2024-04-22T13:05:56.113 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-22T13:05:56.113 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsync-tester}} --name leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi --owner scheduled_leonidus@teuthology --archive /home/teuthworker/archive/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/7668268 --verbose -- /home/teuthworker/archive/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/7668268/orig.config.yaml 2024-04-22T13:05:56.177 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-22T13:05:56.271 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/7668268 branch: wip-lusov-quiescer description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsync-tester}} email: null first_in_suite: false job_id: '7668268' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4362 sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f sleep_before_teardown: 0 subset: 222/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d69f13d827952bb07bb9b7973bca14f97d764d6f/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi012.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCMRG+i3N2wfp8qUYHf6PByN9QrxqQ5vHStVHMeCK3alBQVk04pgrOkt8rCWv7WB0mqXVnbI1YDXI91sjsEVcJs= smithi111.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBlJ/yZTn4or8VxiCJYg37wDBwsYxZsl3eg+scecgl8+x8S9HMVhGjDwEMa491ys8Ytq8fZxWkj1ls+X7qazNg8= smithi134.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI5UvDhcAUouIUIuURUmnHBxRN2ond3DIYi7qGd5N2cCG4gcA0x6qff/x3lF44Y91U5xiTXg+BpD6Fb1XI2XfFg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/fsync-tester.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-22_12:36:42 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-22T13:05:56.271 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d69f13d827952bb07bb9b7973bca14f97d764d6f/qa; will attempt to use it 2024-04-22T13:05:56.272 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d69f13d827952bb07bb9b7973bca14f97d764d6f/qa/tasks 2024-04-22T13:05:56.272 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-22T13:05:56.274 INFO:teuthology.task.internal:Checking packages... 2024-04-22T13:05:56.300 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd69f13d827952bb07bb9b7973bca14f97d764d6f' 2024-04-22T13:05:56.300 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-22T13:05:56.300 INFO:teuthology.packaging:ref: None 2024-04-22T13:05:56.300 INFO:teuthology.packaging:tag: None 2024-04-22T13:05:56.300 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-22T13:05:56.300 INFO:teuthology.packaging:sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-22T13:05:56.300 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-22T13:05:56.650 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3156.gd69f13d8 2024-04-22T13:05:56.652 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-22T13:05:56.659 INFO:teuthology.task.internal:no buildpackages task found 2024-04-22T13:05:56.660 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-22T13:05:56.683 INFO:teuthology.task.internal:Saving configuration 2024-04-22T13:05:56.709 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-22T13:05:56.730 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-22T13:05:56.756 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi012.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/7668268', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 13:00:29.465346', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCMRG+i3N2wfp8qUYHf6PByN9QrxqQ5vHStVHMeCK3alBQVk04pgrOkt8rCWv7WB0mqXVnbI1YDXI91sjsEVcJs='} 2024-04-22T13:05:56.776 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi111.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/7668268', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 13:00:29.461566', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBlJ/yZTn4or8VxiCJYg37wDBwsYxZsl3eg+scecgl8+x8S9HMVhGjDwEMa491ys8Ytq8fZxWkj1ls+X7qazNg8='} 2024-04-22T13:05:56.799 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/7668268', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 13:00:29.463558', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI5UvDhcAUouIUIuURUmnHBxRN2ond3DIYi7qGd5N2cCG4gcA0x6qff/x3lF44Y91U5xiTXg+BpD6Fb1XI2XfFg='} 2024-04-22T13:05:56.799 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-22T13:05:56.806 INFO:teuthology.task.internal:roles: ubuntu@smithi012.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-22T13:05:56.806 INFO:teuthology.task.internal:roles: ubuntu@smithi111.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-22T13:05:56.806 INFO:teuthology.task.internal:roles: ubuntu@smithi134.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-22T13:05:56.806 INFO:teuthology.run_tasks:Running task console_log... 2024-04-22T13:05:56.912 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fdbb4d140d0>, signals=[15]) 2024-04-22T13:05:56.913 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-22T13:05:56.920 INFO:teuthology.task.internal:Opening connections... 2024-04-22T13:05:56.920 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi012.front.sepia.ceph.com 2024-04-22T13:05:56.922 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T13:05:56.999 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi111.front.sepia.ceph.com 2024-04-22T13:05:57.000 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T13:05:57.074 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi134.front.sepia.ceph.com 2024-04-22T13:05:57.074 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T13:05:57.149 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-22T13:05:57.157 DEBUG:teuthology.orchestra.run.smithi012:> uname -m 2024-04-22T13:05:57.176 INFO:teuthology.orchestra.run.smithi012.stdout:x86_64 2024-04-22T13:05:57.176 DEBUG:teuthology.orchestra.run.smithi012:> cat /etc/os-release 2024-04-22T13:05:57.234 INFO:teuthology.orchestra.run.smithi012.stdout:NAME="CentOS Stream" 2024-04-22T13:05:57.234 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION="9" 2024-04-22T13:05:57.234 INFO:teuthology.orchestra.run.smithi012.stdout:ID="centos" 2024-04-22T13:05:57.234 INFO:teuthology.orchestra.run.smithi012.stdout:ID_LIKE="rhel fedora" 2024-04-22T13:05:57.234 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_ID="9" 2024-04-22T13:05:57.234 INFO:teuthology.orchestra.run.smithi012.stdout:PLATFORM_ID="platform:el9" 2024-04-22T13:05:57.234 INFO:teuthology.orchestra.run.smithi012.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-22T13:05:57.235 INFO:teuthology.orchestra.run.smithi012.stdout:ANSI_COLOR="0;31" 2024-04-22T13:05:57.235 INFO:teuthology.orchestra.run.smithi012.stdout:LOGO="fedora-logo-icon" 2024-04-22T13:05:57.235 INFO:teuthology.orchestra.run.smithi012.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-22T13:05:57.235 INFO:teuthology.orchestra.run.smithi012.stdout:HOME_URL="https://centos.org/" 2024-04-22T13:05:57.235 INFO:teuthology.orchestra.run.smithi012.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-22T13:05:57.235 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-22T13:05:57.235 INFO:teuthology.orchestra.run.smithi012.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-22T13:05:57.236 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server 2024-04-22T13:05:57.261 DEBUG:teuthology.orchestra.run.smithi111:> uname -m 2024-04-22T13:05:57.278 INFO:teuthology.orchestra.run.smithi111.stdout:x86_64 2024-04-22T13:05:57.278 DEBUG:teuthology.orchestra.run.smithi111:> cat /etc/os-release 2024-04-22T13:05:57.336 INFO:teuthology.orchestra.run.smithi111.stdout:NAME="CentOS Stream" 2024-04-22T13:05:57.336 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION="9" 2024-04-22T13:05:57.336 INFO:teuthology.orchestra.run.smithi111.stdout:ID="centos" 2024-04-22T13:05:57.336 INFO:teuthology.orchestra.run.smithi111.stdout:ID_LIKE="rhel fedora" 2024-04-22T13:05:57.336 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION_ID="9" 2024-04-22T13:05:57.336 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM_ID="platform:el9" 2024-04-22T13:05:57.336 INFO:teuthology.orchestra.run.smithi111.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-22T13:05:57.336 INFO:teuthology.orchestra.run.smithi111.stdout:ANSI_COLOR="0;31" 2024-04-22T13:05:57.336 INFO:teuthology.orchestra.run.smithi111.stdout:LOGO="fedora-logo-icon" 2024-04-22T13:05:57.336 INFO:teuthology.orchestra.run.smithi111.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-22T13:05:57.336 INFO:teuthology.orchestra.run.smithi111.stdout:HOME_URL="https://centos.org/" 2024-04-22T13:05:57.336 INFO:teuthology.orchestra.run.smithi111.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-22T13:05:57.336 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-22T13:05:57.337 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-22T13:05:57.337 INFO:teuthology.lock.ops:Updating smithi111.front.sepia.ceph.com on lock server 2024-04-22T13:05:57.360 DEBUG:teuthology.orchestra.run.smithi134:> uname -m 2024-04-22T13:05:57.378 INFO:teuthology.orchestra.run.smithi134.stdout:x86_64 2024-04-22T13:05:57.378 DEBUG:teuthology.orchestra.run.smithi134:> cat /etc/os-release 2024-04-22T13:05:57.436 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="CentOS Stream" 2024-04-22T13:05:57.436 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="9" 2024-04-22T13:05:57.436 INFO:teuthology.orchestra.run.smithi134.stdout:ID="centos" 2024-04-22T13:05:57.436 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE="rhel fedora" 2024-04-22T13:05:57.436 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="9" 2024-04-22T13:05:57.436 INFO:teuthology.orchestra.run.smithi134.stdout:PLATFORM_ID="platform:el9" 2024-04-22T13:05:57.436 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-22T13:05:57.436 INFO:teuthology.orchestra.run.smithi134.stdout:ANSI_COLOR="0;31" 2024-04-22T13:05:57.436 INFO:teuthology.orchestra.run.smithi134.stdout:LOGO="fedora-logo-icon" 2024-04-22T13:05:57.436 INFO:teuthology.orchestra.run.smithi134.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-22T13:05:57.436 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://centos.org/" 2024-04-22T13:05:57.436 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-22T13:05:57.437 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-22T13:05:57.437 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-22T13:05:57.437 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server 2024-04-22T13:05:57.460 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-22T13:05:57.471 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-22T13:05:57.482 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-22T13:05:57.482 DEBUG:teuthology.orchestra.run.smithi012:> test '!' -e /home/ubuntu/cephtest 2024-04-22T13:05:57.485 DEBUG:teuthology.orchestra.run.smithi111:> test '!' -e /home/ubuntu/cephtest 2024-04-22T13:05:57.487 DEBUG:teuthology.orchestra.run.smithi134:> test '!' -e /home/ubuntu/cephtest 2024-04-22T13:05:57.502 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-22T13:05:57.510 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-22T13:05:57.510 DEBUG:teuthology.orchestra.run.smithi012:> test -z $(ls -A /var/lib/ceph) 2024-04-22T13:05:57.542 DEBUG:teuthology.orchestra.run.smithi111:> test -z $(ls -A /var/lib/ceph) 2024-04-22T13:05:57.545 DEBUG:teuthology.orchestra.run.smithi134:> test -z $(ls -A /var/lib/ceph) 2024-04-22T13:05:57.567 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-22T13:05:57.649 INFO:teuthology.run_tasks:Running task kernel... 2024-04-22T13:05:57.663 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-22T13:05:57.664 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-22T13:05:57.664 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-22T13:05:57.664 DEBUG:teuthology.orchestra.run.smithi012:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-22T13:05:57.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T13:05:57.681 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2024-04-22T13:05:57.739 INFO:teuthology.orchestra.run.smithi012.stdout:5.14.0-437.el9.x86_64 2024-04-22T13:05:57.739 INFO:teuthology.task.kernel:Running kernel on smithi012: 5.14.0-437.el9.x86_64 2024-04-22T13:05:57.739 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum install -y kernel 2024-04-22T13:06:08.985 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - BaseOS 866 kB/s | 8.0 MB 00:09 2024-04-22T13:06:11.027 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-22T13:06:21.960 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - CRB 872 kB/s | 6.2 MB 00:07 2024-04-22T13:06:23.752 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 16 kB 00:00 2024-04-22T13:06:26.678 INFO:teuthology.orchestra.run.smithi012.stdout:Extra Packages for Enterprise Linux 7.5 MB/s | 21 MB 00:02 2024-04-22T13:06:31.836 INFO:teuthology.orchestra.run.smithi012.stdout:lab-extras 48 kB/s | 1.7 kB 00:00 2024-04-22T13:06:33.758 INFO:teuthology.orchestra.run.smithi012.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-22T13:06:33.758 INFO:teuthology.orchestra.run.smithi012.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-22T13:06:33.848 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-04-22T13:06:33.850 INFO:teuthology.orchestra.run.smithi012.stdout:Nothing to do. 2024-04-22T13:06:33.850 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-04-22T13:06:33.971 DEBUG:teuthology.orchestra.run.smithi012:> echo no | sudo yum reinstall kernel || true 2024-04-22T13:06:34.492 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:00:03 ago on Mon 22 Apr 2024 01:06:31 PM UTC. 2024-04-22T13:06:34.615 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-04-22T13:06:34.633 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-04-22T13:06:34.633 INFO:teuthology.orchestra.run.smithi012.stdout: Package Architecture Version Repository Size 2024-04-22T13:06:34.633 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-04-22T13:06:34.633 INFO:teuthology.orchestra.run.smithi012.stdout:Reinstalling: 2024-04-22T13:06:34.633 INFO:teuthology.orchestra.run.smithi012.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-22T13:06:34.634 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:06:34.634 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-04-22T13:06:34.634 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-04-22T13:06:34.634 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:06:34.634 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 255 k 2024-04-22T13:06:34.634 INFO:teuthology.orchestra.run.smithi012.stdout:Installed size: 0 2024-04-22T13:06:34.634 INFO:teuthology.orchestra.run.smithi012.stderr:Operation aborted. 2024-04-22T13:06:34.696 INFO:teuthology.orchestra.run.smithi012.stdout:Is this ok [y/N]: 2024-04-22T13:06:34.696 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum reinstall -y kernel || true 2024-04-22T13:06:35.213 INFO:teuthology.orchestra.run.smithi012.stdout:Last metadata expiration check: 0:00:04 ago on Mon 22 Apr 2024 01:06:31 PM UTC. 2024-04-22T13:06:35.334 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-04-22T13:06:35.335 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-04-22T13:06:35.335 INFO:teuthology.orchestra.run.smithi012.stdout: Package Architecture Version Repository Size 2024-04-22T13:06:35.335 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-04-22T13:06:35.335 INFO:teuthology.orchestra.run.smithi012.stdout:Reinstalling: 2024-04-22T13:06:35.335 INFO:teuthology.orchestra.run.smithi012.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-22T13:06:35.335 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:06:35.335 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-04-22T13:06:35.335 INFO:teuthology.orchestra.run.smithi012.stdout:================================================================================ 2024-04-22T13:06:35.335 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:06:35.336 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 255 k 2024-04-22T13:06:35.336 INFO:teuthology.orchestra.run.smithi012.stdout:Installed size: 0 2024-04-22T13:06:35.336 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2024-04-22T13:06:36.154 INFO:teuthology.orchestra.run.smithi012.stdout:kernel-5.14.0-437.el9.x86_64.rpm 340 kB/s | 255 kB 00:00 2024-04-22T13:06:36.154 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-04-22T13:06:36.154 INFO:teuthology.orchestra.run.smithi012.stdout:Total 311 kB/s | 255 kB 00:00 2024-04-22T13:06:36.175 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2024-04-22T13:06:36.392 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2024-04-22T13:06:36.392 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2024-04-22T13:06:36.397 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2024-04-22T13:06:36.398 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2024-04-22T13:06:36.535 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2024-04-22T13:06:36.605 INFO:teuthology.orchestra.run.smithi012.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-22T13:06:36.605 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T13:06:37.244 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T13:06:37.244 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-22T13:06:37.966 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T13:06:37.966 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:06:37.966 INFO:teuthology.orchestra.run.smithi012.stdout:Reinstalled: 2024-04-22T13:06:37.966 INFO:teuthology.orchestra.run.smithi012.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-22T13:06:37.966 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:06:37.966 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-04-22T13:06:38.143 DEBUG:teuthology.orchestra.run.smithi012:> rpm -q kernel | sort -rV | head -n 1 2024-04-22T13:06:38.190 INFO:teuthology.orchestra.run.smithi012.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-22T13:06:38.191 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-22T13:06:38.191 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-22T13:06:38.192 DEBUG:teuthology.parallel:result is None 2024-04-22T13:06:38.192 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-22T13:06:38.199 INFO:teuthology.task.internal:Creating test directory... 2024-04-22T13:06:38.199 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-22T13:06:38.201 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-22T13:06:38.204 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-22T13:06:38.222 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-22T13:06:38.228 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-22T13:06:38.332 INFO:teuthology.task.internal:Creating archive directory... 2024-04-22T13:06:38.333 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-22T13:06:38.335 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-22T13:06:38.338 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-22T13:06:38.370 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-22T13:06:38.377 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-22T13:06:38.377 DEBUG:teuthology.orchestra.run.smithi111:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-22T13:06:38.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T13:06:38.417 DEBUG:teuthology.orchestra.run.smithi134:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-22T13:06:38.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T13:06:38.433 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-22T13:06:38.436 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-22T13:06:38.459 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-22T13:06:38.474 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T13:06:38.489 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T13:06:38.495 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T13:06:38.513 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T13:06:38.518 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T13:06:38.535 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T13:06:38.537 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-22T13:06:38.545 INFO:teuthology.task.internal:Configuring sudo... 2024-04-22T13:06:38.545 DEBUG:teuthology.orchestra.run.smithi012:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-22T13:06:38.547 DEBUG:teuthology.orchestra.run.smithi111:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-22T13:06:38.557 DEBUG:teuthology.orchestra.run.smithi134:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-22T13:06:38.583 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-22T13:06:38.592 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-22T13:06:38.592 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-22T13:06:38.611 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-22T13:06:38.626 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-22T13:06:38.641 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T13:06:38.709 DEBUG:teuthology.orchestra.run.smithi012:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T13:06:38.752 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T13:06:38.834 DEBUG:teuthology.orchestra.run.smithi012:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T13:06:38.872 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-22T13:06:38.872 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-22T13:06:38.937 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T13:06:38.982 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T13:06:39.053 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T13:06:39.098 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T13:06:39.137 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-22T13:06:39.137 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-22T13:06:39.202 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T13:06:39.246 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T13:06:39.295 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T13:06:39.379 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T13:06:39.420 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-22T13:06:39.420 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-22T13:06:39.485 DEBUG:teuthology.orchestra.run.smithi012:> sudo service rsyslog restart 2024-04-22T13:06:39.487 DEBUG:teuthology.orchestra.run.smithi111:> sudo service rsyslog restart 2024-04-22T13:06:39.489 DEBUG:teuthology.orchestra.run.smithi134:> sudo service rsyslog restart 2024-04-22T13:06:39.520 INFO:teuthology.orchestra.run.smithi012.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T13:06:39.521 INFO:teuthology.orchestra.run.smithi111.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T13:06:39.566 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T13:06:40.018 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-22T13:06:40.028 INFO:teuthology.task.internal:Starting timer... 2024-04-22T13:06:40.028 INFO:teuthology.run_tasks:Running task pcp... 2024-04-22T13:06:40.038 INFO:teuthology.run_tasks:Running task selinux... 2024-04-22T13:06:40.046 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-22T13:06:40.046 DEBUG:teuthology.orchestra.run.smithi012:> sudo service auditd rotate 2024-04-22T13:06:40.102 INFO:teuthology.orchestra.run.smithi012.stdout:Rotating logs: 2024-04-22T13:06:40.104 DEBUG:teuthology.orchestra.run.smithi111:> sudo service auditd rotate 2024-04-22T13:06:40.149 INFO:teuthology.orchestra.run.smithi111.stdout:Rotating logs: 2024-04-22T13:06:40.151 DEBUG:teuthology.orchestra.run.smithi134:> sudo service auditd rotate 2024-04-22T13:06:40.210 INFO:teuthology.orchestra.run.smithi134.stdout:Rotating logs: 2024-04-22T13:06:40.212 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-22T13:06:40.212 DEBUG:teuthology.orchestra.run.smithi012:> /usr/sbin/getenforce 2024-04-22T13:06:40.236 INFO:teuthology.orchestra.run.smithi012.stdout:Permissive 2024-04-22T13:06:40.237 DEBUG:teuthology.orchestra.run.smithi111:> /usr/sbin/getenforce 2024-04-22T13:06:40.268 INFO:teuthology.orchestra.run.smithi111.stdout:Permissive 2024-04-22T13:06:40.268 DEBUG:teuthology.orchestra.run.smithi134:> /usr/sbin/getenforce 2024-04-22T13:06:40.294 INFO:teuthology.orchestra.run.smithi134.stdout:Permissive 2024-04-22T13:06:40.294 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi012.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi111.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi134.front.sepia.ceph.com': 'permissive'} 2024-04-22T13:06:40.295 DEBUG:teuthology.orchestra.run.smithi012:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T13:06:40.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T13:06:40.319 DEBUG:teuthology.orchestra.run.smithi111:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T13:06:40.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T13:06:40.346 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T13:06:40.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T13:06:40.372 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-22T13:06:40.372 DEBUG:teuthology.orchestra.run.smithi012:> sudo /usr/sbin/setenforce permissive 2024-04-22T13:06:40.397 DEBUG:teuthology.orchestra.run.smithi111:> sudo /usr/sbin/setenforce permissive 2024-04-22T13:06:40.423 DEBUG:teuthology.orchestra.run.smithi134:> sudo /usr/sbin/setenforce permissive 2024-04-22T13:06:40.455 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-22T13:06:40.465 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-22T13:06:40.471 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-22T13:06:40.604 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-22T13:06:40.619 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-22T13:06:40.621 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi012.front.sepia.ceph.com,smithi111.front.sepia.ceph.com,smithi134.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-22T13:10:39.549 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi012.front.sepia.ceph.com'), Remote(name='ubuntu@smithi111.front.sepia.ceph.com'), Remote(name='ubuntu@smithi134.front.sepia.ceph.com')] 2024-04-22T13:10:39.550 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-04-22T13:10:39.552 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T13:10:39.629 DEBUG:teuthology.orchestra.run.smithi012:> true 2024-04-22T13:10:39.705 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-04-22T13:10:39.706 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-04-22T13:10:39.706 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T13:10:39.780 DEBUG:teuthology.orchestra.run.smithi111:> true 2024-04-22T13:10:39.813 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-04-22T13:10:39.814 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-04-22T13:10:39.814 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T13:10:39.887 DEBUG:teuthology.orchestra.run.smithi134:> true 2024-04-22T13:10:39.965 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-04-22T13:10:39.965 INFO:teuthology.run_tasks:Running task clock... 2024-04-22T13:10:39.978 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-22T13:10:39.978 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-22T13:10:39.979 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T13:10:39.981 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-22T13:10:39.981 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T13:10:39.983 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-22T13:10:39.983 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T13:10:40.008 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-22T13:10:40.009 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-22T13:10:40.021 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-22T13:10:40.023 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-22T13:10:40.033 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-22T13:10:40.042 INFO:teuthology.orchestra.run.smithi111.stderr:sudo: ntpd: command not found 2024-04-22T13:10:40.046 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-22T13:10:40.047 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: ntpd: command not found 2024-04-22T13:10:40.053 INFO:teuthology.orchestra.run.smithi111.stdout:506 Cannot talk to daemon 2024-04-22T13:10:40.058 INFO:teuthology.orchestra.run.smithi012.stdout:506 Cannot talk to daemon 2024-04-22T13:10:40.065 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-22T13:10:40.068 INFO:teuthology.orchestra.run.smithi134.stderr:sudo: ntpd: command not found 2024-04-22T13:10:40.071 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-22T13:10:40.078 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-22T13:10:40.079 INFO:teuthology.orchestra.run.smithi134.stdout:506 Cannot talk to daemon 2024-04-22T13:10:40.085 INFO:teuthology.orchestra.run.smithi012.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-22T13:10:40.091 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-22T13:10:40.104 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-22T13:10:40.135 INFO:teuthology.orchestra.run.smithi111.stderr:bash: line 1: ntpq: command not found 2024-04-22T13:10:40.138 INFO:teuthology.orchestra.run.smithi111.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T13:10:40.138 INFO:teuthology.orchestra.run.smithi111.stdout:=============================================================================== 2024-04-22T13:10:40.139 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T13:10:40.139 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T13:10:40.139 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T13:10:40.139 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T13:10:40.142 INFO:teuthology.orchestra.run.smithi012.stderr:bash: line 1: ntpq: command not found 2024-04-22T13:10:40.145 INFO:teuthology.orchestra.run.smithi012.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T13:10:40.145 INFO:teuthology.orchestra.run.smithi012.stdout:=============================================================================== 2024-04-22T13:10:40.146 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T13:10:40.146 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T13:10:40.146 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T13:10:40.146 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T13:10:40.158 INFO:teuthology.orchestra.run.smithi134.stderr:bash: line 1: ntpq: command not found 2024-04-22T13:10:40.162 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T13:10:40.162 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2024-04-22T13:10:40.162 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T13:10:40.162 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T13:10:40.162 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T13:10:40.162 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T13:10:40.163 INFO:teuthology.run_tasks:Running task install... 2024-04-22T13:10:40.171 DEBUG:teuthology.task.install:project ceph 2024-04-22T13:10:40.171 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'}} 2024-04-22T13:10:40.171 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'} 2024-04-22T13:10:40.171 INFO:teuthology.task.install:Using flavor: default 2024-04-22T13:10:40.177 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-22T13:10:40.177 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-22T13:10:40.177 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-22T13:10:40.178 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-22T13:10:40.180 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-22T13:10:40.180 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-22T13:10:40.181 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-22T13:10:40.181 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-22T13:10:40.332 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-22T13:10:40.332 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-22T13:10:40.339 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-22T13:10:40.339 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-22T13:10:40.342 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-22T13:10:40.342 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-22T13:10:40.636 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-22T13:10:40.636 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-22T13:10:40.636 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-22T13:10:40.645 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-22T13:10:40.645 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-22T13:10:40.645 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-22T13:10:40.650 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-22T13:10:40.650 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-22T13:10:40.650 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-22T13:10:40.661 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-22T13:10:40.661 DEBUG:teuthology.orchestra.run.smithi134:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-22T13:10:40.674 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-22T13:10:40.674 DEBUG:teuthology.orchestra.run.smithi012:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-22T13:10:40.676 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-22T13:10:40.676 DEBUG:teuthology.orchestra.run.smithi111:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-22T13:10:40.726 DEBUG:teuthology.orchestra.run.smithi134:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-22T13:10:40.741 DEBUG:teuthology.orchestra.run.smithi111:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-22T13:10:40.743 DEBUG:teuthology.orchestra.run.smithi012:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-22T13:10:40.798 DEBUG:teuthology.orchestra.run.smithi134:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-22T13:10:40.815 DEBUG:teuthology.orchestra.run.smithi111:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-22T13:10:40.821 DEBUG:teuthology.orchestra.run.smithi012:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-22T13:10:40.861 INFO:teuthology.orchestra.run.smithi134.stdout:check_obsoletes = 1 2024-04-22T13:10:40.863 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum clean all 2024-04-22T13:10:40.877 INFO:teuthology.orchestra.run.smithi111.stdout:check_obsoletes = 1 2024-04-22T13:10:40.879 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum clean all 2024-04-22T13:10:40.886 INFO:teuthology.orchestra.run.smithi012.stdout:check_obsoletes = 1 2024-04-22T13:10:40.888 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum clean all 2024-04-22T13:10:41.085 INFO:teuthology.orchestra.run.smithi134.stdout:48 files removed 2024-04-22T13:10:41.103 INFO:teuthology.orchestra.run.smithi111.stdout:48 files removed 2024-04-22T13:10:41.113 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-22T13:10:41.124 INFO:teuthology.orchestra.run.smithi012.stdout:48 files removed 2024-04-22T13:10:41.131 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-22T13:10:41.151 DEBUG:teuthology.orchestra.run.smithi012:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-22T13:10:42.301 INFO:teuthology.orchestra.run.smithi111.stdout:ceph packages for x86_64 92 kB/s | 83 kB 00:00 2024-04-22T13:10:42.572 INFO:teuthology.orchestra.run.smithi134.stdout:ceph packages for x86_64 70 kB/s | 83 kB 00:01 2024-04-22T13:10:42.573 INFO:teuthology.orchestra.run.smithi012.stdout:ceph packages for x86_64 72 kB/s | 83 kB 00:01 2024-04-22T13:10:43.309 INFO:teuthology.orchestra.run.smithi111.stdout:ceph noarch packages 12 kB/s | 12 kB 00:00 2024-04-22T13:10:43.497 INFO:teuthology.orchestra.run.smithi134.stdout:ceph noarch packages 13 kB/s | 12 kB 00:00 2024-04-22T13:10:43.632 INFO:teuthology.orchestra.run.smithi012.stdout:ceph noarch packages 11 kB/s | 12 kB 00:01 2024-04-22T13:10:44.060 INFO:teuthology.orchestra.run.smithi111.stdout:ceph source packages 2.7 kB/s | 1.9 kB 00:00 2024-04-22T13:10:44.273 INFO:teuthology.orchestra.run.smithi134.stdout:ceph source packages 2.6 kB/s | 1.9 kB 00:00 2024-04-22T13:10:44.424 INFO:teuthology.orchestra.run.smithi012.stdout:ceph source packages 2.5 kB/s | 1.9 kB 00:00 2024-04-22T13:10:45.040 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-22T13:10:46.773 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - BaseOS 3.3 MB/s | 8.0 MB 00:02 2024-04-22T13:10:47.025 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-22T13:10:53.557 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - AppStream 3.3 MB/s | 19 MB 00:05 2024-04-22T13:10:55.026 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - BaseOS 754 kB/s | 8.0 MB 00:10 2024-04-22T13:10:57.998 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - CRB 6.8 MB/s | 6.2 MB 00:00 2024-04-22T13:10:58.590 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - CRB 800 kB/s | 6.2 MB 00:07 2024-04-22T13:11:00.308 INFO:teuthology.orchestra.run.smithi012.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-04-22T13:11:01.376 INFO:teuthology.orchestra.run.smithi012.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-04-22T13:11:06.701 INFO:teuthology.orchestra.run.smithi012.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-22T13:11:07.773 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - Extras packages 1.9 kB/s | 16 kB 00:08 2024-04-22T13:11:08.911 INFO:teuthology.orchestra.run.smithi012.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T13:11:08.911 INFO:teuthology.orchestra.run.smithi012.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T13:11:08.923 INFO:teuthology.orchestra.run.smithi012.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-22T13:11:08.924 INFO:teuthology.orchestra.run.smithi012.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-22T13:11:08.925 INFO:teuthology.orchestra.run.smithi012.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-22T13:11:08.925 INFO:teuthology.orchestra.run.smithi012.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-22T13:11:08.927 INFO:teuthology.orchestra.run.smithi012.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-22T13:11:08.929 INFO:teuthology.orchestra.run.smithi012.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-22T13:11:08.930 INFO:teuthology.orchestra.run.smithi012.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-22T13:11:08.996 INFO:teuthology.orchestra.run.smithi012.stdout:Dependencies resolved. 2024-04-22T13:11:09.006 INFO:teuthology.orchestra.run.smithi012.stdout:======================================================================================= 2024-04-22T13:11:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: Package Arch Version Repository Size 2024-04-22T13:11:09.006 INFO:teuthology.orchestra.run.smithi012.stdout:======================================================================================= 2024-04-22T13:11:09.006 INFO:teuthology.orchestra.run.smithi012.stdout:Installing: 2024-04-22T13:11:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-22T13:11:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-22T13:11:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-22T13:11:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-22T13:11:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-22T13:11:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-22T13:11:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-22T13:11:09.006 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-22T13:11:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-22T13:11:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-22T13:11:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-22T13:11:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-22T13:11:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-22T13:11:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-22T13:11:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-22T13:11:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-22T13:11:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-22T13:11:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-22T13:11:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-22T13:11:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-22T13:11:09.007 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-22T13:11:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-22T13:11:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-22T13:11:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-22T13:11:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-22T13:11:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-22T13:11:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-22T13:11:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-22T13:11:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-22T13:11:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-22T13:11:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-22T13:11:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-22T13:11:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-22T13:11:09.008 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-22T13:11:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-22T13:11:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-22T13:11:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-22T13:11:09.009 INFO:teuthology.orchestra.run.smithi012.stdout:Upgrading: 2024-04-22T13:11:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-22T13:11:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-22T13:11:09.009 INFO:teuthology.orchestra.run.smithi012.stdout:Installing dependencies: 2024-04-22T13:11:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-22T13:11:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-22T13:11:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-22T13:11:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-22T13:11:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-22T13:11:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-22T13:11:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-22T13:11:09.009 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-22T13:11:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-22T13:11:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-22T13:11:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-22T13:11:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-22T13:11:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-22T13:11:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-22T13:11:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-22T13:11:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-22T13:11:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-22T13:11:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-22T13:11:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-22T13:11:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-22T13:11:09.010 INFO:teuthology.orchestra.run.smithi012.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-22T13:11:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-22T13:11:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-22T13:11:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-22T13:11:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-22T13:11:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-22T13:11:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-22T13:11:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-22T13:11:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-22T13:11:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-22T13:11:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-22T13:11:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-22T13:11:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-22T13:11:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-22T13:11:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-22T13:11:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-22T13:11:09.011 INFO:teuthology.orchestra.run.smithi012.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-22T13:11:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-22T13:11:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-22T13:11:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-22T13:11:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-22T13:11:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-22T13:11:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-22T13:11:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-22T13:11:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-22T13:11:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-22T13:11:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-22T13:11:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-22T13:11:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-22T13:11:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-22T13:11:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-22T13:11:09.012 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-22T13:11:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-22T13:11:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-22T13:11:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-22T13:11:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-22T13:11:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-22T13:11:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-22T13:11:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-22T13:11:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-22T13:11:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-22T13:11:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-22T13:11:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-22T13:11:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-22T13:11:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-22T13:11:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-22T13:11:09.013 INFO:teuthology.orchestra.run.smithi012.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-22T13:11:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-22T13:11:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-22T13:11:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-22T13:11:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-22T13:11:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-22T13:11:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-22T13:11:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-22T13:11:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-22T13:11:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-22T13:11:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-22T13:11:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-22T13:11:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-22T13:11:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-22T13:11:09.014 INFO:teuthology.orchestra.run.smithi012.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-22T13:11:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-22T13:11:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-22T13:11:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-22T13:11:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-22T13:11:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-22T13:11:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-22T13:11:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-22T13:11:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-22T13:11:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-22T13:11:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-22T13:11:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-22T13:11:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-22T13:11:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-22T13:11:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-22T13:11:09.015 INFO:teuthology.orchestra.run.smithi012.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-22T13:11:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-22T13:11:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-22T13:11:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-22T13:11:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-22T13:11:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-22T13:11:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-22T13:11:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-22T13:11:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-22T13:11:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-22T13:11:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-22T13:11:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-22T13:11:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-22T13:11:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-22T13:11:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-22T13:11:09.016 INFO:teuthology.orchestra.run.smithi012.stdout:Installing weak dependencies: 2024-04-22T13:11:09.016 INFO:teuthology.orchestra.run.smithi012.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-22T13:11:09.017 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:11:09.017 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction Summary 2024-04-22T13:11:09.017 INFO:teuthology.orchestra.run.smithi012.stdout:======================================================================================= 2024-04-22T13:11:09.017 INFO:teuthology.orchestra.run.smithi012.stdout:Install 148 Packages 2024-04-22T13:11:09.017 INFO:teuthology.orchestra.run.smithi012.stdout:Upgrade 2 Packages 2024-04-22T13:11:09.017 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:11:09.017 INFO:teuthology.orchestra.run.smithi012.stdout:Total download size: 218 M 2024-04-22T13:11:09.017 INFO:teuthology.orchestra.run.smithi012.stdout:Downloading Packages: 2024-04-22T13:11:09.433 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - AppStream 1.4 MB/s | 19 MB 00:13 2024-04-22T13:11:09.516 INFO:teuthology.orchestra.run.smithi134.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-22T13:11:10.244 INFO:teuthology.orchestra.run.smithi012.stdout:(1/150): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 17 kB/s | 6.6 kB 00:00 2024-04-22T13:11:11.060 INFO:teuthology.orchestra.run.smithi012.stdout:(2/150): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 4.4 MB/s | 5.4 MB 00:01 2024-04-22T13:11:11.144 INFO:teuthology.orchestra.run.smithi012.stdout:(3/150): ceph-immutable-object-cache-19.0.0-315 1.7 MB/s | 145 kB 00:00 2024-04-22T13:11:11.236 INFO:teuthology.orchestra.run.smithi012.stdout:(4/150): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 897 kB/s | 890 kB 00:00 2024-04-22T13:11:11.370 INFO:teuthology.orchestra.run.smithi012.stdout:(5/150): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 10 MB/s | 2.4 MB 00:00 2024-04-22T13:11:11.795 INFO:teuthology.orchestra.run.smithi012.stdout:(6/150): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 11 MB/s | 4.7 MB 00:00 2024-04-22T13:11:12.303 INFO:teuthology.orchestra.run.smithi012.stdout:(7/150): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 1.4 MB/s | 1.5 MB 00:01 2024-04-22T13:11:12.712 INFO:teuthology.orchestra.run.smithi012.stdout:(8/150): ceph-common-19.0.0-3156.gd69f13d8.el7. 7.4 MB/s | 21 MB 00:02 2024-04-22T13:11:12.787 INFO:teuthology.orchestra.run.smithi012.stdout:(9/150): ceph-selinux-19.0.0-3156.gd69f13d8.el7 336 kB/s | 25 kB 00:00 2024-04-22T13:11:13.071 INFO:teuthology.orchestra.run.smithi012.stdout:(10/150): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 13 MB/s | 17 MB 00:01 2024-04-22T13:11:13.162 INFO:teuthology.orchestra.run.smithi012.stdout:(11/150): cephfs-mirror-19.0.0-3156.gd69f13d8.e 2.4 MB/s | 225 kB 00:00 2024-04-22T13:11:13.238 INFO:teuthology.orchestra.run.smithi012.stdout:(12/150): cephfs-shell-19.0.0-3156.gd69f13d8.el 361 kB/s | 27 kB 00:00 2024-04-22T13:11:13.313 INFO:teuthology.orchestra.run.smithi012.stdout:(13/150): libcephfs-devel-19.0.0-3156.gd69f13d8 423 kB/s | 32 kB 00:00 2024-04-22T13:11:13.438 INFO:teuthology.orchestra.run.smithi012.stdout:(14/150): libcephfs2-19.0.0-3156.gd69f13d8.el7. 5.8 MB/s | 747 kB 00:00 2024-04-22T13:11:13.539 INFO:teuthology.orchestra.run.smithi012.stdout:(15/150): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.6 MB/s | 163 kB 00:00 2024-04-22T13:11:13.648 INFO:teuthology.orchestra.run.smithi012.stdout:(16/150): librados-devel-19.0.0-3156.gd69f13d8. 1.1 MB/s | 126 kB 00:00 2024-04-22T13:11:13.748 INFO:teuthology.orchestra.run.smithi012.stdout:(17/150): libradosstriper1-19.0.0-3156.gd69f13d 4.9 MB/s | 500 kB 00:00 2024-04-22T13:11:14.286 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 6.2 MB 00:01 2024-04-22T13:11:14.323 INFO:teuthology.orchestra.run.smithi012.stdout:(18/150): librgw2-19.0.0-3156.gd69f13d8.el7.x86 9.3 MB/s | 5.4 MB 00:00 2024-04-22T13:11:14.440 INFO:teuthology.orchestra.run.smithi012.stdout:(19/150): ceph-radosgw-19.0.0-3156.gd69f13d8.el 5.0 MB/s | 11 MB 00:02 2024-04-22T13:11:14.466 INFO:teuthology.orchestra.run.smithi012.stdout:(20/150): python3-ceph-argparse-19.0.0-3156.gd6 317 kB/s | 45 kB 00:00 2024-04-22T13:11:14.532 INFO:teuthology.orchestra.run.smithi012.stdout:(21/150): python3-ceph-common-19.0.0-3156.gd69f 1.4 MB/s | 135 kB 00:00 2024-04-22T13:11:14.549 INFO:teuthology.orchestra.run.smithi134.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-22T13:11:14.558 INFO:teuthology.orchestra.run.smithi012.stdout:(22/150): python3-cephfs-19.0.0-3156.gd69f13d8. 1.7 MB/s | 161 kB 00:00 2024-04-22T13:11:14.616 INFO:teuthology.orchestra.run.smithi012.stdout:(23/150): python3-rados-19.0.0-3156.gd69f13d8.e 3.7 MB/s | 320 kB 00:00 2024-04-22T13:11:14.650 INFO:teuthology.orchestra.run.smithi012.stdout:(24/150): python3-rbd-19.0.0-3156.gd69f13d8.el7 3.2 MB/s | 298 kB 00:00 2024-04-22T13:11:14.700 INFO:teuthology.orchestra.run.smithi012.stdout:(25/150): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.2 MB/s | 99 kB 00:00 2024-04-22T13:11:14.725 INFO:teuthology.orchestra.run.smithi012.stdout:(26/150): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.1 MB/s | 85 kB 00:00 2024-04-22T13:11:14.817 INFO:teuthology.orchestra.run.smithi012.stdout:(27/150): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.8 MB/s | 170 kB 00:00 2024-04-22T13:11:14.901 INFO:teuthology.orchestra.run.smithi012.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3156.g 410 kB/s | 34 kB 00:00 2024-04-22T13:11:14.985 INFO:teuthology.orchestra.run.smithi012.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.7 MB/s | 142 kB 00:00 2024-04-22T13:11:15.043 INFO:teuthology.orchestra.run.smithi012.stdout:(30/150): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 9.0 MB/s | 3.1 MB 00:00 2024-04-22T13:11:15.402 INFO:teuthology.orchestra.run.smithi012.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3156.gd69f1 8.5 MB/s | 3.5 MB 00:00 2024-04-22T13:11:15.510 INFO:teuthology.orchestra.run.smithi012.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3156.gd6 2.2 MB/s | 249 kB 00:00 2024-04-22T13:11:15.611 INFO:teuthology.orchestra.run.smithi012.stdout:(33/150): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 492 kB/s | 49 kB 00:00 2024-04-22T13:11:15.761 INFO:teuthology.orchestra.run.smithi012.stdout:(34/150): ceph-mgr-diskprediction-local-19.0.0- 10 MB/s | 7.4 MB 00:00 2024-04-22T13:11:15.786 INFO:teuthology.orchestra.run.smithi012.stdout:(35/150): ceph-prometheus-alerts-19.0.0-3156.gd 94 kB/s | 16 kB 00:00 2024-04-22T13:11:15.853 INFO:teuthology.orchestra.run.smithi012.stdout:(36/150): ceph-volume-19.0.0-3156.gd69f13d8.el7 2.9 MB/s | 273 kB 00:00 2024-04-22T13:11:15.887 INFO:teuthology.orchestra.run.smithi012.stdout:(37/150): cephadm-19.0.0-3156.gd69f13d8.el7.noa 7.4 MB/s | 762 kB 00:00 2024-04-22T13:11:15.929 INFO:teuthology.orchestra.run.smithi012.stdout:(38/150): cephfs-top-19.0.0-3156.gd69f13d8.el7. 242 kB/s | 18 kB 00:00 2024-04-22T13:11:15.962 INFO:teuthology.orchestra.run.smithi012.stdout:(39/150): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 80 kB 00:00 2024-04-22T13:11:16.030 INFO:teuthology.orchestra.run.smithi012.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-04-22T13:11:16.063 INFO:teuthology.orchestra.run.smithi012.stdout:(41/150): libgfortran-11.4.1-3.el9.x86_64.rpm 5.9 MB/s | 803 kB 00:00 2024-04-22T13:11:16.149 INFO:teuthology.orchestra.run.smithi012.stdout:(42/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-04-22T13:11:16.249 INFO:teuthology.orchestra.run.smithi012.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-04-22T13:11:16.291 INFO:teuthology.orchestra.run.smithi012.stdout:(44/150): python3-cryptography-36.0.1-4.el9.x86 5.4 MB/s | 1.2 MB 00:00 2024-04-22T13:11:16.333 INFO:teuthology.orchestra.run.smithi012.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-04-22T13:11:16.375 INFO:teuthology.orchestra.run.smithi012.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-04-22T13:11:16.419 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 16 kB 00:00 2024-04-22T13:11:16.425 INFO:teuthology.orchestra.run.smithi012.stdout:(47/150): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-04-22T13:11:16.534 INFO:teuthology.orchestra.run.smithi012.stdout:(48/150): automake-1.16.2-8.el9.noarch.rpm 4.2 MB/s | 689 kB 00:00 2024-04-22T13:11:16.634 INFO:teuthology.orchestra.run.smithi134.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T13:11:16.634 INFO:teuthology.orchestra.run.smithi134.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T13:11:16.646 INFO:teuthology.orchestra.run.smithi134.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-22T13:11:16.646 INFO:teuthology.orchestra.run.smithi134.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-22T13:11:16.647 INFO:teuthology.orchestra.run.smithi134.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-22T13:11:16.648 INFO:teuthology.orchestra.run.smithi134.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-22T13:11:16.650 INFO:teuthology.orchestra.run.smithi134.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-22T13:11:16.652 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-22T13:11:16.653 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-22T13:11:16.712 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-04-22T13:11:16.739 INFO:teuthology.orchestra.run.smithi134.stdout:======================================================================================= 2024-04-22T13:11:16.739 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2024-04-22T13:11:16.739 INFO:teuthology.orchestra.run.smithi134.stdout:======================================================================================= 2024-04-22T13:11:16.739 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-04-22T13:11:16.739 INFO:teuthology.orchestra.run.smithi134.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-22T13:11:16.739 INFO:teuthology.orchestra.run.smithi134.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-22T13:11:16.739 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-22T13:11:16.739 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-22T13:11:16.739 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-22T13:11:16.739 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-22T13:11:16.740 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-22T13:11:16.740 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-22T13:11:16.740 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-22T13:11:16.740 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-22T13:11:16.740 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-22T13:11:16.740 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-22T13:11:16.740 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-22T13:11:16.740 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-22T13:11:16.740 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-22T13:11:16.740 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-22T13:11:16.740 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-22T13:11:16.740 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-22T13:11:16.740 INFO:teuthology.orchestra.run.smithi134.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-22T13:11:16.741 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-22T13:11:16.741 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-22T13:11:16.741 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-22T13:11:16.741 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-22T13:11:16.741 INFO:teuthology.orchestra.run.smithi134.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-22T13:11:16.741 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-22T13:11:16.741 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-22T13:11:16.741 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-22T13:11:16.741 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-22T13:11:16.741 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-22T13:11:16.741 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-22T13:11:16.741 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-22T13:11:16.742 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-22T13:11:16.742 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-22T13:11:16.742 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-22T13:11:16.742 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-22T13:11:16.742 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-22T13:11:16.742 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-22T13:11:16.742 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrading: 2024-04-22T13:11:16.742 INFO:teuthology.orchestra.run.smithi134.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-22T13:11:16.742 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-22T13:11:16.742 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2024-04-22T13:11:16.742 INFO:teuthology.orchestra.run.smithi134.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-22T13:11:16.742 INFO:teuthology.orchestra.run.smithi134.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-22T13:11:16.742 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-22T13:11:16.743 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-22T13:11:16.743 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-22T13:11:16.743 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-22T13:11:16.743 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-22T13:11:16.743 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-22T13:11:16.743 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-22T13:11:16.743 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-22T13:11:16.743 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-22T13:11:16.743 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-22T13:11:16.743 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-22T13:11:16.743 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-22T13:11:16.743 INFO:teuthology.orchestra.run.smithi134.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-22T13:11:16.743 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-22T13:11:16.744 INFO:teuthology.orchestra.run.smithi134.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-22T13:11:16.744 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-22T13:11:16.744 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-22T13:11:16.744 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-22T13:11:16.744 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-22T13:11:16.744 INFO:teuthology.orchestra.run.smithi134.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-22T13:11:16.744 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-22T13:11:16.744 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-22T13:11:16.744 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-22T13:11:16.744 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-22T13:11:16.744 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-22T13:11:16.744 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-22T13:11:16.744 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-22T13:11:16.745 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-22T13:11:16.745 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-22T13:11:16.745 INFO:teuthology.orchestra.run.smithi134.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-22T13:11:16.745 INFO:teuthology.orchestra.run.smithi134.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-22T13:11:16.745 INFO:teuthology.orchestra.run.smithi134.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-22T13:11:16.745 INFO:teuthology.orchestra.run.smithi134.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-22T13:11:16.745 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-22T13:11:16.745 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-22T13:11:16.745 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-22T13:11:16.745 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-22T13:11:16.745 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-22T13:11:16.745 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-22T13:11:16.745 INFO:teuthology.orchestra.run.smithi134.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-22T13:11:16.745 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-22T13:11:16.746 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-22T13:11:16.746 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-22T13:11:16.746 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-22T13:11:16.746 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-22T13:11:16.746 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-22T13:11:16.746 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-22T13:11:16.746 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-22T13:11:16.746 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-22T13:11:16.746 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-22T13:11:16.746 INFO:teuthology.orchestra.run.smithi134.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-22T13:11:16.746 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-22T13:11:16.746 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-22T13:11:16.746 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-22T13:11:16.747 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-22T13:11:16.747 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-22T13:11:16.747 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-22T13:11:16.747 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-22T13:11:16.747 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-22T13:11:16.747 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-22T13:11:16.747 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-22T13:11:16.747 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-22T13:11:16.747 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-22T13:11:16.747 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-22T13:11:16.747 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-22T13:11:16.747 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-22T13:11:16.747 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-22T13:11:16.747 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-22T13:11:16.747 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-22T13:11:16.748 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-22T13:11:16.748 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-22T13:11:16.748 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-22T13:11:16.748 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-22T13:11:16.748 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-22T13:11:16.748 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-22T13:11:16.748 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-22T13:11:16.748 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-22T13:11:16.748 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-22T13:11:16.748 INFO:teuthology.orchestra.run.smithi134.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-22T13:11:16.748 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-22T13:11:16.748 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-22T13:11:16.748 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-22T13:11:16.748 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-22T13:11:16.748 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-22T13:11:16.749 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-22T13:11:16.749 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-22T13:11:16.749 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-22T13:11:16.749 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-22T13:11:16.749 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-22T13:11:16.749 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-22T13:11:16.749 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-22T13:11:16.749 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-22T13:11:16.749 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-22T13:11:16.749 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-22T13:11:16.749 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-22T13:11:16.749 INFO:teuthology.orchestra.run.smithi134.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-22T13:11:16.750 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-22T13:11:16.750 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-22T13:11:16.750 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-22T13:11:16.750 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-22T13:11:16.750 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-22T13:11:16.750 INFO:teuthology.orchestra.run.smithi134.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-22T13:11:16.750 INFO:teuthology.orchestra.run.smithi134.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-22T13:11:16.750 INFO:teuthology.orchestra.run.smithi134.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-22T13:11:16.750 INFO:teuthology.orchestra.run.smithi134.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-22T13:11:16.750 INFO:teuthology.orchestra.run.smithi134.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-22T13:11:16.750 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-22T13:11:16.750 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-22T13:11:16.750 INFO:teuthology.orchestra.run.smithi134.stdout:Installing weak dependencies: 2024-04-22T13:11:16.750 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-22T13:11:16.751 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-22T13:11:16.751 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-04-22T13:11:16.751 INFO:teuthology.orchestra.run.smithi134.stdout:======================================================================================= 2024-04-22T13:11:16.751 INFO:teuthology.orchestra.run.smithi134.stdout:Install 148 Packages 2024-04-22T13:11:16.751 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrade 2 Packages 2024-04-22T13:11:16.751 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-22T13:11:16.751 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 218 M 2024-04-22T13:11:16.751 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-04-22T13:11:17.001 INFO:teuthology.orchestra.run.smithi012.stdout:(49/150): ceph-test-19.0.0-3156.gd69f13d8.el7.x 12 MB/s | 49 MB 00:04 2024-04-22T13:11:17.042 INFO:teuthology.orchestra.run.smithi012.stdout:(50/150): bison-3.7.4-5.el9.x86_64.rpm 1.5 MB/s | 943 kB 00:00 2024-04-22T13:11:17.084 INFO:teuthology.orchestra.run.smithi012.stdout:(51/150): boost-program-options-1.75.0-8.el9.x8 194 kB/s | 107 kB 00:00 2024-04-22T13:11:17.102 INFO:teuthology.orchestra.run.smithi012.stdout:(52/150): elfutils-libelf-devel-0.190-2.el9.x86 214 kB/s | 21 kB 00:00 2024-04-22T13:11:17.135 INFO:teuthology.orchestra.run.smithi012.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 652 kB/s | 33 kB 00:00 2024-04-22T13:11:17.160 INFO:teuthology.orchestra.run.smithi012.stdout:(54/150): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-04-22T13:11:17.194 INFO:teuthology.orchestra.run.smithi012.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 313 kB/s | 18 kB 00:00 2024-04-22T13:11:17.219 INFO:teuthology.orchestra.run.smithi012.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-04-22T13:11:17.278 INFO:teuthology.orchestra.run.smithi012.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-04-22T13:11:17.303 INFO:teuthology.orchestra.run.smithi012.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 543 kB/s | 45 kB 00:00 2024-04-22T13:11:17.403 INFO:teuthology.orchestra.run.smithi012.stdout:(59/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.9 MB/s | 3.0 MB 00:00 2024-04-22T13:11:17.504 INFO:teuthology.orchestra.run.smithi012.stdout:(60/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 255 kB/s | 25 kB 00:00 2024-04-22T13:11:17.545 INFO:teuthology.orchestra.run.smithi012.stdout:(61/150): libtool-2.4.6-45.el9.x86_64.rpm 2.3 MB/s | 582 kB 00:00 2024-04-22T13:11:17.579 INFO:teuthology.orchestra.run.smithi012.stdout:(62/150): librdkafka-1.6.1-102.el9.x86_64.rpm 2.3 MB/s | 662 kB 00:00 2024-04-22T13:11:17.604 INFO:teuthology.orchestra.run.smithi012.stdout:(63/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 471 kB/s | 47 kB 00:00 2024-04-22T13:11:17.655 INFO:teuthology.orchestra.run.smithi012.stdout:(64/150): openblas-0.3.21-2.el9.x86_64.rpm 706 kB/s | 35 kB 00:00 2024-04-22T13:11:17.680 INFO:teuthology.orchestra.run.smithi012.stdout:(65/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-04-22T13:11:17.705 INFO:teuthology.orchestra.run.smithi012.stdout:(66/150): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-04-22T13:11:17.914 INFO:teuthology.orchestra.run.smithi012.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-04-22T13:11:18.064 INFO:teuthology.orchestra.run.smithi012.stdout:(68/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 11 MB/s | 4.1 MB 00:00 2024-04-22T13:11:18.131 INFO:teuthology.orchestra.run.smithi012.stdout:(69/150): openblas-openmp-0.3.21-2.el9.x86_64.r 10 MB/s | 4.7 MB 00:00 2024-04-22T13:11:18.145 INFO:teuthology.orchestra.run.smithi134.stdout:(1/150): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 13 kB/s | 6.6 kB 00:00 2024-04-22T13:11:18.172 INFO:teuthology.orchestra.run.smithi012.stdout:(70/150): postgresql-contrib-13.14-1.el9.x86_64 3.4 MB/s | 911 kB 00:00 2024-04-22T13:11:18.215 INFO:teuthology.orchestra.run.smithi012.stdout:(71/150): postgresql-private-libs-13.14-1.el9.x 887 kB/s | 133 kB 00:00 2024-04-22T13:11:18.282 INFO:teuthology.orchestra.run.smithi012.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-22T13:11:18.332 INFO:teuthology.orchestra.run.smithi012.stdout:(73/150): protobuf-3.14.0-13.el9.x86_64.rpm 6.3 MB/s | 1.0 MB 00:00 2024-04-22T13:11:18.415 INFO:teuthology.orchestra.run.smithi012.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-04-22T13:11:18.507 INFO:teuthology.orchestra.run.smithi012.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-04-22T13:11:18.583 INFO:teuthology.orchestra.run.smithi012.stdout:(76/150): postgresql-server-13.14-1.el9.x86_64. 13 MB/s | 5.8 MB 00:00 2024-04-22T13:11:18.641 INFO:teuthology.orchestra.run.smithi012.stdout:(77/150): python3-jsonpatch-1.21-16.el9.noarch. 192 kB/s | 26 kB 00:00 2024-04-22T13:11:18.667 INFO:teuthology.orchestra.run.smithi012.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 229 kB/s | 19 kB 00:00 2024-04-22T13:11:18.726 INFO:teuthology.orchestra.run.smithi012.stdout:(79/150): python3-markupsafe-1.1.1-12.el9.x86_6 597 kB/s | 35 kB 00:00 2024-04-22T13:11:18.742 INFO:teuthology.orchestra.run.smithi012.stdout:(80/150): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-04-22T13:11:18.901 INFO:teuthology.orchestra.run.smithi134.stdout:(2/150): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.2 MB/s | 890 kB 00:00 2024-04-22T13:11:18.938 INFO:teuthology.orchestra.run.smithi012.stdout:(81/150): python3-babel-2.9.1-2.el9.noarch.rpm 9.1 MB/s | 6.0 MB 00:00 2024-04-22T13:11:18.959 INFO:teuthology.orchestra.run.smithi012.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 250 kB 00:00 2024-04-22T13:11:19.043 INFO:teuthology.orchestra.run.smithi012.stdout:(83/150): python3-packaging-20.9-5.el9.noarch.r 927 kB/s | 77 kB 00:00 2024-04-22T13:11:19.068 INFO:teuthology.orchestra.run.smithi012.stdout:(84/150): python3-oauthlib-3.1.1-5.el9.noarch.r 1.6 MB/s | 222 kB 00:00 2024-04-22T13:11:19.110 INFO:teuthology.orchestra.run.smithi012.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 633 kB/s | 42 kB 00:00 2024-04-22T13:11:19.169 INFO:teuthology.orchestra.run.smithi012.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-04-22T13:11:19.184 INFO:teuthology.orchestra.run.smithi134.stdout:(3/150): ceph-common-19.0.0-3156.gd69f13d8.el7. 14 MB/s | 21 MB 00:01 2024-04-22T13:11:19.201 INFO:teuthology.orchestra.run.smithi134.stdout:(4/150): ceph-immutable-object-cache-19.0.0-315 482 kB/s | 145 kB 00:00 2024-04-22T13:11:19.211 INFO:teuthology.orchestra.run.smithi012.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 159 kB 00:00 2024-04-22T13:11:19.344 INFO:teuthology.orchestra.run.smithi012.stdout:(88/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.2 MB/s | 5.1 MB 00:00 2024-04-22T13:11:19.361 INFO:teuthology.orchestra.run.smithi012.stdout:(89/150): python3-pytz-2021.1-5.el9.noarch.rpm 339 kB/s | 51 kB 00:00 2024-04-22T13:11:19.386 INFO:teuthology.orchestra.run.smithi012.stdout:(90/150): python3-pyasn1-modules-0.4.8-6.el9.no 1.3 MB/s | 279 kB 00:00 2024-04-22T13:11:19.412 INFO:teuthology.orchestra.run.smithi012.stdout:(91/150): python3-requests-oauthlib-1.3.0-12.el 799 kB/s | 54 kB 00:00 2024-04-22T13:11:19.470 INFO:teuthology.orchestra.run.smithi012.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 498 kB/s | 42 kB 00:00 2024-04-22T13:11:19.487 INFO:teuthology.orchestra.run.smithi012.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 590 kB/s | 44 kB 00:00 2024-04-22T13:11:19.526 INFO:teuthology.orchestra.run.smithi134.stdout:(5/150): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 2.8 MB/s | 5.4 MB 00:01 2024-04-22T13:11:19.538 INFO:teuthology.orchestra.run.smithi012.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-04-22T13:11:19.579 INFO:teuthology.orchestra.run.smithi134.stdout:(6/150): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 6.0 MB/s | 2.4 MB 00:00 2024-04-22T13:11:19.588 INFO:teuthology.orchestra.run.smithi012.stdout:(95/150): qatlib-24.02.0-1.el9.x86_64.rpm 1.8 MB/s | 221 kB 00:00 2024-04-22T13:11:19.661 INFO:teuthology.orchestra.run.smithi111.stdout:Extra Packages for Enterprise Linux 6.7 MB/s | 21 MB 00:03 2024-04-22T13:11:19.663 INFO:teuthology.orchestra.run.smithi012.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 379 kB/s | 47 kB 00:00 2024-04-22T13:11:19.722 INFO:teuthology.orchestra.run.smithi012.stdout:(97/150): socat-1.7.4.1-5.el9.x86_64.rpm 2.2 MB/s | 305 kB 00:00 2024-04-22T13:11:19.729 INFO:teuthology.orchestra.run.smithi134.stdout:(7/150): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 2.8 MB/s | 1.5 MB 00:00 2024-04-22T13:11:19.747 INFO:teuthology.orchestra.run.smithi012.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 692 kB/s | 58 kB 00:00 2024-04-22T13:11:19.805 INFO:teuthology.orchestra.run.smithi012.stdout:(99/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-04-22T13:11:19.831 INFO:teuthology.orchestra.run.smithi012.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 761 kB/s | 64 kB 00:00 2024-04-22T13:11:19.889 INFO:teuthology.orchestra.run.smithi012.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 536 kB/s | 45 kB 00:00 2024-04-22T13:11:19.982 INFO:teuthology.orchestra.run.smithi012.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 148 kB/s | 22 kB 00:00 2024-04-22T13:11:20.340 INFO:teuthology.orchestra.run.smithi012.stdout:(103/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-04-22T13:11:20.362 INFO:teuthology.orchestra.run.smithi134.stdout:(8/150): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 5.6 MB/s | 4.7 MB 00:00 2024-04-22T13:11:20.374 INFO:teuthology.orchestra.run.smithi012.stdout:(104/150): protobuf-compiler-3.14.0-13.el9.x86_ 1.7 MB/s | 863 kB 00:00 2024-04-22T13:11:20.538 INFO:teuthology.orchestra.run.smithi134.stdout:(9/150): ceph-osd-19.0.0-3156.gd69f13d8.el7.x86 18 MB/s | 17 MB 00:00 2024-04-22T13:11:20.563 INFO:teuthology.orchestra.run.smithi134.stdout:(10/150): ceph-selinux-19.0.0-3156.gd69f13d8.el 125 kB/s | 25 kB 00:00 2024-04-22T13:11:20.641 INFO:teuthology.orchestra.run.smithi012.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 73 kB/s | 19 kB 00:00 2024-04-22T13:11:20.666 INFO:teuthology.orchestra.run.smithi012.stdout:(106/150): abseil-cpp-20211102.0-3.el9.x86_64.r 801 kB/s | 548 kB 00:00 2024-04-22T13:11:20.680 INFO:teuthology.orchestra.run.smithi134.stdout:(11/150): cephfs-mirror-19.0.0-3156.gd69f13d8.e 1.9 MB/s | 225 kB 00:00 2024-04-22T13:11:20.796 INFO:teuthology.orchestra.run.smithi134.stdout:(12/150): cephfs-shell-19.0.0-3156.gd69f13d8.el 232 kB/s | 27 kB 00:00 2024-04-22T13:11:20.850 INFO:teuthology.orchestra.run.smithi012.stdout:(107/150): gperftools-libs-2.9.1-2.el9.x86_64.r 607 kB/s | 309 kB 00:00 2024-04-22T13:11:20.905 INFO:teuthology.orchestra.run.smithi134.stdout:(13/150): libcephfs-devel-19.0.0-3156.gd69f13d8 293 kB/s | 32 kB 00:00 2024-04-22T13:11:20.933 INFO:teuthology.orchestra.run.smithi012.stdout:(108/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 97 kB/s | 26 kB 00:00 2024-04-22T13:11:21.193 INFO:teuthology.orchestra.run.smithi012.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 142 kB/s | 49 kB 00:00 2024-04-22T13:11:21.276 INFO:teuthology.orchestra.run.smithi012.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 197 kB/s | 67 kB 00:00 2024-04-22T13:11:21.314 INFO:teuthology.orchestra.run.smithi134.stdout:(14/150): libcephfs2-19.0.0-3156.gd69f13d8.el7. 1.8 MB/s | 747 kB 00:00 2024-04-22T13:11:21.431 INFO:teuthology.orchestra.run.smithi134.stdout:(15/150): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.4 MB/s | 163 kB 00:00 2024-04-22T13:11:21.539 INFO:teuthology.orchestra.run.smithi134.stdout:(16/150): librados-devel-19.0.0-3156.gd69f13d8. 1.1 MB/s | 126 kB 00:00 2024-04-22T13:11:21.634 INFO:teuthology.orchestra.run.smithi012.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 342 kB/s | 151 kB 00:00 2024-04-22T13:11:21.739 INFO:teuthology.orchestra.run.smithi134.stdout:(17/150): libradosstriper1-19.0.0-3156.gd69f13d 2.4 MB/s | 500 kB 00:00 2024-04-22T13:11:21.943 INFO:teuthology.orchestra.run.smithi012.stdout:(112/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.2 MB/s | 838 kB 00:00 2024-04-22T13:11:22.010 INFO:teuthology.orchestra.run.smithi012.stdout:(113/150): libarrow-9.0.0-10.el9.x86_64.rpm 3.2 MB/s | 4.4 MB 00:01 2024-04-22T13:11:22.023 INFO:teuthology.orchestra.run.smithi134.stdout:(18/150): ceph-radosgw-19.0.0-3156.gd69f13d8.el 4.6 MB/s | 11 MB 00:02 2024-04-22T13:11:22.115 INFO:teuthology.orchestra.run.smithi134.stdout:(19/150): python3-ceph-argparse-19.0.0-3156.gd6 494 kB/s | 45 kB 00:00 2024-04-22T13:11:22.240 INFO:teuthology.orchestra.run.smithi134.stdout:(20/150): python3-ceph-common-19.0.0-3156.gd69f 1.1 MB/s | 135 kB 00:00 2024-04-22T13:11:22.243 INFO:teuthology.orchestra.run.smithi012.stdout:(114/150): python3-asyncssh-2.13.2-5.el9.noarch 900 kB/s | 548 kB 00:00 2024-04-22T13:11:22.277 INFO:teuthology.orchestra.run.smithi012.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 120 kB/s | 32 kB 00:00 2024-04-22T13:11:22.302 INFO:teuthology.orchestra.run.smithi012.stdout:(116/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 121 kB/s | 43 kB 00:00 2024-04-22T13:11:22.357 INFO:teuthology.orchestra.run.smithi134.stdout:(21/150): python3-cephfs-19.0.0-3156.gd69f13d8. 1.4 MB/s | 161 kB 00:00 2024-04-22T13:11:22.483 INFO:teuthology.orchestra.run.smithi134.stdout:(22/150): python3-rados-19.0.0-3156.gd69f13d8.e 2.5 MB/s | 320 kB 00:00 2024-04-22T13:11:22.511 INFO:teuthology.orchestra.run.smithi012.stdout:(117/150): python3-certifi-2021.10.8-2.el9.noar 56 kB/s | 15 kB 00:00 2024-04-22T13:11:22.608 INFO:teuthology.orchestra.run.smithi134.stdout:(23/150): python3-rbd-19.0.0-3156.gd69f13d8.el7 2.3 MB/s | 298 kB 00:00 2024-04-22T13:11:22.703 INFO:teuthology.orchestra.run.smithi012.stdout:(118/150): python3-cheroot-8.6.0-4.el9.noarch.r 404 kB/s | 172 kB 00:00 2024-04-22T13:11:22.725 INFO:teuthology.orchestra.run.smithi134.stdout:(24/150): python3-rgw-19.0.0-3156.gd69f13d8.el7 854 kB/s | 99 kB 00:00 2024-04-22T13:11:22.800 INFO:teuthology.orchestra.run.smithi134.stdout:(25/150): librgw2-19.0.0-3156.gd69f13d8.el7.x86 5.1 MB/s | 5.4 MB 00:01 2024-04-22T13:11:22.826 INFO:teuthology.orchestra.run.smithi134.stdout:(26/150): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 849 kB/s | 85 kB 00:00 2024-04-22T13:11:22.886 INFO:teuthology.orchestra.run.smithi012.stdout:(119/150): python3-cherrypy-18.6.1-2.el9.noarch 614 kB/s | 358 kB 00:00 2024-04-22T13:11:22.970 INFO:teuthology.orchestra.run.smithi012.stdout:(120/150): python3-colorama-0.4.4-12.el9.noarch 132 kB/s | 35 kB 00:00 2024-04-22T13:11:23.012 INFO:teuthology.orchestra.run.smithi012.stdout:(121/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 605 kB/s | 303 kB 00:00 2024-04-22T13:11:23.260 INFO:teuthology.orchestra.run.smithi134.stdout:(27/150): ceph-test-19.0.0-3156.gd69f13d8.el7.x 18 MB/s | 49 MB 00:02 2024-04-22T13:11:23.284 INFO:teuthology.orchestra.run.smithi134.stdout:(28/150): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 372 kB/s | 170 kB 00:00 2024-04-22T13:11:23.310 INFO:teuthology.orchestra.run.smithi134.stdout:(29/150): ceph-grafana-dashboards-19.0.0-3156.g 687 kB/s | 34 kB 00:00 2024-04-22T13:11:23.368 INFO:teuthology.orchestra.run.smithi134.stdout:(30/150): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 5.4 MB/s | 3.1 MB 00:00 2024-04-22T13:11:23.394 INFO:teuthology.orchestra.run.smithi134.stdout:(31/150): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.3 MB/s | 142 kB 00:00 2024-04-22T13:11:23.395 INFO:teuthology.orchestra.run.smithi012.stdout:(122/150): python3-google-auth-2.28.2-1.el9.noa 398 kB/s | 203 kB 00:00 2024-04-22T13:11:23.437 INFO:teuthology.orchestra.run.smithi012.stdout:(123/150): python3-grpcio-tools-1.46.7-10.el9.x 340 kB/s | 144 kB 00:00 2024-04-22T13:11:23.536 INFO:teuthology.orchestra.run.smithi134.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3156.gd6 1.7 MB/s | 249 kB 00:00 2024-04-22T13:11:23.586 INFO:teuthology.orchestra.run.smithi134.stdout:(33/150): ceph-mgr-dashboard-19.0.0-3156.gd69f1 13 MB/s | 3.5 MB 00:00 2024-04-22T13:11:23.588 INFO:teuthology.orchestra.run.smithi012.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 56 kB/s | 11 kB 00:00 2024-04-22T13:11:23.619 INFO:teuthology.orchestra.run.smithi134.stdout:(34/150): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 591 kB/s | 49 kB 00:00 2024-04-22T13:11:23.645 INFO:teuthology.orchestra.run.smithi134.stdout:(35/150): ceph-prometheus-alerts-19.0.0-3156.gd 280 kB/s | 16 kB 00:00 2024-04-22T13:11:23.696 INFO:teuthology.orchestra.run.smithi012.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 68 kB/s | 18 kB 00:00 2024-04-22T13:11:23.720 INFO:teuthology.orchestra.run.smithi134.stdout:(36/150): cephadm-19.0.0-3156.gd69f13d8.el7.noa 9.9 MB/s | 762 kB 00:00 2024-04-22T13:11:23.746 INFO:teuthology.orchestra.run.smithi134.stdout:(37/150): ceph-volume-19.0.0-3156.gd69f13d8.el7 2.1 MB/s | 273 kB 00:00 2024-04-22T13:11:23.762 INFO:teuthology.orchestra.run.smithi134.stdout:(38/150): cephfs-top-19.0.0-3156.gd69f13d8.el7. 436 kB/s | 18 kB 00:00 2024-04-22T13:11:23.846 INFO:teuthology.orchestra.run.smithi012.stdout:(126/150): python3-grpcio-1.46.7-10.el9.x86_64. 2.3 MB/s | 2.0 MB 00:00 2024-04-22T13:11:23.863 INFO:teuthology.orchestra.run.smithi012.stdout:(127/150): python3-jaraco-collections-3.0.0-8.e 84 kB/s | 23 kB 00:00 2024-04-22T13:11:23.955 INFO:teuthology.orchestra.run.smithi012.stdout:(128/150): python3-jaraco-functools-3.5.0-2.el9 75 kB/s | 19 kB 00:00 2024-04-22T13:11:24.105 INFO:teuthology.orchestra.run.smithi012.stdout:(129/150): python3-jaraco-text-3.2.0-6.el9.noar 76 kB/s | 20 kB 00:00 2024-04-22T13:11:24.221 INFO:teuthology.orchestra.run.smithi134.stdout:(39/150): fuse-2.9.9-15.el9.x86_64.rpm 169 kB/s | 80 kB 00:00 2024-04-22T13:11:24.298 INFO:teuthology.orchestra.run.smithi012.stdout:(130/150): python3-logutils-0.3.5-21.el9.noarch 135 kB/s | 46 kB 00:00 2024-04-22T13:11:24.456 INFO:teuthology.orchestra.run.smithi012.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 225 kB/s | 79 kB 00:00 2024-04-22T13:11:24.471 INFO:teuthology.orchestra.run.smithi134.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 782 kB/s | 195 kB 00:00 2024-04-22T13:11:24.572 INFO:teuthology.orchestra.run.smithi134.stdout:(41/150): ceph-mgr-diskprediction-local-19.0.0- 6.1 MB/s | 7.4 MB 00:01 2024-04-22T13:11:24.573 INFO:teuthology.orchestra.run.smithi012.stdout:(132/150): python3-kubernetes-26.1.0-2.el9.noar 1.4 MB/s | 1.0 MB 00:00 2024-04-22T13:11:24.611 INFO:teuthology.orchestra.run.smithi111.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-22T13:11:24.648 INFO:teuthology.orchestra.run.smithi012.stdout:(133/150): python3-natsort-7.1.1-5.el9.noarch.r 165 kB/s | 58 kB 00:00 2024-04-22T13:11:24.699 INFO:teuthology.orchestra.run.smithi134.stdout:(42/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-04-22T13:11:24.813 INFO:teuthology.orchestra.run.smithi134.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 915 kB/s | 106 kB 00:00 2024-04-22T13:11:24.840 INFO:teuthology.orchestra.run.smithi012.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 61 kB/s | 16 kB 00:00 2024-04-22T13:11:24.897 INFO:teuthology.orchestra.run.smithi134.stdout:(44/150): libgfortran-11.4.1-3.el9.x86_64.rpm 708 kB/s | 803 kB 00:01 2024-04-22T13:11:24.931 INFO:teuthology.orchestra.run.smithi134.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-04-22T13:11:24.957 INFO:teuthology.orchestra.run.smithi012.stdout:(135/150): python3-pecan-1.4.2-3.el9.noarch.rpm 544 kB/s | 272 kB 00:00 2024-04-22T13:11:25.014 INFO:teuthology.orchestra.run.smithi134.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-04-22T13:11:25.056 INFO:teuthology.orchestra.run.smithi134.stdout:(47/150): python3-urllib3-1.26.5-5.el9.noarch.r 1.7 MB/s | 215 kB 00:00 2024-04-22T13:11:25.074 INFO:teuthology.orchestra.run.smithi012.stdout:(136/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 211 kB/s | 90 kB 00:00 2024-04-22T13:11:25.099 INFO:teuthology.orchestra.run.smithi012.stdout:(137/150): python3-pyperclip-1.8.2-2.el9.noarch 109 kB/s | 28 kB 00:00 2024-04-22T13:11:25.225 INFO:teuthology.orchestra.run.smithi012.stdout:(138/150): python3-repoze-lru-0.7-16.el9.noarch 115 kB/s | 31 kB 00:00 2024-04-22T13:11:25.450 INFO:teuthology.orchestra.run.smithi012.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 169 kB/s | 59 kB 00:00 2024-04-22T13:11:25.473 INFO:teuthology.orchestra.run.smithi134.stdout:(48/150): automake-1.16.2-8.el9.noarch.rpm 1.5 MB/s | 689 kB 00:00 2024-04-22T13:11:25.492 INFO:teuthology.orchestra.run.smithi012.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 134 kB/s | 36 kB 00:00 2024-04-22T13:11:25.548 INFO:teuthology.orchestra.run.smithi134.stdout:(49/150): bison-3.7.4-5.el9.x86_64.rpm 1.9 MB/s | 943 kB 00:00 2024-04-22T13:11:25.574 INFO:teuthology.orchestra.run.smithi134.stdout:(50/150): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 107 kB 00:00 2024-04-22T13:11:25.583 INFO:teuthology.orchestra.run.smithi012.stdout:(141/150): python3-routes-2.5.1-5.el9.noarch.rp 370 kB/s | 188 kB 00:00 2024-04-22T13:11:25.607 INFO:teuthology.orchestra.run.smithi134.stdout:(51/150): python3-cryptography-36.0.1-4.el9.x86 1.2 MB/s | 1.2 MB 00:01 2024-04-22T13:11:25.632 INFO:teuthology.orchestra.run.smithi134.stdout:(52/150): elfutils-libelf-devel-0.190-2.el9.x86 257 kB/s | 21 kB 00:00 2024-04-22T13:11:25.708 INFO:teuthology.orchestra.run.smithi134.stdout:(53/150): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-04-22T13:11:25.783 INFO:teuthology.orchestra.run.smithi134.stdout:(54/150): flexiblas-3.0.4-8.el9.x86_64.rpm 186 kB/s | 33 kB 00:00 2024-04-22T13:11:25.808 INFO:teuthology.orchestra.run.smithi134.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 181 kB/s | 18 kB 00:00 2024-04-22T13:11:25.858 INFO:teuthology.orchestra.run.smithi134.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-04-22T13:11:25.875 INFO:teuthology.orchestra.run.smithi134.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-04-22T13:11:25.917 INFO:teuthology.orchestra.run.smithi012.stdout:(142/150): python3-typing-extensions-4.4.0-2.el 109 kB/s | 51 kB 00:00 2024-04-22T13:11:25.975 INFO:teuthology.orchestra.run.smithi134.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 387 kB/s | 45 kB 00:00 2024-04-22T13:11:26.009 INFO:teuthology.orchestra.run.smithi012.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 211 kB/s | 90 kB 00:00 2024-04-22T13:11:26.034 INFO:teuthology.orchestra.run.smithi012.stdout:(144/150): python3-webob-1.8.7-6.el9.noarch.rpm 423 kB/s | 230 kB 00:00 2024-04-22T13:11:26.059 INFO:teuthology.orchestra.run.smithi134.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 3.5 MB/s | 662 kB 00:00 2024-04-22T13:11:26.135 INFO:teuthology.orchestra.run.smithi134.stdout:(60/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 339 kB/s | 25 kB 00:00 2024-04-22T13:11:26.193 INFO:teuthology.orchestra.run.smithi134.stdout:(61/150): libtool-2.4.6-45.el9.x86_64.rpm 2.6 MB/s | 582 kB 00:00 2024-04-22T13:11:26.218 INFO:teuthology.orchestra.run.smithi134.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 568 kB/s | 47 kB 00:00 2024-04-22T13:11:26.276 INFO:teuthology.orchestra.run.smithi012.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 75 kB/s | 20 kB 00:00 2024-04-22T13:11:26.302 INFO:teuthology.orchestra.run.smithi134.stdout:(63/150): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-04-22T13:11:26.336 INFO:teuthology.orchestra.run.smithi134.stdout:(64/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-04-22T13:11:26.386 INFO:teuthology.orchestra.run.smithi134.stdout:(65/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.0 MB/s | 3.0 MB 00:00 2024-04-22T13:11:26.411 INFO:teuthology.orchestra.run.smithi134.stdout:(66/150): openblas-0.3.21-2.el9.x86_64.rpm 323 kB/s | 35 kB 00:00 2024-04-22T13:11:26.518 INFO:teuthology.orchestra.run.smithi012.stdout:(146/150): python3-werkzeug-2.0.3-3.el9.1.noarc 711 kB/s | 427 kB 00:00 2024-04-22T13:11:26.543 INFO:teuthology.orchestra.run.smithi012.stdout:(147/150): re2-20211101-3.el9.x86_64.rpm 377 kB/s | 192 kB 00:00 2024-04-22T13:11:26.695 INFO:teuthology.orchestra.run.smithi111.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T13:11:26.695 INFO:teuthology.orchestra.run.smithi111.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T13:11:26.706 INFO:teuthology.orchestra.run.smithi111.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-22T13:11:26.707 INFO:teuthology.orchestra.run.smithi111.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-22T13:11:26.707 INFO:teuthology.orchestra.run.smithi111.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-22T13:11:26.708 INFO:teuthology.orchestra.run.smithi111.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-22T13:11:26.710 INFO:teuthology.orchestra.run.smithi111.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-22T13:11:26.712 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-22T13:11:26.713 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-22T13:11:26.772 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-04-22T13:11:26.790 INFO:teuthology.orchestra.run.smithi111.stdout:======================================================================================= 2024-04-22T13:11:26.790 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repository Size 2024-04-22T13:11:26.790 INFO:teuthology.orchestra.run.smithi111.stdout:======================================================================================= 2024-04-22T13:11:26.790 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2024-04-22T13:11:26.790 INFO:teuthology.orchestra.run.smithi111.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-22T13:11:26.790 INFO:teuthology.orchestra.run.smithi111.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-22T13:11:26.791 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-22T13:11:26.791 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-22T13:11:26.791 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-22T13:11:26.791 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-22T13:11:26.791 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-22T13:11:26.791 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-22T13:11:26.791 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-22T13:11:26.791 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-22T13:11:26.791 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-22T13:11:26.791 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-22T13:11:26.791 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-22T13:11:26.791 INFO:teuthology.orchestra.run.smithi111.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-22T13:11:26.792 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-22T13:11:26.792 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-22T13:11:26.792 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-22T13:11:26.792 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-22T13:11:26.792 INFO:teuthology.orchestra.run.smithi111.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-22T13:11:26.792 INFO:teuthology.orchestra.run.smithi111.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-22T13:11:26.792 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-22T13:11:26.792 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-22T13:11:26.792 INFO:teuthology.orchestra.run.smithi111.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-22T13:11:26.792 INFO:teuthology.orchestra.run.smithi111.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-22T13:11:26.792 INFO:teuthology.orchestra.run.smithi111.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-22T13:11:26.792 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-22T13:11:26.792 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-22T13:11:26.793 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-22T13:11:26.793 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-22T13:11:26.793 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-22T13:11:26.793 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-22T13:11:26.793 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-22T13:11:26.793 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-22T13:11:26.793 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-22T13:11:26.793 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-22T13:11:26.793 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-22T13:11:26.793 INFO:teuthology.orchestra.run.smithi111.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-22T13:11:26.793 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrading: 2024-04-22T13:11:26.793 INFO:teuthology.orchestra.run.smithi111.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-22T13:11:26.794 INFO:teuthology.orchestra.run.smithi111.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-22T13:11:26.794 INFO:teuthology.orchestra.run.smithi111.stdout:Installing dependencies: 2024-04-22T13:11:26.794 INFO:teuthology.orchestra.run.smithi111.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-22T13:11:26.794 INFO:teuthology.orchestra.run.smithi111.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-22T13:11:26.794 INFO:teuthology.orchestra.run.smithi111.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-22T13:11:26.794 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-22T13:11:26.794 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-22T13:11:26.794 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-22T13:11:26.794 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-22T13:11:26.794 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-22T13:11:26.794 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-22T13:11:26.794 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-22T13:11:26.795 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-22T13:11:26.795 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-22T13:11:26.795 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-22T13:11:26.795 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-22T13:11:26.795 INFO:teuthology.orchestra.run.smithi111.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-22T13:11:26.795 INFO:teuthology.orchestra.run.smithi111.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-22T13:11:26.795 INFO:teuthology.orchestra.run.smithi111.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-22T13:11:26.795 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-22T13:11:26.795 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-22T13:11:26.795 INFO:teuthology.orchestra.run.smithi111.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-22T13:11:26.796 INFO:teuthology.orchestra.run.smithi111.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-22T13:11:26.796 INFO:teuthology.orchestra.run.smithi111.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-22T13:11:26.796 INFO:teuthology.orchestra.run.smithi111.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-22T13:11:26.796 INFO:teuthology.orchestra.run.smithi111.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-22T13:11:26.796 INFO:teuthology.orchestra.run.smithi111.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-22T13:11:26.796 INFO:teuthology.orchestra.run.smithi111.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-22T13:11:26.796 INFO:teuthology.orchestra.run.smithi111.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-22T13:11:26.796 INFO:teuthology.orchestra.run.smithi111.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-22T13:11:26.796 INFO:teuthology.orchestra.run.smithi111.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-22T13:11:26.796 INFO:teuthology.orchestra.run.smithi111.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-22T13:11:26.796 INFO:teuthology.orchestra.run.smithi111.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-22T13:11:26.796 INFO:teuthology.orchestra.run.smithi111.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-22T13:11:26.797 INFO:teuthology.orchestra.run.smithi111.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-22T13:11:26.797 INFO:teuthology.orchestra.run.smithi111.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-22T13:11:26.797 INFO:teuthology.orchestra.run.smithi111.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-22T13:11:26.797 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-22T13:11:26.797 INFO:teuthology.orchestra.run.smithi111.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-22T13:11:26.797 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-22T13:11:26.797 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-22T13:11:26.797 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-22T13:11:26.797 INFO:teuthology.orchestra.run.smithi111.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-22T13:11:26.797 INFO:teuthology.orchestra.run.smithi111.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-22T13:11:26.797 INFO:teuthology.orchestra.run.smithi111.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-22T13:11:26.797 INFO:teuthology.orchestra.run.smithi111.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-22T13:11:26.798 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-22T13:11:26.798 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-22T13:11:26.798 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-22T13:11:26.798 INFO:teuthology.orchestra.run.smithi111.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-22T13:11:26.798 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-22T13:11:26.798 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-22T13:11:26.798 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-22T13:11:26.798 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-22T13:11:26.798 INFO:teuthology.orchestra.run.smithi111.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-22T13:11:26.798 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-22T13:11:26.799 INFO:teuthology.orchestra.run.smithi111.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-22T13:11:26.799 INFO:teuthology.orchestra.run.smithi111.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-22T13:11:26.799 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-22T13:11:26.799 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-22T13:11:26.799 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-22T13:11:26.799 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-22T13:11:26.799 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-22T13:11:26.799 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-22T13:11:26.799 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-22T13:11:26.799 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-22T13:11:26.800 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-22T13:11:26.800 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-22T13:11:26.800 INFO:teuthology.orchestra.run.smithi111.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-22T13:11:26.800 INFO:teuthology.orchestra.run.smithi111.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-22T13:11:26.800 INFO:teuthology.orchestra.run.smithi111.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-22T13:11:26.800 INFO:teuthology.orchestra.run.smithi111.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-22T13:11:26.800 INFO:teuthology.orchestra.run.smithi111.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-22T13:11:26.800 INFO:teuthology.orchestra.run.smithi111.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-22T13:11:26.800 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-22T13:11:26.800 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-22T13:11:26.800 INFO:teuthology.orchestra.run.smithi111.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-22T13:11:26.800 INFO:teuthology.orchestra.run.smithi111.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-22T13:11:26.801 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-22T13:11:26.801 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-22T13:11:26.801 INFO:teuthology.orchestra.run.smithi111.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-22T13:11:26.801 INFO:teuthology.orchestra.run.smithi111.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-22T13:11:26.801 INFO:teuthology.orchestra.run.smithi111.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-22T13:11:26.801 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-22T13:11:26.801 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-22T13:11:26.801 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-22T13:11:26.801 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-22T13:11:26.801 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-22T13:11:26.801 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-22T13:11:26.801 INFO:teuthology.orchestra.run.smithi111.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-22T13:11:26.802 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-22T13:11:26.802 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-22T13:11:26.802 INFO:teuthology.orchestra.run.smithi111.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-22T13:11:26.802 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-22T13:11:26.802 INFO:teuthology.orchestra.run.smithi111.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-22T13:11:26.802 INFO:teuthology.orchestra.run.smithi111.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-22T13:11:26.802 INFO:teuthology.orchestra.run.smithi111.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-22T13:11:26.802 INFO:teuthology.orchestra.run.smithi111.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-22T13:11:26.802 INFO:teuthology.orchestra.run.smithi111.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-22T13:11:26.802 INFO:teuthology.orchestra.run.smithi111.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-22T13:11:26.802 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-22T13:11:26.803 INFO:teuthology.orchestra.run.smithi111.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-22T13:11:26.803 INFO:teuthology.orchestra.run.smithi111.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-22T13:11:26.803 INFO:teuthology.orchestra.run.smithi111.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-22T13:11:26.803 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-22T13:11:26.803 INFO:teuthology.orchestra.run.smithi111.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-22T13:11:26.803 INFO:teuthology.orchestra.run.smithi111.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-22T13:11:26.803 INFO:teuthology.orchestra.run.smithi111.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-22T13:11:26.803 INFO:teuthology.orchestra.run.smithi111.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-22T13:11:26.803 INFO:teuthology.orchestra.run.smithi111.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-22T13:11:26.803 INFO:teuthology.orchestra.run.smithi111.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-22T13:11:26.803 INFO:teuthology.orchestra.run.smithi111.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-22T13:11:26.804 INFO:teuthology.orchestra.run.smithi111.stdout:Installing weak dependencies: 2024-04-22T13:11:26.804 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-22T13:11:26.804 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-22T13:11:26.804 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2024-04-22T13:11:26.804 INFO:teuthology.orchestra.run.smithi111.stdout:======================================================================================= 2024-04-22T13:11:26.804 INFO:teuthology.orchestra.run.smithi111.stdout:Install 148 Packages 2024-04-22T13:11:26.804 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrade 2 Packages 2024-04-22T13:11:26.804 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-22T13:11:26.805 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 218 M 2024-04-22T13:11:26.805 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2024-04-22T13:11:26.811 INFO:teuthology.orchestra.run.smithi134.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 4.0 MB/s | 1.6 MB 00:00 2024-04-22T13:11:27.110 INFO:teuthology.orchestra.run.smithi012.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 1.9 MB/s | 1.6 MB 00:00 2024-04-22T13:11:27.145 INFO:teuthology.orchestra.run.smithi134.stdout:(68/150): postgresql-contrib-13.14-1.el9.x86_64 2.7 MB/s | 911 kB 00:00 2024-04-22T13:11:27.204 INFO:teuthology.orchestra.run.smithi134.stdout:(69/150): openblas-openmp-0.3.21-2.el9.x86_64.r 5.5 MB/s | 4.7 MB 00:00 2024-04-22T13:11:27.229 INFO:teuthology.orchestra.run.smithi134.stdout:(70/150): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-04-22T13:11:27.269 INFO:teuthology.orchestra.run.smithi012.stdout:(149/150): librados2-19.0.0-3156.gd69f13d8.el7. 4.5 MB/s | 3.4 MB 00:00 2024-04-22T13:11:27.319 INFO:teuthology.orchestra.run.smithi012.stdout:(150/150): librbd1-19.0.0-3156.gd69f13d8.el7.x8 4.1 MB/s | 3.1 MB 00:00 2024-04-22T13:11:27.322 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-04-22T13:11:27.322 INFO:teuthology.orchestra.run.smithi012.stdout:Total 12 MB/s | 218 MB 00:18 2024-04-22T13:11:27.412 INFO:teuthology.orchestra.run.smithi134.stdout:(71/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 4.0 MB/s | 4.1 MB 00:01 2024-04-22T13:11:27.446 INFO:teuthology.orchestra.run.smithi134.stdout:(72/150): protobuf-3.14.0-13.el9.x86_64.rpm 4.7 MB/s | 1.0 MB 00:00 2024-04-22T13:11:27.480 INFO:teuthology.orchestra.run.smithi134.stdout:(73/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-22T13:11:27.546 INFO:teuthology.orchestra.run.smithi134.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-04-22T13:11:27.622 INFO:teuthology.orchestra.run.smithi134.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-04-22T13:11:27.697 INFO:teuthology.orchestra.run.smithi134.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 342 kB/s | 26 kB 00:00 2024-04-22T13:11:27.773 INFO:teuthology.orchestra.run.smithi134.stdout:(77/150): python3-jsonpointer-2.0-4.el9.noarch. 256 kB/s | 19 kB 00:00 2024-04-22T13:11:27.848 INFO:teuthology.orchestra.run.smithi134.stdout:(78/150): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-04-22T13:11:27.965 INFO:teuthology.orchestra.run.smithi134.stdout:(79/150): postgresql-server-13.14-1.el9.x86_64. 7.6 MB/s | 5.8 MB 00:00 2024-04-22T13:11:27.990 INFO:teuthology.orchestra.run.smithi134.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 245 kB/s | 35 kB 00:00 2024-04-22T13:11:28.073 INFO:teuthology.orchestra.run.smithi134.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-04-22T13:11:28.165 INFO:teuthology.orchestra.run.smithi134.stdout:(82/150): python3-babel-2.9.1-2.el9.noarch.rpm 8.3 MB/s | 6.0 MB 00:00 2024-04-22T13:11:28.191 INFO:teuthology.orchestra.run.smithi134.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 1.9 MB/s | 222 kB 00:00 2024-04-22T13:11:28.241 INFO:teuthology.orchestra.run.smithi134.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-04-22T13:11:28.266 INFO:teuthology.orchestra.run.smithi134.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 562 kB/s | 42 kB 00:00 2024-04-22T13:11:28.316 INFO:teuthology.orchestra.run.smithi134.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-04-22T13:11:28.350 INFO:teuthology.orchestra.run.smithi134.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-04-22T13:11:28.400 INFO:teuthology.orchestra.run.smithi134.stdout:(88/150): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-04-22T13:11:28.404 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction check 2024-04-22T13:11:28.425 INFO:teuthology.orchestra.run.smithi134.stdout:(89/150): python3-pytz-2021.1-5.el9.noarch.rpm 677 kB/s | 51 kB 00:00 2024-04-22T13:11:28.476 INFO:teuthology.orchestra.run.smithi134.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 713 kB/s | 54 kB 00:00 2024-04-22T13:11:28.543 INFO:teuthology.orchestra.run.smithi134.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 625 kB/s | 42 kB 00:00 2024-04-22T13:11:28.543 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction check succeeded. 2024-04-22T13:11:28.543 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction test 2024-04-22T13:11:28.627 INFO:teuthology.orchestra.run.smithi134.stdout:(92/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 530 kB/s | 44 kB 00:00 2024-04-22T13:11:28.710 INFO:teuthology.orchestra.run.smithi134.stdout:(93/150): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-04-22T13:11:28.777 INFO:teuthology.orchestra.run.smithi134.stdout:(94/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.3 MB/s | 5.1 MB 00:00 2024-04-22T13:11:28.802 INFO:teuthology.orchestra.run.smithi134.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 391 kB/s | 36 kB 00:00 2024-04-22T13:11:28.852 INFO:teuthology.orchestra.run.smithi134.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 631 kB/s | 47 kB 00:00 2024-04-22T13:11:28.886 INFO:teuthology.orchestra.run.smithi134.stdout:(97/150): socat-1.7.4.1-5.el9.x86_64.rpm 3.6 MB/s | 305 kB 00:00 2024-04-22T13:11:28.953 INFO:teuthology.orchestra.run.smithi134.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 576 kB/s | 58 kB 00:00 2024-04-22T13:11:29.045 INFO:teuthology.orchestra.run.smithi134.stdout:(99/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 694 kB/s | 64 kB 00:00 2024-04-22T13:11:29.412 INFO:teuthology.orchestra.run.smithi134.stdout:(100/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 122 kB/s | 45 kB 00:00 2024-04-22T13:11:29.520 INFO:teuthology.orchestra.run.smithi134.stdout:(101/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 148 kB/s | 94 kB 00:00 2024-04-22T13:11:29.679 INFO:teuthology.orchestra.run.smithi134.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 83 kB/s | 22 kB 00:00 2024-04-22T13:11:29.988 INFO:teuthology.orchestra.run.smithi134.stdout:(103/150): protobuf-compiler-3.14.0-13.el9.x86_ 1.8 MB/s | 863 kB 00:00 2024-04-22T13:11:30.088 INFO:teuthology.orchestra.run.smithi134.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 1.3 MB/s | 548 kB 00:00 2024-04-22T13:11:30.188 INFO:teuthology.orchestra.run.smithi134.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 194 kB/s | 19 kB 00:00 2024-04-22T13:11:30.314 INFO:teuthology.orchestra.run.smithi134.stdout:(106/150): gperftools-libs-2.9.1-2.el9.x86_64.r 950 kB/s | 309 kB 00:00 2024-04-22T13:11:30.377 INFO:teuthology.orchestra.run.smithi012.stdout:Transaction test succeeded. 2024-04-22T13:11:30.377 INFO:teuthology.orchestra.run.smithi012.stdout:Running transaction 2024-04-22T13:11:30.497 INFO:teuthology.orchestra.run.smithi134.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 142 kB/s | 26 kB 00:00 2024-04-22T13:11:30.606 INFO:teuthology.orchestra.run.smithi134.stdout:(108/150): liboath-2.6.7-2.el9.x86_64.rpm 448 kB/s | 49 kB 00:00 2024-04-22T13:11:30.698 INFO:teuthology.orchestra.run.smithi134.stdout:(109/150): libunwind-1.6.2-1.el9.x86_64.rpm 737 kB/s | 67 kB 00:00 2024-04-22T13:11:30.940 INFO:teuthology.orchestra.run.smithi134.stdout:(110/150): luarocks-3.9.2-1.el9.noarch.rpm 626 kB/s | 151 kB 00:00 2024-04-22T13:11:31.215 INFO:teuthology.orchestra.run.smithi134.stdout:(111/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.0 MB/s | 838 kB 00:00 2024-04-22T13:11:31.390 INFO:teuthology.orchestra.run.smithi134.stdout:(112/150): python3-asyncssh-2.13.2-5.el9.noarch 3.1 MB/s | 548 kB 00:00 2024-04-22T13:11:31.515 INFO:teuthology.orchestra.run.smithi134.stdout:(113/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 348 kB/s | 43 kB 00:00 2024-04-22T13:11:31.624 INFO:teuthology.orchestra.run.smithi134.stdout:(114/150): python3-cachetools-4.2.4-1.el9.noarc 297 kB/s | 32 kB 00:00 2024-04-22T13:11:31.741 INFO:teuthology.orchestra.run.smithi134.stdout:(115/150): python3-certifi-2021.10.8-2.el9.noar 128 kB/s | 15 kB 00:00 2024-04-22T13:11:31.849 INFO:teuthology.orchestra.run.smithi134.stdout:(116/150): libarrow-9.0.0-10.el9.x86_64.rpm 2.7 MB/s | 4.4 MB 00:01 2024-04-22T13:11:31.875 INFO:teuthology.orchestra.run.smithi134.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-04-22T13:11:32.193 INFO:teuthology.orchestra.run.smithi134.stdout:(118/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 954 kB/s | 303 kB 00:00 2024-04-22T13:11:32.242 INFO:teuthology.orchestra.run.smithi134.stdout:(119/150): python3-cherrypy-18.6.1-2.el9.noarch 914 kB/s | 358 kB 00:00 2024-04-22T13:11:32.350 INFO:teuthology.orchestra.run.smithi134.stdout:(120/150): python3-colorama-0.4.4-12.el9.noarch 224 kB/s | 35 kB 00:00 2024-04-22T13:11:32.376 INFO:teuthology.orchestra.run.smithi134.stdout:(121/150): python3-google-auth-2.28.2-1.el9.noa 1.5 MB/s | 203 kB 00:00 2024-04-22T13:11:32.534 INFO:teuthology.orchestra.run.smithi134.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 913 kB/s | 144 kB 00:00 2024-04-22T13:11:32.693 INFO:teuthology.orchestra.run.smithi134.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 67 kB/s | 11 kB 00:00 2024-04-22T13:11:32.827 INFO:teuthology.orchestra.run.smithi134.stdout:(124/150): python3-jaraco-classes-3.2.1-5.el9.n 133 kB/s | 18 kB 00:00 2024-04-22T13:11:32.960 INFO:teuthology.orchestra.run.smithi134.stdout:(125/150): python3-grpcio-1.46.7-10.el9.x86_64. 3.3 MB/s | 2.0 MB 00:00 2024-04-22T13:11:32.986 INFO:teuthology.orchestra.run.smithi134.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 146 kB/s | 23 kB 00:00 2024-04-22T13:11:33.094 INFO:teuthology.orchestra.run.smithi134.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 145 kB/s | 19 kB 00:00 2024-04-22T13:11:33.128 INFO:teuthology.orchestra.run.smithi134.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 139 kB/s | 20 kB 00:00 2024-04-22T13:11:33.353 INFO:teuthology.orchestra.run.smithi134.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 206 kB/s | 46 kB 00:00 2024-04-22T13:11:33.437 INFO:teuthology.orchestra.run.smithi134.stdout:(130/150): python3-kubernetes-26.1.0-2.el9.noar 3.0 MB/s | 1.0 MB 00:00 2024-04-22T13:11:33.462 INFO:teuthology.orchestra.run.smithi134.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 727 kB/s | 79 kB 00:00 2024-04-22T13:11:33.546 INFO:teuthology.orchestra.run.smithi134.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 532 kB/s | 58 kB 00:00 2024-04-22T13:11:33.638 INFO:teuthology.orchestra.run.smithi134.stdout:(133/150): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-04-22T13:11:33.680 INFO:teuthology.orchestra.run.smithi134.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 123 kB/s | 16 kB 00:00 2024-04-22T13:11:33.780 INFO:teuthology.orchestra.run.smithi134.stdout:(135/150): python3-pyperclip-1.8.2-2.el9.noarch 283 kB/s | 28 kB 00:00 2024-04-22T13:11:33.805 INFO:teuthology.orchestra.run.smithi134.stdout:(136/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 538 kB/s | 90 kB 00:00 2024-04-22T13:11:33.880 INFO:teuthology.orchestra.run.smithi134.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 307 kB/s | 31 kB 00:00 2024-04-22T13:11:33.906 INFO:teuthology.orchestra.run.smithi134.stdout:(138/150): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-22T13:11:33.973 INFO:teuthology.orchestra.run.smithi134.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 643 kB/s | 59 kB 00:00 2024-04-22T13:11:33.998 INFO:teuthology.orchestra.run.smithi134.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 389 kB/s | 36 kB 00:00 2024-04-22T13:11:34.081 INFO:teuthology.orchestra.run.smithi134.stdout:(141/150): python3-typing-extensions-4.4.0-2.el 470 kB/s | 51 kB 00:00 2024-04-22T13:11:34.107 INFO:teuthology.orchestra.run.smithi134.stdout:(142/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-04-22T13:11:34.157 INFO:teuthology.orchestra.run.smithi134.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-22T13:11:34.249 INFO:teuthology.orchestra.run.smithi134.stdout:(144/150): python3-zc-lockfile-2.0-10.el9.noarc 218 kB/s | 20 kB 00:00 2024-04-22T13:11:34.483 INFO:teuthology.orchestra.run.smithi134.stdout:(145/150): python3-werkzeug-2.0.3-3.el9.1.noarc 1.1 MB/s | 427 kB 00:00 2024-04-22T13:11:34.508 INFO:teuthology.orchestra.run.smithi134.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 741 kB/s | 192 kB 00:00 2024-04-22T13:11:34.883 INFO:teuthology.orchestra.run.smithi134.stdout:(147/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.5 MB/s | 16 MB 00:06 2024-04-22T13:11:34.958 INFO:teuthology.orchestra.run.smithi134.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 3.3 MB/s | 1.6 MB 00:00 2024-04-22T13:11:35.233 INFO:teuthology.orchestra.run.smithi134.stdout:(149/150): librbd1-19.0.0-3156.gd69f13d8.el7.x8 9.0 MB/s | 3.1 MB 00:00 2024-04-22T13:11:35.284 INFO:teuthology.orchestra.run.smithi134.stdout:(150/150): librados2-19.0.0-3156.gd69f13d8.el7. 4.4 MB/s | 3.4 MB 00:00 2024-04-22T13:11:35.288 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-04-22T13:11:35.289 INFO:teuthology.orchestra.run.smithi134.stdout:Total 12 MB/s | 218 MB 00:18 2024-04-22T13:11:36.259 INFO:teuthology.orchestra.run.smithi012.stdout: Preparing : 1/1 2024-04-22T13:11:36.329 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-22T13:11:36.350 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-04-22T13:11:36.362 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-22T13:11:36.406 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-22T13:11:36.475 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-04-22T13:11:36.475 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-04-22T13:11:36.751 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-22T13:11:36.855 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 5/152 2024-04-22T13:11:36.963 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 5/152 2024-04-22T13:11:36.989 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/152 2024-04-22T13:11:37.065 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/152 2024-04-22T13:11:37.107 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/152 2024-04-22T13:11:37.135 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-22T13:11:37.161 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-22T13:11:37.328 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-22T13:11:37.384 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-22T13:11:37.411 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/152 2024-04-22T13:11:37.517 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/152 2024-04-22T13:11:37.554 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 13/152 2024-04-22T13:11:37.617 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 13/152 2024-04-22T13:11:37.638 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-22T13:11:37.665 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-22T13:11:37.814 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-22T13:11:37.847 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-22T13:11:37.958 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-22T13:11:38.056 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-22T13:11:38.087 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-22T13:11:38.123 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-22T13:11:38.173 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-04-22T13:11:38.173 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-04-22T13:11:38.200 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-22T13:11:38.239 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-22T13:11:38.299 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-22T13:11:38.332 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-22T13:11:38.359 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-22T13:11:38.433 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-22T13:11:38.484 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-22T13:11:38.511 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-22T13:11:38.556 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-22T13:11:38.636 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-22T13:11:38.666 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 30/152 2024-04-22T13:11:38.710 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 31/152 2024-04-22T13:11:38.738 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 32/152 2024-04-22T13:11:38.816 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-22T13:11:38.862 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-22T13:11:38.919 INFO:teuthology.orchestra.run.smithi111.stdout:(1/150): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 20 kB/s | 6.6 kB 00:00 2024-04-22T13:11:38.942 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-22T13:11:39.099 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-22T13:11:39.158 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-22T13:11:39.253 INFO:teuthology.orchestra.run.smithi111.stdout:(2/150): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 8.2 MB/s | 5.4 MB 00:00 2024-04-22T13:11:39.320 INFO:teuthology.orchestra.run.smithi111.stdout:(3/150): ceph-immutable-object-cache-19.0.0-315 2.1 MB/s | 145 kB 00:00 2024-04-22T13:11:39.579 INFO:teuthology.orchestra.run.smithi111.stdout:(4/150): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 9.1 MB/s | 2.4 MB 00:00 2024-04-22T13:11:39.737 INFO:teuthology.orchestra.run.smithi111.stdout:(5/150): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 9.5 MB/s | 1.5 MB 00:00 2024-04-22T13:11:39.738 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-22T13:11:39.832 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-22T13:11:39.864 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-22T13:11:39.908 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-22T13:11:39.946 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 42/152 2024-04-22T13:11:39.979 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-22T13:11:40.037 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-22T13:11:40.111 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-22T13:11:40.155 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-22T13:11:40.192 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-22T13:11:40.240 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-22T13:11:40.274 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-22T13:11:40.313 INFO:teuthology.orchestra.run.smithi111.stdout:(6/150): ceph-common-19.0.0-3156.gd69f13d8.el7. 12 MB/s | 21 MB 00:01 2024-04-22T13:11:40.318 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-22T13:11:40.361 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-22T13:11:40.380 INFO:teuthology.orchestra.run.smithi111.stdout:(7/150): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 7.3 MB/s | 4.7 MB 00:00 2024-04-22T13:11:40.507 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-22T13:11:40.522 INFO:teuthology.orchestra.run.smithi111.stdout:(8/150): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 556 kB/s | 890 kB 00:01 2024-04-22T13:11:40.548 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-22T13:11:40.578 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-22T13:11:40.621 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-22T13:11:40.639 INFO:teuthology.orchestra.run.smithi111.stdout:(9/150): ceph-selinux-19.0.0-3156.gd69f13d8.el7 215 kB/s | 25 kB 00:00 2024-04-22T13:11:40.721 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-22T13:11:40.753 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-22T13:11:40.794 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-22T13:11:40.827 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-22T13:11:40.851 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-22T13:11:40.931 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-22T13:11:41.061 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-22T13:11:41.092 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-22T13:11:41.129 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-22T13:11:41.163 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-22T13:11:41.197 INFO:teuthology.orchestra.run.smithi111.stdout:(10/150): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 19 MB/s | 17 MB 00:00 2024-04-22T13:11:41.208 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-22T13:11:41.247 INFO:teuthology.orchestra.run.smithi111.stdout:(11/150): cephfs-mirror-19.0.0-3156.gd69f13d8.e 4.4 MB/s | 225 kB 00:00 2024-04-22T13:11:41.254 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-22T13:11:41.293 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-22T13:11:41.298 INFO:teuthology.orchestra.run.smithi111.stdout:(12/150): cephfs-shell-19.0.0-3156.gd69f13d8.el 539 kB/s | 27 kB 00:00 2024-04-22T13:11:41.322 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-22T13:11:41.340 INFO:teuthology.orchestra.run.smithi111.stdout:(13/150): libcephfs-devel-19.0.0-3156.gd69f13d8 764 kB/s | 32 kB 00:00 2024-04-22T13:11:41.357 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-22T13:11:41.390 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-22T13:11:41.415 INFO:teuthology.orchestra.run.smithi111.stdout:(14/150): libcephfs2-19.0.0-3156.gd69f13d8.el7. 9.8 MB/s | 747 kB 00:00 2024-04-22T13:11:41.433 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-22T13:11:41.466 INFO:teuthology.orchestra.run.smithi111.stdout:(15/150): libcephsqlite-19.0.0-3156.gd69f13d8.e 3.2 MB/s | 163 kB 00:00 2024-04-22T13:11:41.466 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-22T13:11:41.516 INFO:teuthology.orchestra.run.smithi111.stdout:(16/150): librados-devel-19.0.0-3156.gd69f13d8. 2.5 MB/s | 126 kB 00:00 2024-04-22T13:11:41.558 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-22T13:11:41.582 INFO:teuthology.orchestra.run.smithi111.stdout:(17/150): libradosstriper1-19.0.0-3156.gd69f13d 7.4 MB/s | 500 kB 00:00 2024-04-22T13:11:41.631 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-22T13:11:41.662 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-22T13:11:41.710 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-22T13:11:41.822 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-22T13:11:41.873 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-22T13:11:41.908 INFO:teuthology.orchestra.run.smithi111.stdout:(18/150): librgw2-19.0.0-3156.gd69f13d8.el7.x86 16 MB/s | 5.4 MB 00:00 2024-04-22T13:11:41.958 INFO:teuthology.orchestra.run.smithi111.stdout:(19/150): python3-ceph-argparse-19.0.0-3156.gd6 907 kB/s | 45 kB 00:00 2024-04-22T13:11:42.075 INFO:teuthology.orchestra.run.smithi111.stdout:(20/150): ceph-radosgw-19.0.0-3156.gd69f13d8.el 6.2 MB/s | 11 MB 00:01 2024-04-22T13:11:42.101 INFO:teuthology.orchestra.run.smithi111.stdout:(21/150): python3-ceph-common-19.0.0-3156.gd69f 945 kB/s | 135 kB 00:00 2024-04-22T13:11:42.126 INFO:teuthology.orchestra.run.smithi111.stdout:(22/150): python3-cephfs-19.0.0-3156.gd69f13d8. 3.1 MB/s | 161 kB 00:00 2024-04-22T13:11:42.151 INFO:teuthology.orchestra.run.smithi111.stdout:(23/150): python3-rados-19.0.0-3156.gd69f13d8.e 6.3 MB/s | 320 kB 00:00 2024-04-22T13:11:42.185 INFO:teuthology.orchestra.run.smithi111.stdout:(24/150): python3-rbd-19.0.0-3156.gd69f13d8.el7 5.0 MB/s | 298 kB 00:00 2024-04-22T13:11:42.210 INFO:teuthology.orchestra.run.smithi111.stdout:(25/150): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.7 MB/s | 99 kB 00:00 2024-04-22T13:11:42.235 INFO:teuthology.orchestra.run.smithi111.stdout:(26/150): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.7 MB/s | 85 kB 00:00 2024-04-22T13:11:42.302 INFO:teuthology.orchestra.run.smithi111.stdout:(27/150): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 2.5 MB/s | 170 kB 00:00 2024-04-22T13:11:42.394 INFO:teuthology.orchestra.run.smithi111.stdout:(28/150): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 17 MB/s | 3.1 MB 00:00 2024-04-22T13:11:42.419 INFO:teuthology.orchestra.run.smithi111.stdout:(29/150): ceph-grafana-dashboards-19.0.0-3156.g 292 kB/s | 34 kB 00:00 2024-04-22T13:11:42.445 INFO:teuthology.orchestra.run.smithi111.stdout:(30/150): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 2.8 MB/s | 142 kB 00:00 2024-04-22T13:11:42.669 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-22T13:11:42.719 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-22T13:11:42.752 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-22T13:11:42.868 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-22T13:11:42.895 INFO:teuthology.orchestra.run.smithi111.stdout:(31/150): ceph-mgr-diskprediction-local-19.0.0- 16 MB/s | 7.4 MB 00:00 2024-04-22T13:11:42.920 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-22T13:11:42.979 INFO:teuthology.orchestra.run.smithi111.stdout:(32/150): ceph-mgr-dashboard-19.0.0-3156.gd69f1 6.3 MB/s | 3.5 MB 00:00 2024-04-22T13:11:43.004 INFO:teuthology.orchestra.run.smithi111.stdout:(33/150): ceph-mgr-modules-core-19.0.0-3156.gd6 2.2 MB/s | 249 kB 00:00 2024-04-22T13:11:43.012 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-22T13:11:43.030 INFO:teuthology.orchestra.run.smithi111.stdout:(34/150): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 980 kB/s | 49 kB 00:00 2024-04-22T13:11:43.046 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-22T13:11:43.055 INFO:teuthology.orchestra.run.smithi111.stdout:(35/150): ceph-prometheus-alerts-19.0.0-3156.gd 326 kB/s | 16 kB 00:00 2024-04-22T13:11:43.080 INFO:teuthology.orchestra.run.smithi111.stdout:(36/150): ceph-volume-19.0.0-3156.gd69f13d8.el7 5.3 MB/s | 273 kB 00:00 2024-04-22T13:11:43.122 INFO:teuthology.orchestra.run.smithi111.stdout:(37/150): cephadm-19.0.0-3156.gd69f13d8.el7.noa 11 MB/s | 762 kB 00:00 2024-04-22T13:11:43.147 INFO:teuthology.orchestra.run.smithi111.stdout:(38/150): cephfs-top-19.0.0-3156.gd69f13d8.el7. 272 kB/s | 18 kB 00:00 2024-04-22T13:11:43.230 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-22T13:11:43.252 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-22T13:11:43.332 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-22T13:11:43.664 INFO:teuthology.orchestra.run.smithi111.stdout:(39/150): fuse-2.9.9-15.el9.x86_64.rpm 148 kB/s | 80 kB 00:00 2024-04-22T13:11:43.940 INFO:teuthology.orchestra.run.smithi111.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 711 kB/s | 195 kB 00:00 2024-04-22T13:11:44.006 INFO:teuthology.orchestra.run.smithi111.stdout:(41/150): libgfortran-11.4.1-3.el9.x86_64.rpm 935 kB/s | 803 kB 00:00 2024-04-22T13:11:44.043 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-22T13:11:44.059 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-04-22T13:11:44.128 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-22T13:11:44.132 INFO:teuthology.orchestra.run.smithi111.stdout:(42/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-04-22T13:11:44.172 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-22T13:11:44.212 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-22T13:11:44.240 INFO:teuthology.orchestra.run.smithi111.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 984 kB/s | 106 kB 00:00 2024-04-22T13:11:44.291 INFO:teuthology.orchestra.run.smithi111.stdout:(44/150): python3-cryptography-36.0.1-4.el9.x86 4.3 MB/s | 1.2 MB 00:00 2024-04-22T13:11:44.348 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-22T13:11:44.349 INFO:teuthology.orchestra.run.smithi111.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-04-22T13:11:44.399 INFO:teuthology.orchestra.run.smithi111.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-04-22T13:11:44.508 INFO:teuthology.orchestra.run.smithi111.stdout:(47/150): python3-urllib3-1.26.5-5.el9.noarch.r 1.3 MB/s | 215 kB 00:00 2024-04-22T13:11:44.600 INFO:teuthology.orchestra.run.smithi111.stdout:(48/150): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 689 kB 00:00 2024-04-22T13:11:44.650 INFO:teuthology.orchestra.run.smithi111.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-04-22T13:11:44.681 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-22T13:11:44.692 INFO:teuthology.orchestra.run.smithi111.stdout:(50/150): elfutils-libelf-devel-0.190-2.el9.x86 518 kB/s | 21 kB 00:00 2024-04-22T13:11:44.742 INFO:teuthology.orchestra.run.smithi111.stdout:(51/150): bison-3.7.4-5.el9.x86_64.rpm 3.9 MB/s | 943 kB 00:00 2024-04-22T13:11:44.768 INFO:teuthology.orchestra.run.smithi111.stdout:(52/150): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-04-22T13:11:44.779 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 5/152 2024-04-22T13:11:44.785 INFO:teuthology.orchestra.run.smithi111.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 780 kB/s | 33 kB 00:00 2024-04-22T13:11:44.835 INFO:teuthology.orchestra.run.smithi111.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-04-22T13:11:44.877 INFO:teuthology.orchestra.run.smithi111.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 268 kB/s | 11 kB 00:00 2024-04-22T13:11:44.899 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 5/152 2024-04-22T13:11:44.927 INFO:teuthology.orchestra.run.smithi111.stdout:(56/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-04-22T13:11:44.930 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/152 2024-04-22T13:11:44.954 INFO:teuthology.orchestra.run.smithi111.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-04-22T13:11:44.978 INFO:teuthology.orchestra.run.smithi111.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-04-22T13:11:45.020 INFO:teuthology.orchestra.run.smithi111.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-04-22T13:11:45.020 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/152 2024-04-22T13:11:45.045 INFO:teuthology.orchestra.run.smithi111.stdout:(60/150): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-04-22T13:11:45.063 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/152 2024-04-22T13:11:45.070 INFO:teuthology.orchestra.run.smithi111.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 507 kB/s | 25 kB 00:00 2024-04-22T13:11:45.095 INFO:teuthology.orchestra.run.smithi111.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-04-22T13:11:45.101 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-22T13:11:45.121 INFO:teuthology.orchestra.run.smithi111.stdout:(63/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-22T13:11:45.126 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-22T13:11:45.146 INFO:teuthology.orchestra.run.smithi111.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 3.6 MB/s | 188 kB 00:00 2024-04-22T13:11:45.172 INFO:teuthology.orchestra.run.smithi111.stdout:(65/150): openblas-0.3.21-2.el9.x86_64.rpm 702 kB/s | 35 kB 00:00 2024-04-22T13:11:45.283 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-22T13:11:45.322 INFO:teuthology.orchestra.run.smithi111.stdout:(66/150): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-04-22T13:11:45.332 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-22T13:11:45.360 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/152 2024-04-22T13:11:45.406 INFO:teuthology.orchestra.run.smithi111.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-04-22T13:11:45.463 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/152 2024-04-22T13:11:45.473 INFO:teuthology.orchestra.run.smithi111.stdout:(68/150): postgresql-contrib-13.14-1.el9.x86_64 13 MB/s | 911 kB 00:00 2024-04-22T13:11:45.486 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 13/152 2024-04-22T13:11:45.511 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-22T13:11:45.531 INFO:teuthology.orchestra.run.smithi111.stdout:(69/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 12 MB/s | 4.1 MB 00:00 2024-04-22T13:11:45.549 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 13/152 2024-04-22T13:11:45.567 INFO:teuthology.orchestra.run.smithi111.stdout:(70/150): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-04-22T13:11:45.570 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-22T13:11:45.591 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-22T13:11:45.606 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-22T13:11:45.634 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-22T13:11:45.648 INFO:teuthology.orchestra.run.smithi111.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-04-22T13:11:45.667 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-22T13:11:45.690 INFO:teuthology.orchestra.run.smithi111.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 2.1 MB/s | 87 kB 00:00 2024-04-22T13:11:45.770 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-22T13:11:45.803 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-22T13:11:45.910 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-22T13:11:45.924 INFO:teuthology.orchestra.run.smithi111.stdout:(73/150): python3-babel-2.9.1-2.el9.noarch.rpm 26 MB/s | 6.0 MB 00:00 2024-04-22T13:11:45.991 INFO:teuthology.orchestra.run.smithi111.stdout:(74/150): postgresql-server-13.14-1.el9.x86_64. 13 MB/s | 5.8 MB 00:00 2024-04-22T13:11:45.995 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-22T13:11:45.998 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-22T13:11:46.016 INFO:teuthology.orchestra.run.smithi111.stdout:(75/150): python3-devel-3.9.18-3.el9.x86_64.rpm 2.6 MB/s | 244 kB 00:00 2024-04-22T13:11:46.041 INFO:teuthology.orchestra.run.smithi111.stdout:(76/150): python3-jinja2-2.11.3-5.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-04-22T13:11:46.044 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-22T13:11:46.046 INFO:teuthology.orchestra.run.smithi012.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 97/152 2024-04-22T13:11:46.067 INFO:teuthology.orchestra.run.smithi111.stdout:(77/150): python3-jsonpatch-1.21-16.el9.noarch. 509 kB/s | 26 kB 00:00 2024-04-22T13:11:46.079 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-22T13:11:46.101 INFO:teuthology.orchestra.run.smithi111.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 326 kB/s | 19 kB 00:00 2024-04-22T13:11:46.138 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 97/152 2024-04-22T13:11:46.145 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-22T13:11:46.168 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 98/152 2024-04-22T13:11:46.186 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-22T13:11:46.204 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-22T13:11:46.209 INFO:teuthology.orchestra.run.smithi111.stdout:(79/150): python3-mako-1.1.4-6.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-04-22T13:11:46.230 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-22T13:11:46.287 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-22T13:11:46.315 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-22T13:11:46.342 INFO:teuthology.orchestra.run.smithi111.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 144 kB/s | 35 kB 00:00 2024-04-22T13:11:46.385 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-22T13:11:46.393 INFO:teuthology.orchestra.run.smithi111.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.8 MB/s | 250 kB 00:00 2024-04-22T13:11:46.446 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-22T13:11:46.468 INFO:teuthology.orchestra.run.smithi111.stdout:(82/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 5.1 MB 00:00 2024-04-22T13:11:46.477 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-22T13:11:46.494 INFO:teuthology.orchestra.run.smithi111.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-04-22T13:11:46.519 INFO:teuthology.orchestra.run.smithi111.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-22T13:11:46.520 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-22T13:11:46.536 INFO:teuthology.orchestra.run.smithi111.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-04-22T13:11:46.570 INFO:teuthology.orchestra.run.smithi111.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-22T13:11:46.595 INFO:teuthology.orchestra.run.smithi111.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-04-22T13:11:46.597 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-22T13:11:46.620 INFO:teuthology.orchestra.run.smithi111.stdout:(88/150): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-22T13:11:46.631 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 30/152 2024-04-22T13:11:46.637 INFO:teuthology.orchestra.run.smithi111.stdout:(89/150): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-04-22T13:11:46.662 INFO:teuthology.orchestra.run.smithi111.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-04-22T13:11:46.674 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 31/152 2024-04-22T13:11:46.703 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 32/152 2024-04-22T13:11:46.709 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-22T13:11:46.713 INFO:teuthology.orchestra.run.smithi111.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 830 kB/s | 42 kB 00:00 2024-04-22T13:11:46.741 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 101/152 2024-04-22T13:11:46.763 INFO:teuthology.orchestra.run.smithi111.stdout:(92/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 888 kB/s | 44 kB 00:00 2024-04-22T13:11:46.775 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-22T13:11:46.804 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 101/152 2024-04-22T13:11:46.822 INFO:teuthology.orchestra.run.smithi111.stdout:(93/150): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-22T13:11:46.826 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 102/152 2024-04-22T13:11:46.826 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-22T13:11:46.872 INFO:teuthology.orchestra.run.smithi111.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 720 kB/s | 36 kB 00:00 2024-04-22T13:11:46.901 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-22T13:11:46.922 INFO:teuthology.orchestra.run.smithi111.stdout:(95/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 950 kB/s | 47 kB 00:00 2024-04-22T13:11:46.981 INFO:teuthology.orchestra.run.smithi111.stdout:(96/150): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-04-22T13:11:47.039 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-22T13:11:47.039 INFO:teuthology.orchestra.run.smithi111.stdout:(97/150): uuid-1.6.2-55.el9.x86_64.rpm 993 kB/s | 58 kB 00:00 2024-04-22T13:11:47.090 INFO:teuthology.orchestra.run.smithi111.stdout:(98/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-22T13:11:47.104 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-22T13:11:47.140 INFO:teuthology.orchestra.run.smithi111.stdout:(99/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-22T13:11:47.199 INFO:teuthology.orchestra.run.smithi111.stdout:(100/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 769 kB/s | 45 kB 00:00 2024-04-22T13:11:47.249 INFO:teuthology.orchestra.run.smithi111.stdout:(101/150): lua-devel-5.4.4-4.el9.x86_64.rpm 446 kB/s | 22 kB 00:00 2024-04-22T13:11:47.316 INFO:teuthology.orchestra.run.smithi111.stdout:(102/150): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-04-22T13:11:47.566 INFO:teuthology.orchestra.run.smithi111.stdout:(103/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-04-22T13:11:47.644 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-22T13:11:47.733 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-22T13:11:47.761 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-22T13:11:47.804 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-22T13:11:47.842 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 42/152 2024-04-22T13:11:47.875 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-22T13:11:47.931 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-22T13:11:48.013 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-22T13:11:48.043 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-22T13:11:48.066 INFO:teuthology.orchestra.run.smithi111.stdout:(104/150): gperftools-libs-2.9.1-2.el9.x86_64.r 618 kB/s | 309 kB 00:00 2024-04-22T13:11:48.079 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-22T13:11:48.126 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-22T13:11:48.162 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-22T13:11:48.175 INFO:teuthology.orchestra.run.smithi111.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 180 kB/s | 19 kB 00:00 2024-04-22T13:11:48.205 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-22T13:11:48.248 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-22T13:11:48.292 INFO:teuthology.orchestra.run.smithi111.stdout:(106/150): abseil-cpp-20211102.0-3.el9.x86_64.r 562 kB/s | 548 kB 00:00 2024-04-22T13:11:48.386 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-22T13:11:48.401 INFO:teuthology.orchestra.run.smithi111.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 240 kB/s | 26 kB 00:00 2024-04-22T13:11:48.436 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-22T13:11:48.466 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-22T13:11:48.507 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-22T13:11:48.526 INFO:teuthology.orchestra.run.smithi111.stdout:(108/150): liboath-2.6.7-2.el9.x86_64.rpm 389 kB/s | 49 kB 00:00 2024-04-22T13:11:48.612 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-22T13:11:48.649 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-22T13:11:48.660 INFO:teuthology.orchestra.run.smithi111.stdout:(109/150): libunwind-1.6.2-1.el9.x86_64.rpm 505 kB/s | 67 kB 00:00 2024-04-22T13:11:48.689 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-22T13:11:48.723 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-22T13:11:48.748 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-22T13:11:48.823 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-22T13:11:48.843 INFO:teuthology.orchestra.run.smithi111.stdout:(110/150): libarrow-9.0.0-10.el9.x86_64.rpm 6.6 MB/s | 4.4 MB 00:00 2024-04-22T13:11:48.868 INFO:teuthology.orchestra.run.smithi111.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 724 kB/s | 151 kB 00:00 2024-04-22T13:11:48.951 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-22T13:11:48.997 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-22T13:11:49.027 INFO:teuthology.orchestra.run.smithi111.stdout:(112/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.5 MB/s | 838 kB 00:00 2024-04-22T13:11:49.050 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-22T13:11:49.084 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-22T13:11:49.129 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-22T13:11:49.136 INFO:teuthology.orchestra.run.smithi111.stdout:(113/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 401 kB/s | 43 kB 00:00 2024-04-22T13:11:49.166 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-22T13:11:49.167 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/152 2024-04-22T13:11:49.197 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-22T13:11:49.198 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/152 2024-04-22T13:11:49.227 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-22T13:11:49.236 INFO:teuthology.orchestra.run.smithi111.stdout:(114/150): python3-cachetools-4.2.4-1.el9.noarc 323 kB/s | 32 kB 00:00 2024-04-22T13:11:49.261 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-22T13:11:49.295 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-22T13:11:49.337 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-22T13:11:49.353 INFO:teuthology.orchestra.run.smithi111.stdout:(115/150): python3-certifi-2021.10.8-2.el9.noar 128 kB/s | 15 kB 00:00 2024-04-22T13:11:49.371 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-22T13:11:49.450 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-22T13:11:49.461 INFO:teuthology.orchestra.run.smithi111.stdout:(116/150): python3-asyncssh-2.13.2-5.el9.noarch 925 kB/s | 548 kB 00:00 2024-04-22T13:11:49.486 INFO:teuthology.orchestra.run.smithi111.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-04-22T13:11:49.493 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-22T13:11:49.609 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-22T13:11:49.620 INFO:teuthology.orchestra.run.smithi111.stdout:(118/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-04-22T13:11:49.656 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-22T13:11:49.737 INFO:teuthology.orchestra.run.smithi111.stdout:(119/150): python3-colorama-0.4.4-12.el9.noarch 302 kB/s | 35 kB 00:00 2024-04-22T13:11:49.764 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-22T13:11:49.764 INFO:teuthology.orchestra.run.smithi111.stdout:(120/150): python3-cherrypy-18.6.1-2.el9.noarch 1.2 MB/s | 358 kB 00:00 2024-04-22T13:11:49.768 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/152 2024-04-22T13:11:49.800 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/152 2024-04-22T13:11:49.835 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-22T13:11:49.884 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/152 2024-04-22T13:11:49.884 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-22T13:11:49.884 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-22T13:11:49.884 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:11:49.918 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/152 2024-04-22T13:11:49.956 INFO:teuthology.orchestra.run.smithi111.stdout:(121/150): python3-google-auth-2.28.2-1.el9.noa 927 kB/s | 203 kB 00:00 2024-04-22T13:11:50.132 INFO:teuthology.orchestra.run.smithi111.stdout:(122/150): python3-grpcio-1.46.7-10.el9.x86_64. 5.6 MB/s | 2.0 MB 00:00 2024-04-22T13:11:50.157 INFO:teuthology.orchestra.run.smithi111.stdout:(123/150): python3-grpcio-tools-1.46.7-10.el9.x 721 kB/s | 144 kB 00:00 2024-04-22T13:11:50.232 INFO:teuthology.orchestra.run.smithi111.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-04-22T13:11:50.257 INFO:teuthology.orchestra.run.smithi111.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-04-22T13:11:50.333 INFO:teuthology.orchestra.run.smithi111.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-04-22T13:11:50.358 INFO:teuthology.orchestra.run.smithi111.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-04-22T13:11:50.433 INFO:teuthology.orchestra.run.smithi111.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-04-22T13:11:50.559 INFO:teuthology.orchestra.run.smithi111.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 369 kB/s | 46 kB 00:00 2024-04-22T13:11:50.585 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-22T13:11:50.631 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-22T13:11:50.667 INFO:teuthology.orchestra.run.smithi111.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 734 kB/s | 79 kB 00:00 2024-04-22T13:11:50.676 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-22T13:11:50.776 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-22T13:11:50.792 INFO:teuthology.orchestra.run.smithi111.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 462 kB/s | 58 kB 00:00 2024-04-22T13:11:50.832 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-22T13:11:50.918 INFO:teuthology.orchestra.run.smithi111.stdout:(132/150): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-04-22T13:11:50.920 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-22T13:11:50.959 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-22T13:11:50.984 INFO:teuthology.orchestra.run.smithi111.stdout:(133/150): python3-kubernetes-26.1.0-2.el9.noar 1.6 MB/s | 1.0 MB 00:00 2024-04-22T13:11:51.018 INFO:teuthology.orchestra.run.smithi111.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-04-22T13:11:51.094 INFO:teuthology.orchestra.run.smithi111.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 828 kB/s | 90 kB 00:00 2024-04-22T13:11:51.127 INFO:teuthology.orchestra.run.smithi111.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 260 kB/s | 28 kB 00:00 2024-04-22T13:11:51.135 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-22T13:11:51.165 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-22T13:11:51.210 INFO:teuthology.orchestra.run.smithi111.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 263 kB/s | 31 kB 00:00 2024-04-22T13:11:51.242 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-22T13:11:51.244 INFO:teuthology.orchestra.run.smithi111.stdout:(138/150): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-04-22T13:11:51.328 INFO:teuthology.orchestra.run.smithi111.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 505 kB/s | 59 kB 00:00 2024-04-22T13:11:51.353 INFO:teuthology.orchestra.run.smithi111.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 329 kB/s | 36 kB 00:00 2024-04-22T13:11:51.428 INFO:teuthology.orchestra.run.smithi111.stdout:(141/150): python3-typing-extensions-4.4.0-2.el 509 kB/s | 51 kB 00:00 2024-04-22T13:11:51.470 INFO:teuthology.orchestra.run.smithi111.stdout:(142/150): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-04-22T13:11:51.530 INFO:teuthology.orchestra.run.smithi111.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 895 kB/s | 90 kB 00:00 2024-04-22T13:11:51.604 INFO:teuthology.orchestra.run.smithi111.stdout:(144/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-04-22T13:11:51.638 INFO:teuthology.orchestra.run.smithi111.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 184 kB/s | 20 kB 00:00 2024-04-22T13:11:51.738 INFO:teuthology.orchestra.run.smithi111.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-04-22T13:11:51.904 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-22T13:11:52.023 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-22T13:11:52.097 INFO:teuthology.orchestra.run.smithi111.stdout:(147/150): librados2-19.0.0-3156.gd69f13d8.el7. 9.5 MB/s | 3.4 MB 00:00 2024-04-22T13:11:52.297 INFO:teuthology.orchestra.run.smithi111.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 2.4 MB/s | 1.6 MB 00:00 2024-04-22T13:11:52.422 INFO:teuthology.orchestra.run.smithi111.stdout:(149/150): librbd1-19.0.0-3156.gd69f13d8.el7.x8 9.7 MB/s | 3.1 MB 00:00 2024-04-22T13:11:53.278 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-22T13:11:53.360 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-22T13:11:53.388 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-22T13:11:53.421 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-22T13:11:53.734 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-22T13:11:53.800 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 97/152 2024-04-22T13:11:53.890 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 97/152 2024-04-22T13:11:53.921 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 98/152 2024-04-22T13:11:53.966 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-22T13:11:54.443 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-22T13:11:54.470 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 101/152 2024-04-22T13:11:54.524 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 101/152 2024-04-22T13:11:54.546 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 102/152 2024-04-22T13:11:56.812 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/152 2024-04-22T13:11:56.918 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/152 2024-04-22T13:11:57.458 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/152 2024-04-22T13:11:57.496 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/152 2024-04-22T13:11:57.578 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/152 2024-04-22T13:11:57.578 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-22T13:11:57.578 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-22T13:11:57.578 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-22T13:11:57.605 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/152 2024-04-22T13:12:01.787 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/152 2024-04-22T13:12:01.787 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /sys 2024-04-22T13:12:01.787 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /proc 2024-04-22T13:12:01.787 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /mnt 2024-04-22T13:12:01.787 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /var/tmp 2024-04-22T13:12:01.787 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /home 2024-04-22T13:12:01.787 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /root 2024-04-22T13:12:01.787 INFO:teuthology.orchestra.run.smithi012.stdout:skipping the directory /tmp 2024-04-22T13:12:01.787 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:12:01.971 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 106/152 2024-04-22T13:12:01.990 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 106/152 2024-04-22T13:12:01.990 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:12:01.990 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-22T13:12:01.990 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:12:02.380 INFO:teuthology.orchestra.run.smithi111.stdout:(150/150): ceph-test-19.0.0-3156.gd69f13d8.el7. 2.3 MB/s | 49 MB 00:21 2024-04-22T13:12:02.383 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-04-22T13:12:02.383 INFO:teuthology.orchestra.run.smithi111.stdout:Total 6.1 MB/s | 218 MB 00:35 2024-04-22T13:12:03.398 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2024-04-22T13:12:03.522 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2024-04-22T13:12:03.523 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2024-04-22T13:12:03.532 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/152 2024-04-22T13:12:03.559 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/152 2024-04-22T13:12:03.560 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:12:03.560 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-22T13:12:03.560 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T13:12:03.560 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T13:12:03.560 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:12:03.791 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 108/152 2024-04-22T13:12:03.815 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 108/152 2024-04-22T13:12:03.815 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:12:03.815 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-22T13:12:03.815 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T13:12:03.815 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T13:12:03.815 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:12:04.284 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/152 2024-04-22T13:12:04.308 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/152 2024-04-22T13:12:04.308 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:12:04.308 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-22T13:12:04.308 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T13:12:04.308 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T13:12:04.308 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:12:04.415 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-22T13:12:04.482 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-22T13:12:04.546 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-22T13:12:04.742 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-22T13:12:04.794 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-22T13:12:04.892 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-22T13:12:04.990 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-22T13:12:05.142 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-22T13:12:05.185 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-22T13:12:05.217 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-22T13:12:05.237 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2024-04-22T13:12:05.237 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2024-04-22T13:12:05.253 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-22T13:12:05.691 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 121/152 2024-04-22T13:12:05.718 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 121/152 2024-04-22T13:12:05.751 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 122/152 2024-04-22T13:12:05.814 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 123/152 2024-04-22T13:12:05.841 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 124/152 2024-04-22T13:12:06.941 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 124/152 2024-04-22T13:12:06.976 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 125/152 2024-04-22T13:12:07.992 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 125/152 2024-04-22T13:12:08.027 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 126/152 2024-04-22T13:12:08.138 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 126/152 2024-04-22T13:12:08.305 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 127/152 2024-04-22T13:12:08.329 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 128/152 2024-04-22T13:12:08.353 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 128/152 2024-04-22T13:12:08.353 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:12:08.353 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-22T13:12:08.354 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T13:12:08.354 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T13:12:08.354 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:12:08.404 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 129/152 2024-04-22T13:12:08.416 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 129/152 2024-04-22T13:12:08.518 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/152 2024-04-22T13:12:08.571 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 131/152 2024-04-22T13:12:08.782 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/152 2024-04-22T13:12:08.782 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /sys 2024-04-22T13:12:08.782 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /proc 2024-04-22T13:12:08.782 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /mnt 2024-04-22T13:12:08.782 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /var/tmp 2024-04-22T13:12:08.783 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /home 2024-04-22T13:12:08.783 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /root 2024-04-22T13:12:08.783 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /tmp 2024-04-22T13:12:08.783 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-22T13:12:08.966 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 106/152 2024-04-22T13:12:08.983 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 106/152 2024-04-22T13:12:08.983 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:12:08.983 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-22T13:12:08.983 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-22T13:12:09.561 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-22T13:12:09.598 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/152 2024-04-22T13:12:09.622 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/152 2024-04-22T13:12:09.622 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:12:09.622 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-22T13:12:09.622 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T13:12:09.623 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T13:12:09.623 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:12:09.665 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 134/152 2024-04-22T13:12:09.688 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 134/152 2024-04-22T13:12:09.689 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:12:09.689 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-22T13:12:09.689 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:12:09.733 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/152 2024-04-22T13:12:09.761 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/152 2024-04-22T13:12:09.761 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:12:09.761 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-22T13:12:09.761 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-22T13:12:09.761 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-22T13:12:09.761 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:12:10.083 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 136/152 2024-04-22T13:12:10.110 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 136/152 2024-04-22T13:12:10.110 INFO:teuthology.orchestra.run.smithi012.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:12:10.110 INFO:teuthology.orchestra.run.smithi012.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-22T13:12:10.110 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T13:12:10.110 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T13:12:10.110 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:12:10.470 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/152 2024-04-22T13:12:10.495 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/152 2024-04-22T13:12:10.495 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:12:10.495 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-22T13:12:10.495 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T13:12:10.495 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T13:12:10.496 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-22T13:12:10.720 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 108/152 2024-04-22T13:12:10.743 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 108/152 2024-04-22T13:12:10.743 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:12:10.743 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-22T13:12:10.743 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T13:12:10.743 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T13:12:10.743 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-22T13:12:10.995 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2024-04-22T13:12:11.065 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-22T13:12:11.090 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-22T13:12:11.142 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-22T13:12:11.179 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/152 2024-04-22T13:12:11.201 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/152 2024-04-22T13:12:11.201 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:12:11.202 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-22T13:12:11.202 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T13:12:11.202 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T13:12:11.202 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-22T13:12:11.309 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-22T13:12:11.358 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-22T13:12:11.413 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-22T13:12:11.470 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-22T13:12:11.575 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 5/152 2024-04-22T13:12:11.599 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-22T13:12:11.646 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-22T13:12:11.696 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 5/152 2024-04-22T13:12:11.725 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/152 2024-04-22T13:12:11.733 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-22T13:12:11.816 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/152 2024-04-22T13:12:11.821 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-22T13:12:11.859 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/152 2024-04-22T13:12:11.897 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-22T13:12:11.931 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-22T13:12:11.961 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-22T13:12:12.005 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-22T13:12:12.036 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-22T13:12:12.073 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-22T13:12:12.087 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-22T13:12:12.136 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-22T13:12:12.164 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/152 2024-04-22T13:12:12.250 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/152 2024-04-22T13:12:12.273 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 13/152 2024-04-22T13:12:12.336 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 13/152 2024-04-22T13:12:12.358 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-22T13:12:12.376 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-22T13:12:12.516 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 121/152 2024-04-22T13:12:12.533 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-22T13:12:12.546 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 121/152 2024-04-22T13:12:12.566 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-22T13:12:12.578 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 122/152 2024-04-22T13:12:12.640 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 123/152 2024-04-22T13:12:12.669 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 124/152 2024-04-22T13:12:12.673 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-22T13:12:12.763 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-22T13:12:12.798 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-22T13:12:12.834 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-22T13:12:12.909 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-22T13:12:12.950 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-22T13:12:12.993 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-22T13:12:13.034 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-22T13:12:13.062 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-22T13:12:13.132 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-22T13:12:13.185 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-22T13:12:13.214 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-22T13:12:13.249 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-22T13:12:13.327 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-22T13:12:13.361 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 30/152 2024-04-22T13:12:13.395 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 31/152 2024-04-22T13:12:13.424 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 32/152 2024-04-22T13:12:13.497 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-22T13:12:13.539 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-22T13:12:13.615 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-22T13:12:13.702 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 124/152 2024-04-22T13:12:13.746 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 125/152 2024-04-22T13:12:13.753 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-22T13:12:13.810 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-22T13:12:14.353 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-22T13:12:14.439 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-22T13:12:14.475 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-22T13:12:14.517 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-22T13:12:14.555 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 42/152 2024-04-22T13:12:14.589 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-22T13:12:14.645 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-22T13:12:14.705 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 125/152 2024-04-22T13:12:14.719 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-22T13:12:14.738 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 126/152 2024-04-22T13:12:14.756 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-22T13:12:14.792 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-22T13:12:14.839 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 126/152 2024-04-22T13:12:14.840 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-22T13:12:14.875 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-22T13:12:14.918 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-22T13:12:14.954 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/152 2024-04-22T13:12:14.961 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-22T13:12:14.996 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/152 2024-04-22T13:12:15.002 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 127/152 2024-04-22T13:12:15.031 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 128/152 2024-04-22T13:12:15.045 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/152 2024-04-22T13:12:15.054 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 128/152 2024-04-22T13:12:15.054 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:12:15.054 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-22T13:12:15.054 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T13:12:15.054 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T13:12:15.055 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-22T13:12:15.085 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-22T13:12:15.098 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 129/152 2024-04-22T13:12:15.100 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-22T13:12:15.110 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 129/152 2024-04-22T13:12:15.112 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-22T13:12:15.141 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-22T13:12:15.171 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-22T13:12:15.208 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/152 2024-04-22T13:12:15.213 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-22T13:12:15.264 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 131/152 2024-04-22T13:12:15.309 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-22T13:12:15.313 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 142/152 2024-04-22T13:12:15.346 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-22T13:12:15.367 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-22T13:12:15.386 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-22T13:12:15.396 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 144/152 2024-04-22T13:12:15.411 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-22T13:12:15.436 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-22T13:12:15.513 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-22T13:12:15.641 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-22T13:12:15.677 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-22T13:12:15.705 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-22T13:12:15.740 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-22T13:12:15.784 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-22T13:12:15.821 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-22T13:12:15.861 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-22T13:12:15.891 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-22T13:12:15.917 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-22T13:12:15.950 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-22T13:12:15.991 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T13:12:15.992 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-22T13:12:16.026 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-22T13:12:16.027 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T13:12:16.048 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T13:12:16.115 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-22T13:12:16.165 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-22T13:12:16.190 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-22T13:12:16.196 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-22T13:12:16.218 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/152 2024-04-22T13:12:16.240 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/152 2024-04-22T13:12:16.241 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:12:16.241 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-22T13:12:16.241 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T13:12:16.241 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T13:12:16.241 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-22T13:12:16.245 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-22T13:12:16.285 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 134/152 2024-04-22T13:12:16.307 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 134/152 2024-04-22T13:12:16.307 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:12:16.308 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-22T13:12:16.308 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-22T13:12:16.352 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/152 2024-04-22T13:12:16.353 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-22T13:12:16.375 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/152 2024-04-22T13:12:16.375 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:12:16.376 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-22T13:12:16.376 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-22T13:12:16.376 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-22T13:12:16.376 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-22T13:12:16.415 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-22T13:12:16.678 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 136/152 2024-04-22T13:12:16.701 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 136/152 2024-04-22T13:12:16.701 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:12:16.701 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-22T13:12:16.701 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T13:12:16.701 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T13:12:16.701 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-22T13:12:16.904 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 146/152 2024-04-22T13:12:16.990 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-22T13:12:17.033 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-22T13:12:17.127 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-22T13:12:17.161 INFO:teuthology.orchestra.run.smithi012.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-22T13:12:17.161 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-22T13:12:17.167 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-22T13:12:17.212 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-22T13:12:17.245 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-22T13:12:17.245 INFO:teuthology.orchestra.run.smithi012.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T13:12:17.246 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-22T13:12:17.351 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-22T13:12:17.404 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-22T13:12:17.494 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-22T13:12:17.531 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-22T13:12:17.707 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-22T13:12:17.736 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-22T13:12:17.813 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-22T13:12:18.481 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-22T13:12:18.603 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-22T13:12:19.877 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-22T13:12:19.966 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-22T13:12:19.994 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-22T13:12:20.027 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-22T13:12:20.338 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-22T13:12:20.381 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 97/152 2024-04-22T13:12:20.480 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 97/152 2024-04-22T13:12:20.511 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 98/152 2024-04-22T13:12:20.547 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-22T13:12:21.035 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-22T13:12:21.067 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 101/152 2024-04-22T13:12:21.138 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 101/152 2024-04-22T13:12:21.160 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 102/152 2024-04-22T13:12:21.281 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/152 2024-04-22T13:12:21.322 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/152 2024-04-22T13:12:21.363 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/152 2024-04-22T13:12:21.395 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-22T13:12:21.422 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-22T13:12:21.606 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 142/152 2024-04-22T13:12:21.652 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-22T13:12:21.682 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 144/152 2024-04-22T13:12:22.248 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T13:12:22.288 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T13:12:22.308 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T13:12:23.077 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 146/152 2024-04-22T13:12:23.140 INFO:teuthology.orchestra.run.smithi012.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T13:12:23.140 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/152 2024-04-22T13:12:23.140 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/152 2024-04-22T13:12:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/152 2024-04-22T13:12:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/152 2024-04-22T13:12:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/152 2024-04-22T13:12:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/152 2024-04-22T13:12:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/152 2024-04-22T13:12:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/152 2024-04-22T13:12:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/152 2024-04-22T13:12:23.141 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/152 2024-04-22T13:12:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/152 2024-04-22T13:12:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/152 2024-04-22T13:12:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/152 2024-04-22T13:12:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/152 2024-04-22T13:12:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/152 2024-04-22T13:12:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/152 2024-04-22T13:12:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/152 2024-04-22T13:12:23.142 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/152 2024-04-22T13:12:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/152 2024-04-22T13:12:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/152 2024-04-22T13:12:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/152 2024-04-22T13:12:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/152 2024-04-22T13:12:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/152 2024-04-22T13:12:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/152 2024-04-22T13:12:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/152 2024-04-22T13:12:23.143 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/152 2024-04-22T13:12:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/152 2024-04-22T13:12:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/152 2024-04-22T13:12:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/152 2024-04-22T13:12:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/152 2024-04-22T13:12:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/152 2024-04-22T13:12:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/152 2024-04-22T13:12:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/152 2024-04-22T13:12:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/152 2024-04-22T13:12:23.144 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/152 2024-04-22T13:12:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/152 2024-04-22T13:12:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/152 2024-04-22T13:12:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/152 2024-04-22T13:12:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/152 2024-04-22T13:12:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-22T13:12:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-22T13:12:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-22T13:12:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-22T13:12:23.145 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-22T13:12:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-22T13:12:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-22T13:12:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-22T13:12:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-22T13:12:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-22T13:12:23.146 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-22T13:12:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-22T13:12:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-22T13:12:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-22T13:12:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-22T13:12:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-22T13:12:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-22T13:12:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-22T13:12:23.147 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-22T13:12:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-22T13:12:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-22T13:12:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-22T13:12:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-22T13:12:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-22T13:12:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-22T13:12:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-22T13:12:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-22T13:12:23.148 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-22T13:12:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-22T13:12:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-22T13:12:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-22T13:12:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-22T13:12:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-22T13:12:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-22T13:12:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-22T13:12:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-22T13:12:23.149 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-22T13:12:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-22T13:12:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-22T13:12:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-22T13:12:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-22T13:12:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-22T13:12:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-22T13:12:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-22T13:12:23.150 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-22T13:12:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-22T13:12:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-22T13:12:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-22T13:12:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-22T13:12:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-22T13:12:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-22T13:12:23.151 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-22T13:12:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-22T13:12:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-22T13:12:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-22T13:12:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-22T13:12:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-22T13:12:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-22T13:12:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-22T13:12:23.152 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-22T13:12:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-22T13:12:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-22T13:12:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-22T13:12:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-22T13:12:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-22T13:12:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-22T13:12:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-22T13:12:23.153 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-22T13:12:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-22T13:12:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-22T13:12:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-22T13:12:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-22T13:12:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-22T13:12:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-22T13:12:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-22T13:12:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-22T13:12:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-22T13:12:23.154 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-22T13:12:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-22T13:12:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-22T13:12:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-22T13:12:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-22T13:12:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-22T13:12:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-22T13:12:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-22T13:12:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-22T13:12:23.155 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-22T13:12:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-22T13:12:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-22T13:12:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-22T13:12:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-22T13:12:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-22T13:12:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-22T13:12:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-22T13:12:23.156 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-22T13:12:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-22T13:12:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-22T13:12:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-22T13:12:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-22T13:12:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-22T13:12:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-22T13:12:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-22T13:12:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-22T13:12:23.157 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-22T13:12:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-22T13:12:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-22T13:12:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-22T13:12:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-22T13:12:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-22T13:12:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/152 2024-04-22T13:12:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-22T13:12:23.158 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/152 2024-04-22T13:12:23.159 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-22T13:12:23.200 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-22T13:12:23.281 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-22T13:12:23.313 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-22T13:12:23.313 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-22T13:12:23.337 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/152 2024-04-22T13:12:23.374 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/152 2024-04-22T13:12:23.396 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-22T13:12:23.396 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T13:12:23.913 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/152 2024-04-22T13:12:23.960 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/152 2024-04-22T13:12:24.041 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/152 2024-04-22T13:12:24.042 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-22T13:12:24.042 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-22T13:12:24.042 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-22T13:12:24.070 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/152 2024-04-22T13:12:25.241 INFO:teuthology.orchestra.run.smithi012.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T13:12:25.241 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:12:25.241 INFO:teuthology.orchestra.run.smithi012.stdout:Upgraded: 2024-04-22T13:12:25.241 INFO:teuthology.orchestra.run.smithi012.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:25.241 INFO:teuthology.orchestra.run.smithi012.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:25.241 INFO:teuthology.orchestra.run.smithi012.stdout:Installed: 2024-04-22T13:12:25.241 INFO:teuthology.orchestra.run.smithi012.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-22T13:12:25.241 INFO:teuthology.orchestra.run.smithi012.stdout: automake-1.16.2-8.el9.noarch 2024-04-22T13:12:25.241 INFO:teuthology.orchestra.run.smithi012.stdout: bison-3.7.4-5.el9.x86_64 2024-04-22T13:12:25.242 INFO:teuthology.orchestra.run.smithi012.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-22T13:12:25.242 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:25.242 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:25.242 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:25.242 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:25.242 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:12:25.242 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:25.242 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:25.242 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:25.242 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:12:25.242 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:12:25.242 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:12:25.242 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:12:25.242 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:12:25.243 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:25.243 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:25.243 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:12:25.243 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:25.243 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:25.243 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:25.243 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:12:25.243 INFO:teuthology.orchestra.run.smithi012.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:12:25.243 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:25.243 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:25.243 INFO:teuthology.orchestra.run.smithi012.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:12:25.243 INFO:teuthology.orchestra.run.smithi012.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-22T13:12:25.243 INFO:teuthology.orchestra.run.smithi012.stdout: flex-2.6.4-9.el9.x86_64 2024-04-22T13:12:25.243 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-22T13:12:25.245 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-22T13:12:25.245 INFO:teuthology.orchestra.run.smithi012.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-22T13:12:25.245 INFO:teuthology.orchestra.run.smithi012.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-22T13:12:25.245 INFO:teuthology.orchestra.run.smithi012.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-22T13:12:25.245 INFO:teuthology.orchestra.run.smithi012.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-22T13:12:25.245 INFO:teuthology.orchestra.run.smithi012.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-22T13:12:25.245 INFO:teuthology.orchestra.run.smithi012.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-22T13:12:25.245 INFO:teuthology.orchestra.run.smithi012.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-22T13:12:25.245 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:25.245 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:25.246 INFO:teuthology.orchestra.run.smithi012.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:25.246 INFO:teuthology.orchestra.run.smithi012.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-22T13:12:25.246 INFO:teuthology.orchestra.run.smithi012.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-22T13:12:25.246 INFO:teuthology.orchestra.run.smithi012.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-22T13:12:25.246 INFO:teuthology.orchestra.run.smithi012.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-22T13:12:25.246 INFO:teuthology.orchestra.run.smithi012.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-22T13:12:25.246 INFO:teuthology.orchestra.run.smithi012.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:25.246 INFO:teuthology.orchestra.run.smithi012.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:25.246 INFO:teuthology.orchestra.run.smithi012.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-22T13:12:25.246 INFO:teuthology.orchestra.run.smithi012.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:25.246 INFO:teuthology.orchestra.run.smithi012.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-22T13:12:25.246 INFO:teuthology.orchestra.run.smithi012.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-22T13:12:25.247 INFO:teuthology.orchestra.run.smithi012.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-22T13:12:25.247 INFO:teuthology.orchestra.run.smithi012.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-22T13:12:25.247 INFO:teuthology.orchestra.run.smithi012.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-22T13:12:25.247 INFO:teuthology.orchestra.run.smithi012.stdout: lua-5.4.4-4.el9.x86_64 2024-04-22T13:12:25.247 INFO:teuthology.orchestra.run.smithi012.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-22T13:12:25.247 INFO:teuthology.orchestra.run.smithi012.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-22T13:12:25.247 INFO:teuthology.orchestra.run.smithi012.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-22T13:12:25.247 INFO:teuthology.orchestra.run.smithi012.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-22T13:12:25.247 INFO:teuthology.orchestra.run.smithi012.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-22T13:12:25.247 INFO:teuthology.orchestra.run.smithi012.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-22T13:12:25.247 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-13.14-1.el9.x86_64 2024-04-22T13:12:25.247 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-22T13:12:25.247 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-22T13:12:25.247 INFO:teuthology.orchestra.run.smithi012.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-22T13:12:25.248 INFO:teuthology.orchestra.run.smithi012.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-22T13:12:25.248 INFO:teuthology.orchestra.run.smithi012.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-22T13:12:25.248 INFO:teuthology.orchestra.run.smithi012.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-22T13:12:25.248 INFO:teuthology.orchestra.run.smithi012.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-22T13:12:25.248 INFO:teuthology.orchestra.run.smithi012.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-22T13:12:25.248 INFO:teuthology.orchestra.run.smithi012.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-22T13:12:25.248 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-22T13:12:25.248 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:25.248 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:25.248 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:25.248 INFO:teuthology.orchestra.run.smithi012.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-22T13:12:25.249 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-22T13:12:25.250 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-22T13:12:25.250 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-22T13:12:25.250 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-22T13:12:25.250 INFO:teuthology.orchestra.run.smithi012.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-22T13:12:25.250 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-22T13:12:25.250 INFO:teuthology.orchestra.run.smithi012.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-22T13:12:25.250 INFO:teuthology.orchestra.run.smithi012.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-22T13:12:25.250 INFO:teuthology.orchestra.run.smithi012.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-22T13:12:25.250 INFO:teuthology.orchestra.run.smithi012.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-22T13:12:25.250 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-22T13:12:25.250 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-22T13:12:25.251 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-22T13:12:25.251 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-22T13:12:25.251 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-22T13:12:25.251 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-22T13:12:25.251 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-22T13:12:25.251 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-22T13:12:25.251 INFO:teuthology.orchestra.run.smithi012.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-22T13:12:25.251 INFO:teuthology.orchestra.run.smithi012.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-22T13:12:25.251 INFO:teuthology.orchestra.run.smithi012.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-22T13:12:25.251 INFO:teuthology.orchestra.run.smithi012.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-22T13:12:25.251 INFO:teuthology.orchestra.run.smithi012.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-22T13:12:25.251 INFO:teuthology.orchestra.run.smithi012.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-22T13:12:25.252 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-22T13:12:25.252 INFO:teuthology.orchestra.run.smithi012.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-22T13:12:25.252 INFO:teuthology.orchestra.run.smithi012.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-22T13:12:25.252 INFO:teuthology.orchestra.run.smithi012.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-22T13:12:25.252 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-22T13:12:25.252 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ply-3.11-14.el9.noarch 2024-04-22T13:12:25.252 INFO:teuthology.orchestra.run.smithi012.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-22T13:12:25.252 INFO:teuthology.orchestra.run.smithi012.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-22T13:12:25.252 INFO:teuthology.orchestra.run.smithi012.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-22T13:12:25.252 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-22T13:12:25.252 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-22T13:12:25.252 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-22T13:12:25.252 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-22T13:12:25.252 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-22T13:12:25.253 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-22T13:12:25.253 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:25.253 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:25.253 INFO:teuthology.orchestra.run.smithi012.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-22T13:12:25.253 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-22T13:12:25.253 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-22T13:12:25.253 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:25.253 INFO:teuthology.orchestra.run.smithi012.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-22T13:12:25.253 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-22T13:12:25.253 INFO:teuthology.orchestra.run.smithi012.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-22T13:12:25.253 INFO:teuthology.orchestra.run.smithi012.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-22T13:12:25.253 INFO:teuthology.orchestra.run.smithi012.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-22T13:12:25.253 INFO:teuthology.orchestra.run.smithi012.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-22T13:12:25.253 INFO:teuthology.orchestra.run.smithi012.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-22T13:12:25.254 INFO:teuthology.orchestra.run.smithi012.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-22T13:12:25.254 INFO:teuthology.orchestra.run.smithi012.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-22T13:12:25.254 INFO:teuthology.orchestra.run.smithi012.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-22T13:12:25.254 INFO:teuthology.orchestra.run.smithi012.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-22T13:12:25.254 INFO:teuthology.orchestra.run.smithi012.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-22T13:12:25.254 INFO:teuthology.orchestra.run.smithi012.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-22T13:12:25.254 INFO:teuthology.orchestra.run.smithi012.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-22T13:12:25.254 INFO:teuthology.orchestra.run.smithi012.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-22T13:12:25.254 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:25.254 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:25.254 INFO:teuthology.orchestra.run.smithi012.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:25.254 INFO:teuthology.orchestra.run.smithi012.stdout: re2-1:20211101-3.el9.x86_64 2024-04-22T13:12:25.254 INFO:teuthology.orchestra.run.smithi012.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-22T13:12:25.254 INFO:teuthology.orchestra.run.smithi012.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-22T13:12:25.255 INFO:teuthology.orchestra.run.smithi012.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-22T13:12:25.255 INFO:teuthology.orchestra.run.smithi012.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-22T13:12:25.255 INFO:teuthology.orchestra.run.smithi012.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-22T13:12:25.255 INFO:teuthology.orchestra.run.smithi012.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-22T13:12:25.255 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:12:25.255 INFO:teuthology.orchestra.run.smithi012.stdout:Complete! 2024-04-22T13:12:25.570 DEBUG:teuthology.parallel:result is None 2024-04-22T13:12:29.297 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T13:12:29.297 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/152 2024-04-22T13:12:29.297 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/152 2024-04-22T13:12:29.298 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/152 2024-04-22T13:12:29.298 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/152 2024-04-22T13:12:29.298 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/152 2024-04-22T13:12:29.298 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/152 2024-04-22T13:12:29.298 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/152 2024-04-22T13:12:29.298 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/152 2024-04-22T13:12:29.298 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/152 2024-04-22T13:12:29.298 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/152 2024-04-22T13:12:29.298 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/152 2024-04-22T13:12:29.298 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/152 2024-04-22T13:12:29.298 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/152 2024-04-22T13:12:29.298 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/152 2024-04-22T13:12:29.298 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/152 2024-04-22T13:12:29.299 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/152 2024-04-22T13:12:29.299 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/152 2024-04-22T13:12:29.299 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/152 2024-04-22T13:12:29.299 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/152 2024-04-22T13:12:29.299 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/152 2024-04-22T13:12:29.299 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/152 2024-04-22T13:12:29.299 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/152 2024-04-22T13:12:29.299 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/152 2024-04-22T13:12:29.299 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/152 2024-04-22T13:12:29.299 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/152 2024-04-22T13:12:29.299 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/152 2024-04-22T13:12:29.300 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/152 2024-04-22T13:12:29.300 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/152 2024-04-22T13:12:29.300 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/152 2024-04-22T13:12:29.300 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/152 2024-04-22T13:12:29.311 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/152 2024-04-22T13:12:29.311 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/152 2024-04-22T13:12:29.312 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/152 2024-04-22T13:12:29.312 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/152 2024-04-22T13:12:29.312 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/152 2024-04-22T13:12:29.312 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/152 2024-04-22T13:12:29.312 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/152 2024-04-22T13:12:29.312 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/152 2024-04-22T13:12:29.312 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/152 2024-04-22T13:12:29.312 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-22T13:12:29.312 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-22T13:12:29.312 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-22T13:12:29.312 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-22T13:12:29.313 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-22T13:12:29.313 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-22T13:12:29.313 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-22T13:12:29.313 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-22T13:12:29.313 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-22T13:12:29.313 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-22T13:12:29.313 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-22T13:12:29.313 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-22T13:12:29.313 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-22T13:12:29.313 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-22T13:12:29.313 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-22T13:12:29.313 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-22T13:12:29.313 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-22T13:12:29.313 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-22T13:12:29.314 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-22T13:12:29.314 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-22T13:12:29.314 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-22T13:12:29.314 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-22T13:12:29.314 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-22T13:12:29.314 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-22T13:12:29.314 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-22T13:12:29.314 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-22T13:12:29.314 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-22T13:12:29.314 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-22T13:12:29.314 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-22T13:12:29.314 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-22T13:12:29.314 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-22T13:12:29.314 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-22T13:12:29.315 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-22T13:12:29.315 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-22T13:12:29.315 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-22T13:12:29.315 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-22T13:12:29.315 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-22T13:12:29.315 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-22T13:12:29.315 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-22T13:12:29.315 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-22T13:12:29.315 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-22T13:12:29.315 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-22T13:12:29.315 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-22T13:12:29.315 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-22T13:12:29.315 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-22T13:12:29.315 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-22T13:12:29.315 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-22T13:12:29.316 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-22T13:12:29.316 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-22T13:12:29.316 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-22T13:12:29.316 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-22T13:12:29.316 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-22T13:12:29.316 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-22T13:12:29.316 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-22T13:12:29.316 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-22T13:12:29.316 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-22T13:12:29.316 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-22T13:12:29.316 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-22T13:12:29.316 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-22T13:12:29.317 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-22T13:12:29.317 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-22T13:12:29.317 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-22T13:12:29.317 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-22T13:12:29.317 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-22T13:12:29.317 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-22T13:12:29.317 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-22T13:12:29.317 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-22T13:12:29.317 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-22T13:12:29.317 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-22T13:12:29.317 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-22T13:12:29.318 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-22T13:12:29.318 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-22T13:12:29.318 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-22T13:12:29.318 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-22T13:12:29.318 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-22T13:12:29.318 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-22T13:12:29.318 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-22T13:12:29.318 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-22T13:12:29.318 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-22T13:12:29.318 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-22T13:12:29.318 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-22T13:12:29.318 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-22T13:12:29.318 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-22T13:12:29.318 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-22T13:12:29.319 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-22T13:12:29.319 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-22T13:12:29.319 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-22T13:12:29.319 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-22T13:12:29.319 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-22T13:12:29.319 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-22T13:12:29.319 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-22T13:12:29.319 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-22T13:12:29.319 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-22T13:12:29.319 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-22T13:12:29.319 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-22T13:12:29.319 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-22T13:12:29.319 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-22T13:12:29.319 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-22T13:12:29.320 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-22T13:12:29.320 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-22T13:12:29.320 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-22T13:12:29.320 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-22T13:12:29.320 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-22T13:12:29.320 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-22T13:12:29.320 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-22T13:12:29.320 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-22T13:12:29.320 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-22T13:12:29.321 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-22T13:12:29.321 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-22T13:12:29.321 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/152 2024-04-22T13:12:29.321 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-22T13:12:29.321 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/152 2024-04-22T13:12:31.366 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T13:12:31.366 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-22T13:12:31.367 INFO:teuthology.orchestra.run.smithi134.stdout:Upgraded: 2024-04-22T13:12:31.367 INFO:teuthology.orchestra.run.smithi134.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:31.367 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:31.367 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-04-22T13:12:31.367 INFO:teuthology.orchestra.run.smithi134.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-22T13:12:31.367 INFO:teuthology.orchestra.run.smithi134.stdout: automake-1.16.2-8.el9.noarch 2024-04-22T13:12:31.367 INFO:teuthology.orchestra.run.smithi134.stdout: bison-3.7.4-5.el9.x86_64 2024-04-22T13:12:31.367 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-22T13:12:31.367 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:31.367 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:31.367 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:31.367 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:31.367 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:12:31.368 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:31.368 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:31.368 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:31.368 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:12:31.368 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:12:31.368 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:12:31.368 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:12:31.368 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:12:31.368 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:31.368 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:31.368 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:12:31.368 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:31.368 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:31.368 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:31.368 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:12:31.369 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:12:31.369 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:31.369 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:31.369 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:12:31.369 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-22T13:12:31.369 INFO:teuthology.orchestra.run.smithi134.stdout: flex-2.6.4-9.el9.x86_64 2024-04-22T13:12:31.369 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-22T13:12:31.369 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-22T13:12:31.369 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-22T13:12:31.369 INFO:teuthology.orchestra.run.smithi134.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-22T13:12:31.369 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-22T13:12:31.369 INFO:teuthology.orchestra.run.smithi134.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-22T13:12:31.369 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-22T13:12:31.370 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-22T13:12:31.370 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-22T13:12:31.370 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:31.370 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:31.370 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:31.370 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-22T13:12:31.370 INFO:teuthology.orchestra.run.smithi134.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-22T13:12:31.370 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-22T13:12:31.370 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-22T13:12:31.370 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-22T13:12:31.370 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:31.370 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:31.370 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-22T13:12:31.370 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:31.371 INFO:teuthology.orchestra.run.smithi134.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-22T13:12:31.371 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-22T13:12:31.371 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-22T13:12:31.371 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-22T13:12:31.371 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-22T13:12:31.371 INFO:teuthology.orchestra.run.smithi134.stdout: lua-5.4.4-4.el9.x86_64 2024-04-22T13:12:31.371 INFO:teuthology.orchestra.run.smithi134.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-22T13:12:31.371 INFO:teuthology.orchestra.run.smithi134.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-22T13:12:31.371 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-22T13:12:31.371 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-22T13:12:31.371 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-22T13:12:31.371 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-22T13:12:31.371 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-13.14-1.el9.x86_64 2024-04-22T13:12:31.371 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-22T13:12:31.371 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-22T13:12:31.372 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-22T13:12:31.372 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-22T13:12:31.372 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-22T13:12:31.372 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-22T13:12:31.372 INFO:teuthology.orchestra.run.smithi134.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-22T13:12:31.372 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-22T13:12:31.372 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-22T13:12:31.372 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-22T13:12:31.372 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:31.372 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:31.372 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:31.372 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-22T13:12:31.372 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-22T13:12:31.373 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-22T13:12:31.373 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-22T13:12:31.373 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-22T13:12:31.373 INFO:teuthology.orchestra.run.smithi134.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-22T13:12:31.373 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-22T13:12:31.373 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-22T13:12:31.373 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-22T13:12:31.373 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-22T13:12:31.373 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-22T13:12:31.373 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-22T13:12:31.373 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-22T13:12:31.373 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-22T13:12:31.373 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-22T13:12:31.374 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-22T13:12:31.374 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-22T13:12:31.374 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-22T13:12:31.374 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-22T13:12:31.374 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-22T13:12:31.374 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-22T13:12:31.374 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-22T13:12:31.374 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-22T13:12:31.374 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-22T13:12:31.374 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-22T13:12:31.374 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-22T13:12:31.374 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-22T13:12:31.374 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-22T13:12:31.375 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-22T13:12:31.375 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-22T13:12:31.375 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply-3.11-14.el9.noarch 2024-04-22T13:12:31.375 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-22T13:12:31.375 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-22T13:12:31.375 INFO:teuthology.orchestra.run.smithi134.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-22T13:12:31.375 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-22T13:12:31.377 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-22T13:12:31.377 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-22T13:12:31.377 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-22T13:12:31.377 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-22T13:12:31.377 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-22T13:12:31.377 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:31.378 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:31.378 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-22T13:12:31.378 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-22T13:12:31.378 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-22T13:12:31.378 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:31.378 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-22T13:12:31.378 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-22T13:12:31.378 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-22T13:12:31.378 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-22T13:12:31.378 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-22T13:12:31.378 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-22T13:12:31.378 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-22T13:12:31.378 INFO:teuthology.orchestra.run.smithi134.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-22T13:12:31.378 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-22T13:12:31.379 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-22T13:12:31.379 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-22T13:12:31.379 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-22T13:12:31.379 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-22T13:12:31.379 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-22T13:12:31.379 INFO:teuthology.orchestra.run.smithi134.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-22T13:12:31.379 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:31.379 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:31.379 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:31.379 INFO:teuthology.orchestra.run.smithi134.stdout: re2-1:20211101-3.el9.x86_64 2024-04-22T13:12:31.379 INFO:teuthology.orchestra.run.smithi134.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-22T13:12:31.379 INFO:teuthology.orchestra.run.smithi134.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-22T13:12:31.379 INFO:teuthology.orchestra.run.smithi134.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-22T13:12:31.379 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-22T13:12:31.379 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-22T13:12:31.380 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-22T13:12:31.380 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-22T13:12:31.380 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-04-22T13:12:31.745 DEBUG:teuthology.parallel:result is None 2024-04-22T13:12:35.303 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/152 2024-04-22T13:12:35.304 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /sys 2024-04-22T13:12:35.304 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /proc 2024-04-22T13:12:35.304 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /mnt 2024-04-22T13:12:35.304 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /var/tmp 2024-04-22T13:12:35.304 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /home 2024-04-22T13:12:35.304 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /root 2024-04-22T13:12:35.304 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /tmp 2024-04-22T13:12:35.304 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-22T13:12:35.491 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 106/152 2024-04-22T13:12:35.506 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 106/152 2024-04-22T13:12:35.506 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:12:35.506 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-22T13:12:35.506 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-22T13:12:36.977 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/152 2024-04-22T13:12:37.003 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/152 2024-04-22T13:12:37.003 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:12:37.003 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-22T13:12:37.003 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T13:12:37.003 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T13:12:37.003 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-22T13:12:37.219 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 108/152 2024-04-22T13:12:37.241 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 108/152 2024-04-22T13:12:37.242 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:12:37.242 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-22T13:12:37.242 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T13:12:37.242 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T13:12:37.242 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-22T13:12:37.679 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/152 2024-04-22T13:12:37.700 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/152 2024-04-22T13:12:37.700 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:12:37.700 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-22T13:12:37.701 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T13:12:37.701 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T13:12:37.701 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-22T13:12:37.809 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-22T13:12:37.866 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-22T13:12:37.921 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-22T13:12:38.108 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-22T13:12:38.155 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-22T13:12:38.233 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-22T13:12:38.322 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-22T13:12:38.461 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-22T13:12:38.505 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-22T13:12:38.537 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-22T13:12:38.564 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-22T13:12:39.025 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 121/152 2024-04-22T13:12:39.055 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 121/152 2024-04-22T13:12:39.086 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 122/152 2024-04-22T13:12:39.149 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 123/152 2024-04-22T13:12:39.177 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 124/152 2024-04-22T13:12:40.219 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 124/152 2024-04-22T13:12:40.255 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 125/152 2024-04-22T13:12:41.215 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 125/152 2024-04-22T13:12:41.238 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 126/152 2024-04-22T13:12:41.341 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 126/152 2024-04-22T13:12:41.494 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 127/152 2024-04-22T13:12:41.523 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 128/152 2024-04-22T13:12:41.546 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 128/152 2024-04-22T13:12:41.546 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:12:41.546 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-22T13:12:41.546 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T13:12:41.547 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T13:12:41.547 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-22T13:12:41.591 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 129/152 2024-04-22T13:12:41.602 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 129/152 2024-04-22T13:12:41.699 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/152 2024-04-22T13:12:41.748 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 131/152 2024-04-22T13:12:42.669 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-22T13:12:42.701 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/152 2024-04-22T13:12:42.723 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/152 2024-04-22T13:12:42.723 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:12:42.723 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-22T13:12:42.723 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T13:12:42.724 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T13:12:42.724 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-22T13:12:42.769 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 134/152 2024-04-22T13:12:42.790 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 134/152 2024-04-22T13:12:42.790 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:12:42.790 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-22T13:12:42.790 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-22T13:12:42.836 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/152 2024-04-22T13:12:42.858 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/152 2024-04-22T13:12:42.858 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:12:42.859 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-22T13:12:42.859 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-22T13:12:42.859 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-22T13:12:42.859 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-22T13:12:43.170 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 136/152 2024-04-22T13:12:43.193 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 136/152 2024-04-22T13:12:43.193 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T13:12:43.193 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-22T13:12:43.193 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T13:12:43.193 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T13:12:43.193 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-22T13:12:47.782 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/152 2024-04-22T13:12:47.823 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/152 2024-04-22T13:12:47.864 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/152 2024-04-22T13:12:47.896 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-22T13:12:47.923 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-22T13:12:48.113 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 142/152 2024-04-22T13:12:48.161 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-22T13:12:48.191 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 144/152 2024-04-22T13:12:48.766 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T13:12:48.805 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T13:12:48.826 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-22T13:12:49.619 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 146/152 2024-04-22T13:12:49.710 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-22T13:12:49.751 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-22T13:12:49.835 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-22T13:12:49.864 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-22T13:12:49.865 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-22T13:12:49.948 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-22T13:12:49.948 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T13:12:55.818 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T13:12:55.819 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/152 2024-04-22T13:12:55.819 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/152 2024-04-22T13:12:55.819 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/152 2024-04-22T13:12:55.819 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/152 2024-04-22T13:12:55.819 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/152 2024-04-22T13:12:55.819 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/152 2024-04-22T13:12:55.819 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/152 2024-04-22T13:12:55.819 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/152 2024-04-22T13:12:55.819 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/152 2024-04-22T13:12:55.819 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/152 2024-04-22T13:12:55.819 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/152 2024-04-22T13:12:55.819 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/152 2024-04-22T13:12:55.819 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/152 2024-04-22T13:12:55.820 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/152 2024-04-22T13:12:55.820 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/152 2024-04-22T13:12:55.820 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/152 2024-04-22T13:12:55.820 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/152 2024-04-22T13:12:55.820 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/152 2024-04-22T13:12:55.820 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/152 2024-04-22T13:12:55.820 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/152 2024-04-22T13:12:55.820 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/152 2024-04-22T13:12:55.820 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/152 2024-04-22T13:12:55.820 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/152 2024-04-22T13:12:55.820 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/152 2024-04-22T13:12:55.820 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/152 2024-04-22T13:12:55.820 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/152 2024-04-22T13:12:55.820 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/152 2024-04-22T13:12:55.821 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/152 2024-04-22T13:12:55.821 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/152 2024-04-22T13:12:55.821 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/152 2024-04-22T13:12:55.832 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/152 2024-04-22T13:12:55.832 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/152 2024-04-22T13:12:55.832 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/152 2024-04-22T13:12:55.832 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/152 2024-04-22T13:12:55.832 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/152 2024-04-22T13:12:55.832 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/152 2024-04-22T13:12:55.832 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/152 2024-04-22T13:12:55.833 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/152 2024-04-22T13:12:55.833 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/152 2024-04-22T13:12:55.833 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-22T13:12:55.833 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-22T13:12:55.833 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-22T13:12:55.833 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-22T13:12:55.833 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-22T13:12:55.833 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-22T13:12:55.833 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-22T13:12:55.833 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-22T13:12:55.834 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-22T13:12:55.834 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-22T13:12:55.834 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-22T13:12:55.834 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-22T13:12:55.834 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-22T13:12:55.834 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-22T13:12:55.834 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-22T13:12:55.834 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-22T13:12:55.834 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-22T13:12:55.834 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-22T13:12:55.834 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-22T13:12:55.834 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-22T13:12:55.834 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-22T13:12:55.835 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-22T13:12:55.835 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-22T13:12:55.835 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-22T13:12:55.835 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-22T13:12:55.835 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-22T13:12:55.835 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-22T13:12:55.835 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-22T13:12:55.835 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-22T13:12:55.835 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-22T13:12:55.835 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-22T13:12:55.835 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-22T13:12:55.835 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-22T13:12:55.835 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-22T13:12:55.835 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-22T13:12:55.836 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-22T13:12:55.836 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-22T13:12:55.836 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-22T13:12:55.836 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-22T13:12:55.836 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-22T13:12:55.836 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-22T13:12:55.836 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-22T13:12:55.836 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-22T13:12:55.836 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-22T13:12:55.836 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-22T13:12:55.836 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-22T13:12:55.836 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-22T13:12:55.836 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-22T13:12:55.836 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-22T13:12:55.837 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-22T13:12:55.837 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-22T13:12:55.837 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-22T13:12:55.837 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-22T13:12:55.837 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-22T13:12:55.837 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-22T13:12:55.837 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-22T13:12:55.837 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-22T13:12:55.837 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-22T13:12:55.837 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-22T13:12:55.837 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-22T13:12:55.837 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-22T13:12:55.837 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-22T13:12:55.838 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-22T13:12:55.838 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-22T13:12:55.838 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-22T13:12:55.838 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-22T13:12:55.838 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-22T13:12:55.838 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-22T13:12:55.839 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-22T13:12:55.839 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-22T13:12:55.839 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-22T13:12:55.839 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-22T13:12:55.839 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-22T13:12:55.839 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-22T13:12:55.839 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-22T13:12:55.840 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-22T13:12:55.840 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-22T13:12:55.840 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-22T13:12:55.840 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-22T13:12:55.840 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-22T13:12:55.840 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-22T13:12:55.841 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-22T13:12:55.841 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-22T13:12:55.841 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-22T13:12:55.841 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-22T13:12:55.841 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-22T13:12:55.841 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-22T13:12:55.842 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-22T13:12:55.842 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-22T13:12:55.842 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-22T13:12:55.842 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-22T13:12:55.842 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-22T13:12:55.842 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-22T13:12:55.842 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-22T13:12:55.843 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-22T13:12:55.843 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-22T13:12:55.843 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-22T13:12:55.843 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-22T13:12:55.844 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-22T13:12:55.844 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-22T13:12:55.844 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-22T13:12:55.844 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-22T13:12:55.844 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-22T13:12:55.844 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-22T13:12:55.844 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-22T13:12:55.845 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-22T13:12:55.845 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-22T13:12:55.845 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-22T13:12:55.845 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-22T13:12:55.845 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/152 2024-04-22T13:12:55.845 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-22T13:12:55.846 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/152 2024-04-22T13:12:57.884 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-22T13:12:57.885 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-22T13:12:57.885 INFO:teuthology.orchestra.run.smithi111.stdout:Upgraded: 2024-04-22T13:12:57.885 INFO:teuthology.orchestra.run.smithi111.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:57.885 INFO:teuthology.orchestra.run.smithi111.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:57.885 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2024-04-22T13:12:57.885 INFO:teuthology.orchestra.run.smithi111.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-22T13:12:57.885 INFO:teuthology.orchestra.run.smithi111.stdout: automake-1.16.2-8.el9.noarch 2024-04-22T13:12:57.885 INFO:teuthology.orchestra.run.smithi111.stdout: bison-3.7.4-5.el9.x86_64 2024-04-22T13:12:57.885 INFO:teuthology.orchestra.run.smithi111.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-22T13:12:57.885 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:57.885 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:57.885 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:57.886 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:57.886 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:12:57.886 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:57.886 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:57.886 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:57.886 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:12:57.886 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:12:57.886 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:12:57.886 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:12:57.886 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:12:57.886 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:57.886 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:57.887 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:12:57.887 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:57.887 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:57.887 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:57.887 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:12:57.887 INFO:teuthology.orchestra.run.smithi111.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:12:57.887 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:57.887 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:57.887 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-22T13:12:57.887 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-22T13:12:57.887 INFO:teuthology.orchestra.run.smithi111.stdout: flex-2.6.4-9.el9.x86_64 2024-04-22T13:12:57.887 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-22T13:12:57.888 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-22T13:12:57.888 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-22T13:12:57.888 INFO:teuthology.orchestra.run.smithi111.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-22T13:12:57.888 INFO:teuthology.orchestra.run.smithi111.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-22T13:12:57.888 INFO:teuthology.orchestra.run.smithi111.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-22T13:12:57.888 INFO:teuthology.orchestra.run.smithi111.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-22T13:12:57.888 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-22T13:12:57.888 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-22T13:12:57.888 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:57.888 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:57.888 INFO:teuthology.orchestra.run.smithi111.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:57.888 INFO:teuthology.orchestra.run.smithi111.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-22T13:12:57.889 INFO:teuthology.orchestra.run.smithi111.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-22T13:12:57.889 INFO:teuthology.orchestra.run.smithi111.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-22T13:12:57.889 INFO:teuthology.orchestra.run.smithi111.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-22T13:12:57.889 INFO:teuthology.orchestra.run.smithi111.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-22T13:12:57.889 INFO:teuthology.orchestra.run.smithi111.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:57.889 INFO:teuthology.orchestra.run.smithi111.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:57.889 INFO:teuthology.orchestra.run.smithi111.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-22T13:12:57.889 INFO:teuthology.orchestra.run.smithi111.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:57.889 INFO:teuthology.orchestra.run.smithi111.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-22T13:12:57.889 INFO:teuthology.orchestra.run.smithi111.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-22T13:12:57.889 INFO:teuthology.orchestra.run.smithi111.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-22T13:12:57.889 INFO:teuthology.orchestra.run.smithi111.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-22T13:12:57.889 INFO:teuthology.orchestra.run.smithi111.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-22T13:12:57.890 INFO:teuthology.orchestra.run.smithi111.stdout: lua-5.4.4-4.el9.x86_64 2024-04-22T13:12:57.890 INFO:teuthology.orchestra.run.smithi111.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-22T13:12:57.890 INFO:teuthology.orchestra.run.smithi111.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-22T13:12:57.890 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-22T13:12:57.890 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-22T13:12:57.890 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-22T13:12:57.890 INFO:teuthology.orchestra.run.smithi111.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-22T13:12:57.890 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-13.14-1.el9.x86_64 2024-04-22T13:12:57.890 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-22T13:12:57.890 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-22T13:12:57.891 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-22T13:12:57.891 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-22T13:12:57.891 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-22T13:12:57.891 INFO:teuthology.orchestra.run.smithi111.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-22T13:12:57.891 INFO:teuthology.orchestra.run.smithi111.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-22T13:12:57.891 INFO:teuthology.orchestra.run.smithi111.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-22T13:12:57.891 INFO:teuthology.orchestra.run.smithi111.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-22T13:12:57.891 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-22T13:12:57.891 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:57.891 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:57.891 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:57.891 INFO:teuthology.orchestra.run.smithi111.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-22T13:12:57.891 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-22T13:12:57.891 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-22T13:12:57.892 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-22T13:12:57.892 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-22T13:12:57.892 INFO:teuthology.orchestra.run.smithi111.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-22T13:12:57.892 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-22T13:12:57.892 INFO:teuthology.orchestra.run.smithi111.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-22T13:12:57.892 INFO:teuthology.orchestra.run.smithi111.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-22T13:12:57.892 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-22T13:12:57.892 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-22T13:12:57.892 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-22T13:12:57.892 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-22T13:12:57.892 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-22T13:12:57.892 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-22T13:12:57.892 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-22T13:12:57.892 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-22T13:12:57.893 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-22T13:12:57.893 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-22T13:12:57.893 INFO:teuthology.orchestra.run.smithi111.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-22T13:12:57.893 INFO:teuthology.orchestra.run.smithi111.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-22T13:12:57.893 INFO:teuthology.orchestra.run.smithi111.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-22T13:12:57.893 INFO:teuthology.orchestra.run.smithi111.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-22T13:12:57.893 INFO:teuthology.orchestra.run.smithi111.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-22T13:12:57.893 INFO:teuthology.orchestra.run.smithi111.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-22T13:12:57.893 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-22T13:12:57.893 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-22T13:12:57.893 INFO:teuthology.orchestra.run.smithi111.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-22T13:12:57.893 INFO:teuthology.orchestra.run.smithi111.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-22T13:12:57.893 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-22T13:12:57.893 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ply-3.11-14.el9.noarch 2024-04-22T13:12:57.894 INFO:teuthology.orchestra.run.smithi111.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-22T13:12:57.894 INFO:teuthology.orchestra.run.smithi111.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-22T13:12:57.894 INFO:teuthology.orchestra.run.smithi111.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-22T13:12:57.894 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-22T13:12:57.894 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-22T13:12:57.894 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-22T13:12:57.894 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-22T13:12:57.894 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-22T13:12:57.894 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-22T13:12:57.894 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:57.894 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:57.894 INFO:teuthology.orchestra.run.smithi111.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-22T13:12:57.894 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-22T13:12:57.894 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-22T13:12:57.895 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:57.895 INFO:teuthology.orchestra.run.smithi111.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-22T13:12:57.895 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-22T13:12:57.895 INFO:teuthology.orchestra.run.smithi111.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-22T13:12:57.895 INFO:teuthology.orchestra.run.smithi111.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-22T13:12:57.895 INFO:teuthology.orchestra.run.smithi111.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-22T13:12:57.895 INFO:teuthology.orchestra.run.smithi111.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-22T13:12:57.895 INFO:teuthology.orchestra.run.smithi111.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-22T13:12:57.895 INFO:teuthology.orchestra.run.smithi111.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-22T13:12:57.895 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-22T13:12:57.895 INFO:teuthology.orchestra.run.smithi111.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-22T13:12:57.895 INFO:teuthology.orchestra.run.smithi111.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-22T13:12:57.895 INFO:teuthology.orchestra.run.smithi111.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-22T13:12:57.895 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-22T13:12:57.896 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-22T13:12:57.896 INFO:teuthology.orchestra.run.smithi111.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-22T13:12:57.896 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:57.896 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:57.896 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-22T13:12:57.896 INFO:teuthology.orchestra.run.smithi111.stdout: re2-1:20211101-3.el9.x86_64 2024-04-22T13:12:57.896 INFO:teuthology.orchestra.run.smithi111.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-22T13:12:57.896 INFO:teuthology.orchestra.run.smithi111.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-22T13:12:57.896 INFO:teuthology.orchestra.run.smithi111.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-22T13:12:57.896 INFO:teuthology.orchestra.run.smithi111.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-22T13:12:57.896 INFO:teuthology.orchestra.run.smithi111.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-22T13:12:57.897 INFO:teuthology.orchestra.run.smithi111.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-22T13:12:57.897 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-04-22T13:12:57.897 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-04-22T13:12:58.295 DEBUG:teuthology.parallel:result is None 2024-04-22T13:12:58.296 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-22T13:12:58.451 DEBUG:teuthology.orchestra.run.smithi012:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-22T13:12:58.478 INFO:teuthology.orchestra.run.smithi012.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-22T13:12:58.479 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-22T13:12:58.479 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-22T13:12:58.481 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-22T13:12:58.623 DEBUG:teuthology.orchestra.run.smithi111:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-22T13:12:58.643 INFO:teuthology.orchestra.run.smithi111.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-22T13:12:58.643 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-22T13:12:58.643 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-22T13:12:58.645 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-22T13:12:58.905 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-22T13:12:58.931 INFO:teuthology.orchestra.run.smithi134.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-22T13:12:58.932 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-22T13:12:58.932 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-22T13:12:58.934 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-22T13:12:58.934 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-22T13:12:58.934 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-22T13:12:58.967 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-22T13:12:58.967 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-22T13:12:58.993 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-22T13:12:58.993 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-22T13:12:59.024 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-22T13:12:59.025 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-22T13:12:59.025 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/daemon-helper 2024-04-22T13:12:59.053 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-22T13:12:59.120 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-22T13:12:59.120 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/daemon-helper 2024-04-22T13:12:59.144 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-22T13:12:59.207 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-22T13:12:59.208 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/daemon-helper 2024-04-22T13:12:59.235 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-22T13:12:59.303 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-22T13:12:59.303 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-22T13:12:59.303 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-22T13:12:59.331 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-22T13:12:59.397 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-22T13:12:59.397 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-22T13:12:59.421 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-22T13:12:59.484 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-22T13:12:59.484 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-22T13:12:59.507 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-22T13:12:59.571 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-22T13:12:59.571 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-22T13:12:59.571 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/stdin-killer 2024-04-22T13:12:59.596 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-22T13:12:59.660 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-22T13:12:59.661 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/stdin-killer 2024-04-22T13:12:59.683 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-22T13:12:59.747 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-22T13:12:59.747 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/stdin-killer 2024-04-22T13:12:59.771 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-22T13:12:59.836 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-22T13:12:59.957 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-22T13:12:59.957 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-22T13:12:59.957 INFO:tasks.cephadm:Cluster fsid is 0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:12:59.958 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-22T13:12:59.958 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.12', 'mon.b': '172.21.15.111', 'mon.c': '172.21.15.134'} 2024-04-22T13:12:59.958 INFO:tasks.cephadm:First mon is mon.a on smithi012 2024-04-22T13:12:59.958 INFO:tasks.cephadm:First mgr is x 2024-04-22T13:12:59.958 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-22T13:12:59.958 DEBUG:teuthology.orchestra.run.smithi012:> sudo hostname $(hostname -s) 2024-04-22T13:12:59.984 DEBUG:teuthology.orchestra.run.smithi111:> sudo hostname $(hostname -s) 2024-04-22T13:13:00.008 DEBUG:teuthology.orchestra.run.smithi134:> sudo hostname $(hostname -s) 2024-04-22T13:13:00.031 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-22T13:13:00.031 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-22T13:13:00.333 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78795/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3156-gd69f13d8', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3156.gd69f13d8'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-22 12:00:06.590653', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-22T13:13:00.481 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 d69f13d827952bb07bb9b7973bca14f97d764d6f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-22T13:13:00.483 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm 2024-04-22T13:13:00.483 DEBUG:teuthology.orchestra.run.smithi012:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-22T13:13:02.321 INFO:teuthology.orchestra.run.smithi012.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 22 13:13 /home/ubuntu/cephtest/cephadm 2024-04-22T13:13:02.322 DEBUG:teuthology.orchestra.run.smithi111:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-22T13:13:04.018 INFO:teuthology.orchestra.run.smithi111.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 22 13:13 /home/ubuntu/cephtest/cephadm 2024-04-22T13:13:04.019 DEBUG:teuthology.orchestra.run.smithi134:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-22T13:13:05.291 INFO:teuthology.orchestra.run.smithi134.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 22 13:13 /home/ubuntu/cephtest/cephadm 2024-04-22T13:13:05.291 DEBUG:teuthology.orchestra.run.smithi012:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-22T13:13:05.311 DEBUG:teuthology.orchestra.run.smithi111:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-22T13:13:05.329 DEBUG:teuthology.orchestra.run.smithi134:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-22T13:13:05.365 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f on all hosts... 2024-04-22T13:13:05.365 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-22T13:13:05.367 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-22T13:13:05.371 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-22T13:13:06.787 INFO:teuthology.orchestra.run.smithi134.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-22T13:13:06.941 INFO:teuthology.orchestra.run.smithi012.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-22T13:13:07.121 INFO:teuthology.orchestra.run.smithi111.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-22T13:13:47.283 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-04-22T13:13:47.284 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-22T13:13:47.284 INFO:teuthology.orchestra.run.smithi111.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-22T13:13:47.284 INFO:teuthology.orchestra.run.smithi111.stdout: "repo_digests": [ 2024-04-22T13:13:47.284 INFO:teuthology.orchestra.run.smithi111.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-22T13:13:47.284 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-04-22T13:13:47.284 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-04-22T13:13:48.444 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-04-22T13:13:48.445 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-22T13:13:48.445 INFO:teuthology.orchestra.run.smithi134.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-22T13:13:48.445 INFO:teuthology.orchestra.run.smithi134.stdout: "repo_digests": [ 2024-04-22T13:13:48.445 INFO:teuthology.orchestra.run.smithi134.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-22T13:13:48.445 INFO:teuthology.orchestra.run.smithi134.stdout: ] 2024-04-22T13:13:48.445 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-04-22T13:13:48.466 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:13:48.466 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-22T13:13:48.466 INFO:teuthology.orchestra.run.smithi012.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-22T13:13:48.466 INFO:teuthology.orchestra.run.smithi012.stdout: "repo_digests": [ 2024-04-22T13:13:48.467 INFO:teuthology.orchestra.run.smithi012.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-22T13:13:48.467 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-22T13:13:48.467 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:13:48.493 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /etc/ceph 2024-04-22T13:13:48.527 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /etc/ceph 2024-04-22T13:13:48.560 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /etc/ceph 2024-04-22T13:13:48.586 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 777 /etc/ceph 2024-04-22T13:13:48.611 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 777 /etc/ceph 2024-04-22T13:13:48.637 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 777 /etc/ceph 2024-04-22T13:13:48.660 INFO:tasks.cephadm:Writing seed config... 2024-04-22T13:13:48.661 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-22T13:13:48.661 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-22T13:13:48.661 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-22T13:13:48.661 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-22T13:13:48.661 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-22T13:13:48.661 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-22T13:13:48.662 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-22T13:13:48.662 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-22T13:13:48.662 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-22T13:13:48.662 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-22T13:13:48.662 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-22T13:13:48.662 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-22T13:13:48.662 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-22T13:13:48.662 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-22T13:13:48.662 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-22T13:13:48.662 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-22T13:13:48.662 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-22T13:13:48.662 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-22T13:13:48.662 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-22T13:13:48.663 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-22T13:13:48.663 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-22T13:13:48.663 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-22T13:13:48.663 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-22T13:13:48.663 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-22T13:13:48.663 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-22T13:13:48.663 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-22T13:13:48.663 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-22T13:13:48.663 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-22T13:13:48.663 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-22T13:13:48.663 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-22T13:13:48.663 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-22T13:13:48.663 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-22T13:13:48.663 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-22T13:13:48.663 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-22T13:13:48.664 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-22T13:13:48.664 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-22T13:13:48.664 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-22T13:13:48.664 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-22T13:13:48.664 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-22T13:13:48.664 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-22T13:13:48.664 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-22T13:13:48.664 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-22T13:13:48.664 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-22T13:13:48.664 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-22T13:13:48.664 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-22T13:13:48.664 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-22T13:13:48.664 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-22T13:13:48.665 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-22T13:13:48.665 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-22T13:13:48.665 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-22T13:13:48.665 DEBUG:teuthology.orchestra.run.smithi012:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-22T13:13:48.682 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 0aede5ea-00aa-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-22T13:13:48.682 DEBUG:teuthology.orchestra.run.smithi012:mon.a> sudo journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mon.a.service 2024-04-22T13:13:48.724 DEBUG:teuthology.orchestra.run.smithi012:mgr.x> sudo journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mgr.x.service 2024-04-22T13:13:48.767 INFO:tasks.cephadm:Bootstrapping... 2024-04-22T13:13:48.768 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f -v bootstrap --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.12 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-22T13:13:48.964 INFO:teuthology.orchestra.run.smithi012.stdout:-------------------------------------------------------------------------------- 2024-04-22T13:13:48.965 INFO:teuthology.orchestra.run.smithi012.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f', '-v', 'bootstrap', '--fsid', '0aede5ea-00aa-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.12', '--skip-admin-label'] 2024-04-22T13:13:48.965 INFO:teuthology.orchestra.run.smithi012.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-22T13:13:48.965 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying podman|docker is present... 2024-04-22T13:13:48.991 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-22T13:13:48.991 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying lvm2 is present... 2024-04-22T13:13:48.991 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying time synchronization is in place... 2024-04-22T13:13:48.999 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-22T13:13:48.999 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-22T13:13:49.005 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-22T13:13:49.005 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-04-22T13:13:49.011 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout enabled 2024-04-22T13:13:49.017 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout active 2024-04-22T13:13:49.018 INFO:teuthology.orchestra.run.smithi012.stdout:Unit chronyd.service is enabled and running 2024-04-22T13:13:49.018 INFO:teuthology.orchestra.run.smithi012.stdout:Repeating the final host check... 2024-04-22T13:13:49.041 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-22T13:13:49.041 INFO:teuthology.orchestra.run.smithi012.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-22T13:13:49.042 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl is present 2024-04-22T13:13:49.042 INFO:teuthology.orchestra.run.smithi012.stdout:lvcreate is present 2024-04-22T13:13:49.048 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-22T13:13:49.048 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-22T13:13:49.054 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-22T13:13:49.054 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-04-22T13:13:49.060 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout enabled 2024-04-22T13:13:49.067 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout active 2024-04-22T13:13:49.067 INFO:teuthology.orchestra.run.smithi012.stdout:Unit chronyd.service is enabled and running 2024-04-22T13:13:49.067 INFO:teuthology.orchestra.run.smithi012.stdout:Host looks OK 2024-04-22T13:13:49.067 INFO:teuthology.orchestra.run.smithi012.stdout:Cluster fsid: 0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:13:49.067 INFO:teuthology.orchestra.run.smithi012.stdout:Acquiring lock 139998437899424 on /run/cephadm/0aede5ea-00aa-11ef-bc93-c7b262605968.lock 2024-04-22T13:13:49.067 INFO:teuthology.orchestra.run.smithi012.stdout:Lock 139998437899424 acquired on /run/cephadm/0aede5ea-00aa-11ef-bc93-c7b262605968.lock 2024-04-22T13:13:49.067 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying IP 172.21.15.12 port 3300 ... 2024-04-22T13:13:49.068 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying IP 172.21.15.12 port 6789 ... 2024-04-22T13:13:49.068 INFO:teuthology.orchestra.run.smithi012.stdout:Base mon IP(s) is [172.21.15.12:3300, 172.21.15.12:6789], mon addrv is [v2:172.21.15.12:3300,v1:172.21.15.12:6789] 2024-04-22T13:13:49.071 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.12 metric 100 2024-04-22T13:13:49.071 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.12 metric 100 2024-04-22T13:13:49.073 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-22T13:13:49.073 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-22T13:13:49.073 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-22T13:13:49.077 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-22T13:13:49.077 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-22T13:13:49.077 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-22T13:13:49.077 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-22T13:13:49.077 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:25ea/64 scope link noprefixroute 2024-04-22T13:13:49.077 INFO:teuthology.orchestra.run.smithi012.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-22T13:13:49.077 INFO:teuthology.orchestra.run.smithi012.stdout:Mon IP `172.21.15.12` is in CIDR network `172.21.0.0/20` 2024-04-22T13:13:49.077 INFO:teuthology.orchestra.run.smithi012.stdout:Mon IP `172.21.15.12` is in CIDR network `172.21.0.0/20` 2024-04-22T13:13:49.077 INFO:teuthology.orchestra.run.smithi012.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-22T13:13:49.078 INFO:teuthology.orchestra.run.smithi012.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-22T13:13:49.078 INFO:teuthology.orchestra.run.smithi012.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-22T13:13:49.535 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stdout da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-22T13:13:49.535 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-22T13:13:49.535 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Getting image source signatures 2024-04-22T13:13:49.535 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Copying blob sha256:7f03586bcf9c6a2b4d690c118c4aa5b771f9c6f252d75a20fb283dd07093a738 2024-04-22T13:13:49.535 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-22T13:13:49.535 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Copying config sha256:da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-22T13:13:49.535 INFO:teuthology.orchestra.run.smithi012.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-22T13:13:50.463 INFO:teuthology.orchestra.run.smithi012.stdout:ceph: stdout ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-22T13:13:50.464 INFO:teuthology.orchestra.run.smithi012.stdout:Ceph version: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-22T13:13:50.464 INFO:teuthology.orchestra.run.smithi012.stdout:Extracting ceph user uid/gid from container image... 2024-04-22T13:13:59.902 INFO:teuthology.orchestra.run.smithi012.stdout:stat: stdout 167 167 2024-04-22T13:13:59.902 INFO:teuthology.orchestra.run.smithi012.stdout:Creating initial keys... 2024-04-22T13:14:03.114 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph-authtool: stdout AQCaYiZmO8PRIhAA0hkgLV4g2cnWmeHkwXX+iA== 2024-04-22T13:14:04.073 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph-authtool: stdout AQCbYiZmypD8GBAAlfbVvKyXIQ07UV3FUb1IBA== 2024-04-22T13:14:04.980 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph-authtool: stdout AQCcYiZmo878FxAAMtkwQGSRMyu9p5KaUgKcXg== 2024-04-22T13:14:04.981 INFO:teuthology.orchestra.run.smithi012.stdout:Creating initial monmap... 2024-04-22T13:14:05.947 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-22T13:14:05.947 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-22T13:14:05.947 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:14:05.947 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-22T13:14:05.947 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool for a [v2:172.21.15.12:3300,v1:172.21.15.12:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-22T13:14:05.947 INFO:teuthology.orchestra.run.smithi012.stdout:setting min_mon_release = quincy 2024-04-22T13:14:05.947 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: set fsid to 0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:14:05.947 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-22T13:14:05.947 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:14:05.947 INFO:teuthology.orchestra.run.smithi012.stdout:Creating mon... 2024-04-22T13:14:07.261 INFO:teuthology.orchestra.run.smithi012.stdout:create mon.a on 2024-04-22T13:14:07.485 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-22T13:14:07.637 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-22T13:14:07.921 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-0aede5ea-00aa-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-0aede5ea-00aa-11ef-bc93-c7b262605968.target. 2024-04-22T13:14:07.921 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-0aede5ea-00aa-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-0aede5ea-00aa-11ef-bc93-c7b262605968.target. 2024-04-22T13:14:08.654 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mon.a 2024-04-22T13:14:08.654 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-22T13:14:08.842 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0aede5ea-00aa-11ef-bc93-c7b262605968.target.wants/ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@.service. 2024-04-22T13:14:09.408 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-22T13:14:09.408 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-04-22T13:14:09.415 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-22T13:14:09.415 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-04-22T13:14:09.416 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-04-22T13:14:09.416 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to enable service . firewalld.service is not available 2024-04-22T13:14:09.416 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mon to start... 2024-04-22T13:14:09.416 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mon... 2024-04-22T13:14:09.541 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:09 smithi012 systemd[1]: Started Ceph mon.a for 0aede5ea-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:14:10.504 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout cluster: 2024-04-22T13:14:10.504 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout id: 0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:14:10.504 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-22T13:14:10.504 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-04-22T13:14:10.504 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout services: 2024-04-22T13:14:10.504 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.580493s) 2024-04-22T13:14:10.504 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-22T13:14:10.504 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-22T13:14:10.504 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-04-22T13:14:10.505 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout data: 2024-04-22T13:14:10.505 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-22T13:14:10.505 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-22T13:14:10.505 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-22T13:14:10.505 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout pgs: 2024-04-22T13:14:10.505 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-04-22T13:14:10.505 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.858+0000 7f59ebeb1700 1 Processor -- start 2024-04-22T13:14:10.505 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.858+0000 7f59ebeb1700 1 -- start start 2024-04-22T13:14:10.505 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.858+0000 7f59ebeb1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e41090a0 0x7f59e410b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:10.505 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.859+0000 7f59ebeb1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f59e407bde0 con 0x7f59e41090a0 2024-04-22T13:14:10.505 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.859+0000 7f59e9c4d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e41090a0 0x7f59e410b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:10.505 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.859+0000 7f59e9c4d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e41090a0 0x7f59e410b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58116/0 (socket says 172.21.15.12:58116) 2024-04-22T13:14:10.505 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.859+0000 7f59e9c4d700 1 -- 172.21.15.12:0/4009710731 learned_addr learned my addr 172.21.15.12:0/4009710731 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:10.505 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.859+0000 7f59e9c4d700 1 -- 172.21.15.12:0/4009710731 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e410b9d0 con 0x7f59e41090a0 2024-04-22T13:14:10.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.860+0000 7f59e9c4d700 1 --2- 172.21.15.12:0/4009710731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e41090a0 0x7f59e410b490 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f59e00098a0 tx=0x7f59e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a55bde7bcf862cb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:10.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.860+0000 7f59e8c4b700 1 -- 172.21.15.12:0/4009710731 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59e0004160 con 0x7f59e41090a0 2024-04-22T13:14:10.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.860+0000 7f59e8c4b700 1 -- 172.21.15.12:0/4009710731 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59e0004300 con 0x7f59e41090a0 2024-04-22T13:14:10.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.860+0000 7f59e8c4b700 1 -- 172.21.15.12:0/4009710731 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59e00036a0 con 0x7f59e41090a0 2024-04-22T13:14:10.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.860+0000 7f59ebeb1700 1 -- 172.21.15.12:0/4009710731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e41090a0 msgr2=0x7f59e410b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:10.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.860+0000 7f59ebeb1700 1 --2- 172.21.15.12:0/4009710731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e41090a0 0x7f59e410b490 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f59e00098a0 tx=0x7f59e0009bb0 comp rx=0 tx=0).stop 2024-04-22T13:14:10.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.860+0000 7f59ebeb1700 1 -- 172.21.15.12:0/4009710731 shutdown_connections 2024-04-22T13:14:10.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.860+0000 7f59ebeb1700 1 --2- 172.21.15.12:0/4009710731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e41090a0 0x7f59e410b490 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:10.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.860+0000 7f59ebeb1700 1 -- 172.21.15.12:0/4009710731 >> 172.21.15.12:0/4009710731 conn(0x7f59e4102a50 msgr2=0x7f59e4104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:10.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.861+0000 7f59ebeb1700 1 -- 172.21.15.12:0/4009710731 shutdown_connections 2024-04-22T13:14:10.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.861+0000 7f59ebeb1700 1 -- 172.21.15.12:0/4009710731 wait complete. 2024-04-22T13:14:10.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.861+0000 7f59ebeb1700 1 Processor -- start 2024-04-22T13:14:10.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.861+0000 7f59ebeb1700 1 -- start start 2024-04-22T13:14:10.507 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.862+0000 7f59ebeb1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e41090a0 0x7f59e419bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:10.507 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.862+0000 7f59ebeb1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f59e410bde0 con 0x7f59e41090a0 2024-04-22T13:14:10.507 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.862+0000 7f59e9c4d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e41090a0 0x7f59e419bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:10.507 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.862+0000 7f59e9c4d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e41090a0 0x7f59e419bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58124/0 (socket says 172.21.15.12:58124) 2024-04-22T13:14:10.507 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.862+0000 7f59e9c4d700 1 -- 172.21.15.12:0/3209368497 learned_addr learned my addr 172.21.15.12:0/3209368497 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:10.507 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.862+0000 7f59e9c4d700 1 -- 172.21.15.12:0/3209368497 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e419c1b0 con 0x7f59e41090a0 2024-04-22T13:14:10.507 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.862+0000 7f59e9c4d700 1 --2- 172.21.15.12:0/3209368497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e41090a0 0x7f59e419bc70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f59e0009fb0 tx=0x7f59e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:10.507 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.862+0000 7f59d6ffd700 1 -- 172.21.15.12:0/3209368497 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59e0003de0 con 0x7f59e41090a0 2024-04-22T13:14:10.507 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.862+0000 7f59d6ffd700 1 -- 172.21.15.12:0/3209368497 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59e0011630 con 0x7f59e41090a0 2024-04-22T13:14:10.507 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.862+0000 7f59ebeb1700 1 -- 172.21.15.12:0/3209368497 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59e419c440 con 0x7f59e41090a0 2024-04-22T13:14:10.507 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.862+0000 7f59d6ffd700 1 -- 172.21.15.12:0/3209368497 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59e0011930 con 0x7f59e41090a0 2024-04-22T13:14:10.507 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.862+0000 7f59ebeb1700 1 -- 172.21.15.12:0/3209368497 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59e419f130 con 0x7f59e41090a0 2024-04-22T13:14:10.507 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.863+0000 7f59d6ffd700 1 -- 172.21.15.12:0/3209368497 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f59e0024480 con 0x7f59e41090a0 2024-04-22T13:14:10.507 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.863+0000 7f59d6ffd700 1 -- 172.21.15.12:0/3209368497 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f59e001a7a0 con 0x7f59e41090a0 2024-04-22T13:14:10.508 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.863+0000 7f59ebeb1700 1 -- 172.21.15.12:0/3209368497 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59e40640a0 con 0x7f59e41090a0 2024-04-22T13:14:10.508 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.866+0000 7f59d6ffd700 1 -- 172.21.15.12:0/3209368497 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f59e0033080 con 0x7f59e41090a0 2024-04-22T13:14:10.508 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.941+0000 7f59ebeb1700 1 -- 172.21.15.12:0/3209368497 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f59e402e1b0 con 0x7f59e41090a0 2024-04-22T13:14:10.508 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.941+0000 7f59d6ffd700 1 -- 172.21.15.12:0/3209368497 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f59e001a9f0 con 0x7f59e41090a0 2024-04-22T13:14:10.508 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.942+0000 7f59ebeb1700 1 -- 172.21.15.12:0/3209368497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e41090a0 msgr2=0x7f59e419bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:10.508 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.942+0000 7f59ebeb1700 1 --2- 172.21.15.12:0/3209368497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e41090a0 0x7f59e419bc70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f59e0009fb0 tx=0x7f59e0004750 comp rx=0 tx=0).stop 2024-04-22T13:14:10.508 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.942+0000 7f59ebeb1700 1 -- 172.21.15.12:0/3209368497 shutdown_connections 2024-04-22T13:14:10.508 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.942+0000 7f59ebeb1700 1 --2- 172.21.15.12:0/3209368497 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f59e41090a0 0x7f59e419bc70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:10.508 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.942+0000 7f59ebeb1700 1 -- 172.21.15.12:0/3209368497 >> 172.21.15.12:0/3209368497 conn(0x7f59e4102a50 msgr2=0x7f59e4103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:10.508 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.942+0000 7f59ebeb1700 1 -- 172.21.15.12:0/3209368497 shutdown_connections 2024-04-22T13:14:10.508 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:09.942+0000 7f59ebeb1700 1 -- 172.21.15.12:0/3209368497 wait complete. 2024-04-22T13:14:10.508 INFO:teuthology.orchestra.run.smithi012.stdout:mon is available 2024-04-22T13:14:10.508 INFO:teuthology.orchestra.run.smithi012.stdout:Assimilating anything we can from ceph.conf... 2024-04-22T13:14:10.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:10 smithi012 ceph-mon[27886]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-22T13:14:10.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:10 smithi012 ceph-mon[27886]: monmap epoch 1 2024-04-22T13:14:10.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:10 smithi012 ceph-mon[27886]: fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:14:10.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:10 smithi012 ceph-mon[27886]: last_changed 2024-04-22T13:14:05.291228+0000 2024-04-22T13:14:10.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:10 smithi012 ceph-mon[27886]: created 2024-04-22T13:14:05.291228+0000 2024-04-22T13:14:10.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:10 smithi012 ceph-mon[27886]: min_mon_release 19 (squid) 2024-04-22T13:14:10.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:10 smithi012 ceph-mon[27886]: election_strategy: 1 2024-04-22T13:14:10.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:10 smithi012 ceph-mon[27886]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-04-22T13:14:10.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:10 smithi012 ceph-mon[27886]: fsmap 2024-04-22T13:14:10.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:10 smithi012 ceph-mon[27886]: osdmap e1: 0 total, 0 up, 0 in 2024-04-22T13:14:10.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:10 smithi012 ceph-mon[27886]: mgrmap e1: no daemons active 2024-04-22T13:14:10.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:10 smithi012 ceph-mon[27886]: from='client.? 172.21.15.12:0/3209368497' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-22T13:14:11.584 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:11 smithi012 ceph-mon[27886]: from='client.? 172.21.15.12:0/2946297482' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-22T13:14:11.584 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:11 smithi012 ceph-mon[27886]: from='client.? 172.21.15.12:0/2946297482' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-22T13:14:11.594 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-04-22T13:14:11.594 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [global] 2024-04-22T13:14:11.594 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout fsid = 0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:14:11.594 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-22T13:14:11.594 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.12:3300,v1:172.21.15.12:6789] 2024-04-22T13:14:11.594 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-22T13:14:11.594 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-22T13:14:11.594 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-22T13:14:11.594 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-22T13:14:11.595 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-04-22T13:14:11.595 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-22T13:14:11.595 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-22T13:14:11.595 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-04-22T13:14:11.595 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [osd] 2024-04-22T13:14:11.595 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-22T13:14:11.595 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-22T13:14:11.595 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-22T13:14:11.595 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-22T13:14:11.595 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-22T13:14:11.595 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-22T13:14:11.595 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.918+0000 7fdf0edda700 1 Processor -- start 2024-04-22T13:14:11.595 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.918+0000 7fdf0edda700 1 -- start start 2024-04-22T13:14:11.596 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.918+0000 7fdf0edda700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf08108560 0x7fdf08108980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:11.596 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.918+0000 7fdf0edda700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdf08108f50 con 0x7fdf08108560 2024-04-22T13:14:11.596 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.918+0000 7fdf0cb76700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf08108560 0x7fdf08108980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:11.596 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.918+0000 7fdf0cb76700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf08108560 0x7fdf08108980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58140/0 (socket says 172.21.15.12:58140) 2024-04-22T13:14:11.596 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.918+0000 7fdf0cb76700 1 -- 172.21.15.12:0/34207101 learned_addr learned my addr 172.21.15.12:0/34207101 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:11.596 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.919+0000 7fdf0cb76700 1 -- 172.21.15.12:0/34207101 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf08109710 con 0x7fdf08108560 2024-04-22T13:14:11.596 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.919+0000 7fdf0cb76700 1 --2- 172.21.15.12:0/34207101 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf08108560 0x7fdf08108980 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdefc0098a0 tx=0x7fdefc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de229b7ceb955d7a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:11.596 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.919+0000 7fdf077fe700 1 -- 172.21.15.12:0/34207101 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdefc004160 con 0x7fdf08108560 2024-04-22T13:14:11.596 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.919+0000 7fdf077fe700 1 -- 172.21.15.12:0/34207101 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdefc004300 con 0x7fdf08108560 2024-04-22T13:14:11.596 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.919+0000 7fdf077fe700 1 -- 172.21.15.12:0/34207101 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdefc0036a0 con 0x7fdf08108560 2024-04-22T13:14:11.596 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.919+0000 7fdf0edda700 1 -- 172.21.15.12:0/34207101 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf08108560 msgr2=0x7fdf08108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:11.596 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.919+0000 7fdf0edda700 1 --2- 172.21.15.12:0/34207101 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf08108560 0x7fdf08108980 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fdefc0098a0 tx=0x7fdefc009bb0 comp rx=0 tx=0).stop 2024-04-22T13:14:11.596 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.920+0000 7fdf0edda700 1 -- 172.21.15.12:0/34207101 shutdown_connections 2024-04-22T13:14:11.596 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.920+0000 7fdf0edda700 1 --2- 172.21.15.12:0/34207101 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf08108560 0x7fdf08108980 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:11.597 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.920+0000 7fdf0edda700 1 -- 172.21.15.12:0/34207101 >> 172.21.15.12:0/34207101 conn(0x7fdf08103b40 msgr2=0x7fdf08105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:11.597 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.920+0000 7fdf0edda700 1 -- 172.21.15.12:0/34207101 shutdown_connections 2024-04-22T13:14:11.597 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.920+0000 7fdf0edda700 1 -- 172.21.15.12:0/34207101 wait complete. 2024-04-22T13:14:11.597 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.920+0000 7fdf0edda700 1 Processor -- start 2024-04-22T13:14:11.597 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.920+0000 7fdf0edda700 1 -- start start 2024-04-22T13:14:11.597 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.920+0000 7fdf0edda700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf08108560 0x7fdf0819bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:11.597 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.920+0000 7fdf0edda700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdf08109c40 con 0x7fdf08108560 2024-04-22T13:14:11.597 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.921+0000 7fdf0cb76700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf08108560 0x7fdf0819bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:11.597 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.921+0000 7fdf0cb76700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf08108560 0x7fdf0819bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58148/0 (socket says 172.21.15.12:58148) 2024-04-22T13:14:11.597 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.921+0000 7fdf0cb76700 1 -- 172.21.15.12:0/2946297482 learned_addr learned my addr 172.21.15.12:0/2946297482 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:11.597 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.921+0000 7fdf0cb76700 1 -- 172.21.15.12:0/2946297482 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf0819c1e0 con 0x7fdf08108560 2024-04-22T13:14:11.597 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.921+0000 7fdf0cb76700 1 --2- 172.21.15.12:0/2946297482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf08108560 0x7fdf0819bca0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fdefc000c00 tx=0x7fdefc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:11.597 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.921+0000 7fdf05ffb700 1 -- 172.21.15.12:0/2946297482 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdefc003db0 con 0x7fdf08108560 2024-04-22T13:14:11.597 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.921+0000 7fdf05ffb700 1 -- 172.21.15.12:0/2946297482 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdefc011630 con 0x7fdf08108560 2024-04-22T13:14:11.598 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.921+0000 7fdf0edda700 1 -- 172.21.15.12:0/2946297482 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf0819c470 con 0x7fdf08108560 2024-04-22T13:14:11.598 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.921+0000 7fdf05ffb700 1 -- 172.21.15.12:0/2946297482 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdefc011950 con 0x7fdf08108560 2024-04-22T13:14:11.598 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.921+0000 7fdf0edda700 1 -- 172.21.15.12:0/2946297482 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf0819f160 con 0x7fdf08108560 2024-04-22T13:14:11.598 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.921+0000 7fdf05ffb700 1 -- 172.21.15.12:0/2946297482 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdefc01a460 con 0x7fdf08108560 2024-04-22T13:14:11.598 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.922+0000 7fdf05ffb700 1 -- 172.21.15.12:0/2946297482 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdefc01b780 con 0x7fdf08108560 2024-04-22T13:14:11.598 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.922+0000 7fdf0edda700 1 -- 172.21.15.12:0/2946297482 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf080640a0 con 0x7fdf08108560 2024-04-22T13:14:11.598 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.925+0000 7fdf05ffb700 1 -- 172.21.15.12:0/2946297482 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fdefc031020 con 0x7fdf08108560 2024-04-22T13:14:11.598 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.995+0000 7fdf0edda700 1 -- 172.21.15.12:0/2946297482 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fdf0802e1f0 con 0x7fdf08108560 2024-04-22T13:14:11.598 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.998+0000 7fdf05ffb700 1 -- 172.21.15.12:0/2946297482 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+492 (secure 0 0 0) 0x7fdefc01b9d0 con 0x7fdf08108560 2024-04-22T13:14:11.598 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.999+0000 7fdf05ffb700 1 -- 172.21.15.12:0/2946297482 <== mon.0 v2:172.21.15.12:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fdefc01ad20 con 0x7fdf08108560 2024-04-22T13:14:11.598 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.999+0000 7fdf0edda700 1 -- 172.21.15.12:0/2946297482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf08108560 msgr2=0x7fdf0819bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:11.598 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.999+0000 7fdf0edda700 1 --2- 172.21.15.12:0/2946297482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf08108560 0x7fdf0819bca0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fdefc000c00 tx=0x7fdefc004750 comp rx=0 tx=0).stop 2024-04-22T13:14:11.598 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.999+0000 7fdf0edda700 1 -- 172.21.15.12:0/2946297482 shutdown_connections 2024-04-22T13:14:11.599 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.999+0000 7fdf0edda700 1 --2- 172.21.15.12:0/2946297482 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdf08108560 0x7fdf0819bca0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:11.599 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:10.999+0000 7fdf0edda700 1 -- 172.21.15.12:0/2946297482 >> 172.21.15.12:0/2946297482 conn(0x7fdf08103b40 msgr2=0x7fdf081928c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:11.599 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:11.000+0000 7fdf0edda700 1 -- 172.21.15.12:0/2946297482 shutdown_connections 2024-04-22T13:14:11.599 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:11.000+0000 7fdf0edda700 1 -- 172.21.15.12:0/2946297482 wait complete. 2024-04-22T13:14:11.599 INFO:teuthology.orchestra.run.smithi012.stdout:Generating new minimal ceph.conf... 2024-04-22T13:14:12.634 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:12 smithi012 ceph-mon[27886]: from='client.? 172.21.15.12:0/1187907184' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:14:12.641 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.008+0000 7fec4f7c9700 1 Processor -- start 2024-04-22T13:14:12.641 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.008+0000 7fec4f7c9700 1 -- start start 2024-04-22T13:14:12.641 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.008+0000 7fec4f7c9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec4807de80 0x7fec4807c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:12.641 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.008+0000 7fec4f7c9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fec4807c900 con 0x7fec4807de80 2024-04-22T13:14:12.641 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.008+0000 7fec4d565700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec4807de80 0x7fec4807c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:12.641 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.008+0000 7fec4d565700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec4807de80 0x7fec4807c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46076/0 (socket says 172.21.15.12:46076) 2024-04-22T13:14:12.642 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.008+0000 7fec4d565700 1 -- 172.21.15.12:0/3427559330 learned_addr learned my addr 172.21.15.12:0/3427559330 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:12.642 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.008+0000 7fec4d565700 1 -- 172.21.15.12:0/3427559330 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4807ca80 con 0x7fec4807de80 2024-04-22T13:14:12.642 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.009+0000 7fec4d565700 1 --2- 172.21.15.12:0/3427559330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec4807de80 0x7fec4807c330 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fec440098a0 tx=0x7fec44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b34572685ba31c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:12.642 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.009+0000 7fec3ffff700 1 -- 172.21.15.12:0/3427559330 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec44004160 con 0x7fec4807de80 2024-04-22T13:14:12.642 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.009+0000 7fec3ffff700 1 -- 172.21.15.12:0/3427559330 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fec44004300 con 0x7fec4807de80 2024-04-22T13:14:12.642 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.009+0000 7fec3ffff700 1 -- 172.21.15.12:0/3427559330 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec44011af0 con 0x7fec4807de80 2024-04-22T13:14:12.642 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.009+0000 7fec4f7c9700 1 -- 172.21.15.12:0/3427559330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec4807de80 msgr2=0x7fec4807c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:12.642 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.009+0000 7fec4f7c9700 1 --2- 172.21.15.12:0/3427559330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec4807de80 0x7fec4807c330 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fec440098a0 tx=0x7fec44009bb0 comp rx=0 tx=0).stop 2024-04-22T13:14:12.642 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.009+0000 7fec4f7c9700 1 -- 172.21.15.12:0/3427559330 shutdown_connections 2024-04-22T13:14:12.642 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.009+0000 7fec4f7c9700 1 --2- 172.21.15.12:0/3427559330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec4807de80 0x7fec4807c330 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:12.642 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.009+0000 7fec4f7c9700 1 -- 172.21.15.12:0/3427559330 >> 172.21.15.12:0/3427559330 conn(0x7fec48103b40 msgr2=0x7fec48105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:12.642 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.010+0000 7fec4f7c9700 1 -- 172.21.15.12:0/3427559330 shutdown_connections 2024-04-22T13:14:12.642 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.010+0000 7fec4f7c9700 1 -- 172.21.15.12:0/3427559330 wait complete. 2024-04-22T13:14:12.642 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.010+0000 7fec4f7c9700 1 Processor -- start 2024-04-22T13:14:12.643 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.010+0000 7fec4f7c9700 1 -- start start 2024-04-22T13:14:12.643 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.010+0000 7fec4f7c9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec4807de80 0x7fec4819e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:12.643 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.010+0000 7fec4f7c9700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fec4810ea90 con 0x7fec4807de80 2024-04-22T13:14:12.643 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.010+0000 7fec4d565700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec4807de80 0x7fec4819e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:12.643 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.010+0000 7fec4d565700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec4807de80 0x7fec4819e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46088/0 (socket says 172.21.15.12:46088) 2024-04-22T13:14:12.643 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.010+0000 7fec4d565700 1 -- 172.21.15.12:0/1187907184 learned_addr learned my addr 172.21.15.12:0/1187907184 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:12.643 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.011+0000 7fec4d565700 1 -- 172.21.15.12:0/1187907184 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4819ec10 con 0x7fec4807de80 2024-04-22T13:14:12.643 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.011+0000 7fec4d565700 1 --2- 172.21.15.12:0/1187907184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec4807de80 0x7fec4819e6d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fec44011ac0 tx=0x7fec44011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:12.643 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.011+0000 7fec3e7fc700 1 -- 172.21.15.12:0/1187907184 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec44003970 con 0x7fec4807de80 2024-04-22T13:14:12.643 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.011+0000 7fec3e7fc700 1 -- 172.21.15.12:0/1187907184 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fec44003b10 con 0x7fec4807de80 2024-04-22T13:14:12.643 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.011+0000 7fec3e7fc700 1 -- 172.21.15.12:0/1187907184 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec44024610 con 0x7fec4807de80 2024-04-22T13:14:12.643 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.011+0000 7fec4f7c9700 1 -- 172.21.15.12:0/1187907184 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec4819eea0 con 0x7fec4807de80 2024-04-22T13:14:12.643 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.011+0000 7fec4f7c9700 1 -- 172.21.15.12:0/1187907184 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec481a1b90 con 0x7fec4807de80 2024-04-22T13:14:12.643 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.011+0000 7fec3e7fc700 1 -- 172.21.15.12:0/1187907184 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fec44024bd0 con 0x7fec4807de80 2024-04-22T13:14:12.643 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.011+0000 7fec3e7fc700 1 -- 172.21.15.12:0/1187907184 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fec44019700 con 0x7fec4807de80 2024-04-22T13:14:12.644 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.012+0000 7fec4f7c9700 1 -- 172.21.15.12:0/1187907184 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec480640a0 con 0x7fec4807de80 2024-04-22T13:14:12.644 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.014+0000 7fec3e7fc700 1 -- 172.21.15.12:0/1187907184 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fec4401a9e0 con 0x7fec4807de80 2024-04-22T13:14:12.644 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.085+0000 7fec4f7c9700 1 -- 172.21.15.12:0/1187907184 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fec481a20c0 con 0x7fec4807de80 2024-04-22T13:14:12.644 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.085+0000 7fec3e7fc700 1 -- 172.21.15.12:0/1187907184 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fec4401a430 con 0x7fec4807de80 2024-04-22T13:14:12.644 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.086+0000 7fec4f7c9700 1 -- 172.21.15.12:0/1187907184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec4807de80 msgr2=0x7fec4819e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:12.644 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.086+0000 7fec4f7c9700 1 --2- 172.21.15.12:0/1187907184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec4807de80 0x7fec4819e6d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fec44011ac0 tx=0x7fec44011ba0 comp rx=0 tx=0).stop 2024-04-22T13:14:12.644 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.086+0000 7fec4f7c9700 1 -- 172.21.15.12:0/1187907184 shutdown_connections 2024-04-22T13:14:12.644 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.086+0000 7fec4f7c9700 1 --2- 172.21.15.12:0/1187907184 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fec4807de80 0x7fec4819e6d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:12.644 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.086+0000 7fec4f7c9700 1 -- 172.21.15.12:0/1187907184 >> 172.21.15.12:0/1187907184 conn(0x7fec48103b40 msgr2=0x7fec48104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:12.644 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.086+0000 7fec4f7c9700 1 -- 172.21.15.12:0/1187907184 shutdown_connections 2024-04-22T13:14:12.644 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:12.086+0000 7fec4f7c9700 1 -- 172.21.15.12:0/1187907184 wait complete. 2024-04-22T13:14:12.644 INFO:teuthology.orchestra.run.smithi012.stdout:Restarting the monitor... 2024-04-22T13:14:12.896 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:12 smithi012 systemd[1]: Stopping Ceph mon.a for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:14:13.287 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:12 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mon-a[27882]: 2024-04-22T13:14:12.896+0000 7f9925d10700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:14:13.287 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:12 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mon-a[27882]: 2024-04-22T13:14:12.896+0000 7f9925d10700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-22T13:14:13.287 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:13 smithi012 podman[28135]: 2024-04-22 13:14:13.050969838 +0000 UTC m=+0.244730218 container died b946abe52a5b3b4e716525dc847c4f437e86c9a6def6183a0e90ce870608b79e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-22T13:14:13.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:13 smithi012 podman[28135]: 2024-04-22 13:14:13.387912953 +0000 UTC m=+0.581673340 container cleanup b946abe52a5b3b4e716525dc847c4f437e86c9a6def6183a0e90ce870608b79e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-22T13:14:13.562 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:13 smithi012 bash[28135]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mon-a 2024-04-22T13:14:13.840 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:13 smithi012 podman[28147]: 2024-04-22 13:14:13.563858288 +0000 UTC m=+0.510219894 container remove b946abe52a5b3b4e716525dc847c4f437e86c9a6def6183a0e90ce870608b79e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-22T13:14:13.840 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:13 smithi012 systemd[1]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-22T13:14:13.840 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:13 smithi012 systemd[1]: Stopped Ceph mon.a for 0aede5ea-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:14:13.840 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:13 smithi012 systemd[1]: Starting Ceph mon.a for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:14:14.307 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 podman[28247]: 2024-04-22 13:14:14.01462416 +0000 UTC m=+0.107772293 container create f2fedfae476176dd590a89c0a4f08f37920f761ca34ffa424bc61e3b5b5e8b84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-22T13:14:14.307 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 podman[28247]: 2024-04-22 13:14:13.925222195 +0000 UTC m=+0.018370308 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-22T13:14:14.307 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 podman[28247]: 2024-04-22 13:14:14.225715595 +0000 UTC m=+0.318863693 container init f2fedfae476176dd590a89c0a4f08f37920f761ca34ffa424bc61e3b5b5e8b84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-22T13:14:14.307 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 podman[28247]: 2024-04-22 13:14:14.228898521 +0000 UTC m=+0.322046620 container start f2fedfae476176dd590a89c0a4f08f37920f761ca34ffa424bc61e3b5b5e8b84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-22T13:14:14.307 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: set uid:gid to 167:167 (ceph:ceph) 2024-04-22T13:14:14.307 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-22T13:14:14.307 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: pidfile_write: ignore empty --pid-file 2024-04-22T13:14:14.307 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: load: jerasure load: lrc 2024-04-22T13:14:14.307 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: RocksDB version: 7.9.2 2024-04-22T13:14:14.308 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Git sha 0 2024-04-22T13:14:14.308 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-22T13:14:14.308 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: DB SUMMARY 2024-04-22T13:14:14.308 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: DB Session ID: UPI1Y6F5X8H5REWPJH9S 2024-04-22T13:14:14.308 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: CURRENT file: CURRENT 2024-04-22T13:14:14.308 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: IDENTITY file: IDENTITY 2024-04-22T13:14:14.308 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-22T13:14:14.308 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-22T13:14:14.308 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105474 ; 2024-04-22T13:14:14.308 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.error_if_exists: 0 2024-04-22T13:14:14.309 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.create_if_missing: 0 2024-04-22T13:14:14.309 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.paranoid_checks: 1 2024-04-22T13:14:14.309 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-22T13:14:14.310 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-22T13:14:14.310 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-22T13:14:14.310 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.env: 0x55f6b0474cc0 2024-04-22T13:14:14.310 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.fs: PosixFileSystem 2024-04-22T13:14:14.310 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.info_log: 0x55f6b2149be0 2024-04-22T13:14:14.310 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.max_file_opening_threads: 16 2024-04-22T13:14:14.310 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.statistics: (nil) 2024-04-22T13:14:14.310 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.use_fsync: 0 2024-04-22T13:14:14.310 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.max_log_file_size: 0 2024-04-22T13:14:14.310 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-22T13:14:14.310 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-22T13:14:14.310 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.keep_log_file_num: 1000 2024-04-22T13:14:14.310 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.recycle_log_file_num: 0 2024-04-22T13:14:14.311 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.allow_fallocate: 1 2024-04-22T13:14:14.311 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.allow_mmap_reads: 0 2024-04-22T13:14:14.311 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.allow_mmap_writes: 0 2024-04-22T13:14:14.311 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.use_direct_reads: 0 2024-04-22T13:14:14.311 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-22T13:14:14.311 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.create_missing_column_families: 0 2024-04-22T13:14:14.311 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.db_log_dir: 2024-04-22T13:14:14.311 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.wal_dir: 2024-04-22T13:14:14.311 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-22T13:14:14.311 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-22T13:14:14.311 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-22T13:14:14.311 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-22T13:14:14.311 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-22T13:14:14.311 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-22T13:14:14.311 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.advise_random_on_open: 1 2024-04-22T13:14:14.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.db_write_buffer_size: 0 2024-04-22T13:14:14.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.write_buffer_manager: 0x55f6b20dd0e0 2024-04-22T13:14:14.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-22T13:14:14.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-22T13:14:14.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-22T13:14:14.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.rate_limiter: (nil) 2024-04-22T13:14:14.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-22T13:14:14.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.wal_recovery_mode: 2 2024-04-22T13:14:14.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.enable_thread_tracking: 0 2024-04-22T13:14:14.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.enable_pipelined_write: 0 2024-04-22T13:14:14.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.unordered_write: 0 2024-04-22T13:14:14.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-22T13:14:14.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-22T13:14:14.312 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-22T13:14:14.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-22T13:14:14.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.row_cache: None 2024-04-22T13:14:14.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.wal_filter: None 2024-04-22T13:14:14.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-22T13:14:14.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.allow_ingest_behind: 0 2024-04-22T13:14:14.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.two_write_queues: 0 2024-04-22T13:14:14.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.manual_wal_flush: 0 2024-04-22T13:14:14.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.wal_compression: 0 2024-04-22T13:14:14.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.atomic_flush: 0 2024-04-22T13:14:14.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-22T13:14:14.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-22T13:14:14.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-22T13:14:14.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.log_readahead_size: 0 2024-04-22T13:14:14.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-22T13:14:14.313 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.best_efforts_recovery: 0 2024-04-22T13:14:14.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-22T13:14:14.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-22T13:14:14.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.allow_data_in_errors: 0 2024-04-22T13:14:14.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.db_host_id: __hostname__ 2024-04-22T13:14:14.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-22T13:14:14.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.max_background_jobs: 2 2024-04-22T13:14:14.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.max_background_compactions: -1 2024-04-22T13:14:14.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.max_subcompactions: 1 2024-04-22T13:14:14.314 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-22T13:14:14.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-22T13:14:14.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-22T13:14:14.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.max_total_wal_size: 0 2024-04-22T13:14:14.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-22T13:14:14.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-22T13:14:14.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-22T13:14:14.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-22T13:14:14.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.max_open_files: -1 2024-04-22T13:14:14.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.bytes_per_sync: 0 2024-04-22T13:14:14.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-22T13:14:14.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-22T13:14:14.315 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.compaction_readahead_size: 0 2024-04-22T13:14:14.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.max_background_flushes: -1 2024-04-22T13:14:14.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Compression algorithms supported: 2024-04-22T13:14:14.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-22T13:14:14.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: kZSTD supported: 0 2024-04-22T13:14:14.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: kXpressCompression supported: 0 2024-04-22T13:14:14.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: kLZ4HCCompression supported: 1 2024-04-22T13:14:14.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: kZlibCompression supported: 1 2024-04-22T13:14:14.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: kSnappyCompression supported: 1 2024-04-22T13:14:14.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: kLZ4Compression supported: 1 2024-04-22T13:14:14.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: kBZip2Compression supported: 0 2024-04-22T13:14:14.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-22T13:14:14.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-22T13:14:14.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-22T13:14:14.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-22T13:14:14.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-22T13:14:14.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.merge_operator: 2024-04-22T13:14:14.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.compaction_filter: None 2024-04-22T13:14:14.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.compaction_filter_factory: None 2024-04-22T13:14:14.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.sst_partitioner_factory: None 2024-04-22T13:14:14.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-22T13:14:14.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-22T13:14:14.317 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f6b2149dc0) 2024-04-22T13:14:14.317 INFO:journalctl@ceph.mon.a.smithi012.stdout: cache_index_and_filter_blocks: 1 2024-04-22T13:14:14.317 INFO:journalctl@ceph.mon.a.smithi012.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-22T13:14:14.317 INFO:journalctl@ceph.mon.a.smithi012.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-22T13:14:14.317 INFO:journalctl@ceph.mon.a.smithi012.stdout: pin_top_level_index_and_filter: 1 2024-04-22T13:14:14.318 INFO:journalctl@ceph.mon.a.smithi012.stdout: index_type: 0 2024-04-22T13:14:14.318 INFO:journalctl@ceph.mon.a.smithi012.stdout: data_block_index_type: 0 2024-04-22T13:14:14.318 INFO:journalctl@ceph.mon.a.smithi012.stdout: index_shortening: 1 2024-04-22T13:14:14.318 INFO:journalctl@ceph.mon.a.smithi012.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-22T13:14:14.318 INFO:journalctl@ceph.mon.a.smithi012.stdout: checksum: 4 2024-04-22T13:14:14.318 INFO:journalctl@ceph.mon.a.smithi012.stdout: no_block_cache: 0 2024-04-22T13:14:14.318 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache: 0x55f6b210f1f0 2024-04-22T13:14:14.318 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache_name: BinnedLRUCache 2024-04-22T13:14:14.318 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache_options: 2024-04-22T13:14:14.318 INFO:journalctl@ceph.mon.a.smithi012.stdout: capacity : 536870912 2024-04-22T13:14:14.318 INFO:journalctl@ceph.mon.a.smithi012.stdout: num_shard_bits : 4 2024-04-22T13:14:14.318 INFO:journalctl@ceph.mon.a.smithi012.stdout: strict_capacity_limit : 0 2024-04-22T13:14:14.318 INFO:journalctl@ceph.mon.a.smithi012.stdout: high_pri_pool_ratio: 0.000 2024-04-22T13:14:14.318 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_cache_compressed: (nil) 2024-04-22T13:14:14.318 INFO:journalctl@ceph.mon.a.smithi012.stdout: persistent_cache: (nil) 2024-04-22T13:14:14.319 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_size: 4096 2024-04-22T13:14:14.319 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_size_deviation: 10 2024-04-22T13:14:14.319 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_restart_interval: 16 2024-04-22T13:14:14.319 INFO:journalctl@ceph.mon.a.smithi012.stdout: index_block_restart_interval: 1 2024-04-22T13:14:14.319 INFO:journalctl@ceph.mon.a.smithi012.stdout: metadata_block_size: 4096 2024-04-22T13:14:14.319 INFO:journalctl@ceph.mon.a.smithi012.stdout: partition_filters: 0 2024-04-22T13:14:14.319 INFO:journalctl@ceph.mon.a.smithi012.stdout: use_delta_encoding: 1 2024-04-22T13:14:14.319 INFO:journalctl@ceph.mon.a.smithi012.stdout: filter_policy: bloomfilter 2024-04-22T13:14:14.319 INFO:journalctl@ceph.mon.a.smithi012.stdout: whole_key_filtering: 1 2024-04-22T13:14:14.319 INFO:journalctl@ceph.mon.a.smithi012.stdout: verify_compression: 0 2024-04-22T13:14:14.319 INFO:journalctl@ceph.mon.a.smithi012.stdout: read_amp_bytes_per_bit: 0 2024-04-22T13:14:14.319 INFO:journalctl@ceph.mon.a.smithi012.stdout: format_version: 5 2024-04-22T13:14:14.319 INFO:journalctl@ceph.mon.a.smithi012.stdout: enable_index_compression: 1 2024-04-22T13:14:14.319 INFO:journalctl@ceph.mon.a.smithi012.stdout: block_align: 0 2024-04-22T13:14:14.319 INFO:journalctl@ceph.mon.a.smithi012.stdout: max_auto_readahead_size: 262144 2024-04-22T13:14:14.320 INFO:journalctl@ceph.mon.a.smithi012.stdout: prepopulate_block_cache: 0 2024-04-22T13:14:14.320 INFO:journalctl@ceph.mon.a.smithi012.stdout: initial_auto_readahead_size: 8192 2024-04-22T13:14:14.320 INFO:journalctl@ceph.mon.a.smithi012.stdout: num_file_reads_for_auto_readahead: 2 2024-04-22T13:14:14.320 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.write_buffer_size: 33554432 2024-04-22T13:14:14.320 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.max_write_buffer_number: 2 2024-04-22T13:14:14.320 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.compression: NoCompression 2024-04-22T13:14:14.320 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.bottommost_compression: Disabled 2024-04-22T13:14:14.320 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.prefix_extractor: nullptr 2024-04-22T13:14:14.320 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-22T13:14:14.320 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.num_levels: 7 2024-04-22T13:14:14.320 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-22T13:14:14.320 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-22T13:14:14.320 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-22T13:14:14.320 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-22T13:14:14.320 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-22T13:14:14.321 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-22T13:14:14.321 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-22T13:14:14.321 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-22T13:14:14.321 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-22T13:14:14.321 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-22T13:14:14.321 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-22T13:14:14.321 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-22T13:14:14.321 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-22T13:14:14.322 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.compression_opts.level: 32767 2024-04-22T13:14:14.322 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.compression_opts.strategy: 0 2024-04-22T13:14:14.322 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-22T13:14:14.322 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-22T13:14:14.322 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-22T13:14:14.322 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-22T13:14:14.322 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.compression_opts.enabled: false 2024-04-22T13:14:14.322 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-22T13:14:14.322 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-22T13:14:14.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-22T13:14:14.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-22T13:14:14.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.target_file_size_base: 67108864 2024-04-22T13:14:14.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-22T13:14:14.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-22T13:14:14.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-22T13:14:14.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-22T13:14:14.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-22T13:14:14.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-22T13:14:14.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-22T13:14:14.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-22T13:14:14.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-22T13:14:14.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-22T13:14:14.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-22T13:14:14.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-22T13:14:14.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-22T13:14:14.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-22T13:14:14.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.arena_block_size: 1048576 2024-04-22T13:14:14.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-22T13:14:14.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-22T13:14:14.324 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.disable_auto_compactions: 0 2024-04-22T13:14:14.325 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-22T13:14:14.325 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-22T13:14:14.325 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-22T13:14:14.325 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-22T13:14:14.325 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-22T13:14:14.325 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-22T13:14:14.325 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-22T13:14:14.326 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-22T13:14:14.326 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-22T13:14:14.326 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-22T13:14:14.326 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-22T13:14:14.326 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.inplace_update_support: 0 2024-04-22T13:14:14.327 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-22T13:14:14.327 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-22T13:14:14.327 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-22T13:14:14.328 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-22T13:14:14.329 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.bloom_locality: 0 2024-04-22T13:14:14.329 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.max_successive_merges: 0 2024-04-22T13:14:14.329 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-22T13:14:14.329 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.paranoid_file_checks: 0 2024-04-22T13:14:14.329 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.force_consistency_checks: 1 2024-04-22T13:14:14.329 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.report_bg_io_stats: 0 2024-04-22T13:14:14.330 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.ttl: 2592000 2024-04-22T13:14:14.330 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-22T13:14:14.330 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-22T13:14:14.330 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-22T13:14:14.330 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.enable_blob_files: false 2024-04-22T13:14:14.331 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.min_blob_size: 0 2024-04-22T13:14:14.331 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.blob_file_size: 268435456 2024-04-22T13:14:14.331 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-22T13:14:14.331 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-22T13:14:14.331 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-22T13:14:14.331 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-22T13:14:14.332 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-22T13:14:14.332 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.blob_file_starting_level: 0 2024-04-22T13:14:14.332 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-22T13:14:14.332 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-22T13:14:14.332 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-22T13:14:14.332 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5809e8b6-6eb1-4078-9e65-4a544e246e10 2024-04-22T13:14:14.333 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713791654258575, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-22T13:14:14.333 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-22T13:14:14.333 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713791654259608, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100559, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98453, "index_size": 298, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14272, "raw_average_key_size": 49, "raw_value_size": 90717, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713791654, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5809e8b6-6eb1-4078-9e65-4a544e246e10", "db_session_id": "UPI1Y6F5X8H5REWPJH9S", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-22T13:14:14.333 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713791654259691, "job": 1, "event": "recovery_finished"} 2024-04-22T13:14:14.333 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-22T13:14:14.333 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-22T13:14:14.334 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f6b2224000 2024-04-22T13:14:14.334 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: DB pointer 0x55f6b220e000 2024-04-22T13:14:14.334 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-22T13:14:14.334 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-22T13:14:14.334 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** DB Stats ** 2024-04-22T13:14:14.334 INFO:journalctl@ceph.mon.a.smithi012.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-22T13:14:14.334 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-22T13:14:14.335 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-22T13:14:14.335 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-22T13:14:14.335 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-22T13:14:14.335 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-22T13:14:14.335 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-22T13:14:14.335 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-04-22T13:14:14.335 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** Compaction Stats [default] ** 2024-04-22T13:14:14.336 INFO:journalctl@ceph.mon.a.smithi012.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-22T13:14:14.336 INFO:journalctl@ceph.mon.a.smithi012.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-22T13:14:14.336 INFO:journalctl@ceph.mon.a.smithi012.stdout: L0 2/0 100.06 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 158.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-22T13:14:14.336 INFO:journalctl@ceph.mon.a.smithi012.stdout: Sum 2/0 100.06 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 158.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-22T13:14:14.336 INFO:journalctl@ceph.mon.a.smithi012.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 158.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-22T13:14:14.336 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-04-22T13:14:14.336 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** Compaction Stats [default] ** 2024-04-22T13:14:14.337 INFO:journalctl@ceph.mon.a.smithi012.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-22T13:14:14.337 INFO:journalctl@ceph.mon.a.smithi012.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-22T13:14:14.337 INFO:journalctl@ceph.mon.a.smithi012.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 158.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-22T13:14:14.337 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-04-22T13:14:14.337 INFO:journalctl@ceph.mon.a.smithi012.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-22T13:14:14.337 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-04-22T13:14:14.337 INFO:journalctl@ceph.mon.a.smithi012.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-22T13:14:14.337 INFO:journalctl@ceph.mon.a.smithi012.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-22T13:14:14.338 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-22T13:14:14.338 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-22T13:14:14.338 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-22T13:14:14.338 INFO:journalctl@ceph.mon.a.smithi012.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-22T13:14:14.338 INFO:journalctl@ceph.mon.a.smithi012.stdout: Cumulative compaction: 0.00 GB write, 31.33 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-22T13:14:14.338 INFO:journalctl@ceph.mon.a.smithi012.stdout: Interval compaction: 0.00 GB write, 31.33 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-22T13:14:14.339 INFO:journalctl@ceph.mon.a.smithi012.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-22T13:14:14.339 INFO:journalctl@ceph.mon.a.smithi012.stdout: Block cache BinnedLRUCache@0x55f6b210f1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-22T13:14:14.339 INFO:journalctl@ceph.mon.a.smithi012.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-22T13:14:14.339 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-04-22T13:14:14.339 INFO:journalctl@ceph.mon.a.smithi012.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-22T13:14:14.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: starting mon.a rank 0 at public addrs [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] at bind addrs [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:14:14.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: mon.a@-1(???) e1 preinit fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:14:14.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: mon.a@-1(???).mds e1 new map 2024-04-22T13:14:14.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: mon.a@-1(???).mds e1 print_map 2024-04-22T13:14:14.340 INFO:journalctl@ceph.mon.a.smithi012.stdout: e1 2024-04-22T13:14:14.340 INFO:journalctl@ceph.mon.a.smithi012.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T13:14:14.340 INFO:journalctl@ceph.mon.a.smithi012.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-22T13:14:14.340 INFO:journalctl@ceph.mon.a.smithi012.stdout: legacy client fscid: -1 2024-04-22T13:14:14.341 INFO:journalctl@ceph.mon.a.smithi012.stdout: 2024-04-22T13:14:14.341 INFO:journalctl@ceph.mon.a.smithi012.stdout: No filesystems configured 2024-04-22T13:14:14.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-22T13:14:14.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-22T13:14:14.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-22T13:14:14.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-22T13:14:14.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-22T13:14:14.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 bash[28247]: f2fedfae476176dd590a89c0a4f08f37920f761ca34ffa424bc61e3b5b5e8b84 2024-04-22T13:14:14.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 systemd[1]: Started Ceph mon.a for 0aede5ea-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:14:14.342 INFO:teuthology.orchestra.run.smithi012.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-22T13:14:14.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-22T13:14:14.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: monmap epoch 1 2024-04-22T13:14:14.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:14:14.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: last_changed 2024-04-22T13:14:05.291228+0000 2024-04-22T13:14:14.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: created 2024-04-22T13:14:05.291228+0000 2024-04-22T13:14:14.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: min_mon_release 19 (squid) 2024-04-22T13:14:14.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: election_strategy: 1 2024-04-22T13:14:14.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-04-22T13:14:14.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: fsmap 2024-04-22T13:14:14.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: osdmap e1: 0 total, 0 up, 0 in 2024-04-22T13:14:14.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:14 smithi012 ceph-mon[28265]: mgrmap e1: no daemons active 2024-04-22T13:14:15.420 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.749+0000 7f85ffc1f700 1 Processor -- start 2024-04-22T13:14:15.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.749+0000 7f85ffc1f700 1 -- start start 2024-04-22T13:14:15.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.749+0000 7f85ffc1f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85f8108fb0 0x7f85f810b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:15.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.749+0000 7f85ffc1f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f85f8063d60 con 0x7f85f8108fb0 2024-04-22T13:14:15.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.750+0000 7f85fd9bb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85f8108fb0 0x7f85f810b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:15.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.750+0000 7f85fd9bb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85f8108fb0 0x7f85f810b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46090/0 (socket says 172.21.15.12:46090) 2024-04-22T13:14:15.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.750+0000 7f85fd9bb700 1 -- 172.21.15.12:0/3746305473 learned_addr learned my addr 172.21.15.12:0/3746305473 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:15.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.750+0000 7f85fd9bb700 1 -- 172.21.15.12:0/3746305473 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f810b8e0 con 0x7f85f8108fb0 2024-04-22T13:14:15.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.750+0000 7f85fd9bb700 1 --2- 172.21.15.12:0/3746305473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85f8108fb0 0x7f85f810b3a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f85f40098a0 tx=0x7f85f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a90d65d8bb7898 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:15.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.750+0000 7f85fc9b9700 1 -- 172.21.15.12:0/3746305473 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85f4004160 con 0x7f85f8108fb0 2024-04-22T13:14:15.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.751+0000 7f85fc9b9700 1 -- 172.21.15.12:0/3746305473 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f85f4004300 con 0x7f85f8108fb0 2024-04-22T13:14:15.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.751+0000 7f85fc9b9700 1 -- 172.21.15.12:0/3746305473 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85f4011af0 con 0x7f85f8108fb0 2024-04-22T13:14:15.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.751+0000 7f85ffc1f700 1 -- 172.21.15.12:0/3746305473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85f8108fb0 msgr2=0x7f85f810b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:15.421 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.751+0000 7f85ffc1f700 1 --2- 172.21.15.12:0/3746305473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85f8108fb0 0x7f85f810b3a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f85f40098a0 tx=0x7f85f4009bb0 comp rx=0 tx=0).stop 2024-04-22T13:14:15.422 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.751+0000 7f85ffc1f700 1 -- 172.21.15.12:0/3746305473 shutdown_connections 2024-04-22T13:14:15.422 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.751+0000 7f85ffc1f700 1 --2- 172.21.15.12:0/3746305473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85f8108fb0 0x7f85f810b3a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:15.422 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.751+0000 7f85ffc1f700 1 -- 172.21.15.12:0/3746305473 >> 172.21.15.12:0/3746305473 conn(0x7f85f8102a40 msgr2=0x7f85f8104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:15.422 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.751+0000 7f85ffc1f700 1 -- 172.21.15.12:0/3746305473 shutdown_connections 2024-04-22T13:14:15.422 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.751+0000 7f85ffc1f700 1 -- 172.21.15.12:0/3746305473 wait complete. 2024-04-22T13:14:15.422 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.752+0000 7f85ffc1f700 1 Processor -- start 2024-04-22T13:14:15.422 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.752+0000 7f85ffc1f700 1 -- start start 2024-04-22T13:14:15.422 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.752+0000 7f85ffc1f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85f8108fb0 0x7f85f819c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:15.422 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.752+0000 7f85ffc1f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f85f810c8b0 con 0x7f85f8108fb0 2024-04-22T13:14:15.423 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.752+0000 7f85fd9bb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85f8108fb0 0x7f85f819c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:15.423 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.752+0000 7f85fd9bb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85f8108fb0 0x7f85f819c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46104/0 (socket says 172.21.15.12:46104) 2024-04-22T13:14:15.423 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.752+0000 7f85fd9bb700 1 -- 172.21.15.12:0/1504023220 learned_addr learned my addr 172.21.15.12:0/1504023220 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:15.423 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.752+0000 7f85fd9bb700 1 -- 172.21.15.12:0/1504023220 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f819c990 con 0x7f85f8108fb0 2024-04-22T13:14:15.423 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.752+0000 7f85fd9bb700 1 --2- 172.21.15.12:0/1504023220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85f8108fb0 0x7f85f819c450 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f85f4011ac0 tx=0x7f85f4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:15.423 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.753+0000 7f85eeffd700 1 -- 172.21.15.12:0/1504023220 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85f4003b20 con 0x7f85f8108fb0 2024-04-22T13:14:15.423 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.753+0000 7f85eeffd700 1 -- 172.21.15.12:0/1504023220 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f85f401a8c0 con 0x7f85f8108fb0 2024-04-22T13:14:15.423 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.753+0000 7f85ffc1f700 1 -- 172.21.15.12:0/1504023220 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85f819cc20 con 0x7f85f8108fb0 2024-04-22T13:14:15.423 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.753+0000 7f85eeffd700 1 -- 172.21.15.12:0/1504023220 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85f40248d0 con 0x7f85f8108fb0 2024-04-22T13:14:15.423 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.753+0000 7f85ffc1f700 1 -- 172.21.15.12:0/1504023220 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85f819f910 con 0x7f85f8108fb0 2024-04-22T13:14:15.423 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.753+0000 7f85eeffd700 1 -- 172.21.15.12:0/1504023220 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f85f4022020 con 0x7f85f8108fb0 2024-04-22T13:14:15.423 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.753+0000 7f85eeffd700 1 -- 172.21.15.12:0/1504023220 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f85f40193c0 con 0x7f85f8108fb0 2024-04-22T13:14:15.423 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.754+0000 7f85ffc1f700 1 -- 172.21.15.12:0/1504023220 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85f80640a0 con 0x7f85f8108fb0 2024-04-22T13:14:15.423 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.756+0000 7f85eeffd700 1 -- 172.21.15.12:0/1504023220 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f85f4042e50 con 0x7f85f8108fb0 2024-04-22T13:14:15.423 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.828+0000 7f85ffc1f700 1 -- 172.21.15.12:0/1504023220 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f85f819fae0 con 0x7f85f8108fb0 2024-04-22T13:14:15.424 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.830+0000 7f85eeffd700 1 -- 172.21.15.12:0/1504023220 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f85f4031020 con 0x7f85f8108fb0 2024-04-22T13:14:15.424 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.831+0000 7f85ffc1f700 1 -- 172.21.15.12:0/1504023220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85f8108fb0 msgr2=0x7f85f819c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:15.424 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.831+0000 7f85ffc1f700 1 --2- 172.21.15.12:0/1504023220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85f8108fb0 0x7f85f819c450 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f85f4011ac0 tx=0x7f85f4011ba0 comp rx=0 tx=0).stop 2024-04-22T13:14:15.424 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.831+0000 7f85ffc1f700 1 -- 172.21.15.12:0/1504023220 shutdown_connections 2024-04-22T13:14:15.424 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.831+0000 7f85ffc1f700 1 --2- 172.21.15.12:0/1504023220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85f8108fb0 0x7f85f819c450 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:15.424 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.831+0000 7f85ffc1f700 1 -- 172.21.15.12:0/1504023220 >> 172.21.15.12:0/1504023220 conn(0x7f85f8102a40 msgr2=0x7f85f81034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:15.424 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.831+0000 7f85ffc1f700 1 -- 172.21.15.12:0/1504023220 shutdown_connections 2024-04-22T13:14:15.424 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:14.831+0000 7f85ffc1f700 1 -- 172.21.15.12:0/1504023220 wait complete. 2024-04-22T13:14:15.424 INFO:teuthology.orchestra.run.smithi012.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-22T13:14:15.446 INFO:teuthology.orchestra.run.smithi012.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-22T13:14:15.446 INFO:teuthology.orchestra.run.smithi012.stdout:Creating mgr... 2024-04-22T13:14:15.447 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-22T13:14:15.447 INFO:teuthology.orchestra.run.smithi012.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-22T13:14:15.711 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mgr.x 2024-04-22T13:14:15.712 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-22T13:14:15.888 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0aede5ea-00aa-11ef-bc93-c7b262605968.target.wants/ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@.service. 2024-04-22T13:14:16.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:15 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1504023220' entity='client.admin' 2024-04-22T13:14:16.090 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:15 smithi012 systemd[1]: Starting Ceph mgr.x for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:14:16.376 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:16 smithi012 podman[28485]: 2024-04-22 13:14:16.116742834 +0000 UTC m=+0.109361180 container create 822680d55dcd56b91d5bc4f10bb989d0f7f38bc01c560d035e060fbb4006c424 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-04-22T13:14:16.376 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:16 smithi012 podman[28485]: 2024-04-22 13:14:16.02566363 +0000 UTC m=+0.018282002 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-22T13:14:16.376 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:16 smithi012 podman[28485]: 2024-04-22 13:14:16.310690846 +0000 UTC m=+0.303309194 container init 822680d55dcd56b91d5bc4f10bb989d0f7f38bc01c560d035e060fbb4006c424 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-22T13:14:16.376 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:16 smithi012 podman[28485]: 2024-04-22 13:14:16.313332098 +0000 UTC m=+0.305950445 container start 822680d55dcd56b91d5bc4f10bb989d0f7f38bc01c560d035e060fbb4006c424 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-22T13:14:16.376 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:16 smithi012 bash[28485]: 822680d55dcd56b91d5bc4f10bb989d0f7f38bc01c560d035e060fbb4006c424 2024-04-22T13:14:16.392 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-22T13:14:16.392 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-04-22T13:14:16.400 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-22T13:14:16.401 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-04-22T13:14:16.401 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-04-22T13:14:16.401 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to enable service . firewalld.service is not available 2024-04-22T13:14:16.411 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-22T13:14:16.411 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-04-22T13:14:16.420 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-22T13:14:16.420 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-04-22T13:14:16.420 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-04-22T13:14:16.420 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-22T13:14:16.420 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr to start... 2024-04-22T13:14:16.421 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr... 2024-04-22T13:14:16.672 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:16 smithi012 systemd[1]: Started Ceph mgr.x for 0aede5ea-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:14:16.672 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:16 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:16.482+0000 7f4c57ed1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T13:14:16.980 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:16 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:16.679+0000 7f4c57ed1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:14:16.980 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:16 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:16.785+0000 7f4c57ed1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T13:14:17.323 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:17 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/840102122' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-22T13:14:17.324 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:17 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:17.117+0000 7f4c57ed1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T13:14:17.324 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:17 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:17.205+0000 7f4c57ed1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T13:14:17.601 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-04-22T13:14:17.606 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-04-22T13:14:17.606 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "0aede5ea-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:14:17.606 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-04-22T13:14:17.606 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-22T13:14:17.606 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-22T13:14:17.606 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-22T13:14:17.606 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:14:17.606 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-22T13:14:17.606 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-22T13:14:17.607 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-04-22T13:14:17.607 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-04-22T13:14:17.607 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-22T13:14:17.607 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "a" 2024-04-22T13:14:17.607 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-04-22T13:14:17.607 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-22T13:14:17.607 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-22T13:14:17.607 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:14:17.607 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-22T13:14:17.607 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-22T13:14:17.607 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:14:17.607 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-22T13:14:17.607 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:14:17.607 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-22T13:14:17.607 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-22T13:14:17.608 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-22T13:14:17.608 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-22T13:14:17.608 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-22T13:14:17.608 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-22T13:14:17.608 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:14:17.608 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-22T13:14:17.608 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-22T13:14:17.608 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-22T13:14:17.608 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-22T13:14:17.608 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-22T13:14:17.608 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-22T13:14:17.608 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-22T13:14:17.608 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-22T13:14:17.608 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-22T13:14:17.608 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:14:17.608 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-22T13:14:17.609 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:14:17.609 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-22T13:14:17.609 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-22T13:14:17.609 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:14:17.609 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-22T13:14:17.609 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-22T13:14:17.609 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-22T13:14:17.609 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-22T13:14:17.609 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-22T13:14:17.609 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-22T13:14:17.609 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-04-22T13:14:17.609 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-04-22T13:14:17.609 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T13:14:17.609 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:14:17.609 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-22T13:14:17.609 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:14:17.615 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-04-22T13:14:09.363588+0000", 2024-04-22T13:14:17.615 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T13:14:17.615 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:14:17.615 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-22T13:14:17.615 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-04-22T13:14:17.615 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.899+0000 7f9d93a2d700 1 Processor -- start 2024-04-22T13:14:17.615 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.899+0000 7f9d93a2d700 1 -- start start 2024-04-22T13:14:17.615 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.899+0000 7f9d93a2d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d8c108570 0x7f9d8c108990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:17.615 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.899+0000 7f9d93a2d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c108f60 con 0x7f9d8c108570 2024-04-22T13:14:17.615 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.899+0000 7f9d917c9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d8c108570 0x7f9d8c108990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:17.616 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.900+0000 7f9d917c9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d8c108570 0x7f9d8c108990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46130/0 (socket says 172.21.15.12:46130) 2024-04-22T13:14:17.616 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.900+0000 7f9d917c9700 1 -- 172.21.15.12:0/265332868 learned_addr learned my addr 172.21.15.12:0/265332868 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:17.616 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.900+0000 7f9d917c9700 1 -- 172.21.15.12:0/265332868 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8c109720 con 0x7f9d8c108570 2024-04-22T13:14:17.616 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.900+0000 7f9d917c9700 1 --2- 172.21.15.12:0/265332868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d8c108570 0x7f9d8c108990 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9d880098a0 tx=0x7f9d88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fce25ccaef143b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:17.616 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.900+0000 7f9d83fff700 1 -- 172.21.15.12:0/265332868 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d88004160 con 0x7f9d8c108570 2024-04-22T13:14:17.616 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.900+0000 7f9d83fff700 1 -- 172.21.15.12:0/265332868 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9d88004300 con 0x7f9d8c108570 2024-04-22T13:14:17.616 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.900+0000 7f9d83fff700 1 -- 172.21.15.12:0/265332868 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d88011af0 con 0x7f9d8c108570 2024-04-22T13:14:17.616 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.901+0000 7f9d93a2d700 1 -- 172.21.15.12:0/265332868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d8c108570 msgr2=0x7f9d8c108990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:17.616 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.901+0000 7f9d93a2d700 1 --2- 172.21.15.12:0/265332868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d8c108570 0x7f9d8c108990 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9d880098a0 tx=0x7f9d88009bb0 comp rx=0 tx=0).stop 2024-04-22T13:14:17.616 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.901+0000 7f9d93a2d700 1 -- 172.21.15.12:0/265332868 shutdown_connections 2024-04-22T13:14:17.616 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.901+0000 7f9d93a2d700 1 --2- 172.21.15.12:0/265332868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d8c108570 0x7f9d8c108990 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:17.616 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.901+0000 7f9d93a2d700 1 -- 172.21.15.12:0/265332868 >> 172.21.15.12:0/265332868 conn(0x7f9d8c103b70 msgr2=0x7f9d8c105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:17.616 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.901+0000 7f9d93a2d700 1 -- 172.21.15.12:0/265332868 shutdown_connections 2024-04-22T13:14:17.616 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.901+0000 7f9d93a2d700 1 -- 172.21.15.12:0/265332868 wait complete. 2024-04-22T13:14:17.617 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.902+0000 7f9d93a2d700 1 Processor -- start 2024-04-22T13:14:17.617 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.902+0000 7f9d93a2d700 1 -- start start 2024-04-22T13:14:17.617 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.902+0000 7f9d93a2d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d8c108570 0x7f9d8c19c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:17.617 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.902+0000 7f9d93a2d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c10cad0 con 0x7f9d8c108570 2024-04-22T13:14:17.617 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.902+0000 7f9d917c9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d8c108570 0x7f9d8c19c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:17.617 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.902+0000 7f9d917c9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d8c108570 0x7f9d8c19c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46132/0 (socket says 172.21.15.12:46132) 2024-04-22T13:14:17.617 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.902+0000 7f9d917c9700 1 -- 172.21.15.12:0/840102122 learned_addr learned my addr 172.21.15.12:0/840102122 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:17.617 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.902+0000 7f9d917c9700 1 -- 172.21.15.12:0/840102122 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8c19ca10 con 0x7f9d8c108570 2024-04-22T13:14:17.617 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.903+0000 7f9d917c9700 1 --2- 172.21.15.12:0/840102122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d8c108570 0x7f9d8c19c4d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009fb0 tx=0x7f9d88011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:17.617 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.903+0000 7f9d827fc700 1 -- 172.21.15.12:0/840102122 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d880036a0 con 0x7f9d8c108570 2024-04-22T13:14:17.617 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.903+0000 7f9d93a2d700 1 -- 172.21.15.12:0/840102122 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d8c19cca0 con 0x7f9d8c108570 2024-04-22T13:14:17.617 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.903+0000 7f9d827fc700 1 -- 172.21.15.12:0/840102122 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9d8801a430 con 0x7f9d8c108570 2024-04-22T13:14:17.617 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.903+0000 7f9d827fc700 1 -- 172.21.15.12:0/840102122 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d88019550 con 0x7f9d8c108570 2024-04-22T13:14:17.617 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.903+0000 7f9d93a2d700 1 -- 172.21.15.12:0/840102122 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d8c19f990 con 0x7f9d8c108570 2024-04-22T13:14:17.617 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.903+0000 7f9d827fc700 1 -- 172.21.15.12:0/840102122 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9d88019b90 con 0x7f9d8c108570 2024-04-22T13:14:17.618 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.903+0000 7f9d827fc700 1 -- 172.21.15.12:0/840102122 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9d88024730 con 0x7f9d8c108570 2024-04-22T13:14:17.618 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.904+0000 7f9d93a2d700 1 -- 172.21.15.12:0/840102122 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d8c0640a0 con 0x7f9d8c108570 2024-04-22T13:14:17.618 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.906+0000 7f9d827fc700 1 -- 172.21.15.12:0/840102122 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f9d88043e50 con 0x7f9d8c108570 2024-04-22T13:14:17.618 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.981+0000 7f9d93a2d700 1 -- 172.21.15.12:0/840102122 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f9d8c19fee0 con 0x7f9d8c108570 2024-04-22T13:14:17.618 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.982+0000 7f9d827fc700 1 -- 172.21.15.12:0/840102122 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f9d88032020 con 0x7f9d8c108570 2024-04-22T13:14:17.618 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.983+0000 7f9d93a2d700 1 -- 172.21.15.12:0/840102122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d8c108570 msgr2=0x7f9d8c19c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:17.618 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.983+0000 7f9d93a2d700 1 --2- 172.21.15.12:0/840102122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d8c108570 0x7f9d8c19c4d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009fb0 tx=0x7f9d88011940 comp rx=0 tx=0).stop 2024-04-22T13:14:17.618 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.983+0000 7f9d93a2d700 1 -- 172.21.15.12:0/840102122 shutdown_connections 2024-04-22T13:14:17.618 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.983+0000 7f9d93a2d700 1 --2- 172.21.15.12:0/840102122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d8c108570 0x7f9d8c19c4d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:17.618 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.983+0000 7f9d93a2d700 1 -- 172.21.15.12:0/840102122 >> 172.21.15.12:0/840102122 conn(0x7f9d8c103b70 msgr2=0x7f9d8c104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:17.618 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.983+0000 7f9d93a2d700 1 -- 172.21.15.12:0/840102122 shutdown_connections 2024-04-22T13:14:17.618 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:16.983+0000 7f9d93a2d700 1 -- 172.21.15.12:0/840102122 wait complete. 2024-04-22T13:14:17.618 INFO:teuthology.orchestra.run.smithi012.stdout:mgr not available, waiting (1/15)... 2024-04-22T13:14:18.834 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:18 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:18.484+0000 7f4c57ed1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T13:14:18.834 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:18 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:18.659+0000 7f4c57ed1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T13:14:18.834 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:18 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:18.835+0000 7f4c57ed1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T13:14:19.438 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:19 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:19.132+0000 7f4c57ed1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T13:14:19.515 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:19 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:19.227+0000 7f4c57ed1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T13:14:19.515 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:19 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:19.438+0000 7f4c57ed1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T13:14:19.739 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:19 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:19.639+0000 7f4c57ed1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:14:20.293 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:20 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3384195594' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-22T13:14:20.589 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:20 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:20.294+0000 7f4c57ed1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T13:14:20.806 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-04-22T13:14:20.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-04-22T13:14:20.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "0aede5ea-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:14:20.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-04-22T13:14:20.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-22T13:14:20.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-22T13:14:20.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-22T13:14:20.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:14:20.807 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-22T13:14:20.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-22T13:14:20.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-04-22T13:14:20.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-04-22T13:14:20.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-22T13:14:20.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "a" 2024-04-22T13:14:20.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-04-22T13:14:20.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-22T13:14:20.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-22T13:14:20.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:14:20.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-22T13:14:20.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-22T13:14:20.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:14:20.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-22T13:14:20.808 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:14:20.809 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-22T13:14:20.809 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-22T13:14:20.809 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-22T13:14:20.809 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-22T13:14:20.809 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-22T13:14:20.809 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-22T13:14:20.809 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:14:20.809 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-22T13:14:20.809 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-22T13:14:20.809 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-22T13:14:20.809 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-22T13:14:20.809 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-22T13:14:20.809 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-22T13:14:20.809 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-22T13:14:20.810 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-22T13:14:20.810 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-22T13:14:20.810 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:14:20.810 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-22T13:14:20.810 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:14:20.810 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-22T13:14:20.810 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-22T13:14:20.810 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:14:20.810 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-22T13:14:20.810 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-22T13:14:20.810 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-22T13:14:20.810 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-22T13:14:20.810 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-22T13:14:20.810 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-22T13:14:20.810 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-04-22T13:14:20.811 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-04-22T13:14:20.811 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T13:14:20.811 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:14:20.811 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-22T13:14:20.811 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:14:20.811 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-04-22T13:14:09.363588+0000", 2024-04-22T13:14:20.811 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T13:14:20.811 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:14:20.811 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-22T13:14:20.811 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-04-22T13:14:20.811 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.038+0000 7fc4a1c48700 1 Processor -- start 2024-04-22T13:14:20.811 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.038+0000 7fc4a1c48700 1 -- start start 2024-04-22T13:14:20.811 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.038+0000 7fc4a1c48700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc49c107440 0x7fc49c107860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:20.811 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.038+0000 7fc4a1c48700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c107e30 con 0x7fc49c107440 2024-04-22T13:14:20.812 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.038+0000 7fc49b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc49c107440 0x7fc49c107860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:20.812 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.038+0000 7fc49b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc49c107440 0x7fc49c107860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46138/0 (socket says 172.21.15.12:46138) 2024-04-22T13:14:20.812 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.038+0000 7fc49b7fe700 1 -- 172.21.15.12:0/1936585094 learned_addr learned my addr 172.21.15.12:0/1936585094 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:20.812 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.039+0000 7fc49b7fe700 1 -- 172.21.15.12:0/1936585094 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49c108640 con 0x7fc49c107440 2024-04-22T13:14:20.812 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.039+0000 7fc49b7fe700 1 --2- 172.21.15.12:0/1936585094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc49c107440 0x7fc49c107860 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0098a0 tx=0x7fc48c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41df67725cb8d77 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:20.812 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.039+0000 7fc49a7fc700 1 -- 172.21.15.12:0/1936585094 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc48c004160 con 0x7fc49c107440 2024-04-22T13:14:20.812 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.039+0000 7fc49a7fc700 1 -- 172.21.15.12:0/1936585094 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc48c004300 con 0x7fc49c107440 2024-04-22T13:14:20.812 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.039+0000 7fc49a7fc700 1 -- 172.21.15.12:0/1936585094 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc48c011af0 con 0x7fc49c107440 2024-04-22T13:14:20.812 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.040+0000 7fc4a1c48700 1 -- 172.21.15.12:0/1936585094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc49c107440 msgr2=0x7fc49c107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:20.812 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.040+0000 7fc4a1c48700 1 --2- 172.21.15.12:0/1936585094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc49c107440 0x7fc49c107860 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0098a0 tx=0x7fc48c009bb0 comp rx=0 tx=0).stop 2024-04-22T13:14:20.812 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.040+0000 7fc4a1c48700 1 -- 172.21.15.12:0/1936585094 shutdown_connections 2024-04-22T13:14:20.812 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.040+0000 7fc4a1c48700 1 --2- 172.21.15.12:0/1936585094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc49c107440 0x7fc49c107860 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:20.812 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.040+0000 7fc4a1c48700 1 -- 172.21.15.12:0/1936585094 >> 172.21.15.12:0/1936585094 conn(0x7fc49c102a40 msgr2=0x7fc49c104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:20.812 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.040+0000 7fc4a1c48700 1 -- 172.21.15.12:0/1936585094 shutdown_connections 2024-04-22T13:14:20.813 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.040+0000 7fc4a1c48700 1 -- 172.21.15.12:0/1936585094 wait complete. 2024-04-22T13:14:20.813 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.040+0000 7fc4a1c48700 1 Processor -- start 2024-04-22T13:14:20.813 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.041+0000 7fc4a1c48700 1 -- start start 2024-04-22T13:14:20.813 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.041+0000 7fc4a1c48700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc49c107440 0x7fc49c19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:20.813 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.041+0000 7fc4a1c48700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c10b9f0 con 0x7fc49c107440 2024-04-22T13:14:20.813 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.041+0000 7fc49b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc49c107440 0x7fc49c19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:20.813 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.041+0000 7fc49b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc49c107440 0x7fc49c19c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46144/0 (socket says 172.21.15.12:46144) 2024-04-22T13:14:20.813 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.041+0000 7fc49b7fe700 1 -- 172.21.15.12:0/3384195594 learned_addr learned my addr 172.21.15.12:0/3384195594 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:20.813 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.041+0000 7fc49b7fe700 1 -- 172.21.15.12:0/3384195594 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49c19ca60 con 0x7fc49c107440 2024-04-22T13:14:20.813 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.041+0000 7fc49b7fe700 1 --2- 172.21.15.12:0/3384195594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc49c107440 0x7fc49c19c520 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc48c000c00 tx=0x7fc48c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:20.813 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.042+0000 7fc498ff9700 1 -- 172.21.15.12:0/3384195594 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc48c003b20 con 0x7fc49c107440 2024-04-22T13:14:20.813 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.042+0000 7fc498ff9700 1 -- 172.21.15.12:0/3384195594 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc48c01a8c0 con 0x7fc49c107440 2024-04-22T13:14:20.813 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.042+0000 7fc4a1c48700 1 -- 172.21.15.12:0/3384195594 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc49c19ccf0 con 0x7fc49c107440 2024-04-22T13:14:20.813 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.042+0000 7fc498ff9700 1 -- 172.21.15.12:0/3384195594 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc48c024940 con 0x7fc49c107440 2024-04-22T13:14:20.814 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.042+0000 7fc4a1c48700 1 -- 172.21.15.12:0/3384195594 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc49c19f990 con 0x7fc49c107440 2024-04-22T13:14:20.814 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.042+0000 7fc498ff9700 1 -- 172.21.15.12:0/3384195594 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc48c022020 con 0x7fc49c107440 2024-04-22T13:14:20.814 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.042+0000 7fc498ff9700 1 -- 172.21.15.12:0/3384195594 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc48c019610 con 0x7fc49c107440 2024-04-22T13:14:20.814 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.043+0000 7fc4a1c48700 1 -- 172.21.15.12:0/3384195594 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc49c0640a0 con 0x7fc49c107440 2024-04-22T13:14:20.814 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.045+0000 7fc498ff9700 1 -- 172.21.15.12:0/3384195594 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc48c01aa70 con 0x7fc49c107440 2024-04-22T13:14:20.814 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.118+0000 7fc4a1c48700 1 -- 172.21.15.12:0/3384195594 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc49c02e210 con 0x7fc49c107440 2024-04-22T13:14:20.814 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.118+0000 7fc498ff9700 1 -- 172.21.15.12:0/3384195594 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fc48c01a430 con 0x7fc49c107440 2024-04-22T13:14:20.814 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.119+0000 7fc4a1c48700 1 -- 172.21.15.12:0/3384195594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc49c107440 msgr2=0x7fc49c19c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:20.814 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.119+0000 7fc4a1c48700 1 --2- 172.21.15.12:0/3384195594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc49c107440 0x7fc49c19c520 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc48c000c00 tx=0x7fc48c011b70 comp rx=0 tx=0).stop 2024-04-22T13:14:20.814 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.119+0000 7fc4a1c48700 1 -- 172.21.15.12:0/3384195594 shutdown_connections 2024-04-22T13:14:20.814 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.119+0000 7fc4a1c48700 1 --2- 172.21.15.12:0/3384195594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc49c107440 0x7fc49c19c520 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:20.814 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.119+0000 7fc4a1c48700 1 -- 172.21.15.12:0/3384195594 >> 172.21.15.12:0/3384195594 conn(0x7fc49c102a40 msgr2=0x7fc49c1038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:20.814 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.120+0000 7fc4a1c48700 1 -- 172.21.15.12:0/3384195594 shutdown_connections 2024-04-22T13:14:20.814 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:20.120+0000 7fc4a1c48700 1 -- 172.21.15.12:0/3384195594 wait complete. 2024-04-22T13:14:20.814 INFO:teuthology.orchestra.run.smithi012.stdout:mgr not available, waiting (2/15)... 2024-04-22T13:14:21.039 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:20 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:20.733+0000 7f4c57ed1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T13:14:21.039 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:20 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:20.833+0000 7f4c57ed1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T13:14:21.039 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:21 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:21.040+0000 7f4c57ed1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T13:14:21.839 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:21 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:21.584+0000 7f4c57ed1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T13:14:21.839 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:21 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:21.679+0000 7f4c57ed1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T13:14:22.281 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:21 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:21.928+0000 7f4c57ed1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T13:14:22.281 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:22 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:22.065+0000 7f4c57ed1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T13:14:22.281 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:22 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:22.163+0000 7f4c57ed1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T13:14:22.281 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:22 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:22.282+0000 7f4c57ed1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T13:14:22.920 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:22 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:22.594+0000 7f4c57ed1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T13:14:23.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:23 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/257403124' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-22T13:14:23.590 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:23 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:23.233+0000 7f4c57ed1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T13:14:23.590 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:23 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:23.465+0000 7f4c57ed1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T13:14:23.922 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-04-22T13:14:23.923 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-04-22T13:14:23.923 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "0aede5ea-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:14:23.923 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-04-22T13:14:23.923 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-22T13:14:23.923 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-22T13:14:23.923 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-22T13:14:23.923 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:14:23.923 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-22T13:14:23.923 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-22T13:14:23.923 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-04-22T13:14:23.923 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-04-22T13:14:23.923 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-22T13:14:23.923 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "a" 2024-04-22T13:14:23.923 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-04-22T13:14:23.923 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-04-22T13:14:23.924 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-22T13:14:23.924 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:14:23.924 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-22T13:14:23.924 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-22T13:14:23.924 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:14:23.924 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-22T13:14:23.924 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:14:23.924 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-22T13:14:23.924 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-22T13:14:23.924 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-22T13:14:23.924 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-22T13:14:23.924 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-22T13:14:23.924 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-22T13:14:23.924 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:14:23.924 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-22T13:14:23.925 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-22T13:14:23.925 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-22T13:14:23.925 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-22T13:14:23.925 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-22T13:14:23.925 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-22T13:14:23.925 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-22T13:14:23.925 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-22T13:14:23.925 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-22T13:14:23.925 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:14:23.925 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-22T13:14:23.925 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:14:23.925 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-22T13:14:23.925 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-22T13:14:23.925 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:14:23.926 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-22T13:14:23.926 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-22T13:14:23.926 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-22T13:14:23.926 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-22T13:14:23.926 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-22T13:14:23.926 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-22T13:14:23.926 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-04-22T13:14:23.926 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-04-22T13:14:23.926 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T13:14:23.926 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:14:23.926 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-22T13:14:23.926 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:14:23.926 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-04-22T13:14:09.363588+0000", 2024-04-22T13:14:23.927 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T13:14:23.927 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:14:23.927 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-22T13:14:23.927 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-04-22T13:14:23.927 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.233+0000 7f8da548e700 1 Processor -- start 2024-04-22T13:14:23.927 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.233+0000 7f8da548e700 1 -- start start 2024-04-22T13:14:23.927 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.233+0000 7f8da548e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8da010a770 0x7f8da010ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:23.927 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.233+0000 7f8da548e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8da010b160 con 0x7f8da010a770 2024-04-22T13:14:23.927 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.233+0000 7f8d9effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8da010a770 0x7f8da010ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:23.927 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.233+0000 7f8d9effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8da010a770 0x7f8da010ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60508/0 (socket says 172.21.15.12:60508) 2024-04-22T13:14:23.927 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.233+0000 7f8d9effd700 1 -- 172.21.15.12:0/515628063 learned_addr learned my addr 172.21.15.12:0/515628063 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:23.927 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.234+0000 7f8d9effd700 1 -- 172.21.15.12:0/515628063 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da010b920 con 0x7f8da010a770 2024-04-22T13:14:23.927 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.234+0000 7f8d9effd700 1 --2- 172.21.15.12:0/515628063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8da010a770 0x7f8da010ab90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8d900098a0 tx=0x7f8d90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c4491b44284ba9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:23.927 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.234+0000 7f8d9e7fc700 1 -- 172.21.15.12:0/515628063 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d90004160 con 0x7f8da010a770 2024-04-22T13:14:23.928 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.234+0000 7f8d9e7fc700 1 -- 172.21.15.12:0/515628063 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8d90004300 con 0x7f8da010a770 2024-04-22T13:14:23.928 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.234+0000 7f8d9e7fc700 1 -- 172.21.15.12:0/515628063 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d90011af0 con 0x7f8da010a770 2024-04-22T13:14:23.928 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.235+0000 7f8da548e700 1 -- 172.21.15.12:0/515628063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8da010a770 msgr2=0x7f8da010ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:23.928 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.235+0000 7f8da548e700 1 --2- 172.21.15.12:0/515628063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8da010a770 0x7f8da010ab90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8d900098a0 tx=0x7f8d90009bb0 comp rx=0 tx=0).stop 2024-04-22T13:14:23.928 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.235+0000 7f8da548e700 1 -- 172.21.15.12:0/515628063 shutdown_connections 2024-04-22T13:14:23.928 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.235+0000 7f8da548e700 1 --2- 172.21.15.12:0/515628063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8da010a770 0x7f8da010ab90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:23.928 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.235+0000 7f8da548e700 1 -- 172.21.15.12:0/515628063 >> 172.21.15.12:0/515628063 conn(0x7f8da007db60 msgr2=0x7f8da01084e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:23.928 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.235+0000 7f8da548e700 1 -- 172.21.15.12:0/515628063 shutdown_connections 2024-04-22T13:14:23.928 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.235+0000 7f8da548e700 1 -- 172.21.15.12:0/515628063 wait complete. 2024-04-22T13:14:23.928 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.236+0000 7f8da548e700 1 Processor -- start 2024-04-22T13:14:23.928 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.236+0000 7f8da548e700 1 -- start start 2024-04-22T13:14:23.928 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.236+0000 7f8da548e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8da010a770 0x7f8da019e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:23.928 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.236+0000 7f8da548e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8da010ecd0 con 0x7f8da010a770 2024-04-22T13:14:23.928 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.236+0000 7f8d9effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8da010a770 0x7f8da019e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:23.929 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.236+0000 7f8d9effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8da010a770 0x7f8da019e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60520/0 (socket says 172.21.15.12:60520) 2024-04-22T13:14:23.929 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.236+0000 7f8d9effd700 1 -- 172.21.15.12:0/257403124 learned_addr learned my addr 172.21.15.12:0/257403124 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:23.929 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.236+0000 7f8d9effd700 1 -- 172.21.15.12:0/257403124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da019ebd0 con 0x7f8da010a770 2024-04-22T13:14:23.929 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.236+0000 7f8d9effd700 1 --2- 172.21.15.12:0/257403124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8da010a770 0x7f8da019e690 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8d90009fb0 tx=0x7f8d90011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:23.929 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.237+0000 7f8d9cff9700 1 -- 172.21.15.12:0/257403124 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d900036a0 con 0x7f8da010a770 2024-04-22T13:14:23.929 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.237+0000 7f8da548e700 1 -- 172.21.15.12:0/257403124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8da019ee60 con 0x7f8da010a770 2024-04-22T13:14:23.929 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.237+0000 7f8d9cff9700 1 -- 172.21.15.12:0/257403124 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8d9001a430 con 0x7f8da010a770 2024-04-22T13:14:23.929 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.237+0000 7f8d9cff9700 1 -- 172.21.15.12:0/257403124 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d90019550 con 0x7f8da010a770 2024-04-22T13:14:23.929 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.237+0000 7f8da548e700 1 -- 172.21.15.12:0/257403124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8da01a1b50 con 0x7f8da010a770 2024-04-22T13:14:23.929 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.237+0000 7f8d9cff9700 1 -- 172.21.15.12:0/257403124 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8d90019be0 con 0x7f8da010a770 2024-04-22T13:14:23.929 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.237+0000 7f8d9cff9700 1 -- 172.21.15.12:0/257403124 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8d90024730 con 0x7f8da010a770 2024-04-22T13:14:23.929 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.238+0000 7f8da548e700 1 -- 172.21.15.12:0/257403124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8da00640a0 con 0x7f8da010a770 2024-04-22T13:14:23.929 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.240+0000 7f8d9cff9700 1 -- 172.21.15.12:0/257403124 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f8d90043e50 con 0x7f8da010a770 2024-04-22T13:14:23.929 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.314+0000 7f8da548e700 1 -- 172.21.15.12:0/257403124 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f8da01a2090 con 0x7f8da010a770 2024-04-22T13:14:23.930 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.314+0000 7f8d9cff9700 1 -- 172.21.15.12:0/257403124 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f8d90032020 con 0x7f8da010a770 2024-04-22T13:14:23.930 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.316+0000 7f8da548e700 1 -- 172.21.15.12:0/257403124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8da010a770 msgr2=0x7f8da019e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:23.930 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.316+0000 7f8da548e700 1 --2- 172.21.15.12:0/257403124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8da010a770 0x7f8da019e690 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8d90009fb0 tx=0x7f8d90011940 comp rx=0 tx=0).stop 2024-04-22T13:14:23.930 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.316+0000 7f8da548e700 1 -- 172.21.15.12:0/257403124 shutdown_connections 2024-04-22T13:14:23.930 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.316+0000 7f8da548e700 1 --2- 172.21.15.12:0/257403124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8da010a770 0x7f8da019e690 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:23.930 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.316+0000 7f8da548e700 1 -- 172.21.15.12:0/257403124 >> 172.21.15.12:0/257403124 conn(0x7f8da007db60 msgr2=0x7f8da0107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:23.930 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.316+0000 7f8da548e700 1 -- 172.21.15.12:0/257403124 shutdown_connections 2024-04-22T13:14:23.930 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:23.316+0000 7f8da548e700 1 -- 172.21.15.12:0/257403124 wait complete. 2024-04-22T13:14:23.930 INFO:teuthology.orchestra.run.smithi012.stdout:mgr not available, waiting (3/15)... 2024-04-22T13:14:24.840 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:24 smithi012 ceph-mon[28265]: Activating manager daemon x 2024-04-22T13:14:24.840 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:24 smithi012 ceph-mon[28265]: mgrmap e2: x(active, starting, since 0.00308945s) 2024-04-22T13:14:24.840 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:24 smithi012 ceph-mon[28265]: from='mgr.14100 172.21.15.12:0/2197258377' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T13:14:24.840 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:24 smithi012 ceph-mon[28265]: from='mgr.14100 172.21.15.12:0/2197258377' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T13:14:24.840 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:24 smithi012 ceph-mon[28265]: from='mgr.14100 172.21.15.12:0/2197258377' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T13:14:24.840 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:24 smithi012 ceph-mon[28265]: from='mgr.14100 172.21.15.12:0/2197258377' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:14:24.840 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:24 smithi012 ceph-mon[28265]: from='mgr.14100 172.21.15.12:0/2197258377' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-22T13:14:24.840 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:24 smithi012 ceph-mon[28265]: Manager daemon x is now available 2024-04-22T13:14:24.840 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:24 smithi012 ceph-mon[28265]: from='mgr.14100 172.21.15.12:0/2197258377' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-22T13:14:24.840 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:24 smithi012 ceph-mon[28265]: from='mgr.14100 172.21.15.12:0/2197258377' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-22T13:14:24.840 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:24 smithi012 ceph-mon[28265]: from='mgr.14100 172.21.15.12:0/2197258377' entity='mgr.x' 2024-04-22T13:14:24.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:24 smithi012 ceph-mon[28265]: from='mgr.14100 172.21.15.12:0/2197258377' entity='mgr.x' 2024-04-22T13:14:24.841 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:24 smithi012 ceph-mon[28265]: from='mgr.14100 172.21.15.12:0/2197258377' entity='mgr.x' 2024-04-22T13:14:25.840 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:25 smithi012 ceph-mon[28265]: mgrmap e3: x(active, since 1.00704s) 2024-04-22T13:14:26.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:26 smithi012 ceph-mon[28265]: mgrmap e4: x(active, since 2s) 2024-04-22T13:14:27.271 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-04-22T13:14:27.271 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-04-22T13:14:27.271 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsid": "0aede5ea-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:14:27.271 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "health": { 2024-04-22T13:14:27.271 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-22T13:14:27.272 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-22T13:14:27.272 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-22T13:14:27.272 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:14:27.272 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-22T13:14:27.272 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-22T13:14:27.272 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 0 2024-04-22T13:14:27.272 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-04-22T13:14:27.272 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-22T13:14:27.272 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "a" 2024-04-22T13:14:27.272 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-04-22T13:14:27.272 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-22T13:14:27.272 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-22T13:14:27.272 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:14:27.272 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-22T13:14:27.272 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-22T13:14:27.273 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:14:27.273 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-22T13:14:27.273 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:14:27.273 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-22T13:14:27.273 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-22T13:14:27.273 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-22T13:14:27.273 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-22T13:14:27.273 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-22T13:14:27.273 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-22T13:14:27.273 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:14:27.273 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-22T13:14:27.273 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-22T13:14:27.273 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-22T13:14:27.273 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-22T13:14:27.273 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-22T13:14:27.274 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-22T13:14:27.274 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-22T13:14:27.274 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-22T13:14:27.274 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-22T13:14:27.274 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:14:27.274 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-22T13:14:27.274 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:14:27.274 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-22T13:14:27.274 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-22T13:14:27.274 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:14:27.274 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-22T13:14:27.274 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-22T13:14:27.274 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-22T13:14:27.274 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-22T13:14:27.275 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-22T13:14:27.275 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-22T13:14:27.275 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "restful" 2024-04-22T13:14:27.275 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ], 2024-04-22T13:14:27.275 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T13:14:27.275 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:14:27.275 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-22T13:14:27.275 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-22T13:14:27.275 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "modified": "2024-04-22T13:14:09.363588+0000", 2024-04-22T13:14:27.275 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-22T13:14:27.275 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout }, 2024-04-22T13:14:27.275 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-22T13:14:27.275 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-04-22T13:14:27.275 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.367+0000 7f7e32b61700 1 Processor -- start 2024-04-22T13:14:27.275 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.367+0000 7f7e32b61700 1 -- start start 2024-04-22T13:14:27.276 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.367+0000 7f7e32b61700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e2c108b20 0x7f7e2c108f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:27.276 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.367+0000 7f7e32b61700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c107060 con 0x7f7e2c108b20 2024-04-22T13:14:27.276 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.367+0000 7f7e308fd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e2c108b20 0x7f7e2c108f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:27.276 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.367+0000 7f7e308fd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e2c108b20 0x7f7e2c108f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60606/0 (socket says 172.21.15.12:60606) 2024-04-22T13:14:27.276 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.367+0000 7f7e308fd700 1 -- 172.21.15.12:0/488253989 learned_addr learned my addr 172.21.15.12:0/488253989 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:27.276 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.367+0000 7f7e308fd700 1 -- 172.21.15.12:0/488253989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2c1071e0 con 0x7f7e2c108b20 2024-04-22T13:14:27.276 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.368+0000 7f7e308fd700 1 --2- 172.21.15.12:0/488253989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e2c108b20 0x7f7e2c108f40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7e200098a0 tx=0x7f7e20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eab01417f16a87 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:27.276 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.368+0000 7f7e2b7fe700 1 -- 172.21.15.12:0/488253989 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e20004160 con 0x7f7e2c108b20 2024-04-22T13:14:27.276 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.368+0000 7f7e2b7fe700 1 -- 172.21.15.12:0/488253989 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7e20004300 con 0x7f7e2c108b20 2024-04-22T13:14:27.276 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.368+0000 7f7e2b7fe700 1 -- 172.21.15.12:0/488253989 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e20011af0 con 0x7f7e2c108b20 2024-04-22T13:14:27.276 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.368+0000 7f7e32b61700 1 -- 172.21.15.12:0/488253989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e2c108b20 msgr2=0x7f7e2c108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:27.276 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.368+0000 7f7e32b61700 1 --2- 172.21.15.12:0/488253989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e2c108b20 0x7f7e2c108f40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7e200098a0 tx=0x7f7e20009bb0 comp rx=0 tx=0).stop 2024-04-22T13:14:27.276 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.368+0000 7f7e32b61700 1 -- 172.21.15.12:0/488253989 shutdown_connections 2024-04-22T13:14:27.276 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.369+0000 7f7e32b61700 1 --2- 172.21.15.12:0/488253989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e2c108b20 0x7f7e2c108f40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:27.277 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.369+0000 7f7e32b61700 1 -- 172.21.15.12:0/488253989 >> 172.21.15.12:0/488253989 conn(0x7f7e2c102a60 msgr2=0x7f7e2c104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:27.277 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.369+0000 7f7e32b61700 1 -- 172.21.15.12:0/488253989 shutdown_connections 2024-04-22T13:14:27.277 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.369+0000 7f7e32b61700 1 -- 172.21.15.12:0/488253989 wait complete. 2024-04-22T13:14:27.277 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.369+0000 7f7e32b61700 1 Processor -- start 2024-04-22T13:14:27.277 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.369+0000 7f7e32b61700 1 -- start start 2024-04-22T13:14:27.277 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.369+0000 7f7e32b61700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e2c108b20 0x7f7e2c19c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:27.277 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.369+0000 7f7e32b61700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c10b9f0 con 0x7f7e2c108b20 2024-04-22T13:14:27.277 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.370+0000 7f7e308fd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e2c108b20 0x7f7e2c19c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:27.277 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.370+0000 7f7e308fd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e2c108b20 0x7f7e2c19c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60608/0 (socket says 172.21.15.12:60608) 2024-04-22T13:14:27.277 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.370+0000 7f7e308fd700 1 -- 172.21.15.12:0/247609146 learned_addr learned my addr 172.21.15.12:0/247609146 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:27.277 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.370+0000 7f7e308fd700 1 -- 172.21.15.12:0/247609146 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2c19cb80 con 0x7f7e2c108b20 2024-04-22T13:14:27.278 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.370+0000 7f7e308fd700 1 --2- 172.21.15.12:0/247609146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e2c108b20 0x7f7e2c19c640 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7e20009fb0 tx=0x7f7e20011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:27.278 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.370+0000 7f7e29ffb700 1 -- 172.21.15.12:0/247609146 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e200036a0 con 0x7f7e2c108b20 2024-04-22T13:14:27.278 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.370+0000 7f7e29ffb700 1 -- 172.21.15.12:0/247609146 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7e2001a430 con 0x7f7e2c108b20 2024-04-22T13:14:27.278 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.370+0000 7f7e29ffb700 1 -- 172.21.15.12:0/247609146 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e20019550 con 0x7f7e2c108b20 2024-04-22T13:14:27.278 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.370+0000 7f7e32b61700 1 -- 172.21.15.12:0/247609146 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e2c19ce10 con 0x7f7e2c108b20 2024-04-22T13:14:27.278 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.371+0000 7f7e32b61700 1 -- 172.21.15.12:0/247609146 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e2c19fb00 con 0x7f7e2c108b20 2024-04-22T13:14:27.278 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.371+0000 7f7e29ffb700 1 -- 172.21.15.12:0/247609146 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f7e2001a8e0 con 0x7f7e2c108b20 2024-04-22T13:14:27.278 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.372+0000 7f7e32b61700 1 -- 172.21.15.12:0/247609146 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e2c10bb70 con 0x7f7e2c108b20 2024-04-22T13:14:27.278 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.372+0000 7f7e29ffb700 1 --2- 172.21.15.12:0/247609146 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f7e1c0395b0 0x7f7e1c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:27.278 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.372+0000 7f7e29ffb700 1 -- 172.21.15.12:0/247609146 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7e2004d800 con 0x7f7e2c108b20 2024-04-22T13:14:27.278 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.372+0000 7f7e2bfff700 1 --2- 172.21.15.12:0/247609146 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f7e1c0395b0 0x7f7e1c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:27.278 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.372+0000 7f7e2bfff700 1 --2- 172.21.15.12:0/247609146 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f7e1c0395b0 0x7f7e1c03ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7e18006fd0 tx=0x7f7e18006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:27.278 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.377+0000 7f7e29ffb700 1 -- 172.21.15.12:0/247609146 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7e20020240 con 0x7f7e2c108b20 2024-04-22T13:14:27.279 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.669+0000 7f7e32b61700 1 -- 172.21.15.12:0/247609146 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7e2c02e260 con 0x7f7e2c108b20 2024-04-22T13:14:27.279 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.669+0000 7f7e29ffb700 1 -- 172.21.15.12:0/247609146 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f7e2001a690 con 0x7f7e2c108b20 2024-04-22T13:14:27.279 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.673+0000 7f7e32b61700 1 -- 172.21.15.12:0/247609146 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f7e1c0395b0 msgr2=0x7f7e1c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:27.279 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.673+0000 7f7e32b61700 1 --2- 172.21.15.12:0/247609146 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f7e1c0395b0 0x7f7e1c03ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7e18006fd0 tx=0x7f7e18006e40 comp rx=0 tx=0).stop 2024-04-22T13:14:27.279 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.673+0000 7f7e32b61700 1 -- 172.21.15.12:0/247609146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e2c108b20 msgr2=0x7f7e2c19c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:27.279 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.673+0000 7f7e32b61700 1 --2- 172.21.15.12:0/247609146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e2c108b20 0x7f7e2c19c640 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7e20009fb0 tx=0x7f7e20011940 comp rx=0 tx=0).stop 2024-04-22T13:14:27.279 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.673+0000 7f7e32b61700 1 -- 172.21.15.12:0/247609146 shutdown_connections 2024-04-22T13:14:27.279 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.673+0000 7f7e32b61700 1 --2- 172.21.15.12:0/247609146 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f7e1c0395b0 0x7f7e1c03ba70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:27.279 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.673+0000 7f7e32b61700 1 --2- 172.21.15.12:0/247609146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7e2c108b20 0x7f7e2c19c640 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:27.279 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.673+0000 7f7e32b61700 1 -- 172.21.15.12:0/247609146 >> 172.21.15.12:0/247609146 conn(0x7f7e2c102a60 msgr2=0x7f7e2c103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:27.279 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.673+0000 7f7e32b61700 1 -- 172.21.15.12:0/247609146 shutdown_connections 2024-04-22T13:14:27.280 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:26.673+0000 7f7e32b61700 1 -- 172.21.15.12:0/247609146 wait complete. 2024-04-22T13:14:27.280 INFO:teuthology.orchestra.run.smithi012.stdout:mgr is available 2024-04-22T13:14:27.643 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:27 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/247609146' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-22T13:14:28.448 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-04-22T13:14:28.449 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [global] 2024-04-22T13:14:28.449 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout fsid = 0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:14:28.449 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-22T13:14:28.449 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-22T13:14:28.449 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-22T13:14:28.449 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-22T13:14:28.449 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-22T13:14:28.449 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-04-22T13:14:28.449 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-22T13:14:28.449 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-22T13:14:28.449 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 2024-04-22T13:14:28.449 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout [osd] 2024-04-22T13:14:28.449 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-22T13:14:28.449 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-22T13:14:28.450 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-22T13:14:28.450 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-22T13:14:28.450 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-22T13:14:28.450 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-22T13:14:28.450 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.674+0000 7f0eb2516700 1 Processor -- start 2024-04-22T13:14:28.450 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.674+0000 7f0eb2516700 1 -- start start 2024-04-22T13:14:28.450 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.674+0000 7f0eb2516700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0eac108550 0x7f0eac108970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:28.450 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.674+0000 7f0eb2516700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac108f40 con 0x7f0eac108550 2024-04-22T13:14:28.450 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.674+0000 7f0eabfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0eac108550 0x7f0eac108970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:28.450 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.675+0000 7f0eabfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0eac108550 0x7f0eac108970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60622/0 (socket says 172.21.15.12:60622) 2024-04-22T13:14:28.451 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.675+0000 7f0eabfff700 1 -- 172.21.15.12:0/2000659419 learned_addr learned my addr 172.21.15.12:0/2000659419 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:28.451 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.675+0000 7f0eabfff700 1 -- 172.21.15.12:0/2000659419 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eac109700 con 0x7f0eac108550 2024-04-22T13:14:28.451 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.675+0000 7f0eabfff700 1 --2- 172.21.15.12:0/2000659419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0eac108550 0x7f0eac108970 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0098a0 tx=0x7f0e9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa19d6978473e57 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:28.451 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.675+0000 7f0eaaffd700 1 -- 172.21.15.12:0/2000659419 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e9c004160 con 0x7f0eac108550 2024-04-22T13:14:28.451 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.675+0000 7f0eaaffd700 1 -- 172.21.15.12:0/2000659419 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0e9c004300 con 0x7f0eac108550 2024-04-22T13:14:28.451 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.675+0000 7f0eaaffd700 1 -- 172.21.15.12:0/2000659419 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e9c011af0 con 0x7f0eac108550 2024-04-22T13:14:28.451 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.676+0000 7f0eb2516700 1 -- 172.21.15.12:0/2000659419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0eac108550 msgr2=0x7f0eac108970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:28.451 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.676+0000 7f0eb2516700 1 --2- 172.21.15.12:0/2000659419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0eac108550 0x7f0eac108970 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0098a0 tx=0x7f0e9c009bb0 comp rx=0 tx=0).stop 2024-04-22T13:14:28.451 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.676+0000 7f0eb2516700 1 -- 172.21.15.12:0/2000659419 shutdown_connections 2024-04-22T13:14:28.451 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.676+0000 7f0eb2516700 1 --2- 172.21.15.12:0/2000659419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0eac108550 0x7f0eac108970 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:28.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.676+0000 7f0eb2516700 1 -- 172.21.15.12:0/2000659419 >> 172.21.15.12:0/2000659419 conn(0x7f0eac103b10 msgr2=0x7f0eac105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:28.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.676+0000 7f0eb2516700 1 -- 172.21.15.12:0/2000659419 shutdown_connections 2024-04-22T13:14:28.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.676+0000 7f0eb2516700 1 -- 172.21.15.12:0/2000659419 wait complete. 2024-04-22T13:14:28.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.676+0000 7f0eb2516700 1 Processor -- start 2024-04-22T13:14:28.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.676+0000 7f0eb2516700 1 -- start start 2024-04-22T13:14:28.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.677+0000 7f0eb2516700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0eac108550 0x7f0eac19e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:28.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.677+0000 7f0eb2516700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac10cab0 con 0x7f0eac108550 2024-04-22T13:14:28.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.677+0000 7f0eabfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0eac108550 0x7f0eac19e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:28.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.677+0000 7f0eabfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0eac108550 0x7f0eac19e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60624/0 (socket says 172.21.15.12:60624) 2024-04-22T13:14:28.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.677+0000 7f0eabfff700 1 -- 172.21.15.12:0/3871539208 learned_addr learned my addr 172.21.15.12:0/3871539208 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:28.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.677+0000 7f0eabfff700 1 -- 172.21.15.12:0/3871539208 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eac19ec50 con 0x7f0eac108550 2024-04-22T13:14:28.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.677+0000 7f0eabfff700 1 --2- 172.21.15.12:0/3871539208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0eac108550 0x7f0eac19e710 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0117b0 tx=0x7f0e9c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:28.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.677+0000 7f0ea97fa700 1 -- 172.21.15.12:0/3871539208 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e9c003af0 con 0x7f0eac108550 2024-04-22T13:14:28.452 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.677+0000 7f0ea97fa700 1 -- 172.21.15.12:0/3871539208 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0e9c020070 con 0x7f0eac108550 2024-04-22T13:14:28.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.677+0000 7f0eb2516700 1 -- 172.21.15.12:0/3871539208 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eac19eee0 con 0x7f0eac108550 2024-04-22T13:14:28.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.678+0000 7f0ea97fa700 1 -- 172.21.15.12:0/3871539208 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e9c019630 con 0x7f0eac108550 2024-04-22T13:14:28.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.678+0000 7f0eb2516700 1 -- 172.21.15.12:0/3871539208 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eac1a1bd0 con 0x7f0eac108550 2024-04-22T13:14:28.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.678+0000 7f0ea97fa700 1 -- 172.21.15.12:0/3871539208 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f0e9c0198a0 con 0x7f0eac108550 2024-04-22T13:14:28.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.678+0000 7f0ea97fa700 1 --2- 172.21.15.12:0/3871539208 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f0e940395b0 0x7f0e9403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:28.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.679+0000 7f0ea97fa700 1 -- 172.21.15.12:0/3871539208 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0e9c04db20 con 0x7f0eac108550 2024-04-22T13:14:28.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.679+0000 7f0eab7fe700 1 --2- 172.21.15.12:0/3871539208 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f0e940395b0 0x7f0e9403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:28.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.679+0000 7f0eb2516700 1 -- 172.21.15.12:0/3871539208 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eac0640a0 con 0x7f0eac108550 2024-04-22T13:14:28.453 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.679+0000 7f0eab7fe700 1 --2- 172.21.15.12:0/3871539208 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f0e940395b0 0x7f0e9403ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0006fd0 tx=0x7f0ea0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:28.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.684+0000 7f0ea97fa700 1 -- 172.21.15.12:0/3871539208 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0e9c04d540 con 0x7f0eac108550 2024-04-22T13:14:28.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.892+0000 7f0eb2516700 1 -- 172.21.15.12:0/3871539208 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f0eac1a1da0 con 0x7f0eac108550 2024-04-22T13:14:28.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.893+0000 7f0ea97fa700 1 -- 172.21.15.12:0/3871539208 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7f0e9c02bd00 con 0x7f0eac108550 2024-04-22T13:14:28.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.895+0000 7f0eb2516700 1 -- 172.21.15.12:0/3871539208 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f0e940395b0 msgr2=0x7f0e9403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:28.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.895+0000 7f0eb2516700 1 --2- 172.21.15.12:0/3871539208 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f0e940395b0 0x7f0e9403ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0006fd0 tx=0x7f0ea0006e40 comp rx=0 tx=0).stop 2024-04-22T13:14:28.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.895+0000 7f0eb2516700 1 -- 172.21.15.12:0/3871539208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0eac108550 msgr2=0x7f0eac19e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:28.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.895+0000 7f0eb2516700 1 --2- 172.21.15.12:0/3871539208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0eac108550 0x7f0eac19e710 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0117b0 tx=0x7f0e9c011b70 comp rx=0 tx=0).stop 2024-04-22T13:14:28.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.896+0000 7f0eb2516700 1 -- 172.21.15.12:0/3871539208 shutdown_connections 2024-04-22T13:14:28.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.896+0000 7f0eb2516700 1 --2- 172.21.15.12:0/3871539208 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f0e940395b0 0x7f0e9403ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:28.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.896+0000 7f0eb2516700 1 --2- 172.21.15.12:0/3871539208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0eac108550 0x7f0eac19e710 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:28.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.896+0000 7f0eb2516700 1 -- 172.21.15.12:0/3871539208 >> 172.21.15.12:0/3871539208 conn(0x7f0eac103b10 msgr2=0x7f0eac104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:28.454 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.896+0000 7f0eb2516700 1 -- 172.21.15.12:0/3871539208 shutdown_connections 2024-04-22T13:14:28.455 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:27.896+0000 7f0eb2516700 1 -- 172.21.15.12:0/3871539208 wait complete. 2024-04-22T13:14:28.455 INFO:teuthology.orchestra.run.smithi012.stdout:Enabling cephadm module... 2024-04-22T13:14:28.694 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:28 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3871539208' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-22T13:14:29.777 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:29 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1327557057' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-22T13:14:29.777 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:29 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: ignoring --setuser ceph since I am not root 2024-04-22T13:14:29.777 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:29 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: ignoring --setgroup ceph since I am not root 2024-04-22T13:14:29.777 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:29 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:29.650+0000 7f7537b52200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T13:14:30.060 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:29 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:29.842+0000 7f7537b52200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:14:30.061 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:29 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:29.949+0000 7f7537b52200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T13:14:30.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.903+0000 7f9278446700 1 Processor -- start 2024-04-22T13:14:30.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.903+0000 7f9278446700 1 -- start start 2024-04-22T13:14:30.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.904+0000 7f9278446700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9270106e30 0x7f9270107250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:30.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.904+0000 7f9278446700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f927007be40 con 0x7f9270106e30 2024-04-22T13:14:30.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.904+0000 7f92761e2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9270106e30 0x7f9270107250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:30.068 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.904+0000 7f92761e2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9270106e30 0x7f9270107250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60630/0 (socket says 172.21.15.12:60630) 2024-04-22T13:14:30.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.904+0000 7f92761e2700 1 -- 172.21.15.12:0/830915697 learned_addr learned my addr 172.21.15.12:0/830915697 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:30.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.904+0000 7f92761e2700 1 -- 172.21.15.12:0/830915697 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9270107790 con 0x7f9270106e30 2024-04-22T13:14:30.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.904+0000 7f92761e2700 1 --2- 172.21.15.12:0/830915697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9270106e30 0x7f9270107250 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f926c0098a0 tx=0x7f926c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a95590e1bbe869d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:30.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.904+0000 7f92751e0700 1 -- 172.21.15.12:0/830915697 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f926c004160 con 0x7f9270106e30 2024-04-22T13:14:30.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.905+0000 7f92751e0700 1 -- 172.21.15.12:0/830915697 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f926c004300 con 0x7f9270106e30 2024-04-22T13:14:30.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.905+0000 7f92751e0700 1 -- 172.21.15.12:0/830915697 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f926c011af0 con 0x7f9270106e30 2024-04-22T13:14:30.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.905+0000 7f9278446700 1 -- 172.21.15.12:0/830915697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9270106e30 msgr2=0x7f9270107250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:30.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.905+0000 7f9278446700 1 --2- 172.21.15.12:0/830915697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9270106e30 0x7f9270107250 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f926c0098a0 tx=0x7f926c009bb0 comp rx=0 tx=0).stop 2024-04-22T13:14:30.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.905+0000 7f9278446700 1 -- 172.21.15.12:0/830915697 shutdown_connections 2024-04-22T13:14:30.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.905+0000 7f9278446700 1 --2- 172.21.15.12:0/830915697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9270106e30 0x7f9270107250 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:30.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.905+0000 7f9278446700 1 -- 172.21.15.12:0/830915697 >> 172.21.15.12:0/830915697 conn(0x7f9270102a50 msgr2=0x7f9270104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:30.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.905+0000 7f9278446700 1 -- 172.21.15.12:0/830915697 shutdown_connections 2024-04-22T13:14:30.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.905+0000 7f9278446700 1 -- 172.21.15.12:0/830915697 wait complete. 2024-04-22T13:14:30.069 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.906+0000 7f9278446700 1 Processor -- start 2024-04-22T13:14:30.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.906+0000 7f9278446700 1 -- start start 2024-04-22T13:14:30.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.906+0000 7f9278446700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9270106e30 0x7f927019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:30.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.906+0000 7f9278446700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f92701087c0 con 0x7f9270106e30 2024-04-22T13:14:30.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.906+0000 7f92761e2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9270106e30 0x7f927019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:30.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.906+0000 7f92761e2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9270106e30 0x7f927019e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60632/0 (socket says 172.21.15.12:60632) 2024-04-22T13:14:30.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.906+0000 7f92761e2700 1 -- 172.21.15.12:0/1327557057 learned_addr learned my addr 172.21.15.12:0/1327557057 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:30.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.907+0000 7f92761e2700 1 -- 172.21.15.12:0/1327557057 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f927019ebe0 con 0x7f9270106e30 2024-04-22T13:14:30.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.907+0000 7f92761e2700 1 --2- 172.21.15.12:0/1327557057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9270106e30 0x7f927019e6a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f926c011ac0 tx=0x7f926c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:30.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.907+0000 7f92677fe700 1 -- 172.21.15.12:0/1327557057 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f926c003970 con 0x7f9270106e30 2024-04-22T13:14:30.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.907+0000 7f92677fe700 1 -- 172.21.15.12:0/1327557057 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f926c003b10 con 0x7f9270106e30 2024-04-22T13:14:30.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.907+0000 7f9278446700 1 -- 172.21.15.12:0/1327557057 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f927019ee70 con 0x7f9270106e30 2024-04-22T13:14:30.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.907+0000 7f92677fe700 1 -- 172.21.15.12:0/1327557057 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f926c024610 con 0x7f9270106e30 2024-04-22T13:14:30.070 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.907+0000 7f9278446700 1 -- 172.21.15.12:0/1327557057 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92701a1b60 con 0x7f9270106e30 2024-04-22T13:14:30.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.908+0000 7f9278446700 1 -- 172.21.15.12:0/1327557057 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92700640a0 con 0x7f9270106e30 2024-04-22T13:14:30.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.908+0000 7f92677fe700 1 -- 172.21.15.12:0/1327557057 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f926c024890 con 0x7f9270106e30 2024-04-22T13:14:30.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.908+0000 7f92677fe700 1 --2- 172.21.15.12:0/1327557057 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f925c039560 0x7f925c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:30.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.908+0000 7f92677fe700 1 -- 172.21.15.12:0/1327557057 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f926c04da30 con 0x7f9270106e30 2024-04-22T13:14:30.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.908+0000 7f92759e1700 1 --2- 172.21.15.12:0/1327557057 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f925c039560 0x7f925c03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:30.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.909+0000 7f92759e1700 1 --2- 172.21.15.12:0/1327557057 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f925c039560 0x7f925c03ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9260006fd0 tx=0x7f9260006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:30.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:28.913+0000 7f92677fe700 1 -- 172.21.15.12:0/1327557057 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f926c04d450 con 0x7f9270106e30 2024-04-22T13:14:30.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:29.181+0000 7f9278446700 1 -- 172.21.15.12:0/1327557057 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f92700008d0 con 0x7f9270106e30 2024-04-22T13:14:30.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:29.479+0000 7f92677fe700 1 -- 172.21.15.12:0/1327557057 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f926c00bc70 con 0x7f9270106e30 2024-04-22T13:14:30.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:29.481+0000 7f92677fe700 1 -- 172.21.15.12:0/1327557057 <== mon.0 v2:172.21.15.12:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f926c02d910 con 0x7f9270106e30 2024-04-22T13:14:30.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:29.482+0000 7f9278446700 1 -- 172.21.15.12:0/1327557057 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f925c039560 msgr2=0x7f925c03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:30.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:29.482+0000 7f9278446700 1 --2- 172.21.15.12:0/1327557057 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f925c039560 0x7f925c03ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9260006fd0 tx=0x7f9260006e40 comp rx=0 tx=0).stop 2024-04-22T13:14:30.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:29.482+0000 7f9278446700 1 -- 172.21.15.12:0/1327557057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9270106e30 msgr2=0x7f927019e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:30.071 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:29.482+0000 7f9278446700 1 --2- 172.21.15.12:0/1327557057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9270106e30 0x7f927019e6a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f926c011ac0 tx=0x7f926c011ba0 comp rx=0 tx=0).stop 2024-04-22T13:14:30.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:29.482+0000 7f9278446700 1 -- 172.21.15.12:0/1327557057 shutdown_connections 2024-04-22T13:14:30.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:29.482+0000 7f9278446700 1 --2- 172.21.15.12:0/1327557057 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f925c039560 0x7f925c03ba20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:30.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:29.482+0000 7f9278446700 1 --2- 172.21.15.12:0/1327557057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9270106e30 0x7f927019e6a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:30.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:29.482+0000 7f9278446700 1 -- 172.21.15.12:0/1327557057 >> 172.21.15.12:0/1327557057 conn(0x7f9270102a50 msgr2=0x7f9270195340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:30.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:29.482+0000 7f9278446700 1 -- 172.21.15.12:0/1327557057 shutdown_connections 2024-04-22T13:14:30.072 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:29.482+0000 7f9278446700 1 -- 172.21.15.12:0/1327557057 wait complete. 2024-04-22T13:14:30.311 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:30 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:30.274+0000 7f7537b52200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T13:14:30.712 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:30 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:30.365+0000 7f7537b52200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T13:14:30.713 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:30 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1327557057' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-22T13:14:30.713 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:30 smithi012 ceph-mon[28265]: mgrmap e5: x(active, since 6s) 2024-04-22T13:14:31.400 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-04-22T13:14:31.401 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-22T13:14:31.401 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-22T13:14:31.401 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-22T13:14:31.401 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-22T13:14:31.401 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-04-22T13:14:31.401 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.508+0000 7f15d4941700 1 Processor -- start 2024-04-22T13:14:31.401 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.508+0000 7f15d4941700 1 -- start start 2024-04-22T13:14:31.401 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.509+0000 7f15d4941700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15d0108570 0x7f15d0108990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:31.401 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.509+0000 7f15d4941700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f15d0108f60 con 0x7f15d0108570 2024-04-22T13:14:31.401 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.509+0000 7f15ce59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15d0108570 0x7f15d0108990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:31.401 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.509+0000 7f15ce59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15d0108570 0x7f15d0108990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60666/0 (socket says 172.21.15.12:60666) 2024-04-22T13:14:31.401 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.509+0000 7f15ce59c700 1 -- 172.21.15.12:0/1123294913 learned_addr learned my addr 172.21.15.12:0/1123294913 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:31.401 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.509+0000 7f15ce59c700 1 -- 172.21.15.12:0/1123294913 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d0109720 con 0x7f15d0108570 2024-04-22T13:14:31.402 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.509+0000 7f15ce59c700 1 --2- 172.21.15.12:0/1123294913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15d0108570 0x7f15d0108990 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f15c40098a0 tx=0x7f15c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe5eef459c0edb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:31.402 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.509+0000 7f15cd59a700 1 -- 172.21.15.12:0/1123294913 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15c4004160 con 0x7f15d0108570 2024-04-22T13:14:31.402 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.509+0000 7f15cd59a700 1 -- 172.21.15.12:0/1123294913 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f15c4004300 con 0x7f15d0108570 2024-04-22T13:14:31.402 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.509+0000 7f15cd59a700 1 -- 172.21.15.12:0/1123294913 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15c4011af0 con 0x7f15d0108570 2024-04-22T13:14:31.402 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.510+0000 7f15d4941700 1 -- 172.21.15.12:0/1123294913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15d0108570 msgr2=0x7f15d0108990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:31.402 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.510+0000 7f15d4941700 1 --2- 172.21.15.12:0/1123294913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15d0108570 0x7f15d0108990 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f15c40098a0 tx=0x7f15c4009bb0 comp rx=0 tx=0).stop 2024-04-22T13:14:31.402 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.510+0000 7f15d4941700 1 -- 172.21.15.12:0/1123294913 shutdown_connections 2024-04-22T13:14:31.402 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.510+0000 7f15d4941700 1 --2- 172.21.15.12:0/1123294913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15d0108570 0x7f15d0108990 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:31.402 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.510+0000 7f15d4941700 1 -- 172.21.15.12:0/1123294913 >> 172.21.15.12:0/1123294913 conn(0x7f15d0103b70 msgr2=0x7f15d0105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:31.402 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.510+0000 7f15d4941700 1 -- 172.21.15.12:0/1123294913 shutdown_connections 2024-04-22T13:14:31.402 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.510+0000 7f15d4941700 1 -- 172.21.15.12:0/1123294913 wait complete. 2024-04-22T13:14:31.402 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.511+0000 7f15d4941700 1 Processor -- start 2024-04-22T13:14:31.403 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.511+0000 7f15d4941700 1 -- start start 2024-04-22T13:14:31.403 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.511+0000 7f15d4941700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15d0108570 0x7f15d007db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:31.403 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.511+0000 7f15d4941700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f15d010cad0 con 0x7f15d0108570 2024-04-22T13:14:31.403 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.511+0000 7f15ce59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15d0108570 0x7f15d007db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:31.403 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.511+0000 7f15ce59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15d0108570 0x7f15d007db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60680/0 (socket says 172.21.15.12:60680) 2024-04-22T13:14:31.403 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.511+0000 7f15ce59c700 1 -- 172.21.15.12:0/580852666 learned_addr learned my addr 172.21.15.12:0/580852666 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:31.403 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.512+0000 7f15ce59c700 1 -- 172.21.15.12:0/580852666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d007e0c0 con 0x7f15d0108570 2024-04-22T13:14:31.403 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.512+0000 7f15ce59c700 1 --2- 172.21.15.12:0/580852666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15d0108570 0x7f15d007db80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f15c40117b0 tx=0x7f15c4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:31.403 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.512+0000 7f15bf7fe700 1 -- 172.21.15.12:0/580852666 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15c4003af0 con 0x7f15d0108570 2024-04-22T13:14:31.403 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.512+0000 7f15d4941700 1 -- 172.21.15.12:0/580852666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d007c2b0 con 0x7f15d0108570 2024-04-22T13:14:31.403 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.512+0000 7f15bf7fe700 1 -- 172.21.15.12:0/580852666 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f15c4020070 con 0x7f15d0108570 2024-04-22T13:14:31.403 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.512+0000 7f15bf7fe700 1 -- 172.21.15.12:0/580852666 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15c4019630 con 0x7f15d0108570 2024-04-22T13:14:31.403 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.512+0000 7f15d4941700 1 -- 172.21.15.12:0/580852666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d007c7f0 con 0x7f15d0108570 2024-04-22T13:14:31.403 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.513+0000 7f15d4941700 1 -- 172.21.15.12:0/580852666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d00640a0 con 0x7f15d0108570 2024-04-22T13:14:31.404 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.513+0000 7f15bf7fe700 1 -- 172.21.15.12:0/580852666 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f15c40198f0 con 0x7f15d0108570 2024-04-22T13:14:31.404 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.513+0000 7f15bf7fe700 1 --2- 172.21.15.12:0/580852666 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f15b80395b0 0x7f15b803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:31.404 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.513+0000 7f15bf7fe700 1 -- 172.21.15.12:0/580852666 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f15c404dca0 con 0x7f15d0108570 2024-04-22T13:14:31.404 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.513+0000 7f15cdd9b700 1 -- 172.21.15.12:0/580852666 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f15b80395b0 msgr2=0x7f15b803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2091570832 2024-04-22T13:14:31.404 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.513+0000 7f15cdd9b700 1 --2- 172.21.15.12:0/580852666 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f15b80395b0 0x7f15b803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T13:14:31.404 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.524+0000 7f15bf7fe700 1 -- 172.21.15.12:0/580852666 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f15c40042a0 con 0x7f15d0108570 2024-04-22T13:14:31.404 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.714+0000 7f15cdd9b700 1 -- 172.21.15.12:0/580852666 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f15b80395b0 msgr2=0x7f15b803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2091570832 2024-04-22T13:14:31.404 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.714+0000 7f15cdd9b700 1 --2- 172.21.15.12:0/580852666 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f15b80395b0 0x7f15b803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T13:14:31.404 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.839+0000 7f15d4941700 1 -- 172.21.15.12:0/580852666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f15d007cfe0 con 0x7f15d0108570 2024-04-22T13:14:31.404 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.839+0000 7f15bf7fe700 1 -- 172.21.15.12:0/580852666 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f15c404d6c0 con 0x7f15d0108570 2024-04-22T13:14:31.404 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.841+0000 7f15d4941700 1 -- 172.21.15.12:0/580852666 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f15b80395b0 msgr2=0x7f15b803ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T13:14:31.404 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.841+0000 7f15d4941700 1 --2- 172.21.15.12:0/580852666 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f15b80395b0 0x7f15b803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:31.404 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.841+0000 7f15d4941700 1 -- 172.21.15.12:0/580852666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15d0108570 msgr2=0x7f15d007db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:31.405 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.841+0000 7f15d4941700 1 --2- 172.21.15.12:0/580852666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15d0108570 0x7f15d007db80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f15c40117b0 tx=0x7f15c4011b70 comp rx=0 tx=0).stop 2024-04-22T13:14:31.405 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.841+0000 7f15d4941700 1 -- 172.21.15.12:0/580852666 shutdown_connections 2024-04-22T13:14:31.405 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.841+0000 7f15d4941700 1 --2- 172.21.15.12:0/580852666 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f15b80395b0 0x7f15b803ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:31.405 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.842+0000 7f15d4941700 1 --2- 172.21.15.12:0/580852666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f15d0108570 0x7f15d007db80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:31.405 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.842+0000 7f15d4941700 1 -- 172.21.15.12:0/580852666 >> 172.21.15.12:0/580852666 conn(0x7f15d0103b70 msgr2=0x7f15d0104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:31.405 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.842+0000 7f15d4941700 1 -- 172.21.15.12:0/580852666 shutdown_connections 2024-04-22T13:14:31.405 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:30.842+0000 7f15d4941700 1 -- 172.21.15.12:0/580852666 wait complete. 2024-04-22T13:14:31.405 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for the mgr to restart... 2024-04-22T13:14:31.405 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr epoch 5... 2024-04-22T13:14:31.809 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:31 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/580852666' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-22T13:14:31.810 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:31 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:31.632+0000 7f7537b52200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T13:14:31.810 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:31 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:31.810+0000 7f7537b52200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T13:14:32.089 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:31 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:31.992+0000 7f7537b52200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T13:14:32.589 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:32 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:32.296+0000 7f7537b52200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T13:14:32.589 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:32 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:32.392+0000 7f7537b52200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T13:14:33.089 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:32 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:32.603+0000 7f7537b52200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T13:14:33.089 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:32 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:32.804+0000 7f7537b52200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:14:33.839 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:33 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:33.431+0000 7f7537b52200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T13:14:34.181 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:33 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:33.871+0000 7f7537b52200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T13:14:34.181 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:33 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:33.971+0000 7f7537b52200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T13:14:34.181 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:34 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:34.182+0000 7f7537b52200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T13:14:35.047 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:34 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:34.726+0000 7f7537b52200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T13:14:35.047 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:34 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:34.822+0000 7f7537b52200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T13:14:35.314 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:35 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:35.076+0000 7f7537b52200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T13:14:35.314 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:35 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:35.215+0000 7f7537b52200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T13:14:35.315 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:35 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:35.314+0000 7f7537b52200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T13:14:35.590 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:35 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:35.435+0000 7f7537b52200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T13:14:36.089 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:35 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:35.746+0000 7f7537b52200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T13:14:36.653 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:36 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:36.375+0000 7f7537b52200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T13:14:36.653 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:36 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:36.601+0000 7f7537b52200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T13:14:37.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:36 smithi012 ceph-mon[28265]: Active manager daemon x restarted 2024-04-22T13:14:37.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:36 smithi012 ceph-mon[28265]: Activating manager daemon x 2024-04-22T13:14:37.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:36 smithi012 ceph-mon[28265]: osdmap e2: 0 total, 0 up, 0 in 2024-04-22T13:14:37.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:36 smithi012 ceph-mon[28265]: mgrmap e6: x(active, starting, since 0.00332097s) 2024-04-22T13:14:37.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:36 smithi012 ceph-mon[28265]: from='mgr.14120 172.21.15.12:0/2157619606' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:14:37.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:36 smithi012 ceph-mon[28265]: from='mgr.14120 172.21.15.12:0/2157619606' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-22T13:14:37.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:36 smithi012 ceph-mon[28265]: from='mgr.14120 172.21.15.12:0/2157619606' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T13:14:37.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:36 smithi012 ceph-mon[28265]: from='mgr.14120 172.21.15.12:0/2157619606' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T13:14:37.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:36 smithi012 ceph-mon[28265]: from='mgr.14120 172.21.15.12:0/2157619606' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T13:14:37.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:36 smithi012 ceph-mon[28265]: Manager daemon x is now available 2024-04-22T13:14:37.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:36 smithi012 ceph-mon[28265]: from='mgr.14120 172.21.15.12:0/2157619606' entity='mgr.x' 2024-04-22T13:14:37.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:36 smithi012 ceph-mon[28265]: from='mgr.14120 172.21.15.12:0/2157619606' entity='mgr.x' 2024-04-22T13:14:37.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:36 smithi012 ceph-mon[28265]: from='mgr.14120 172.21.15.12:0/2157619606' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:14:37.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:36 smithi012 ceph-mon[28265]: from='mgr.14120 172.21.15.12:0/2157619606' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:14:37.904 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:37 smithi012 ceph-mon[28265]: Found migration_current of "None". Setting to last migration. 2024-04-22T13:14:37.904 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:37 smithi012 ceph-mon[28265]: from='mgr.14120 172.21.15.12:0/2157619606' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-22T13:14:37.905 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:37 smithi012 ceph-mon[28265]: from='mgr.14120 172.21.15.12:0/2157619606' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-22T13:14:37.905 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:37 smithi012 ceph-mon[28265]: mgrmap e7: x(active, since 1.00509s) 2024-04-22T13:14:38.181 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-04-22T13:14:38.182 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-22T13:14:38.182 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-22T13:14:38.182 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-04-22T13:14:38.182 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.039+0000 7f4ac7fe0700 1 Processor -- start 2024-04-22T13:14:38.182 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.040+0000 7f4ac7fe0700 1 -- start start 2024-04-22T13:14:38.182 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.040+0000 7f4ac7fe0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ac0108fb0 0x7f4ac010b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:38.182 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.040+0000 7f4ac7fe0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac0063d60 con 0x7f4ac0108fb0 2024-04-22T13:14:38.182 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.040+0000 7f4ac5d7c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ac0108fb0 0x7f4ac010b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:38.182 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.040+0000 7f4ac5d7c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ac0108fb0 0x7f4ac010b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42188/0 (socket says 172.21.15.12:42188) 2024-04-22T13:14:38.182 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.040+0000 7f4ac5d7c700 1 -- 172.21.15.12:0/1137846093 learned_addr learned my addr 172.21.15.12:0/1137846093 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:38.182 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.040+0000 7f4ac5d7c700 1 -- 172.21.15.12:0/1137846093 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac010b8e0 con 0x7f4ac0108fb0 2024-04-22T13:14:38.182 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.041+0000 7f4ac5d7c700 1 --2- 172.21.15.12:0/1137846093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ac0108fb0 0x7f4ac010b3a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4abc0098a0 tx=0x7f4abc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85992701fd3da129 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:38.182 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.041+0000 7f4ac4d7a700 1 -- 172.21.15.12:0/1137846093 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4abc004160 con 0x7f4ac0108fb0 2024-04-22T13:14:38.183 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.041+0000 7f4ac4d7a700 1 -- 172.21.15.12:0/1137846093 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4abc004300 con 0x7f4ac0108fb0 2024-04-22T13:14:38.183 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.041+0000 7f4ac4d7a700 1 -- 172.21.15.12:0/1137846093 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4abc011af0 con 0x7f4ac0108fb0 2024-04-22T13:14:38.183 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.041+0000 7f4ac7fe0700 1 -- 172.21.15.12:0/1137846093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ac0108fb0 msgr2=0x7f4ac010b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:38.183 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.041+0000 7f4ac7fe0700 1 --2- 172.21.15.12:0/1137846093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ac0108fb0 0x7f4ac010b3a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4abc0098a0 tx=0x7f4abc009bb0 comp rx=0 tx=0).stop 2024-04-22T13:14:38.183 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.041+0000 7f4ac7fe0700 1 -- 172.21.15.12:0/1137846093 shutdown_connections 2024-04-22T13:14:38.183 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.042+0000 7f4ac7fe0700 1 --2- 172.21.15.12:0/1137846093 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ac0108fb0 0x7f4ac010b3a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:38.183 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.042+0000 7f4ac7fe0700 1 -- 172.21.15.12:0/1137846093 >> 172.21.15.12:0/1137846093 conn(0x7f4ac0102a40 msgr2=0x7f4ac0104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:38.183 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.042+0000 7f4ac7fe0700 1 -- 172.21.15.12:0/1137846093 shutdown_connections 2024-04-22T13:14:38.183 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.042+0000 7f4ac7fe0700 1 -- 172.21.15.12:0/1137846093 wait complete. 2024-04-22T13:14:38.183 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.042+0000 7f4ac7fe0700 1 Processor -- start 2024-04-22T13:14:38.183 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.042+0000 7f4ac7fe0700 1 -- start start 2024-04-22T13:14:38.183 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.042+0000 7f4ac7fe0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ac0108fb0 0x7f4ac019c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:38.183 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.043+0000 7f4ac7fe0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac010c8b0 con 0x7f4ac0108fb0 2024-04-22T13:14:38.184 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.043+0000 7f4ac5d7c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ac0108fb0 0x7f4ac019c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:38.184 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.043+0000 7f4ac5d7c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ac0108fb0 0x7f4ac019c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42198/0 (socket says 172.21.15.12:42198) 2024-04-22T13:14:38.184 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.043+0000 7f4ac5d7c700 1 -- 172.21.15.12:0/806476552 learned_addr learned my addr 172.21.15.12:0/806476552 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:38.184 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.043+0000 7f4ac5d7c700 1 -- 172.21.15.12:0/806476552 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac019c990 con 0x7f4ac0108fb0 2024-04-22T13:14:38.184 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.043+0000 7f4ac5d7c700 1 --2- 172.21.15.12:0/806476552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ac0108fb0 0x7f4ac019c450 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4abc011ac0 tx=0x7f4abc011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:38.184 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.043+0000 7f4ab2ffd700 1 -- 172.21.15.12:0/806476552 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4abc003b20 con 0x7f4ac0108fb0 2024-04-22T13:14:38.184 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.043+0000 7f4ab2ffd700 1 -- 172.21.15.12:0/806476552 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4abc01a8c0 con 0x7f4ac0108fb0 2024-04-22T13:14:38.184 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.043+0000 7f4ac7fe0700 1 -- 172.21.15.12:0/806476552 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ac019cc20 con 0x7f4ac0108fb0 2024-04-22T13:14:38.184 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.043+0000 7f4ab2ffd700 1 -- 172.21.15.12:0/806476552 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4abc0248d0 con 0x7f4ac0108fb0 2024-04-22T13:14:38.184 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.043+0000 7f4ac7fe0700 1 -- 172.21.15.12:0/806476552 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ac019f910 con 0x7f4ac0108fb0 2024-04-22T13:14:38.184 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.044+0000 7f4ab2ffd700 1 -- 172.21.15.12:0/806476552 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f4abc022020 con 0x7f4ac0108fb0 2024-04-22T13:14:38.184 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.044+0000 7f4ab2ffd700 1 --2- 172.21.15.12:0/806476552 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f4aac0395b0 0x7f4aac03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:38.184 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.044+0000 7f4ac7fe0700 1 -- 172.21.15.12:0/806476552 --> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4ac0069ba0 con 0x7f4aac0395b0 2024-04-22T13:14:38.184 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.044+0000 7f4ab2ffd700 1 -- 172.21.15.12:0/806476552 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4abc04d500 con 0x7f4ac0108fb0 2024-04-22T13:14:38.185 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.044+0000 7f4ac557b700 1 -- 172.21.15.12:0/806476552 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f4aac0395b0 msgr2=0x7f4aac03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2091570832 2024-04-22T13:14:38.185 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.044+0000 7f4ac557b700 1 --2- 172.21.15.12:0/806476552 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f4aac0395b0 0x7f4aac03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T13:14:38.185 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.245+0000 7f4ac557b700 1 -- 172.21.15.12:0/806476552 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f4aac0395b0 msgr2=0x7f4aac03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2091570832 2024-04-22T13:14:38.185 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.245+0000 7f4ac557b700 1 --2- 172.21.15.12:0/806476552 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f4aac0395b0 0x7f4aac03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T13:14:38.185 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.646+0000 7f4ac557b700 1 -- 172.21.15.12:0/806476552 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f4aac0395b0 msgr2=0x7f4aac03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2091570832 2024-04-22T13:14:38.185 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:32.646+0000 7f4ac557b700 1 --2- 172.21.15.12:0/806476552 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f4aac0395b0 0x7f4aac03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T13:14:38.185 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:33.447+0000 7f4ac557b700 1 -- 172.21.15.12:0/806476552 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f4aac0395b0 msgr2=0x7f4aac03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2091570832 2024-04-22T13:14:38.185 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:33.447+0000 7f4ac557b700 1 --2- 172.21.15.12:0/806476552 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f4aac0395b0 0x7f4aac03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T13:14:38.185 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:35.048+0000 7f4ac557b700 1 -- 172.21.15.12:0/806476552 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f4aac0395b0 msgr2=0x7f4aac03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2091570832 2024-04-22T13:14:38.185 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:35.048+0000 7f4ac557b700 1 --2- 172.21.15.12:0/806476552 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f4aac0395b0 0x7f4aac03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-22T13:14:38.185 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:36.605+0000 7f4ab2ffd700 1 -- 172.21.15.12:0/806476552 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f4abc02d3f0 con 0x7f4ac0108fb0 2024-04-22T13:14:38.185 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:36.605+0000 7f4ab2ffd700 1 -- 172.21.15.12:0/806476552 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f4aac0395b0 msgr2=0x7f4aac03ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T13:14:38.185 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:36.605+0000 7f4ab2ffd700 1 --2- 172.21.15.12:0/806476552 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f4aac0395b0 0x7f4aac03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:38.185 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:37.606+0000 7f4ab2ffd700 1 -- 172.21.15.12:0/806476552 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f4abc04f320 con 0x7f4ac0108fb0 2024-04-22T13:14:38.186 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:37.606+0000 7f4ab2ffd700 1 --2- 172.21.15.12:0/806476552 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f4aac03ccb0 0x7f4aac03f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:38.186 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:37.606+0000 7f4ab2ffd700 1 -- 172.21.15.12:0/806476552 --> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4abc012ea0 con 0x7f4aac03ccb0 2024-04-22T13:14:38.186 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:37.607+0000 7f4ac557b700 1 --2- 172.21.15.12:0/806476552 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f4aac03ccb0 0x7f4aac03f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:38.186 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:37.608+0000 7f4ac557b700 1 --2- 172.21.15.12:0/806476552 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f4aac03ccb0 0x7f4aac03f0a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4003a10 tx=0x7f4ab4009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:38.186 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:37.608+0000 7f4ab2ffd700 1 -- 172.21.15.12:0/806476552 <== mgr.14120 v2:172.21.15.12:6800/2009831940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f4abc012ea0 con 0x7f4aac03ccb0 2024-04-22T13:14:38.186 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:37.615+0000 7f4ac7fe0700 1 -- 172.21.15.12:0/806476552 --> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f4ac0103c20 con 0x7f4aac03ccb0 2024-04-22T13:14:38.186 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:37.615+0000 7f4ab2ffd700 1 -- 172.21.15.12:0/806476552 <== mgr.14120 v2:172.21.15.12:6800/2009831940 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f4ac0103c20 con 0x7f4aac03ccb0 2024-04-22T13:14:38.186 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:37.615+0000 7f4ac7fe0700 1 -- 172.21.15.12:0/806476552 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f4aac03ccb0 msgr2=0x7f4aac03f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:38.186 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:37.615+0000 7f4ac7fe0700 1 --2- 172.21.15.12:0/806476552 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f4aac03ccb0 0x7f4aac03f0a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4003a10 tx=0x7f4ab4009290 comp rx=0 tx=0).stop 2024-04-22T13:14:38.186 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:37.615+0000 7f4ac7fe0700 1 -- 172.21.15.12:0/806476552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ac0108fb0 msgr2=0x7f4ac019c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:38.186 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:37.615+0000 7f4ac7fe0700 1 --2- 172.21.15.12:0/806476552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ac0108fb0 0x7f4ac019c450 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4abc011ac0 tx=0x7f4abc011ba0 comp rx=0 tx=0).stop 2024-04-22T13:14:38.186 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:37.615+0000 7f4ac7fe0700 1 -- 172.21.15.12:0/806476552 shutdown_connections 2024-04-22T13:14:38.187 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:37.615+0000 7f4ac7fe0700 1 --2- 172.21.15.12:0/806476552 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f4aac03ccb0 0x7f4aac03f0a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:38.187 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:37.615+0000 7f4ac7fe0700 1 --2- 172.21.15.12:0/806476552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ac0108fb0 0x7f4ac019c450 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:38.187 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:37.615+0000 7f4ac7fe0700 1 --2- 172.21.15.12:0/806476552 >> [v2:172.21.15.12:6800/2091570832,v1:172.21.15.12:6801/2091570832] conn(0x7f4aac0395b0 0x7f4aac03ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:38.187 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:37.615+0000 7f4ac7fe0700 1 -- 172.21.15.12:0/806476552 >> 172.21.15.12:0/806476552 conn(0x7f4ac0102a40 msgr2=0x7f4ac0103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:38.187 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:37.615+0000 7f4ac7fe0700 1 -- 172.21.15.12:0/806476552 shutdown_connections 2024-04-22T13:14:38.187 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:37.615+0000 7f4ac7fe0700 1 -- 172.21.15.12:0/806476552 wait complete. 2024-04-22T13:14:38.187 INFO:teuthology.orchestra.run.smithi012.stdout:mgr epoch 5 is available 2024-04-22T13:14:38.187 INFO:teuthology.orchestra.run.smithi012.stdout:Setting orchestrator backend to cephadm... 2024-04-22T13:14:38.796 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:38 smithi012 ceph-mon[28265]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-22T13:14:38.796 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:38 smithi012 ceph-mon[28265]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-22T13:14:38.796 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:38 smithi012 ceph-mon[28265]: from='mgr.14120 172.21.15.12:0/2157619606' entity='mgr.x' 2024-04-22T13:14:38.796 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:38 smithi012 ceph-mon[28265]: from='mgr.14120 172.21.15.12:0/2157619606' entity='mgr.x' 2024-04-22T13:14:39.385 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.562+0000 7f31001dd700 1 Processor -- start 2024-04-22T13:14:39.385 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.562+0000 7f31001dd700 1 -- start start 2024-04-22T13:14:39.385 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.562+0000 7f31001dd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30f807cde0 0x7f30f807b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:39.385 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.562+0000 7f31001dd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f30f807b810 con 0x7f30f807cde0 2024-04-22T13:14:39.385 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.563+0000 7f30fdf79700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30f807cde0 0x7f30f807b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:39.385 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.563+0000 7f30fdf79700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30f807cde0 0x7f30f807b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42256/0 (socket says 172.21.15.12:42256) 2024-04-22T13:14:39.385 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.563+0000 7f30fdf79700 1 -- 172.21.15.12:0/2439803439 learned_addr learned my addr 172.21.15.12:0/2439803439 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:39.386 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.563+0000 7f30fdf79700 1 -- 172.21.15.12:0/2439803439 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f807b990 con 0x7f30f807cde0 2024-04-22T13:14:39.386 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.563+0000 7f30fdf79700 1 --2- 172.21.15.12:0/2439803439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30f807cde0 0x7f30f807b240 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f30e80098a0 tx=0x7f30e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4410a64c3521f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:39.386 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.563+0000 7f30fcf77700 1 -- 172.21.15.12:0/2439803439 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30e8004160 con 0x7f30f807cde0 2024-04-22T13:14:39.386 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.563+0000 7f30fcf77700 1 -- 172.21.15.12:0/2439803439 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f30e8004300 con 0x7f30f807cde0 2024-04-22T13:14:39.386 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.563+0000 7f30fcf77700 1 -- 172.21.15.12:0/2439803439 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30e8011af0 con 0x7f30f807cde0 2024-04-22T13:14:39.386 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.564+0000 7f31001dd700 1 -- 172.21.15.12:0/2439803439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30f807cde0 msgr2=0x7f30f807b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:39.386 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.564+0000 7f31001dd700 1 --2- 172.21.15.12:0/2439803439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30f807cde0 0x7f30f807b240 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f30e80098a0 tx=0x7f30e8009bb0 comp rx=0 tx=0).stop 2024-04-22T13:14:39.386 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.564+0000 7f31001dd700 1 -- 172.21.15.12:0/2439803439 shutdown_connections 2024-04-22T13:14:39.386 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.564+0000 7f31001dd700 1 --2- 172.21.15.12:0/2439803439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30f807cde0 0x7f30f807b240 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:39.386 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.564+0000 7f31001dd700 1 -- 172.21.15.12:0/2439803439 >> 172.21.15.12:0/2439803439 conn(0x7f30f8103b50 msgr2=0x7f30f8105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:39.386 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.564+0000 7f31001dd700 1 -- 172.21.15.12:0/2439803439 shutdown_connections 2024-04-22T13:14:39.387 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.564+0000 7f31001dd700 1 -- 172.21.15.12:0/2439803439 wait complete. 2024-04-22T13:14:39.387 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.565+0000 7f31001dd700 1 Processor -- start 2024-04-22T13:14:39.387 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.565+0000 7f31001dd700 1 -- start start 2024-04-22T13:14:39.387 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.565+0000 7f31001dd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30f807cde0 0x7f30f819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:39.387 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.565+0000 7f31001dd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f30f810eaf0 con 0x7f30f807cde0 2024-04-22T13:14:39.387 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.565+0000 7f30fdf79700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30f807cde0 0x7f30f819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:39.387 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.565+0000 7f30fdf79700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30f807cde0 0x7f30f819e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42264/0 (socket says 172.21.15.12:42264) 2024-04-22T13:14:39.387 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.565+0000 7f30fdf79700 1 -- 172.21.15.12:0/107763921 learned_addr learned my addr 172.21.15.12:0/107763921 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:39.387 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.565+0000 7f30fdf79700 1 -- 172.21.15.12:0/107763921 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f819ebc0 con 0x7f30f807cde0 2024-04-22T13:14:39.387 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.566+0000 7f30fdf79700 1 --2- 172.21.15.12:0/107763921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30f807cde0 0x7f30f819e680 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f30e8003f40 tx=0x7f30e8011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:39.387 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.566+0000 7f30eeffd700 1 -- 172.21.15.12:0/107763921 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30e80038b0 con 0x7f30f807cde0 2024-04-22T13:14:39.387 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.566+0000 7f30eeffd700 1 -- 172.21.15.12:0/107763921 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f30e8003a50 con 0x7f30f807cde0 2024-04-22T13:14:39.387 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.566+0000 7f31001dd700 1 -- 172.21.15.12:0/107763921 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30f819ee50 con 0x7f30f807cde0 2024-04-22T13:14:39.388 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.566+0000 7f30eeffd700 1 -- 172.21.15.12:0/107763921 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30e801a5c0 con 0x7f30f807cde0 2024-04-22T13:14:39.388 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.566+0000 7f31001dd700 1 -- 172.21.15.12:0/107763921 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30f81a1b40 con 0x7f30f807cde0 2024-04-22T13:14:39.388 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.567+0000 7f30eeffd700 1 -- 172.21.15.12:0/107763921 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f30e801a870 con 0x7f30f807cde0 2024-04-22T13:14:39.388 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.567+0000 7f30eeffd700 1 --2- 172.21.15.12:0/107763921 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f30e40394a0 0x7f30e403b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:39.388 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.567+0000 7f30eeffd700 1 -- 172.21.15.12:0/107763921 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f30e804d8d0 con 0x7f30f807cde0 2024-04-22T13:14:39.388 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.567+0000 7f30fd778700 1 --2- 172.21.15.12:0/107763921 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f30e40394a0 0x7f30e403b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:39.388 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.567+0000 7f31001dd700 1 -- 172.21.15.12:0/107763921 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30f80640a0 con 0x7f30f807cde0 2024-04-22T13:14:39.388 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.567+0000 7f30fd778700 1 --2- 172.21.15.12:0/107763921 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f30e40394a0 0x7f30e403b960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f30f4006fd0 tx=0x7f30f4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:39.388 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.574+0000 7f30eeffd700 1 -- 172.21.15.12:0/107763921 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f30e800b3e0 con 0x7f30f807cde0 2024-04-22T13:14:39.388 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.797+0000 7f31001dd700 1 -- 172.21.15.12:0/107763921 --> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f30f8104d70 con 0x7f30e40394a0 2024-04-22T13:14:39.388 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.804+0000 7f30eeffd700 1 -- 172.21.15.12:0/107763921 <== mgr.14120 v2:172.21.15.12:6800/2009831940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f30f8104d70 con 0x7f30e40394a0 2024-04-22T13:14:39.388 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.806+0000 7f31001dd700 1 -- 172.21.15.12:0/107763921 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f30e40394a0 msgr2=0x7f30e403b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:39.388 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.806+0000 7f31001dd700 1 --2- 172.21.15.12:0/107763921 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f30e40394a0 0x7f30e403b960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f30f4006fd0 tx=0x7f30f4006e40 comp rx=0 tx=0).stop 2024-04-22T13:14:39.388 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.806+0000 7f31001dd700 1 -- 172.21.15.12:0/107763921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30f807cde0 msgr2=0x7f30f819e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:39.388 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.806+0000 7f31001dd700 1 --2- 172.21.15.12:0/107763921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30f807cde0 0x7f30f819e680 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f30e8003f40 tx=0x7f30e8011890 comp rx=0 tx=0).stop 2024-04-22T13:14:39.389 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.806+0000 7f31001dd700 1 -- 172.21.15.12:0/107763921 shutdown_connections 2024-04-22T13:14:39.389 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.806+0000 7f31001dd700 1 --2- 172.21.15.12:0/107763921 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f30e40394a0 0x7f30e403b960 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:39.389 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.806+0000 7f31001dd700 1 --2- 172.21.15.12:0/107763921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f30f807cde0 0x7f30f819e680 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:39.389 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.806+0000 7f31001dd700 1 -- 172.21.15.12:0/107763921 >> 172.21.15.12:0/107763921 conn(0x7f30f8103b50 msgr2=0x7f30f8104660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:39.389 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.807+0000 7f31001dd700 1 -- 172.21.15.12:0/107763921 shutdown_connections 2024-04-22T13:14:39.389 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:38.807+0000 7f31001dd700 1 -- 172.21.15.12:0/107763921 wait complete. 2024-04-22T13:14:39.896 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:39 smithi012 ceph-mon[28265]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:14:39.897 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:39 smithi012 ceph-mon[28265]: from='mgr.14120 172.21.15.12:0/2157619606' entity='mgr.x' 2024-04-22T13:14:39.897 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:39 smithi012 ceph-mon[28265]: from='mgr.14120 172.21.15.12:0/2157619606' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:14:39.897 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:39 smithi012 ceph-mon[28265]: mgrmap e8: x(active, since 2s) 2024-04-22T13:14:40.716 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-22T13:14:40.716 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.893+0000 7fb493c13700 1 Processor -- start 2024-04-22T13:14:40.716 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.893+0000 7fb493c13700 1 -- start start 2024-04-22T13:14:40.716 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.893+0000 7fb493c13700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb48c07de40 0x7fb48c07c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:40.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.893+0000 7fb493c13700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb48c07c830 con 0x7fb48c07de40 2024-04-22T13:14:40.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.893+0000 7fb4919af700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb48c07de40 0x7fb48c07c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:40.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.894+0000 7fb4919af700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb48c07de40 0x7fb48c07c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42276/0 (socket says 172.21.15.12:42276) 2024-04-22T13:14:40.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.894+0000 7fb4919af700 1 -- 172.21.15.12:0/525553669 learned_addr learned my addr 172.21.15.12:0/525553669 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:40.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.894+0000 7fb4919af700 1 -- 172.21.15.12:0/525553669 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48c07c9b0 con 0x7fb48c07de40 2024-04-22T13:14:40.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.894+0000 7fb4919af700 1 --2- 172.21.15.12:0/525553669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb48c07de40 0x7fb48c07c2f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb4880098a0 tx=0x7fb488009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f94e752536b7dfd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:40.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.894+0000 7fb4909ad700 1 -- 172.21.15.12:0/525553669 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb488004160 con 0x7fb48c07de40 2024-04-22T13:14:40.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.894+0000 7fb4909ad700 1 -- 172.21.15.12:0/525553669 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb488004300 con 0x7fb48c07de40 2024-04-22T13:14:40.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.894+0000 7fb4909ad700 1 -- 172.21.15.12:0/525553669 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb488011af0 con 0x7fb48c07de40 2024-04-22T13:14:40.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.895+0000 7fb493c13700 1 -- 172.21.15.12:0/525553669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb48c07de40 msgr2=0x7fb48c07c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:40.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.895+0000 7fb493c13700 1 --2- 172.21.15.12:0/525553669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb48c07de40 0x7fb48c07c2f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb4880098a0 tx=0x7fb488009bb0 comp rx=0 tx=0).stop 2024-04-22T13:14:40.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.895+0000 7fb493c13700 1 -- 172.21.15.12:0/525553669 shutdown_connections 2024-04-22T13:14:40.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.895+0000 7fb493c13700 1 --2- 172.21.15.12:0/525553669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb48c07de40 0x7fb48c07c2f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:40.717 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.895+0000 7fb493c13700 1 -- 172.21.15.12:0/525553669 >> 172.21.15.12:0/525553669 conn(0x7fb48c103b40 msgr2=0x7fb48c105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:40.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.895+0000 7fb493c13700 1 -- 172.21.15.12:0/525553669 shutdown_connections 2024-04-22T13:14:40.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.895+0000 7fb493c13700 1 -- 172.21.15.12:0/525553669 wait complete. 2024-04-22T13:14:40.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.895+0000 7fb493c13700 1 Processor -- start 2024-04-22T13:14:40.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.895+0000 7fb493c13700 1 -- start start 2024-04-22T13:14:40.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.896+0000 7fb493c13700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb48c07de40 0x7fb48c19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:40.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.896+0000 7fb493c13700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb48c10ead0 con 0x7fb48c07de40 2024-04-22T13:14:40.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.896+0000 7fb4919af700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb48c07de40 0x7fb48c19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:40.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.896+0000 7fb4919af700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb48c07de40 0x7fb48c19e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42280/0 (socket says 172.21.15.12:42280) 2024-04-22T13:14:40.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.896+0000 7fb4919af700 1 -- 172.21.15.12:0/1374221019 learned_addr learned my addr 172.21.15.12:0/1374221019 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:40.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.896+0000 7fb4919af700 1 -- 172.21.15.12:0/1374221019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48c19ec10 con 0x7fb48c07de40 2024-04-22T13:14:40.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.896+0000 7fb4919af700 1 --2- 172.21.15.12:0/1374221019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb48c07de40 0x7fb48c19e6d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb488009fb0 tx=0x7fb488011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:40.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.896+0000 7fb482ffd700 1 -- 172.21.15.12:0/1374221019 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4880036a0 con 0x7fb48c07de40 2024-04-22T13:14:40.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.896+0000 7fb482ffd700 1 -- 172.21.15.12:0/1374221019 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb48801a430 con 0x7fb48c07de40 2024-04-22T13:14:40.718 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.896+0000 7fb493c13700 1 -- 172.21.15.12:0/1374221019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb48c19eea0 con 0x7fb48c07de40 2024-04-22T13:14:40.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.896+0000 7fb482ffd700 1 -- 172.21.15.12:0/1374221019 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb488019550 con 0x7fb48c07de40 2024-04-22T13:14:40.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.897+0000 7fb493c13700 1 -- 172.21.15.12:0/1374221019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb48c1a1b90 con 0x7fb48c07de40 2024-04-22T13:14:40.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.897+0000 7fb482ffd700 1 -- 172.21.15.12:0/1374221019 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fb488019800 con 0x7fb48c07de40 2024-04-22T13:14:40.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.897+0000 7fb493c13700 1 -- 172.21.15.12:0/1374221019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb48c069ba0 con 0x7fb48c07de40 2024-04-22T13:14:40.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.898+0000 7fb482ffd700 1 --2- 172.21.15.12:0/1374221019 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fb4780395b0 0x7fb47803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:40.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.898+0000 7fb482ffd700 1 -- 172.21.15.12:0/1374221019 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb48804da70 con 0x7fb48c07de40 2024-04-22T13:14:40.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.898+0000 7fb4911ae700 1 --2- 172.21.15.12:0/1374221019 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fb4780395b0 0x7fb47803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:40.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.899+0000 7fb4911ae700 1 --2- 172.21.15.12:0/1374221019 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fb4780395b0 0x7fb47803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb47c006fd0 tx=0x7fb47c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:40.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:39.905+0000 7fb482ffd700 1 -- 172.21.15.12:0/1374221019 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb48802d960 con 0x7fb48c07de40 2024-04-22T13:14:40.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:40.114+0000 7fb493c13700 1 -- 172.21.15.12:0/1374221019 --> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fb48c104dd0 con 0x7fb4780395b0 2024-04-22T13:14:40.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:40.115+0000 7fb482ffd700 1 -- 172.21.15.12:0/1374221019 <== mgr.14120 v2:172.21.15.12:6800/2009831940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fb48c104dd0 con 0x7fb4780395b0 2024-04-22T13:14:40.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:40.117+0000 7fb493c13700 1 -- 172.21.15.12:0/1374221019 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fb4780395b0 msgr2=0x7fb47803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:40.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:40.117+0000 7fb493c13700 1 --2- 172.21.15.12:0/1374221019 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fb4780395b0 0x7fb47803ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb47c006fd0 tx=0x7fb47c006e40 comp rx=0 tx=0).stop 2024-04-22T13:14:40.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:40.117+0000 7fb493c13700 1 -- 172.21.15.12:0/1374221019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb48c07de40 msgr2=0x7fb48c19e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:40.719 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:40.117+0000 7fb493c13700 1 --2- 172.21.15.12:0/1374221019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb48c07de40 0x7fb48c19e6d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb488009fb0 tx=0x7fb488011940 comp rx=0 tx=0).stop 2024-04-22T13:14:40.720 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:40.117+0000 7fb493c13700 1 -- 172.21.15.12:0/1374221019 shutdown_connections 2024-04-22T13:14:40.720 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:40.117+0000 7fb493c13700 1 --2- 172.21.15.12:0/1374221019 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fb4780395b0 0x7fb47803ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:40.720 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:40.117+0000 7fb493c13700 1 --2- 172.21.15.12:0/1374221019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb48c07de40 0x7fb48c19e6d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:40.720 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:40.117+0000 7fb493c13700 1 -- 172.21.15.12:0/1374221019 >> 172.21.15.12:0/1374221019 conn(0x7fb48c103b40 msgr2=0x7fb48c1046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:40.720 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:40.117+0000 7fb493c13700 1 -- 172.21.15.12:0/1374221019 shutdown_connections 2024-04-22T13:14:40.720 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:40.117+0000 7fb493c13700 1 -- 172.21.15.12:0/1374221019 wait complete. 2024-04-22T13:14:40.720 INFO:teuthology.orchestra.run.smithi012.stdout:Generating ssh key... 2024-04-22T13:14:41.036 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:40 smithi012 ceph-mon[28265]: [22/Apr/2024:13:14:39] ENGINE Bus STARTING 2024-04-22T13:14:41.036 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:40 smithi012 ceph-mon[28265]: [22/Apr/2024:13:14:39] ENGINE Serving on https://172.21.15.12:7150 2024-04-22T13:14:41.036 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:40 smithi012 ceph-mon[28265]: [22/Apr/2024:13:14:39] ENGINE Serving on http://172.21.15.12:8765 2024-04-22T13:14:41.036 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:40 smithi012 ceph-mon[28265]: [22/Apr/2024:13:14:39] ENGINE Bus STARTED 2024-04-22T13:14:41.036 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:40 smithi012 ceph-mon[28265]: from='mgr.14120 172.21.15.12:0/2157619606' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:14:41.036 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:40 smithi012 ceph-mon[28265]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:14:42.022 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:41 smithi012 ceph-mon[28265]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:14:42.022 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:41 smithi012 ceph-mon[28265]: Generating ssh key... 2024-04-22T13:14:42.022 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:41 smithi012 ceph-mon[28265]: from='mgr.14120 172.21.15.12:0/2157619606' entity='mgr.x' 2024-04-22T13:14:42.022 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:41 smithi012 ceph-mon[28265]: from='mgr.14120 172.21.15.12:0/2157619606' entity='mgr.x' 2024-04-22T13:14:42.023 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:41 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: Generating public/private rsa key pair. 2024-04-22T13:14:42.023 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:41 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: Your identification has been saved in /tmp/tmp5ag02o7g/key. 2024-04-22T13:14:42.023 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:41 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: Your public key has been saved in /tmp/tmp5ag02o7g/key.pub. 2024-04-22T13:14:42.023 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:41 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: The key fingerprint is: 2024-04-22T13:14:42.023 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:41 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: SHA256:hrnGL+qFBUxvM7YooCyW7UY0apuyqWiwdIn8pgsZFBw ceph-0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:14:42.023 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:41 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: The key's randomart image is: 2024-04-22T13:14:42.023 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:41 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: +---[RSA 3072]----+ 2024-04-22T13:14:42.023 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:41 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: |.E. . | 2024-04-22T13:14:42.023 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:41 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: | ..o . | 2024-04-22T13:14:42.023 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:41 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: |.. oo * | 2024-04-22T13:14:42.023 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:41 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: |+.= .= * | 2024-04-22T13:14:42.023 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:41 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: |=*ooo = S | 2024-04-22T13:14:42.023 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:41 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: |=*=+ + o | 2024-04-22T13:14:42.023 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:41 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: |*+oo. = | 2024-04-22T13:14:42.024 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:41 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: |=+.o o.. | 2024-04-22T13:14:42.024 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:41 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: |Bo+.o. .. | 2024-04-22T13:14:42.024 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:41 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: +----[SHA256]-----+ 2024-04-22T13:14:42.306 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.142+0000 7f6de720a700 1 Processor -- start 2024-04-22T13:14:42.306 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.142+0000 7f6de720a700 1 -- start start 2024-04-22T13:14:42.306 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.142+0000 7f6de720a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6de0106e20 0x7f6de0107240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:42.306 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.142+0000 7f6de720a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6de007be40 con 0x7f6de0106e20 2024-04-22T13:14:42.306 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.142+0000 7f6de4fa6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6de0106e20 0x7f6de0107240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:42.306 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.142+0000 7f6de4fa6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6de0106e20 0x7f6de0107240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42286/0 (socket says 172.21.15.12:42286) 2024-04-22T13:14:42.306 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.142+0000 7f6de4fa6700 1 -- 172.21.15.12:0/552515460 learned_addr learned my addr 172.21.15.12:0/552515460 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:42.306 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.143+0000 7f6de4fa6700 1 -- 172.21.15.12:0/552515460 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de0107780 con 0x7f6de0106e20 2024-04-22T13:14:42.306 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.143+0000 7f6de4fa6700 1 --2- 172.21.15.12:0/552515460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6de0106e20 0x7f6de0107240 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6dd40098a0 tx=0x7f6dd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ace11ba12a5a7b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:42.307 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.143+0000 7f6ddf7fe700 1 -- 172.21.15.12:0/552515460 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6dd4004160 con 0x7f6de0106e20 2024-04-22T13:14:42.307 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.143+0000 7f6ddf7fe700 1 -- 172.21.15.12:0/552515460 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6dd4004300 con 0x7f6de0106e20 2024-04-22T13:14:42.307 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.143+0000 7f6ddf7fe700 1 -- 172.21.15.12:0/552515460 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6dd4011af0 con 0x7f6de0106e20 2024-04-22T13:14:42.307 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.143+0000 7f6de720a700 1 -- 172.21.15.12:0/552515460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6de0106e20 msgr2=0x7f6de0107240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:42.307 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.143+0000 7f6de720a700 1 --2- 172.21.15.12:0/552515460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6de0106e20 0x7f6de0107240 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6dd40098a0 tx=0x7f6dd4009bb0 comp rx=0 tx=0).stop 2024-04-22T13:14:42.307 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.144+0000 7f6de720a700 1 -- 172.21.15.12:0/552515460 shutdown_connections 2024-04-22T13:14:42.307 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.144+0000 7f6de720a700 1 --2- 172.21.15.12:0/552515460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6de0106e20 0x7f6de0107240 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:42.307 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.144+0000 7f6de720a700 1 -- 172.21.15.12:0/552515460 >> 172.21.15.12:0/552515460 conn(0x7f6de0102a40 msgr2=0x7f6de0104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:42.307 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.144+0000 7f6de720a700 1 -- 172.21.15.12:0/552515460 shutdown_connections 2024-04-22T13:14:42.308 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.144+0000 7f6de720a700 1 -- 172.21.15.12:0/552515460 wait complete. 2024-04-22T13:14:42.308 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.144+0000 7f6de720a700 1 Processor -- start 2024-04-22T13:14:42.308 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.144+0000 7f6de720a700 1 -- start start 2024-04-22T13:14:42.308 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.144+0000 7f6de720a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6de0106e20 0x7f6de019e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:42.308 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.144+0000 7f6de720a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6de01087b0 con 0x7f6de0106e20 2024-04-22T13:14:42.308 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.145+0000 7f6de4fa6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6de0106e20 0x7f6de019e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:42.308 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.145+0000 7f6de4fa6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6de0106e20 0x7f6de019e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42300/0 (socket says 172.21.15.12:42300) 2024-04-22T13:14:42.308 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.145+0000 7f6de4fa6700 1 -- 172.21.15.12:0/1893650523 learned_addr learned my addr 172.21.15.12:0/1893650523 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:42.308 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.145+0000 7f6de4fa6700 1 -- 172.21.15.12:0/1893650523 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de019ec00 con 0x7f6de0106e20 2024-04-22T13:14:42.308 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.145+0000 7f6de4fa6700 1 --2- 172.21.15.12:0/1893650523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6de0106e20 0x7f6de019e6c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009fb0 tx=0x7f6dd4011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:42.308 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.145+0000 7f6dddffb700 1 -- 172.21.15.12:0/1893650523 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6dd40036a0 con 0x7f6de0106e20 2024-04-22T13:14:42.308 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.145+0000 7f6dddffb700 1 -- 172.21.15.12:0/1893650523 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6dd401a430 con 0x7f6de0106e20 2024-04-22T13:14:42.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.145+0000 7f6dddffb700 1 -- 172.21.15.12:0/1893650523 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6dd4019550 con 0x7f6de0106e20 2024-04-22T13:14:42.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.145+0000 7f6de720a700 1 -- 172.21.15.12:0/1893650523 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de019ee90 con 0x7f6de0106e20 2024-04-22T13:14:42.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.145+0000 7f6de720a700 1 -- 172.21.15.12:0/1893650523 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de01a1b80 con 0x7f6de0106e20 2024-04-22T13:14:42.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.146+0000 7f6dddffb700 1 -- 172.21.15.12:0/1893650523 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f6dd401a8e0 con 0x7f6de0106e20 2024-04-22T13:14:42.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.146+0000 7f6de720a700 1 -- 172.21.15.12:0/1893650523 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6de00640a0 con 0x7f6de0106e20 2024-04-22T13:14:42.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.147+0000 7f6dddffb700 1 --2- 172.21.15.12:0/1893650523 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f6dd00395b0 0x7f6dd003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:42.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.147+0000 7f6dddffb700 1 -- 172.21.15.12:0/1893650523 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6dd404d8d0 con 0x7f6de0106e20 2024-04-22T13:14:42.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.147+0000 7f6ddffff700 1 --2- 172.21.15.12:0/1893650523 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f6dd00395b0 0x7f6dd003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:42.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.147+0000 7f6ddffff700 1 --2- 172.21.15.12:0/1893650523 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f6dd00395b0 0x7f6dd003ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc006fd0 tx=0x7f6dcc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:42.309 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.152+0000 7f6dddffb700 1 -- 172.21.15.12:0/1893650523 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6dd400b3e0 con 0x7f6de0106e20 2024-04-22T13:14:42.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.360+0000 7f6de720a700 1 -- 172.21.15.12:0/1893650523 --> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f6de00008d0 con 0x7f6dd00395b0 2024-04-22T13:14:42.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.678+0000 7f6dddffb700 1 -- 172.21.15.12:0/1893650523 <== mgr.14120 v2:172.21.15.12:6800/2009831940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6de00008d0 con 0x7f6dd00395b0 2024-04-22T13:14:42.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.680+0000 7f6de720a700 1 -- 172.21.15.12:0/1893650523 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f6dd00395b0 msgr2=0x7f6dd003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:42.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.680+0000 7f6de720a700 1 --2- 172.21.15.12:0/1893650523 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f6dd00395b0 0x7f6dd003ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc006fd0 tx=0x7f6dcc006e40 comp rx=0 tx=0).stop 2024-04-22T13:14:42.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.680+0000 7f6de720a700 1 -- 172.21.15.12:0/1893650523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6de0106e20 msgr2=0x7f6de019e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:42.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.680+0000 7f6de720a700 1 --2- 172.21.15.12:0/1893650523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6de0106e20 0x7f6de019e6c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009fb0 tx=0x7f6dd4011940 comp rx=0 tx=0).stop 2024-04-22T13:14:42.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.680+0000 7f6de720a700 1 -- 172.21.15.12:0/1893650523 shutdown_connections 2024-04-22T13:14:42.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.680+0000 7f6de720a700 1 --2- 172.21.15.12:0/1893650523 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f6dd00395b0 0x7f6dd003ba70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:42.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.680+0000 7f6de720a700 1 --2- 172.21.15.12:0/1893650523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6de0106e20 0x7f6de019e6c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:42.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.680+0000 7f6de720a700 1 -- 172.21.15.12:0/1893650523 >> 172.21.15.12:0/1893650523 conn(0x7f6de0102a40 msgr2=0x7f6de0195330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:42.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.680+0000 7f6de720a700 1 -- 172.21.15.12:0/1893650523 shutdown_connections 2024-04-22T13:14:42.310 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:41.680+0000 7f6de720a700 1 -- 172.21.15.12:0/1893650523 wait complete. 2024-04-22T13:14:43.553 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQLZu8FDpJVceSCKOLD/XtW0gKG+KKFHIpsZPld7ZT1NQs4U9PUXQ+91wgyjHvmBzPksEks09W/u8ddq4iy/ponZEvcCoIg3Cpoo0RG/o45Pu772m1KCXwXDkYWyWum7IwLy5gBQ6cqy8TvpT+u1ETzKdpNZPeMvHqg8UNdEjVKT20+4DznKohEw5r1NiwRl9jhrQRUZIOZILuooql79K2LELSUDaxU0bGG5NUNTiGGgfjF/y7xjgmsSmu6T3y8v3+ESwD7D9u0SioZhOW95mkYQ1iRGhcmVzzOzSqduzmjzFgygo6AixsH4bUimWX0+ZdXs7UreU7HKZ+4+h2tx+6AZ/qwhCpoO0zhwZd0NkQAG45yv79yxvZMqYeRY9yyvyJkqt/Kwibli80nTcZ4DmwsAZzeMLZuG+qmu/zdOYhqFU1Qrb/V5Ee2pu0LDYyTek0PfWR3SOijN5rZrglCv1wPAquvFkt/qeCNgSeVSJ9A6Z6tat8JgyEioCURLXTXgE= ceph-0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:14:43.553 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.730+0000 7f1842e5f700 1 Processor -- start 2024-04-22T13:14:43.554 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.730+0000 7f1842e5f700 1 -- start start 2024-04-22T13:14:43.554 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.730+0000 7f1842e5f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f183c07de80 0x7f183c07c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:43.554 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.730+0000 7f1842e5f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f183c07c900 con 0x7f183c07de80 2024-04-22T13:14:43.554 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.730+0000 7f1840bfb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f183c07de80 0x7f183c07c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:43.554 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.730+0000 7f1840bfb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f183c07de80 0x7f183c07c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54420/0 (socket says 172.21.15.12:54420) 2024-04-22T13:14:43.554 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.730+0000 7f1840bfb700 1 -- 172.21.15.12:0/2643686699 learned_addr learned my addr 172.21.15.12:0/2643686699 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:43.554 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.731+0000 7f1840bfb700 1 -- 172.21.15.12:0/2643686699 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183c07ca80 con 0x7f183c07de80 2024-04-22T13:14:43.554 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.731+0000 7f1840bfb700 1 --2- 172.21.15.12:0/2643686699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f183c07de80 0x7f183c07c330 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f18300098a0 tx=0x7f1830009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1192f91b49f56821 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:43.554 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.731+0000 7f183b7fe700 1 -- 172.21.15.12:0/2643686699 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1830004160 con 0x7f183c07de80 2024-04-22T13:14:43.554 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.731+0000 7f183b7fe700 1 -- 172.21.15.12:0/2643686699 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1830004300 con 0x7f183c07de80 2024-04-22T13:14:43.554 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.731+0000 7f183b7fe700 1 -- 172.21.15.12:0/2643686699 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1830011af0 con 0x7f183c07de80 2024-04-22T13:14:43.554 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.732+0000 7f1842e5f700 1 -- 172.21.15.12:0/2643686699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f183c07de80 msgr2=0x7f183c07c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:43.554 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.732+0000 7f1842e5f700 1 --2- 172.21.15.12:0/2643686699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f183c07de80 0x7f183c07c330 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f18300098a0 tx=0x7f1830009bb0 comp rx=0 tx=0).stop 2024-04-22T13:14:43.554 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.732+0000 7f1842e5f700 1 -- 172.21.15.12:0/2643686699 shutdown_connections 2024-04-22T13:14:43.555 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.732+0000 7f1842e5f700 1 --2- 172.21.15.12:0/2643686699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f183c07de80 0x7f183c07c330 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:43.555 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.732+0000 7f1842e5f700 1 -- 172.21.15.12:0/2643686699 >> 172.21.15.12:0/2643686699 conn(0x7f183c103b40 msgr2=0x7f183c105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:43.555 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.732+0000 7f1842e5f700 1 -- 172.21.15.12:0/2643686699 shutdown_connections 2024-04-22T13:14:43.555 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.732+0000 7f1842e5f700 1 -- 172.21.15.12:0/2643686699 wait complete. 2024-04-22T13:14:43.555 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.732+0000 7f1842e5f700 1 Processor -- start 2024-04-22T13:14:43.555 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.732+0000 7f1842e5f700 1 -- start start 2024-04-22T13:14:43.555 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.733+0000 7f1842e5f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f183c07de80 0x7f183c19e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:43.555 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.733+0000 7f1842e5f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f183c10ea70 con 0x7f183c07de80 2024-04-22T13:14:43.555 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.733+0000 7f1840bfb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f183c07de80 0x7f183c19e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:43.555 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.733+0000 7f1840bfb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f183c07de80 0x7f183c19e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54436/0 (socket says 172.21.15.12:54436) 2024-04-22T13:14:43.556 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.733+0000 7f1840bfb700 1 -- 172.21.15.12:0/3535915570 learned_addr learned my addr 172.21.15.12:0/3535915570 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:43.556 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.733+0000 7f1840bfb700 1 -- 172.21.15.12:0/3535915570 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183c19ebf0 con 0x7f183c07de80 2024-04-22T13:14:43.556 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.733+0000 7f1840bfb700 1 --2- 172.21.15.12:0/3535915570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f183c07de80 0x7f183c19e6b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1830009fb0 tx=0x7f1830011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:43.556 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.733+0000 7f1839ffb700 1 -- 172.21.15.12:0/3535915570 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18300036a0 con 0x7f183c07de80 2024-04-22T13:14:43.556 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.733+0000 7f1839ffb700 1 -- 172.21.15.12:0/3535915570 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f183001a430 con 0x7f183c07de80 2024-04-22T13:14:43.556 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.733+0000 7f1842e5f700 1 -- 172.21.15.12:0/3535915570 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f183c19ee80 con 0x7f183c07de80 2024-04-22T13:14:43.556 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.733+0000 7f1839ffb700 1 -- 172.21.15.12:0/3535915570 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1830019550 con 0x7f183c07de80 2024-04-22T13:14:43.556 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.734+0000 7f1842e5f700 1 -- 172.21.15.12:0/3535915570 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f183c1a1b70 con 0x7f183c07de80 2024-04-22T13:14:43.556 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.734+0000 7f1839ffb700 1 -- 172.21.15.12:0/3535915570 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f1830019850 con 0x7f183c07de80 2024-04-22T13:14:43.556 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.734+0000 7f1842e5f700 1 -- 172.21.15.12:0/3535915570 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f183c0640a0 con 0x7f183c07de80 2024-04-22T13:14:43.556 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.735+0000 7f1839ffb700 1 --2- 172.21.15.12:0/3535915570 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f182c039560 0x7f182c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:43.556 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.735+0000 7f1839ffb700 1 -- 172.21.15.12:0/3535915570 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f183004db90 con 0x7f183c07de80 2024-04-22T13:14:43.556 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.735+0000 7f183bfff700 1 --2- 172.21.15.12:0/3535915570 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f182c039560 0x7f182c03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:43.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.735+0000 7f183bfff700 1 --2- 172.21.15.12:0/3535915570 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f182c039560 0x7f182c03ba20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1828006fd0 tx=0x7f1828006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:43.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.740+0000 7f1839ffb700 1 -- 172.21.15.12:0/3535915570 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f183000fda0 con 0x7f183c07de80 2024-04-22T13:14:43.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.954+0000 7f1842e5f700 1 -- 172.21.15.12:0/3535915570 --> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f183c104df0 con 0x7f182c039560 2024-04-22T13:14:43.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.955+0000 7f1839ffb700 1 -- 172.21.15.12:0/3535915570 <== mgr.14120 v2:172.21.15.12:6800/2009831940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f183c104df0 con 0x7f182c039560 2024-04-22T13:14:43.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.957+0000 7f1842e5f700 1 -- 172.21.15.12:0/3535915570 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f182c039560 msgr2=0x7f182c03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:43.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.957+0000 7f1842e5f700 1 --2- 172.21.15.12:0/3535915570 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f182c039560 0x7f182c03ba20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1828006fd0 tx=0x7f1828006e40 comp rx=0 tx=0).stop 2024-04-22T13:14:43.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.957+0000 7f1842e5f700 1 -- 172.21.15.12:0/3535915570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f183c07de80 msgr2=0x7f183c19e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:43.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.957+0000 7f1842e5f700 1 --2- 172.21.15.12:0/3535915570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f183c07de80 0x7f183c19e6b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1830009fb0 tx=0x7f1830011940 comp rx=0 tx=0).stop 2024-04-22T13:14:43.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.957+0000 7f1842e5f700 1 -- 172.21.15.12:0/3535915570 shutdown_connections 2024-04-22T13:14:43.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.957+0000 7f1842e5f700 1 --2- 172.21.15.12:0/3535915570 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f182c039560 0x7f182c03ba20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:43.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.957+0000 7f1842e5f700 1 --2- 172.21.15.12:0/3535915570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f183c07de80 0x7f183c19e6b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:43.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.957+0000 7f1842e5f700 1 -- 172.21.15.12:0/3535915570 >> 172.21.15.12:0/3535915570 conn(0x7f183c103b40 msgr2=0x7f183c1046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:43.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.957+0000 7f1842e5f700 1 -- 172.21.15.12:0/3535915570 shutdown_connections 2024-04-22T13:14:43.557 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:42.957+0000 7f1842e5f700 1 -- 172.21.15.12:0/3535915570 wait complete. 2024-04-22T13:14:43.558 INFO:teuthology.orchestra.run.smithi012.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-22T13:14:43.558 INFO:teuthology.orchestra.run.smithi012.stdout:Adding key to root@localhost authorized_keys... 2024-04-22T13:14:43.558 INFO:teuthology.orchestra.run.smithi012.stdout:Adding host smithi012... 2024-04-22T13:14:43.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:43 smithi012 ceph-mon[28265]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:14:44.761 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:44 smithi012 ceph-mon[28265]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi012", "addr": "172.21.15.12", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:14:45.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:45 smithi012 ceph-mon[28265]: Deploying cephadm binary to smithi012 2024-04-22T13:14:48.976 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Added host 'smithi012' with addr '172.21.15.12' 2024-04-22T13:14:48.977 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.949+0000 7fc04b9af700 1 Processor -- start 2024-04-22T13:14:48.977 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.949+0000 7fc04b9af700 1 -- start start 2024-04-22T13:14:48.977 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.949+0000 7fc04b9af700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc044108ff0 0x7fc04410b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:48.977 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.950+0000 7fc04b9af700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc044063d20 con 0x7fc044108ff0 2024-04-22T13:14:48.977 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.950+0000 7fc04974b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc044108ff0 0x7fc04410b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:48.977 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.950+0000 7fc04974b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc044108ff0 0x7fc04410b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54448/0 (socket says 172.21.15.12:54448) 2024-04-22T13:14:48.977 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.950+0000 7fc04974b700 1 -- 172.21.15.12:0/3217182777 learned_addr learned my addr 172.21.15.12:0/3217182777 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:48.977 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.950+0000 7fc04974b700 1 -- 172.21.15.12:0/3217182777 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04410b920 con 0x7fc044108ff0 2024-04-22T13:14:48.977 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.950+0000 7fc04974b700 1 --2- 172.21.15.12:0/3217182777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc044108ff0 0x7fc04410b3e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc03c009e80 tx=0x7fc03c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c16cb2108895b79 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:48.977 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.950+0000 7fc043fff700 1 -- 172.21.15.12:0/3217182777 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc03c017070 con 0x7fc044108ff0 2024-04-22T13:14:48.977 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.950+0000 7fc043fff700 1 -- 172.21.15.12:0/3217182777 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc03c00b8c0 con 0x7fc044108ff0 2024-04-22T13:14:48.978 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.950+0000 7fc043fff700 1 -- 172.21.15.12:0/3217182777 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc03c004490 con 0x7fc044108ff0 2024-04-22T13:14:48.978 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.951+0000 7fc04b9af700 1 -- 172.21.15.12:0/3217182777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc044108ff0 msgr2=0x7fc04410b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:48.978 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.951+0000 7fc04b9af700 1 --2- 172.21.15.12:0/3217182777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc044108ff0 0x7fc04410b3e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc03c009e80 tx=0x7fc03c00b330 comp rx=0 tx=0).stop 2024-04-22T13:14:48.978 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.951+0000 7fc04b9af700 1 -- 172.21.15.12:0/3217182777 shutdown_connections 2024-04-22T13:14:48.978 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.951+0000 7fc04b9af700 1 --2- 172.21.15.12:0/3217182777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc044108ff0 0x7fc04410b3e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:48.978 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.951+0000 7fc04b9af700 1 -- 172.21.15.12:0/3217182777 >> 172.21.15.12:0/3217182777 conn(0x7fc044102a40 msgr2=0x7fc044104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:48.978 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.951+0000 7fc04b9af700 1 -- 172.21.15.12:0/3217182777 shutdown_connections 2024-04-22T13:14:48.978 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.951+0000 7fc04b9af700 1 -- 172.21.15.12:0/3217182777 wait complete. 2024-04-22T13:14:48.978 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.952+0000 7fc04b9af700 1 Processor -- start 2024-04-22T13:14:48.978 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.952+0000 7fc04b9af700 1 -- start start 2024-04-22T13:14:48.978 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.952+0000 7fc04b9af700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc044108ff0 0x7fc04419c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:48.978 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.952+0000 7fc04b9af700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc04410c8e0 con 0x7fc044108ff0 2024-04-22T13:14:48.978 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.952+0000 7fc04974b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc044108ff0 0x7fc04419c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:48.979 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.952+0000 7fc04974b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc044108ff0 0x7fc04419c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54454/0 (socket says 172.21.15.12:54454) 2024-04-22T13:14:48.979 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.952+0000 7fc04974b700 1 -- 172.21.15.12:0/2666669609 learned_addr learned my addr 172.21.15.12:0/2666669609 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:48.979 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.952+0000 7fc04974b700 1 -- 172.21.15.12:0/2666669609 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04419c930 con 0x7fc044108ff0 2024-04-22T13:14:48.979 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.953+0000 7fc04974b700 1 --2- 172.21.15.12:0/2666669609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc044108ff0 0x7fc04419c3f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc03c00b7b0 tx=0x7fc03c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:48.979 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.953+0000 7fc0427fc700 1 -- 172.21.15.12:0/2666669609 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc03c020020 con 0x7fc044108ff0 2024-04-22T13:14:48.979 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.953+0000 7fc04b9af700 1 -- 172.21.15.12:0/2666669609 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc04419cbc0 con 0x7fc044108ff0 2024-04-22T13:14:48.979 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.953+0000 7fc04b9af700 1 -- 172.21.15.12:0/2666669609 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc04419f8b0 con 0x7fc044108ff0 2024-04-22T13:14:48.979 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.953+0000 7fc0427fc700 1 -- 172.21.15.12:0/2666669609 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc03c012430 con 0x7fc044108ff0 2024-04-22T13:14:48.979 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.953+0000 7fc0427fc700 1 -- 172.21.15.12:0/2666669609 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc03c017040 con 0x7fc044108ff0 2024-04-22T13:14:48.979 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.954+0000 7fc0427fc700 1 -- 172.21.15.12:0/2666669609 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fc03c023020 con 0x7fc044108ff0 2024-04-22T13:14:48.979 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.954+0000 7fc0427fc700 1 --2- 172.21.15.12:0/2666669609 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fc02c039560 0x7fc02c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:48.979 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.954+0000 7fc04b9af700 1 -- 172.21.15.12:0/2666669609 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc030005360 con 0x7fc044108ff0 2024-04-22T13:14:48.979 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.954+0000 7fc0427fc700 1 -- 172.21.15.12:0/2666669609 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc03c00f050 con 0x7fc044108ff0 2024-04-22T13:14:48.979 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.954+0000 7fc048f4a700 1 --2- 172.21.15.12:0/2666669609 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fc02c039560 0x7fc02c03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:48.980 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.954+0000 7fc048f4a700 1 --2- 172.21.15.12:0/2666669609 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fc02c039560 0x7fc02c03ba20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc038006fd0 tx=0x7fc038006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:48.980 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:43.959+0000 7fc0427fc700 1 -- 172.21.15.12:0/2666669609 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc03c02a030 con 0x7fc044108ff0 2024-04-22T13:14:48.980 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:44.172+0000 7fc04b9af700 1 -- 172.21.15.12:0/2666669609 --> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi012", "addr": "172.21.15.12", "target": ["mon-mgr", ""]}) -- 0x7fc030000bf0 con 0x7fc02c039560 2024-04-22T13:14:48.980 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:48.384+0000 7fc0427fc700 1 -- 172.21.15.12:0/2666669609 <== mgr.14120 v2:172.21.15.12:6800/2009831940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fc030000bf0 con 0x7fc02c039560 2024-04-22T13:14:48.980 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:48.386+0000 7fc04b9af700 1 -- 172.21.15.12:0/2666669609 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fc02c039560 msgr2=0x7fc02c03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:48.980 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:48.386+0000 7fc04b9af700 1 --2- 172.21.15.12:0/2666669609 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fc02c039560 0x7fc02c03ba20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc038006fd0 tx=0x7fc038006e40 comp rx=0 tx=0).stop 2024-04-22T13:14:48.980 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:48.386+0000 7fc04b9af700 1 -- 172.21.15.12:0/2666669609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc044108ff0 msgr2=0x7fc04419c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:48.980 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:48.386+0000 7fc04b9af700 1 --2- 172.21.15.12:0/2666669609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc044108ff0 0x7fc04419c3f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc03c00b7b0 tx=0x7fc03c00be20 comp rx=0 tx=0).stop 2024-04-22T13:14:48.980 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:48.387+0000 7fc04b9af700 1 -- 172.21.15.12:0/2666669609 shutdown_connections 2024-04-22T13:14:48.980 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:48.387+0000 7fc04b9af700 1 --2- 172.21.15.12:0/2666669609 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fc02c039560 0x7fc02c03ba20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:48.980 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:48.387+0000 7fc04b9af700 1 --2- 172.21.15.12:0/2666669609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc044108ff0 0x7fc04419c3f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:48.980 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:48.387+0000 7fc04b9af700 1 -- 172.21.15.12:0/2666669609 >> 172.21.15.12:0/2666669609 conn(0x7fc044102a40 msgr2=0x7fc0441034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:48.980 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:48.387+0000 7fc04b9af700 1 -- 172.21.15.12:0/2666669609 shutdown_connections 2024-04-22T13:14:48.980 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:48.387+0000 7fc04b9af700 1 -- 172.21.15.12:0/2666669609 wait complete. 2024-04-22T13:14:48.980 INFO:teuthology.orchestra.run.smithi012.stdout:Deploying unmanaged mon service... 2024-04-22T13:14:49.536 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:49 smithi012 ceph-mon[28265]: from='mgr.14120 172.21.15.12:0/2157619606' entity='mgr.x' 2024-04-22T13:14:49.537 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:49 smithi012 ceph-mon[28265]: Added host smithi012 2024-04-22T13:14:49.537 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:49 smithi012 ceph-mon[28265]: from='mgr.14120 172.21.15.12:0/2157619606' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:14:50.512 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-22T13:14:50.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.537+0000 7fd47bee7700 1 Processor -- start 2024-04-22T13:14:50.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.538+0000 7fd47bee7700 1 -- start start 2024-04-22T13:14:50.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.538+0000 7fd47bee7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4741067f0 0x7fd474106c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:50.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.538+0000 7fd47bee7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd4741071e0 con 0x7fd4741067f0 2024-04-22T13:14:50.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.538+0000 7fd479c83700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4741067f0 0x7fd474106c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:50.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.538+0000 7fd479c83700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4741067f0 0x7fd474106c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54460/0 (socket says 172.21.15.12:54460) 2024-04-22T13:14:50.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.538+0000 7fd479c83700 1 -- 172.21.15.12:0/380488292 learned_addr learned my addr 172.21.15.12:0/380488292 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:50.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.538+0000 7fd479c83700 1 -- 172.21.15.12:0/380488292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd474109780 con 0x7fd4741067f0 2024-04-22T13:14:50.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.539+0000 7fd479c83700 1 --2- 172.21.15.12:0/380488292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4741067f0 0x7fd474106c10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd4700098a0 tx=0x7fd470009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c897acad993810e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:50.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.539+0000 7fd478c81700 1 -- 172.21.15.12:0/380488292 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd470004160 con 0x7fd4741067f0 2024-04-22T13:14:50.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.539+0000 7fd478c81700 1 -- 172.21.15.12:0/380488292 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd470004300 con 0x7fd4741067f0 2024-04-22T13:14:50.513 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.539+0000 7fd478c81700 1 -- 172.21.15.12:0/380488292 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd470011af0 con 0x7fd4741067f0 2024-04-22T13:14:50.514 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.539+0000 7fd47bee7700 1 -- 172.21.15.12:0/380488292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4741067f0 msgr2=0x7fd474106c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:50.514 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.539+0000 7fd47bee7700 1 --2- 172.21.15.12:0/380488292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4741067f0 0x7fd474106c10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd4700098a0 tx=0x7fd470009bb0 comp rx=0 tx=0).stop 2024-04-22T13:14:50.514 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.539+0000 7fd47bee7700 1 -- 172.21.15.12:0/380488292 shutdown_connections 2024-04-22T13:14:50.514 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.539+0000 7fd47bee7700 1 --2- 172.21.15.12:0/380488292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4741067f0 0x7fd474106c10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:50.514 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.539+0000 7fd47bee7700 1 -- 172.21.15.12:0/380488292 >> 172.21.15.12:0/380488292 conn(0x7fd474102140 msgr2=0x7fd474104560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:50.514 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.539+0000 7fd47bee7700 1 -- 172.21.15.12:0/380488292 shutdown_connections 2024-04-22T13:14:50.514 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.539+0000 7fd47bee7700 1 -- 172.21.15.12:0/380488292 wait complete. 2024-04-22T13:14:50.514 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.540+0000 7fd47bee7700 1 Processor -- start 2024-04-22T13:14:50.514 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.540+0000 7fd47bee7700 1 -- start start 2024-04-22T13:14:50.514 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.540+0000 7fd47bee7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4741067f0 0x7fd47419c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:50.514 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.540+0000 7fd47bee7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd47410cb30 con 0x7fd4741067f0 2024-04-22T13:14:50.514 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.540+0000 7fd479c83700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4741067f0 0x7fd47419c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:50.514 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.540+0000 7fd479c83700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4741067f0 0x7fd47419c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54472/0 (socket says 172.21.15.12:54472) 2024-04-22T13:14:50.514 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.540+0000 7fd479c83700 1 -- 172.21.15.12:0/2147293267 learned_addr learned my addr 172.21.15.12:0/2147293267 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:50.515 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.541+0000 7fd479c83700 1 -- 172.21.15.12:0/2147293267 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd47419c9a0 con 0x7fd4741067f0 2024-04-22T13:14:50.515 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.541+0000 7fd479c83700 1 --2- 172.21.15.12:0/2147293267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4741067f0 0x7fd47419c460 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd470003f40 tx=0x7fd470011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:50.515 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.541+0000 7fd46affd700 1 -- 172.21.15.12:0/2147293267 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4700038b0 con 0x7fd4741067f0 2024-04-22T13:14:50.515 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.541+0000 7fd47bee7700 1 -- 172.21.15.12:0/2147293267 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd47419cc30 con 0x7fd4741067f0 2024-04-22T13:14:50.515 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.541+0000 7fd46affd700 1 -- 172.21.15.12:0/2147293267 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd470003a50 con 0x7fd4741067f0 2024-04-22T13:14:50.515 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.541+0000 7fd46affd700 1 -- 172.21.15.12:0/2147293267 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd47001a5c0 con 0x7fd4741067f0 2024-04-22T13:14:50.515 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.541+0000 7fd47bee7700 1 -- 172.21.15.12:0/2147293267 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd47419f920 con 0x7fd4741067f0 2024-04-22T13:14:50.515 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.542+0000 7fd46affd700 1 -- 172.21.15.12:0/2147293267 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fd470024430 con 0x7fd4741067f0 2024-04-22T13:14:50.515 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.542+0000 7fd47bee7700 1 -- 172.21.15.12:0/2147293267 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4740640a0 con 0x7fd4741067f0 2024-04-22T13:14:50.515 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.542+0000 7fd46affd700 1 --2- 172.21.15.12:0/2147293267 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fd460039560 0x7fd46003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:50.515 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.542+0000 7fd46affd700 1 -- 172.21.15.12:0/2147293267 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd47004dae0 con 0x7fd4741067f0 2024-04-22T13:14:50.515 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.542+0000 7fd479482700 1 --2- 172.21.15.12:0/2147293267 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fd460039560 0x7fd46003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:50.515 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.543+0000 7fd479482700 1 --2- 172.21.15.12:0/2147293267 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fd460039560 0x7fd46003ba20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd464006fd0 tx=0x7fd464006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:50.515 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.548+0000 7fd46affd700 1 -- 172.21.15.12:0/2147293267 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd47002d910 con 0x7fd4741067f0 2024-04-22T13:14:50.515 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.784+0000 7fd47bee7700 1 -- 172.21.15.12:0/2147293267 --> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fd474103300 con 0x7fd460039560 2024-04-22T13:14:50.516 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.791+0000 7fd46affd700 1 -- 172.21.15.12:0/2147293267 <== mgr.14120 v2:172.21.15.12:6800/2009831940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd474103300 con 0x7fd460039560 2024-04-22T13:14:50.516 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.794+0000 7fd47bee7700 1 -- 172.21.15.12:0/2147293267 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fd460039560 msgr2=0x7fd46003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:50.516 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.794+0000 7fd47bee7700 1 --2- 172.21.15.12:0/2147293267 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fd460039560 0x7fd46003ba20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd464006fd0 tx=0x7fd464006e40 comp rx=0 tx=0).stop 2024-04-22T13:14:50.516 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.794+0000 7fd47bee7700 1 -- 172.21.15.12:0/2147293267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4741067f0 msgr2=0x7fd47419c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:50.516 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.794+0000 7fd47bee7700 1 --2- 172.21.15.12:0/2147293267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4741067f0 0x7fd47419c460 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd470003f40 tx=0x7fd470011890 comp rx=0 tx=0).stop 2024-04-22T13:14:50.516 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.794+0000 7fd47bee7700 1 -- 172.21.15.12:0/2147293267 shutdown_connections 2024-04-22T13:14:50.516 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.794+0000 7fd47bee7700 1 --2- 172.21.15.12:0/2147293267 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fd460039560 0x7fd46003ba20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:50.516 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.794+0000 7fd47bee7700 1 --2- 172.21.15.12:0/2147293267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4741067f0 0x7fd47419c460 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:50.516 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.794+0000 7fd47bee7700 1 -- 172.21.15.12:0/2147293267 >> 172.21.15.12:0/2147293267 conn(0x7fd474102140 msgr2=0x7fd474102bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:50.516 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.794+0000 7fd47bee7700 1 -- 172.21.15.12:0/2147293267 shutdown_connections 2024-04-22T13:14:50.516 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:49.794+0000 7fd47bee7700 1 -- 172.21.15.12:0/2147293267 wait complete. 2024-04-22T13:14:50.516 INFO:teuthology.orchestra.run.smithi012.stdout:Deploying unmanaged mgr service... 2024-04-22T13:14:51.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:50 smithi012 ceph-mon[28265]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:14:51.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:50 smithi012 ceph-mon[28265]: Saving service mon spec with placement count:5 2024-04-22T13:14:51.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:50 smithi012 ceph-mon[28265]: from='mgr.14120 172.21.15.12:0/2157619606' entity='mgr.x' 2024-04-22T13:14:52.456 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-22T13:14:52.457 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.360+0000 7f93d04a0700 1 Processor -- start 2024-04-22T13:14:52.457 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.360+0000 7f93d04a0700 1 -- start start 2024-04-22T13:14:52.457 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.360+0000 7f93d04a0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93c8107440 0x7f93c8107860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:52.457 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.360+0000 7f93d04a0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f93c8107e30 con 0x7f93c8107440 2024-04-22T13:14:52.457 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.361+0000 7f93ce23c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93c8107440 0x7f93c8107860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:52.457 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.361+0000 7f93ce23c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93c8107440 0x7f93c8107860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54480/0 (socket says 172.21.15.12:54480) 2024-04-22T13:14:52.457 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.361+0000 7f93ce23c700 1 -- 172.21.15.12:0/952002747 learned_addr learned my addr 172.21.15.12:0/952002747 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:52.457 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.361+0000 7f93ce23c700 1 -- 172.21.15.12:0/952002747 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c8108640 con 0x7f93c8107440 2024-04-22T13:14:52.457 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.361+0000 7f93ce23c700 1 --2- 172.21.15.12:0/952002747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93c8107440 0x7f93c8107860 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f93c400bb40 tx=0x7f93c400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ff7d4866f91334 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:52.457 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.361+0000 7f93cd23a700 1 -- 172.21.15.12:0/952002747 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93c400dbe0 con 0x7f93c8107440 2024-04-22T13:14:52.457 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.361+0000 7f93cd23a700 1 -- 172.21.15.12:0/952002747 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f93c40044c0 con 0x7f93c8107440 2024-04-22T13:14:52.457 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.361+0000 7f93cd23a700 1 -- 172.21.15.12:0/952002747 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93c4010470 con 0x7f93c8107440 2024-04-22T13:14:52.457 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.362+0000 7f93d04a0700 1 -- 172.21.15.12:0/952002747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93c8107440 msgr2=0x7f93c8107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:52.458 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.362+0000 7f93d04a0700 1 --2- 172.21.15.12:0/952002747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93c8107440 0x7f93c8107860 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f93c400bb40 tx=0x7f93c400be50 comp rx=0 tx=0).stop 2024-04-22T13:14:52.458 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.362+0000 7f93d04a0700 1 -- 172.21.15.12:0/952002747 shutdown_connections 2024-04-22T13:14:52.458 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.362+0000 7f93d04a0700 1 --2- 172.21.15.12:0/952002747 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93c8107440 0x7f93c8107860 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:52.458 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.362+0000 7f93d04a0700 1 -- 172.21.15.12:0/952002747 >> 172.21.15.12:0/952002747 conn(0x7f93c8102a40 msgr2=0x7f93c8104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:52.458 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.362+0000 7f93d04a0700 1 -- 172.21.15.12:0/952002747 shutdown_connections 2024-04-22T13:14:52.458 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.362+0000 7f93d04a0700 1 -- 172.21.15.12:0/952002747 wait complete. 2024-04-22T13:14:52.458 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.362+0000 7f93d04a0700 1 Processor -- start 2024-04-22T13:14:52.458 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.363+0000 7f93d04a0700 1 -- start start 2024-04-22T13:14:52.458 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.363+0000 7f93d04a0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93c8107440 0x7f93c819e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:52.458 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.363+0000 7f93d04a0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f93c807afe0 con 0x7f93c8107440 2024-04-22T13:14:52.458 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.363+0000 7f93ce23c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93c8107440 0x7f93c819e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:52.458 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.363+0000 7f93ce23c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93c8107440 0x7f93c819e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54492/0 (socket says 172.21.15.12:54492) 2024-04-22T13:14:52.458 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.363+0000 7f93ce23c700 1 -- 172.21.15.12:0/189524986 learned_addr learned my addr 172.21.15.12:0/189524986 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:52.458 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.363+0000 7f93ce23c700 1 -- 172.21.15.12:0/189524986 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c819eb90 con 0x7f93c8107440 2024-04-22T13:14:52.459 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.363+0000 7f93ce23c700 1 --2- 172.21.15.12:0/189524986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93c8107440 0x7f93c819e650 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f93c400d650 tx=0x7f93c4003ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:52.459 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.364+0000 7f93bf7fe700 1 -- 172.21.15.12:0/189524986 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93c4010770 con 0x7f93c8107440 2024-04-22T13:14:52.459 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.364+0000 7f93bf7fe700 1 -- 172.21.15.12:0/189524986 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f93c401d070 con 0x7f93c8107440 2024-04-22T13:14:52.459 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.364+0000 7f93d04a0700 1 -- 172.21.15.12:0/189524986 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93c819ee20 con 0x7f93c8107440 2024-04-22T13:14:52.459 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.364+0000 7f93bf7fe700 1 -- 172.21.15.12:0/189524986 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93c4019c10 con 0x7f93c8107440 2024-04-22T13:14:52.459 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.364+0000 7f93d04a0700 1 -- 172.21.15.12:0/189524986 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93c81a1b10 con 0x7f93c8107440 2024-04-22T13:14:52.459 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.364+0000 7f93d04a0700 1 -- 172.21.15.12:0/189524986 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93c80640a0 con 0x7f93c8107440 2024-04-22T13:14:52.459 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.365+0000 7f93bf7fe700 1 -- 172.21.15.12:0/189524986 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f93c4028060 con 0x7f93c8107440 2024-04-22T13:14:52.459 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.365+0000 7f93bf7fe700 1 --2- 172.21.15.12:0/189524986 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f93b4039560 0x7f93b403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:52.459 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.365+0000 7f93bf7fe700 1 -- 172.21.15.12:0/189524986 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f93c404db20 con 0x7f93c8107440 2024-04-22T13:14:52.459 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.365+0000 7f93cda3b700 1 --2- 172.21.15.12:0/189524986 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f93b4039560 0x7f93b403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:52.459 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.365+0000 7f93cda3b700 1 --2- 172.21.15.12:0/189524986 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f93b4039560 0x7f93b403ba20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f93b8006fd0 tx=0x7f93b8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:52.459 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.370+0000 7f93bf7fe700 1 -- 172.21.15.12:0/189524986 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f93c4015070 con 0x7f93c8107440 2024-04-22T13:14:52.459 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.594+0000 7f93d04a0700 1 -- 172.21.15.12:0/189524986 --> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f93c8075460 con 0x7f93b4039560 2024-04-22T13:14:52.460 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.598+0000 7f93bf7fe700 1 -- 172.21.15.12:0/189524986 <== mgr.14120 v2:172.21.15.12:6800/2009831940 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f93c8075460 con 0x7f93b4039560 2024-04-22T13:14:52.460 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.600+0000 7f93d04a0700 1 -- 172.21.15.12:0/189524986 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f93b4039560 msgr2=0x7f93b403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:52.460 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.600+0000 7f93d04a0700 1 --2- 172.21.15.12:0/189524986 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f93b4039560 0x7f93b403ba20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f93b8006fd0 tx=0x7f93b8006e40 comp rx=0 tx=0).stop 2024-04-22T13:14:52.460 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.600+0000 7f93d04a0700 1 -- 172.21.15.12:0/189524986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93c8107440 msgr2=0x7f93c819e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:52.460 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.600+0000 7f93d04a0700 1 --2- 172.21.15.12:0/189524986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93c8107440 0x7f93c819e650 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f93c400d650 tx=0x7f93c4003ff0 comp rx=0 tx=0).stop 2024-04-22T13:14:52.460 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.600+0000 7f93d04a0700 1 -- 172.21.15.12:0/189524986 shutdown_connections 2024-04-22T13:14:52.460 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.600+0000 7f93d04a0700 1 --2- 172.21.15.12:0/189524986 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f93b4039560 0x7f93b403ba20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:52.460 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.600+0000 7f93d04a0700 1 --2- 172.21.15.12:0/189524986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f93c8107440 0x7f93c819e650 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:52.460 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.600+0000 7f93d04a0700 1 -- 172.21.15.12:0/189524986 >> 172.21.15.12:0/189524986 conn(0x7f93c8102a40 msgr2=0x7f93c8190860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:52.460 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.600+0000 7f93d04a0700 1 -- 172.21.15.12:0/189524986 shutdown_connections 2024-04-22T13:14:52.460 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:51.600+0000 7f93d04a0700 1 -- 172.21.15.12:0/189524986 wait complete. 2024-04-22T13:14:52.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:52 smithi012 ceph-mon[28265]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:14:52.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:52 smithi012 ceph-mon[28265]: Saving service mgr spec with placement count:2 2024-04-22T13:14:52.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:52 smithi012 ceph-mon[28265]: from='mgr.14120 172.21.15.12:0/2157619606' entity='mgr.x' 2024-04-22T13:14:52.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:52 smithi012 ceph-mon[28265]: from='mgr.14120 172.21.15.12:0/2157619606' entity='mgr.x' 2024-04-22T13:14:52.734 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:52 smithi012 ceph-mon[28265]: from='mgr.14120 172.21.15.12:0/2157619606' entity='mgr.x' 2024-04-22T13:14:54.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:54 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2467382' entity='client.admin' 2024-04-22T13:14:54.417 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:54 smithi012 ceph-mon[28265]: from='mgr.14120 172.21.15.12:0/2157619606' entity='mgr.x' 2024-04-22T13:14:54.423 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.942+0000 7fcb4d983700 1 Processor -- start 2024-04-22T13:14:54.423 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.942+0000 7fcb4d983700 1 -- start start 2024-04-22T13:14:54.423 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.942+0000 7fcb4d983700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb48108fc0 0x7fcb4810b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:54.423 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.942+0000 7fcb4d983700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4807bec0 con 0x7fcb48108fc0 2024-04-22T13:14:54.423 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.943+0000 7fcb46ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb48108fc0 0x7fcb4810b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:54.423 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.943+0000 7fcb46ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb48108fc0 0x7fcb4810b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36280/0 (socket says 172.21.15.12:36280) 2024-04-22T13:14:54.423 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.943+0000 7fcb46ffd700 1 -- 172.21.15.12:0/820861379 learned_addr learned my addr 172.21.15.12:0/820861379 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:54.423 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.943+0000 7fcb46ffd700 1 -- 172.21.15.12:0/820861379 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb4810b8f0 con 0x7fcb48108fc0 2024-04-22T13:14:54.423 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.943+0000 7fcb46ffd700 1 --2- 172.21.15.12:0/820861379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb48108fc0 0x7fcb4810b3b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c0098a0 tx=0x7fcb3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6148bd5057e85528 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:54.423 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.943+0000 7fcb45ffb700 1 -- 172.21.15.12:0/820861379 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb3c004160 con 0x7fcb48108fc0 2024-04-22T13:14:54.423 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.943+0000 7fcb45ffb700 1 -- 172.21.15.12:0/820861379 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb3c004300 con 0x7fcb48108fc0 2024-04-22T13:14:54.424 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.943+0000 7fcb45ffb700 1 -- 172.21.15.12:0/820861379 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb3c011a20 con 0x7fcb48108fc0 2024-04-22T13:14:54.424 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.944+0000 7fcb4d983700 1 -- 172.21.15.12:0/820861379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb48108fc0 msgr2=0x7fcb4810b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:54.424 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.944+0000 7fcb4d983700 1 --2- 172.21.15.12:0/820861379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb48108fc0 0x7fcb4810b3b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c0098a0 tx=0x7fcb3c009bb0 comp rx=0 tx=0).stop 2024-04-22T13:14:54.424 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.944+0000 7fcb4d983700 1 -- 172.21.15.12:0/820861379 shutdown_connections 2024-04-22T13:14:54.424 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.944+0000 7fcb4d983700 1 --2- 172.21.15.12:0/820861379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb48108fc0 0x7fcb4810b3b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:54.424 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.944+0000 7fcb4d983700 1 -- 172.21.15.12:0/820861379 >> 172.21.15.12:0/820861379 conn(0x7fcb48102a50 msgr2=0x7fcb48104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:54.424 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.944+0000 7fcb4d983700 1 -- 172.21.15.12:0/820861379 shutdown_connections 2024-04-22T13:14:54.424 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.944+0000 7fcb4d983700 1 -- 172.21.15.12:0/820861379 wait complete. 2024-04-22T13:14:54.424 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.945+0000 7fcb4d983700 1 Processor -- start 2024-04-22T13:14:54.424 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.945+0000 7fcb4d983700 1 -- start start 2024-04-22T13:14:54.424 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.945+0000 7fcb4d983700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb48108fc0 0x7fcb4819c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:54.424 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.945+0000 7fcb4d983700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4810c900 con 0x7fcb48108fc0 2024-04-22T13:14:54.424 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.945+0000 7fcb46ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb48108fc0 0x7fcb4819c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:54.425 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.945+0000 7fcb46ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb48108fc0 0x7fcb4819c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36294/0 (socket says 172.21.15.12:36294) 2024-04-22T13:14:54.425 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.945+0000 7fcb46ffd700 1 -- 172.21.15.12:0/2467382 learned_addr learned my addr 172.21.15.12:0/2467382 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:54.425 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.946+0000 7fcb46ffd700 1 -- 172.21.15.12:0/2467382 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb4819c960 con 0x7fcb48108fc0 2024-04-22T13:14:54.425 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.946+0000 7fcb46ffd700 1 --2- 172.21.15.12:0/2467382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb48108fc0 0x7fcb4819c420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c0116e0 tx=0x7fcb3c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:54.425 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.946+0000 7fcb4c981700 1 -- 172.21.15.12:0/2467382 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb3c003b20 con 0x7fcb48108fc0 2024-04-22T13:14:54.425 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.946+0000 7fcb4c981700 1 -- 172.21.15.12:0/2467382 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb3c01b8f0 con 0x7fcb48108fc0 2024-04-22T13:14:54.425 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.946+0000 7fcb4c981700 1 -- 172.21.15.12:0/2467382 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb3c0238c0 con 0x7fcb48108fc0 2024-04-22T13:14:54.425 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.946+0000 7fcb4d983700 1 -- 172.21.15.12:0/2467382 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb4819cbf0 con 0x7fcb48108fc0 2024-04-22T13:14:54.425 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.946+0000 7fcb4d983700 1 -- 172.21.15.12:0/2467382 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb4819f8e0 con 0x7fcb48108fc0 2024-04-22T13:14:54.425 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.947+0000 7fcb4d983700 1 -- 172.21.15.12:0/2467382 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb480640a0 con 0x7fcb48108fc0 2024-04-22T13:14:54.425 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.947+0000 7fcb4c981700 1 -- 172.21.15.12:0/2467382 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fcb3c01baa0 con 0x7fcb48108fc0 2024-04-22T13:14:54.425 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.947+0000 7fcb4c981700 1 --2- 172.21.15.12:0/2467382 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fcb30039560 0x7fcb3003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:54.425 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.948+0000 7fcb4c981700 1 -- 172.21.15.12:0/2467382 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcb3c04e080 con 0x7fcb48108fc0 2024-04-22T13:14:54.425 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.948+0000 7fcb467fc700 1 --2- 172.21.15.12:0/2467382 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fcb30039560 0x7fcb3003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:54.426 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.948+0000 7fcb467fc700 1 --2- 172.21.15.12:0/2467382 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fcb30039560 0x7fcb3003ba20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcb38006fd0 tx=0x7fcb38006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:54.426 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:52.953+0000 7fcb4c981700 1 -- 172.21.15.12:0/2467382 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb3c01f030 con 0x7fcb48108fc0 2024-04-22T13:14:54.426 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:53.178+0000 7fcb4d983700 1 -- 172.21.15.12:0/2467382 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fcb4819fe60 con 0x7fcb48108fc0 2024-04-22T13:14:54.426 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:53.180+0000 7fcb4c981700 1 -- 172.21.15.12:0/2467382 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fcb3c01a3d0 con 0x7fcb48108fc0 2024-04-22T13:14:54.426 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:53.183+0000 7fcb4d983700 1 -- 172.21.15.12:0/2467382 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fcb30039560 msgr2=0x7fcb3003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:54.426 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:53.183+0000 7fcb4d983700 1 --2- 172.21.15.12:0/2467382 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fcb30039560 0x7fcb3003ba20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcb38006fd0 tx=0x7fcb38006e40 comp rx=0 tx=0).stop 2024-04-22T13:14:54.426 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:53.183+0000 7fcb4d983700 1 -- 172.21.15.12:0/2467382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb48108fc0 msgr2=0x7fcb4819c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:54.426 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:53.183+0000 7fcb4d983700 1 --2- 172.21.15.12:0/2467382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb48108fc0 0x7fcb4819c420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c0116e0 tx=0x7fcb3c011aa0 comp rx=0 tx=0).stop 2024-04-22T13:14:54.426 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:53.183+0000 7fcb4d983700 1 -- 172.21.15.12:0/2467382 shutdown_connections 2024-04-22T13:14:54.426 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:53.183+0000 7fcb4d983700 1 --2- 172.21.15.12:0/2467382 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fcb30039560 0x7fcb3003ba20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:54.426 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:53.183+0000 7fcb4d983700 1 --2- 172.21.15.12:0/2467382 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcb48108fc0 0x7fcb4819c420 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:54.426 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:53.183+0000 7fcb4d983700 1 -- 172.21.15.12:0/2467382 >> 172.21.15.12:0/2467382 conn(0x7fcb48102a50 msgr2=0x7fcb481034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:54.426 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:53.183+0000 7fcb4d983700 1 -- 172.21.15.12:0/2467382 shutdown_connections 2024-04-22T13:14:54.426 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:53.183+0000 7fcb4d983700 1 -- 172.21.15.12:0/2467382 wait complete. 2024-04-22T13:14:55.729 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.878+0000 7f4d3b232700 1 Processor -- start 2024-04-22T13:14:55.729 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.878+0000 7f4d3b232700 1 -- start start 2024-04-22T13:14:55.729 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.878+0000 7f4d3b232700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d34108570 0x7f4d34108990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:55.729 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.878+0000 7f4d3b232700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4d34108f60 con 0x7f4d34108570 2024-04-22T13:14:55.729 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.879+0000 7f4d38fce700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d34108570 0x7f4d34108990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:55.730 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.879+0000 7f4d38fce700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d34108570 0x7f4d34108990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36304/0 (socket says 172.21.15.12:36304) 2024-04-22T13:14:55.730 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.879+0000 7f4d38fce700 1 -- 172.21.15.12:0/2258537867 learned_addr learned my addr 172.21.15.12:0/2258537867 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:55.730 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.879+0000 7f4d38fce700 1 -- 172.21.15.12:0/2258537867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d34109720 con 0x7f4d34108570 2024-04-22T13:14:55.730 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.879+0000 7f4d38fce700 1 --2- 172.21.15.12:0/2258537867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d34108570 0x7f4d34108990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4d280098a0 tx=0x7f4d28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81c8a6a1317aece server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:55.730 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.879+0000 7f4d337fe700 1 -- 172.21.15.12:0/2258537867 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d28004160 con 0x7f4d34108570 2024-04-22T13:14:55.730 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.879+0000 7f4d337fe700 1 -- 172.21.15.12:0/2258537867 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d28004300 con 0x7f4d34108570 2024-04-22T13:14:55.730 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.879+0000 7f4d337fe700 1 -- 172.21.15.12:0/2258537867 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d28011a20 con 0x7f4d34108570 2024-04-22T13:14:55.730 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.880+0000 7f4d3b232700 1 -- 172.21.15.12:0/2258537867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d34108570 msgr2=0x7f4d34108990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:55.730 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.880+0000 7f4d3b232700 1 --2- 172.21.15.12:0/2258537867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d34108570 0x7f4d34108990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4d280098a0 tx=0x7f4d28009bb0 comp rx=0 tx=0).stop 2024-04-22T13:14:55.730 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.880+0000 7f4d3b232700 1 -- 172.21.15.12:0/2258537867 shutdown_connections 2024-04-22T13:14:55.730 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.880+0000 7f4d3b232700 1 --2- 172.21.15.12:0/2258537867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d34108570 0x7f4d34108990 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:55.730 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.880+0000 7f4d3b232700 1 -- 172.21.15.12:0/2258537867 >> 172.21.15.12:0/2258537867 conn(0x7f4d34103b70 msgr2=0x7f4d34105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:55.730 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.880+0000 7f4d3b232700 1 -- 172.21.15.12:0/2258537867 shutdown_connections 2024-04-22T13:14:55.730 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.880+0000 7f4d3b232700 1 -- 172.21.15.12:0/2258537867 wait complete. 2024-04-22T13:14:55.731 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.881+0000 7f4d3b232700 1 Processor -- start 2024-04-22T13:14:55.731 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.881+0000 7f4d3b232700 1 -- start start 2024-04-22T13:14:55.731 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.881+0000 7f4d3b232700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d34108570 0x7f4d3419c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:55.731 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.881+0000 7f4d3b232700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3410cad0 con 0x7f4d34108570 2024-04-22T13:14:55.731 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.881+0000 7f4d38fce700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d34108570 0x7f4d3419c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:55.731 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.881+0000 7f4d38fce700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d34108570 0x7f4d3419c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36320/0 (socket says 172.21.15.12:36320) 2024-04-22T13:14:55.731 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.881+0000 7f4d38fce700 1 -- 172.21.15.12:0/371530690 learned_addr learned my addr 172.21.15.12:0/371530690 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:55.731 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.881+0000 7f4d38fce700 1 -- 172.21.15.12:0/371530690 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d3419c9b0 con 0x7f4d34108570 2024-04-22T13:14:55.731 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.882+0000 7f4d38fce700 1 --2- 172.21.15.12:0/371530690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d34108570 0x7f4d3419c470 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4d28000c00 tx=0x7f4d28011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:55.731 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.882+0000 7f4d31ffb700 1 -- 172.21.15.12:0/371530690 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d28003aa0 con 0x7f4d34108570 2024-04-22T13:14:55.731 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.882+0000 7f4d31ffb700 1 -- 172.21.15.12:0/371530690 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d2801b8f0 con 0x7f4d34108570 2024-04-22T13:14:55.731 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.882+0000 7f4d3b232700 1 -- 172.21.15.12:0/371530690 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d3419cc40 con 0x7f4d34108570 2024-04-22T13:14:55.731 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.882+0000 7f4d31ffb700 1 -- 172.21.15.12:0/371530690 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d28023890 con 0x7f4d34108570 2024-04-22T13:14:55.731 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.882+0000 7f4d3b232700 1 -- 172.21.15.12:0/371530690 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d3419f930 con 0x7f4d34108570 2024-04-22T13:14:55.731 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.883+0000 7f4d31ffb700 1 -- 172.21.15.12:0/371530690 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f4d2802c3f0 con 0x7f4d34108570 2024-04-22T13:14:55.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.883+0000 7f4d31ffb700 1 --2- 172.21.15.12:0/371530690 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f4d24039600 0x7f4d2403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:55.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.883+0000 7f4d31ffb700 1 -- 172.21.15.12:0/371530690 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4d2801bda0 con 0x7f4d34108570 2024-04-22T13:14:55.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.883+0000 7f4d3b232700 1 -- 172.21.15.12:0/371530690 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d340640a0 con 0x7f4d34108570 2024-04-22T13:14:55.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.883+0000 7f4d33fff700 1 --2- 172.21.15.12:0/371530690 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f4d24039600 0x7f4d2403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:55.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.884+0000 7f4d33fff700 1 --2- 172.21.15.12:0/371530690 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f4d24039600 0x7f4d2403bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4d20006fd0 tx=0x7f4d20006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:55.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:54.889+0000 7f4d31ffb700 1 -- 172.21.15.12:0/371530690 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4d2801f030 con 0x7f4d34108570 2024-04-22T13:14:55.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:55.100+0000 7f4d3b232700 1 -- 172.21.15.12:0/371530690 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f4d3419fdc0 con 0x7f4d34108570 2024-04-22T13:14:55.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:55.102+0000 7f4d31ffb700 1 -- 172.21.15.12:0/371530690 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f4d2801baa0 con 0x7f4d34108570 2024-04-22T13:14:55.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:55.105+0000 7f4d3b232700 1 -- 172.21.15.12:0/371530690 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f4d24039600 msgr2=0x7f4d2403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:55.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:55.105+0000 7f4d3b232700 1 --2- 172.21.15.12:0/371530690 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f4d24039600 0x7f4d2403bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4d20006fd0 tx=0x7f4d20006e40 comp rx=0 tx=0).stop 2024-04-22T13:14:55.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:55.105+0000 7f4d3b232700 1 -- 172.21.15.12:0/371530690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d34108570 msgr2=0x7f4d3419c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:55.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:55.105+0000 7f4d3b232700 1 --2- 172.21.15.12:0/371530690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d34108570 0x7f4d3419c470 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4d28000c00 tx=0x7f4d28011aa0 comp rx=0 tx=0).stop 2024-04-22T13:14:55.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:55.105+0000 7f4d3b232700 1 -- 172.21.15.12:0/371530690 shutdown_connections 2024-04-22T13:14:55.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:55.105+0000 7f4d3b232700 1 --2- 172.21.15.12:0/371530690 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f4d24039600 0x7f4d2403bac0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:55.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:55.105+0000 7f4d3b232700 1 --2- 172.21.15.12:0/371530690 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d34108570 0x7f4d3419c470 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:55.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:55.105+0000 7f4d3b232700 1 -- 172.21.15.12:0/371530690 >> 172.21.15.12:0/371530690 conn(0x7f4d34103b70 msgr2=0x7f4d341046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:55.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:55.105+0000 7f4d3b232700 1 -- 172.21.15.12:0/371530690 shutdown_connections 2024-04-22T13:14:55.733 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:55.105+0000 7f4d3b232700 1 -- 172.21.15.12:0/371530690 wait complete. 2024-04-22T13:14:55.733 INFO:teuthology.orchestra.run.smithi012.stdout:Enabling the dashboard module... 2024-04-22T13:14:56.306 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:56 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/371530690' entity='client.admin' 2024-04-22T13:14:56.306 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:56 smithi012 ceph-mon[28265]: from='mgr.14120 172.21.15.12:0/2157619606' entity='mgr.x' 2024-04-22T13:14:57.292 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:57 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/981672452' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-22T13:14:57.292 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:57 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: ignoring --setuser ceph since I am not root 2024-04-22T13:14:57.292 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:57 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: ignoring --setgroup ceph since I am not root 2024-04-22T13:14:57.292 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:57 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:57.293+0000 7f80f9c8a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T13:14:57.589 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:57 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:57.485+0000 7f80f9c8a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:14:57.914 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:57 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:57.592+0000 7f80f9c8a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T13:14:57.914 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:57 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:57.915+0000 7f80f9c8a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T13:14:57.982 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.171+0000 7f991f8ca700 1 Processor -- start 2024-04-22T13:14:57.982 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.171+0000 7f991f8ca700 1 -- start start 2024-04-22T13:14:57.982 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.172+0000 7f991f8ca700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9918108fb0 0x7f991810b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:57.982 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.172+0000 7f991f8ca700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9918063d60 con 0x7f9918108fb0 2024-04-22T13:14:57.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.172+0000 7f991d666700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9918108fb0 0x7f991810b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:57.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.172+0000 7f991d666700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9918108fb0 0x7f991810b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36330/0 (socket says 172.21.15.12:36330) 2024-04-22T13:14:57.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.172+0000 7f991d666700 1 -- 172.21.15.12:0/1607737241 learned_addr learned my addr 172.21.15.12:0/1607737241 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:57.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.172+0000 7f991d666700 1 -- 172.21.15.12:0/1607737241 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991810b8e0 con 0x7f9918108fb0 2024-04-22T13:14:57.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.172+0000 7f991d666700 1 --2- 172.21.15.12:0/1607737241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9918108fb0 0x7f991810b3a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f99140098a0 tx=0x7f9914009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb5e9b6432f05e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:57.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.173+0000 7f990ffff700 1 -- 172.21.15.12:0/1607737241 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9914004160 con 0x7f9918108fb0 2024-04-22T13:14:57.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.173+0000 7f990ffff700 1 -- 172.21.15.12:0/1607737241 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9914004300 con 0x7f9918108fb0 2024-04-22T13:14:57.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.173+0000 7f990ffff700 1 -- 172.21.15.12:0/1607737241 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9914011a20 con 0x7f9918108fb0 2024-04-22T13:14:57.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.173+0000 7f991f8ca700 1 -- 172.21.15.12:0/1607737241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9918108fb0 msgr2=0x7f991810b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:57.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.173+0000 7f991f8ca700 1 --2- 172.21.15.12:0/1607737241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9918108fb0 0x7f991810b3a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f99140098a0 tx=0x7f9914009bb0 comp rx=0 tx=0).stop 2024-04-22T13:14:57.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.174+0000 7f991f8ca700 1 -- 172.21.15.12:0/1607737241 shutdown_connections 2024-04-22T13:14:57.983 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.174+0000 7f991f8ca700 1 --2- 172.21.15.12:0/1607737241 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9918108fb0 0x7f991810b3a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:57.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.174+0000 7f991f8ca700 1 -- 172.21.15.12:0/1607737241 >> 172.21.15.12:0/1607737241 conn(0x7f9918102a40 msgr2=0x7f9918104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:57.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.174+0000 7f991f8ca700 1 -- 172.21.15.12:0/1607737241 shutdown_connections 2024-04-22T13:14:57.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.174+0000 7f991f8ca700 1 -- 172.21.15.12:0/1607737241 wait complete. 2024-04-22T13:14:57.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.174+0000 7f991f8ca700 1 Processor -- start 2024-04-22T13:14:57.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.174+0000 7f991f8ca700 1 -- start start 2024-04-22T13:14:57.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.175+0000 7f991f8ca700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9918108fb0 0x7f991819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:57.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.175+0000 7f991f8ca700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f991810c8f0 con 0x7f9918108fb0 2024-04-22T13:14:57.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.175+0000 7f991d666700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9918108fb0 0x7f991819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:57.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.175+0000 7f991d666700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9918108fb0 0x7f991819e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36332/0 (socket says 172.21.15.12:36332) 2024-04-22T13:14:57.984 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.175+0000 7f991d666700 1 -- 172.21.15.12:0/981672452 learned_addr learned my addr 172.21.15.12:0/981672452 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:57.985 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.175+0000 7f991d666700 1 -- 172.21.15.12:0/981672452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991819eb60 con 0x7f9918108fb0 2024-04-22T13:14:57.985 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.176+0000 7f991d666700 1 --2- 172.21.15.12:0/981672452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9918108fb0 0x7f991819e620 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f99140116e0 tx=0x7f9914011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:57.985 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.176+0000 7f990e7fc700 1 -- 172.21.15.12:0/981672452 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9914003a60 con 0x7f9918108fb0 2024-04-22T13:14:57.985 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.176+0000 7f990e7fc700 1 -- 172.21.15.12:0/981672452 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f991401b8f0 con 0x7f9918108fb0 2024-04-22T13:14:57.985 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.176+0000 7f990e7fc700 1 -- 172.21.15.12:0/981672452 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9914003a60 con 0x7f9918108fb0 2024-04-22T13:14:57.985 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.176+0000 7f991f8ca700 1 -- 172.21.15.12:0/981672452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f991819edf0 con 0x7f9918108fb0 2024-04-22T13:14:57.985 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.177+0000 7f991f8ca700 1 -- 172.21.15.12:0/981672452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99181a1ae0 con 0x7f9918108fb0 2024-04-22T13:14:57.985 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.177+0000 7f991f8ca700 1 -- 172.21.15.12:0/981672452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99180640a0 con 0x7f9918108fb0 2024-04-22T13:14:57.985 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.178+0000 7f990e7fc700 1 -- 172.21.15.12:0/981672452 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f991401b440 con 0x7f9918108fb0 2024-04-22T13:14:57.985 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.178+0000 7f990e7fc700 1 --2- 172.21.15.12:0/981672452 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f9904039940 0x7f990403be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:57.985 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.178+0000 7f990e7fc700 1 -- 172.21.15.12:0/981672452 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f991404d680 con 0x7f9918108fb0 2024-04-22T13:14:57.985 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.178+0000 7f991ce65700 1 --2- 172.21.15.12:0/981672452 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f9904039940 0x7f990403be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:57.985 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.178+0000 7f991ce65700 1 --2- 172.21.15.12:0/981672452 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f9904039940 0x7f990403be00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9908006fd0 tx=0x7f9908006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:57.985 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.183+0000 7f990e7fc700 1 -- 172.21.15.12:0/981672452 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9914028070 con 0x7f9918108fb0 2024-04-22T13:14:57.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:56.453+0000 7f991f8ca700 1 -- 172.21.15.12:0/981672452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f99181a2090 con 0x7f9918108fb0 2024-04-22T13:14:57.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:57.106+0000 7f990e7fc700 1 -- 172.21.15.12:0/981672452 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f991401f030 con 0x7f9918108fb0 2024-04-22T13:14:57.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:57.108+0000 7f990e7fc700 1 -- 172.21.15.12:0/981672452 <== mon.0 v2:172.21.15.12:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f99140223b0 con 0x7f9918108fb0 2024-04-22T13:14:57.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:57.110+0000 7f991f8ca700 1 -- 172.21.15.12:0/981672452 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f9904039940 msgr2=0x7f990403be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:57.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:57.110+0000 7f991f8ca700 1 --2- 172.21.15.12:0/981672452 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f9904039940 0x7f990403be00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9908006fd0 tx=0x7f9908006e40 comp rx=0 tx=0).stop 2024-04-22T13:14:57.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:57.110+0000 7f991f8ca700 1 -- 172.21.15.12:0/981672452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9918108fb0 msgr2=0x7f991819e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:57.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:57.110+0000 7f991f8ca700 1 --2- 172.21.15.12:0/981672452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9918108fb0 0x7f991819e620 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f99140116e0 tx=0x7f9914011aa0 comp rx=0 tx=0).stop 2024-04-22T13:14:57.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:57.110+0000 7f991f8ca700 1 -- 172.21.15.12:0/981672452 shutdown_connections 2024-04-22T13:14:57.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:57.110+0000 7f991f8ca700 1 --2- 172.21.15.12:0/981672452 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f9904039940 0x7f990403be00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:57.986 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:57.110+0000 7f991f8ca700 1 --2- 172.21.15.12:0/981672452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9918108fb0 0x7f991819e620 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:57.987 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:57.110+0000 7f991f8ca700 1 -- 172.21.15.12:0/981672452 >> 172.21.15.12:0/981672452 conn(0x7f9918102a40 msgr2=0x7f9918103430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:57.987 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:57.110+0000 7f991f8ca700 1 -- 172.21.15.12:0/981672452 shutdown_connections 2024-04-22T13:14:57.987 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:57.110+0000 7f991f8ca700 1 -- 172.21.15.12:0/981672452 wait complete. 2024-04-22T13:14:58.224 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:58 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/981672452' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-22T13:14:58.225 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:58 smithi012 ceph-mon[28265]: mgrmap e9: x(active, since 20s) 2024-04-22T13:14:58.225 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:58 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:58.011+0000 7f80f9c8a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T13:14:59.362 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:14:59 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/351336625' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-22T13:14:59.363 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:59 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:59.278+0000 7f80f9c8a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T13:14:59.371 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-04-22T13:14:59.371 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-22T13:14:59.371 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-22T13:14:59.371 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-22T13:14:59.371 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-22T13:14:59.371 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-04-22T13:14:59.371 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.460+0000 7f4c13fd3700 1 Processor -- start 2024-04-22T13:14:59.371 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.461+0000 7f4c13fd3700 1 -- start start 2024-04-22T13:14:59.371 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.461+0000 7f4c13fd3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c0c073970 0x7f4c0c073d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:59.371 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.461+0000 7f4c13fd3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c074360 con 0x7f4c0c073970 2024-04-22T13:14:59.371 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.462+0000 7f4c11d6f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c0c073970 0x7f4c0c073d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:59.371 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.462+0000 7f4c11d6f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c0c073970 0x7f4c0c073d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36356/0 (socket says 172.21.15.12:36356) 2024-04-22T13:14:59.372 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.462+0000 7f4c11d6f700 1 -- 172.21.15.12:0/3241833247 learned_addr learned my addr 172.21.15.12:0/3241833247 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:59.372 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.463+0000 7f4c11d6f700 1 -- 172.21.15.12:0/3241833247 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0c10fdb0 con 0x7f4c0c073970 2024-04-22T13:14:59.372 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.464+0000 7f4c11d6f700 1 --2- 172.21.15.12:0/3241833247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c0c073970 0x7f4c0c073d90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4c04009e80 tx=0x7f4c0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982559f797d80fa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:59.372 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.464+0000 7f4c10d6d700 1 -- 172.21.15.12:0/3241833247 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c04017070 con 0x7f4c0c073970 2024-04-22T13:14:59.372 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.465+0000 7f4c10d6d700 1 -- 172.21.15.12:0/3241833247 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c0400b8c0 con 0x7f4c0c073970 2024-04-22T13:14:59.372 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.466+0000 7f4c13fd3700 1 -- 172.21.15.12:0/3241833247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c0c073970 msgr2=0x7f4c0c073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:59.372 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.466+0000 7f4c10d6d700 1 -- 172.21.15.12:0/3241833247 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c04017070 con 0x7f4c0c073970 2024-04-22T13:14:59.372 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.466+0000 7f4c13fd3700 1 --2- 172.21.15.12:0/3241833247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c0c073970 0x7f4c0c073d90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4c04009e80 tx=0x7f4c0400b330 comp rx=0 tx=0).stop 2024-04-22T13:14:59.372 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.466+0000 7f4c13fd3700 1 -- 172.21.15.12:0/3241833247 shutdown_connections 2024-04-22T13:14:59.372 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.466+0000 7f4c13fd3700 1 --2- 172.21.15.12:0/3241833247 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c0c073970 0x7f4c0c073d90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:59.372 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.466+0000 7f4c13fd3700 1 -- 172.21.15.12:0/3241833247 >> 172.21.15.12:0/3241833247 conn(0x7f4c0c06f050 msgr2=0x7f4c0c071470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:59.372 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.466+0000 7f4c13fd3700 1 -- 172.21.15.12:0/3241833247 shutdown_connections 2024-04-22T13:14:59.372 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.466+0000 7f4c13fd3700 1 -- 172.21.15.12:0/3241833247 wait complete. 2024-04-22T13:14:59.373 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.467+0000 7f4c13fd3700 1 Processor -- start 2024-04-22T13:14:59.373 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.467+0000 7f4c13fd3700 1 -- start start 2024-04-22T13:14:59.373 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.467+0000 7f4c13fd3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c0c073970 0x7f4c0c11f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:59.373 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.468+0000 7f4c13fd3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c113000 con 0x7f4c0c073970 2024-04-22T13:14:59.373 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.469+0000 7f4c11d6f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c0c073970 0x7f4c0c11f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:14:59.373 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.469+0000 7f4c11d6f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c0c073970 0x7f4c0c11f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36364/0 (socket says 172.21.15.12:36364) 2024-04-22T13:14:59.373 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.469+0000 7f4c11d6f700 1 -- 172.21.15.12:0/351336625 learned_addr learned my addr 172.21.15.12:0/351336625 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:14:59.373 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.476+0000 7f4c11d6f700 1 -- 172.21.15.12:0/351336625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0c11fb20 con 0x7f4c0c073970 2024-04-22T13:14:59.373 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.476+0000 7f4c11d6f700 1 --2- 172.21.15.12:0/351336625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c0c073970 0x7f4c0c11f5e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4c04003fa0 tx=0x7f4c04004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:14:59.373 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.476+0000 7f4c02ffd700 1 -- 172.21.15.12:0/351336625 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c04020020 con 0x7f4c0c073970 2024-04-22T13:14:59.373 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.476+0000 7f4c13fd3700 1 -- 172.21.15.12:0/351336625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c0c11dd10 con 0x7f4c0c073970 2024-04-22T13:14:59.373 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.476+0000 7f4c02ffd700 1 -- 172.21.15.12:0/351336625 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c04004620 con 0x7f4c0c073970 2024-04-22T13:14:59.373 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.477+0000 7f4c13fd3700 1 -- 172.21.15.12:0/351336625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c0c11e250 con 0x7f4c0c073970 2024-04-22T13:14:59.373 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.477+0000 7f4c02ffd700 1 -- 172.21.15.12:0/351336625 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c04017040 con 0x7f4c0c073970 2024-04-22T13:14:59.373 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.477+0000 7f4c02ffd700 1 -- 172.21.15.12:0/351336625 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f4c04023020 con 0x7f4c0c073970 2024-04-22T13:14:59.374 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.478+0000 7f4c02ffd700 1 --2- 172.21.15.12:0/351336625 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f4bf80395f0 0x7f4bf803bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:14:59.374 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.478+0000 7f4c02ffd700 1 -- 172.21.15.12:0/351336625 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4c0400f050 con 0x7f4c0c073970 2024-04-22T13:14:59.374 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.478+0000 7f4c1156e700 1 -- 172.21.15.12:0/351336625 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f4bf80395f0 msgr2=0x7f4bf803bab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2009831940 2024-04-22T13:14:59.374 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.478+0000 7f4c1156e700 1 --2- 172.21.15.12:0/351336625 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f4bf80395f0 0x7f4bf803bab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T13:14:59.374 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.478+0000 7f4c13fd3700 1 -- 172.21.15.12:0/351336625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c0c113180 con 0x7f4c0c073970 2024-04-22T13:14:59.374 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.484+0000 7f4c02ffd700 1 -- 172.21.15.12:0/351336625 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4c0402a020 con 0x7f4c0c073970 2024-04-22T13:14:59.374 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.679+0000 7f4c1156e700 1 -- 172.21.15.12:0/351336625 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f4bf80395f0 msgr2=0x7f4bf803bab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2009831940 2024-04-22T13:14:59.374 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.679+0000 7f4c1156e700 1 --2- 172.21.15.12:0/351336625 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f4bf80395f0 0x7f4bf803bab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T13:14:59.374 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.752+0000 7f4c13fd3700 1 -- 172.21.15.12:0/351336625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f4c0c11ea70 con 0x7f4c0c073970 2024-04-22T13:14:59.374 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.753+0000 7f4c02ffd700 1 -- 172.21.15.12:0/351336625 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f4c04007310 con 0x7f4c0c073970 2024-04-22T13:14:59.374 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.755+0000 7f4c13fd3700 1 -- 172.21.15.12:0/351336625 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f4bf80395f0 msgr2=0x7f4bf803bab0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T13:14:59.374 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.755+0000 7f4c13fd3700 1 --2- 172.21.15.12:0/351336625 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f4bf80395f0 0x7f4bf803bab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:59.374 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.755+0000 7f4c13fd3700 1 -- 172.21.15.12:0/351336625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c0c073970 msgr2=0x7f4c0c11f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:14:59.374 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.755+0000 7f4c13fd3700 1 --2- 172.21.15.12:0/351336625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c0c073970 0x7f4c0c11f5e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4c04003fa0 tx=0x7f4c04004310 comp rx=0 tx=0).stop 2024-04-22T13:14:59.375 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.755+0000 7f4c13fd3700 1 -- 172.21.15.12:0/351336625 shutdown_connections 2024-04-22T13:14:59.375 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.755+0000 7f4c13fd3700 1 --2- 172.21.15.12:0/351336625 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7f4bf80395f0 0x7f4bf803bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:59.375 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.755+0000 7f4c13fd3700 1 --2- 172.21.15.12:0/351336625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4c0c073970 0x7f4c0c11f5e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:14:59.375 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.755+0000 7f4c13fd3700 1 -- 172.21.15.12:0/351336625 >> 172.21.15.12:0/351336625 conn(0x7f4c0c06f050 msgr2=0x7f4c0c070b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:14:59.375 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.755+0000 7f4c13fd3700 1 -- 172.21.15.12:0/351336625 shutdown_connections 2024-04-22T13:14:59.375 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:58.755+0000 7f4c13fd3700 1 -- 172.21.15.12:0/351336625 wait complete. 2024-04-22T13:14:59.375 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for the mgr to restart... 2024-04-22T13:14:59.375 INFO:teuthology.orchestra.run.smithi012.stdout:Waiting for mgr epoch 9... 2024-04-22T13:14:59.653 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:59 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:59.469+0000 7f80f9c8a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T13:14:59.654 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:59 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:59.654+0000 7f80f9c8a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T13:15:00.266 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:14:59 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:14:59.960+0000 7f80f9c8a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T13:15:00.266 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:15:00 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:15:00.056+0000 7f80f9c8a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T13:15:00.266 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:15:00 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:15:00.267+0000 7f80f9c8a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T13:15:00.589 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:15:00 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:15:00.469+0000 7f80f9c8a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:15:01.537 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:15:01 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:15:01.096+0000 7f80f9c8a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T13:15:01.839 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:15:01 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:15:01.539+0000 7f80f9c8a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T13:15:01.839 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:15:01 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:15:01.639+0000 7f80f9c8a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T13:15:02.339 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:15:01 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:15:01.850+0000 7f80f9c8a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T13:15:02.748 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:15:02 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:15:02.399+0000 7f80f9c8a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T13:15:02.748 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:15:02 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:15:02.496+0000 7f80f9c8a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T13:15:03.089 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:15:02 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:15:02.750+0000 7f80f9c8a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T13:15:03.089 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:15:02 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:15:02.889+0000 7f80f9c8a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T13:15:03.089 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:15:02 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:15:02.987+0000 7f80f9c8a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T13:15:03.418 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:15:03 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:15:03.107+0000 7f80f9c8a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T13:15:03.419 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:15:03 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:15:03.420+0000 7f80f9c8a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T13:15:04.330 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:15:04 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:15:04.050+0000 7f80f9c8a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T13:15:04.330 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:15:04 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:15:04.277+0000 7f80f9c8a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T13:15:04.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:04 smithi012 ceph-mon[28265]: Active manager daemon x restarted 2024-04-22T13:15:04.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:04 smithi012 ceph-mon[28265]: Activating manager daemon x 2024-04-22T13:15:04.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:04 smithi012 ceph-mon[28265]: osdmap e3: 0 total, 0 up, 0 in 2024-04-22T13:15:04.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:04 smithi012 ceph-mon[28265]: mgrmap e10: x(active, starting, since 0.00343536s) 2024-04-22T13:15:04.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:15:04.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-22T13:15:04.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T13:15:04.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T13:15:04.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T13:15:04.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:04 smithi012 ceph-mon[28265]: Manager daemon x is now available 2024-04-22T13:15:04.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:15:04.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-22T13:15:05.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:05 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-22T13:15:05.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:05 smithi012 ceph-mon[28265]: [22/Apr/2024:13:15:04] ENGINE Bus STARTING 2024-04-22T13:15:05.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:05 smithi012 ceph-mon[28265]: [22/Apr/2024:13:15:05] ENGINE Serving on https://172.21.15.12:7150 2024-04-22T13:15:05.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:05 smithi012 ceph-mon[28265]: mgrmap e11: x(active, since 1.00654s) 2024-04-22T13:15:06.374 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout { 2024-04-22T13:15:06.382 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-22T13:15:06.383 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-22T13:15:06.383 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout } 2024-04-22T13:15:06.383 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.804+0000 7fe3000cc700 1 Processor -- start 2024-04-22T13:15:06.383 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.804+0000 7fe3000cc700 1 -- start start 2024-04-22T13:15:06.383 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.804+0000 7fe3000cc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f8108560 0x7fe2f8108980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:06.383 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.804+0000 7fe3000cc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f8108f50 con 0x7fe2f8108560 2024-04-22T13:15:06.383 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.804+0000 7fe2fde68700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f8108560 0x7fe2f8108980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:06.383 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.804+0000 7fe2fde68700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f8108560 0x7fe2f8108980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36368/0 (socket says 172.21.15.12:36368) 2024-04-22T13:15:06.383 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.804+0000 7fe2fde68700 1 -- 172.21.15.12:0/3612392274 learned_addr learned my addr 172.21.15.12:0/3612392274 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:15:06.383 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.805+0000 7fe2fde68700 1 -- 172.21.15.12:0/3612392274 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f8109710 con 0x7fe2f8108560 2024-04-22T13:15:06.383 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.805+0000 7fe2fde68700 1 --2- 172.21.15.12:0/3612392274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f8108560 0x7fe2f8108980 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe2f40098a0 tx=0x7fe2f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e00c3de5075434f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:06.383 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.805+0000 7fe2fce66700 1 -- 172.21.15.12:0/3612392274 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2f4004160 con 0x7fe2f8108560 2024-04-22T13:15:06.383 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.805+0000 7fe2fce66700 1 -- 172.21.15.12:0/3612392274 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2f4004300 con 0x7fe2f8108560 2024-04-22T13:15:06.384 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.805+0000 7fe2fce66700 1 -- 172.21.15.12:0/3612392274 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2f4011a20 con 0x7fe2f8108560 2024-04-22T13:15:06.384 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.806+0000 7fe3000cc700 1 -- 172.21.15.12:0/3612392274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f8108560 msgr2=0x7fe2f8108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:06.384 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.806+0000 7fe3000cc700 1 --2- 172.21.15.12:0/3612392274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f8108560 0x7fe2f8108980 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe2f40098a0 tx=0x7fe2f4009bb0 comp rx=0 tx=0).stop 2024-04-22T13:15:06.384 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.806+0000 7fe3000cc700 1 -- 172.21.15.12:0/3612392274 shutdown_connections 2024-04-22T13:15:06.384 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.806+0000 7fe3000cc700 1 --2- 172.21.15.12:0/3612392274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f8108560 0x7fe2f8108980 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:06.384 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.806+0000 7fe3000cc700 1 -- 172.21.15.12:0/3612392274 >> 172.21.15.12:0/3612392274 conn(0x7fe2f8103b40 msgr2=0x7fe2f8105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:06.384 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.806+0000 7fe3000cc700 1 -- 172.21.15.12:0/3612392274 shutdown_connections 2024-04-22T13:15:06.384 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.806+0000 7fe3000cc700 1 -- 172.21.15.12:0/3612392274 wait complete. 2024-04-22T13:15:06.384 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.807+0000 7fe3000cc700 1 Processor -- start 2024-04-22T13:15:06.384 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.807+0000 7fe3000cc700 1 -- start start 2024-04-22T13:15:06.384 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.807+0000 7fe3000cc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f8108560 0x7fe2f819c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:06.384 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.807+0000 7fe3000cc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f810cac0 con 0x7fe2f8108560 2024-04-22T13:15:06.384 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.807+0000 7fe2fde68700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f8108560 0x7fe2f819c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:06.384 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.807+0000 7fe2fde68700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f8108560 0x7fe2f819c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36378/0 (socket says 172.21.15.12:36378) 2024-04-22T13:15:06.385 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.807+0000 7fe2fde68700 1 -- 172.21.15.12:0/2245787242 learned_addr learned my addr 172.21.15.12:0/2245787242 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:15:06.385 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.807+0000 7fe2fde68700 1 -- 172.21.15.12:0/2245787242 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f819ca00 con 0x7fe2f8108560 2024-04-22T13:15:06.385 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.807+0000 7fe2fde68700 1 --2- 172.21.15.12:0/2245787242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f8108560 0x7fe2f819c4c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe2f40119f0 tx=0x7fe2f4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:06.385 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.808+0000 7fe2eeffd700 1 -- 172.21.15.12:0/2245787242 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2f40039e0 con 0x7fe2f8108560 2024-04-22T13:15:06.385 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.808+0000 7fe3000cc700 1 -- 172.21.15.12:0/2245787242 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2f819cc90 con 0x7fe2f8108560 2024-04-22T13:15:06.385 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.808+0000 7fe2eeffd700 1 -- 172.21.15.12:0/2245787242 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2f401b8f0 con 0x7fe2f8108560 2024-04-22T13:15:06.385 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.808+0000 7fe2eeffd700 1 -- 172.21.15.12:0/2245787242 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2f40196d0 con 0x7fe2f8108560 2024-04-22T13:15:06.386 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.808+0000 7fe3000cc700 1 -- 172.21.15.12:0/2245787242 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2f819f980 con 0x7fe2f8108560 2024-04-22T13:15:06.386 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.809+0000 7fe2eeffd700 1 -- 172.21.15.12:0/2245787242 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fe2f4019950 con 0x7fe2f8108560 2024-04-22T13:15:06.386 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.809+0000 7fe2eeffd700 1 --2- 172.21.15.12:0/2245787242 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fe2e40395b0 0x7fe2e403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:06.386 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.809+0000 7fe2eeffd700 1 -- 172.21.15.12:0/2245787242 --> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe2e403c180 con 0x7fe2e40395b0 2024-04-22T13:15:06.386 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.809+0000 7fe2fd667700 1 -- 172.21.15.12:0/2245787242 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fe2e40395b0 msgr2=0x7fe2e403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2009831940 2024-04-22T13:15:06.387 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.809+0000 7fe2eeffd700 1 -- 172.21.15.12:0/2245787242 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe2f404e030 con 0x7fe2f8108560 2024-04-22T13:15:06.387 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:14:59.809+0000 7fe2fd667700 1 --2- 172.21.15.12:0/2245787242 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fe2e40395b0 0x7fe2e403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T13:15:06.387 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:00.010+0000 7fe2fd667700 1 -- 172.21.15.12:0/2245787242 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fe2e40395b0 msgr2=0x7fe2e403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2009831940 2024-04-22T13:15:06.387 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:00.010+0000 7fe2fd667700 1 --2- 172.21.15.12:0/2245787242 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fe2e40395b0 0x7fe2e403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T13:15:06.387 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:00.411+0000 7fe2fd667700 1 -- 172.21.15.12:0/2245787242 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fe2e40395b0 msgr2=0x7fe2e403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2009831940 2024-04-22T13:15:06.387 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:00.411+0000 7fe2fd667700 1 --2- 172.21.15.12:0/2245787242 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fe2e40395b0 0x7fe2e403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T13:15:06.387 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:01.212+0000 7fe2fd667700 1 -- 172.21.15.12:0/2245787242 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fe2e40395b0 msgr2=0x7fe2e403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2009831940 2024-04-22T13:15:06.387 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:01.212+0000 7fe2fd667700 1 --2- 172.21.15.12:0/2245787242 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fe2e40395b0 0x7fe2e403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-22T13:15:06.387 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:02.813+0000 7fe2fd667700 1 -- 172.21.15.12:0/2245787242 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fe2e40395b0 msgr2=0x7fe2e403ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/2009831940 2024-04-22T13:15:06.387 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:02.813+0000 7fe2fd667700 1 --2- 172.21.15.12:0/2245787242 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fe2e40395b0 0x7fe2e403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-22T13:15:06.387 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:04.281+0000 7fe2eeffd700 1 -- 172.21.15.12:0/2245787242 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7fe2f4019ce0 con 0x7fe2f8108560 2024-04-22T13:15:06.387 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:04.281+0000 7fe2eeffd700 1 -- 172.21.15.12:0/2245787242 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fe2e40395b0 msgr2=0x7fe2e403ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T13:15:06.387 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:04.281+0000 7fe2eeffd700 1 --2- 172.21.15.12:0/2245787242 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fe2e40395b0 0x7fe2e403ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:06.388 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:05.283+0000 7fe2eeffd700 1 -- 172.21.15.12:0/2245787242 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7fe2f401a3d0 con 0x7fe2f8108560 2024-04-22T13:15:06.388 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:05.283+0000 7fe2eeffd700 1 --2- 172.21.15.12:0/2245787242 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe2e403cfe0 0x7fe2e403f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:06.388 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:05.283+0000 7fe2eeffd700 1 -- 172.21.15.12:0/2245787242 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe2e403c180 con 0x7fe2e403cfe0 2024-04-22T13:15:06.388 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:05.284+0000 7fe2fd667700 1 --2- 172.21.15.12:0/2245787242 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe2e403cfe0 0x7fe2e403f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:06.388 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:05.285+0000 7fe2fd667700 1 --2- 172.21.15.12:0/2245787242 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe2e403cfe0 0x7fe2e403f3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8003a10 tx=0x7fe2e80092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:06.388 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:05.285+0000 7fe2eeffd700 1 -- 172.21.15.12:0/2245787242 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fe2e403c180 con 0x7fe2e403cfe0 2024-04-22T13:15:06.388 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:05.292+0000 7fe3000cc700 1 -- 172.21.15.12:0/2245787242 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fe2f8069ba0 con 0x7fe2e403cfe0 2024-04-22T13:15:06.388 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:05.293+0000 7fe2eeffd700 1 -- 172.21.15.12:0/2245787242 <== mgr.14152 v2:172.21.15.12:6800/1802873236 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fe2f8069ba0 con 0x7fe2e403cfe0 2024-04-22T13:15:06.388 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:05.293+0000 7fe3000cc700 1 -- 172.21.15.12:0/2245787242 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe2e403cfe0 msgr2=0x7fe2e403f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:06.388 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:05.293+0000 7fe3000cc700 1 --2- 172.21.15.12:0/2245787242 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe2e403cfe0 0x7fe2e403f3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8003a10 tx=0x7fe2e80092b0 comp rx=0 tx=0).stop 2024-04-22T13:15:06.388 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:05.293+0000 7fe3000cc700 1 -- 172.21.15.12:0/2245787242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f8108560 msgr2=0x7fe2f819c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:06.388 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:05.293+0000 7fe3000cc700 1 --2- 172.21.15.12:0/2245787242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f8108560 0x7fe2f819c4c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe2f40119f0 tx=0x7fe2f4011ad0 comp rx=0 tx=0).stop 2024-04-22T13:15:06.388 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:05.293+0000 7fe3000cc700 1 -- 172.21.15.12:0/2245787242 shutdown_connections 2024-04-22T13:15:06.389 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:05.293+0000 7fe3000cc700 1 --2- 172.21.15.12:0/2245787242 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe2e403cfe0 0x7fe2e403f3d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:06.389 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:05.293+0000 7fe3000cc700 1 --2- 172.21.15.12:0/2245787242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe2f8108560 0x7fe2f819c4c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:06.389 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:05.293+0000 7fe3000cc700 1 --2- 172.21.15.12:0/2245787242 >> [v2:172.21.15.12:6800/2009831940,v1:172.21.15.12:6801/2009831940] conn(0x7fe2e40395b0 0x7fe2e403ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:06.389 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:05.293+0000 7fe3000cc700 1 -- 172.21.15.12:0/2245787242 >> 172.21.15.12:0/2245787242 conn(0x7fe2f8103b40 msgr2=0x7fe2f81046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:06.389 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:05.293+0000 7fe3000cc700 1 -- 172.21.15.12:0/2245787242 shutdown_connections 2024-04-22T13:15:06.389 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:05.293+0000 7fe3000cc700 1 -- 172.21.15.12:0/2245787242 wait complete. 2024-04-22T13:15:06.389 INFO:teuthology.orchestra.run.smithi012.stdout:mgr epoch 9 is available 2024-04-22T13:15:06.389 INFO:teuthology.orchestra.run.smithi012.stdout:Generating a dashboard self-signed certificate... 2024-04-22T13:15:06.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:06 smithi012 ceph-mon[28265]: [22/Apr/2024:13:15:05] ENGINE Serving on http://172.21.15.12:8765 2024-04-22T13:15:06.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:06 smithi012 ceph-mon[28265]: [22/Apr/2024:13:15:05] ENGINE Bus STARTED 2024-04-22T13:15:06.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:06 smithi012 ceph-mon[28265]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-22T13:15:06.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:06 smithi012 ceph-mon[28265]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-22T13:15:07.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:07 smithi012 ceph-mon[28265]: mgrmap e12: x(active, since 2s) 2024-04-22T13:15:07.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:07 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:07.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:07 smithi012 ceph-mon[28265]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:15:07.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:07 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:07.591 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:07 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:07.751 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-22T13:15:07.751 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.844+0000 7fbb4efab700 1 Processor -- start 2024-04-22T13:15:07.751 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.844+0000 7fbb4efab700 1 -- start start 2024-04-22T13:15:07.751 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.845+0000 7fbb4efab700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb48108ff0 0x7fbb4810b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:07.751 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.845+0000 7fbb4efab700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbb48063d20 con 0x7fbb48108ff0 2024-04-22T13:15:07.751 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.845+0000 7fbb4cd47700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb48108ff0 0x7fbb4810b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:07.751 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.845+0000 7fbb4cd47700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb48108ff0 0x7fbb4810b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42918/0 (socket says 172.21.15.12:42918) 2024-04-22T13:15:07.751 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.845+0000 7fbb4cd47700 1 -- 172.21.15.12:0/3003434348 learned_addr learned my addr 172.21.15.12:0/3003434348 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:15:07.752 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.845+0000 7fbb4cd47700 1 -- 172.21.15.12:0/3003434348 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb4810b920 con 0x7fbb48108ff0 2024-04-22T13:15:07.752 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.845+0000 7fbb4cd47700 1 --2- 172.21.15.12:0/3003434348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb48108ff0 0x7fbb4810b3e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c0098a0 tx=0x7fbb3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7de5b89ef20ddaf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:07.752 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.845+0000 7fbb477fe700 1 -- 172.21.15.12:0/3003434348 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb3c004160 con 0x7fbb48108ff0 2024-04-22T13:15:07.752 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.845+0000 7fbb477fe700 1 -- 172.21.15.12:0/3003434348 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb3c004300 con 0x7fbb48108ff0 2024-04-22T13:15:07.752 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.845+0000 7fbb477fe700 1 -- 172.21.15.12:0/3003434348 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb3c011a20 con 0x7fbb48108ff0 2024-04-22T13:15:07.752 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.846+0000 7fbb4efab700 1 -- 172.21.15.12:0/3003434348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb48108ff0 msgr2=0x7fbb4810b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:07.752 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.846+0000 7fbb4efab700 1 --2- 172.21.15.12:0/3003434348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb48108ff0 0x7fbb4810b3e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c0098a0 tx=0x7fbb3c009bb0 comp rx=0 tx=0).stop 2024-04-22T13:15:07.752 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.846+0000 7fbb4efab700 1 -- 172.21.15.12:0/3003434348 shutdown_connections 2024-04-22T13:15:07.752 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.846+0000 7fbb4efab700 1 --2- 172.21.15.12:0/3003434348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb48108ff0 0x7fbb4810b3e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:07.752 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.846+0000 7fbb4efab700 1 -- 172.21.15.12:0/3003434348 >> 172.21.15.12:0/3003434348 conn(0x7fbb48102a40 msgr2=0x7fbb48104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:07.752 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.846+0000 7fbb4efab700 1 -- 172.21.15.12:0/3003434348 shutdown_connections 2024-04-22T13:15:07.752 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.846+0000 7fbb4efab700 1 -- 172.21.15.12:0/3003434348 wait complete. 2024-04-22T13:15:07.752 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.847+0000 7fbb4efab700 1 Processor -- start 2024-04-22T13:15:07.753 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.847+0000 7fbb4efab700 1 -- start start 2024-04-22T13:15:07.753 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.847+0000 7fbb4efab700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb48108ff0 0x7fbb4819c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:07.753 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.847+0000 7fbb4efab700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4810c920 con 0x7fbb48108ff0 2024-04-22T13:15:07.753 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.847+0000 7fbb4cd47700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb48108ff0 0x7fbb4819c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:07.753 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.847+0000 7fbb4cd47700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb48108ff0 0x7fbb4819c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42924/0 (socket says 172.21.15.12:42924) 2024-04-22T13:15:07.753 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.847+0000 7fbb4cd47700 1 -- 172.21.15.12:0/1885683309 learned_addr learned my addr 172.21.15.12:0/1885683309 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:15:07.753 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.847+0000 7fbb4cd47700 1 -- 172.21.15.12:0/1885683309 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb4819c920 con 0x7fbb48108ff0 2024-04-22T13:15:07.753 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.847+0000 7fbb4cd47700 1 --2- 172.21.15.12:0/1885683309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb48108ff0 0x7fbb4819c3e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c0116e0 tx=0x7fbb3c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:07.753 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.848+0000 7fbb45ffb700 1 -- 172.21.15.12:0/1885683309 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb3c003a60 con 0x7fbb48108ff0 2024-04-22T13:15:07.753 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.848+0000 7fbb4efab700 1 -- 172.21.15.12:0/1885683309 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb4819cbb0 con 0x7fbb48108ff0 2024-04-22T13:15:07.753 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.848+0000 7fbb45ffb700 1 -- 172.21.15.12:0/1885683309 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb3c01b8f0 con 0x7fbb48108ff0 2024-04-22T13:15:07.753 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.848+0000 7fbb45ffb700 1 -- 172.21.15.12:0/1885683309 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb3c019860 con 0x7fbb48108ff0 2024-04-22T13:15:07.753 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.848+0000 7fbb4efab700 1 -- 172.21.15.12:0/1885683309 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb4819f8a0 con 0x7fbb48108ff0 2024-04-22T13:15:07.753 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.849+0000 7fbb45ffb700 1 -- 172.21.15.12:0/1885683309 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fbb3c019b60 con 0x7fbb48108ff0 2024-04-22T13:15:07.754 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.849+0000 7fbb45ffb700 1 --2- 172.21.15.12:0/1885683309 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fbb380395b0 0x7fbb3803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:07.754 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.849+0000 7fbb4efab700 1 -- 172.21.15.12:0/1885683309 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb480640a0 con 0x7fbb48108ff0 2024-04-22T13:15:07.754 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.849+0000 7fbb45ffb700 1 -- 172.21.15.12:0/1885683309 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbb3c01b440 con 0x7fbb48108ff0 2024-04-22T13:15:07.754 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.849+0000 7fbb47fff700 1 --2- 172.21.15.12:0/1885683309 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fbb380395b0 0x7fbb3803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:07.754 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.850+0000 7fbb47fff700 1 --2- 172.21.15.12:0/1885683309 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fbb380395b0 0x7fbb3803ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbb34006fd0 tx=0x7fbb34006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:07.754 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:06.855+0000 7fbb45ffb700 1 -- 172.21.15.12:0/1885683309 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbb3c01f030 con 0x7fbb48108ff0 2024-04-22T13:15:07.754 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:07.068+0000 7fbb4efab700 1 -- 172.21.15.12:0/1885683309 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fbb48103bc0 con 0x7fbb380395b0 2024-04-22T13:15:07.754 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:07.145+0000 7fbb45ffb700 1 -- 172.21.15.12:0/1885683309 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fbb48103bc0 con 0x7fbb380395b0 2024-04-22T13:15:07.754 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:07.147+0000 7fbb4efab700 1 -- 172.21.15.12:0/1885683309 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fbb380395b0 msgr2=0x7fbb3803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:07.754 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:07.147+0000 7fbb4efab700 1 --2- 172.21.15.12:0/1885683309 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fbb380395b0 0x7fbb3803ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbb34006fd0 tx=0x7fbb34006e40 comp rx=0 tx=0).stop 2024-04-22T13:15:07.754 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:07.147+0000 7fbb4efab700 1 -- 172.21.15.12:0/1885683309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb48108ff0 msgr2=0x7fbb4819c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:07.754 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:07.147+0000 7fbb4efab700 1 --2- 172.21.15.12:0/1885683309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb48108ff0 0x7fbb4819c3e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c0116e0 tx=0x7fbb3c011aa0 comp rx=0 tx=0).stop 2024-04-22T13:15:07.754 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:07.147+0000 7fbb4efab700 1 -- 172.21.15.12:0/1885683309 shutdown_connections 2024-04-22T13:15:07.754 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:07.147+0000 7fbb4efab700 1 --2- 172.21.15.12:0/1885683309 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fbb380395b0 0x7fbb3803ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:07.755 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:07.147+0000 7fbb4efab700 1 --2- 172.21.15.12:0/1885683309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb48108ff0 0x7fbb4819c3e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:07.755 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:07.147+0000 7fbb4efab700 1 -- 172.21.15.12:0/1885683309 >> 172.21.15.12:0/1885683309 conn(0x7fbb48102a40 msgr2=0x7fbb481034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:07.755 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:07.147+0000 7fbb4efab700 1 -- 172.21.15.12:0/1885683309 shutdown_connections 2024-04-22T13:15:07.755 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:07.147+0000 7fbb4efab700 1 -- 172.21.15.12:0/1885683309 wait complete. 2024-04-22T13:15:07.755 INFO:teuthology.orchestra.run.smithi012.stdout:Creating initial admin user... 2024-04-22T13:15:09.343 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:09 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:09.343 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:09 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:10.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:10 smithi012 ceph-mon[28265]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:15:10.204 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$SgULT.lOfx.IdczIZgMYU.SlCVbVWxb0itpXp7ftwa0877bdsQHFm", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713791708, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-22T13:15:10.204 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.182+0000 7fb0ab091700 1 Processor -- start 2024-04-22T13:15:10.205 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.182+0000 7fb0ab091700 1 -- start start 2024-04-22T13:15:10.205 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.182+0000 7fb0ab091700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0a4108570 0x7fb0a4108990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:10.205 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.182+0000 7fb0ab091700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a4108f60 con 0x7fb0a4108570 2024-04-22T13:15:10.205 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.182+0000 7fb0a8e2d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0a4108570 0x7fb0a4108990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:10.205 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.182+0000 7fb0a8e2d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0a4108570 0x7fb0a4108990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42936/0 (socket says 172.21.15.12:42936) 2024-04-22T13:15:10.205 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.182+0000 7fb0a8e2d700 1 -- 172.21.15.12:0/4245500868 learned_addr learned my addr 172.21.15.12:0/4245500868 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:15:10.205 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.183+0000 7fb0a8e2d700 1 -- 172.21.15.12:0/4245500868 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a4109720 con 0x7fb0a4108570 2024-04-22T13:15:10.205 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.183+0000 7fb0a8e2d700 1 --2- 172.21.15.12:0/4245500868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0a4108570 0x7fb0a4108990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb0980098a0 tx=0x7fb098009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571af929fb4de416 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:10.205 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.183+0000 7fb0a37fe700 1 -- 172.21.15.12:0/4245500868 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb098004160 con 0x7fb0a4108570 2024-04-22T13:15:10.205 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.183+0000 7fb0a37fe700 1 -- 172.21.15.12:0/4245500868 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb098004300 con 0x7fb0a4108570 2024-04-22T13:15:10.205 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.183+0000 7fb0a37fe700 1 -- 172.21.15.12:0/4245500868 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb098011a20 con 0x7fb0a4108570 2024-04-22T13:15:10.206 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.184+0000 7fb0ab091700 1 -- 172.21.15.12:0/4245500868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0a4108570 msgr2=0x7fb0a4108990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:10.206 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.184+0000 7fb0ab091700 1 --2- 172.21.15.12:0/4245500868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0a4108570 0x7fb0a4108990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb0980098a0 tx=0x7fb098009bb0 comp rx=0 tx=0).stop 2024-04-22T13:15:10.206 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.184+0000 7fb0ab091700 1 -- 172.21.15.12:0/4245500868 shutdown_connections 2024-04-22T13:15:10.206 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.184+0000 7fb0ab091700 1 --2- 172.21.15.12:0/4245500868 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0a4108570 0x7fb0a4108990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:10.206 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.184+0000 7fb0ab091700 1 -- 172.21.15.12:0/4245500868 >> 172.21.15.12:0/4245500868 conn(0x7fb0a4103b70 msgr2=0x7fb0a4105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:10.206 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.184+0000 7fb0ab091700 1 -- 172.21.15.12:0/4245500868 shutdown_connections 2024-04-22T13:15:10.206 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.184+0000 7fb0ab091700 1 -- 172.21.15.12:0/4245500868 wait complete. 2024-04-22T13:15:10.206 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.184+0000 7fb0ab091700 1 Processor -- start 2024-04-22T13:15:10.206 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.184+0000 7fb0ab091700 1 -- start start 2024-04-22T13:15:10.206 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.184+0000 7fb0ab091700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0a4108570 0x7fb0a419e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:10.206 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.184+0000 7fb0ab091700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a410cad0 con 0x7fb0a4108570 2024-04-22T13:15:10.206 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.185+0000 7fb0a8e2d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0a4108570 0x7fb0a419e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:10.206 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.185+0000 7fb0a8e2d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0a4108570 0x7fb0a419e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42948/0 (socket says 172.21.15.12:42948) 2024-04-22T13:15:10.207 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.185+0000 7fb0a8e2d700 1 -- 172.21.15.12:0/1954969051 learned_addr learned my addr 172.21.15.12:0/1954969051 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:15:10.207 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.185+0000 7fb0a8e2d700 1 -- 172.21.15.12:0/1954969051 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a419ebb0 con 0x7fb0a4108570 2024-04-22T13:15:10.207 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.185+0000 7fb0a8e2d700 1 --2- 172.21.15.12:0/1954969051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0a4108570 0x7fb0a419e670 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb0980119f0 tx=0x7fb098011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:10.207 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.185+0000 7fb0a1ffb700 1 -- 172.21.15.12:0/1954969051 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0980039e0 con 0x7fb0a4108570 2024-04-22T13:15:10.207 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.185+0000 7fb0ab091700 1 -- 172.21.15.12:0/1954969051 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0a419ee40 con 0x7fb0a4108570 2024-04-22T13:15:10.207 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.185+0000 7fb0a1ffb700 1 -- 172.21.15.12:0/1954969051 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb09801b8f0 con 0x7fb0a4108570 2024-04-22T13:15:10.207 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.185+0000 7fb0a1ffb700 1 -- 172.21.15.12:0/1954969051 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0980237a0 con 0x7fb0a4108570 2024-04-22T13:15:10.207 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.185+0000 7fb0ab091700 1 -- 172.21.15.12:0/1954969051 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0a41a1b30 con 0x7fb0a4108570 2024-04-22T13:15:10.207 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.186+0000 7fb0a1ffb700 1 -- 172.21.15.12:0/1954969051 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fb098023aa0 con 0x7fb0a4108570 2024-04-22T13:15:10.207 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.186+0000 7fb0ab091700 1 -- 172.21.15.12:0/1954969051 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0a40640a0 con 0x7fb0a4108570 2024-04-22T13:15:10.207 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.187+0000 7fb0a1ffb700 1 --2- 172.21.15.12:0/1954969051 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb0940395b0 0x7fb09403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:10.207 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.187+0000 7fb0a1ffb700 1 -- 172.21.15.12:0/1954969051 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb09804dfa0 con 0x7fb0a4108570 2024-04-22T13:15:10.207 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.187+0000 7fb0a3fff700 1 --2- 172.21.15.12:0/1954969051 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb0940395b0 0x7fb09403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:10.207 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.187+0000 7fb0a3fff700 1 --2- 172.21.15.12:0/1954969051 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb0940395b0 0x7fb09403ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb090006fd0 tx=0x7fb090006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:10.207 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.192+0000 7fb0a1ffb700 1 -- 172.21.15.12:0/1954969051 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb09801f030 con 0x7fb0a4108570 2024-04-22T13:15:10.208 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.420+0000 7fb0ab091700 1 -- 172.21.15.12:0/1954969051 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fb0a41a1dc0 con 0x7fb0940395b0 2024-04-22T13:15:10.208 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.666+0000 7fb0a1ffb700 1 -- 172.21.15.12:0/1954969051 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fb0a41a1dc0 con 0x7fb0940395b0 2024-04-22T13:15:10.208 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.669+0000 7fb0ab091700 1 -- 172.21.15.12:0/1954969051 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb0940395b0 msgr2=0x7fb09403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:10.208 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.669+0000 7fb0ab091700 1 --2- 172.21.15.12:0/1954969051 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb0940395b0 0x7fb09403ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb090006fd0 tx=0x7fb090006e40 comp rx=0 tx=0).stop 2024-04-22T13:15:10.208 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.669+0000 7fb0ab091700 1 -- 172.21.15.12:0/1954969051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0a4108570 msgr2=0x7fb0a419e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:10.208 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.669+0000 7fb0ab091700 1 --2- 172.21.15.12:0/1954969051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0a4108570 0x7fb0a419e670 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb0980119f0 tx=0x7fb098011ad0 comp rx=0 tx=0).stop 2024-04-22T13:15:10.208 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.669+0000 7fb0ab091700 1 -- 172.21.15.12:0/1954969051 shutdown_connections 2024-04-22T13:15:10.208 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.669+0000 7fb0ab091700 1 --2- 172.21.15.12:0/1954969051 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb0940395b0 0x7fb09403ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:10.208 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.669+0000 7fb0ab091700 1 --2- 172.21.15.12:0/1954969051 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0a4108570 0x7fb0a419e670 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:10.208 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.669+0000 7fb0ab091700 1 -- 172.21.15.12:0/1954969051 >> 172.21.15.12:0/1954969051 conn(0x7fb0a4103b70 msgr2=0x7fb0a4104620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:10.208 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.669+0000 7fb0ab091700 1 -- 172.21.15.12:0/1954969051 shutdown_connections 2024-04-22T13:15:10.208 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:08.669+0000 7fb0ab091700 1 -- 172.21.15.12:0/1954969051 wait complete. 2024-04-22T13:15:10.208 INFO:teuthology.orchestra.run.smithi012.stdout:Fetching dashboard port number... 2024-04-22T13:15:11.193 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:11 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2434476149' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-22T13:15:11.502 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stdout 8443 2024-04-22T13:15:11.502 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.659+0000 7f4b39a6e700 1 Processor -- start 2024-04-22T13:15:11.503 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.659+0000 7f4b39a6e700 1 -- start start 2024-04-22T13:15:11.503 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.659+0000 7f4b39a6e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b34108570 0x7f4b34108990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:11.503 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.659+0000 7f4b39a6e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4b34108f60 con 0x7f4b34108570 2024-04-22T13:15:11.503 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.659+0000 7f4b337fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b34108570 0x7f4b34108990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:11.503 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.659+0000 7f4b337fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b34108570 0x7f4b34108990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42964/0 (socket says 172.21.15.12:42964) 2024-04-22T13:15:11.503 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.659+0000 7f4b337fe700 1 -- 172.21.15.12:0/1089934976 learned_addr learned my addr 172.21.15.12:0/1089934976 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:15:11.503 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.660+0000 7f4b337fe700 1 -- 172.21.15.12:0/1089934976 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b34109720 con 0x7f4b34108570 2024-04-22T13:15:11.503 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.660+0000 7f4b337fe700 1 --2- 172.21.15.12:0/1089934976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b34108570 0x7f4b34108990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4b240098a0 tx=0x7f4b24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e742b447fb952a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:11.503 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.660+0000 7f4b327fc700 1 -- 172.21.15.12:0/1089934976 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b24004160 con 0x7f4b34108570 2024-04-22T13:15:11.503 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.660+0000 7f4b327fc700 1 -- 172.21.15.12:0/1089934976 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b24004300 con 0x7f4b34108570 2024-04-22T13:15:11.503 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.660+0000 7f4b327fc700 1 -- 172.21.15.12:0/1089934976 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b24011a20 con 0x7f4b34108570 2024-04-22T13:15:11.503 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.660+0000 7f4b39a6e700 1 -- 172.21.15.12:0/1089934976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b34108570 msgr2=0x7f4b34108990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:11.503 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.660+0000 7f4b39a6e700 1 --2- 172.21.15.12:0/1089934976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b34108570 0x7f4b34108990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4b240098a0 tx=0x7f4b24009bb0 comp rx=0 tx=0).stop 2024-04-22T13:15:11.504 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.660+0000 7f4b39a6e700 1 -- 172.21.15.12:0/1089934976 shutdown_connections 2024-04-22T13:15:11.504 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.660+0000 7f4b39a6e700 1 --2- 172.21.15.12:0/1089934976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b34108570 0x7f4b34108990 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:11.504 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.660+0000 7f4b39a6e700 1 -- 172.21.15.12:0/1089934976 >> 172.21.15.12:0/1089934976 conn(0x7f4b34103b70 msgr2=0x7f4b34105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:11.504 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.661+0000 7f4b39a6e700 1 -- 172.21.15.12:0/1089934976 shutdown_connections 2024-04-22T13:15:11.504 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.661+0000 7f4b39a6e700 1 -- 172.21.15.12:0/1089934976 wait complete. 2024-04-22T13:15:11.504 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.661+0000 7f4b39a6e700 1 Processor -- start 2024-04-22T13:15:11.504 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.661+0000 7f4b39a6e700 1 -- start start 2024-04-22T13:15:11.504 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.662+0000 7f4b39a6e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b34108570 0x7f4b3419c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:11.504 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.662+0000 7f4b39a6e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3410cad0 con 0x7f4b34108570 2024-04-22T13:15:11.504 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.662+0000 7f4b337fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b34108570 0x7f4b3419c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:11.504 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.662+0000 7f4b337fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b34108570 0x7f4b3419c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42974/0 (socket says 172.21.15.12:42974) 2024-04-22T13:15:11.504 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.662+0000 7f4b337fe700 1 -- 172.21.15.12:0/2434476149 learned_addr learned my addr 172.21.15.12:0/2434476149 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:15:11.504 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.662+0000 7f4b337fe700 1 -- 172.21.15.12:0/2434476149 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b3419c9b0 con 0x7f4b34108570 2024-04-22T13:15:11.504 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.662+0000 7f4b337fe700 1 --2- 172.21.15.12:0/2434476149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b34108570 0x7f4b3419c470 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4b24000c00 tx=0x7f4b24011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:11.505 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.662+0000 7f4b30ff9700 1 -- 172.21.15.12:0/2434476149 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b24003aa0 con 0x7f4b34108570 2024-04-22T13:15:11.505 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.662+0000 7f4b30ff9700 1 -- 172.21.15.12:0/2434476149 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b2401b8f0 con 0x7f4b34108570 2024-04-22T13:15:11.505 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.662+0000 7f4b30ff9700 1 -- 172.21.15.12:0/2434476149 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b24023890 con 0x7f4b34108570 2024-04-22T13:15:11.505 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.662+0000 7f4b39a6e700 1 -- 172.21.15.12:0/2434476149 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b3419cc40 con 0x7f4b34108570 2024-04-22T13:15:11.505 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.662+0000 7f4b39a6e700 1 -- 172.21.15.12:0/2434476149 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b3419f930 con 0x7f4b34108570 2024-04-22T13:15:11.505 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.663+0000 7f4b30ff9700 1 -- 172.21.15.12:0/2434476149 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f4b2402c3f0 con 0x7f4b34108570 2024-04-22T13:15:11.505 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.663+0000 7f4b30ff9700 1 --2- 172.21.15.12:0/2434476149 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f4b1c039600 0x7f4b1c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:11.505 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.663+0000 7f4b30ff9700 1 -- 172.21.15.12:0/2434476149 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4b2404dfe0 con 0x7f4b34108570 2024-04-22T13:15:11.505 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.663+0000 7f4b39a6e700 1 -- 172.21.15.12:0/2434476149 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b340640a0 con 0x7f4b34108570 2024-04-22T13:15:11.505 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.664+0000 7f4b32ffd700 1 --2- 172.21.15.12:0/2434476149 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f4b1c039600 0x7f4b1c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:11.505 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.664+0000 7f4b32ffd700 1 --2- 172.21.15.12:0/2434476149 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f4b1c039600 0x7f4b1c03bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4b28006fd0 tx=0x7f4b28006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:11.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.669+0000 7f4b30ff9700 1 -- 172.21.15.12:0/2434476149 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b2401f030 con 0x7f4b34108570 2024-04-22T13:15:11.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.877+0000 7f4b39a6e700 1 -- 172.21.15.12:0/2434476149 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f4b3419fb00 con 0x7f4b34108570 2024-04-22T13:15:11.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.878+0000 7f4b30ff9700 1 -- 172.21.15.12:0/2434476149 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f4b2401bda0 con 0x7f4b34108570 2024-04-22T13:15:11.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.880+0000 7f4b39a6e700 1 -- 172.21.15.12:0/2434476149 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f4b1c039600 msgr2=0x7f4b1c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:11.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.880+0000 7f4b39a6e700 1 --2- 172.21.15.12:0/2434476149 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f4b1c039600 0x7f4b1c03bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4b28006fd0 tx=0x7f4b28006e40 comp rx=0 tx=0).stop 2024-04-22T13:15:11.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.880+0000 7f4b39a6e700 1 -- 172.21.15.12:0/2434476149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b34108570 msgr2=0x7f4b3419c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:11.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.880+0000 7f4b39a6e700 1 --2- 172.21.15.12:0/2434476149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b34108570 0x7f4b3419c470 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4b24000c00 tx=0x7f4b24011aa0 comp rx=0 tx=0).stop 2024-04-22T13:15:11.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.880+0000 7f4b39a6e700 1 -- 172.21.15.12:0/2434476149 shutdown_connections 2024-04-22T13:15:11.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.880+0000 7f4b39a6e700 1 --2- 172.21.15.12:0/2434476149 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f4b1c039600 0x7f4b1c03bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:11.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.880+0000 7f4b39a6e700 1 --2- 172.21.15.12:0/2434476149 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b34108570 0x7f4b3419c470 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:11.506 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.880+0000 7f4b39a6e700 1 -- 172.21.15.12:0/2434476149 >> 172.21.15.12:0/2434476149 conn(0x7f4b34103b70 msgr2=0x7f4b341046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:11.507 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.880+0000 7f4b39a6e700 1 -- 172.21.15.12:0/2434476149 shutdown_connections 2024-04-22T13:15:11.507 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:10.880+0000 7f4b39a6e700 1 -- 172.21.15.12:0/2434476149 wait complete. 2024-04-22T13:15:11.509 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-22T13:15:11.509 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout disabled 2024-04-22T13:15:11.517 INFO:teuthology.orchestra.run.smithi012.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-22T13:15:11.517 INFO:teuthology.orchestra.run.smithi012.stdout:systemctl: stdout inactive 2024-04-22T13:15:11.517 INFO:teuthology.orchestra.run.smithi012.stdout:firewalld.service is not enabled 2024-04-22T13:15:11.517 INFO:teuthology.orchestra.run.smithi012.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-22T13:15:11.519 INFO:teuthology.orchestra.run.smithi012.stdout:Ceph Dashboard is now available at: 2024-04-22T13:15:11.519 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:15:11.519 INFO:teuthology.orchestra.run.smithi012.stdout: URL: https://smithi012:8443/ 2024-04-22T13:15:11.519 INFO:teuthology.orchestra.run.smithi012.stdout: User: admin 2024-04-22T13:15:11.519 INFO:teuthology.orchestra.run.smithi012.stdout: Password: 1zpwc1dl37 2024-04-22T13:15:11.519 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:15:11.520 INFO:teuthology.orchestra.run.smithi012.stdout:Saving cluster configuration to /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config directory 2024-04-22T13:15:11.520 INFO:teuthology.orchestra.run.smithi012.stdout:Enabling autotune for osd_memory_target 2024-04-22T13:15:12.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:12 smithi012 ceph-mon[28265]: mgrmap e13: x(active, since 6s) 2024-04-22T13:15:12.726 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.906+0000 7f3ee6c38700 1 Processor -- start 2024-04-22T13:15:12.726 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.907+0000 7f3ee6c38700 1 -- start start 2024-04-22T13:15:12.726 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.907+0000 7f3ee6c38700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ee007de40 0x7f3ee007c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:12.726 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.907+0000 7f3ee6c38700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee007c830 con 0x7f3ee007de40 2024-04-22T13:15:12.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.907+0000 7f3ee49d4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ee007de40 0x7f3ee007c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:12.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.907+0000 7f3ee49d4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ee007de40 0x7f3ee007c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39698/0 (socket says 172.21.15.12:39698) 2024-04-22T13:15:12.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.907+0000 7f3ee49d4700 1 -- 172.21.15.12:0/3514599720 learned_addr learned my addr 172.21.15.12:0/3514599720 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:15:12.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.907+0000 7f3ee49d4700 1 -- 172.21.15.12:0/3514599720 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee007c9b0 con 0x7f3ee007de40 2024-04-22T13:15:12.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.907+0000 7f3ee49d4700 1 --2- 172.21.15.12:0/3514599720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ee007de40 0x7f3ee007c2f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3ed40098a0 tx=0x7f3ed4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbbf97850fee4d57 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:12.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.908+0000 7f3edf7fe700 1 -- 172.21.15.12:0/3514599720 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ed4004160 con 0x7f3ee007de40 2024-04-22T13:15:12.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.908+0000 7f3edf7fe700 1 -- 172.21.15.12:0/3514599720 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ed4004300 con 0x7f3ee007de40 2024-04-22T13:15:12.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.908+0000 7f3edf7fe700 1 -- 172.21.15.12:0/3514599720 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ed4011a20 con 0x7f3ee007de40 2024-04-22T13:15:12.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.908+0000 7f3ee6c38700 1 -- 172.21.15.12:0/3514599720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ee007de40 msgr2=0x7f3ee007c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:12.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.908+0000 7f3ee6c38700 1 --2- 172.21.15.12:0/3514599720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ee007de40 0x7f3ee007c2f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3ed40098a0 tx=0x7f3ed4009bb0 comp rx=0 tx=0).stop 2024-04-22T13:15:12.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.908+0000 7f3ee6c38700 1 -- 172.21.15.12:0/3514599720 shutdown_connections 2024-04-22T13:15:12.727 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.908+0000 7f3ee6c38700 1 --2- 172.21.15.12:0/3514599720 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ee007de40 0x7f3ee007c2f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:12.728 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.908+0000 7f3ee6c38700 1 -- 172.21.15.12:0/3514599720 >> 172.21.15.12:0/3514599720 conn(0x7f3ee0103b40 msgr2=0x7f3ee0105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:12.728 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.908+0000 7f3ee6c38700 1 -- 172.21.15.12:0/3514599720 shutdown_connections 2024-04-22T13:15:12.728 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.908+0000 7f3ee6c38700 1 -- 172.21.15.12:0/3514599720 wait complete. 2024-04-22T13:15:12.728 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.909+0000 7f3ee6c38700 1 Processor -- start 2024-04-22T13:15:12.728 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.909+0000 7f3ee6c38700 1 -- start start 2024-04-22T13:15:12.728 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.909+0000 7f3ee6c38700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ee007de40 0x7f3ee019e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:12.728 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.909+0000 7f3ee6c38700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee010eac0 con 0x7f3ee007de40 2024-04-22T13:15:12.728 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.909+0000 7f3ee49d4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ee007de40 0x7f3ee019e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:12.728 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.909+0000 7f3ee49d4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ee007de40 0x7f3ee019e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39704/0 (socket says 172.21.15.12:39704) 2024-04-22T13:15:12.729 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.909+0000 7f3ee49d4700 1 -- 172.21.15.12:0/2938721841 learned_addr learned my addr 172.21.15.12:0/2938721841 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:15:12.729 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.909+0000 7f3ee49d4700 1 -- 172.21.15.12:0/2938721841 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee019ebb0 con 0x7f3ee007de40 2024-04-22T13:15:12.729 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.910+0000 7f3ee49d4700 1 --2- 172.21.15.12:0/2938721841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ee007de40 0x7f3ee019e670 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3ed40116e0 tx=0x7f3ed4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:12.729 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.910+0000 7f3eddffb700 1 -- 172.21.15.12:0/2938721841 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ed4003b20 con 0x7f3ee007de40 2024-04-22T13:15:12.729 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.910+0000 7f3eddffb700 1 -- 172.21.15.12:0/2938721841 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ed401b8f0 con 0x7f3ee007de40 2024-04-22T13:15:12.729 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.910+0000 7f3ee6c38700 1 -- 172.21.15.12:0/2938721841 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ee019ee40 con 0x7f3ee007de40 2024-04-22T13:15:12.729 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.910+0000 7f3eddffb700 1 -- 172.21.15.12:0/2938721841 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ed40238c0 con 0x7f3ee007de40 2024-04-22T13:15:12.729 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.910+0000 7f3ee6c38700 1 -- 172.21.15.12:0/2938721841 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ee01a1b30 con 0x7f3ee007de40 2024-04-22T13:15:12.729 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.911+0000 7f3eddffb700 1 -- 172.21.15.12:0/2938721841 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3ed401baa0 con 0x7f3ee007de40 2024-04-22T13:15:12.729 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.911+0000 7f3eddffb700 1 --2- 172.21.15.12:0/2938721841 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f3ed0039650 0x7f3ed003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:12.729 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.911+0000 7f3ee6c38700 1 -- 172.21.15.12:0/2938721841 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ee00640a0 con 0x7f3ee007de40 2024-04-22T13:15:12.729 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.911+0000 7f3eddffb700 1 -- 172.21.15.12:0/2938721841 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3ed404e250 con 0x7f3ee007de40 2024-04-22T13:15:12.729 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.911+0000 7f3edffff700 1 --2- 172.21.15.12:0/2938721841 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f3ed0039650 0x7f3ed003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:12.730 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.912+0000 7f3edffff700 1 --2- 172.21.15.12:0/2938721841 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f3ed0039650 0x7f3ed003bb10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc006fd0 tx=0x7f3ecc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:12.730 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:11.916+0000 7f3eddffb700 1 -- 172.21.15.12:0/2938721841 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ed401f030 con 0x7f3ee007de40 2024-04-22T13:15:12.730 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:12.124+0000 7f3ee6c38700 1 -- 172.21.15.12:0/2938721841 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f3ee01a2120 con 0x7f3ee007de40 2024-04-22T13:15:12.730 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:12.124+0000 7f3eddffb700 1 -- 172.21.15.12:0/2938721841 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f3ed402bb90 con 0x7f3ee007de40 2024-04-22T13:15:12.730 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:12.126+0000 7f3ee6c38700 1 -- 172.21.15.12:0/2938721841 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f3ed0039650 msgr2=0x7f3ed003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:12.730 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:12.126+0000 7f3ee6c38700 1 --2- 172.21.15.12:0/2938721841 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f3ed0039650 0x7f3ed003bb10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc006fd0 tx=0x7f3ecc006e40 comp rx=0 tx=0).stop 2024-04-22T13:15:12.730 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:12.126+0000 7f3ee6c38700 1 -- 172.21.15.12:0/2938721841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ee007de40 msgr2=0x7f3ee019e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:12.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:12.126+0000 7f3ee6c38700 1 --2- 172.21.15.12:0/2938721841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ee007de40 0x7f3ee019e670 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3ed40116e0 tx=0x7f3ed4011aa0 comp rx=0 tx=0).stop 2024-04-22T13:15:12.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:12.126+0000 7f3ee6c38700 1 -- 172.21.15.12:0/2938721841 shutdown_connections 2024-04-22T13:15:12.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:12.126+0000 7f3ee6c38700 1 --2- 172.21.15.12:0/2938721841 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f3ed0039650 0x7f3ed003bb10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:12.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:12.126+0000 7f3ee6c38700 1 --2- 172.21.15.12:0/2938721841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ee007de40 0x7f3ee019e670 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:12.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:12.126+0000 7f3ee6c38700 1 -- 172.21.15.12:0/2938721841 >> 172.21.15.12:0/2938721841 conn(0x7f3ee0103b40 msgr2=0x7f3ee0104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:12.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:12.126+0000 7f3ee6c38700 1 -- 172.21.15.12:0/2938721841 shutdown_connections 2024-04-22T13:15:12.732 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:12.126+0000 7f3ee6c38700 1 -- 172.21.15.12:0/2938721841 wait complete. 2024-04-22T13:15:14.023 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.160+0000 7fb023098700 1 Processor -- start 2024-04-22T13:15:14.024 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.160+0000 7fb023098700 1 -- start start 2024-04-22T13:15:14.024 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.161+0000 7fb023098700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb01c107440 0x7fb01c107860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:14.024 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.161+0000 7fb023098700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c107e30 con 0x7fb01c107440 2024-04-22T13:15:14.024 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.161+0000 7fb020e34700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb01c107440 0x7fb01c107860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:14.024 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.161+0000 7fb020e34700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb01c107440 0x7fb01c107860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39710/0 (socket says 172.21.15.12:39710) 2024-04-22T13:15:14.024 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.161+0000 7fb020e34700 1 -- 172.21.15.12:0/738984446 learned_addr learned my addr 172.21.15.12:0/738984446 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:15:14.024 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.161+0000 7fb020e34700 1 -- 172.21.15.12:0/738984446 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01c108640 con 0x7fb01c107440 2024-04-22T13:15:14.024 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.161+0000 7fb020e34700 1 --2- 172.21.15.12:0/738984446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb01c107440 0x7fb01c107860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0098a0 tx=0x7fb00c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb9b37ab2894888 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:14.024 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.161+0000 7fb01b7fe700 1 -- 172.21.15.12:0/738984446 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb00c004160 con 0x7fb01c107440 2024-04-22T13:15:14.024 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.161+0000 7fb01b7fe700 1 -- 172.21.15.12:0/738984446 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb00c004300 con 0x7fb01c107440 2024-04-22T13:15:14.025 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.161+0000 7fb01b7fe700 1 -- 172.21.15.12:0/738984446 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb00c011a20 con 0x7fb01c107440 2024-04-22T13:15:14.025 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.162+0000 7fb023098700 1 -- 172.21.15.12:0/738984446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb01c107440 msgr2=0x7fb01c107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:14.025 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.162+0000 7fb023098700 1 --2- 172.21.15.12:0/738984446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb01c107440 0x7fb01c107860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0098a0 tx=0x7fb00c009bb0 comp rx=0 tx=0).stop 2024-04-22T13:15:14.025 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.162+0000 7fb023098700 1 -- 172.21.15.12:0/738984446 shutdown_connections 2024-04-22T13:15:14.025 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.162+0000 7fb023098700 1 --2- 172.21.15.12:0/738984446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb01c107440 0x7fb01c107860 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:14.025 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.162+0000 7fb023098700 1 -- 172.21.15.12:0/738984446 >> 172.21.15.12:0/738984446 conn(0x7fb01c102a40 msgr2=0x7fb01c104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:14.025 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.162+0000 7fb023098700 1 -- 172.21.15.12:0/738984446 shutdown_connections 2024-04-22T13:15:14.025 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.162+0000 7fb023098700 1 -- 172.21.15.12:0/738984446 wait complete. 2024-04-22T13:15:14.025 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.163+0000 7fb023098700 1 Processor -- start 2024-04-22T13:15:14.025 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.163+0000 7fb023098700 1 -- start start 2024-04-22T13:15:14.025 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.163+0000 7fb023098700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb01c107440 0x7fb01c19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:14.025 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.163+0000 7fb023098700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c10b9f0 con 0x7fb01c107440 2024-04-22T13:15:14.025 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.163+0000 7fb020e34700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb01c107440 0x7fb01c19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:14.026 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.163+0000 7fb020e34700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb01c107440 0x7fb01c19c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39716/0 (socket says 172.21.15.12:39716) 2024-04-22T13:15:14.026 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.163+0000 7fb020e34700 1 -- 172.21.15.12:0/1945056453 learned_addr learned my addr 172.21.15.12:0/1945056453 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:15:14.026 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.163+0000 7fb020e34700 1 -- 172.21.15.12:0/1945056453 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01c19c990 con 0x7fb01c107440 2024-04-22T13:15:14.026 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.164+0000 7fb020e34700 1 --2- 172.21.15.12:0/1945056453 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb01c107440 0x7fb01c19c450 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb00c009520 tx=0x7fb00c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:14.026 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.164+0000 7fb019ffb700 1 -- 172.21.15.12:0/1945056453 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb00c0039e0 con 0x7fb01c107440 2024-04-22T13:15:14.026 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.164+0000 7fb019ffb700 1 -- 172.21.15.12:0/1945056453 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb00c01b8f0 con 0x7fb01c107440 2024-04-22T13:15:14.026 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.164+0000 7fb023098700 1 -- 172.21.15.12:0/1945056453 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb01c19cc20 con 0x7fb01c107440 2024-04-22T13:15:14.026 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.164+0000 7fb019ffb700 1 -- 172.21.15.12:0/1945056453 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb00c019770 con 0x7fb01c107440 2024-04-22T13:15:14.026 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.164+0000 7fb023098700 1 -- 172.21.15.12:0/1945056453 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb01c19f8c0 con 0x7fb01c107440 2024-04-22T13:15:14.026 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.165+0000 7fb019ffb700 1 -- 172.21.15.12:0/1945056453 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fb00c019a20 con 0x7fb01c107440 2024-04-22T13:15:14.027 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.165+0000 7fb023098700 1 -- 172.21.15.12:0/1945056453 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb01c0640a0 con 0x7fb01c107440 2024-04-22T13:15:14.027 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.165+0000 7fb019ffb700 1 --2- 172.21.15.12:0/1945056453 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb0040396f0 0x7fb00403bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:14.027 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.165+0000 7fb019ffb700 1 -- 172.21.15.12:0/1945056453 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb00c04e270 con 0x7fb01c107440 2024-04-22T13:15:14.027 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.165+0000 7fb01bfff700 1 --2- 172.21.15.12:0/1945056453 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb0040396f0 0x7fb00403bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:14.027 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.166+0000 7fb01bfff700 1 --2- 172.21.15.12:0/1945056453 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb0040396f0 0x7fb00403bbb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb010006fd0 tx=0x7fb010006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:14.027 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.171+0000 7fb019ffb700 1 -- 172.21.15.12:0/1945056453 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb00c01f030 con 0x7fb01c107440 2024-04-22T13:15:14.028 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.490+0000 7fb023098700 1 -- 172.21.15.12:0/1945056453 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fb01c02e250 con 0x7fb01c107440 2024-04-22T13:15:14.028 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.493+0000 7fb019ffb700 1 -- 172.21.15.12:0/1945056453 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fb00c00ba30 con 0x7fb01c107440 2024-04-22T13:15:14.028 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.495+0000 7fb023098700 1 -- 172.21.15.12:0/1945056453 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb0040396f0 msgr2=0x7fb00403bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:14.028 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.495+0000 7fb023098700 1 --2- 172.21.15.12:0/1945056453 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb0040396f0 0x7fb00403bbb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb010006fd0 tx=0x7fb010006e40 comp rx=0 tx=0).stop 2024-04-22T13:15:14.028 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.495+0000 7fb023098700 1 -- 172.21.15.12:0/1945056453 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb01c107440 msgr2=0x7fb01c19c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:14.028 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.495+0000 7fb023098700 1 --2- 172.21.15.12:0/1945056453 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb01c107440 0x7fb01c19c450 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb00c009520 tx=0x7fb00c011aa0 comp rx=0 tx=0).stop 2024-04-22T13:15:14.028 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.495+0000 7fb023098700 1 -- 172.21.15.12:0/1945056453 shutdown_connections 2024-04-22T13:15:14.028 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.495+0000 7fb023098700 1 --2- 172.21.15.12:0/1945056453 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb0040396f0 0x7fb00403bbb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:14.028 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.495+0000 7fb023098700 1 --2- 172.21.15.12:0/1945056453 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb01c107440 0x7fb01c19c450 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:14.028 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.495+0000 7fb023098700 1 -- 172.21.15.12:0/1945056453 >> 172.21.15.12:0/1945056453 conn(0x7fb01c102a40 msgr2=0x7fb01c103880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:14.028 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.495+0000 7fb023098700 1 -- 172.21.15.12:0/1945056453 shutdown_connections 2024-04-22T13:15:14.028 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr 2024-04-22T13:15:13.495+0000 7fb023098700 1 -- 172.21.15.12:0/1945056453 wait complete. 2024-04-22T13:15:14.029 INFO:teuthology.orchestra.run.smithi012.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-22T13:15:14.029 INFO:teuthology.orchestra.run.smithi012.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-22T13:15:14.029 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:15:14.029 INFO:teuthology.orchestra.run.smithi012.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-22T13:15:14.029 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:15:14.029 INFO:teuthology.orchestra.run.smithi012.stdout:Or, if you are only running a single cluster on this host: 2024-04-22T13:15:14.029 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:15:14.029 INFO:teuthology.orchestra.run.smithi012.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-22T13:15:14.029 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:15:14.029 INFO:teuthology.orchestra.run.smithi012.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-22T13:15:14.029 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:15:14.030 INFO:teuthology.orchestra.run.smithi012.stdout: ceph telemetry on 2024-04-22T13:15:14.030 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:15:14.030 INFO:teuthology.orchestra.run.smithi012.stdout:For more information see: 2024-04-22T13:15:14.030 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:15:14.030 INFO:teuthology.orchestra.run.smithi012.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-22T13:15:14.030 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:15:14.030 INFO:teuthology.orchestra.run.smithi012.stdout:Bootstrap complete. 2024-04-22T13:15:14.060 INFO:tasks.cephadm:Fetching config... 2024-04-22T13:15:14.061 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-22T13:15:14.061 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-22T13:15:14.077 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-22T13:15:14.077 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-22T13:15:14.077 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-22T13:15:14.133 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-22T13:15:14.133 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-22T13:15:14.133 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-22T13:15:14.198 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-22T13:15:14.198 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-22T13:15:14.198 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-22T13:15:14.254 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-22T13:15:14.254 DEBUG:teuthology.orchestra.run.smithi012:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQLZu8FDpJVceSCKOLD/XtW0gKG+KKFHIpsZPld7ZT1NQs4U9PUXQ+91wgyjHvmBzPksEks09W/u8ddq4iy/ponZEvcCoIg3Cpoo0RG/o45Pu772m1KCXwXDkYWyWum7IwLy5gBQ6cqy8TvpT+u1ETzKdpNZPeMvHqg8UNdEjVKT20+4DznKohEw5r1NiwRl9jhrQRUZIOZILuooql79K2LELSUDaxU0bGG5NUNTiGGgfjF/y7xjgmsSmu6T3y8v3+ESwD7D9u0SioZhOW95mkYQ1iRGhcmVzzOzSqduzmjzFgygo6AixsH4bUimWX0+ZdXs7UreU7HKZ+4+h2tx+6AZ/qwhCpoO0zhwZd0NkQAG45yv79yxvZMqYeRY9yyvyJkqt/Kwibli80nTcZ4DmwsAZzeMLZuG+qmu/zdOYhqFU1Qrb/V5Ee2pu0LDYyTek0PfWR3SOijN5rZrglCv1wPAquvFkt/qeCNgSeVSJ9A6Z6tat8JgyEioCURLXTXgE= ceph-0aede5ea-00aa-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-22T13:15:14.330 INFO:teuthology.orchestra.run.smithi012.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQLZu8FDpJVceSCKOLD/XtW0gKG+KKFHIpsZPld7ZT1NQs4U9PUXQ+91wgyjHvmBzPksEks09W/u8ddq4iy/ponZEvcCoIg3Cpoo0RG/o45Pu772m1KCXwXDkYWyWum7IwLy5gBQ6cqy8TvpT+u1ETzKdpNZPeMvHqg8UNdEjVKT20+4DznKohEw5r1NiwRl9jhrQRUZIOZILuooql79K2LELSUDaxU0bGG5NUNTiGGgfjF/y7xjgmsSmu6T3y8v3+ESwD7D9u0SioZhOW95mkYQ1iRGhcmVzzOzSqduzmjzFgygo6AixsH4bUimWX0+ZdXs7UreU7HKZ+4+h2tx+6AZ/qwhCpoO0zhwZd0NkQAG45yv79yxvZMqYeRY9yyvyJkqt/Kwibli80nTcZ4DmwsAZzeMLZuG+qmu/zdOYhqFU1Qrb/V5Ee2pu0LDYyTek0PfWR3SOijN5rZrglCv1wPAquvFkt/qeCNgSeVSJ9A6Z6tat8JgyEioCURLXTXgE= ceph-0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:15:14.342 DEBUG:teuthology.orchestra.run.smithi111:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQLZu8FDpJVceSCKOLD/XtW0gKG+KKFHIpsZPld7ZT1NQs4U9PUXQ+91wgyjHvmBzPksEks09W/u8ddq4iy/ponZEvcCoIg3Cpoo0RG/o45Pu772m1KCXwXDkYWyWum7IwLy5gBQ6cqy8TvpT+u1ETzKdpNZPeMvHqg8UNdEjVKT20+4DznKohEw5r1NiwRl9jhrQRUZIOZILuooql79K2LELSUDaxU0bGG5NUNTiGGgfjF/y7xjgmsSmu6T3y8v3+ESwD7D9u0SioZhOW95mkYQ1iRGhcmVzzOzSqduzmjzFgygo6AixsH4bUimWX0+ZdXs7UreU7HKZ+4+h2tx+6AZ/qwhCpoO0zhwZd0NkQAG45yv79yxvZMqYeRY9yyvyJkqt/Kwibli80nTcZ4DmwsAZzeMLZuG+qmu/zdOYhqFU1Qrb/V5Ee2pu0LDYyTek0PfWR3SOijN5rZrglCv1wPAquvFkt/qeCNgSeVSJ9A6Z6tat8JgyEioCURLXTXgE= ceph-0aede5ea-00aa-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-22T13:15:14.382 INFO:teuthology.orchestra.run.smithi111.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQLZu8FDpJVceSCKOLD/XtW0gKG+KKFHIpsZPld7ZT1NQs4U9PUXQ+91wgyjHvmBzPksEks09W/u8ddq4iy/ponZEvcCoIg3Cpoo0RG/o45Pu772m1KCXwXDkYWyWum7IwLy5gBQ6cqy8TvpT+u1ETzKdpNZPeMvHqg8UNdEjVKT20+4DznKohEw5r1NiwRl9jhrQRUZIOZILuooql79K2LELSUDaxU0bGG5NUNTiGGgfjF/y7xjgmsSmu6T3y8v3+ESwD7D9u0SioZhOW95mkYQ1iRGhcmVzzOzSqduzmjzFgygo6AixsH4bUimWX0+ZdXs7UreU7HKZ+4+h2tx+6AZ/qwhCpoO0zhwZd0NkQAG45yv79yxvZMqYeRY9yyvyJkqt/Kwibli80nTcZ4DmwsAZzeMLZuG+qmu/zdOYhqFU1Qrb/V5Ee2pu0LDYyTek0PfWR3SOijN5rZrglCv1wPAquvFkt/qeCNgSeVSJ9A6Z6tat8JgyEioCURLXTXgE= ceph-0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:15:14.393 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQLZu8FDpJVceSCKOLD/XtW0gKG+KKFHIpsZPld7ZT1NQs4U9PUXQ+91wgyjHvmBzPksEks09W/u8ddq4iy/ponZEvcCoIg3Cpoo0RG/o45Pu772m1KCXwXDkYWyWum7IwLy5gBQ6cqy8TvpT+u1ETzKdpNZPeMvHqg8UNdEjVKT20+4DznKohEw5r1NiwRl9jhrQRUZIOZILuooql79K2LELSUDaxU0bGG5NUNTiGGgfjF/y7xjgmsSmu6T3y8v3+ESwD7D9u0SioZhOW95mkYQ1iRGhcmVzzOzSqduzmjzFgygo6AixsH4bUimWX0+ZdXs7UreU7HKZ+4+h2tx+6AZ/qwhCpoO0zhwZd0NkQAG45yv79yxvZMqYeRY9yyvyJkqt/Kwibli80nTcZ4DmwsAZzeMLZuG+qmu/zdOYhqFU1Qrb/V5Ee2pu0LDYyTek0PfWR3SOijN5rZrglCv1wPAquvFkt/qeCNgSeVSJ9A6Z6tat8JgyEioCURLXTXgE= ceph-0aede5ea-00aa-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-22T13:15:14.432 INFO:teuthology.orchestra.run.smithi134.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQLZu8FDpJVceSCKOLD/XtW0gKG+KKFHIpsZPld7ZT1NQs4U9PUXQ+91wgyjHvmBzPksEks09W/u8ddq4iy/ponZEvcCoIg3Cpoo0RG/o45Pu772m1KCXwXDkYWyWum7IwLy5gBQ6cqy8TvpT+u1ETzKdpNZPeMvHqg8UNdEjVKT20+4DznKohEw5r1NiwRl9jhrQRUZIOZILuooql79K2LELSUDaxU0bGG5NUNTiGGgfjF/y7xjgmsSmu6T3y8v3+ESwD7D9u0SioZhOW95mkYQ1iRGhcmVzzOzSqduzmjzFgygo6AixsH4bUimWX0+ZdXs7UreU7HKZ+4+h2tx+6AZ/qwhCpoO0zhwZd0NkQAG45yv79yxvZMqYeRY9yyvyJkqt/Kwibli80nTcZ4DmwsAZzeMLZuG+qmu/zdOYhqFU1Qrb/V5Ee2pu0LDYyTek0PfWR3SOijN5rZrglCv1wPAquvFkt/qeCNgSeVSJ9A6Z6tat8JgyEioCURLXTXgE= ceph-0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:15:14.444 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-22T13:15:14.683 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:15:14.805 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:14 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1945056453' entity='client.admin' 2024-04-22T13:15:17.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.353+0000 7f840b428700 1 -- 172.21.15.12:0/971036689 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84041024d0 msgr2=0x7f84041028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:17.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.353+0000 7f840b428700 1 --2- 172.21.15.12:0/971036689 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84041024d0 0x7f84041028f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8400009910 tx=0x7f8400009c20 comp rx=0 tx=0).stop 2024-04-22T13:15:17.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.353+0000 7f840b428700 1 -- 172.21.15.12:0/971036689 shutdown_connections 2024-04-22T13:15:17.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.353+0000 7f840b428700 1 --2- 172.21.15.12:0/971036689 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84041024d0 0x7f84041028f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:17.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.353+0000 7f840b428700 1 -- 172.21.15.12:0/971036689 >> 172.21.15.12:0/971036689 conn(0x7f84040fda90 msgr2=0x7f84040ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:17.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.353+0000 7f840b428700 1 -- 172.21.15.12:0/971036689 shutdown_connections 2024-04-22T13:15:17.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.353+0000 7f840b428700 1 -- 172.21.15.12:0/971036689 wait complete. 2024-04-22T13:15:17.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.354+0000 7f840b428700 1 Processor -- start 2024-04-22T13:15:17.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.354+0000 7f840b428700 1 -- start start 2024-04-22T13:15:17.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.354+0000 7f840b428700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84041024d0 0x7f8404199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:17.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.354+0000 7f840b428700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8404108410 con 0x7f84041024d0 2024-04-22T13:15:17.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.354+0000 7f84091c4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84041024d0 0x7f8404199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:17.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.354+0000 7f84091c4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84041024d0 0x7f8404199ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39732/0 (socket says 172.21.15.12:39732) 2024-04-22T13:15:17.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.354+0000 7f84091c4700 1 -- 172.21.15.12:0/4173386937 learned_addr learned my addr 172.21.15.12:0/4173386937 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:15:17.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.354+0000 7f84091c4700 1 -- 172.21.15.12:0/4173386937 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f840419a220 con 0x7f84041024d0 2024-04-22T13:15:17.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.354+0000 7f84091c4700 1 --2- 172.21.15.12:0/4173386937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84041024d0 0x7f8404199ce0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8400005800 tx=0x7f8400009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:17.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.355+0000 7f83fa7fc700 1 -- 172.21.15.12:0/4173386937 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8400027070 con 0x7f84041024d0 2024-04-22T13:15:17.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.355+0000 7f83fa7fc700 1 -- 172.21.15.12:0/4173386937 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f840000be10 con 0x7f84041024d0 2024-04-22T13:15:17.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.355+0000 7f83fa7fc700 1 -- 172.21.15.12:0/4173386937 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f840000f460 con 0x7f84041024d0 2024-04-22T13:15:17.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.355+0000 7f840b428700 1 -- 172.21.15.12:0/4173386937 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f840419a4b0 con 0x7f84041024d0 2024-04-22T13:15:17.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.355+0000 7f840b428700 1 -- 172.21.15.12:0/4173386937 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f840419d190 con 0x7f84041024d0 2024-04-22T13:15:17.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.356+0000 7f83fa7fc700 1 -- 172.21.15.12:0/4173386937 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f840002a070 con 0x7f84041024d0 2024-04-22T13:15:17.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.356+0000 7f840b428700 1 -- 172.21.15.12:0/4173386937 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8404108b90 con 0x7f84041024d0 2024-04-22T13:15:17.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.356+0000 7f83fa7fc700 1 --2- 172.21.15.12:0/4173386937 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f83f00396b0 0x7f83f003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:17.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.356+0000 7f83fa7fc700 1 -- 172.21.15.12:0/4173386937 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f840001e030 con 0x7f84041024d0 2024-04-22T13:15:17.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.356+0000 7f84089c3700 1 --2- 172.21.15.12:0/4173386937 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f83f00396b0 0x7f83f003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:17.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.357+0000 7f84089c3700 1 --2- 172.21.15.12:0/4173386937 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f83f00396b0 0x7f83f003bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f83f4006fd0 tx=0x7f83f4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:17.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.361+0000 7f83fa7fc700 1 -- 172.21.15.12:0/4173386937 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8400013070 con 0x7f84041024d0 2024-04-22T13:15:17.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.571+0000 7f840b428700 1 -- 172.21.15.12:0/4173386937 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f840402e090 con 0x7f84041024d0 2024-04-22T13:15:17.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.573+0000 7f83fa7fc700 1 -- 172.21.15.12:0/4173386937 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f8400031020 con 0x7f84041024d0 2024-04-22T13:15:17.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.576+0000 7f840b428700 1 -- 172.21.15.12:0/4173386937 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f83f00396b0 msgr2=0x7f83f003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:17.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.576+0000 7f840b428700 1 --2- 172.21.15.12:0/4173386937 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f83f00396b0 0x7f83f003bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f83f4006fd0 tx=0x7f83f4006e40 comp rx=0 tx=0).stop 2024-04-22T13:15:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.576+0000 7f840b428700 1 -- 172.21.15.12:0/4173386937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84041024d0 msgr2=0x7f8404199ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.576+0000 7f840b428700 1 --2- 172.21.15.12:0/4173386937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84041024d0 0x7f8404199ce0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8400005800 tx=0x7f8400009f60 comp rx=0 tx=0).stop 2024-04-22T13:15:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.576+0000 7f840b428700 1 -- 172.21.15.12:0/4173386937 shutdown_connections 2024-04-22T13:15:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.576+0000 7f840b428700 1 --2- 172.21.15.12:0/4173386937 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f83f00396b0 0x7f83f003bb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.576+0000 7f840b428700 1 --2- 172.21.15.12:0/4173386937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84041024d0 0x7f8404199ce0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.576+0000 7f840b428700 1 -- 172.21.15.12:0/4173386937 >> 172.21.15.12:0/4173386937 conn(0x7f84040fda90 msgr2=0x7f84040ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.576+0000 7f840b428700 1 -- 172.21.15.12:0/4173386937 shutdown_connections 2024-04-22T13:15:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:17.576+0000 7f840b428700 1 -- 172.21.15.12:0/4173386937 wait complete. 2024-04-22T13:15:18.725 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:18 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/4173386937' entity='client.admin' 2024-04-22T13:15:18.726 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:18 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:18.726 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:18 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:18.726 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:18 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi012", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:15:18.726 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:18 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:18.726 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:18 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:15:18.726 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:18 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:18.760 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-22T13:15:18.760 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-22T13:15:19.007 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:15:20.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.529+0000 7f5b3a34a700 1 -- 172.21.15.12:0/3574119682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b34102f80 msgr2=0x7f5b34105370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:20.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.529+0000 7f5b3a34a700 1 --2- 172.21.15.12:0/3574119682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b34102f80 0x7f5b34105370 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5b24009910 tx=0x7f5b24009c20 comp rx=0 tx=0).stop 2024-04-22T13:15:20.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.529+0000 7f5b3a34a700 1 -- 172.21.15.12:0/3574119682 shutdown_connections 2024-04-22T13:15:20.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.529+0000 7f5b3a34a700 1 --2- 172.21.15.12:0/3574119682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b34102f80 0x7f5b34105370 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:20.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.529+0000 7f5b3a34a700 1 -- 172.21.15.12:0/3574119682 >> 172.21.15.12:0/3574119682 conn(0x7f5b340fc950 msgr2=0x7f5b340fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:20.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.529+0000 7f5b3a34a700 1 -- 172.21.15.12:0/3574119682 shutdown_connections 2024-04-22T13:15:20.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.529+0000 7f5b3a34a700 1 -- 172.21.15.12:0/3574119682 wait complete. 2024-04-22T13:15:20.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.529+0000 7f5b3a34a700 1 Processor -- start 2024-04-22T13:15:20.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.530+0000 7f5b3a34a700 1 -- start start 2024-04-22T13:15:20.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.530+0000 7f5b3a34a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b34102f80 0x7f5b34197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:20.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.530+0000 7f5b3a34a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5b341083d0 con 0x7f5b34102f80 2024-04-22T13:15:20.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.530+0000 7f5b33fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b34102f80 0x7f5b34197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:20.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.530+0000 7f5b33fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b34102f80 0x7f5b34197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39738/0 (socket says 172.21.15.12:39738) 2024-04-22T13:15:20.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.530+0000 7f5b33fff700 1 -- 172.21.15.12:0/1319421972 learned_addr learned my addr 172.21.15.12:0/1319421972 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:15:20.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.531+0000 7f5b33fff700 1 -- 172.21.15.12:0/1319421972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b34197f80 con 0x7f5b34102f80 2024-04-22T13:15:20.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.531+0000 7f5b33fff700 1 --2- 172.21.15.12:0/1319421972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b34102f80 0x7f5b34197a40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5b24005b40 tx=0x7f5b24004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:20.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.531+0000 7f5b317fa700 1 -- 172.21.15.12:0/1319421972 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b24027070 con 0x7f5b34102f80 2024-04-22T13:15:20.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.531+0000 7f5b317fa700 1 -- 172.21.15.12:0/1319421972 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b2400be10 con 0x7f5b34102f80 2024-04-22T13:15:20.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.531+0000 7f5b3a34a700 1 -- 172.21.15.12:0/1319421972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b34198270 con 0x7f5b34102f80 2024-04-22T13:15:20.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.531+0000 7f5b317fa700 1 -- 172.21.15.12:0/1319421972 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b24022de0 con 0x7f5b34102f80 2024-04-22T13:15:20.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.531+0000 7f5b3a34a700 1 -- 172.21.15.12:0/1319421972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b3419af00 con 0x7f5b34102f80 2024-04-22T13:15:20.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.532+0000 7f5b317fa700 1 -- 172.21.15.12:0/1319421972 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5b2402a070 con 0x7f5b34102f80 2024-04-22T13:15:20.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.532+0000 7f5b317fa700 1 --2- 172.21.15.12:0/1319421972 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f5b1c039650 0x7f5b1c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:20.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.532+0000 7f5b317fa700 1 -- 172.21.15.12:0/1319421972 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5b2401e030 con 0x7f5b34102f80 2024-04-22T13:15:20.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.532+0000 7f5b337fe700 1 --2- 172.21.15.12:0/1319421972 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f5b1c039650 0x7f5b1c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:20.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.532+0000 7f5b3a34a700 1 -- 172.21.15.12:0/1319421972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b34108b50 con 0x7f5b34102f80 2024-04-22T13:15:20.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.533+0000 7f5b337fe700 1 --2- 172.21.15.12:0/1319421972 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f5b1c039650 0x7f5b1c03bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5b28006fd0 tx=0x7f5b28006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:20.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.538+0000 7f5b317fa700 1 -- 172.21.15.12:0/1319421972 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5b24031070 con 0x7f5b34102f80 2024-04-22T13:15:20.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.749+0000 7f5b3a34a700 1 -- 172.21.15.12:0/1319421972 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f5b3402e0b0 con 0x7f5b1c039650 2024-04-22T13:15:20.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.753+0000 7f5b317fa700 1 -- 172.21.15.12:0/1319421972 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5b3402e0b0 con 0x7f5b1c039650 2024-04-22T13:15:20.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.755+0000 7f5b3a34a700 1 -- 172.21.15.12:0/1319421972 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f5b1c039650 msgr2=0x7f5b1c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:20.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.755+0000 7f5b3a34a700 1 --2- 172.21.15.12:0/1319421972 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f5b1c039650 0x7f5b1c03bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5b28006fd0 tx=0x7f5b28006e40 comp rx=0 tx=0).stop 2024-04-22T13:15:20.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.755+0000 7f5b3a34a700 1 -- 172.21.15.12:0/1319421972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b34102f80 msgr2=0x7f5b34197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:20.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.755+0000 7f5b3a34a700 1 --2- 172.21.15.12:0/1319421972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b34102f80 0x7f5b34197a40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5b24005b40 tx=0x7f5b24004f80 comp rx=0 tx=0).stop 2024-04-22T13:15:20.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.755+0000 7f5b3a34a700 1 -- 172.21.15.12:0/1319421972 shutdown_connections 2024-04-22T13:15:20.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.755+0000 7f5b3a34a700 1 --2- 172.21.15.12:0/1319421972 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f5b1c039650 0x7f5b1c03bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:20.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.755+0000 7f5b3a34a700 1 --2- 172.21.15.12:0/1319421972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5b34102f80 0x7f5b34197a40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:20.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.755+0000 7f5b3a34a700 1 -- 172.21.15.12:0/1319421972 >> 172.21.15.12:0/1319421972 conn(0x7f5b340fc950 msgr2=0x7f5b340fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:20.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.755+0000 7f5b3a34a700 1 -- 172.21.15.12:0/1319421972 shutdown_connections 2024-04-22T13:15:20.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:20.755+0000 7f5b3a34a700 1 -- 172.21.15.12:0/1319421972 wait complete. 2024-04-22T13:15:21.528 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi111 2024-04-22T13:15:21.528 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-22T13:15:21.528 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/etc/ceph/ceph.conf 2024-04-22T13:15:21.546 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-22T13:15:21.547 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:15:21.605 INFO:tasks.cephadm:Adding host smithi111 to orchestrator... 2024-04-22T13:15:21.605 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph orch host add smithi111 2024-04-22T13:15:21.854 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:15:22.083 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:21 smithi012 ceph-mon[28265]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:15:22.083 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:21 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:22.083 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:21 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:15:22.083 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:21 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:15:22.083 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:21 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:15:22.083 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:21 smithi012 ceph-mon[28265]: Updating smithi012:/etc/ceph/ceph.conf 2024-04-22T13:15:22.083 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:21 smithi012 ceph-mon[28265]: Updating smithi012:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:15:22.083 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:21 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:22.083 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:21 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:22.083 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:21 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:22.805 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:22 smithi012 ceph-mon[28265]: Updating smithi012:/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:15:22.805 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:22 smithi012 ceph-mon[28265]: Updating smithi012:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T13:15:23.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:23.285+0000 7fc6e37e4700 1 -- 172.21.15.12:0/1936829530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6dc1015f0 msgr2=0x7fc6dc1019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:23.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:23.285+0000 7fc6e37e4700 1 --2- 172.21.15.12:0/1936829530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6dc1015f0 0x7fc6dc1019d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8009910 tx=0x7fc6d8009c20 comp rx=0 tx=0).stop 2024-04-22T13:15:23.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:23.285+0000 7fc6e37e4700 1 -- 172.21.15.12:0/1936829530 shutdown_connections 2024-04-22T13:15:23.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:23.285+0000 7fc6e37e4700 1 --2- 172.21.15.12:0/1936829530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6dc1015f0 0x7fc6dc1019d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:23.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:23.285+0000 7fc6e37e4700 1 -- 172.21.15.12:0/1936829530 >> 172.21.15.12:0/1936829530 conn(0x7fc6dc076ea0 msgr2=0x7fc6dc0772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:23.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:23.285+0000 7fc6e37e4700 1 -- 172.21.15.12:0/1936829530 shutdown_connections 2024-04-22T13:15:23.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:23.285+0000 7fc6e37e4700 1 -- 172.21.15.12:0/1936829530 wait complete. 2024-04-22T13:15:23.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:23.285+0000 7fc6e37e4700 1 Processor -- start 2024-04-22T13:15:23.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:23.286+0000 7fc6e37e4700 1 -- start start 2024-04-22T13:15:23.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:23.286+0000 7fc6e37e4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6dc198fa0 0x7fc6dc199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:23.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:23.286+0000 7fc6e37e4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc6dc10b820 con 0x7fc6dc198fa0 2024-04-22T13:15:23.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:23.286+0000 7fc6e1580700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6dc198fa0 0x7fc6dc199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:23.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:23.286+0000 7fc6e1580700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6dc198fa0 0x7fc6dc199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57668/0 (socket says 172.21.15.12:57668) 2024-04-22T13:15:23.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:23.286+0000 7fc6e1580700 1 -- 172.21.15.12:0/1542805815 learned_addr learned my addr 172.21.15.12:0/1542805815 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:15:23.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:23.286+0000 7fc6e1580700 1 -- 172.21.15.12:0/1542805815 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6dc1998c0 con 0x7fc6dc198fa0 2024-04-22T13:15:23.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:23.286+0000 7fc6e1580700 1 --2- 172.21.15.12:0/1542805815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6dc198fa0 0x7fc6dc199380 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800b5c0 tx=0x7fc6d8005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:23.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:23.286+0000 7fc6d27fc700 1 -- 172.21.15.12:0/1542805815 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6d8026070 con 0x7fc6dc198fa0 2024-04-22T13:15:23.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:23.286+0000 7fc6d27fc700 1 -- 172.21.15.12:0/1542805815 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6d800b6f0 con 0x7fc6dc198fa0 2024-04-22T13:15:23.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:23.287+0000 7fc6e37e4700 1 -- 172.21.15.12:0/1542805815 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6dc199bb0 con 0x7fc6dc198fa0 2024-04-22T13:15:23.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:23.287+0000 7fc6d27fc700 1 -- 172.21.15.12:0/1542805815 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6d8003810 con 0x7fc6dc198fa0 2024-04-22T13:15:23.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:23.287+0000 7fc6e37e4700 1 -- 172.21.15.12:0/1542805815 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6dc1a01d0 con 0x7fc6dc198fa0 2024-04-22T13:15:23.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:23.287+0000 7fc6d27fc700 1 -- 172.21.15.12:0/1542805815 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fc6d8029070 con 0x7fc6dc198fa0 2024-04-22T13:15:23.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:23.288+0000 7fc6d27fc700 1 --2- 172.21.15.12:0/1542805815 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc6c8039650 0x7fc6c803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:23.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:23.288+0000 7fc6d27fc700 1 -- 172.21.15.12:0/1542805815 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc6d801e070 con 0x7fc6dc198fa0 2024-04-22T13:15:23.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:23.288+0000 7fc6e37e4700 1 -- 172.21.15.12:0/1542805815 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6dc102a80 con 0x7fc6dc198fa0 2024-04-22T13:15:23.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:23.288+0000 7fc6e0d7f700 1 --2- 172.21.15.12:0/1542805815 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc6c8039650 0x7fc6c803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:23.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:23.288+0000 7fc6e0d7f700 1 --2- 172.21.15.12:0/1542805815 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc6c8039650 0x7fc6c803bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc006fd0 tx=0x7fc6cc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:23.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:23.293+0000 7fc6d27fc700 1 -- 172.21.15.12:0/1542805815 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc6d8030070 con 0x7fc6dc198fa0 2024-04-22T13:15:23.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:23.506+0000 7fc6e37e4700 1 -- 172.21.15.12:0/1542805815 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi111", "target": ["mon-mgr", ""]}) -- 0x7fc6dc0640a0 con 0x7fc6c8039650 2024-04-22T13:15:24.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:24 smithi012 ceph-mon[28265]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi111", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:15:24.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:24 smithi012 ceph-mon[28265]: Deploying cephadm binary to smithi111 2024-04-22T13:15:25.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:25 smithi012 ceph-mon[28265]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:15:27.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:27 smithi012 ceph-mon[28265]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:15:27.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:27.773+0000 7fc6d27fc700 1 -- 172.21.15.12:0/1542805815 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fc6dc0640a0 con 0x7fc6c8039650 2024-04-22T13:15:27.772 INFO:teuthology.orchestra.run.smithi012.stdout:Added host 'smithi111' with addr '172.21.15.111' 2024-04-22T13:15:27.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:27.776+0000 7fc6e37e4700 1 -- 172.21.15.12:0/1542805815 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc6c8039650 msgr2=0x7fc6c803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:27.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:27.776+0000 7fc6e37e4700 1 --2- 172.21.15.12:0/1542805815 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc6c8039650 0x7fc6c803bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc006fd0 tx=0x7fc6cc006e40 comp rx=0 tx=0).stop 2024-04-22T13:15:27.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:27.776+0000 7fc6e37e4700 1 -- 172.21.15.12:0/1542805815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6dc198fa0 msgr2=0x7fc6dc199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:27.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:27.776+0000 7fc6e37e4700 1 --2- 172.21.15.12:0/1542805815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6dc198fa0 0x7fc6dc199380 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800b5c0 tx=0x7fc6d8005780 comp rx=0 tx=0).stop 2024-04-22T13:15:27.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:27.776+0000 7fc6e37e4700 1 -- 172.21.15.12:0/1542805815 shutdown_connections 2024-04-22T13:15:27.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:27.776+0000 7fc6e37e4700 1 --2- 172.21.15.12:0/1542805815 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc6c8039650 0x7fc6c803bb10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:27.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:27.776+0000 7fc6e37e4700 1 --2- 172.21.15.12:0/1542805815 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc6dc198fa0 0x7fc6dc199380 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:27.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:27.776+0000 7fc6e37e4700 1 -- 172.21.15.12:0/1542805815 >> 172.21.15.12:0/1542805815 conn(0x7fc6dc076ea0 msgr2=0x7fc6dc109110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:27.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:27.776+0000 7fc6e37e4700 1 -- 172.21.15.12:0/1542805815 shutdown_connections 2024-04-22T13:15:27.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:27.776+0000 7fc6e37e4700 1 -- 172.21.15.12:0/1542805815 wait complete. 2024-04-22T13:15:28.428 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-22T13:15:28.676 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:15:29.046 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:28 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:29.046 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:28 smithi012 ceph-mon[28265]: Added host smithi111 2024-04-22T13:15:29.046 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:28 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:15:29.046 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:28 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:30.020 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:29 smithi012 ceph-mon[28265]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:15:30.020 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:29 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:30.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.243+0000 7ff358e24700 1 -- 172.21.15.12:0/1931955804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3541015f0 msgr2=0x7ff3541019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:30.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.243+0000 7ff358e24700 1 --2- 172.21.15.12:0/1931955804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3541015f0 0x7ff3541019d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff34c009910 tx=0x7ff34c009c20 comp rx=0 tx=0).stop 2024-04-22T13:15:30.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.243+0000 7ff358e24700 1 -- 172.21.15.12:0/1931955804 shutdown_connections 2024-04-22T13:15:30.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.243+0000 7ff358e24700 1 --2- 172.21.15.12:0/1931955804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3541015f0 0x7ff3541019d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:30.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.243+0000 7ff358e24700 1 -- 172.21.15.12:0/1931955804 >> 172.21.15.12:0/1931955804 conn(0x7ff3540fcea0 msgr2=0x7ff3540ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:30.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.243+0000 7ff358e24700 1 -- 172.21.15.12:0/1931955804 shutdown_connections 2024-04-22T13:15:30.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.243+0000 7ff358e24700 1 -- 172.21.15.12:0/1931955804 wait complete. 2024-04-22T13:15:30.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.244+0000 7ff358e24700 1 Processor -- start 2024-04-22T13:15:30.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.244+0000 7ff358e24700 1 -- start start 2024-04-22T13:15:30.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.244+0000 7ff358e24700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3541015f0 0x7ff354198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:30.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.244+0000 7ff358e24700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff35410b870 con 0x7ff3541015f0 2024-04-22T13:15:30.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.244+0000 7ff35259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3541015f0 0x7ff354198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:30.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.244+0000 7ff35259c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3541015f0 0x7ff354198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57696/0 (socket says 172.21.15.12:57696) 2024-04-22T13:15:30.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.244+0000 7ff35259c700 1 -- 172.21.15.12:0/1033301491 learned_addr learned my addr 172.21.15.12:0/1033301491 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:15:30.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.244+0000 7ff35259c700 1 -- 172.21.15.12:0/1033301491 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff354199290 con 0x7ff3541015f0 2024-04-22T13:15:30.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.245+0000 7ff35259c700 1 --2- 172.21.15.12:0/1033301491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3541015f0 0x7ff354198d50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff34c006010 tx=0x7ff34c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:30.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.245+0000 7ff3437fe700 1 -- 172.21.15.12:0/1033301491 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff34c026070 con 0x7ff3541015f0 2024-04-22T13:15:30.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.245+0000 7ff358e24700 1 -- 172.21.15.12:0/1033301491 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff354199580 con 0x7ff3541015f0 2024-04-22T13:15:30.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.245+0000 7ff3437fe700 1 -- 172.21.15.12:0/1033301491 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff34c00bba0 con 0x7ff3541015f0 2024-04-22T13:15:30.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.245+0000 7ff3437fe700 1 -- 172.21.15.12:0/1033301491 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff34c003910 con 0x7ff3541015f0 2024-04-22T13:15:30.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.245+0000 7ff358e24700 1 -- 172.21.15.12:0/1033301491 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff354199a40 con 0x7ff3541015f0 2024-04-22T13:15:30.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.246+0000 7ff3437fe700 1 -- 172.21.15.12:0/1033301491 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ff34c029070 con 0x7ff3541015f0 2024-04-22T13:15:30.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.246+0000 7ff358e24700 1 -- 172.21.15.12:0/1033301491 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff354102ad0 con 0x7ff3541015f0 2024-04-22T13:15:30.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.246+0000 7ff3437fe700 1 --2- 172.21.15.12:0/1033301491 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ff33c039650 0x7ff33c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:30.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.246+0000 7ff3437fe700 1 -- 172.21.15.12:0/1033301491 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff34c01e070 con 0x7ff3541015f0 2024-04-22T13:15:30.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.246+0000 7ff351d9b700 1 --2- 172.21.15.12:0/1033301491 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ff33c039650 0x7ff33c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:30.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.246+0000 7ff351d9b700 1 --2- 172.21.15.12:0/1033301491 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ff33c039650 0x7ff33c03bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff348006fd0 tx=0x7ff348006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:30.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.251+0000 7ff3437fe700 1 -- 172.21.15.12:0/1033301491 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff34c013070 con 0x7ff3541015f0 2024-04-22T13:15:30.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.466+0000 7ff358e24700 1 -- 172.21.15.12:0/1033301491 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff35406aea0 con 0x7ff33c039650 2024-04-22T13:15:30.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.467+0000 7ff3437fe700 1 -- 172.21.15.12:0/1033301491 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7ff35406aea0 con 0x7ff33c039650 2024-04-22T13:15:30.466 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:15:30.466 INFO:teuthology.orchestra.run.smithi012.stdout:[{"addr": "172.21.15.12", "hostname": "smithi012", "labels": [], "status": ""}, {"addr": "172.21.15.111", "hostname": "smithi111", "labels": [], "status": ""}] 2024-04-22T13:15:30.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.469+0000 7ff358e24700 1 -- 172.21.15.12:0/1033301491 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ff33c039650 msgr2=0x7ff33c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:30.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.469+0000 7ff358e24700 1 --2- 172.21.15.12:0/1033301491 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ff33c039650 0x7ff33c03bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff348006fd0 tx=0x7ff348006e40 comp rx=0 tx=0).stop 2024-04-22T13:15:30.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.469+0000 7ff358e24700 1 -- 172.21.15.12:0/1033301491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3541015f0 msgr2=0x7ff354198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:30.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.469+0000 7ff358e24700 1 --2- 172.21.15.12:0/1033301491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3541015f0 0x7ff354198d50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff34c006010 tx=0x7ff34c005780 comp rx=0 tx=0).stop 2024-04-22T13:15:30.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.469+0000 7ff358e24700 1 -- 172.21.15.12:0/1033301491 shutdown_connections 2024-04-22T13:15:30.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.469+0000 7ff358e24700 1 --2- 172.21.15.12:0/1033301491 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ff33c039650 0x7ff33c03bb10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:30.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.469+0000 7ff358e24700 1 --2- 172.21.15.12:0/1033301491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3541015f0 0x7ff354198d50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:30.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.469+0000 7ff358e24700 1 -- 172.21.15.12:0/1033301491 >> 172.21.15.12:0/1033301491 conn(0x7ff3540fcea0 msgr2=0x7ff354102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:30.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.469+0000 7ff358e24700 1 -- 172.21.15.12:0/1033301491 shutdown_connections 2024-04-22T13:15:30.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:30.469+0000 7ff358e24700 1 -- 172.21.15.12:0/1033301491 wait complete. 2024-04-22T13:15:31.113 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi134 2024-04-22T13:15:31.113 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-22T13:15:31.113 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/etc/ceph/ceph.conf 2024-04-22T13:15:31.132 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-22T13:15:31.132 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:15:31.188 INFO:tasks.cephadm:Adding host smithi134 to orchestrator... 2024-04-22T13:15:31.189 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph orch host add smithi134 2024-04-22T13:15:31.439 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:15:31.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:31 smithi012 ceph-mon[28265]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:15:31.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:31 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:31.665 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:31 smithi012 ceph-mon[28265]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:15:32.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:32.945+0000 7f6fbb265700 1 -- 172.21.15.12:0/1607887407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fb41015b0 msgr2=0x7f6fb4101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:32.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:32.945+0000 7f6fbb265700 1 --2- 172.21.15.12:0/1607887407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fb41015b0 0x7f6fb4101990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4009910 tx=0x7f6fa4009c20 comp rx=0 tx=0).stop 2024-04-22T13:15:32.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:32.945+0000 7f6fbb265700 1 -- 172.21.15.12:0/1607887407 shutdown_connections 2024-04-22T13:15:32.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:32.945+0000 7f6fbb265700 1 --2- 172.21.15.12:0/1607887407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fb41015b0 0x7f6fb4101990 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:32.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:32.945+0000 7f6fbb265700 1 -- 172.21.15.12:0/1607887407 >> 172.21.15.12:0/1607887407 conn(0x7f6fb40fce40 msgr2=0x7f6fb40ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:32.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:32.945+0000 7f6fbb265700 1 -- 172.21.15.12:0/1607887407 shutdown_connections 2024-04-22T13:15:32.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:32.946+0000 7f6fbb265700 1 -- 172.21.15.12:0/1607887407 wait complete. 2024-04-22T13:15:32.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:32.946+0000 7f6fbb265700 1 Processor -- start 2024-04-22T13:15:32.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:32.946+0000 7f6fbb265700 1 -- start start 2024-04-22T13:15:32.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:32.946+0000 7f6fbb265700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fb41015b0 0x7f6fb4198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:32.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:32.946+0000 7f6fbb265700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb410b830 con 0x7f6fb41015b0 2024-04-22T13:15:32.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:32.946+0000 7f6fb9001700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fb41015b0 0x7f6fb4198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:32.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:32.946+0000 7f6fb9001700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fb41015b0 0x7f6fb4198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59682/0 (socket says 172.21.15.12:59682) 2024-04-22T13:15:32.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:32.946+0000 7f6fb9001700 1 -- 172.21.15.12:0/3668677067 learned_addr learned my addr 172.21.15.12:0/3668677067 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:15:32.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:32.947+0000 7f6fb9001700 1 -- 172.21.15.12:0/3668677067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb4199240 con 0x7f6fb41015b0 2024-04-22T13:15:32.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:32.947+0000 7f6fb9001700 1 --2- 172.21.15.12:0/3668677067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fb41015b0 0x7f6fb4198d00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6fa40056d0 tx=0x7f6fa40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:32.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:32.947+0000 7f6fb27fc700 1 -- 172.21.15.12:0/3668677067 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fa4026070 con 0x7f6fb41015b0 2024-04-22T13:15:32.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:32.947+0000 7f6fb27fc700 1 -- 172.21.15.12:0/3668677067 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fa400bba0 con 0x7f6fb41015b0 2024-04-22T13:15:32.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:32.947+0000 7f6fbb265700 1 -- 172.21.15.12:0/3668677067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fb4199530 con 0x7f6fb41015b0 2024-04-22T13:15:32.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:32.947+0000 7f6fb27fc700 1 -- 172.21.15.12:0/3668677067 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fa4003920 con 0x7f6fb41015b0 2024-04-22T13:15:32.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:32.947+0000 7f6fbb265700 1 -- 172.21.15.12:0/3668677067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fb41999f0 con 0x7f6fb41015b0 2024-04-22T13:15:32.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:32.948+0000 7f6fb27fc700 1 -- 172.21.15.12:0/3668677067 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6fa4029070 con 0x7f6fb41015b0 2024-04-22T13:15:32.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:32.948+0000 7f6fbb265700 1 -- 172.21.15.12:0/3668677067 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fb4102a90 con 0x7f6fb41015b0 2024-04-22T13:15:32.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:32.948+0000 7f6fb27fc700 1 --2- 172.21.15.12:0/3668677067 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f6f9c0396b0 0x7f6f9c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:32.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:32.948+0000 7f6fb27fc700 1 -- 172.21.15.12:0/3668677067 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6fa401e070 con 0x7f6fb41015b0 2024-04-22T13:15:32.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:32.948+0000 7f6fb8800700 1 --2- 172.21.15.12:0/3668677067 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f6f9c0396b0 0x7f6f9c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:32.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:32.949+0000 7f6fb8800700 1 --2- 172.21.15.12:0/3668677067 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f6f9c0396b0 0x7f6f9c03bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8006fd0 tx=0x7f6fa8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:32.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:32.953+0000 7f6fb27fc700 1 -- 172.21.15.12:0/3668677067 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6fa4013070 con 0x7f6fb41015b0 2024-04-22T13:15:33.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:33.163+0000 7f6fbb265700 1 -- 172.21.15.12:0/3668677067 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}) -- 0x7f6fb406aea0 con 0x7f6f9c0396b0 2024-04-22T13:15:33.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:33 smithi012 ceph-mon[28265]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:15:33.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:33 smithi012 ceph-mon[28265]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi134", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:15:34.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:34 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:34.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:34 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:34.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:34 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:34.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:34 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:34.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:34 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi111", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:15:34.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:34 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:15:34.840 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:34 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:15:34.840 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:34 smithi012 ceph-mon[28265]: Updating smithi111:/etc/ceph/ceph.conf 2024-04-22T13:15:34.840 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:34 smithi012 ceph-mon[28265]: Deploying cephadm binary to smithi134 2024-04-22T13:15:34.840 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:34 smithi012 ceph-mon[28265]: Updating smithi111:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:15:34.840 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:34 smithi012 ceph-mon[28265]: Updating smithi111:/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:15:35.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:35 smithi012 ceph-mon[28265]: Updating smithi111:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T13:15:35.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:35 smithi012 ceph-mon[28265]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:15:37.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:37.325+0000 7f6fb27fc700 1 -- 172.21.15.12:0/3668677067 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f6fb406aea0 con 0x7f6f9c0396b0 2024-04-22T13:15:37.325 INFO:teuthology.orchestra.run.smithi012.stdout:Added host 'smithi134' with addr '172.21.15.134' 2024-04-22T13:15:37.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:37.327+0000 7f6fbb265700 1 -- 172.21.15.12:0/3668677067 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f6f9c0396b0 msgr2=0x7f6f9c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:37.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:37.327+0000 7f6fbb265700 1 --2- 172.21.15.12:0/3668677067 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f6f9c0396b0 0x7f6f9c03bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8006fd0 tx=0x7f6fa8006e40 comp rx=0 tx=0).stop 2024-04-22T13:15:37.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:37.327+0000 7f6fbb265700 1 -- 172.21.15.12:0/3668677067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fb41015b0 msgr2=0x7f6fb4198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:37.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:37.327+0000 7f6fbb265700 1 --2- 172.21.15.12:0/3668677067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fb41015b0 0x7f6fb4198d00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6fa40056d0 tx=0x7f6fa40057b0 comp rx=0 tx=0).stop 2024-04-22T13:15:37.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:37.327+0000 7f6fbb265700 1 -- 172.21.15.12:0/3668677067 shutdown_connections 2024-04-22T13:15:37.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:37.327+0000 7f6fbb265700 1 --2- 172.21.15.12:0/3668677067 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f6f9c0396b0 0x7f6f9c03bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:37.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:37.327+0000 7f6fbb265700 1 --2- 172.21.15.12:0/3668677067 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fb41015b0 0x7f6fb4198d00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:37.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:37.327+0000 7f6fbb265700 1 -- 172.21.15.12:0/3668677067 >> 172.21.15.12:0/3668677067 conn(0x7f6fb40fce40 msgr2=0x7f6fb4102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:37.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:37.327+0000 7f6fbb265700 1 -- 172.21.15.12:0/3668677067 shutdown_connections 2024-04-22T13:15:37.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:37.327+0000 7f6fbb265700 1 -- 172.21.15.12:0/3668677067 wait complete. 2024-04-22T13:15:37.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:37 smithi012 ceph-mon[28265]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:15:37.916 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:37 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:37.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:37 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:37.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:37 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:37.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:37 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:37.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:37 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:15:37.917 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:37 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:37.945 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-22T13:15:38.186 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:15:39.257 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:38 smithi012 ceph-mon[28265]: Added host smithi134 2024-04-22T13:15:39.257 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:38 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:39.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:39.788+0000 7f7ebe4b1700 1 -- 172.21.15.12:0/2880901682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb81015f0 msgr2=0x7f7eb81019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:39.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:39.788+0000 7f7ebe4b1700 1 --2- 172.21.15.12:0/2880901682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb81015f0 0x7f7eb81019d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8009910 tx=0x7f7ea8009c20 comp rx=0 tx=0).stop 2024-04-22T13:15:39.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:39.788+0000 7f7ebe4b1700 1 -- 172.21.15.12:0/2880901682 shutdown_connections 2024-04-22T13:15:39.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:39.788+0000 7f7ebe4b1700 1 --2- 172.21.15.12:0/2880901682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb81015f0 0x7f7eb81019d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:39.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:39.788+0000 7f7ebe4b1700 1 -- 172.21.15.12:0/2880901682 >> 172.21.15.12:0/2880901682 conn(0x7f7eb80fcea0 msgr2=0x7f7eb80ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:39.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:39.788+0000 7f7ebe4b1700 1 -- 172.21.15.12:0/2880901682 shutdown_connections 2024-04-22T13:15:39.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:39.788+0000 7f7ebe4b1700 1 -- 172.21.15.12:0/2880901682 wait complete. 2024-04-22T13:15:39.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:39.789+0000 7f7ebe4b1700 1 Processor -- start 2024-04-22T13:15:39.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:39.789+0000 7f7ebe4b1700 1 -- start start 2024-04-22T13:15:39.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:39.789+0000 7f7ebe4b1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb81015f0 0x7f7eb8198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:39.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:39.789+0000 7f7ebe4b1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb810b870 con 0x7f7eb81015f0 2024-04-22T13:15:39.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:39.790+0000 7f7eb7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb81015f0 0x7f7eb8198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:39.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:39.790+0000 7f7eb7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb81015f0 0x7f7eb8198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59708/0 (socket says 172.21.15.12:59708) 2024-04-22T13:15:39.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:39.790+0000 7f7eb7fff700 1 -- 172.21.15.12:0/3534416578 learned_addr learned my addr 172.21.15.12:0/3534416578 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:15:39.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:39.790+0000 7f7eb7fff700 1 -- 172.21.15.12:0/3534416578 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb8199230 con 0x7f7eb81015f0 2024-04-22T13:15:39.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:39.790+0000 7f7eb7fff700 1 --2- 172.21.15.12:0/3534416578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb81015f0 0x7f7eb8198cf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8006010 tx=0x7f7ea8005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:39.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:39.790+0000 7f7eb57fa700 1 -- 172.21.15.12:0/3534416578 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ea8026070 con 0x7f7eb81015f0 2024-04-22T13:15:39.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:39.790+0000 7f7eb57fa700 1 -- 172.21.15.12:0/3534416578 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ea800bba0 con 0x7f7eb81015f0 2024-04-22T13:15:39.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:39.790+0000 7f7ebe4b1700 1 -- 172.21.15.12:0/3534416578 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb8199520 con 0x7f7eb81015f0 2024-04-22T13:15:39.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:39.790+0000 7f7eb57fa700 1 -- 172.21.15.12:0/3534416578 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ea8003910 con 0x7f7eb81015f0 2024-04-22T13:15:39.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:39.790+0000 7f7ebe4b1700 1 -- 172.21.15.12:0/3534416578 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb81999e0 con 0x7f7eb81015f0 2024-04-22T13:15:39.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:39.791+0000 7f7eb57fa700 1 -- 172.21.15.12:0/3534416578 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f7ea8029070 con 0x7f7eb81015f0 2024-04-22T13:15:39.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:39.791+0000 7f7eb57fa700 1 --2- 172.21.15.12:0/3534416578 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f7ea0039700 0x7f7ea003bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:39.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:39.791+0000 7f7eb57fa700 1 -- 172.21.15.12:0/3534416578 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7ea801e070 con 0x7f7eb81015f0 2024-04-22T13:15:39.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:39.791+0000 7f7ebe4b1700 1 -- 172.21.15.12:0/3534416578 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eb8102ad0 con 0x7f7eb81015f0 2024-04-22T13:15:39.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:39.791+0000 7f7eb77fe700 1 --2- 172.21.15.12:0/3534416578 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f7ea0039700 0x7f7ea003bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:39.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:39.792+0000 7f7eb77fe700 1 --2- 172.21.15.12:0/3534416578 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f7ea0039700 0x7f7ea003bbc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7eac006fd0 tx=0x7f7eac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:39.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:39.797+0000 7f7eb57fa700 1 -- 172.21.15.12:0/3534416578 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ea8030020 con 0x7f7eb81015f0 2024-04-22T13:15:40.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:40.012+0000 7f7ebe4b1700 1 -- 172.21.15.12:0/3534416578 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7eb80640a0 con 0x7f7ea0039700 2024-04-22T13:15:40.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:40.013+0000 7f7eb57fa700 1 -- 172.21.15.12:0/3534416578 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f7eb80640a0 con 0x7f7ea0039700 2024-04-22T13:15:40.014 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:15:40.014 INFO:teuthology.orchestra.run.smithi012.stdout:[{"addr": "172.21.15.12", "hostname": "smithi012", "labels": [], "status": ""}, {"addr": "172.21.15.111", "hostname": "smithi111", "labels": [], "status": ""}, {"addr": "172.21.15.134", "hostname": "smithi134", "labels": [], "status": ""}] 2024-04-22T13:15:40.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:40.015+0000 7f7ebe4b1700 1 -- 172.21.15.12:0/3534416578 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f7ea0039700 msgr2=0x7f7ea003bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:40.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:40.015+0000 7f7ebe4b1700 1 --2- 172.21.15.12:0/3534416578 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f7ea0039700 0x7f7ea003bbc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7eac006fd0 tx=0x7f7eac006e40 comp rx=0 tx=0).stop 2024-04-22T13:15:40.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:40.015+0000 7f7ebe4b1700 1 -- 172.21.15.12:0/3534416578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb81015f0 msgr2=0x7f7eb8198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:40.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:40.015+0000 7f7ebe4b1700 1 --2- 172.21.15.12:0/3534416578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb81015f0 0x7f7eb8198cf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8006010 tx=0x7f7ea8005780 comp rx=0 tx=0).stop 2024-04-22T13:15:40.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:40.015+0000 7f7ebe4b1700 1 -- 172.21.15.12:0/3534416578 shutdown_connections 2024-04-22T13:15:40.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:40.015+0000 7f7ebe4b1700 1 --2- 172.21.15.12:0/3534416578 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f7ea0039700 0x7f7ea003bbc0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:40.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:40.015+0000 7f7ebe4b1700 1 --2- 172.21.15.12:0/3534416578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7eb81015f0 0x7f7eb8198cf0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:40.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:40.015+0000 7f7ebe4b1700 1 -- 172.21.15.12:0/3534416578 >> 172.21.15.12:0/3534416578 conn(0x7f7eb80fcea0 msgr2=0x7f7eb8102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:40.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:40.016+0000 7f7ebe4b1700 1 -- 172.21.15.12:0/3534416578 shutdown_connections 2024-04-22T13:15:40.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:40.016+0000 7f7ebe4b1700 1 -- 172.21.15.12:0/3534416578 wait complete. 2024-04-22T13:15:40.320 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:39 smithi012 ceph-mon[28265]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:15:40.641 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-22T13:15:40.641 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-22T13:15:40.887 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:15:41.239 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:40 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:41.240 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:40 smithi012 ceph-mon[28265]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:15:42.118 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:41 smithi012 ceph-mon[28265]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:15:42.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.346+0000 7f87b86ba700 1 -- 172.21.15.12:0/1339389162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87b01074d0 msgr2=0x7f87b01078b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:42.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.346+0000 7f87b86ba700 1 --2- 172.21.15.12:0/1339389162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87b01074d0 0x7f87b01078b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f87ac009910 tx=0x7f87ac009c20 comp rx=0 tx=0).stop 2024-04-22T13:15:42.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.347+0000 7f87b86ba700 1 -- 172.21.15.12:0/1339389162 shutdown_connections 2024-04-22T13:15:42.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.347+0000 7f87b86ba700 1 --2- 172.21.15.12:0/1339389162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87b01074d0 0x7f87b01078b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:42.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.347+0000 7f87b86ba700 1 -- 172.21.15.12:0/1339389162 >> 172.21.15.12:0/1339389162 conn(0x7f87b0078970 msgr2=0x7f87b0078d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:42.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.347+0000 7f87b86ba700 1 -- 172.21.15.12:0/1339389162 shutdown_connections 2024-04-22T13:15:42.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.347+0000 7f87b86ba700 1 -- 172.21.15.12:0/1339389162 wait complete. 2024-04-22T13:15:42.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.347+0000 7f87b86ba700 1 Processor -- start 2024-04-22T13:15:42.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.347+0000 7f87b86ba700 1 -- start start 2024-04-22T13:15:42.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.348+0000 7f87b86ba700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87b01074d0 0x7f87b00fdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:42.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.348+0000 7f87b86ba700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f87b010b8b0 con 0x7f87b01074d0 2024-04-22T13:15:42.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.348+0000 7f87b6456700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87b01074d0 0x7f87b00fdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:42.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.348+0000 7f87b6456700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87b01074d0 0x7f87b00fdfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47572/0 (socket says 172.21.15.12:47572) 2024-04-22T13:15:42.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.348+0000 7f87b6456700 1 -- 172.21.15.12:0/3691676967 learned_addr learned my addr 172.21.15.12:0/3691676967 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:15:42.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.348+0000 7f87b6456700 1 -- 172.21.15.12:0/3691676967 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b00ffe70 con 0x7f87b01074d0 2024-04-22T13:15:42.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.348+0000 7f87b6456700 1 --2- 172.21.15.12:0/3691676967 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87b01074d0 0x7f87b00fdfd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f87ac006010 tx=0x7f87ac005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:42.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.348+0000 7f87a77fe700 1 -- 172.21.15.12:0/3691676967 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87ac026070 con 0x7f87b01074d0 2024-04-22T13:15:42.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.348+0000 7f87a77fe700 1 -- 172.21.15.12:0/3691676967 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87ac00bc00 con 0x7f87b01074d0 2024-04-22T13:15:42.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.348+0000 7f87b86ba700 1 -- 172.21.15.12:0/3691676967 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87b00fe510 con 0x7f87b01074d0 2024-04-22T13:15:42.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.348+0000 7f87a77fe700 1 -- 172.21.15.12:0/3691676967 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87ac003910 con 0x7f87b01074d0 2024-04-22T13:15:42.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.348+0000 7f87b86ba700 1 -- 172.21.15.12:0/3691676967 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87b00fe9d0 con 0x7f87b01074d0 2024-04-22T13:15:42.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.349+0000 7f87a77fe700 1 -- 172.21.15.12:0/3691676967 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f87ac029070 con 0x7f87b01074d0 2024-04-22T13:15:42.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.349+0000 7f87b86ba700 1 -- 172.21.15.12:0/3691676967 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87b0102c70 con 0x7f87b01074d0 2024-04-22T13:15:42.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.349+0000 7f87a77fe700 1 --2- 172.21.15.12:0/3691676967 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f879c039650 0x7f879c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:42.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.349+0000 7f87a77fe700 1 -- 172.21.15.12:0/3691676967 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f87ac01e070 con 0x7f87b01074d0 2024-04-22T13:15:42.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.350+0000 7f87b5c55700 1 --2- 172.21.15.12:0/3691676967 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f879c039650 0x7f879c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:42.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.350+0000 7f87b5c55700 1 --2- 172.21.15.12:0/3691676967 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f879c039650 0x7f879c03bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f87a0006fd0 tx=0x7f87a0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:42.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.355+0000 7f87a77fe700 1 -- 172.21.15.12:0/3691676967 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f87ac013070 con 0x7f87b01074d0 2024-04-22T13:15:42.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.568+0000 7f87b86ba700 1 -- 172.21.15.12:0/3691676967 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f87b00ff4e0 con 0x7f87b01074d0 2024-04-22T13:15:42.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.926+0000 7f87a77fe700 1 -- 172.21.15.12:0/3691676967 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f87ac00fe20 con 0x7f87b01074d0 2024-04-22T13:15:42.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.929+0000 7f87b86ba700 1 -- 172.21.15.12:0/3691676967 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f879c039650 msgr2=0x7f879c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:42.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.929+0000 7f87b86ba700 1 --2- 172.21.15.12:0/3691676967 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f879c039650 0x7f879c03bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f87a0006fd0 tx=0x7f87a0006e40 comp rx=0 tx=0).stop 2024-04-22T13:15:42.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.929+0000 7f87b86ba700 1 -- 172.21.15.12:0/3691676967 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87b01074d0 msgr2=0x7f87b00fdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:42.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.929+0000 7f87b86ba700 1 --2- 172.21.15.12:0/3691676967 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87b01074d0 0x7f87b00fdfd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f87ac006010 tx=0x7f87ac005780 comp rx=0 tx=0).stop 2024-04-22T13:15:42.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.930+0000 7f87b86ba700 1 -- 172.21.15.12:0/3691676967 shutdown_connections 2024-04-22T13:15:42.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.930+0000 7f87b86ba700 1 --2- 172.21.15.12:0/3691676967 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f879c039650 0x7f879c03bb10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:42.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.930+0000 7f87b86ba700 1 --2- 172.21.15.12:0/3691676967 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f87b01074d0 0x7f87b00fdfd0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:42.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.930+0000 7f87b86ba700 1 -- 172.21.15.12:0/3691676967 >> 172.21.15.12:0/3691676967 conn(0x7f87b0078970 msgr2=0x7f87b0104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:42.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.930+0000 7f87b86ba700 1 -- 172.21.15.12:0/3691676967 shutdown_connections 2024-04-22T13:15:42.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:15:42.930+0000 7f87b86ba700 1 -- 172.21.15.12:0/3691676967 wait complete. 2024-04-22T13:15:42.930 INFO:teuthology.orchestra.run.smithi012.stderr:adjusted tunables profile to default 2024-04-22T13:15:43.257 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:42 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3691676967' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-22T13:15:43.578 INFO:tasks.cephadm:Adding mon.a on smithi012 2024-04-22T13:15:43.578 INFO:tasks.cephadm:Adding mon.b on smithi111 2024-04-22T13:15:43.578 INFO:tasks.cephadm:Adding mon.c on smithi134 2024-04-22T13:15:43.579 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi012:172.21.15.12=a;smithi111:172.21.15.111=b;smithi134:172.21.15.134=c' 2024-04-22T13:15:43.819 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:15:44.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:43 smithi012 ceph-mon[28265]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:15:44.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:43 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3691676967' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-22T13:15:44.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:43 smithi012 ceph-mon[28265]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T13:15:44.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:44.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:44.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:44.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:44.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi134", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:15:44.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:15:44.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:15:44.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:43 smithi012 ceph-mon[28265]: Updating smithi134:/etc/ceph/ceph.conf 2024-04-22T13:15:44.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:44.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:44.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:45.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.315+0000 7f8924ce3700 1 -- 172.21.15.134:0/2438672276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89201084e0 msgr2=0x7f89201088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:45.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.315+0000 7f8924ce3700 1 --2- 172.21.15.134:0/2438672276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89201084e0 0x7f89201088c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8910009910 tx=0x7f8910009c20 comp rx=0 tx=0).stop 2024-04-22T13:15:45.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.315+0000 7f8924ce3700 1 -- 172.21.15.134:0/2438672276 shutdown_connections 2024-04-22T13:15:45.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.315+0000 7f8924ce3700 1 --2- 172.21.15.134:0/2438672276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89201084e0 0x7f89201088c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:45.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.315+0000 7f8924ce3700 1 -- 172.21.15.134:0/2438672276 >> 172.21.15.134:0/2438672276 conn(0x7f89200fdf60 msgr2=0x7f8920100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:45.314 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.315+0000 7f8924ce3700 1 -- 172.21.15.134:0/2438672276 shutdown_connections 2024-04-22T13:15:45.315 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.315+0000 7f8924ce3700 1 -- 172.21.15.134:0/2438672276 wait complete. 2024-04-22T13:15:45.316 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.316+0000 7f8924ce3700 1 Processor -- start 2024-04-22T13:15:45.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.316+0000 7f8924ce3700 1 -- start start 2024-04-22T13:15:45.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.316+0000 7f8924ce3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89201084e0 0x7f8920197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:45.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.316+0000 7f8924ce3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f892010c940 con 0x7f89201084e0 2024-04-22T13:15:45.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.317+0000 7f891e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89201084e0 0x7f8920197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:45.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.317+0000 7f891e59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89201084e0 0x7f8920197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.134:57706/0 (socket says 172.21.15.134:57706) 2024-04-22T13:15:45.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.317+0000 7f891e59c700 1 -- 172.21.15.134:0/599194710 learned_addr learned my addr 172.21.15.134:0/599194710 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-22T13:15:45.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.317+0000 7f891e59c700 1 -- 172.21.15.134:0/599194710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89201983d0 con 0x7f89201084e0 2024-04-22T13:15:45.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.317+0000 7f891e59c700 1 --2- 172.21.15.134:0/599194710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89201084e0 0x7f8920197e90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f89100056d0 tx=0x7f89100057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:45.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.317+0000 7f890f7fe700 1 -- 172.21.15.134:0/599194710 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8910026070 con 0x7f89201084e0 2024-04-22T13:15:45.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.317+0000 7f8924ce3700 1 -- 172.21.15.134:0/599194710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89201986c0 con 0x7f89201084e0 2024-04-22T13:15:45.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.317+0000 7f890f7fe700 1 -- 172.21.15.134:0/599194710 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f891000bba0 con 0x7f89201084e0 2024-04-22T13:15:45.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.317+0000 7f890f7fe700 1 -- 172.21.15.134:0/599194710 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8910003920 con 0x7f89201084e0 2024-04-22T13:15:45.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.317+0000 7f8924ce3700 1 -- 172.21.15.134:0/599194710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f892019c760 con 0x7f89201084e0 2024-04-22T13:15:45.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.318+0000 7f8924ce3700 1 -- 172.21.15.134:0/599194710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8920103b90 con 0x7f89201084e0 2024-04-22T13:15:45.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.318+0000 7f890f7fe700 1 -- 172.21.15.134:0/599194710 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8910029070 con 0x7f89201084e0 2024-04-22T13:15:45.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.318+0000 7f890f7fe700 1 --2- 172.21.15.134:0/599194710 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f8908039650 0x7f890803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:45.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.318+0000 7f890f7fe700 1 -- 172.21.15.134:0/599194710 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f891001e070 con 0x7f89201084e0 2024-04-22T13:15:45.318 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.319+0000 7f891dd9b700 1 --2- 172.21.15.134:0/599194710 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f8908039650 0x7f890803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:45.319 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.319+0000 7f891dd9b700 1 --2- 172.21.15.134:0/599194710 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f8908039650 0x7f890803bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8914006fd0 tx=0x7f8914006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:45.322 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.324+0000 7f890f7fe700 1 -- 172.21.15.134:0/599194710 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8910013070 con 0x7f89201084e0 2024-04-22T13:15:45.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:44 smithi012 ceph-mon[28265]: Updating smithi134:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:15:45.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:44 smithi012 ceph-mon[28265]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:15:45.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:44 smithi012 ceph-mon[28265]: Updating smithi134:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T13:15:45.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.530+0000 7f8924ce3700 1 -- 172.21.15.134:0/599194710 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi012:172.21.15.12=a;smithi111:172.21.15.111=b;smithi134:172.21.15.134=c", "target": ["mon-mgr", ""]}) -- 0x7f89201991f0 con 0x7f8908039650 2024-04-22T13:15:45.536 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.537+0000 7f890f7fe700 1 -- 172.21.15.134:0/599194710 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f89201991f0 con 0x7f8908039650 2024-04-22T13:15:45.536 INFO:teuthology.orchestra.run.smithi134.stdout:Scheduled mon update... 2024-04-22T13:15:45.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.540+0000 7f8924ce3700 1 -- 172.21.15.134:0/599194710 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f8908039650 msgr2=0x7f890803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:45.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.540+0000 7f8924ce3700 1 --2- 172.21.15.134:0/599194710 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f8908039650 0x7f890803bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8914006fd0 tx=0x7f8914006e40 comp rx=0 tx=0).stop 2024-04-22T13:15:45.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.540+0000 7f8924ce3700 1 -- 172.21.15.134:0/599194710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89201084e0 msgr2=0x7f8920197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:45.538 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.540+0000 7f8924ce3700 1 --2- 172.21.15.134:0/599194710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89201084e0 0x7f8920197e90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f89100056d0 tx=0x7f89100057b0 comp rx=0 tx=0).stop 2024-04-22T13:15:45.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.540+0000 7f8924ce3700 1 -- 172.21.15.134:0/599194710 shutdown_connections 2024-04-22T13:15:45.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.540+0000 7f8924ce3700 1 --2- 172.21.15.134:0/599194710 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f8908039650 0x7f890803bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:45.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.540+0000 7f8924ce3700 1 --2- 172.21.15.134:0/599194710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f89201084e0 0x7f8920197e90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:45.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.540+0000 7f8924ce3700 1 -- 172.21.15.134:0/599194710 >> 172.21.15.134:0/599194710 conn(0x7f89200fdf60 msgr2=0x7f8920109ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:45.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.540+0000 7f8924ce3700 1 -- 172.21.15.134:0/599194710 shutdown_connections 2024-04-22T13:15:45.539 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:45.540+0000 7f8924ce3700 1 -- 172.21.15.134:0/599194710 wait complete. 2024-04-22T13:15:46.160 DEBUG:teuthology.orchestra.run.smithi111:mon.b> sudo journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mon.b.service 2024-04-22T13:15:46.163 DEBUG:teuthology.orchestra.run.smithi134:mon.c> sudo journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mon.c.service 2024-04-22T13:15:46.165 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-22T13:15:46.165 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T13:15:46.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:45 smithi012 ceph-mon[28265]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:15:46.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:45 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:46.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:45 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:15:46.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:45 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:15:46.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:45 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:15:46.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:45 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:46.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:45 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T13:15:46.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:45 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:15:47.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:46 smithi012 ceph-mon[28265]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi012:172.21.15.12=a;smithi111:172.21.15.111=b;smithi134:172.21.15.134=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:15:47.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:46 smithi012 ceph-mon[28265]: Saving service mon spec with placement smithi012:172.21.15.12=a;smithi111:172.21.15.111=b;smithi134:172.21.15.134=c;count:3 2024-04-22T13:15:47.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:46 smithi012 ceph-mon[28265]: Deploying daemon mon.c on smithi134 2024-04-22T13:15:47.549 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:15:48.240 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:47 smithi012 ceph-mon[28265]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:15:50.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.149+0000 7ffa8bbda700 1 -- 172.21.15.134:0/1593076076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa841015b0 msgr2=0x7ffa84101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:50.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.149+0000 7ffa8bbda700 1 --2- 172.21.15.134:0/1593076076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa841015b0 0x7ffa84101990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ffa74009910 tx=0x7ffa74009c20 comp rx=0 tx=0).stop 2024-04-22T13:15:50.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.149+0000 7ffa8bbda700 1 -- 172.21.15.134:0/1593076076 shutdown_connections 2024-04-22T13:15:50.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.149+0000 7ffa8bbda700 1 --2- 172.21.15.134:0/1593076076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa841015b0 0x7ffa84101990 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:50.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.149+0000 7ffa8bbda700 1 -- 172.21.15.134:0/1593076076 >> 172.21.15.134:0/1593076076 conn(0x7ffa840fce60 msgr2=0x7ffa840ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:50.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.149+0000 7ffa8bbda700 1 -- 172.21.15.134:0/1593076076 shutdown_connections 2024-04-22T13:15:50.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.149+0000 7ffa8bbda700 1 -- 172.21.15.134:0/1593076076 wait complete. 2024-04-22T13:15:50.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.149+0000 7ffa8bbda700 1 Processor -- start 2024-04-22T13:15:50.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.150+0000 7ffa8bbda700 1 -- start start 2024-04-22T13:15:50.153 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.150+0000 7ffa8bbda700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa841015b0 0x7ffa84198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:50.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.150+0000 7ffa8bbda700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8410b830 con 0x7ffa841015b0 2024-04-22T13:15:50.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.150+0000 7ffa89976700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa841015b0 0x7ffa84198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:50.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.150+0000 7ffa89976700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa841015b0 0x7ffa84198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.134:57728/0 (socket says 172.21.15.134:57728) 2024-04-22T13:15:50.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.150+0000 7ffa89976700 1 -- 172.21.15.134:0/2442761209 learned_addr learned my addr 172.21.15.134:0/2442761209 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-22T13:15:50.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.150+0000 7ffa89976700 1 -- 172.21.15.134:0/2442761209 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa84199230 con 0x7ffa841015b0 2024-04-22T13:15:50.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.151+0000 7ffa89976700 1 --2- 172.21.15.134:0/2442761209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa841015b0 0x7ffa84198cf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ffa740056d0 tx=0x7ffa740057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:50.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.151+0000 7ffa7affd700 1 -- 172.21.15.134:0/2442761209 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa74026070 con 0x7ffa841015b0 2024-04-22T13:15:50.154 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.151+0000 7ffa8bbda700 1 -- 172.21.15.134:0/2442761209 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa84199520 con 0x7ffa841015b0 2024-04-22T13:15:50.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.151+0000 7ffa7affd700 1 -- 172.21.15.134:0/2442761209 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa7400bba0 con 0x7ffa841015b0 2024-04-22T13:15:50.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.151+0000 7ffa7affd700 1 -- 172.21.15.134:0/2442761209 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa74003920 con 0x7ffa841015b0 2024-04-22T13:15:50.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.151+0000 7ffa8bbda700 1 -- 172.21.15.134:0/2442761209 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa841999e0 con 0x7ffa841015b0 2024-04-22T13:15:50.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.152+0000 7ffa7affd700 1 -- 172.21.15.134:0/2442761209 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ffa74029070 con 0x7ffa841015b0 2024-04-22T13:15:50.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.152+0000 7ffa7affd700 1 --2- 172.21.15.134:0/2442761209 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ffa70039650 0x7ffa7003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:50.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.152+0000 7ffa7affd700 1 -- 172.21.15.134:0/2442761209 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ffa7401e070 con 0x7ffa841015b0 2024-04-22T13:15:50.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.152+0000 7ffa8bbda700 1 -- 172.21.15.134:0/2442761209 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa84102a90 con 0x7ffa841015b0 2024-04-22T13:15:50.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.152+0000 7ffa89175700 1 --2- 172.21.15.134:0/2442761209 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ffa70039650 0x7ffa7003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:50.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.153+0000 7ffa89175700 1 --2- 172.21.15.134:0/2442761209 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ffa70039650 0x7ffa7003bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ffa80006fd0 tx=0x7ffa80006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:50.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.158+0000 7ffa7affd700 1 -- 172.21.15.134:0/2442761209 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa74013070 con 0x7ffa841015b0 2024-04-22T13:15:50.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:49 smithi012 ceph-mon[28265]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:15:50.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.454+0000 7ffa8bbda700 1 -- 172.21.15.134:0/2442761209 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ffa840640a0 con 0x7ffa841015b0 2024-04-22T13:15:50.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.455+0000 7ffa7affd700 1 -- 172.21.15.134:0/2442761209 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7ffa74030020 con 0x7ffa841015b0 2024-04-22T13:15:50.453 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-22T13:15:50.453 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":1,"fsid":"0aede5ea-00aa-11ef-bc93-c7b262605968","modified":"2024-04-22T13:14:05.291228Z","created":"2024-04-22T13:14:05.291228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-22T13:15:50.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.457+0000 7ffa8bbda700 1 -- 172.21.15.134:0/2442761209 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ffa70039650 msgr2=0x7ffa7003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:50.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.457+0000 7ffa8bbda700 1 --2- 172.21.15.134:0/2442761209 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ffa70039650 0x7ffa7003bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ffa80006fd0 tx=0x7ffa80006e40 comp rx=0 tx=0).stop 2024-04-22T13:15:50.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.457+0000 7ffa8bbda700 1 -- 172.21.15.134:0/2442761209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa841015b0 msgr2=0x7ffa84198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:50.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.457+0000 7ffa8bbda700 1 --2- 172.21.15.134:0/2442761209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa841015b0 0x7ffa84198cf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ffa740056d0 tx=0x7ffa740057b0 comp rx=0 tx=0).stop 2024-04-22T13:15:50.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.457+0000 7ffa8bbda700 1 -- 172.21.15.134:0/2442761209 shutdown_connections 2024-04-22T13:15:50.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.457+0000 7ffa8bbda700 1 --2- 172.21.15.134:0/2442761209 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ffa70039650 0x7ffa7003bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:50.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.457+0000 7ffa8bbda700 1 --2- 172.21.15.134:0/2442761209 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa841015b0 0x7ffa84198cf0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:50.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.457+0000 7ffa8bbda700 1 -- 172.21.15.134:0/2442761209 >> 172.21.15.134:0/2442761209 conn(0x7ffa840fce60 msgr2=0x7ffa84102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:50.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.457+0000 7ffa8bbda700 1 -- 172.21.15.134:0/2442761209 shutdown_connections 2024-04-22T13:15:50.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:50.457+0000 7ffa8bbda700 1 -- 172.21.15.134:0/2442761209 wait complete. 2024-04-22T13:15:50.457 INFO:teuthology.orchestra.run.smithi134.stderr:dumped monmap epoch 1 2024-04-22T13:15:52.336 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-22T13:15:52.337 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T13:15:52.581 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:15:56.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:15:55 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:56.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:55 smithi012 ceph-mon[28265]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:15:56.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:55 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:15:56.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:55 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:15:56.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:55 smithi012 ceph-mon[28265]: mon.a calling monitor election 2024-04-22T13:15:56.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:55 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:15:56.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:55 smithi012 ceph-mon[28265]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:15:56.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:55 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:15:56.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:55 smithi012 ceph-mon[28265]: mon.c calling monitor election 2024-04-22T13:15:56.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:55 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:15:56.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:55 smithi012 ceph-mon[28265]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:15:56.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:55 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:15:56.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:55 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:15:56.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:55 smithi012 ceph-mon[28265]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-22T13:15:56.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:55 smithi012 ceph-mon[28265]: monmap epoch 2 2024-04-22T13:15:56.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:55 smithi012 ceph-mon[28265]: fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:15:56.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:55 smithi012 ceph-mon[28265]: last_changed 2024-04-22T13:15:50.841600+0000 2024-04-22T13:15:56.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:55 smithi012 ceph-mon[28265]: created 2024-04-22T13:14:05.291228+0000 2024-04-22T13:15:56.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:55 smithi012 ceph-mon[28265]: min_mon_release 19 (squid) 2024-04-22T13:15:56.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:55 smithi012 ceph-mon[28265]: election_strategy: 1 2024-04-22T13:15:56.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:55 smithi012 ceph-mon[28265]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-04-22T13:15:56.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:55 smithi012 ceph-mon[28265]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-04-22T13:15:56.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:55 smithi012 ceph-mon[28265]: fsmap 2024-04-22T13:15:56.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:55 smithi012 ceph-mon[28265]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T13:15:56.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:55 smithi012 ceph-mon[28265]: mgrmap e13: x(active, since 51s) 2024-04-22T13:15:56.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:55 smithi012 ceph-mon[28265]: overall HEALTH_OK 2024-04-22T13:15:56.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:55 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:56.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:55 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:56.960 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.962+0000 7fd40a04d700 1 -- 172.21.15.134:0/3803712035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3ec0035c0 msgr2=0x7fd3ec005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:56.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.962+0000 7fd40a04d700 1 --2- 172.21.15.134:0/3803712035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3ec0035c0 0x7fd3ec005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd404078b70 tx=0x7fd3f4009c60 comp rx=0 tx=0).stop 2024-04-22T13:15:56.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.962+0000 7fd40a04d700 1 -- 172.21.15.134:0/3803712035 shutdown_connections 2024-04-22T13:15:56.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.962+0000 7fd40a04d700 1 --2- 172.21.15.134:0/3803712035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd3ec0035c0 0x7fd3ec005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:56.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.962+0000 7fd40a04d700 1 -- 172.21.15.134:0/3803712035 >> 172.21.15.134:0/3803712035 conn(0x7fd404077860 msgr2=0x7fd404077c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:56.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.962+0000 7fd40a04d700 1 -- 172.21.15.134:0/3803712035 shutdown_connections 2024-04-22T13:15:56.961 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.962+0000 7fd40a04d700 1 -- 172.21.15.134:0/3803712035 wait complete. 2024-04-22T13:15:56.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.963+0000 7fd40a04d700 1 Processor -- start 2024-04-22T13:15:56.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.963+0000 7fd40a04d700 1 -- start start 2024-04-22T13:15:56.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.964+0000 7fd40a04d700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3ec0035c0 0x7fd404198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:56.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.964+0000 7fd40a04d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd404198560 0x7fd40419c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:56.963 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.964+0000 7fd40a04d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd4041082b0 con 0x7fd404198560 2024-04-22T13:15:56.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.964+0000 7fd40a04d700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd404108130 con 0x7fd3ec0035c0 2024-04-22T13:15:56.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.964+0000 7fd4037fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3ec0035c0 0x7fd404198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:56.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.964+0000 7fd4037fe700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3ec0035c0 0x7fd404198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:39520/0 (socket says 172.21.15.134:39520) 2024-04-22T13:15:56.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.964+0000 7fd4037fe700 1 -- 172.21.15.134:0/3647410636 learned_addr learned my addr 172.21.15.134:0/3647410636 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-22T13:15:56.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.964+0000 7fd4037fe700 1 -- 172.21.15.134:0/3647410636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3ec0035c0 msgr2=0x7fd404198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-22T13:15:56.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.964+0000 7fd4037fe700 1 -- 172.21.15.134:0/3647410636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3ec0035c0 msgr2=0x7fd404198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-22T13:15:56.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.964+0000 7fd4037fe700 1 --2- 172.21.15.134:0/3647410636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3ec0035c0 0x7fd404198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T13:15:56.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.964+0000 7fd4037fe700 1 --2- 172.21.15.134:0/3647410636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3ec0035c0 0x7fd404198020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T13:15:56.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.964+0000 7fd402ffd700 1 --2- 172.21.15.134:0/3647410636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd404198560 0x7fd40419c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:56.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.964+0000 7fd402ffd700 1 -- 172.21.15.134:0/3647410636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3ec0035c0 msgr2=0x7fd404198020 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T13:15:56.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.964+0000 7fd402ffd700 1 --2- 172.21.15.134:0/3647410636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3ec0035c0 0x7fd404198020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:56.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.964+0000 7fd402ffd700 1 -- 172.21.15.134:0/3647410636 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd404198ba0 con 0x7fd404198560 2024-04-22T13:15:56.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.965+0000 7fd402ffd700 1 --2- 172.21.15.134:0/3647410636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd404198560 0x7fd40419c8f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd3f800b4d0 tx=0x7fd3f800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:56.964 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.965+0000 7fd400ff9700 1 -- 172.21.15.134:0/3647410636 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd3f8011630 con 0x7fd404198560 2024-04-22T13:15:56.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.965+0000 7fd400ff9700 1 -- 172.21.15.134:0/3647410636 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3f8011c80 con 0x7fd404198560 2024-04-22T13:15:56.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.965+0000 7fd40a04d700 1 -- 172.21.15.134:0/3647410636 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd40419ce30 con 0x7fd404198560 2024-04-22T13:15:56.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.965+0000 7fd400ff9700 1 -- 172.21.15.134:0/3647410636 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd3f8010da0 con 0x7fd404198560 2024-04-22T13:15:56.965 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.965+0000 7fd40a04d700 1 -- 172.21.15.134:0/3647410636 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd40419d390 con 0x7fd404198560 2024-04-22T13:15:56.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.966+0000 7fd400ff9700 1 -- 172.21.15.134:0/3647410636 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd3f8018020 con 0x7fd404198560 2024-04-22T13:15:56.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.966+0000 7fd40a04d700 1 -- 172.21.15.134:0/3647410636 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd404102340 con 0x7fd404198560 2024-04-22T13:15:56.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.966+0000 7fd400ff9700 1 --2- 172.21.15.134:0/3647410636 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fd3f0039460 0x7fd3f003b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:15:56.966 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.966+0000 7fd400ff9700 1 -- 172.21.15.134:0/3647410636 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd3f804c1f0 con 0x7fd404198560 2024-04-22T13:15:56.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.966+0000 7fd4037fe700 1 --2- 172.21.15.134:0/3647410636 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fd3f0039460 0x7fd3f003b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:15:56.967 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.967+0000 7fd4037fe700 1 --2- 172.21.15.134:0/3647410636 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fd3f0039460 0x7fd3f003b920 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4009120 tx=0x7fd3f4009050 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:15:56.970 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:56.972+0000 7fd400ff9700 1 -- 172.21.15.134:0/3647410636 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd3f801f020 con 0x7fd404198560 2024-04-22T13:15:57.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:15:56 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:57.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:15:56 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T13:15:57.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:15:56 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:15:57.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:15:56 smithi134 ceph-mon[31338]: Deploying daemon mon.b on smithi111 2024-04-22T13:15:57.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:15:56 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:15:57.251 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:57.253+0000 7fd40a04d700 1 -- 172.21.15.134:0/3647410636 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd40419d6a0 con 0x7fd404198560 2024-04-22T13:15:57.252 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:57.254+0000 7fd400ff9700 1 -- 172.21.15.134:0/3647410636 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fd3f8018570 con 0x7fd404198560 2024-04-22T13:15:57.253 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-22T13:15:57.253 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":2,"fsid":"0aede5ea-00aa-11ef-bc93-c7b262605968","modified":"2024-04-22T13:15:50.841600Z","created":"2024-04-22T13:14:05.291228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-22T13:15:57.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:57.256+0000 7fd40a04d700 1 -- 172.21.15.134:0/3647410636 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fd3f0039460 msgr2=0x7fd3f003b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:57.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:57.256+0000 7fd40a04d700 1 --2- 172.21.15.134:0/3647410636 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fd3f0039460 0x7fd3f003b920 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4009120 tx=0x7fd3f4009050 comp rx=0 tx=0).stop 2024-04-22T13:15:57.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:57.256+0000 7fd40a04d700 1 -- 172.21.15.134:0/3647410636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd404198560 msgr2=0x7fd40419c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:15:57.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:57.256+0000 7fd40a04d700 1 --2- 172.21.15.134:0/3647410636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd404198560 0x7fd40419c8f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd3f800b4d0 tx=0x7fd3f800b890 comp rx=0 tx=0).stop 2024-04-22T13:15:57.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:57.256+0000 7fd40a04d700 1 -- 172.21.15.134:0/3647410636 shutdown_connections 2024-04-22T13:15:57.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:57.256+0000 7fd40a04d700 1 --2- 172.21.15.134:0/3647410636 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fd3f0039460 0x7fd3f003b920 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:57.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:57.256+0000 7fd40a04d700 1 --2- 172.21.15.134:0/3647410636 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd3ec0035c0 0x7fd404198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:57.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:57.256+0000 7fd40a04d700 1 --2- 172.21.15.134:0/3647410636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd404198560 0x7fd40419c8f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:15:57.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:57.256+0000 7fd40a04d700 1 -- 172.21.15.134:0/3647410636 >> 172.21.15.134:0/3647410636 conn(0x7fd404077860 msgr2=0x7fd4040ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:15:57.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:57.256+0000 7fd40a04d700 1 -- 172.21.15.134:0/3647410636 shutdown_connections 2024-04-22T13:15:57.255 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:15:57.256+0000 7fd40a04d700 1 -- 172.21.15.134:0/3647410636 wait complete. 2024-04-22T13:15:57.256 INFO:teuthology.orchestra.run.smithi134.stderr:dumped monmap epoch 2 2024-04-22T13:15:57.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:56 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:15:57.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:56 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T13:15:57.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:56 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:15:57.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:56 smithi012 ceph-mon[28265]: Deploying daemon mon.b on smithi111 2024-04-22T13:15:57.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:56 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:15:58.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:15:57 smithi134 ceph-mon[31338]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:15:58.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:15:57 smithi134 ceph-mon[31338]: from='client.? 172.21.15.134:0/3647410636' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T13:15:58.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:57 smithi012 ceph-mon[28265]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:15:58.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:15:57 smithi012 ceph-mon[28265]: from='client.? 172.21.15.134:0/3647410636' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T13:15:58.908 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-22T13:15:58.908 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-22T13:15:59.151 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:15:59.844 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:15:59 smithi111 systemd[1]: Started Ceph mon.b for 0aede5ea-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:16:04.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.978+0000 7f5ec3254700 1 -- 172.21.15.134:0/4119001447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ea40035c0 msgr2=0x7f5ea4005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:16:04.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.978+0000 7f5ec3254700 1 --2- 172.21.15.134:0/4119001447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ea40035c0 0x7f5ea4005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc06a480 tx=0x7f5eb4009e80 comp rx=0 tx=0).stop 2024-04-22T13:16:04.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.978+0000 7f5ec3254700 1 -- 172.21.15.134:0/4119001447 shutdown_connections 2024-04-22T13:16:04.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.978+0000 7f5ec3254700 1 --2- 172.21.15.134:0/4119001447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ea40035c0 0x7f5ea4005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:04.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.978+0000 7f5ec3254700 1 -- 172.21.15.134:0/4119001447 >> 172.21.15.134:0/4119001447 conn(0x7f5ebc0fcb50 msgr2=0x7f5ebc0fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:16:04.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.978+0000 7f5ec3254700 1 -- 172.21.15.134:0/4119001447 shutdown_connections 2024-04-22T13:16:04.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.978+0000 7f5ec3254700 1 -- 172.21.15.134:0/4119001447 wait complete. 2024-04-22T13:16:04.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.979+0000 7f5ec3254700 1 Processor -- start 2024-04-22T13:16:04.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.979+0000 7f5ec3254700 1 -- start start 2024-04-22T13:16:04.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.979+0000 7f5ec3254700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ea40035c0 0x7f5ebc0fd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:16:04.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.979+0000 7f5ec3254700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ebc0fd540 0x7f5ebc0fd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:16:04.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.979+0000 7f5ec3254700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ebc1a4190 0x7f5ebc1a6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:16:04.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.979+0000 7f5ec3254700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc109380 con 0x7f5ebc0fd540 2024-04-22T13:16:04.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.979+0000 7f5ec3254700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc109200 con 0x7f5ea40035c0 2024-04-22T13:16:04.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.979+0000 7f5ec3254700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc109500 con 0x7f5ebc1a4190 2024-04-22T13:16:04.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.979+0000 7f5ec17f1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ebc1a4190 0x7f5ebc1a6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:16:04.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.979+0000 7f5ec0ff0700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ea40035c0 0x7f5ebc0fd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:16:04.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.979+0000 7f5ec17f1700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ebc1a4190 0x7f5ebc1a6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:52162/0 (socket says 172.21.15.134:52162) 2024-04-22T13:16:04.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.980+0000 7f5ec17f1700 1 -- 172.21.15.134:0/1468553707 learned_addr learned my addr 172.21.15.134:0/1468553707 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-22T13:16:04.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.980+0000 7f5ebbfff700 1 --2- 172.21.15.134:0/1468553707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ebc0fd540 0x7f5ebc0fd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:16:04.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.980+0000 7f5ec17f1700 1 -- 172.21.15.134:0/1468553707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ebc1a4190 msgr2=0x7f5ebc1a6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-22T13:16:04.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.980+0000 7f5ec0ff0700 1 -- 172.21.15.134:0/1468553707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ea40035c0 msgr2=0x7f5ebc0fd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-22T13:16:04.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.980+0000 7f5ec17f1700 1 -- 172.21.15.134:0/1468553707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ebc1a4190 msgr2=0x7f5ebc1a6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-22T13:16:04.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.980+0000 7f5ec0ff0700 1 -- 172.21.15.134:0/1468553707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ea40035c0 msgr2=0x7f5ebc0fd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-22T13:16:04.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.980+0000 7f5ec0ff0700 1 --2- 172.21.15.134:0/1468553707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ea40035c0 0x7f5ebc0fd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T13:16:04.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.980+0000 7f5ec17f1700 1 --2- 172.21.15.134:0/1468553707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ebc1a4190 0x7f5ebc1a6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-22T13:16:04.982 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.980+0000 7f5ec0ff0700 1 --2- 172.21.15.134:0/1468553707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ea40035c0 0x7f5ebc0fd000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T13:16:04.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.980+0000 7f5ec17f1700 1 --2- 172.21.15.134:0/1468553707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ebc1a4190 0x7f5ebc1a6580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T13:16:04.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.980+0000 7f5ebbfff700 1 -- 172.21.15.134:0/1468553707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ebc1a4190 msgr2=0x7f5ebc1a6580 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T13:16:04.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.980+0000 7f5ebbfff700 1 --2- 172.21.15.134:0/1468553707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ebc1a4190 0x7f5ebc1a6580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:04.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.980+0000 7f5ebbfff700 1 -- 172.21.15.134:0/1468553707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ea40035c0 msgr2=0x7f5ebc0fd000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T13:16:04.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.980+0000 7f5ebbfff700 1 --2- 172.21.15.134:0/1468553707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ea40035c0 0x7f5ebc0fd000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:04.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.980+0000 7f5ebbfff700 1 -- 172.21.15.134:0/1468553707 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ebc1a6cb0 con 0x7f5ebc0fd540 2024-04-22T13:16:04.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.980+0000 7f5ebbfff700 1 --2- 172.21.15.134:0/1468553707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ebc0fd540 0x7f5ebc0fd9c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000b550 tx=0x7f5eb000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:16:04.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.980+0000 7f5eb9ffb700 1 -- 172.21.15.134:0/1468553707 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0004230 con 0x7f5ebc0fd540 2024-04-22T13:16:04.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.980+0000 7f5eb9ffb700 1 -- 172.21.15.134:0/1468553707 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5eb00043d0 con 0x7f5ebc0fd540 2024-04-22T13:16:04.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.980+0000 7f5eb9ffb700 1 -- 172.21.15.134:0/1468553707 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0011b90 con 0x7f5ebc0fd540 2024-04-22T13:16:04.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.980+0000 7f5ec3254700 1 -- 172.21.15.134:0/1468553707 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ebc1a6fa0 con 0x7f5ebc0fd540 2024-04-22T13:16:04.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.981+0000 7f5ec3254700 1 -- 172.21.15.134:0/1468553707 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ebc1a74d0 con 0x7f5ebc0fd540 2024-04-22T13:16:04.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.982+0000 7f5eb9ffb700 1 -- 172.21.15.134:0/1468553707 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5eb001a400 con 0x7f5ebc0fd540 2024-04-22T13:16:04.983 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.982+0000 7f5eb9ffb700 1 --2- 172.21.15.134:0/1468553707 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f5ea8039ae0 0x7f5ea803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:16:04.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.982+0000 7f5eb9ffb700 1 -- 172.21.15.134:0/1468553707 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5eb004c360 con 0x7f5ebc0fd540 2024-04-22T13:16:04.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.982+0000 7f5ec3254700 1 -- 172.21.15.134:0/1468553707 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ebc0640a0 con 0x7f5ebc0fd540 2024-04-22T13:16:04.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.982+0000 7f5ec0ff0700 1 --2- 172.21.15.134:0/1468553707 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f5ea8039ae0 0x7f5ea803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:16:04.984 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.983+0000 7f5ec0ff0700 1 --2- 172.21.15.134:0/1468553707 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f5ea8039ae0 0x7f5ea803bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009a30 tx=0x7f5eb400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:16:04.987 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:04.988+0000 7f5eb9ffb700 1 -- 172.21.15.134:0/1468553707 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5eb00151c0 con 0x7f5ebc0fd540 2024-04-22T13:16:05.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:04 smithi012 ceph-mon[28265]: mon.c calling monitor election 2024-04-22T13:16:05.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:16:05.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:04 smithi012 ceph-mon[28265]: mon.a calling monitor election 2024-04-22T13:16:05.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:16:05.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:16:05.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:04 smithi012 ceph-mon[28265]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:05.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:16:05.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:16:05.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:04 smithi012 ceph-mon[28265]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:05.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:16:05.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:16:05.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:16:05.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:04 smithi012 ceph-mon[28265]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-22T13:16:05.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:04 smithi012 ceph-mon[28265]: monmap epoch 3 2024-04-22T13:16:05.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:04 smithi012 ceph-mon[28265]: fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:16:05.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:04 smithi012 ceph-mon[28265]: last_changed 2024-04-22T13:15:59.711221+0000 2024-04-22T13:16:05.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:04 smithi012 ceph-mon[28265]: created 2024-04-22T13:14:05.291228+0000 2024-04-22T13:16:05.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:04 smithi012 ceph-mon[28265]: min_mon_release 19 (squid) 2024-04-22T13:16:05.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:04 smithi012 ceph-mon[28265]: election_strategy: 1 2024-04-22T13:16:05.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:04 smithi012 ceph-mon[28265]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-04-22T13:16:05.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:04 smithi012 ceph-mon[28265]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-04-22T13:16:05.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:04 smithi012 ceph-mon[28265]: 2: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.b 2024-04-22T13:16:05.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:04 smithi012 ceph-mon[28265]: fsmap 2024-04-22T13:16:05.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:04 smithi012 ceph-mon[28265]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T13:16:05.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:04 smithi012 ceph-mon[28265]: mgrmap e13: x(active, since 60s) 2024-04-22T13:16:05.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:04 smithi012 ceph-mon[28265]: overall HEALTH_OK 2024-04-22T13:16:05.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:05.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:05.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:05.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:05.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:04 smithi134 ceph-mon[31338]: mon.c calling monitor election 2024-04-22T13:16:05.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:04 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:16:05.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:04 smithi134 ceph-mon[31338]: mon.a calling monitor election 2024-04-22T13:16:05.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:04 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:16:05.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:04 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:16:05.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:04 smithi134 ceph-mon[31338]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:05.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:04 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:16:05.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:04 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:16:05.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:04 smithi134 ceph-mon[31338]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:05.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:04 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:16:05.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:04 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:16:05.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:04 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:16:05.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:04 smithi134 ceph-mon[31338]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-22T13:16:05.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:04 smithi134 ceph-mon[31338]: monmap epoch 3 2024-04-22T13:16:05.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:04 smithi134 ceph-mon[31338]: fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:16:05.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:04 smithi134 ceph-mon[31338]: last_changed 2024-04-22T13:15:59.711221+0000 2024-04-22T13:16:05.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:04 smithi134 ceph-mon[31338]: created 2024-04-22T13:14:05.291228+0000 2024-04-22T13:16:05.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:04 smithi134 ceph-mon[31338]: min_mon_release 19 (squid) 2024-04-22T13:16:05.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:04 smithi134 ceph-mon[31338]: election_strategy: 1 2024-04-22T13:16:05.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:04 smithi134 ceph-mon[31338]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-04-22T13:16:05.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:04 smithi134 ceph-mon[31338]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-04-22T13:16:05.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:04 smithi134 ceph-mon[31338]: 2: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.b 2024-04-22T13:16:05.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:04 smithi134 ceph-mon[31338]: fsmap 2024-04-22T13:16:05.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:04 smithi134 ceph-mon[31338]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T13:16:05.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:04 smithi134 ceph-mon[31338]: mgrmap e13: x(active, since 60s) 2024-04-22T13:16:05.182 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:04 smithi134 ceph-mon[31338]: overall HEALTH_OK 2024-04-22T13:16:05.182 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:04 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:05.182 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:04 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:05.182 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:04 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:05.182 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:04 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:05.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:05.282+0000 7f5ec3254700 1 -- 172.21.15.134:0/1468553707 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f5ebc1a7890 con 0x7f5ebc0fd540 2024-04-22T13:16:05.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:05.283+0000 7f5eb9ffb700 1 -- 172.21.15.134:0/1468553707 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f5eb0018510 con 0x7f5ebc0fd540 2024-04-22T13:16:05.282 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-04-22T13:16:05.282 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":3,"fsid":"0aede5ea-00aa-11ef-bc93-c7b262605968","modified":"2024-04-22T13:15:59.711221Z","created":"2024-04-22T13:14:05.291228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:3300","nonce":0},{"type":"v1","addr":"172.21.15.12:6789","nonce":0}]},"addr":"172.21.15.12:6789/0","public_addr":"172.21.15.12:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:3300","nonce":0},{"type":"v1","addr":"172.21.15.134:6789","nonce":0}]},"addr":"172.21.15.134:6789/0","public_addr":"172.21.15.134:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:3300","nonce":0},{"type":"v1","addr":"172.21.15.111:6789","nonce":0}]},"addr":"172.21.15.111:6789/0","public_addr":"172.21.15.111:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-22T13:16:05.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:05.285+0000 7f5ec3254700 1 -- 172.21.15.134:0/1468553707 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f5ea8039ae0 msgr2=0x7f5ea803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:16:05.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:05.285+0000 7f5ec3254700 1 --2- 172.21.15.134:0/1468553707 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f5ea8039ae0 0x7f5ea803bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009a30 tx=0x7f5eb400b040 comp rx=0 tx=0).stop 2024-04-22T13:16:05.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:05.285+0000 7f5ec3254700 1 -- 172.21.15.134:0/1468553707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ebc0fd540 msgr2=0x7f5ebc0fd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:16:05.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:05.285+0000 7f5ec3254700 1 --2- 172.21.15.134:0/1468553707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ebc0fd540 0x7f5ebc0fd9c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000b550 tx=0x7f5eb000b910 comp rx=0 tx=0).stop 2024-04-22T13:16:05.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:05.285+0000 7f5ec3254700 1 -- 172.21.15.134:0/1468553707 shutdown_connections 2024-04-22T13:16:05.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:05.286+0000 7f5ec3254700 1 --2- 172.21.15.134:0/1468553707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ea40035c0 0x7f5ebc0fd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:05.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:05.286+0000 7f5ec3254700 1 --2- 172.21.15.134:0/1468553707 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f5ea8039ae0 0x7f5ea803bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:05.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:05.286+0000 7f5ec3254700 1 --2- 172.21.15.134:0/1468553707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ebc1a4190 0x7f5ebc1a6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:05.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:05.286+0000 7f5ec3254700 1 --2- 172.21.15.134:0/1468553707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ebc0fd540 0x7f5ebc0fd9c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:05.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:05.286+0000 7f5ec3254700 1 -- 172.21.15.134:0/1468553707 >> 172.21.15.134:0/1468553707 conn(0x7f5ebc0fcb50 msgr2=0x7f5ebc0ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:16:05.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:05.286+0000 7f5ec3254700 1 -- 172.21.15.134:0/1468553707 shutdown_connections 2024-04-22T13:16:05.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:05.286+0000 7f5ec3254700 1 -- 172.21.15.134:0/1468553707 wait complete. 2024-04-22T13:16:05.286 INFO:teuthology.orchestra.run.smithi134.stderr:dumped monmap epoch 3 2024-04-22T13:16:05.902 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:05 smithi134 ceph-mon[31338]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:05.902 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:05 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:16:05.902 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:05 smithi134 ceph-mon[31338]: from='client.? 172.21.15.134:0/1468553707' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T13:16:05.902 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:05 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:16:05.935 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-22T13:16:05.935 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-22T13:16:06.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:05 smithi012 ceph-mon[28265]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:06.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:05 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:16:06.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:05 smithi012 ceph-mon[28265]: from='client.? 172.21.15.134:0/1468553707' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T13:16:06.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:05 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:16:06.193 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:16:06.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: mon.c calling monitor election 2024-04-22T13:16:06.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:16:06.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: mon.a calling monitor election 2024-04-22T13:16:06.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:16:06.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:16:06.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:06.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:16:06.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:16:06.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:06.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:16:06.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:16:06.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:16:06.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-22T13:16:06.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: monmap epoch 3 2024-04-22T13:16:06.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:16:06.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: last_changed 2024-04-22T13:15:59.711221+0000 2024-04-22T13:16:06.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: created 2024-04-22T13:14:05.291228+0000 2024-04-22T13:16:06.981 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: min_mon_release 19 (squid) 2024-04-22T13:16:06.981 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: election_strategy: 1 2024-04-22T13:16:06.981 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-04-22T13:16:06.981 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-04-22T13:16:06.981 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: 2: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.b 2024-04-22T13:16:06.981 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: fsmap 2024-04-22T13:16:06.981 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T13:16:06.981 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: mgrmap e13: x(active, since 60s) 2024-04-22T13:16:06.981 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: overall HEALTH_OK 2024-04-22T13:16:06.981 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:06.981 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:06.981 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:06.981 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:06.982 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:06.982 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:16:06.982 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: from='client.? 172.21.15.134:0/1468553707' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-22T13:16:06.982 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:06 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:16:07.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.822+0000 7f81a532c700 1 -- 172.21.15.12:0/3865864035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81a01015b0 msgr2=0x7f81a0101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:16:07.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.822+0000 7f81a532c700 1 --2- 172.21.15.12:0/3865864035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81a01015b0 0x7f81a0101990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8188009910 tx=0x7f8188009c20 comp rx=0 tx=0).stop 2024-04-22T13:16:07.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.822+0000 7f81a532c700 1 -- 172.21.15.12:0/3865864035 shutdown_connections 2024-04-22T13:16:07.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.822+0000 7f81a532c700 1 --2- 172.21.15.12:0/3865864035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81a01015b0 0x7f81a0101990 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:07.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.822+0000 7f81a532c700 1 -- 172.21.15.12:0/3865864035 >> 172.21.15.12:0/3865864035 conn(0x7f81a00fce40 msgr2=0x7f81a00ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:16:07.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.822+0000 7f81a532c700 1 -- 172.21.15.12:0/3865864035 shutdown_connections 2024-04-22T13:16:07.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.822+0000 7f81a532c700 1 -- 172.21.15.12:0/3865864035 wait complete. 2024-04-22T13:16:07.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.823+0000 7f81a532c700 1 Processor -- start 2024-04-22T13:16:07.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.823+0000 7f81a532c700 1 -- start start 2024-04-22T13:16:07.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.824+0000 7f81a532c700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81a01015b0 0x7f81a01981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:16:07.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.824+0000 7f81a532c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81a0198700 0x7f81a019ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:16:07.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.824+0000 7f81a532c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81a019cfd0 0x7f81a019d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:16:07.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.824+0000 7f81a532c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f81a010b830 con 0x7f81a019cfd0 2024-04-22T13:16:07.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.824+0000 7f81a532c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f81a010b6b0 con 0x7f81a0198700 2024-04-22T13:16:07.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.824+0000 7f81a532c700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f81a010b9b0 con 0x7f81a01015b0 2024-04-22T13:16:07.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.824+0000 7f819f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81a019cfd0 0x7f81a019d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:16:07.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.824+0000 7f819f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81a019cfd0 0x7f81a019d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33356/0 (socket says 172.21.15.12:33356) 2024-04-22T13:16:07.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.824+0000 7f819f7fe700 1 -- 172.21.15.12:0/846723549 learned_addr learned my addr 172.21.15.12:0/846723549 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:16:07.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.824+0000 7f819effd700 1 --2- 172.21.15.12:0/846723549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81a01015b0 0x7f81a01981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:16:07.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.824+0000 7f819e7fc700 1 --2- 172.21.15.12:0/846723549 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81a0198700 0x7f81a019ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:16:07.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.824+0000 7f819f7fe700 1 -- 172.21.15.12:0/846723549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81a01015b0 msgr2=0x7f81a01981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:16:07.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.824+0000 7f819f7fe700 1 --2- 172.21.15.12:0/846723549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81a01015b0 0x7f81a01981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:07.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.824+0000 7f819f7fe700 1 -- 172.21.15.12:0/846723549 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81a0198700 msgr2=0x7f81a019ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:16:07.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.824+0000 7f819f7fe700 1 --2- 172.21.15.12:0/846723549 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81a0198700 0x7f81a019ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:07.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.824+0000 7f819f7fe700 1 -- 172.21.15.12:0/846723549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a01a4d40 con 0x7f81a019cfd0 2024-04-22T13:16:07.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.825+0000 7f819f7fe700 1 --2- 172.21.15.12:0/846723549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81a019cfd0 0x7f81a019d480 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f819800ba50 tx=0x7f819800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:16:07.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.825+0000 7f8197fff700 1 -- 172.21.15.12:0/846723549 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198019070 con 0x7f81a019cfd0 2024-04-22T13:16:07.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.825+0000 7f8197fff700 1 -- 172.21.15.12:0/846723549 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f819800cde0 con 0x7f81a019cfd0 2024-04-22T13:16:07.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.825+0000 7f81a532c700 1 -- 172.21.15.12:0/846723549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a01a4f70 con 0x7f81a019cfd0 2024-04-22T13:16:07.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.825+0000 7f8197fff700 1 -- 172.21.15.12:0/846723549 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198007350 con 0x7f81a019cfd0 2024-04-22T13:16:07.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.825+0000 7f81a532c700 1 -- 172.21.15.12:0/846723549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a01a53c0 con 0x7f81a019cfd0 2024-04-22T13:16:07.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.826+0000 7f8197fff700 1 -- 172.21.15.12:0/846723549 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f819801c070 con 0x7f81a019cfd0 2024-04-22T13:16:07.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.826+0000 7f81a532c700 1 -- 172.21.15.12:0/846723549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81a00640a0 con 0x7f81a019cfd0 2024-04-22T13:16:07.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.826+0000 7f8197fff700 1 --2- 172.21.15.12:0/846723549 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f818c039ae0 0x7f818c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:16:07.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.826+0000 7f8197fff700 1 -- 172.21.15.12:0/846723549 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f819804bdf0 con 0x7f81a019cfd0 2024-04-22T13:16:07.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.826+0000 7f819effd700 1 --2- 172.21.15.12:0/846723549 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f818c039ae0 0x7f818c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:16:07.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.827+0000 7f819effd700 1 --2- 172.21.15.12:0/846723549 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f818c039ae0 0x7f818c03bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8188006010 tx=0x7f818801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:16:07.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:07.832+0000 7f8197fff700 1 -- 172.21.15.12:0/846723549 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f819801e440 con 0x7f81a019cfd0 2024-04-22T13:16:07.930 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: mon.b calling monitor election 2024-04-22T13:16:07.930 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:07.930 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: Updating smithi012:/etc/ceph/ceph.conf 2024-04-22T13:16:07.930 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: Updating smithi111:/etc/ceph/ceph.conf 2024-04-22T13:16:07.930 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: Updating smithi134:/etc/ceph/ceph.conf 2024-04-22T13:16:07.930 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: mon.b calling monitor election 2024-04-22T13:16:07.930 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: mon.c calling monitor election 2024-04-22T13:16:07.930 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: mon.a calling monitor election 2024-04-22T13:16:07.930 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-22T13:16:07.930 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: monmap epoch 3 2024-04-22T13:16:07.930 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:16:07.931 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: last_changed 2024-04-22T13:15:59.711221+0000 2024-04-22T13:16:07.931 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: created 2024-04-22T13:14:05.291228+0000 2024-04-22T13:16:07.931 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: min_mon_release 19 (squid) 2024-04-22T13:16:07.931 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: election_strategy: 1 2024-04-22T13:16:07.931 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-04-22T13:16:07.931 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-04-22T13:16:07.931 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: 2: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.b 2024-04-22T13:16:07.931 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: fsmap 2024-04-22T13:16:07.931 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T13:16:07.931 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: mgrmap e13: x(active, since 62s) 2024-04-22T13:16:07.931 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: overall HEALTH_OK 2024-04-22T13:16:07.931 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: Updating smithi012:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:16:07.931 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: Updating smithi111:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:16:07.931 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: Updating smithi134:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:16:07.932 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.932 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.932 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.932 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.932 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.932 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.932 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.932 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.932 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.932 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.932 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.932 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T13:16:07.932 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T13:16:07.932 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:07 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:07.950 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: mon.b calling monitor election 2024-04-22T13:16:07.950 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:07.950 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: Updating smithi012:/etc/ceph/ceph.conf 2024-04-22T13:16:07.950 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: Updating smithi111:/etc/ceph/ceph.conf 2024-04-22T13:16:07.950 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: Updating smithi134:/etc/ceph/ceph.conf 2024-04-22T13:16:07.950 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: mon.b calling monitor election 2024-04-22T13:16:07.951 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: mon.c calling monitor election 2024-04-22T13:16:07.951 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: mon.a calling monitor election 2024-04-22T13:16:07.951 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-22T13:16:07.951 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: monmap epoch 3 2024-04-22T13:16:07.951 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:16:07.951 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: last_changed 2024-04-22T13:15:59.711221+0000 2024-04-22T13:16:07.951 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: created 2024-04-22T13:14:05.291228+0000 2024-04-22T13:16:07.951 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: min_mon_release 19 (squid) 2024-04-22T13:16:07.951 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: election_strategy: 1 2024-04-22T13:16:07.951 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-04-22T13:16:07.951 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-04-22T13:16:07.951 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: 2: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.b 2024-04-22T13:16:07.952 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: fsmap 2024-04-22T13:16:07.952 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T13:16:07.952 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: mgrmap e13: x(active, since 62s) 2024-04-22T13:16:07.952 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: overall HEALTH_OK 2024-04-22T13:16:07.952 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: Updating smithi012:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:16:07.952 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: Updating smithi111:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:16:07.952 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: Updating smithi134:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:16:07.952 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.953 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.953 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.953 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.953 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.953 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.953 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.953 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.953 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.953 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.953 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.953 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T13:16:07.954 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T13:16:07.954 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:07 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:07.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: mon.b calling monitor election 2024-04-22T13:16:07.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:07.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: Updating smithi012:/etc/ceph/ceph.conf 2024-04-22T13:16:07.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: Updating smithi111:/etc/ceph/ceph.conf 2024-04-22T13:16:07.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: Updating smithi134:/etc/ceph/ceph.conf 2024-04-22T13:16:07.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: mon.b calling monitor election 2024-04-22T13:16:07.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: mon.c calling monitor election 2024-04-22T13:16:07.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: mon.a calling monitor election 2024-04-22T13:16:07.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-22T13:16:07.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: monmap epoch 3 2024-04-22T13:16:07.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:16:07.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: last_changed 2024-04-22T13:15:59.711221+0000 2024-04-22T13:16:07.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: created 2024-04-22T13:14:05.291228+0000 2024-04-22T13:16:07.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: min_mon_release 19 (squid) 2024-04-22T13:16:07.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: election_strategy: 1 2024-04-22T13:16:07.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: 0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-04-22T13:16:07.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: 1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.c 2024-04-22T13:16:07.981 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: 2: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.b 2024-04-22T13:16:07.981 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: fsmap 2024-04-22T13:16:07.981 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: osdmap e4: 0 total, 0 up, 0 in 2024-04-22T13:16:07.981 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: mgrmap e13: x(active, since 62s) 2024-04-22T13:16:07.981 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: overall HEALTH_OK 2024-04-22T13:16:07.982 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: Updating smithi012:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:16:07.982 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: Updating smithi111:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:16:07.982 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: Updating smithi134:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:16:07.982 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.982 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.982 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.982 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.982 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.982 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.982 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.982 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.982 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.982 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.982 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:07.983 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T13:16:07.983 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T13:16:07.983 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:07 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:08.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:08.043+0000 7f81a532c700 1 -- 172.21.15.12:0/846723549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f81a01a5770 con 0x7f81a019cfd0 2024-04-22T13:16:08.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:08.043+0000 7f8197fff700 1 -- 172.21.15.12:0/846723549 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f8198012340 con 0x7f81a019cfd0 2024-04-22T13:16:08.043 INFO:teuthology.orchestra.run.smithi012.stdout:# minimal ceph.conf for 0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:16:08.043 INFO:teuthology.orchestra.run.smithi012.stdout:[global] 2024-04-22T13:16:08.043 INFO:teuthology.orchestra.run.smithi012.stdout: fsid = 0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:16:08.043 INFO:teuthology.orchestra.run.smithi012.stdout: mon_host = [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] 2024-04-22T13:16:08.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:08.045+0000 7f81a532c700 1 -- 172.21.15.12:0/846723549 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f818c039ae0 msgr2=0x7f818c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:16:08.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:08.045+0000 7f81a532c700 1 --2- 172.21.15.12:0/846723549 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f818c039ae0 0x7f818c03bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8188006010 tx=0x7f818801a040 comp rx=0 tx=0).stop 2024-04-22T13:16:08.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:08.045+0000 7f81a532c700 1 -- 172.21.15.12:0/846723549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81a019cfd0 msgr2=0x7f81a019d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:16:08.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:08.045+0000 7f81a532c700 1 --2- 172.21.15.12:0/846723549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81a019cfd0 0x7f81a019d480 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f819800ba50 tx=0x7f819800bd60 comp rx=0 tx=0).stop 2024-04-22T13:16:08.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:08.045+0000 7f81a532c700 1 -- 172.21.15.12:0/846723549 shutdown_connections 2024-04-22T13:16:08.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:08.045+0000 7f81a532c700 1 --2- 172.21.15.12:0/846723549 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f818c039ae0 0x7f818c03bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:08.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:08.045+0000 7f81a532c700 1 --2- 172.21.15.12:0/846723549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f81a019cfd0 0x7f81a019d480 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:08.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:08.045+0000 7f81a532c700 1 --2- 172.21.15.12:0/846723549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81a01015b0 0x7f81a01981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:08.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:08.045+0000 7f81a532c700 1 --2- 172.21.15.12:0/846723549 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81a0198700 0x7f81a019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:08.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:08.045+0000 7f81a532c700 1 -- 172.21.15.12:0/846723549 >> 172.21.15.12:0/846723549 conn(0x7f81a00fce40 msgr2=0x7f81a006a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:16:08.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:08.045+0000 7f81a532c700 1 -- 172.21.15.12:0/846723549 shutdown_connections 2024-04-22T13:16:08.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:08.045+0000 7f81a532c700 1 -- 172.21.15.12:0/846723549 wait complete. 2024-04-22T13:16:08.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:08 smithi111 ceph-mon[31361]: Reconfiguring mon.a (unknown last config time)... 2024-04-22T13:16:08.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:08 smithi111 ceph-mon[31361]: Reconfiguring daemon mon.a on smithi012 2024-04-22T13:16:08.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:08 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:16:08.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:08 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/846723549' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:09.023 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:08 smithi012 ceph-mon[28265]: Reconfiguring mon.a (unknown last config time)... 2024-04-22T13:16:09.023 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:08 smithi012 ceph-mon[28265]: Reconfiguring daemon mon.a on smithi012 2024-04-22T13:16:09.024 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:08 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:16:09.024 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:08 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/846723549' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:09.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:08 smithi134 ceph-mon[31338]: Reconfiguring mon.a (unknown last config time)... 2024-04-22T13:16:09.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:08 smithi134 ceph-mon[31338]: Reconfiguring daemon mon.a on smithi012 2024-04-22T13:16:09.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:08 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:16:09.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:08 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/846723549' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:09.494 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-22T13:16:09.494 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-22T13:16:09.494 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.conf 2024-04-22T13:16:09.520 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-22T13:16:09.520 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:16:09.586 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-22T13:16:09.586 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/ceph/ceph.conf 2024-04-22T13:16:09.612 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-22T13:16:09.612 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:16:09.675 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-22T13:16:09.675 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.conf 2024-04-22T13:16:09.707 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-22T13:16:09.707 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:16:09.776 INFO:tasks.cephadm:Adding mgr.x on smithi012 2024-04-22T13:16:09.776 INFO:tasks.cephadm:Adding mgr.y on smithi111 2024-04-22T13:16:09.776 INFO:tasks.cephadm:Adding mgr.z on smithi134 2024-04-22T13:16:09.777 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi012=x;smithi111=y;smithi134=z' 2024-04-22T13:16:10.051 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:16:10.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:10 smithi111 ceph-mon[31361]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:10.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:10 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:10.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:10 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:10.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:10 smithi111 ceph-mon[31361]: Reconfiguring mon.b (monmap changed)... 2024-04-22T13:16:10.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:10 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T13:16:10.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:10 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T13:16:10.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:10 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:10.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:10 smithi111 ceph-mon[31361]: Reconfiguring daemon mon.b on smithi111 2024-04-22T13:16:10.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:10 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:10.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:10 smithi012 ceph-mon[28265]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:10.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:10 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:10.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:10 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:10.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:10 smithi012 ceph-mon[28265]: Reconfiguring mon.b (monmap changed)... 2024-04-22T13:16:10.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:10 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T13:16:10.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:10 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T13:16:10.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:10 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:10.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:10 smithi012 ceph-mon[28265]: Reconfiguring daemon mon.b on smithi111 2024-04-22T13:16:10.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:10 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:10.399 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:10 smithi134 ceph-mon[31338]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:10.400 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:10 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:10.400 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:10 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:10.400 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:10 smithi134 ceph-mon[31338]: Reconfiguring mon.b (monmap changed)... 2024-04-22T13:16:10.400 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:10 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T13:16:10.400 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:10 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T13:16:10.400 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:10 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:10.400 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:10 smithi134 ceph-mon[31338]: Reconfiguring daemon mon.b on smithi111 2024-04-22T13:16:10.400 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:10 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:11.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.492+0000 7f116f38f700 1 -- 172.21.15.134:0/3155688742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1168103050 msgr2=0x7f1168105440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:16:11.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.492+0000 7f116f38f700 1 --2- 172.21.15.134:0/3155688742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1168103050 0x7f1168105440 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1158009910 tx=0x7f1158009c20 comp rx=0 tx=0).stop 2024-04-22T13:16:11.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.492+0000 7f116f38f700 1 -- 172.21.15.134:0/3155688742 shutdown_connections 2024-04-22T13:16:11.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.492+0000 7f116f38f700 1 --2- 172.21.15.134:0/3155688742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1168103050 0x7f1168105440 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:11.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.492+0000 7f116f38f700 1 -- 172.21.15.134:0/3155688742 >> 172.21.15.134:0/3155688742 conn(0x7f11680fca80 msgr2=0x7f11680feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:16:11.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.492+0000 7f116f38f700 1 -- 172.21.15.134:0/3155688742 shutdown_connections 2024-04-22T13:16:11.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.492+0000 7f116f38f700 1 -- 172.21.15.134:0/3155688742 wait complete. 2024-04-22T13:16:11.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.492+0000 7f116f38f700 1 Processor -- start 2024-04-22T13:16:11.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.492+0000 7f116f38f700 1 -- start start 2024-04-22T13:16:11.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.493+0000 7f116f38f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1168103050 0x7f1168197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:16:11.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.493+0000 7f116f38f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11681981a0 0x7f116819d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:16:11.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.493+0000 7f116f38f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1168198620 0x7f116819f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:16:11.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.493+0000 7f116f38f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f11681084a0 con 0x7f1168103050 2024-04-22T13:16:11.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.493+0000 7f116f38f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1168108320 con 0x7f1168198620 2024-04-22T13:16:11.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.493+0000 7f116f38f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1168108620 con 0x7f11681981a0 2024-04-22T13:16:11.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.493+0000 7f116c92a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11681981a0 0x7f116819d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:16:11.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.493+0000 7f116c92a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11681981a0 0x7f116819d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:52170/0 (socket says 172.21.15.134:52170) 2024-04-22T13:16:11.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.493+0000 7f116c92a700 1 -- 172.21.15.134:0/1890539921 learned_addr learned my addr 172.21.15.134:0/1890539921 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-22T13:16:11.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.493+0000 7f116d12b700 1 --2- 172.21.15.134:0/1890539921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1168103050 0x7f1168197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:16:11.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.493+0000 7f116d92c700 1 --2- 172.21.15.134:0/1890539921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1168198620 0x7f116819f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:16:11.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.493+0000 7f116c92a700 1 -- 172.21.15.134:0/1890539921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1168198620 msgr2=0x7f116819f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:16:11.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.493+0000 7f116c92a700 1 --2- 172.21.15.134:0/1890539921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1168198620 0x7f116819f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:11.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.493+0000 7f116c92a700 1 -- 172.21.15.134:0/1890539921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1168103050 msgr2=0x7f1168197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:16:11.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.493+0000 7f116c92a700 1 --2- 172.21.15.134:0/1890539921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1168103050 0x7f1168197c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:11.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.493+0000 7f116c92a700 1 -- 172.21.15.134:0/1890539921 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11681a00b0 con 0x7f11681981a0 2024-04-22T13:16:11.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.493+0000 7f116d92c700 1 --2- 172.21.15.134:0/1890539921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1168198620 0x7f116819f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:16:11.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.493+0000 7f116c92a700 1 --2- 172.21.15.134:0/1890539921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11681981a0 0x7f116819d200 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f116400b780 tx=0x7f116400ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:16:11.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.493+0000 7f115e7fc700 1 -- 172.21.15.134:0/1890539921 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164004230 con 0x7f11681981a0 2024-04-22T13:16:11.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.494+0000 7f116f38f700 1 -- 172.21.15.134:0/1890539921 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11681a03a0 con 0x7f11681981a0 2024-04-22T13:16:11.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.494+0000 7f115e7fc700 1 -- 172.21.15.134:0/1890539921 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11640043d0 con 0x7f11681981a0 2024-04-22T13:16:11.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.494+0000 7f115e7fc700 1 -- 172.21.15.134:0/1890539921 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164011b20 con 0x7f11681981a0 2024-04-22T13:16:11.494 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.494+0000 7f116f38f700 1 -- 172.21.15.134:0/1890539921 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11681a0960 con 0x7f11681981a0 2024-04-22T13:16:11.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.494+0000 7f115e7fc700 1 -- 172.21.15.134:0/1890539921 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1164011cc0 con 0x7f11681981a0 2024-04-22T13:16:11.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.495+0000 7f116f38f700 1 -- 172.21.15.134:0/1890539921 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11680640a0 con 0x7f11681981a0 2024-04-22T13:16:11.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.495+0000 7f115e7fc700 1 --2- 172.21.15.134:0/1890539921 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1154039ae0 0x7f115403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:16:11.495 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.495+0000 7f115e7fc700 1 -- 172.21.15.134:0/1890539921 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f116404c3d0 con 0x7f11681981a0 2024-04-22T13:16:11.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.495+0000 7f116d12b700 1 --2- 172.21.15.134:0/1890539921 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1154039ae0 0x7f115403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:16:11.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.495+0000 7f116d12b700 1 --2- 172.21.15.134:0/1890539921 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1154039ae0 0x7f115403bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1158005190 tx=0x7f115801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:16:11.498 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.500+0000 7f115e7fc700 1 -- 172.21.15.134:0/1890539921 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f11640204e0 con 0x7f11681981a0 2024-04-22T13:16:11.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.719+0000 7f116f38f700 1 -- 172.21.15.134:0/1890539921 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi012=x;smithi111=y;smithi134=z", "target": ["mon-mgr", ""]}) -- 0x7f1168062f10 con 0x7f1154039ae0 2024-04-22T13:16:11.723 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.725+0000 7f115e7fc700 1 -- 172.21.15.134:0/1890539921 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1168062f10 con 0x7f1154039ae0 2024-04-22T13:16:11.724 INFO:teuthology.orchestra.run.smithi134.stdout:Scheduled mgr update... 2024-04-22T13:16:11.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.728+0000 7f116f38f700 1 -- 172.21.15.134:0/1890539921 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1154039ae0 msgr2=0x7f115403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:16:11.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.728+0000 7f116f38f700 1 --2- 172.21.15.134:0/1890539921 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1154039ae0 0x7f115403bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1158005190 tx=0x7f115801a040 comp rx=0 tx=0).stop 2024-04-22T13:16:11.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.728+0000 7f116f38f700 1 -- 172.21.15.134:0/1890539921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11681981a0 msgr2=0x7f116819d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:16:11.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.728+0000 7f116f38f700 1 --2- 172.21.15.134:0/1890539921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11681981a0 0x7f116819d200 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f116400b780 tx=0x7f116400ba90 comp rx=0 tx=0).stop 2024-04-22T13:16:11.726 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.728+0000 7f116f38f700 1 -- 172.21.15.134:0/1890539921 shutdown_connections 2024-04-22T13:16:11.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.728+0000 7f116f38f700 1 --2- 172.21.15.134:0/1890539921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1168198620 0x7f116819f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:11.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.728+0000 7f116f38f700 1 --2- 172.21.15.134:0/1890539921 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1154039ae0 0x7f115403bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:11.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.728+0000 7f116f38f700 1 --2- 172.21.15.134:0/1890539921 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1168103050 0x7f1168197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:11.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.728+0000 7f116f38f700 1 --2- 172.21.15.134:0/1890539921 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11681981a0 0x7f116819d200 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:11.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.728+0000 7f116f38f700 1 -- 172.21.15.134:0/1890539921 >> 172.21.15.134:0/1890539921 conn(0x7f11680fca80 msgr2=0x7f11680fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:16:11.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.728+0000 7f116f38f700 1 -- 172.21.15.134:0/1890539921 shutdown_connections 2024-04-22T13:16:11.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:16:11.728+0000 7f116f38f700 1 -- 172.21.15.134:0/1890539921 wait complete. 2024-04-22T13:16:11.842 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:11 smithi134 ceph-mon[31338]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:11.842 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:11 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:11.842 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:11 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:11.842 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:11 smithi134 ceph-mon[31338]: Reconfiguring mon.c (monmap changed)... 2024-04-22T13:16:11.842 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:11 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T13:16:11.842 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:11 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T13:16:11.842 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:11 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:11.842 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:11 smithi134 ceph-mon[31338]: Reconfiguring daemon mon.c on smithi134 2024-04-22T13:16:11.842 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:11 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:11.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:11 smithi111 ceph-mon[31361]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:11.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:11 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:11.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:11 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:11.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:11 smithi111 ceph-mon[31361]: Reconfiguring mon.c (monmap changed)... 2024-04-22T13:16:11.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:11 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T13:16:11.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:11 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T13:16:11.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:11 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:11.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:11 smithi111 ceph-mon[31361]: Reconfiguring daemon mon.c on smithi134 2024-04-22T13:16:11.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:11 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:12.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:11 smithi012 ceph-mon[28265]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:12.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:11 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:12.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:11 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:12.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:11 smithi012 ceph-mon[28265]: Reconfiguring mon.c (monmap changed)... 2024-04-22T13:16:12.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:11 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-22T13:16:12.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:11 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-22T13:16:12.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:11 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:12.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:11 smithi012 ceph-mon[28265]: Reconfiguring daemon mon.c on smithi134 2024-04-22T13:16:12.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:11 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:12.582 DEBUG:teuthology.orchestra.run.smithi111:mgr.y> sudo journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mgr.y.service 2024-04-22T13:16:12.585 DEBUG:teuthology.orchestra.run.smithi134:mgr.z> sudo journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mgr.z.service 2024-04-22T13:16:12.587 INFO:tasks.cephadm:Deploying OSDs... 2024-04-22T13:16:12.588 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-22T13:16:12.588 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/scratch_devs of=/dev/stdout 2024-04-22T13:16:12.607 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-22T13:16:12.607 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_1 2024-04-22T13:16:12.667 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-22T13:16:12.667 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T13:16:12.667 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-22T13:16:12.667 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T13:16:12.667 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T13:16:12.667 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-04-22 13:15:17.310112537 +0000 2024-04-22T13:16:12.667 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-04-22 13:09:53.880566475 +0000 2024-04-22T13:16:12.667 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-04-22 13:09:53.880566475 +0000 2024-04-22T13:16:12.667 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-04-22 13:09:53.880566475 +0000 2024-04-22T13:16:12.667 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-22T13:16:12.735 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-04-22T13:16:12.735 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-04-22T13:16:12.736 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000219892 s, 2.3 MB/s 2024-04-22T13:16:12.737 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-22T13:16:12.798 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_2 2024-04-22T13:16:12.856 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-22T13:16:12.856 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T13:16:12.857 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-22T13:16:12.857 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T13:16:12.857 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T13:16:12.857 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-04-22 13:15:17.340112298 +0000 2024-04-22T13:16:12.857 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-04-22 13:09:54.235563654 +0000 2024-04-22T13:16:12.857 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-04-22 13:09:54.235563654 +0000 2024-04-22T13:16:12.857 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-04-22 13:09:54.235563654 +0000 2024-04-22T13:16:12.857 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-22T13:16:12.924 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-04-22T13:16:12.924 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-04-22T13:16:12.924 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.00023741 s, 2.2 MB/s 2024-04-22T13:16:12.925 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-22T13:16:12.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:12 smithi134 ceph-mon[31338]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi012=x;smithi111=y;smithi134=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:16:12.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:12 smithi134 ceph-mon[31338]: Saving service mgr spec with placement smithi012=x;smithi111=y;smithi134=z;count:3 2024-04-22T13:16:12.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:12 smithi111 ceph-mon[31361]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi012=x;smithi111=y;smithi134=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:16:12.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:12 smithi111 ceph-mon[31361]: Saving service mgr spec with placement smithi012=x;smithi111=y;smithi134=z;count:3 2024-04-22T13:16:12.985 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_3 2024-04-22T13:16:13.043 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-22T13:16:13.043 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T13:16:13.043 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-22T13:16:13.043 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T13:16:13.043 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T13:16:13.043 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-04-22 13:15:17.373112035 +0000 2024-04-22T13:16:13.043 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-04-22 13:09:54.567561016 +0000 2024-04-22T13:16:13.044 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-04-22 13:09:54.567561016 +0000 2024-04-22T13:16:13.044 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-04-22 13:09:54.567561016 +0000 2024-04-22T13:16:13.044 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-22T13:16:13.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:12 smithi012 ceph-mon[28265]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi012=x;smithi111=y;smithi134=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:16:13.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:12 smithi012 ceph-mon[28265]: Saving service mgr spec with placement smithi012=x;smithi111=y;smithi134=z;count:3 2024-04-22T13:16:13.109 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-04-22T13:16:13.109 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-04-22T13:16:13.109 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.00024711 s, 2.1 MB/s 2024-04-22T13:16:13.111 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-22T13:16:13.168 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_4 2024-04-22T13:16:13.224 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-22T13:16:13.224 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T13:16:13.225 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-22T13:16:13.225 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T13:16:13.225 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T13:16:13.225 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-04-22 13:15:17.401111811 +0000 2024-04-22T13:16:13.225 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-04-22 13:09:54.914558259 +0000 2024-04-22T13:16:13.225 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-04-22 13:09:54.914558259 +0000 2024-04-22T13:16:13.225 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-04-22 13:09:54.914558259 +0000 2024-04-22T13:16:13.225 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-22T13:16:13.288 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-04-22T13:16:13.288 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-04-22T13:16:13.288 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000218462 s, 2.3 MB/s 2024-04-22T13:16:13.289 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-22T13:16:13.347 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-04-22T13:16:13.347 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/scratch_devs of=/dev/stdout 2024-04-22T13:16:13.366 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-22T13:16:13.366 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_1 2024-04-22T13:16:13.423 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-22T13:16:13.423 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T13:16:13.424 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-22T13:16:13.424 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T13:16:13.424 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T13:16:13.424 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-04-22 13:15:59.173642605 +0000 2024-04-22T13:16:13.424 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-04-22 13:10:21.274086641 +0000 2024-04-22T13:16:13.424 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-04-22 13:10:21.274086641 +0000 2024-04-22T13:16:13.424 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-04-22 13:10:21.274086641 +0000 2024-04-22T13:16:13.424 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-22T13:16:13.490 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-04-22T13:16:13.490 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-04-22T13:16:13.490 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000284897 s, 1.8 MB/s 2024-04-22T13:16:13.491 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-22T13:16:13.551 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_2 2024-04-22T13:16:13.607 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-22T13:16:13.608 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T13:16:13.608 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-22T13:16:13.608 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T13:16:13.608 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T13:16:13.608 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-04-22 13:15:59.173642605 +0000 2024-04-22T13:16:13.608 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-04-22 13:10:21.603080259 +0000 2024-04-22T13:16:13.608 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-04-22 13:10:21.603080259 +0000 2024-04-22T13:16:13.608 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-04-22 13:10:21.603080259 +0000 2024-04-22T13:16:13.608 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-22T13:16:13.673 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-04-22T13:16:13.673 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-04-22T13:16:13.673 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000216606 s, 2.4 MB/s 2024-04-22T13:16:13.674 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-22T13:16:13.731 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_3 2024-04-22T13:16:13.787 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-22T13:16:13.787 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T13:16:13.787 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-22T13:16:13.787 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T13:16:13.787 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T13:16:13.788 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-04-22 13:15:59.174642586 +0000 2024-04-22T13:16:13.788 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-04-22 13:10:21.952073490 +0000 2024-04-22T13:16:13.788 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-04-22 13:10:21.952073490 +0000 2024-04-22T13:16:13.788 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-04-22 13:10:21.952073490 +0000 2024-04-22T13:16:13.788 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-22T13:16:13.849 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-04-22T13:16:13.850 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-04-22T13:16:13.850 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000173525 s, 3.0 MB/s 2024-04-22T13:16:13.851 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-22T13:16:13.907 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_4 2024-04-22T13:16:13.963 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-22T13:16:13.963 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T13:16:13.963 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-04-22T13:16:13.963 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T13:16:13.963 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T13:16:13.963 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-04-22 13:15:59.174642586 +0000 2024-04-22T13:16:13.963 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-04-22 13:10:22.284067051 +0000 2024-04-22T13:16:13.964 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-04-22 13:10:22.284067051 +0000 2024-04-22T13:16:13.964 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-04-22 13:10:22.284067051 +0000 2024-04-22T13:16:13.964 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-22T13:16:14.025 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-04-22T13:16:14.025 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-04-22T13:16:14.026 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.00024229 s, 2.1 MB/s 2024-04-22T13:16:14.027 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-22T13:16:14.083 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-04-22T13:16:14.083 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/scratch_devs of=/dev/stdout 2024-04-22T13:16:14.100 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-22T13:16:14.100 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_1 2024-04-22T13:16:14.155 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-22T13:16:14.156 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T13:16:14.156 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-22T13:16:14.156 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T13:16:14.156 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T13:16:14.156 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-04-22 13:15:49.889249127 +0000 2024-04-22T13:16:14.156 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-04-22 13:10:18.972297814 +0000 2024-04-22T13:16:14.156 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-04-22 13:10:18.972297814 +0000 2024-04-22T13:16:14.156 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-04-22 13:10:18.972297814 +0000 2024-04-22T13:16:14.156 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-22T13:16:14.219 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-04-22T13:16:14.220 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-04-22T13:16:14.220 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000209638 s, 2.4 MB/s 2024-04-22T13:16:14.221 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-22T13:16:14.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:14 smithi111 ceph-mon[31361]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:14.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:14 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:14.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:14 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:14.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:14 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:16:14.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:14 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:14.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:14 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:16:14.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:14 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:14.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:14 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T13:16:14.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:14 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-22T13:16:14.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:14 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T13:16:14.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:14 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:14.277 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_2 2024-04-22T13:16:14.331 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:14 smithi134 ceph-mon[31338]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:14.331 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:14 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:14.331 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:14 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:14.331 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:14 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:16:14.331 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:14 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:14.331 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:14 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:16:14.331 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:14 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:14.331 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:14 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T13:16:14.331 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:14 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-22T13:16:14.331 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:14 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T13:16:14.331 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:14 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:14.332 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-22T13:16:14.332 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T13:16:14.332 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-22T13:16:14.332 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T13:16:14.333 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T13:16:14.333 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-04-22 13:15:49.889249127 +0000 2024-04-22T13:16:14.333 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-04-22 13:10:19.297289173 +0000 2024-04-22T13:16:14.333 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-04-22 13:10:19.297289173 +0000 2024-04-22T13:16:14.333 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-04-22 13:10:19.297289173 +0000 2024-04-22T13:16:14.333 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-22T13:16:14.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:14 smithi012 ceph-mon[28265]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:14.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:14 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:14.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:14 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:14.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:14 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:16:14.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:14 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:14.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:14 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:16:14.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:14 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:14.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:14 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T13:16:14.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:14 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-22T13:16:14.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:14 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T13:16:14.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:14 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:14.397 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-04-22T13:16:14.398 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-04-22T13:16:14.398 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000236665 s, 2.2 MB/s 2024-04-22T13:16:14.399 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-22T13:16:14.455 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_3 2024-04-22T13:16:14.510 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-22T13:16:14.510 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T13:16:14.510 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-22T13:16:14.510 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T13:16:14.510 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T13:16:14.510 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-04-22 13:15:49.889249127 +0000 2024-04-22T13:16:14.510 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-04-22 13:10:19.640280055 +0000 2024-04-22T13:16:14.510 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-04-22 13:10:19.640280055 +0000 2024-04-22T13:16:14.511 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-04-22 13:10:19.640280055 +0000 2024-04-22T13:16:14.511 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-22T13:16:14.573 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-04-22T13:16:14.573 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-04-22T13:16:14.573 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000211345 s, 2.4 MB/s 2024-04-22T13:16:14.574 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-22T13:16:14.630 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_4 2024-04-22T13:16:14.685 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-22T13:16:14.685 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T13:16:14.686 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-22T13:16:14.686 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T13:16:14.686 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T13:16:14.686 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-04-22 13:15:49.889249127 +0000 2024-04-22T13:16:14.686 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-04-22 13:10:19.973271202 +0000 2024-04-22T13:16:14.686 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-04-22 13:10:19.973271202 +0000 2024-04-22T13:16:14.686 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-04-22 13:10:19.973271202 +0000 2024-04-22T13:16:14.686 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-22T13:16:14.751 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-04-22T13:16:14.751 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-04-22T13:16:14.751 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000205079 s, 2.5 MB/s 2024-04-22T13:16:14.752 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-22T13:16:14.808 INFO:tasks.cephadm:Deploying osd.0 on smithi012 with /dev/vg_nvme/lv_4... 2024-04-22T13:16:14.809 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-22T13:16:15.059 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:16:15.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:15 smithi012 ceph-mon[28265]: Deploying daemon mgr.z on smithi134 2024-04-22T13:16:15.393 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:15 smithi134 ceph-mon[31338]: Deploying daemon mgr.z on smithi134 2024-04-22T13:16:15.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:15 smithi111 ceph-mon[31361]: Deploying daemon mgr.z on smithi134 2024-04-22T13:16:16.084 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:16:16 smithi134 podman[33188]: 2024-04-22 13:16:15.903586733 +0000 UTC m=+0.017825747 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:16:16.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:16 smithi012 ceph-mon[28265]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:16.355 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:16 smithi134 ceph-mon[31338]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:16.355 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:16:16 smithi134 podman[33188]: 2024-04-22 13:16:16.201991566 +0000 UTC m=+0.316230581 container init bed71e9456c44b97207188b4a9e0f9f0ef79b9efc40b917c76cd66a7bdfd979d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-22T13:16:16.355 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:16:16 smithi134 podman[33188]: 2024-04-22 13:16:16.204498968 +0000 UTC m=+0.318737983 container start bed71e9456c44b97207188b4a9e0f9f0ef79b9efc40b917c76cd66a7bdfd979d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5) 2024-04-22T13:16:16.355 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:16:16 smithi134 ceph-mgr[33207]: -- 172.21.15.134:0/3987547950 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559dced0e5a0 con 0x559dcece2800 2024-04-22T13:16:16.355 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:16:16 smithi134 bash[33188]: bed71e9456c44b97207188b4a9e0f9f0ef79b9efc40b917c76cd66a7bdfd979d 2024-04-22T13:16:16.355 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:16:16 smithi134 systemd[1]: Started Ceph mgr.z for 0aede5ea-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:16:16.355 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:16:16 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:16:16.356+0000 7fe393673200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T13:16:16.379 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:16 smithi111 ceph-mon[31361]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:16.639 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:16:16 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:16:16.538+0000 7fe393673200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:16:16.639 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:16:16 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:16:16.641+0000 7fe393673200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T13:16:16.927 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-22T13:16:16.927 INFO:teuthology.orchestra.run.smithi012.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-22T13:16:16.927 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10+0 records in 2024-04-22T13:16:16.927 INFO:teuthology.orchestra.run.smithi012.stderr:10+0 records out 2024-04-22T13:16:16.927 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112421 s, 933 MB/s 2024-04-22T13:16:16.927 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping successful for: 2024-04-22T13:16:17.337 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:16:16 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:16:16.949+0000 7fe393673200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T13:16:17.337 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:16:17 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:16:17.032+0000 7fe393673200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T13:16:17.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:17 smithi012 ceph-mon[28265]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:17.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:17 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:17.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:17 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:17.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:17 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:17.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:17 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T13:16:17.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:17 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-22T13:16:17.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:17 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T13:16:17.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:17 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:17.529 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:17 smithi012 ceph-mon[28265]: Deploying daemon mgr.y on smithi111 2024-04-22T13:16:17.559 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi012:vg_nvme/lv_4 2024-04-22T13:16:17.616 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:17 smithi111 ceph-mon[31361]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:17.617 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:17 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:17.617 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:17 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:17.617 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:17 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:17.617 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:17 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T13:16:17.617 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:17 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-22T13:16:17.617 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:17 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T13:16:17.617 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:17 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:17.617 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:17 smithi111 ceph-mon[31361]: Deploying daemon mgr.y on smithi111 2024-04-22T13:16:17.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:17 smithi134 ceph-mon[31338]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:17.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:17 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:17.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:17 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:17.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:17 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:17.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:17 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T13:16:17.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:17 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-22T13:16:17.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:17 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T13:16:17.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:17 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:17.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:17 smithi134 ceph-mon[31338]: Deploying daemon mgr.y on smithi111 2024-04-22T13:16:17.802 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:16:18.579 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:16:18 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:16:18.229+0000 7fe393673200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T13:16:18.579 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:16:18 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:16:18.398+0000 7fe393673200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T13:16:18.579 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:16:18 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:16:18.581+0000 7fe393673200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T13:16:18.866 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:16:18 smithi111 systemd[1]: Started Ceph mgr.y for 0aede5ea-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:16:19.158 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:16:18 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:16:18.863+0000 7fe50267f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T13:16:19.159 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:16:19 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:16:19.053+0000 7fe50267f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:16:19.159 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:16:19 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:16:19.159+0000 7fe50267f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T13:16:19.164 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:16:18 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:16:18.867+0000 7fe393673200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T13:16:19.164 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:16:18 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:16:18.959+0000 7fe393673200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T13:16:19.164 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:16:19 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:16:19.165+0000 7fe393673200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T13:16:19.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.393+0000 7fde1fde2700 1 -- 172.21.15.12:0/455466994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde18108790 msgr2=0x7fde1810ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:16:19.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.393+0000 7fde1fde2700 1 --2- 172.21.15.12:0/455466994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde18108790 0x7fde1810ac00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fde1000b0a0 tx=0x7fde1000b3b0 comp rx=0 tx=0).stop 2024-04-22T13:16:19.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.393+0000 7fde1fde2700 1 -- 172.21.15.12:0/455466994 shutdown_connections 2024-04-22T13:16:19.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.393+0000 7fde1fde2700 1 --2- 172.21.15.12:0/455466994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde18103530 0x7fde18105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:19.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.393+0000 7fde1fde2700 1 --2- 172.21.15.12:0/455466994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde18108790 0x7fde1810ac00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:19.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.393+0000 7fde1fde2700 1 --2- 172.21.15.12:0/455466994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde18105e60 0x7fde18108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:19.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.393+0000 7fde1fde2700 1 -- 172.21.15.12:0/455466994 >> 172.21.15.12:0/455466994 conn(0x7fde180fcf60 msgr2=0x7fde180ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:16:19.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.393+0000 7fde1fde2700 1 -- 172.21.15.12:0/455466994 shutdown_connections 2024-04-22T13:16:19.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.393+0000 7fde1fde2700 1 -- 172.21.15.12:0/455466994 wait complete. 2024-04-22T13:16:19.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.394+0000 7fde1fde2700 1 Processor -- start 2024-04-22T13:16:19.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.394+0000 7fde1fde2700 1 -- start start 2024-04-22T13:16:19.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.395+0000 7fde1fde2700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde18103530 0x7fde18198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:16:19.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.395+0000 7fde1fde2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde18105e60 0x7fde18199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:16:19.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.395+0000 7fde1fde2700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde18108790 0x7fde181a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:16:19.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.395+0000 7fde1fde2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fde1810e580 con 0x7fde18105e60 2024-04-22T13:16:19.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.395+0000 7fde1fde2700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fde1810e400 con 0x7fde18108790 2024-04-22T13:16:19.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.395+0000 7fde1fde2700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fde1810e700 con 0x7fde18103530 2024-04-22T13:16:19.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.395+0000 7fde1d37d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde18105e60 0x7fde18199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:16:19.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.395+0000 7fde1d37d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde18105e60 0x7fde18199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54086/0 (socket says 172.21.15.12:54086) 2024-04-22T13:16:19.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.395+0000 7fde1d37d700 1 -- 172.21.15.12:0/4187862337 learned_addr learned my addr 172.21.15.12:0/4187862337 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:16:19.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.395+0000 7fde1db7e700 1 --2- 172.21.15.12:0/4187862337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde18103530 0x7fde18198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:16:19.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.395+0000 7fde1e37f700 1 --2- 172.21.15.12:0/4187862337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde18108790 0x7fde181a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:16:19.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.395+0000 7fde1d37d700 1 -- 172.21.15.12:0/4187862337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde18103530 msgr2=0x7fde18198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:16:19.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.395+0000 7fde1d37d700 1 --2- 172.21.15.12:0/4187862337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde18103530 0x7fde18198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:19.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.395+0000 7fde1d37d700 1 -- 172.21.15.12:0/4187862337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde18108790 msgr2=0x7fde181a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:16:19.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.395+0000 7fde1d37d700 1 --2- 172.21.15.12:0/4187862337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde18108790 0x7fde181a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:19.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.395+0000 7fde1d37d700 1 -- 172.21.15.12:0/4187862337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde181a0830 con 0x7fde18105e60 2024-04-22T13:16:19.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.395+0000 7fde1db7e700 1 --2- 172.21.15.12:0/4187862337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde18103530 0x7fde18198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:16:19.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.395+0000 7fde1d37d700 1 --2- 172.21.15.12:0/4187862337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde18105e60 0x7fde18199110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fde0800e9f0 tx=0x7fde0800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:16:19.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.395+0000 7fde0effd700 1 -- 172.21.15.12:0/4187862337 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0800cbd0 con 0x7fde18105e60 2024-04-22T13:16:19.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.396+0000 7fde0effd700 1 -- 172.21.15.12:0/4187862337 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde0800ee90 con 0x7fde18105e60 2024-04-22T13:16:19.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.396+0000 7fde1fde2700 1 -- 172.21.15.12:0/4187862337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde181a0b20 con 0x7fde18105e60 2024-04-22T13:16:19.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.396+0000 7fde0effd700 1 -- 172.21.15.12:0/4187862337 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde08005590 con 0x7fde18105e60 2024-04-22T13:16:19.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.396+0000 7fde1fde2700 1 -- 172.21.15.12:0/4187862337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde181a1010 con 0x7fde18105e60 2024-04-22T13:16:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.397+0000 7fde0effd700 1 -- 172.21.15.12:0/4187862337 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fde08021440 con 0x7fde18105e60 2024-04-22T13:16:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.397+0000 7fde1fde2700 1 -- 172.21.15.12:0/4187862337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde18069730 con 0x7fde18105e60 2024-04-22T13:16:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.397+0000 7fde0effd700 1 --2- 172.21.15.12:0/4187862337 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fde04039890 0x7fde0403bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:16:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.397+0000 7fde0effd700 1 -- 172.21.15.12:0/4187862337 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fde0804cc60 con 0x7fde18105e60 2024-04-22T13:16:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.397+0000 7fde1db7e700 1 --2- 172.21.15.12:0/4187862337 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fde04039890 0x7fde0403bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:16:19.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.397+0000 7fde1db7e700 1 --2- 172.21.15.12:0/4187862337 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fde04039890 0x7fde0403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fde14011490 tx=0x7fde14009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:16:19.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.402+0000 7fde0effd700 1 -- 172.21.15.12:0/4187862337 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fde08016bd0 con 0x7fde18105e60 2024-04-22T13:16:19.429 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:16:19 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:16:19.361+0000 7fe393673200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:16:19.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:19.624+0000 7fde1fde2700 1 -- 172.21.15.12:0/4187862337 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fde18062e80 con 0x7fde04039890 2024-04-22T13:16:19.875 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:16:19 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:16:19.484+0000 7fe50267f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T13:16:19.875 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:16:19 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:16:19.580+0000 7fe50267f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T13:16:20.179 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:16:19 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:16:19.995+0000 7fe393673200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T13:16:20.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:19 smithi134 ceph-mon[31338]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:20.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:19 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:20.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:19 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:20.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:19 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:20.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:19 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:20.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:19 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:16:20.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:19 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:20.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:19 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:16:20.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:19 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:16:20.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:19 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:20.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:19 smithi111 ceph-mon[31361]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:20.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:19 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:20.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:19 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:20.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:19 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:20.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:19 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:20.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:19 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:16:20.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:19 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:20.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:19 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:16:20.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:19 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:16:20.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:19 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:20.252 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:19 smithi012 ceph-mon[28265]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:20.252 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:19 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:20.252 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:19 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:20.252 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:19 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:20.253 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:19 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:20.253 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:19 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:16:20.253 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:19 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:20.253 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:19 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:16:20.253 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:19 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:16:20.253 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:19 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:20.679 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:16:20 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:16:20.425+0000 7fe393673200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T13:16:20.679 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:16:20 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:16:20.521+0000 7fe393673200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T13:16:21.178 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:16:20 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:16:20.842+0000 7fe50267f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T13:16:21.178 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:16:21 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:16:21.010+0000 7fe50267f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T13:16:21.178 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:16:21 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:16:21.179+0000 7fe50267f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T13:16:21.179 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:20 smithi111 ceph-mon[31361]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:16:21.179 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:20 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:21.179 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:20 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:21.179 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:20 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:21.179 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:20 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:21.179 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:20 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:21.179 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:20 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:16:21.179 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:20 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:21.179 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:20 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T13:16:21.179 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:20 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T13:16:21.179 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:20 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:21.180 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:16:20 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:16:20.721+0000 7fe393673200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T13:16:21.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:20 smithi134 ceph-mon[31338]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:16:21.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:20 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:21.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:20 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:21.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:20 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:21.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:20 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:21.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:20 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:21.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:20 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:16:21.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:20 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:21.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:20 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T13:16:21.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:20 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T13:16:21.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:20 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:21.212 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:20 smithi012 ceph-mon[28265]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:16:21.212 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:20 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:21.212 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:20 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:21.212 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:20 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:21.212 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:20 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:21.212 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:20 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:21.212 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:20 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:16:21.212 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:20 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:21.212 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:20 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-22T13:16:21.212 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:20 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-22T13:16:21.212 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:20 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:21.571 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:16:21 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:16:21.245+0000 7fe393673200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T13:16:21.571 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:16:21 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:16:21.334+0000 7fe393673200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T13:16:21.728 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:16:21 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:16:21.462+0000 7fe50267f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T13:16:21.729 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:16:21 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:16:21.553+0000 7fe50267f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T13:16:21.881 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:16:21 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:16:21.572+0000 7fe393673200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T13:16:21.881 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:16:21 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:16:21.705+0000 7fe393673200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T13:16:21.881 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:16:21 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:16:21.799+0000 7fe393673200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T13:16:22.179 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:16:21 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:16:21.914+0000 7fe393673200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T13:16:22.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:21 smithi134 ceph-mon[31338]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:22.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:21 smithi134 ceph-mon[31338]: Reconfiguring mgr.x (unknown last config time)... 2024-04-22T13:16:22.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:21 smithi134 ceph-mon[31338]: Reconfiguring daemon mgr.x on smithi012 2024-04-22T13:16:22.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:21 smithi111 ceph-mon[31361]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:22.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:21 smithi111 ceph-mon[31361]: Reconfiguring mgr.x (unknown last config time)... 2024-04-22T13:16:22.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:21 smithi111 ceph-mon[31361]: Reconfiguring daemon mgr.x on smithi012 2024-04-22T13:16:22.229 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:16:21 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:16:21.756+0000 7fe50267f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T13:16:22.229 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:16:21 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:16:21.946+0000 7fe50267f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:16:22.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:21 smithi012 ceph-mon[28265]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:22.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:21 smithi012 ceph-mon[28265]: Reconfiguring mgr.x (unknown last config time)... 2024-04-22T13:16:22.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:21 smithi012 ceph-mon[28265]: Reconfiguring daemon mgr.x on smithi012 2024-04-22T13:16:22.679 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:16:22 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:16:22.218+0000 7fe393673200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T13:16:22.884 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:16:22 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:16:22.540+0000 7fe50267f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T13:16:23.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:22 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/496085087' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d9e2365-3a64-4b13-96fc-c217b1cee2ef"}]: dispatch 2024-04-22T13:16:23.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:22 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/496085087' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d9e2365-3a64-4b13-96fc-c217b1cee2ef"}]': finished 2024-04-22T13:16:23.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:22 smithi012 ceph-mon[28265]: osdmap e5: 1 total, 0 up, 1 in 2024-04-22T13:16:23.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:22 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:16:23.179 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:16:22 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:16:22.812+0000 7fe393673200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T13:16:23.179 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:16:23 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:16:23.023+0000 7fe393673200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T13:16:23.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:22 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/496085087' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d9e2365-3a64-4b13-96fc-c217b1cee2ef"}]: dispatch 2024-04-22T13:16:23.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:22 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/496085087' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d9e2365-3a64-4b13-96fc-c217b1cee2ef"}]': finished 2024-04-22T13:16:23.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:22 smithi134 ceph-mon[31338]: osdmap e5: 1 total, 0 up, 1 in 2024-04-22T13:16:23.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:22 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:16:23.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:22 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/496085087' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d9e2365-3a64-4b13-96fc-c217b1cee2ef"}]: dispatch 2024-04-22T13:16:23.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:22 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/496085087' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d9e2365-3a64-4b13-96fc-c217b1cee2ef"}]': finished 2024-04-22T13:16:23.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:22 smithi111 ceph-mon[31361]: osdmap e5: 1 total, 0 up, 1 in 2024-04-22T13:16:23.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:22 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:16:23.229 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:16:22 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:16:22.958+0000 7fe50267f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T13:16:23.229 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:16:23 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:16:23.053+0000 7fe50267f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T13:16:23.728 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:16:23 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:16:23.254+0000 7fe50267f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T13:16:23.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:23.892+0000 7fde0effd700 1 -- 172.21.15.12:0/4187862337 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7fde08081ff0 con 0x7fde18105e60 2024-04-22T13:16:24.104 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:16:23 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:16:23.771+0000 7fe50267f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T13:16:24.104 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:16:23 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:16:23.862+0000 7fe50267f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T13:16:24.104 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:23 smithi111 ceph-mon[31361]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:24.104 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:23 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/3681425329' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:16:24.104 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:23 smithi111 ceph-mon[31361]: Standby manager daemon z started 2024-04-22T13:16:24.105 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:23 smithi111 ceph-mon[31361]: from='mgr.? 172.21.15.134:0/1375867045' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-22T13:16:24.105 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:23 smithi111 ceph-mon[31361]: from='mgr.? 172.21.15.134:0/1375867045' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T13:16:24.105 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:23 smithi111 ceph-mon[31361]: from='mgr.? 172.21.15.134:0/1375867045' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-22T13:16:24.105 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:23 smithi111 ceph-mon[31361]: from='mgr.? 172.21.15.134:0/1375867045' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T13:16:24.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:23 smithi134 ceph-mon[31338]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:24.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:23 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/3681425329' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:16:24.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:23 smithi134 ceph-mon[31338]: Standby manager daemon z started 2024-04-22T13:16:24.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:23 smithi134 ceph-mon[31338]: from='mgr.? 172.21.15.134:0/1375867045' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-22T13:16:24.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:23 smithi134 ceph-mon[31338]: from='mgr.? 172.21.15.134:0/1375867045' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T13:16:24.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:23 smithi134 ceph-mon[31338]: from='mgr.? 172.21.15.134:0/1375867045' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-22T13:16:24.181 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:23 smithi134 ceph-mon[31338]: from='mgr.? 172.21.15.134:0/1375867045' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T13:16:24.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:23 smithi012 ceph-mon[28265]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:24.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:23 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3681425329' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:16:24.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:23 smithi012 ceph-mon[28265]: Standby manager daemon z started 2024-04-22T13:16:24.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:23 smithi012 ceph-mon[28265]: from='mgr.? 172.21.15.134:0/1375867045' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-22T13:16:24.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:23 smithi012 ceph-mon[28265]: from='mgr.? 172.21.15.134:0/1375867045' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T13:16:24.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:23 smithi012 ceph-mon[28265]: from='mgr.? 172.21.15.134:0/1375867045' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-22T13:16:24.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:23 smithi012 ceph-mon[28265]: from='mgr.? 172.21.15.134:0/1375867045' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T13:16:24.442 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:16:24 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:16:24.104+0000 7fe50267f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T13:16:24.443 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:16:24 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:16:24.236+0000 7fe50267f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T13:16:24.443 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:16:24 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:16:24.330+0000 7fe50267f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T13:16:24.443 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:16:24 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:16:24.444+0000 7fe50267f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T13:16:25.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:24 smithi134 ceph-mon[31338]: mgrmap e14: x(active, since 79s), standbys: z 2024-04-22T13:16:25.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:24 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-22T13:16:25.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:24 smithi111 ceph-mon[31361]: mgrmap e14: x(active, since 79s), standbys: z 2024-04-22T13:16:25.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:24 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-22T13:16:25.229 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:16:24 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:16:24.761+0000 7fe50267f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T13:16:25.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:24 smithi012 ceph-mon[28265]: mgrmap e14: x(active, since 79s), standbys: z 2024-04-22T13:16:25.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:24 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-22T13:16:25.728 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:16:25 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:16:25.354+0000 7fe50267f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T13:16:25.729 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:16:25 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:16:25.565+0000 7fe50267f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T13:16:25.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:25.918+0000 7fde0effd700 1 -- 172.21.15.12:0/4187862337 <== mon.0 v2:172.21.15.12:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fde0801f340 con 0x7fde18105e60 2024-04-22T13:16:26.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:25 smithi134 ceph-mon[31338]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:26.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:25 smithi134 ceph-mon[31338]: Standby manager daemon y started 2024-04-22T13:16:26.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:25 smithi134 ceph-mon[31338]: from='mgr.? 172.21.15.111:0/3595967305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-22T13:16:26.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:25 smithi134 ceph-mon[31338]: from='mgr.? 172.21.15.111:0/3595967305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T13:16:26.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:25 smithi134 ceph-mon[31338]: from='mgr.? 172.21.15.111:0/3595967305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-22T13:16:26.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:25 smithi134 ceph-mon[31338]: from='mgr.? 172.21.15.111:0/3595967305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T13:16:26.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:25 smithi111 ceph-mon[31361]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:26.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:25 smithi111 ceph-mon[31361]: Standby manager daemon y started 2024-04-22T13:16:26.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:25 smithi111 ceph-mon[31361]: from='mgr.? 172.21.15.111:0/3595967305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-22T13:16:26.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:25 smithi111 ceph-mon[31361]: from='mgr.? 172.21.15.111:0/3595967305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T13:16:26.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:25 smithi111 ceph-mon[31361]: from='mgr.? 172.21.15.111:0/3595967305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-22T13:16:26.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:25 smithi111 ceph-mon[31361]: from='mgr.? 172.21.15.111:0/3595967305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T13:16:26.276 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:25 smithi012 ceph-mon[28265]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:26.276 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:25 smithi012 ceph-mon[28265]: Standby manager daemon y started 2024-04-22T13:16:26.276 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:25 smithi012 ceph-mon[28265]: from='mgr.? 172.21.15.111:0/3595967305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-22T13:16:26.276 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:25 smithi012 ceph-mon[28265]: from='mgr.? 172.21.15.111:0/3595967305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T13:16:26.276 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:25 smithi012 ceph-mon[28265]: from='mgr.? 172.21.15.111:0/3595967305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-22T13:16:26.276 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:25 smithi012 ceph-mon[28265]: from='mgr.? 172.21.15.111:0/3595967305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T13:16:27.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:26 smithi134 ceph-mon[31338]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-04-22T13:16:27.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:26 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-22T13:16:27.208 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:26 smithi012 ceph-mon[28265]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-04-22T13:16:27.208 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:26 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-22T13:16:27.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:26 smithi111 ceph-mon[31361]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-04-22T13:16:27.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:26 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-22T13:16:27.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:27 smithi111 ceph-mon[31361]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:28.034 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:27 smithi012 ceph-mon[28265]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:28.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:27 smithi134 ceph-mon[31338]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:29.151 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:29 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:29.151 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:29 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:29.151 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:29 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:16:29.151 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:29 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:29.151 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:29 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:16:29.151 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:29 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:29.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:29 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:29.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:29 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:29.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:29 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:16:29.430 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:29 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:29.430 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:29 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:16:29.430 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:29 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:29.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:29 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:29.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:29 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:29.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:29 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:16:29.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:29 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:29.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:29 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:16:29.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:29 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:30.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:30 smithi134 ceph-mon[31338]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:30.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:30 smithi111 ceph-mon[31361]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:30.496 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:30 smithi012 ceph-mon[28265]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:31.374 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:31 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-22T13:16:31.375 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:31 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:31.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:31 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-22T13:16:31.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:31 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:31.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:31 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-22T13:16:31.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:31 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:32.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:32 smithi012 ceph-mon[28265]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:32.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:32 smithi012 ceph-mon[28265]: Deploying daemon osd.0 on smithi012 2024-04-22T13:16:32.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:32 smithi134 ceph-mon[31338]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:32.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:32 smithi134 ceph-mon[31338]: Deploying daemon osd.0 on smithi012 2024-04-22T13:16:32.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:32 smithi111 ceph-mon[31361]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:32.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:32 smithi111 ceph-mon[31361]: Deploying daemon osd.0 on smithi012 2024-04-22T13:16:34.241 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:34 smithi012 ceph-mon[28265]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:34.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:34 smithi134 ceph-mon[31338]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:34.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:34 smithi111 ceph-mon[31361]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:36.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:36 smithi134 ceph-mon[31338]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:36.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:36 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:36.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:36 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:36.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:36 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:16:36.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:36 smithi111 ceph-mon[31361]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:36.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:36 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:36.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:36 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:36.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:36 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:16:36.579 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:36 smithi012 ceph-mon[28265]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:36.579 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:36 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:36.579 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:36 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:36.579 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:36 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:16:38.247 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:38 smithi012 ceph-mon[28265]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:38.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:38 smithi111 ceph-mon[31361]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:38.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:38 smithi134 ceph-mon[31338]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:39.681 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 0 on host 'smithi012' 2024-04-22T13:16:39.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:39.683+0000 7fde0effd700 1 -- 172.21.15.12:0/4187862337 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fde18062e80 con 0x7fde04039890 2024-04-22T13:16:39.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:39.685+0000 7fde1fde2700 1 -- 172.21.15.12:0/4187862337 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fde04039890 msgr2=0x7fde0403bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:16:39.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:39.685+0000 7fde1fde2700 1 --2- 172.21.15.12:0/4187862337 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fde04039890 0x7fde0403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fde14011490 tx=0x7fde14009260 comp rx=0 tx=0).stop 2024-04-22T13:16:39.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:39.685+0000 7fde1fde2700 1 -- 172.21.15.12:0/4187862337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde18105e60 msgr2=0x7fde18199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:16:39.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:39.685+0000 7fde1fde2700 1 --2- 172.21.15.12:0/4187862337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde18105e60 0x7fde18199110 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fde0800e9f0 tx=0x7fde0800ed00 comp rx=0 tx=0).stop 2024-04-22T13:16:39.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:39.685+0000 7fde1fde2700 1 -- 172.21.15.12:0/4187862337 shutdown_connections 2024-04-22T13:16:39.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:39.685+0000 7fde1fde2700 1 --2- 172.21.15.12:0/4187862337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde18108790 0x7fde181a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:39.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:39.685+0000 7fde1fde2700 1 --2- 172.21.15.12:0/4187862337 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fde04039890 0x7fde0403bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:39.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:39.685+0000 7fde1fde2700 1 --2- 172.21.15.12:0/4187862337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde18103530 0x7fde18198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:39.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:39.685+0000 7fde1fde2700 1 --2- 172.21.15.12:0/4187862337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fde18105e60 0x7fde18199110 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:39.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:39.685+0000 7fde1fde2700 1 -- 172.21.15.12:0/4187862337 >> 172.21.15.12:0/4187862337 conn(0x7fde180fcf60 msgr2=0x7fde18103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:16:39.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:39.686+0000 7fde1fde2700 1 -- 172.21.15.12:0/4187862337 shutdown_connections 2024-04-22T13:16:39.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:16:39.686+0000 7fde1fde2700 1 -- 172.21.15.12:0/4187862337 wait complete. 2024-04-22T13:16:39.886 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:39 smithi012 ceph-mon[28265]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:39.886 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:39 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:39.887 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:39 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:39.887 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:39 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:39.887 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:39 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:16:39.887 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:39 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:39.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:39 smithi134 ceph-mon[31338]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:39.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:39 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:39.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:39 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:39.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:39 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:39.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:39 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:16:39.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:39 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:39.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:39 smithi111 ceph-mon[31361]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:39.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:39 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:39.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:39 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:39.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:39 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:39.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:39 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:16:39.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:39 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:40.289 DEBUG:teuthology.orchestra.run.smithi012:osd.0> sudo journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.0.service 2024-04-22T13:16:40.291 INFO:tasks.cephadm:Deploying osd.1 on smithi111 with /dev/vg_nvme/lv_4... 2024-04-22T13:16:40.292 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-22T13:16:40.533 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T13:16:40.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:40 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:40.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:40 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:40.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:40 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:16:40.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:40 smithi012 ceph-mon[28265]: from='osd.0 [v2:172.21.15.12:6802/1191350632,v1:172.21.15.12:6803/1191350632]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-22T13:16:40.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:40 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:40.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:40 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:40.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:40 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:16:40.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:40 smithi134 ceph-mon[31338]: from='osd.0 [v2:172.21.15.12:6802/1191350632,v1:172.21.15.12:6803/1191350632]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-22T13:16:40.962 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:40 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:40.962 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:40 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:40.962 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:40 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:16:40.962 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:40 smithi111 ceph-mon[31361]: from='osd.0 [v2:172.21.15.12:6802/1191350632,v1:172.21.15.12:6803/1191350632]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-22T13:16:41.835 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:41 smithi111 ceph-mon[31361]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:41.835 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:41 smithi111 ceph-mon[31361]: from='osd.0 [v2:172.21.15.12:6802/1191350632,v1:172.21.15.12:6803/1191350632]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-22T13:16:41.836 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:41 smithi111 ceph-mon[31361]: osdmap e6: 1 total, 0 up, 1 in 2024-04-22T13:16:41.836 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:41 smithi111 ceph-mon[31361]: from='osd.0 [v2:172.21.15.12:6802/1191350632,v1:172.21.15.12:6803/1191350632]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-22T13:16:41.836 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:41 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:16:41.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:41 smithi012 ceph-mon[28265]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:41.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:41 smithi012 ceph-mon[28265]: from='osd.0 [v2:172.21.15.12:6802/1191350632,v1:172.21.15.12:6803/1191350632]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-22T13:16:41.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:41 smithi012 ceph-mon[28265]: osdmap e6: 1 total, 0 up, 1 in 2024-04-22T13:16:41.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:41 smithi012 ceph-mon[28265]: from='osd.0 [v2:172.21.15.12:6802/1191350632,v1:172.21.15.12:6803/1191350632]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-22T13:16:41.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:41 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:16:41.839 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 22 13:16:41 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-0[42725]: 2024-04-22T13:16:41.686+0000 7f9654465700 -1 osd.0 0 waiting for initial osdmap 2024-04-22T13:16:41.839 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 22 13:16:41 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-0[42725]: 2024-04-22T13:16:41.690+0000 7f9650258700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T13:16:42.183 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:41 smithi134 ceph-mon[31338]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:42.183 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:41 smithi134 ceph-mon[31338]: from='osd.0 [v2:172.21.15.12:6802/1191350632,v1:172.21.15.12:6803/1191350632]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-22T13:16:42.183 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:41 smithi134 ceph-mon[31338]: osdmap e6: 1 total, 0 up, 1 in 2024-04-22T13:16:42.183 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:41 smithi134 ceph-mon[31338]: from='osd.0 [v2:172.21.15.12:6802/1191350632,v1:172.21.15.12:6803/1191350632]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-22T13:16:42.183 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:41 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:16:42.403 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-22T13:16:42.403 INFO:teuthology.orchestra.run.smithi111.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-22T13:16:42.404 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10+0 records in 2024-04-22T13:16:42.404 INFO:teuthology.orchestra.run.smithi111.stderr:10+0 records out 2024-04-22T13:16:42.404 INFO:teuthology.orchestra.run.smithi111.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111507 s, 940 MB/s 2024-04-22T13:16:42.404 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 2024-04-22T13:16:42.404 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping successful for: 2024-04-22T13:16:42.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:42 smithi111 ceph-mon[31361]: from='osd.0 [v2:172.21.15.12:6802/1191350632,v1:172.21.15.12:6803/1191350632]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-04-22T13:16:42.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:42 smithi111 ceph-mon[31361]: osdmap e7: 1 total, 0 up, 1 in 2024-04-22T13:16:42.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:42 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:16:42.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:42 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:16:43.045 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_4 2024-04-22T13:16:43.069 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:42 smithi012 ceph-mon[28265]: from='osd.0 [v2:172.21.15.12:6802/1191350632,v1:172.21.15.12:6803/1191350632]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-04-22T13:16:43.069 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:42 smithi012 ceph-mon[28265]: osdmap e7: 1 total, 0 up, 1 in 2024-04-22T13:16:43.069 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:42 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:16:43.069 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:42 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:16:43.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:42 smithi134 ceph-mon[31338]: from='osd.0 [v2:172.21.15.12:6802/1191350632,v1:172.21.15.12:6803/1191350632]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-04-22T13:16:43.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:42 smithi134 ceph-mon[31338]: osdmap e7: 1 total, 0 up, 1 in 2024-04-22T13:16:43.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:42 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:16:43.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:42 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:16:43.281 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T13:16:43.958 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:43 smithi111 ceph-mon[31361]: purged_snaps scrub starts 2024-04-22T13:16:43.959 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:43 smithi111 ceph-mon[31361]: purged_snaps scrub ok 2024-04-22T13:16:43.959 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:43 smithi111 ceph-mon[31361]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:43.959 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:43 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:16:43.959 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:43 smithi111 ceph-mon[31361]: osd.0 [v2:172.21.15.12:6802/1191350632,v1:172.21.15.12:6803/1191350632] boot 2024-04-22T13:16:43.959 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:43 smithi111 ceph-mon[31361]: osdmap e8: 1 total, 1 up, 1 in 2024-04-22T13:16:43.959 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:43 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:16:43.959 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:43 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:43.959 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:43 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:43.959 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:43 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:16:43.959 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:43 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:43.959 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:43 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:43.959 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:43 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:16:43.959 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:43 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:44.080 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:43 smithi134 ceph-mon[31338]: purged_snaps scrub starts 2024-04-22T13:16:44.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:43 smithi134 ceph-mon[31338]: purged_snaps scrub ok 2024-04-22T13:16:44.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:43 smithi134 ceph-mon[31338]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:44.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:43 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:16:44.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:43 smithi134 ceph-mon[31338]: osd.0 [v2:172.21.15.12:6802/1191350632,v1:172.21.15.12:6803/1191350632] boot 2024-04-22T13:16:44.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:43 smithi134 ceph-mon[31338]: osdmap e8: 1 total, 1 up, 1 in 2024-04-22T13:16:44.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:43 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:16:44.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:43 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:44.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:43 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:44.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:43 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:16:44.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:43 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:44.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:43 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:44.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:43 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:16:44.081 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:43 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:44.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:43 smithi012 ceph-mon[28265]: purged_snaps scrub starts 2024-04-22T13:16:44.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:43 smithi012 ceph-mon[28265]: purged_snaps scrub ok 2024-04-22T13:16:44.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:43 smithi012 ceph-mon[28265]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-22T13:16:44.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:16:44.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:43 smithi012 ceph-mon[28265]: osd.0 [v2:172.21.15.12:6802/1191350632,v1:172.21.15.12:6803/1191350632] boot 2024-04-22T13:16:44.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:43 smithi012 ceph-mon[28265]: osdmap e8: 1 total, 1 up, 1 in 2024-04-22T13:16:44.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:16:44.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:44.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:44.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:16:44.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:44.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:44.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:16:44.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:16:44.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.742+0000 7ff71ebd5700 1 -- 172.21.15.111:0/1824673387 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff718103540 msgr2=0x7ff718109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:16:44.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.742+0000 7ff71ebd5700 1 --2- 172.21.15.111:0/1824673387 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff718103540 0x7ff718109df0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff71000b0d0 tx=0x7ff71000b3e0 comp rx=0 tx=0).stop 2024-04-22T13:16:44.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.742+0000 7ff71ebd5700 1 -- 172.21.15.111:0/1824673387 shutdown_connections 2024-04-22T13:16:44.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.742+0000 7ff71ebd5700 1 --2- 172.21.15.111:0/1824673387 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff718103540 0x7ff718109df0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:44.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.742+0000 7ff71ebd5700 1 --2- 172.21.15.111:0/1824673387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7181019a0 0x7ff718101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:44.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.742+0000 7ff71ebd5700 1 --2- 172.21.15.111:0/1824673387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff718102ba0 0x7ff718103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:44.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.742+0000 7ff71ebd5700 1 -- 172.21.15.111:0/1824673387 >> 172.21.15.111:0/1824673387 conn(0x7ff7180fcf40 msgr2=0x7ff7180ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:16:44.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.743+0000 7ff71ebd5700 1 -- 172.21.15.111:0/1824673387 shutdown_connections 2024-04-22T13:16:44.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.743+0000 7ff71ebd5700 1 -- 172.21.15.111:0/1824673387 wait complete. 2024-04-22T13:16:44.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.743+0000 7ff71ebd5700 1 Processor -- start 2024-04-22T13:16:44.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.743+0000 7ff71ebd5700 1 -- start start 2024-04-22T13:16:44.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.743+0000 7ff71ebd5700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7181019a0 0x7ff71806bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:16:44.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.743+0000 7ff71ebd5700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff718102ba0 0x7ff71819d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:16:44.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.743+0000 7ff71ebd5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff718103540 0x7ff71819ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:16:44.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.743+0000 7ff71ebd5700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff71810d420 con 0x7ff718103540 2024-04-22T13:16:44.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.743+0000 7ff71ebd5700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff71810d2a0 con 0x7ff7181019a0 2024-04-22T13:16:44.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.743+0000 7ff71ebd5700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff71810d5a0 con 0x7ff718102ba0 2024-04-22T13:16:44.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.743+0000 7ff71c971700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7181019a0 0x7ff71806bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:16:44.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.743+0000 7ff71c971700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7181019a0 0x7ff71806bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33422/0 (socket says 172.21.15.111:33422) 2024-04-22T13:16:44.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.743+0000 7ff71c971700 1 -- 172.21.15.111:0/1994549177 learned_addr learned my addr 172.21.15.111:0/1994549177 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-22T13:16:44.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.743+0000 7ff717fff700 1 --2- 172.21.15.111:0/1994549177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff718102ba0 0x7ff71819d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:16:44.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.743+0000 7ff71d172700 1 --2- 172.21.15.111:0/1994549177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff718103540 0x7ff71819ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:16:44.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.744+0000 7ff71c971700 1 -- 172.21.15.111:0/1994549177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff718102ba0 msgr2=0x7ff71819d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:16:44.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.744+0000 7ff71c971700 1 --2- 172.21.15.111:0/1994549177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff718102ba0 0x7ff71819d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:44.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.744+0000 7ff71c971700 1 -- 172.21.15.111:0/1994549177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff718103540 msgr2=0x7ff71819ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:16:44.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.744+0000 7ff71c971700 1 --2- 172.21.15.111:0/1994549177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff718103540 0x7ff71819ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:16:44.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.744+0000 7ff71c971700 1 -- 172.21.15.111:0/1994549177 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7181a0840 con 0x7ff7181019a0 2024-04-22T13:16:44.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.744+0000 7ff71c971700 1 --2- 172.21.15.111:0/1994549177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7181019a0 0x7ff71806bd50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff70c00d980 tx=0x7ff70c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:16:44.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.744+0000 7ff715ffb700 1 -- 172.21.15.111:0/1994549177 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70c016070 con 0x7ff7181019a0 2024-04-22T13:16:44.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.744+0000 7ff715ffb700 1 -- 172.21.15.111:0/1994549177 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff70c00fd40 con 0x7ff7181019a0 2024-04-22T13:16:44.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.744+0000 7ff71ebd5700 1 -- 172.21.15.111:0/1994549177 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7181a0b30 con 0x7ff7181019a0 2024-04-22T13:16:44.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.744+0000 7ff715ffb700 1 -- 172.21.15.111:0/1994549177 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70c019360 con 0x7ff7181019a0 2024-04-22T13:16:44.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.744+0000 7ff71ebd5700 1 -- 172.21.15.111:0/1994549177 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7181a10c0 con 0x7ff7181019a0 2024-04-22T13:16:44.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.745+0000 7ff71ebd5700 1 -- 172.21.15.111:0/1994549177 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff718069730 con 0x7ff7181019a0 2024-04-22T13:16:44.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.746+0000 7ff715ffb700 1 -- 172.21.15.111:0/1994549177 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7ff70c019070 con 0x7ff7181019a0 2024-04-22T13:16:44.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.746+0000 7ff715ffb700 1 --2- 172.21.15.111:0/1994549177 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ff7080a3c30 0x7ff7080a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:16:44.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.747+0000 7ff715ffb700 1 -- 172.21.15.111:0/1994549177 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7ff70c0cdfd0 con 0x7ff7181019a0 2024-04-22T13:16:44.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.747+0000 7ff717fff700 1 --2- 172.21.15.111:0/1994549177 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ff7080a3c30 0x7ff7080a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:16:44.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.747+0000 7ff717fff700 1 --2- 172.21.15.111:0/1994549177 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ff7080a3c30 0x7ff7080a60f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff704005950 tx=0x7ff70400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:16:44.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.751+0000 7ff715ffb700 1 -- 172.21.15.111:0/1994549177 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff70c09c420 con 0x7ff7181019a0 2024-04-22T13:16:44.956 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:16:44.957+0000 7ff71ebd5700 1 -- 172.21.15.111:0/1994549177 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff718062e80 con 0x7ff7080a3c30 2024-04-22T13:16:44.956 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:44 smithi111 ceph-mon[31361]: Detected new or changed devices on smithi012 2024-04-22T13:16:44.956 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:44 smithi111 ceph-mon[31361]: Adjusting osd_memory_target on smithi012 to 17093M 2024-04-22T13:16:44.956 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:44 smithi111 ceph-mon[31361]: osdmap e9: 1 total, 1 up, 1 in 2024-04-22T13:16:45.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:44 smithi012 ceph-mon[28265]: Detected new or changed devices on smithi012 2024-04-22T13:16:45.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:44 smithi012 ceph-mon[28265]: Adjusting osd_memory_target on smithi012 to 17093M 2024-04-22T13:16:45.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:44 smithi012 ceph-mon[28265]: osdmap e9: 1 total, 1 up, 1 in 2024-04-22T13:16:45.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:44 smithi134 ceph-mon[31338]: Detected new or changed devices on smithi012 2024-04-22T13:16:45.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:44 smithi134 ceph-mon[31338]: Adjusting osd_memory_target on smithi012 to 17093M 2024-04-22T13:16:45.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:44 smithi134 ceph-mon[31338]: osdmap e9: 1 total, 1 up, 1 in 2024-04-22T13:16:45.799 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:45 smithi111 ceph-mon[31361]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:16:45.799 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:45 smithi111 ceph-mon[31361]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:16:45.799 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:45 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:16:45.799 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:45 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:16:45.799 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:45 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:46.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:45 smithi012 ceph-mon[28265]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:16:46.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:45 smithi012 ceph-mon[28265]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:16:46.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:45 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:16:46.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:45 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:16:46.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:45 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:46.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:45 smithi134 ceph-mon[31338]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:16:46.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:45 smithi134 ceph-mon[31338]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:16:46.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:45 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:16:46.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:45 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:16:46.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:45 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:47.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:47 smithi111 ceph-mon[31361]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:16:47.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:47 smithi111 ceph-mon[31361]: from='client.? 172.21.15.111:0/2855800314' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1499c783-2bd7-48ce-a444-34add42bcf31"}]: dispatch 2024-04-22T13:16:47.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:47 smithi111 ceph-mon[31361]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1499c783-2bd7-48ce-a444-34add42bcf31"}]: dispatch 2024-04-22T13:16:47.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:47 smithi111 ceph-mon[31361]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1499c783-2bd7-48ce-a444-34add42bcf31"}]': finished 2024-04-22T13:16:47.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:47 smithi111 ceph-mon[31361]: osdmap e10: 2 total, 1 up, 2 in 2024-04-22T13:16:47.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:47 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:16:48.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:47 smithi012 ceph-mon[28265]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:16:48.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:47 smithi012 ceph-mon[28265]: from='client.? 172.21.15.111:0/2855800314' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1499c783-2bd7-48ce-a444-34add42bcf31"}]: dispatch 2024-04-22T13:16:48.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:47 smithi012 ceph-mon[28265]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1499c783-2bd7-48ce-a444-34add42bcf31"}]: dispatch 2024-04-22T13:16:48.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:47 smithi012 ceph-mon[28265]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1499c783-2bd7-48ce-a444-34add42bcf31"}]': finished 2024-04-22T13:16:48.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:47 smithi012 ceph-mon[28265]: osdmap e10: 2 total, 1 up, 2 in 2024-04-22T13:16:48.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:47 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:16:48.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:47 smithi134 ceph-mon[31338]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:16:48.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:47 smithi134 ceph-mon[31338]: from='client.? 172.21.15.111:0/2855800314' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1499c783-2bd7-48ce-a444-34add42bcf31"}]: dispatch 2024-04-22T13:16:48.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:47 smithi134 ceph-mon[31338]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1499c783-2bd7-48ce-a444-34add42bcf31"}]: dispatch 2024-04-22T13:16:48.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:47 smithi134 ceph-mon[31338]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1499c783-2bd7-48ce-a444-34add42bcf31"}]': finished 2024-04-22T13:16:48.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:47 smithi134 ceph-mon[31338]: osdmap e10: 2 total, 1 up, 2 in 2024-04-22T13:16:48.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:47 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:16:48.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:48 smithi111 ceph-mon[31361]: from='client.? 172.21.15.111:0/549274515' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:16:48.990 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:48 smithi012 ceph-mon[28265]: from='client.? 172.21.15.111:0/549274515' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:16:49.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:48 smithi134 ceph-mon[31338]: from='client.? 172.21.15.111:0/549274515' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:16:49.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:49 smithi111 ceph-mon[31361]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:16:50.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:49 smithi012 ceph-mon[28265]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:16:50.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:49 smithi134 ceph-mon[31338]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:16:51.852 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:51 smithi111 ceph-mon[31361]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:16:52.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:51 smithi012 ceph-mon[28265]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:16:52.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:51 smithi134 ceph-mon[31338]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:16:53.777 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:53 smithi111 ceph-mon[31361]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:16:54.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:53 smithi012 ceph-mon[28265]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:16:54.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:53 smithi134 ceph-mon[31338]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:16:55.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:55 smithi111 ceph-mon[31361]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:16:55.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:55 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-22T13:16:55.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:55 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:55.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:55 smithi111 ceph-mon[31361]: Deploying daemon osd.1 on smithi111 2024-04-22T13:16:56.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:55 smithi012 ceph-mon[28265]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:16:56.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:55 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-22T13:16:56.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:55 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:56.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:55 smithi012 ceph-mon[28265]: Deploying daemon osd.1 on smithi111 2024-04-22T13:16:56.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:55 smithi134 ceph-mon[31338]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:16:56.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:55 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-22T13:16:56.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:55 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:16:56.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:55 smithi134 ceph-mon[31338]: Deploying daemon osd.1 on smithi111 2024-04-22T13:16:57.870 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:57 smithi111 ceph-mon[31361]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:16:58.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:57 smithi012 ceph-mon[28265]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:16:58.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:57 smithi134 ceph-mon[31338]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:17:00.006 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:16:59 smithi111 ceph-mon[31361]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:17:00.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:16:59 smithi012 ceph-mon[28265]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:17:00.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:16:59 smithi134 ceph-mon[31338]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:17:01.358 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:01 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:01.359 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:01 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:01.359 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:01 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:17:01.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:01 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:01.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:01 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:01.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:01 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:17:01.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:01 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:01.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:01 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:01.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:01 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:17:02.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:02 smithi111 ceph-mon[31361]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:17:02.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:02 smithi012 ceph-mon[28265]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:17:02.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:02 smithi134 ceph-mon[31338]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:17:03.423 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 1 on host 'smithi111' 2024-04-22T13:17:03.424 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:03.425+0000 7ff715ffb700 1 -- 172.21.15.111:0/1994549177 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff718062e80 con 0x7ff7080a3c30 2024-04-22T13:17:03.425 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:03.427+0000 7ff71ebd5700 1 -- 172.21.15.111:0/1994549177 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ff7080a3c30 msgr2=0x7ff7080a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:03.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:03.427+0000 7ff71ebd5700 1 --2- 172.21.15.111:0/1994549177 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ff7080a3c30 0x7ff7080a60f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff704005950 tx=0x7ff70400b410 comp rx=0 tx=0).stop 2024-04-22T13:17:03.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:03.427+0000 7ff71ebd5700 1 -- 172.21.15.111:0/1994549177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7181019a0 msgr2=0x7ff71806bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:03.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:03.427+0000 7ff71ebd5700 1 --2- 172.21.15.111:0/1994549177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7181019a0 0x7ff71806bd50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff70c00d980 tx=0x7ff70c00dc90 comp rx=0 tx=0).stop 2024-04-22T13:17:03.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:03.427+0000 7ff71ebd5700 1 -- 172.21.15.111:0/1994549177 shutdown_connections 2024-04-22T13:17:03.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:03.427+0000 7ff71ebd5700 1 --2- 172.21.15.111:0/1994549177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7181019a0 0x7ff71806bd50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:03.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:03.427+0000 7ff71ebd5700 1 --2- 172.21.15.111:0/1994549177 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ff7080a3c30 0x7ff7080a60f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:03.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:03.427+0000 7ff71ebd5700 1 --2- 172.21.15.111:0/1994549177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff718103540 0x7ff71819ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:03.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:03.427+0000 7ff71ebd5700 1 --2- 172.21.15.111:0/1994549177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff718102ba0 0x7ff71819d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:03.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:03.427+0000 7ff71ebd5700 1 -- 172.21.15.111:0/1994549177 >> 172.21.15.111:0/1994549177 conn(0x7ff7180fcf40 msgr2=0x7ff718105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:03.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:03.427+0000 7ff71ebd5700 1 -- 172.21.15.111:0/1994549177 shutdown_connections 2024-04-22T13:17:03.426 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:03.428+0000 7ff71ebd5700 1 -- 172.21.15.111:0/1994549177 wait complete. 2024-04-22T13:17:04.055 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:03 smithi111 ceph-mon[31361]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:17:04.056 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:03 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:04.056 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:03 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:04.056 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:03 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:04.056 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:03 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:17:04.056 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:03 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:04.056 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:03 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:04.056 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:03 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:04.056 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:03 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:17:04.092 DEBUG:teuthology.orchestra.run.smithi111:osd.1> sudo journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.1.service 2024-04-22T13:17:04.094 INFO:tasks.cephadm:Deploying osd.2 on smithi134 with /dev/vg_nvme/lv_4... 2024-04-22T13:17:04.094 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-22T13:17:04.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:03 smithi134 ceph-mon[31338]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:17:04.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:03 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:04.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:03 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:04.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:03 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:04.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:03 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:17:04.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:03 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:04.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:03 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:04.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:03 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:04.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:03 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:17:04.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:03 smithi012 ceph-mon[28265]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:17:04.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:03 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:04.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:03 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:04.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:03 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:04.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:03 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:17:04.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:03 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:04.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:03 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:04.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:03 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:04.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:03 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:17:04.340 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:17:05.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:04 smithi134 ceph-mon[31338]: from='osd.1 [v2:172.21.15.111:6800/417496465,v1:172.21.15.111:6801/417496465]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-22T13:17:05.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:04 smithi134 ceph-mon[31338]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-22T13:17:05.204 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:04 smithi111 ceph-mon[31361]: from='osd.1 [v2:172.21.15.111:6800/417496465,v1:172.21.15.111:6801/417496465]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-22T13:17:05.204 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:04 smithi111 ceph-mon[31361]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-22T13:17:05.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:04 smithi012 ceph-mon[28265]: from='osd.1 [v2:172.21.15.111:6800/417496465,v1:172.21.15.111:6801/417496465]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-22T13:17:05.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:04 smithi012 ceph-mon[28265]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-22T13:17:06.102 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-22T13:17:06.102 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-22T13:17:06.103 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2024-04-22T13:17:06.103 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2024-04-22T13:17:06.103 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112159 s, 935 MB/s 2024-04-22T13:17:06.103 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2024-04-22T13:17:06.166 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:05 smithi111 ceph-mon[31361]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:17:06.166 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:05 smithi111 ceph-mon[31361]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-22T13:17:06.166 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:05 smithi111 ceph-mon[31361]: from='osd.1 [v2:172.21.15.111:6800/417496465,v1:172.21.15.111:6801/417496465]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-22T13:17:06.166 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:05 smithi111 ceph-mon[31361]: osdmap e11: 2 total, 1 up, 2 in 2024-04-22T13:17:06.166 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:05 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:17:06.166 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:05 smithi111 ceph-mon[31361]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-22T13:17:06.167 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 22 13:17:05 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-1[35956]: 2024-04-22T13:17:05.896+0000 7f5b6fabc700 -1 osd.1 0 waiting for initial osdmap 2024-04-22T13:17:06.167 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 22 13:17:05 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-1[35956]: 2024-04-22T13:17:05.900+0000 7f5b6b8af700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T13:17:06.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:05 smithi134 ceph-mon[31338]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:17:06.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:05 smithi134 ceph-mon[31338]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-22T13:17:06.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:05 smithi134 ceph-mon[31338]: from='osd.1 [v2:172.21.15.111:6800/417496465,v1:172.21.15.111:6801/417496465]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-22T13:17:06.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:05 smithi134 ceph-mon[31338]: osdmap e11: 2 total, 1 up, 2 in 2024-04-22T13:17:06.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:05 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:17:06.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:05 smithi134 ceph-mon[31338]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-22T13:17:06.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:05 smithi012 ceph-mon[28265]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:17:06.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:05 smithi012 ceph-mon[28265]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-22T13:17:06.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:05 smithi012 ceph-mon[28265]: from='osd.1 [v2:172.21.15.111:6800/417496465,v1:172.21.15.111:6801/417496465]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-22T13:17:06.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:05 smithi012 ceph-mon[28265]: osdmap e11: 2 total, 1 up, 2 in 2024-04-22T13:17:06.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:05 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:17:06.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:05 smithi012 ceph-mon[28265]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-22T13:17:06.722 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_4 2024-04-22T13:17:06.964 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:17:07.086 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:06 smithi134 ceph-mon[31338]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-22T13:17:07.086 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:06 smithi134 ceph-mon[31338]: osdmap e12: 2 total, 1 up, 2 in 2024-04-22T13:17:07.086 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:06 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:17:07.086 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:06 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:17:07.086 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:06 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:07.086 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:06 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:07.086 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:06 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:17:07.087 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:06 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:07.087 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:06 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:07.087 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:06 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:17:07.087 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:06 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:07.087 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:06 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:17:07.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:06 smithi111 ceph-mon[31361]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-22T13:17:07.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:06 smithi111 ceph-mon[31361]: osdmap e12: 2 total, 1 up, 2 in 2024-04-22T13:17:07.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:06 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:17:07.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:06 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:17:07.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:06 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:07.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:06 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:07.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:06 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:17:07.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:06 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:07.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:06 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:07.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:06 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:17:07.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:06 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:07.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:06 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:17:07.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:06 smithi012 ceph-mon[28265]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-22T13:17:07.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:06 smithi012 ceph-mon[28265]: osdmap e12: 2 total, 1 up, 2 in 2024-04-22T13:17:07.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:06 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:17:07.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:06 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:17:07.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:06 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:07.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:06 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:07.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:06 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:17:07.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:06 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:07.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:06 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:07.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:06 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:17:07.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:06 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:07.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:06 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:17:08.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:07 smithi134 ceph-mon[31338]: purged_snaps scrub starts 2024-04-22T13:17:08.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:07 smithi134 ceph-mon[31338]: purged_snaps scrub ok 2024-04-22T13:17:08.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:07 smithi134 ceph-mon[31338]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:17:08.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:07 smithi134 ceph-mon[31338]: Detected new or changed devices on smithi111 2024-04-22T13:17:08.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:07 smithi134 ceph-mon[31338]: Adjusting osd_memory_target on smithi111 to 17088M 2024-04-22T13:17:08.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:07 smithi134 ceph-mon[31338]: osd.1 [v2:172.21.15.111:6800/417496465,v1:172.21.15.111:6801/417496465] boot 2024-04-22T13:17:08.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:07 smithi134 ceph-mon[31338]: osdmap e13: 2 total, 2 up, 2 in 2024-04-22T13:17:08.180 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:07 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:17:08.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:07 smithi111 ceph-mon[31361]: purged_snaps scrub starts 2024-04-22T13:17:08.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:07 smithi111 ceph-mon[31361]: purged_snaps scrub ok 2024-04-22T13:17:08.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:07 smithi111 ceph-mon[31361]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:17:08.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:07 smithi111 ceph-mon[31361]: Detected new or changed devices on smithi111 2024-04-22T13:17:08.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:07 smithi111 ceph-mon[31361]: Adjusting osd_memory_target on smithi111 to 17088M 2024-04-22T13:17:08.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:07 smithi111 ceph-mon[31361]: osd.1 [v2:172.21.15.111:6800/417496465,v1:172.21.15.111:6801/417496465] boot 2024-04-22T13:17:08.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:07 smithi111 ceph-mon[31361]: osdmap e13: 2 total, 2 up, 2 in 2024-04-22T13:17:08.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:07 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:17:08.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.339+0000 7f707366b700 1 -- 172.21.15.134:0/2098000233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f706c103c80 msgr2=0x7f706c106110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:08.339 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.339+0000 7f707366b700 1 --2- 172.21.15.134:0/2098000233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f706c103c80 0x7f706c106110 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f705c009990 tx=0x7f705c009ca0 comp rx=0 tx=0).stop 2024-04-22T13:17:08.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.339+0000 7f707366b700 1 -- 172.21.15.134:0/2098000233 shutdown_connections 2024-04-22T13:17:08.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.339+0000 7f707366b700 1 --2- 172.21.15.134:0/2098000233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f706c101310 0x7f706c103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:08.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.339+0000 7f707366b700 1 --2- 172.21.15.134:0/2098000233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f706c106650 0x7f706c106ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:08.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.339+0000 7f707366b700 1 --2- 172.21.15.134:0/2098000233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f706c103c80 0x7f706c106110 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:08.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.339+0000 7f707366b700 1 -- 172.21.15.134:0/2098000233 >> 172.21.15.134:0/2098000233 conn(0x7f706c0fcf30 msgr2=0x7f706c0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:08.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.339+0000 7f707366b700 1 -- 172.21.15.134:0/2098000233 shutdown_connections 2024-04-22T13:17:08.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.339+0000 7f707366b700 1 -- 172.21.15.134:0/2098000233 wait complete. 2024-04-22T13:17:08.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.340+0000 7f707366b700 1 Processor -- start 2024-04-22T13:17:08.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.340+0000 7f707366b700 1 -- start start 2024-04-22T13:17:08.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.340+0000 7f707366b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f706c101310 0x7f706c19f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:08.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.340+0000 7f707366b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f706c103c80 0x7f706c19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:08.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.340+0000 7f707366b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f706c106650 0x7f706c1a6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:08.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.340+0000 7f707366b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f706c114af0 con 0x7f706c101310 2024-04-22T13:17:08.340 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.340+0000 7f707366b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f706c114970 con 0x7f706c106650 2024-04-22T13:17:08.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.340+0000 7f707366b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f706c114c70 con 0x7f706c103c80 2024-04-22T13:17:08.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.340+0000 7f7070c06700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f706c103c80 0x7f706c19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:08.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.340+0000 7f7070c06700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f706c103c80 0x7f706c19f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:44500/0 (socket says 172.21.15.134:44500) 2024-04-22T13:17:08.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.340+0000 7f7070c06700 1 -- 172.21.15.134:0/2649970738 learned_addr learned my addr 172.21.15.134:0/2649970738 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-22T13:17:08.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.340+0000 7f7071407700 1 --2- 172.21.15.134:0/2649970738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f706c101310 0x7f706c19f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:08.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.340+0000 7f7071c08700 1 --2- 172.21.15.134:0/2649970738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f706c106650 0x7f706c1a6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:08.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.340+0000 7f7070c06700 1 -- 172.21.15.134:0/2649970738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f706c106650 msgr2=0x7f706c1a6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:08.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.340+0000 7f7070c06700 1 --2- 172.21.15.134:0/2649970738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f706c106650 0x7f706c1a6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:08.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.340+0000 7f7070c06700 1 -- 172.21.15.134:0/2649970738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f706c101310 msgr2=0x7f706c19f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:08.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.340+0000 7f7070c06700 1 --2- 172.21.15.134:0/2649970738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f706c101310 0x7f706c19f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:08.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.340+0000 7f7070c06700 1 -- 172.21.15.134:0/2649970738 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f706c1a6df0 con 0x7f706c103c80 2024-04-22T13:17:08.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.340+0000 7f7071407700 1 --2- 172.21.15.134:0/2649970738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f706c101310 0x7f706c19f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:17:08.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.340+0000 7f7071c08700 1 --2- 172.21.15.134:0/2649970738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f706c106650 0x7f706c1a6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:17:08.341 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.340+0000 7f7070c06700 1 --2- 172.21.15.134:0/2649970738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f706c103c80 0x7f706c19f6d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f705c005910 tx=0x7f705c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:08.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.341+0000 7f70627fc700 1 -- 172.21.15.134:0/2649970738 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705c027070 con 0x7f706c103c80 2024-04-22T13:17:08.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.341+0000 7f707366b700 1 -- 172.21.15.134:0/2649970738 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f706c1a7080 con 0x7f706c103c80 2024-04-22T13:17:08.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.341+0000 7f70627fc700 1 -- 172.21.15.134:0/2649970738 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f705c0041c0 con 0x7f706c103c80 2024-04-22T13:17:08.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.341+0000 7f70627fc700 1 -- 172.21.15.134:0/2649970738 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705c00f460 con 0x7f706c103c80 2024-04-22T13:17:08.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.341+0000 7f707366b700 1 -- 172.21.15.134:0/2649970738 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f706c1a7590 con 0x7f706c103c80 2024-04-22T13:17:08.342 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.341+0000 7f707366b700 1 -- 172.21.15.134:0/2649970738 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f706c069730 con 0x7f706c103c80 2024-04-22T13:17:08.342 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:07 smithi012 ceph-mon[28265]: purged_snaps scrub starts 2024-04-22T13:17:08.343 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:07 smithi012 ceph-mon[28265]: purged_snaps scrub ok 2024-04-22T13:17:08.343 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:07 smithi012 ceph-mon[28265]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-22T13:17:08.343 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:07 smithi012 ceph-mon[28265]: Detected new or changed devices on smithi111 2024-04-22T13:17:08.343 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:07 smithi012 ceph-mon[28265]: Adjusting osd_memory_target on smithi111 to 17088M 2024-04-22T13:17:08.343 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:07 smithi012 ceph-mon[28265]: osd.1 [v2:172.21.15.111:6800/417496465,v1:172.21.15.111:6801/417496465] boot 2024-04-22T13:17:08.343 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:07 smithi012 ceph-mon[28265]: osdmap e13: 2 total, 2 up, 2 in 2024-04-22T13:17:08.343 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:07 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:17:08.343 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.343+0000 7f70627fc700 1 -- 172.21.15.134:0/2649970738 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f705c02a070 con 0x7f706c103c80 2024-04-22T13:17:08.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.343+0000 7f70627fc700 1 --2- 172.21.15.134:0/2649970738 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f70580a3bb0 0x7f70580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:08.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.343+0000 7f70627fc700 1 -- 172.21.15.134:0/2649970738 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f705c0d0090 con 0x7f706c103c80 2024-04-22T13:17:08.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.343+0000 7f7071407700 1 --2- 172.21.15.134:0/2649970738 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f70580a3bb0 0x7f70580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:08.344 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.344+0000 7f7071407700 1 --2- 172.21.15.134:0/2649970738 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f70580a3bb0 0x7f70580a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7068005950 tx=0x7f7068005f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:08.345 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.347+0000 7f70627fc700 1 -- 172.21.15.134:0/2649970738 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f705c031070 con 0x7f706c103c80 2024-04-22T13:17:08.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:08.555+0000 7f707366b700 1 -- 172.21.15.134:0/2649970738 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f706c062e80 con 0x7f70580a3bb0 2024-04-22T13:17:09.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:08 smithi134 ceph-mon[31338]: osdmap e14: 2 total, 2 up, 2 in 2024-04-22T13:17:09.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:08 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:17:09.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:08 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:17:09.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:08 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:09.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:08 smithi111 ceph-mon[31361]: osdmap e14: 2 total, 2 up, 2 in 2024-04-22T13:17:09.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:08 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:17:09.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:08 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:17:09.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:08 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:09.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:08 smithi012 ceph-mon[28265]: osdmap e14: 2 total, 2 up, 2 in 2024-04-22T13:17:09.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:08 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:17:09.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:08 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:17:09.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:08 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:10.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:09 smithi134 ceph-mon[31338]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:10.133 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:09 smithi134 ceph-mon[31338]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:17:10.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:09 smithi111 ceph-mon[31361]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:10.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:09 smithi111 ceph-mon[31361]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:17:10.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:09 smithi012 ceph-mon[28265]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:10.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:09 smithi012 ceph-mon[28265]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:17:12.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:11 smithi111 ceph-mon[31361]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:12.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:11 smithi111 ceph-mon[31361]: from='client.? 172.21.15.134:0/3350722342' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71a9ebc3-c622-434e-a6c2-f48358f384f5"}]: dispatch 2024-04-22T13:17:12.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:11 smithi111 ceph-mon[31361]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71a9ebc3-c622-434e-a6c2-f48358f384f5"}]: dispatch 2024-04-22T13:17:12.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:11 smithi111 ceph-mon[31361]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71a9ebc3-c622-434e-a6c2-f48358f384f5"}]': finished 2024-04-22T13:17:12.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:11 smithi111 ceph-mon[31361]: osdmap e15: 3 total, 2 up, 3 in 2024-04-22T13:17:12.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:11 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:17:12.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:11 smithi111 ceph-mon[31361]: from='client.? 172.21.15.134:0/930778815' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:17:12.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:11 smithi012 ceph-mon[28265]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:12.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:11 smithi012 ceph-mon[28265]: from='client.? 172.21.15.134:0/3350722342' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71a9ebc3-c622-434e-a6c2-f48358f384f5"}]: dispatch 2024-04-22T13:17:12.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:11 smithi012 ceph-mon[28265]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71a9ebc3-c622-434e-a6c2-f48358f384f5"}]: dispatch 2024-04-22T13:17:12.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:11 smithi012 ceph-mon[28265]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71a9ebc3-c622-434e-a6c2-f48358f384f5"}]': finished 2024-04-22T13:17:12.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:11 smithi012 ceph-mon[28265]: osdmap e15: 3 total, 2 up, 3 in 2024-04-22T13:17:12.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:11 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:17:12.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:11 smithi012 ceph-mon[28265]: from='client.? 172.21.15.134:0/930778815' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:17:12.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:11 smithi134 ceph-mon[31338]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:12.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:11 smithi134 ceph-mon[31338]: from='client.? 172.21.15.134:0/3350722342' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71a9ebc3-c622-434e-a6c2-f48358f384f5"}]: dispatch 2024-04-22T13:17:12.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:11 smithi134 ceph-mon[31338]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71a9ebc3-c622-434e-a6c2-f48358f384f5"}]: dispatch 2024-04-22T13:17:12.430 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:11 smithi134 ceph-mon[31338]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71a9ebc3-c622-434e-a6c2-f48358f384f5"}]': finished 2024-04-22T13:17:12.430 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:11 smithi134 ceph-mon[31338]: osdmap e15: 3 total, 2 up, 3 in 2024-04-22T13:17:12.430 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:11 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:17:12.430 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:11 smithi134 ceph-mon[31338]: from='client.? 172.21.15.134:0/930778815' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:17:14.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:13 smithi111 ceph-mon[31361]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:14.231 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:13 smithi134 ceph-mon[31338]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:14.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:13 smithi012 ceph-mon[28265]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:16.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:15 smithi111 ceph-mon[31361]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:16.283 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:15 smithi134 ceph-mon[31338]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:16.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:15 smithi012 ceph-mon[28265]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:18.226 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:17 smithi134 ceph-mon[31338]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:18.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:17 smithi111 ceph-mon[31361]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:18.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:17 smithi012 ceph-mon[28265]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:19.161 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:18 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-22T13:17:19.162 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:18 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:19.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:18 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-22T13:17:19.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:18 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:19.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:18 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-22T13:17:19.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:18 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:20.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:19 smithi111 ceph-mon[31361]: Deploying daemon osd.2 on smithi134 2024-04-22T13:17:20.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:19 smithi111 ceph-mon[31361]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:20.279 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:19 smithi134 ceph-mon[31338]: Deploying daemon osd.2 on smithi134 2024-04-22T13:17:20.279 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:19 smithi134 ceph-mon[31338]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:20.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:19 smithi012 ceph-mon[28265]: Deploying daemon osd.2 on smithi134 2024-04-22T13:17:20.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:19 smithi012 ceph-mon[28265]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:22.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:21 smithi134 ceph-mon[31338]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:22.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:21 smithi111 ceph-mon[31361]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:22.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:21 smithi012 ceph-mon[28265]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:24.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:23 smithi134 ceph-mon[31338]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:24.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:23 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:24.068 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:23 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:24.069 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:23 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:17:24.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:23 smithi111 ceph-mon[31361]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:24.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:23 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:24.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:23 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:24.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:23 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:17:24.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:23 smithi012 ceph-mon[28265]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:24.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:23 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:24.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:23 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:24.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:23 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:17:26.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:25 smithi111 ceph-mon[31361]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:26.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:25 smithi012 ceph-mon[28265]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:26.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:25 smithi134 ceph-mon[31338]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:26.772 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 2 on host 'smithi134' 2024-04-22T13:17:26.772 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:26.772+0000 7f70627fc700 1 -- 172.21.15.134:0/2649970738 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f706c062e80 con 0x7f70580a3bb0 2024-04-22T13:17:26.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:26.774+0000 7f707366b700 1 -- 172.21.15.134:0/2649970738 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f70580a3bb0 msgr2=0x7f70580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:26.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:26.774+0000 7f707366b700 1 --2- 172.21.15.134:0/2649970738 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f70580a3bb0 0x7f70580a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7068005950 tx=0x7f7068005f30 comp rx=0 tx=0).stop 2024-04-22T13:17:26.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:26.774+0000 7f707366b700 1 -- 172.21.15.134:0/2649970738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f706c103c80 msgr2=0x7f706c19f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:26.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:26.774+0000 7f707366b700 1 --2- 172.21.15.134:0/2649970738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f706c103c80 0x7f706c19f6d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f705c005910 tx=0x7f705c009580 comp rx=0 tx=0).stop 2024-04-22T13:17:26.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:26.774+0000 7f707366b700 1 -- 172.21.15.134:0/2649970738 shutdown_connections 2024-04-22T13:17:26.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:26.774+0000 7f707366b700 1 --2- 172.21.15.134:0/2649970738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f706c106650 0x7f706c1a6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:26.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:26.774+0000 7f707366b700 1 --2- 172.21.15.134:0/2649970738 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f70580a3bb0 0x7f70580a6070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:26.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:26.774+0000 7f707366b700 1 --2- 172.21.15.134:0/2649970738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f706c101310 0x7f706c19f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:26.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:26.774+0000 7f707366b700 1 --2- 172.21.15.134:0/2649970738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f706c103c80 0x7f706c19f6d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:26.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:26.774+0000 7f707366b700 1 -- 172.21.15.134:0/2649970738 >> 172.21.15.134:0/2649970738 conn(0x7f706c0fcf30 msgr2=0x7f706c101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:26.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:26.775+0000 7f707366b700 1 -- 172.21.15.134:0/2649970738 shutdown_connections 2024-04-22T13:17:26.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:17:26.775+0000 7f707366b700 1 -- 172.21.15.134:0/2649970738 wait complete. 2024-04-22T13:17:27.350 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:27 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:27.350 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:27 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:27.350 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:27 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:27.350 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:27 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:17:27.350 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:27 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:27.350 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:27 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:27.350 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:27 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:27.350 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:27 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:17:27.378 DEBUG:teuthology.orchestra.run.smithi134:osd.2> sudo journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.2.service 2024-04-22T13:17:27.380 INFO:tasks.cephadm:Deploying osd.3 on smithi012 with /dev/vg_nvme/lv_3... 2024-04-22T13:17:27.380 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-22T13:17:27.407 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:27 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:27.408 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:27 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:27.408 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:27 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:27.408 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:27 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:17:27.408 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:27 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:27.408 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:27 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:27.408 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:27 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:27.408 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:27 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:17:27.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:27 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:27.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:27 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:27.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:27 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:27.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:27 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:17:27.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:27 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:27.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:27 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:27.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:27 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:27.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:27 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:17:27.633 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:17:28.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:28 smithi012 ceph-mon[28265]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:28.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:28 smithi012 ceph-mon[28265]: from='osd.2 [v2:172.21.15.134:6800/1403153998,v1:172.21.15.134:6801/1403153998]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-22T13:17:28.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:28 smithi012 ceph-mon[28265]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-22T13:17:28.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:28 smithi134 ceph-mon[31338]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:28.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:28 smithi134 ceph-mon[31338]: from='osd.2 [v2:172.21.15.134:6800/1403153998,v1:172.21.15.134:6801/1403153998]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-22T13:17:28.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:28 smithi134 ceph-mon[31338]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-22T13:17:28.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:28 smithi111 ceph-mon[31361]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:28.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:28 smithi111 ceph-mon[31361]: from='osd.2 [v2:172.21.15.134:6800/1403153998,v1:172.21.15.134:6801/1403153998]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-22T13:17:28.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:28 smithi111 ceph-mon[31361]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-22T13:17:29.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:29 smithi012 ceph-mon[28265]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-22T13:17:29.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:29 smithi012 ceph-mon[28265]: osdmap e16: 3 total, 2 up, 3 in 2024-04-22T13:17:29.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:29 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:17:29.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:29 smithi012 ceph-mon[28265]: from='osd.2 [v2:172.21.15.134:6800/1403153998,v1:172.21.15.134:6801/1403153998]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-22T13:17:29.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:29 smithi012 ceph-mon[28265]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-22T13:17:29.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:29 smithi111 ceph-mon[31361]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-22T13:17:29.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:29 smithi111 ceph-mon[31361]: osdmap e16: 3 total, 2 up, 3 in 2024-04-22T13:17:29.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:29 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:17:29.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:29 smithi111 ceph-mon[31361]: from='osd.2 [v2:172.21.15.134:6800/1403153998,v1:172.21.15.134:6801/1403153998]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-22T13:17:29.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:29 smithi111 ceph-mon[31361]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-22T13:17:29.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:29 smithi134 ceph-mon[31338]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-22T13:17:29.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:29 smithi134 ceph-mon[31338]: osdmap e16: 3 total, 2 up, 3 in 2024-04-22T13:17:29.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:29 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:17:29.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:29 smithi134 ceph-mon[31338]: from='osd.2 [v2:172.21.15.134:6800/1403153998,v1:172.21.15.134:6801/1403153998]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-22T13:17:29.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:29 smithi134 ceph-mon[31338]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-22T13:17:29.742 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-22T13:17:29.742 INFO:teuthology.orchestra.run.smithi012.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-22T13:17:29.742 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10+0 records in 2024-04-22T13:17:29.742 INFO:teuthology.orchestra.run.smithi012.stderr:10+0 records out 2024-04-22T13:17:29.742 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0118494 s, 885 MB/s 2024-04-22T13:17:29.742 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping successful for: 2024-04-22T13:17:30.328 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:30 smithi134 ceph-mon[31338]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:30.328 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:30 smithi134 ceph-mon[31338]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-22T13:17:30.328 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:30 smithi134 ceph-mon[31338]: osdmap e17: 3 total, 2 up, 3 in 2024-04-22T13:17:30.328 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:30 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:17:30.328 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:30 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:17:30.328 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:30 smithi134 ceph-mon[31338]: from='osd.2 ' entity='osd.2' 2024-04-22T13:17:30.329 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 22 13:17:30 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-2[36460]: 2024-04-22T13:17:30.087+0000 7ff2ed046700 -1 osd.2 0 waiting for initial osdmap 2024-04-22T13:17:30.329 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 22 13:17:30 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-2[36460]: 2024-04-22T13:17:30.224+0000 7ff2e4e31700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T13:17:30.443 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:30 smithi012 ceph-mon[28265]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:30.443 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:30 smithi012 ceph-mon[28265]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-22T13:17:30.443 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:30 smithi012 ceph-mon[28265]: osdmap e17: 3 total, 2 up, 3 in 2024-04-22T13:17:30.443 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:30 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:17:30.443 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:30 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:17:30.443 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:30 smithi012 ceph-mon[28265]: from='osd.2 ' entity='osd.2' 2024-04-22T13:17:30.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:30 smithi111 ceph-mon[31361]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:30.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:30 smithi111 ceph-mon[31361]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-22T13:17:30.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:30 smithi111 ceph-mon[31361]: osdmap e17: 3 total, 2 up, 3 in 2024-04-22T13:17:30.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:30 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:17:30.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:30 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:17:30.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:30 smithi111 ceph-mon[31361]: from='osd.2 ' entity='osd.2' 2024-04-22T13:17:30.533 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi012:vg_nvme/lv_3 2024-04-22T13:17:30.819 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:17:31.281 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:31 smithi012 ceph-mon[28265]: purged_snaps scrub starts 2024-04-22T13:17:31.282 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:31 smithi012 ceph-mon[28265]: purged_snaps scrub ok 2024-04-22T13:17:31.282 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:31 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:17:31.282 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:31 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:31.282 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:31 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:31.282 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:31 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:17:31.282 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:31 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:31.283 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:31 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:31.283 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:31 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:17:31.283 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:31 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:31.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:31 smithi111 ceph-mon[31361]: purged_snaps scrub starts 2024-04-22T13:17:31.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:31 smithi111 ceph-mon[31361]: purged_snaps scrub ok 2024-04-22T13:17:31.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:31 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:17:31.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:31 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:31.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:31 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:31.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:31 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:17:31.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:31 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:31.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:31 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:31.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:31 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:17:31.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:31 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:31.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:31 smithi134 ceph-mon[31338]: purged_snaps scrub starts 2024-04-22T13:17:31.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:31 smithi134 ceph-mon[31338]: purged_snaps scrub ok 2024-04-22T13:17:31.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:31 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:17:31.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:31 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:31.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:31 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:31.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:31 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:17:31.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:31 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:31.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:31 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:31.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:31 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:17:31.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:31 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:32.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:32 smithi012 ceph-mon[28265]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:32.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:32 smithi012 ceph-mon[28265]: Detected new or changed devices on smithi134 2024-04-22T13:17:32.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:32 smithi012 ceph-mon[28265]: Adjusting osd_memory_target on smithi134 to 17088M 2024-04-22T13:17:32.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:32 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:17:32.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:32 smithi012 ceph-mon[28265]: osd.2 [v2:172.21.15.134:6800/1403153998,v1:172.21.15.134:6801/1403153998] boot 2024-04-22T13:17:32.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:32 smithi012 ceph-mon[28265]: osdmap e18: 3 total, 3 up, 3 in 2024-04-22T13:17:32.316 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:32 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:17:32.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:32 smithi111 ceph-mon[31361]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:32.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:32 smithi111 ceph-mon[31361]: Detected new or changed devices on smithi134 2024-04-22T13:17:32.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:32 smithi111 ceph-mon[31361]: Adjusting osd_memory_target on smithi134 to 17088M 2024-04-22T13:17:32.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:32 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:17:32.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:32 smithi111 ceph-mon[31361]: osd.2 [v2:172.21.15.134:6800/1403153998,v1:172.21.15.134:6801/1403153998] boot 2024-04-22T13:17:32.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:32 smithi111 ceph-mon[31361]: osdmap e18: 3 total, 3 up, 3 in 2024-04-22T13:17:32.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:32 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:17:32.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.572+0000 7f00c2b15700 1 -- 172.21.15.12:0/2603532574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00bc1047b0 msgr2=0x7f00bc10b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:32.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.572+0000 7f00c2b15700 1 --2- 172.21.15.12:0/2603532574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00bc1047b0 0x7f00bc10b060 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f00b400b0d0 tx=0x7f00b400b3e0 comp rx=0 tx=0).stop 2024-04-22T13:17:32.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.572+0000 7f00c2b15700 1 -- 172.21.15.12:0/2603532574 shutdown_connections 2024-04-22T13:17:32.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.572+0000 7f00c2b15700 1 --2- 172.21.15.12:0/2603532574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00bc100d90 0x7f00bc1011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:32.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.572+0000 7f00c2b15700 1 --2- 172.21.15.12:0/2603532574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00bc1047b0 0x7f00bc10b060 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:32.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.572+0000 7f00c2b15700 1 --2- 172.21.15.12:0/2603532574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00bc103e10 0x7f00bc104270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:32.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.572+0000 7f00c2b15700 1 -- 172.21.15.12:0/2603532574 >> 172.21.15.12:0/2603532574 conn(0x7f00bc0fc6b0 msgr2=0x7f00bc0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:32.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.572+0000 7f00c2b15700 1 -- 172.21.15.12:0/2603532574 shutdown_connections 2024-04-22T13:17:32.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.572+0000 7f00c2b15700 1 -- 172.21.15.12:0/2603532574 wait complete. 2024-04-22T13:17:32.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.572+0000 7f00c2b15700 1 Processor -- start 2024-04-22T13:17:32.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.573+0000 7f00c2b15700 1 -- start start 2024-04-22T13:17:32.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.573+0000 7f00c2b15700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00bc100d90 0x7f00bc103770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:32.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.573+0000 7f00c2b15700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00bc103e10 0x7f00bc101dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:32.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.573+0000 7f00c2b15700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00bc1047b0 0x7f00bc102300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:32.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.573+0000 7f00c2b15700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc10e550 con 0x7f00bc100d90 2024-04-22T13:17:32.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.573+0000 7f00c2b15700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc10e3d0 con 0x7f00bc1047b0 2024-04-22T13:17:32.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.573+0000 7f00c2b15700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc10e6d0 con 0x7f00bc103e10 2024-04-22T13:17:32.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.573+0000 7f00c08b1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00bc100d90 0x7f00bc103770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:32.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.573+0000 7f00c08b1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00bc100d90 0x7f00bc103770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:46850/0 (socket says 172.21.15.12:46850) 2024-04-22T13:17:32.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.573+0000 7f00c08b1700 1 -- 172.21.15.12:0/2366815643 learned_addr learned my addr 172.21.15.12:0/2366815643 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:17:32.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.573+0000 7f00c10b2700 1 --2- 172.21.15.12:0/2366815643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00bc1047b0 0x7f00bc102300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:32.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.573+0000 7f00bbfff700 1 --2- 172.21.15.12:0/2366815643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00bc103e10 0x7f00bc101dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:32.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.573+0000 7f00c08b1700 1 -- 172.21.15.12:0/2366815643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00bc103e10 msgr2=0x7f00bc101dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:32.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.573+0000 7f00c08b1700 1 --2- 172.21.15.12:0/2366815643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00bc103e10 0x7f00bc101dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:32.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.573+0000 7f00c08b1700 1 -- 172.21.15.12:0/2366815643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00bc1047b0 msgr2=0x7f00bc102300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:32.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.573+0000 7f00c08b1700 1 --2- 172.21.15.12:0/2366815643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00bc1047b0 0x7f00bc102300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:32.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.573+0000 7f00c08b1700 1 -- 172.21.15.12:0/2366815643 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00bc102bb0 con 0x7f00bc100d90 2024-04-22T13:17:32.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.573+0000 7f00c10b2700 1 --2- 172.21.15.12:0/2366815643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00bc1047b0 0x7f00bc102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:17:32.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.573+0000 7f00c08b1700 1 --2- 172.21.15.12:0/2366815643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00bc100d90 0x7f00bc103770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f00b000e970 tx=0x7f00b000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:32.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.573+0000 7f00b9ffb700 1 -- 172.21.15.12:0/2366815643 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b000cbd0 con 0x7f00bc100d90 2024-04-22T13:17:32.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.574+0000 7f00b9ffb700 1 -- 172.21.15.12:0/2366815643 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00b00044d0 con 0x7f00bc100d90 2024-04-22T13:17:32.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.574+0000 7f00c2b15700 1 -- 172.21.15.12:0/2366815643 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00bc1a2cc0 con 0x7f00bc100d90 2024-04-22T13:17:32.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.574+0000 7f00b9ffb700 1 -- 172.21.15.12:0/2366815643 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b0003e40 con 0x7f00bc100d90 2024-04-22T13:17:32.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.574+0000 7f00c2b15700 1 -- 172.21.15.12:0/2366815643 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00bc1a3180 con 0x7f00bc100d90 2024-04-22T13:17:32.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.575+0000 7f00c2b15700 1 -- 172.21.15.12:0/2366815643 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00bc069730 con 0x7f00bc100d90 2024-04-22T13:17:32.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.576+0000 7f00b9ffb700 1 -- 172.21.15.12:0/2366815643 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f00b0003fe0 con 0x7f00bc100d90 2024-04-22T13:17:32.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.577+0000 7f00b9ffb700 1 --2- 172.21.15.12:0/2366815643 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f00ac0a3bb0 0x7f00ac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:32.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.577+0000 7f00bbfff700 1 --2- 172.21.15.12:0/2366815643 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f00ac0a3bb0 0x7f00ac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:32.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.577+0000 7f00b9ffb700 1 -- 172.21.15.12:0/2366815643 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 2851+0+0 (secure 0 0 0) 0x7f00b00cf250 con 0x7f00bc100d90 2024-04-22T13:17:32.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.578+0000 7f00bbfff700 1 --2- 172.21.15.12:0/2366815643 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f00ac0a3bb0 0x7f00ac0a6070 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f00a8005950 tx=0x7f00a80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:32.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.582+0000 7f00b9ffb700 1 -- 172.21.15.12:0/2366815643 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f00b009d250 con 0x7f00bc100d90 2024-04-22T13:17:32.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:32 smithi134 ceph-mon[31338]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-22T13:17:32.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:32 smithi134 ceph-mon[31338]: Detected new or changed devices on smithi134 2024-04-22T13:17:32.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:32 smithi134 ceph-mon[31338]: Adjusting osd_memory_target on smithi134 to 17088M 2024-04-22T13:17:32.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:32 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:17:32.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:32 smithi134 ceph-mon[31338]: osd.2 [v2:172.21.15.134:6800/1403153998,v1:172.21.15.134:6801/1403153998] boot 2024-04-22T13:17:32.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:32 smithi134 ceph-mon[31338]: osdmap e18: 3 total, 3 up, 3 in 2024-04-22T13:17:32.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:32 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:17:32.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:32.799+0000 7f00c2b15700 1 -- 172.21.15.12:0/2366815643 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f00bc062e80 con 0x7f00ac0a3bb0 2024-04-22T13:17:33.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:33 smithi111 ceph-mon[31361]: osdmap e19: 3 total, 3 up, 3 in 2024-04-22T13:17:33.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:33 smithi111 ceph-mon[31361]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:33.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:33 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-22T13:17:33.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:33 smithi111 ceph-mon[31361]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:17:33.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:33 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:17:33.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:33 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:17:33.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:33 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:33.537 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:33 smithi012 ceph-mon[28265]: osdmap e19: 3 total, 3 up, 3 in 2024-04-22T13:17:33.537 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:33 smithi012 ceph-mon[28265]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:33.537 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:33 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-22T13:17:33.537 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:33 smithi012 ceph-mon[28265]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:17:33.537 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:33 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:17:33.537 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:33 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:17:33.537 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:33 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:33.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:33 smithi134 ceph-mon[31338]: osdmap e19: 3 total, 3 up, 3 in 2024-04-22T13:17:33.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:33 smithi134 ceph-mon[31338]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:33.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:33 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-22T13:17:33.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:33 smithi134 ceph-mon[31338]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:17:33.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:33 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:17:33.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:33 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:17:33.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:33 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:34.460 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:34 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-22T13:17:34.460 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:34 smithi134 ceph-mon[31338]: osdmap e20: 3 total, 3 up, 3 in 2024-04-22T13:17:34.461 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:34 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-22T13:17:34.461 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 22 13:17:34 smithi134 sudo[40521]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-22T13:17:34.461 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 22 13:17:34 smithi134 sudo[40521]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T13:17:34.461 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 22 13:17:34 smithi134 sudo[40521]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T13:17:34.461 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 22 13:17:34 smithi134 sudo[40521]: pam_unix(sudo:session): session closed for user root 2024-04-22T13:17:34.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:34 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-22T13:17:34.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:34 smithi111 ceph-mon[31361]: osdmap e20: 3 total, 3 up, 3 in 2024-04-22T13:17:34.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:34 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-22T13:17:34.478 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 22 13:17:34 smithi111 sudo[40093]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-22T13:17:34.478 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 22 13:17:34 smithi111 sudo[40093]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T13:17:34.479 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 22 13:17:34 smithi111 sudo[40093]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T13:17:34.479 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 22 13:17:34 smithi111 sudo[40093]: pam_unix(sudo:session): session closed for user root 2024-04-22T13:17:34.479 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 22 13:17:34 smithi111 sudo[40096]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-22T13:17:34.479 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 22 13:17:34 smithi111 sudo[40096]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T13:17:34.479 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 22 13:17:34 smithi111 sudo[40096]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T13:17:34.479 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 22 13:17:34 smithi111 sudo[40096]: pam_unix(sudo:session): session closed for user root 2024-04-22T13:17:34.508 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:34 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-22T13:17:34.508 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:34 smithi012 ceph-mon[28265]: osdmap e20: 3 total, 3 up, 3 in 2024-04-22T13:17:34.509 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:34 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-22T13:17:34.509 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 22 13:17:34 smithi012 sudo[47558]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-22T13:17:34.509 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 22 13:17:34 smithi012 sudo[47558]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T13:17:34.509 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 22 13:17:34 smithi012 sudo[47558]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T13:17:34.509 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 22 13:17:34 smithi012 sudo[47558]: pam_unix(sudo:session): session closed for user root 2024-04-22T13:17:34.509 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 22 13:17:34 smithi012 sudo[47561]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-22T13:17:34.509 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 22 13:17:34 smithi012 sudo[47561]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T13:17:34.509 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 22 13:17:34 smithi012 sudo[47561]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T13:17:34.509 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 22 13:17:34 smithi012 sudo[47561]: pam_unix(sudo:session): session closed for user root 2024-04-22T13:17:34.787 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:34 smithi012 sudo[47585]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-22T13:17:34.787 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:34 smithi012 sudo[47585]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T13:17:34.787 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:34 smithi012 sudo[47585]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T13:17:34.787 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:34 smithi012 sudo[47585]: pam_unix(sudo:session): session closed for user root 2024-04-22T13:17:34.788 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:34 smithi012 sudo[47588]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-22T13:17:34.788 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:34 smithi012 sudo[47588]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T13:17:34.788 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:34 smithi012 sudo[47588]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T13:17:34.788 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:34 smithi012 sudo[47588]: pam_unix(sudo:session): session closed for user root 2024-04-22T13:17:34.929 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 22 13:17:34 smithi134 sudo[40524]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-22T13:17:34.929 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 22 13:17:34 smithi134 sudo[40524]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T13:17:34.929 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 22 13:17:34 smithi134 sudo[40524]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T13:17:34.929 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 22 13:17:34 smithi134 sudo[40524]: pam_unix(sudo:session): session closed for user root 2024-04-22T13:17:34.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:34 smithi134 sudo[40527]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-22T13:17:34.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:34 smithi134 sudo[40527]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T13:17:34.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:34 smithi134 sudo[40527]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T13:17:34.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:34 smithi134 sudo[40527]: pam_unix(sudo:session): session closed for user root 2024-04-22T13:17:34.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:34 smithi134 sudo[40530]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-22T13:17:34.930 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:34 smithi134 sudo[40530]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T13:17:34.930 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:34 smithi134 sudo[40530]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T13:17:34.930 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:34 smithi134 sudo[40530]: pam_unix(sudo:session): session closed for user root 2024-04-22T13:17:34.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:34 smithi111 sudo[40099]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-22T13:17:34.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:34 smithi111 sudo[40099]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T13:17:34.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:34 smithi111 sudo[40099]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T13:17:34.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:34 smithi111 sudo[40099]: pam_unix(sudo:session): session closed for user root 2024-04-22T13:17:34.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:34 smithi111 sudo[40102]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-22T13:17:34.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:34 smithi111 sudo[40102]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-22T13:17:34.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:34 smithi111 sudo[40102]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-22T13:17:34.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:34 smithi111 sudo[40102]: pam_unix(sudo:session): session closed for user root 2024-04-22T13:17:35.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:35.225+0000 7f00b9ffb700 1 -- 172.21.15.12:0/2366815643 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f00b00991c0 con 0x7f00bc100d90 2024-04-22T13:17:35.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:35 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-22T13:17:35.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:35 smithi111 ceph-mon[31361]: osdmap e21: 3 total, 3 up, 3 in 2024-04-22T13:17:35.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:35 smithi111 ceph-mon[31361]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:35.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:35 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:35.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:35 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:35.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:35 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:35.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:35 smithi111 ceph-mon[31361]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T13:17:35.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:35 smithi111 ceph-mon[31361]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T13:17:35.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:35 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:17:35.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:35 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:17:35.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:35 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:17:35.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:35 smithi111 ceph-mon[31361]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T13:17:35.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:35 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:17:35.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:35 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:17:35.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:35 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:17:35.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:35 smithi111 ceph-mon[31361]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T13:17:35.480 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:35 smithi111 ceph-mon[31361]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T13:17:35.480 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:35 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:17:35.480 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:35 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:17:35.480 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:35 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:17:35.480 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:35 smithi111 ceph-mon[31361]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T13:17:35.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:35 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-22T13:17:35.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:35 smithi012 ceph-mon[28265]: osdmap e21: 3 total, 3 up, 3 in 2024-04-22T13:17:35.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:35 smithi012 ceph-mon[28265]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:35.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:35 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:35.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:35 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:35.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:35 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:35.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:35 smithi012 ceph-mon[28265]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T13:17:35.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:35 smithi012 ceph-mon[28265]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T13:17:35.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:35 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:17:35.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:35 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:17:35.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:35 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:17:35.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:35 smithi012 ceph-mon[28265]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T13:17:35.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:35 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:17:35.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:35 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:17:35.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:35 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:17:35.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:35 smithi012 ceph-mon[28265]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T13:17:35.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:35 smithi012 ceph-mon[28265]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T13:17:35.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:35 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:17:35.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:35 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:17:35.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:35 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:17:35.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:35 smithi012 ceph-mon[28265]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T13:17:35.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:35 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-22T13:17:35.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:35 smithi134 ceph-mon[31338]: osdmap e21: 3 total, 3 up, 3 in 2024-04-22T13:17:35.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:35 smithi134 ceph-mon[31338]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:35.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:35 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:35.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:35 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:35.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:35 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:35.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:35 smithi134 ceph-mon[31338]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T13:17:35.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:35 smithi134 ceph-mon[31338]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T13:17:35.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:35 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:17:35.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:35 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:17:35.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:35 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:17:35.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:35 smithi134 ceph-mon[31338]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T13:17:35.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:35 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:17:35.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:35 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:17:35.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:35 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:17:35.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:35 smithi134 ceph-mon[31338]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T13:17:35.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:35 smithi134 ceph-mon[31338]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-22T13:17:35.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:35 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:17:35.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:35 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:17:35.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:35 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:17:35.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:35 smithi134 ceph-mon[31338]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-22T13:17:36.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:36 smithi111 ceph-mon[31361]: osdmap e22: 3 total, 3 up, 3 in 2024-04-22T13:17:36.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:36 smithi111 ceph-mon[31361]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-22T13:17:36.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:36 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/507546271' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6dd00db2-a286-48ba-8666-56ffdd5f35ba"}]: dispatch 2024-04-22T13:17:36.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:36 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/507546271' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6dd00db2-a286-48ba-8666-56ffdd5f35ba"}]': finished 2024-04-22T13:17:36.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:36 smithi111 ceph-mon[31361]: osdmap e23: 4 total, 3 up, 4 in 2024-04-22T13:17:36.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:36 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:17:36.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:36 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1918166660' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:17:36.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:36 smithi012 ceph-mon[28265]: osdmap e22: 3 total, 3 up, 3 in 2024-04-22T13:17:36.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:36 smithi012 ceph-mon[28265]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-22T13:17:36.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:36 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/507546271' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6dd00db2-a286-48ba-8666-56ffdd5f35ba"}]: dispatch 2024-04-22T13:17:36.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:36 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/507546271' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6dd00db2-a286-48ba-8666-56ffdd5f35ba"}]': finished 2024-04-22T13:17:36.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:36 smithi012 ceph-mon[28265]: osdmap e23: 4 total, 3 up, 4 in 2024-04-22T13:17:36.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:36 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:17:36.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:36 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1918166660' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:17:36.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:36 smithi134 ceph-mon[31338]: osdmap e22: 3 total, 3 up, 3 in 2024-04-22T13:17:36.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:36 smithi134 ceph-mon[31338]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-22T13:17:36.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:36 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/507546271' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6dd00db2-a286-48ba-8666-56ffdd5f35ba"}]: dispatch 2024-04-22T13:17:36.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:36 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/507546271' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6dd00db2-a286-48ba-8666-56ffdd5f35ba"}]': finished 2024-04-22T13:17:36.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:36 smithi134 ceph-mon[31338]: osdmap e23: 4 total, 3 up, 4 in 2024-04-22T13:17:36.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:36 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:17:36.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:36 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1918166660' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:17:37.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:37 smithi111 ceph-mon[31361]: pgmap v89: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:37.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:37 smithi012 ceph-mon[28265]: pgmap v89: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:37.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:37 smithi134 ceph-mon[31338]: pgmap v89: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:39.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:39 smithi134 ceph-mon[31338]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:39.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:39 smithi111 ceph-mon[31361]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:39.764 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:39 smithi012 ceph-mon[28265]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:41.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:41 smithi134 ceph-mon[31338]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:41.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:41 smithi111 ceph-mon[31361]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:41.729 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:41 smithi012 ceph-mon[28265]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:43.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:43 smithi134 ceph-mon[31338]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:43.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:43 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-22T13:17:43.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:43 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:43.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:43 smithi134 ceph-mon[31338]: Deploying daemon osd.3 on smithi012 2024-04-22T13:17:43.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:43 smithi111 ceph-mon[31361]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:43.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:43 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-22T13:17:43.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:43 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:43.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:43 smithi111 ceph-mon[31361]: Deploying daemon osd.3 on smithi012 2024-04-22T13:17:43.756 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:43 smithi012 ceph-mon[28265]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:43.756 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-22T13:17:43.756 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:43.756 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:43 smithi012 ceph-mon[28265]: Deploying daemon osd.3 on smithi012 2024-04-22T13:17:45.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:45 smithi134 ceph-mon[31338]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:45.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:45 smithi111 ceph-mon[31361]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:45.765 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:45 smithi012 ceph-mon[28265]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:47.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:47 smithi134 ceph-mon[31338]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:47.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:47 smithi111 ceph-mon[31361]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:47.808 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:47 smithi012 ceph-mon[28265]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:49.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:49 smithi134 ceph-mon[31338]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:49.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:49 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:49.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:49 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:49.930 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:49 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:17:49.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:49 smithi111 ceph-mon[31361]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:49.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:49 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:49.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:49 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:49.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:49 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:17:50.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:49 smithi012 ceph-mon[28265]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:50.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:49 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:50.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:49 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:50.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:49 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:17:51.935 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:51 smithi134 ceph-mon[31338]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:51.936 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:51 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:51.936 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:51 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:51.936 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:51 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:51.936 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:51 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:17:51.937 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:51 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:51.959 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:51 smithi012 ceph-mon[28265]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:51.959 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:51 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:51.960 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:51 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:51.960 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:51 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:51.960 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:51 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:17:51.960 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:51 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:51.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:51 smithi111 ceph-mon[31361]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:51.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:51 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:51.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:51 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:51.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:51 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:51.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:51 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:17:51.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:51 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:52.281 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 3 on host 'smithi012' 2024-04-22T13:17:52.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:52.277+0000 7f00b9ffb700 1 -- 172.21.15.12:0/2366815643 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f00bc062e80 con 0x7f00ac0a3bb0 2024-04-22T13:17:52.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:52.280+0000 7f00c2b15700 1 -- 172.21.15.12:0/2366815643 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f00ac0a3bb0 msgr2=0x7f00ac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:52.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:52.280+0000 7f00c2b15700 1 --2- 172.21.15.12:0/2366815643 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f00ac0a3bb0 0x7f00ac0a6070 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f00a8005950 tx=0x7f00a80058e0 comp rx=0 tx=0).stop 2024-04-22T13:17:52.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:52.280+0000 7f00c2b15700 1 -- 172.21.15.12:0/2366815643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00bc100d90 msgr2=0x7f00bc103770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:52.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:52.280+0000 7f00c2b15700 1 --2- 172.21.15.12:0/2366815643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00bc100d90 0x7f00bc103770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f00b000e970 tx=0x7f00b000ed30 comp rx=0 tx=0).stop 2024-04-22T13:17:52.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:52.280+0000 7f00c2b15700 1 -- 172.21.15.12:0/2366815643 shutdown_connections 2024-04-22T13:17:52.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:52.280+0000 7f00c2b15700 1 --2- 172.21.15.12:0/2366815643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00bc1047b0 0x7f00bc102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:52.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:52.280+0000 7f00c2b15700 1 --2- 172.21.15.12:0/2366815643 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f00ac0a3bb0 0x7f00ac0a6070 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:52.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:52.280+0000 7f00c2b15700 1 --2- 172.21.15.12:0/2366815643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00bc100d90 0x7f00bc103770 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:52.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:52.280+0000 7f00c2b15700 1 --2- 172.21.15.12:0/2366815643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00bc103e10 0x7f00bc101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:52.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:52.280+0000 7f00c2b15700 1 -- 172.21.15.12:0/2366815643 >> 172.21.15.12:0/2366815643 conn(0x7f00bc0fc6b0 msgr2=0x7f00bc107110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:52.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:52.280+0000 7f00c2b15700 1 -- 172.21.15.12:0/2366815643 shutdown_connections 2024-04-22T13:17:52.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:17:52.280+0000 7f00c2b15700 1 -- 172.21.15.12:0/2366815643 wait complete. 2024-04-22T13:17:52.909 DEBUG:teuthology.orchestra.run.smithi012:osd.3> sudo journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.3.service 2024-04-22T13:17:52.911 INFO:tasks.cephadm:Deploying osd.4 on smithi111 with /dev/vg_nvme/lv_3... 2024-04-22T13:17:52.911 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-22T13:17:53.156 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T13:17:53.546 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:53 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:53.546 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:53 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:53.546 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:53 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:17:53.546 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:53 smithi111 ceph-mon[31361]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:53.546 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:53 smithi111 ceph-mon[31361]: from='osd.3 [v2:172.21.15.12:6810/3359066681,v1:172.21.15.12:6811/3359066681]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-22T13:17:53.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:53 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:53.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:53 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:53.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:53 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:17:53.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:53 smithi012 ceph-mon[28265]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:53.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:53 smithi012 ceph-mon[28265]: from='osd.3 [v2:172.21.15.12:6810/3359066681,v1:172.21.15.12:6811/3359066681]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-22T13:17:53.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:53 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:53.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:53 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:53.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:53 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:17:53.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:53 smithi134 ceph-mon[31338]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:53.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:53 smithi134 ceph-mon[31338]: from='osd.3 [v2:172.21.15.12:6810/3359066681,v1:172.21.15.12:6811/3359066681]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-22T13:17:54.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:54 smithi012 ceph-mon[28265]: from='osd.3 [v2:172.21.15.12:6810/3359066681,v1:172.21.15.12:6811/3359066681]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-22T13:17:54.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:54 smithi012 ceph-mon[28265]: osdmap e24: 4 total, 3 up, 4 in 2024-04-22T13:17:54.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:54 smithi012 ceph-mon[28265]: from='osd.3 [v2:172.21.15.12:6810/3359066681,v1:172.21.15.12:6811/3359066681]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-22T13:17:54.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:54 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:17:54.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:54 smithi111 ceph-mon[31361]: from='osd.3 [v2:172.21.15.12:6810/3359066681,v1:172.21.15.12:6811/3359066681]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-22T13:17:54.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:54 smithi111 ceph-mon[31361]: osdmap e24: 4 total, 3 up, 4 in 2024-04-22T13:17:54.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:54 smithi111 ceph-mon[31361]: from='osd.3 [v2:172.21.15.12:6810/3359066681,v1:172.21.15.12:6811/3359066681]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-22T13:17:54.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:54 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:17:54.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:54 smithi134 ceph-mon[31338]: from='osd.3 [v2:172.21.15.12:6810/3359066681,v1:172.21.15.12:6811/3359066681]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-22T13:17:54.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:54 smithi134 ceph-mon[31338]: osdmap e24: 4 total, 3 up, 4 in 2024-04-22T13:17:54.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:54 smithi134 ceph-mon[31338]: from='osd.3 [v2:172.21.15.12:6810/3359066681,v1:172.21.15.12:6811/3359066681]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-22T13:17:54.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:54 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:17:55.082 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-22T13:17:55.082 INFO:teuthology.orchestra.run.smithi111.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-22T13:17:55.082 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10+0 records in 2024-04-22T13:17:55.082 INFO:teuthology.orchestra.run.smithi111.stderr:10+0 records out 2024-04-22T13:17:55.082 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128544 s, 816 MB/s 2024-04-22T13:17:55.082 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping successful for: 2024-04-22T13:17:55.558 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:55 smithi111 ceph-mon[31361]: from='osd.3 [v2:172.21.15.12:6810/3359066681,v1:172.21.15.12:6811/3359066681]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-04-22T13:17:55.558 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:55 smithi111 ceph-mon[31361]: osdmap e25: 4 total, 3 up, 4 in 2024-04-22T13:17:55.558 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:55 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:17:55.558 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:55 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:17:55.559 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:55 smithi111 ceph-mon[31361]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:55.559 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:55 smithi111 ceph-mon[31361]: from='osd.3 [v2:172.21.15.12:6810/3359066681,v1:172.21.15.12:6811/3359066681]' entity='osd.3' 2024-04-22T13:17:55.559 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:55 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:17:55.571 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:55 smithi012 ceph-mon[28265]: from='osd.3 [v2:172.21.15.12:6810/3359066681,v1:172.21.15.12:6811/3359066681]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-04-22T13:17:55.571 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:55 smithi012 ceph-mon[28265]: osdmap e25: 4 total, 3 up, 4 in 2024-04-22T13:17:55.571 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:55 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:17:55.571 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:55 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:17:55.571 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:55 smithi012 ceph-mon[28265]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:55.571 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:55 smithi012 ceph-mon[28265]: from='osd.3 [v2:172.21.15.12:6810/3359066681,v1:172.21.15.12:6811/3359066681]' entity='osd.3' 2024-04-22T13:17:55.571 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:55 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:17:55.572 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 22 13:17:55 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-3[49663]: 2024-04-22T13:17:55.212+0000 7fb49f115700 -1 osd.3 0 waiting for initial osdmap 2024-04-22T13:17:55.572 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 22 13:17:55 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-3[49663]: 2024-04-22T13:17:55.297+0000 7fb496f00700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T13:17:55.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:55 smithi134 ceph-mon[31338]: from='osd.3 [v2:172.21.15.12:6810/3359066681,v1:172.21.15.12:6811/3359066681]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-04-22T13:17:55.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:55 smithi134 ceph-mon[31338]: osdmap e25: 4 total, 3 up, 4 in 2024-04-22T13:17:55.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:55 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:17:55.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:55 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:17:55.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:55 smithi134 ceph-mon[31338]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:55.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:55 smithi134 ceph-mon[31338]: from='osd.3 [v2:172.21.15.12:6810/3359066681,v1:172.21.15.12:6811/3359066681]' entity='osd.3' 2024-04-22T13:17:55.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:55 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:17:55.756 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_3 2024-04-22T13:17:55.996 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T13:17:56.371 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:56 smithi111 ceph-mon[31361]: purged_snaps scrub starts 2024-04-22T13:17:56.371 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:56 smithi111 ceph-mon[31361]: purged_snaps scrub ok 2024-04-22T13:17:56.371 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:56 smithi111 ceph-mon[31361]: osdmap e26: 4 total, 3 up, 4 in 2024-04-22T13:17:56.371 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:56 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:17:56.371 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:56 smithi111 ceph-mon[31361]: Detected new or changed devices on smithi012 2024-04-22T13:17:56.371 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:56 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:56.371 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:56 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:56.371 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:56 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:17:56.371 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:56 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:17:56.371 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:56 smithi111 ceph-mon[31361]: Adjusting osd_memory_target on smithi012 to 8546M 2024-04-22T13:17:56.371 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:56 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:56.371 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:56 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:56.372 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:56 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:17:56.372 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:56 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:56.372 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:56 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:17:56.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:56 smithi012 ceph-mon[28265]: purged_snaps scrub starts 2024-04-22T13:17:56.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:56 smithi012 ceph-mon[28265]: purged_snaps scrub ok 2024-04-22T13:17:56.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:56 smithi012 ceph-mon[28265]: osdmap e26: 4 total, 3 up, 4 in 2024-04-22T13:17:56.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:56 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:17:56.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:56 smithi012 ceph-mon[28265]: Detected new or changed devices on smithi012 2024-04-22T13:17:56.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:56 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:56.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:56 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:56.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:56 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:17:56.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:56 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:17:56.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:56 smithi012 ceph-mon[28265]: Adjusting osd_memory_target on smithi012 to 8546M 2024-04-22T13:17:56.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:56 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:56.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:56 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:56.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:56 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:17:56.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:56 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:56.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:56 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:17:56.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:56 smithi134 ceph-mon[31338]: purged_snaps scrub starts 2024-04-22T13:17:56.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:56 smithi134 ceph-mon[31338]: purged_snaps scrub ok 2024-04-22T13:17:56.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:56 smithi134 ceph-mon[31338]: osdmap e26: 4 total, 3 up, 4 in 2024-04-22T13:17:56.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:56 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:17:56.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:56 smithi134 ceph-mon[31338]: Detected new or changed devices on smithi012 2024-04-22T13:17:56.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:56 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:56.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:56 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:56.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:56 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:17:56.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:56 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:17:56.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:56 smithi134 ceph-mon[31338]: Adjusting osd_memory_target on smithi012 to 8546M 2024-04-22T13:17:56.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:56 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:56.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:56 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:56.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:56 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:17:56.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:56 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:17:56.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:56 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:17:57.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.508+0000 7f0e7dc22700 1 -- 172.21.15.111:0/1464030469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e78103540 msgr2=0x7f0e78109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:57.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.508+0000 7f0e7dc22700 1 --2- 172.21.15.111:0/1464030469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e78103540 0x7f0e78109df0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0e7000b0d0 tx=0x7f0e7000b3e0 comp rx=0 tx=0).stop 2024-04-22T13:17:57.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.508+0000 7f0e7dc22700 1 -- 172.21.15.111:0/1464030469 shutdown_connections 2024-04-22T13:17:57.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.508+0000 7f0e7dc22700 1 --2- 172.21.15.111:0/1464030469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e78103540 0x7f0e78109df0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:57.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.508+0000 7f0e7dc22700 1 --2- 172.21.15.111:0/1464030469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e781019a0 0x7f0e78101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:57.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.508+0000 7f0e7dc22700 1 --2- 172.21.15.111:0/1464030469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e78102ba0 0x7f0e78103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:57.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.508+0000 7f0e7dc22700 1 -- 172.21.15.111:0/1464030469 >> 172.21.15.111:0/1464030469 conn(0x7f0e780fcf40 msgr2=0x7f0e780ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:17:57.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.509+0000 7f0e7dc22700 1 -- 172.21.15.111:0/1464030469 shutdown_connections 2024-04-22T13:17:57.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.509+0000 7f0e7dc22700 1 -- 172.21.15.111:0/1464030469 wait complete. 2024-04-22T13:17:57.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.509+0000 7f0e7dc22700 1 Processor -- start 2024-04-22T13:17:57.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.509+0000 7f0e7dc22700 1 -- start start 2024-04-22T13:17:57.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.509+0000 7f0e7dc22700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e781019a0 0x7f0e7806bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:57.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.510+0000 7f0e7dc22700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e78102ba0 0x7f0e7819d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:57.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.510+0000 7f0e7dc22700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e78103540 0x7f0e7819fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:57.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.510+0000 7f0e7dc22700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7810d420 con 0x7f0e78102ba0 2024-04-22T13:17:57.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.510+0000 7f0e7dc22700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7810d2a0 con 0x7f0e78103540 2024-04-22T13:17:57.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.510+0000 7f0e7dc22700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7810d5a0 con 0x7f0e781019a0 2024-04-22T13:17:57.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.510+0000 7f0e77fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e78103540 0x7f0e7819fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:57.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.510+0000 7f0e77fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e78103540 0x7f0e7819fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57706/0 (socket says 172.21.15.111:57706) 2024-04-22T13:17:57.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.510+0000 7f0e77fff700 1 -- 172.21.15.111:0/3551113525 learned_addr learned my addr 172.21.15.111:0/3551113525 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-22T13:17:57.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.510+0000 7f0e777fe700 1 --2- 172.21.15.111:0/3551113525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e781019a0 0x7f0e7806bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:57.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.510+0000 7f0e6ffff700 1 --2- 172.21.15.111:0/3551113525 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e78102ba0 0x7f0e7819d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:57.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.510+0000 7f0e77fff700 1 -- 172.21.15.111:0/3551113525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e781019a0 msgr2=0x7f0e7806bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:57.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.510+0000 7f0e77fff700 1 --2- 172.21.15.111:0/3551113525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e781019a0 0x7f0e7806bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:57.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.510+0000 7f0e77fff700 1 -- 172.21.15.111:0/3551113525 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e78102ba0 msgr2=0x7f0e7819d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:17:57.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.510+0000 7f0e77fff700 1 --2- 172.21.15.111:0/3551113525 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e78102ba0 0x7f0e7819d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:17:57.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.510+0000 7f0e77fff700 1 -- 172.21.15.111:0/3551113525 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e781a0730 con 0x7f0e78103540 2024-04-22T13:17:57.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.510+0000 7f0e6ffff700 1 --2- 172.21.15.111:0/3551113525 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e78102ba0 0x7f0e7819d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:17:57.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.510+0000 7f0e77fff700 1 --2- 172.21.15.111:0/3551113525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e78103540 0x7f0e7819fe60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0e70009040 tx=0x7f0e70004e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:57.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.510+0000 7f0e757fa700 1 -- 172.21.15.111:0/3551113525 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7001f030 con 0x7f0e78103540 2024-04-22T13:17:57.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.510+0000 7f0e757fa700 1 -- 172.21.15.111:0/3551113525 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e70007a80 con 0x7f0e78103540 2024-04-22T13:17:57.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.510+0000 7f0e7dc22700 1 -- 172.21.15.111:0/3551113525 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e781a09c0 con 0x7f0e78103540 2024-04-22T13:17:57.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.510+0000 7f0e757fa700 1 -- 172.21.15.111:0/3551113525 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e700047e0 con 0x7f0e78103540 2024-04-22T13:17:57.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.510+0000 7f0e7dc22700 1 -- 172.21.15.111:0/3551113525 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e781a0e10 con 0x7f0e78103540 2024-04-22T13:17:57.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.511+0000 7f0e7dc22700 1 -- 172.21.15.111:0/3551113525 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e78069730 con 0x7f0e78103540 2024-04-22T13:17:57.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.512+0000 7f0e757fa700 1 -- 172.21.15.111:0/3551113525 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0e70003a00 con 0x7f0e78103540 2024-04-22T13:17:57.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.513+0000 7f0e757fa700 1 --2- 172.21.15.111:0/3551113525 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f0e640a3c40 0x7f0e640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:17:57.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.513+0000 7f0e757fa700 1 -- 172.21.15.111:0/3551113525 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3698+0+0 (secure 0 0 0) 0x7f0e7000f020 con 0x7f0e78103540 2024-04-22T13:17:57.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.513+0000 7f0e777fe700 1 --2- 172.21.15.111:0/3551113525 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f0e640a3c40 0x7f0e640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:17:57.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.513+0000 7f0e777fe700 1 --2- 172.21.15.111:0/3551113525 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f0e640a3c40 0x7f0e640a6100 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0e78102a00 tx=0x7f0e6800b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:17:57.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.518+0000 7f0e757fa700 1 -- 172.21.15.111:0/3551113525 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0e70029020 con 0x7f0e78103540 2024-04-22T13:17:57.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:57 smithi012 ceph-mon[28265]: osd.3 [v2:172.21.15.12:6810/3359066681,v1:172.21.15.12:6811/3359066681] boot 2024-04-22T13:17:57.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:57 smithi012 ceph-mon[28265]: osdmap e27: 4 total, 4 up, 4 in 2024-04-22T13:17:57.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:57 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:17:57.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:57 smithi012 ceph-mon[28265]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:57.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:57 smithi012 ceph-mon[28265]: osdmap e28: 4 total, 4 up, 4 in 2024-04-22T13:17:57.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:57 smithi134 ceph-mon[31338]: osd.3 [v2:172.21.15.12:6810/3359066681,v1:172.21.15.12:6811/3359066681] boot 2024-04-22T13:17:57.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:57 smithi134 ceph-mon[31338]: osdmap e27: 4 total, 4 up, 4 in 2024-04-22T13:17:57.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:57 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:17:57.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:57 smithi134 ceph-mon[31338]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:57.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:57 smithi134 ceph-mon[31338]: osdmap e28: 4 total, 4 up, 4 in 2024-04-22T13:17:57.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:17:57.726+0000 7f0e7dc22700 1 -- 172.21.15.111:0/3551113525 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0e78062e80 con 0x7f0e640a3c40 2024-04-22T13:17:57.724 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:57 smithi111 ceph-mon[31361]: osd.3 [v2:172.21.15.12:6810/3359066681,v1:172.21.15.12:6811/3359066681] boot 2024-04-22T13:17:57.724 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:57 smithi111 ceph-mon[31361]: osdmap e27: 4 total, 4 up, 4 in 2024-04-22T13:17:57.724 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:57 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:17:57.724 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:57 smithi111 ceph-mon[31361]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-22T13:17:57.724 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:57 smithi111 ceph-mon[31361]: osdmap e28: 4 total, 4 up, 4 in 2024-04-22T13:17:58.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:58 smithi012 ceph-mon[28265]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:17:58.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:58 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:17:58.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:58 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:17:58.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:58 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:58.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:58 smithi012 ceph-mon[28265]: osdmap e29: 4 total, 4 up, 4 in 2024-04-22T13:17:58.631 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:58 smithi111 ceph-mon[31361]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:17:58.631 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:58 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:17:58.631 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:58 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:17:58.631 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:58 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:58.631 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:58 smithi111 ceph-mon[31361]: osdmap e29: 4 total, 4 up, 4 in 2024-04-22T13:17:58.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:58 smithi134 ceph-mon[31338]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:17:58.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:58 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:17:58.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:58 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:17:58.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:58 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:17:58.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:58 smithi134 ceph-mon[31338]: osdmap e29: 4 total, 4 up, 4 in 2024-04-22T13:17:59.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:59 smithi111 ceph-mon[31361]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:17:59.543 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:17:59 smithi111 ceph-mon[31361]: osdmap e30: 4 total, 4 up, 4 in 2024-04-22T13:17:59.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:59 smithi012 ceph-mon[28265]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:17:59.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:17:59 smithi012 ceph-mon[28265]: osdmap e30: 4 total, 4 up, 4 in 2024-04-22T13:17:59.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:59 smithi134 ceph-mon[31338]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:17:59.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:17:59 smithi134 ceph-mon[31338]: osdmap e30: 4 total, 4 up, 4 in 2024-04-22T13:18:01.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:01 smithi134 ceph-mon[31338]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:18:01.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:01 smithi134 ceph-mon[31338]: osdmap e31: 4 total, 4 up, 4 in 2024-04-22T13:18:01.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:01 smithi134 ceph-mon[31338]: from='client.? 172.21.15.111:0/2247483198' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e29afd7-354c-4506-aaa0-b43eb1a19a83"}]: dispatch 2024-04-22T13:18:01.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:01 smithi134 ceph-mon[31338]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e29afd7-354c-4506-aaa0-b43eb1a19a83"}]: dispatch 2024-04-22T13:18:01.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:01 smithi134 ceph-mon[31338]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e29afd7-354c-4506-aaa0-b43eb1a19a83"}]': finished 2024-04-22T13:18:01.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:01 smithi134 ceph-mon[31338]: osdmap e32: 5 total, 4 up, 5 in 2024-04-22T13:18:01.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:01 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:18:01.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:01 smithi134 ceph-mon[31338]: from='client.? 172.21.15.111:0/3152743345' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:18:01.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:01 smithi111 ceph-mon[31361]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:18:01.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:01 smithi111 ceph-mon[31361]: osdmap e31: 4 total, 4 up, 4 in 2024-04-22T13:18:01.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:01 smithi111 ceph-mon[31361]: from='client.? 172.21.15.111:0/2247483198' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e29afd7-354c-4506-aaa0-b43eb1a19a83"}]: dispatch 2024-04-22T13:18:01.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:01 smithi111 ceph-mon[31361]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e29afd7-354c-4506-aaa0-b43eb1a19a83"}]: dispatch 2024-04-22T13:18:01.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:01 smithi111 ceph-mon[31361]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e29afd7-354c-4506-aaa0-b43eb1a19a83"}]': finished 2024-04-22T13:18:01.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:01 smithi111 ceph-mon[31361]: osdmap e32: 5 total, 4 up, 5 in 2024-04-22T13:18:01.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:01 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:18:01.729 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:01 smithi111 ceph-mon[31361]: from='client.? 172.21.15.111:0/3152743345' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:18:01.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:01 smithi012 ceph-mon[28265]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:18:01.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:01 smithi012 ceph-mon[28265]: osdmap e31: 4 total, 4 up, 4 in 2024-04-22T13:18:01.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:01 smithi012 ceph-mon[28265]: from='client.? 172.21.15.111:0/2247483198' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e29afd7-354c-4506-aaa0-b43eb1a19a83"}]: dispatch 2024-04-22T13:18:01.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:01 smithi012 ceph-mon[28265]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e29afd7-354c-4506-aaa0-b43eb1a19a83"}]: dispatch 2024-04-22T13:18:01.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:01 smithi012 ceph-mon[28265]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e29afd7-354c-4506-aaa0-b43eb1a19a83"}]': finished 2024-04-22T13:18:01.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:01 smithi012 ceph-mon[28265]: osdmap e32: 5 total, 4 up, 5 in 2024-04-22T13:18:01.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:01 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:18:01.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:01 smithi012 ceph-mon[28265]: from='client.? 172.21.15.111:0/3152743345' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:18:03.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:03 smithi134 ceph-mon[31338]: pgmap v111: 1 pgs: 1 remapped+peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:18:03.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:03 smithi111 ceph-mon[31361]: pgmap v111: 1 pgs: 1 remapped+peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:18:03.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:03 smithi012 ceph-mon[28265]: pgmap v111: 1 pgs: 1 remapped+peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:18:05.621 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:05 smithi111 ceph-mon[31361]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-22T13:18:05.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:05 smithi134 ceph-mon[31338]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-22T13:18:05.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:05 smithi012 ceph-mon[28265]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-22T13:18:07.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:07 smithi134 ceph-mon[31338]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-22T13:18:07.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:07 smithi111 ceph-mon[31361]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-22T13:18:07.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:07 smithi012 ceph-mon[28265]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-22T13:18:08.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:08 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-22T13:18:08.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:08 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:08.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:08 smithi111 ceph-mon[31361]: Deploying daemon osd.4 on smithi111 2024-04-22T13:18:08.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:08 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-22T13:18:08.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:08 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:08.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:08 smithi134 ceph-mon[31338]: Deploying daemon osd.4 on smithi111 2024-04-22T13:18:08.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:08 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-22T13:18:08.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:08 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:08.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:08 smithi012 ceph-mon[28265]: Deploying daemon osd.4 on smithi111 2024-04-22T13:18:09.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:09 smithi111 ceph-mon[31361]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-22T13:18:09.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:09 smithi134 ceph-mon[31338]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-22T13:18:09.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:09 smithi012 ceph-mon[28265]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-22T13:18:11.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:11 smithi111 ceph-mon[31361]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-22T13:18:11.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:11 smithi134 ceph-mon[31338]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-22T13:18:11.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:11 smithi012 ceph-mon[28265]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-22T13:18:14.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:14 smithi012 ceph-mon[28265]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-22T13:18:14.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:14 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:14.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:14 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:14.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:14 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:18:14.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:14 smithi134 ceph-mon[31338]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-22T13:18:14.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:14 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:14.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:14 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:14.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:14 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:18:14.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:14 smithi111 ceph-mon[31361]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-22T13:18:14.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:14 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:14.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:14 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:14.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:14 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:18:16.041 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:18:16.043+0000 7f0e757fa700 1 -- 172.21.15.111:0/3551113525 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0e78062e80 con 0x7f0e640a3c40 2024-04-22T13:18:16.042 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 4 on host 'smithi111' 2024-04-22T13:18:16.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:18:16.045+0000 7f0e7dc22700 1 -- 172.21.15.111:0/3551113525 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f0e640a3c40 msgr2=0x7f0e640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:16.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:18:16.045+0000 7f0e7dc22700 1 --2- 172.21.15.111:0/3551113525 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f0e640a3c40 0x7f0e640a6100 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0e78102a00 tx=0x7f0e6800b5b0 comp rx=0 tx=0).stop 2024-04-22T13:18:16.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:18:16.045+0000 7f0e7dc22700 1 -- 172.21.15.111:0/3551113525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e78103540 msgr2=0x7f0e7819fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:16.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:18:16.045+0000 7f0e7dc22700 1 --2- 172.21.15.111:0/3551113525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e78103540 0x7f0e7819fe60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0e70009040 tx=0x7f0e70004e00 comp rx=0 tx=0).stop 2024-04-22T13:18:16.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:18:16.045+0000 7f0e7dc22700 1 -- 172.21.15.111:0/3551113525 shutdown_connections 2024-04-22T13:18:16.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:18:16.045+0000 7f0e7dc22700 1 --2- 172.21.15.111:0/3551113525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e78103540 0x7f0e7819fe60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:16.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:18:16.045+0000 7f0e7dc22700 1 --2- 172.21.15.111:0/3551113525 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f0e640a3c40 0x7f0e640a6100 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:16.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:18:16.045+0000 7f0e7dc22700 1 --2- 172.21.15.111:0/3551113525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e781019a0 0x7f0e7806bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:16.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:18:16.045+0000 7f0e7dc22700 1 --2- 172.21.15.111:0/3551113525 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0e78102ba0 0x7f0e7819d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:16.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:18:16.045+0000 7f0e7dc22700 1 -- 172.21.15.111:0/3551113525 >> 172.21.15.111:0/3551113525 conn(0x7f0e780fcf40 msgr2=0x7f0e78105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:16.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:18:16.045+0000 7f0e7dc22700 1 -- 172.21.15.111:0/3551113525 shutdown_connections 2024-04-22T13:18:16.044 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:18:16.045+0000 7f0e7dc22700 1 -- 172.21.15.111:0/3551113525 wait complete. 2024-04-22T13:18:16.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:16 smithi111 ceph-mon[31361]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-22T13:18:16.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:16 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:16.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:16 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:16.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:16 smithi012 ceph-mon[28265]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-22T13:18:16.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:16 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:16.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:16 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:16.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:16 smithi134 ceph-mon[31338]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-22T13:18:16.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:16 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:16.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:16 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:16.891 DEBUG:teuthology.orchestra.run.smithi111:osd.4> sudo journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.4.service 2024-04-22T13:18:16.893 INFO:tasks.cephadm:Deploying osd.5 on smithi134 with /dev/vg_nvme/lv_3... 2024-04-22T13:18:16.894 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-22T13:18:17.145 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:18:17.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:17 smithi134 ceph-mon[31338]: from='osd.4 [v2:172.21.15.111:6808/2886349872,v1:172.21.15.111:6809/2886349872]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T13:18:17.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:17 smithi134 ceph-mon[31338]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T13:18:17.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:17 smithi111 ceph-mon[31361]: from='osd.4 [v2:172.21.15.111:6808/2886349872,v1:172.21.15.111:6809/2886349872]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T13:18:17.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:17 smithi111 ceph-mon[31361]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T13:18:17.228 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 22 13:18:16 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-4[42939]: 2024-04-22T13:18:16.932+0000 7fd69f6ff7c0 -1 osd.4 0 log_to_monitors true 2024-04-22T13:18:17.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:17 smithi012 ceph-mon[28265]: from='osd.4 [v2:172.21.15.111:6808/2886349872,v1:172.21.15.111:6809/2886349872]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T13:18:17.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:17 smithi012 ceph-mon[28265]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-22T13:18:18.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:18 smithi111 ceph-mon[31361]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:18:18.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:18 smithi111 ceph-mon[31361]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-22T13:18:18.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:18 smithi111 ceph-mon[31361]: osdmap e33: 5 total, 4 up, 5 in 2024-04-22T13:18:18.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:18 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:18:18.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:18 smithi111 ceph-mon[31361]: from='osd.4 [v2:172.21.15.111:6808/2886349872,v1:172.21.15.111:6809/2886349872]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-22T13:18:18.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:18 smithi111 ceph-mon[31361]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-22T13:18:18.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:18 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:18.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:18 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:18.294 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:18 smithi134 ceph-mon[31338]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:18:18.294 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:18 smithi134 ceph-mon[31338]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-22T13:18:18.294 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:18 smithi134 ceph-mon[31338]: osdmap e33: 5 total, 4 up, 5 in 2024-04-22T13:18:18.295 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:18 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:18:18.295 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:18 smithi134 ceph-mon[31338]: from='osd.4 [v2:172.21.15.111:6808/2886349872,v1:172.21.15.111:6809/2886349872]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-22T13:18:18.295 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:18 smithi134 ceph-mon[31338]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-22T13:18:18.295 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:18 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:18.295 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:18 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:18.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:18 smithi012 ceph-mon[28265]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:18:18.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:18 smithi012 ceph-mon[28265]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-22T13:18:18.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:18 smithi012 ceph-mon[28265]: osdmap e33: 5 total, 4 up, 5 in 2024-04-22T13:18:18.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:18 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:18:18.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:18 smithi012 ceph-mon[28265]: from='osd.4 [v2:172.21.15.111:6808/2886349872,v1:172.21.15.111:6809/2886349872]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-22T13:18:18.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:18 smithi012 ceph-mon[28265]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-22T13:18:18.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:18 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:18.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:18 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:18.983 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-22T13:18:18.984 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-22T13:18:18.984 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2024-04-22T13:18:18.984 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2024-04-22T13:18:18.984 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113833 s, 921 MB/s 2024-04-22T13:18:18.984 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2024-04-22T13:18:19.140 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:19 smithi111 ceph-mon[31361]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-22T13:18:19.140 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:19 smithi111 ceph-mon[31361]: osdmap e34: 5 total, 4 up, 5 in 2024-04-22T13:18:19.141 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:19 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:18:19.141 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:19 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:18:19.141 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:19 smithi111 ceph-mon[31361]: from='osd.4 ' entity='osd.4' 2024-04-22T13:18:19.141 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 22 13:18:18 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-4[42939]: 2024-04-22T13:18:18.948+0000 7fd694d64700 -1 osd.4 0 waiting for initial osdmap 2024-04-22T13:18:19.141 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 22 13:18:19 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-4[42939]: 2024-04-22T13:18:19.104+0000 7fd68cb4f700 -1 osd.4 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T13:18:19.328 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:19 smithi134 ceph-mon[31338]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-22T13:18:19.328 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:19 smithi134 ceph-mon[31338]: osdmap e34: 5 total, 4 up, 5 in 2024-04-22T13:18:19.328 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:19 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:18:19.329 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:19 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:18:19.329 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:19 smithi134 ceph-mon[31338]: from='osd.4 ' entity='osd.4' 2024-04-22T13:18:19.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:19 smithi012 ceph-mon[28265]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-22T13:18:19.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:19 smithi012 ceph-mon[28265]: osdmap e34: 5 total, 4 up, 5 in 2024-04-22T13:18:19.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:19 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:18:19.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:19 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:18:19.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:19 smithi012 ceph-mon[28265]: from='osd.4 ' entity='osd.4' 2024-04-22T13:18:19.638 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_3 2024-04-22T13:18:19.872 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:18:20.264 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:20 smithi134 ceph-mon[31338]: purged_snaps scrub starts 2024-04-22T13:18:20.264 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:20 smithi134 ceph-mon[31338]: purged_snaps scrub ok 2024-04-22T13:18:20.264 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:20 smithi134 ceph-mon[31338]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:18:20.264 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:20 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:18:20.264 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:20 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:18:20.311 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:20 smithi111 ceph-mon[31361]: purged_snaps scrub starts 2024-04-22T13:18:20.311 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:20 smithi111 ceph-mon[31361]: purged_snaps scrub ok 2024-04-22T13:18:20.311 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:20 smithi111 ceph-mon[31361]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:18:20.311 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:20 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:18:20.311 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:20 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:18:20.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:20 smithi012 ceph-mon[28265]: purged_snaps scrub starts 2024-04-22T13:18:20.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:20 smithi012 ceph-mon[28265]: purged_snaps scrub ok 2024-04-22T13:18:20.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:20 smithi012 ceph-mon[28265]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:18:20.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:20 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:18:20.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:20 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:18:21.203 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:21 smithi134 ceph-mon[31338]: osd.4 [v2:172.21.15.111:6808/2886349872,v1:172.21.15.111:6809/2886349872] boot 2024-04-22T13:18:21.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:21 smithi134 ceph-mon[31338]: osdmap e35: 5 total, 5 up, 5 in 2024-04-22T13:18:21.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:21 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:18:21.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:21 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:21.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:21 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:21.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:21 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:18:21.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:21 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:18:21.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:21 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:21.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:21 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:21.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:21 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:18:21.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:21 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:21.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:21 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:18:21.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:21 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:21.204 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:21 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:18:21.205 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:21 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:21.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:21 smithi012 ceph-mon[28265]: osd.4 [v2:172.21.15.111:6808/2886349872,v1:172.21.15.111:6809/2886349872] boot 2024-04-22T13:18:21.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:21 smithi012 ceph-mon[28265]: osdmap e35: 5 total, 5 up, 5 in 2024-04-22T13:18:21.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:21 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:18:21.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:21 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:21.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:21 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:21.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:21 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:18:21.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:21 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:18:21.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:21 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:21.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:21 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:21.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:21 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:18:21.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:21 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:21.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:21 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:18:21.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:21 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:21.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:21 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:18:21.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:21 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:21.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.416+0000 7f60ad503700 1 -- 172.21.15.134:0/222184559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60a8102aa0 msgr2=0x7f60a8102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:21.415 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.416+0000 7f60ad503700 1 --2- 172.21.15.134:0/222184559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60a8102aa0 0x7f60a8102ec0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6098009990 tx=0x7f6098009ca0 comp rx=0 tx=0).stop 2024-04-22T13:18:21.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.416+0000 7f60ad503700 1 -- 172.21.15.134:0/222184559 shutdown_connections 2024-04-22T13:18:21.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.416+0000 7f60ad503700 1 --2- 172.21.15.134:0/222184559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60a8104640 0x7f60a810aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:21.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.416+0000 7f60ad503700 1 --2- 172.21.15.134:0/222184559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60a8102aa0 0x7f60a8102ec0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:21.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.416+0000 7f60ad503700 1 --2- 172.21.15.134:0/222184559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60a8103ca0 0x7f60a8104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:21.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.416+0000 7f60ad503700 1 -- 172.21.15.134:0/222184559 >> 172.21.15.134:0/222184559 conn(0x7f60a80fe040 msgr2=0x7f60a8100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:21.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.416+0000 7f60ad503700 1 -- 172.21.15.134:0/222184559 shutdown_connections 2024-04-22T13:18:21.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.416+0000 7f60ad503700 1 -- 172.21.15.134:0/222184559 wait complete. 2024-04-22T13:18:21.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.416+0000 7f60ad503700 1 Processor -- start 2024-04-22T13:18:21.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.416+0000 7f60ad503700 1 -- start start 2024-04-22T13:18:21.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.417+0000 7f60ad503700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60a8102aa0 0x7f60a8198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:21.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.417+0000 7f60ad503700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60a8103ca0 0x7f60a8199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:21.416 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.417+0000 7f60ad503700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60a8104640 0x7f60a81a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:21.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.417+0000 7f60ad503700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f60a810e530 con 0x7f60a8104640 2024-04-22T13:18:21.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.417+0000 7f60ad503700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f60a810e3b0 con 0x7f60a8103ca0 2024-04-22T13:18:21.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.417+0000 7f60ad503700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f60a810e6b0 con 0x7f60a8102aa0 2024-04-22T13:18:21.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.417+0000 7f60a6ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60a8102aa0 0x7f60a8198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:21.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.417+0000 7f60a6ffd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60a8102aa0 0x7f60a8198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:39342/0 (socket says 172.21.15.134:39342) 2024-04-22T13:18:21.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.417+0000 7f60a6ffd700 1 -- 172.21.15.134:0/1003062096 learned_addr learned my addr 172.21.15.134:0/1003062096 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-22T13:18:21.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.417+0000 7f609ffff700 1 --2- 172.21.15.134:0/1003062096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60a8103ca0 0x7f60a8199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:21.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.417+0000 7f60a77fe700 1 --2- 172.21.15.134:0/1003062096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60a8104640 0x7f60a81a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:21.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.417+0000 7f60a6ffd700 1 -- 172.21.15.134:0/1003062096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60a8103ca0 msgr2=0x7f60a8199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:21.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.417+0000 7f60a6ffd700 1 --2- 172.21.15.134:0/1003062096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60a8103ca0 0x7f60a8199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:21.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.417+0000 7f60a6ffd700 1 -- 172.21.15.134:0/1003062096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60a8104640 msgr2=0x7f60a81a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:21.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.417+0000 7f60a6ffd700 1 --2- 172.21.15.134:0/1003062096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60a8104640 0x7f60a81a01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:21.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.417+0000 7f60a6ffd700 1 -- 172.21.15.134:0/1003062096 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a81a08a0 con 0x7f60a8102aa0 2024-04-22T13:18:21.417 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.417+0000 7f60a6ffd700 1 --2- 172.21.15.134:0/1003062096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60a8102aa0 0x7f60a8198c50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6098000c00 tx=0x7f6098009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:21.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.417+0000 7f60a4ff9700 1 -- 172.21.15.134:0/1003062096 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6098027070 con 0x7f60a8102aa0 2024-04-22T13:18:21.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.417+0000 7f60ad503700 1 -- 172.21.15.134:0/1003062096 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a81a0b30 con 0x7f60a8102aa0 2024-04-22T13:18:21.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.417+0000 7f60a4ff9700 1 -- 172.21.15.134:0/1003062096 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60980041c0 con 0x7f60a8102aa0 2024-04-22T13:18:21.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.417+0000 7f60a4ff9700 1 -- 172.21.15.134:0/1003062096 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609800f460 con 0x7f60a8102aa0 2024-04-22T13:18:21.418 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.417+0000 7f60ad503700 1 -- 172.21.15.134:0/1003062096 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a81a0f80 con 0x7f60a8102aa0 2024-04-22T13:18:21.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.418+0000 7f60ad503700 1 -- 172.21.15.134:0/1003062096 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60a8069730 con 0x7f60a8102aa0 2024-04-22T13:18:21.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.419+0000 7f60a4ff9700 1 -- 172.21.15.134:0/1003062096 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f609802a070 con 0x7f60a8102aa0 2024-04-22T13:18:21.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.420+0000 7f60a4ff9700 1 --2- 172.21.15.134:0/1003062096 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f60940a3c40 0x7f60940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:21.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.420+0000 7f60a4ff9700 1 -- 172.21.15.134:0/1003062096 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7f60980d0320 con 0x7f60a8102aa0 2024-04-22T13:18:21.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.420+0000 7f609ffff700 1 --2- 172.21.15.134:0/1003062096 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f60940a3c40 0x7f60940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:21.419 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.421+0000 7f609ffff700 1 --2- 172.21.15.134:0/1003062096 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f60940a3c40 0x7f60940a6100 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6090005950 tx=0x7f60900058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:21.422 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.425+0000 7f60a4ff9700 1 -- 172.21.15.134:0/1003062096 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f609805eff0 con 0x7f60a8102aa0 2024-04-22T13:18:21.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:21 smithi111 ceph-mon[31361]: osd.4 [v2:172.21.15.111:6808/2886349872,v1:172.21.15.111:6809/2886349872] boot 2024-04-22T13:18:21.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:21 smithi111 ceph-mon[31361]: osdmap e35: 5 total, 5 up, 5 in 2024-04-22T13:18:21.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:21 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:18:21.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:21 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:21.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:21 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:21.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:21 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:18:21.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:21 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:18:21.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:21 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:21.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:21 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:21.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:21 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:18:21.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:21 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:21.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:21 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:18:21.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:21 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:21.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:21 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:18:21.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:21 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:21.628 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:21.630+0000 7f60ad503700 1 -- 172.21.15.134:0/1003062096 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f60a8062e80 con 0x7f60940a3c40 2024-04-22T13:18:22.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:22 smithi012 ceph-mon[28265]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:18:22.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:22 smithi012 ceph-mon[28265]: Detected new or changed devices on smithi111 2024-04-22T13:18:22.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:22 smithi012 ceph-mon[28265]: Adjusting osd_memory_target on smithi111 to 8544M 2024-04-22T13:18:22.340 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:22 smithi012 ceph-mon[28265]: osdmap e36: 5 total, 5 up, 5 in 2024-04-22T13:18:22.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:22 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:18:22.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:22 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:18:22.341 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:22 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:22.351 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:22 smithi134 ceph-mon[31338]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:18:22.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:22 smithi134 ceph-mon[31338]: Detected new or changed devices on smithi111 2024-04-22T13:18:22.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:22 smithi134 ceph-mon[31338]: Adjusting osd_memory_target on smithi111 to 8544M 2024-04-22T13:18:22.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:22 smithi134 ceph-mon[31338]: osdmap e36: 5 total, 5 up, 5 in 2024-04-22T13:18:22.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:22 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:18:22.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:22 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:18:22.353 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:22 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:22.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:22 smithi111 ceph-mon[31361]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-22T13:18:22.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:22 smithi111 ceph-mon[31361]: Detected new or changed devices on smithi111 2024-04-22T13:18:22.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:22 smithi111 ceph-mon[31361]: Adjusting osd_memory_target on smithi111 to 8544M 2024-04-22T13:18:22.480 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:22 smithi111 ceph-mon[31361]: osdmap e36: 5 total, 5 up, 5 in 2024-04-22T13:18:22.480 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:22 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:18:22.481 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:22 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:18:22.481 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:22 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:23.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:23 smithi012 ceph-mon[28265]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:18:23.349 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:23 smithi134 ceph-mon[31338]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:18:23.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:23 smithi111 ceph-mon[31361]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:18:24.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:24 smithi012 ceph-mon[28265]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:24.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:24 smithi134 ceph-mon[31338]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:24.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:24 smithi111 ceph-mon[31361]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:25.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:25 smithi134 ceph-mon[31338]: from='client.? 172.21.15.134:0/2610180659' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f40f02d-0ba2-4842-a03d-2a1a04d1010d"}]: dispatch 2024-04-22T13:18:25.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:25 smithi134 ceph-mon[31338]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f40f02d-0ba2-4842-a03d-2a1a04d1010d"}]: dispatch 2024-04-22T13:18:25.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:25 smithi134 ceph-mon[31338]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f40f02d-0ba2-4842-a03d-2a1a04d1010d"}]': finished 2024-04-22T13:18:25.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:25 smithi134 ceph-mon[31338]: osdmap e37: 6 total, 5 up, 6 in 2024-04-22T13:18:25.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:25 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:18:25.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:25 smithi134 ceph-mon[31338]: from='client.? 172.21.15.134:0/2320825055' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:18:25.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:25 smithi111 ceph-mon[31361]: from='client.? 172.21.15.134:0/2610180659' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f40f02d-0ba2-4842-a03d-2a1a04d1010d"}]: dispatch 2024-04-22T13:18:25.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:25 smithi111 ceph-mon[31361]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f40f02d-0ba2-4842-a03d-2a1a04d1010d"}]: dispatch 2024-04-22T13:18:25.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:25 smithi111 ceph-mon[31361]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f40f02d-0ba2-4842-a03d-2a1a04d1010d"}]': finished 2024-04-22T13:18:25.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:25 smithi111 ceph-mon[31361]: osdmap e37: 6 total, 5 up, 6 in 2024-04-22T13:18:25.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:25 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:18:25.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:25 smithi111 ceph-mon[31361]: from='client.? 172.21.15.134:0/2320825055' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:18:25.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:25 smithi012 ceph-mon[28265]: from='client.? 172.21.15.134:0/2610180659' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f40f02d-0ba2-4842-a03d-2a1a04d1010d"}]: dispatch 2024-04-22T13:18:25.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:25 smithi012 ceph-mon[28265]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f40f02d-0ba2-4842-a03d-2a1a04d1010d"}]: dispatch 2024-04-22T13:18:25.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:25 smithi012 ceph-mon[28265]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f40f02d-0ba2-4842-a03d-2a1a04d1010d"}]': finished 2024-04-22T13:18:25.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:25 smithi012 ceph-mon[28265]: osdmap e37: 6 total, 5 up, 6 in 2024-04-22T13:18:25.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:25 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:18:25.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:25 smithi012 ceph-mon[28265]: from='client.? 172.21.15.134:0/2320825055' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:18:26.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:26 smithi134 ceph-mon[31338]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:26.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:26 smithi111 ceph-mon[31361]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:26.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:26 smithi012 ceph-mon[28265]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:28.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:28 smithi134 ceph-mon[31338]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:28.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:28 smithi111 ceph-mon[31361]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:28.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:28 smithi012 ceph-mon[28265]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:30.375 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:30 smithi134 ceph-mon[31338]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:30.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:30 smithi111 ceph-mon[31361]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:30.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:30 smithi012 ceph-mon[28265]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:32.358 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:32 smithi134 ceph-mon[31338]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:32.359 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:32 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-22T13:18:32.359 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:32 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:32.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:32 smithi111 ceph-mon[31361]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:32.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:32 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-22T13:18:32.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:32 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:32.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:32 smithi012 ceph-mon[28265]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:32.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:32 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-22T13:18:32.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:32 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:33.256 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:33 smithi134 ceph-mon[31338]: Deploying daemon osd.5 on smithi134 2024-04-22T13:18:33.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:33 smithi111 ceph-mon[31361]: Deploying daemon osd.5 on smithi134 2024-04-22T13:18:33.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:33 smithi012 ceph-mon[28265]: Deploying daemon osd.5 on smithi134 2024-04-22T13:18:34.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:34 smithi134 ceph-mon[31338]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:34.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:34 smithi111 ceph-mon[31361]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:34.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:34 smithi012 ceph-mon[28265]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:36.251 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:36 smithi134 ceph-mon[31338]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:36.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:36 smithi111 ceph-mon[31361]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:36.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:36 smithi012 ceph-mon[28265]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:38.077 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:37 smithi134 ceph-mon[31338]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:38.077 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:37 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:38.078 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:37 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:38.078 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:37 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:18:38.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:37 smithi012 ceph-mon[28265]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:38.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:37 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:38.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:37 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:38.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:37 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:18:38.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:37 smithi111 ceph-mon[31361]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:38.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:37 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:38.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:37 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:38.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:37 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:18:40.154 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:40 smithi134 ceph-mon[31338]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:40.154 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:40 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:40.155 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:40 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:40.155 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:40 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:40.155 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:40 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:18:40.155 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:40 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:40.247 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 5 on host 'smithi134' 2024-04-22T13:18:40.247 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:40.248+0000 7f60a4ff9700 1 -- 172.21.15.134:0/1003062096 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f60a8062e80 con 0x7f60940a3c40 2024-04-22T13:18:40.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:40.250+0000 7f60ad503700 1 -- 172.21.15.134:0/1003062096 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f60940a3c40 msgr2=0x7f60940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:40.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:40.250+0000 7f60ad503700 1 --2- 172.21.15.134:0/1003062096 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f60940a3c40 0x7f60940a6100 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6090005950 tx=0x7f60900058e0 comp rx=0 tx=0).stop 2024-04-22T13:18:40.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:40.250+0000 7f60ad503700 1 -- 172.21.15.134:0/1003062096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60a8102aa0 msgr2=0x7f60a8198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:40.248 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:40.250+0000 7f60ad503700 1 --2- 172.21.15.134:0/1003062096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60a8102aa0 0x7f60a8198c50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6098000c00 tx=0x7f6098009580 comp rx=0 tx=0).stop 2024-04-22T13:18:40.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:40.251+0000 7f60ad503700 1 -- 172.21.15.134:0/1003062096 shutdown_connections 2024-04-22T13:18:40.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:40.251+0000 7f60ad503700 1 --2- 172.21.15.134:0/1003062096 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f60940a3c40 0x7f60940a6100 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:40.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:40.251+0000 7f60ad503700 1 --2- 172.21.15.134:0/1003062096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60a8104640 0x7f60a81a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:40.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:40.251+0000 7f60ad503700 1 --2- 172.21.15.134:0/1003062096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60a8102aa0 0x7f60a8198c50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:40.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:40.251+0000 7f60ad503700 1 --2- 172.21.15.134:0/1003062096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60a8103ca0 0x7f60a8199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:40.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:40.251+0000 7f60ad503700 1 -- 172.21.15.134:0/1003062096 >> 172.21.15.134:0/1003062096 conn(0x7f60a80fe040 msgr2=0x7f60a8107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:40.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:40.251+0000 7f60ad503700 1 -- 172.21.15.134:0/1003062096 shutdown_connections 2024-04-22T13:18:40.249 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:18:40.251+0000 7f60ad503700 1 -- 172.21.15.134:0/1003062096 wait complete. 2024-04-22T13:18:40.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:40 smithi012 ceph-mon[28265]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:40.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:40 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:40.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:40 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:40.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:40 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:40.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:40 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:18:40.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:40 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:40.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:40 smithi111 ceph-mon[31361]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:40.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:40 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:40.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:40 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:40.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:40 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:40.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:40 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:18:40.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:40 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:40.888 DEBUG:teuthology.orchestra.run.smithi134:osd.5> sudo journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.5.service 2024-04-22T13:18:40.890 INFO:tasks.cephadm:Deploying osd.6 on smithi012 with /dev/vg_nvme/lv_2... 2024-04-22T13:18:40.890 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-22T13:18:41.146 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:18:41.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:41 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:41.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:41 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:18:41.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:41 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:41.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:41 smithi134 ceph-mon[31338]: from='osd.5 [v2:172.21.15.134:6808/3071977602,v1:172.21.15.134:6809/3071977602]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T13:18:41.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:41 smithi134 ceph-mon[31338]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T13:18:41.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:41 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:41.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:41 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:18:41.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:41 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:41.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:41 smithi111 ceph-mon[31361]: from='osd.5 [v2:172.21.15.134:6808/3071977602,v1:172.21.15.134:6809/3071977602]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T13:18:41.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:41 smithi111 ceph-mon[31361]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T13:18:41.499 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:41 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:41.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:41 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:18:41.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:41 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:41.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:41 smithi012 ceph-mon[28265]: from='osd.5 [v2:172.21.15.134:6808/3071977602,v1:172.21.15.134:6809/3071977602]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T13:18:41.500 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:41 smithi012 ceph-mon[28265]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-22T13:18:42.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:42 smithi111 ceph-mon[31361]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:42.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:42 smithi111 ceph-mon[31361]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-22T13:18:42.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:42 smithi111 ceph-mon[31361]: from='osd.5 [v2:172.21.15.134:6808/3071977602,v1:172.21.15.134:6809/3071977602]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-22T13:18:42.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:42 smithi111 ceph-mon[31361]: osdmap e38: 6 total, 5 up, 6 in 2024-04-22T13:18:42.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:42 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:18:42.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:42 smithi111 ceph-mon[31361]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-22T13:18:42.506 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:42 smithi012 ceph-mon[28265]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:42.506 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:42 smithi012 ceph-mon[28265]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-22T13:18:42.507 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:42 smithi012 ceph-mon[28265]: from='osd.5 [v2:172.21.15.134:6808/3071977602,v1:172.21.15.134:6809/3071977602]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-22T13:18:42.507 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:42 smithi012 ceph-mon[28265]: osdmap e38: 6 total, 5 up, 6 in 2024-04-22T13:18:42.507 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:42 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:18:42.507 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:42 smithi012 ceph-mon[28265]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-22T13:18:42.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:42 smithi134 ceph-mon[31338]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:42.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:42 smithi134 ceph-mon[31338]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-22T13:18:42.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:42 smithi134 ceph-mon[31338]: from='osd.5 [v2:172.21.15.134:6808/3071977602,v1:172.21.15.134:6809/3071977602]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-22T13:18:42.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:42 smithi134 ceph-mon[31338]: osdmap e38: 6 total, 5 up, 6 in 2024-04-22T13:18:42.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:42 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:18:42.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:42 smithi134 ceph-mon[31338]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-22T13:18:42.680 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:18:42 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:18:42.219+0000 7fcf4d936700 -1 osd.5 0 waiting for initial osdmap 2024-04-22T13:18:42.680 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:18:42 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:18:42.229+0000 7fcf49729700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T13:18:43.104 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-22T13:18:43.104 INFO:teuthology.orchestra.run.smithi012.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-22T13:18:43.104 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10+0 records in 2024-04-22T13:18:43.104 INFO:teuthology.orchestra.run.smithi012.stderr:10+0 records out 2024-04-22T13:18:43.104 INFO:teuthology.orchestra.run.smithi012.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110986 s, 945 MB/s 2024-04-22T13:18:43.105 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 2024-04-22T13:18:43.105 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping successful for: 2024-04-22T13:18:43.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:43 smithi012 ceph-mon[28265]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-22T13:18:43.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:43 smithi012 ceph-mon[28265]: osdmap e39: 6 total, 5 up, 6 in 2024-04-22T13:18:43.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:18:43.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:18:43.414 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:43 smithi012 ceph-mon[28265]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:43.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:43 smithi134 ceph-mon[31338]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-22T13:18:43.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:43 smithi134 ceph-mon[31338]: osdmap e39: 6 total, 5 up, 6 in 2024-04-22T13:18:43.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:43 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:18:43.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:43 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:18:43.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:43 smithi134 ceph-mon[31338]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:43.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:43 smithi111 ceph-mon[31361]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-22T13:18:43.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:43 smithi111 ceph-mon[31361]: osdmap e39: 6 total, 5 up, 6 in 2024-04-22T13:18:43.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:43 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:18:43.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:43 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:18:43.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:43 smithi111 ceph-mon[31361]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-22T13:18:43.755 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi012:vg_nvme/lv_2 2024-04-22T13:18:44.001 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:18:44.420 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:44 smithi012 ceph-mon[28265]: purged_snaps scrub starts 2024-04-22T13:18:44.420 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:44 smithi012 ceph-mon[28265]: purged_snaps scrub ok 2024-04-22T13:18:44.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:44 smithi012 ceph-mon[28265]: osd.5 [v2:172.21.15.134:6808/3071977602,v1:172.21.15.134:6809/3071977602] boot 2024-04-22T13:18:44.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:44 smithi012 ceph-mon[28265]: osdmap e40: 6 total, 6 up, 6 in 2024-04-22T13:18:44.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:44 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:18:44.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:44 smithi012 ceph-mon[28265]: Detected new or changed devices on smithi134 2024-04-22T13:18:44.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:44 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:44.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:44 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:44.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:44 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:18:44.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:44 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:18:44.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:44 smithi012 ceph-mon[28265]: Adjusting osd_memory_target on smithi134 to 8544M 2024-04-22T13:18:44.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:44 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:44.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:44 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:44.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:44 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:18:44.421 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:44 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:44.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:44 smithi111 ceph-mon[31361]: purged_snaps scrub starts 2024-04-22T13:18:44.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:44 smithi111 ceph-mon[31361]: purged_snaps scrub ok 2024-04-22T13:18:44.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:44 smithi111 ceph-mon[31361]: osd.5 [v2:172.21.15.134:6808/3071977602,v1:172.21.15.134:6809/3071977602] boot 2024-04-22T13:18:44.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:44 smithi111 ceph-mon[31361]: osdmap e40: 6 total, 6 up, 6 in 2024-04-22T13:18:44.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:44 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:18:44.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:44 smithi111 ceph-mon[31361]: Detected new or changed devices on smithi134 2024-04-22T13:18:44.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:44 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:44.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:44 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:44.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:44 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:18:44.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:44 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:18:44.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:44 smithi111 ceph-mon[31361]: Adjusting osd_memory_target on smithi134 to 8544M 2024-04-22T13:18:44.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:44 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:44.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:44 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:44.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:44 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:18:44.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:44 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:44.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:44 smithi134 ceph-mon[31338]: purged_snaps scrub starts 2024-04-22T13:18:44.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:44 smithi134 ceph-mon[31338]: purged_snaps scrub ok 2024-04-22T13:18:44.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:44 smithi134 ceph-mon[31338]: osd.5 [v2:172.21.15.134:6808/3071977602,v1:172.21.15.134:6809/3071977602] boot 2024-04-22T13:18:44.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:44 smithi134 ceph-mon[31338]: osdmap e40: 6 total, 6 up, 6 in 2024-04-22T13:18:44.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:44 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:18:44.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:44 smithi134 ceph-mon[31338]: Detected new or changed devices on smithi134 2024-04-22T13:18:44.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:44 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:44.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:44 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:44.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:44 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:18:44.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:44 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:18:44.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:44 smithi134 ceph-mon[31338]: Adjusting osd_memory_target on smithi134 to 8544M 2024-04-22T13:18:44.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:44 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:44.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:44 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:44.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:44 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:18:44.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:44 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:18:45.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:45 smithi012 ceph-mon[28265]: osdmap e41: 6 total, 6 up, 6 in 2024-04-22T13:18:45.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:45 smithi012 ceph-mon[28265]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:18:45.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:45 smithi012 ceph-mon[28265]: osdmap e42: 6 total, 6 up, 6 in 2024-04-22T13:18:45.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.563+0000 7f8104643700 1 -- 172.21.15.12:0/3723089599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80fc102ac0 msgr2=0x7f80fc102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:45.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.563+0000 7f8104643700 1 --2- 172.21.15.12:0/3723089599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80fc102ac0 0x7f80fc102ee0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f80f80099c0 tx=0x7f80f8009cd0 comp rx=0 tx=0).stop 2024-04-22T13:18:45.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.563+0000 7f8104643700 1 -- 172.21.15.12:0/3723089599 shutdown_connections 2024-04-22T13:18:45.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.563+0000 7f8104643700 1 --2- 172.21.15.12:0/3723089599 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80fc104660 0x7f80fc10af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:45.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.563+0000 7f8104643700 1 --2- 172.21.15.12:0/3723089599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80fc102ac0 0x7f80fc102ee0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:45.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.563+0000 7f8104643700 1 --2- 172.21.15.12:0/3723089599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80fc103cc0 0x7f80fc104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:45.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.563+0000 7f8104643700 1 -- 172.21.15.12:0/3723089599 >> 172.21.15.12:0/3723089599 conn(0x7f80fc0fe060 msgr2=0x7f80fc1004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:18:45.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.563+0000 7f8104643700 1 -- 172.21.15.12:0/3723089599 shutdown_connections 2024-04-22T13:18:45.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.563+0000 7f8104643700 1 -- 172.21.15.12:0/3723089599 wait complete. 2024-04-22T13:18:45.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.564+0000 7f8104643700 1 Processor -- start 2024-04-22T13:18:45.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.564+0000 7f8104643700 1 -- start start 2024-04-22T13:18:45.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.564+0000 7f8104643700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80fc102ac0 0x7f80fc198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:45.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.564+0000 7f8104643700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80fc103cc0 0x7f80fc199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:45.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.564+0000 7f8104643700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80fc104660 0x7f80fc1a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:45.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.564+0000 7f8104643700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc10e600 con 0x7f80fc104660 2024-04-22T13:18:45.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.564+0000 7f8104643700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc10e480 con 0x7f80fc102ac0 2024-04-22T13:18:45.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.564+0000 7f8104643700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc10e780 con 0x7f80fc103cc0 2024-04-22T13:18:45.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.565+0000 7f8102be0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80fc104660 0x7f80fc1a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:45.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.565+0000 7f81023df700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80fc102ac0 0x7f80fc198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:45.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.565+0000 7f8102be0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80fc104660 0x7f80fc1a0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52586/0 (socket says 172.21.15.12:52586) 2024-04-22T13:18:45.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.565+0000 7f8102be0700 1 -- 172.21.15.12:0/4227384874 learned_addr learned my addr 172.21.15.12:0/4227384874 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:18:45.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.565+0000 7f8101bde700 1 --2- 172.21.15.12:0/4227384874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80fc103cc0 0x7f80fc199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:45.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.565+0000 7f8102be0700 1 -- 172.21.15.12:0/4227384874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80fc103cc0 msgr2=0x7f80fc199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:45.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.565+0000 7f8102be0700 1 --2- 172.21.15.12:0/4227384874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80fc103cc0 0x7f80fc199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:45.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.565+0000 7f8102be0700 1 -- 172.21.15.12:0/4227384874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80fc102ac0 msgr2=0x7f80fc198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:18:45.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.565+0000 7f8102be0700 1 --2- 172.21.15.12:0/4227384874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80fc102ac0 0x7f80fc198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:18:45.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.565+0000 7f8102be0700 1 -- 172.21.15.12:0/4227384874 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80fc1a0940 con 0x7f80fc104660 2024-04-22T13:18:45.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.565+0000 7f81023df700 1 --2- 172.21.15.12:0/4227384874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80fc102ac0 0x7f80fc198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:18:45.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.565+0000 7f8102be0700 1 --2- 172.21.15.12:0/4227384874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80fc104660 0x7f80fc1a0270 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f80f400ada0 tx=0x7f80f400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:45.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.565+0000 7f80f37fe700 1 -- 172.21.15.12:0/4227384874 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f4017020 con 0x7f80fc104660 2024-04-22T13:18:45.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.565+0000 7f80f37fe700 1 -- 172.21.15.12:0/4227384874 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80f400ce30 con 0x7f80fc104660 2024-04-22T13:18:45.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.565+0000 7f8104643700 1 -- 172.21.15.12:0/4227384874 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80fc074cd0 con 0x7f80fc104660 2024-04-22T13:18:45.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.565+0000 7f80f37fe700 1 -- 172.21.15.12:0/4227384874 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f4005350 con 0x7f80fc104660 2024-04-22T13:18:45.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.565+0000 7f8104643700 1 -- 172.21.15.12:0/4227384874 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80fc075190 con 0x7f80fc104660 2024-04-22T13:18:45.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.566+0000 7f8104643700 1 -- 172.21.15.12:0/4227384874 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80fc069730 con 0x7f80fc104660 2024-04-22T13:18:45.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.568+0000 7f80f37fe700 1 -- 172.21.15.12:0/4227384874 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f80f400c980 con 0x7f80fc104660 2024-04-22T13:18:45.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.569+0000 7f80f37fe700 1 --2- 172.21.15.12:0/4227384874 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f80e80a3c40 0x7f80e80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:18:45.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.569+0000 7f81023df700 1 --2- 172.21.15.12:0/4227384874 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f80e80a3c40 0x7f80e80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:18:45.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.569+0000 7f80f37fe700 1 -- 172.21.15.12:0/4227384874 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4562+0+0 (secure 0 0 0) 0x7f80f40cf270 con 0x7f80fc104660 2024-04-22T13:18:45.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.569+0000 7f81023df700 1 --2- 172.21.15.12:0/4227384874 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f80e80a3c40 0x7f80e80a6100 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f80f8009f50 tx=0x7f80f8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:18:45.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.573+0000 7f80f37fe700 1 -- 172.21.15.12:0/4227384874 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f80f4021020 con 0x7f80fc104660 2024-04-22T13:18:45.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:45 smithi134 ceph-mon[31338]: osdmap e41: 6 total, 6 up, 6 in 2024-04-22T13:18:45.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:45 smithi134 ceph-mon[31338]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:18:45.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:45 smithi134 ceph-mon[31338]: osdmap e42: 6 total, 6 up, 6 in 2024-04-22T13:18:45.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:45 smithi111 ceph-mon[31361]: osdmap e41: 6 total, 6 up, 6 in 2024-04-22T13:18:45.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:45 smithi111 ceph-mon[31361]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:18:45.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:45 smithi111 ceph-mon[31361]: osdmap e42: 6 total, 6 up, 6 in 2024-04-22T13:18:45.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:18:45.794+0000 7f8104643700 1 -- 172.21.15.12:0/4227384874 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f80fc062f40 con 0x7f80e80a3c40 2024-04-22T13:18:46.550 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:46 smithi012 ceph-mon[28265]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:18:46.550 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:46 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:18:46.550 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:46 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:18:46.550 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:46 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:46.580 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:46 smithi134 ceph-mon[31338]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:18:46.580 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:46 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:18:46.580 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:46 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:18:46.580 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:46 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:46.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:46 smithi111 ceph-mon[31361]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:18:46.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:46 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:18:46.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:46 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:18:46.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:46 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:47.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:47 smithi012 ceph-mon[28265]: osdmap e43: 6 total, 6 up, 6 in 2024-04-22T13:18:47.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:47 smithi012 ceph-mon[28265]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:18:47.629 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:47 smithi111 ceph-mon[31361]: osdmap e43: 6 total, 6 up, 6 in 2024-04-22T13:18:47.629 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:47 smithi111 ceph-mon[31361]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:18:47.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:47 smithi134 ceph-mon[31338]: osdmap e43: 6 total, 6 up, 6 in 2024-04-22T13:18:47.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:47 smithi134 ceph-mon[31338]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:18:48.244 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:48 smithi012 ceph-mon[28265]: osdmap e44: 6 total, 6 up, 6 in 2024-04-22T13:18:48.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:48 smithi134 ceph-mon[31338]: osdmap e44: 6 total, 6 up, 6 in 2024-04-22T13:18:48.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:48 smithi111 ceph-mon[31361]: osdmap e44: 6 total, 6 up, 6 in 2024-04-22T13:18:49.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:49 smithi012 ceph-mon[28265]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-04-22T13:18:49.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:49 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/7643910' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb0c17a9-6a8e-4be6-8660-f73cfa8200ca"}]: dispatch 2024-04-22T13:18:49.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:49 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/7643910' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb0c17a9-6a8e-4be6-8660-f73cfa8200ca"}]': finished 2024-04-22T13:18:49.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:49 smithi012 ceph-mon[28265]: osdmap e45: 7 total, 6 up, 7 in 2024-04-22T13:18:49.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:49 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:18:49.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:49 smithi134 ceph-mon[31338]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-04-22T13:18:49.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:49 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/7643910' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb0c17a9-6a8e-4be6-8660-f73cfa8200ca"}]: dispatch 2024-04-22T13:18:49.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:49 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/7643910' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb0c17a9-6a8e-4be6-8660-f73cfa8200ca"}]': finished 2024-04-22T13:18:49.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:49 smithi134 ceph-mon[31338]: osdmap e45: 7 total, 6 up, 7 in 2024-04-22T13:18:49.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:49 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:18:49.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:49 smithi111 ceph-mon[31361]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-04-22T13:18:49.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:49 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/7643910' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bb0c17a9-6a8e-4be6-8660-f73cfa8200ca"}]: dispatch 2024-04-22T13:18:49.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:49 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/7643910' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bb0c17a9-6a8e-4be6-8660-f73cfa8200ca"}]': finished 2024-04-22T13:18:49.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:49 smithi111 ceph-mon[31361]: osdmap e45: 7 total, 6 up, 7 in 2024-04-22T13:18:49.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:49 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:18:50.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:50 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1900176817' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:18:50.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:50 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1900176817' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:18:50.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:50 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1900176817' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:18:51.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:51 smithi012 ceph-mon[28265]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-04-22T13:18:51.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:51 smithi134 ceph-mon[31338]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-04-22T13:18:51.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:51 smithi111 ceph-mon[31361]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-04-22T13:18:53.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:53 smithi134 ceph-mon[31338]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-22T13:18:53.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:53 smithi111 ceph-mon[31361]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-22T13:18:53.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:53 smithi012 ceph-mon[28265]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-22T13:18:55.668 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:55 smithi012 ceph-mon[28265]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-22T13:18:55.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:55 smithi134 ceph-mon[31338]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-22T13:18:55.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:55 smithi111 ceph-mon[31361]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-22T13:18:57.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:57 smithi012 ceph-mon[28265]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-22T13:18:57.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:57 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-22T13:18:57.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:57 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:57.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:57 smithi012 ceph-mon[28265]: Deploying daemon osd.6 on smithi012 2024-04-22T13:18:57.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:57 smithi134 ceph-mon[31338]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-22T13:18:57.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:57 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-22T13:18:57.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:57 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:57.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:57 smithi134 ceph-mon[31338]: Deploying daemon osd.6 on smithi012 2024-04-22T13:18:57.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:57 smithi111 ceph-mon[31361]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-22T13:18:57.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:57 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-22T13:18:57.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:57 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:18:57.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:57 smithi111 ceph-mon[31361]: Deploying daemon osd.6 on smithi012 2024-04-22T13:18:59.568 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:18:59 smithi012 ceph-mon[28265]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:18:59.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:18:59 smithi134 ceph-mon[31338]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:18:59.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:18:59 smithi111 ceph-mon[31361]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:19:01.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:01 smithi012 ceph-mon[28265]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:19:01.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:01 smithi134 ceph-mon[31338]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:19:01.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:01 smithi111 ceph-mon[31361]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:19:02.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:02 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:02.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:02 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:02.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:02 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:02.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:02 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:02.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:02 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:02.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:02 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:02.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:02 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:02.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:02 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:02.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:02 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:03.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:03 smithi012 ceph-mon[28265]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:19:03.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:03 smithi134 ceph-mon[31338]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:19:03.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:03 smithi111 ceph-mon[31361]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:19:05.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:05 smithi134 ceph-mon[31338]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:19:05.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:05 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:05.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:05 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:05.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:05 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:05.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:05 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:05.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:05 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:05.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:05 smithi111 ceph-mon[31361]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:19:05.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:05 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:05.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:05 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:05.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:05 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:05.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:05 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:05.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:05 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:05.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:05 smithi012 ceph-mon[28265]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:19:05.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:05 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:05.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:05 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:05.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:05 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:05.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:05 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:05.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:05 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:06.256 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 6 on host 'smithi012' 2024-04-22T13:19:06.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:19:06.259+0000 7f80f37fe700 1 -- 172.21.15.12:0/4227384874 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f80fc062f40 con 0x7f80e80a3c40 2024-04-22T13:19:06.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:19:06.261+0000 7f8104643700 1 -- 172.21.15.12:0/4227384874 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f80e80a3c40 msgr2=0x7f80e80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:06.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:19:06.261+0000 7f8104643700 1 --2- 172.21.15.12:0/4227384874 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f80e80a3c40 0x7f80e80a6100 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f80f8009f50 tx=0x7f80f8000bc0 comp rx=0 tx=0).stop 2024-04-22T13:19:06.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:19:06.261+0000 7f8104643700 1 -- 172.21.15.12:0/4227384874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80fc104660 msgr2=0x7f80fc1a0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:06.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:19:06.261+0000 7f8104643700 1 --2- 172.21.15.12:0/4227384874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80fc104660 0x7f80fc1a0270 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f80f400ada0 tx=0x7f80f400c330 comp rx=0 tx=0).stop 2024-04-22T13:19:06.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:19:06.261+0000 7f8104643700 1 -- 172.21.15.12:0/4227384874 shutdown_connections 2024-04-22T13:19:06.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:19:06.261+0000 7f8104643700 1 --2- 172.21.15.12:0/4227384874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80fc102ac0 0x7f80fc198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:06.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:19:06.261+0000 7f8104643700 1 --2- 172.21.15.12:0/4227384874 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f80e80a3c40 0x7f80e80a6100 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:06.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:19:06.261+0000 7f8104643700 1 --2- 172.21.15.12:0/4227384874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f80fc104660 0x7f80fc1a0270 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:06.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:19:06.261+0000 7f8104643700 1 --2- 172.21.15.12:0/4227384874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80fc103cc0 0x7f80fc199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:06.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:19:06.261+0000 7f8104643700 1 -- 172.21.15.12:0/4227384874 >> 172.21.15.12:0/4227384874 conn(0x7f80fc0fe060 msgr2=0x7f80fc106fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:19:06.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:19:06.261+0000 7f8104643700 1 -- 172.21.15.12:0/4227384874 shutdown_connections 2024-04-22T13:19:06.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:19:06.261+0000 7f8104643700 1 -- 172.21.15.12:0/4227384874 wait complete. 2024-04-22T13:19:06.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:06 smithi134 ceph-mon[31338]: from='osd.6 [v2:172.21.15.12:6818/3819619299,v1:172.21.15.12:6819/3819619299]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-22T13:19:06.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:06 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:06.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:06 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:06.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:06 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:06.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:06 smithi111 ceph-mon[31361]: from='osd.6 [v2:172.21.15.12:6818/3819619299,v1:172.21.15.12:6819/3819619299]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-22T13:19:06.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:06 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:06.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:06 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:06.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:06 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:06.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:06 smithi012 ceph-mon[28265]: from='osd.6 [v2:172.21.15.12:6818/3819619299,v1:172.21.15.12:6819/3819619299]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-22T13:19:06.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:06 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:06.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:06 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:06.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:06 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:06.992 DEBUG:teuthology.orchestra.run.smithi012:osd.6> sudo journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.6.service 2024-04-22T13:19:06.995 INFO:tasks.cephadm:Deploying osd.7 on smithi111 with /dev/vg_nvme/lv_2... 2024-04-22T13:19:06.995 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-22T13:19:07.239 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T13:19:07.535 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:07 smithi012 ceph-mon[28265]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:19:07.535 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:07 smithi012 ceph-mon[28265]: from='osd.6 [v2:172.21.15.12:6818/3819619299,v1:172.21.15.12:6819/3819619299]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-22T13:19:07.535 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:07 smithi012 ceph-mon[28265]: osdmap e46: 7 total, 6 up, 7 in 2024-04-22T13:19:07.535 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:07 smithi012 ceph-mon[28265]: from='osd.6 [v2:172.21.15.12:6818/3819619299,v1:172.21.15.12:6819/3819619299]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-22T13:19:07.535 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:07 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:19:07.678 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:07 smithi111 ceph-mon[31361]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:19:07.678 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:07 smithi111 ceph-mon[31361]: from='osd.6 [v2:172.21.15.12:6818/3819619299,v1:172.21.15.12:6819/3819619299]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-22T13:19:07.678 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:07 smithi111 ceph-mon[31361]: osdmap e46: 7 total, 6 up, 7 in 2024-04-22T13:19:07.678 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:07 smithi111 ceph-mon[31361]: from='osd.6 [v2:172.21.15.12:6818/3819619299,v1:172.21.15.12:6819/3819619299]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-22T13:19:07.678 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:07 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:19:07.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:07 smithi134 ceph-mon[31338]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:19:07.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:07 smithi134 ceph-mon[31338]: from='osd.6 [v2:172.21.15.12:6818/3819619299,v1:172.21.15.12:6819/3819619299]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-22T13:19:07.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:07 smithi134 ceph-mon[31338]: osdmap e46: 7 total, 6 up, 7 in 2024-04-22T13:19:07.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:07 smithi134 ceph-mon[31338]: from='osd.6 [v2:172.21.15.12:6818/3819619299,v1:172.21.15.12:6819/3819619299]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-22T13:19:07.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:07 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:19:08.577 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:19:08 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:19:08.315+0000 7fb75e8c2700 -1 osd.6 0 waiting for initial osdmap 2024-04-22T13:19:08.578 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:19:08 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:19:08.386+0000 7fb7566ad700 -1 osd.6 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T13:19:08.578 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:08 smithi012 ceph-mon[28265]: from='osd.6 [v2:172.21.15.12:6818/3819619299,v1:172.21.15.12:6819/3819619299]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-04-22T13:19:08.578 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:08 smithi012 ceph-mon[28265]: osdmap e47: 7 total, 6 up, 7 in 2024-04-22T13:19:08.578 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:08 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:19:08.578 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:08 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:19:08.578 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:08 smithi012 ceph-mon[28265]: from='osd.6 [v2:172.21.15.12:6818/3819619299,v1:172.21.15.12:6819/3819619299]' entity='osd.6' 2024-04-22T13:19:08.578 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:08 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:19:08.637 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:08 smithi111 ceph-mon[31361]: from='osd.6 [v2:172.21.15.12:6818/3819619299,v1:172.21.15.12:6819/3819619299]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-04-22T13:19:08.637 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:08 smithi111 ceph-mon[31361]: osdmap e47: 7 total, 6 up, 7 in 2024-04-22T13:19:08.637 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:08 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:19:08.637 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:08 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:19:08.637 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:08 smithi111 ceph-mon[31361]: from='osd.6 [v2:172.21.15.12:6818/3819619299,v1:172.21.15.12:6819/3819619299]' entity='osd.6' 2024-04-22T13:19:08.637 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:08 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:19:08.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:08 smithi134 ceph-mon[31338]: from='osd.6 [v2:172.21.15.12:6818/3819619299,v1:172.21.15.12:6819/3819619299]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-04-22T13:19:08.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:08 smithi134 ceph-mon[31338]: osdmap e47: 7 total, 6 up, 7 in 2024-04-22T13:19:08.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:08 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:19:08.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:08 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:19:08.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:08 smithi134 ceph-mon[31338]: from='osd.6 [v2:172.21.15.12:6818/3819619299,v1:172.21.15.12:6819/3819619299]' entity='osd.6' 2024-04-22T13:19:08.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:08 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:19:09.183 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-22T13:19:09.183 INFO:teuthology.orchestra.run.smithi111.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-22T13:19:09.183 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10+0 records in 2024-04-22T13:19:09.183 INFO:teuthology.orchestra.run.smithi111.stderr:10+0 records out 2024-04-22T13:19:09.183 INFO:teuthology.orchestra.run.smithi111.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112097 s, 935 MB/s 2024-04-22T13:19:09.183 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping successful for: 2024-04-22T13:19:09.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:09 smithi111 ceph-mon[31361]: purged_snaps scrub starts 2024-04-22T13:19:09.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:09 smithi111 ceph-mon[31361]: purged_snaps scrub ok 2024-04-22T13:19:09.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:09 smithi111 ceph-mon[31361]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:19:09.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:09 smithi111 ceph-mon[31361]: osd.6 [v2:172.21.15.12:6818/3819619299,v1:172.21.15.12:6819/3819619299] boot 2024-04-22T13:19:09.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:09 smithi111 ceph-mon[31361]: osdmap e48: 7 total, 7 up, 7 in 2024-04-22T13:19:09.729 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:09 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:19:09.736 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:09 smithi012 ceph-mon[28265]: purged_snaps scrub starts 2024-04-22T13:19:09.737 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:09 smithi012 ceph-mon[28265]: purged_snaps scrub ok 2024-04-22T13:19:09.737 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:09 smithi012 ceph-mon[28265]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:19:09.737 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:09 smithi012 ceph-mon[28265]: osd.6 [v2:172.21.15.12:6818/3819619299,v1:172.21.15.12:6819/3819619299] boot 2024-04-22T13:19:09.737 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:09 smithi012 ceph-mon[28265]: osdmap e48: 7 total, 7 up, 7 in 2024-04-22T13:19:09.737 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:09 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:19:09.837 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_2 2024-04-22T13:19:09.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:09 smithi134 ceph-mon[31338]: purged_snaps scrub starts 2024-04-22T13:19:09.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:09 smithi134 ceph-mon[31338]: purged_snaps scrub ok 2024-04-22T13:19:09.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:09 smithi134 ceph-mon[31338]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-22T13:19:09.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:09 smithi134 ceph-mon[31338]: osd.6 [v2:172.21.15.12:6818/3819619299,v1:172.21.15.12:6819/3819619299] boot 2024-04-22T13:19:09.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:09 smithi134 ceph-mon[31338]: osdmap e48: 7 total, 7 up, 7 in 2024-04-22T13:19:09.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:09 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:19:10.082 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T13:19:11.530 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:11 smithi111 ceph-mon[31361]: Detected new or changed devices on smithi012 2024-04-22T13:19:11.530 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:11 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:11.530 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:11 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:11.531 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:11 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:19:11.531 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:11 smithi111 ceph-mon[31361]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:11.531 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:11 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:19:11.531 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:11 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:19:11.531 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:11 smithi111 ceph-mon[31361]: Adjusting osd_memory_target on smithi012 to 5697M 2024-04-22T13:19:11.531 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:11 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:11.531 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:11 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:11.531 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:11 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:11.531 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:11 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:11.531 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:11 smithi111 ceph-mon[31361]: osdmap e49: 7 total, 7 up, 7 in 2024-04-22T13:19:11.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.561+0000 7faed5d86700 1 -- 172.21.15.111:0/132771675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faed0103540 msgr2=0x7faed0109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:11.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.561+0000 7faed5d86700 1 --2- 172.21.15.111:0/132771675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faed0103540 0x7faed0109df0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7faec800b0d0 tx=0x7faec800b3e0 comp rx=0 tx=0).stop 2024-04-22T13:19:11.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.561+0000 7faed5d86700 1 -- 172.21.15.111:0/132771675 shutdown_connections 2024-04-22T13:19:11.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.561+0000 7faed5d86700 1 --2- 172.21.15.111:0/132771675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faed0103540 0x7faed0109df0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:11.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.561+0000 7faed5d86700 1 --2- 172.21.15.111:0/132771675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faed01019a0 0x7faed0101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:11.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.561+0000 7faed5d86700 1 --2- 172.21.15.111:0/132771675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faed0102ba0 0x7faed0103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:11.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.561+0000 7faed5d86700 1 -- 172.21.15.111:0/132771675 >> 172.21.15.111:0/132771675 conn(0x7faed00fcf40 msgr2=0x7faed00ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:19:11.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.561+0000 7faed5d86700 1 -- 172.21.15.111:0/132771675 shutdown_connections 2024-04-22T13:19:11.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.561+0000 7faed5d86700 1 -- 172.21.15.111:0/132771675 wait complete. 2024-04-22T13:19:11.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.561+0000 7faed5d86700 1 Processor -- start 2024-04-22T13:19:11.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.561+0000 7faed5d86700 1 -- start start 2024-04-22T13:19:11.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.562+0000 7faed5d86700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faed01019a0 0x7faed019ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:19:11.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.562+0000 7faed5d86700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faed0102ba0 0x7faed019b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:19:11.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.562+0000 7faed5d86700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faed0103540 0x7faed01a22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:19:11.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.562+0000 7faed5d86700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7faed010d420 con 0x7faed0102ba0 2024-04-22T13:19:11.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.562+0000 7faed5d86700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faed010d2a0 con 0x7faed01019a0 2024-04-22T13:19:11.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.562+0000 7faed5d86700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faed010d5a0 con 0x7faed0103540 2024-04-22T13:19:11.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.562+0000 7faecf7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faed01019a0 0x7faed019ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:19:11.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.562+0000 7faecf7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faed01019a0 0x7faed019ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46494/0 (socket says 172.21.15.111:46494) 2024-04-22T13:19:11.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.562+0000 7faecf7fe700 1 -- 172.21.15.111:0/1622506789 learned_addr learned my addr 172.21.15.111:0/1622506789 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-22T13:19:11.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.562+0000 7faecffff700 1 --2- 172.21.15.111:0/1622506789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faed0103540 0x7faed01a22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:19:11.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.562+0000 7faecf7fe700 1 -- 172.21.15.111:0/1622506789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faed0103540 msgr2=0x7faed01a22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:11.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.562+0000 7faecf7fe700 1 --2- 172.21.15.111:0/1622506789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faed0103540 0x7faed01a22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:11.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.562+0000 7faecf7fe700 1 -- 172.21.15.111:0/1622506789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faed0102ba0 msgr2=0x7faed019b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:11.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.562+0000 7faeceffd700 1 --2- 172.21.15.111:0/1622506789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faed0102ba0 0x7faed019b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:19:11.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.562+0000 7faecf7fe700 1 --2- 172.21.15.111:0/1622506789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faed0102ba0 0x7faed019b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:11.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.562+0000 7faecf7fe700 1 -- 172.21.15.111:0/1622506789 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed01a2990 con 0x7faed01019a0 2024-04-22T13:19:11.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.562+0000 7faecffff700 1 --2- 172.21.15.111:0/1622506789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faed0103540 0x7faed01a22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:19:11.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.562+0000 7faeceffd700 1 --2- 172.21.15.111:0/1622506789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faed0102ba0 0x7faed019b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:19:11.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.562+0000 7faecf7fe700 1 --2- 172.21.15.111:0/1622506789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faed01019a0 0x7faed019ad30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7faec000d980 tx=0x7faec000dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:19:11.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.562+0000 7faeccff9700 1 -- 172.21.15.111:0/1622506789 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec0017030 con 0x7faed01019a0 2024-04-22T13:19:11.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.562+0000 7faeccff9700 1 -- 172.21.15.111:0/1622506789 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faec00044d0 con 0x7faed01019a0 2024-04-22T13:19:11.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.562+0000 7faed5d86700 1 -- 172.21.15.111:0/1622506789 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed01a2c80 con 0x7faed01019a0 2024-04-22T13:19:11.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.562+0000 7faeccff9700 1 -- 172.21.15.111:0/1622506789 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec0003aa0 con 0x7faed01019a0 2024-04-22T13:19:11.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.563+0000 7faed5d86700 1 -- 172.21.15.111:0/1622506789 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faed01a31c0 con 0x7faed01019a0 2024-04-22T13:19:11.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.564+0000 7faed5d86700 1 -- 172.21.15.111:0/1622506789 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faed0069730 con 0x7faed01019a0 2024-04-22T13:19:11.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.564+0000 7faeccff9700 1 -- 172.21.15.111:0/1622506789 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7faec001a070 con 0x7faed01019a0 2024-04-22T13:19:11.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.565+0000 7faeccff9700 1 --2- 172.21.15.111:0/1622506789 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7faeb80a3cc0 0x7faeb80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:19:11.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.565+0000 7faeccff9700 1 -- 172.21.15.111:0/1622506789 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4965+0+0 (secure 0 0 0) 0x7faec00cf060 con 0x7faed01019a0 2024-04-22T13:19:11.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.565+0000 7faeceffd700 1 --2- 172.21.15.111:0/1622506789 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7faeb80a3cc0 0x7faeb80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:19:11.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.566+0000 7faeceffd700 1 --2- 172.21.15.111:0/1622506789 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7faeb80a3cc0 0x7faeb80a6180 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faec4005fd0 tx=0x7faec4005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:19:11.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.570+0000 7faeccff9700 1 -- 172.21.15.111:0/1622506789 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faec009c750 con 0x7faed01019a0 2024-04-22T13:19:11.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:11 smithi012 ceph-mon[28265]: Detected new or changed devices on smithi012 2024-04-22T13:19:11.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:11 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:11.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:11 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:11.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:11 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:19:11.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:11 smithi012 ceph-mon[28265]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:11.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:11 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:19:11.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:11 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:19:11.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:11 smithi012 ceph-mon[28265]: Adjusting osd_memory_target on smithi012 to 5697M 2024-04-22T13:19:11.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:11 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:11.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:11 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:11.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:11 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:11.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:11 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:11.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:11 smithi012 ceph-mon[28265]: osdmap e49: 7 total, 7 up, 7 in 2024-04-22T13:19:11.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:11 smithi134 ceph-mon[31338]: Detected new or changed devices on smithi012 2024-04-22T13:19:11.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:11 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:11.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:11 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:11.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:11 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:19:11.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:11 smithi134 ceph-mon[31338]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:11.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:11 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:19:11.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:11 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:19:11.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:11 smithi134 ceph-mon[31338]: Adjusting osd_memory_target on smithi012 to 5697M 2024-04-22T13:19:11.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:11 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:11.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:11 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:11.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:11 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:11.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:11 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:11.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:11 smithi134 ceph-mon[31338]: osdmap e49: 7 total, 7 up, 7 in 2024-04-22T13:19:11.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:11.777+0000 7faed5d86700 1 -- 172.21.15.111:0/1622506789 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7faed00ff0e0 con 0x7faeb80a3cc0 2024-04-22T13:19:12.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:12 smithi012 ceph-mon[28265]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:19:12.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:12 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:19:12.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:12 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:19:12.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:12 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:12.641 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:12 smithi111 ceph-mon[31361]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:19:12.641 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:12 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:19:12.641 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:12 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:19:12.642 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:12 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:12.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:12 smithi134 ceph-mon[31338]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:19:12.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:12 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:19:12.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:12 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:19:12.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:12 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:13.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:13 smithi012 ceph-mon[28265]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:13.655 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:13 smithi111 ceph-mon[31361]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:13.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:13 smithi134 ceph-mon[31338]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:15.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:15 smithi134 ceph-mon[31338]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:15.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:15 smithi134 ceph-mon[31338]: from='client.? 172.21.15.111:0/4023327284' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a93d6af-569e-4e45-b697-9e978c1c1004"}]: dispatch 2024-04-22T13:19:15.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:15 smithi134 ceph-mon[31338]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a93d6af-569e-4e45-b697-9e978c1c1004"}]: dispatch 2024-04-22T13:19:15.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:15 smithi134 ceph-mon[31338]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a93d6af-569e-4e45-b697-9e978c1c1004"}]': finished 2024-04-22T13:19:15.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:15 smithi134 ceph-mon[31338]: osdmap e50: 8 total, 7 up, 8 in 2024-04-22T13:19:15.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:15 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:19:15.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:15 smithi134 ceph-mon[31338]: from='client.? 172.21.15.111:0/3050714682' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:19:15.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:15 smithi111 ceph-mon[31361]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:15.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:15 smithi111 ceph-mon[31361]: from='client.? 172.21.15.111:0/4023327284' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a93d6af-569e-4e45-b697-9e978c1c1004"}]: dispatch 2024-04-22T13:19:15.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:15 smithi111 ceph-mon[31361]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a93d6af-569e-4e45-b697-9e978c1c1004"}]: dispatch 2024-04-22T13:19:15.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:15 smithi111 ceph-mon[31361]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a93d6af-569e-4e45-b697-9e978c1c1004"}]': finished 2024-04-22T13:19:15.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:15 smithi111 ceph-mon[31361]: osdmap e50: 8 total, 7 up, 8 in 2024-04-22T13:19:15.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:15 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:19:15.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:15 smithi111 ceph-mon[31361]: from='client.? 172.21.15.111:0/3050714682' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:19:15.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:15 smithi012 ceph-mon[28265]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:15.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:15 smithi012 ceph-mon[28265]: from='client.? 172.21.15.111:0/4023327284' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a93d6af-569e-4e45-b697-9e978c1c1004"}]: dispatch 2024-04-22T13:19:15.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:15 smithi012 ceph-mon[28265]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a93d6af-569e-4e45-b697-9e978c1c1004"}]: dispatch 2024-04-22T13:19:15.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:15 smithi012 ceph-mon[28265]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a93d6af-569e-4e45-b697-9e978c1c1004"}]': finished 2024-04-22T13:19:15.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:15 smithi012 ceph-mon[28265]: osdmap e50: 8 total, 7 up, 8 in 2024-04-22T13:19:15.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:15 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:19:15.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:15 smithi012 ceph-mon[28265]: from='client.? 172.21.15.111:0/3050714682' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:19:17.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:17 smithi134 ceph-mon[31338]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:17.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:17 smithi111 ceph-mon[31361]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:17.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:17 smithi012 ceph-mon[28265]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:19.640 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:19 smithi012 ceph-mon[28265]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:19.658 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:19 smithi111 ceph-mon[31361]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:19.671 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:19 smithi134 ceph-mon[31338]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:21.666 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:21 smithi111 ceph-mon[31361]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:21.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:21 smithi134 ceph-mon[31338]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:21.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:21 smithi012 ceph-mon[28265]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:22.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:22 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-22T13:19:22.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:22 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:22.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:22 smithi134 ceph-mon[31338]: Deploying daemon osd.7 on smithi111 2024-04-22T13:19:22.687 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:22 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-22T13:19:22.687 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:22 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:22.688 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:22 smithi111 ceph-mon[31361]: Deploying daemon osd.7 on smithi111 2024-04-22T13:19:22.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:22 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-22T13:19:22.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:22 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:22.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:22 smithi012 ceph-mon[28265]: Deploying daemon osd.7 on smithi111 2024-04-22T13:19:23.384 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:23 smithi111 ceph-mon[31361]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:23.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:23 smithi134 ceph-mon[31338]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:23.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:23 smithi012 ceph-mon[28265]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:25.586 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:25 smithi111 ceph-mon[31361]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:25.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:25 smithi134 ceph-mon[31338]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:25.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:25 smithi012 ceph-mon[28265]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:27.664 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:27 smithi111 ceph-mon[31361]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:27.664 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:27 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:27.664 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:27 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:27.664 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:27 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:27.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:27 smithi134 ceph-mon[31338]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:27.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:27 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:27.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:27 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:27.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:27 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:27.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:27 smithi012 ceph-mon[28265]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:27.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:27 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:27.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:27 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:27.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:27 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:29.620 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:29 smithi111 ceph-mon[31361]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:29.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:29 smithi134 ceph-mon[31338]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:29.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:29 smithi012 ceph-mon[28265]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:30.843 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:30.844+0000 7faeccff9700 1 -- 172.21.15.111:0/1622506789 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7faed00ff0e0 con 0x7faeb80a3cc0 2024-04-22T13:19:30.843 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 7 on host 'smithi111' 2024-04-22T13:19:30.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:30.847+0000 7faed5d86700 1 -- 172.21.15.111:0/1622506789 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7faeb80a3cc0 msgr2=0x7faeb80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:30.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:30.847+0000 7faed5d86700 1 --2- 172.21.15.111:0/1622506789 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7faeb80a3cc0 0x7faeb80a6180 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faec4005fd0 tx=0x7faec4005ee0 comp rx=0 tx=0).stop 2024-04-22T13:19:30.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:30.847+0000 7faed5d86700 1 -- 172.21.15.111:0/1622506789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faed01019a0 msgr2=0x7faed019ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:30.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:30.847+0000 7faed5d86700 1 --2- 172.21.15.111:0/1622506789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faed01019a0 0x7faed019ad30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7faec000d980 tx=0x7faec000dd40 comp rx=0 tx=0).stop 2024-04-22T13:19:30.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:30.847+0000 7faed5d86700 1 -- 172.21.15.111:0/1622506789 shutdown_connections 2024-04-22T13:19:30.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:30.847+0000 7faed5d86700 1 --2- 172.21.15.111:0/1622506789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faed01019a0 0x7faed019ad30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:30.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:30.847+0000 7faed5d86700 1 --2- 172.21.15.111:0/1622506789 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7faeb80a3cc0 0x7faeb80a6180 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:30.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:30.847+0000 7faed5d86700 1 --2- 172.21.15.111:0/1622506789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faed0103540 0x7faed01a22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:30.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:30.847+0000 7faed5d86700 1 --2- 172.21.15.111:0/1622506789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faed0102ba0 0x7faed019b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:30.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:30.847+0000 7faed5d86700 1 -- 172.21.15.111:0/1622506789 >> 172.21.15.111:0/1622506789 conn(0x7faed00fcf40 msgr2=0x7faed00fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:19:30.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:30.847+0000 7faed5d86700 1 -- 172.21.15.111:0/1622506789 shutdown_connections 2024-04-22T13:19:30.845 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:19:30.847+0000 7faed5d86700 1 -- 172.21.15.111:0/1622506789 wait complete. 2024-04-22T13:19:31.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:31 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:31.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:31 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:31.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:31 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:31.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:31 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:31.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:31 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:31.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:31 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:31.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:31 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:31.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:31 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:31.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:31 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:31.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:31 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:31.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:31 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:31.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:31 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:31.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:31 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:31.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:31 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:31.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:31 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:31.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:31 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:31.462 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:31 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:31.463 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:31 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:31.463 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:31 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:31.463 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:31 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:31.463 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:31 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:31.463 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:31 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:31.463 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:31 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:31.463 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:31 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:31.492 DEBUG:teuthology.orchestra.run.smithi111:osd.7> sudo journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.7.service 2024-04-22T13:19:31.494 INFO:tasks.cephadm:Deploying osd.8 on smithi134 with /dev/vg_nvme/lv_2... 2024-04-22T13:19:31.494 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-22T13:19:31.745 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:19:32.130 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:32 smithi134 ceph-mon[31338]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:32.130 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:32 smithi134 ceph-mon[31338]: from='osd.7 [v2:172.21.15.111:6816/3113856282,v1:172.21.15.111:6817/3113856282]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-22T13:19:32.130 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:32 smithi134 ceph-mon[31338]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-22T13:19:32.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:32 smithi012 ceph-mon[28265]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:32.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:32 smithi012 ceph-mon[28265]: from='osd.7 [v2:172.21.15.111:6816/3113856282,v1:172.21.15.111:6817/3113856282]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-22T13:19:32.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:32 smithi012 ceph-mon[28265]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-22T13:19:32.368 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:32 smithi111 ceph-mon[31361]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:32.369 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:32 smithi111 ceph-mon[31361]: from='osd.7 [v2:172.21.15.111:6816/3113856282,v1:172.21.15.111:6817/3113856282]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-22T13:19:32.369 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:32 smithi111 ceph-mon[31361]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-22T13:19:33.229 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:33 smithi134 ceph-mon[31338]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-22T13:19:33.230 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:33 smithi134 ceph-mon[31338]: from='osd.7 [v2:172.21.15.111:6816/3113856282,v1:172.21.15.111:6817/3113856282]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-22T13:19:33.230 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:33 smithi134 ceph-mon[31338]: osdmap e51: 8 total, 7 up, 8 in 2024-04-22T13:19:33.230 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:33 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:19:33.230 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:33 smithi134 ceph-mon[31338]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-22T13:19:33.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:33 smithi012 ceph-mon[28265]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-22T13:19:33.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:33 smithi012 ceph-mon[28265]: from='osd.7 [v2:172.21.15.111:6816/3113856282,v1:172.21.15.111:6817/3113856282]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-22T13:19:33.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:33 smithi012 ceph-mon[28265]: osdmap e51: 8 total, 7 up, 8 in 2024-04-22T13:19:33.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:33 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:19:33.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:33 smithi012 ceph-mon[28265]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-22T13:19:33.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:33 smithi111 ceph-mon[31361]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-22T13:19:33.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:33 smithi111 ceph-mon[31361]: from='osd.7 [v2:172.21.15.111:6816/3113856282,v1:172.21.15.111:6817/3113856282]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-22T13:19:33.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:33 smithi111 ceph-mon[31361]: osdmap e51: 8 total, 7 up, 8 in 2024-04-22T13:19:33.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:33 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:19:33.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:33 smithi111 ceph-mon[31361]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-22T13:19:33.594 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-22T13:19:33.594 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-22T13:19:33.594 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2024-04-22T13:19:33.594 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2024-04-22T13:19:33.594 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011222 s, 934 MB/s 2024-04-22T13:19:33.594 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2024-04-22T13:19:34.121 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:34 smithi111 ceph-mon[31361]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:34.122 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:34 smithi111 ceph-mon[31361]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-22T13:19:34.122 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:34 smithi111 ceph-mon[31361]: osdmap e52: 8 total, 7 up, 8 in 2024-04-22T13:19:34.122 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:34 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:19:34.122 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:34 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:19:34.122 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:34 smithi111 ceph-mon[31361]: from='osd.7 ' entity='osd.7' 2024-04-22T13:19:34.122 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:34 smithi111 ceph-mon[31361]: osdmap e53: 8 total, 7 up, 8 in 2024-04-22T13:19:34.122 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:34 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:19:34.122 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:19:33 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:19:33.976+0000 7f11b59dc700 -1 osd.7 0 waiting for initial osdmap 2024-04-22T13:19:34.122 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:19:34 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:19:34.123+0000 7f11ad7c7700 -1 osd.7 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T13:19:34.149 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:34 smithi134 ceph-mon[31338]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:34.149 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:34 smithi134 ceph-mon[31338]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-22T13:19:34.149 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:34 smithi134 ceph-mon[31338]: osdmap e52: 8 total, 7 up, 8 in 2024-04-22T13:19:34.149 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:34 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:19:34.149 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:34 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:19:34.149 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:34 smithi134 ceph-mon[31338]: from='osd.7 ' entity='osd.7' 2024-04-22T13:19:34.149 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:34 smithi134 ceph-mon[31338]: osdmap e53: 8 total, 7 up, 8 in 2024-04-22T13:19:34.149 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:34 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:19:34.177 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_2 2024-04-22T13:19:34.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:34 smithi012 ceph-mon[28265]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:34.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:34 smithi012 ceph-mon[28265]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-22T13:19:34.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:34 smithi012 ceph-mon[28265]: osdmap e52: 8 total, 7 up, 8 in 2024-04-22T13:19:34.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:34 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:19:34.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:34 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:19:34.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:34 smithi012 ceph-mon[28265]: from='osd.7 ' entity='osd.7' 2024-04-22T13:19:34.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:34 smithi012 ceph-mon[28265]: osdmap e53: 8 total, 7 up, 8 in 2024-04-22T13:19:34.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:34 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:19:34.416 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:19:35.345 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:35 smithi012 ceph-mon[28265]: purged_snaps scrub starts 2024-04-22T13:19:35.345 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:35 smithi012 ceph-mon[28265]: purged_snaps scrub ok 2024-04-22T13:19:35.345 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:35 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:19:35.345 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:35 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:35.345 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:35 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:35.345 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:35 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:19:35.345 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:35 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:19:35.345 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:35 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:19:35.345 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:35 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:35.345 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:35 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:35.345 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:35 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:35.346 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:35 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:35.414 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:35 smithi134 ceph-mon[31338]: purged_snaps scrub starts 2024-04-22T13:19:35.415 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:35 smithi134 ceph-mon[31338]: purged_snaps scrub ok 2024-04-22T13:19:35.415 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:35 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:19:35.415 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:35 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:35.415 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:35 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:35.415 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:35 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:19:35.415 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:35 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:19:35.415 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:35 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:19:35.415 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:35 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:35.415 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:35 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:35.415 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:35 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:35.416 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:35 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:35.487 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:35 smithi111 ceph-mon[31361]: purged_snaps scrub starts 2024-04-22T13:19:35.487 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:35 smithi111 ceph-mon[31361]: purged_snaps scrub ok 2024-04-22T13:19:35.487 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:35 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:19:35.487 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:35 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:35.487 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:35 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:35.487 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:35 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:19:35.487 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:35 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:19:35.487 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:35 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:19:35.487 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:35 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:35.488 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:35 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:35.488 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:35 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:35.488 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:35 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:35.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.763+0000 7fd429c38700 1 -- 172.21.15.134:0/460328094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd424101ce0 msgr2=0x7fd424102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:35.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.763+0000 7fd429c38700 1 --2- 172.21.15.134:0/460328094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd424101ce0 0x7fd424102160 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd414009990 tx=0x7fd414009ca0 comp rx=0 tx=0).stop 2024-04-22T13:19:35.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.764+0000 7fd429c38700 1 -- 172.21.15.134:0/460328094 shutdown_connections 2024-04-22T13:19:35.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.764+0000 7fd429c38700 1 --2- 172.21.15.134:0/460328094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd424101380 0x7fd4241017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:35.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.764+0000 7fd429c38700 1 --2- 172.21.15.134:0/460328094 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd4241026a0 0x7fd424102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:35.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.764+0000 7fd429c38700 1 --2- 172.21.15.134:0/460328094 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd424101ce0 0x7fd424102160 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:35.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.764+0000 7fd429c38700 1 -- 172.21.15.134:0/460328094 >> 172.21.15.134:0/460328094 conn(0x7fd4240fcf40 msgr2=0x7fd4240ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:19:35.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.764+0000 7fd429c38700 1 -- 172.21.15.134:0/460328094 shutdown_connections 2024-04-22T13:19:35.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.764+0000 7fd429c38700 1 -- 172.21.15.134:0/460328094 wait complete. 2024-04-22T13:19:35.776 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.764+0000 7fd429c38700 1 Processor -- start 2024-04-22T13:19:35.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.764+0000 7fd429c38700 1 -- start start 2024-04-22T13:19:35.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.764+0000 7fd429c38700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd424101380 0x7fd42419ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:19:35.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.764+0000 7fd429c38700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd424101ce0 0x7fd42419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:19:35.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.764+0000 7fd429c38700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4241026a0 0x7fd4241a2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:19:35.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.765+0000 7fd429c38700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd4241107a0 con 0x7fd424101ce0 2024-04-22T13:19:35.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.765+0000 7fd429c38700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd424110620 con 0x7fd424101380 2024-04-22T13:19:35.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.765+0000 7fd429c38700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd424110920 con 0x7fd4241026a0 2024-04-22T13:19:35.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.765+0000 7fd423fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4241026a0 0x7fd4241a2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:19:35.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.765+0000 7fd423fff700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4241026a0 0x7fd4241a2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:50474/0 (socket says 172.21.15.134:50474) 2024-04-22T13:19:35.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.765+0000 7fd423fff700 1 -- 172.21.15.134:0/3885456450 learned_addr learned my addr 172.21.15.134:0/3885456450 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-22T13:19:35.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.765+0000 7fd422ffd700 1 --2- 172.21.15.134:0/3885456450 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd424101ce0 0x7fd42419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:19:35.777 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.765+0000 7fd4237fe700 1 --2- 172.21.15.134:0/3885456450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd424101380 0x7fd42419ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:19:35.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.765+0000 7fd423fff700 1 -- 172.21.15.134:0/3885456450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd424101380 msgr2=0x7fd42419ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:35.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.765+0000 7fd423fff700 1 --2- 172.21.15.134:0/3885456450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd424101380 0x7fd42419ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:35.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.765+0000 7fd423fff700 1 -- 172.21.15.134:0/3885456450 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd424101ce0 msgr2=0x7fd42419b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:35.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.765+0000 7fd423fff700 1 --2- 172.21.15.134:0/3885456450 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd424101ce0 0x7fd42419b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:35.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.765+0000 7fd423fff700 1 -- 172.21.15.134:0/3885456450 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4241a2ad0 con 0x7fd4241026a0 2024-04-22T13:19:35.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.765+0000 7fd423fff700 1 --2- 172.21.15.134:0/3885456450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4241026a0 0x7fd4241a2430 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd41c00b9a0 tx=0x7fd41c00bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:19:35.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.765+0000 7fd420ff9700 1 -- 172.21.15.134:0/3885456450 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c00d850 con 0x7fd4241026a0 2024-04-22T13:19:35.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.765+0000 7fd420ff9700 1 -- 172.21.15.134:0/3885456450 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd41c00e420 con 0x7fd4241026a0 2024-04-22T13:19:35.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.765+0000 7fd429c38700 1 -- 172.21.15.134:0/3885456450 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4241a2dc0 con 0x7fd4241026a0 2024-04-22T13:19:35.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.765+0000 7fd420ff9700 1 -- 172.21.15.134:0/3885456450 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c017640 con 0x7fd4241026a0 2024-04-22T13:19:35.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.765+0000 7fd429c38700 1 -- 172.21.15.134:0/3885456450 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4241a32a0 con 0x7fd4241026a0 2024-04-22T13:19:35.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.766+0000 7fd429c38700 1 -- 172.21.15.134:0/3885456450 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd424069730 con 0x7fd4241026a0 2024-04-22T13:19:35.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.767+0000 7fd420ff9700 1 -- 172.21.15.134:0/3885456450 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd41c00d9f0 con 0x7fd4241026a0 2024-04-22T13:19:35.778 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.768+0000 7fd420ff9700 1 --2- 172.21.15.134:0/3885456450 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fd40c0a3c40 0x7fd40c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:19:35.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.768+0000 7fd420ff9700 1 -- 172.21.15.134:0/3885456450 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5430+0+0 (secure 0 0 0) 0x7fd41c0cf530 con 0x7fd4241026a0 2024-04-22T13:19:35.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.768+0000 7fd4237fe700 1 --2- 172.21.15.134:0/3885456450 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fd40c0a3c40 0x7fd40c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:19:35.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.769+0000 7fd4237fe700 1 --2- 172.21.15.134:0/3885456450 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fd40c0a3c40 0x7fd40c0a6100 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd418009c60 tx=0x7fd418009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:19:35.779 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.772+0000 7fd420ff9700 1 -- 172.21.15.134:0/3885456450 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd41c013030 con 0x7fd4241026a0 2024-04-22T13:19:35.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:35.981+0000 7fd429c38700 1 -- 172.21.15.134:0/3885456450 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd424062e80 con 0x7fd40c0a3c40 2024-04-22T13:19:36.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:36 smithi012 ceph-mon[28265]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:36.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:36 smithi012 ceph-mon[28265]: Detected new or changed devices on smithi111 2024-04-22T13:19:36.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:36 smithi012 ceph-mon[28265]: Adjusting osd_memory_target on smithi111 to 5696M 2024-04-22T13:19:36.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:36 smithi012 ceph-mon[28265]: osd.7 [v2:172.21.15.111:6816/3113856282,v1:172.21.15.111:6817/3113856282] boot 2024-04-22T13:19:36.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:36 smithi012 ceph-mon[28265]: osdmap e54: 8 total, 8 up, 8 in 2024-04-22T13:19:36.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:36 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:19:36.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:36 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:19:36.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:36 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:19:36.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:36 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:36.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:36 smithi012 ceph-mon[28265]: osdmap e55: 8 total, 8 up, 8 in 2024-04-22T13:19:36.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:36 smithi134 ceph-mon[31338]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:36.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:36 smithi134 ceph-mon[31338]: Detected new or changed devices on smithi111 2024-04-22T13:19:36.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:36 smithi134 ceph-mon[31338]: Adjusting osd_memory_target on smithi111 to 5696M 2024-04-22T13:19:36.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:36 smithi134 ceph-mon[31338]: osd.7 [v2:172.21.15.111:6816/3113856282,v1:172.21.15.111:6817/3113856282] boot 2024-04-22T13:19:36.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:36 smithi134 ceph-mon[31338]: osdmap e54: 8 total, 8 up, 8 in 2024-04-22T13:19:36.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:36 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:19:36.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:36 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:19:36.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:36 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:19:36.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:36 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:36.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:36 smithi134 ceph-mon[31338]: osdmap e55: 8 total, 8 up, 8 in 2024-04-22T13:19:36.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:36 smithi111 ceph-mon[31361]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-22T13:19:36.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:36 smithi111 ceph-mon[31361]: Detected new or changed devices on smithi111 2024-04-22T13:19:36.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:36 smithi111 ceph-mon[31361]: Adjusting osd_memory_target on smithi111 to 5696M 2024-04-22T13:19:36.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:36 smithi111 ceph-mon[31361]: osd.7 [v2:172.21.15.111:6816/3113856282,v1:172.21.15.111:6817/3113856282] boot 2024-04-22T13:19:36.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:36 smithi111 ceph-mon[31361]: osdmap e54: 8 total, 8 up, 8 in 2024-04-22T13:19:36.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:36 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:19:36.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:36 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:19:36.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:36 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:19:36.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:36 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:36.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:36 smithi111 ceph-mon[31361]: osdmap e55: 8 total, 8 up, 8 in 2024-04-22T13:19:37.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:37 smithi134 ceph-mon[31338]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:19:37.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:37 smithi134 ceph-mon[31338]: osdmap e56: 8 total, 8 up, 8 in 2024-04-22T13:19:37.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:37 smithi012 ceph-mon[28265]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:19:37.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:37 smithi012 ceph-mon[28265]: osdmap e56: 8 total, 8 up, 8 in 2024-04-22T13:19:37.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:37 smithi111 ceph-mon[31361]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:19:37.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:37 smithi111 ceph-mon[31361]: osdmap e56: 8 total, 8 up, 8 in 2024-04-22T13:19:38.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:38 smithi012 ceph-mon[28265]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:38.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:38 smithi134 ceph-mon[31338]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:38.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:38 smithi111 ceph-mon[31361]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:39.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:39 smithi134 ceph-mon[31338]: osdmap e57: 8 total, 8 up, 8 in 2024-04-22T13:19:39.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:39 smithi134 ceph-mon[31338]: from='client.? 172.21.15.134:0/4264257949' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e21acc68-cf50-4f4f-b1ba-cf6c195b4955"}]: dispatch 2024-04-22T13:19:39.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:39 smithi134 ceph-mon[31338]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e21acc68-cf50-4f4f-b1ba-cf6c195b4955"}]: dispatch 2024-04-22T13:19:39.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:39 smithi134 ceph-mon[31338]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e21acc68-cf50-4f4f-b1ba-cf6c195b4955"}]': finished 2024-04-22T13:19:39.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:39 smithi134 ceph-mon[31338]: osdmap e58: 9 total, 8 up, 9 in 2024-04-22T13:19:39.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:39 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:19:39.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:39 smithi111 ceph-mon[31361]: osdmap e57: 8 total, 8 up, 8 in 2024-04-22T13:19:39.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:39 smithi111 ceph-mon[31361]: from='client.? 172.21.15.134:0/4264257949' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e21acc68-cf50-4f4f-b1ba-cf6c195b4955"}]: dispatch 2024-04-22T13:19:39.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:39 smithi111 ceph-mon[31361]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e21acc68-cf50-4f4f-b1ba-cf6c195b4955"}]: dispatch 2024-04-22T13:19:39.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:39 smithi111 ceph-mon[31361]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e21acc68-cf50-4f4f-b1ba-cf6c195b4955"}]': finished 2024-04-22T13:19:39.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:39 smithi111 ceph-mon[31361]: osdmap e58: 9 total, 8 up, 9 in 2024-04-22T13:19:39.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:39 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:19:39.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:39 smithi012 ceph-mon[28265]: osdmap e57: 8 total, 8 up, 8 in 2024-04-22T13:19:39.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:39 smithi012 ceph-mon[28265]: from='client.? 172.21.15.134:0/4264257949' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e21acc68-cf50-4f4f-b1ba-cf6c195b4955"}]: dispatch 2024-04-22T13:19:39.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:39 smithi012 ceph-mon[28265]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e21acc68-cf50-4f4f-b1ba-cf6c195b4955"}]: dispatch 2024-04-22T13:19:39.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:39 smithi012 ceph-mon[28265]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e21acc68-cf50-4f4f-b1ba-cf6c195b4955"}]': finished 2024-04-22T13:19:39.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:39 smithi012 ceph-mon[28265]: osdmap e58: 9 total, 8 up, 9 in 2024-04-22T13:19:39.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:39 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:19:40.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:40 smithi134 ceph-mon[31338]: pgmap v184: 1 pgs: 1 unknown; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:40.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:40 smithi134 ceph-mon[31338]: from='client.? 172.21.15.134:0/3956610564' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:19:40.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:40 smithi111 ceph-mon[31361]: pgmap v184: 1 pgs: 1 unknown; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:40.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:40 smithi111 ceph-mon[31361]: from='client.? 172.21.15.134:0/3956610564' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:19:40.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:40 smithi012 ceph-mon[28265]: pgmap v184: 1 pgs: 1 unknown; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:40.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:40 smithi012 ceph-mon[28265]: from='client.? 172.21.15.134:0/3956610564' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:19:42.346 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:42 smithi134 ceph-mon[31338]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:42.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:42 smithi111 ceph-mon[31361]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:42.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:42 smithi012 ceph-mon[28265]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:44.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:44 smithi134 ceph-mon[31338]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:44.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:44 smithi111 ceph-mon[31361]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:44.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:44 smithi012 ceph-mon[28265]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:46.406 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:46 smithi134 ceph-mon[31338]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:46.406 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:46 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-22T13:19:46.406 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:46 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:46.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:46 smithi111 ceph-mon[31361]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:46.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:46 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-22T13:19:46.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:46 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:46.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:46 smithi012 ceph-mon[28265]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:46.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:46 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-22T13:19:46.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:46 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:47.379 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:47 smithi111 ceph-mon[31361]: Deploying daemon osd.8 on smithi134 2024-04-22T13:19:47.394 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:47 smithi134 ceph-mon[31338]: Deploying daemon osd.8 on smithi134 2024-04-22T13:19:47.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:47 smithi012 ceph-mon[28265]: Deploying daemon osd.8 on smithi134 2024-04-22T13:19:48.418 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:48 smithi134 ceph-mon[31338]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:48.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:48 smithi111 ceph-mon[31361]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:48.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:48 smithi012 ceph-mon[28265]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:50.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:50 smithi134 ceph-mon[31338]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:50.390 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:50 smithi012 ceph-mon[28265]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:50.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:50 smithi111 ceph-mon[31361]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:52.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:52 smithi012 ceph-mon[28265]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:52.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:52 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:52.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:52 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:52.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:52 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:52.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:52 smithi134 ceph-mon[31338]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:52.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:52 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:52.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:52 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:52.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:52 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:52.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:52 smithi111 ceph-mon[31361]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:52.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:52 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:52.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:52 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:52.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:52 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:54.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:54 smithi012 ceph-mon[28265]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:54.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:54 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:54.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:54 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:54.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:54 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:54.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:54 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:54.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:54 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:54.356 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:54 smithi134 ceph-mon[31338]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:54.356 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:54 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:54.356 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:54 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:54.356 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:54 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:54.357 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:54 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:54.357 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:54 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:54.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:54 smithi111 ceph-mon[31361]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:54.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:54 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:54.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:54 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:54.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:54 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:54.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:54 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:54.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:54 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:54.698 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 8 on host 'smithi134' 2024-04-22T13:19:54.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:54.700+0000 7fd420ff9700 1 -- 172.21.15.134:0/3885456450 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd424062e80 con 0x7fd40c0a3c40 2024-04-22T13:19:54.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:54.703+0000 7fd429c38700 1 -- 172.21.15.134:0/3885456450 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fd40c0a3c40 msgr2=0x7fd40c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:54.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:54.703+0000 7fd429c38700 1 --2- 172.21.15.134:0/3885456450 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fd40c0a3c40 0x7fd40c0a6100 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd418009c60 tx=0x7fd418009260 comp rx=0 tx=0).stop 2024-04-22T13:19:54.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:54.703+0000 7fd429c38700 1 -- 172.21.15.134:0/3885456450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4241026a0 msgr2=0x7fd4241a2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:19:54.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:54.703+0000 7fd429c38700 1 --2- 172.21.15.134:0/3885456450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4241026a0 0x7fd4241a2430 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd41c00b9a0 tx=0x7fd41c00bcb0 comp rx=0 tx=0).stop 2024-04-22T13:19:54.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:54.703+0000 7fd429c38700 1 -- 172.21.15.134:0/3885456450 shutdown_connections 2024-04-22T13:19:54.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:54.703+0000 7fd429c38700 1 --2- 172.21.15.134:0/3885456450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd424101380 0x7fd42419ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:54.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:54.703+0000 7fd429c38700 1 --2- 172.21.15.134:0/3885456450 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fd40c0a3c40 0x7fd40c0a6100 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:54.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:54.703+0000 7fd429c38700 1 --2- 172.21.15.134:0/3885456450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd4241026a0 0x7fd4241a2430 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:54.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:54.703+0000 7fd429c38700 1 --2- 172.21.15.134:0/3885456450 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd424101ce0 0x7fd42419b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:19:54.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:54.703+0000 7fd429c38700 1 -- 172.21.15.134:0/3885456450 >> 172.21.15.134:0/3885456450 conn(0x7fd4240fcf40 msgr2=0x7fd4240fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:19:54.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:54.703+0000 7fd429c38700 1 -- 172.21.15.134:0/3885456450 shutdown_connections 2024-04-22T13:19:54.701 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:19:54.703+0000 7fd429c38700 1 -- 172.21.15.134:0/3885456450 wait complete. 2024-04-22T13:19:55.286 DEBUG:teuthology.orchestra.run.smithi134:osd.8> sudo journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.8.service 2024-04-22T13:19:55.288 INFO:tasks.cephadm:Deploying osd.9 on smithi012 with /dev/vg_nvme/lv_1... 2024-04-22T13:19:55.288 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-22T13:19:55.546 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:19:55.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:55 smithi134 ceph-mon[31338]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:55.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:55 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:55.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:55 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:55.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:55 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:55.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:55 smithi134 ceph-mon[31338]: from='osd.8 [v2:172.21.15.134:6816/1689362191,v1:172.21.15.134:6817/1689362191]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-22T13:19:55.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:55 smithi134 ceph-mon[31338]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-22T13:19:55.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:55 smithi111 ceph-mon[31361]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:55.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:55 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:55.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:55 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:55.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:55 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:55.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:55 smithi111 ceph-mon[31361]: from='osd.8 [v2:172.21.15.134:6816/1689362191,v1:172.21.15.134:6817/1689362191]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-22T13:19:55.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:55 smithi111 ceph-mon[31361]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-22T13:19:55.978 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:55 smithi012 ceph-mon[28265]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:55.979 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:55 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:55.979 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:55 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:55.979 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:55 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:19:55.979 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:55 smithi012 ceph-mon[28265]: from='osd.8 [v2:172.21.15.134:6816/1689362191,v1:172.21.15.134:6817/1689362191]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-22T13:19:55.979 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:55 smithi012 ceph-mon[28265]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-22T13:19:56.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:56 smithi134 ceph-mon[31338]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-22T13:19:56.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:56 smithi134 ceph-mon[31338]: from='osd.8 [v2:172.21.15.134:6816/1689362191,v1:172.21.15.134:6817/1689362191]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-22T13:19:56.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:56 smithi134 ceph-mon[31338]: osdmap e59: 9 total, 8 up, 9 in 2024-04-22T13:19:56.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:56 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:19:56.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:56 smithi134 ceph-mon[31338]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-22T13:19:56.929 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:19:56 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:19:56.666+0000 7f4bf6e3d700 -1 osd.8 0 waiting for initial osdmap 2024-04-22T13:19:56.929 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:19:56 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:19:56.681+0000 7f4bf2c30700 -1 osd.8 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T13:19:56.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:56 smithi111 ceph-mon[31361]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-22T13:19:56.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:56 smithi111 ceph-mon[31361]: from='osd.8 [v2:172.21.15.134:6816/1689362191,v1:172.21.15.134:6817/1689362191]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-22T13:19:56.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:56 smithi111 ceph-mon[31361]: osdmap e59: 9 total, 8 up, 9 in 2024-04-22T13:19:56.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:56 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:19:56.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:56 smithi111 ceph-mon[31361]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-22T13:19:56.982 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:56 smithi012 ceph-mon[28265]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-22T13:19:56.982 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:56 smithi012 ceph-mon[28265]: from='osd.8 [v2:172.21.15.134:6816/1689362191,v1:172.21.15.134:6817/1689362191]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-22T13:19:56.982 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:56 smithi012 ceph-mon[28265]: osdmap e59: 9 total, 8 up, 9 in 2024-04-22T13:19:56.982 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:56 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:19:56.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:56 smithi012 ceph-mon[28265]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-22T13:19:57.681 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-22T13:19:57.682 INFO:teuthology.orchestra.run.smithi012.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-22T13:19:57.682 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10+0 records in 2024-04-22T13:19:57.682 INFO:teuthology.orchestra.run.smithi012.stderr:10+0 records out 2024-04-22T13:19:57.682 INFO:teuthology.orchestra.run.smithi012.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125352 s, 837 MB/s 2024-04-22T13:19:57.682 INFO:teuthology.orchestra.run.smithi012.stderr:--> Zapping successful for: 2024-04-22T13:19:57.851 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:57 smithi134 ceph-mon[31338]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:57.851 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:57 smithi134 ceph-mon[31338]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-22T13:19:57.851 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:57 smithi134 ceph-mon[31338]: osdmap e60: 9 total, 8 up, 9 in 2024-04-22T13:19:57.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:57 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:19:57.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:57 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:19:57.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:57 smithi134 ceph-mon[31338]: osd.8 [v2:172.21.15.134:6816/1689362191,v1:172.21.15.134:6817/1689362191] boot 2024-04-22T13:19:57.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:57 smithi134 ceph-mon[31338]: osdmap e61: 9 total, 9 up, 9 in 2024-04-22T13:19:57.852 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:57 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:19:57.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:57 smithi111 ceph-mon[31361]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:57.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:57 smithi111 ceph-mon[31361]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-22T13:19:57.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:57 smithi111 ceph-mon[31361]: osdmap e60: 9 total, 8 up, 9 in 2024-04-22T13:19:57.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:57 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:19:57.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:57 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:19:57.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:57 smithi111 ceph-mon[31361]: osd.8 [v2:172.21.15.134:6816/1689362191,v1:172.21.15.134:6817/1689362191] boot 2024-04-22T13:19:57.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:57 smithi111 ceph-mon[31361]: osdmap e61: 9 total, 9 up, 9 in 2024-04-22T13:19:57.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:57 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:19:58.006 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:57 smithi012 ceph-mon[28265]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-22T13:19:58.006 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:57 smithi012 ceph-mon[28265]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-22T13:19:58.007 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:57 smithi012 ceph-mon[28265]: osdmap e60: 9 total, 8 up, 9 in 2024-04-22T13:19:58.007 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:57 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:19:58.007 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:57 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:19:58.007 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:57 smithi012 ceph-mon[28265]: osd.8 [v2:172.21.15.134:6816/1689362191,v1:172.21.15.134:6817/1689362191] boot 2024-04-22T13:19:58.007 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:57 smithi012 ceph-mon[28265]: osdmap e61: 9 total, 9 up, 9 in 2024-04-22T13:19:58.007 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:57 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:19:58.579 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi012:vg_nvme/lv_1 2024-04-22T13:19:58.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:58 smithi012 ceph-mon[28265]: purged_snaps scrub starts 2024-04-22T13:19:58.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:58 smithi012 ceph-mon[28265]: purged_snaps scrub ok 2024-04-22T13:19:58.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:58 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:58.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:58 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:58.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:58 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:19:58.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:58 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:19:58.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:58 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:19:58.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:58 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:58.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:58 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:58.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:58 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:58.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:58 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:58.856 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:19:58.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:58 smithi134 ceph-mon[31338]: purged_snaps scrub starts 2024-04-22T13:19:58.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:58 smithi134 ceph-mon[31338]: purged_snaps scrub ok 2024-04-22T13:19:58.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:58 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:58.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:58 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:58.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:58 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:19:58.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:58 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:19:58.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:58 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:19:58.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:58 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:58.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:58 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:58.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:58 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:58.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:58 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:58.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:58 smithi111 ceph-mon[31361]: purged_snaps scrub starts 2024-04-22T13:19:58.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:58 smithi111 ceph-mon[31361]: purged_snaps scrub ok 2024-04-22T13:19:58.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:58 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:58.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:58 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:58.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:58 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:19:58.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:58 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:19:58.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:58 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:19:58.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:58 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:58.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:58 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:19:58.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:58 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:19:58.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:58 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:19:59.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:59 smithi012 ceph-mon[28265]: Detected new or changed devices on smithi134 2024-04-22T13:19:59.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:59 smithi012 ceph-mon[28265]: Adjusting osd_memory_target on smithi134 to 5696M 2024-04-22T13:19:59.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:59 smithi012 ceph-mon[28265]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:19:59.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:19:59 smithi012 ceph-mon[28265]: osdmap e62: 9 total, 9 up, 9 in 2024-04-22T13:19:59.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:59 smithi134 ceph-mon[31338]: Detected new or changed devices on smithi134 2024-04-22T13:19:59.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:59 smithi134 ceph-mon[31338]: Adjusting osd_memory_target on smithi134 to 5696M 2024-04-22T13:19:59.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:59 smithi134 ceph-mon[31338]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:19:59.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:19:59 smithi134 ceph-mon[31338]: osdmap e62: 9 total, 9 up, 9 in 2024-04-22T13:19:59.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:59 smithi111 ceph-mon[31361]: Detected new or changed devices on smithi134 2024-04-22T13:19:59.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:59 smithi111 ceph-mon[31361]: Adjusting osd_memory_target on smithi134 to 5696M 2024-04-22T13:19:59.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:59 smithi111 ceph-mon[31361]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:19:59.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:19:59 smithi111 ceph-mon[31361]: osdmap e62: 9 total, 9 up, 9 in 2024-04-22T13:20:00.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.564+0000 7fea12ce4700 1 -- 172.21.15.12:0/1017397654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea0c102ba0 msgr2=0x7fea0c103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:20:00.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.564+0000 7fea12ce4700 1 --2- 172.21.15.12:0/1017397654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea0c102ba0 0x7fea0c103000 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe9f80099c0 tx=0x7fe9f8009cd0 comp rx=0 tx=0).stop 2024-04-22T13:20:00.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.564+0000 7fea12ce4700 1 -- 172.21.15.12:0/1017397654 shutdown_connections 2024-04-22T13:20:00.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.564+0000 7fea12ce4700 1 --2- 172.21.15.12:0/1017397654 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea0c103540 0x7fea0c109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:00.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.564+0000 7fea12ce4700 1 --2- 172.21.15.12:0/1017397654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea0c1019a0 0x7fea0c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:00.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.564+0000 7fea12ce4700 1 --2- 172.21.15.12:0/1017397654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea0c102ba0 0x7fea0c103000 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:00.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.564+0000 7fea12ce4700 1 -- 172.21.15.12:0/1017397654 >> 172.21.15.12:0/1017397654 conn(0x7fea0c0fcf40 msgr2=0x7fea0c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:20:00.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.564+0000 7fea12ce4700 1 -- 172.21.15.12:0/1017397654 shutdown_connections 2024-04-22T13:20:00.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.564+0000 7fea12ce4700 1 -- 172.21.15.12:0/1017397654 wait complete. 2024-04-22T13:20:00.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.565+0000 7fea12ce4700 1 Processor -- start 2024-04-22T13:20:00.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.565+0000 7fea12ce4700 1 -- start start 2024-04-22T13:20:00.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.565+0000 7fea12ce4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea0c1019a0 0x7fea0c105930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:20:00.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.565+0000 7fea12ce4700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea0c102ba0 0x7fea0c103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:20:00.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.565+0000 7fea12ce4700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea0c103540 0x7fea0c1044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:20:00.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.565+0000 7fea12ce4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c10d490 con 0x7fea0c1019a0 2024-04-22T13:20:00.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.565+0000 7fea12ce4700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c10d310 con 0x7fea0c102ba0 2024-04-22T13:20:00.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.565+0000 7fea12ce4700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fea0c10d610 con 0x7fea0c103540 2024-04-22T13:20:00.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.565+0000 7fea10a80700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea0c1019a0 0x7fea0c105930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:20:00.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.565+0000 7fea10a80700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea0c1019a0 0x7fea0c105930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43046/0 (socket says 172.21.15.12:43046) 2024-04-22T13:20:00.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.565+0000 7fea10a80700 1 -- 172.21.15.12:0/3425424793 learned_addr learned my addr 172.21.15.12:0/3425424793 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:20:00.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.565+0000 7fea0bfff700 1 --2- 172.21.15.12:0/3425424793 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea0c102ba0 0x7fea0c103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:20:00.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.565+0000 7fea11281700 1 --2- 172.21.15.12:0/3425424793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea0c103540 0x7fea0c1044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:20:00.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.565+0000 7fea10a80700 1 -- 172.21.15.12:0/3425424793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea0c103540 msgr2=0x7fea0c1044c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:20:00.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.565+0000 7fea10a80700 1 --2- 172.21.15.12:0/3425424793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea0c103540 0x7fea0c1044c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:00.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.565+0000 7fea10a80700 1 -- 172.21.15.12:0/3425424793 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea0c102ba0 msgr2=0x7fea0c103f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:20:00.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.565+0000 7fea10a80700 1 --2- 172.21.15.12:0/3425424793 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea0c102ba0 0x7fea0c103f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:00.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.565+0000 7fea10a80700 1 -- 172.21.15.12:0/3425424793 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea0c104d70 con 0x7fea0c1019a0 2024-04-22T13:20:00.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.566+0000 7fea10a80700 1 --2- 172.21.15.12:0/3425424793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea0c1019a0 0x7fea0c105930 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fea0000c970 tx=0x7fea0000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:20:00.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.566+0000 7fea09ffb700 1 -- 172.21.15.12:0/3425424793 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00004220 con 0x7fea0c1019a0 2024-04-22T13:20:00.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.566+0000 7fea09ffb700 1 -- 172.21.15.12:0/3425424793 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea00007d50 con 0x7fea0c1019a0 2024-04-22T13:20:00.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.566+0000 7fea12ce4700 1 -- 172.21.15.12:0/3425424793 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea0c1a2de0 con 0x7fea0c1019a0 2024-04-22T13:20:00.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.566+0000 7fea09ffb700 1 -- 172.21.15.12:0/3425424793 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00003d00 con 0x7fea0c1019a0 2024-04-22T13:20:00.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.566+0000 7fea12ce4700 1 -- 172.21.15.12:0/3425424793 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea0c1a31e0 con 0x7fea0c1019a0 2024-04-22T13:20:00.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.567+0000 7fea12ce4700 1 -- 172.21.15.12:0/3425424793 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea0c1a3470 con 0x7fea0c1019a0 2024-04-22T13:20:00.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.568+0000 7fea09ffb700 1 -- 172.21.15.12:0/3425424793 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fea000078a0 con 0x7fea0c1019a0 2024-04-22T13:20:00.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.569+0000 7fea09ffb700 1 --2- 172.21.15.12:0/3425424793 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe9fc0a3c40 0x7fe9fc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:20:00.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.569+0000 7fea0bfff700 1 --2- 172.21.15.12:0/3425424793 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe9fc0a3c40 0x7fe9fc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:20:00.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.569+0000 7fea09ffb700 1 -- 172.21.15.12:0/3425424793 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5858+0+0 (secure 0 0 0) 0x7fea000cf630 con 0x7fea0c1019a0 2024-04-22T13:20:00.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.569+0000 7fea0bfff700 1 --2- 172.21.15.12:0/3425424793 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe9fc0a3c40 0x7fe9fc0a6100 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009990 tx=0x7fe9f8005c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:20:00.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.573+0000 7fea09ffb700 1 -- 172.21.15.12:0/3425424793 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fea00014030 con 0x7fea0c1019a0 2024-04-22T13:20:00.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:00.795+0000 7fea12ce4700 1 -- 172.21.15.12:0/3425424793 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fea0c0ff190 con 0x7fe9fc0a3c40 2024-04-22T13:20:00.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:00 smithi111 ceph-mon[31361]: osdmap e63: 9 total, 9 up, 9 in 2024-04-22T13:20:00.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:00 smithi111 ceph-mon[31361]: overall HEALTH_OK 2024-04-22T13:20:01.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:00 smithi012 ceph-mon[28265]: osdmap e63: 9 total, 9 up, 9 in 2024-04-22T13:20:01.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:00 smithi012 ceph-mon[28265]: overall HEALTH_OK 2024-04-22T13:20:01.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:00 smithi134 ceph-mon[31338]: osdmap e63: 9 total, 9 up, 9 in 2024-04-22T13:20:01.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:00 smithi134 ceph-mon[31338]: overall HEALTH_OK 2024-04-22T13:20:01.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:01 smithi111 ceph-mon[31361]: pgmap v201: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-22T13:20:01.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:01 smithi111 ceph-mon[31361]: osdmap e64: 9 total, 9 up, 9 in 2024-04-22T13:20:01.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:01 smithi111 ceph-mon[31361]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:20:01.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:01 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:20:01.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:01 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:20:01.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:01 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:02.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:01 smithi012 ceph-mon[28265]: pgmap v201: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-22T13:20:02.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:01 smithi012 ceph-mon[28265]: osdmap e64: 9 total, 9 up, 9 in 2024-04-22T13:20:02.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:01 smithi012 ceph-mon[28265]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:20:02.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:01 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:20:02.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:01 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:20:02.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:01 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:02.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:01 smithi134 ceph-mon[31338]: pgmap v201: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-22T13:20:02.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:01 smithi134 ceph-mon[31338]: osdmap e64: 9 total, 9 up, 9 in 2024-04-22T13:20:02.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:01 smithi134 ceph-mon[31338]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi012:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:20:02.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:01 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:20:02.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:01 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:20:02.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:01 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:02.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:02 smithi012 ceph-mon[28265]: osdmap e65: 9 total, 9 up, 9 in 2024-04-22T13:20:02.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:02 smithi111 ceph-mon[31361]: osdmap e65: 9 total, 9 up, 9 in 2024-04-22T13:20:03.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:02 smithi134 ceph-mon[31338]: osdmap e65: 9 total, 9 up, 9 in 2024-04-22T13:20:03.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:03 smithi111 ceph-mon[31361]: pgmap v204: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-22T13:20:03.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:03 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2015028402' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd745869-7f5f-48a8-b412-bb9c92b586e0"}]: dispatch 2024-04-22T13:20:03.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:03 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2015028402' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd745869-7f5f-48a8-b412-bb9c92b586e0"}]': finished 2024-04-22T13:20:03.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:03 smithi111 ceph-mon[31361]: osdmap e66: 10 total, 9 up, 10 in 2024-04-22T13:20:03.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:03 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:20:04.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:03 smithi012 ceph-mon[28265]: pgmap v204: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-22T13:20:04.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:03 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2015028402' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd745869-7f5f-48a8-b412-bb9c92b586e0"}]: dispatch 2024-04-22T13:20:04.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:03 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2015028402' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd745869-7f5f-48a8-b412-bb9c92b586e0"}]': finished 2024-04-22T13:20:04.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:03 smithi012 ceph-mon[28265]: osdmap e66: 10 total, 9 up, 10 in 2024-04-22T13:20:04.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:03 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:20:04.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:03 smithi134 ceph-mon[31338]: pgmap v204: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-22T13:20:04.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:03 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2015028402' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd745869-7f5f-48a8-b412-bb9c92b586e0"}]: dispatch 2024-04-22T13:20:04.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:03 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2015028402' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd745869-7f5f-48a8-b412-bb9c92b586e0"}]': finished 2024-04-22T13:20:04.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:03 smithi134 ceph-mon[31338]: osdmap e66: 10 total, 9 up, 10 in 2024-04-22T13:20:04.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:03 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:20:04.588 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:20:04 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:20:04.301+0000 7f80894fb700 -1 calc_pg_upmaps abort due to max <= 0 2024-04-22T13:20:04.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:04 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/651184349' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:20:05.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:04 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/651184349' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:20:05.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:04 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/651184349' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:20:05.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:05 smithi111 ceph-mon[31361]: pgmap v206: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-04-22T13:20:06.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:05 smithi012 ceph-mon[28265]: pgmap v206: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-04-22T13:20:06.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:05 smithi134 ceph-mon[31338]: pgmap v206: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-04-22T13:20:07.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:07 smithi111 ceph-mon[31361]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:20:08.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:07 smithi012 ceph-mon[28265]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:20:08.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:07 smithi134 ceph-mon[31338]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:20:09.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:09 smithi111 ceph-mon[31361]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:20:09.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:09 smithi012 ceph-mon[28265]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:20:10.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:09 smithi134 ceph-mon[31338]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:20:11.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:11 smithi111 ceph-mon[31361]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:20:11.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:11 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-22T13:20:11.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:11 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:12.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:11 smithi012 ceph-mon[28265]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:20:12.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:11 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-22T13:20:12.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:11 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:12.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:11 smithi134 ceph-mon[31338]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:20:12.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:11 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-22T13:20:12.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:11 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:12.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:12 smithi111 ceph-mon[31361]: Deploying daemon osd.9 on smithi012 2024-04-22T13:20:13.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:12 smithi012 ceph-mon[28265]: Deploying daemon osd.9 on smithi012 2024-04-22T13:20:13.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:12 smithi134 ceph-mon[31338]: Deploying daemon osd.9 on smithi012 2024-04-22T13:20:13.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:13 smithi111 ceph-mon[31361]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:20:13.994 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:13 smithi012 ceph-mon[28265]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:20:14.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:13 smithi134 ceph-mon[31338]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:20:16.051 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:15 smithi012 ceph-mon[28265]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:20:16.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:15 smithi134 ceph-mon[31338]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:20:16.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:15 smithi111 ceph-mon[31361]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:20:17.981 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:17 smithi134 ceph-mon[31338]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:20:17.981 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:17 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:17.981 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:17 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:17.981 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:17 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:20:18.029 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:17 smithi012 ceph-mon[28265]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:20:18.029 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:17 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:18.029 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:17 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:18.029 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:17 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:20:18.030 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:17 smithi111 ceph-mon[31361]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:20:18.030 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:17 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:18.030 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:17 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:18.030 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:17 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:20:19.939 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:19 smithi012 ceph-mon[28265]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:20:20.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:19 smithi134 ceph-mon[31338]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:20:20.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:19 smithi111 ceph-mon[31361]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:20:21.164 INFO:teuthology.orchestra.run.smithi012.stdout:Created osd(s) 9 on host 'smithi012' 2024-04-22T13:20:21.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:21.167+0000 7fea09ffb700 1 -- 172.21.15.12:0/3425424793 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fea0c0ff190 con 0x7fe9fc0a3c40 2024-04-22T13:20:21.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:21.170+0000 7fea12ce4700 1 -- 172.21.15.12:0/3425424793 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe9fc0a3c40 msgr2=0x7fe9fc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:20:21.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:21.170+0000 7fea12ce4700 1 --2- 172.21.15.12:0/3425424793 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe9fc0a3c40 0x7fe9fc0a6100 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009990 tx=0x7fe9f8005c20 comp rx=0 tx=0).stop 2024-04-22T13:20:21.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:21.170+0000 7fea12ce4700 1 -- 172.21.15.12:0/3425424793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea0c1019a0 msgr2=0x7fea0c105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:20:21.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:21.170+0000 7fea12ce4700 1 --2- 172.21.15.12:0/3425424793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea0c1019a0 0x7fea0c105930 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fea0000c970 tx=0x7fea0000cc80 comp rx=0 tx=0).stop 2024-04-22T13:20:21.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:21.170+0000 7fea12ce4700 1 -- 172.21.15.12:0/3425424793 shutdown_connections 2024-04-22T13:20:21.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:21.170+0000 7fea12ce4700 1 --2- 172.21.15.12:0/3425424793 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe9fc0a3c40 0x7fe9fc0a6100 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:21.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:21.170+0000 7fea12ce4700 1 --2- 172.21.15.12:0/3425424793 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea0c103540 0x7fea0c1044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:21.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:21.170+0000 7fea12ce4700 1 --2- 172.21.15.12:0/3425424793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea0c1019a0 0x7fea0c105930 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:21.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:21.170+0000 7fea12ce4700 1 --2- 172.21.15.12:0/3425424793 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea0c102ba0 0x7fea0c103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:21.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:21.170+0000 7fea12ce4700 1 -- 172.21.15.12:0/3425424793 >> 172.21.15.12:0/3425424793 conn(0x7fea0c0fcf40 msgr2=0x7fea0c0fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:20:21.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:21.170+0000 7fea12ce4700 1 -- 172.21.15.12:0/3425424793 shutdown_connections 2024-04-22T13:20:21.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:20:21.170+0000 7fea12ce4700 1 -- 172.21.15.12:0/3425424793 wait complete. 2024-04-22T13:20:21.750 DEBUG:teuthology.orchestra.run.smithi012:osd.9> sudo journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.9.service 2024-04-22T13:20:21.752 INFO:tasks.cephadm:Deploying osd.10 on smithi111 with /dev/vg_nvme/lv_1... 2024-04-22T13:20:21.753 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-22T13:20:21.998 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T13:20:22.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:22 smithi134 ceph-mon[31338]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:20:22.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:22 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:22.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:22 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:22.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:22 smithi134 ceph-mon[31338]: from='osd.9 [v2:172.21.15.12:6826/4292385779,v1:172.21.15.12:6827/4292385779]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-22T13:20:22.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:22 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:22.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:22 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:22.444 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:22 smithi111 ceph-mon[31361]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:20:22.444 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:22 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:22.444 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:22 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:22.444 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:22 smithi111 ceph-mon[31361]: from='osd.9 [v2:172.21.15.12:6826/4292385779,v1:172.21.15.12:6827/4292385779]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-22T13:20:22.444 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:22 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:22.444 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:22 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:22.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:22 smithi012 ceph-mon[28265]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:20:22.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:22 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:22.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:22 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:22.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:22 smithi012 ceph-mon[28265]: from='osd.9 [v2:172.21.15.12:6826/4292385779,v1:172.21.15.12:6827/4292385779]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-22T13:20:22.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:22 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:22.574 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:22 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:23.421 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:23 smithi111 ceph-mon[31361]: from='osd.9 [v2:172.21.15.12:6826/4292385779,v1:172.21.15.12:6827/4292385779]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-22T13:20:23.421 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:23 smithi111 ceph-mon[31361]: osdmap e67: 10 total, 9 up, 10 in 2024-04-22T13:20:23.421 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:23 smithi111 ceph-mon[31361]: from='osd.9 [v2:172.21.15.12:6826/4292385779,v1:172.21.15.12:6827/4292385779]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-22T13:20:23.422 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:23 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:20:23.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:23 smithi134 ceph-mon[31338]: from='osd.9 [v2:172.21.15.12:6826/4292385779,v1:172.21.15.12:6827/4292385779]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-22T13:20:23.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:23 smithi134 ceph-mon[31338]: osdmap e67: 10 total, 9 up, 10 in 2024-04-22T13:20:23.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:23 smithi134 ceph-mon[31338]: from='osd.9 [v2:172.21.15.12:6826/4292385779,v1:172.21.15.12:6827/4292385779]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-22T13:20:23.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:23 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:20:23.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:23 smithi012 ceph-mon[28265]: from='osd.9 [v2:172.21.15.12:6826/4292385779,v1:172.21.15.12:6827/4292385779]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-22T13:20:23.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:23 smithi012 ceph-mon[28265]: osdmap e67: 10 total, 9 up, 10 in 2024-04-22T13:20:23.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:23 smithi012 ceph-mon[28265]: from='osd.9 [v2:172.21.15.12:6826/4292385779,v1:172.21.15.12:6827/4292385779]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]: dispatch 2024-04-22T13:20:23.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:23 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:20:23.589 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:20:23 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:20:23.152+0000 7f52001bd700 -1 osd.9 0 waiting for initial osdmap 2024-04-22T13:20:23.589 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:20:23 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:20:23.168+0000 7f51fbfb0700 -1 osd.9 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T13:20:23.975 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-22T13:20:23.976 INFO:teuthology.orchestra.run.smithi111.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-22T13:20:23.976 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10+0 records in 2024-04-22T13:20:23.976 INFO:teuthology.orchestra.run.smithi111.stderr:10+0 records out 2024-04-22T13:20:23.976 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128876 s, 814 MB/s 2024-04-22T13:20:23.976 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping successful for: 2024-04-22T13:20:24.312 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:24 smithi111 ceph-mon[31361]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:20:24.312 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:24 smithi111 ceph-mon[31361]: from='osd.9 [v2:172.21.15.12:6826/4292385779,v1:172.21.15.12:6827/4292385779]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-04-22T13:20:24.312 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:24 smithi111 ceph-mon[31361]: osdmap e68: 10 total, 9 up, 10 in 2024-04-22T13:20:24.312 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:24 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:20:24.312 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:24 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:20:24.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:24 smithi012 ceph-mon[28265]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:20:24.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:24 smithi012 ceph-mon[28265]: from='osd.9 [v2:172.21.15.12:6826/4292385779,v1:172.21.15.12:6827/4292385779]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-04-22T13:20:24.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:24 smithi012 ceph-mon[28265]: osdmap e68: 10 total, 9 up, 10 in 2024-04-22T13:20:24.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:24 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:20:24.339 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:24 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:20:24.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:24 smithi134 ceph-mon[31338]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:20:24.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:24 smithi134 ceph-mon[31338]: from='osd.9 [v2:172.21.15.12:6826/4292385779,v1:172.21.15.12:6827/4292385779]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi012", "root=default"]}]': finished 2024-04-22T13:20:24.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:24 smithi134 ceph-mon[31338]: osdmap e68: 10 total, 9 up, 10 in 2024-04-22T13:20:24.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:24 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:20:24.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:24 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:20:24.630 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_1 2024-04-22T13:20:24.864 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T13:20:25.298 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:25 smithi111 ceph-mon[31361]: purged_snaps scrub starts 2024-04-22T13:20:25.298 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:25 smithi111 ceph-mon[31361]: purged_snaps scrub ok 2024-04-22T13:20:25.298 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:25 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:20:25.299 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:25 smithi111 ceph-mon[31361]: osd.9 [v2:172.21.15.12:6826/4292385779,v1:172.21.15.12:6827/4292385779] boot 2024-04-22T13:20:25.299 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:25 smithi111 ceph-mon[31361]: osdmap e69: 10 total, 10 up, 10 in 2024-04-22T13:20:25.299 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:25 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:20:25.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:25 smithi134 ceph-mon[31338]: purged_snaps scrub starts 2024-04-22T13:20:25.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:25 smithi134 ceph-mon[31338]: purged_snaps scrub ok 2024-04-22T13:20:25.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:25 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:20:25.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:25 smithi134 ceph-mon[31338]: osd.9 [v2:172.21.15.12:6826/4292385779,v1:172.21.15.12:6827/4292385779] boot 2024-04-22T13:20:25.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:25 smithi134 ceph-mon[31338]: osdmap e69: 10 total, 10 up, 10 in 2024-04-22T13:20:25.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:25 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:20:25.462 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:25 smithi012 ceph-mon[28265]: purged_snaps scrub starts 2024-04-22T13:20:25.462 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:25 smithi012 ceph-mon[28265]: purged_snaps scrub ok 2024-04-22T13:20:25.462 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:25 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:20:25.462 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:25 smithi012 ceph-mon[28265]: osd.9 [v2:172.21.15.12:6826/4292385779,v1:172.21.15.12:6827/4292385779] boot 2024-04-22T13:20:25.462 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:25 smithi012 ceph-mon[28265]: osdmap e69: 10 total, 10 up, 10 in 2024-04-22T13:20:25.463 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:25 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:20:26.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:26 smithi111 ceph-mon[31361]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:20:26.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:26 smithi111 ceph-mon[31361]: osdmap e70: 10 total, 10 up, 10 in 2024-04-22T13:20:26.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:26 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:26.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:26 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:26.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:26 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:20:26.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:26 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:20:26.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:26 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:20:26.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:26 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:20:26.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:26 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:26.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:26 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:26.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:26 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:20:26.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:26 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:26.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:26 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:20:26.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:26 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:26.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:26 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:20:26.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:26 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:26.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:26 smithi134 ceph-mon[31338]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:20:26.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:26 smithi134 ceph-mon[31338]: osdmap e70: 10 total, 10 up, 10 in 2024-04-22T13:20:26.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:26 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:26.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:26 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:26.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:26 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:20:26.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:26 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:20:26.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:26 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:20:26.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:26 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:20:26.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:26 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:26.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:26 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:26.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:26 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:20:26.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:26 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:26.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:26 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:20:26.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:26 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:26.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:26 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:20:26.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:26 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:26.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.452+0000 7f2797634700 1 -- 172.21.15.111:0/2012909435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27901014f0 msgr2=0x7f279010b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:20:26.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.452+0000 7f2797634700 1 --2- 172.21.15.111:0/2012909435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27901014f0 0x7f279010b0c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f278800b0a0 tx=0x7f278800b3b0 comp rx=0 tx=0).stop 2024-04-22T13:20:26.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.452+0000 7f2797634700 1 -- 172.21.15.111:0/2012909435 shutdown_connections 2024-04-22T13:20:26.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.452+0000 7f2797634700 1 --2- 172.21.15.111:0/2012909435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27901014f0 0x7f279010b0c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:26.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.452+0000 7f2797634700 1 --2- 172.21.15.111:0/2012909435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27901040d0 0x7f27901044f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:26.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.452+0000 7f2797634700 1 --2- 172.21.15.111:0/2012909435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2790100b50 0x7f2790100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:26.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.452+0000 7f2797634700 1 -- 172.21.15.111:0/2012909435 >> 172.21.15.111:0/2012909435 conn(0x7f27900fc6b0 msgr2=0x7f27900feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:20:26.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.452+0000 7f2797634700 1 -- 172.21.15.111:0/2012909435 shutdown_connections 2024-04-22T13:20:26.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.452+0000 7f2797634700 1 -- 172.21.15.111:0/2012909435 wait complete. 2024-04-22T13:20:26.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.453+0000 7f2797634700 1 Processor -- start 2024-04-22T13:20:26.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.453+0000 7f2797634700 1 -- start start 2024-04-22T13:20:26.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.453+0000 7f2797634700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2790100b50 0x7f2790198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:20:26.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.453+0000 7f2797634700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27901014f0 0x7f2790199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:20:26.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.453+0000 7f2797634700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27901040d0 0x7f27901a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:20:26.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.453+0000 7f2797634700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f279010e5b0 con 0x7f2790100b50 2024-04-22T13:20:26.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.453+0000 7f2797634700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f279010e430 con 0x7f27901014f0 2024-04-22T13:20:26.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.453+0000 7f2797634700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f279010e730 con 0x7f27901040d0 2024-04-22T13:20:26.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.453+0000 7f2794bcf700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27901014f0 0x7f2790199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:20:26.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.453+0000 7f2794bcf700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27901014f0 0x7f2790199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45632/0 (socket says 172.21.15.111:45632) 2024-04-22T13:20:26.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.453+0000 7f2794bcf700 1 -- 172.21.15.111:0/4065959731 learned_addr learned my addr 172.21.15.111:0/4065959731 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-04-22T13:20:26.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.453+0000 7f2795bd1700 1 --2- 172.21.15.111:0/4065959731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27901040d0 0x7f27901a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:20:26.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.453+0000 7f2794bcf700 1 -- 172.21.15.111:0/4065959731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27901040d0 msgr2=0x7f27901a00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:20:26.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.453+0000 7f2794bcf700 1 --2- 172.21.15.111:0/4065959731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27901040d0 0x7f27901a00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:26.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.453+0000 7f2794bcf700 1 -- 172.21.15.111:0/4065959731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2790100b50 msgr2=0x7f2790198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:20:26.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.453+0000 7f2794bcf700 1 --2- 172.21.15.111:0/4065959731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2790100b50 0x7f2790198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:26.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.453+0000 7f2794bcf700 1 -- 172.21.15.111:0/4065959731 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27901a0760 con 0x7f27901014f0 2024-04-22T13:20:26.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.453+0000 7f2795bd1700 1 --2- 172.21.15.111:0/4065959731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27901040d0 0x7f27901a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:20:26.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.453+0000 7f2794bcf700 1 --2- 172.21.15.111:0/4065959731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27901014f0 0x7f2790199040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f278c00c9b0 tx=0x7f278c00cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:20:26.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.454+0000 7f27867fc700 1 -- 172.21.15.111:0/4065959731 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c007b00 con 0x7f27901014f0 2024-04-22T13:20:26.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.454+0000 7f27867fc700 1 -- 172.21.15.111:0/4065959731 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f278c007ca0 con 0x7f27901014f0 2024-04-22T13:20:26.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.454+0000 7f27867fc700 1 -- 172.21.15.111:0/4065959731 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c0186f0 con 0x7f27901014f0 2024-04-22T13:20:26.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.454+0000 7f2797634700 1 -- 172.21.15.111:0/4065959731 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27901a0a50 con 0x7f27901014f0 2024-04-22T13:20:26.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.454+0000 7f2797634700 1 -- 172.21.15.111:0/4065959731 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27901a1060 con 0x7f27901014f0 2024-04-22T13:20:26.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.455+0000 7f2797634700 1 -- 172.21.15.111:0/4065959731 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2790069730 con 0x7f27901014f0 2024-04-22T13:20:26.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.456+0000 7f27867fc700 1 -- 172.21.15.111:0/4065959731 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f278c00f450 con 0x7f27901014f0 2024-04-22T13:20:26.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.457+0000 7f27867fc700 1 --2- 172.21.15.111:0/4065959731 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f277c0a3cc0 0x7f277c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:20:26.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.457+0000 7f27867fc700 1 -- 172.21.15.111:0/4065959731 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6261+0+0 (secure 0 0 0) 0x7f278c0cfe40 con 0x7f27901014f0 2024-04-22T13:20:26.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.457+0000 7f27953d0700 1 --2- 172.21.15.111:0/4065959731 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f277c0a3cc0 0x7f277c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:20:26.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.457+0000 7f27953d0700 1 --2- 172.21.15.111:0/4065959731 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f277c0a3cc0 0x7f277c0a6180 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f279006aed0 tx=0x7f2780006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:20:26.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.460+0000 7f27867fc700 1 -- 172.21.15.111:0/4065959731 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f278c014030 con 0x7f27901014f0 2024-04-22T13:20:26.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:26 smithi012 ceph-mon[28265]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-22T13:20:26.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:26 smithi012 ceph-mon[28265]: osdmap e70: 10 total, 10 up, 10 in 2024-04-22T13:20:26.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:26 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:26.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:26 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:26.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:26 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:20:26.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:26 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:20:26.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:26 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:20:26.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:26 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:20:26.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:26 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:26.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:26 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:26.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:26 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:20:26.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:26 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:26.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:26 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:20:26.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:26 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:26.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:26 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:20:26.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:26 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:26.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:26.667+0000 7f2797634700 1 -- 172.21.15.111:0/4065959731 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f2790062e80 con 0x7f277c0a3cc0 2024-04-22T13:20:27.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:27 smithi134 ceph-mon[31338]: Detected new or changed devices on smithi012 2024-04-22T13:20:27.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:27 smithi134 ceph-mon[31338]: Adjusting osd_memory_target on smithi012 to 4273M 2024-04-22T13:20:27.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:27 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:20:27.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:27 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:20:27.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:27 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:27.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:27 smithi111 ceph-mon[31361]: Detected new or changed devices on smithi012 2024-04-22T13:20:27.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:27 smithi111 ceph-mon[31361]: Adjusting osd_memory_target on smithi012 to 4273M 2024-04-22T13:20:27.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:27 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:20:27.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:27 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:20:27.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:27 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:27.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:27 smithi012 ceph-mon[28265]: Detected new or changed devices on smithi012 2024-04-22T13:20:27.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:27 smithi012 ceph-mon[28265]: Adjusting osd_memory_target on smithi012 to 4273M 2024-04-22T13:20:27.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:27 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:20:27.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:27 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:20:27.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:27 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:28.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:28 smithi134 ceph-mon[31338]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:28.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:28 smithi134 ceph-mon[31338]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:20:28.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:28 smithi111 ceph-mon[31361]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:28.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:28 smithi111 ceph-mon[31361]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:20:28.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:28 smithi012 ceph-mon[28265]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:28.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:28 smithi012 ceph-mon[28265]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:20:30.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:30 smithi012 ceph-mon[28265]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:30.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:30 smithi012 ceph-mon[28265]: from='client.? 172.21.15.111:0/1366761417' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "798b1abd-f6c7-4a5f-8da0-fdae49e50d2c"}]: dispatch 2024-04-22T13:20:30.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:30 smithi012 ceph-mon[28265]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "798b1abd-f6c7-4a5f-8da0-fdae49e50d2c"}]: dispatch 2024-04-22T13:20:30.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:30 smithi012 ceph-mon[28265]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "798b1abd-f6c7-4a5f-8da0-fdae49e50d2c"}]': finished 2024-04-22T13:20:30.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:30 smithi012 ceph-mon[28265]: osdmap e71: 11 total, 10 up, 11 in 2024-04-22T13:20:30.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:30 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:20:30.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:30 smithi012 ceph-mon[28265]: from='client.? 172.21.15.111:0/3637255787' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:20:30.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:30 smithi134 ceph-mon[31338]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:30.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:30 smithi134 ceph-mon[31338]: from='client.? 172.21.15.111:0/1366761417' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "798b1abd-f6c7-4a5f-8da0-fdae49e50d2c"}]: dispatch 2024-04-22T13:20:30.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:30 smithi134 ceph-mon[31338]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "798b1abd-f6c7-4a5f-8da0-fdae49e50d2c"}]: dispatch 2024-04-22T13:20:30.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:30 smithi134 ceph-mon[31338]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "798b1abd-f6c7-4a5f-8da0-fdae49e50d2c"}]': finished 2024-04-22T13:20:30.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:30 smithi134 ceph-mon[31338]: osdmap e71: 11 total, 10 up, 11 in 2024-04-22T13:20:30.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:30 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:20:30.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:30 smithi134 ceph-mon[31338]: from='client.? 172.21.15.111:0/3637255787' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:20:30.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:30 smithi111 ceph-mon[31361]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:30.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:30 smithi111 ceph-mon[31361]: from='client.? 172.21.15.111:0/1366761417' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "798b1abd-f6c7-4a5f-8da0-fdae49e50d2c"}]: dispatch 2024-04-22T13:20:30.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:30 smithi111 ceph-mon[31361]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "798b1abd-f6c7-4a5f-8da0-fdae49e50d2c"}]: dispatch 2024-04-22T13:20:30.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:30 smithi111 ceph-mon[31361]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "798b1abd-f6c7-4a5f-8da0-fdae49e50d2c"}]': finished 2024-04-22T13:20:30.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:30 smithi111 ceph-mon[31361]: osdmap e71: 11 total, 10 up, 11 in 2024-04-22T13:20:30.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:30 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:20:30.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:30 smithi111 ceph-mon[31361]: from='client.? 172.21.15.111:0/3637255787' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:20:32.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:32 smithi111 ceph-mon[31361]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:32.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:32 smithi012 ceph-mon[28265]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:32.680 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:32 smithi134 ceph-mon[31338]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:34.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:34 smithi111 ceph-mon[31361]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:34.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:34 smithi012 ceph-mon[28265]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:34.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:34 smithi134 ceph-mon[31338]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:36.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:36 smithi111 ceph-mon[31361]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:36.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:36 smithi012 ceph-mon[28265]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:36.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:36 smithi134 ceph-mon[31338]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:37.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:37 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-22T13:20:37.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:37 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:37.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:37 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-22T13:20:37.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:37 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:37.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:37 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-22T13:20:37.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:37 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:38.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:38 smithi111 ceph-mon[31361]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:38.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:38 smithi111 ceph-mon[31361]: Deploying daemon osd.10 on smithi111 2024-04-22T13:20:38.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:38 smithi012 ceph-mon[28265]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:38.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:38 smithi012 ceph-mon[28265]: Deploying daemon osd.10 on smithi111 2024-04-22T13:20:38.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:38 smithi134 ceph-mon[31338]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:38.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:38 smithi134 ceph-mon[31338]: Deploying daemon osd.10 on smithi111 2024-04-22T13:20:40.359 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:40 smithi111 ceph-mon[31361]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:40.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:40 smithi012 ceph-mon[28265]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:40.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:40 smithi134 ceph-mon[31338]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:42.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:42 smithi111 ceph-mon[31361]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:42.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:42 smithi012 ceph-mon[28265]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:42.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:42 smithi134 ceph-mon[31338]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:43.682 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:43 smithi111 ceph-mon[31361]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:43.683 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:43 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:43.683 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:43 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:43.683 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:43 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:20:43.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:43 smithi012 ceph-mon[28265]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:43.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:43.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:43.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:20:43.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:43 smithi134 ceph-mon[31338]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:43.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:43 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:43.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:43 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:43.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:43 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:20:45.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:45 smithi111 ceph-mon[31361]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:45.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:45.835+0000 7f27867fc700 1 -- 172.21.15.111:0/4065959731 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f2790062e80 con 0x7f277c0a3cc0 2024-04-22T13:20:45.833 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 10 on host 'smithi111' 2024-04-22T13:20:45.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:45.839+0000 7f2797634700 1 -- 172.21.15.111:0/4065959731 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f277c0a3cc0 msgr2=0x7f277c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:20:45.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:45.839+0000 7f2797634700 1 --2- 172.21.15.111:0/4065959731 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f277c0a3cc0 0x7f277c0a6180 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f279006aed0 tx=0x7f2780006d20 comp rx=0 tx=0).stop 2024-04-22T13:20:45.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:45.839+0000 7f2797634700 1 -- 172.21.15.111:0/4065959731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27901014f0 msgr2=0x7f2790199040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:20:45.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:45.839+0000 7f2797634700 1 --2- 172.21.15.111:0/4065959731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27901014f0 0x7f2790199040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f278c00c9b0 tx=0x7f278c00cd70 comp rx=0 tx=0).stop 2024-04-22T13:20:45.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:45.839+0000 7f2797634700 1 -- 172.21.15.111:0/4065959731 shutdown_connections 2024-04-22T13:20:45.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:45.839+0000 7f2797634700 1 --2- 172.21.15.111:0/4065959731 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f277c0a3cc0 0x7f277c0a6180 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:45.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:45.839+0000 7f2797634700 1 --2- 172.21.15.111:0/4065959731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27901040d0 0x7f27901a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:45.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:45.839+0000 7f2797634700 1 --2- 172.21.15.111:0/4065959731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2790100b50 0x7f2790198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:45.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:45.839+0000 7f2797634700 1 --2- 172.21.15.111:0/4065959731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27901014f0 0x7f2790199040 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:45.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:45.839+0000 7f2797634700 1 -- 172.21.15.111:0/4065959731 >> 172.21.15.111:0/4065959731 conn(0x7f27900fc6b0 msgr2=0x7f27901070a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:20:45.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:45.839+0000 7f2797634700 1 -- 172.21.15.111:0/4065959731 shutdown_connections 2024-04-22T13:20:45.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-04-22T13:20:45.839+0000 7f2797634700 1 -- 172.21.15.111:0/4065959731 wait complete. 2024-04-22T13:20:45.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:45 smithi012 ceph-mon[28265]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:45.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:45 smithi134 ceph-mon[31338]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:46.417 DEBUG:teuthology.orchestra.run.smithi111:osd.10> sudo journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.10.service 2024-04-22T13:20:46.419 INFO:tasks.cephadm:Deploying osd.11 on smithi134 with /dev/vg_nvme/lv_1... 2024-04-22T13:20:46.420 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-22T13:20:46.668 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:20:47.058 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:46 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:47.058 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:46 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:47.059 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:46 smithi134 ceph-mon[31338]: from='osd.10 [v2:172.21.15.111:6824/3338566976,v1:172.21.15.111:6825/3338566976]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-22T13:20:47.059 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:46 smithi134 ceph-mon[31338]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-22T13:20:47.059 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:46 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:47.059 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:46 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:47.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:46 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:47.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:46 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:47.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:46 smithi012 ceph-mon[28265]: from='osd.10 [v2:172.21.15.111:6824/3338566976,v1:172.21.15.111:6825/3338566976]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-22T13:20:47.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:46 smithi012 ceph-mon[28265]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-22T13:20:47.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:46 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:47.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:46 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:47.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:46 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:47.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:46 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:47.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:46 smithi111 ceph-mon[31361]: from='osd.10 [v2:172.21.15.111:6824/3338566976,v1:172.21.15.111:6825/3338566976]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-22T13:20:47.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:46 smithi111 ceph-mon[31361]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-22T13:20:47.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:46 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:47.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:46 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:48.021 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:47 smithi134 ceph-mon[31338]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:48.022 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:47 smithi134 ceph-mon[31338]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-22T13:20:48.022 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:47 smithi134 ceph-mon[31338]: osdmap e72: 11 total, 10 up, 11 in 2024-04-22T13:20:48.022 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:47 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:20:48.022 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:47 smithi134 ceph-mon[31338]: from='osd.10 [v2:172.21.15.111:6824/3338566976,v1:172.21.15.111:6825/3338566976]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-22T13:20:48.022 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:47 smithi134 ceph-mon[31338]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-22T13:20:48.078 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:47 smithi111 ceph-mon[31361]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:48.078 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:47 smithi111 ceph-mon[31361]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-22T13:20:48.079 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:47 smithi111 ceph-mon[31361]: osdmap e72: 11 total, 10 up, 11 in 2024-04-22T13:20:48.079 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:47 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:20:48.079 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:47 smithi111 ceph-mon[31361]: from='osd.10 [v2:172.21.15.111:6824/3338566976,v1:172.21.15.111:6825/3338566976]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-22T13:20:48.079 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:47 smithi111 ceph-mon[31361]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-22T13:20:48.079 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:20:47 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:20:47.817+0000 7f2638e71700 -1 osd.10 0 waiting for initial osdmap 2024-04-22T13:20:48.079 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:20:47 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:20:47.831+0000 7f2634c64700 -1 osd.10 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T13:20:48.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:47 smithi012 ceph-mon[28265]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:48.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:47 smithi012 ceph-mon[28265]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-22T13:20:48.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:47 smithi012 ceph-mon[28265]: osdmap e72: 11 total, 10 up, 11 in 2024-04-22T13:20:48.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:47 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:20:48.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:47 smithi012 ceph-mon[28265]: from='osd.10 [v2:172.21.15.111:6824/3338566976,v1:172.21.15.111:6825/3338566976]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-22T13:20:48.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:47 smithi012 ceph-mon[28265]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-04-22T13:20:48.592 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-22T13:20:48.592 INFO:teuthology.orchestra.run.smithi134.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-22T13:20:48.592 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10+0 records in 2024-04-22T13:20:48.592 INFO:teuthology.orchestra.run.smithi134.stderr:10+0 records out 2024-04-22T13:20:48.592 INFO:teuthology.orchestra.run.smithi134.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111483 s, 941 MB/s 2024-04-22T13:20:48.592 INFO:teuthology.orchestra.run.smithi134.stderr:--> Zapping successful for: 2024-04-22T13:20:48.919 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:48 smithi134 ceph-mon[31338]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-22T13:20:48.919 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:48 smithi134 ceph-mon[31338]: osdmap e73: 11 total, 10 up, 11 in 2024-04-22T13:20:48.920 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:48 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:20:48.920 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:48 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:20:48.920 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:48 smithi134 ceph-mon[31338]: osd.10 [v2:172.21.15.111:6824/3338566976,v1:172.21.15.111:6825/3338566976] boot 2024-04-22T13:20:48.920 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:48 smithi134 ceph-mon[31338]: osdmap e74: 11 total, 11 up, 11 in 2024-04-22T13:20:48.920 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:48 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:20:49.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:48 smithi012 ceph-mon[28265]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-22T13:20:49.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:48 smithi012 ceph-mon[28265]: osdmap e73: 11 total, 10 up, 11 in 2024-04-22T13:20:49.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:48 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:20:49.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:48 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:20:49.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:48 smithi012 ceph-mon[28265]: osd.10 [v2:172.21.15.111:6824/3338566976,v1:172.21.15.111:6825/3338566976] boot 2024-04-22T13:20:49.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:48 smithi012 ceph-mon[28265]: osdmap e74: 11 total, 11 up, 11 in 2024-04-22T13:20:49.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:48 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:20:49.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:48 smithi111 ceph-mon[31361]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-04-22T13:20:49.155 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:48 smithi111 ceph-mon[31361]: osdmap e73: 11 total, 10 up, 11 in 2024-04-22T13:20:49.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:48 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:20:49.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:48 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:20:49.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:48 smithi111 ceph-mon[31361]: osd.10 [v2:172.21.15.111:6824/3338566976,v1:172.21.15.111:6825/3338566976] boot 2024-04-22T13:20:49.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:48 smithi111 ceph-mon[31361]: osdmap e74: 11 total, 11 up, 11 in 2024-04-22T13:20:49.156 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:48 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:20:49.330 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi134:vg_nvme/lv_1 2024-04-22T13:20:49.564 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:20:49.990 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:49 smithi134 ceph-mon[31338]: purged_snaps scrub starts 2024-04-22T13:20:49.990 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:49 smithi134 ceph-mon[31338]: purged_snaps scrub ok 2024-04-22T13:20:49.990 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:49 smithi134 ceph-mon[31338]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:49.990 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:49 smithi134 ceph-mon[31338]: osdmap e75: 11 total, 11 up, 11 in 2024-04-22T13:20:50.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:49 smithi012 ceph-mon[28265]: purged_snaps scrub starts 2024-04-22T13:20:50.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:49 smithi012 ceph-mon[28265]: purged_snaps scrub ok 2024-04-22T13:20:50.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:49 smithi012 ceph-mon[28265]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:50.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:49 smithi012 ceph-mon[28265]: osdmap e75: 11 total, 11 up, 11 in 2024-04-22T13:20:50.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:49 smithi111 ceph-mon[31361]: purged_snaps scrub starts 2024-04-22T13:20:50.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:49 smithi111 ceph-mon[31361]: purged_snaps scrub ok 2024-04-22T13:20:50.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:49 smithi111 ceph-mon[31361]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-22T13:20:50.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:49 smithi111 ceph-mon[31361]: osdmap e75: 11 total, 11 up, 11 in 2024-04-22T13:20:50.980 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:50 smithi134 ceph-mon[31338]: Detected new or changed devices on smithi111 2024-04-22T13:20:50.980 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:50 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:50.980 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:50 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:50.981 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:50 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:20:50.981 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:50 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:20:50.981 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:50 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:20:50.981 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:50 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:20:50.981 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:50 smithi134 ceph-mon[31338]: Adjusting osd_memory_target on smithi111 to 4272M 2024-04-22T13:20:50.981 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:50 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:50.981 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:50 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:50.981 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:50 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:20:50.981 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:50 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:50.981 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:50 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:20:50.981 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:50 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:50.981 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:50 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:20:50.981 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:50 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:50.981 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:50 smithi134 ceph-mon[31338]: osdmap e76: 11 total, 11 up, 11 in 2024-04-22T13:20:51.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.178+0000 7f9d9ceda700 1 -- 172.21.15.134:0/2531976944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d98103560 msgr2=0x7f9d98109e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:20:51.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.178+0000 7f9d9ceda700 1 --2- 172.21.15.134:0/2531976944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d98103560 0x7f9d98109e10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000b0a0 tx=0x7f9d9000b3b0 comp rx=0 tx=0).stop 2024-04-22T13:20:51.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.178+0000 7f9d9ceda700 1 -- 172.21.15.134:0/2531976944 shutdown_connections 2024-04-22T13:20:51.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.178+0000 7f9d9ceda700 1 --2- 172.21.15.134:0/2531976944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d98103560 0x7f9d98109e10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:51.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.178+0000 7f9d9ceda700 1 --2- 172.21.15.134:0/2531976944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d981019c0 0x7f9d98101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:51.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.178+0000 7f9d9ceda700 1 --2- 172.21.15.134:0/2531976944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d98102bc0 0x7f9d98103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:51.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.178+0000 7f9d9ceda700 1 -- 172.21.15.134:0/2531976944 >> 172.21.15.134:0/2531976944 conn(0x7f9d980fcf60 msgr2=0x7f9d980ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:20:51.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.178+0000 7f9d9ceda700 1 -- 172.21.15.134:0/2531976944 shutdown_connections 2024-04-22T13:20:51.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.178+0000 7f9d9ceda700 1 -- 172.21.15.134:0/2531976944 wait complete. 2024-04-22T13:20:51.177 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.178+0000 7f9d9ceda700 1 Processor -- start 2024-04-22T13:20:51.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.178+0000 7f9d9ceda700 1 -- start start 2024-04-22T13:20:51.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.179+0000 7f9d9ceda700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d981019c0 0x7f9d98198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:20:51.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.179+0000 7f9d9ceda700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d98102bc0 0x7f9d98199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:20:51.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.179+0000 7f9d9ceda700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d98103560 0x7f9d981a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:20:51.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.179+0000 7f9d9ceda700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9810d460 con 0x7f9d981019c0 2024-04-22T13:20:51.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.179+0000 7f9d9ceda700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9810d2e0 con 0x7f9d98103560 2024-04-22T13:20:51.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.179+0000 7f9d9ceda700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9810d5e0 con 0x7f9d98102bc0 2024-04-22T13:20:51.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.179+0000 7f9d95d9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d98102bc0 0x7f9d98199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:20:51.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.179+0000 7f9d95d9b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d98102bc0 0x7f9d98199070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:45982/0 (socket says 172.21.15.134:45982) 2024-04-22T13:20:51.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.179+0000 7f9d95d9b700 1 -- 172.21.15.134:0/2732196956 learned_addr learned my addr 172.21.15.134:0/2732196956 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-22T13:20:51.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.179+0000 7f9d9659c700 1 --2- 172.21.15.134:0/2732196956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d981019c0 0x7f9d98198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:20:51.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.179+0000 7f9d96d9d700 1 --2- 172.21.15.134:0/2732196956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d98103560 0x7f9d981a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:20:51.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.179+0000 7f9d95d9b700 1 -- 172.21.15.134:0/2732196956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d98103560 msgr2=0x7f9d981a00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:20:51.178 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.179+0000 7f9d95d9b700 1 --2- 172.21.15.134:0/2732196956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d98103560 0x7f9d981a00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:51.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.179+0000 7f9d95d9b700 1 -- 172.21.15.134:0/2732196956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d981019c0 msgr2=0x7f9d98198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:20:51.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.179+0000 7f9d95d9b700 1 --2- 172.21.15.134:0/2732196956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d981019c0 0x7f9d98198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:20:51.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.179+0000 7f9d95d9b700 1 -- 172.21.15.134:0/2732196956 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d981a0790 con 0x7f9d98102bc0 2024-04-22T13:20:51.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.179+0000 7f9d96d9d700 1 --2- 172.21.15.134:0/2732196956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d98103560 0x7f9d981a00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:20:51.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.179+0000 7f9d9659c700 1 --2- 172.21.15.134:0/2732196956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d981019c0 0x7f9d98198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:20:51.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.179+0000 7f9d95d9b700 1 --2- 172.21.15.134:0/2732196956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d98102bc0 0x7f9d98199070 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00c9a0 tx=0x7f9d8c00ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:20:51.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.179+0000 7f9d877fe700 1 -- 172.21.15.134:0/2732196956 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c007a60 con 0x7f9d98102bc0 2024-04-22T13:20:51.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.179+0000 7f9d877fe700 1 -- 172.21.15.134:0/2732196956 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d8c004ce0 con 0x7f9d98102bc0 2024-04-22T13:20:51.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.179+0000 7f9d877fe700 1 -- 172.21.15.134:0/2732196956 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c005560 con 0x7f9d98102bc0 2024-04-22T13:20:51.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.179+0000 7f9d9ceda700 1 -- 172.21.15.134:0/2732196956 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d981a0a80 con 0x7f9d98102bc0 2024-04-22T13:20:51.179 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.180+0000 7f9d9ceda700 1 -- 172.21.15.134:0/2732196956 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d981a0f70 con 0x7f9d98102bc0 2024-04-22T13:20:51.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.180+0000 7f9d9ceda700 1 -- 172.21.15.134:0/2732196956 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d98069730 con 0x7f9d98102bc0 2024-04-22T13:20:51.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.181+0000 7f9d877fe700 1 -- 172.21.15.134:0/2732196956 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9d8c007c00 con 0x7f9d98102bc0 2024-04-22T13:20:51.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.182+0000 7f9d877fe700 1 --2- 172.21.15.134:0/2732196956 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f9d800a3c40 0x7f9d800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:20:51.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.182+0000 7f9d9659c700 1 --2- 172.21.15.134:0/2732196956 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f9d800a3c40 0x7f9d800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:20:51.180 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.183+0000 7f9d877fe700 1 -- 172.21.15.134:0/2732196956 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6723+0+0 (secure 0 0 0) 0x7f9d8c0d01a0 con 0x7f9d98102bc0 2024-04-22T13:20:51.181 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.183+0000 7f9d9659c700 1 --2- 172.21.15.134:0/2732196956 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f9d800a3c40 0x7f9d800a6100 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9d98102a20 tx=0x7f9d88009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:20:51.183 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.186+0000 7f9d877fe700 1 -- 172.21.15.134:0/2732196956 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9d8c025020 con 0x7f9d98102bc0 2024-04-22T13:20:51.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:50 smithi111 ceph-mon[31361]: Detected new or changed devices on smithi111 2024-04-22T13:20:51.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:50 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:51.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:50 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:51.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:50 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:20:51.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:50 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:20:51.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:50 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:20:51.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:50 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:20:51.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:50 smithi111 ceph-mon[31361]: Adjusting osd_memory_target on smithi111 to 4272M 2024-04-22T13:20:51.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:50 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:51.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:50 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:51.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:50 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:20:51.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:50 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:51.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:50 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:20:51.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:50 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:51.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:50 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:20:51.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:50 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:51.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:50 smithi111 ceph-mon[31361]: osdmap e76: 11 total, 11 up, 11 in 2024-04-22T13:20:51.284 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:50 smithi012 ceph-mon[28265]: Detected new or changed devices on smithi111 2024-04-22T13:20:51.284 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:50 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:51.284 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:50 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:51.284 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:50 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:20:51.284 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:50 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:20:51.284 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:50 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:20:51.284 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:50 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:20:51.284 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:50 smithi012 ceph-mon[28265]: Adjusting osd_memory_target on smithi111 to 4272M 2024-04-22T13:20:51.284 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:50 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:51.284 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:50 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:51.284 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:50 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:20:51.284 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:50 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:51.284 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:50 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:20:51.285 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:50 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:51.285 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:50 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:20:51.285 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:50 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:20:51.285 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:50 smithi012 ceph-mon[28265]: osdmap e76: 11 total, 11 up, 11 in 2024-04-22T13:20:51.392 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:20:51.395+0000 7f9d9ceda700 1 -- 172.21.15.134:0/2732196956 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f9d980ff170 con 0x7f9d800a3c40 2024-04-22T13:20:52.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:52 smithi012 ceph-mon[28265]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:20:52.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:52 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:20:52.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:52 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:20:52.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:52 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:52.417 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:52 smithi134 ceph-mon[31338]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:20:52.417 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:52 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:20:52.417 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:52 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:20:52.417 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:52 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:52.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:52 smithi111 ceph-mon[31361]: pgmap v238: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:20:52.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:52 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-22T13:20:52.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:52 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-22T13:20:52.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:52 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:20:53.326 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:53 smithi134 ceph-mon[31338]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:20:53.326 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:53 smithi134 ceph-mon[31338]: osdmap e77: 11 total, 11 up, 11 in 2024-04-22T13:20:53.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:53 smithi012 ceph-mon[28265]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:20:53.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:53 smithi012 ceph-mon[28265]: osdmap e77: 11 total, 11 up, 11 in 2024-04-22T13:20:53.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:53 smithi111 ceph-mon[31361]: from='client.24346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi134:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:20:53.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:53 smithi111 ceph-mon[31361]: osdmap e77: 11 total, 11 up, 11 in 2024-04-22T13:20:54.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:54 smithi012 ceph-mon[28265]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:20:54.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:54 smithi012 ceph-mon[28265]: osdmap e78: 11 total, 11 up, 11 in 2024-04-22T13:20:54.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:54 smithi134 ceph-mon[31338]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:20:54.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:54 smithi134 ceph-mon[31338]: osdmap e78: 11 total, 11 up, 11 in 2024-04-22T13:20:54.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:54 smithi111 ceph-mon[31361]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:20:54.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:54 smithi111 ceph-mon[31361]: osdmap e78: 11 total, 11 up, 11 in 2024-04-22T13:20:55.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:55 smithi134 ceph-mon[31338]: from='client.? 172.21.15.134:0/729623843' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b18fc5f3-1049-4871-b060-e99290273cfb"}]: dispatch 2024-04-22T13:20:55.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:55 smithi134 ceph-mon[31338]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b18fc5f3-1049-4871-b060-e99290273cfb"}]: dispatch 2024-04-22T13:20:55.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:55 smithi134 ceph-mon[31338]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b18fc5f3-1049-4871-b060-e99290273cfb"}]': finished 2024-04-22T13:20:55.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:55 smithi134 ceph-mon[31338]: osdmap e79: 12 total, 11 up, 12 in 2024-04-22T13:20:55.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:55 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:20:55.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:55 smithi134 ceph-mon[31338]: from='client.? 172.21.15.134:0/2533005603' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:20:55.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:55 smithi111 ceph-mon[31361]: from='client.? 172.21.15.134:0/729623843' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b18fc5f3-1049-4871-b060-e99290273cfb"}]: dispatch 2024-04-22T13:20:55.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:55 smithi111 ceph-mon[31361]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b18fc5f3-1049-4871-b060-e99290273cfb"}]: dispatch 2024-04-22T13:20:55.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:55 smithi111 ceph-mon[31361]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b18fc5f3-1049-4871-b060-e99290273cfb"}]': finished 2024-04-22T13:20:55.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:55 smithi111 ceph-mon[31361]: osdmap e79: 12 total, 11 up, 12 in 2024-04-22T13:20:55.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:55 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:20:55.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:55 smithi111 ceph-mon[31361]: from='client.? 172.21.15.134:0/2533005603' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:20:55.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:55 smithi012 ceph-mon[28265]: from='client.? 172.21.15.134:0/729623843' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b18fc5f3-1049-4871-b060-e99290273cfb"}]: dispatch 2024-04-22T13:20:55.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:55 smithi012 ceph-mon[28265]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b18fc5f3-1049-4871-b060-e99290273cfb"}]: dispatch 2024-04-22T13:20:55.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:55 smithi012 ceph-mon[28265]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b18fc5f3-1049-4871-b060-e99290273cfb"}]': finished 2024-04-22T13:20:55.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:55 smithi012 ceph-mon[28265]: osdmap e79: 12 total, 11 up, 12 in 2024-04-22T13:20:55.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:55 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:20:55.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:55 smithi012 ceph-mon[28265]: from='client.? 172.21.15.134:0/2533005603' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-22T13:20:56.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:56 smithi111 ceph-mon[31361]: pgmap v244: 1 pgs: 1 remapped+peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:20:56.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:56 smithi012 ceph-mon[28265]: pgmap v244: 1 pgs: 1 remapped+peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:20:56.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:56 smithi134 ceph-mon[31338]: pgmap v244: 1 pgs: 1 remapped+peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:20:58.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:20:58 smithi134 ceph-mon[31338]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-22T13:20:58.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:20:58 smithi111 ceph-mon[31361]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-22T13:20:58.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:20:58 smithi012 ceph-mon[28265]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-22T13:21:00.452 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:00 smithi134 ceph-mon[31338]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-22T13:21:00.475 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:00 smithi111 ceph-mon[31361]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-22T13:21:00.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:00 smithi012 ceph-mon[28265]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-22T13:21:02.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:02 smithi111 ceph-mon[31361]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-22T13:21:02.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:02 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-22T13:21:02.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:02 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:02.512 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:02 smithi134 ceph-mon[31338]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-22T13:21:02.512 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:02 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-22T13:21:02.512 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:02 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:02.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:02 smithi012 ceph-mon[28265]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-22T13:21:02.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:02 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-22T13:21:02.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:02 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:03.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:03 smithi111 ceph-mon[31361]: Deploying daemon osd.11 on smithi134 2024-04-22T13:21:03.514 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:03 smithi134 ceph-mon[31338]: Deploying daemon osd.11 on smithi134 2024-04-22T13:21:03.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:03 smithi012 ceph-mon[28265]: Deploying daemon osd.11 on smithi134 2024-04-22T13:21:04.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:04 smithi134 ceph-mon[31338]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-22T13:21:04.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:04 smithi111 ceph-mon[31361]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-22T13:21:04.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:04 smithi012 ceph-mon[28265]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-22T13:21:06.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:06 smithi134 ceph-mon[31338]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-04-22T13:21:06.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:06 smithi111 ceph-mon[31361]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-04-22T13:21:06.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:06 smithi012 ceph-mon[28265]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-04-22T13:21:08.351 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:08 smithi134 ceph-mon[31338]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-22T13:21:08.351 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:08 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:08.351 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:08 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:21:08.352 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:08 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:08.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:08 smithi111 ceph-mon[31361]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-22T13:21:08.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:08 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:08.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:08 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:21:08.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:08 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:08.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:08 smithi012 ceph-mon[28265]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-22T13:21:08.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:08 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:08.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:08 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:21:08.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:08 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:10.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:10 smithi111 ceph-mon[31361]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:21:10.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:10 smithi012 ceph-mon[28265]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:21:10.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:10 smithi134 ceph-mon[31338]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:21:11.216 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:11.218+0000 7f9d877fe700 1 -- 172.21.15.134:0/2732196956 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f9d980ff170 con 0x7f9d800a3c40 2024-04-22T13:21:11.217 INFO:teuthology.orchestra.run.smithi134.stdout:Created osd(s) 11 on host 'smithi134' 2024-04-22T13:21:11.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:11.221+0000 7f9d9ceda700 1 -- 172.21.15.134:0/2732196956 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f9d800a3c40 msgr2=0x7f9d800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:11.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:11.221+0000 7f9d9ceda700 1 --2- 172.21.15.134:0/2732196956 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f9d800a3c40 0x7f9d800a6100 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9d98102a20 tx=0x7f9d88009260 comp rx=0 tx=0).stop 2024-04-22T13:21:11.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:11.221+0000 7f9d9ceda700 1 -- 172.21.15.134:0/2732196956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d98102bc0 msgr2=0x7f9d98199070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:11.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:11.221+0000 7f9d9ceda700 1 --2- 172.21.15.134:0/2732196956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d98102bc0 0x7f9d98199070 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00c9a0 tx=0x7f9d8c00ccb0 comp rx=0 tx=0).stop 2024-04-22T13:21:11.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:11.221+0000 7f9d9ceda700 1 -- 172.21.15.134:0/2732196956 shutdown_connections 2024-04-22T13:21:11.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:11.221+0000 7f9d9ceda700 1 --2- 172.21.15.134:0/2732196956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d98103560 0x7f9d981a00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:11.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:11.221+0000 7f9d9ceda700 1 --2- 172.21.15.134:0/2732196956 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f9d800a3c40 0x7f9d800a6100 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:11.218 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:11.221+0000 7f9d9ceda700 1 --2- 172.21.15.134:0/2732196956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9d981019c0 0x7f9d98198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:11.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:11.221+0000 7f9d9ceda700 1 --2- 172.21.15.134:0/2732196956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9d98102bc0 0x7f9d98199070 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:11.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:11.221+0000 7f9d9ceda700 1 -- 172.21.15.134:0/2732196956 >> 172.21.15.134:0/2732196956 conn(0x7f9d980fcf60 msgr2=0x7f9d980fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:11.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:11.221+0000 7f9d9ceda700 1 -- 172.21.15.134:0/2732196956 shutdown_connections 2024-04-22T13:21:11.219 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:11.221+0000 7f9d9ceda700 1 -- 172.21.15.134:0/2732196956 wait complete. 2024-04-22T13:21:11.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:11 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:11.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:11 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:11.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:11 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:11.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:11 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:21:11.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:11 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:11.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:11 smithi134 ceph-mon[31338]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:21:11.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:11 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:11.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:11 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:11.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:11 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:21:11.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:11 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:11.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:11 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:11.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:11 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:11.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:11 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:21:11.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:11 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:11.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:11 smithi012 ceph-mon[28265]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:21:11.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:11 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:11.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:11 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:11.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:11 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:21:11.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:11 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:11.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:11 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:11.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:11 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:11.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:11 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:21:11.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:11 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:11.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:11 smithi111 ceph-mon[31361]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:21:11.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:11 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:11.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:11 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:11.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:11 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:21:11.913 DEBUG:teuthology.orchestra.run.smithi134:osd.11> sudo journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.11.service 2024-04-22T13:21:11.915 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-22T13:21:11.916 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-22T13:21:12.172 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:21:12.413 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:12 smithi134 ceph-mon[31338]: from='osd.11 [v2:172.21.15.134:6824/578640428,v1:172.21.15.134:6825/578640428]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-22T13:21:12.413 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:12 smithi134 ceph-mon[31338]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-22T13:21:12.534 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:12 smithi012 ceph-mon[28265]: from='osd.11 [v2:172.21.15.134:6824/578640428,v1:172.21.15.134:6825/578640428]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-22T13:21:12.537 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:12 smithi012 ceph-mon[28265]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-22T13:21:12.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:12 smithi111 ceph-mon[31361]: from='osd.11 [v2:172.21.15.134:6824/578640428,v1:172.21.15.134:6825/578640428]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-22T13:21:12.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:12 smithi111 ceph-mon[31361]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-22T13:21:13.357 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:13 smithi134 ceph-mon[31338]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-22T13:21:13.358 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:13 smithi134 ceph-mon[31338]: from='osd.11 [v2:172.21.15.134:6824/578640428,v1:172.21.15.134:6825/578640428]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-22T13:21:13.358 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:13 smithi134 ceph-mon[31338]: osdmap e80: 12 total, 11 up, 12 in 2024-04-22T13:21:13.358 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:13 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:21:13.358 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:13 smithi134 ceph-mon[31338]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-22T13:21:13.358 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:13 smithi134 ceph-mon[31338]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:21:13.555 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:13 smithi012 ceph-mon[28265]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-22T13:21:13.555 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:13 smithi012 ceph-mon[28265]: from='osd.11 [v2:172.21.15.134:6824/578640428,v1:172.21.15.134:6825/578640428]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-22T13:21:13.556 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:13 smithi012 ceph-mon[28265]: osdmap e80: 12 total, 11 up, 12 in 2024-04-22T13:21:13.556 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:13 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:21:13.556 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:13 smithi012 ceph-mon[28265]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-22T13:21:13.556 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:13 smithi012 ceph-mon[28265]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:21:13.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:13 smithi111 ceph-mon[31361]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-22T13:21:13.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:13 smithi111 ceph-mon[31361]: from='osd.11 [v2:172.21.15.134:6824/578640428,v1:172.21.15.134:6825/578640428]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-22T13:21:13.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:13 smithi111 ceph-mon[31361]: osdmap e80: 12 total, 11 up, 12 in 2024-04-22T13:21:13.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:13 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:21:13.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:13 smithi111 ceph-mon[31361]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]: dispatch 2024-04-22T13:21:13.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:13 smithi111 ceph-mon[31361]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:21:13.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.920+0000 7fb78886f700 1 -- 172.21.15.12:0/273311632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb784104660 msgr2=0x7fb78410af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:13.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.920+0000 7fb78886f700 1 --2- 172.21.15.12:0/273311632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb784104660 0x7fb78410af10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb77c00b0a0 tx=0x7fb77c00b3b0 comp rx=0 tx=0).stop 2024-04-22T13:21:13.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.920+0000 7fb78886f700 1 -- 172.21.15.12:0/273311632 shutdown_connections 2024-04-22T13:21:13.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.920+0000 7fb78886f700 1 --2- 172.21.15.12:0/273311632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb784104660 0x7fb78410af10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:13.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.920+0000 7fb78886f700 1 --2- 172.21.15.12:0/273311632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb784102ac0 0x7fb784102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:13.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.920+0000 7fb78886f700 1 --2- 172.21.15.12:0/273311632 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb784103cc0 0x7fb784104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:13.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.920+0000 7fb78886f700 1 -- 172.21.15.12:0/273311632 >> 172.21.15.12:0/273311632 conn(0x7fb7840fe060 msgr2=0x7fb7841004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:13.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.920+0000 7fb78886f700 1 -- 172.21.15.12:0/273311632 shutdown_connections 2024-04-22T13:21:13.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.920+0000 7fb78886f700 1 -- 172.21.15.12:0/273311632 wait complete. 2024-04-22T13:21:13.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.921+0000 7fb78886f700 1 Processor -- start 2024-04-22T13:21:13.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.921+0000 7fb78886f700 1 -- start start 2024-04-22T13:21:13.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.921+0000 7fb78886f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb784102ac0 0x7fb784198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:13.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.921+0000 7fb78886f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb784103cc0 0x7fb7841990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:13.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.921+0000 7fb78886f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb784104660 0x7fb7841a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:13.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.921+0000 7fb78886f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb78410e540 con 0x7fb784103cc0 2024-04-22T13:21:13.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.921+0000 7fb78886f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb78410e3c0 con 0x7fb784102ac0 2024-04-22T13:21:13.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.921+0000 7fb78886f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb78410e6c0 con 0x7fb784104660 2024-04-22T13:21:13.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.921+0000 7fb781d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb784103cc0 0x7fb7841990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:13.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.921+0000 7fb781d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb784103cc0 0x7fb7841990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58158/0 (socket says 172.21.15.12:58158) 2024-04-22T13:21:13.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.921+0000 7fb781d9b700 1 -- 172.21.15.12:0/2102351580 learned_addr learned my addr 172.21.15.12:0/2102351580 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:21:13.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.921+0000 7fb782d9d700 1 --2- 172.21.15.12:0/2102351580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb784104660 0x7fb7841a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:13.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.921+0000 7fb78259c700 1 --2- 172.21.15.12:0/2102351580 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb784102ac0 0x7fb784198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:13.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.922+0000 7fb781d9b700 1 -- 172.21.15.12:0/2102351580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb784104660 msgr2=0x7fb7841a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:13.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.922+0000 7fb781d9b700 1 --2- 172.21.15.12:0/2102351580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb784104660 0x7fb7841a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:13.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.922+0000 7fb781d9b700 1 -- 172.21.15.12:0/2102351580 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb784102ac0 msgr2=0x7fb784198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:13.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.922+0000 7fb781d9b700 1 --2- 172.21.15.12:0/2102351580 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb784102ac0 0x7fb784198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:13.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.922+0000 7fb781d9b700 1 -- 172.21.15.12:0/2102351580 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7841a07e0 con 0x7fb784103cc0 2024-04-22T13:21:13.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.922+0000 7fb781d9b700 1 --2- 172.21.15.12:0/2102351580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb784103cc0 0x7fb7841990c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb77800d980 tx=0x7fb77800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:13.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.922+0000 7fb7737fe700 1 -- 172.21.15.12:0/2102351580 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb778016070 con 0x7fb784103cc0 2024-04-22T13:21:13.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.922+0000 7fb7737fe700 1 -- 172.21.15.12:0/2102351580 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb778009d80 con 0x7fb784103cc0 2024-04-22T13:21:13.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.922+0000 7fb7737fe700 1 -- 172.21.15.12:0/2102351580 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77800f3c0 con 0x7fb784103cc0 2024-04-22T13:21:13.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.922+0000 7fb78886f700 1 -- 172.21.15.12:0/2102351580 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7841a0ad0 con 0x7fb784103cc0 2024-04-22T13:21:13.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.922+0000 7fb78886f700 1 -- 172.21.15.12:0/2102351580 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7841a0fc0 con 0x7fb784103cc0 2024-04-22T13:21:13.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.923+0000 7fb78886f700 1 -- 172.21.15.12:0/2102351580 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb784069730 con 0x7fb784103cc0 2024-04-22T13:21:13.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.924+0000 7fb7737fe700 1 -- 172.21.15.12:0/2102351580 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb778019070 con 0x7fb784103cc0 2024-04-22T13:21:13.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.925+0000 7fb7737fe700 1 --2- 172.21.15.12:0/2102351580 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb76c0a3c40 0x7fb76c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:13.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.925+0000 7fb7737fe700 1 -- 172.21.15.12:0/2102351580 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6847+0+0 (secure 0 0 0) 0x7fb7780cf8b0 con 0x7fb784103cc0 2024-04-22T13:21:13.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.925+0000 7fb78259c700 1 --2- 172.21.15.12:0/2102351580 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb76c0a3c40 0x7fb76c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:13.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.925+0000 7fb78259c700 1 --2- 172.21.15.12:0/2102351580 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb76c0a3c40 0x7fb76c0a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb774009680 tx=0x7fb774009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:13.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:13.929+0000 7fb7737fe700 1 -- 172.21.15.12:0/2102351580 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb77809c850 con 0x7fb784103cc0 2024-04-22T13:21:14.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:14.135+0000 7fb78886f700 1 -- 172.21.15.12:0/2102351580 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fb784063650 con 0x7fb784103cc0 2024-04-22T13:21:14.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:14.135+0000 7fb7737fe700 1 -- 172.21.15.12:0/2102351580 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7fb77809c150 con 0x7fb784103cc0 2024-04-22T13:21:14.132 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:21:14.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:14.137+0000 7fb78886f700 1 -- 172.21.15.12:0/2102351580 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb76c0a3c40 msgr2=0x7fb76c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:14.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:14.137+0000 7fb78886f700 1 --2- 172.21.15.12:0/2102351580 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb76c0a3c40 0x7fb76c0a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb774009680 tx=0x7fb774009260 comp rx=0 tx=0).stop 2024-04-22T13:21:14.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:14.137+0000 7fb78886f700 1 -- 172.21.15.12:0/2102351580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb784103cc0 msgr2=0x7fb7841990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:14.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:14.137+0000 7fb78886f700 1 --2- 172.21.15.12:0/2102351580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb784103cc0 0x7fb7841990c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb77800d980 tx=0x7fb77800dc90 comp rx=0 tx=0).stop 2024-04-22T13:21:14.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:14.137+0000 7fb78886f700 1 -- 172.21.15.12:0/2102351580 shutdown_connections 2024-04-22T13:21:14.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:14.137+0000 7fb78886f700 1 --2- 172.21.15.12:0/2102351580 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb784102ac0 0x7fb784198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:14.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:14.137+0000 7fb78886f700 1 --2- 172.21.15.12:0/2102351580 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb76c0a3c40 0x7fb76c0a6100 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:14.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:14.137+0000 7fb78886f700 1 --2- 172.21.15.12:0/2102351580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb784104660 0x7fb7841a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:14.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:14.137+0000 7fb78886f700 1 --2- 172.21.15.12:0/2102351580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb784103cc0 0x7fb7841990c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:14.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:14.137+0000 7fb78886f700 1 -- 172.21.15.12:0/2102351580 >> 172.21.15.12:0/2102351580 conn(0x7fb7840fe060 msgr2=0x7fb784106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:14.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:14.137+0000 7fb78886f700 1 -- 172.21.15.12:0/2102351580 shutdown_connections 2024-04-22T13:21:14.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:14.138+0000 7fb78886f700 1 -- 172.21.15.12:0/2102351580 wait complete. 2024-04-22T13:21:14.406 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:14 smithi012 ceph-mon[28265]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-22T13:21:14.406 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:14 smithi012 ceph-mon[28265]: osdmap e81: 12 total, 11 up, 12 in 2024-04-22T13:21:14.406 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:14 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:21:14.406 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:14 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:21:14.407 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:14 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2102351580' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-22T13:21:14.407 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:14 smithi012 ceph-mon[28265]: from='osd.11 ' entity='osd.11' 2024-04-22T13:21:14.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:14 smithi134 ceph-mon[31338]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-22T13:21:14.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:14 smithi134 ceph-mon[31338]: osdmap e81: 12 total, 11 up, 12 in 2024-04-22T13:21:14.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:14 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:21:14.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:14 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:21:14.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:14 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2102351580' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-22T13:21:14.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:14 smithi134 ceph-mon[31338]: from='osd.11 ' entity='osd.11' 2024-04-22T13:21:14.429 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:21:14 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:21:14.183+0000 7f4c914b0700 -1 osd.11 0 waiting for initial osdmap 2024-04-22T13:21:14.429 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:21:14 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:21:14.328+0000 7f4c8929b700 -1 osd.11 81 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T13:21:14.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:14 smithi111 ceph-mon[31361]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi134", "root=default"]}]': finished 2024-04-22T13:21:14.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:14 smithi111 ceph-mon[31361]: osdmap e81: 12 total, 11 up, 12 in 2024-04-22T13:21:14.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:14 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:21:14.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:14 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:21:14.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:14 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2102351580' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-22T13:21:14.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:14 smithi111 ceph-mon[31361]: from='osd.11 ' entity='osd.11' 2024-04-22T13:21:14.734 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":81,"num_osds":12,"num_up_osds":11,"osd_up_since":1713792048,"num_in_osds":12,"osd_in_since":1713792054,"num_remapped_pgs":0} 2024-04-22T13:21:15.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:15 smithi012 ceph-mon[28265]: purged_snaps scrub starts 2024-04-22T13:21:15.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:15 smithi012 ceph-mon[28265]: purged_snaps scrub ok 2024-04-22T13:21:15.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:15 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:21:15.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:15 smithi012 ceph-mon[28265]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:21:15.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:15 smithi012 ceph-mon[28265]: Detected new or changed devices on smithi134 2024-04-22T13:21:15.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:15 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:15.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:15 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:15.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:15 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:21:15.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:15 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:21:15.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:15 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:21:15.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:15 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:21:15.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:15 smithi012 ceph-mon[28265]: Adjusting osd_memory_target on smithi134 to 4272M 2024-04-22T13:21:15.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:15 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:15.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:15 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:15.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:15 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:21:15.590 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:15 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:15.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:15 smithi134 ceph-mon[31338]: purged_snaps scrub starts 2024-04-22T13:21:15.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:15 smithi134 ceph-mon[31338]: purged_snaps scrub ok 2024-04-22T13:21:15.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:15 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:21:15.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:15 smithi134 ceph-mon[31338]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:21:15.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:15 smithi134 ceph-mon[31338]: Detected new or changed devices on smithi134 2024-04-22T13:21:15.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:15 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:15.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:15 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:15.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:15 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:21:15.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:15 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:21:15.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:15 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:21:15.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:15 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:21:15.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:15 smithi134 ceph-mon[31338]: Adjusting osd_memory_target on smithi134 to 4272M 2024-04-22T13:21:15.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:15 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:15.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:15 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:15.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:15 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:21:15.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:15 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:15.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:15 smithi111 ceph-mon[31361]: purged_snaps scrub starts 2024-04-22T13:21:15.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:15 smithi111 ceph-mon[31361]: purged_snaps scrub ok 2024-04-22T13:21:15.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:15 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:21:15.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:15 smithi111 ceph-mon[31361]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-22T13:21:15.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:15 smithi111 ceph-mon[31361]: Detected new or changed devices on smithi134 2024-04-22T13:21:15.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:15 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:15.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:15 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:15.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:15 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:21:15.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:15 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:21:15.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:15 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:21:15.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:15 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:21:15.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:15 smithi111 ceph-mon[31361]: Adjusting osd_memory_target on smithi134 to 4272M 2024-04-22T13:21:15.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:15 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:15.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:15 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:15.729 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:15 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:21:15.729 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:15 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:15.735 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-22T13:21:15.983 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:21:16.434 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:16 smithi012 ceph-mon[28265]: osd.11 [v2:172.21.15.134:6824/578640428,v1:172.21.15.134:6825/578640428] boot 2024-04-22T13:21:16.434 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:16 smithi012 ceph-mon[28265]: osdmap e82: 12 total, 12 up, 12 in 2024-04-22T13:21:16.434 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:16 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:21:16.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:16 smithi134 ceph-mon[31338]: osd.11 [v2:172.21.15.134:6824/578640428,v1:172.21.15.134:6825/578640428] boot 2024-04-22T13:21:16.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:16 smithi134 ceph-mon[31338]: osdmap e82: 12 total, 12 up, 12 in 2024-04-22T13:21:16.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:16 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:21:16.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:16 smithi111 ceph-mon[31361]: osd.11 [v2:172.21.15.134:6824/578640428,v1:172.21.15.134:6825/578640428] boot 2024-04-22T13:21:16.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:16 smithi111 ceph-mon[31361]: osdmap e82: 12 total, 12 up, 12 in 2024-04-22T13:21:16.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:16 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:21:17.493 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:17 smithi012 ceph-mon[28265]: osdmap e83: 12 total, 12 up, 12 in 2024-04-22T13:21:17.494 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:17 smithi012 ceph-mon[28265]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:17.579 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:17 smithi134 ceph-mon[31338]: osdmap e83: 12 total, 12 up, 12 in 2024-04-22T13:21:17.580 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:17 smithi134 ceph-mon[31338]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:17.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:17 smithi111 ceph-mon[31361]: osdmap e83: 12 total, 12 up, 12 in 2024-04-22T13:21:17.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:17 smithi111 ceph-mon[31361]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:17.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.870+0000 7fcfa951b700 1 -- 172.21.15.12:0/2133419076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfa4103560 msgr2=0x7fcfa4109e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:17.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.870+0000 7fcfa951b700 1 --2- 172.21.15.12:0/2133419076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfa4103560 0x7fcfa4109e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00b0d0 tx=0x7fcf9c00b3e0 comp rx=0 tx=0).stop 2024-04-22T13:21:17.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.870+0000 7fcfa951b700 1 -- 172.21.15.12:0/2133419076 shutdown_connections 2024-04-22T13:21:17.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.870+0000 7fcfa951b700 1 --2- 172.21.15.12:0/2133419076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfa41019c0 0x7fcfa4101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:17.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.870+0000 7fcfa951b700 1 --2- 172.21.15.12:0/2133419076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfa4103560 0x7fcfa4109e10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:17.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.870+0000 7fcfa951b700 1 --2- 172.21.15.12:0/2133419076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfa4102bc0 0x7fcfa4103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:17.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.870+0000 7fcfa951b700 1 -- 172.21.15.12:0/2133419076 >> 172.21.15.12:0/2133419076 conn(0x7fcfa40fcf60 msgr2=0x7fcfa40ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:17.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.871+0000 7fcfa951b700 1 -- 172.21.15.12:0/2133419076 shutdown_connections 2024-04-22T13:21:17.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.871+0000 7fcfa951b700 1 -- 172.21.15.12:0/2133419076 wait complete. 2024-04-22T13:21:17.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.871+0000 7fcfa951b700 1 Processor -- start 2024-04-22T13:21:17.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.872+0000 7fcfa951b700 1 -- start start 2024-04-22T13:21:17.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.872+0000 7fcfa951b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfa41019c0 0x7fcfa419ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:17.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.872+0000 7fcfa951b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfa4102bc0 0x7fcfa419b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:17.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.872+0000 7fcfa951b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfa4103560 0x7fcfa41a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:17.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.872+0000 7fcfa951b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa410d490 con 0x7fcfa4102bc0 2024-04-22T13:21:17.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.872+0000 7fcfa951b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa410d310 con 0x7fcfa41019c0 2024-04-22T13:21:17.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.872+0000 7fcfa951b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa410d610 con 0x7fcfa4103560 2024-04-22T13:21:17.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.872+0000 7fcfa27fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfa4102bc0 0x7fcfa419b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:17.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.872+0000 7fcfa27fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfa4102bc0 0x7fcfa419b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58172/0 (socket says 172.21.15.12:58172) 2024-04-22T13:21:17.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.872+0000 7fcfa27fc700 1 -- 172.21.15.12:0/2070024447 learned_addr learned my addr 172.21.15.12:0/2070024447 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:21:17.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.872+0000 7fcfa37fe700 1 --2- 172.21.15.12:0/2070024447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfa4103560 0x7fcfa41a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:17.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.872+0000 7fcfa2ffd700 1 --2- 172.21.15.12:0/2070024447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfa41019c0 0x7fcfa419ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:17.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.872+0000 7fcfa27fc700 1 -- 172.21.15.12:0/2070024447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfa4103560 msgr2=0x7fcfa41a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:17.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.872+0000 7fcfa27fc700 1 --2- 172.21.15.12:0/2070024447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfa4103560 0x7fcfa41a22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:17.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.872+0000 7fcfa27fc700 1 -- 172.21.15.12:0/2070024447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfa41019c0 msgr2=0x7fcfa419ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:17.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.872+0000 7fcfa27fc700 1 --2- 172.21.15.12:0/2070024447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfa41019c0 0x7fcfa419ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:17.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.872+0000 7fcfa27fc700 1 -- 172.21.15.12:0/2070024447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa41a2970 con 0x7fcfa4102bc0 2024-04-22T13:21:17.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.872+0000 7fcfa2ffd700 1 --2- 172.21.15.12:0/2070024447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfa41019c0 0x7fcfa419ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:21:17.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.872+0000 7fcfa37fe700 1 --2- 172.21.15.12:0/2070024447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfa4103560 0x7fcfa41a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:21:17.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.873+0000 7fcfa27fc700 1 --2- 172.21.15.12:0/2070024447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfa4102bc0 0x7fcfa419b260 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcf9800d950 tx=0x7fcf9800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:17.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.873+0000 7fcf8bfff700 1 -- 172.21.15.12:0/2070024447 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf98016070 con 0x7fcfa4102bc0 2024-04-22T13:21:17.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.873+0000 7fcfa951b700 1 -- 172.21.15.12:0/2070024447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfa41a2c60 con 0x7fcfa4102bc0 2024-04-22T13:21:17.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.873+0000 7fcf8bfff700 1 -- 172.21.15.12:0/2070024447 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf98010460 con 0x7fcfa4102bc0 2024-04-22T13:21:17.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.873+0000 7fcf8bfff700 1 -- 172.21.15.12:0/2070024447 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9800f620 con 0x7fcfa4102bc0 2024-04-22T13:21:17.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.873+0000 7fcfa951b700 1 -- 172.21.15.12:0/2070024447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfa41a31a0 con 0x7fcfa4102bc0 2024-04-22T13:21:17.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.874+0000 7fcfa951b700 1 -- 172.21.15.12:0/2070024447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfa4069730 con 0x7fcfa4102bc0 2024-04-22T13:21:17.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.876+0000 7fcf8bfff700 1 -- 172.21.15.12:0/2070024447 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcf98019070 con 0x7fcfa4102bc0 2024-04-22T13:21:17.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.876+0000 7fcf8bfff700 1 --2- 172.21.15.12:0/2070024447 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fcf8c0a3c40 0x7fcf8c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:17.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.876+0000 7fcfa2ffd700 1 --2- 172.21.15.12:0/2070024447 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fcf8c0a3c40 0x7fcf8c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:17.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.876+0000 7fcf8bfff700 1 -- 172.21.15.12:0/2070024447 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fcf980cff90 con 0x7fcfa4102bc0 2024-04-22T13:21:17.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.877+0000 7fcfa2ffd700 1 --2- 172.21.15.12:0/2070024447 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fcf8c0a3c40 0x7fcf8c0a6100 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcfa419c200 tx=0x7fcf9400a2b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:17.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:17.880+0000 7fcf8bfff700 1 -- 172.21.15.12:0/2070024447 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcf9809ce10 con 0x7fcfa4102bc0 2024-04-22T13:21:18.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:18.086+0000 7fcfa951b700 1 -- 172.21.15.12:0/2070024447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fcfa41a3430 con 0x7fcfa4102bc0 2024-04-22T13:21:18.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:18.087+0000 7fcf8bfff700 1 -- 172.21.15.12:0/2070024447 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v83) ==== 74+0+133 (secure 0 0 0) 0x7fcf9809c710 con 0x7fcfa4102bc0 2024-04-22T13:21:18.084 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:21:18.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:18.089+0000 7fcfa951b700 1 -- 172.21.15.12:0/2070024447 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fcf8c0a3c40 msgr2=0x7fcf8c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:18.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:18.089+0000 7fcfa951b700 1 --2- 172.21.15.12:0/2070024447 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fcf8c0a3c40 0x7fcf8c0a6100 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcfa419c200 tx=0x7fcf9400a2b0 comp rx=0 tx=0).stop 2024-04-22T13:21:18.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:18.089+0000 7fcfa951b700 1 -- 172.21.15.12:0/2070024447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfa4102bc0 msgr2=0x7fcfa419b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:18.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:18.089+0000 7fcfa951b700 1 --2- 172.21.15.12:0/2070024447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfa4102bc0 0x7fcfa419b260 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcf9800d950 tx=0x7fcf9800dd10 comp rx=0 tx=0).stop 2024-04-22T13:21:18.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:18.089+0000 7fcfa951b700 1 -- 172.21.15.12:0/2070024447 shutdown_connections 2024-04-22T13:21:18.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:18.089+0000 7fcfa951b700 1 --2- 172.21.15.12:0/2070024447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcfa41019c0 0x7fcfa419ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:18.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:18.089+0000 7fcfa951b700 1 --2- 172.21.15.12:0/2070024447 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fcf8c0a3c40 0x7fcf8c0a6100 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:18.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:18.089+0000 7fcfa951b700 1 --2- 172.21.15.12:0/2070024447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcfa4103560 0x7fcfa41a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:18.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:18.089+0000 7fcfa951b700 1 --2- 172.21.15.12:0/2070024447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcfa4102bc0 0x7fcfa419b260 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:18.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:18.089+0000 7fcfa951b700 1 -- 172.21.15.12:0/2070024447 >> 172.21.15.12:0/2070024447 conn(0x7fcfa40fcf60 msgr2=0x7fcfa40fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:18.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:18.090+0000 7fcfa951b700 1 -- 172.21.15.12:0/2070024447 shutdown_connections 2024-04-22T13:21:18.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:18.090+0000 7fcfa951b700 1 -- 172.21.15.12:0/2070024447 wait complete. 2024-04-22T13:21:18.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:18 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2070024447' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-22T13:21:18.592 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:18 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2070024447' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-22T13:21:18.629 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:18 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2070024447' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-22T13:21:18.714 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":83,"num_osds":12,"num_up_osds":12,"osd_up_since":1713792075,"num_in_osds":12,"osd_in_since":1713792054,"num_remapped_pgs":0} 2024-04-22T13:21:18.715 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-22T13:21:18.965 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:21:19.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:19 smithi012 ceph-mon[28265]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:19.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:19 smithi134 ceph-mon[31338]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:19.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:19 smithi111 ceph-mon[31361]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:20.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.610+0000 7f3f34c0f700 1 -- 172.21.15.12:0/2067549982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f30108b50 msgr2=0x7f3f30108f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:20.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.610+0000 7f3f34c0f700 1 --2- 172.21.15.12:0/2067549982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f30108b50 0x7f3f30108f30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3f200099c0 tx=0x7f3f20009cd0 comp rx=0 tx=0).stop 2024-04-22T13:21:20.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.610+0000 7f3f34c0f700 1 -- 172.21.15.12:0/2067549982 shutdown_connections 2024-04-22T13:21:20.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.610+0000 7f3f34c0f700 1 --2- 172.21.15.12:0/2067549982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f3006bb40 0x7f3f3010f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:20.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.610+0000 7f3f34c0f700 1 --2- 172.21.15.12:0/2067549982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f30108b50 0x7f3f30108f30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.610+0000 7f3f34c0f700 1 --2- 172.21.15.12:0/2067549982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f3006b180 0x7f3f3006b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.610+0000 7f3f34c0f700 1 -- 172.21.15.12:0/2067549982 >> 172.21.15.12:0/2067549982 conn(0x7f3f30078740 msgr2=0x7f3f30078b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.610+0000 7f3f34c0f700 1 -- 172.21.15.12:0/2067549982 shutdown_connections 2024-04-22T13:21:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.610+0000 7f3f34c0f700 1 -- 172.21.15.12:0/2067549982 wait complete. 2024-04-22T13:21:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.611+0000 7f3f34c0f700 1 Processor -- start 2024-04-22T13:21:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.611+0000 7f3f34c0f700 1 -- start start 2024-04-22T13:21:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.611+0000 7f3f34c0f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f3006b180 0x7f3f30199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.611+0000 7f3f34c0f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f3006bb40 0x7f3f30199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.611+0000 7f3f34c0f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f30108b50 0x7f3f3019db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.611+0000 7f3f34c0f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3f30112d40 con 0x7f3f30108b50 2024-04-22T13:21:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.611+0000 7f3f34c0f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3f30112bc0 con 0x7f3f3006bb40 2024-04-22T13:21:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.611+0000 7f3f34c0f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3f30112ec0 con 0x7f3f3006b180 2024-04-22T13:21:20.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.611+0000 7f3f2ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f30108b50 0x7f3f3019db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.611+0000 7f3f2ed9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f30108b50 0x7f3f3019db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58200/0 (socket says 172.21.15.12:58200) 2024-04-22T13:21:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.611+0000 7f3f2ed9d700 1 -- 172.21.15.12:0/2177994870 learned_addr learned my addr 172.21.15.12:0/2177994870 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:21:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.611+0000 7f3f2e59c700 1 --2- 172.21.15.12:0/2177994870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f3006b180 0x7f3f30199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.611+0000 7f3f2dd9b700 1 --2- 172.21.15.12:0/2177994870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f3006bb40 0x7f3f30199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.611+0000 7f3f2ed9d700 1 -- 172.21.15.12:0/2177994870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f3006b180 msgr2=0x7f3f30199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.611+0000 7f3f2ed9d700 1 --2- 172.21.15.12:0/2177994870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f3006b180 0x7f3f30199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.611+0000 7f3f2ed9d700 1 -- 172.21.15.12:0/2177994870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f3006bb40 msgr2=0x7f3f30199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.611+0000 7f3f2ed9d700 1 --2- 172.21.15.12:0/2177994870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f3006bb40 0x7f3f30199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.611+0000 7f3f2ed9d700 1 -- 172.21.15.12:0/2177994870 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3019e260 con 0x7f3f30108b50 2024-04-22T13:21:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.612+0000 7f3f2e59c700 1 --2- 172.21.15.12:0/2177994870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f3006b180 0x7f3f30199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:21:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.612+0000 7f3f2dd9b700 1 --2- 172.21.15.12:0/2177994870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f3006bb40 0x7f3f30199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:21:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.612+0000 7f3f2ed9d700 1 --2- 172.21.15.12:0/2177994870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f30108b50 0x7f3f3019db20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800b9e0 tx=0x7f3f2800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.612+0000 7f3f1f7fe700 1 -- 172.21.15.12:0/2177994870 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f28019070 con 0x7f3f30108b50 2024-04-22T13:21:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.612+0000 7f3f1f7fe700 1 -- 172.21.15.12:0/2177994870 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f2800ce00 con 0x7f3f30108b50 2024-04-22T13:21:20.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.612+0000 7f3f1f7fe700 1 -- 172.21.15.12:0/2177994870 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f28007370 con 0x7f3f30108b50 2024-04-22T13:21:20.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.612+0000 7f3f34c0f700 1 -- 172.21.15.12:0/2177994870 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f3019e550 con 0x7f3f30108b50 2024-04-22T13:21:20.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.612+0000 7f3f34c0f700 1 -- 172.21.15.12:0/2177994870 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f301a5dd0 con 0x7f3f30108b50 2024-04-22T13:21:20.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.613+0000 7f3f34c0f700 1 -- 172.21.15.12:0/2177994870 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f30069730 con 0x7f3f30108b50 2024-04-22T13:21:20.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.614+0000 7f3f1f7fe700 1 -- 172.21.15.12:0/2177994870 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3f2801c070 con 0x7f3f30108b50 2024-04-22T13:21:20.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.615+0000 7f3f1f7fe700 1 --2- 172.21.15.12:0/2177994870 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f3f180a3c40 0x7f3f180a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:20.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.615+0000 7f3f2e59c700 1 --2- 172.21.15.12:0/2177994870 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f3f180a3c40 0x7f3f180a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:20.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.615+0000 7f3f1f7fe700 1 -- 172.21.15.12:0/2177994870 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f3f280cf780 con 0x7f3f30108b50 2024-04-22T13:21:20.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.615+0000 7f3f2e59c700 1 --2- 172.21.15.12:0/2177994870 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f3f180a3c40 0x7f3f180a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3f20000c00 tx=0x7f3f20005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:20.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.619+0000 7f3f1f7fe700 1 -- 172.21.15.12:0/2177994870 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3f2809c600 con 0x7f3f30108b50 2024-04-22T13:21:20.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.829+0000 7f3f34c0f700 1 -- 172.21.15.12:0/2177994870 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3f3019a5b0 con 0x7f3f30108b50 2024-04-22T13:21:20.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.830+0000 7f3f1f7fe700 1 -- 172.21.15.12:0/2177994870 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+18470 (secure 0 0 0) 0x7f3f2809bf00 con 0x7f3f30108b50 2024-04-22T13:21:20.827 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:21:20.827 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":83,"fsid":"0aede5ea-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:14:09.362918+0000","modified":"2024-04-22T13:21:16.272831+0000","last_up_change":"2024-04-22T13:21:15.263393+0000","last_in_change":"2024-04-22T13:20:54.174685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:17:32.372434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d9e2365-3a64-4b13-96fc-c217b1cee2ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6803","nonce":1191350632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6805","nonce":1191350632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6809","nonce":1191350632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6807","nonce":1191350632}]},"public_addr":"172.21.15.12:6803/1191350632","cluster_addr":"172.21.15.12:6805/1191350632","heartbeat_back_addr":"172.21.15.12:6809/1191350632","heartbeat_front_addr":"172.21.15.12:6807/1191350632","state":["exists","up"]},{"osd":1,"uuid":"1499c783-2bd7-48ce-a444-34add42bcf31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6801","nonce":417496465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6803","nonce":417496465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6807","nonce":417496465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6805","nonce":417496465}]},"public_addr":"172.21.15.111:6801/417496465","cluster_addr":"172.21.15.111:6803/417496465","heartbeat_back_addr":"172.21.15.111:6807/417496465","heartbeat_front_addr":"172.21.15.111:6805/417496465","state":["exists","up"]},{"osd":2,"uuid":"71a9ebc3-c622-434e-a6c2-f48358f384f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6801","nonce":1403153998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6803","nonce":1403153998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6807","nonce":1403153998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6805","nonce":1403153998}]},"public_addr":"172.21.15.134:6801/1403153998","cluster_addr":"172.21.15.134:6803/1403153998","heartbeat_back_addr":"172.21.15.134:6807/1403153998","heartbeat_front_addr":"172.21.15.134:6805/1403153998","state":["exists","up"]},{"osd":3,"uuid":"6dd00db2-a286-48ba-8666-56ffdd5f35ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6811","nonce":3359066681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6813","nonce":3359066681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6817","nonce":3359066681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6815","nonce":3359066681}]},"public_addr":"172.21.15.12:6811/3359066681","cluster_addr":"172.21.15.12:6813/3359066681","heartbeat_back_addr":"172.21.15.12:6817/3359066681","heartbeat_front_addr":"172.21.15.12:6815/3359066681","state":["exists","up"]},{"osd":4,"uuid":"1e29afd7-354c-4506-aaa0-b43eb1a19a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6809","nonce":2886349872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6811","nonce":2886349872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6815","nonce":2886349872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6813","nonce":2886349872}]},"public_addr":"172.21.15.111:6809/2886349872","cluster_addr":"172.21.15.111:6811/2886349872","heartbeat_back_addr":"172.21.15.111:6815/2886349872","heartbeat_front_addr":"172.21.15.111:6813/2886349872","state":["exists","up"]},{"osd":5,"uuid":"8f40f02d-0ba2-4842-a03d-2a1a04d1010d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6809","nonce":3071977602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6811","nonce":3071977602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6815","nonce":3071977602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6813","nonce":3071977602}]},"public_addr":"172.21.15.134:6809/3071977602","cluster_addr":"172.21.15.134:6811/3071977602","heartbeat_back_addr":"172.21.15.134:6815/3071977602","heartbeat_front_addr":"172.21.15.134:6813/3071977602","state":["exists","up"]},{"osd":6,"uuid":"bb0c17a9-6a8e-4be6-8660-f73cfa8200ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6819","nonce":3819619299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6821","nonce":3819619299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6825","nonce":3819619299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6823","nonce":3819619299}]},"public_addr":"172.21.15.12:6819/3819619299","cluster_addr":"172.21.15.12:6821/3819619299","heartbeat_back_addr":"172.21.15.12:6825/3819619299","heartbeat_front_addr":"172.21.15.12:6823/3819619299","state":["exists","up"]},{"osd":7,"uuid":"0a93d6af-569e-4e45-b697-9e978c1c1004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6817","nonce":3113856282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6819","nonce":3113856282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6823","nonce":3113856282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6821","nonce":3113856282}]},"public_addr":"172.21.15.111:6817/3113856282","cluster_addr":"172.21.15.111:6819/3113856282","heartbeat_back_addr":"172.21.15.111:6823/3113856282","heartbeat_front_addr":"172.21.15.111:6821/3113856282","state":["exists","up"]},{"osd":8,"uuid":"e21acc68-cf50-4f4f-b1ba-cf6c195b4955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6817","nonce":1689362191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6819","nonce":1689362191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6823","nonce":1689362191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6821","nonce":1689362191}]},"public_addr":"172.21.15.134:6817/1689362191","cluster_addr":"172.21.15.134:6819/1689362191","heartbeat_back_addr":"172.21.15.134:6823/1689362191","heartbeat_front_addr":"172.21.15.134:6821/1689362191","state":["exists","up"]},{"osd":9,"uuid":"fd745869-7f5f-48a8-b412-bb9c92b586e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6827","nonce":4292385779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6829","nonce":4292385779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6833","nonce":4292385779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6831","nonce":4292385779}]},"public_addr":"172.21.15.12:6827/4292385779","cluster_addr":"172.21.15.12:6829/4292385779","heartbeat_back_addr":"172.21.15.12:6833/4292385779","heartbeat_front_addr":"172.21.15.12:6831/4292385779","state":["exists","up"]},{"osd":10,"uuid":"798b1abd-f6c7-4a5f-8da0-fdae49e50d2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6825","nonce":3338566976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6827","nonce":3338566976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6831","nonce":3338566976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6829","nonce":3338566976}]},"public_addr":"172.21.15.111:6825/3338566976","cluster_addr":"172.21.15.111:6827/3338566976","heartbeat_back_addr":"172.21.15.111:6831/3338566976","heartbeat_front_addr":"172.21.15.111:6829/3338566976","state":["exists","up"]},{"osd":11,"uuid":"b18fc5f3-1049-4871-b060-e99290273cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6825","nonce":578640428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6827","nonce":578640428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6831","nonce":578640428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6829","nonce":578640428}]},"public_addr":"172.21.15.134:6825/578640428","cluster_addr":"172.21.15.134:6827/578640428","heartbeat_back_addr":"172.21.15.134:6831/578640428","heartbeat_front_addr":"172.21.15.134:6829/578640428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:16:40.828608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:05.052528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:28.297316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:53.496497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:17.947259+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:41.712894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:07.105945+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:32.129688+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:56.008516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:22.239867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:47.334692+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:12.301276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/2593601362":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/2519382069":"2024-04-23T13:15:04.279990+0000","172.21.15.12:6801/2009831940":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3509412187":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1007749062":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3588480095":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1904962176":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2009831940":"2024-04-23T13:15:04.279990+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:21:20.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.832+0000 7f3f34c0f700 1 -- 172.21.15.12:0/2177994870 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f3f180a3c40 msgr2=0x7f3f180a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:20.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.832+0000 7f3f34c0f700 1 --2- 172.21.15.12:0/2177994870 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f3f180a3c40 0x7f3f180a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3f20000c00 tx=0x7f3f20005fb0 comp rx=0 tx=0).stop 2024-04-22T13:21:20.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.832+0000 7f3f34c0f700 1 -- 172.21.15.12:0/2177994870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f30108b50 msgr2=0x7f3f3019db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:20.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.832+0000 7f3f34c0f700 1 --2- 172.21.15.12:0/2177994870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f30108b50 0x7f3f3019db20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800b9e0 tx=0x7f3f2800bda0 comp rx=0 tx=0).stop 2024-04-22T13:21:20.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.832+0000 7f3f34c0f700 1 -- 172.21.15.12:0/2177994870 shutdown_connections 2024-04-22T13:21:20.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.832+0000 7f3f34c0f700 1 --2- 172.21.15.12:0/2177994870 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f3f180a3c40 0x7f3f180a6100 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:20.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.832+0000 7f3f34c0f700 1 --2- 172.21.15.12:0/2177994870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f30108b50 0x7f3f3019db20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:20.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.832+0000 7f3f34c0f700 1 --2- 172.21.15.12:0/2177994870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f3006b180 0x7f3f30199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:20.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.832+0000 7f3f34c0f700 1 --2- 172.21.15.12:0/2177994870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f3006bb40 0x7f3f30199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:20.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.832+0000 7f3f34c0f700 1 -- 172.21.15.12:0/2177994870 >> 172.21.15.12:0/2177994870 conn(0x7f3f30078740 msgr2=0x7f3f301009e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:20.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.832+0000 7f3f34c0f700 1 -- 172.21.15.12:0/2177994870 shutdown_connections 2024-04-22T13:21:20.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:20.832+0000 7f3f34c0f700 1 -- 172.21.15.12:0/2177994870 wait complete. 2024-04-22T13:21:21.537 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T13:17:32.372434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T13:21:21.537 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-22T13:21:21.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:21 smithi134 ceph-mon[31338]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:21.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:21 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2177994870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:21:21.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:21 smithi111 ceph-mon[31361]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:21.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:21 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2177994870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:21:21.787 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:21:21.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:21 smithi012 ceph-mon[28265]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:21.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:21 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2177994870' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:21:23.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.494+0000 7fb5bb1cc700 1 -- 172.21.15.12:0/2019564543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5b4102a90 msgr2=0x7fb5b4102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:23.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.494+0000 7fb5bb1cc700 1 --2- 172.21.15.12:0/2019564543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5b4102a90 0x7fb5b4102f10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8009990 tx=0x7fb5a8009ca0 comp rx=0 tx=0).stop 2024-04-22T13:21:23.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.494+0000 7fb5bb1cc700 1 -- 172.21.15.12:0/2019564543 shutdown_connections 2024-04-22T13:21:23.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.494+0000 7fb5bb1cc700 1 --2- 172.21.15.12:0/2019564543 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5b4103450 0x7fb5b410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:23.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.495+0000 7fb5bb1cc700 1 --2- 172.21.15.12:0/2019564543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5b4108a50 0x7fb5b4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:23.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.495+0000 7fb5bb1cc700 1 --2- 172.21.15.12:0/2019564543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5b4102a90 0x7fb5b4102f10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:23.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.495+0000 7fb5bb1cc700 1 -- 172.21.15.12:0/2019564543 >> 172.21.15.12:0/2019564543 conn(0x7fb5b4079ba0 msgr2=0x7fb5b4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:23.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.495+0000 7fb5bb1cc700 1 -- 172.21.15.12:0/2019564543 shutdown_connections 2024-04-22T13:21:23.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.495+0000 7fb5bb1cc700 1 -- 172.21.15.12:0/2019564543 wait complete. 2024-04-22T13:21:23.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.495+0000 7fb5bb1cc700 1 Processor -- start 2024-04-22T13:21:23.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.495+0000 7fb5bb1cc700 1 -- start start 2024-04-22T13:21:23.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.495+0000 7fb5bb1cc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5b4102a90 0x7fb5b4199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:23.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.495+0000 7fb5bb1cc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5b4103450 0x7fb5b41996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:23.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.495+0000 7fb5bb1cc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5b4108a50 0x7fb5b419da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:23.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.495+0000 7fb5bb1cc700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b4112c60 con 0x7fb5b4108a50 2024-04-22T13:21:23.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.495+0000 7fb5bb1cc700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b4112ae0 con 0x7fb5b4102a90 2024-04-22T13:21:23.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.495+0000 7fb5bb1cc700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b4112de0 con 0x7fb5b4103450 2024-04-22T13:21:23.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.495+0000 7fb5b9769700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5b4108a50 0x7fb5b419da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:23.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.496+0000 7fb5b9769700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5b4108a50 0x7fb5b419da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39788/0 (socket says 172.21.15.12:39788) 2024-04-22T13:21:23.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.496+0000 7fb5b9769700 1 -- 172.21.15.12:0/4060759971 learned_addr learned my addr 172.21.15.12:0/4060759971 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:21:23.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.496+0000 7fb5b9769700 1 -- 172.21.15.12:0/4060759971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5b4103450 msgr2=0x7fb5b41996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:23.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.496+0000 7fb5b3fff700 1 --2- 172.21.15.12:0/4060759971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5b4103450 0x7fb5b41996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:23.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.496+0000 7fb5b8f68700 1 --2- 172.21.15.12:0/4060759971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5b4102a90 0x7fb5b4199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:23.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.496+0000 7fb5b9769700 1 --2- 172.21.15.12:0/4060759971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5b4103450 0x7fb5b41996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:23.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.496+0000 7fb5b9769700 1 -- 172.21.15.12:0/4060759971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5b4102a90 msgr2=0x7fb5b4199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:23.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.496+0000 7fb5b9769700 1 --2- 172.21.15.12:0/4060759971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5b4102a90 0x7fb5b4199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:23.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.496+0000 7fb5b9769700 1 -- 172.21.15.12:0/4060759971 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b419e1a0 con 0x7fb5b4108a50 2024-04-22T13:21:23.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.496+0000 7fb5b3fff700 1 --2- 172.21.15.12:0/4060759971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5b4103450 0x7fb5b41996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:21:23.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.496+0000 7fb5b8f68700 1 --2- 172.21.15.12:0/4060759971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5b4102a90 0x7fb5b4199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:21:23.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.496+0000 7fb5b9769700 1 --2- 172.21.15.12:0/4060759971 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5b4108a50 0x7fb5b419da60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac007ee0 tx=0x7fb5ac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:23.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.496+0000 7fb5b1ffb700 1 -- 172.21.15.12:0/4060759971 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac00dbd0 con 0x7fb5b4108a50 2024-04-22T13:21:23.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.496+0000 7fb5b1ffb700 1 -- 172.21.15.12:0/4060759971 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5ac00dd70 con 0x7fb5b4108a50 2024-04-22T13:21:23.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.496+0000 7fb5bb1cc700 1 -- 172.21.15.12:0/4060759971 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5b419e490 con 0x7fb5b4108a50 2024-04-22T13:21:23.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.496+0000 7fb5b1ffb700 1 -- 172.21.15.12:0/4060759971 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac005990 con 0x7fb5b4108a50 2024-04-22T13:21:23.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.497+0000 7fb5bb1cc700 1 -- 172.21.15.12:0/4060759971 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5b41a5d10 con 0x7fb5b4108a50 2024-04-22T13:21:23.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.497+0000 7fb5bb1cc700 1 -- 172.21.15.12:0/4060759971 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5b4069730 con 0x7fb5b4108a50 2024-04-22T13:21:23.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.498+0000 7fb5b1ffb700 1 -- 172.21.15.12:0/4060759971 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb5ac01a020 con 0x7fb5b4108a50 2024-04-22T13:21:23.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.499+0000 7fb5b1ffb700 1 --2- 172.21.15.12:0/4060759971 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb59c0a3c40 0x7fb59c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:23.497 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:23 smithi012 ceph-mon[28265]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:23.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.499+0000 7fb5b1ffb700 1 -- 172.21.15.12:0/4060759971 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fb5ac0cf7f0 con 0x7fb5b4108a50 2024-04-22T13:21:23.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.499+0000 7fb5b8f68700 1 --2- 172.21.15.12:0/4060759971 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb59c0a3c40 0x7fb59c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:23.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.500+0000 7fb5b8f68700 1 --2- 172.21.15.12:0/4060759971 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb59c0a3c40 0x7fb59c0a6100 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4007900 tx=0x7fb5a4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:23.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.504+0000 7fb5b1ffb700 1 -- 172.21.15.12:0/4060759971 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb5ac010070 con 0x7fb5b4108a50 2024-04-22T13:21:23.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:23 smithi134 ceph-mon[31338]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:23.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.709+0000 7fb5bb1cc700 1 -- 172.21.15.12:0/4060759971 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb5b402df80 con 0x7fb5b4108a50 2024-04-22T13:21:23.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.709+0000 7fb5b1ffb700 1 -- 172.21.15.12:0/4060759971 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7fb5ac09c670 con 0x7fb5b4108a50 2024-04-22T13:21:23.706 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-22T13:21:23.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.711+0000 7fb5bb1cc700 1 -- 172.21.15.12:0/4060759971 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb59c0a3c40 msgr2=0x7fb59c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:23.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.712+0000 7fb5bb1cc700 1 --2- 172.21.15.12:0/4060759971 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb59c0a3c40 0x7fb59c0a6100 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4007900 tx=0x7fb5a4008040 comp rx=0 tx=0).stop 2024-04-22T13:21:23.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.712+0000 7fb5bb1cc700 1 -- 172.21.15.12:0/4060759971 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5b4108a50 msgr2=0x7fb5b419da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:23.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.712+0000 7fb5bb1cc700 1 --2- 172.21.15.12:0/4060759971 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5b4108a50 0x7fb5b419da60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac007ee0 tx=0x7fb5ac00d3b0 comp rx=0 tx=0).stop 2024-04-22T13:21:23.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.712+0000 7fb5bb1cc700 1 -- 172.21.15.12:0/4060759971 shutdown_connections 2024-04-22T13:21:23.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.712+0000 7fb5bb1cc700 1 --2- 172.21.15.12:0/4060759971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5b4102a90 0x7fb5b4199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:23.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.712+0000 7fb5bb1cc700 1 --2- 172.21.15.12:0/4060759971 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb59c0a3c40 0x7fb59c0a6100 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:23.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.712+0000 7fb5bb1cc700 1 --2- 172.21.15.12:0/4060759971 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5b4108a50 0x7fb5b419da60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:23.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.712+0000 7fb5bb1cc700 1 --2- 172.21.15.12:0/4060759971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5b4103450 0x7fb5b41996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:23.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.712+0000 7fb5bb1cc700 1 -- 172.21.15.12:0/4060759971 >> 172.21.15.12:0/4060759971 conn(0x7fb5b4079ba0 msgr2=0x7fb5b4101910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:23.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.712+0000 7fb5bb1cc700 1 -- 172.21.15.12:0/4060759971 shutdown_connections 2024-04-22T13:21:23.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:23.712+0000 7fb5bb1cc700 1 -- 172.21.15.12:0/4060759971 wait complete. 2024-04-22T13:21:23.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:23 smithi111 ceph-mon[31361]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:24.258 INFO:tasks.cephadm:Adding mds.a on smithi012 2024-04-22T13:21:24.259 INFO:tasks.cephadm:Adding mds.d on smithi012 2024-04-22T13:21:24.259 INFO:tasks.cephadm:Adding mds.g on smithi012 2024-04-22T13:21:24.259 INFO:tasks.cephadm:Adding mds.j on smithi012 2024-04-22T13:21:24.259 INFO:tasks.cephadm:Adding mds.b on smithi111 2024-04-22T13:21:24.259 INFO:tasks.cephadm:Adding mds.e on smithi111 2024-04-22T13:21:24.259 INFO:tasks.cephadm:Adding mds.h on smithi111 2024-04-22T13:21:24.259 INFO:tasks.cephadm:Adding mds.k on smithi111 2024-04-22T13:21:24.260 INFO:tasks.cephadm:Adding mds.c on smithi134 2024-04-22T13:21:24.260 INFO:tasks.cephadm:Adding mds.f on smithi134 2024-04-22T13:21:24.260 INFO:tasks.cephadm:Adding mds.i on smithi134 2024-04-22T13:21:24.260 INFO:tasks.cephadm:Adding mds.l on smithi134 2024-04-22T13:21:24.260 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l' 2024-04-22T13:21:24.509 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:21:24.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:24 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/4060759971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:21:24.630 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:24 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/4060759971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:21:24.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:24 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/4060759971' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:21:25.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:25 smithi134 ceph-mon[31338]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:25.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:25 smithi111 ceph-mon[31361]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:25.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:25 smithi012 ceph-mon[28265]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:25.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.893+0000 7f09ab855700 1 -- 172.21.15.134:0/2888917575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a4103c80 msgr2=0x7f09a4106110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:25.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.893+0000 7f09ab855700 1 --2- 172.21.15.134:0/2888917575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a4103c80 0x7f09a4106110 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0994009990 tx=0x7f0994009ca0 comp rx=0 tx=0).stop 2024-04-22T13:21:25.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.893+0000 7f09ab855700 1 -- 172.21.15.134:0/2888917575 shutdown_connections 2024-04-22T13:21:25.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.893+0000 7f09ab855700 1 --2- 172.21.15.134:0/2888917575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09a4101310 0x7f09a4103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:25.893 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.893+0000 7f09ab855700 1 --2- 172.21.15.134:0/2888917575 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09a4106650 0x7f09a4106ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:25.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.893+0000 7f09ab855700 1 --2- 172.21.15.134:0/2888917575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a4103c80 0x7f09a4106110 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:25.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.893+0000 7f09ab855700 1 -- 172.21.15.134:0/2888917575 >> 172.21.15.134:0/2888917575 conn(0x7f09a40fcf70 msgr2=0x7f09a40ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:25.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.893+0000 7f09ab855700 1 -- 172.21.15.134:0/2888917575 shutdown_connections 2024-04-22T13:21:25.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.893+0000 7f09ab855700 1 -- 172.21.15.134:0/2888917575 wait complete. 2024-04-22T13:21:25.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.894+0000 7f09ab855700 1 Processor -- start 2024-04-22T13:21:25.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.894+0000 7f09ab855700 1 -- start start 2024-04-22T13:21:25.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.894+0000 7f09ab855700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09a4101310 0x7f09a419f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:25.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.894+0000 7f09ab855700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09a4103c80 0x7f09a419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:25.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.894+0000 7f09ab855700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a4106650 0x7f09a41a6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:25.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.894+0000 7f09ab855700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f09a4114aa0 con 0x7f09a4101310 2024-04-22T13:21:25.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.894+0000 7f09ab855700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f09a4114920 con 0x7f09a4103c80 2024-04-22T13:21:25.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.894+0000 7f09ab855700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f09a4114c20 con 0x7f09a4106650 2024-04-22T13:21:25.894 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.894+0000 7f09a9df2700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a4106650 0x7f09a41a6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:25.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.894+0000 7f09a9df2700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a4106650 0x7f09a41a6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:51580/0 (socket says 172.21.15.134:51580) 2024-04-22T13:21:25.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.894+0000 7f09a9df2700 1 -- 172.21.15.134:0/3034704133 learned_addr learned my addr 172.21.15.134:0/3034704133 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-04-22T13:21:25.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.894+0000 7f09a8df0700 1 --2- 172.21.15.134:0/3034704133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09a4103c80 0x7f09a419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:25.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.894+0000 7f09a95f1700 1 --2- 172.21.15.134:0/3034704133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09a4101310 0x7f09a419f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:25.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.894+0000 7f09a9df2700 1 -- 172.21.15.134:0/3034704133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09a4103c80 msgr2=0x7f09a419f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:25.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.894+0000 7f09a9df2700 1 --2- 172.21.15.134:0/3034704133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09a4103c80 0x7f09a419f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:25.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.894+0000 7f09a9df2700 1 -- 172.21.15.134:0/3034704133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09a4101310 msgr2=0x7f09a419f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:25.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.894+0000 7f09a9df2700 1 --2- 172.21.15.134:0/3034704133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09a4101310 0x7f09a419f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:25.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.894+0000 7f09a9df2700 1 -- 172.21.15.134:0/3034704133 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a41a6dd0 con 0x7f09a4106650 2024-04-22T13:21:25.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.895+0000 7f09a9df2700 1 --2- 172.21.15.134:0/3034704133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a4106650 0x7f09a41a6730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f099c007f10 tx=0x7f099c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:25.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.895+0000 7f099a7fc700 1 -- 172.21.15.134:0/3034704133 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c004bb0 con 0x7f09a4106650 2024-04-22T13:21:25.895 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.895+0000 7f099a7fc700 1 -- 172.21.15.134:0/3034704133 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f099c004d50 con 0x7f09a4106650 2024-04-22T13:21:25.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.895+0000 7f09ab855700 1 -- 172.21.15.134:0/3034704133 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a41a70c0 con 0x7f09a4106650 2024-04-22T13:21:25.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.895+0000 7f099a7fc700 1 -- 172.21.15.134:0/3034704133 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c0058e0 con 0x7f09a4106650 2024-04-22T13:21:25.896 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.895+0000 7f09ab855700 1 -- 172.21.15.134:0/3034704133 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a41a75b0 con 0x7f09a4106650 2024-04-22T13:21:25.897 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.896+0000 7f09ab855700 1 -- 172.21.15.134:0/3034704133 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09a4069730 con 0x7f09a4106650 2024-04-22T13:21:25.897 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.897+0000 7f099a7fc700 1 -- 172.21.15.134:0/3034704133 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f099c01a020 con 0x7f09a4106650 2024-04-22T13:21:25.897 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.897+0000 7f099a7fc700 1 --2- 172.21.15.134:0/3034704133 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f09900a3c40 0x7f09900a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:25.897 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.898+0000 7f099a7fc700 1 -- 172.21.15.134:0/3034704133 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f099c0cfd10 con 0x7f09a4106650 2024-04-22T13:21:25.897 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.898+0000 7f09a95f1700 1 --2- 172.21.15.134:0/3034704133 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f09900a3c40 0x7f09900a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:25.897 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.898+0000 7f09a95f1700 1 --2- 172.21.15.134:0/3034704133 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f09900a3c40 0x7f09900a6100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f09a0005950 tx=0x7f09a000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:25.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:25.901+0000 7f099a7fc700 1 -- 172.21.15.134:0/3034704133 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f099c010070 con 0x7f09a4106650 2024-04-22T13:21:26.105 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:26.107+0000 7f09ab855700 1 -- 172.21.15.134:0/3034704133 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l", "target": ["mon-mgr", ""]}) -- 0x7f09a4063140 con 0x7f09900a3c40 2024-04-22T13:21:26.114 INFO:teuthology.orchestra.run.smithi134.stdout:Scheduled mds.all update... 2024-04-22T13:21:26.114 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:26.115+0000 7f099a7fc700 1 -- 172.21.15.134:0/3034704133 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f09a4063140 con 0x7f09900a3c40 2024-04-22T13:21:26.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:26.118+0000 7f09ab855700 1 -- 172.21.15.134:0/3034704133 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f09900a3c40 msgr2=0x7f09900a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:26.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:26.118+0000 7f09ab855700 1 --2- 172.21.15.134:0/3034704133 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f09900a3c40 0x7f09900a6100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f09a0005950 tx=0x7f09a000a300 comp rx=0 tx=0).stop 2024-04-22T13:21:26.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:26.118+0000 7f09ab855700 1 -- 172.21.15.134:0/3034704133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a4106650 msgr2=0x7f09a41a6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:26.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:26.118+0000 7f09ab855700 1 --2- 172.21.15.134:0/3034704133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a4106650 0x7f09a41a6730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f099c007f10 tx=0x7f099c00d3b0 comp rx=0 tx=0).stop 2024-04-22T13:21:26.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:26.118+0000 7f09ab855700 1 -- 172.21.15.134:0/3034704133 shutdown_connections 2024-04-22T13:21:26.116 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:26.118+0000 7f09ab855700 1 --2- 172.21.15.134:0/3034704133 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f09900a3c40 0x7f09900a6100 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:26.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:26.118+0000 7f09ab855700 1 --2- 172.21.15.134:0/3034704133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09a4106650 0x7f09a41a6730 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:26.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:26.118+0000 7f09ab855700 1 --2- 172.21.15.134:0/3034704133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09a4101310 0x7f09a419f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:26.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:26.118+0000 7f09ab855700 1 --2- 172.21.15.134:0/3034704133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09a4103c80 0x7f09a419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:26.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:26.118+0000 7f09ab855700 1 -- 172.21.15.134:0/3034704133 >> 172.21.15.134:0/3034704133 conn(0x7f09a40fcf70 msgr2=0x7f09a4101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:26.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:26.118+0000 7f09ab855700 1 -- 172.21.15.134:0/3034704133 shutdown_connections 2024-04-22T13:21:26.117 INFO:teuthology.orchestra.run.smithi134.stderr:2024-04-22T13:21:26.118+0000 7f09ab855700 1 -- 172.21.15.134:0/3034704133 wait complete. 2024-04-22T13:21:26.725 DEBUG:teuthology.orchestra.run.smithi012:mds.a> sudo journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.a.service 2024-04-22T13:21:26.728 DEBUG:teuthology.orchestra.run.smithi012:mds.d> sudo journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.d.service 2024-04-22T13:21:26.771 DEBUG:teuthology.orchestra.run.smithi012:mds.g> sudo journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.g.service 2024-04-22T13:21:26.815 DEBUG:teuthology.orchestra.run.smithi012:mds.j> sudo journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.j.service 2024-04-22T13:21:26.857 DEBUG:teuthology.orchestra.run.smithi111:mds.b> sudo journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.b.service 2024-04-22T13:21:26.860 DEBUG:teuthology.orchestra.run.smithi111:mds.e> sudo journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.e.service 2024-04-22T13:21:26.904 DEBUG:teuthology.orchestra.run.smithi111:mds.h> sudo journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.h.service 2024-04-22T13:21:26.947 DEBUG:teuthology.orchestra.run.smithi111:mds.k> sudo journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.k.service 2024-04-22T13:21:26.990 DEBUG:teuthology.orchestra.run.smithi134:mds.c> sudo journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.c.service 2024-04-22T13:21:26.992 DEBUG:teuthology.orchestra.run.smithi134:mds.f> sudo journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.f.service 2024-04-22T13:21:27.035 DEBUG:teuthology.orchestra.run.smithi134:mds.i> sudo journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.i.service 2024-04-22T13:21:27.078 DEBUG:teuthology.orchestra.run.smithi134:mds.l> sudo journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.l.service 2024-04-22T13:21:27.121 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-22T13:21:27.121 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:21:27.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:27 smithi111 ceph-mon[31361]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:21:27.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:27 smithi111 ceph-mon[31361]: Saving service mds.all spec with placement smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l;count:12 2024-04-22T13:21:27.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:27 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:27.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:27 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:21:27.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:27 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:27.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:27 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:21:27.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:27 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:27.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:27 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:27.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:27 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:27.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:27 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:27.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.260+0000 7fe69ad67640 1 -- 172.21.15.12:0/1560192821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe694103190 msgr2=0x7fe694103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:27.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.260+0000 7fe69ad67640 1 --2- 172.21.15.12:0/1560192821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe694103190 0x7fe694103610 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe684009900 tx=0x7fe68402bc30 comp rx=0 tx=0).stop 2024-04-22T13:21:27.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.260+0000 7fe69ad67640 1 -- 172.21.15.12:0/1560192821 shutdown_connections 2024-04-22T13:21:27.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.260+0000 7fe69ad67640 1 --2- 172.21.15.12:0/1560192821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe694103b50 0x7fe69410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:27.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.260+0000 7fe69ad67640 1 --2- 172.21.15.12:0/1560192821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe694103190 0x7fe694103610 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:27.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.260+0000 7fe69ad67640 1 --2- 172.21.15.12:0/1560192821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe694101fa0 0x7fe6941023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:27.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.260+0000 7fe69ad67640 1 -- 172.21.15.12:0/1560192821 >> 172.21.15.12:0/1560192821 conn(0x7fe6940fd530 msgr2=0x7fe6940ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:27.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.261+0000 7fe69ad67640 1 -- 172.21.15.12:0/1560192821 shutdown_connections 2024-04-22T13:21:27.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.261+0000 7fe69ad67640 1 -- 172.21.15.12:0/1560192821 wait complete. 2024-04-22T13:21:27.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.261+0000 7fe69ad67640 1 Processor -- start 2024-04-22T13:21:27.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.261+0000 7fe69ad67640 1 -- start start 2024-04-22T13:21:27.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.261+0000 7fe69ad67640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe694101fa0 0x7fe694198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:27.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.261+0000 7fe69ad67640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe694103190 0x7fe694198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:27.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.261+0000 7fe69ad67640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe694103b50 0x7fe69419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:27.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.261+0000 7fe69ad67640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe69410dae0 con 0x7fe694103b50 2024-04-22T13:21:27.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.261+0000 7fe69ad67640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe69410d960 con 0x7fe694101fa0 2024-04-22T13:21:27.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.261+0000 7fe69ad67640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe69410dc60 con 0x7fe694103190 2024-04-22T13:21:27.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.261+0000 7fe699304640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe694103b50 0x7fe69419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:27.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.261+0000 7fe699304640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe694103b50 0x7fe69419f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39802/0 (socket says 172.21.15.12:39802) 2024-04-22T13:21:27.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.261+0000 7fe699304640 1 -- 172.21.15.12:0/4099214610 learned_addr learned my addr 172.21.15.12:0/4099214610 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:21:27.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.261+0000 7fe68bfff640 1 --2- 172.21.15.12:0/4099214610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe694103190 0x7fe694198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:27.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.262+0000 7fe698b03640 1 --2- 172.21.15.12:0/4099214610 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe694101fa0 0x7fe694198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:27.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.262+0000 7fe699304640 1 -- 172.21.15.12:0/4099214610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe694103190 msgr2=0x7fe694198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:27.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.262+0000 7fe699304640 1 --2- 172.21.15.12:0/4099214610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe694103190 0x7fe694198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:27.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.262+0000 7fe699304640 1 -- 172.21.15.12:0/4099214610 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe694101fa0 msgr2=0x7fe694198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:27.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.262+0000 7fe699304640 1 --2- 172.21.15.12:0/4099214610 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe694101fa0 0x7fe694198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:27.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.262+0000 7fe699304640 1 -- 172.21.15.12:0/4099214610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69419fd70 con 0x7fe694103b50 2024-04-22T13:21:27.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.262+0000 7fe699304640 1 --2- 172.21.15.12:0/4099214610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe694103b50 0x7fe69419f700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe68c00eb10 tx=0x7fe68c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:27.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.262+0000 7fe689ffb640 1 -- 172.21.15.12:0/4099214610 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c004b20 con 0x7fe694103b50 2024-04-22T13:21:27.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.262+0000 7fe689ffb640 1 -- 172.21.15.12:0/4099214610 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe68c004cc0 con 0x7fe694103b50 2024-04-22T13:21:27.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.262+0000 7fe69ad67640 1 -- 172.21.15.12:0/4099214610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6941a0060 con 0x7fe694103b50 2024-04-22T13:21:27.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.262+0000 7fe689ffb640 1 -- 172.21.15.12:0/4099214610 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c010670 con 0x7fe694103b50 2024-04-22T13:21:27.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.262+0000 7fe69ad67640 1 -- 172.21.15.12:0/4099214610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6941a0530 con 0x7fe694103b50 2024-04-22T13:21:27.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.263+0000 7fe69ad67640 1 -- 172.21.15.12:0/4099214610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6941092c0 con 0x7fe694103b50 2024-04-22T13:21:27.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.264+0000 7fe689ffb640 1 -- 172.21.15.12:0/4099214610 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe68c005020 con 0x7fe694103b50 2024-04-22T13:21:27.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.265+0000 7fe689ffb640 1 --2- 172.21.15.12:0/4099214610 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe67c0a3bb0 0x7fe67c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:27.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.265+0000 7fe689ffb640 1 -- 172.21.15.12:0/4099214610 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fe68c0d0450 con 0x7fe694103b50 2024-04-22T13:21:27.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.265+0000 7fe698b03640 1 --2- 172.21.15.12:0/4099214610 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe67c0a3bb0 0x7fe67c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:27.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.265+0000 7fe698b03640 1 --2- 172.21.15.12:0/4099214610 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe67c0a3bb0 0x7fe67c0a6070 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe690005fd0 tx=0x7fe69000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:27.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.270+0000 7fe689ffb640 1 -- 172.21.15.12:0/4099214610 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe68c09d2d0 con 0x7fe694103b50 2024-04-22T13:21:27.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:27 smithi134 ceph-mon[31338]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:21:27.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:27 smithi134 ceph-mon[31338]: Saving service mds.all spec with placement smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l;count:12 2024-04-22T13:21:27.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:27 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:27.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:27 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:21:27.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:27 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:27.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:27 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:21:27.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:27 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:27.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:27 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:27.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:27 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:27.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:27 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:27.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.457+0000 7fe69ad67640 1 -- 172.21.15.12:0/4099214610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe694062b50 con 0x7fe694103b50 2024-04-22T13:21:27.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.458+0000 7fe689ffb640 1 -- 172.21.15.12:0/4099214610 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+18470 (secure 0 0 0) 0x7fe68c09cbd0 con 0x7fe694103b50 2024-04-22T13:21:27.455 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:21:27.455 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":83,"fsid":"0aede5ea-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:14:09.362918+0000","modified":"2024-04-22T13:21:16.272831+0000","last_up_change":"2024-04-22T13:21:15.263393+0000","last_in_change":"2024-04-22T13:20:54.174685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:17:32.372434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d9e2365-3a64-4b13-96fc-c217b1cee2ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6803","nonce":1191350632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6805","nonce":1191350632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6809","nonce":1191350632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6807","nonce":1191350632}]},"public_addr":"172.21.15.12:6803/1191350632","cluster_addr":"172.21.15.12:6805/1191350632","heartbeat_back_addr":"172.21.15.12:6809/1191350632","heartbeat_front_addr":"172.21.15.12:6807/1191350632","state":["exists","up"]},{"osd":1,"uuid":"1499c783-2bd7-48ce-a444-34add42bcf31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6801","nonce":417496465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6803","nonce":417496465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6807","nonce":417496465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6805","nonce":417496465}]},"public_addr":"172.21.15.111:6801/417496465","cluster_addr":"172.21.15.111:6803/417496465","heartbeat_back_addr":"172.21.15.111:6807/417496465","heartbeat_front_addr":"172.21.15.111:6805/417496465","state":["exists","up"]},{"osd":2,"uuid":"71a9ebc3-c622-434e-a6c2-f48358f384f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6801","nonce":1403153998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6803","nonce":1403153998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6807","nonce":1403153998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6805","nonce":1403153998}]},"public_addr":"172.21.15.134:6801/1403153998","cluster_addr":"172.21.15.134:6803/1403153998","heartbeat_back_addr":"172.21.15.134:6807/1403153998","heartbeat_front_addr":"172.21.15.134:6805/1403153998","state":["exists","up"]},{"osd":3,"uuid":"6dd00db2-a286-48ba-8666-56ffdd5f35ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6811","nonce":3359066681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6813","nonce":3359066681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6817","nonce":3359066681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6815","nonce":3359066681}]},"public_addr":"172.21.15.12:6811/3359066681","cluster_addr":"172.21.15.12:6813/3359066681","heartbeat_back_addr":"172.21.15.12:6817/3359066681","heartbeat_front_addr":"172.21.15.12:6815/3359066681","state":["exists","up"]},{"osd":4,"uuid":"1e29afd7-354c-4506-aaa0-b43eb1a19a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6809","nonce":2886349872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6811","nonce":2886349872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6815","nonce":2886349872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6813","nonce":2886349872}]},"public_addr":"172.21.15.111:6809/2886349872","cluster_addr":"172.21.15.111:6811/2886349872","heartbeat_back_addr":"172.21.15.111:6815/2886349872","heartbeat_front_addr":"172.21.15.111:6813/2886349872","state":["exists","up"]},{"osd":5,"uuid":"8f40f02d-0ba2-4842-a03d-2a1a04d1010d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6809","nonce":3071977602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6811","nonce":3071977602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6815","nonce":3071977602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6813","nonce":3071977602}]},"public_addr":"172.21.15.134:6809/3071977602","cluster_addr":"172.21.15.134:6811/3071977602","heartbeat_back_addr":"172.21.15.134:6815/3071977602","heartbeat_front_addr":"172.21.15.134:6813/3071977602","state":["exists","up"]},{"osd":6,"uuid":"bb0c17a9-6a8e-4be6-8660-f73cfa8200ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6819","nonce":3819619299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6821","nonce":3819619299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6825","nonce":3819619299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6823","nonce":3819619299}]},"public_addr":"172.21.15.12:6819/3819619299","cluster_addr":"172.21.15.12:6821/3819619299","heartbeat_back_addr":"172.21.15.12:6825/3819619299","heartbeat_front_addr":"172.21.15.12:6823/3819619299","state":["exists","up"]},{"osd":7,"uuid":"0a93d6af-569e-4e45-b697-9e978c1c1004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6817","nonce":3113856282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6819","nonce":3113856282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6823","nonce":3113856282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6821","nonce":3113856282}]},"public_addr":"172.21.15.111:6817/3113856282","cluster_addr":"172.21.15.111:6819/3113856282","heartbeat_back_addr":"172.21.15.111:6823/3113856282","heartbeat_front_addr":"172.21.15.111:6821/3113856282","state":["exists","up"]},{"osd":8,"uuid":"e21acc68-cf50-4f4f-b1ba-cf6c195b4955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6817","nonce":1689362191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6819","nonce":1689362191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6823","nonce":1689362191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6821","nonce":1689362191}]},"public_addr":"172.21.15.134:6817/1689362191","cluster_addr":"172.21.15.134:6819/1689362191","heartbeat_back_addr":"172.21.15.134:6823/1689362191","heartbeat_front_addr":"172.21.15.134:6821/1689362191","state":["exists","up"]},{"osd":9,"uuid":"fd745869-7f5f-48a8-b412-bb9c92b586e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6827","nonce":4292385779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6829","nonce":4292385779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6833","nonce":4292385779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6831","nonce":4292385779}]},"public_addr":"172.21.15.12:6827/4292385779","cluster_addr":"172.21.15.12:6829/4292385779","heartbeat_back_addr":"172.21.15.12:6833/4292385779","heartbeat_front_addr":"172.21.15.12:6831/4292385779","state":["exists","up"]},{"osd":10,"uuid":"798b1abd-f6c7-4a5f-8da0-fdae49e50d2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6825","nonce":3338566976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6827","nonce":3338566976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6831","nonce":3338566976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6829","nonce":3338566976}]},"public_addr":"172.21.15.111:6825/3338566976","cluster_addr":"172.21.15.111:6827/3338566976","heartbeat_back_addr":"172.21.15.111:6831/3338566976","heartbeat_front_addr":"172.21.15.111:6829/3338566976","state":["exists","up"]},{"osd":11,"uuid":"b18fc5f3-1049-4871-b060-e99290273cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6825","nonce":578640428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6827","nonce":578640428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6831","nonce":578640428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6829","nonce":578640428}]},"public_addr":"172.21.15.134:6825/578640428","cluster_addr":"172.21.15.134:6827/578640428","heartbeat_back_addr":"172.21.15.134:6831/578640428","heartbeat_front_addr":"172.21.15.134:6829/578640428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:16:40.828608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:05.052528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:28.297316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:53.496497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:17.947259+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:41.712894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:07.105945+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:32.129688+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:56.008516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:22.239867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:47.334692+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:12.301276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/2593601362":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/2519382069":"2024-04-23T13:15:04.279990+0000","172.21.15.12:6801/2009831940":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3509412187":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1007749062":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3588480095":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1904962176":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2009831940":"2024-04-23T13:15:04.279990+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:21:27.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.460+0000 7fe69ad67640 1 -- 172.21.15.12:0/4099214610 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe67c0a3bb0 msgr2=0x7fe67c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:27.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.460+0000 7fe69ad67640 1 --2- 172.21.15.12:0/4099214610 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe67c0a3bb0 0x7fe67c0a6070 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe690005fd0 tx=0x7fe69000b410 comp rx=0 tx=0).stop 2024-04-22T13:21:27.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.460+0000 7fe69ad67640 1 -- 172.21.15.12:0/4099214610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe694103b50 msgr2=0x7fe69419f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:27.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.460+0000 7fe69ad67640 1 --2- 172.21.15.12:0/4099214610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe694103b50 0x7fe69419f700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe68c00eb10 tx=0x7fe68c00c2d0 comp rx=0 tx=0).stop 2024-04-22T13:21:27.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.460+0000 7fe69ad67640 1 -- 172.21.15.12:0/4099214610 shutdown_connections 2024-04-22T13:21:27.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.460+0000 7fe69ad67640 1 --2- 172.21.15.12:0/4099214610 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe67c0a3bb0 0x7fe67c0a6070 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:27.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.460+0000 7fe69ad67640 1 --2- 172.21.15.12:0/4099214610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe694103b50 0x7fe69419f700 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:27.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.460+0000 7fe69ad67640 1 --2- 172.21.15.12:0/4099214610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe694103190 0x7fe694198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:27.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.460+0000 7fe69ad67640 1 --2- 172.21.15.12:0/4099214610 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe694101fa0 0x7fe694198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:27.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.460+0000 7fe69ad67640 1 -- 172.21.15.12:0/4099214610 >> 172.21.15.12:0/4099214610 conn(0x7fe6940fd530 msgr2=0x7fe694106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:27.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.460+0000 7fe69ad67640 1 -- 172.21.15.12:0/4099214610 shutdown_connections 2024-04-22T13:21:27.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.460+0000 7fe69ad67640 1 -- 172.21.15.12:0/4099214610 wait complete. 2024-04-22T13:21:27.466 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:27 smithi012 ceph-mon[28265]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:21:27.466 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:27 smithi012 ceph-mon[28265]: Saving service mds.all spec with placement smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l;count:12 2024-04-22T13:21:27.467 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:27 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:27.467 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:27 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:21:27.467 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:27 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:27.467 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:27 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:21:27.467 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:27 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:27.467 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:27 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:27.467 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:27 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:27.467 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:27 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:27.468 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T13:17:32.372434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T13:21:27.469 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T13:21:27.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.576+0000 7f41332e0640 1 -- 172.21.15.12:0/1010760633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f412c100e90 msgr2=0x7f412c1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:27.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.576+0000 7f41332e0640 1 --2- 172.21.15.12:0/1010760633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f412c100e90 0x7f412c1012b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f411c009ca0 tx=0x7f411c02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:21:27.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.577+0000 7f41332e0640 1 -- 172.21.15.12:0/1010760633 shutdown_connections 2024-04-22T13:21:27.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.577+0000 7f41332e0640 1 --2- 172.21.15.12:0/1010760633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f412c1029a0 0x7f412c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:27.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.577+0000 7f41332e0640 1 --2- 172.21.15.12:0/1010760633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f412c101fe0 0x7f412c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:27.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.577+0000 7f41332e0640 1 --2- 172.21.15.12:0/1010760633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f412c100e90 0x7f412c1012b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:27.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.577+0000 7f41332e0640 1 -- 172.21.15.12:0/1010760633 >> 172.21.15.12:0/1010760633 conn(0x7f412c0fc400 msgr2=0x7f412c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:27.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.577+0000 7f41332e0640 1 -- 172.21.15.12:0/1010760633 shutdown_connections 2024-04-22T13:21:27.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.577+0000 7f41332e0640 1 -- 172.21.15.12:0/1010760633 wait complete. 2024-04-22T13:21:27.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.577+0000 7f41332e0640 1 Processor -- start 2024-04-22T13:21:27.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.577+0000 7f41332e0640 1 -- start start 2024-04-22T13:21:27.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.577+0000 7f41332e0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f412c100e90 0x7f412c069c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:27.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.577+0000 7f41332e0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f412c101fe0 0x7f412c19cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:27.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.577+0000 7f41332e0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f412c1029a0 0x7f412c19f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:27.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.577+0000 7f41332e0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f412c10c9c0 con 0x7f412c1029a0 2024-04-22T13:21:27.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.577+0000 7f41332e0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f412c10c840 con 0x7f412c101fe0 2024-04-22T13:21:27.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.577+0000 7f41332e0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f412c10cb40 con 0x7f412c100e90 2024-04-22T13:21:27.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.578+0000 7f413187d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f412c1029a0 0x7f412c19f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:27.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.578+0000 7f413107c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f412c100e90 0x7f412c069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:27.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.578+0000 7f413187d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f412c1029a0 0x7f412c19f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39818/0 (socket says 172.21.15.12:39818) 2024-04-22T13:21:27.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.578+0000 7f413107c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f412c100e90 0x7f412c069c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.12:54388/0 (socket says 172.21.15.12:54388) 2024-04-22T13:21:27.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.578+0000 7f413187d640 1 -- 172.21.15.12:0/4160001277 learned_addr learned my addr 172.21.15.12:0/4160001277 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:21:27.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.578+0000 7f413087b640 1 --2- 172.21.15.12:0/4160001277 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f412c101fe0 0x7f412c19cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:27.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.578+0000 7f413187d640 1 -- 172.21.15.12:0/4160001277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f412c100e90 msgr2=0x7f412c069c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:27.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.578+0000 7f413187d640 1 --2- 172.21.15.12:0/4160001277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f412c100e90 0x7f412c069c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:27.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.578+0000 7f413187d640 1 -- 172.21.15.12:0/4160001277 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f412c101fe0 msgr2=0x7f412c19cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:27.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.578+0000 7f413187d640 1 --2- 172.21.15.12:0/4160001277 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f412c101fe0 0x7f412c19cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:27.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.578+0000 7f413187d640 1 -- 172.21.15.12:0/4160001277 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f412c19fe30 con 0x7f412c1029a0 2024-04-22T13:21:27.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.578+0000 7f413087b640 1 --2- 172.21.15.12:0/4160001277 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f412c101fe0 0x7f412c19cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:21:27.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.578+0000 7f413187d640 1 --2- 172.21.15.12:0/4160001277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f412c1029a0 0x7f412c19f570 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f412400d750 tx=0x7f412400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:27.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.578+0000 7f412a7fc640 1 -- 172.21.15.12:0/4160001277 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4124017070 con 0x7f412c1029a0 2024-04-22T13:21:27.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.578+0000 7f41332e0640 1 -- 172.21.15.12:0/4160001277 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f412c1a0120 con 0x7f412c1029a0 2024-04-22T13:21:27.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.578+0000 7f412a7fc640 1 -- 172.21.15.12:0/4160001277 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41240054d0 con 0x7f412c1029a0 2024-04-22T13:21:27.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.578+0000 7f412a7fc640 1 -- 172.21.15.12:0/4160001277 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4124004c60 con 0x7f412c1029a0 2024-04-22T13:21:27.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.578+0000 7f41332e0640 1 -- 172.21.15.12:0/4160001277 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f412c1a0670 con 0x7f412c1029a0 2024-04-22T13:21:27.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.579+0000 7f41332e0640 1 -- 172.21.15.12:0/4160001277 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f412c108310 con 0x7f412c1029a0 2024-04-22T13:21:27.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.580+0000 7f412a7fc640 1 -- 172.21.15.12:0/4160001277 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4124009bd0 con 0x7f412c1029a0 2024-04-22T13:21:27.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.581+0000 7f412a7fc640 1 --2- 172.21.15.12:0/4160001277 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f41140a3bb0 0x7f41140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:27.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.581+0000 7f413107c640 1 --2- 172.21.15.12:0/4160001277 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f41140a3bb0 0x7f41140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:27.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.581+0000 7f412a7fc640 1 -- 172.21.15.12:0/4160001277 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f41240cfd60 con 0x7f412c1029a0 2024-04-22T13:21:27.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.582+0000 7f413107c640 1 --2- 172.21.15.12:0/4160001277 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f41140a3bb0 0x7f41140a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f411c005e30 tx=0x7f411c000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:27.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.585+0000 7f412a7fc640 1 -- 172.21.15.12:0/4160001277 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f412409cbe0 con 0x7f412c1029a0 2024-04-22T13:21:27.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.771+0000 7f41332e0640 1 -- 172.21.15.12:0/4160001277 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f412c02d7c0 con 0x7f412c1029a0 2024-04-22T13:21:27.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.771+0000 7f412a7fc640 1 -- 172.21.15.12:0/4160001277 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f412409c4e0 con 0x7f412c1029a0 2024-04-22T13:21:27.768 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-22T13:21:27.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.773+0000 7f41332e0640 1 -- 172.21.15.12:0/4160001277 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f41140a3bb0 msgr2=0x7f41140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:27.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.773+0000 7f41332e0640 1 --2- 172.21.15.12:0/4160001277 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f41140a3bb0 0x7f41140a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f411c005e30 tx=0x7f411c000f40 comp rx=0 tx=0).stop 2024-04-22T13:21:27.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.773+0000 7f41332e0640 1 -- 172.21.15.12:0/4160001277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f412c1029a0 msgr2=0x7f412c19f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:27.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.774+0000 7f41332e0640 1 --2- 172.21.15.12:0/4160001277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f412c1029a0 0x7f412c19f570 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f412400d750 tx=0x7f412400dc10 comp rx=0 tx=0).stop 2024-04-22T13:21:27.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.774+0000 7f41332e0640 1 -- 172.21.15.12:0/4160001277 shutdown_connections 2024-04-22T13:21:27.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.774+0000 7f41332e0640 1 --2- 172.21.15.12:0/4160001277 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f41140a3bb0 0x7f41140a6070 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:27.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.774+0000 7f41332e0640 1 --2- 172.21.15.12:0/4160001277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f412c1029a0 0x7f412c19f570 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:27.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.774+0000 7f41332e0640 1 --2- 172.21.15.12:0/4160001277 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f412c101fe0 0x7f412c19cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:27.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.774+0000 7f41332e0640 1 --2- 172.21.15.12:0/4160001277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f412c100e90 0x7f412c069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:27.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.774+0000 7f41332e0640 1 -- 172.21.15.12:0/4160001277 >> 172.21.15.12:0/4160001277 conn(0x7f412c0fc400 msgr2=0x7f412c1054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:27.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.774+0000 7f41332e0640 1 -- 172.21.15.12:0/4160001277 shutdown_connections 2024-04-22T13:21:27.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.774+0000 7f41332e0640 1 -- 172.21.15.12:0/4160001277 wait complete. 2024-04-22T13:21:27.781 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:21:27.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.885+0000 7f14f0d7e640 1 -- 172.21.15.12:0/2695604365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14ec101fa0 msgr2=0x7f14ec1023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:27.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.885+0000 7f14f0d7e640 1 --2- 172.21.15.12:0/2695604365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14ec101fa0 0x7f14ec1023c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009900 tx=0x7f14e402bc30 comp rx=0 tx=0).stop 2024-04-22T13:21:27.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.886+0000 7f14f0d7e640 1 -- 172.21.15.12:0/2695604365 shutdown_connections 2024-04-22T13:21:27.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.886+0000 7f14f0d7e640 1 --2- 172.21.15.12:0/2695604365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14ec103b50 0x7f14ec10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:27.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.886+0000 7f14f0d7e640 1 --2- 172.21.15.12:0/2695604365 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14ec103190 0x7f14ec103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:27.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.886+0000 7f14f0d7e640 1 --2- 172.21.15.12:0/2695604365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14ec101fa0 0x7f14ec1023c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:27.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.886+0000 7f14f0d7e640 1 -- 172.21.15.12:0/2695604365 >> 172.21.15.12:0/2695604365 conn(0x7f14ec0fd530 msgr2=0x7f14ec0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:27.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.886+0000 7f14f0d7e640 1 -- 172.21.15.12:0/2695604365 shutdown_connections 2024-04-22T13:21:27.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.886+0000 7f14f0d7e640 1 -- 172.21.15.12:0/2695604365 wait complete. 2024-04-22T13:21:27.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.886+0000 7f14f0d7e640 1 Processor -- start 2024-04-22T13:21:27.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.886+0000 7f14f0d7e640 1 -- start start 2024-04-22T13:21:27.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.886+0000 7f14f0d7e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14ec101fa0 0x7f14ec198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:27.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.886+0000 7f14f0d7e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14ec103190 0x7f14ec198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:27.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.887+0000 7f14f0d7e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14ec103b50 0x7f14ec19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:27.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.887+0000 7f14f0d7e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec10daa0 con 0x7f14ec103b50 2024-04-22T13:21:27.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.887+0000 7f14f0d7e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec10d920 con 0x7f14ec101fa0 2024-04-22T13:21:27.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.887+0000 7f14f0d7e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec10dc20 con 0x7f14ec103190 2024-04-22T13:21:27.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.887+0000 7f14ead9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14ec103b50 0x7f14ec19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:27.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.887+0000 7f14ead9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14ec103b50 0x7f14ec19f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39832/0 (socket says 172.21.15.12:39832) 2024-04-22T13:21:27.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.887+0000 7f14ead9d640 1 -- 172.21.15.12:0/1515535257 learned_addr learned my addr 172.21.15.12:0/1515535257 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:21:27.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.887+0000 7f14e9d9b640 1 --2- 172.21.15.12:0/1515535257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14ec103190 0x7f14ec198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:27.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.887+0000 7f14ea59c640 1 --2- 172.21.15.12:0/1515535257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14ec101fa0 0x7f14ec198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:27.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.887+0000 7f14ead9d640 1 -- 172.21.15.12:0/1515535257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14ec103190 msgr2=0x7f14ec198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:27.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.887+0000 7f14ead9d640 1 --2- 172.21.15.12:0/1515535257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14ec103190 0x7f14ec198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:27.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.887+0000 7f14ead9d640 1 -- 172.21.15.12:0/1515535257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14ec101fa0 msgr2=0x7f14ec198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:27.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.887+0000 7f14ead9d640 1 --2- 172.21.15.12:0/1515535257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14ec101fa0 0x7f14ec198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:27.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.887+0000 7f14ead9d640 1 -- 172.21.15.12:0/1515535257 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ec19fd30 con 0x7f14ec103b50 2024-04-22T13:21:27.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.887+0000 7f14ead9d640 1 --2- 172.21.15.12:0/1515535257 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14ec103b50 0x7f14ec19f6c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00cce0 tx=0x7f14dc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:27.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.887+0000 7f14db7fe640 1 -- 172.21.15.12:0/1515535257 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc00f450 con 0x7f14ec103b50 2024-04-22T13:21:27.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.887+0000 7f14db7fe640 1 -- 172.21.15.12:0/1515535257 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14dc00faa0 con 0x7f14ec103b50 2024-04-22T13:21:27.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.887+0000 7f14f0d7e640 1 -- 172.21.15.12:0/1515535257 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14ec19ffc0 con 0x7f14ec103b50 2024-04-22T13:21:27.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.887+0000 7f14db7fe640 1 -- 172.21.15.12:0/1515535257 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14dc018b70 con 0x7f14ec103b50 2024-04-22T13:21:27.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.887+0000 7f14f0d7e640 1 -- 172.21.15.12:0/1515535257 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14ec1a0490 con 0x7f14ec103b50 2024-04-22T13:21:27.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.888+0000 7f14f0d7e640 1 -- 172.21.15.12:0/1515535257 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14ec1092c0 con 0x7f14ec103b50 2024-04-22T13:21:27.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.889+0000 7f14db7fe640 1 -- 172.21.15.12:0/1515535257 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f14dc018d10 con 0x7f14ec103b50 2024-04-22T13:21:27.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.890+0000 7f14db7fe640 1 --2- 172.21.15.12:0/1515535257 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f14d40a3bb0 0x7f14d40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:27.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.890+0000 7f14ea59c640 1 --2- 172.21.15.12:0/1515535257 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f14d40a3bb0 0x7f14d40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:27.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.890+0000 7f14db7fe640 1 -- 172.21.15.12:0/1515535257 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f14dc0d0310 con 0x7f14ec103b50 2024-04-22T13:21:27.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.891+0000 7f14ea59c640 1 --2- 172.21.15.12:0/1515535257 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f14d40a3bb0 0x7f14d40a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009230 tx=0x7f14e4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:27.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:27.895+0000 7f14db7fe640 1 -- 172.21.15.12:0/1515535257 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f14dc014030 con 0x7f14ec103b50 2024-04-22T13:21:28.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:28.118+0000 7f14f0d7e640 1 -- 172.21.15.12:0/1515535257 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14ec062b50 con 0x7f14ec103b50 2024-04-22T13:21:28.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:28.119+0000 7f14db7fe640 1 -- 172.21.15.12:0/1515535257 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f14dc09d190 con 0x7f14ec103b50 2024-04-22T13:21:28.116 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:21:28.116 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-22T13:21:28.116 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 1 2024-04-22T13:21:28.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:28.121+0000 7f14f0d7e640 1 -- 172.21.15.12:0/1515535257 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f14d40a3bb0 msgr2=0x7f14d40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:28.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:28.121+0000 7f14f0d7e640 1 --2- 172.21.15.12:0/1515535257 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f14d40a3bb0 0x7f14d40a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009230 tx=0x7f14e4005fd0 comp rx=0 tx=0).stop 2024-04-22T13:21:28.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:28.121+0000 7f14f0d7e640 1 -- 172.21.15.12:0/1515535257 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14ec103b50 msgr2=0x7f14ec19f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:28.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:28.121+0000 7f14f0d7e640 1 --2- 172.21.15.12:0/1515535257 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14ec103b50 0x7f14ec19f6c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00cce0 tx=0x7f14dc00a620 comp rx=0 tx=0).stop 2024-04-22T13:21:28.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:28.121+0000 7f14f0d7e640 1 -- 172.21.15.12:0/1515535257 shutdown_connections 2024-04-22T13:21:28.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:28.121+0000 7f14f0d7e640 1 --2- 172.21.15.12:0/1515535257 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f14d40a3bb0 0x7f14d40a6070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:28.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:28.121+0000 7f14f0d7e640 1 --2- 172.21.15.12:0/1515535257 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14ec103b50 0x7f14ec19f6c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:28.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:28.121+0000 7f14f0d7e640 1 --2- 172.21.15.12:0/1515535257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14ec103190 0x7f14ec198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:28.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:28.121+0000 7f14f0d7e640 1 --2- 172.21.15.12:0/1515535257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14ec101fa0 0x7f14ec198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:28.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:28.121+0000 7f14f0d7e640 1 -- 172.21.15.12:0/1515535257 >> 172.21.15.12:0/1515535257 conn(0x7f14ec0fd530 msgr2=0x7f14ec106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:28.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:28.121+0000 7f14f0d7e640 1 -- 172.21.15.12:0/1515535257 shutdown_connections 2024-04-22T13:21:28.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:28.121+0000 7f14f0d7e640 1 -- 172.21.15.12:0/1515535257 wait complete. 2024-04-22T13:21:28.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:28 smithi134 ceph-mon[31338]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:28.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:28 smithi134 ceph-mon[31338]: Deploying daemon mds.l on smithi134 2024-04-22T13:21:28.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:28 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/4099214610' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:21:28.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:28 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/4160001277' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:21:28.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:28 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1515535257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:28.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:28 smithi111 ceph-mon[31361]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:28.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:28 smithi111 ceph-mon[31361]: Deploying daemon mds.l on smithi134 2024-04-22T13:21:28.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:28 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/4099214610' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:21:28.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:28 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/4160001277' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:21:28.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:28 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1515535257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:28.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:28 smithi012 ceph-mon[28265]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:28.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:28 smithi012 ceph-mon[28265]: Deploying daemon mds.l on smithi134 2024-04-22T13:21:28.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:28 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/4099214610' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:21:28.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:28 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/4160001277' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:21:28.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:28 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1515535257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:29.178 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 22 13:21:28 smithi134 systemd[1]: Starting Ceph mds.l for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:21:29.460 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 22 13:21:29 smithi134 podman[61843]: 2024-04-22 13:21:29.099127033 +0000 UTC m=+0.018794775 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:21:30.237 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 22 13:21:29 smithi134 podman[61843]: 2024-04-22 13:21:29.220039402 +0000 UTC m=+0.139707118 container create a9d4a04af81444ff2a58793619ac037e6593743062f91453f7580721b291faab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-22T13:21:30.237 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 22 13:21:29 smithi134 podman[61843]: 2024-04-22 13:21:29.397206451 +0000 UTC m=+0.316874175 container init a9d4a04af81444ff2a58793619ac037e6593743062f91453f7580721b291faab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-22T13:21:30.237 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 22 13:21:29 smithi134 podman[61843]: 2024-04-22 13:21:29.399590991 +0000 UTC m=+0.319258714 container start a9d4a04af81444ff2a58793619ac037e6593743062f91453f7580721b291faab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-22T13:21:30.237 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 22 13:21:29 smithi134 ceph-mds[61862]: -- 172.21.15.134:0/666791073 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b7d9bfda40 con 0x55b7d9bd1000 2024-04-22T13:21:30.237 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 22 13:21:29 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-l[61858]: starting mds.l at 2024-04-22T13:21:30.237 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 22 13:21:29 smithi134 bash[61843]: a9d4a04af81444ff2a58793619ac037e6593743062f91453f7580721b291faab 2024-04-22T13:21:30.237 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:21:30.239 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:30 smithi111 ceph-mon[31361]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:30.239 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:30 smithi111 ceph-mon[31361]: mds.? [v2:172.21.15.134:6832/4235830781,v1:172.21.15.134:6833/4235830781] up:boot 2024-04-22T13:21:30.239 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:30 smithi111 ceph-mon[31361]: fsmap 1 up:standby 2024-04-22T13:21:30.239 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:30 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-22T13:21:30.239 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:30 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:30.239 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:30 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:30.239 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:30 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:30.239 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:30 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:30.239 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:30 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:30.239 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:30 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:30.239 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 22 13:21:29 smithi134 systemd[1]: Started Ceph mds.l for 0aede5ea-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:21:30.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.356+0000 7fa56d984640 1 -- 172.21.15.12:0/3477393612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa568103190 msgr2=0x7fa568103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:30.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.356+0000 7fa56d984640 1 --2- 172.21.15.12:0/3477393612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa568103190 0x7fa568103610 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa55c009900 tx=0x7fa55c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:21:30.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.356+0000 7fa56d984640 1 -- 172.21.15.12:0/3477393612 shutdown_connections 2024-04-22T13:21:30.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.356+0000 7fa56d984640 1 --2- 172.21.15.12:0/3477393612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa568103b50 0x7fa56810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:30.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.356+0000 7fa56d984640 1 --2- 172.21.15.12:0/3477393612 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa568103190 0x7fa568103610 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:30.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.356+0000 7fa56d984640 1 --2- 172.21.15.12:0/3477393612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa568101fa0 0x7fa5681023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:30.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.356+0000 7fa56d984640 1 -- 172.21.15.12:0/3477393612 >> 172.21.15.12:0/3477393612 conn(0x7fa5680fd530 msgr2=0x7fa5680ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:30.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.356+0000 7fa56d984640 1 -- 172.21.15.12:0/3477393612 shutdown_connections 2024-04-22T13:21:30.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.356+0000 7fa56d984640 1 -- 172.21.15.12:0/3477393612 wait complete. 2024-04-22T13:21:30.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.356+0000 7fa56d984640 1 Processor -- start 2024-04-22T13:21:30.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.357+0000 7fa56d984640 1 -- start start 2024-04-22T13:21:30.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.357+0000 7fa56d984640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa568101fa0 0x7fa568075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:30.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.357+0000 7fa56d984640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa568103190 0x7fa568074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:30.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.357+0000 7fa56d984640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa568103b50 0x7fa568074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:30.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.357+0000 7fa56d984640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa56810db30 con 0x7fa568103b50 2024-04-22T13:21:30.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.357+0000 7fa56d984640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa56810d9b0 con 0x7fa568101fa0 2024-04-22T13:21:30.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.357+0000 7fa56d984640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa56810dcb0 con 0x7fa568103190 2024-04-22T13:21:30.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.357+0000 7fa5677fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa568103b50 0x7fa568074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:30.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.357+0000 7fa5677fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa568103b50 0x7fa568074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39850/0 (socket says 172.21.15.12:39850) 2024-04-22T13:21:30.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.357+0000 7fa5677fe640 1 -- 172.21.15.12:0/558626059 learned_addr learned my addr 172.21.15.12:0/558626059 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:21:30.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.357+0000 7fa566ffd640 1 --2- 172.21.15.12:0/558626059 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa568101fa0 0x7fa568075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:30.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.357+0000 7fa5667fc640 1 --2- 172.21.15.12:0/558626059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa568103190 0x7fa568074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:30.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.357+0000 7fa5677fe640 1 -- 172.21.15.12:0/558626059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa568103190 msgr2=0x7fa568074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:30.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.357+0000 7fa5677fe640 1 --2- 172.21.15.12:0/558626059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa568103190 0x7fa568074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:30.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.357+0000 7fa5677fe640 1 -- 172.21.15.12:0/558626059 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa568101fa0 msgr2=0x7fa568075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:30.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.357+0000 7fa5677fe640 1 --2- 172.21.15.12:0/558626059 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa568101fa0 0x7fa568075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:30.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.357+0000 7fa5677fe640 1 -- 172.21.15.12:0/558626059 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa568075110 con 0x7fa568103b50 2024-04-22T13:21:30.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.357+0000 7fa5677fe640 1 --2- 172.21.15.12:0/558626059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa568103b50 0x7fa568074880 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa56000cad0 tx=0x7fa56000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:30.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.358+0000 7fa56c982640 1 -- 172.21.15.12:0/558626059 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56000ae20 con 0x7fa568103b50 2024-04-22T13:21:30.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.358+0000 7fa56c982640 1 -- 172.21.15.12:0/558626059 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa560005020 con 0x7fa568103b50 2024-04-22T13:21:30.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.358+0000 7fa56d984640 1 -- 172.21.15.12:0/558626059 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5681a22e0 con 0x7fa568103b50 2024-04-22T13:21:30.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.358+0000 7fa56c982640 1 -- 172.21.15.12:0/558626059 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56000f6b0 con 0x7fa568103b50 2024-04-22T13:21:30.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.358+0000 7fa56d984640 1 -- 172.21.15.12:0/558626059 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5681a2800 con 0x7fa568103b50 2024-04-22T13:21:30.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.359+0000 7fa56d984640 1 -- 172.21.15.12:0/558626059 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa568068de0 con 0x7fa568103b50 2024-04-22T13:21:30.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.360+0000 7fa56c982640 1 -- 172.21.15.12:0/558626059 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa5600051d0 con 0x7fa568103b50 2024-04-22T13:21:30.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.360+0000 7fa56c982640 1 --2- 172.21.15.12:0/558626059 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa5500a3bb0 0x7fa5500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:30.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.361+0000 7fa566ffd640 1 --2- 172.21.15.12:0/558626059 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa5500a3bb0 0x7fa5500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:30.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.361+0000 7fa56c982640 1 -- 172.21.15.12:0/558626059 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fa5600cff90 con 0x7fa568103b50 2024-04-22T13:21:30.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.361+0000 7fa566ffd640 1 --2- 172.21.15.12:0/558626059 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa5500a3bb0 0x7fa5500a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa558005fd0 tx=0x7fa55800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:30.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.365+0000 7fa56c982640 1 -- 172.21.15.12:0/558626059 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa560014030 con 0x7fa568103b50 2024-04-22T13:21:30.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:30 smithi134 ceph-mon[31338]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:30.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:30 smithi134 ceph-mon[31338]: mds.? [v2:172.21.15.134:6832/4235830781,v1:172.21.15.134:6833/4235830781] up:boot 2024-04-22T13:21:30.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:30 smithi134 ceph-mon[31338]: fsmap 1 up:standby 2024-04-22T13:21:30.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:30 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-22T13:21:30.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:30 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:30.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:30 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:30.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:30 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:30.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:30 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:30.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:30 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:30.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:30 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:30.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:30 smithi012 ceph-mon[28265]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:30.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:30 smithi012 ceph-mon[28265]: mds.? [v2:172.21.15.134:6832/4235830781,v1:172.21.15.134:6833/4235830781] up:boot 2024-04-22T13:21:30.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:30 smithi012 ceph-mon[28265]: fsmap 1 up:standby 2024-04-22T13:21:30.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:30 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-22T13:21:30.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:30 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:30.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:30 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:30.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:30 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:30.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:30 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:30.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:30 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:30.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:30 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:30.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.591+0000 7fa56d984640 1 -- 172.21.15.12:0/558626059 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa56802d5c0 con 0x7fa568103b50 2024-04-22T13:21:30.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.592+0000 7fa56c982640 1 -- 172.21.15.12:0/558626059 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fa56009ce10 con 0x7fa568103b50 2024-04-22T13:21:30.590 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:21:30.590 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-22T13:21:30.591 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 2 2024-04-22T13:21:30.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.594+0000 7fa56d984640 1 -- 172.21.15.12:0/558626059 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa5500a3bb0 msgr2=0x7fa5500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:30.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.594+0000 7fa56d984640 1 --2- 172.21.15.12:0/558626059 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa5500a3bb0 0x7fa5500a6070 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa558005fd0 tx=0x7fa55800b410 comp rx=0 tx=0).stop 2024-04-22T13:21:30.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.594+0000 7fa56d984640 1 -- 172.21.15.12:0/558626059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa568103b50 msgr2=0x7fa568074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:30.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.594+0000 7fa56d984640 1 --2- 172.21.15.12:0/558626059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa568103b50 0x7fa568074880 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa56000cad0 tx=0x7fa56000cf90 comp rx=0 tx=0).stop 2024-04-22T13:21:30.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.594+0000 7fa56d984640 1 -- 172.21.15.12:0/558626059 shutdown_connections 2024-04-22T13:21:30.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.594+0000 7fa56d984640 1 --2- 172.21.15.12:0/558626059 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa5500a3bb0 0x7fa5500a6070 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:30.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.594+0000 7fa56d984640 1 --2- 172.21.15.12:0/558626059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa568103b50 0x7fa568074880 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:30.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.594+0000 7fa56d984640 1 --2- 172.21.15.12:0/558626059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa568103190 0x7fa568074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:30.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.594+0000 7fa56d984640 1 --2- 172.21.15.12:0/558626059 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa568101fa0 0x7fa568075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:30.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.594+0000 7fa56d984640 1 -- 172.21.15.12:0/558626059 >> 172.21.15.12:0/558626059 conn(0x7fa5680fd530 msgr2=0x7fa568106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:30.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.594+0000 7fa56d984640 1 -- 172.21.15.12:0/558626059 shutdown_connections 2024-04-22T13:21:30.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:30.594+0000 7fa56d984640 1 -- 172.21.15.12:0/558626059 wait complete. 2024-04-22T13:21:31.376 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:31 smithi111 ceph-mon[31361]: Deploying daemon mds.k on smithi111 2024-04-22T13:21:31.376 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:31 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/558626059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:31.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:31 smithi134 ceph-mon[31338]: Deploying daemon mds.k on smithi111 2024-04-22T13:21:31.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:31 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/558626059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:31.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:31 smithi012 ceph-mon[28265]: Deploying daemon mds.k on smithi111 2024-04-22T13:21:31.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:31 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/558626059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:31.728 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 22 13:21:31 smithi111 systemd[1]: Starting Ceph mds.k for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:21:32.047 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 22 13:21:31 smithi111 podman[61228]: 2024-04-22 13:21:31.681886026 +0000 UTC m=+0.018167391 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:21:32.047 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 22 13:21:31 smithi111 podman[61228]: 2024-04-22 13:21:31.781372925 +0000 UTC m=+0.117654289 container create 363efbebf019617358b97ce9e3fb7b339e9262b6e82851d8ca37f80787cfe553 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-22T13:21:32.047 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 22 13:21:32 smithi111 podman[61228]: 2024-04-22 13:21:32.025328169 +0000 UTC m=+0.361609537 container init 363efbebf019617358b97ce9e3fb7b339e9262b6e82851d8ca37f80787cfe553 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-22T13:21:32.047 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 22 13:21:32 smithi111 podman[61228]: 2024-04-22 13:21:32.027956546 +0000 UTC m=+0.364237920 container start 363efbebf019617358b97ce9e3fb7b339e9262b6e82851d8ca37f80787cfe553 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-k, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-04-22T13:21:32.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:32 smithi134 ceph-mon[31338]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:32.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:32 smithi111 ceph-mon[31361]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:32.477 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 22 13:21:32 smithi111 ceph-mds[61247]: -- 172.21.15.111:0/2283822147 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561daeef9a40 con 0x561daeecd000 2024-04-22T13:21:32.478 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 22 13:21:32 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-k[61243]: starting mds.k at 2024-04-22T13:21:32.478 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 22 13:21:32 smithi111 bash[61228]: 363efbebf019617358b97ce9e3fb7b339e9262b6e82851d8ca37f80787cfe553 2024-04-22T13:21:32.478 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 22 13:21:32 smithi111 systemd[1]: Started Ceph mds.k for 0aede5ea-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:21:32.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:32 smithi012 ceph-mon[28265]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:32.603 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:21:32.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.712+0000 7fe5d212b640 1 -- 172.21.15.12:0/3801658948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5cc103190 msgr2=0x7fe5cc103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:32.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.712+0000 7fe5d212b640 1 --2- 172.21.15.12:0/3801658948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5cc103190 0x7fe5cc103610 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0009900 tx=0x7fe5c002bc30 comp rx=0 tx=0).stop 2024-04-22T13:21:32.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.712+0000 7fe5d212b640 1 -- 172.21.15.12:0/3801658948 shutdown_connections 2024-04-22T13:21:32.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.712+0000 7fe5d212b640 1 --2- 172.21.15.12:0/3801658948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5cc103b50 0x7fe5cc10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:32.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.712+0000 7fe5d212b640 1 --2- 172.21.15.12:0/3801658948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5cc103190 0x7fe5cc103610 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:32.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.712+0000 7fe5d212b640 1 --2- 172.21.15.12:0/3801658948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5cc101fa0 0x7fe5cc1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:32.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.712+0000 7fe5d212b640 1 -- 172.21.15.12:0/3801658948 >> 172.21.15.12:0/3801658948 conn(0x7fe5cc0fd530 msgr2=0x7fe5cc0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:32.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.712+0000 7fe5d212b640 1 -- 172.21.15.12:0/3801658948 shutdown_connections 2024-04-22T13:21:32.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.712+0000 7fe5d212b640 1 -- 172.21.15.12:0/3801658948 wait complete. 2024-04-22T13:21:32.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.712+0000 7fe5d212b640 1 Processor -- start 2024-04-22T13:21:32.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.713+0000 7fe5d212b640 1 -- start start 2024-04-22T13:21:32.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.713+0000 7fe5d212b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5cc101fa0 0x7fe5cc198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:32.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.713+0000 7fe5d212b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5cc103190 0x7fe5cc1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:32.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.713+0000 7fe5d212b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5cc103b50 0x7fe5cc19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:32.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.713+0000 7fe5d212b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe5cc10dae0 con 0x7fe5cc103b50 2024-04-22T13:21:32.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.713+0000 7fe5d212b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe5cc10d960 con 0x7fe5cc101fa0 2024-04-22T13:21:32.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.713+0000 7fe5d212b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe5cc10dc60 con 0x7fe5cc103190 2024-04-22T13:21:32.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.713+0000 7fe5cbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5cc103b50 0x7fe5cc19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:32.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.713+0000 7fe5cb7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5cc101fa0 0x7fe5cc198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:32.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.713+0000 7fe5cbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5cc103b50 0x7fe5cc19f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41720/0 (socket says 172.21.15.12:41720) 2024-04-22T13:21:32.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.713+0000 7fe5cbfff640 1 -- 172.21.15.12:0/455067256 learned_addr learned my addr 172.21.15.12:0/455067256 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:21:32.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.713+0000 7fe5caffd640 1 --2- 172.21.15.12:0/455067256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5cc103190 0x7fe5cc1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:32.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.713+0000 7fe5cbfff640 1 -- 172.21.15.12:0/455067256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5cc103190 msgr2=0x7fe5cc1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:32.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.713+0000 7fe5cbfff640 1 --2- 172.21.15.12:0/455067256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5cc103190 0x7fe5cc1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:32.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.713+0000 7fe5cbfff640 1 -- 172.21.15.12:0/455067256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5cc101fa0 msgr2=0x7fe5cc198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:32.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.713+0000 7fe5cbfff640 1 --2- 172.21.15.12:0/455067256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5cc101fa0 0x7fe5cc198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:32.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.713+0000 7fe5cbfff640 1 -- 172.21.15.12:0/455067256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5cc19fd90 con 0x7fe5cc103b50 2024-04-22T13:21:32.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.713+0000 7fe5cb7fe640 1 --2- 172.21.15.12:0/455067256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5cc101fa0 0x7fe5cc198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:21:32.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.713+0000 7fe5caffd640 1 --2- 172.21.15.12:0/455067256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5cc103190 0x7fe5cc1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:21:32.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.713+0000 7fe5cbfff640 1 --2- 172.21.15.12:0/455067256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5cc103b50 0x7fe5cc19f720 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe5c400e970 tx=0x7fe5c400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:32.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.714+0000 7fe5c8ff9640 1 -- 172.21.15.12:0/455067256 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c400ccc0 con 0x7fe5cc103b50 2024-04-22T13:21:32.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.714+0000 7fe5d212b640 1 -- 172.21.15.12:0/455067256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5cc1a0080 con 0x7fe5cc103b50 2024-04-22T13:21:32.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.714+0000 7fe5c8ff9640 1 -- 172.21.15.12:0/455067256 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5c400ce60 con 0x7fe5cc103b50 2024-04-22T13:21:32.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.714+0000 7fe5c8ff9640 1 -- 172.21.15.12:0/455067256 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4010430 con 0x7fe5cc103b50 2024-04-22T13:21:32.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.714+0000 7fe5d212b640 1 -- 172.21.15.12:0/455067256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5cc1a05d0 con 0x7fe5cc103b50 2024-04-22T13:21:32.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.715+0000 7fe5d212b640 1 -- 172.21.15.12:0/455067256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5cc109360 con 0x7fe5cc103b50 2024-04-22T13:21:32.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.716+0000 7fe5c8ff9640 1 -- 172.21.15.12:0/455067256 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe5c4004fb0 con 0x7fe5cc103b50 2024-04-22T13:21:32.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.716+0000 7fe5c8ff9640 1 --2- 172.21.15.12:0/455067256 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe5b40a3bb0 0x7fe5b40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:32.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.717+0000 7fe5c8ff9640 1 -- 172.21.15.12:0/455067256 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fe5c40d0600 con 0x7fe5cc103b50 2024-04-22T13:21:32.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.717+0000 7fe5cb7fe640 1 --2- 172.21.15.12:0/455067256 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe5b40a3bb0 0x7fe5b40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:32.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.717+0000 7fe5cb7fe640 1 --2- 172.21.15.12:0/455067256 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe5b40a3bb0 0x7fe5b40a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc102ff0 tx=0x7fe5bc009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:32.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.720+0000 7fe5c8ff9640 1 -- 172.21.15.12:0/455067256 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe5c4053ff0 con 0x7fe5cc103b50 2024-04-22T13:21:32.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.955+0000 7fe5d212b640 1 -- 172.21.15.12:0/455067256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5cc02d5c0 con 0x7fe5cc103b50 2024-04-22T13:21:32.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.955+0000 7fe5c8ff9640 1 -- 172.21.15.12:0/455067256 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7fe5c409d480 con 0x7fe5cc103b50 2024-04-22T13:21:32.952 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:21:32.952 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-22T13:21:32.953 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 3 2024-04-22T13:21:32.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.958+0000 7fe5d212b640 1 -- 172.21.15.12:0/455067256 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe5b40a3bb0 msgr2=0x7fe5b40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:32.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.958+0000 7fe5d212b640 1 --2- 172.21.15.12:0/455067256 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe5b40a3bb0 0x7fe5b40a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc102ff0 tx=0x7fe5bc009670 comp rx=0 tx=0).stop 2024-04-22T13:21:32.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.958+0000 7fe5d212b640 1 -- 172.21.15.12:0/455067256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5cc103b50 msgr2=0x7fe5cc19f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:32.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.958+0000 7fe5d212b640 1 --2- 172.21.15.12:0/455067256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5cc103b50 0x7fe5cc19f720 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe5c400e970 tx=0x7fe5c400ed70 comp rx=0 tx=0).stop 2024-04-22T13:21:32.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.958+0000 7fe5d212b640 1 -- 172.21.15.12:0/455067256 shutdown_connections 2024-04-22T13:21:32.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.958+0000 7fe5d212b640 1 --2- 172.21.15.12:0/455067256 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe5b40a3bb0 0x7fe5b40a6070 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:32.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.958+0000 7fe5d212b640 1 --2- 172.21.15.12:0/455067256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe5cc103b50 0x7fe5cc19f720 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:32.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.958+0000 7fe5d212b640 1 --2- 172.21.15.12:0/455067256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5cc103190 0x7fe5cc1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:32.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.958+0000 7fe5d212b640 1 --2- 172.21.15.12:0/455067256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5cc101fa0 0x7fe5cc198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:32.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.958+0000 7fe5d212b640 1 -- 172.21.15.12:0/455067256 >> 172.21.15.12:0/455067256 conn(0x7fe5cc0fd530 msgr2=0x7fe5cc106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:32.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.958+0000 7fe5d212b640 1 -- 172.21.15.12:0/455067256 shutdown_connections 2024-04-22T13:21:32.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:32.958+0000 7fe5d212b640 1 -- 172.21.15.12:0/455067256 wait complete. 2024-04-22T13:21:33.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:33 smithi134 ceph-mon[31338]: mds.? [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] up:boot 2024-04-22T13:21:33.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:33 smithi134 ceph-mon[31338]: fsmap 2 up:standby 2024-04-22T13:21:33.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:33 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-22T13:21:33.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:33 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:33.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:33 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:33.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:33 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:33.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:33 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:33.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:33 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:33.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:33 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:33.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:33 smithi134 ceph-mon[31338]: Deploying daemon mds.j on smithi012 2024-04-22T13:21:33.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:33 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/455067256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:33.433 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:33 smithi111 ceph-mon[31361]: mds.? [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] up:boot 2024-04-22T13:21:33.433 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:33 smithi111 ceph-mon[31361]: fsmap 2 up:standby 2024-04-22T13:21:33.433 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:33 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-22T13:21:33.433 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:33 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:33.433 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:33 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:33.433 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:33 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:33.433 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:33 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:33.434 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:33 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:33.434 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:33 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:33.434 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:33 smithi111 ceph-mon[31361]: Deploying daemon mds.j on smithi012 2024-04-22T13:21:33.434 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:33 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/455067256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:33.541 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:33 smithi012 ceph-mon[28265]: mds.? [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] up:boot 2024-04-22T13:21:33.541 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:33 smithi012 ceph-mon[28265]: fsmap 2 up:standby 2024-04-22T13:21:33.541 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:33 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-22T13:21:33.542 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:33 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:33.542 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:33 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:33.542 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:33 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:33.542 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:33 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:33.542 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:33 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:33.542 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:33 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:33.542 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:33 smithi012 ceph-mon[28265]: Deploying daemon mds.j on smithi012 2024-04-22T13:21:33.542 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:33 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/455067256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:34.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:34 smithi111 ceph-mon[31361]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:34.560 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:34 smithi012 ceph-mon[28265]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:34.560 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 22 13:21:34 smithi012 systemd[1]: Starting Ceph mds.j for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:21:34.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:34 smithi134 ceph-mon[31338]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:34.824 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 22 13:21:34 smithi012 podman[69023]: 2024-04-22 13:21:34.4672187 +0000 UTC m=+0.018608171 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:21:34.824 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 22 13:21:34 smithi012 podman[69023]: 2024-04-22 13:21:34.564750463 +0000 UTC m=+0.116139931 container create 4322eb25c6620ba43d35597f19a3b3ede7e464016affd1e04f6b6d8f2fa5a38c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-j, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-04-22T13:21:34.824 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 22 13:21:34 smithi012 podman[69023]: 2024-04-22 13:21:34.800507041 +0000 UTC m=+0.351896501 container init 4322eb25c6620ba43d35597f19a3b3ede7e464016affd1e04f6b6d8f2fa5a38c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-j, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-22T13:21:34.824 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 22 13:21:34 smithi012 podman[69023]: 2024-04-22 13:21:34.803249656 +0000 UTC m=+0.354639122 container start 4322eb25c6620ba43d35597f19a3b3ede7e464016affd1e04f6b6d8f2fa5a38c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-j, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-22T13:21:34.824 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 22 13:21:34 smithi012 ceph-mds[69043]: -- 172.21.15.12:0/3465375018 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5637c3a8da40 con 0x5637c3a60800 2024-04-22T13:21:34.966 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:21:35.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.076+0000 7f6b9da48640 1 -- 172.21.15.12:0/2645143045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b98101fc0 msgr2=0x7f6b98102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:35.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.076+0000 7f6b9da48640 1 --2- 172.21.15.12:0/2645143045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b98101fc0 0x7f6b98102440 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6b88009900 tx=0x7f6b8802bc30 comp rx=0 tx=0).stop 2024-04-22T13:21:35.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.076+0000 7f6b9da48640 1 -- 172.21.15.12:0/2645143045 shutdown_connections 2024-04-22T13:21:35.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.076+0000 7f6b9da48640 1 --2- 172.21.15.12:0/2645143045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b98102980 0x7f6b981092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:35.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.076+0000 7f6b9da48640 1 --2- 172.21.15.12:0/2645143045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b98101fc0 0x7f6b98102440 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:35.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.076+0000 7f6b9da48640 1 --2- 172.21.15.12:0/2645143045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b98100e70 0x7f6b98101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:35.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.076+0000 7f6b9da48640 1 -- 172.21.15.12:0/2645143045 >> 172.21.15.12:0/2645143045 conn(0x7f6b980fc3e0 msgr2=0x7f6b980fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:35.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.076+0000 7f6b9da48640 1 -- 172.21.15.12:0/2645143045 shutdown_connections 2024-04-22T13:21:35.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.076+0000 7f6b9da48640 1 -- 172.21.15.12:0/2645143045 wait complete. 2024-04-22T13:21:35.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.077+0000 7f6b9da48640 1 Processor -- start 2024-04-22T13:21:35.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.077+0000 7f6b9da48640 1 -- start start 2024-04-22T13:21:35.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.077+0000 7f6b9da48640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b98100e70 0x7f6b981970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:35.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.077+0000 7f6b9da48640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b98101fc0 0x7f6b981975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:35.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.077+0000 7f6b9da48640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b98102980 0x7f6b9819e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:35.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.077+0000 7f6b9da48640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9810c9a0 con 0x7f6b98102980 2024-04-22T13:21:35.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.077+0000 7f6b9da48640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9810c820 con 0x7f6b98100e70 2024-04-22T13:21:35.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.077+0000 7f6b9da48640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9810cb20 con 0x7f6b98101fc0 2024-04-22T13:21:35.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.077+0000 7f6b977fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b98102980 0x7f6b9819e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:35.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.077+0000 7f6b977fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b98102980 0x7f6b9819e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41762/0 (socket says 172.21.15.12:41762) 2024-04-22T13:21:35.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.077+0000 7f6b977fe640 1 -- 172.21.15.12:0/3606679722 learned_addr learned my addr 172.21.15.12:0/3606679722 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:21:35.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.078+0000 7f6b96ffd640 1 --2- 172.21.15.12:0/3606679722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b98100e70 0x7f6b981970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:35.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.078+0000 7f6b967fc640 1 --2- 172.21.15.12:0/3606679722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b98101fc0 0x7f6b981975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:35.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.078+0000 7f6b977fe640 1 -- 172.21.15.12:0/3606679722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b98101fc0 msgr2=0x7f6b981975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:35.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.078+0000 7f6b977fe640 1 --2- 172.21.15.12:0/3606679722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b98101fc0 0x7f6b981975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:35.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.078+0000 7f6b977fe640 1 -- 172.21.15.12:0/3606679722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b98100e70 msgr2=0x7f6b981970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:35.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.078+0000 7f6b977fe640 1 --2- 172.21.15.12:0/3606679722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b98100e70 0x7f6b981970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:35.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.078+0000 7f6b977fe640 1 -- 172.21.15.12:0/3606679722 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9819ecc0 con 0x7f6b98102980 2024-04-22T13:21:35.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.078+0000 7f6b967fc640 1 --2- 172.21.15.12:0/3606679722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b98101fc0 0x7f6b981975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:21:35.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.078+0000 7f6b977fe640 1 --2- 172.21.15.12:0/3606679722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b98102980 0x7f6b9819e650 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6b9000eaa0 tx=0x7f6b9000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:35.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.078+0000 7f6b9ca46640 1 -- 172.21.15.12:0/3606679722 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9000cdb0 con 0x7f6b98102980 2024-04-22T13:21:35.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.078+0000 7f6b9da48640 1 -- 172.21.15.12:0/3606679722 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b9819efb0 con 0x7f6b98102980 2024-04-22T13:21:35.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.078+0000 7f6b9ca46640 1 -- 172.21.15.12:0/3606679722 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b90005020 con 0x7f6b98102980 2024-04-22T13:21:35.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.078+0000 7f6b9ca46640 1 -- 172.21.15.12:0/3606679722 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b900116d0 con 0x7f6b98102980 2024-04-22T13:21:35.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.078+0000 7f6b9da48640 1 -- 172.21.15.12:0/3606679722 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b9819f470 con 0x7f6b98102980 2024-04-22T13:21:35.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.080+0000 7f6b9da48640 1 -- 172.21.15.12:0/3606679722 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b98108250 con 0x7f6b98102980 2024-04-22T13:21:35.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.080+0000 7f6b9ca46640 1 -- 172.21.15.12:0/3606679722 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6b900051d0 con 0x7f6b98102980 2024-04-22T13:21:35.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.081+0000 7f6b9ca46640 1 --2- 172.21.15.12:0/3606679722 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f6b800a3bb0 0x7f6b800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:35.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.081+0000 7f6b96ffd640 1 --2- 172.21.15.12:0/3606679722 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f6b800a3bb0 0x7f6b800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:35.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.081+0000 7f6b9ca46640 1 -- 172.21.15.12:0/3606679722 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f6b900d0160 con 0x7f6b98102980 2024-04-22T13:21:35.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.081+0000 7f6b96ffd640 1 --2- 172.21.15.12:0/3606679722 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f6b800a3bb0 0x7f6b800a6070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00bbe0 tx=0x7f6b8c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:35.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.090+0000 7f6b9ca46640 1 -- 172.21.15.12:0/3606679722 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6b90016030 con 0x7f6b98102980 2024-04-22T13:21:35.087 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 22 13:21:34 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-j[69039]: starting mds.j at 2024-04-22T13:21:35.087 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 22 13:21:34 smithi012 bash[69023]: 4322eb25c6620ba43d35597f19a3b3ede7e464016affd1e04f6b6d8f2fa5a38c 2024-04-22T13:21:35.087 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 22 13:21:34 smithi012 systemd[1]: Started Ceph mds.j for 0aede5ea-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:21:35.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.321+0000 7f6b9da48640 1 -- 172.21.15.12:0/3606679722 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b9802d770 con 0x7f6b98102980 2024-04-22T13:21:35.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.321+0000 7f6b9ca46640 1 -- 172.21.15.12:0/3606679722 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7f6b9009cfe0 con 0x7f6b98102980 2024-04-22T13:21:35.319 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:21:35.319 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-22T13:21:35.319 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 4 2024-04-22T13:21:35.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.323+0000 7f6b9da48640 1 -- 172.21.15.12:0/3606679722 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f6b800a3bb0 msgr2=0x7f6b800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:35.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.323+0000 7f6b9da48640 1 --2- 172.21.15.12:0/3606679722 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f6b800a3bb0 0x7f6b800a6070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00bbe0 tx=0x7f6b8c00b390 comp rx=0 tx=0).stop 2024-04-22T13:21:35.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.323+0000 7f6b9da48640 1 -- 172.21.15.12:0/3606679722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b98102980 msgr2=0x7f6b9819e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:35.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.323+0000 7f6b9da48640 1 --2- 172.21.15.12:0/3606679722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b98102980 0x7f6b9819e650 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6b9000eaa0 tx=0x7f6b9000ef60 comp rx=0 tx=0).stop 2024-04-22T13:21:35.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.323+0000 7f6b9da48640 1 -- 172.21.15.12:0/3606679722 shutdown_connections 2024-04-22T13:21:35.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.323+0000 7f6b9da48640 1 --2- 172.21.15.12:0/3606679722 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f6b800a3bb0 0x7f6b800a6070 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:35.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.323+0000 7f6b9da48640 1 --2- 172.21.15.12:0/3606679722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b98102980 0x7f6b9819e650 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:35.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.323+0000 7f6b9da48640 1 --2- 172.21.15.12:0/3606679722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b98101fc0 0x7f6b981975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:35.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.323+0000 7f6b9da48640 1 --2- 172.21.15.12:0/3606679722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b98100e70 0x7f6b981970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:35.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.323+0000 7f6b9da48640 1 -- 172.21.15.12:0/3606679722 >> 172.21.15.12:0/3606679722 conn(0x7f6b980fc3e0 msgr2=0x7f6b98105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:35.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.324+0000 7f6b9da48640 1 -- 172.21.15.12:0/3606679722 shutdown_connections 2024-04-22T13:21:35.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:35.324+0000 7f6b9da48640 1 -- 172.21.15.12:0/3606679722 wait complete. 2024-04-22T13:21:36.190 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:35 smithi134 ceph-mon[31338]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:36.190 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:35 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:36.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:35 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:36.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:35 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:36.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:35 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:36.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:35 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:36.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:35 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:36.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:35 smithi134 ceph-mon[31338]: Deploying daemon mds.d on smithi012 2024-04-22T13:21:36.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:35 smithi134 ceph-mon[31338]: mds.? [v2:172.21.15.12:6834/1590002838,v1:172.21.15.12:6835/1590002838] up:boot 2024-04-22T13:21:36.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:35 smithi134 ceph-mon[31338]: fsmap 3 up:standby 2024-04-22T13:21:36.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:35 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-22T13:21:36.191 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:35 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/3606679722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:36.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:35 smithi111 ceph-mon[31361]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:36.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:35 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:36.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:35 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:36.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:35 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:36.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:35 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:36.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:35 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:36.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:35 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:36.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:35 smithi111 ceph-mon[31361]: Deploying daemon mds.d on smithi012 2024-04-22T13:21:36.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:35 smithi111 ceph-mon[31361]: mds.? [v2:172.21.15.12:6834/1590002838,v1:172.21.15.12:6835/1590002838] up:boot 2024-04-22T13:21:36.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:35 smithi111 ceph-mon[31361]: fsmap 3 up:standby 2024-04-22T13:21:36.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:35 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-22T13:21:36.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:35 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/3606679722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:36.246 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:35 smithi012 ceph-mon[28265]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:36.246 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:35 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:36.246 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:35 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:36.246 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:35 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:36.246 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:35 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:36.246 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:35 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:36.246 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:35 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:36.247 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:35 smithi012 ceph-mon[28265]: Deploying daemon mds.d on smithi012 2024-04-22T13:21:36.247 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:35 smithi012 ceph-mon[28265]: mds.? [v2:172.21.15.12:6834/1590002838,v1:172.21.15.12:6835/1590002838] up:boot 2024-04-22T13:21:36.247 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:35 smithi012 ceph-mon[28265]: fsmap 3 up:standby 2024-04-22T13:21:36.247 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:35 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-22T13:21:36.247 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:35 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3606679722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:37.280 INFO:journalctl@ceph.mds.d.smithi012.stdout:Apr 22 13:21:37 smithi012 systemd[1]: Starting Ceph mds.d for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:21:37.333 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:21:37.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.446+0000 7fb6fe4c0640 1 -- 172.21.15.12:0/3603362426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6f8107cb0 msgr2=0x7fb6f810a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:37.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.446+0000 7fb6fe4c0640 1 --2- 172.21.15.12:0/3603362426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6f8107cb0 0x7fb6f810a0a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0009ca0 tx=0x7fb6f002c120 comp rx=0 tx=0).stop 2024-04-22T13:21:37.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.446+0000 7fb6fe4c0640 1 -- 172.21.15.12:0/3603362426 shutdown_connections 2024-04-22T13:21:37.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.446+0000 7fb6fe4c0640 1 --2- 172.21.15.12:0/3603362426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6f8107cb0 0x7fb6f810a0a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:37.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.446+0000 7fb6fe4c0640 1 --2- 172.21.15.12:0/3603362426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6f8105380 0x7fb6f8107770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:37.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.446+0000 7fb6fe4c0640 1 --2- 172.21.15.12:0/3603362426 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6f8102a50 0x7fb6f8104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:37.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.446+0000 7fb6fe4c0640 1 -- 172.21.15.12:0/3603362426 >> 172.21.15.12:0/3603362426 conn(0x7fb6f80fc3e0 msgr2=0x7fb6f80fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:37.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.446+0000 7fb6fe4c0640 1 -- 172.21.15.12:0/3603362426 shutdown_connections 2024-04-22T13:21:37.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.446+0000 7fb6fe4c0640 1 -- 172.21.15.12:0/3603362426 wait complete. 2024-04-22T13:21:37.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.446+0000 7fb6fe4c0640 1 Processor -- start 2024-04-22T13:21:37.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.446+0000 7fb6fe4c0640 1 -- start start 2024-04-22T13:21:37.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.447+0000 7fb6fe4c0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6f8102a50 0x7fb6f8102370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:37.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.447+0000 7fb6fe4c0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6f8105380 0x7fb6f8100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:37.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.447+0000 7fb6fe4c0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6f8107cb0 0x7fb6f8100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:37.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.447+0000 7fb6fe4c0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f810db10 con 0x7fb6f8102a50 2024-04-22T13:21:37.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.447+0000 7fb6fe4c0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f810d990 con 0x7fb6f8107cb0 2024-04-22T13:21:37.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.447+0000 7fb6fe4c0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f810dc90 con 0x7fb6f8105380 2024-04-22T13:21:37.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.447+0000 7fb6f7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6f8102a50 0x7fb6f8102370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:37.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.447+0000 7fb6f7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6f8102a50 0x7fb6f8102370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41778/0 (socket says 172.21.15.12:41778) 2024-04-22T13:21:37.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.447+0000 7fb6f7fff640 1 -- 172.21.15.12:0/3326724356 learned_addr learned my addr 172.21.15.12:0/3326724356 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:21:37.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.447+0000 7fb6fca5d640 1 --2- 172.21.15.12:0/3326724356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6f8107cb0 0x7fb6f8100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:37.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.447+0000 7fb6f7fff640 1 -- 172.21.15.12:0/3326724356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6f8105380 msgr2=0x7fb6f8100a00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-22T13:21:37.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.447+0000 7fb6f7fff640 1 --2- 172.21.15.12:0/3326724356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6f8105380 0x7fb6f8100a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:37.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.447+0000 7fb6f7fff640 1 -- 172.21.15.12:0/3326724356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6f8107cb0 msgr2=0x7fb6f8100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:37.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.447+0000 7fb6f7fff640 1 --2- 172.21.15.12:0/3326724356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6f8107cb0 0x7fb6f8100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:37.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.447+0000 7fb6f7fff640 1 -- 172.21.15.12:0/3326724356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f8101800 con 0x7fb6f8102a50 2024-04-22T13:21:37.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.447+0000 7fb6f7fff640 1 --2- 172.21.15.12:0/3326724356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6f8102a50 0x7fb6f8102370 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800d950 tx=0x7fb6e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:37.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.448+0000 7fb6f57fa640 1 -- 172.21.15.12:0/3326724356 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8017030 con 0x7fb6f8102a50 2024-04-22T13:21:37.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.448+0000 7fb6fe4c0640 1 -- 172.21.15.12:0/3326724356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f81a2340 con 0x7fb6f8102a50 2024-04-22T13:21:37.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.448+0000 7fb6f57fa640 1 -- 172.21.15.12:0/3326724356 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6e8004020 con 0x7fb6f8102a50 2024-04-22T13:21:37.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.448+0000 7fb6f57fa640 1 -- 172.21.15.12:0/3326724356 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e80047a0 con 0x7fb6f8102a50 2024-04-22T13:21:37.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.448+0000 7fb6fe4c0640 1 -- 172.21.15.12:0/3326724356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f81a28b0 con 0x7fb6f8102a50 2024-04-22T13:21:37.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.449+0000 7fb6fe4c0640 1 -- 172.21.15.12:0/3326724356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f80fded0 con 0x7fb6f8102a50 2024-04-22T13:21:37.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.450+0000 7fb6f57fa640 1 -- 172.21.15.12:0/3326724356 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb6e8009a30 con 0x7fb6f8102a50 2024-04-22T13:21:37.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.450+0000 7fb6f57fa640 1 --2- 172.21.15.12:0/3326724356 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb6e00a3bb0 0x7fb6e00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:37.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.451+0000 7fb6f77fe640 1 --2- 172.21.15.12:0/3326724356 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb6e00a3bb0 0x7fb6e00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:37.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.451+0000 7fb6f57fa640 1 -- 172.21.15.12:0/3326724356 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fb6e80cfa10 con 0x7fb6f8102a50 2024-04-22T13:21:37.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.451+0000 7fb6f77fe640 1 --2- 172.21.15.12:0/3326724356 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb6e00a3bb0 0x7fb6e00a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb6f81020e0 tx=0x7fb6ec006bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:37.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.455+0000 7fb6f57fa640 1 -- 172.21.15.12:0/3326724356 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb6e809c890 con 0x7fb6f8102a50 2024-04-22T13:21:37.587 INFO:journalctl@ceph.mds.d.smithi012.stdout:Apr 22 13:21:37 smithi012 podman[69367]: 2024-04-22 13:21:37.186949783 +0000 UTC m=+0.018383834 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:21:37.588 INFO:journalctl@ceph.mds.d.smithi012.stdout:Apr 22 13:21:37 smithi012 podman[69367]: 2024-04-22 13:21:37.285516838 +0000 UTC m=+0.116950893 container create b96099ca848db41e053410667ba075056942af1501b3173c097e07841d8804e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-22T13:21:37.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.684+0000 7fb6fe4c0640 1 -- 172.21.15.12:0/3326724356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6f802d5c0 con 0x7fb6f8102a50 2024-04-22T13:21:37.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.684+0000 7fb6f57fa640 1 -- 172.21.15.12:0/3326724356 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7fb6e809c190 con 0x7fb6f8102a50 2024-04-22T13:21:37.681 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:21:37.681 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-22T13:21:37.681 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 4 2024-04-22T13:21:37.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.686+0000 7fb6fe4c0640 1 -- 172.21.15.12:0/3326724356 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb6e00a3bb0 msgr2=0x7fb6e00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:37.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.686+0000 7fb6fe4c0640 1 --2- 172.21.15.12:0/3326724356 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb6e00a3bb0 0x7fb6e00a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb6f81020e0 tx=0x7fb6ec006bd0 comp rx=0 tx=0).stop 2024-04-22T13:21:37.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.686+0000 7fb6fe4c0640 1 -- 172.21.15.12:0/3326724356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6f8102a50 msgr2=0x7fb6f8102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:37.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.686+0000 7fb6fe4c0640 1 --2- 172.21.15.12:0/3326724356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6f8102a50 0x7fb6f8102370 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800d950 tx=0x7fb6e800de10 comp rx=0 tx=0).stop 2024-04-22T13:21:37.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.686+0000 7fb6fe4c0640 1 -- 172.21.15.12:0/3326724356 shutdown_connections 2024-04-22T13:21:37.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.686+0000 7fb6fe4c0640 1 --2- 172.21.15.12:0/3326724356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6f8107cb0 0x7fb6f8100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:37.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.686+0000 7fb6fe4c0640 1 --2- 172.21.15.12:0/3326724356 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6f8105380 0x7fb6f8100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:37.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.686+0000 7fb6fe4c0640 1 --2- 172.21.15.12:0/3326724356 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb6e00a3bb0 0x7fb6e00a6070 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:37.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.686+0000 7fb6fe4c0640 1 --2- 172.21.15.12:0/3326724356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb6f8102a50 0x7fb6f8102370 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:37.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.686+0000 7fb6fe4c0640 1 -- 172.21.15.12:0/3326724356 >> 172.21.15.12:0/3326724356 conn(0x7fb6f80fc3e0 msgr2=0x7fb6f8103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:37.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.687+0000 7fb6fe4c0640 1 -- 172.21.15.12:0/3326724356 shutdown_connections 2024-04-22T13:21:37.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:37.687+0000 7fb6fe4c0640 1 -- 172.21.15.12:0/3326724356 wait complete. 2024-04-22T13:21:37.934 INFO:journalctl@ceph.mds.d.smithi012.stdout:Apr 22 13:21:37 smithi012 podman[69367]: 2024-04-22 13:21:37.66509126 +0000 UTC m=+0.496525324 container init b96099ca848db41e053410667ba075056942af1501b3173c097e07841d8804e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T13:21:37.934 INFO:journalctl@ceph.mds.d.smithi012.stdout:Apr 22 13:21:37 smithi012 podman[69367]: 2024-04-22 13:21:37.667866049 +0000 UTC m=+0.499300102 container start b96099ca848db41e053410667ba075056942af1501b3173c097e07841d8804e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-22T13:21:37.934 INFO:journalctl@ceph.mds.d.smithi012.stdout:Apr 22 13:21:37 smithi012 ceph-mds[69425]: -- 172.21.15.12:0/1783174935 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558f96961a40 con 0x558f96934c00 2024-04-22T13:21:37.935 INFO:journalctl@ceph.mds.d.smithi012.stdout:Apr 22 13:21:37 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-d[69402]: starting mds.d at 2024-04-22T13:21:37.935 INFO:journalctl@ceph.mds.d.smithi012.stdout:Apr 22 13:21:37 smithi012 bash[69367]: b96099ca848db41e053410667ba075056942af1501b3173c097e07841d8804e6 2024-04-22T13:21:37.935 INFO:journalctl@ceph.mds.d.smithi012.stdout:Apr 22 13:21:37 smithi012 systemd[1]: Started Ceph mds.d for 0aede5ea-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:21:38.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:37 smithi111 ceph-mon[31361]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:38.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:37 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/3326724356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:38.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:37 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:38.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:37 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:38.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:37 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:38.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:37 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:38.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:37 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:38.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:37 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:38.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:37 smithi012 ceph-mon[28265]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:38.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:37 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3326724356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:38.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:37 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:38.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:37 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:38.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:37 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:38.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:37 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:38.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:37 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:38.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:37 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:38.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:37 smithi134 ceph-mon[31338]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:38.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:37 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/3326724356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:38.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:37 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:38.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:37 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:38.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:37 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:38.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:37 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:38.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:37 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:38.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:37 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:38.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:38 smithi111 ceph-mon[31361]: Deploying daemon mds.e on smithi111 2024-04-22T13:21:38.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:38 smithi111 ceph-mon[31361]: mds.? [v2:172.21.15.12:6836/3677153272,v1:172.21.15.12:6837/3677153272] up:boot 2024-04-22T13:21:38.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:38 smithi111 ceph-mon[31361]: fsmap 4 up:standby 2024-04-22T13:21:38.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:38 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-22T13:21:39.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:38 smithi012 ceph-mon[28265]: Deploying daemon mds.e on smithi111 2024-04-22T13:21:39.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:38 smithi012 ceph-mon[28265]: mds.? [v2:172.21.15.12:6836/3677153272,v1:172.21.15.12:6837/3677153272] up:boot 2024-04-22T13:21:39.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:38 smithi012 ceph-mon[28265]: fsmap 4 up:standby 2024-04-22T13:21:39.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:38 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-22T13:21:39.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:38 smithi134 ceph-mon[31338]: Deploying daemon mds.e on smithi111 2024-04-22T13:21:39.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:38 smithi134 ceph-mon[31338]: mds.? [v2:172.21.15.12:6836/3677153272,v1:172.21.15.12:6837/3677153272] up:boot 2024-04-22T13:21:39.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:38 smithi134 ceph-mon[31338]: fsmap 4 up:standby 2024-04-22T13:21:39.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:38 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-22T13:21:39.697 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:21:39.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.815+0000 7fe1e7474640 1 -- 172.21.15.12:0/2172771547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1e0103b50 msgr2=0x7fe1e010a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:39.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.815+0000 7fe1e7474640 1 --2- 172.21.15.12:0/2172771547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1e0103b50 0x7fe1e010a3e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8009ca0 tx=0x7fe1d802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:21:39.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.816+0000 7fe1e7474640 1 -- 172.21.15.12:0/2172771547 shutdown_connections 2024-04-22T13:21:39.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.816+0000 7fe1e7474640 1 --2- 172.21.15.12:0/2172771547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1e0103b50 0x7fe1e010a3e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:39.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.816+0000 7fe1e7474640 1 --2- 172.21.15.12:0/2172771547 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1e0103190 0x7fe1e0103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:39.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.816+0000 7fe1e7474640 1 --2- 172.21.15.12:0/2172771547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1e0101fa0 0x7fe1e01023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:39.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.816+0000 7fe1e7474640 1 -- 172.21.15.12:0/2172771547 >> 172.21.15.12:0/2172771547 conn(0x7fe1e00fd530 msgr2=0x7fe1e00ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:39.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.816+0000 7fe1e7474640 1 -- 172.21.15.12:0/2172771547 shutdown_connections 2024-04-22T13:21:39.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.816+0000 7fe1e7474640 1 -- 172.21.15.12:0/2172771547 wait complete. 2024-04-22T13:21:39.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.816+0000 7fe1e7474640 1 Processor -- start 2024-04-22T13:21:39.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.816+0000 7fe1e7474640 1 -- start start 2024-04-22T13:21:39.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.816+0000 7fe1e7474640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1e0101fa0 0x7fe1e0075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:39.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.816+0000 7fe1e7474640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1e0103190 0x7fe1e0074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:39.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.817+0000 7fe1e7474640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1e0103b50 0x7fe1e0074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:39.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.817+0000 7fe1e7474640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e010db40 con 0x7fe1e0103190 2024-04-22T13:21:39.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.817+0000 7fe1e7474640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e010d9c0 con 0x7fe1e0101fa0 2024-04-22T13:21:39.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.817+0000 7fe1e7474640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e010dcc0 con 0x7fe1e0103b50 2024-04-22T13:21:39.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.817+0000 7fe1e4a0f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1e0103190 0x7fe1e0074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:39.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.817+0000 7fe1e4a0f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1e0103190 0x7fe1e0074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41804/0 (socket says 172.21.15.12:41804) 2024-04-22T13:21:39.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.817+0000 7fe1e5210640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1e0101fa0 0x7fe1e0075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:39.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.817+0000 7fe1e4a0f640 1 -- 172.21.15.12:0/1155128292 learned_addr learned my addr 172.21.15.12:0/1155128292 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:21:39.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.817+0000 7fe1e5a11640 1 --2- 172.21.15.12:0/1155128292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1e0103b50 0x7fe1e0074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:39.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.817+0000 7fe1e4a0f640 1 -- 172.21.15.12:0/1155128292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1e0103b50 msgr2=0x7fe1e0074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:39.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.817+0000 7fe1e4a0f640 1 --2- 172.21.15.12:0/1155128292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1e0103b50 0x7fe1e0074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:39.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.817+0000 7fe1e4a0f640 1 -- 172.21.15.12:0/1155128292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1e0101fa0 msgr2=0x7fe1e0075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:39.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.817+0000 7fe1e4a0f640 1 --2- 172.21.15.12:0/1155128292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1e0101fa0 0x7fe1e0075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:39.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.817+0000 7fe1e4a0f640 1 -- 172.21.15.12:0/1155128292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e0075110 con 0x7fe1e0103190 2024-04-22T13:21:39.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.817+0000 7fe1e4a0f640 1 --2- 172.21.15.12:0/1155128292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1e0103190 0x7fe1e0074340 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe1d000e9f0 tx=0x7fe1d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:39.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.817+0000 7fe1d67fc640 1 -- 172.21.15.12:0/1155128292 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d000ccc0 con 0x7fe1e0103190 2024-04-22T13:21:39.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.817+0000 7fe1e7474640 1 -- 172.21.15.12:0/1155128292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1e01a22b0 con 0x7fe1e0103190 2024-04-22T13:21:39.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.817+0000 7fe1d67fc640 1 -- 172.21.15.12:0/1155128292 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1d00108e0 con 0x7fe1e0103190 2024-04-22T13:21:39.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.817+0000 7fe1d67fc640 1 -- 172.21.15.12:0/1155128292 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0018880 con 0x7fe1e0103190 2024-04-22T13:21:39.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.817+0000 7fe1e7474640 1 -- 172.21.15.12:0/1155128292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1e01a2800 con 0x7fe1e0103190 2024-04-22T13:21:39.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.818+0000 7fe1e7474640 1 -- 172.21.15.12:0/1155128292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1e01092c0 con 0x7fe1e0103190 2024-04-22T13:21:39.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.820+0000 7fe1d67fc640 1 -- 172.21.15.12:0/1155128292 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe1d0010430 con 0x7fe1e0103190 2024-04-22T13:21:39.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.820+0000 7fe1d67fc640 1 --2- 172.21.15.12:0/1155128292 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe1cc0a3bb0 0x7fe1cc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:39.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.820+0000 7fe1e5210640 1 --2- 172.21.15.12:0/1155128292 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe1cc0a3bb0 0x7fe1cc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:39.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.820+0000 7fe1d67fc640 1 -- 172.21.15.12:0/1155128292 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fe1d00d0820 con 0x7fe1e0103190 2024-04-22T13:21:39.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.821+0000 7fe1e5210640 1 --2- 172.21.15.12:0/1155128292 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe1cc0a3bb0 0x7fe1cc0a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0102ff0 tx=0x7fe1dc009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:39.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:39.824+0000 7fe1d67fc640 1 -- 172.21.15.12:0/1155128292 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe1d009d6a0 con 0x7fe1e0103190 2024-04-22T13:21:39.948 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 22 13:21:39 smithi111 systemd[1]: Starting Ceph mds.e for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:21:40.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:40.051+0000 7fe1e7474640 1 -- 172.21.15.12:0/1155128292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1e002d810 con 0x7fe1e0103190 2024-04-22T13:21:40.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:40.051+0000 7fe1d67fc640 1 -- 172.21.15.12:0/1155128292 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3874 (secure 0 0 0) 0x7fe1d009cfa0 con 0x7fe1e0103190 2024-04-22T13:21:40.049 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:21:40.049 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-22T13:21:40.049 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 5 2024-04-22T13:21:40.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:40.054+0000 7fe1e7474640 1 -- 172.21.15.12:0/1155128292 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe1cc0a3bb0 msgr2=0x7fe1cc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:40.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:40.054+0000 7fe1e7474640 1 --2- 172.21.15.12:0/1155128292 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe1cc0a3bb0 0x7fe1cc0a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0102ff0 tx=0x7fe1dc009110 comp rx=0 tx=0).stop 2024-04-22T13:21:40.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:40.054+0000 7fe1e7474640 1 -- 172.21.15.12:0/1155128292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1e0103190 msgr2=0x7fe1e0074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:40.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:40.054+0000 7fe1e7474640 1 --2- 172.21.15.12:0/1155128292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1e0103190 0x7fe1e0074340 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe1d000e9f0 tx=0x7fe1d000edf0 comp rx=0 tx=0).stop 2024-04-22T13:21:40.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:40.054+0000 7fe1e7474640 1 -- 172.21.15.12:0/1155128292 shutdown_connections 2024-04-22T13:21:40.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:40.054+0000 7fe1e7474640 1 --2- 172.21.15.12:0/1155128292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1e0103b50 0x7fe1e0074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:40.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:40.054+0000 7fe1e7474640 1 --2- 172.21.15.12:0/1155128292 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe1cc0a3bb0 0x7fe1cc0a6070 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:40.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:40.054+0000 7fe1e7474640 1 --2- 172.21.15.12:0/1155128292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1e0103190 0x7fe1e0074340 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:40.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:40.054+0000 7fe1e7474640 1 --2- 172.21.15.12:0/1155128292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1e0101fa0 0x7fe1e0075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:40.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:40.054+0000 7fe1e7474640 1 -- 172.21.15.12:0/1155128292 >> 172.21.15.12:0/1155128292 conn(0x7fe1e00fd530 msgr2=0x7fe1e0106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:40.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:40.054+0000 7fe1e7474640 1 -- 172.21.15.12:0/1155128292 shutdown_connections 2024-04-22T13:21:40.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:40.054+0000 7fe1e7474640 1 -- 172.21.15.12:0/1155128292 wait complete. 2024-04-22T13:21:40.061 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:39 smithi012 ceph-mon[28265]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:40.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:39 smithi111 ceph-mon[31361]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:40.227 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 22 13:21:40 smithi111 podman[61551]: 2024-04-22 13:21:39.930686596 +0000 UTC m=+0.017864345 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:21:40.227 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 22 13:21:40 smithi111 podman[61551]: 2024-04-22 13:21:40.051089748 +0000 UTC m=+0.138267490 container create 933813d49ecada9e59885b27ff94001926ff0147d5570b56874d2ebc34c87f6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-e, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True) 2024-04-22T13:21:40.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:39 smithi134 ceph-mon[31338]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:40.727 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 22 13:21:40 smithi111 podman[61551]: 2024-04-22 13:21:40.287007166 +0000 UTC m=+0.374184906 container init 933813d49ecada9e59885b27ff94001926ff0147d5570b56874d2ebc34c87f6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T13:21:40.727 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 22 13:21:40 smithi111 podman[61551]: 2024-04-22 13:21:40.289739776 +0000 UTC m=+0.376917518 container start 933813d49ecada9e59885b27ff94001926ff0147d5570b56874d2ebc34c87f6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-e, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T13:21:40.728 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 22 13:21:40 smithi111 ceph-mds[61570]: -- 172.21.15.111:0/314375206 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56206d977a40 con 0x56206d94a800 2024-04-22T13:21:40.728 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 22 13:21:40 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-e[61566]: starting mds.e at 2024-04-22T13:21:40.728 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 22 13:21:40 smithi111 bash[61551]: 933813d49ecada9e59885b27ff94001926ff0147d5570b56874d2ebc34c87f6c 2024-04-22T13:21:40.728 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 22 13:21:40 smithi111 systemd[1]: Started Ceph mds.e for 0aede5ea-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:21:41.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:40 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1155128292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:41.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:40 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:41.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:40 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:41.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:40 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:41.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:40 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:41.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:40 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:41.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:40 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:41.271 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:40 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1155128292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:41.271 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:40 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:41.271 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:40 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:41.271 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:40 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:41.271 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:40 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:41.271 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:40 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:41.271 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:40 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:41.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:40 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1155128292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:41.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:40 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:41.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:40 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:41.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:40 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:41.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:40 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:41.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:40 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:41.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:40 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:42.062 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:21:42.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.180+0000 7fd7ba0e9640 1 -- 172.21.15.12:0/1412135231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7b4102a40 msgr2=0x7fd7b4104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:42.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.180+0000 7fd7ba0e9640 1 --2- 172.21.15.12:0/1412135231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7b4102a40 0x7fd7b4104e30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4009900 tx=0x7fd7a402bc30 comp rx=0 tx=0).stop 2024-04-22T13:21:42.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.180+0000 7fd7ba0e9640 1 -- 172.21.15.12:0/1412135231 shutdown_connections 2024-04-22T13:21:42.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.180+0000 7fd7ba0e9640 1 --2- 172.21.15.12:0/1412135231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7b4107ca0 0x7fd7b410a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:42.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.180+0000 7fd7ba0e9640 1 --2- 172.21.15.12:0/1412135231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7b4105370 0x7fd7b4107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:42.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.180+0000 7fd7ba0e9640 1 --2- 172.21.15.12:0/1412135231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7b4102a40 0x7fd7b4104e30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:42.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.180+0000 7fd7ba0e9640 1 -- 172.21.15.12:0/1412135231 >> 172.21.15.12:0/1412135231 conn(0x7fd7b40fc470 msgr2=0x7fd7b40fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:42.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.180+0000 7fd7ba0e9640 1 -- 172.21.15.12:0/1412135231 shutdown_connections 2024-04-22T13:21:42.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.180+0000 7fd7ba0e9640 1 -- 172.21.15.12:0/1412135231 wait complete. 2024-04-22T13:21:42.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.181+0000 7fd7ba0e9640 1 Processor -- start 2024-04-22T13:21:42.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.181+0000 7fd7ba0e9640 1 -- start start 2024-04-22T13:21:42.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.181+0000 7fd7ba0e9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7b4102a40 0x7fd7b41981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:42.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.181+0000 7fd7ba0e9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7b4105370 0x7fd7b4198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:42.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.181+0000 7fd7ba0e9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7b4107ca0 0x7fd7b419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:42.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.181+0000 7fd7ba0e9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b410db00 con 0x7fd7b4105370 2024-04-22T13:21:42.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.181+0000 7fd7ba0e9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b410d980 con 0x7fd7b4102a40 2024-04-22T13:21:42.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.181+0000 7fd7ba0e9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b410dc80 con 0x7fd7b4107ca0 2024-04-22T13:21:42.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.181+0000 7fd7b2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7b4105370 0x7fd7b4198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:42.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.181+0000 7fd7b2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7b4105370 0x7fd7b4198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57084/0 (socket says 172.21.15.12:57084) 2024-04-22T13:21:42.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.181+0000 7fd7b2ffd640 1 -- 172.21.15.12:0/4061007834 learned_addr learned my addr 172.21.15.12:0/4061007834 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:21:42.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.181+0000 7fd7b3fff640 1 --2- 172.21.15.12:0/4061007834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7b4107ca0 0x7fd7b419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:42.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.182+0000 7fd7b37fe640 1 --2- 172.21.15.12:0/4061007834 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7b4102a40 0x7fd7b41981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:42.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.182+0000 7fd7b2ffd640 1 -- 172.21.15.12:0/4061007834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7b4107ca0 msgr2=0x7fd7b419f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:42.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.182+0000 7fd7b2ffd640 1 --2- 172.21.15.12:0/4061007834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7b4107ca0 0x7fd7b419f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:42.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.182+0000 7fd7b2ffd640 1 -- 172.21.15.12:0/4061007834 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7b4102a40 msgr2=0x7fd7b41981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:42.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.182+0000 7fd7b2ffd640 1 --2- 172.21.15.12:0/4061007834 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7b4102a40 0x7fd7b41981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:42.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.182+0000 7fd7b2ffd640 1 -- 172.21.15.12:0/4061007834 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b419fe00 con 0x7fd7b4105370 2024-04-22T13:21:42.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.182+0000 7fd7b3fff640 1 --2- 172.21.15.12:0/4061007834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7b4107ca0 0x7fd7b419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:21:42.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.182+0000 7fd7b37fe640 1 --2- 172.21.15.12:0/4061007834 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7b4102a40 0x7fd7b41981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:21:42.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.182+0000 7fd7b2ffd640 1 --2- 172.21.15.12:0/4061007834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7b4105370 0x7fd7b4198720 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800e9f0 tx=0x7fd7a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:42.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.182+0000 7fd7b0ff9640 1 -- 172.21.15.12:0/4061007834 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a800ccc0 con 0x7fd7b4105370 2024-04-22T13:21:42.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.182+0000 7fd7ba0e9640 1 -- 172.21.15.12:0/4061007834 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7b41a00f0 con 0x7fd7b4105370 2024-04-22T13:21:42.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.182+0000 7fd7b0ff9640 1 -- 172.21.15.12:0/4061007834 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7a80054d0 con 0x7fd7b4105370 2024-04-22T13:21:42.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.182+0000 7fd7b0ff9640 1 -- 172.21.15.12:0/4061007834 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8010430 con 0x7fd7b4105370 2024-04-22T13:21:42.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.182+0000 7fd7ba0e9640 1 -- 172.21.15.12:0/4061007834 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7b41a05f0 con 0x7fd7b4105370 2024-04-22T13:21:42.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.183+0000 7fd7ba0e9640 1 -- 172.21.15.12:0/4061007834 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7b40fdf60 con 0x7fd7b4105370 2024-04-22T13:21:42.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.184+0000 7fd7b0ff9640 1 -- 172.21.15.12:0/4061007834 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd7a80046c0 con 0x7fd7b4105370 2024-04-22T13:21:42.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.185+0000 7fd7b0ff9640 1 --2- 172.21.15.12:0/4061007834 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fd79c0a3bb0 0x7fd79c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:42.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.185+0000 7fd7b37fe640 1 --2- 172.21.15.12:0/4061007834 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fd79c0a3bb0 0x7fd79c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:42.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.185+0000 7fd7b0ff9640 1 -- 172.21.15.12:0/4061007834 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fd7a80d04d0 con 0x7fd7b4105370 2024-04-22T13:21:42.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.185+0000 7fd7b37fe640 1 --2- 172.21.15.12:0/4061007834 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fd79c0a3bb0 0x7fd79c0a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd7a40096c0 tx=0x7fd7a4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:42.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.189+0000 7fd7b0ff9640 1 -- 172.21.15.12:0/4061007834 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd7a809d2b0 con 0x7fd7b4105370 2024-04-22T13:21:42.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:41 smithi111 ceph-mon[31361]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:42.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:41 smithi111 ceph-mon[31361]: Deploying daemon mds.f on smithi134 2024-04-22T13:21:42.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:41 smithi111 ceph-mon[31361]: mds.? [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] up:boot 2024-04-22T13:21:42.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:41 smithi111 ceph-mon[31361]: fsmap 5 up:standby 2024-04-22T13:21:42.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:41 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-22T13:21:42.298 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:41 smithi134 ceph-mon[31338]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:42.298 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:41 smithi134 ceph-mon[31338]: Deploying daemon mds.f on smithi134 2024-04-22T13:21:42.298 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:41 smithi134 ceph-mon[31338]: mds.? [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] up:boot 2024-04-22T13:21:42.298 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:41 smithi134 ceph-mon[31338]: fsmap 5 up:standby 2024-04-22T13:21:42.298 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:41 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-22T13:21:42.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:41 smithi012 ceph-mon[28265]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:42.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:41 smithi012 ceph-mon[28265]: Deploying daemon mds.f on smithi134 2024-04-22T13:21:42.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:41 smithi012 ceph-mon[28265]: mds.? [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] up:boot 2024-04-22T13:21:42.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:41 smithi012 ceph-mon[28265]: fsmap 5 up:standby 2024-04-22T13:21:42.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:41 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-22T13:21:42.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.418+0000 7fd7ba0e9640 1 -- 172.21.15.12:0/4061007834 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7b402d5c0 con 0x7fd7b4105370 2024-04-22T13:21:42.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.418+0000 7fd7b0ff9640 1 -- 172.21.15.12:0/4061007834 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4702 (secure 0 0 0) 0x7fd7a809cbb0 con 0x7fd7b4105370 2024-04-22T13:21:42.415 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:21:42.415 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-22T13:21:42.415 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 6 2024-04-22T13:21:42.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.420+0000 7fd7ba0e9640 1 -- 172.21.15.12:0/4061007834 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fd79c0a3bb0 msgr2=0x7fd79c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:42.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.420+0000 7fd7ba0e9640 1 --2- 172.21.15.12:0/4061007834 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fd79c0a3bb0 0x7fd79c0a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd7a40096c0 tx=0x7fd7a4000bc0 comp rx=0 tx=0).stop 2024-04-22T13:21:42.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.420+0000 7fd7ba0e9640 1 -- 172.21.15.12:0/4061007834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7b4105370 msgr2=0x7fd7b4198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:42.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.420+0000 7fd7ba0e9640 1 --2- 172.21.15.12:0/4061007834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7b4105370 0x7fd7b4198720 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800e9f0 tx=0x7fd7a800eeb0 comp rx=0 tx=0).stop 2024-04-22T13:21:42.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.420+0000 7fd7ba0e9640 1 -- 172.21.15.12:0/4061007834 shutdown_connections 2024-04-22T13:21:42.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.420+0000 7fd7ba0e9640 1 --2- 172.21.15.12:0/4061007834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7b4107ca0 0x7fd7b419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:42.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.420+0000 7fd7ba0e9640 1 --2- 172.21.15.12:0/4061007834 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fd79c0a3bb0 0x7fd79c0a6070 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:42.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.420+0000 7fd7ba0e9640 1 --2- 172.21.15.12:0/4061007834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7b4105370 0x7fd7b4198720 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:42.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.420+0000 7fd7ba0e9640 1 --2- 172.21.15.12:0/4061007834 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7b4102a40 0x7fd7b41981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:42.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.420+0000 7fd7ba0e9640 1 -- 172.21.15.12:0/4061007834 >> 172.21.15.12:0/4061007834 conn(0x7fd7b40fc470 msgr2=0x7fd7b41033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:42.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.421+0000 7fd7ba0e9640 1 -- 172.21.15.12:0/4061007834 shutdown_connections 2024-04-22T13:21:42.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:42.421+0000 7fd7ba0e9640 1 -- 172.21.15.12:0/4061007834 wait complete. 2024-04-22T13:21:42.677 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 22 13:21:42 smithi134 systemd[1]: Starting Ceph mds.f for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:21:42.928 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 22 13:21:42 smithi134 podman[62151]: 2024-04-22 13:21:42.680402182 +0000 UTC m=+0.096897049 container create 8ad0c7dc27649152d1bfde057e2cb35d7b4ab1053c5a9c18a98159bb142eec21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.schema-version=1.0) 2024-04-22T13:21:42.928 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 22 13:21:42 smithi134 podman[62151]: 2024-04-22 13:21:42.601459221 +0000 UTC m=+0.017954099 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:21:42.929 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 22 13:21:42 smithi134 podman[62151]: 2024-04-22 13:21:42.899461121 +0000 UTC m=+0.315955986 container init 8ad0c7dc27649152d1bfde057e2cb35d7b4ab1053c5a9c18a98159bb142eec21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-f, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-22T13:21:42.929 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 22 13:21:42 smithi134 podman[62151]: 2024-04-22 13:21:42.901916144 +0000 UTC m=+0.318411012 container start 8ad0c7dc27649152d1bfde057e2cb35d7b4ab1053c5a9c18a98159bb142eec21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-f, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-22T13:21:42.929 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 22 13:21:42 smithi134 ceph-mds[62170]: -- 172.21.15.134:0/1089516703 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561e1670fa40 con 0x561e166e2800 2024-04-22T13:21:42.929 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 22 13:21:42 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-f[62166]: starting mds.f at 2024-04-22T13:21:43.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:42 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/4061007834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:43.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:42 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/4061007834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:43.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:42 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/4061007834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:43.428 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 22 13:21:43 smithi134 bash[62151]: 8ad0c7dc27649152d1bfde057e2cb35d7b4ab1053c5a9c18a98159bb142eec21 2024-04-22T13:21:43.428 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 22 13:21:43 smithi134 systemd[1]: Started Ceph mds.f for 0aede5ea-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:21:44.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:43 smithi111 ceph-mon[31361]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:44.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:43 smithi111 ceph-mon[31361]: mds.? [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] up:boot 2024-04-22T13:21:44.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:43 smithi111 ceph-mon[31361]: fsmap 6 up:standby 2024-04-22T13:21:44.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:43 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T13:21:44.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:43 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:44.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:43 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:44.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:43 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:44.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:43 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:44.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:43 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:44.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:43 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:44.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:43 smithi111 ceph-mon[31361]: Deploying daemon mds.c on smithi134 2024-04-22T13:21:44.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:43 smithi012 ceph-mon[28265]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:44.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:43 smithi012 ceph-mon[28265]: mds.? [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] up:boot 2024-04-22T13:21:44.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:43 smithi012 ceph-mon[28265]: fsmap 6 up:standby 2024-04-22T13:21:44.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T13:21:44.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:44.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:44.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:44.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:44.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:44.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:43 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:44.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:43 smithi012 ceph-mon[28265]: Deploying daemon mds.c on smithi134 2024-04-22T13:21:44.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:43 smithi134 ceph-mon[31338]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:44.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:43 smithi134 ceph-mon[31338]: mds.? [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] up:boot 2024-04-22T13:21:44.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:43 smithi134 ceph-mon[31338]: fsmap 6 up:standby 2024-04-22T13:21:44.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:43 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T13:21:44.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:43 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:44.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:43 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:44.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:43 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:44.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:43 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:44.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:43 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:44.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:43 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:44.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:43 smithi134 ceph-mon[31338]: Deploying daemon mds.c on smithi134 2024-04-22T13:21:44.429 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:21:44.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.547+0000 7f18f93ce640 1 -- 172.21.15.12:0/339451124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18f4103180 msgr2=0x7f18f4103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:44.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.547+0000 7f18f93ce640 1 --2- 172.21.15.12:0/339451124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18f4103180 0x7f18f4103600 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f18e8009900 tx=0x7f18e802bc30 comp rx=0 tx=0).stop 2024-04-22T13:21:44.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.547+0000 7f18f93ce640 1 -- 172.21.15.12:0/339451124 shutdown_connections 2024-04-22T13:21:44.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.547+0000 7f18f93ce640 1 --2- 172.21.15.12:0/339451124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18f4103b40 0x7f18f410a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:44.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.547+0000 7f18f93ce640 1 --2- 172.21.15.12:0/339451124 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18f4103180 0x7f18f4103600 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:44.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.547+0000 7f18f93ce640 1 --2- 172.21.15.12:0/339451124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18f4101f90 0x7f18f41023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:44.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.547+0000 7f18f93ce640 1 -- 172.21.15.12:0/339451124 >> 172.21.15.12:0/339451124 conn(0x7f18f40fd500 msgr2=0x7f18f40ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:44.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.547+0000 7f18f93ce640 1 -- 172.21.15.12:0/339451124 shutdown_connections 2024-04-22T13:21:44.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.547+0000 7f18f93ce640 1 -- 172.21.15.12:0/339451124 wait complete. 2024-04-22T13:21:44.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.547+0000 7f18f93ce640 1 Processor -- start 2024-04-22T13:21:44.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.548+0000 7f18f93ce640 1 -- start start 2024-04-22T13:21:44.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.548+0000 7f18f93ce640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18f4101f90 0x7f18f4198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:44.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.548+0000 7f18f93ce640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18f4103180 0x7f18f41986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:44.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.548+0000 7f18f93ce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18f4103b40 0x7f18f419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:44.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.548+0000 7f18f93ce640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f18f410dad0 con 0x7f18f4101f90 2024-04-22T13:21:44.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.548+0000 7f18f93ce640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f18f410d950 con 0x7f18f4103b40 2024-04-22T13:21:44.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.548+0000 7f18f93ce640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f18f410dc50 con 0x7f18f4103180 2024-04-22T13:21:44.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.548+0000 7f18f2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18f4101f90 0x7f18f4198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:44.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.548+0000 7f18f2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18f4101f90 0x7f18f4198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57100/0 (socket says 172.21.15.12:57100) 2024-04-22T13:21:44.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.548+0000 7f18f2ffd640 1 -- 172.21.15.12:0/3024248677 learned_addr learned my addr 172.21.15.12:0/3024248677 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:21:44.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.548+0000 7f18f27fc640 1 --2- 172.21.15.12:0/3024248677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18f4103180 0x7f18f41986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:44.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.548+0000 7f18f37fe640 1 --2- 172.21.15.12:0/3024248677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18f4103b40 0x7f18f419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:44.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.548+0000 7f18f2ffd640 1 -- 172.21.15.12:0/3024248677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18f4103180 msgr2=0x7f18f41986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:44.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.548+0000 7f18f2ffd640 1 --2- 172.21.15.12:0/3024248677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18f4103180 0x7f18f41986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:44.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.548+0000 7f18f2ffd640 1 -- 172.21.15.12:0/3024248677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18f4103b40 msgr2=0x7f18f419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:44.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.548+0000 7f18f2ffd640 1 --2- 172.21.15.12:0/3024248677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18f4103b40 0x7f18f419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:44.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.548+0000 7f18f2ffd640 1 -- 172.21.15.12:0/3024248677 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18f419fd80 con 0x7f18f4101f90 2024-04-22T13:21:44.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.549+0000 7f18f2ffd640 1 --2- 172.21.15.12:0/3024248677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18f4101f90 0x7f18f4198160 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f18e400d770 tx=0x7f18e400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:44.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.549+0000 7f18dbfff640 1 -- 172.21.15.12:0/3024248677 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e4017070 con 0x7f18f4101f90 2024-04-22T13:21:44.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.549+0000 7f18f93ce640 1 -- 172.21.15.12:0/3024248677 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18f41a0070 con 0x7f18f4101f90 2024-04-22T13:21:44.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.549+0000 7f18dbfff640 1 -- 172.21.15.12:0/3024248677 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18e4009c30 con 0x7f18f4101f90 2024-04-22T13:21:44.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.549+0000 7f18dbfff640 1 -- 172.21.15.12:0/3024248677 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e4003cc0 con 0x7f18f4101f90 2024-04-22T13:21:44.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.549+0000 7f18f93ce640 1 -- 172.21.15.12:0/3024248677 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18f41a05c0 con 0x7f18f4101f90 2024-04-22T13:21:44.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.550+0000 7f18f93ce640 1 -- 172.21.15.12:0/3024248677 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18f41092b0 con 0x7f18f4101f90 2024-04-22T13:21:44.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.551+0000 7f18dbfff640 1 -- 172.21.15.12:0/3024248677 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f18e4003e60 con 0x7f18f4101f90 2024-04-22T13:21:44.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.552+0000 7f18dbfff640 1 --2- 172.21.15.12:0/3024248677 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f18dc0a3bb0 0x7f18dc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:44.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.552+0000 7f18dbfff640 1 -- 172.21.15.12:0/3024248677 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f18e40cfe80 con 0x7f18f4101f90 2024-04-22T13:21:44.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.552+0000 7f18f27fc640 1 --2- 172.21.15.12:0/3024248677 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f18dc0a3bb0 0x7f18dc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:44.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.552+0000 7f18f27fc640 1 --2- 172.21.15.12:0/3024248677 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f18dc0a3bb0 0x7f18dc0a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f18f4199670 tx=0x7f18e8005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:44.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.556+0000 7f18dbfff640 1 -- 172.21.15.12:0/3024248677 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f18e4009de0 con 0x7f18f4101f90 2024-04-22T13:21:44.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.787+0000 7f18f93ce640 1 -- 172.21.15.12:0/3024248677 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18f402d5c0 con 0x7f18f4101f90 2024-04-22T13:21:44.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.787+0000 7f18dbfff640 1 -- 172.21.15.12:0/3024248677 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5530 (secure 0 0 0) 0x7f18e409cc60 con 0x7f18f4101f90 2024-04-22T13:21:44.785 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:21:44.785 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-22T13:21:44.786 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 7 2024-04-22T13:21:44.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.790+0000 7f18f93ce640 1 -- 172.21.15.12:0/3024248677 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f18dc0a3bb0 msgr2=0x7f18dc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:44.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.790+0000 7f18f93ce640 1 --2- 172.21.15.12:0/3024248677 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f18dc0a3bb0 0x7f18dc0a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f18f4199670 tx=0x7f18e8005be0 comp rx=0 tx=0).stop 2024-04-22T13:21:44.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.790+0000 7f18f93ce640 1 -- 172.21.15.12:0/3024248677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18f4101f90 msgr2=0x7f18f4198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:44.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.790+0000 7f18f93ce640 1 --2- 172.21.15.12:0/3024248677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18f4101f90 0x7f18f4198160 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f18e400d770 tx=0x7f18e400dc30 comp rx=0 tx=0).stop 2024-04-22T13:21:44.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.790+0000 7f18f93ce640 1 -- 172.21.15.12:0/3024248677 shutdown_connections 2024-04-22T13:21:44.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.790+0000 7f18f93ce640 1 --2- 172.21.15.12:0/3024248677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18f4103b40 0x7f18f419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:44.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.790+0000 7f18f93ce640 1 --2- 172.21.15.12:0/3024248677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18f4103180 0x7f18f41986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:44.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.790+0000 7f18f93ce640 1 --2- 172.21.15.12:0/3024248677 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f18dc0a3bb0 0x7f18dc0a6070 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:44.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.790+0000 7f18f93ce640 1 --2- 172.21.15.12:0/3024248677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18f4101f90 0x7f18f4198160 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:44.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.790+0000 7f18f93ce640 1 -- 172.21.15.12:0/3024248677 >> 172.21.15.12:0/3024248677 conn(0x7f18f40fd500 msgr2=0x7f18f4106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:44.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.790+0000 7f18f93ce640 1 -- 172.21.15.12:0/3024248677 shutdown_connections 2024-04-22T13:21:44.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:44.790+0000 7f18f93ce640 1 -- 172.21.15.12:0/3024248677 wait complete. 2024-04-22T13:21:45.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:44 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3024248677' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:45.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:44 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/3024248677' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:45.178 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 22 13:21:45 smithi134 systemd[1]: Starting Ceph mds.c for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:21:45.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:44 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/3024248677' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:45.511 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 22 13:21:45 smithi134 podman[62462]: 2024-04-22 13:21:45.1596677 +0000 UTC m=+0.018173610 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:21:45.511 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 22 13:21:45 smithi134 podman[62462]: 2024-04-22 13:21:45.275016781 +0000 UTC m=+0.133522692 container create 43ee81f18aaa61c7ab7be2a43d45adec793586eca87d518c0cae167334e629d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-c, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-22T13:21:45.511 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 22 13:21:45 smithi134 podman[62462]: 2024-04-22 13:21:45.489134947 +0000 UTC m=+0.347640857 container init 43ee81f18aaa61c7ab7be2a43d45adec793586eca87d518c0cae167334e629d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD) 2024-04-22T13:21:45.511 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 22 13:21:45 smithi134 podman[62462]: 2024-04-22 13:21:45.491694571 +0000 UTC m=+0.350200489 container start 43ee81f18aaa61c7ab7be2a43d45adec793586eca87d518c0cae167334e629d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-22T13:21:45.511 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 22 13:21:45 smithi134 ceph-mds[62481]: -- 172.21.15.134:0/2252566136 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563debd5fa40 con 0x563debd32800 2024-04-22T13:21:45.928 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 22 13:21:45 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-c[62477]: starting mds.c at 2024-04-22T13:21:45.928 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 22 13:21:45 smithi134 bash[62462]: 43ee81f18aaa61c7ab7be2a43d45adec793586eca87d518c0cae167334e629d5 2024-04-22T13:21:45.928 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 22 13:21:45 smithi134 systemd[1]: Started Ceph mds.c for 0aede5ea-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:21:46.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:45 smithi111 ceph-mon[31361]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:46.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:45 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:46.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:45 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:46.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:45 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:46.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:45 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:46.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:45 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:46.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:45 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:46.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:45 smithi012 ceph-mon[28265]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:46.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:45 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:46.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:45 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:46.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:45 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:46.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:45 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:46.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:45 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:46.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:45 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:46.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:45 smithi134 ceph-mon[31338]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:46.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:45 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:46.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:45 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:46.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:45 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:46.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:45 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:46.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:45 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:46.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:45 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:46.800 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:21:46.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.917+0000 7fc1ffc70640 1 -- 172.21.15.12:0/1011228854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1f8101b30 msgr2=0x7fc1f8101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:46.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.917+0000 7fc1ffc70640 1 --2- 172.21.15.12:0/1011228854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1f8101b30 0x7fc1f8101fb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0009ca0 tx=0x7fc1f002c100 comp rx=0 tx=0).stop 2024-04-22T13:21:46.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.917+0000 7fc1ffc70640 1 -- 172.21.15.12:0/1011228854 shutdown_connections 2024-04-22T13:21:46.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.917+0000 7fc1ffc70640 1 --2- 172.21.15.12:0/1011228854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1f8101b30 0x7fc1f8101fb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:46.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.917+0000 7fc1ffc70640 1 --2- 172.21.15.12:0/1011228854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1f8101190 0x7fc1f81015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:46.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.917+0000 7fc1ffc70640 1 --2- 172.21.15.12:0/1011228854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1f8100830 0x7fc1f8100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:46.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.917+0000 7fc1ffc70640 1 -- 172.21.15.12:0/1011228854 >> 172.21.15.12:0/1011228854 conn(0x7fc1f80fc470 msgr2=0x7fc1f80fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:46.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.917+0000 7fc1ffc70640 1 -- 172.21.15.12:0/1011228854 shutdown_connections 2024-04-22T13:21:46.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.917+0000 7fc1ffc70640 1 -- 172.21.15.12:0/1011228854 wait complete. 2024-04-22T13:21:46.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.918+0000 7fc1ffc70640 1 Processor -- start 2024-04-22T13:21:46.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.918+0000 7fc1ffc70640 1 -- start start 2024-04-22T13:21:46.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.918+0000 7fc1ffc70640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1f8100830 0x7fc1f819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:46.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.918+0000 7fc1ffc70640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1f8101190 0x7fc1f819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:46.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.918+0000 7fc1ffc70640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1f8101b30 0x7fc1f81a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:46.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.918+0000 7fc1ffc70640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f810fd10 con 0x7fc1f8100830 2024-04-22T13:21:46.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.918+0000 7fc1ffc70640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f810fb90 con 0x7fc1f8101b30 2024-04-22T13:21:46.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.918+0000 7fc1ffc70640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f810fe90 con 0x7fc1f8101190 2024-04-22T13:21:46.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.918+0000 7fc1fda0c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1f8100830 0x7fc1f819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:46.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.918+0000 7fc1fda0c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1f8100830 0x7fc1f819a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57112/0 (socket says 172.21.15.12:57112) 2024-04-22T13:21:46.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.918+0000 7fc1fda0c640 1 -- 172.21.15.12:0/1134673607 learned_addr learned my addr 172.21.15.12:0/1134673607 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:21:46.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.919+0000 7fc1fe20d640 1 --2- 172.21.15.12:0/1134673607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1f8101b30 0x7fc1f81a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:46.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.919+0000 7fc1fd20b640 1 --2- 172.21.15.12:0/1134673607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1f8101190 0x7fc1f819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:46.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.919+0000 7fc1fda0c640 1 -- 172.21.15.12:0/1134673607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1f8101190 msgr2=0x7fc1f819a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:46.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.919+0000 7fc1fda0c640 1 --2- 172.21.15.12:0/1134673607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1f8101190 0x7fc1f819a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:46.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.919+0000 7fc1fda0c640 1 -- 172.21.15.12:0/1134673607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1f8101b30 msgr2=0x7fc1f81a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:46.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.919+0000 7fc1fda0c640 1 --2- 172.21.15.12:0/1134673607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1f8101b30 0x7fc1f81a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:46.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.919+0000 7fc1fda0c640 1 -- 172.21.15.12:0/1134673607 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f81a1ed0 con 0x7fc1f8100830 2024-04-22T13:21:46.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.919+0000 7fc1fe20d640 1 --2- 172.21.15.12:0/1134673607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1f8101b30 0x7fc1f81a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:21:46.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.919+0000 7fc1fda0c640 1 --2- 172.21.15.12:0/1134673607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1f8100830 0x7fc1f819a2b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400d770 tx=0x7fc1f400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:46.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.919+0000 7fc1eeffd640 1 -- 172.21.15.12:0/1134673607 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4017070 con 0x7fc1f8100830 2024-04-22T13:21:46.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.919+0000 7fc1eeffd640 1 -- 172.21.15.12:0/1134673607 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f40054d0 con 0x7fc1f8100830 2024-04-22T13:21:46.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.919+0000 7fc1ffc70640 1 -- 172.21.15.12:0/1134673607 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1f81a21c0 con 0x7fc1f8100830 2024-04-22T13:21:46.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.919+0000 7fc1eeffd640 1 -- 172.21.15.12:0/1134673607 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4004c30 con 0x7fc1f8100830 2024-04-22T13:21:46.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.919+0000 7fc1ffc70640 1 -- 172.21.15.12:0/1134673607 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1f81a26c0 con 0x7fc1f8100830 2024-04-22T13:21:46.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.920+0000 7fc1ffc70640 1 -- 172.21.15.12:0/1134673607 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1f810b650 con 0x7fc1f8100830 2024-04-22T13:21:46.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.921+0000 7fc1eeffd640 1 -- 172.21.15.12:0/1134673607 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc1f4004e10 con 0x7fc1f8100830 2024-04-22T13:21:46.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.922+0000 7fc1eeffd640 1 --2- 172.21.15.12:0/1134673607 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc1e40a3bb0 0x7fc1e40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:46.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.922+0000 7fc1fd20b640 1 --2- 172.21.15.12:0/1134673607 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc1e40a3bb0 0x7fc1e40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:46.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.922+0000 7fc1eeffd640 1 -- 172.21.15.12:0/1134673607 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fc1f40cfb70 con 0x7fc1f8100830 2024-04-22T13:21:46.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.922+0000 7fc1fd20b640 1 --2- 172.21.15.12:0/1134673607 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc1e40a3bb0 0x7fc1e40a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8005e10 tx=0x7fc1e8005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:46.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:46.927+0000 7fc1eeffd640 1 -- 172.21.15.12:0/1134673607 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc1f409c950 con 0x7fc1f8100830 2024-04-22T13:21:47.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:46 smithi012 ceph-mon[28265]: Deploying daemon mds.h on smithi111 2024-04-22T13:21:47.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:46 smithi012 ceph-mon[28265]: mds.? [v2:172.21.15.134:6836/3631577545,v1:172.21.15.134:6837/3631577545] up:boot 2024-04-22T13:21:47.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:46 smithi012 ceph-mon[28265]: fsmap 7 up:standby 2024-04-22T13:21:47.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:46 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-22T13:21:47.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:47.154+0000 7fc1ffc70640 1 -- 172.21.15.12:0/1134673607 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1f802d5c0 con 0x7fc1f8100830 2024-04-22T13:21:47.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:47.155+0000 7fc1eeffd640 1 -- 172.21.15.12:0/1134673607 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6358 (secure 0 0 0) 0x7fc1f409c250 con 0x7fc1f8100830 2024-04-22T13:21:47.153 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:21:47.153 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-22T13:21:47.153 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 8 2024-04-22T13:21:47.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:47.157+0000 7fc1ffc70640 1 -- 172.21.15.12:0/1134673607 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc1e40a3bb0 msgr2=0x7fc1e40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:47.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:47.157+0000 7fc1ffc70640 1 --2- 172.21.15.12:0/1134673607 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc1e40a3bb0 0x7fc1e40a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8005e10 tx=0x7fc1e8005da0 comp rx=0 tx=0).stop 2024-04-22T13:21:47.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:47.157+0000 7fc1ffc70640 1 -- 172.21.15.12:0/1134673607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1f8100830 msgr2=0x7fc1f819a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:47.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:47.157+0000 7fc1ffc70640 1 --2- 172.21.15.12:0/1134673607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1f8100830 0x7fc1f819a2b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400d770 tx=0x7fc1f400dc30 comp rx=0 tx=0).stop 2024-04-22T13:21:47.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:47.157+0000 7fc1ffc70640 1 -- 172.21.15.12:0/1134673607 shutdown_connections 2024-04-22T13:21:47.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:47.157+0000 7fc1ffc70640 1 --2- 172.21.15.12:0/1134673607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1f8101b30 0x7fc1f81a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:47.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:47.157+0000 7fc1ffc70640 1 --2- 172.21.15.12:0/1134673607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1f8101190 0x7fc1f819a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:47.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:47.157+0000 7fc1ffc70640 1 --2- 172.21.15.12:0/1134673607 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc1e40a3bb0 0x7fc1e40a6070 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:47.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:47.157+0000 7fc1ffc70640 1 --2- 172.21.15.12:0/1134673607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc1f8100830 0x7fc1f819a2b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:47.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:47.157+0000 7fc1ffc70640 1 -- 172.21.15.12:0/1134673607 >> 172.21.15.12:0/1134673607 conn(0x7fc1f80fc470 msgr2=0x7fc1f80fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:47.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:47.157+0000 7fc1ffc70640 1 -- 172.21.15.12:0/1134673607 shutdown_connections 2024-04-22T13:21:47.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:47.157+0000 7fc1ffc70640 1 -- 172.21.15.12:0/1134673607 wait complete. 2024-04-22T13:21:47.295 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:46 smithi111 ceph-mon[31361]: Deploying daemon mds.h on smithi111 2024-04-22T13:21:47.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:46 smithi111 ceph-mon[31361]: mds.? [v2:172.21.15.134:6836/3631577545,v1:172.21.15.134:6837/3631577545] up:boot 2024-04-22T13:21:47.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:46 smithi111 ceph-mon[31361]: fsmap 7 up:standby 2024-04-22T13:21:47.296 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:46 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-22T13:21:47.330 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:46 smithi134 ceph-mon[31338]: Deploying daemon mds.h on smithi111 2024-04-22T13:21:47.330 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:46 smithi134 ceph-mon[31338]: mds.? [v2:172.21.15.134:6836/3631577545,v1:172.21.15.134:6837/3631577545] up:boot 2024-04-22T13:21:47.330 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:46 smithi134 ceph-mon[31338]: fsmap 7 up:standby 2024-04-22T13:21:47.330 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:46 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-22T13:21:48.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:47 smithi111 ceph-mon[31361]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:48.228 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:47 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1134673607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:48.228 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 22 13:21:48 smithi111 podman[61856]: 2024-04-22 13:21:48.154896528 +0000 UTC m=+0.120164945 container create 2081b13c1b65210248b8024205cffa724f493ca66d7d41c96e396ec89ca336d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-h, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-22T13:21:48.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:47 smithi012 ceph-mon[28265]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:48.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:47 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1134673607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:48.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:47 smithi134 ceph-mon[31338]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:48.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:47 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1134673607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:48.512 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 22 13:21:48 smithi111 podman[61856]: 2024-04-22 13:21:48.415763221 +0000 UTC m=+0.381031638 container init 2081b13c1b65210248b8024205cffa724f493ca66d7d41c96e396ec89ca336d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-h, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-22T13:21:48.512 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 22 13:21:48 smithi111 podman[61856]: 2024-04-22 13:21:48.418438628 +0000 UTC m=+0.383707048 container start 2081b13c1b65210248b8024205cffa724f493ca66d7d41c96e396ec89ca336d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-22T13:21:48.512 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 22 13:21:48 smithi111 ceph-mds[61875]: -- 172.21.15.111:0/1692499344 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a190593a40 con 0x55a190566c00 2024-04-22T13:21:48.512 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 22 13:21:48 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-h[61871]: starting mds.h at 2024-04-22T13:21:48.977 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 22 13:21:48 smithi111 bash[61856]: 2081b13c1b65210248b8024205cffa724f493ca66d7d41c96e396ec89ca336d0 2024-04-22T13:21:48.977 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 22 13:21:48 smithi111 systemd[1]: Started Ceph mds.h for 0aede5ea-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:21:49.166 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:21:49.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.283+0000 7f1d30f6d640 1 -- 172.21.15.12:0/3847928987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d2c100e70 msgr2=0x7f1d2c101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:49.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.283+0000 7f1d30f6d640 1 --2- 172.21.15.12:0/3847928987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d2c100e70 0x7f1d2c101290 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1d20009900 tx=0x7f1d2002bc30 comp rx=0 tx=0).stop 2024-04-22T13:21:49.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.283+0000 7f1d30f6d640 1 -- 172.21.15.12:0/3847928987 shutdown_connections 2024-04-22T13:21:49.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.283+0000 7f1d30f6d640 1 --2- 172.21.15.12:0/3847928987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d2c102980 0x7f1d2c1092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:49.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.283+0000 7f1d30f6d640 1 --2- 172.21.15.12:0/3847928987 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d2c101fc0 0x7f1d2c102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:49.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.283+0000 7f1d30f6d640 1 --2- 172.21.15.12:0/3847928987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d2c100e70 0x7f1d2c101290 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:49.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.283+0000 7f1d30f6d640 1 -- 172.21.15.12:0/3847928987 >> 172.21.15.12:0/3847928987 conn(0x7f1d2c0fc3e0 msgr2=0x7f1d2c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:49.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.283+0000 7f1d30f6d640 1 -- 172.21.15.12:0/3847928987 shutdown_connections 2024-04-22T13:21:49.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.283+0000 7f1d30f6d640 1 -- 172.21.15.12:0/3847928987 wait complete. 2024-04-22T13:21:49.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.284+0000 7f1d30f6d640 1 Processor -- start 2024-04-22T13:21:49.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.284+0000 7f1d30f6d640 1 -- start start 2024-04-22T13:21:49.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.284+0000 7f1d30f6d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d2c100e70 0x7f1d2c069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:49.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.284+0000 7f1d30f6d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d2c101fc0 0x7f1d2c19cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:49.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.284+0000 7f1d30f6d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d2c102980 0x7f1d2c19f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:49.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.284+0000 7f1d30f6d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c10c9c0 con 0x7f1d2c100e70 2024-04-22T13:21:49.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.284+0000 7f1d30f6d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c10c840 con 0x7f1d2c101fc0 2024-04-22T13:21:49.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.284+0000 7f1d30f6d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c10cb40 con 0x7f1d2c102980 2024-04-22T13:21:49.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.284+0000 7f1d2a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d2c100e70 0x7f1d2c069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:49.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.284+0000 7f1d2a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d2c100e70 0x7f1d2c069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57128/0 (socket says 172.21.15.12:57128) 2024-04-22T13:21:49.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.284+0000 7f1d2a59c640 1 -- 172.21.15.12:0/1967218874 learned_addr learned my addr 172.21.15.12:0/1967218874 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:21:49.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.284+0000 7f1d2ad9d640 1 --2- 172.21.15.12:0/1967218874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d2c102980 0x7f1d2c19f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:49.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.284+0000 7f1d29d9b640 1 --2- 172.21.15.12:0/1967218874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d2c101fc0 0x7f1d2c19cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:49.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.284+0000 7f1d2a59c640 1 -- 172.21.15.12:0/1967218874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d2c102980 msgr2=0x7f1d2c19f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:49.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.284+0000 7f1d2a59c640 1 --2- 172.21.15.12:0/1967218874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d2c102980 0x7f1d2c19f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:49.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.284+0000 7f1d2a59c640 1 -- 172.21.15.12:0/1967218874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d2c101fc0 msgr2=0x7f1d2c19cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:49.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.284+0000 7f1d2a59c640 1 --2- 172.21.15.12:0/1967218874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d2c101fc0 0x7f1d2c19cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:49.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.284+0000 7f1d2a59c640 1 -- 172.21.15.12:0/1967218874 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d2c19fe10 con 0x7f1d2c100e70 2024-04-22T13:21:49.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.284+0000 7f1d29d9b640 1 --2- 172.21.15.12:0/1967218874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d2c101fc0 0x7f1d2c19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:21:49.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.285+0000 7f1d2a59c640 1 --2- 172.21.15.12:0/1967218874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d2c100e70 0x7f1d2c069c50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1d20006010 tx=0x7f1d20004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:49.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.285+0000 7f1d1b7fe640 1 -- 172.21.15.12:0/1967218874 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20043020 con 0x7f1d2c100e70 2024-04-22T13:21:49.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.285+0000 7f1d1b7fe640 1 -- 172.21.15.12:0/1967218874 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d20004440 con 0x7f1d2c100e70 2024-04-22T13:21:49.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.285+0000 7f1d30f6d640 1 -- 172.21.15.12:0/1967218874 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d2c1a0040 con 0x7f1d2c100e70 2024-04-22T13:21:49.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.285+0000 7f1d1b7fe640 1 -- 172.21.15.12:0/1967218874 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20003bf0 con 0x7f1d2c100e70 2024-04-22T13:21:49.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.285+0000 7f1d30f6d640 1 -- 172.21.15.12:0/1967218874 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d2c1a05e0 con 0x7f1d2c100e70 2024-04-22T13:21:49.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.286+0000 7f1d30f6d640 1 -- 172.21.15.12:0/1967218874 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d2c108220 con 0x7f1d2c100e70 2024-04-22T13:21:49.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.287+0000 7f1d1b7fe640 1 -- 172.21.15.12:0/1967218874 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1d20046020 con 0x7f1d2c100e70 2024-04-22T13:21:49.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.288+0000 7f1d1b7fe640 1 --2- 172.21.15.12:0/1967218874 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1d140a3c30 0x7f1d140a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:49.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.288+0000 7f1d29d9b640 1 --2- 172.21.15.12:0/1967218874 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1d140a3c30 0x7f1d140a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:49.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.288+0000 7f1d1b7fe640 1 -- 172.21.15.12:0/1967218874 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f1d200f22a0 con 0x7f1d2c100e70 2024-04-22T13:21:49.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.288+0000 7f1d29d9b640 1 --2- 172.21.15.12:0/1967218874 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1d140a3c30 0x7f1d140a60f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c0058c0 tx=0x7f1d1c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:49.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.293+0000 7f1d1b7fe640 1 -- 172.21.15.12:0/1967218874 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1d2004d020 con 0x7f1d2c100e70 2024-04-22T13:21:49.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.519+0000 7f1d30f6d640 1 -- 172.21.15.12:0/1967218874 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d2c02d5c0 con 0x7f1d2c100e70 2024-04-22T13:21:49.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.520+0000 7f1d1b7fe640 1 -- 172.21.15.12:0/1967218874 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7186 (secure 0 0 0) 0x7f1d200bf120 con 0x7f1d2c100e70 2024-04-22T13:21:49.518 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:21:49.518 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-22T13:21:49.518 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 9 2024-04-22T13:21:49.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.522+0000 7f1d30f6d640 1 -- 172.21.15.12:0/1967218874 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1d140a3c30 msgr2=0x7f1d140a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:49.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.522+0000 7f1d30f6d640 1 --2- 172.21.15.12:0/1967218874 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1d140a3c30 0x7f1d140a60f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c0058c0 tx=0x7f1d1c005850 comp rx=0 tx=0).stop 2024-04-22T13:21:49.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.522+0000 7f1d30f6d640 1 -- 172.21.15.12:0/1967218874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d2c100e70 msgr2=0x7f1d2c069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:49.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.522+0000 7f1d30f6d640 1 --2- 172.21.15.12:0/1967218874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d2c100e70 0x7f1d2c069c50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1d20006010 tx=0x7f1d20004040 comp rx=0 tx=0).stop 2024-04-22T13:21:49.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.522+0000 7f1d30f6d640 1 -- 172.21.15.12:0/1967218874 shutdown_connections 2024-04-22T13:21:49.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.522+0000 7f1d30f6d640 1 --2- 172.21.15.12:0/1967218874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d2c102980 0x7f1d2c19f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:49.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.522+0000 7f1d30f6d640 1 --2- 172.21.15.12:0/1967218874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d2c101fc0 0x7f1d2c19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:49.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.522+0000 7f1d30f6d640 1 --2- 172.21.15.12:0/1967218874 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1d140a3c30 0x7f1d140a60f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:49.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.522+0000 7f1d30f6d640 1 --2- 172.21.15.12:0/1967218874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1d2c100e70 0x7f1d2c069c50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:49.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.522+0000 7f1d30f6d640 1 -- 172.21.15.12:0/1967218874 >> 172.21.15.12:0/1967218874 conn(0x7f1d2c0fc3e0 msgr2=0x7f1d2c1053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:49.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.522+0000 7f1d30f6d640 1 -- 172.21.15.12:0/1967218874 shutdown_connections 2024-04-22T13:21:49.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:49.522+0000 7f1d30f6d640 1 -- 172.21.15.12:0/1967218874 wait complete. 2024-04-22T13:21:49.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:49 smithi012 ceph-mon[28265]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:49.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:49 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:49.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:49 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:49.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:49 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:49.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:49 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:49.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:49 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:49.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:49 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:49.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:49 smithi012 ceph-mon[28265]: Deploying daemon mds.i on smithi134 2024-04-22T13:21:49.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:49 smithi012 ceph-mon[28265]: mds.? [v2:172.21.15.111:6836/4218407470,v1:172.21.15.111:6837/4218407470] up:boot 2024-04-22T13:21:49.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:49 smithi012 ceph-mon[28265]: fsmap 8 up:standby 2024-04-22T13:21:49.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:49 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-22T13:21:49.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:49 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1967218874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:49.841 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:49 smithi111 ceph-mon[31361]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:49.841 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:49 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:49.841 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:49 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:49.842 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:49 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:49.842 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:49 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:49.842 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:49 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:49.842 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:49 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:49.842 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:49 smithi111 ceph-mon[31361]: Deploying daemon mds.i on smithi134 2024-04-22T13:21:49.842 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:49 smithi111 ceph-mon[31361]: mds.? [v2:172.21.15.111:6836/4218407470,v1:172.21.15.111:6837/4218407470] up:boot 2024-04-22T13:21:49.842 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:49 smithi111 ceph-mon[31361]: fsmap 8 up:standby 2024-04-22T13:21:49.842 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:49 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-22T13:21:49.842 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:49 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1967218874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:49.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:49 smithi134 ceph-mon[31338]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:49.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:49 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:49.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:49 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:49.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:49 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:49.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:49 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:49.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:49 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:49.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:49 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:49.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:49 smithi134 ceph-mon[31338]: Deploying daemon mds.i on smithi134 2024-04-22T13:21:49.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:49 smithi134 ceph-mon[31338]: mds.? [v2:172.21.15.111:6836/4218407470,v1:172.21.15.111:6837/4218407470] up:boot 2024-04-22T13:21:49.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:49 smithi134 ceph-mon[31338]: fsmap 8 up:standby 2024-04-22T13:21:49.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:49 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-22T13:21:49.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:49 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1967218874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:50.678 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 22 13:21:50 smithi134 systemd[1]: Starting Ceph mds.i for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:21:51.010 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 22 13:21:50 smithi134 podman[62786]: 2024-04-22 13:21:50.752408206 +0000 UTC m=+0.094383165 container create 59128799688f5b0448c78803b026b3666e987dc24535d5ed386fef2f9cde26c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-i, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-22T13:21:51.010 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 22 13:21:50 smithi134 podman[62786]: 2024-04-22 13:21:50.675828622 +0000 UTC m=+0.017803598 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:21:51.010 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 22 13:21:50 smithi134 podman[62786]: 2024-04-22 13:21:50.988162423 +0000 UTC m=+0.330137383 container init 59128799688f5b0448c78803b026b3666e987dc24535d5ed386fef2f9cde26c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-22T13:21:51.010 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 22 13:21:50 smithi134 podman[62786]: 2024-04-22 13:21:50.990625827 +0000 UTC m=+0.332600802 container start 59128799688f5b0448c78803b026b3666e987dc24535d5ed386fef2f9cde26c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-i, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-22T13:21:51.428 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 22 13:21:51 smithi134 ceph-mds[62805]: -- 172.21.15.134:0/885435716 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55968a615a40 con 0x55968a5e8800 2024-04-22T13:21:51.428 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 22 13:21:51 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-i[62801]: starting mds.i at 2024-04-22T13:21:51.428 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 22 13:21:51 smithi134 bash[62786]: 59128799688f5b0448c78803b026b3666e987dc24535d5ed386fef2f9cde26c3 2024-04-22T13:21:51.428 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 22 13:21:51 smithi134 systemd[1]: Started Ceph mds.i for 0aede5ea-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:21:51.532 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:21:51.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.637+0000 7faeb739d640 1 -- 172.21.15.12:0/1066576275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faeb0103180 msgr2=0x7faeb0103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:51.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.637+0000 7faeb739d640 1 --2- 172.21.15.12:0/1066576275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faeb0103180 0x7faeb0103600 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faea0009900 tx=0x7faea002bc30 comp rx=0 tx=0).stop 2024-04-22T13:21:51.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.637+0000 7faeb739d640 1 -- 172.21.15.12:0/1066576275 shutdown_connections 2024-04-22T13:21:51.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.637+0000 7faeb739d640 1 --2- 172.21.15.12:0/1066576275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faeb0103b40 0x7faeb010a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:51.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.637+0000 7faeb739d640 1 --2- 172.21.15.12:0/1066576275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faeb0103180 0x7faeb0103600 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:51.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.637+0000 7faeb739d640 1 --2- 172.21.15.12:0/1066576275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faeb0101f90 0x7faeb01023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:51.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.637+0000 7faeb739d640 1 -- 172.21.15.12:0/1066576275 >> 172.21.15.12:0/1066576275 conn(0x7faeb00fd500 msgr2=0x7faeb00ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:51.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.637+0000 7faeb739d640 1 -- 172.21.15.12:0/1066576275 shutdown_connections 2024-04-22T13:21:51.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.637+0000 7faeb739d640 1 -- 172.21.15.12:0/1066576275 wait complete. 2024-04-22T13:21:51.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.638+0000 7faeb739d640 1 Processor -- start 2024-04-22T13:21:51.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.638+0000 7faeb739d640 1 -- start start 2024-04-22T13:21:51.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.638+0000 7faeb739d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faeb0101f90 0x7faeb0075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:51.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.638+0000 7faeb739d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faeb0103180 0x7faeb0074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:51.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.638+0000 7faeb739d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faeb0103b40 0x7faeb0074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:51.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.638+0000 7faeb739d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7faeb010db50 con 0x7faeb0103180 2024-04-22T13:21:51.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.638+0000 7faeb739d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faeb010d9d0 con 0x7faeb0101f90 2024-04-22T13:21:51.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.638+0000 7faeb739d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faeb010dcd0 con 0x7faeb0103b40 2024-04-22T13:21:51.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.638+0000 7faeb4938640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faeb0103180 0x7faeb0074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:51.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.638+0000 7faeb4938640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faeb0103180 0x7faeb0074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36976/0 (socket says 172.21.15.12:36976) 2024-04-22T13:21:51.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.638+0000 7faeb4938640 1 -- 172.21.15.12:0/250193582 learned_addr learned my addr 172.21.15.12:0/250193582 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:21:51.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.638+0000 7faeb593a640 1 --2- 172.21.15.12:0/250193582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faeb0103b40 0x7faeb0074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:51.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.639+0000 7faeb5139640 1 --2- 172.21.15.12:0/250193582 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faeb0101f90 0x7faeb0075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:51.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.639+0000 7faeb4938640 1 -- 172.21.15.12:0/250193582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faeb0103b40 msgr2=0x7faeb0074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:51.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.639+0000 7faeb4938640 1 --2- 172.21.15.12:0/250193582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faeb0103b40 0x7faeb0074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:51.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.639+0000 7faeb4938640 1 -- 172.21.15.12:0/250193582 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faeb0101f90 msgr2=0x7faeb0075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:51.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.639+0000 7faeb4938640 1 --2- 172.21.15.12:0/250193582 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faeb0101f90 0x7faeb0075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:51.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.639+0000 7faeb4938640 1 -- 172.21.15.12:0/250193582 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb0075110 con 0x7faeb0103180 2024-04-22T13:21:51.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.639+0000 7faeb593a640 1 --2- 172.21.15.12:0/250193582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faeb0103b40 0x7faeb0074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:21:51.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.639+0000 7faeb4938640 1 --2- 172.21.15.12:0/250193582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faeb0103180 0x7faeb0074340 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7faea0006010 tx=0x7faea00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:51.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.639+0000 7faea67fc640 1 -- 172.21.15.12:0/250193582 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0043020 con 0x7faeb0103180 2024-04-22T13:21:51.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.639+0000 7faeb739d640 1 -- 172.21.15.12:0/250193582 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeb01a2310 con 0x7faeb0103180 2024-04-22T13:21:51.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.639+0000 7faea67fc640 1 -- 172.21.15.12:0/250193582 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faea0004aa0 con 0x7faeb0103180 2024-04-22T13:21:51.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.639+0000 7faea67fc640 1 -- 172.21.15.12:0/250193582 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0003c70 con 0x7faeb0103180 2024-04-22T13:21:51.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.639+0000 7faeb739d640 1 -- 172.21.15.12:0/250193582 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeb01a2930 con 0x7faeb0103180 2024-04-22T13:21:51.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.640+0000 7faeb739d640 1 -- 172.21.15.12:0/250193582 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeb0109330 con 0x7faeb0103180 2024-04-22T13:21:51.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.641+0000 7faea67fc640 1 -- 172.21.15.12:0/250193582 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7faea0046020 con 0x7faeb0103180 2024-04-22T13:21:51.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.642+0000 7faea67fc640 1 --2- 172.21.15.12:0/250193582 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fae9c0a3bb0 0x7fae9c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:51.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.642+0000 7faea67fc640 1 -- 172.21.15.12:0/250193582 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7faea00f23c0 con 0x7faeb0103180 2024-04-22T13:21:51.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.642+0000 7faeb5139640 1 --2- 172.21.15.12:0/250193582 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fae9c0a3bb0 0x7fae9c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:51.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.643+0000 7faeb5139640 1 --2- 172.21.15.12:0/250193582 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fae9c0a3bb0 0x7fae9c0a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7faeb0102fe0 tx=0x7faeac008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:51.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.647+0000 7faea67fc640 1 -- 172.21.15.12:0/250193582 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faea004d020 con 0x7faeb0103180 2024-04-22T13:21:51.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.886+0000 7faeb739d640 1 -- 172.21.15.12:0/250193582 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faeb002d5c0 con 0x7faeb0103180 2024-04-22T13:21:51.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.887+0000 7faea67fc640 1 -- 172.21.15.12:0/250193582 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8016 (secure 0 0 0) 0x7faea00bf240 con 0x7faeb0103180 2024-04-22T13:21:51.885 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:21:51.885 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-22T13:21:51.885 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 10 2024-04-22T13:21:51.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.889+0000 7faeb739d640 1 -- 172.21.15.12:0/250193582 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fae9c0a3bb0 msgr2=0x7fae9c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:51.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.889+0000 7faeb739d640 1 --2- 172.21.15.12:0/250193582 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fae9c0a3bb0 0x7fae9c0a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7faeb0102fe0 tx=0x7faeac008040 comp rx=0 tx=0).stop 2024-04-22T13:21:51.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.889+0000 7faeb739d640 1 -- 172.21.15.12:0/250193582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faeb0103180 msgr2=0x7faeb0074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:51.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.889+0000 7faeb739d640 1 --2- 172.21.15.12:0/250193582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faeb0103180 0x7faeb0074340 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7faea0006010 tx=0x7faea00042f0 comp rx=0 tx=0).stop 2024-04-22T13:21:51.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.889+0000 7faeb739d640 1 -- 172.21.15.12:0/250193582 shutdown_connections 2024-04-22T13:21:51.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.889+0000 7faeb739d640 1 --2- 172.21.15.12:0/250193582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faeb0103b40 0x7faeb0074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:51.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.889+0000 7faeb739d640 1 --2- 172.21.15.12:0/250193582 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fae9c0a3bb0 0x7fae9c0a6070 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:51.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.889+0000 7faeb739d640 1 --2- 172.21.15.12:0/250193582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faeb0103180 0x7faeb0074340 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:51.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.889+0000 7faeb739d640 1 --2- 172.21.15.12:0/250193582 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faeb0101f90 0x7faeb0075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:51.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.889+0000 7faeb739d640 1 -- 172.21.15.12:0/250193582 >> 172.21.15.12:0/250193582 conn(0x7faeb00fd500 msgr2=0x7faeb0106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:51.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.889+0000 7faeb739d640 1 -- 172.21.15.12:0/250193582 shutdown_connections 2024-04-22T13:21:51.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:51.889+0000 7faeb739d640 1 -- 172.21.15.12:0/250193582 wait complete. 2024-04-22T13:21:52.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:52 smithi134 ceph-mon[31338]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:52.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:52 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:52.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:52 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:52.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:52 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:52.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:52 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:52.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:52 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:52.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:52 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:52.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:52 smithi134 ceph-mon[31338]: Deploying daemon mds.g on smithi012 2024-04-22T13:21:52.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:52 smithi134 ceph-mon[31338]: mds.? [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] up:boot 2024-04-22T13:21:52.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:52 smithi134 ceph-mon[31338]: fsmap 9 up:standby 2024-04-22T13:21:52.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:52 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-22T13:21:52.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:52 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/250193582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:52.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:52 smithi111 ceph-mon[31361]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:52.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:52 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:52.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:52 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:52.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:52 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:52.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:52 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:52.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:52 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:52.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:52 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:52.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:52 smithi111 ceph-mon[31361]: Deploying daemon mds.g on smithi012 2024-04-22T13:21:52.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:52 smithi111 ceph-mon[31361]: mds.? [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] up:boot 2024-04-22T13:21:52.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:52 smithi111 ceph-mon[31361]: fsmap 9 up:standby 2024-04-22T13:21:52.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:52 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-22T13:21:52.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:52 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/250193582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:52.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:52 smithi012 ceph-mon[28265]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:52.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:52 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:52.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:52 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:52.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:52 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:52.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:52 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:52.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:52 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:52.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:52 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:52.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:52 smithi012 ceph-mon[28265]: Deploying daemon mds.g on smithi012 2024-04-22T13:21:52.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:52 smithi012 ceph-mon[28265]: mds.? [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] up:boot 2024-04-22T13:21:52.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:52 smithi012 ceph-mon[28265]: fsmap 9 up:standby 2024-04-22T13:21:52.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:52 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-22T13:21:52.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:52 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/250193582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:53.753 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 22 13:21:53 smithi012 podman[69973]: 2024-04-22 13:21:53.626368833 +0000 UTC m=+0.018763210 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:21:53.754 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 22 13:21:53 smithi012 podman[69973]: 2024-04-22 13:21:53.757463447 +0000 UTC m=+0.149857813 container create 2b9a2ed95a8dd6e490b8b24d126f5ad764b53b2ee98999a99d1219efcba650be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-22T13:21:53.899 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:21:54.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.007+0000 7f00c086d640 1 -- 172.21.15.12:0/1342372195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00bc101fc0 msgr2=0x7f00bc102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:54.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.007+0000 7f00c086d640 1 --2- 172.21.15.12:0/1342372195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00bc101fc0 0x7f00bc102440 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f00b0009900 tx=0x7f00b002bc30 comp rx=0 tx=0).stop 2024-04-22T13:21:54.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.007+0000 7f00c086d640 1 -- 172.21.15.12:0/1342372195 shutdown_connections 2024-04-22T13:21:54.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.007+0000 7f00c086d640 1 --2- 172.21.15.12:0/1342372195 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00bc102980 0x7f00bc1092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:54.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.007+0000 7f00c086d640 1 --2- 172.21.15.12:0/1342372195 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00bc101fc0 0x7f00bc102440 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:54.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.007+0000 7f00c086d640 1 --2- 172.21.15.12:0/1342372195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00bc100e70 0x7f00bc101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:54.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.007+0000 7f00c086d640 1 -- 172.21.15.12:0/1342372195 >> 172.21.15.12:0/1342372195 conn(0x7f00bc0fc3e0 msgr2=0x7f00bc0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:54.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.007+0000 7f00c086d640 1 -- 172.21.15.12:0/1342372195 shutdown_connections 2024-04-22T13:21:54.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.007+0000 7f00c086d640 1 -- 172.21.15.12:0/1342372195 wait complete. 2024-04-22T13:21:54.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.008+0000 7f00c086d640 1 Processor -- start 2024-04-22T13:21:54.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.008+0000 7f00c086d640 1 -- start start 2024-04-22T13:21:54.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.008+0000 7f00c086d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00bc100e70 0x7f00bc069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:54.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.008+0000 7f00c086d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00bc101fc0 0x7f00bc19cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:54.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.008+0000 7f00c086d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00bc102980 0x7f00bc19f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:54.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.008+0000 7f00c086d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc10c9d0 con 0x7f00bc101fc0 2024-04-22T13:21:54.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.008+0000 7f00c086d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc10c850 con 0x7f00bc102980 2024-04-22T13:21:54.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.008+0000 7f00c086d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc10cb50 con 0x7f00bc100e70 2024-04-22T13:21:54.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.008+0000 7f00b9d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00bc101fc0 0x7f00bc19cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:54.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.008+0000 7f00b9d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00bc101fc0 0x7f00bc19cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36996/0 (socket says 172.21.15.12:36996) 2024-04-22T13:21:54.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.008+0000 7f00b9d9b640 1 -- 172.21.15.12:0/3381318529 learned_addr learned my addr 172.21.15.12:0/3381318529 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:21:54.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.008+0000 7f00ba59c640 1 --2- 172.21.15.12:0/3381318529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00bc100e70 0x7f00bc069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:54.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.008+0000 7f00bad9d640 1 --2- 172.21.15.12:0/3381318529 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00bc102980 0x7f00bc19f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:54.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.008+0000 7f00b9d9b640 1 -- 172.21.15.12:0/3381318529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00bc100e70 msgr2=0x7f00bc069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:54.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.008+0000 7f00b9d9b640 1 --2- 172.21.15.12:0/3381318529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00bc100e70 0x7f00bc069c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:54.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.008+0000 7f00b9d9b640 1 -- 172.21.15.12:0/3381318529 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00bc102980 msgr2=0x7f00bc19f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:54.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.008+0000 7f00b9d9b640 1 --2- 172.21.15.12:0/3381318529 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00bc102980 0x7f00bc19f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:54.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.008+0000 7f00b9d9b640 1 -- 172.21.15.12:0/3381318529 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00bc19fe70 con 0x7f00bc101fc0 2024-04-22T13:21:54.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.008+0000 7f00bad9d640 1 --2- 172.21.15.12:0/3381318529 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00bc102980 0x7f00bc19f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:21:54.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.008+0000 7f00b9d9b640 1 --2- 172.21.15.12:0/3381318529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00bc101fc0 0x7f00bc19cf40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f00b0000c00 tx=0x7f00b0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:54.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.009+0000 7f00ab7fe640 1 -- 172.21.15.12:0/3381318529 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b0043020 con 0x7f00bc101fc0 2024-04-22T13:21:54.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.009+0000 7f00c086d640 1 -- 172.21.15.12:0/3381318529 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00bc1a00a0 con 0x7f00bc101fc0 2024-04-22T13:21:54.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.009+0000 7f00ab7fe640 1 -- 172.21.15.12:0/3381318529 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00b0004aa0 con 0x7f00bc101fc0 2024-04-22T13:21:54.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.009+0000 7f00ab7fe640 1 -- 172.21.15.12:0/3381318529 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b0003d00 con 0x7f00bc101fc0 2024-04-22T13:21:54.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.009+0000 7f00c086d640 1 -- 172.21.15.12:0/3381318529 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00bc1a05f0 con 0x7f00bc101fc0 2024-04-22T13:21:54.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.010+0000 7f00c086d640 1 -- 172.21.15.12:0/3381318529 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00bc1082a0 con 0x7f00bc101fc0 2024-04-22T13:21:54.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.011+0000 7f00ab7fe640 1 -- 172.21.15.12:0/3381318529 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f00b0046020 con 0x7f00bc101fc0 2024-04-22T13:21:54.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.011+0000 7f00ab7fe640 1 --2- 172.21.15.12:0/3381318529 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f00a40a3bb0 0x7f00a40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:54.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.012+0000 7f00ba59c640 1 --2- 172.21.15.12:0/3381318529 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f00a40a3bb0 0x7f00a40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:54.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.012+0000 7f00ab7fe640 1 -- 172.21.15.12:0/3381318529 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f00b00f2980 con 0x7f00bc101fc0 2024-04-22T13:21:54.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.012+0000 7f00ba59c640 1 --2- 172.21.15.12:0/3381318529 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f00a40a3bb0 0x7f00a40a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f00ac005d00 tx=0x7f00ac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:54.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.015+0000 7f00ab7fe640 1 -- 172.21.15.12:0/3381318529 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f00b004d020 con 0x7f00bc101fc0 2024-04-22T13:21:54.027 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 22 13:21:54 smithi012 podman[69973]: 2024-04-22 13:21:54.00196773 +0000 UTC m=+0.394362107 container init 2b9a2ed95a8dd6e490b8b24d126f5ad764b53b2ee98999a99d1219efcba650be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2) 2024-04-22T13:21:54.027 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 22 13:21:54 smithi012 podman[69973]: 2024-04-22 13:21:54.005185686 +0000 UTC m=+0.397580052 container start 2b9a2ed95a8dd6e490b8b24d126f5ad764b53b2ee98999a99d1219efcba650be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-22T13:21:54.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.255+0000 7f00c086d640 1 -- 172.21.15.12:0/3381318529 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00bc02d660 con 0x7f00bc101fc0 2024-04-22T13:21:54.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.256+0000 7f00ab7fe640 1 -- 172.21.15.12:0/3381318529 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8842 (secure 0 0 0) 0x7f00b00bf800 con 0x7f00bc101fc0 2024-04-22T13:21:54.254 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:21:54.254 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-22T13:21:54.254 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 11 2024-04-22T13:21:54.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.258+0000 7f00c086d640 1 -- 172.21.15.12:0/3381318529 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f00a40a3bb0 msgr2=0x7f00a40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:54.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.258+0000 7f00c086d640 1 --2- 172.21.15.12:0/3381318529 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f00a40a3bb0 0x7f00a40a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f00ac005d00 tx=0x7f00ac00b410 comp rx=0 tx=0).stop 2024-04-22T13:21:54.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.258+0000 7f00c086d640 1 -- 172.21.15.12:0/3381318529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00bc101fc0 msgr2=0x7f00bc19cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:54.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.258+0000 7f00c086d640 1 --2- 172.21.15.12:0/3381318529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00bc101fc0 0x7f00bc19cf40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f00b0000c00 tx=0x7f00b0004270 comp rx=0 tx=0).stop 2024-04-22T13:21:54.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.258+0000 7f00c086d640 1 -- 172.21.15.12:0/3381318529 shutdown_connections 2024-04-22T13:21:54.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.258+0000 7f00c086d640 1 --2- 172.21.15.12:0/3381318529 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00bc102980 0x7f00bc19f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:54.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.258+0000 7f00c086d640 1 --2- 172.21.15.12:0/3381318529 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f00a40a3bb0 0x7f00a40a6070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:54.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.258+0000 7f00c086d640 1 --2- 172.21.15.12:0/3381318529 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f00bc101fc0 0x7f00bc19cf40 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:54.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.258+0000 7f00c086d640 1 --2- 172.21.15.12:0/3381318529 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00bc100e70 0x7f00bc069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:54.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.258+0000 7f00c086d640 1 -- 172.21.15.12:0/3381318529 >> 172.21.15.12:0/3381318529 conn(0x7f00bc0fc3e0 msgr2=0x7f00bc105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:54.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.258+0000 7f00c086d640 1 -- 172.21.15.12:0/3381318529 shutdown_connections 2024-04-22T13:21:54.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:54.258+0000 7f00c086d640 1 -- 172.21.15.12:0/3381318529 wait complete. 2024-04-22T13:21:54.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:54 smithi012 ceph-mon[28265]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:54.338 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 22 13:21:54 smithi012 ceph-mds[70015]: -- 172.21.15.12:0/2718981872 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b6d4831a40 con 0x55b6d4805000 2024-04-22T13:21:54.338 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 22 13:21:54 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-g[69989]: starting mds.g at 2024-04-22T13:21:54.338 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 22 13:21:54 smithi012 bash[69973]: 2b9a2ed95a8dd6e490b8b24d126f5ad764b53b2ee98999a99d1219efcba650be 2024-04-22T13:21:54.338 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 22 13:21:54 smithi012 systemd[1]: Started Ceph mds.g for 0aede5ea-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:21:54.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:54 smithi134 ceph-mon[31338]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:54.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:54 smithi111 ceph-mon[31361]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:55.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:55 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:55.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:55 smithi012 ceph-mon[28265]: mds.? [v2:172.21.15.12:6838/4083188176,v1:172.21.15.12:6839/4083188176] up:boot 2024-04-22T13:21:55.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:55 smithi012 ceph-mon[28265]: fsmap 10 up:standby 2024-04-22T13:21:55.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:55 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-22T13:21:55.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:55 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:55.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:55 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:55.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:55 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:55.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:55 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:55.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:55 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:55.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:55 smithi012 ceph-mon[28265]: Deploying daemon mds.a on smithi012 2024-04-22T13:21:55.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:55 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3381318529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:55.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:55 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:55.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:55 smithi134 ceph-mon[31338]: mds.? [v2:172.21.15.12:6838/4083188176,v1:172.21.15.12:6839/4083188176] up:boot 2024-04-22T13:21:55.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:55 smithi134 ceph-mon[31338]: fsmap 10 up:standby 2024-04-22T13:21:55.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:55 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-22T13:21:55.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:55 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:55.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:55 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:55.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:55 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:55.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:55 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:55.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:55 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:55.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:55 smithi134 ceph-mon[31338]: Deploying daemon mds.a on smithi012 2024-04-22T13:21:55.429 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:55 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/3381318529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:55.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:55 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:55.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:55 smithi111 ceph-mon[31361]: mds.? [v2:172.21.15.12:6838/4083188176,v1:172.21.15.12:6839/4083188176] up:boot 2024-04-22T13:21:55.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:55 smithi111 ceph-mon[31361]: fsmap 10 up:standby 2024-04-22T13:21:55.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:55 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-22T13:21:55.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:55 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:55.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:55 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:55.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:55 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:55.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:55 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:55.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:55 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:55.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:55 smithi111 ceph-mon[31361]: Deploying daemon mds.a on smithi012 2024-04-22T13:21:55.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:55 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/3381318529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:56.268 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:21:56.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.411+0000 7ff4d65d5640 1 -- 172.21.15.12:0/1014407429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4d0101fd0 msgr2=0x7ff4d0102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:56.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.411+0000 7ff4d65d5640 1 --2- 172.21.15.12:0/1014407429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4d0101fd0 0x7ff4d0102450 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009900 tx=0x7ff4c402bc30 comp rx=0 tx=0).stop 2024-04-22T13:21:56.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.411+0000 7ff4d65d5640 1 -- 172.21.15.12:0/1014407429 shutdown_connections 2024-04-22T13:21:56.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.411+0000 7ff4d65d5640 1 --2- 172.21.15.12:0/1014407429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4d0102990 0x7ff4d01092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:56.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.411+0000 7ff4d65d5640 1 --2- 172.21.15.12:0/1014407429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4d0101fd0 0x7ff4d0102450 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:56.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.411+0000 7ff4d65d5640 1 --2- 172.21.15.12:0/1014407429 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4d0100e80 0x7ff4d01012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:56.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.411+0000 7ff4d65d5640 1 -- 172.21.15.12:0/1014407429 >> 172.21.15.12:0/1014407429 conn(0x7ff4d00fc410 msgr2=0x7ff4d00fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:56.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.411+0000 7ff4d65d5640 1 -- 172.21.15.12:0/1014407429 shutdown_connections 2024-04-22T13:21:56.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.411+0000 7ff4d65d5640 1 -- 172.21.15.12:0/1014407429 wait complete. 2024-04-22T13:21:56.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.411+0000 7ff4d65d5640 1 Processor -- start 2024-04-22T13:21:56.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.411+0000 7ff4d65d5640 1 -- start start 2024-04-22T13:21:56.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.412+0000 7ff4d65d5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4d0100e80 0x7ff4d0198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:56.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.412+0000 7ff4d65d5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4d0101fd0 0x7ff4d01986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:56.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.412+0000 7ff4d65d5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4d0102990 0x7ff4d019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:56.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.412+0000 7ff4d65d5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d010ca10 con 0x7ff4d0101fd0 2024-04-22T13:21:56.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.412+0000 7ff4d65d5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d010c890 con 0x7ff4d0102990 2024-04-22T13:21:56.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.412+0000 7ff4d65d5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d010cb90 con 0x7ff4d0100e80 2024-04-22T13:21:56.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.412+0000 7ff4cffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4d0100e80 0x7ff4d0198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:56.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.412+0000 7ff4d4b72640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4d0102990 0x7ff4d019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:56.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.412+0000 7ff4cffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4d0100e80 0x7ff4d0198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.12:46912/0 (socket says 172.21.15.12:46912) 2024-04-22T13:21:56.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.412+0000 7ff4cf7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4d0101fd0 0x7ff4d01986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:56.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.412+0000 7ff4cffff640 1 -- 172.21.15.12:0/3635578932 learned_addr learned my addr 172.21.15.12:0/3635578932 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:21:56.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.412+0000 7ff4cf7fe640 1 -- 172.21.15.12:0/3635578932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4d0100e80 msgr2=0x7ff4d0198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:56.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.412+0000 7ff4cf7fe640 1 --2- 172.21.15.12:0/3635578932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4d0100e80 0x7ff4d0198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:56.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.412+0000 7ff4cf7fe640 1 -- 172.21.15.12:0/3635578932 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4d0102990 msgr2=0x7ff4d019f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:56.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.412+0000 7ff4cf7fe640 1 --2- 172.21.15.12:0/3635578932 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4d0102990 0x7ff4d019f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:56.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.412+0000 7ff4cf7fe640 1 -- 172.21.15.12:0/3635578932 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d019fd90 con 0x7ff4d0101fd0 2024-04-22T13:21:56.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.413+0000 7ff4cf7fe640 1 --2- 172.21.15.12:0/3635578932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4d0101fd0 0x7ff4d01986b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4036040 tx=0x7ff4c40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:56.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.413+0000 7ff4cd7fa640 1 -- 172.21.15.12:0/3635578932 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4043020 con 0x7ff4d0101fd0 2024-04-22T13:21:56.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.413+0000 7ff4d65d5640 1 -- 172.21.15.12:0/3635578932 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4d01a0020 con 0x7ff4d0101fd0 2024-04-22T13:21:56.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.413+0000 7ff4d65d5640 1 -- 172.21.15.12:0/3635578932 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4d01a0590 con 0x7ff4d0101fd0 2024-04-22T13:21:56.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.413+0000 7ff4cd7fa640 1 -- 172.21.15.12:0/3635578932 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4c4004a80 con 0x7ff4d0101fd0 2024-04-22T13:21:56.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.413+0000 7ff4cd7fa640 1 -- 172.21.15.12:0/3635578932 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4003ca0 con 0x7ff4d0101fd0 2024-04-22T13:21:56.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.414+0000 7ff4d65d5640 1 -- 172.21.15.12:0/3635578932 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4d01a0900 con 0x7ff4d0101fd0 2024-04-22T13:21:56.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.415+0000 7ff4cd7fa640 1 -- 172.21.15.12:0/3635578932 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff4c4046020 con 0x7ff4d0101fd0 2024-04-22T13:21:56.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.416+0000 7ff4cd7fa640 1 --2- 172.21.15.12:0/3635578932 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ff4b80a3c30 0x7ff4b80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:56.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.416+0000 7ff4cd7fa640 1 -- 172.21.15.12:0/3635578932 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7ff4c40f2710 con 0x7ff4d0101fd0 2024-04-22T13:21:56.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.416+0000 7ff4cffff640 1 --2- 172.21.15.12:0/3635578932 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ff4b80a3c30 0x7ff4b80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:56.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.416+0000 7ff4cffff640 1 --2- 172.21.15.12:0/3635578932 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ff4b80a3c30 0x7ff4b80a60f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff4c000ac90 tx=0x7ff4c000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:56.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.420+0000 7ff4cd7fa640 1 -- 172.21.15.12:0/3635578932 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff4c404d020 con 0x7ff4d0101fd0 2024-04-22T13:21:56.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:56 smithi134 ceph-mon[31338]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:56.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:56 smithi111 ceph-mon[31361]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:56.565 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:56 smithi012 ceph-mon[28265]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:56.566 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 22 13:21:56 smithi012 systemd[1]: Starting Ceph mds.a for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:21:56.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.655+0000 7ff4d65d5640 1 -- 172.21.15.12:0/3635578932 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4d01a0ad0 con 0x7ff4d0101fd0 2024-04-22T13:21:56.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.655+0000 7ff4cd7fa640 1 -- 172.21.15.12:0/3635578932 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8842 (secure 0 0 0) 0x7ff4c40bf590 con 0x7ff4d0101fd0 2024-04-22T13:21:56.653 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:21:56.653 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-22T13:21:56.653 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 11 2024-04-22T13:21:56.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.657+0000 7ff4d65d5640 1 -- 172.21.15.12:0/3635578932 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ff4b80a3c30 msgr2=0x7ff4b80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:56.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.657+0000 7ff4d65d5640 1 --2- 172.21.15.12:0/3635578932 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ff4b80a3c30 0x7ff4b80a60f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff4c000ac90 tx=0x7ff4c000a380 comp rx=0 tx=0).stop 2024-04-22T13:21:56.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.657+0000 7ff4d65d5640 1 -- 172.21.15.12:0/3635578932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4d0101fd0 msgr2=0x7ff4d01986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:56.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.657+0000 7ff4d65d5640 1 --2- 172.21.15.12:0/3635578932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4d0101fd0 0x7ff4d01986b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4036040 tx=0x7ff4c40042f0 comp rx=0 tx=0).stop 2024-04-22T13:21:56.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.658+0000 7ff4d65d5640 1 -- 172.21.15.12:0/3635578932 shutdown_connections 2024-04-22T13:21:56.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.658+0000 7ff4d65d5640 1 --2- 172.21.15.12:0/3635578932 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4d0102990 0x7ff4d019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:56.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.658+0000 7ff4d65d5640 1 --2- 172.21.15.12:0/3635578932 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ff4b80a3c30 0x7ff4b80a60f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:56.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.658+0000 7ff4d65d5640 1 --2- 172.21.15.12:0/3635578932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff4d0101fd0 0x7ff4d01986b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:56.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.658+0000 7ff4d65d5640 1 --2- 172.21.15.12:0/3635578932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4d0100e80 0x7ff4d0198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:56.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.658+0000 7ff4d65d5640 1 -- 172.21.15.12:0/3635578932 >> 172.21.15.12:0/3635578932 conn(0x7ff4d00fc410 msgr2=0x7ff4d00fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:56.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.658+0000 7ff4d65d5640 1 -- 172.21.15.12:0/3635578932 shutdown_connections 2024-04-22T13:21:56.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:56.658+0000 7ff4d65d5640 1 -- 172.21.15.12:0/3635578932 wait complete. 2024-04-22T13:21:56.827 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 22 13:21:56 smithi012 podman[70359]: 2024-04-22 13:21:56.569306453 +0000 UTC m=+0.115449764 container create 130122b5463070ae43d225b0e631940da7b15627ae7a59665d73b58d86f0d4bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-a, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T13:21:56.828 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 22 13:21:56 smithi012 podman[70359]: 2024-04-22 13:21:56.47272087 +0000 UTC m=+0.018864195 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:21:56.828 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 22 13:21:56 smithi012 podman[70359]: 2024-04-22 13:21:56.831158641 +0000 UTC m=+0.377301964 container init 130122b5463070ae43d225b0e631940da7b15627ae7a59665d73b58d86f0d4bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-22T13:21:57.087 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 22 13:21:56 smithi012 podman[70359]: 2024-04-22 13:21:56.834450339 +0000 UTC m=+0.380593658 container start 130122b5463070ae43d225b0e631940da7b15627ae7a59665d73b58d86f0d4bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-a, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-22T13:21:57.088 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 22 13:21:56 smithi012 ceph-mds[70380]: -- 172.21.15.12:0/398367722 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5571975f1a40 con 0x5571975c5000 2024-04-22T13:21:57.088 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 22 13:21:56 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-a[70376]: starting mds.a at 2024-04-22T13:21:57.088 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 22 13:21:56 smithi012 bash[70359]: 130122b5463070ae43d225b0e631940da7b15627ae7a59665d73b58d86f0d4bb 2024-04-22T13:21:57.088 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 22 13:21:56 smithi012 systemd[1]: Started Ceph mds.a for 0aede5ea-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:21:57.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:57 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/3635578932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:57.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:57 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:57.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:57 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:57.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:57 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:57.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:57 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:57.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:57 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:57.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:57 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:57.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:57 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3635578932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:57.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:57 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:57.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:57 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:57.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:57 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:57.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:57 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:57.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:57 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:57.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:57 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:57.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:57 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/3635578932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:57.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:57 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:57.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:57 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:57.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:57 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:21:57.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:57 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-22T13:21:57.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:57 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-22T13:21:57.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:57 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:21:58.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:58 smithi134 ceph-mon[31338]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:58.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:58 smithi134 ceph-mon[31338]: Deploying daemon mds.b on smithi111 2024-04-22T13:21:58.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:58 smithi134 ceph-mon[31338]: mds.? [v2:172.21.15.12:6840/763294965,v1:172.21.15.12:6841/763294965] up:boot 2024-04-22T13:21:58.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:58 smithi134 ceph-mon[31338]: fsmap 11 up:standby 2024-04-22T13:21:58.428 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:58 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-22T13:21:58.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:58 smithi111 ceph-mon[31361]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:58.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:58 smithi111 ceph-mon[31361]: Deploying daemon mds.b on smithi111 2024-04-22T13:21:58.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:58 smithi111 ceph-mon[31361]: mds.? [v2:172.21.15.12:6840/763294965,v1:172.21.15.12:6841/763294965] up:boot 2024-04-22T13:21:58.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:58 smithi111 ceph-mon[31361]: fsmap 11 up:standby 2024-04-22T13:21:58.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:58 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-22T13:21:58.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:58 smithi012 ceph-mon[28265]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:21:58.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:58 smithi012 ceph-mon[28265]: Deploying daemon mds.b on smithi111 2024-04-22T13:21:58.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:58 smithi012 ceph-mon[28265]: mds.? [v2:172.21.15.12:6840/763294965,v1:172.21.15.12:6841/763294965] up:boot 2024-04-22T13:21:58.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:58 smithi012 ceph-mon[28265]: fsmap 11 up:standby 2024-04-22T13:21:58.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:58 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-22T13:21:58.666 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:21:58.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.786+0000 7f0016c43640 1 -- 172.21.15.12:0/3921844271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0010100830 msgr2=0x7f0010100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:58.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.786+0000 7f0016c43640 1 --2- 172.21.15.12:0/3921844271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0010100830 0x7f0010100c50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efff8009900 tx=0x7efff802bc30 comp rx=0 tx=0).stop 2024-04-22T13:21:58.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.786+0000 7f0016c43640 1 -- 172.21.15.12:0/3921844271 shutdown_connections 2024-04-22T13:21:58.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.786+0000 7f0016c43640 1 --2- 172.21.15.12:0/3921844271 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0010101b30 0x7f0010101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:58.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.786+0000 7f0016c43640 1 --2- 172.21.15.12:0/3921844271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0010101190 0x7f00101015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:58.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.786+0000 7f0016c43640 1 --2- 172.21.15.12:0/3921844271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0010100830 0x7f0010100c50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:58.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.786+0000 7f0016c43640 1 -- 172.21.15.12:0/3921844271 >> 172.21.15.12:0/3921844271 conn(0x7f00100fc470 msgr2=0x7f00100fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:58.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.786+0000 7f0016c43640 1 -- 172.21.15.12:0/3921844271 shutdown_connections 2024-04-22T13:21:58.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.786+0000 7f0016c43640 1 -- 172.21.15.12:0/3921844271 wait complete. 2024-04-22T13:21:58.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.787+0000 7f0016c43640 1 Processor -- start 2024-04-22T13:21:58.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.787+0000 7f0016c43640 1 -- start start 2024-04-22T13:21:58.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.787+0000 7f0016c43640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0010100830 0x7f001019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:58.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.787+0000 7f0016c43640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0010101190 0x7f001019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:58.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.787+0000 7f0016c43640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0010101b30 0x7f00101a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:58.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.787+0000 7f0016c43640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f001010fd60 con 0x7f0010101b30 2024-04-22T13:21:58.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.787+0000 7f0016c43640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f001010fbe0 con 0x7f0010100830 2024-04-22T13:21:58.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.787+0000 7f0016c43640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f001010fee0 con 0x7f0010101190 2024-04-22T13:21:58.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.787+0000 7f00151e0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0010101b30 0x7f00101a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:58.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.787+0000 7f00149df640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0010100830 0x7f001019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:58.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.787+0000 7f00151e0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0010101b30 0x7f00101a1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37066/0 (socket says 172.21.15.12:37066) 2024-04-22T13:21:58.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.787+0000 7f00151e0640 1 -- 172.21.15.12:0/1548087132 learned_addr learned my addr 172.21.15.12:0/1548087132 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:21:58.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.787+0000 7f000ffff640 1 --2- 172.21.15.12:0/1548087132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0010101190 0x7f001019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:58.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.788+0000 7f00151e0640 1 -- 172.21.15.12:0/1548087132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0010101190 msgr2=0x7f001019a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:58.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.788+0000 7f00151e0640 1 --2- 172.21.15.12:0/1548087132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0010101190 0x7f001019a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:58.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.788+0000 7f00151e0640 1 -- 172.21.15.12:0/1548087132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0010100830 msgr2=0x7f001019a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:58.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.788+0000 7f00151e0640 1 --2- 172.21.15.12:0/1548087132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0010100830 0x7f001019a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:58.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.788+0000 7f00151e0640 1 -- 172.21.15.12:0/1548087132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00101a1fd0 con 0x7f0010101b30 2024-04-22T13:21:58.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.788+0000 7f00149df640 1 --2- 172.21.15.12:0/1548087132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0010100830 0x7f001019a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:21:58.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.788+0000 7f00151e0640 1 --2- 172.21.15.12:0/1548087132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0010101b30 0x7f00101a1960 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f000800d950 tx=0x7f000800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:58.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.788+0000 7f000dffb640 1 -- 172.21.15.12:0/1548087132 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0008017030 con 0x7f0010101b30 2024-04-22T13:21:58.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.788+0000 7f0016c43640 1 -- 172.21.15.12:0/1548087132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00101a22c0 con 0x7f0010101b30 2024-04-22T13:21:58.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.788+0000 7f000dffb640 1 -- 172.21.15.12:0/1548087132 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0008004020 con 0x7f0010101b30 2024-04-22T13:21:58.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.788+0000 7f000dffb640 1 -- 172.21.15.12:0/1548087132 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0008003b50 con 0x7f0010101b30 2024-04-22T13:21:58.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.788+0000 7f0016c43640 1 -- 172.21.15.12:0/1548087132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00101a27c0 con 0x7f0010101b30 2024-04-22T13:21:58.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.789+0000 7f0016c43640 1 -- 172.21.15.12:0/1548087132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f001010b700 con 0x7f0010101b30 2024-04-22T13:21:58.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.790+0000 7f000dffb640 1 -- 172.21.15.12:0/1548087132 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f000801a070 con 0x7f0010101b30 2024-04-22T13:21:58.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.791+0000 7f000dffb640 1 --2- 172.21.15.12:0/1548087132 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efffc0a3bb0 0x7efffc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:21:58.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.791+0000 7f00149df640 1 --2- 172.21.15.12:0/1548087132 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efffc0a3bb0 0x7efffc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:21:58.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.791+0000 7f000dffb640 1 -- 172.21.15.12:0/1548087132 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f00080cf720 con 0x7f0010101b30 2024-04-22T13:21:58.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.792+0000 7f00149df640 1 --2- 172.21.15.12:0/1548087132 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efffc0a3bb0 0x7efffc0a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efff8000c00 tx=0x7efff802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:21:58.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:58.796+0000 7f000dffb640 1 -- 172.21.15.12:0/1548087132 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0008021070 con 0x7f0010101b30 2024-04-22T13:21:59.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:59.023+0000 7f0016c43640 1 -- 172.21.15.12:0/1548087132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0010062b50 con 0x7f0010101b30 2024-04-22T13:21:59.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:59.024+0000 7f000dffb640 1 -- 172.21.15.12:0/1548087132 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9665 (secure 0 0 0) 0x7f000809c500 con 0x7f0010101b30 2024-04-22T13:21:59.021 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:21:59.021 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-22T13:21:59.022 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 12 2024-04-22T13:21:59.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:59.025+0000 7f0016c43640 1 -- 172.21.15.12:0/1548087132 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efffc0a3bb0 msgr2=0x7efffc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:59.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:59.026+0000 7f0016c43640 1 --2- 172.21.15.12:0/1548087132 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efffc0a3bb0 0x7efffc0a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efff8000c00 tx=0x7efff802c7e0 comp rx=0 tx=0).stop 2024-04-22T13:21:59.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:59.026+0000 7f0016c43640 1 -- 172.21.15.12:0/1548087132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0010101b30 msgr2=0x7f00101a1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:21:59.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:59.026+0000 7f0016c43640 1 --2- 172.21.15.12:0/1548087132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0010101b30 0x7f00101a1960 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f000800d950 tx=0x7f000800de10 comp rx=0 tx=0).stop 2024-04-22T13:21:59.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:59.026+0000 7f0016c43640 1 -- 172.21.15.12:0/1548087132 shutdown_connections 2024-04-22T13:21:59.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:59.026+0000 7f0016c43640 1 --2- 172.21.15.12:0/1548087132 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efffc0a3bb0 0x7efffc0a6070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:59.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:59.026+0000 7f0016c43640 1 --2- 172.21.15.12:0/1548087132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0010101b30 0x7f00101a1960 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:59.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:59.026+0000 7f0016c43640 1 --2- 172.21.15.12:0/1548087132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0010101190 0x7f001019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:59.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:59.026+0000 7f0016c43640 1 --2- 172.21.15.12:0/1548087132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0010100830 0x7f001019a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:21:59.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:59.026+0000 7f0016c43640 1 -- 172.21.15.12:0/1548087132 >> 172.21.15.12:0/1548087132 conn(0x7f00100fc470 msgr2=0x7f00100fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:21:59.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:59.026+0000 7f0016c43640 1 -- 172.21.15.12:0/1548087132 shutdown_connections 2024-04-22T13:21:59.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:21:59.026+0000 7f0016c43640 1 -- 172.21.15.12:0/1548087132 wait complete. 2024-04-22T13:21:59.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:21:59 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1548087132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:59.450 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:21:59 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1548087132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:59.451 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 22 13:21:59 smithi111 systemd[1]: Starting Ceph mds.b for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:21:59.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:21:59 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1548087132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:21:59.715 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 22 13:21:59 smithi111 podman[62176]: 2024-04-22 13:21:59.356721271 +0000 UTC m=+0.017866075 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:21:59.715 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 22 13:21:59 smithi111 podman[62176]: 2024-04-22 13:21:59.475068786 +0000 UTC m=+0.136213580 container create ee7dfcb8429a7318584ae12e241b31777c839341a5a86c9838940194ceec7517 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-22T13:21:59.715 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 22 13:21:59 smithi111 podman[62176]: 2024-04-22 13:21:59.693928227 +0000 UTC m=+0.355073020 container init ee7dfcb8429a7318584ae12e241b31777c839341a5a86c9838940194ceec7517 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-22T13:21:59.715 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 22 13:21:59 smithi111 podman[62176]: 2024-04-22 13:21:59.696479671 +0000 UTC m=+0.357624480 container start ee7dfcb8429a7318584ae12e241b31777c839341a5a86c9838940194ceec7517 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T13:21:59.977 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 22 13:21:59 smithi111 ceph-mds[62195]: -- 172.21.15.111:0/231030077 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5575e378fa40 con 0x5575e3762800 2024-04-22T13:21:59.977 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 22 13:21:59 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-b[62191]: starting mds.b at 2024-04-22T13:21:59.977 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 22 13:21:59 smithi111 bash[62176]: ee7dfcb8429a7318584ae12e241b31777c839341a5a86c9838940194ceec7517 2024-04-22T13:21:59.977 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 22 13:21:59 smithi111 systemd[1]: Started Ceph mds.b for 0aede5ea-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:22:00.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:00 smithi111 ceph-mon[31361]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:22:00.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:00 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:00.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:00 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:00.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:00 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:00.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:00 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:00.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:00 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:00.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:00 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:22:00.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:00 smithi012 ceph-mon[28265]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:22:00.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:00 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:00.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:00 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:00.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:00 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:00.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:00 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:00.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:00 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:00.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:00 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:22:00.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:00 smithi134 ceph-mon[31338]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:22:00.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:00 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:00.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:00 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:00.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:00 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:00.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:00 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:00.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:00 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:00.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:00 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:22:01.036 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:22:01.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.144+0000 7fe1f31b2640 1 -- 172.21.15.12:0/3421426130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1ec102a80 msgr2=0x7fe1ec104e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:01.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.144+0000 7fe1f31b2640 1 --2- 172.21.15.12:0/3421426130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1ec102a80 0x7fe1ec104e70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0009900 tx=0x7fe1e002bc30 comp rx=0 tx=0).stop 2024-04-22T13:22:01.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.144+0000 7fe1f31b2640 1 -- 172.21.15.12:0/3421426130 shutdown_connections 2024-04-22T13:22:01.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.144+0000 7fe1f31b2640 1 --2- 172.21.15.12:0/3421426130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1ec107ce0 0x7fe1ec10a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:01.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.144+0000 7fe1f31b2640 1 --2- 172.21.15.12:0/3421426130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1ec1053b0 0x7fe1ec1077a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:01.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.144+0000 7fe1f31b2640 1 --2- 172.21.15.12:0/3421426130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1ec102a80 0x7fe1ec104e70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:01.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.144+0000 7fe1f31b2640 1 -- 172.21.15.12:0/3421426130 >> 172.21.15.12:0/3421426130 conn(0x7fe1ec0fc410 msgr2=0x7fe1ec0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:01.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.144+0000 7fe1f31b2640 1 -- 172.21.15.12:0/3421426130 shutdown_connections 2024-04-22T13:22:01.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.144+0000 7fe1f31b2640 1 -- 172.21.15.12:0/3421426130 wait complete. 2024-04-22T13:22:01.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.144+0000 7fe1f31b2640 1 Processor -- start 2024-04-22T13:22:01.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.144+0000 7fe1f31b2640 1 -- start start 2024-04-22T13:22:01.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.145+0000 7fe1f31b2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1ec102a80 0x7fe1ec1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:01.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.145+0000 7fe1f31b2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1ec1053b0 0x7fe1ec198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:01.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.145+0000 7fe1f31b2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1ec107ce0 0x7fe1ec19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:01.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.145+0000 7fe1f31b2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ec10dae0 con 0x7fe1ec107ce0 2024-04-22T13:22:01.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.145+0000 7fe1f31b2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ec10d960 con 0x7fe1ec1053b0 2024-04-22T13:22:01.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.145+0000 7fe1f31b2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ec10dc60 con 0x7fe1ec102a80 2024-04-22T13:22:01.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.145+0000 7fe1f174f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1ec107ce0 0x7fe1ec19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:01.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.145+0000 7fe1f0f4e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1ec102a80 0x7fe1ec1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:01.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.145+0000 7fe1f174f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1ec107ce0 0x7fe1ec19f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37094/0 (socket says 172.21.15.12:37094) 2024-04-22T13:22:01.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.145+0000 7fe1f174f640 1 -- 172.21.15.12:0/3847267867 learned_addr learned my addr 172.21.15.12:0/3847267867 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:01.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.145+0000 7fe1ebfff640 1 --2- 172.21.15.12:0/3847267867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1ec1053b0 0x7fe1ec198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:01.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.145+0000 7fe1f174f640 1 -- 172.21.15.12:0/3847267867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1ec102a80 msgr2=0x7fe1ec1981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:01.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.145+0000 7fe1f174f640 1 --2- 172.21.15.12:0/3847267867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1ec102a80 0x7fe1ec1981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:01.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.145+0000 7fe1f174f640 1 -- 172.21.15.12:0/3847267867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1ec1053b0 msgr2=0x7fe1ec198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:01.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.145+0000 7fe1f174f640 1 --2- 172.21.15.12:0/3847267867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1ec1053b0 0x7fe1ec198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:01.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.145+0000 7fe1f174f640 1 -- 172.21.15.12:0/3847267867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1ec19fe10 con 0x7fe1ec107ce0 2024-04-22T13:22:01.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.145+0000 7fe1ebfff640 1 --2- 172.21.15.12:0/3847267867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1ec1053b0 0x7fe1ec198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:22:01.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.145+0000 7fe1f174f640 1 --2- 172.21.15.12:0/3847267867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1ec107ce0 0x7fe1ec19f7a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400cad0 tx=0x7fe1e400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:01.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.145+0000 7fe1e9ffb640 1 -- 172.21.15.12:0/3847267867 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e400ae20 con 0x7fe1ec107ce0 2024-04-22T13:22:01.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.145+0000 7fe1e9ffb640 1 -- 172.21.15.12:0/3847267867 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1e4005020 con 0x7fe1ec107ce0 2024-04-22T13:22:01.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.145+0000 7fe1f31b2640 1 -- 172.21.15.12:0/3847267867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1ec1a0100 con 0x7fe1ec107ce0 2024-04-22T13:22:01.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.145+0000 7fe1e9ffb640 1 -- 172.21.15.12:0/3847267867 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e400f6b0 con 0x7fe1ec107ce0 2024-04-22T13:22:01.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.146+0000 7fe1f31b2640 1 -- 172.21.15.12:0/3847267867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1ec1a0600 con 0x7fe1ec107ce0 2024-04-22T13:22:01.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.147+0000 7fe1f31b2640 1 -- 172.21.15.12:0/3847267867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1ec0fdf00 con 0x7fe1ec107ce0 2024-04-22T13:22:01.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.148+0000 7fe1e9ffb640 1 -- 172.21.15.12:0/3847267867 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe1e40051d0 con 0x7fe1ec107ce0 2024-04-22T13:22:01.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.148+0000 7fe1e9ffb640 1 --2- 172.21.15.12:0/3847267867 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe1dc0a3bb0 0x7fe1dc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:01.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.148+0000 7fe1f0f4e640 1 --2- 172.21.15.12:0/3847267867 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe1dc0a3bb0 0x7fe1dc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:01.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.148+0000 7fe1e9ffb640 1 -- 172.21.15.12:0/3847267867 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fe1e40d0020 con 0x7fe1ec107ce0 2024-04-22T13:22:01.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.149+0000 7fe1f0f4e640 1 --2- 172.21.15.12:0/3847267867 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe1dc0a3bb0 0x7fe1dc0a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0009230 tx=0x7fe1e0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:01.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.153+0000 7fe1e9ffb640 1 -- 172.21.15.12:0/3847267867 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe1e4014030 con 0x7fe1ec107ce0 2024-04-22T13:22:01.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.394+0000 7fe1f31b2640 1 -- 172.21.15.12:0/3847267867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1ec062b50 con 0x7fe1ec107ce0 2024-04-22T13:22:01.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.395+0000 7fe1e9ffb640 1 -- 172.21.15.12:0/3847267867 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10494 (secure 0 0 0) 0x7fe1e409cea0 con 0x7fe1ec107ce0 2024-04-22T13:22:01.392 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:22:01.392 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14631,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14643,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24406,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24484,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24509,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-22T13:22:01.393 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 13 2024-04-22T13:22:01.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.397+0000 7fe1f31b2640 1 -- 172.21.15.12:0/3847267867 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe1dc0a3bb0 msgr2=0x7fe1dc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:01.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.397+0000 7fe1f31b2640 1 --2- 172.21.15.12:0/3847267867 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe1dc0a3bb0 0x7fe1dc0a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0009230 tx=0x7fe1e0005fd0 comp rx=0 tx=0).stop 2024-04-22T13:22:01.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.397+0000 7fe1f31b2640 1 -- 172.21.15.12:0/3847267867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1ec107ce0 msgr2=0x7fe1ec19f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:01.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.397+0000 7fe1f31b2640 1 --2- 172.21.15.12:0/3847267867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1ec107ce0 0x7fe1ec19f7a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400cad0 tx=0x7fe1e400cf90 comp rx=0 tx=0).stop 2024-04-22T13:22:01.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.397+0000 7fe1f31b2640 1 -- 172.21.15.12:0/3847267867 shutdown_connections 2024-04-22T13:22:01.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.397+0000 7fe1f31b2640 1 --2- 172.21.15.12:0/3847267867 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe1dc0a3bb0 0x7fe1dc0a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:01.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.397+0000 7fe1f31b2640 1 --2- 172.21.15.12:0/3847267867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe1ec107ce0 0x7fe1ec19f7a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:01.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.397+0000 7fe1f31b2640 1 --2- 172.21.15.12:0/3847267867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1ec1053b0 0x7fe1ec198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:01.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.397+0000 7fe1f31b2640 1 --2- 172.21.15.12:0/3847267867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1ec102a80 0x7fe1ec1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:01.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.397+0000 7fe1f31b2640 1 -- 172.21.15.12:0/3847267867 >> 172.21.15.12:0/3847267867 conn(0x7fe1ec0fc410 msgr2=0x7fe1ec1033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:01.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.397+0000 7fe1f31b2640 1 -- 172.21.15.12:0/3847267867 shutdown_connections 2024-04-22T13:22:01.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.397+0000 7fe1f31b2640 1 -- 172.21.15.12:0/3847267867 wait complete. 2024-04-22T13:22:01.407 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:22:01.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:01 smithi111 ceph-mon[31361]: mds.? [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] up:boot 2024-04-22T13:22:01.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:01 smithi111 ceph-mon[31361]: fsmap 12 up:standby 2024-04-22T13:22:01.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:01 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-22T13:22:01.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.519+0000 7f6b33724640 1 -- 172.21.15.12:0/3110698041 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b2c1029f0 msgr2=0x7f6b2c104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:01.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.519+0000 7f6b33724640 1 --2- 172.21.15.12:0/3110698041 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b2c1029f0 0x7f6b2c104de0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6b28009900 tx=0x7f6b2802bc30 comp rx=0 tx=0).stop 2024-04-22T13:22:01.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.519+0000 7f6b33724640 1 -- 172.21.15.12:0/3110698041 shutdown_connections 2024-04-22T13:22:01.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.519+0000 7f6b33724640 1 --2- 172.21.15.12:0/3110698041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b2c107c50 0x7f6b2c10a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:01.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.519+0000 7f6b33724640 1 --2- 172.21.15.12:0/3110698041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b2c105320 0x7f6b2c107710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:01.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.519+0000 7f6b33724640 1 --2- 172.21.15.12:0/3110698041 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b2c1029f0 0x7f6b2c104de0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:01.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.519+0000 7f6b33724640 1 -- 172.21.15.12:0/3110698041 >> 172.21.15.12:0/3110698041 conn(0x7f6b2c0fc3e0 msgr2=0x7f6b2c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:01.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.519+0000 7f6b33724640 1 -- 172.21.15.12:0/3110698041 shutdown_connections 2024-04-22T13:22:01.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.519+0000 7f6b33724640 1 -- 172.21.15.12:0/3110698041 wait complete. 2024-04-22T13:22:01.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.520+0000 7f6b33724640 1 Processor -- start 2024-04-22T13:22:01.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.520+0000 7f6b33724640 1 -- start start 2024-04-22T13:22:01.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.520+0000 7f6b33724640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b2c1029f0 0x7f6b2c102360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:01.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.520+0000 7f6b33724640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b2c105320 0x7f6b2c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:01.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.520+0000 7f6b33724640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b2c107c50 0x7f6b2c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:01.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.520+0000 7f6b33724640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c10daf0 con 0x7f6b2c105320 2024-04-22T13:22:01.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.520+0000 7f6b33724640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c10d970 con 0x7f6b2c107c50 2024-04-22T13:22:01.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.520+0000 7f6b33724640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c10dc70 con 0x7f6b2c1029f0 2024-04-22T13:22:01.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.520+0000 7f6b30cbf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b2c105320 0x7f6b2c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:01.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.520+0000 7f6b30cbf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b2c105320 0x7f6b2c1009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:37114/0 (socket says 172.21.15.12:37114) 2024-04-22T13:22:01.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.520+0000 7f6b30cbf640 1 -- 172.21.15.12:0/401974256 learned_addr learned my addr 172.21.15.12:0/401974256 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:01.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.520+0000 7f6b314c0640 1 --2- 172.21.15.12:0/401974256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b2c1029f0 0x7f6b2c102360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:01.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.520+0000 7f6b31cc1640 1 --2- 172.21.15.12:0/401974256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b2c107c50 0x7f6b2c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:01.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.520+0000 7f6b30cbf640 1 -- 172.21.15.12:0/401974256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b2c1029f0 msgr2=0x7f6b2c102360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:01.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.521+0000 7f6b30cbf640 1 --2- 172.21.15.12:0/401974256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b2c1029f0 0x7f6b2c102360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:01.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.521+0000 7f6b30cbf640 1 -- 172.21.15.12:0/401974256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b2c107c50 msgr2=0x7f6b2c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:01.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.521+0000 7f6b30cbf640 1 --2- 172.21.15.12:0/401974256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b2c107c50 0x7f6b2c100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:01.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.521+0000 7f6b30cbf640 1 -- 172.21.15.12:0/401974256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2c101770 con 0x7f6b2c105320 2024-04-22T13:22:01.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.521+0000 7f6b30cbf640 1 --2- 172.21.15.12:0/401974256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b2c105320 0x7f6b2c1009a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00e9c0 tx=0x7f6b1c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:01.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.521+0000 7f6b227fc640 1 -- 172.21.15.12:0/401974256 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c00cd80 con 0x7f6b2c105320 2024-04-22T13:22:01.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.521+0000 7f6b227fc640 1 -- 172.21.15.12:0/401974256 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b1c0054d0 con 0x7f6b2c105320 2024-04-22T13:22:01.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.521+0000 7f6b33724640 1 -- 172.21.15.12:0/401974256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b2c1a2300 con 0x7f6b2c105320 2024-04-22T13:22:01.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.521+0000 7f6b227fc640 1 -- 172.21.15.12:0/401974256 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c010430 con 0x7f6b2c105320 2024-04-22T13:22:01.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.521+0000 7f6b33724640 1 -- 172.21.15.12:0/401974256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b2c1a28a0 con 0x7f6b2c105320 2024-04-22T13:22:01.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.522+0000 7f6b33724640 1 -- 172.21.15.12:0/401974256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b2c0fded0 con 0x7f6b2c105320 2024-04-22T13:22:01.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.523+0000 7f6b227fc640 1 -- 172.21.15.12:0/401974256 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6b1c004fd0 con 0x7f6b2c105320 2024-04-22T13:22:01.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.524+0000 7f6b227fc640 1 --2- 172.21.15.12:0/401974256 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f6b180a3c30 0x7f6b180a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:01.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.524+0000 7f6b314c0640 1 --2- 172.21.15.12:0/401974256 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f6b180a3c30 0x7f6b180a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:01.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.524+0000 7f6b227fc640 1 -- 172.21.15.12:0/401974256 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f6b1c0d0320 con 0x7f6b2c105320 2024-04-22T13:22:01.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.524+0000 7f6b314c0640 1 --2- 172.21.15.12:0/401974256 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f6b180a3c30 0x7f6b180a60f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6b28000c00 tx=0x7f6b2802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:01.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.529+0000 7f6b227fc640 1 -- 172.21.15.12:0/401974256 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6b1c09d1a0 con 0x7f6b2c105320 2024-04-22T13:22:01.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:01 smithi012 ceph-mon[28265]: mds.? [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] up:boot 2024-04-22T13:22:01.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:01 smithi012 ceph-mon[28265]: fsmap 12 up:standby 2024-04-22T13:22:01.589 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:01 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-22T13:22:01.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:01 smithi134 ceph-mon[31338]: mds.? [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] up:boot 2024-04-22T13:22:01.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:01 smithi134 ceph-mon[31338]: fsmap 12 up:standby 2024-04-22T13:22:01.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:01 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-22T13:22:01.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.753+0000 7f6b33724640 1 -- 172.21.15.12:0/401974256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b2c02d770 con 0x7f6b2c105320 2024-04-22T13:22:01.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.755+0000 7f6b227fc640 1 -- 172.21.15.12:0/401974256 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+18470 (secure 0 0 0) 0x7f6b1c09caa0 con 0x7f6b2c105320 2024-04-22T13:22:01.752 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:22:01.753 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":83,"fsid":"0aede5ea-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:14:09.362918+0000","modified":"2024-04-22T13:21:16.272831+0000","last_up_change":"2024-04-22T13:21:15.263393+0000","last_in_change":"2024-04-22T13:20:54.174685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:17:32.372434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d9e2365-3a64-4b13-96fc-c217b1cee2ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6803","nonce":1191350632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6805","nonce":1191350632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6809","nonce":1191350632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6807","nonce":1191350632}]},"public_addr":"172.21.15.12:6803/1191350632","cluster_addr":"172.21.15.12:6805/1191350632","heartbeat_back_addr":"172.21.15.12:6809/1191350632","heartbeat_front_addr":"172.21.15.12:6807/1191350632","state":["exists","up"]},{"osd":1,"uuid":"1499c783-2bd7-48ce-a444-34add42bcf31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6801","nonce":417496465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6803","nonce":417496465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6807","nonce":417496465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6805","nonce":417496465}]},"public_addr":"172.21.15.111:6801/417496465","cluster_addr":"172.21.15.111:6803/417496465","heartbeat_back_addr":"172.21.15.111:6807/417496465","heartbeat_front_addr":"172.21.15.111:6805/417496465","state":["exists","up"]},{"osd":2,"uuid":"71a9ebc3-c622-434e-a6c2-f48358f384f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6801","nonce":1403153998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6803","nonce":1403153998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6807","nonce":1403153998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6805","nonce":1403153998}]},"public_addr":"172.21.15.134:6801/1403153998","cluster_addr":"172.21.15.134:6803/1403153998","heartbeat_back_addr":"172.21.15.134:6807/1403153998","heartbeat_front_addr":"172.21.15.134:6805/1403153998","state":["exists","up"]},{"osd":3,"uuid":"6dd00db2-a286-48ba-8666-56ffdd5f35ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6811","nonce":3359066681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6813","nonce":3359066681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6817","nonce":3359066681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6815","nonce":3359066681}]},"public_addr":"172.21.15.12:6811/3359066681","cluster_addr":"172.21.15.12:6813/3359066681","heartbeat_back_addr":"172.21.15.12:6817/3359066681","heartbeat_front_addr":"172.21.15.12:6815/3359066681","state":["exists","up"]},{"osd":4,"uuid":"1e29afd7-354c-4506-aaa0-b43eb1a19a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6809","nonce":2886349872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6811","nonce":2886349872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6815","nonce":2886349872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6813","nonce":2886349872}]},"public_addr":"172.21.15.111:6809/2886349872","cluster_addr":"172.21.15.111:6811/2886349872","heartbeat_back_addr":"172.21.15.111:6815/2886349872","heartbeat_front_addr":"172.21.15.111:6813/2886349872","state":["exists","up"]},{"osd":5,"uuid":"8f40f02d-0ba2-4842-a03d-2a1a04d1010d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6809","nonce":3071977602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6811","nonce":3071977602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6815","nonce":3071977602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6813","nonce":3071977602}]},"public_addr":"172.21.15.134:6809/3071977602","cluster_addr":"172.21.15.134:6811/3071977602","heartbeat_back_addr":"172.21.15.134:6815/3071977602","heartbeat_front_addr":"172.21.15.134:6813/3071977602","state":["exists","up"]},{"osd":6,"uuid":"bb0c17a9-6a8e-4be6-8660-f73cfa8200ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6819","nonce":3819619299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6821","nonce":3819619299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6825","nonce":3819619299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6823","nonce":3819619299}]},"public_addr":"172.21.15.12:6819/3819619299","cluster_addr":"172.21.15.12:6821/3819619299","heartbeat_back_addr":"172.21.15.12:6825/3819619299","heartbeat_front_addr":"172.21.15.12:6823/3819619299","state":["exists","up"]},{"osd":7,"uuid":"0a93d6af-569e-4e45-b697-9e978c1c1004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6817","nonce":3113856282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6819","nonce":3113856282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6823","nonce":3113856282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6821","nonce":3113856282}]},"public_addr":"172.21.15.111:6817/3113856282","cluster_addr":"172.21.15.111:6819/3113856282","heartbeat_back_addr":"172.21.15.111:6823/3113856282","heartbeat_front_addr":"172.21.15.111:6821/3113856282","state":["exists","up"]},{"osd":8,"uuid":"e21acc68-cf50-4f4f-b1ba-cf6c195b4955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6817","nonce":1689362191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6819","nonce":1689362191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6823","nonce":1689362191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6821","nonce":1689362191}]},"public_addr":"172.21.15.134:6817/1689362191","cluster_addr":"172.21.15.134:6819/1689362191","heartbeat_back_addr":"172.21.15.134:6823/1689362191","heartbeat_front_addr":"172.21.15.134:6821/1689362191","state":["exists","up"]},{"osd":9,"uuid":"fd745869-7f5f-48a8-b412-bb9c92b586e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6827","nonce":4292385779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6829","nonce":4292385779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6833","nonce":4292385779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6831","nonce":4292385779}]},"public_addr":"172.21.15.12:6827/4292385779","cluster_addr":"172.21.15.12:6829/4292385779","heartbeat_back_addr":"172.21.15.12:6833/4292385779","heartbeat_front_addr":"172.21.15.12:6831/4292385779","state":["exists","up"]},{"osd":10,"uuid":"798b1abd-f6c7-4a5f-8da0-fdae49e50d2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6825","nonce":3338566976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6827","nonce":3338566976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6831","nonce":3338566976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6829","nonce":3338566976}]},"public_addr":"172.21.15.111:6825/3338566976","cluster_addr":"172.21.15.111:6827/3338566976","heartbeat_back_addr":"172.21.15.111:6831/3338566976","heartbeat_front_addr":"172.21.15.111:6829/3338566976","state":["exists","up"]},{"osd":11,"uuid":"b18fc5f3-1049-4871-b060-e99290273cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6825","nonce":578640428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6827","nonce":578640428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6831","nonce":578640428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6829","nonce":578640428}]},"public_addr":"172.21.15.134:6825/578640428","cluster_addr":"172.21.15.134:6827/578640428","heartbeat_back_addr":"172.21.15.134:6831/578640428","heartbeat_front_addr":"172.21.15.134:6829/578640428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:16:40.828608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:05.052528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:28.297316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:53.496497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:17.947259+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:41.712894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:07.105945+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:32.129688+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:56.008516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:22.239867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:47.334692+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:12.301276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/2593601362":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/2519382069":"2024-04-23T13:15:04.279990+0000","172.21.15.12:6801/2009831940":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3509412187":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1007749062":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3588480095":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1904962176":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2009831940":"2024-04-23T13:15:04.279990+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:22:01.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.757+0000 7f6b33724640 1 -- 172.21.15.12:0/401974256 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f6b180a3c30 msgr2=0x7f6b180a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:01.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.757+0000 7f6b33724640 1 --2- 172.21.15.12:0/401974256 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f6b180a3c30 0x7f6b180a60f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6b28000c00 tx=0x7f6b2802c7e0 comp rx=0 tx=0).stop 2024-04-22T13:22:01.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.757+0000 7f6b33724640 1 -- 172.21.15.12:0/401974256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b2c105320 msgr2=0x7f6b2c1009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:01.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.757+0000 7f6b33724640 1 --2- 172.21.15.12:0/401974256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b2c105320 0x7f6b2c1009a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00e9c0 tx=0x7f6b1c00ee80 comp rx=0 tx=0).stop 2024-04-22T13:22:01.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.757+0000 7f6b33724640 1 -- 172.21.15.12:0/401974256 shutdown_connections 2024-04-22T13:22:01.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.757+0000 7f6b33724640 1 --2- 172.21.15.12:0/401974256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b2c107c50 0x7f6b2c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:01.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.757+0000 7f6b33724640 1 --2- 172.21.15.12:0/401974256 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f6b180a3c30 0x7f6b180a60f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:01.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.757+0000 7f6b33724640 1 --2- 172.21.15.12:0/401974256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6b2c105320 0x7f6b2c1009a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:01.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.757+0000 7f6b33724640 1 --2- 172.21.15.12:0/401974256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6b2c1029f0 0x7f6b2c102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:01.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.757+0000 7f6b33724640 1 -- 172.21.15.12:0/401974256 >> 172.21.15.12:0/401974256 conn(0x7f6b2c0fc3e0 msgr2=0x7f6b2c1033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:01.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.757+0000 7f6b33724640 1 -- 172.21.15.12:0/401974256 shutdown_connections 2024-04-22T13:22:01.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.757+0000 7f6b33724640 1 -- 172.21.15.12:0/401974256 wait complete. 2024-04-22T13:22:01.766 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T13:17:32.372434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T13:22:01.767 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T13:22:01.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.879+0000 7f5715411640 1 -- 172.21.15.12:0/3914882994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5710101160 msgr2=0x7f57101015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:01.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.879+0000 7f5715411640 1 --2- 172.21.15.12:0/3914882994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5710101160 0x7f57101015e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f56f8009900 tx=0x7f56f802bc30 comp rx=0 tx=0).stop 2024-04-22T13:22:01.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.879+0000 7f5715411640 1 -- 172.21.15.12:0/3914882994 shutdown_connections 2024-04-22T13:22:01.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.879+0000 7f5715411640 1 --2- 172.21.15.12:0/3914882994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5710101b20 0x7f57101093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:01.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.879+0000 7f5715411640 1 --2- 172.21.15.12:0/3914882994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5710101160 0x7f57101015e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:01.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.879+0000 7f5715411640 1 --2- 172.21.15.12:0/3914882994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5710102430 0x7f5710102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:01.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.879+0000 7f5715411640 1 -- 172.21.15.12:0/3914882994 >> 172.21.15.12:0/3914882994 conn(0x7f57100fbb90 msgr2=0x7f57100fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:01.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.879+0000 7f5715411640 1 -- 172.21.15.12:0/3914882994 shutdown_connections 2024-04-22T13:22:01.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.879+0000 7f5715411640 1 -- 172.21.15.12:0/3914882994 wait complete. 2024-04-22T13:22:01.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.880+0000 7f5715411640 1 Processor -- start 2024-04-22T13:22:01.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.880+0000 7f5715411640 1 -- start start 2024-04-22T13:22:01.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.880+0000 7f5715411640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5710101160 0x7f5710198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:01.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.880+0000 7f5715411640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5710101b20 0x7f5710198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:01.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.880+0000 7f5715411640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5710102430 0x7f571019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:01.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.880+0000 7f5715411640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f571010c9e0 con 0x7f5710101b20 2024-04-22T13:22:01.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.880+0000 7f5715411640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f571010c860 con 0x7f5710102430 2024-04-22T13:22:01.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.880+0000 7f5715411640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f571010cb60 con 0x7f5710101160 2024-04-22T13:22:01.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.880+0000 7f5707fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5710101b20 0x7f5710198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:01.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.880+0000 7f5707fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5710101b20 0x7f5710198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42480/0 (socket says 172.21.15.12:42480) 2024-04-22T13:22:01.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.880+0000 7f5707fff640 1 -- 172.21.15.12:0/3277093741 learned_addr learned my addr 172.21.15.12:0/3277093741 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:01.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.880+0000 7f570f7fe640 1 --2- 172.21.15.12:0/3277093741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5710102430 0x7f571019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:01.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.880+0000 7f570effd640 1 --2- 172.21.15.12:0/3277093741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5710101160 0x7f5710198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:01.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.880+0000 7f5707fff640 1 -- 172.21.15.12:0/3277093741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5710101160 msgr2=0x7f5710198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:01.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.880+0000 7f5707fff640 1 --2- 172.21.15.12:0/3277093741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5710101160 0x7f5710198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:01.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.880+0000 7f5707fff640 1 -- 172.21.15.12:0/3277093741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5710102430 msgr2=0x7f571019f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:01.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.880+0000 7f5707fff640 1 --2- 172.21.15.12:0/3277093741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5710102430 0x7f571019f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:01.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.880+0000 7f5707fff640 1 -- 172.21.15.12:0/3277093741 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f571019fe50 con 0x7f5710101b20 2024-04-22T13:22:01.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.880+0000 7f570f7fe640 1 --2- 172.21.15.12:0/3277093741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5710102430 0x7f571019f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:22:01.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.880+0000 7f570effd640 1 --2- 172.21.15.12:0/3277093741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5710101160 0x7f5710198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:22:01.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.881+0000 7f5707fff640 1 --2- 172.21.15.12:0/3277093741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5710101b20 0x7f5710198770 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f56f8005ab0 tx=0x7f56f80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:01.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.881+0000 7f570cff9640 1 -- 172.21.15.12:0/3277093741 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f8043020 con 0x7f5710101b20 2024-04-22T13:22:01.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.881+0000 7f570cff9640 1 -- 172.21.15.12:0/3277093741 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56f8004a50 con 0x7f5710101b20 2024-04-22T13:22:01.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.881+0000 7f5715411640 1 -- 172.21.15.12:0/3277093741 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57101a00e0 con 0x7f5710101b20 2024-04-22T13:22:01.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.881+0000 7f570cff9640 1 -- 172.21.15.12:0/3277093741 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f8003bc0 con 0x7f5710101b20 2024-04-22T13:22:01.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.881+0000 7f5715411640 1 -- 172.21.15.12:0/3277093741 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57101a0630 con 0x7f5710101b20 2024-04-22T13:22:01.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.882+0000 7f5715411640 1 -- 172.21.15.12:0/3277093741 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57101a0830 con 0x7f5710101b20 2024-04-22T13:22:01.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.883+0000 7f570cff9640 1 -- 172.21.15.12:0/3277093741 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f56f8046020 con 0x7f5710101b20 2024-04-22T13:22:01.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.884+0000 7f570cff9640 1 --2- 172.21.15.12:0/3277093741 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f56fc0a3bb0 0x7f56fc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:01.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.884+0000 7f570effd640 1 --2- 172.21.15.12:0/3277093741 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f56fc0a3bb0 0x7f56fc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:01.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.884+0000 7f570cff9640 1 -- 172.21.15.12:0/3277093741 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f56f80f2a40 con 0x7f5710101b20 2024-04-22T13:22:01.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.884+0000 7f570effd640 1 --2- 172.21.15.12:0/3277093741 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f56fc0a3bb0 0x7f56fc0a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f571006a2b0 tx=0x7f5700005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:01.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:01.887+0000 7f570cff9640 1 -- 172.21.15.12:0/3277093741 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f56f804d020 con 0x7f5710101b20 2024-04-22T13:22:02.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.077+0000 7f5715411640 1 -- 172.21.15.12:0/3277093741 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f57101a0a00 con 0x7f5710101b20 2024-04-22T13:22:02.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.077+0000 7f570cff9640 1 -- 172.21.15.12:0/3277093741 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f56f803e450 con 0x7f5710101b20 2024-04-22T13:22:02.075 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-22T13:22:02.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.079+0000 7f5715411640 1 -- 172.21.15.12:0/3277093741 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f56fc0a3bb0 msgr2=0x7f56fc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:02.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.079+0000 7f5715411640 1 --2- 172.21.15.12:0/3277093741 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f56fc0a3bb0 0x7f56fc0a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f571006a2b0 tx=0x7f5700005f90 comp rx=0 tx=0).stop 2024-04-22T13:22:02.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.079+0000 7f5715411640 1 -- 172.21.15.12:0/3277093741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5710101b20 msgr2=0x7f5710198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:02.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.079+0000 7f5715411640 1 --2- 172.21.15.12:0/3277093741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5710101b20 0x7f5710198770 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f56f8005ab0 tx=0x7f56f80042f0 comp rx=0 tx=0).stop 2024-04-22T13:22:02.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.079+0000 7f5715411640 1 -- 172.21.15.12:0/3277093741 shutdown_connections 2024-04-22T13:22:02.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.079+0000 7f5715411640 1 --2- 172.21.15.12:0/3277093741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5710102430 0x7f571019f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:02.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.079+0000 7f5715411640 1 --2- 172.21.15.12:0/3277093741 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f56fc0a3bb0 0x7f56fc0a6070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:02.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.079+0000 7f5715411640 1 --2- 172.21.15.12:0/3277093741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5710101b20 0x7f5710198770 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:02.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.079+0000 7f5715411640 1 --2- 172.21.15.12:0/3277093741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5710101160 0x7f5710198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:02.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.079+0000 7f5715411640 1 -- 172.21.15.12:0/3277093741 >> 172.21.15.12:0/3277093741 conn(0x7f57100fbb90 msgr2=0x7f57101000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:02.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.079+0000 7f5715411640 1 -- 172.21.15.12:0/3277093741 shutdown_connections 2024-04-22T13:22:02.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.079+0000 7f5715411640 1 -- 172.21.15.12:0/3277093741 wait complete. 2024-04-22T13:22:02.089 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-22T13:22:02.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.196+0000 7fa5213ca640 1 -- 172.21.15.12:0/1007972193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa51c102a70 msgr2=0x7fa51c104e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:02.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.196+0000 7fa5213ca640 1 --2- 172.21.15.12:0/1007972193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa51c102a70 0x7fa51c104e60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa510009900 tx=0x7fa51002bc30 comp rx=0 tx=0).stop 2024-04-22T13:22:02.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.196+0000 7fa5213ca640 1 -- 172.21.15.12:0/1007972193 shutdown_connections 2024-04-22T13:22:02.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.196+0000 7fa5213ca640 1 --2- 172.21.15.12:0/1007972193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa51c107cd0 0x7fa51c10a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:02.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.196+0000 7fa5213ca640 1 --2- 172.21.15.12:0/1007972193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa51c1053a0 0x7fa51c107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:02.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.196+0000 7fa5213ca640 1 --2- 172.21.15.12:0/1007972193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa51c102a70 0x7fa51c104e60 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:02.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.196+0000 7fa5213ca640 1 -- 172.21.15.12:0/1007972193 >> 172.21.15.12:0/1007972193 conn(0x7fa51c0fc400 msgr2=0x7fa51c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:02.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.196+0000 7fa5213ca640 1 -- 172.21.15.12:0/1007972193 shutdown_connections 2024-04-22T13:22:02.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.196+0000 7fa5213ca640 1 -- 172.21.15.12:0/1007972193 wait complete. 2024-04-22T13:22:02.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.196+0000 7fa5213ca640 1 Processor -- start 2024-04-22T13:22:02.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.196+0000 7fa5213ca640 1 -- start start 2024-04-22T13:22:02.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.197+0000 7fa5213ca640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa51c102a70 0x7fa51c198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:02.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.197+0000 7fa5213ca640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa51c1053a0 0x7fa51c198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:02.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.197+0000 7fa5213ca640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa51c107cd0 0x7fa51c19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:02.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.197+0000 7fa5213ca640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa51c10dad0 con 0x7fa51c102a70 2024-04-22T13:22:02.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.197+0000 7fa5213ca640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa51c10d950 con 0x7fa51c107cd0 2024-04-22T13:22:02.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.197+0000 7fa5213ca640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa51c10dc50 con 0x7fa51c1053a0 2024-04-22T13:22:02.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.197+0000 7fa51affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa51c102a70 0x7fa51c198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:02.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.197+0000 7fa51a7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa51c1053a0 0x7fa51c198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:02.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.197+0000 7fa51affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa51c102a70 0x7fa51c198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42504/0 (socket says 172.21.15.12:42504) 2024-04-22T13:22:02.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.197+0000 7fa51affd640 1 -- 172.21.15.12:0/2408976905 learned_addr learned my addr 172.21.15.12:0/2408976905 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:02.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.197+0000 7fa51b7fe640 1 --2- 172.21.15.12:0/2408976905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa51c107cd0 0x7fa51c19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:02.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.197+0000 7fa51affd640 1 -- 172.21.15.12:0/2408976905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa51c1053a0 msgr2=0x7fa51c198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:02.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.197+0000 7fa51affd640 1 --2- 172.21.15.12:0/2408976905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa51c1053a0 0x7fa51c198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:02.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.197+0000 7fa51affd640 1 -- 172.21.15.12:0/2408976905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa51c107cd0 msgr2=0x7fa51c19f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:02.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.197+0000 7fa51affd640 1 --2- 172.21.15.12:0/2408976905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa51c107cd0 0x7fa51c19f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:02.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.197+0000 7fa51affd640 1 -- 172.21.15.12:0/2408976905 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa51c19fe70 con 0x7fa51c102a70 2024-04-22T13:22:02.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.197+0000 7fa51b7fe640 1 --2- 172.21.15.12:0/2408976905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa51c107cd0 0x7fa51c19f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:22:02.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.197+0000 7fa51a7fc640 1 --2- 172.21.15.12:0/2408976905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa51c1053a0 0x7fa51c198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:22:02.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.197+0000 7fa51affd640 1 --2- 172.21.15.12:0/2408976905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa51c102a70 0x7fa51c198250 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa510006010 tx=0x7fa510005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:02.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.198+0000 7fa503fff640 1 -- 172.21.15.12:0/2408976905 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa510043020 con 0x7fa51c102a70 2024-04-22T13:22:02.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.198+0000 7fa503fff640 1 -- 172.21.15.12:0/2408976905 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5100043d0 con 0x7fa51c102a70 2024-04-22T13:22:02.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.198+0000 7fa5213ca640 1 -- 172.21.15.12:0/2408976905 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa51c1a0100 con 0x7fa51c102a70 2024-04-22T13:22:02.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.198+0000 7fa503fff640 1 -- 172.21.15.12:0/2408976905 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa510003b80 con 0x7fa51c102a70 2024-04-22T13:22:02.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.198+0000 7fa5213ca640 1 -- 172.21.15.12:0/2408976905 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa51c1a0650 con 0x7fa51c102a70 2024-04-22T13:22:02.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.199+0000 7fa5213ca640 1 -- 172.21.15.12:0/2408976905 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa51c0fdef0 con 0x7fa51c102a70 2024-04-22T13:22:02.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.200+0000 7fa503fff640 1 -- 172.21.15.12:0/2408976905 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa510046020 con 0x7fa51c102a70 2024-04-22T13:22:02.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.200+0000 7fa503fff640 1 --2- 172.21.15.12:0/2408976905 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa5040a3bb0 0x7fa5040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:02.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.201+0000 7fa51a7fc640 1 --2- 172.21.15.12:0/2408976905 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa5040a3bb0 0x7fa5040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:02.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.201+0000 7fa503fff640 1 -- 172.21.15.12:0/2408976905 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fa5100f21d0 con 0x7fa51c102a70 2024-04-22T13:22:02.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.201+0000 7fa51a7fc640 1 --2- 172.21.15.12:0/2408976905 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa5040a3bb0 0x7fa5040a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa51c199760 tx=0x7fa50c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:02.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.205+0000 7fa503fff640 1 -- 172.21.15.12:0/2408976905 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa51004d020 con 0x7fa51c102a70 2024-04-22T13:22:02.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.430+0000 7fa5213ca640 1 -- 172.21.15.12:0/2408976905 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa51c062b50 con 0x7fa51c102a70 2024-04-22T13:22:02.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.431+0000 7fa503fff640 1 -- 172.21.15.12:0/2408976905 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) ==== 84+0+65 (secure 0 0 0) 0x7fa5100befb0 con 0x7fa51c102a70 2024-04-22T13:22:02.428 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:22:02.429 INFO:teuthology.orchestra.run.smithi012.stdout:[ 2024-04-22T13:22:02.429 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:22:02.429 INFO:teuthology.orchestra.run.smithi012.stdout: "poolnum": 1, 2024-04-22T13:22:02.429 INFO:teuthology.orchestra.run.smithi012.stdout: "poolname": ".mgr" 2024-04-22T13:22:02.429 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:22:02.429 INFO:teuthology.orchestra.run.smithi012.stdout:] 2024-04-22T13:22:02.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.433+0000 7fa5213ca640 1 -- 172.21.15.12:0/2408976905 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa5040a3bb0 msgr2=0x7fa5040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:02.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.433+0000 7fa5213ca640 1 --2- 172.21.15.12:0/2408976905 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa5040a3bb0 0x7fa5040a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa51c199760 tx=0x7fa50c005f90 comp rx=0 tx=0).stop 2024-04-22T13:22:02.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.433+0000 7fa5213ca640 1 -- 172.21.15.12:0/2408976905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa51c102a70 msgr2=0x7fa51c198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:02.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.433+0000 7fa5213ca640 1 --2- 172.21.15.12:0/2408976905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa51c102a70 0x7fa51c198250 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa510006010 tx=0x7fa510005e40 comp rx=0 tx=0).stop 2024-04-22T13:22:02.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.433+0000 7fa5213ca640 1 -- 172.21.15.12:0/2408976905 shutdown_connections 2024-04-22T13:22:02.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.433+0000 7fa5213ca640 1 --2- 172.21.15.12:0/2408976905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa51c107cd0 0x7fa51c19f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:02.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.433+0000 7fa5213ca640 1 --2- 172.21.15.12:0/2408976905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa51c1053a0 0x7fa51c198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:02.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.433+0000 7fa5213ca640 1 --2- 172.21.15.12:0/2408976905 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa5040a3bb0 0x7fa5040a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:02.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.433+0000 7fa5213ca640 1 --2- 172.21.15.12:0/2408976905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa51c102a70 0x7fa51c198250 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:02.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.433+0000 7fa5213ca640 1 -- 172.21.15.12:0/2408976905 >> 172.21.15.12:0/2408976905 conn(0x7fa51c0fc400 msgr2=0x7fa51c104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:02.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.433+0000 7fa5213ca640 1 -- 172.21.15.12:0/2408976905 shutdown_connections 2024-04-22T13:22:02.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.433+0000 7fa5213ca640 1 -- 172.21.15.12:0/2408976905 wait complete. 2024-04-22T13:22:02.441 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:02 smithi012 ceph-mon[28265]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:22:02.441 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:02 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3847267867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:22:02.441 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:02 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:02.441 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:02 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:02.441 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:02 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/401974256' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:22:02.441 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:02 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:02.441 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:02 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:02.441 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:02 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:02.441 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:02 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:02.441 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:02 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:02.441 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:02 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:22:02.441 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:02 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:02.441 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:02 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:22:02.442 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:02 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3277093741' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:22:02.442 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-22T13:22:02.443 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-22T13:22:02.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:02 smithi111 ceph-mon[31361]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:22:02.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:02 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/3847267867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:22:02.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:02 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:02.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:02 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:02.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:02 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/401974256' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:22:02.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:02 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:02.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:02 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:02.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:02 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:02.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:02 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:02.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:02 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:02.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:02 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:22:02.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:02 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:02.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:02 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:22:02.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:02 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/3277093741' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:22:02.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.557+0000 7f40952d2640 1 -- 172.21.15.12:0/1062073326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4090100ea0 msgr2=0x7f40901012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:02.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.557+0000 7f40952d2640 1 --2- 172.21.15.12:0/1062073326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4090100ea0 0x7f40901012c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4084009900 tx=0x7f408402bc30 comp rx=0 tx=0).stop 2024-04-22T13:22:02.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.557+0000 7f40952d2640 1 -- 172.21.15.12:0/1062073326 shutdown_connections 2024-04-22T13:22:02.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.557+0000 7f40952d2640 1 --2- 172.21.15.12:0/1062073326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40901029b0 0x7f40901092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:02.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.557+0000 7f40952d2640 1 --2- 172.21.15.12:0/1062073326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4090101ff0 0x7f4090102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:02.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.557+0000 7f40952d2640 1 --2- 172.21.15.12:0/1062073326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4090100ea0 0x7f40901012c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:02.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.557+0000 7f40952d2640 1 -- 172.21.15.12:0/1062073326 >> 172.21.15.12:0/1062073326 conn(0x7f40900fc410 msgr2=0x7f40900fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:02.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.557+0000 7f40952d2640 1 -- 172.21.15.12:0/1062073326 shutdown_connections 2024-04-22T13:22:02.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.557+0000 7f40952d2640 1 -- 172.21.15.12:0/1062073326 wait complete. 2024-04-22T13:22:02.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.558+0000 7f40952d2640 1 Processor -- start 2024-04-22T13:22:02.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.558+0000 7f40952d2640 1 -- start start 2024-04-22T13:22:02.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.558+0000 7f40952d2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4090100ea0 0x7f4090198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:02.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.558+0000 7f40952d2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4090101ff0 0x7f40901987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:02.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.558+0000 7f40952d2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40901029b0 0x7f409019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:02.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.558+0000 7f40952d2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f409010c9d0 con 0x7f40901029b0 2024-04-22T13:22:02.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.558+0000 7f40952d2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f409010c850 con 0x7f4090100ea0 2024-04-22T13:22:02.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.558+0000 7f40952d2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f409010cb50 con 0x7f4090101ff0 2024-04-22T13:22:02.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.558+0000 7f408f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40901029b0 0x7f409019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:02.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.559+0000 7f408f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40901029b0 0x7f409019f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42526/0 (socket says 172.21.15.12:42526) 2024-04-22T13:22:02.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.559+0000 7f408f7fe640 1 -- 172.21.15.12:0/2487511941 learned_addr learned my addr 172.21.15.12:0/2487511941 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:02.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.559+0000 7f408effd640 1 --2- 172.21.15.12:0/2487511941 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4090100ea0 0x7f4090198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:02.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.559+0000 7f408e7fc640 1 --2- 172.21.15.12:0/2487511941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4090101ff0 0x7f40901987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:02.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.559+0000 7f408f7fe640 1 -- 172.21.15.12:0/2487511941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4090101ff0 msgr2=0x7f40901987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:02.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.559+0000 7f408f7fe640 1 --2- 172.21.15.12:0/2487511941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4090101ff0 0x7f40901987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:02.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.559+0000 7f408f7fe640 1 -- 172.21.15.12:0/2487511941 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4090100ea0 msgr2=0x7f4090198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:02.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.559+0000 7f408f7fe640 1 --2- 172.21.15.12:0/2487511941 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4090100ea0 0x7f4090198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:02.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.559+0000 7f408f7fe640 1 -- 172.21.15.12:0/2487511941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409019fe80 con 0x7f40901029b0 2024-04-22T13:22:02.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.559+0000 7f408effd640 1 --2- 172.21.15.12:0/2487511941 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4090100ea0 0x7f4090198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:22:02.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.559+0000 7f408f7fe640 1 --2- 172.21.15.12:0/2487511941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40901029b0 0x7f409019f810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f408800efd0 tx=0x7f408800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:02.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.559+0000 7f4077fff640 1 -- 172.21.15.12:0/2487511941 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4088003b70 con 0x7f40901029b0 2024-04-22T13:22:02.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.559+0000 7f40952d2640 1 -- 172.21.15.12:0/2487511941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40901a0170 con 0x7f40901029b0 2024-04-22T13:22:02.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.559+0000 7f4077fff640 1 -- 172.21.15.12:0/2487511941 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4088003d10 con 0x7f40901029b0 2024-04-22T13:22:02.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.559+0000 7f4077fff640 1 -- 172.21.15.12:0/2487511941 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40880108a0 con 0x7f40901029b0 2024-04-22T13:22:02.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.559+0000 7f40952d2640 1 -- 172.21.15.12:0/2487511941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40901a0670 con 0x7f40901029b0 2024-04-22T13:22:02.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.560+0000 7f40952d2640 1 -- 172.21.15.12:0/2487511941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40901a0930 con 0x7f40901029b0 2024-04-22T13:22:02.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.562+0000 7f4077fff640 1 -- 172.21.15.12:0/2487511941 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4088004b40 con 0x7f40901029b0 2024-04-22T13:22:02.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.563+0000 7f4077fff640 1 --2- 172.21.15.12:0/2487511941 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f40780a3bb0 0x7f40780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:02.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.563+0000 7f4077fff640 1 -- 172.21.15.12:0/2487511941 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f40880d0750 con 0x7f40901029b0 2024-04-22T13:22:02.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.563+0000 7f408effd640 1 --2- 172.21.15.12:0/2487511941 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f40780a3bb0 0x7f40780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:02.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.563+0000 7f408effd640 1 --2- 172.21.15.12:0/2487511941 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f40780a3bb0 0x7f40780a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4084009230 tx=0x7f4084005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:02.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.567+0000 7f4077fff640 1 -- 172.21.15.12:0/2487511941 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f408809d5d0 con 0x7f40901029b0 2024-04-22T13:22:02.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:02 smithi134 ceph-mon[31338]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:22:02.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:02 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/3847267867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:22:02.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:02 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:02.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:02 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:02.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:02 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/401974256' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:22:02.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:02 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:02.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:02 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:02.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:02 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:02.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:02 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:02.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:02 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:02.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:02 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:22:02.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:02 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:02.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:02 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:22:02.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:02 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/3277093741' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:22:02.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:02.763+0000 7f40952d2640 1 -- 172.21.15.12:0/2487511941 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f40901a0c80 con 0x7f40901029b0 2024-04-22T13:22:03.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.216+0000 7f4077fff640 1 -- 172.21.15.12:0/2487511941 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) ==== 138+0+0 (secure 0 0 0) 0x7f408809ced0 con 0x7f40901029b0 2024-04-22T13:22:03.214 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_metadata' created 2024-04-22T13:22:03.215 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:03 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2408976905' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-22T13:22:03.215 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:03 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2487511941' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-22T13:22:03.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.219+0000 7f40952d2640 1 -- 172.21.15.12:0/2487511941 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f40780a3bb0 msgr2=0x7f40780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:03.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.219+0000 7f40952d2640 1 --2- 172.21.15.12:0/2487511941 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f40780a3bb0 0x7f40780a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4084009230 tx=0x7f4084005fd0 comp rx=0 tx=0).stop 2024-04-22T13:22:03.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.219+0000 7f40952d2640 1 -- 172.21.15.12:0/2487511941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40901029b0 msgr2=0x7f409019f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:03.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.219+0000 7f40952d2640 1 --2- 172.21.15.12:0/2487511941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40901029b0 0x7f409019f810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f408800efd0 tx=0x7f408800c6a0 comp rx=0 tx=0).stop 2024-04-22T13:22:03.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.219+0000 7f40952d2640 1 -- 172.21.15.12:0/2487511941 shutdown_connections 2024-04-22T13:22:03.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.219+0000 7f40952d2640 1 --2- 172.21.15.12:0/2487511941 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f40780a3bb0 0x7f40780a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:03.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.219+0000 7f40952d2640 1 --2- 172.21.15.12:0/2487511941 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f40901029b0 0x7f409019f810 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:03.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.219+0000 7f40952d2640 1 --2- 172.21.15.12:0/2487511941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4090101ff0 0x7f40901987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:03.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.219+0000 7f40952d2640 1 --2- 172.21.15.12:0/2487511941 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4090100ea0 0x7f4090198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:03.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.219+0000 7f40952d2640 1 -- 172.21.15.12:0/2487511941 >> 172.21.15.12:0/2487511941 conn(0x7f40900fc410 msgr2=0x7f409006a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:03.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.219+0000 7f40952d2640 1 -- 172.21.15.12:0/2487511941 shutdown_connections 2024-04-22T13:22:03.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.219+0000 7f40952d2640 1 -- 172.21.15.12:0/2487511941 wait complete. 2024-04-22T13:22:03.231 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-22T13:22:03.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:03 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2408976905' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-22T13:22:03.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:03 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2487511941' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-22T13:22:03.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.349+0000 7faa8270e640 1 -- 172.21.15.12:0/1399241163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa7c1029a0 msgr2=0x7faa7c1092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:03.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.349+0000 7faa8270e640 1 --2- 172.21.15.12:0/1399241163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa7c1029a0 0x7faa7c1092c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faa74009be0 tx=0x7faa7402bef0 comp rx=0 tx=0).stop 2024-04-22T13:22:03.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.349+0000 7faa8270e640 1 -- 172.21.15.12:0/1399241163 shutdown_connections 2024-04-22T13:22:03.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.349+0000 7faa8270e640 1 --2- 172.21.15.12:0/1399241163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa7c1029a0 0x7faa7c1092c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:03.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.349+0000 7faa8270e640 1 --2- 172.21.15.12:0/1399241163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa7c101fe0 0x7faa7c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:03.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.349+0000 7faa8270e640 1 --2- 172.21.15.12:0/1399241163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa7c100e90 0x7faa7c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:03.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.349+0000 7faa8270e640 1 -- 172.21.15.12:0/1399241163 >> 172.21.15.12:0/1399241163 conn(0x7faa7c0fc400 msgr2=0x7faa7c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:03.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.349+0000 7faa8270e640 1 -- 172.21.15.12:0/1399241163 shutdown_connections 2024-04-22T13:22:03.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.349+0000 7faa8270e640 1 -- 172.21.15.12:0/1399241163 wait complete. 2024-04-22T13:22:03.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.350+0000 7faa8270e640 1 Processor -- start 2024-04-22T13:22:03.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.350+0000 7faa8270e640 1 -- start start 2024-04-22T13:22:03.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.350+0000 7faa8270e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa7c100e90 0x7faa7c1970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:03.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.350+0000 7faa8270e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa7c101fe0 0x7faa7c197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:03.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.350+0000 7faa8270e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa7c1029a0 0x7faa7c19e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:03.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.350+0000 7faa8270e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7faa7c10ca30 con 0x7faa7c1029a0 2024-04-22T13:22:03.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.350+0000 7faa8270e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faa7c10c8b0 con 0x7faa7c101fe0 2024-04-22T13:22:03.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.350+0000 7faa8270e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faa7c10cbb0 con 0x7faa7c100e90 2024-04-22T13:22:03.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.350+0000 7faa80cab640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa7c1029a0 0x7faa7c19e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:03.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.350+0000 7faa7bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa7c100e90 0x7faa7c1970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:03.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.350+0000 7faa7bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa7c100e90 0x7faa7c1970d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.12:42966/0 (socket says 172.21.15.12:42966) 2024-04-22T13:22:03.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.350+0000 7faa80cab640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa7c1029a0 0x7faa7c19e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42540/0 (socket says 172.21.15.12:42540) 2024-04-22T13:22:03.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.350+0000 7faa7bfff640 1 -- 172.21.15.12:0/4287923786 learned_addr learned my addr 172.21.15.12:0/4287923786 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:03.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.350+0000 7faa7b7fe640 1 --2- 172.21.15.12:0/4287923786 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa7c101fe0 0x7faa7c197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:03.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.351+0000 7faa7bfff640 1 -- 172.21.15.12:0/4287923786 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa7c101fe0 msgr2=0x7faa7c197610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:03.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.351+0000 7faa7bfff640 1 --2- 172.21.15.12:0/4287923786 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa7c101fe0 0x7faa7c197610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:03.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.351+0000 7faa7bfff640 1 -- 172.21.15.12:0/4287923786 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa7c1029a0 msgr2=0x7faa7c19e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:03.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.351+0000 7faa7bfff640 1 --2- 172.21.15.12:0/4287923786 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa7c1029a0 0x7faa7c19e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:03.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.351+0000 7faa7bfff640 1 -- 172.21.15.12:0/4287923786 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa7c19ecf0 con 0x7faa7c100e90 2024-04-22T13:22:03.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.351+0000 7faa80cab640 1 --2- 172.21.15.12:0/4287923786 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa7c1029a0 0x7faa7c19e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:22:03.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.351+0000 7faa7bfff640 1 --2- 172.21.15.12:0/4287923786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa7c100e90 0x7faa7c1970d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00e9c0 tx=0x7faa6c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:03.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.351+0000 7faa797fa640 1 -- 172.21.15.12:0/4287923786 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c00ccc0 con 0x7faa7c100e90 2024-04-22T13:22:03.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.351+0000 7faa797fa640 1 -- 172.21.15.12:0/4287923786 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa6c0054d0 con 0x7faa7c100e90 2024-04-22T13:22:03.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.351+0000 7faa8270e640 1 -- 172.21.15.12:0/4287923786 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa7c19efe0 con 0x7faa7c100e90 2024-04-22T13:22:03.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.351+0000 7faa797fa640 1 -- 172.21.15.12:0/4287923786 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c010430 con 0x7faa7c100e90 2024-04-22T13:22:03.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.351+0000 7faa8270e640 1 -- 172.21.15.12:0/4287923786 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa7c19f4a0 con 0x7faa7c100e90 2024-04-22T13:22:03.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.352+0000 7faa8270e640 1 -- 172.21.15.12:0/4287923786 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa7c1081a0 con 0x7faa7c100e90 2024-04-22T13:22:03.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.353+0000 7faa797fa640 1 -- 172.21.15.12:0/4287923786 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7faa6c004f80 con 0x7faa7c100e90 2024-04-22T13:22:03.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.354+0000 7faa797fa640 1 --2- 172.21.15.12:0/4287923786 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7faa640a3c30 0x7faa640a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:03.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.354+0000 7faa7b7fe640 1 --2- 172.21.15.12:0/4287923786 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7faa640a3c30 0x7faa640a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:03.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.354+0000 7faa797fa640 1 -- 172.21.15.12:0/4287923786 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7497+0+0 (secure 0 0 0) 0x7faa6c0d05f0 con 0x7faa7c100e90 2024-04-22T13:22:03.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.354+0000 7faa7b7fe640 1 --2- 172.21.15.12:0/4287923786 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7faa640a3c30 0x7faa640a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faa7c1985e0 tx=0x7faa70005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:03.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.358+0000 7faa797fa640 1 -- 172.21.15.12:0/4287923786 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faa6c09d260 con 0x7faa7c100e90 2024-04-22T13:22:03.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:03.571+0000 7faa8270e640 1 -- 172.21.15.12:0/4287923786 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7faa7c02d5c0 con 0x7faa7c100e90 2024-04-22T13:22:03.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:03 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2408976905' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-22T13:22:03.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:03 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2487511941' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-22T13:22:04.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.231+0000 7faa797fa640 1 -- 172.21.15.12:0/4287923786 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v85) ==== 170+0+0 (secure 0 0 0) 0x7faa6c09cb60 con 0x7faa7c100e90 2024-04-22T13:22:04.229 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data' created 2024-04-22T13:22:04.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.234+0000 7faa8270e640 1 -- 172.21.15.12:0/4287923786 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7faa640a3c30 msgr2=0x7faa640a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:04.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.234+0000 7faa8270e640 1 --2- 172.21.15.12:0/4287923786 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7faa640a3c30 0x7faa640a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faa7c1985e0 tx=0x7faa70005f90 comp rx=0 tx=0).stop 2024-04-22T13:22:04.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.234+0000 7faa8270e640 1 -- 172.21.15.12:0/4287923786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa7c100e90 msgr2=0x7faa7c1970d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:04.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.234+0000 7faa8270e640 1 --2- 172.21.15.12:0/4287923786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa7c100e90 0x7faa7c1970d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00e9c0 tx=0x7faa6c00ee80 comp rx=0 tx=0).stop 2024-04-22T13:22:04.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.237+0000 7faa8270e640 1 -- 172.21.15.12:0/4287923786 shutdown_connections 2024-04-22T13:22:04.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.237+0000 7faa8270e640 1 --2- 172.21.15.12:0/4287923786 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7faa640a3c30 0x7faa640a60f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:04.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.237+0000 7faa8270e640 1 --2- 172.21.15.12:0/4287923786 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa7c1029a0 0x7faa7c19e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:04.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.237+0000 7faa8270e640 1 --2- 172.21.15.12:0/4287923786 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa7c101fe0 0x7faa7c197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:04.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.237+0000 7faa8270e640 1 --2- 172.21.15.12:0/4287923786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa7c100e90 0x7faa7c1970d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:04.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.237+0000 7faa8270e640 1 -- 172.21.15.12:0/4287923786 >> 172.21.15.12:0/4287923786 conn(0x7faa7c0fc400 msgr2=0x7faa7c105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:04.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.237+0000 7faa8270e640 1 -- 172.21.15.12:0/4287923786 shutdown_connections 2024-04-22T13:22:04.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.237+0000 7faa8270e640 1 -- 172.21.15.12:0/4287923786 wait complete. 2024-04-22T13:22:04.253 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-22T13:22:04.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.362+0000 7f44196ba640 1 -- 172.21.15.12:0/1925249749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4414101f90 msgr2=0x7f44141023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:04.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.362+0000 7f44196ba640 1 --2- 172.21.15.12:0/1925249749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4414101f90 0x7f44141023b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4404009900 tx=0x7f440402bc30 comp rx=0 tx=0).stop 2024-04-22T13:22:04.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.362+0000 7f44196ba640 1 -- 172.21.15.12:0/1925249749 shutdown_connections 2024-04-22T13:22:04.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.362+0000 7f44196ba640 1 --2- 172.21.15.12:0/1925249749 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4414103b40 0x7f441410a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:04.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.362+0000 7f44196ba640 1 --2- 172.21.15.12:0/1925249749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4414103180 0x7f4414103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:04.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.362+0000 7f44196ba640 1 --2- 172.21.15.12:0/1925249749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4414101f90 0x7f44141023b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:04.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.362+0000 7f44196ba640 1 -- 172.21.15.12:0/1925249749 >> 172.21.15.12:0/1925249749 conn(0x7f44140fd500 msgr2=0x7f44140ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.362+0000 7f44196ba640 1 -- 172.21.15.12:0/1925249749 shutdown_connections 2024-04-22T13:22:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.362+0000 7f44196ba640 1 -- 172.21.15.12:0/1925249749 wait complete. 2024-04-22T13:22:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.363+0000 7f44196ba640 1 Processor -- start 2024-04-22T13:22:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.363+0000 7f44196ba640 1 -- start start 2024-04-22T13:22:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.363+0000 7f44196ba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4414101f90 0x7f4414075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.363+0000 7f44196ba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4414103180 0x7f4414074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.363+0000 7f44196ba640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4414103b40 0x7f4414074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.363+0000 7f44196ba640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f441410dad0 con 0x7f4414101f90 2024-04-22T13:22:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.363+0000 7f44196ba640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f441410d950 con 0x7f4414103b40 2024-04-22T13:22:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.363+0000 7f44196ba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f441410dc50 con 0x7f4414103180 2024-04-22T13:22:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.363+0000 7f4412ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4414101f90 0x7f4414075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.363+0000 7f440bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4414103180 0x7f4414074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:04.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.363+0000 7f440bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4414103180 0x7f4414074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.12:42990/0 (socket says 172.21.15.12:42990) 2024-04-22T13:22:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.363+0000 7f4412ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4414101f90 0x7f4414075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42560/0 (socket says 172.21.15.12:42560) 2024-04-22T13:22:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.363+0000 7f440bfff640 1 -- 172.21.15.12:0/784733308 learned_addr learned my addr 172.21.15.12:0/784733308 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.363+0000 7f44137fe640 1 --2- 172.21.15.12:0/784733308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4414103b40 0x7f4414074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.364+0000 7f4412ffd640 1 -- 172.21.15.12:0/784733308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4414103180 msgr2=0x7f4414074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.364+0000 7f4412ffd640 1 --2- 172.21.15.12:0/784733308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4414103180 0x7f4414074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.364+0000 7f4412ffd640 1 -- 172.21.15.12:0/784733308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4414103b40 msgr2=0x7f4414074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.364+0000 7f4412ffd640 1 --2- 172.21.15.12:0/784733308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4414103b40 0x7f4414074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.364+0000 7f4412ffd640 1 -- 172.21.15.12:0/784733308 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4414075110 con 0x7f4414101f90 2024-04-22T13:22:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.364+0000 7f440bfff640 1 --2- 172.21.15.12:0/784733308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4414103180 0x7f4414074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:22:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.364+0000 7f4412ffd640 1 --2- 172.21.15.12:0/784733308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4414101f90 0x7f4414075d00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4404036040 tx=0x7f4404005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.364+0000 7f4410ff9640 1 -- 172.21.15.12:0/784733308 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4404043020 con 0x7f4414101f90 2024-04-22T13:22:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.364+0000 7f44196ba640 1 -- 172.21.15.12:0/784733308 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44141a2310 con 0x7f4414101f90 2024-04-22T13:22:04.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.364+0000 7f4410ff9640 1 -- 172.21.15.12:0/784733308 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4404004350 con 0x7f4414101f90 2024-04-22T13:22:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.364+0000 7f4410ff9640 1 -- 172.21.15.12:0/784733308 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4404003b20 con 0x7f4414101f90 2024-04-22T13:22:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.364+0000 7f44196ba640 1 -- 172.21.15.12:0/784733308 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44141a2880 con 0x7f4414101f90 2024-04-22T13:22:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.365+0000 7f44196ba640 1 -- 172.21.15.12:0/784733308 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44141093f0 con 0x7f4414101f90 2024-04-22T13:22:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.369+0000 7f4410ff9640 1 -- 172.21.15.12:0/784733308 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4404046020 con 0x7f4414101f90 2024-04-22T13:22:04.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.370+0000 7f4410ff9640 1 --2- 172.21.15.12:0/784733308 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f44000a3bb0 0x7f44000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:04.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.370+0000 7f440bfff640 1 --2- 172.21.15.12:0/784733308 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f44000a3bb0 0x7f44000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:04.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.370+0000 7f4410ff9640 1 -- 172.21.15.12:0/784733308 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7879+0+0 (secure 0 0 0) 0x7f44040f2a10 con 0x7f4414101f90 2024-04-22T13:22:04.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.370+0000 7f440bfff640 1 --2- 172.21.15.12:0/784733308 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f44000a3bb0 0x7f44000a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f43fc0095f0 tx=0x7f43fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:04.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.373+0000 7f4410ff9640 1 -- 172.21.15.12:0/784733308 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f440404d020 con 0x7f4414101f90 2024-04-22T13:22:04.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:04 smithi111 ceph-mon[31361]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:22:04.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:04 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2487511941' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-22T13:22:04.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:04 smithi111 ceph-mon[31361]: osdmap e84: 12 total, 12 up, 12 in 2024-04-22T13:22:04.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:04 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:04.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:04 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:04.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:04 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:04.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:04 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:04.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:04 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/4287923786' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-22T13:22:04.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:04 smithi111 ceph-mon[31361]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-22T13:22:04.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:04 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:04.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:04 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:04.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:04 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:04.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:04 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:22:04.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:04 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:04.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:04 smithi012 ceph-mon[28265]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:22:04.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:04 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2487511941' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-22T13:22:04.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:04 smithi012 ceph-mon[28265]: osdmap e84: 12 total, 12 up, 12 in 2024-04-22T13:22:04.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:04.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:04.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:04.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:04.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:04 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/4287923786' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-22T13:22:04.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:04 smithi012 ceph-mon[28265]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-22T13:22:04.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:04.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:04.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:04.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:22:04.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:04.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:04.600+0000 7f44196ba640 1 -- 172.21.15.12:0/784733308 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f441402d5c0 con 0x7f4414101f90 2024-04-22T13:22:04.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:04 smithi134 ceph-mon[31338]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:22:04.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:04 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2487511941' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-22T13:22:04.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:04 smithi134 ceph-mon[31338]: osdmap e84: 12 total, 12 up, 12 in 2024-04-22T13:22:04.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:04 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:04.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:04 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:04.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:04 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:04.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:04 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:04.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:04 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/4287923786' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-22T13:22:04.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:04 smithi134 ceph-mon[31338]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-22T13:22:04.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:04 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:04.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:04 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:04.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:04 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:22:04.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:04 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:22:04.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:04 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:05.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.365+0000 7f4410ff9640 1 -- 172.21.15.12:0/784733308 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f44040bf540 con 0x7f4414101f90 2024-04-22T13:22:05.364 INFO:teuthology.orchestra.run.smithi012.stderr:new fs with metadata pool 2 and data pool 3 2024-04-22T13:22:05.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.369+0000 7f44196ba640 1 -- 172.21.15.12:0/784733308 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f44000a3bb0 msgr2=0x7f44000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:05.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.369+0000 7f44196ba640 1 --2- 172.21.15.12:0/784733308 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f44000a3bb0 0x7f44000a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f43fc0095f0 tx=0x7f43fc005f90 comp rx=0 tx=0).stop 2024-04-22T13:22:05.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.369+0000 7f44196ba640 1 -- 172.21.15.12:0/784733308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4414101f90 msgr2=0x7f4414075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:05.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.369+0000 7f44196ba640 1 --2- 172.21.15.12:0/784733308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4414101f90 0x7f4414075d00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4404036040 tx=0x7f4404005dc0 comp rx=0 tx=0).stop 2024-04-22T13:22:05.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.369+0000 7f44196ba640 1 -- 172.21.15.12:0/784733308 shutdown_connections 2024-04-22T13:22:05.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.369+0000 7f44196ba640 1 --2- 172.21.15.12:0/784733308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4414103b40 0x7f4414074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:05.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.369+0000 7f44196ba640 1 --2- 172.21.15.12:0/784733308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4414103180 0x7f4414074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:05.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.369+0000 7f44196ba640 1 --2- 172.21.15.12:0/784733308 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f44000a3bb0 0x7f44000a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:05.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.369+0000 7f44196ba640 1 --2- 172.21.15.12:0/784733308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4414101f90 0x7f4414075d00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:05.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.369+0000 7f44196ba640 1 -- 172.21.15.12:0/784733308 >> 172.21.15.12:0/784733308 conn(0x7f44140fd500 msgr2=0x7f44141065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:05.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.369+0000 7f44196ba640 1 -- 172.21.15.12:0/784733308 shutdown_connections 2024-04-22T13:22:05.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.370+0000 7f44196ba640 1 -- 172.21.15.12:0/784733308 wait complete. 2024-04-22T13:22:05.381 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:05 smithi012 ceph-mon[28265]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-22T13:22:05.381 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:05 smithi012 ceph-mon[28265]: osdmap e85: 12 total, 12 up, 12 in 2024-04-22T13:22:05.381 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:05 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:05.381 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:05 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/784733308' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-22T13:22:05.381 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:05 smithi012 ceph-mon[28265]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-22T13:22:05.382 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:22:05.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:05 smithi111 ceph-mon[31361]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-22T13:22:05.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:05 smithi111 ceph-mon[31361]: osdmap e85: 12 total, 12 up, 12 in 2024-04-22T13:22:05.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:05 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:05.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:05 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/784733308' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-22T13:22:05.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:05 smithi111 ceph-mon[31361]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-22T13:22:05.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.490+0000 7f579e031640 1 -- 172.21.15.12:0/1860290100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5798100e70 msgr2=0x7f5798101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:05.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.490+0000 7f579e031640 1 --2- 172.21.15.12:0/1860290100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5798100e70 0x7f5798101290 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5788009900 tx=0x7f578802bc30 comp rx=0 tx=0).stop 2024-04-22T13:22:05.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.490+0000 7f579e031640 1 -- 172.21.15.12:0/1860290100 shutdown_connections 2024-04-22T13:22:05.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.490+0000 7f579e031640 1 --2- 172.21.15.12:0/1860290100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5798102a10 0x7f5798109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:05.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.490+0000 7f579e031640 1 --2- 172.21.15.12:0/1860290100 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5798102050 0x7f57981024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:05.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.490+0000 7f579e031640 1 --2- 172.21.15.12:0/1860290100 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5798100e70 0x7f5798101290 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:05.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.490+0000 7f579e031640 1 -- 172.21.15.12:0/1860290100 >> 172.21.15.12:0/1860290100 conn(0x7f57980fc470 msgr2=0x7f57980fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:05.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.490+0000 7f579e031640 1 -- 172.21.15.12:0/1860290100 shutdown_connections 2024-04-22T13:22:05.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.490+0000 7f579e031640 1 -- 172.21.15.12:0/1860290100 wait complete. 2024-04-22T13:22:05.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.490+0000 7f579e031640 1 Processor -- start 2024-04-22T13:22:05.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.491+0000 7f579e031640 1 -- start start 2024-04-22T13:22:05.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.491+0000 7f579e031640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5798100e70 0x7f57981980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:05.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.491+0000 7f579e031640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5798102050 0x7f5798198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:05.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.491+0000 7f579e031640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5798102a10 0x7f579819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:05.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.491+0000 7f579e031640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f579810c9e0 con 0x7f5798102050 2024-04-22T13:22:05.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.491+0000 7f579e031640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f579810c860 con 0x7f5798100e70 2024-04-22T13:22:05.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.491+0000 7f579e031640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f579810cb60 con 0x7f5798102a10 2024-04-22T13:22:05.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.491+0000 7f5796ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5798102050 0x7f5798198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:05.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.491+0000 7f5796ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5798102050 0x7f5798198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42578/0 (socket says 172.21.15.12:42578) 2024-04-22T13:22:05.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.491+0000 7f5796ffd640 1 -- 172.21.15.12:0/3979499052 learned_addr learned my addr 172.21.15.12:0/3979499052 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:05.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.491+0000 7f57977fe640 1 --2- 172.21.15.12:0/3979499052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5798100e70 0x7f57981980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:05.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.491+0000 7f5797fff640 1 --2- 172.21.15.12:0/3979499052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5798102a10 0x7f579819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:05.491 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.491+0000 7f5796ffd640 1 -- 172.21.15.12:0/3979499052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5798102a10 msgr2=0x7f579819f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:05.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.491+0000 7f5796ffd640 1 --2- 172.21.15.12:0/3979499052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5798102a10 0x7f579819f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:05.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.491+0000 7f5796ffd640 1 -- 172.21.15.12:0/3979499052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5798100e70 msgr2=0x7f57981980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:05.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.491+0000 7f5796ffd640 1 --2- 172.21.15.12:0/3979499052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5798100e70 0x7f57981980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:05.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.491+0000 7f5796ffd640 1 -- 172.21.15.12:0/3979499052 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f579819fd10 con 0x7f5798102050 2024-04-22T13:22:05.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.491+0000 7f5796ffd640 1 --2- 172.21.15.12:0/3979499052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5798102050 0x7f5798198630 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f578c00c920 tx=0x7f578c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:05.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.492+0000 7f5794ff9640 1 -- 172.21.15.12:0/3979499052 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c007c70 con 0x7f5798102050 2024-04-22T13:22:05.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.492+0000 7f5794ff9640 1 -- 172.21.15.12:0/3979499052 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f578c007e10 con 0x7f5798102050 2024-04-22T13:22:05.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.492+0000 7f579e031640 1 -- 172.21.15.12:0/3979499052 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f579819ffa0 con 0x7f5798102050 2024-04-22T13:22:05.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.492+0000 7f5794ff9640 1 -- 172.21.15.12:0/3979499052 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c018820 con 0x7f5798102050 2024-04-22T13:22:05.492 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.492+0000 7f579e031640 1 -- 172.21.15.12:0/3979499052 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57981a04c0 con 0x7f5798102050 2024-04-22T13:22:05.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.493+0000 7f579e031640 1 -- 172.21.15.12:0/3979499052 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57981a0750 con 0x7f5798102050 2024-04-22T13:22:05.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.494+0000 7f5794ff9640 1 -- 172.21.15.12:0/3979499052 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f578c00f450 con 0x7f5798102050 2024-04-22T13:22:05.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.494+0000 7f5794ff9640 1 --2- 172.21.15.12:0/3979499052 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f57800a3bb0 0x7f57800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:05.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.494+0000 7f57977fe640 1 --2- 172.21.15.12:0/3979499052 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f57800a3bb0 0x7f57800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:05.493 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.495+0000 7f5794ff9640 1 -- 172.21.15.12:0/3979499052 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f578c0d02f0 con 0x7f5798102050 2024-04-22T13:22:05.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.495+0000 7f57977fe640 1 --2- 172.21.15.12:0/3979499052 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f57800a3bb0 0x7f57800a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5788009230 tx=0x7f5788005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:05.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.499+0000 7f5794ff9640 1 -- 172.21.15.12:0/3979499052 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f578c014030 con 0x7f5798102050 2024-04-22T13:22:05.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:05 smithi134 ceph-mon[31338]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-22T13:22:05.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:05 smithi134 ceph-mon[31338]: osdmap e85: 12 total, 12 up, 12 in 2024-04-22T13:22:05.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:05 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:22:05.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:05 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/784733308' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-22T13:22:05.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:05 smithi134 ceph-mon[31338]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-22T13:22:05.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.688+0000 7f579e031640 1 -- 172.21.15.12:0/3979499052 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f57981a0a40 con 0x7f5798102050 2024-04-22T13:22:05.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.690+0000 7f5794ff9640 1 -- 172.21.15.12:0/3979499052 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22246 (secure 0 0 0) 0x7f578c09ce20 con 0x7f5798102050 2024-04-22T13:22:05.688 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:22:05.689 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":86,"fsid":"0aede5ea-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:14:09.362918+0000","modified":"2024-04-22T13:22:04.602609+0000","last_up_change":"2024-04-22T13:21:15.263393+0000","last_in_change":"2024-04-22T13:20:54.174685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:17:32.372434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:22:02.768799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:22:03.578730+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d9e2365-3a64-4b13-96fc-c217b1cee2ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6803","nonce":1191350632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6805","nonce":1191350632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6809","nonce":1191350632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6807","nonce":1191350632}]},"public_addr":"172.21.15.12:6803/1191350632","cluster_addr":"172.21.15.12:6805/1191350632","heartbeat_back_addr":"172.21.15.12:6809/1191350632","heartbeat_front_addr":"172.21.15.12:6807/1191350632","state":["exists","up"]},{"osd":1,"uuid":"1499c783-2bd7-48ce-a444-34add42bcf31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6801","nonce":417496465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6803","nonce":417496465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6807","nonce":417496465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6805","nonce":417496465}]},"public_addr":"172.21.15.111:6801/417496465","cluster_addr":"172.21.15.111:6803/417496465","heartbeat_back_addr":"172.21.15.111:6807/417496465","heartbeat_front_addr":"172.21.15.111:6805/417496465","state":["exists","up"]},{"osd":2,"uuid":"71a9ebc3-c622-434e-a6c2-f48358f384f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6801","nonce":1403153998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6803","nonce":1403153998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6807","nonce":1403153998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6805","nonce":1403153998}]},"public_addr":"172.21.15.134:6801/1403153998","cluster_addr":"172.21.15.134:6803/1403153998","heartbeat_back_addr":"172.21.15.134:6807/1403153998","heartbeat_front_addr":"172.21.15.134:6805/1403153998","state":["exists","up"]},{"osd":3,"uuid":"6dd00db2-a286-48ba-8666-56ffdd5f35ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6811","nonce":3359066681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6813","nonce":3359066681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6817","nonce":3359066681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6815","nonce":3359066681}]},"public_addr":"172.21.15.12:6811/3359066681","cluster_addr":"172.21.15.12:6813/3359066681","heartbeat_back_addr":"172.21.15.12:6817/3359066681","heartbeat_front_addr":"172.21.15.12:6815/3359066681","state":["exists","up"]},{"osd":4,"uuid":"1e29afd7-354c-4506-aaa0-b43eb1a19a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6809","nonce":2886349872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6811","nonce":2886349872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6815","nonce":2886349872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6813","nonce":2886349872}]},"public_addr":"172.21.15.111:6809/2886349872","cluster_addr":"172.21.15.111:6811/2886349872","heartbeat_back_addr":"172.21.15.111:6815/2886349872","heartbeat_front_addr":"172.21.15.111:6813/2886349872","state":["exists","up"]},{"osd":5,"uuid":"8f40f02d-0ba2-4842-a03d-2a1a04d1010d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6809","nonce":3071977602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6811","nonce":3071977602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6815","nonce":3071977602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6813","nonce":3071977602}]},"public_addr":"172.21.15.134:6809/3071977602","cluster_addr":"172.21.15.134:6811/3071977602","heartbeat_back_addr":"172.21.15.134:6815/3071977602","heartbeat_front_addr":"172.21.15.134:6813/3071977602","state":["exists","up"]},{"osd":6,"uuid":"bb0c17a9-6a8e-4be6-8660-f73cfa8200ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6819","nonce":3819619299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6821","nonce":3819619299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6825","nonce":3819619299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6823","nonce":3819619299}]},"public_addr":"172.21.15.12:6819/3819619299","cluster_addr":"172.21.15.12:6821/3819619299","heartbeat_back_addr":"172.21.15.12:6825/3819619299","heartbeat_front_addr":"172.21.15.12:6823/3819619299","state":["exists","up"]},{"osd":7,"uuid":"0a93d6af-569e-4e45-b697-9e978c1c1004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6817","nonce":3113856282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6819","nonce":3113856282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6823","nonce":3113856282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6821","nonce":3113856282}]},"public_addr":"172.21.15.111:6817/3113856282","cluster_addr":"172.21.15.111:6819/3113856282","heartbeat_back_addr":"172.21.15.111:6823/3113856282","heartbeat_front_addr":"172.21.15.111:6821/3113856282","state":["exists","up"]},{"osd":8,"uuid":"e21acc68-cf50-4f4f-b1ba-cf6c195b4955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6817","nonce":1689362191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6819","nonce":1689362191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6823","nonce":1689362191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6821","nonce":1689362191}]},"public_addr":"172.21.15.134:6817/1689362191","cluster_addr":"172.21.15.134:6819/1689362191","heartbeat_back_addr":"172.21.15.134:6823/1689362191","heartbeat_front_addr":"172.21.15.134:6821/1689362191","state":["exists","up"]},{"osd":9,"uuid":"fd745869-7f5f-48a8-b412-bb9c92b586e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6827","nonce":4292385779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6829","nonce":4292385779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6833","nonce":4292385779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6831","nonce":4292385779}]},"public_addr":"172.21.15.12:6827/4292385779","cluster_addr":"172.21.15.12:6829/4292385779","heartbeat_back_addr":"172.21.15.12:6833/4292385779","heartbeat_front_addr":"172.21.15.12:6831/4292385779","state":["exists","up"]},{"osd":10,"uuid":"798b1abd-f6c7-4a5f-8da0-fdae49e50d2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6825","nonce":3338566976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6827","nonce":3338566976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6831","nonce":3338566976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6829","nonce":3338566976}]},"public_addr":"172.21.15.111:6825/3338566976","cluster_addr":"172.21.15.111:6827/3338566976","heartbeat_back_addr":"172.21.15.111:6831/3338566976","heartbeat_front_addr":"172.21.15.111:6829/3338566976","state":["exists","up"]},{"osd":11,"uuid":"b18fc5f3-1049-4871-b060-e99290273cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6825","nonce":578640428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6827","nonce":578640428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6831","nonce":578640428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6829","nonce":578640428}]},"public_addr":"172.21.15.134:6825/578640428","cluster_addr":"172.21.15.134:6827/578640428","heartbeat_back_addr":"172.21.15.134:6831/578640428","heartbeat_front_addr":"172.21.15.134:6829/578640428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:16:40.828608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:05.052528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:28.297316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:53.496497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:17.947259+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:41.712894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:07.105945+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:32.129688+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:56.008516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:22.239867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:47.334692+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:12.301276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/2593601362":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/2519382069":"2024-04-23T13:15:04.279990+0000","172.21.15.12:6801/2009831940":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3509412187":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1007749062":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3588480095":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1904962176":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2009831940":"2024-04-23T13:15:04.279990+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:22:05.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.692+0000 7f579e031640 1 -- 172.21.15.12:0/3979499052 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f57800a3bb0 msgr2=0x7f57800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:05.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.692+0000 7f579e031640 1 --2- 172.21.15.12:0/3979499052 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f57800a3bb0 0x7f57800a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5788009230 tx=0x7f5788005fd0 comp rx=0 tx=0).stop 2024-04-22T13:22:05.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.692+0000 7f579e031640 1 -- 172.21.15.12:0/3979499052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5798102050 msgr2=0x7f5798198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:05.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.692+0000 7f579e031640 1 --2- 172.21.15.12:0/3979499052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5798102050 0x7f5798198630 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f578c00c920 tx=0x7f578c00cde0 comp rx=0 tx=0).stop 2024-04-22T13:22:05.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.692+0000 7f579e031640 1 -- 172.21.15.12:0/3979499052 shutdown_connections 2024-04-22T13:22:05.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.692+0000 7f579e031640 1 --2- 172.21.15.12:0/3979499052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5798102a10 0x7f579819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:05.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.692+0000 7f579e031640 1 --2- 172.21.15.12:0/3979499052 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f57800a3bb0 0x7f57800a6070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:05.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.692+0000 7f579e031640 1 --2- 172.21.15.12:0/3979499052 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5798102050 0x7f5798198630 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:05.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.692+0000 7f579e031640 1 --2- 172.21.15.12:0/3979499052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5798100e70 0x7f57981980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:05.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.692+0000 7f579e031640 1 -- 172.21.15.12:0/3979499052 >> 172.21.15.12:0/3979499052 conn(0x7f57980fc470 msgr2=0x7f57980fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:05.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.692+0000 7f579e031640 1 -- 172.21.15.12:0/3979499052 shutdown_connections 2024-04-22T13:22:05.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.692+0000 7f579e031640 1 -- 172.21.15.12:0/3979499052 wait complete. 2024-04-22T13:22:05.702 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:22:05.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.813+0000 7f18a01bd640 1 -- 172.21.15.12:0/1450307288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1898100e80 msgr2=0x7f18981012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:05.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.813+0000 7f18a01bd640 1 --2- 172.21.15.12:0/1450307288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1898100e80 0x7f18981012a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1894009900 tx=0x7f189402bc30 comp rx=0 tx=0).stop 2024-04-22T13:22:05.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.813+0000 7f18a01bd640 1 -- 172.21.15.12:0/1450307288 shutdown_connections 2024-04-22T13:22:05.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.813+0000 7f18a01bd640 1 --2- 172.21.15.12:0/1450307288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1898102990 0x7f18981092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:05.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.813+0000 7f18a01bd640 1 --2- 172.21.15.12:0/1450307288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1898101fd0 0x7f1898102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:05.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.813+0000 7f18a01bd640 1 --2- 172.21.15.12:0/1450307288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1898100e80 0x7f18981012a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:05.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.813+0000 7f18a01bd640 1 -- 172.21.15.12:0/1450307288 >> 172.21.15.12:0/1450307288 conn(0x7f18980fc410 msgr2=0x7f18980fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:05.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.813+0000 7f18a01bd640 1 -- 172.21.15.12:0/1450307288 shutdown_connections 2024-04-22T13:22:05.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.813+0000 7f18a01bd640 1 -- 172.21.15.12:0/1450307288 wait complete. 2024-04-22T13:22:05.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.814+0000 7f18a01bd640 1 Processor -- start 2024-04-22T13:22:05.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.814+0000 7f18a01bd640 1 -- start start 2024-04-22T13:22:05.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.814+0000 7f18a01bd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1898100e80 0x7f189806b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:05.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.814+0000 7f18a01bd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1898101fd0 0x7f189819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:05.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.814+0000 7f18a01bd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1898102990 0x7f189819f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:05.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.814+0000 7f18a01bd640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f189810c9b0 con 0x7f1898101fd0 2024-04-22T13:22:05.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.814+0000 7f18a01bd640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f189810c830 con 0x7f1898102990 2024-04-22T13:22:05.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.814+0000 7f18a01bd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f189810cb30 con 0x7f1898100e80 2024-04-22T13:22:05.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.814+0000 7f189d758640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1898101fd0 0x7f189819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:05.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.814+0000 7f189d758640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1898101fd0 0x7f189819ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42604/0 (socket says 172.21.15.12:42604) 2024-04-22T13:22:05.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.814+0000 7f189d758640 1 -- 172.21.15.12:0/209192652 learned_addr learned my addr 172.21.15.12:0/209192652 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:05.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.814+0000 7f189e75a640 1 --2- 172.21.15.12:0/209192652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1898102990 0x7f189819f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:05.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.814+0000 7f189df59640 1 --2- 172.21.15.12:0/209192652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1898100e80 0x7f189806b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:05.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.814+0000 7f189d758640 1 -- 172.21.15.12:0/209192652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1898100e80 msgr2=0x7f189806b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:05.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.814+0000 7f189d758640 1 --2- 172.21.15.12:0/209192652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1898100e80 0x7f189806b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:05.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.814+0000 7f189d758640 1 -- 172.21.15.12:0/209192652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1898102990 msgr2=0x7f189819f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:05.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.814+0000 7f189d758640 1 --2- 172.21.15.12:0/209192652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1898102990 0x7f189819f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:05.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.814+0000 7f189d758640 1 -- 172.21.15.12:0/209192652 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f189819fd60 con 0x7f1898101fd0 2024-04-22T13:22:05.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.815+0000 7f189d758640 1 --2- 172.21.15.12:0/209192652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1898101fd0 0x7f189819ce60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f188800c970 tx=0x7f188800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:05.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.815+0000 7f188effd640 1 -- 172.21.15.12:0/209192652 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1888004220 con 0x7f1898101fd0 2024-04-22T13:22:05.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.815+0000 7f188effd640 1 -- 172.21.15.12:0/209192652 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18880043c0 con 0x7f1898101fd0 2024-04-22T13:22:05.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.815+0000 7f18a01bd640 1 -- 172.21.15.12:0/209192652 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f189819fff0 con 0x7f1898101fd0 2024-04-22T13:22:05.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.815+0000 7f188effd640 1 -- 172.21.15.12:0/209192652 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188800f450 con 0x7f1898101fd0 2024-04-22T13:22:05.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.815+0000 7f18a01bd640 1 -- 172.21.15.12:0/209192652 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18981a0510 con 0x7f1898101fd0 2024-04-22T13:22:05.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.816+0000 7f18a01bd640 1 -- 172.21.15.12:0/209192652 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1898108190 con 0x7f1898101fd0 2024-04-22T13:22:05.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.817+0000 7f188effd640 1 -- 172.21.15.12:0/209192652 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1888003610 con 0x7f1898101fd0 2024-04-22T13:22:05.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.818+0000 7f188effd640 1 --2- 172.21.15.12:0/209192652 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f18840a3bb0 0x7f18840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:05.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.818+0000 7f189df59640 1 --2- 172.21.15.12:0/209192652 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f18840a3bb0 0x7f18840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:05.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.818+0000 7f188effd640 1 -- 172.21.15.12:0/209192652 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f18880d0430 con 0x7f1898101fd0 2024-04-22T13:22:05.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.818+0000 7f189df59640 1 --2- 172.21.15.12:0/209192652 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f18840a3bb0 0x7f18840a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1894000c00 tx=0x7f189402c800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:05.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:05.821+0000 7f188effd640 1 -- 172.21.15.12:0/209192652 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1888014030 con 0x7f1898101fd0 2024-04-22T13:22:06.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.009+0000 7f18a01bd640 1 -- 172.21.15.12:0/209192652 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f189802d810 con 0x7f1898101fd0 2024-04-22T13:22:06.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.010+0000 7f188effd640 1 -- 172.21.15.12:0/209192652 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22246 (secure 0 0 0) 0x7f188809cf60 con 0x7f1898101fd0 2024-04-22T13:22:06.008 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:22:06.009 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":86,"fsid":"0aede5ea-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:14:09.362918+0000","modified":"2024-04-22T13:22:04.602609+0000","last_up_change":"2024-04-22T13:21:15.263393+0000","last_in_change":"2024-04-22T13:20:54.174685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:17:32.372434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:22:02.768799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:22:03.578730+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d9e2365-3a64-4b13-96fc-c217b1cee2ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6803","nonce":1191350632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6805","nonce":1191350632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6809","nonce":1191350632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6807","nonce":1191350632}]},"public_addr":"172.21.15.12:6803/1191350632","cluster_addr":"172.21.15.12:6805/1191350632","heartbeat_back_addr":"172.21.15.12:6809/1191350632","heartbeat_front_addr":"172.21.15.12:6807/1191350632","state":["exists","up"]},{"osd":1,"uuid":"1499c783-2bd7-48ce-a444-34add42bcf31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6801","nonce":417496465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6803","nonce":417496465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6807","nonce":417496465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6805","nonce":417496465}]},"public_addr":"172.21.15.111:6801/417496465","cluster_addr":"172.21.15.111:6803/417496465","heartbeat_back_addr":"172.21.15.111:6807/417496465","heartbeat_front_addr":"172.21.15.111:6805/417496465","state":["exists","up"]},{"osd":2,"uuid":"71a9ebc3-c622-434e-a6c2-f48358f384f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6801","nonce":1403153998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6803","nonce":1403153998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6807","nonce":1403153998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6805","nonce":1403153998}]},"public_addr":"172.21.15.134:6801/1403153998","cluster_addr":"172.21.15.134:6803/1403153998","heartbeat_back_addr":"172.21.15.134:6807/1403153998","heartbeat_front_addr":"172.21.15.134:6805/1403153998","state":["exists","up"]},{"osd":3,"uuid":"6dd00db2-a286-48ba-8666-56ffdd5f35ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6811","nonce":3359066681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6813","nonce":3359066681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6817","nonce":3359066681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6815","nonce":3359066681}]},"public_addr":"172.21.15.12:6811/3359066681","cluster_addr":"172.21.15.12:6813/3359066681","heartbeat_back_addr":"172.21.15.12:6817/3359066681","heartbeat_front_addr":"172.21.15.12:6815/3359066681","state":["exists","up"]},{"osd":4,"uuid":"1e29afd7-354c-4506-aaa0-b43eb1a19a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6809","nonce":2886349872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6811","nonce":2886349872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6815","nonce":2886349872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6813","nonce":2886349872}]},"public_addr":"172.21.15.111:6809/2886349872","cluster_addr":"172.21.15.111:6811/2886349872","heartbeat_back_addr":"172.21.15.111:6815/2886349872","heartbeat_front_addr":"172.21.15.111:6813/2886349872","state":["exists","up"]},{"osd":5,"uuid":"8f40f02d-0ba2-4842-a03d-2a1a04d1010d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6809","nonce":3071977602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6811","nonce":3071977602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6815","nonce":3071977602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6813","nonce":3071977602}]},"public_addr":"172.21.15.134:6809/3071977602","cluster_addr":"172.21.15.134:6811/3071977602","heartbeat_back_addr":"172.21.15.134:6815/3071977602","heartbeat_front_addr":"172.21.15.134:6813/3071977602","state":["exists","up"]},{"osd":6,"uuid":"bb0c17a9-6a8e-4be6-8660-f73cfa8200ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6819","nonce":3819619299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6821","nonce":3819619299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6825","nonce":3819619299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6823","nonce":3819619299}]},"public_addr":"172.21.15.12:6819/3819619299","cluster_addr":"172.21.15.12:6821/3819619299","heartbeat_back_addr":"172.21.15.12:6825/3819619299","heartbeat_front_addr":"172.21.15.12:6823/3819619299","state":["exists","up"]},{"osd":7,"uuid":"0a93d6af-569e-4e45-b697-9e978c1c1004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6817","nonce":3113856282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6819","nonce":3113856282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6823","nonce":3113856282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6821","nonce":3113856282}]},"public_addr":"172.21.15.111:6817/3113856282","cluster_addr":"172.21.15.111:6819/3113856282","heartbeat_back_addr":"172.21.15.111:6823/3113856282","heartbeat_front_addr":"172.21.15.111:6821/3113856282","state":["exists","up"]},{"osd":8,"uuid":"e21acc68-cf50-4f4f-b1ba-cf6c195b4955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6817","nonce":1689362191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6819","nonce":1689362191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6823","nonce":1689362191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6821","nonce":1689362191}]},"public_addr":"172.21.15.134:6817/1689362191","cluster_addr":"172.21.15.134:6819/1689362191","heartbeat_back_addr":"172.21.15.134:6823/1689362191","heartbeat_front_addr":"172.21.15.134:6821/1689362191","state":["exists","up"]},{"osd":9,"uuid":"fd745869-7f5f-48a8-b412-bb9c92b586e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6827","nonce":4292385779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6829","nonce":4292385779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6833","nonce":4292385779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6831","nonce":4292385779}]},"public_addr":"172.21.15.12:6827/4292385779","cluster_addr":"172.21.15.12:6829/4292385779","heartbeat_back_addr":"172.21.15.12:6833/4292385779","heartbeat_front_addr":"172.21.15.12:6831/4292385779","state":["exists","up"]},{"osd":10,"uuid":"798b1abd-f6c7-4a5f-8da0-fdae49e50d2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6825","nonce":3338566976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6827","nonce":3338566976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6831","nonce":3338566976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6829","nonce":3338566976}]},"public_addr":"172.21.15.111:6825/3338566976","cluster_addr":"172.21.15.111:6827/3338566976","heartbeat_back_addr":"172.21.15.111:6831/3338566976","heartbeat_front_addr":"172.21.15.111:6829/3338566976","state":["exists","up"]},{"osd":11,"uuid":"b18fc5f3-1049-4871-b060-e99290273cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6825","nonce":578640428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6827","nonce":578640428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6831","nonce":578640428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6829","nonce":578640428}]},"public_addr":"172.21.15.134:6825/578640428","cluster_addr":"172.21.15.134:6827/578640428","heartbeat_back_addr":"172.21.15.134:6831/578640428","heartbeat_front_addr":"172.21.15.134:6829/578640428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:16:40.828608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:05.052528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:28.297316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:53.496497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:17.947259+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:41.712894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:07.105945+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:32.129688+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:56.008516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:22.239867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:47.334692+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:12.301276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/2593601362":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/2519382069":"2024-04-23T13:15:04.279990+0000","172.21.15.12:6801/2009831940":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3509412187":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1007749062":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3588480095":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1904962176":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2009831940":"2024-04-23T13:15:04.279990+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:22:06.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.013+0000 7f18a01bd640 1 -- 172.21.15.12:0/209192652 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f18840a3bb0 msgr2=0x7f18840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:06.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.013+0000 7f18a01bd640 1 --2- 172.21.15.12:0/209192652 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f18840a3bb0 0x7f18840a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1894000c00 tx=0x7f189402c800 comp rx=0 tx=0).stop 2024-04-22T13:22:06.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.013+0000 7f18a01bd640 1 -- 172.21.15.12:0/209192652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1898101fd0 msgr2=0x7f189819ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:06.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.013+0000 7f18a01bd640 1 --2- 172.21.15.12:0/209192652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1898101fd0 0x7f189819ce60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f188800c970 tx=0x7f188800ce30 comp rx=0 tx=0).stop 2024-04-22T13:22:06.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.013+0000 7f18a01bd640 1 -- 172.21.15.12:0/209192652 shutdown_connections 2024-04-22T13:22:06.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.013+0000 7f18a01bd640 1 --2- 172.21.15.12:0/209192652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1898102990 0x7f189819f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:06.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.013+0000 7f18a01bd640 1 --2- 172.21.15.12:0/209192652 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f18840a3bb0 0x7f18840a6070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:06.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.013+0000 7f18a01bd640 1 --2- 172.21.15.12:0/209192652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1898101fd0 0x7f189819ce60 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:06.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.013+0000 7f18a01bd640 1 --2- 172.21.15.12:0/209192652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1898100e80 0x7f189806b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:06.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.013+0000 7f18a01bd640 1 -- 172.21.15.12:0/209192652 >> 172.21.15.12:0/209192652 conn(0x7f18980fc410 msgr2=0x7f1898105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:06.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.013+0000 7f18a01bd640 1 -- 172.21.15.12:0/209192652 shutdown_connections 2024-04-22T13:22:06.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.013+0000 7f18a01bd640 1 -- 172.21.15.12:0/209192652 wait complete. 2024-04-22T13:22:06.022 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-22T13:22:06.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.129+0000 7f1e314b3640 1 -- 172.21.15.12:0/1733726406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e2c1029a0 msgr2=0x7f1e2c1092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:06.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.129+0000 7f1e314b3640 1 --2- 172.21.15.12:0/1733726406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e2c1029a0 0x7f1e2c1092c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c009ca0 tx=0x7f1e1c02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:22:06.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.129+0000 7f1e314b3640 1 -- 172.21.15.12:0/1733726406 shutdown_connections 2024-04-22T13:22:06.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.129+0000 7f1e314b3640 1 --2- 172.21.15.12:0/1733726406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e2c1029a0 0x7f1e2c1092c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:06.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.129+0000 7f1e314b3640 1 --2- 172.21.15.12:0/1733726406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e2c101fe0 0x7f1e2c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:06.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.129+0000 7f1e314b3640 1 --2- 172.21.15.12:0/1733726406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e2c100e90 0x7f1e2c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:06.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.129+0000 7f1e314b3640 1 -- 172.21.15.12:0/1733726406 >> 172.21.15.12:0/1733726406 conn(0x7f1e2c0fc400 msgr2=0x7f1e2c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:06.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.129+0000 7f1e314b3640 1 -- 172.21.15.12:0/1733726406 shutdown_connections 2024-04-22T13:22:06.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.129+0000 7f1e314b3640 1 -- 172.21.15.12:0/1733726406 wait complete. 2024-04-22T13:22:06.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.130+0000 7f1e314b3640 1 Processor -- start 2024-04-22T13:22:06.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.130+0000 7f1e314b3640 1 -- start start 2024-04-22T13:22:06.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.130+0000 7f1e314b3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e2c100e90 0x7f1e2c069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:06.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.130+0000 7f1e314b3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e2c101fe0 0x7f1e2c19cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:06.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.130+0000 7f1e314b3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e2c1029a0 0x7f1e2c19f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:06.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.130+0000 7f1e314b3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2c10c9f0 con 0x7f1e2c100e90 2024-04-22T13:22:06.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.130+0000 7f1e314b3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2c10c870 con 0x7f1e2c1029a0 2024-04-22T13:22:06.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.130+0000 7f1e314b3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2c10cb70 con 0x7f1e2c101fe0 2024-04-22T13:22:06.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.130+0000 7f1e2affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e2c100e90 0x7f1e2c069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:06.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.130+0000 7f1e2affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e2c100e90 0x7f1e2c069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42626/0 (socket says 172.21.15.12:42626) 2024-04-22T13:22:06.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.130+0000 7f1e2affd640 1 -- 172.21.15.12:0/198548488 learned_addr learned my addr 172.21.15.12:0/198548488 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:06.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.131+0000 7f1e2a7fc640 1 --2- 172.21.15.12:0/198548488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e2c101fe0 0x7f1e2c19cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:06.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.131+0000 7f1e2b7fe640 1 --2- 172.21.15.12:0/198548488 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e2c1029a0 0x7f1e2c19f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:06.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.131+0000 7f1e2affd640 1 -- 172.21.15.12:0/198548488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e2c101fe0 msgr2=0x7f1e2c19cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:06.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.131+0000 7f1e2affd640 1 --2- 172.21.15.12:0/198548488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e2c101fe0 0x7f1e2c19cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:06.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.131+0000 7f1e2affd640 1 -- 172.21.15.12:0/198548488 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e2c1029a0 msgr2=0x7f1e2c19f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:06.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.131+0000 7f1e2affd640 1 --2- 172.21.15.12:0/198548488 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e2c1029a0 0x7f1e2c19f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:06.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.131+0000 7f1e2affd640 1 -- 172.21.15.12:0/198548488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2c19fe20 con 0x7f1e2c100e90 2024-04-22T13:22:06.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.131+0000 7f1e2a7fc640 1 --2- 172.21.15.12:0/198548488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e2c101fe0 0x7f1e2c19cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:22:06.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.131+0000 7f1e2b7fe640 1 --2- 172.21.15.12:0/198548488 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e2c1029a0 0x7f1e2c19f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:22:06.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.131+0000 7f1e2affd640 1 --2- 172.21.15.12:0/198548488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e2c100e90 0x7f1e2c069c50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1e2400d770 tx=0x7f1e2400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:06.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.131+0000 7f1e23fff640 1 -- 172.21.15.12:0/198548488 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e24017070 con 0x7f1e2c100e90 2024-04-22T13:22:06.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.131+0000 7f1e23fff640 1 -- 172.21.15.12:0/198548488 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e240054d0 con 0x7f1e2c100e90 2024-04-22T13:22:06.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.131+0000 7f1e314b3640 1 -- 172.21.15.12:0/198548488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e2c1a00b0 con 0x7f1e2c100e90 2024-04-22T13:22:06.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.131+0000 7f1e23fff640 1 -- 172.21.15.12:0/198548488 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e24004c30 con 0x7f1e2c100e90 2024-04-22T13:22:06.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.131+0000 7f1e314b3640 1 -- 172.21.15.12:0/198548488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e2c1a0600 con 0x7f1e2c100e90 2024-04-22T13:22:06.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.132+0000 7f1e314b3640 1 -- 172.21.15.12:0/198548488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e2c1081a0 con 0x7f1e2c100e90 2024-04-22T13:22:06.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.133+0000 7f1e23fff640 1 -- 172.21.15.12:0/198548488 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1e24004dd0 con 0x7f1e2c100e90 2024-04-22T13:22:06.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.134+0000 7f1e23fff640 1 --2- 172.21.15.12:0/198548488 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1e180a3c30 0x7f1e180a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:06.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.134+0000 7f1e23fff640 1 -- 172.21.15.12:0/198548488 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f1e240d01d0 con 0x7f1e2c100e90 2024-04-22T13:22:06.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.134+0000 7f1e2a7fc640 1 --2- 172.21.15.12:0/198548488 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1e180a3c30 0x7f1e180a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:06.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.134+0000 7f1e2a7fc640 1 --2- 172.21.15.12:0/198548488 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1e180a3c30 0x7f1e180a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1e140058c0 tx=0x7f1e14005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:06.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.138+0000 7f1e23fff640 1 -- 172.21.15.12:0/198548488 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1e2409cd80 con 0x7f1e2c100e90 2024-04-22T13:22:06.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:06 smithi012 ceph-mon[28265]: pgmap v285: 97 pgs: 8 creating+peering, 1 active+clean, 88 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:22:06.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:06 smithi012 ceph-mon[28265]: osdmap e86: 12 total, 12 up, 12 in 2024-04-22T13:22:06.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:06 smithi012 ceph-mon[28265]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-22T13:22:06.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:06 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/784733308' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-22T13:22:06.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:06 smithi012 ceph-mon[28265]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-22T13:22:06.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:06 smithi012 ceph-mon[28265]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-22T13:22:06.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:06 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3979499052' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:22:06.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:06 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/209192652' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:22:06.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:06.368+0000 7f1e314b3640 1 -- 172.21.15.12:0/198548488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f1e2c02d5c0 con 0x7f1e2c100e90 2024-04-22T13:22:06.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:06 smithi111 ceph-mon[31361]: pgmap v285: 97 pgs: 8 creating+peering, 1 active+clean, 88 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:22:06.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:06 smithi111 ceph-mon[31361]: osdmap e86: 12 total, 12 up, 12 in 2024-04-22T13:22:06.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:06 smithi111 ceph-mon[31361]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-22T13:22:06.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:06 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/784733308' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-22T13:22:06.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:06 smithi111 ceph-mon[31361]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-22T13:22:06.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:06 smithi111 ceph-mon[31361]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-22T13:22:06.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:06 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/3979499052' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:22:06.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:06 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/209192652' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:22:06.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:06 smithi134 ceph-mon[31338]: pgmap v285: 97 pgs: 8 creating+peering, 1 active+clean, 88 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:22:06.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:06 smithi134 ceph-mon[31338]: osdmap e86: 12 total, 12 up, 12 in 2024-04-22T13:22:06.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:06 smithi134 ceph-mon[31338]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-22T13:22:06.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:06 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/784733308' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-22T13:22:06.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:06 smithi134 ceph-mon[31338]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-22T13:22:06.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:06 smithi134 ceph-mon[31338]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-22T13:22:06.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:06 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/3979499052' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:22:06.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:06 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/209192652' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:22:07.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.376+0000 7f1e23fff640 1 -- 172.21.15.12:0/198548488 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f1e2409c680 con 0x7f1e2c100e90 2024-04-22T13:22:07.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.380+0000 7f1e314b3640 1 -- 172.21.15.12:0/198548488 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1e180a3c30 msgr2=0x7f1e180a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:07.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.380+0000 7f1e314b3640 1 --2- 172.21.15.12:0/198548488 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1e180a3c30 0x7f1e180a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1e140058c0 tx=0x7f1e14005850 comp rx=0 tx=0).stop 2024-04-22T13:22:07.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.380+0000 7f1e314b3640 1 -- 172.21.15.12:0/198548488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e2c100e90 msgr2=0x7f1e2c069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:07.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.380+0000 7f1e314b3640 1 --2- 172.21.15.12:0/198548488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e2c100e90 0x7f1e2c069c50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1e2400d770 tx=0x7f1e2400dc30 comp rx=0 tx=0).stop 2024-04-22T13:22:07.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.380+0000 7f1e314b3640 1 -- 172.21.15.12:0/198548488 shutdown_connections 2024-04-22T13:22:07.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.380+0000 7f1e314b3640 1 --2- 172.21.15.12:0/198548488 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e2c1029a0 0x7f1e2c19f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:07.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.380+0000 7f1e314b3640 1 --2- 172.21.15.12:0/198548488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1e2c101fe0 0x7f1e2c19cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:07.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.380+0000 7f1e314b3640 1 --2- 172.21.15.12:0/198548488 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1e180a3c30 0x7f1e180a60f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:07.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.380+0000 7f1e314b3640 1 --2- 172.21.15.12:0/198548488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1e2c100e90 0x7f1e2c069c50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:07.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.380+0000 7f1e314b3640 1 -- 172.21.15.12:0/198548488 >> 172.21.15.12:0/198548488 conn(0x7f1e2c0fc400 msgr2=0x7f1e2c105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:07.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.380+0000 7f1e314b3640 1 -- 172.21.15.12:0/198548488 shutdown_connections 2024-04-22T13:22:07.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.380+0000 7f1e314b3640 1 -- 172.21.15.12:0/198548488 wait complete. 2024-04-22T13:22:07.391 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-22T13:22:07.391 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-22T13:22:07.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.504+0000 7fef7d68c640 1 -- 172.21.15.12:0/3222096828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef78101fb0 msgr2=0x7fef781023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:07.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.504+0000 7fef7d68c640 1 --2- 172.21.15.12:0/3222096828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef78101fb0 0x7fef781023d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fef68009900 tx=0x7fef6802bc30 comp rx=0 tx=0).stop 2024-04-22T13:22:07.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.504+0000 7fef7d68c640 1 -- 172.21.15.12:0/3222096828 shutdown_connections 2024-04-22T13:22:07.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.504+0000 7fef7d68c640 1 --2- 172.21.15.12:0/3222096828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef78103b60 0x7fef7810a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:07.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.504+0000 7fef7d68c640 1 --2- 172.21.15.12:0/3222096828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef781031a0 0x7fef78103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:07.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.504+0000 7fef7d68c640 1 --2- 172.21.15.12:0/3222096828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef78101fb0 0x7fef781023d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:07.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.504+0000 7fef7d68c640 1 -- 172.21.15.12:0/3222096828 >> 172.21.15.12:0/3222096828 conn(0x7fef780fd520 msgr2=0x7fef780ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:07.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.504+0000 7fef7d68c640 1 -- 172.21.15.12:0/3222096828 shutdown_connections 2024-04-22T13:22:07.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.504+0000 7fef7d68c640 1 -- 172.21.15.12:0/3222096828 wait complete. 2024-04-22T13:22:07.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.505+0000 7fef7d68c640 1 Processor -- start 2024-04-22T13:22:07.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.505+0000 7fef7d68c640 1 -- start start 2024-04-22T13:22:07.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.505+0000 7fef7d68c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef78101fb0 0x7fef78198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:07.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.505+0000 7fef7d68c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef781031a0 0x7fef781987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:07.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.505+0000 7fef7d68c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef78103b60 0x7fef7819f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:07.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.505+0000 7fef7d68c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fef7810db80 con 0x7fef78103b60 2024-04-22T13:22:07.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.505+0000 7fef7d68c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fef7810da00 con 0x7fef78101fb0 2024-04-22T13:22:07.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.505+0000 7fef7d68c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fef7810dd00 con 0x7fef781031a0 2024-04-22T13:22:07.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.505+0000 7fef777fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef78103b60 0x7fef7819f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:07.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.505+0000 7fef777fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef78103b60 0x7fef7819f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42638/0 (socket says 172.21.15.12:42638) 2024-04-22T13:22:07.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.505+0000 7fef777fe640 1 -- 172.21.15.12:0/3431074556 learned_addr learned my addr 172.21.15.12:0/3431074556 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:07.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.505+0000 7fef76ffd640 1 --2- 172.21.15.12:0/3431074556 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef78101fb0 0x7fef78198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:07.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.505+0000 7fef777fe640 1 -- 172.21.15.12:0/3431074556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef781031a0 msgr2=0x7fef781987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:07.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.505+0000 7fef777fe640 1 --2- 172.21.15.12:0/3431074556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef781031a0 0x7fef781987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:07.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.505+0000 7fef777fe640 1 -- 172.21.15.12:0/3431074556 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef78101fb0 msgr2=0x7fef78198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:07.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.506+0000 7fef777fe640 1 --2- 172.21.15.12:0/3431074556 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef78101fb0 0x7fef78198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:07.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.506+0000 7fef777fe640 1 -- 172.21.15.12:0/3431074556 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef7819fea0 con 0x7fef78103b60 2024-04-22T13:22:07.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.506+0000 7fef777fe640 1 --2- 172.21.15.12:0/3431074556 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef78103b60 0x7fef7819f830 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fef7000cad0 tx=0x7fef7000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:07.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.506+0000 7fef74ff9640 1 -- 172.21.15.12:0/3431074556 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7000ae20 con 0x7fef78103b60 2024-04-22T13:22:07.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.506+0000 7fef74ff9640 1 -- 172.21.15.12:0/3431074556 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef70005020 con 0x7fef78103b60 2024-04-22T13:22:07.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.506+0000 7fef7d68c640 1 -- 172.21.15.12:0/3431074556 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef781a0190 con 0x7fef78103b60 2024-04-22T13:22:07.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.506+0000 7fef74ff9640 1 -- 172.21.15.12:0/3431074556 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7000f6b0 con 0x7fef78103b60 2024-04-22T13:22:07.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.506+0000 7fef7d68c640 1 -- 172.21.15.12:0/3431074556 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef781a0710 con 0x7fef78103b60 2024-04-22T13:22:07.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.507+0000 7fef7d68c640 1 -- 172.21.15.12:0/3431074556 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef78109370 con 0x7fef78103b60 2024-04-22T13:22:07.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.508+0000 7fef74ff9640 1 -- 172.21.15.12:0/3431074556 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fef700051d0 con 0x7fef78103b60 2024-04-22T13:22:07.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.509+0000 7fef74ff9640 1 --2- 172.21.15.12:0/3431074556 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fef640a3bb0 0x7fef640a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:07.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.509+0000 7fef76ffd640 1 --2- 172.21.15.12:0/3431074556 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fef640a3bb0 0x7fef640a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:07.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.509+0000 7fef74ff9640 1 -- 172.21.15.12:0/3431074556 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fef700d04c0 con 0x7fef78103b60 2024-04-22T13:22:07.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.509+0000 7fef76ffd640 1 --2- 172.21.15.12:0/3431074556 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fef640a3bb0 0x7fef640a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fef6802c240 tx=0x7fef68000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:07.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.513+0000 7fef74ff9640 1 -- 172.21.15.12:0/3431074556 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fef70014030 con 0x7fef78103b60 2024-04-22T13:22:07.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:07 smithi012 ceph-mon[28265]: osdmap e87: 12 total, 12 up, 12 in 2024-04-22T13:22:07.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:07 smithi012 ceph-mon[28265]: mds.? [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] up:active 2024-04-22T13:22:07.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:07 smithi012 ceph-mon[28265]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-22T13:22:07.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:07 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/198548488' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-22T13:22:07.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:07 smithi012 ceph-mon[28265]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-22T13:22:07.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:07 smithi134 ceph-mon[31338]: osdmap e87: 12 total, 12 up, 12 in 2024-04-22T13:22:07.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:07 smithi134 ceph-mon[31338]: mds.? [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] up:active 2024-04-22T13:22:07.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:07 smithi134 ceph-mon[31338]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-22T13:22:07.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:07 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/198548488' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-22T13:22:07.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:07 smithi134 ceph-mon[31338]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-22T13:22:07.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:07 smithi111 ceph-mon[31361]: osdmap e87: 12 total, 12 up, 12 in 2024-04-22T13:22:07.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:07 smithi111 ceph-mon[31361]: mds.? [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] up:active 2024-04-22T13:22:07.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:07 smithi111 ceph-mon[31361]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-22T13:22:07.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:07 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/198548488' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-22T13:22:07.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:07 smithi111 ceph-mon[31361]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-22T13:22:07.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:07.740+0000 7fef7d68c640 1 -- 172.21.15.12:0/3431074556 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fef78062b50 con 0x7fef78103b60 2024-04-22T13:22:08.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.378+0000 7fef74ff9640 1 -- 172.21.15.12:0/3431074556 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fef7009cf50 con 0x7fef78103b60 2024-04-22T13:22:08.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.381+0000 7fef7d68c640 1 -- 172.21.15.12:0/3431074556 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fef640a3bb0 msgr2=0x7fef640a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:08.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.381+0000 7fef7d68c640 1 --2- 172.21.15.12:0/3431074556 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fef640a3bb0 0x7fef640a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fef6802c240 tx=0x7fef68000bc0 comp rx=0 tx=0).stop 2024-04-22T13:22:08.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.381+0000 7fef7d68c640 1 -- 172.21.15.12:0/3431074556 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef78103b60 msgr2=0x7fef7819f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:08.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.381+0000 7fef7d68c640 1 --2- 172.21.15.12:0/3431074556 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef78103b60 0x7fef7819f830 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fef7000cad0 tx=0x7fef7000cf90 comp rx=0 tx=0).stop 2024-04-22T13:22:08.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.381+0000 7fef7d68c640 1 -- 172.21.15.12:0/3431074556 shutdown_connections 2024-04-22T13:22:08.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.381+0000 7fef7d68c640 1 --2- 172.21.15.12:0/3431074556 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fef640a3bb0 0x7fef640a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:08.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.381+0000 7fef7d68c640 1 --2- 172.21.15.12:0/3431074556 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef78103b60 0x7fef7819f830 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:08.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.381+0000 7fef7d68c640 1 --2- 172.21.15.12:0/3431074556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef781031a0 0x7fef781987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:08.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.381+0000 7fef7d68c640 1 --2- 172.21.15.12:0/3431074556 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef78101fb0 0x7fef78198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:08.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.381+0000 7fef7d68c640 1 -- 172.21.15.12:0/3431074556 >> 172.21.15.12:0/3431074556 conn(0x7fef780fd520 msgr2=0x7fef78106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:08.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.381+0000 7fef7d68c640 1 -- 172.21.15.12:0/3431074556 shutdown_connections 2024-04-22T13:22:08.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.381+0000 7fef7d68c640 1 -- 172.21.15.12:0/3431074556 wait complete. 2024-04-22T13:22:08.395 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-22T13:22:08.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.508+0000 7f18bb606640 1 -- 172.21.15.12:0/2556103961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18b4107c90 msgr2=0x7f18b410a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:08.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.508+0000 7f18bb606640 1 --2- 172.21.15.12:0/2556103961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18b4107c90 0x7f18b410a0e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f18ac009ca0 tx=0x7f18ac02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:22:08.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.508+0000 7f18bb606640 1 -- 172.21.15.12:0/2556103961 shutdown_connections 2024-04-22T13:22:08.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.508+0000 7f18bb606640 1 --2- 172.21.15.12:0/2556103961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18b4107c90 0x7f18b410a0e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:08.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.508+0000 7f18bb606640 1 --2- 172.21.15.12:0/2556103961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18b4105360 0x7f18b4107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:08.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.508+0000 7f18bb606640 1 --2- 172.21.15.12:0/2556103961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18b4102a30 0x7f18b4104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:08.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.508+0000 7f18bb606640 1 -- 172.21.15.12:0/2556103961 >> 172.21.15.12:0/2556103961 conn(0x7f18b40fc420 msgr2=0x7f18b40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:08.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.508+0000 7f18bb606640 1 -- 172.21.15.12:0/2556103961 shutdown_connections 2024-04-22T13:22:08.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.508+0000 7f18bb606640 1 -- 172.21.15.12:0/2556103961 wait complete. 2024-04-22T13:22:08.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.509+0000 7f18bb606640 1 Processor -- start 2024-04-22T13:22:08.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.509+0000 7f18bb606640 1 -- start start 2024-04-22T13:22:08.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.509+0000 7f18bb606640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18b4102a30 0x7f18b41982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:08.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.509+0000 7f18bb606640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18b4105360 0x7f18b41987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:08.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.509+0000 7f18bb606640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18b4107c90 0x7f18b419f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:08.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.509+0000 7f18bb606640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f18b410dbd0 con 0x7f18b4105360 2024-04-22T13:22:08.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.509+0000 7f18bb606640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f18b410da50 con 0x7f18b4107c90 2024-04-22T13:22:08.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.509+0000 7f18bb606640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f18b410dd50 con 0x7f18b4102a30 2024-04-22T13:22:08.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.509+0000 7f18b8ba1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18b4105360 0x7f18b41987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:08.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.509+0000 7f18b8ba1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18b4105360 0x7f18b41987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42660/0 (socket says 172.21.15.12:42660) 2024-04-22T13:22:08.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.509+0000 7f18b8ba1640 1 -- 172.21.15.12:0/1082750528 learned_addr learned my addr 172.21.15.12:0/1082750528 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:08.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.509+0000 7f18b93a2640 1 --2- 172.21.15.12:0/1082750528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18b4102a30 0x7f18b41982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:08.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.509+0000 7f18b9ba3640 1 --2- 172.21.15.12:0/1082750528 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18b4107c90 0x7f18b419f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:08.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.509+0000 7f18b8ba1640 1 -- 172.21.15.12:0/1082750528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18b4102a30 msgr2=0x7f18b41982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:08.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.509+0000 7f18b8ba1640 1 --2- 172.21.15.12:0/1082750528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18b4102a30 0x7f18b41982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:08.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.509+0000 7f18b8ba1640 1 -- 172.21.15.12:0/1082750528 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18b4107c90 msgr2=0x7f18b419f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:08.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.509+0000 7f18b8ba1640 1 --2- 172.21.15.12:0/1082750528 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18b4107c90 0x7f18b419f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:08.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.509+0000 7f18b8ba1640 1 -- 172.21.15.12:0/1082750528 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b419fec0 con 0x7f18b4105360 2024-04-22T13:22:08.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.509+0000 7f18b9ba3640 1 --2- 172.21.15.12:0/1082750528 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18b4107c90 0x7f18b419f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:22:08.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.509+0000 7f18b8ba1640 1 --2- 172.21.15.12:0/1082750528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18b4105360 0x7f18b41987e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f18a400b450 tx=0x7f18a400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:08.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.510+0000 7f18aa7fc640 1 -- 172.21.15.12:0/1082750528 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a4015020 con 0x7f18b4105360 2024-04-22T13:22:08.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.510+0000 7f18aa7fc640 1 -- 172.21.15.12:0/1082750528 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18a4011d40 con 0x7f18b4105360 2024-04-22T13:22:08.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.510+0000 7f18bb606640 1 -- 172.21.15.12:0/1082750528 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b41a01b0 con 0x7f18b4105360 2024-04-22T13:22:08.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.510+0000 7f18aa7fc640 1 -- 172.21.15.12:0/1082750528 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a400f340 con 0x7f18b4105360 2024-04-22T13:22:08.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.510+0000 7f18bb606640 1 -- 172.21.15.12:0/1082750528 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b41a0780 con 0x7f18b4105360 2024-04-22T13:22:08.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.511+0000 7f18bb606640 1 -- 172.21.15.12:0/1082750528 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18b40fdf10 con 0x7f18b4105360 2024-04-22T13:22:08.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.512+0000 7f18aa7fc640 1 -- 172.21.15.12:0/1082750528 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f18a4018020 con 0x7f18b4105360 2024-04-22T13:22:08.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.512+0000 7f18aa7fc640 1 --2- 172.21.15.12:0/1082750528 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f18a00a3bb0 0x7f18a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:08.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.512+0000 7f18b93a2640 1 --2- 172.21.15.12:0/1082750528 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f18a00a3bb0 0x7f18a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:08.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.513+0000 7f18aa7fc640 1 -- 172.21.15.12:0/1082750528 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f18a40cfdf0 con 0x7f18b4105360 2024-04-22T13:22:08.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.513+0000 7f18b93a2640 1 --2- 172.21.15.12:0/1082750528 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f18a00a3bb0 0x7f18a00a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f18b0009890 tx=0x7f18b0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:08.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.517+0000 7f18aa7fc640 1 -- 172.21.15.12:0/1082750528 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f18a401f020 con 0x7f18b4105360 2024-04-22T13:22:08.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:08 smithi012 ceph-mon[28265]: pgmap v288: 97 pgs: 22 active+clean, 24 creating+peering, 51 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-22T13:22:08.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:08 smithi012 ceph-mon[28265]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-22T13:22:08.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:08 smithi012 ceph-mon[28265]: Cluster is now healthy 2024-04-22T13:22:08.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:08 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/198548488' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-22T13:22:08.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:08 smithi012 ceph-mon[28265]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-22T13:22:08.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:08 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3431074556' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-22T13:22:08.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:08 smithi134 ceph-mon[31338]: pgmap v288: 97 pgs: 22 active+clean, 24 creating+peering, 51 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-22T13:22:08.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:08 smithi134 ceph-mon[31338]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-22T13:22:08.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:08 smithi134 ceph-mon[31338]: Cluster is now healthy 2024-04-22T13:22:08.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:08 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/198548488' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-22T13:22:08.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:08 smithi134 ceph-mon[31338]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-22T13:22:08.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:08 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/3431074556' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-22T13:22:08.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:08 smithi111 ceph-mon[31361]: pgmap v288: 97 pgs: 22 active+clean, 24 creating+peering, 51 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-22T13:22:08.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:08 smithi111 ceph-mon[31361]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-22T13:22:08.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:08 smithi111 ceph-mon[31361]: Cluster is now healthy 2024-04-22T13:22:08.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:08 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/198548488' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-22T13:22:08.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:08 smithi111 ceph-mon[31361]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-22T13:22:08.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:08 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/3431074556' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-22T13:22:08.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:08.742+0000 7f18bb606640 1 -- 172.21.15.12:0/1082750528 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f18b402d810 con 0x7f18b4105360 2024-04-22T13:22:09.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.367+0000 7f18aa7fc640 1 -- 172.21.15.12:0/1082750528 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f18a400d040 con 0x7f18b4105360 2024-04-22T13:22:09.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.369+0000 7f18bb606640 1 -- 172.21.15.12:0/1082750528 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f18a00a3bb0 msgr2=0x7f18a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:09.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.369+0000 7f18bb606640 1 --2- 172.21.15.12:0/1082750528 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f18a00a3bb0 0x7f18a00a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f18b0009890 tx=0x7f18b0008040 comp rx=0 tx=0).stop 2024-04-22T13:22:09.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.369+0000 7f18bb606640 1 -- 172.21.15.12:0/1082750528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18b4105360 msgr2=0x7f18b41987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:09.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.369+0000 7f18bb606640 1 --2- 172.21.15.12:0/1082750528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18b4105360 0x7f18b41987e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f18a400b450 tx=0x7f18a400b910 comp rx=0 tx=0).stop 2024-04-22T13:22:09.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.369+0000 7f18bb606640 1 -- 172.21.15.12:0/1082750528 shutdown_connections 2024-04-22T13:22:09.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.369+0000 7f18bb606640 1 --2- 172.21.15.12:0/1082750528 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18b4107c90 0x7f18b419f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:09.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.369+0000 7f18bb606640 1 --2- 172.21.15.12:0/1082750528 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f18a00a3bb0 0x7f18a00a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:09.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.369+0000 7f18bb606640 1 --2- 172.21.15.12:0/1082750528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18b4105360 0x7f18b41987e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:09.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.369+0000 7f18bb606640 1 --2- 172.21.15.12:0/1082750528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18b4102a30 0x7f18b41982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:09.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.370+0000 7f18bb606640 1 -- 172.21.15.12:0/1082750528 >> 172.21.15.12:0/1082750528 conn(0x7f18b40fc420 msgr2=0x7f18b41032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:09.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.370+0000 7f18bb606640 1 -- 172.21.15.12:0/1082750528 shutdown_connections 2024-04-22T13:22:09.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.370+0000 7f18bb606640 1 -- 172.21.15.12:0/1082750528 wait complete. 2024-04-22T13:22:09.385 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-22T13:22:09.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.498+0000 7fa152aac640 1 -- 172.21.15.12:0/3375469584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa14c100e60 msgr2=0x7fa14c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:09.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.498+0000 7fa152aac640 1 --2- 172.21.15.12:0/3375469584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa14c100e60 0x7fa14c101280 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa13c009900 tx=0x7fa13c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:22:09.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.498+0000 7fa152aac640 1 -- 172.21.15.12:0/3375469584 shutdown_connections 2024-04-22T13:22:09.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.498+0000 7fa152aac640 1 --2- 172.21.15.12:0/3375469584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa14c102a00 0x7fa14c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:09.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.498+0000 7fa152aac640 1 --2- 172.21.15.12:0/3375469584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa14c102040 0x7fa14c1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:09.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.498+0000 7fa152aac640 1 --2- 172.21.15.12:0/3375469584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa14c100e60 0x7fa14c101280 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:09.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.498+0000 7fa152aac640 1 -- 172.21.15.12:0/3375469584 >> 172.21.15.12:0/3375469584 conn(0x7fa14c0fc440 msgr2=0x7fa14c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:09.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.498+0000 7fa152aac640 1 -- 172.21.15.12:0/3375469584 shutdown_connections 2024-04-22T13:22:09.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.498+0000 7fa152aac640 1 -- 172.21.15.12:0/3375469584 wait complete. 2024-04-22T13:22:09.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.499+0000 7fa152aac640 1 Processor -- start 2024-04-22T13:22:09.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.499+0000 7fa152aac640 1 -- start start 2024-04-22T13:22:09.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.499+0000 7fa152aac640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa14c100e60 0x7fa14c198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:09.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.499+0000 7fa152aac640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa14c102040 0x7fa14c1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:09.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.499+0000 7fa152aac640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa14c102a00 0x7fa14c19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:09.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.499+0000 7fa152aac640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c10db30 con 0x7fa14c102a00 2024-04-22T13:22:09.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.499+0000 7fa152aac640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c10d9b0 con 0x7fa14c102040 2024-04-22T13:22:09.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.499+0000 7fa152aac640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c10dcb0 con 0x7fa14c100e60 2024-04-22T13:22:09.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.499+0000 7fa151049640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa14c102a00 0x7fa14c19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:09.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.499+0000 7fa151049640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa14c102a00 0x7fa14c19f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42678/0 (socket says 172.21.15.12:42678) 2024-04-22T13:22:09.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.499+0000 7fa151049640 1 -- 172.21.15.12:0/1048336288 learned_addr learned my addr 172.21.15.12:0/1048336288 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:09.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.499+0000 7fa14bfff640 1 --2- 172.21.15.12:0/1048336288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa14c102040 0x7fa14c1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:09.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.499+0000 7fa151049640 1 -- 172.21.15.12:0/1048336288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa14c100e60 msgr2=0x7fa14c198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:09.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.500+0000 7fa150848640 1 --2- 172.21.15.12:0/1048336288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa14c100e60 0x7fa14c198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:09.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.500+0000 7fa151049640 1 --2- 172.21.15.12:0/1048336288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa14c100e60 0x7fa14c198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:09.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.500+0000 7fa151049640 1 -- 172.21.15.12:0/1048336288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa14c102040 msgr2=0x7fa14c1987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:09.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.500+0000 7fa151049640 1 --2- 172.21.15.12:0/1048336288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa14c102040 0x7fa14c1987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:09.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.500+0000 7fa151049640 1 -- 172.21.15.12:0/1048336288 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14c19fe40 con 0x7fa14c102a00 2024-04-22T13:22:09.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.500+0000 7fa14bfff640 1 --2- 172.21.15.12:0/1048336288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa14c102040 0x7fa14c1987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:22:09.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.500+0000 7fa150848640 1 --2- 172.21.15.12:0/1048336288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa14c100e60 0x7fa14c198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:22:09.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.500+0000 7fa151049640 1 --2- 172.21.15.12:0/1048336288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa14c102a00 0x7fa14c19f7d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa14400cad0 tx=0x7fa14400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:09.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.500+0000 7fa149ffb640 1 -- 172.21.15.12:0/1048336288 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14400ae20 con 0x7fa14c102a00 2024-04-22T13:22:09.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.500+0000 7fa149ffb640 1 -- 172.21.15.12:0/1048336288 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa144005020 con 0x7fa14c102a00 2024-04-22T13:22:09.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.500+0000 7fa152aac640 1 -- 172.21.15.12:0/1048336288 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa14c1a0130 con 0x7fa14c102a00 2024-04-22T13:22:09.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.500+0000 7fa149ffb640 1 -- 172.21.15.12:0/1048336288 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14400f6b0 con 0x7fa14c102a00 2024-04-22T13:22:09.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.500+0000 7fa152aac640 1 -- 172.21.15.12:0/1048336288 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa14c1a0680 con 0x7fa14c102a00 2024-04-22T13:22:09.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.501+0000 7fa152aac640 1 -- 172.21.15.12:0/1048336288 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa14c108320 con 0x7fa14c102a00 2024-04-22T13:22:09.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.502+0000 7fa149ffb640 1 -- 172.21.15.12:0/1048336288 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa1440051d0 con 0x7fa14c102a00 2024-04-22T13:22:09.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.503+0000 7fa149ffb640 1 --2- 172.21.15.12:0/1048336288 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa1340a3bb0 0x7fa1340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:09.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.503+0000 7fa150848640 1 --2- 172.21.15.12:0/1048336288 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa1340a3bb0 0x7fa1340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:09.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.503+0000 7fa149ffb640 1 -- 172.21.15.12:0/1048336288 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fa1440d0370 con 0x7fa14c102a00 2024-04-22T13:22:09.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.503+0000 7fa150848640 1 --2- 172.21.15.12:0/1048336288 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa1340a3bb0 0x7fa1340a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0096c0 tx=0x7fa13c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:09.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.508+0000 7fa149ffb640 1 -- 172.21.15.12:0/1048336288 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa144014030 con 0x7fa14c102a00 2024-04-22T13:22:09.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:09 smithi012 ceph-mon[28265]: pgmap v289: 97 pgs: 66 active+clean, 19 creating+peering, 12 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-04-22T13:22:09.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:09 smithi012 ceph-mon[28265]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-22T13:22:09.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:09 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3431074556' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-22T13:22:09.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:09 smithi012 ceph-mon[28265]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-22T13:22:09.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:09 smithi012 ceph-mon[28265]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-22T13:22:09.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:09 smithi012 ceph-mon[28265]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-22T13:22:09.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:09 smithi012 ceph-mon[28265]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-22T13:22:09.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:09 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1082750528' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-22T13:22:09.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:09 smithi134 ceph-mon[31338]: pgmap v289: 97 pgs: 66 active+clean, 19 creating+peering, 12 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-04-22T13:22:09.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:09 smithi134 ceph-mon[31338]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-22T13:22:09.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:09 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/3431074556' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-22T13:22:09.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:09 smithi134 ceph-mon[31338]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-22T13:22:09.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:09 smithi134 ceph-mon[31338]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-22T13:22:09.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:09 smithi134 ceph-mon[31338]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-22T13:22:09.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:09 smithi134 ceph-mon[31338]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-22T13:22:09.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:09 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1082750528' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-22T13:22:09.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:09 smithi111 ceph-mon[31361]: pgmap v289: 97 pgs: 66 active+clean, 19 creating+peering, 12 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-04-22T13:22:09.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:09 smithi111 ceph-mon[31361]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-22T13:22:09.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:09 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/3431074556' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-22T13:22:09.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:09 smithi111 ceph-mon[31361]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-22T13:22:09.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:09 smithi111 ceph-mon[31361]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-22T13:22:09.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:09 smithi111 ceph-mon[31361]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-22T13:22:09.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:09 smithi111 ceph-mon[31361]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-22T13:22:09.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:09 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1082750528' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-22T13:22:09.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:09.736+0000 7fa152aac640 1 -- 172.21.15.12:0/1048336288 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fa14c02d5c0 con 0x7fa14c102a00 2024-04-22T13:22:10.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.377+0000 7fa149ffb640 1 -- 172.21.15.12:0/1048336288 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7fa14409cea0 con 0x7fa14c102a00 2024-04-22T13:22:10.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.380+0000 7fa152aac640 1 -- 172.21.15.12:0/1048336288 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa1340a3bb0 msgr2=0x7fa1340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:10.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.380+0000 7fa152aac640 1 --2- 172.21.15.12:0/1048336288 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa1340a3bb0 0x7fa1340a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0096c0 tx=0x7fa13c000bc0 comp rx=0 tx=0).stop 2024-04-22T13:22:10.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.380+0000 7fa152aac640 1 -- 172.21.15.12:0/1048336288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa14c102a00 msgr2=0x7fa14c19f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:10.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.380+0000 7fa152aac640 1 --2- 172.21.15.12:0/1048336288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa14c102a00 0x7fa14c19f7d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa14400cad0 tx=0x7fa14400cf90 comp rx=0 tx=0).stop 2024-04-22T13:22:10.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.380+0000 7fa152aac640 1 -- 172.21.15.12:0/1048336288 shutdown_connections 2024-04-22T13:22:10.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.380+0000 7fa152aac640 1 --2- 172.21.15.12:0/1048336288 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa1340a3bb0 0x7fa1340a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:10.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.380+0000 7fa152aac640 1 --2- 172.21.15.12:0/1048336288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa14c102a00 0x7fa14c19f7d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:10.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.380+0000 7fa152aac640 1 --2- 172.21.15.12:0/1048336288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa14c102040 0x7fa14c1987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:10.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.380+0000 7fa152aac640 1 --2- 172.21.15.12:0/1048336288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa14c100e60 0x7fa14c198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:10.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.380+0000 7fa152aac640 1 -- 172.21.15.12:0/1048336288 >> 172.21.15.12:0/1048336288 conn(0x7fa14c0fc440 msgr2=0x7fa14c1054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:10.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.380+0000 7fa152aac640 1 -- 172.21.15.12:0/1048336288 shutdown_connections 2024-04-22T13:22:10.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.380+0000 7fa152aac640 1 -- 172.21.15.12:0/1048336288 wait complete. 2024-04-22T13:22:10.395 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-04-22T13:22:10.395 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-22T13:22:10.477 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 22 13:22:10 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-b[62191]: 2024-04-22T13:22:10.203+0000 7fa512624700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-22T13:22:10.477 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 22 13:22:10 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-b[62191]: 2024-04-22T13:22:10.203+0000 7fa512624700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-22T13:22:10.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:10 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1082750528' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-22T13:22:10.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:10 smithi111 ceph-mon[31361]: mds.? [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] up:active 2024-04-22T13:22:10.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:10 smithi111 ceph-mon[31361]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-22T13:22:10.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:10 smithi111 ceph-mon[31361]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2024-04-22T13:22:10.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:10 smithi111 ceph-mon[31361]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2024-04-22T13:22:10.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:10 smithi111 ceph-mon[31361]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-04-22T13:22:10.478 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:10 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1048336288' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-22T13:22:10.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.510+0000 7f3062632640 1 -- 172.21.15.12:0/2822143836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f305c103b60 msgr2=0x7f305c10a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:10.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.510+0000 7f3062632640 1 --2- 172.21.15.12:0/2822143836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f305c103b60 0x7f305c10a3f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3054009ca0 tx=0x7f305402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:22:10.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.510+0000 7f3062632640 1 -- 172.21.15.12:0/2822143836 shutdown_connections 2024-04-22T13:22:10.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.510+0000 7f3062632640 1 --2- 172.21.15.12:0/2822143836 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f305c103b60 0x7f305c10a3f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:10.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.510+0000 7f3062632640 1 --2- 172.21.15.12:0/2822143836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c1031a0 0x7f305c103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:10.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.510+0000 7f3062632640 1 --2- 172.21.15.12:0/2822143836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f305c101fb0 0x7f305c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:10.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.510+0000 7f3062632640 1 -- 172.21.15.12:0/2822143836 >> 172.21.15.12:0/2822143836 conn(0x7f305c0fd520 msgr2=0x7f305c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:10.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.510+0000 7f3062632640 1 -- 172.21.15.12:0/2822143836 shutdown_connections 2024-04-22T13:22:10.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.510+0000 7f3062632640 1 -- 172.21.15.12:0/2822143836 wait complete. 2024-04-22T13:22:10.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.511+0000 7f3062632640 1 Processor -- start 2024-04-22T13:22:10.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.511+0000 7f3062632640 1 -- start start 2024-04-22T13:22:10.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.511+0000 7f3062632640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f305c101fb0 0x7f305c1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:10.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.511+0000 7f3062632640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c1031a0 0x7f305c198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:10.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.511+0000 7f3062632640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f305c103b60 0x7f305c19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:10.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.511+0000 7f3062632640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f305c10db30 con 0x7f305c101fb0 2024-04-22T13:22:10.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.511+0000 7f3062632640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f305c10d9b0 con 0x7f305c103b60 2024-04-22T13:22:10.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.511+0000 7f3062632640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f305c10dcb0 con 0x7f305c1031a0 2024-04-22T13:22:10.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.511+0000 7f305bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f305c101fb0 0x7f305c1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:10.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.511+0000 7f305bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f305c101fb0 0x7f305c1981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42698/0 (socket says 172.21.15.12:42698) 2024-04-22T13:22:10.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.511+0000 7f305bfff640 1 -- 172.21.15.12:0/724529547 learned_addr learned my addr 172.21.15.12:0/724529547 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:10.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.511+0000 7f3060bcf640 1 --2- 172.21.15.12:0/724529547 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f305c103b60 0x7f305c19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:10.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.511+0000 7f305b7fe640 1 --2- 172.21.15.12:0/724529547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c1031a0 0x7f305c198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:10.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.511+0000 7f305bfff640 1 -- 172.21.15.12:0/724529547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c1031a0 msgr2=0x7f305c198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:10.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.511+0000 7f305bfff640 1 --2- 172.21.15.12:0/724529547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c1031a0 0x7f305c198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:10.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.511+0000 7f305bfff640 1 -- 172.21.15.12:0/724529547 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f305c103b60 msgr2=0x7f305c19f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:10.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.511+0000 7f305bfff640 1 --2- 172.21.15.12:0/724529547 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f305c103b60 0x7f305c19f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:10.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.511+0000 7f305bfff640 1 -- 172.21.15.12:0/724529547 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f305c19fe10 con 0x7f305c101fb0 2024-04-22T13:22:10.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.512+0000 7f305bfff640 1 --2- 172.21.15.12:0/724529547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f305c101fb0 0x7f305c1981f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f304c00e9c0 tx=0x7f304c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:10.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.512+0000 7f30597fa640 1 -- 172.21.15.12:0/724529547 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c00cd80 con 0x7f305c101fb0 2024-04-22T13:22:10.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.512+0000 7f30597fa640 1 -- 172.21.15.12:0/724529547 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f304c0035f0 con 0x7f305c101fb0 2024-04-22T13:22:10.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.512+0000 7f3062632640 1 -- 172.21.15.12:0/724529547 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f305c1a0100 con 0x7f305c101fb0 2024-04-22T13:22:10.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.512+0000 7f30597fa640 1 -- 172.21.15.12:0/724529547 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c010690 con 0x7f305c101fb0 2024-04-22T13:22:10.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.512+0000 7f3062632640 1 -- 172.21.15.12:0/724529547 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f305c1a06a0 con 0x7f305c101fb0 2024-04-22T13:22:10.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.513+0000 7f3062632640 1 -- 172.21.15.12:0/724529547 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f305c1093a0 con 0x7f305c101fb0 2024-04-22T13:22:10.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.514+0000 7f30597fa640 1 -- 172.21.15.12:0/724529547 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f304c0037a0 con 0x7f305c101fb0 2024-04-22T13:22:10.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.514+0000 7f30597fa640 1 --2- 172.21.15.12:0/724529547 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f30440a3c30 0x7f30440a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:10.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.515+0000 7f305b7fe640 1 --2- 172.21.15.12:0/724529547 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f30440a3c30 0x7f30440a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:10.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.515+0000 7f30597fa640 1 -- 172.21.15.12:0/724529547 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f304c0d0640 con 0x7f305c101fb0 2024-04-22T13:22:10.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.515+0000 7f305b7fe640 1 --2- 172.21.15.12:0/724529547 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f30440a3c30 0x7f30440a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f305c199700 tx=0x7f3050008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:10.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.519+0000 7f30597fa640 1 -- 172.21.15.12:0/724529547 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f304c09d170 con 0x7f305c101fb0 2024-04-22T13:22:10.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:10 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1082750528' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-22T13:22:10.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:10 smithi134 ceph-mon[31338]: mds.? [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] up:active 2024-04-22T13:22:10.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:10 smithi134 ceph-mon[31338]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-22T13:22:10.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:10 smithi134 ceph-mon[31338]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2024-04-22T13:22:10.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:10 smithi134 ceph-mon[31338]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2024-04-22T13:22:10.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:10 smithi134 ceph-mon[31338]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-04-22T13:22:10.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:10 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1048336288' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-22T13:22:10.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:10.740+0000 7f3062632640 1 -- 172.21.15.12:0/724529547 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f305c062390 con 0x7f30440a3c30 2024-04-22T13:22:10.743 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:10 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1082750528' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-22T13:22:10.743 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:10 smithi012 ceph-mon[28265]: mds.? [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] up:active 2024-04-22T13:22:10.743 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:10 smithi012 ceph-mon[28265]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-22T13:22:10.743 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:10 smithi012 ceph-mon[28265]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2024-04-22T13:22:10.743 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:10 smithi012 ceph-mon[28265]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2024-04-22T13:22:10.743 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:10 smithi012 ceph-mon[28265]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-04-22T13:22:10.744 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:10 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1048336288' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-22T13:22:11.087 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:22:10 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:22:10.744+0000 7f805c083700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:22:11.088 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:22:10 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:22:10.744+0000 7f805c083700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:22:11.088 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:22:10 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:22:10.744+0000 7f805c083700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:22:11.088 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:22:10 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:22:10.744+0000 7f805c083700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:22:11.088 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:22:10 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:22:10.744+0000 7f805c083700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:22:11.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.402+0000 7f30597fa640 1 -- 172.21.15.12:0/724529547 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f305c062390 con 0x7f30440a3c30 2024-04-22T13:22:11.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.405+0000 7f3062632640 1 -- 172.21.15.12:0/724529547 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f30440a3c30 msgr2=0x7f30440a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:11.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.405+0000 7f3062632640 1 --2- 172.21.15.12:0/724529547 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f30440a3c30 0x7f30440a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f305c199700 tx=0x7f3050008040 comp rx=0 tx=0).stop 2024-04-22T13:22:11.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.405+0000 7f3062632640 1 -- 172.21.15.12:0/724529547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f305c101fb0 msgr2=0x7f305c1981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:11.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.405+0000 7f3062632640 1 --2- 172.21.15.12:0/724529547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f305c101fb0 0x7f305c1981f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f304c00e9c0 tx=0x7f304c00ee80 comp rx=0 tx=0).stop 2024-04-22T13:22:11.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.405+0000 7f3062632640 1 -- 172.21.15.12:0/724529547 shutdown_connections 2024-04-22T13:22:11.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.405+0000 7f3062632640 1 --2- 172.21.15.12:0/724529547 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f305c103b60 0x7f305c19f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:11.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.405+0000 7f3062632640 1 --2- 172.21.15.12:0/724529547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f305c1031a0 0x7f305c198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:11.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.405+0000 7f3062632640 1 --2- 172.21.15.12:0/724529547 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f30440a3c30 0x7f30440a60f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:11.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.405+0000 7f3062632640 1 --2- 172.21.15.12:0/724529547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f305c101fb0 0x7f305c1981f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:11.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.405+0000 7f3062632640 1 -- 172.21.15.12:0/724529547 >> 172.21.15.12:0/724529547 conn(0x7f305c0fd520 msgr2=0x7f305c106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:11.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.405+0000 7f3062632640 1 -- 172.21.15.12:0/724529547 shutdown_connections 2024-04-22T13:22:11.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.405+0000 7f3062632640 1 -- 172.21.15.12:0/724529547 wait complete. 2024-04-22T13:22:11.416 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-22T13:22:11.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.532+0000 7f2f8ec43640 1 -- 172.21.15.12:0/701963408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f88102a10 msgr2=0x7f2f88104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:11.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.532+0000 7f2f8ec43640 1 --2- 172.21.15.12:0/701963408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f88102a10 0x7f2f88104e00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2f80009ca0 tx=0x7f2f8002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:22:11.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.532+0000 7f2f8ec43640 1 -- 172.21.15.12:0/701963408 shutdown_connections 2024-04-22T13:22:11.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.532+0000 7f2f8ec43640 1 --2- 172.21.15.12:0/701963408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f88107c70 0x7f2f8810a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:11.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.532+0000 7f2f8ec43640 1 --2- 172.21.15.12:0/701963408 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f88105340 0x7f2f88107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:11.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.532+0000 7f2f8ec43640 1 --2- 172.21.15.12:0/701963408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f88102a10 0x7f2f88104e00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:11.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.532+0000 7f2f8ec43640 1 -- 172.21.15.12:0/701963408 >> 172.21.15.12:0/701963408 conn(0x7f2f880fc400 msgr2=0x7f2f880fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:11.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.532+0000 7f2f8ec43640 1 -- 172.21.15.12:0/701963408 shutdown_connections 2024-04-22T13:22:11.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.532+0000 7f2f8ec43640 1 -- 172.21.15.12:0/701963408 wait complete. 2024-04-22T13:22:11.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.532+0000 7f2f8ec43640 1 Processor -- start 2024-04-22T13:22:11.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.532+0000 7f2f8ec43640 1 -- start start 2024-04-22T13:22:11.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.533+0000 7f2f8ec43640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f88102a10 0x7f2f88198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:11.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.533+0000 7f2f8ec43640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f88105340 0x7f2f88198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:11.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.533+0000 7f2f8ec43640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f88107c70 0x7f2f8819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:11.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.533+0000 7f2f8ec43640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8810dac0 con 0x7f2f88105340 2024-04-22T13:22:11.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.533+0000 7f2f8ec43640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8810d940 con 0x7f2f88102a10 2024-04-22T13:22:11.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.533+0000 7f2f8ec43640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8810dc40 con 0x7f2f88107c70 2024-04-22T13:22:11.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.533+0000 7f2f87fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f88105340 0x7f2f88198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:11.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.533+0000 7f2f8c9df640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f88102a10 0x7f2f88198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:11.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.533+0000 7f2f87fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f88105340 0x7f2f88198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42720/0 (socket says 172.21.15.12:42720) 2024-04-22T13:22:11.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.533+0000 7f2f87fff640 1 -- 172.21.15.12:0/1105761552 learned_addr learned my addr 172.21.15.12:0/1105761552 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:11.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.533+0000 7f2f8d1e0640 1 --2- 172.21.15.12:0/1105761552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f88107c70 0x7f2f8819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:11.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.533+0000 7f2f8c9df640 1 -- 172.21.15.12:0/1105761552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f88107c70 msgr2=0x7f2f8819f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:11.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.533+0000 7f2f8c9df640 1 --2- 172.21.15.12:0/1105761552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f88107c70 0x7f2f8819f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:11.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.533+0000 7f2f8c9df640 1 -- 172.21.15.12:0/1105761552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f88105340 msgr2=0x7f2f88198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:11.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.533+0000 7f2f8c9df640 1 --2- 172.21.15.12:0/1105761552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f88105340 0x7f2f88198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:11.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.533+0000 7f2f8c9df640 1 -- 172.21.15.12:0/1105761552 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f8819fe60 con 0x7f2f88102a10 2024-04-22T13:22:11.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.533+0000 7f2f8c9df640 1 --2- 172.21.15.12:0/1105761552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f88102a10 0x7f2f88198240 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2f80006010 tx=0x7f2f800346e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:11.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.533+0000 7f2f85ffb640 1 -- 172.21.15.12:0/1105761552 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f80041020 con 0x7f2f88102a10 2024-04-22T13:22:11.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.533+0000 7f2f8ec43640 1 -- 172.21.15.12:0/1105761552 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f881a00f0 con 0x7f2f88102a10 2024-04-22T13:22:11.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.534+0000 7f2f85ffb640 1 -- 172.21.15.12:0/1105761552 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f80034ca0 con 0x7f2f88102a10 2024-04-22T13:22:11.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.534+0000 7f2f85ffb640 1 -- 172.21.15.12:0/1105761552 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f80039040 con 0x7f2f88102a10 2024-04-22T13:22:11.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.534+0000 7f2f8ec43640 1 -- 172.21.15.12:0/1105761552 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f881a0600 con 0x7f2f88102a10 2024-04-22T13:22:11.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.535+0000 7f2f8ec43640 1 -- 172.21.15.12:0/1105761552 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f880fdef0 con 0x7f2f88102a10 2024-04-22T13:22:11.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.536+0000 7f2f85ffb640 1 -- 172.21.15.12:0/1105761552 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2f80044020 con 0x7f2f88102a10 2024-04-22T13:22:11.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.536+0000 7f2f85ffb640 1 --2- 172.21.15.12:0/1105761552 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f2f700a3bb0 0x7f2f700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:11.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.537+0000 7f2f87fff640 1 --2- 172.21.15.12:0/1105761552 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f2f700a3bb0 0x7f2f700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:11.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.537+0000 7f2f85ffb640 1 -- 172.21.15.12:0/1105761552 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f2f800f2410 con 0x7f2f88102a10 2024-04-22T13:22:11.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.537+0000 7f2f87fff640 1 --2- 172.21.15.12:0/1105761552 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f2f700a3bb0 0x7f2f700a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2f88199750 tx=0x7f2f7c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:11.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.545+0000 7f2f85ffb640 1 -- 172.21.15.12:0/1105761552 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2f8004b020 con 0x7f2f88102a10 2024-04-22T13:22:11.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:11 smithi134 ceph-mon[31338]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 166 B/s rd, 2.6 KiB/s wr, 12 op/s 2024-04-22T13:22:11.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:11 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1048336288' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-22T13:22:11.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:11 smithi134 ceph-mon[31338]: mds.? [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] up:active 2024-04-22T13:22:11.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:11 smithi134 ceph-mon[31338]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-22T13:22:11.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:11 smithi134 ceph-mon[31338]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-04-22T13:22:11.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:11 smithi134 ceph-mon[31338]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:creating} 2 up:standby-replay 6 up:standby 2024-04-22T13:22:11.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:11 smithi134 ceph-mon[31338]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-04-22T13:22:11.679 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:11 smithi134 ceph-mon[31338]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:22:11.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:11 smithi111 ceph-mon[31361]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 166 B/s rd, 2.6 KiB/s wr, 12 op/s 2024-04-22T13:22:11.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:11 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1048336288' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-22T13:22:11.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:11 smithi111 ceph-mon[31361]: mds.? [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] up:active 2024-04-22T13:22:11.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:11 smithi111 ceph-mon[31361]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-22T13:22:11.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:11 smithi111 ceph-mon[31361]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-04-22T13:22:11.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:11 smithi111 ceph-mon[31361]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:creating} 2 up:standby-replay 6 up:standby 2024-04-22T13:22:11.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:11 smithi111 ceph-mon[31361]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-04-22T13:22:11.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:11 smithi111 ceph-mon[31361]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:22:11.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.761+0000 7f2f8ec43640 1 -- 172.21.15.12:0/1105761552 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f2f88062390 con 0x7f2f700a3bb0 2024-04-22T13:22:11.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.784+0000 7f2f85ffb640 1 -- 172.21.15.12:0/1105761552 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2f88062390 con 0x7f2f700a3bb0 2024-04-22T13:22:11.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.786+0000 7f2f8ec43640 1 -- 172.21.15.12:0/1105761552 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f2f700a3bb0 msgr2=0x7f2f700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:11.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.786+0000 7f2f8ec43640 1 --2- 172.21.15.12:0/1105761552 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f2f700a3bb0 0x7f2f700a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2f88199750 tx=0x7f2f7c005f90 comp rx=0 tx=0).stop 2024-04-22T13:22:11.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.786+0000 7f2f8ec43640 1 -- 172.21.15.12:0/1105761552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f88102a10 msgr2=0x7f2f88198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:11.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.786+0000 7f2f8ec43640 1 --2- 172.21.15.12:0/1105761552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f88102a10 0x7f2f88198240 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2f80006010 tx=0x7f2f800346e0 comp rx=0 tx=0).stop 2024-04-22T13:22:11.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.786+0000 7f2f8ec43640 1 -- 172.21.15.12:0/1105761552 shutdown_connections 2024-04-22T13:22:11.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.786+0000 7f2f8ec43640 1 --2- 172.21.15.12:0/1105761552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f88107c70 0x7f2f8819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:11.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.786+0000 7f2f8ec43640 1 --2- 172.21.15.12:0/1105761552 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f2f700a3bb0 0x7f2f700a6070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:11.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.786+0000 7f2f8ec43640 1 --2- 172.21.15.12:0/1105761552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2f88105340 0x7f2f88198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:11.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.786+0000 7f2f8ec43640 1 --2- 172.21.15.12:0/1105761552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f88102a10 0x7f2f88198240 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:11.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.786+0000 7f2f8ec43640 1 -- 172.21.15.12:0/1105761552 >> 172.21.15.12:0/1105761552 conn(0x7f2f880fc400 msgr2=0x7f2f881048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:11.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.786+0000 7f2f8ec43640 1 -- 172.21.15.12:0/1105761552 shutdown_connections 2024-04-22T13:22:11.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.786+0000 7f2f8ec43640 1 -- 172.21.15.12:0/1105761552 wait complete. 2024-04-22T13:22:11.794 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:11 smithi012 ceph-mon[28265]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 166 B/s rd, 2.6 KiB/s wr, 12 op/s 2024-04-22T13:22:11.794 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:11 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1048336288' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-22T13:22:11.794 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:11 smithi012 ceph-mon[28265]: mds.? [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] up:active 2024-04-22T13:22:11.794 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:11 smithi012 ceph-mon[28265]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-22T13:22:11.794 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:11 smithi012 ceph-mon[28265]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-04-22T13:22:11.794 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:11 smithi012 ceph-mon[28265]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:creating} 2 up:standby-replay 6 up:standby 2024-04-22T13:22:11.794 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:11 smithi012 ceph-mon[28265]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-04-22T13:22:11.794 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:11 smithi012 ceph-mon[28265]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:22:11.795 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-22T13:22:11.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.904+0000 7f8a11f32640 1 -- 172.21.15.12:0/734161872 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a0c107c90 msgr2=0x7f8a0c10a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:11.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.904+0000 7f8a11f32640 1 --2- 172.21.15.12:0/734161872 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a0c107c90 0x7f8a0c10a0e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009ca0 tx=0x7f8a0402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:22:11.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.904+0000 7f8a11f32640 1 -- 172.21.15.12:0/734161872 shutdown_connections 2024-04-22T13:22:11.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.904+0000 7f8a11f32640 1 --2- 172.21.15.12:0/734161872 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a0c107c90 0x7f8a0c10a0e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:11.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.904+0000 7f8a11f32640 1 --2- 172.21.15.12:0/734161872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a0c105360 0x7f8a0c107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:11.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.904+0000 7f8a11f32640 1 --2- 172.21.15.12:0/734161872 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a0c102a30 0x7f8a0c104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:11.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.904+0000 7f8a11f32640 1 -- 172.21.15.12:0/734161872 >> 172.21.15.12:0/734161872 conn(0x7f8a0c0fc440 msgr2=0x7f8a0c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:11.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.904+0000 7f8a11f32640 1 -- 172.21.15.12:0/734161872 shutdown_connections 2024-04-22T13:22:11.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.904+0000 7f8a11f32640 1 -- 172.21.15.12:0/734161872 wait complete. 2024-04-22T13:22:11.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.905+0000 7f8a11f32640 1 Processor -- start 2024-04-22T13:22:11.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.905+0000 7f8a11f32640 1 -- start start 2024-04-22T13:22:11.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.905+0000 7f8a11f32640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a0c102a30 0x7f8a0c198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:11.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.905+0000 7f8a11f32640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a0c105360 0x7f8a0c1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:11.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.905+0000 7f8a11f32640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a0c107c90 0x7f8a0c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:11.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.905+0000 7f8a11f32640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c10daf0 con 0x7f8a0c105360 2024-04-22T13:22:11.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.905+0000 7f8a11f32640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c10d970 con 0x7f8a0c107c90 2024-04-22T13:22:11.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.905+0000 7f8a11f32640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c10dc70 con 0x7f8a0c102a30 2024-04-22T13:22:11.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.905+0000 7f8a0affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a0c105360 0x7f8a0c1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:11.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.905+0000 7f8a0affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a0c105360 0x7f8a0c1986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39804/0 (socket says 172.21.15.12:39804) 2024-04-22T13:22:11.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.905+0000 7f8a0affd640 1 -- 172.21.15.12:0/629754128 learned_addr learned my addr 172.21.15.12:0/629754128 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:11.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.905+0000 7f8a0bfff640 1 --2- 172.21.15.12:0/629754128 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a0c107c90 0x7f8a0c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:11.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.905+0000 7f8a0b7fe640 1 --2- 172.21.15.12:0/629754128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a0c102a30 0x7f8a0c198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:11.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.905+0000 7f8a0affd640 1 -- 172.21.15.12:0/629754128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a0c102a30 msgr2=0x7f8a0c198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:11.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.905+0000 7f8a0affd640 1 --2- 172.21.15.12:0/629754128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a0c102a30 0x7f8a0c198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:11.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.905+0000 7f8a0affd640 1 -- 172.21.15.12:0/629754128 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a0c107c90 msgr2=0x7f8a0c19f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:11.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.905+0000 7f8a0affd640 1 --2- 172.21.15.12:0/629754128 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a0c107c90 0x7f8a0c19f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:11.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.905+0000 7f8a0affd640 1 -- 172.21.15.12:0/629754128 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0c19fd90 con 0x7f8a0c105360 2024-04-22T13:22:11.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.905+0000 7f8a0b7fe640 1 --2- 172.21.15.12:0/629754128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a0c102a30 0x7f8a0c198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:22:11.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.906+0000 7f8a0affd640 1 --2- 172.21.15.12:0/629754128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a0c105360 0x7f8a0c1986b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000d740 tx=0x7f8a0000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:11.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.906+0000 7f8a08ff9640 1 -- 172.21.15.12:0/629754128 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a00017070 con 0x7f8a0c105360 2024-04-22T13:22:11.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.906+0000 7f8a08ff9640 1 -- 172.21.15.12:0/629754128 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a0000de50 con 0x7f8a0c105360 2024-04-22T13:22:11.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.906+0000 7f8a11f32640 1 -- 172.21.15.12:0/629754128 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a0c1a0080 con 0x7f8a0c105360 2024-04-22T13:22:11.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.906+0000 7f8a08ff9640 1 -- 172.21.15.12:0/629754128 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a00004ce0 con 0x7f8a0c105360 2024-04-22T13:22:11.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.906+0000 7f8a11f32640 1 -- 172.21.15.12:0/629754128 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a0c1a0580 con 0x7f8a0c105360 2024-04-22T13:22:11.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.907+0000 7f8a11f32640 1 -- 172.21.15.12:0/629754128 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a0c0fdf30 con 0x7f8a0c105360 2024-04-22T13:22:11.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.908+0000 7f8a08ff9640 1 -- 172.21.15.12:0/629754128 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8a00004e80 con 0x7f8a0c105360 2024-04-22T13:22:11.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.908+0000 7f8a08ff9640 1 --2- 172.21.15.12:0/629754128 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f89f40a3bb0 0x7f89f40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:11.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.909+0000 7f8a0b7fe640 1 --2- 172.21.15.12:0/629754128 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f89f40a3bb0 0x7f89f40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:11.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.909+0000 7f8a08ff9640 1 -- 172.21.15.12:0/629754128 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f8a000cfcf0 con 0x7f8a0c105360 2024-04-22T13:22:11.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.909+0000 7f8a0b7fe640 1 --2- 172.21.15.12:0/629754128 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f89f40a3bb0 0x7f89f40a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f89fc009b50 tx=0x7f89fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:11.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:11.913+0000 7f8a08ff9640 1 -- 172.21.15.12:0/629754128 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8a0009c780 con 0x7f8a0c105360 2024-04-22T13:22:12.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.129+0000 7f8a11f32640 1 -- 172.21.15.12:0/629754128 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f8a0c062420 con 0x7f89f40a3bb0 2024-04-22T13:22:12.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.134+0000 7f8a08ff9640 1 -- 172.21.15.12:0/629754128 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f8a0c062420 con 0x7f89f40a3bb0 2024-04-22T13:22:12.132 INFO:teuthology.orchestra.run.smithi012.stdout:/volumes/qa/sv_0/0ae47f1a-0fc9-4838-bef2-3331165cace3 2024-04-22T13:22:12.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.136+0000 7f8a11f32640 1 -- 172.21.15.12:0/629754128 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f89f40a3bb0 msgr2=0x7f89f40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:12.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.136+0000 7f8a11f32640 1 --2- 172.21.15.12:0/629754128 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f89f40a3bb0 0x7f89f40a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f89fc009b50 tx=0x7f89fc005f90 comp rx=0 tx=0).stop 2024-04-22T13:22:12.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.136+0000 7f8a11f32640 1 -- 172.21.15.12:0/629754128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a0c105360 msgr2=0x7f8a0c1986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:12.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.136+0000 7f8a11f32640 1 --2- 172.21.15.12:0/629754128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a0c105360 0x7f8a0c1986b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000d740 tx=0x7f8a0000dc00 comp rx=0 tx=0).stop 2024-04-22T13:22:12.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.136+0000 7f8a11f32640 1 -- 172.21.15.12:0/629754128 shutdown_connections 2024-04-22T13:22:12.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.136+0000 7f8a11f32640 1 --2- 172.21.15.12:0/629754128 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a0c107c90 0x7f8a0c19f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:12.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.136+0000 7f8a11f32640 1 --2- 172.21.15.12:0/629754128 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f89f40a3bb0 0x7f89f40a6070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:12.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.136+0000 7f8a11f32640 1 --2- 172.21.15.12:0/629754128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8a0c105360 0x7f8a0c1986b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:12.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.136+0000 7f8a11f32640 1 --2- 172.21.15.12:0/629754128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a0c102a30 0x7f8a0c198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:12.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.136+0000 7f8a11f32640 1 -- 172.21.15.12:0/629754128 >> 172.21.15.12:0/629754128 conn(0x7f8a0c0fc440 msgr2=0x7f8a0c103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:12.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.136+0000 7f8a11f32640 1 -- 172.21.15.12:0/629754128 shutdown_connections 2024-04-22T13:22:12.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.136+0000 7f8a11f32640 1 -- 172.21.15.12:0/629754128 wait complete. 2024-04-22T13:22:12.145 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-22T13:22:12.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.255+0000 7fc593fa4640 1 -- 172.21.15.12:0/1700812983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc58c102a00 msgr2=0x7fc58c109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:12.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.255+0000 7fc593fa4640 1 --2- 172.21.15.12:0/1700812983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc58c102a00 0x7fc58c109320 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc584009ca0 tx=0x7fc58402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:22:12.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.255+0000 7fc593fa4640 1 -- 172.21.15.12:0/1700812983 shutdown_connections 2024-04-22T13:22:12.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.255+0000 7fc593fa4640 1 --2- 172.21.15.12:0/1700812983 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc58c102a00 0x7fc58c109320 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:12.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.255+0000 7fc593fa4640 1 --2- 172.21.15.12:0/1700812983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc58c102040 0x7fc58c1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:12.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.255+0000 7fc593fa4640 1 --2- 172.21.15.12:0/1700812983 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc58c100e60 0x7fc58c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:12.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.255+0000 7fc593fa4640 1 -- 172.21.15.12:0/1700812983 >> 172.21.15.12:0/1700812983 conn(0x7fc58c0fc440 msgr2=0x7fc58c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:12.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.255+0000 7fc593fa4640 1 -- 172.21.15.12:0/1700812983 shutdown_connections 2024-04-22T13:22:12.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.255+0000 7fc593fa4640 1 -- 172.21.15.12:0/1700812983 wait complete. 2024-04-22T13:22:12.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.255+0000 7fc593fa4640 1 Processor -- start 2024-04-22T13:22:12.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.255+0000 7fc593fa4640 1 -- start start 2024-04-22T13:22:12.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.256+0000 7fc593fa4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc58c100e60 0x7fc58c1970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:12.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.256+0000 7fc593fa4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc58c102040 0x7fc58c1975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:12.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.256+0000 7fc593fa4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc58c102a00 0x7fc58c19e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:12.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.256+0000 7fc593fa4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc58c10ca10 con 0x7fc58c102040 2024-04-22T13:22:12.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.256+0000 7fc593fa4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc58c10c890 con 0x7fc58c102a00 2024-04-22T13:22:12.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.256+0000 7fc593fa4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc58c10cb90 con 0x7fc58c100e60 2024-04-22T13:22:12.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.256+0000 7fc59153f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc58c102040 0x7fc58c1975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:12.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.256+0000 7fc59153f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc58c102040 0x7fc58c1975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39822/0 (socket says 172.21.15.12:39822) 2024-04-22T13:22:12.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.256+0000 7fc59153f640 1 -- 172.21.15.12:0/2826415280 learned_addr learned my addr 172.21.15.12:0/2826415280 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:12.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.256+0000 7fc592541640 1 --2- 172.21.15.12:0/2826415280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc58c102a00 0x7fc58c19e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:12.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.256+0000 7fc591d40640 1 --2- 172.21.15.12:0/2826415280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc58c100e60 0x7fc58c1970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:12.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.256+0000 7fc59153f640 1 -- 172.21.15.12:0/2826415280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc58c100e60 msgr2=0x7fc58c1970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:12.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.256+0000 7fc59153f640 1 --2- 172.21.15.12:0/2826415280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc58c100e60 0x7fc58c1970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:12.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.256+0000 7fc59153f640 1 -- 172.21.15.12:0/2826415280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc58c102a00 msgr2=0x7fc58c19e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:12.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.256+0000 7fc59153f640 1 --2- 172.21.15.12:0/2826415280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc58c102a00 0x7fc58c19e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:12.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.256+0000 7fc59153f640 1 -- 172.21.15.12:0/2826415280 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58c19ecd0 con 0x7fc58c102040 2024-04-22T13:22:12.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.256+0000 7fc59153f640 1 --2- 172.21.15.12:0/2826415280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc58c102040 0x7fc58c1975f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00ea60 tx=0x7fc57c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:12.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.256+0000 7fc582ffd640 1 -- 172.21.15.12:0/2826415280 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c00cde0 con 0x7fc58c102040 2024-04-22T13:22:12.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.256+0000 7fc582ffd640 1 -- 172.21.15.12:0/2826415280 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc57c0035f0 con 0x7fc58c102040 2024-04-22T13:22:12.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.256+0000 7fc593fa4640 1 -- 172.21.15.12:0/2826415280 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc58c19efc0 con 0x7fc58c102040 2024-04-22T13:22:12.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.256+0000 7fc582ffd640 1 -- 172.21.15.12:0/2826415280 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c010690 con 0x7fc58c102040 2024-04-22T13:22:12.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.256+0000 7fc593fa4640 1 -- 172.21.15.12:0/2826415280 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc58c19f480 con 0x7fc58c102040 2024-04-22T13:22:12.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.257+0000 7fc593fa4640 1 -- 172.21.15.12:0/2826415280 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc58c108200 con 0x7fc58c102040 2024-04-22T13:22:12.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.259+0000 7fc582ffd640 1 -- 172.21.15.12:0/2826415280 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc57c0037a0 con 0x7fc58c102040 2024-04-22T13:22:12.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.259+0000 7fc582ffd640 1 --2- 172.21.15.12:0/2826415280 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc5780a3bb0 0x7fc5780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:12.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.259+0000 7fc591d40640 1 --2- 172.21.15.12:0/2826415280 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc5780a3bb0 0x7fc5780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:12.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.259+0000 7fc582ffd640 1 -- 172.21.15.12:0/2826415280 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fc57c0d0600 con 0x7fc58c102040 2024-04-22T13:22:12.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.260+0000 7fc591d40640 1 --2- 172.21.15.12:0/2826415280 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc5780a3bb0 0x7fc5780a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc5880058c0 tx=0x7fc58800a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:12.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.264+0000 7fc582ffd640 1 -- 172.21.15.12:0/2826415280 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc57c09d130 con 0x7fc58c102040 2024-04-22T13:22:12.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.485+0000 7fc593fa4640 1 -- 172.21.15.12:0/2826415280 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc58c062390 con 0x7fc5780a3bb0 2024-04-22T13:22:12.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.509+0000 7fc582ffd640 1 -- 172.21.15.12:0/2826415280 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc58c062390 con 0x7fc5780a3bb0 2024-04-22T13:22:12.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.511+0000 7fc593fa4640 1 -- 172.21.15.12:0/2826415280 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc5780a3bb0 msgr2=0x7fc5780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:12.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.511+0000 7fc593fa4640 1 --2- 172.21.15.12:0/2826415280 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc5780a3bb0 0x7fc5780a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc5880058c0 tx=0x7fc58800a4c0 comp rx=0 tx=0).stop 2024-04-22T13:22:12.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.511+0000 7fc593fa4640 1 -- 172.21.15.12:0/2826415280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc58c102040 msgr2=0x7fc58c1975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:12.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.511+0000 7fc593fa4640 1 --2- 172.21.15.12:0/2826415280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc58c102040 0x7fc58c1975f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00ea60 tx=0x7fc57c00ef20 comp rx=0 tx=0).stop 2024-04-22T13:22:12.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.511+0000 7fc593fa4640 1 -- 172.21.15.12:0/2826415280 shutdown_connections 2024-04-22T13:22:12.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.511+0000 7fc593fa4640 1 --2- 172.21.15.12:0/2826415280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc58c102a00 0x7fc58c19e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:12.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.511+0000 7fc593fa4640 1 --2- 172.21.15.12:0/2826415280 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc5780a3bb0 0x7fc5780a6070 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:12.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.511+0000 7fc593fa4640 1 --2- 172.21.15.12:0/2826415280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc58c102040 0x7fc58c1975f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:12.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.511+0000 7fc593fa4640 1 --2- 172.21.15.12:0/2826415280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc58c100e60 0x7fc58c1970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:12.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.511+0000 7fc593fa4640 1 -- 172.21.15.12:0/2826415280 >> 172.21.15.12:0/2826415280 conn(0x7fc58c0fc440 msgr2=0x7fc58c1053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:12.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.511+0000 7fc593fa4640 1 -- 172.21.15.12:0/2826415280 shutdown_connections 2024-04-22T13:22:12.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.511+0000 7fc593fa4640 1 -- 172.21.15.12:0/2826415280 wait complete. 2024-04-22T13:22:12.519 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:12 smithi012 ceph-mon[28265]: mds.? [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] up:active 2024-04-22T13:22:12.519 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:12 smithi012 ceph-mon[28265]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-22T13:22:12.519 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:12 smithi012 ceph-mon[28265]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-22T13:22:12.519 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:12 smithi012 ceph-mon[28265]: Cluster is now healthy 2024-04-22T13:22:12.519 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:12 smithi012 ceph-mon[28265]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:active} 2 up:standby-replay 6 up:standby 2024-04-22T13:22:12.519 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:12 smithi012 ceph-mon[28265]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2024-04-22T13:22:12.519 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:12 smithi012 ceph-mon[28265]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-04-22T13:22:12.519 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:12 smithi012 ceph-mon[28265]: from='client.24584 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:22:12.519 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:12 smithi012 ceph-mon[28265]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:22:12.520 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-22T13:22:12.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.629+0000 7f95ec655640 1 -- 172.21.15.12:0/469235596 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95e4101830 msgr2=0x7f95e4101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:12.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.629+0000 7f95ec655640 1 --2- 172.21.15.12:0/469235596 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95e4101830 0x7f95e4101cb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009900 tx=0x7f95d402bc30 comp rx=0 tx=0).stop 2024-04-22T13:22:12.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.630+0000 7f95ec655640 1 -- 172.21.15.12:0/469235596 shutdown_connections 2024-04-22T13:22:12.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.630+0000 7f95ec655640 1 --2- 172.21.15.12:0/469235596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f95e41021f0 0x7f95e4108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:12.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.630+0000 7f95ec655640 1 --2- 172.21.15.12:0/469235596 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95e4101830 0x7f95e4101cb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:12.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.630+0000 7f95ec655640 1 --2- 172.21.15.12:0/469235596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95e4100640 0x7f95e4100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:12.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.630+0000 7f95ec655640 1 -- 172.21.15.12:0/469235596 >> 172.21.15.12:0/469235596 conn(0x7f95e40fbbb0 msgr2=0x7f95e40fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:12.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.630+0000 7f95ec655640 1 -- 172.21.15.12:0/469235596 shutdown_connections 2024-04-22T13:22:12.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.630+0000 7f95ec655640 1 -- 172.21.15.12:0/469235596 wait complete. 2024-04-22T13:22:12.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.630+0000 7f95ec655640 1 Processor -- start 2024-04-22T13:22:12.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.630+0000 7f95ec655640 1 -- start start 2024-04-22T13:22:12.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.630+0000 7f95ec655640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95e4100640 0x7f95e410bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:12.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.630+0000 7f95ec655640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95e4101830 0x7f95e410a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:12.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.630+0000 7f95ec655640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f95e41021f0 0x7f95e410a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:12.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.630+0000 7f95ec655640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f95e410dbb0 con 0x7f95e4100640 2024-04-22T13:22:12.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.630+0000 7f95ec655640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f95e410da30 con 0x7f95e41021f0 2024-04-22T13:22:12.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.630+0000 7f95ec655640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f95e410dd30 con 0x7f95e4101830 2024-04-22T13:22:12.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.630+0000 7f95ea3f1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95e4100640 0x7f95e410bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:12.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.630+0000 7f95ea3f1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95e4100640 0x7f95e410bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39842/0 (socket says 172.21.15.12:39842) 2024-04-22T13:22:12.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.631+0000 7f95ea3f1640 1 -- 172.21.15.12:0/311181610 learned_addr learned my addr 172.21.15.12:0/311181610 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:12.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.631+0000 7f95eabf2640 1 --2- 172.21.15.12:0/311181610 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f95e41021f0 0x7f95e410a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:12.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.631+0000 7f95e9bf0640 1 --2- 172.21.15.12:0/311181610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95e4101830 0x7f95e410a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:12.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.631+0000 7f95ea3f1640 1 -- 172.21.15.12:0/311181610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95e4101830 msgr2=0x7f95e410a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:12.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.631+0000 7f95ea3f1640 1 --2- 172.21.15.12:0/311181610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95e4101830 0x7f95e410a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:12.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.631+0000 7f95ea3f1640 1 -- 172.21.15.12:0/311181610 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f95e41021f0 msgr2=0x7f95e410a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:12.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.631+0000 7f95ea3f1640 1 --2- 172.21.15.12:0/311181610 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f95e41021f0 0x7f95e410a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:12.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.631+0000 7f95ea3f1640 1 -- 172.21.15.12:0/311181610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e410b060 con 0x7f95e4100640 2024-04-22T13:22:12.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.631+0000 7f95e9bf0640 1 --2- 172.21.15.12:0/311181610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95e4101830 0x7f95e410a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:22:12.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.631+0000 7f95ea3f1640 1 --2- 172.21.15.12:0/311181610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95e4100640 0x7f95e410bc50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00b450 tx=0x7f95dc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:12.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.631+0000 7f95db7fe640 1 -- 172.21.15.12:0/311181610 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc015020 con 0x7f95e4100640 2024-04-22T13:22:12.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.631+0000 7f95db7fe640 1 -- 172.21.15.12:0/311181610 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95dc011d40 con 0x7f95e4100640 2024-04-22T13:22:12.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.631+0000 7f95ec655640 1 -- 172.21.15.12:0/311181610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95e41a2380 con 0x7f95e4100640 2024-04-22T13:22:12.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.631+0000 7f95db7fe640 1 -- 172.21.15.12:0/311181610 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc00f340 con 0x7f95e4100640 2024-04-22T13:22:12.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.631+0000 7f95ec655640 1 -- 172.21.15.12:0/311181610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95e41a2850 con 0x7f95e4100640 2024-04-22T13:22:12.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.632+0000 7f95ec655640 1 -- 172.21.15.12:0/311181610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95e4107960 con 0x7f95e4100640 2024-04-22T13:22:12.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.633+0000 7f95db7fe640 1 -- 172.21.15.12:0/311181610 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f95dc018020 con 0x7f95e4100640 2024-04-22T13:22:12.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.634+0000 7f95db7fe640 1 --2- 172.21.15.12:0/311181610 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f95d00a3bb0 0x7f95d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:12.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.634+0000 7f95e9bf0640 1 --2- 172.21.15.12:0/311181610 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f95d00a3bb0 0x7f95d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:12.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.634+0000 7f95db7fe640 1 -- 172.21.15.12:0/311181610 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f95dc0cfe10 con 0x7f95e4100640 2024-04-22T13:22:12.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.634+0000 7f95e9bf0640 1 --2- 172.21.15.12:0/311181610 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f95d00a3bb0 0x7f95d00a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f95d4006010 tx=0x7f95d402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:12.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.638+0000 7f95db7fe640 1 -- 172.21.15.12:0/311181610 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f95dc01f020 con 0x7f95e4100640 2024-04-22T13:22:12.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:12 smithi134 ceph-mon[31338]: mds.? [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] up:active 2024-04-22T13:22:12.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:12 smithi134 ceph-mon[31338]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-22T13:22:12.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:12 smithi134 ceph-mon[31338]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-22T13:22:12.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:12 smithi134 ceph-mon[31338]: Cluster is now healthy 2024-04-22T13:22:12.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:12 smithi134 ceph-mon[31338]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:active} 2 up:standby-replay 6 up:standby 2024-04-22T13:22:12.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:12 smithi134 ceph-mon[31338]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2024-04-22T13:22:12.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:12 smithi134 ceph-mon[31338]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-04-22T13:22:12.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:12 smithi134 ceph-mon[31338]: from='client.24584 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:22:12.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:12 smithi134 ceph-mon[31338]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:22:12.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:12 smithi111 ceph-mon[31361]: mds.? [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] up:active 2024-04-22T13:22:12.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:12 smithi111 ceph-mon[31361]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-22T13:22:12.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:12 smithi111 ceph-mon[31361]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-22T13:22:12.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:12 smithi111 ceph-mon[31361]: Cluster is now healthy 2024-04-22T13:22:12.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:12 smithi111 ceph-mon[31361]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=f=up:active,3=e=up:active} 2 up:standby-replay 6 up:standby 2024-04-22T13:22:12.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:12 smithi111 ceph-mon[31361]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2024-04-22T13:22:12.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:12 smithi111 ceph-mon[31361]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-04-22T13:22:12.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:12 smithi111 ceph-mon[31361]: from='client.24584 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:22:12.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:12 smithi111 ceph-mon[31361]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:22:12.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.858+0000 7f95ec655640 1 -- 172.21.15.12:0/311181610 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f95e4062420 con 0x7f95d00a3bb0 2024-04-22T13:22:12.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.862+0000 7f95db7fe640 1 -- 172.21.15.12:0/311181610 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f95e4062420 con 0x7f95d00a3bb0 2024-04-22T13:22:12.860 INFO:teuthology.orchestra.run.smithi012.stdout:/volumes/qa/sv_1/cd6a71e5-f19a-4226-8ca1-746deb442c27 2024-04-22T13:22:12.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.864+0000 7f95ec655640 1 -- 172.21.15.12:0/311181610 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f95d00a3bb0 msgr2=0x7f95d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:12.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.864+0000 7f95ec655640 1 --2- 172.21.15.12:0/311181610 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f95d00a3bb0 0x7f95d00a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f95d4006010 tx=0x7f95d402c7e0 comp rx=0 tx=0).stop 2024-04-22T13:22:12.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.864+0000 7f95ec655640 1 -- 172.21.15.12:0/311181610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95e4100640 msgr2=0x7f95e410bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:12.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.864+0000 7f95ec655640 1 --2- 172.21.15.12:0/311181610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95e4100640 0x7f95e410bc50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00b450 tx=0x7f95dc00b910 comp rx=0 tx=0).stop 2024-04-22T13:22:12.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.864+0000 7f95ec655640 1 -- 172.21.15.12:0/311181610 shutdown_connections 2024-04-22T13:22:12.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.865+0000 7f95ec655640 1 --2- 172.21.15.12:0/311181610 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f95e41021f0 0x7f95e410a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:12.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.865+0000 7f95ec655640 1 --2- 172.21.15.12:0/311181610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95e4101830 0x7f95e410a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:12.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.865+0000 7f95ec655640 1 --2- 172.21.15.12:0/311181610 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f95d00a3bb0 0x7f95d00a6070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:12.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.865+0000 7f95ec655640 1 --2- 172.21.15.12:0/311181610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95e4100640 0x7f95e410bc50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:12.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.865+0000 7f95ec655640 1 -- 172.21.15.12:0/311181610 >> 172.21.15.12:0/311181610 conn(0x7f95e40fbbb0 msgr2=0x7f95e4104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:12.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.865+0000 7f95ec655640 1 -- 172.21.15.12:0/311181610 shutdown_connections 2024-04-22T13:22:12.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.865+0000 7f95ec655640 1 -- 172.21.15.12:0/311181610 wait complete. 2024-04-22T13:22:12.873 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:22:12.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.977+0000 7fdd4d332640 1 -- 172.21.15.12:0/3939348504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd481029f0 msgr2=0x7fdd48104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:12.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.977+0000 7fdd4d332640 1 --2- 172.21.15.12:0/3939348504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd481029f0 0x7fdd48104de0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdd38009ca0 tx=0x7fdd3802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:22:12.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.977+0000 7fdd4d332640 1 -- 172.21.15.12:0/3939348504 shutdown_connections 2024-04-22T13:22:12.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.977+0000 7fdd4d332640 1 --2- 172.21.15.12:0/3939348504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd48107c50 0x7fdd4810a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:12.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.977+0000 7fdd4d332640 1 --2- 172.21.15.12:0/3939348504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd48105320 0x7fdd48107710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:12.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.977+0000 7fdd4d332640 1 --2- 172.21.15.12:0/3939348504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd481029f0 0x7fdd48104de0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:12.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.977+0000 7fdd4d332640 1 -- 172.21.15.12:0/3939348504 >> 172.21.15.12:0/3939348504 conn(0x7fdd480fc3e0 msgr2=0x7fdd480fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:12.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.977+0000 7fdd4d332640 1 -- 172.21.15.12:0/3939348504 shutdown_connections 2024-04-22T13:22:12.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.977+0000 7fdd4d332640 1 -- 172.21.15.12:0/3939348504 wait complete. 2024-04-22T13:22:12.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.978+0000 7fdd4d332640 1 Processor -- start 2024-04-22T13:22:12.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.978+0000 7fdd4d332640 1 -- start start 2024-04-22T13:22:12.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.978+0000 7fdd4d332640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd481029f0 0x7fdd48102360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:12.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.978+0000 7fdd4d332640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd48105320 0x7fdd481009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:12.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.978+0000 7fdd4d332640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd48107c50 0x7fdd48100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:12.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.978+0000 7fdd4d332640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4810dac0 con 0x7fdd481029f0 2024-04-22T13:22:12.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.978+0000 7fdd4d332640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4810d940 con 0x7fdd48107c50 2024-04-22T13:22:12.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.978+0000 7fdd4d332640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4810dc40 con 0x7fdd48105320 2024-04-22T13:22:12.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.978+0000 7fdd46ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd481029f0 0x7fdd48102360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:12.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.978+0000 7fdd46ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd481029f0 0x7fdd48102360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39858/0 (socket says 172.21.15.12:39858) 2024-04-22T13:22:12.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.978+0000 7fdd46ffd640 1 -- 172.21.15.12:0/1151434084 learned_addr learned my addr 172.21.15.12:0/1151434084 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:12.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.978+0000 7fdd477fe640 1 --2- 172.21.15.12:0/1151434084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd48107c50 0x7fdd48100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:12.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.979+0000 7fdd46ffd640 1 -- 172.21.15.12:0/1151434084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd48105320 msgr2=0x7fdd481009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:12.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.979+0000 7fdd46ffd640 1 --2- 172.21.15.12:0/1151434084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd48105320 0x7fdd481009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:12.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.979+0000 7fdd46ffd640 1 -- 172.21.15.12:0/1151434084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd48107c50 msgr2=0x7fdd48100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:12.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.979+0000 7fdd46ffd640 1 --2- 172.21.15.12:0/1151434084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd48107c50 0x7fdd48100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:12.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.979+0000 7fdd46ffd640 1 -- 172.21.15.12:0/1151434084 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd48101770 con 0x7fdd481029f0 2024-04-22T13:22:12.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.979+0000 7fdd477fe640 1 --2- 172.21.15.12:0/1151434084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd48107c50 0x7fdd48100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:22:12.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.979+0000 7fdd46ffd640 1 --2- 172.21.15.12:0/1151434084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd481029f0 0x7fdd48102360 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdd38005e30 tx=0x7fdd38034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:12.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.979+0000 7fdd2ffff640 1 -- 172.21.15.12:0/1151434084 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd38039040 con 0x7fdd481029f0 2024-04-22T13:22:12.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.979+0000 7fdd2ffff640 1 -- 172.21.15.12:0/1151434084 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd380325c0 con 0x7fdd481029f0 2024-04-22T13:22:12.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.979+0000 7fdd4d332640 1 -- 172.21.15.12:0/1151434084 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd481a2280 con 0x7fdd481029f0 2024-04-22T13:22:12.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.979+0000 7fdd2ffff640 1 -- 172.21.15.12:0/1151434084 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd380042a0 con 0x7fdd481029f0 2024-04-22T13:22:12.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.979+0000 7fdd4d332640 1 -- 172.21.15.12:0/1151434084 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd481a27a0 con 0x7fdd481029f0 2024-04-22T13:22:12.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.980+0000 7fdd4d332640 1 -- 172.21.15.12:0/1151434084 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd480fded0 con 0x7fdd481029f0 2024-04-22T13:22:12.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.981+0000 7fdd2ffff640 1 -- 172.21.15.12:0/1151434084 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdd38046030 con 0x7fdd481029f0 2024-04-22T13:22:12.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.982+0000 7fdd2ffff640 1 --2- 172.21.15.12:0/1151434084 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fdd300a3bb0 0x7fdd300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:12.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.982+0000 7fdd467fc640 1 --2- 172.21.15.12:0/1151434084 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fdd300a3bb0 0x7fdd300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:12.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.982+0000 7fdd2ffff640 1 -- 172.21.15.12:0/1151434084 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fdd380f2240 con 0x7fdd481029f0 2024-04-22T13:22:12.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.982+0000 7fdd467fc640 1 --2- 172.21.15.12:0/1151434084 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fdd300a3bb0 0x7fdd300a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0058c0 tx=0x7fdd3c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:12.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:12.986+0000 7fdd2ffff640 1 -- 172.21.15.12:0/1151434084 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd3804d060 con 0x7fdd481029f0 2024-04-22T13:22:13.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.212+0000 7fdd4d332640 1 -- 172.21.15.12:0/1151434084 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd4802d5c0 con 0x7fdd481029f0 2024-04-22T13:22:13.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.213+0000 7fdd2ffff640 1 -- 172.21.15.12:0/1151434084 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11942 (secure 0 0 0) 0x7fdd380bee00 con 0x7fdd481029f0 2024-04-22T13:22:13.210 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:22:13.210 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:22:12.398056+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24448,24509,24443,24484,24407]},"id":1}]} 2024-04-22T13:22:13.211 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 27 2024-04-22T13:22:13.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.214+0000 7fdd4d332640 1 -- 172.21.15.12:0/1151434084 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fdd300a3bb0 msgr2=0x7fdd300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:13.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.214+0000 7fdd4d332640 1 --2- 172.21.15.12:0/1151434084 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fdd300a3bb0 0x7fdd300a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0058c0 tx=0x7fdd3c005850 comp rx=0 tx=0).stop 2024-04-22T13:22:13.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.214+0000 7fdd4d332640 1 -- 172.21.15.12:0/1151434084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd481029f0 msgr2=0x7fdd48102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:13.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.214+0000 7fdd4d332640 1 --2- 172.21.15.12:0/1151434084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd481029f0 0x7fdd48102360 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdd38005e30 tx=0x7fdd38034760 comp rx=0 tx=0).stop 2024-04-22T13:22:13.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.215+0000 7fdd4d332640 1 -- 172.21.15.12:0/1151434084 shutdown_connections 2024-04-22T13:22:13.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.215+0000 7fdd4d332640 1 --2- 172.21.15.12:0/1151434084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdd48107c50 0x7fdd48100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:13.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.215+0000 7fdd4d332640 1 --2- 172.21.15.12:0/1151434084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd48105320 0x7fdd481009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:13.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.215+0000 7fdd4d332640 1 --2- 172.21.15.12:0/1151434084 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fdd300a3bb0 0x7fdd300a6070 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:13.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.215+0000 7fdd4d332640 1 --2- 172.21.15.12:0/1151434084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdd481029f0 0x7fdd48102360 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:13.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.215+0000 7fdd4d332640 1 -- 172.21.15.12:0/1151434084 >> 172.21.15.12:0/1151434084 conn(0x7fdd480fc3e0 msgr2=0x7fdd481034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:13.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.215+0000 7fdd4d332640 1 -- 172.21.15.12:0/1151434084 shutdown_connections 2024-04-22T13:22:13.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.215+0000 7fdd4d332640 1 -- 172.21.15.12:0/1151434084 wait complete. 2024-04-22T13:22:13.224 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:22:13.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.331+0000 7efde7bbd640 1 -- 172.21.15.12:0/612223413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efde0105390 msgr2=0x7efde0107780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:13.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.331+0000 7efde7bbd640 1 --2- 172.21.15.12:0/612223413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efde0105390 0x7efde0107780 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efdd0009900 tx=0x7efdd002bc30 comp rx=0 tx=0).stop 2024-04-22T13:22:13.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.331+0000 7efde7bbd640 1 -- 172.21.15.12:0/612223413 shutdown_connections 2024-04-22T13:22:13.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.331+0000 7efde7bbd640 1 --2- 172.21.15.12:0/612223413 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efde0107cc0 0x7efde010a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:13.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.331+0000 7efde7bbd640 1 --2- 172.21.15.12:0/612223413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efde0105390 0x7efde0107780 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:13.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.331+0000 7efde7bbd640 1 --2- 172.21.15.12:0/612223413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efde0102a60 0x7efde0104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:13.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.331+0000 7efde7bbd640 1 -- 172.21.15.12:0/612223413 >> 172.21.15.12:0/612223413 conn(0x7efde00fc410 msgr2=0x7efde00fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:13.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.331+0000 7efde7bbd640 1 -- 172.21.15.12:0/612223413 shutdown_connections 2024-04-22T13:22:13.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.331+0000 7efde7bbd640 1 -- 172.21.15.12:0/612223413 wait complete. 2024-04-22T13:22:13.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.331+0000 7efde7bbd640 1 Processor -- start 2024-04-22T13:22:13.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.331+0000 7efde7bbd640 1 -- start start 2024-04-22T13:22:13.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.332+0000 7efde7bbd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efde0102a60 0x7efde0198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:13.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.332+0000 7efde7bbd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efde0105390 0x7efde01987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:13.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.332+0000 7efde7bbd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efde0107cc0 0x7efde019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:13.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.332+0000 7efde7bbd640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efde010dab0 con 0x7efde0102a60 2024-04-22T13:22:13.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.332+0000 7efde7bbd640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efde010d930 con 0x7efde0107cc0 2024-04-22T13:22:13.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.332+0000 7efde7bbd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efde010dc30 con 0x7efde0105390 2024-04-22T13:22:13.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.332+0000 7efde5959640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efde0102a60 0x7efde0198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:13.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.332+0000 7efde5959640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efde0102a60 0x7efde0198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39878/0 (socket says 172.21.15.12:39878) 2024-04-22T13:22:13.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.332+0000 7efde5959640 1 -- 172.21.15.12:0/4178940128 learned_addr learned my addr 172.21.15.12:0/4178940128 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:13.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.332+0000 7efde615a640 1 --2- 172.21.15.12:0/4178940128 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efde0107cc0 0x7efde019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:13.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.332+0000 7efde5158640 1 --2- 172.21.15.12:0/4178940128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efde0105390 0x7efde01987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:13.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.332+0000 7efde5959640 1 -- 172.21.15.12:0/4178940128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efde0105390 msgr2=0x7efde01987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:13.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.332+0000 7efde5959640 1 --2- 172.21.15.12:0/4178940128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efde0105390 0x7efde01987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:13.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.332+0000 7efde5959640 1 -- 172.21.15.12:0/4178940128 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efde0107cc0 msgr2=0x7efde019f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:13.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.332+0000 7efde5959640 1 --2- 172.21.15.12:0/4178940128 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efde0107cc0 0x7efde019f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:13.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.332+0000 7efde5959640 1 -- 172.21.15.12:0/4178940128 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde019fe80 con 0x7efde0102a60 2024-04-22T13:22:13.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.332+0000 7efde5158640 1 --2- 172.21.15.12:0/4178940128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efde0105390 0x7efde01987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:22:13.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.332+0000 7efde5959640 1 --2- 172.21.15.12:0/4178940128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efde0102a60 0x7efde0198260 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efddc00d770 tx=0x7efddc00db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:13.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.332+0000 7efdd6ffd640 1 -- 172.21.15.12:0/4178940128 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc017070 con 0x7efde0102a60 2024-04-22T13:22:13.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.332+0000 7efdd6ffd640 1 -- 172.21.15.12:0/4178940128 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efddc009820 con 0x7efde0102a60 2024-04-22T13:22:13.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.332+0000 7efde7bbd640 1 -- 172.21.15.12:0/4178940128 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efde01a0170 con 0x7efde0102a60 2024-04-22T13:22:13.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.332+0000 7efdd6ffd640 1 -- 172.21.15.12:0/4178940128 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc004bc0 con 0x7efde0102a60 2024-04-22T13:22:13.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.332+0000 7efde7bbd640 1 -- 172.21.15.12:0/4178940128 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efde01a0640 con 0x7efde0102a60 2024-04-22T13:22:13.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.333+0000 7efde7bbd640 1 -- 172.21.15.12:0/4178940128 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efde00fdf00 con 0x7efde0102a60 2024-04-22T13:22:13.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.334+0000 7efdd6ffd640 1 -- 172.21.15.12:0/4178940128 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efddc0099d0 con 0x7efde0102a60 2024-04-22T13:22:13.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.335+0000 7efdd6ffd640 1 --2- 172.21.15.12:0/4178940128 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efdcc0a3bb0 0x7efdcc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:13.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.335+0000 7efdd6ffd640 1 -- 172.21.15.12:0/4178940128 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7efddc0cfd70 con 0x7efde0102a60 2024-04-22T13:22:13.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.335+0000 7efde5158640 1 --2- 172.21.15.12:0/4178940128 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efdcc0a3bb0 0x7efdcc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:13.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.335+0000 7efde5158640 1 --2- 172.21.15.12:0/4178940128 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efdcc0a3bb0 0x7efdcc0a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efdd0006010 tx=0x7efdd002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:13.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.340+0000 7efdd6ffd640 1 -- 172.21.15.12:0/4178940128 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efddc09c8a0 con 0x7efde0102a60 2024-04-22T13:22:13.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.413+0000 7efdd6ffd640 1 -- 172.21.15.12:0/4178940128 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7efddc009cf0 con 0x7efde0102a60 2024-04-22T13:22:13.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.533+0000 7efde7bbd640 1 -- 172.21.15.12:0/4178940128 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efde0062b50 con 0x7efde0102a60 2024-04-22T13:22:13.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.534+0000 7efdd6ffd640 1 -- 172.21.15.12:0/4178940128 <== mon.0 v2:172.21.15.12:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22233 (secure 0 0 0) 0x7efddc0609e0 con 0x7efde0102a60 2024-04-22T13:22:13.532 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:22:13.532 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":87,"fsid":"0aede5ea-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:14:09.362918+0000","modified":"2024-04-22T13:22:06.227428+0000","last_up_change":"2024-04-22T13:21:15.263393+0000","last_in_change":"2024-04-22T13:20:54.174685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:17:32.372434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:22:02.768799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:22:03.578730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d9e2365-3a64-4b13-96fc-c217b1cee2ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6803","nonce":1191350632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6805","nonce":1191350632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6809","nonce":1191350632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6807","nonce":1191350632}]},"public_addr":"172.21.15.12:6803/1191350632","cluster_addr":"172.21.15.12:6805/1191350632","heartbeat_back_addr":"172.21.15.12:6809/1191350632","heartbeat_front_addr":"172.21.15.12:6807/1191350632","state":["exists","up"]},{"osd":1,"uuid":"1499c783-2bd7-48ce-a444-34add42bcf31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6801","nonce":417496465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6803","nonce":417496465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6807","nonce":417496465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6805","nonce":417496465}]},"public_addr":"172.21.15.111:6801/417496465","cluster_addr":"172.21.15.111:6803/417496465","heartbeat_back_addr":"172.21.15.111:6807/417496465","heartbeat_front_addr":"172.21.15.111:6805/417496465","state":["exists","up"]},{"osd":2,"uuid":"71a9ebc3-c622-434e-a6c2-f48358f384f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6801","nonce":1403153998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6803","nonce":1403153998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6807","nonce":1403153998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6805","nonce":1403153998}]},"public_addr":"172.21.15.134:6801/1403153998","cluster_addr":"172.21.15.134:6803/1403153998","heartbeat_back_addr":"172.21.15.134:6807/1403153998","heartbeat_front_addr":"172.21.15.134:6805/1403153998","state":["exists","up"]},{"osd":3,"uuid":"6dd00db2-a286-48ba-8666-56ffdd5f35ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6811","nonce":3359066681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6813","nonce":3359066681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6817","nonce":3359066681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6815","nonce":3359066681}]},"public_addr":"172.21.15.12:6811/3359066681","cluster_addr":"172.21.15.12:6813/3359066681","heartbeat_back_addr":"172.21.15.12:6817/3359066681","heartbeat_front_addr":"172.21.15.12:6815/3359066681","state":["exists","up"]},{"osd":4,"uuid":"1e29afd7-354c-4506-aaa0-b43eb1a19a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6809","nonce":2886349872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6811","nonce":2886349872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6815","nonce":2886349872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6813","nonce":2886349872}]},"public_addr":"172.21.15.111:6809/2886349872","cluster_addr":"172.21.15.111:6811/2886349872","heartbeat_back_addr":"172.21.15.111:6815/2886349872","heartbeat_front_addr":"172.21.15.111:6813/2886349872","state":["exists","up"]},{"osd":5,"uuid":"8f40f02d-0ba2-4842-a03d-2a1a04d1010d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6809","nonce":3071977602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6811","nonce":3071977602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6815","nonce":3071977602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6813","nonce":3071977602}]},"public_addr":"172.21.15.134:6809/3071977602","cluster_addr":"172.21.15.134:6811/3071977602","heartbeat_back_addr":"172.21.15.134:6815/3071977602","heartbeat_front_addr":"172.21.15.134:6813/3071977602","state":["exists","up"]},{"osd":6,"uuid":"bb0c17a9-6a8e-4be6-8660-f73cfa8200ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6819","nonce":3819619299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6821","nonce":3819619299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6825","nonce":3819619299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6823","nonce":3819619299}]},"public_addr":"172.21.15.12:6819/3819619299","cluster_addr":"172.21.15.12:6821/3819619299","heartbeat_back_addr":"172.21.15.12:6825/3819619299","heartbeat_front_addr":"172.21.15.12:6823/3819619299","state":["exists","up"]},{"osd":7,"uuid":"0a93d6af-569e-4e45-b697-9e978c1c1004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6817","nonce":3113856282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6819","nonce":3113856282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6823","nonce":3113856282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6821","nonce":3113856282}]},"public_addr":"172.21.15.111:6817/3113856282","cluster_addr":"172.21.15.111:6819/3113856282","heartbeat_back_addr":"172.21.15.111:6823/3113856282","heartbeat_front_addr":"172.21.15.111:6821/3113856282","state":["exists","up"]},{"osd":8,"uuid":"e21acc68-cf50-4f4f-b1ba-cf6c195b4955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6817","nonce":1689362191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6819","nonce":1689362191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6823","nonce":1689362191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6821","nonce":1689362191}]},"public_addr":"172.21.15.134:6817/1689362191","cluster_addr":"172.21.15.134:6819/1689362191","heartbeat_back_addr":"172.21.15.134:6823/1689362191","heartbeat_front_addr":"172.21.15.134:6821/1689362191","state":["exists","up"]},{"osd":9,"uuid":"fd745869-7f5f-48a8-b412-bb9c92b586e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6827","nonce":4292385779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6829","nonce":4292385779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6833","nonce":4292385779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6831","nonce":4292385779}]},"public_addr":"172.21.15.12:6827/4292385779","cluster_addr":"172.21.15.12:6829/4292385779","heartbeat_back_addr":"172.21.15.12:6833/4292385779","heartbeat_front_addr":"172.21.15.12:6831/4292385779","state":["exists","up"]},{"osd":10,"uuid":"798b1abd-f6c7-4a5f-8da0-fdae49e50d2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6825","nonce":3338566976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6827","nonce":3338566976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6831","nonce":3338566976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6829","nonce":3338566976}]},"public_addr":"172.21.15.111:6825/3338566976","cluster_addr":"172.21.15.111:6827/3338566976","heartbeat_back_addr":"172.21.15.111:6831/3338566976","heartbeat_front_addr":"172.21.15.111:6829/3338566976","state":["exists","up"]},{"osd":11,"uuid":"b18fc5f3-1049-4871-b060-e99290273cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6825","nonce":578640428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6827","nonce":578640428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6831","nonce":578640428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6829","nonce":578640428}]},"public_addr":"172.21.15.134:6825/578640428","cluster_addr":"172.21.15.134:6827/578640428","heartbeat_back_addr":"172.21.15.134:6831/578640428","heartbeat_front_addr":"172.21.15.134:6829/578640428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:16:40.828608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:05.052528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:28.297316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:53.496497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:17.947259+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:41.712894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:07.105945+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:32.129688+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:56.008516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:22.239867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:47.334692+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:12.301276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/2593601362":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/2519382069":"2024-04-23T13:15:04.279990+0000","172.21.15.12:6801/2009831940":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3509412187":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1007749062":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3588480095":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1904962176":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2009831940":"2024-04-23T13:15:04.279990+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:22:13.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.536+0000 7efde7bbd640 1 -- 172.21.15.12:0/4178940128 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efdcc0a3bb0 msgr2=0x7efdcc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:13.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.536+0000 7efde7bbd640 1 --2- 172.21.15.12:0/4178940128 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efdcc0a3bb0 0x7efdcc0a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efdd0006010 tx=0x7efdd002c7e0 comp rx=0 tx=0).stop 2024-04-22T13:22:13.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.536+0000 7efde7bbd640 1 -- 172.21.15.12:0/4178940128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efde0102a60 msgr2=0x7efde0198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:13.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.536+0000 7efde7bbd640 1 --2- 172.21.15.12:0/4178940128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efde0102a60 0x7efde0198260 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efddc00d770 tx=0x7efddc00db70 comp rx=0 tx=0).stop 2024-04-22T13:22:13.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.536+0000 7efde7bbd640 1 -- 172.21.15.12:0/4178940128 shutdown_connections 2024-04-22T13:22:13.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.536+0000 7efde7bbd640 1 --2- 172.21.15.12:0/4178940128 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efde0107cc0 0x7efde019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:13.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.536+0000 7efde7bbd640 1 --2- 172.21.15.12:0/4178940128 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efde0105390 0x7efde01987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:13.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.536+0000 7efde7bbd640 1 --2- 172.21.15.12:0/4178940128 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efdcc0a3bb0 0x7efdcc0a6070 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:13.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.536+0000 7efde7bbd640 1 --2- 172.21.15.12:0/4178940128 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efde0102a60 0x7efde0198260 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:13.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.536+0000 7efde7bbd640 1 -- 172.21.15.12:0/4178940128 >> 172.21.15.12:0/4178940128 conn(0x7efde00fc410 msgr2=0x7efde01034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:13.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.537+0000 7efde7bbd640 1 -- 172.21.15.12:0/4178940128 shutdown_connections 2024-04-22T13:22:13.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.537+0000 7efde7bbd640 1 -- 172.21.15.12:0/4178940128 wait complete. 2024-04-22T13:22:13.544 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:13 smithi012 ceph-mon[28265]: pgmap v291: 97 pgs: 97 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 3.7 KiB/s wr, 15 op/s 2024-04-22T13:22:13.544 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:13 smithi012 ceph-mon[28265]: mds.? [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] up:active 2024-04-22T13:22:13.544 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:13 smithi012 ceph-mon[28265]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2024-04-22T13:22:13.544 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:13 smithi012 ceph-mon[28265]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-22T13:22:13.544 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:13 smithi012 ceph-mon[28265]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:22:13.544 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:13 smithi012 ceph-mon[28265]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:22:13.544 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:13 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1151434084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:22:13.546 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-22T13:22:13.546 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T13:22:13.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.650+0000 7effe7e32640 1 -- 172.21.15.12:0/2011934470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe0101fa0 msgr2=0x7effe01023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:13.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.650+0000 7effe7e32640 1 --2- 172.21.15.12:0/2011934470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe0101fa0 0x7effe01023c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7effdc009900 tx=0x7effdc02bc30 comp rx=0 tx=0).stop 2024-04-22T13:22:13.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.650+0000 7effe7e32640 1 -- 172.21.15.12:0/2011934470 shutdown_connections 2024-04-22T13:22:13.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.650+0000 7effe7e32640 1 --2- 172.21.15.12:0/2011934470 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7effe0103b50 0x7effe010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:13.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.650+0000 7effe7e32640 1 --2- 172.21.15.12:0/2011934470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effe0103190 0x7effe0103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:13.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.650+0000 7effe7e32640 1 --2- 172.21.15.12:0/2011934470 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe0101fa0 0x7effe01023c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:13.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.650+0000 7effe7e32640 1 -- 172.21.15.12:0/2011934470 >> 172.21.15.12:0/2011934470 conn(0x7effe00fd530 msgr2=0x7effe00ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:13.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.650+0000 7effe7e32640 1 -- 172.21.15.12:0/2011934470 shutdown_connections 2024-04-22T13:22:13.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.650+0000 7effe7e32640 1 -- 172.21.15.12:0/2011934470 wait complete. 2024-04-22T13:22:13.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.651+0000 7effe7e32640 1 Processor -- start 2024-04-22T13:22:13.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.651+0000 7effe7e32640 1 -- start start 2024-04-22T13:22:13.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.651+0000 7effe7e32640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7effe0101fa0 0x7effe01981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:13.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.651+0000 7effe7e32640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effe0103190 0x7effe0198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:13.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.651+0000 7effe7e32640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe0103b50 0x7effe019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:13.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.651+0000 7effe7e32640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7effe010db20 con 0x7effe0103b50 2024-04-22T13:22:13.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.651+0000 7effe7e32640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7effe010d9a0 con 0x7effe0101fa0 2024-04-22T13:22:13.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.651+0000 7effe7e32640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7effe010dca0 con 0x7effe0103190 2024-04-22T13:22:13.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.651+0000 7effe63cf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe0103b50 0x7effe019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:13.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.651+0000 7effe5bce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7effe0101fa0 0x7effe01981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:13.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.651+0000 7effe5bce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7effe0101fa0 0x7effe01981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:44812/0 (socket says 172.21.15.12:44812) 2024-04-22T13:22:13.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.651+0000 7effe63cf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe0103b50 0x7effe019f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39894/0 (socket says 172.21.15.12:39894) 2024-04-22T13:22:13.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.651+0000 7effe5bce640 1 -- 172.21.15.12:0/4093128486 learned_addr learned my addr 172.21.15.12:0/4093128486 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:13.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.652+0000 7effe53cd640 1 --2- 172.21.15.12:0/4093128486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effe0103190 0x7effe0198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:13.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.652+0000 7effe63cf640 1 -- 172.21.15.12:0/4093128486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effe0103190 msgr2=0x7effe0198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:13.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.652+0000 7effe63cf640 1 --2- 172.21.15.12:0/4093128486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effe0103190 0x7effe0198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:13.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.652+0000 7effe63cf640 1 -- 172.21.15.12:0/4093128486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7effe0101fa0 msgr2=0x7effe01981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:13.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.652+0000 7effe63cf640 1 --2- 172.21.15.12:0/4093128486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7effe0101fa0 0x7effe01981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:13.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.652+0000 7effe63cf640 1 -- 172.21.15.12:0/4093128486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe019fe00 con 0x7effe0103b50 2024-04-22T13:22:13.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.652+0000 7effe63cf640 1 --2- 172.21.15.12:0/4093128486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe0103b50 0x7effe019f790 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7effd8009fd0 tx=0x7effd800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:13.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.652+0000 7effd6ffd640 1 -- 172.21.15.12:0/4093128486 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd800ce20 con 0x7effe0103b50 2024-04-22T13:22:13.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.652+0000 7effe7e32640 1 -- 172.21.15.12:0/4093128486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effe01a00f0 con 0x7effe0103b50 2024-04-22T13:22:13.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.652+0000 7effd6ffd640 1 -- 172.21.15.12:0/4093128486 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effd8005020 con 0x7effe0103b50 2024-04-22T13:22:13.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.652+0000 7effd6ffd640 1 -- 172.21.15.12:0/4093128486 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd8010690 con 0x7effe0103b50 2024-04-22T13:22:13.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.652+0000 7effe7e32640 1 -- 172.21.15.12:0/4093128486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effe01a0670 con 0x7effe0103b50 2024-04-22T13:22:13.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.653+0000 7effe7e32640 1 -- 172.21.15.12:0/4093128486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effe01092c0 con 0x7effe0103b50 2024-04-22T13:22:13.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.654+0000 7effd6ffd640 1 -- 172.21.15.12:0/4093128486 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7effd80054d0 con 0x7effe0103b50 2024-04-22T13:22:13.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.655+0000 7effd6ffd640 1 --2- 172.21.15.12:0/4093128486 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7effcc0a3c40 0x7effcc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:13.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.655+0000 7effe5bce640 1 --2- 172.21.15.12:0/4093128486 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7effcc0a3c40 0x7effcc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:13.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.655+0000 7effd6ffd640 1 -- 172.21.15.12:0/4093128486 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7effd80d0b50 con 0x7effe0103b50 2024-04-22T13:22:13.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.656+0000 7effe5bce640 1 --2- 172.21.15.12:0/4093128486 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7effcc0a3c40 0x7effcc0a6100 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7effdc02c840 tx=0x7effdc0095a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:13.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.660+0000 7effd6ffd640 1 -- 172.21.15.12:0/4093128486 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7effd809d580 con 0x7effe0103b50 2024-04-22T13:22:13.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:13 smithi134 ceph-mon[31338]: pgmap v291: 97 pgs: 97 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 3.7 KiB/s wr, 15 op/s 2024-04-22T13:22:13.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:13 smithi134 ceph-mon[31338]: mds.? [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] up:active 2024-04-22T13:22:13.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:13 smithi134 ceph-mon[31338]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2024-04-22T13:22:13.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:13 smithi134 ceph-mon[31338]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-22T13:22:13.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:13 smithi134 ceph-mon[31338]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:22:13.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:13 smithi134 ceph-mon[31338]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:22:13.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:13 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1151434084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:22:13.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:13 smithi111 ceph-mon[31361]: pgmap v291: 97 pgs: 97 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 3.7 KiB/s wr, 15 op/s 2024-04-22T13:22:13.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:13 smithi111 ceph-mon[31361]: mds.? [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] up:active 2024-04-22T13:22:13.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:13 smithi111 ceph-mon[31361]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2024-04-22T13:22:13.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:13 smithi111 ceph-mon[31361]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-22T13:22:13.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:13 smithi111 ceph-mon[31361]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:22:13.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:13 smithi111 ceph-mon[31361]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:22:13.728 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:13 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1151434084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:22:13.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.844+0000 7effe7e32640 1 -- 172.21.15.12:0/4093128486 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7effe0062390 con 0x7effcc0a3c40 2024-04-22T13:22:13.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.851+0000 7effd6ffd640 1 -- 172.21.15.12:0/4093128486 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258732 (secure 0 0 0) 0x7effe0062390 con 0x7effcc0a3c40 2024-04-22T13:22:13.852 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:22:13.853 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-22T13:22:13.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.855+0000 7effe7e32640 1 -- 172.21.15.12:0/4093128486 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7effcc0a3c40 msgr2=0x7effcc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:13.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.855+0000 7effe7e32640 1 --2- 172.21.15.12:0/4093128486 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7effcc0a3c40 0x7effcc0a6100 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7effdc02c840 tx=0x7effdc0095a0 comp rx=0 tx=0).stop 2024-04-22T13:22:13.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.855+0000 7effe7e32640 1 -- 172.21.15.12:0/4093128486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe0103b50 msgr2=0x7effe019f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:13.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.855+0000 7effe7e32640 1 --2- 172.21.15.12:0/4093128486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe0103b50 0x7effe019f790 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7effd8009fd0 tx=0x7effd800ef10 comp rx=0 tx=0).stop 2024-04-22T13:22:13.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.855+0000 7effe7e32640 1 -- 172.21.15.12:0/4093128486 shutdown_connections 2024-04-22T13:22:13.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.855+0000 7effe7e32640 1 --2- 172.21.15.12:0/4093128486 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7effcc0a3c40 0x7effcc0a6100 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:13.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.855+0000 7effe7e32640 1 --2- 172.21.15.12:0/4093128486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effe0103b50 0x7effe019f790 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:13.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.855+0000 7effe7e32640 1 --2- 172.21.15.12:0/4093128486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7effe0103190 0x7effe0198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:13.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.855+0000 7effe7e32640 1 --2- 172.21.15.12:0/4093128486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7effe0101fa0 0x7effe01981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:13.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.855+0000 7effe7e32640 1 -- 172.21.15.12:0/4093128486 >> 172.21.15.12:0/4093128486 conn(0x7effe00fd530 msgr2=0x7effe0106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:13.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.855+0000 7effe7e32640 1 -- 172.21.15.12:0/4093128486 shutdown_connections 2024-04-22T13:22:13.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:13.855+0000 7effe7e32640 1 -- 172.21.15.12:0/4093128486 wait complete. 2024-04-22T13:22:13.863 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-04-22T13:22:12.362549+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595287,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":140,"num_read_kb":123,"num_write":299,"num_write_kb":2968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":221,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374468,"kb_used_data":12792,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567356,"statfs":{"total":1151940427776,"available":1151556972544,"internally_reserved":0,"allocated":13099008,"data_stored":7103716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4919,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":118,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001849"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.399900+0000","last_change":"2024-04-22T13:22:05.254388+0000","last_active":"2024-04-22T13:22:08.399900+0000","last_peered":"2024-04-22T13:22:08.399900+0000","last_clean":"2024-04-22T13:22:08.399900+0000","last_became_active":"2024-04-22T13:22:05.254042+0000","last_became_peered":"2024-04-22T13:22:05.254042+0000","last_unstale":"2024-04-22T13:22:08.399900+0000","last_undegraded":"2024-04-22T13:22:08.399900+0000","last_fullsized":"2024-04-22T13:22:08.399900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:07:56.139653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.237324+0000","last_change":"2024-04-22T13:22:05.247879+0000","last_active":"2024-04-22T13:22:06.237324+0000","last_peered":"2024-04-22T13:22:06.237324+0000","last_clean":"2024-04-22T13:22:06.237324+0000","last_became_active":"2024-04-22T13:22:05.247508+0000","last_became_peered":"2024-04-22T13:22:05.247508+0000","last_unstale":"2024-04-22T13:22:06.237324+0000","last_undegraded":"2024-04-22T13:22:06.237324+0000","last_fullsized":"2024-04-22T13:22:06.237324+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:55:51.928658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.237413+0000","last_change":"2024-04-22T13:22:05.257970+0000","last_active":"2024-04-22T13:22:06.237413+0000","last_peered":"2024-04-22T13:22:06.237413+0000","last_clean":"2024-04-22T13:22:06.237413+0000","last_became_active":"2024-04-22T13:22:05.257652+0000","last_became_peered":"2024-04-22T13:22:05.257652+0000","last_unstale":"2024-04-22T13:22:06.237413+0000","last_undegraded":"2024-04-22T13:22:06.237413+0000","last_fullsized":"2024-04-22T13:22:06.237413+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:42:17.476349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:07.872534+0000","last_change":"2024-04-22T13:22:05.249186+0000","last_active":"2024-04-22T13:22:07.872534+0000","last_peered":"2024-04-22T13:22:07.872534+0000","last_clean":"2024-04-22T13:22:07.872534+0000","last_became_active":"2024-04-22T13:22:05.248814+0000","last_became_peered":"2024-04-22T13:22:05.248814+0000","last_unstale":"2024-04-22T13:22:07.872534+0000","last_undegraded":"2024-04-22T13:22:07.872534+0000","last_fullsized":"2024-04-22T13:22:07.872534+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:57.450794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.237721+0000","last_change":"2024-04-22T13:22:05.251953+0000","last_active":"2024-04-22T13:22:06.237721+0000","last_peered":"2024-04-22T13:22:06.237721+0000","last_clean":"2024-04-22T13:22:06.237721+0000","last_became_active":"2024-04-22T13:22:05.251547+0000","last_became_peered":"2024-04-22T13:22:05.251547+0000","last_unstale":"2024-04-22T13:22:06.237721+0000","last_undegraded":"2024-04-22T13:22:06.237721+0000","last_fullsized":"2024-04-22T13:22:06.237721+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:36:34.809878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.259562+0000","last_change":"2024-04-22T13:22:05.250619+0000","last_active":"2024-04-22T13:22:08.259562+0000","last_peered":"2024-04-22T13:22:08.259562+0000","last_clean":"2024-04-22T13:22:08.259562+0000","last_became_active":"2024-04-22T13:22:05.250283+0000","last_became_peered":"2024-04-22T13:22:05.250283+0000","last_unstale":"2024-04-22T13:22:08.259562+0000","last_undegraded":"2024-04-22T13:22:08.259562+0000","last_fullsized":"2024-04-22T13:22:08.259562+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:43:10.029253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.242404+0000","last_change":"2024-04-22T13:22:05.250418+0000","last_active":"2024-04-22T13:22:06.242404+0000","last_peered":"2024-04-22T13:22:06.242404+0000","last_clean":"2024-04-22T13:22:06.242404+0000","last_became_active":"2024-04-22T13:22:05.250097+0000","last_became_peered":"2024-04-22T13:22:05.250097+0000","last_unstale":"2024-04-22T13:22:06.242404+0000","last_undegraded":"2024-04-22T13:22:06.242404+0000","last_fullsized":"2024-04-22T13:22:06.242404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:34:59.598950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.378148+0000","last_change":"2024-04-22T13:22:05.244762+0000","last_active":"2024-04-22T13:22:06.378148+0000","last_peered":"2024-04-22T13:22:06.378148+0000","last_clean":"2024-04-22T13:22:06.378148+0000","last_became_active":"2024-04-22T13:22:05.244430+0000","last_became_peered":"2024-04-22T13:22:05.244430+0000","last_unstale":"2024-04-22T13:22:06.378148+0000","last_undegraded":"2024-04-22T13:22:06.378148+0000","last_fullsized":"2024-04-22T13:22:06.378148+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:33:01.582851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.258784+0000","last_change":"2024-04-22T13:22:05.257596+0000","last_active":"2024-04-22T13:22:05.258784+0000","last_peered":"2024-04-22T13:22:05.258784+0000","last_clean":"2024-04-22T13:22:05.258784+0000","last_became_active":"2024-04-22T13:22:05.257277+0000","last_became_peered":"2024-04-22T13:22:05.257277+0000","last_unstale":"2024-04-22T13:22:05.258784+0000","last_undegraded":"2024-04-22T13:22:05.258784+0000","last_fullsized":"2024-04-22T13:22:05.258784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:41:57.442610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.259432+0000","last_change":"2024-04-22T13:22:05.252177+0000","last_active":"2024-04-22T13:22:08.259432+0000","last_peered":"2024-04-22T13:22:08.259432+0000","last_clean":"2024-04-22T13:22:08.259432+0000","last_became_active":"2024-04-22T13:22:05.251794+0000","last_became_peered":"2024-04-22T13:22:05.251794+0000","last_unstale":"2024-04-22T13:22:08.259432+0000","last_undegraded":"2024-04-22T13:22:08.259432+0000","last_fullsized":"2024-04-22T13:22:08.259432+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:08:59.500540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.242883+0000","last_change":"2024-04-22T13:22:05.256209+0000","last_active":"2024-04-22T13:22:06.242883+0000","last_peered":"2024-04-22T13:22:06.242883+0000","last_clean":"2024-04-22T13:22:06.242883+0000","last_became_active":"2024-04-22T13:22:05.255504+0000","last_became_peered":"2024-04-22T13:22:05.255504+0000","last_unstale":"2024-04-22T13:22:06.242883+0000","last_undegraded":"2024-04-22T13:22:06.242883+0000","last_fullsized":"2024-04-22T13:22:06.242883+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:32:08.190921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:07.872481+0000","last_change":"2024-04-22T13:22:05.253939+0000","last_active":"2024-04-22T13:22:07.872481+0000","last_peered":"2024-04-22T13:22:07.872481+0000","last_clean":"2024-04-22T13:22:07.872481+0000","last_became_active":"2024-04-22T13:22:05.252939+0000","last_became_peered":"2024-04-22T13:22:05.252939+0000","last_unstale":"2024-04-22T13:22:07.872481+0000","last_undegraded":"2024-04-22T13:22:07.872481+0000","last_fullsized":"2024-04-22T13:22:07.872481+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:01:32.255047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.258921+0000","last_change":"2024-04-22T13:22:05.246667+0000","last_active":"2024-04-22T13:22:08.258921+0000","last_peered":"2024-04-22T13:22:08.258921+0000","last_clean":"2024-04-22T13:22:08.258921+0000","last_became_active":"2024-04-22T13:22:05.246365+0000","last_became_peered":"2024-04-22T13:22:05.246365+0000","last_unstale":"2024-04-22T13:22:08.258921+0000","last_undegraded":"2024-04-22T13:22:08.258921+0000","last_fullsized":"2024-04-22T13:22:08.258921+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:41:55.533430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.236961+0000","last_change":"2024-04-22T13:22:05.251982+0000","last_active":"2024-04-22T13:22:06.236961+0000","last_peered":"2024-04-22T13:22:06.236961+0000","last_clean":"2024-04-22T13:22:06.236961+0000","last_became_active":"2024-04-22T13:22:05.251575+0000","last_became_peered":"2024-04-22T13:22:05.251575+0000","last_unstale":"2024-04-22T13:22:06.236961+0000","last_undegraded":"2024-04-22T13:22:06.236961+0000","last_fullsized":"2024-04-22T13:22:06.236961+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:28:05.343140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.236910+0000","last_change":"2024-04-22T13:22:05.251056+0000","last_active":"2024-04-22T13:22:06.236910+0000","last_peered":"2024-04-22T13:22:06.236910+0000","last_clean":"2024-04-22T13:22:06.236910+0000","last_became_active":"2024-04-22T13:22:05.250652+0000","last_became_peered":"2024-04-22T13:22:05.250652+0000","last_unstale":"2024-04-22T13:22:06.236910+0000","last_undegraded":"2024-04-22T13:22:06.236910+0000","last_fullsized":"2024-04-22T13:22:06.236910+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:08:28.629965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.258372+0000","last_change":"2024-04-22T13:22:05.256877+0000","last_active":"2024-04-22T13:22:05.258372+0000","last_peered":"2024-04-22T13:22:05.258372+0000","last_clean":"2024-04-22T13:22:05.258372+0000","last_became_active":"2024-04-22T13:22:05.256074+0000","last_became_peered":"2024-04-22T13:22:05.256074+0000","last_unstale":"2024-04-22T13:22:05.258372+0000","last_undegraded":"2024-04-22T13:22:05.258372+0000","last_fullsized":"2024-04-22T13:22:05.258372+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:53:54.082142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.400172+0000","last_change":"2024-04-22T13:22:05.255598+0000","last_active":"2024-04-22T13:22:08.400172+0000","last_peered":"2024-04-22T13:22:08.400172+0000","last_clean":"2024-04-22T13:22:08.400172+0000","last_became_active":"2024-04-22T13:22:05.255239+0000","last_became_peered":"2024-04-22T13:22:05.255239+0000","last_unstale":"2024-04-22T13:22:08.400172+0000","last_undegraded":"2024-04-22T13:22:08.400172+0000","last_fullsized":"2024-04-22T13:22:08.400172+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:38:20.547882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.374676+0000","last_change":"2024-04-22T13:22:05.254232+0000","last_active":"2024-04-22T13:22:06.374676+0000","last_peered":"2024-04-22T13:22:06.374676+0000","last_clean":"2024-04-22T13:22:06.374676+0000","last_became_active":"2024-04-22T13:22:05.253907+0000","last_became_peered":"2024-04-22T13:22:05.253907+0000","last_unstale":"2024-04-22T13:22:06.374676+0000","last_undegraded":"2024-04-22T13:22:06.374676+0000","last_fullsized":"2024-04-22T13:22:06.374676+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:15:49.931112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:07.530160+0000","last_change":"2024-04-22T13:22:05.251518+0000","last_active":"2024-04-22T13:22:07.530160+0000","last_peered":"2024-04-22T13:22:07.530160+0000","last_clean":"2024-04-22T13:22:07.530160+0000","last_became_active":"2024-04-22T13:22:05.250478+0000","last_became_peered":"2024-04-22T13:22:05.250478+0000","last_unstale":"2024-04-22T13:22:07.530160+0000","last_undegraded":"2024-04-22T13:22:07.530160+0000","last_fullsized":"2024-04-22T13:22:07.530160+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:25:26.758692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:09.385907+0000","last_change":"2024-04-22T13:22:04.241365+0000","last_active":"2024-04-22T13:22:09.385907+0000","last_peered":"2024-04-22T13:22:09.385907+0000","last_clean":"2024-04-22T13:22:09.385907+0000","last_became_active":"2024-04-22T13:22:04.240822+0000","last_became_peered":"2024-04-22T13:22:04.240822+0000","last_unstale":"2024-04-22T13:22:09.385907+0000","last_undegraded":"2024-04-22T13:22:09.385907+0000","last_fullsized":"2024-04-22T13:22:09.385907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:20:23.928534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.378627+0000","last_change":"2024-04-22T13:22:05.249735+0000","last_active":"2024-04-22T13:22:06.378627+0000","last_peered":"2024-04-22T13:22:06.378627+0000","last_clean":"2024-04-22T13:22:06.378627+0000","last_became_active":"2024-04-22T13:22:05.249423+0000","last_became_peered":"2024-04-22T13:22:05.249423+0000","last_unstale":"2024-04-22T13:22:06.378627+0000","last_undegraded":"2024-04-22T13:22:06.378627+0000","last_fullsized":"2024-04-22T13:22:06.378627+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:38:49.425865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:07.295335+0000","last_change":"2024-04-22T13:22:04.248872+0000","last_active":"2024-04-22T13:22:07.295335+0000","last_peered":"2024-04-22T13:22:07.295335+0000","last_clean":"2024-04-22T13:22:07.295335+0000","last_became_active":"2024-04-22T13:22:04.248466+0000","last_became_peered":"2024-04-22T13:22:04.248466+0000","last_unstale":"2024-04-22T13:22:07.295335+0000","last_undegraded":"2024-04-22T13:22:07.295335+0000","last_fullsized":"2024-04-22T13:22:07.295335+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:38:10.871602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.237442+0000","last_change":"2024-04-22T13:22:05.258005+0000","last_active":"2024-04-22T13:22:06.237442+0000","last_peered":"2024-04-22T13:22:06.237442+0000","last_clean":"2024-04-22T13:22:06.237442+0000","last_became_active":"2024-04-22T13:22:05.257615+0000","last_became_peered":"2024-04-22T13:22:05.257615+0000","last_unstale":"2024-04-22T13:22:06.237442+0000","last_undegraded":"2024-04-22T13:22:06.237442+0000","last_fullsized":"2024-04-22T13:22:06.237442+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:21:18.030530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.238113+0000","last_change":"2024-04-22T13:22:04.246273+0000","last_active":"2024-04-22T13:22:06.238113+0000","last_peered":"2024-04-22T13:22:06.238113+0000","last_clean":"2024-04-22T13:22:06.238113+0000","last_became_active":"2024-04-22T13:22:04.245969+0000","last_became_peered":"2024-04-22T13:22:04.245969+0000","last_unstale":"2024-04-22T13:22:06.238113+0000","last_undegraded":"2024-04-22T13:22:06.238113+0000","last_fullsized":"2024-04-22T13:22:06.238113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:10.260192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.260743+0000","last_change":"2024-04-22T13:22:05.260299+0000","last_active":"2024-04-22T13:22:05.260743+0000","last_peered":"2024-04-22T13:22:05.260743+0000","last_clean":"2024-04-22T13:22:05.260743+0000","last_became_active":"2024-04-22T13:22:05.259634+0000","last_became_peered":"2024-04-22T13:22:05.259634+0000","last_unstale":"2024-04-22T13:22:05.260743+0000","last_undegraded":"2024-04-22T13:22:05.260743+0000","last_fullsized":"2024-04-22T13:22:05.260743+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:30:20.883785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.237885+0000","last_change":"2024-04-22T13:22:04.243843+0000","last_active":"2024-04-22T13:22:06.237885+0000","last_peered":"2024-04-22T13:22:06.237885+0000","last_clean":"2024-04-22T13:22:06.237885+0000","last_became_active":"2024-04-22T13:22:04.243465+0000","last_became_peered":"2024-04-22T13:22:04.243465+0000","last_unstale":"2024-04-22T13:22:06.237885+0000","last_undegraded":"2024-04-22T13:22:06.237885+0000","last_fullsized":"2024-04-22T13:22:06.237885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:48:36.431697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.260233+0000","last_change":"2024-04-22T13:22:05.259789+0000","last_active":"2024-04-22T13:22:05.260233+0000","last_peered":"2024-04-22T13:22:05.260233+0000","last_clean":"2024-04-22T13:22:05.260233+0000","last_became_active":"2024-04-22T13:22:05.259395+0000","last_became_peered":"2024-04-22T13:22:05.259395+0000","last_unstale":"2024-04-22T13:22:05.260233+0000","last_undegraded":"2024-04-22T13:22:05.260233+0000","last_fullsized":"2024-04-22T13:22:05.260233+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:35.446141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.234534+0000","last_change":"2024-04-22T13:22:04.240615+0000","last_active":"2024-04-22T13:22:05.234534+0000","last_peered":"2024-04-22T13:22:05.234534+0000","last_clean":"2024-04-22T13:22:05.234534+0000","last_became_active":"2024-04-22T13:22:04.240248+0000","last_became_peered":"2024-04-22T13:22:04.240248+0000","last_unstale":"2024-04-22T13:22:05.234534+0000","last_undegraded":"2024-04-22T13:22:05.234534+0000","last_fullsized":"2024-04-22T13:22:05.234534+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:55:05.948979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.237633+0000","last_change":"2024-04-22T13:22:05.253788+0000","last_active":"2024-04-22T13:22:06.237633+0000","last_peered":"2024-04-22T13:22:06.237633+0000","last_clean":"2024-04-22T13:22:06.237633+0000","last_became_active":"2024-04-22T13:22:05.252578+0000","last_became_peered":"2024-04-22T13:22:05.252578+0000","last_unstale":"2024-04-22T13:22:06.237633+0000","last_undegraded":"2024-04-22T13:22:06.237633+0000","last_fullsized":"2024-04-22T13:22:06.237633+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:04:11.769930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.259120+0000","last_change":"2024-04-22T13:22:04.241170+0000","last_active":"2024-04-22T13:22:08.259120+0000","last_peered":"2024-04-22T13:22:08.259120+0000","last_clean":"2024-04-22T13:22:08.259120+0000","last_became_active":"2024-04-22T13:22:04.240830+0000","last_became_peered":"2024-04-22T13:22:04.240830+0000","last_unstale":"2024-04-22T13:22:08.259120+0000","last_undegraded":"2024-04-22T13:22:08.259120+0000","last_fullsized":"2024-04-22T13:22:08.259120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:51:30.520613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.246158+0000","last_change":"2024-04-22T13:22:05.237900+0000","last_active":"2024-04-22T13:22:05.246158+0000","last_peered":"2024-04-22T13:22:05.246158+0000","last_clean":"2024-04-22T13:22:05.246158+0000","last_became_active":"2024-04-22T13:22:05.237507+0000","last_became_peered":"2024-04-22T13:22:05.237507+0000","last_unstale":"2024-04-22T13:22:05.246158+0000","last_undegraded":"2024-04-22T13:22:05.246158+0000","last_fullsized":"2024-04-22T13:22:05.246158+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:41:43.910207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:09.386142+0000","last_change":"2024-04-22T13:22:04.236078+0000","last_active":"2024-04-22T13:22:09.386142+0000","last_peered":"2024-04-22T13:22:09.386142+0000","last_clean":"2024-04-22T13:22:09.386142+0000","last_became_active":"2024-04-22T13:22:04.235710+0000","last_became_peered":"2024-04-22T13:22:04.235710+0000","last_unstale":"2024-04-22T13:22:09.386142+0000","last_undegraded":"2024-04-22T13:22:09.386142+0000","last_fullsized":"2024-04-22T13:22:09.386142+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:37:55.335237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.378097+0000","last_change":"2024-04-22T13:22:05.243152+0000","last_active":"2024-04-22T13:22:06.378097+0000","last_peered":"2024-04-22T13:22:06.378097+0000","last_clean":"2024-04-22T13:22:06.378097+0000","last_became_active":"2024-04-22T13:22:05.242772+0000","last_became_peered":"2024-04-22T13:22:05.242772+0000","last_unstale":"2024-04-22T13:22:06.378097+0000","last_undegraded":"2024-04-22T13:22:06.378097+0000","last_fullsized":"2024-04-22T13:22:06.378097+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:33:42.359679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.234359+0000","last_change":"2024-04-22T13:22:04.244381+0000","last_active":"2024-04-22T13:22:05.234359+0000","last_peered":"2024-04-22T13:22:05.234359+0000","last_clean":"2024-04-22T13:22:05.234359+0000","last_became_active":"2024-04-22T13:22:04.243746+0000","last_became_peered":"2024-04-22T13:22:04.243746+0000","last_unstale":"2024-04-22T13:22:05.234359+0000","last_undegraded":"2024-04-22T13:22:05.234359+0000","last_fullsized":"2024-04-22T13:22:05.234359+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:15:36.019930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.374485+0000","last_change":"2024-04-22T13:22:05.245394+0000","last_active":"2024-04-22T13:22:06.374485+0000","last_peered":"2024-04-22T13:22:06.374485+0000","last_clean":"2024-04-22T13:22:06.374485+0000","last_became_active":"2024-04-22T13:22:05.243931+0000","last_became_peered":"2024-04-22T13:22:05.243931+0000","last_unstale":"2024-04-22T13:22:06.374485+0000","last_undegraded":"2024-04-22T13:22:06.374485+0000","last_fullsized":"2024-04-22T13:22:06.374485+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:11:10.790842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.399036+0000","last_change":"2024-04-22T13:22:04.242583+0000","last_active":"2024-04-22T13:22:08.399036+0000","last_peered":"2024-04-22T13:22:08.399036+0000","last_clean":"2024-04-22T13:22:08.399036+0000","last_became_active":"2024-04-22T13:22:04.242261+0000","last_became_peered":"2024-04-22T13:22:04.242261+0000","last_unstale":"2024-04-22T13:22:08.399036+0000","last_undegraded":"2024-04-22T13:22:08.399036+0000","last_fullsized":"2024-04-22T13:22:08.399036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:37:47.618573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.258274+0000","last_change":"2024-04-22T13:22:05.255293+0000","last_active":"2024-04-22T13:22:05.258274+0000","last_peered":"2024-04-22T13:22:05.258274+0000","last_clean":"2024-04-22T13:22:05.258274+0000","last_became_active":"2024-04-22T13:22:05.253949+0000","last_became_peered":"2024-04-22T13:22:05.253949+0000","last_unstale":"2024-04-22T13:22:05.258274+0000","last_undegraded":"2024-04-22T13:22:05.258274+0000","last_fullsized":"2024-04-22T13:22:05.258274+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:02:23.442708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.397641+0000","last_change":"2024-04-22T13:22:04.241529+0000","last_active":"2024-04-22T13:22:08.397641+0000","last_peered":"2024-04-22T13:22:08.397641+0000","last_clean":"2024-04-22T13:22:08.397641+0000","last_became_active":"2024-04-22T13:22:04.241164+0000","last_became_peered":"2024-04-22T13:22:04.241164+0000","last_unstale":"2024-04-22T13:22:08.397641+0000","last_undegraded":"2024-04-22T13:22:08.397641+0000","last_fullsized":"2024-04-22T13:22:08.397641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:16:49.387181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.237429+0000","last_change":"2024-04-22T13:22:05.241623+0000","last_active":"2024-04-22T13:22:06.237429+0000","last_peered":"2024-04-22T13:22:06.237429+0000","last_clean":"2024-04-22T13:22:06.237429+0000","last_became_active":"2024-04-22T13:22:05.241150+0000","last_became_peered":"2024-04-22T13:22:05.241150+0000","last_unstale":"2024-04-22T13:22:06.237429+0000","last_undegraded":"2024-04-22T13:22:06.237429+0000","last_fullsized":"2024-04-22T13:22:06.237429+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:17:28.412366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'3","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.377801+0000","last_change":"2024-04-22T13:22:04.247852+0000","last_active":"2024-04-22T13:22:05.377801+0000","last_peered":"2024-04-22T13:22:05.377801+0000","last_clean":"2024-04-22T13:22:05.377801+0000","last_became_active":"2024-04-22T13:22:04.247550+0000","last_became_peered":"2024-04-22T13:22:04.247550+0000","last_unstale":"2024-04-22T13:22:05.377801+0000","last_undegraded":"2024-04-22T13:22:05.377801+0000","last_fullsized":"2024-04-22T13:22:05.377801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:55:01.928435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.260640+0000","last_change":"2024-04-22T13:22:05.260050+0000","last_active":"2024-04-22T13:22:05.260640+0000","last_peered":"2024-04-22T13:22:05.260640+0000","last_clean":"2024-04-22T13:22:05.260640+0000","last_became_active":"2024-04-22T13:22:05.259098+0000","last_became_peered":"2024-04-22T13:22:05.259098+0000","last_unstale":"2024-04-22T13:22:05.260640+0000","last_undegraded":"2024-04-22T13:22:05.260640+0000","last_fullsized":"2024-04-22T13:22:05.260640+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:24:52.552016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'8","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.398117+0000","last_change":"2024-04-22T13:22:04.240233+0000","last_active":"2024-04-22T13:22:08.398117+0000","last_peered":"2024-04-22T13:22:08.398117+0000","last_clean":"2024-04-22T13:22:08.398117+0000","last_became_active":"2024-04-22T13:22:04.239890+0000","last_became_peered":"2024-04-22T13:22:04.239890+0000","last_unstale":"2024-04-22T13:22:08.398117+0000","last_undegraded":"2024-04-22T13:22:08.398117+0000","last_fullsized":"2024-04-22T13:22:08.398117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:04:21.547422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.237968+0000","last_change":"2024-04-22T13:22:05.257756+0000","last_active":"2024-04-22T13:22:06.237968+0000","last_peered":"2024-04-22T13:22:06.237968+0000","last_clean":"2024-04-22T13:22:06.237968+0000","last_became_active":"2024-04-22T13:22:05.257442+0000","last_became_peered":"2024-04-22T13:22:05.257442+0000","last_unstale":"2024-04-22T13:22:06.237968+0000","last_undegraded":"2024-04-22T13:22:06.237968+0000","last_fullsized":"2024-04-22T13:22:06.237968+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:16:53.465689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:11.402777+0000","last_change":"2024-04-22T13:22:04.240815+0000","last_active":"2024-04-22T13:22:11.402777+0000","last_peered":"2024-04-22T13:22:11.402777+0000","last_clean":"2024-04-22T13:22:11.402777+0000","last_became_active":"2024-04-22T13:22:04.240218+0000","last_became_peered":"2024-04-22T13:22:04.240218+0000","last_unstale":"2024-04-22T13:22:11.402777+0000","last_undegraded":"2024-04-22T13:22:11.402777+0000","last_fullsized":"2024-04-22T13:22:11.402777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:29:29.148764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.258649+0000","last_change":"2024-04-22T13:22:05.248797+0000","last_active":"2024-04-22T13:22:08.258649+0000","last_peered":"2024-04-22T13:22:08.258649+0000","last_clean":"2024-04-22T13:22:08.258649+0000","last_became_active":"2024-04-22T13:22:05.248435+0000","last_became_peered":"2024-04-22T13:22:05.248435+0000","last_unstale":"2024-04-22T13:22:08.258649+0000","last_undegraded":"2024-04-22T13:22:08.258649+0000","last_fullsized":"2024-04-22T13:22:08.258649+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:32:31.970494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.234419+0000","last_change":"2024-04-22T13:22:04.245303+0000","last_active":"2024-04-22T13:22:05.234419+0000","last_peered":"2024-04-22T13:22:05.234419+0000","last_clean":"2024-04-22T13:22:05.234419+0000","last_became_active":"2024-04-22T13:22:04.244903+0000","last_became_peered":"2024-04-22T13:22:04.244903+0000","last_unstale":"2024-04-22T13:22:05.234419+0000","last_undegraded":"2024-04-22T13:22:05.234419+0000","last_fullsized":"2024-04-22T13:22:05.234419+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:34:32.174858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.242794+0000","last_change":"2024-04-22T13:22:05.246318+0000","last_active":"2024-04-22T13:22:06.242794+0000","last_peered":"2024-04-22T13:22:06.242794+0000","last_clean":"2024-04-22T13:22:06.242794+0000","last_became_active":"2024-04-22T13:22:05.245899+0000","last_became_peered":"2024-04-22T13:22:05.245899+0000","last_unstale":"2024-04-22T13:22:06.242794+0000","last_undegraded":"2024-04-22T13:22:06.242794+0000","last_fullsized":"2024-04-22T13:22:06.242794+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:27:42.789058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:10.392090+0000","last_change":"2024-04-22T13:22:04.240612+0000","last_active":"2024-04-22T13:22:10.392090+0000","last_peered":"2024-04-22T13:22:10.392090+0000","last_clean":"2024-04-22T13:22:10.392090+0000","last_became_active":"2024-04-22T13:22:04.240290+0000","last_became_peered":"2024-04-22T13:22:04.240290+0000","last_unstale":"2024-04-22T13:22:10.392090+0000","last_undegraded":"2024-04-22T13:22:10.392090+0000","last_fullsized":"2024-04-22T13:22:10.392090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:00:50.285553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.242390+0000","last_change":"2024-04-22T13:22:05.253205+0000","last_active":"2024-04-22T13:22:06.242390+0000","last_peered":"2024-04-22T13:22:06.242390+0000","last_clean":"2024-04-22T13:22:06.242390+0000","last_became_active":"2024-04-22T13:22:05.252865+0000","last_became_peered":"2024-04-22T13:22:05.252865+0000","last_unstale":"2024-04-22T13:22:06.242390+0000","last_undegraded":"2024-04-22T13:22:06.242390+0000","last_fullsized":"2024-04-22T13:22:06.242390+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:19:12.482371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'8","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.237479+0000","last_change":"2024-04-22T13:22:04.248938+0000","last_active":"2024-04-22T13:22:06.237479+0000","last_peered":"2024-04-22T13:22:06.237479+0000","last_clean":"2024-04-22T13:22:06.237479+0000","last_became_active":"2024-04-22T13:22:04.248440+0000","last_became_peered":"2024-04-22T13:22:04.248440+0000","last_unstale":"2024-04-22T13:22:06.237479+0000","last_undegraded":"2024-04-22T13:22:06.237479+0000","last_fullsized":"2024-04-22T13:22:06.237479+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:46:50.928067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.256079+0000","last_change":"2024-04-22T13:22:05.254664+0000","last_active":"2024-04-22T13:22:05.256079+0000","last_peered":"2024-04-22T13:22:05.256079+0000","last_clean":"2024-04-22T13:22:05.256079+0000","last_became_active":"2024-04-22T13:22:05.254304+0000","last_became_peered":"2024-04-22T13:22:05.254304+0000","last_unstale":"2024-04-22T13:22:05.256079+0000","last_undegraded":"2024-04-22T13:22:05.256079+0000","last_fullsized":"2024-04-22T13:22:05.256079+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:18:57.622836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.238119+0000","last_change":"2024-04-22T13:22:04.244128+0000","last_active":"2024-04-22T13:22:06.238119+0000","last_peered":"2024-04-22T13:22:06.238119+0000","last_clean":"2024-04-22T13:22:06.238119+0000","last_became_active":"2024-04-22T13:22:04.243738+0000","last_became_peered":"2024-04-22T13:22:04.243738+0000","last_unstale":"2024-04-22T13:22:06.238119+0000","last_undegraded":"2024-04-22T13:22:06.238119+0000","last_fullsized":"2024-04-22T13:22:06.238119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:23:52.220584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.399335+0000","last_change":"2024-04-22T13:22:05.250910+0000","last_active":"2024-04-22T13:22:08.399335+0000","last_peered":"2024-04-22T13:22:08.399335+0000","last_clean":"2024-04-22T13:22:08.399335+0000","last_became_active":"2024-04-22T13:22:05.250410+0000","last_became_peered":"2024-04-22T13:22:05.250410+0000","last_unstale":"2024-04-22T13:22:08.399335+0000","last_undegraded":"2024-04-22T13:22:08.399335+0000","last_fullsized":"2024-04-22T13:22:08.399335+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:04:25.492677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.374499+0000","last_change":"2024-04-22T13:22:04.245889+0000","last_active":"2024-04-22T13:22:06.374499+0000","last_peered":"2024-04-22T13:22:06.374499+0000","last_clean":"2024-04-22T13:22:06.374499+0000","last_became_active":"2024-04-22T13:22:04.245474+0000","last_became_peered":"2024-04-22T13:22:04.245474+0000","last_unstale":"2024-04-22T13:22:06.374499+0000","last_undegraded":"2024-04-22T13:22:06.374499+0000","last_fullsized":"2024-04-22T13:22:06.374499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:22:12.626950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.259356+0000","last_change":"2024-04-22T13:22:05.250978+0000","last_active":"2024-04-22T13:22:08.259356+0000","last_peered":"2024-04-22T13:22:08.259356+0000","last_clean":"2024-04-22T13:22:08.259356+0000","last_became_active":"2024-04-22T13:22:05.250463+0000","last_became_peered":"2024-04-22T13:22:05.250463+0000","last_unstale":"2024-04-22T13:22:08.259356+0000","last_undegraded":"2024-04-22T13:22:08.259356+0000","last_fullsized":"2024-04-22T13:22:08.259356+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:45:18.591072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.242578+0000","last_change":"2024-04-22T13:22:04.242503+0000","last_active":"2024-04-22T13:22:06.242578+0000","last_peered":"2024-04-22T13:22:06.242578+0000","last_clean":"2024-04-22T13:22:06.242578+0000","last_became_active":"2024-04-22T13:22:04.242143+0000","last_became_peered":"2024-04-22T13:22:04.242143+0000","last_unstale":"2024-04-22T13:22:06.242578+0000","last_undegraded":"2024-04-22T13:22:06.242578+0000","last_fullsized":"2024-04-22T13:22:06.242578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:40.236377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.259848+0000","last_change":"2024-04-22T13:22:05.252387+0000","last_active":"2024-04-22T13:22:08.259848+0000","last_peered":"2024-04-22T13:22:08.259848+0000","last_clean":"2024-04-22T13:22:08.259848+0000","last_became_active":"2024-04-22T13:22:05.252033+0000","last_became_peered":"2024-04-22T13:22:05.252033+0000","last_unstale":"2024-04-22T13:22:08.259848+0000","last_undegraded":"2024-04-22T13:22:08.259848+0000","last_fullsized":"2024-04-22T13:22:08.259848+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:55:01.791283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.249496+0000","last_change":"2024-04-22T13:22:05.248394+0000","last_active":"2024-04-22T13:22:05.249496+0000","last_peered":"2024-04-22T13:22:05.249496+0000","last_clean":"2024-04-22T13:22:05.249496+0000","last_became_active":"2024-04-22T13:22:05.248024+0000","last_became_peered":"2024-04-22T13:22:05.248024+0000","last_unstale":"2024-04-22T13:22:05.249496+0000","last_undegraded":"2024-04-22T13:22:05.249496+0000","last_fullsized":"2024-04-22T13:22:05.249496+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:35:39.136654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:09.386308+0000","last_change":"2024-04-22T13:22:04.247692+0000","last_active":"2024-04-22T13:22:09.386308+0000","last_peered":"2024-04-22T13:22:09.386308+0000","last_clean":"2024-04-22T13:22:09.386308+0000","last_became_active":"2024-04-22T13:22:04.247345+0000","last_became_peered":"2024-04-22T13:22:04.247345+0000","last_unstale":"2024-04-22T13:22:09.386308+0000","last_undegraded":"2024-04-22T13:22:09.386308+0000","last_fullsized":"2024-04-22T13:22:09.386308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:07:49.803248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.378466+0000","last_change":"2024-04-22T13:22:05.241022+0000","last_active":"2024-04-22T13:22:06.378466+0000","last_peered":"2024-04-22T13:22:06.378466+0000","last_clean":"2024-04-22T13:22:06.378466+0000","last_became_active":"2024-04-22T13:22:05.240635+0000","last_became_peered":"2024-04-22T13:22:05.240635+0000","last_unstale":"2024-04-22T13:22:06.378466+0000","last_undegraded":"2024-04-22T13:22:06.378466+0000","last_fullsized":"2024-04-22T13:22:06.378466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:50:02.012260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:10.391925+0000","last_change":"2024-04-22T13:22:04.236817+0000","last_active":"2024-04-22T13:22:10.391925+0000","last_peered":"2024-04-22T13:22:10.391925+0000","last_clean":"2024-04-22T13:22:10.391925+0000","last_became_active":"2024-04-22T13:22:04.236484+0000","last_became_peered":"2024-04-22T13:22:04.236484+0000","last_unstale":"2024-04-22T13:22:10.391925+0000","last_undegraded":"2024-04-22T13:22:10.391925+0000","last_fullsized":"2024-04-22T13:22:10.391925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:27:41.053327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":350,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.242350+0000","last_change":"2024-04-22T13:20:53.094123+0000","last_active":"2024-04-22T13:22:06.242350+0000","last_peered":"2024-04-22T13:22:06.242350+0000","last_clean":"2024-04-22T13:22:06.242350+0000","last_became_active":"2024-04-22T13:20:53.093664+0000","last_became_peered":"2024-04-22T13:20:53.093664+0000","last_unstale":"2024-04-22T13:22:06.242350+0000","last_undegraded":"2024-04-22T13:22:06.242350+0000","last_fullsized":"2024-04-22T13:22:06.242350+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:17:33.201786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:17:33.201786+0000","last_clean_scrub_stamp":"2024-04-22T13:17:33.201786+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:46:58.986399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.258555+0000","last_change":"2024-04-22T13:22:05.247766+0000","last_active":"2024-04-22T13:22:08.258555+0000","last_peered":"2024-04-22T13:22:08.258555+0000","last_clean":"2024-04-22T13:22:08.258555+0000","last_became_active":"2024-04-22T13:22:05.247362+0000","last_became_peered":"2024-04-22T13:22:05.247362+0000","last_unstale":"2024-04-22T13:22:08.258555+0000","last_undegraded":"2024-04-22T13:22:08.258555+0000","last_fullsized":"2024-04-22T13:22:08.258555+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:25:00.126638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:09.385197+0000","last_change":"2024-04-22T13:22:04.243493+0000","last_active":"2024-04-22T13:22:09.385197+0000","last_peered":"2024-04-22T13:22:09.385197+0000","last_clean":"2024-04-22T13:22:09.385197+0000","last_became_active":"2024-04-22T13:22:04.243190+0000","last_became_peered":"2024-04-22T13:22:04.243190+0000","last_unstale":"2024-04-22T13:22:09.385197+0000","last_undegraded":"2024-04-22T13:22:09.385197+0000","last_fullsized":"2024-04-22T13:22:09.385197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:15:09.025152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.238040+0000","last_change":"2024-04-22T13:22:04.249222+0000","last_active":"2024-04-22T13:22:06.238040+0000","last_peered":"2024-04-22T13:22:06.238040+0000","last_clean":"2024-04-22T13:22:06.238040+0000","last_became_active":"2024-04-22T13:22:04.248926+0000","last_became_peered":"2024-04-22T13:22:04.248926+0000","last_unstale":"2024-04-22T13:22:06.238040+0000","last_undegraded":"2024-04-22T13:22:06.238040+0000","last_fullsized":"2024-04-22T13:22:06.238040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:33:50.168426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.258780+0000","last_change":"2024-04-22T13:22:05.248199+0000","last_active":"2024-04-22T13:22:08.258780+0000","last_peered":"2024-04-22T13:22:08.258780+0000","last_clean":"2024-04-22T13:22:08.258780+0000","last_became_active":"2024-04-22T13:22:05.247873+0000","last_became_peered":"2024-04-22T13:22:05.247873+0000","last_unstale":"2024-04-22T13:22:08.258780+0000","last_undegraded":"2024-04-22T13:22:08.258780+0000","last_fullsized":"2024-04-22T13:22:08.258780+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:40:29.353901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:09.384236+0000","last_change":"2024-04-22T13:22:04.243474+0000","last_active":"2024-04-22T13:22:09.384236+0000","last_peered":"2024-04-22T13:22:09.384236+0000","last_clean":"2024-04-22T13:22:09.384236+0000","last_became_active":"2024-04-22T13:22:04.243151+0000","last_became_peered":"2024-04-22T13:22:04.243151+0000","last_unstale":"2024-04-22T13:22:09.384236+0000","last_undegraded":"2024-04-22T13:22:09.384236+0000","last_fullsized":"2024-04-22T13:22:09.384236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:18:25.140715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.378634+0000","last_change":"2024-04-22T13:22:05.243120+0000","last_active":"2024-04-22T13:22:06.378634+0000","last_peered":"2024-04-22T13:22:06.378634+0000","last_clean":"2024-04-22T13:22:06.378634+0000","last_became_active":"2024-04-22T13:22:05.242726+0000","last_became_peered":"2024-04-22T13:22:05.242726+0000","last_unstale":"2024-04-22T13:22:06.378634+0000","last_undegraded":"2024-04-22T13:22:06.378634+0000","last_fullsized":"2024-04-22T13:22:06.378634+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:06:16.087884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.400718+0000","last_change":"2024-04-22T13:22:04.239672+0000","last_active":"2024-04-22T13:22:08.400718+0000","last_peered":"2024-04-22T13:22:08.400718+0000","last_clean":"2024-04-22T13:22:08.400718+0000","last_became_active":"2024-04-22T13:22:04.238776+0000","last_became_peered":"2024-04-22T13:22:04.238776+0000","last_unstale":"2024-04-22T13:22:08.400718+0000","last_undegraded":"2024-04-22T13:22:08.400718+0000","last_fullsized":"2024-04-22T13:22:08.400718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:43:21.752547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.237383+0000","last_change":"2024-04-22T13:22:05.242089+0000","last_active":"2024-04-22T13:22:06.237383+0000","last_peered":"2024-04-22T13:22:06.237383+0000","last_clean":"2024-04-22T13:22:06.237383+0000","last_became_active":"2024-04-22T13:22:05.241681+0000","last_became_peered":"2024-04-22T13:22:05.241681+0000","last_unstale":"2024-04-22T13:22:06.237383+0000","last_undegraded":"2024-04-22T13:22:06.237383+0000","last_fullsized":"2024-04-22T13:22:06.237383+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:58:49.906188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.250782+0000","last_change":"2024-04-22T13:22:04.248647+0000","last_active":"2024-04-22T13:22:05.250782+0000","last_peered":"2024-04-22T13:22:05.250782+0000","last_clean":"2024-04-22T13:22:05.250782+0000","last_became_active":"2024-04-22T13:22:04.248408+0000","last_became_peered":"2024-04-22T13:22:04.248408+0000","last_unstale":"2024-04-22T13:22:05.250782+0000","last_undegraded":"2024-04-22T13:22:05.250782+0000","last_fullsized":"2024-04-22T13:22:05.250782+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:16:26.244223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.237585+0000","last_change":"2024-04-22T13:22:05.250120+0000","last_active":"2024-04-22T13:22:06.237585+0000","last_peered":"2024-04-22T13:22:06.237585+0000","last_clean":"2024-04-22T13:22:06.237585+0000","last_became_active":"2024-04-22T13:22:05.247936+0000","last_became_peered":"2024-04-22T13:22:05.247936+0000","last_unstale":"2024-04-22T13:22:06.237585+0000","last_undegraded":"2024-04-22T13:22:06.237585+0000","last_fullsized":"2024-04-22T13:22:06.237585+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:27:38.437918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:11.401215+0000","last_change":"2024-04-22T13:22:04.244026+0000","last_active":"2024-04-22T13:22:11.401215+0000","last_peered":"2024-04-22T13:22:11.401215+0000","last_clean":"2024-04-22T13:22:11.401215+0000","last_became_active":"2024-04-22T13:22:04.243518+0000","last_became_peered":"2024-04-22T13:22:04.243518+0000","last_unstale":"2024-04-22T13:22:11.401215+0000","last_undegraded":"2024-04-22T13:22:11.401215+0000","last_fullsized":"2024-04-22T13:22:11.401215+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:54:48.791874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.398466+0000","last_change":"2024-04-22T13:22:05.252674+0000","last_active":"2024-04-22T13:22:08.398466+0000","last_peered":"2024-04-22T13:22:08.398466+0000","last_clean":"2024-04-22T13:22:08.398466+0000","last_became_active":"2024-04-22T13:22:05.251087+0000","last_became_peered":"2024-04-22T13:22:05.251087+0000","last_unstale":"2024-04-22T13:22:08.398466+0000","last_undegraded":"2024-04-22T13:22:08.398466+0000","last_fullsized":"2024-04-22T13:22:08.398466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:20:06.873412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:10.392209+0000","last_change":"2024-04-22T13:22:04.244409+0000","last_active":"2024-04-22T13:22:10.392209+0000","last_peered":"2024-04-22T13:22:10.392209+0000","last_clean":"2024-04-22T13:22:10.392209+0000","last_became_active":"2024-04-22T13:22:04.244104+0000","last_became_peered":"2024-04-22T13:22:04.244104+0000","last_unstale":"2024-04-22T13:22:10.392209+0000","last_undegraded":"2024-04-22T13:22:10.392209+0000","last_fullsized":"2024-04-22T13:22:10.392209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:39:14.714755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.378315+0000","last_change":"2024-04-22T13:22:05.245730+0000","last_active":"2024-04-22T13:22:06.378315+0000","last_peered":"2024-04-22T13:22:06.378315+0000","last_clean":"2024-04-22T13:22:06.378315+0000","last_became_active":"2024-04-22T13:22:05.245366+0000","last_became_peered":"2024-04-22T13:22:05.245366+0000","last_unstale":"2024-04-22T13:22:06.378315+0000","last_undegraded":"2024-04-22T13:22:06.378315+0000","last_fullsized":"2024-04-22T13:22:06.378315+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:31.935258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.237657+0000","last_change":"2024-04-22T13:22:04.246683+0000","last_active":"2024-04-22T13:22:06.237657+0000","last_peered":"2024-04-22T13:22:06.237657+0000","last_clean":"2024-04-22T13:22:06.237657+0000","last_became_active":"2024-04-22T13:22:04.246372+0000","last_became_peered":"2024-04-22T13:22:04.246372+0000","last_unstale":"2024-04-22T13:22:06.237657+0000","last_undegraded":"2024-04-22T13:22:06.237657+0000","last_fullsized":"2024-04-22T13:22:06.237657+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:01:13.464077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.242859+0000","last_change":"2024-04-22T13:22:05.242146+0000","last_active":"2024-04-22T13:22:06.242859+0000","last_peered":"2024-04-22T13:22:06.242859+0000","last_clean":"2024-04-22T13:22:06.242859+0000","last_became_active":"2024-04-22T13:22:05.241802+0000","last_became_peered":"2024-04-22T13:22:05.241802+0000","last_unstale":"2024-04-22T13:22:06.242859+0000","last_undegraded":"2024-04-22T13:22:06.242859+0000","last_fullsized":"2024-04-22T13:22:06.242859+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:49:49.002123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:10.392663+0000","last_change":"2024-04-22T13:22:04.248884+0000","last_active":"2024-04-22T13:22:10.392663+0000","last_peered":"2024-04-22T13:22:10.392663+0000","last_clean":"2024-04-22T13:22:10.392663+0000","last_became_active":"2024-04-22T13:22:04.248393+0000","last_became_peered":"2024-04-22T13:22:04.248393+0000","last_unstale":"2024-04-22T13:22:10.392663+0000","last_undegraded":"2024-04-22T13:22:10.392663+0000","last_fullsized":"2024-04-22T13:22:10.392663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:36:14.687457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.242948+0000","last_change":"2024-04-22T13:22:05.256881+0000","last_active":"2024-04-22T13:22:06.242948+0000","last_peered":"2024-04-22T13:22:06.242948+0000","last_clean":"2024-04-22T13:22:06.242948+0000","last_became_active":"2024-04-22T13:22:05.255967+0000","last_became_peered":"2024-04-22T13:22:05.255967+0000","last_unstale":"2024-04-22T13:22:06.242948+0000","last_undegraded":"2024-04-22T13:22:06.242948+0000","last_fullsized":"2024-04-22T13:22:06.242948+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:24:21.137344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.234442+0000","last_change":"2024-04-22T13:22:04.247303+0000","last_active":"2024-04-22T13:22:05.234442+0000","last_peered":"2024-04-22T13:22:05.234442+0000","last_clean":"2024-04-22T13:22:05.234442+0000","last_became_active":"2024-04-22T13:22:04.246742+0000","last_became_peered":"2024-04-22T13:22:04.246742+0000","last_unstale":"2024-04-22T13:22:05.234442+0000","last_undegraded":"2024-04-22T13:22:05.234442+0000","last_fullsized":"2024-04-22T13:22:05.234442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:45:03.189042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.242622+0000","last_change":"2024-04-22T13:22:05.251754+0000","last_active":"2024-04-22T13:22:06.242622+0000","last_peered":"2024-04-22T13:22:06.242622+0000","last_clean":"2024-04-22T13:22:06.242622+0000","last_became_active":"2024-04-22T13:22:05.251155+0000","last_became_peered":"2024-04-22T13:22:05.251155+0000","last_unstale":"2024-04-22T13:22:06.242622+0000","last_undegraded":"2024-04-22T13:22:06.242622+0000","last_fullsized":"2024-04-22T13:22:06.242622+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:08:46.682600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.258891+0000","last_change":"2024-04-22T13:22:04.243382+0000","last_active":"2024-04-22T13:22:08.258891+0000","last_peered":"2024-04-22T13:22:08.258891+0000","last_clean":"2024-04-22T13:22:08.258891+0000","last_became_active":"2024-04-22T13:22:04.243055+0000","last_became_peered":"2024-04-22T13:22:04.243055+0000","last_unstale":"2024-04-22T13:22:08.258891+0000","last_undegraded":"2024-04-22T13:22:08.258891+0000","last_fullsized":"2024-04-22T13:22:08.258891+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:19:58.379645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.242846+0000","last_change":"2024-04-22T13:22:05.249527+0000","last_active":"2024-04-22T13:22:06.242846+0000","last_peered":"2024-04-22T13:22:06.242846+0000","last_clean":"2024-04-22T13:22:06.242846+0000","last_became_active":"2024-04-22T13:22:05.249207+0000","last_became_peered":"2024-04-22T13:22:05.249207+0000","last_unstale":"2024-04-22T13:22:06.242846+0000","last_undegraded":"2024-04-22T13:22:06.242846+0000","last_fullsized":"2024-04-22T13:22:06.242846+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:50:02.755288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.398479+0000","last_change":"2024-04-22T13:22:05.257812+0000","last_active":"2024-04-22T13:22:08.398479+0000","last_peered":"2024-04-22T13:22:08.398479+0000","last_clean":"2024-04-22T13:22:08.398479+0000","last_became_active":"2024-04-22T13:22:05.257470+0000","last_became_peered":"2024-04-22T13:22:05.257470+0000","last_unstale":"2024-04-22T13:22:08.398479+0000","last_undegraded":"2024-04-22T13:22:08.398479+0000","last_fullsized":"2024-04-22T13:22:08.398479+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:47:25.343965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:07.529883+0000","last_change":"2024-04-22T13:22:05.253839+0000","last_active":"2024-04-22T13:22:07.529883+0000","last_peered":"2024-04-22T13:22:07.529883+0000","last_clean":"2024-04-22T13:22:07.529883+0000","last_became_active":"2024-04-22T13:22:05.253499+0000","last_became_peered":"2024-04-22T13:22:05.253499+0000","last_unstale":"2024-04-22T13:22:07.529883+0000","last_undegraded":"2024-04-22T13:22:07.529883+0000","last_fullsized":"2024-04-22T13:22:07.529883+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:40:50.456951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:07.529803+0000","last_change":"2024-04-22T13:22:05.247882+0000","last_active":"2024-04-22T13:22:07.529803+0000","last_peered":"2024-04-22T13:22:07.529803+0000","last_clean":"2024-04-22T13:22:07.529803+0000","last_became_active":"2024-04-22T13:22:05.247525+0000","last_became_peered":"2024-04-22T13:22:05.247525+0000","last_unstale":"2024-04-22T13:22:07.529803+0000","last_undegraded":"2024-04-22T13:22:07.529803+0000","last_fullsized":"2024-04-22T13:22:07.529803+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:46:04.349076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.237060+0000","last_change":"2024-04-22T13:22:05.243716+0000","last_active":"2024-04-22T13:22:06.237060+0000","last_peered":"2024-04-22T13:22:06.237060+0000","last_clean":"2024-04-22T13:22:06.237060+0000","last_became_active":"2024-04-22T13:22:05.243397+0000","last_became_peered":"2024-04-22T13:22:05.243397+0000","last_unstale":"2024-04-22T13:22:06.237060+0000","last_undegraded":"2024-04-22T13:22:06.237060+0000","last_fullsized":"2024-04-22T13:22:06.237060+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:16:21.236339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.258905+0000","last_change":"2024-04-22T13:22:05.246042+0000","last_active":"2024-04-22T13:22:08.258905+0000","last_peered":"2024-04-22T13:22:08.258905+0000","last_clean":"2024-04-22T13:22:08.258905+0000","last_became_active":"2024-04-22T13:22:05.245669+0000","last_became_peered":"2024-04-22T13:22:05.245669+0000","last_unstale":"2024-04-22T13:22:08.258905+0000","last_undegraded":"2024-04-22T13:22:08.258905+0000","last_fullsized":"2024-04-22T13:22:08.258905+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:49:05.539641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.399059+0000","last_change":"2024-04-22T13:22:05.257009+0000","last_active":"2024-04-22T13:22:08.399059+0000","last_peered":"2024-04-22T13:22:08.399059+0000","last_clean":"2024-04-22T13:22:08.399059+0000","last_became_active":"2024-04-22T13:22:05.254327+0000","last_became_peered":"2024-04-22T13:22:05.254327+0000","last_unstale":"2024-04-22T13:22:08.399059+0000","last_undegraded":"2024-04-22T13:22:08.399059+0000","last_fullsized":"2024-04-22T13:22:08.399059+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:29:12.482525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:07.530313+0000","last_change":"2024-04-22T13:22:05.260575+0000","last_active":"2024-04-22T13:22:07.530313+0000","last_peered":"2024-04-22T13:22:07.530313+0000","last_clean":"2024-04-22T13:22:07.530313+0000","last_became_active":"2024-04-22T13:22:05.260181+0000","last_became_peered":"2024-04-22T13:22:05.260181+0000","last_unstale":"2024-04-22T13:22:07.530313+0000","last_undegraded":"2024-04-22T13:22:07.530313+0000","last_fullsized":"2024-04-22T13:22:07.530313+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:49:20.064368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.378496+0000","last_change":"2024-04-22T13:22:05.248199+0000","last_active":"2024-04-22T13:22:06.378496+0000","last_peered":"2024-04-22T13:22:06.378496+0000","last_clean":"2024-04-22T13:22:06.378496+0000","last_became_active":"2024-04-22T13:22:05.247858+0000","last_became_peered":"2024-04-22T13:22:05.247858+0000","last_unstale":"2024-04-22T13:22:06.378496+0000","last_undegraded":"2024-04-22T13:22:06.378496+0000","last_fullsized":"2024-04-22T13:22:06.378496+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:49:21.228105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.246736+0000","last_change":"2024-04-22T13:22:05.239137+0000","last_active":"2024-04-22T13:22:05.246736+0000","last_peered":"2024-04-22T13:22:05.246736+0000","last_clean":"2024-04-22T13:22:05.246736+0000","last_became_active":"2024-04-22T13:22:05.238807+0000","last_became_peered":"2024-04-22T13:22:05.238807+0000","last_unstale":"2024-04-22T13:22:05.246736+0000","last_undegraded":"2024-04-22T13:22:05.246736+0000","last_fullsized":"2024-04-22T13:22:05.246736+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:39:02.950022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.378236+0000","last_change":"2024-04-22T13:22:05.249080+0000","last_active":"2024-04-22T13:22:06.378236+0000","last_peered":"2024-04-22T13:22:06.378236+0000","last_clean":"2024-04-22T13:22:06.378236+0000","last_became_active":"2024-04-22T13:22:05.248733+0000","last_became_peered":"2024-04-22T13:22:05.248733+0000","last_unstale":"2024-04-22T13:22:06.378236+0000","last_undegraded":"2024-04-22T13:22:06.378236+0000","last_fullsized":"2024-04-22T13:22:06.378236+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:58.735941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.258236+0000","last_change":"2024-04-22T13:22:05.255490+0000","last_active":"2024-04-22T13:22:05.258236+0000","last_peered":"2024-04-22T13:22:05.258236+0000","last_clean":"2024-04-22T13:22:05.258236+0000","last_became_active":"2024-04-22T13:22:05.253644+0000","last_became_peered":"2024-04-22T13:22:05.253644+0000","last_unstale":"2024-04-22T13:22:05.258236+0000","last_undegraded":"2024-04-22T13:22:05.258236+0000","last_fullsized":"2024-04-22T13:22:05.258236+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:42:05.958106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.258994+0000","last_change":"2024-04-22T13:22:05.240988+0000","last_active":"2024-04-22T13:22:08.258994+0000","last_peered":"2024-04-22T13:22:08.258994+0000","last_clean":"2024-04-22T13:22:08.258994+0000","last_became_active":"2024-04-22T13:22:05.240642+0000","last_became_peered":"2024-04-22T13:22:05.240642+0000","last_unstale":"2024-04-22T13:22:08.258994+0000","last_undegraded":"2024-04-22T13:22:08.258994+0000","last_fullsized":"2024-04-22T13:22:08.258994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:41:29.001979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.398806+0000","last_change":"2024-04-22T13:22:05.252212+0000","last_active":"2024-04-22T13:22:08.398806+0000","last_peered":"2024-04-22T13:22:08.398806+0000","last_clean":"2024-04-22T13:22:08.398806+0000","last_became_active":"2024-04-22T13:22:05.251852+0000","last_became_peered":"2024-04-22T13:22:05.251852+0000","last_unstale":"2024-04-22T13:22:08.398806+0000","last_undegraded":"2024-04-22T13:22:08.398806+0000","last_fullsized":"2024-04-22T13:22:08.398806+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:13:55.132501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4919,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":118,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":33906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318287,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":446155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579922,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28520,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716632,"statfs":{"total":95995035648,"available":95965831168,"internally_reserved":0,"allocated":1536000,"data_stored":1028567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743448,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":445781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005084,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":445567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234018,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32676,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712476,"statfs":{"total":95995035648,"available":95961575424,"internally_reserved":0,"allocated":1536000,"data_stored":1036535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430247,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":929792,"data_stored":444963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":438005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":445053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":455147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117047,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":445393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":942080,"data_stored":445607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691883,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28508,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716644,"statfs":{"total":95995035648,"available":95965843456,"internally_reserved":0,"allocated":1523712,"data_stored":1026943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":22867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T13:22:13.868 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T13:22:14.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.007+0000 7faf60c25640 1 -- 172.21.15.12:0/737763544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf5c102a90 msgr2=0x7faf5c104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:14.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.007+0000 7faf60c25640 1 --2- 172.21.15.12:0/737763544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf5c102a90 0x7faf5c104e80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7faf4c009ca0 tx=0x7faf4c02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:22:14.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.008+0000 7faf60c25640 1 -- 172.21.15.12:0/737763544 shutdown_connections 2024-04-22T13:22:14.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.008+0000 7faf60c25640 1 --2- 172.21.15.12:0/737763544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf5c107cf0 0x7faf5c10a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:14.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.008+0000 7faf60c25640 1 --2- 172.21.15.12:0/737763544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf5c1053c0 0x7faf5c1077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:14.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.008+0000 7faf60c25640 1 --2- 172.21.15.12:0/737763544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf5c102a90 0x7faf5c104e80 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:14.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.008+0000 7faf60c25640 1 -- 172.21.15.12:0/737763544 >> 172.21.15.12:0/737763544 conn(0x7faf5c0fc3e0 msgr2=0x7faf5c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:14.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.008+0000 7faf60c25640 1 -- 172.21.15.12:0/737763544 shutdown_connections 2024-04-22T13:22:14.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.008+0000 7faf60c25640 1 -- 172.21.15.12:0/737763544 wait complete. 2024-04-22T13:22:14.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.008+0000 7faf60c25640 1 Processor -- start 2024-04-22T13:22:14.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.008+0000 7faf60c25640 1 -- start start 2024-04-22T13:22:14.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.009+0000 7faf60c25640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf5c102a90 0x7faf5c1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:14.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.009+0000 7faf60c25640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf5c1053c0 0x7faf5c198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:14.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.009+0000 7faf60c25640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf5c107cf0 0x7faf5c19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:14.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.009+0000 7faf60c25640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7faf5c10dac0 con 0x7faf5c1053c0 2024-04-22T13:22:14.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.009+0000 7faf60c25640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faf5c10d940 con 0x7faf5c102a90 2024-04-22T13:22:14.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.009+0000 7faf60c25640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faf5c10dc40 con 0x7faf5c107cf0 2024-04-22T13:22:14.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.009+0000 7faf59d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf5c1053c0 0x7faf5c198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:14.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.009+0000 7faf59d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf5c1053c0 0x7faf5c198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39908/0 (socket says 172.21.15.12:39908) 2024-04-22T13:22:14.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.009+0000 7faf59d9b640 1 -- 172.21.15.12:0/1314827479 learned_addr learned my addr 172.21.15.12:0/1314827479 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:14.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.009+0000 7faf5ad9d640 1 --2- 172.21.15.12:0/1314827479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf5c107cf0 0x7faf5c19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:14.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.009+0000 7faf5a59c640 1 --2- 172.21.15.12:0/1314827479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf5c102a90 0x7faf5c1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:14.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.009+0000 7faf59d9b640 1 -- 172.21.15.12:0/1314827479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf5c107cf0 msgr2=0x7faf5c19f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:14.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.009+0000 7faf59d9b640 1 --2- 172.21.15.12:0/1314827479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf5c107cf0 0x7faf5c19f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:14.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.009+0000 7faf59d9b640 1 -- 172.21.15.12:0/1314827479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf5c102a90 msgr2=0x7faf5c1981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:14.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.009+0000 7faf59d9b640 1 --2- 172.21.15.12:0/1314827479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf5c102a90 0x7faf5c1981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:14.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.009+0000 7faf59d9b640 1 -- 172.21.15.12:0/1314827479 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf5c19fe10 con 0x7faf5c1053c0 2024-04-22T13:22:14.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.009+0000 7faf5a59c640 1 --2- 172.21.15.12:0/1314827479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf5c102a90 0x7faf5c1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:22:14.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.009+0000 7faf59d9b640 1 --2- 172.21.15.12:0/1314827479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf5c1053c0 0x7faf5c198730 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faf5000c970 tx=0x7faf5000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:14.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.009+0000 7faf4b7fe640 1 -- 172.21.15.12:0/1314827479 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf50007bb0 con 0x7faf5c1053c0 2024-04-22T13:22:14.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.009+0000 7faf60c25640 1 -- 172.21.15.12:0/1314827479 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf5c1a0100 con 0x7faf5c1053c0 2024-04-22T13:22:14.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.009+0000 7faf4b7fe640 1 -- 172.21.15.12:0/1314827479 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf50007d50 con 0x7faf5c1053c0 2024-04-22T13:22:14.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.009+0000 7faf4b7fe640 1 -- 172.21.15.12:0/1314827479 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5000f450 con 0x7faf5c1053c0 2024-04-22T13:22:14.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.009+0000 7faf60c25640 1 -- 172.21.15.12:0/1314827479 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf5c1a06a0 con 0x7faf5c1053c0 2024-04-22T13:22:14.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.010+0000 7faf60c25640 1 -- 172.21.15.12:0/1314827479 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf5c0fded0 con 0x7faf5c1053c0 2024-04-22T13:22:14.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.011+0000 7faf4b7fe640 1 -- 172.21.15.12:0/1314827479 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7faf50004ec0 con 0x7faf5c1053c0 2024-04-22T13:22:14.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.012+0000 7faf4b7fe640 1 --2- 172.21.15.12:0/1314827479 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7faf440a3c40 0x7faf440a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:14.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.012+0000 7faf5a59c640 1 --2- 172.21.15.12:0/1314827479 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7faf440a3c40 0x7faf440a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:14.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.012+0000 7faf4b7fe640 1 -- 172.21.15.12:0/1314827479 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7faf500d01a0 con 0x7faf5c1053c0 2024-04-22T13:22:14.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.013+0000 7faf5a59c640 1 --2- 172.21.15.12:0/1314827479 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7faf440a3c40 0x7faf440a6100 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7faf4c037040 tx=0x7faf4c005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:14.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.017+0000 7faf4b7fe640 1 -- 172.21.15.12:0/1314827479 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faf50014030 con 0x7faf5c1053c0 2024-04-22T13:22:14.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.199+0000 7faf60c25640 1 -- 172.21.15.12:0/1314827479 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faf5c062390 con 0x7faf440a3c40 2024-04-22T13:22:14.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.206+0000 7faf4b7fe640 1 -- 172.21.15.12:0/1314827479 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258732 (secure 0 0 0) 0x7faf5c062390 con 0x7faf440a3c40 2024-04-22T13:22:14.206 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:22:14.208 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-22T13:22:14.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.209+0000 7faf60c25640 1 -- 172.21.15.12:0/1314827479 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7faf440a3c40 msgr2=0x7faf440a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:14.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.209+0000 7faf60c25640 1 --2- 172.21.15.12:0/1314827479 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7faf440a3c40 0x7faf440a6100 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7faf4c037040 tx=0x7faf4c005ef0 comp rx=0 tx=0).stop 2024-04-22T13:22:14.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.209+0000 7faf60c25640 1 -- 172.21.15.12:0/1314827479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf5c1053c0 msgr2=0x7faf5c198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:14.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.209+0000 7faf60c25640 1 --2- 172.21.15.12:0/1314827479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf5c1053c0 0x7faf5c198730 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faf5000c970 tx=0x7faf5000ce30 comp rx=0 tx=0).stop 2024-04-22T13:22:14.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.209+0000 7faf60c25640 1 -- 172.21.15.12:0/1314827479 shutdown_connections 2024-04-22T13:22:14.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.209+0000 7faf60c25640 1 --2- 172.21.15.12:0/1314827479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf5c107cf0 0x7faf5c19f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:14.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.209+0000 7faf60c25640 1 --2- 172.21.15.12:0/1314827479 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7faf440a3c40 0x7faf440a6100 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:14.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.209+0000 7faf60c25640 1 --2- 172.21.15.12:0/1314827479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf5c1053c0 0x7faf5c198730 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:14.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.209+0000 7faf60c25640 1 --2- 172.21.15.12:0/1314827479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf5c102a90 0x7faf5c1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:14.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.209+0000 7faf60c25640 1 -- 172.21.15.12:0/1314827479 >> 172.21.15.12:0/1314827479 conn(0x7faf5c0fc3e0 msgr2=0x7faf5c103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:14.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.209+0000 7faf60c25640 1 -- 172.21.15.12:0/1314827479 shutdown_connections 2024-04-22T13:22:14.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:14.209+0000 7faf60c25640 1 -- 172.21.15.12:0/1314827479 wait complete. 2024-04-22T13:22:14.218 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-04-22T13:22:12.362549+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595287,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":140,"num_read_kb":123,"num_write":299,"num_write_kb":2968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":221,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374468,"kb_used_data":12792,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567356,"statfs":{"total":1151940427776,"available":1151556972544,"internally_reserved":0,"allocated":13099008,"data_stored":7103716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4919,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":118,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001849"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.399900+0000","last_change":"2024-04-22T13:22:05.254388+0000","last_active":"2024-04-22T13:22:08.399900+0000","last_peered":"2024-04-22T13:22:08.399900+0000","last_clean":"2024-04-22T13:22:08.399900+0000","last_became_active":"2024-04-22T13:22:05.254042+0000","last_became_peered":"2024-04-22T13:22:05.254042+0000","last_unstale":"2024-04-22T13:22:08.399900+0000","last_undegraded":"2024-04-22T13:22:08.399900+0000","last_fullsized":"2024-04-22T13:22:08.399900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:07:56.139653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.237324+0000","last_change":"2024-04-22T13:22:05.247879+0000","last_active":"2024-04-22T13:22:06.237324+0000","last_peered":"2024-04-22T13:22:06.237324+0000","last_clean":"2024-04-22T13:22:06.237324+0000","last_became_active":"2024-04-22T13:22:05.247508+0000","last_became_peered":"2024-04-22T13:22:05.247508+0000","last_unstale":"2024-04-22T13:22:06.237324+0000","last_undegraded":"2024-04-22T13:22:06.237324+0000","last_fullsized":"2024-04-22T13:22:06.237324+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:55:51.928658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.237413+0000","last_change":"2024-04-22T13:22:05.257970+0000","last_active":"2024-04-22T13:22:06.237413+0000","last_peered":"2024-04-22T13:22:06.237413+0000","last_clean":"2024-04-22T13:22:06.237413+0000","last_became_active":"2024-04-22T13:22:05.257652+0000","last_became_peered":"2024-04-22T13:22:05.257652+0000","last_unstale":"2024-04-22T13:22:06.237413+0000","last_undegraded":"2024-04-22T13:22:06.237413+0000","last_fullsized":"2024-04-22T13:22:06.237413+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:42:17.476349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:07.872534+0000","last_change":"2024-04-22T13:22:05.249186+0000","last_active":"2024-04-22T13:22:07.872534+0000","last_peered":"2024-04-22T13:22:07.872534+0000","last_clean":"2024-04-22T13:22:07.872534+0000","last_became_active":"2024-04-22T13:22:05.248814+0000","last_became_peered":"2024-04-22T13:22:05.248814+0000","last_unstale":"2024-04-22T13:22:07.872534+0000","last_undegraded":"2024-04-22T13:22:07.872534+0000","last_fullsized":"2024-04-22T13:22:07.872534+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:57.450794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.237721+0000","last_change":"2024-04-22T13:22:05.251953+0000","last_active":"2024-04-22T13:22:06.237721+0000","last_peered":"2024-04-22T13:22:06.237721+0000","last_clean":"2024-04-22T13:22:06.237721+0000","last_became_active":"2024-04-22T13:22:05.251547+0000","last_became_peered":"2024-04-22T13:22:05.251547+0000","last_unstale":"2024-04-22T13:22:06.237721+0000","last_undegraded":"2024-04-22T13:22:06.237721+0000","last_fullsized":"2024-04-22T13:22:06.237721+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:36:34.809878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.259562+0000","last_change":"2024-04-22T13:22:05.250619+0000","last_active":"2024-04-22T13:22:08.259562+0000","last_peered":"2024-04-22T13:22:08.259562+0000","last_clean":"2024-04-22T13:22:08.259562+0000","last_became_active":"2024-04-22T13:22:05.250283+0000","last_became_peered":"2024-04-22T13:22:05.250283+0000","last_unstale":"2024-04-22T13:22:08.259562+0000","last_undegraded":"2024-04-22T13:22:08.259562+0000","last_fullsized":"2024-04-22T13:22:08.259562+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:43:10.029253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.242404+0000","last_change":"2024-04-22T13:22:05.250418+0000","last_active":"2024-04-22T13:22:06.242404+0000","last_peered":"2024-04-22T13:22:06.242404+0000","last_clean":"2024-04-22T13:22:06.242404+0000","last_became_active":"2024-04-22T13:22:05.250097+0000","last_became_peered":"2024-04-22T13:22:05.250097+0000","last_unstale":"2024-04-22T13:22:06.242404+0000","last_undegraded":"2024-04-22T13:22:06.242404+0000","last_fullsized":"2024-04-22T13:22:06.242404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:34:59.598950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.378148+0000","last_change":"2024-04-22T13:22:05.244762+0000","last_active":"2024-04-22T13:22:06.378148+0000","last_peered":"2024-04-22T13:22:06.378148+0000","last_clean":"2024-04-22T13:22:06.378148+0000","last_became_active":"2024-04-22T13:22:05.244430+0000","last_became_peered":"2024-04-22T13:22:05.244430+0000","last_unstale":"2024-04-22T13:22:06.378148+0000","last_undegraded":"2024-04-22T13:22:06.378148+0000","last_fullsized":"2024-04-22T13:22:06.378148+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:33:01.582851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.258784+0000","last_change":"2024-04-22T13:22:05.257596+0000","last_active":"2024-04-22T13:22:05.258784+0000","last_peered":"2024-04-22T13:22:05.258784+0000","last_clean":"2024-04-22T13:22:05.258784+0000","last_became_active":"2024-04-22T13:22:05.257277+0000","last_became_peered":"2024-04-22T13:22:05.257277+0000","last_unstale":"2024-04-22T13:22:05.258784+0000","last_undegraded":"2024-04-22T13:22:05.258784+0000","last_fullsized":"2024-04-22T13:22:05.258784+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:41:57.442610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.259432+0000","last_change":"2024-04-22T13:22:05.252177+0000","last_active":"2024-04-22T13:22:08.259432+0000","last_peered":"2024-04-22T13:22:08.259432+0000","last_clean":"2024-04-22T13:22:08.259432+0000","last_became_active":"2024-04-22T13:22:05.251794+0000","last_became_peered":"2024-04-22T13:22:05.251794+0000","last_unstale":"2024-04-22T13:22:08.259432+0000","last_undegraded":"2024-04-22T13:22:08.259432+0000","last_fullsized":"2024-04-22T13:22:08.259432+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:08:59.500540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.242883+0000","last_change":"2024-04-22T13:22:05.256209+0000","last_active":"2024-04-22T13:22:06.242883+0000","last_peered":"2024-04-22T13:22:06.242883+0000","last_clean":"2024-04-22T13:22:06.242883+0000","last_became_active":"2024-04-22T13:22:05.255504+0000","last_became_peered":"2024-04-22T13:22:05.255504+0000","last_unstale":"2024-04-22T13:22:06.242883+0000","last_undegraded":"2024-04-22T13:22:06.242883+0000","last_fullsized":"2024-04-22T13:22:06.242883+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:32:08.190921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:07.872481+0000","last_change":"2024-04-22T13:22:05.253939+0000","last_active":"2024-04-22T13:22:07.872481+0000","last_peered":"2024-04-22T13:22:07.872481+0000","last_clean":"2024-04-22T13:22:07.872481+0000","last_became_active":"2024-04-22T13:22:05.252939+0000","last_became_peered":"2024-04-22T13:22:05.252939+0000","last_unstale":"2024-04-22T13:22:07.872481+0000","last_undegraded":"2024-04-22T13:22:07.872481+0000","last_fullsized":"2024-04-22T13:22:07.872481+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:01:32.255047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.258921+0000","last_change":"2024-04-22T13:22:05.246667+0000","last_active":"2024-04-22T13:22:08.258921+0000","last_peered":"2024-04-22T13:22:08.258921+0000","last_clean":"2024-04-22T13:22:08.258921+0000","last_became_active":"2024-04-22T13:22:05.246365+0000","last_became_peered":"2024-04-22T13:22:05.246365+0000","last_unstale":"2024-04-22T13:22:08.258921+0000","last_undegraded":"2024-04-22T13:22:08.258921+0000","last_fullsized":"2024-04-22T13:22:08.258921+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:41:55.533430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.236961+0000","last_change":"2024-04-22T13:22:05.251982+0000","last_active":"2024-04-22T13:22:06.236961+0000","last_peered":"2024-04-22T13:22:06.236961+0000","last_clean":"2024-04-22T13:22:06.236961+0000","last_became_active":"2024-04-22T13:22:05.251575+0000","last_became_peered":"2024-04-22T13:22:05.251575+0000","last_unstale":"2024-04-22T13:22:06.236961+0000","last_undegraded":"2024-04-22T13:22:06.236961+0000","last_fullsized":"2024-04-22T13:22:06.236961+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:28:05.343140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.236910+0000","last_change":"2024-04-22T13:22:05.251056+0000","last_active":"2024-04-22T13:22:06.236910+0000","last_peered":"2024-04-22T13:22:06.236910+0000","last_clean":"2024-04-22T13:22:06.236910+0000","last_became_active":"2024-04-22T13:22:05.250652+0000","last_became_peered":"2024-04-22T13:22:05.250652+0000","last_unstale":"2024-04-22T13:22:06.236910+0000","last_undegraded":"2024-04-22T13:22:06.236910+0000","last_fullsized":"2024-04-22T13:22:06.236910+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:08:28.629965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.258372+0000","last_change":"2024-04-22T13:22:05.256877+0000","last_active":"2024-04-22T13:22:05.258372+0000","last_peered":"2024-04-22T13:22:05.258372+0000","last_clean":"2024-04-22T13:22:05.258372+0000","last_became_active":"2024-04-22T13:22:05.256074+0000","last_became_peered":"2024-04-22T13:22:05.256074+0000","last_unstale":"2024-04-22T13:22:05.258372+0000","last_undegraded":"2024-04-22T13:22:05.258372+0000","last_fullsized":"2024-04-22T13:22:05.258372+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:53:54.082142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.400172+0000","last_change":"2024-04-22T13:22:05.255598+0000","last_active":"2024-04-22T13:22:08.400172+0000","last_peered":"2024-04-22T13:22:08.400172+0000","last_clean":"2024-04-22T13:22:08.400172+0000","last_became_active":"2024-04-22T13:22:05.255239+0000","last_became_peered":"2024-04-22T13:22:05.255239+0000","last_unstale":"2024-04-22T13:22:08.400172+0000","last_undegraded":"2024-04-22T13:22:08.400172+0000","last_fullsized":"2024-04-22T13:22:08.400172+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:38:20.547882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.374676+0000","last_change":"2024-04-22T13:22:05.254232+0000","last_active":"2024-04-22T13:22:06.374676+0000","last_peered":"2024-04-22T13:22:06.374676+0000","last_clean":"2024-04-22T13:22:06.374676+0000","last_became_active":"2024-04-22T13:22:05.253907+0000","last_became_peered":"2024-04-22T13:22:05.253907+0000","last_unstale":"2024-04-22T13:22:06.374676+0000","last_undegraded":"2024-04-22T13:22:06.374676+0000","last_fullsized":"2024-04-22T13:22:06.374676+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:15:49.931112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:07.530160+0000","last_change":"2024-04-22T13:22:05.251518+0000","last_active":"2024-04-22T13:22:07.530160+0000","last_peered":"2024-04-22T13:22:07.530160+0000","last_clean":"2024-04-22T13:22:07.530160+0000","last_became_active":"2024-04-22T13:22:05.250478+0000","last_became_peered":"2024-04-22T13:22:05.250478+0000","last_unstale":"2024-04-22T13:22:07.530160+0000","last_undegraded":"2024-04-22T13:22:07.530160+0000","last_fullsized":"2024-04-22T13:22:07.530160+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:25:26.758692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:09.385907+0000","last_change":"2024-04-22T13:22:04.241365+0000","last_active":"2024-04-22T13:22:09.385907+0000","last_peered":"2024-04-22T13:22:09.385907+0000","last_clean":"2024-04-22T13:22:09.385907+0000","last_became_active":"2024-04-22T13:22:04.240822+0000","last_became_peered":"2024-04-22T13:22:04.240822+0000","last_unstale":"2024-04-22T13:22:09.385907+0000","last_undegraded":"2024-04-22T13:22:09.385907+0000","last_fullsized":"2024-04-22T13:22:09.385907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:20:23.928534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.378627+0000","last_change":"2024-04-22T13:22:05.249735+0000","last_active":"2024-04-22T13:22:06.378627+0000","last_peered":"2024-04-22T13:22:06.378627+0000","last_clean":"2024-04-22T13:22:06.378627+0000","last_became_active":"2024-04-22T13:22:05.249423+0000","last_became_peered":"2024-04-22T13:22:05.249423+0000","last_unstale":"2024-04-22T13:22:06.378627+0000","last_undegraded":"2024-04-22T13:22:06.378627+0000","last_fullsized":"2024-04-22T13:22:06.378627+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:38:49.425865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:07.295335+0000","last_change":"2024-04-22T13:22:04.248872+0000","last_active":"2024-04-22T13:22:07.295335+0000","last_peered":"2024-04-22T13:22:07.295335+0000","last_clean":"2024-04-22T13:22:07.295335+0000","last_became_active":"2024-04-22T13:22:04.248466+0000","last_became_peered":"2024-04-22T13:22:04.248466+0000","last_unstale":"2024-04-22T13:22:07.295335+0000","last_undegraded":"2024-04-22T13:22:07.295335+0000","last_fullsized":"2024-04-22T13:22:07.295335+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:38:10.871602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.237442+0000","last_change":"2024-04-22T13:22:05.258005+0000","last_active":"2024-04-22T13:22:06.237442+0000","last_peered":"2024-04-22T13:22:06.237442+0000","last_clean":"2024-04-22T13:22:06.237442+0000","last_became_active":"2024-04-22T13:22:05.257615+0000","last_became_peered":"2024-04-22T13:22:05.257615+0000","last_unstale":"2024-04-22T13:22:06.237442+0000","last_undegraded":"2024-04-22T13:22:06.237442+0000","last_fullsized":"2024-04-22T13:22:06.237442+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:21:18.030530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.238113+0000","last_change":"2024-04-22T13:22:04.246273+0000","last_active":"2024-04-22T13:22:06.238113+0000","last_peered":"2024-04-22T13:22:06.238113+0000","last_clean":"2024-04-22T13:22:06.238113+0000","last_became_active":"2024-04-22T13:22:04.245969+0000","last_became_peered":"2024-04-22T13:22:04.245969+0000","last_unstale":"2024-04-22T13:22:06.238113+0000","last_undegraded":"2024-04-22T13:22:06.238113+0000","last_fullsized":"2024-04-22T13:22:06.238113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:10.260192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.260743+0000","last_change":"2024-04-22T13:22:05.260299+0000","last_active":"2024-04-22T13:22:05.260743+0000","last_peered":"2024-04-22T13:22:05.260743+0000","last_clean":"2024-04-22T13:22:05.260743+0000","last_became_active":"2024-04-22T13:22:05.259634+0000","last_became_peered":"2024-04-22T13:22:05.259634+0000","last_unstale":"2024-04-22T13:22:05.260743+0000","last_undegraded":"2024-04-22T13:22:05.260743+0000","last_fullsized":"2024-04-22T13:22:05.260743+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:30:20.883785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.237885+0000","last_change":"2024-04-22T13:22:04.243843+0000","last_active":"2024-04-22T13:22:06.237885+0000","last_peered":"2024-04-22T13:22:06.237885+0000","last_clean":"2024-04-22T13:22:06.237885+0000","last_became_active":"2024-04-22T13:22:04.243465+0000","last_became_peered":"2024-04-22T13:22:04.243465+0000","last_unstale":"2024-04-22T13:22:06.237885+0000","last_undegraded":"2024-04-22T13:22:06.237885+0000","last_fullsized":"2024-04-22T13:22:06.237885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:48:36.431697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.260233+0000","last_change":"2024-04-22T13:22:05.259789+0000","last_active":"2024-04-22T13:22:05.260233+0000","last_peered":"2024-04-22T13:22:05.260233+0000","last_clean":"2024-04-22T13:22:05.260233+0000","last_became_active":"2024-04-22T13:22:05.259395+0000","last_became_peered":"2024-04-22T13:22:05.259395+0000","last_unstale":"2024-04-22T13:22:05.260233+0000","last_undegraded":"2024-04-22T13:22:05.260233+0000","last_fullsized":"2024-04-22T13:22:05.260233+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:35.446141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.234534+0000","last_change":"2024-04-22T13:22:04.240615+0000","last_active":"2024-04-22T13:22:05.234534+0000","last_peered":"2024-04-22T13:22:05.234534+0000","last_clean":"2024-04-22T13:22:05.234534+0000","last_became_active":"2024-04-22T13:22:04.240248+0000","last_became_peered":"2024-04-22T13:22:04.240248+0000","last_unstale":"2024-04-22T13:22:05.234534+0000","last_undegraded":"2024-04-22T13:22:05.234534+0000","last_fullsized":"2024-04-22T13:22:05.234534+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:55:05.948979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.237633+0000","last_change":"2024-04-22T13:22:05.253788+0000","last_active":"2024-04-22T13:22:06.237633+0000","last_peered":"2024-04-22T13:22:06.237633+0000","last_clean":"2024-04-22T13:22:06.237633+0000","last_became_active":"2024-04-22T13:22:05.252578+0000","last_became_peered":"2024-04-22T13:22:05.252578+0000","last_unstale":"2024-04-22T13:22:06.237633+0000","last_undegraded":"2024-04-22T13:22:06.237633+0000","last_fullsized":"2024-04-22T13:22:06.237633+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:04:11.769930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.259120+0000","last_change":"2024-04-22T13:22:04.241170+0000","last_active":"2024-04-22T13:22:08.259120+0000","last_peered":"2024-04-22T13:22:08.259120+0000","last_clean":"2024-04-22T13:22:08.259120+0000","last_became_active":"2024-04-22T13:22:04.240830+0000","last_became_peered":"2024-04-22T13:22:04.240830+0000","last_unstale":"2024-04-22T13:22:08.259120+0000","last_undegraded":"2024-04-22T13:22:08.259120+0000","last_fullsized":"2024-04-22T13:22:08.259120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:51:30.520613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.246158+0000","last_change":"2024-04-22T13:22:05.237900+0000","last_active":"2024-04-22T13:22:05.246158+0000","last_peered":"2024-04-22T13:22:05.246158+0000","last_clean":"2024-04-22T13:22:05.246158+0000","last_became_active":"2024-04-22T13:22:05.237507+0000","last_became_peered":"2024-04-22T13:22:05.237507+0000","last_unstale":"2024-04-22T13:22:05.246158+0000","last_undegraded":"2024-04-22T13:22:05.246158+0000","last_fullsized":"2024-04-22T13:22:05.246158+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:41:43.910207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:09.386142+0000","last_change":"2024-04-22T13:22:04.236078+0000","last_active":"2024-04-22T13:22:09.386142+0000","last_peered":"2024-04-22T13:22:09.386142+0000","last_clean":"2024-04-22T13:22:09.386142+0000","last_became_active":"2024-04-22T13:22:04.235710+0000","last_became_peered":"2024-04-22T13:22:04.235710+0000","last_unstale":"2024-04-22T13:22:09.386142+0000","last_undegraded":"2024-04-22T13:22:09.386142+0000","last_fullsized":"2024-04-22T13:22:09.386142+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:37:55.335237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.378097+0000","last_change":"2024-04-22T13:22:05.243152+0000","last_active":"2024-04-22T13:22:06.378097+0000","last_peered":"2024-04-22T13:22:06.378097+0000","last_clean":"2024-04-22T13:22:06.378097+0000","last_became_active":"2024-04-22T13:22:05.242772+0000","last_became_peered":"2024-04-22T13:22:05.242772+0000","last_unstale":"2024-04-22T13:22:06.378097+0000","last_undegraded":"2024-04-22T13:22:06.378097+0000","last_fullsized":"2024-04-22T13:22:06.378097+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:33:42.359679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.234359+0000","last_change":"2024-04-22T13:22:04.244381+0000","last_active":"2024-04-22T13:22:05.234359+0000","last_peered":"2024-04-22T13:22:05.234359+0000","last_clean":"2024-04-22T13:22:05.234359+0000","last_became_active":"2024-04-22T13:22:04.243746+0000","last_became_peered":"2024-04-22T13:22:04.243746+0000","last_unstale":"2024-04-22T13:22:05.234359+0000","last_undegraded":"2024-04-22T13:22:05.234359+0000","last_fullsized":"2024-04-22T13:22:05.234359+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:15:36.019930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.374485+0000","last_change":"2024-04-22T13:22:05.245394+0000","last_active":"2024-04-22T13:22:06.374485+0000","last_peered":"2024-04-22T13:22:06.374485+0000","last_clean":"2024-04-22T13:22:06.374485+0000","last_became_active":"2024-04-22T13:22:05.243931+0000","last_became_peered":"2024-04-22T13:22:05.243931+0000","last_unstale":"2024-04-22T13:22:06.374485+0000","last_undegraded":"2024-04-22T13:22:06.374485+0000","last_fullsized":"2024-04-22T13:22:06.374485+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:11:10.790842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.399036+0000","last_change":"2024-04-22T13:22:04.242583+0000","last_active":"2024-04-22T13:22:08.399036+0000","last_peered":"2024-04-22T13:22:08.399036+0000","last_clean":"2024-04-22T13:22:08.399036+0000","last_became_active":"2024-04-22T13:22:04.242261+0000","last_became_peered":"2024-04-22T13:22:04.242261+0000","last_unstale":"2024-04-22T13:22:08.399036+0000","last_undegraded":"2024-04-22T13:22:08.399036+0000","last_fullsized":"2024-04-22T13:22:08.399036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:37:47.618573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.258274+0000","last_change":"2024-04-22T13:22:05.255293+0000","last_active":"2024-04-22T13:22:05.258274+0000","last_peered":"2024-04-22T13:22:05.258274+0000","last_clean":"2024-04-22T13:22:05.258274+0000","last_became_active":"2024-04-22T13:22:05.253949+0000","last_became_peered":"2024-04-22T13:22:05.253949+0000","last_unstale":"2024-04-22T13:22:05.258274+0000","last_undegraded":"2024-04-22T13:22:05.258274+0000","last_fullsized":"2024-04-22T13:22:05.258274+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:02:23.442708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.397641+0000","last_change":"2024-04-22T13:22:04.241529+0000","last_active":"2024-04-22T13:22:08.397641+0000","last_peered":"2024-04-22T13:22:08.397641+0000","last_clean":"2024-04-22T13:22:08.397641+0000","last_became_active":"2024-04-22T13:22:04.241164+0000","last_became_peered":"2024-04-22T13:22:04.241164+0000","last_unstale":"2024-04-22T13:22:08.397641+0000","last_undegraded":"2024-04-22T13:22:08.397641+0000","last_fullsized":"2024-04-22T13:22:08.397641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:16:49.387181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.237429+0000","last_change":"2024-04-22T13:22:05.241623+0000","last_active":"2024-04-22T13:22:06.237429+0000","last_peered":"2024-04-22T13:22:06.237429+0000","last_clean":"2024-04-22T13:22:06.237429+0000","last_became_active":"2024-04-22T13:22:05.241150+0000","last_became_peered":"2024-04-22T13:22:05.241150+0000","last_unstale":"2024-04-22T13:22:06.237429+0000","last_undegraded":"2024-04-22T13:22:06.237429+0000","last_fullsized":"2024-04-22T13:22:06.237429+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:17:28.412366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'3","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.377801+0000","last_change":"2024-04-22T13:22:04.247852+0000","last_active":"2024-04-22T13:22:05.377801+0000","last_peered":"2024-04-22T13:22:05.377801+0000","last_clean":"2024-04-22T13:22:05.377801+0000","last_became_active":"2024-04-22T13:22:04.247550+0000","last_became_peered":"2024-04-22T13:22:04.247550+0000","last_unstale":"2024-04-22T13:22:05.377801+0000","last_undegraded":"2024-04-22T13:22:05.377801+0000","last_fullsized":"2024-04-22T13:22:05.377801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:55:01.928435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.260640+0000","last_change":"2024-04-22T13:22:05.260050+0000","last_active":"2024-04-22T13:22:05.260640+0000","last_peered":"2024-04-22T13:22:05.260640+0000","last_clean":"2024-04-22T13:22:05.260640+0000","last_became_active":"2024-04-22T13:22:05.259098+0000","last_became_peered":"2024-04-22T13:22:05.259098+0000","last_unstale":"2024-04-22T13:22:05.260640+0000","last_undegraded":"2024-04-22T13:22:05.260640+0000","last_fullsized":"2024-04-22T13:22:05.260640+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:24:52.552016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'8","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.398117+0000","last_change":"2024-04-22T13:22:04.240233+0000","last_active":"2024-04-22T13:22:08.398117+0000","last_peered":"2024-04-22T13:22:08.398117+0000","last_clean":"2024-04-22T13:22:08.398117+0000","last_became_active":"2024-04-22T13:22:04.239890+0000","last_became_peered":"2024-04-22T13:22:04.239890+0000","last_unstale":"2024-04-22T13:22:08.398117+0000","last_undegraded":"2024-04-22T13:22:08.398117+0000","last_fullsized":"2024-04-22T13:22:08.398117+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:04:21.547422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.237968+0000","last_change":"2024-04-22T13:22:05.257756+0000","last_active":"2024-04-22T13:22:06.237968+0000","last_peered":"2024-04-22T13:22:06.237968+0000","last_clean":"2024-04-22T13:22:06.237968+0000","last_became_active":"2024-04-22T13:22:05.257442+0000","last_became_peered":"2024-04-22T13:22:05.257442+0000","last_unstale":"2024-04-22T13:22:06.237968+0000","last_undegraded":"2024-04-22T13:22:06.237968+0000","last_fullsized":"2024-04-22T13:22:06.237968+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:16:53.465689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:11.402777+0000","last_change":"2024-04-22T13:22:04.240815+0000","last_active":"2024-04-22T13:22:11.402777+0000","last_peered":"2024-04-22T13:22:11.402777+0000","last_clean":"2024-04-22T13:22:11.402777+0000","last_became_active":"2024-04-22T13:22:04.240218+0000","last_became_peered":"2024-04-22T13:22:04.240218+0000","last_unstale":"2024-04-22T13:22:11.402777+0000","last_undegraded":"2024-04-22T13:22:11.402777+0000","last_fullsized":"2024-04-22T13:22:11.402777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:29:29.148764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.258649+0000","last_change":"2024-04-22T13:22:05.248797+0000","last_active":"2024-04-22T13:22:08.258649+0000","last_peered":"2024-04-22T13:22:08.258649+0000","last_clean":"2024-04-22T13:22:08.258649+0000","last_became_active":"2024-04-22T13:22:05.248435+0000","last_became_peered":"2024-04-22T13:22:05.248435+0000","last_unstale":"2024-04-22T13:22:08.258649+0000","last_undegraded":"2024-04-22T13:22:08.258649+0000","last_fullsized":"2024-04-22T13:22:08.258649+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:32:31.970494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.234419+0000","last_change":"2024-04-22T13:22:04.245303+0000","last_active":"2024-04-22T13:22:05.234419+0000","last_peered":"2024-04-22T13:22:05.234419+0000","last_clean":"2024-04-22T13:22:05.234419+0000","last_became_active":"2024-04-22T13:22:04.244903+0000","last_became_peered":"2024-04-22T13:22:04.244903+0000","last_unstale":"2024-04-22T13:22:05.234419+0000","last_undegraded":"2024-04-22T13:22:05.234419+0000","last_fullsized":"2024-04-22T13:22:05.234419+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:34:32.174858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.242794+0000","last_change":"2024-04-22T13:22:05.246318+0000","last_active":"2024-04-22T13:22:06.242794+0000","last_peered":"2024-04-22T13:22:06.242794+0000","last_clean":"2024-04-22T13:22:06.242794+0000","last_became_active":"2024-04-22T13:22:05.245899+0000","last_became_peered":"2024-04-22T13:22:05.245899+0000","last_unstale":"2024-04-22T13:22:06.242794+0000","last_undegraded":"2024-04-22T13:22:06.242794+0000","last_fullsized":"2024-04-22T13:22:06.242794+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:27:42.789058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:10.392090+0000","last_change":"2024-04-22T13:22:04.240612+0000","last_active":"2024-04-22T13:22:10.392090+0000","last_peered":"2024-04-22T13:22:10.392090+0000","last_clean":"2024-04-22T13:22:10.392090+0000","last_became_active":"2024-04-22T13:22:04.240290+0000","last_became_peered":"2024-04-22T13:22:04.240290+0000","last_unstale":"2024-04-22T13:22:10.392090+0000","last_undegraded":"2024-04-22T13:22:10.392090+0000","last_fullsized":"2024-04-22T13:22:10.392090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:00:50.285553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.242390+0000","last_change":"2024-04-22T13:22:05.253205+0000","last_active":"2024-04-22T13:22:06.242390+0000","last_peered":"2024-04-22T13:22:06.242390+0000","last_clean":"2024-04-22T13:22:06.242390+0000","last_became_active":"2024-04-22T13:22:05.252865+0000","last_became_peered":"2024-04-22T13:22:05.252865+0000","last_unstale":"2024-04-22T13:22:06.242390+0000","last_undegraded":"2024-04-22T13:22:06.242390+0000","last_fullsized":"2024-04-22T13:22:06.242390+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:19:12.482371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'8","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.237479+0000","last_change":"2024-04-22T13:22:04.248938+0000","last_active":"2024-04-22T13:22:06.237479+0000","last_peered":"2024-04-22T13:22:06.237479+0000","last_clean":"2024-04-22T13:22:06.237479+0000","last_became_active":"2024-04-22T13:22:04.248440+0000","last_became_peered":"2024-04-22T13:22:04.248440+0000","last_unstale":"2024-04-22T13:22:06.237479+0000","last_undegraded":"2024-04-22T13:22:06.237479+0000","last_fullsized":"2024-04-22T13:22:06.237479+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:46:50.928067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.256079+0000","last_change":"2024-04-22T13:22:05.254664+0000","last_active":"2024-04-22T13:22:05.256079+0000","last_peered":"2024-04-22T13:22:05.256079+0000","last_clean":"2024-04-22T13:22:05.256079+0000","last_became_active":"2024-04-22T13:22:05.254304+0000","last_became_peered":"2024-04-22T13:22:05.254304+0000","last_unstale":"2024-04-22T13:22:05.256079+0000","last_undegraded":"2024-04-22T13:22:05.256079+0000","last_fullsized":"2024-04-22T13:22:05.256079+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:18:57.622836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.238119+0000","last_change":"2024-04-22T13:22:04.244128+0000","last_active":"2024-04-22T13:22:06.238119+0000","last_peered":"2024-04-22T13:22:06.238119+0000","last_clean":"2024-04-22T13:22:06.238119+0000","last_became_active":"2024-04-22T13:22:04.243738+0000","last_became_peered":"2024-04-22T13:22:04.243738+0000","last_unstale":"2024-04-22T13:22:06.238119+0000","last_undegraded":"2024-04-22T13:22:06.238119+0000","last_fullsized":"2024-04-22T13:22:06.238119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:23:52.220584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.399335+0000","last_change":"2024-04-22T13:22:05.250910+0000","last_active":"2024-04-22T13:22:08.399335+0000","last_peered":"2024-04-22T13:22:08.399335+0000","last_clean":"2024-04-22T13:22:08.399335+0000","last_became_active":"2024-04-22T13:22:05.250410+0000","last_became_peered":"2024-04-22T13:22:05.250410+0000","last_unstale":"2024-04-22T13:22:08.399335+0000","last_undegraded":"2024-04-22T13:22:08.399335+0000","last_fullsized":"2024-04-22T13:22:08.399335+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:04:25.492677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.374499+0000","last_change":"2024-04-22T13:22:04.245889+0000","last_active":"2024-04-22T13:22:06.374499+0000","last_peered":"2024-04-22T13:22:06.374499+0000","last_clean":"2024-04-22T13:22:06.374499+0000","last_became_active":"2024-04-22T13:22:04.245474+0000","last_became_peered":"2024-04-22T13:22:04.245474+0000","last_unstale":"2024-04-22T13:22:06.374499+0000","last_undegraded":"2024-04-22T13:22:06.374499+0000","last_fullsized":"2024-04-22T13:22:06.374499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:22:12.626950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.259356+0000","last_change":"2024-04-22T13:22:05.250978+0000","last_active":"2024-04-22T13:22:08.259356+0000","last_peered":"2024-04-22T13:22:08.259356+0000","last_clean":"2024-04-22T13:22:08.259356+0000","last_became_active":"2024-04-22T13:22:05.250463+0000","last_became_peered":"2024-04-22T13:22:05.250463+0000","last_unstale":"2024-04-22T13:22:08.259356+0000","last_undegraded":"2024-04-22T13:22:08.259356+0000","last_fullsized":"2024-04-22T13:22:08.259356+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:45:18.591072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.242578+0000","last_change":"2024-04-22T13:22:04.242503+0000","last_active":"2024-04-22T13:22:06.242578+0000","last_peered":"2024-04-22T13:22:06.242578+0000","last_clean":"2024-04-22T13:22:06.242578+0000","last_became_active":"2024-04-22T13:22:04.242143+0000","last_became_peered":"2024-04-22T13:22:04.242143+0000","last_unstale":"2024-04-22T13:22:06.242578+0000","last_undegraded":"2024-04-22T13:22:06.242578+0000","last_fullsized":"2024-04-22T13:22:06.242578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:40.236377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.259848+0000","last_change":"2024-04-22T13:22:05.252387+0000","last_active":"2024-04-22T13:22:08.259848+0000","last_peered":"2024-04-22T13:22:08.259848+0000","last_clean":"2024-04-22T13:22:08.259848+0000","last_became_active":"2024-04-22T13:22:05.252033+0000","last_became_peered":"2024-04-22T13:22:05.252033+0000","last_unstale":"2024-04-22T13:22:08.259848+0000","last_undegraded":"2024-04-22T13:22:08.259848+0000","last_fullsized":"2024-04-22T13:22:08.259848+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:55:01.791283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.249496+0000","last_change":"2024-04-22T13:22:05.248394+0000","last_active":"2024-04-22T13:22:05.249496+0000","last_peered":"2024-04-22T13:22:05.249496+0000","last_clean":"2024-04-22T13:22:05.249496+0000","last_became_active":"2024-04-22T13:22:05.248024+0000","last_became_peered":"2024-04-22T13:22:05.248024+0000","last_unstale":"2024-04-22T13:22:05.249496+0000","last_undegraded":"2024-04-22T13:22:05.249496+0000","last_fullsized":"2024-04-22T13:22:05.249496+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:35:39.136654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'4","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:09.386308+0000","last_change":"2024-04-22T13:22:04.247692+0000","last_active":"2024-04-22T13:22:09.386308+0000","last_peered":"2024-04-22T13:22:09.386308+0000","last_clean":"2024-04-22T13:22:09.386308+0000","last_became_active":"2024-04-22T13:22:04.247345+0000","last_became_peered":"2024-04-22T13:22:04.247345+0000","last_unstale":"2024-04-22T13:22:09.386308+0000","last_undegraded":"2024-04-22T13:22:09.386308+0000","last_fullsized":"2024-04-22T13:22:09.386308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:07:49.803248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.378466+0000","last_change":"2024-04-22T13:22:05.241022+0000","last_active":"2024-04-22T13:22:06.378466+0000","last_peered":"2024-04-22T13:22:06.378466+0000","last_clean":"2024-04-22T13:22:06.378466+0000","last_became_active":"2024-04-22T13:22:05.240635+0000","last_became_peered":"2024-04-22T13:22:05.240635+0000","last_unstale":"2024-04-22T13:22:06.378466+0000","last_undegraded":"2024-04-22T13:22:06.378466+0000","last_fullsized":"2024-04-22T13:22:06.378466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:50:02.012260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:10.391925+0000","last_change":"2024-04-22T13:22:04.236817+0000","last_active":"2024-04-22T13:22:10.391925+0000","last_peered":"2024-04-22T13:22:10.391925+0000","last_clean":"2024-04-22T13:22:10.391925+0000","last_became_active":"2024-04-22T13:22:04.236484+0000","last_became_peered":"2024-04-22T13:22:04.236484+0000","last_unstale":"2024-04-22T13:22:10.391925+0000","last_undegraded":"2024-04-22T13:22:10.391925+0000","last_fullsized":"2024-04-22T13:22:10.391925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:27:41.053327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":350,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.242350+0000","last_change":"2024-04-22T13:20:53.094123+0000","last_active":"2024-04-22T13:22:06.242350+0000","last_peered":"2024-04-22T13:22:06.242350+0000","last_clean":"2024-04-22T13:22:06.242350+0000","last_became_active":"2024-04-22T13:20:53.093664+0000","last_became_peered":"2024-04-22T13:20:53.093664+0000","last_unstale":"2024-04-22T13:22:06.242350+0000","last_undegraded":"2024-04-22T13:22:06.242350+0000","last_fullsized":"2024-04-22T13:22:06.242350+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:17:33.201786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:17:33.201786+0000","last_clean_scrub_stamp":"2024-04-22T13:17:33.201786+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:46:58.986399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.258555+0000","last_change":"2024-04-22T13:22:05.247766+0000","last_active":"2024-04-22T13:22:08.258555+0000","last_peered":"2024-04-22T13:22:08.258555+0000","last_clean":"2024-04-22T13:22:08.258555+0000","last_became_active":"2024-04-22T13:22:05.247362+0000","last_became_peered":"2024-04-22T13:22:05.247362+0000","last_unstale":"2024-04-22T13:22:08.258555+0000","last_undegraded":"2024-04-22T13:22:08.258555+0000","last_fullsized":"2024-04-22T13:22:08.258555+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:25:00.126638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:09.385197+0000","last_change":"2024-04-22T13:22:04.243493+0000","last_active":"2024-04-22T13:22:09.385197+0000","last_peered":"2024-04-22T13:22:09.385197+0000","last_clean":"2024-04-22T13:22:09.385197+0000","last_became_active":"2024-04-22T13:22:04.243190+0000","last_became_peered":"2024-04-22T13:22:04.243190+0000","last_unstale":"2024-04-22T13:22:09.385197+0000","last_undegraded":"2024-04-22T13:22:09.385197+0000","last_fullsized":"2024-04-22T13:22:09.385197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:15:09.025152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.238040+0000","last_change":"2024-04-22T13:22:04.249222+0000","last_active":"2024-04-22T13:22:06.238040+0000","last_peered":"2024-04-22T13:22:06.238040+0000","last_clean":"2024-04-22T13:22:06.238040+0000","last_became_active":"2024-04-22T13:22:04.248926+0000","last_became_peered":"2024-04-22T13:22:04.248926+0000","last_unstale":"2024-04-22T13:22:06.238040+0000","last_undegraded":"2024-04-22T13:22:06.238040+0000","last_fullsized":"2024-04-22T13:22:06.238040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:33:50.168426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.258780+0000","last_change":"2024-04-22T13:22:05.248199+0000","last_active":"2024-04-22T13:22:08.258780+0000","last_peered":"2024-04-22T13:22:08.258780+0000","last_clean":"2024-04-22T13:22:08.258780+0000","last_became_active":"2024-04-22T13:22:05.247873+0000","last_became_peered":"2024-04-22T13:22:05.247873+0000","last_unstale":"2024-04-22T13:22:08.258780+0000","last_undegraded":"2024-04-22T13:22:08.258780+0000","last_fullsized":"2024-04-22T13:22:08.258780+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:40:29.353901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:09.384236+0000","last_change":"2024-04-22T13:22:04.243474+0000","last_active":"2024-04-22T13:22:09.384236+0000","last_peered":"2024-04-22T13:22:09.384236+0000","last_clean":"2024-04-22T13:22:09.384236+0000","last_became_active":"2024-04-22T13:22:04.243151+0000","last_became_peered":"2024-04-22T13:22:04.243151+0000","last_unstale":"2024-04-22T13:22:09.384236+0000","last_undegraded":"2024-04-22T13:22:09.384236+0000","last_fullsized":"2024-04-22T13:22:09.384236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:18:25.140715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.378634+0000","last_change":"2024-04-22T13:22:05.243120+0000","last_active":"2024-04-22T13:22:06.378634+0000","last_peered":"2024-04-22T13:22:06.378634+0000","last_clean":"2024-04-22T13:22:06.378634+0000","last_became_active":"2024-04-22T13:22:05.242726+0000","last_became_peered":"2024-04-22T13:22:05.242726+0000","last_unstale":"2024-04-22T13:22:06.378634+0000","last_undegraded":"2024-04-22T13:22:06.378634+0000","last_fullsized":"2024-04-22T13:22:06.378634+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:06:16.087884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.400718+0000","last_change":"2024-04-22T13:22:04.239672+0000","last_active":"2024-04-22T13:22:08.400718+0000","last_peered":"2024-04-22T13:22:08.400718+0000","last_clean":"2024-04-22T13:22:08.400718+0000","last_became_active":"2024-04-22T13:22:04.238776+0000","last_became_peered":"2024-04-22T13:22:04.238776+0000","last_unstale":"2024-04-22T13:22:08.400718+0000","last_undegraded":"2024-04-22T13:22:08.400718+0000","last_fullsized":"2024-04-22T13:22:08.400718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:43:21.752547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.237383+0000","last_change":"2024-04-22T13:22:05.242089+0000","last_active":"2024-04-22T13:22:06.237383+0000","last_peered":"2024-04-22T13:22:06.237383+0000","last_clean":"2024-04-22T13:22:06.237383+0000","last_became_active":"2024-04-22T13:22:05.241681+0000","last_became_peered":"2024-04-22T13:22:05.241681+0000","last_unstale":"2024-04-22T13:22:06.237383+0000","last_undegraded":"2024-04-22T13:22:06.237383+0000","last_fullsized":"2024-04-22T13:22:06.237383+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:58:49.906188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.250782+0000","last_change":"2024-04-22T13:22:04.248647+0000","last_active":"2024-04-22T13:22:05.250782+0000","last_peered":"2024-04-22T13:22:05.250782+0000","last_clean":"2024-04-22T13:22:05.250782+0000","last_became_active":"2024-04-22T13:22:04.248408+0000","last_became_peered":"2024-04-22T13:22:04.248408+0000","last_unstale":"2024-04-22T13:22:05.250782+0000","last_undegraded":"2024-04-22T13:22:05.250782+0000","last_fullsized":"2024-04-22T13:22:05.250782+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:16:26.244223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.237585+0000","last_change":"2024-04-22T13:22:05.250120+0000","last_active":"2024-04-22T13:22:06.237585+0000","last_peered":"2024-04-22T13:22:06.237585+0000","last_clean":"2024-04-22T13:22:06.237585+0000","last_became_active":"2024-04-22T13:22:05.247936+0000","last_became_peered":"2024-04-22T13:22:05.247936+0000","last_unstale":"2024-04-22T13:22:06.237585+0000","last_undegraded":"2024-04-22T13:22:06.237585+0000","last_fullsized":"2024-04-22T13:22:06.237585+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:27:38.437918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:11.401215+0000","last_change":"2024-04-22T13:22:04.244026+0000","last_active":"2024-04-22T13:22:11.401215+0000","last_peered":"2024-04-22T13:22:11.401215+0000","last_clean":"2024-04-22T13:22:11.401215+0000","last_became_active":"2024-04-22T13:22:04.243518+0000","last_became_peered":"2024-04-22T13:22:04.243518+0000","last_unstale":"2024-04-22T13:22:11.401215+0000","last_undegraded":"2024-04-22T13:22:11.401215+0000","last_fullsized":"2024-04-22T13:22:11.401215+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:54:48.791874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.398466+0000","last_change":"2024-04-22T13:22:05.252674+0000","last_active":"2024-04-22T13:22:08.398466+0000","last_peered":"2024-04-22T13:22:08.398466+0000","last_clean":"2024-04-22T13:22:08.398466+0000","last_became_active":"2024-04-22T13:22:05.251087+0000","last_became_peered":"2024-04-22T13:22:05.251087+0000","last_unstale":"2024-04-22T13:22:08.398466+0000","last_undegraded":"2024-04-22T13:22:08.398466+0000","last_fullsized":"2024-04-22T13:22:08.398466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:20:06.873412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:10.392209+0000","last_change":"2024-04-22T13:22:04.244409+0000","last_active":"2024-04-22T13:22:10.392209+0000","last_peered":"2024-04-22T13:22:10.392209+0000","last_clean":"2024-04-22T13:22:10.392209+0000","last_became_active":"2024-04-22T13:22:04.244104+0000","last_became_peered":"2024-04-22T13:22:04.244104+0000","last_unstale":"2024-04-22T13:22:10.392209+0000","last_undegraded":"2024-04-22T13:22:10.392209+0000","last_fullsized":"2024-04-22T13:22:10.392209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:39:14.714755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.378315+0000","last_change":"2024-04-22T13:22:05.245730+0000","last_active":"2024-04-22T13:22:06.378315+0000","last_peered":"2024-04-22T13:22:06.378315+0000","last_clean":"2024-04-22T13:22:06.378315+0000","last_became_active":"2024-04-22T13:22:05.245366+0000","last_became_peered":"2024-04-22T13:22:05.245366+0000","last_unstale":"2024-04-22T13:22:06.378315+0000","last_undegraded":"2024-04-22T13:22:06.378315+0000","last_fullsized":"2024-04-22T13:22:06.378315+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:31.935258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.237657+0000","last_change":"2024-04-22T13:22:04.246683+0000","last_active":"2024-04-22T13:22:06.237657+0000","last_peered":"2024-04-22T13:22:06.237657+0000","last_clean":"2024-04-22T13:22:06.237657+0000","last_became_active":"2024-04-22T13:22:04.246372+0000","last_became_peered":"2024-04-22T13:22:04.246372+0000","last_unstale":"2024-04-22T13:22:06.237657+0000","last_undegraded":"2024-04-22T13:22:06.237657+0000","last_fullsized":"2024-04-22T13:22:06.237657+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:01:13.464077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.242859+0000","last_change":"2024-04-22T13:22:05.242146+0000","last_active":"2024-04-22T13:22:06.242859+0000","last_peered":"2024-04-22T13:22:06.242859+0000","last_clean":"2024-04-22T13:22:06.242859+0000","last_became_active":"2024-04-22T13:22:05.241802+0000","last_became_peered":"2024-04-22T13:22:05.241802+0000","last_unstale":"2024-04-22T13:22:06.242859+0000","last_undegraded":"2024-04-22T13:22:06.242859+0000","last_fullsized":"2024-04-22T13:22:06.242859+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:49:49.002123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:10.392663+0000","last_change":"2024-04-22T13:22:04.248884+0000","last_active":"2024-04-22T13:22:10.392663+0000","last_peered":"2024-04-22T13:22:10.392663+0000","last_clean":"2024-04-22T13:22:10.392663+0000","last_became_active":"2024-04-22T13:22:04.248393+0000","last_became_peered":"2024-04-22T13:22:04.248393+0000","last_unstale":"2024-04-22T13:22:10.392663+0000","last_undegraded":"2024-04-22T13:22:10.392663+0000","last_fullsized":"2024-04-22T13:22:10.392663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:36:14.687457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.242948+0000","last_change":"2024-04-22T13:22:05.256881+0000","last_active":"2024-04-22T13:22:06.242948+0000","last_peered":"2024-04-22T13:22:06.242948+0000","last_clean":"2024-04-22T13:22:06.242948+0000","last_became_active":"2024-04-22T13:22:05.255967+0000","last_became_peered":"2024-04-22T13:22:05.255967+0000","last_unstale":"2024-04-22T13:22:06.242948+0000","last_undegraded":"2024-04-22T13:22:06.242948+0000","last_fullsized":"2024-04-22T13:22:06.242948+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:24:21.137344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.234442+0000","last_change":"2024-04-22T13:22:04.247303+0000","last_active":"2024-04-22T13:22:05.234442+0000","last_peered":"2024-04-22T13:22:05.234442+0000","last_clean":"2024-04-22T13:22:05.234442+0000","last_became_active":"2024-04-22T13:22:04.246742+0000","last_became_peered":"2024-04-22T13:22:04.246742+0000","last_unstale":"2024-04-22T13:22:05.234442+0000","last_undegraded":"2024-04-22T13:22:05.234442+0000","last_fullsized":"2024-04-22T13:22:05.234442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:45:03.189042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.242622+0000","last_change":"2024-04-22T13:22:05.251754+0000","last_active":"2024-04-22T13:22:06.242622+0000","last_peered":"2024-04-22T13:22:06.242622+0000","last_clean":"2024-04-22T13:22:06.242622+0000","last_became_active":"2024-04-22T13:22:05.251155+0000","last_became_peered":"2024-04-22T13:22:05.251155+0000","last_unstale":"2024-04-22T13:22:06.242622+0000","last_undegraded":"2024-04-22T13:22:06.242622+0000","last_fullsized":"2024-04-22T13:22:06.242622+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:08:46.682600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.258891+0000","last_change":"2024-04-22T13:22:04.243382+0000","last_active":"2024-04-22T13:22:08.258891+0000","last_peered":"2024-04-22T13:22:08.258891+0000","last_clean":"2024-04-22T13:22:08.258891+0000","last_became_active":"2024-04-22T13:22:04.243055+0000","last_became_peered":"2024-04-22T13:22:04.243055+0000","last_unstale":"2024-04-22T13:22:08.258891+0000","last_undegraded":"2024-04-22T13:22:08.258891+0000","last_fullsized":"2024-04-22T13:22:08.258891+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:19:58.379645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.242846+0000","last_change":"2024-04-22T13:22:05.249527+0000","last_active":"2024-04-22T13:22:06.242846+0000","last_peered":"2024-04-22T13:22:06.242846+0000","last_clean":"2024-04-22T13:22:06.242846+0000","last_became_active":"2024-04-22T13:22:05.249207+0000","last_became_peered":"2024-04-22T13:22:05.249207+0000","last_unstale":"2024-04-22T13:22:06.242846+0000","last_undegraded":"2024-04-22T13:22:06.242846+0000","last_fullsized":"2024-04-22T13:22:06.242846+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:50:02.755288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.398479+0000","last_change":"2024-04-22T13:22:05.257812+0000","last_active":"2024-04-22T13:22:08.398479+0000","last_peered":"2024-04-22T13:22:08.398479+0000","last_clean":"2024-04-22T13:22:08.398479+0000","last_became_active":"2024-04-22T13:22:05.257470+0000","last_became_peered":"2024-04-22T13:22:05.257470+0000","last_unstale":"2024-04-22T13:22:08.398479+0000","last_undegraded":"2024-04-22T13:22:08.398479+0000","last_fullsized":"2024-04-22T13:22:08.398479+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:47:25.343965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:07.529883+0000","last_change":"2024-04-22T13:22:05.253839+0000","last_active":"2024-04-22T13:22:07.529883+0000","last_peered":"2024-04-22T13:22:07.529883+0000","last_clean":"2024-04-22T13:22:07.529883+0000","last_became_active":"2024-04-22T13:22:05.253499+0000","last_became_peered":"2024-04-22T13:22:05.253499+0000","last_unstale":"2024-04-22T13:22:07.529883+0000","last_undegraded":"2024-04-22T13:22:07.529883+0000","last_fullsized":"2024-04-22T13:22:07.529883+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:40:50.456951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:07.529803+0000","last_change":"2024-04-22T13:22:05.247882+0000","last_active":"2024-04-22T13:22:07.529803+0000","last_peered":"2024-04-22T13:22:07.529803+0000","last_clean":"2024-04-22T13:22:07.529803+0000","last_became_active":"2024-04-22T13:22:05.247525+0000","last_became_peered":"2024-04-22T13:22:05.247525+0000","last_unstale":"2024-04-22T13:22:07.529803+0000","last_undegraded":"2024-04-22T13:22:07.529803+0000","last_fullsized":"2024-04-22T13:22:07.529803+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:46:04.349076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.237060+0000","last_change":"2024-04-22T13:22:05.243716+0000","last_active":"2024-04-22T13:22:06.237060+0000","last_peered":"2024-04-22T13:22:06.237060+0000","last_clean":"2024-04-22T13:22:06.237060+0000","last_became_active":"2024-04-22T13:22:05.243397+0000","last_became_peered":"2024-04-22T13:22:05.243397+0000","last_unstale":"2024-04-22T13:22:06.237060+0000","last_undegraded":"2024-04-22T13:22:06.237060+0000","last_fullsized":"2024-04-22T13:22:06.237060+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:16:21.236339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.258905+0000","last_change":"2024-04-22T13:22:05.246042+0000","last_active":"2024-04-22T13:22:08.258905+0000","last_peered":"2024-04-22T13:22:08.258905+0000","last_clean":"2024-04-22T13:22:08.258905+0000","last_became_active":"2024-04-22T13:22:05.245669+0000","last_became_peered":"2024-04-22T13:22:05.245669+0000","last_unstale":"2024-04-22T13:22:08.258905+0000","last_undegraded":"2024-04-22T13:22:08.258905+0000","last_fullsized":"2024-04-22T13:22:08.258905+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:49:05.539641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.399059+0000","last_change":"2024-04-22T13:22:05.257009+0000","last_active":"2024-04-22T13:22:08.399059+0000","last_peered":"2024-04-22T13:22:08.399059+0000","last_clean":"2024-04-22T13:22:08.399059+0000","last_became_active":"2024-04-22T13:22:05.254327+0000","last_became_peered":"2024-04-22T13:22:05.254327+0000","last_unstale":"2024-04-22T13:22:08.399059+0000","last_undegraded":"2024-04-22T13:22:08.399059+0000","last_fullsized":"2024-04-22T13:22:08.399059+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:29:12.482525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:07.530313+0000","last_change":"2024-04-22T13:22:05.260575+0000","last_active":"2024-04-22T13:22:07.530313+0000","last_peered":"2024-04-22T13:22:07.530313+0000","last_clean":"2024-04-22T13:22:07.530313+0000","last_became_active":"2024-04-22T13:22:05.260181+0000","last_became_peered":"2024-04-22T13:22:05.260181+0000","last_unstale":"2024-04-22T13:22:07.530313+0000","last_undegraded":"2024-04-22T13:22:07.530313+0000","last_fullsized":"2024-04-22T13:22:07.530313+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:49:20.064368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.378496+0000","last_change":"2024-04-22T13:22:05.248199+0000","last_active":"2024-04-22T13:22:06.378496+0000","last_peered":"2024-04-22T13:22:06.378496+0000","last_clean":"2024-04-22T13:22:06.378496+0000","last_became_active":"2024-04-22T13:22:05.247858+0000","last_became_peered":"2024-04-22T13:22:05.247858+0000","last_unstale":"2024-04-22T13:22:06.378496+0000","last_undegraded":"2024-04-22T13:22:06.378496+0000","last_fullsized":"2024-04-22T13:22:06.378496+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:49:21.228105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.246736+0000","last_change":"2024-04-22T13:22:05.239137+0000","last_active":"2024-04-22T13:22:05.246736+0000","last_peered":"2024-04-22T13:22:05.246736+0000","last_clean":"2024-04-22T13:22:05.246736+0000","last_became_active":"2024-04-22T13:22:05.238807+0000","last_became_peered":"2024-04-22T13:22:05.238807+0000","last_unstale":"2024-04-22T13:22:05.246736+0000","last_undegraded":"2024-04-22T13:22:05.246736+0000","last_fullsized":"2024-04-22T13:22:05.246736+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:39:02.950022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:06.378236+0000","last_change":"2024-04-22T13:22:05.249080+0000","last_active":"2024-04-22T13:22:06.378236+0000","last_peered":"2024-04-22T13:22:06.378236+0000","last_clean":"2024-04-22T13:22:06.378236+0000","last_became_active":"2024-04-22T13:22:05.248733+0000","last_became_peered":"2024-04-22T13:22:05.248733+0000","last_unstale":"2024-04-22T13:22:06.378236+0000","last_undegraded":"2024-04-22T13:22:06.378236+0000","last_fullsized":"2024-04-22T13:22:06.378236+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:58.735941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-22T13:22:05.258236+0000","last_change":"2024-04-22T13:22:05.255490+0000","last_active":"2024-04-22T13:22:05.258236+0000","last_peered":"2024-04-22T13:22:05.258236+0000","last_clean":"2024-04-22T13:22:05.258236+0000","last_became_active":"2024-04-22T13:22:05.253644+0000","last_became_peered":"2024-04-22T13:22:05.253644+0000","last_unstale":"2024-04-22T13:22:05.258236+0000","last_undegraded":"2024-04-22T13:22:05.258236+0000","last_fullsized":"2024-04-22T13:22:05.258236+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:42:05.958106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.258994+0000","last_change":"2024-04-22T13:22:05.240988+0000","last_active":"2024-04-22T13:22:08.258994+0000","last_peered":"2024-04-22T13:22:08.258994+0000","last_clean":"2024-04-22T13:22:08.258994+0000","last_became_active":"2024-04-22T13:22:05.240642+0000","last_became_peered":"2024-04-22T13:22:05.240642+0000","last_unstale":"2024-04-22T13:22:08.258994+0000","last_undegraded":"2024-04-22T13:22:08.258994+0000","last_fullsized":"2024-04-22T13:22:08.258994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:41:29.001979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-22T13:22:08.398806+0000","last_change":"2024-04-22T13:22:05.252212+0000","last_active":"2024-04-22T13:22:08.398806+0000","last_peered":"2024-04-22T13:22:08.398806+0000","last_clean":"2024-04-22T13:22:08.398806+0000","last_became_active":"2024-04-22T13:22:05.251852+0000","last_became_peered":"2024-04-22T13:22:05.251852+0000","last_unstale":"2024-04-22T13:22:08.398806+0000","last_undegraded":"2024-04-22T13:22:08.398806+0000","last_fullsized":"2024-04-22T13:22:08.398806+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:13:55.132501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4919,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":118,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":33906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318287,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":446155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579922,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28520,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716632,"statfs":{"total":95995035648,"available":95965831168,"internally_reserved":0,"allocated":1536000,"data_stored":1028567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743448,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":445781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005084,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":445567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234018,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32676,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712476,"statfs":{"total":95995035648,"available":95961575424,"internally_reserved":0,"allocated":1536000,"data_stored":1036535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430247,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":929792,"data_stored":444963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":438005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":445053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":455147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117047,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":445393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":942080,"data_stored":445607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691883,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28508,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716644,"statfs":{"total":95995035648,"available":95965843456,"internally_reserved":0,"allocated":1523712,"data_stored":1026943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":22867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T13:22:14.222 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-22T13:22:14.222 INFO:tasks.cephadm:Setting up client nodes... 2024-04-22T13:22:14.223 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-22T13:22:14.503 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:22:14.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:14 smithi012 ceph-mon[28265]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-22T13:22:14.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:14 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/4178940128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:22:14.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:14 smithi012 ceph-mon[28265]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:22:14.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:14 smithi012 ceph-mon[28265]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:22:14.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:14 smithi134 ceph-mon[31338]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-22T13:22:14.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:14 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/4178940128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:22:14.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:14 smithi134 ceph-mon[31338]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:22:14.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:14 smithi134 ceph-mon[31338]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:22:14.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:14 smithi111 ceph-mon[31361]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-22T13:22:14.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:14 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/4178940128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:22:14.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:14 smithi111 ceph-mon[31361]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:22:14.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:14 smithi111 ceph-mon[31361]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:22:15.477 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:15 smithi111 ceph-mon[31361]: pgmap v292: 97 pgs: 97 active+clean; 620 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 10 KiB/s wr, 25 op/s 2024-04-22T13:22:15.477 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 22 13:22:15 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-b[62191]: 2024-04-22T13:22:15.203+0000 7fa512624700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-04-22T13:22:15.478 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 22 13:22:15 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-b[62191]: 2024-04-22T13:22:15.203+0000 7fa512624700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-04-22T13:22:15.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:15 smithi134 ceph-mon[31338]: pgmap v292: 97 pgs: 97 active+clean; 620 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 10 KiB/s wr, 25 op/s 2024-04-22T13:22:15.707 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:15 smithi012 ceph-mon[28265]: pgmap v292: 97 pgs: 97 active+clean; 620 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 10 KiB/s wr, 25 op/s 2024-04-22T13:22:16.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.195+0000 7fa197052700 1 -- 172.21.15.12:0/2444920352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa190103570 msgr2=0x7fa190105960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:16.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.195+0000 7fa197052700 1 --2- 172.21.15.12:0/2444920352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa190103570 0x7fa190105960 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa184009990 tx=0x7fa184009ca0 comp rx=0 tx=0).stop 2024-04-22T13:22:16.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.195+0000 7fa197052700 1 -- 172.21.15.12:0/2444920352 shutdown_connections 2024-04-22T13:22:16.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.195+0000 7fa197052700 1 --2- 172.21.15.12:0/2444920352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1901087d0 0x7fa19010ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:16.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.195+0000 7fa197052700 1 --2- 172.21.15.12:0/2444920352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa190103570 0x7fa190105960 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:16.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.195+0000 7fa197052700 1 --2- 172.21.15.12:0/2444920352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa190105ea0 0x7fa190108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:16.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.195+0000 7fa197052700 1 -- 172.21.15.12:0/2444920352 >> 172.21.15.12:0/2444920352 conn(0x7fa1900fcfa0 msgr2=0x7fa1900ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:16.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.195+0000 7fa197052700 1 -- 172.21.15.12:0/2444920352 shutdown_connections 2024-04-22T13:22:16.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.195+0000 7fa197052700 1 -- 172.21.15.12:0/2444920352 wait complete. 2024-04-22T13:22:16.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.195+0000 7fa197052700 1 Processor -- start 2024-04-22T13:22:16.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.195+0000 7fa197052700 1 -- start start 2024-04-22T13:22:16.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.196+0000 7fa197052700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa190103570 0x7fa190198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:16.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.196+0000 7fa197052700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa190105ea0 0x7fa190199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:16.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.196+0000 7fa197052700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1901087d0 0x7fa1901a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:16.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.196+0000 7fa197052700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa19010e560 con 0x7fa1901087d0 2024-04-22T13:22:16.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.196+0000 7fa197052700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa19010e3e0 con 0x7fa190105ea0 2024-04-22T13:22:16.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.196+0000 7fa197052700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa19010e6e0 con 0x7fa190103570 2024-04-22T13:22:16.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.196+0000 7fa1955ef700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1901087d0 0x7fa1901a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:16.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.196+0000 7fa1955ef700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1901087d0 0x7fa1901a0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39918/0 (socket says 172.21.15.12:39918) 2024-04-22T13:22:16.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.196+0000 7fa1955ef700 1 -- 172.21.15.12:0/2249141410 learned_addr learned my addr 172.21.15.12:0/2249141410 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:16.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.196+0000 7fa194dee700 1 --2- 172.21.15.12:0/2249141410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa190103570 0x7fa190198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:16.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.196+0000 7fa18ffff700 1 --2- 172.21.15.12:0/2249141410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa190105ea0 0x7fa190199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:16.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.196+0000 7fa1955ef700 1 -- 172.21.15.12:0/2249141410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa190103570 msgr2=0x7fa190198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:16.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.196+0000 7fa1955ef700 1 --2- 172.21.15.12:0/2249141410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa190103570 0x7fa190198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:16.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.196+0000 7fa1955ef700 1 -- 172.21.15.12:0/2249141410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa190105ea0 msgr2=0x7fa190199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:16.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.196+0000 7fa1955ef700 1 --2- 172.21.15.12:0/2249141410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa190105ea0 0x7fa190199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:16.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.196+0000 7fa1955ef700 1 -- 172.21.15.12:0/2249141410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1901a07a0 con 0x7fa1901087d0 2024-04-22T13:22:16.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.196+0000 7fa1955ef700 1 --2- 172.21.15.12:0/2249141410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1901087d0 0x7fa1901a0100 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa18800add0 tx=0x7fa18800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:16.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.196+0000 7fa18dffb700 1 -- 172.21.15.12:0/2249141410 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa188017020 con 0x7fa1901087d0 2024-04-22T13:22:16.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.196+0000 7fa18dffb700 1 -- 172.21.15.12:0/2249141410 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa18800ae00 con 0x7fa1901087d0 2024-04-22T13:22:16.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.196+0000 7fa197052700 1 -- 172.21.15.12:0/2249141410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1901a0a90 con 0x7fa1901087d0 2024-04-22T13:22:16.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.197+0000 7fa18dffb700 1 -- 172.21.15.12:0/2249141410 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1880053a0 con 0x7fa1901087d0 2024-04-22T13:22:16.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.197+0000 7fa197052700 1 -- 172.21.15.12:0/2249141410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1901a0f80 con 0x7fa1901087d0 2024-04-22T13:22:16.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.198+0000 7fa197052700 1 -- 172.21.15.12:0/2249141410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa190050f80 con 0x7fa1901087d0 2024-04-22T13:22:16.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.199+0000 7fa18dffb700 1 -- 172.21.15.12:0/2249141410 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa188004ad0 con 0x7fa1901087d0 2024-04-22T13:22:16.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.199+0000 7fa18dffb700 1 --2- 172.21.15.12:0/2249141410 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa1800a3d80 0x7fa1800a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:16.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.199+0000 7fa18dffb700 1 -- 172.21.15.12:0/2249141410 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fa1880cfa40 con 0x7fa1901087d0 2024-04-22T13:22:16.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.199+0000 7fa194dee700 1 --2- 172.21.15.12:0/2249141410 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa1800a3d80 0x7fa1800a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:16.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.200+0000 7fa194dee700 1 --2- 172.21.15.12:0/2249141410 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa1800a3d80 0x7fa1800a6240 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa18400b5c0 tx=0x7fa184005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:16.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.204+0000 7fa18dffb700 1 -- 172.21.15.12:0/2249141410 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa188021020 con 0x7fa1901087d0 2024-04-22T13:22:16.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.502+0000 7fa197052700 1 -- 172.21.15.12:0/2249141410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fa190063650 con 0x7fa1901087d0 2024-04-22T13:22:16.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.504+0000 7fa18dffb700 1 -- 172.21.15.12:0/2249141410 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fa18809c500 con 0x7fa1901087d0 2024-04-22T13:22:16.502 INFO:teuthology.orchestra.run.smithi012.stdout:[client.0] 2024-04-22T13:22:16.502 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQCIZCZmdOkRHhAAQoJqYc2GbTEyawKiyPPuOg== 2024-04-22T13:22:16.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.506+0000 7fa197052700 1 -- 172.21.15.12:0/2249141410 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa1800a3d80 msgr2=0x7fa1800a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:16.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.506+0000 7fa197052700 1 --2- 172.21.15.12:0/2249141410 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa1800a3d80 0x7fa1800a6240 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa18400b5c0 tx=0x7fa184005fb0 comp rx=0 tx=0).stop 2024-04-22T13:22:16.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.506+0000 7fa197052700 1 -- 172.21.15.12:0/2249141410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1901087d0 msgr2=0x7fa1901a0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:16.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.506+0000 7fa197052700 1 --2- 172.21.15.12:0/2249141410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1901087d0 0x7fa1901a0100 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa18800add0 tx=0x7fa18800c330 comp rx=0 tx=0).stop 2024-04-22T13:22:16.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.506+0000 7fa197052700 1 -- 172.21.15.12:0/2249141410 shutdown_connections 2024-04-22T13:22:16.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.506+0000 7fa197052700 1 --2- 172.21.15.12:0/2249141410 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa1800a3d80 0x7fa1800a6240 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:16.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.506+0000 7fa197052700 1 --2- 172.21.15.12:0/2249141410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1901087d0 0x7fa1901a0100 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:16.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.506+0000 7fa197052700 1 --2- 172.21.15.12:0/2249141410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa190103570 0x7fa190198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:16.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.506+0000 7fa197052700 1 --2- 172.21.15.12:0/2249141410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa190105ea0 0x7fa190199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:16.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.506+0000 7fa197052700 1 -- 172.21.15.12:0/2249141410 >> 172.21.15.12:0/2249141410 conn(0x7fa1900fcfa0 msgr2=0x7fa190103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:16.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.506+0000 7fa197052700 1 -- 172.21.15.12:0/2249141410 shutdown_connections 2024-04-22T13:22:16.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:16.507+0000 7fa197052700 1 -- 172.21.15.12:0/2249141410 wait complete. 2024-04-22T13:22:17.217 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-22T13:22:17.217 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-22T13:22:17.218 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-22T13:22:17.257 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-22T13:22:17.258 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-22T13:22:17.258 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-22T13:22:17.547 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:22:17.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:17 smithi012 ceph-mon[28265]: pgmap v293: 97 pgs: 97 active+clean; 621 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 10 KiB/s wr, 28 op/s 2024-04-22T13:22:17.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:17 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2249141410' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-22T13:22:17.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:17 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2249141410' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-22T13:22:17.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:17 smithi134 ceph-mon[31338]: pgmap v293: 97 pgs: 97 active+clean; 621 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 10 KiB/s wr, 28 op/s 2024-04-22T13:22:17.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:17 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2249141410' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-22T13:22:17.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:17 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2249141410' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-22T13:22:17.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:17 smithi111 ceph-mon[31361]: pgmap v293: 97 pgs: 97 active+clean; 621 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 10 KiB/s wr, 28 op/s 2024-04-22T13:22:17.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:17 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2249141410' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-22T13:22:17.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:17 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2249141410' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-22T13:22:19.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.286+0000 7fe7c087b700 1 -- 172.21.15.12:0/3351516787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7bc102a90 msgr2=0x7fe7bc102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:19.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.286+0000 7fe7c087b700 1 --2- 172.21.15.12:0/3351516787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7bc102a90 0x7fe7bc102f10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009990 tx=0x7fe7b0009ca0 comp rx=0 tx=0).stop 2024-04-22T13:22:19.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.286+0000 7fe7c087b700 1 -- 172.21.15.12:0/3351516787 shutdown_connections 2024-04-22T13:22:19.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.286+0000 7fe7c087b700 1 --2- 172.21.15.12:0/3351516787 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7bc108a80 0x7fe7bc108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:19.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.286+0000 7fe7c087b700 1 --2- 172.21.15.12:0/3351516787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7bc103450 0x7fe7bc10f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:19.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.286+0000 7fe7c087b700 1 --2- 172.21.15.12:0/3351516787 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7bc102a90 0x7fe7bc102f10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:19.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.286+0000 7fe7c087b700 1 -- 172.21.15.12:0/3351516787 >> 172.21.15.12:0/3351516787 conn(0x7fe7bc079bf0 msgr2=0x7fe7bc07a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:19.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.286+0000 7fe7c087b700 1 -- 172.21.15.12:0/3351516787 shutdown_connections 2024-04-22T13:22:19.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.286+0000 7fe7c087b700 1 -- 172.21.15.12:0/3351516787 wait complete. 2024-04-22T13:22:19.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.287+0000 7fe7c087b700 1 Processor -- start 2024-04-22T13:22:19.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.287+0000 7fe7c087b700 1 -- start start 2024-04-22T13:22:19.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.287+0000 7fe7c087b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7bc102a90 0x7fe7bc1991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:19.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.287+0000 7fe7c087b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7bc103450 0x7fe7bc1996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:19.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.287+0000 7fe7c087b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7bc108a80 0x7fe7bc19da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:19.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.287+0000 7fe7c087b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc112cb0 con 0x7fe7bc103450 2024-04-22T13:22:19.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.287+0000 7fe7c087b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc112b30 con 0x7fe7bc102a90 2024-04-22T13:22:19.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.287+0000 7fe7c087b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc112e30 con 0x7fe7bc108a80 2024-04-22T13:22:19.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.287+0000 7fe7b9d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7bc103450 0x7fe7bc1996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:19.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.287+0000 7fe7b9d9b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7bc103450 0x7fe7bc1996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39938/0 (socket says 172.21.15.12:39938) 2024-04-22T13:22:19.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.287+0000 7fe7b9d9b700 1 -- 172.21.15.12:0/894172644 learned_addr learned my addr 172.21.15.12:0/894172644 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:19.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.287+0000 7fe7ba59c700 1 --2- 172.21.15.12:0/894172644 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7bc102a90 0x7fe7bc1991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:19.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.287+0000 7fe7bad9d700 1 --2- 172.21.15.12:0/894172644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7bc108a80 0x7fe7bc19da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:19.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.287+0000 7fe7b9d9b700 1 -- 172.21.15.12:0/894172644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7bc108a80 msgr2=0x7fe7bc19da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:19.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.287+0000 7fe7b9d9b700 1 --2- 172.21.15.12:0/894172644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7bc108a80 0x7fe7bc19da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:19.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.287+0000 7fe7b9d9b700 1 -- 172.21.15.12:0/894172644 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7bc102a90 msgr2=0x7fe7bc1991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:19.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.287+0000 7fe7b9d9b700 1 --2- 172.21.15.12:0/894172644 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7bc102a90 0x7fe7bc1991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:19.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.287+0000 7fe7b9d9b700 1 -- 172.21.15.12:0/894172644 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7bc19e160 con 0x7fe7bc103450 2024-04-22T13:22:19.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.287+0000 7fe7bad9d700 1 --2- 172.21.15.12:0/894172644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7bc108a80 0x7fe7bc19da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:22:19.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.288+0000 7fe7b9d9b700 1 --2- 172.21.15.12:0/894172644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7bc103450 0x7fe7bc1996e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0000c00 tx=0x7fe7b00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:19.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.288+0000 7fe7ab7fe700 1 -- 172.21.15.12:0/894172644 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0027070 con 0x7fe7bc103450 2024-04-22T13:22:19.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.288+0000 7fe7c087b700 1 -- 172.21.15.12:0/894172644 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7bc19e3f0 con 0x7fe7bc103450 2024-04-22T13:22:19.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.288+0000 7fe7ab7fe700 1 -- 172.21.15.12:0/894172644 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7b0003f20 con 0x7fe7bc103450 2024-04-22T13:22:19.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.288+0000 7fe7ab7fe700 1 -- 172.21.15.12:0/894172644 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0022cf0 con 0x7fe7bc103450 2024-04-22T13:22:19.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.288+0000 7fe7c087b700 1 -- 172.21.15.12:0/894172644 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7bc1a5cd0 con 0x7fe7bc103450 2024-04-22T13:22:19.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.289+0000 7fe7c087b700 1 -- 172.21.15.12:0/894172644 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7bc069730 con 0x7fe7bc103450 2024-04-22T13:22:19.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.290+0000 7fe7ab7fe700 1 -- 172.21.15.12:0/894172644 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe7b002a070 con 0x7fe7bc103450 2024-04-22T13:22:19.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.291+0000 7fe7ab7fe700 1 --2- 172.21.15.12:0/894172644 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe7a40a3cd0 0x7fe7a40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:19.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.291+0000 7fe7ba59c700 1 --2- 172.21.15.12:0/894172644 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe7a40a3cd0 0x7fe7a40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:19.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.291+0000 7fe7ab7fe700 1 -- 172.21.15.12:0/894172644 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fe7b00d11c0 con 0x7fe7bc103450 2024-04-22T13:22:19.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.291+0000 7fe7ba59c700 1 --2- 172.21.15.12:0/894172644 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe7a40a3cd0 0x7fe7a40a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac009a10 tx=0x7fe7ac009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:19.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.296+0000 7fe7ab7fe700 1 -- 172.21.15.12:0/894172644 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe7b0013070 con 0x7fe7bc103450 2024-04-22T13:22:19.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.566+0000 7fe7c087b700 1 -- 172.21.15.12:0/894172644 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fe7bc02df80 con 0x7fe7bc103450 2024-04-22T13:22:19.564 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:19 smithi012 ceph-mon[28265]: pgmap v294: 97 pgs: 97 active+clean; 621 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.7 KiB/s wr, 24 op/s 2024-04-22T13:22:19.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.572+0000 7fe7ab7fe700 1 -- 172.21.15.12:0/894172644 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264997 (secure 0 0 0) 0x7fe7b009dc80 con 0x7fe7bc103450 2024-04-22T13:22:19.572 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:22:19.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.574+0000 7fe7c087b700 1 -- 172.21.15.12:0/894172644 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe7a40a3cd0 msgr2=0x7fe7a40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:19.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.574+0000 7fe7c087b700 1 --2- 172.21.15.12:0/894172644 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe7a40a3cd0 0x7fe7a40a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac009a10 tx=0x7fe7ac009190 comp rx=0 tx=0).stop 2024-04-22T13:22:19.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.574+0000 7fe7c087b700 1 -- 172.21.15.12:0/894172644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7bc103450 msgr2=0x7fe7bc1996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:19.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.574+0000 7fe7c087b700 1 --2- 172.21.15.12:0/894172644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7bc103450 0x7fe7bc1996e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0000c00 tx=0x7fe7b00047b0 comp rx=0 tx=0).stop 2024-04-22T13:22:19.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.574+0000 7fe7c087b700 1 -- 172.21.15.12:0/894172644 shutdown_connections 2024-04-22T13:22:19.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.574+0000 7fe7c087b700 1 --2- 172.21.15.12:0/894172644 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7bc102a90 0x7fe7bc1991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:19.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.574+0000 7fe7c087b700 1 --2- 172.21.15.12:0/894172644 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe7a40a3cd0 0x7fe7a40a6190 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:19.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.574+0000 7fe7c087b700 1 --2- 172.21.15.12:0/894172644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe7bc108a80 0x7fe7bc19da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:19.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.574+0000 7fe7c087b700 1 --2- 172.21.15.12:0/894172644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe7bc103450 0x7fe7bc1996e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:19.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.574+0000 7fe7c087b700 1 -- 172.21.15.12:0/894172644 >> 172.21.15.12:0/894172644 conn(0x7fe7bc079bf0 msgr2=0x7fe7bc101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:19.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.574+0000 7fe7c087b700 1 -- 172.21.15.12:0/894172644 shutdown_connections 2024-04-22T13:22:19.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:19.574+0000 7fe7c087b700 1 -- 172.21.15.12:0/894172644 wait complete. 2024-04-22T13:22:19.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:19 smithi134 ceph-mon[31338]: pgmap v294: 97 pgs: 97 active+clean; 621 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.7 KiB/s wr, 24 op/s 2024-04-22T13:22:19.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:19 smithi111 ceph-mon[31361]: pgmap v294: 97 pgs: 97 active+clean; 621 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 8.7 KiB/s wr, 24 op/s 2024-04-22T13:22:20.199 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":1802873236},{"type":"v1","addr":"172.21.15.12:6801","nonce":1802873236}]},"active_addr":"172.21.15.12:6801/1802873236","active_change":"2024-04-22T13:15:04.280145+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.12:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":3535299028}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":856202072}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":3010129696}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":2837479598}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.12:0","nonce":2578446968}]}]} 2024-04-22T13:22:20.206 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-22T13:22:20.206 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-22T13:22:20.206 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-22T13:22:20.456 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:22:20.572 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:20 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/894172644' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-22T13:22:20.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:20 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/894172644' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-22T13:22:20.721 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:20 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/894172644' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-22T13:22:21.546 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:21 smithi012 ceph-mon[28265]: pgmap v295: 97 pgs: 97 active+clean; 626 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.5 KiB/s wr, 26 op/s 2024-04-22T13:22:21.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:21 smithi134 ceph-mon[31338]: pgmap v295: 97 pgs: 97 active+clean; 626 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.5 KiB/s wr, 26 op/s 2024-04-22T13:22:21.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:21 smithi111 ceph-mon[31361]: pgmap v295: 97 pgs: 97 active+clean; 626 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.5 KiB/s wr, 26 op/s 2024-04-22T13:22:22.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.054+0000 7f34f3755700 1 -- 172.21.15.12:0/1525102737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34ec06b980 msgr2=0x7f34ec10f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:22.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.054+0000 7f34f3755700 1 --2- 172.21.15.12:0/1525102737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34ec06b980 0x7f34ec10f8f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f34e4009990 tx=0x7f34e4009ca0 comp rx=0 tx=0).stop 2024-04-22T13:22:22.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.054+0000 7f34f3755700 1 -- 172.21.15.12:0/1525102737 shutdown_connections 2024-04-22T13:22:22.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.054+0000 7f34f3755700 1 --2- 172.21.15.12:0/1525102737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34ec06b980 0x7f34ec10f8f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:22.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.054+0000 7f34f3755700 1 --2- 172.21.15.12:0/1525102737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34ec108a20 0x7f34ec108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:22.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.054+0000 7f34f3755700 1 --2- 172.21.15.12:0/1525102737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34ec06afe0 0x7f34ec06b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:22.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.054+0000 7f34f3755700 1 -- 172.21.15.12:0/1525102737 >> 172.21.15.12:0/1525102737 conn(0x7f34ec0788b0 msgr2=0x7f34ec078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:22.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.054+0000 7f34f3755700 1 -- 172.21.15.12:0/1525102737 shutdown_connections 2024-04-22T13:22:22.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.054+0000 7f34f3755700 1 -- 172.21.15.12:0/1525102737 wait complete. 2024-04-22T13:22:22.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.055+0000 7f34f3755700 1 Processor -- start 2024-04-22T13:22:22.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.055+0000 7f34f3755700 1 -- start start 2024-04-22T13:22:22.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.055+0000 7f34f3755700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34ec06afe0 0x7f34ec1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:22.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.055+0000 7f34f3755700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34ec06b980 0x7f34ec1997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:22.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.055+0000 7f34f3755700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34ec108a20 0x7f34ec19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:22.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.055+0000 7f34f3755700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f34ec112ca0 con 0x7f34ec06b980 2024-04-22T13:22:22.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.055+0000 7f34f3755700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f34ec112b20 con 0x7f34ec06afe0 2024-04-22T13:22:22.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.055+0000 7f34f3755700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f34ec112e20 con 0x7f34ec108a20 2024-04-22T13:22:22.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.055+0000 7f34f0cf0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34ec06b980 0x7f34ec1997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:22.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.055+0000 7f34f0cf0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34ec06b980 0x7f34ec1997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48502/0 (socket says 172.21.15.12:48502) 2024-04-22T13:22:22.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.055+0000 7f34f0cf0700 1 -- 172.21.15.12:0/2786127871 learned_addr learned my addr 172.21.15.12:0/2786127871 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:22.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.055+0000 7f34f14f1700 1 --2- 172.21.15.12:0/2786127871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34ec06afe0 0x7f34ec1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:22.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.055+0000 7f34f1cf2700 1 --2- 172.21.15.12:0/2786127871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34ec108a20 0x7f34ec19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:22.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.055+0000 7f34f0cf0700 1 -- 172.21.15.12:0/2786127871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34ec108a20 msgr2=0x7f34ec19db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:22.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.055+0000 7f34f0cf0700 1 --2- 172.21.15.12:0/2786127871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34ec108a20 0x7f34ec19db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:22.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.055+0000 7f34f0cf0700 1 -- 172.21.15.12:0/2786127871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34ec06afe0 msgr2=0x7f34ec1992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:22.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.055+0000 7f34f0cf0700 1 --2- 172.21.15.12:0/2786127871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34ec06afe0 0x7f34ec1992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:22.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.055+0000 7f34f0cf0700 1 -- 172.21.15.12:0/2786127871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34ec19e2c0 con 0x7f34ec06b980 2024-04-22T13:22:22.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.057+0000 7f34f0cf0700 1 --2- 172.21.15.12:0/2786127871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34ec06b980 0x7f34ec1997f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00b530 tx=0x7f34dc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:22.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.057+0000 7f34e27fc700 1 -- 172.21.15.12:0/2786127871 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34dc015020 con 0x7f34ec06b980 2024-04-22T13:22:22.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.057+0000 7f34f3755700 1 -- 172.21.15.12:0/2786127871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34ec19e5b0 con 0x7f34ec06b980 2024-04-22T13:22:22.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.057+0000 7f34e27fc700 1 -- 172.21.15.12:0/2786127871 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34dc011ce0 con 0x7f34ec06b980 2024-04-22T13:22:22.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.057+0000 7f34e27fc700 1 -- 172.21.15.12:0/2786127871 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34dc010dd0 con 0x7f34ec06b980 2024-04-22T13:22:22.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.057+0000 7f34f3755700 1 -- 172.21.15.12:0/2786127871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34ec1a5e30 con 0x7f34ec06b980 2024-04-22T13:22:22.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.058+0000 7f34f3755700 1 -- 172.21.15.12:0/2786127871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34ec10b020 con 0x7f34ec06b980 2024-04-22T13:22:22.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.059+0000 7f34e27fc700 1 -- 172.21.15.12:0/2786127871 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f34dc018020 con 0x7f34ec06b980 2024-04-22T13:22:22.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.060+0000 7f34e27fc700 1 --2- 172.21.15.12:0/2786127871 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f34d80a3cd0 0x7f34d80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:22.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.060+0000 7f34f14f1700 1 --2- 172.21.15.12:0/2786127871 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f34d80a3cd0 0x7f34d80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:22.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.060+0000 7f34e27fc700 1 -- 172.21.15.12:0/2786127871 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f34dc0cf990 con 0x7f34ec06b980 2024-04-22T13:22:22.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.060+0000 7f34f14f1700 1 --2- 172.21.15.12:0/2786127871 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f34d80a3cd0 0x7f34d80a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f34e800a400 tx=0x7f34e800d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:22.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.064+0000 7f34e27fc700 1 -- 172.21.15.12:0/2786127871 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f34dc01f020 con 0x7f34ec06b980 2024-04-22T13:22:22.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.278+0000 7f34f3755700 1 -- 172.21.15.12:0/2786127871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34ec10aa00 con 0x7f34ec06b980 2024-04-22T13:22:22.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.279+0000 7f34e27fc700 1 -- 172.21.15.12:0/2786127871 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22233 (secure 0 0 0) 0x7f34dc09c4d0 con 0x7f34ec06b980 2024-04-22T13:22:22.276 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:22:22.276 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":87,"fsid":"0aede5ea-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:14:09.362918+0000","modified":"2024-04-22T13:22:06.227428+0000","last_up_change":"2024-04-22T13:21:15.263393+0000","last_in_change":"2024-04-22T13:20:54.174685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:17:32.372434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:22:02.768799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:22:03.578730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d9e2365-3a64-4b13-96fc-c217b1cee2ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6803","nonce":1191350632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6805","nonce":1191350632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6809","nonce":1191350632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6807","nonce":1191350632}]},"public_addr":"172.21.15.12:6803/1191350632","cluster_addr":"172.21.15.12:6805/1191350632","heartbeat_back_addr":"172.21.15.12:6809/1191350632","heartbeat_front_addr":"172.21.15.12:6807/1191350632","state":["exists","up"]},{"osd":1,"uuid":"1499c783-2bd7-48ce-a444-34add42bcf31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6801","nonce":417496465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6803","nonce":417496465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6807","nonce":417496465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6805","nonce":417496465}]},"public_addr":"172.21.15.111:6801/417496465","cluster_addr":"172.21.15.111:6803/417496465","heartbeat_back_addr":"172.21.15.111:6807/417496465","heartbeat_front_addr":"172.21.15.111:6805/417496465","state":["exists","up"]},{"osd":2,"uuid":"71a9ebc3-c622-434e-a6c2-f48358f384f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6801","nonce":1403153998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6803","nonce":1403153998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6807","nonce":1403153998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6805","nonce":1403153998}]},"public_addr":"172.21.15.134:6801/1403153998","cluster_addr":"172.21.15.134:6803/1403153998","heartbeat_back_addr":"172.21.15.134:6807/1403153998","heartbeat_front_addr":"172.21.15.134:6805/1403153998","state":["exists","up"]},{"osd":3,"uuid":"6dd00db2-a286-48ba-8666-56ffdd5f35ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6811","nonce":3359066681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6813","nonce":3359066681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6817","nonce":3359066681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6815","nonce":3359066681}]},"public_addr":"172.21.15.12:6811/3359066681","cluster_addr":"172.21.15.12:6813/3359066681","heartbeat_back_addr":"172.21.15.12:6817/3359066681","heartbeat_front_addr":"172.21.15.12:6815/3359066681","state":["exists","up"]},{"osd":4,"uuid":"1e29afd7-354c-4506-aaa0-b43eb1a19a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6809","nonce":2886349872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6811","nonce":2886349872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6815","nonce":2886349872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6813","nonce":2886349872}]},"public_addr":"172.21.15.111:6809/2886349872","cluster_addr":"172.21.15.111:6811/2886349872","heartbeat_back_addr":"172.21.15.111:6815/2886349872","heartbeat_front_addr":"172.21.15.111:6813/2886349872","state":["exists","up"]},{"osd":5,"uuid":"8f40f02d-0ba2-4842-a03d-2a1a04d1010d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6809","nonce":3071977602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6811","nonce":3071977602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6815","nonce":3071977602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6813","nonce":3071977602}]},"public_addr":"172.21.15.134:6809/3071977602","cluster_addr":"172.21.15.134:6811/3071977602","heartbeat_back_addr":"172.21.15.134:6815/3071977602","heartbeat_front_addr":"172.21.15.134:6813/3071977602","state":["exists","up"]},{"osd":6,"uuid":"bb0c17a9-6a8e-4be6-8660-f73cfa8200ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6819","nonce":3819619299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6821","nonce":3819619299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6825","nonce":3819619299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6823","nonce":3819619299}]},"public_addr":"172.21.15.12:6819/3819619299","cluster_addr":"172.21.15.12:6821/3819619299","heartbeat_back_addr":"172.21.15.12:6825/3819619299","heartbeat_front_addr":"172.21.15.12:6823/3819619299","state":["exists","up"]},{"osd":7,"uuid":"0a93d6af-569e-4e45-b697-9e978c1c1004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6817","nonce":3113856282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6819","nonce":3113856282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6823","nonce":3113856282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6821","nonce":3113856282}]},"public_addr":"172.21.15.111:6817/3113856282","cluster_addr":"172.21.15.111:6819/3113856282","heartbeat_back_addr":"172.21.15.111:6823/3113856282","heartbeat_front_addr":"172.21.15.111:6821/3113856282","state":["exists","up"]},{"osd":8,"uuid":"e21acc68-cf50-4f4f-b1ba-cf6c195b4955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6817","nonce":1689362191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6819","nonce":1689362191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6823","nonce":1689362191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6821","nonce":1689362191}]},"public_addr":"172.21.15.134:6817/1689362191","cluster_addr":"172.21.15.134:6819/1689362191","heartbeat_back_addr":"172.21.15.134:6823/1689362191","heartbeat_front_addr":"172.21.15.134:6821/1689362191","state":["exists","up"]},{"osd":9,"uuid":"fd745869-7f5f-48a8-b412-bb9c92b586e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6827","nonce":4292385779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6829","nonce":4292385779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6833","nonce":4292385779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6831","nonce":4292385779}]},"public_addr":"172.21.15.12:6827/4292385779","cluster_addr":"172.21.15.12:6829/4292385779","heartbeat_back_addr":"172.21.15.12:6833/4292385779","heartbeat_front_addr":"172.21.15.12:6831/4292385779","state":["exists","up"]},{"osd":10,"uuid":"798b1abd-f6c7-4a5f-8da0-fdae49e50d2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6825","nonce":3338566976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6827","nonce":3338566976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6831","nonce":3338566976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6829","nonce":3338566976}]},"public_addr":"172.21.15.111:6825/3338566976","cluster_addr":"172.21.15.111:6827/3338566976","heartbeat_back_addr":"172.21.15.111:6831/3338566976","heartbeat_front_addr":"172.21.15.111:6829/3338566976","state":["exists","up"]},{"osd":11,"uuid":"b18fc5f3-1049-4871-b060-e99290273cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6825","nonce":578640428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6827","nonce":578640428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6831","nonce":578640428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6829","nonce":578640428}]},"public_addr":"172.21.15.134:6825/578640428","cluster_addr":"172.21.15.134:6827/578640428","heartbeat_back_addr":"172.21.15.134:6831/578640428","heartbeat_front_addr":"172.21.15.134:6829/578640428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:16:40.828608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:05.052528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:28.297316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:53.496497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:17.947259+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:41.712894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:07.105945+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:32.129688+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:56.008516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:22.239867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:47.334692+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:12.301276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/2593601362":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/2519382069":"2024-04-23T13:15:04.279990+0000","172.21.15.12:6801/2009831940":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3509412187":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1007749062":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3588480095":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1904962176":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2009831940":"2024-04-23T13:15:04.279990+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:22:22.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.281+0000 7f34f3755700 1 -- 172.21.15.12:0/2786127871 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f34d80a3cd0 msgr2=0x7f34d80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:22.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.281+0000 7f34f3755700 1 --2- 172.21.15.12:0/2786127871 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f34d80a3cd0 0x7f34d80a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f34e800a400 tx=0x7f34e800d040 comp rx=0 tx=0).stop 2024-04-22T13:22:22.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.281+0000 7f34f3755700 1 -- 172.21.15.12:0/2786127871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34ec06b980 msgr2=0x7f34ec1997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:22.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.281+0000 7f34f3755700 1 --2- 172.21.15.12:0/2786127871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34ec06b980 0x7f34ec1997f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00b530 tx=0x7f34dc00b840 comp rx=0 tx=0).stop 2024-04-22T13:22:22.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.281+0000 7f34f3755700 1 -- 172.21.15.12:0/2786127871 shutdown_connections 2024-04-22T13:22:22.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.281+0000 7f34f3755700 1 --2- 172.21.15.12:0/2786127871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34ec06afe0 0x7f34ec1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:22.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.281+0000 7f34f3755700 1 --2- 172.21.15.12:0/2786127871 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f34d80a3cd0 0x7f34d80a6190 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:22.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.281+0000 7f34f3755700 1 --2- 172.21.15.12:0/2786127871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34ec108a20 0x7f34ec19db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:22.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.281+0000 7f34f3755700 1 --2- 172.21.15.12:0/2786127871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34ec06b980 0x7f34ec1997f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:22.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.281+0000 7f34f3755700 1 -- 172.21.15.12:0/2786127871 >> 172.21.15.12:0/2786127871 conn(0x7f34ec0788b0 msgr2=0x7f34ec0ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:22.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.281+0000 7f34f3755700 1 -- 172.21.15.12:0/2786127871 shutdown_connections 2024-04-22T13:22:22.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:22.281+0000 7f34f3755700 1 -- 172.21.15.12:0/2786127871 wait complete. 2024-04-22T13:22:22.573 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:22 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2786127871' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:22:22.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:22 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2786127871' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:22:22.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:22 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2786127871' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:22:22.932 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-22T13:22:22.932 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-22T13:22:23.187 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:22:23.579 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:23 smithi012 ceph-mon[28265]: pgmap v296: 97 pgs: 97 active+clean; 626 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.0 KiB/s wr, 23 op/s 2024-04-22T13:22:23.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:23 smithi134 ceph-mon[31338]: pgmap v296: 97 pgs: 97 active+clean; 626 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.0 KiB/s wr, 23 op/s 2024-04-22T13:22:23.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:23 smithi111 ceph-mon[31361]: pgmap v296: 97 pgs: 97 active+clean; 626 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.0 KiB/s wr, 23 op/s 2024-04-22T13:22:24.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.822+0000 7f3ab13b7700 1 -- 172.21.15.12:0/3204961972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3aac06bbb0 msgr2=0x7f3aac10f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:24.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.822+0000 7f3ab13b7700 1 --2- 172.21.15.12:0/3204961972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3aac06bbb0 0x7f3aac10f930 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3aa400b0a0 tx=0x7f3aa400b3b0 comp rx=0 tx=0).stop 2024-04-22T13:22:24.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.822+0000 7f3ab13b7700 1 -- 172.21.15.12:0/3204961972 shutdown_connections 2024-04-22T13:22:24.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.822+0000 7f3ab13b7700 1 --2- 172.21.15.12:0/3204961972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3aac108a60 0x7f3aac108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:24.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.822+0000 7f3ab13b7700 1 --2- 172.21.15.12:0/3204961972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3aac06bbb0 0x7f3aac10f930 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:24.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.822+0000 7f3ab13b7700 1 --2- 172.21.15.12:0/3204961972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3aac06b1f0 0x7f3aac06b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:24.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.822+0000 7f3ab13b7700 1 -- 172.21.15.12:0/3204961972 >> 172.21.15.12:0/3204961972 conn(0x7f3aac0789e0 msgr2=0x7f3aac078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:24.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.822+0000 7f3ab13b7700 1 -- 172.21.15.12:0/3204961972 shutdown_connections 2024-04-22T13:22:24.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.822+0000 7f3ab13b7700 1 -- 172.21.15.12:0/3204961972 wait complete. 2024-04-22T13:22:24.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.823+0000 7f3ab13b7700 1 Processor -- start 2024-04-22T13:22:24.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.823+0000 7f3ab13b7700 1 -- start start 2024-04-22T13:22:24.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.823+0000 7f3ab13b7700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3aac06b1f0 0x7f3aac1019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:24.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.823+0000 7f3ab13b7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3aac06bbb0 0x7f3aac101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:24.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.823+0000 7f3ab13b7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3aac108a60 0x7f3aac102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:24.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.823+0000 7f3ab13b7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac112ca0 con 0x7f3aac06bbb0 2024-04-22T13:22:24.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.823+0000 7f3ab13b7700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac112b20 con 0x7f3aac108a60 2024-04-22T13:22:24.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.823+0000 7f3ab13b7700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac112e20 con 0x7f3aac06b1f0 2024-04-22T13:22:24.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.823+0000 7f3aaa7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3aac06bbb0 0x7f3aac101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:24.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.823+0000 7f3aaa7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3aac06bbb0 0x7f3aac101ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48522/0 (socket says 172.21.15.12:48522) 2024-04-22T13:22:24.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.823+0000 7f3aaa7fc700 1 -- 172.21.15.12:0/1975332424 learned_addr learned my addr 172.21.15.12:0/1975332424 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:24.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.824+0000 7f3aaaffd700 1 --2- 172.21.15.12:0/1975332424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3aac06b1f0 0x7f3aac1019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:24.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.824+0000 7f3aab7fe700 1 --2- 172.21.15.12:0/1975332424 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3aac108a60 0x7f3aac102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:24.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.824+0000 7f3aaa7fc700 1 -- 172.21.15.12:0/1975332424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3aac06b1f0 msgr2=0x7f3aac1019b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:24.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.824+0000 7f3aaa7fc700 1 --2- 172.21.15.12:0/1975332424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3aac06b1f0 0x7f3aac1019b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:24.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.824+0000 7f3aaa7fc700 1 -- 172.21.15.12:0/1975332424 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3aac108a60 msgr2=0x7f3aac102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:24.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.824+0000 7f3aaa7fc700 1 --2- 172.21.15.12:0/1975332424 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3aac108a60 0x7f3aac102430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:24.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.824+0000 7f3aaa7fc700 1 -- 172.21.15.12:0/1975332424 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aac1a7580 con 0x7f3aac06bbb0 2024-04-22T13:22:24.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.824+0000 7f3aab7fe700 1 --2- 172.21.15.12:0/1975332424 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3aac108a60 0x7f3aac102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:22:24.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.824+0000 7f3aaaffd700 1 --2- 172.21.15.12:0/1975332424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3aac06b1f0 0x7f3aac1019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:22:24.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.824+0000 7f3aaa7fc700 1 --2- 172.21.15.12:0/1975332424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3aac06bbb0 0x7f3aac101ef0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000c9b0 tx=0x7f3aa000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:24.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.824+0000 7f3a93fff700 1 -- 172.21.15.12:0/1975332424 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0007b00 con 0x7f3aac06bbb0 2024-04-22T13:22:24.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.824+0000 7f3ab13b7700 1 -- 172.21.15.12:0/1975332424 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aac1a7870 con 0x7f3aac06bbb0 2024-04-22T13:22:24.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.824+0000 7f3a93fff700 1 -- 172.21.15.12:0/1975332424 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aa0007ca0 con 0x7f3aac06bbb0 2024-04-22T13:22:24.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.824+0000 7f3a93fff700 1 -- 172.21.15.12:0/1975332424 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0018730 con 0x7f3aac06bbb0 2024-04-22T13:22:24.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.824+0000 7f3ab13b7700 1 -- 172.21.15.12:0/1975332424 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aac1a7e00 con 0x7f3aac06bbb0 2024-04-22T13:22:24.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.825+0000 7f3ab13b7700 1 -- 172.21.15.12:0/1975332424 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aac069730 con 0x7f3aac06bbb0 2024-04-22T13:22:24.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.826+0000 7f3a93fff700 1 -- 172.21.15.12:0/1975332424 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3aa000f450 con 0x7f3aac06bbb0 2024-04-22T13:22:24.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.827+0000 7f3a93fff700 1 --2- 172.21.15.12:0/1975332424 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f3a940a3d50 0x7f3a940a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:24.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.827+0000 7f3aaaffd700 1 --2- 172.21.15.12:0/1975332424 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f3a940a3d50 0x7f3a940a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:24.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.827+0000 7f3a93fff700 1 -- 172.21.15.12:0/1975332424 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f3aa00d0500 con 0x7f3aac06bbb0 2024-04-22T13:22:24.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.828+0000 7f3aaaffd700 1 --2- 172.21.15.12:0/1975332424 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f3a940a3d50 0x7f3a940a6210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c005e10 tx=0x7f3a9c005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:24.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:24.832+0000 7f3a93fff700 1 -- 172.21.15.12:0/1975332424 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3aa0014030 con 0x7f3aac06bbb0 2024-04-22T13:22:25.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:25.043+0000 7f3ab13b7700 1 -- 172.21.15.12:0/1975332424 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3aac02df80 con 0x7f3aac06bbb0 2024-04-22T13:22:25.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:25.044+0000 7f3a93fff700 1 -- 172.21.15.12:0/1975332424 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22233 (secure 0 0 0) 0x7f3aa009ce30 con 0x7f3aac06bbb0 2024-04-22T13:22:25.041 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:22:25.041 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":87,"fsid":"0aede5ea-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:14:09.362918+0000","modified":"2024-04-22T13:22:06.227428+0000","last_up_change":"2024-04-22T13:21:15.263393+0000","last_in_change":"2024-04-22T13:20:54.174685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:17:32.372434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:22:02.768799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:22:03.578730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d9e2365-3a64-4b13-96fc-c217b1cee2ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6803","nonce":1191350632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6805","nonce":1191350632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6809","nonce":1191350632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6807","nonce":1191350632}]},"public_addr":"172.21.15.12:6803/1191350632","cluster_addr":"172.21.15.12:6805/1191350632","heartbeat_back_addr":"172.21.15.12:6809/1191350632","heartbeat_front_addr":"172.21.15.12:6807/1191350632","state":["exists","up"]},{"osd":1,"uuid":"1499c783-2bd7-48ce-a444-34add42bcf31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6801","nonce":417496465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6803","nonce":417496465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6807","nonce":417496465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6805","nonce":417496465}]},"public_addr":"172.21.15.111:6801/417496465","cluster_addr":"172.21.15.111:6803/417496465","heartbeat_back_addr":"172.21.15.111:6807/417496465","heartbeat_front_addr":"172.21.15.111:6805/417496465","state":["exists","up"]},{"osd":2,"uuid":"71a9ebc3-c622-434e-a6c2-f48358f384f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6801","nonce":1403153998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6803","nonce":1403153998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6807","nonce":1403153998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6805","nonce":1403153998}]},"public_addr":"172.21.15.134:6801/1403153998","cluster_addr":"172.21.15.134:6803/1403153998","heartbeat_back_addr":"172.21.15.134:6807/1403153998","heartbeat_front_addr":"172.21.15.134:6805/1403153998","state":["exists","up"]},{"osd":3,"uuid":"6dd00db2-a286-48ba-8666-56ffdd5f35ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6811","nonce":3359066681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6813","nonce":3359066681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6817","nonce":3359066681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6815","nonce":3359066681}]},"public_addr":"172.21.15.12:6811/3359066681","cluster_addr":"172.21.15.12:6813/3359066681","heartbeat_back_addr":"172.21.15.12:6817/3359066681","heartbeat_front_addr":"172.21.15.12:6815/3359066681","state":["exists","up"]},{"osd":4,"uuid":"1e29afd7-354c-4506-aaa0-b43eb1a19a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6809","nonce":2886349872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6811","nonce":2886349872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6815","nonce":2886349872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6813","nonce":2886349872}]},"public_addr":"172.21.15.111:6809/2886349872","cluster_addr":"172.21.15.111:6811/2886349872","heartbeat_back_addr":"172.21.15.111:6815/2886349872","heartbeat_front_addr":"172.21.15.111:6813/2886349872","state":["exists","up"]},{"osd":5,"uuid":"8f40f02d-0ba2-4842-a03d-2a1a04d1010d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6809","nonce":3071977602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6811","nonce":3071977602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6815","nonce":3071977602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6813","nonce":3071977602}]},"public_addr":"172.21.15.134:6809/3071977602","cluster_addr":"172.21.15.134:6811/3071977602","heartbeat_back_addr":"172.21.15.134:6815/3071977602","heartbeat_front_addr":"172.21.15.134:6813/3071977602","state":["exists","up"]},{"osd":6,"uuid":"bb0c17a9-6a8e-4be6-8660-f73cfa8200ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6819","nonce":3819619299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6821","nonce":3819619299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6825","nonce":3819619299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6823","nonce":3819619299}]},"public_addr":"172.21.15.12:6819/3819619299","cluster_addr":"172.21.15.12:6821/3819619299","heartbeat_back_addr":"172.21.15.12:6825/3819619299","heartbeat_front_addr":"172.21.15.12:6823/3819619299","state":["exists","up"]},{"osd":7,"uuid":"0a93d6af-569e-4e45-b697-9e978c1c1004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6817","nonce":3113856282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6819","nonce":3113856282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6823","nonce":3113856282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6821","nonce":3113856282}]},"public_addr":"172.21.15.111:6817/3113856282","cluster_addr":"172.21.15.111:6819/3113856282","heartbeat_back_addr":"172.21.15.111:6823/3113856282","heartbeat_front_addr":"172.21.15.111:6821/3113856282","state":["exists","up"]},{"osd":8,"uuid":"e21acc68-cf50-4f4f-b1ba-cf6c195b4955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6817","nonce":1689362191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6819","nonce":1689362191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6823","nonce":1689362191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6821","nonce":1689362191}]},"public_addr":"172.21.15.134:6817/1689362191","cluster_addr":"172.21.15.134:6819/1689362191","heartbeat_back_addr":"172.21.15.134:6823/1689362191","heartbeat_front_addr":"172.21.15.134:6821/1689362191","state":["exists","up"]},{"osd":9,"uuid":"fd745869-7f5f-48a8-b412-bb9c92b586e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6827","nonce":4292385779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6829","nonce":4292385779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6833","nonce":4292385779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6831","nonce":4292385779}]},"public_addr":"172.21.15.12:6827/4292385779","cluster_addr":"172.21.15.12:6829/4292385779","heartbeat_back_addr":"172.21.15.12:6833/4292385779","heartbeat_front_addr":"172.21.15.12:6831/4292385779","state":["exists","up"]},{"osd":10,"uuid":"798b1abd-f6c7-4a5f-8da0-fdae49e50d2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6825","nonce":3338566976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6827","nonce":3338566976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6831","nonce":3338566976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6829","nonce":3338566976}]},"public_addr":"172.21.15.111:6825/3338566976","cluster_addr":"172.21.15.111:6827/3338566976","heartbeat_back_addr":"172.21.15.111:6831/3338566976","heartbeat_front_addr":"172.21.15.111:6829/3338566976","state":["exists","up"]},{"osd":11,"uuid":"b18fc5f3-1049-4871-b060-e99290273cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6825","nonce":578640428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6827","nonce":578640428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6831","nonce":578640428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6829","nonce":578640428}]},"public_addr":"172.21.15.134:6825/578640428","cluster_addr":"172.21.15.134:6827/578640428","heartbeat_back_addr":"172.21.15.134:6831/578640428","heartbeat_front_addr":"172.21.15.134:6829/578640428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:16:40.828608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:05.052528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:28.297316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:53.496497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:17.947259+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:41.712894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:07.105945+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:32.129688+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:56.008516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:22.239867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:47.334692+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:12.301276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/2593601362":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/2519382069":"2024-04-23T13:15:04.279990+0000","172.21.15.12:6801/2009831940":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3509412187":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1007749062":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3588480095":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1904962176":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2009831940":"2024-04-23T13:15:04.279990+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:22:25.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:25.046+0000 7f3ab13b7700 1 -- 172.21.15.12:0/1975332424 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f3a940a3d50 msgr2=0x7f3a940a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:25.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:25.046+0000 7f3ab13b7700 1 --2- 172.21.15.12:0/1975332424 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f3a940a3d50 0x7f3a940a6210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c005e10 tx=0x7f3a9c005d60 comp rx=0 tx=0).stop 2024-04-22T13:22:25.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:25.046+0000 7f3ab13b7700 1 -- 172.21.15.12:0/1975332424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3aac06bbb0 msgr2=0x7f3aac101ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:25.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:25.046+0000 7f3ab13b7700 1 --2- 172.21.15.12:0/1975332424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3aac06bbb0 0x7f3aac101ef0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000c9b0 tx=0x7f3aa000cd70 comp rx=0 tx=0).stop 2024-04-22T13:22:25.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:25.046+0000 7f3ab13b7700 1 -- 172.21.15.12:0/1975332424 shutdown_connections 2024-04-22T13:22:25.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:25.046+0000 7f3ab13b7700 1 --2- 172.21.15.12:0/1975332424 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3aac108a60 0x7f3aac102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:25.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:25.046+0000 7f3ab13b7700 1 --2- 172.21.15.12:0/1975332424 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f3a940a3d50 0x7f3a940a6210 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:25.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:25.046+0000 7f3ab13b7700 1 --2- 172.21.15.12:0/1975332424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3aac06b1f0 0x7f3aac1019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:25.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:25.046+0000 7f3ab13b7700 1 --2- 172.21.15.12:0/1975332424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3aac06bbb0 0x7f3aac101ef0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:25.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:25.046+0000 7f3ab13b7700 1 -- 172.21.15.12:0/1975332424 >> 172.21.15.12:0/1975332424 conn(0x7f3aac0789e0 msgr2=0x7f3aac0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:25.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:25.046+0000 7f3ab13b7700 1 -- 172.21.15.12:0/1975332424 shutdown_connections 2024-04-22T13:22:25.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:25.046+0000 7f3ab13b7700 1 -- 172.21.15.12:0/1975332424 wait complete. 2024-04-22T13:22:25.570 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-22T13:22:25.571 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-22T13:22:25.571 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-22T13:22:25.571 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-22T13:22:25.572 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-22T13:22:25.572 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-22T13:22:25.572 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-22T13:22:25.573 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-22T13:22:25.573 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-22T13:22:25.574 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-22T13:22:25.574 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-22T13:22:25.574 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-22T13:22:25.614 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:25 smithi012 ceph-mon[28265]: pgmap v297: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 KiB/s wr, 21 op/s 2024-04-22T13:22:25.614 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:25 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1975332424' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:22:25.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:25 smithi111 ceph-mon[31361]: pgmap v297: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 KiB/s wr, 21 op/s 2024-04-22T13:22:25.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:25 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1975332424' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:22:25.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:25 smithi134 ceph-mon[31338]: pgmap v297: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.3 KiB/s wr, 21 op/s 2024-04-22T13:22:25.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:25 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1975332424' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:22:26.321 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:22:26.325 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:22:26.332 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:22:26.333 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:22:26.333 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:22:26.334 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:22:26.334 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:22:26.366 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:22:26.371 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:22:26.380 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:22:27.718 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:27 smithi012 ceph-mon[28265]: pgmap v298: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.2 KiB/s wr, 14 op/s 2024-04-22T13:22:27.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:27 smithi111 ceph-mon[31361]: pgmap v298: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.2 KiB/s wr, 14 op/s 2024-04-22T13:22:27.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:27 smithi134 ceph-mon[31338]: pgmap v298: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.2 KiB/s wr, 14 op/s 2024-04-22T13:22:29.045 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:22:29.047 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:22:29.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:29 smithi111 ceph-mon[31361]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-04-22T13:22:29.757 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:29 smithi012 ceph-mon[28265]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-04-22T13:22:29.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:29 smithi134 ceph-mon[31338]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-04-22T13:22:31.581 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:31 smithi012 ceph-mon[28265]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-04-22T13:22:31.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:31 smithi111 ceph-mon[31361]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-04-22T13:22:31.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:31 smithi134 ceph-mon[31338]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-04-22T13:22:33.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:33 smithi012 ceph-mon[28265]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-22T13:22:33.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:33 smithi111 ceph-mon[31361]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-22T13:22:33.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:33 smithi134 ceph-mon[31338]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-22T13:22:34.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.381+0000 7efc0b016700 1 -- 172.21.15.12:0/1926589568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc04107940 msgr2=0x7efc04107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:34.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.381+0000 7efc0b016700 1 --2- 172.21.15.12:0/1926589568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc04107940 0x7efc04107d20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efbf80099c0 tx=0x7efbf8009cd0 comp rx=0 tx=0).stop 2024-04-22T13:22:34.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.381+0000 7efc0b016700 1 -- 172.21.15.12:0/1926589568 shutdown_connections 2024-04-22T13:22:34.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.381+0000 7efc0b016700 1 --2- 172.21.15.12:0/1926589568 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc041034b0 0x7efc0410f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:34.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.381+0000 7efc0b016700 1 --2- 172.21.15.12:0/1926589568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc04107940 0x7efc04107d20 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:34.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.381+0000 7efc0b016700 1 --2- 172.21.15.12:0/1926589568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc04102af0 0x7efc04102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:34.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.381+0000 7efc0b016700 1 -- 172.21.15.12:0/1926589568 >> 172.21.15.12:0/1926589568 conn(0x7efc04079ba0 msgr2=0x7efc04079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:34.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.381+0000 7efc0b016700 1 -- 172.21.15.12:0/1926589568 shutdown_connections 2024-04-22T13:22:34.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.381+0000 7efc0b016700 1 -- 172.21.15.12:0/1926589568 wait complete. 2024-04-22T13:22:34.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.382+0000 7efc0b016700 1 Processor -- start 2024-04-22T13:22:34.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.382+0000 7efc0b016700 1 -- start start 2024-04-22T13:22:34.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.383+0000 7efc0b016700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc04102af0 0x7efc04199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:34.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.383+0000 7efc0b016700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc041034b0 0x7efc04199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:34.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.383+0000 7efc0b016700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc04107940 0x7efc0419d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:34.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.383+0000 7efc0b016700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efc04112c40 con 0x7efc04102af0 2024-04-22T13:22:34.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.383+0000 7efc0b016700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efc04112ac0 con 0x7efc041034b0 2024-04-22T13:22:34.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.383+0000 7efc0b016700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efc04112dc0 con 0x7efc04107940 2024-04-22T13:22:34.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.383+0000 7efc08db2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc04102af0 0x7efc04199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:34.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.383+0000 7efc08db2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc04102af0 0x7efc04199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38766/0 (socket says 172.21.15.12:38766) 2024-04-22T13:22:34.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.383+0000 7efc08db2700 1 -- 172.21.15.12:0/3672576403 learned_addr learned my addr 172.21.15.12:0/3672576403 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:34.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.383+0000 7efc03fff700 1 --2- 172.21.15.12:0/3672576403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc041034b0 0x7efc04199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:34.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.383+0000 7efc095b3700 1 --2- 172.21.15.12:0/3672576403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc04107940 0x7efc0419d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:34.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.383+0000 7efc08db2700 1 -- 172.21.15.12:0/3672576403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc04107940 msgr2=0x7efc0419d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:34.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.383+0000 7efc08db2700 1 --2- 172.21.15.12:0/3672576403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc04107940 0x7efc0419d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:34.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.383+0000 7efc08db2700 1 -- 172.21.15.12:0/3672576403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc041034b0 msgr2=0x7efc04199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:34.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.383+0000 7efc08db2700 1 --2- 172.21.15.12:0/3672576403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc041034b0 0x7efc04199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:34.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.383+0000 7efc08db2700 1 -- 172.21.15.12:0/3672576403 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0419e120 con 0x7efc04102af0 2024-04-22T13:22:34.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.383+0000 7efc08db2700 1 --2- 172.21.15.12:0/3672576403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc04102af0 0x7efc04199110 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7efbf800b580 tx=0x7efbf80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:34.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.383+0000 7efc01ffb700 1 -- 172.21.15.12:0/3672576403 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf8027070 con 0x7efc04102af0 2024-04-22T13:22:34.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.384+0000 7efc0b016700 1 -- 172.21.15.12:0/3672576403 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc0419e3b0 con 0x7efc04102af0 2024-04-22T13:22:34.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.384+0000 7efc01ffb700 1 -- 172.21.15.12:0/3672576403 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbf8003f20 con 0x7efc04102af0 2024-04-22T13:22:34.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.384+0000 7efc01ffb700 1 -- 172.21.15.12:0/3672576403 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf8022d70 con 0x7efc04102af0 2024-04-22T13:22:34.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.384+0000 7efc0b016700 1 -- 172.21.15.12:0/3672576403 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc041a5c90 con 0x7efc04102af0 2024-04-22T13:22:34.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.385+0000 7efc0b016700 1 -- 172.21.15.12:0/3672576403 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efc040631d0 con 0x7efc04102af0 2024-04-22T13:22:34.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.386+0000 7efc01ffb700 1 -- 172.21.15.12:0/3672576403 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7efbf8022500 con 0x7efc04102af0 2024-04-22T13:22:34.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.387+0000 7efc01ffb700 1 --2- 172.21.15.12:0/3672576403 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efbf40a3d50 0x7efbf40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:34.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.387+0000 7efc03fff700 1 --2- 172.21.15.12:0/3672576403 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efbf40a3d50 0x7efbf40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:34.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.387+0000 7efc01ffb700 1 -- 172.21.15.12:0/3672576403 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7efbf80d16d0 con 0x7efc04102af0 2024-04-22T13:22:34.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.387+0000 7efc01ffb700 1 --2- 172.21.15.12:0/3672576403 >> [v2:172.21.15.12:6826/4292385779,v1:172.21.15.12:6827/4292385779] conn(0x7efbf40ad400 0x7efbf40af820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:34.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.388+0000 7efc01ffb700 1 -- 172.21.15.12:0/3672576403 --> [v2:172.21.15.12:6826/4292385779,v1:172.21.15.12:6827/4292385779] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efbf8019e80 con 0x7efbf40ad400 2024-04-22T13:22:34.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.388+0000 7efc01ffb700 1 -- 172.21.15.12:0/3672576403 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7efbf80d1bb0 con 0x7efc04102af0 2024-04-22T13:22:34.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.388+0000 7efc095b3700 1 --2- 172.21.15.12:0/3672576403 >> [v2:172.21.15.12:6826/4292385779,v1:172.21.15.12:6827/4292385779] conn(0x7efbf40ad400 0x7efbf40af820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:34.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.388+0000 7efc03fff700 1 --2- 172.21.15.12:0/3672576403 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efbf40a3d50 0x7efbf40a6210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efc0407a210 tx=0x7efbf00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:34.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.388+0000 7efc095b3700 1 --2- 172.21.15.12:0/3672576403 >> [v2:172.21.15.12:6826/4292385779,v1:172.21.15.12:6827/4292385779] conn(0x7efbf40ad400 0x7efbf40af820 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:34.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.389+0000 7efc01ffb700 1 -- 172.21.15.12:0/3672576403 <== osd.9 v2:172.21.15.12:6826/4292385779 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efbf8019e80 con 0x7efbf40ad400 2024-04-22T13:22:34.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.406+0000 7efc0b016700 1 -- 172.21.15.12:0/3672576403 --> [v2:172.21.15.12:6826/4292385779,v1:172.21.15.12:6827/4292385779] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efc040635d0 con 0x7efbf40ad400 2024-04-22T13:22:34.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.406+0000 7efc01ffb700 1 -- 172.21.15.12:0/3672576403 <== osd.9 v2:172.21.15.12:6826/4292385779 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7efc040635d0 con 0x7efbf40ad400 2024-04-22T13:22:34.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.406+0000 7efc0b016700 1 -- 172.21.15.12:0/3672576403 >> [v2:172.21.15.12:6826/4292385779,v1:172.21.15.12:6827/4292385779] conn(0x7efbf40ad400 msgr2=0x7efbf40af820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:34.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.406+0000 7efc0b016700 1 --2- 172.21.15.12:0/3672576403 >> [v2:172.21.15.12:6826/4292385779,v1:172.21.15.12:6827/4292385779] conn(0x7efbf40ad400 0x7efbf40af820 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:34.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.406+0000 7efc0b016700 1 -- 172.21.15.12:0/3672576403 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efbf40a3d50 msgr2=0x7efbf40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:34.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.406+0000 7efc0b016700 1 --2- 172.21.15.12:0/3672576403 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efbf40a3d50 0x7efbf40a6210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efc0407a210 tx=0x7efbf00073d0 comp rx=0 tx=0).stop 2024-04-22T13:22:34.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.406+0000 7efc0b016700 1 -- 172.21.15.12:0/3672576403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc04102af0 msgr2=0x7efc04199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:34.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.406+0000 7efc0b016700 1 --2- 172.21.15.12:0/3672576403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc04102af0 0x7efc04199110 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7efbf800b580 tx=0x7efbf80047b0 comp rx=0 tx=0).stop 2024-04-22T13:22:34.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.406+0000 7efc095b3700 1 -- 172.21.15.12:0/3672576403 reap_dead start 2024-04-22T13:22:34.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.406+0000 7efc0b016700 1 -- 172.21.15.12:0/3672576403 shutdown_connections 2024-04-22T13:22:34.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.406+0000 7efc0b016700 1 -- 172.21.15.12:0/3672576403 >> 172.21.15.12:0/3672576403 conn(0x7efc04079ba0 msgr2=0x7efc04100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:34.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.407+0000 7efc0b016700 1 -- 172.21.15.12:0/3672576403 shutdown_connections 2024-04-22T13:22:34.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:34.407+0000 7efc0b016700 1 -- 172.21.15.12:0/3672576403 wait complete. 2024-04-22T13:22:35.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:35 smithi111 ceph-mon[31361]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-22T13:22:35.742 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:35 smithi012 ceph-mon[28265]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-22T13:22:35.776 INFO:teuthology.orchestra.run.smithi012.stdout:296352743453 2024-04-22T13:22:35.777 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-22T13:22:35.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:35 smithi134 ceph-mon[31338]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-22T13:22:37.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:37 smithi111 ceph-mon[31361]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-22T13:22:37.765 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:37 smithi012 ceph-mon[28265]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-22T13:22:37.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:37 smithi134 ceph-mon[31338]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-22T13:22:39.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.076+0000 7f11915f4700 1 -- 172.21.15.12:0/4211954886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f118c103480 msgr2=0x7f118c10f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:39.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.076+0000 7f11915f4700 1 --2- 172.21.15.12:0/4211954886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f118c103480 0x7f118c10f8d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f118400b0a0 tx=0x7f118400b3b0 comp rx=0 tx=0).stop 2024-04-22T13:22:39.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.076+0000 7f11915f4700 1 -- 172.21.15.12:0/4211954886 shutdown_connections 2024-04-22T13:22:39.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.076+0000 7f11915f4700 1 --2- 172.21.15.12:0/4211954886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f118c103480 0x7f118c10f8d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:39.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.076+0000 7f11915f4700 1 --2- 172.21.15.12:0/4211954886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f118c108a70 0x7f118c108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:39.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.076+0000 7f11915f4700 1 --2- 172.21.15.12:0/4211954886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f118c102ac0 0x7f118c102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:39.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.076+0000 7f11915f4700 1 -- 172.21.15.12:0/4211954886 >> 172.21.15.12:0/4211954886 conn(0x7f118c079bb0 msgr2=0x7f118c079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:39.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.076+0000 7f11915f4700 1 -- 172.21.15.12:0/4211954886 shutdown_connections 2024-04-22T13:22:39.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.076+0000 7f11915f4700 1 -- 172.21.15.12:0/4211954886 wait complete. 2024-04-22T13:22:39.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.076+0000 7f11915f4700 1 Processor -- start 2024-04-22T13:22:39.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.077+0000 7f11915f4700 1 -- start start 2024-04-22T13:22:39.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.077+0000 7f11915f4700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f118c102ac0 0x7f118c074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:39.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.077+0000 7f11915f4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f118c103480 0x7f118c075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:39.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.077+0000 7f11915f4700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f118c108a70 0x7f118c0757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:39.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.077+0000 7f11915f4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f118c112c20 con 0x7f118c103480 2024-04-22T13:22:39.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.077+0000 7f11915f4700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f118c112aa0 con 0x7f118c102ac0 2024-04-22T13:22:39.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.077+0000 7f11915f4700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f118c112da0 con 0x7f118c108a70 2024-04-22T13:22:39.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.077+0000 7f118a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f118c103480 0x7f118c075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:39.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.077+0000 7f118a7fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f118c103480 0x7f118c075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38784/0 (socket says 172.21.15.12:38784) 2024-04-22T13:22:39.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.077+0000 7f118a7fc700 1 -- 172.21.15.12:0/417969781 learned_addr learned my addr 172.21.15.12:0/417969781 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:39.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.077+0000 7f118affd700 1 --2- 172.21.15.12:0/417969781 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f118c102ac0 0x7f118c074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:39.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.077+0000 7f118b7fe700 1 --2- 172.21.15.12:0/417969781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f118c108a70 0x7f118c0757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:39.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.077+0000 7f118a7fc700 1 -- 172.21.15.12:0/417969781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f118c108a70 msgr2=0x7f118c0757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:39.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.077+0000 7f118a7fc700 1 --2- 172.21.15.12:0/417969781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f118c108a70 0x7f118c0757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:39.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.077+0000 7f118a7fc700 1 -- 172.21.15.12:0/417969781 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f118c102ac0 msgr2=0x7f118c074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:39.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.077+0000 7f118a7fc700 1 --2- 172.21.15.12:0/417969781 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f118c102ac0 0x7f118c074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:39.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.077+0000 7f118a7fc700 1 -- 172.21.15.12:0/417969781 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f118c1a7490 con 0x7f118c103480 2024-04-22T13:22:39.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.078+0000 7f118a7fc700 1 --2- 172.21.15.12:0/417969781 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f118c103480 0x7f118c075280 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f117c00c9b0 tx=0x7f117c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:39.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.078+0000 7f1183fff700 1 -- 172.21.15.12:0/417969781 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c007b00 con 0x7f118c103480 2024-04-22T13:22:39.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.078+0000 7f11915f4700 1 -- 172.21.15.12:0/417969781 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f118c1a7780 con 0x7f118c103480 2024-04-22T13:22:39.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.078+0000 7f1183fff700 1 -- 172.21.15.12:0/417969781 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f117c00ce50 con 0x7f118c103480 2024-04-22T13:22:39.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.078+0000 7f1183fff700 1 -- 172.21.15.12:0/417969781 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c018660 con 0x7f118c103480 2024-04-22T13:22:39.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.078+0000 7f11915f4700 1 -- 172.21.15.12:0/417969781 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f118c1a7d10 con 0x7f118c103480 2024-04-22T13:22:39.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.079+0000 7f11915f4700 1 -- 172.21.15.12:0/417969781 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f118c108080 con 0x7f118c103480 2024-04-22T13:22:39.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.080+0000 7f1183fff700 1 -- 172.21.15.12:0/417969781 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f117c00f450 con 0x7f118c103480 2024-04-22T13:22:39.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.081+0000 7f1183fff700 1 --2- 172.21.15.12:0/417969781 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f11780a3cd0 0x7f11780a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:39.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.081+0000 7f118affd700 1 --2- 172.21.15.12:0/417969781 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f11780a3cd0 0x7f11780a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:39.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.081+0000 7f1183fff700 1 -- 172.21.15.12:0/417969781 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f117c0d0740 con 0x7f118c103480 2024-04-22T13:22:39.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.081+0000 7f1183fff700 1 --2- 172.21.15.12:0/417969781 >> [v2:172.21.15.134:6824/578640428,v1:172.21.15.134:6825/578640428] conn(0x7f11780ad4b0 0x7f11780af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:39.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.081+0000 7f1183fff700 1 -- 172.21.15.12:0/417969781 --> [v2:172.21.15.134:6824/578640428,v1:172.21.15.134:6825/578640428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f11780aff80 con 0x7f11780ad4b0 2024-04-22T13:22:39.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.081+0000 7f1183fff700 1 -- 172.21.15.12:0/417969781 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f117c0d0c20 con 0x7f118c103480 2024-04-22T13:22:39.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.081+0000 7f118affd700 1 --2- 172.21.15.12:0/417969781 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f11780a3cd0 0x7f11780a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f117400abd0 tx=0x7f117400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:39.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.081+0000 7f118b7fe700 1 --2- 172.21.15.12:0/417969781 >> [v2:172.21.15.134:6824/578640428,v1:172.21.15.134:6825/578640428] conn(0x7f11780ad4b0 0x7f11780af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:39.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.082+0000 7f118b7fe700 1 --2- 172.21.15.12:0/417969781 >> [v2:172.21.15.134:6824/578640428,v1:172.21.15.134:6825/578640428] conn(0x7f11780ad4b0 0x7f11780af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:39.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.083+0000 7f1183fff700 1 -- 172.21.15.12:0/417969781 <== osd.11 v2:172.21.15.134:6824/578640428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f11780aff80 con 0x7f11780ad4b0 2024-04-22T13:22:39.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.099+0000 7f11915f4700 1 -- 172.21.15.12:0/417969781 --> [v2:172.21.15.134:6824/578640428,v1:172.21.15.134:6825/578640428] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f118c1041a0 con 0x7f11780ad4b0 2024-04-22T13:22:39.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.099+0000 7f1183fff700 1 -- 172.21.15.12:0/417969781 <== osd.11 v2:172.21.15.134:6824/578640428 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f118c1041a0 con 0x7f11780ad4b0 2024-04-22T13:22:39.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.100+0000 7f11915f4700 1 -- 172.21.15.12:0/417969781 >> [v2:172.21.15.134:6824/578640428,v1:172.21.15.134:6825/578640428] conn(0x7f11780ad4b0 msgr2=0x7f11780af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:39.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.100+0000 7f11915f4700 1 --2- 172.21.15.12:0/417969781 >> [v2:172.21.15.134:6824/578640428,v1:172.21.15.134:6825/578640428] conn(0x7f11780ad4b0 0x7f11780af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:39.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.100+0000 7f11915f4700 1 -- 172.21.15.12:0/417969781 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f11780a3cd0 msgr2=0x7f11780a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:39.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.100+0000 7f11915f4700 1 --2- 172.21.15.12:0/417969781 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f11780a3cd0 0x7f11780a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f117400abd0 tx=0x7f117400a380 comp rx=0 tx=0).stop 2024-04-22T13:22:39.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.100+0000 7f11915f4700 1 -- 172.21.15.12:0/417969781 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f118c103480 msgr2=0x7f118c075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:39.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.100+0000 7f11915f4700 1 --2- 172.21.15.12:0/417969781 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f118c103480 0x7f118c075280 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f117c00c9b0 tx=0x7f117c00ccc0 comp rx=0 tx=0).stop 2024-04-22T13:22:39.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.100+0000 7f118b7fe700 1 -- 172.21.15.12:0/417969781 reap_dead start 2024-04-22T13:22:39.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.100+0000 7f11915f4700 1 -- 172.21.15.12:0/417969781 shutdown_connections 2024-04-22T13:22:39.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.100+0000 7f11915f4700 1 -- 172.21.15.12:0/417969781 >> 172.21.15.12:0/417969781 conn(0x7f118c079bb0 msgr2=0x7f118c106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:39.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.100+0000 7f11915f4700 1 -- 172.21.15.12:0/417969781 shutdown_connections 2024-04-22T13:22:39.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.100+0000 7f11915f4700 1 -- 172.21.15.12:0/417969781 wait complete. 2024-04-22T13:22:39.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.713+0000 7fd16c85b700 1 -- 172.21.15.12:0/2014835483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd168102300 msgr2=0x7fd16810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:39.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.713+0000 7fd16c85b700 1 --2- 172.21.15.12:0/2014835483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd168102300 0x7fd16810e7f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd16000b0a0 tx=0x7fd16000b3b0 comp rx=0 tx=0).stop 2024-04-22T13:22:39.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.713+0000 7fd16c85b700 1 -- 172.21.15.12:0/2014835483 shutdown_connections 2024-04-22T13:22:39.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.713+0000 7fd16c85b700 1 --2- 172.21.15.12:0/2014835483 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd168107990 0x7fd168107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:39.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.713+0000 7fd16c85b700 1 --2- 172.21.15.12:0/2014835483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd168102300 0x7fd16810e7f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:39.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.713+0000 7fd16c85b700 1 --2- 172.21.15.12:0/2014835483 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd168101940 0x7fd168101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:39.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.713+0000 7fd16c85b700 1 -- 172.21.15.12:0/2014835483 >> 172.21.15.12:0/2014835483 conn(0x7fd1680788f0 msgr2=0x7fd168078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:39.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.713+0000 7fd16c85b700 1 -- 172.21.15.12:0/2014835483 shutdown_connections 2024-04-22T13:22:39.711 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:39 smithi012 ceph-mon[28265]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-22T13:22:39.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.714+0000 7fd16c85b700 1 -- 172.21.15.12:0/2014835483 wait complete. 2024-04-22T13:22:39.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.714+0000 7fd16c85b700 1 Processor -- start 2024-04-22T13:22:39.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.714+0000 7fd16c85b700 1 -- start start 2024-04-22T13:22:39.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.715+0000 7fd16c85b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd168101940 0x7fd168199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:39.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.715+0000 7fd16c85b700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd168102300 0x7fd16819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:39.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.715+0000 7fd16c85b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd168107990 0x7fd1681a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:39.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.715+0000 7fd16c85b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd168111af0 con 0x7fd168101940 2024-04-22T13:22:39.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.715+0000 7fd16c85b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd168111970 con 0x7fd168107990 2024-04-22T13:22:39.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.715+0000 7fd16c85b700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd168111c70 con 0x7fd168102300 2024-04-22T13:22:39.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.715+0000 7fd16659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd168101940 0x7fd168199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:39.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.715+0000 7fd16659c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd168101940 0x7fd168199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38806/0 (socket says 172.21.15.12:38806) 2024-04-22T13:22:39.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.715+0000 7fd16659c700 1 -- 172.21.15.12:0/3658778435 learned_addr learned my addr 172.21.15.12:0/3658778435 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:39.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.715+0000 7fd165d9b700 1 --2- 172.21.15.12:0/3658778435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd168102300 0x7fd16819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:39.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.715+0000 7fd166d9d700 1 --2- 172.21.15.12:0/3658778435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd168107990 0x7fd1681a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:39.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.715+0000 7fd16659c700 1 -- 172.21.15.12:0/3658778435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd168102300 msgr2=0x7fd16819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:39.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.715+0000 7fd16659c700 1 --2- 172.21.15.12:0/3658778435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd168102300 0x7fd16819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:39.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.715+0000 7fd16659c700 1 -- 172.21.15.12:0/3658778435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd168107990 msgr2=0x7fd1681a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:39.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.715+0000 7fd16659c700 1 --2- 172.21.15.12:0/3658778435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd168107990 0x7fd1681a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:39.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.715+0000 7fd16659c700 1 -- 172.21.15.12:0/3658778435 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1681a54f0 con 0x7fd168101940 2024-04-22T13:22:39.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.715+0000 7fd16659c700 1 --2- 172.21.15.12:0/3658778435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd168101940 0x7fd168199f30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd15800c920 tx=0x7fd15800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:39.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.716+0000 7fd1577fe700 1 -- 172.21.15.12:0/3658778435 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd158004a30 con 0x7fd168101940 2024-04-22T13:22:39.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.716+0000 7fd16c85b700 1 -- 172.21.15.12:0/3658778435 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1681a56c0 con 0x7fd168101940 2024-04-22T13:22:39.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.716+0000 7fd1577fe700 1 -- 172.21.15.12:0/3658778435 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1580054d0 con 0x7fd168101940 2024-04-22T13:22:39.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.716+0000 7fd1577fe700 1 -- 172.21.15.12:0/3658778435 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd158007bd0 con 0x7fd168101940 2024-04-22T13:22:39.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.716+0000 7fd16c85b700 1 -- 172.21.15.12:0/3658778435 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1681a5c00 con 0x7fd168101940 2024-04-22T13:22:39.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.716+0000 7fd16c85b700 1 -- 172.21.15.12:0/3658778435 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd168063150 con 0x7fd168101940 2024-04-22T13:22:39.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.718+0000 7fd1577fe700 1 -- 172.21.15.12:0/3658778435 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd158004bd0 con 0x7fd168101940 2024-04-22T13:22:39.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.718+0000 7fd1577fe700 1 --2- 172.21.15.12:0/3658778435 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fd1500a3cd0 0x7fd1500a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:39.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.718+0000 7fd165d9b700 1 --2- 172.21.15.12:0/3658778435 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fd1500a3cd0 0x7fd1500a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:39.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.718+0000 7fd1577fe700 1 -- 172.21.15.12:0/3658778435 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fd1580d02c0 con 0x7fd168101940 2024-04-22T13:22:39.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.719+0000 7fd1577fe700 1 --2- 172.21.15.12:0/3658778435 >> [v2:172.21.15.111:6808/2886349872,v1:172.21.15.111:6809/2886349872] conn(0x7fd1500ad4b0 0x7fd1500af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:39.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.719+0000 7fd1577fe700 1 -- 172.21.15.12:0/3658778435 --> [v2:172.21.15.111:6808/2886349872,v1:172.21.15.111:6809/2886349872] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd1500aff80 con 0x7fd1500ad4b0 2024-04-22T13:22:39.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.719+0000 7fd1577fe700 1 -- 172.21.15.12:0/3658778435 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fd1580d07a0 con 0x7fd168101940 2024-04-22T13:22:39.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.719+0000 7fd165d9b700 1 --2- 172.21.15.12:0/3658778435 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fd1500a3cd0 0x7fd1500a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd168078ed0 tx=0x7fd15c00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:39.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.719+0000 7fd166d9d700 1 --2- 172.21.15.12:0/3658778435 >> [v2:172.21.15.111:6808/2886349872,v1:172.21.15.111:6809/2886349872] conn(0x7fd1500ad4b0 0x7fd1500af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:39.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.720+0000 7fd166d9d700 1 --2- 172.21.15.12:0/3658778435 >> [v2:172.21.15.111:6808/2886349872,v1:172.21.15.111:6809/2886349872] conn(0x7fd1500ad4b0 0x7fd1500af8d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:39.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.720+0000 7fd1577fe700 1 -- 172.21.15.12:0/3658778435 <== osd.4 v2:172.21.15.111:6808/2886349872 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd1500aff80 con 0x7fd1500ad4b0 2024-04-22T13:22:39.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:39 smithi111 ceph-mon[31361]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-22T13:22:39.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.737+0000 7fd16c85b700 1 -- 172.21.15.12:0/3658778435 --> [v2:172.21.15.111:6808/2886349872,v1:172.21.15.111:6809/2886349872] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd1680635d0 con 0x7fd1500ad4b0 2024-04-22T13:22:39.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.737+0000 7fd1577fe700 1 -- 172.21.15.12:0/3658778435 <== osd.4 v2:172.21.15.111:6808/2886349872 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd1680635d0 con 0x7fd1500ad4b0 2024-04-22T13:22:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.738+0000 7fd16c85b700 1 -- 172.21.15.12:0/3658778435 >> [v2:172.21.15.111:6808/2886349872,v1:172.21.15.111:6809/2886349872] conn(0x7fd1500ad4b0 msgr2=0x7fd1500af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.738+0000 7fd16c85b700 1 --2- 172.21.15.12:0/3658778435 >> [v2:172.21.15.111:6808/2886349872,v1:172.21.15.111:6809/2886349872] conn(0x7fd1500ad4b0 0x7fd1500af8d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.738+0000 7fd16c85b700 1 -- 172.21.15.12:0/3658778435 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fd1500a3cd0 msgr2=0x7fd1500a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.738+0000 7fd16c85b700 1 --2- 172.21.15.12:0/3658778435 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fd1500a3cd0 0x7fd1500a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd168078ed0 tx=0x7fd15c00b3c0 comp rx=0 tx=0).stop 2024-04-22T13:22:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.738+0000 7fd16c85b700 1 -- 172.21.15.12:0/3658778435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd168101940 msgr2=0x7fd168199f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.738+0000 7fd16c85b700 1 --2- 172.21.15.12:0/3658778435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd168101940 0x7fd168199f30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd15800c920 tx=0x7fd15800cc30 comp rx=0 tx=0).stop 2024-04-22T13:22:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.738+0000 7fd166d9d700 1 -- 172.21.15.12:0/3658778435 reap_dead start 2024-04-22T13:22:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.738+0000 7fd16c85b700 1 -- 172.21.15.12:0/3658778435 shutdown_connections 2024-04-22T13:22:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.738+0000 7fd16c85b700 1 -- 172.21.15.12:0/3658778435 >> 172.21.15.12:0/3658778435 conn(0x7fd1680788f0 msgr2=0x7fd1680ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.738+0000 7fd16c85b700 1 -- 172.21.15.12:0/3658778435 shutdown_connections 2024-04-22T13:22:39.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:39.738+0000 7fd16c85b700 1 -- 172.21.15.12:0/3658778435 wait complete. 2024-04-22T13:22:39.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:39 smithi134 ceph-mon[31338]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-22T13:22:40.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.879+0000 7fe10d684700 1 -- 172.21.15.12:0/1495114719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe10806a750 msgr2=0x7fe10806abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:40.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.879+0000 7fe10d684700 1 --2- 172.21.15.12:0/1495114719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe10806a750 0x7fe10806abb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0009990 tx=0x7fe0f0009ca0 comp rx=0 tx=0).stop 2024-04-22T13:22:40.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.879+0000 7fe10d684700 1 -- 172.21.15.12:0/1495114719 shutdown_connections 2024-04-22T13:22:40.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.879+0000 7fe10d684700 1 --2- 172.21.15.12:0/1495114719 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe108108a30 0x7fe108108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:40.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.879+0000 7fe10d684700 1 --2- 172.21.15.12:0/1495114719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe10806b0f0 0x7fe10810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:40.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.879+0000 7fe10d684700 1 --2- 172.21.15.12:0/1495114719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe10806a750 0x7fe10806abb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:40.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.879+0000 7fe10d684700 1 -- 172.21.15.12:0/1495114719 >> 172.21.15.12:0/1495114719 conn(0x7fe1080788f0 msgr2=0x7fe108078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:40.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.879+0000 7fe10d684700 1 -- 172.21.15.12:0/1495114719 shutdown_connections 2024-04-22T13:22:40.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.879+0000 7fe10d684700 1 -- 172.21.15.12:0/1495114719 wait complete. 2024-04-22T13:22:40.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.879+0000 7fe10d684700 1 Processor -- start 2024-04-22T13:22:40.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.879+0000 7fe10d684700 1 -- start start 2024-04-22T13:22:40.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.880+0000 7fe10d684700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe10806a750 0x7fe1081990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:40.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.880+0000 7fe10d684700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe10806b0f0 0x7fe108199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:40.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.880+0000 7fe10d684700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe108108a30 0x7fe10819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:40.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.880+0000 7fe10d684700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe108112bd0 con 0x7fe10806b0f0 2024-04-22T13:22:40.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.880+0000 7fe10d684700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe108112a50 con 0x7fe10806a750 2024-04-22T13:22:40.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.880+0000 7fe10d684700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe108112d50 con 0x7fe108108a30 2024-04-22T13:22:40.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.880+0000 7fe0fffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe10806b0f0 0x7fe108199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:40.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.880+0000 7fe0fffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe10806b0f0 0x7fe108199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38824/0 (socket says 172.21.15.12:38824) 2024-04-22T13:22:40.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.880+0000 7fe0fffff700 1 -- 172.21.15.12:0/982089205 learned_addr learned my addr 172.21.15.12:0/982089205 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:40.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.880+0000 7fe106ffd700 1 --2- 172.21.15.12:0/982089205 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe10806a750 0x7fe1081990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:40.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.880+0000 7fe1077fe700 1 --2- 172.21.15.12:0/982089205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe108108a30 0x7fe10819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:40.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.880+0000 7fe0fffff700 1 -- 172.21.15.12:0/982089205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe108108a30 msgr2=0x7fe10819d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:40.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.880+0000 7fe0fffff700 1 --2- 172.21.15.12:0/982089205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe108108a30 0x7fe10819d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:40.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.880+0000 7fe0fffff700 1 -- 172.21.15.12:0/982089205 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe10806a750 msgr2=0x7fe1081990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:40.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.880+0000 7fe0fffff700 1 --2- 172.21.15.12:0/982089205 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe10806a750 0x7fe1081990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:40.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.880+0000 7fe0fffff700 1 -- 172.21.15.12:0/982089205 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe10819e0a0 con 0x7fe10806b0f0 2024-04-22T13:22:40.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.880+0000 7fe0fffff700 1 --2- 172.21.15.12:0/982089205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe10806b0f0 0x7fe108199620 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0000c00 tx=0x7fe0f00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:40.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.880+0000 7fe104ff9700 1 -- 172.21.15.12:0/982089205 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0027070 con 0x7fe10806b0f0 2024-04-22T13:22:40.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.880+0000 7fe104ff9700 1 -- 172.21.15.12:0/982089205 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0f00041d0 con 0x7fe10806b0f0 2024-04-22T13:22:40.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.880+0000 7fe10d684700 1 -- 172.21.15.12:0/982089205 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe10819e330 con 0x7fe10806b0f0 2024-04-22T13:22:40.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.880+0000 7fe104ff9700 1 -- 172.21.15.12:0/982089205 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f000f460 con 0x7fe10806b0f0 2024-04-22T13:22:40.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.880+0000 7fe10d684700 1 -- 172.21.15.12:0/982089205 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1081a5c10 con 0x7fe10806b0f0 2024-04-22T13:22:40.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.881+0000 7fe10d684700 1 -- 172.21.15.12:0/982089205 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe1080631d0 con 0x7fe10806b0f0 2024-04-22T13:22:40.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.883+0000 7fe104ff9700 1 -- 172.21.15.12:0/982089205 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe0f002a070 con 0x7fe10806b0f0 2024-04-22T13:22:40.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.883+0000 7fe104ff9700 1 --2- 172.21.15.12:0/982089205 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe0f40a3cd0 0x7fe0f40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:40.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.883+0000 7fe104ff9700 1 -- 172.21.15.12:0/982089205 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fe0f00d1bb0 con 0x7fe10806b0f0 2024-04-22T13:22:40.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.883+0000 7fe106ffd700 1 --2- 172.21.15.12:0/982089205 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe0f40a3cd0 0x7fe0f40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:40.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.883+0000 7fe104ff9700 1 --2- 172.21.15.12:0/982089205 >> [v2:172.21.15.134:6816/1689362191,v1:172.21.15.134:6817/1689362191] conn(0x7fe0f40ad4b0 0x7fe0f40af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:40.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.883+0000 7fe104ff9700 1 -- 172.21.15.12:0/982089205 --> [v2:172.21.15.134:6816/1689362191,v1:172.21.15.134:6817/1689362191] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe0f00043d0 con 0x7fe0f40ad4b0 2024-04-22T13:22:40.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.883+0000 7fe104ff9700 1 -- 172.21.15.12:0/982089205 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fe0f00d1f60 con 0x7fe10806b0f0 2024-04-22T13:22:40.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.884+0000 7fe106ffd700 1 --2- 172.21.15.12:0/982089205 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe0f40a3cd0 0x7fe0f40a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8005fd0 tx=0x7fe0f800af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:40.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.884+0000 7fe1077fe700 1 --2- 172.21.15.12:0/982089205 >> [v2:172.21.15.134:6816/1689362191,v1:172.21.15.134:6817/1689362191] conn(0x7fe0f40ad4b0 0x7fe0f40af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:40.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.884+0000 7fe1077fe700 1 --2- 172.21.15.12:0/982089205 >> [v2:172.21.15.134:6816/1689362191,v1:172.21.15.134:6817/1689362191] conn(0x7fe0f40ad4b0 0x7fe0f40af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:40.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.885+0000 7fe104ff9700 1 -- 172.21.15.12:0/982089205 <== osd.8 v2:172.21.15.134:6816/1689362191 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe0f00043d0 con 0x7fe0f40ad4b0 2024-04-22T13:22:40.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.902+0000 7fe10d684700 1 -- 172.21.15.12:0/982089205 --> [v2:172.21.15.134:6816/1689362191,v1:172.21.15.134:6817/1689362191] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe108063650 con 0x7fe0f40ad4b0 2024-04-22T13:22:40.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.903+0000 7fe104ff9700 1 -- 172.21.15.12:0/982089205 <== osd.8 v2:172.21.15.134:6816/1689362191 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe108063650 con 0x7fe0f40ad4b0 2024-04-22T13:22:40.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.903+0000 7fe10d684700 1 -- 172.21.15.12:0/982089205 >> [v2:172.21.15.134:6816/1689362191,v1:172.21.15.134:6817/1689362191] conn(0x7fe0f40ad4b0 msgr2=0x7fe0f40af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:40.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.903+0000 7fe10d684700 1 --2- 172.21.15.12:0/982089205 >> [v2:172.21.15.134:6816/1689362191,v1:172.21.15.134:6817/1689362191] conn(0x7fe0f40ad4b0 0x7fe0f40af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:40.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.903+0000 7fe10d684700 1 -- 172.21.15.12:0/982089205 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe0f40a3cd0 msgr2=0x7fe0f40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:40.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.903+0000 7fe10d684700 1 --2- 172.21.15.12:0/982089205 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe0f40a3cd0 0x7fe0f40a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8005fd0 tx=0x7fe0f800af20 comp rx=0 tx=0).stop 2024-04-22T13:22:40.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.903+0000 7fe10d684700 1 -- 172.21.15.12:0/982089205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe10806b0f0 msgr2=0x7fe108199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:40.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.903+0000 7fe10d684700 1 --2- 172.21.15.12:0/982089205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe10806b0f0 0x7fe108199620 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0000c00 tx=0x7fe0f00047b0 comp rx=0 tx=0).stop 2024-04-22T13:22:40.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.903+0000 7fe1077fe700 1 -- 172.21.15.12:0/982089205 reap_dead start 2024-04-22T13:22:40.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.903+0000 7fe10d684700 1 -- 172.21.15.12:0/982089205 shutdown_connections 2024-04-22T13:22:40.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.903+0000 7fe10d684700 1 -- 172.21.15.12:0/982089205 >> 172.21.15.12:0/982089205 conn(0x7fe1080788f0 msgr2=0x7fe1080ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:40.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.903+0000 7fe10d684700 1 -- 172.21.15.12:0/982089205 shutdown_connections 2024-04-22T13:22:40.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:40.903+0000 7fe10d684700 1 -- 172.21.15.12:0/982089205 wait complete. 2024-04-22T13:22:41.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:41 smithi111 ceph-mon[31361]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-22T13:22:41.782 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:41 smithi012 ceph-mon[28265]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-22T13:22:41.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:41 smithi134 ceph-mon[31338]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-22T13:22:42.207 INFO:teuthology.orchestra.run.smithi012.stdout:150323855416 2024-04-22T13:22:42.208 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-22T13:22:42.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.437+0000 7fc9bf2de700 1 -- 172.21.15.12:0/1761237081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9b8107990 msgr2=0x7fc9b8107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:42.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.437+0000 7fc9bf2de700 1 --2- 172.21.15.12:0/1761237081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9b8107990 0x7fc9b8107d70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4009990 tx=0x7fc9b4009ca0 comp rx=0 tx=0).stop 2024-04-22T13:22:42.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.437+0000 7fc9bf2de700 1 -- 172.21.15.12:0/1761237081 shutdown_connections 2024-04-22T13:22:42.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.437+0000 7fc9bf2de700 1 --2- 172.21.15.12:0/1761237081 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9b8102300 0x7fc9b810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:42.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.437+0000 7fc9bf2de700 1 --2- 172.21.15.12:0/1761237081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9b8107990 0x7fc9b8107d70 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:42.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.437+0000 7fc9bf2de700 1 --2- 172.21.15.12:0/1761237081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9b8101940 0x7fc9b8101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:42.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.437+0000 7fc9bf2de700 1 -- 172.21.15.12:0/1761237081 >> 172.21.15.12:0/1761237081 conn(0x7fc9b80788f0 msgr2=0x7fc9b8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:42.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.437+0000 7fc9bf2de700 1 -- 172.21.15.12:0/1761237081 shutdown_connections 2024-04-22T13:22:42.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.437+0000 7fc9bf2de700 1 -- 172.21.15.12:0/1761237081 wait complete. 2024-04-22T13:22:42.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.438+0000 7fc9bf2de700 1 Processor -- start 2024-04-22T13:22:42.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.438+0000 7fc9bf2de700 1 -- start start 2024-04-22T13:22:42.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.438+0000 7fc9bf2de700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9b8101940 0x7fc9b8199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:42.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.438+0000 7fc9bf2de700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9b8102300 0x7fc9b819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:42.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.438+0000 7fc9bf2de700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9b8107990 0x7fc9b81a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:42.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.438+0000 7fc9bf2de700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b8111b80 con 0x7fc9b8102300 2024-04-22T13:22:42.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.438+0000 7fc9bf2de700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b8111a00 con 0x7fc9b8101940 2024-04-22T13:22:42.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.438+0000 7fc9bf2de700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b8111d00 con 0x7fc9b8107990 2024-04-22T13:22:42.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.438+0000 7fc9bc879700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9b8102300 0x7fc9b819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:42.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.438+0000 7fc9bc879700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9b8102300 0x7fc9b819a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34014/0 (socket says 172.21.15.12:34014) 2024-04-22T13:22:42.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.439+0000 7fc9bc879700 1 -- 172.21.15.12:0/3734874123 learned_addr learned my addr 172.21.15.12:0/3734874123 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:42.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.439+0000 7fc9bd07a700 1 --2- 172.21.15.12:0/3734874123 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9b8101940 0x7fc9b8199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:42.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.439+0000 7fc9bd87b700 1 --2- 172.21.15.12:0/3734874123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9b8107990 0x7fc9b81a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:42.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.439+0000 7fc9bc879700 1 -- 172.21.15.12:0/3734874123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9b8107990 msgr2=0x7fc9b81a5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:42.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.439+0000 7fc9bc879700 1 --2- 172.21.15.12:0/3734874123 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9b8107990 0x7fc9b81a5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:42.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.439+0000 7fc9bc879700 1 -- 172.21.15.12:0/3734874123 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9b8101940 msgr2=0x7fc9b8199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:42.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.439+0000 7fc9bc879700 1 --2- 172.21.15.12:0/3734874123 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9b8101940 0x7fc9b8199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:42.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.439+0000 7fc9bc879700 1 -- 172.21.15.12:0/3734874123 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b81a5580 con 0x7fc9b8102300 2024-04-22T13:22:42.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.439+0000 7fc9bd07a700 1 --2- 172.21.15.12:0/3734874123 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9b8101940 0x7fc9b8199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:22:42.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.439+0000 7fc9bc879700 1 --2- 172.21.15.12:0/3734874123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9b8102300 0x7fc9b819a500 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800c9b0 tx=0x7fc9a800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:42.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.439+0000 7fc9ae7fc700 1 -- 172.21.15.12:0/3734874123 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a8007b00 con 0x7fc9b8102300 2024-04-22T13:22:42.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.439+0000 7fc9bf2de700 1 -- 172.21.15.12:0/3734874123 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9b81a5750 con 0x7fc9b8102300 2024-04-22T13:22:42.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.439+0000 7fc9ae7fc700 1 -- 172.21.15.12:0/3734874123 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9a8007ca0 con 0x7fc9b8102300 2024-04-22T13:22:42.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.439+0000 7fc9ae7fc700 1 -- 172.21.15.12:0/3734874123 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a8018730 con 0x7fc9b8102300 2024-04-22T13:22:42.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.439+0000 7fc9bf2de700 1 -- 172.21.15.12:0/3734874123 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9b81a5c60 con 0x7fc9b8102300 2024-04-22T13:22:42.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.440+0000 7fc9bf2de700 1 -- 172.21.15.12:0/3734874123 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc9b8063110 con 0x7fc9b8102300 2024-04-22T13:22:42.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.443+0000 7fc9ae7fc700 1 -- 172.21.15.12:0/3734874123 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc9a800f450 con 0x7fc9b8102300 2024-04-22T13:22:42.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.443+0000 7fc9ae7fc700 1 --2- 172.21.15.12:0/3734874123 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc9a40a3d10 0x7fc9a40a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:42.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.443+0000 7fc9bd07a700 1 --2- 172.21.15.12:0/3734874123 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc9a40a3d10 0x7fc9a40a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:42.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.443+0000 7fc9ae7fc700 1 -- 172.21.15.12:0/3734874123 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fc9a80d0910 con 0x7fc9b8102300 2024-04-22T13:22:42.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.444+0000 7fc9ae7fc700 1 --2- 172.21.15.12:0/3734874123 >> [v2:172.21.15.111:6824/3338566976,v1:172.21.15.111:6825/3338566976] conn(0x7fc9a40ad4f0 0x7fc9a40af910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:42.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.444+0000 7fc9ae7fc700 1 -- 172.21.15.12:0/3734874123 --> [v2:172.21.15.111:6824/3338566976,v1:172.21.15.111:6825/3338566976] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc9a8007e50 con 0x7fc9a40ad4f0 2024-04-22T13:22:42.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.444+0000 7fc9ae7fc700 1 -- 172.21.15.12:0/3734874123 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fc9a80d0df0 con 0x7fc9b8102300 2024-04-22T13:22:42.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.444+0000 7fc9bd07a700 1 --2- 172.21.15.12:0/3734874123 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc9a40a3d10 0x7fc9a40a61d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc9b400b5c0 tx=0x7fc9b401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:42.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.444+0000 7fc9bd87b700 1 --2- 172.21.15.12:0/3734874123 >> [v2:172.21.15.111:6824/3338566976,v1:172.21.15.111:6825/3338566976] conn(0x7fc9a40ad4f0 0x7fc9a40af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:42.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.444+0000 7fc9bd87b700 1 --2- 172.21.15.12:0/3734874123 >> [v2:172.21.15.111:6824/3338566976,v1:172.21.15.111:6825/3338566976] conn(0x7fc9a40ad4f0 0x7fc9a40af910 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:42.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.445+0000 7fc9ae7fc700 1 -- 172.21.15.12:0/3734874123 <== osd.10 v2:172.21.15.111:6824/3338566976 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc9a8007e50 con 0x7fc9a40ad4f0 2024-04-22T13:22:42.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.462+0000 7fc9bf2de700 1 -- 172.21.15.12:0/3734874123 --> [v2:172.21.15.111:6824/3338566976,v1:172.21.15.111:6825/3338566976] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc9b8063320 con 0x7fc9a40ad4f0 2024-04-22T13:22:42.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.463+0000 7fc9ae7fc700 1 -- 172.21.15.12:0/3734874123 <== osd.10 v2:172.21.15.111:6824/3338566976 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc9b8063320 con 0x7fc9a40ad4f0 2024-04-22T13:22:42.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.463+0000 7fc9bf2de700 1 -- 172.21.15.12:0/3734874123 >> [v2:172.21.15.111:6824/3338566976,v1:172.21.15.111:6825/3338566976] conn(0x7fc9a40ad4f0 msgr2=0x7fc9a40af910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:42.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.463+0000 7fc9bf2de700 1 --2- 172.21.15.12:0/3734874123 >> [v2:172.21.15.111:6824/3338566976,v1:172.21.15.111:6825/3338566976] conn(0x7fc9a40ad4f0 0x7fc9a40af910 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:42.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.463+0000 7fc9bf2de700 1 -- 172.21.15.12:0/3734874123 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc9a40a3d10 msgr2=0x7fc9a40a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:42.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.463+0000 7fc9bf2de700 1 --2- 172.21.15.12:0/3734874123 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc9a40a3d10 0x7fc9a40a61d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc9b400b5c0 tx=0x7fc9b401a040 comp rx=0 tx=0).stop 2024-04-22T13:22:42.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.463+0000 7fc9bf2de700 1 -- 172.21.15.12:0/3734874123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9b8102300 msgr2=0x7fc9b819a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:42.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.463+0000 7fc9bf2de700 1 --2- 172.21.15.12:0/3734874123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc9b8102300 0x7fc9b819a500 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800c9b0 tx=0x7fc9a800cd70 comp rx=0 tx=0).stop 2024-04-22T13:22:42.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.463+0000 7fc9bd87b700 1 -- 172.21.15.12:0/3734874123 reap_dead start 2024-04-22T13:22:42.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.463+0000 7fc9bf2de700 1 -- 172.21.15.12:0/3734874123 shutdown_connections 2024-04-22T13:22:42.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.463+0000 7fc9bf2de700 1 -- 172.21.15.12:0/3734874123 >> 172.21.15.12:0/3734874123 conn(0x7fc9b80788f0 msgr2=0x7fc9b81005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:42.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.463+0000 7fc9bf2de700 1 -- 172.21.15.12:0/3734874123 shutdown_connections 2024-04-22T13:22:42.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:42.464+0000 7fc9bf2de700 1 -- 172.21.15.12:0/3734874123 wait complete. 2024-04-22T13:22:43.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.175+0000 7f682d314700 1 -- 172.21.15.12:0/1776431058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f682806bb30 msgr2=0x7f682810f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:43.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.175+0000 7f682d314700 1 --2- 172.21.15.12:0/1776431058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f682806bb30 0x7f682810f8d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f682000b0a0 tx=0x7f682000b3b0 comp rx=0 tx=0).stop 2024-04-22T13:22:43.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.175+0000 7f682d314700 1 -- 172.21.15.12:0/1776431058 shutdown_connections 2024-04-22T13:22:43.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.175+0000 7f682d314700 1 --2- 172.21.15.12:0/1776431058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f682806bb30 0x7f682810f8d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:43.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.175+0000 7f682d314700 1 --2- 172.21.15.12:0/1776431058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6828108a70 0x7f6828108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:43.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.175+0000 7f682d314700 1 --2- 172.21.15.12:0/1776431058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f682806b170 0x7f682806b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:43.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.175+0000 7f682d314700 1 -- 172.21.15.12:0/1776431058 >> 172.21.15.12:0/1776431058 conn(0x7f68280777b0 msgr2=0x7f6828077bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:43.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.175+0000 7f682d314700 1 -- 172.21.15.12:0/1776431058 shutdown_connections 2024-04-22T13:22:43.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.175+0000 7f682d314700 1 -- 172.21.15.12:0/1776431058 wait complete. 2024-04-22T13:22:43.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.176+0000 7f682d314700 1 Processor -- start 2024-04-22T13:22:43.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.176+0000 7f682d314700 1 -- start start 2024-04-22T13:22:43.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.176+0000 7f682d314700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f682806b170 0x7f6828101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:43.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.176+0000 7f682d314700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f682806bb30 0x7f68281022a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:43.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.176+0000 7f682d314700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6828108a70 0x7f68281027e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:43.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.176+0000 7f682d314700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6828112cb0 con 0x7f6828108a70 2024-04-22T13:22:43.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.176+0000 7f682d314700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6828112b30 con 0x7f682806b170 2024-04-22T13:22:43.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.176+0000 7f682d314700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6828112e30 con 0x7f682806bb30 2024-04-22T13:22:43.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.176+0000 7f68277fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6828108a70 0x7f68281027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:43.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.176+0000 7f68277fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6828108a70 0x7f68281027e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34036/0 (socket says 172.21.15.12:34036) 2024-04-22T13:22:43.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.176+0000 7f68277fe700 1 -- 172.21.15.12:0/58483561 learned_addr learned my addr 172.21.15.12:0/58483561 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:43.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.176+0000 7f6826ffd700 1 --2- 172.21.15.12:0/58483561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f682806b170 0x7f6828101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:43.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.177+0000 7f68267fc700 1 --2- 172.21.15.12:0/58483561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f682806bb30 0x7f68281022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:43.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.177+0000 7f68277fe700 1 -- 172.21.15.12:0/58483561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f682806bb30 msgr2=0x7f68281022a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:43.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.177+0000 7f68277fe700 1 --2- 172.21.15.12:0/58483561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f682806bb30 0x7f68281022a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:43.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.177+0000 7f68277fe700 1 -- 172.21.15.12:0/58483561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f682806b170 msgr2=0x7f6828101d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:43.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.177+0000 7f68277fe700 1 --2- 172.21.15.12:0/58483561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f682806b170 0x7f6828101d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:43.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.177+0000 7f68277fe700 1 -- 172.21.15.12:0/58483561 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68281a7650 con 0x7f6828108a70 2024-04-22T13:22:43.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.177+0000 7f68277fe700 1 --2- 172.21.15.12:0/58483561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6828108a70 0x7f68281027e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f68200062a0 tx=0x7f6820004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:43.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.177+0000 7f681ffff700 1 -- 172.21.15.12:0/58483561 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6820021070 con 0x7f6828108a70 2024-04-22T13:22:43.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.177+0000 7f682d314700 1 -- 172.21.15.12:0/58483561 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68281a78e0 con 0x7f6828108a70 2024-04-22T13:22:43.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.177+0000 7f681ffff700 1 -- 172.21.15.12:0/58483561 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6820007b80 con 0x7f6828108a70 2024-04-22T13:22:43.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.177+0000 7f681ffff700 1 -- 172.21.15.12:0/58483561 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68200052a0 con 0x7f6828108a70 2024-04-22T13:22:43.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.177+0000 7f682d314700 1 -- 172.21.15.12:0/58483561 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68281a7de0 con 0x7f6828108a70 2024-04-22T13:22:43.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.178+0000 7f682d314700 1 -- 172.21.15.12:0/58483561 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6828063150 con 0x7f6828108a70 2024-04-22T13:22:43.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.179+0000 7f681ffff700 1 -- 172.21.15.12:0/58483561 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6820024070 con 0x7f6828108a70 2024-04-22T13:22:43.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.180+0000 7f681ffff700 1 --2- 172.21.15.12:0/58483561 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f68140a3cd0 0x7f68140a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:43.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.180+0000 7f681ffff700 1 -- 172.21.15.12:0/58483561 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f682000f020 con 0x7f6828108a70 2024-04-22T13:22:43.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.180+0000 7f6826ffd700 1 --2- 172.21.15.12:0/58483561 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f68140a3cd0 0x7f68140a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:43.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.180+0000 7f681ffff700 1 --2- 172.21.15.12:0/58483561 >> [v2:172.21.15.134:6800/1403153998,v1:172.21.15.134:6801/1403153998] conn(0x7f68140ad4b0 0x7f68140af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:43.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.180+0000 7f681ffff700 1 -- 172.21.15.12:0/58483561 --> [v2:172.21.15.134:6800/1403153998,v1:172.21.15.134:6801/1403153998] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f68140aff80 con 0x7f68140ad4b0 2024-04-22T13:22:43.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.180+0000 7f681ffff700 1 -- 172.21.15.12:0/58483561 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f68200d20a0 con 0x7f6828108a70 2024-04-22T13:22:43.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.180+0000 7f6826ffd700 1 --2- 172.21.15.12:0/58483561 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f68140a3cd0 0x7f68140a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6818009a10 tx=0x7f6818009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:43.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.181+0000 7f68267fc700 1 --2- 172.21.15.12:0/58483561 >> [v2:172.21.15.134:6800/1403153998,v1:172.21.15.134:6801/1403153998] conn(0x7f68140ad4b0 0x7f68140af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:43.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.181+0000 7f68267fc700 1 --2- 172.21.15.12:0/58483561 >> [v2:172.21.15.134:6800/1403153998,v1:172.21.15.134:6801/1403153998] conn(0x7f68140ad4b0 0x7f68140af8d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:43.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.182+0000 7f681ffff700 1 -- 172.21.15.12:0/58483561 <== osd.2 v2:172.21.15.134:6800/1403153998 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f68140aff80 con 0x7f68140ad4b0 2024-04-22T13:22:43.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.198+0000 7f682d314700 1 -- 172.21.15.12:0/58483561 --> [v2:172.21.15.134:6800/1403153998,v1:172.21.15.134:6801/1403153998] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f68280635d0 con 0x7f68140ad4b0 2024-04-22T13:22:43.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.199+0000 7f681ffff700 1 -- 172.21.15.12:0/58483561 <== osd.2 v2:172.21.15.134:6800/1403153998 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f68280635d0 con 0x7f68140ad4b0 2024-04-22T13:22:43.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.199+0000 7f682d314700 1 -- 172.21.15.12:0/58483561 >> [v2:172.21.15.134:6800/1403153998,v1:172.21.15.134:6801/1403153998] conn(0x7f68140ad4b0 msgr2=0x7f68140af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:43.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.199+0000 7f682d314700 1 --2- 172.21.15.12:0/58483561 >> [v2:172.21.15.134:6800/1403153998,v1:172.21.15.134:6801/1403153998] conn(0x7f68140ad4b0 0x7f68140af8d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:43.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.199+0000 7f682d314700 1 -- 172.21.15.12:0/58483561 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f68140a3cd0 msgr2=0x7f68140a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:43.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.199+0000 7f682d314700 1 --2- 172.21.15.12:0/58483561 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f68140a3cd0 0x7f68140a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6818009a10 tx=0x7f6818009190 comp rx=0 tx=0).stop 2024-04-22T13:22:43.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.199+0000 7f682d314700 1 -- 172.21.15.12:0/58483561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6828108a70 msgr2=0x7f68281027e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:43.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.199+0000 7f682d314700 1 --2- 172.21.15.12:0/58483561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6828108a70 0x7f68281027e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f68200062a0 tx=0x7f6820004ce0 comp rx=0 tx=0).stop 2024-04-22T13:22:43.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.199+0000 7f68277fe700 1 -- 172.21.15.12:0/58483561 reap_dead start 2024-04-22T13:22:43.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.199+0000 7f682d314700 1 -- 172.21.15.12:0/58483561 shutdown_connections 2024-04-22T13:22:43.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.199+0000 7f682d314700 1 -- 172.21.15.12:0/58483561 >> 172.21.15.12:0/58483561 conn(0x7f68280777b0 msgr2=0x7f6828100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:43.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.199+0000 7f682d314700 1 -- 172.21.15.12:0/58483561 shutdown_connections 2024-04-22T13:22:43.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:43.199+0000 7f682d314700 1 -- 172.21.15.12:0/58483561 wait complete. 2024-04-22T13:22:43.653 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:43 smithi012 ceph-mon[28265]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-22T13:22:43.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:43 smithi111 ceph-mon[31361]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-22T13:22:43.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:43 smithi134 ceph-mon[31338]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-22T13:22:45.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:45 smithi012 ceph-mon[28265]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-22T13:22:45.685 INFO:teuthology.orchestra.run.smithi012.stdout:261993005091 2024-04-22T13:22:45.685 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-22T13:22:45.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:45 smithi111 ceph-mon[31361]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-22T13:22:45.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:45 smithi134 ceph-mon[31338]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-22T13:22:46.552 INFO:teuthology.orchestra.run.smithi012.stdout:77309411395 2024-04-22T13:22:46.552 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-22T13:22:46.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.817+0000 7f784e20f700 1 -- 172.21.15.12:0/598065479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7848105830 msgr2=0x7f7848109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:46.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.817+0000 7f784e20f700 1 --2- 172.21.15.12:0/598065479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7848105830 0x7f7848109880 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f783c0099c0 tx=0x7f783c009cd0 comp rx=0 tx=0).stop 2024-04-22T13:22:46.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.817+0000 7f784e20f700 1 -- 172.21.15.12:0/598065479 shutdown_connections 2024-04-22T13:22:46.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.817+0000 7f784e20f700 1 --2- 172.21.15.12:0/598065479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7848104e80 0x7f7848105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:46.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.817+0000 7f784e20f700 1 --2- 172.21.15.12:0/598065479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7848109f60 0x7f7848111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:46.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.817+0000 7f784e20f700 1 --2- 172.21.15.12:0/598065479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7848105830 0x7f7848109880 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:46.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.817+0000 7f784e20f700 1 -- 172.21.15.12:0/598065479 >> 172.21.15.12:0/598065479 conn(0x7f78481006f0 msgr2=0x7f7848102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:46.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.817+0000 7f784e20f700 1 -- 172.21.15.12:0/598065479 shutdown_connections 2024-04-22T13:22:46.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.817+0000 7f784e20f700 1 -- 172.21.15.12:0/598065479 wait complete. 2024-04-22T13:22:46.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.818+0000 7f784e20f700 1 Processor -- start 2024-04-22T13:22:46.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.818+0000 7f784e20f700 1 -- start start 2024-04-22T13:22:46.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.818+0000 7f784e20f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7848104e80 0x7f784819b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:46.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.818+0000 7f784e20f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7848105830 0x7f784819b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:46.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.818+0000 7f784e20f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7848109f60 0x7f784819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:46.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.818+0000 7f784e20f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7848114ea0 con 0x7f7848105830 2024-04-22T13:22:46.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.818+0000 7f784e20f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7848114d20 con 0x7f7848104e80 2024-04-22T13:22:46.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.818+0000 7f784e20f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7848115020 con 0x7f7848109f60 2024-04-22T13:22:46.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.818+0000 7f7846ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7848105830 0x7f784819b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:46.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.818+0000 7f7846ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7848105830 0x7f784819b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34098/0 (socket says 172.21.15.12:34098) 2024-04-22T13:22:46.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.818+0000 7f7846ffd700 1 -- 172.21.15.12:0/2952891746 learned_addr learned my addr 172.21.15.12:0/2952891746 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:46.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.819+0000 7f78477fe700 1 --2- 172.21.15.12:0/2952891746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7848104e80 0x7f784819b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:46.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.819+0000 7f7846ffd700 1 -- 172.21.15.12:0/2952891746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7848109f60 msgr2=0x7f784819fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:46.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.819+0000 7f7846ffd700 1 --2- 172.21.15.12:0/2952891746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7848109f60 0x7f784819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:46.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.819+0000 7f7846ffd700 1 -- 172.21.15.12:0/2952891746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7848104e80 msgr2=0x7f784819b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:46.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.819+0000 7f7846ffd700 1 --2- 172.21.15.12:0/2952891746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7848104e80 0x7f784819b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:46.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.819+0000 7f7846ffd700 1 -- 172.21.15.12:0/2952891746 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78481a0460 con 0x7f7848105830 2024-04-22T13:22:46.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.819+0000 7f7846ffd700 1 --2- 172.21.15.12:0/2952891746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7848105830 0x7f784819b990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f783c006010 tx=0x7f783c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:46.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.819+0000 7f7844ff9700 1 -- 172.21.15.12:0/2952891746 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c027070 con 0x7f7848105830 2024-04-22T13:22:46.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.819+0000 7f784e20f700 1 -- 172.21.15.12:0/2952891746 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78481a06f0 con 0x7f7848105830 2024-04-22T13:22:46.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.819+0000 7f7844ff9700 1 -- 172.21.15.12:0/2952891746 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f783c003fa0 con 0x7f7848105830 2024-04-22T13:22:46.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.819+0000 7f7844ff9700 1 -- 172.21.15.12:0/2952891746 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c022dd0 con 0x7f7848105830 2024-04-22T13:22:46.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.819+0000 7f784e20f700 1 -- 172.21.15.12:0/2952891746 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78481a7fd0 con 0x7f7848105830 2024-04-22T13:22:46.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.820+0000 7f784e20f700 1 -- 172.21.15.12:0/2952891746 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7848063150 con 0x7f7848105830 2024-04-22T13:22:46.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.821+0000 7f7844ff9700 1 -- 172.21.15.12:0/2952891746 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f783c02a070 con 0x7f7848105830 2024-04-22T13:22:46.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.822+0000 7f7844ff9700 1 --2- 172.21.15.12:0/2952891746 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f78300a3cd0 0x7f78300a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:46.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.822+0000 7f78477fe700 1 --2- 172.21.15.12:0/2952891746 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f78300a3cd0 0x7f78300a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:46.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.822+0000 7f7844ff9700 1 -- 172.21.15.12:0/2952891746 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f783c0d1910 con 0x7f7848105830 2024-04-22T13:22:46.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.822+0000 7f7844ff9700 1 --2- 172.21.15.12:0/2952891746 >> [v2:172.21.15.12:6802/1191350632,v1:172.21.15.12:6803/1191350632] conn(0x7f78300ad4b0 0x7f78300af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:46.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.822+0000 7f7844ff9700 1 -- 172.21.15.12:0/2952891746 --> [v2:172.21.15.12:6802/1191350632,v1:172.21.15.12:6803/1191350632] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f78300aff80 con 0x7f78300ad4b0 2024-04-22T13:22:46.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.822+0000 7f7844ff9700 1 -- 172.21.15.12:0/2952891746 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f783c0d1cc0 con 0x7f7848105830 2024-04-22T13:22:46.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.822+0000 7f7847fff700 1 --2- 172.21.15.12:0/2952891746 >> [v2:172.21.15.12:6802/1191350632,v1:172.21.15.12:6803/1191350632] conn(0x7f78300ad4b0 0x7f78300af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:46.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.822+0000 7f78477fe700 1 --2- 172.21.15.12:0/2952891746 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f78300a3cd0 0x7f78300a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7838006fd0 tx=0x7f7838008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:46.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.823+0000 7f7847fff700 1 --2- 172.21.15.12:0/2952891746 >> [v2:172.21.15.12:6802/1191350632,v1:172.21.15.12:6803/1191350632] conn(0x7f78300ad4b0 0x7f78300af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:46.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.823+0000 7f7844ff9700 1 -- 172.21.15.12:0/2952891746 <== osd.0 v2:172.21.15.12:6802/1191350632 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f78300aff80 con 0x7f78300ad4b0 2024-04-22T13:22:46.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.840+0000 7f784e20f700 1 -- 172.21.15.12:0/2952891746 --> [v2:172.21.15.12:6802/1191350632,v1:172.21.15.12:6803/1191350632] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7848063550 con 0x7f78300ad4b0 2024-04-22T13:22:46.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.841+0000 7f7844ff9700 1 -- 172.21.15.12:0/2952891746 <== osd.0 v2:172.21.15.12:6802/1191350632 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7848063550 con 0x7f78300ad4b0 2024-04-22T13:22:46.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.841+0000 7f784e20f700 1 -- 172.21.15.12:0/2952891746 >> [v2:172.21.15.12:6802/1191350632,v1:172.21.15.12:6803/1191350632] conn(0x7f78300ad4b0 msgr2=0x7f78300af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:46.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.841+0000 7f784e20f700 1 --2- 172.21.15.12:0/2952891746 >> [v2:172.21.15.12:6802/1191350632,v1:172.21.15.12:6803/1191350632] conn(0x7f78300ad4b0 0x7f78300af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:46.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.841+0000 7f784e20f700 1 -- 172.21.15.12:0/2952891746 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f78300a3cd0 msgr2=0x7f78300a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:46.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.841+0000 7f784e20f700 1 --2- 172.21.15.12:0/2952891746 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f78300a3cd0 0x7f78300a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7838006fd0 tx=0x7f7838008040 comp rx=0 tx=0).stop 2024-04-22T13:22:46.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.841+0000 7f784e20f700 1 -- 172.21.15.12:0/2952891746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7848105830 msgr2=0x7f784819b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:46.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.841+0000 7f784e20f700 1 --2- 172.21.15.12:0/2952891746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7848105830 0x7f784819b990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f783c006010 tx=0x7f783c0047b0 comp rx=0 tx=0).stop 2024-04-22T13:22:46.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.841+0000 7f7847fff700 1 -- 172.21.15.12:0/2952891746 reap_dead start 2024-04-22T13:22:46.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.841+0000 7f784e20f700 1 -- 172.21.15.12:0/2952891746 shutdown_connections 2024-04-22T13:22:46.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.841+0000 7f784e20f700 1 -- 172.21.15.12:0/2952891746 >> 172.21.15.12:0/2952891746 conn(0x7f78481006f0 msgr2=0x7f7848101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:46.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.841+0000 7f784e20f700 1 -- 172.21.15.12:0/2952891746 shutdown_connections 2024-04-22T13:22:46.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.842+0000 7f784e20f700 1 -- 172.21.15.12:0/2952891746 wait complete. 2024-04-22T13:22:47.698 INFO:teuthology.orchestra.run.smithi012.stdout:317827579929 2024-04-22T13:22:47.698 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-22T13:22:47.722 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:47 smithi012 ceph-mon[28265]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-22T13:22:47.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:47 smithi111 ceph-mon[31361]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-22T13:22:47.729 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:47 smithi134 ceph-mon[31338]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-22T13:22:48.085 INFO:teuthology.orchestra.run.smithi012.stdout:352187318293 2024-04-22T13:22:48.085 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-22T13:22:48.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.748+0000 7efdcb15f700 1 -- 172.21.15.12:0/4249178890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdc4108a20 msgr2=0x7efdc4108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:48.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.748+0000 7efdcb15f700 1 --2- 172.21.15.12:0/4249178890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdc4108a20 0x7efdc4108e00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009990 tx=0x7efdb8009ca0 comp rx=0 tx=0).stop 2024-04-22T13:22:48.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.748+0000 7efdcb15f700 1 -- 172.21.15.12:0/4249178890 shutdown_connections 2024-04-22T13:22:48.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.748+0000 7efdcb15f700 1 --2- 172.21.15.12:0/4249178890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdc406b980 0x7efdc410f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:48.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.748+0000 7efdcb15f700 1 --2- 172.21.15.12:0/4249178890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdc4108a20 0x7efdc4108e00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:48.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.748+0000 7efdcb15f700 1 --2- 172.21.15.12:0/4249178890 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efdc406afe0 0x7efdc406b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:48.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.748+0000 7efdcb15f700 1 -- 172.21.15.12:0/4249178890 >> 172.21.15.12:0/4249178890 conn(0x7efdc40788b0 msgr2=0x7efdc4078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:48.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.748+0000 7efdcb15f700 1 -- 172.21.15.12:0/4249178890 shutdown_connections 2024-04-22T13:22:48.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.748+0000 7efdcb15f700 1 -- 172.21.15.12:0/4249178890 wait complete. 2024-04-22T13:22:48.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.749+0000 7efdcb15f700 1 Processor -- start 2024-04-22T13:22:48.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.749+0000 7efdcb15f700 1 -- start start 2024-04-22T13:22:48.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.749+0000 7efdcb15f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdc406afe0 0x7efdc41990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:48.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.749+0000 7efdcb15f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdc406b980 0x7efdc4199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:48.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.749+0000 7efdcb15f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efdc4108a20 0x7efdc419d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:48.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.749+0000 7efdcb15f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efdc4112bc0 con 0x7efdc406b980 2024-04-22T13:22:48.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.749+0000 7efdcb15f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efdc4112a40 con 0x7efdc4108a20 2024-04-22T13:22:48.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.749+0000 7efdcb15f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efdc4112d40 con 0x7efdc406afe0 2024-04-22T13:22:48.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.749+0000 7efdc3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdc406b980 0x7efdc4199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:48.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.749+0000 7efdc3fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdc406b980 0x7efdc4199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34070/0 (socket says 172.21.15.12:34070) 2024-04-22T13:22:48.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.749+0000 7efdc3fff700 1 -- 172.21.15.12:0/1951426594 learned_addr learned my addr 172.21.15.12:0/1951426594 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:48.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.749+0000 7efdc96fc700 1 --2- 172.21.15.12:0/1951426594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efdc4108a20 0x7efdc419d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.749+0000 7efdc8efb700 1 --2- 172.21.15.12:0/1951426594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdc406afe0 0x7efdc41990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.750+0000 7efdc3fff700 1 -- 172.21.15.12:0/1951426594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdc406afe0 msgr2=0x7efdc41990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.750+0000 7efdc3fff700 1 --2- 172.21.15.12:0/1951426594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdc406afe0 0x7efdc41990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.750+0000 7efdc3fff700 1 -- 172.21.15.12:0/1951426594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efdc4108a20 msgr2=0x7efdc419d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.750+0000 7efdc3fff700 1 --2- 172.21.15.12:0/1951426594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efdc4108a20 0x7efdc419d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.750+0000 7efdc3fff700 1 -- 172.21.15.12:0/1951426594 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc419e0e0 con 0x7efdc406b980 2024-04-22T13:22:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.750+0000 7efdc3fff700 1 --2- 172.21.15.12:0/1951426594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdc406b980 0x7efdc4199610 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efdb000d980 tx=0x7efdb000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.750+0000 7efdc1ffb700 1 -- 172.21.15.12:0/1951426594 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb0016070 con 0x7efdc406b980 2024-04-22T13:22:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.750+0000 7efdcb15f700 1 -- 172.21.15.12:0/1951426594 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdc419e3d0 con 0x7efdc406b980 2024-04-22T13:22:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.750+0000 7efdc1ffb700 1 -- 172.21.15.12:0/1951426594 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdb0004830 con 0x7efdc406b980 2024-04-22T13:22:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.750+0000 7efdc1ffb700 1 -- 172.21.15.12:0/1951426594 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb0005540 con 0x7efdc406b980 2024-04-22T13:22:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.750+0000 7efdcb15f700 1 -- 172.21.15.12:0/1951426594 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdc4101930 con 0x7efdc406b980 2024-04-22T13:22:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.751+0000 7efdcb15f700 1 -- 172.21.15.12:0/1951426594 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efdc40631d0 con 0x7efdc406b980 2024-04-22T13:22:48.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.752+0000 7efdc1ffb700 1 -- 172.21.15.12:0/1951426594 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7efdb0004220 con 0x7efdc406b980 2024-04-22T13:22:48.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.753+0000 7efdc1ffb700 1 --2- 172.21.15.12:0/1951426594 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efdb40a3cd0 0x7efdb40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:48.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.753+0000 7efdc8efb700 1 --2- 172.21.15.12:0/1951426594 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efdb40a3cd0 0x7efdb40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:48.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.753+0000 7efdc1ffb700 1 -- 172.21.15.12:0/1951426594 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7efdb00d0170 con 0x7efdc406b980 2024-04-22T13:22:48.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.753+0000 7efdc1ffb700 1 --2- 172.21.15.12:0/1951426594 >> [v2:172.21.15.134:6808/3071977602,v1:172.21.15.134:6809/3071977602] conn(0x7efdb40ad4b0 0x7efdb40af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:48.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.753+0000 7efdc1ffb700 1 -- 172.21.15.12:0/1951426594 --> [v2:172.21.15.134:6808/3071977602,v1:172.21.15.134:6809/3071977602] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efdb0013ec0 con 0x7efdb40ad4b0 2024-04-22T13:22:48.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.753+0000 7efdc1ffb700 1 -- 172.21.15.12:0/1951426594 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7efdb00d7050 con 0x7efdc406b980 2024-04-22T13:22:48.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.753+0000 7efdc8efb700 1 --2- 172.21.15.12:0/1951426594 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efdb40a3cd0 0x7efdb40a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efdb80098e0 tx=0x7efdb8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:48.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.754+0000 7efdc96fc700 1 --2- 172.21.15.12:0/1951426594 >> [v2:172.21.15.134:6808/3071977602,v1:172.21.15.134:6809/3071977602] conn(0x7efdb40ad4b0 0x7efdb40af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:48.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.754+0000 7efdc96fc700 1 --2- 172.21.15.12:0/1951426594 >> [v2:172.21.15.134:6808/3071977602,v1:172.21.15.134:6809/3071977602] conn(0x7efdb40ad4b0 0x7efdb40af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:48.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.755+0000 7efdc1ffb700 1 -- 172.21.15.12:0/1951426594 <== osd.5 v2:172.21.15.134:6808/3071977602 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efdb0013ec0 con 0x7efdb40ad4b0 2024-04-22T13:22:48.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.771+0000 7efdcb15f700 1 -- 172.21.15.12:0/1951426594 --> [v2:172.21.15.134:6808/3071977602,v1:172.21.15.134:6809/3071977602] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efdc4063650 con 0x7efdb40ad4b0 2024-04-22T13:22:48.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.772+0000 7efdc1ffb700 1 -- 172.21.15.12:0/1951426594 <== osd.5 v2:172.21.15.134:6808/3071977602 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7efdc4063650 con 0x7efdb40ad4b0 2024-04-22T13:22:48.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.773+0000 7efdcb15f700 1 -- 172.21.15.12:0/1951426594 >> [v2:172.21.15.134:6808/3071977602,v1:172.21.15.134:6809/3071977602] conn(0x7efdb40ad4b0 msgr2=0x7efdb40af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:48.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.773+0000 7efdcb15f700 1 --2- 172.21.15.12:0/1951426594 >> [v2:172.21.15.134:6808/3071977602,v1:172.21.15.134:6809/3071977602] conn(0x7efdb40ad4b0 0x7efdb40af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:48.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.773+0000 7efdcb15f700 1 -- 172.21.15.12:0/1951426594 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efdb40a3cd0 msgr2=0x7efdb40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:48.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.773+0000 7efdcb15f700 1 --2- 172.21.15.12:0/1951426594 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efdb40a3cd0 0x7efdb40a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efdb80098e0 tx=0x7efdb8005960 comp rx=0 tx=0).stop 2024-04-22T13:22:48.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.773+0000 7efdcb15f700 1 -- 172.21.15.12:0/1951426594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdc406b980 msgr2=0x7efdc4199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:48.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.773+0000 7efdcb15f700 1 --2- 172.21.15.12:0/1951426594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdc406b980 0x7efdc4199610 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efdb000d980 tx=0x7efdb000dc90 comp rx=0 tx=0).stop 2024-04-22T13:22:48.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.773+0000 7efdc96fc700 1 -- 172.21.15.12:0/1951426594 reap_dead start 2024-04-22T13:22:48.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.773+0000 7efdcb15f700 1 -- 172.21.15.12:0/1951426594 shutdown_connections 2024-04-22T13:22:48.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.773+0000 7efdcb15f700 1 -- 172.21.15.12:0/1951426594 >> 172.21.15.12:0/1951426594 conn(0x7efdc40788b0 msgr2=0x7efdc40ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:48.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.773+0000 7efdcb15f700 1 -- 172.21.15.12:0/1951426594 shutdown_connections 2024-04-22T13:22:48.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.773+0000 7efdcb15f700 1 -- 172.21.15.12:0/1951426594 wait complete. 2024-04-22T13:22:49.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.219+0000 7f9398102700 1 -- 172.21.15.12:0/2647563060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9390108a50 msgr2=0x7f9390108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:49.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.219+0000 7f9398102700 1 --2- 172.21.15.12:0/2647563060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9390108a50 0x7f9390108e30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9380009990 tx=0x7f9380009ca0 comp rx=0 tx=0).stop 2024-04-22T13:22:49.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.219+0000 7f9398102700 1 -- 172.21.15.12:0/2647563060 shutdown_connections 2024-04-22T13:22:49.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.219+0000 7f9398102700 1 --2- 172.21.15.12:0/2647563060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f939006bab0 0x7f939010f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:49.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.219+0000 7f9398102700 1 --2- 172.21.15.12:0/2647563060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9390108a50 0x7f9390108e30 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:49.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.219+0000 7f9398102700 1 --2- 172.21.15.12:0/2647563060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f939006b110 0x7f939006b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:49.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.219+0000 7f9398102700 1 -- 172.21.15.12:0/2647563060 >> 172.21.15.12:0/2647563060 conn(0x7f93900788f0 msgr2=0x7f9390078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:49.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.219+0000 7f9398102700 1 -- 172.21.15.12:0/2647563060 shutdown_connections 2024-04-22T13:22:49.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.219+0000 7f9398102700 1 -- 172.21.15.12:0/2647563060 wait complete. 2024-04-22T13:22:49.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.220+0000 7f9398102700 1 Processor -- start 2024-04-22T13:22:49.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.220+0000 7f9398102700 1 -- start start 2024-04-22T13:22:49.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.220+0000 7f9398102700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f939006b110 0x7f9390199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:49.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.220+0000 7f9398102700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f939006bab0 0x7f93901997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:49.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.220+0000 7f9398102700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9390108a50 0x7f939019db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:49.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.220+0000 7f9398102700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9390112cd0 con 0x7f9390108a50 2024-04-22T13:22:49.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.220+0000 7f9398102700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9390112b50 con 0x7f939006bab0 2024-04-22T13:22:49.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.220+0000 7f9398102700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9390112e50 con 0x7f939006b110 2024-04-22T13:22:49.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.220+0000 7f939669f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9390108a50 0x7f939019db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:49.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.220+0000 7f939669f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9390108a50 0x7f939019db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34108/0 (socket says 172.21.15.12:34108) 2024-04-22T13:22:49.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.220+0000 7f939669f700 1 -- 172.21.15.12:0/1537924979 learned_addr learned my addr 172.21.15.12:0/1537924979 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:49.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.221+0000 7f939569d700 1 --2- 172.21.15.12:0/1537924979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f939006bab0 0x7f93901997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:49.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.221+0000 7f9395e9e700 1 --2- 172.21.15.12:0/1537924979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f939006b110 0x7f9390199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:49.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.221+0000 7f939669f700 1 -- 172.21.15.12:0/1537924979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f939006b110 msgr2=0x7f9390199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:49.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.221+0000 7f939669f700 1 --2- 172.21.15.12:0/1537924979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f939006b110 0x7f9390199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:49.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.221+0000 7f939669f700 1 -- 172.21.15.12:0/1537924979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f939006bab0 msgr2=0x7f93901997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:49.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.221+0000 7f939669f700 1 --2- 172.21.15.12:0/1537924979 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f939006bab0 0x7f93901997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:49.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.221+0000 7f939669f700 1 -- 172.21.15.12:0/1537924979 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939019e290 con 0x7f9390108a50 2024-04-22T13:22:49.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.221+0000 7f9395e9e700 1 --2- 172.21.15.12:0/1537924979 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f939006b110 0x7f9390199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:22:49.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.221+0000 7f939669f700 1 --2- 172.21.15.12:0/1537924979 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9390108a50 0x7f939019db50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f938800b970 tx=0x7f938800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:49.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.221+0000 7f938effd700 1 -- 172.21.15.12:0/1537924979 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388019070 con 0x7f9390108a50 2024-04-22T13:22:49.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.221+0000 7f9398102700 1 -- 172.21.15.12:0/1537924979 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f939019e580 con 0x7f9390108a50 2024-04-22T13:22:49.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.221+0000 7f938effd700 1 -- 172.21.15.12:0/1537924979 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f938800cde0 con 0x7f9390108a50 2024-04-22T13:22:49.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.221+0000 7f938effd700 1 -- 172.21.15.12:0/1537924979 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388007370 con 0x7f9390108a50 2024-04-22T13:22:49.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.221+0000 7f9398102700 1 -- 172.21.15.12:0/1537924979 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93901a5e00 con 0x7f9390108a50 2024-04-22T13:22:49.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.222+0000 7f9398102700 1 -- 172.21.15.12:0/1537924979 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9390063150 con 0x7f9390108a50 2024-04-22T13:22:49.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.223+0000 7f938effd700 1 -- 172.21.15.12:0/1537924979 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f938801c070 con 0x7f9390108a50 2024-04-22T13:22:49.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.224+0000 7f938effd700 1 --2- 172.21.15.12:0/1537924979 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f93780a3cd0 0x7f93780a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:49.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.224+0000 7f9395e9e700 1 --2- 172.21.15.12:0/1537924979 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f93780a3cd0 0x7f93780a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:49.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.224+0000 7f938effd700 1 -- 172.21.15.12:0/1537924979 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f93880d01d0 con 0x7f9390108a50 2024-04-22T13:22:49.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.224+0000 7f938effd700 1 --2- 172.21.15.12:0/1537924979 >> [v2:172.21.15.111:6800/417496465,v1:172.21.15.111:6801/417496465] conn(0x7f93780ad4b0 0x7f93780af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:49.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.224+0000 7f938effd700 1 -- 172.21.15.12:0/1537924979 --> [v2:172.21.15.111:6800/417496465,v1:172.21.15.111:6801/417496465] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f93780aff80 con 0x7f93780ad4b0 2024-04-22T13:22:49.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.224+0000 7f938effd700 1 -- 172.21.15.12:0/1537924979 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f93880d06b0 con 0x7f9390108a50 2024-04-22T13:22:49.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.224+0000 7f9395e9e700 1 --2- 172.21.15.12:0/1537924979 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f93780a3cd0 0x7f93780a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9380006010 tx=0x7f938000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:49.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.225+0000 7f939569d700 1 --2- 172.21.15.12:0/1537924979 >> [v2:172.21.15.111:6800/417496465,v1:172.21.15.111:6801/417496465] conn(0x7f93780ad4b0 0x7f93780af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:49.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.225+0000 7f939569d700 1 --2- 172.21.15.12:0/1537924979 >> [v2:172.21.15.111:6800/417496465,v1:172.21.15.111:6801/417496465] conn(0x7f93780ad4b0 0x7f93780af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:49.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.226+0000 7f938effd700 1 -- 172.21.15.12:0/1537924979 <== osd.1 v2:172.21.15.111:6800/417496465 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f93780aff80 con 0x7f93780ad4b0 2024-04-22T13:22:49.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.242+0000 7f9398102700 1 -- 172.21.15.12:0/1537924979 --> [v2:172.21.15.111:6800/417496465,v1:172.21.15.111:6801/417496465] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f93900635d0 con 0x7f93780ad4b0 2024-04-22T13:22:49.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.243+0000 7f938effd700 1 -- 172.21.15.12:0/1537924979 <== osd.1 v2:172.21.15.111:6800/417496465 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f93900635d0 con 0x7f93780ad4b0 2024-04-22T13:22:49.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.243+0000 7f9398102700 1 -- 172.21.15.12:0/1537924979 >> [v2:172.21.15.111:6800/417496465,v1:172.21.15.111:6801/417496465] conn(0x7f93780ad4b0 msgr2=0x7f93780af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:49.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.243+0000 7f9398102700 1 --2- 172.21.15.12:0/1537924979 >> [v2:172.21.15.111:6800/417496465,v1:172.21.15.111:6801/417496465] conn(0x7f93780ad4b0 0x7f93780af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:49.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.243+0000 7f9398102700 1 -- 172.21.15.12:0/1537924979 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f93780a3cd0 msgr2=0x7f93780a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:49.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.243+0000 7f9398102700 1 --2- 172.21.15.12:0/1537924979 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f93780a3cd0 0x7f93780a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9380006010 tx=0x7f938000b540 comp rx=0 tx=0).stop 2024-04-22T13:22:49.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.243+0000 7f9398102700 1 -- 172.21.15.12:0/1537924979 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9390108a50 msgr2=0x7f939019db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:49.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.243+0000 7f9398102700 1 --2- 172.21.15.12:0/1537924979 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9390108a50 0x7f939019db50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f938800b970 tx=0x7f938800bc80 comp rx=0 tx=0).stop 2024-04-22T13:22:49.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.243+0000 7f939669f700 1 -- 172.21.15.12:0/1537924979 reap_dead start 2024-04-22T13:22:49.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.243+0000 7f9398102700 1 -- 172.21.15.12:0/1537924979 shutdown_connections 2024-04-22T13:22:49.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.243+0000 7f9398102700 1 -- 172.21.15.12:0/1537924979 >> 172.21.15.12:0/1537924979 conn(0x7f93900788f0 msgr2=0x7f93900ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:49.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.244+0000 7f9398102700 1 -- 172.21.15.12:0/1537924979 shutdown_connections 2024-04-22T13:22:49.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:49.244+0000 7f9398102700 1 -- 172.21.15.12:0/1537924979 wait complete. 2024-04-22T13:22:49.562 INFO:teuthology.orchestra.run.smithi012.stdout:34359738444 2024-04-22T13:22:49.563 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-22T13:22:49.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:49 smithi111 ceph-mon[31361]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-22T13:22:49.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:49 smithi012 ceph-mon[28265]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-22T13:22:49.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:49 smithi134 ceph-mon[31338]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-22T13:22:49.987 INFO:teuthology.orchestra.run.smithi012.stdout:171798691891 2024-04-22T13:22:49.987 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-22T13:22:50.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.542+0000 7f19e8037700 1 -- 172.21.15.12:0/2028923029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19e0103450 msgr2=0x7f19e010f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:50.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.542+0000 7f19e8037700 1 --2- 172.21.15.12:0/2028923029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19e0103450 0x7f19e010f8b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f19d800b0a0 tx=0x7f19d800b3b0 comp rx=0 tx=0).stop 2024-04-22T13:22:50.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.542+0000 7f19e8037700 1 -- 172.21.15.12:0/2028923029 shutdown_connections 2024-04-22T13:22:50.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.542+0000 7f19e8037700 1 --2- 172.21.15.12:0/2028923029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19e0103450 0x7f19e010f8b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:50.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.542+0000 7f19e8037700 1 --2- 172.21.15.12:0/2028923029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19e0108a50 0x7f19e0108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:50.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.542+0000 7f19e8037700 1 --2- 172.21.15.12:0/2028923029 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19e0102a90 0x7f19e0102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:50.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.542+0000 7f19e8037700 1 -- 172.21.15.12:0/2028923029 >> 172.21.15.12:0/2028923029 conn(0x7f19e0079ba0 msgr2=0x7f19e0079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:50.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.542+0000 7f19e8037700 1 -- 172.21.15.12:0/2028923029 shutdown_connections 2024-04-22T13:22:50.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.542+0000 7f19e8037700 1 -- 172.21.15.12:0/2028923029 wait complete. 2024-04-22T13:22:50.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.543+0000 7f19e8037700 1 Processor -- start 2024-04-22T13:22:50.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.543+0000 7f19e8037700 1 -- start start 2024-04-22T13:22:50.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.543+0000 7f19e8037700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19e0102a90 0x7f19e0199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:50.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.543+0000 7f19e8037700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19e0103450 0x7f19e0199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:50.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.543+0000 7f19e8037700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19e0108a50 0x7f19e019db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:50.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.543+0000 7f19e8037700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f19e0112ce0 con 0x7f19e0102a90 2024-04-22T13:22:50.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.543+0000 7f19e8037700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f19e0112b60 con 0x7f19e0103450 2024-04-22T13:22:50.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.543+0000 7f19e8037700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f19e0112e60 con 0x7f19e0108a50 2024-04-22T13:22:50.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.543+0000 7f19e5dd3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19e0102a90 0x7f19e0199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:50.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.543+0000 7f19e5dd3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19e0102a90 0x7f19e0199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34120/0 (socket says 172.21.15.12:34120) 2024-04-22T13:22:50.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.543+0000 7f19e5dd3700 1 -- 172.21.15.12:0/1079831684 learned_addr learned my addr 172.21.15.12:0/1079831684 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:50.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.543+0000 7f19e65d4700 1 --2- 172.21.15.12:0/1079831684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19e0108a50 0x7f19e019db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:50.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.543+0000 7f19e55d2700 1 --2- 172.21.15.12:0/1079831684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19e0103450 0x7f19e0199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:50.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.544+0000 7f19e5dd3700 1 -- 172.21.15.12:0/1079831684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19e0108a50 msgr2=0x7f19e019db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:50.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.544+0000 7f19e5dd3700 1 --2- 172.21.15.12:0/1079831684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19e0108a50 0x7f19e019db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:50.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.544+0000 7f19e5dd3700 1 -- 172.21.15.12:0/1079831684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19e0103450 msgr2=0x7f19e0199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:50.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.544+0000 7f19e5dd3700 1 --2- 172.21.15.12:0/1079831684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19e0103450 0x7f19e0199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:50.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.544+0000 7f19e5dd3700 1 -- 172.21.15.12:0/1079831684 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e019e260 con 0x7f19e0102a90 2024-04-22T13:22:50.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.544+0000 7f19e5dd3700 1 --2- 172.21.15.12:0/1079831684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19e0102a90 0x7f19e0199250 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00b530 tx=0x7f19dc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:50.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.544+0000 7f19d6ffd700 1 -- 172.21.15.12:0/1079831684 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc015020 con 0x7f19e0102a90 2024-04-22T13:22:50.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.544+0000 7f19e8037700 1 -- 172.21.15.12:0/1079831684 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e019e550 con 0x7f19e0102a90 2024-04-22T13:22:50.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.544+0000 7f19d6ffd700 1 -- 172.21.15.12:0/1079831684 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19dc011ce0 con 0x7f19e0102a90 2024-04-22T13:22:50.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.544+0000 7f19d6ffd700 1 -- 172.21.15.12:0/1079831684 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc010e40 con 0x7f19e0102a90 2024-04-22T13:22:50.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.544+0000 7f19e8037700 1 -- 172.21.15.12:0/1079831684 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e01a5dd0 con 0x7f19e0102a90 2024-04-22T13:22:50.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.545+0000 7f19e8037700 1 -- 172.21.15.12:0/1079831684 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f19e00631d0 con 0x7f19e0102a90 2024-04-22T13:22:50.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.546+0000 7f19d6ffd700 1 -- 172.21.15.12:0/1079831684 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f19dc018020 con 0x7f19e0102a90 2024-04-22T13:22:50.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.547+0000 7f19d6ffd700 1 --2- 172.21.15.12:0/1079831684 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f19cc0a3cd0 0x7f19cc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:50.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.547+0000 7f19e55d2700 1 --2- 172.21.15.12:0/1079831684 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f19cc0a3cd0 0x7f19cc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:50.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.547+0000 7f19d6ffd700 1 -- 172.21.15.12:0/1079831684 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f19dc0d0390 con 0x7f19e0102a90 2024-04-22T13:22:50.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.547+0000 7f19d6ffd700 1 --2- 172.21.15.12:0/1079831684 >> [v2:172.21.15.12:6818/3819619299,v1:172.21.15.12:6819/3819619299] conn(0x7f19cc0ad4b0 0x7f19cc0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:50.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.547+0000 7f19d6ffd700 1 -- 172.21.15.12:0/1079831684 --> [v2:172.21.15.12:6818/3819619299,v1:172.21.15.12:6819/3819619299] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f19cc0aff80 con 0x7f19cc0ad4b0 2024-04-22T13:22:50.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.547+0000 7f19d6ffd700 1 -- 172.21.15.12:0/1079831684 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f19dc0d0740 con 0x7f19e0102a90 2024-04-22T13:22:50.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.547+0000 7f19e65d4700 1 --2- 172.21.15.12:0/1079831684 >> [v2:172.21.15.12:6818/3819619299,v1:172.21.15.12:6819/3819619299] conn(0x7f19cc0ad4b0 0x7f19cc0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:50.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.547+0000 7f19e55d2700 1 --2- 172.21.15.12:0/1079831684 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f19cc0a3cd0 0x7f19cc0a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f19d0006fd0 tx=0x7f19d0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:50.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.548+0000 7f19e65d4700 1 --2- 172.21.15.12:0/1079831684 >> [v2:172.21.15.12:6818/3819619299,v1:172.21.15.12:6819/3819619299] conn(0x7f19cc0ad4b0 0x7f19cc0af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:50.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.548+0000 7f19d6ffd700 1 -- 172.21.15.12:0/1079831684 <== osd.6 v2:172.21.15.12:6818/3819619299 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f19cc0aff80 con 0x7f19cc0ad4b0 2024-04-22T13:22:50.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.565+0000 7f19e8037700 1 -- 172.21.15.12:0/1079831684 --> [v2:172.21.15.12:6818/3819619299,v1:172.21.15.12:6819/3819619299] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f19e0063650 con 0x7f19cc0ad4b0 2024-04-22T13:22:50.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.565+0000 7f19d6ffd700 1 -- 172.21.15.12:0/1079831684 <== osd.6 v2:172.21.15.12:6818/3819619299 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f19e0063650 con 0x7f19cc0ad4b0 2024-04-22T13:22:50.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.565+0000 7f19e8037700 1 -- 172.21.15.12:0/1079831684 >> [v2:172.21.15.12:6818/3819619299,v1:172.21.15.12:6819/3819619299] conn(0x7f19cc0ad4b0 msgr2=0x7f19cc0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:50.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.565+0000 7f19e8037700 1 --2- 172.21.15.12:0/1079831684 >> [v2:172.21.15.12:6818/3819619299,v1:172.21.15.12:6819/3819619299] conn(0x7f19cc0ad4b0 0x7f19cc0af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:50.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.565+0000 7f19e8037700 1 -- 172.21.15.12:0/1079831684 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f19cc0a3cd0 msgr2=0x7f19cc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:50.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.565+0000 7f19e8037700 1 --2- 172.21.15.12:0/1079831684 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f19cc0a3cd0 0x7f19cc0a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f19d0006fd0 tx=0x7f19d0008040 comp rx=0 tx=0).stop 2024-04-22T13:22:50.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.565+0000 7f19e8037700 1 -- 172.21.15.12:0/1079831684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19e0102a90 msgr2=0x7f19e0199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:50.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.565+0000 7f19e8037700 1 --2- 172.21.15.12:0/1079831684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19e0102a90 0x7f19e0199250 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00b530 tx=0x7f19dc00b840 comp rx=0 tx=0).stop 2024-04-22T13:22:50.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.566+0000 7f19e65d4700 1 -- 172.21.15.12:0/1079831684 reap_dead start 2024-04-22T13:22:50.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.566+0000 7f19e8037700 1 -- 172.21.15.12:0/1079831684 shutdown_connections 2024-04-22T13:22:50.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.566+0000 7f19e8037700 1 -- 172.21.15.12:0/1079831684 >> 172.21.15.12:0/1079831684 conn(0x7f19e0079ba0 msgr2=0x7f19e0101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:50.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.566+0000 7f19e8037700 1 -- 172.21.15.12:0/1079831684 shutdown_connections 2024-04-22T13:22:50.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:50.566+0000 7f19e8037700 1 -- 172.21.15.12:0/1079831684 wait complete. 2024-04-22T13:22:50.977 INFO:teuthology.orchestra.run.smithi012.stdout:55834574920 2024-04-22T13:22:50.977 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-22T13:22:51.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.258+0000 7f133e974700 1 -- 172.21.15.12:0/515465946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1338102ac0 msgr2=0x7f1338102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:51.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.258+0000 7f133e974700 1 --2- 172.21.15.12:0/515465946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1338102ac0 0x7f1338102f40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f132c009990 tx=0x7f132c009ca0 comp rx=0 tx=0).stop 2024-04-22T13:22:51.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.258+0000 7f133e974700 1 -- 172.21.15.12:0/515465946 shutdown_connections 2024-04-22T13:22:51.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.258+0000 7f133e974700 1 --2- 172.21.15.12:0/515465946 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1338103480 0x7f133810f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:51.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.258+0000 7f133e974700 1 --2- 172.21.15.12:0/515465946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1338108ab0 0x7f1338108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:51.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.258+0000 7f133e974700 1 --2- 172.21.15.12:0/515465946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1338102ac0 0x7f1338102f40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:51.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.258+0000 7f133e974700 1 -- 172.21.15.12:0/515465946 >> 172.21.15.12:0/515465946 conn(0x7f1338079bf0 msgr2=0x7f133807a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:51.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.258+0000 7f133e974700 1 -- 172.21.15.12:0/515465946 shutdown_connections 2024-04-22T13:22:51.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.258+0000 7f133e974700 1 -- 172.21.15.12:0/515465946 wait complete. 2024-04-22T13:22:51.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.259+0000 7f133e974700 1 Processor -- start 2024-04-22T13:22:51.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.259+0000 7f133e974700 1 -- start start 2024-04-22T13:22:51.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.259+0000 7f133e974700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1338102ac0 0x7f1338074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:51.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.259+0000 7f133e974700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1338103480 0x7f1338075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:51.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.259+0000 7f133e974700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1338108ab0 0x7f13380757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:51.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.259+0000 7f133e974700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1338112d50 con 0x7f1338108ab0 2024-04-22T13:22:51.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.259+0000 7f133e974700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1338112bd0 con 0x7f1338102ac0 2024-04-22T13:22:51.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.259+0000 7f133e974700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1338112ed0 con 0x7f1338103480 2024-04-22T13:22:51.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.260+0000 7f133cf11700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1338108ab0 0x7f13380757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:51.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.260+0000 7f133cf11700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1338108ab0 0x7f13380757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34058/0 (socket says 172.21.15.12:34058) 2024-04-22T13:22:51.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.260+0000 7f133cf11700 1 -- 172.21.15.12:0/1351675800 learned_addr learned my addr 172.21.15.12:0/1351675800 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:51.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.260+0000 7f1337fff700 1 --2- 172.21.15.12:0/1351675800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1338102ac0 0x7f1338074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:51.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.260+0000 7f13377fe700 1 --2- 172.21.15.12:0/1351675800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1338103480 0x7f1338075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:51.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.260+0000 7f133cf11700 1 -- 172.21.15.12:0/1351675800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1338103480 msgr2=0x7f1338075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:51.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.260+0000 7f133cf11700 1 --2- 172.21.15.12:0/1351675800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1338103480 0x7f1338075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:51.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.260+0000 7f133cf11700 1 -- 172.21.15.12:0/1351675800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1338102ac0 msgr2=0x7f1338074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:51.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.260+0000 7f133cf11700 1 --2- 172.21.15.12:0/1351675800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1338102ac0 0x7f1338074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:51.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.260+0000 7f133cf11700 1 -- 172.21.15.12:0/1351675800 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13381a7810 con 0x7f1338108ab0 2024-04-22T13:22:51.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.260+0000 7f13377fe700 1 --2- 172.21.15.12:0/1351675800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1338103480 0x7f1338075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:22:51.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.260+0000 7f133cf11700 1 --2- 172.21.15.12:0/1351675800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1338108ab0 0x7f13380757c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f133000b970 tx=0x7f133000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:51.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.260+0000 7f13357fa700 1 -- 172.21.15.12:0/1351675800 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1330019070 con 0x7f1338108ab0 2024-04-22T13:22:51.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.260+0000 7f13357fa700 1 -- 172.21.15.12:0/1351675800 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f133000cde0 con 0x7f1338108ab0 2024-04-22T13:22:51.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.260+0000 7f133e974700 1 -- 172.21.15.12:0/1351675800 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13381a7b00 con 0x7f1338108ab0 2024-04-22T13:22:51.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.260+0000 7f13357fa700 1 -- 172.21.15.12:0/1351675800 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1330007370 con 0x7f1338108ab0 2024-04-22T13:22:51.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.260+0000 7f133e974700 1 -- 172.21.15.12:0/1351675800 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13381a7ff0 con 0x7f1338108ab0 2024-04-22T13:22:51.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.261+0000 7f133e974700 1 -- 172.21.15.12:0/1351675800 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f13380630d0 con 0x7f1338108ab0 2024-04-22T13:22:51.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.262+0000 7f13357fa700 1 -- 172.21.15.12:0/1351675800 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f133001c070 con 0x7f1338108ab0 2024-04-22T13:22:51.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.263+0000 7f13357fa700 1 --2- 172.21.15.12:0/1351675800 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f13200a3cd0 0x7f13200a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:51.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.263+0000 7f1337fff700 1 --2- 172.21.15.12:0/1351675800 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f13200a3cd0 0x7f13200a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:51.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.263+0000 7f13357fa700 1 -- 172.21.15.12:0/1351675800 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f13300d0210 con 0x7f1338108ab0 2024-04-22T13:22:51.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.263+0000 7f13357fa700 1 --2- 172.21.15.12:0/1351675800 >> [v2:172.21.15.12:6810/3359066681,v1:172.21.15.12:6811/3359066681] conn(0x7f13200ad4b0 0x7f13200af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:51.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.263+0000 7f13357fa700 1 -- 172.21.15.12:0/1351675800 --> [v2:172.21.15.12:6810/3359066681,v1:172.21.15.12:6811/3359066681] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f13200aff80 con 0x7f13200ad4b0 2024-04-22T13:22:51.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.263+0000 7f13357fa700 1 -- 172.21.15.12:0/1351675800 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f13300d06f0 con 0x7f1338108ab0 2024-04-22T13:22:51.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.264+0000 7f13377fe700 1 --2- 172.21.15.12:0/1351675800 >> [v2:172.21.15.12:6810/3359066681,v1:172.21.15.12:6811/3359066681] conn(0x7f13200ad4b0 0x7f13200af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:51.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.264+0000 7f1337fff700 1 --2- 172.21.15.12:0/1351675800 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f13200a3cd0 0x7f13200a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1328009a10 tx=0x7f1328009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:51.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.264+0000 7f13377fe700 1 --2- 172.21.15.12:0/1351675800 >> [v2:172.21.15.12:6810/3359066681,v1:172.21.15.12:6811/3359066681] conn(0x7f13200ad4b0 0x7f13200af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:51.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.265+0000 7f13357fa700 1 -- 172.21.15.12:0/1351675800 <== osd.3 v2:172.21.15.12:6810/3359066681 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f13200aff80 con 0x7f13200ad4b0 2024-04-22T13:22:51.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.281+0000 7f133e974700 1 -- 172.21.15.12:0/1351675800 --> [v2:172.21.15.12:6810/3359066681,v1:172.21.15.12:6811/3359066681] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f13380634d0 con 0x7f13200ad4b0 2024-04-22T13:22:51.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.281+0000 7f13357fa700 1 -- 172.21.15.12:0/1351675800 <== osd.3 v2:172.21.15.12:6810/3359066681 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f13380634d0 con 0x7f13200ad4b0 2024-04-22T13:22:51.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.282+0000 7f133e974700 1 -- 172.21.15.12:0/1351675800 >> [v2:172.21.15.12:6810/3359066681,v1:172.21.15.12:6811/3359066681] conn(0x7f13200ad4b0 msgr2=0x7f13200af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:51.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.282+0000 7f133e974700 1 --2- 172.21.15.12:0/1351675800 >> [v2:172.21.15.12:6810/3359066681,v1:172.21.15.12:6811/3359066681] conn(0x7f13200ad4b0 0x7f13200af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:51.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.282+0000 7f133e974700 1 -- 172.21.15.12:0/1351675800 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f13200a3cd0 msgr2=0x7f13200a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:51.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.282+0000 7f133e974700 1 --2- 172.21.15.12:0/1351675800 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f13200a3cd0 0x7f13200a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1328009a10 tx=0x7f1328009190 comp rx=0 tx=0).stop 2024-04-22T13:22:51.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.282+0000 7f133e974700 1 -- 172.21.15.12:0/1351675800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1338108ab0 msgr2=0x7f13380757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:51.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.282+0000 7f133e974700 1 --2- 172.21.15.12:0/1351675800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1338108ab0 0x7f13380757c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f133000b970 tx=0x7f133000bc80 comp rx=0 tx=0).stop 2024-04-22T13:22:51.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.282+0000 7f133cf11700 1 -- 172.21.15.12:0/1351675800 reap_dead start 2024-04-22T13:22:51.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.282+0000 7f133e974700 1 -- 172.21.15.12:0/1351675800 shutdown_connections 2024-04-22T13:22:51.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.282+0000 7f133e974700 1 -- 172.21.15.12:0/1351675800 >> 172.21.15.12:0/1351675800 conn(0x7f1338079bf0 msgr2=0x7f1338100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:51.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.282+0000 7f133e974700 1 -- 172.21.15.12:0/1351675800 shutdown_connections 2024-04-22T13:22:51.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:46.282+0000 7f133e974700 1 -- 172.21.15.12:0/1351675800 wait complete. 2024-04-22T13:22:51.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:51 smithi111 ceph-mon[31361]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-22T13:22:51.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:51 smithi012 ceph-mon[28265]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-22T13:22:51.881 INFO:teuthology.orchestra.run.smithi012.stdout:115964117054 2024-04-22T13:22:51.881 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-22T13:22:51.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:51 smithi134 ceph-mon[31338]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-22T13:22:52.096 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:22:52.098 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:22:52.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.623+0000 7fcf62c22700 1 -- 172.21.15.12:0/459009956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf5c107980 msgr2=0x7fcf5c107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:52.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.623+0000 7fcf62c22700 1 --2- 172.21.15.12:0/459009956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf5c107980 0x7fcf5c107d60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009990 tx=0x7fcf50009ca0 comp rx=0 tx=0).stop 2024-04-22T13:22:52.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.623+0000 7fcf62c22700 1 -- 172.21.15.12:0/459009956 shutdown_connections 2024-04-22T13:22:52.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.623+0000 7fcf62c22700 1 --2- 172.21.15.12:0/459009956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf5c1022f0 0x7fcf5c10e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:52.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.623+0000 7fcf62c22700 1 --2- 172.21.15.12:0/459009956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf5c107980 0x7fcf5c107d60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:52.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.623+0000 7fcf62c22700 1 --2- 172.21.15.12:0/459009956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf5c101930 0x7fcf5c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:52.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.623+0000 7fcf62c22700 1 -- 172.21.15.12:0/459009956 >> 172.21.15.12:0/459009956 conn(0x7fcf5c0788b0 msgr2=0x7fcf5c078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:52.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.623+0000 7fcf62c22700 1 -- 172.21.15.12:0/459009956 shutdown_connections 2024-04-22T13:22:52.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.623+0000 7fcf62c22700 1 -- 172.21.15.12:0/459009956 wait complete. 2024-04-22T13:22:52.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.623+0000 7fcf62c22700 1 Processor -- start 2024-04-22T13:22:52.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.624+0000 7fcf62c22700 1 -- start start 2024-04-22T13:22:52.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.624+0000 7fcf62c22700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf5c101930 0x7fcf5c199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:52.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.624+0000 7fcf62c22700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf5c1022f0 0x7fcf5c199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:52.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.624+0000 7fcf62c22700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf5c107980 0x7fcf5c19d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:52.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.624+0000 7fcf62c22700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c111b50 con 0x7fcf5c107980 2024-04-22T13:22:52.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.624+0000 7fcf62c22700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c1119d0 con 0x7fcf5c101930 2024-04-22T13:22:52.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.624+0000 7fcf62c22700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5c111cd0 con 0x7fcf5c1022f0 2024-04-22T13:22:52.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.624+0000 7fcf611bf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf5c107980 0x7fcf5c19d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:52.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.624+0000 7fcf611bf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf5c107980 0x7fcf5c19d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:57976/0 (socket says 172.21.15.12:57976) 2024-04-22T13:22:52.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.624+0000 7fcf611bf700 1 -- 172.21.15.12:0/248321083 learned_addr learned my addr 172.21.15.12:0/248321083 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:22:52.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.624+0000 7fcf5bfff700 1 --2- 172.21.15.12:0/248321083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf5c1022f0 0x7fcf5c199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:52.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.624+0000 7fcf609be700 1 --2- 172.21.15.12:0/248321083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf5c101930 0x7fcf5c199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:52.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.624+0000 7fcf611bf700 1 -- 172.21.15.12:0/248321083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf5c1022f0 msgr2=0x7fcf5c199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:52.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.624+0000 7fcf611bf700 1 --2- 172.21.15.12:0/248321083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf5c1022f0 0x7fcf5c199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:52.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.624+0000 7fcf611bf700 1 -- 172.21.15.12:0/248321083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf5c101930 msgr2=0x7fcf5c199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:52.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.624+0000 7fcf611bf700 1 --2- 172.21.15.12:0/248321083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf5c101930 0x7fcf5c199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:52.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.624+0000 7fcf611bf700 1 -- 172.21.15.12:0/248321083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf5c19e120 con 0x7fcf5c107980 2024-04-22T13:22:52.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.624+0000 7fcf611bf700 1 --2- 172.21.15.12:0/248321083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf5c107980 0x7fcf5c19d9e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcf5400b970 tx=0x7fcf5400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:52.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.625+0000 7fcf59ffb700 1 -- 172.21.15.12:0/248321083 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5400d860 con 0x7fcf5c107980 2024-04-22T13:22:52.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.625+0000 7fcf62c22700 1 -- 172.21.15.12:0/248321083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf5c19e410 con 0x7fcf5c107980 2024-04-22T13:22:52.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.625+0000 7fcf59ffb700 1 -- 172.21.15.12:0/248321083 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf5400e430 con 0x7fcf5c107980 2024-04-22T13:22:52.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.625+0000 7fcf59ffb700 1 -- 172.21.15.12:0/248321083 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf54017660 con 0x7fcf5c107980 2024-04-22T13:22:52.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.625+0000 7fcf62c22700 1 -- 172.21.15.12:0/248321083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf5c1a5c90 con 0x7fcf5c107980 2024-04-22T13:22:52.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.626+0000 7fcf62c22700 1 -- 172.21.15.12:0/248321083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcf5c106980 con 0x7fcf5c107980 2024-04-22T13:22:52.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.627+0000 7fcf59ffb700 1 -- 172.21.15.12:0/248321083 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcf5400da00 con 0x7fcf5c107980 2024-04-22T13:22:52.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.628+0000 7fcf59ffb700 1 --2- 172.21.15.12:0/248321083 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fcf4c0a3cd0 0x7fcf4c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:52.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.628+0000 7fcf609be700 1 --2- 172.21.15.12:0/248321083 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fcf4c0a3cd0 0x7fcf4c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:52.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.628+0000 7fcf59ffb700 1 -- 172.21.15.12:0/248321083 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7fcf540d04d0 con 0x7fcf5c107980 2024-04-22T13:22:52.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.628+0000 7fcf59ffb700 1 --2- 172.21.15.12:0/248321083 >> [v2:172.21.15.111:6816/3113856282,v1:172.21.15.111:6817/3113856282] conn(0x7fcf4c0ad4b0 0x7fcf4c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:22:52.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.628+0000 7fcf59ffb700 1 -- 172.21.15.12:0/248321083 --> [v2:172.21.15.111:6816/3113856282,v1:172.21.15.111:6817/3113856282] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcf54011ec0 con 0x7fcf4c0ad4b0 2024-04-22T13:22:52.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.628+0000 7fcf59ffb700 1 -- 172.21.15.12:0/248321083 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fcf540d0880 con 0x7fcf5c107980 2024-04-22T13:22:52.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.628+0000 7fcf609be700 1 --2- 172.21.15.12:0/248321083 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fcf4c0a3cd0 0x7fcf4c0a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcf500098e0 tx=0x7fcf50005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:52.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.628+0000 7fcf5bfff700 1 --2- 172.21.15.12:0/248321083 >> [v2:172.21.15.111:6816/3113856282,v1:172.21.15.111:6817/3113856282] conn(0x7fcf4c0ad4b0 0x7fcf4c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:22:52.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.629+0000 7fcf5bfff700 1 --2- 172.21.15.12:0/248321083 >> [v2:172.21.15.111:6816/3113856282,v1:172.21.15.111:6817/3113856282] conn(0x7fcf4c0ad4b0 0x7fcf4c0af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:22:52.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.630+0000 7fcf59ffb700 1 -- 172.21.15.12:0/248321083 <== osd.7 v2:172.21.15.111:6816/3113856282 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcf54011ec0 con 0x7fcf4c0ad4b0 2024-04-22T13:22:52.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.647+0000 7fcf62c22700 1 -- 172.21.15.12:0/248321083 --> [v2:172.21.15.111:6816/3113856282,v1:172.21.15.111:6817/3113856282] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcf5c103050 con 0x7fcf4c0ad4b0 2024-04-22T13:22:52.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.648+0000 7fcf59ffb700 1 -- 172.21.15.12:0/248321083 <== osd.7 v2:172.21.15.111:6816/3113856282 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcf5c103050 con 0x7fcf4c0ad4b0 2024-04-22T13:22:52.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.648+0000 7fcf62c22700 1 -- 172.21.15.12:0/248321083 >> [v2:172.21.15.111:6816/3113856282,v1:172.21.15.111:6817/3113856282] conn(0x7fcf4c0ad4b0 msgr2=0x7fcf4c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:52.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.648+0000 7fcf62c22700 1 --2- 172.21.15.12:0/248321083 >> [v2:172.21.15.111:6816/3113856282,v1:172.21.15.111:6817/3113856282] conn(0x7fcf4c0ad4b0 0x7fcf4c0af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:22:52.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.648+0000 7fcf62c22700 1 -- 172.21.15.12:0/248321083 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fcf4c0a3cd0 msgr2=0x7fcf4c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:52.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.648+0000 7fcf62c22700 1 --2- 172.21.15.12:0/248321083 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fcf4c0a3cd0 0x7fcf4c0a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcf500098e0 tx=0x7fcf50005960 comp rx=0 tx=0).stop 2024-04-22T13:22:52.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.648+0000 7fcf62c22700 1 -- 172.21.15.12:0/248321083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf5c107980 msgr2=0x7fcf5c19d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:22:52.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.648+0000 7fcf62c22700 1 --2- 172.21.15.12:0/248321083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf5c107980 0x7fcf5c19d9e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcf5400b970 tx=0x7fcf5400bc80 comp rx=0 tx=0).stop 2024-04-22T13:22:52.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.648+0000 7fcf611bf700 1 -- 172.21.15.12:0/248321083 reap_dead start 2024-04-22T13:22:52.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.649+0000 7fcf62c22700 1 -- 172.21.15.12:0/248321083 shutdown_connections 2024-04-22T13:22:52.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.649+0000 7fcf62c22700 1 -- 172.21.15.12:0/248321083 >> 172.21.15.12:0/248321083 conn(0x7fcf5c0788b0 msgr2=0x7fcf5c06b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:22:52.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.649+0000 7fcf62c22700 1 -- 172.21.15.12:0/248321083 shutdown_connections 2024-04-22T13:22:52.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:22:52.649+0000 7fcf62c22700 1 -- 172.21.15.12:0/248321083 wait complete. 2024-04-22T13:22:53.078 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:22:53.079 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:22:53.080 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:22:53.099 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:22:53.570 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:53 smithi012 ceph-mon[28265]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-22T13:22:53.596 INFO:teuthology.orchestra.run.smithi012.stdout:206158430256 2024-04-22T13:22:53.597 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-22T13:22:53.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:53 smithi111 ceph-mon[31361]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-22T13:22:53.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:53 smithi134 ceph-mon[31338]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-22T13:22:54.412 INFO:teuthology.orchestra.run.smithi012.stdout:231928234027 2024-04-22T13:22:54.413 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-22T13:22:55.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:55 smithi012 ceph-mon[28265]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-04-22T13:22:55.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:55 smithi111 ceph-mon[31361]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-04-22T13:22:55.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:55 smithi134 ceph-mon[31338]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-04-22T13:22:57.208 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:22:57.208 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:22:57.209 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:22:57.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:57 smithi012 ceph-mon[28265]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-22T13:22:57.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:57 smithi134 ceph-mon[31338]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-22T13:22:57.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:57 smithi111 ceph-mon[31361]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-22T13:22:59.588 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:22:59 smithi012 ceph-mon[28265]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-22T13:22:59.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:22:59 smithi134 ceph-mon[31338]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-22T13:22:59.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:22:59 smithi111 ceph-mon[31361]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-22T13:23:00.629 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:23:00.630 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:23:01.664 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:01 smithi012 ceph-mon[28265]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-22T13:23:01.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:01 smithi134 ceph-mon[31338]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-22T13:23:01.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:01 smithi111 ceph-mon[31361]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-22T13:23:02.697 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:02 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/14068992' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-22T13:23:02.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:02 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/14068992' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-22T13:23:02.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:02 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/14068992' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-22T13:23:03.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.239+0000 7f3807a61700 1 -- 172.21.15.12:0/1186444452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3800108a70 msgr2=0x7f3800108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:03.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.239+0000 7f3807a61700 1 --2- 172.21.15.12:0/1186444452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3800108a70 0x7f3800108e50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f37f000b0a0 tx=0x7f37f000b3b0 comp rx=0 tx=0).stop 2024-04-22T13:23:03.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.239+0000 7f3807a61700 1 -- 172.21.15.12:0/1186444452 shutdown_connections 2024-04-22T13:23:03.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.239+0000 7f3807a61700 1 --2- 172.21.15.12:0/1186444452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3800103460 0x7f380010f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:03.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.239+0000 7f3807a61700 1 --2- 172.21.15.12:0/1186444452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3800108a70 0x7f3800108e50 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:03.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.239+0000 7f3807a61700 1 --2- 172.21.15.12:0/1186444452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3800102ac0 0x7f3800102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:03.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.239+0000 7f3807a61700 1 -- 172.21.15.12:0/1186444452 >> 172.21.15.12:0/1186444452 conn(0x7f3800079b10 msgr2=0x7f3800079f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:03.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.240+0000 7f3807a61700 1 -- 172.21.15.12:0/1186444452 shutdown_connections 2024-04-22T13:23:03.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.240+0000 7f3807a61700 1 -- 172.21.15.12:0/1186444452 wait complete. 2024-04-22T13:23:03.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.240+0000 7f3807a61700 1 Processor -- start 2024-04-22T13:23:03.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.240+0000 7f3807a61700 1 -- start start 2024-04-22T13:23:03.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.240+0000 7f3807a61700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3800102ac0 0x7f3800074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:03.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.240+0000 7f3807a61700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3800103460 0x7f3800075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:03.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.240+0000 7f3807a61700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3800108a70 0x7f38000757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:03.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.240+0000 7f3807a61700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3800112c10 con 0x7f3800102ac0 2024-04-22T13:23:03.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.240+0000 7f3807a61700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3800112a90 con 0x7f3800103460 2024-04-22T13:23:03.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.240+0000 7f3807a61700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3800112d90 con 0x7f3800108a70 2024-04-22T13:23:03.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.241+0000 7f38057fd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3800102ac0 0x7f3800074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:03.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.241+0000 7f38057fd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3800102ac0 0x7f3800074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59192/0 (socket says 172.21.15.12:59192) 2024-04-22T13:23:03.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.241+0000 7f38057fd700 1 -- 172.21.15.12:0/3385064927 learned_addr learned my addr 172.21.15.12:0/3385064927 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:03.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.241+0000 7f3805ffe700 1 --2- 172.21.15.12:0/3385064927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3800108a70 0x7f38000757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:03.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.241+0000 7f3804ffc700 1 --2- 172.21.15.12:0/3385064927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3800103460 0x7f3800075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:03.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.241+0000 7f38057fd700 1 -- 172.21.15.12:0/3385064927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3800108a70 msgr2=0x7f38000757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:03.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.241+0000 7f38057fd700 1 --2- 172.21.15.12:0/3385064927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3800108a70 0x7f38000757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:03.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.241+0000 7f38057fd700 1 -- 172.21.15.12:0/3385064927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3800103460 msgr2=0x7f3800075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:03.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.241+0000 7f38057fd700 1 --2- 172.21.15.12:0/3385064927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3800103460 0x7f3800075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:03.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.241+0000 7f38057fd700 1 -- 172.21.15.12:0/3385064927 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38001a7650 con 0x7f3800102ac0 2024-04-22T13:23:03.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.241+0000 7f38057fd700 1 --2- 172.21.15.12:0/3385064927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3800102ac0 0x7f3800074d40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f37f000b070 tx=0x7f37f0004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:03.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.241+0000 7f37fe7fc700 1 -- 172.21.15.12:0/3385064927 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f0021070 con 0x7f3800102ac0 2024-04-22T13:23:03.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.241+0000 7f37fe7fc700 1 -- 172.21.15.12:0/3385064927 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37f0007c80 con 0x7f3800102ac0 2024-04-22T13:23:03.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.241+0000 7f3807a61700 1 -- 172.21.15.12:0/3385064927 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38001a78e0 con 0x7f3800102ac0 2024-04-22T13:23:03.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.241+0000 7f37fe7fc700 1 -- 172.21.15.12:0/3385064927 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f001d450 con 0x7f3800102ac0 2024-04-22T13:23:03.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.241+0000 7f3807a61700 1 -- 172.21.15.12:0/3385064927 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38001a7de0 con 0x7f3800102ac0 2024-04-22T13:23:03.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.242+0000 7f3807a61700 1 -- 172.21.15.12:0/3385064927 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3800069730 con 0x7f3800102ac0 2024-04-22T13:23:03.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.244+0000 7f37fe7fc700 1 -- 172.21.15.12:0/3385064927 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f37f0004000 con 0x7f3800102ac0 2024-04-22T13:23:03.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.244+0000 7f37fe7fc700 1 --2- 172.21.15.12:0/3385064927 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f37e80a3d50 0x7f37e80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:03.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.244+0000 7f3804ffc700 1 --2- 172.21.15.12:0/3385064927 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f37e80a3d50 0x7f37e80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:03.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.244+0000 7f37fe7fc700 1 -- 172.21.15.12:0/3385064927 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f37f000f020 con 0x7f3800102ac0 2024-04-22T13:23:03.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.245+0000 7f3804ffc700 1 --2- 172.21.15.12:0/3385064927 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f37e80a3d50 0x7f37e80a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f37f4005fd0 tx=0x7f37f400a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:03.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.248+0000 7f37fe7fc700 1 -- 172.21.15.12:0/3385064927 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f37f002a020 con 0x7f3800102ac0 2024-04-22T13:23:03.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.464+0000 7f3807a61700 1 -- 172.21.15.12:0/3385064927 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f380002df80 con 0x7f3800102ac0 2024-04-22T13:23:03.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.465+0000 7f37fe7fc700 1 -- 172.21.15.12:0/3385064927 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f37f009e810 con 0x7f3800102ac0 2024-04-22T13:23:03.461 INFO:teuthology.orchestra.run.smithi012.stdout:150323855421 2024-04-22T13:23:03.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.467+0000 7f3807a61700 1 -- 172.21.15.12:0/3385064927 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f37e80a3d50 msgr2=0x7f37e80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:03.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.467+0000 7f3807a61700 1 --2- 172.21.15.12:0/3385064927 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f37e80a3d50 0x7f37e80a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f37f4005fd0 tx=0x7f37f400a560 comp rx=0 tx=0).stop 2024-04-22T13:23:03.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.467+0000 7f3807a61700 1 -- 172.21.15.12:0/3385064927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3800102ac0 msgr2=0x7f3800074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:03.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.467+0000 7f3807a61700 1 --2- 172.21.15.12:0/3385064927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3800102ac0 0x7f3800074d40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f37f000b070 tx=0x7f37f0004db0 comp rx=0 tx=0).stop 2024-04-22T13:23:03.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.467+0000 7f3807a61700 1 -- 172.21.15.12:0/3385064927 shutdown_connections 2024-04-22T13:23:03.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.467+0000 7f3807a61700 1 --2- 172.21.15.12:0/3385064927 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f37e80a3d50 0x7f37e80a6210 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:03.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.467+0000 7f3807a61700 1 --2- 172.21.15.12:0/3385064927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3800108a70 0x7f38000757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:03.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.467+0000 7f3807a61700 1 --2- 172.21.15.12:0/3385064927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3800102ac0 0x7f3800074d40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:03.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.467+0000 7f3807a61700 1 --2- 172.21.15.12:0/3385064927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3800103460 0x7f3800075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:03.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.467+0000 7f3807a61700 1 -- 172.21.15.12:0/3385064927 >> 172.21.15.12:0/3385064927 conn(0x7f3800079b10 msgr2=0x7f3800100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:03.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.467+0000 7f3807a61700 1 -- 172.21.15.12:0/3385064927 shutdown_connections 2024-04-22T13:23:03.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:03.467+0000 7f3807a61700 1 -- 172.21.15.12:0/3385064927 wait complete. 2024-04-22T13:23:03.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.158+0000 7f64e5e0a700 1 -- 172.21.15.12:0/4217553433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64e006bb40 msgr2=0x7f64e010f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:03.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.158+0000 7f64e5e0a700 1 --2- 172.21.15.12:0/4217553433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64e006bb40 0x7f64e010f920 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f64d800b0d0 tx=0x7f64d800b3e0 comp rx=0 tx=0).stop 2024-04-22T13:23:03.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.158+0000 7f64e5e0a700 1 -- 172.21.15.12:0/4217553433 shutdown_connections 2024-04-22T13:23:03.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.158+0000 7f64e5e0a700 1 --2- 172.21.15.12:0/4217553433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64e006bb40 0x7f64e010f920 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:03.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.158+0000 7f64e5e0a700 1 --2- 172.21.15.12:0/4217553433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e0108b50 0x7f64e0108f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:03.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.158+0000 7f64e5e0a700 1 --2- 172.21.15.12:0/4217553433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64e006b180 0x7f64e006b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:03.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.158+0000 7f64e5e0a700 1 -- 172.21.15.12:0/4217553433 >> 172.21.15.12:0/4217553433 conn(0x7f64e0078740 msgr2=0x7f64e0078b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:03.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.158+0000 7f64e5e0a700 1 -- 172.21.15.12:0/4217553433 shutdown_connections 2024-04-22T13:23:03.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.158+0000 7f64e5e0a700 1 -- 172.21.15.12:0/4217553433 wait complete. 2024-04-22T13:23:03.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.159+0000 7f64e5e0a700 1 Processor -- start 2024-04-22T13:23:03.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.159+0000 7f64e5e0a700 1 -- start start 2024-04-22T13:23:03.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.159+0000 7f64e5e0a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64e006b180 0x7f64e0199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:03.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.159+0000 7f64e5e0a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64e006bb40 0x7f64e01996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:03.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.159+0000 7f64e5e0a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e0108b50 0x7f64e019d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:03.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.159+0000 7f64e5e0a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f64e0112c70 con 0x7f64e006b180 2024-04-22T13:23:03.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.159+0000 7f64e5e0a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f64e0112af0 con 0x7f64e006bb40 2024-04-22T13:23:03.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.159+0000 7f64e5e0a700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f64e0112df0 con 0x7f64e0108b50 2024-04-22T13:23:03.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.159+0000 7f64df7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64e006b180 0x7f64e0199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:03.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.159+0000 7f64df7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64e006b180 0x7f64e0199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59166/0 (socket says 172.21.15.12:59166) 2024-04-22T13:23:03.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.159+0000 7f64df7fe700 1 -- 172.21.15.12:0/14068992 learned_addr learned my addr 172.21.15.12:0/14068992 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:03.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.159+0000 7f64deffd700 1 --2- 172.21.15.12:0/14068992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64e006bb40 0x7f64e01996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:03.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.160+0000 7f64dffff700 1 --2- 172.21.15.12:0/14068992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e0108b50 0x7f64e019d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:03.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.160+0000 7f64df7fe700 1 -- 172.21.15.12:0/14068992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e0108b50 msgr2=0x7f64e019d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:03.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.160+0000 7f64df7fe700 1 --2- 172.21.15.12:0/14068992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e0108b50 0x7f64e019d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:03.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.160+0000 7f64df7fe700 1 -- 172.21.15.12:0/14068992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64e006bb40 msgr2=0x7f64e01996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:03.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.160+0000 7f64df7fe700 1 --2- 172.21.15.12:0/14068992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64e006bb40 0x7f64e01996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:03.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.160+0000 7f64df7fe700 1 -- 172.21.15.12:0/14068992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e019e120 con 0x7f64e006b180 2024-04-22T13:23:03.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.160+0000 7f64dffff700 1 --2- 172.21.15.12:0/14068992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e0108b50 0x7f64e019d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:23:03.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.160+0000 7f64df7fe700 1 --2- 172.21.15.12:0/14068992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64e006b180 0x7f64e0199160 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f64d000d980 tx=0x7f64d000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:03.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.160+0000 7f64dcff9700 1 -- 172.21.15.12:0/14068992 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d0016070 con 0x7f64e006b180 2024-04-22T13:23:03.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.160+0000 7f64e5e0a700 1 -- 172.21.15.12:0/14068992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64e019e410 con 0x7f64e006b180 2024-04-22T13:23:03.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.160+0000 7f64dcff9700 1 -- 172.21.15.12:0/14068992 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64d0004830 con 0x7f64e006b180 2024-04-22T13:23:03.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.160+0000 7f64dcff9700 1 -- 172.21.15.12:0/14068992 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d00054c0 con 0x7f64e006b180 2024-04-22T13:23:03.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.160+0000 7f64e5e0a700 1 -- 172.21.15.12:0/14068992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64e01a5c90 con 0x7f64e006b180 2024-04-22T13:23:03.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.161+0000 7f64e5e0a700 1 -- 172.21.15.12:0/14068992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64e0069730 con 0x7f64e006b180 2024-04-22T13:23:03.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.162+0000 7f64dcff9700 1 -- 172.21.15.12:0/14068992 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f64d0004220 con 0x7f64e006b180 2024-04-22T13:23:03.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.163+0000 7f64dcff9700 1 --2- 172.21.15.12:0/14068992 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f64c80a3cd0 0x7f64c80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:03.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.163+0000 7f64deffd700 1 --2- 172.21.15.12:0/14068992 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f64c80a3cd0 0x7f64c80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:03.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.163+0000 7f64dcff9700 1 -- 172.21.15.12:0/14068992 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7979+0+0 (secure 0 0 0) 0x7f64d00d0250 con 0x7f64e006b180 2024-04-22T13:23:03.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.163+0000 7f64deffd700 1 --2- 172.21.15.12:0/14068992 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f64c80a3cd0 0x7f64c80a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f64e0078d60 tx=0x7f64d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:03.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.168+0000 7f64dcff9700 1 -- 172.21.15.12:0/14068992 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f64d001f020 con 0x7f64e006b180 2024-04-22T13:23:03.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.379+0000 7f64e5e0a700 1 -- 172.21.15.12:0/14068992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f64e019a470 con 0x7f64e006b180 2024-04-22T13:23:03.571 INFO:teuthology.orchestra.run.smithi012.stdout:296352743459 2024-04-22T13:23:03.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.379+0000 7f64dcff9700 1 -- 172.21.15.12:0/14068992 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f64d009cb80 con 0x7f64e006b180 2024-04-22T13:23:03.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.381+0000 7f64e5e0a700 1 -- 172.21.15.12:0/14068992 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f64c80a3cd0 msgr2=0x7f64c80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:03.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.381+0000 7f64e5e0a700 1 --2- 172.21.15.12:0/14068992 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f64c80a3cd0 0x7f64c80a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f64e0078d60 tx=0x7f64d4009260 comp rx=0 tx=0).stop 2024-04-22T13:23:03.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.381+0000 7f64e5e0a700 1 -- 172.21.15.12:0/14068992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64e006b180 msgr2=0x7f64e0199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:03.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.381+0000 7f64e5e0a700 1 --2- 172.21.15.12:0/14068992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64e006b180 0x7f64e0199160 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f64d000d980 tx=0x7f64d000dc90 comp rx=0 tx=0).stop 2024-04-22T13:23:03.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.382+0000 7f64e5e0a700 1 -- 172.21.15.12:0/14068992 shutdown_connections 2024-04-22T13:23:03.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.382+0000 7f64e5e0a700 1 --2- 172.21.15.12:0/14068992 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f64c80a3cd0 0x7f64c80a6190 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:03.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.382+0000 7f64e5e0a700 1 --2- 172.21.15.12:0/14068992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e0108b50 0x7f64e019d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:03.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.382+0000 7f64e5e0a700 1 --2- 172.21.15.12:0/14068992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64e006b180 0x7f64e0199160 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:03.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.382+0000 7f64e5e0a700 1 --2- 172.21.15.12:0/14068992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64e006bb40 0x7f64e01996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:03.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.382+0000 7f64e5e0a700 1 -- 172.21.15.12:0/14068992 >> 172.21.15.12:0/14068992 conn(0x7f64e0078740 msgr2=0x7f64e0100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:03.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.382+0000 7f64e5e0a700 1 -- 172.21.15.12:0/14068992 shutdown_connections 2024-04-22T13:23:03.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:02.382+0000 7f64e5e0a700 1 -- 172.21.15.12:0/14068992 wait complete. 2024-04-22T13:23:03.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:03 smithi012 ceph-mon[28265]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-22T13:23:03.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:03 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3385064927' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-22T13:23:03.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:03 smithi134 ceph-mon[31338]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-22T13:23:03.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:03 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/3385064927' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-22T13:23:03.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:03 smithi111 ceph-mon[31361]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-22T13:23:03.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:03 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/3385064927' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-22T13:23:04.194 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743453 got 296352743459 for osd.9 2024-04-22T13:23:04.194 DEBUG:teuthology.parallel:result is None 2024-04-22T13:23:04.808 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:23:04.808 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-22T13:23:04.808 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-22T13:23:04.808 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-04-22T13:23:04.808 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-04-22T13:23:04.809 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-04-22T13:23:04.809 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-22T13:23:04.809 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:04 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-04-22T13:23:04.824 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:04 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:23:04.824 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:04 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-22T13:23:04.824 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:04 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-22T13:23:04.824 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:04 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-04-22T13:23:04.824 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:04 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-04-22T13:23:04.824 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:04 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-04-22T13:23:04.825 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:04 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-22T13:23:04.825 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:04 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-04-22T13:23:04.840 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:04 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:23:04.840 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:04 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-22T13:23:04.840 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:04 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-22T13:23:04.840 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:04 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-04-22T13:23:04.840 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:04 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-04-22T13:23:04.840 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:04 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-04-22T13:23:04.840 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:04 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-22T13:23:04.840 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:04 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-04-22T13:23:05.809 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:05 smithi012 ceph-mon[28265]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-22T13:23:05.809 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:05 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-22T13:23:05.809 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:05 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-22T13:23:05.809 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:05 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-04-22T13:23:05.810 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:05 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-04-22T13:23:05.810 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:05 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-04-22T13:23:05.810 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:05 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-22T13:23:05.810 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:05 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-04-22T13:23:05.810 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:05 smithi012 ceph-mon[28265]: osdmap e88: 12 total, 12 up, 12 in 2024-04-22T13:23:05.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:05 smithi134 ceph-mon[31338]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-22T13:23:05.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:05 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-22T13:23:05.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:05 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-22T13:23:05.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:05 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-04-22T13:23:05.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:05 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-04-22T13:23:05.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:05 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-04-22T13:23:05.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:05 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-22T13:23:05.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:05 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-04-22T13:23:05.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:05 smithi134 ceph-mon[31338]: osdmap e88: 12 total, 12 up, 12 in 2024-04-22T13:23:05.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:05 smithi111 ceph-mon[31361]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-22T13:23:05.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:05 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-22T13:23:05.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:05 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-22T13:23:05.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:05 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-04-22T13:23:05.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:05 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-04-22T13:23:05.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:05 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-04-22T13:23:05.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:05 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-22T13:23:05.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:05 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-04-22T13:23:05.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:05 smithi111 ceph-mon[31361]: osdmap e88: 12 total, 12 up, 12 in 2024-04-22T13:23:06.834 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:06 smithi012 ceph-mon[28265]: osdmap e89: 12 total, 12 up, 12 in 2024-04-22T13:23:06.834 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:06 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:23:06.834 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:06 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:23:06.834 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:06 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:23:06.834 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:06 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:23:06.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:06 smithi134 ceph-mon[31338]: osdmap e89: 12 total, 12 up, 12 in 2024-04-22T13:23:06.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:06 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:23:06.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:06 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:23:06.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:06 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:23:06.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:06 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:23:06.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:06 smithi111 ceph-mon[31361]: osdmap e89: 12 total, 12 up, 12 in 2024-04-22T13:23:06.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:06 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:23:06.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:06 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:23:06.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:06 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:23:06.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:06 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:23:07.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.621+0000 7efc741d7700 1 -- 172.21.15.12:0/3055314122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc6c107920 msgr2=0x7efc6c107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:07.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.621+0000 7efc741d7700 1 --2- 172.21.15.12:0/3055314122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc6c107920 0x7efc6c107d00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7efc60009990 tx=0x7efc60009ca0 comp rx=0 tx=0).stop 2024-04-22T13:23:07.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.621+0000 7efc741d7700 1 -- 172.21.15.12:0/3055314122 shutdown_connections 2024-04-22T13:23:07.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.621+0000 7efc741d7700 1 --2- 172.21.15.12:0/3055314122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc6c102330 0x7efc6c10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:07.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.621+0000 7efc741d7700 1 --2- 172.21.15.12:0/3055314122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc6c107920 0x7efc6c107d00 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:07.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.621+0000 7efc741d7700 1 --2- 172.21.15.12:0/3055314122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc6c101970 0x7efc6c101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:07.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.621+0000 7efc741d7700 1 -- 172.21.15.12:0/3055314122 >> 172.21.15.12:0/3055314122 conn(0x7efc6c0779b0 msgr2=0x7efc6c077dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:07.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.621+0000 7efc741d7700 1 -- 172.21.15.12:0/3055314122 shutdown_connections 2024-04-22T13:23:07.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.621+0000 7efc741d7700 1 -- 172.21.15.12:0/3055314122 wait complete. 2024-04-22T13:23:07.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.622+0000 7efc741d7700 1 Processor -- start 2024-04-22T13:23:07.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.622+0000 7efc741d7700 1 -- start start 2024-04-22T13:23:07.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.622+0000 7efc741d7700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc6c101970 0x7efc6c1034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:07.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.622+0000 7efc741d7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc6c102330 0x7efc6c103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:07.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.622+0000 7efc741d7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc6c107920 0x7efc6c103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:07.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.622+0000 7efc741d7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c111b00 con 0x7efc6c102330 2024-04-22T13:23:07.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.622+0000 7efc741d7700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c111980 con 0x7efc6c107920 2024-04-22T13:23:07.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.622+0000 7efc741d7700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c111c80 con 0x7efc6c101970 2024-04-22T13:23:07.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.622+0000 7efc71772700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc6c102330 0x7efc6c103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:07.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.622+0000 7efc71772700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc6c102330 0x7efc6c103a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59202/0 (socket says 172.21.15.12:59202) 2024-04-22T13:23:07.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.622+0000 7efc71772700 1 -- 172.21.15.12:0/2205923393 learned_addr learned my addr 172.21.15.12:0/2205923393 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:07.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.623+0000 7efc72774700 1 --2- 172.21.15.12:0/2205923393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc6c107920 0x7efc6c103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:07.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.623+0000 7efc71f73700 1 --2- 172.21.15.12:0/2205923393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc6c101970 0x7efc6c1034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:07.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.623+0000 7efc71772700 1 -- 172.21.15.12:0/2205923393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc6c101970 msgr2=0x7efc6c1034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:07.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.623+0000 7efc71772700 1 --2- 172.21.15.12:0/2205923393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc6c101970 0x7efc6c1034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:07.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.623+0000 7efc71772700 1 -- 172.21.15.12:0/2205923393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc6c107920 msgr2=0x7efc6c103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:07.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.623+0000 7efc71772700 1 --2- 172.21.15.12:0/2205923393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc6c107920 0x7efc6c103f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:07.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.623+0000 7efc71772700 1 -- 172.21.15.12:0/2205923393 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6c1a7660 con 0x7efc6c102330 2024-04-22T13:23:07.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.623+0000 7efc71f73700 1 --2- 172.21.15.12:0/2205923393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc6c101970 0x7efc6c1034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:23:07.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.623+0000 7efc71772700 1 --2- 172.21.15.12:0/2205923393 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc6c102330 0x7efc6c103a30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efc6800d950 tx=0x7efc6800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:07.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.623+0000 7efc5effd700 1 -- 172.21.15.12:0/2205923393 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc68016070 con 0x7efc6c102330 2024-04-22T13:23:07.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.623+0000 7efc5effd700 1 -- 172.21.15.12:0/2205923393 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc68010460 con 0x7efc6c102330 2024-04-22T13:23:07.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.623+0000 7efc741d7700 1 -- 172.21.15.12:0/2205923393 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc6c1a7950 con 0x7efc6c102330 2024-04-22T13:23:07.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.623+0000 7efc5effd700 1 -- 172.21.15.12:0/2205923393 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6800f620 con 0x7efc6c102330 2024-04-22T13:23:07.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.623+0000 7efc741d7700 1 -- 172.21.15.12:0/2205923393 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc6c1a7e90 con 0x7efc6c102330 2024-04-22T13:23:07.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.624+0000 7efc741d7700 1 -- 172.21.15.12:0/2205923393 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc6c103060 con 0x7efc6c102330 2024-04-22T13:23:07.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.625+0000 7efc5effd700 1 -- 172.21.15.12:0/2205923393 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7efc68019070 con 0x7efc6c102330 2024-04-22T13:23:07.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.626+0000 7efc5effd700 1 --2- 172.21.15.12:0/2205923393 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efc580a3cd0 0x7efc580a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:07.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.626+0000 7efc71f73700 1 --2- 172.21.15.12:0/2205923393 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efc580a3cd0 0x7efc580a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:07.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.626+0000 7efc5effd700 1 -- 172.21.15.12:0/2205923393 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7efc680d0210 con 0x7efc6c102330 2024-04-22T13:23:07.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.626+0000 7efc71f73700 1 --2- 172.21.15.12:0/2205923393 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efc580a3cd0 0x7efc580a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efc6000b5c0 tx=0x7efc60005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:07.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.631+0000 7efc5effd700 1 -- 172.21.15.12:0/2205923393 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efc6809ca60 con 0x7efc6c102330 2024-04-22T13:23:07.729 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:07 smithi012 ceph-mon[28265]: pgmap v320: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2024-04-22T13:23:07.729 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:07 smithi012 ceph-mon[28265]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-04-22T13:23:07.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.845+0000 7efc741d7700 1 -- 172.21.15.12:0/2205923393 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7efc6c02df80 con 0x7efc6c102330 2024-04-22T13:23:07.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.846+0000 7efc5effd700 1 -- 172.21.15.12:0/2205923393 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efc6809c360 con 0x7efc6c102330 2024-04-22T13:23:07.842 INFO:teuthology.orchestra.run.smithi012.stdout:55834574923 2024-04-22T13:23:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.848+0000 7efc741d7700 1 -- 172.21.15.12:0/2205923393 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efc580a3cd0 msgr2=0x7efc580a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.848+0000 7efc741d7700 1 --2- 172.21.15.12:0/2205923393 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efc580a3cd0 0x7efc580a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efc6000b5c0 tx=0x7efc60005fb0 comp rx=0 tx=0).stop 2024-04-22T13:23:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.848+0000 7efc741d7700 1 -- 172.21.15.12:0/2205923393 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc6c102330 msgr2=0x7efc6c103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.848+0000 7efc741d7700 1 --2- 172.21.15.12:0/2205923393 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc6c102330 0x7efc6c103a30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efc6800d950 tx=0x7efc6800dd10 comp rx=0 tx=0).stop 2024-04-22T13:23:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.848+0000 7efc741d7700 1 -- 172.21.15.12:0/2205923393 shutdown_connections 2024-04-22T13:23:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.848+0000 7efc741d7700 1 --2- 172.21.15.12:0/2205923393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc6c107920 0x7efc6c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.848+0000 7efc741d7700 1 --2- 172.21.15.12:0/2205923393 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efc580a3cd0 0x7efc580a6190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.848+0000 7efc741d7700 1 --2- 172.21.15.12:0/2205923393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc6c101970 0x7efc6c1034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:07.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.848+0000 7efc741d7700 1 --2- 172.21.15.12:0/2205923393 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc6c102330 0x7efc6c103a30 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:07.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.848+0000 7efc741d7700 1 -- 172.21.15.12:0/2205923393 >> 172.21.15.12:0/2205923393 conn(0x7efc6c0779b0 msgr2=0x7efc6c06b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:07.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.848+0000 7efc741d7700 1 -- 172.21.15.12:0/2205923393 shutdown_connections 2024-04-22T13:23:07.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:07.849+0000 7efc741d7700 1 -- 172.21.15.12:0/2205923393 wait complete. 2024-04-22T13:23:07.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:07 smithi134 ceph-mon[31338]: pgmap v320: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2024-04-22T13:23:07.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:07 smithi134 ceph-mon[31338]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-04-22T13:23:07.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:07 smithi111 ceph-mon[31361]: pgmap v320: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2024-04-22T13:23:07.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:07 smithi111 ceph-mon[31361]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-04-22T13:23:08.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.107+0000 7f91576c0700 1 -- 172.21.15.12:0/3527175006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f915006a750 msgr2=0x7f915006ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:08.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.107+0000 7f91576c0700 1 --2- 172.21.15.12:0/3527175006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f915006a750 0x7f915006ab30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f914800b0a0 tx=0x7f914800b3b0 comp rx=0 tx=0).stop 2024-04-22T13:23:08.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.107+0000 7f91576c0700 1 -- 172.21.15.12:0/3527175006 shutdown_connections 2024-04-22T13:23:08.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.107+0000 7f91576c0700 1 --2- 172.21.15.12:0/3527175006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f915010f4f0 0x7f91501118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:08.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.107+0000 7f91576c0700 1 --2- 172.21.15.12:0/3527175006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f915006a750 0x7f915006ab30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:08.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.107+0000 7f91576c0700 1 --2- 172.21.15.12:0/3527175006 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f915006b070 0x7f915010efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:08.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.107+0000 7f91576c0700 1 -- 172.21.15.12:0/3527175006 >> 172.21.15.12:0/3527175006 conn(0x7f91500788f0 msgr2=0x7f9150078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:08.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.107+0000 7f91576c0700 1 -- 172.21.15.12:0/3527175006 shutdown_connections 2024-04-22T13:23:08.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.107+0000 7f91576c0700 1 -- 172.21.15.12:0/3527175006 wait complete. 2024-04-22T13:23:08.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.108+0000 7f91576c0700 1 Processor -- start 2024-04-22T13:23:08.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.108+0000 7f91576c0700 1 -- start start 2024-04-22T13:23:08.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.108+0000 7f91576c0700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f915006a750 0x7f915019b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:08.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.108+0000 7f91576c0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f915006b070 0x7f915019b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:08.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.108+0000 7f91576c0700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f915010f4f0 0x7f915019fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:08.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.108+0000 7f91576c0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9150114e00 con 0x7f915006b070 2024-04-22T13:23:08.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.108+0000 7f91576c0700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9150114c80 con 0x7f915006a750 2024-04-22T13:23:08.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.108+0000 7f91576c0700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9150114f80 con 0x7f915010f4f0 2024-04-22T13:23:08.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.108+0000 7f9154c5b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f915006b070 0x7f915019b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:08.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.108+0000 7f9154c5b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f915006b070 0x7f915019b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59216/0 (socket says 172.21.15.12:59216) 2024-04-22T13:23:08.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.108+0000 7f9154c5b700 1 -- 172.21.15.12:0/1101269181 learned_addr learned my addr 172.21.15.12:0/1101269181 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:08.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.108+0000 7f915545c700 1 --2- 172.21.15.12:0/1101269181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f915006a750 0x7f915019b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:08.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.108+0000 7f9155c5d700 1 --2- 172.21.15.12:0/1101269181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f915010f4f0 0x7f915019fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:08.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.109+0000 7f9154c5b700 1 -- 172.21.15.12:0/1101269181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f915010f4f0 msgr2=0x7f915019fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:08.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.109+0000 7f9154c5b700 1 --2- 172.21.15.12:0/1101269181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f915010f4f0 0x7f915019fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:08.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.109+0000 7f9154c5b700 1 -- 172.21.15.12:0/1101269181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f915006a750 msgr2=0x7f915019b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:08.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.109+0000 7f9154c5b700 1 --2- 172.21.15.12:0/1101269181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f915006a750 0x7f915019b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:08.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.109+0000 7f9154c5b700 1 -- 172.21.15.12:0/1101269181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91501a01f0 con 0x7f915006b070 2024-04-22T13:23:08.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.109+0000 7f9154c5b700 1 --2- 172.21.15.12:0/1101269181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f915006b070 0x7f915019b770 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f914400c9b0 tx=0x7f914400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:08.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.109+0000 7f914e7fc700 1 -- 172.21.15.12:0/1101269181 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144007b00 con 0x7f915006b070 2024-04-22T13:23:08.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.109+0000 7f914e7fc700 1 -- 172.21.15.12:0/1101269181 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9144007ca0 con 0x7f915006b070 2024-04-22T13:23:08.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.109+0000 7f91576c0700 1 -- 172.21.15.12:0/1101269181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91501a04e0 con 0x7f915006b070 2024-04-22T13:23:08.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.109+0000 7f914e7fc700 1 -- 172.21.15.12:0/1101269181 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144018660 con 0x7f915006b070 2024-04-22T13:23:08.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.109+0000 7f91576c0700 1 -- 172.21.15.12:0/1101269181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91501a7d60 con 0x7f915006b070 2024-04-22T13:23:08.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.110+0000 7f91576c0700 1 -- 172.21.15.12:0/1101269181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9150069730 con 0x7f915006b070 2024-04-22T13:23:08.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.111+0000 7f914e7fc700 1 -- 172.21.15.12:0/1101269181 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f914400f450 con 0x7f915006b070 2024-04-22T13:23:08.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.112+0000 7f914e7fc700 1 --2- 172.21.15.12:0/1101269181 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f91380a3cd0 0x7f91380a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:08.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.112+0000 7f915545c700 1 --2- 172.21.15.12:0/1101269181 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f91380a3cd0 0x7f91380a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:08.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.112+0000 7f914e7fc700 1 -- 172.21.15.12:0/1101269181 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f91440d0520 con 0x7f915006b070 2024-04-22T13:23:08.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.112+0000 7f915545c700 1 --2- 172.21.15.12:0/1101269181 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f91380a3cd0 0x7f91380a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9148009fd0 tx=0x7f914800b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:08.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.117+0000 7f914e7fc700 1 -- 172.21.15.12:0/1101269181 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9144014030 con 0x7f915006b070 2024-04-22T13:23:08.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.309+0000 7f4dd991f700 1 -- 172.21.15.12:0/2070275968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd4107920 msgr2=0x7f4dd4107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:08.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.309+0000 7f4dd991f700 1 --2- 172.21.15.12:0/2070275968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd4107920 0x7f4dd4107d00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009990 tx=0x7f4dc8009ca0 comp rx=0 tx=0).stop 2024-04-22T13:23:08.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.309+0000 7f4dd991f700 1 -- 172.21.15.12:0/2070275968 shutdown_connections 2024-04-22T13:23:08.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.309+0000 7f4dd991f700 1 --2- 172.21.15.12:0/2070275968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dd4102330 0x7f4dd410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:08.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.309+0000 7f4dd991f700 1 --2- 172.21.15.12:0/2070275968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd4107920 0x7f4dd4107d00 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:08.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.309+0000 7f4dd991f700 1 --2- 172.21.15.12:0/2070275968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dd4101970 0x7f4dd4101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:08.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.309+0000 7f4dd991f700 1 -- 172.21.15.12:0/2070275968 >> 172.21.15.12:0/2070275968 conn(0x7f4dd40788f0 msgr2=0x7f4dd4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:08.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.309+0000 7f4dd991f700 1 -- 172.21.15.12:0/2070275968 shutdown_connections 2024-04-22T13:23:08.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.309+0000 7f4dd991f700 1 -- 172.21.15.12:0/2070275968 wait complete. 2024-04-22T13:23:08.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.310+0000 7f4dd991f700 1 Processor -- start 2024-04-22T13:23:08.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.310+0000 7f4dd991f700 1 -- start start 2024-04-22T13:23:08.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.310+0000 7f4dd991f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dd4101970 0x7f4dd4199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:08.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.310+0000 7f4dd991f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd4102330 0x7f4dd41997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:08.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.310+0000 7f4dd991f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dd4107920 0x7f4dd419dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:08.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.310+0000 7f4dd991f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd4111b00 con 0x7f4dd4102330 2024-04-22T13:23:08.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.310+0000 7f4dd991f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd4111980 con 0x7f4dd4101970 2024-04-22T13:23:08.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.310+0000 7f4dd991f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd4111c80 con 0x7f4dd4107920 2024-04-22T13:23:08.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.310+0000 7f4dd27fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd4102330 0x7f4dd41997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:08.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.310+0000 7f4dd27fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd4102330 0x7f4dd41997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59226/0 (socket says 172.21.15.12:59226) 2024-04-22T13:23:08.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.310+0000 7f4dd27fc700 1 -- 172.21.15.12:0/2052051679 learned_addr learned my addr 172.21.15.12:0/2052051679 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:08.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.310+0000 7f4dd2ffd700 1 --2- 172.21.15.12:0/2052051679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dd4101970 0x7f4dd4199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:08.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.311+0000 7f4dd37fe700 1 --2- 172.21.15.12:0/2052051679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dd4107920 0x7f4dd419dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:08.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.311+0000 7f4dd27fc700 1 -- 172.21.15.12:0/2052051679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dd4107920 msgr2=0x7f4dd419dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:08.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.311+0000 7f4dd27fc700 1 --2- 172.21.15.12:0/2052051679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dd4107920 0x7f4dd419dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:08.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.311+0000 7f4dd27fc700 1 -- 172.21.15.12:0/2052051679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dd4101970 msgr2=0x7f4dd4199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:08.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.311+0000 7f4dd27fc700 1 --2- 172.21.15.12:0/2052051679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dd4101970 0x7f4dd4199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:08.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.311+0000 7f4dd27fc700 1 -- 172.21.15.12:0/2052051679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd419e220 con 0x7f4dd4102330 2024-04-22T13:23:08.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.311+0000 7f4dd27fc700 1 --2- 172.21.15.12:0/2052051679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd4102330 0x7f4dd41997a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4dc400d950 tx=0x7f4dc400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:08.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.311+0000 7f4dd891d700 1 -- 172.21.15.12:0/2052051679 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc4016070 con 0x7f4dd4102330 2024-04-22T13:23:08.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.311+0000 7f4dd991f700 1 -- 172.21.15.12:0/2052051679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dd419e510 con 0x7f4dd4102330 2024-04-22T13:23:08.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.311+0000 7f4dd891d700 1 -- 172.21.15.12:0/2052051679 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dc4009df0 con 0x7f4dd4102330 2024-04-22T13:23:08.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.311+0000 7f4dd891d700 1 -- 172.21.15.12:0/2052051679 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc400f3c0 con 0x7f4dd4102330 2024-04-22T13:23:08.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.311+0000 7f4dd991f700 1 -- 172.21.15.12:0/2052051679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dd41a5d90 con 0x7f4dd4102330 2024-04-22T13:23:08.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.312+0000 7f4dd991f700 1 -- 172.21.15.12:0/2052051679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dd4069730 con 0x7f4dd4102330 2024-04-22T13:23:08.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.314+0000 7f4dd891d700 1 -- 172.21.15.12:0/2052051679 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4dc4019070 con 0x7f4dd4102330 2024-04-22T13:23:08.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.314+0000 7f4dd891d700 1 --2- 172.21.15.12:0/2052051679 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f4dbc0a3cd0 0x7f4dbc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:08.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.315+0000 7f4dd2ffd700 1 --2- 172.21.15.12:0/2052051679 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f4dbc0a3cd0 0x7f4dbc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:08.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.315+0000 7f4dd891d700 1 -- 172.21.15.12:0/2052051679 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f4dc40d01b0 con 0x7f4dd4102330 2024-04-22T13:23:08.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.315+0000 7f4dd2ffd700 1 --2- 172.21.15.12:0/2052051679 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f4dbc0a3cd0 0x7f4dbc0a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4dc800b5c0 tx=0x7f4dc8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:08.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.321+0000 7f4dd891d700 1 -- 172.21.15.12:0/2052051679 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4dc409ca00 con 0x7f4dd4102330 2024-04-22T13:23:08.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.332+0000 7f91576c0700 1 -- 172.21.15.12:0/1101269181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f915002df80 con 0x7f915006b070 2024-04-22T13:23:08.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.332+0000 7f914e7fc700 1 -- 172.21.15.12:0/1101269181 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f914409cdd0 con 0x7f915006b070 2024-04-22T13:23:08.328 INFO:teuthology.orchestra.run.smithi012.stdout:115964117058 2024-04-22T13:23:08.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.334+0000 7f91576c0700 1 -- 172.21.15.12:0/1101269181 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f91380a3cd0 msgr2=0x7f91380a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:08.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.334+0000 7f91576c0700 1 --2- 172.21.15.12:0/1101269181 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f91380a3cd0 0x7f91380a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9148009fd0 tx=0x7f914800b7b0 comp rx=0 tx=0).stop 2024-04-22T13:23:08.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.334+0000 7f91576c0700 1 -- 172.21.15.12:0/1101269181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f915006b070 msgr2=0x7f915019b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:08.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.334+0000 7f91576c0700 1 --2- 172.21.15.12:0/1101269181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f915006b070 0x7f915019b770 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f914400c9b0 tx=0x7f914400ccc0 comp rx=0 tx=0).stop 2024-04-22T13:23:08.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.334+0000 7f91576c0700 1 -- 172.21.15.12:0/1101269181 shutdown_connections 2024-04-22T13:23:08.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.334+0000 7f91576c0700 1 --2- 172.21.15.12:0/1101269181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f915006a750 0x7f915019b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:08.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.334+0000 7f91576c0700 1 --2- 172.21.15.12:0/1101269181 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f91380a3cd0 0x7f91380a6190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:08.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.334+0000 7f91576c0700 1 --2- 172.21.15.12:0/1101269181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f915010f4f0 0x7f915019fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:08.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.334+0000 7f91576c0700 1 --2- 172.21.15.12:0/1101269181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f915006b070 0x7f915019b770 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:08.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.334+0000 7f91576c0700 1 -- 172.21.15.12:0/1101269181 >> 172.21.15.12:0/1101269181 conn(0x7f91500788f0 msgr2=0x7f91501108a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:08.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.335+0000 7f91576c0700 1 -- 172.21.15.12:0/1101269181 shutdown_connections 2024-04-22T13:23:08.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.335+0000 7f91576c0700 1 -- 172.21.15.12:0/1101269181 wait complete. 2024-04-22T13:23:08.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.537+0000 7f4dd991f700 1 -- 172.21.15.12:0/2052051679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f4dd402df80 con 0x7f4dd4102330 2024-04-22T13:23:08.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.538+0000 7f4dd891d700 1 -- 172.21.15.12:0/2052051679 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f4dc409c300 con 0x7f4dd4102330 2024-04-22T13:23:08.538 INFO:teuthology.orchestra.run.smithi012.stdout:352187318299 2024-04-22T13:23:08.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.540+0000 7f4dd991f700 1 -- 172.21.15.12:0/2052051679 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f4dbc0a3cd0 msgr2=0x7f4dbc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:08.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.540+0000 7f4dd991f700 1 --2- 172.21.15.12:0/2052051679 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f4dbc0a3cd0 0x7f4dbc0a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4dc800b5c0 tx=0x7f4dc8005960 comp rx=0 tx=0).stop 2024-04-22T13:23:08.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.540+0000 7f4dd991f700 1 -- 172.21.15.12:0/2052051679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd4102330 msgr2=0x7f4dd41997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:08.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.540+0000 7f4dd991f700 1 --2- 172.21.15.12:0/2052051679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd4102330 0x7f4dd41997a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4dc400d950 tx=0x7f4dc400dc60 comp rx=0 tx=0).stop 2024-04-22T13:23:08.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.540+0000 7f4dd991f700 1 -- 172.21.15.12:0/2052051679 shutdown_connections 2024-04-22T13:23:08.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.540+0000 7f4dd991f700 1 --2- 172.21.15.12:0/2052051679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4dd4101970 0x7f4dd4199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:08.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.540+0000 7f4dd991f700 1 --2- 172.21.15.12:0/2052051679 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f4dbc0a3cd0 0x7f4dbc0a6190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:08.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.540+0000 7f4dd991f700 1 --2- 172.21.15.12:0/2052051679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4dd4107920 0x7f4dd419dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:08.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.540+0000 7f4dd991f700 1 --2- 172.21.15.12:0/2052051679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4dd4102330 0x7f4dd41997a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:08.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.540+0000 7f4dd991f700 1 -- 172.21.15.12:0/2052051679 >> 172.21.15.12:0/2052051679 conn(0x7f4dd40788f0 msgr2=0x7f4dd406b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:08.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.540+0000 7f4dd991f700 1 -- 172.21.15.12:0/2052051679 shutdown_connections 2024-04-22T13:23:08.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.540+0000 7f4dd991f700 1 -- 172.21.15.12:0/2052051679 wait complete. 2024-04-22T13:23:08.558 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:08 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2205923393' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-22T13:23:08.558 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:08 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1101269181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-22T13:23:08.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:08 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2205923393' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-22T13:23:08.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:08 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1101269181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-22T13:23:08.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:08 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2205923393' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-22T13:23:08.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:08 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1101269181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-22T13:23:09.561 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:09 smithi012 ceph-mon[28265]: pgmap v321: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-22T13:23:09.561 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:09 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2052051679' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-22T13:23:09.561 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:09 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2446340347' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-22T13:23:09.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.553+0000 7f1f4619a700 1 -- 172.21.15.12:0/2004198869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f40101940 msgr2=0x7f1f40101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:09.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.553+0000 7f1f4619a700 1 --2- 172.21.15.12:0/2004198869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f40101940 0x7f1f40101dc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1f34009990 tx=0x7f1f34009ca0 comp rx=0 tx=0).stop 2024-04-22T13:23:09.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.553+0000 7f1f4619a700 1 -- 172.21.15.12:0/2004198869 shutdown_connections 2024-04-22T13:23:09.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.553+0000 7f1f4619a700 1 --2- 172.21.15.12:0/2004198869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f40102300 0x7f1f4010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:09.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.553+0000 7f1f4619a700 1 --2- 172.21.15.12:0/2004198869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f40107990 0x7f1f40107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:09.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.553+0000 7f1f4619a700 1 --2- 172.21.15.12:0/2004198869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f40101940 0x7f1f40101dc0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:09.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.553+0000 7f1f4619a700 1 -- 172.21.15.12:0/2004198869 >> 172.21.15.12:0/2004198869 conn(0x7f1f400788f0 msgr2=0x7f1f40078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:09.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.553+0000 7f1f4619a700 1 -- 172.21.15.12:0/2004198869 shutdown_connections 2024-04-22T13:23:09.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.553+0000 7f1f4619a700 1 -- 172.21.15.12:0/2004198869 wait complete. 2024-04-22T13:23:09.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.553+0000 7f1f4619a700 1 Processor -- start 2024-04-22T13:23:09.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.554+0000 7f1f4619a700 1 -- start start 2024-04-22T13:23:09.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.554+0000 7f1f4619a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f40101940 0x7f1f40199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:09.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.554+0000 7f1f4619a700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f40102300 0x7f1f4019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:09.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.554+0000 7f1f4619a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f40107990 0x7f1f401a5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:09.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.554+0000 7f1f4619a700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1f40111b00 con 0x7f1f40101940 2024-04-22T13:23:09.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.554+0000 7f1f4619a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1f40111980 con 0x7f1f40107990 2024-04-22T13:23:09.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.554+0000 7f1f4619a700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1f40111c80 con 0x7f1f40102300 2024-04-22T13:23:09.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.554+0000 7f1f3f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f40101940 0x7f1f40199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:09.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.554+0000 7f1f3f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f40101940 0x7f1f40199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59248/0 (socket says 172.21.15.12:59248) 2024-04-22T13:23:09.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.554+0000 7f1f3f7fe700 1 -- 172.21.15.12:0/2446340347 learned_addr learned my addr 172.21.15.12:0/2446340347 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:09.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.554+0000 7f1f3effd700 1 --2- 172.21.15.12:0/2446340347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f40102300 0x7f1f4019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:09.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.554+0000 7f1f3ffff700 1 --2- 172.21.15.12:0/2446340347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f40107990 0x7f1f401a5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:09.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.554+0000 7f1f3f7fe700 1 -- 172.21.15.12:0/2446340347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f40102300 msgr2=0x7f1f4019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:09.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.554+0000 7f1f3f7fe700 1 --2- 172.21.15.12:0/2446340347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f40102300 0x7f1f4019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:09.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.554+0000 7f1f3f7fe700 1 -- 172.21.15.12:0/2446340347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f40107990 msgr2=0x7f1f401a5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:09.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.554+0000 7f1f3f7fe700 1 --2- 172.21.15.12:0/2446340347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f40107990 0x7f1f401a5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:09.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.554+0000 7f1f3f7fe700 1 -- 172.21.15.12:0/2446340347 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f401a5560 con 0x7f1f40101940 2024-04-22T13:23:09.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.554+0000 7f1f3f7fe700 1 --2- 172.21.15.12:0/2446340347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f40101940 0x7f1f40199fa0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000c970 tx=0x7f1f3000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:09.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.555+0000 7f1f3cff9700 1 -- 172.21.15.12:0/2446340347 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f30007aa0 con 0x7f1f40101940 2024-04-22T13:23:09.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.555+0000 7f1f4619a700 1 -- 172.21.15.12:0/2446340347 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f401a5790 con 0x7f1f40101940 2024-04-22T13:23:09.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.555+0000 7f1f3cff9700 1 -- 172.21.15.12:0/2446340347 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f30004ce0 con 0x7f1f40101940 2024-04-22T13:23:09.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.555+0000 7f1f3cff9700 1 -- 172.21.15.12:0/2446340347 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f30005510 con 0x7f1f40101940 2024-04-22T13:23:09.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.555+0000 7f1f4619a700 1 -- 172.21.15.12:0/2446340347 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f401a5ca0 con 0x7f1f40101940 2024-04-22T13:23:09.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.556+0000 7f1f4619a700 1 -- 172.21.15.12:0/2446340347 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f40109eb0 con 0x7f1f40101940 2024-04-22T13:23:09.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.557+0000 7f1f3cff9700 1 -- 172.21.15.12:0/2446340347 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1f30007c40 con 0x7f1f40101940 2024-04-22T13:23:09.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.557+0000 7f1f3cff9700 1 --2- 172.21.15.12:0/2446340347 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1f280a3cd0 0x7f1f280a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:09.688 INFO:teuthology.orchestra.run.smithi012.stdout:317827579934 2024-04-22T13:23:09.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.558+0000 7f1f3effd700 1 --2- 172.21.15.12:0/2446340347 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1f280a3cd0 0x7f1f280a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:09.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.558+0000 7f1f3cff9700 1 -- 172.21.15.12:0/2446340347 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f1f300d05a0 con 0x7f1f40101940 2024-04-22T13:23:09.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.558+0000 7f1f3effd700 1 --2- 172.21.15.12:0/2446340347 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1f280a3cd0 0x7f1f280a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400b5c0 tx=0x7f1f34005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:09.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.561+0000 7f1f3cff9700 1 -- 172.21.15.12:0/2446340347 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1f3009cf10 con 0x7f1f40101940 2024-04-22T13:23:09.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.776+0000 7f1f4619a700 1 -- 172.21.15.12:0/2446340347 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f1f4006ad30 con 0x7f1f40101940 2024-04-22T13:23:09.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.776+0000 7f1f3cff9700 1 -- 172.21.15.12:0/2446340347 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f1f3009c810 con 0x7f1f40101940 2024-04-22T13:23:09.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.778+0000 7f1f4619a700 1 -- 172.21.15.12:0/2446340347 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1f280a3cd0 msgr2=0x7f1f280a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:09.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.778+0000 7f1f4619a700 1 --2- 172.21.15.12:0/2446340347 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1f280a3cd0 0x7f1f280a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400b5c0 tx=0x7f1f34005960 comp rx=0 tx=0).stop 2024-04-22T13:23:09.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.778+0000 7f1f4619a700 1 -- 172.21.15.12:0/2446340347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f40101940 msgr2=0x7f1f40199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:09.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.778+0000 7f1f4619a700 1 --2- 172.21.15.12:0/2446340347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f40101940 0x7f1f40199fa0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000c970 tx=0x7f1f3000cc80 comp rx=0 tx=0).stop 2024-04-22T13:23:09.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.779+0000 7f1f4619a700 1 -- 172.21.15.12:0/2446340347 shutdown_connections 2024-04-22T13:23:09.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.779+0000 7f1f4619a700 1 --2- 172.21.15.12:0/2446340347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f40107990 0x7f1f401a5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:09.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.779+0000 7f1f4619a700 1 --2- 172.21.15.12:0/2446340347 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1f280a3cd0 0x7f1f280a6190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:09.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.779+0000 7f1f4619a700 1 --2- 172.21.15.12:0/2446340347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f40101940 0x7f1f40199fa0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:09.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.779+0000 7f1f4619a700 1 --2- 172.21.15.12:0/2446340347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f40102300 0x7f1f4019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:09.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.779+0000 7f1f4619a700 1 -- 172.21.15.12:0/2446340347 >> 172.21.15.12:0/2446340347 conn(0x7f1f400788f0 msgr2=0x7f1f400ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:09.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.779+0000 7f1f4619a700 1 -- 172.21.15.12:0/2446340347 shutdown_connections 2024-04-22T13:23:09.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:08.779+0000 7f1f4619a700 1 -- 172.21.15.12:0/2446340347 wait complete. 2024-04-22T13:23:09.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:09 smithi134 ceph-mon[31338]: pgmap v321: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-22T13:23:09.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:09 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2052051679' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-22T13:23:09.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:09 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2446340347' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-22T13:23:09.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:09 smithi111 ceph-mon[31361]: pgmap v321: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-22T13:23:09.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:09 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2052051679' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-22T13:23:09.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:09 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2446340347' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-22T13:23:10.304 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579929 got 317827579934 for osd.10 2024-04-22T13:23:10.304 DEBUG:teuthology.parallel:result is None 2024-04-22T13:23:10.688 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:23:10.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:10 smithi012 ceph-mon[28265]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-04-22T13:23:10.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:10 smithi012 ceph-mon[28265]: Cluster is now healthy 2024-04-22T13:23:10.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:10 smithi134 ceph-mon[31338]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-04-22T13:23:10.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:10 smithi134 ceph-mon[31338]: Cluster is now healthy 2024-04-22T13:23:10.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:10 smithi111 ceph-mon[31361]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-04-22T13:23:10.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:10 smithi111 ceph-mon[31361]: Cluster is now healthy 2024-04-22T13:23:11.110 INFO:tasks.cephadm.ceph_manager.ceph:need seq 352187318293 got 352187318299 for osd.11 2024-04-22T13:23:11.110 DEBUG:teuthology.parallel:result is None 2024-04-22T13:23:11.561 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855416 got 150323855421 for osd.4 2024-04-22T13:23:11.561 DEBUG:teuthology.parallel:result is None 2024-04-22T13:23:11.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.764+0000 7f962b14f700 1 -- 172.21.15.12:0/3471712622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f962406bbb0 msgr2=0x7f962410f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:11.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.764+0000 7f962b14f700 1 --2- 172.21.15.12:0/3471712622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f962406bbb0 0x7f962410f930 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f961c00b0a0 tx=0x7f961c00b3b0 comp rx=0 tx=0).stop 2024-04-22T13:23:11.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.764+0000 7f962b14f700 1 -- 172.21.15.12:0/3471712622 shutdown_connections 2024-04-22T13:23:11.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.764+0000 7f962b14f700 1 --2- 172.21.15.12:0/3471712622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9624108a60 0x7f9624108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:11.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.764+0000 7f962b14f700 1 --2- 172.21.15.12:0/3471712622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f962406bbb0 0x7f962410f930 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:11.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.764+0000 7f962b14f700 1 --2- 172.21.15.12:0/3471712622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f962406b1f0 0x7f962406b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:11.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.764+0000 7f962b14f700 1 -- 172.21.15.12:0/3471712622 >> 172.21.15.12:0/3471712622 conn(0x7f96240789e0 msgr2=0x7f9624078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:11.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.764+0000 7f962b14f700 1 -- 172.21.15.12:0/3471712622 shutdown_connections 2024-04-22T13:23:11.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.764+0000 7f962b14f700 1 -- 172.21.15.12:0/3471712622 wait complete. 2024-04-22T13:23:11.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.765+0000 7f962b14f700 1 Processor -- start 2024-04-22T13:23:11.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.765+0000 7f962b14f700 1 -- start start 2024-04-22T13:23:11.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.765+0000 7f962b14f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f962406b1f0 0x7f96241990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:11.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.765+0000 7f962b14f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f962406bbb0 0x7f9624199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:11.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.765+0000 7f962b14f700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9624108a60 0x7f962419d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:11.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.765+0000 7f962b14f700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9624112ca0 con 0x7f962406b1f0 2024-04-22T13:23:11.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.765+0000 7f962b14f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9624112b20 con 0x7f962406bbb0 2024-04-22T13:23:11.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.765+0000 7f962b14f700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9624112e20 con 0x7f9624108a60 2024-04-22T13:23:11.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.765+0000 7f9628eeb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f962406b1f0 0x7f96241990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:11.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.765+0000 7f9628eeb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f962406b1f0 0x7f96241990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49288/0 (socket says 172.21.15.12:49288) 2024-04-22T13:23:11.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.765+0000 7f9628eeb700 1 -- 172.21.15.12:0/1725853063 learned_addr learned my addr 172.21.15.12:0/1725853063 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:11.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.765+0000 7f9623fff700 1 --2- 172.21.15.12:0/1725853063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f962406bbb0 0x7f9624199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:11.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.766+0000 7f96296ec700 1 --2- 172.21.15.12:0/1725853063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9624108a60 0x7f962419d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:11.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.766+0000 7f9628eeb700 1 -- 172.21.15.12:0/1725853063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9624108a60 msgr2=0x7f962419d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:11.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.766+0000 7f9628eeb700 1 --2- 172.21.15.12:0/1725853063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9624108a60 0x7f962419d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:11.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.766+0000 7f9628eeb700 1 -- 172.21.15.12:0/1725853063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f962406bbb0 msgr2=0x7f9624199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:11.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.766+0000 7f9628eeb700 1 --2- 172.21.15.12:0/1725853063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f962406bbb0 0x7f9624199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:11.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.766+0000 7f9628eeb700 1 -- 172.21.15.12:0/1725853063 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f962419e0b0 con 0x7f962406b1f0 2024-04-22T13:23:11.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.766+0000 7f9623fff700 1 --2- 172.21.15.12:0/1725853063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f962406bbb0 0x7f9624199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:23:11.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.766+0000 7f96296ec700 1 --2- 172.21.15.12:0/1725853063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9624108a60 0x7f962419d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:23:11.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.766+0000 7f9628eeb700 1 --2- 172.21.15.12:0/1725853063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f962406b1f0 0x7f96241990f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f961400c9b0 tx=0x7f961400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:11.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.766+0000 7f9621ffb700 1 -- 172.21.15.12:0/1725853063 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614007b00 con 0x7f962406b1f0 2024-04-22T13:23:11.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.766+0000 7f9621ffb700 1 -- 172.21.15.12:0/1725853063 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9614007ca0 con 0x7f962406b1f0 2024-04-22T13:23:11.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.766+0000 7f9621ffb700 1 -- 172.21.15.12:0/1725853063 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614018730 con 0x7f962406b1f0 2024-04-22T13:23:11.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.766+0000 7f962b14f700 1 -- 172.21.15.12:0/1725853063 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f962419e3a0 con 0x7f962406b1f0 2024-04-22T13:23:11.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.766+0000 7f962b14f700 1 -- 172.21.15.12:0/1725853063 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96241a5c20 con 0x7f962406b1f0 2024-04-22T13:23:11.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.767+0000 7f962b14f700 1 -- 172.21.15.12:0/1725853063 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9624069730 con 0x7f962406b1f0 2024-04-22T13:23:11.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.768+0000 7f9621ffb700 1 -- 172.21.15.12:0/1725853063 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f961400f450 con 0x7f962406b1f0 2024-04-22T13:23:11.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.769+0000 7f9621ffb700 1 --2- 172.21.15.12:0/1725853063 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f96100a3cd0 0x7f96100a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:11.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.769+0000 7f9623fff700 1 --2- 172.21.15.12:0/1725853063 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f96100a3cd0 0x7f96100a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:11.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.769+0000 7f9621ffb700 1 -- 172.21.15.12:0/1725853063 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f96140d05f0 con 0x7f962406b1f0 2024-04-22T13:23:11.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.769+0000 7f9623fff700 1 --2- 172.21.15.12:0/1725853063 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f96100a3cd0 0x7f96100a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9624078ee0 tx=0x7f960c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:11.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.774+0000 7f9621ffb700 1 -- 172.21.15.12:0/1725853063 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9614014030 con 0x7f962406b1f0 2024-04-22T13:23:11.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:11 smithi012 ceph-mon[28265]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2024-04-22T13:23:11.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:11 smithi134 ceph-mon[31338]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2024-04-22T13:23:11.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:11 smithi111 ceph-mon[31361]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2024-04-22T13:23:11.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.986+0000 7f962b14f700 1 -- 172.21.15.12:0/1725853063 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f962419a400 con 0x7f962406b1f0 2024-04-22T13:23:11.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.986+0000 7f9621ffb700 1 -- 172.21.15.12:0/1725853063 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f961409cea0 con 0x7f962406b1f0 2024-04-22T13:23:11.983 INFO:teuthology.orchestra.run.smithi012.stdout:171798691896 2024-04-22T13:23:11.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.988+0000 7f962b14f700 1 -- 172.21.15.12:0/1725853063 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f96100a3cd0 msgr2=0x7f96100a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:11.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.988+0000 7f962b14f700 1 --2- 172.21.15.12:0/1725853063 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f96100a3cd0 0x7f96100a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9624078ee0 tx=0x7f960c008040 comp rx=0 tx=0).stop 2024-04-22T13:23:11.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.988+0000 7f962b14f700 1 -- 172.21.15.12:0/1725853063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f962406b1f0 msgr2=0x7f96241990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:11.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.988+0000 7f962b14f700 1 --2- 172.21.15.12:0/1725853063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f962406b1f0 0x7f96241990f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f961400c9b0 tx=0x7f961400cd70 comp rx=0 tx=0).stop 2024-04-22T13:23:11.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.988+0000 7f962b14f700 1 -- 172.21.15.12:0/1725853063 shutdown_connections 2024-04-22T13:23:11.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.988+0000 7f962b14f700 1 --2- 172.21.15.12:0/1725853063 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f96100a3cd0 0x7f96100a6190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:11.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.988+0000 7f962b14f700 1 --2- 172.21.15.12:0/1725853063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9624108a60 0x7f962419d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:11.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.988+0000 7f962b14f700 1 --2- 172.21.15.12:0/1725853063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f962406b1f0 0x7f96241990f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:11.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.988+0000 7f962b14f700 1 --2- 172.21.15.12:0/1725853063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f962406bbb0 0x7f9624199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:11.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.988+0000 7f962b14f700 1 -- 172.21.15.12:0/1725853063 >> 172.21.15.12:0/1725853063 conn(0x7f96240789e0 msgr2=0x7f96240ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:11.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.989+0000 7f962b14f700 1 -- 172.21.15.12:0/1725853063 shutdown_connections 2024-04-22T13:23:11.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:11.989+0000 7f962b14f700 1 -- 172.21.15.12:0/1725853063 wait complete. 2024-04-22T13:23:12.146 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117054 got 115964117058 for osd.3 2024-04-22T13:23:12.147 DEBUG:teuthology.parallel:result is None 2024-04-22T13:23:12.690 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:12 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1725853063' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-22T13:23:12.720 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691891 got 171798691896 for osd.5 2024-04-22T13:23:12.720 DEBUG:teuthology.parallel:result is None 2024-04-22T13:23:12.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:12 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1725853063' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-22T13:23:12.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:12 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1725853063' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-22T13:23:13.166 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574920 got 55834574923 for osd.1 2024-04-22T13:23:13.166 DEBUG:teuthology.parallel:result is None 2024-04-22T13:23:13.663 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:13 smithi012 ceph-mon[28265]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-22T13:23:13.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:13 smithi134 ceph-mon[31338]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-22T13:23:13.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:13 smithi111 ceph-mon[31361]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-22T13:23:14.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.251+0000 7f4a38837700 1 -- 172.21.15.12:0/1172252606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a34107990 msgr2=0x7f4a34107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:14.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.251+0000 7f4a38837700 1 --2- 172.21.15.12:0/1172252606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a34107990 0x7f4a34107d70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4a24009990 tx=0x7f4a24009ca0 comp rx=0 tx=0).stop 2024-04-22T13:23:14.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.251+0000 7f4a38837700 1 -- 172.21.15.12:0/1172252606 shutdown_connections 2024-04-22T13:23:14.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.251+0000 7f4a38837700 1 --2- 172.21.15.12:0/1172252606 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a34102300 0x7f4a3410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:14.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.251+0000 7f4a38837700 1 --2- 172.21.15.12:0/1172252606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a34107990 0x7f4a34107d70 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:14.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.251+0000 7f4a38837700 1 --2- 172.21.15.12:0/1172252606 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a34101940 0x7f4a34101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:14.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.251+0000 7f4a38837700 1 -- 172.21.15.12:0/1172252606 >> 172.21.15.12:0/1172252606 conn(0x7f4a340788f0 msgr2=0x7f4a34078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:14.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.251+0000 7f4a38837700 1 -- 172.21.15.12:0/1172252606 shutdown_connections 2024-04-22T13:23:14.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.251+0000 7f4a38837700 1 -- 172.21.15.12:0/1172252606 wait complete. 2024-04-22T13:23:14.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.252+0000 7f4a38837700 1 Processor -- start 2024-04-22T13:23:14.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.252+0000 7f4a38837700 1 -- start start 2024-04-22T13:23:14.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.252+0000 7f4a38837700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a34101940 0x7f4a34199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:14.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.252+0000 7f4a38837700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a34102300 0x7f4a3419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:14.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.252+0000 7f4a38837700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a34107990 0x7f4a341a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:14.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.252+0000 7f4a38837700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4a34111b60 con 0x7f4a34107990 2024-04-22T13:23:14.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.252+0000 7f4a38837700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4a341119e0 con 0x7f4a34102300 2024-04-22T13:23:14.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.252+0000 7f4a38837700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4a34111ce0 con 0x7f4a34101940 2024-04-22T13:23:14.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.252+0000 7f4a32d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a34107990 0x7f4a341a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:14.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.252+0000 7f4a32d9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a34107990 0x7f4a341a5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49348/0 (socket says 172.21.15.12:49348) 2024-04-22T13:23:14.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.252+0000 7f4a32d9d700 1 -- 172.21.15.12:0/1850297660 learned_addr learned my addr 172.21.15.12:0/1850297660 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:14.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.252+0000 7f4a31d9b700 1 --2- 172.21.15.12:0/1850297660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a34102300 0x7f4a3419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:14.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.253+0000 7f4a3259c700 1 --2- 172.21.15.12:0/1850297660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a34101940 0x7f4a34199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:14.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.253+0000 7f4a32d9d700 1 -- 172.21.15.12:0/1850297660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a34101940 msgr2=0x7f4a34199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:14.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.253+0000 7f4a32d9d700 1 --2- 172.21.15.12:0/1850297660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a34101940 0x7f4a34199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:14.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.253+0000 7f4a32d9d700 1 -- 172.21.15.12:0/1850297660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a34102300 msgr2=0x7f4a3419a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:14.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.253+0000 7f4a32d9d700 1 --2- 172.21.15.12:0/1850297660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a34102300 0x7f4a3419a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:14.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.253+0000 7f4a32d9d700 1 -- 172.21.15.12:0/1850297660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a341a5580 con 0x7f4a34107990 2024-04-22T13:23:14.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.253+0000 7f4a3259c700 1 --2- 172.21.15.12:0/1850297660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a34101940 0x7f4a34199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:23:14.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.253+0000 7f4a32d9d700 1 --2- 172.21.15.12:0/1850297660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a34107990 0x7f4a341a5040 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c00b970 tx=0x7f4a2c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:14.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.253+0000 7f4a237fe700 1 -- 172.21.15.12:0/1850297660 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c019070 con 0x7f4a34107990 2024-04-22T13:23:14.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.253+0000 7f4a38837700 1 -- 172.21.15.12:0/1850297660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a341a5750 con 0x7f4a34107990 2024-04-22T13:23:14.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.253+0000 7f4a237fe700 1 -- 172.21.15.12:0/1850297660 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a2c00cde0 con 0x7f4a34107990 2024-04-22T13:23:14.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.253+0000 7f4a237fe700 1 -- 172.21.15.12:0/1850297660 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c007370 con 0x7f4a34107990 2024-04-22T13:23:14.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.253+0000 7f4a38837700 1 -- 172.21.15.12:0/1850297660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a341a5cb0 con 0x7f4a34107990 2024-04-22T13:23:14.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.254+0000 7f4a38837700 1 -- 172.21.15.12:0/1850297660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a34109eb0 con 0x7f4a34107990 2024-04-22T13:23:14.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.255+0000 7f4a237fe700 1 -- 172.21.15.12:0/1850297660 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4a2c01c070 con 0x7f4a34107990 2024-04-22T13:23:14.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.256+0000 7f4a237fe700 1 --2- 172.21.15.12:0/1850297660 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f4a1c0a3cd0 0x7f4a1c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:14.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.256+0000 7f4a3259c700 1 --2- 172.21.15.12:0/1850297660 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f4a1c0a3cd0 0x7f4a1c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:14.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.256+0000 7f4a237fe700 1 -- 172.21.15.12:0/1850297660 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f4a2c0cf970 con 0x7f4a34107990 2024-04-22T13:23:14.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.256+0000 7f4a3259c700 1 --2- 172.21.15.12:0/1850297660 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f4a1c0a3cd0 0x7f4a1c0a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4a24006010 tx=0x7f4a2401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:14.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.260+0000 7f4a237fe700 1 -- 172.21.15.12:0/1850297660 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4a2c09c2d0 con 0x7f4a34107990 2024-04-22T13:23:14.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.474+0000 7f4a38837700 1 -- 172.21.15.12:0/1850297660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f4a3406acc0 con 0x7f4a34107990 2024-04-22T13:23:14.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.474+0000 7f4a237fe700 1 -- 172.21.15.12:0/1850297660 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4a2c09bbd0 con 0x7f4a34107990 2024-04-22T13:23:14.471 INFO:teuthology.orchestra.run.smithi012.stdout:261993005097 2024-04-22T13:23:14.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.476+0000 7f4a38837700 1 -- 172.21.15.12:0/1850297660 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f4a1c0a3cd0 msgr2=0x7f4a1c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:14.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.476+0000 7f4a38837700 1 --2- 172.21.15.12:0/1850297660 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f4a1c0a3cd0 0x7f4a1c0a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4a24006010 tx=0x7f4a2401a040 comp rx=0 tx=0).stop 2024-04-22T13:23:14.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.476+0000 7f4a38837700 1 -- 172.21.15.12:0/1850297660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a34107990 msgr2=0x7f4a341a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:14.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.476+0000 7f4a38837700 1 --2- 172.21.15.12:0/1850297660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a34107990 0x7f4a341a5040 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c00b970 tx=0x7f4a2c00bc80 comp rx=0 tx=0).stop 2024-04-22T13:23:14.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.476+0000 7f4a38837700 1 -- 172.21.15.12:0/1850297660 shutdown_connections 2024-04-22T13:23:14.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.476+0000 7f4a38837700 1 --2- 172.21.15.12:0/1850297660 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f4a1c0a3cd0 0x7f4a1c0a6190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:14.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.476+0000 7f4a38837700 1 --2- 172.21.15.12:0/1850297660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a34107990 0x7f4a341a5040 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:14.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.476+0000 7f4a38837700 1 --2- 172.21.15.12:0/1850297660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a34101940 0x7f4a34199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:14.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.476+0000 7f4a38837700 1 --2- 172.21.15.12:0/1850297660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a34102300 0x7f4a3419a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:14.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.476+0000 7f4a38837700 1 -- 172.21.15.12:0/1850297660 >> 172.21.15.12:0/1850297660 conn(0x7f4a340788f0 msgr2=0x7f4a34100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:14.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.477+0000 7f4a38837700 1 -- 172.21.15.12:0/1850297660 shutdown_connections 2024-04-22T13:23:14.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:14.477+0000 7f4a38837700 1 -- 172.21.15.12:0/1850297660 wait complete. 2024-04-22T13:23:14.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:14 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1185523137' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-22T13:23:14.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:14 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1620276038' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-22T13:23:14.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:14 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1850297660' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-22T13:23:14.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:14 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1185523137' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-22T13:23:14.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:14 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1620276038' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-22T13:23:14.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:14 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1850297660' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-22T13:23:14.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:14 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1185523137' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-22T13:23:14.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:14 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1620276038' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-22T13:23:14.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:14 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1850297660' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-22T13:23:15.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.583+0000 7fbbe5729700 1 -- 172.21.15.12:0/2804902570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbe0074c70 msgr2=0x7fbbe0075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:15.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.583+0000 7fbbe5729700 1 --2- 172.21.15.12:0/2804902570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbe0074c70 0x7fbbe0075050 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0009990 tx=0x7fbbd0009ca0 comp rx=0 tx=0).stop 2024-04-22T13:23:15.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.583+0000 7fbbe5729700 1 -- 172.21.15.12:0/2804902570 shutdown_connections 2024-04-22T13:23:15.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.583+0000 7fbbe5729700 1 --2- 172.21.15.12:0/2804902570 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbe010f470 0x7fbbe0111860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:15.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.583+0000 7fbbe5729700 1 --2- 172.21.15.12:0/2804902570 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbe0074c70 0x7fbbe0075050 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:15.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.583+0000 7fbbe5729700 1 --2- 172.21.15.12:0/2804902570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbe0075590 0x7fbbe010ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:15.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.583+0000 7fbbe5729700 1 -- 172.21.15.12:0/2804902570 >> 172.21.15.12:0/2804902570 conn(0x7fbbe0079bf0 msgr2=0x7fbbe007a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:15.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.583+0000 7fbbe5729700 1 -- 172.21.15.12:0/2804902570 shutdown_connections 2024-04-22T13:23:15.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.583+0000 7fbbe5729700 1 -- 172.21.15.12:0/2804902570 wait complete. 2024-04-22T13:23:15.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.583+0000 7fbbe5729700 1 Processor -- start 2024-04-22T13:23:15.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.584+0000 7fbbe5729700 1 -- start start 2024-04-22T13:23:15.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.584+0000 7fbbe5729700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbe0074c70 0x7fbbe019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:15.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.584+0000 7fbbe5729700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbe0075590 0x7fbbe019b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:15.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.584+0000 7fbbe5729700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbe010f470 0x7fbbe019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:15.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.584+0000 7fbbe5729700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe0114ec0 con 0x7fbbe0074c70 2024-04-22T13:23:15.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.584+0000 7fbbe5729700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe0114d40 con 0x7fbbe0075590 2024-04-22T13:23:15.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.584+0000 7fbbe5729700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe0115040 con 0x7fbbe010f470 2024-04-22T13:23:15.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.584+0000 7fbbdeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbe0074c70 0x7fbbe019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:15.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.584+0000 7fbbdeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbe0074c70 0x7fbbe019b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49308/0 (socket says 172.21.15.12:49308) 2024-04-22T13:23:15.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.584+0000 7fbbdeffd700 1 -- 172.21.15.12:0/1185523137 learned_addr learned my addr 172.21.15.12:0/1185523137 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:15.292 INFO:teuthology.orchestra.run.smithi012.stdout:231928234031 2024-04-22T13:23:15.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.584+0000 7fbbde7fc700 1 --2- 172.21.15.12:0/1185523137 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbe0075590 0x7fbbe019b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:15.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.584+0000 7fbbdf7fe700 1 --2- 172.21.15.12:0/1185523137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbe010f470 0x7fbbe019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:15.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.584+0000 7fbbdeffd700 1 -- 172.21.15.12:0/1185523137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbe010f470 msgr2=0x7fbbe019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:15.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.584+0000 7fbbdeffd700 1 --2- 172.21.15.12:0/1185523137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbe010f470 0x7fbbe019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:15.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.584+0000 7fbbdeffd700 1 -- 172.21.15.12:0/1185523137 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbe0075590 msgr2=0x7fbbe019b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:15.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.584+0000 7fbbdeffd700 1 --2- 172.21.15.12:0/1185523137 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbe0075590 0x7fbbe019b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:15.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.584+0000 7fbbdeffd700 1 -- 172.21.15.12:0/1185523137 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe01a03e0 con 0x7fbbe0074c70 2024-04-22T13:23:15.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.584+0000 7fbbdeffd700 1 --2- 172.21.15.12:0/1185523137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbe0074c70 0x7fbbe019b3d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0000c00 tx=0x7fbbd00057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:15.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.585+0000 7fbbc7fff700 1 -- 172.21.15.12:0/1185523137 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0027070 con 0x7fbbe0074c70 2024-04-22T13:23:15.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.585+0000 7fbbc7fff700 1 -- 172.21.15.12:0/1185523137 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbd0022470 con 0x7fbbe0074c70 2024-04-22T13:23:15.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.585+0000 7fbbe5729700 1 -- 172.21.15.12:0/1185523137 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe01a0670 con 0x7fbbe0074c70 2024-04-22T13:23:15.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.585+0000 7fbbc7fff700 1 -- 172.21.15.12:0/1185523137 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0017650 con 0x7fbbe0074c70 2024-04-22T13:23:15.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.585+0000 7fbbe5729700 1 -- 172.21.15.12:0/1185523137 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe01a7f50 con 0x7fbbe0074c70 2024-04-22T13:23:15.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.586+0000 7fbbe5729700 1 -- 172.21.15.12:0/1185523137 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbe0069730 con 0x7fbbe0074c70 2024-04-22T13:23:15.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.587+0000 7fbbc7fff700 1 -- 172.21.15.12:0/1185523137 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbbd002a070 con 0x7fbbe0074c70 2024-04-22T13:23:15.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.587+0000 7fbbc7fff700 1 --2- 172.21.15.12:0/1185523137 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fbbc80a3cd0 0x7fbbc80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:15.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.588+0000 7fbbde7fc700 1 --2- 172.21.15.12:0/1185523137 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fbbc80a3cd0 0x7fbbc80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:15.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.588+0000 7fbbc7fff700 1 -- 172.21.15.12:0/1185523137 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fbbd00d12b0 con 0x7fbbe0074c70 2024-04-22T13:23:15.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.588+0000 7fbbde7fc700 1 --2- 172.21.15.12:0/1185523137 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fbbc80a3cd0 0x7fbbc80a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbbe01025e0 tx=0x7fbbd40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:15.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.592+0000 7fbbc7fff700 1 -- 172.21.15.12:0/1185523137 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbbd0013070 con 0x7fbbe0074c70 2024-04-22T13:23:15.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.812+0000 7fbbe5729700 1 -- 172.21.15.12:0/1185523137 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fbbe002df80 con 0x7fbbe0074c70 2024-04-22T13:23:15.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.812+0000 7fbbc7fff700 1 -- 172.21.15.12:0/1185523137 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbbd009db90 con 0x7fbbe0074c70 2024-04-22T13:23:15.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.814+0000 7fbbe5729700 1 -- 172.21.15.12:0/1185523137 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fbbc80a3cd0 msgr2=0x7fbbc80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:15.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.814+0000 7fbbe5729700 1 --2- 172.21.15.12:0/1185523137 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fbbc80a3cd0 0x7fbbc80a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbbe01025e0 tx=0x7fbbd40073d0 comp rx=0 tx=0).stop 2024-04-22T13:23:15.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.814+0000 7fbbe5729700 1 -- 172.21.15.12:0/1185523137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbe0074c70 msgr2=0x7fbbe019b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:15.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.814+0000 7fbbe5729700 1 --2- 172.21.15.12:0/1185523137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbe0074c70 0x7fbbe019b3d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0000c00 tx=0x7fbbd00057c0 comp rx=0 tx=0).stop 2024-04-22T13:23:15.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.815+0000 7fbbe5729700 1 -- 172.21.15.12:0/1185523137 shutdown_connections 2024-04-22T13:23:15.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.815+0000 7fbbe5729700 1 --2- 172.21.15.12:0/1185523137 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fbbc80a3cd0 0x7fbbc80a6190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:15.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.815+0000 7fbbe5729700 1 --2- 172.21.15.12:0/1185523137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbbe010f470 0x7fbbe019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:15.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.815+0000 7fbbe5729700 1 --2- 172.21.15.12:0/1185523137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbbe0074c70 0x7fbbe019b3d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:15.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.815+0000 7fbbe5729700 1 --2- 172.21.15.12:0/1185523137 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbe0075590 0x7fbbe019b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:15.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.815+0000 7fbbe5729700 1 -- 172.21.15.12:0/1185523137 >> 172.21.15.12:0/1185523137 conn(0x7fbbe0079bf0 msgr2=0x7fbbe00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:15.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.815+0000 7fbbe5729700 1 -- 172.21.15.12:0/1185523137 shutdown_connections 2024-04-22T13:23:15.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.815+0000 7fbbe5729700 1 -- 172.21.15.12:0/1185523137 wait complete. 2024-04-22T13:23:15.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:15 smithi012 ceph-mon[28265]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-22T13:23:15.883 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234027 got 231928234031 for osd.7 2024-04-22T13:23:15.883 DEBUG:teuthology.parallel:result is None 2024-04-22T13:23:15.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:15 smithi134 ceph-mon[31338]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-22T13:23:15.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:15 smithi111 ceph-mon[31361]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-22T13:23:16.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.684+0000 7fe058655700 1 -- 172.21.15.12:0/1402460528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe05006b980 msgr2=0x7fe05010f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:16.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.684+0000 7fe058655700 1 --2- 172.21.15.12:0/1402460528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe05006b980 0x7fe05010f8f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe04800b0a0 tx=0x7fe04800b3b0 comp rx=0 tx=0).stop 2024-04-22T13:23:16.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.684+0000 7fe058655700 1 -- 172.21.15.12:0/1402460528 shutdown_connections 2024-04-22T13:23:16.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.684+0000 7fe058655700 1 --2- 172.21.15.12:0/1402460528 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe050108a20 0x7fe050108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:16.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.684+0000 7fe058655700 1 --2- 172.21.15.12:0/1402460528 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe05006b980 0x7fe05010f8f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:16.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.684+0000 7fe058655700 1 --2- 172.21.15.12:0/1402460528 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe05006afe0 0x7fe05006b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:16.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.684+0000 7fe058655700 1 -- 172.21.15.12:0/1402460528 >> 172.21.15.12:0/1402460528 conn(0x7fe0500788b0 msgr2=0x7fe050078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:16.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.684+0000 7fe058655700 1 -- 172.21.15.12:0/1402460528 shutdown_connections 2024-04-22T13:23:16.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.684+0000 7fe058655700 1 -- 172.21.15.12:0/1402460528 wait complete. 2024-04-22T13:23:16.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.685+0000 7fe058655700 1 Processor -- start 2024-04-22T13:23:16.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.685+0000 7fe058655700 1 -- start start 2024-04-22T13:23:16.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.685+0000 7fe058655700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe05006afe0 0x7fe050199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:16.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.685+0000 7fe058655700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe05006b980 0x7fe050199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:16.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.685+0000 7fe058655700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe050108a20 0x7fe05019daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:16.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.685+0000 7fe058655700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe050112cb0 con 0x7fe05006afe0 2024-04-22T13:23:16.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.685+0000 7fe058655700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe050112b30 con 0x7fe05006b980 2024-04-22T13:23:16.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.685+0000 7fe058655700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe050112e30 con 0x7fe050108a20 2024-04-22T13:23:16.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.685+0000 7fe0563f1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe05006afe0 0x7fe050199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:16.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.685+0000 7fe0563f1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe05006afe0 0x7fe050199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49328/0 (socket says 172.21.15.12:49328) 2024-04-22T13:23:16.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.685+0000 7fe0563f1700 1 -- 172.21.15.12:0/1620276038 learned_addr learned my addr 172.21.15.12:0/1620276038 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:16.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.686+0000 7fe056bf2700 1 --2- 172.21.15.12:0/1620276038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe050108a20 0x7fe05019daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:16.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.686+0000 7fe055bf0700 1 --2- 172.21.15.12:0/1620276038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe05006b980 0x7fe050199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:16.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.686+0000 7fe0563f1700 1 -- 172.21.15.12:0/1620276038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe050108a20 msgr2=0x7fe05019daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:16.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.686+0000 7fe0563f1700 1 --2- 172.21.15.12:0/1620276038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe050108a20 0x7fe05019daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:16.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.686+0000 7fe0563f1700 1 -- 172.21.15.12:0/1620276038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe05006b980 msgr2=0x7fe050199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:16.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.686+0000 7fe0563f1700 1 --2- 172.21.15.12:0/1620276038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe05006b980 0x7fe050199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:16.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.686+0000 7fe0563f1700 1 -- 172.21.15.12:0/1620276038 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe05019e230 con 0x7fe05006afe0 2024-04-22T13:23:16.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.686+0000 7fe0563f1700 1 --2- 172.21.15.12:0/1620276038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe05006afe0 0x7fe050199220 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe04c00c970 tx=0x7fe04c00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:16.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.686+0000 7fe0477fe700 1 -- 172.21.15.12:0/1620276038 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04c007aa0 con 0x7fe05006afe0 2024-04-22T13:23:16.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.686+0000 7fe0477fe700 1 -- 172.21.15.12:0/1620276038 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe04c004ce0 con 0x7fe05006afe0 2024-04-22T13:23:16.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.686+0000 7fe058655700 1 -- 172.21.15.12:0/1620276038 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe05019e520 con 0x7fe05006afe0 2024-04-22T13:23:16.712 INFO:teuthology.orchestra.run.smithi012.stdout:34359738449 2024-04-22T13:23:16.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.686+0000 7fe0477fe700 1 -- 172.21.15.12:0/1620276038 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04c0055c0 con 0x7fe05006afe0 2024-04-22T13:23:16.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.686+0000 7fe058655700 1 -- 172.21.15.12:0/1620276038 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0501a5da0 con 0x7fe05006afe0 2024-04-22T13:23:16.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.687+0000 7fe058655700 1 -- 172.21.15.12:0/1620276038 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe050069730 con 0x7fe05006afe0 2024-04-22T13:23:16.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.688+0000 7fe0477fe700 1 -- 172.21.15.12:0/1620276038 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe04c007c40 con 0x7fe05006afe0 2024-04-22T13:23:16.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.689+0000 7fe0477fe700 1 --2- 172.21.15.12:0/1620276038 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe03c0a3cd0 0x7fe03c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:16.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.689+0000 7fe055bf0700 1 --2- 172.21.15.12:0/1620276038 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe03c0a3cd0 0x7fe03c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:16.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.689+0000 7fe0477fe700 1 -- 172.21.15.12:0/1620276038 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fe04c0d0690 con 0x7fe05006afe0 2024-04-22T13:23:16.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.689+0000 7fe055bf0700 1 --2- 172.21.15.12:0/1620276038 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe03c0a3cd0 0x7fe03c0a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe040006fd0 tx=0x7fe040008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:16.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.697+0000 7fe0477fe700 1 -- 172.21.15.12:0/1620276038 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe04c09d000 con 0x7fe05006afe0 2024-04-22T13:23:16.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.914+0000 7fe058655700 1 -- 172.21.15.12:0/1620276038 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fe05019a580 con 0x7fe05006afe0 2024-04-22T13:23:16.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.915+0000 7fe0477fe700 1 -- 172.21.15.12:0/1620276038 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe04c09c900 con 0x7fe05006afe0 2024-04-22T13:23:16.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.917+0000 7fe058655700 1 -- 172.21.15.12:0/1620276038 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe03c0a3cd0 msgr2=0x7fe03c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:16.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.917+0000 7fe058655700 1 --2- 172.21.15.12:0/1620276038 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe03c0a3cd0 0x7fe03c0a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe040006fd0 tx=0x7fe040008040 comp rx=0 tx=0).stop 2024-04-22T13:23:16.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.917+0000 7fe058655700 1 -- 172.21.15.12:0/1620276038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe05006afe0 msgr2=0x7fe050199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:16.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.917+0000 7fe058655700 1 --2- 172.21.15.12:0/1620276038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe05006afe0 0x7fe050199220 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe04c00c970 tx=0x7fe04c00cd30 comp rx=0 tx=0).stop 2024-04-22T13:23:16.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.917+0000 7fe058655700 1 -- 172.21.15.12:0/1620276038 shutdown_connections 2024-04-22T13:23:16.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.917+0000 7fe058655700 1 --2- 172.21.15.12:0/1620276038 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe03c0a3cd0 0x7fe03c0a6190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:16.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.917+0000 7fe058655700 1 --2- 172.21.15.12:0/1620276038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe050108a20 0x7fe05019daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:16.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.917+0000 7fe058655700 1 --2- 172.21.15.12:0/1620276038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe05006afe0 0x7fe050199220 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:16.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.917+0000 7fe058655700 1 --2- 172.21.15.12:0/1620276038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe05006b980 0x7fe050199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:16.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.917+0000 7fe058655700 1 -- 172.21.15.12:0/1620276038 >> 172.21.15.12:0/1620276038 conn(0x7fe0500788b0 msgr2=0x7fe0500ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:16.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.917+0000 7fe058655700 1 -- 172.21.15.12:0/1620276038 shutdown_connections 2024-04-22T13:23:16.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:13.917+0000 7fe058655700 1 -- 172.21.15.12:0/1620276038 wait complete. 2024-04-22T13:23:17.268 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738444 got 34359738449 for osd.0 2024-04-22T13:23:17.268 DEBUG:teuthology.parallel:result is None 2024-04-22T13:23:17.667 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:17 smithi012 ceph-mon[28265]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-22T13:23:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.713+0000 7ff3f166e700 1 -- 172.21.15.12:0/3874090976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3ec102ac0 msgr2=0x7ff3ec102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.713+0000 7ff3f166e700 1 --2- 172.21.15.12:0/3874090976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3ec102ac0 0x7ff3ec102f40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0009990 tx=0x7ff3e0009ca0 comp rx=0 tx=0).stop 2024-04-22T13:23:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.713+0000 7ff3f166e700 1 -- 172.21.15.12:0/3874090976 shutdown_connections 2024-04-22T13:23:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.713+0000 7ff3f166e700 1 --2- 172.21.15.12:0/3874090976 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3ec103480 0x7ff3ec10f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.713+0000 7ff3f166e700 1 --2- 172.21.15.12:0/3874090976 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3ec108a70 0x7ff3ec108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.713+0000 7ff3f166e700 1 --2- 172.21.15.12:0/3874090976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3ec102ac0 0x7ff3ec102f40 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.713+0000 7ff3f166e700 1 -- 172.21.15.12:0/3874090976 >> 172.21.15.12:0/3874090976 conn(0x7ff3ec079bb0 msgr2=0x7ff3ec079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.713+0000 7ff3f166e700 1 -- 172.21.15.12:0/3874090976 shutdown_connections 2024-04-22T13:23:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.713+0000 7ff3f166e700 1 -- 172.21.15.12:0/3874090976 wait complete. 2024-04-22T13:23:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.714+0000 7ff3f166e700 1 Processor -- start 2024-04-22T13:23:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.714+0000 7ff3f166e700 1 -- start start 2024-04-22T13:23:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.714+0000 7ff3f166e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3ec102ac0 0x7ff3ec19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.714+0000 7ff3f166e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3ec103480 0x7ff3ec19b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.714+0000 7ff3f166e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3ec108a70 0x7ff3ec19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.714+0000 7ff3f166e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec112cd0 con 0x7ff3ec102ac0 2024-04-22T13:23:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.714+0000 7ff3f166e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec112b50 con 0x7ff3ec108a70 2024-04-22T13:23:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.714+0000 7ff3f166e700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec112e50 con 0x7ff3ec103480 2024-04-22T13:23:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.714+0000 7ff3eaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3ec102ac0 0x7ff3ec19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.714+0000 7ff3eaffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3ec102ac0 0x7ff3ec19b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49356/0 (socket says 172.21.15.12:49356) 2024-04-22T13:23:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.714+0000 7ff3eaffd700 1 -- 172.21.15.12:0/1716421211 learned_addr learned my addr 172.21.15.12:0/1716421211 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.715+0000 7ff3eb7fe700 1 --2- 172.21.15.12:0/1716421211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3ec108a70 0x7ff3ec19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.715+0000 7ff3ea7fc700 1 --2- 172.21.15.12:0/1716421211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3ec103480 0x7ff3ec19b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.715+0000 7ff3eaffd700 1 -- 172.21.15.12:0/1716421211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3ec103480 msgr2=0x7ff3ec19b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.715+0000 7ff3eaffd700 1 --2- 172.21.15.12:0/1716421211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3ec103480 0x7ff3ec19b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.715+0000 7ff3eaffd700 1 -- 172.21.15.12:0/1716421211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3ec108a70 msgr2=0x7ff3ec19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:17.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.715+0000 7ff3eaffd700 1 --2- 172.21.15.12:0/1716421211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3ec108a70 0x7ff3ec19fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:17.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.715+0000 7ff3eaffd700 1 -- 172.21.15.12:0/1716421211 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3ec1a03d0 con 0x7ff3ec102ac0 2024-04-22T13:23:17.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.715+0000 7ff3ea7fc700 1 --2- 172.21.15.12:0/1716421211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3ec103480 0x7ff3ec19b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:23:17.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.715+0000 7ff3eaffd700 1 --2- 172.21.15.12:0/1716421211 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3ec102ac0 0x7ff3ec19b3c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc00b530 tx=0x7ff3dc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:17.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.715+0000 7ff3d3fff700 1 -- 172.21.15.12:0/1716421211 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc015020 con 0x7ff3ec102ac0 2024-04-22T13:23:17.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.715+0000 7ff3d3fff700 1 -- 172.21.15.12:0/1716421211 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3dc011ce0 con 0x7ff3ec102ac0 2024-04-22T13:23:17.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.715+0000 7ff3f166e700 1 -- 172.21.15.12:0/1716421211 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3ec1a06c0 con 0x7ff3ec102ac0 2024-04-22T13:23:17.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.715+0000 7ff3d3fff700 1 -- 172.21.15.12:0/1716421211 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc010e40 con 0x7ff3ec102ac0 2024-04-22T13:23:17.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.715+0000 7ff3f166e700 1 -- 172.21.15.12:0/1716421211 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3ec1a7f40 con 0x7ff3ec102ac0 2024-04-22T13:23:17.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.716+0000 7ff3f166e700 1 -- 172.21.15.12:0/1716421211 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3ec069730 con 0x7ff3ec102ac0 2024-04-22T13:23:17.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.717+0000 7ff3d3fff700 1 -- 172.21.15.12:0/1716421211 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff3dc018020 con 0x7ff3ec102ac0 2024-04-22T13:23:17.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.718+0000 7ff3d3fff700 1 --2- 172.21.15.12:0/1716421211 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ff3d40a3cd0 0x7ff3d40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:17.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.718+0000 7ff3ea7fc700 1 --2- 172.21.15.12:0/1716421211 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ff3d40a3cd0 0x7ff3d40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:17.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.718+0000 7ff3d3fff700 1 -- 172.21.15.12:0/1716421211 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7ff3dc0cfce0 con 0x7ff3ec102ac0 2024-04-22T13:23:17.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.718+0000 7ff3ea7fc700 1 --2- 172.21.15.12:0/1716421211 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ff3d40a3cd0 0x7ff3d40a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff3e000b5c0 tx=0x7ff3e0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:17.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.723+0000 7ff3d3fff700 1 -- 172.21.15.12:0/1716421211 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff3dc01f020 con 0x7ff3ec102ac0 2024-04-22T13:23:17.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.925+0000 7fab604b0700 1 -- 172.21.15.12:0/1981521812 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab5806a750 msgr2=0x7fab5806abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:17.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.925+0000 7fab604b0700 1 --2- 172.21.15.12:0/1981521812 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab5806a750 0x7fab5806abb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fab48009990 tx=0x7fab48009ca0 comp rx=0 tx=0).stop 2024-04-22T13:23:17.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.925+0000 7fab604b0700 1 -- 172.21.15.12:0/1981521812 shutdown_connections 2024-04-22T13:23:17.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.925+0000 7fab604b0700 1 --2- 172.21.15.12:0/1981521812 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab5806b0f0 0x7fab5810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:17.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.925+0000 7fab604b0700 1 --2- 172.21.15.12:0/1981521812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab58108a30 0x7fab58108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:17.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.925+0000 7fab604b0700 1 --2- 172.21.15.12:0/1981521812 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab5806a750 0x7fab5806abb0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:17.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.925+0000 7fab604b0700 1 -- 172.21.15.12:0/1981521812 >> 172.21.15.12:0/1981521812 conn(0x7fab580788f0 msgr2=0x7fab58078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:17.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.925+0000 7fab604b0700 1 -- 172.21.15.12:0/1981521812 shutdown_connections 2024-04-22T13:23:17.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.925+0000 7fab604b0700 1 -- 172.21.15.12:0/1981521812 wait complete. 2024-04-22T13:23:17.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.926+0000 7fab604b0700 1 Processor -- start 2024-04-22T13:23:17.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.926+0000 7fab604b0700 1 -- start start 2024-04-22T13:23:17.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.926+0000 7fab604b0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab5806a750 0x7fab5819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:17.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.926+0000 7fab604b0700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab5806b0f0 0x7fab5819b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:17.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.926+0000 7fab604b0700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab58108a30 0x7fab5819fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:17.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.926+0000 7fab604b0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fab58112bd0 con 0x7fab5806a750 2024-04-22T13:23:17.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.926+0000 7fab604b0700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fab58112a50 con 0x7fab5806b0f0 2024-04-22T13:23:17.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.926+0000 7fab604b0700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fab58112d50 con 0x7fab58108a30 2024-04-22T13:23:17.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.926+0000 7fab5e24c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab5806a750 0x7fab5819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:17.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.926+0000 7fab5e24c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab5806a750 0x7fab5819b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49372/0 (socket says 172.21.15.12:49372) 2024-04-22T13:23:17.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.926+0000 7fab5e24c700 1 -- 172.21.15.12:0/1988419743 learned_addr learned my addr 172.21.15.12:0/1988419743 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:17.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.927+0000 7fab5da4b700 1 --2- 172.21.15.12:0/1988419743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab5806b0f0 0x7fab5819b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:17.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.927+0000 7fab5ea4d700 1 --2- 172.21.15.12:0/1988419743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab58108a30 0x7fab5819fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:17.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.927+0000 7fab5e24c700 1 -- 172.21.15.12:0/1988419743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab58108a30 msgr2=0x7fab5819fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:17.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.927+0000 7fab5e24c700 1 --2- 172.21.15.12:0/1988419743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab58108a30 0x7fab5819fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:17.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.927+0000 7fab5e24c700 1 -- 172.21.15.12:0/1988419743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab5806b0f0 msgr2=0x7fab5819b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:17.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.927+0000 7fab5e24c700 1 --2- 172.21.15.12:0/1988419743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab5806b0f0 0x7fab5819b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:17.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.927+0000 7fab5e24c700 1 -- 172.21.15.12:0/1988419743 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab581a0370 con 0x7fab5806a750 2024-04-22T13:23:17.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.927+0000 7fab5ea4d700 1 --2- 172.21.15.12:0/1988419743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab58108a30 0x7fab5819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:23:17.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.927+0000 7fab5e24c700 1 --2- 172.21.15.12:0/1988419743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab5806a750 0x7fab5819b360 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fab5400d980 tx=0x7fab5400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:17.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.927+0000 7fab4f7fe700 1 -- 172.21.15.12:0/1988419743 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54017070 con 0x7fab5806a750 2024-04-22T13:23:17.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.927+0000 7fab4f7fe700 1 -- 172.21.15.12:0/1988419743 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab54005020 con 0x7fab5806a750 2024-04-22T13:23:17.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.927+0000 7fab604b0700 1 -- 172.21.15.12:0/1988419743 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab581a0660 con 0x7fab5806a750 2024-04-22T13:23:17.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.927+0000 7fab4f7fe700 1 -- 172.21.15.12:0/1988419743 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54009b50 con 0x7fab5806a750 2024-04-22T13:23:17.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.927+0000 7fab604b0700 1 -- 172.21.15.12:0/1988419743 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab581a7ee0 con 0x7fab5806a750 2024-04-22T13:23:17.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.928+0000 7fab604b0700 1 -- 172.21.15.12:0/1988419743 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab58104170 con 0x7fab5806a750 2024-04-22T13:23:17.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.930+0000 7fab4f7fe700 1 -- 172.21.15.12:0/1988419743 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fab540051d0 con 0x7fab5806a750 2024-04-22T13:23:17.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.931+0000 7fab4f7fe700 1 --2- 172.21.15.12:0/1988419743 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fab440a3cd0 0x7fab440a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:17.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.931+0000 7fab5da4b700 1 --2- 172.21.15.12:0/1988419743 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fab440a3cd0 0x7fab440a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:17.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.931+0000 7fab4f7fe700 1 -- 172.21.15.12:0/1988419743 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fab540d07c0 con 0x7fab5806a750 2024-04-22T13:23:17.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:17 smithi134 ceph-mon[31338]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-22T13:23:17.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.931+0000 7fab5da4b700 1 --2- 172.21.15.12:0/1988419743 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fab440a3cd0 0x7fab440a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fab4800b5c0 tx=0x7fab48005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:17.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.934+0000 7fab4f7fe700 1 -- 172.21.15.12:0/1988419743 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fab54020020 con 0x7fab5806a750 2024-04-22T13:23:17.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.937+0000 7ff3f166e700 1 -- 172.21.15.12:0/1716421211 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7ff3ec1a8110 con 0x7ff3ec102ac0 2024-04-22T13:23:17.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.938+0000 7ff3d3fff700 1 -- 172.21.15.12:0/1716421211 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff3dc09c6a0 con 0x7ff3ec102ac0 2024-04-22T13:23:17.934 INFO:teuthology.orchestra.run.smithi012.stdout:77309411401 2024-04-22T13:23:17.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.940+0000 7ff3f166e700 1 -- 172.21.15.12:0/1716421211 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ff3d40a3cd0 msgr2=0x7ff3d40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:17.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.940+0000 7ff3f166e700 1 --2- 172.21.15.12:0/1716421211 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ff3d40a3cd0 0x7ff3d40a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff3e000b5c0 tx=0x7ff3e0005fb0 comp rx=0 tx=0).stop 2024-04-22T13:23:17.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.940+0000 7ff3f166e700 1 -- 172.21.15.12:0/1716421211 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3ec102ac0 msgr2=0x7ff3ec19b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:17.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.940+0000 7ff3f166e700 1 --2- 172.21.15.12:0/1716421211 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3ec102ac0 0x7ff3ec19b3c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc00b530 tx=0x7ff3dc00b840 comp rx=0 tx=0).stop 2024-04-22T13:23:17.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.940+0000 7ff3f166e700 1 -- 172.21.15.12:0/1716421211 shutdown_connections 2024-04-22T13:23:17.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.940+0000 7ff3f166e700 1 --2- 172.21.15.12:0/1716421211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3ec108a70 0x7ff3ec19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:17.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.940+0000 7ff3f166e700 1 --2- 172.21.15.12:0/1716421211 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ff3d40a3cd0 0x7ff3d40a6190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:17.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.940+0000 7ff3f166e700 1 --2- 172.21.15.12:0/1716421211 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3ec102ac0 0x7ff3ec19b3c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:17.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.940+0000 7ff3f166e700 1 --2- 172.21.15.12:0/1716421211 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3ec103480 0x7ff3ec19b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:17.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.940+0000 7ff3f166e700 1 -- 172.21.15.12:0/1716421211 >> 172.21.15.12:0/1716421211 conn(0x7ff3ec079bb0 msgr2=0x7ff3ec101550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:17.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.940+0000 7ff3f166e700 1 -- 172.21.15.12:0/1716421211 shutdown_connections 2024-04-22T13:23:17.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:17.940+0000 7ff3f166e700 1 -- 172.21.15.12:0/1716421211 wait complete. 2024-04-22T13:23:17.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:17 smithi111 ceph-mon[31361]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-22T13:23:18.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:18.146+0000 7fab604b0700 1 -- 172.21.15.12:0/1988419743 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fab5802df80 con 0x7fab5806a750 2024-04-22T13:23:18.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:18.147+0000 7fab4f7fe700 1 -- 172.21.15.12:0/1988419743 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fab5409d010 con 0x7fab5806a750 2024-04-22T13:23:18.143 INFO:teuthology.orchestra.run.smithi012.stdout:206158430261 2024-04-22T13:23:18.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:18.149+0000 7fab604b0700 1 -- 172.21.15.12:0/1988419743 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fab440a3cd0 msgr2=0x7fab440a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:18.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:18.149+0000 7fab604b0700 1 --2- 172.21.15.12:0/1988419743 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fab440a3cd0 0x7fab440a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fab4800b5c0 tx=0x7fab48005960 comp rx=0 tx=0).stop 2024-04-22T13:23:18.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:18.149+0000 7fab604b0700 1 -- 172.21.15.12:0/1988419743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab5806a750 msgr2=0x7fab5819b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:18.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:18.149+0000 7fab604b0700 1 --2- 172.21.15.12:0/1988419743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab5806a750 0x7fab5819b360 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fab5400d980 tx=0x7fab5400dc90 comp rx=0 tx=0).stop 2024-04-22T13:23:18.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:18.149+0000 7fab604b0700 1 -- 172.21.15.12:0/1988419743 shutdown_connections 2024-04-22T13:23:18.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:18.149+0000 7fab604b0700 1 --2- 172.21.15.12:0/1988419743 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fab440a3cd0 0x7fab440a6190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:18.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:18.149+0000 7fab604b0700 1 --2- 172.21.15.12:0/1988419743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab58108a30 0x7fab5819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:18.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:18.149+0000 7fab604b0700 1 --2- 172.21.15.12:0/1988419743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab5806a750 0x7fab5819b360 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:18.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:18.149+0000 7fab604b0700 1 --2- 172.21.15.12:0/1988419743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab5806b0f0 0x7fab5819b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:18.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:18.149+0000 7fab604b0700 1 -- 172.21.15.12:0/1988419743 >> 172.21.15.12:0/1988419743 conn(0x7fab580788f0 msgr2=0x7fab5810df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:18.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:18.149+0000 7fab604b0700 1 -- 172.21.15.12:0/1988419743 shutdown_connections 2024-04-22T13:23:18.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:18.149+0000 7fab604b0700 1 -- 172.21.15.12:0/1988419743 wait complete. 2024-04-22T13:23:18.732 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:18 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1716421211' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-22T13:23:18.732 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:18 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1988419743' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-22T13:23:18.763 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430256 got 206158430261 for osd.6 2024-04-22T13:23:18.763 DEBUG:teuthology.parallel:result is None 2024-04-22T13:23:18.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:18 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1716421211' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-22T13:23:18.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:18 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1988419743' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-22T13:23:18.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:18 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1716421211' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-22T13:23:18.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:18 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1988419743' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-22T13:23:19.365 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005091 got 261993005097 for osd.8 2024-04-22T13:23:19.366 DEBUG:teuthology.parallel:result is None 2024-04-22T13:23:19.848 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:19 smithi012 ceph-mon[28265]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-22T13:23:19.849 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411395 got 77309411401 for osd.2 2024-04-22T13:23:19.849 DEBUG:teuthology.parallel:result is None 2024-04-22T13:23:19.849 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-22T13:23:19.849 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-22T13:23:19.878 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:19 smithi111 ceph-mon[31361]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-22T13:23:19.919 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:19 smithi134 ceph-mon[31338]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-22T13:23:20.621 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:23:21.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:21 smithi012 ceph-mon[28265]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-22T13:23:21.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:21 smithi134 ceph-mon[31338]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-22T13:23:21.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:21 smithi111 ceph-mon[31361]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-22T13:23:22.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.360+0000 7fc2ce0ab700 1 -- 172.21.15.12:0/3554074617 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2c8108a70 msgr2=0x7fc2c8108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:22.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.360+0000 7fc2ce0ab700 1 --2- 172.21.15.12:0/3554074617 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2c8108a70 0x7fc2c8108e50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8009990 tx=0x7fc2b8009ca0 comp rx=0 tx=0).stop 2024-04-22T13:23:22.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.360+0000 7fc2ce0ab700 1 -- 172.21.15.12:0/3554074617 shutdown_connections 2024-04-22T13:23:22.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.360+0000 7fc2ce0ab700 1 --2- 172.21.15.12:0/3554074617 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2c8103480 0x7fc2c810f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:22.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.361+0000 7fc2ce0ab700 1 --2- 172.21.15.12:0/3554074617 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2c8108a70 0x7fc2c8108e50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:22.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.361+0000 7fc2ce0ab700 1 --2- 172.21.15.12:0/3554074617 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2c8102ac0 0x7fc2c8102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:22.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.361+0000 7fc2ce0ab700 1 -- 172.21.15.12:0/3554074617 >> 172.21.15.12:0/3554074617 conn(0x7fc2c8079bb0 msgr2=0x7fc2c8079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:22.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.361+0000 7fc2ce0ab700 1 -- 172.21.15.12:0/3554074617 shutdown_connections 2024-04-22T13:23:22.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.361+0000 7fc2ce0ab700 1 -- 172.21.15.12:0/3554074617 wait complete. 2024-04-22T13:23:22.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.361+0000 7fc2ce0ab700 1 Processor -- start 2024-04-22T13:23:22.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.361+0000 7fc2ce0ab700 1 -- start start 2024-04-22T13:23:22.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.362+0000 7fc2ce0ab700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2c8102ac0 0x7fc2c8199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:22.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.362+0000 7fc2ce0ab700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2c8103480 0x7fc2c8199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:22.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.362+0000 7fc2ce0ab700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2c8108a70 0x7fc2c819d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:22.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.362+0000 7fc2ce0ab700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c8112cd0 con 0x7fc2c8103480 2024-04-22T13:23:22.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.362+0000 7fc2ce0ab700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c8112b50 con 0x7fc2c8108a70 2024-04-22T13:23:22.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.362+0000 7fc2ce0ab700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c8112e50 con 0x7fc2c8102ac0 2024-04-22T13:23:22.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.362+0000 7fc2c6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2c8103480 0x7fc2c8199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:22.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.362+0000 7fc2c6ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2c8103480 0x7fc2c8199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47728/0 (socket says 172.21.15.12:47728) 2024-04-22T13:23:22.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.362+0000 7fc2c6ffd700 1 -- 172.21.15.12:0/1228926882 learned_addr learned my addr 172.21.15.12:0/1228926882 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:22.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.362+0000 7fc2c77fe700 1 --2- 172.21.15.12:0/1228926882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2c8102ac0 0x7fc2c8199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:22.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.362+0000 7fc2c7fff700 1 --2- 172.21.15.12:0/1228926882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2c8108a70 0x7fc2c819d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:22.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.362+0000 7fc2c6ffd700 1 -- 172.21.15.12:0/1228926882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2c8102ac0 msgr2=0x7fc2c8199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:22.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.362+0000 7fc2c6ffd700 1 --2- 172.21.15.12:0/1228926882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2c8102ac0 0x7fc2c8199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:22.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.362+0000 7fc2c6ffd700 1 -- 172.21.15.12:0/1228926882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2c8108a70 msgr2=0x7fc2c819d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:22.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.362+0000 7fc2c6ffd700 1 --2- 172.21.15.12:0/1228926882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2c8108a70 0x7fc2c819d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:22.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.362+0000 7fc2c6ffd700 1 -- 172.21.15.12:0/1228926882 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c819e110 con 0x7fc2c8103480 2024-04-22T13:23:22.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.362+0000 7fc2c6ffd700 1 --2- 172.21.15.12:0/1228926882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2c8103480 0x7fc2c8199640 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00c9b0 tx=0x7fc2bc00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:22.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.362+0000 7fc2c4ff9700 1 -- 172.21.15.12:0/1228926882 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc007b00 con 0x7fc2c8103480 2024-04-22T13:23:22.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.363+0000 7fc2ce0ab700 1 -- 172.21.15.12:0/1228926882 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2c819e400 con 0x7fc2c8103480 2024-04-22T13:23:22.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.363+0000 7fc2c4ff9700 1 -- 172.21.15.12:0/1228926882 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2bc007ca0 con 0x7fc2c8103480 2024-04-22T13:23:22.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.363+0000 7fc2c4ff9700 1 -- 172.21.15.12:0/1228926882 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc018730 con 0x7fc2c8103480 2024-04-22T13:23:22.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.363+0000 7fc2ce0ab700 1 -- 172.21.15.12:0/1228926882 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2c81a5c80 con 0x7fc2c8103480 2024-04-22T13:23:22.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.364+0000 7fc2ce0ab700 1 -- 172.21.15.12:0/1228926882 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2c8069730 con 0x7fc2c8103480 2024-04-22T13:23:22.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.366+0000 7fc2c4ff9700 1 -- 172.21.15.12:0/1228926882 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc2bc00f450 con 0x7fc2c8103480 2024-04-22T13:23:22.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.366+0000 7fc2c4ff9700 1 --2- 172.21.15.12:0/1228926882 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc2b00a3cd0 0x7fc2b00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:22.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.366+0000 7fc2c77fe700 1 --2- 172.21.15.12:0/1228926882 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc2b00a3cd0 0x7fc2b00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:22.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.366+0000 7fc2c4ff9700 1 -- 172.21.15.12:0/1228926882 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fc2bc0d0910 con 0x7fc2c8103480 2024-04-22T13:23:22.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.367+0000 7fc2c77fe700 1 --2- 172.21.15.12:0/1228926882 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc2b00a3cd0 0x7fc2b00a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8009750 tx=0x7fc2b801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:22.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.370+0000 7fc2c4ff9700 1 -- 172.21.15.12:0/1228926882 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc2bc014030 con 0x7fc2c8103480 2024-04-22T13:23:22.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.595+0000 7fc2ce0ab700 1 -- 172.21.15.12:0/1228926882 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc2c8062e80 con 0x7fc2b00a3cd0 2024-04-22T13:23:22.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.603+0000 7fc2c4ff9700 1 -- 172.21.15.12:0/1228926882 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258759 (secure 0 0 0) 0x7fc2c8062e80 con 0x7fc2b00a3cd0 2024-04-22T13:23:22.607 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:23:22.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.606+0000 7fc2ce0ab700 1 -- 172.21.15.12:0/1228926882 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc2b00a3cd0 msgr2=0x7fc2b00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:22.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.606+0000 7fc2ce0ab700 1 --2- 172.21.15.12:0/1228926882 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc2b00a3cd0 0x7fc2b00a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8009750 tx=0x7fc2b801a040 comp rx=0 tx=0).stop 2024-04-22T13:23:22.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.606+0000 7fc2ce0ab700 1 -- 172.21.15.12:0/1228926882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2c8103480 msgr2=0x7fc2c8199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:22.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.606+0000 7fc2ce0ab700 1 --2- 172.21.15.12:0/1228926882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2c8103480 0x7fc2c8199640 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00c9b0 tx=0x7fc2bc00cd70 comp rx=0 tx=0).stop 2024-04-22T13:23:22.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.606+0000 7fc2ce0ab700 1 -- 172.21.15.12:0/1228926882 shutdown_connections 2024-04-22T13:23:22.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.606+0000 7fc2ce0ab700 1 --2- 172.21.15.12:0/1228926882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2c8108a70 0x7fc2c819d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:22.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.606+0000 7fc2ce0ab700 1 --2- 172.21.15.12:0/1228926882 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc2b00a3cd0 0x7fc2b00a6190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:22.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.606+0000 7fc2ce0ab700 1 --2- 172.21.15.12:0/1228926882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2c8102ac0 0x7fc2c8199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:22.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.606+0000 7fc2ce0ab700 1 --2- 172.21.15.12:0/1228926882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2c8103480 0x7fc2c8199640 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:22.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.606+0000 7fc2ce0ab700 1 -- 172.21.15.12:0/1228926882 >> 172.21.15.12:0/1228926882 conn(0x7fc2c8079bb0 msgr2=0x7fc2c81017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:22.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.606+0000 7fc2ce0ab700 1 -- 172.21.15.12:0/1228926882 shutdown_connections 2024-04-22T13:23:22.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:22.606+0000 7fc2ce0ab700 1 -- 172.21.15.12:0/1228926882 wait complete. 2024-04-22T13:23:22.612 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-22T13:23:23.296 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-04-22T13:23:22.380891+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656880,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":860,"num_read_kb":551,"num_write":418,"num_write_kb":3071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":343,"ondisk_log_size":343,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375212,"kb_used_data":13500,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566612,"statfs":{"total":1151940427776,"available":1151556210688,"internally_reserved":0,"allocated":13824000,"data_stored":7514280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003087"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.506048+0000","last_change":"2024-04-22T13:22:05.254388+0000","last_active":"2024-04-22T13:23:05.506048+0000","last_peered":"2024-04-22T13:23:05.506048+0000","last_clean":"2024-04-22T13:23:05.506048+0000","last_became_active":"2024-04-22T13:22:05.254042+0000","last_became_peered":"2024-04-22T13:22:05.254042+0000","last_unstale":"2024-04-22T13:23:05.506048+0000","last_undegraded":"2024-04-22T13:23:05.506048+0000","last_fullsized":"2024-04-22T13:23:05.506048+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:07:56.139653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508850+0000","last_change":"2024-04-22T13:22:05.247879+0000","last_active":"2024-04-22T13:23:05.508850+0000","last_peered":"2024-04-22T13:23:05.508850+0000","last_clean":"2024-04-22T13:23:05.508850+0000","last_became_active":"2024-04-22T13:22:05.247508+0000","last_became_peered":"2024-04-22T13:22:05.247508+0000","last_unstale":"2024-04-22T13:23:05.508850+0000","last_undegraded":"2024-04-22T13:23:05.508850+0000","last_fullsized":"2024-04-22T13:23:05.508850+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:55:51.928658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508776+0000","last_change":"2024-04-22T13:22:05.257970+0000","last_active":"2024-04-22T13:23:05.508776+0000","last_peered":"2024-04-22T13:23:05.508776+0000","last_clean":"2024-04-22T13:23:05.508776+0000","last_became_active":"2024-04-22T13:22:05.257652+0000","last_became_peered":"2024-04-22T13:22:05.257652+0000","last_unstale":"2024-04-22T13:23:05.508776+0000","last_undegraded":"2024-04-22T13:23:05.508776+0000","last_fullsized":"2024-04-22T13:23:05.508776+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:42:17.476349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.990505+0000","last_change":"2024-04-22T13:22:05.249186+0000","last_active":"2024-04-22T13:23:05.990505+0000","last_peered":"2024-04-22T13:23:05.990505+0000","last_clean":"2024-04-22T13:23:05.990505+0000","last_became_active":"2024-04-22T13:22:05.248814+0000","last_became_peered":"2024-04-22T13:22:05.248814+0000","last_unstale":"2024-04-22T13:23:05.990505+0000","last_undegraded":"2024-04-22T13:23:05.990505+0000","last_fullsized":"2024-04-22T13:23:05.990505+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:57.450794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508196+0000","last_change":"2024-04-22T13:22:05.251953+0000","last_active":"2024-04-22T13:23:05.508196+0000","last_peered":"2024-04-22T13:23:05.508196+0000","last_clean":"2024-04-22T13:23:05.508196+0000","last_became_active":"2024-04-22T13:22:05.251547+0000","last_became_peered":"2024-04-22T13:22:05.251547+0000","last_unstale":"2024-04-22T13:23:05.508196+0000","last_undegraded":"2024-04-22T13:23:05.508196+0000","last_fullsized":"2024-04-22T13:23:05.508196+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:36:34.809878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.510044+0000","last_change":"2024-04-22T13:23:05.510044+0000","last_active":"2024-04-22T13:23:05.510044+0000","last_peered":"2024-04-22T13:23:05.510044+0000","last_clean":"2024-04-22T13:23:05.510044+0000","last_became_active":"2024-04-22T13:23:05.509677+0000","last_became_peered":"2024-04-22T13:23:05.509677+0000","last_unstale":"2024-04-22T13:23:05.510044+0000","last_undegraded":"2024-04-22T13:23:05.510044+0000","last_fullsized":"2024-04-22T13:23:05.510044+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:46.979291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508380+0000","last_change":"2024-04-22T13:22:05.250418+0000","last_active":"2024-04-22T13:23:05.508380+0000","last_peered":"2024-04-22T13:23:05.508380+0000","last_clean":"2024-04-22T13:23:05.508380+0000","last_became_active":"2024-04-22T13:22:05.250097+0000","last_became_peered":"2024-04-22T13:22:05.250097+0000","last_unstale":"2024-04-22T13:23:05.508380+0000","last_undegraded":"2024-04-22T13:23:05.508380+0000","last_fullsized":"2024-04-22T13:23:05.508380+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:34:59.598950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.990304+0000","last_change":"2024-04-22T13:22:05.244762+0000","last_active":"2024-04-22T13:23:05.990304+0000","last_peered":"2024-04-22T13:23:05.990304+0000","last_clean":"2024-04-22T13:23:05.990304+0000","last_became_active":"2024-04-22T13:22:05.244430+0000","last_became_peered":"2024-04-22T13:22:05.244430+0000","last_unstale":"2024-04-22T13:23:05.990304+0000","last_undegraded":"2024-04-22T13:23:05.990304+0000","last_fullsized":"2024-04-22T13:23:05.990304+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:33:01.582851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.506546+0000","last_change":"2024-04-22T13:22:05.257596+0000","last_active":"2024-04-22T13:23:05.506546+0000","last_peered":"2024-04-22T13:23:05.506546+0000","last_clean":"2024-04-22T13:23:05.506546+0000","last_became_active":"2024-04-22T13:22:05.257277+0000","last_became_peered":"2024-04-22T13:22:05.257277+0000","last_unstale":"2024-04-22T13:23:05.506546+0000","last_undegraded":"2024-04-22T13:23:05.506546+0000","last_fullsized":"2024-04-22T13:23:05.506546+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:09:44.715965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.507131+0000","last_change":"2024-04-22T13:22:05.252177+0000","last_active":"2024-04-22T13:23:05.507131+0000","last_peered":"2024-04-22T13:23:05.507131+0000","last_clean":"2024-04-22T13:23:05.507131+0000","last_became_active":"2024-04-22T13:22:05.251794+0000","last_became_peered":"2024-04-22T13:22:05.251794+0000","last_unstale":"2024-04-22T13:23:05.507131+0000","last_undegraded":"2024-04-22T13:23:05.507131+0000","last_fullsized":"2024-04-22T13:23:05.507131+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:08:59.500540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508555+0000","last_change":"2024-04-22T13:22:05.256209+0000","last_active":"2024-04-22T13:23:05.508555+0000","last_peered":"2024-04-22T13:23:05.508555+0000","last_clean":"2024-04-22T13:23:05.508555+0000","last_became_active":"2024-04-22T13:22:05.255504+0000","last_became_peered":"2024-04-22T13:22:05.255504+0000","last_unstale":"2024-04-22T13:23:05.508555+0000","last_undegraded":"2024-04-22T13:23:05.508555+0000","last_fullsized":"2024-04-22T13:23:05.508555+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:32:08.190921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.509837+0000","last_change":"2024-04-22T13:23:05.509837+0000","last_active":"2024-04-22T13:23:05.509837+0000","last_peered":"2024-04-22T13:23:05.509837+0000","last_clean":"2024-04-22T13:23:05.509837+0000","last_became_active":"2024-04-22T13:23:05.509421+0000","last_became_peered":"2024-04-22T13:23:05.509421+0000","last_unstale":"2024-04-22T13:23:05.509837+0000","last_undegraded":"2024-04-22T13:23:05.509837+0000","last_fullsized":"2024-04-22T13:23:05.509837+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:30:01.103373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.863247+0000","last_change":"2024-04-22T13:22:05.246667+0000","last_active":"2024-04-22T13:23:05.863247+0000","last_peered":"2024-04-22T13:23:05.863247+0000","last_clean":"2024-04-22T13:23:05.863247+0000","last_became_active":"2024-04-22T13:22:05.246365+0000","last_became_peered":"2024-04-22T13:22:05.246365+0000","last_unstale":"2024-04-22T13:23:05.863247+0000","last_undegraded":"2024-04-22T13:23:05.863247+0000","last_fullsized":"2024-04-22T13:23:05.863247+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:41:55.533430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508257+0000","last_change":"2024-04-22T13:22:05.251982+0000","last_active":"2024-04-22T13:23:05.508257+0000","last_peered":"2024-04-22T13:23:05.508257+0000","last_clean":"2024-04-22T13:23:05.508257+0000","last_became_active":"2024-04-22T13:22:05.251575+0000","last_became_peered":"2024-04-22T13:22:05.251575+0000","last_unstale":"2024-04-22T13:23:05.508257+0000","last_undegraded":"2024-04-22T13:23:05.508257+0000","last_fullsized":"2024-04-22T13:23:05.508257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:28:05.343140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508282+0000","last_change":"2024-04-22T13:22:05.251056+0000","last_active":"2024-04-22T13:23:05.508282+0000","last_peered":"2024-04-22T13:23:05.508282+0000","last_clean":"2024-04-22T13:23:05.508282+0000","last_became_active":"2024-04-22T13:22:05.250652+0000","last_became_peered":"2024-04-22T13:22:05.250652+0000","last_unstale":"2024-04-22T13:23:05.508282+0000","last_undegraded":"2024-04-22T13:23:05.508282+0000","last_fullsized":"2024-04-22T13:23:05.508282+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:08:28.629965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.505896+0000","last_change":"2024-04-22T13:22:05.256877+0000","last_active":"2024-04-22T13:23:05.505896+0000","last_peered":"2024-04-22T13:23:05.505896+0000","last_clean":"2024-04-22T13:23:05.505896+0000","last_became_active":"2024-04-22T13:22:05.256074+0000","last_became_peered":"2024-04-22T13:22:05.256074+0000","last_unstale":"2024-04-22T13:23:05.505896+0000","last_undegraded":"2024-04-22T13:23:05.505896+0000","last_fullsized":"2024-04-22T13:23:05.505896+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:08:53.607626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.506174+0000","last_change":"2024-04-22T13:22:05.255598+0000","last_active":"2024-04-22T13:23:05.506174+0000","last_peered":"2024-04-22T13:23:05.506174+0000","last_clean":"2024-04-22T13:23:05.506174+0000","last_became_active":"2024-04-22T13:22:05.255239+0000","last_became_peered":"2024-04-22T13:22:05.255239+0000","last_unstale":"2024-04-22T13:23:05.506174+0000","last_undegraded":"2024-04-22T13:23:05.506174+0000","last_fullsized":"2024-04-22T13:23:05.506174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:38:20.547882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.505358+0000","last_change":"2024-04-22T13:22:05.254232+0000","last_active":"2024-04-22T13:23:05.505358+0000","last_peered":"2024-04-22T13:23:05.505358+0000","last_clean":"2024-04-22T13:23:05.505358+0000","last_became_active":"2024-04-22T13:22:05.253907+0000","last_became_peered":"2024-04-22T13:22:05.253907+0000","last_unstale":"2024-04-22T13:23:05.505358+0000","last_undegraded":"2024-04-22T13:23:05.505358+0000","last_fullsized":"2024-04-22T13:23:05.505358+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:15:49.931112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.504381+0000","last_change":"2024-04-22T13:22:05.251518+0000","last_active":"2024-04-22T13:23:05.504381+0000","last_peered":"2024-04-22T13:23:05.504381+0000","last_clean":"2024-04-22T13:23:05.504381+0000","last_became_active":"2024-04-22T13:22:05.250478+0000","last_became_peered":"2024-04-22T13:22:05.250478+0000","last_unstale":"2024-04-22T13:23:05.504381+0000","last_undegraded":"2024-04-22T13:23:05.504381+0000","last_fullsized":"2024-04-22T13:23:05.504381+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:25:26.758692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":101,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:18.496965+0000","last_change":"2024-04-22T13:22:04.241365+0000","last_active":"2024-04-22T13:23:18.496965+0000","last_peered":"2024-04-22T13:23:18.496965+0000","last_clean":"2024-04-22T13:23:18.496965+0000","last_became_active":"2024-04-22T13:22:04.240822+0000","last_became_peered":"2024-04-22T13:22:04.240822+0000","last_unstale":"2024-04-22T13:23:18.496965+0000","last_undegraded":"2024-04-22T13:23:18.496965+0000","last_fullsized":"2024-04-22T13:23:18.496965+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:20:23.928534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.876262+0000","last_change":"2024-04-22T13:23:05.876262+0000","last_active":"2024-04-22T13:23:05.876262+0000","last_peered":"2024-04-22T13:23:05.876262+0000","last_clean":"2024-04-22T13:23:05.876262+0000","last_became_active":"2024-04-22T13:23:05.875872+0000","last_became_peered":"2024-04-22T13:23:05.875872+0000","last_unstale":"2024-04-22T13:23:05.876262+0000","last_undegraded":"2024-04-22T13:23:05.876262+0000","last_fullsized":"2024-04-22T13:23:05.876262+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:29:33.092887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.990905+0000","last_change":"2024-04-22T13:22:04.248872+0000","last_active":"2024-04-22T13:23:05.990905+0000","last_peered":"2024-04-22T13:23:05.990905+0000","last_clean":"2024-04-22T13:23:05.990905+0000","last_became_active":"2024-04-22T13:22:04.248466+0000","last_became_peered":"2024-04-22T13:22:04.248466+0000","last_unstale":"2024-04-22T13:23:05.990905+0000","last_undegraded":"2024-04-22T13:23:05.990905+0000","last_fullsized":"2024-04-22T13:23:05.990905+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:38:10.871602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.509167+0000","last_change":"2024-04-22T13:22:05.258005+0000","last_active":"2024-04-22T13:23:05.509167+0000","last_peered":"2024-04-22T13:23:05.509167+0000","last_clean":"2024-04-22T13:23:05.509167+0000","last_became_active":"2024-04-22T13:22:05.257615+0000","last_became_peered":"2024-04-22T13:22:05.257615+0000","last_unstale":"2024-04-22T13:23:05.509167+0000","last_undegraded":"2024-04-22T13:23:05.509167+0000","last_fullsized":"2024-04-22T13:23:05.509167+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:21:18.030530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508625+0000","last_change":"2024-04-22T13:22:04.246273+0000","last_active":"2024-04-22T13:23:05.508625+0000","last_peered":"2024-04-22T13:23:05.508625+0000","last_clean":"2024-04-22T13:23:05.508625+0000","last_became_active":"2024-04-22T13:22:04.245969+0000","last_became_peered":"2024-04-22T13:22:04.245969+0000","last_unstale":"2024-04-22T13:23:05.508625+0000","last_undegraded":"2024-04-22T13:23:05.508625+0000","last_fullsized":"2024-04-22T13:23:05.508625+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:10.260192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.506145+0000","last_change":"2024-04-22T13:22:05.260299+0000","last_active":"2024-04-22T13:23:05.506145+0000","last_peered":"2024-04-22T13:23:05.506145+0000","last_clean":"2024-04-22T13:23:05.506145+0000","last_became_active":"2024-04-22T13:22:05.259634+0000","last_became_peered":"2024-04-22T13:22:05.259634+0000","last_unstale":"2024-04-22T13:23:05.506145+0000","last_undegraded":"2024-04-22T13:23:05.506145+0000","last_fullsized":"2024-04-22T13:23:05.506145+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:16:44.421652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508504+0000","last_change":"2024-04-22T13:22:04.243843+0000","last_active":"2024-04-22T13:23:05.508504+0000","last_peered":"2024-04-22T13:23:05.508504+0000","last_clean":"2024-04-22T13:23:05.508504+0000","last_became_active":"2024-04-22T13:22:04.243465+0000","last_became_peered":"2024-04-22T13:22:04.243465+0000","last_unstale":"2024-04-22T13:23:05.508504+0000","last_undegraded":"2024-04-22T13:23:05.508504+0000","last_fullsized":"2024-04-22T13:23:05.508504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:48:36.431697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.874474+0000","last_change":"2024-04-22T13:22:05.259789+0000","last_active":"2024-04-22T13:23:05.874474+0000","last_peered":"2024-04-22T13:23:05.874474+0000","last_clean":"2024-04-22T13:23:05.874474+0000","last_became_active":"2024-04-22T13:22:05.259395+0000","last_became_peered":"2024-04-22T13:22:05.259395+0000","last_unstale":"2024-04-22T13:23:05.874474+0000","last_undegraded":"2024-04-22T13:23:05.874474+0000","last_fullsized":"2024-04-22T13:23:05.874474+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:43:51.430561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":95,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:16.466964+0000","last_change":"2024-04-22T13:22:04.240615+0000","last_active":"2024-04-22T13:23:16.466964+0000","last_peered":"2024-04-22T13:23:16.466964+0000","last_clean":"2024-04-22T13:23:16.466964+0000","last_became_active":"2024-04-22T13:22:04.240248+0000","last_became_peered":"2024-04-22T13:22:04.240248+0000","last_unstale":"2024-04-22T13:23:16.466964+0000","last_undegraded":"2024-04-22T13:23:16.466964+0000","last_fullsized":"2024-04-22T13:23:16.466964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:55:05.948979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.507615+0000","last_change":"2024-04-22T13:22:05.253788+0000","last_active":"2024-04-22T13:23:05.507615+0000","last_peered":"2024-04-22T13:23:05.507615+0000","last_clean":"2024-04-22T13:23:05.507615+0000","last_became_active":"2024-04-22T13:22:05.252578+0000","last_became_peered":"2024-04-22T13:22:05.252578+0000","last_unstale":"2024-04-22T13:23:05.507615+0000","last_undegraded":"2024-04-22T13:23:05.507615+0000","last_fullsized":"2024-04-22T13:23:05.507615+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:04:11.769930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.507141+0000","last_change":"2024-04-22T13:22:04.241170+0000","last_active":"2024-04-22T13:23:05.507141+0000","last_peered":"2024-04-22T13:23:05.507141+0000","last_clean":"2024-04-22T13:23:05.507141+0000","last_became_active":"2024-04-22T13:22:04.240830+0000","last_became_peered":"2024-04-22T13:22:04.240830+0000","last_unstale":"2024-04-22T13:23:05.507141+0000","last_undegraded":"2024-04-22T13:23:05.507141+0000","last_fullsized":"2024-04-22T13:23:05.507141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:51:30.520613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.874397+0000","last_change":"2024-04-22T13:22:05.237900+0000","last_active":"2024-04-22T13:23:05.874397+0000","last_peered":"2024-04-22T13:23:05.874397+0000","last_clean":"2024-04-22T13:23:05.874397+0000","last_became_active":"2024-04-22T13:22:05.237507+0000","last_became_peered":"2024-04-22T13:22:05.237507+0000","last_unstale":"2024-04-22T13:23:05.874397+0000","last_undegraded":"2024-04-22T13:23:05.874397+0000","last_fullsized":"2024-04-22T13:23:05.874397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:26:24.307803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.863158+0000","last_change":"2024-04-22T13:22:04.236078+0000","last_active":"2024-04-22T13:23:05.863158+0000","last_peered":"2024-04-22T13:23:05.863158+0000","last_clean":"2024-04-22T13:23:05.863158+0000","last_became_active":"2024-04-22T13:22:04.235710+0000","last_became_peered":"2024-04-22T13:22:04.235710+0000","last_unstale":"2024-04-22T13:23:05.863158+0000","last_undegraded":"2024-04-22T13:23:05.863158+0000","last_fullsized":"2024-04-22T13:23:05.863158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:37:55.335237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.990253+0000","last_change":"2024-04-22T13:22:05.243152+0000","last_active":"2024-04-22T13:23:05.990253+0000","last_peered":"2024-04-22T13:23:05.990253+0000","last_clean":"2024-04-22T13:23:05.990253+0000","last_became_active":"2024-04-22T13:22:05.242772+0000","last_became_peered":"2024-04-22T13:22:05.242772+0000","last_unstale":"2024-04-22T13:23:05.990253+0000","last_undegraded":"2024-04-22T13:23:05.990253+0000","last_fullsized":"2024-04-22T13:23:05.990253+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:33:42.359679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.874282+0000","last_change":"2024-04-22T13:22:04.244381+0000","last_active":"2024-04-22T13:23:05.874282+0000","last_peered":"2024-04-22T13:23:05.874282+0000","last_clean":"2024-04-22T13:23:05.874282+0000","last_became_active":"2024-04-22T13:22:04.243746+0000","last_became_peered":"2024-04-22T13:22:04.243746+0000","last_unstale":"2024-04-22T13:23:05.874282+0000","last_undegraded":"2024-04-22T13:23:05.874282+0000","last_fullsized":"2024-04-22T13:23:05.874282+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:15:36.019930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.505047+0000","last_change":"2024-04-22T13:22:05.245394+0000","last_active":"2024-04-22T13:23:05.505047+0000","last_peered":"2024-04-22T13:23:05.505047+0000","last_clean":"2024-04-22T13:23:05.505047+0000","last_became_active":"2024-04-22T13:22:05.243931+0000","last_became_peered":"2024-04-22T13:22:05.243931+0000","last_unstale":"2024-04-22T13:23:05.505047+0000","last_undegraded":"2024-04-22T13:23:05.505047+0000","last_fullsized":"2024-04-22T13:23:05.505047+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:11:10.790842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":160,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:18.469173+0000","last_change":"2024-04-22T13:22:04.242583+0000","last_active":"2024-04-22T13:23:18.469173+0000","last_peered":"2024-04-22T13:23:18.469173+0000","last_clean":"2024-04-22T13:23:18.469173+0000","last_became_active":"2024-04-22T13:22:04.242261+0000","last_became_peered":"2024-04-22T13:22:04.242261+0000","last_unstale":"2024-04-22T13:23:18.469173+0000","last_undegraded":"2024-04-22T13:23:18.469173+0000","last_fullsized":"2024-04-22T13:23:18.469173+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:37:47.618573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":137,"num_read_kb":137,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.506448+0000","last_change":"2024-04-22T13:22:05.255293+0000","last_active":"2024-04-22T13:23:05.506448+0000","last_peered":"2024-04-22T13:23:05.506448+0000","last_clean":"2024-04-22T13:23:05.506448+0000","last_became_active":"2024-04-22T13:22:05.253949+0000","last_became_peered":"2024-04-22T13:22:05.253949+0000","last_unstale":"2024-04-22T13:23:05.506448+0000","last_undegraded":"2024-04-22T13:23:05.506448+0000","last_fullsized":"2024-04-22T13:23:05.506448+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:41:08.754989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508720+0000","last_change":"2024-04-22T13:22:04.241529+0000","last_active":"2024-04-22T13:23:05.508720+0000","last_peered":"2024-04-22T13:23:05.508720+0000","last_clean":"2024-04-22T13:23:05.508720+0000","last_became_active":"2024-04-22T13:22:04.241164+0000","last_became_peered":"2024-04-22T13:22:04.241164+0000","last_unstale":"2024-04-22T13:23:05.508720+0000","last_undegraded":"2024-04-22T13:23:05.508720+0000","last_fullsized":"2024-04-22T13:23:05.508720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:16:49.387181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.509076+0000","last_change":"2024-04-22T13:22:05.241623+0000","last_active":"2024-04-22T13:23:05.509076+0000","last_peered":"2024-04-22T13:23:05.509076+0000","last_clean":"2024-04-22T13:23:05.509076+0000","last_became_active":"2024-04-22T13:22:05.241150+0000","last_became_peered":"2024-04-22T13:22:05.241150+0000","last_unstale":"2024-04-22T13:23:05.509076+0000","last_undegraded":"2024-04-22T13:23:05.509076+0000","last_fullsized":"2024-04-22T13:23:05.509076+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:17:28.412366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.874565+0000","last_change":"2024-04-22T13:22:04.247852+0000","last_active":"2024-04-22T13:23:05.874565+0000","last_peered":"2024-04-22T13:23:05.874565+0000","last_clean":"2024-04-22T13:23:05.874565+0000","last_became_active":"2024-04-22T13:22:04.247550+0000","last_became_peered":"2024-04-22T13:22:04.247550+0000","last_unstale":"2024-04-22T13:23:05.874565+0000","last_undegraded":"2024-04-22T13:23:05.874565+0000","last_fullsized":"2024-04-22T13:23:05.874565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:55:01.928435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.506217+0000","last_change":"2024-04-22T13:22:05.260050+0000","last_active":"2024-04-22T13:23:05.506217+0000","last_peered":"2024-04-22T13:23:05.506217+0000","last_clean":"2024-04-22T13:23:05.506217+0000","last_became_active":"2024-04-22T13:22:05.259098+0000","last_became_peered":"2024-04-22T13:22:05.259098+0000","last_unstale":"2024-04-22T13:23:05.506217+0000","last_undegraded":"2024-04-22T13:23:05.506217+0000","last_fullsized":"2024-04-22T13:23:05.506217+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:05:06.245492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'29","reported_seq":196,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:18.469135+0000","last_change":"2024-04-22T13:22:04.240233+0000","last_active":"2024-04-22T13:23:18.469135+0000","last_peered":"2024-04-22T13:23:18.469135+0000","last_clean":"2024-04-22T13:23:18.469135+0000","last_became_active":"2024-04-22T13:22:04.239890+0000","last_became_peered":"2024-04-22T13:22:04.239890+0000","last_unstale":"2024-04-22T13:23:18.469135+0000","last_undegraded":"2024-04-22T13:23:18.469135+0000","last_fullsized":"2024-04-22T13:23:18.469135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:04:21.547422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59465,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":148,"num_read_kb":132,"num_write":31,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508400+0000","last_change":"2024-04-22T13:22:05.257756+0000","last_active":"2024-04-22T13:23:05.508400+0000","last_peered":"2024-04-22T13:23:05.508400+0000","last_clean":"2024-04-22T13:23:05.508400+0000","last_became_active":"2024-04-22T13:22:05.257442+0000","last_became_peered":"2024-04-22T13:22:05.257442+0000","last_unstale":"2024-04-22T13:23:05.508400+0000","last_undegraded":"2024-04-22T13:23:05.508400+0000","last_fullsized":"2024-04-22T13:23:05.508400+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:16:53.465689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.504279+0000","last_change":"2024-04-22T13:22:04.240815+0000","last_active":"2024-04-22T13:23:05.504279+0000","last_peered":"2024-04-22T13:23:05.504279+0000","last_clean":"2024-04-22T13:23:05.504279+0000","last_became_active":"2024-04-22T13:22:04.240218+0000","last_became_peered":"2024-04-22T13:22:04.240218+0000","last_unstale":"2024-04-22T13:23:05.504279+0000","last_undegraded":"2024-04-22T13:23:05.504279+0000","last_fullsized":"2024-04-22T13:23:05.504279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:29:29.148764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.863069+0000","last_change":"2024-04-22T13:22:05.248797+0000","last_active":"2024-04-22T13:23:05.863069+0000","last_peered":"2024-04-22T13:23:05.863069+0000","last_clean":"2024-04-22T13:23:05.863069+0000","last_became_active":"2024-04-22T13:22:05.248435+0000","last_became_peered":"2024-04-22T13:22:05.248435+0000","last_unstale":"2024-04-22T13:23:05.863069+0000","last_undegraded":"2024-04-22T13:23:05.863069+0000","last_fullsized":"2024-04-22T13:23:05.863069+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:32:31.970494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.874115+0000","last_change":"2024-04-22T13:22:04.245303+0000","last_active":"2024-04-22T13:23:05.874115+0000","last_peered":"2024-04-22T13:23:05.874115+0000","last_clean":"2024-04-22T13:23:05.874115+0000","last_became_active":"2024-04-22T13:22:04.244903+0000","last_became_peered":"2024-04-22T13:22:04.244903+0000","last_unstale":"2024-04-22T13:23:05.874115+0000","last_undegraded":"2024-04-22T13:23:05.874115+0000","last_fullsized":"2024-04-22T13:23:05.874115+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:34:32.174858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508763+0000","last_change":"2024-04-22T13:22:05.246318+0000","last_active":"2024-04-22T13:23:05.508763+0000","last_peered":"2024-04-22T13:23:05.508763+0000","last_clean":"2024-04-22T13:23:05.508763+0000","last_became_active":"2024-04-22T13:22:05.245899+0000","last_became_peered":"2024-04-22T13:22:05.245899+0000","last_unstale":"2024-04-22T13:23:05.508763+0000","last_undegraded":"2024-04-22T13:23:05.508763+0000","last_fullsized":"2024-04-22T13:23:05.508763+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:27:42.789058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":96,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:19.510505+0000","last_change":"2024-04-22T13:22:04.240612+0000","last_active":"2024-04-22T13:23:19.510505+0000","last_peered":"2024-04-22T13:23:19.510505+0000","last_clean":"2024-04-22T13:23:19.510505+0000","last_became_active":"2024-04-22T13:22:04.240290+0000","last_became_peered":"2024-04-22T13:22:04.240290+0000","last_unstale":"2024-04-22T13:23:19.510505+0000","last_undegraded":"2024-04-22T13:23:19.510505+0000","last_fullsized":"2024-04-22T13:23:19.510505+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:00:50.285553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508228+0000","last_change":"2024-04-22T13:22:05.253205+0000","last_active":"2024-04-22T13:23:05.508228+0000","last_peered":"2024-04-22T13:23:05.508228+0000","last_clean":"2024-04-22T13:23:05.508228+0000","last_became_active":"2024-04-22T13:22:05.252865+0000","last_became_peered":"2024-04-22T13:22:05.252865+0000","last_unstale":"2024-04-22T13:23:05.508228+0000","last_undegraded":"2024-04-22T13:23:05.508228+0000","last_fullsized":"2024-04-22T13:23:05.508228+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:19:12.482371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.507514+0000","last_change":"2024-04-22T13:22:04.248938+0000","last_active":"2024-04-22T13:23:05.507514+0000","last_peered":"2024-04-22T13:23:05.507514+0000","last_clean":"2024-04-22T13:23:05.507514+0000","last_became_active":"2024-04-22T13:22:04.248440+0000","last_became_peered":"2024-04-22T13:22:04.248440+0000","last_unstale":"2024-04-22T13:23:05.507514+0000","last_undegraded":"2024-04-22T13:23:05.507514+0000","last_fullsized":"2024-04-22T13:23:05.507514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:46:50.928067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.874410+0000","last_change":"2024-04-22T13:22:05.254664+0000","last_active":"2024-04-22T13:23:05.874410+0000","last_peered":"2024-04-22T13:23:05.874410+0000","last_clean":"2024-04-22T13:23:05.874410+0000","last_became_active":"2024-04-22T13:22:05.254304+0000","last_became_peered":"2024-04-22T13:22:05.254304+0000","last_unstale":"2024-04-22T13:23:05.874410+0000","last_undegraded":"2024-04-22T13:23:05.874410+0000","last_fullsized":"2024-04-22T13:23:05.874410+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:58:49.345573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508830+0000","last_change":"2024-04-22T13:22:04.244128+0000","last_active":"2024-04-22T13:23:05.508830+0000","last_peered":"2024-04-22T13:23:05.508830+0000","last_clean":"2024-04-22T13:23:05.508830+0000","last_became_active":"2024-04-22T13:22:04.243738+0000","last_became_peered":"2024-04-22T13:22:04.243738+0000","last_unstale":"2024-04-22T13:23:05.508830+0000","last_undegraded":"2024-04-22T13:23:05.508830+0000","last_fullsized":"2024-04-22T13:23:05.508830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:23:52.220584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.505822+0000","last_change":"2024-04-22T13:22:05.250910+0000","last_active":"2024-04-22T13:23:05.505822+0000","last_peered":"2024-04-22T13:23:05.505822+0000","last_clean":"2024-04-22T13:23:05.505822+0000","last_became_active":"2024-04-22T13:22:05.250410+0000","last_became_peered":"2024-04-22T13:22:05.250410+0000","last_unstale":"2024-04-22T13:23:05.505822+0000","last_undegraded":"2024-04-22T13:23:05.505822+0000","last_fullsized":"2024-04-22T13:23:05.505822+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:04:25.492677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.504909+0000","last_change":"2024-04-22T13:22:04.245889+0000","last_active":"2024-04-22T13:23:05.504909+0000","last_peered":"2024-04-22T13:23:05.504909+0000","last_clean":"2024-04-22T13:23:05.504909+0000","last_became_active":"2024-04-22T13:22:04.245474+0000","last_became_peered":"2024-04-22T13:22:04.245474+0000","last_unstale":"2024-04-22T13:23:05.504909+0000","last_undegraded":"2024-04-22T13:23:05.504909+0000","last_fullsized":"2024-04-22T13:23:05.504909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:22:12.626950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.507250+0000","last_change":"2024-04-22T13:22:05.250978+0000","last_active":"2024-04-22T13:23:05.507250+0000","last_peered":"2024-04-22T13:23:05.507250+0000","last_clean":"2024-04-22T13:23:05.507250+0000","last_became_active":"2024-04-22T13:22:05.250463+0000","last_became_peered":"2024-04-22T13:22:05.250463+0000","last_unstale":"2024-04-22T13:23:05.507250+0000","last_undegraded":"2024-04-22T13:23:05.507250+0000","last_fullsized":"2024-04-22T13:23:05.507250+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:45:18.591072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":93,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:18.497555+0000","last_change":"2024-04-22T13:22:04.242503+0000","last_active":"2024-04-22T13:23:18.497555+0000","last_peered":"2024-04-22T13:23:18.497555+0000","last_clean":"2024-04-22T13:23:18.497555+0000","last_became_active":"2024-04-22T13:22:04.242143+0000","last_became_peered":"2024-04-22T13:22:04.242143+0000","last_unstale":"2024-04-22T13:23:18.497555+0000","last_undegraded":"2024-04-22T13:23:18.497555+0000","last_fullsized":"2024-04-22T13:23:18.497555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:40.236377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.507316+0000","last_change":"2024-04-22T13:22:05.252387+0000","last_active":"2024-04-22T13:23:05.507316+0000","last_peered":"2024-04-22T13:23:05.507316+0000","last_clean":"2024-04-22T13:23:05.507316+0000","last_became_active":"2024-04-22T13:22:05.252033+0000","last_became_peered":"2024-04-22T13:22:05.252033+0000","last_unstale":"2024-04-22T13:23:05.507316+0000","last_undegraded":"2024-04-22T13:23:05.507316+0000","last_fullsized":"2024-04-22T13:23:05.507316+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:55:01.791283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.874416+0000","last_change":"2024-04-22T13:22:05.248394+0000","last_active":"2024-04-22T13:23:05.874416+0000","last_peered":"2024-04-22T13:23:05.874416+0000","last_clean":"2024-04-22T13:23:05.874416+0000","last_became_active":"2024-04-22T13:22:05.248024+0000","last_became_peered":"2024-04-22T13:22:05.248024+0000","last_unstale":"2024-04-22T13:23:05.874416+0000","last_undegraded":"2024-04-22T13:23:05.874416+0000","last_fullsized":"2024-04-22T13:23:05.874416+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:49:36.556136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":92,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:17.506027+0000","last_change":"2024-04-22T13:22:04.247692+0000","last_active":"2024-04-22T13:23:17.506027+0000","last_peered":"2024-04-22T13:23:17.506027+0000","last_clean":"2024-04-22T13:23:17.506027+0000","last_became_active":"2024-04-22T13:22:04.247345+0000","last_became_peered":"2024-04-22T13:22:04.247345+0000","last_unstale":"2024-04-22T13:23:17.506027+0000","last_undegraded":"2024-04-22T13:23:17.506027+0000","last_fullsized":"2024-04-22T13:23:17.506027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:07:49.803248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.990908+0000","last_change":"2024-04-22T13:22:05.241022+0000","last_active":"2024-04-22T13:23:05.990908+0000","last_peered":"2024-04-22T13:23:05.990908+0000","last_clean":"2024-04-22T13:23:05.990908+0000","last_became_active":"2024-04-22T13:22:05.240635+0000","last_became_peered":"2024-04-22T13:22:05.240635+0000","last_unstale":"2024-04-22T13:23:05.990908+0000","last_undegraded":"2024-04-22T13:23:05.990908+0000","last_fullsized":"2024-04-22T13:23:05.990908+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:50:02.012260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.990537+0000","last_change":"2024-04-22T13:22:04.236817+0000","last_active":"2024-04-22T13:23:05.990537+0000","last_peered":"2024-04-22T13:23:05.990537+0000","last_clean":"2024-04-22T13:23:05.990537+0000","last_became_active":"2024-04-22T13:22:04.236484+0000","last_became_peered":"2024-04-22T13:22:04.236484+0000","last_unstale":"2024-04-22T13:23:05.990537+0000","last_undegraded":"2024-04-22T13:23:05.990537+0000","last_fullsized":"2024-04-22T13:23:05.990537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:27:41.053327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":354,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508269+0000","last_change":"2024-04-22T13:20:53.094123+0000","last_active":"2024-04-22T13:23:05.508269+0000","last_peered":"2024-04-22T13:23:05.508269+0000","last_clean":"2024-04-22T13:23:05.508269+0000","last_became_active":"2024-04-22T13:20:53.093664+0000","last_became_peered":"2024-04-22T13:20:53.093664+0000","last_unstale":"2024-04-22T13:23:05.508269+0000","last_undegraded":"2024-04-22T13:23:05.508269+0000","last_fullsized":"2024-04-22T13:23:05.508269+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:17:33.201786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:17:33.201786+0000","last_clean_scrub_stamp":"2024-04-22T13:17:33.201786+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:46:58.986399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.862908+0000","last_change":"2024-04-22T13:22:05.247766+0000","last_active":"2024-04-22T13:23:05.862908+0000","last_peered":"2024-04-22T13:23:05.862908+0000","last_clean":"2024-04-22T13:23:05.862908+0000","last_became_active":"2024-04-22T13:22:05.247362+0000","last_became_peered":"2024-04-22T13:22:05.247362+0000","last_unstale":"2024-04-22T13:23:05.862908+0000","last_undegraded":"2024-04-22T13:23:05.862908+0000","last_fullsized":"2024-04-22T13:23:05.862908+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:25:00.126638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508388+0000","last_change":"2024-04-22T13:22:04.243493+0000","last_active":"2024-04-22T13:23:05.508388+0000","last_peered":"2024-04-22T13:23:05.508388+0000","last_clean":"2024-04-22T13:23:05.508388+0000","last_became_active":"2024-04-22T13:22:04.243190+0000","last_became_peered":"2024-04-22T13:22:04.243190+0000","last_unstale":"2024-04-22T13:23:05.508388+0000","last_undegraded":"2024-04-22T13:23:05.508388+0000","last_fullsized":"2024-04-22T13:23:05.508388+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:15:09.025152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508498+0000","last_change":"2024-04-22T13:22:04.249222+0000","last_active":"2024-04-22T13:23:05.508498+0000","last_peered":"2024-04-22T13:23:05.508498+0000","last_clean":"2024-04-22T13:23:05.508498+0000","last_became_active":"2024-04-22T13:22:04.248926+0000","last_became_peered":"2024-04-22T13:22:04.248926+0000","last_unstale":"2024-04-22T13:23:05.508498+0000","last_undegraded":"2024-04-22T13:23:05.508498+0000","last_fullsized":"2024-04-22T13:23:05.508498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:33:50.168426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.863308+0000","last_change":"2024-04-22T13:22:05.248199+0000","last_active":"2024-04-22T13:23:05.863308+0000","last_peered":"2024-04-22T13:23:05.863308+0000","last_clean":"2024-04-22T13:23:05.863308+0000","last_became_active":"2024-04-22T13:22:05.247873+0000","last_became_peered":"2024-04-22T13:22:05.247873+0000","last_unstale":"2024-04-22T13:23:05.863308+0000","last_undegraded":"2024-04-22T13:23:05.863308+0000","last_fullsized":"2024-04-22T13:23:05.863308+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:40:29.353901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'12","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.505242+0000","last_change":"2024-04-22T13:22:04.243474+0000","last_active":"2024-04-22T13:23:05.505242+0000","last_peered":"2024-04-22T13:23:05.505242+0000","last_clean":"2024-04-22T13:23:05.505242+0000","last_became_active":"2024-04-22T13:22:04.243151+0000","last_became_peered":"2024-04-22T13:22:04.243151+0000","last_unstale":"2024-04-22T13:23:05.505242+0000","last_undegraded":"2024-04-22T13:23:05.505242+0000","last_fullsized":"2024-04-22T13:23:05.505242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:18:25.140715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.990761+0000","last_change":"2024-04-22T13:22:05.243120+0000","last_active":"2024-04-22T13:23:05.990761+0000","last_peered":"2024-04-22T13:23:05.990761+0000","last_clean":"2024-04-22T13:23:05.990761+0000","last_became_active":"2024-04-22T13:22:05.242726+0000","last_became_peered":"2024-04-22T13:22:05.242726+0000","last_unstale":"2024-04-22T13:23:05.990761+0000","last_undegraded":"2024-04-22T13:23:05.990761+0000","last_fullsized":"2024-04-22T13:23:05.990761+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:06:16.087884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.990760+0000","last_change":"2024-04-22T13:22:04.239672+0000","last_active":"2024-04-22T13:23:05.990760+0000","last_peered":"2024-04-22T13:23:05.990760+0000","last_clean":"2024-04-22T13:23:05.990760+0000","last_became_active":"2024-04-22T13:22:04.238776+0000","last_became_peered":"2024-04-22T13:22:04.238776+0000","last_unstale":"2024-04-22T13:23:05.990760+0000","last_undegraded":"2024-04-22T13:23:05.990760+0000","last_fullsized":"2024-04-22T13:23:05.990760+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:43:21.752547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508977+0000","last_change":"2024-04-22T13:22:05.242089+0000","last_active":"2024-04-22T13:23:05.508977+0000","last_peered":"2024-04-22T13:23:05.508977+0000","last_clean":"2024-04-22T13:23:05.508977+0000","last_became_active":"2024-04-22T13:22:05.241681+0000","last_became_peered":"2024-04-22T13:22:05.241681+0000","last_unstale":"2024-04-22T13:23:05.508977+0000","last_undegraded":"2024-04-22T13:23:05.508977+0000","last_fullsized":"2024-04-22T13:23:05.508977+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:58:49.906188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.513123+0000","last_change":"2024-04-22T13:23:05.513123+0000","last_active":"2024-04-22T13:23:05.513123+0000","last_peered":"2024-04-22T13:23:05.513123+0000","last_clean":"2024-04-22T13:23:05.513123+0000","last_became_active":"2024-04-22T13:23:05.509367+0000","last_became_peered":"2024-04-22T13:23:05.509367+0000","last_unstale":"2024-04-22T13:23:05.513123+0000","last_undegraded":"2024-04-22T13:23:05.513123+0000","last_fullsized":"2024-04-22T13:23:05.513123+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:28:41.770957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.507644+0000","last_change":"2024-04-22T13:22:05.250120+0000","last_active":"2024-04-22T13:23:05.507644+0000","last_peered":"2024-04-22T13:23:05.507644+0000","last_clean":"2024-04-22T13:23:05.507644+0000","last_became_active":"2024-04-22T13:22:05.247936+0000","last_became_peered":"2024-04-22T13:22:05.247936+0000","last_unstale":"2024-04-22T13:23:05.507644+0000","last_undegraded":"2024-04-22T13:23:05.507644+0000","last_fullsized":"2024-04-22T13:23:05.507644+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:27:38.437918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.990406+0000","last_change":"2024-04-22T13:22:04.244026+0000","last_active":"2024-04-22T13:23:05.990406+0000","last_peered":"2024-04-22T13:23:05.990406+0000","last_clean":"2024-04-22T13:23:05.990406+0000","last_became_active":"2024-04-22T13:22:04.243518+0000","last_became_peered":"2024-04-22T13:22:04.243518+0000","last_unstale":"2024-04-22T13:23:05.990406+0000","last_undegraded":"2024-04-22T13:23:05.990406+0000","last_fullsized":"2024-04-22T13:23:05.990406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:54:48.791874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.505929+0000","last_change":"2024-04-22T13:22:05.252674+0000","last_active":"2024-04-22T13:23:05.505929+0000","last_peered":"2024-04-22T13:23:05.505929+0000","last_clean":"2024-04-22T13:23:05.505929+0000","last_became_active":"2024-04-22T13:22:05.251087+0000","last_became_peered":"2024-04-22T13:22:05.251087+0000","last_unstale":"2024-04-22T13:23:05.505929+0000","last_undegraded":"2024-04-22T13:23:05.505929+0000","last_fullsized":"2024-04-22T13:23:05.505929+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:20:06.873412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.504220+0000","last_change":"2024-04-22T13:22:04.244409+0000","last_active":"2024-04-22T13:23:05.504220+0000","last_peered":"2024-04-22T13:23:05.504220+0000","last_clean":"2024-04-22T13:23:05.504220+0000","last_became_active":"2024-04-22T13:22:04.244104+0000","last_became_peered":"2024-04-22T13:22:04.244104+0000","last_unstale":"2024-04-22T13:23:05.504220+0000","last_undegraded":"2024-04-22T13:23:05.504220+0000","last_fullsized":"2024-04-22T13:23:05.504220+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:39:14.714755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.990777+0000","last_change":"2024-04-22T13:22:05.245730+0000","last_active":"2024-04-22T13:23:05.990777+0000","last_peered":"2024-04-22T13:23:05.990777+0000","last_clean":"2024-04-22T13:23:05.990777+0000","last_became_active":"2024-04-22T13:22:05.245366+0000","last_became_peered":"2024-04-22T13:22:05.245366+0000","last_unstale":"2024-04-22T13:23:05.990777+0000","last_undegraded":"2024-04-22T13:23:05.990777+0000","last_fullsized":"2024-04-22T13:23:05.990777+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:31.935258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.514632+0000","last_change":"2024-04-22T13:23:05.514197+0000","last_active":"2024-04-22T13:23:05.514632+0000","last_peered":"2024-04-22T13:23:05.514632+0000","last_clean":"2024-04-22T13:23:05.514632+0000","last_became_active":"2024-04-22T13:23:05.509431+0000","last_became_peered":"2024-04-22T13:23:05.509431+0000","last_unstale":"2024-04-22T13:23:05.514632+0000","last_undegraded":"2024-04-22T13:23:05.514632+0000","last_fullsized":"2024-04-22T13:23:05.514632+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:43:59.825567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508622+0000","last_change":"2024-04-22T13:22:05.242146+0000","last_active":"2024-04-22T13:23:05.508622+0000","last_peered":"2024-04-22T13:23:05.508622+0000","last_clean":"2024-04-22T13:23:05.508622+0000","last_became_active":"2024-04-22T13:22:05.241802+0000","last_became_peered":"2024-04-22T13:22:05.241802+0000","last_unstale":"2024-04-22T13:23:05.508622+0000","last_undegraded":"2024-04-22T13:23:05.508622+0000","last_fullsized":"2024-04-22T13:23:05.508622+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:49:49.002123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":96,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:20.500478+0000","last_change":"2024-04-22T13:22:04.248884+0000","last_active":"2024-04-22T13:23:20.500478+0000","last_peered":"2024-04-22T13:23:20.500478+0000","last_clean":"2024-04-22T13:23:20.500478+0000","last_became_active":"2024-04-22T13:22:04.248393+0000","last_became_peered":"2024-04-22T13:22:04.248393+0000","last_unstale":"2024-04-22T13:23:20.500478+0000","last_undegraded":"2024-04-22T13:23:20.500478+0000","last_fullsized":"2024-04-22T13:23:20.500478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:36:14.687457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508653+0000","last_change":"2024-04-22T13:22:05.256881+0000","last_active":"2024-04-22T13:23:05.508653+0000","last_peered":"2024-04-22T13:23:05.508653+0000","last_clean":"2024-04-22T13:23:05.508653+0000","last_became_active":"2024-04-22T13:22:05.255967+0000","last_became_peered":"2024-04-22T13:22:05.255967+0000","last_unstale":"2024-04-22T13:23:05.508653+0000","last_undegraded":"2024-04-22T13:23:05.508653+0000","last_fullsized":"2024-04-22T13:23:05.508653+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:24:21.137344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.874347+0000","last_change":"2024-04-22T13:22:04.247303+0000","last_active":"2024-04-22T13:23:05.874347+0000","last_peered":"2024-04-22T13:23:05.874347+0000","last_clean":"2024-04-22T13:23:05.874347+0000","last_became_active":"2024-04-22T13:22:04.246742+0000","last_became_peered":"2024-04-22T13:22:04.246742+0000","last_unstale":"2024-04-22T13:23:05.874347+0000","last_undegraded":"2024-04-22T13:23:05.874347+0000","last_fullsized":"2024-04-22T13:23:05.874347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:45:03.189042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508435+0000","last_change":"2024-04-22T13:22:05.251754+0000","last_active":"2024-04-22T13:23:05.508435+0000","last_peered":"2024-04-22T13:23:05.508435+0000","last_clean":"2024-04-22T13:23:05.508435+0000","last_became_active":"2024-04-22T13:22:05.251155+0000","last_became_peered":"2024-04-22T13:22:05.251155+0000","last_unstale":"2024-04-22T13:23:05.508435+0000","last_undegraded":"2024-04-22T13:23:05.508435+0000","last_fullsized":"2024-04-22T13:23:05.508435+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:08:46.682600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.863343+0000","last_change":"2024-04-22T13:22:04.243382+0000","last_active":"2024-04-22T13:23:05.863343+0000","last_peered":"2024-04-22T13:23:05.863343+0000","last_clean":"2024-04-22T13:23:05.863343+0000","last_became_active":"2024-04-22T13:22:04.243055+0000","last_became_peered":"2024-04-22T13:22:04.243055+0000","last_unstale":"2024-04-22T13:23:05.863343+0000","last_undegraded":"2024-04-22T13:23:05.863343+0000","last_fullsized":"2024-04-22T13:23:05.863343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:19:58.379645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508551+0000","last_change":"2024-04-22T13:22:05.249527+0000","last_active":"2024-04-22T13:23:05.508551+0000","last_peered":"2024-04-22T13:23:05.508551+0000","last_clean":"2024-04-22T13:23:05.508551+0000","last_became_active":"2024-04-22T13:22:05.249207+0000","last_became_peered":"2024-04-22T13:22:05.249207+0000","last_unstale":"2024-04-22T13:23:05.508551+0000","last_undegraded":"2024-04-22T13:23:05.508551+0000","last_fullsized":"2024-04-22T13:23:05.508551+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:50:02.755288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508324+0000","last_change":"2024-04-22T13:23:05.508324+0000","last_active":"2024-04-22T13:23:05.508324+0000","last_peered":"2024-04-22T13:23:05.508324+0000","last_clean":"2024-04-22T13:23:05.508324+0000","last_became_active":"2024-04-22T13:23:05.507814+0000","last_became_peered":"2024-04-22T13:23:05.507814+0000","last_unstale":"2024-04-22T13:23:05.508324+0000","last_undegraded":"2024-04-22T13:23:05.508324+0000","last_fullsized":"2024-04-22T13:23:05.508324+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:46:20.523715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.875683+0000","last_change":"2024-04-22T13:23:05.875683+0000","last_active":"2024-04-22T13:23:05.875683+0000","last_peered":"2024-04-22T13:23:05.875683+0000","last_clean":"2024-04-22T13:23:05.875683+0000","last_became_active":"2024-04-22T13:23:05.875150+0000","last_became_peered":"2024-04-22T13:23:05.875150+0000","last_unstale":"2024-04-22T13:23:05.875683+0000","last_undegraded":"2024-04-22T13:23:05.875683+0000","last_fullsized":"2024-04-22T13:23:05.875683+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:12:47.592230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.504119+0000","last_change":"2024-04-22T13:22:05.247882+0000","last_active":"2024-04-22T13:23:05.504119+0000","last_peered":"2024-04-22T13:23:05.504119+0000","last_clean":"2024-04-22T13:23:05.504119+0000","last_became_active":"2024-04-22T13:22:05.247525+0000","last_became_peered":"2024-04-22T13:22:05.247525+0000","last_unstale":"2024-04-22T13:23:05.504119+0000","last_undegraded":"2024-04-22T13:23:05.504119+0000","last_fullsized":"2024-04-22T13:23:05.504119+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:46:04.349076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508413+0000","last_change":"2024-04-22T13:22:05.243716+0000","last_active":"2024-04-22T13:23:05.508413+0000","last_peered":"2024-04-22T13:23:05.508413+0000","last_clean":"2024-04-22T13:23:05.508413+0000","last_became_active":"2024-04-22T13:22:05.243397+0000","last_became_peered":"2024-04-22T13:22:05.243397+0000","last_unstale":"2024-04-22T13:23:05.508413+0000","last_undegraded":"2024-04-22T13:23:05.508413+0000","last_fullsized":"2024-04-22T13:23:05.508413+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:16:21.236339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.863393+0000","last_change":"2024-04-22T13:22:05.246042+0000","last_active":"2024-04-22T13:23:05.863393+0000","last_peered":"2024-04-22T13:23:05.863393+0000","last_clean":"2024-04-22T13:23:05.863393+0000","last_became_active":"2024-04-22T13:22:05.245669+0000","last_became_peered":"2024-04-22T13:22:05.245669+0000","last_unstale":"2024-04-22T13:23:05.863393+0000","last_undegraded":"2024-04-22T13:23:05.863393+0000","last_fullsized":"2024-04-22T13:23:05.863393+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:49:05.539641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.505796+0000","last_change":"2024-04-22T13:22:05.257009+0000","last_active":"2024-04-22T13:23:05.505796+0000","last_peered":"2024-04-22T13:23:05.505796+0000","last_clean":"2024-04-22T13:23:05.505796+0000","last_became_active":"2024-04-22T13:22:05.254327+0000","last_became_peered":"2024-04-22T13:22:05.254327+0000","last_unstale":"2024-04-22T13:23:05.505796+0000","last_undegraded":"2024-04-22T13:23:05.505796+0000","last_fullsized":"2024-04-22T13:23:05.505796+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:29:12.482525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.504704+0000","last_change":"2024-04-22T13:22:05.260575+0000","last_active":"2024-04-22T13:23:05.504704+0000","last_peered":"2024-04-22T13:23:05.504704+0000","last_clean":"2024-04-22T13:23:05.504704+0000","last_became_active":"2024-04-22T13:22:05.260181+0000","last_became_peered":"2024-04-22T13:22:05.260181+0000","last_unstale":"2024-04-22T13:23:05.504704+0000","last_undegraded":"2024-04-22T13:23:05.504704+0000","last_fullsized":"2024-04-22T13:23:05.504704+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:49:20.064368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.990636+0000","last_change":"2024-04-22T13:22:05.248199+0000","last_active":"2024-04-22T13:23:05.990636+0000","last_peered":"2024-04-22T13:23:05.990636+0000","last_clean":"2024-04-22T13:23:05.990636+0000","last_became_active":"2024-04-22T13:22:05.247858+0000","last_became_peered":"2024-04-22T13:22:05.247858+0000","last_unstale":"2024-04-22T13:23:05.990636+0000","last_undegraded":"2024-04-22T13:23:05.990636+0000","last_fullsized":"2024-04-22T13:23:05.990636+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:49:21.228105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.874325+0000","last_change":"2024-04-22T13:22:05.239137+0000","last_active":"2024-04-22T13:23:05.874325+0000","last_peered":"2024-04-22T13:23:05.874325+0000","last_clean":"2024-04-22T13:23:05.874325+0000","last_became_active":"2024-04-22T13:22:05.238807+0000","last_became_peered":"2024-04-22T13:22:05.238807+0000","last_unstale":"2024-04-22T13:23:05.874325+0000","last_undegraded":"2024-04-22T13:23:05.874325+0000","last_fullsized":"2024-04-22T13:23:05.874325+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:02:48.587097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.990356+0000","last_change":"2024-04-22T13:22:05.249080+0000","last_active":"2024-04-22T13:23:05.990356+0000","last_peered":"2024-04-22T13:23:05.990356+0000","last_clean":"2024-04-22T13:23:05.990356+0000","last_became_active":"2024-04-22T13:22:05.248733+0000","last_became_peered":"2024-04-22T13:22:05.248733+0000","last_unstale":"2024-04-22T13:23:05.990356+0000","last_undegraded":"2024-04-22T13:23:05.990356+0000","last_fullsized":"2024-04-22T13:23:05.990356+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:58.735941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.506059+0000","last_change":"2024-04-22T13:22:05.255490+0000","last_active":"2024-04-22T13:23:05.506059+0000","last_peered":"2024-04-22T13:23:05.506059+0000","last_clean":"2024-04-22T13:23:05.506059+0000","last_became_active":"2024-04-22T13:22:05.253644+0000","last_became_peered":"2024-04-22T13:22:05.253644+0000","last_unstale":"2024-04-22T13:23:05.506059+0000","last_undegraded":"2024-04-22T13:23:05.506059+0000","last_fullsized":"2024-04-22T13:23:05.506059+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:31:01.816182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.863180+0000","last_change":"2024-04-22T13:22:05.240988+0000","last_active":"2024-04-22T13:23:05.863180+0000","last_peered":"2024-04-22T13:23:05.863180+0000","last_clean":"2024-04-22T13:23:05.863180+0000","last_became_active":"2024-04-22T13:22:05.240642+0000","last_became_peered":"2024-04-22T13:22:05.240642+0000","last_unstale":"2024-04-22T13:23:05.863180+0000","last_undegraded":"2024-04-22T13:23:05.863180+0000","last_fullsized":"2024-04-22T13:23:05.863180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:41:29.001979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.505682+0000","last_change":"2024-04-22T13:22:05.252212+0000","last_active":"2024-04-22T13:23:05.505682+0000","last_peered":"2024-04-22T13:23:05.505682+0000","last_clean":"2024-04-22T13:23:05.505682+0000","last_became_active":"2024-04-22T13:22:05.251852+0000","last_became_peered":"2024-04-22T13:22:05.251852+0000","last_unstale":"2024-04-22T13:23:05.505682+0000","last_undegraded":"2024-04-22T13:23:05.505682+0000","last_fullsized":"2024-04-22T13:23:05.505682+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:13:55.132501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66278,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":724,"num_read_kb":432,"num_write":235,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":198834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318302,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":463162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579937,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1667072,"data_stored":1114047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743463,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":463092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005099,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":463470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234033,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32732,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712420,"statfs":{"total":95995035648,"available":95961518080,"internally_reserved":0,"allocated":1585152,"data_stored":1055988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430262,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":974848,"data_stored":462809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":523187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":464148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1056768,"data_stored":523407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117062,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":991232,"data_stored":462634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855425,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":463061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691898,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1597440,"data_stored":1055275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T13:23:23.301 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-22T13:23:23.571 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:23:23.887 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:23 smithi012 ceph-mon[28265]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-22T13:23:23.887 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:23 smithi012 ceph-mon[28265]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:23:23.887 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:23 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:23:23.887 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:23 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:23:23.887 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:23 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:23.887 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:23 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:23:23.887 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:23 smithi012 ceph-mon[28265]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:23:23.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:23 smithi134 ceph-mon[31338]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-22T13:23:23.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:23 smithi134 ceph-mon[31338]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:23:23.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:23 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:23:23.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:23 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:23:23.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:23 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:23.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:23 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:23:23.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:23 smithi134 ceph-mon[31338]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:23:23.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:23 smithi111 ceph-mon[31361]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-22T13:23:23.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:23 smithi111 ceph-mon[31361]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:23:23.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:23 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:23:23.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:23 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:23:23.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:23 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:23:23.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:23 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:23:23.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:23 smithi111 ceph-mon[31361]: from='mgr.14152 172.21.15.12:0/2692025120' entity='mgr.x' 2024-04-22T13:23:25.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.237+0000 7fa851719700 1 -- 172.21.15.12:0/281572448 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa84c108a50 msgr2=0x7fa84c108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:25.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.237+0000 7fa851719700 1 --2- 172.21.15.12:0/281572448 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa84c108a50 0x7fa84c108e30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa84400b0a0 tx=0x7fa84400b3b0 comp rx=0 tx=0).stop 2024-04-22T13:23:25.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.237+0000 7fa851719700 1 -- 172.21.15.12:0/281572448 shutdown_connections 2024-04-22T13:23:25.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.237+0000 7fa851719700 1 --2- 172.21.15.12:0/281572448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa84c103450 0x7fa84c10f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:25.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.237+0000 7fa851719700 1 --2- 172.21.15.12:0/281572448 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa84c108a50 0x7fa84c108e30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:25.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.237+0000 7fa851719700 1 --2- 172.21.15.12:0/281572448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa84c102a90 0x7fa84c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:25.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.237+0000 7fa851719700 1 -- 172.21.15.12:0/281572448 >> 172.21.15.12:0/281572448 conn(0x7fa84c079ba0 msgr2=0x7fa84c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:25.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.237+0000 7fa851719700 1 -- 172.21.15.12:0/281572448 shutdown_connections 2024-04-22T13:23:25.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.237+0000 7fa851719700 1 -- 172.21.15.12:0/281572448 wait complete. 2024-04-22T13:23:25.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.237+0000 7fa851719700 1 Processor -- start 2024-04-22T13:23:25.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.238+0000 7fa851719700 1 -- start start 2024-04-22T13:23:25.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.238+0000 7fa851719700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa84c102a90 0x7fa84c1990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:25.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.238+0000 7fa851719700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa84c103450 0x7fa84c199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:25.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.238+0000 7fa851719700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa84c108a50 0x7fa84c19d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:25.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.238+0000 7fa851719700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c112be0 con 0x7fa84c108a50 2024-04-22T13:23:25.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.238+0000 7fa851719700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c112a60 con 0x7fa84c103450 2024-04-22T13:23:25.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.238+0000 7fa851719700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c112d60 con 0x7fa84c102a90 2024-04-22T13:23:25.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.238+0000 7fa84b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa84c108a50 0x7fa84c19d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:25.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.238+0000 7fa84b7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa84c108a50 0x7fa84c19d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47750/0 (socket says 172.21.15.12:47750) 2024-04-22T13:23:25.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.238+0000 7fa84b7fe700 1 -- 172.21.15.12:0/3203694150 learned_addr learned my addr 172.21.15.12:0/3203694150 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:25.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.238+0000 7fa84affd700 1 --2- 172.21.15.12:0/3203694150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa84c102a90 0x7fa84c1990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:25.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.238+0000 7fa84a7fc700 1 --2- 172.21.15.12:0/3203694150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa84c103450 0x7fa84c199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:25.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.238+0000 7fa84b7fe700 1 -- 172.21.15.12:0/3203694150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa84c102a90 msgr2=0x7fa84c1990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:25.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.238+0000 7fa84b7fe700 1 --2- 172.21.15.12:0/3203694150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa84c102a90 0x7fa84c1990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:25.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.238+0000 7fa84b7fe700 1 -- 172.21.15.12:0/3203694150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa84c103450 msgr2=0x7fa84c199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:25.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.238+0000 7fa84b7fe700 1 --2- 172.21.15.12:0/3203694150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa84c103450 0x7fa84c199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:25.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.238+0000 7fa84b7fe700 1 -- 172.21.15.12:0/3203694150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84c19e090 con 0x7fa84c108a50 2024-04-22T13:23:25.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.238+0000 7fa84b7fe700 1 --2- 172.21.15.12:0/3203694150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa84c108a50 0x7fa84c19d950 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa83c00c900 tx=0x7fa83c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:25.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.239+0000 7fa843fff700 1 -- 172.21.15.12:0/3203694150 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c0078d0 con 0x7fa84c108a50 2024-04-22T13:23:25.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.239+0000 7fa843fff700 1 -- 172.21.15.12:0/3203694150 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa83c00f450 con 0x7fa84c108a50 2024-04-22T13:23:25.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.239+0000 7fa843fff700 1 -- 172.21.15.12:0/3203694150 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c018620 con 0x7fa84c108a50 2024-04-22T13:23:25.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.239+0000 7fa851719700 1 -- 172.21.15.12:0/3203694150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa84c19e380 con 0x7fa84c108a50 2024-04-22T13:23:25.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.239+0000 7fa851719700 1 -- 172.21.15.12:0/3203694150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa84c1a5c00 con 0x7fa84c108a50 2024-04-22T13:23:25.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.240+0000 7fa851719700 1 -- 172.21.15.12:0/3203694150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa84c069730 con 0x7fa84c108a50 2024-04-22T13:23:25.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.241+0000 7fa843fff700 1 -- 172.21.15.12:0/3203694150 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa83c007a70 con 0x7fa84c108a50 2024-04-22T13:23:25.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.242+0000 7fa843fff700 1 --2- 172.21.15.12:0/3203694150 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa8380a3cd0 0x7fa8380a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:25.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.242+0000 7fa843fff700 1 -- 172.21.15.12:0/3203694150 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fa83c0d0380 con 0x7fa84c108a50 2024-04-22T13:23:25.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.242+0000 7fa84affd700 1 --2- 172.21.15.12:0/3203694150 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa8380a3cd0 0x7fa8380a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:25.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.243+0000 7fa84affd700 1 --2- 172.21.15.12:0/3203694150 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa8380a3cd0 0x7fa8380a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa844009fd0 tx=0x7fa84400b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:25.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.247+0000 7fa843fff700 1 -- 172.21.15.12:0/3203694150 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa83c014030 con 0x7fa84c108a50 2024-04-22T13:23:25.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.453+0000 7fa851719700 1 -- 172.21.15.12:0/3203694150 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa84c062e80 con 0x7fa8380a3cd0 2024-04-22T13:23:25.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.463+0000 7fa843fff700 1 -- 172.21.15.12:0/3203694150 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258760 (secure 0 0 0) 0x7fa84c062e80 con 0x7fa8380a3cd0 2024-04-22T13:23:25.463 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:23:25.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.466+0000 7fa851719700 1 -- 172.21.15.12:0/3203694150 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa8380a3cd0 msgr2=0x7fa8380a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:25.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.466+0000 7fa851719700 1 --2- 172.21.15.12:0/3203694150 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa8380a3cd0 0x7fa8380a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa844009fd0 tx=0x7fa84400b7b0 comp rx=0 tx=0).stop 2024-04-22T13:23:25.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.466+0000 7fa851719700 1 -- 172.21.15.12:0/3203694150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa84c108a50 msgr2=0x7fa84c19d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:25.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.466+0000 7fa851719700 1 --2- 172.21.15.12:0/3203694150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa84c108a50 0x7fa84c19d950 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa83c00c900 tx=0x7fa83c00cc10 comp rx=0 tx=0).stop 2024-04-22T13:23:25.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.466+0000 7fa851719700 1 -- 172.21.15.12:0/3203694150 shutdown_connections 2024-04-22T13:23:25.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.466+0000 7fa851719700 1 --2- 172.21.15.12:0/3203694150 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa8380a3cd0 0x7fa8380a6190 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:25.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.466+0000 7fa851719700 1 --2- 172.21.15.12:0/3203694150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa84c108a50 0x7fa84c19d950 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:25.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.466+0000 7fa851719700 1 --2- 172.21.15.12:0/3203694150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa84c102a90 0x7fa84c1990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:25.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.466+0000 7fa851719700 1 --2- 172.21.15.12:0/3203694150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa84c103450 0x7fa84c199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:25.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.466+0000 7fa851719700 1 -- 172.21.15.12:0/3203694150 >> 172.21.15.12:0/3203694150 conn(0x7fa84c079ba0 msgr2=0x7fa84c1016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:25.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.466+0000 7fa851719700 1 -- 172.21.15.12:0/3203694150 shutdown_connections 2024-04-22T13:23:25.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:25.466+0000 7fa851719700 1 -- 172.21.15.12:0/3203694150 wait complete. 2024-04-22T13:23:25.465 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-22T13:23:25.815 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:25 smithi012 ceph-mon[28265]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-22T13:23:25.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:25 smithi134 ceph-mon[31338]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-22T13:23:25.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:25 smithi111 ceph-mon[31361]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-22T13:23:26.171 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2024-04-22T13:23:24.381394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656880,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":885,"num_read_kb":566,"num_write":418,"num_write_kb":3071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":343,"ondisk_log_size":343,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375212,"kb_used_data":13500,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566612,"statfs":{"total":1151940427776,"available":1151556210688,"internally_reserved":0,"allocated":13824000,"data_stored":7514280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":125,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003047"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.506048+0000","last_change":"2024-04-22T13:22:05.254388+0000","last_active":"2024-04-22T13:23:05.506048+0000","last_peered":"2024-04-22T13:23:05.506048+0000","last_clean":"2024-04-22T13:23:05.506048+0000","last_became_active":"2024-04-22T13:22:05.254042+0000","last_became_peered":"2024-04-22T13:22:05.254042+0000","last_unstale":"2024-04-22T13:23:05.506048+0000","last_undegraded":"2024-04-22T13:23:05.506048+0000","last_fullsized":"2024-04-22T13:23:05.506048+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:07:56.139653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508850+0000","last_change":"2024-04-22T13:22:05.247879+0000","last_active":"2024-04-22T13:23:05.508850+0000","last_peered":"2024-04-22T13:23:05.508850+0000","last_clean":"2024-04-22T13:23:05.508850+0000","last_became_active":"2024-04-22T13:22:05.247508+0000","last_became_peered":"2024-04-22T13:22:05.247508+0000","last_unstale":"2024-04-22T13:23:05.508850+0000","last_undegraded":"2024-04-22T13:23:05.508850+0000","last_fullsized":"2024-04-22T13:23:05.508850+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:55:51.928658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508776+0000","last_change":"2024-04-22T13:22:05.257970+0000","last_active":"2024-04-22T13:23:05.508776+0000","last_peered":"2024-04-22T13:23:05.508776+0000","last_clean":"2024-04-22T13:23:05.508776+0000","last_became_active":"2024-04-22T13:22:05.257652+0000","last_became_peered":"2024-04-22T13:22:05.257652+0000","last_unstale":"2024-04-22T13:23:05.508776+0000","last_undegraded":"2024-04-22T13:23:05.508776+0000","last_fullsized":"2024-04-22T13:23:05.508776+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:42:17.476349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.990505+0000","last_change":"2024-04-22T13:22:05.249186+0000","last_active":"2024-04-22T13:23:05.990505+0000","last_peered":"2024-04-22T13:23:05.990505+0000","last_clean":"2024-04-22T13:23:05.990505+0000","last_became_active":"2024-04-22T13:22:05.248814+0000","last_became_peered":"2024-04-22T13:22:05.248814+0000","last_unstale":"2024-04-22T13:23:05.990505+0000","last_undegraded":"2024-04-22T13:23:05.990505+0000","last_fullsized":"2024-04-22T13:23:05.990505+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:57.450794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508196+0000","last_change":"2024-04-22T13:22:05.251953+0000","last_active":"2024-04-22T13:23:05.508196+0000","last_peered":"2024-04-22T13:23:05.508196+0000","last_clean":"2024-04-22T13:23:05.508196+0000","last_became_active":"2024-04-22T13:22:05.251547+0000","last_became_peered":"2024-04-22T13:22:05.251547+0000","last_unstale":"2024-04-22T13:23:05.508196+0000","last_undegraded":"2024-04-22T13:23:05.508196+0000","last_fullsized":"2024-04-22T13:23:05.508196+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:36:34.809878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.510044+0000","last_change":"2024-04-22T13:23:05.510044+0000","last_active":"2024-04-22T13:23:05.510044+0000","last_peered":"2024-04-22T13:23:05.510044+0000","last_clean":"2024-04-22T13:23:05.510044+0000","last_became_active":"2024-04-22T13:23:05.509677+0000","last_became_peered":"2024-04-22T13:23:05.509677+0000","last_unstale":"2024-04-22T13:23:05.510044+0000","last_undegraded":"2024-04-22T13:23:05.510044+0000","last_fullsized":"2024-04-22T13:23:05.510044+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:46.979291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508380+0000","last_change":"2024-04-22T13:22:05.250418+0000","last_active":"2024-04-22T13:23:05.508380+0000","last_peered":"2024-04-22T13:23:05.508380+0000","last_clean":"2024-04-22T13:23:05.508380+0000","last_became_active":"2024-04-22T13:22:05.250097+0000","last_became_peered":"2024-04-22T13:22:05.250097+0000","last_unstale":"2024-04-22T13:23:05.508380+0000","last_undegraded":"2024-04-22T13:23:05.508380+0000","last_fullsized":"2024-04-22T13:23:05.508380+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:34:59.598950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.990304+0000","last_change":"2024-04-22T13:22:05.244762+0000","last_active":"2024-04-22T13:23:05.990304+0000","last_peered":"2024-04-22T13:23:05.990304+0000","last_clean":"2024-04-22T13:23:05.990304+0000","last_became_active":"2024-04-22T13:22:05.244430+0000","last_became_peered":"2024-04-22T13:22:05.244430+0000","last_unstale":"2024-04-22T13:23:05.990304+0000","last_undegraded":"2024-04-22T13:23:05.990304+0000","last_fullsized":"2024-04-22T13:23:05.990304+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:33:01.582851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.506546+0000","last_change":"2024-04-22T13:22:05.257596+0000","last_active":"2024-04-22T13:23:05.506546+0000","last_peered":"2024-04-22T13:23:05.506546+0000","last_clean":"2024-04-22T13:23:05.506546+0000","last_became_active":"2024-04-22T13:22:05.257277+0000","last_became_peered":"2024-04-22T13:22:05.257277+0000","last_unstale":"2024-04-22T13:23:05.506546+0000","last_undegraded":"2024-04-22T13:23:05.506546+0000","last_fullsized":"2024-04-22T13:23:05.506546+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:09:44.715965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.507131+0000","last_change":"2024-04-22T13:22:05.252177+0000","last_active":"2024-04-22T13:23:05.507131+0000","last_peered":"2024-04-22T13:23:05.507131+0000","last_clean":"2024-04-22T13:23:05.507131+0000","last_became_active":"2024-04-22T13:22:05.251794+0000","last_became_peered":"2024-04-22T13:22:05.251794+0000","last_unstale":"2024-04-22T13:23:05.507131+0000","last_undegraded":"2024-04-22T13:23:05.507131+0000","last_fullsized":"2024-04-22T13:23:05.507131+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:08:59.500540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508555+0000","last_change":"2024-04-22T13:22:05.256209+0000","last_active":"2024-04-22T13:23:05.508555+0000","last_peered":"2024-04-22T13:23:05.508555+0000","last_clean":"2024-04-22T13:23:05.508555+0000","last_became_active":"2024-04-22T13:22:05.255504+0000","last_became_peered":"2024-04-22T13:22:05.255504+0000","last_unstale":"2024-04-22T13:23:05.508555+0000","last_undegraded":"2024-04-22T13:23:05.508555+0000","last_fullsized":"2024-04-22T13:23:05.508555+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:32:08.190921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.509837+0000","last_change":"2024-04-22T13:23:05.509837+0000","last_active":"2024-04-22T13:23:05.509837+0000","last_peered":"2024-04-22T13:23:05.509837+0000","last_clean":"2024-04-22T13:23:05.509837+0000","last_became_active":"2024-04-22T13:23:05.509421+0000","last_became_peered":"2024-04-22T13:23:05.509421+0000","last_unstale":"2024-04-22T13:23:05.509837+0000","last_undegraded":"2024-04-22T13:23:05.509837+0000","last_fullsized":"2024-04-22T13:23:05.509837+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:30:01.103373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.863247+0000","last_change":"2024-04-22T13:22:05.246667+0000","last_active":"2024-04-22T13:23:05.863247+0000","last_peered":"2024-04-22T13:23:05.863247+0000","last_clean":"2024-04-22T13:23:05.863247+0000","last_became_active":"2024-04-22T13:22:05.246365+0000","last_became_peered":"2024-04-22T13:22:05.246365+0000","last_unstale":"2024-04-22T13:23:05.863247+0000","last_undegraded":"2024-04-22T13:23:05.863247+0000","last_fullsized":"2024-04-22T13:23:05.863247+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:41:55.533430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508257+0000","last_change":"2024-04-22T13:22:05.251982+0000","last_active":"2024-04-22T13:23:05.508257+0000","last_peered":"2024-04-22T13:23:05.508257+0000","last_clean":"2024-04-22T13:23:05.508257+0000","last_became_active":"2024-04-22T13:22:05.251575+0000","last_became_peered":"2024-04-22T13:22:05.251575+0000","last_unstale":"2024-04-22T13:23:05.508257+0000","last_undegraded":"2024-04-22T13:23:05.508257+0000","last_fullsized":"2024-04-22T13:23:05.508257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:28:05.343140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508282+0000","last_change":"2024-04-22T13:22:05.251056+0000","last_active":"2024-04-22T13:23:05.508282+0000","last_peered":"2024-04-22T13:23:05.508282+0000","last_clean":"2024-04-22T13:23:05.508282+0000","last_became_active":"2024-04-22T13:22:05.250652+0000","last_became_peered":"2024-04-22T13:22:05.250652+0000","last_unstale":"2024-04-22T13:23:05.508282+0000","last_undegraded":"2024-04-22T13:23:05.508282+0000","last_fullsized":"2024-04-22T13:23:05.508282+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:08:28.629965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.505896+0000","last_change":"2024-04-22T13:22:05.256877+0000","last_active":"2024-04-22T13:23:05.505896+0000","last_peered":"2024-04-22T13:23:05.505896+0000","last_clean":"2024-04-22T13:23:05.505896+0000","last_became_active":"2024-04-22T13:22:05.256074+0000","last_became_peered":"2024-04-22T13:22:05.256074+0000","last_unstale":"2024-04-22T13:23:05.505896+0000","last_undegraded":"2024-04-22T13:23:05.505896+0000","last_fullsized":"2024-04-22T13:23:05.505896+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:08:53.607626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.506174+0000","last_change":"2024-04-22T13:22:05.255598+0000","last_active":"2024-04-22T13:23:05.506174+0000","last_peered":"2024-04-22T13:23:05.506174+0000","last_clean":"2024-04-22T13:23:05.506174+0000","last_became_active":"2024-04-22T13:22:05.255239+0000","last_became_peered":"2024-04-22T13:22:05.255239+0000","last_unstale":"2024-04-22T13:23:05.506174+0000","last_undegraded":"2024-04-22T13:23:05.506174+0000","last_fullsized":"2024-04-22T13:23:05.506174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:38:20.547882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.505358+0000","last_change":"2024-04-22T13:22:05.254232+0000","last_active":"2024-04-22T13:23:05.505358+0000","last_peered":"2024-04-22T13:23:05.505358+0000","last_clean":"2024-04-22T13:23:05.505358+0000","last_became_active":"2024-04-22T13:22:05.253907+0000","last_became_peered":"2024-04-22T13:22:05.253907+0000","last_unstale":"2024-04-22T13:23:05.505358+0000","last_undegraded":"2024-04-22T13:23:05.505358+0000","last_fullsized":"2024-04-22T13:23:05.505358+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:15:49.931112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.504381+0000","last_change":"2024-04-22T13:22:05.251518+0000","last_active":"2024-04-22T13:23:05.504381+0000","last_peered":"2024-04-22T13:23:05.504381+0000","last_clean":"2024-04-22T13:23:05.504381+0000","last_became_active":"2024-04-22T13:22:05.250478+0000","last_became_peered":"2024-04-22T13:22:05.250478+0000","last_unstale":"2024-04-22T13:23:05.504381+0000","last_undegraded":"2024-04-22T13:23:05.504381+0000","last_fullsized":"2024-04-22T13:23:05.504381+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:25:26.758692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":101,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:18.496965+0000","last_change":"2024-04-22T13:22:04.241365+0000","last_active":"2024-04-22T13:23:18.496965+0000","last_peered":"2024-04-22T13:23:18.496965+0000","last_clean":"2024-04-22T13:23:18.496965+0000","last_became_active":"2024-04-22T13:22:04.240822+0000","last_became_peered":"2024-04-22T13:22:04.240822+0000","last_unstale":"2024-04-22T13:23:18.496965+0000","last_undegraded":"2024-04-22T13:23:18.496965+0000","last_fullsized":"2024-04-22T13:23:18.496965+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:20:23.928534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.876262+0000","last_change":"2024-04-22T13:23:05.876262+0000","last_active":"2024-04-22T13:23:05.876262+0000","last_peered":"2024-04-22T13:23:05.876262+0000","last_clean":"2024-04-22T13:23:05.876262+0000","last_became_active":"2024-04-22T13:23:05.875872+0000","last_became_peered":"2024-04-22T13:23:05.875872+0000","last_unstale":"2024-04-22T13:23:05.876262+0000","last_undegraded":"2024-04-22T13:23:05.876262+0000","last_fullsized":"2024-04-22T13:23:05.876262+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:29:33.092887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.990905+0000","last_change":"2024-04-22T13:22:04.248872+0000","last_active":"2024-04-22T13:23:05.990905+0000","last_peered":"2024-04-22T13:23:05.990905+0000","last_clean":"2024-04-22T13:23:05.990905+0000","last_became_active":"2024-04-22T13:22:04.248466+0000","last_became_peered":"2024-04-22T13:22:04.248466+0000","last_unstale":"2024-04-22T13:23:05.990905+0000","last_undegraded":"2024-04-22T13:23:05.990905+0000","last_fullsized":"2024-04-22T13:23:05.990905+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:38:10.871602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.509167+0000","last_change":"2024-04-22T13:22:05.258005+0000","last_active":"2024-04-22T13:23:05.509167+0000","last_peered":"2024-04-22T13:23:05.509167+0000","last_clean":"2024-04-22T13:23:05.509167+0000","last_became_active":"2024-04-22T13:22:05.257615+0000","last_became_peered":"2024-04-22T13:22:05.257615+0000","last_unstale":"2024-04-22T13:23:05.509167+0000","last_undegraded":"2024-04-22T13:23:05.509167+0000","last_fullsized":"2024-04-22T13:23:05.509167+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:21:18.030530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508625+0000","last_change":"2024-04-22T13:22:04.246273+0000","last_active":"2024-04-22T13:23:05.508625+0000","last_peered":"2024-04-22T13:23:05.508625+0000","last_clean":"2024-04-22T13:23:05.508625+0000","last_became_active":"2024-04-22T13:22:04.245969+0000","last_became_peered":"2024-04-22T13:22:04.245969+0000","last_unstale":"2024-04-22T13:23:05.508625+0000","last_undegraded":"2024-04-22T13:23:05.508625+0000","last_fullsized":"2024-04-22T13:23:05.508625+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:10.260192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.506145+0000","last_change":"2024-04-22T13:22:05.260299+0000","last_active":"2024-04-22T13:23:05.506145+0000","last_peered":"2024-04-22T13:23:05.506145+0000","last_clean":"2024-04-22T13:23:05.506145+0000","last_became_active":"2024-04-22T13:22:05.259634+0000","last_became_peered":"2024-04-22T13:22:05.259634+0000","last_unstale":"2024-04-22T13:23:05.506145+0000","last_undegraded":"2024-04-22T13:23:05.506145+0000","last_fullsized":"2024-04-22T13:23:05.506145+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:16:44.421652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508504+0000","last_change":"2024-04-22T13:22:04.243843+0000","last_active":"2024-04-22T13:23:05.508504+0000","last_peered":"2024-04-22T13:23:05.508504+0000","last_clean":"2024-04-22T13:23:05.508504+0000","last_became_active":"2024-04-22T13:22:04.243465+0000","last_became_peered":"2024-04-22T13:22:04.243465+0000","last_unstale":"2024-04-22T13:23:05.508504+0000","last_undegraded":"2024-04-22T13:23:05.508504+0000","last_fullsized":"2024-04-22T13:23:05.508504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:48:36.431697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.874474+0000","last_change":"2024-04-22T13:22:05.259789+0000","last_active":"2024-04-22T13:23:05.874474+0000","last_peered":"2024-04-22T13:23:05.874474+0000","last_clean":"2024-04-22T13:23:05.874474+0000","last_became_active":"2024-04-22T13:22:05.259395+0000","last_became_peered":"2024-04-22T13:22:05.259395+0000","last_unstale":"2024-04-22T13:23:05.874474+0000","last_undegraded":"2024-04-22T13:23:05.874474+0000","last_fullsized":"2024-04-22T13:23:05.874474+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:43:51.430561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'5","reported_seq":100,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:21.473099+0000","last_change":"2024-04-22T13:22:04.240615+0000","last_active":"2024-04-22T13:23:21.473099+0000","last_peered":"2024-04-22T13:23:21.473099+0000","last_clean":"2024-04-22T13:23:21.473099+0000","last_became_active":"2024-04-22T13:22:04.240248+0000","last_became_peered":"2024-04-22T13:22:04.240248+0000","last_unstale":"2024-04-22T13:23:21.473099+0000","last_undegraded":"2024-04-22T13:23:21.473099+0000","last_fullsized":"2024-04-22T13:23:21.473099+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:55:05.948979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.507615+0000","last_change":"2024-04-22T13:22:05.253788+0000","last_active":"2024-04-22T13:23:05.507615+0000","last_peered":"2024-04-22T13:23:05.507615+0000","last_clean":"2024-04-22T13:23:05.507615+0000","last_became_active":"2024-04-22T13:22:05.252578+0000","last_became_peered":"2024-04-22T13:22:05.252578+0000","last_unstale":"2024-04-22T13:23:05.507615+0000","last_undegraded":"2024-04-22T13:23:05.507615+0000","last_fullsized":"2024-04-22T13:23:05.507615+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:04:11.769930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.507141+0000","last_change":"2024-04-22T13:22:04.241170+0000","last_active":"2024-04-22T13:23:05.507141+0000","last_peered":"2024-04-22T13:23:05.507141+0000","last_clean":"2024-04-22T13:23:05.507141+0000","last_became_active":"2024-04-22T13:22:04.240830+0000","last_became_peered":"2024-04-22T13:22:04.240830+0000","last_unstale":"2024-04-22T13:23:05.507141+0000","last_undegraded":"2024-04-22T13:23:05.507141+0000","last_fullsized":"2024-04-22T13:23:05.507141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:51:30.520613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.874397+0000","last_change":"2024-04-22T13:22:05.237900+0000","last_active":"2024-04-22T13:23:05.874397+0000","last_peered":"2024-04-22T13:23:05.874397+0000","last_clean":"2024-04-22T13:23:05.874397+0000","last_became_active":"2024-04-22T13:22:05.237507+0000","last_became_peered":"2024-04-22T13:22:05.237507+0000","last_unstale":"2024-04-22T13:23:05.874397+0000","last_undegraded":"2024-04-22T13:23:05.874397+0000","last_fullsized":"2024-04-22T13:23:05.874397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:26:24.307803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.863158+0000","last_change":"2024-04-22T13:22:04.236078+0000","last_active":"2024-04-22T13:23:05.863158+0000","last_peered":"2024-04-22T13:23:05.863158+0000","last_clean":"2024-04-22T13:23:05.863158+0000","last_became_active":"2024-04-22T13:22:04.235710+0000","last_became_peered":"2024-04-22T13:22:04.235710+0000","last_unstale":"2024-04-22T13:23:05.863158+0000","last_undegraded":"2024-04-22T13:23:05.863158+0000","last_fullsized":"2024-04-22T13:23:05.863158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:37:55.335237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.990253+0000","last_change":"2024-04-22T13:22:05.243152+0000","last_active":"2024-04-22T13:23:05.990253+0000","last_peered":"2024-04-22T13:23:05.990253+0000","last_clean":"2024-04-22T13:23:05.990253+0000","last_became_active":"2024-04-22T13:22:05.242772+0000","last_became_peered":"2024-04-22T13:22:05.242772+0000","last_unstale":"2024-04-22T13:23:05.990253+0000","last_undegraded":"2024-04-22T13:23:05.990253+0000","last_fullsized":"2024-04-22T13:23:05.990253+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:33:42.359679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.874282+0000","last_change":"2024-04-22T13:22:04.244381+0000","last_active":"2024-04-22T13:23:05.874282+0000","last_peered":"2024-04-22T13:23:05.874282+0000","last_clean":"2024-04-22T13:23:05.874282+0000","last_became_active":"2024-04-22T13:22:04.243746+0000","last_became_peered":"2024-04-22T13:22:04.243746+0000","last_unstale":"2024-04-22T13:23:05.874282+0000","last_undegraded":"2024-04-22T13:23:05.874282+0000","last_fullsized":"2024-04-22T13:23:05.874282+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:15:36.019930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.505047+0000","last_change":"2024-04-22T13:22:05.245394+0000","last_active":"2024-04-22T13:23:05.505047+0000","last_peered":"2024-04-22T13:23:05.505047+0000","last_clean":"2024-04-22T13:23:05.505047+0000","last_became_active":"2024-04-22T13:22:05.243931+0000","last_became_peered":"2024-04-22T13:22:05.243931+0000","last_unstale":"2024-04-22T13:23:05.505047+0000","last_undegraded":"2024-04-22T13:23:05.505047+0000","last_fullsized":"2024-04-22T13:23:05.505047+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:11:10.790842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":170,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:23.475754+0000","last_change":"2024-04-22T13:22:04.242583+0000","last_active":"2024-04-22T13:23:23.475754+0000","last_peered":"2024-04-22T13:23:23.475754+0000","last_clean":"2024-04-22T13:23:23.475754+0000","last_became_active":"2024-04-22T13:22:04.242261+0000","last_became_peered":"2024-04-22T13:22:04.242261+0000","last_unstale":"2024-04-22T13:23:23.475754+0000","last_undegraded":"2024-04-22T13:23:23.475754+0000","last_fullsized":"2024-04-22T13:23:23.475754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:37:47.618573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":147,"num_read_kb":147,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.506448+0000","last_change":"2024-04-22T13:22:05.255293+0000","last_active":"2024-04-22T13:23:05.506448+0000","last_peered":"2024-04-22T13:23:05.506448+0000","last_clean":"2024-04-22T13:23:05.506448+0000","last_became_active":"2024-04-22T13:22:05.253949+0000","last_became_peered":"2024-04-22T13:22:05.253949+0000","last_unstale":"2024-04-22T13:23:05.506448+0000","last_undegraded":"2024-04-22T13:23:05.506448+0000","last_fullsized":"2024-04-22T13:23:05.506448+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:41:08.754989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508720+0000","last_change":"2024-04-22T13:22:04.241529+0000","last_active":"2024-04-22T13:23:05.508720+0000","last_peered":"2024-04-22T13:23:05.508720+0000","last_clean":"2024-04-22T13:23:05.508720+0000","last_became_active":"2024-04-22T13:22:04.241164+0000","last_became_peered":"2024-04-22T13:22:04.241164+0000","last_unstale":"2024-04-22T13:23:05.508720+0000","last_undegraded":"2024-04-22T13:23:05.508720+0000","last_fullsized":"2024-04-22T13:23:05.508720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:16:49.387181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.509076+0000","last_change":"2024-04-22T13:22:05.241623+0000","last_active":"2024-04-22T13:23:05.509076+0000","last_peered":"2024-04-22T13:23:05.509076+0000","last_clean":"2024-04-22T13:23:05.509076+0000","last_became_active":"2024-04-22T13:22:05.241150+0000","last_became_peered":"2024-04-22T13:22:05.241150+0000","last_unstale":"2024-04-22T13:23:05.509076+0000","last_undegraded":"2024-04-22T13:23:05.509076+0000","last_fullsized":"2024-04-22T13:23:05.509076+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:17:28.412366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.874565+0000","last_change":"2024-04-22T13:22:04.247852+0000","last_active":"2024-04-22T13:23:05.874565+0000","last_peered":"2024-04-22T13:23:05.874565+0000","last_clean":"2024-04-22T13:23:05.874565+0000","last_became_active":"2024-04-22T13:22:04.247550+0000","last_became_peered":"2024-04-22T13:22:04.247550+0000","last_unstale":"2024-04-22T13:23:05.874565+0000","last_undegraded":"2024-04-22T13:23:05.874565+0000","last_fullsized":"2024-04-22T13:23:05.874565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:55:01.928435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.506217+0000","last_change":"2024-04-22T13:22:05.260050+0000","last_active":"2024-04-22T13:23:05.506217+0000","last_peered":"2024-04-22T13:23:05.506217+0000","last_clean":"2024-04-22T13:23:05.506217+0000","last_became_active":"2024-04-22T13:22:05.259098+0000","last_became_peered":"2024-04-22T13:22:05.259098+0000","last_unstale":"2024-04-22T13:23:05.506217+0000","last_undegraded":"2024-04-22T13:23:05.506217+0000","last_fullsized":"2024-04-22T13:23:05.506217+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:05:06.245492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'29","reported_seq":206,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:23.475714+0000","last_change":"2024-04-22T13:22:04.240233+0000","last_active":"2024-04-22T13:23:23.475714+0000","last_peered":"2024-04-22T13:23:23.475714+0000","last_clean":"2024-04-22T13:23:23.475714+0000","last_became_active":"2024-04-22T13:22:04.239890+0000","last_became_peered":"2024-04-22T13:22:04.239890+0000","last_unstale":"2024-04-22T13:23:23.475714+0000","last_undegraded":"2024-04-22T13:23:23.475714+0000","last_fullsized":"2024-04-22T13:23:23.475714+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:04:21.547422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59465,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":158,"num_read_kb":137,"num_write":31,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508400+0000","last_change":"2024-04-22T13:22:05.257756+0000","last_active":"2024-04-22T13:23:05.508400+0000","last_peered":"2024-04-22T13:23:05.508400+0000","last_clean":"2024-04-22T13:23:05.508400+0000","last_became_active":"2024-04-22T13:22:05.257442+0000","last_became_peered":"2024-04-22T13:22:05.257442+0000","last_unstale":"2024-04-22T13:23:05.508400+0000","last_undegraded":"2024-04-22T13:23:05.508400+0000","last_fullsized":"2024-04-22T13:23:05.508400+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:16:53.465689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.504279+0000","last_change":"2024-04-22T13:22:04.240815+0000","last_active":"2024-04-22T13:23:05.504279+0000","last_peered":"2024-04-22T13:23:05.504279+0000","last_clean":"2024-04-22T13:23:05.504279+0000","last_became_active":"2024-04-22T13:22:04.240218+0000","last_became_peered":"2024-04-22T13:22:04.240218+0000","last_unstale":"2024-04-22T13:23:05.504279+0000","last_undegraded":"2024-04-22T13:23:05.504279+0000","last_fullsized":"2024-04-22T13:23:05.504279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:29:29.148764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.863069+0000","last_change":"2024-04-22T13:22:05.248797+0000","last_active":"2024-04-22T13:23:05.863069+0000","last_peered":"2024-04-22T13:23:05.863069+0000","last_clean":"2024-04-22T13:23:05.863069+0000","last_became_active":"2024-04-22T13:22:05.248435+0000","last_became_peered":"2024-04-22T13:22:05.248435+0000","last_unstale":"2024-04-22T13:23:05.863069+0000","last_undegraded":"2024-04-22T13:23:05.863069+0000","last_fullsized":"2024-04-22T13:23:05.863069+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:32:31.970494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.874115+0000","last_change":"2024-04-22T13:22:04.245303+0000","last_active":"2024-04-22T13:23:05.874115+0000","last_peered":"2024-04-22T13:23:05.874115+0000","last_clean":"2024-04-22T13:23:05.874115+0000","last_became_active":"2024-04-22T13:22:04.244903+0000","last_became_peered":"2024-04-22T13:22:04.244903+0000","last_unstale":"2024-04-22T13:23:05.874115+0000","last_undegraded":"2024-04-22T13:23:05.874115+0000","last_fullsized":"2024-04-22T13:23:05.874115+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:34:32.174858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508763+0000","last_change":"2024-04-22T13:22:05.246318+0000","last_active":"2024-04-22T13:23:05.508763+0000","last_peered":"2024-04-22T13:23:05.508763+0000","last_clean":"2024-04-22T13:23:05.508763+0000","last_became_active":"2024-04-22T13:22:05.245899+0000","last_became_peered":"2024-04-22T13:22:05.245899+0000","last_unstale":"2024-04-22T13:23:05.508763+0000","last_undegraded":"2024-04-22T13:23:05.508763+0000","last_fullsized":"2024-04-22T13:23:05.508763+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:27:42.789058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":96,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:19.510505+0000","last_change":"2024-04-22T13:22:04.240612+0000","last_active":"2024-04-22T13:23:19.510505+0000","last_peered":"2024-04-22T13:23:19.510505+0000","last_clean":"2024-04-22T13:23:19.510505+0000","last_became_active":"2024-04-22T13:22:04.240290+0000","last_became_peered":"2024-04-22T13:22:04.240290+0000","last_unstale":"2024-04-22T13:23:19.510505+0000","last_undegraded":"2024-04-22T13:23:19.510505+0000","last_fullsized":"2024-04-22T13:23:19.510505+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:00:50.285553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508228+0000","last_change":"2024-04-22T13:22:05.253205+0000","last_active":"2024-04-22T13:23:05.508228+0000","last_peered":"2024-04-22T13:23:05.508228+0000","last_clean":"2024-04-22T13:23:05.508228+0000","last_became_active":"2024-04-22T13:22:05.252865+0000","last_became_peered":"2024-04-22T13:22:05.252865+0000","last_unstale":"2024-04-22T13:23:05.508228+0000","last_undegraded":"2024-04-22T13:23:05.508228+0000","last_fullsized":"2024-04-22T13:23:05.508228+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:19:12.482371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.507514+0000","last_change":"2024-04-22T13:22:04.248938+0000","last_active":"2024-04-22T13:23:05.507514+0000","last_peered":"2024-04-22T13:23:05.507514+0000","last_clean":"2024-04-22T13:23:05.507514+0000","last_became_active":"2024-04-22T13:22:04.248440+0000","last_became_peered":"2024-04-22T13:22:04.248440+0000","last_unstale":"2024-04-22T13:23:05.507514+0000","last_undegraded":"2024-04-22T13:23:05.507514+0000","last_fullsized":"2024-04-22T13:23:05.507514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:46:50.928067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.874410+0000","last_change":"2024-04-22T13:22:05.254664+0000","last_active":"2024-04-22T13:23:05.874410+0000","last_peered":"2024-04-22T13:23:05.874410+0000","last_clean":"2024-04-22T13:23:05.874410+0000","last_became_active":"2024-04-22T13:22:05.254304+0000","last_became_peered":"2024-04-22T13:22:05.254304+0000","last_unstale":"2024-04-22T13:23:05.874410+0000","last_undegraded":"2024-04-22T13:23:05.874410+0000","last_fullsized":"2024-04-22T13:23:05.874410+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:58:49.345573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508830+0000","last_change":"2024-04-22T13:22:04.244128+0000","last_active":"2024-04-22T13:23:05.508830+0000","last_peered":"2024-04-22T13:23:05.508830+0000","last_clean":"2024-04-22T13:23:05.508830+0000","last_became_active":"2024-04-22T13:22:04.243738+0000","last_became_peered":"2024-04-22T13:22:04.243738+0000","last_unstale":"2024-04-22T13:23:05.508830+0000","last_undegraded":"2024-04-22T13:23:05.508830+0000","last_fullsized":"2024-04-22T13:23:05.508830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:23:52.220584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.505822+0000","last_change":"2024-04-22T13:22:05.250910+0000","last_active":"2024-04-22T13:23:05.505822+0000","last_peered":"2024-04-22T13:23:05.505822+0000","last_clean":"2024-04-22T13:23:05.505822+0000","last_became_active":"2024-04-22T13:22:05.250410+0000","last_became_peered":"2024-04-22T13:22:05.250410+0000","last_unstale":"2024-04-22T13:23:05.505822+0000","last_undegraded":"2024-04-22T13:23:05.505822+0000","last_fullsized":"2024-04-22T13:23:05.505822+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:04:25.492677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.504909+0000","last_change":"2024-04-22T13:22:04.245889+0000","last_active":"2024-04-22T13:23:05.504909+0000","last_peered":"2024-04-22T13:23:05.504909+0000","last_clean":"2024-04-22T13:23:05.504909+0000","last_became_active":"2024-04-22T13:22:04.245474+0000","last_became_peered":"2024-04-22T13:22:04.245474+0000","last_unstale":"2024-04-22T13:23:05.504909+0000","last_undegraded":"2024-04-22T13:23:05.504909+0000","last_fullsized":"2024-04-22T13:23:05.504909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:22:12.626950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.507250+0000","last_change":"2024-04-22T13:22:05.250978+0000","last_active":"2024-04-22T13:23:05.507250+0000","last_peered":"2024-04-22T13:23:05.507250+0000","last_clean":"2024-04-22T13:23:05.507250+0000","last_became_active":"2024-04-22T13:22:05.250463+0000","last_became_peered":"2024-04-22T13:22:05.250463+0000","last_unstale":"2024-04-22T13:23:05.507250+0000","last_undegraded":"2024-04-22T13:23:05.507250+0000","last_fullsized":"2024-04-22T13:23:05.507250+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:45:18.591072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":93,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:18.497555+0000","last_change":"2024-04-22T13:22:04.242503+0000","last_active":"2024-04-22T13:23:18.497555+0000","last_peered":"2024-04-22T13:23:18.497555+0000","last_clean":"2024-04-22T13:23:18.497555+0000","last_became_active":"2024-04-22T13:22:04.242143+0000","last_became_peered":"2024-04-22T13:22:04.242143+0000","last_unstale":"2024-04-22T13:23:18.497555+0000","last_undegraded":"2024-04-22T13:23:18.497555+0000","last_fullsized":"2024-04-22T13:23:18.497555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:40.236377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.507316+0000","last_change":"2024-04-22T13:22:05.252387+0000","last_active":"2024-04-22T13:23:05.507316+0000","last_peered":"2024-04-22T13:23:05.507316+0000","last_clean":"2024-04-22T13:23:05.507316+0000","last_became_active":"2024-04-22T13:22:05.252033+0000","last_became_peered":"2024-04-22T13:22:05.252033+0000","last_unstale":"2024-04-22T13:23:05.507316+0000","last_undegraded":"2024-04-22T13:23:05.507316+0000","last_fullsized":"2024-04-22T13:23:05.507316+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:55:01.791283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.874416+0000","last_change":"2024-04-22T13:22:05.248394+0000","last_active":"2024-04-22T13:23:05.874416+0000","last_peered":"2024-04-22T13:23:05.874416+0000","last_clean":"2024-04-22T13:23:05.874416+0000","last_became_active":"2024-04-22T13:22:05.248024+0000","last_became_peered":"2024-04-22T13:22:05.248024+0000","last_unstale":"2024-04-22T13:23:05.874416+0000","last_undegraded":"2024-04-22T13:23:05.874416+0000","last_fullsized":"2024-04-22T13:23:05.874416+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:49:36.556136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":92,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:17.506027+0000","last_change":"2024-04-22T13:22:04.247692+0000","last_active":"2024-04-22T13:23:17.506027+0000","last_peered":"2024-04-22T13:23:17.506027+0000","last_clean":"2024-04-22T13:23:17.506027+0000","last_became_active":"2024-04-22T13:22:04.247345+0000","last_became_peered":"2024-04-22T13:22:04.247345+0000","last_unstale":"2024-04-22T13:23:17.506027+0000","last_undegraded":"2024-04-22T13:23:17.506027+0000","last_fullsized":"2024-04-22T13:23:17.506027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:07:49.803248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.990908+0000","last_change":"2024-04-22T13:22:05.241022+0000","last_active":"2024-04-22T13:23:05.990908+0000","last_peered":"2024-04-22T13:23:05.990908+0000","last_clean":"2024-04-22T13:23:05.990908+0000","last_became_active":"2024-04-22T13:22:05.240635+0000","last_became_peered":"2024-04-22T13:22:05.240635+0000","last_unstale":"2024-04-22T13:23:05.990908+0000","last_undegraded":"2024-04-22T13:23:05.990908+0000","last_fullsized":"2024-04-22T13:23:05.990908+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:50:02.012260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.990537+0000","last_change":"2024-04-22T13:22:04.236817+0000","last_active":"2024-04-22T13:23:05.990537+0000","last_peered":"2024-04-22T13:23:05.990537+0000","last_clean":"2024-04-22T13:23:05.990537+0000","last_became_active":"2024-04-22T13:22:04.236484+0000","last_became_peered":"2024-04-22T13:22:04.236484+0000","last_unstale":"2024-04-22T13:23:05.990537+0000","last_undegraded":"2024-04-22T13:23:05.990537+0000","last_fullsized":"2024-04-22T13:23:05.990537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:27:41.053327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":354,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508269+0000","last_change":"2024-04-22T13:20:53.094123+0000","last_active":"2024-04-22T13:23:05.508269+0000","last_peered":"2024-04-22T13:23:05.508269+0000","last_clean":"2024-04-22T13:23:05.508269+0000","last_became_active":"2024-04-22T13:20:53.093664+0000","last_became_peered":"2024-04-22T13:20:53.093664+0000","last_unstale":"2024-04-22T13:23:05.508269+0000","last_undegraded":"2024-04-22T13:23:05.508269+0000","last_fullsized":"2024-04-22T13:23:05.508269+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:17:33.201786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:17:33.201786+0000","last_clean_scrub_stamp":"2024-04-22T13:17:33.201786+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:46:58.986399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.862908+0000","last_change":"2024-04-22T13:22:05.247766+0000","last_active":"2024-04-22T13:23:05.862908+0000","last_peered":"2024-04-22T13:23:05.862908+0000","last_clean":"2024-04-22T13:23:05.862908+0000","last_became_active":"2024-04-22T13:22:05.247362+0000","last_became_peered":"2024-04-22T13:22:05.247362+0000","last_unstale":"2024-04-22T13:23:05.862908+0000","last_undegraded":"2024-04-22T13:23:05.862908+0000","last_fullsized":"2024-04-22T13:23:05.862908+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:25:00.126638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508388+0000","last_change":"2024-04-22T13:22:04.243493+0000","last_active":"2024-04-22T13:23:05.508388+0000","last_peered":"2024-04-22T13:23:05.508388+0000","last_clean":"2024-04-22T13:23:05.508388+0000","last_became_active":"2024-04-22T13:22:04.243190+0000","last_became_peered":"2024-04-22T13:22:04.243190+0000","last_unstale":"2024-04-22T13:23:05.508388+0000","last_undegraded":"2024-04-22T13:23:05.508388+0000","last_fullsized":"2024-04-22T13:23:05.508388+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:15:09.025152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508498+0000","last_change":"2024-04-22T13:22:04.249222+0000","last_active":"2024-04-22T13:23:05.508498+0000","last_peered":"2024-04-22T13:23:05.508498+0000","last_clean":"2024-04-22T13:23:05.508498+0000","last_became_active":"2024-04-22T13:22:04.248926+0000","last_became_peered":"2024-04-22T13:22:04.248926+0000","last_unstale":"2024-04-22T13:23:05.508498+0000","last_undegraded":"2024-04-22T13:23:05.508498+0000","last_fullsized":"2024-04-22T13:23:05.508498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:33:50.168426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.863308+0000","last_change":"2024-04-22T13:22:05.248199+0000","last_active":"2024-04-22T13:23:05.863308+0000","last_peered":"2024-04-22T13:23:05.863308+0000","last_clean":"2024-04-22T13:23:05.863308+0000","last_became_active":"2024-04-22T13:22:05.247873+0000","last_became_peered":"2024-04-22T13:22:05.247873+0000","last_unstale":"2024-04-22T13:23:05.863308+0000","last_undegraded":"2024-04-22T13:23:05.863308+0000","last_fullsized":"2024-04-22T13:23:05.863308+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:40:29.353901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'12","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.505242+0000","last_change":"2024-04-22T13:22:04.243474+0000","last_active":"2024-04-22T13:23:05.505242+0000","last_peered":"2024-04-22T13:23:05.505242+0000","last_clean":"2024-04-22T13:23:05.505242+0000","last_became_active":"2024-04-22T13:22:04.243151+0000","last_became_peered":"2024-04-22T13:22:04.243151+0000","last_unstale":"2024-04-22T13:23:05.505242+0000","last_undegraded":"2024-04-22T13:23:05.505242+0000","last_fullsized":"2024-04-22T13:23:05.505242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:18:25.140715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.990761+0000","last_change":"2024-04-22T13:22:05.243120+0000","last_active":"2024-04-22T13:23:05.990761+0000","last_peered":"2024-04-22T13:23:05.990761+0000","last_clean":"2024-04-22T13:23:05.990761+0000","last_became_active":"2024-04-22T13:22:05.242726+0000","last_became_peered":"2024-04-22T13:22:05.242726+0000","last_unstale":"2024-04-22T13:23:05.990761+0000","last_undegraded":"2024-04-22T13:23:05.990761+0000","last_fullsized":"2024-04-22T13:23:05.990761+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:06:16.087884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.990760+0000","last_change":"2024-04-22T13:22:04.239672+0000","last_active":"2024-04-22T13:23:05.990760+0000","last_peered":"2024-04-22T13:23:05.990760+0000","last_clean":"2024-04-22T13:23:05.990760+0000","last_became_active":"2024-04-22T13:22:04.238776+0000","last_became_peered":"2024-04-22T13:22:04.238776+0000","last_unstale":"2024-04-22T13:23:05.990760+0000","last_undegraded":"2024-04-22T13:23:05.990760+0000","last_fullsized":"2024-04-22T13:23:05.990760+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:43:21.752547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508977+0000","last_change":"2024-04-22T13:22:05.242089+0000","last_active":"2024-04-22T13:23:05.508977+0000","last_peered":"2024-04-22T13:23:05.508977+0000","last_clean":"2024-04-22T13:23:05.508977+0000","last_became_active":"2024-04-22T13:22:05.241681+0000","last_became_peered":"2024-04-22T13:22:05.241681+0000","last_unstale":"2024-04-22T13:23:05.508977+0000","last_undegraded":"2024-04-22T13:23:05.508977+0000","last_fullsized":"2024-04-22T13:23:05.508977+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:58:49.906188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.513123+0000","last_change":"2024-04-22T13:23:05.513123+0000","last_active":"2024-04-22T13:23:05.513123+0000","last_peered":"2024-04-22T13:23:05.513123+0000","last_clean":"2024-04-22T13:23:05.513123+0000","last_became_active":"2024-04-22T13:23:05.509367+0000","last_became_peered":"2024-04-22T13:23:05.509367+0000","last_unstale":"2024-04-22T13:23:05.513123+0000","last_undegraded":"2024-04-22T13:23:05.513123+0000","last_fullsized":"2024-04-22T13:23:05.513123+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:28:41.770957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.507644+0000","last_change":"2024-04-22T13:22:05.250120+0000","last_active":"2024-04-22T13:23:05.507644+0000","last_peered":"2024-04-22T13:23:05.507644+0000","last_clean":"2024-04-22T13:23:05.507644+0000","last_became_active":"2024-04-22T13:22:05.247936+0000","last_became_peered":"2024-04-22T13:22:05.247936+0000","last_unstale":"2024-04-22T13:23:05.507644+0000","last_undegraded":"2024-04-22T13:23:05.507644+0000","last_fullsized":"2024-04-22T13:23:05.507644+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:27:38.437918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.990406+0000","last_change":"2024-04-22T13:22:04.244026+0000","last_active":"2024-04-22T13:23:05.990406+0000","last_peered":"2024-04-22T13:23:05.990406+0000","last_clean":"2024-04-22T13:23:05.990406+0000","last_became_active":"2024-04-22T13:22:04.243518+0000","last_became_peered":"2024-04-22T13:22:04.243518+0000","last_unstale":"2024-04-22T13:23:05.990406+0000","last_undegraded":"2024-04-22T13:23:05.990406+0000","last_fullsized":"2024-04-22T13:23:05.990406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:54:48.791874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.505929+0000","last_change":"2024-04-22T13:22:05.252674+0000","last_active":"2024-04-22T13:23:05.505929+0000","last_peered":"2024-04-22T13:23:05.505929+0000","last_clean":"2024-04-22T13:23:05.505929+0000","last_became_active":"2024-04-22T13:22:05.251087+0000","last_became_peered":"2024-04-22T13:22:05.251087+0000","last_unstale":"2024-04-22T13:23:05.505929+0000","last_undegraded":"2024-04-22T13:23:05.505929+0000","last_fullsized":"2024-04-22T13:23:05.505929+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:20:06.873412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.504220+0000","last_change":"2024-04-22T13:22:04.244409+0000","last_active":"2024-04-22T13:23:05.504220+0000","last_peered":"2024-04-22T13:23:05.504220+0000","last_clean":"2024-04-22T13:23:05.504220+0000","last_became_active":"2024-04-22T13:22:04.244104+0000","last_became_peered":"2024-04-22T13:22:04.244104+0000","last_unstale":"2024-04-22T13:23:05.504220+0000","last_undegraded":"2024-04-22T13:23:05.504220+0000","last_fullsized":"2024-04-22T13:23:05.504220+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:39:14.714755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.990777+0000","last_change":"2024-04-22T13:22:05.245730+0000","last_active":"2024-04-22T13:23:05.990777+0000","last_peered":"2024-04-22T13:23:05.990777+0000","last_clean":"2024-04-22T13:23:05.990777+0000","last_became_active":"2024-04-22T13:22:05.245366+0000","last_became_peered":"2024-04-22T13:22:05.245366+0000","last_unstale":"2024-04-22T13:23:05.990777+0000","last_undegraded":"2024-04-22T13:23:05.990777+0000","last_fullsized":"2024-04-22T13:23:05.990777+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:31.935258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.514632+0000","last_change":"2024-04-22T13:23:05.514197+0000","last_active":"2024-04-22T13:23:05.514632+0000","last_peered":"2024-04-22T13:23:05.514632+0000","last_clean":"2024-04-22T13:23:05.514632+0000","last_became_active":"2024-04-22T13:23:05.509431+0000","last_became_peered":"2024-04-22T13:23:05.509431+0000","last_unstale":"2024-04-22T13:23:05.514632+0000","last_undegraded":"2024-04-22T13:23:05.514632+0000","last_fullsized":"2024-04-22T13:23:05.514632+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:43:59.825567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508622+0000","last_change":"2024-04-22T13:22:05.242146+0000","last_active":"2024-04-22T13:23:05.508622+0000","last_peered":"2024-04-22T13:23:05.508622+0000","last_clean":"2024-04-22T13:23:05.508622+0000","last_became_active":"2024-04-22T13:22:05.241802+0000","last_became_peered":"2024-04-22T13:22:05.241802+0000","last_unstale":"2024-04-22T13:23:05.508622+0000","last_undegraded":"2024-04-22T13:23:05.508622+0000","last_fullsized":"2024-04-22T13:23:05.508622+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:49:49.002123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":96,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:20.500478+0000","last_change":"2024-04-22T13:22:04.248884+0000","last_active":"2024-04-22T13:23:20.500478+0000","last_peered":"2024-04-22T13:23:20.500478+0000","last_clean":"2024-04-22T13:23:20.500478+0000","last_became_active":"2024-04-22T13:22:04.248393+0000","last_became_peered":"2024-04-22T13:22:04.248393+0000","last_unstale":"2024-04-22T13:23:20.500478+0000","last_undegraded":"2024-04-22T13:23:20.500478+0000","last_fullsized":"2024-04-22T13:23:20.500478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:36:14.687457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508653+0000","last_change":"2024-04-22T13:22:05.256881+0000","last_active":"2024-04-22T13:23:05.508653+0000","last_peered":"2024-04-22T13:23:05.508653+0000","last_clean":"2024-04-22T13:23:05.508653+0000","last_became_active":"2024-04-22T13:22:05.255967+0000","last_became_peered":"2024-04-22T13:22:05.255967+0000","last_unstale":"2024-04-22T13:23:05.508653+0000","last_undegraded":"2024-04-22T13:23:05.508653+0000","last_fullsized":"2024-04-22T13:23:05.508653+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:24:21.137344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.874347+0000","last_change":"2024-04-22T13:22:04.247303+0000","last_active":"2024-04-22T13:23:05.874347+0000","last_peered":"2024-04-22T13:23:05.874347+0000","last_clean":"2024-04-22T13:23:05.874347+0000","last_became_active":"2024-04-22T13:22:04.246742+0000","last_became_peered":"2024-04-22T13:22:04.246742+0000","last_unstale":"2024-04-22T13:23:05.874347+0000","last_undegraded":"2024-04-22T13:23:05.874347+0000","last_fullsized":"2024-04-22T13:23:05.874347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:45:03.189042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'1","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508435+0000","last_change":"2024-04-22T13:22:05.251754+0000","last_active":"2024-04-22T13:23:05.508435+0000","last_peered":"2024-04-22T13:23:05.508435+0000","last_clean":"2024-04-22T13:23:05.508435+0000","last_became_active":"2024-04-22T13:22:05.251155+0000","last_became_peered":"2024-04-22T13:22:05.251155+0000","last_unstale":"2024-04-22T13:23:05.508435+0000","last_undegraded":"2024-04-22T13:23:05.508435+0000","last_fullsized":"2024-04-22T13:23:05.508435+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:08:46.682600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.863343+0000","last_change":"2024-04-22T13:22:04.243382+0000","last_active":"2024-04-22T13:23:05.863343+0000","last_peered":"2024-04-22T13:23:05.863343+0000","last_clean":"2024-04-22T13:23:05.863343+0000","last_became_active":"2024-04-22T13:22:04.243055+0000","last_became_peered":"2024-04-22T13:22:04.243055+0000","last_unstale":"2024-04-22T13:23:05.863343+0000","last_undegraded":"2024-04-22T13:23:05.863343+0000","last_fullsized":"2024-04-22T13:23:05.863343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:19:58.379645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508551+0000","last_change":"2024-04-22T13:22:05.249527+0000","last_active":"2024-04-22T13:23:05.508551+0000","last_peered":"2024-04-22T13:23:05.508551+0000","last_clean":"2024-04-22T13:23:05.508551+0000","last_became_active":"2024-04-22T13:22:05.249207+0000","last_became_peered":"2024-04-22T13:22:05.249207+0000","last_unstale":"2024-04-22T13:23:05.508551+0000","last_undegraded":"2024-04-22T13:23:05.508551+0000","last_fullsized":"2024-04-22T13:23:05.508551+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:50:02.755288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508324+0000","last_change":"2024-04-22T13:23:05.508324+0000","last_active":"2024-04-22T13:23:05.508324+0000","last_peered":"2024-04-22T13:23:05.508324+0000","last_clean":"2024-04-22T13:23:05.508324+0000","last_became_active":"2024-04-22T13:23:05.507814+0000","last_became_peered":"2024-04-22T13:23:05.507814+0000","last_unstale":"2024-04-22T13:23:05.508324+0000","last_undegraded":"2024-04-22T13:23:05.508324+0000","last_fullsized":"2024-04-22T13:23:05.508324+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:46:20.523715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.875683+0000","last_change":"2024-04-22T13:23:05.875683+0000","last_active":"2024-04-22T13:23:05.875683+0000","last_peered":"2024-04-22T13:23:05.875683+0000","last_clean":"2024-04-22T13:23:05.875683+0000","last_became_active":"2024-04-22T13:23:05.875150+0000","last_became_peered":"2024-04-22T13:23:05.875150+0000","last_unstale":"2024-04-22T13:23:05.875683+0000","last_undegraded":"2024-04-22T13:23:05.875683+0000","last_fullsized":"2024-04-22T13:23:05.875683+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:12:47.592230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.504119+0000","last_change":"2024-04-22T13:22:05.247882+0000","last_active":"2024-04-22T13:23:05.504119+0000","last_peered":"2024-04-22T13:23:05.504119+0000","last_clean":"2024-04-22T13:23:05.504119+0000","last_became_active":"2024-04-22T13:22:05.247525+0000","last_became_peered":"2024-04-22T13:22:05.247525+0000","last_unstale":"2024-04-22T13:23:05.504119+0000","last_undegraded":"2024-04-22T13:23:05.504119+0000","last_fullsized":"2024-04-22T13:23:05.504119+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:46:04.349076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.508413+0000","last_change":"2024-04-22T13:22:05.243716+0000","last_active":"2024-04-22T13:23:05.508413+0000","last_peered":"2024-04-22T13:23:05.508413+0000","last_clean":"2024-04-22T13:23:05.508413+0000","last_became_active":"2024-04-22T13:22:05.243397+0000","last_became_peered":"2024-04-22T13:22:05.243397+0000","last_unstale":"2024-04-22T13:23:05.508413+0000","last_undegraded":"2024-04-22T13:23:05.508413+0000","last_fullsized":"2024-04-22T13:23:05.508413+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:16:21.236339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.863393+0000","last_change":"2024-04-22T13:22:05.246042+0000","last_active":"2024-04-22T13:23:05.863393+0000","last_peered":"2024-04-22T13:23:05.863393+0000","last_clean":"2024-04-22T13:23:05.863393+0000","last_became_active":"2024-04-22T13:22:05.245669+0000","last_became_peered":"2024-04-22T13:22:05.245669+0000","last_unstale":"2024-04-22T13:23:05.863393+0000","last_undegraded":"2024-04-22T13:23:05.863393+0000","last_fullsized":"2024-04-22T13:23:05.863393+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:49:05.539641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.505796+0000","last_change":"2024-04-22T13:22:05.257009+0000","last_active":"2024-04-22T13:23:05.505796+0000","last_peered":"2024-04-22T13:23:05.505796+0000","last_clean":"2024-04-22T13:23:05.505796+0000","last_became_active":"2024-04-22T13:22:05.254327+0000","last_became_peered":"2024-04-22T13:22:05.254327+0000","last_unstale":"2024-04-22T13:23:05.505796+0000","last_undegraded":"2024-04-22T13:23:05.505796+0000","last_fullsized":"2024-04-22T13:23:05.505796+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:29:12.482525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.504704+0000","last_change":"2024-04-22T13:22:05.260575+0000","last_active":"2024-04-22T13:23:05.504704+0000","last_peered":"2024-04-22T13:23:05.504704+0000","last_clean":"2024-04-22T13:23:05.504704+0000","last_became_active":"2024-04-22T13:22:05.260181+0000","last_became_peered":"2024-04-22T13:22:05.260181+0000","last_unstale":"2024-04-22T13:23:05.504704+0000","last_undegraded":"2024-04-22T13:23:05.504704+0000","last_fullsized":"2024-04-22T13:23:05.504704+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:49:20.064368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.990636+0000","last_change":"2024-04-22T13:22:05.248199+0000","last_active":"2024-04-22T13:23:05.990636+0000","last_peered":"2024-04-22T13:23:05.990636+0000","last_clean":"2024-04-22T13:23:05.990636+0000","last_became_active":"2024-04-22T13:22:05.247858+0000","last_became_peered":"2024-04-22T13:22:05.247858+0000","last_unstale":"2024-04-22T13:23:05.990636+0000","last_undegraded":"2024-04-22T13:23:05.990636+0000","last_fullsized":"2024-04-22T13:23:05.990636+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:49:21.228105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.874325+0000","last_change":"2024-04-22T13:22:05.239137+0000","last_active":"2024-04-22T13:23:05.874325+0000","last_peered":"2024-04-22T13:23:05.874325+0000","last_clean":"2024-04-22T13:23:05.874325+0000","last_became_active":"2024-04-22T13:22:05.238807+0000","last_became_peered":"2024-04-22T13:22:05.238807+0000","last_unstale":"2024-04-22T13:23:05.874325+0000","last_undegraded":"2024-04-22T13:23:05.874325+0000","last_fullsized":"2024-04-22T13:23:05.874325+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:02:48.587097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.990356+0000","last_change":"2024-04-22T13:22:05.249080+0000","last_active":"2024-04-22T13:23:05.990356+0000","last_peered":"2024-04-22T13:23:05.990356+0000","last_clean":"2024-04-22T13:23:05.990356+0000","last_became_active":"2024-04-22T13:22:05.248733+0000","last_became_peered":"2024-04-22T13:22:05.248733+0000","last_unstale":"2024-04-22T13:23:05.990356+0000","last_undegraded":"2024-04-22T13:23:05.990356+0000","last_fullsized":"2024-04-22T13:23:05.990356+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:58.735941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.506059+0000","last_change":"2024-04-22T13:22:05.255490+0000","last_active":"2024-04-22T13:23:05.506059+0000","last_peered":"2024-04-22T13:23:05.506059+0000","last_clean":"2024-04-22T13:23:05.506059+0000","last_became_active":"2024-04-22T13:22:05.253644+0000","last_became_peered":"2024-04-22T13:22:05.253644+0000","last_unstale":"2024-04-22T13:23:05.506059+0000","last_undegraded":"2024-04-22T13:23:05.506059+0000","last_fullsized":"2024-04-22T13:23:05.506059+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:31:01.816182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.863180+0000","last_change":"2024-04-22T13:22:05.240988+0000","last_active":"2024-04-22T13:23:05.863180+0000","last_peered":"2024-04-22T13:23:05.863180+0000","last_clean":"2024-04-22T13:23:05.863180+0000","last_became_active":"2024-04-22T13:22:05.240642+0000","last_became_peered":"2024-04-22T13:22:05.240642+0000","last_unstale":"2024-04-22T13:23:05.863180+0000","last_undegraded":"2024-04-22T13:23:05.863180+0000","last_fullsized":"2024-04-22T13:23:05.863180+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:41:29.001979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-22T13:23:05.505682+0000","last_change":"2024-04-22T13:22:05.252212+0000","last_active":"2024-04-22T13:23:05.505682+0000","last_peered":"2024-04-22T13:23:05.505682+0000","last_clean":"2024-04-22T13:23:05.505682+0000","last_became_active":"2024-04-22T13:22:05.251852+0000","last_became_peered":"2024-04-22T13:22:05.251852+0000","last_unstale":"2024-04-22T13:23:05.505682+0000","last_undegraded":"2024-04-22T13:23:05.505682+0000","last_fullsized":"2024-04-22T13:23:05.505682+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:13:55.132501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66278,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":749,"num_read_kb":447,"num_write":235,"num_write_kb":131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":198834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318302,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":463162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579938,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1667072,"data_stored":1114047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743463,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":463092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005100,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":463470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234033,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32732,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712420,"statfs":{"total":95995035648,"available":95961518080,"internally_reserved":0,"allocated":1585152,"data_stored":1055988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430262,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":974848,"data_stored":462809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":523187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":464148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1056768,"data_stored":523407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117062,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":991232,"data_stored":462634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855425,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":463061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691899,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1597440,"data_stored":1055275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T13:23:26.175 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-22T13:23:26.175 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-22T13:23:26.175 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-22T13:23:26.175 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-22T13:23:26.424 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:23:26.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:26 smithi012 ceph-mon[28265]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:23:26.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:26 smithi134 ceph-mon[31338]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:23:26.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:26 smithi111 ceph-mon[31361]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:23:27.794 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:27 smithi012 ceph-mon[28265]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-22T13:23:27.964 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:27 smithi134 ceph-mon[31338]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-22T13:23:27.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:27 smithi111 ceph-mon[31361]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-22T13:23:28.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.025+0000 7faf9e548700 1 -- 172.21.15.12:0/3714798233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf98103450 msgr2=0x7faf9810f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:28.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.025+0000 7faf9e548700 1 --2- 172.21.15.12:0/3714798233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf98103450 0x7faf9810f8b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7faf9000b0a0 tx=0x7faf9000b3b0 comp rx=0 tx=0).stop 2024-04-22T13:23:28.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.025+0000 7faf9e548700 1 -- 172.21.15.12:0/3714798233 shutdown_connections 2024-04-22T13:23:28.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.025+0000 7faf9e548700 1 --2- 172.21.15.12:0/3714798233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf98108a50 0x7faf98108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:28.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.025+0000 7faf9e548700 1 --2- 172.21.15.12:0/3714798233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf98103450 0x7faf9810f8b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:28.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.025+0000 7faf9e548700 1 --2- 172.21.15.12:0/3714798233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf98102a90 0x7faf98102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:28.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.025+0000 7faf9e548700 1 -- 172.21.15.12:0/3714798233 >> 172.21.15.12:0/3714798233 conn(0x7faf98079ba0 msgr2=0x7faf98079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:28.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.025+0000 7faf9e548700 1 -- 172.21.15.12:0/3714798233 shutdown_connections 2024-04-22T13:23:28.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.025+0000 7faf9e548700 1 -- 172.21.15.12:0/3714798233 wait complete. 2024-04-22T13:23:28.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.026+0000 7faf9e548700 1 Processor -- start 2024-04-22T13:23:28.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.026+0000 7faf9e548700 1 -- start start 2024-04-22T13:23:28.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.026+0000 7faf9e548700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf98102a90 0x7faf98074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:28.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.026+0000 7faf9e548700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf98103450 0x7faf98075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:28.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.026+0000 7faf9e548700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf98108a50 0x7faf980757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:28.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.026+0000 7faf9e548700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7faf98112c20 con 0x7faf98102a90 2024-04-22T13:23:28.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.026+0000 7faf9e548700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faf98112aa0 con 0x7faf98103450 2024-04-22T13:23:28.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.026+0000 7faf9e548700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faf98112da0 con 0x7faf98108a50 2024-04-22T13:23:28.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.026+0000 7faf97fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf98102a90 0x7faf98074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:28.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.026+0000 7faf97fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf98102a90 0x7faf98074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47768/0 (socket says 172.21.15.12:47768) 2024-04-22T13:23:28.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.026+0000 7faf97fff700 1 -- 172.21.15.12:0/3057687230 learned_addr learned my addr 172.21.15.12:0/3057687230 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:28.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.026+0000 7faf9cae5700 1 --2- 172.21.15.12:0/3057687230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf98108a50 0x7faf980757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:28.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.027+0000 7faf977fe700 1 --2- 172.21.15.12:0/3057687230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf98103450 0x7faf98075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:28.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.027+0000 7faf97fff700 1 -- 172.21.15.12:0/3057687230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf98108a50 msgr2=0x7faf980757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:28.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.027+0000 7faf97fff700 1 --2- 172.21.15.12:0/3057687230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf98108a50 0x7faf980757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:28.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.027+0000 7faf97fff700 1 -- 172.21.15.12:0/3057687230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf98103450 msgr2=0x7faf98075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:28.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.027+0000 7faf97fff700 1 --2- 172.21.15.12:0/3057687230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf98103450 0x7faf98075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:28.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.027+0000 7faf97fff700 1 -- 172.21.15.12:0/3057687230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf981a7610 con 0x7faf98102a90 2024-04-22T13:23:28.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.027+0000 7faf977fe700 1 --2- 172.21.15.12:0/3057687230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf98103450 0x7faf98075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:23:28.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.027+0000 7faf97fff700 1 --2- 172.21.15.12:0/3057687230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf98102a90 0x7faf98074d40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7faf8800c9b0 tx=0x7faf8800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:28.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.027+0000 7faf957fa700 1 -- 172.21.15.12:0/3057687230 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf88007b00 con 0x7faf98102a90 2024-04-22T13:23:28.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.027+0000 7faf9e548700 1 -- 172.21.15.12:0/3057687230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf981a7900 con 0x7faf98102a90 2024-04-22T13:23:28.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.027+0000 7faf957fa700 1 -- 172.21.15.12:0/3057687230 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf88007ca0 con 0x7faf98102a90 2024-04-22T13:23:28.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.027+0000 7faf957fa700 1 -- 172.21.15.12:0/3057687230 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf88018660 con 0x7faf98102a90 2024-04-22T13:23:28.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.027+0000 7faf9e548700 1 -- 172.21.15.12:0/3057687230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf981a7e90 con 0x7faf98102a90 2024-04-22T13:23:28.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.028+0000 7faf9e548700 1 -- 172.21.15.12:0/3057687230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf98069730 con 0x7faf98102a90 2024-04-22T13:23:28.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.029+0000 7faf957fa700 1 -- 172.21.15.12:0/3057687230 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7faf8800f450 con 0x7faf98102a90 2024-04-22T13:23:28.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.030+0000 7faf957fa700 1 --2- 172.21.15.12:0/3057687230 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7faf800a3cd0 0x7faf800a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:28.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.030+0000 7faf977fe700 1 --2- 172.21.15.12:0/3057687230 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7faf800a3cd0 0x7faf800a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:28.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.030+0000 7faf957fa700 1 -- 172.21.15.12:0/3057687230 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7faf880d0780 con 0x7faf98102a90 2024-04-22T13:23:28.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.031+0000 7faf977fe700 1 --2- 172.21.15.12:0/3057687230 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7faf800a3cd0 0x7faf800a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7faf9807a1f0 tx=0x7faf8c00b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:28.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.034+0000 7faf957fa700 1 -- 172.21.15.12:0/3057687230 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faf88014030 con 0x7faf98102a90 2024-04-22T13:23:28.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.290+0000 7faf9e548700 1 -- 172.21.15.12:0/3057687230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7faf9802df80 con 0x7faf98102a90 2024-04-22T13:23:28.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.290+0000 7faf957fa700 1 -- 172.21.15.12:0/3057687230 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7faf8809d060 con 0x7faf98102a90 2024-04-22T13:23:28.287 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:23:28.287 INFO:teuthology.orchestra.run.smithi012.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-22T13:23:28.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.292+0000 7faf9e548700 1 -- 172.21.15.12:0/3057687230 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7faf800a3cd0 msgr2=0x7faf800a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:28.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.292+0000 7faf9e548700 1 --2- 172.21.15.12:0/3057687230 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7faf800a3cd0 0x7faf800a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7faf9807a1f0 tx=0x7faf8c00b2c0 comp rx=0 tx=0).stop 2024-04-22T13:23:28.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.292+0000 7faf9e548700 1 -- 172.21.15.12:0/3057687230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf98102a90 msgr2=0x7faf98074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:28.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.292+0000 7faf9e548700 1 --2- 172.21.15.12:0/3057687230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf98102a90 0x7faf98074d40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7faf8800c9b0 tx=0x7faf8800ccc0 comp rx=0 tx=0).stop 2024-04-22T13:23:28.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.293+0000 7faf9e548700 1 -- 172.21.15.12:0/3057687230 shutdown_connections 2024-04-22T13:23:28.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.293+0000 7faf9e548700 1 --2- 172.21.15.12:0/3057687230 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7faf800a3cd0 0x7faf800a6190 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:28.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.293+0000 7faf9e548700 1 --2- 172.21.15.12:0/3057687230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf98108a50 0x7faf980757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:28.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.293+0000 7faf9e548700 1 --2- 172.21.15.12:0/3057687230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faf98102a90 0x7faf98074d40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:28.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.293+0000 7faf9e548700 1 --2- 172.21.15.12:0/3057687230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf98103450 0x7faf98075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:28.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.293+0000 7faf9e548700 1 -- 172.21.15.12:0/3057687230 >> 172.21.15.12:0/3057687230 conn(0x7faf98079ba0 msgr2=0x7faf98101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:28.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.293+0000 7faf9e548700 1 -- 172.21.15.12:0/3057687230 shutdown_connections 2024-04-22T13:23:28.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:28.293+0000 7faf9e548700 1 -- 172.21.15.12:0/3057687230 wait complete. 2024-04-22T13:23:28.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:28 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/3057687230' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-22T13:23:28.947 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:28 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3057687230' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-22T13:23:28.976 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-22T13:23:28.976 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:23:28.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:28 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/3057687230' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-22T13:23:29.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.086+0000 7f128ddfd640 1 -- 172.21.15.12:0/1963884021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1288107ca0 msgr2=0x7f128810a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:29.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.086+0000 7f128ddfd640 1 --2- 172.21.15.12:0/1963884021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1288107ca0 0x7f128810a0f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1280009ca0 tx=0x7f128002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.086+0000 7f128ddfd640 1 -- 172.21.15.12:0/1963884021 shutdown_connections 2024-04-22T13:23:29.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.086+0000 7f128ddfd640 1 --2- 172.21.15.12:0/1963884021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1288107ca0 0x7f128810a0f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.086+0000 7f128ddfd640 1 --2- 172.21.15.12:0/1963884021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1288105370 0x7f1288107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.086+0000 7f128ddfd640 1 --2- 172.21.15.12:0/1963884021 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1288102a40 0x7f1288104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.086+0000 7f128ddfd640 1 -- 172.21.15.12:0/1963884021 >> 172.21.15.12:0/1963884021 conn(0x7f12880fc470 msgr2=0x7f12880fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:29.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.086+0000 7f128ddfd640 1 -- 172.21.15.12:0/1963884021 shutdown_connections 2024-04-22T13:23:29.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.086+0000 7f128ddfd640 1 -- 172.21.15.12:0/1963884021 wait complete. 2024-04-22T13:23:29.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.087+0000 7f128ddfd640 1 Processor -- start 2024-04-22T13:23:29.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.087+0000 7f128ddfd640 1 -- start start 2024-04-22T13:23:29.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.087+0000 7f128ddfd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1288102a40 0x7f1288198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:29.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.087+0000 7f128ddfd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1288105370 0x7f12881986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:29.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.087+0000 7f128ddfd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1288107ca0 0x7f128819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:29.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.087+0000 7f128ddfd640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f128810db60 con 0x7f1288105370 2024-04-22T13:23:29.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.087+0000 7f128ddfd640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f128810d9e0 con 0x7f1288107ca0 2024-04-22T13:23:29.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.087+0000 7f128ddfd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f128810dce0 con 0x7f1288102a40 2024-04-22T13:23:29.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.087+0000 7f1286ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1288105370 0x7f12881986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:29.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.087+0000 7f1286ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1288105370 0x7f12881986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47780/0 (socket says 172.21.15.12:47780) 2024-04-22T13:23:29.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.087+0000 7f1286ffd640 1 -- 172.21.15.12:0/374878575 learned_addr learned my addr 172.21.15.12:0/374878575 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:29.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.087+0000 7f1287fff640 1 --2- 172.21.15.12:0/374878575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1288107ca0 0x7f128819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:29.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.087+0000 7f12877fe640 1 --2- 172.21.15.12:0/374878575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1288102a40 0x7f1288198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:29.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.087+0000 7f1286ffd640 1 -- 172.21.15.12:0/374878575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1288102a40 msgr2=0x7f1288198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:29.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.088+0000 7f1286ffd640 1 --2- 172.21.15.12:0/374878575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1288102a40 0x7f1288198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.088+0000 7f1286ffd640 1 -- 172.21.15.12:0/374878575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1288107ca0 msgr2=0x7f128819f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:29.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.088+0000 7f1286ffd640 1 --2- 172.21.15.12:0/374878575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1288107ca0 0x7f128819f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.088+0000 7f1286ffd640 1 -- 172.21.15.12:0/374878575 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f128819fda0 con 0x7f1288105370 2024-04-22T13:23:29.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.088+0000 7f1287fff640 1 --2- 172.21.15.12:0/374878575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1288107ca0 0x7f128819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:23:29.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.088+0000 7f1286ffd640 1 --2- 172.21.15.12:0/374878575 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1288105370 0x7f12881986c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f127800b450 tx=0x7f127800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:29.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.088+0000 7f1284ff9640 1 -- 172.21.15.12:0/374878575 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1278015020 con 0x7f1288105370 2024-04-22T13:23:29.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.088+0000 7f128ddfd640 1 -- 172.21.15.12:0/374878575 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12881a0090 con 0x7f1288105370 2024-04-22T13:23:29.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.088+0000 7f1284ff9640 1 -- 172.21.15.12:0/374878575 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1278011d40 con 0x7f1288105370 2024-04-22T13:23:29.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.088+0000 7f1284ff9640 1 -- 172.21.15.12:0/374878575 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127800f340 con 0x7f1288105370 2024-04-22T13:23:29.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.088+0000 7f128ddfd640 1 -- 172.21.15.12:0/374878575 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12881a0590 con 0x7f1288105370 2024-04-22T13:23:29.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.089+0000 7f128ddfd640 1 -- 172.21.15.12:0/374878575 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12880fdf60 con 0x7f1288105370 2024-04-22T13:23:29.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.090+0000 7f1284ff9640 1 -- 172.21.15.12:0/374878575 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1278018020 con 0x7f1288105370 2024-04-22T13:23:29.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.091+0000 7f1284ff9640 1 --2- 172.21.15.12:0/374878575 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f12700a3c40 0x7f12700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:29.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.091+0000 7f12877fe640 1 --2- 172.21.15.12:0/374878575 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f12700a3c40 0x7f12700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:29.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.091+0000 7f1284ff9640 1 -- 172.21.15.12:0/374878575 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f12780cfff0 con 0x7f1288105370 2024-04-22T13:23:29.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.091+0000 7f12877fe640 1 --2- 172.21.15.12:0/374878575 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f12700a3c40 0x7f12700a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f127c005e10 tx=0x7f127c005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:29.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.095+0000 7f1284ff9640 1 -- 172.21.15.12:0/374878575 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f127801f020 con 0x7f1288105370 2024-04-22T13:23:29.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.281+0000 7f128ddfd640 1 -- 172.21.15.12:0/374878575 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f128802d810 con 0x7f1288105370 2024-04-22T13:23:29.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.283+0000 7f1284ff9640 1 -- 172.21.15.12:0/374878575 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22620 (secure 0 0 0) 0x7f127800d040 con 0x7f1288105370 2024-04-22T13:23:29.280 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:23:29.280 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"0aede5ea-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:14:09.362918+0000","modified":"2024-04-22T13:23:05.500756+0000","last_up_change":"2024-04-22T13:21:15.263393+0000","last_in_change":"2024-04-22T13:20:54.174685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:17:32.372434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:22:02.768799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:22:03.578730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d9e2365-3a64-4b13-96fc-c217b1cee2ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6803","nonce":1191350632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6805","nonce":1191350632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6809","nonce":1191350632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6807","nonce":1191350632}]},"public_addr":"172.21.15.12:6803/1191350632","cluster_addr":"172.21.15.12:6805/1191350632","heartbeat_back_addr":"172.21.15.12:6809/1191350632","heartbeat_front_addr":"172.21.15.12:6807/1191350632","state":["exists","up"]},{"osd":1,"uuid":"1499c783-2bd7-48ce-a444-34add42bcf31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6801","nonce":417496465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6803","nonce":417496465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6807","nonce":417496465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6805","nonce":417496465}]},"public_addr":"172.21.15.111:6801/417496465","cluster_addr":"172.21.15.111:6803/417496465","heartbeat_back_addr":"172.21.15.111:6807/417496465","heartbeat_front_addr":"172.21.15.111:6805/417496465","state":["exists","up"]},{"osd":2,"uuid":"71a9ebc3-c622-434e-a6c2-f48358f384f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6801","nonce":1403153998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6803","nonce":1403153998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6807","nonce":1403153998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6805","nonce":1403153998}]},"public_addr":"172.21.15.134:6801/1403153998","cluster_addr":"172.21.15.134:6803/1403153998","heartbeat_back_addr":"172.21.15.134:6807/1403153998","heartbeat_front_addr":"172.21.15.134:6805/1403153998","state":["exists","up"]},{"osd":3,"uuid":"6dd00db2-a286-48ba-8666-56ffdd5f35ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6811","nonce":3359066681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6813","nonce":3359066681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6817","nonce":3359066681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6815","nonce":3359066681}]},"public_addr":"172.21.15.12:6811/3359066681","cluster_addr":"172.21.15.12:6813/3359066681","heartbeat_back_addr":"172.21.15.12:6817/3359066681","heartbeat_front_addr":"172.21.15.12:6815/3359066681","state":["exists","up"]},{"osd":4,"uuid":"1e29afd7-354c-4506-aaa0-b43eb1a19a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6809","nonce":2886349872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6811","nonce":2886349872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6815","nonce":2886349872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6813","nonce":2886349872}]},"public_addr":"172.21.15.111:6809/2886349872","cluster_addr":"172.21.15.111:6811/2886349872","heartbeat_back_addr":"172.21.15.111:6815/2886349872","heartbeat_front_addr":"172.21.15.111:6813/2886349872","state":["exists","up"]},{"osd":5,"uuid":"8f40f02d-0ba2-4842-a03d-2a1a04d1010d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6809","nonce":3071977602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6811","nonce":3071977602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6815","nonce":3071977602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6813","nonce":3071977602}]},"public_addr":"172.21.15.134:6809/3071977602","cluster_addr":"172.21.15.134:6811/3071977602","heartbeat_back_addr":"172.21.15.134:6815/3071977602","heartbeat_front_addr":"172.21.15.134:6813/3071977602","state":["exists","up"]},{"osd":6,"uuid":"bb0c17a9-6a8e-4be6-8660-f73cfa8200ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6819","nonce":3819619299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6821","nonce":3819619299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6825","nonce":3819619299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6823","nonce":3819619299}]},"public_addr":"172.21.15.12:6819/3819619299","cluster_addr":"172.21.15.12:6821/3819619299","heartbeat_back_addr":"172.21.15.12:6825/3819619299","heartbeat_front_addr":"172.21.15.12:6823/3819619299","state":["exists","up"]},{"osd":7,"uuid":"0a93d6af-569e-4e45-b697-9e978c1c1004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6817","nonce":3113856282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6819","nonce":3113856282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6823","nonce":3113856282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6821","nonce":3113856282}]},"public_addr":"172.21.15.111:6817/3113856282","cluster_addr":"172.21.15.111:6819/3113856282","heartbeat_back_addr":"172.21.15.111:6823/3113856282","heartbeat_front_addr":"172.21.15.111:6821/3113856282","state":["exists","up"]},{"osd":8,"uuid":"e21acc68-cf50-4f4f-b1ba-cf6c195b4955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6817","nonce":1689362191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6819","nonce":1689362191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6823","nonce":1689362191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6821","nonce":1689362191}]},"public_addr":"172.21.15.134:6817/1689362191","cluster_addr":"172.21.15.134:6819/1689362191","heartbeat_back_addr":"172.21.15.134:6823/1689362191","heartbeat_front_addr":"172.21.15.134:6821/1689362191","state":["exists","up"]},{"osd":9,"uuid":"fd745869-7f5f-48a8-b412-bb9c92b586e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6827","nonce":4292385779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6829","nonce":4292385779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6833","nonce":4292385779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6831","nonce":4292385779}]},"public_addr":"172.21.15.12:6827/4292385779","cluster_addr":"172.21.15.12:6829/4292385779","heartbeat_back_addr":"172.21.15.12:6833/4292385779","heartbeat_front_addr":"172.21.15.12:6831/4292385779","state":["exists","up"]},{"osd":10,"uuid":"798b1abd-f6c7-4a5f-8da0-fdae49e50d2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6825","nonce":3338566976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6827","nonce":3338566976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6831","nonce":3338566976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6829","nonce":3338566976}]},"public_addr":"172.21.15.111:6825/3338566976","cluster_addr":"172.21.15.111:6827/3338566976","heartbeat_back_addr":"172.21.15.111:6831/3338566976","heartbeat_front_addr":"172.21.15.111:6829/3338566976","state":["exists","up"]},{"osd":11,"uuid":"b18fc5f3-1049-4871-b060-e99290273cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6825","nonce":578640428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6827","nonce":578640428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6831","nonce":578640428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6829","nonce":578640428}]},"public_addr":"172.21.15.134:6825/578640428","cluster_addr":"172.21.15.134:6827/578640428","heartbeat_back_addr":"172.21.15.134:6831/578640428","heartbeat_front_addr":"172.21.15.134:6829/578640428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:16:40.828608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:05.052528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:28.297316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:53.496497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:17.947259+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:41.712894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:07.105945+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:32.129688+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:56.008516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:22.239867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:47.334692+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:12.301276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/2593601362":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/2519382069":"2024-04-23T13:15:04.279990+0000","172.21.15.12:6801/2009831940":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3509412187":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1007749062":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3588480095":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1904962176":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2009831940":"2024-04-23T13:15:04.279990+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:23:29.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.285+0000 7f128ddfd640 1 -- 172.21.15.12:0/374878575 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f12700a3c40 msgr2=0x7f12700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:29.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.285+0000 7f128ddfd640 1 --2- 172.21.15.12:0/374878575 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f12700a3c40 0x7f12700a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f127c005e10 tx=0x7f127c005da0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.285+0000 7f128ddfd640 1 -- 172.21.15.12:0/374878575 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1288105370 msgr2=0x7f12881986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:29.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.285+0000 7f128ddfd640 1 --2- 172.21.15.12:0/374878575 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1288105370 0x7f12881986c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f127800b450 tx=0x7f127800b910 comp rx=0 tx=0).stop 2024-04-22T13:23:29.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.285+0000 7f128ddfd640 1 -- 172.21.15.12:0/374878575 shutdown_connections 2024-04-22T13:23:29.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.285+0000 7f128ddfd640 1 --2- 172.21.15.12:0/374878575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1288107ca0 0x7f128819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.285+0000 7f128ddfd640 1 --2- 172.21.15.12:0/374878575 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f12700a3c40 0x7f12700a6100 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.285+0000 7f128ddfd640 1 --2- 172.21.15.12:0/374878575 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1288105370 0x7f12881986c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.285+0000 7f128ddfd640 1 --2- 172.21.15.12:0/374878575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1288102a40 0x7f1288198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.285+0000 7f128ddfd640 1 -- 172.21.15.12:0/374878575 >> 172.21.15.12:0/374878575 conn(0x7f12880fc470 msgr2=0x7f1288103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:29.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.285+0000 7f128ddfd640 1 -- 172.21.15.12:0/374878575 shutdown_connections 2024-04-22T13:23:29.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.285+0000 7f128ddfd640 1 -- 172.21.15.12:0/374878575 wait complete. 2024-04-22T13:23:29.293 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T13:17:32.372434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T13:22:02.768799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T13:22:03.578730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T13:23:29.293 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T13:23:29.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.403+0000 7efc0ce69640 1 -- 172.21.15.12:0/1901593410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc08100820 msgr2=0x7efc08100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:29.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.403+0000 7efc0ce69640 1 --2- 172.21.15.12:0/1901593410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc08100820 0x7efc08100c40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7efbf8009ca0 tx=0x7efbf802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.403+0000 7efc0ce69640 1 -- 172.21.15.12:0/1901593410 shutdown_connections 2024-04-22T13:23:29.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.403+0000 7efc0ce69640 1 --2- 172.21.15.12:0/1901593410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc08101b20 0x7efc08101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.403+0000 7efc0ce69640 1 --2- 172.21.15.12:0/1901593410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc08101180 0x7efc081015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.403+0000 7efc0ce69640 1 --2- 172.21.15.12:0/1901593410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc08100820 0x7efc08100c40 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.403+0000 7efc0ce69640 1 -- 172.21.15.12:0/1901593410 >> 172.21.15.12:0/1901593410 conn(0x7efc080fc440 msgr2=0x7efc080fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:29.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.403+0000 7efc0ce69640 1 -- 172.21.15.12:0/1901593410 shutdown_connections 2024-04-22T13:23:29.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.403+0000 7efc0ce69640 1 -- 172.21.15.12:0/1901593410 wait complete. 2024-04-22T13:23:29.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.404+0000 7efc0ce69640 1 Processor -- start 2024-04-22T13:23:29.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.404+0000 7efc0ce69640 1 -- start start 2024-04-22T13:23:29.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.404+0000 7efc0ce69640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc08100820 0x7efc0819a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:29.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.404+0000 7efc0ce69640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc08101180 0x7efc0819a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:29.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.404+0000 7efc0ce69640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc08101b20 0x7efc081a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:29.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.404+0000 7efc0ce69640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efc0810fd00 con 0x7efc08101b20 2024-04-22T13:23:29.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.404+0000 7efc0ce69640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efc0810fb80 con 0x7efc08100820 2024-04-22T13:23:29.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.404+0000 7efc0ce69640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efc0810fe80 con 0x7efc08101180 2024-04-22T13:23:29.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.404+0000 7efc06d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc08101b20 0x7efc081a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:29.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.404+0000 7efc06d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc08101b20 0x7efc081a19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47806/0 (socket says 172.21.15.12:47806) 2024-04-22T13:23:29.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.404+0000 7efc06d9d640 1 -- 172.21.15.12:0/4252416954 learned_addr learned my addr 172.21.15.12:0/4252416954 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:29.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.405+0000 7efc05d9b640 1 --2- 172.21.15.12:0/4252416954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc08101180 0x7efc0819a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:29.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.405+0000 7efc0659c640 1 --2- 172.21.15.12:0/4252416954 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc08100820 0x7efc0819a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:29.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.405+0000 7efc06d9d640 1 -- 172.21.15.12:0/4252416954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc08101180 msgr2=0x7efc0819a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:29.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.405+0000 7efc06d9d640 1 --2- 172.21.15.12:0/4252416954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc08101180 0x7efc0819a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.405+0000 7efc06d9d640 1 -- 172.21.15.12:0/4252416954 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc08100820 msgr2=0x7efc0819a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:29.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.405+0000 7efc06d9d640 1 --2- 172.21.15.12:0/4252416954 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc08100820 0x7efc0819a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.405+0000 7efc06d9d640 1 -- 172.21.15.12:0/4252416954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc081a2030 con 0x7efc08101b20 2024-04-22T13:23:29.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.405+0000 7efc0659c640 1 --2- 172.21.15.12:0/4252416954 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc08100820 0x7efc0819a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:23:29.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.405+0000 7efc05d9b640 1 --2- 172.21.15.12:0/4252416954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc08101180 0x7efc0819a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:23:29.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.405+0000 7efc06d9d640 1 --2- 172.21.15.12:0/4252416954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc08101b20 0x7efc081a19c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7efc0000c970 tx=0x7efc0000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:29.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.405+0000 7efbf77fe640 1 -- 172.21.15.12:0/4252416954 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc00007bb0 con 0x7efc08101b20 2024-04-22T13:23:29.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.405+0000 7efc0ce69640 1 -- 172.21.15.12:0/4252416954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc081a2320 con 0x7efc08101b20 2024-04-22T13:23:29.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.405+0000 7efbf77fe640 1 -- 172.21.15.12:0/4252416954 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc00007d50 con 0x7efc08101b20 2024-04-22T13:23:29.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.405+0000 7efbf77fe640 1 -- 172.21.15.12:0/4252416954 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc00018770 con 0x7efc08101b20 2024-04-22T13:23:29.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.405+0000 7efc0ce69640 1 -- 172.21.15.12:0/4252416954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc081a2820 con 0x7efc08101b20 2024-04-22T13:23:29.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.406+0000 7efc0ce69640 1 -- 172.21.15.12:0/4252416954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc0810b570 con 0x7efc08101b20 2024-04-22T13:23:29.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.407+0000 7efbf77fe640 1 -- 172.21.15.12:0/4252416954 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7efc0000f990 con 0x7efc08101b20 2024-04-22T13:23:29.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.408+0000 7efbf77fe640 1 --2- 172.21.15.12:0/4252416954 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efbf00a3c40 0x7efbf00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:29.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.408+0000 7efc0659c640 1 --2- 172.21.15.12:0/4252416954 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efbf00a3c40 0x7efbf00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:29.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.408+0000 7efbf77fe640 1 -- 172.21.15.12:0/4252416954 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7efc000d0670 con 0x7efc08101b20 2024-04-22T13:23:29.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.408+0000 7efc0659c640 1 --2- 172.21.15.12:0/4252416954 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efbf00a3c40 0x7efbf00a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efbf8000f80 tx=0x7efbf80324c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:29.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.413+0000 7efbf77fe640 1 -- 172.21.15.12:0/4252416954 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efc00014030 con 0x7efc08101b20 2024-04-22T13:23:29.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.601+0000 7efc0ce69640 1 -- 172.21.15.12:0/4252416954 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efc08062b50 con 0x7efc08101b20 2024-04-22T13:23:29.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.601+0000 7efbf77fe640 1 -- 172.21.15.12:0/4252416954 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7efc0009cf20 con 0x7efc08101b20 2024-04-22T13:23:29.598 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-22T13:23:29.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.603+0000 7efc0ce69640 1 -- 172.21.15.12:0/4252416954 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efbf00a3c40 msgr2=0x7efbf00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:29.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.603+0000 7efc0ce69640 1 --2- 172.21.15.12:0/4252416954 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efbf00a3c40 0x7efbf00a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efbf8000f80 tx=0x7efbf80324c0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.603+0000 7efc0ce69640 1 -- 172.21.15.12:0/4252416954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc08101b20 msgr2=0x7efc081a19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:29.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.603+0000 7efc0ce69640 1 --2- 172.21.15.12:0/4252416954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc08101b20 0x7efc081a19c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7efc0000c970 tx=0x7efc0000ce30 comp rx=0 tx=0).stop 2024-04-22T13:23:29.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.603+0000 7efc0ce69640 1 -- 172.21.15.12:0/4252416954 shutdown_connections 2024-04-22T13:23:29.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.603+0000 7efc0ce69640 1 --2- 172.21.15.12:0/4252416954 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efbf00a3c40 0x7efbf00a6100 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.603+0000 7efc0ce69640 1 --2- 172.21.15.12:0/4252416954 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc08101b20 0x7efc081a19c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.603+0000 7efc0ce69640 1 --2- 172.21.15.12:0/4252416954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc08101180 0x7efc0819a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.603+0000 7efc0ce69640 1 --2- 172.21.15.12:0/4252416954 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efc08100820 0x7efc0819a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.603+0000 7efc0ce69640 1 -- 172.21.15.12:0/4252416954 >> 172.21.15.12:0/4252416954 conn(0x7efc080fc440 msgr2=0x7efc080fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:29.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.603+0000 7efc0ce69640 1 -- 172.21.15.12:0/4252416954 shutdown_connections 2024-04-22T13:23:29.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.603+0000 7efc0ce69640 1 -- 172.21.15.12:0/4252416954 wait complete. 2024-04-22T13:23:29.611 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T13:23:29.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:29 smithi134 ceph-mon[31338]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-22T13:23:29.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:29 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/374878575' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:23:29.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:29 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/4252416954' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:23:29.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.719+0000 7f92261ef640 1 -- 172.21.15.12:0/569568284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9220103190 msgr2=0x7f9220103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:29.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.720+0000 7f92261ef640 1 --2- 172.21.15.12:0/569568284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9220103190 0x7f9220103610 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9214009900 tx=0x7f921402bc30 comp rx=0 tx=0).stop 2024-04-22T13:23:29.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.720+0000 7f92261ef640 1 -- 172.21.15.12:0/569568284 shutdown_connections 2024-04-22T13:23:29.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.720+0000 7f92261ef640 1 --2- 172.21.15.12:0/569568284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9220103b50 0x7f922010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.720+0000 7f92261ef640 1 --2- 172.21.15.12:0/569568284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9220103190 0x7f9220103610 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.720+0000 7f92261ef640 1 --2- 172.21.15.12:0/569568284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9220101fa0 0x7f92201023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.720+0000 7f92261ef640 1 -- 172.21.15.12:0/569568284 >> 172.21.15.12:0/569568284 conn(0x7f92200fd530 msgr2=0x7f92200ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:29.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.720+0000 7f92261ef640 1 -- 172.21.15.12:0/569568284 shutdown_connections 2024-04-22T13:23:29.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.720+0000 7f92261ef640 1 -- 172.21.15.12:0/569568284 wait complete. 2024-04-22T13:23:29.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.720+0000 7f92261ef640 1 Processor -- start 2024-04-22T13:23:29.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.720+0000 7f92261ef640 1 -- start start 2024-04-22T13:23:29.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.721+0000 7f92261ef640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9220101fa0 0x7f9220198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:29.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.721+0000 7f92261ef640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9220103190 0x7f92201987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:29.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.721+0000 7f92261ef640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9220103b50 0x7f922019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:29.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.721+0000 7f92261ef640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f922010db40 con 0x7f9220101fa0 2024-04-22T13:23:29.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.721+0000 7f92261ef640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f922010d9c0 con 0x7f9220103b50 2024-04-22T13:23:29.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.721+0000 7f92261ef640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f922010dcc0 con 0x7f9220103190 2024-04-22T13:23:29.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.721+0000 7f921f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9220101fa0 0x7f9220198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.721+0000 7f921f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9220101fa0 0x7f9220198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47818/0 (socket says 172.21.15.12:47818) 2024-04-22T13:23:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.721+0000 7f921f7fe640 1 -- 172.21.15.12:0/2026031197 learned_addr learned my addr 172.21.15.12:0/2026031197 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.721+0000 7f921effd640 1 --2- 172.21.15.12:0/2026031197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9220103190 0x7f92201987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.721+0000 7f921f7fe640 1 -- 172.21.15.12:0/2026031197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9220103190 msgr2=0x7f92201987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.721+0000 7f921ffff640 1 --2- 172.21.15.12:0/2026031197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9220103b50 0x7f922019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.721+0000 7f921f7fe640 1 --2- 172.21.15.12:0/2026031197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9220103190 0x7f92201987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.721+0000 7f921f7fe640 1 -- 172.21.15.12:0/2026031197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9220103b50 msgr2=0x7f922019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.721+0000 7f921f7fe640 1 --2- 172.21.15.12:0/2026031197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9220103b50 0x7f922019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.721+0000 7f921f7fe640 1 -- 172.21.15.12:0/2026031197 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f922019feb0 con 0x7f9220101fa0 2024-04-22T13:23:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.721+0000 7f921ffff640 1 --2- 172.21.15.12:0/2026031197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9220103b50 0x7f922019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:23:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.721+0000 7f921f7fe640 1 --2- 172.21.15.12:0/2026031197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9220101fa0 0x7f9220198290 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f921000b9c0 tx=0x7f921000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.721+0000 7f921cff9640 1 -- 172.21.15.12:0/2026031197 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9210019070 con 0x7f9220101fa0 2024-04-22T13:23:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.721+0000 7f921cff9640 1 -- 172.21.15.12:0/2026031197 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f921000cba0 con 0x7f9220101fa0 2024-04-22T13:23:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.721+0000 7f92261ef640 1 -- 172.21.15.12:0/2026031197 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92201a01a0 con 0x7f9220101fa0 2024-04-22T13:23:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.721+0000 7f921cff9640 1 -- 172.21.15.12:0/2026031197 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92100125a0 con 0x7f9220101fa0 2024-04-22T13:23:29.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.721+0000 7f92261ef640 1 -- 172.21.15.12:0/2026031197 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92201a0770 con 0x7f9220101fa0 2024-04-22T13:23:29.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.722+0000 7f92261ef640 1 -- 172.21.15.12:0/2026031197 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9220109390 con 0x7f9220101fa0 2024-04-22T13:23:29.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.723+0000 7f921cff9640 1 -- 172.21.15.12:0/2026031197 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f921001c070 con 0x7f9220101fa0 2024-04-22T13:23:29.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.724+0000 7f921cff9640 1 --2- 172.21.15.12:0/2026031197 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f92080a3c40 0x7f92080a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:29.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.724+0000 7f921effd640 1 --2- 172.21.15.12:0/2026031197 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f92080a3c40 0x7f92080a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:29.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.724+0000 7f921cff9640 1 -- 172.21.15.12:0/2026031197 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f92100cfa60 con 0x7f9220101fa0 2024-04-22T13:23:29.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.724+0000 7f921effd640 1 --2- 172.21.15.12:0/2026031197 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f92080a3c40 0x7f92080a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f92201997a0 tx=0x7f9214005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:29.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.729+0000 7f921cff9640 1 -- 172.21.15.12:0/2026031197 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f921009c420 con 0x7f9220101fa0 2024-04-22T13:23:29.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.916+0000 7f92261ef640 1 -- 172.21.15.12:0/2026031197 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9220062b50 con 0x7f9220101fa0 2024-04-22T13:23:29.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.916+0000 7f921cff9640 1 -- 172.21.15.12:0/2026031197 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f921009bd20 con 0x7f9220101fa0 2024-04-22T13:23:29.913 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-22T13:23:29.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.918+0000 7f92261ef640 1 -- 172.21.15.12:0/2026031197 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f92080a3c40 msgr2=0x7f92080a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:29.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.918+0000 7f92261ef640 1 --2- 172.21.15.12:0/2026031197 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f92080a3c40 0x7f92080a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f92201997a0 tx=0x7f9214005be0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.918+0000 7f92261ef640 1 -- 172.21.15.12:0/2026031197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9220101fa0 msgr2=0x7f9220198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:29.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.918+0000 7f92261ef640 1 --2- 172.21.15.12:0/2026031197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9220101fa0 0x7f9220198290 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f921000b9c0 tx=0x7f921000be80 comp rx=0 tx=0).stop 2024-04-22T13:23:29.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.918+0000 7f92261ef640 1 -- 172.21.15.12:0/2026031197 shutdown_connections 2024-04-22T13:23:29.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.918+0000 7f92261ef640 1 --2- 172.21.15.12:0/2026031197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9220103b50 0x7f922019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.918+0000 7f92261ef640 1 --2- 172.21.15.12:0/2026031197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9220103190 0x7f92201987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.918+0000 7f92261ef640 1 --2- 172.21.15.12:0/2026031197 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f92080a3c40 0x7f92080a6100 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.918+0000 7f92261ef640 1 --2- 172.21.15.12:0/2026031197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9220101fa0 0x7f9220198290 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:29.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.918+0000 7f92261ef640 1 -- 172.21.15.12:0/2026031197 >> 172.21.15.12:0/2026031197 conn(0x7f92200fd530 msgr2=0x7f9220106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:29.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.918+0000 7f92261ef640 1 -- 172.21.15.12:0/2026031197 shutdown_connections 2024-04-22T13:23:29.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:29.918+0000 7f92261ef640 1 -- 172.21.15.12:0/2026031197 wait complete. 2024-04-22T13:23:29.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:29 smithi012 ceph-mon[28265]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-22T13:23:29.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:29 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/374878575' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:23:29.925 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:29 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/4252416954' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:23:29.926 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T13:23:29.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:29 smithi111 ceph-mon[31361]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-22T13:23:29.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:29 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/374878575' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:23:29.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:29 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/4252416954' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:23:30.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.035+0000 7f224ee4a640 1 -- 172.21.15.12:0/1426057761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22481031a0 msgr2=0x7f2248103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:30.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.035+0000 7f224ee4a640 1 --2- 172.21.15.12:0/1426057761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22481031a0 0x7f2248103620 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f223c009900 tx=0x7f223c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:23:30.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.035+0000 7f224ee4a640 1 -- 172.21.15.12:0/1426057761 shutdown_connections 2024-04-22T13:23:30.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.035+0000 7f224ee4a640 1 --2- 172.21.15.12:0/1426057761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2248103b60 0x7f224810a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:30.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.035+0000 7f224ee4a640 1 --2- 172.21.15.12:0/1426057761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f22481031a0 0x7f2248103620 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:30.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.035+0000 7f224ee4a640 1 --2- 172.21.15.12:0/1426057761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2248101fb0 0x7f22481023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:30.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.035+0000 7f224ee4a640 1 -- 172.21.15.12:0/1426057761 >> 172.21.15.12:0/1426057761 conn(0x7f22480fd520 msgr2=0x7f22480ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:30.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.035+0000 7f224ee4a640 1 -- 172.21.15.12:0/1426057761 shutdown_connections 2024-04-22T13:23:30.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.035+0000 7f224ee4a640 1 -- 172.21.15.12:0/1426057761 wait complete. 2024-04-22T13:23:30.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.035+0000 7f224ee4a640 1 Processor -- start 2024-04-22T13:23:30.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.035+0000 7f224ee4a640 1 -- start start 2024-04-22T13:23:30.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.035+0000 7f224ee4a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2248101fb0 0x7f22481981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:30.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.035+0000 7f224ee4a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22481031a0 0x7f2248198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:30.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.035+0000 7f224ee4a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2248103b60 0x7f224819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:30.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.035+0000 7f224ee4a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f224810daf0 con 0x7f2248103b60 2024-04-22T13:23:30.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.035+0000 7f224ee4a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f224810d970 con 0x7f22481031a0 2024-04-22T13:23:30.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.035+0000 7f224ee4a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f224810dc70 con 0x7f2248101fb0 2024-04-22T13:23:30.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.036+0000 7f224d3e7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2248103b60 0x7f224819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:30.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.036+0000 7f224d3e7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2248103b60 0x7f224819f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47844/0 (socket says 172.21.15.12:47844) 2024-04-22T13:23:30.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.036+0000 7f224d3e7640 1 -- 172.21.15.12:0/1363765627 learned_addr learned my addr 172.21.15.12:0/1363765627 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:30.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.036+0000 7f224cbe6640 1 --2- 172.21.15.12:0/1363765627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2248101fb0 0x7f22481981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:30.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.036+0000 7f2247fff640 1 --2- 172.21.15.12:0/1363765627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22481031a0 0x7f2248198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:30.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.036+0000 7f224d3e7640 1 -- 172.21.15.12:0/1363765627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2248101fb0 msgr2=0x7f22481981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:30.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.036+0000 7f224d3e7640 1 --2- 172.21.15.12:0/1363765627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2248101fb0 0x7f22481981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:30.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.036+0000 7f224d3e7640 1 -- 172.21.15.12:0/1363765627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22481031a0 msgr2=0x7f2248198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:30.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.036+0000 7f224d3e7640 1 --2- 172.21.15.12:0/1363765627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22481031a0 0x7f2248198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:30.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.036+0000 7f224d3e7640 1 -- 172.21.15.12:0/1363765627 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224819fe10 con 0x7f2248103b60 2024-04-22T13:23:30.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.036+0000 7f224d3e7640 1 --2- 172.21.15.12:0/1363765627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2248103b60 0x7f224819f7a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f224000eaa0 tx=0x7f224000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:30.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.036+0000 7f2245ffb640 1 -- 172.21.15.12:0/1363765627 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224000cdb0 con 0x7f2248103b60 2024-04-22T13:23:30.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.036+0000 7f224ee4a640 1 -- 172.21.15.12:0/1363765627 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22481a0100 con 0x7f2248103b60 2024-04-22T13:23:30.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.036+0000 7f2245ffb640 1 -- 172.21.15.12:0/1363765627 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2240005020 con 0x7f2248103b60 2024-04-22T13:23:30.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.036+0000 7f2245ffb640 1 -- 172.21.15.12:0/1363765627 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22400116d0 con 0x7f2248103b60 2024-04-22T13:23:30.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.036+0000 7f224ee4a640 1 -- 172.21.15.12:0/1363765627 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22481a0600 con 0x7f2248103b60 2024-04-22T13:23:30.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.037+0000 7f224ee4a640 1 -- 172.21.15.12:0/1363765627 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22481093a0 con 0x7f2248103b60 2024-04-22T13:23:30.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.038+0000 7f2245ffb640 1 -- 172.21.15.12:0/1363765627 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f22400051d0 con 0x7f2248103b60 2024-04-22T13:23:30.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.039+0000 7f2245ffb640 1 --2- 172.21.15.12:0/1363765627 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f22300a3c40 0x7f22300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:30.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.039+0000 7f2245ffb640 1 -- 172.21.15.12:0/1363765627 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f22400d0810 con 0x7f2248103b60 2024-04-22T13:23:30.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.039+0000 7f224cbe6640 1 --2- 172.21.15.12:0/1363765627 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f22300a3c40 0x7f22300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:30.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.039+0000 7f224cbe6640 1 --2- 172.21.15.12:0/1363765627 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f22300a3c40 0x7f22300a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2238005fd0 tx=0x7f2238005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:30.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.044+0000 7f2245ffb640 1 -- 172.21.15.12:0/1363765627 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2240016030 con 0x7f2248103b60 2024-04-22T13:23:30.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.230+0000 7f224ee4a640 1 -- 172.21.15.12:0/1363765627 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2248062b50 con 0x7f2248103b60 2024-04-22T13:23:30.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.231+0000 7f2245ffb640 1 -- 172.21.15.12:0/1363765627 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f224009d0c0 con 0x7f2248103b60 2024-04-22T13:23:30.227 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-22T13:23:30.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.232+0000 7f224ee4a640 1 -- 172.21.15.12:0/1363765627 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f22300a3c40 msgr2=0x7f22300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:30.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.232+0000 7f224ee4a640 1 --2- 172.21.15.12:0/1363765627 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f22300a3c40 0x7f22300a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2238005fd0 tx=0x7f2238005850 comp rx=0 tx=0).stop 2024-04-22T13:23:30.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.232+0000 7f224ee4a640 1 -- 172.21.15.12:0/1363765627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2248103b60 msgr2=0x7f224819f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:30.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.232+0000 7f224ee4a640 1 --2- 172.21.15.12:0/1363765627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2248103b60 0x7f224819f7a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f224000eaa0 tx=0x7f224000ef60 comp rx=0 tx=0).stop 2024-04-22T13:23:30.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.233+0000 7f224ee4a640 1 -- 172.21.15.12:0/1363765627 shutdown_connections 2024-04-22T13:23:30.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.233+0000 7f224ee4a640 1 --2- 172.21.15.12:0/1363765627 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f22300a3c40 0x7f22300a6100 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:30.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.233+0000 7f224ee4a640 1 --2- 172.21.15.12:0/1363765627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2248103b60 0x7f224819f7a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:30.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.233+0000 7f224ee4a640 1 --2- 172.21.15.12:0/1363765627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22481031a0 0x7f2248198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:30.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.233+0000 7f224ee4a640 1 --2- 172.21.15.12:0/1363765627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2248101fb0 0x7f22481981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:30.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.233+0000 7f224ee4a640 1 -- 172.21.15.12:0/1363765627 >> 172.21.15.12:0/1363765627 conn(0x7f22480fd520 msgr2=0x7f2248106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:30.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.233+0000 7f224ee4a640 1 -- 172.21.15.12:0/1363765627 shutdown_connections 2024-04-22T13:23:30.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.233+0000 7f224ee4a640 1 -- 172.21.15.12:0/1363765627 wait complete. 2024-04-22T13:23:30.240 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:23:30.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.345+0000 7f050b055640 1 -- 172.21.15.12:0/2194107452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0504102a10 msgr2=0x7f0504109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:30.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.345+0000 7f050b055640 1 --2- 172.21.15.12:0/2194107452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0504102a10 0x7f0504109330 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009ca0 tx=0x7f04fc02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:23:30.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.345+0000 7f050b055640 1 -- 172.21.15.12:0/2194107452 shutdown_connections 2024-04-22T13:23:30.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.345+0000 7f050b055640 1 --2- 172.21.15.12:0/2194107452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0504102a10 0x7f0504109330 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:30.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.345+0000 7f050b055640 1 --2- 172.21.15.12:0/2194107452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0504102050 0x7f05041024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:30.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.345+0000 7f050b055640 1 --2- 172.21.15.12:0/2194107452 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0504100e70 0x7f0504101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:30.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.345+0000 7f050b055640 1 -- 172.21.15.12:0/2194107452 >> 172.21.15.12:0/2194107452 conn(0x7f05040fc470 msgr2=0x7f05040fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:30.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.345+0000 7f050b055640 1 -- 172.21.15.12:0/2194107452 shutdown_connections 2024-04-22T13:23:30.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.345+0000 7f050b055640 1 -- 172.21.15.12:0/2194107452 wait complete. 2024-04-22T13:23:30.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.346+0000 7f050b055640 1 Processor -- start 2024-04-22T13:23:30.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.346+0000 7f050b055640 1 -- start start 2024-04-22T13:23:30.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.346+0000 7f050b055640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0504100e70 0x7f0504104e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:30.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.346+0000 7f050b055640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0504102050 0x7f05041034c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:30.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.346+0000 7f050b055640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0504102a10 0x7f0504103a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:30.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.346+0000 7f050b055640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f050410ca80 con 0x7f0504100e70 2024-04-22T13:23:30.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.346+0000 7f050b055640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f050410c900 con 0x7f0504102050 2024-04-22T13:23:30.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.346+0000 7f050b055640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f050410cc00 con 0x7f0504102a10 2024-04-22T13:23:30.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.346+0000 7f0508df1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0504100e70 0x7f0504104e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:30.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.346+0000 7f0508df1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0504100e70 0x7f0504104e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47858/0 (socket says 172.21.15.12:47858) 2024-04-22T13:23:30.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.346+0000 7f0508df1640 1 -- 172.21.15.12:0/1905515635 learned_addr learned my addr 172.21.15.12:0/1905515635 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:30.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.346+0000 7f0503fff640 1 --2- 172.21.15.12:0/1905515635 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0504102050 0x7f05041034c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:30.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.346+0000 7f05095f2640 1 --2- 172.21.15.12:0/1905515635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0504102a10 0x7f0504103a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:30.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.346+0000 7f0508df1640 1 -- 172.21.15.12:0/1905515635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0504102a10 msgr2=0x7f0504103a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:30.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.346+0000 7f0508df1640 1 --2- 172.21.15.12:0/1905515635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0504102a10 0x7f0504103a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:30.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.346+0000 7f0508df1640 1 -- 172.21.15.12:0/1905515635 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0504102050 msgr2=0x7f05041034c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:30.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.346+0000 7f0508df1640 1 --2- 172.21.15.12:0/1905515635 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0504102050 0x7f05041034c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:30.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.346+0000 7f0508df1640 1 -- 172.21.15.12:0/1905515635 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0504104290 con 0x7f0504100e70 2024-04-22T13:23:30.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.346+0000 7f0508df1640 1 --2- 172.21.15.12:0/1905515635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0504100e70 0x7f0504104e80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f04f800b450 tx=0x7f04f800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:30.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.347+0000 7f0501ffb640 1 -- 172.21.15.12:0/1905515635 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f8015020 con 0x7f0504100e70 2024-04-22T13:23:30.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.347+0000 7f050b055640 1 -- 172.21.15.12:0/1905515635 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05041a2360 con 0x7f0504100e70 2024-04-22T13:23:30.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.347+0000 7f0501ffb640 1 -- 172.21.15.12:0/1905515635 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04f8011d40 con 0x7f0504100e70 2024-04-22T13:23:30.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.347+0000 7f0501ffb640 1 -- 172.21.15.12:0/1905515635 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f800f340 con 0x7f0504100e70 2024-04-22T13:23:30.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.347+0000 7f050b055640 1 -- 172.21.15.12:0/1905515635 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05041a28b0 con 0x7f0504100e70 2024-04-22T13:23:30.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.348+0000 7f050b055640 1 -- 172.21.15.12:0/1905515635 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05041a2b70 con 0x7f0504100e70 2024-04-22T13:23:30.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.349+0000 7f0501ffb640 1 -- 172.21.15.12:0/1905515635 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f04f8018020 con 0x7f0504100e70 2024-04-22T13:23:30.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.349+0000 7f0501ffb640 1 --2- 172.21.15.12:0/1905515635 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f04f40a3c40 0x7f04f40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:30.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.349+0000 7f0503fff640 1 --2- 172.21.15.12:0/1905515635 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f04f40a3c40 0x7f04f40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:30.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.350+0000 7f0501ffb640 1 -- 172.21.15.12:0/1905515635 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f04f80cff80 con 0x7f0504100e70 2024-04-22T13:23:30.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.350+0000 7f0503fff640 1 --2- 172.21.15.12:0/1905515635 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f04f40a3c40 0x7f04f40a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0504104bf0 tx=0x7f04f000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:30.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.354+0000 7f0501ffb640 1 -- 172.21.15.12:0/1905515635 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f04f801f020 con 0x7f0504100e70 2024-04-22T13:23:30.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.581+0000 7f050b055640 1 -- 172.21.15.12:0/1905515635 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05041a2d40 con 0x7f0504100e70 2024-04-22T13:23:30.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.582+0000 7f0501ffb640 1 -- 172.21.15.12:0/1905515635 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11942 (secure 0 0 0) 0x7f04f800d040 con 0x7f0504100e70 2024-04-22T13:23:30.579 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:23:30.579 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:22:12.398056+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24448,24509,24443,24484,24407]},"id":1}]} 2024-04-22T13:23:30.579 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 27 2024-04-22T13:23:30.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.584+0000 7f050b055640 1 -- 172.21.15.12:0/1905515635 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f04f40a3c40 msgr2=0x7f04f40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:30.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.584+0000 7f050b055640 1 --2- 172.21.15.12:0/1905515635 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f04f40a3c40 0x7f04f40a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0504104bf0 tx=0x7f04f000b390 comp rx=0 tx=0).stop 2024-04-22T13:23:30.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.584+0000 7f050b055640 1 -- 172.21.15.12:0/1905515635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0504100e70 msgr2=0x7f0504104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:30.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.584+0000 7f050b055640 1 --2- 172.21.15.12:0/1905515635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0504100e70 0x7f0504104e80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f04f800b450 tx=0x7f04f800b910 comp rx=0 tx=0).stop 2024-04-22T13:23:30.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.584+0000 7f050b055640 1 -- 172.21.15.12:0/1905515635 shutdown_connections 2024-04-22T13:23:30.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.584+0000 7f050b055640 1 --2- 172.21.15.12:0/1905515635 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0504102a10 0x7f0504103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:30.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.584+0000 7f050b055640 1 --2- 172.21.15.12:0/1905515635 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0504102050 0x7f05041034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:30.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.584+0000 7f050b055640 1 --2- 172.21.15.12:0/1905515635 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f04f40a3c40 0x7f04f40a6100 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:30.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.584+0000 7f050b055640 1 --2- 172.21.15.12:0/1905515635 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0504100e70 0x7f0504104e80 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:30.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.584+0000 7f050b055640 1 -- 172.21.15.12:0/1905515635 >> 172.21.15.12:0/1905515635 conn(0x7f05040fc470 msgr2=0x7f05040fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:30.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.584+0000 7f050b055640 1 -- 172.21.15.12:0/1905515635 shutdown_connections 2024-04-22T13:23:30.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.584+0000 7f050b055640 1 -- 172.21.15.12:0/1905515635 wait complete. 2024-04-22T13:23:30.592 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:23:30.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.698+0000 7fdb84e48640 1 -- 172.21.15.12:0/3138025167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb80102980 msgr2=0x7fdb801092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:30.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.698+0000 7fdb84e48640 1 --2- 172.21.15.12:0/3138025167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb80102980 0x7fdb801092a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdb78009900 tx=0x7fdb7802bc30 comp rx=0 tx=0).stop 2024-04-22T13:23:30.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.699+0000 7fdb84e48640 1 -- 172.21.15.12:0/3138025167 shutdown_connections 2024-04-22T13:23:30.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.699+0000 7fdb84e48640 1 --2- 172.21.15.12:0/3138025167 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb80102980 0x7fdb801092a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:30.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.699+0000 7fdb84e48640 1 --2- 172.21.15.12:0/3138025167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb80101fc0 0x7fdb80102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:30.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.699+0000 7fdb84e48640 1 --2- 172.21.15.12:0/3138025167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb80100e70 0x7fdb80101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:30.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.699+0000 7fdb84e48640 1 -- 172.21.15.12:0/3138025167 >> 172.21.15.12:0/3138025167 conn(0x7fdb800fc3e0 msgr2=0x7fdb800fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:30.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.699+0000 7fdb84e48640 1 -- 172.21.15.12:0/3138025167 shutdown_connections 2024-04-22T13:23:30.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.699+0000 7fdb84e48640 1 -- 172.21.15.12:0/3138025167 wait complete. 2024-04-22T13:23:30.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.699+0000 7fdb84e48640 1 Processor -- start 2024-04-22T13:23:30.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.699+0000 7fdb84e48640 1 -- start start 2024-04-22T13:23:30.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.700+0000 7fdb84e48640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb80100e70 0x7fdb80104df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:30.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.700+0000 7fdb84e48640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb80101fc0 0x7fdb80103430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:30.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.700+0000 7fdb84e48640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb80102980 0x7fdb80103970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:30.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.700+0000 7fdb84e48640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8010c9d0 con 0x7fdb80100e70 2024-04-22T13:23:30.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.700+0000 7fdb84e48640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8010c850 con 0x7fdb80102980 2024-04-22T13:23:30.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.700+0000 7fdb84e48640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8010cb50 con 0x7fdb80101fc0 2024-04-22T13:23:30.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.700+0000 7fdb7e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb80100e70 0x7fdb80104df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:30.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.700+0000 7fdb7e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb80100e70 0x7fdb80104df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47874/0 (socket says 172.21.15.12:47874) 2024-04-22T13:23:30.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.700+0000 7fdb7e59c640 1 -- 172.21.15.12:0/71034727 learned_addr learned my addr 172.21.15.12:0/71034727 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:30.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.700+0000 7fdb7ed9d640 1 --2- 172.21.15.12:0/71034727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb80102980 0x7fdb80103970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:30.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.700+0000 7fdb7dd9b640 1 --2- 172.21.15.12:0/71034727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb80101fc0 0x7fdb80103430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:30.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.700+0000 7fdb7e59c640 1 -- 172.21.15.12:0/71034727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb80101fc0 msgr2=0x7fdb80103430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:30.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.700+0000 7fdb7e59c640 1 --2- 172.21.15.12:0/71034727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb80101fc0 0x7fdb80103430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:30.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.700+0000 7fdb7e59c640 1 -- 172.21.15.12:0/71034727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb80102980 msgr2=0x7fdb80103970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:30.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.700+0000 7fdb7e59c640 1 --2- 172.21.15.12:0/71034727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb80102980 0x7fdb80103970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:30.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.700+0000 7fdb7e59c640 1 -- 172.21.15.12:0/71034727 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb80104200 con 0x7fdb80100e70 2024-04-22T13:23:30.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.700+0000 7fdb7dd9b640 1 --2- 172.21.15.12:0/71034727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb80101fc0 0x7fdb80103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:23:30.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.700+0000 7fdb7e59c640 1 --2- 172.21.15.12:0/71034727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb80100e70 0x7fdb80104df0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000eab0 tx=0x7fdb7000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:30.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.700+0000 7fdb6f7fe640 1 -- 172.21.15.12:0/71034727 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7000ce40 con 0x7fdb80100e70 2024-04-22T13:23:30.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.700+0000 7fdb84e48640 1 -- 172.21.15.12:0/71034727 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb801a22f0 con 0x7fdb80100e70 2024-04-22T13:23:30.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.700+0000 7fdb6f7fe640 1 -- 172.21.15.12:0/71034727 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb70011920 con 0x7fdb80100e70 2024-04-22T13:23:30.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.700+0000 7fdb6f7fe640 1 -- 172.21.15.12:0/71034727 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb70010a30 con 0x7fdb80100e70 2024-04-22T13:23:30.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.700+0000 7fdb84e48640 1 -- 172.21.15.12:0/71034727 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb801a2840 con 0x7fdb80100e70 2024-04-22T13:23:30.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.701+0000 7fdb84e48640 1 -- 172.21.15.12:0/71034727 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb801a2b60 con 0x7fdb80100e70 2024-04-22T13:23:30.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.702+0000 7fdb6f7fe640 1 -- 172.21.15.12:0/71034727 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdb70010bd0 con 0x7fdb80100e70 2024-04-22T13:23:30.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.703+0000 7fdb6f7fe640 1 --2- 172.21.15.12:0/71034727 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fdb680a3c40 0x7fdb680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:30.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.703+0000 7fdb7dd9b640 1 --2- 172.21.15.12:0/71034727 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fdb680a3c40 0x7fdb680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:30.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.703+0000 7fdb6f7fe640 1 -- 172.21.15.12:0/71034727 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fdb700d0bd0 con 0x7fdb80100e70 2024-04-22T13:23:30.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.704+0000 7fdb7dd9b640 1 --2- 172.21.15.12:0/71034727 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fdb680a3c40 0x7fdb680a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdb80104b60 tx=0x7fdb74005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:30.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.708+0000 7fdb6f7fe640 1 -- 172.21.15.12:0/71034727 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdb70016030 con 0x7fdb80100e70 2024-04-22T13:23:30.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.895+0000 7fdb84e48640 1 -- 172.21.15.12:0/71034727 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdb801a2d30 con 0x7fdb80100e70 2024-04-22T13:23:30.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.897+0000 7fdb6f7fe640 1 -- 172.21.15.12:0/71034727 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22620 (secure 0 0 0) 0x7fdb7009d480 con 0x7fdb80100e70 2024-04-22T13:23:30.893 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:23:30.894 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"0aede5ea-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:14:09.362918+0000","modified":"2024-04-22T13:23:05.500756+0000","last_up_change":"2024-04-22T13:21:15.263393+0000","last_in_change":"2024-04-22T13:20:54.174685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:17:32.372434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:22:02.768799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:22:03.578730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d9e2365-3a64-4b13-96fc-c217b1cee2ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6803","nonce":1191350632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6805","nonce":1191350632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6809","nonce":1191350632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6807","nonce":1191350632}]},"public_addr":"172.21.15.12:6803/1191350632","cluster_addr":"172.21.15.12:6805/1191350632","heartbeat_back_addr":"172.21.15.12:6809/1191350632","heartbeat_front_addr":"172.21.15.12:6807/1191350632","state":["exists","up"]},{"osd":1,"uuid":"1499c783-2bd7-48ce-a444-34add42bcf31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6801","nonce":417496465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6803","nonce":417496465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6807","nonce":417496465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6805","nonce":417496465}]},"public_addr":"172.21.15.111:6801/417496465","cluster_addr":"172.21.15.111:6803/417496465","heartbeat_back_addr":"172.21.15.111:6807/417496465","heartbeat_front_addr":"172.21.15.111:6805/417496465","state":["exists","up"]},{"osd":2,"uuid":"71a9ebc3-c622-434e-a6c2-f48358f384f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6801","nonce":1403153998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6803","nonce":1403153998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6807","nonce":1403153998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6805","nonce":1403153998}]},"public_addr":"172.21.15.134:6801/1403153998","cluster_addr":"172.21.15.134:6803/1403153998","heartbeat_back_addr":"172.21.15.134:6807/1403153998","heartbeat_front_addr":"172.21.15.134:6805/1403153998","state":["exists","up"]},{"osd":3,"uuid":"6dd00db2-a286-48ba-8666-56ffdd5f35ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6811","nonce":3359066681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6813","nonce":3359066681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6817","nonce":3359066681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6815","nonce":3359066681}]},"public_addr":"172.21.15.12:6811/3359066681","cluster_addr":"172.21.15.12:6813/3359066681","heartbeat_back_addr":"172.21.15.12:6817/3359066681","heartbeat_front_addr":"172.21.15.12:6815/3359066681","state":["exists","up"]},{"osd":4,"uuid":"1e29afd7-354c-4506-aaa0-b43eb1a19a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6809","nonce":2886349872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6811","nonce":2886349872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6815","nonce":2886349872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6813","nonce":2886349872}]},"public_addr":"172.21.15.111:6809/2886349872","cluster_addr":"172.21.15.111:6811/2886349872","heartbeat_back_addr":"172.21.15.111:6815/2886349872","heartbeat_front_addr":"172.21.15.111:6813/2886349872","state":["exists","up"]},{"osd":5,"uuid":"8f40f02d-0ba2-4842-a03d-2a1a04d1010d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6809","nonce":3071977602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6811","nonce":3071977602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6815","nonce":3071977602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6813","nonce":3071977602}]},"public_addr":"172.21.15.134:6809/3071977602","cluster_addr":"172.21.15.134:6811/3071977602","heartbeat_back_addr":"172.21.15.134:6815/3071977602","heartbeat_front_addr":"172.21.15.134:6813/3071977602","state":["exists","up"]},{"osd":6,"uuid":"bb0c17a9-6a8e-4be6-8660-f73cfa8200ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6819","nonce":3819619299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6821","nonce":3819619299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6825","nonce":3819619299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6823","nonce":3819619299}]},"public_addr":"172.21.15.12:6819/3819619299","cluster_addr":"172.21.15.12:6821/3819619299","heartbeat_back_addr":"172.21.15.12:6825/3819619299","heartbeat_front_addr":"172.21.15.12:6823/3819619299","state":["exists","up"]},{"osd":7,"uuid":"0a93d6af-569e-4e45-b697-9e978c1c1004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6817","nonce":3113856282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6819","nonce":3113856282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6823","nonce":3113856282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6821","nonce":3113856282}]},"public_addr":"172.21.15.111:6817/3113856282","cluster_addr":"172.21.15.111:6819/3113856282","heartbeat_back_addr":"172.21.15.111:6823/3113856282","heartbeat_front_addr":"172.21.15.111:6821/3113856282","state":["exists","up"]},{"osd":8,"uuid":"e21acc68-cf50-4f4f-b1ba-cf6c195b4955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6817","nonce":1689362191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6819","nonce":1689362191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6823","nonce":1689362191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6821","nonce":1689362191}]},"public_addr":"172.21.15.134:6817/1689362191","cluster_addr":"172.21.15.134:6819/1689362191","heartbeat_back_addr":"172.21.15.134:6823/1689362191","heartbeat_front_addr":"172.21.15.134:6821/1689362191","state":["exists","up"]},{"osd":9,"uuid":"fd745869-7f5f-48a8-b412-bb9c92b586e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6827","nonce":4292385779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6829","nonce":4292385779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6833","nonce":4292385779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6831","nonce":4292385779}]},"public_addr":"172.21.15.12:6827/4292385779","cluster_addr":"172.21.15.12:6829/4292385779","heartbeat_back_addr":"172.21.15.12:6833/4292385779","heartbeat_front_addr":"172.21.15.12:6831/4292385779","state":["exists","up"]},{"osd":10,"uuid":"798b1abd-f6c7-4a5f-8da0-fdae49e50d2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6825","nonce":3338566976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6827","nonce":3338566976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6831","nonce":3338566976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6829","nonce":3338566976}]},"public_addr":"172.21.15.111:6825/3338566976","cluster_addr":"172.21.15.111:6827/3338566976","heartbeat_back_addr":"172.21.15.111:6831/3338566976","heartbeat_front_addr":"172.21.15.111:6829/3338566976","state":["exists","up"]},{"osd":11,"uuid":"b18fc5f3-1049-4871-b060-e99290273cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6825","nonce":578640428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6827","nonce":578640428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6831","nonce":578640428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6829","nonce":578640428}]},"public_addr":"172.21.15.134:6825/578640428","cluster_addr":"172.21.15.134:6827/578640428","heartbeat_back_addr":"172.21.15.134:6831/578640428","heartbeat_front_addr":"172.21.15.134:6829/578640428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:16:40.828608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:05.052528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:28.297316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:53.496497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:17.947259+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:41.712894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:07.105945+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:32.129688+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:56.008516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:22.239867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:47.334692+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:12.301276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/2593601362":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/2519382069":"2024-04-23T13:15:04.279990+0000","172.21.15.12:6801/2009831940":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3509412187":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1007749062":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3588480095":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1904962176":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2009831940":"2024-04-23T13:15:04.279990+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:23:30.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.899+0000 7fdb84e48640 1 -- 172.21.15.12:0/71034727 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fdb680a3c40 msgr2=0x7fdb680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:30.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.899+0000 7fdb84e48640 1 --2- 172.21.15.12:0/71034727 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fdb680a3c40 0x7fdb680a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdb80104b60 tx=0x7fdb74005e70 comp rx=0 tx=0).stop 2024-04-22T13:23:30.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.899+0000 7fdb84e48640 1 -- 172.21.15.12:0/71034727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb80100e70 msgr2=0x7fdb80104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:30.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.899+0000 7fdb84e48640 1 --2- 172.21.15.12:0/71034727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb80100e70 0x7fdb80104df0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000eab0 tx=0x7fdb7000ef70 comp rx=0 tx=0).stop 2024-04-22T13:23:30.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.899+0000 7fdb84e48640 1 -- 172.21.15.12:0/71034727 shutdown_connections 2024-04-22T13:23:30.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.899+0000 7fdb84e48640 1 --2- 172.21.15.12:0/71034727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb80102980 0x7fdb80103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:30.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.899+0000 7fdb84e48640 1 --2- 172.21.15.12:0/71034727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb80101fc0 0x7fdb80103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:30.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.899+0000 7fdb84e48640 1 --2- 172.21.15.12:0/71034727 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fdb680a3c40 0x7fdb680a6100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:30.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.899+0000 7fdb84e48640 1 --2- 172.21.15.12:0/71034727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb80100e70 0x7fdb80104df0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:30.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.899+0000 7fdb84e48640 1 -- 172.21.15.12:0/71034727 >> 172.21.15.12:0/71034727 conn(0x7fdb800fc3e0 msgr2=0x7fdb800fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:30.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.899+0000 7fdb84e48640 1 -- 172.21.15.12:0/71034727 shutdown_connections 2024-04-22T13:23:30.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:30.899+0000 7fdb84e48640 1 -- 172.21.15.12:0/71034727 wait complete. 2024-04-22T13:23:30.905 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:30 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2026031197' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:23:30.905 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:30 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1363765627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:23:30.905 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:30 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1905515635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:23:30.907 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T13:17:32.372434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T13:22:02.768799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T13:22:03.578730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T13:23:30.907 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T13:23:30.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:30 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2026031197' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:23:30.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:30 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1363765627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:23:30.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:30 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1905515635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:23:30.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:30 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2026031197' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:23:30.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:30 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1363765627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:23:30.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:30 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1905515635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:23:31.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.015+0000 7f12065db640 1 -- 172.21.15.12:0/2777644666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1200100e90 msgr2=0x7f12001012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:31.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.015+0000 7f12065db640 1 --2- 172.21.15.12:0/2777644666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1200100e90 0x7f12001012b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009900 tx=0x7f11f002bc30 comp rx=0 tx=0).stop 2024-04-22T13:23:31.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.015+0000 7f12065db640 1 -- 172.21.15.12:0/2777644666 shutdown_connections 2024-04-22T13:23:31.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.015+0000 7f12065db640 1 --2- 172.21.15.12:0/2777644666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12001029a0 0x7f12001092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.015+0000 7f12065db640 1 --2- 172.21.15.12:0/2777644666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1200101fe0 0x7f1200102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.015+0000 7f12065db640 1 --2- 172.21.15.12:0/2777644666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1200100e90 0x7f12001012b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.015+0000 7f12065db640 1 -- 172.21.15.12:0/2777644666 >> 172.21.15.12:0/2777644666 conn(0x7f12000fc400 msgr2=0x7f12000fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:31.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.015+0000 7f12065db640 1 -- 172.21.15.12:0/2777644666 shutdown_connections 2024-04-22T13:23:31.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.015+0000 7f12065db640 1 -- 172.21.15.12:0/2777644666 wait complete. 2024-04-22T13:23:31.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.016+0000 7f12065db640 1 Processor -- start 2024-04-22T13:23:31.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.016+0000 7f12065db640 1 -- start start 2024-04-22T13:23:31.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.016+0000 7f12065db640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1200100e90 0x7f1200069c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:31.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.016+0000 7f12065db640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1200101fe0 0x7f120019cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:31.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.016+0000 7f12065db640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12001029a0 0x7f120019f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:31.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.016+0000 7f12065db640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f120010ca40 con 0x7f1200100e90 2024-04-22T13:23:31.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.016+0000 7f12065db640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f120010c8c0 con 0x7f12001029a0 2024-04-22T13:23:31.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.016+0000 7f12065db640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f120010cbc0 con 0x7f1200101fe0 2024-04-22T13:23:31.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.016+0000 7f11fffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1200100e90 0x7f1200069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:31.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.016+0000 7f11fffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1200100e90 0x7f1200069c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47890/0 (socket says 172.21.15.12:47890) 2024-04-22T13:23:31.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.016+0000 7f11fffff640 1 -- 172.21.15.12:0/1136941171 learned_addr learned my addr 172.21.15.12:0/1136941171 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:31.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.017+0000 7f1204b78640 1 --2- 172.21.15.12:0/1136941171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12001029a0 0x7f120019f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:31.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.017+0000 7f11ff7fe640 1 --2- 172.21.15.12:0/1136941171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1200101fe0 0x7f120019cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:31.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.017+0000 7f11fffff640 1 -- 172.21.15.12:0/1136941171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1200101fe0 msgr2=0x7f120019cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:31.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.017+0000 7f11fffff640 1 --2- 172.21.15.12:0/1136941171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1200101fe0 0x7f120019cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.017+0000 7f11fffff640 1 -- 172.21.15.12:0/1136941171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12001029a0 msgr2=0x7f120019f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:31.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.017+0000 7f11fffff640 1 --2- 172.21.15.12:0/1136941171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12001029a0 0x7f120019f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.017+0000 7f11fffff640 1 -- 172.21.15.12:0/1136941171 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120019fe30 con 0x7f1200100e90 2024-04-22T13:23:31.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.017+0000 7f1204b78640 1 --2- 172.21.15.12:0/1136941171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12001029a0 0x7f120019f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:23:31.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.017+0000 7f11fffff640 1 --2- 172.21.15.12:0/1136941171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1200100e90 0x7f1200069c70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f11f0036040 tx=0x7f11f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:31.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.017+0000 7f11fd7fa640 1 -- 172.21.15.12:0/1136941171 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f0043020 con 0x7f1200100e90 2024-04-22T13:23:31.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.017+0000 7f12065db640 1 -- 172.21.15.12:0/1136941171 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12001a00c0 con 0x7f1200100e90 2024-04-22T13:23:31.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.017+0000 7f11fd7fa640 1 -- 172.21.15.12:0/1136941171 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11f00054c0 con 0x7f1200100e90 2024-04-22T13:23:31.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.017+0000 7f11fd7fa640 1 -- 172.21.15.12:0/1136941171 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f0003db0 con 0x7f1200100e90 2024-04-22T13:23:31.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.017+0000 7f12065db640 1 -- 172.21.15.12:0/1136941171 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12001a0690 con 0x7f1200100e90 2024-04-22T13:23:31.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.018+0000 7f12065db640 1 -- 172.21.15.12:0/1136941171 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12001081a0 con 0x7f1200100e90 2024-04-22T13:23:31.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.020+0000 7f11fd7fa640 1 -- 172.21.15.12:0/1136941171 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f11f0046020 con 0x7f1200100e90 2024-04-22T13:23:31.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.021+0000 7f11fd7fa640 1 --2- 172.21.15.12:0/1136941171 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f11e80a3c40 0x7f11e80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:31.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.021+0000 7f11ff7fe640 1 --2- 172.21.15.12:0/1136941171 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f11e80a3c40 0x7f11e80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:31.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.021+0000 7f11fd7fa640 1 -- 172.21.15.12:0/1136941171 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f11f00f2e00 con 0x7f1200100e90 2024-04-22T13:23:31.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.021+0000 7f11ff7fe640 1 --2- 172.21.15.12:0/1136941171 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f11e80a3c40 0x7f11e80a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f11f40058c0 tx=0x7f11f4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:31.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.024+0000 7f11fd7fa640 1 -- 172.21.15.12:0/1136941171 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f11f004d020 con 0x7f1200100e90 2024-04-22T13:23:31.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.209+0000 7f12065db640 1 -- 172.21.15.12:0/1136941171 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f120002d810 con 0x7f1200100e90 2024-04-22T13:23:31.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.210+0000 7f11fd7fa640 1 -- 172.21.15.12:0/1136941171 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f11f00bf760 con 0x7f1200100e90 2024-04-22T13:23:31.206 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-22T13:23:31.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.211+0000 7f12065db640 1 -- 172.21.15.12:0/1136941171 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f11e80a3c40 msgr2=0x7f11e80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:31.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.211+0000 7f12065db640 1 --2- 172.21.15.12:0/1136941171 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f11e80a3c40 0x7f11e80a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f11f40058c0 tx=0x7f11f4005850 comp rx=0 tx=0).stop 2024-04-22T13:23:31.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.211+0000 7f12065db640 1 -- 172.21.15.12:0/1136941171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1200100e90 msgr2=0x7f1200069c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:31.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.211+0000 7f12065db640 1 --2- 172.21.15.12:0/1136941171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1200100e90 0x7f1200069c70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f11f0036040 tx=0x7f11f00042f0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.212+0000 7f12065db640 1 -- 172.21.15.12:0/1136941171 shutdown_connections 2024-04-22T13:23:31.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.212+0000 7f12065db640 1 --2- 172.21.15.12:0/1136941171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12001029a0 0x7f120019f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.212+0000 7f12065db640 1 --2- 172.21.15.12:0/1136941171 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1200101fe0 0x7f120019cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.212+0000 7f12065db640 1 --2- 172.21.15.12:0/1136941171 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f11e80a3c40 0x7f11e80a6100 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.212+0000 7f12065db640 1 --2- 172.21.15.12:0/1136941171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1200100e90 0x7f1200069c70 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.212+0000 7f12065db640 1 -- 172.21.15.12:0/1136941171 >> 172.21.15.12:0/1136941171 conn(0x7f12000fc400 msgr2=0x7f1200105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:31.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.212+0000 7f12065db640 1 -- 172.21.15.12:0/1136941171 shutdown_connections 2024-04-22T13:23:31.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.212+0000 7f12065db640 1 -- 172.21.15.12:0/1136941171 wait complete. 2024-04-22T13:23:31.219 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T13:23:31.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.327+0000 7f09906ce640 1 -- 172.21.15.12:0/2683674372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0988102a00 msgr2=0x7f0988109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:31.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.327+0000 7f09906ce640 1 --2- 172.21.15.12:0/2683674372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0988102a00 0x7f0988109320 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0980009ca0 tx=0x7f098002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.327+0000 7f09906ce640 1 -- 172.21.15.12:0/2683674372 shutdown_connections 2024-04-22T13:23:31.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.327+0000 7f09906ce640 1 --2- 172.21.15.12:0/2683674372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0988102a00 0x7f0988109320 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.327+0000 7f09906ce640 1 --2- 172.21.15.12:0/2683674372 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0988102040 0x7f09881024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.327+0000 7f09906ce640 1 --2- 172.21.15.12:0/2683674372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0988100e60 0x7f0988101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.327+0000 7f09906ce640 1 -- 172.21.15.12:0/2683674372 >> 172.21.15.12:0/2683674372 conn(0x7f09880fc440 msgr2=0x7f09880fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:31.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.327+0000 7f09906ce640 1 -- 172.21.15.12:0/2683674372 shutdown_connections 2024-04-22T13:23:31.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.327+0000 7f09906ce640 1 -- 172.21.15.12:0/2683674372 wait complete. 2024-04-22T13:23:31.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.328+0000 7f09906ce640 1 Processor -- start 2024-04-22T13:23:31.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.328+0000 7f09906ce640 1 -- start start 2024-04-22T13:23:31.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.328+0000 7f09906ce640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0988100e60 0x7f0988198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:31.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.328+0000 7f09906ce640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0988102040 0x7f0988198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:31.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.328+0000 7f09906ce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0988102a00 0x7f098819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:31.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.328+0000 7f09906ce640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f098810db30 con 0x7f0988102040 2024-04-22T13:23:31.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.328+0000 7f09906ce640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f098810d9b0 con 0x7f0988102a00 2024-04-22T13:23:31.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.328+0000 7f09906ce640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f098810dcb0 con 0x7f0988100e60 2024-04-22T13:23:31.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.328+0000 7f098dc69640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0988102040 0x7f0988198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:31.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.328+0000 7f098dc69640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0988102040 0x7f0988198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:47912/0 (socket says 172.21.15.12:47912) 2024-04-22T13:23:31.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.328+0000 7f098dc69640 1 -- 172.21.15.12:0/2694657095 learned_addr learned my addr 172.21.15.12:0/2694657095 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:31.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.329+0000 7f098e46a640 1 --2- 172.21.15.12:0/2694657095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0988100e60 0x7f0988198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:31.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.329+0000 7f098dc69640 1 -- 172.21.15.12:0/2694657095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0988100e60 msgr2=0x7f0988198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:31.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.329+0000 7f098dc69640 1 --2- 172.21.15.12:0/2694657095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0988100e60 0x7f0988198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.329+0000 7f098dc69640 1 -- 172.21.15.12:0/2694657095 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0988102a00 msgr2=0x7f098819f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:31.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.329+0000 7f098ec6b640 1 --2- 172.21.15.12:0/2694657095 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0988102a00 0x7f098819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:31.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.329+0000 7f098dc69640 1 --2- 172.21.15.12:0/2694657095 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0988102a00 0x7f098819f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.329+0000 7f098dc69640 1 -- 172.21.15.12:0/2694657095 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f098819fd40 con 0x7f0988102040 2024-04-22T13:23:31.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.329+0000 7f098e46a640 1 --2- 172.21.15.12:0/2694657095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0988100e60 0x7f0988198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:23:31.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.329+0000 7f098dc69640 1 --2- 172.21.15.12:0/2694657095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0988102040 0x7f0988198660 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f098400c920 tx=0x7f098400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:31.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.329+0000 7f097f7fe640 1 -- 172.21.15.12:0/2694657095 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0984007c70 con 0x7f0988102040 2024-04-22T13:23:31.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.329+0000 7f097f7fe640 1 -- 172.21.15.12:0/2694657095 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0984007e10 con 0x7f0988102040 2024-04-22T13:23:31.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.329+0000 7f09906ce640 1 -- 172.21.15.12:0/2694657095 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f098819ffd0 con 0x7f0988102040 2024-04-22T13:23:31.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.329+0000 7f097f7fe640 1 -- 172.21.15.12:0/2694657095 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0984018820 con 0x7f0988102040 2024-04-22T13:23:31.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.329+0000 7f09906ce640 1 -- 172.21.15.12:0/2694657095 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09881a0540 con 0x7f0988102040 2024-04-22T13:23:31.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.330+0000 7f09906ce640 1 -- 172.21.15.12:0/2694657095 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09881081a0 con 0x7f0988102040 2024-04-22T13:23:31.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.331+0000 7f097f7fe640 1 -- 172.21.15.12:0/2694657095 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f098400f450 con 0x7f0988102040 2024-04-22T13:23:31.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.332+0000 7f097f7fe640 1 --2- 172.21.15.12:0/2694657095 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f09740a3cc0 0x7f09740a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:31.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.332+0000 7f098e46a640 1 --2- 172.21.15.12:0/2694657095 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f09740a3cc0 0x7f09740a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:31.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.332+0000 7f097f7fe640 1 -- 172.21.15.12:0/2694657095 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f09840d0730 con 0x7f0988102040 2024-04-22T13:23:31.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.332+0000 7f098e46a640 1 --2- 172.21.15.12:0/2694657095 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f09740a3cc0 0x7f09740a6180 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0988101bb0 tx=0x7f0978005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:31.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.336+0000 7f097f7fe640 1 -- 172.21.15.12:0/2694657095 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0984014030 con 0x7f0988102040 2024-04-22T13:23:31.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.529+0000 7f09906ce640 1 -- 172.21.15.12:0/2694657095 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f098802d7b0 con 0x7f0988102040 2024-04-22T13:23:31.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.530+0000 7f097f7fe640 1 -- 172.21.15.12:0/2694657095 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f098409cf80 con 0x7f0988102040 2024-04-22T13:23:31.526 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-22T13:23:31.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.532+0000 7f09906ce640 1 -- 172.21.15.12:0/2694657095 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f09740a3cc0 msgr2=0x7f09740a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:31.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.532+0000 7f09906ce640 1 --2- 172.21.15.12:0/2694657095 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f09740a3cc0 0x7f09740a6180 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0988101bb0 tx=0x7f0978005e70 comp rx=0 tx=0).stop 2024-04-22T13:23:31.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.532+0000 7f09906ce640 1 -- 172.21.15.12:0/2694657095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0988102040 msgr2=0x7f0988198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:31.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.532+0000 7f09906ce640 1 --2- 172.21.15.12:0/2694657095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0988102040 0x7f0988198660 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f098400c920 tx=0x7f098400cde0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.532+0000 7f09906ce640 1 -- 172.21.15.12:0/2694657095 shutdown_connections 2024-04-22T13:23:31.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.532+0000 7f09906ce640 1 --2- 172.21.15.12:0/2694657095 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0988102a00 0x7f098819f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.532+0000 7f09906ce640 1 --2- 172.21.15.12:0/2694657095 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f09740a3cc0 0x7f09740a6180 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.532+0000 7f09906ce640 1 --2- 172.21.15.12:0/2694657095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0988102040 0x7f0988198660 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.532+0000 7f09906ce640 1 --2- 172.21.15.12:0/2694657095 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0988100e60 0x7f0988198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.532+0000 7f09906ce640 1 -- 172.21.15.12:0/2694657095 >> 172.21.15.12:0/2694657095 conn(0x7f09880fc440 msgr2=0x7f0988105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:31.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.532+0000 7f09906ce640 1 -- 172.21.15.12:0/2694657095 shutdown_connections 2024-04-22T13:23:31.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.532+0000 7f09906ce640 1 -- 172.21.15.12:0/2694657095 wait complete. 2024-04-22T13:23:31.539 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T13:23:31.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.648+0000 7fc7bded5640 1 -- 172.21.15.12:0/1334114109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7b8100e90 msgr2=0x7fc7b81012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:31.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.648+0000 7fc7bded5640 1 --2- 172.21.15.12:0/1334114109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7b8100e90 0x7fc7b81012b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8009900 tx=0x7fc7a802bc30 comp rx=0 tx=0).stop 2024-04-22T13:23:31.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.648+0000 7fc7bded5640 1 -- 172.21.15.12:0/1334114109 shutdown_connections 2024-04-22T13:23:31.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.648+0000 7fc7bded5640 1 --2- 172.21.15.12:0/1334114109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b81029a0 0x7fc7b81092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.648+0000 7fc7bded5640 1 --2- 172.21.15.12:0/1334114109 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7b8101fe0 0x7fc7b8102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.648+0000 7fc7bded5640 1 --2- 172.21.15.12:0/1334114109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7b8100e90 0x7fc7b81012b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.648+0000 7fc7bded5640 1 -- 172.21.15.12:0/1334114109 >> 172.21.15.12:0/1334114109 conn(0x7fc7b80fc400 msgr2=0x7fc7b80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:31.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.648+0000 7fc7bded5640 1 -- 172.21.15.12:0/1334114109 shutdown_connections 2024-04-22T13:23:31.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.648+0000 7fc7bded5640 1 -- 172.21.15.12:0/1334114109 wait complete. 2024-04-22T13:23:31.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.648+0000 7fc7bded5640 1 Processor -- start 2024-04-22T13:23:31.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.649+0000 7fc7bded5640 1 -- start start 2024-04-22T13:23:31.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.649+0000 7fc7bded5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7b8100e90 0x7fc7b8198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:31.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.649+0000 7fc7bded5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b8101fe0 0x7fc7b8198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:31.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.649+0000 7fc7bded5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7b81029a0 0x7fc7b819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:31.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.649+0000 7fc7bded5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b810c9c0 con 0x7fc7b81029a0 2024-04-22T13:23:31.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.649+0000 7fc7bded5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b810c840 con 0x7fc7b8100e90 2024-04-22T13:23:31.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.649+0000 7fc7bded5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b810cb40 con 0x7fc7b8101fe0 2024-04-22T13:23:31.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.649+0000 7fc7b7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7b81029a0 0x7fc7b819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:31.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.649+0000 7fc7b7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7b81029a0 0x7fc7b819f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34750/0 (socket says 172.21.15.12:34750) 2024-04-22T13:23:31.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.649+0000 7fc7b7fff640 1 -- 172.21.15.12:0/2531868478 learned_addr learned my addr 172.21.15.12:0/2531868478 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:31.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.649+0000 7fc7b6ffd640 1 --2- 172.21.15.12:0/2531868478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b8101fe0 0x7fc7b8198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:31.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.649+0000 7fc7b77fe640 1 --2- 172.21.15.12:0/2531868478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7b8100e90 0x7fc7b8198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:31.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.649+0000 7fc7b7fff640 1 -- 172.21.15.12:0/2531868478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b8101fe0 msgr2=0x7fc7b8198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:31.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.649+0000 7fc7b7fff640 1 --2- 172.21.15.12:0/2531868478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b8101fe0 0x7fc7b8198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.649+0000 7fc7b7fff640 1 -- 172.21.15.12:0/2531868478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7b8100e90 msgr2=0x7fc7b8198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:31.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.649+0000 7fc7b7fff640 1 --2- 172.21.15.12:0/2531868478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7b8100e90 0x7fc7b8198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.649+0000 7fc7b7fff640 1 -- 172.21.15.12:0/2531868478 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b819fe30 con 0x7fc7b81029a0 2024-04-22T13:23:31.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.649+0000 7fc7b6ffd640 1 --2- 172.21.15.12:0/2531868478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b8101fe0 0x7fc7b8198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:23:31.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.649+0000 7fc7b7fff640 1 --2- 172.21.15.12:0/2531868478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7b81029a0 0x7fc7b819f7c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000cad0 tx=0x7fc7b000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:31.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.649+0000 7fc7b4ff9640 1 -- 172.21.15.12:0/2531868478 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b000ae20 con 0x7fc7b81029a0 2024-04-22T13:23:31.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.650+0000 7fc7b4ff9640 1 -- 172.21.15.12:0/2531868478 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7b0005020 con 0x7fc7b81029a0 2024-04-22T13:23:31.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.650+0000 7fc7bded5640 1 -- 172.21.15.12:0/2531868478 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b81a0120 con 0x7fc7b81029a0 2024-04-22T13:23:31.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.650+0000 7fc7b4ff9640 1 -- 172.21.15.12:0/2531868478 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b000f6b0 con 0x7fc7b81029a0 2024-04-22T13:23:31.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.650+0000 7fc7bded5640 1 -- 172.21.15.12:0/2531868478 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b81a0620 con 0x7fc7b81029a0 2024-04-22T13:23:31.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.651+0000 7fc7bded5640 1 -- 172.21.15.12:0/2531868478 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7b81a08e0 con 0x7fc7b81029a0 2024-04-22T13:23:31.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.653+0000 7fc7b4ff9640 1 -- 172.21.15.12:0/2531868478 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc7b00051d0 con 0x7fc7b81029a0 2024-04-22T13:23:31.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.653+0000 7fc7b4ff9640 1 --2- 172.21.15.12:0/2531868478 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc7a00a3c40 0x7fc7a00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:31.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.653+0000 7fc7b77fe640 1 --2- 172.21.15.12:0/2531868478 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc7a00a3c40 0x7fc7a00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:31.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.653+0000 7fc7b4ff9640 1 -- 172.21.15.12:0/2531868478 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fc7b00d0a60 con 0x7fc7b81029a0 2024-04-22T13:23:31.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.654+0000 7fc7b77fe640 1 --2- 172.21.15.12:0/2531868478 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc7a00a3c40 0x7fc7a00a6100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8000c00 tx=0x7fc7a802c800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:31.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.657+0000 7fc7b4ff9640 1 -- 172.21.15.12:0/2531868478 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc7b0014030 con 0x7fc7b81029a0 2024-04-22T13:23:31.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:31 smithi012 ceph-mon[28265]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-22T13:23:31.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:31 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/71034727' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:23:31.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:31 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1136941171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:23:31.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:31 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2694657095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:23:31.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.850+0000 7fc7bded5640 1 -- 172.21.15.12:0/2531868478 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc7b81a0ab0 con 0x7fc7b81029a0 2024-04-22T13:23:31.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.851+0000 7fc7b4ff9640 1 -- 172.21.15.12:0/2531868478 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7fc7b009d310 con 0x7fc7b81029a0 2024-04-22T13:23:31.847 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-22T13:23:31.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.853+0000 7fc7bded5640 1 -- 172.21.15.12:0/2531868478 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc7a00a3c40 msgr2=0x7fc7a00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:31.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.853+0000 7fc7bded5640 1 --2- 172.21.15.12:0/2531868478 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc7a00a3c40 0x7fc7a00a6100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8000c00 tx=0x7fc7a802c800 comp rx=0 tx=0).stop 2024-04-22T13:23:31.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.853+0000 7fc7bded5640 1 -- 172.21.15.12:0/2531868478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7b81029a0 msgr2=0x7fc7b819f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:31.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.853+0000 7fc7bded5640 1 --2- 172.21.15.12:0/2531868478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7b81029a0 0x7fc7b819f7c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000cad0 tx=0x7fc7b000cf90 comp rx=0 tx=0).stop 2024-04-22T13:23:31.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.853+0000 7fc7bded5640 1 -- 172.21.15.12:0/2531868478 shutdown_connections 2024-04-22T13:23:31.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.853+0000 7fc7bded5640 1 --2- 172.21.15.12:0/2531868478 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc7a00a3c40 0x7fc7a00a6100 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.853+0000 7fc7bded5640 1 --2- 172.21.15.12:0/2531868478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7b81029a0 0x7fc7b819f7c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.853+0000 7fc7bded5640 1 --2- 172.21.15.12:0/2531868478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b8101fe0 0x7fc7b8198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.853+0000 7fc7bded5640 1 --2- 172.21.15.12:0/2531868478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7b8100e90 0x7fc7b8198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.853+0000 7fc7bded5640 1 -- 172.21.15.12:0/2531868478 >> 172.21.15.12:0/2531868478 conn(0x7fc7b80fc400 msgr2=0x7fc7b8069110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:31.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.853+0000 7fc7bded5640 1 -- 172.21.15.12:0/2531868478 shutdown_connections 2024-04-22T13:23:31.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.853+0000 7fc7bded5640 1 -- 172.21.15.12:0/2531868478 wait complete. 2024-04-22T13:23:31.861 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:23:31.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:31 smithi134 ceph-mon[31338]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-22T13:23:31.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:31 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/71034727' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:23:31.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:31 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1136941171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:23:31.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:31 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2694657095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:23:31.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.965+0000 7f7069c81640 1 -- 172.21.15.12:0/2435411404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7064107c60 msgr2=0x7f706410a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:31.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.965+0000 7f7069c81640 1 --2- 172.21.15.12:0/2435411404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7064107c60 0x7f706410a0b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f705c009ca0 tx=0x7f705c02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.965+0000 7f7069c81640 1 -- 172.21.15.12:0/2435411404 shutdown_connections 2024-04-22T13:23:31.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.965+0000 7f7069c81640 1 --2- 172.21.15.12:0/2435411404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7064107c60 0x7f706410a0b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.965+0000 7f7069c81640 1 --2- 172.21.15.12:0/2435411404 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7064105330 0x7f7064107720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.965+0000 7f7069c81640 1 --2- 172.21.15.12:0/2435411404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7064102a00 0x7f7064104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.965+0000 7f7069c81640 1 -- 172.21.15.12:0/2435411404 >> 172.21.15.12:0/2435411404 conn(0x7f70640fc410 msgr2=0x7f70640fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:31.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.965+0000 7f7069c81640 1 -- 172.21.15.12:0/2435411404 shutdown_connections 2024-04-22T13:23:31.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.965+0000 7f7069c81640 1 -- 172.21.15.12:0/2435411404 wait complete. 2024-04-22T13:23:31.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.966+0000 7f7069c81640 1 Processor -- start 2024-04-22T13:23:31.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.966+0000 7f7069c81640 1 -- start start 2024-04-22T13:23:31.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.966+0000 7f7069c81640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7064102a00 0x7f7064198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:31.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.966+0000 7f7069c81640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7064105330 0x7f7064198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:31.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.966+0000 7f7069c81640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7064107c60 0x7f706419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:31.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.966+0000 7f7069c81640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f706410dad0 con 0x7f7064102a00 2024-04-22T13:23:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.966+0000 7f7069c81640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f706410d950 con 0x7f7064105330 2024-04-22T13:23:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.966+0000 7f7069c81640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f706410dc50 con 0x7f7064107c60 2024-04-22T13:23:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.966+0000 7f70637fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7064102a00 0x7f7064198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.966+0000 7f70637fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7064102a00 0x7f7064198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34766/0 (socket says 172.21.15.12:34766) 2024-04-22T13:23:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.966+0000 7f70637fe640 1 -- 172.21.15.12:0/2187005344 learned_addr learned my addr 172.21.15.12:0/2187005344 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.966+0000 7f7062ffd640 1 --2- 172.21.15.12:0/2187005344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7064105330 0x7f7064198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.967+0000 7f7063fff640 1 --2- 172.21.15.12:0/2187005344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7064107c60 0x7f706419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.967+0000 7f70637fe640 1 -- 172.21.15.12:0/2187005344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7064107c60 msgr2=0x7f706419f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.967+0000 7f70637fe640 1 --2- 172.21.15.12:0/2187005344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7064107c60 0x7f706419f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.967+0000 7f70637fe640 1 -- 172.21.15.12:0/2187005344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7064105330 msgr2=0x7f7064198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.967+0000 7f70637fe640 1 --2- 172.21.15.12:0/2187005344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7064105330 0x7f7064198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.967+0000 7f70637fe640 1 -- 172.21.15.12:0/2187005344 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f706419fd70 con 0x7f7064102a00 2024-04-22T13:23:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.967+0000 7f70637fe640 1 --2- 172.21.15.12:0/2187005344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7064102a00 0x7f7064198150 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f705400c970 tx=0x7f705400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:31.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.967+0000 7f7060ff9640 1 -- 172.21.15.12:0/2187005344 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7054007bb0 con 0x7f7064102a00 2024-04-22T13:23:31.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.967+0000 7f7060ff9640 1 -- 172.21.15.12:0/2187005344 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7054007d50 con 0x7f7064102a00 2024-04-22T13:23:31.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.967+0000 7f7060ff9640 1 -- 172.21.15.12:0/2187005344 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705400f450 con 0x7f7064102a00 2024-04-22T13:23:31.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.967+0000 7f7069c81640 1 -- 172.21.15.12:0/2187005344 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70641a0060 con 0x7f7064102a00 2024-04-22T13:23:31.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.967+0000 7f7069c81640 1 -- 172.21.15.12:0/2187005344 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70641a0560 con 0x7f7064102a00 2024-04-22T13:23:31.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.968+0000 7f7069c81640 1 -- 172.21.15.12:0/2187005344 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70640fdf00 con 0x7f7064102a00 2024-04-22T13:23:31.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.969+0000 7f7060ff9640 1 -- 172.21.15.12:0/2187005344 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7054004f20 con 0x7f7064102a00 2024-04-22T13:23:31.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.970+0000 7f7060ff9640 1 --2- 172.21.15.12:0/2187005344 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f704c0a3c40 0x7f704c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:31.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.970+0000 7f7062ffd640 1 --2- 172.21.15.12:0/2187005344 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f704c0a3c40 0x7f704c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:31.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.970+0000 7f7060ff9640 1 -- 172.21.15.12:0/2187005344 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f70540d0330 con 0x7f7064102a00 2024-04-22T13:23:31.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.970+0000 7f7062ffd640 1 --2- 172.21.15.12:0/2187005344 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f704c0a3c40 0x7f704c0a6100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7064199660 tx=0x7f705800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:31.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:31.974+0000 7f7060ff9640 1 -- 172.21.15.12:0/2187005344 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7054014030 con 0x7f7064102a00 2024-04-22T13:23:31.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:31 smithi111 ceph-mon[31361]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-22T13:23:31.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:31 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/71034727' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:23:31.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:31 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1136941171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:23:31.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:31 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2694657095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:23:32.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.200+0000 7f7069c81640 1 -- 172.21.15.12:0/2187005344 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f706402d5c0 con 0x7f7064102a00 2024-04-22T13:23:32.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.201+0000 7f7060ff9640 1 -- 172.21.15.12:0/2187005344 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11942 (secure 0 0 0) 0x7f705409cb80 con 0x7f7064102a00 2024-04-22T13:23:32.197 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:23:32.197 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:22:12.398056+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24448,24509,24443,24484,24407]},"id":1}]} 2024-04-22T13:23:32.197 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 27 2024-04-22T13:23:32.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.203+0000 7f7069c81640 1 -- 172.21.15.12:0/2187005344 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f704c0a3c40 msgr2=0x7f704c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:32.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.203+0000 7f7069c81640 1 --2- 172.21.15.12:0/2187005344 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f704c0a3c40 0x7f704c0a6100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7064199660 tx=0x7f705800b390 comp rx=0 tx=0).stop 2024-04-22T13:23:32.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.203+0000 7f7069c81640 1 -- 172.21.15.12:0/2187005344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7064102a00 msgr2=0x7f7064198150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:32.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.203+0000 7f7069c81640 1 --2- 172.21.15.12:0/2187005344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7064102a00 0x7f7064198150 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f705400c970 tx=0x7f705400ce30 comp rx=0 tx=0).stop 2024-04-22T13:23:32.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.203+0000 7f7069c81640 1 -- 172.21.15.12:0/2187005344 shutdown_connections 2024-04-22T13:23:32.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.203+0000 7f7069c81640 1 --2- 172.21.15.12:0/2187005344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7064107c60 0x7f706419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.203+0000 7f7069c81640 1 --2- 172.21.15.12:0/2187005344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7064105330 0x7f7064198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.203+0000 7f7069c81640 1 --2- 172.21.15.12:0/2187005344 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f704c0a3c40 0x7f704c0a6100 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.203+0000 7f7069c81640 1 --2- 172.21.15.12:0/2187005344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7064102a00 0x7f7064198150 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.203+0000 7f7069c81640 1 -- 172.21.15.12:0/2187005344 >> 172.21.15.12:0/2187005344 conn(0x7f70640fc410 msgr2=0x7f7064103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:32.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.203+0000 7f7069c81640 1 -- 172.21.15.12:0/2187005344 shutdown_connections 2024-04-22T13:23:32.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.203+0000 7f7069c81640 1 -- 172.21.15.12:0/2187005344 wait complete. 2024-04-22T13:23:32.211 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:23:32.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.317+0000 7fb116031640 1 -- 172.21.15.12:0/3997385423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb110101fc0 msgr2=0x7fb110102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:32.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.317+0000 7fb116031640 1 --2- 172.21.15.12:0/3997385423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb110101fc0 0x7fb110102440 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb104009900 tx=0x7fb10402bc30 comp rx=0 tx=0).stop 2024-04-22T13:23:32.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.317+0000 7fb116031640 1 -- 172.21.15.12:0/3997385423 shutdown_connections 2024-04-22T13:23:32.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.317+0000 7fb116031640 1 --2- 172.21.15.12:0/3997385423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb110102980 0x7fb1101092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.317+0000 7fb116031640 1 --2- 172.21.15.12:0/3997385423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb110101fc0 0x7fb110102440 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.317+0000 7fb116031640 1 --2- 172.21.15.12:0/3997385423 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb110100e70 0x7fb110101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.317+0000 7fb116031640 1 -- 172.21.15.12:0/3997385423 >> 172.21.15.12:0/3997385423 conn(0x7fb1100fc3e0 msgr2=0x7fb1100fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:32.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.317+0000 7fb116031640 1 -- 172.21.15.12:0/3997385423 shutdown_connections 2024-04-22T13:23:32.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.317+0000 7fb116031640 1 -- 172.21.15.12:0/3997385423 wait complete. 2024-04-22T13:23:32.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.318+0000 7fb116031640 1 Processor -- start 2024-04-22T13:23:32.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.318+0000 7fb116031640 1 -- start start 2024-04-22T13:23:32.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.318+0000 7fb116031640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb110100e70 0x7fb1101981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:32.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.318+0000 7fb116031640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb110101fc0 0x7fb110198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:32.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.318+0000 7fb116031640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb110102980 0x7fb11019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:32.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.318+0000 7fb116031640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb11010c9a0 con 0x7fb110102980 2024-04-22T13:23:32.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.318+0000 7fb116031640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb11010c820 con 0x7fb110100e70 2024-04-22T13:23:32.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.318+0000 7fb116031640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb11010cb20 con 0x7fb110101fc0 2024-04-22T13:23:32.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.318+0000 7fb10ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb110102980 0x7fb11019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:32.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.318+0000 7fb10ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb110102980 0x7fb11019f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34790/0 (socket says 172.21.15.12:34790) 2024-04-22T13:23:32.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.318+0000 7fb10ffff640 1 -- 172.21.15.12:0/3041653584 learned_addr learned my addr 172.21.15.12:0/3041653584 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:32.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.318+0000 7fb10f7fe640 1 --2- 172.21.15.12:0/3041653584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb110100e70 0x7fb1101981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:32.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.318+0000 7fb10effd640 1 --2- 172.21.15.12:0/3041653584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb110101fc0 0x7fb110198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:32.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.318+0000 7fb10ffff640 1 -- 172.21.15.12:0/3041653584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb110101fc0 msgr2=0x7fb110198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:32.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.318+0000 7fb10ffff640 1 --2- 172.21.15.12:0/3041653584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb110101fc0 0x7fb110198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.318+0000 7fb10ffff640 1 -- 172.21.15.12:0/3041653584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb110100e70 msgr2=0x7fb1101981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:32.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.318+0000 7fb10ffff640 1 --2- 172.21.15.12:0/3041653584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb110100e70 0x7fb1101981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.318+0000 7fb10ffff640 1 -- 172.21.15.12:0/3041653584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11019fe10 con 0x7fb110102980 2024-04-22T13:23:32.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.319+0000 7fb10ffff640 1 --2- 172.21.15.12:0/3041653584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb110102980 0x7fb11019f7a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb10800cad0 tx=0x7fb10800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:32.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.319+0000 7fb10cff9640 1 -- 172.21.15.12:0/3041653584 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10800ae20 con 0x7fb110102980 2024-04-22T13:23:32.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.319+0000 7fb10cff9640 1 -- 172.21.15.12:0/3041653584 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb108005020 con 0x7fb110102980 2024-04-22T13:23:32.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.319+0000 7fb116031640 1 -- 172.21.15.12:0/3041653584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1101a0100 con 0x7fb110102980 2024-04-22T13:23:32.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.319+0000 7fb10cff9640 1 -- 172.21.15.12:0/3041653584 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10800f6b0 con 0x7fb110102980 2024-04-22T13:23:32.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.319+0000 7fb116031640 1 -- 172.21.15.12:0/3041653584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1101a05d0 con 0x7fb110102980 2024-04-22T13:23:32.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.320+0000 7fb116031640 1 -- 172.21.15.12:0/3041653584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1101a0890 con 0x7fb110102980 2024-04-22T13:23:32.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.321+0000 7fb10cff9640 1 -- 172.21.15.12:0/3041653584 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb1080051d0 con 0x7fb110102980 2024-04-22T13:23:32.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.321+0000 7fb10cff9640 1 --2- 172.21.15.12:0/3041653584 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb0f80a3c40 0x7fb0f80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:32.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.322+0000 7fb10f7fe640 1 --2- 172.21.15.12:0/3041653584 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb0f80a3c40 0x7fb0f80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:32.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.322+0000 7fb10cff9640 1 -- 172.21.15.12:0/3041653584 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fb1080d0730 con 0x7fb110102980 2024-04-22T13:23:32.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.322+0000 7fb10f7fe640 1 --2- 172.21.15.12:0/3041653584 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb0f80a3c40 0x7fb0f80a6100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb1000058c0 tx=0x7fb10000a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:32.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.326+0000 7fb10cff9640 1 -- 172.21.15.12:0/3041653584 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb108014030 con 0x7fb110102980 2024-04-22T13:23:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.515+0000 7fb116031640 1 -- 172.21.15.12:0/3041653584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1101a0a60 con 0x7fb110102980 2024-04-22T13:23:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.517+0000 7fb10cff9640 1 -- 172.21.15.12:0/3041653584 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22620 (secure 0 0 0) 0x7fb10809cf80 con 0x7fb110102980 2024-04-22T13:23:32.513 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:23:32.514 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"0aede5ea-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:14:09.362918+0000","modified":"2024-04-22T13:23:05.500756+0000","last_up_change":"2024-04-22T13:21:15.263393+0000","last_in_change":"2024-04-22T13:20:54.174685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:17:32.372434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:22:02.768799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:22:03.578730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d9e2365-3a64-4b13-96fc-c217b1cee2ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6803","nonce":1191350632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6805","nonce":1191350632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6809","nonce":1191350632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6807","nonce":1191350632}]},"public_addr":"172.21.15.12:6803/1191350632","cluster_addr":"172.21.15.12:6805/1191350632","heartbeat_back_addr":"172.21.15.12:6809/1191350632","heartbeat_front_addr":"172.21.15.12:6807/1191350632","state":["exists","up"]},{"osd":1,"uuid":"1499c783-2bd7-48ce-a444-34add42bcf31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6801","nonce":417496465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6803","nonce":417496465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6807","nonce":417496465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6805","nonce":417496465}]},"public_addr":"172.21.15.111:6801/417496465","cluster_addr":"172.21.15.111:6803/417496465","heartbeat_back_addr":"172.21.15.111:6807/417496465","heartbeat_front_addr":"172.21.15.111:6805/417496465","state":["exists","up"]},{"osd":2,"uuid":"71a9ebc3-c622-434e-a6c2-f48358f384f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6801","nonce":1403153998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6803","nonce":1403153998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6807","nonce":1403153998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6805","nonce":1403153998}]},"public_addr":"172.21.15.134:6801/1403153998","cluster_addr":"172.21.15.134:6803/1403153998","heartbeat_back_addr":"172.21.15.134:6807/1403153998","heartbeat_front_addr":"172.21.15.134:6805/1403153998","state":["exists","up"]},{"osd":3,"uuid":"6dd00db2-a286-48ba-8666-56ffdd5f35ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6811","nonce":3359066681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6813","nonce":3359066681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6817","nonce":3359066681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6815","nonce":3359066681}]},"public_addr":"172.21.15.12:6811/3359066681","cluster_addr":"172.21.15.12:6813/3359066681","heartbeat_back_addr":"172.21.15.12:6817/3359066681","heartbeat_front_addr":"172.21.15.12:6815/3359066681","state":["exists","up"]},{"osd":4,"uuid":"1e29afd7-354c-4506-aaa0-b43eb1a19a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6809","nonce":2886349872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6811","nonce":2886349872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6815","nonce":2886349872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6813","nonce":2886349872}]},"public_addr":"172.21.15.111:6809/2886349872","cluster_addr":"172.21.15.111:6811/2886349872","heartbeat_back_addr":"172.21.15.111:6815/2886349872","heartbeat_front_addr":"172.21.15.111:6813/2886349872","state":["exists","up"]},{"osd":5,"uuid":"8f40f02d-0ba2-4842-a03d-2a1a04d1010d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6809","nonce":3071977602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6811","nonce":3071977602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6815","nonce":3071977602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6813","nonce":3071977602}]},"public_addr":"172.21.15.134:6809/3071977602","cluster_addr":"172.21.15.134:6811/3071977602","heartbeat_back_addr":"172.21.15.134:6815/3071977602","heartbeat_front_addr":"172.21.15.134:6813/3071977602","state":["exists","up"]},{"osd":6,"uuid":"bb0c17a9-6a8e-4be6-8660-f73cfa8200ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6819","nonce":3819619299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6821","nonce":3819619299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6825","nonce":3819619299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6823","nonce":3819619299}]},"public_addr":"172.21.15.12:6819/3819619299","cluster_addr":"172.21.15.12:6821/3819619299","heartbeat_back_addr":"172.21.15.12:6825/3819619299","heartbeat_front_addr":"172.21.15.12:6823/3819619299","state":["exists","up"]},{"osd":7,"uuid":"0a93d6af-569e-4e45-b697-9e978c1c1004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6817","nonce":3113856282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6819","nonce":3113856282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6823","nonce":3113856282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6821","nonce":3113856282}]},"public_addr":"172.21.15.111:6817/3113856282","cluster_addr":"172.21.15.111:6819/3113856282","heartbeat_back_addr":"172.21.15.111:6823/3113856282","heartbeat_front_addr":"172.21.15.111:6821/3113856282","state":["exists","up"]},{"osd":8,"uuid":"e21acc68-cf50-4f4f-b1ba-cf6c195b4955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6817","nonce":1689362191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6819","nonce":1689362191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6823","nonce":1689362191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6821","nonce":1689362191}]},"public_addr":"172.21.15.134:6817/1689362191","cluster_addr":"172.21.15.134:6819/1689362191","heartbeat_back_addr":"172.21.15.134:6823/1689362191","heartbeat_front_addr":"172.21.15.134:6821/1689362191","state":["exists","up"]},{"osd":9,"uuid":"fd745869-7f5f-48a8-b412-bb9c92b586e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6827","nonce":4292385779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6829","nonce":4292385779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6833","nonce":4292385779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6831","nonce":4292385779}]},"public_addr":"172.21.15.12:6827/4292385779","cluster_addr":"172.21.15.12:6829/4292385779","heartbeat_back_addr":"172.21.15.12:6833/4292385779","heartbeat_front_addr":"172.21.15.12:6831/4292385779","state":["exists","up"]},{"osd":10,"uuid":"798b1abd-f6c7-4a5f-8da0-fdae49e50d2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6825","nonce":3338566976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6827","nonce":3338566976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6831","nonce":3338566976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6829","nonce":3338566976}]},"public_addr":"172.21.15.111:6825/3338566976","cluster_addr":"172.21.15.111:6827/3338566976","heartbeat_back_addr":"172.21.15.111:6831/3338566976","heartbeat_front_addr":"172.21.15.111:6829/3338566976","state":["exists","up"]},{"osd":11,"uuid":"b18fc5f3-1049-4871-b060-e99290273cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6825","nonce":578640428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6827","nonce":578640428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6831","nonce":578640428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6829","nonce":578640428}]},"public_addr":"172.21.15.134:6825/578640428","cluster_addr":"172.21.15.134:6827/578640428","heartbeat_back_addr":"172.21.15.134:6831/578640428","heartbeat_front_addr":"172.21.15.134:6829/578640428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:16:40.828608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:05.052528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:28.297316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:53.496497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:17.947259+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:41.712894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:07.105945+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:32.129688+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:56.008516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:22.239867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:47.334692+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:12.301276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/2593601362":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/2519382069":"2024-04-23T13:15:04.279990+0000","172.21.15.12:6801/2009831940":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3509412187":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1007749062":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3588480095":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1904962176":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2009831940":"2024-04-23T13:15:04.279990+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:23:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.519+0000 7fb116031640 1 -- 172.21.15.12:0/3041653584 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb0f80a3c40 msgr2=0x7fb0f80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.519+0000 7fb116031640 1 --2- 172.21.15.12:0/3041653584 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb0f80a3c40 0x7fb0f80a6100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb1000058c0 tx=0x7fb10000a4c0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.519+0000 7fb116031640 1 -- 172.21.15.12:0/3041653584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb110102980 msgr2=0x7fb11019f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.519+0000 7fb116031640 1 --2- 172.21.15.12:0/3041653584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb110102980 0x7fb11019f7a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb10800cad0 tx=0x7fb10800cf90 comp rx=0 tx=0).stop 2024-04-22T13:23:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.519+0000 7fb116031640 1 -- 172.21.15.12:0/3041653584 shutdown_connections 2024-04-22T13:23:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.519+0000 7fb116031640 1 --2- 172.21.15.12:0/3041653584 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb0f80a3c40 0x7fb0f80a6100 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.519+0000 7fb116031640 1 --2- 172.21.15.12:0/3041653584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb110102980 0x7fb11019f7a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.519+0000 7fb116031640 1 --2- 172.21.15.12:0/3041653584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb110101fc0 0x7fb110198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.519+0000 7fb116031640 1 --2- 172.21.15.12:0/3041653584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb110100e70 0x7fb1101981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.519+0000 7fb116031640 1 -- 172.21.15.12:0/3041653584 >> 172.21.15.12:0/3041653584 conn(0x7fb1100fc3e0 msgr2=0x7fb110069130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:32.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.520+0000 7fb116031640 1 -- 172.21.15.12:0/3041653584 shutdown_connections 2024-04-22T13:23:32.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.520+0000 7fb116031640 1 -- 172.21.15.12:0/3041653584 wait complete. 2024-04-22T13:23:32.528 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:23:32.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.633+0000 7ff60abdb640 1 -- 172.21.15.12:0/2944102903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff604105320 msgr2=0x7ff604107710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:32.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.633+0000 7ff60abdb640 1 --2- 172.21.15.12:0/2944102903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff604105320 0x7ff604107710 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009900 tx=0x7ff5f802bc30 comp rx=0 tx=0).stop 2024-04-22T13:23:32.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.633+0000 7ff60abdb640 1 -- 172.21.15.12:0/2944102903 shutdown_connections 2024-04-22T13:23:32.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.633+0000 7ff60abdb640 1 --2- 172.21.15.12:0/2944102903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff604107c50 0x7ff60410a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.633+0000 7ff60abdb640 1 --2- 172.21.15.12:0/2944102903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff604105320 0x7ff604107710 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.633+0000 7ff60abdb640 1 --2- 172.21.15.12:0/2944102903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6041029f0 0x7ff604104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.633+0000 7ff60abdb640 1 -- 172.21.15.12:0/2944102903 >> 172.21.15.12:0/2944102903 conn(0x7ff6040fc3e0 msgr2=0x7ff6040fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:32.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.633+0000 7ff60abdb640 1 -- 172.21.15.12:0/2944102903 shutdown_connections 2024-04-22T13:23:32.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.633+0000 7ff60abdb640 1 -- 172.21.15.12:0/2944102903 wait complete. 2024-04-22T13:23:32.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.633+0000 7ff60abdb640 1 Processor -- start 2024-04-22T13:23:32.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.633+0000 7ff60abdb640 1 -- start start 2024-04-22T13:23:32.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.633+0000 7ff60abdb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6041029f0 0x7ff604198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:32.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.633+0000 7ff60abdb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff604105320 0x7ff604198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:32.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.633+0000 7ff60abdb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff604107c50 0x7ff60419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:32.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.633+0000 7ff60abdb640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff60410da80 con 0x7ff6041029f0 2024-04-22T13:23:32.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.633+0000 7ff60abdb640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff60410d900 con 0x7ff604107c50 2024-04-22T13:23:32.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.633+0000 7ff60abdb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff60410dc00 con 0x7ff604105320 2024-04-22T13:23:32.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.634+0000 7ff608977640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6041029f0 0x7ff604198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:32.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.634+0000 7ff608977640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6041029f0 0x7ff604198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34798/0 (socket says 172.21.15.12:34798) 2024-04-22T13:23:32.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.634+0000 7ff608977640 1 -- 172.21.15.12:0/2855791810 learned_addr learned my addr 172.21.15.12:0/2855791810 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:32.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.634+0000 7ff609178640 1 --2- 172.21.15.12:0/2855791810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff604107c50 0x7ff60419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:32.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.634+0000 7ff603fff640 1 --2- 172.21.15.12:0/2855791810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff604105320 0x7ff604198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:32.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.634+0000 7ff608977640 1 -- 172.21.15.12:0/2855791810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff604105320 msgr2=0x7ff604198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:32.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.634+0000 7ff608977640 1 --2- 172.21.15.12:0/2855791810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff604105320 0x7ff604198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.634+0000 7ff608977640 1 -- 172.21.15.12:0/2855791810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff604107c50 msgr2=0x7ff60419f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:32.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.634+0000 7ff608977640 1 --2- 172.21.15.12:0/2855791810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff604107c50 0x7ff60419f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.634+0000 7ff608977640 1 -- 172.21.15.12:0/2855791810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60419fdd0 con 0x7ff6041029f0 2024-04-22T13:23:32.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.634+0000 7ff609178640 1 --2- 172.21.15.12:0/2855791810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff604107c50 0x7ff60419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:23:32.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.634+0000 7ff608977640 1 --2- 172.21.15.12:0/2855791810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6041029f0 0x7ff604198200 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400d950 tx=0x7ff5f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:32.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.634+0000 7ff601ffb640 1 -- 172.21.15.12:0/2855791810 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4017070 con 0x7ff6041029f0 2024-04-22T13:23:32.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.634+0000 7ff601ffb640 1 -- 172.21.15.12:0/2855791810 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5f4009c30 con 0x7ff6041029f0 2024-04-22T13:23:32.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.634+0000 7ff60abdb640 1 -- 172.21.15.12:0/2855791810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6041a00c0 con 0x7ff6041029f0 2024-04-22T13:23:32.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.634+0000 7ff601ffb640 1 -- 172.21.15.12:0/2855791810 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f400f620 con 0x7ff6041029f0 2024-04-22T13:23:32.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.634+0000 7ff60abdb640 1 -- 172.21.15.12:0/2855791810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6041a0660 con 0x7ff6041029f0 2024-04-22T13:23:32.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.635+0000 7ff60abdb640 1 -- 172.21.15.12:0/2855791810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6040fded0 con 0x7ff6041029f0 2024-04-22T13:23:32.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.636+0000 7ff601ffb640 1 -- 172.21.15.12:0/2855791810 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff5f4010460 con 0x7ff6041029f0 2024-04-22T13:23:32.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.637+0000 7ff601ffb640 1 --2- 172.21.15.12:0/2855791810 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ff5ec0a3cc0 0x7ff5ec0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:32.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.637+0000 7ff603fff640 1 --2- 172.21.15.12:0/2855791810 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ff5ec0a3cc0 0x7ff5ec0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:32.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.637+0000 7ff601ffb640 1 -- 172.21.15.12:0/2855791810 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7ff5f40d01c0 con 0x7ff6041029f0 2024-04-22T13:23:32.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.638+0000 7ff603fff640 1 --2- 172.21.15.12:0/2855791810 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ff5ec0a3cc0 0x7ff5ec0a6180 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff5f802c840 tx=0x7ff5f8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:32.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.642+0000 7ff601ffb640 1 -- 172.21.15.12:0/2855791810 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff5f409ca70 con 0x7ff6041029f0 2024-04-22T13:23:32.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:32 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2531868478' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:23:32.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:32 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2187005344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:23:32.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:32 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3041653584' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:23:32.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.869+0000 7ff60abdb640 1 -- 172.21.15.12:0/2855791810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff60402d5c0 con 0x7ff6041029f0 2024-04-22T13:23:32.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.870+0000 7ff601ffb640 1 -- 172.21.15.12:0/2855791810 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11942 (secure 0 0 0) 0x7ff5f4009de0 con 0x7ff6041029f0 2024-04-22T13:23:32.866 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:23:32.866 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:22:12.398056+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24448,24509,24443,24484,24407]},"id":1}]} 2024-04-22T13:23:32.866 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 27 2024-04-22T13:23:32.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.872+0000 7ff60abdb640 1 -- 172.21.15.12:0/2855791810 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ff5ec0a3cc0 msgr2=0x7ff5ec0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:32.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.872+0000 7ff60abdb640 1 --2- 172.21.15.12:0/2855791810 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ff5ec0a3cc0 0x7ff5ec0a6180 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff5f802c840 tx=0x7ff5f8000bc0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.872+0000 7ff60abdb640 1 -- 172.21.15.12:0/2855791810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6041029f0 msgr2=0x7ff604198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:32.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.872+0000 7ff60abdb640 1 --2- 172.21.15.12:0/2855791810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6041029f0 0x7ff604198200 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400d950 tx=0x7ff5f400de10 comp rx=0 tx=0).stop 2024-04-22T13:23:32.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.872+0000 7ff60abdb640 1 -- 172.21.15.12:0/2855791810 shutdown_connections 2024-04-22T13:23:32.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.872+0000 7ff60abdb640 1 --2- 172.21.15.12:0/2855791810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff604107c50 0x7ff60419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.872+0000 7ff60abdb640 1 --2- 172.21.15.12:0/2855791810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff604105320 0x7ff604198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.872+0000 7ff60abdb640 1 --2- 172.21.15.12:0/2855791810 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ff5ec0a3cc0 0x7ff5ec0a6180 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.872+0000 7ff60abdb640 1 --2- 172.21.15.12:0/2855791810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff6041029f0 0x7ff604198200 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.872+0000 7ff60abdb640 1 -- 172.21.15.12:0/2855791810 >> 172.21.15.12:0/2855791810 conn(0x7ff6040fc3e0 msgr2=0x7ff604103330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:32.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.872+0000 7ff60abdb640 1 -- 172.21.15.12:0/2855791810 shutdown_connections 2024-04-22T13:23:32.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.872+0000 7ff60abdb640 1 -- 172.21.15.12:0/2855791810 wait complete. 2024-04-22T13:23:32.879 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-22T13:22:04.602553+0000', 'modified': '2024-04-22T13:22:12.398056+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24509, 'mds_1': 24484, 'mds_2': 24448, 'mds_3': 24443, 'mds_4': 24407}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14631': {'gid': 14631, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.12:6839/4083188176', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6838', 'nonce': 4083188176}, {'type': 'v1', 'addr': '172.21.15.12:6839', 'nonce': 4083188176}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14643': {'gid': 14643, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.12:6841/763294965', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6840', 'nonce': 763294965}, {'type': 'v1', 'addr': '172.21.15.12:6841', 'nonce': 763294965}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24406': {'gid': 24406, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6833/4235830781', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 4235830781}, {'type': 'v1', 'addr': '172.21.15.134:6833', 'nonce': 4235830781}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'k', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.111:6833/1902687060', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6832', 'nonce': 1902687060}, {'type': 'v1', 'addr': '172.21.15.111:6833', 'nonce': 1902687060}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'e', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.111:6835/2665640170', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6834', 'nonce': 2665640170}, {'type': 'v1', 'addr': '172.21.15.111:6835', 'nonce': 2665640170}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'f', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.134:6835/3927126579', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 3927126579}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 3927126579}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6837/3631577545', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6836', 'nonce': 3631577545}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 3631577545}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.111:6837/4218407470', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6836', 'nonce': 4218407470}, {'type': 'v1', 'addr': '172.21.15.111:6837', 'nonce': 4218407470}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.134:6839/1488513960', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 1488513960}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 1488513960}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24509': {'gid': 24509, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.111:6839/2126656770', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6838', 'nonce': 2126656770}, {'type': 'v1', 'addr': '172.21.15.111:6839', 'nonce': 2126656770}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24509, 'qdb_cluster': [24448, 24509, 24443, 24484, 24407]} 2024-04-22T13:23:32.880 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-22T13:23:32.880 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-22T13:23:32.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:32 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2531868478' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:23:32.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:32 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2187005344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:23:32.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:32 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/3041653584' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:23:32.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:32 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2531868478' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:23:32.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:32 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2187005344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:23:32.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:32 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/3041653584' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:23:32.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.986+0000 7f6ffa65c640 1 -- 172.21.15.12:0/3596384880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ff4101fa0 msgr2=0x7f6ff41023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:32.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.986+0000 7f6ffa65c640 1 --2- 172.21.15.12:0/3596384880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ff4101fa0 0x7f6ff41023c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009900 tx=0x7f6fe402bc30 comp rx=0 tx=0).stop 2024-04-22T13:23:32.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.986+0000 7f6ffa65c640 1 -- 172.21.15.12:0/3596384880 shutdown_connections 2024-04-22T13:23:32.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.986+0000 7f6ffa65c640 1 --2- 172.21.15.12:0/3596384880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ff4103b50 0x7f6ff410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.986+0000 7f6ffa65c640 1 --2- 172.21.15.12:0/3596384880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ff4103190 0x7f6ff4103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.986+0000 7f6ffa65c640 1 --2- 172.21.15.12:0/3596384880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ff4101fa0 0x7f6ff41023c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.986+0000 7f6ffa65c640 1 -- 172.21.15.12:0/3596384880 >> 172.21.15.12:0/3596384880 conn(0x7f6ff40fd530 msgr2=0x7f6ff40ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:32.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.986+0000 7f6ffa65c640 1 -- 172.21.15.12:0/3596384880 shutdown_connections 2024-04-22T13:23:32.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.986+0000 7f6ffa65c640 1 -- 172.21.15.12:0/3596384880 wait complete. 2024-04-22T13:23:32.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.987+0000 7f6ffa65c640 1 Processor -- start 2024-04-22T13:23:32.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.987+0000 7f6ffa65c640 1 -- start start 2024-04-22T13:23:32.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.987+0000 7f6ffa65c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ff4101fa0 0x7f6ff41981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:32.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.987+0000 7f6ffa65c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ff4103190 0x7f6ff41986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:32.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.987+0000 7f6ffa65c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ff4103b50 0x7f6ff419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:32.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.987+0000 7f6ffa65c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff410dae0 con 0x7f6ff4101fa0 2024-04-22T13:23:32.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.987+0000 7f6ffa65c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff410d960 con 0x7f6ff4103b50 2024-04-22T13:23:32.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.987+0000 7f6ffa65c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff410dc60 con 0x7f6ff4103190 2024-04-22T13:23:32.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.987+0000 7f6ff3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ff4101fa0 0x7f6ff41981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:32.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.987+0000 7f6ff3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ff4101fa0 0x7f6ff41981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34808/0 (socket says 172.21.15.12:34808) 2024-04-22T13:23:32.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.987+0000 7f6ff3fff640 1 -- 172.21.15.12:0/4128498756 learned_addr learned my addr 172.21.15.12:0/4128498756 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:32.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.987+0000 7f6ff3fff640 1 -- 172.21.15.12:0/4128498756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ff4103190 msgr2=0x7f6ff41986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:32.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.987+0000 7f6ff3fff640 1 --2- 172.21.15.12:0/4128498756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ff4103190 0x7f6ff41986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.987+0000 7f6ff3fff640 1 -- 172.21.15.12:0/4128498756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ff4103b50 msgr2=0x7f6ff419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:32.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.987+0000 7f6ff3fff640 1 --2- 172.21.15.12:0/4128498756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ff4103b50 0x7f6ff419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.987+0000 7f6ff3fff640 1 -- 172.21.15.12:0/4128498756 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff419fdc0 con 0x7f6ff4101fa0 2024-04-22T13:23:32.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.988+0000 7f6ff3fff640 1 --2- 172.21.15.12:0/4128498756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ff4101fa0 0x7f6ff41981a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4000c00 tx=0x7f6fe4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:32.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.988+0000 7f6ff17fa640 1 -- 172.21.15.12:0/4128498756 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4043020 con 0x7f6ff4101fa0 2024-04-22T13:23:32.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.988+0000 7f6ff17fa640 1 -- 172.21.15.12:0/4128498756 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fe4004440 con 0x7f6ff4101fa0 2024-04-22T13:23:32.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.988+0000 7f6ffa65c640 1 -- 172.21.15.12:0/4128498756 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ff41a0050 con 0x7f6ff4101fa0 2024-04-22T13:23:32.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.988+0000 7f6ff17fa640 1 -- 172.21.15.12:0/4128498756 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4003c90 con 0x7f6ff4101fa0 2024-04-22T13:23:32.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.988+0000 7f6ffa65c640 1 -- 172.21.15.12:0/4128498756 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ff41a05a0 con 0x7f6ff4101fa0 2024-04-22T13:23:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.991+0000 7f6ff17fa640 1 -- 172.21.15.12:0/4128498756 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6fe4046020 con 0x7f6ff4101fa0 2024-04-22T13:23:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.991+0000 7f6ffd252740 1 Processor -- start 2024-04-22T13:23:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.991+0000 7f6ffd252740 1 -- start start 2024-04-22T13:23:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.991+0000 7f6ffd252740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a7ad1e40 0x5632a7ad2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.991+0000 7f6ffd252740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5632a7ad2830 0x5632a7ad2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.991+0000 7f6ffd252740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a7ad32d0 0x5632a7ad9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.991+0000 7f6ffd252740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5632a7a36b10 con 0x5632a7ad1e40 2024-04-22T13:23:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.991+0000 7f6ffd252740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5632a79d4240 con 0x5632a7ad2830 2024-04-22T13:23:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.991+0000 7f6ffd252740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5632a7a32ad0 con 0x5632a7ad32d0 2024-04-22T13:23:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.991+0000 7f6ff8bf9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a7ad1e40 0x5632a7ad2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.991+0000 7f6ff8bf9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a7ad1e40 0x5632a7ad2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34824/0 (socket says 172.21.15.12:34824) 2024-04-22T13:23:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.991+0000 7f6ff8bf9640 1 -- 172.21.15.12:0/1803455911 learned_addr learned my addr 172.21.15.12:0/1803455911 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.991+0000 7f6ff37fe640 1 --2- 172.21.15.12:0/1803455911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a7ad32d0 0x5632a7ad9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.991+0000 7f6ff3fff640 1 --2- 172.21.15.12:0/1803455911 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5632a7ad2830 0x5632a7ad2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.991+0000 7f6ff17fa640 1 --2- 172.21.15.12:0/4128498756 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f6fdc0a3c40 0x7f6fdc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.991+0000 7f6ff8bf9640 1 -- 172.21.15.12:0/1803455911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a7ad32d0 msgr2=0x5632a7ad9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.991+0000 7f6ff8bf9640 1 --2- 172.21.15.12:0/1803455911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a7ad32d0 0x5632a7ad9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.991+0000 7f6ff8bf9640 1 -- 172.21.15.12:0/1803455911 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5632a7ad2830 msgr2=0x5632a7ad2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.991+0000 7f6ff8bf9640 1 --2- 172.21.15.12:0/1803455911 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5632a7ad2830 0x5632a7ad2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.991+0000 7f6ff8bf9640 1 -- 172.21.15.12:0/1803455911 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632a77492d0 con 0x5632a7ad1e40 2024-04-22T13:23:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.991+0000 7f6ff37fe640 1 --2- 172.21.15.12:0/1803455911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a7ad32d0 0x5632a7ad9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:23:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.992+0000 7f6ff8bf9640 1 --2- 172.21.15.12:0/4128498756 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f6fdc0a3c40 0x7f6fdc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.992+0000 7f6ff17fa640 1 -- 172.21.15.12:0/4128498756 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f6fe40f2770 con 0x7f6ff4101fa0 2024-04-22T13:23:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.992+0000 7f6ff8bf9640 1 --2- 172.21.15.12:0/1803455911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a7ad1e40 0x5632a7ad2260 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6fec009fd0 tx=0x7f6fec00e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=674bce72a68ff6cc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.992+0000 7f6fda7fc640 1 -- 172.21.15.12:0/1803455911 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec012e20 con 0x5632a7ad1e40 2024-04-22T13:23:32.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.992+0000 7f6fda7fc640 1 -- 172.21.15.12:0/1803455911 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fec01bd90 con 0x5632a7ad1e40 2024-04-22T13:23:32.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.992+0000 7f6fda7fc640 1 -- 172.21.15.12:0/1803455911 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec01acb0 con 0x5632a7ad1e40 2024-04-22T13:23:32.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.992+0000 7f6ff8bf9640 1 --2- 172.21.15.12:0/4128498756 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f6fdc0a3c40 0x7f6fdc0a6100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6fec01f000 tx=0x7f6fec020000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:32.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.992+0000 7f6ffd252740 1 -- 172.21.15.12:0/1803455911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a7ad1e40 msgr2=0x5632a7ad2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:32.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.992+0000 7f6ffd252740 1 --2- 172.21.15.12:0/1803455911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a7ad1e40 0x5632a7ad2260 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6fec009fd0 tx=0x7f6fec00e830 comp rx=0 tx=0).stop 2024-04-22T13:23:32.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.993+0000 7f6ffd252740 1 -- 172.21.15.12:0/1803455911 shutdown_connections 2024-04-22T13:23:32.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.993+0000 7f6ffd252740 1 --2- 172.21.15.12:0/1803455911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a7ad32d0 0x5632a7ad9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.993+0000 7f6ffd252740 1 --2- 172.21.15.12:0/1803455911 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5632a7ad2830 0x5632a7ad2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.993+0000 7f6ffd252740 1 --2- 172.21.15.12:0/1803455911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a7ad1e40 0x5632a7ad2260 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.993+0000 7f6ffd252740 1 -- 172.21.15.12:0/1803455911 >> 172.21.15.12:0/1803455911 conn(0x5632a7acdf40 msgr2=0x5632a7acc490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:32.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.993+0000 7f6ffd252740 1 -- 172.21.15.12:0/1803455911 shutdown_connections 2024-04-22T13:23:32.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.993+0000 7f6ffd252740 1 -- 172.21.15.12:0/1803455911 wait complete. 2024-04-22T13:23:32.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.993+0000 7f6ffd252740 1 -- start start 2024-04-22T13:23:32.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.993+0000 7f6ffd252740 1 Processor -- start 2024-04-22T13:23:32.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.993+0000 7f6ffd252740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a7ad1e40 0x5632a7adbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:32.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.993+0000 7f6ffd252740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a7ad2830 0x5632a7adc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:32.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.993+0000 7f6ffd252740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5632a7ad32d0 0x5632a7ae3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:32.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.993+0000 7f6ffd252740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5632a7ada4b0 con 0x5632a7ad2830 2024-04-22T13:23:32.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.993+0000 7f6ffd252740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5632a79bf190 con 0x5632a7ad32d0 2024-04-22T13:23:32.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.993+0000 7f6ffd252740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5632a7ada630 con 0x5632a7ad1e40 2024-04-22T13:23:32.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.993+0000 7f6ffd252740 10 client.0 _ll_register_callbacks cb 0x5632a79ba8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:23:32.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.993+0000 7f6ff37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a7ad2830 0x5632a7adc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:32.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.994+0000 7f6ff37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a7ad2830 0x5632a7adc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34826/0 (socket says 172.21.15.12:34826) 2024-04-22T13:23:32.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.994+0000 7f6ff37fe640 1 -- 172.21.15.12:0/2543654718 learned_addr learned my addr 172.21.15.12:0/2543654718 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:32.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.994+0000 7f6ff8bf9640 1 --2- 172.21.15.12:0/2543654718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5632a7ad32d0 0x5632a7ae3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:32.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.994+0000 7f6ff3fff640 1 --2- 172.21.15.12:0/2543654718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a7ad1e40 0x5632a7adbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:32.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.994+0000 7f6ff37fe640 1 -- 172.21.15.12:0/2543654718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a7ad1e40 msgr2=0x5632a7adbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:32.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.994+0000 7f6ff37fe640 1 --2- 172.21.15.12:0/2543654718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a7ad1e40 0x5632a7adbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.994+0000 7f6ff37fe640 1 -- 172.21.15.12:0/2543654718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5632a7ad32d0 msgr2=0x5632a7ae3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:32.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.994+0000 7f6ff37fe640 1 --2- 172.21.15.12:0/2543654718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5632a7ad32d0 0x5632a7ae3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.994+0000 7f6ff37fe640 1 -- 172.21.15.12:0/2543654718 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632a78136b0 con 0x5632a7ad2830 2024-04-22T13:23:32.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.994+0000 7f6ff3fff640 1 --2- 172.21.15.12:0/2543654718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a7ad1e40 0x5632a7adbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:23:32.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.994+0000 7f6ff8bf9640 1 --2- 172.21.15.12:0/2543654718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5632a7ad32d0 0x5632a7ae3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:23:32.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.994+0000 7f6ff37fe640 1 --2- 172.21.15.12:0/2543654718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a7ad2830 0x5632a7adc420 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6fe800e670 tx=0x7f6fe800eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c001ddcc769010 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:32.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.994+0000 7f6fd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:23:32.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.994+0000 7f6fd37fe640 1 -- 172.21.15.12:0/2543654718 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe800aa30 con 0x5632a7ad2830 2024-04-22T13:23:32.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.994+0000 7f6fd37fe640 1 -- 172.21.15.12:0/2543654718 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f6fe800abd0 con 0x5632a7ad2830 2024-04-22T13:23:32.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.994+0000 7f6ffd252740 1 -- 172.21.15.12:0/2543654718 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fec01bd90 con 0x5632a7ad2830 2024-04-22T13:23:32.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.994+0000 7f6fd37fe640 1 -- 172.21.15.12:0/2543654718 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6fe8021070 con 0x5632a7ad2830 2024-04-22T13:23:32.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.994+0000 7f6ffd252740 10 client.15057 fetch_fsmap learned FSMap version 27 2024-04-22T13:23:32.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.994+0000 7f6ffd252740 1 -- 172.21.15.12:0/2543654718 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5632a7ae68a0 con 0x5632a7ad2830 2024-04-22T13:23:32.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.994+0000 7f6fd37fe640 1 -- 172.21.15.12:0/2543654718 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fec01bd90 con 0x5632a7ad2830 2024-04-22T13:23:32.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.995+0000 7f6fd37fe640 1 -- 172.21.15.12:0/2543654718 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6fe8019030 con 0x5632a7ad2830 2024-04-22T13:23:32.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.995+0000 7f6ffd252740 10 client.15057 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:23:32.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.995+0000 7f6ffd252740 10 client.15057 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-22T13:23:32.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.995+0000 7f6ffd252740 20 client.15057 populate_metadata read hostname 'smithi012' 2024-04-22T13:23:32.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.995+0000 7f6ffd252740 1 --2- 172.21.15.12:0/2543654718 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5632a7ae8fa0 0x5632a7aea410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:32.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.995+0000 7f6ffd252740 4 client.15057 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:23:32.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.995+0000 7f6ffd252740 1 -- 172.21.15.12:0/2543654718 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632a7a44530 con 0x5632a7ae8fa0 2024-04-22T13:23:32.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.995+0000 7f6ff3fff640 1 --2- 172.21.15.12:0/2543654718 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5632a7ae8fa0 0x5632a7aea410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:32.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.995+0000 7f6fd37fe640 10 client.15057 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:23:32.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.995+0000 7f6ff3fff640 1 --2- 172.21.15.12:0/2543654718 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5632a7ae8fa0 0x5632a7aea410 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6a7ae763abf082ba server_cookie=86190b6394138a5b in_seq=0 out_seq=0 2024-04-22T13:23:32.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.996+0000 7f6fd37fe640 1 -- 172.21.15.12:0/2543654718 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5632a7a44530 con 0x5632a7ae8fa0 2024-04-22T13:23:32.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.996+0000 7f6fd37fe640 10 client.15057 handle_command_reply: tid=0 2024-04-22T13:23:32.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.996+0000 7f6ffd252740 1 client.15057 shutdown 2024-04-22T13:23:32.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.996+0000 7f6ffd252740 1 -- 172.21.15.12:0/2543654718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a7ad2830 msgr2=0x5632a7adc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:32.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.996+0000 7f6ffd252740 1 --2- 172.21.15.12:0/2543654718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a7ad2830 0x5632a7adc420 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6fe800e670 tx=0x7f6fe800eb30 comp rx=0 tx=0).stop 2024-04-22T13:23:32.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.996+0000 7f6ffd252740 1 -- 172.21.15.12:0/2543654718 shutdown_connections 2024-04-22T13:23:32.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.996+0000 7f6ffd252740 1 --2- 172.21.15.12:0/2543654718 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5632a7ae8fa0 0x5632a7aea410 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.996+0000 7f6ffd252740 1 --2- 172.21.15.12:0/2543654718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5632a7ad32d0 0x5632a7ae3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.996+0000 7f6ffd252740 1 --2- 172.21.15.12:0/2543654718 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a7ad2830 0x5632a7adc420 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.996+0000 7f6ffd252740 1 --2- 172.21.15.12:0/2543654718 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a7ad1e40 0x5632a7adbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:32.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.996+0000 7f6ffd252740 1 -- 172.21.15.12:0/2543654718 >> 172.21.15.12:0/2543654718 conn(0x5632a7acdf40 msgr2=0x5632a7acba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:32.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.996+0000 7f6fd37fe640 0 client.15057 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:23:32.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.997+0000 7f6ffd252740 1 -- 172.21.15.12:0/2543654718 shutdown_connections 2024-04-22T13:23:32.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.997+0000 7f6ffd252740 1 -- 172.21.15.12:0/2543654718 wait complete. 2024-04-22T13:23:32.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:32.997+0000 7f6ffd252740 20 client.15057 trim_cache size 0 max 16384 2024-04-22T13:23:33.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.015+0000 7f6ffd252740 1 Processor -- start 2024-04-22T13:23:33.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.015+0000 7f6ffd252740 1 -- start start 2024-04-22T13:23:33.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.015+0000 7f6ffd252740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5632a7ad1e40 0x5632a7acc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.015+0000 7f6ffd252740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a7ad2830 0x5632a7ae3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.015+0000 7f6ffd252740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a7ad32d0 0x5632a7acc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.015+0000 7f6ffd252740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5632a7ae8750 con 0x5632a7ad2830 2024-04-22T13:23:33.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.015+0000 7f6ffd252740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5632a7ae8e20 con 0x5632a7ad1e40 2024-04-22T13:23:33.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.015+0000 7f6ffd252740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5632a7ae94f0 con 0x5632a7ad32d0 2024-04-22T13:23:33.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.015+0000 7f6ff37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a7ad2830 0x5632a7ae3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.015+0000 7f6ff37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a7ad2830 0x5632a7ae3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34838/0 (socket says 172.21.15.12:34838) 2024-04-22T13:23:33.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.015+0000 7f6ff37fe640 1 -- 172.21.15.12:0/887397264 learned_addr learned my addr 172.21.15.12:0/887397264 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:33.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.015+0000 7f6ff37fe640 1 -- 172.21.15.12:0/887397264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a7ad32d0 msgr2=0x5632a7acc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.015+0000 7f6ff37fe640 1 --2- 172.21.15.12:0/887397264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a7ad32d0 0x5632a7acc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.015+0000 7f6ff37fe640 1 -- 172.21.15.12:0/887397264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5632a7ad1e40 msgr2=0x5632a7acc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.015+0000 7f6ff37fe640 1 --2- 172.21.15.12:0/887397264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5632a7ad1e40 0x5632a7acc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.015+0000 7f6ff37fe640 1 -- 172.21.15.12:0/887397264 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632a7acd830 con 0x5632a7ad2830 2024-04-22T13:23:33.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.016+0000 7f6ff37fe640 1 --2- 172.21.15.12:0/887397264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a7ad2830 0x5632a7ae3420 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8001eb0 tx=0x7f6fe8005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafc70b69aa8b4d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:33.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.016+0000 7f6fd9ffb640 1 -- 172.21.15.12:0/887397264 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe802c070 con 0x5632a7ad2830 2024-04-22T13:23:33.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.016+0000 7f6fd9ffb640 1 -- 172.21.15.12:0/887397264 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fe801a970 con 0x5632a7ad2830 2024-04-22T13:23:33.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.016+0000 7f6fd9ffb640 1 -- 172.21.15.12:0/887397264 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe8010690 con 0x5632a7ad2830 2024-04-22T13:23:33.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.016+0000 7f6ffd252740 1 -- 172.21.15.12:0/887397264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a7ad2830 msgr2=0x5632a7ae3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.016+0000 7f6ffd252740 1 --2- 172.21.15.12:0/887397264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a7ad2830 0x5632a7ae3420 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8001eb0 tx=0x7f6fe8005b10 comp rx=0 tx=0).stop 2024-04-22T13:23:33.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.016+0000 7f6ffd252740 1 -- 172.21.15.12:0/887397264 shutdown_connections 2024-04-22T13:23:33.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.016+0000 7f6ffd252740 1 --2- 172.21.15.12:0/887397264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a7ad32d0 0x5632a7acc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.016+0000 7f6ffd252740 1 --2- 172.21.15.12:0/887397264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a7ad2830 0x5632a7ae3420 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.016+0000 7f6ffd252740 1 --2- 172.21.15.12:0/887397264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5632a7ad1e40 0x5632a7acc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.016+0000 7f6ffd252740 1 -- 172.21.15.12:0/887397264 >> 172.21.15.12:0/887397264 conn(0x5632a7acdf40 msgr2=0x5632a7ad6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:33.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.016+0000 7f6ffd252740 1 -- 172.21.15.12:0/887397264 shutdown_connections 2024-04-22T13:23:33.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.016+0000 7f6ffd252740 1 -- 172.21.15.12:0/887397264 wait complete. 2024-04-22T13:23:33.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.017+0000 7f6ffd252740 1 -- start start 2024-04-22T13:23:33.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.017+0000 7f6ffd252740 1 Processor -- start 2024-04-22T13:23:33.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.017+0000 7f6ffd252740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5632a7ad1e40 0x5632a7adf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.017+0000 7f6ffd252740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a7ad2830 0x5632a7ae18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.017+0000 7f6ffd252740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a7ad32d0 0x5632a7ae1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.017+0000 7f6ffd252740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5632a7aea290 con 0x5632a7ad2830 2024-04-22T13:23:33.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.017+0000 7f6ffd252740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5632a7ae9bc0 con 0x5632a7ad1e40 2024-04-22T13:23:33.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.017+0000 7f6ffd252740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5632a7aea960 con 0x5632a7ad32d0 2024-04-22T13:23:33.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.017+0000 7f6ffd252740 10 client.0 _ll_register_callbacks cb 0x7f6fe800c330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:23:33.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.017+0000 7f6ff37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a7ad2830 0x5632a7ae18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.017+0000 7f6ff37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a7ad2830 0x5632a7ae18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34854/0 (socket says 172.21.15.12:34854) 2024-04-22T13:23:33.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.017+0000 7f6ff37fe640 1 -- 172.21.15.12:0/2294586966 learned_addr learned my addr 172.21.15.12:0/2294586966 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:33.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.017+0000 7f6ff3fff640 1 --2- 172.21.15.12:0/2294586966 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5632a7ad1e40 0x5632a7adf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.017+0000 7f6ff8bf9640 1 --2- 172.21.15.12:0/2294586966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a7ad32d0 0x5632a7ae1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.017+0000 7f6ff37fe640 1 -- 172.21.15.12:0/2294586966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a7ad32d0 msgr2=0x5632a7ae1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.017+0000 7f6ff37fe640 1 --2- 172.21.15.12:0/2294586966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a7ad32d0 0x5632a7ae1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.017+0000 7f6ff37fe640 1 -- 172.21.15.12:0/2294586966 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5632a7ad1e40 msgr2=0x5632a7adf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.017+0000 7f6ff37fe640 1 --2- 172.21.15.12:0/2294586966 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5632a7ad1e40 0x5632a7adf360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.018+0000 7f6ff37fe640 1 -- 172.21.15.12:0/2294586966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632a7ae24e0 con 0x5632a7ad2830 2024-04-22T13:23:33.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.018+0000 7f6ff8bf9640 1 --2- 172.21.15.12:0/2294586966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a7ad32d0 0x5632a7ae1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:23:33.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.018+0000 7f6ff37fe640 1 --2- 172.21.15.12:0/2294586966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a7ad2830 0x5632a7ae18b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6fe801a6f0 tx=0x7f6fe801add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d064371d85396d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:33.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.018+0000 7f6fd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:23:33.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.018+0000 7f6fd37fe640 1 -- 172.21.15.12:0/2294586966 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe802c040 con 0x5632a7ad2830 2024-04-22T13:23:33.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.018+0000 7f6fd37fe640 1 -- 172.21.15.12:0/2294586966 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f6fe8004220 con 0x5632a7ad2830 2024-04-22T13:23:33.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.018+0000 7f6ffd252740 1 -- 172.21.15.12:0/2294586966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fe801a970 con 0x5632a7ad2830 2024-04-22T13:23:33.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.018+0000 7f6fd37fe640 1 -- 172.21.15.12:0/2294586966 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6fe8004460 con 0x5632a7ad2830 2024-04-22T13:23:33.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.018+0000 7f6ffd252740 10 client.15063 fetch_fsmap learned FSMap version 27 2024-04-22T13:23:33.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.018+0000 7f6ffd252740 1 -- 172.21.15.12:0/2294586966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5632a7b4ef50 con 0x5632a7ad2830 2024-04-22T13:23:33.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.018+0000 7f6fd37fe640 1 -- 172.21.15.12:0/2294586966 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fe801a970 con 0x5632a7ad2830 2024-04-22T13:23:33.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.018+0000 7f6fd37fe640 1 -- 172.21.15.12:0/2294586966 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6fe8019040 con 0x5632a7ad2830 2024-04-22T13:23:33.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.018+0000 7f6ffd252740 10 client.15063 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:23:33.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.018+0000 7f6ffd252740 10 client.15063 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-22T13:23:33.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.018+0000 7f6ffd252740 20 client.15063 populate_metadata read hostname 'smithi012' 2024-04-22T13:23:33.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.018+0000 7f6ffd252740 1 --2- 172.21.15.12:0/2294586966 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5632a7b53560 0x5632a7b52270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.018+0000 7f6ffd252740 4 client.15063 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T13:23:33.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.018+0000 7f6ffd252740 1 -- 172.21.15.12:0/2294586966 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "status"}) -- 0x5632a7a3e3a0 con 0x5632a7b53560 2024-04-22T13:23:33.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.019+0000 7f6ff3fff640 1 --2- 172.21.15.12:0/2294586966 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5632a7b53560 0x5632a7b52270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.019+0000 7f6ff3fff640 1 --2- 172.21.15.12:0/2294586966 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5632a7b53560 0x5632a7b52270 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8914126a653c2a70 server_cookie=5206cf3131ff51e6 in_seq=0 out_seq=0 2024-04-22T13:23:33.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.019+0000 7f6fd37fe640 10 client.15063 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:23:33.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.019+0000 7f6fd37fe640 1 -- 172.21.15.12:0/2294586966 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5632a7a3e3a0 con 0x5632a7b53560 2024-04-22T13:23:33.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.019+0000 7f6fd37fe640 10 client.15063 handle_command_reply: tid=0 2024-04-22T13:23:33.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.019+0000 7f6ffd252740 1 client.15063 shutdown 2024-04-22T13:23:33.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.020+0000 7f6ffd252740 1 -- 172.21.15.12:0/2294586966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a7ad2830 msgr2=0x5632a7ae18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.020+0000 7f6ffd252740 1 --2- 172.21.15.12:0/2294586966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a7ad2830 0x5632a7ae18b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6fe801a6f0 tx=0x7f6fe801add0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.020+0000 7f6ffd252740 1 -- 172.21.15.12:0/2294586966 shutdown_connections 2024-04-22T13:23:33.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.020+0000 7f6ffd252740 1 --2- 172.21.15.12:0/2294586966 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5632a7b53560 0x5632a7b52270 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.020+0000 7f6ffd252740 1 --2- 172.21.15.12:0/2294586966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632a7ad32d0 0x5632a7ae1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.020+0000 7f6ffd252740 1 --2- 172.21.15.12:0/2294586966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632a7ad2830 0x5632a7ae18b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.020+0000 7f6fd37fe640 0 client.15063 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:23:33.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.020+0000 7f6ffd252740 1 --2- 172.21.15.12:0/2294586966 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5632a7ad1e40 0x5632a7adf360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.020+0000 7f6ffd252740 1 -- 172.21.15.12:0/2294586966 >> 172.21.15.12:0/2294586966 conn(0x5632a7acdf40 msgr2=0x5632a7ad0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:33.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.020+0000 7f6ffd252740 1 -- 172.21.15.12:0/2294586966 shutdown_connections 2024-04-22T13:23:33.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.020+0000 7f6ffd252740 1 -- 172.21.15.12:0/2294586966 wait complete. 2024-04-22T13:23:33.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.020+0000 7f6ffd252740 20 client.15063 trim_cache size 0 max 16384 2024-04-22T13:23:33.023 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:23:33.023 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "0aede5ea-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:23:33.023 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 4, 2024-04-22T13:23:33.023 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24407, 2024-04-22T13:23:33.023 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-04-22T13:23:33.023 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-04-22T13:23:33.023 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-04-22T13:23:33.023 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 81.627432744999993, 2024-04-22T13:23:33.023 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 27, 2024-04-22T13:23:33.024 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 89, 2024-04-22T13:23:33.024 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 87, 2024-04-22T13:23:33.024 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 120.967062398 2024-04-22T13:23:33.024 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:23:33.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.020+0000 7f6ffa65c640 1 -- 172.21.15.12:0/4128498756 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f6fdc0a3c40 msgr2=0x7f6fdc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.020+0000 7f6ffa65c640 1 --2- 172.21.15.12:0/4128498756 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f6fdc0a3c40 0x7f6fdc0a6100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6fec01f000 tx=0x7f6fec020000 comp rx=0 tx=0).stop 2024-04-22T13:23:33.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.020+0000 7f6ffa65c640 1 -- 172.21.15.12:0/4128498756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ff4101fa0 msgr2=0x7f6ff41981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.020+0000 7f6ffa65c640 1 --2- 172.21.15.12:0/4128498756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ff4101fa0 0x7f6ff41981a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4000c00 tx=0x7f6fe4004040 comp rx=0 tx=0).stop 2024-04-22T13:23:33.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.020+0000 7f6ffa65c640 1 -- 172.21.15.12:0/4128498756 shutdown_connections 2024-04-22T13:23:33.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.020+0000 7f6ffa65c640 1 --2- 172.21.15.12:0/4128498756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ff4103b50 0x7f6ff419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.020+0000 7f6ffa65c640 1 --2- 172.21.15.12:0/4128498756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ff4103190 0x7f6ff41986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.020+0000 7f6ffa65c640 1 --2- 172.21.15.12:0/4128498756 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f6fdc0a3c40 0x7f6fdc0a6100 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.020+0000 7f6ffa65c640 1 --2- 172.21.15.12:0/4128498756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ff4101fa0 0x7f6ff41981a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.020+0000 7f6ffa65c640 1 -- 172.21.15.12:0/4128498756 >> 172.21.15.12:0/4128498756 conn(0x7f6ff40fd530 msgr2=0x7f6ff4106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:33.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.020+0000 7f6ffa65c640 1 -- 172.21.15.12:0/4128498756 shutdown_connections 2024-04-22T13:23:33.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.020+0000 7f6ffa65c640 1 -- 172.21.15.12:0/4128498756 wait complete. 2024-04-22T13:23:33.027 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-22T13:23:33.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.133+0000 7fc2cc350640 1 -- 172.21.15.12:0/1445805841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2c4101fb0 msgr2=0x7fc2c41023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.133+0000 7fc2cc350640 1 --2- 172.21.15.12:0/1445805841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2c4101fb0 0x7fc2c41023d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009900 tx=0x7fc2c002bc30 comp rx=0 tx=0).stop 2024-04-22T13:23:33.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.133+0000 7fc2cc350640 1 -- 172.21.15.12:0/1445805841 shutdown_connections 2024-04-22T13:23:33.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.133+0000 7fc2cc350640 1 --2- 172.21.15.12:0/1445805841 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2c4103b60 0x7fc2c410a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.133+0000 7fc2cc350640 1 --2- 172.21.15.12:0/1445805841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2c41031a0 0x7fc2c4103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.133+0000 7fc2cc350640 1 --2- 172.21.15.12:0/1445805841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2c4101fb0 0x7fc2c41023d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.133+0000 7fc2cc350640 1 -- 172.21.15.12:0/1445805841 >> 172.21.15.12:0/1445805841 conn(0x7fc2c40fd520 msgr2=0x7fc2c40ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:33.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.133+0000 7fc2cc350640 1 -- 172.21.15.12:0/1445805841 shutdown_connections 2024-04-22T13:23:33.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.133+0000 7fc2cc350640 1 -- 172.21.15.12:0/1445805841 wait complete. 2024-04-22T13:23:33.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.134+0000 7fc2cc350640 1 Processor -- start 2024-04-22T13:23:33.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.134+0000 7fc2cc350640 1 -- start start 2024-04-22T13:23:33.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.134+0000 7fc2cc350640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2c4101fb0 0x7fc2c4198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.134+0000 7fc2cc350640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2c41031a0 0x7fc2c41987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.134+0000 7fc2cc350640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2c4103b60 0x7fc2c419f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.134+0000 7fc2cc350640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c410daf0 con 0x7fc2c4101fb0 2024-04-22T13:23:33.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.134+0000 7fc2cc350640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c410d970 con 0x7fc2c41031a0 2024-04-22T13:23:33.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.134+0000 7fc2cc350640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c410dc70 con 0x7fc2c4103b60 2024-04-22T13:23:33.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.134+0000 7fc2ca0ec640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2c4101fb0 0x7fc2c4198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.134+0000 7fc2ca0ec640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2c4101fb0 0x7fc2c4198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34872/0 (socket says 172.21.15.12:34872) 2024-04-22T13:23:33.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.134+0000 7fc2ca0ec640 1 -- 172.21.15.12:0/2038054000 learned_addr learned my addr 172.21.15.12:0/2038054000 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:33.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.134+0000 7fc2ca0ec640 1 -- 172.21.15.12:0/2038054000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2c4103b60 msgr2=0x7fc2c419f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.134+0000 7fc2ca0ec640 1 --2- 172.21.15.12:0/2038054000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2c4103b60 0x7fc2c419f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.134+0000 7fc2ca0ec640 1 -- 172.21.15.12:0/2038054000 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2c41031a0 msgr2=0x7fc2c41987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.134+0000 7fc2c98eb640 1 --2- 172.21.15.12:0/2038054000 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2c41031a0 0x7fc2c41987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.134+0000 7fc2ca0ec640 1 --2- 172.21.15.12:0/2038054000 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2c41031a0 0x7fc2c41987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.134+0000 7fc2ca0ec640 1 -- 172.21.15.12:0/2038054000 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c419fe30 con 0x7fc2c4101fb0 2024-04-22T13:23:33.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.134+0000 7fc2c98eb640 1 --2- 172.21.15.12:0/2038054000 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2c41031a0 0x7fc2c41987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:23:33.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.135+0000 7fc2ca0ec640 1 --2- 172.21.15.12:0/2038054000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2c4101fb0 0x7fc2c4198260 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0036040 tx=0x7fc2c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:33.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.135+0000 7fc2bb7fe640 1 -- 172.21.15.12:0/2038054000 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0043020 con 0x7fc2c4101fb0 2024-04-22T13:23:33.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.135+0000 7fc2bb7fe640 1 -- 172.21.15.12:0/2038054000 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c0004aa0 con 0x7fc2c4101fb0 2024-04-22T13:23:33.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.135+0000 7fc2cc350640 1 -- 172.21.15.12:0/2038054000 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2c41a00c0 con 0x7fc2c4101fb0 2024-04-22T13:23:33.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.135+0000 7fc2bb7fe640 1 -- 172.21.15.12:0/2038054000 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0003de0 con 0x7fc2c4101fb0 2024-04-22T13:23:33.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.135+0000 7fc2cc350640 1 -- 172.21.15.12:0/2038054000 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2c41a0660 con 0x7fc2c4101fb0 2024-04-22T13:23:33.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.137+0000 7fc2bb7fe640 1 -- 172.21.15.12:0/2038054000 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc2c0046020 con 0x7fc2c4101fb0 2024-04-22T13:23:33.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.137+0000 7fc2cf01c740 1 Processor -- start 2024-04-22T13:23:33.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.138+0000 7fc2cf01c740 1 -- start start 2024-04-22T13:23:33.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.138+0000 7fc2cf01c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625556a2e40 0x5625556a3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.138+0000 7fc2cf01c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625556a3830 0x5625556a3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.138+0000 7fc2cf01c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625556a42d0 0x5625556aaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.138+0000 7fc2cf01c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x562555607b10 con 0x5625556a42d0 2024-04-22T13:23:33.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.138+0000 7fc2cf01c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5625555a5240 con 0x5625556a3830 2024-04-22T13:23:33.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.138+0000 7fc2cf01c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562555603ad0 con 0x5625556a2e40 2024-04-22T13:23:33.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.138+0000 7fc2c98eb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625556a42d0 0x5625556aaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.138+0000 7fc2bb7fe640 1 --2- 172.21.15.12:0/2038054000 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc2b00a3cc0 0x7fc2b00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.138+0000 7fc2c98eb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625556a42d0 0x5625556aaeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34876/0 (socket says 172.21.15.12:34876) 2024-04-22T13:23:33.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.138+0000 7fc2c98eb640 1 -- 172.21.15.12:0/4132991929 learned_addr learned my addr 172.21.15.12:0/4132991929 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:33.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.138+0000 7fc2ca0ec640 1 --2- 172.21.15.12:0/4132991929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625556a3830 0x5625556a3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.138+0000 7fc2ca8ed640 1 --2- 172.21.15.12:0/4132991929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625556a2e40 0x5625556a3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.138+0000 7fc2ca8ed640 1 --2- 172.21.15.12:0/2038054000 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc2b00a3cc0 0x7fc2b00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.138+0000 7fc2bb7fe640 1 -- 172.21.15.12:0/2038054000 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fc2c00f2890 con 0x7fc2c4101fb0 2024-04-22T13:23:33.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.138+0000 7fc2c98eb640 1 -- 172.21.15.12:0/4132991929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625556a2e40 msgr2=0x5625556a3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.138+0000 7fc2c98eb640 1 --2- 172.21.15.12:0/4132991929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625556a2e40 0x5625556a3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.138+0000 7fc2c98eb640 1 -- 172.21.15.12:0/4132991929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625556a3830 msgr2=0x5625556a3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.138+0000 7fc2c98eb640 1 --2- 172.21.15.12:0/4132991929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625556a3830 0x5625556a3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.138+0000 7fc2c98eb640 1 -- 172.21.15.12:0/4132991929 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56255531a2d0 con 0x5625556a42d0 2024-04-22T13:23:33.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.139+0000 7fc2c98eb640 1 --2- 172.21.15.12:0/4132991929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625556a42d0 0x5625556aaeb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc2b400bb40 tx=0x7fc2b400d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2356a8a58f5a17e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:33.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.139+0000 7fc2ca8ed640 1 --2- 172.21.15.12:0/2038054000 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc2b00a3cc0 0x7fc2b00a6180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x56255569e200 tx=0x7fc2bc005e00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:33.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.139+0000 7fc2b8bfe640 1 -- 172.21.15.12:0/4132991929 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b4017420 con 0x5625556a42d0 2024-04-22T13:23:33.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.139+0000 7fc2b8bfe640 1 -- 172.21.15.12:0/4132991929 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2b4017a70 con 0x5625556a42d0 2024-04-22T13:23:33.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.139+0000 7fc2b8bfe640 1 -- 172.21.15.12:0/4132991929 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b400fac0 con 0x5625556a42d0 2024-04-22T13:23:33.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.139+0000 7fc2cf01c740 1 -- 172.21.15.12:0/4132991929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625556a42d0 msgr2=0x5625556aaeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.139+0000 7fc2cf01c740 1 --2- 172.21.15.12:0/4132991929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625556a42d0 0x5625556aaeb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc2b400bb40 tx=0x7fc2b400d340 comp rx=0 tx=0).stop 2024-04-22T13:23:33.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.140+0000 7fc2cf01c740 1 -- 172.21.15.12:0/4132991929 shutdown_connections 2024-04-22T13:23:33.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.140+0000 7fc2cf01c740 1 --2- 172.21.15.12:0/4132991929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625556a42d0 0x5625556aaeb0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.140+0000 7fc2cf01c740 1 --2- 172.21.15.12:0/4132991929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625556a3830 0x5625556a3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.140+0000 7fc2cf01c740 1 --2- 172.21.15.12:0/4132991929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625556a2e40 0x5625556a3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.140+0000 7fc2cf01c740 1 -- 172.21.15.12:0/4132991929 >> 172.21.15.12:0/4132991929 conn(0x56255569ef40 msgr2=0x56255569d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:33.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.140+0000 7fc2cf01c740 1 -- 172.21.15.12:0/4132991929 shutdown_connections 2024-04-22T13:23:33.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.140+0000 7fc2cf01c740 1 -- 172.21.15.12:0/4132991929 wait complete. 2024-04-22T13:23:33.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.140+0000 7fc2cf01c740 1 -- start start 2024-04-22T13:23:33.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.140+0000 7fc2cf01c740 1 Processor -- start 2024-04-22T13:23:33.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.140+0000 7fc2cf01c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625556a2e40 0x5625556ace50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.140+0000 7fc2cf01c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625556a3830 0x5625556ad390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.140+0000 7fc2cf01c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625556a42d0 0x5625556b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.140+0000 7fc2cf01c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5625556ab4b0 con 0x5625556a2e40 2024-04-22T13:23:33.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.140+0000 7fc2cf01c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x562555590190 con 0x5625556a42d0 2024-04-22T13:23:33.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.140+0000 7fc2cf01c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5625556ab630 con 0x5625556a3830 2024-04-22T13:23:33.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.140+0000 7fc2cf01c740 10 client.0 _ll_register_callbacks cb 0x56255558b8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:23:33.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.140+0000 7fc2ca0ec640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625556a2e40 0x5625556ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.140+0000 7fc2ca0ec640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625556a2e40 0x5625556ace50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34890/0 (socket says 172.21.15.12:34890) 2024-04-22T13:23:33.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.140+0000 7fc2ca0ec640 1 -- 172.21.15.12:0/1733957894 learned_addr learned my addr 172.21.15.12:0/1733957894 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:33.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.140+0000 7fc2ca8ed640 1 --2- 172.21.15.12:0/1733957894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625556a42d0 0x5625556b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.141+0000 7fc2ca0ec640 1 -- 172.21.15.12:0/1733957894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625556a3830 msgr2=0x5625556ad390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.141+0000 7fc2ca0ec640 1 --2- 172.21.15.12:0/1733957894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625556a3830 0x5625556ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.141+0000 7fc2ca0ec640 1 -- 172.21.15.12:0/1733957894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625556a42d0 msgr2=0x5625556b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.141+0000 7fc2ca0ec640 1 --2- 172.21.15.12:0/1733957894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625556a42d0 0x5625556b43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.141+0000 7fc2ca0ec640 1 -- 172.21.15.12:0/1733957894 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625553e46b0 con 0x5625556a2e40 2024-04-22T13:23:33.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.141+0000 7fc2ca8ed640 1 --2- 172.21.15.12:0/1733957894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625556a42d0 0x5625556b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:23:33.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.141+0000 7fc2ca0ec640 1 --2- 172.21.15.12:0/1733957894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625556a2e40 0x5625556ace50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc2c00bb0d0 tx=0x7fc2c008d790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9750a74403212b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:33.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.141+0000 7fc2adffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:23:33.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.141+0000 7fc2adffb640 1 -- 172.21.15.12:0/1733957894 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0004c50 con 0x5625556a2e40 2024-04-22T13:23:33.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.141+0000 7fc2adffb640 1 -- 172.21.15.12:0/1733957894 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fc2c00035f0 con 0x5625556a2e40 2024-04-22T13:23:33.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.141+0000 7fc2cf01c740 1 -- 172.21.15.12:0/1733957894 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2b4017a70 con 0x5625556a2e40 2024-04-22T13:23:33.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.141+0000 7fc2adffb640 1 -- 172.21.15.12:0/1733957894 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc2c00898f0 con 0x5625556a2e40 2024-04-22T13:23:33.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.141+0000 7fc2cf01c740 10 client.15075 fetch_fsmap learned FSMap version 27 2024-04-22T13:23:33.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.141+0000 7fc2cf01c740 1 -- 172.21.15.12:0/1733957894 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5625556b7740 con 0x5625556a2e40 2024-04-22T13:23:33.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.141+0000 7fc2adffb640 1 -- 172.21.15.12:0/1733957894 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2b4017a70 con 0x5625556a2e40 2024-04-22T13:23:33.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.141+0000 7fc2adffb640 1 -- 172.21.15.12:0/1733957894 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc2c00beed0 con 0x5625556a2e40 2024-04-22T13:23:33.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.141+0000 7fc2cf01c740 10 client.15075 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:23:33.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.141+0000 7fc2cf01c740 10 client.15075 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-22T13:23:33.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.141+0000 7fc2cf01c740 20 client.15075 populate_metadata read hostname 'smithi012' 2024-04-22T13:23:33.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.141+0000 7fc2cf01c740 1 --2- 172.21.15.12:0/1733957894 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5625556b9ec0 0x5625556bb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.141+0000 7fc2cf01c740 4 client.15075 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:23:33.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.141+0000 7fc2cf01c740 1 -- 172.21.15.12:0/1733957894 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562555615530 con 0x5625556b9ec0 2024-04-22T13:23:33.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.142+0000 7fc2c98eb640 1 --2- 172.21.15.12:0/1733957894 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5625556b9ec0 0x5625556bb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.142+0000 7fc2c98eb640 1 --2- 172.21.15.12:0/1733957894 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5625556b9ec0 0x5625556bb330 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=477e073aa8225657 server_cookie=917a26870f0898ca in_seq=0 out_seq=0 2024-04-22T13:23:33.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.142+0000 7fc2adffb640 10 client.15075 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:23:33.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.143+0000 7fc2adffb640 1 -- 172.21.15.12:0/1733957894 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562555615530 con 0x5625556b9ec0 2024-04-22T13:23:33.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.143+0000 7fc2adffb640 10 client.15075 handle_command_reply: tid=0 2024-04-22T13:23:33.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.143+0000 7fc2cf01c740 1 client.15075 shutdown 2024-04-22T13:23:33.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.143+0000 7fc2cf01c740 1 -- 172.21.15.12:0/1733957894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625556a2e40 msgr2=0x5625556ace50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.143+0000 7fc2cf01c740 1 --2- 172.21.15.12:0/1733957894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625556a2e40 0x5625556ace50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc2c00bb0d0 tx=0x7fc2c008d790 comp rx=0 tx=0).stop 2024-04-22T13:23:33.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.143+0000 7fc2cf01c740 1 -- 172.21.15.12:0/1733957894 shutdown_connections 2024-04-22T13:23:33.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.143+0000 7fc2cf01c740 1 --2- 172.21.15.12:0/1733957894 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5625556b9ec0 0x5625556bb330 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.143+0000 7fc2cf01c740 1 --2- 172.21.15.12:0/1733957894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625556a42d0 0x5625556b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.143+0000 7fc2adffb640 0 client.15075 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:23:33.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.143+0000 7fc2cf01c740 1 --2- 172.21.15.12:0/1733957894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625556a3830 0x5625556ad390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.143+0000 7fc2cf01c740 1 --2- 172.21.15.12:0/1733957894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625556a2e40 0x5625556ace50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.143+0000 7fc2cf01c740 1 -- 172.21.15.12:0/1733957894 >> 172.21.15.12:0/1733957894 conn(0x56255569ef40 msgr2=0x56255569ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:33.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.143+0000 7fc2cf01c740 1 -- 172.21.15.12:0/1733957894 shutdown_connections 2024-04-22T13:23:33.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.143+0000 7fc2cf01c740 1 -- 172.21.15.12:0/1733957894 wait complete. 2024-04-22T13:23:33.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.143+0000 7fc2cf01c740 20 client.15075 trim_cache size 0 max 16384 2024-04-22T13:23:33.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.161+0000 7fc2cf01c740 1 Processor -- start 2024-04-22T13:23:33.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.162+0000 7fc2cf01c740 1 -- start start 2024-04-22T13:23:33.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.162+0000 7fc2cf01c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625556a2e40 0x5625556aca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.162+0000 7fc2cf01c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625556a3830 0x5625556b4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.162+0000 7fc2cf01c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625556a42d0 0x5625556acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.162+0000 7fc2cf01c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5625556b9670 con 0x5625556a3830 2024-04-22T13:23:33.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.162+0000 7fc2cf01c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5625556b9d40 con 0x5625556a2e40 2024-04-22T13:23:33.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.162+0000 7fc2cf01c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5625556ba410 con 0x5625556a42d0 2024-04-22T13:23:33.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.162+0000 7fc2c98eb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625556a3830 0x5625556b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.162+0000 7fc2c98eb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625556a3830 0x5625556b4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34906/0 (socket says 172.21.15.12:34906) 2024-04-22T13:23:33.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.162+0000 7fc2c98eb640 1 -- 172.21.15.12:0/1267051361 learned_addr learned my addr 172.21.15.12:0/1267051361 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:33.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.162+0000 7fc2c98eb640 1 -- 172.21.15.12:0/1267051361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625556a42d0 msgr2=0x5625556acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.162+0000 7fc2c98eb640 1 --2- 172.21.15.12:0/1267051361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625556a42d0 0x5625556acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.162+0000 7fc2c98eb640 1 -- 172.21.15.12:0/1267051361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625556a2e40 msgr2=0x5625556aca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.162+0000 7fc2c98eb640 1 --2- 172.21.15.12:0/1267051361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625556a2e40 0x5625556aca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.162+0000 7fc2c98eb640 1 -- 172.21.15.12:0/1267051361 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56255569e230 con 0x5625556a3830 2024-04-22T13:23:33.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.162+0000 7fc2c98eb640 1 --2- 172.21.15.12:0/1267051361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625556a3830 0x5625556b4390 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x56255569e410 tx=0x7fc2b400df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f3686f0ece5ff3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:33.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.162+0000 7fc2affff640 1 -- 172.21.15.12:0/1267051361 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b4027250 con 0x5625556a3830 2024-04-22T13:23:33.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.162+0000 7fc2affff640 1 -- 172.21.15.12:0/1267051361 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2b402b030 con 0x5625556a3830 2024-04-22T13:23:33.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.162+0000 7fc2affff640 1 -- 172.21.15.12:0/1267051361 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b400f8c0 con 0x5625556a3830 2024-04-22T13:23:33.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.163+0000 7fc2cf01c740 1 -- 172.21.15.12:0/1267051361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625556a3830 msgr2=0x5625556b4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.163+0000 7fc2cf01c740 1 --2- 172.21.15.12:0/1267051361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625556a3830 0x5625556b4390 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x56255569e410 tx=0x7fc2b400df10 comp rx=0 tx=0).stop 2024-04-22T13:23:33.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.163+0000 7fc2cf01c740 1 -- 172.21.15.12:0/1267051361 shutdown_connections 2024-04-22T13:23:33.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.163+0000 7fc2cf01c740 1 --2- 172.21.15.12:0/1267051361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625556a42d0 0x5625556acfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.163+0000 7fc2cf01c740 1 --2- 172.21.15.12:0/1267051361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625556a3830 0x5625556b4390 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.163+0000 7fc2cf01c740 1 --2- 172.21.15.12:0/1267051361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625556a2e40 0x5625556aca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.163+0000 7fc2cf01c740 1 -- 172.21.15.12:0/1267051361 >> 172.21.15.12:0/1267051361 conn(0x56255569ef40 msgr2=0x5625556b8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:33.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.163+0000 7fc2cf01c740 1 -- 172.21.15.12:0/1267051361 shutdown_connections 2024-04-22T13:23:33.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.163+0000 7fc2cf01c740 1 -- 172.21.15.12:0/1267051361 wait complete. 2024-04-22T13:23:33.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.163+0000 7fc2cf01c740 1 -- start start 2024-04-22T13:23:33.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.163+0000 7fc2cf01c740 1 Processor -- start 2024-04-22T13:23:33.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.163+0000 7fc2cf01c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625556a2e40 0x5625556afc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.163+0000 7fc2cf01c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625556a3830 0x5625556b0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.163+0000 7fc2cf01c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625556a42d0 0x5625556b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.163+0000 7fc2cf01c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5625556bb1b0 con 0x5625556a3830 2024-04-22T13:23:33.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.163+0000 7fc2cf01c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5625556baae0 con 0x5625556a2e40 2024-04-22T13:23:33.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.163+0000 7fc2cf01c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5625556bb880 con 0x5625556a42d0 2024-04-22T13:23:33.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.163+0000 7fc2cf01c740 10 client.0 _ll_register_callbacks cb 0x7fc2c0004510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:23:33.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.164+0000 7fc2c98eb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625556a3830 0x5625556b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.164+0000 7fc2c98eb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625556a3830 0x5625556b0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34914/0 (socket says 172.21.15.12:34914) 2024-04-22T13:23:33.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.164+0000 7fc2c98eb640 1 -- 172.21.15.12:0/1142419488 learned_addr learned my addr 172.21.15.12:0/1142419488 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:33.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.164+0000 7fc2ca8ed640 1 --2- 172.21.15.12:0/1142419488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625556a42d0 0x5625556b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.164+0000 7fc2c98eb640 1 -- 172.21.15.12:0/1142419488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625556a42d0 msgr2=0x5625556b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.164+0000 7fc2c98eb640 1 --2- 172.21.15.12:0/1142419488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625556a42d0 0x5625556b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.164+0000 7fc2c98eb640 1 -- 172.21.15.12:0/1142419488 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625556a2e40 msgr2=0x5625556afc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.164+0000 7fc2c98eb640 1 --2- 172.21.15.12:0/1142419488 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625556a2e40 0x5625556afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.164+0000 7fc2c98eb640 1 -- 172.21.15.12:0/1142419488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625556b3790 con 0x5625556a3830 2024-04-22T13:23:33.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.164+0000 7fc2ca8ed640 1 --2- 172.21.15.12:0/1142419488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625556a42d0 0x5625556b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:23:33.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.164+0000 7fc2c98eb640 1 --2- 172.21.15.12:0/1142419488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625556a3830 0x5625556b0180 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4005940 tx=0x7fc2b400b5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5317c36930a904 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:33.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.164+0000 7fc2adffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:23:33.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.164+0000 7fc2adffb640 1 -- 172.21.15.12:0/1142419488 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b4027250 con 0x5625556a3830 2024-04-22T13:23:33.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.164+0000 7fc2adffb640 1 -- 172.21.15.12:0/1142419488 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fc2b40273f0 con 0x5625556a3830 2024-04-22T13:23:33.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.164+0000 7fc2cf01c740 1 -- 172.21.15.12:0/1142419488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2b402b030 con 0x5625556a3830 2024-04-22T13:23:33.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.164+0000 7fc2adffb640 1 -- 172.21.15.12:0/1142419488 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc2b4040020 con 0x5625556a3830 2024-04-22T13:23:33.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.164+0000 7fc2cf01c740 10 client.15081 fetch_fsmap learned FSMap version 27 2024-04-22T13:23:33.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.164+0000 7fc2cf01c740 1 -- 172.21.15.12:0/1142419488 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562555720030 con 0x5625556a3830 2024-04-22T13:23:33.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.164+0000 7fc2adffb640 1 -- 172.21.15.12:0/1142419488 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2b402b030 con 0x5625556a3830 2024-04-22T13:23:33.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.165+0000 7fc2adffb640 1 -- 172.21.15.12:0/1142419488 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc2b40047c0 con 0x5625556a3830 2024-04-22T13:23:33.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.165+0000 7fc2cf01c740 10 client.15081 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:23:33.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.165+0000 7fc2cf01c740 10 client.15081 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-22T13:23:33.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.165+0000 7fc2cf01c740 20 client.15081 populate_metadata read hostname 'smithi012' 2024-04-22T13:23:33.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.165+0000 7fc2cf01c740 1 --2- 172.21.15.12:0/1142419488 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5625557245d0 0x5625557232e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.165+0000 7fc2cf01c740 4 client.15081 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T13:23:33.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.165+0000 7fc2cf01c740 1 -- 172.21.15.12:0/1142419488 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "status"}) -- 0x56255560f3a0 con 0x5625557245d0 2024-04-22T13:23:33.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.165+0000 7fc2ca0ec640 1 --2- 172.21.15.12:0/1142419488 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5625557245d0 0x5625557232e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.165+0000 7fc2adffb640 10 client.15081 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:23:33.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.165+0000 7fc2ca0ec640 1 --2- 172.21.15.12:0/1142419488 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5625557245d0 0x5625557232e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e869d5cdd703b66f server_cookie=4d5d9762c1867fed in_seq=0 out_seq=0 2024-04-22T13:23:33.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.165+0000 7fc2adffb640 1 -- 172.21.15.12:0/1142419488 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56255560f3a0 con 0x5625557245d0 2024-04-22T13:23:33.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.165+0000 7fc2adffb640 10 client.15081 handle_command_reply: tid=0 2024-04-22T13:23:33.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.165+0000 7fc2cf01c740 1 client.15081 shutdown 2024-04-22T13:23:33.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.166+0000 7fc2cf01c740 1 -- 172.21.15.12:0/1142419488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625556a3830 msgr2=0x5625556b0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.166+0000 7fc2cf01c740 1 --2- 172.21.15.12:0/1142419488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625556a3830 0x5625556b0180 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4005940 tx=0x7fc2b400b5f0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.166+0000 7fc2cf01c740 1 -- 172.21.15.12:0/1142419488 shutdown_connections 2024-04-22T13:23:33.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.166+0000 7fc2cf01c740 1 --2- 172.21.15.12:0/1142419488 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5625557245d0 0x5625557232e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.166+0000 7fc2cf01c740 1 --2- 172.21.15.12:0/1142419488 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5625556a42d0 0x5625556b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.166+0000 7fc2adffb640 0 client.15081 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:23:33.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.166+0000 7fc2cf01c740 1 --2- 172.21.15.12:0/1142419488 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625556a3830 0x5625556b0180 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.166+0000 7fc2cf01c740 1 --2- 172.21.15.12:0/1142419488 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625556a2e40 0x5625556afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.166+0000 7fc2cf01c740 1 -- 172.21.15.12:0/1142419488 >> 172.21.15.12:0/1142419488 conn(0x56255569ef40 msgr2=0x5625556b8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:33.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.166+0000 7fc2cf01c740 1 -- 172.21.15.12:0/1142419488 shutdown_connections 2024-04-22T13:23:33.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.166+0000 7fc2cf01c740 1 -- 172.21.15.12:0/1142419488 wait complete. 2024-04-22T13:23:33.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.166+0000 7fc2cf01c740 20 client.15081 trim_cache size 0 max 16384 2024-04-22T13:23:33.168 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:23:33.168 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "0aede5ea-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:23:33.168 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 3, 2024-04-22T13:23:33.168 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24443, 2024-04-22T13:23:33.169 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-04-22T13:23:33.169 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-04-22T13:23:33.169 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-04-22T13:23:33.169 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 82.782132802000007, 2024-04-22T13:23:33.169 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 27, 2024-04-22T13:23:33.169 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 89, 2024-04-22T13:23:33.169 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 87, 2024-04-22T13:23:33.169 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 112.851496113 2024-04-22T13:23:33.169 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:23:33.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.166+0000 7fc2cc350640 1 -- 172.21.15.12:0/2038054000 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc2b00a3cc0 msgr2=0x7fc2b00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.166+0000 7fc2cc350640 1 --2- 172.21.15.12:0/2038054000 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc2b00a3cc0 0x7fc2b00a6180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x56255569e200 tx=0x7fc2bc005e00 comp rx=0 tx=0).stop 2024-04-22T13:23:33.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.166+0000 7fc2cc350640 1 -- 172.21.15.12:0/2038054000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2c4101fb0 msgr2=0x7fc2c4198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.166+0000 7fc2cc350640 1 --2- 172.21.15.12:0/2038054000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2c4101fb0 0x7fc2c4198260 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0036040 tx=0x7fc2c00042f0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.166+0000 7fc2cc350640 1 -- 172.21.15.12:0/2038054000 shutdown_connections 2024-04-22T13:23:33.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.166+0000 7fc2cc350640 1 --2- 172.21.15.12:0/2038054000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2c4103b60 0x7fc2c419f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.166+0000 7fc2cc350640 1 --2- 172.21.15.12:0/2038054000 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2c41031a0 0x7fc2c41987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.166+0000 7fc2cc350640 1 --2- 172.21.15.12:0/2038054000 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc2b00a3cc0 0x7fc2b00a6180 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.166+0000 7fc2cc350640 1 --2- 172.21.15.12:0/2038054000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2c4101fb0 0x7fc2c4198260 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.166+0000 7fc2cc350640 1 -- 172.21.15.12:0/2038054000 >> 172.21.15.12:0/2038054000 conn(0x7fc2c40fd520 msgr2=0x7fc2c41065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:33.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.166+0000 7fc2cc350640 1 -- 172.21.15.12:0/2038054000 shutdown_connections 2024-04-22T13:23:33.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.166+0000 7fc2cc350640 1 -- 172.21.15.12:0/2038054000 wait complete. 2024-04-22T13:23:33.173 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-22T13:23:33.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.282+0000 7f60374e4640 1 -- 172.21.15.12:0/3462200252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6030101fe0 msgr2=0x7f6030102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.282+0000 7f60374e4640 1 --2- 172.21.15.12:0/3462200252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6030101fe0 0x7f6030102460 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6020009900 tx=0x7f602002bc30 comp rx=0 tx=0).stop 2024-04-22T13:23:33.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.282+0000 7f60374e4640 1 -- 172.21.15.12:0/3462200252 shutdown_connections 2024-04-22T13:23:33.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.282+0000 7f60374e4640 1 --2- 172.21.15.12:0/3462200252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60301029a0 0x7f60301092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.282+0000 7f60374e4640 1 --2- 172.21.15.12:0/3462200252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6030101fe0 0x7f6030102460 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.282+0000 7f60374e4640 1 --2- 172.21.15.12:0/3462200252 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6030100e90 0x7f60301012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.282+0000 7f60374e4640 1 -- 172.21.15.12:0/3462200252 >> 172.21.15.12:0/3462200252 conn(0x7f60300fc400 msgr2=0x7f60300fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:33.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.282+0000 7f60374e4640 1 -- 172.21.15.12:0/3462200252 shutdown_connections 2024-04-22T13:23:33.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.282+0000 7f60374e4640 1 -- 172.21.15.12:0/3462200252 wait complete. 2024-04-22T13:23:33.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.282+0000 7f60374e4640 1 Processor -- start 2024-04-22T13:23:33.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.282+0000 7f60374e4640 1 -- start start 2024-04-22T13:23:33.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.283+0000 7f60374e4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6030100e90 0x7f60301981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.283+0000 7f60374e4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6030101fe0 0x7f6030198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.283+0000 7f60374e4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60301029a0 0x7f603019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.283+0000 7f60374e4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f603010c9d0 con 0x7f60301029a0 2024-04-22T13:23:33.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.283+0000 7f60374e4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f603010c850 con 0x7f6030101fe0 2024-04-22T13:23:33.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.283+0000 7f60374e4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f603010cb50 con 0x7f6030100e90 2024-04-22T13:23:33.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.283+0000 7f6035a81640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60301029a0 0x7f603019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.283+0000 7f6035a81640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60301029a0 0x7f603019f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34944/0 (socket says 172.21.15.12:34944) 2024-04-22T13:23:33.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.283+0000 7f6035a81640 1 -- 172.21.15.12:0/1250061828 learned_addr learned my addr 172.21.15.12:0/1250061828 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:33.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.283+0000 7f6034a7f640 1 --2- 172.21.15.12:0/1250061828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6030101fe0 0x7f6030198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.283+0000 7f6035280640 1 --2- 172.21.15.12:0/1250061828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6030100e90 0x7f60301981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.283+0000 7f6035a81640 1 -- 172.21.15.12:0/1250061828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6030100e90 msgr2=0x7f60301981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.283+0000 7f6035a81640 1 --2- 172.21.15.12:0/1250061828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6030100e90 0x7f60301981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.283+0000 7f6035a81640 1 -- 172.21.15.12:0/1250061828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6030101fe0 msgr2=0x7f6030198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.283+0000 7f6035a81640 1 --2- 172.21.15.12:0/1250061828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6030101fe0 0x7f6030198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.283+0000 7f6035a81640 1 -- 172.21.15.12:0/1250061828 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603019fe00 con 0x7f60301029a0 2024-04-22T13:23:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.283+0000 7f6034a7f640 1 --2- 172.21.15.12:0/1250061828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6030101fe0 0x7f6030198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:23:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.283+0000 7f6035a81640 1 --2- 172.21.15.12:0/1250061828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60301029a0 0x7f603019f790 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f602800cce0 tx=0x7f602800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.283+0000 7f60267fc640 1 -- 172.21.15.12:0/1250061828 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602800f450 con 0x7f60301029a0 2024-04-22T13:23:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.283+0000 7f60267fc640 1 -- 172.21.15.12:0/1250061828 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f602800faa0 con 0x7f60301029a0 2024-04-22T13:23:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.283+0000 7f60374e4640 1 -- 172.21.15.12:0/1250061828 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60301a00f0 con 0x7f60301029a0 2024-04-22T13:23:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.283+0000 7f60267fc640 1 -- 172.21.15.12:0/1250061828 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6028018b70 con 0x7f60301029a0 2024-04-22T13:23:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.283+0000 7f60374e4640 1 -- 172.21.15.12:0/1250061828 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60301a05c0 con 0x7f60301029a0 2024-04-22T13:23:33.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.286+0000 7f60267fc640 1 -- 172.21.15.12:0/1250061828 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6028018d10 con 0x7f60301029a0 2024-04-22T13:23:33.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.286+0000 7f603a1b4740 1 Processor -- start 2024-04-22T13:23:33.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.286+0000 7f603a1b4740 1 -- start start 2024-04-22T13:23:33.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.286+0000 7f603a1b4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578093b0e40 0x5578093b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.286+0000 7f603a1b4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578093b1830 0x5578093b1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.286+0000 7f603a1b4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578093b22d0 0x5578093b8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.286+0000 7f603a1b4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557809315b10 con 0x5578093b22d0 2024-04-22T13:23:33.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.286+0000 7f603a1b4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5578092b3240 con 0x5578093b1830 2024-04-22T13:23:33.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.286+0000 7f603a1b4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557809311ad0 con 0x5578093b0e40 2024-04-22T13:23:33.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.286+0000 7f6034a7f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578093b22d0 0x5578093b8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.286+0000 7f6034a7f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578093b22d0 0x5578093b8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34960/0 (socket says 172.21.15.12:34960) 2024-04-22T13:23:33.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.286+0000 7f6034a7f640 1 -- 172.21.15.12:0/1807907471 learned_addr learned my addr 172.21.15.12:0/1807907471 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:33.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.287+0000 7f60267fc640 1 --2- 172.21.15.12:0/1250061828 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f601c0a3c40 0x7f601c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.287+0000 7f6034a7f640 1 --2- 172.21.15.12:0/1807907471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578093b0e40 0x5578093b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.287+0000 7f6035280640 1 --2- 172.21.15.12:0/1807907471 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578093b1830 0x5578093b1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.287+0000 7f6035a81640 1 --2- 172.21.15.12:0/1250061828 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f601c0a3c40 0x7f601c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.287+0000 7f6034a7f640 1 -- 172.21.15.12:0/1807907471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578093b0e40 msgr2=0x5578093b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.287+0000 7f6034a7f640 1 --2- 172.21.15.12:0/1807907471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578093b0e40 0x5578093b1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.287+0000 7f6034a7f640 1 -- 172.21.15.12:0/1807907471 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578093b1830 msgr2=0x5578093b1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.287+0000 7f6034a7f640 1 --2- 172.21.15.12:0/1807907471 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578093b1830 0x5578093b1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.287+0000 7f6034a7f640 1 -- 172.21.15.12:0/1807907471 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578090282d0 con 0x5578093b22d0 2024-04-22T13:23:33.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.287+0000 7f6035280640 1 --2- 172.21.15.12:0/1807907471 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578093b1830 0x5578093b1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:23:33.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.287+0000 7f60267fc640 1 -- 172.21.15.12:0/1250061828 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f60280d0ab0 con 0x7f60301029a0 2024-04-22T13:23:33.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.287+0000 7f6034a7f640 1 --2- 172.21.15.12:0/1807907471 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578093b22d0 0x5578093b8eb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6020009900 tx=0x7f6020009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c3e3e4aa41363b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:33.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.287+0000 7f601b7fe640 1 -- 172.21.15.12:0/1807907471 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60200042c0 con 0x5578093b22d0 2024-04-22T13:23:33.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.287+0000 7f601b7fe640 1 -- 172.21.15.12:0/1807907471 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6020048070 con 0x5578093b22d0 2024-04-22T13:23:33.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.287+0000 7f6035a81640 1 --2- 172.21.15.12:0/1250061828 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f601c0a3c40 0x7f601c0a6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6028062440 tx=0x7f602806c530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:33.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.287+0000 7f601b7fe640 1 -- 172.21.15.12:0/1807907471 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6020004c20 con 0x5578093b22d0 2024-04-22T13:23:33.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.288+0000 7f603a1b4740 1 -- 172.21.15.12:0/1807907471 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578093b22d0 msgr2=0x5578093b8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.288+0000 7f603a1b4740 1 --2- 172.21.15.12:0/1807907471 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578093b22d0 0x5578093b8eb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6020009900 tx=0x7f6020009560 comp rx=0 tx=0).stop 2024-04-22T13:23:33.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.288+0000 7f603a1b4740 1 -- 172.21.15.12:0/1807907471 shutdown_connections 2024-04-22T13:23:33.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.288+0000 7f603a1b4740 1 --2- 172.21.15.12:0/1807907471 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578093b22d0 0x5578093b8eb0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.288+0000 7f603a1b4740 1 --2- 172.21.15.12:0/1807907471 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578093b1830 0x5578093b1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.288+0000 7f603a1b4740 1 --2- 172.21.15.12:0/1807907471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578093b0e40 0x5578093b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.288+0000 7f603a1b4740 1 -- 172.21.15.12:0/1807907471 >> 172.21.15.12:0/1807907471 conn(0x5578093acf40 msgr2=0x5578093ab490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:33.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.288+0000 7f603a1b4740 1 -- 172.21.15.12:0/1807907471 shutdown_connections 2024-04-22T13:23:33.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.288+0000 7f603a1b4740 1 -- 172.21.15.12:0/1807907471 wait complete. 2024-04-22T13:23:33.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.288+0000 7f603a1b4740 1 -- start start 2024-04-22T13:23:33.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.288+0000 7f603a1b4740 1 Processor -- start 2024-04-22T13:23:33.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.289+0000 7f603a1b4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578093b0e40 0x5578093bae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.289+0000 7f603a1b4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578093b1830 0x5578093bb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.289+0000 7f603a1b4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578093b22d0 0x5578093c2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.289+0000 7f603a1b4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5578093b9450 con 0x5578093b0e40 2024-04-22T13:23:33.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.289+0000 7f603a1b4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55780929e190 con 0x5578093b1830 2024-04-22T13:23:33.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.289+0000 7f603a1b4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5578093b95d0 con 0x5578093b22d0 2024-04-22T13:23:33.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.289+0000 7f603a1b4740 10 client.0 _ll_register_callbacks cb 0x5578092998b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:23:33.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.289+0000 7f6034a7f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578093b0e40 0x5578093bae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.289+0000 7f6034a7f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578093b0e40 0x5578093bae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34972/0 (socket says 172.21.15.12:34972) 2024-04-22T13:23:33.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.289+0000 7f6034a7f640 1 -- 172.21.15.12:0/3318481103 learned_addr learned my addr 172.21.15.12:0/3318481103 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:33.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.289+0000 7f6034a7f640 1 --2- 172.21.15.12:0/3318481103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578093b22d0 0x5578093c2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.289+0000 7f6035280640 1 --2- 172.21.15.12:0/3318481103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578093b1830 0x5578093bb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.289+0000 7f6034a7f640 1 -- 172.21.15.12:0/3318481103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578093b22d0 msgr2=0x5578093c2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.289+0000 7f6034a7f640 1 --2- 172.21.15.12:0/3318481103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578093b22d0 0x5578093c2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.289+0000 7f6034a7f640 1 -- 172.21.15.12:0/3318481103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578093b1830 msgr2=0x5578093bb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.289+0000 7f6034a7f640 1 --2- 172.21.15.12:0/3318481103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578093b1830 0x5578093bb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.289+0000 7f6034a7f640 1 -- 172.21.15.12:0/3318481103 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578090f26b0 con 0x5578093b0e40 2024-04-22T13:23:33.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.289+0000 7f6034a7f640 1 --2- 172.21.15.12:0/3318481103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578093b0e40 0x5578093bae90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f60200044f0 tx=0x7f6020009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ed0f6daf6e88a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:33.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.289+0000 7f6018ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:23:33.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.290+0000 7f6018ff9640 1 -- 172.21.15.12:0/3318481103 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6020003a00 con 0x5578093b0e40 2024-04-22T13:23:33.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.290+0000 7f6018ff9640 1 -- 172.21.15.12:0/3318481103 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f6020003ba0 con 0x5578093b0e40 2024-04-22T13:23:33.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.290+0000 7f603a1b4740 1 -- 172.21.15.12:0/3318481103 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6020048070 con 0x5578093b0e40 2024-04-22T13:23:33.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.290+0000 7f6018ff9640 1 -- 172.21.15.12:0/3318481103 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6020003da0 con 0x5578093b0e40 2024-04-22T13:23:33.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.290+0000 7f603a1b4740 10 client.15093 fetch_fsmap learned FSMap version 27 2024-04-22T13:23:33.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.290+0000 7f603a1b4740 1 -- 172.21.15.12:0/3318481103 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5578093c5780 con 0x5578093b0e40 2024-04-22T13:23:33.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.290+0000 7f6018ff9640 1 -- 172.21.15.12:0/3318481103 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6020048070 con 0x5578093b0e40 2024-04-22T13:23:33.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.290+0000 7f6018ff9640 1 -- 172.21.15.12:0/3318481103 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f6020043c80 con 0x5578093b0e40 2024-04-22T13:23:33.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.290+0000 7f603a1b4740 10 client.15093 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:23:33.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.290+0000 7f603a1b4740 10 client.15093 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-22T13:23:33.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.290+0000 7f603a1b4740 20 client.15093 populate_metadata read hostname 'smithi012' 2024-04-22T13:23:33.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.290+0000 7f603a1b4740 1 --2- 172.21.15.12:0/3318481103 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x5578093c78e0 0x5578093c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.290+0000 7f603a1b4740 4 client.15093 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:23:33.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.290+0000 7f603a1b4740 1 -- 172.21.15.12:0/3318481103 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557809323530 con 0x5578093c78e0 2024-04-22T13:23:33.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.291+0000 7f6035280640 1 --2- 172.21.15.12:0/3318481103 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x5578093c78e0 0x5578093c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.291+0000 7f6035280640 1 --2- 172.21.15.12:0/3318481103 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x5578093c78e0 0x5578093c8d50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c33f39344a8f7dcb server_cookie=74d3820b00b0f842 in_seq=0 out_seq=0 2024-04-22T13:23:33.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.291+0000 7f6018ff9640 10 client.15093 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:23:33.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.292+0000 7f6018ff9640 1 -- 172.21.15.12:0/3318481103 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557809323530 con 0x5578093c78e0 2024-04-22T13:23:33.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.292+0000 7f6018ff9640 10 client.15093 handle_command_reply: tid=0 2024-04-22T13:23:33.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.292+0000 7f603a1b4740 1 client.15093 shutdown 2024-04-22T13:23:33.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.292+0000 7f603a1b4740 1 -- 172.21.15.12:0/3318481103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578093b0e40 msgr2=0x5578093bae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.292+0000 7f603a1b4740 1 --2- 172.21.15.12:0/3318481103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578093b0e40 0x5578093bae90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f60200044f0 tx=0x7f6020009650 comp rx=0 tx=0).stop 2024-04-22T13:23:33.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.292+0000 7f603a1b4740 1 -- 172.21.15.12:0/3318481103 shutdown_connections 2024-04-22T13:23:33.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.292+0000 7f603a1b4740 1 --2- 172.21.15.12:0/3318481103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578093b22d0 0x5578093c2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.292+0000 7f603a1b4740 1 --2- 172.21.15.12:0/3318481103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578093b1830 0x5578093bb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.292+0000 7f603a1b4740 1 --2- 172.21.15.12:0/3318481103 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x5578093c78e0 0x5578093c8d50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.292+0000 7f603a1b4740 1 --2- 172.21.15.12:0/3318481103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578093b0e40 0x5578093bae90 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.292+0000 7f603a1b4740 1 -- 172.21.15.12:0/3318481103 >> 172.21.15.12:0/3318481103 conn(0x5578093acf40 msgr2=0x5578093aaa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:33.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.292+0000 7f6018ff9640 0 client.15093 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:23:33.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.292+0000 7f603a1b4740 1 -- 172.21.15.12:0/3318481103 shutdown_connections 2024-04-22T13:23:33.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.292+0000 7f603a1b4740 1 -- 172.21.15.12:0/3318481103 wait complete. 2024-04-22T13:23:33.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.292+0000 7f603a1b4740 20 client.15093 trim_cache size 0 max 16384 2024-04-22T13:23:33.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.310+0000 7f603a1b4740 1 Processor -- start 2024-04-22T13:23:33.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.310+0000 7f603a1b4740 1 -- start start 2024-04-22T13:23:33.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.310+0000 7f603a1b4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578093b0e40 0x5578093ab350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.310+0000 7f603a1b4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578093b1830 0x5578093b8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.310+0000 7f603a1b4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578093b22d0 0x5578093b5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.310+0000 7f603a1b4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5578093c7090 con 0x5578093b1830 2024-04-22T13:23:33.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.310+0000 7f603a1b4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5578093c7760 con 0x5578093b22d0 2024-04-22T13:23:33.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.310+0000 7f603a1b4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5578093c7e30 con 0x5578093b0e40 2024-04-22T13:23:33.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.310+0000 7f6035280640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578093b1830 0x5578093b8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.310+0000 7f6035280640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578093b1830 0x5578093b8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34986/0 (socket says 172.21.15.12:34986) 2024-04-22T13:23:33.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.310+0000 7f6035280640 1 -- 172.21.15.12:0/3739213539 learned_addr learned my addr 172.21.15.12:0/3739213539 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:33.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.311+0000 7f6034a7f640 1 --2- 172.21.15.12:0/3739213539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578093b22d0 0x5578093b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.311+0000 7f6035280640 1 -- 172.21.15.12:0/3739213539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578093b0e40 msgr2=0x5578093ab350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.311+0000 7f6035280640 1 --2- 172.21.15.12:0/3739213539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578093b0e40 0x5578093ab350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.311+0000 7f6035280640 1 -- 172.21.15.12:0/3739213539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578093b22d0 msgr2=0x5578093b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.311+0000 7f6035280640 1 --2- 172.21.15.12:0/3739213539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578093b22d0 0x5578093b5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.311+0000 7f6035280640 1 -- 172.21.15.12:0/3739213539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578093ac630 con 0x5578093b1830 2024-04-22T13:23:33.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.311+0000 7f6035280640 1 --2- 172.21.15.12:0/3739213539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578093b1830 0x5578093b8ef0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f602c004700 tx=0x7f602c013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3587b91fee8f5b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:33.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.311+0000 7f601affd640 1 -- 172.21.15.12:0/3739213539 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c005020 con 0x5578093b1830 2024-04-22T13:23:33.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.311+0000 7f601affd640 1 -- 172.21.15.12:0/3739213539 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f602c0047a0 con 0x5578093b1830 2024-04-22T13:23:33.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.311+0000 7f601affd640 1 -- 172.21.15.12:0/3739213539 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c01d290 con 0x5578093b1830 2024-04-22T13:23:33.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.311+0000 7f603a1b4740 1 -- 172.21.15.12:0/3739213539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578093b1830 msgr2=0x5578093b8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.311+0000 7f603a1b4740 1 --2- 172.21.15.12:0/3739213539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578093b1830 0x5578093b8ef0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f602c004700 tx=0x7f602c013830 comp rx=0 tx=0).stop 2024-04-22T13:23:33.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.312+0000 7f603a1b4740 1 -- 172.21.15.12:0/3739213539 shutdown_connections 2024-04-22T13:23:33.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.312+0000 7f603a1b4740 1 --2- 172.21.15.12:0/3739213539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578093b22d0 0x5578093b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.312+0000 7f603a1b4740 1 --2- 172.21.15.12:0/3739213539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578093b1830 0x5578093b8ef0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.312+0000 7f603a1b4740 1 --2- 172.21.15.12:0/3739213539 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578093b0e40 0x5578093ab350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.312+0000 7f603a1b4740 1 -- 172.21.15.12:0/3739213539 >> 172.21.15.12:0/3739213539 conn(0x5578093acf40 msgr2=0x5578093a6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:33.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.312+0000 7f603a1b4740 1 -- 172.21.15.12:0/3739213539 shutdown_connections 2024-04-22T13:23:33.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.312+0000 7f603a1b4740 1 -- 172.21.15.12:0/3739213539 wait complete. 2024-04-22T13:23:33.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.312+0000 7f603a1b4740 1 -- start start 2024-04-22T13:23:33.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.312+0000 7f603a1b4740 1 Processor -- start 2024-04-22T13:23:33.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.312+0000 7f603a1b4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578093b0e40 0x5578093bddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.312+0000 7f603a1b4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578093b1830 0x5578093b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.312+0000 7f603a1b4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578093b22d0 0x55780942f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.312+0000 7f603a1b4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5578093c8bd0 con 0x5578093b1830 2024-04-22T13:23:33.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.312+0000 7f603a1b4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5578093c8500 con 0x5578093b0e40 2024-04-22T13:23:33.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.312+0000 7f603a1b4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5578093c92a0 con 0x5578093b22d0 2024-04-22T13:23:33.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.312+0000 7f603a1b4740 10 client.0 _ll_register_callbacks cb 0x7f602002cc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:23:33.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.312+0000 7f6035280640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578093b1830 0x5578093b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.312+0000 7f6035280640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578093b1830 0x5578093b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34988/0 (socket says 172.21.15.12:34988) 2024-04-22T13:23:33.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.312+0000 7f6035280640 1 -- 172.21.15.12:0/1342841874 learned_addr learned my addr 172.21.15.12:0/1342841874 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:33.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.313+0000 7f6034a7f640 1 --2- 172.21.15.12:0/1342841874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578093b22d0 0x55780942f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.313+0000 7f6034a7f640 1 --2- 172.21.15.12:0/1342841874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578093b0e40 0x5578093bddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.313+0000 7f6035280640 1 -- 172.21.15.12:0/1342841874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578093b22d0 msgr2=0x55780942f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.313+0000 7f6035280640 1 --2- 172.21.15.12:0/1342841874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578093b22d0 0x55780942f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.313+0000 7f6035280640 1 -- 172.21.15.12:0/1342841874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578093b0e40 msgr2=0x5578093bddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.313+0000 7f6035280640 1 --2- 172.21.15.12:0/1342841874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578093b0e40 0x5578093bddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.313+0000 7f6035280640 1 -- 172.21.15.12:0/1342841874 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55780942ffe0 con 0x5578093b1830 2024-04-22T13:23:33.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.313+0000 7f6034a7f640 1 --2- 172.21.15.12:0/1342841874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578093b22d0 0x55780942f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:23:33.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.313+0000 7f6035280640 1 --2- 172.21.15.12:0/1342841874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578093b1830 0x5578093b4e90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f602c005c50 tx=0x7f602c005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=746d76b2ab6366c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:33.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.313+0000 7f6018ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:23:33.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.313+0000 7f6018ff9640 1 -- 172.21.15.12:0/1342841874 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c004220 con 0x5578093b1830 2024-04-22T13:23:33.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.313+0000 7f6018ff9640 1 -- 172.21.15.12:0/1342841874 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f602c0043c0 con 0x5578093b1830 2024-04-22T13:23:33.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.313+0000 7f603a1b4740 1 -- 172.21.15.12:0/1342841874 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f602c0047a0 con 0x5578093b1830 2024-04-22T13:23:33.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.313+0000 7f6018ff9640 1 -- 172.21.15.12:0/1342841874 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f602c035070 con 0x5578093b1830 2024-04-22T13:23:33.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.313+0000 7f603a1b4740 10 client.15099 fetch_fsmap learned FSMap version 27 2024-04-22T13:23:33.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.313+0000 7f603a1b4740 1 -- 172.21.15.12:0/1342841874 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5578094312e0 con 0x5578093b1830 2024-04-22T13:23:33.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.313+0000 7f6018ff9640 1 -- 172.21.15.12:0/1342841874 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f602c0047a0 con 0x5578093b1830 2024-04-22T13:23:33.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.314+0000 7f6018ff9640 1 -- 172.21.15.12:0/1342841874 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f602c003b00 con 0x5578093b1830 2024-04-22T13:23:33.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.314+0000 7f603a1b4740 10 client.15099 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:23:33.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.314+0000 7f603a1b4740 10 client.15099 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-22T13:23:33.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.314+0000 7f603a1b4740 20 client.15099 populate_metadata read hostname 'smithi012' 2024-04-22T13:23:33.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.314+0000 7f603a1b4740 1 --2- 172.21.15.12:0/1342841874 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x557809435bc0 0x557809432000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.314+0000 7f603a1b4740 4 client.15099 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T13:23:33.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.314+0000 7f603a1b4740 1 -- 172.21.15.12:0/1342841874 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "status"}) -- 0x55780931d3a0 con 0x557809435bc0 2024-04-22T13:23:33.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.314+0000 7f6034a7f640 1 --2- 172.21.15.12:0/1342841874 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x557809435bc0 0x557809432000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.314+0000 7f6018ff9640 10 client.15099 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:23:33.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.314+0000 7f6034a7f640 1 --2- 172.21.15.12:0/1342841874 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x557809435bc0 0x557809432000 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2a08f9283e182bfa server_cookie=29715220f7cc623 in_seq=0 out_seq=0 2024-04-22T13:23:33.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.314+0000 7f6018ff9640 1 -- 172.21.15.12:0/1342841874 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55780931d3a0 con 0x557809435bc0 2024-04-22T13:23:33.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.314+0000 7f6018ff9640 10 client.15099 handle_command_reply: tid=0 2024-04-22T13:23:33.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.314+0000 7f603a1b4740 1 client.15099 shutdown 2024-04-22T13:23:33.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.315+0000 7f603a1b4740 1 -- 172.21.15.12:0/1342841874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578093b1830 msgr2=0x5578093b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.315+0000 7f603a1b4740 1 --2- 172.21.15.12:0/1342841874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578093b1830 0x5578093b4e90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f602c005c50 tx=0x7f602c005250 comp rx=0 tx=0).stop 2024-04-22T13:23:33.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.315+0000 7f603a1b4740 1 -- 172.21.15.12:0/1342841874 shutdown_connections 2024-04-22T13:23:33.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.315+0000 7f603a1b4740 1 --2- 172.21.15.12:0/1342841874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578093b22d0 0x55780942f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.315+0000 7f603a1b4740 1 --2- 172.21.15.12:0/1342841874 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x557809435bc0 0x557809432000 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.315+0000 7f603a1b4740 1 --2- 172.21.15.12:0/1342841874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578093b1830 0x5578093b4e90 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.315+0000 7f6018ff9640 0 client.15099 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:23:33.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.315+0000 7f603a1b4740 1 --2- 172.21.15.12:0/1342841874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578093b0e40 0x5578093bddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.315+0000 7f603a1b4740 1 -- 172.21.15.12:0/1342841874 >> 172.21.15.12:0/1342841874 conn(0x5578093acf40 msgr2=0x5578093b5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:33.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.315+0000 7f603a1b4740 1 -- 172.21.15.12:0/1342841874 shutdown_connections 2024-04-22T13:23:33.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.315+0000 7f603a1b4740 1 -- 172.21.15.12:0/1342841874 wait complete. 2024-04-22T13:23:33.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.315+0000 7f603a1b4740 20 client.15099 trim_cache size 0 max 16384 2024-04-22T13:23:33.317 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:23:33.317 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "0aede5ea-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:23:33.317 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 2, 2024-04-22T13:23:33.317 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24448, 2024-04-22T13:23:33.318 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-04-22T13:23:33.318 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-04-22T13:23:33.318 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-04-22T13:23:33.318 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 83.940079702000006, 2024-04-22T13:23:33.318 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 27, 2024-04-22T13:23:33.318 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 89, 2024-04-22T13:23:33.318 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 87, 2024-04-22T13:23:33.318 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 110.38781281 2024-04-22T13:23:33.318 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:23:33.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.315+0000 7f60374e4640 1 -- 172.21.15.12:0/1250061828 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f601c0a3c40 msgr2=0x7f601c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.315+0000 7f60374e4640 1 --2- 172.21.15.12:0/1250061828 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f601c0a3c40 0x7f601c0a6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6028062440 tx=0x7f602806c530 comp rx=0 tx=0).stop 2024-04-22T13:23:33.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.315+0000 7f60374e4640 1 -- 172.21.15.12:0/1250061828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60301029a0 msgr2=0x7f603019f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.315+0000 7f60374e4640 1 --2- 172.21.15.12:0/1250061828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60301029a0 0x7f603019f790 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f602800cce0 tx=0x7f602800a620 comp rx=0 tx=0).stop 2024-04-22T13:23:33.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.315+0000 7f60374e4640 1 -- 172.21.15.12:0/1250061828 shutdown_connections 2024-04-22T13:23:33.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.315+0000 7f60374e4640 1 --2- 172.21.15.12:0/1250061828 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f601c0a3c40 0x7f601c0a6100 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.315+0000 7f60374e4640 1 --2- 172.21.15.12:0/1250061828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f60301029a0 0x7f603019f790 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.315+0000 7f60374e4640 1 --2- 172.21.15.12:0/1250061828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6030101fe0 0x7f6030198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.315+0000 7f60374e4640 1 --2- 172.21.15.12:0/1250061828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6030100e90 0x7f60301981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.315+0000 7f60374e4640 1 -- 172.21.15.12:0/1250061828 >> 172.21.15.12:0/1250061828 conn(0x7f60300fc400 msgr2=0x7f60300690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:33.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.315+0000 7f60374e4640 1 -- 172.21.15.12:0/1250061828 shutdown_connections 2024-04-22T13:23:33.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.316+0000 7f60374e4640 1 -- 172.21.15.12:0/1250061828 wait complete. 2024-04-22T13:23:33.322 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-22T13:23:33.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.429+0000 7fbd1dac1640 1 -- 172.21.15.12:0/3825226728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd18105400 msgr2=0x7fbd181077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.429+0000 7fbd1dac1640 1 --2- 172.21.15.12:0/3825226728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd18105400 0x7fbd181077f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c009900 tx=0x7fbd0c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:23:33.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.429+0000 7fbd1dac1640 1 -- 172.21.15.12:0/3825226728 shutdown_connections 2024-04-22T13:23:33.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.429+0000 7fbd1dac1640 1 --2- 172.21.15.12:0/3825226728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd18107d30 0x7fbd1810a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.429+0000 7fbd1dac1640 1 --2- 172.21.15.12:0/3825226728 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd18105400 0x7fbd181077f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.429+0000 7fbd1dac1640 1 --2- 172.21.15.12:0/3825226728 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd18102ad0 0x7fbd18104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.429+0000 7fbd1dac1640 1 -- 172.21.15.12:0/3825226728 >> 172.21.15.12:0/3825226728 conn(0x7fbd180fc400 msgr2=0x7fbd180fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:33.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.430+0000 7fbd1dac1640 1 -- 172.21.15.12:0/3825226728 shutdown_connections 2024-04-22T13:23:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.430+0000 7fbd1dac1640 1 -- 172.21.15.12:0/3825226728 wait complete. 2024-04-22T13:23:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.430+0000 7fbd1dac1640 1 Processor -- start 2024-04-22T13:23:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.430+0000 7fbd1dac1640 1 -- start start 2024-04-22T13:23:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.430+0000 7fbd1dac1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd18102ad0 0x7fbd18102440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.430+0000 7fbd1dac1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd18105400 0x7fbd18100a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.430+0000 7fbd1dac1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd18107d30 0x7fbd18100fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.430+0000 7fbd1dac1640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1810db30 con 0x7fbd18102ad0 2024-04-22T13:23:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.430+0000 7fbd1dac1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1810d9b0 con 0x7fbd18107d30 2024-04-22T13:23:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.430+0000 7fbd1dac1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1810dcb0 con 0x7fbd18105400 2024-04-22T13:23:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.431+0000 7fbd177fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd18102ad0 0x7fbd18102440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.431+0000 7fbd177fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd18102ad0 0x7fbd18102440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35000/0 (socket says 172.21.15.12:35000) 2024-04-22T13:23:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.431+0000 7fbd177fe640 1 -- 172.21.15.12:0/4225891740 learned_addr learned my addr 172.21.15.12:0/4225891740 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.431+0000 7fbd16ffd640 1 --2- 172.21.15.12:0/4225891740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd18105400 0x7fbd18100a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.431+0000 7fbd17fff640 1 --2- 172.21.15.12:0/4225891740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd18107d30 0x7fbd18100fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.431+0000 7fbd177fe640 1 -- 172.21.15.12:0/4225891740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd18105400 msgr2=0x7fbd18100a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.431+0000 7fbd177fe640 1 --2- 172.21.15.12:0/4225891740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd18105400 0x7fbd18100a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.431+0000 7fbd177fe640 1 -- 172.21.15.12:0/4225891740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd18107d30 msgr2=0x7fbd18100fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.431+0000 7fbd177fe640 1 --2- 172.21.15.12:0/4225891740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd18107d30 0x7fbd18100fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.431+0000 7fbd177fe640 1 -- 172.21.15.12:0/4225891740 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd18101850 con 0x7fbd18102ad0 2024-04-22T13:23:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.431+0000 7fbd177fe640 1 --2- 172.21.15.12:0/4225891740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd18102ad0 0x7fbd18102440 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbd0800d770 tx=0x7fbd0800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.431+0000 7fbd14ff9640 1 -- 172.21.15.12:0/4225891740 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd08017070 con 0x7fbd18102ad0 2024-04-22T13:23:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.431+0000 7fbd14ff9640 1 -- 172.21.15.12:0/4225891740 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd080054d0 con 0x7fbd18102ad0 2024-04-22T13:23:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.431+0000 7fbd1dac1640 1 -- 172.21.15.12:0/4225891740 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd181a2330 con 0x7fbd18102ad0 2024-04-22T13:23:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.431+0000 7fbd14ff9640 1 -- 172.21.15.12:0/4225891740 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd08004c30 con 0x7fbd18102ad0 2024-04-22T13:23:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.431+0000 7fbd1dac1640 1 -- 172.21.15.12:0/4225891740 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd181a2800 con 0x7fbd18102ad0 2024-04-22T13:23:33.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.434+0000 7fbd2078d740 1 Processor -- start 2024-04-22T13:23:33.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.434+0000 7fbd14ff9640 1 -- 172.21.15.12:0/4225891740 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbd08004e50 con 0x7fbd18102ad0 2024-04-22T13:23:33.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.434+0000 7fbd2078d740 1 -- start start 2024-04-22T13:23:33.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.434+0000 7fbd2078d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557d719d3e40 0x557d719d4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.434+0000 7fbd2078d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d719d4830 0x557d719d4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.434+0000 7fbd2078d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d719d52d0 0x557d719dbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.434+0000 7fbd2078d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557d71938b10 con 0x557d719d52d0 2024-04-22T13:23:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.434+0000 7fbd2078d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557d718d6240 con 0x557d719d3e40 2024-04-22T13:23:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.434+0000 7fbd2078d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557d71934ad0 con 0x557d719d4830 2024-04-22T13:23:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.434+0000 7fbd16ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d719d52d0 0x557d719dbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.434+0000 7fbd16ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d719d52d0 0x557d719dbeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35016/0 (socket says 172.21.15.12:35016) 2024-04-22T13:23:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.434+0000 7fbd16ffd640 1 -- 172.21.15.12:0/2262147368 learned_addr learned my addr 172.21.15.12:0/2262147368 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.434+0000 7fbd177fe640 1 --2- 172.21.15.12:0/2262147368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d719d4830 0x557d719d4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.434+0000 7fbd17fff640 1 --2- 172.21.15.12:0/2262147368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557d719d3e40 0x557d719d4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.435+0000 7fbd16ffd640 1 -- 172.21.15.12:0/2262147368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d719d4830 msgr2=0x557d719d4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.435+0000 7fbd16ffd640 1 --2- 172.21.15.12:0/2262147368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d719d4830 0x557d719d4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.435+0000 7fbd16ffd640 1 -- 172.21.15.12:0/2262147368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557d719d3e40 msgr2=0x557d719d4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.435+0000 7fbd16ffd640 1 --2- 172.21.15.12:0/2262147368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557d719d3e40 0x557d719d4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.435+0000 7fbd16ffd640 1 -- 172.21.15.12:0/2262147368 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d7164b2d0 con 0x557d719d52d0 2024-04-22T13:23:33.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.435+0000 7fbd14ff9640 1 --2- 172.21.15.12:0/4225891740 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fbd000a3c40 0x7fbd000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.435+0000 7fbd16ffd640 1 --2- 172.21.15.12:0/2262147368 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d719d52d0 0x557d719dbeb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c000c00 tx=0x7fbd0c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc0fc5e6ef0c4d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:33.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.435+0000 7fbd17fff640 1 --2- 172.21.15.12:0/4225891740 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fbd000a3c40 0x7fbd000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.435+0000 7fbd14ff9640 1 -- 172.21.15.12:0/4225891740 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fbd080cff90 con 0x7fbd18102ad0 2024-04-22T13:23:33.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.435+0000 7fbcfdffb640 1 -- 172.21.15.12:0/2262147368 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c045020 con 0x557d719d52d0 2024-04-22T13:23:33.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.435+0000 7fbcfdffb640 1 -- 172.21.15.12:0/2262147368 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd0c02cb20 con 0x557d719d52d0 2024-04-22T13:23:33.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.435+0000 7fbcfdffb640 1 -- 172.21.15.12:0/2262147368 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c004920 con 0x557d719d52d0 2024-04-22T13:23:33.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.436+0000 7fbd2078d740 1 -- 172.21.15.12:0/2262147368 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d719d52d0 msgr2=0x557d719dbeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.436+0000 7fbd17fff640 1 --2- 172.21.15.12:0/4225891740 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fbd000a3c40 0x7fbd000a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x557d719cf200 tx=0x7fbd10007520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:33.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.436+0000 7fbd2078d740 1 --2- 172.21.15.12:0/2262147368 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d719d52d0 0x557d719dbeb0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c000c00 tx=0x7fbd0c005e80 comp rx=0 tx=0).stop 2024-04-22T13:23:33.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.436+0000 7fbd2078d740 1 -- 172.21.15.12:0/2262147368 shutdown_connections 2024-04-22T13:23:33.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.436+0000 7fbd2078d740 1 --2- 172.21.15.12:0/2262147368 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d719d52d0 0x557d719dbeb0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.436+0000 7fbd2078d740 1 --2- 172.21.15.12:0/2262147368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d719d4830 0x557d719d4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.436+0000 7fbd2078d740 1 --2- 172.21.15.12:0/2262147368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557d719d3e40 0x557d719d4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.436+0000 7fbd2078d740 1 -- 172.21.15.12:0/2262147368 >> 172.21.15.12:0/2262147368 conn(0x557d719cff40 msgr2=0x557d719ce490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:33.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.436+0000 7fbd2078d740 1 -- 172.21.15.12:0/2262147368 shutdown_connections 2024-04-22T13:23:33.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.436+0000 7fbd2078d740 1 -- 172.21.15.12:0/2262147368 wait complete. 2024-04-22T13:23:33.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.436+0000 7fbd2078d740 1 -- start start 2024-04-22T13:23:33.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.436+0000 7fbd2078d740 1 Processor -- start 2024-04-22T13:23:33.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.437+0000 7fbd2078d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557d719d3e40 0x557d719dde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.437+0000 7fbd2078d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d719d4830 0x557d719de3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.437+0000 7fbd2078d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d719d52d0 0x557d719e5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.437+0000 7fbd2078d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557d719dc450 con 0x557d719d4830 2024-04-22T13:23:33.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.437+0000 7fbd2078d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557d718c1190 con 0x557d719d3e40 2024-04-22T13:23:33.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.437+0000 7fbd2078d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557d719dc5d0 con 0x557d719d52d0 2024-04-22T13:23:33.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.437+0000 7fbd16ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d719d4830 0x557d719de3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.437+0000 7fbd2078d740 10 client.0 _ll_register_callbacks cb 0x557d718bc8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:23:33.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.437+0000 7fbd16ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d719d4830 0x557d719de3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35028/0 (socket says 172.21.15.12:35028) 2024-04-22T13:23:33.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.437+0000 7fbd16ffd640 1 -- 172.21.15.12:0/1252993218 learned_addr learned my addr 172.21.15.12:0/1252993218 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:33.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.437+0000 7fbd177fe640 1 --2- 172.21.15.12:0/1252993218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557d719d3e40 0x557d719dde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.437+0000 7fbd17fff640 1 --2- 172.21.15.12:0/1252993218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d719d52d0 0x557d719e5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.437+0000 7fbd16ffd640 1 -- 172.21.15.12:0/1252993218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d719d52d0 msgr2=0x557d719e5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.437+0000 7fbd16ffd640 1 --2- 172.21.15.12:0/1252993218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d719d52d0 0x557d719e5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.437+0000 7fbd16ffd640 1 -- 172.21.15.12:0/1252993218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557d719d3e40 msgr2=0x557d719dde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.437+0000 7fbd16ffd640 1 --2- 172.21.15.12:0/1252993218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557d719d3e40 0x557d719dde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.437+0000 7fbd16ffd640 1 -- 172.21.15.12:0/1252993218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d717156b0 con 0x557d719d4830 2024-04-22T13:23:33.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.437+0000 7fbd16ffd640 1 --2- 172.21.15.12:0/1252993218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d719d4830 0x557d719de3c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c02c240 tx=0x7fbd0c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23051b6f3c84624e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:33.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.437+0000 7fbcf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:23:33.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.437+0000 7fbcf6ffd640 1 -- 172.21.15.12:0/1252993218 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c045070 con 0x557d719d4830 2024-04-22T13:23:33.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.437+0000 7fbcf6ffd640 1 -- 172.21.15.12:0/1252993218 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fbd0c003e20 con 0x557d719d4830 2024-04-22T13:23:33.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.437+0000 7fbd2078d740 1 -- 172.21.15.12:0/1252993218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd0c02cb20 con 0x557d719d4830 2024-04-22T13:23:33.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.438+0000 7fbcf6ffd640 1 -- 172.21.15.12:0/1252993218 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbd0c005620 con 0x557d719d4830 2024-04-22T13:23:33.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.438+0000 7fbd2078d740 10 client.15111 fetch_fsmap learned FSMap version 27 2024-04-22T13:23:33.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.438+0000 7fbd2078d740 1 -- 172.21.15.12:0/1252993218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557d719e8770 con 0x557d719d4830 2024-04-22T13:23:33.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.438+0000 7fbcf6ffd640 1 -- 172.21.15.12:0/1252993218 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd0c02cb20 con 0x557d719d4830 2024-04-22T13:23:33.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.438+0000 7fbcf6ffd640 1 -- 172.21.15.12:0/1252993218 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbd0c04c460 con 0x557d719d4830 2024-04-22T13:23:33.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.438+0000 7fbd2078d740 10 client.15111 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:23:33.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.438+0000 7fbd2078d740 10 client.15111 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-22T13:23:33.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.438+0000 7fbd2078d740 20 client.15111 populate_metadata read hostname 'smithi012' 2024-04-22T13:23:33.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.438+0000 7fbd2078d740 1 --2- 172.21.15.12:0/1252993218 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x557d719ea9a0 0x557d719ebe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.438+0000 7fbd2078d740 4 client.15111 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:23:33.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.438+0000 7fbd2078d740 1 -- 172.21.15.12:0/1252993218 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d71946530 con 0x557d719ea9a0 2024-04-22T13:23:33.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.438+0000 7fbd177fe640 1 --2- 172.21.15.12:0/1252993218 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x557d719ea9a0 0x557d719ebe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.439+0000 7fbd177fe640 1 --2- 172.21.15.12:0/1252993218 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x557d719ea9a0 0x557d719ebe10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f1422fe11ba5c3a5 server_cookie=97c899a4069dbca4 in_seq=0 out_seq=0 2024-04-22T13:23:33.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.439+0000 7fbcf6ffd640 10 client.15111 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:23:33.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.440+0000 7fbcf6ffd640 1 -- 172.21.15.12:0/1252993218 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557d71946530 con 0x557d719ea9a0 2024-04-22T13:23:33.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.440+0000 7fbcf6ffd640 10 client.15111 handle_command_reply: tid=0 2024-04-22T13:23:33.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.440+0000 7fbd2078d740 1 client.15111 shutdown 2024-04-22T13:23:33.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.440+0000 7fbd2078d740 1 -- 172.21.15.12:0/1252993218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d719d4830 msgr2=0x557d719de3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.440+0000 7fbd2078d740 1 --2- 172.21.15.12:0/1252993218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d719d4830 0x557d719de3c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c02c240 tx=0x7fbd0c003820 comp rx=0 tx=0).stop 2024-04-22T13:23:33.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.440+0000 7fbd2078d740 1 -- 172.21.15.12:0/1252993218 shutdown_connections 2024-04-22T13:23:33.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.440+0000 7fbd2078d740 1 --2- 172.21.15.12:0/1252993218 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x557d719ea9a0 0x557d719ebe10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.440+0000 7fbd2078d740 1 --2- 172.21.15.12:0/1252993218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d719d52d0 0x557d719e5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.440+0000 7fbd2078d740 1 --2- 172.21.15.12:0/1252993218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d719d4830 0x557d719de3c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.440+0000 7fbd2078d740 1 --2- 172.21.15.12:0/1252993218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557d719d3e40 0x557d719dde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.440+0000 7fbcf6ffd640 0 client.15111 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:23:33.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.440+0000 7fbd2078d740 1 -- 172.21.15.12:0/1252993218 >> 172.21.15.12:0/1252993218 conn(0x557d719cff40 msgr2=0x557d719cda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:33.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.440+0000 7fbd2078d740 1 -- 172.21.15.12:0/1252993218 shutdown_connections 2024-04-22T13:23:33.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.440+0000 7fbd2078d740 1 -- 172.21.15.12:0/1252993218 wait complete. 2024-04-22T13:23:33.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.440+0000 7fbd2078d740 20 client.15111 trim_cache size 0 max 16384 2024-04-22T13:23:33.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.458+0000 7fbd2078d740 1 Processor -- start 2024-04-22T13:23:33.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.458+0000 7fbd2078d740 1 -- start start 2024-04-22T13:23:33.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.458+0000 7fbd2078d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d719d3e40 0x557d719d7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.458+0000 7fbd2078d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d719d4830 0x557d719e53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.458+0000 7fbd2078d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557d719d52d0 0x557d719d8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.458+0000 7fbd2078d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557d719ea150 con 0x557d719d3e40 2024-04-22T13:23:33.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.458+0000 7fbd2078d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557d719ea820 con 0x557d719d52d0 2024-04-22T13:23:33.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.458+0000 7fbd2078d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557d719eaef0 con 0x557d719d4830 2024-04-22T13:23:33.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.458+0000 7fbd177fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d719d3e40 0x557d719d7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.458+0000 7fbd177fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d719d3e40 0x557d719d7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35036/0 (socket says 172.21.15.12:35036) 2024-04-22T13:23:33.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.458+0000 7fbd177fe640 1 -- 172.21.15.12:0/2994952876 learned_addr learned my addr 172.21.15.12:0/2994952876 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:33.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.459+0000 7fbd177fe640 1 -- 172.21.15.12:0/2994952876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d719d4830 msgr2=0x557d719e53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.459+0000 7fbd177fe640 1 --2- 172.21.15.12:0/2994952876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d719d4830 0x557d719e53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.459+0000 7fbd177fe640 1 -- 172.21.15.12:0/2994952876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557d719d52d0 msgr2=0x557d719d8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.459+0000 7fbd177fe640 1 --2- 172.21.15.12:0/2994952876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557d719d52d0 0x557d719d8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.459+0000 7fbd177fe640 1 -- 172.21.15.12:0/2994952876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d719cf630 con 0x557d719d3e40 2024-04-22T13:23:33.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.459+0000 7fbd177fe640 1 --2- 172.21.15.12:0/2994952876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d719d3e40 0x557d719d7ca0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbd080cd9a0 tx=0x7fbd080d2480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df534c9b2e4b2d04 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:33.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.459+0000 7fbcfd7fa640 1 -- 172.21.15.12:0/2994952876 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd08020040 con 0x557d719d3e40 2024-04-22T13:23:33.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.459+0000 7fbcfd7fa640 1 -- 172.21.15.12:0/2994952876 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd08005680 con 0x557d719d3e40 2024-04-22T13:23:33.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.459+0000 7fbcfd7fa640 1 -- 172.21.15.12:0/2994952876 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0809a4f0 con 0x557d719d3e40 2024-04-22T13:23:33.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.459+0000 7fbd2078d740 1 -- 172.21.15.12:0/2994952876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d719d3e40 msgr2=0x557d719d7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.459+0000 7fbd2078d740 1 --2- 172.21.15.12:0/2994952876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d719d3e40 0x557d719d7ca0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbd080cd9a0 tx=0x7fbd080d2480 comp rx=0 tx=0).stop 2024-04-22T13:23:33.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.460+0000 7fbd2078d740 1 -- 172.21.15.12:0/2994952876 shutdown_connections 2024-04-22T13:23:33.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.460+0000 7fbd2078d740 1 --2- 172.21.15.12:0/2994952876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557d719d52d0 0x557d719d8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.460+0000 7fbd2078d740 1 --2- 172.21.15.12:0/2994952876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d719d4830 0x557d719e53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.460+0000 7fbd2078d740 1 --2- 172.21.15.12:0/2994952876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d719d3e40 0x557d719d7ca0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.460+0000 7fbd2078d740 1 -- 172.21.15.12:0/2994952876 >> 172.21.15.12:0/2994952876 conn(0x557d719cff40 msgr2=0x557d719c9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:33.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.460+0000 7fbd2078d740 1 -- 172.21.15.12:0/2994952876 shutdown_connections 2024-04-22T13:23:33.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.460+0000 7fbd2078d740 1 -- 172.21.15.12:0/2994952876 wait complete. 2024-04-22T13:23:33.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.460+0000 7fbd2078d740 1 -- start start 2024-04-22T13:23:33.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.460+0000 7fbd2078d740 1 Processor -- start 2024-04-22T13:23:33.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.460+0000 7fbd2078d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d719d3e40 0x557d719e13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.460+0000 7fbd2078d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557d719d4830 0x557d719e38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.460+0000 7fbd2078d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d719d52d0 0x557d719e3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.460+0000 7fbd2078d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557d719ebc90 con 0x557d719d52d0 2024-04-22T13:23:33.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.460+0000 7fbd2078d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557d719eb5c0 con 0x557d719d4830 2024-04-22T13:23:33.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.460+0000 7fbd2078d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557d719ec360 con 0x557d719d3e40 2024-04-22T13:23:33.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.460+0000 7fbd2078d740 10 client.0 _ll_register_callbacks cb 0x7fbd10007940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:23:33.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.460+0000 7fbd17fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d719d52d0 0x557d719e3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.460+0000 7fbd17fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d719d52d0 0x557d719e3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35042/0 (socket says 172.21.15.12:35042) 2024-04-22T13:23:33.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.460+0000 7fbd17fff640 1 -- 172.21.15.12:0/141114147 learned_addr learned my addr 172.21.15.12:0/141114147 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:33.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.461+0000 7fbd177fe640 1 --2- 172.21.15.12:0/141114147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d719d3e40 0x557d719e13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.461+0000 7fbd16ffd640 1 --2- 172.21.15.12:0/141114147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557d719d4830 0x557d719e38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.461+0000 7fbd17fff640 1 -- 172.21.15.12:0/141114147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d719d3e40 msgr2=0x557d719e13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.461+0000 7fbd17fff640 1 --2- 172.21.15.12:0/141114147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d719d3e40 0x557d719e13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.461+0000 7fbd17fff640 1 -- 172.21.15.12:0/141114147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557d719d4830 msgr2=0x557d719e38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.461+0000 7fbd17fff640 1 --2- 172.21.15.12:0/141114147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557d719d4830 0x557d719e38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.461+0000 7fbd17fff640 1 -- 172.21.15.12:0/141114147 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d719e4750 con 0x557d719d52d0 2024-04-22T13:23:33.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.461+0000 7fbd16ffd640 1 --2- 172.21.15.12:0/141114147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557d719d4830 0x557d719e38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:23:33.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.461+0000 7fbd17fff640 1 --2- 172.21.15.12:0/141114147 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d719d52d0 0x557d719e3f20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbd1001ece0 tx=0x7fbd1001c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a89264aba3e8416 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:33.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.461+0000 7fbcf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:23:33.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.461+0000 7fbcf6ffd640 1 -- 172.21.15.12:0/141114147 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1001e260 con 0x557d719d52d0 2024-04-22T13:23:33.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.461+0000 7fbcf6ffd640 1 -- 172.21.15.12:0/141114147 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fbd10016030 con 0x557d719d52d0 2024-04-22T13:23:33.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.461+0000 7fbd2078d740 1 -- 172.21.15.12:0/141114147 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd08005680 con 0x557d719d52d0 2024-04-22T13:23:33.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.461+0000 7fbcf6ffd640 1 -- 172.21.15.12:0/141114147 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbd10003970 con 0x557d719d52d0 2024-04-22T13:23:33.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.461+0000 7fbd2078d740 10 client.15117 fetch_fsmap learned FSMap version 27 2024-04-22T13:23:33.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.461+0000 7fbd2078d740 1 -- 172.21.15.12:0/141114147 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557d71a50a80 con 0x557d719d52d0 2024-04-22T13:23:33.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.461+0000 7fbcf6ffd640 1 -- 172.21.15.12:0/141114147 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd08005680 con 0x557d719d52d0 2024-04-22T13:23:33.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.462+0000 7fbcf6ffd640 1 -- 172.21.15.12:0/141114147 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fbd1001d5e0 con 0x557d719d52d0 2024-04-22T13:23:33.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.462+0000 7fbd2078d740 10 client.15117 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:23:33.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.462+0000 7fbd2078d740 10 client.15117 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-22T13:23:33.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.462+0000 7fbd2078d740 20 client.15117 populate_metadata read hostname 'smithi012' 2024-04-22T13:23:33.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.462+0000 7fbd2078d740 1 --2- 172.21.15.12:0/141114147 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x557d71a55650 0x557d71a54360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.462+0000 7fbd2078d740 4 client.15117 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T13:23:33.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.462+0000 7fbd2078d740 1 -- 172.21.15.12:0/141114147 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "status"}) -- 0x557d719403a0 con 0x557d71a55650 2024-04-22T13:23:33.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.462+0000 7fbd177fe640 1 --2- 172.21.15.12:0/141114147 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x557d71a55650 0x557d71a54360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.462+0000 7fbd177fe640 1 --2- 172.21.15.12:0/141114147 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x557d71a55650 0x557d71a54360 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=64923a2d660b8769 server_cookie=49b389fa2d822276 in_seq=0 out_seq=0 2024-04-22T13:23:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.462+0000 7fbcf6ffd640 10 client.15117 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:23:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.462+0000 7fbcf6ffd640 1 -- 172.21.15.12:0/141114147 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x557d719403a0 con 0x557d71a55650 2024-04-22T13:23:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.462+0000 7fbcf6ffd640 10 client.15117 handle_command_reply: tid=0 2024-04-22T13:23:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.462+0000 7fbd2078d740 1 client.15117 shutdown 2024-04-22T13:23:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.463+0000 7fbd2078d740 1 -- 172.21.15.12:0/141114147 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d719d52d0 msgr2=0x557d719e3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.463+0000 7fbd2078d740 1 --2- 172.21.15.12:0/141114147 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d719d52d0 0x557d719e3f20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbd1001ece0 tx=0x7fbd1001c7e0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.463+0000 7fbd2078d740 1 -- 172.21.15.12:0/141114147 shutdown_connections 2024-04-22T13:23:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.463+0000 7fbd2078d740 1 --2- 172.21.15.12:0/141114147 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x557d71a55650 0x557d71a54360 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.463+0000 7fbd2078d740 1 --2- 172.21.15.12:0/141114147 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557d719d52d0 0x557d719e3f20 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.463+0000 7fbd2078d740 1 --2- 172.21.15.12:0/141114147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557d719d4830 0x557d719e38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.463+0000 7fbcf6ffd640 0 client.15117 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:23:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.463+0000 7fbd2078d740 1 --2- 172.21.15.12:0/141114147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d719d3e40 0x557d719e13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.463+0000 7fbd2078d740 1 -- 172.21.15.12:0/141114147 >> 172.21.15.12:0/141114147 conn(0x557d719cff40 msgr2=0x557d719d8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.463+0000 7fbd2078d740 1 -- 172.21.15.12:0/141114147 shutdown_connections 2024-04-22T13:23:33.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.463+0000 7fbd2078d740 1 -- 172.21.15.12:0/141114147 wait complete. 2024-04-22T13:23:33.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.463+0000 7fbd2078d740 20 client.15117 trim_cache size 0 max 16384 2024-04-22T13:23:33.466 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:23:33.466 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "0aede5ea-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:23:33.466 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 1, 2024-04-22T13:23:33.466 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24484, 2024-04-22T13:23:33.466 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-04-22T13:23:33.466 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-04-22T13:23:33.466 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-04-22T13:23:33.466 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 85.074300777000005, 2024-04-22T13:23:33.466 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 27, 2024-04-22T13:23:33.467 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 89, 2024-04-22T13:23:33.467 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 87, 2024-04-22T13:23:33.467 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 102.446896346 2024-04-22T13:23:33.467 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:23:33.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.463+0000 7fbd1dac1640 1 -- 172.21.15.12:0/4225891740 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fbd000a3c40 msgr2=0x7fbd000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.463+0000 7fbd1dac1640 1 --2- 172.21.15.12:0/4225891740 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fbd000a3c40 0x7fbd000a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x557d719cf200 tx=0x7fbd10007520 comp rx=0 tx=0).stop 2024-04-22T13:23:33.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.463+0000 7fbd1dac1640 1 -- 172.21.15.12:0/4225891740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd18102ad0 msgr2=0x7fbd18102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.463+0000 7fbd1dac1640 1 --2- 172.21.15.12:0/4225891740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd18102ad0 0x7fbd18102440 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbd0800d770 tx=0x7fbd0800dc30 comp rx=0 tx=0).stop 2024-04-22T13:23:33.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.463+0000 7fbd1dac1640 1 -- 172.21.15.12:0/4225891740 shutdown_connections 2024-04-22T13:23:33.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.463+0000 7fbd1dac1640 1 --2- 172.21.15.12:0/4225891740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd18107d30 0x7fbd18100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.463+0000 7fbd1dac1640 1 --2- 172.21.15.12:0/4225891740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd18105400 0x7fbd18100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.463+0000 7fbd1dac1640 1 --2- 172.21.15.12:0/4225891740 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fbd000a3c40 0x7fbd000a6100 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.463+0000 7fbd1dac1640 1 --2- 172.21.15.12:0/4225891740 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd18102ad0 0x7fbd18102440 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.463+0000 7fbd1dac1640 1 -- 172.21.15.12:0/4225891740 >> 172.21.15.12:0/4225891740 conn(0x7fbd180fc400 msgr2=0x7fbd18103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:33.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.463+0000 7fbd1dac1640 1 -- 172.21.15.12:0/4225891740 shutdown_connections 2024-04-22T13:23:33.468 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.463+0000 7fbd1dac1640 1 -- 172.21.15.12:0/4225891740 wait complete. 2024-04-22T13:23:33.471 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-22T13:23:33.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.576+0000 7f278a074640 1 -- 172.21.15.12:0/1388148055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2784100800 msgr2=0x7f2784100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.576+0000 7f278a074640 1 --2- 172.21.15.12:0/1388148055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2784100800 0x7f2784100c20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2774009ca0 tx=0x7f277402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.577+0000 7f278a074640 1 -- 172.21.15.12:0/1388148055 shutdown_connections 2024-04-22T13:23:33.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.577+0000 7f278a074640 1 --2- 172.21.15.12:0/1388148055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2784101b00 0x7f2784101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.577+0000 7f278a074640 1 --2- 172.21.15.12:0/1388148055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2784101160 0x7f27841015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.577+0000 7f278a074640 1 --2- 172.21.15.12:0/1388148055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2784100800 0x7f2784100c20 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.577+0000 7f278a074640 1 -- 172.21.15.12:0/1388148055 >> 172.21.15.12:0/1388148055 conn(0x7f27840fc400 msgr2=0x7f27840fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:33.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.577+0000 7f278a074640 1 -- 172.21.15.12:0/1388148055 shutdown_connections 2024-04-22T13:23:33.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.577+0000 7f278a074640 1 -- 172.21.15.12:0/1388148055 wait complete. 2024-04-22T13:23:33.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.577+0000 7f278a074640 1 Processor -- start 2024-04-22T13:23:33.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.577+0000 7f278a074640 1 -- start start 2024-04-22T13:23:33.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.578+0000 7f278a074640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2784100800 0x7f278419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.578+0000 7f278a074640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2784101160 0x7f278419a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.578+0000 7f278a074640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2784101b00 0x7f27841a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.578+0000 7f278a074640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f278410fce0 con 0x7f2784100800 2024-04-22T13:23:33.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.578+0000 7f278a074640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f278410fb60 con 0x7f2784101b00 2024-04-22T13:23:33.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.578+0000 7f278a074640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f278410fe60 con 0x7f2784101160 2024-04-22T13:23:33.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.578+0000 7f27837fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2784100800 0x7f278419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.578+0000 7f27837fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2784100800 0x7f278419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35058/0 (socket says 172.21.15.12:35058) 2024-04-22T13:23:33.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.578+0000 7f27837fe640 1 -- 172.21.15.12:0/2235584516 learned_addr learned my addr 172.21.15.12:0/2235584516 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:33.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.578+0000 7f2782ffd640 1 --2- 172.21.15.12:0/2235584516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2784101160 0x7f278419a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.578+0000 7f2783fff640 1 --2- 172.21.15.12:0/2235584516 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2784101b00 0x7f27841a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.578+0000 7f27837fe640 1 -- 172.21.15.12:0/2235584516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2784101160 msgr2=0x7f278419a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.578+0000 7f27837fe640 1 --2- 172.21.15.12:0/2235584516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2784101160 0x7f278419a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.578+0000 7f27837fe640 1 -- 172.21.15.12:0/2235584516 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2784101b00 msgr2=0x7f27841a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.578+0000 7f27837fe640 1 --2- 172.21.15.12:0/2235584516 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2784101b00 0x7f27841a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.578+0000 7f27837fe640 1 -- 172.21.15.12:0/2235584516 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27841a2010 con 0x7f2784100800 2024-04-22T13:23:33.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.578+0000 7f27837fe640 1 --2- 172.21.15.12:0/2235584516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2784100800 0x7f278419a440 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2774005e30 tx=0x7f27740346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:33.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.578+0000 7f2780ff9640 1 -- 172.21.15.12:0/2235584516 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2774041020 con 0x7f2784100800 2024-04-22T13:23:33.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.578+0000 7f2780ff9640 1 -- 172.21.15.12:0/2235584516 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2774034ca0 con 0x7f2784100800 2024-04-22T13:23:33.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.578+0000 7f278a074640 1 -- 172.21.15.12:0/2235584516 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27841a22a0 con 0x7f2784100800 2024-04-22T13:23:33.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.578+0000 7f2780ff9640 1 -- 172.21.15.12:0/2235584516 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2774039040 con 0x7f2784100800 2024-04-22T13:23:33.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.578+0000 7f278a074640 1 -- 172.21.15.12:0/2235584516 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27841a2760 con 0x7f2784100800 2024-04-22T13:23:33.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.581+0000 7f2780ff9640 1 -- 172.21.15.12:0/2235584516 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2774044020 con 0x7f2784100800 2024-04-22T13:23:33.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.581+0000 7f278cc6e740 1 Processor -- start 2024-04-22T13:23:33.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.581+0000 7f278cc6e740 1 -- start start 2024-04-22T13:23:33.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.581+0000 7f278cc6e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8587c3e40 0x55b8587c4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.581+0000 7f278cc6e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b8587c4830 0x55b8587c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.581+0000 7f278cc6e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8587c52d0 0x55b8587cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.581+0000 7f278cc6e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b858728b10 con 0x55b8587c3e40 2024-04-22T13:23:33.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.581+0000 7f278cc6e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b8586c6240 con 0x55b8587c4830 2024-04-22T13:23:33.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.581+0000 7f278cc6e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b858724ad0 con 0x55b8587c52d0 2024-04-22T13:23:33.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.581+0000 7f2783fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8587c3e40 0x55b8587c4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.581+0000 7f2783fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8587c3e40 0x55b8587c4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35066/0 (socket says 172.21.15.12:35066) 2024-04-22T13:23:33.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.581+0000 7f2783fff640 1 -- 172.21.15.12:0/3194864522 learned_addr learned my addr 172.21.15.12:0/3194864522 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:33.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.582+0000 7f2780ff9640 1 --2- 172.21.15.12:0/2235584516 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f27700a3c40 0x7f27700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.582+0000 7f2783fff640 1 --2- 172.21.15.12:0/2235584516 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f27700a3c40 0x7f27700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.582+0000 7f2780ff9640 1 -- 172.21.15.12:0/2235584516 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f27740f2360 con 0x7f2784100800 2024-04-22T13:23:33.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.582+0000 7f2783fff640 1 -- 172.21.15.12:0/3194864522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8587c52d0 msgr2=0x55b8587cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.582+0000 7f2783fff640 1 --2- 172.21.15.12:0/3194864522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8587c52d0 0x55b8587cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.582+0000 7f27837fe640 1 --2- 172.21.15.12:0/3194864522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b8587c4830 0x55b8587c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.582+0000 7f2783fff640 1 -- 172.21.15.12:0/3194864522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b8587c4830 msgr2=0x55b8587c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.582+0000 7f2783fff640 1 --2- 172.21.15.12:0/3194864522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b8587c4830 0x55b8587c4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.582+0000 7f2783fff640 1 -- 172.21.15.12:0/3194864522 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b85843b2d0 con 0x55b8587c3e40 2024-04-22T13:23:33.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.582+0000 7f2783fff640 1 --2- 172.21.15.12:0/3194864522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8587c3e40 0x55b8587c4260 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f277c013850 tx=0x7f277c013d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4075b326df4c35 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:33.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.582+0000 7f2779ffb640 1 -- 172.21.15.12:0/3194864522 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c022020 con 0x55b8587c3e40 2024-04-22T13:23:33.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.582+0000 7f2779ffb640 1 -- 172.21.15.12:0/3194864522 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f277c003aa0 con 0x55b8587c3e40 2024-04-22T13:23:33.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.582+0000 7f2779ffb640 1 -- 172.21.15.12:0/3194864522 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c01e680 con 0x55b8587c3e40 2024-04-22T13:23:33.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.583+0000 7f278cc6e740 1 -- 172.21.15.12:0/3194864522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8587c3e40 msgr2=0x55b8587c4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.583+0000 7f2783fff640 1 --2- 172.21.15.12:0/2235584516 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f27700a3c40 0x7f27700a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f277c023000 tx=0x7f277c021000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:33.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.583+0000 7f278cc6e740 1 --2- 172.21.15.12:0/3194864522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8587c3e40 0x55b8587c4260 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f277c013850 tx=0x7f277c013d10 comp rx=0 tx=0).stop 2024-04-22T13:23:33.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.583+0000 7f278cc6e740 1 -- 172.21.15.12:0/3194864522 shutdown_connections 2024-04-22T13:23:33.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.583+0000 7f278cc6e740 1 --2- 172.21.15.12:0/3194864522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8587c52d0 0x55b8587cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.583+0000 7f278cc6e740 1 --2- 172.21.15.12:0/3194864522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b8587c4830 0x55b8587c4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.583+0000 7f278cc6e740 1 --2- 172.21.15.12:0/3194864522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8587c3e40 0x55b8587c4260 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.583+0000 7f278cc6e740 1 -- 172.21.15.12:0/3194864522 >> 172.21.15.12:0/3194864522 conn(0x55b8587bff40 msgr2=0x55b8587be490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:33.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.583+0000 7f278cc6e740 1 -- 172.21.15.12:0/3194864522 shutdown_connections 2024-04-22T13:23:33.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.583+0000 7f278cc6e740 1 -- 172.21.15.12:0/3194864522 wait complete. 2024-04-22T13:23:33.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.583+0000 7f278cc6e740 1 -- start start 2024-04-22T13:23:33.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.583+0000 7f278cc6e740 1 Processor -- start 2024-04-22T13:23:33.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.584+0000 7f278cc6e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8587c3e40 0x55b8587cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.584+0000 7f278cc6e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b8587c4830 0x55b8587ce450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.584+0000 7f278cc6e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8587c52d0 0x55b8587d5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.584+0000 7f278cc6e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b8587cc4b0 con 0x55b8587c52d0 2024-04-22T13:23:33.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.584+0000 7f278cc6e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b8586b1190 con 0x55b8587c4830 2024-04-22T13:23:33.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.584+0000 7f278cc6e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b8587cc630 con 0x55b8587c3e40 2024-04-22T13:23:33.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.584+0000 7f278cc6e740 10 client.0 _ll_register_callbacks cb 0x55b8586ac8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:23:33.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.584+0000 7f2783fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8587c52d0 0x55b8587d5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.584+0000 7f2783fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8587c52d0 0x55b8587d5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35068/0 (socket says 172.21.15.12:35068) 2024-04-22T13:23:33.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.584+0000 7f2783fff640 1 -- 172.21.15.12:0/3796353242 learned_addr learned my addr 172.21.15.12:0/3796353242 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:33.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.584+0000 7f27837fe640 1 --2- 172.21.15.12:0/3796353242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8587c3e40 0x55b8587cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.584+0000 7f2782ffd640 1 --2- 172.21.15.12:0/3796353242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b8587c4830 0x55b8587ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.584+0000 7f2783fff640 1 -- 172.21.15.12:0/3796353242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8587c3e40 msgr2=0x55b8587cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.584+0000 7f2783fff640 1 --2- 172.21.15.12:0/3796353242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8587c3e40 0x55b8587cdf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.584+0000 7f2783fff640 1 -- 172.21.15.12:0/3796353242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b8587c4830 msgr2=0x55b8587ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.584+0000 7f2783fff640 1 --2- 172.21.15.12:0/3796353242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b8587c4830 0x55b8587ce450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.584+0000 7f2783fff640 1 -- 172.21.15.12:0/3796353242 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8585056b0 con 0x55b8587c52d0 2024-04-22T13:23:33.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.584+0000 7f2782ffd640 1 --2- 172.21.15.12:0/3796353242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b8587c4830 0x55b8587ce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:23:33.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.584+0000 7f2783fff640 1 --2- 172.21.15.12:0/3796353242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8587c52d0 0x55b8587d5490 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f277c004000 tx=0x7f277c00f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1a44caeece326c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:33.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.584+0000 7f2762ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:23:33.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.584+0000 7f2762ffd640 1 -- 172.21.15.12:0/3796353242 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c030060 con 0x55b8587c52d0 2024-04-22T13:23:33.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.584+0000 7f2762ffd640 1 -- 172.21.15.12:0/3796353242 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f277c01e620 con 0x55b8587c52d0 2024-04-22T13:23:33.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.584+0000 7f278cc6e740 1 -- 172.21.15.12:0/3796353242 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f277c003aa0 con 0x55b8587c52d0 2024-04-22T13:23:33.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.585+0000 7f2762ffd640 1 -- 172.21.15.12:0/3796353242 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f277c022070 con 0x55b8587c52d0 2024-04-22T13:23:33.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.585+0000 7f278cc6e740 10 client.15129 fetch_fsmap learned FSMap version 27 2024-04-22T13:23:33.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.585+0000 7f278cc6e740 1 -- 172.21.15.12:0/3796353242 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b8587d8870 con 0x55b8587c52d0 2024-04-22T13:23:33.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.585+0000 7f2762ffd640 1 -- 172.21.15.12:0/3796353242 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f277c003aa0 con 0x55b8587c52d0 2024-04-22T13:23:33.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.585+0000 7f2762ffd640 1 -- 172.21.15.12:0/3796353242 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f277c01ec80 con 0x55b8587c52d0 2024-04-22T13:23:33.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.585+0000 7f278cc6e740 10 client.15129 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:23:33.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.585+0000 7f278cc6e740 10 client.15129 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-22T13:23:33.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.585+0000 7f278cc6e740 20 client.15129 populate_metadata read hostname 'smithi012' 2024-04-22T13:23:33.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.585+0000 7f278cc6e740 1 --2- 172.21.15.12:0/3796353242 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b8587dacf0 0x55b8587dc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.585+0000 7f278cc6e740 4 client.15129 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:23:33.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.585+0000 7f278cc6e740 1 -- 172.21.15.12:0/3796353242 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b858736530 con 0x55b8587dacf0 2024-04-22T13:23:33.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.585+0000 7f27837fe640 1 --2- 172.21.15.12:0/3796353242 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b8587dacf0 0x55b8587dc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.586+0000 7f2762ffd640 10 client.15129 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:23:33.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.586+0000 7f27837fe640 1 --2- 172.21.15.12:0/3796353242 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b8587dacf0 0x55b8587dc160 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd9e4396ec59320d server_cookie=93953c1e86e95f3e in_seq=0 out_seq=0 2024-04-22T13:23:33.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.587+0000 7f2762ffd640 1 -- 172.21.15.12:0/3796353242 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b858736530 con 0x55b8587dacf0 2024-04-22T13:23:33.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.587+0000 7f2762ffd640 10 client.15129 handle_command_reply: tid=0 2024-04-22T13:23:33.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.587+0000 7f278cc6e740 1 client.15129 shutdown 2024-04-22T13:23:33.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.587+0000 7f278cc6e740 1 -- 172.21.15.12:0/3796353242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8587c52d0 msgr2=0x55b8587d5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.587+0000 7f278cc6e740 1 --2- 172.21.15.12:0/3796353242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8587c52d0 0x55b8587d5490 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f277c004000 tx=0x7f277c00f850 comp rx=0 tx=0).stop 2024-04-22T13:23:33.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.587+0000 7f278cc6e740 1 -- 172.21.15.12:0/3796353242 shutdown_connections 2024-04-22T13:23:33.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.587+0000 7f278cc6e740 1 --2- 172.21.15.12:0/3796353242 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b8587dacf0 0x55b8587dc160 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.587+0000 7f278cc6e740 1 --2- 172.21.15.12:0/3796353242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8587c52d0 0x55b8587d5490 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.587+0000 7f2762ffd640 0 client.15129 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:23:33.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.587+0000 7f278cc6e740 1 --2- 172.21.15.12:0/3796353242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b8587c4830 0x55b8587ce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.587+0000 7f278cc6e740 1 --2- 172.21.15.12:0/3796353242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8587c3e40 0x55b8587cdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.587+0000 7f278cc6e740 1 -- 172.21.15.12:0/3796353242 >> 172.21.15.12:0/3796353242 conn(0x55b8587bff40 msgr2=0x55b8587bda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:33.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.587+0000 7f278cc6e740 1 -- 172.21.15.12:0/3796353242 shutdown_connections 2024-04-22T13:23:33.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.587+0000 7f278cc6e740 1 -- 172.21.15.12:0/3796353242 wait complete. 2024-04-22T13:23:33.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.587+0000 7f278cc6e740 20 client.15129 trim_cache size 0 max 16384 2024-04-22T13:23:33.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.605+0000 7f278cc6e740 1 Processor -- start 2024-04-22T13:23:33.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.605+0000 7f278cc6e740 1 -- start start 2024-04-22T13:23:33.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.605+0000 7f278cc6e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8587c3e40 0x55b8587b9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.605+0000 7f278cc6e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8587c4830 0x55b8587d5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.605+0000 7f278cc6e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b8587c52d0 0x55b8587cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.605+0000 7f278cc6e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b8587da4a0 con 0x55b8587c4830 2024-04-22T13:23:33.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.605+0000 7f278cc6e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b8587dab70 con 0x55b8587c52d0 2024-04-22T13:23:33.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.605+0000 7f278cc6e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b8587db240 con 0x55b8587c3e40 2024-04-22T13:23:33.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.605+0000 7f2782ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8587c4830 0x55b8587d5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.605+0000 7f2782ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8587c4830 0x55b8587d5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35082/0 (socket says 172.21.15.12:35082) 2024-04-22T13:23:33.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.605+0000 7f2782ffd640 1 -- 172.21.15.12:0/2788146090 learned_addr learned my addr 172.21.15.12:0/2788146090 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:33.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.605+0000 7f2782ffd640 1 -- 172.21.15.12:0/2788146090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8587c3e40 msgr2=0x55b8587b9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.605+0000 7f2782ffd640 1 --2- 172.21.15.12:0/2788146090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8587c3e40 0x55b8587b9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.606+0000 7f2782ffd640 1 -- 172.21.15.12:0/2788146090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b8587c52d0 msgr2=0x55b8587cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.606+0000 7f2782ffd640 1 --2- 172.21.15.12:0/2788146090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b8587c52d0 0x55b8587cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.606+0000 7f2782ffd640 1 -- 172.21.15.12:0/2788146090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8587bec00 con 0x55b8587c4830 2024-04-22T13:23:33.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.606+0000 7f2782ffd640 1 --2- 172.21.15.12:0/2788146090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8587c4830 0x55b8587d5450 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f276c009bb0 tx=0x7f276c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cbbe923f69f2f88 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:33.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.606+0000 7f27797fa640 1 -- 172.21.15.12:0/2788146090 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c01b440 con 0x55b8587c4830 2024-04-22T13:23:33.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.606+0000 7f27797fa640 1 -- 172.21.15.12:0/2788146090 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f276c00f8c0 con 0x55b8587c4830 2024-04-22T13:23:33.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.606+0000 7f27797fa640 1 -- 172.21.15.12:0/2788146090 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c01a910 con 0x55b8587c4830 2024-04-22T13:23:33.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.606+0000 7f278cc6e740 1 -- 172.21.15.12:0/2788146090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8587c4830 msgr2=0x55b8587d5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.606+0000 7f278cc6e740 1 --2- 172.21.15.12:0/2788146090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8587c4830 0x55b8587d5450 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f276c009bb0 tx=0x7f276c012820 comp rx=0 tx=0).stop 2024-04-22T13:23:33.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.606+0000 7f278cc6e740 1 -- 172.21.15.12:0/2788146090 shutdown_connections 2024-04-22T13:23:33.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.606+0000 7f278cc6e740 1 --2- 172.21.15.12:0/2788146090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b8587c52d0 0x55b8587cdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.606+0000 7f278cc6e740 1 --2- 172.21.15.12:0/2788146090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8587c4830 0x55b8587d5450 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.606+0000 7f278cc6e740 1 --2- 172.21.15.12:0/2788146090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8587c3e40 0x55b8587b9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.606+0000 7f278cc6e740 1 -- 172.21.15.12:0/2788146090 >> 172.21.15.12:0/2788146090 conn(0x55b8587bff40 msgr2=0x55b8587d9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:33.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.606+0000 7f278cc6e740 1 -- 172.21.15.12:0/2788146090 shutdown_connections 2024-04-22T13:23:33.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.606+0000 7f278cc6e740 1 -- 172.21.15.12:0/2788146090 wait complete. 2024-04-22T13:23:33.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.607+0000 7f278cc6e740 1 -- start start 2024-04-22T13:23:33.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.607+0000 7f278cc6e740 1 Processor -- start 2024-04-22T13:23:33.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.607+0000 7f278cc6e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8587c3e40 0x55b8587d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.607+0000 7f278cc6e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8587c4830 0x55b8587d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.607+0000 7f278cc6e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b8587c52d0 0x55b8587d3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.607+0000 7f278cc6e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b8587dbfe0 con 0x55b8587c4830 2024-04-22T13:23:33.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.607+0000 7f278cc6e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b8587db910 con 0x55b8587c52d0 2024-04-22T13:23:33.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.607+0000 7f278cc6e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b8587dc6b0 con 0x55b8587c3e40 2024-04-22T13:23:33.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.607+0000 7f278cc6e740 10 client.0 _ll_register_callbacks cb 0x7f277c013620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:23:33.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.607+0000 7f2782ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8587c4830 0x55b8587d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.607+0000 7f2782ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8587c4830 0x55b8587d1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35086/0 (socket says 172.21.15.12:35086) 2024-04-22T13:23:33.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.607+0000 7f2782ffd640 1 -- 172.21.15.12:0/750789429 learned_addr learned my addr 172.21.15.12:0/750789429 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:33.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.607+0000 7f27837fe640 1 --2- 172.21.15.12:0/750789429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8587c3e40 0x55b8587d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.608+0000 7f2782ffd640 1 -- 172.21.15.12:0/750789429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8587c3e40 msgr2=0x55b8587d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.608+0000 7f2782ffd640 1 --2- 172.21.15.12:0/750789429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8587c3e40 0x55b8587d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.608+0000 7f2783fff640 1 --2- 172.21.15.12:0/750789429 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b8587c52d0 0x55b8587d3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.608+0000 7f2782ffd640 1 -- 172.21.15.12:0/750789429 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b8587c52d0 msgr2=0x55b8587d3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:33.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.608+0000 7f2782ffd640 1 --2- 172.21.15.12:0/750789429 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b8587c52d0 0x55b8587d3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.608+0000 7f2782ffd640 1 -- 172.21.15.12:0/750789429 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8587d4560 con 0x55b8587c4830 2024-04-22T13:23:33.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.608+0000 7f27837fe640 1 --2- 172.21.15.12:0/750789429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8587c3e40 0x55b8587d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:23:33.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.608+0000 7f2782ffd640 1 --2- 172.21.15.12:0/750789429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8587c4830 0x55b8587d1120 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f276c009b80 tx=0x7f276c0098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327cde6e7deb8e87 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:33.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.608+0000 7f2762ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:23:33.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.608+0000 7f2762ffd640 1 -- 172.21.15.12:0/750789429 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c028020 con 0x55b8587c4830 2024-04-22T13:23:33.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.608+0000 7f2762ffd640 1 -- 172.21.15.12:0/750789429 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f276c010d80 con 0x55b8587c4830 2024-04-22T13:23:33.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.608+0000 7f278cc6e740 1 -- 172.21.15.12:0/750789429 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f276c00f8c0 con 0x55b8587c4830 2024-04-22T13:23:33.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.608+0000 7f2762ffd640 1 -- 172.21.15.12:0/750789429 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f276c020040 con 0x55b8587c4830 2024-04-22T13:23:33.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.608+0000 7f278cc6e740 10 client.15135 fetch_fsmap learned FSMap version 27 2024-04-22T13:23:33.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.608+0000 7f278cc6e740 1 -- 172.21.15.12:0/750789429 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b858840ca0 con 0x55b8587c4830 2024-04-22T13:23:33.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.608+0000 7f2762ffd640 1 -- 172.21.15.12:0/750789429 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f276c00f8c0 con 0x55b8587c4830 2024-04-22T13:23:33.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.608+0000 7f2762ffd640 1 -- 172.21.15.12:0/750789429 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f276c023030 con 0x55b8587c4830 2024-04-22T13:23:33.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.608+0000 7f278cc6e740 10 client.15135 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:23:33.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.609+0000 7f278cc6e740 10 client.15135 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-22T13:23:33.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.609+0000 7f278cc6e740 20 client.15135 populate_metadata read hostname 'smithi012' 2024-04-22T13:23:33.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.609+0000 7f278cc6e740 1 --2- 172.21.15.12:0/750789429 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b858845580 0x55b858844290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:33.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.609+0000 7f278cc6e740 4 client.15135 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T13:23:33.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.609+0000 7f278cc6e740 1 -- 172.21.15.12:0/750789429 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "status"}) -- 0x55b8587303a0 con 0x55b858845580 2024-04-22T13:23:33.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.609+0000 7f27837fe640 1 --2- 172.21.15.12:0/750789429 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b858845580 0x55b858844290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:33.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.609+0000 7f2762ffd640 10 client.15135 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:23:33.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.609+0000 7f27837fe640 1 --2- 172.21.15.12:0/750789429 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b858845580 0x55b858844290 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1075dfdf93a267a4 server_cookie=f8e771e562332e4a in_seq=0 out_seq=0 2024-04-22T13:23:33.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.609+0000 7f2762ffd640 1 -- 172.21.15.12:0/750789429 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b8587303a0 con 0x55b858845580 2024-04-22T13:23:33.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.609+0000 7f2762ffd640 10 client.15135 handle_command_reply: tid=0 2024-04-22T13:23:33.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.609+0000 7f278cc6e740 1 client.15135 shutdown 2024-04-22T13:23:33.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.610+0000 7f278cc6e740 1 -- 172.21.15.12:0/750789429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8587c4830 msgr2=0x55b8587d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.610+0000 7f278cc6e740 1 --2- 172.21.15.12:0/750789429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8587c4830 0x55b8587d1120 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f276c009b80 tx=0x7f276c0098f0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.610+0000 7f278cc6e740 1 -- 172.21.15.12:0/750789429 shutdown_connections 2024-04-22T13:23:33.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.610+0000 7f278cc6e740 1 --2- 172.21.15.12:0/750789429 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b858845580 0x55b858844290 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.610+0000 7f278cc6e740 1 --2- 172.21.15.12:0/750789429 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b8587c52d0 0x55b8587d3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.610+0000 7f278cc6e740 1 --2- 172.21.15.12:0/750789429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b8587c4830 0x55b8587d1120 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.610+0000 7f2762ffd640 0 client.15135 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:23:33.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.610+0000 7f278cc6e740 1 --2- 172.21.15.12:0/750789429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b8587c3e40 0x55b8587d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.610+0000 7f278cc6e740 1 -- 172.21.15.12:0/750789429 >> 172.21.15.12:0/750789429 conn(0x55b8587bff40 msgr2=0x55b8587cdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:33.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.610+0000 7f278cc6e740 1 -- 172.21.15.12:0/750789429 shutdown_connections 2024-04-22T13:23:33.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.610+0000 7f278cc6e740 1 -- 172.21.15.12:0/750789429 wait complete. 2024-04-22T13:23:33.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.610+0000 7f278cc6e740 20 client.15135 trim_cache size 0 max 16384 2024-04-22T13:23:33.612 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:23:33.612 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "0aede5ea-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:23:33.613 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 0, 2024-04-22T13:23:33.613 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24509, 2024-04-22T13:23:33.613 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-04-22T13:23:33.613 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-04-22T13:23:33.613 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-04-22T13:23:33.613 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 88.241476237000001, 2024-04-22T13:23:33.613 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 27, 2024-04-22T13:23:33.613 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 89, 2024-04-22T13:23:33.613 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 87, 2024-04-22T13:23:33.613 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 93.888883523999993 2024-04-22T13:23:33.613 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:23:33.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.610+0000 7f278a074640 1 -- 172.21.15.12:0/2235584516 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f27700a3c40 msgr2=0x7f27700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.610+0000 7f278a074640 1 --2- 172.21.15.12:0/2235584516 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f27700a3c40 0x7f27700a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f277c023000 tx=0x7f277c021000 comp rx=0 tx=0).stop 2024-04-22T13:23:33.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.610+0000 7f278a074640 1 -- 172.21.15.12:0/2235584516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2784100800 msgr2=0x7f278419a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:33.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.610+0000 7f278a074640 1 --2- 172.21.15.12:0/2235584516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2784100800 0x7f278419a440 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2774005e30 tx=0x7f27740346f0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.610+0000 7f278a074640 1 -- 172.21.15.12:0/2235584516 shutdown_connections 2024-04-22T13:23:33.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.610+0000 7f278a074640 1 --2- 172.21.15.12:0/2235584516 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2784101b00 0x7f27841a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.610+0000 7f278a074640 1 --2- 172.21.15.12:0/2235584516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2784101160 0x7f278419a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.610+0000 7f278a074640 1 --2- 172.21.15.12:0/2235584516 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f27700a3c40 0x7f27700a6100 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.610+0000 7f278a074640 1 --2- 172.21.15.12:0/2235584516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2784100800 0x7f278419a440 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:33.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.610+0000 7f278a074640 1 -- 172.21.15.12:0/2235584516 >> 172.21.15.12:0/2235584516 conn(0x7f27840fc400 msgr2=0x7f27840fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:33.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.611+0000 7f278a074640 1 -- 172.21.15.12:0/2235584516 shutdown_connections 2024-04-22T13:23:33.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:33.611+0000 7f278a074640 1 -- 172.21.15.12:0/2235584516 wait complete. 2024-04-22T13:23:33.618 INFO:tasks.cephadm:Setup complete, yielding 2024-04-22T13:23:33.618 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-22T13:23:33.627 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-04-22T13:23:33.627 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-22T13:23:33.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:33 smithi012 ceph-mon[28265]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-22T13:23:33.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:33 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2855791810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:23:33.872 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:23:33.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:33 smithi134 ceph-mon[31338]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-22T13:23:33.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:33 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2855791810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:23:33.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:33 smithi111 ceph-mon[31361]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-22T13:23:33.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:33 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2855791810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:23:35.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.442+0000 7efdd04f3700 1 -- 172.21.15.12:0/517894507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdc81019a0 msgr2=0x7efdc8101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:35.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.442+0000 7efdd04f3700 1 --2- 172.21.15.12:0/517894507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdc81019a0 0x7efdc8101dc0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7efdb80099c0 tx=0x7efdb8009cd0 comp rx=0 tx=0).stop 2024-04-22T13:23:35.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.442+0000 7efdd04f3700 1 -- 172.21.15.12:0/517894507 shutdown_connections 2024-04-22T13:23:35.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.442+0000 7efdd04f3700 1 --2- 172.21.15.12:0/517894507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efdc8103540 0x7efdc8109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:35.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.442+0000 7efdd04f3700 1 --2- 172.21.15.12:0/517894507 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdc81019a0 0x7efdc8101dc0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:35.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.442+0000 7efdd04f3700 1 --2- 172.21.15.12:0/517894507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdc8102ba0 0x7efdc8103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:35.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.442+0000 7efdd04f3700 1 -- 172.21.15.12:0/517894507 >> 172.21.15.12:0/517894507 conn(0x7efdc80fcf40 msgr2=0x7efdc80ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:35.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.443+0000 7efdd04f3700 1 -- 172.21.15.12:0/517894507 shutdown_connections 2024-04-22T13:23:35.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.443+0000 7efdd04f3700 1 -- 172.21.15.12:0/517894507 wait complete. 2024-04-22T13:23:35.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.443+0000 7efdd04f3700 1 Processor -- start 2024-04-22T13:23:35.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.443+0000 7efdd04f3700 1 -- start start 2024-04-22T13:23:35.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.443+0000 7efdd04f3700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efdc81019a0 0x7efdc8198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:35.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.443+0000 7efdd04f3700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdc8102ba0 0x7efdc8199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:35.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.443+0000 7efdd04f3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdc8103540 0x7efdc81a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:35.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.443+0000 7efdd04f3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efdc810d430 con 0x7efdc8103540 2024-04-22T13:23:35.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.443+0000 7efdd04f3700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efdc810d2b0 con 0x7efdc81019a0 2024-04-22T13:23:35.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.443+0000 7efdd04f3700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efdc810d5b0 con 0x7efdc8102ba0 2024-04-22T13:23:35.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.444+0000 7efdcea90700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdc8103540 0x7efdc81a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:35.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.444+0000 7efdcea90700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdc8103540 0x7efdc81a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35118/0 (socket says 172.21.15.12:35118) 2024-04-22T13:23:35.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.444+0000 7efdcea90700 1 -- 172.21.15.12:0/1198203381 learned_addr learned my addr 172.21.15.12:0/1198203381 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:35.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.444+0000 7efdcda8e700 1 --2- 172.21.15.12:0/1198203381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdc8102ba0 0x7efdc8199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:35.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.444+0000 7efdce28f700 1 --2- 172.21.15.12:0/1198203381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efdc81019a0 0x7efdc8198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:35.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.444+0000 7efdcea90700 1 -- 172.21.15.12:0/1198203381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdc8102ba0 msgr2=0x7efdc8199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:35.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.444+0000 7efdcea90700 1 --2- 172.21.15.12:0/1198203381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdc8102ba0 0x7efdc8199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:35.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.444+0000 7efdcea90700 1 -- 172.21.15.12:0/1198203381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efdc81019a0 msgr2=0x7efdc8198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:35.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.444+0000 7efdcea90700 1 --2- 172.21.15.12:0/1198203381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efdc81019a0 0x7efdc8198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:35.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.444+0000 7efdcea90700 1 -- 172.21.15.12:0/1198203381 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc81a0860 con 0x7efdc8103540 2024-04-22T13:23:35.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.444+0000 7efdce28f700 1 --2- 172.21.15.12:0/1198203381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efdc81019a0 0x7efdc8198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:23:35.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.444+0000 7efdcea90700 1 --2- 172.21.15.12:0/1198203381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdc8103540 0x7efdc81a0190 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7efdc0007d70 tx=0x7efdc000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:35.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.444+0000 7efdbf7fe700 1 -- 172.21.15.12:0/1198203381 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc0016020 con 0x7efdc8103540 2024-04-22T13:23:35.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.444+0000 7efdbf7fe700 1 -- 172.21.15.12:0/1198203381 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdc000ad40 con 0x7efdc8103540 2024-04-22T13:23:35.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.444+0000 7efdd04f3700 1 -- 172.21.15.12:0/1198203381 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdc81a0b50 con 0x7efdc8103540 2024-04-22T13:23:35.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.444+0000 7efdbf7fe700 1 -- 172.21.15.12:0/1198203381 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc00115e0 con 0x7efdc8103540 2024-04-22T13:23:35.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.444+0000 7efdd04f3700 1 -- 172.21.15.12:0/1198203381 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdc81a1090 con 0x7efdc8103540 2024-04-22T13:23:35.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.445+0000 7efdd04f3700 1 -- 172.21.15.12:0/1198203381 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdc8069730 con 0x7efdc8103540 2024-04-22T13:23:35.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.446+0000 7efdbf7fe700 1 -- 172.21.15.12:0/1198203381 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7efdc0019020 con 0x7efdc8103540 2024-04-22T13:23:35.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.447+0000 7efdbf7fe700 1 --2- 172.21.15.12:0/1198203381 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efdb40a3cc0 0x7efdb40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:35.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.447+0000 7efdce28f700 1 --2- 172.21.15.12:0/1198203381 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efdb40a3cc0 0x7efdb40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:35.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.447+0000 7efdbf7fe700 1 -- 172.21.15.12:0/1198203381 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7efdc00d0340 con 0x7efdc8103540 2024-04-22T13:23:35.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.448+0000 7efdce28f700 1 --2- 172.21.15.12:0/1198203381 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efdb40a3cc0 0x7efdb40a6180 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009990 tx=0x7efdb8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:35.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.452+0000 7efdbf7fe700 1 -- 172.21.15.12:0/1198203381 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efdc0020020 con 0x7efdc8103540 2024-04-22T13:23:35.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.681+0000 7efdd04f3700 1 -- 172.21.15.12:0/1198203381 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7efdc80ff140 con 0x7efdb40a3cc0 2024-04-22T13:23:35.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.682+0000 7efdbf7fe700 1 -- 172.21.15.12:0/1198203381 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7efdc80ff140 con 0x7efdb40a3cc0 2024-04-22T13:23:35.678 INFO:teuthology.orchestra.run.smithi012.stdout:Backend: cephadm 2024-04-22T13:23:35.678 INFO:teuthology.orchestra.run.smithi012.stdout:Available: Yes 2024-04-22T13:23:35.678 INFO:teuthology.orchestra.run.smithi012.stdout:Paused: No 2024-04-22T13:23:35.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.684+0000 7efdd04f3700 1 -- 172.21.15.12:0/1198203381 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efdb40a3cc0 msgr2=0x7efdb40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:35.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.684+0000 7efdd04f3700 1 --2- 172.21.15.12:0/1198203381 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efdb40a3cc0 0x7efdb40a6180 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009990 tx=0x7efdb8005fb0 comp rx=0 tx=0).stop 2024-04-22T13:23:35.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.684+0000 7efdd04f3700 1 -- 172.21.15.12:0/1198203381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdc8103540 msgr2=0x7efdc81a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:35.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.684+0000 7efdd04f3700 1 --2- 172.21.15.12:0/1198203381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdc8103540 0x7efdc81a0190 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7efdc0007d70 tx=0x7efdc000a340 comp rx=0 tx=0).stop 2024-04-22T13:23:35.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.684+0000 7efdd04f3700 1 -- 172.21.15.12:0/1198203381 shutdown_connections 2024-04-22T13:23:35.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.684+0000 7efdd04f3700 1 --2- 172.21.15.12:0/1198203381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efdc81019a0 0x7efdc8198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:35.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.684+0000 7efdd04f3700 1 --2- 172.21.15.12:0/1198203381 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efdb40a3cc0 0x7efdb40a6180 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:35.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.684+0000 7efdd04f3700 1 --2- 172.21.15.12:0/1198203381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efdc8103540 0x7efdc81a0190 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:35.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.684+0000 7efdd04f3700 1 --2- 172.21.15.12:0/1198203381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efdc8102ba0 0x7efdc8199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:35.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.684+0000 7efdd04f3700 1 -- 172.21.15.12:0/1198203381 >> 172.21.15.12:0/1198203381 conn(0x7efdc80fcf40 msgr2=0x7efdc80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:35.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.684+0000 7efdd04f3700 1 -- 172.21.15.12:0/1198203381 shutdown_connections 2024-04-22T13:23:35.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:35.684+0000 7efdd04f3700 1 -- 172.21.15.12:0/1198203381 wait complete. 2024-04-22T13:23:35.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:35 smithi012 ceph-mon[28265]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-22T13:23:35.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:35 smithi012 ceph-mon[28265]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-22T13:23:35.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:35 smithi134 ceph-mon[31338]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-22T13:23:35.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:35 smithi134 ceph-mon[31338]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-22T13:23:35.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:35 smithi111 ceph-mon[31361]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-22T13:23:35.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:35 smithi111 ceph-mon[31361]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-22T13:23:36.381 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-22T13:23:36.641 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:23:36.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:36 smithi134 ceph-mon[31338]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:23:36.932 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:36 smithi012 ceph-mon[28265]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:23:36.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:36 smithi111 ceph-mon[31361]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:23:37.812 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:37 smithi012 ceph-mon[28265]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-22T13:23:37.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:37 smithi134 ceph-mon[31338]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-22T13:23:37.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:37 smithi111 ceph-mon[31361]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-22T13:23:38.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.283+0000 7fb1022d7700 1 -- 172.21.15.12:0/3225487912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0fc102ac0 msgr2=0x7fb0fc102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:38.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.283+0000 7fb1022d7700 1 --2- 172.21.15.12:0/3225487912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0fc102ac0 0x7fb0fc102ee0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec0099c0 tx=0x7fb0ec009cd0 comp rx=0 tx=0).stop 2024-04-22T13:23:38.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.283+0000 7fb1022d7700 1 -- 172.21.15.12:0/3225487912 shutdown_connections 2024-04-22T13:23:38.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.283+0000 7fb1022d7700 1 --2- 172.21.15.12:0/3225487912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0fc104660 0x7fb0fc10af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:38.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.283+0000 7fb1022d7700 1 --2- 172.21.15.12:0/3225487912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0fc102ac0 0x7fb0fc102ee0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:38.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.283+0000 7fb1022d7700 1 --2- 172.21.15.12:0/3225487912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0fc103cc0 0x7fb0fc104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:38.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.283+0000 7fb1022d7700 1 -- 172.21.15.12:0/3225487912 >> 172.21.15.12:0/3225487912 conn(0x7fb0fc0fe060 msgr2=0x7fb0fc1004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:38.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.283+0000 7fb1022d7700 1 -- 172.21.15.12:0/3225487912 shutdown_connections 2024-04-22T13:23:38.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.283+0000 7fb1022d7700 1 -- 172.21.15.12:0/3225487912 wait complete. 2024-04-22T13:23:38.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.283+0000 7fb1022d7700 1 Processor -- start 2024-04-22T13:23:38.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.283+0000 7fb1022d7700 1 -- start start 2024-04-22T13:23:38.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.284+0000 7fb1022d7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0fc102ac0 0x7fb0fc198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:38.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.284+0000 7fb1022d7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0fc103cc0 0x7fb0fc199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:38.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.284+0000 7fb1022d7700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0fc104660 0x7fb0fc1a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:38.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.284+0000 7fb1022d7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc10e550 con 0x7fb0fc102ac0 2024-04-22T13:23:38.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.284+0000 7fb1022d7700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc10e3d0 con 0x7fb0fc103cc0 2024-04-22T13:23:38.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.284+0000 7fb1022d7700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc10e6d0 con 0x7fb0fc104660 2024-04-22T13:23:38.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.284+0000 7fb0fbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0fc102ac0 0x7fb0fc198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:38.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.284+0000 7fb0fbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0fc102ac0 0x7fb0fc198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35144/0 (socket says 172.21.15.12:35144) 2024-04-22T13:23:38.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.284+0000 7fb0fbfff700 1 -- 172.21.15.12:0/1489725999 learned_addr learned my addr 172.21.15.12:0/1489725999 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:38.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.284+0000 7fb0fb7fe700 1 --2- 172.21.15.12:0/1489725999 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0fc103cc0 0x7fb0fc199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:38.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.284+0000 7fb100874700 1 --2- 172.21.15.12:0/1489725999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0fc104660 0x7fb0fc1a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:38.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.284+0000 7fb0fbfff700 1 -- 172.21.15.12:0/1489725999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0fc104660 msgr2=0x7fb0fc1a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:38.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.284+0000 7fb0fbfff700 1 --2- 172.21.15.12:0/1489725999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0fc104660 0x7fb0fc1a01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:38.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.284+0000 7fb0fbfff700 1 -- 172.21.15.12:0/1489725999 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0fc103cc0 msgr2=0x7fb0fc199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:38.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.284+0000 7fb0fbfff700 1 --2- 172.21.15.12:0/1489725999 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0fc103cc0 0x7fb0fc199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:38.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.284+0000 7fb0fbfff700 1 -- 172.21.15.12:0/1489725999 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0fc1a0890 con 0x7fb0fc102ac0 2024-04-22T13:23:38.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.284+0000 7fb0fb7fe700 1 --2- 172.21.15.12:0/1489725999 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0fc103cc0 0x7fb0fc199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:23:38.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.284+0000 7fb100874700 1 --2- 172.21.15.12:0/1489725999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0fc104660 0x7fb0fc1a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:23:38.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.284+0000 7fb0fbfff700 1 --2- 172.21.15.12:0/1489725999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0fc102ac0 0x7fb0fc198c40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00fa00 tx=0x7fb0ec00fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:38.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.285+0000 7fb0f97fa700 1 -- 172.21.15.12:0/1489725999 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec027070 con 0x7fb0fc102ac0 2024-04-22T13:23:38.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.285+0000 7fb0f97fa700 1 -- 172.21.15.12:0/1489725999 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0ec005940 con 0x7fb0fc102ac0 2024-04-22T13:23:38.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.285+0000 7fb1022d7700 1 -- 172.21.15.12:0/1489725999 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0fc1a0b20 con 0x7fb0fc102ac0 2024-04-22T13:23:38.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.285+0000 7fb0f97fa700 1 -- 172.21.15.12:0/1489725999 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec017650 con 0x7fb0fc102ac0 2024-04-22T13:23:38.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.285+0000 7fb1022d7700 1 -- 172.21.15.12:0/1489725999 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0fc1a1000 con 0x7fb0fc102ac0 2024-04-22T13:23:38.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.286+0000 7fb1022d7700 1 -- 172.21.15.12:0/1489725999 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0fc069730 con 0x7fb0fc102ac0 2024-04-22T13:23:38.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.287+0000 7fb0f97fa700 1 -- 172.21.15.12:0/1489725999 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb0ec02a070 con 0x7fb0fc102ac0 2024-04-22T13:23:38.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.287+0000 7fb0f97fa700 1 --2- 172.21.15.12:0/1489725999 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb0e40a3c40 0x7fb0e40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:38.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.287+0000 7fb0fb7fe700 1 --2- 172.21.15.12:0/1489725999 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb0e40a3c40 0x7fb0e40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:38.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.288+0000 7fb0f97fa700 1 -- 172.21.15.12:0/1489725999 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fb0ec0d13a0 con 0x7fb0fc102ac0 2024-04-22T13:23:38.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.288+0000 7fb0fb7fe700 1 --2- 172.21.15.12:0/1489725999 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb0e40a3c40 0x7fb0e40a6100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0005fd0 tx=0x7fb0f0005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:38.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.292+0000 7fb0f97fa700 1 -- 172.21.15.12:0/1489725999 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb0ec05eff0 con 0x7fb0fc102ac0 2024-04-22T13:23:38.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.510+0000 7fb1022d7700 1 -- 172.21.15.12:0/1489725999 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fb0fc062e80 con 0x7fb0e40a3c40 2024-04-22T13:23:38.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.524+0000 7fb0f97fa700 1 -- 172.21.15.12:0/1489725999 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7fb0fc062e80 con 0x7fb0e40a3c40 2024-04-22T13:23:38.520 INFO:teuthology.orchestra.run.smithi012.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-22T13:23:38.520 INFO:teuthology.orchestra.run.smithi012.stdout:mds.a smithi012 running (102s) 15s ago 101s 19.4M - 19.0.0-3156-gd69f13d8 da4b004c09e8 130122b54630 2024-04-22T13:23:38.521 INFO:teuthology.orchestra.run.smithi012.stdout:mds.b smithi111 running (99s) 32s ago 98s 26.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 ee7dfcb8429a 2024-04-22T13:23:38.521 INFO:teuthology.orchestra.run.smithi012.stdout:mds.c smithi134 running (113s) 32s ago 112s 16.8M - 19.0.0-3156-gd69f13d8 da4b004c09e8 43ee81f18aaa 2024-04-22T13:23:38.521 INFO:teuthology.orchestra.run.smithi012.stdout:mds.d smithi012 running (2m) 15s ago 2m 14.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 b96099ca848d 2024-04-22T13:23:38.521 INFO:teuthology.orchestra.run.smithi012.stdout:mds.e smithi111 running (118s) 32s ago 118s 17.7M - 19.0.0-3156-gd69f13d8 da4b004c09e8 933813d49eca 2024-04-22T13:23:38.521 INFO:teuthology.orchestra.run.smithi012.stdout:mds.f smithi134 running (115s) 32s ago 115s 16.8M - 19.0.0-3156-gd69f13d8 da4b004c09e8 8ad0c7dc2764 2024-04-22T13:23:38.521 INFO:teuthology.orchestra.run.smithi012.stdout:mds.g smithi012 running (104s) 15s ago 104s 17.6M - 19.0.0-3156-gd69f13d8 da4b004c09e8 2b9a2ed95a8d 2024-04-22T13:23:38.521 INFO:teuthology.orchestra.run.smithi012.stdout:mds.h smithi111 running (110s) 32s ago 109s 18.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 2081b13c1b65 2024-04-22T13:23:38.521 INFO:teuthology.orchestra.run.smithi012.stdout:mds.i smithi134 running (107s) 32s ago 107s 20.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 59128799688f 2024-04-22T13:23:38.521 INFO:teuthology.orchestra.run.smithi012.stdout:mds.j smithi012 running (2m) 15s ago 2m 15.4M - 19.0.0-3156-gd69f13d8 da4b004c09e8 4322eb25c662 2024-04-22T13:23:38.521 INFO:teuthology.orchestra.run.smithi012.stdout:mds.k smithi111 running (2m) 32s ago 2m 16.7M - 19.0.0-3156-gd69f13d8 da4b004c09e8 363efbebf019 2024-04-22T13:23:38.521 INFO:teuthology.orchestra.run.smithi012.stdout:mds.l smithi134 running (2m) 32s ago 2m 19.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 a9d4a04af814 2024-04-22T13:23:38.521 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.x smithi012 *:9283,8765 running (9m) 15s ago 9m 515M - 19.0.0-3156-gd69f13d8 da4b004c09e8 822680d55dcd 2024-04-22T13:23:38.521 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.y smithi111 *:8443,8765 running (7m) 32s ago 7m 454M - 19.0.0-3156-gd69f13d8 da4b004c09e8 094e71b40f5b 2024-04-22T13:23:38.521 INFO:teuthology.orchestra.run.smithi012.stdout:mgr.z smithi134 *:8443,8765 running (7m) 32s ago 7m 451M - 19.0.0-3156-gd69f13d8 da4b004c09e8 bed71e9456c4 2024-04-22T13:23:38.521 INFO:teuthology.orchestra.run.smithi012.stdout:mon.a smithi012 running (9m) 15s ago 9m 51.1M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 f2fedfae4761 2024-04-22T13:23:38.522 INFO:teuthology.orchestra.run.smithi012.stdout:mon.b smithi111 running (7m) 32s ago 7m 39.1M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 d0da6c572569 2024-04-22T13:23:38.522 INFO:teuthology.orchestra.run.smithi012.stdout:mon.c smithi134 running (7m) 32s ago 7m 38.4M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 b5335653a227 2024-04-22T13:23:38.522 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 smithi012 running (7m) 15s ago 7m 50.7M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 8bc0836a9f03 2024-04-22T13:23:38.522 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 smithi111 running (6m) 32s ago 6m 49.2M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 673de813ef92 2024-04-22T13:23:38.522 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 smithi134 running (6m) 32s ago 6m 61.5M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 948065a127b1 2024-04-22T13:23:38.522 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 smithi012 running (5m) 15s ago 5m 54.3M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 5f98e37eebf9 2024-04-22T13:23:38.522 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 smithi111 running (5m) 32s ago 5m 51.8M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 a2f7b06a1201 2024-04-22T13:23:38.522 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 smithi134 running (5m) 32s ago 5m 46.7M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 3fb7405bf104 2024-04-22T13:23:38.522 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6 smithi012 running (4m) 15s ago 4m 51.1M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 0c2b48670cd5 2024-04-22T13:23:38.522 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7 smithi111 running (4m) 32s ago 4m 56.9M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 20104481e22c 2024-04-22T13:23:38.522 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8 smithi134 running (3m) 32s ago 3m 46.2M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 dcfeaa679a8e 2024-04-22T13:23:38.522 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9 smithi012 running (3m) 15s ago 3m 44.9M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 e72530e1e36c 2024-04-22T13:23:38.522 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10 smithi111 running (2m) 32s ago 2m 45.2M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 99f24f1b918d 2024-04-22T13:23:38.522 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11 smithi134 running (2m) 32s ago 2m 51.0M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 23ebe1ae87f2 2024-04-22T13:23:38.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.527+0000 7fb1022d7700 1 -- 172.21.15.12:0/1489725999 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb0e40a3c40 msgr2=0x7fb0e40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:38.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.527+0000 7fb1022d7700 1 --2- 172.21.15.12:0/1489725999 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb0e40a3c40 0x7fb0e40a6100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0005fd0 tx=0x7fb0f0005ee0 comp rx=0 tx=0).stop 2024-04-22T13:23:38.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.527+0000 7fb1022d7700 1 -- 172.21.15.12:0/1489725999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0fc102ac0 msgr2=0x7fb0fc198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:38.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.527+0000 7fb1022d7700 1 --2- 172.21.15.12:0/1489725999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0fc102ac0 0x7fb0fc198c40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec00fa00 tx=0x7fb0ec00fae0 comp rx=0 tx=0).stop 2024-04-22T13:23:38.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.527+0000 7fb1022d7700 1 -- 172.21.15.12:0/1489725999 shutdown_connections 2024-04-22T13:23:38.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.527+0000 7fb1022d7700 1 --2- 172.21.15.12:0/1489725999 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb0e40a3c40 0x7fb0e40a6100 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:38.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.527+0000 7fb1022d7700 1 --2- 172.21.15.12:0/1489725999 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0fc104660 0x7fb0fc1a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:38.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.527+0000 7fb1022d7700 1 --2- 172.21.15.12:0/1489725999 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb0fc102ac0 0x7fb0fc198c40 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:38.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.527+0000 7fb1022d7700 1 --2- 172.21.15.12:0/1489725999 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0fc103cc0 0x7fb0fc199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:38.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.527+0000 7fb1022d7700 1 -- 172.21.15.12:0/1489725999 >> 172.21.15.12:0/1489725999 conn(0x7fb0fc0fe060 msgr2=0x7fb0fc1070e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:38.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.527+0000 7fb1022d7700 1 -- 172.21.15.12:0/1489725999 shutdown_connections 2024-04-22T13:23:38.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:38.527+0000 7fb1022d7700 1 -- 172.21.15.12:0/1489725999 wait complete. 2024-04-22T13:23:39.166 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-22T13:23:39.415 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:23:39.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:39 smithi012 ceph-mon[28265]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-22T13:23:39.823 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:39 smithi012 ceph-mon[28265]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:23:39.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:39 smithi111 ceph-mon[31361]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-22T13:23:39.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:39 smithi111 ceph-mon[31361]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:23:40.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:39 smithi134 ceph-mon[31338]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-22T13:23:40.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:39 smithi134 ceph-mon[31338]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:23:41.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.118+0000 7f16ec048700 1 -- 172.21.15.12:0/189655982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16e4102ac0 msgr2=0x7f16e4102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:41.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.118+0000 7f16ec048700 1 --2- 172.21.15.12:0/189655982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16e4102ac0 0x7f16e4102ee0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f16e00099c0 tx=0x7f16e0009cd0 comp rx=0 tx=0).stop 2024-04-22T13:23:41.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.118+0000 7f16ec048700 1 -- 172.21.15.12:0/189655982 shutdown_connections 2024-04-22T13:23:41.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.118+0000 7f16ec048700 1 --2- 172.21.15.12:0/189655982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16e4104660 0x7f16e410af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:41.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.118+0000 7f16ec048700 1 --2- 172.21.15.12:0/189655982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16e4102ac0 0x7f16e4102ee0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:41.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.118+0000 7f16ec048700 1 --2- 172.21.15.12:0/189655982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16e4103cc0 0x7f16e4104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:41.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.118+0000 7f16ec048700 1 -- 172.21.15.12:0/189655982 >> 172.21.15.12:0/189655982 conn(0x7f16e40fe060 msgr2=0x7f16e41004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:41.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.118+0000 7f16ec048700 1 -- 172.21.15.12:0/189655982 shutdown_connections 2024-04-22T13:23:41.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.118+0000 7f16ec048700 1 -- 172.21.15.12:0/189655982 wait complete. 2024-04-22T13:23:41.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.119+0000 7f16ec048700 1 Processor -- start 2024-04-22T13:23:41.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.119+0000 7f16ec048700 1 -- start start 2024-04-22T13:23:41.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.119+0000 7f16ec048700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16e4102ac0 0x7f16e4198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:41.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.119+0000 7f16ec048700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16e4103cc0 0x7f16e41991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:41.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.119+0000 7f16ec048700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16e4104660 0x7f16e41a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:41.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.119+0000 7f16ec048700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f16e410e530 con 0x7f16e4102ac0 2024-04-22T13:23:41.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.119+0000 7f16ec048700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f16e410e3b0 con 0x7f16e4104660 2024-04-22T13:23:41.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.119+0000 7f16ec048700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f16e410e6b0 con 0x7f16e4103cc0 2024-04-22T13:23:41.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.120+0000 7f16e9de4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16e4102ac0 0x7f16e4198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:41.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.120+0000 7f16e9de4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16e4102ac0 0x7f16e4198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35168/0 (socket says 172.21.15.12:35168) 2024-04-22T13:23:41.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.120+0000 7f16e9de4700 1 -- 172.21.15.12:0/1828997846 learned_addr learned my addr 172.21.15.12:0/1828997846 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:41.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.120+0000 7f16ea5e5700 1 --2- 172.21.15.12:0/1828997846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16e4104660 0x7f16e41a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:41.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.120+0000 7f16e95e3700 1 --2- 172.21.15.12:0/1828997846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16e4103cc0 0x7f16e41991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:41.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.120+0000 7f16e9de4700 1 -- 172.21.15.12:0/1828997846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16e4103cc0 msgr2=0x7f16e41991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:41.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.120+0000 7f16e9de4700 1 --2- 172.21.15.12:0/1828997846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16e4103cc0 0x7f16e41991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:41.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.120+0000 7f16e9de4700 1 -- 172.21.15.12:0/1828997846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16e4104660 msgr2=0x7f16e41a01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:41.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.120+0000 7f16e9de4700 1 --2- 172.21.15.12:0/1828997846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16e4104660 0x7f16e41a01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:41.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.120+0000 7f16e9de4700 1 -- 172.21.15.12:0/1828997846 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e41a0880 con 0x7f16e4102ac0 2024-04-22T13:23:41.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.120+0000 7f16e9de4700 1 --2- 172.21.15.12:0/1828997846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16e4102ac0 0x7f16e4198c80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f16e00036f0 tx=0x7f16e000f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:41.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.120+0000 7f16d6ffd700 1 -- 172.21.15.12:0/1828997846 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0009e60 con 0x7f16e4102ac0 2024-04-22T13:23:41.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.120+0000 7f16ec048700 1 -- 172.21.15.12:0/1828997846 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e41a0b10 con 0x7f16e4102ac0 2024-04-22T13:23:41.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.120+0000 7f16d6ffd700 1 -- 172.21.15.12:0/1828997846 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16e0005940 con 0x7f16e4102ac0 2024-04-22T13:23:41.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.120+0000 7f16ec048700 1 -- 172.21.15.12:0/1828997846 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e41a1040 con 0x7f16e4102ac0 2024-04-22T13:23:41.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.121+0000 7f16d6ffd700 1 -- 172.21.15.12:0/1828997846 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e00054f0 con 0x7f16e4102ac0 2024-04-22T13:23:41.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.122+0000 7f16ec048700 1 -- 172.21.15.12:0/1828997846 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16e4069730 con 0x7f16e4102ac0 2024-04-22T13:23:41.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.123+0000 7f16d6ffd700 1 -- 172.21.15.12:0/1828997846 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f16e000fb80 con 0x7f16e4102ac0 2024-04-22T13:23:41.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.123+0000 7f16d6ffd700 1 --2- 172.21.15.12:0/1828997846 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f16d00a3c40 0x7f16d00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:41.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.123+0000 7f16e95e3700 1 --2- 172.21.15.12:0/1828997846 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f16d00a3c40 0x7f16d00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:41.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.123+0000 7f16d6ffd700 1 -- 172.21.15.12:0/1828997846 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f16e00d1180 con 0x7f16e4102ac0 2024-04-22T13:23:41.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.124+0000 7f16e95e3700 1 --2- 172.21.15.12:0/1828997846 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f16d00a3c40 0x7f16d00a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009c60 tx=0x7f16d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:41.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.128+0000 7f16d6ffd700 1 -- 172.21.15.12:0/1828997846 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f16e0013070 con 0x7f16e4102ac0 2024-04-22T13:23:41.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.346+0000 7f16ec048700 1 -- 172.21.15.12:0/1828997846 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f16e4062e80 con 0x7f16d00a3c40 2024-04-22T13:23:41.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.352+0000 7f16d6ffd700 1 -- 172.21.15.12:0/1828997846 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f16e4062e80 con 0x7f16d00a3c40 2024-04-22T13:23:41.349 INFO:teuthology.orchestra.run.smithi012.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-22T13:23:41.349 INFO:teuthology.orchestra.run.smithi012.stdout:mds.all 12/12 35s ago 2m smithi012=a;smithi012=d;smithi012=g;smithi012=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi134=c;smithi134=f;smithi134=i;smithi134=l;count:12 2024-04-22T13:23:41.349 INFO:teuthology.orchestra.run.smithi012.stdout:mgr 3/3 35s ago 7m smithi012=x;smithi111=y;smithi134=z;count:3 2024-04-22T13:23:41.349 INFO:teuthology.orchestra.run.smithi012.stdout:mon 3/3 35s ago 7m smithi012:172.21.15.12=a;smithi111:172.21.15.111=b;smithi134:172.21.15.134=c;count:3 2024-04-22T13:23:41.349 INFO:teuthology.orchestra.run.smithi012.stdout:osd 12 35s ago - 2024-04-22T13:23:41.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.355+0000 7f16ec048700 1 -- 172.21.15.12:0/1828997846 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f16d00a3c40 msgr2=0x7f16d00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:41.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.355+0000 7f16ec048700 1 --2- 172.21.15.12:0/1828997846 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f16d00a3c40 0x7f16d00a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f16d8009c60 tx=0x7f16d8009260 comp rx=0 tx=0).stop 2024-04-22T13:23:41.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.355+0000 7f16ec048700 1 -- 172.21.15.12:0/1828997846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16e4102ac0 msgr2=0x7f16e4198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:41.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.355+0000 7f16ec048700 1 --2- 172.21.15.12:0/1828997846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16e4102ac0 0x7f16e4198c80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f16e00036f0 tx=0x7f16e000f710 comp rx=0 tx=0).stop 2024-04-22T13:23:41.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.355+0000 7f16ec048700 1 -- 172.21.15.12:0/1828997846 shutdown_connections 2024-04-22T13:23:41.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.355+0000 7f16ec048700 1 --2- 172.21.15.12:0/1828997846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f16e4104660 0x7f16e41a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:41.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.355+0000 7f16ec048700 1 --2- 172.21.15.12:0/1828997846 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f16d00a3c40 0x7f16d00a6100 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:41.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.355+0000 7f16ec048700 1 --2- 172.21.15.12:0/1828997846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f16e4102ac0 0x7f16e4198c80 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:41.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.355+0000 7f16ec048700 1 --2- 172.21.15.12:0/1828997846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f16e4103cc0 0x7f16e41991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:41.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.355+0000 7f16ec048700 1 -- 172.21.15.12:0/1828997846 >> 172.21.15.12:0/1828997846 conn(0x7f16e40fe060 msgr2=0x7f16e41070e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:41.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.355+0000 7f16ec048700 1 -- 172.21.15.12:0/1828997846 shutdown_connections 2024-04-22T13:23:41.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:41.355+0000 7f16ec048700 1 -- 172.21.15.12:0/1828997846 wait complete. 2024-04-22T13:23:41.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:41 smithi111 ceph-mon[31361]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-22T13:23:42.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:41 smithi012 ceph-mon[28265]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-22T13:23:42.159 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-22T13:23:42.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:41 smithi134 ceph-mon[31338]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-22T13:23:42.407 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:23:42.789 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:42 smithi012 ceph-mon[28265]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:23:42.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:42 smithi111 ceph-mon[31361]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:23:43.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:42 smithi134 ceph-mon[31338]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:23:43.824 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:43 smithi012 ceph-mon[28265]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-22T13:23:43.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:43 smithi111 ceph-mon[31361]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-22T13:23:44.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.177+0000 7efcc5925700 1 -- 172.21.15.12:0/2123845374 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcc0100b10 msgr2=0x7efcc0100f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:44.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.177+0000 7efcc5925700 1 --2- 172.21.15.12:0/2123845374 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcc0100b10 0x7efcc0100f90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7efcb4009990 tx=0x7efcb4009ca0 comp rx=0 tx=0).stop 2024-04-22T13:23:44.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.177+0000 7efcc5925700 1 -- 172.21.15.12:0/2123845374 shutdown_connections 2024-04-22T13:23:44.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.177+0000 7efcc5925700 1 --2- 172.21.15.12:0/2123845374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcc0103010 0x7efcc0103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:44.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.177+0000 7efcc5925700 1 --2- 172.21.15.12:0/2123845374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcc01014d0 0x7efcc0101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:44.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.177+0000 7efcc5925700 1 --2- 172.21.15.12:0/2123845374 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcc0100b10 0x7efcc0100f90 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:44.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.177+0000 7efcc5925700 1 -- 172.21.15.12:0/2123845374 >> 172.21.15.12:0/2123845374 conn(0x7efcc00fc6d0 msgr2=0x7efcc00feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:44.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.177+0000 7efcc5925700 1 -- 172.21.15.12:0/2123845374 shutdown_connections 2024-04-22T13:23:44.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.177+0000 7efcc5925700 1 -- 172.21.15.12:0/2123845374 wait complete. 2024-04-22T13:23:44.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.178+0000 7efcc5925700 1 Processor -- start 2024-04-22T13:23:44.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.178+0000 7efcc5925700 1 -- start start 2024-04-22T13:23:44.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.178+0000 7efcc5925700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcc0100b10 0x7efcc019adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:44.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.178+0000 7efcc5925700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcc01014d0 0x7efcc019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:44.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.178+0000 7efcc5925700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcc0103010 0x7efcc01a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:44.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.178+0000 7efcc5925700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efcc0110760 con 0x7efcc0100b10 2024-04-22T13:23:44.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.178+0000 7efcc5925700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efcc01105e0 con 0x7efcc0103010 2024-04-22T13:23:44.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.178+0000 7efcc5925700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efcc01108e0 con 0x7efcc01014d0 2024-04-22T13:23:44.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.178+0000 7efcbeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcc0100b10 0x7efcc019adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:44.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.178+0000 7efcbeffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcc0100b10 0x7efcc019adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45668/0 (socket says 172.21.15.12:45668) 2024-04-22T13:23:44.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.178+0000 7efcbeffd700 1 -- 172.21.15.12:0/577507660 learned_addr learned my addr 172.21.15.12:0/577507660 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:44.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.178+0000 7efcbe7fc700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcc01014d0 0x7efcc019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:44.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.178+0000 7efcbf7fe700 1 --2- 172.21.15.12:0/577507660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcc0103010 0x7efcc01a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:44.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.179+0000 7efcbeffd700 1 -- 172.21.15.12:0/577507660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcc01014d0 msgr2=0x7efcc019b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:44.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.179+0000 7efcbeffd700 1 --2- 172.21.15.12:0/577507660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcc01014d0 0x7efcc019b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:44.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.179+0000 7efcbeffd700 1 -- 172.21.15.12:0/577507660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcc0103010 msgr2=0x7efcc01a23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:44.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.179+0000 7efcbeffd700 1 --2- 172.21.15.12:0/577507660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcc0103010 0x7efcc01a23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:44.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.179+0000 7efcbeffd700 1 -- 172.21.15.12:0/577507660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc01a2a50 con 0x7efcc0100b10 2024-04-22T13:23:44.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.179+0000 7efcbe7fc700 1 --2- 172.21.15.12:0/577507660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcc01014d0 0x7efcc019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:23:44.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.179+0000 7efcbeffd700 1 --2- 172.21.15.12:0/577507660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcc0100b10 0x7efcc019adf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7efcb000b560 tx=0x7efcb000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:44.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.179+0000 7efcc4923700 1 -- 172.21.15.12:0/577507660 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0015020 con 0x7efcc0100b10 2024-04-22T13:23:44.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.179+0000 7efcc4923700 1 -- 172.21.15.12:0/577507660 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcb0011ce0 con 0x7efcc0100b10 2024-04-22T13:23:44.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:43 smithi134 ceph-mon[31338]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-22T13:23:44.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.179+0000 7efcc5925700 1 -- 172.21.15.12:0/577507660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc01a2d40 con 0x7efcc0100b10 2024-04-22T13:23:44.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.179+0000 7efcc4923700 1 -- 172.21.15.12:0/577507660 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb0010d40 con 0x7efcc0100b10 2024-04-22T13:23:44.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.179+0000 7efcc5925700 1 -- 172.21.15.12:0/577507660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc01a3290 con 0x7efcc0100b10 2024-04-22T13:23:44.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.180+0000 7efcc5925700 1 -- 172.21.15.12:0/577507660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcc0069730 con 0x7efcc0100b10 2024-04-22T13:23:44.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.182+0000 7efcc4923700 1 -- 172.21.15.12:0/577507660 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7efcb0018020 con 0x7efcc0100b10 2024-04-22T13:23:44.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.183+0000 7efcc4923700 1 --2- 172.21.15.12:0/577507660 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efca80a3cc0 0x7efca80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:44.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.183+0000 7efcbe7fc700 1 --2- 172.21.15.12:0/577507660 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efca80a3cc0 0x7efca80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:44.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.183+0000 7efcc4923700 1 -- 172.21.15.12:0/577507660 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7efcb00cffb0 con 0x7efcc0100b10 2024-04-22T13:23:44.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.183+0000 7efcbe7fc700 1 --2- 172.21.15.12:0/577507660 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efca80a3cc0 0x7efca80a6180 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7efcb400b5c0 tx=0x7efcb4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:44.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.186+0000 7efcc4923700 1 -- 172.21.15.12:0/577507660 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efcb001f020 con 0x7efcc0100b10 2024-04-22T13:23:44.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.402+0000 7efcc5925700 1 -- 172.21.15.12:0/577507660 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7efcc0062e80 con 0x7efca80a3cc0 2024-04-22T13:23:44.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.403+0000 7efcc4923700 1 -- 172.21.15.12:0/577507660 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7efcc0062e80 con 0x7efca80a3cc0 2024-04-22T13:23:44.399 INFO:teuthology.orchestra.run.smithi012.stdout:HOST ADDR LABELS STATUS 2024-04-22T13:23:44.399 INFO:teuthology.orchestra.run.smithi012.stdout:smithi012 172.21.15.12 2024-04-22T13:23:44.400 INFO:teuthology.orchestra.run.smithi012.stdout:smithi111 172.21.15.111 2024-04-22T13:23:44.400 INFO:teuthology.orchestra.run.smithi012.stdout:smithi134 172.21.15.134 2024-04-22T13:23:44.400 INFO:teuthology.orchestra.run.smithi012.stdout:3 hosts in cluster 2024-04-22T13:23:44.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.405+0000 7efcc5925700 1 -- 172.21.15.12:0/577507660 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efca80a3cc0 msgr2=0x7efca80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:44.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.405+0000 7efcc5925700 1 --2- 172.21.15.12:0/577507660 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efca80a3cc0 0x7efca80a6180 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7efcb400b5c0 tx=0x7efcb4005fb0 comp rx=0 tx=0).stop 2024-04-22T13:23:44.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.405+0000 7efcc5925700 1 -- 172.21.15.12:0/577507660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcc0100b10 msgr2=0x7efcc019adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:44.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.405+0000 7efcc5925700 1 --2- 172.21.15.12:0/577507660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcc0100b10 0x7efcc019adf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7efcb000b560 tx=0x7efcb000b870 comp rx=0 tx=0).stop 2024-04-22T13:23:44.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.405+0000 7efcc5925700 1 -- 172.21.15.12:0/577507660 shutdown_connections 2024-04-22T13:23:44.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.405+0000 7efcc5925700 1 --2- 172.21.15.12:0/577507660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcc0103010 0x7efcc01a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:44.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.405+0000 7efcc5925700 1 --2- 172.21.15.12:0/577507660 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7efca80a3cc0 0x7efca80a6180 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:44.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.405+0000 7efcc5925700 1 --2- 172.21.15.12:0/577507660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efcc0100b10 0x7efcc019adf0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:44.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.405+0000 7efcc5925700 1 --2- 172.21.15.12:0/577507660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcc01014d0 0x7efcc019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:44.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.405+0000 7efcc5925700 1 -- 172.21.15.12:0/577507660 >> 172.21.15.12:0/577507660 conn(0x7efcc00fc6d0 msgr2=0x7efcc00fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:44.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.405+0000 7efcc5925700 1 -- 172.21.15.12:0/577507660 shutdown_connections 2024-04-22T13:23:44.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:44.405+0000 7efcc5925700 1 -- 172.21.15.12:0/577507660 wait complete. 2024-04-22T13:23:45.127 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-22T13:23:45.380 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:23:45.805 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:45 smithi012 ceph-mon[28265]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-22T13:23:45.805 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:45 smithi012 ceph-mon[28265]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:23:45.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:45 smithi111 ceph-mon[31361]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-22T13:23:45.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:45 smithi111 ceph-mon[31361]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:23:46.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:45 smithi134 ceph-mon[31338]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-22T13:23:46.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:45 smithi134 ceph-mon[31338]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:23:47.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.281+0000 7ffa83489700 1 -- 172.21.15.12:0/1805928623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa7c103540 msgr2=0x7ffa7c109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:47.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.281+0000 7ffa83489700 1 --2- 172.21.15.12:0/1805928623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa7c103540 0x7ffa7c109df0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400b0d0 tx=0x7ffa7400b3e0 comp rx=0 tx=0).stop 2024-04-22T13:23:47.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.281+0000 7ffa83489700 1 -- 172.21.15.12:0/1805928623 shutdown_connections 2024-04-22T13:23:47.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.281+0000 7ffa83489700 1 --2- 172.21.15.12:0/1805928623 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa7c103540 0x7ffa7c109df0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:47.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.281+0000 7ffa83489700 1 --2- 172.21.15.12:0/1805928623 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa7c1019a0 0x7ffa7c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:47.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.281+0000 7ffa83489700 1 --2- 172.21.15.12:0/1805928623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa7c102ba0 0x7ffa7c103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:47.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.281+0000 7ffa83489700 1 -- 172.21.15.12:0/1805928623 >> 172.21.15.12:0/1805928623 conn(0x7ffa7c0fcf40 msgr2=0x7ffa7c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:47.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.281+0000 7ffa83489700 1 -- 172.21.15.12:0/1805928623 shutdown_connections 2024-04-22T13:23:47.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.281+0000 7ffa83489700 1 -- 172.21.15.12:0/1805928623 wait complete. 2024-04-22T13:23:47.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.282+0000 7ffa83489700 1 Processor -- start 2024-04-22T13:23:47.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.282+0000 7ffa83489700 1 -- start start 2024-04-22T13:23:47.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.282+0000 7ffa83489700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa7c1019a0 0x7ffa7c06bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:47.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.282+0000 7ffa83489700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa7c102ba0 0x7ffa7c19d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:47.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.282+0000 7ffa83489700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa7c103540 0x7ffa7c19fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:47.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.282+0000 7ffa83489700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c10d4b0 con 0x7ffa7c1019a0 2024-04-22T13:23:47.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.282+0000 7ffa83489700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c10d330 con 0x7ffa7c102ba0 2024-04-22T13:23:47.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.282+0000 7ffa83489700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c10d630 con 0x7ffa7c103540 2024-04-22T13:23:47.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.282+0000 7ffa81225700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa7c1019a0 0x7ffa7c06bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:47.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.282+0000 7ffa81225700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa7c1019a0 0x7ffa7c06bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45684/0 (socket says 172.21.15.12:45684) 2024-04-22T13:23:47.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.282+0000 7ffa81225700 1 -- 172.21.15.12:0/1094176777 learned_addr learned my addr 172.21.15.12:0/1094176777 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:47.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.282+0000 7ffa80a24700 1 --2- 172.21.15.12:0/1094176777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa7c102ba0 0x7ffa7c19d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:47.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.282+0000 7ffa81a26700 1 --2- 172.21.15.12:0/1094176777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa7c103540 0x7ffa7c19fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:47.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.282+0000 7ffa81225700 1 -- 172.21.15.12:0/1094176777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa7c103540 msgr2=0x7ffa7c19fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:47.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.282+0000 7ffa81225700 1 --2- 172.21.15.12:0/1094176777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa7c103540 0x7ffa7c19fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:47.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.282+0000 7ffa81225700 1 -- 172.21.15.12:0/1094176777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa7c102ba0 msgr2=0x7ffa7c19d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:47.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.282+0000 7ffa81225700 1 --2- 172.21.15.12:0/1094176777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa7c102ba0 0x7ffa7c19d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:47.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.282+0000 7ffa81225700 1 -- 172.21.15.12:0/1094176777 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7c1a0790 con 0x7ffa7c1019a0 2024-04-22T13:23:47.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.282+0000 7ffa81225700 1 --2- 172.21.15.12:0/1094176777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa7c1019a0 0x7ffa7c06bcf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800b500 tx=0x7ffa7800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:47.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.283+0000 7ffa727fc700 1 -- 172.21.15.12:0/1094176777 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa78015020 con 0x7ffa7c1019a0 2024-04-22T13:23:47.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.283+0000 7ffa727fc700 1 -- 172.21.15.12:0/1094176777 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa78011ce0 con 0x7ffa7c1019a0 2024-04-22T13:23:47.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.283+0000 7ffa83489700 1 -- 172.21.15.12:0/1094176777 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa7c1a0a80 con 0x7ffa7c1019a0 2024-04-22T13:23:47.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.283+0000 7ffa727fc700 1 -- 172.21.15.12:0/1094176777 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa78010d20 con 0x7ffa7c1019a0 2024-04-22T13:23:47.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.283+0000 7ffa83489700 1 -- 172.21.15.12:0/1094176777 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa7c1a1040 con 0x7ffa7c1019a0 2024-04-22T13:23:47.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.284+0000 7ffa83489700 1 -- 172.21.15.12:0/1094176777 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa7c069730 con 0x7ffa7c1019a0 2024-04-22T13:23:47.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.285+0000 7ffa727fc700 1 -- 172.21.15.12:0/1094176777 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ffa78018020 con 0x7ffa7c1019a0 2024-04-22T13:23:47.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.285+0000 7ffa727fc700 1 --2- 172.21.15.12:0/1094176777 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ffa680a3cc0 0x7ffa680a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:47.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.286+0000 7ffa727fc700 1 -- 172.21.15.12:0/1094176777 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7ffa780cfad0 con 0x7ffa7c1019a0 2024-04-22T13:23:47.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.286+0000 7ffa80a24700 1 --2- 172.21.15.12:0/1094176777 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ffa680a3cc0 0x7ffa680a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:47.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.286+0000 7ffa80a24700 1 --2- 172.21.15.12:0/1094176777 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ffa680a3cc0 0x7ffa680a6180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c06c250 tx=0x7ffa6c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:47.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.290+0000 7ffa727fc700 1 -- 172.21.15.12:0/1094176777 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa7801f020 con 0x7ffa7c1019a0 2024-04-22T13:23:47.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.504+0000 7ffa83489700 1 -- 172.21.15.12:0/1094176777 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7ffa7c062e80 con 0x7ffa680a3cc0 2024-04-22T13:23:47.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.506+0000 7ffa727fc700 1 -- 172.21.15.12:0/1094176777 <== mgr.14152 v2:172.21.15.12:6800/1802873236 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7ffa7c062e80 con 0x7ffa680a3cc0 2024-04-22T13:23:47.502 INFO:teuthology.orchestra.run.smithi012.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-22T13:23:47.502 INFO:teuthology.orchestra.run.smithi012.stdout:smithi012 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331005J400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-22T13:23:47.502 INFO:teuthology.orchestra.run.smithi012.stdout:smithi111 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400VZ400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-22T13:23:47.502 INFO:teuthology.orchestra.run.smithi012.stdout:smithi134 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62420002400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-22T13:23:47.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.508+0000 7ffa83489700 1 -- 172.21.15.12:0/1094176777 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ffa680a3cc0 msgr2=0x7ffa680a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:47.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.508+0000 7ffa83489700 1 --2- 172.21.15.12:0/1094176777 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ffa680a3cc0 0x7ffa680a6180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c06c250 tx=0x7ffa6c00a380 comp rx=0 tx=0).stop 2024-04-22T13:23:47.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.508+0000 7ffa83489700 1 -- 172.21.15.12:0/1094176777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa7c1019a0 msgr2=0x7ffa7c06bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:47.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.508+0000 7ffa83489700 1 --2- 172.21.15.12:0/1094176777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa7c1019a0 0x7ffa7c06bcf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800b500 tx=0x7ffa7800b810 comp rx=0 tx=0).stop 2024-04-22T13:23:47.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.508+0000 7ffa83489700 1 -- 172.21.15.12:0/1094176777 shutdown_connections 2024-04-22T13:23:47.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.508+0000 7ffa83489700 1 --2- 172.21.15.12:0/1094176777 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7ffa680a3cc0 0x7ffa680a6180 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:47.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.508+0000 7ffa83489700 1 --2- 172.21.15.12:0/1094176777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa7c103540 0x7ffa7c19fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:47.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.508+0000 7ffa83489700 1 --2- 172.21.15.12:0/1094176777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffa7c1019a0 0x7ffa7c06bcf0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:47.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.508+0000 7ffa83489700 1 --2- 172.21.15.12:0/1094176777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa7c102ba0 0x7ffa7c19d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:47.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.508+0000 7ffa83489700 1 -- 172.21.15.12:0/1094176777 >> 172.21.15.12:0/1094176777 conn(0x7ffa7c0fcf40 msgr2=0x7ffa7c105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:47.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.508+0000 7ffa83489700 1 -- 172.21.15.12:0/1094176777 shutdown_connections 2024-04-22T13:23:47.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:47.508+0000 7ffa83489700 1 -- 172.21.15.12:0/1094176777 wait complete. 2024-04-22T13:23:47.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:47 smithi111 ceph-mon[31361]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-22T13:23:47.980 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:47 smithi134 ceph-mon[31338]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-22T13:23:48.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:47 smithi012 ceph-mon[28265]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-22T13:23:48.122 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-22T13:23:48.131 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-04-22T13:23:48.131 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-22T13:23:48.382 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:23:48.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:48 smithi012 ceph-mon[28265]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:23:48.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:48 smithi111 ceph-mon[31361]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:23:49.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:48 smithi134 ceph-mon[31338]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:23:49.736 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:49 smithi012 ceph-mon[28265]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-22T13:23:49.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.939+0000 7fe39e794700 1 -- 172.21.15.12:0/3347461915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe398103540 msgr2=0x7fe398109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:49.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.939+0000 7fe39e794700 1 --2- 172.21.15.12:0/3347461915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe398103540 0x7fe398109df0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe39000b0d0 tx=0x7fe39000b3e0 comp rx=0 tx=0).stop 2024-04-22T13:23:49.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.939+0000 7fe39e794700 1 -- 172.21.15.12:0/3347461915 shutdown_connections 2024-04-22T13:23:49.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.939+0000 7fe39e794700 1 --2- 172.21.15.12:0/3347461915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe3981019a0 0x7fe398101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:49.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.939+0000 7fe39e794700 1 --2- 172.21.15.12:0/3347461915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe398103540 0x7fe398109df0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:49.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.939+0000 7fe39e794700 1 --2- 172.21.15.12:0/3347461915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe398102ba0 0x7fe398103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:49.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.939+0000 7fe39e794700 1 -- 172.21.15.12:0/3347461915 >> 172.21.15.12:0/3347461915 conn(0x7fe3980fcf40 msgr2=0x7fe3980ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:49.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.939+0000 7fe39e794700 1 -- 172.21.15.12:0/3347461915 shutdown_connections 2024-04-22T13:23:49.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.939+0000 7fe39e794700 1 -- 172.21.15.12:0/3347461915 wait complete. 2024-04-22T13:23:49.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.940+0000 7fe39e794700 1 Processor -- start 2024-04-22T13:23:49.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.940+0000 7fe39e794700 1 -- start start 2024-04-22T13:23:49.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.940+0000 7fe39e794700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3981019a0 0x7fe398105930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:49.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.940+0000 7fe39e794700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe398102ba0 0x7fe398103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:49.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.940+0000 7fe39e794700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe398103540 0x7fe3981044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:49.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.940+0000 7fe39e794700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe39810d490 con 0x7fe398102ba0 2024-04-22T13:23:49.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.940+0000 7fe39e794700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe39810d310 con 0x7fe398103540 2024-04-22T13:23:49.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.940+0000 7fe39e794700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe39810d610 con 0x7fe3981019a0 2024-04-22T13:23:49.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.940+0000 7fe3977fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe398102ba0 0x7fe398103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:49.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.940+0000 7fe3977fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe398102ba0 0x7fe398103f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45704/0 (socket says 172.21.15.12:45704) 2024-04-22T13:23:49.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.940+0000 7fe3977fe700 1 -- 172.21.15.12:0/3714057462 learned_addr learned my addr 172.21.15.12:0/3714057462 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:49.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.940+0000 7fe397fff700 1 --2- 172.21.15.12:0/3714057462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3981019a0 0x7fe398105930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:49.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.940+0000 7fe39cd31700 1 --2- 172.21.15.12:0/3714057462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe398103540 0x7fe3981044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:49.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.940+0000 7fe3977fe700 1 -- 172.21.15.12:0/3714057462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3981019a0 msgr2=0x7fe398105930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:49.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.940+0000 7fe3977fe700 1 --2- 172.21.15.12:0/3714057462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3981019a0 0x7fe398105930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:49.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.940+0000 7fe3977fe700 1 -- 172.21.15.12:0/3714057462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe398103540 msgr2=0x7fe3981044c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:49.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.940+0000 7fe3977fe700 1 --2- 172.21.15.12:0/3714057462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe398103540 0x7fe3981044c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:49.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.940+0000 7fe3977fe700 1 -- 172.21.15.12:0/3714057462 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe398104d70 con 0x7fe398102ba0 2024-04-22T13:23:49.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.940+0000 7fe39cd31700 1 --2- 172.21.15.12:0/3714057462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe398103540 0x7fe3981044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:23:49.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.941+0000 7fe3977fe700 1 --2- 172.21.15.12:0/3714057462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe398102ba0 0x7fe398103f80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe38000b530 tx=0x7fe38000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:49.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.941+0000 7fe3957fa700 1 -- 172.21.15.12:0/3714057462 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe380015020 con 0x7fe398102ba0 2024-04-22T13:23:49.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.941+0000 7fe3957fa700 1 -- 172.21.15.12:0/3714057462 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe380011ce0 con 0x7fe398102ba0 2024-04-22T13:23:49.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.941+0000 7fe39e794700 1 -- 172.21.15.12:0/3714057462 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3981a2d50 con 0x7fe398102ba0 2024-04-22T13:23:49.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.941+0000 7fe3957fa700 1 -- 172.21.15.12:0/3714057462 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe380010d20 con 0x7fe398102ba0 2024-04-22T13:23:49.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.941+0000 7fe39e794700 1 -- 172.21.15.12:0/3714057462 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3981a32b0 con 0x7fe398102ba0 2024-04-22T13:23:49.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.942+0000 7fe39e794700 1 -- 172.21.15.12:0/3714057462 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe398069760 con 0x7fe398102ba0 2024-04-22T13:23:49.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.943+0000 7fe3957fa700 1 -- 172.21.15.12:0/3714057462 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe380018020 con 0x7fe398102ba0 2024-04-22T13:23:49.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.944+0000 7fe3957fa700 1 --2- 172.21.15.12:0/3714057462 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe3840a3cc0 0x7fe3840a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:49.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.944+0000 7fe3957fa700 1 -- 172.21.15.12:0/3714057462 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fe3800d0110 con 0x7fe398102ba0 2024-04-22T13:23:49.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.944+0000 7fe397fff700 1 --2- 172.21.15.12:0/3714057462 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe3840a3cc0 0x7fe3840a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:49.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.944+0000 7fe397fff700 1 --2- 172.21.15.12:0/3714057462 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe3840a3cc0 0x7fe3840a6180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe388005fd0 tx=0x7fe388005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:49.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:49.947+0000 7fe3957fa700 1 -- 172.21.15.12:0/3714057462 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe398069760 con 0x7fe398102ba0 2024-04-22T13:23:49.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:49 smithi111 ceph-mon[31361]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-22T13:23:50.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:49 smithi134 ceph-mon[31338]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-22T13:23:50.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:50.202+0000 7fe39e794700 1 -- 172.21.15.12:0/3714057462 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fe3981a35f0 con 0x7fe398102ba0 2024-04-22T13:23:50.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:50.203+0000 7fe3957fa700 1 -- 172.21.15.12:0/3714057462 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3097 (secure 0 0 0) 0x7fe38001f020 con 0x7fe398102ba0 2024-04-22T13:23:50.200 INFO:teuthology.orchestra.run.smithi012.stdout:e27 2024-04-22T13:23:50.200 INFO:teuthology.orchestra.run.smithi012.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-22T13:23:50.200 INFO:teuthology.orchestra.run.smithi012.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-22T13:23:50.200 INFO:teuthology.orchestra.run.smithi012.stdout:legacy client fscid: 1 2024-04-22T13:23:50.201 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:23:50.201 INFO:teuthology.orchestra.run.smithi012.stdout:Filesystem 'cephfs' (1) 2024-04-22T13:23:50.201 INFO:teuthology.orchestra.run.smithi012.stdout:fs_name cephfs 2024-04-22T13:23:50.201 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 27 2024-04-22T13:23:50.201 INFO:teuthology.orchestra.run.smithi012.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-22T13:23:50.201 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-04-22T13:22:04.602553+0000 2024-04-22T13:23:50.201 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-04-22T13:22:12.398056+0000 2024-04-22T13:23:50.201 INFO:teuthology.orchestra.run.smithi012.stdout:tableserver 0 2024-04-22T13:23:50.201 INFO:teuthology.orchestra.run.smithi012.stdout:root 0 2024-04-22T13:23:50.201 INFO:teuthology.orchestra.run.smithi012.stdout:session_timeout 300 2024-04-22T13:23:50.201 INFO:teuthology.orchestra.run.smithi012.stdout:session_autoclose 300 2024-04-22T13:23:50.202 INFO:teuthology.orchestra.run.smithi012.stdout:max_file_size 1099511627776 2024-04-22T13:23:50.202 INFO:teuthology.orchestra.run.smithi012.stdout:max_xattr_size 65536 2024-04-22T13:23:50.202 INFO:teuthology.orchestra.run.smithi012.stdout:required_client_features {} 2024-04-22T13:23:50.202 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure 0 2024-04-22T13:23:50.202 INFO:teuthology.orchestra.run.smithi012.stdout:last_failure_osd_epoch 0 2024-04-22T13:23:50.202 INFO:teuthology.orchestra.run.smithi012.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-22T13:23:50.202 INFO:teuthology.orchestra.run.smithi012.stdout:max_mds 5 2024-04-22T13:23:50.202 INFO:teuthology.orchestra.run.smithi012.stdout:in 0,1,2,3,4 2024-04-22T13:23:50.202 INFO:teuthology.orchestra.run.smithi012.stdout:up {0=24509,1=24484,2=24448,3=24443,4=24407} 2024-04-22T13:23:50.202 INFO:teuthology.orchestra.run.smithi012.stdout:failed 2024-04-22T13:23:50.202 INFO:teuthology.orchestra.run.smithi012.stdout:damaged 2024-04-22T13:23:50.202 INFO:teuthology.orchestra.run.smithi012.stdout:stopped 2024-04-22T13:23:50.203 INFO:teuthology.orchestra.run.smithi012.stdout:data_pools [3] 2024-04-22T13:23:50.203 INFO:teuthology.orchestra.run.smithi012.stdout:metadata_pool 2 2024-04-22T13:23:50.203 INFO:teuthology.orchestra.run.smithi012.stdout:inline_data disabled 2024-04-22T13:23:50.203 INFO:teuthology.orchestra.run.smithi012.stdout:balancer 2024-04-22T13:23:50.203 INFO:teuthology.orchestra.run.smithi012.stdout:bal_rank_mask -1 2024-04-22T13:23:50.203 INFO:teuthology.orchestra.run.smithi012.stdout:standby_count_wanted 0 2024-04-22T13:23:50.203 INFO:teuthology.orchestra.run.smithi012.stdout:qdb_cluster leader: 24509 members: 24448,24509,24443,24484,24407 2024-04-22T13:23:50.203 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.b{0:24509} state up:active seq 3 addr [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T13:23:50.203 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.h{0:24470} state up:standby-replay seq 1 addr [v2:172.21.15.111:6836/4218407470,v1:172.21.15.111:6837/4218407470] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T13:23:50.203 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.i{1:24484} state up:active seq 6 addr [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T13:23:50.203 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.c{1:24460} state up:standby-replay seq 1 addr [v2:172.21.15.134:6836/3631577545,v1:172.21.15.134:6837/3631577545] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T13:23:50.203 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.f{2:24448} state up:active seq 8 addr [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T13:23:50.204 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.l{2:24406} state up:standby-replay seq 1 addr [v2:172.21.15.134:6832/4235830781,v1:172.21.15.134:6833/4235830781] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T13:23:50.204 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.e{3:24443} state up:active seq 9 addr [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T13:23:50.204 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.a{3:14643} state up:standby-replay seq 1 addr [v2:172.21.15.12:6840/763294965,v1:172.21.15.12:6841/763294965] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T13:23:50.204 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.k{4:24407} state up:active seq 11 addr [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T13:23:50.204 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.g{4:14631} state up:standby-replay seq 1 addr [v2:172.21.15.12:6838/4083188176,v1:172.21.15.12:6839/4083188176] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T13:23:50.204 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:23:50.204 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:23:50.204 INFO:teuthology.orchestra.run.smithi012.stdout:Standby daemons: 2024-04-22T13:23:50.204 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:23:50.204 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.j{-1:14547} state up:standby seq 1 addr [v2:172.21.15.12:6834/1590002838,v1:172.21.15.12:6835/1590002838] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T13:23:50.204 INFO:teuthology.orchestra.run.smithi012.stdout:[mds.d{-1:14565} state up:standby seq 1 addr [v2:172.21.15.12:6836/3677153272,v1:172.21.15.12:6837/3677153272] compat {c=[1],r=[1],i=[1fff]}] 2024-04-22T13:23:50.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:50.205+0000 7fe39e794700 1 -- 172.21.15.12:0/3714057462 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe3840a3cc0 msgr2=0x7fe3840a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:50.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:50.205+0000 7fe39e794700 1 --2- 172.21.15.12:0/3714057462 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe3840a3cc0 0x7fe3840a6180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe388005fd0 tx=0x7fe388005e50 comp rx=0 tx=0).stop 2024-04-22T13:23:50.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:50.205+0000 7fe39e794700 1 -- 172.21.15.12:0/3714057462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe398102ba0 msgr2=0x7fe398103f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:50.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:50.205+0000 7fe39e794700 1 --2- 172.21.15.12:0/3714057462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe398102ba0 0x7fe398103f80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe38000b530 tx=0x7fe38000b8f0 comp rx=0 tx=0).stop 2024-04-22T13:23:50.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:50.205+0000 7fe39e794700 1 -- 172.21.15.12:0/3714057462 shutdown_connections 2024-04-22T13:23:50.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:50.205+0000 7fe39e794700 1 --2- 172.21.15.12:0/3714057462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe398103540 0x7fe3981044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:50.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:50.205+0000 7fe39e794700 1 --2- 172.21.15.12:0/3714057462 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe3840a3cc0 0x7fe3840a6180 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:50.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:50.205+0000 7fe39e794700 1 --2- 172.21.15.12:0/3714057462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3981019a0 0x7fe398105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:50.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:50.205+0000 7fe39e794700 1 --2- 172.21.15.12:0/3714057462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe398102ba0 0x7fe398103f80 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:50.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:50.205+0000 7fe39e794700 1 -- 172.21.15.12:0/3714057462 >> 172.21.15.12:0/3714057462 conn(0x7fe3980fcf40 msgr2=0x7fe3980fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:50.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:50.205+0000 7fe39e794700 1 -- 172.21.15.12:0/3714057462 shutdown_connections 2024-04-22T13:23:50.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:50.205+0000 7fe39e794700 1 -- 172.21.15.12:0/3714057462 wait complete. 2024-04-22T13:23:50.206 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 27 2024-04-22T13:23:50.832 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:50 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3714057462' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T13:23:50.861 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-22T13:23:50.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:50 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/3714057462' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T13:23:51.109 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:23:51.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:50 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/3714057462' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-22T13:23:51.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:51 smithi111 ceph-mon[31361]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-22T13:23:52.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:51 smithi012 ceph-mon[28265]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-22T13:23:52.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:51 smithi134 ceph-mon[31338]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-22T13:23:52.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.897+0000 7f06647cd700 1 -- 172.21.15.12:0/1413489014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f065c1019a0 msgr2=0x7f065c101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:52.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.897+0000 7f06647cd700 1 --2- 172.21.15.12:0/1413489014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f065c1019a0 0x7f065c101dc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f064c00b0d0 tx=0x7f064c00b3e0 comp rx=0 tx=0).stop 2024-04-22T13:23:52.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.897+0000 7f06647cd700 1 -- 172.21.15.12:0/1413489014 shutdown_connections 2024-04-22T13:23:52.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.897+0000 7f06647cd700 1 --2- 172.21.15.12:0/1413489014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f065c103540 0x7f065c109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:52.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.897+0000 7f06647cd700 1 --2- 172.21.15.12:0/1413489014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f065c1019a0 0x7f065c101dc0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:52.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.897+0000 7f06647cd700 1 --2- 172.21.15.12:0/1413489014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f065c102ba0 0x7f065c103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:52.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.897+0000 7f06647cd700 1 -- 172.21.15.12:0/1413489014 >> 172.21.15.12:0/1413489014 conn(0x7f065c0fcf40 msgr2=0x7f065c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:52.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.897+0000 7f06647cd700 1 -- 172.21.15.12:0/1413489014 shutdown_connections 2024-04-22T13:23:52.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.897+0000 7f06647cd700 1 -- 172.21.15.12:0/1413489014 wait complete. 2024-04-22T13:23:52.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.898+0000 7f06647cd700 1 Processor -- start 2024-04-22T13:23:52.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.898+0000 7f06647cd700 1 -- start start 2024-04-22T13:23:52.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.898+0000 7f06647cd700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f065c1019a0 0x7f065c198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:52.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.898+0000 7f06647cd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f065c102ba0 0x7f065c1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:52.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.898+0000 7f06647cd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f065c103540 0x7f065c1a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:52.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.898+0000 7f06647cd700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f065c10d4e0 con 0x7f065c103540 2024-04-22T13:23:52.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.898+0000 7f06647cd700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f065c10d360 con 0x7f065c102ba0 2024-04-22T13:23:52.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.898+0000 7f06647cd700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f065c10d660 con 0x7f065c1019a0 2024-04-22T13:23:52.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.898+0000 7f0661d68700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f065c102ba0 0x7f065c1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:52.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.898+0000 7f0662d6a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f065c103540 0x7f065c1a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:52.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.898+0000 7f0662569700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f065c1019a0 0x7f065c198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:52.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.898+0000 7f0661d68700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f065c102ba0 0x7f065c1991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:43748/0 (socket says 172.21.15.12:43748) 2024-04-22T13:23:52.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.898+0000 7f0662d6a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f065c103540 0x7f065c1a0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59618/0 (socket says 172.21.15.12:59618) 2024-04-22T13:23:52.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.898+0000 7f0661d68700 1 -- 172.21.15.12:0/1236507917 learned_addr learned my addr 172.21.15.12:0/1236507917 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:52.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.899+0000 7f0662569700 1 -- 172.21.15.12:0/1236507917 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f065c102ba0 msgr2=0x7f065c1991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:52.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.899+0000 7f0662569700 1 --2- 172.21.15.12:0/1236507917 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f065c102ba0 0x7f065c1991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:52.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.899+0000 7f0662569700 1 -- 172.21.15.12:0/1236507917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f065c103540 msgr2=0x7f065c1a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:52.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.899+0000 7f0662569700 1 --2- 172.21.15.12:0/1236507917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f065c103540 0x7f065c1a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:52.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.899+0000 7f0662569700 1 -- 172.21.15.12:0/1236507917 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065c1a0910 con 0x7f065c1019a0 2024-04-22T13:23:52.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.899+0000 7f0662d6a700 1 --2- 172.21.15.12:0/1236507917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f065c103540 0x7f065c1a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:23:52.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.899+0000 7f0662569700 1 --2- 172.21.15.12:0/1236507917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f065c1019a0 0x7f065c198c90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f064c00d010 tx=0x7f064c007a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:52.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.899+0000 7f065b7fe700 1 -- 172.21.15.12:0/1236507917 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064c003c00 con 0x7f065c1019a0 2024-04-22T13:23:52.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.899+0000 7f06647cd700 1 -- 172.21.15.12:0/1236507917 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f065c1a0ba0 con 0x7f065c1019a0 2024-04-22T13:23:52.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.899+0000 7f065b7fe700 1 -- 172.21.15.12:0/1236507917 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f064c003da0 con 0x7f065c1019a0 2024-04-22T13:23:52.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.899+0000 7f065b7fe700 1 -- 172.21.15.12:0/1236507917 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064c01b8b0 con 0x7f065c1019a0 2024-04-22T13:23:52.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.899+0000 7f06647cd700 1 -- 172.21.15.12:0/1236507917 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f065c1a1050 con 0x7f065c1019a0 2024-04-22T13:23:52.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.900+0000 7f06647cd700 1 -- 172.21.15.12:0/1236507917 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f065c069730 con 0x7f065c1019a0 2024-04-22T13:23:52.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.901+0000 7f065b7fe700 1 -- 172.21.15.12:0/1236507917 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f064c0055b0 con 0x7f065c1019a0 2024-04-22T13:23:52.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.902+0000 7f065b7fe700 1 --2- 172.21.15.12:0/1236507917 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f06440a3c40 0x7f06440a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:52.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.902+0000 7f0661d68700 1 --2- 172.21.15.12:0/1236507917 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f06440a3c40 0x7f06440a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:52.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.902+0000 7f065b7fe700 1 -- 172.21.15.12:0/1236507917 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f064c00f020 con 0x7f065c1019a0 2024-04-22T13:23:52.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.903+0000 7f0661d68700 1 --2- 172.21.15.12:0/1236507917 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f06440a3c40 0x7f06440a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f065c19a1a0 tx=0x7f0650005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:52.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:52.906+0000 7f065b7fe700 1 -- 172.21.15.12:0/1236507917 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f064c09e220 con 0x7f065c1019a0 2024-04-22T13:23:53.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:53.118+0000 7f06647cd700 1 -- 172.21.15.12:0/1236507917 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f065c1a1420 con 0x7f065c1019a0 2024-04-22T13:23:53.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:53.120+0000 7f065b7fe700 1 -- 172.21.15.12:0/1236507917 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v89) ==== 56+0+5264 (secure 0 0 0) 0x7f064c09db20 con 0x7f065c1019a0 2024-04-22T13:23:53.117 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 89 2024-04-22T13:23:53.117 INFO:teuthology.orchestra.run.smithi012.stdout:fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:23:53.117 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-04-22T13:14:09.362918+0000 2024-04-22T13:23:53.117 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-04-22T13:23:05.500756+0000 2024-04-22T13:23:53.117 INFO:teuthology.orchestra.run.smithi012.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-22T13:23:53.117 INFO:teuthology.orchestra.run.smithi012.stdout:crush_version 26 2024-04-22T13:23:53.117 INFO:teuthology.orchestra.run.smithi012.stdout:full_ratio 0.95 2024-04-22T13:23:53.117 INFO:teuthology.orchestra.run.smithi012.stdout:backfillfull_ratio 0.9 2024-04-22T13:23:53.118 INFO:teuthology.orchestra.run.smithi012.stdout:nearfull_ratio 0.85 2024-04-22T13:23:53.118 INFO:teuthology.orchestra.run.smithi012.stdout:require_min_compat_client luminous 2024-04-22T13:23:53.118 INFO:teuthology.orchestra.run.smithi012.stdout:min_compat_client luminous 2024-04-22T13:23:53.118 INFO:teuthology.orchestra.run.smithi012.stdout:require_osd_release squid 2024-04-22T13:23:53.118 INFO:teuthology.orchestra.run.smithi012.stdout:stretch_mode_enabled false 2024-04-22T13:23:53.118 INFO:teuthology.orchestra.run.smithi012.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-22T13:23:53.118 INFO:teuthology.orchestra.run.smithi012.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-22T13:23:53.118 INFO:teuthology.orchestra.run.smithi012.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 87 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-22T13:23:53.118 INFO:teuthology.orchestra.run.smithi012.stdout:max_osd 12 2024-04-22T13:23:53.118 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 up in weight 1 up_from 8 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6802/1191350632,v1:172.21.15.12:6803/1191350632] [v2:172.21.15.12:6804/1191350632,v1:172.21.15.12:6805/1191350632] exists,up 6d9e2365-3a64-4b13-96fc-c217b1cee2ef 2024-04-22T13:23:53.118 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6800/417496465,v1:172.21.15.111:6801/417496465] [v2:172.21.15.111:6802/417496465,v1:172.21.15.111:6803/417496465] exists,up 1499c783-2bd7-48ce-a444-34add42bcf31 2024-04-22T13:23:53.118 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 up in weight 1 up_from 18 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6800/1403153998,v1:172.21.15.134:6801/1403153998] [v2:172.21.15.134:6802/1403153998,v1:172.21.15.134:6803/1403153998] exists,up 71a9ebc3-c622-434e-a6c2-f48358f384f5 2024-04-22T13:23:53.118 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 up in weight 1 up_from 27 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6810/3359066681,v1:172.21.15.12:6811/3359066681] [v2:172.21.15.12:6812/3359066681,v1:172.21.15.12:6813/3359066681] exists,up 6dd00db2-a286-48ba-8666-56ffdd5f35ba 2024-04-22T13:23:53.118 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 up in weight 1 up_from 35 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6808/2886349872,v1:172.21.15.111:6809/2886349872] [v2:172.21.15.111:6810/2886349872,v1:172.21.15.111:6811/2886349872] exists,up 1e29afd7-354c-4506-aaa0-b43eb1a19a83 2024-04-22T13:23:53.119 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 up in weight 1 up_from 40 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6808/3071977602,v1:172.21.15.134:6809/3071977602] [v2:172.21.15.134:6810/3071977602,v1:172.21.15.134:6811/3071977602] exists,up 8f40f02d-0ba2-4842-a03d-2a1a04d1010d 2024-04-22T13:23:53.119 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6 up in weight 1 up_from 48 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6818/3819619299,v1:172.21.15.12:6819/3819619299] [v2:172.21.15.12:6820/3819619299,v1:172.21.15.12:6821/3819619299] exists,up bb0c17a9-6a8e-4be6-8660-f73cfa8200ca 2024-04-22T13:23:53.119 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7 up in weight 1 up_from 54 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6816/3113856282,v1:172.21.15.111:6817/3113856282] [v2:172.21.15.111:6818/3113856282,v1:172.21.15.111:6819/3113856282] exists,up 0a93d6af-569e-4e45-b697-9e978c1c1004 2024-04-22T13:23:53.119 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8 up in weight 1 up_from 61 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6816/1689362191,v1:172.21.15.134:6817/1689362191] [v2:172.21.15.134:6818/1689362191,v1:172.21.15.134:6819/1689362191] exists,up e21acc68-cf50-4f4f-b1ba-cf6c195b4955 2024-04-22T13:23:53.119 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9 up in weight 1 up_from 69 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6826/4292385779,v1:172.21.15.12:6827/4292385779] [v2:172.21.15.12:6828/4292385779,v1:172.21.15.12:6829/4292385779] exists,up fd745869-7f5f-48a8-b412-bb9c92b586e0 2024-04-22T13:23:53.119 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10 up in weight 1 up_from 74 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6824/3338566976,v1:172.21.15.111:6825/3338566976] [v2:172.21.15.111:6826/3338566976,v1:172.21.15.111:6827/3338566976] exists,up 798b1abd-f6c7-4a5f-8da0-fdae49e50d2c 2024-04-22T13:23:53.119 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11 up in weight 1 up_from 82 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.134:6824/578640428,v1:172.21.15.134:6825/578640428] [v2:172.21.15.134:6826/578640428,v1:172.21.15.134:6827/578640428] exists,up b18fc5f3-1049-4871-b060-e99290273cfb 2024-04-22T13:23:53.119 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 2.8 [2,11] 2024-04-22T13:23:53.119 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 2.b [8,11] 2024-04-22T13:23:53.119 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.1e [3,9] 2024-04-22T13:23:53.119 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.20 [5,9] 2024-04-22T13:23:53.119 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.21 [4,6] 2024-04-22T13:23:53.119 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.34 [4,9] 2024-04-22T13:23:53.119 INFO:teuthology.orchestra.run.smithi012.stdout:pg_upmap_items 3.3a [4,8] 2024-04-22T13:23:53.119 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/2593601362 expires 2024-04-23T13:15:04.279990+0000 2024-04-22T13:23:53.119 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/2519382069 expires 2024-04-23T13:15:04.279990+0000 2024-04-22T13:23:53.119 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:6801/2009831940 expires 2024-04-23T13:15:04.279990+0000 2024-04-22T13:23:53.120 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/3509412187 expires 2024-04-23T13:14:36.603653+0000 2024-04-22T13:23:53.120 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/1007749062 expires 2024-04-23T13:15:04.279990+0000 2024-04-22T13:23:53.120 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/3588480095 expires 2024-04-23T13:14:36.603653+0000 2024-04-22T13:23:53.120 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:6800/2091570832 expires 2024-04-23T13:14:36.603653+0000 2024-04-22T13:23:53.120 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:0/1904962176 expires 2024-04-23T13:14:36.603653+0000 2024-04-22T13:23:53.120 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:6801/2091570832 expires 2024-04-23T13:14:36.603653+0000 2024-04-22T13:23:53.120 INFO:teuthology.orchestra.run.smithi012.stdout:blocklist 172.21.15.12:6800/2009831940 expires 2024-04-23T13:15:04.279990+0000 2024-04-22T13:23:53.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:53.122+0000 7f06647cd700 1 -- 172.21.15.12:0/1236507917 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f06440a3c40 msgr2=0x7f06440a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:53.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:53.122+0000 7f06647cd700 1 --2- 172.21.15.12:0/1236507917 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f06440a3c40 0x7f06440a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f065c19a1a0 tx=0x7f0650005f90 comp rx=0 tx=0).stop 2024-04-22T13:23:53.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:53.122+0000 7f06647cd700 1 -- 172.21.15.12:0/1236507917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f065c1019a0 msgr2=0x7f065c198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:53.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:53.122+0000 7f06647cd700 1 --2- 172.21.15.12:0/1236507917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f065c1019a0 0x7f065c198c90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f064c00d010 tx=0x7f064c007a90 comp rx=0 tx=0).stop 2024-04-22T13:23:53.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:53.122+0000 7f06647cd700 1 -- 172.21.15.12:0/1236507917 shutdown_connections 2024-04-22T13:23:53.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:53.122+0000 7f06647cd700 1 --2- 172.21.15.12:0/1236507917 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f06440a3c40 0x7f06440a6100 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:53.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:53.122+0000 7f06647cd700 1 --2- 172.21.15.12:0/1236507917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f065c103540 0x7f065c1a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:53.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:53.122+0000 7f06647cd700 1 --2- 172.21.15.12:0/1236507917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f065c1019a0 0x7f065c198c90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:53.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:53.122+0000 7f06647cd700 1 --2- 172.21.15.12:0/1236507917 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f065c102ba0 0x7f065c1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:53.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:53.122+0000 7f06647cd700 1 -- 172.21.15.12:0/1236507917 >> 172.21.15.12:0/1236507917 conn(0x7f065c0fcf40 msgr2=0x7f065c0fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:53.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:53.122+0000 7f06647cd700 1 -- 172.21.15.12:0/1236507917 shutdown_connections 2024-04-22T13:23:53.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:53.122+0000 7f06647cd700 1 -- 172.21.15.12:0/1236507917 wait complete. 2024-04-22T13:23:53.790 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:53 smithi012 ceph-mon[28265]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-22T13:23:53.790 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:53 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1236507917' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-22T13:23:53.819 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-22T13:23:53.908 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:23:53.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:53 smithi111 ceph-mon[31361]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-22T13:23:53.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:53 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1236507917' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-22T13:23:54.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.017+0000 7fd609843640 1 -- 172.21.15.12:0/3810417877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd604105370 msgr2=0x7fd604107760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:54.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.017+0000 7fd609843640 1 --2- 172.21.15.12:0/3810417877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd604105370 0x7fd604107760 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009900 tx=0x7fd5f802bc30 comp rx=0 tx=0).stop 2024-04-22T13:23:54.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.017+0000 7fd609843640 1 -- 172.21.15.12:0/3810417877 shutdown_connections 2024-04-22T13:23:54.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.017+0000 7fd609843640 1 --2- 172.21.15.12:0/3810417877 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd604107ca0 0x7fd60410a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.017+0000 7fd609843640 1 --2- 172.21.15.12:0/3810417877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd604105370 0x7fd604107760 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.017+0000 7fd609843640 1 --2- 172.21.15.12:0/3810417877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd604102a40 0x7fd604104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.017+0000 7fd609843640 1 -- 172.21.15.12:0/3810417877 >> 172.21.15.12:0/3810417877 conn(0x7fd6040fc470 msgr2=0x7fd6040fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:54.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.017+0000 7fd609843640 1 -- 172.21.15.12:0/3810417877 shutdown_connections 2024-04-22T13:23:54.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.017+0000 7fd609843640 1 -- 172.21.15.12:0/3810417877 wait complete. 2024-04-22T13:23:54.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.017+0000 7fd609843640 1 Processor -- start 2024-04-22T13:23:54.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.017+0000 7fd609843640 1 -- start start 2024-04-22T13:23:54.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.018+0000 7fd609843640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd604102a40 0x7fd60419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:54.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.018+0000 7fd609843640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd604105370 0x7fd60419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:54.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.018+0000 7fd609843640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd604107ca0 0x7fd6041a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:54.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.018+0000 7fd609843640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd604101860 con 0x7fd604107ca0 2024-04-22T13:23:54.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.018+0000 7fd609843640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd6041016e0 con 0x7fd604102a40 2024-04-22T13:23:54.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.018+0000 7fd609843640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd6041019e0 con 0x7fd604105370 2024-04-22T13:23:54.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.018+0000 7fd6037fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd604107ca0 0x7fd6041a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:54.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.018+0000 7fd6037fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd604107ca0 0x7fd6041a19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59640/0 (socket says 172.21.15.12:59640) 2024-04-22T13:23:54.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.018+0000 7fd6037fe640 1 -- 172.21.15.12:0/3443397500 learned_addr learned my addr 172.21.15.12:0/3443397500 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:54.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.018+0000 7fd6027fc640 1 --2- 172.21.15.12:0/3443397500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd604105370 0x7fd60419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:54.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.018+0000 7fd602ffd640 1 --2- 172.21.15.12:0/3443397500 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd604102a40 0x7fd60419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:54.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.018+0000 7fd6037fe640 1 -- 172.21.15.12:0/3443397500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd604105370 msgr2=0x7fd60419a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:54.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.018+0000 7fd6037fe640 1 --2- 172.21.15.12:0/3443397500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd604105370 0x7fd60419a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.018+0000 7fd6037fe640 1 -- 172.21.15.12:0/3443397500 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd604102a40 msgr2=0x7fd60419a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:54.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.018+0000 7fd6037fe640 1 --2- 172.21.15.12:0/3443397500 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd604102a40 0x7fd60419a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.018+0000 7fd6037fe640 1 -- 172.21.15.12:0/3443397500 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6041a2020 con 0x7fd604107ca0 2024-04-22T13:23:54.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.018+0000 7fd6037fe640 1 --2- 172.21.15.12:0/3443397500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd604107ca0 0x7fd6041a19b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc009fd0 tx=0x7fd5fc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:54.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.018+0000 7fd608841640 1 -- 172.21.15.12:0/3443397500 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc017070 con 0x7fd604107ca0 2024-04-22T13:23:54.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.018+0000 7fd609843640 1 -- 172.21.15.12:0/3443397500 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6041a2310 con 0x7fd604107ca0 2024-04-22T13:23:54.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.018+0000 7fd608841640 1 -- 172.21.15.12:0/3443397500 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5fc003e20 con 0x7fd604107ca0 2024-04-22T13:23:54.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.018+0000 7fd608841640 1 -- 172.21.15.12:0/3443397500 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc010460 con 0x7fd604107ca0 2024-04-22T13:23:54.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.018+0000 7fd609843640 1 -- 172.21.15.12:0/3443397500 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6041a27e0 con 0x7fd604107ca0 2024-04-22T13:23:54.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.019+0000 7fd609843640 1 -- 172.21.15.12:0/3443397500 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6041a2a40 con 0x7fd604107ca0 2024-04-22T13:23:54.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.021+0000 7fd608841640 1 -- 172.21.15.12:0/3443397500 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd5fc003970 con 0x7fd604107ca0 2024-04-22T13:23:54.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.021+0000 7fd608841640 1 --2- 172.21.15.12:0/3443397500 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fd5ec0a3bb0 0x7fd5ec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:54.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.022+0000 7fd602ffd640 1 --2- 172.21.15.12:0/3443397500 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fd5ec0a3bb0 0x7fd5ec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:54.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.022+0000 7fd608841640 1 -- 172.21.15.12:0/3443397500 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fd5fc0d0240 con 0x7fd604107ca0 2024-04-22T13:23:54.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.022+0000 7fd602ffd640 1 --2- 172.21.15.12:0/3443397500 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fd5ec0a3bb0 0x7fd5ec0a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4005fd0 tx=0x7fd5f4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:54.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.026+0000 7fd608841640 1 -- 172.21.15.12:0/3443397500 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd5fc09cc50 con 0x7fd604107ca0 2024-04-22T13:23:54.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:53 smithi134 ceph-mon[31338]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-22T13:23:54.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:53 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1236507917' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-22T13:23:54.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.229+0000 7fd609843640 1 -- 172.21.15.12:0/3443397500 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6041a2d30 con 0x7fd604107ca0 2024-04-22T13:23:54.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.230+0000 7fd608841640 1 -- 172.21.15.12:0/3443397500 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22620 (secure 0 0 0) 0x7fd5fc09c550 con 0x7fd604107ca0 2024-04-22T13:23:54.228 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:23:54.228 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"0aede5ea-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:14:09.362918+0000","modified":"2024-04-22T13:23:05.500756+0000","last_up_change":"2024-04-22T13:21:15.263393+0000","last_in_change":"2024-04-22T13:20:54.174685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:17:32.372434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:22:02.768799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:22:03.578730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d9e2365-3a64-4b13-96fc-c217b1cee2ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6803","nonce":1191350632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6805","nonce":1191350632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6809","nonce":1191350632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6807","nonce":1191350632}]},"public_addr":"172.21.15.12:6803/1191350632","cluster_addr":"172.21.15.12:6805/1191350632","heartbeat_back_addr":"172.21.15.12:6809/1191350632","heartbeat_front_addr":"172.21.15.12:6807/1191350632","state":["exists","up"]},{"osd":1,"uuid":"1499c783-2bd7-48ce-a444-34add42bcf31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6801","nonce":417496465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6803","nonce":417496465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6807","nonce":417496465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6805","nonce":417496465}]},"public_addr":"172.21.15.111:6801/417496465","cluster_addr":"172.21.15.111:6803/417496465","heartbeat_back_addr":"172.21.15.111:6807/417496465","heartbeat_front_addr":"172.21.15.111:6805/417496465","state":["exists","up"]},{"osd":2,"uuid":"71a9ebc3-c622-434e-a6c2-f48358f384f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6801","nonce":1403153998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6803","nonce":1403153998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6807","nonce":1403153998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6805","nonce":1403153998}]},"public_addr":"172.21.15.134:6801/1403153998","cluster_addr":"172.21.15.134:6803/1403153998","heartbeat_back_addr":"172.21.15.134:6807/1403153998","heartbeat_front_addr":"172.21.15.134:6805/1403153998","state":["exists","up"]},{"osd":3,"uuid":"6dd00db2-a286-48ba-8666-56ffdd5f35ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6811","nonce":3359066681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6813","nonce":3359066681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6817","nonce":3359066681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6815","nonce":3359066681}]},"public_addr":"172.21.15.12:6811/3359066681","cluster_addr":"172.21.15.12:6813/3359066681","heartbeat_back_addr":"172.21.15.12:6817/3359066681","heartbeat_front_addr":"172.21.15.12:6815/3359066681","state":["exists","up"]},{"osd":4,"uuid":"1e29afd7-354c-4506-aaa0-b43eb1a19a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6809","nonce":2886349872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6811","nonce":2886349872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6815","nonce":2886349872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6813","nonce":2886349872}]},"public_addr":"172.21.15.111:6809/2886349872","cluster_addr":"172.21.15.111:6811/2886349872","heartbeat_back_addr":"172.21.15.111:6815/2886349872","heartbeat_front_addr":"172.21.15.111:6813/2886349872","state":["exists","up"]},{"osd":5,"uuid":"8f40f02d-0ba2-4842-a03d-2a1a04d1010d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6809","nonce":3071977602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6811","nonce":3071977602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6815","nonce":3071977602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6813","nonce":3071977602}]},"public_addr":"172.21.15.134:6809/3071977602","cluster_addr":"172.21.15.134:6811/3071977602","heartbeat_back_addr":"172.21.15.134:6815/3071977602","heartbeat_front_addr":"172.21.15.134:6813/3071977602","state":["exists","up"]},{"osd":6,"uuid":"bb0c17a9-6a8e-4be6-8660-f73cfa8200ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6819","nonce":3819619299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6821","nonce":3819619299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6825","nonce":3819619299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6823","nonce":3819619299}]},"public_addr":"172.21.15.12:6819/3819619299","cluster_addr":"172.21.15.12:6821/3819619299","heartbeat_back_addr":"172.21.15.12:6825/3819619299","heartbeat_front_addr":"172.21.15.12:6823/3819619299","state":["exists","up"]},{"osd":7,"uuid":"0a93d6af-569e-4e45-b697-9e978c1c1004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6817","nonce":3113856282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6819","nonce":3113856282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6823","nonce":3113856282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6821","nonce":3113856282}]},"public_addr":"172.21.15.111:6817/3113856282","cluster_addr":"172.21.15.111:6819/3113856282","heartbeat_back_addr":"172.21.15.111:6823/3113856282","heartbeat_front_addr":"172.21.15.111:6821/3113856282","state":["exists","up"]},{"osd":8,"uuid":"e21acc68-cf50-4f4f-b1ba-cf6c195b4955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6817","nonce":1689362191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6819","nonce":1689362191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6823","nonce":1689362191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6821","nonce":1689362191}]},"public_addr":"172.21.15.134:6817/1689362191","cluster_addr":"172.21.15.134:6819/1689362191","heartbeat_back_addr":"172.21.15.134:6823/1689362191","heartbeat_front_addr":"172.21.15.134:6821/1689362191","state":["exists","up"]},{"osd":9,"uuid":"fd745869-7f5f-48a8-b412-bb9c92b586e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6827","nonce":4292385779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6829","nonce":4292385779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6833","nonce":4292385779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6831","nonce":4292385779}]},"public_addr":"172.21.15.12:6827/4292385779","cluster_addr":"172.21.15.12:6829/4292385779","heartbeat_back_addr":"172.21.15.12:6833/4292385779","heartbeat_front_addr":"172.21.15.12:6831/4292385779","state":["exists","up"]},{"osd":10,"uuid":"798b1abd-f6c7-4a5f-8da0-fdae49e50d2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6825","nonce":3338566976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6827","nonce":3338566976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6831","nonce":3338566976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6829","nonce":3338566976}]},"public_addr":"172.21.15.111:6825/3338566976","cluster_addr":"172.21.15.111:6827/3338566976","heartbeat_back_addr":"172.21.15.111:6831/3338566976","heartbeat_front_addr":"172.21.15.111:6829/3338566976","state":["exists","up"]},{"osd":11,"uuid":"b18fc5f3-1049-4871-b060-e99290273cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6825","nonce":578640428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6827","nonce":578640428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6831","nonce":578640428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6829","nonce":578640428}]},"public_addr":"172.21.15.134:6825/578640428","cluster_addr":"172.21.15.134:6827/578640428","heartbeat_back_addr":"172.21.15.134:6831/578640428","heartbeat_front_addr":"172.21.15.134:6829/578640428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:16:40.828608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:05.052528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:28.297316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:53.496497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:17.947259+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:41.712894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:07.105945+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:32.129688+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:56.008516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:22.239867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:47.334692+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:12.301276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/2593601362":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/2519382069":"2024-04-23T13:15:04.279990+0000","172.21.15.12:6801/2009831940":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3509412187":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1007749062":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3588480095":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1904962176":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2009831940":"2024-04-23T13:15:04.279990+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:23:54.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.232+0000 7fd609843640 1 -- 172.21.15.12:0/3443397500 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fd5ec0a3bb0 msgr2=0x7fd5ec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:54.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.232+0000 7fd609843640 1 --2- 172.21.15.12:0/3443397500 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fd5ec0a3bb0 0x7fd5ec0a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4005fd0 tx=0x7fd5f4005850 comp rx=0 tx=0).stop 2024-04-22T13:23:54.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.232+0000 7fd609843640 1 -- 172.21.15.12:0/3443397500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd604107ca0 msgr2=0x7fd6041a19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:54.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.232+0000 7fd609843640 1 --2- 172.21.15.12:0/3443397500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd604107ca0 0x7fd6041a19b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc009fd0 tx=0x7fd5fc00b260 comp rx=0 tx=0).stop 2024-04-22T13:23:54.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.232+0000 7fd609843640 1 -- 172.21.15.12:0/3443397500 shutdown_connections 2024-04-22T13:23:54.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.232+0000 7fd609843640 1 --2- 172.21.15.12:0/3443397500 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fd5ec0a3bb0 0x7fd5ec0a6070 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.232+0000 7fd609843640 1 --2- 172.21.15.12:0/3443397500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd604107ca0 0x7fd6041a19b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.232+0000 7fd609843640 1 --2- 172.21.15.12:0/3443397500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd604105370 0x7fd60419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.232+0000 7fd609843640 1 --2- 172.21.15.12:0/3443397500 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd604102a40 0x7fd60419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.232+0000 7fd609843640 1 -- 172.21.15.12:0/3443397500 >> 172.21.15.12:0/3443397500 conn(0x7fd6040fc470 msgr2=0x7fd6041080e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:54.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.233+0000 7fd609843640 1 -- 172.21.15.12:0/3443397500 shutdown_connections 2024-04-22T13:23:54.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.233+0000 7fd609843640 1 -- 172.21.15.12:0/3443397500 wait complete. 2024-04-22T13:23:54.241 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T13:17:32.372434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T13:22:02.768799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T13:22:03.578730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T13:23:54.242 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T13:23:54.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.350+0000 7f9b3570d640 1 -- 172.21.15.12:0/1030505114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b30102a00 msgr2=0x7f9b30109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:54.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.350+0000 7f9b3570d640 1 --2- 172.21.15.12:0/1030505114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b30102a00 0x7f9b30109320 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9b28009ca0 tx=0x7f9b2802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.350+0000 7f9b3570d640 1 -- 172.21.15.12:0/1030505114 shutdown_connections 2024-04-22T13:23:54.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.350+0000 7f9b3570d640 1 --2- 172.21.15.12:0/1030505114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b30102a00 0x7f9b30109320 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.350+0000 7f9b3570d640 1 --2- 172.21.15.12:0/1030505114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b30102040 0x7f9b301024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.350+0000 7f9b3570d640 1 --2- 172.21.15.12:0/1030505114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b30100e60 0x7f9b30101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.350+0000 7f9b3570d640 1 -- 172.21.15.12:0/1030505114 >> 172.21.15.12:0/1030505114 conn(0x7f9b300fc440 msgr2=0x7f9b300fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:54.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.350+0000 7f9b3570d640 1 -- 172.21.15.12:0/1030505114 shutdown_connections 2024-04-22T13:23:54.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.350+0000 7f9b3570d640 1 -- 172.21.15.12:0/1030505114 wait complete. 2024-04-22T13:23:54.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.350+0000 7f9b3570d640 1 Processor -- start 2024-04-22T13:23:54.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.351+0000 7f9b3570d640 1 -- start start 2024-04-22T13:23:54.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.351+0000 7f9b3570d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b30100e60 0x7f9b301970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:54.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.351+0000 7f9b3570d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b30102040 0x7f9b301975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:54.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.351+0000 7f9b3570d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b30102a00 0x7f9b3019e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:54.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.351+0000 7f9b3570d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3010ca00 con 0x7f9b30102040 2024-04-22T13:23:54.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.351+0000 7f9b3570d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3010c880 con 0x7f9b30100e60 2024-04-22T13:23:54.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.351+0000 7f9b3570d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3010cb80 con 0x7f9b30102a00 2024-04-22T13:23:54.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.351+0000 7f9b2e7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b30102040 0x7f9b301975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:54.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.351+0000 7f9b2e7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b30102040 0x7f9b301975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59656/0 (socket says 172.21.15.12:59656) 2024-04-22T13:23:54.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.351+0000 7f9b2e7fc640 1 -- 172.21.15.12:0/378792081 learned_addr learned my addr 172.21.15.12:0/378792081 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:54.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.351+0000 7f9b2effd640 1 --2- 172.21.15.12:0/378792081 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b30100e60 0x7f9b301970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:54.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.351+0000 7f9b2f7fe640 1 --2- 172.21.15.12:0/378792081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b30102a00 0x7f9b3019e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:54.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.351+0000 7f9b2e7fc640 1 -- 172.21.15.12:0/378792081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b30102a00 msgr2=0x7f9b3019e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:54.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.351+0000 7f9b2e7fc640 1 --2- 172.21.15.12:0/378792081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b30102a00 0x7f9b3019e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.351+0000 7f9b2e7fc640 1 -- 172.21.15.12:0/378792081 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b30100e60 msgr2=0x7f9b301970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:54.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.351+0000 7f9b2e7fc640 1 --2- 172.21.15.12:0/378792081 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b30100e60 0x7f9b301970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.351+0000 7f9b2e7fc640 1 -- 172.21.15.12:0/378792081 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b3019ecc0 con 0x7f9b30102040 2024-04-22T13:23:54.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.351+0000 7f9b2f7fe640 1 --2- 172.21.15.12:0/378792081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b30102a00 0x7f9b3019e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:23:54.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.351+0000 7f9b2e7fc640 1 --2- 172.21.15.12:0/378792081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b30102040 0x7f9b301975e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9b2400e9f0 tx=0x7f9b2400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:54.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.351+0000 7f9b17fff640 1 -- 172.21.15.12:0/378792081 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2400cd80 con 0x7f9b30102040 2024-04-22T13:23:54.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.352+0000 7f9b3570d640 1 -- 172.21.15.12:0/378792081 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b3019efb0 con 0x7f9b30102040 2024-04-22T13:23:54.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.352+0000 7f9b17fff640 1 -- 172.21.15.12:0/378792081 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b240054d0 con 0x7f9b30102040 2024-04-22T13:23:54.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.352+0000 7f9b17fff640 1 -- 172.21.15.12:0/378792081 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b24010690 con 0x7f9b30102040 2024-04-22T13:23:54.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.352+0000 7f9b3570d640 1 -- 172.21.15.12:0/378792081 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b3019f420 con 0x7f9b30102040 2024-04-22T13:23:54.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.352+0000 7f9b3570d640 1 -- 172.21.15.12:0/378792081 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b30108250 con 0x7f9b30102040 2024-04-22T13:23:54.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.354+0000 7f9b17fff640 1 -- 172.21.15.12:0/378792081 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9b24004e30 con 0x7f9b30102040 2024-04-22T13:23:54.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.354+0000 7f9b17fff640 1 --2- 172.21.15.12:0/378792081 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f9b180a3bb0 0x7f9b180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:54.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.354+0000 7f9b2effd640 1 --2- 172.21.15.12:0/378792081 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f9b180a3bb0 0x7f9b180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:54.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.354+0000 7f9b17fff640 1 -- 172.21.15.12:0/378792081 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f9b240d0a50 con 0x7f9b30102040 2024-04-22T13:23:54.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.355+0000 7f9b2effd640 1 --2- 172.21.15.12:0/378792081 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f9b180a3bb0 0x7f9b180a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9b30101bb0 tx=0x7f9b20005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:54.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.358+0000 7f9b17fff640 1 -- 172.21.15.12:0/378792081 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9b24053ff0 con 0x7f9b30102040 2024-04-22T13:23:54.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.549+0000 7f9b3570d640 1 -- 172.21.15.12:0/378792081 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9b3002d810 con 0x7f9b30102040 2024-04-22T13:23:54.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.549+0000 7f9b17fff640 1 -- 172.21.15.12:0/378792081 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f9b2409d460 con 0x7f9b30102040 2024-04-22T13:23:54.547 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-22T13:23:54.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.551+0000 7f9b3570d640 1 -- 172.21.15.12:0/378792081 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f9b180a3bb0 msgr2=0x7f9b180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:54.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.551+0000 7f9b3570d640 1 --2- 172.21.15.12:0/378792081 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f9b180a3bb0 0x7f9b180a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9b30101bb0 tx=0x7f9b20005f90 comp rx=0 tx=0).stop 2024-04-22T13:23:54.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.551+0000 7f9b3570d640 1 -- 172.21.15.12:0/378792081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b30102040 msgr2=0x7f9b301975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:54.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.551+0000 7f9b3570d640 1 --2- 172.21.15.12:0/378792081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b30102040 0x7f9b301975e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9b2400e9f0 tx=0x7f9b2400edf0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.551+0000 7f9b3570d640 1 -- 172.21.15.12:0/378792081 shutdown_connections 2024-04-22T13:23:54.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.551+0000 7f9b3570d640 1 --2- 172.21.15.12:0/378792081 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b30102a00 0x7f9b3019e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.551+0000 7f9b3570d640 1 --2- 172.21.15.12:0/378792081 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f9b180a3bb0 0x7f9b180a6070 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.551+0000 7f9b3570d640 1 --2- 172.21.15.12:0/378792081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9b30102040 0x7f9b301975e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.551+0000 7f9b3570d640 1 --2- 172.21.15.12:0/378792081 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b30100e60 0x7f9b301970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.551+0000 7f9b3570d640 1 -- 172.21.15.12:0/378792081 >> 172.21.15.12:0/378792081 conn(0x7f9b300fc440 msgr2=0x7f9b30105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:54.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.552+0000 7f9b3570d640 1 -- 172.21.15.12:0/378792081 shutdown_connections 2024-04-22T13:23:54.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.552+0000 7f9b3570d640 1 -- 172.21.15.12:0/378792081 wait complete. 2024-04-22T13:23:54.560 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T13:23:54.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.668+0000 7faca7663640 1 -- 172.21.15.12:0/166537825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faca0102040 msgr2=0x7faca01024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:54.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.668+0000 7faca7663640 1 --2- 172.21.15.12:0/166537825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faca0102040 0x7faca01024c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009900 tx=0x7fac9c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:23:54.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.668+0000 7faca7663640 1 -- 172.21.15.12:0/166537825 shutdown_connections 2024-04-22T13:23:54.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.668+0000 7faca7663640 1 --2- 172.21.15.12:0/166537825 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faca0102a00 0x7faca0109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.668+0000 7faca7663640 1 --2- 172.21.15.12:0/166537825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faca0102040 0x7faca01024c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.668+0000 7faca7663640 1 --2- 172.21.15.12:0/166537825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faca0100e60 0x7faca0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.668+0000 7faca7663640 1 -- 172.21.15.12:0/166537825 >> 172.21.15.12:0/166537825 conn(0x7faca00fc440 msgr2=0x7faca00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:54.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.668+0000 7faca7663640 1 -- 172.21.15.12:0/166537825 shutdown_connections 2024-04-22T13:23:54.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.668+0000 7faca7663640 1 -- 172.21.15.12:0/166537825 wait complete. 2024-04-22T13:23:54.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.668+0000 7faca7663640 1 Processor -- start 2024-04-22T13:23:54.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.668+0000 7faca7663640 1 -- start start 2024-04-22T13:23:54.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.669+0000 7faca7663640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faca0100e60 0x7faca0104e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:54.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.669+0000 7faca7663640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faca0102040 0x7faca01034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:54.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.669+0000 7faca7663640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faca0102a00 0x7faca01039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:54.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.669+0000 7faca7663640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7faca010ca70 con 0x7faca0100e60 2024-04-22T13:23:54.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.669+0000 7faca7663640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faca010c8f0 con 0x7faca0102a00 2024-04-22T13:23:54.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.669+0000 7faca7663640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faca010cbf0 con 0x7faca0102040 2024-04-22T13:23:54.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.669+0000 7faca53ff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faca0100e60 0x7faca0104e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:54.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.669+0000 7faca53ff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faca0100e60 0x7faca0104e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59672/0 (socket says 172.21.15.12:59672) 2024-04-22T13:23:54.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.669+0000 7faca53ff640 1 -- 172.21.15.12:0/249832158 learned_addr learned my addr 172.21.15.12:0/249832158 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:54.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.669+0000 7faca5c00640 1 --2- 172.21.15.12:0/249832158 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faca0102a00 0x7faca01039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:54.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.669+0000 7faca4bfe640 1 --2- 172.21.15.12:0/249832158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faca0102040 0x7faca01034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:54.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.669+0000 7faca53ff640 1 -- 172.21.15.12:0/249832158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faca0102040 msgr2=0x7faca01034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:54.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.669+0000 7faca53ff640 1 --2- 172.21.15.12:0/249832158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faca0102040 0x7faca01034b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.669+0000 7faca53ff640 1 -- 172.21.15.12:0/249832158 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faca0102a00 msgr2=0x7faca01039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:54.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.669+0000 7faca53ff640 1 --2- 172.21.15.12:0/249832158 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faca0102a00 0x7faca01039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.669+0000 7faca53ff640 1 -- 172.21.15.12:0/249832158 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca0104280 con 0x7faca0100e60 2024-04-22T13:23:54.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.669+0000 7faca4bfe640 1 --2- 172.21.15.12:0/249832158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faca0102040 0x7faca01034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:23:54.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.669+0000 7faca53ff640 1 --2- 172.21.15.12:0/249832158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faca0100e60 0x7faca0104e70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fac9000b450 tx=0x7fac9000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:54.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.669+0000 7fac967fc640 1 -- 172.21.15.12:0/249832158 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac90015020 con 0x7faca0100e60 2024-04-22T13:23:54.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.669+0000 7faca7663640 1 -- 172.21.15.12:0/249832158 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca01a23b0 con 0x7faca0100e60 2024-04-22T13:23:54.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.669+0000 7fac967fc640 1 -- 172.21.15.12:0/249832158 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac90011d40 con 0x7faca0100e60 2024-04-22T13:23:54.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.669+0000 7fac967fc640 1 -- 172.21.15.12:0/249832158 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9000f340 con 0x7faca0100e60 2024-04-22T13:23:54.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.670+0000 7faca7663640 1 -- 172.21.15.12:0/249832158 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca01a2930 con 0x7faca0100e60 2024-04-22T13:23:54.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.671+0000 7faca7663640 1 -- 172.21.15.12:0/249832158 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faca01a2bf0 con 0x7faca0100e60 2024-04-22T13:23:54.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.672+0000 7fac967fc640 1 -- 172.21.15.12:0/249832158 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fac90018020 con 0x7faca0100e60 2024-04-22T13:23:54.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.672+0000 7fac967fc640 1 --2- 172.21.15.12:0/249832158 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fac8c0a3bb0 0x7fac8c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:54.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.672+0000 7faca4bfe640 1 --2- 172.21.15.12:0/249832158 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fac8c0a3bb0 0x7fac8c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:54.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.672+0000 7fac967fc640 1 -- 172.21.15.12:0/249832158 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fac900d0560 con 0x7faca0100e60 2024-04-22T13:23:54.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.673+0000 7faca4bfe640 1 --2- 172.21.15.12:0/249832158 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fac8c0a3bb0 0x7fac8c0a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0096c0 tx=0x7fac9c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:54.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.676+0000 7fac967fc640 1 -- 172.21.15.12:0/249832158 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fac9001f020 con 0x7faca0100e60 2024-04-22T13:23:54.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:54 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3443397500' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:23:54.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:54 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/378792081' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:23:54.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.863+0000 7faca7663640 1 -- 172.21.15.12:0/249832158 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7faca01a2dc0 con 0x7faca0100e60 2024-04-22T13:23:54.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.864+0000 7fac967fc640 1 -- 172.21.15.12:0/249832158 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7fac9000d040 con 0x7faca0100e60 2024-04-22T13:23:54.861 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-22T13:23:54.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.866+0000 7faca7663640 1 -- 172.21.15.12:0/249832158 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fac8c0a3bb0 msgr2=0x7fac8c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:54.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.866+0000 7faca7663640 1 --2- 172.21.15.12:0/249832158 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fac8c0a3bb0 0x7fac8c0a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0096c0 tx=0x7fac9c000bc0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.866+0000 7faca7663640 1 -- 172.21.15.12:0/249832158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faca0100e60 msgr2=0x7faca0104e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:54.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.866+0000 7faca7663640 1 --2- 172.21.15.12:0/249832158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faca0100e60 0x7faca0104e70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fac9000b450 tx=0x7fac9000b910 comp rx=0 tx=0).stop 2024-04-22T13:23:54.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.866+0000 7faca7663640 1 -- 172.21.15.12:0/249832158 shutdown_connections 2024-04-22T13:23:54.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.866+0000 7faca7663640 1 --2- 172.21.15.12:0/249832158 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faca0102a00 0x7faca01039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.866+0000 7faca7663640 1 --2- 172.21.15.12:0/249832158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faca0102040 0x7faca01034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.866+0000 7faca7663640 1 --2- 172.21.15.12:0/249832158 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fac8c0a3bb0 0x7fac8c0a6070 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.866+0000 7faca7663640 1 --2- 172.21.15.12:0/249832158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faca0100e60 0x7faca0104e70 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.866+0000 7faca7663640 1 -- 172.21.15.12:0/249832158 >> 172.21.15.12:0/249832158 conn(0x7faca00fc440 msgr2=0x7faca00fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:54.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.866+0000 7faca7663640 1 -- 172.21.15.12:0/249832158 shutdown_connections 2024-04-22T13:23:54.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.866+0000 7faca7663640 1 -- 172.21.15.12:0/249832158 wait complete. 2024-04-22T13:23:54.874 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T13:23:54.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:54 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/3443397500' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:23:54.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:54 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/378792081' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:23:54.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.982+0000 7f73233fb640 1 -- 172.21.15.12:0/1824774260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f731c105360 msgr2=0x7f731c107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:54.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.982+0000 7f73233fb640 1 --2- 172.21.15.12:0/1824774260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f731c105360 0x7f731c107750 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f730c009900 tx=0x7f730c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:23:54.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.982+0000 7f73233fb640 1 -- 172.21.15.12:0/1824774260 shutdown_connections 2024-04-22T13:23:54.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.982+0000 7f73233fb640 1 --2- 172.21.15.12:0/1824774260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f731c107c90 0x7f731c10a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.982+0000 7f73233fb640 1 --2- 172.21.15.12:0/1824774260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f731c105360 0x7f731c107750 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.982+0000 7f73233fb640 1 --2- 172.21.15.12:0/1824774260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f731c102a30 0x7f731c104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.982+0000 7f73233fb640 1 -- 172.21.15.12:0/1824774260 >> 172.21.15.12:0/1824774260 conn(0x7f731c0fc440 msgr2=0x7f731c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:54.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.982+0000 7f73233fb640 1 -- 172.21.15.12:0/1824774260 shutdown_connections 2024-04-22T13:23:54.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.982+0000 7f73233fb640 1 -- 172.21.15.12:0/1824774260 wait complete. 2024-04-22T13:23:54.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.983+0000 7f73233fb640 1 Processor -- start 2024-04-22T13:23:54.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.983+0000 7f73233fb640 1 -- start start 2024-04-22T13:23:54.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.983+0000 7f73233fb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f731c102a30 0x7f731c198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:54.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.983+0000 7f73233fb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f731c105360 0x7f731c1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:54.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.983+0000 7f73233fb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f731c107c90 0x7f731c19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:54.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.983+0000 7f73233fb640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f731c10dab0 con 0x7f731c107c90 2024-04-22T13:23:54.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.983+0000 7f73233fb640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f731c10d930 con 0x7f731c105360 2024-04-22T13:23:54.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.983+0000 7f73233fb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f731c10dc30 con 0x7f731c102a30 2024-04-22T13:23:54.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.983+0000 7f7321998640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f731c107c90 0x7f731c19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:54.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.983+0000 7f7321998640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f731c107c90 0x7f731c19f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59696/0 (socket says 172.21.15.12:59696) 2024-04-22T13:23:54.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.983+0000 7f7321998640 1 -- 172.21.15.12:0/2944771492 learned_addr learned my addr 172.21.15.12:0/2944771492 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:54.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.983+0000 7f7321197640 1 --2- 172.21.15.12:0/2944771492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f731c102a30 0x7f731c198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:54.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.983+0000 7f7320996640 1 --2- 172.21.15.12:0/2944771492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f731c105360 0x7f731c1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:54.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.983+0000 7f7321998640 1 -- 172.21.15.12:0/2944771492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f731c102a30 msgr2=0x7f731c198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:54.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.983+0000 7f7321998640 1 --2- 172.21.15.12:0/2944771492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f731c102a30 0x7f731c198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.983+0000 7f7321998640 1 -- 172.21.15.12:0/2944771492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f731c105360 msgr2=0x7f731c1987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:54.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.983+0000 7f7321998640 1 --2- 172.21.15.12:0/2944771492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f731c105360 0x7f731c1987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:54.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.983+0000 7f7321998640 1 -- 172.21.15.12:0/2944771492 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f731c19fe90 con 0x7f731c107c90 2024-04-22T13:23:54.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.984+0000 7f7321998640 1 --2- 172.21.15.12:0/2944771492 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f731c107c90 0x7f731c19f820 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f731400eaa0 tx=0x7f731400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:54.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.984+0000 7f73127fc640 1 -- 172.21.15.12:0/2944771492 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731400cdb0 con 0x7f731c107c90 2024-04-22T13:23:54.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.984+0000 7f73127fc640 1 -- 172.21.15.12:0/2944771492 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7314005020 con 0x7f731c107c90 2024-04-22T13:23:54.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.984+0000 7f73233fb640 1 -- 172.21.15.12:0/2944771492 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f731c1a0180 con 0x7f731c107c90 2024-04-22T13:23:54.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.984+0000 7f73127fc640 1 -- 172.21.15.12:0/2944771492 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73140116d0 con 0x7f731c107c90 2024-04-22T13:23:54.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.984+0000 7f73233fb640 1 -- 172.21.15.12:0/2944771492 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f731c1a0650 con 0x7f731c107c90 2024-04-22T13:23:54.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.985+0000 7f73233fb640 1 -- 172.21.15.12:0/2944771492 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f731c0fdf30 con 0x7f731c107c90 2024-04-22T13:23:54.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.986+0000 7f73127fc640 1 -- 172.21.15.12:0/2944771492 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f73140051d0 con 0x7f731c107c90 2024-04-22T13:23:54.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.986+0000 7f73127fc640 1 --2- 172.21.15.12:0/2944771492 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f73080a3bb0 0x7f73080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:54.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.987+0000 7f7321197640 1 --2- 172.21.15.12:0/2944771492 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f73080a3bb0 0x7f73080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:54.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.987+0000 7f73127fc640 1 -- 172.21.15.12:0/2944771492 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f73140d0510 con 0x7f731c107c90 2024-04-22T13:23:54.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.987+0000 7f7321197640 1 --2- 172.21.15.12:0/2944771492 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f73080a3bb0 0x7f73080a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7318005d00 tx=0x7f731800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:54.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:54.991+0000 7f73127fc640 1 -- 172.21.15.12:0/2944771492 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7314016030 con 0x7f731c107c90 2024-04-22T13:23:55.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.179+0000 7f73233fb640 1 -- 172.21.15.12:0/2944771492 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f731c062b50 con 0x7f731c107c90 2024-04-22T13:23:55.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.180+0000 7f73127fc640 1 -- 172.21.15.12:0/2944771492 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f731409c870 con 0x7f731c107c90 2024-04-22T13:23:55.177 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-22T13:23:55.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:54 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/3443397500' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:23:55.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:54 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/378792081' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:23:55.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.181+0000 7f73233fb640 1 -- 172.21.15.12:0/2944771492 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f73080a3bb0 msgr2=0x7f73080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:55.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.181+0000 7f73233fb640 1 --2- 172.21.15.12:0/2944771492 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f73080a3bb0 0x7f73080a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7318005d00 tx=0x7f731800b410 comp rx=0 tx=0).stop 2024-04-22T13:23:55.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.182+0000 7f73233fb640 1 -- 172.21.15.12:0/2944771492 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f731c107c90 msgr2=0x7f731c19f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:55.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.182+0000 7f73233fb640 1 --2- 172.21.15.12:0/2944771492 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f731c107c90 0x7f731c19f820 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f731400eaa0 tx=0x7f731400ef60 comp rx=0 tx=0).stop 2024-04-22T13:23:55.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.182+0000 7f73233fb640 1 -- 172.21.15.12:0/2944771492 shutdown_connections 2024-04-22T13:23:55.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.182+0000 7f73233fb640 1 --2- 172.21.15.12:0/2944771492 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f73080a3bb0 0x7f73080a6070 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.182+0000 7f73233fb640 1 --2- 172.21.15.12:0/2944771492 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f731c107c90 0x7f731c19f820 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.182+0000 7f73233fb640 1 --2- 172.21.15.12:0/2944771492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f731c105360 0x7f731c1987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.182+0000 7f73233fb640 1 --2- 172.21.15.12:0/2944771492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f731c102a30 0x7f731c198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.182+0000 7f73233fb640 1 -- 172.21.15.12:0/2944771492 >> 172.21.15.12:0/2944771492 conn(0x7f731c0fc440 msgr2=0x7f731c1048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:55.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.182+0000 7f73233fb640 1 -- 172.21.15.12:0/2944771492 shutdown_connections 2024-04-22T13:23:55.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.182+0000 7f73233fb640 1 -- 172.21.15.12:0/2944771492 wait complete. 2024-04-22T13:23:55.190 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:23:55.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.295+0000 7f0731796640 1 -- 172.21.15.12:0/2063524430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f072c100e70 msgr2=0x7f072c101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:55.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.295+0000 7f0731796640 1 --2- 172.21.15.12:0/2063524430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f072c100e70 0x7f072c101290 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f071c009900 tx=0x7f071c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:23:55.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.295+0000 7f0731796640 1 -- 172.21.15.12:0/2063524430 shutdown_connections 2024-04-22T13:23:55.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.295+0000 7f0731796640 1 --2- 172.21.15.12:0/2063524430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f072c102a10 0x7f072c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.295+0000 7f0731796640 1 --2- 172.21.15.12:0/2063524430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f072c102050 0x7f072c1024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.295+0000 7f0731796640 1 --2- 172.21.15.12:0/2063524430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f072c100e70 0x7f072c101290 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.295+0000 7f0731796640 1 -- 172.21.15.12:0/2063524430 >> 172.21.15.12:0/2063524430 conn(0x7f072c0fc470 msgr2=0x7f072c0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:55.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.295+0000 7f0731796640 1 -- 172.21.15.12:0/2063524430 shutdown_connections 2024-04-22T13:23:55.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.295+0000 7f0731796640 1 -- 172.21.15.12:0/2063524430 wait complete. 2024-04-22T13:23:55.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.295+0000 7f0731796640 1 Processor -- start 2024-04-22T13:23:55.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.295+0000 7f0731796640 1 -- start start 2024-04-22T13:23:55.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.296+0000 7f0731796640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f072c100e70 0x7f072c197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:55.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.296+0000 7f0731796640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f072c102050 0x7f072c197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:55.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.296+0000 7f0731796640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f072c102a10 0x7f072c19e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:55.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.296+0000 7f0731796640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f072c10c9e0 con 0x7f072c102050 2024-04-22T13:23:55.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.296+0000 7f0731796640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f072c10c860 con 0x7f072c100e70 2024-04-22T13:23:55.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.296+0000 7f0731796640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f072c10cb60 con 0x7f072c102a10 2024-04-22T13:23:55.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.296+0000 7f072a7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f072c102050 0x7f072c197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:55.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.296+0000 7f072a7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f072c102050 0x7f072c197560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59714/0 (socket says 172.21.15.12:59714) 2024-04-22T13:23:55.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.296+0000 7f072a7fc640 1 -- 172.21.15.12:0/3217435250 learned_addr learned my addr 172.21.15.12:0/3217435250 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:55.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.296+0000 7f072b7fe640 1 --2- 172.21.15.12:0/3217435250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f072c102a10 0x7f072c19e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:55.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.296+0000 7f072affd640 1 --2- 172.21.15.12:0/3217435250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f072c100e70 0x7f072c197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:55.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.296+0000 7f072a7fc640 1 -- 172.21.15.12:0/3217435250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f072c102a10 msgr2=0x7f072c19e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:55.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.296+0000 7f072a7fc640 1 --2- 172.21.15.12:0/3217435250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f072c102a10 0x7f072c19e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.296+0000 7f072a7fc640 1 -- 172.21.15.12:0/3217435250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f072c100e70 msgr2=0x7f072c197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:55.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.296+0000 7f072a7fc640 1 --2- 172.21.15.12:0/3217435250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f072c100e70 0x7f072c197020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.296+0000 7f072a7fc640 1 -- 172.21.15.12:0/3217435250 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f072c19ec40 con 0x7f072c102050 2024-04-22T13:23:55.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.296+0000 7f072a7fc640 1 --2- 172.21.15.12:0/3217435250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f072c102050 0x7f072c197560 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f072000c920 tx=0x7f072000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:55.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.296+0000 7f0713fff640 1 -- 172.21.15.12:0/3217435250 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0720007c70 con 0x7f072c102050 2024-04-22T13:23:55.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.296+0000 7f0713fff640 1 -- 172.21.15.12:0/3217435250 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0720007e10 con 0x7f072c102050 2024-04-22T13:23:55.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.296+0000 7f0731796640 1 -- 172.21.15.12:0/3217435250 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f072c19ef30 con 0x7f072c102050 2024-04-22T13:23:55.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.296+0000 7f0713fff640 1 -- 172.21.15.12:0/3217435250 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0720018820 con 0x7f072c102050 2024-04-22T13:23:55.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.296+0000 7f0731796640 1 -- 172.21.15.12:0/3217435250 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f072c19f3f0 con 0x7f072c102050 2024-04-22T13:23:55.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.297+0000 7f0731796640 1 -- 172.21.15.12:0/3217435250 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f072c1081b0 con 0x7f072c102050 2024-04-22T13:23:55.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.299+0000 7f0713fff640 1 -- 172.21.15.12:0/3217435250 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f072000f450 con 0x7f072c102050 2024-04-22T13:23:55.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.300+0000 7f0713fff640 1 --2- 172.21.15.12:0/3217435250 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f07140a3bb0 0x7f07140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:55.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.300+0000 7f072affd640 1 --2- 172.21.15.12:0/3217435250 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f07140a3bb0 0x7f07140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:55.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.300+0000 7f0713fff640 1 -- 172.21.15.12:0/3217435250 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f07200d0a00 con 0x7f072c102050 2024-04-22T13:23:55.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.300+0000 7f072affd640 1 --2- 172.21.15.12:0/3217435250 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f07140a3bb0 0x7f07140a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f071c036040 tx=0x7f071c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:55.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.303+0000 7f0713fff640 1 -- 172.21.15.12:0/3217435250 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f072001d020 con 0x7f072c102050 2024-04-22T13:23:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.528+0000 7f0731796640 1 -- 172.21.15.12:0/3217435250 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f072c02d810 con 0x7f072c102050 2024-04-22T13:23:55.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.528+0000 7f0713fff640 1 -- 172.21.15.12:0/3217435250 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11942 (secure 0 0 0) 0x7f0720014030 con 0x7f072c102050 2024-04-22T13:23:55.526 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:23:55.526 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:22:12.398056+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24448,24509,24443,24484,24407]},"id":1}]} 2024-04-22T13:23:55.526 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 27 2024-04-22T13:23:55.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.530+0000 7f0731796640 1 -- 172.21.15.12:0/3217435250 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f07140a3bb0 msgr2=0x7f07140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:55.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.530+0000 7f0731796640 1 --2- 172.21.15.12:0/3217435250 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f07140a3bb0 0x7f07140a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f071c036040 tx=0x7f071c000bc0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.530+0000 7f0731796640 1 -- 172.21.15.12:0/3217435250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f072c102050 msgr2=0x7f072c197560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:55.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.530+0000 7f0731796640 1 --2- 172.21.15.12:0/3217435250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f072c102050 0x7f072c197560 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f072000c920 tx=0x7f072000cde0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.530+0000 7f0731796640 1 -- 172.21.15.12:0/3217435250 shutdown_connections 2024-04-22T13:23:55.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.530+0000 7f0731796640 1 --2- 172.21.15.12:0/3217435250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f072c102a10 0x7f072c19e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.530+0000 7f0731796640 1 --2- 172.21.15.12:0/3217435250 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f07140a3bb0 0x7f07140a6070 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.530+0000 7f0731796640 1 --2- 172.21.15.12:0/3217435250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f072c102050 0x7f072c197560 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.530+0000 7f0731796640 1 --2- 172.21.15.12:0/3217435250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f072c100e70 0x7f072c197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.530+0000 7f0731796640 1 -- 172.21.15.12:0/3217435250 >> 172.21.15.12:0/3217435250 conn(0x7f072c0fc470 msgr2=0x7f072c105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:55.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.531+0000 7f0731796640 1 -- 172.21.15.12:0/3217435250 shutdown_connections 2024-04-22T13:23:55.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.531+0000 7f0731796640 1 -- 172.21.15.12:0/3217435250 wait complete. 2024-04-22T13:23:55.540 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:23:55.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.644+0000 7f802f4e0640 1 -- 172.21.15.12:0/2498993117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8028101fa0 msgr2=0x7f80281023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:55.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.644+0000 7f802f4e0640 1 --2- 172.21.15.12:0/2498993117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8028101fa0 0x7f80281023c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8020009900 tx=0x7f802002bc30 comp rx=0 tx=0).stop 2024-04-22T13:23:55.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.644+0000 7f802f4e0640 1 -- 172.21.15.12:0/2498993117 shutdown_connections 2024-04-22T13:23:55.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.644+0000 7f802f4e0640 1 --2- 172.21.15.12:0/2498993117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8028103b50 0x7f802810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.644+0000 7f802f4e0640 1 --2- 172.21.15.12:0/2498993117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8028103190 0x7f8028103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.644+0000 7f802f4e0640 1 --2- 172.21.15.12:0/2498993117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8028101fa0 0x7f80281023c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.644+0000 7f802f4e0640 1 -- 172.21.15.12:0/2498993117 >> 172.21.15.12:0/2498993117 conn(0x7f80280fd530 msgr2=0x7f80280ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:55.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.645+0000 7f802f4e0640 1 -- 172.21.15.12:0/2498993117 shutdown_connections 2024-04-22T13:23:55.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.645+0000 7f802f4e0640 1 -- 172.21.15.12:0/2498993117 wait complete. 2024-04-22T13:23:55.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.645+0000 7f802f4e0640 1 Processor -- start 2024-04-22T13:23:55.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.645+0000 7f802f4e0640 1 -- start start 2024-04-22T13:23:55.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.645+0000 7f802f4e0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8028101fa0 0x7f8028198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:55.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.645+0000 7f802f4e0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8028103190 0x7f8028198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:55.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.645+0000 7f802f4e0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8028103b50 0x7f802819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:55.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.645+0000 7f802f4e0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f802810db70 con 0x7f8028103190 2024-04-22T13:23:55.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.645+0000 7f802f4e0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f802810d9f0 con 0x7f8028103b50 2024-04-22T13:23:55.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.645+0000 7f802f4e0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f802810dcf0 con 0x7f8028101fa0 2024-04-22T13:23:55.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.645+0000 7f802ca7b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8028103190 0x7f8028198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:55.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.645+0000 7f802ca7b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8028103190 0x7f8028198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59730/0 (socket says 172.21.15.12:59730) 2024-04-22T13:23:55.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.645+0000 7f802ca7b640 1 -- 172.21.15.12:0/2147216364 learned_addr learned my addr 172.21.15.12:0/2147216364 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:55.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.646+0000 7f802da7d640 1 --2- 172.21.15.12:0/2147216364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8028103b50 0x7f802819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:55.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.646+0000 7f802d27c640 1 --2- 172.21.15.12:0/2147216364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8028101fa0 0x7f8028198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:55.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.646+0000 7f802ca7b640 1 -- 172.21.15.12:0/2147216364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8028101fa0 msgr2=0x7f8028198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:55.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.646+0000 7f802ca7b640 1 --2- 172.21.15.12:0/2147216364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8028101fa0 0x7f8028198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.646+0000 7f802ca7b640 1 -- 172.21.15.12:0/2147216364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8028103b50 msgr2=0x7f802819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:55.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.646+0000 7f802ca7b640 1 --2- 172.21.15.12:0/2147216364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8028103b50 0x7f802819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.646+0000 7f802ca7b640 1 -- 172.21.15.12:0/2147216364 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f802819fe30 con 0x7f8028103190 2024-04-22T13:23:55.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.646+0000 7f802ca7b640 1 --2- 172.21.15.12:0/2147216364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8028103190 0x7f8028198750 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f801c00b4a0 tx=0x7f801c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:55.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.646+0000 7f80267fc640 1 -- 172.21.15.12:0/2147216364 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c014070 con 0x7f8028103190 2024-04-22T13:23:55.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.646+0000 7f80267fc640 1 -- 172.21.15.12:0/2147216364 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f801c010ac0 con 0x7f8028103190 2024-04-22T13:23:55.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.646+0000 7f802f4e0640 1 -- 172.21.15.12:0/2147216364 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80281a0120 con 0x7f8028103190 2024-04-22T13:23:55.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.646+0000 7f80267fc640 1 -- 172.21.15.12:0/2147216364 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c017c30 con 0x7f8028103190 2024-04-22T13:23:55.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.646+0000 7f802f4e0640 1 -- 172.21.15.12:0/2147216364 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80281a0740 con 0x7f8028103190 2024-04-22T13:23:55.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.647+0000 7f802f4e0640 1 -- 172.21.15.12:0/2147216364 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80281092c0 con 0x7f8028103190 2024-04-22T13:23:55.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.648+0000 7f80267fc640 1 -- 172.21.15.12:0/2147216364 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f801c017070 con 0x7f8028103190 2024-04-22T13:23:55.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.649+0000 7f80267fc640 1 --2- 172.21.15.12:0/2147216364 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f80100a3c30 0x7f80100a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:55.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.649+0000 7f802d27c640 1 --2- 172.21.15.12:0/2147216364 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f80100a3c30 0x7f80100a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:55.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.649+0000 7f80267fc640 1 -- 172.21.15.12:0/2147216364 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f801c0cfb50 con 0x7f8028103190 2024-04-22T13:23:55.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.649+0000 7f802d27c640 1 --2- 172.21.15.12:0/2147216364 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f80100a3c30 0x7f80100a60f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f802002c240 tx=0x7f8020005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:55.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.653+0000 7f80267fc640 1 -- 172.21.15.12:0/2147216364 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f801c09c500 con 0x7f8028103190 2024-04-22T13:23:55.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:55 smithi012 ceph-mon[28265]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-22T13:23:55.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:55 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/249832158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:23:55.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:55 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2944771492' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:23:55.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:55 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3217435250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:23:55.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.844+0000 7f802f4e0640 1 -- 172.21.15.12:0/2147216364 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8028062b50 con 0x7f8028103190 2024-04-22T13:23:55.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.846+0000 7f80267fc640 1 -- 172.21.15.12:0/2147216364 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22620 (secure 0 0 0) 0x7f801c09be00 con 0x7f8028103190 2024-04-22T13:23:55.844 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:23:55.844 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"0aede5ea-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:14:09.362918+0000","modified":"2024-04-22T13:23:05.500756+0000","last_up_change":"2024-04-22T13:21:15.263393+0000","last_in_change":"2024-04-22T13:20:54.174685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:17:32.372434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:22:02.768799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:22:03.578730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d9e2365-3a64-4b13-96fc-c217b1cee2ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6803","nonce":1191350632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6805","nonce":1191350632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6809","nonce":1191350632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6807","nonce":1191350632}]},"public_addr":"172.21.15.12:6803/1191350632","cluster_addr":"172.21.15.12:6805/1191350632","heartbeat_back_addr":"172.21.15.12:6809/1191350632","heartbeat_front_addr":"172.21.15.12:6807/1191350632","state":["exists","up"]},{"osd":1,"uuid":"1499c783-2bd7-48ce-a444-34add42bcf31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6801","nonce":417496465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6803","nonce":417496465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6807","nonce":417496465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6805","nonce":417496465}]},"public_addr":"172.21.15.111:6801/417496465","cluster_addr":"172.21.15.111:6803/417496465","heartbeat_back_addr":"172.21.15.111:6807/417496465","heartbeat_front_addr":"172.21.15.111:6805/417496465","state":["exists","up"]},{"osd":2,"uuid":"71a9ebc3-c622-434e-a6c2-f48358f384f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6801","nonce":1403153998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6803","nonce":1403153998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6807","nonce":1403153998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6805","nonce":1403153998}]},"public_addr":"172.21.15.134:6801/1403153998","cluster_addr":"172.21.15.134:6803/1403153998","heartbeat_back_addr":"172.21.15.134:6807/1403153998","heartbeat_front_addr":"172.21.15.134:6805/1403153998","state":["exists","up"]},{"osd":3,"uuid":"6dd00db2-a286-48ba-8666-56ffdd5f35ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6811","nonce":3359066681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6813","nonce":3359066681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6817","nonce":3359066681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6815","nonce":3359066681}]},"public_addr":"172.21.15.12:6811/3359066681","cluster_addr":"172.21.15.12:6813/3359066681","heartbeat_back_addr":"172.21.15.12:6817/3359066681","heartbeat_front_addr":"172.21.15.12:6815/3359066681","state":["exists","up"]},{"osd":4,"uuid":"1e29afd7-354c-4506-aaa0-b43eb1a19a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6809","nonce":2886349872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6811","nonce":2886349872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6815","nonce":2886349872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6813","nonce":2886349872}]},"public_addr":"172.21.15.111:6809/2886349872","cluster_addr":"172.21.15.111:6811/2886349872","heartbeat_back_addr":"172.21.15.111:6815/2886349872","heartbeat_front_addr":"172.21.15.111:6813/2886349872","state":["exists","up"]},{"osd":5,"uuid":"8f40f02d-0ba2-4842-a03d-2a1a04d1010d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6809","nonce":3071977602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6811","nonce":3071977602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6815","nonce":3071977602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6813","nonce":3071977602}]},"public_addr":"172.21.15.134:6809/3071977602","cluster_addr":"172.21.15.134:6811/3071977602","heartbeat_back_addr":"172.21.15.134:6815/3071977602","heartbeat_front_addr":"172.21.15.134:6813/3071977602","state":["exists","up"]},{"osd":6,"uuid":"bb0c17a9-6a8e-4be6-8660-f73cfa8200ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6819","nonce":3819619299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6821","nonce":3819619299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6825","nonce":3819619299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6823","nonce":3819619299}]},"public_addr":"172.21.15.12:6819/3819619299","cluster_addr":"172.21.15.12:6821/3819619299","heartbeat_back_addr":"172.21.15.12:6825/3819619299","heartbeat_front_addr":"172.21.15.12:6823/3819619299","state":["exists","up"]},{"osd":7,"uuid":"0a93d6af-569e-4e45-b697-9e978c1c1004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6817","nonce":3113856282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6819","nonce":3113856282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6823","nonce":3113856282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6821","nonce":3113856282}]},"public_addr":"172.21.15.111:6817/3113856282","cluster_addr":"172.21.15.111:6819/3113856282","heartbeat_back_addr":"172.21.15.111:6823/3113856282","heartbeat_front_addr":"172.21.15.111:6821/3113856282","state":["exists","up"]},{"osd":8,"uuid":"e21acc68-cf50-4f4f-b1ba-cf6c195b4955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6817","nonce":1689362191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6819","nonce":1689362191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6823","nonce":1689362191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6821","nonce":1689362191}]},"public_addr":"172.21.15.134:6817/1689362191","cluster_addr":"172.21.15.134:6819/1689362191","heartbeat_back_addr":"172.21.15.134:6823/1689362191","heartbeat_front_addr":"172.21.15.134:6821/1689362191","state":["exists","up"]},{"osd":9,"uuid":"fd745869-7f5f-48a8-b412-bb9c92b586e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6827","nonce":4292385779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6829","nonce":4292385779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6833","nonce":4292385779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6831","nonce":4292385779}]},"public_addr":"172.21.15.12:6827/4292385779","cluster_addr":"172.21.15.12:6829/4292385779","heartbeat_back_addr":"172.21.15.12:6833/4292385779","heartbeat_front_addr":"172.21.15.12:6831/4292385779","state":["exists","up"]},{"osd":10,"uuid":"798b1abd-f6c7-4a5f-8da0-fdae49e50d2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6825","nonce":3338566976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6827","nonce":3338566976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6831","nonce":3338566976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6829","nonce":3338566976}]},"public_addr":"172.21.15.111:6825/3338566976","cluster_addr":"172.21.15.111:6827/3338566976","heartbeat_back_addr":"172.21.15.111:6831/3338566976","heartbeat_front_addr":"172.21.15.111:6829/3338566976","state":["exists","up"]},{"osd":11,"uuid":"b18fc5f3-1049-4871-b060-e99290273cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6825","nonce":578640428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6827","nonce":578640428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6831","nonce":578640428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6829","nonce":578640428}]},"public_addr":"172.21.15.134:6825/578640428","cluster_addr":"172.21.15.134:6827/578640428","heartbeat_back_addr":"172.21.15.134:6831/578640428","heartbeat_front_addr":"172.21.15.134:6829/578640428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:16:40.828608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:05.052528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:28.297316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:53.496497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:17.947259+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:41.712894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:07.105945+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:32.129688+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:56.008516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:22.239867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:47.334692+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:12.301276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/2593601362":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/2519382069":"2024-04-23T13:15:04.279990+0000","172.21.15.12:6801/2009831940":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3509412187":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1007749062":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3588480095":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1904962176":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2009831940":"2024-04-23T13:15:04.279990+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:23:55.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.848+0000 7f802f4e0640 1 -- 172.21.15.12:0/2147216364 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f80100a3c30 msgr2=0x7f80100a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:55.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.848+0000 7f802f4e0640 1 --2- 172.21.15.12:0/2147216364 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f80100a3c30 0x7f80100a60f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f802002c240 tx=0x7f8020005fd0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.848+0000 7f802f4e0640 1 -- 172.21.15.12:0/2147216364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8028103190 msgr2=0x7f8028198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:55.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.848+0000 7f802f4e0640 1 --2- 172.21.15.12:0/2147216364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8028103190 0x7f8028198750 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f801c00b4a0 tx=0x7f801c00b960 comp rx=0 tx=0).stop 2024-04-22T13:23:55.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.848+0000 7f802f4e0640 1 -- 172.21.15.12:0/2147216364 shutdown_connections 2024-04-22T13:23:55.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.848+0000 7f802f4e0640 1 --2- 172.21.15.12:0/2147216364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8028103b50 0x7f802819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.848+0000 7f802f4e0640 1 --2- 172.21.15.12:0/2147216364 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f80100a3c30 0x7f80100a60f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.848+0000 7f802f4e0640 1 --2- 172.21.15.12:0/2147216364 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8028103190 0x7f8028198750 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.848+0000 7f802f4e0640 1 --2- 172.21.15.12:0/2147216364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8028101fa0 0x7f8028198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.848+0000 7f802f4e0640 1 -- 172.21.15.12:0/2147216364 >> 172.21.15.12:0/2147216364 conn(0x7f80280fd530 msgr2=0x7f8028106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:55.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.849+0000 7f802f4e0640 1 -- 172.21.15.12:0/2147216364 shutdown_connections 2024-04-22T13:23:55.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.849+0000 7f802f4e0640 1 -- 172.21.15.12:0/2147216364 wait complete. 2024-04-22T13:23:55.857 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T13:17:32.372434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T13:22:02.768799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T13:22:03.578730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T13:23:55.858 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T13:23:55.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.965+0000 7fb594ab3640 1 -- 172.21.15.12:0/3380006699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5901029a0 msgr2=0x7fb5901092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:55.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.965+0000 7fb594ab3640 1 --2- 172.21.15.12:0/3380006699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5901029a0 0x7fb5901092c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb588009900 tx=0x7fb58802bc30 comp rx=0 tx=0).stop 2024-04-22T13:23:55.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.966+0000 7fb594ab3640 1 -- 172.21.15.12:0/3380006699 shutdown_connections 2024-04-22T13:23:55.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.966+0000 7fb594ab3640 1 --2- 172.21.15.12:0/3380006699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5901029a0 0x7fb5901092c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.966+0000 7fb594ab3640 1 --2- 172.21.15.12:0/3380006699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb590101fe0 0x7fb590102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.966+0000 7fb594ab3640 1 --2- 172.21.15.12:0/3380006699 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb590100e90 0x7fb5901012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.966+0000 7fb594ab3640 1 -- 172.21.15.12:0/3380006699 >> 172.21.15.12:0/3380006699 conn(0x7fb5900fc400 msgr2=0x7fb5900fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:55.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.966+0000 7fb594ab3640 1 -- 172.21.15.12:0/3380006699 shutdown_connections 2024-04-22T13:23:55.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.966+0000 7fb594ab3640 1 -- 172.21.15.12:0/3380006699 wait complete. 2024-04-22T13:23:55.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.966+0000 7fb594ab3640 1 Processor -- start 2024-04-22T13:23:55.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.966+0000 7fb594ab3640 1 -- start start 2024-04-22T13:23:55.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.966+0000 7fb594ab3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb590100e90 0x7fb590069c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:55.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.967+0000 7fb594ab3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb590101fe0 0x7fb59019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:55.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.967+0000 7fb594ab3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5901029a0 0x7fb59019f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:55.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.967+0000 7fb594ab3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb59010c9e0 con 0x7fb590101fe0 2024-04-22T13:23:55.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.967+0000 7fb594ab3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb59010c860 con 0x7fb5901029a0 2024-04-22T13:23:55.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.967+0000 7fb594ab3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb59010cb60 con 0x7fb590100e90 2024-04-22T13:23:55.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.967+0000 7fb58dd9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb590101fe0 0x7fb59019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:55.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.967+0000 7fb58dd9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb590101fe0 0x7fb59019ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59758/0 (socket says 172.21.15.12:59758) 2024-04-22T13:23:55.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.967+0000 7fb58dd9b640 1 -- 172.21.15.12:0/1090078053 learned_addr learned my addr 172.21.15.12:0/1090078053 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:55.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.967+0000 7fb58ed9d640 1 --2- 172.21.15.12:0/1090078053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5901029a0 0x7fb59019f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:55.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.967+0000 7fb58e59c640 1 --2- 172.21.15.12:0/1090078053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb590100e90 0x7fb590069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:55.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.967+0000 7fb58dd9b640 1 -- 172.21.15.12:0/1090078053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb590100e90 msgr2=0x7fb590069c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:55.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.967+0000 7fb58dd9b640 1 --2- 172.21.15.12:0/1090078053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb590100e90 0x7fb590069c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.967+0000 7fb58dd9b640 1 -- 172.21.15.12:0/1090078053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5901029a0 msgr2=0x7fb59019f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:55.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.967+0000 7fb58dd9b640 1 --2- 172.21.15.12:0/1090078053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5901029a0 0x7fb59019f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:55.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.967+0000 7fb58dd9b640 1 -- 172.21.15.12:0/1090078053 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb59019fd60 con 0x7fb590101fe0 2024-04-22T13:23:55.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.967+0000 7fb58e59c640 1 --2- 172.21.15.12:0/1090078053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb590100e90 0x7fb590069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:23:55.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.967+0000 7fb58dd9b640 1 --2- 172.21.15.12:0/1090078053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb590101fe0 0x7fb59019ce60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb58400d950 tx=0x7fb58400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:55.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.967+0000 7fb57f7fe640 1 -- 172.21.15.12:0/1090078053 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb584017070 con 0x7fb590101fe0 2024-04-22T13:23:55.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.967+0000 7fb594ab3640 1 -- 172.21.15.12:0/1090078053 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb59019fff0 con 0x7fb590101fe0 2024-04-22T13:23:55.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.967+0000 7fb57f7fe640 1 -- 172.21.15.12:0/1090078053 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb584009c30 con 0x7fb590101fe0 2024-04-22T13:23:55.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.967+0000 7fb57f7fe640 1 -- 172.21.15.12:0/1090078053 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58400f620 con 0x7fb590101fe0 2024-04-22T13:23:55.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.967+0000 7fb594ab3640 1 -- 172.21.15.12:0/1090078053 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5901a04c0 con 0x7fb590101fe0 2024-04-22T13:23:55.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.968+0000 7fb594ab3640 1 -- 172.21.15.12:0/1090078053 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5901081a0 con 0x7fb590101fe0 2024-04-22T13:23:55.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.970+0000 7fb57f7fe640 1 -- 172.21.15.12:0/1090078053 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb58401a070 con 0x7fb590101fe0 2024-04-22T13:23:55.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.970+0000 7fb57f7fe640 1 --2- 172.21.15.12:0/1090078053 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb5780a3bb0 0x7fb5780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:55.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.970+0000 7fb57f7fe640 1 -- 172.21.15.12:0/1090078053 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fb5840cfda0 con 0x7fb590101fe0 2024-04-22T13:23:55.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.970+0000 7fb58e59c640 1 --2- 172.21.15.12:0/1090078053 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb5780a3bb0 0x7fb5780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:55.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.971+0000 7fb58e59c640 1 --2- 172.21.15.12:0/1090078053 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb5780a3bb0 0x7fb5780a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb580008e00 tx=0x7fb580008350 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:55.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:55.974+0000 7fb57f7fe640 1 -- 172.21.15.12:0/1090078053 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb584009de0 con 0x7fb590101fe0 2024-04-22T13:23:55.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:55 smithi111 ceph-mon[31361]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-22T13:23:55.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:55 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/249832158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:23:55.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:55 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2944771492' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:23:55.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:55 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/3217435250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:23:56.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.161+0000 7fb594ab3640 1 -- 172.21.15.12:0/1090078053 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb59002d810 con 0x7fb590101fe0 2024-04-22T13:23:56.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.162+0000 7fb57f7fe640 1 -- 172.21.15.12:0/1090078053 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fb58409c7b0 con 0x7fb590101fe0 2024-04-22T13:23:56.159 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-22T13:23:56.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.164+0000 7fb594ab3640 1 -- 172.21.15.12:0/1090078053 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb5780a3bb0 msgr2=0x7fb5780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:56.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.164+0000 7fb594ab3640 1 --2- 172.21.15.12:0/1090078053 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb5780a3bb0 0x7fb5780a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb580008e00 tx=0x7fb580008350 comp rx=0 tx=0).stop 2024-04-22T13:23:56.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.164+0000 7fb594ab3640 1 -- 172.21.15.12:0/1090078053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb590101fe0 msgr2=0x7fb59019ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:56.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.164+0000 7fb594ab3640 1 --2- 172.21.15.12:0/1090078053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb590101fe0 0x7fb59019ce60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb58400d950 tx=0x7fb58400de10 comp rx=0 tx=0).stop 2024-04-22T13:23:56.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.164+0000 7fb594ab3640 1 -- 172.21.15.12:0/1090078053 shutdown_connections 2024-04-22T13:23:56.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.164+0000 7fb594ab3640 1 --2- 172.21.15.12:0/1090078053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5901029a0 0x7fb59019f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:56.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.164+0000 7fb594ab3640 1 --2- 172.21.15.12:0/1090078053 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fb5780a3bb0 0x7fb5780a6070 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:56.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.164+0000 7fb594ab3640 1 --2- 172.21.15.12:0/1090078053 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb590101fe0 0x7fb59019ce60 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:56.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.164+0000 7fb594ab3640 1 --2- 172.21.15.12:0/1090078053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb590100e90 0x7fb590069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:56.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.164+0000 7fb594ab3640 1 -- 172.21.15.12:0/1090078053 >> 172.21.15.12:0/1090078053 conn(0x7fb5900fc400 msgr2=0x7fb590105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:56.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.164+0000 7fb594ab3640 1 -- 172.21.15.12:0/1090078053 shutdown_connections 2024-04-22T13:23:56.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.164+0000 7fb594ab3640 1 -- 172.21.15.12:0/1090078053 wait complete. 2024-04-22T13:23:56.173 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T13:23:56.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:55 smithi134 ceph-mon[31338]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-22T13:23:56.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:55 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/249832158' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:23:56.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:55 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2944771492' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:23:56.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:55 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/3217435250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:23:56.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.281+0000 7f83e4a14640 1 -- 172.21.15.12:0/3450958286 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83e0102a30 msgr2=0x7f83e0104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:56.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.281+0000 7f83e4a14640 1 --2- 172.21.15.12:0/3450958286 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83e0102a30 0x7f83e0104e20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f83d0009ca0 tx=0x7f83d002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:23:56.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.281+0000 7f83e4a14640 1 -- 172.21.15.12:0/3450958286 shutdown_connections 2024-04-22T13:23:56.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.281+0000 7f83e4a14640 1 --2- 172.21.15.12:0/3450958286 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83e0107c90 0x7f83e010a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:56.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.281+0000 7f83e4a14640 1 --2- 172.21.15.12:0/3450958286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83e0105360 0x7f83e0107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:56.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.281+0000 7f83e4a14640 1 --2- 172.21.15.12:0/3450958286 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83e0102a30 0x7f83e0104e20 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:56.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.281+0000 7f83e4a14640 1 -- 172.21.15.12:0/3450958286 >> 172.21.15.12:0/3450958286 conn(0x7f83e00fc440 msgr2=0x7f83e00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:56.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.281+0000 7f83e4a14640 1 -- 172.21.15.12:0/3450958286 shutdown_connections 2024-04-22T13:23:56.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.281+0000 7f83e4a14640 1 -- 172.21.15.12:0/3450958286 wait complete. 2024-04-22T13:23:56.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.282+0000 7f83e4a14640 1 Processor -- start 2024-04-22T13:23:56.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.282+0000 7f83e4a14640 1 -- start start 2024-04-22T13:23:56.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.282+0000 7f83e4a14640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83e0102a30 0x7f83e0198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:56.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.282+0000 7f83e4a14640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83e0105360 0x7f83e01987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:56.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.282+0000 7f83e4a14640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83e0107c90 0x7f83e019f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:56.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.282+0000 7f83e4a14640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f83e010db50 con 0x7f83e0107c90 2024-04-22T13:23:56.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.282+0000 7f83e4a14640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f83e010d9d0 con 0x7f83e0105360 2024-04-22T13:23:56.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.282+0000 7f83e4a14640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f83e010dcd0 con 0x7f83e0102a30 2024-04-22T13:23:56.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.282+0000 7f83ded9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83e0107c90 0x7f83e019f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:56.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.282+0000 7f83de59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83e0102a30 0x7f83e0198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:56.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.282+0000 7f83ded9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83e0107c90 0x7f83e019f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59768/0 (socket says 172.21.15.12:59768) 2024-04-22T13:23:56.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.282+0000 7f83ded9d640 1 -- 172.21.15.12:0/2664103345 learned_addr learned my addr 172.21.15.12:0/2664103345 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:56.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.282+0000 7f83ddd9b640 1 --2- 172.21.15.12:0/2664103345 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83e0105360 0x7f83e01987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:56.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.282+0000 7f83ded9d640 1 -- 172.21.15.12:0/2664103345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83e0102a30 msgr2=0x7f83e0198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:56.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.282+0000 7f83ded9d640 1 --2- 172.21.15.12:0/2664103345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83e0102a30 0x7f83e0198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:56.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.282+0000 7f83ded9d640 1 -- 172.21.15.12:0/2664103345 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83e0105360 msgr2=0x7f83e01987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:56.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.282+0000 7f83ded9d640 1 --2- 172.21.15.12:0/2664103345 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83e0105360 0x7f83e01987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:56.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.282+0000 7f83ded9d640 1 -- 172.21.15.12:0/2664103345 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e019fea0 con 0x7f83e0107c90 2024-04-22T13:23:56.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.283+0000 7f83ded9d640 1 --2- 172.21.15.12:0/2664103345 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83e0107c90 0x7f83e019f830 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009fd0 tx=0x7f83d800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:56.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.283+0000 7f83cf7fe640 1 -- 172.21.15.12:0/2664103345 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d8011830 con 0x7f83e0107c90 2024-04-22T13:23:56.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.283+0000 7f83e4a14640 1 -- 172.21.15.12:0/2664103345 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83e01a0190 con 0x7f83e0107c90 2024-04-22T13:23:56.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.283+0000 7f83cf7fe640 1 -- 172.21.15.12:0/2664103345 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83d8016070 con 0x7f83e0107c90 2024-04-22T13:23:56.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.283+0000 7f83cf7fe640 1 -- 172.21.15.12:0/2664103345 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d8010de0 con 0x7f83e0107c90 2024-04-22T13:23:56.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.283+0000 7f83e4a14640 1 -- 172.21.15.12:0/2664103345 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83e01a0690 con 0x7f83e0107c90 2024-04-22T13:23:56.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.284+0000 7f83e4a14640 1 -- 172.21.15.12:0/2664103345 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83e00fdf30 con 0x7f83e0107c90 2024-04-22T13:23:56.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.285+0000 7f83cf7fe640 1 -- 172.21.15.12:0/2664103345 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f83d80119d0 con 0x7f83e0107c90 2024-04-22T13:23:56.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.286+0000 7f83cf7fe640 1 --2- 172.21.15.12:0/2664103345 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f83c80a3bb0 0x7f83c80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:56.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.286+0000 7f83de59c640 1 --2- 172.21.15.12:0/2664103345 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f83c80a3bb0 0x7f83c80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:56.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.286+0000 7f83cf7fe640 1 -- 172.21.15.12:0/2664103345 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f83d80cfa90 con 0x7f83e0107c90 2024-04-22T13:23:56.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.286+0000 7f83de59c640 1 --2- 172.21.15.12:0/2664103345 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f83c80a3bb0 0x7f83c80a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f83d0032520 tx=0x7f83d0000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:56.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.290+0000 7f83cf7fe640 1 -- 172.21.15.12:0/2664103345 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f83d809c440 con 0x7f83e0107c90 2024-04-22T13:23:56.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.476+0000 7f83e4a14640 1 -- 172.21.15.12:0/2664103345 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f83e0062b50 con 0x7f83e0107c90 2024-04-22T13:23:56.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.476+0000 7f83cf7fe640 1 -- 172.21.15.12:0/2664103345 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f83d809bd40 con 0x7f83e0107c90 2024-04-22T13:23:56.473 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-22T13:23:56.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.478+0000 7f83e4a14640 1 -- 172.21.15.12:0/2664103345 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f83c80a3bb0 msgr2=0x7f83c80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:56.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.478+0000 7f83e4a14640 1 --2- 172.21.15.12:0/2664103345 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f83c80a3bb0 0x7f83c80a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f83d0032520 tx=0x7f83d0000f40 comp rx=0 tx=0).stop 2024-04-22T13:23:56.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.478+0000 7f83e4a14640 1 -- 172.21.15.12:0/2664103345 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83e0107c90 msgr2=0x7f83e019f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:56.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.478+0000 7f83e4a14640 1 --2- 172.21.15.12:0/2664103345 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83e0107c90 0x7f83e019f830 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009fd0 tx=0x7f83d800b8e0 comp rx=0 tx=0).stop 2024-04-22T13:23:56.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.478+0000 7f83e4a14640 1 -- 172.21.15.12:0/2664103345 shutdown_connections 2024-04-22T13:23:56.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.478+0000 7f83e4a14640 1 --2- 172.21.15.12:0/2664103345 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f83c80a3bb0 0x7f83c80a6070 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:56.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.478+0000 7f83e4a14640 1 --2- 172.21.15.12:0/2664103345 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f83e0107c90 0x7f83e019f830 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:56.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.478+0000 7f83e4a14640 1 --2- 172.21.15.12:0/2664103345 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83e0105360 0x7f83e01987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:56.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.478+0000 7f83e4a14640 1 --2- 172.21.15.12:0/2664103345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83e0102a30 0x7f83e0198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:56.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.478+0000 7f83e4a14640 1 -- 172.21.15.12:0/2664103345 >> 172.21.15.12:0/2664103345 conn(0x7f83e00fc440 msgr2=0x7f83e0103490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:56.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.478+0000 7f83e4a14640 1 -- 172.21.15.12:0/2664103345 shutdown_connections 2024-04-22T13:23:56.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.478+0000 7f83e4a14640 1 -- 172.21.15.12:0/2664103345 wait complete. 2024-04-22T13:23:56.486 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T13:23:56.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.593+0000 7fc5b623f640 1 -- 172.21.15.12:0/795895852 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5b0107c90 msgr2=0x7fc5b010a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:56.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.593+0000 7fc5b623f640 1 --2- 172.21.15.12:0/795895852 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5b0107c90 0x7fc5b010a0e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8009ca0 tx=0x7fc5a802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:23:56.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.593+0000 7fc5b623f640 1 -- 172.21.15.12:0/795895852 shutdown_connections 2024-04-22T13:23:56.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.593+0000 7fc5b623f640 1 --2- 172.21.15.12:0/795895852 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5b0107c90 0x7fc5b010a0e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:56.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.593+0000 7fc5b623f640 1 --2- 172.21.15.12:0/795895852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5b0105360 0x7fc5b0107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:56.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.593+0000 7fc5b623f640 1 --2- 172.21.15.12:0/795895852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5b0102a30 0x7fc5b0104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:56.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.593+0000 7fc5b623f640 1 -- 172.21.15.12:0/795895852 >> 172.21.15.12:0/795895852 conn(0x7fc5b00fc440 msgr2=0x7fc5b00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:56.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.593+0000 7fc5b623f640 1 -- 172.21.15.12:0/795895852 shutdown_connections 2024-04-22T13:23:56.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.593+0000 7fc5b623f640 1 -- 172.21.15.12:0/795895852 wait complete. 2024-04-22T13:23:56.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.594+0000 7fc5b623f640 1 Processor -- start 2024-04-22T13:23:56.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.594+0000 7fc5b623f640 1 -- start start 2024-04-22T13:23:56.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.594+0000 7fc5b623f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5b0102a30 0x7fc5b01023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:56.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.594+0000 7fc5b623f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5b0105360 0x7fc5b01009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:56.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.594+0000 7fc5b623f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5b0107c90 0x7fc5b0100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:56.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.594+0000 7fc5b623f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b010db50 con 0x7fc5b0102a30 2024-04-22T13:23:56.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.594+0000 7fc5b623f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b010d9d0 con 0x7fc5b0105360 2024-04-22T13:23:56.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.594+0000 7fc5b623f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b010dcd0 con 0x7fc5b0107c90 2024-04-22T13:23:56.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.594+0000 7fc5af7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5b0102a30 0x7fc5b01023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:56.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.594+0000 7fc5af7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5b0102a30 0x7fc5b01023a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59782/0 (socket says 172.21.15.12:59782) 2024-04-22T13:23:56.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.594+0000 7fc5af7fe640 1 -- 172.21.15.12:0/1994035554 learned_addr learned my addr 172.21.15.12:0/1994035554 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:56.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.594+0000 7fc5affff640 1 --2- 172.21.15.12:0/1994035554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5b0107c90 0x7fc5b0100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:56.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.594+0000 7fc5aeffd640 1 --2- 172.21.15.12:0/1994035554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5b0105360 0x7fc5b01009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:56.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.595+0000 7fc5af7fe640 1 -- 172.21.15.12:0/1994035554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5b0107c90 msgr2=0x7fc5b0100f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:56.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.595+0000 7fc5af7fe640 1 --2- 172.21.15.12:0/1994035554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5b0107c90 0x7fc5b0100f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:56.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.595+0000 7fc5af7fe640 1 -- 172.21.15.12:0/1994035554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5b0105360 msgr2=0x7fc5b01009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:56.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.595+0000 7fc5af7fe640 1 --2- 172.21.15.12:0/1994035554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5b0105360 0x7fc5b01009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:56.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.595+0000 7fc5af7fe640 1 -- 172.21.15.12:0/1994035554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b01017b0 con 0x7fc5b0102a30 2024-04-22T13:23:56.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.595+0000 7fc5affff640 1 --2- 172.21.15.12:0/1994035554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5b0107c90 0x7fc5b0100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:23:56.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.595+0000 7fc5af7fe640 1 --2- 172.21.15.12:0/1994035554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5b0102a30 0x7fc5b01023a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc5a000b4a0 tx=0x7fc5a000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:56.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.595+0000 7fc5acff9640 1 -- 172.21.15.12:0/1994035554 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0014070 con 0x7fc5b0102a30 2024-04-22T13:23:56.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.595+0000 7fc5acff9640 1 -- 172.21.15.12:0/1994035554 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5a0010ac0 con 0x7fc5b0102a30 2024-04-22T13:23:56.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.595+0000 7fc5acff9640 1 -- 172.21.15.12:0/1994035554 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0017c30 con 0x7fc5b0102a30 2024-04-22T13:23:56.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.595+0000 7fc5b623f640 1 -- 172.21.15.12:0/1994035554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5b01a2340 con 0x7fc5b0102a30 2024-04-22T13:23:56.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.595+0000 7fc5b623f640 1 -- 172.21.15.12:0/1994035554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5b01a2890 con 0x7fc5b0102a30 2024-04-22T13:23:56.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.596+0000 7fc5b623f640 1 -- 172.21.15.12:0/1994035554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5b00fdf30 con 0x7fc5b0102a30 2024-04-22T13:23:56.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.597+0000 7fc5acff9640 1 -- 172.21.15.12:0/1994035554 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc5a0017070 con 0x7fc5b0102a30 2024-04-22T13:23:56.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.597+0000 7fc5acff9640 1 --2- 172.21.15.12:0/1994035554 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc5980a3c30 0x7fc5980a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:56.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.598+0000 7fc5aeffd640 1 --2- 172.21.15.12:0/1994035554 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc5980a3c30 0x7fc5980a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:56.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.598+0000 7fc5acff9640 1 -- 172.21.15.12:0/1994035554 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fc5a00cf800 con 0x7fc5b0102a30 2024-04-22T13:23:56.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.598+0000 7fc5aeffd640 1 --2- 172.21.15.12:0/1994035554 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc5980a3c30 0x7fc5980a60f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0102110 tx=0x7fc5a400a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:56.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.602+0000 7fc5acff9640 1 -- 172.21.15.12:0/1994035554 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc5a009c210 con 0x7fc5b0102a30 2024-04-22T13:23:56.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.788+0000 7fc5b623f640 1 -- 172.21.15.12:0/1994035554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc5b0062b50 con 0x7fc5b0102a30 2024-04-22T13:23:56.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.789+0000 7fc5acff9640 1 -- 172.21.15.12:0/1994035554 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7fc5a009bb10 con 0x7fc5b0102a30 2024-04-22T13:23:56.786 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-22T13:23:56.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.791+0000 7fc5b623f640 1 -- 172.21.15.12:0/1994035554 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc5980a3c30 msgr2=0x7fc5980a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:56.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.791+0000 7fc5b623f640 1 --2- 172.21.15.12:0/1994035554 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc5980a3c30 0x7fc5980a60f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0102110 tx=0x7fc5a400a4c0 comp rx=0 tx=0).stop 2024-04-22T13:23:56.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.791+0000 7fc5b623f640 1 -- 172.21.15.12:0/1994035554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5b0102a30 msgr2=0x7fc5b01023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:56.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.791+0000 7fc5b623f640 1 --2- 172.21.15.12:0/1994035554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5b0102a30 0x7fc5b01023a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc5a000b4a0 tx=0x7fc5a000b960 comp rx=0 tx=0).stop 2024-04-22T13:23:56.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.791+0000 7fc5b623f640 1 -- 172.21.15.12:0/1994035554 shutdown_connections 2024-04-22T13:23:56.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.791+0000 7fc5b623f640 1 --2- 172.21.15.12:0/1994035554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5b0107c90 0x7fc5b0100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:56.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.791+0000 7fc5b623f640 1 --2- 172.21.15.12:0/1994035554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5b0105360 0x7fc5b01009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:56.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.791+0000 7fc5b623f640 1 --2- 172.21.15.12:0/1994035554 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc5980a3c30 0x7fc5980a60f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:56.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.791+0000 7fc5b623f640 1 --2- 172.21.15.12:0/1994035554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5b0102a30 0x7fc5b01023a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:56.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.791+0000 7fc5b623f640 1 -- 172.21.15.12:0/1994035554 >> 172.21.15.12:0/1994035554 conn(0x7fc5b00fc440 msgr2=0x7fc5b0103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:56.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.791+0000 7fc5b623f640 1 -- 172.21.15.12:0/1994035554 shutdown_connections 2024-04-22T13:23:56.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.791+0000 7fc5b623f640 1 -- 172.21.15.12:0/1994035554 wait complete. 2024-04-22T13:23:56.798 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:56 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2147216364' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:23:56.798 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:56 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1090078053' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:23:56.798 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:56 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2664103345' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:23:56.799 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:23:56.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.903+0000 7f85f937e640 1 -- 172.21.15.12:0/393239065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85f4101f80 msgr2=0x7f85f41023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:56.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.903+0000 7f85f937e640 1 --2- 172.21.15.12:0/393239065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85f4101f80 0x7f85f41023a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009900 tx=0x7f85e402bc30 comp rx=0 tx=0).stop 2024-04-22T13:23:56.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.903+0000 7f85f937e640 1 -- 172.21.15.12:0/393239065 shutdown_connections 2024-04-22T13:23:56.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.903+0000 7f85f937e640 1 --2- 172.21.15.12:0/393239065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85f4103b30 0x7f85f410a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:56.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.903+0000 7f85f937e640 1 --2- 172.21.15.12:0/393239065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85f4103170 0x7f85f41035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:56.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.903+0000 7f85f937e640 1 --2- 172.21.15.12:0/393239065 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85f4101f80 0x7f85f41023a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:56.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.903+0000 7f85f937e640 1 -- 172.21.15.12:0/393239065 >> 172.21.15.12:0/393239065 conn(0x7f85f40fd530 msgr2=0x7f85f40ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:56.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.903+0000 7f85f937e640 1 -- 172.21.15.12:0/393239065 shutdown_connections 2024-04-22T13:23:56.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.903+0000 7f85f937e640 1 -- 172.21.15.12:0/393239065 wait complete. 2024-04-22T13:23:56.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.904+0000 7f85f937e640 1 Processor -- start 2024-04-22T13:23:56.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.904+0000 7f85f937e640 1 -- start start 2024-04-22T13:23:56.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.904+0000 7f85f937e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85f4101f80 0x7f85f4198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:56.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.904+0000 7f85f937e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85f4103170 0x7f85f41986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:56.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.904+0000 7f85f937e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85f4103b30 0x7f85f419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:56.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.904+0000 7f85f937e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f85f410dac0 con 0x7f85f4103170 2024-04-22T13:23:56.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.904+0000 7f85f937e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f85f410d940 con 0x7f85f4103b30 2024-04-22T13:23:56.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.904+0000 7f85f937e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f85f410dc40 con 0x7f85f4101f80 2024-04-22T13:23:56.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.904+0000 7f85f27fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85f4103170 0x7f85f41986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:56.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.904+0000 7f85f27fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85f4103170 0x7f85f41986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59790/0 (socket says 172.21.15.12:59790) 2024-04-22T13:23:56.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.904+0000 7f85f27fc640 1 -- 172.21.15.12:0/3979169932 learned_addr learned my addr 172.21.15.12:0/3979169932 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:56.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.904+0000 7f85f2ffd640 1 --2- 172.21.15.12:0/3979169932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85f4101f80 0x7f85f4198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:56.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.905+0000 7f85f37fe640 1 --2- 172.21.15.12:0/3979169932 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85f4103b30 0x7f85f419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:56.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.905+0000 7f85f27fc640 1 -- 172.21.15.12:0/3979169932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85f4101f80 msgr2=0x7f85f4198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:56.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.905+0000 7f85f27fc640 1 --2- 172.21.15.12:0/3979169932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85f4101f80 0x7f85f4198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:56.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.905+0000 7f85f27fc640 1 -- 172.21.15.12:0/3979169932 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85f4103b30 msgr2=0x7f85f419f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:56.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.905+0000 7f85f27fc640 1 --2- 172.21.15.12:0/3979169932 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85f4103b30 0x7f85f419f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:56.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.905+0000 7f85f27fc640 1 -- 172.21.15.12:0/3979169932 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f419fd90 con 0x7f85f4103170 2024-04-22T13:23:56.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.905+0000 7f85f37fe640 1 --2- 172.21.15.12:0/3979169932 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85f4103b30 0x7f85f419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:23:56.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.905+0000 7f85f27fc640 1 --2- 172.21.15.12:0/3979169932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85f4103170 0x7f85f41986b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f85e800c920 tx=0x7f85e800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:56.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.905+0000 7f85dbfff640 1 -- 172.21.15.12:0/3979169932 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e8007c70 con 0x7f85f4103170 2024-04-22T13:23:56.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.905+0000 7f85f937e640 1 -- 172.21.15.12:0/3979169932 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85f41a0080 con 0x7f85f4103170 2024-04-22T13:23:56.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.905+0000 7f85dbfff640 1 -- 172.21.15.12:0/3979169932 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85e8007e10 con 0x7f85f4103170 2024-04-22T13:23:56.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.905+0000 7f85dbfff640 1 -- 172.21.15.12:0/3979169932 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e8018820 con 0x7f85f4103170 2024-04-22T13:23:56.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.905+0000 7f85f937e640 1 -- 172.21.15.12:0/3979169932 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85f41a0620 con 0x7f85f4103170 2024-04-22T13:23:56.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.906+0000 7f85f937e640 1 -- 172.21.15.12:0/3979169932 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85f4109370 con 0x7f85f4103170 2024-04-22T13:23:56.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.907+0000 7f85dbfff640 1 -- 172.21.15.12:0/3979169932 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f85e800f450 con 0x7f85f4103170 2024-04-22T13:23:56.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.908+0000 7f85dbfff640 1 --2- 172.21.15.12:0/3979169932 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f85dc0a3bb0 0x7f85dc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:56.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.908+0000 7f85f2ffd640 1 --2- 172.21.15.12:0/3979169932 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f85dc0a3bb0 0x7f85dc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:56.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.908+0000 7f85dbfff640 1 -- 172.21.15.12:0/3979169932 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f85e80d0520 con 0x7f85f4103170 2024-04-22T13:23:56.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.908+0000 7f85f2ffd640 1 --2- 172.21.15.12:0/3979169932 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f85dc0a3bb0 0x7f85dc0a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009230 tx=0x7f85e4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:56.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:56.912+0000 7f85dbfff640 1 -- 172.21.15.12:0/3979169932 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f85e8014030 con 0x7f85f4103170 2024-04-22T13:23:56.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:56 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2147216364' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:23:56.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:56 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1090078053' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:23:56.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:56 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2664103345' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:23:57.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.138+0000 7f85f937e640 1 -- 172.21.15.12:0/3979169932 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85f4062b50 con 0x7f85f4103170 2024-04-22T13:23:57.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.139+0000 7f85dbfff640 1 -- 172.21.15.12:0/3979169932 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11942 (secure 0 0 0) 0x7f85e809cf30 con 0x7f85f4103170 2024-04-22T13:23:57.136 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:23:57.136 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:22:12.398056+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24448,24509,24443,24484,24407]},"id":1}]} 2024-04-22T13:23:57.136 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 27 2024-04-22T13:23:57.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.141+0000 7f85f937e640 1 -- 172.21.15.12:0/3979169932 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f85dc0a3bb0 msgr2=0x7f85dc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.141+0000 7f85f937e640 1 --2- 172.21.15.12:0/3979169932 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f85dc0a3bb0 0x7f85dc0a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009230 tx=0x7f85e4005fd0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.141+0000 7f85f937e640 1 -- 172.21.15.12:0/3979169932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85f4103170 msgr2=0x7f85f41986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.141+0000 7f85f937e640 1 --2- 172.21.15.12:0/3979169932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85f4103170 0x7f85f41986b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f85e800c920 tx=0x7f85e800cde0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.141+0000 7f85f937e640 1 -- 172.21.15.12:0/3979169932 shutdown_connections 2024-04-22T13:23:57.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.141+0000 7f85f937e640 1 --2- 172.21.15.12:0/3979169932 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85f4103b30 0x7f85f419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.141+0000 7f85f937e640 1 --2- 172.21.15.12:0/3979169932 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f85dc0a3bb0 0x7f85dc0a6070 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.141+0000 7f85f937e640 1 --2- 172.21.15.12:0/3979169932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85f4103170 0x7f85f41986b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.141+0000 7f85f937e640 1 --2- 172.21.15.12:0/3979169932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85f4101f80 0x7f85f4198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.141+0000 7f85f937e640 1 -- 172.21.15.12:0/3979169932 >> 172.21.15.12:0/3979169932 conn(0x7f85f40fd530 msgr2=0x7f85f4106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:57.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.141+0000 7f85f937e640 1 -- 172.21.15.12:0/3979169932 shutdown_connections 2024-04-22T13:23:57.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.141+0000 7f85f937e640 1 -- 172.21.15.12:0/3979169932 wait complete. 2024-04-22T13:23:57.149 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:23:57.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:56 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2147216364' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:23:57.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:56 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1090078053' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:23:57.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:56 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2664103345' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:23:57.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.255+0000 7fc8b4d04640 1 -- 172.21.15.12:0/3321177600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8b0102980 msgr2=0x7fc8b01092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.255+0000 7fc8b4d04640 1 --2- 172.21.15.12:0/3321177600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8b0102980 0x7fc8b01092a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009ca0 tx=0x7fc8a802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.255+0000 7fc8b4d04640 1 -- 172.21.15.12:0/3321177600 shutdown_connections 2024-04-22T13:23:57.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.255+0000 7fc8b4d04640 1 --2- 172.21.15.12:0/3321177600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8b0102980 0x7fc8b01092a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.255+0000 7fc8b4d04640 1 --2- 172.21.15.12:0/3321177600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8b0101fc0 0x7fc8b0102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.255+0000 7fc8b4d04640 1 --2- 172.21.15.12:0/3321177600 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8b0100e70 0x7fc8b0101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.255+0000 7fc8b4d04640 1 -- 172.21.15.12:0/3321177600 >> 172.21.15.12:0/3321177600 conn(0x7fc8b00fc3e0 msgr2=0x7fc8b00fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:57.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.255+0000 7fc8b4d04640 1 -- 172.21.15.12:0/3321177600 shutdown_connections 2024-04-22T13:23:57.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.255+0000 7fc8b4d04640 1 -- 172.21.15.12:0/3321177600 wait complete. 2024-04-22T13:23:57.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.256+0000 7fc8b4d04640 1 Processor -- start 2024-04-22T13:23:57.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.256+0000 7fc8b4d04640 1 -- start start 2024-04-22T13:23:57.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.256+0000 7fc8b4d04640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8b0100e70 0x7fc8b0069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.256+0000 7fc8b4d04640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8b0101fc0 0x7fc8b019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.256+0000 7fc8b4d04640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8b0102980 0x7fc8b019f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.256+0000 7fc8b4d04640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b010c9d0 con 0x7fc8b0102980 2024-04-22T13:23:57.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.256+0000 7fc8b4d04640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b010c850 con 0x7fc8b0101fc0 2024-04-22T13:23:57.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.256+0000 7fc8b4d04640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b010cb50 con 0x7fc8b0100e70 2024-04-22T13:23:57.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.257+0000 7fc8aed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8b0102980 0x7fc8b019f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.257+0000 7fc8aed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8b0102980 0x7fc8b019f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59808/0 (socket says 172.21.15.12:59808) 2024-04-22T13:23:57.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.257+0000 7fc8ae59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8b0100e70 0x7fc8b0069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.257+0000 7fc8aed9d640 1 -- 172.21.15.12:0/484665437 learned_addr learned my addr 172.21.15.12:0/484665437 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:57.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.257+0000 7fc8add9b640 1 --2- 172.21.15.12:0/484665437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8b0101fc0 0x7fc8b019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.257+0000 7fc8aed9d640 1 -- 172.21.15.12:0/484665437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8b0100e70 msgr2=0x7fc8b0069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.257+0000 7fc8aed9d640 1 --2- 172.21.15.12:0/484665437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8b0100e70 0x7fc8b0069c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.257+0000 7fc8aed9d640 1 -- 172.21.15.12:0/484665437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8b0101fc0 msgr2=0x7fc8b019ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.257+0000 7fc8aed9d640 1 --2- 172.21.15.12:0/484665437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8b0101fc0 0x7fc8b019ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.257+0000 7fc8aed9d640 1 -- 172.21.15.12:0/484665437 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b019fda0 con 0x7fc8b0102980 2024-04-22T13:23:57.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.257+0000 7fc8aed9d640 1 --2- 172.21.15.12:0/484665437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8b0102980 0x7fc8b019f490 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc8a802c700 tx=0x7fc8a80346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:57.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.257+0000 7fc89f7fe640 1 -- 172.21.15.12:0/484665437 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8041020 con 0x7fc8b0102980 2024-04-22T13:23:57.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.257+0000 7fc89f7fe640 1 -- 172.21.15.12:0/484665437 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8a8034ca0 con 0x7fc8b0102980 2024-04-22T13:23:57.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.257+0000 7fc8b4d04640 1 -- 172.21.15.12:0/484665437 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8b019ffd0 con 0x7fc8b0102980 2024-04-22T13:23:57.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.257+0000 7fc89f7fe640 1 -- 172.21.15.12:0/484665437 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8039040 con 0x7fc8b0102980 2024-04-22T13:23:57.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.257+0000 7fc8b4d04640 1 -- 172.21.15.12:0/484665437 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8b01a0490 con 0x7fc8b0102980 2024-04-22T13:23:57.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.258+0000 7fc8b4d04640 1 -- 172.21.15.12:0/484665437 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8b0108180 con 0x7fc8b0102980 2024-04-22T13:23:57.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.259+0000 7fc89f7fe640 1 -- 172.21.15.12:0/484665437 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc8a8044020 con 0x7fc8b0102980 2024-04-22T13:23:57.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.260+0000 7fc89f7fe640 1 --2- 172.21.15.12:0/484665437 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc8980a3c30 0x7fc8980a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.260+0000 7fc8ae59c640 1 --2- 172.21.15.12:0/484665437 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc8980a3c30 0x7fc8980a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.260+0000 7fc89f7fe640 1 -- 172.21.15.12:0/484665437 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fc8a80f20c0 con 0x7fc8b0102980 2024-04-22T13:23:57.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.260+0000 7fc8ae59c640 1 --2- 172.21.15.12:0/484665437 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc8980a3c30 0x7fc8980a60f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0009bd0 tx=0x7fc8a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:57.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.265+0000 7fc89f7fe640 1 -- 172.21.15.12:0/484665437 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc8a804b020 con 0x7fc8b0102980 2024-04-22T13:23:57.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.454+0000 7fc8b4d04640 1 -- 172.21.15.12:0/484665437 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8b0062bf0 con 0x7fc8b0102980 2024-04-22T13:23:57.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.456+0000 7fc89f7fe640 1 -- 172.21.15.12:0/484665437 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22620 (secure 0 0 0) 0x7fc8a80bea70 con 0x7fc8b0102980 2024-04-22T13:23:57.453 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:23:57.453 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":89,"fsid":"0aede5ea-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:14:09.362918+0000","modified":"2024-04-22T13:23:05.500756+0000","last_up_change":"2024-04-22T13:21:15.263393+0000","last_in_change":"2024-04-22T13:20:54.174685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:17:32.372434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:22:02.768799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:22:03.578730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d9e2365-3a64-4b13-96fc-c217b1cee2ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6803","nonce":1191350632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6805","nonce":1191350632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6809","nonce":1191350632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6807","nonce":1191350632}]},"public_addr":"172.21.15.12:6803/1191350632","cluster_addr":"172.21.15.12:6805/1191350632","heartbeat_back_addr":"172.21.15.12:6809/1191350632","heartbeat_front_addr":"172.21.15.12:6807/1191350632","state":["exists","up"]},{"osd":1,"uuid":"1499c783-2bd7-48ce-a444-34add42bcf31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6801","nonce":417496465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6803","nonce":417496465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6807","nonce":417496465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6805","nonce":417496465}]},"public_addr":"172.21.15.111:6801/417496465","cluster_addr":"172.21.15.111:6803/417496465","heartbeat_back_addr":"172.21.15.111:6807/417496465","heartbeat_front_addr":"172.21.15.111:6805/417496465","state":["exists","up"]},{"osd":2,"uuid":"71a9ebc3-c622-434e-a6c2-f48358f384f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6801","nonce":1403153998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6803","nonce":1403153998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6807","nonce":1403153998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6805","nonce":1403153998}]},"public_addr":"172.21.15.134:6801/1403153998","cluster_addr":"172.21.15.134:6803/1403153998","heartbeat_back_addr":"172.21.15.134:6807/1403153998","heartbeat_front_addr":"172.21.15.134:6805/1403153998","state":["exists","up"]},{"osd":3,"uuid":"6dd00db2-a286-48ba-8666-56ffdd5f35ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6811","nonce":3359066681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6813","nonce":3359066681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6817","nonce":3359066681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6815","nonce":3359066681}]},"public_addr":"172.21.15.12:6811/3359066681","cluster_addr":"172.21.15.12:6813/3359066681","heartbeat_back_addr":"172.21.15.12:6817/3359066681","heartbeat_front_addr":"172.21.15.12:6815/3359066681","state":["exists","up"]},{"osd":4,"uuid":"1e29afd7-354c-4506-aaa0-b43eb1a19a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6809","nonce":2886349872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6811","nonce":2886349872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6815","nonce":2886349872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6813","nonce":2886349872}]},"public_addr":"172.21.15.111:6809/2886349872","cluster_addr":"172.21.15.111:6811/2886349872","heartbeat_back_addr":"172.21.15.111:6815/2886349872","heartbeat_front_addr":"172.21.15.111:6813/2886349872","state":["exists","up"]},{"osd":5,"uuid":"8f40f02d-0ba2-4842-a03d-2a1a04d1010d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6809","nonce":3071977602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6811","nonce":3071977602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6815","nonce":3071977602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6813","nonce":3071977602}]},"public_addr":"172.21.15.134:6809/3071977602","cluster_addr":"172.21.15.134:6811/3071977602","heartbeat_back_addr":"172.21.15.134:6815/3071977602","heartbeat_front_addr":"172.21.15.134:6813/3071977602","state":["exists","up"]},{"osd":6,"uuid":"bb0c17a9-6a8e-4be6-8660-f73cfa8200ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6819","nonce":3819619299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6821","nonce":3819619299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6825","nonce":3819619299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6823","nonce":3819619299}]},"public_addr":"172.21.15.12:6819/3819619299","cluster_addr":"172.21.15.12:6821/3819619299","heartbeat_back_addr":"172.21.15.12:6825/3819619299","heartbeat_front_addr":"172.21.15.12:6823/3819619299","state":["exists","up"]},{"osd":7,"uuid":"0a93d6af-569e-4e45-b697-9e978c1c1004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6817","nonce":3113856282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6819","nonce":3113856282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6823","nonce":3113856282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6821","nonce":3113856282}]},"public_addr":"172.21.15.111:6817/3113856282","cluster_addr":"172.21.15.111:6819/3113856282","heartbeat_back_addr":"172.21.15.111:6823/3113856282","heartbeat_front_addr":"172.21.15.111:6821/3113856282","state":["exists","up"]},{"osd":8,"uuid":"e21acc68-cf50-4f4f-b1ba-cf6c195b4955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6817","nonce":1689362191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6819","nonce":1689362191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6823","nonce":1689362191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6821","nonce":1689362191}]},"public_addr":"172.21.15.134:6817/1689362191","cluster_addr":"172.21.15.134:6819/1689362191","heartbeat_back_addr":"172.21.15.134:6823/1689362191","heartbeat_front_addr":"172.21.15.134:6821/1689362191","state":["exists","up"]},{"osd":9,"uuid":"fd745869-7f5f-48a8-b412-bb9c92b586e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6827","nonce":4292385779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6829","nonce":4292385779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6833","nonce":4292385779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6831","nonce":4292385779}]},"public_addr":"172.21.15.12:6827/4292385779","cluster_addr":"172.21.15.12:6829/4292385779","heartbeat_back_addr":"172.21.15.12:6833/4292385779","heartbeat_front_addr":"172.21.15.12:6831/4292385779","state":["exists","up"]},{"osd":10,"uuid":"798b1abd-f6c7-4a5f-8da0-fdae49e50d2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6825","nonce":3338566976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6827","nonce":3338566976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6831","nonce":3338566976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6829","nonce":3338566976}]},"public_addr":"172.21.15.111:6825/3338566976","cluster_addr":"172.21.15.111:6827/3338566976","heartbeat_back_addr":"172.21.15.111:6831/3338566976","heartbeat_front_addr":"172.21.15.111:6829/3338566976","state":["exists","up"]},{"osd":11,"uuid":"b18fc5f3-1049-4871-b060-e99290273cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6825","nonce":578640428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6827","nonce":578640428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6831","nonce":578640428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6829","nonce":578640428}]},"public_addr":"172.21.15.134:6825/578640428","cluster_addr":"172.21.15.134:6827/578640428","heartbeat_back_addr":"172.21.15.134:6831/578640428","heartbeat_front_addr":"172.21.15.134:6829/578640428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:16:40.828608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:05.052528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:28.297316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:53.496497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:17.947259+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:41.712894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:07.105945+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:32.129688+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:56.008516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:22.239867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:47.334692+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:12.301276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/2593601362":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/2519382069":"2024-04-23T13:15:04.279990+0000","172.21.15.12:6801/2009831940":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3509412187":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1007749062":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3588480095":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1904962176":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2009831940":"2024-04-23T13:15:04.279990+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:23:57.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.458+0000 7fc8b4d04640 1 -- 172.21.15.12:0/484665437 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc8980a3c30 msgr2=0x7fc8980a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.458+0000 7fc8b4d04640 1 --2- 172.21.15.12:0/484665437 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc8980a3c30 0x7fc8980a60f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0009bd0 tx=0x7fc8a0005f90 comp rx=0 tx=0).stop 2024-04-22T13:23:57.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.458+0000 7fc8b4d04640 1 -- 172.21.15.12:0/484665437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8b0102980 msgr2=0x7fc8b019f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.458+0000 7fc8b4d04640 1 --2- 172.21.15.12:0/484665437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8b0102980 0x7fc8b019f490 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc8a802c700 tx=0x7fc8a80346d0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.458+0000 7fc8b4d04640 1 -- 172.21.15.12:0/484665437 shutdown_connections 2024-04-22T13:23:57.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.458+0000 7fc8b4d04640 1 --2- 172.21.15.12:0/484665437 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fc8980a3c30 0x7fc8980a60f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.458+0000 7fc8b4d04640 1 --2- 172.21.15.12:0/484665437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc8b0102980 0x7fc8b019f490 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.458+0000 7fc8b4d04640 1 --2- 172.21.15.12:0/484665437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8b0101fc0 0x7fc8b019ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.458+0000 7fc8b4d04640 1 --2- 172.21.15.12:0/484665437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8b0100e70 0x7fc8b0069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.458+0000 7fc8b4d04640 1 -- 172.21.15.12:0/484665437 >> 172.21.15.12:0/484665437 conn(0x7fc8b00fc3e0 msgr2=0x7fc8b0105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:57.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.458+0000 7fc8b4d04640 1 -- 172.21.15.12:0/484665437 shutdown_connections 2024-04-22T13:23:57.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.458+0000 7fc8b4d04640 1 -- 172.21.15.12:0/484665437 wait complete. 2024-04-22T13:23:57.467 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-22T13:22:04.602553+0000', 'modified': '2024-04-22T13:22:12.398056+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24509, 'mds_1': 24484, 'mds_2': 24448, 'mds_3': 24443, 'mds_4': 24407}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14631': {'gid': 14631, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.12:6839/4083188176', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6838', 'nonce': 4083188176}, {'type': 'v1', 'addr': '172.21.15.12:6839', 'nonce': 4083188176}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14643': {'gid': 14643, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.12:6841/763294965', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6840', 'nonce': 763294965}, {'type': 'v1', 'addr': '172.21.15.12:6841', 'nonce': 763294965}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24406': {'gid': 24406, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6833/4235830781', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 4235830781}, {'type': 'v1', 'addr': '172.21.15.134:6833', 'nonce': 4235830781}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'k', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.111:6833/1902687060', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6832', 'nonce': 1902687060}, {'type': 'v1', 'addr': '172.21.15.111:6833', 'nonce': 1902687060}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'e', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.111:6835/2665640170', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6834', 'nonce': 2665640170}, {'type': 'v1', 'addr': '172.21.15.111:6835', 'nonce': 2665640170}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'f', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.134:6835/3927126579', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 3927126579}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 3927126579}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6837/3631577545', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6836', 'nonce': 3631577545}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 3631577545}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.111:6837/4218407470', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6836', 'nonce': 4218407470}, {'type': 'v1', 'addr': '172.21.15.111:6837', 'nonce': 4218407470}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.134:6839/1488513960', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 1488513960}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 1488513960}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24509': {'gid': 24509, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.111:6839/2126656770', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6838', 'nonce': 2126656770}, {'type': 'v1', 'addr': '172.21.15.111:6839', 'nonce': 2126656770}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24509, 'qdb_cluster': [24448, 24509, 24443, 24484, 24407]} 2024-04-22T13:23:57.467 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-22T13:23:57.467 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-22T13:23:57.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.573+0000 7f38c84d9640 1 -- 172.21.15.12:0/506121553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38c01031a0 msgr2=0x7f38c0103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.573+0000 7f38c84d9640 1 --2- 172.21.15.12:0/506121553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38c01031a0 0x7f38c0103620 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009900 tx=0x7f38b002bc30 comp rx=0 tx=0).stop 2024-04-22T13:23:57.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.573+0000 7f38c84d9640 1 -- 172.21.15.12:0/506121553 shutdown_connections 2024-04-22T13:23:57.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.573+0000 7f38c84d9640 1 --2- 172.21.15.12:0/506121553 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38c0103b60 0x7f38c010a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.573+0000 7f38c84d9640 1 --2- 172.21.15.12:0/506121553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38c01031a0 0x7f38c0103620 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.573+0000 7f38c84d9640 1 --2- 172.21.15.12:0/506121553 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38c0101fb0 0x7f38c01023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.573+0000 7f38c84d9640 1 -- 172.21.15.12:0/506121553 >> 172.21.15.12:0/506121553 conn(0x7f38c00fd520 msgr2=0x7f38c00ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:57.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.573+0000 7f38c84d9640 1 -- 172.21.15.12:0/506121553 shutdown_connections 2024-04-22T13:23:57.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.573+0000 7f38c84d9640 1 -- 172.21.15.12:0/506121553 wait complete. 2024-04-22T13:23:57.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.574+0000 7f38c84d9640 1 Processor -- start 2024-04-22T13:23:57.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.574+0000 7f38c84d9640 1 -- start start 2024-04-22T13:23:57.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.574+0000 7f38c84d9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38c0101fb0 0x7f38c0198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.574+0000 7f38c84d9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38c01031a0 0x7f38c01986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.574+0000 7f38c84d9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38c0103b60 0x7f38c019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.574+0000 7f38c84d9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f38c010daf0 con 0x7f38c0103b60 2024-04-22T13:23:57.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.574+0000 7f38c84d9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f38c010d970 con 0x7f38c01031a0 2024-04-22T13:23:57.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.574+0000 7f38c84d9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f38c010dc70 con 0x7f38c0101fb0 2024-04-22T13:23:57.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.574+0000 7f38c6a76640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38c0103b60 0x7f38c019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.574+0000 7f38c6a76640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38c0103b60 0x7f38c019f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59834/0 (socket says 172.21.15.12:59834) 2024-04-22T13:23:57.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.574+0000 7f38c6a76640 1 -- 172.21.15.12:0/3368032532 learned_addr learned my addr 172.21.15.12:0/3368032532 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:57.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.574+0000 7f38c6275640 1 --2- 172.21.15.12:0/3368032532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38c0101fb0 0x7f38c0198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.574+0000 7f38c5a74640 1 --2- 172.21.15.12:0/3368032532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38c01031a0 0x7f38c01986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.574+0000 7f38c6a76640 1 -- 172.21.15.12:0/3368032532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38c0101fb0 msgr2=0x7f38c0198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.574+0000 7f38c6a76640 1 --2- 172.21.15.12:0/3368032532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38c0101fb0 0x7f38c0198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.574+0000 7f38c6a76640 1 -- 172.21.15.12:0/3368032532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38c01031a0 msgr2=0x7f38c01986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.574+0000 7f38c6a76640 1 --2- 172.21.15.12:0/3368032532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38c01031a0 0x7f38c01986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.575+0000 7f38c6a76640 1 -- 172.21.15.12:0/3368032532 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c019fda0 con 0x7f38c0103b60 2024-04-22T13:23:57.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.575+0000 7f38c6275640 1 --2- 172.21.15.12:0/3368032532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38c0101fb0 0x7f38c0198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:23:57.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.575+0000 7f38c5a74640 1 --2- 172.21.15.12:0/3368032532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38c01031a0 0x7f38c01986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:23:57.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.575+0000 7f38c6a76640 1 --2- 172.21.15.12:0/3368032532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38c0103b60 0x7f38c019f730 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f38b800cce0 tx=0x7f38b800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:57.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.575+0000 7f38b77fe640 1 -- 172.21.15.12:0/3368032532 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b800f450 con 0x7f38c0103b60 2024-04-22T13:23:57.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.575+0000 7f38b77fe640 1 -- 172.21.15.12:0/3368032532 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38b800faa0 con 0x7f38c0103b60 2024-04-22T13:23:57.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.575+0000 7f38c84d9640 1 -- 172.21.15.12:0/3368032532 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38c01a0090 con 0x7f38c0103b60 2024-04-22T13:23:57.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.575+0000 7f38b77fe640 1 -- 172.21.15.12:0/3368032532 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b8018b70 con 0x7f38c0103b60 2024-04-22T13:23:57.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.575+0000 7f38c84d9640 1 -- 172.21.15.12:0/3368032532 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38c01a0590 con 0x7f38c0103b60 2024-04-22T13:23:57.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.577+0000 7f38b77fe640 1 -- 172.21.15.12:0/3368032532 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f38b8018d10 con 0x7f38c0103b60 2024-04-22T13:23:57.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.578+0000 7f38cb1a9740 1 Processor -- start 2024-04-22T13:23:57.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.578+0000 7f38cb1a9740 1 -- start start 2024-04-22T13:23:57.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.578+0000 7f38cb1a9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c9dbfb0e40 0x55c9dbfb1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.578+0000 7f38cb1a9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9dbfb1830 0x55c9dbfb1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.578+0000 7f38cb1a9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9dbfb22d0 0x55c9dbfb8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.578+0000 7f38cb1a9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c9dbf15b10 con 0x55c9dbfb1830 2024-04-22T13:23:57.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.578+0000 7f38cb1a9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c9dbeb3240 con 0x55c9dbfb0e40 2024-04-22T13:23:57.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.578+0000 7f38cb1a9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c9dbf11ad0 con 0x55c9dbfb22d0 2024-04-22T13:23:57.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.578+0000 7f38c6275640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9dbfb1830 0x55c9dbfb1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.578+0000 7f38c6275640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9dbfb1830 0x55c9dbfb1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59838/0 (socket says 172.21.15.12:59838) 2024-04-22T13:23:57.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.578+0000 7f38c6275640 1 -- 172.21.15.12:0/209048115 learned_addr learned my addr 172.21.15.12:0/209048115 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:57.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.578+0000 7f38b77fe640 1 --2- 172.21.15.12:0/3368032532 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f38ac0a3bb0 0x7f38ac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.579+0000 7f38c5a74640 1 --2- 172.21.15.12:0/209048115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c9dbfb0e40 0x55c9dbfb1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.579+0000 7f38c5a74640 1 --2- 172.21.15.12:0/209048115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9dbfb22d0 0x55c9dbfb8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.579+0000 7f38c6a76640 1 --2- 172.21.15.12:0/3368032532 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f38ac0a3bb0 0x7f38ac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.579+0000 7f38c6275640 1 -- 172.21.15.12:0/209048115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9dbfb22d0 msgr2=0x55c9dbfb8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:57.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.579+0000 7f38c6275640 1 --2- 172.21.15.12:0/209048115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9dbfb22d0 0x55c9dbfb8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.579+0000 7f38c6275640 1 -- 172.21.15.12:0/209048115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c9dbfb0e40 msgr2=0x55c9dbfb1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:57.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.579+0000 7f38c6275640 1 --2- 172.21.15.12:0/209048115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c9dbfb0e40 0x55c9dbfb1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.579+0000 7f38c6275640 1 -- 172.21.15.12:0/209048115 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9dbc282d0 con 0x55c9dbfb1830 2024-04-22T13:23:57.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.579+0000 7f38c5a74640 1 --2- 172.21.15.12:0/209048115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9dbfb22d0 0x55c9dbfb8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:23:57.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.579+0000 7f38b77fe640 1 -- 172.21.15.12:0/3368032532 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f38b80d08c0 con 0x7f38c0103b60 2024-04-22T13:23:57.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.579+0000 7f38c6275640 1 --2- 172.21.15.12:0/209048115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9dbfb1830 0x55c9dbfb1cb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009cd0 tx=0x7f38bc0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70eebdd8d02aa7d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:57.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.579+0000 7f38b4bfe640 1 -- 172.21.15.12:0/209048115 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc012450 con 0x55c9dbfb1830 2024-04-22T13:23:57.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.579+0000 7f38c6a76640 1 --2- 172.21.15.12:0/3368032532 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f38ac0a3bb0 0x7f38ac0a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f38b8099030 tx=0x7f38b80c7ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:57.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.579+0000 7f38b4bfe640 1 -- 172.21.15.12:0/209048115 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38bc012aa0 con 0x55c9dbfb1830 2024-04-22T13:23:57.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.579+0000 7f38b4bfe640 1 -- 172.21.15.12:0/209048115 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc00fbb0 con 0x55c9dbfb1830 2024-04-22T13:23:57.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.580+0000 7f38cb1a9740 1 -- 172.21.15.12:0/209048115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9dbfb1830 msgr2=0x55c9dbfb1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.580+0000 7f38cb1a9740 1 --2- 172.21.15.12:0/209048115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9dbfb1830 0x55c9dbfb1cb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009cd0 tx=0x7f38bc0117b0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.580+0000 7f38cb1a9740 1 -- 172.21.15.12:0/209048115 shutdown_connections 2024-04-22T13:23:57.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.580+0000 7f38cb1a9740 1 --2- 172.21.15.12:0/209048115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9dbfb22d0 0x55c9dbfb8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.580+0000 7f38cb1a9740 1 --2- 172.21.15.12:0/209048115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9dbfb1830 0x55c9dbfb1cb0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.580+0000 7f38cb1a9740 1 --2- 172.21.15.12:0/209048115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c9dbfb0e40 0x55c9dbfb1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.580+0000 7f38cb1a9740 1 -- 172.21.15.12:0/209048115 >> 172.21.15.12:0/209048115 conn(0x55c9dbfacf40 msgr2=0x55c9dbfab490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.580+0000 7f38cb1a9740 1 -- 172.21.15.12:0/209048115 shutdown_connections 2024-04-22T13:23:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.580+0000 7f38cb1a9740 1 -- 172.21.15.12:0/209048115 wait complete. 2024-04-22T13:23:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.580+0000 7f38cb1a9740 1 -- start start 2024-04-22T13:23:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.580+0000 7f38cb1a9740 1 Processor -- start 2024-04-22T13:23:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.581+0000 7f38cb1a9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9dbfb0e40 0x55c9dbfbae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.581+0000 7f38cb1a9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c9dbfb1830 0x55c9dbfbb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.581+0000 7f38cb1a9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9dbfb22d0 0x55c9dbfc2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.581+0000 7f38cb1a9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c9dbfb94b0 con 0x55c9dbfb0e40 2024-04-22T13:23:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.581+0000 7f38cb1a9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c9dbe9e190 con 0x55c9dbfb1830 2024-04-22T13:23:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.581+0000 7f38cb1a9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c9dbfb9630 con 0x55c9dbfb22d0 2024-04-22T13:23:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.581+0000 7f38c5a74640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9dbfb0e40 0x55c9dbfbae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.581+0000 7f38cb1a9740 10 client.0 _ll_register_callbacks cb 0x55c9dbe998b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:23:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.581+0000 7f38c5a74640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9dbfb0e40 0x55c9dbfbae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59848/0 (socket says 172.21.15.12:59848) 2024-04-22T13:23:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.581+0000 7f38c5a74640 1 -- 172.21.15.12:0/3148146974 learned_addr learned my addr 172.21.15.12:0/3148146974 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.581+0000 7f38c5a74640 1 -- 172.21.15.12:0/3148146974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9dbfb22d0 msgr2=0x55c9dbfc2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.581+0000 7f38c6275640 1 --2- 172.21.15.12:0/3148146974 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c9dbfb1830 0x55c9dbfbb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.581+0000 7f38c5a74640 1 --2- 172.21.15.12:0/3148146974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9dbfb22d0 0x55c9dbfc2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.581+0000 7f38c5a74640 1 -- 172.21.15.12:0/3148146974 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c9dbfb1830 msgr2=0x55c9dbfbb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.581+0000 7f38c5a74640 1 --2- 172.21.15.12:0/3148146974 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c9dbfb1830 0x55c9dbfbb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.581+0000 7f38c5a74640 1 -- 172.21.15.12:0/3148146974 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9dbcf26b0 con 0x55c9dbfb0e40 2024-04-22T13:23:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.581+0000 7f38c5a74640 1 --2- 172.21.15.12:0/3148146974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9dbfb0e40 0x55c9dbfbae60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009900 tx=0x7f38b0004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b88902cd77edf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.581+0000 7f38a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:23:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.581+0000 7f38a9ffb640 1 -- 172.21.15.12:0/3148146974 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b0045070 con 0x55c9dbfb0e40 2024-04-22T13:23:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.581+0000 7f38cb1a9740 1 -- 172.21.15.12:0/3148146974 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38bc012aa0 con 0x55c9dbfb0e40 2024-04-22T13:23:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.581+0000 7f38a9ffb640 1 -- 172.21.15.12:0/3148146974 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f38b0005040 con 0x55c9dbfb0e40 2024-04-22T13:23:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.582+0000 7f38a9ffb640 1 -- 172.21.15.12:0/3148146974 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f38b0051050 con 0x55c9dbfb0e40 2024-04-22T13:23:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.582+0000 7f38cb1a9740 10 client.15255 fetch_fsmap learned FSMap version 27 2024-04-22T13:23:57.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.582+0000 7f38cb1a9740 1 -- 172.21.15.12:0/3148146974 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c9dbfc56c0 con 0x55c9dbfb0e40 2024-04-22T13:23:57.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.582+0000 7f38a9ffb640 1 -- 172.21.15.12:0/3148146974 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38bc012aa0 con 0x55c9dbfb0e40 2024-04-22T13:23:57.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.582+0000 7f38a9ffb640 1 -- 172.21.15.12:0/3148146974 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f38b00035f0 con 0x55c9dbfb0e40 2024-04-22T13:23:57.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.582+0000 7f38cb1a9740 10 client.15255 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:23:57.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.582+0000 7f38cb1a9740 10 client.15255 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-22T13:23:57.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.582+0000 7f38cb1a9740 20 client.15255 populate_metadata read hostname 'smithi012' 2024-04-22T13:23:57.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.582+0000 7f38cb1a9740 1 --2- 172.21.15.12:0/3148146974 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55c9dbfc7da0 0x55c9dbfc9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.582+0000 7f38cb1a9740 4 client.15255 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:23:57.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.582+0000 7f38cb1a9740 1 -- 172.21.15.12:0/3148146974 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9dbf23530 con 0x55c9dbfc7da0 2024-04-22T13:23:57.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.582+0000 7f38c6275640 1 --2- 172.21.15.12:0/3148146974 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55c9dbfc7da0 0x55c9dbfc9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.583+0000 7f38c6275640 1 --2- 172.21.15.12:0/3148146974 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55c9dbfc7da0 0x55c9dbfc9210 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bf2c0542c96d9fa7 server_cookie=d66bb61b995622e1 in_seq=0 out_seq=0 2024-04-22T13:23:57.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.583+0000 7f38a9ffb640 10 client.15255 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:23:57.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.583+0000 7f38a9ffb640 1 -- 172.21.15.12:0/3148146974 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c9dbf23530 con 0x55c9dbfc7da0 2024-04-22T13:23:57.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.583+0000 7f38a9ffb640 10 client.15255 handle_command_reply: tid=0 2024-04-22T13:23:57.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.583+0000 7f38cb1a9740 1 client.15255 shutdown 2024-04-22T13:23:57.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.584+0000 7f38cb1a9740 1 -- 172.21.15.12:0/3148146974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9dbfb0e40 msgr2=0x55c9dbfbae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.584+0000 7f38cb1a9740 1 --2- 172.21.15.12:0/3148146974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9dbfb0e40 0x55c9dbfbae60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009900 tx=0x7f38b0004250 comp rx=0 tx=0).stop 2024-04-22T13:23:57.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.584+0000 7f38cb1a9740 1 -- 172.21.15.12:0/3148146974 shutdown_connections 2024-04-22T13:23:57.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.584+0000 7f38cb1a9740 1 --2- 172.21.15.12:0/3148146974 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55c9dbfc7da0 0x55c9dbfc9210 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.584+0000 7f38cb1a9740 1 --2- 172.21.15.12:0/3148146974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9dbfb22d0 0x55c9dbfc2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.584+0000 7f38cb1a9740 1 --2- 172.21.15.12:0/3148146974 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c9dbfb1830 0x55c9dbfbb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.584+0000 7f38a9ffb640 0 client.15255 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:23:57.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.584+0000 7f38cb1a9740 1 --2- 172.21.15.12:0/3148146974 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9dbfb0e40 0x55c9dbfbae60 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.584+0000 7f38cb1a9740 1 -- 172.21.15.12:0/3148146974 >> 172.21.15.12:0/3148146974 conn(0x55c9dbfacf40 msgr2=0x55c9dbfaaa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:57.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.584+0000 7f38cb1a9740 1 -- 172.21.15.12:0/3148146974 shutdown_connections 2024-04-22T13:23:57.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.584+0000 7f38cb1a9740 1 -- 172.21.15.12:0/3148146974 wait complete. 2024-04-22T13:23:57.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.584+0000 7f38cb1a9740 20 client.15255 trim_cache size 0 max 16384 2024-04-22T13:23:57.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.602+0000 7f38cb1a9740 1 Processor -- start 2024-04-22T13:23:57.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.602+0000 7f38cb1a9740 1 -- start start 2024-04-22T13:23:57.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.602+0000 7f38cb1a9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9dbfb0e40 0x55c9dbfb5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.602+0000 7f38cb1a9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c9dbfb1830 0x55c9dbfc2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.602+0000 7f38cb1a9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9dbfb22d0 0x55c9dbfb5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.602+0000 7f38cb1a9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c9dbfc7550 con 0x55c9dbfb22d0 2024-04-22T13:23:57.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.602+0000 7f38cb1a9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c9dbfc7c20 con 0x55c9dbfb1830 2024-04-22T13:23:57.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.602+0000 7f38cb1a9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c9dbfc82f0 con 0x55c9dbfb0e40 2024-04-22T13:23:57.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.602+0000 7f38c5a74640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9dbfb22d0 0x55c9dbfb5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.602+0000 7f38c5a74640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9dbfb22d0 0x55c9dbfb5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59856/0 (socket says 172.21.15.12:59856) 2024-04-22T13:23:57.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.602+0000 7f38c5a74640 1 -- 172.21.15.12:0/681507456 learned_addr learned my addr 172.21.15.12:0/681507456 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:57.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.602+0000 7f38c6275640 1 --2- 172.21.15.12:0/681507456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c9dbfb1830 0x55c9dbfc2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.602+0000 7f38c5a74640 1 --2- 172.21.15.12:0/681507456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9dbfb0e40 0x55c9dbfb5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.603+0000 7f38c5a74640 1 -- 172.21.15.12:0/681507456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9dbfb0e40 msgr2=0x55c9dbfb5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:57.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.603+0000 7f38c5a74640 1 --2- 172.21.15.12:0/681507456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9dbfb0e40 0x55c9dbfb5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.603+0000 7f38c5a74640 1 -- 172.21.15.12:0/681507456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c9dbfb1830 msgr2=0x55c9dbfc2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:57.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.603+0000 7f38c5a74640 1 --2- 172.21.15.12:0/681507456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c9dbfb1830 0x55c9dbfc2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.603+0000 7f38c5a74640 1 -- 172.21.15.12:0/681507456 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9dbfb8ae0 con 0x55c9dbfb22d0 2024-04-22T13:23:57.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.603+0000 7f38c6275640 1 --2- 172.21.15.12:0/681507456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c9dbfb1830 0x55c9dbfc2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:23:57.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.603+0000 7f38c5a74640 1 --2- 172.21.15.12:0/681507456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9dbfb22d0 0x55c9dbfb5d80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f38b0000c00 tx=0x7f38b0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5fc9b661e8d8978 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:57.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.603+0000 7f38abfff640 1 -- 172.21.15.12:0/681507456 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b0045040 con 0x55c9dbfb22d0 2024-04-22T13:23:57.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.603+0000 7f38abfff640 1 -- 172.21.15.12:0/681507456 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38b00054e0 con 0x55c9dbfb22d0 2024-04-22T13:23:57.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.603+0000 7f38abfff640 1 -- 172.21.15.12:0/681507456 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b0004c30 con 0x55c9dbfb22d0 2024-04-22T13:23:57.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.603+0000 7f38cb1a9740 1 -- 172.21.15.12:0/681507456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9dbfb22d0 msgr2=0x55c9dbfb5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.603+0000 7f38cb1a9740 1 --2- 172.21.15.12:0/681507456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9dbfb22d0 0x55c9dbfb5d80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f38b0000c00 tx=0x7f38b0009580 comp rx=0 tx=0).stop 2024-04-22T13:23:57.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.603+0000 7f38cb1a9740 1 -- 172.21.15.12:0/681507456 shutdown_connections 2024-04-22T13:23:57.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.603+0000 7f38cb1a9740 1 --2- 172.21.15.12:0/681507456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9dbfb22d0 0x55c9dbfb5d80 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.603+0000 7f38cb1a9740 1 --2- 172.21.15.12:0/681507456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c9dbfb1830 0x55c9dbfc2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.603+0000 7f38cb1a9740 1 --2- 172.21.15.12:0/681507456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9dbfb0e40 0x55c9dbfb5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.603+0000 7f38cb1a9740 1 -- 172.21.15.12:0/681507456 >> 172.21.15.12:0/681507456 conn(0x55c9dbfacf40 msgr2=0x55c9dbfc6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:57.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.604+0000 7f38cb1a9740 1 -- 172.21.15.12:0/681507456 shutdown_connections 2024-04-22T13:23:57.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.604+0000 7f38cb1a9740 1 -- 172.21.15.12:0/681507456 wait complete. 2024-04-22T13:23:57.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.604+0000 7f38cb1a9740 1 -- start start 2024-04-22T13:23:57.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.604+0000 7f38cb1a9740 1 Processor -- start 2024-04-22T13:23:57.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.604+0000 7f38cb1a9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c9dbfb0e40 0x55c9dbfbe310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.604+0000 7f38cb1a9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9dbfb1830 0x55c9dbfc0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.604+0000 7f38cb1a9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9dbfb22d0 0x55c9dbfc0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.604+0000 7f38cb1a9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c9dbfc9090 con 0x55c9dbfb1830 2024-04-22T13:23:57.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.604+0000 7f38cb1a9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c9dbfc89c0 con 0x55c9dbfb0e40 2024-04-22T13:23:57.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.604+0000 7f38cb1a9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c9dbfc9760 con 0x55c9dbfb22d0 2024-04-22T13:23:57.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.604+0000 7f38cb1a9740 10 client.0 _ll_register_callbacks cb 0x7f38bc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:23:57.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.604+0000 7f38c6275640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9dbfb1830 0x55c9dbfc0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.604+0000 7f38c6275640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9dbfb1830 0x55c9dbfc0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59864/0 (socket says 172.21.15.12:59864) 2024-04-22T13:23:57.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.604+0000 7f38c6275640 1 -- 172.21.15.12:0/928922183 learned_addr learned my addr 172.21.15.12:0/928922183 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:57.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.604+0000 7f38c5a74640 1 --2- 172.21.15.12:0/928922183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c9dbfb0e40 0x55c9dbfbe310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.604+0000 7f38c6275640 1 -- 172.21.15.12:0/928922183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9dbfb22d0 msgr2=0x55c9dbfc0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:57.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.604+0000 7f38c6275640 1 --2- 172.21.15.12:0/928922183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9dbfb22d0 0x55c9dbfc0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.604+0000 7f38c6275640 1 -- 172.21.15.12:0/928922183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c9dbfb0e40 msgr2=0x55c9dbfbe310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:57.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.604+0000 7f38c6275640 1 --2- 172.21.15.12:0/928922183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c9dbfb0e40 0x55c9dbfbe310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.604+0000 7f38c6275640 1 -- 172.21.15.12:0/928922183 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9dbfc1710 con 0x55c9dbfb1830 2024-04-22T13:23:57.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.604+0000 7f38c5a74640 1 --2- 172.21.15.12:0/928922183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c9dbfb0e40 0x55c9dbfbe310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:23:57.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.605+0000 7f38c6275640 1 --2- 172.21.15.12:0/928922183 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9dbfb1830 0x55c9dbfc0860 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x55c9dbfac830 tx=0x7f38bc00f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6468736e794498ae server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:57.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.605+0000 7f38a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:23:57.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.605+0000 7f38a9ffb640 1 -- 172.21.15.12:0/928922183 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc02c070 con 0x55c9dbfb1830 2024-04-22T13:23:57.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.605+0000 7f38a9ffb640 1 -- 172.21.15.12:0/928922183 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f38bc005450 con 0x55c9dbfb1830 2024-04-22T13:23:57.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.605+0000 7f38cb1a9740 1 -- 172.21.15.12:0/928922183 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38b00054e0 con 0x55c9dbfb1830 2024-04-22T13:23:57.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.605+0000 7f38a9ffb640 1 -- 172.21.15.12:0/928922183 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f38bc005650 con 0x55c9dbfb1830 2024-04-22T13:23:57.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.605+0000 7f38cb1a9740 10 client.15261 fetch_fsmap learned FSMap version 27 2024-04-22T13:23:57.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.605+0000 7f38cb1a9740 1 -- 172.21.15.12:0/928922183 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c9dc02df00 con 0x55c9dbfb1830 2024-04-22T13:23:57.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.605+0000 7f38a9ffb640 1 -- 172.21.15.12:0/928922183 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38b00054e0 con 0x55c9dbfb1830 2024-04-22T13:23:57.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.605+0000 7f38a9ffb640 1 -- 172.21.15.12:0/928922183 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f38bc00f8b0 con 0x55c9dbfb1830 2024-04-22T13:23:57.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.605+0000 7f38cb1a9740 10 client.15261 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:23:57.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.605+0000 7f38cb1a9740 10 client.15261 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-22T13:23:57.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.605+0000 7f38cb1a9740 20 client.15261 populate_metadata read hostname 'smithi012' 2024-04-22T13:23:57.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.605+0000 7f38cb1a9740 1 --2- 172.21.15.12:0/928922183 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55c9dc0327b0 0x55c9dc0314c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.605+0000 7f38cb1a9740 4 client.15261 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T13:23:57.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.605+0000 7f38cb1a9740 1 -- 172.21.15.12:0/928922183 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "status"}) -- 0x55c9dbf1d3a0 con 0x55c9dc0327b0 2024-04-22T13:23:57.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.606+0000 7f38c5a74640 1 --2- 172.21.15.12:0/928922183 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55c9dc0327b0 0x55c9dc0314c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.606+0000 7f38c5a74640 1 --2- 172.21.15.12:0/928922183 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55c9dc0327b0 0x55c9dc0314c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b6b9265b9028b191 server_cookie=a2eb2fabf1633e60 in_seq=0 out_seq=0 2024-04-22T13:23:57.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.606+0000 7f38a9ffb640 10 client.15261 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:23:57.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.606+0000 7f38a9ffb640 1 -- 172.21.15.12:0/928922183 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55c9dbf1d3a0 con 0x55c9dc0327b0 2024-04-22T13:23:57.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.606+0000 7f38a9ffb640 10 client.15261 handle_command_reply: tid=0 2024-04-22T13:23:57.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.606+0000 7f38cb1a9740 1 client.15261 shutdown 2024-04-22T13:23:57.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.606+0000 7f38cb1a9740 1 -- 172.21.15.12:0/928922183 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9dbfb1830 msgr2=0x55c9dbfc0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.606+0000 7f38cb1a9740 1 --2- 172.21.15.12:0/928922183 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9dbfb1830 0x55c9dbfc0860 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x55c9dbfac830 tx=0x7f38bc00f5a0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.606+0000 7f38cb1a9740 1 -- 172.21.15.12:0/928922183 shutdown_connections 2024-04-22T13:23:57.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.606+0000 7f38cb1a9740 1 --2- 172.21.15.12:0/928922183 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55c9dc0327b0 0x55c9dc0314c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.606+0000 7f38cb1a9740 1 --2- 172.21.15.12:0/928922183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c9dbfb22d0 0x55c9dbfc0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.606+0000 7f38a9ffb640 0 client.15261 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:23:57.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.606+0000 7f38cb1a9740 1 --2- 172.21.15.12:0/928922183 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c9dbfb1830 0x55c9dbfc0860 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.606+0000 7f38cb1a9740 1 --2- 172.21.15.12:0/928922183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c9dbfb0e40 0x55c9dbfbe310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.606+0000 7f38cb1a9740 1 -- 172.21.15.12:0/928922183 >> 172.21.15.12:0/928922183 conn(0x55c9dbfacf40 msgr2=0x55c9dbfab400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:57.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.606+0000 7f38cb1a9740 1 -- 172.21.15.12:0/928922183 shutdown_connections 2024-04-22T13:23:57.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.606+0000 7f38cb1a9740 1 -- 172.21.15.12:0/928922183 wait complete. 2024-04-22T13:23:57.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.606+0000 7f38cb1a9740 20 client.15261 trim_cache size 0 max 16384 2024-04-22T13:23:57.610 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:23:57.610 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "0aede5ea-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:23:57.610 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 4, 2024-04-22T13:23:57.610 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24407, 2024-04-22T13:23:57.610 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-04-22T13:23:57.610 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-04-22T13:23:57.610 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-04-22T13:23:57.610 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 106.215136766, 2024-04-22T13:23:57.610 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 27, 2024-04-22T13:23:57.610 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 89, 2024-04-22T13:23:57.610 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 87, 2024-04-22T13:23:57.611 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 145.554763713 2024-04-22T13:23:57.611 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:23:57.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.607+0000 7f38c84d9640 1 -- 172.21.15.12:0/3368032532 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f38ac0a3bb0 msgr2=0x7f38ac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.607+0000 7f38c84d9640 1 --2- 172.21.15.12:0/3368032532 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f38ac0a3bb0 0x7f38ac0a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f38b8099030 tx=0x7f38b80c7ce0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.607+0000 7f38c84d9640 1 -- 172.21.15.12:0/3368032532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38c0103b60 msgr2=0x7f38c019f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.607+0000 7f38c84d9640 1 --2- 172.21.15.12:0/3368032532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38c0103b60 0x7f38c019f730 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f38b800cce0 tx=0x7f38b800a620 comp rx=0 tx=0).stop 2024-04-22T13:23:57.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.607+0000 7f38c84d9640 1 -- 172.21.15.12:0/3368032532 shutdown_connections 2024-04-22T13:23:57.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.607+0000 7f38c84d9640 1 --2- 172.21.15.12:0/3368032532 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f38ac0a3bb0 0x7f38ac0a6070 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.607+0000 7f38c84d9640 1 --2- 172.21.15.12:0/3368032532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f38c0103b60 0x7f38c019f730 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.607+0000 7f38c84d9640 1 --2- 172.21.15.12:0/3368032532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38c01031a0 0x7f38c01986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.607+0000 7f38c84d9640 1 --2- 172.21.15.12:0/3368032532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38c0101fb0 0x7f38c0198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.607+0000 7f38c84d9640 1 -- 172.21.15.12:0/3368032532 >> 172.21.15.12:0/3368032532 conn(0x7f38c00fd520 msgr2=0x7f38c01064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:57.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.607+0000 7f38c84d9640 1 -- 172.21.15.12:0/3368032532 shutdown_connections 2024-04-22T13:23:57.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.607+0000 7f38c84d9640 1 -- 172.21.15.12:0/3368032532 wait complete. 2024-04-22T13:23:57.615 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-22T13:23:57.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.720+0000 7f8b67055640 1 -- 172.21.15.12:0/1611568371 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b60103b50 msgr2=0x7f8b6010a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.720+0000 7f8b67055640 1 --2- 172.21.15.12:0/1611568371 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b60103b50 0x7f8b6010a3e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009be0 tx=0x7f8b5802bef0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.720+0000 7f8b67055640 1 -- 172.21.15.12:0/1611568371 shutdown_connections 2024-04-22T13:23:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.720+0000 7f8b67055640 1 --2- 172.21.15.12:0/1611568371 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b60103b50 0x7f8b6010a3e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.720+0000 7f8b67055640 1 --2- 172.21.15.12:0/1611568371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b60103190 0x7f8b60103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.720+0000 7f8b67055640 1 --2- 172.21.15.12:0/1611568371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b60101fa0 0x7f8b601023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.720+0000 7f8b67055640 1 -- 172.21.15.12:0/1611568371 >> 172.21.15.12:0/1611568371 conn(0x7f8b600fd530 msgr2=0x7f8b600ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.720+0000 7f8b67055640 1 -- 172.21.15.12:0/1611568371 shutdown_connections 2024-04-22T13:23:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.720+0000 7f8b67055640 1 -- 172.21.15.12:0/1611568371 wait complete. 2024-04-22T13:23:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.721+0000 7f8b67055640 1 Processor -- start 2024-04-22T13:23:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.721+0000 7f8b67055640 1 -- start start 2024-04-22T13:23:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.721+0000 7f8b67055640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b60101fa0 0x7f8b60198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.721+0000 7f8b67055640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b60103190 0x7f8b60198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.721+0000 7f8b67055640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b60103b50 0x7f8b6019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.721+0000 7f8b67055640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6010dae0 con 0x7f8b60103190 2024-04-22T13:23:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.721+0000 7f8b67055640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6010d960 con 0x7f8b60101fa0 2024-04-22T13:23:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.721+0000 7f8b67055640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6010dc60 con 0x7f8b60103b50 2024-04-22T13:23:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.721+0000 7f8b5ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b60103190 0x7f8b60198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.721+0000 7f8b5ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b60103190 0x7f8b60198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59890/0 (socket says 172.21.15.12:59890) 2024-04-22T13:23:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.721+0000 7f8b5ffff640 1 -- 172.21.15.12:0/4122186326 learned_addr learned my addr 172.21.15.12:0/4122186326 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.721+0000 7f8b64df1640 1 --2- 172.21.15.12:0/4122186326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b60101fa0 0x7f8b60198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.721+0000 7f8b655f2640 1 --2- 172.21.15.12:0/4122186326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b60103b50 0x7f8b6019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.721+0000 7f8b5ffff640 1 -- 172.21.15.12:0/4122186326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b60103b50 msgr2=0x7f8b6019f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.721+0000 7f8b5ffff640 1 --2- 172.21.15.12:0/4122186326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b60103b50 0x7f8b6019f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.721+0000 7f8b5ffff640 1 -- 172.21.15.12:0/4122186326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b60101fa0 msgr2=0x7f8b60198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.721+0000 7f8b5ffff640 1 --2- 172.21.15.12:0/4122186326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b60101fa0 0x7f8b60198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.722+0000 7f8b5ffff640 1 -- 172.21.15.12:0/4122186326 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b6019fd60 con 0x7f8b60103190 2024-04-22T13:23:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.722+0000 7f8b5ffff640 1 --2- 172.21.15.12:0/4122186326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b60103190 0x7f8b60198680 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c00d950 tx=0x7f8b4c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.722+0000 7f8b5dffb640 1 -- 172.21.15.12:0/4122186326 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c017070 con 0x7f8b60103190 2024-04-22T13:23:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.722+0000 7f8b67055640 1 -- 172.21.15.12:0/4122186326 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b6019fff0 con 0x7f8b60103190 2024-04-22T13:23:57.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.722+0000 7f8b5dffb640 1 -- 172.21.15.12:0/4122186326 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b4c009c30 con 0x7f8b60103190 2024-04-22T13:23:57.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.722+0000 7f8b5dffb640 1 -- 172.21.15.12:0/4122186326 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c00f620 con 0x7f8b60103190 2024-04-22T13:23:57.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.722+0000 7f8b67055640 1 -- 172.21.15.12:0/4122186326 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b601a0510 con 0x7f8b60103190 2024-04-22T13:23:57.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.724+0000 7f8b5dffb640 1 -- 172.21.15.12:0/4122186326 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8b4c010460 con 0x7f8b60103190 2024-04-22T13:23:57.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.725+0000 7f8b69c46740 1 Processor -- start 2024-04-22T13:23:57.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.725+0000 7f8b69c46740 1 -- start start 2024-04-22T13:23:57.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.725+0000 7f8b69c46740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b17802c250 0x55b17802c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.725+0000 7f8b69c46740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b17802cc40 0x55b17802d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.725+0000 7f8b69c46740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b17802d6e0 0x55b1780342c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.725+0000 7f8b69c46740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b177f8cb10 con 0x55b17802c250 2024-04-22T13:23:57.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.725+0000 7f8b69c46740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b177f2a240 con 0x55b17802cc40 2024-04-22T13:23:57.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.725+0000 7f8b69c46740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b177f88ad0 con 0x55b17802d6e0 2024-04-22T13:23:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.725+0000 7f8b5dffb640 1 --2- 172.21.15.12:0/4122186326 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f8b500a3bb0 0x7f8b500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.725+0000 7f8b655f2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b17802c250 0x55b17802c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.725+0000 7f8b655f2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b17802c250 0x55b17802c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59904/0 (socket says 172.21.15.12:59904) 2024-04-22T13:23:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.725+0000 7f8b655f2640 1 -- 172.21.15.12:0/1301879578 learned_addr learned my addr 172.21.15.12:0/1301879578 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.725+0000 7f8b655f2640 1 --2- 172.21.15.12:0/4122186326 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f8b500a3bb0 0x7f8b500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.725+0000 7f8b64df1640 1 --2- 172.21.15.12:0/1301879578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b17802cc40 0x55b17802d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.725+0000 7f8b5dffb640 1 -- 172.21.15.12:0/4122186326 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f8b4c0cff80 con 0x7f8b60103190 2024-04-22T13:23:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.725+0000 7f8b5ffff640 1 --2- 172.21.15.12:0/1301879578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b17802d6e0 0x55b1780342c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.726+0000 7f8b655f2640 1 -- 172.21.15.12:0/1301879578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b17802d6e0 msgr2=0x55b1780342c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.726+0000 7f8b655f2640 1 --2- 172.21.15.12:0/1301879578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b17802d6e0 0x55b1780342c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.726+0000 7f8b655f2640 1 -- 172.21.15.12:0/1301879578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b17802cc40 msgr2=0x55b17802d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.726+0000 7f8b655f2640 1 --2- 172.21.15.12:0/1301879578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b17802cc40 0x55b17802d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.726+0000 7f8b655f2640 1 -- 172.21.15.12:0/1301879578 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b177c9f2d0 con 0x55b17802c250 2024-04-22T13:23:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.726+0000 7f8b655f2640 1 --2- 172.21.15.12:0/1301879578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b17802c250 0x55b17802c670 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009be0 tx=0x7f8b5802dc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54193450350fada2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:57.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.726+0000 7f8b655f2640 1 --2- 172.21.15.12:0/4122186326 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f8b500a3bb0 0x7f8b500a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8b58035000 tx=0x7f8b58049000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:57.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.726+0000 7f8b4affd640 1 -- 172.21.15.12:0/1301879578 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b58003e30 con 0x55b17802c250 2024-04-22T13:23:57.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.726+0000 7f8b4affd640 1 -- 172.21.15.12:0/1301879578 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b580044f0 con 0x55b17802c250 2024-04-22T13:23:57.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.726+0000 7f8b4affd640 1 -- 172.21.15.12:0/1301879578 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b58034660 con 0x55b17802c250 2024-04-22T13:23:57.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.727+0000 7f8b69c46740 1 -- 172.21.15.12:0/1301879578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b17802c250 msgr2=0x55b17802c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.727+0000 7f8b69c46740 1 --2- 172.21.15.12:0/1301879578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b17802c250 0x55b17802c670 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009be0 tx=0x7f8b5802dc80 comp rx=0 tx=0).stop 2024-04-22T13:23:57.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.727+0000 7f8b69c46740 1 -- 172.21.15.12:0/1301879578 shutdown_connections 2024-04-22T13:23:57.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.727+0000 7f8b69c46740 1 --2- 172.21.15.12:0/1301879578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b17802d6e0 0x55b1780342c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.727+0000 7f8b69c46740 1 --2- 172.21.15.12:0/1301879578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b17802cc40 0x55b17802d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.727+0000 7f8b69c46740 1 --2- 172.21.15.12:0/1301879578 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b17802c250 0x55b17802c670 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.727+0000 7f8b69c46740 1 -- 172.21.15.12:0/1301879578 >> 172.21.15.12:0/1301879578 conn(0x55b178023f40 msgr2=0x55b178022490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:57.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.727+0000 7f8b69c46740 1 -- 172.21.15.12:0/1301879578 shutdown_connections 2024-04-22T13:23:57.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.727+0000 7f8b69c46740 1 -- 172.21.15.12:0/1301879578 wait complete. 2024-04-22T13:23:57.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.727+0000 7f8b69c46740 1 -- start start 2024-04-22T13:23:57.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.727+0000 7f8b69c46740 1 Processor -- start 2024-04-22T13:23:57.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.728+0000 7f8b69c46740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b17802c250 0x55b178036270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.728+0000 7f8b69c46740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b17802cc40 0x55b1780367b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.728+0000 7f8b69c46740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b17802d6e0 0x55b17803d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.728+0000 7f8b69c46740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b178034860 con 0x55b17802d6e0 2024-04-22T13:23:57.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.728+0000 7f8b69c46740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b177f15190 con 0x55b17802cc40 2024-04-22T13:23:57.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.728+0000 7f8b69c46740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b1780349e0 con 0x55b17802c250 2024-04-22T13:23:57.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.728+0000 7f8b69c46740 10 client.0 _ll_register_callbacks cb 0x55b177f108b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:23:57.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.728+0000 7f8b655f2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b17802d6e0 0x55b17803d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.728+0000 7f8b655f2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b17802d6e0 0x55b17803d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59912/0 (socket says 172.21.15.12:59912) 2024-04-22T13:23:57.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.728+0000 7f8b655f2640 1 -- 172.21.15.12:0/3509076318 learned_addr learned my addr 172.21.15.12:0/3509076318 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:57.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.728+0000 7f8b5ffff640 1 --2- 172.21.15.12:0/3509076318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b17802cc40 0x55b1780367b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.728+0000 7f8b64df1640 1 --2- 172.21.15.12:0/3509076318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b17802c250 0x55b178036270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.728+0000 7f8b655f2640 1 -- 172.21.15.12:0/3509076318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b17802c250 msgr2=0x55b178036270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:57.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.728+0000 7f8b655f2640 1 --2- 172.21.15.12:0/3509076318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b17802c250 0x55b178036270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.728+0000 7f8b655f2640 1 -- 172.21.15.12:0/3509076318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b17802cc40 msgr2=0x55b1780367b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:57.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.728+0000 7f8b655f2640 1 --2- 172.21.15.12:0/3509076318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b17802cc40 0x55b1780367b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.728+0000 7f8b655f2640 1 -- 172.21.15.12:0/3509076318 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b177d696b0 con 0x55b17802d6e0 2024-04-22T13:23:57.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.728+0000 7f8b5ffff640 1 --2- 172.21.15.12:0/3509076318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b17802cc40 0x55b1780367b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:23:57.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.728+0000 7f8b655f2640 1 --2- 172.21.15.12:0/3509076318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b17802d6e0 0x55b17803d7f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8b58034fd0 tx=0x7f8b5802d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c9825cf5f55e487 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:57.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.728+0000 7f8b3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:23:57.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.729+0000 7f8b3ffff640 1 -- 172.21.15.12:0/3509076318 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b58003e30 con 0x55b17802d6e0 2024-04-22T13:23:57.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.729+0000 7f8b3ffff640 1 -- 172.21.15.12:0/3509076318 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f8b58004240 con 0x55b17802d6e0 2024-04-22T13:23:57.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.729+0000 7f8b69c46740 1 -- 172.21.15.12:0/3509076318 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b580044f0 con 0x55b17802d6e0 2024-04-22T13:23:57.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.730+0000 7f8b3ffff640 1 -- 172.21.15.12:0/3509076318 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8b58065050 con 0x55b17802d6e0 2024-04-22T13:23:57.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.730+0000 7f8b69c46740 10 client.15273 fetch_fsmap learned FSMap version 27 2024-04-22T13:23:57.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.730+0000 7f8b69c46740 1 -- 172.21.15.12:0/3509076318 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b178040b60 con 0x55b17802d6e0 2024-04-22T13:23:57.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.730+0000 7f8b3ffff640 1 -- 172.21.15.12:0/3509076318 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b580044f0 con 0x55b17802d6e0 2024-04-22T13:23:57.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.730+0000 7f8b3ffff640 1 -- 172.21.15.12:0/3509076318 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8b58005050 con 0x55b17802d6e0 2024-04-22T13:23:57.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.730+0000 7f8b69c46740 10 client.15273 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:23:57.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.730+0000 7f8b69c46740 10 client.15273 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-22T13:23:57.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.730+0000 7f8b69c46740 20 client.15273 populate_metadata read hostname 'smithi012' 2024-04-22T13:23:57.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.730+0000 7f8b69c46740 1 --2- 172.21.15.12:0/3509076318 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55b178042d90 0x55b178044200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.730+0000 7f8b69c46740 4 client.15273 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:23:57.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.730+0000 7f8b69c46740 1 -- 172.21.15.12:0/3509076318 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b177f9a530 con 0x55b178042d90 2024-04-22T13:23:57.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.730+0000 7f8b64df1640 1 --2- 172.21.15.12:0/3509076318 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55b178042d90 0x55b178044200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.731+0000 7f8b64df1640 1 --2- 172.21.15.12:0/3509076318 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55b178042d90 0x55b178044200 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=591ed05d078933e5 server_cookie=d9598ffca8958592 in_seq=0 out_seq=0 2024-04-22T13:23:57.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.731+0000 7f8b3ffff640 10 client.15273 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:23:57.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.731+0000 7f8b3ffff640 1 -- 172.21.15.12:0/3509076318 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b177f9a530 con 0x55b178042d90 2024-04-22T13:23:57.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.731+0000 7f8b3ffff640 10 client.15273 handle_command_reply: tid=0 2024-04-22T13:23:57.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.731+0000 7f8b69c46740 1 client.15273 shutdown 2024-04-22T13:23:57.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.732+0000 7f8b69c46740 1 -- 172.21.15.12:0/3509076318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b17802d6e0 msgr2=0x55b17803d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.732+0000 7f8b69c46740 1 --2- 172.21.15.12:0/3509076318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b17802d6e0 0x55b17803d7f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8b58034fd0 tx=0x7f8b5802d9c0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.732+0000 7f8b69c46740 1 -- 172.21.15.12:0/3509076318 shutdown_connections 2024-04-22T13:23:57.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.732+0000 7f8b69c46740 1 --2- 172.21.15.12:0/3509076318 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55b178042d90 0x55b178044200 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.732+0000 7f8b69c46740 1 --2- 172.21.15.12:0/3509076318 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b17802d6e0 0x55b17803d7f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.732+0000 7f8b3ffff640 0 client.15273 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:23:57.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.732+0000 7f8b69c46740 1 --2- 172.21.15.12:0/3509076318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b17802cc40 0x55b1780367b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.732+0000 7f8b69c46740 1 --2- 172.21.15.12:0/3509076318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b17802c250 0x55b178036270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.732+0000 7f8b69c46740 1 -- 172.21.15.12:0/3509076318 >> 172.21.15.12:0/3509076318 conn(0x55b178023f40 msgr2=0x55b178021a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:57.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.732+0000 7f8b69c46740 1 -- 172.21.15.12:0/3509076318 shutdown_connections 2024-04-22T13:23:57.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.732+0000 7f8b69c46740 1 -- 172.21.15.12:0/3509076318 wait complete. 2024-04-22T13:23:57.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.732+0000 7f8b69c46740 20 client.15273 trim_cache size 0 max 16384 2024-04-22T13:23:57.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.750+0000 7f8b69c46740 1 Processor -- start 2024-04-22T13:23:57.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.750+0000 7f8b69c46740 1 -- start start 2024-04-22T13:23:57.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.750+0000 7f8b69c46740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b17802c250 0x55b178021170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.750+0000 7f8b69c46740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b17802cc40 0x55b17803d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.750+0000 7f8b69c46740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b17802d6e0 0x55b1780216b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.750+0000 7f8b69c46740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b178042540 con 0x55b17802c250 2024-04-22T13:23:57.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.750+0000 7f8b69c46740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b178042c10 con 0x55b17802cc40 2024-04-22T13:23:57.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.750+0000 7f8b69c46740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b1780432e0 con 0x55b17802d6e0 2024-04-22T13:23:57.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.750+0000 7f8b64df1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b17802c250 0x55b178021170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.750+0000 7f8b64df1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b17802c250 0x55b178021170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59914/0 (socket says 172.21.15.12:59914) 2024-04-22T13:23:57.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.750+0000 7f8b64df1640 1 -- 172.21.15.12:0/3393268174 learned_addr learned my addr 172.21.15.12:0/3393268174 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:57.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.751+0000 7f8b64df1640 1 -- 172.21.15.12:0/3393268174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b17802d6e0 msgr2=0x55b1780216b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:57.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.751+0000 7f8b64df1640 1 --2- 172.21.15.12:0/3393268174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b17802d6e0 0x55b1780216b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.751+0000 7f8b5ffff640 1 --2- 172.21.15.12:0/3393268174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b17802cc40 0x55b17803d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.751+0000 7f8b64df1640 1 -- 172.21.15.12:0/3393268174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b17802cc40 msgr2=0x55b17803d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:57.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.751+0000 7f8b64df1640 1 --2- 172.21.15.12:0/3393268174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b17802cc40 0x55b17803d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.751+0000 7f8b64df1640 1 -- 172.21.15.12:0/3393268174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1780373f0 con 0x55b17802c250 2024-04-22T13:23:57.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.751+0000 7f8b5ffff640 1 --2- 172.21.15.12:0/3393268174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b17802cc40 0x55b17803d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:23:57.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.751+0000 7f8b64df1640 1 --2- 172.21.15.12:0/3393268174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b17802c250 0x55b178021170 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x55b178023830 tx=0x7f8b5400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7741338ed63cd6e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:57.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.751+0000 7f8b4a7fc640 1 -- 172.21.15.12:0/3393268174 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b54026070 con 0x55b17802c250 2024-04-22T13:23:57.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.751+0000 7f8b4a7fc640 1 -- 172.21.15.12:0/3393268174 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b54003c40 con 0x55b17802c250 2024-04-22T13:23:57.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.751+0000 7f8b4a7fc640 1 -- 172.21.15.12:0/3393268174 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b54011860 con 0x55b17802c250 2024-04-22T13:23:57.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.752+0000 7f8b69c46740 1 -- 172.21.15.12:0/3393268174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b17802c250 msgr2=0x55b178021170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.752+0000 7f8b69c46740 1 --2- 172.21.15.12:0/3393268174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b17802c250 0x55b178021170 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x55b178023830 tx=0x7f8b5400b390 comp rx=0 tx=0).stop 2024-04-22T13:23:57.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.752+0000 7f8b69c46740 1 -- 172.21.15.12:0/3393268174 shutdown_connections 2024-04-22T13:23:57.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.752+0000 7f8b69c46740 1 --2- 172.21.15.12:0/3393268174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b17802d6e0 0x55b1780216b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.752+0000 7f8b69c46740 1 --2- 172.21.15.12:0/3393268174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b17802cc40 0x55b17803d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.752+0000 7f8b69c46740 1 --2- 172.21.15.12:0/3393268174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b17802c250 0x55b178021170 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.752+0000 7f8b69c46740 1 -- 172.21.15.12:0/3393268174 >> 172.21.15.12:0/3393268174 conn(0x55b178023f40 msgr2=0x55b17801d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:57.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.752+0000 7f8b69c46740 1 -- 172.21.15.12:0/3393268174 shutdown_connections 2024-04-22T13:23:57.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.752+0000 7f8b69c46740 1 -- 172.21.15.12:0/3393268174 wait complete. 2024-04-22T13:23:57.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.752+0000 7f8b69c46740 1 -- start start 2024-04-22T13:23:57.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.752+0000 7f8b69c46740 1 Processor -- start 2024-04-22T13:23:57.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.752+0000 7f8b69c46740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b17802c250 0x55b178039190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.752+0000 7f8b69c46740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b17802cc40 0x55b1780396d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.752+0000 7f8b69c46740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b17802d6e0 0x55b17803c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.752+0000 7f8b69c46740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b178044080 con 0x55b17802d6e0 2024-04-22T13:23:57.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.752+0000 7f8b69c46740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b1780439b0 con 0x55b17802c250 2024-04-22T13:23:57.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.752+0000 7f8b69c46740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b178044750 con 0x55b17802cc40 2024-04-22T13:23:57.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.752+0000 7f8b69c46740 10 client.0 _ll_register_callbacks cb 0x7f8b5400b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:23:57.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.753+0000 7f8b655f2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b17802d6e0 0x55b17803c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.753+0000 7f8b655f2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b17802d6e0 0x55b17803c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59928/0 (socket says 172.21.15.12:59928) 2024-04-22T13:23:57.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.753+0000 7f8b655f2640 1 -- 172.21.15.12:0/2869768608 learned_addr learned my addr 172.21.15.12:0/2869768608 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:57.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.753+0000 7f8b655f2640 1 -- 172.21.15.12:0/2869768608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b17802cc40 msgr2=0x55b1780396d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:57.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.753+0000 7f8b655f2640 1 --2- 172.21.15.12:0/2869768608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b17802cc40 0x55b1780396d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.753+0000 7f8b655f2640 1 -- 172.21.15.12:0/2869768608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b17802c250 msgr2=0x55b178039190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:57.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.753+0000 7f8b64df1640 1 --2- 172.21.15.12:0/2869768608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b17802c250 0x55b178039190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.753+0000 7f8b655f2640 1 --2- 172.21.15.12:0/2869768608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b17802c250 0x55b178039190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.753+0000 7f8b655f2640 1 -- 172.21.15.12:0/2869768608 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b17803cce0 con 0x55b17802d6e0 2024-04-22T13:23:57.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.753+0000 7f8b64df1640 1 --2- 172.21.15.12:0/2869768608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b17802c250 0x55b178039190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:23:57.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.753+0000 7f8b655f2640 1 --2- 172.21.15.12:0/2869768608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b17802d6e0 0x55b17803c6f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8b58033780 tx=0x7f8b58005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d46546af3888e34 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:57.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.753+0000 7f8b3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:23:57.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.753+0000 7f8b3ffff640 1 -- 172.21.15.12:0/2869768608 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b58009480 con 0x55b17802d6e0 2024-04-22T13:23:57.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.753+0000 7f8b69c46740 1 -- 172.21.15.12:0/2869768608 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b54003c40 con 0x55b17802d6e0 2024-04-22T13:23:57.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.753+0000 7f8b3ffff640 1 -- 172.21.15.12:0/2869768608 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f8b58004040 con 0x55b17802d6e0 2024-04-22T13:23:57.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.753+0000 7f8b3ffff640 1 -- 172.21.15.12:0/2869768608 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8b58004240 con 0x55b17802d6e0 2024-04-22T13:23:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.753+0000 7f8b69c46740 10 client.15279 fetch_fsmap learned FSMap version 27 2024-04-22T13:23:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.753+0000 7f8b69c46740 1 -- 172.21.15.12:0/2869768608 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b1780a8fa0 con 0x55b17802d6e0 2024-04-22T13:23:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.754+0000 7f8b3ffff640 1 -- 172.21.15.12:0/2869768608 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b54003c40 con 0x55b17802d6e0 2024-04-22T13:23:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.754+0000 7f8b3ffff640 1 -- 172.21.15.12:0/2869768608 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8b58004ff0 con 0x55b17802d6e0 2024-04-22T13:23:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.754+0000 7f8b69c46740 10 client.15279 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:23:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.754+0000 7f8b69c46740 10 client.15279 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-22T13:23:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.754+0000 7f8b69c46740 20 client.15279 populate_metadata read hostname 'smithi012' 2024-04-22T13:23:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.754+0000 7f8b69c46740 1 --2- 172.21.15.12:0/2869768608 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55b1780adbf0 0x55b1780ac900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.754+0000 7f8b69c46740 4 client.15279 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T13:23:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.754+0000 7f8b69c46740 1 -- 172.21.15.12:0/2869768608 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "status"}) -- 0x55b177f943a0 con 0x55b1780adbf0 2024-04-22T13:23:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.754+0000 7f8b64df1640 1 --2- 172.21.15.12:0/2869768608 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55b1780adbf0 0x55b1780ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.754+0000 7f8b64df1640 1 --2- 172.21.15.12:0/2869768608 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55b1780adbf0 0x55b1780ac900 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6c08ff41553a047f server_cookie=3f55793f8bc190c4 in_seq=0 out_seq=0 2024-04-22T13:23:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.754+0000 7f8b3ffff640 10 client.15279 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:23:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.755+0000 7f8b3ffff640 1 -- 172.21.15.12:0/2869768608 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55b177f943a0 con 0x55b1780adbf0 2024-04-22T13:23:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.755+0000 7f8b3ffff640 10 client.15279 handle_command_reply: tid=0 2024-04-22T13:23:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.755+0000 7f8b69c46740 1 client.15279 shutdown 2024-04-22T13:23:57.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.755+0000 7f8b69c46740 1 -- 172.21.15.12:0/2869768608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b17802d6e0 msgr2=0x55b17803c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.755+0000 7f8b69c46740 1 --2- 172.21.15.12:0/2869768608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b17802d6e0 0x55b17803c6f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8b58033780 tx=0x7f8b58005910 comp rx=0 tx=0).stop 2024-04-22T13:23:57.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.755+0000 7f8b69c46740 1 -- 172.21.15.12:0/2869768608 shutdown_connections 2024-04-22T13:23:57.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.755+0000 7f8b69c46740 1 --2- 172.21.15.12:0/2869768608 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55b1780adbf0 0x55b1780ac900 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.755+0000 7f8b69c46740 1 --2- 172.21.15.12:0/2869768608 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b17802d6e0 0x55b17803c6f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.755+0000 7f8b69c46740 1 --2- 172.21.15.12:0/2869768608 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b17802cc40 0x55b1780396d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.755+0000 7f8b69c46740 1 --2- 172.21.15.12:0/2869768608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b17802c250 0x55b178039190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.755+0000 7f8b3ffff640 0 client.15279 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:23:57.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.755+0000 7f8b69c46740 1 -- 172.21.15.12:0/2869768608 >> 172.21.15.12:0/2869768608 conn(0x55b178023f40 msgr2=0x55b178021170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:57.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.755+0000 7f8b69c46740 1 -- 172.21.15.12:0/2869768608 shutdown_connections 2024-04-22T13:23:57.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.755+0000 7f8b69c46740 1 -- 172.21.15.12:0/2869768608 wait complete. 2024-04-22T13:23:57.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.755+0000 7f8b69c46740 20 client.15279 trim_cache size 0 max 16384 2024-04-22T13:23:57.758 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:23:57.759 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "0aede5ea-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:23:57.759 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 3, 2024-04-22T13:23:57.759 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24443, 2024-04-22T13:23:57.759 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-04-22T13:23:57.759 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-04-22T13:23:57.759 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-04-22T13:23:57.759 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 107.372313022, 2024-04-22T13:23:57.759 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 27, 2024-04-22T13:23:57.759 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 89, 2024-04-22T13:23:57.759 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 87, 2024-04-22T13:23:57.759 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 137.441677903 2024-04-22T13:23:57.759 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:23:57.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.756+0000 7f8b67055640 1 -- 172.21.15.12:0/4122186326 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f8b500a3bb0 msgr2=0x7f8b500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.756+0000 7f8b67055640 1 --2- 172.21.15.12:0/4122186326 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f8b500a3bb0 0x7f8b500a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8b58035000 tx=0x7f8b58049000 comp rx=0 tx=0).stop 2024-04-22T13:23:57.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.756+0000 7f8b67055640 1 -- 172.21.15.12:0/4122186326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b60103190 msgr2=0x7f8b60198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.756+0000 7f8b67055640 1 --2- 172.21.15.12:0/4122186326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b60103190 0x7f8b60198680 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c00d950 tx=0x7f8b4c00de10 comp rx=0 tx=0).stop 2024-04-22T13:23:57.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.756+0000 7f8b67055640 1 -- 172.21.15.12:0/4122186326 shutdown_connections 2024-04-22T13:23:57.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.756+0000 7f8b67055640 1 --2- 172.21.15.12:0/4122186326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b60103b50 0x7f8b6019f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.756+0000 7f8b67055640 1 --2- 172.21.15.12:0/4122186326 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f8b500a3bb0 0x7f8b500a6070 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.756+0000 7f8b67055640 1 --2- 172.21.15.12:0/4122186326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8b60103190 0x7f8b60198680 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.756+0000 7f8b67055640 1 --2- 172.21.15.12:0/4122186326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b60101fa0 0x7f8b60198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.756+0000 7f8b67055640 1 -- 172.21.15.12:0/4122186326 >> 172.21.15.12:0/4122186326 conn(0x7f8b600fd530 msgr2=0x7f8b60106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:57.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.756+0000 7f8b67055640 1 -- 172.21.15.12:0/4122186326 shutdown_connections 2024-04-22T13:23:57.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.756+0000 7f8b67055640 1 -- 172.21.15.12:0/4122186326 wait complete. 2024-04-22T13:23:57.764 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-22T13:23:57.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.869+0000 7f1cb7a55640 1 -- 172.21.15.12:0/2986567558 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cb0101fa0 msgr2=0x7f1cb01023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.869+0000 7f1cb7a55640 1 --2- 172.21.15.12:0/2986567558 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cb0101fa0 0x7f1cb01023c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1cac009900 tx=0x7f1cac02bc30 comp rx=0 tx=0).stop 2024-04-22T13:23:57.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.869+0000 7f1cb7a55640 1 -- 172.21.15.12:0/2986567558 shutdown_connections 2024-04-22T13:23:57.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.869+0000 7f1cb7a55640 1 --2- 172.21.15.12:0/2986567558 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cb0103b50 0x7f1cb010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.869+0000 7f1cb7a55640 1 --2- 172.21.15.12:0/2986567558 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cb0103190 0x7f1cb0103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.869+0000 7f1cb7a55640 1 --2- 172.21.15.12:0/2986567558 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cb0101fa0 0x7f1cb01023c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.869+0000 7f1cb7a55640 1 -- 172.21.15.12:0/2986567558 >> 172.21.15.12:0/2986567558 conn(0x7f1cb00fd530 msgr2=0x7f1cb00ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.869+0000 7f1cb7a55640 1 -- 172.21.15.12:0/2986567558 shutdown_connections 2024-04-22T13:23:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.869+0000 7f1cb7a55640 1 -- 172.21.15.12:0/2986567558 wait complete. 2024-04-22T13:23:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.870+0000 7f1cb7a55640 1 Processor -- start 2024-04-22T13:23:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.870+0000 7f1cb7a55640 1 -- start start 2024-04-22T13:23:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.870+0000 7f1cb7a55640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cb0101fa0 0x7f1cb0198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.870+0000 7f1cb7a55640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cb0103190 0x7f1cb0198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.870+0000 7f1cb7a55640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cb0103b50 0x7f1cb019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.870+0000 7f1cb7a55640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb010db30 con 0x7f1cb0103190 2024-04-22T13:23:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.870+0000 7f1cb7a55640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb010d9b0 con 0x7f1cb0101fa0 2024-04-22T13:23:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.870+0000 7f1cb7a55640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb010dcb0 con 0x7f1cb0103b50 2024-04-22T13:23:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.870+0000 7f1cb4ff0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cb0103190 0x7f1cb0198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.870+0000 7f1cb4ff0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cb0103190 0x7f1cb0198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59942/0 (socket says 172.21.15.12:59942) 2024-04-22T13:23:57.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.870+0000 7f1cb4ff0640 1 -- 172.21.15.12:0/733041566 learned_addr learned my addr 172.21.15.12:0/733041566 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.870+0000 7f1cb57f1640 1 --2- 172.21.15.12:0/733041566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cb0101fa0 0x7f1cb0198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.870+0000 7f1cb5ff2640 1 --2- 172.21.15.12:0/733041566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cb0103b50 0x7f1cb019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.870+0000 7f1cb4ff0640 1 -- 172.21.15.12:0/733041566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cb0103b50 msgr2=0x7f1cb019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.870+0000 7f1cb4ff0640 1 --2- 172.21.15.12:0/733041566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cb0103b50 0x7f1cb019f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.870+0000 7f1cb4ff0640 1 -- 172.21.15.12:0/733041566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cb0101fa0 msgr2=0x7f1cb0198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.870+0000 7f1cb4ff0640 1 --2- 172.21.15.12:0/733041566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cb0101fa0 0x7f1cb0198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.870+0000 7f1cb4ff0640 1 -- 172.21.15.12:0/733041566 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb019fe60 con 0x7f1cb0103190 2024-04-22T13:23:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.870+0000 7f1cb5ff2640 1 --2- 172.21.15.12:0/733041566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cb0103b50 0x7f1cb019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:23:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.870+0000 7f1cb4ff0640 1 --2- 172.21.15.12:0/733041566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cb0103190 0x7f1cb0198780 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000d770 tx=0x7f1ca000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.871+0000 7f1ca67fc640 1 -- 172.21.15.12:0/733041566 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0017070 con 0x7f1cb0103190 2024-04-22T13:23:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.871+0000 7f1ca67fc640 1 -- 172.21.15.12:0/733041566 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ca00054d0 con 0x7f1cb0103190 2024-04-22T13:23:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.871+0000 7f1cb7a55640 1 -- 172.21.15.12:0/733041566 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb01a0150 con 0x7f1cb0103190 2024-04-22T13:23:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.871+0000 7f1ca67fc640 1 -- 172.21.15.12:0/733041566 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0004c30 con 0x7f1cb0103190 2024-04-22T13:23:57.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.871+0000 7f1cb7a55640 1 -- 172.21.15.12:0/733041566 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb01a06a0 con 0x7f1cb0103190 2024-04-22T13:23:57.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.873+0000 7f1ca67fc640 1 -- 172.21.15.12:0/733041566 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1ca0004e10 con 0x7f1cb0103190 2024-04-22T13:23:57.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.873+0000 7f1cba646740 1 Processor -- start 2024-04-22T13:23:57.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.873+0000 7f1cba646740 1 -- start start 2024-04-22T13:23:57.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.873+0000 7f1cba646740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559e017b3e40 0x559e017b4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.873+0000 7f1cba646740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e017b4830 0x559e017b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.873+0000 7f1cba646740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e017b52d0 0x559e017bbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.873+0000 7f1cba646740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559e01718b10 con 0x559e017b4830 2024-04-22T13:23:57.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.873+0000 7f1cba646740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559e016b6240 con 0x559e017b3e40 2024-04-22T13:23:57.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.873+0000 7f1cba646740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559e01714ad0 con 0x559e017b52d0 2024-04-22T13:23:57.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.874+0000 7f1ca67fc640 1 --2- 172.21.15.12:0/733041566 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1c9c0a3c30 0x7f1c9c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.874+0000 7f1cb57f1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e017b4830 0x559e017b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.874+0000 7f1cb57f1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e017b4830 0x559e017b4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59952/0 (socket says 172.21.15.12:59952) 2024-04-22T13:23:57.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.874+0000 7f1cb57f1640 1 -- 172.21.15.12:0/529083390 learned_addr learned my addr 172.21.15.12:0/529083390 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:57.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.874+0000 7f1cb5ff2640 1 --2- 172.21.15.12:0/733041566 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1c9c0a3c30 0x7f1c9c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.874+0000 7f1cb4ff0640 1 --2- 172.21.15.12:0/529083390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e017b52d0 0x559e017bbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.874+0000 7f1ca67fc640 1 -- 172.21.15.12:0/733041566 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f1ca00cfe10 con 0x7f1cb0103190 2024-04-22T13:23:57.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.874+0000 7f1cb5ff2640 1 --2- 172.21.15.12:0/529083390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559e017b3e40 0x559e017b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.874+0000 7f1cb57f1640 1 -- 172.21.15.12:0/529083390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e017b52d0 msgr2=0x559e017bbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:57.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.874+0000 7f1cb57f1640 1 --2- 172.21.15.12:0/529083390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e017b52d0 0x559e017bbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.874+0000 7f1cb57f1640 1 -- 172.21.15.12:0/529083390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559e017b3e40 msgr2=0x559e017b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:57.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.874+0000 7f1cb57f1640 1 --2- 172.21.15.12:0/529083390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559e017b3e40 0x559e017b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.874+0000 7f1cb57f1640 1 -- 172.21.15.12:0/529083390 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e0142b2d0 con 0x559e017b4830 2024-04-22T13:23:57.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.874+0000 7f1cb4ff0640 1 --2- 172.21.15.12:0/529083390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e017b52d0 0x559e017bbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:23:57.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.874+0000 7f1cb5ff2640 1 --2- 172.21.15.12:0/529083390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559e017b3e40 0x559e017b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:23:57.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.874+0000 7f1cb5ff2640 1 --2- 172.21.15.12:0/733041566 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1c9c0a3c30 0x7f1c9c0a60f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x559e017af200 tx=0x7f1ca8009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:57.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.874+0000 7f1cb57f1640 1 --2- 172.21.15.12:0/529083390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e017b4830 0x559e017b4cb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1cac000c00 tx=0x7f1cac005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09ef9084766ccda server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:57.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.874+0000 7f1c9b7fe640 1 -- 172.21.15.12:0/529083390 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac045020 con 0x559e017b4830 2024-04-22T13:23:57.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.874+0000 7f1c9b7fe640 1 -- 172.21.15.12:0/529083390 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cac02c9e0 con 0x559e017b4830 2024-04-22T13:23:57.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.874+0000 7f1c9b7fe640 1 -- 172.21.15.12:0/529083390 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac0046c0 con 0x559e017b4830 2024-04-22T13:23:57.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.875+0000 7f1cba646740 1 -- 172.21.15.12:0/529083390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e017b4830 msgr2=0x559e017b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.875+0000 7f1cba646740 1 --2- 172.21.15.12:0/529083390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e017b4830 0x559e017b4cb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1cac000c00 tx=0x7f1cac005dd0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.875+0000 7f1cba646740 1 -- 172.21.15.12:0/529083390 shutdown_connections 2024-04-22T13:23:57.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.875+0000 7f1cba646740 1 --2- 172.21.15.12:0/529083390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e017b52d0 0x559e017bbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.875+0000 7f1cba646740 1 --2- 172.21.15.12:0/529083390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e017b4830 0x559e017b4cb0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.875+0000 7f1cba646740 1 --2- 172.21.15.12:0/529083390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559e017b3e40 0x559e017b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.875+0000 7f1cba646740 1 -- 172.21.15.12:0/529083390 >> 172.21.15.12:0/529083390 conn(0x559e017aff40 msgr2=0x559e017ae490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:57.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.875+0000 7f1cba646740 1 -- 172.21.15.12:0/529083390 shutdown_connections 2024-04-22T13:23:57.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.875+0000 7f1cba646740 1 -- 172.21.15.12:0/529083390 wait complete. 2024-04-22T13:23:57.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.875+0000 7f1cba646740 1 -- start start 2024-04-22T13:23:57.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.875+0000 7f1cba646740 1 Processor -- start 2024-04-22T13:23:57.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.876+0000 7f1cba646740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559e017b3e40 0x559e017bde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.876+0000 7f1cba646740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e017b4830 0x559e017be340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.876+0000 7f1cba646740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e017b52d0 0x559e017c5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.876+0000 7f1cba646740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559e017bc450 con 0x559e017b52d0 2024-04-22T13:23:57.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.876+0000 7f1cba646740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559e016a1190 con 0x559e017b3e40 2024-04-22T13:23:57.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.876+0000 7f1cba646740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559e017bc5d0 con 0x559e017b4830 2024-04-22T13:23:57.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.876+0000 7f1cba646740 10 client.0 _ll_register_callbacks cb 0x559e0169c8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:23:57.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.876+0000 7f1cb5ff2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e017b52d0 0x559e017c5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.876+0000 7f1cb5ff2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e017b52d0 0x559e017c5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59960/0 (socket says 172.21.15.12:59960) 2024-04-22T13:23:57.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.876+0000 7f1cb5ff2640 1 -- 172.21.15.12:0/1269085132 learned_addr learned my addr 172.21.15.12:0/1269085132 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:57.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.876+0000 7f1cb4ff0640 1 --2- 172.21.15.12:0/1269085132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e017b4830 0x559e017be340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.876+0000 7f1cb57f1640 1 --2- 172.21.15.12:0/1269085132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559e017b3e40 0x559e017bde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.876+0000 7f1cb5ff2640 1 -- 172.21.15.12:0/1269085132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e017b4830 msgr2=0x559e017be340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:57.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.876+0000 7f1cb5ff2640 1 --2- 172.21.15.12:0/1269085132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e017b4830 0x559e017be340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.876+0000 7f1cb5ff2640 1 -- 172.21.15.12:0/1269085132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559e017b3e40 msgr2=0x559e017bde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:57.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.876+0000 7f1cb5ff2640 1 --2- 172.21.15.12:0/1269085132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559e017b3e40 0x559e017bde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.876+0000 7f1cb5ff2640 1 -- 172.21.15.12:0/1269085132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e014f56b0 con 0x559e017b52d0 2024-04-22T13:23:57.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.876+0000 7f1cb5ff2640 1 --2- 172.21.15.12:0/1269085132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e017b52d0 0x559e017c5310 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1ca800faa0 tx=0x7f1ca8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eabff5781634f8f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:57.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.876+0000 7f1c98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:23:57.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.877+0000 7f1c98ff9640 1 -- 172.21.15.12:0/1269085132 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8024030 con 0x559e017b52d0 2024-04-22T13:23:57.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.877+0000 7f1c98ff9640 1 -- 172.21.15.12:0/1269085132 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f1ca8012b80 con 0x559e017b52d0 2024-04-22T13:23:57.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.877+0000 7f1cba646740 1 -- 172.21.15.12:0/1269085132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cac02c9e0 con 0x559e017b52d0 2024-04-22T13:23:57.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.877+0000 7f1c98ff9640 1 -- 172.21.15.12:0/1269085132 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1ca8012d80 con 0x559e017b52d0 2024-04-22T13:23:57.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.877+0000 7f1cba646740 10 client.15291 fetch_fsmap learned FSMap version 27 2024-04-22T13:23:57.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.877+0000 7f1cba646740 1 -- 172.21.15.12:0/1269085132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559e017c86b0 con 0x559e017b52d0 2024-04-22T13:23:57.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.877+0000 7f1c98ff9640 1 -- 172.21.15.12:0/1269085132 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cac02c9e0 con 0x559e017b52d0 2024-04-22T13:23:57.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.877+0000 7f1c98ff9640 1 -- 172.21.15.12:0/1269085132 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1ca801d050 con 0x559e017b52d0 2024-04-22T13:23:57.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.877+0000 7f1cba646740 10 client.15291 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:23:57.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.877+0000 7f1cba646740 10 client.15291 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-22T13:23:57.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.877+0000 7f1cba646740 20 client.15291 populate_metadata read hostname 'smithi012' 2024-04-22T13:23:57.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.877+0000 7f1cba646740 1 --2- 172.21.15.12:0/1269085132 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x559e017cadf0 0x559e017cc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.877+0000 7f1cba646740 4 client.15291 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:23:57.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.877+0000 7f1cba646740 1 -- 172.21.15.12:0/1269085132 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e01726530 con 0x559e017cadf0 2024-04-22T13:23:57.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.878+0000 7f1cb57f1640 1 --2- 172.21.15.12:0/1269085132 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x559e017cadf0 0x559e017cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.878+0000 7f1cb57f1640 1 --2- 172.21.15.12:0/1269085132 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x559e017cadf0 0x559e017cc260 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b8763bd1d2cc8b1b server_cookie=68930dfbfaabb25b in_seq=0 out_seq=0 2024-04-22T13:23:57.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.878+0000 7f1c98ff9640 10 client.15291 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:23:57.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.879+0000 7f1c98ff9640 1 -- 172.21.15.12:0/1269085132 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559e01726530 con 0x559e017cadf0 2024-04-22T13:23:57.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.879+0000 7f1c98ff9640 10 client.15291 handle_command_reply: tid=0 2024-04-22T13:23:57.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.879+0000 7f1cba646740 1 client.15291 shutdown 2024-04-22T13:23:57.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.879+0000 7f1cba646740 1 -- 172.21.15.12:0/1269085132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e017b52d0 msgr2=0x559e017c5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.879+0000 7f1cba646740 1 --2- 172.21.15.12:0/1269085132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e017b52d0 0x559e017c5310 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1ca800faa0 tx=0x7f1ca8004a40 comp rx=0 tx=0).stop 2024-04-22T13:23:57.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.879+0000 7f1cba646740 1 -- 172.21.15.12:0/1269085132 shutdown_connections 2024-04-22T13:23:57.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.879+0000 7f1cba646740 1 --2- 172.21.15.12:0/1269085132 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x559e017cadf0 0x559e017cc260 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.879+0000 7f1cba646740 1 --2- 172.21.15.12:0/1269085132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e017b52d0 0x559e017c5310 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.879+0000 7f1cba646740 1 --2- 172.21.15.12:0/1269085132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e017b4830 0x559e017be340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.879+0000 7f1c98ff9640 0 client.15291 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:23:57.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.879+0000 7f1cba646740 1 --2- 172.21.15.12:0/1269085132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559e017b3e40 0x559e017bde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.879+0000 7f1cba646740 1 -- 172.21.15.12:0/1269085132 >> 172.21.15.12:0/1269085132 conn(0x559e017aff40 msgr2=0x559e017ada00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:57.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.879+0000 7f1cba646740 1 -- 172.21.15.12:0/1269085132 shutdown_connections 2024-04-22T13:23:57.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.879+0000 7f1cba646740 1 -- 172.21.15.12:0/1269085132 wait complete. 2024-04-22T13:23:57.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.879+0000 7f1cba646740 20 client.15291 trim_cache size 0 max 16384 2024-04-22T13:23:57.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.897+0000 7f1cba646740 1 Processor -- start 2024-04-22T13:23:57.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.897+0000 7f1cba646740 1 -- start start 2024-04-22T13:23:57.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.897+0000 7f1cba646740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e017b3e40 0x559e017c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.897+0000 7f1cba646740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559e017b4830 0x559e017c5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.898+0000 7f1cba646740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e017b52d0 0x559e017bbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.898+0000 7f1cba646740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559e017ca5a0 con 0x559e017b3e40 2024-04-22T13:23:57.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.898+0000 7f1cba646740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559e017cac70 con 0x559e017b4830 2024-04-22T13:23:57.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.898+0000 7f1cba646740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559e017cb340 con 0x559e017b52d0 2024-04-22T13:23:57.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.898+0000 7f1cb57f1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e017b3e40 0x559e017c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.898+0000 7f1cb57f1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e017b3e40 0x559e017c9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59964/0 (socket says 172.21.15.12:59964) 2024-04-22T13:23:57.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.898+0000 7f1cb57f1640 1 -- 172.21.15.12:0/3176797821 learned_addr learned my addr 172.21.15.12:0/3176797821 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:57.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.898+0000 7f1cb5ff2640 1 --2- 172.21.15.12:0/3176797821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e017b52d0 0x559e017bbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.898+0000 7f1cb57f1640 1 -- 172.21.15.12:0/3176797821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e017b52d0 msgr2=0x559e017bbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:57.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.898+0000 7f1cb4ff0640 1 --2- 172.21.15.12:0/3176797821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559e017b4830 0x559e017c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.898+0000 7f1cb57f1640 1 --2- 172.21.15.12:0/3176797821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e017b52d0 0x559e017bbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.898+0000 7f1cb57f1640 1 -- 172.21.15.12:0/3176797821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559e017b4830 msgr2=0x559e017c5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:57.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.898+0000 7f1cb57f1640 1 --2- 172.21.15.12:0/3176797821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559e017b4830 0x559e017c5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.898+0000 7f1cb57f1640 1 -- 172.21.15.12:0/3176797821 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e017bbae0 con 0x559e017b3e40 2024-04-22T13:23:57.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.898+0000 7f1cb5ff2640 1 --2- 172.21.15.12:0/3176797821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e017b52d0 0x559e017bbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:23:57.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.898+0000 7f1cb57f1640 1 --2- 172.21.15.12:0/3176797821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e017b3e40 0x559e017c9b70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1cac009900 tx=0x7f1cac005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82bab91756b29749 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:57.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.898+0000 7f1c9affd640 1 -- 172.21.15.12:0/3176797821 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac04e070 con 0x559e017b3e40 2024-04-22T13:23:57.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.898+0000 7f1c9affd640 1 -- 172.21.15.12:0/3176797821 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cac02c9e0 con 0x559e017b3e40 2024-04-22T13:23:57.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.898+0000 7f1c9affd640 1 -- 172.21.15.12:0/3176797821 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac045040 con 0x559e017b3e40 2024-04-22T13:23:57.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.899+0000 7f1cba646740 1 -- 172.21.15.12:0/3176797821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e017b3e40 msgr2=0x559e017c9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.899+0000 7f1cba646740 1 --2- 172.21.15.12:0/3176797821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e017b3e40 0x559e017c9b70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1cac009900 tx=0x7f1cac005c00 comp rx=0 tx=0).stop 2024-04-22T13:23:57.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.899+0000 7f1cba646740 1 -- 172.21.15.12:0/3176797821 shutdown_connections 2024-04-22T13:23:57.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.899+0000 7f1cba646740 1 --2- 172.21.15.12:0/3176797821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e017b52d0 0x559e017bbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.899+0000 7f1cba646740 1 --2- 172.21.15.12:0/3176797821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559e017b4830 0x559e017c5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.899+0000 7f1cba646740 1 --2- 172.21.15.12:0/3176797821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e017b3e40 0x559e017c9b70 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.899+0000 7f1cba646740 1 -- 172.21.15.12:0/3176797821 >> 172.21.15.12:0/3176797821 conn(0x559e017aff40 msgr2=0x559e017bd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:57.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.899+0000 7f1cba646740 1 -- 172.21.15.12:0/3176797821 shutdown_connections 2024-04-22T13:23:57.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.899+0000 7f1cba646740 1 -- 172.21.15.12:0/3176797821 wait complete. 2024-04-22T13:23:57.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.899+0000 7f1cba646740 1 -- start start 2024-04-22T13:23:57.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.899+0000 7f1cba646740 1 Processor -- start 2024-04-22T13:23:57.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.900+0000 7f1cba646740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e017b3e40 0x559e017c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.900+0000 7f1cba646740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559e017b4830 0x559e017c3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.900+0000 7f1cba646740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e017b52d0 0x559e017c3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.900+0000 7f1cba646740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559e017cc0e0 con 0x559e017b3e40 2024-04-22T13:23:57.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.900+0000 7f1cba646740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559e017cba10 con 0x559e017b4830 2024-04-22T13:23:57.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.900+0000 7f1cba646740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559e017cc7b0 con 0x559e017b52d0 2024-04-22T13:23:57.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.900+0000 7f1cba646740 10 client.0 _ll_register_callbacks cb 0x7f1ca8009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:23:57.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.900+0000 7f1cb57f1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e017b3e40 0x559e017c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.900+0000 7f1cb57f1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e017b3e40 0x559e017c1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59980/0 (socket says 172.21.15.12:59980) 2024-04-22T13:23:57.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.900+0000 7f1cb57f1640 1 -- 172.21.15.12:0/1757433033 learned_addr learned my addr 172.21.15.12:0/1757433033 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:57.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.900+0000 7f1cb5ff2640 1 --2- 172.21.15.12:0/1757433033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e017b52d0 0x559e017c3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.900+0000 7f1cb57f1640 1 -- 172.21.15.12:0/1757433033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e017b52d0 msgr2=0x559e017c3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:57.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.900+0000 7f1cb57f1640 1 --2- 172.21.15.12:0/1757433033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e017b52d0 0x559e017c3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.900+0000 7f1cb57f1640 1 -- 172.21.15.12:0/1757433033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559e017b4830 msgr2=0x559e017c3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:57.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.900+0000 7f1cb57f1640 1 --2- 172.21.15.12:0/1757433033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559e017b4830 0x559e017c3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.900+0000 7f1cb57f1640 1 -- 172.21.15.12:0/1757433033 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e017c41e0 con 0x559e017b3e40 2024-04-22T13:23:57.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.900+0000 7f1cb5ff2640 1 --2- 172.21.15.12:0/1757433033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e017b52d0 0x559e017c3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:23:57.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.900+0000 7f1cb57f1640 1 --2- 172.21.15.12:0/1757433033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e017b3e40 0x559e017c1210 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1cac02c170 tx=0x7f1cac0045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc0065f52c784e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:57.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.900+0000 7f1c98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:23:57.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.900+0000 7f1c98ff9640 1 -- 172.21.15.12:0/1757433033 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac045070 con 0x559e017b3e40 2024-04-22T13:23:57.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.900+0000 7f1c98ff9640 1 -- 172.21.15.12:0/1757433033 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f1cac04f680 con 0x559e017b3e40 2024-04-22T13:23:57.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.900+0000 7f1cba646740 1 -- 172.21.15.12:0/1757433033 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cac02c9e0 con 0x559e017b3e40 2024-04-22T13:23:57.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.900+0000 7f1c98ff9640 1 -- 172.21.15.12:0/1757433033 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1cac04e050 con 0x559e017b3e40 2024-04-22T13:23:57.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.900+0000 7f1cba646740 10 client.15297 fetch_fsmap learned FSMap version 27 2024-04-22T13:23:57.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.900+0000 7f1cba646740 1 -- 172.21.15.12:0/1757433033 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559e017c5410 con 0x559e017b3e40 2024-04-22T13:23:57.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.901+0000 7f1c98ff9640 1 -- 172.21.15.12:0/1757433033 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cac02c9e0 con 0x559e017b3e40 2024-04-22T13:23:57.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.901+0000 7f1c98ff9640 1 -- 172.21.15.12:0/1757433033 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f1ca80140f0 con 0x559e017b3e40 2024-04-22T13:23:57.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.901+0000 7f1cba646740 10 client.15297 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:23:57.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.901+0000 7f1cba646740 10 client.15297 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-22T13:23:57.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.901+0000 7f1cba646740 20 client.15297 populate_metadata read hostname 'smithi012' 2024-04-22T13:23:57.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.901+0000 7f1cba646740 1 --2- 172.21.15.12:0/1757433033 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x559e01834f30 0x559e01833c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:57.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.901+0000 7f1cba646740 4 client.15297 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T13:23:57.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.901+0000 7f1cba646740 1 -- 172.21.15.12:0/1757433033 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "status"}) -- 0x559e017203a0 con 0x559e01834f30 2024-04-22T13:23:57.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.901+0000 7f1cb4ff0640 1 --2- 172.21.15.12:0/1757433033 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x559e01834f30 0x559e01833c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:57.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.901+0000 7f1cb4ff0640 1 --2- 172.21.15.12:0/1757433033 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x559e01834f30 0x559e01833c40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e8ca9085f28619a8 server_cookie=f1ffb4a74d5fb949 in_seq=0 out_seq=0 2024-04-22T13:23:57.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.901+0000 7f1c98ff9640 10 client.15297 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:23:57.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.902+0000 7f1c98ff9640 1 -- 172.21.15.12:0/1757433033 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x559e017203a0 con 0x559e01834f30 2024-04-22T13:23:57.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.902+0000 7f1c98ff9640 10 client.15297 handle_command_reply: tid=0 2024-04-22T13:23:57.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.902+0000 7f1cba646740 1 client.15297 shutdown 2024-04-22T13:23:57.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.902+0000 7f1cba646740 1 -- 172.21.15.12:0/1757433033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e017b3e40 msgr2=0x559e017c1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.902+0000 7f1cba646740 1 --2- 172.21.15.12:0/1757433033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e017b3e40 0x559e017c1210 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1cac02c170 tx=0x7f1cac0045f0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.902+0000 7f1cba646740 1 -- 172.21.15.12:0/1757433033 shutdown_connections 2024-04-22T13:23:57.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.902+0000 7f1cba646740 1 --2- 172.21.15.12:0/1757433033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559e017b52d0 0x559e017c3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.902+0000 7f1cba646740 1 --2- 172.21.15.12:0/1757433033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559e017b4830 0x559e017c3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.902+0000 7f1cba646740 1 --2- 172.21.15.12:0/1757433033 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x559e01834f30 0x559e01833c40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.902+0000 7f1cba646740 1 --2- 172.21.15.12:0/1757433033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559e017b3e40 0x559e017c1210 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.902+0000 7f1c98ff9640 0 client.15297 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:23:57.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.902+0000 7f1cba646740 1 -- 172.21.15.12:0/1757433033 >> 172.21.15.12:0/1757433033 conn(0x559e017aff40 msgr2=0x559e017b8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:57.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.902+0000 7f1cba646740 1 -- 172.21.15.12:0/1757433033 shutdown_connections 2024-04-22T13:23:57.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.902+0000 7f1cba646740 1 -- 172.21.15.12:0/1757433033 wait complete. 2024-04-22T13:23:57.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.902+0000 7f1cba646740 20 client.15297 trim_cache size 0 max 16384 2024-04-22T13:23:57.905 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:23:57.905 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "0aede5ea-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:23:57.905 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 2, 2024-04-22T13:23:57.905 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24448, 2024-04-22T13:23:57.905 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-04-22T13:23:57.905 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-04-22T13:23:57.905 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-04-22T13:23:57.905 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 108.5282983, 2024-04-22T13:23:57.906 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 27, 2024-04-22T13:23:57.906 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 89, 2024-04-22T13:23:57.906 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 87, 2024-04-22T13:23:57.906 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 134.97603060500001 2024-04-22T13:23:57.906 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:23:57.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.903+0000 7f1cb7a55640 1 -- 172.21.15.12:0/733041566 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1c9c0a3c30 msgr2=0x7f1c9c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.903+0000 7f1cb7a55640 1 --2- 172.21.15.12:0/733041566 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1c9c0a3c30 0x7f1c9c0a60f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x559e017af200 tx=0x7f1ca8009530 comp rx=0 tx=0).stop 2024-04-22T13:23:57.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.903+0000 7f1cb7a55640 1 -- 172.21.15.12:0/733041566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cb0103190 msgr2=0x7f1cb0198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:57.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.903+0000 7f1cb7a55640 1 --2- 172.21.15.12:0/733041566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cb0103190 0x7f1cb0198780 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000d770 tx=0x7f1ca000dc30 comp rx=0 tx=0).stop 2024-04-22T13:23:57.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.903+0000 7f1cb7a55640 1 -- 172.21.15.12:0/733041566 shutdown_connections 2024-04-22T13:23:57.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.903+0000 7f1cb7a55640 1 --2- 172.21.15.12:0/733041566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1cb0103b50 0x7f1cb019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.903+0000 7f1cb7a55640 1 --2- 172.21.15.12:0/733041566 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1c9c0a3c30 0x7f1c9c0a60f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.903+0000 7f1cb7a55640 1 --2- 172.21.15.12:0/733041566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1cb0103190 0x7f1cb0198780 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.903+0000 7f1cb7a55640 1 --2- 172.21.15.12:0/733041566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cb0101fa0 0x7f1cb0198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:57.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.903+0000 7f1cb7a55640 1 -- 172.21.15.12:0/733041566 >> 172.21.15.12:0/733041566 conn(0x7f1cb00fd530 msgr2=0x7f1cb01064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:57.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.903+0000 7f1cb7a55640 1 -- 172.21.15.12:0/733041566 shutdown_connections 2024-04-22T13:23:57.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:57.903+0000 7f1cb7a55640 1 -- 172.21.15.12:0/733041566 wait complete. 2024-04-22T13:23:57.911 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-22T13:23:57.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:57 smithi111 ceph-mon[31361]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-22T13:23:57.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:57 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1994035554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:23:57.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:57 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/3979169932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:23:57.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:57 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/484665437' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:23:58.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.015+0000 7f537883a640 1 -- 172.21.15.12:0/2049053583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5374101fa0 msgr2=0x7f53741023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:58.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.016+0000 7f537883a640 1 --2- 172.21.15.12:0/2049053583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5374101fa0 0x7f53741023c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5364009900 tx=0x7f536402bc30 comp rx=0 tx=0).stop 2024-04-22T13:23:58.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.016+0000 7f537883a640 1 -- 172.21.15.12:0/2049053583 shutdown_connections 2024-04-22T13:23:58.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.016+0000 7f537883a640 1 --2- 172.21.15.12:0/2049053583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5374103b50 0x7f537410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.016+0000 7f537883a640 1 --2- 172.21.15.12:0/2049053583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5374103190 0x7f5374103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.016+0000 7f537883a640 1 --2- 172.21.15.12:0/2049053583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5374101fa0 0x7f53741023c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.016+0000 7f537883a640 1 -- 172.21.15.12:0/2049053583 >> 172.21.15.12:0/2049053583 conn(0x7f53740fd530 msgr2=0x7f53740ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:58.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.016+0000 7f537883a640 1 -- 172.21.15.12:0/2049053583 shutdown_connections 2024-04-22T13:23:58.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.016+0000 7f537883a640 1 -- 172.21.15.12:0/2049053583 wait complete. 2024-04-22T13:23:58.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.016+0000 7f537883a640 1 Processor -- start 2024-04-22T13:23:58.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.016+0000 7f537883a640 1 -- start start 2024-04-22T13:23:58.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.017+0000 7f537883a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5374101fa0 0x7f5374198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.017+0000 7f537883a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5374103190 0x7f53741986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.017+0000 7f537883a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5374103b50 0x7f537419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.017+0000 7f537883a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f537410dae0 con 0x7f5374103190 2024-04-22T13:23:58.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.017+0000 7f537883a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f537410d960 con 0x7f5374101fa0 2024-04-22T13:23:58.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.017+0000 7f537883a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f537410dc60 con 0x7f5374103b50 2024-04-22T13:23:58.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.017+0000 7f5371d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5374103190 0x7f53741986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.017+0000 7f5371d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5374103190 0x7f53741986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60006/0 (socket says 172.21.15.12:60006) 2024-04-22T13:23:58.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.017+0000 7f5371d9b640 1 -- 172.21.15.12:0/1048755873 learned_addr learned my addr 172.21.15.12:0/1048755873 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:58.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.017+0000 7f5372d9d640 1 --2- 172.21.15.12:0/1048755873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5374103b50 0x7f537419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.017+0000 7f537259c640 1 --2- 172.21.15.12:0/1048755873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5374101fa0 0x7f5374198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.017+0000 7f5371d9b640 1 -- 172.21.15.12:0/1048755873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5374103b50 msgr2=0x7f537419f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:58.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.017+0000 7f5371d9b640 1 --2- 172.21.15.12:0/1048755873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5374103b50 0x7f537419f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.017+0000 7f5371d9b640 1 -- 172.21.15.12:0/1048755873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5374101fa0 msgr2=0x7f5374198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:58.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.017+0000 7f5371d9b640 1 --2- 172.21.15.12:0/1048755873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5374101fa0 0x7f5374198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.017+0000 7f5371d9b640 1 -- 172.21.15.12:0/1048755873 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537419fd90 con 0x7f5374103190 2024-04-22T13:23:58.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.017+0000 7f5371d9b640 1 --2- 172.21.15.12:0/1048755873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5374103190 0x7f53741986b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f536800c920 tx=0x7f536800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:58.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.017+0000 7f53637fe640 1 -- 172.21.15.12:0/1048755873 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368007c70 con 0x7f5374103190 2024-04-22T13:23:58.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.017+0000 7f53637fe640 1 -- 172.21.15.12:0/1048755873 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5368007e10 con 0x7f5374103190 2024-04-22T13:23:58.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.017+0000 7f537883a640 1 -- 172.21.15.12:0/1048755873 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53741a0080 con 0x7f5374103190 2024-04-22T13:23:58.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.017+0000 7f53637fe640 1 -- 172.21.15.12:0/1048755873 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368018820 con 0x7f5374103190 2024-04-22T13:23:58.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.017+0000 7f537883a640 1 -- 172.21.15.12:0/1048755873 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53741a05d0 con 0x7f5374103190 2024-04-22T13:23:58.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.020+0000 7f53637fe640 1 -- 172.21.15.12:0/1048755873 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f536800f450 con 0x7f5374103190 2024-04-22T13:23:58.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.020+0000 7f537b420740 1 Processor -- start 2024-04-22T13:23:58.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.020+0000 7f537b420740 1 -- start start 2024-04-22T13:23:58.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.020+0000 7f537b420740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56201a0fae40 0x56201a0fb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.020+0000 7f537b420740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56201a0fb830 0x56201a0fbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.020+0000 7f537b420740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56201a0fc2d0 0x56201a102eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.020+0000 7f537b420740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56201a05fb10 con 0x56201a0fae40 2024-04-22T13:23:58.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.020+0000 7f537b420740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x562019ffd240 con 0x56201a0fb830 2024-04-22T13:23:58.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.021+0000 7f537b420740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56201a05bad0 con 0x56201a0fc2d0 2024-04-22T13:23:58.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.021+0000 7f5372d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56201a0fae40 0x56201a0fb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.021+0000 7f5372d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56201a0fae40 0x56201a0fb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60014/0 (socket says 172.21.15.12:60014) 2024-04-22T13:23:58.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.021+0000 7f5372d9d640 1 -- 172.21.15.12:0/3918439583 learned_addr learned my addr 172.21.15.12:0/3918439583 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:58.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.021+0000 7f53637fe640 1 --2- 172.21.15.12:0/1048755873 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f535c0a3bb0 0x7f535c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.021+0000 7f537259c640 1 --2- 172.21.15.12:0/3918439583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56201a0fb830 0x56201a0fbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.021+0000 7f5372d9d640 1 --2- 172.21.15.12:0/1048755873 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f535c0a3bb0 0x7f535c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.021+0000 7f53637fe640 1 -- 172.21.15.12:0/1048755873 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f53680d0590 con 0x7f5374103190 2024-04-22T13:23:58.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.021+0000 7f5371d9b640 1 --2- 172.21.15.12:0/3918439583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56201a0fc2d0 0x56201a102eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.021+0000 7f5372d9d640 1 -- 172.21.15.12:0/3918439583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56201a0fc2d0 msgr2=0x56201a102eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:58.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.021+0000 7f5372d9d640 1 --2- 172.21.15.12:0/3918439583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56201a0fc2d0 0x56201a102eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.021+0000 7f5372d9d640 1 -- 172.21.15.12:0/3918439583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56201a0fb830 msgr2=0x56201a0fbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:58.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.021+0000 7f5372d9d640 1 --2- 172.21.15.12:0/3918439583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56201a0fb830 0x56201a0fbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.021+0000 7f5372d9d640 1 -- 172.21.15.12:0/3918439583 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562019d722d0 con 0x56201a0fae40 2024-04-22T13:23:58.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.021+0000 7f537259c640 1 --2- 172.21.15.12:0/3918439583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56201a0fb830 0x56201a0fbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:23:58.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.021+0000 7f5372d9d640 1 --2- 172.21.15.12:0/3918439583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56201a0fae40 0x56201a0fb260 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f536c0149c0 tx=0x7f536c014e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d63c0378a475d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:58.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.021+0000 7f5372d9d640 1 --2- 172.21.15.12:0/1048755873 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f535c0a3bb0 0x7f535c0a6070 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f536c01c000 tx=0x7f536c01e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:58.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.021+0000 7f5360ff9640 1 -- 172.21.15.12:0/3918439583 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c014050 con 0x56201a0fae40 2024-04-22T13:23:58.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.021+0000 7f5360ff9640 1 -- 172.21.15.12:0/3918439583 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f536c0141f0 con 0x56201a0fae40 2024-04-22T13:23:58.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.021+0000 7f5360ff9640 1 -- 172.21.15.12:0/3918439583 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c013920 con 0x56201a0fae40 2024-04-22T13:23:58.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.022+0000 7f537b420740 1 -- 172.21.15.12:0/3918439583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56201a0fae40 msgr2=0x56201a0fb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:58.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.022+0000 7f537b420740 1 --2- 172.21.15.12:0/3918439583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56201a0fae40 0x56201a0fb260 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f536c0149c0 tx=0x7f536c014e80 comp rx=0 tx=0).stop 2024-04-22T13:23:58.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.022+0000 7f537b420740 1 -- 172.21.15.12:0/3918439583 shutdown_connections 2024-04-22T13:23:58.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.022+0000 7f537b420740 1 --2- 172.21.15.12:0/3918439583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56201a0fc2d0 0x56201a102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.022+0000 7f537b420740 1 --2- 172.21.15.12:0/3918439583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56201a0fb830 0x56201a0fbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.022+0000 7f537b420740 1 --2- 172.21.15.12:0/3918439583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56201a0fae40 0x56201a0fb260 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.022+0000 7f537b420740 1 -- 172.21.15.12:0/3918439583 >> 172.21.15.12:0/3918439583 conn(0x56201a0f6f40 msgr2=0x56201a0f5490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:58.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.022+0000 7f537b420740 1 -- 172.21.15.12:0/3918439583 shutdown_connections 2024-04-22T13:23:58.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.022+0000 7f537b420740 1 -- 172.21.15.12:0/3918439583 wait complete. 2024-04-22T13:23:58.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.023+0000 7f537b420740 1 -- start start 2024-04-22T13:23:58.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.023+0000 7f537b420740 1 Processor -- start 2024-04-22T13:23:58.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.023+0000 7f537b420740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56201a0fae40 0x56201a104e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.023+0000 7f537b420740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56201a0fb830 0x56201a105360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.023+0000 7f537b420740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56201a0fc2d0 0x56201a10c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.023+0000 7f537b420740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56201a103450 con 0x56201a0fb830 2024-04-22T13:23:58.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.023+0000 7f537b420740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x562019fe8190 con 0x56201a0fc2d0 2024-04-22T13:23:58.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.023+0000 7f537b420740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56201a1035d0 con 0x56201a0fae40 2024-04-22T13:23:58.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.023+0000 7f537b420740 10 client.0 _ll_register_callbacks cb 0x562019fe38b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:23:58.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.023+0000 7f5371d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56201a0fb830 0x56201a105360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.023+0000 7f5371d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56201a0fb830 0x56201a105360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60030/0 (socket says 172.21.15.12:60030) 2024-04-22T13:23:58.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.023+0000 7f5371d9b640 1 -- 172.21.15.12:0/3322635615 learned_addr learned my addr 172.21.15.12:0/3322635615 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:58.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.023+0000 7f537259c640 1 --2- 172.21.15.12:0/3322635615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56201a0fae40 0x56201a104e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.023+0000 7f5372d9d640 1 --2- 172.21.15.12:0/3322635615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56201a0fc2d0 0x56201a10c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.023+0000 7f5371d9b640 1 -- 172.21.15.12:0/3322635615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56201a0fae40 msgr2=0x56201a104e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:58.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.023+0000 7f5371d9b640 1 --2- 172.21.15.12:0/3322635615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56201a0fae40 0x56201a104e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.023+0000 7f5371d9b640 1 -- 172.21.15.12:0/3322635615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56201a0fc2d0 msgr2=0x56201a10c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:58.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.023+0000 7f5371d9b640 1 --2- 172.21.15.12:0/3322635615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56201a0fc2d0 0x56201a10c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.023+0000 7f5371d9b640 1 -- 172.21.15.12:0/3322635615 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562019e3c6b0 con 0x56201a0fb830 2024-04-22T13:23:58.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.023+0000 7f5371d9b640 1 --2- 172.21.15.12:0/3322635615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56201a0fb830 0x56201a105360 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f53680ce110 tx=0x7f536809d060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc30a9af0ce90bef server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:58.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.023+0000 7f5359ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:23:58.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.024+0000 7f5359ffb640 1 -- 172.21.15.12:0/3322635615 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53680a5510 con 0x56201a0fb830 2024-04-22T13:23:58.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.024+0000 7f5359ffb640 1 -- 172.21.15.12:0/3322635615 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f536800fa90 con 0x56201a0fb830 2024-04-22T13:23:58.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.024+0000 7f537b420740 1 -- 172.21.15.12:0/3322635615 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f536c0141f0 con 0x56201a0fb830 2024-04-22T13:23:58.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.024+0000 7f5359ffb640 1 -- 172.21.15.12:0/3322635615 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5368098f10 con 0x56201a0fb830 2024-04-22T13:23:58.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.024+0000 7f537b420740 10 client.15309 fetch_fsmap learned FSMap version 27 2024-04-22T13:23:58.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.024+0000 7f537b420740 1 -- 172.21.15.12:0/3322635615 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56201a10f7e0 con 0x56201a0fb830 2024-04-22T13:23:58.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.024+0000 7f5359ffb640 1 -- 172.21.15.12:0/3322635615 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f536c0141f0 con 0x56201a0fb830 2024-04-22T13:23:58.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.024+0000 7f5359ffb640 1 -- 172.21.15.12:0/3322635615 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5368067260 con 0x56201a0fb830 2024-04-22T13:23:58.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.024+0000 7f537b420740 10 client.15309 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:23:58.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.024+0000 7f537b420740 10 client.15309 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-22T13:23:58.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.024+0000 7f537b420740 20 client.15309 populate_metadata read hostname 'smithi012' 2024-04-22T13:23:58.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.024+0000 7f537b420740 1 --2- 172.21.15.12:0/3322635615 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56201a111e30 0x56201a1132a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.024+0000 7f537b420740 4 client.15309 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:23:58.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.024+0000 7f537b420740 1 -- 172.21.15.12:0/3322635615 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56201a06d530 con 0x56201a111e30 2024-04-22T13:23:58.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.025+0000 7f537259c640 1 --2- 172.21.15.12:0/3322635615 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56201a111e30 0x56201a1132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.025+0000 7f537259c640 1 --2- 172.21.15.12:0/3322635615 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56201a111e30 0x56201a1132a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e19f252da80d51c server_cookie=f4b9b04742e55855 in_seq=0 out_seq=0 2024-04-22T13:23:58.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.025+0000 7f5359ffb640 10 client.15309 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:23:58.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.026+0000 7f5359ffb640 1 -- 172.21.15.12:0/3322635615 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56201a06d530 con 0x56201a111e30 2024-04-22T13:23:58.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.026+0000 7f5359ffb640 10 client.15309 handle_command_reply: tid=0 2024-04-22T13:23:58.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.026+0000 7f537b420740 1 client.15309 shutdown 2024-04-22T13:23:58.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.026+0000 7f537b420740 1 -- 172.21.15.12:0/3322635615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56201a0fb830 msgr2=0x56201a105360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:58.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.026+0000 7f537b420740 1 --2- 172.21.15.12:0/3322635615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56201a0fb830 0x56201a105360 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f53680ce110 tx=0x7f536809d060 comp rx=0 tx=0).stop 2024-04-22T13:23:58.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.026+0000 7f537b420740 1 -- 172.21.15.12:0/3322635615 shutdown_connections 2024-04-22T13:23:58.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.026+0000 7f537b420740 1 --2- 172.21.15.12:0/3322635615 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56201a111e30 0x56201a1132a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.026+0000 7f537b420740 1 --2- 172.21.15.12:0/3322635615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56201a0fc2d0 0x56201a10c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.026+0000 7f5359ffb640 0 client.15309 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:23:58.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.026+0000 7f537b420740 1 --2- 172.21.15.12:0/3322635615 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56201a0fb830 0x56201a105360 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.026+0000 7f537b420740 1 --2- 172.21.15.12:0/3322635615 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56201a0fae40 0x56201a104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.026+0000 7f537b420740 1 -- 172.21.15.12:0/3322635615 >> 172.21.15.12:0/3322635615 conn(0x56201a0f6f40 msgr2=0x56201a0f4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:58.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.026+0000 7f537b420740 1 -- 172.21.15.12:0/3322635615 shutdown_connections 2024-04-22T13:23:58.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.026+0000 7f537b420740 1 -- 172.21.15.12:0/3322635615 wait complete. 2024-04-22T13:23:58.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.026+0000 7f537b420740 20 client.15309 trim_cache size 0 max 16384 2024-04-22T13:23:58.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.044+0000 7f537b420740 1 Processor -- start 2024-04-22T13:23:58.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.044+0000 7f537b420740 1 -- start start 2024-04-22T13:23:58.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.044+0000 7f537b420740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56201a0fae40 0x56201a0f5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.044+0000 7f537b420740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56201a0fb830 0x56201a0f5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.044+0000 7f537b420740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56201a0fc2d0 0x56201a10c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.044+0000 7f537b420740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56201a1115e0 con 0x56201a0fb830 2024-04-22T13:23:58.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.044+0000 7f537b420740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56201a111cb0 con 0x56201a0fae40 2024-04-22T13:23:58.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.044+0000 7f537b420740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56201a112380 con 0x56201a0fc2d0 2024-04-22T13:23:58.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.045+0000 7f5371d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56201a0fb830 0x56201a0f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.045+0000 7f5371d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56201a0fb830 0x56201a0f5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60032/0 (socket says 172.21.15.12:60032) 2024-04-22T13:23:58.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.045+0000 7f5371d9b640 1 -- 172.21.15.12:0/2452321986 learned_addr learned my addr 172.21.15.12:0/2452321986 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:58.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.045+0000 7f5371d9b640 1 -- 172.21.15.12:0/2452321986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56201a0fc2d0 msgr2=0x56201a10c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:58.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.045+0000 7f5371d9b640 1 --2- 172.21.15.12:0/2452321986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56201a0fc2d0 0x56201a10c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.045+0000 7f5371d9b640 1 -- 172.21.15.12:0/2452321986 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56201a0fae40 msgr2=0x56201a0f5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:58.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.045+0000 7f5371d9b640 1 --2- 172.21.15.12:0/2452321986 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56201a0fae40 0x56201a0f5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.045+0000 7f5371d9b640 1 -- 172.21.15.12:0/2452321986 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56201a103710 con 0x56201a0fb830 2024-04-22T13:23:58.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.045+0000 7f5371d9b640 1 --2- 172.21.15.12:0/2452321986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56201a0fb830 0x56201a0f5940 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f53680a4fe0 tx=0x7f536800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f64c5df0488d69 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:58.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.045+0000 7f535bfff640 1 -- 172.21.15.12:0/2452321986 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536800fa90 con 0x56201a0fb830 2024-04-22T13:23:58.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.045+0000 7f535bfff640 1 -- 172.21.15.12:0/2452321986 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5368067290 con 0x56201a0fb830 2024-04-22T13:23:58.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.045+0000 7f535bfff640 1 -- 172.21.15.12:0/2452321986 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53680872d0 con 0x56201a0fb830 2024-04-22T13:23:58.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.046+0000 7f537b420740 1 -- 172.21.15.12:0/2452321986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56201a0fb830 msgr2=0x56201a0f5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:58.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.046+0000 7f537b420740 1 --2- 172.21.15.12:0/2452321986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56201a0fb830 0x56201a0f5940 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f53680a4fe0 tx=0x7f536800cf90 comp rx=0 tx=0).stop 2024-04-22T13:23:58.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.046+0000 7f537b420740 1 -- 172.21.15.12:0/2452321986 shutdown_connections 2024-04-22T13:23:58.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.046+0000 7f537b420740 1 --2- 172.21.15.12:0/2452321986 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56201a0fc2d0 0x56201a10c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.046+0000 7f537b420740 1 --2- 172.21.15.12:0/2452321986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56201a0fb830 0x56201a0f5940 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.046+0000 7f537b420740 1 --2- 172.21.15.12:0/2452321986 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56201a0fae40 0x56201a0f5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.046+0000 7f537b420740 1 -- 172.21.15.12:0/2452321986 >> 172.21.15.12:0/2452321986 conn(0x56201a0f6f40 msgr2=0x56201a104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:58.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.046+0000 7f537b420740 1 -- 172.21.15.12:0/2452321986 shutdown_connections 2024-04-22T13:23:58.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.046+0000 7f537b420740 1 -- 172.21.15.12:0/2452321986 wait complete. 2024-04-22T13:23:58.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.046+0000 7f537b420740 1 -- start start 2024-04-22T13:23:58.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.046+0000 7f537b420740 1 Processor -- start 2024-04-22T13:23:58.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.046+0000 7f537b420740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56201a0fae40 0x56201a1080f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.046+0000 7f537b420740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56201a0fb830 0x56201a108630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.046+0000 7f537b420740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56201a0fc2d0 0x56201a10b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.046+0000 7f537b420740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56201a113120 con 0x56201a0fc2d0 2024-04-22T13:23:58.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.046+0000 7f537b420740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56201a112a50 con 0x56201a0fb830 2024-04-22T13:23:58.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.046+0000 7f537b420740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56201a1137f0 con 0x56201a0fae40 2024-04-22T13:23:58.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.046+0000 7f537b420740 10 client.0 _ll_register_callbacks cb 0x7f536800fd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:23:58.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.046+0000 7f5372d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56201a0fc2d0 0x56201a10b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.046+0000 7f5372d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56201a0fc2d0 0x56201a10b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60044/0 (socket says 172.21.15.12:60044) 2024-04-22T13:23:58.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.046+0000 7f5372d9d640 1 -- 172.21.15.12:0/2140984645 learned_addr learned my addr 172.21.15.12:0/2140984645 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:58.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.046+0000 7f537259c640 1 --2- 172.21.15.12:0/2140984645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56201a0fae40 0x56201a1080f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.047+0000 7f5371d9b640 1 --2- 172.21.15.12:0/2140984645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56201a0fb830 0x56201a108630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.047+0000 7f5372d9d640 1 -- 172.21.15.12:0/2140984645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56201a0fae40 msgr2=0x56201a1080f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:58.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.047+0000 7f5372d9d640 1 --2- 172.21.15.12:0/2140984645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56201a0fae40 0x56201a1080f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.047+0000 7f5372d9d640 1 -- 172.21.15.12:0/2140984645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56201a0fb830 msgr2=0x56201a108630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:58.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.047+0000 7f5372d9d640 1 --2- 172.21.15.12:0/2140984645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56201a0fb830 0x56201a108630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.047+0000 7f5372d9d640 1 -- 172.21.15.12:0/2140984645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56201a10bc40 con 0x56201a0fc2d0 2024-04-22T13:23:58.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.047+0000 7f537259c640 1 --2- 172.21.15.12:0/2140984645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56201a0fae40 0x56201a1080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:23:58.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.047+0000 7f5372d9d640 1 --2- 172.21.15.12:0/2140984645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56201a0fc2d0 0x56201a10b650 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x56201a0f6200 tx=0x7f536c0148f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b74e4dbe11a739 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:58.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.047+0000 7f5359ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:23:58.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.047+0000 7f5359ffb640 1 -- 172.21.15.12:0/2140984645 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c02e020 con 0x56201a0fc2d0 2024-04-22T13:23:58.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.047+0000 7f5359ffb640 1 -- 172.21.15.12:0/2140984645 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f536c012080 con 0x56201a0fc2d0 2024-04-22T13:23:58.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.047+0000 7f537b420740 1 -- 172.21.15.12:0/2140984645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5368067290 con 0x56201a0fc2d0 2024-04-22T13:23:58.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.047+0000 7f5359ffb640 1 -- 172.21.15.12:0/2140984645 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f536c009860 con 0x56201a0fc2d0 2024-04-22T13:23:58.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.047+0000 7f537b420740 10 client.15315 fetch_fsmap learned FSMap version 27 2024-04-22T13:23:58.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.047+0000 7f537b420740 1 -- 172.21.15.12:0/2140984645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56201a1784f0 con 0x56201a0fc2d0 2024-04-22T13:23:58.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.047+0000 7f5359ffb640 1 -- 172.21.15.12:0/2140984645 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5368067290 con 0x56201a0fc2d0 2024-04-22T13:23:58.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.047+0000 7f5359ffb640 1 -- 172.21.15.12:0/2140984645 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f536c029040 con 0x56201a0fc2d0 2024-04-22T13:23:58.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.047+0000 7f537b420740 10 client.15315 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:23:58.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.048+0000 7f537b420740 10 client.15315 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-22T13:23:58.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.048+0000 7f537b420740 20 client.15315 populate_metadata read hostname 'smithi012' 2024-04-22T13:23:58.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.048+0000 7f537b420740 1 --2- 172.21.15.12:0/2140984645 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56201a17cdf0 0x56201a17bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.048+0000 7f537b420740 4 client.15315 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T13:23:58.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.048+0000 7f537b420740 1 -- 172.21.15.12:0/2140984645 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "status"}) -- 0x56201a0673a0 con 0x56201a17cdf0 2024-04-22T13:23:58.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.048+0000 7f537259c640 1 --2- 172.21.15.12:0/2140984645 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56201a17cdf0 0x56201a17bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.048+0000 7f5359ffb640 10 client.15315 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:23:58.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.048+0000 7f537259c640 1 --2- 172.21.15.12:0/2140984645 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56201a17cdf0 0x56201a17bb00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f726b4a1bfa2ec7b server_cookie=13cf9d71f7952eed in_seq=0 out_seq=0 2024-04-22T13:23:58.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.048+0000 7f5359ffb640 1 -- 172.21.15.12:0/2140984645 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x56201a0673a0 con 0x56201a17cdf0 2024-04-22T13:23:58.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.048+0000 7f5359ffb640 10 client.15315 handle_command_reply: tid=0 2024-04-22T13:23:58.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.048+0000 7f537b420740 1 client.15315 shutdown 2024-04-22T13:23:58.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.049+0000 7f537b420740 1 -- 172.21.15.12:0/2140984645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56201a0fc2d0 msgr2=0x56201a10b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:58.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.049+0000 7f537b420740 1 --2- 172.21.15.12:0/2140984645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56201a0fc2d0 0x56201a10b650 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x56201a0f6200 tx=0x7f536c0148f0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.049+0000 7f537b420740 1 -- 172.21.15.12:0/2140984645 shutdown_connections 2024-04-22T13:23:58.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.049+0000 7f537b420740 1 --2- 172.21.15.12:0/2140984645 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56201a17cdf0 0x56201a17bb00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.049+0000 7f537b420740 1 --2- 172.21.15.12:0/2140984645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56201a0fc2d0 0x56201a10b650 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.049+0000 7f537b420740 1 --2- 172.21.15.12:0/2140984645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56201a0fb830 0x56201a108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.049+0000 7f537b420740 1 --2- 172.21.15.12:0/2140984645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56201a0fae40 0x56201a1080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.049+0000 7f5359ffb640 0 client.15315 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:23:58.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.049+0000 7f537b420740 1 -- 172.21.15.12:0/2140984645 >> 172.21.15.12:0/2140984645 conn(0x56201a0f6f40 msgr2=0x56201a0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:58.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.049+0000 7f537b420740 1 -- 172.21.15.12:0/2140984645 shutdown_connections 2024-04-22T13:23:58.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.049+0000 7f537b420740 1 -- 172.21.15.12:0/2140984645 wait complete. 2024-04-22T13:23:58.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.049+0000 7f537b420740 20 client.15315 trim_cache size 0 max 16384 2024-04-22T13:23:58.052 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:23:58.052 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "0aede5ea-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:23:58.052 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 1, 2024-04-22T13:23:58.052 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24484, 2024-04-22T13:23:58.052 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-04-22T13:23:58.052 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-04-22T13:23:58.053 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-04-22T13:23:58.053 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 109.661255869, 2024-04-22T13:23:58.053 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 27, 2024-04-22T13:23:58.053 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 89, 2024-04-22T13:23:58.053 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 87, 2024-04-22T13:23:58.053 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 127.033851293 2024-04-22T13:23:58.053 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:23:58.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.049+0000 7f537883a640 1 -- 172.21.15.12:0/1048755873 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f535c0a3bb0 msgr2=0x7f535c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:58.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.049+0000 7f537883a640 1 --2- 172.21.15.12:0/1048755873 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f535c0a3bb0 0x7f535c0a6070 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f536c01c000 tx=0x7f536c01e000 comp rx=0 tx=0).stop 2024-04-22T13:23:58.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.049+0000 7f537883a640 1 -- 172.21.15.12:0/1048755873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5374103190 msgr2=0x7f53741986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:58.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.049+0000 7f537883a640 1 --2- 172.21.15.12:0/1048755873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5374103190 0x7f53741986b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f536800c920 tx=0x7f536800cde0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.049+0000 7f537883a640 1 -- 172.21.15.12:0/1048755873 shutdown_connections 2024-04-22T13:23:58.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.049+0000 7f537883a640 1 --2- 172.21.15.12:0/1048755873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5374103b50 0x7f537419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.049+0000 7f537883a640 1 --2- 172.21.15.12:0/1048755873 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f535c0a3bb0 0x7f535c0a6070 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.049+0000 7f537883a640 1 --2- 172.21.15.12:0/1048755873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5374103190 0x7f53741986b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.049+0000 7f537883a640 1 --2- 172.21.15.12:0/1048755873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5374101fa0 0x7f5374198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.049+0000 7f537883a640 1 -- 172.21.15.12:0/1048755873 >> 172.21.15.12:0/1048755873 conn(0x7f53740fd530 msgr2=0x7f5374106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:58.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.049+0000 7f537883a640 1 -- 172.21.15.12:0/1048755873 shutdown_connections 2024-04-22T13:23:58.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.049+0000 7f537883a640 1 -- 172.21.15.12:0/1048755873 wait complete. 2024-04-22T13:23:58.056 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:57 smithi012 ceph-mon[28265]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-22T13:23:58.056 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:57 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1994035554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:23:58.057 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:57 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3979169932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:23:58.057 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:57 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/484665437' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:23:58.057 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-22T13:23:58.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.162+0000 7fe85ae1f640 1 -- 172.21.15.12:0/1003704622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe854102a80 msgr2=0x7fe854104e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:58.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.162+0000 7fe85ae1f640 1 --2- 172.21.15.12:0/1003704622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe854102a80 0x7fe854104e70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe848009900 tx=0x7fe84802bc30 comp rx=0 tx=0).stop 2024-04-22T13:23:58.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.162+0000 7fe85ae1f640 1 -- 172.21.15.12:0/1003704622 shutdown_connections 2024-04-22T13:23:58.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.162+0000 7fe85ae1f640 1 --2- 172.21.15.12:0/1003704622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe854107ce0 0x7fe85410a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.162+0000 7fe85ae1f640 1 --2- 172.21.15.12:0/1003704622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8541053b0 0x7fe8541077a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.162+0000 7fe85ae1f640 1 --2- 172.21.15.12:0/1003704622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe854102a80 0x7fe854104e70 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.162+0000 7fe85ae1f640 1 -- 172.21.15.12:0/1003704622 >> 172.21.15.12:0/1003704622 conn(0x7fe8540fc410 msgr2=0x7fe8540fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:58.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.162+0000 7fe85ae1f640 1 -- 172.21.15.12:0/1003704622 shutdown_connections 2024-04-22T13:23:58.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.162+0000 7fe85ae1f640 1 -- 172.21.15.12:0/1003704622 wait complete. 2024-04-22T13:23:58.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.163+0000 7fe85ae1f640 1 Processor -- start 2024-04-22T13:23:58.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.163+0000 7fe85ae1f640 1 -- start start 2024-04-22T13:23:58.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.163+0000 7fe85ae1f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe854102a80 0x7fe854198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.163+0000 7fe85ae1f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8541053b0 0x7fe8541987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.163+0000 7fe85ae1f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe854107ce0 0x7fe85419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.163+0000 7fe85ae1f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe85410dae0 con 0x7fe854102a80 2024-04-22T13:23:58.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.163+0000 7fe85ae1f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe85410d960 con 0x7fe854107ce0 2024-04-22T13:23:58.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.163+0000 7fe85ae1f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe85410dc60 con 0x7fe8541053b0 2024-04-22T13:23:58.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.163+0000 7fe858bbb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe854102a80 0x7fe854198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.163+0000 7fe858bbb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe854102a80 0x7fe854198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60056/0 (socket says 172.21.15.12:60056) 2024-04-22T13:23:58.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.163+0000 7fe858bbb640 1 -- 172.21.15.12:0/3686111722 learned_addr learned my addr 172.21.15.12:0/3686111722 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:58.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.163+0000 7fe853fff640 1 --2- 172.21.15.12:0/3686111722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8541053b0 0x7fe8541987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.163+0000 7fe858bbb640 1 -- 172.21.15.12:0/3686111722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8541053b0 msgr2=0x7fe8541987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:58.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.163+0000 7fe8593bc640 1 --2- 172.21.15.12:0/3686111722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe854107ce0 0x7fe85419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.163+0000 7fe858bbb640 1 --2- 172.21.15.12:0/3686111722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8541053b0 0x7fe8541987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.163+0000 7fe858bbb640 1 -- 172.21.15.12:0/3686111722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe854107ce0 msgr2=0x7fe85419f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:58.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.163+0000 7fe858bbb640 1 --2- 172.21.15.12:0/3686111722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe854107ce0 0x7fe85419f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.163+0000 7fe858bbb640 1 -- 172.21.15.12:0/3686111722 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe85419fe80 con 0x7fe854102a80 2024-04-22T13:23:58.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.163+0000 7fe8593bc640 1 --2- 172.21.15.12:0/3686111722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe854107ce0 0x7fe85419f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:23:58.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.164+0000 7fe858bbb640 1 --2- 172.21.15.12:0/3686111722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe854102a80 0x7fe854198260 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe848006010 tx=0x7fe8480040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:58.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.164+0000 7fe851ffb640 1 -- 172.21.15.12:0/3686111722 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe848043020 con 0x7fe854102a80 2024-04-22T13:23:58.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.164+0000 7fe851ffb640 1 -- 172.21.15.12:0/3686111722 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8480044c0 con 0x7fe854102a80 2024-04-22T13:23:58.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.164+0000 7fe851ffb640 1 -- 172.21.15.12:0/3686111722 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe848003c60 con 0x7fe854102a80 2024-04-22T13:23:58.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.164+0000 7fe85ae1f640 1 -- 172.21.15.12:0/3686111722 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8541a0110 con 0x7fe854102a80 2024-04-22T13:23:58.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.164+0000 7fe85ae1f640 1 -- 172.21.15.12:0/3686111722 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8541a0610 con 0x7fe854102a80 2024-04-22T13:23:58.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.166+0000 7fe851ffb640 1 -- 172.21.15.12:0/3686111722 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe848046020 con 0x7fe854102a80 2024-04-22T13:23:58.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.166+0000 7fe85da0d740 1 Processor -- start 2024-04-22T13:23:58.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.166+0000 7fe85da0d740 1 -- start start 2024-04-22T13:23:58.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.166+0000 7fe85da0d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c24de33e40 0x55c24de34260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.166+0000 7fe85da0d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c24de34830 0x55c24de34cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.166+0000 7fe85da0d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c24de352d0 0x55c24de3beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.166+0000 7fe85da0d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c24dd98b10 con 0x55c24de33e40 2024-04-22T13:23:58.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.166+0000 7fe85da0d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c24dd36240 con 0x55c24de352d0 2024-04-22T13:23:58.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.166+0000 7fe85da0d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c24dd94ad0 con 0x55c24de34830 2024-04-22T13:23:58.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.167+0000 7fe8593bc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c24de33e40 0x55c24de34260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.167+0000 7fe8593bc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c24de33e40 0x55c24de34260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60066/0 (socket says 172.21.15.12:60066) 2024-04-22T13:23:58.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.167+0000 7fe8593bc640 1 -- 172.21.15.12:0/3897186814 learned_addr learned my addr 172.21.15.12:0/3897186814 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:58.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.167+0000 7fe853fff640 1 --2- 172.21.15.12:0/3897186814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c24de352d0 0x55c24de3beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.167+0000 7fe858bbb640 1 --2- 172.21.15.12:0/3897186814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c24de34830 0x55c24de34cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.167+0000 7fe8593bc640 1 -- 172.21.15.12:0/3897186814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c24de34830 msgr2=0x55c24de34cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:58.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.167+0000 7fe8593bc640 1 --2- 172.21.15.12:0/3897186814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c24de34830 0x55c24de34cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.167+0000 7fe8593bc640 1 -- 172.21.15.12:0/3897186814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c24de352d0 msgr2=0x55c24de3beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:58.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.167+0000 7fe8593bc640 1 --2- 172.21.15.12:0/3897186814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c24de352d0 0x55c24de3beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.167+0000 7fe8593bc640 1 -- 172.21.15.12:0/3897186814 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c24daab2d0 con 0x55c24de33e40 2024-04-22T13:23:58.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.167+0000 7fe853fff640 1 --2- 172.21.15.12:0/3897186814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c24de352d0 0x55c24de3beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:23:58.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.167+0000 7fe851ffb640 1 --2- 172.21.15.12:0/3686111722 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe8440a3bb0 0x7fe8440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.167+0000 7fe858bbb640 1 --2- 172.21.15.12:0/3686111722 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe8440a3bb0 0x7fe8440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.167+0000 7fe8593bc640 1 --2- 172.21.15.12:0/3897186814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c24de33e40 0x55c24de34260 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe84c008d50 tx=0x7fe84c00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff17bb5761bdc597 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:58.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.167+0000 7fe851ffb640 1 -- 172.21.15.12:0/3686111722 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fe8480f2800 con 0x7fe854102a80 2024-04-22T13:23:58.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.167+0000 7fe83effd640 1 -- 172.21.15.12:0/3897186814 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c00f830 con 0x55c24de33e40 2024-04-22T13:23:58.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.167+0000 7fe83effd640 1 -- 172.21.15.12:0/3897186814 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe84c00de20 con 0x55c24de33e40 2024-04-22T13:23:58.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.167+0000 7fe83effd640 1 -- 172.21.15.12:0/3897186814 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c018e10 con 0x55c24de33e40 2024-04-22T13:23:58.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.168+0000 7fe85da0d740 1 -- 172.21.15.12:0/3897186814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c24de33e40 msgr2=0x55c24de34260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:58.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.168+0000 7fe858bbb640 1 --2- 172.21.15.12:0/3686111722 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe8440a3bb0 0x7fe8440a6070 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe8480ac290 tx=0x7fe8480f6040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:58.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.168+0000 7fe85da0d740 1 --2- 172.21.15.12:0/3897186814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c24de33e40 0x55c24de34260 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe84c008d50 tx=0x7fe84c00d710 comp rx=0 tx=0).stop 2024-04-22T13:23:58.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.168+0000 7fe85da0d740 1 -- 172.21.15.12:0/3897186814 shutdown_connections 2024-04-22T13:23:58.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.168+0000 7fe85da0d740 1 --2- 172.21.15.12:0/3897186814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c24de352d0 0x55c24de3beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.168+0000 7fe85da0d740 1 --2- 172.21.15.12:0/3897186814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c24de34830 0x55c24de34cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.168+0000 7fe85da0d740 1 --2- 172.21.15.12:0/3897186814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c24de33e40 0x55c24de34260 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.168+0000 7fe85da0d740 1 -- 172.21.15.12:0/3897186814 >> 172.21.15.12:0/3897186814 conn(0x55c24de2ff40 msgr2=0x55c24de2e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:58.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.168+0000 7fe85da0d740 1 -- 172.21.15.12:0/3897186814 shutdown_connections 2024-04-22T13:23:58.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.168+0000 7fe85da0d740 1 -- 172.21.15.12:0/3897186814 wait complete. 2024-04-22T13:23:58.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.168+0000 7fe85da0d740 1 -- start start 2024-04-22T13:23:58.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.169+0000 7fe85da0d740 1 Processor -- start 2024-04-22T13:23:58.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.169+0000 7fe85da0d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c24de33e40 0x55c24de3df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.169+0000 7fe85da0d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c24de34830 0x55c24de3e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.169+0000 7fe85da0d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c24de352d0 0x55c24de45450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.169+0000 7fe85da0d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c24de3c4b0 con 0x55c24de352d0 2024-04-22T13:23:58.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.169+0000 7fe85da0d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c24dd21190 con 0x55c24de34830 2024-04-22T13:23:58.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.169+0000 7fe85da0d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c24de3c630 con 0x55c24de33e40 2024-04-22T13:23:58.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.169+0000 7fe85da0d740 10 client.0 _ll_register_callbacks cb 0x55c24dd1c8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:23:58.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.169+0000 7fe8593bc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c24de352d0 0x55c24de45450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.169+0000 7fe8593bc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c24de352d0 0x55c24de45450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60076/0 (socket says 172.21.15.12:60076) 2024-04-22T13:23:58.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.169+0000 7fe8593bc640 1 -- 172.21.15.12:0/3961125317 learned_addr learned my addr 172.21.15.12:0/3961125317 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:58.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.169+0000 7fe853fff640 1 --2- 172.21.15.12:0/3961125317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c24de34830 0x55c24de3e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.169+0000 7fe8593bc640 1 --2- 172.21.15.12:0/3961125317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c24de33e40 0x55c24de3df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.169+0000 7fe8593bc640 1 -- 172.21.15.12:0/3961125317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c24de33e40 msgr2=0x55c24de3df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:58.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.169+0000 7fe8593bc640 1 --2- 172.21.15.12:0/3961125317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c24de33e40 0x55c24de3df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.169+0000 7fe8593bc640 1 -- 172.21.15.12:0/3961125317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c24de34830 msgr2=0x55c24de3e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:58.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.169+0000 7fe8593bc640 1 --2- 172.21.15.12:0/3961125317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c24de34830 0x55c24de3e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.169+0000 7fe8593bc640 1 -- 172.21.15.12:0/3961125317 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c24db756b0 con 0x55c24de352d0 2024-04-22T13:23:58.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.169+0000 7fe8593bc640 1 --2- 172.21.15.12:0/3961125317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c24de352d0 0x55c24de45450 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe84c018570 tx=0x7fe84c008f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88206cbff10806ad server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:58.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.169+0000 7fe833fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:23:58.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.170+0000 7fe833fff640 1 -- 172.21.15.12:0/3961125317 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c008990 con 0x55c24de352d0 2024-04-22T13:23:58.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.170+0000 7fe833fff640 1 -- 172.21.15.12:0/3961125317 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fe84c00f840 con 0x55c24de352d0 2024-04-22T13:23:58.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.170+0000 7fe85da0d740 1 -- 172.21.15.12:0/3961125317 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe84c00de20 con 0x55c24de352d0 2024-04-22T13:23:58.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.170+0000 7fe833fff640 1 -- 172.21.15.12:0/3961125317 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe84c00fa40 con 0x55c24de352d0 2024-04-22T13:23:58.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.170+0000 7fe85da0d740 10 client.15327 fetch_fsmap learned FSMap version 27 2024-04-22T13:23:58.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.170+0000 7fe85da0d740 1 -- 172.21.15.12:0/3961125317 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c24de487c0 con 0x55c24de352d0 2024-04-22T13:23:58.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.170+0000 7fe833fff640 1 -- 172.21.15.12:0/3961125317 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe84c00de20 con 0x55c24de352d0 2024-04-22T13:23:58.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.170+0000 7fe833fff640 1 -- 172.21.15.12:0/3961125317 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe84c023070 con 0x55c24de352d0 2024-04-22T13:23:58.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.170+0000 7fe85da0d740 10 client.15327 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:23:58.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.170+0000 7fe85da0d740 10 client.15327 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-22T13:23:58.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.170+0000 7fe85da0d740 20 client.15327 populate_metadata read hostname 'smithi012' 2024-04-22T13:23:58.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.170+0000 7fe85da0d740 1 --2- 172.21.15.12:0/3961125317 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c24de4aae0 0x55c24de4bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.170+0000 7fe85da0d740 4 client.15327 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:23:58.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.170+0000 7fe85da0d740 1 -- 172.21.15.12:0/3961125317 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c24dda6530 con 0x55c24de4aae0 2024-04-22T13:23:58.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.171+0000 7fe853fff640 1 --2- 172.21.15.12:0/3961125317 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c24de4aae0 0x55c24de4bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.171+0000 7fe833fff640 10 client.15327 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:23:58.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.171+0000 7fe853fff640 1 --2- 172.21.15.12:0/3961125317 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c24de4aae0 0x55c24de4bf50 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a6a044a42d86e51 server_cookie=47c934a4361a73a3 in_seq=0 out_seq=0 2024-04-22T13:23:58.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.172+0000 7fe833fff640 1 -- 172.21.15.12:0/3961125317 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c24dda6530 con 0x55c24de4aae0 2024-04-22T13:23:58.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.172+0000 7fe833fff640 10 client.15327 handle_command_reply: tid=0 2024-04-22T13:23:58.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.172+0000 7fe85da0d740 1 client.15327 shutdown 2024-04-22T13:23:58.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.172+0000 7fe85da0d740 1 -- 172.21.15.12:0/3961125317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c24de352d0 msgr2=0x55c24de45450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:58.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.172+0000 7fe85da0d740 1 --2- 172.21.15.12:0/3961125317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c24de352d0 0x55c24de45450 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe84c018570 tx=0x7fe84c008f60 comp rx=0 tx=0).stop 2024-04-22T13:23:58.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.172+0000 7fe85da0d740 1 -- 172.21.15.12:0/3961125317 shutdown_connections 2024-04-22T13:23:58.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.172+0000 7fe85da0d740 1 --2- 172.21.15.12:0/3961125317 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c24de4aae0 0x55c24de4bf50 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.172+0000 7fe85da0d740 1 --2- 172.21.15.12:0/3961125317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c24de352d0 0x55c24de45450 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.172+0000 7fe833fff640 0 client.15327 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:23:58.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.172+0000 7fe85da0d740 1 --2- 172.21.15.12:0/3961125317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c24de34830 0x55c24de3e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.172+0000 7fe85da0d740 1 --2- 172.21.15.12:0/3961125317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c24de33e40 0x55c24de3df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.172+0000 7fe85da0d740 1 -- 172.21.15.12:0/3961125317 >> 172.21.15.12:0/3961125317 conn(0x55c24de2ff40 msgr2=0x55c24de2da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:58.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.172+0000 7fe85da0d740 1 -- 172.21.15.12:0/3961125317 shutdown_connections 2024-04-22T13:23:58.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.172+0000 7fe85da0d740 1 -- 172.21.15.12:0/3961125317 wait complete. 2024-04-22T13:23:58.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.172+0000 7fe85da0d740 20 client.15327 trim_cache size 0 max 16384 2024-04-22T13:23:58.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:57 smithi134 ceph-mon[31338]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-22T13:23:58.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:57 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1994035554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:23:58.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:57 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/3979169932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:23:58.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:57 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/484665437' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:23:58.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.190+0000 7fe85da0d740 1 Processor -- start 2024-04-22T13:23:58.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.190+0000 7fe85da0d740 1 -- start start 2024-04-22T13:23:58.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.190+0000 7fe85da0d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c24de33e40 0x55c24de2d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.190+0000 7fe85da0d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c24de34830 0x55c24de2d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.190+0000 7fe85da0d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c24de352d0 0x55c24de2dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.190+0000 7fe85da0d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c24de4a290 con 0x55c24de33e40 2024-04-22T13:23:58.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.190+0000 7fe85da0d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c24de4a960 con 0x55c24de34830 2024-04-22T13:23:58.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.190+0000 7fe85da0d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c24de4b030 con 0x55c24de352d0 2024-04-22T13:23:58.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.191+0000 7fe8593bc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c24de33e40 0x55c24de2d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.191+0000 7fe8593bc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c24de33e40 0x55c24de2d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60090/0 (socket says 172.21.15.12:60090) 2024-04-22T13:23:58.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.191+0000 7fe8593bc640 1 -- 172.21.15.12:0/680758710 learned_addr learned my addr 172.21.15.12:0/680758710 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:58.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.191+0000 7fe8593bc640 1 -- 172.21.15.12:0/680758710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c24de352d0 msgr2=0x55c24de2dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:58.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.191+0000 7fe8593bc640 1 --2- 172.21.15.12:0/680758710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c24de352d0 0x55c24de2dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.191+0000 7fe8593bc640 1 -- 172.21.15.12:0/680758710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c24de34830 msgr2=0x55c24de2d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:58.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.191+0000 7fe8593bc640 1 --2- 172.21.15.12:0/680758710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c24de34830 0x55c24de2d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.191+0000 7fe8593bc640 1 -- 172.21.15.12:0/680758710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c24de2fd40 con 0x55c24de33e40 2024-04-22T13:23:58.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.191+0000 7fe8593bc640 1 --2- 172.21.15.12:0/680758710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c24de33e40 0x55c24de2d170 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe84c004ad0 tx=0x7fe84c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4849a72d95d7ec66 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:58.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.191+0000 7fe83e7fc640 1 -- 172.21.15.12:0/680758710 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c00dea0 con 0x55c24de33e40 2024-04-22T13:23:58.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.191+0000 7fe83e7fc640 1 -- 172.21.15.12:0/680758710 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe84c00f840 con 0x55c24de33e40 2024-04-22T13:23:58.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.191+0000 7fe83e7fc640 1 -- 172.21.15.12:0/680758710 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c027e20 con 0x55c24de33e40 2024-04-22T13:23:58.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.192+0000 7fe85da0d740 1 -- 172.21.15.12:0/680758710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c24de33e40 msgr2=0x55c24de2d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:58.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.192+0000 7fe85da0d740 1 --2- 172.21.15.12:0/680758710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c24de33e40 0x55c24de2d170 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe84c004ad0 tx=0x7fe84c005bd0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.192+0000 7fe85da0d740 1 -- 172.21.15.12:0/680758710 shutdown_connections 2024-04-22T13:23:58.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.192+0000 7fe85da0d740 1 --2- 172.21.15.12:0/680758710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c24de352d0 0x55c24de2dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.192+0000 7fe85da0d740 1 --2- 172.21.15.12:0/680758710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c24de34830 0x55c24de2d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.192+0000 7fe85da0d740 1 --2- 172.21.15.12:0/680758710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c24de33e40 0x55c24de2d170 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.192+0000 7fe85da0d740 1 -- 172.21.15.12:0/680758710 >> 172.21.15.12:0/680758710 conn(0x55c24de2ff40 msgr2=0x55c24de388e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:58.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.192+0000 7fe85da0d740 1 -- 172.21.15.12:0/680758710 shutdown_connections 2024-04-22T13:23:58.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.192+0000 7fe85da0d740 1 -- 172.21.15.12:0/680758710 wait complete. 2024-04-22T13:23:58.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.192+0000 7fe85da0d740 1 -- start start 2024-04-22T13:23:58.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.192+0000 7fe85da0d740 1 Processor -- start 2024-04-22T13:23:58.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.192+0000 7fe85da0d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c24de33e40 0x55c24de405f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.192+0000 7fe85da0d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c24de34830 0x55c24de40b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.192+0000 7fe85da0d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c24de352d0 0x55c24de43b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.192+0000 7fe85da0d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c24de4bdd0 con 0x55c24de352d0 2024-04-22T13:23:58.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.192+0000 7fe85da0d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c24de4b700 con 0x55c24de33e40 2024-04-22T13:23:58.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.192+0000 7fe85da0d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c24de4c4a0 con 0x55c24de34830 2024-04-22T13:23:58.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.192+0000 7fe85da0d740 10 client.0 _ll_register_callbacks cb 0x7fe84c008bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:23:58.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.192+0000 7fe8593bc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c24de352d0 0x55c24de43b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.193+0000 7fe8593bc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c24de352d0 0x55c24de43b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60100/0 (socket says 172.21.15.12:60100) 2024-04-22T13:23:58.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.193+0000 7fe8593bc640 1 -- 172.21.15.12:0/2025720653 learned_addr learned my addr 172.21.15.12:0/2025720653 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:58.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.193+0000 7fe853fff640 1 --2- 172.21.15.12:0/2025720653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c24de34830 0x55c24de40b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.193+0000 7fe8593bc640 1 -- 172.21.15.12:0/2025720653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c24de34830 msgr2=0x55c24de40b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:58.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.193+0000 7fe8593bc640 1 --2- 172.21.15.12:0/2025720653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c24de34830 0x55c24de40b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.193+0000 7fe8593bc640 1 -- 172.21.15.12:0/2025720653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c24de33e40 msgr2=0x55c24de405f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:23:58.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.193+0000 7fe8593bc640 1 --2- 172.21.15.12:0/2025720653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c24de33e40 0x55c24de405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.193+0000 7fe8593bc640 1 -- 172.21.15.12:0/2025720653 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c24de44140 con 0x55c24de352d0 2024-04-22T13:23:58.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.193+0000 7fe8593bc640 1 --2- 172.21.15.12:0/2025720653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c24de352d0 0x55c24de43b50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe84c00fa10 tx=0x7fe84c018f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eda0babff02d234 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:58.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.193+0000 7fe833fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:23:58.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.193+0000 7fe833fff640 1 -- 172.21.15.12:0/2025720653 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c018860 con 0x55c24de352d0 2024-04-22T13:23:58.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.193+0000 7fe833fff640 1 -- 172.21.15.12:0/2025720653 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fe84c008830 con 0x55c24de352d0 2024-04-22T13:23:58.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.193+0000 7fe85da0d740 1 -- 172.21.15.12:0/2025720653 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe84c00f840 con 0x55c24de352d0 2024-04-22T13:23:58.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.193+0000 7fe833fff640 1 -- 172.21.15.12:0/2025720653 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe84c027430 con 0x55c24de352d0 2024-04-22T13:23:58.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.193+0000 7fe85da0d740 10 client.15333 fetch_fsmap learned FSMap version 27 2024-04-22T13:23:58.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.193+0000 7fe85da0d740 1 -- 172.21.15.12:0/2025720653 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c24deb1020 con 0x55c24de352d0 2024-04-22T13:23:58.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.193+0000 7fe833fff640 1 -- 172.21.15.12:0/2025720653 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe84c00f840 con 0x55c24de352d0 2024-04-22T13:23:58.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.194+0000 7fe833fff640 1 -- 172.21.15.12:0/2025720653 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe84c023040 con 0x55c24de352d0 2024-04-22T13:23:58.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.194+0000 7fe85da0d740 10 client.15333 fetch_fsmap finished waiting for FSMap version 27 2024-04-22T13:23:58.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.194+0000 7fe85da0d740 10 client.15333 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-22T13:23:58.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.194+0000 7fe85da0d740 20 client.15333 populate_metadata read hostname 'smithi012' 2024-04-22T13:23:58.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.194+0000 7fe85da0d740 1 --2- 172.21.15.12:0/2025720653 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c24deb5820 0x55c24deb2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.194+0000 7fe85da0d740 4 client.15333 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T13:23:58.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.194+0000 7fe85da0d740 1 -- 172.21.15.12:0/2025720653 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "status"}) -- 0x55c24dda03a0 con 0x55c24deb5820 2024-04-22T13:23:58.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.194+0000 7fe853fff640 1 --2- 172.21.15.12:0/2025720653 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c24deb5820 0x55c24deb2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.194+0000 7fe853fff640 1 --2- 172.21.15.12:0/2025720653 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c24deb5820 0x55c24deb2330 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d19bc21d4b79065 server_cookie=61d512c95035f5b2 in_seq=0 out_seq=0 2024-04-22T13:23:58.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.194+0000 7fe833fff640 10 client.15333 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:23:58.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.195+0000 7fe833fff640 1 -- 172.21.15.12:0/2025720653 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c24dda03a0 con 0x55c24deb5820 2024-04-22T13:23:58.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.195+0000 7fe833fff640 10 client.15333 handle_command_reply: tid=0 2024-04-22T13:23:58.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.195+0000 7fe85da0d740 1 client.15333 shutdown 2024-04-22T13:23:58.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.195+0000 7fe85da0d740 1 -- 172.21.15.12:0/2025720653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c24de352d0 msgr2=0x55c24de43b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:58.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.195+0000 7fe85da0d740 1 --2- 172.21.15.12:0/2025720653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c24de352d0 0x55c24de43b50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe84c00fa10 tx=0x7fe84c018f90 comp rx=0 tx=0).stop 2024-04-22T13:23:58.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.195+0000 7fe85da0d740 1 -- 172.21.15.12:0/2025720653 shutdown_connections 2024-04-22T13:23:58.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.195+0000 7fe85da0d740 1 --2- 172.21.15.12:0/2025720653 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c24deb5820 0x55c24deb2330 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.195+0000 7fe85da0d740 1 --2- 172.21.15.12:0/2025720653 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c24de352d0 0x55c24de43b50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.195+0000 7fe85da0d740 1 --2- 172.21.15.12:0/2025720653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c24de34830 0x55c24de40b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.195+0000 7fe833fff640 0 client.15333 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:23:58.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.195+0000 7fe85da0d740 1 --2- 172.21.15.12:0/2025720653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c24de33e40 0x55c24de405f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.195+0000 7fe85da0d740 1 -- 172.21.15.12:0/2025720653 >> 172.21.15.12:0/2025720653 conn(0x55c24de2ff40 msgr2=0x55c24de334d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:58.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.195+0000 7fe85da0d740 1 -- 172.21.15.12:0/2025720653 shutdown_connections 2024-04-22T13:23:58.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.195+0000 7fe85da0d740 1 -- 172.21.15.12:0/2025720653 wait complete. 2024-04-22T13:23:58.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.195+0000 7fe85da0d740 20 client.15333 trim_cache size 0 max 16384 2024-04-22T13:23:58.198 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:23:58.198 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "0aede5ea-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:23:58.198 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 0, 2024-04-22T13:23:58.198 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24509, 2024-04-22T13:23:58.198 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-04-22T13:23:58.198 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-04-22T13:23:58.198 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-04-22T13:23:58.198 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 112.82755752200001, 2024-04-22T13:23:58.198 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 27, 2024-04-22T13:23:58.198 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 89, 2024-04-22T13:23:58.198 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 87, 2024-04-22T13:23:58.198 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 118.474974707 2024-04-22T13:23:58.199 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:23:58.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.195+0000 7fe85ae1f640 1 -- 172.21.15.12:0/3686111722 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe8440a3bb0 msgr2=0x7fe8440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:58.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.195+0000 7fe85ae1f640 1 --2- 172.21.15.12:0/3686111722 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe8440a3bb0 0x7fe8440a6070 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe8480ac290 tx=0x7fe8480f6040 comp rx=0 tx=0).stop 2024-04-22T13:23:58.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.195+0000 7fe85ae1f640 1 -- 172.21.15.12:0/3686111722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe854102a80 msgr2=0x7fe854198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:58.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.195+0000 7fe85ae1f640 1 --2- 172.21.15.12:0/3686111722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe854102a80 0x7fe854198260 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe848006010 tx=0x7fe8480040c0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.196+0000 7fe85ae1f640 1 -- 172.21.15.12:0/3686111722 shutdown_connections 2024-04-22T13:23:58.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.196+0000 7fe85ae1f640 1 --2- 172.21.15.12:0/3686111722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe854107ce0 0x7fe85419f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.196+0000 7fe85ae1f640 1 --2- 172.21.15.12:0/3686111722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8541053b0 0x7fe8541987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.196+0000 7fe85ae1f640 1 --2- 172.21.15.12:0/3686111722 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fe8440a3bb0 0x7fe8440a6070 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.196+0000 7fe85ae1f640 1 --2- 172.21.15.12:0/3686111722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe854102a80 0x7fe854198260 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.196+0000 7fe85ae1f640 1 -- 172.21.15.12:0/3686111722 >> 172.21.15.12:0/3686111722 conn(0x7fe8540fc410 msgr2=0x7fe8541048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:58.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.196+0000 7fe85ae1f640 1 -- 172.21.15.12:0/3686111722 shutdown_connections 2024-04-22T13:23:58.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.196+0000 7fe85ae1f640 1 -- 172.21.15.12:0/3686111722 wait complete. 2024-04-22T13:23:58.204 INFO:teuthology.run_tasks:Running task sequential... 2024-04-22T13:23:58.213 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-22T13:23:58.213 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-22T13:23:58.214 INFO:teuthology.task.print:Enabling mgr modules 2024-04-22T13:23:58.214 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-22T13:23:58.214 INFO:teuthology.task.exec:Executing custom commands... 2024-04-22T13:23:58.214 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-04-22T13:23:58.215 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-22T13:23:58.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.318+0000 7fa85d0fd640 1 -- 172.21.15.12:0/999168901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa858101ba0 msgr2=0x7fa858102020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:58.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.318+0000 7fa85d0fd640 1 --2- 172.21.15.12:0/999168901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa858101ba0 0x7fa858102020 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa850009ca0 tx=0x7fa85002c180 comp rx=0 tx=0).stop 2024-04-22T13:23:58.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.318+0000 7fa85d0fd640 1 -- 172.21.15.12:0/999168901 shutdown_connections 2024-04-22T13:23:58.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.318+0000 7fa85d0fd640 1 --2- 172.21.15.12:0/999168901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa858101ba0 0x7fa858102020 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.318+0000 7fa85d0fd640 1 --2- 172.21.15.12:0/999168901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa858101200 0x7fa858101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.318+0000 7fa85d0fd640 1 --2- 172.21.15.12:0/999168901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8581008a0 0x7fa858100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.318+0000 7fa85d0fd640 1 -- 172.21.15.12:0/999168901 >> 172.21.15.12:0/999168901 conn(0x7fa8580fc420 msgr2=0x7fa8580fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:58.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.318+0000 7fa85d0fd640 1 -- 172.21.15.12:0/999168901 shutdown_connections 2024-04-22T13:23:58.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.318+0000 7fa85d0fd640 1 -- 172.21.15.12:0/999168901 wait complete. 2024-04-22T13:23:58.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.319+0000 7fa85d0fd640 1 Processor -- start 2024-04-22T13:23:58.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.319+0000 7fa85d0fd640 1 -- start start 2024-04-22T13:23:58.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.319+0000 7fa85d0fd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8581008a0 0x7fa85819a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.319+0000 7fa85d0fd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa858101200 0x7fa85819a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.319+0000 7fa85d0fd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa858101ba0 0x7fa8581a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.319+0000 7fa85d0fd640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa85810fd30 con 0x7fa858101ba0 2024-04-22T13:23:58.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.319+0000 7fa85d0fd640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa85810fbb0 con 0x7fa858101200 2024-04-22T13:23:58.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.319+0000 7fa85d0fd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa85810feb0 con 0x7fa8581008a0 2024-04-22T13:23:58.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.319+0000 7fa85759e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa858101ba0 0x7fa8581a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.319+0000 7fa85759e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa858101ba0 0x7fa8581a1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60118/0 (socket says 172.21.15.12:60118) 2024-04-22T13:23:58.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.319+0000 7fa85759e640 1 -- 172.21.15.12:0/2367417421 learned_addr learned my addr 172.21.15.12:0/2367417421 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:58.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.320+0000 7fa856d9d640 1 --2- 172.21.15.12:0/2367417421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8581008a0 0x7fa85819a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.320+0000 7fa85759e640 1 -- 172.21.15.12:0/2367417421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8581008a0 msgr2=0x7fa85819a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:58.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.320+0000 7fa85759e640 1 --2- 172.21.15.12:0/2367417421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8581008a0 0x7fa85819a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.320+0000 7fa85759e640 1 -- 172.21.15.12:0/2367417421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa858101200 msgr2=0x7fa85819a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:58.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.320+0000 7fa85659c640 1 --2- 172.21.15.12:0/2367417421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa858101200 0x7fa85819a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.320+0000 7fa85759e640 1 --2- 172.21.15.12:0/2367417421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa858101200 0x7fa85819a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.320+0000 7fa85759e640 1 -- 172.21.15.12:0/2367417421 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8581a1f90 con 0x7fa858101ba0 2024-04-22T13:23:58.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.320+0000 7fa856d9d640 1 --2- 172.21.15.12:0/2367417421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8581008a0 0x7fa85819a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:23:58.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.320+0000 7fa85759e640 1 --2- 172.21.15.12:0/2367417421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa858101ba0 0x7fa8581a1920 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa850005cc0 tx=0x7fa850003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:58.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.320+0000 7fa83ffff640 1 -- 172.21.15.12:0/2367417421 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850039040 con 0x7fa858101ba0 2024-04-22T13:23:58.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.320+0000 7fa85d0fd640 1 -- 172.21.15.12:0/2367417421 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8581a2220 con 0x7fa858101ba0 2024-04-22T13:23:58.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.320+0000 7fa83ffff640 1 -- 172.21.15.12:0/2367417421 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa850044070 con 0x7fa858101ba0 2024-04-22T13:23:58.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.320+0000 7fa83ffff640 1 -- 172.21.15.12:0/2367417421 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850032970 con 0x7fa858101ba0 2024-04-22T13:23:58.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.320+0000 7fa85d0fd640 1 -- 172.21.15.12:0/2367417421 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8581a2790 con 0x7fa858101ba0 2024-04-22T13:23:58.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.321+0000 7fa85d0fd640 1 -- 172.21.15.12:0/2367417421 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa85810b5f0 con 0x7fa858101ba0 2024-04-22T13:23:58.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.322+0000 7fa83ffff640 1 -- 172.21.15.12:0/2367417421 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa850032b30 con 0x7fa858101ba0 2024-04-22T13:23:58.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.323+0000 7fa83ffff640 1 --2- 172.21.15.12:0/2367417421 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa8400a3c50 0x7fa8400a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.323+0000 7fa83ffff640 1 -- 172.21.15.12:0/2367417421 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fa8500f2b40 con 0x7fa858101ba0 2024-04-22T13:23:58.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.323+0000 7fa856d9d640 1 --2- 172.21.15.12:0/2367417421 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa8400a3c50 0x7fa8400a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.323+0000 7fa856d9d640 1 --2- 172.21.15.12:0/2367417421 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa8400a3c50 0x7fa8400a6110 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa848006fd0 tx=0x7fa848008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:58.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.328+0000 7fa83ffff640 1 -- 172.21.15.12:0/2367417421 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa8500bf4f0 con 0x7fa858101ba0 2024-04-22T13:23:58.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.567+0000 7fa85d0fd640 1 -- 172.21.15.12:0/2367417421 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fa858062b50 con 0x7fa858101ba0 2024-04-22T13:23:58.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.734+0000 7fa83ffff640 1 -- 172.21.15.12:0/2367417421 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fa8500bedf0 con 0x7fa858101ba0 2024-04-22T13:23:58.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.736+0000 7fa85d0fd640 1 -- 172.21.15.12:0/2367417421 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa8400a3c50 msgr2=0x7fa8400a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:58.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.736+0000 7fa85d0fd640 1 --2- 172.21.15.12:0/2367417421 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa8400a3c50 0x7fa8400a6110 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa848006fd0 tx=0x7fa848008040 comp rx=0 tx=0).stop 2024-04-22T13:23:58.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.737+0000 7fa85d0fd640 1 -- 172.21.15.12:0/2367417421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa858101ba0 msgr2=0x7fa8581a1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:58.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.737+0000 7fa85d0fd640 1 --2- 172.21.15.12:0/2367417421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa858101ba0 0x7fa8581a1920 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa850005cc0 tx=0x7fa850003c20 comp rx=0 tx=0).stop 2024-04-22T13:23:58.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.737+0000 7fa85d0fd640 1 -- 172.21.15.12:0/2367417421 shutdown_connections 2024-04-22T13:23:58.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.737+0000 7fa85d0fd640 1 --2- 172.21.15.12:0/2367417421 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7fa8400a3c50 0x7fa8400a6110 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.737+0000 7fa85d0fd640 1 --2- 172.21.15.12:0/2367417421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa858101ba0 0x7fa8581a1920 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.737+0000 7fa85d0fd640 1 --2- 172.21.15.12:0/2367417421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa858101200 0x7fa85819a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.737+0000 7fa85d0fd640 1 --2- 172.21.15.12:0/2367417421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8581008a0 0x7fa85819a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.737+0000 7fa85d0fd640 1 -- 172.21.15.12:0/2367417421 >> 172.21.15.12:0/2367417421 conn(0x7fa8580fc420 msgr2=0x7fa8580fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:58.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.737+0000 7fa85d0fd640 1 -- 172.21.15.12:0/2367417421 shutdown_connections 2024-04-22T13:23:58.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.737+0000 7fa85d0fd640 1 -- 172.21.15.12:0/2367417421 wait complete. 2024-04-22T13:23:58.752 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-22T13:23:58.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.900+0000 7f6351070640 1 -- 172.21.15.12:0/2965853147 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f634c100e60 msgr2=0x7f634c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:58.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.900+0000 7f6351070640 1 --2- 172.21.15.12:0/2965853147 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f634c100e60 0x7f634c101280 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6334009900 tx=0x7f633402bc30 comp rx=0 tx=0).stop 2024-04-22T13:23:58.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.900+0000 7f6351070640 1 -- 172.21.15.12:0/2965853147 shutdown_connections 2024-04-22T13:23:58.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.900+0000 7f6351070640 1 --2- 172.21.15.12:0/2965853147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f634c102a00 0x7f634c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.900+0000 7f6351070640 1 --2- 172.21.15.12:0/2965853147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f634c102040 0x7f634c1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.900+0000 7f6351070640 1 --2- 172.21.15.12:0/2965853147 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f634c100e60 0x7f634c101280 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.900+0000 7f6351070640 1 -- 172.21.15.12:0/2965853147 >> 172.21.15.12:0/2965853147 conn(0x7f634c0fc420 msgr2=0x7f634c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:58.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.900+0000 7f6351070640 1 -- 172.21.15.12:0/2965853147 shutdown_connections 2024-04-22T13:23:58.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.900+0000 7f6351070640 1 -- 172.21.15.12:0/2965853147 wait complete. 2024-04-22T13:23:58.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.901+0000 7f6351070640 1 Processor -- start 2024-04-22T13:23:58.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.901+0000 7f6351070640 1 -- start start 2024-04-22T13:23:58.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.902+0000 7f6351070640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f634c100e60 0x7f634c06a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.902+0000 7f6351070640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f634c102040 0x7f634c19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.902+0000 7f6351070640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f634c102a00 0x7f634c19f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.902+0000 7f6351070640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f634c10c9d0 con 0x7f634c100e60 2024-04-22T13:23:58.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.902+0000 7f6351070640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f634c10c850 con 0x7f634c102040 2024-04-22T13:23:58.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.902+0000 7f6351070640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f634c10cb50 con 0x7f634c102a00 2024-04-22T13:23:58.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.902+0000 7f634ad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f634c100e60 0x7f634c06a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.902+0000 7f634ad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f634c100e60 0x7f634c06a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60144/0 (socket says 172.21.15.12:60144) 2024-04-22T13:23:58.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.902+0000 7f634ad9d640 1 -- 172.21.15.12:0/2600339848 learned_addr learned my addr 172.21.15.12:0/2600339848 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:58.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.902+0000 7f634b59e640 1 --2- 172.21.15.12:0/2600339848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f634c102a00 0x7f634c19f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.902+0000 7f634a59c640 1 --2- 172.21.15.12:0/2600339848 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f634c102040 0x7f634c19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:58.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.902+0000 7f634ad9d640 1 -- 172.21.15.12:0/2600339848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f634c102a00 msgr2=0x7f634c19f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:58.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.902+0000 7f634ad9d640 1 --2- 172.21.15.12:0/2600339848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f634c102a00 0x7f634c19f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.902+0000 7f634ad9d640 1 -- 172.21.15.12:0/2600339848 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f634c102040 msgr2=0x7f634c19ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:58.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.902+0000 7f634ad9d640 1 --2- 172.21.15.12:0/2600339848 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f634c102040 0x7f634c19ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:58.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.902+0000 7f634ad9d640 1 -- 172.21.15.12:0/2600339848 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634c19fd70 con 0x7f634c100e60 2024-04-22T13:23:58.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.902+0000 7f634ad9d640 1 --2- 172.21.15.12:0/2600339848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f634c100e60 0x7f634c06a1d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6334000c00 tx=0x7f6334004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:58.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.902+0000 7f6343fff640 1 -- 172.21.15.12:0/2600339848 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334043020 con 0x7f634c100e60 2024-04-22T13:23:58.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.902+0000 7f6351070640 1 -- 172.21.15.12:0/2600339848 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f634c1a0000 con 0x7f634c100e60 2024-04-22T13:23:58.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.902+0000 7f6343fff640 1 -- 172.21.15.12:0/2600339848 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6334004440 con 0x7f634c100e60 2024-04-22T13:23:58.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.902+0000 7f6343fff640 1 -- 172.21.15.12:0/2600339848 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334003c90 con 0x7f634c100e60 2024-04-22T13:23:58.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.903+0000 7f6351070640 1 -- 172.21.15.12:0/2600339848 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f634c1a0550 con 0x7f634c100e60 2024-04-22T13:23:58.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.904+0000 7f6351070640 1 -- 172.21.15.12:0/2600339848 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f634c108200 con 0x7f634c100e60 2024-04-22T13:23:58.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.905+0000 7f6343fff640 1 -- 172.21.15.12:0/2600339848 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f6334046020 con 0x7f634c100e60 2024-04-22T13:23:58.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.905+0000 7f6343fff640 1 --2- 172.21.15.12:0/2600339848 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f63380a3ca0 0x7f63380a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:58.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.906+0000 7f634a59c640 1 -- 172.21.15.12:0/2600339848 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f63380a3ca0 msgr2=0x7f63380a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1802873236 2024-04-22T13:23:58.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.906+0000 7f6343fff640 1 -- 172.21.15.12:0/2600339848 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f63340f25a0 con 0x7f634c100e60 2024-04-22T13:23:58.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.906+0000 7f634a59c640 1 --2- 172.21.15.12:0/2600339848 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f63380a3ca0 0x7f63380a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T13:23:58.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:58.910+0000 7f6343fff640 1 -- 172.21.15.12:0/2600339848 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f633404d020 con 0x7f634c100e60 2024-04-22T13:23:58.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:58 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2367417421' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-22T13:23:58.977 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:23:58 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: ignoring --setuser ceph since I am not root 2024-04-22T13:23:58.977 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:23:58 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: ignoring --setgroup ceph since I am not root 2024-04-22T13:23:58.977 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:23:58 smithi111 ceph-mgr[32755]: -- 172.21.15.111:0/2745501399 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f328bd05a0 con 0x55f328ba5000 2024-04-22T13:23:58.977 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:23:58 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:23:58.916+0000 7fc2f2023200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T13:23:59.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:58 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2367417421' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-22T13:23:59.087 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:23:58 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: ignoring --setuser ceph since I am not root 2024-04-22T13:23:59.087 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:23:58 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: ignoring --setgroup ceph since I am not root 2024-04-22T13:23:59.087 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:23:58 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:23:58.970+0000 7fdcd00c0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T13:23:59.095 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:58 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2367417421' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-22T13:23:59.095 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:23:58 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: ignoring --setuser ceph since I am not root 2024-04-22T13:23:59.095 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:23:58 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: ignoring --setgroup ceph since I am not root 2024-04-22T13:23:59.095 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:23:58 smithi134 ceph-mgr[33207]: -- 172.21.15.134:0/1587096851 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5618d89925a0 con 0x5618d8967000 2024-04-22T13:23:59.096 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:23:58 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:23:58.915+0000 7f9135fea200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T13:23:59.096 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:23:59 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:23:59.098+0000 7f9135fea200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:23:59.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.107+0000 7f634a59c640 1 -- 172.21.15.12:0/2600339848 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f63380a3ca0 msgr2=0x7f63380a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1802873236 2024-04-22T13:23:59.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.107+0000 7f634a59c640 1 --2- 172.21.15.12:0/2600339848 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f63380a3ca0 0x7f63380a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T13:23:59.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.109+0000 7f6351070640 1 -- 172.21.15.12:0/2600339848 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f634c062b50 con 0x7f634c100e60 2024-04-22T13:23:59.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.113+0000 7f6343fff640 1 -- 172.21.15.12:0/2600339848 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7f63340befb0 con 0x7f634c100e60 2024-04-22T13:23:59.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.115+0000 7f6351070640 1 -- 172.21.15.12:0/2600339848 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f63380a3ca0 msgr2=0x7f63380a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T13:23:59.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.115+0000 7f6351070640 1 --2- 172.21.15.12:0/2600339848 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f63380a3ca0 0x7f63380a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:59.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.116+0000 7f6351070640 1 -- 172.21.15.12:0/2600339848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f634c100e60 msgr2=0x7f634c06a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:59.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.116+0000 7f6351070640 1 --2- 172.21.15.12:0/2600339848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f634c100e60 0x7f634c06a1d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6334000c00 tx=0x7f6334004040 comp rx=0 tx=0).stop 2024-04-22T13:23:59.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.116+0000 7f6351070640 1 -- 172.21.15.12:0/2600339848 shutdown_connections 2024-04-22T13:23:59.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.116+0000 7f6351070640 1 --2- 172.21.15.12:0/2600339848 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f634c102a00 0x7f634c19f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:59.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.116+0000 7f6351070640 1 --2- 172.21.15.12:0/2600339848 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f634c102040 0x7f634c19ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:59.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.116+0000 7f6351070640 1 --2- 172.21.15.12:0/2600339848 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f63380a3ca0 0x7f63380a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:59.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.116+0000 7f6351070640 1 --2- 172.21.15.12:0/2600339848 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f634c100e60 0x7f634c06a1d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:59.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.116+0000 7f6351070640 1 -- 172.21.15.12:0/2600339848 >> 172.21.15.12:0/2600339848 conn(0x7f634c0fc420 msgr2=0x7f634c1053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:59.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.116+0000 7f6351070640 1 -- 172.21.15.12:0/2600339848 shutdown_connections 2024-04-22T13:23:59.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.116+0000 7f6351070640 1 -- 172.21.15.12:0/2600339848 wait complete. 2024-04-22T13:23:59.125 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-22T13:23:59.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.232+0000 7f1f5c596640 1 -- 172.21.15.12:0/2285824164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f54101fc0 msgr2=0x7f1f541023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:59.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.232+0000 7f1f5c596640 1 --2- 172.21.15.12:0/2285824164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f54101fc0 0x7f1f541023e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1f50009900 tx=0x7f1f5002bc30 comp rx=0 tx=0).stop 2024-04-22T13:23:59.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.233+0000 7f1f5c596640 1 -- 172.21.15.12:0/2285824164 shutdown_connections 2024-04-22T13:23:59.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.233+0000 7f1f5c596640 1 --2- 172.21.15.12:0/2285824164 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f54103b70 0x7f1f5410a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:59.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.233+0000 7f1f5c596640 1 --2- 172.21.15.12:0/2285824164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f541031b0 0x7f1f54103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:59.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.233+0000 7f1f5c596640 1 --2- 172.21.15.12:0/2285824164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f54101fc0 0x7f1f541023e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:59.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.233+0000 7f1f5c596640 1 -- 172.21.15.12:0/2285824164 >> 172.21.15.12:0/2285824164 conn(0x7f1f540fd530 msgr2=0x7f1f540ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:59.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.233+0000 7f1f5c596640 1 -- 172.21.15.12:0/2285824164 shutdown_connections 2024-04-22T13:23:59.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.233+0000 7f1f5c596640 1 -- 172.21.15.12:0/2285824164 wait complete. 2024-04-22T13:23:59.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.233+0000 7f1f5c596640 1 Processor -- start 2024-04-22T13:23:59.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.233+0000 7f1f5c596640 1 -- start start 2024-04-22T13:23:59.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.234+0000 7f1f5c596640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f54101fc0 0x7f1f54198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:59.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.234+0000 7f1f5c596640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f541031b0 0x7f1f541986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:59.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.234+0000 7f1f5c596640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f54103b70 0x7f1f5419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:59.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.234+0000 7f1f5c596640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5410dac0 con 0x7f1f54103b70 2024-04-22T13:23:59.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.234+0000 7f1f5c596640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5410d940 con 0x7f1f54101fc0 2024-04-22T13:23:59.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.234+0000 7f1f5c596640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5410dc40 con 0x7f1f541031b0 2024-04-22T13:23:59.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.234+0000 7f1f5ab33640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f54103b70 0x7f1f5419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:59.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.234+0000 7f1f5ab33640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f54103b70 0x7f1f5419f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60154/0 (socket says 172.21.15.12:60154) 2024-04-22T13:23:59.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.234+0000 7f1f5ab33640 1 -- 172.21.15.12:0/1863280959 learned_addr learned my addr 172.21.15.12:0/1863280959 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:23:59.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.234+0000 7f1f5a332640 1 --2- 172.21.15.12:0/1863280959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f54101fc0 0x7f1f54198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:59.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.234+0000 7f1f59b31640 1 --2- 172.21.15.12:0/1863280959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f541031b0 0x7f1f541986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:23:59.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.234+0000 7f1f5ab33640 1 -- 172.21.15.12:0/1863280959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f541031b0 msgr2=0x7f1f541986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:59.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.234+0000 7f1f5ab33640 1 --2- 172.21.15.12:0/1863280959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f541031b0 0x7f1f541986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:59.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.234+0000 7f1f5ab33640 1 -- 172.21.15.12:0/1863280959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f54101fc0 msgr2=0x7f1f54198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:59.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.234+0000 7f1f5ab33640 1 --2- 172.21.15.12:0/1863280959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f54101fc0 0x7f1f54198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:59.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.234+0000 7f1f5ab33640 1 -- 172.21.15.12:0/1863280959 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f5419fd90 con 0x7f1f54103b70 2024-04-22T13:23:59.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.234+0000 7f1f5a332640 1 --2- 172.21.15.12:0/1863280959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f54101fc0 0x7f1f54198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:23:59.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.234+0000 7f1f5ab33640 1 --2- 172.21.15.12:0/1863280959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f54103b70 0x7f1f5419f720 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00eaa0 tx=0x7f1f4c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:23:59.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.234+0000 7f1f4b7fe640 1 -- 172.21.15.12:0/1863280959 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c00cdb0 con 0x7f1f54103b70 2024-04-22T13:23:59.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.235+0000 7f1f4b7fe640 1 -- 172.21.15.12:0/1863280959 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f4c005020 con 0x7f1f54103b70 2024-04-22T13:23:59.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.235+0000 7f1f5c596640 1 -- 172.21.15.12:0/1863280959 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f541a0080 con 0x7f1f54103b70 2024-04-22T13:23:59.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.235+0000 7f1f4b7fe640 1 -- 172.21.15.12:0/1863280959 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c0116d0 con 0x7f1f54103b70 2024-04-22T13:23:59.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.235+0000 7f1f5c596640 1 -- 172.21.15.12:0/1863280959 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f541a0580 con 0x7f1f54103b70 2024-04-22T13:23:59.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.236+0000 7f1f5c596640 1 -- 172.21.15.12:0/1863280959 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f54109350 con 0x7f1f54103b70 2024-04-22T13:23:59.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.237+0000 7f1f4b7fe640 1 -- 172.21.15.12:0/1863280959 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f1f4c0051d0 con 0x7f1f54103b70 2024-04-22T13:23:59.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.237+0000 7f1f4b7fe640 1 --2- 172.21.15.12:0/1863280959 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1f400a3ca0 0x7f1f400a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:23:59.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.237+0000 7f1f5a332640 1 -- 172.21.15.12:0/1863280959 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1f400a3ca0 msgr2=0x7f1f400a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1802873236 2024-04-22T13:23:59.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.237+0000 7f1f4b7fe640 1 -- 172.21.15.12:0/1863280959 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f1f4c0d0580 con 0x7f1f54103b70 2024-04-22T13:23:59.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.237+0000 7f1f5a332640 1 --2- 172.21.15.12:0/1863280959 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1f400a3ca0 0x7f1f400a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T13:23:59.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.242+0000 7f1f4b7fe640 1 -- 172.21.15.12:0/1863280959 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1f4c016030 con 0x7f1f54103b70 2024-04-22T13:23:59.427 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:23:59 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:23:59.202+0000 7f9135fea200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T13:23:59.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.431+0000 7f1f5c596640 1 -- 172.21.15.12:0/1863280959 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f1f54062b50 con 0x7f1f54103b70 2024-04-22T13:23:59.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.435+0000 7f1f4b7fe640 1 -- 172.21.15.12:0/1863280959 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) ==== 133+0+0 (secure 0 0 0) 0x7f1f4c09c940 con 0x7f1f54103b70 2024-04-22T13:23:59.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.437+0000 7f1f5c596640 1 -- 172.21.15.12:0/1863280959 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1f400a3ca0 msgr2=0x7f1f400a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T13:23:59.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.437+0000 7f1f5c596640 1 --2- 172.21.15.12:0/1863280959 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1f400a3ca0 0x7f1f400a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:59.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.437+0000 7f1f5c596640 1 -- 172.21.15.12:0/1863280959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f54103b70 msgr2=0x7f1f5419f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:23:59.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.437+0000 7f1f5c596640 1 --2- 172.21.15.12:0/1863280959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f54103b70 0x7f1f5419f720 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00eaa0 tx=0x7f1f4c00ef60 comp rx=0 tx=0).stop 2024-04-22T13:23:59.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.438+0000 7f1f5c596640 1 -- 172.21.15.12:0/1863280959 shutdown_connections 2024-04-22T13:23:59.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.438+0000 7f1f5c596640 1 --2- 172.21.15.12:0/1863280959 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f1f400a3ca0 0x7f1f400a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:59.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.438+0000 7f1f5c596640 1 --2- 172.21.15.12:0/1863280959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1f54103b70 0x7f1f5419f720 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:59.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.438+0000 7f1f5c596640 1 --2- 172.21.15.12:0/1863280959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f541031b0 0x7f1f541986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:59.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.438+0000 7f1f5c596640 1 --2- 172.21.15.12:0/1863280959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f54101fc0 0x7f1f54198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:23:59.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.438+0000 7f1f5c596640 1 -- 172.21.15.12:0/1863280959 >> 172.21.15.12:0/1863280959 conn(0x7f1f540fd530 msgr2=0x7f1f54106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:23:59.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.438+0000 7f1f5c596640 1 -- 172.21.15.12:0/1863280959 shutdown_connections 2024-04-22T13:23:59.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:23:59.438+0000 7f1f5c596640 1 -- 172.21.15.12:0/1863280959 wait complete. 2024-04-22T13:23:59.445 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:23:59 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:23:59.165+0000 7fdcd00c0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:23:59.445 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:23:59 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:23:59.277+0000 7fdcd00c0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T13:23:59.446 INFO:teuthology.run_tasks:Running task kclient... 2024-04-22T13:23:59.469 INFO:tasks.kclient:Mounting kernel clients... 2024-04-22T13:23:59.469 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-22T13:23:59.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:23:59.469 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-04-22T13:23:59.476 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:23:59 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:23:59.098+0000 7fc2f2023200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:23:59.476 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:23:59 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:23:59.202+0000 7fc2f2023200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T13:23:59.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:23:59.485 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link delete ceph-brx 2024-04-22T13:23:59.552 INFO:teuthology.orchestra.run.smithi012.stderr:Cannot find device "ceph-brx" 2024-04-22T13:23:59.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T13:23:59.554 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-22T13:23:59.554 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-22T13:23:59.554 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-22T13:23:59.554 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-22T13:23:59.732 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:23:59 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:23:59.603+0000 7fdcd00c0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T13:23:59.732 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:23:59 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:23:59.695+0000 7fdcd00c0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T13:23:59.845 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:23:59.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:59 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2367417421' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-22T13:23:59.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:59 smithi134 ceph-mon[31338]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-22T13:23:59.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:59 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2600339848' entity='client.admin' 2024-04-22T13:23:59.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:23:59 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1863280959' entity='client.admin' 2024-04-22T13:23:59.928 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:23:59 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:23:59.508+0000 7f9135fea200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T13:23:59.928 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:23:59 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:23:59.593+0000 7f9135fea200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T13:23:59.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:59 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2367417421' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-22T13:23:59.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:59 smithi111 ceph-mon[31361]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-22T13:23:59.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:59 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2600339848' entity='client.admin' 2024-04-22T13:23:59.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:23:59 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1863280959' entity='client.admin' 2024-04-22T13:23:59.977 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:23:59 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:23:59.510+0000 7fc2f2023200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T13:23:59.977 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:23:59 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:23:59.595+0000 7fc2f2023200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T13:23:59.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:59 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2367417421' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-22T13:23:59.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:59 smithi012 ceph-mon[28265]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-22T13:23:59.983 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:59 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2600339848' entity='client.admin' 2024-04-22T13:23:59.984 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:23:59 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1863280959' entity='client.admin' 2024-04-22T13:24:01.106 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:24:00 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:24:00.767+0000 7f9135fea200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T13:24:01.106 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:24:00 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:24:00.937+0000 7f9135fea200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T13:24:01.106 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:24:01 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:24:01.108+0000 7f9135fea200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T13:24:01.106 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:24:00 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:24:00.771+0000 7fc2f2023200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T13:24:01.107 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:24:00 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:24:00.940+0000 7fc2f2023200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T13:24:01.107 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:24:01 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:24:01.110+0000 7fc2f2023200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T13:24:01.249 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:00 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:00.969+0000 7fdcd00c0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T13:24:01.250 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:01 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:01.156+0000 7fdcd00c0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T13:24:01.535 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:01 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:01.336+0000 7fdcd00c0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T13:24:01.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.586+0000 7f9cb30a2700 1 -- 172.21.15.12:0/3313584885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9cac101930 msgr2=0x7f9cac101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:01.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.586+0000 7f9cb30a2700 1 --2- 172.21.15.12:0/3313584885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9cac101930 0x7f9cac101db0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0009990 tx=0x7f9ca0009ca0 comp rx=0 tx=0).stop 2024-04-22T13:24:01.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.586+0000 7f9cb30a2700 1 -- 172.21.15.12:0/3313584885 shutdown_connections 2024-04-22T13:24:01.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.586+0000 7f9cb30a2700 1 --2- 172.21.15.12:0/3313584885 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cac107980 0x7f9cac107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:01.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.586+0000 7f9cb30a2700 1 --2- 172.21.15.12:0/3313584885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cac1022f0 0x7f9cac10e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:01.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.586+0000 7f9cb30a2700 1 --2- 172.21.15.12:0/3313584885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9cac101930 0x7f9cac101db0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:01.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.586+0000 7f9cb30a2700 1 -- 172.21.15.12:0/3313584885 >> 172.21.15.12:0/3313584885 conn(0x7f9cac0788b0 msgr2=0x7f9cac078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:01.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.587+0000 7f9cb30a2700 1 -- 172.21.15.12:0/3313584885 shutdown_connections 2024-04-22T13:24:01.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.587+0000 7f9cb30a2700 1 -- 172.21.15.12:0/3313584885 wait complete. 2024-04-22T13:24:01.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.587+0000 7f9cb30a2700 1 Processor -- start 2024-04-22T13:24:01.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.587+0000 7f9cb30a2700 1 -- start start 2024-04-22T13:24:01.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.588+0000 7f9cb30a2700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cac101930 0x7f9cac199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:01.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.588+0000 7f9cb30a2700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cac1022f0 0x7f9cac19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:01.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.588+0000 7f9cb30a2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9cac107980 0x7f9cac1a5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:01.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.588+0000 7f9cb30a2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac111b10 con 0x7f9cac107980 2024-04-22T13:24:01.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.588+0000 7f9cb30a2700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac111990 con 0x7f9cac1022f0 2024-04-22T13:24:01.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.588+0000 7f9cb30a2700 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac111c90 con 0x7f9cac101930 2024-04-22T13:24:01.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.588+0000 7f9cb163f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9cac107980 0x7f9cac1a5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:01.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.588+0000 7f9cb0e3e700 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cac101930 0x7f9cac199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:01.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.588+0000 7f9cb163f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9cac107980 0x7f9cac1a5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60186/0 (socket says 172.21.15.12:60186) 2024-04-22T13:24:01.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.588+0000 7f9cb163f700 1 -- 172.21.15.12:0/728767014 learned_addr learned my addr 172.21.15.12:0/728767014 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:01.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.588+0000 7f9cabfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cac1022f0 0x7f9cac19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:01.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.588+0000 7f9cb163f700 1 -- 172.21.15.12:0/728767014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cac101930 msgr2=0x7f9cac199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:01.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.588+0000 7f9cb163f700 1 --2- 172.21.15.12:0/728767014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cac101930 0x7f9cac199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:01.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.588+0000 7f9cb163f700 1 -- 172.21.15.12:0/728767014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cac1022f0 msgr2=0x7f9cac19a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:01.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.588+0000 7f9cb163f700 1 --2- 172.21.15.12:0/728767014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cac1022f0 0x7f9cac19a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:01.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.588+0000 7f9cb163f700 1 -- 172.21.15.12:0/728767014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cac1a5550 con 0x7f9cac107980 2024-04-22T13:24:01.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.588+0000 7f9cb163f700 1 --2- 172.21.15.12:0/728767014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9cac107980 0x7f9cac1a5010 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4010300 tx=0x7f9ca4010610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:01.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.589+0000 7f9ca9ffb700 1 -- 172.21.15.12:0/728767014 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4010bb0 con 0x7f9cac107980 2024-04-22T13:24:01.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.589+0000 7f9ca9ffb700 1 -- 172.21.15.12:0/728767014 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ca4010d50 con 0x7f9cac107980 2024-04-22T13:24:01.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.589+0000 7f9cb30a2700 1 -- 172.21.15.12:0/728767014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cac197de0 con 0x7f9cac107980 2024-04-22T13:24:01.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.589+0000 7f9ca9ffb700 1 -- 172.21.15.12:0/728767014 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca400ce60 con 0x7f9cac107980 2024-04-22T13:24:01.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.589+0000 7f9cb30a2700 1 -- 172.21.15.12:0/728767014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cac198320 con 0x7f9cac107980 2024-04-22T13:24:01.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.590+0000 7f9cb30a2700 1 -- 172.21.15.12:0/728767014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cac069730 con 0x7f9cac107980 2024-04-22T13:24:01.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.591+0000 7f9ca9ffb700 1 -- 172.21.15.12:0/728767014 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f9ca400c450 con 0x7f9cac107980 2024-04-22T13:24:01.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.592+0000 7f9ca9ffb700 1 --2- 172.21.15.12:0/728767014 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f9c940a3d10 0x7f9c940a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:01.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.592+0000 7f9cb0e3e700 1 -- 172.21.15.12:0/728767014 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f9c940a3d10 msgr2=0x7f9c940a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1802873236 2024-04-22T13:24:01.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.592+0000 7f9ca9ffb700 1 -- 172.21.15.12:0/728767014 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f9ca40d4170 con 0x7f9cac107980 2024-04-22T13:24:01.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.592+0000 7f9cb0e3e700 1 --2- 172.21.15.12:0/728767014 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f9c940a3d10 0x7f9c940a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T13:24:01.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.596+0000 7f9ca9ffb700 1 -- 172.21.15.12:0/728767014 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ca40a0b20 con 0x7f9cac107980 2024-04-22T13:24:01.677 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:24:01 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:24:01.395+0000 7f9135fea200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T13:24:01.677 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:24:01 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:24:01.488+0000 7f9135fea200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T13:24:01.687 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:24:01 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:24:01.396+0000 7fc2f2023200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T13:24:01.688 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:24:01 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:24:01.487+0000 7fc2f2023200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T13:24:01.688 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:24:01 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:24:01.691+0000 7fc2f2023200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T13:24:01.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.793+0000 7f9cb0e3e700 1 -- 172.21.15.12:0/728767014 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f9c940a3d10 msgr2=0x7f9c940a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1802873236 2024-04-22T13:24:01.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.793+0000 7f9cb0e3e700 1 --2- 172.21.15.12:0/728767014 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f9c940a3d10 0x7f9c940a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T13:24:01.790 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:01 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:01.662+0000 7fdcd00c0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T13:24:01.790 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:01 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:01.759+0000 7fdcd00c0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T13:24:01.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.853+0000 7f9cb30a2700 1 -- 172.21.15.12:0/728767014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9cac02df80 con 0x7f9cac107980 2024-04-22T13:24:01.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.854+0000 7f9ca9ffb700 1 -- 172.21.15.12:0/728767014 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f9ca40a0420 con 0x7f9cac107980 2024-04-22T13:24:01.851 INFO:teuthology.orchestra.run.smithi012.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-22T13:24:01.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.856+0000 7f9cb30a2700 1 -- 172.21.15.12:0/728767014 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f9c940a3d10 msgr2=0x7f9c940a61d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T13:24:01.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.856+0000 7f9cb30a2700 1 --2- 172.21.15.12:0/728767014 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f9c940a3d10 0x7f9c940a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:01.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.856+0000 7f9cb30a2700 1 -- 172.21.15.12:0/728767014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9cac107980 msgr2=0x7f9cac1a5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:01.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.856+0000 7f9cb30a2700 1 --2- 172.21.15.12:0/728767014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9cac107980 0x7f9cac1a5010 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4010300 tx=0x7f9ca4010610 comp rx=0 tx=0).stop 2024-04-22T13:24:01.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.856+0000 7f9cb30a2700 1 -- 172.21.15.12:0/728767014 shutdown_connections 2024-04-22T13:24:01.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.856+0000 7f9cb30a2700 1 --2- 172.21.15.12:0/728767014 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f9c940a3d10 0x7f9c940a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:01.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.856+0000 7f9cb30a2700 1 --2- 172.21.15.12:0/728767014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9cac107980 0x7f9cac1a5010 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:01.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.856+0000 7f9cb30a2700 1 --2- 172.21.15.12:0/728767014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9cac101930 0x7f9cac199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:01.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.856+0000 7f9cb30a2700 1 --2- 172.21.15.12:0/728767014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cac1022f0 0x7f9cac19a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:01.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.856+0000 7f9cb30a2700 1 -- 172.21.15.12:0/728767014 >> 172.21.15.12:0/728767014 conn(0x7f9cac0788b0 msgr2=0x7f9cac0ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:01.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.856+0000 7f9cb30a2700 1 -- 172.21.15.12:0/728767014 shutdown_connections 2024-04-22T13:24:01.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:01.856+0000 7f9cb30a2700 1 -- 172.21.15.12:0/728767014 wait complete. 2024-04-22T13:24:01.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:01 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/728767014' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-22T13:24:01.977 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:24:01 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:24:01.884+0000 7fc2f2023200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:24:02.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:01 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/728767014' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-22T13:24:02.087 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:01 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:01.979+0000 7fdcd00c0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T13:24:02.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:01 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/728767014' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-22T13:24:02.177 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:24:01 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:24:01.693+0000 7f9135fea200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T13:24:02.177 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:24:01 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:24:01.885+0000 7f9135fea200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:24:02.556 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:02 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:02.183+0000 7fdcd00c0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T13:24:02.585 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-22T13:24:02.585 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-22T13:24:02.585 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi012.front.sepia.ceph.com 2024-04-22T13:24:02.585 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-22T13:24:02.585 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-22T13:24:02.585 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-22T13:24:02.585 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-22T13:24:02.585 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-22T13:24:02.585 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-22T13:24:02.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:24:02.586 DEBUG:teuthology.orchestra.run.smithi012:> ip addr 2024-04-22T13:24:02.602 INFO:teuthology.orchestra.run.smithi012.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-22T13:24:02.602 INFO:teuthology.orchestra.run.smithi012.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-22T13:24:02.602 INFO:teuthology.orchestra.run.smithi012.stdout: inet 127.0.0.1/8 scope host lo 2024-04-22T13:24:02.602 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-04-22T13:24:02.602 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 ::1/128 scope host 2024-04-22T13:24:02.602 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-04-22T13:24:02.602 INFO:teuthology.orchestra.run.smithi012.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T13:24:02.602 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a4 brd ff:ff:ff:ff:ff:ff 2024-04-22T13:24:02.603 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f0 2024-04-22T13:24:02.603 INFO:teuthology.orchestra.run.smithi012.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T13:24:02.603 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether 0c:c4:7a:6b:fd:a5 brd ff:ff:ff:ff:ff:ff 2024-04-22T13:24:02.603 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp7s0f1 2024-04-22T13:24:02.603 INFO:teuthology.orchestra.run.smithi012.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-22T13:24:02.603 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:ea brd ff:ff:ff:ff:ff:ff 2024-04-22T13:24:02.603 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f0 2024-04-22T13:24:02.603 INFO:teuthology.orchestra.run.smithi012.stdout: inet 172.21.15.12/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-22T13:24:02.603 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft 42028sec preferred_lft 42028sec 2024-04-22T13:24:02.603 INFO:teuthology.orchestra.run.smithi012.stdout: inet6 fe80::ae1f:6bff:fef8:25ea/64 scope link noprefixroute 2024-04-22T13:24:02.603 INFO:teuthology.orchestra.run.smithi012.stdout: valid_lft forever preferred_lft forever 2024-04-22T13:24:02.603 INFO:teuthology.orchestra.run.smithi012.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-22T13:24:02.603 INFO:teuthology.orchestra.run.smithi012.stdout: link/ether ac:1f:6b:f8:25:eb brd ff:ff:ff:ff:ff:ff 2024-04-22T13:24:02.603 INFO:teuthology.orchestra.run.smithi012.stdout: altname enp1s0f1 2024-04-22T13:24:02.604 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-22T13:24:02.604 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T13:24:02.604 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-04-22T13:24:02.604 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add name ceph-brx type bridge 2024-04-22T13:24:02.604 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr flush dev ceph-brx 2024-04-22T13:24:02.604 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set ceph-brx up 2024-04-22T13:24:02.604 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-22T13:24:02.604 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-04-22T13:24:02.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T13:24:02.726 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:24:02 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:24:02.476+0000 7fc2f2023200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T13:24:02.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T13:24:02.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:24:02.799 DEBUG:teuthology.orchestra.run.smithi012:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-22T13:24:02.824 INFO:teuthology.orchestra.run.smithi012.stdout:1 2024-04-22T13:24:02.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:24:02.825 DEBUG:teuthology.orchestra.run.smithi012:> route 2024-04-22T13:24:02.897 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:24:02 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:24:02.482+0000 7f9135fea200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T13:24:02.907 INFO:teuthology.orchestra.run.smithi012.stdout:Kernel IP routing table 2024-04-22T13:24:02.907 INFO:teuthology.orchestra.run.smithi012.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-22T13:24:02.907 INFO:teuthology.orchestra.run.smithi012.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-22T13:24:02.907 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-22T13:24:02.907 INFO:teuthology.orchestra.run.smithi012.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-22T13:24:02.907 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T13:24:02.907 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-04-22T13:24:02.908 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-22T13:24:02.908 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-22T13:24:02.908 DEBUG:teuthology.orchestra.run.smithi012:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-22T13:24:02.908 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-04-22T13:24:02.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T13:24:03.087 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:02 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:02.833+0000 7fdcd00c0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T13:24:03.177 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:24:02 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:24:02.900+0000 7f9135fea200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T13:24:03.177 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:24:02 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:24:02.995+0000 7f9135fea200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T13:24:03.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T13:24:03.185 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:24:02 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:24:02.893+0000 7fc2f2023200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T13:24:03.185 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:24:02 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:24:02.989+0000 7fc2f2023200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T13:24:03.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:24:03.189 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-04-22T13:24:03.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:24:03.206 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list-id 2024-04-22T13:24:03.262 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T13:24:03.262 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-04-22T13:24:03.262 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-22T13:24:03.262 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-22T13:24:03.263 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-04-22T13:24:03.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T13:24:03.359 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:03 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:03.297+0000 7fdcd00c0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T13:24:03.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T13:24:03.384 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-22T13:24:03.384 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T13:24:03.384 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-04-22T13:24:03.384 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-22T13:24:03.384 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-22T13:24:03.384 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-22T13:24:03.384 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-22T13:24:03.384 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-22T13:24:03.384 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-04-22T13:24:03.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T13:24:03.477 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:24:03 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:24:03.189+0000 7fc2f2023200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T13:24:03.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T13:24:03.508 DEBUG:teuthology.orchestra.run.smithi012:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-22T13:24:03.508 DEBUG:teuthology.orchestra.run.smithi012:> set -e 2024-04-22T13:24:03.508 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set brx.0 up 2024-04-22T13:24:03.509 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link set dev brx.0 master ceph-brx 2024-04-22T13:24:03.509 DEBUG:teuthology.orchestra.run.smithi012:> ') 2024-04-22T13:24:03.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T13:24:03.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T13:24:03.587 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-22T13:24:03.587 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-22T13:24:03.602 INFO:teuthology.orchestra.run.smithi012.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-22T13:24:03.602 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-22T13:24:03.602 DEBUG:teuthology.orchestra.run.smithi012:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-22T13:24:03.631 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:03 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:03.407+0000 7fdcd00c0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T13:24:03.645 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-22T13:24:03.645 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-22T13:24:03.645 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/cd6a71e5-f19a-4226-8ca1-746deb442c27 2024-04-22T13:24:03.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:24:03.645 DEBUG:teuthology.orchestra.run.smithi012:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/cd6a71e5-f19a-4226-8ca1-746deb442c27 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-22T13:24:03.677 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:24:03 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:24:03.197+0000 7f9135fea200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T13:24:03.739 INFO:teuthology.orchestra.run.smithi012.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-22T13:24:03.739 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-22T13:24:03.857 INFO:teuthology.orchestra.run.smithi012.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-22T13:24:03.857 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-22T13:24:03.857 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: resolved to: "172.21.15.12:3300,172.21.15.111:3300,172.21.15.134:3300" 2024-04-22T13:24:03.858 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: trying mount with new device syntax: 0@0aede5ea-00aa-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/cd6a71e5-f19a-4226-8ca1-746deb442c27 2024-04-22T13:24:03.858 INFO:teuthology.orchestra.run.smithi012.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,name=0,key=0,mon_addr=172.21.15.12:3300/172.21.15.111:3300/172.21.15.134:3300" will pass to kernel 2024-04-22T13:24:03.859 INFO:teuthology.orchestra.run.smithi012.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-22T13:24:03.859 INFO:teuthology.orchestra.run.smithi012.stdout: You just mounted a file system that supports labels which does not 2024-04-22T13:24:03.859 INFO:teuthology.orchestra.run.smithi012.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-22T13:24:03.859 INFO:teuthology.orchestra.run.smithi012.stdout: applications will generate AVC messages and not be allowed access to 2024-04-22T13:24:03.859 INFO:teuthology.orchestra.run.smithi012.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-22T13:24:03.861 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-22T13:24:03.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:24:03.861 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-22T13:24:03.883 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:03 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:03.634+0000 7fdcd00c0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T13:24:03.891 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-22T13:24:03.977 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:24:03 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:24:03.711+0000 7fc2f2023200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T13:24:03.977 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:24:03 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:24:03.803+0000 7fc2f2023200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T13:24:03.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-22T13:24:03.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T13:24:03.989 INFO:teuthology.orchestra.run.smithi012.stdout:client24989 2024-04-22T13:24:03.989 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-22T13:24:03.989 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-22T13:24:04.006 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-22T13:24:04.052 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:24:03 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:24:03.722+0000 7f9135fea200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T13:24:04.053 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:24:03 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:24:03.813+0000 7f9135fea200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T13:24:04.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-22T13:24:04.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T13:24:04.103 INFO:teuthology.orchestra.run.smithi012.stdout:client24989 2024-04-22T13:24:04.104 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-22T13:24:04.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-22T13:24:04.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T13:24:04.161 INFO:teuthology.orchestra.run.smithi012.stdout:client24989 2024-04-22T13:24:04.161 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968.client24989/status') 2024-04-22T13:24:04.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-22T13:24:04.221 INFO:teuthology.orchestra.run.smithi012.stdout:instance: client.24989 (3)172.21.15.12:0/1313989247 2024-04-22T13:24:04.221 INFO:teuthology.orchestra.run.smithi012.stdout:blocklisted: false 2024-04-22T13:24:04.221 INFO:teuthology.orchestra.run.smithi012.stderr:0+1 records in 2024-04-22T13:24:04.221 INFO:teuthology.orchestra.run.smithi012.stderr:0+1 records out 2024-04-22T13:24:04.222 INFO:teuthology.orchestra.run.smithi012.stderr:71 bytes copied, 2.4818e-05 s, 2.9 MB/s 2024-04-22T13:24:04.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-22T13:24:04.229 INFO:teuthology.run_tasks:Running task exec... 2024-04-22T13:24:04.252 INFO:teuthology.task.exec:Executing custom commands... 2024-04-22T13:24:04.252 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-04-22T13:24:04.252 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-22T13:24:04.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.360+0000 7f258809a640 1 -- 172.21.15.12:0/3074868713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2580102a90 msgr2=0x7f2580104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:04.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.360+0000 7f258809a640 1 --2- 172.21.15.12:0/3074868713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2580102a90 0x7f2580104e80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2570009900 tx=0x7f257002bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:04.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.360+0000 7f258809a640 1 -- 172.21.15.12:0/3074868713 shutdown_connections 2024-04-22T13:24:04.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.360+0000 7f258809a640 1 --2- 172.21.15.12:0/3074868713 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2580107cf0 0x7f258010a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:04.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.360+0000 7f258809a640 1 --2- 172.21.15.12:0/3074868713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25801053c0 0x7f25801077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:04.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.360+0000 7f258809a640 1 --2- 172.21.15.12:0/3074868713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2580102a90 0x7f2580104e80 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:04.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.360+0000 7f258809a640 1 -- 172.21.15.12:0/3074868713 >> 172.21.15.12:0/3074868713 conn(0x7f25800fc420 msgr2=0x7f25800fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:04.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.360+0000 7f258809a640 1 -- 172.21.15.12:0/3074868713 shutdown_connections 2024-04-22T13:24:04.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.360+0000 7f258809a640 1 -- 172.21.15.12:0/3074868713 wait complete. 2024-04-22T13:24:04.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.360+0000 7f258809a640 1 Processor -- start 2024-04-22T13:24:04.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.360+0000 7f258809a640 1 -- start start 2024-04-22T13:24:04.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.361+0000 7f258809a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2580102a90 0x7f258019a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:04.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.361+0000 7f258809a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25801053c0 0x7f258019a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:04.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.361+0000 7f258809a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2580107cf0 0x7f25801a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:04.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.361+0000 7f258809a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f258010daf0 con 0x7f25801053c0 2024-04-22T13:24:04.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.361+0000 7f258809a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f258010d970 con 0x7f2580107cf0 2024-04-22T13:24:04.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.361+0000 7f258809a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f258010dc70 con 0x7f2580102a90 2024-04-22T13:24:04.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.361+0000 7f2585635640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25801053c0 0x7f258019a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:04.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.361+0000 7f2585635640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25801053c0 0x7f258019a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34654/0 (socket says 172.21.15.12:34654) 2024-04-22T13:24:04.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.361+0000 7f2585635640 1 -- 172.21.15.12:0/2849659674 learned_addr learned my addr 172.21.15.12:0/2849659674 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:04.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.361+0000 7f2585e36640 1 --2- 172.21.15.12:0/2849659674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2580102a90 0x7f258019a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:04.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.361+0000 7f2586637640 1 --2- 172.21.15.12:0/2849659674 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2580107cf0 0x7f25801a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:04.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.361+0000 7f2585635640 1 -- 172.21.15.12:0/2849659674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2580102a90 msgr2=0x7f258019a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:04.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.361+0000 7f2585635640 1 --2- 172.21.15.12:0/2849659674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2580102a90 0x7f258019a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:04.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.361+0000 7f2585635640 1 -- 172.21.15.12:0/2849659674 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2580107cf0 msgr2=0x7f25801a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:04.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.361+0000 7f2585635640 1 --2- 172.21.15.12:0/2849659674 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2580107cf0 0x7f25801a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:04.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.361+0000 7f2585635640 1 -- 172.21.15.12:0/2849659674 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25801a1fe0 con 0x7f25801053c0 2024-04-22T13:24:04.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.361+0000 7f2585635640 1 --2- 172.21.15.12:0/2849659674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25801053c0 0x7f258019a900 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f257c00e9f0 tx=0x7f257c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:04.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.361+0000 7f2576ffd640 1 -- 172.21.15.12:0/2849659674 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c00ccc0 con 0x7f25801053c0 2024-04-22T13:24:04.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.361+0000 7f2576ffd640 1 -- 172.21.15.12:0/2849659674 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f257c0054d0 con 0x7f25801053c0 2024-04-22T13:24:04.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.361+0000 7f258809a640 1 -- 172.21.15.12:0/2849659674 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25801a22d0 con 0x7f25801053c0 2024-04-22T13:24:04.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.361+0000 7f2576ffd640 1 -- 172.21.15.12:0/2849659674 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c010430 con 0x7f25801053c0 2024-04-22T13:24:04.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.361+0000 7f258809a640 1 -- 172.21.15.12:0/2849659674 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25801a27d0 con 0x7f25801053c0 2024-04-22T13:24:04.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.363+0000 7f258809a640 1 -- 172.21.15.12:0/2849659674 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25801097e0 con 0x7f25801053c0 2024-04-22T13:24:04.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.364+0000 7f2576ffd640 1 -- 172.21.15.12:0/2849659674 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f257c0046c0 con 0x7f25801053c0 2024-04-22T13:24:04.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.365+0000 7f2576ffd640 1 --2- 172.21.15.12:0/2849659674 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f256c0a3ca0 0x7f256c0a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:04.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.365+0000 7f2585e36640 1 -- 172.21.15.12:0/2849659674 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f256c0a3ca0 msgr2=0x7f256c0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1802873236 2024-04-22T13:24:04.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.365+0000 7f2576ffd640 1 -- 172.21.15.12:0/2849659674 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f257c0d0770 con 0x7f25801053c0 2024-04-22T13:24:04.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.365+0000 7f2585e36640 1 --2- 172.21.15.12:0/2849659674 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f256c0a3ca0 0x7f256c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T13:24:04.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.369+0000 7f2576ffd640 1 -- 172.21.15.12:0/2849659674 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f257c09d180 con 0x7f25801053c0 2024-04-22T13:24:04.385 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:24:04 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:24:04.046+0000 7fc2f2023200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T13:24:04.385 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:24:04 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:24:04.179+0000 7fc2f2023200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T13:24:04.385 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:24:04 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:24:04.274+0000 7fc2f2023200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T13:24:04.398 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:24:04 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:24:04.055+0000 7f9135fea200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T13:24:04.398 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:24:04 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:24:04.188+0000 7f9135fea200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T13:24:04.398 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:24:04 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:24:04.284+0000 7f9135fea200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T13:24:04.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.566+0000 7f2585e36640 1 -- 172.21.15.12:0/2849659674 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f256c0a3ca0 msgr2=0x7f256c0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1802873236 2024-04-22T13:24:04.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.566+0000 7f2585e36640 1 --2- 172.21.15.12:0/2849659674 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f256c0a3ca0 0x7f256c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T13:24:04.573 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:04 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:04.205+0000 7fdcd00c0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T13:24:04.573 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:04 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:04.309+0000 7fdcd00c0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T13:24:04.573 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:04 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:04.575+0000 7fdcd00c0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T13:24:04.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.597+0000 7f258809a640 1 -- 172.21.15.12:0/2849659674 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f2580062b50 con 0x7f25801053c0 2024-04-22T13:24:04.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.649+0000 7f2576ffd640 1 -- 172.21.15.12:0/2849659674 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v28) ==== 118+0+0 (secure 0 0 0) 0x7f257c0bd0c0 con 0x7f25801053c0 2024-04-22T13:24:04.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.652+0000 7f258809a640 1 -- 172.21.15.12:0/2849659674 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f256c0a3ca0 msgr2=0x7f256c0a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T13:24:04.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.652+0000 7f258809a640 1 --2- 172.21.15.12:0/2849659674 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f256c0a3ca0 0x7f256c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:04.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.652+0000 7f258809a640 1 -- 172.21.15.12:0/2849659674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25801053c0 msgr2=0x7f258019a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:04.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.652+0000 7f258809a640 1 --2- 172.21.15.12:0/2849659674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25801053c0 0x7f258019a900 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f257c00e9f0 tx=0x7f257c00eeb0 comp rx=0 tx=0).stop 2024-04-22T13:24:04.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.652+0000 7f258809a640 1 -- 172.21.15.12:0/2849659674 shutdown_connections 2024-04-22T13:24:04.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.652+0000 7f258809a640 1 --2- 172.21.15.12:0/2849659674 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2580107cf0 0x7f25801a1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:04.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.652+0000 7f258809a640 1 --2- 172.21.15.12:0/2849659674 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f256c0a3ca0 0x7f256c0a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:04.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.652+0000 7f258809a640 1 --2- 172.21.15.12:0/2849659674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f25801053c0 0x7f258019a900 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:04.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.652+0000 7f258809a640 1 --2- 172.21.15.12:0/2849659674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2580102a90 0x7f258019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:04.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.652+0000 7f258809a640 1 -- 172.21.15.12:0/2849659674 >> 172.21.15.12:0/2849659674 conn(0x7f25800fc420 msgr2=0x7f25801022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:04.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.652+0000 7f258809a640 1 -- 172.21.15.12:0/2849659674 shutdown_connections 2024-04-22T13:24:04.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.652+0000 7f258809a640 1 -- 172.21.15.12:0/2849659674 wait complete. 2024-04-22T13:24:04.652 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:24:04 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:24:04.388+0000 7fc2f2023200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T13:24:04.653 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:24:04 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:24:04.400+0000 7f9135fea200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T13:24:04.661 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-22T13:24:04.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.770+0000 7f2b3feeb640 1 -- 172.21.15.12:0/1653531634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b38102a00 msgr2=0x7f2b38109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:04.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.770+0000 7f2b3feeb640 1 --2- 172.21.15.12:0/1653531634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b38102a00 0x7f2b38109320 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2b30009ca0 tx=0x7f2b3002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:24:04.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.770+0000 7f2b3feeb640 1 -- 172.21.15.12:0/1653531634 shutdown_connections 2024-04-22T13:24:04.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.770+0000 7f2b3feeb640 1 --2- 172.21.15.12:0/1653531634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b38102a00 0x7f2b38109320 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:04.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.770+0000 7f2b3feeb640 1 --2- 172.21.15.12:0/1653531634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b38102040 0x7f2b381024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:04.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.770+0000 7f2b3feeb640 1 --2- 172.21.15.12:0/1653531634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b38100e60 0x7f2b38101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:04.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.770+0000 7f2b3feeb640 1 -- 172.21.15.12:0/1653531634 >> 172.21.15.12:0/1653531634 conn(0x7f2b380fc420 msgr2=0x7f2b380fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:04.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.770+0000 7f2b3feeb640 1 -- 172.21.15.12:0/1653531634 shutdown_connections 2024-04-22T13:24:04.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.770+0000 7f2b3feeb640 1 -- 172.21.15.12:0/1653531634 wait complete. 2024-04-22T13:24:04.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.771+0000 7f2b3feeb640 1 Processor -- start 2024-04-22T13:24:04.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.771+0000 7f2b3feeb640 1 -- start start 2024-04-22T13:24:04.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.771+0000 7f2b3feeb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b38100e60 0x7f2b38104e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:04.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.771+0000 7f2b3feeb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b38102040 0x7f2b381034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:04.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.771+0000 7f2b3feeb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b38102a00 0x7f2b381039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:04.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.771+0000 7f2b3feeb640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3810ca10 con 0x7f2b38102040 2024-04-22T13:24:04.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.771+0000 7f2b3feeb640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3810c890 con 0x7f2b38100e60 2024-04-22T13:24:04.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.771+0000 7f2b3feeb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3810cb90 con 0x7f2b38102a00 2024-04-22T13:24:04.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.771+0000 7f2b3d486640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b38102040 0x7f2b381034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:04.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.771+0000 7f2b3dc87640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b38100e60 0x7f2b38104e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:04.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.771+0000 7f2b3dc87640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b38100e60 0x7f2b38104e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:49384/0 (socket says 172.21.15.12:49384) 2024-04-22T13:24:04.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.771+0000 7f2b3d486640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b38102040 0x7f2b381034b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34686/0 (socket says 172.21.15.12:34686) 2024-04-22T13:24:04.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.771+0000 7f2b3dc87640 1 -- 172.21.15.12:0/1672916618 learned_addr learned my addr 172.21.15.12:0/1672916618 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:04.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.771+0000 7f2b3e488640 1 --2- 172.21.15.12:0/1672916618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b38102a00 0x7f2b381039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:04.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.771+0000 7f2b3dc87640 1 -- 172.21.15.12:0/1672916618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b38102a00 msgr2=0x7f2b381039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:04.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.771+0000 7f2b3dc87640 1 --2- 172.21.15.12:0/1672916618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b38102a00 0x7f2b381039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:04.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.771+0000 7f2b3dc87640 1 -- 172.21.15.12:0/1672916618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b38102040 msgr2=0x7f2b381034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:04.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.771+0000 7f2b3dc87640 1 --2- 172.21.15.12:0/1672916618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b38102040 0x7f2b381034b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:04.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.771+0000 7f2b3dc87640 1 -- 172.21.15.12:0/1672916618 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b381042b0 con 0x7f2b38100e60 2024-04-22T13:24:04.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.771+0000 7f2b3d486640 1 --2- 172.21.15.12:0/1672916618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b38102040 0x7f2b381034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:24:04.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.772+0000 7f2b3dc87640 1 --2- 172.21.15.12:0/1672916618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b38100e60 0x7f2b38104e20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2b2800e9c0 tx=0x7f2b2800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:04.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.772+0000 7f2b2effd640 1 -- 172.21.15.12:0/1672916618 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2800cd80 con 0x7f2b38100e60 2024-04-22T13:24:04.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.772+0000 7f2b3feeb640 1 -- 172.21.15.12:0/1672916618 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b381a2260 con 0x7f2b38100e60 2024-04-22T13:24:04.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.772+0000 7f2b2effd640 1 -- 172.21.15.12:0/1672916618 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b280035f0 con 0x7f2b38100e60 2024-04-22T13:24:04.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.772+0000 7f2b2effd640 1 -- 172.21.15.12:0/1672916618 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b28010690 con 0x7f2b38100e60 2024-04-22T13:24:04.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.772+0000 7f2b3feeb640 1 -- 172.21.15.12:0/1672916618 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b381a2730 con 0x7f2b38100e60 2024-04-22T13:24:04.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.773+0000 7f2b3feeb640 1 -- 172.21.15.12:0/1672916618 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b381a29f0 con 0x7f2b38100e60 2024-04-22T13:24:04.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.774+0000 7f2b2effd640 1 -- 172.21.15.12:0/1672916618 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f2b28004000 con 0x7f2b38100e60 2024-04-22T13:24:04.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.774+0000 7f2b2effd640 1 --2- 172.21.15.12:0/1672916618 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f2b240a3ca0 0x7f2b240a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:04.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.775+0000 7f2b3d486640 1 -- 172.21.15.12:0/1672916618 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f2b240a3ca0 msgr2=0x7f2b240a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1802873236 2024-04-22T13:24:04.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.775+0000 7f2b2effd640 1 -- 172.21.15.12:0/1672916618 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f2b280d0a50 con 0x7f2b38100e60 2024-04-22T13:24:04.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.775+0000 7f2b3d486640 1 --2- 172.21.15.12:0/1672916618 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f2b240a3ca0 0x7f2b240a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-22T13:24:04.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.778+0000 7f2b2effd640 1 -- 172.21.15.12:0/1672916618 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2b2809d400 con 0x7f2b38100e60 2024-04-22T13:24:04.825 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:04 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2849659674' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-22T13:24:04.825 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:04 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:04.723+0000 7fdcd00c0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T13:24:04.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:04 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2849659674' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-22T13:24:04.928 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:24:04 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:24:04.707+0000 7f9135fea200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T13:24:04.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.976+0000 7f2b3d486640 1 -- 172.21.15.12:0/1672916618 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f2b240a3ca0 msgr2=0x7f2b240a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1802873236 2024-04-22T13:24:04.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.976+0000 7f2b3d486640 1 --2- 172.21.15.12:0/1672916618 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f2b240a3ca0 0x7f2b240a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-22T13:24:04.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:04 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2849659674' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-22T13:24:04.977 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:24:04 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:24:04.690+0000 7fc2f2023200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T13:24:04.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:04.997+0000 7f2b3feeb640 1 -- 172.21.15.12:0/1672916618 --> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f2b380fe660 con 0x7f2b240a3ca0 2024-04-22T13:24:05.087 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:04 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:04.827+0000 7fdcd00c0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T13:24:05.087 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:04 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:04.953+0000 7fdcd00c0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T13:24:05.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:05.377+0000 7f2b3d486640 1 -- 172.21.15.12:0/1672916618 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f2b240a3ca0 msgr2=0x7f2b240a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.12:6800/1802873236 2024-04-22T13:24:05.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:05.377+0000 7f2b3d486640 1 --2- 172.21.15.12:0/1672916618 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f2b240a3ca0 0x7f2b240a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-22T13:24:05.587 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:05 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:05.271+0000 7fdcd00c0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T13:24:05.653 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:24:05 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:24:05.306+0000 7f9135fea200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T13:24:05.653 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:24:05 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z[33203]: 2024-04-22T13:24:05.522+0000 7f9135fea200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T13:24:05.653 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:24:05 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:24:05.289+0000 7fc2f2023200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T13:24:05.653 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:24:05 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y[32751]: 2024-04-22T13:24:05.506+0000 7fc2f2023200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T13:24:05.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:05.671+0000 7f2b2effd640 1 -- 172.21.15.12:0/1672916618 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7f2b28099370 con 0x7f2b38100e60 2024-04-22T13:24:05.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:05 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2849659674' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-22T13:24:05.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:05 smithi012 ceph-mon[28265]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-22T13:24:05.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:05 smithi012 ceph-mon[28265]: Standby manager daemon y restarted 2024-04-22T13:24:05.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:05 smithi012 ceph-mon[28265]: Standby manager daemon y started 2024-04-22T13:24:05.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:05 smithi012 ceph-mon[28265]: from='mgr.? 172.21.15.111:0/1384434491' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-22T13:24:05.918 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:05 smithi012 ceph-mon[28265]: from='mgr.? 172.21.15.111:0/1384434491' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T13:24:05.919 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:05 smithi012 ceph-mon[28265]: from='mgr.? 172.21.15.111:0/1384434491' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-22T13:24:05.919 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:05 smithi012 ceph-mon[28265]: from='mgr.? 172.21.15.111:0/1384434491' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T13:24:05.919 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:05 smithi012 ceph-mon[28265]: Standby manager daemon z restarted 2024-04-22T13:24:05.919 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:05 smithi012 ceph-mon[28265]: Standby manager daemon z started 2024-04-22T13:24:05.919 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:05 smithi012 ceph-mon[28265]: from='mgr.? 172.21.15.134:0/1929566507' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-22T13:24:05.919 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:05 smithi012 ceph-mon[28265]: from='mgr.? 172.21.15.134:0/1929566507' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T13:24:05.919 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:05 smithi012 ceph-mon[28265]: from='mgr.? 172.21.15.134:0/1929566507' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-22T13:24:05.919 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:05 smithi012 ceph-mon[28265]: from='mgr.? 172.21.15.134:0/1929566507' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T13:24:05.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:05 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2849659674' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-22T13:24:05.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:05 smithi134 ceph-mon[31338]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-22T13:24:05.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:05 smithi134 ceph-mon[31338]: Standby manager daemon y restarted 2024-04-22T13:24:05.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:05 smithi134 ceph-mon[31338]: Standby manager daemon y started 2024-04-22T13:24:05.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:05 smithi134 ceph-mon[31338]: from='mgr.? 172.21.15.111:0/1384434491' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-22T13:24:05.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:05 smithi134 ceph-mon[31338]: from='mgr.? 172.21.15.111:0/1384434491' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T13:24:05.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:05 smithi134 ceph-mon[31338]: from='mgr.? 172.21.15.111:0/1384434491' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-22T13:24:05.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:05 smithi134 ceph-mon[31338]: from='mgr.? 172.21.15.111:0/1384434491' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T13:24:05.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:05 smithi134 ceph-mon[31338]: Standby manager daemon z restarted 2024-04-22T13:24:05.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:05 smithi134 ceph-mon[31338]: Standby manager daemon z started 2024-04-22T13:24:05.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:05 smithi134 ceph-mon[31338]: from='mgr.? 172.21.15.134:0/1929566507' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-22T13:24:05.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:05 smithi134 ceph-mon[31338]: from='mgr.? 172.21.15.134:0/1929566507' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T13:24:05.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:05 smithi134 ceph-mon[31338]: from='mgr.? 172.21.15.134:0/1929566507' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-22T13:24:05.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:05 smithi134 ceph-mon[31338]: from='mgr.? 172.21.15.134:0/1929566507' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T13:24:05.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:05 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2849659674' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-22T13:24:05.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:05 smithi111 ceph-mon[31361]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-22T13:24:05.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:05 smithi111 ceph-mon[31361]: Standby manager daemon y restarted 2024-04-22T13:24:05.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:05 smithi111 ceph-mon[31361]: Standby manager daemon y started 2024-04-22T13:24:05.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:05 smithi111 ceph-mon[31361]: from='mgr.? 172.21.15.111:0/1384434491' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-22T13:24:05.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:05 smithi111 ceph-mon[31361]: from='mgr.? 172.21.15.111:0/1384434491' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T13:24:05.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:05 smithi111 ceph-mon[31361]: from='mgr.? 172.21.15.111:0/1384434491' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-22T13:24:05.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:05 smithi111 ceph-mon[31361]: from='mgr.? 172.21.15.111:0/1384434491' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T13:24:05.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:05 smithi111 ceph-mon[31361]: Standby manager daemon z restarted 2024-04-22T13:24:05.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:05 smithi111 ceph-mon[31361]: Standby manager daemon z started 2024-04-22T13:24:05.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:05 smithi111 ceph-mon[31361]: from='mgr.? 172.21.15.134:0/1929566507' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-22T13:24:05.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:05 smithi111 ceph-mon[31361]: from='mgr.? 172.21.15.134:0/1929566507' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-22T13:24:05.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:05 smithi111 ceph-mon[31361]: from='mgr.? 172.21.15.134:0/1929566507' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-22T13:24:05.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:05 smithi111 ceph-mon[31361]: from='mgr.? 172.21.15.134:0/1929566507' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-22T13:24:06.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:06.168+0000 7f2b2effd640 1 -- 172.21.15.12:0/1672916618 <== mon.2 v2:172.21.15.111:3300/0 8 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f2b280ce5a0 con 0x7f2b38100e60 2024-04-22T13:24:06.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:06.168+0000 7f2b2effd640 1 -- 172.21.15.12:0/1672916618 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f2b240a3ca0 msgr2=0x7f2b240a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-22T13:24:06.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:06.168+0000 7f2b2effd640 1 --2- 172.21.15.12:0/1672916618 >> [v2:172.21.15.12:6800/1802873236,v1:172.21.15.12:6801/1802873236] conn(0x7f2b240a3ca0 0x7f2b240a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:06.328 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:05 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:05.920+0000 7fdcd00c0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T13:24:06.329 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:06.151+0000 7fdcd00c0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T13:24:06.587 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:06.402+0000 7fdc31a38700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:24:06.587 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:06.402+0000 7fdc31a38700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:24:06.587 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:06.402+0000 7fdc31a38700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:24:06.587 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:06.402+0000 7fdc31a38700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:24:06.588 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:06.402+0000 7fdc31a38700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:24:06.588 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:06.403+0000 7fdc2e9f2700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:24:06.588 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:06.403+0000 7fdc2e9f2700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:24:06.588 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:06.403+0000 7fdc2e9f2700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:24:06.588 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:06.403+0000 7fdc2e9f2700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:24:06.588 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:06.403+0000 7fdc2e9f2700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:24:06.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-22T13:24:06.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: Active manager daemon x restarted 2024-04-22T13:24:06.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: Activating manager daemon x 2024-04-22T13:24:06.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: osdmap e90: 12 total, 12 up, 12 in 2024-04-22T13:24:06.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: mgrmap e21: x(active, starting, since 0.0128332s), standbys: z, y 2024-04-22T13:24:06.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:24:06.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:24:06.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:24:06.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-22T13:24:06.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-22T13:24:06.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-22T13:24:06.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-22T13:24:06.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-22T13:24:06.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-22T13:24:06.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-22T13:24:06.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T13:24:06.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-22T13:24:06.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-22T13:24:06.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-22T13:24:06.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-22T13:24:06.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-22T13:24:06.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-22T13:24:06.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-22T13:24:06.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:24:06.930 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:24:06.930 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:24:06.930 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:24:06.930 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:24:06.930 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:24:06.930 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:24:06.930 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:24:06.930 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:24:06.930 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:24:06.930 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:24:06.930 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:24:06.930 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T13:24:06.930 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T13:24:06.930 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T13:24:06.931 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: Manager daemon x is now available 2024-04-22T13:24:06.931 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:24:06.931 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-22T13:24:06.931 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:06 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-22T13:24:06.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-22T13:24:06.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: Active manager daemon x restarted 2024-04-22T13:24:06.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: Activating manager daemon x 2024-04-22T13:24:06.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: osdmap e90: 12 total, 12 up, 12 in 2024-04-22T13:24:06.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: mgrmap e21: x(active, starting, since 0.0128332s), standbys: z, y 2024-04-22T13:24:06.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:24:06.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:24:06.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:24:06.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-22T13:24:06.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-22T13:24:06.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-22T13:24:06.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-22T13:24:06.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-22T13:24:06.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-22T13:24:06.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-22T13:24:06.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T13:24:06.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-22T13:24:06.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-22T13:24:06.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-22T13:24:06.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-22T13:24:06.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-22T13:24:06.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-22T13:24:06.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-22T13:24:06.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:24:06.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:24:06.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:24:06.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:24:06.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:24:06.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:24:06.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:24:06.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:24:06.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:24:06.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:24:06.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:24:06.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:24:06.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T13:24:06.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T13:24:06.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T13:24:06.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: Manager daemon x is now available 2024-04-22T13:24:06.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:24:06.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-22T13:24:06.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:06 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-22T13:24:07.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-22T13:24:07.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: Active manager daemon x restarted 2024-04-22T13:24:07.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: Activating manager daemon x 2024-04-22T13:24:07.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: osdmap e90: 12 total, 12 up, 12 in 2024-04-22T13:24:07.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: mgrmap e21: x(active, starting, since 0.0128332s), standbys: z, y 2024-04-22T13:24:07.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-22T13:24:07.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-22T13:24:07.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-22T13:24:07.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-22T13:24:07.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-22T13:24:07.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-22T13:24:07.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-22T13:24:07.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-22T13:24:07.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-22T13:24:07.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-22T13:24:07.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T13:24:07.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-22T13:24:07.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-22T13:24:07.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-22T13:24:07.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-22T13:24:07.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-22T13:24:07.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-22T13:24:07.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-22T13:24:07.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-22T13:24:07.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-22T13:24:07.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-22T13:24:07.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-22T13:24:07.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-22T13:24:07.089 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-22T13:24:07.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-22T13:24:07.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-22T13:24:07.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-22T13:24:07.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-22T13:24:07.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-22T13:24:07.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-22T13:24:07.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-22T13:24:07.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-22T13:24:07.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-22T13:24:07.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: Manager daemon x is now available 2024-04-22T13:24:07.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:24:07.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-22T13:24:07.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:06 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-22T13:24:07.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.170+0000 7f2b2effd640 1 -- 172.21.15.12:0/1672916618 <== mon.2 v2:172.21.15.111:3300/0 9 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f2b2804f350 con 0x7f2b38100e60 2024-04-22T13:24:07.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.171+0000 7f2b2effd640 1 --2- 172.21.15.12:0/1672916618 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2b24032330 0x7f2b24032710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:07.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.171+0000 7f2b2effd640 1 -- 172.21.15.12:0/1672916618 --> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f2b24041850 con 0x7f2b24032330 2024-04-22T13:24:07.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.175+0000 7f2b3d486640 1 --2- 172.21.15.12:0/1672916618 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2b24032330 0x7f2b24032710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:07.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.177+0000 7f2b3d486640 1 --2- 172.21.15.12:0/1672916618 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2b24032330 0x7f2b24032710 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2b3400a010 tx=0x7f2b34005f90 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:07.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.198+0000 7f2b2effd640 1 -- 172.21.15.12:0/1672916618 <== mgr.24968 v2:172.21.15.12:6800/1615661596 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f2b24041850 con 0x7f2b24032330 2024-04-22T13:24:07.196 INFO:teuthology.orchestra.run.smithi012.stdout:{} 2024-04-22T13:24:07.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.200+0000 7f2b3feeb640 1 -- 172.21.15.12:0/1672916618 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2b24032330 msgr2=0x7f2b24032710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:07.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.200+0000 7f2b3feeb640 1 --2- 172.21.15.12:0/1672916618 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2b24032330 0x7f2b24032710 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2b3400a010 tx=0x7f2b34005f90 comp rx=0 tx=0).stop 2024-04-22T13:24:07.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.200+0000 7f2b3feeb640 1 -- 172.21.15.12:0/1672916618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b38100e60 msgr2=0x7f2b38104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:07.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.200+0000 7f2b3feeb640 1 --2- 172.21.15.12:0/1672916618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b38100e60 0x7f2b38104e20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2b2800e9c0 tx=0x7f2b2800ee80 comp rx=0 tx=0).stop 2024-04-22T13:24:07.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.200+0000 7f2b3e488640 1 -- 172.21.15.12:0/1672916618 reap_dead start 2024-04-22T13:24:07.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.201+0000 7f2b3feeb640 1 -- 172.21.15.12:0/1672916618 shutdown_connections 2024-04-22T13:24:07.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.201+0000 7f2b3feeb640 1 -- 172.21.15.12:0/1672916618 >> 172.21.15.12:0/1672916618 conn(0x7f2b380fc420 msgr2=0x7f2b380fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:07.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.201+0000 7f2b3feeb640 1 -- 172.21.15.12:0/1672916618 shutdown_connections 2024-04-22T13:24:07.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.201+0000 7f2b3feeb640 1 -- 172.21.15.12:0/1672916618 wait complete. 2024-04-22T13:24:07.210 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-22T13:24:07.221 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-22T13:24:07.232 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:24:07.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.337+0000 7fbc34c70640 1 -- 172.21.15.12:0/3920086411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc30107c60 msgr2=0x7fbc3010a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:07.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.337+0000 7fbc34c70640 1 --2- 172.21.15.12:0/3920086411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc30107c60 0x7fbc3010a0b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fbc28009900 tx=0x7fbc2802bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:07.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.338+0000 7fbc34c70640 1 -- 172.21.15.12:0/3920086411 shutdown_connections 2024-04-22T13:24:07.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.338+0000 7fbc34c70640 1 --2- 172.21.15.12:0/3920086411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc30107c60 0x7fbc3010a0b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:07.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.338+0000 7fbc34c70640 1 --2- 172.21.15.12:0/3920086411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc30105330 0x7fbc30107720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:07.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.338+0000 7fbc34c70640 1 --2- 172.21.15.12:0/3920086411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc30102a00 0x7fbc30104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:07.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.338+0000 7fbc34c70640 1 -- 172.21.15.12:0/3920086411 >> 172.21.15.12:0/3920086411 conn(0x7fbc300fc410 msgr2=0x7fbc300fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:07.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.338+0000 7fbc34c70640 1 -- 172.21.15.12:0/3920086411 shutdown_connections 2024-04-22T13:24:07.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.338+0000 7fbc34c70640 1 -- 172.21.15.12:0/3920086411 wait complete. 2024-04-22T13:24:07.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.338+0000 7fbc34c70640 1 Processor -- start 2024-04-22T13:24:07.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.338+0000 7fbc34c70640 1 -- start start 2024-04-22T13:24:07.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.339+0000 7fbc34c70640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc30102a00 0x7fbc30198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:07.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.339+0000 7fbc34c70640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc30105330 0x7fbc30198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:07.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.339+0000 7fbc34c70640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc30107c60 0x7fbc3019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:07.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.339+0000 7fbc34c70640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3010dab0 con 0x7fbc30102a00 2024-04-22T13:24:07.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.339+0000 7fbc34c70640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3010d930 con 0x7fbc30105330 2024-04-22T13:24:07.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.339+0000 7fbc34c70640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3010dc30 con 0x7fbc30107c60 2024-04-22T13:24:07.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.339+0000 7fbc2e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc30102a00 0x7fbc30198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:07.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.339+0000 7fbc2e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc30102a00 0x7fbc30198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34842/0 (socket says 172.21.15.12:34842) 2024-04-22T13:24:07.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.339+0000 7fbc2e59c640 1 -- 172.21.15.12:0/2897407443 learned_addr learned my addr 172.21.15.12:0/2897407443 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:07.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.339+0000 7fbc2ed9d640 1 --2- 172.21.15.12:0/2897407443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc30107c60 0x7fbc3019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:07.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.339+0000 7fbc2dd9b640 1 --2- 172.21.15.12:0/2897407443 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc30105330 0x7fbc30198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:07.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.339+0000 7fbc2e59c640 1 -- 172.21.15.12:0/2897407443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc30107c60 msgr2=0x7fbc3019f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:07.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.339+0000 7fbc2e59c640 1 --2- 172.21.15.12:0/2897407443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc30107c60 0x7fbc3019f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:07.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.339+0000 7fbc2e59c640 1 -- 172.21.15.12:0/2897407443 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc30105330 msgr2=0x7fbc30198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:07.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.339+0000 7fbc2e59c640 1 --2- 172.21.15.12:0/2897407443 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc30105330 0x7fbc30198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:07.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.339+0000 7fbc2e59c640 1 -- 172.21.15.12:0/2897407443 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc3019fd20 con 0x7fbc30102a00 2024-04-22T13:24:07.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.339+0000 7fbc2e59c640 1 --2- 172.21.15.12:0/2897407443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc30102a00 0x7fbc30198100 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000eab0 tx=0x7fbc2000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:07.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.339+0000 7fbc1f7fe640 1 -- 172.21.15.12:0/2897407443 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2000ce40 con 0x7fbc30102a00 2024-04-22T13:24:07.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.339+0000 7fbc1f7fe640 1 -- 172.21.15.12:0/2897407443 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc20011920 con 0x7fbc30102a00 2024-04-22T13:24:07.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.339+0000 7fbc34c70640 1 -- 172.21.15.12:0/2897407443 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc3019ffb0 con 0x7fbc30102a00 2024-04-22T13:24:07.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.339+0000 7fbc1f7fe640 1 -- 172.21.15.12:0/2897407443 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc20010a30 con 0x7fbc30102a00 2024-04-22T13:24:07.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.339+0000 7fbc34c70640 1 -- 172.21.15.12:0/2897407443 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc301a0470 con 0x7fbc30102a00 2024-04-22T13:24:07.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.341+0000 7fbc34c70640 1 -- 172.21.15.12:0/2897407443 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc300fdf00 con 0x7fbc30102a00 2024-04-22T13:24:07.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.342+0000 7fbc1f7fe640 1 -- 172.21.15.12:0/2897407443 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fbc20010bd0 con 0x7fbc30102a00 2024-04-22T13:24:07.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.343+0000 7fbc1f7fe640 1 --2- 172.21.15.12:0/2897407443 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbc180a39b0 0x7fbc180a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:07.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.343+0000 7fbc2dd9b640 1 --2- 172.21.15.12:0/2897407443 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbc180a39b0 0x7fbc180a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:07.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.343+0000 7fbc1f7fe640 1 -- 172.21.15.12:0/2897407443 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fbc200d0670 con 0x7fbc30102a00 2024-04-22T13:24:07.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.343+0000 7fbc2dd9b640 1 --2- 172.21.15.12:0/2897407443 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbc180a39b0 0x7fbc180a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbc30199610 tx=0x7fbc24005f90 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:07.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.347+0000 7fbc1f7fe640 1 -- 172.21.15.12:0/2897407443 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbc20016030 con 0x7fbc30102a00 2024-04-22T13:24:07.439 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:07 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:07.183+0000 7fdc2c9ee700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:24:07.439 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:07 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:07.183+0000 7fdc2c9ee700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:24:07.439 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:07 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:07.183+0000 7fdc2c9ee700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:24:07.439 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:07 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:07.183+0000 7fdc2c9ee700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:24:07.439 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:24:07 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:24:07.183+0000 7fdc2c9ee700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:24:07.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.538+0000 7fbc34c70640 1 -- 172.21.15.12:0/2897407443 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc30062bf0 con 0x7fbc30102a00 2024-04-22T13:24:07.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.540+0000 7fbc1f7fe640 1 -- 172.21.15.12:0/2897407443 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22997 (secure 0 0 0) 0x7fbc2009d230 con 0x7fbc30102a00 2024-04-22T13:24:07.538 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:24:07.538 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":90,"fsid":"0aede5ea-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:14:09.362918+0000","modified":"2024-04-22T13:24:06.155595+0000","last_up_change":"2024-04-22T13:21:15.263393+0000","last_in_change":"2024-04-22T13:20:54.174685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:17:32.372434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:22:02.768799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:22:03.578730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d9e2365-3a64-4b13-96fc-c217b1cee2ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6803","nonce":1191350632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6805","nonce":1191350632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6809","nonce":1191350632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6807","nonce":1191350632}]},"public_addr":"172.21.15.12:6803/1191350632","cluster_addr":"172.21.15.12:6805/1191350632","heartbeat_back_addr":"172.21.15.12:6809/1191350632","heartbeat_front_addr":"172.21.15.12:6807/1191350632","state":["exists","up"]},{"osd":1,"uuid":"1499c783-2bd7-48ce-a444-34add42bcf31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6801","nonce":417496465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6803","nonce":417496465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6807","nonce":417496465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6805","nonce":417496465}]},"public_addr":"172.21.15.111:6801/417496465","cluster_addr":"172.21.15.111:6803/417496465","heartbeat_back_addr":"172.21.15.111:6807/417496465","heartbeat_front_addr":"172.21.15.111:6805/417496465","state":["exists","up"]},{"osd":2,"uuid":"71a9ebc3-c622-434e-a6c2-f48358f384f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6801","nonce":1403153998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6803","nonce":1403153998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6807","nonce":1403153998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6805","nonce":1403153998}]},"public_addr":"172.21.15.134:6801/1403153998","cluster_addr":"172.21.15.134:6803/1403153998","heartbeat_back_addr":"172.21.15.134:6807/1403153998","heartbeat_front_addr":"172.21.15.134:6805/1403153998","state":["exists","up"]},{"osd":3,"uuid":"6dd00db2-a286-48ba-8666-56ffdd5f35ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6811","nonce":3359066681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6813","nonce":3359066681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6817","nonce":3359066681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6815","nonce":3359066681}]},"public_addr":"172.21.15.12:6811/3359066681","cluster_addr":"172.21.15.12:6813/3359066681","heartbeat_back_addr":"172.21.15.12:6817/3359066681","heartbeat_front_addr":"172.21.15.12:6815/3359066681","state":["exists","up"]},{"osd":4,"uuid":"1e29afd7-354c-4506-aaa0-b43eb1a19a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6809","nonce":2886349872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6811","nonce":2886349872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6815","nonce":2886349872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6813","nonce":2886349872}]},"public_addr":"172.21.15.111:6809/2886349872","cluster_addr":"172.21.15.111:6811/2886349872","heartbeat_back_addr":"172.21.15.111:6815/2886349872","heartbeat_front_addr":"172.21.15.111:6813/2886349872","state":["exists","up"]},{"osd":5,"uuid":"8f40f02d-0ba2-4842-a03d-2a1a04d1010d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6809","nonce":3071977602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6811","nonce":3071977602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6815","nonce":3071977602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6813","nonce":3071977602}]},"public_addr":"172.21.15.134:6809/3071977602","cluster_addr":"172.21.15.134:6811/3071977602","heartbeat_back_addr":"172.21.15.134:6815/3071977602","heartbeat_front_addr":"172.21.15.134:6813/3071977602","state":["exists","up"]},{"osd":6,"uuid":"bb0c17a9-6a8e-4be6-8660-f73cfa8200ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6819","nonce":3819619299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6821","nonce":3819619299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6825","nonce":3819619299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6823","nonce":3819619299}]},"public_addr":"172.21.15.12:6819/3819619299","cluster_addr":"172.21.15.12:6821/3819619299","heartbeat_back_addr":"172.21.15.12:6825/3819619299","heartbeat_front_addr":"172.21.15.12:6823/3819619299","state":["exists","up"]},{"osd":7,"uuid":"0a93d6af-569e-4e45-b697-9e978c1c1004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6817","nonce":3113856282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6819","nonce":3113856282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6823","nonce":3113856282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6821","nonce":3113856282}]},"public_addr":"172.21.15.111:6817/3113856282","cluster_addr":"172.21.15.111:6819/3113856282","heartbeat_back_addr":"172.21.15.111:6823/3113856282","heartbeat_front_addr":"172.21.15.111:6821/3113856282","state":["exists","up"]},{"osd":8,"uuid":"e21acc68-cf50-4f4f-b1ba-cf6c195b4955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6817","nonce":1689362191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6819","nonce":1689362191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6823","nonce":1689362191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6821","nonce":1689362191}]},"public_addr":"172.21.15.134:6817/1689362191","cluster_addr":"172.21.15.134:6819/1689362191","heartbeat_back_addr":"172.21.15.134:6823/1689362191","heartbeat_front_addr":"172.21.15.134:6821/1689362191","state":["exists","up"]},{"osd":9,"uuid":"fd745869-7f5f-48a8-b412-bb9c92b586e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6827","nonce":4292385779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6829","nonce":4292385779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6833","nonce":4292385779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6831","nonce":4292385779}]},"public_addr":"172.21.15.12:6827/4292385779","cluster_addr":"172.21.15.12:6829/4292385779","heartbeat_back_addr":"172.21.15.12:6833/4292385779","heartbeat_front_addr":"172.21.15.12:6831/4292385779","state":["exists","up"]},{"osd":10,"uuid":"798b1abd-f6c7-4a5f-8da0-fdae49e50d2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6825","nonce":3338566976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6827","nonce":3338566976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6831","nonce":3338566976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6829","nonce":3338566976}]},"public_addr":"172.21.15.111:6825/3338566976","cluster_addr":"172.21.15.111:6827/3338566976","heartbeat_back_addr":"172.21.15.111:6831/3338566976","heartbeat_front_addr":"172.21.15.111:6829/3338566976","state":["exists","up"]},{"osd":11,"uuid":"b18fc5f3-1049-4871-b060-e99290273cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6825","nonce":578640428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6827","nonce":578640428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6831","nonce":578640428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6829","nonce":578640428}]},"public_addr":"172.21.15.134:6825/578640428","cluster_addr":"172.21.15.134:6827/578640428","heartbeat_back_addr":"172.21.15.134:6831/578640428","heartbeat_front_addr":"172.21.15.134:6829/578640428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:16:40.828608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:05.052528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:28.297316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:53.496497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:17.947259+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:41.712894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:07.105945+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:32.129688+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:56.008516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:22.239867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:47.334692+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:12.301276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/3535299028":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/2837479598":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/3010129696":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/2593601362":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/2519382069":"2024-04-23T13:15:04.279990+0000","172.21.15.12:6801/2009831940":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/856202072":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/3509412187":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/1802873236":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/1007749062":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3588480095":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1904962176":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/1802873236":"2024-04-23T13:24:06.155564+0000","172.21.15.12:6800/2009831940":"2024-04-23T13:15:04.279990+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:24:07.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.542+0000 7fbc34c70640 1 -- 172.21.15.12:0/2897407443 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbc180a39b0 msgr2=0x7fbc180a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:07.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.542+0000 7fbc34c70640 1 --2- 172.21.15.12:0/2897407443 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbc180a39b0 0x7fbc180a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbc30199610 tx=0x7fbc24005f90 comp rx=0 tx=0).stop 2024-04-22T13:24:07.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.542+0000 7fbc34c70640 1 -- 172.21.15.12:0/2897407443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc30102a00 msgr2=0x7fbc30198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:07.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.542+0000 7fbc34c70640 1 --2- 172.21.15.12:0/2897407443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc30102a00 0x7fbc30198100 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000eab0 tx=0x7fbc2000ef70 comp rx=0 tx=0).stop 2024-04-22T13:24:07.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.542+0000 7fbc34c70640 1 -- 172.21.15.12:0/2897407443 shutdown_connections 2024-04-22T13:24:07.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.542+0000 7fbc34c70640 1 --2- 172.21.15.12:0/2897407443 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbc180a39b0 0x7fbc180a5e70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:07.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.542+0000 7fbc34c70640 1 --2- 172.21.15.12:0/2897407443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbc30107c60 0x7fbc3019f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:07.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.542+0000 7fbc34c70640 1 --2- 172.21.15.12:0/2897407443 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc30105330 0x7fbc30198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:07.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.542+0000 7fbc34c70640 1 --2- 172.21.15.12:0/2897407443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbc30102a00 0x7fbc30198100 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:07.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.542+0000 7fbc34c70640 1 -- 172.21.15.12:0/2897407443 >> 172.21.15.12:0/2897407443 conn(0x7fbc300fc410 msgr2=0x7fbc301032a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:07.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.542+0000 7fbc34c70640 1 -- 172.21.15.12:0/2897407443 shutdown_connections 2024-04-22T13:24:07.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.542+0000 7fbc34c70640 1 -- 172.21.15.12:0/2897407443 wait complete. 2024-04-22T13:24:07.551 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T13:17:32.372434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T13:22:02.768799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T13:22:03.578730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T13:24:07.552 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T13:24:07.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.662+0000 7fc146625640 1 -- 172.21.15.12:0/556424435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc140107c90 msgr2=0x7fc14010a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:07.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.662+0000 7fc146625640 1 --2- 172.21.15.12:0/556424435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc140107c90 0x7fc14010a0e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc138009ca0 tx=0x7fc13802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:24:07.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.662+0000 7fc146625640 1 -- 172.21.15.12:0/556424435 shutdown_connections 2024-04-22T13:24:07.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.662+0000 7fc146625640 1 --2- 172.21.15.12:0/556424435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc140107c90 0x7fc14010a0e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:07.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.662+0000 7fc146625640 1 --2- 172.21.15.12:0/556424435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc140105360 0x7fc140107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:07.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.662+0000 7fc146625640 1 --2- 172.21.15.12:0/556424435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc140102a30 0x7fc140104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:07.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.662+0000 7fc146625640 1 -- 172.21.15.12:0/556424435 >> 172.21.15.12:0/556424435 conn(0x7fc1400fc440 msgr2=0x7fc1400fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:07.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.662+0000 7fc146625640 1 -- 172.21.15.12:0/556424435 shutdown_connections 2024-04-22T13:24:07.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.662+0000 7fc146625640 1 -- 172.21.15.12:0/556424435 wait complete. 2024-04-22T13:24:07.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.663+0000 7fc146625640 1 Processor -- start 2024-04-22T13:24:07.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.663+0000 7fc146625640 1 -- start start 2024-04-22T13:24:07.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.663+0000 7fc146625640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc140102a30 0x7fc1401023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:07.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.663+0000 7fc146625640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc140105360 0x7fc1401009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:07.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.663+0000 7fc146625640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc140107c90 0x7fc140100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:07.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.663+0000 7fc146625640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc14010daf0 con 0x7fc140105360 2024-04-22T13:24:07.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.663+0000 7fc146625640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc14010d970 con 0x7fc140107c90 2024-04-22T13:24:07.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.663+0000 7fc146625640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc14010dc70 con 0x7fc140102a30 2024-04-22T13:24:07.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.663+0000 7fc13f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc140105360 0x7fc1401009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:07.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.663+0000 7fc13f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc140105360 0x7fc1401009e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34856/0 (socket says 172.21.15.12:34856) 2024-04-22T13:24:07.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.663+0000 7fc13f7fe640 1 -- 172.21.15.12:0/1518996181 learned_addr learned my addr 172.21.15.12:0/1518996181 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:07.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.664+0000 7fc144bc2640 1 --2- 172.21.15.12:0/1518996181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc140107c90 0x7fc140100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:07.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.664+0000 7fc13ffff640 1 --2- 172.21.15.12:0/1518996181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc140102a30 0x7fc1401023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:07.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.664+0000 7fc13f7fe640 1 -- 172.21.15.12:0/1518996181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc140102a30 msgr2=0x7fc1401023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:07.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.664+0000 7fc13f7fe640 1 --2- 172.21.15.12:0/1518996181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc140102a30 0x7fc1401023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:07.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.664+0000 7fc13f7fe640 1 -- 172.21.15.12:0/1518996181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc140107c90 msgr2=0x7fc140100f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:07.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.664+0000 7fc13f7fe640 1 --2- 172.21.15.12:0/1518996181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc140107c90 0x7fc140100f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:07.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.664+0000 7fc13f7fe640 1 -- 172.21.15.12:0/1518996181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1401017b0 con 0x7fc140105360 2024-04-22T13:24:07.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.664+0000 7fc13ffff640 1 --2- 172.21.15.12:0/1518996181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc140102a30 0x7fc1401023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:07.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.664+0000 7fc13f7fe640 1 --2- 172.21.15.12:0/1518996181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc140105360 0x7fc1401009e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc12800d770 tx=0x7fc12800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:07.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.664+0000 7fc13d7fa640 1 -- 172.21.15.12:0/1518996181 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc128017070 con 0x7fc140105360 2024-04-22T13:24:07.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.664+0000 7fc146625640 1 -- 172.21.15.12:0/1518996181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1401a22e0 con 0x7fc140105360 2024-04-22T13:24:07.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.664+0000 7fc13d7fa640 1 -- 172.21.15.12:0/1518996181 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc128009c30 con 0x7fc140105360 2024-04-22T13:24:07.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.664+0000 7fc13d7fa640 1 -- 172.21.15.12:0/1518996181 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12800f620 con 0x7fc140105360 2024-04-22T13:24:07.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.664+0000 7fc146625640 1 -- 172.21.15.12:0/1518996181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1401a2830 con 0x7fc140105360 2024-04-22T13:24:07.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.665+0000 7fc146625640 1 -- 172.21.15.12:0/1518996181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1400fdf30 con 0x7fc140105360 2024-04-22T13:24:07.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.666+0000 7fc13d7fa640 1 -- 172.21.15.12:0/1518996181 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fc12801a070 con 0x7fc140105360 2024-04-22T13:24:07.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.667+0000 7fc13d7fa640 1 --2- 172.21.15.12:0/1518996181 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc12c0a39b0 0x7fc12c0a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:07.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.667+0000 7fc13d7fa640 1 -- 172.21.15.12:0/1518996181 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fc1280cfbf0 con 0x7fc140105360 2024-04-22T13:24:07.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.667+0000 7fc13ffff640 1 --2- 172.21.15.12:0/1518996181 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc12c0a39b0 0x7fc12c0a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:07.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.668+0000 7fc13ffff640 1 --2- 172.21.15.12:0/1518996181 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc12c0a39b0 0x7fc12c0a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc130005fd0 tx=0x7fc130005ea0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:07.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.671+0000 7fc13d7fa640 1 -- 172.21.15.12:0/1518996181 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc128009de0 con 0x7fc140105360 2024-04-22T13:24:07.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:07 smithi012 ceph-mon[28265]: [22/Apr/2024:13:24:06] ENGINE Bus STARTING 2024-04-22T13:24:07.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:07 smithi012 ceph-mon[28265]: [22/Apr/2024:13:24:06] ENGINE Serving on http://172.21.15.12:8765 2024-04-22T13:24:07.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:07 smithi012 ceph-mon[28265]: mgrmap e22: x(active, since 1.0204s), standbys: z, y 2024-04-22T13:24:07.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:07 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2897407443' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:24:07.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.858+0000 7fc146625640 1 -- 172.21.15.12:0/1518996181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc14002d810 con 0x7fc140105360 2024-04-22T13:24:07.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.858+0000 7fc13d7fa640 1 -- 172.21.15.12:0/1518996181 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7fc12809c7b0 con 0x7fc140105360 2024-04-22T13:24:07.856 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-22T13:24:07.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.860+0000 7fc146625640 1 -- 172.21.15.12:0/1518996181 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc12c0a39b0 msgr2=0x7fc12c0a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:07.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.860+0000 7fc146625640 1 --2- 172.21.15.12:0/1518996181 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc12c0a39b0 0x7fc12c0a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc130005fd0 tx=0x7fc130005ea0 comp rx=0 tx=0).stop 2024-04-22T13:24:07.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.860+0000 7fc146625640 1 -- 172.21.15.12:0/1518996181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc140105360 msgr2=0x7fc1401009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:07.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.860+0000 7fc146625640 1 --2- 172.21.15.12:0/1518996181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc140105360 0x7fc1401009e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc12800d770 tx=0x7fc12800dc30 comp rx=0 tx=0).stop 2024-04-22T13:24:07.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.860+0000 7fc146625640 1 -- 172.21.15.12:0/1518996181 shutdown_connections 2024-04-22T13:24:07.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.860+0000 7fc146625640 1 --2- 172.21.15.12:0/1518996181 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc12c0a39b0 0x7fc12c0a5e70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:07.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.860+0000 7fc146625640 1 --2- 172.21.15.12:0/1518996181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc140107c90 0x7fc140100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:07.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.860+0000 7fc146625640 1 --2- 172.21.15.12:0/1518996181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc140105360 0x7fc1401009e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:07.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.860+0000 7fc146625640 1 --2- 172.21.15.12:0/1518996181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc140102a30 0x7fc1401023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:07.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.860+0000 7fc146625640 1 -- 172.21.15.12:0/1518996181 >> 172.21.15.12:0/1518996181 conn(0x7fc1400fc440 msgr2=0x7fc140103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:07.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.860+0000 7fc146625640 1 -- 172.21.15.12:0/1518996181 shutdown_connections 2024-04-22T13:24:07.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.860+0000 7fc146625640 1 -- 172.21.15.12:0/1518996181 wait complete. 2024-04-22T13:24:07.869 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T13:24:07.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:07 smithi134 ceph-mon[31338]: [22/Apr/2024:13:24:06] ENGINE Bus STARTING 2024-04-22T13:24:07.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:07 smithi134 ceph-mon[31338]: [22/Apr/2024:13:24:06] ENGINE Serving on http://172.21.15.12:8765 2024-04-22T13:24:07.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:07 smithi134 ceph-mon[31338]: mgrmap e22: x(active, since 1.0204s), standbys: z, y 2024-04-22T13:24:07.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:07 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2897407443' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:24:07.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:07 smithi111 ceph-mon[31361]: [22/Apr/2024:13:24:06] ENGINE Bus STARTING 2024-04-22T13:24:07.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:07 smithi111 ceph-mon[31361]: [22/Apr/2024:13:24:06] ENGINE Serving on http://172.21.15.12:8765 2024-04-22T13:24:07.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:07 smithi111 ceph-mon[31361]: mgrmap e22: x(active, since 1.0204s), standbys: z, y 2024-04-22T13:24:07.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:07 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2897407443' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:24:07.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.988+0000 7f12e1750640 1 -- 172.21.15.12:0/1078648204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12dc103b50 msgr2=0x7f12dc10a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:07.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.988+0000 7f12e1750640 1 --2- 172.21.15.12:0/1078648204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12dc103b50 0x7f12dc10a3e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f12d4009ca0 tx=0x7f12d402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:24:07.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.988+0000 7f12e1750640 1 -- 172.21.15.12:0/1078648204 shutdown_connections 2024-04-22T13:24:07.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.988+0000 7f12e1750640 1 --2- 172.21.15.12:0/1078648204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12dc103b50 0x7f12dc10a3e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:07.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.988+0000 7f12e1750640 1 --2- 172.21.15.12:0/1078648204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12dc103190 0x7f12dc103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:07.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.988+0000 7f12e1750640 1 --2- 172.21.15.12:0/1078648204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12dc101fa0 0x7f12dc1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:07.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.988+0000 7f12e1750640 1 -- 172.21.15.12:0/1078648204 >> 172.21.15.12:0/1078648204 conn(0x7f12dc0fd530 msgr2=0x7f12dc0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:07.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.988+0000 7f12e1750640 1 -- 172.21.15.12:0/1078648204 shutdown_connections 2024-04-22T13:24:07.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.988+0000 7f12e1750640 1 -- 172.21.15.12:0/1078648204 wait complete. 2024-04-22T13:24:07.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.989+0000 7f12e1750640 1 Processor -- start 2024-04-22T13:24:07.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.989+0000 7f12e1750640 1 -- start start 2024-04-22T13:24:07.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.989+0000 7f12e1750640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12dc101fa0 0x7f12dc198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:07.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.989+0000 7f12e1750640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12dc103190 0x7f12dc1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.989+0000 7f12e1750640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12dc103b50 0x7f12dc19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.989+0000 7f12e1750640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f12dc10dae0 con 0x7f12dc103190 2024-04-22T13:24:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.989+0000 7f12e1750640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f12dc10d960 con 0x7f12dc103b50 2024-04-22T13:24:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.989+0000 7f12e1750640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f12dc10dc60 con 0x7f12dc101fa0 2024-04-22T13:24:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.989+0000 7f12da7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12dc103190 0x7f12dc1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.989+0000 7f12da7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12dc103190 0x7f12dc1986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34882/0 (socket says 172.21.15.12:34882) 2024-04-22T13:24:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.989+0000 7f12da7fc640 1 -- 172.21.15.12:0/3069348292 learned_addr learned my addr 172.21.15.12:0/3069348292 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.989+0000 7f12db7fe640 1 --2- 172.21.15.12:0/3069348292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12dc103b50 0x7f12dc19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.989+0000 7f12daffd640 1 --2- 172.21.15.12:0/3069348292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12dc101fa0 0x7f12dc198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.990+0000 7f12da7fc640 1 -- 172.21.15.12:0/3069348292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12dc101fa0 msgr2=0x7f12dc198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.990+0000 7f12da7fc640 1 --2- 172.21.15.12:0/3069348292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12dc101fa0 0x7f12dc198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.990+0000 7f12da7fc640 1 -- 172.21.15.12:0/3069348292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12dc103b50 msgr2=0x7f12dc19f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.990+0000 7f12da7fc640 1 --2- 172.21.15.12:0/3069348292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12dc103b50 0x7f12dc19f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.990+0000 7f12da7fc640 1 -- 172.21.15.12:0/3069348292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12dc19fd90 con 0x7f12dc103190 2024-04-22T13:24:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.990+0000 7f12da7fc640 1 --2- 172.21.15.12:0/3069348292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12dc103190 0x7f12dc1986b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f12d000e9f0 tx=0x7f12d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.990+0000 7f12c3fff640 1 -- 172.21.15.12:0/3069348292 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d000cd80 con 0x7f12dc103190 2024-04-22T13:24:07.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.990+0000 7f12e1750640 1 -- 172.21.15.12:0/3069348292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12dc1a0080 con 0x7f12dc103190 2024-04-22T13:24:07.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.990+0000 7f12c3fff640 1 -- 172.21.15.12:0/3069348292 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12d00054d0 con 0x7f12dc103190 2024-04-22T13:24:07.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.990+0000 7f12c3fff640 1 -- 172.21.15.12:0/3069348292 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d0010690 con 0x7f12dc103190 2024-04-22T13:24:07.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.990+0000 7f12e1750640 1 -- 172.21.15.12:0/3069348292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12dc1a0580 con 0x7f12dc103190 2024-04-22T13:24:07.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.992+0000 7f12e1750640 1 -- 172.21.15.12:0/3069348292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12dc109390 con 0x7f12dc103190 2024-04-22T13:24:07.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.993+0000 7f12c3fff640 1 -- 172.21.15.12:0/3069348292 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f12d0004e30 con 0x7f12dc103190 2024-04-22T13:24:07.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.994+0000 7f12c3fff640 1 --2- 172.21.15.12:0/3069348292 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f12c40a39b0 0x7f12c40a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:07.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.994+0000 7f12c3fff640 1 -- 172.21.15.12:0/3069348292 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f12d00d0720 con 0x7f12dc103190 2024-04-22T13:24:07.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.994+0000 7f12daffd640 1 --2- 172.21.15.12:0/3069348292 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f12c40a39b0 0x7f12c40a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:07.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.994+0000 7f12daffd640 1 --2- 172.21.15.12:0/3069348292 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f12c40a39b0 0x7f12c40a5e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f12dc102ff0 tx=0x7f12cc005f90 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:07.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:07.997+0000 7f12c3fff640 1 -- 172.21.15.12:0/3069348292 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f12d0053ff0 con 0x7f12dc103190 2024-04-22T13:24:08.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.193+0000 7f12e1750640 1 -- 172.21.15.12:0/3069348292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f12dc02d810 con 0x7f12dc103190 2024-04-22T13:24:08.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.194+0000 7f12c3fff640 1 -- 172.21.15.12:0/3069348292 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f12d009d2e0 con 0x7f12dc103190 2024-04-22T13:24:08.191 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-22T13:24:08.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.197+0000 7f12e1750640 1 -- 172.21.15.12:0/3069348292 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f12c40a39b0 msgr2=0x7f12c40a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:08.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.197+0000 7f12e1750640 1 --2- 172.21.15.12:0/3069348292 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f12c40a39b0 0x7f12c40a5e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f12dc102ff0 tx=0x7f12cc005f90 comp rx=0 tx=0).stop 2024-04-22T13:24:08.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.197+0000 7f12e1750640 1 -- 172.21.15.12:0/3069348292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12dc103190 msgr2=0x7f12dc1986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:08.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.197+0000 7f12e1750640 1 --2- 172.21.15.12:0/3069348292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12dc103190 0x7f12dc1986b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f12d000e9f0 tx=0x7f12d000edf0 comp rx=0 tx=0).stop 2024-04-22T13:24:08.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.197+0000 7f12e1750640 1 -- 172.21.15.12:0/3069348292 shutdown_connections 2024-04-22T13:24:08.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.197+0000 7f12e1750640 1 --2- 172.21.15.12:0/3069348292 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f12c40a39b0 0x7f12c40a5e70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:08.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.197+0000 7f12e1750640 1 --2- 172.21.15.12:0/3069348292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f12dc103b50 0x7f12dc19f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:08.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.197+0000 7f12e1750640 1 --2- 172.21.15.12:0/3069348292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f12dc103190 0x7f12dc1986b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:08.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.197+0000 7f12e1750640 1 --2- 172.21.15.12:0/3069348292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12dc101fa0 0x7f12dc198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:08.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.197+0000 7f12e1750640 1 -- 172.21.15.12:0/3069348292 >> 172.21.15.12:0/3069348292 conn(0x7f12dc0fd530 msgr2=0x7f12dc106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:08.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.197+0000 7f12e1750640 1 -- 172.21.15.12:0/3069348292 shutdown_connections 2024-04-22T13:24:08.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.197+0000 7f12e1750640 1 -- 172.21.15.12:0/3069348292 wait complete. 2024-04-22T13:24:08.207 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T13:24:08.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.337+0000 7f1645b4b640 1 -- 172.21.15.12:0/752920810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1640107d40 msgr2=0x7f164010a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:08.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.337+0000 7f1645b4b640 1 --2- 172.21.15.12:0/752920810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1640107d40 0x7f164010a130 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1638009ca0 tx=0x7f163802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:24:08.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.337+0000 7f1645b4b640 1 -- 172.21.15.12:0/752920810 shutdown_connections 2024-04-22T13:24:08.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.337+0000 7f1645b4b640 1 --2- 172.21.15.12:0/752920810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1640107d40 0x7f164010a130 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:08.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.337+0000 7f1645b4b640 1 --2- 172.21.15.12:0/752920810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1640105410 0x7f1640107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:08.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.337+0000 7f1645b4b640 1 --2- 172.21.15.12:0/752920810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1640102ae0 0x7f1640104ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:08.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.337+0000 7f1645b4b640 1 -- 172.21.15.12:0/752920810 >> 172.21.15.12:0/752920810 conn(0x7f16400fc410 msgr2=0x7f16400fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:08.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.337+0000 7f1645b4b640 1 -- 172.21.15.12:0/752920810 shutdown_connections 2024-04-22T13:24:08.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.337+0000 7f1645b4b640 1 -- 172.21.15.12:0/752920810 wait complete. 2024-04-22T13:24:08.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.338+0000 7f1645b4b640 1 Processor -- start 2024-04-22T13:24:08.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.338+0000 7f1645b4b640 1 -- start start 2024-04-22T13:24:08.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.338+0000 7f1645b4b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1640102ae0 0x7f164019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:08.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.338+0000 7f1645b4b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1640105410 0x7f164019a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:08.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.338+0000 7f1645b4b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1640107d40 0x7f16401a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:08.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.338+0000 7f1645b4b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f164010daf0 con 0x7f1640105410 2024-04-22T13:24:08.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.338+0000 7f1645b4b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f164010d970 con 0x7f1640102ae0 2024-04-22T13:24:08.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.338+0000 7f1645b4b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f164010dc70 con 0x7f1640107d40 2024-04-22T13:24:08.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.339+0000 7f163effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1640105410 0x7f164019a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:08.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.339+0000 7f163effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1640105410 0x7f164019a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34924/0 (socket says 172.21.15.12:34924) 2024-04-22T13:24:08.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.339+0000 7f163effd640 1 -- 172.21.15.12:0/916862778 learned_addr learned my addr 172.21.15.12:0/916862778 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:08.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.339+0000 7f163f7fe640 1 --2- 172.21.15.12:0/916862778 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1640102ae0 0x7f164019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:08.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.339+0000 7f163ffff640 1 --2- 172.21.15.12:0/916862778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1640107d40 0x7f16401a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:08.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.339+0000 7f163effd640 1 -- 172.21.15.12:0/916862778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1640107d40 msgr2=0x7f16401a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:08.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.339+0000 7f163effd640 1 --2- 172.21.15.12:0/916862778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1640107d40 0x7f16401a19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:08.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.339+0000 7f163effd640 1 -- 172.21.15.12:0/916862778 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1640102ae0 msgr2=0x7f164019a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:08.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.339+0000 7f163effd640 1 --2- 172.21.15.12:0/916862778 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1640102ae0 0x7f164019a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:08.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.339+0000 7f163effd640 1 -- 172.21.15.12:0/916862778 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16401a2060 con 0x7f1640105410 2024-04-22T13:24:08.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.339+0000 7f163effd640 1 --2- 172.21.15.12:0/916862778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1640105410 0x7f164019a980 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f163400d6f0 tx=0x7f163400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:08.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.339+0000 7f163cff9640 1 -- 172.21.15.12:0/916862778 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634017070 con 0x7f1640105410 2024-04-22T13:24:08.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.339+0000 7f1645b4b640 1 -- 172.21.15.12:0/916862778 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16401a2350 con 0x7f1640105410 2024-04-22T13:24:08.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.339+0000 7f163cff9640 1 -- 172.21.15.12:0/916862778 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1634010460 con 0x7f1640105410 2024-04-22T13:24:08.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.339+0000 7f163cff9640 1 -- 172.21.15.12:0/916862778 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163400f620 con 0x7f1640105410 2024-04-22T13:24:08.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.339+0000 7f1645b4b640 1 -- 172.21.15.12:0/916862778 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16401a28f0 con 0x7f1640105410 2024-04-22T13:24:08.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.341+0000 7f1645b4b640 1 -- 172.21.15.12:0/916862778 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1640109830 con 0x7f1640105410 2024-04-22T13:24:08.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.342+0000 7f163cff9640 1 -- 172.21.15.12:0/916862778 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f163401a070 con 0x7f1640105410 2024-04-22T13:24:08.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.343+0000 7f163cff9640 1 --2- 172.21.15.12:0/916862778 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f16280a39f0 0x7f16280a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:08.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.343+0000 7f163cff9640 1 -- 172.21.15.12:0/916862778 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f16340cfa50 con 0x7f1640105410 2024-04-22T13:24:08.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.343+0000 7f163f7fe640 1 --2- 172.21.15.12:0/916862778 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f16280a39f0 0x7f16280a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:08.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.343+0000 7f163f7fe640 1 --2- 172.21.15.12:0/916862778 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f16280a39f0 0x7f16280a5eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1630009bd0 tx=0x7f1630005f90 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:08.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.347+0000 7f163cff9640 1 -- 172.21.15.12:0/916862778 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f163409c000 con 0x7f1640105410 2024-04-22T13:24:08.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.538+0000 7f1645b4b640 1 -- 172.21.15.12:0/916862778 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1640062b50 con 0x7f1640105410 2024-04-22T13:24:08.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.538+0000 7f163cff9640 1 -- 172.21.15.12:0/916862778 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f16340a0300 con 0x7f1640105410 2024-04-22T13:24:08.535 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-22T13:24:08.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.540+0000 7f1645b4b640 1 -- 172.21.15.12:0/916862778 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f16280a39f0 msgr2=0x7f16280a5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:08.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.540+0000 7f1645b4b640 1 --2- 172.21.15.12:0/916862778 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f16280a39f0 0x7f16280a5eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1630009bd0 tx=0x7f1630005f90 comp rx=0 tx=0).stop 2024-04-22T13:24:08.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.540+0000 7f1645b4b640 1 -- 172.21.15.12:0/916862778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1640105410 msgr2=0x7f164019a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:08.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.540+0000 7f1645b4b640 1 --2- 172.21.15.12:0/916862778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1640105410 0x7f164019a980 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f163400d6f0 tx=0x7f163400dbb0 comp rx=0 tx=0).stop 2024-04-22T13:24:08.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.540+0000 7f1645b4b640 1 -- 172.21.15.12:0/916862778 shutdown_connections 2024-04-22T13:24:08.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.540+0000 7f1645b4b640 1 --2- 172.21.15.12:0/916862778 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f16280a39f0 0x7f16280a5eb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:08.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.540+0000 7f1645b4b640 1 --2- 172.21.15.12:0/916862778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1640107d40 0x7f16401a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:08.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.540+0000 7f1645b4b640 1 --2- 172.21.15.12:0/916862778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1640105410 0x7f164019a980 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:08.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.540+0000 7f1645b4b640 1 --2- 172.21.15.12:0/916862778 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1640102ae0 0x7f164019a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:08.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.540+0000 7f1645b4b640 1 -- 172.21.15.12:0/916862778 >> 172.21.15.12:0/916862778 conn(0x7f16400fc410 msgr2=0x7f16401022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:08.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.541+0000 7f1645b4b640 1 -- 172.21.15.12:0/916862778 shutdown_connections 2024-04-22T13:24:08.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.541+0000 7f1645b4b640 1 -- 172.21.15.12:0/916862778 wait complete. 2024-04-22T13:24:08.549 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:24:08.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.655+0000 7fa7f5e6d640 1 -- 172.21.15.12:0/898544279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7f0101fc0 msgr2=0x7fa7f0102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:08.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.655+0000 7fa7f5e6d640 1 --2- 172.21.15.12:0/898544279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7f0101fc0 0x7fa7f0102440 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4009900 tx=0x7fa7e402bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:08.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.655+0000 7fa7f5e6d640 1 -- 172.21.15.12:0/898544279 shutdown_connections 2024-04-22T13:24:08.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.655+0000 7fa7f5e6d640 1 --2- 172.21.15.12:0/898544279 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7f0102980 0x7fa7f01092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:08.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.655+0000 7fa7f5e6d640 1 --2- 172.21.15.12:0/898544279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7f0101fc0 0x7fa7f0102440 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:08.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.655+0000 7fa7f5e6d640 1 --2- 172.21.15.12:0/898544279 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7f0100e70 0x7fa7f0101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:08.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.655+0000 7fa7f5e6d640 1 -- 172.21.15.12:0/898544279 >> 172.21.15.12:0/898544279 conn(0x7fa7f00fc3e0 msgr2=0x7fa7f00fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:08.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.655+0000 7fa7f5e6d640 1 -- 172.21.15.12:0/898544279 shutdown_connections 2024-04-22T13:24:08.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.655+0000 7fa7f5e6d640 1 -- 172.21.15.12:0/898544279 wait complete. 2024-04-22T13:24:08.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.655+0000 7fa7f5e6d640 1 Processor -- start 2024-04-22T13:24:08.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.655+0000 7fa7f5e6d640 1 -- start start 2024-04-22T13:24:08.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.656+0000 7fa7f5e6d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7f0100e70 0x7fa7f0198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:08.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.656+0000 7fa7f5e6d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7f0101fc0 0x7fa7f0198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:08.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.656+0000 7fa7f5e6d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7f0102980 0x7fa7f019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:08.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.656+0000 7fa7f5e6d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f010ca20 con 0x7fa7f0102980 2024-04-22T13:24:08.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.656+0000 7fa7f5e6d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f010c8a0 con 0x7fa7f0101fc0 2024-04-22T13:24:08.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.656+0000 7fa7f5e6d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f010cba0 con 0x7fa7f0100e70 2024-04-22T13:24:08.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.656+0000 7fa7effff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7f0102980 0x7fa7f019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:08.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.656+0000 7fa7eeffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7f0101fc0 0x7fa7f0198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:08.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.656+0000 7fa7effff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7f0102980 0x7fa7f019f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34942/0 (socket says 172.21.15.12:34942) 2024-04-22T13:24:08.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.656+0000 7fa7effff640 1 -- 172.21.15.12:0/4094535981 learned_addr learned my addr 172.21.15.12:0/4094535981 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:08.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.656+0000 7fa7ef7fe640 1 --2- 172.21.15.12:0/4094535981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7f0100e70 0x7fa7f0198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:08.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.656+0000 7fa7effff640 1 -- 172.21.15.12:0/4094535981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7f0100e70 msgr2=0x7fa7f0198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:08.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.656+0000 7fa7effff640 1 --2- 172.21.15.12:0/4094535981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7f0100e70 0x7fa7f0198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:08.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.656+0000 7fa7effff640 1 -- 172.21.15.12:0/4094535981 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7f0101fc0 msgr2=0x7fa7f0198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:08.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.656+0000 7fa7effff640 1 --2- 172.21.15.12:0/4094535981 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7f0101fc0 0x7fa7f0198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:08.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.656+0000 7fa7effff640 1 -- 172.21.15.12:0/4094535981 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f019fe20 con 0x7fa7f0102980 2024-04-22T13:24:08.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.656+0000 7fa7eeffd640 1 --2- 172.21.15.12:0/4094535981 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7f0101fc0 0x7fa7f0198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:24:08.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.656+0000 7fa7effff640 1 --2- 172.21.15.12:0/4094535981 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7f0102980 0x7fa7f019f7b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800eb10 tx=0x7fa7e800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:08.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.656+0000 7fa7ecff9640 1 -- 172.21.15.12:0/4094535981 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e800ce20 con 0x7fa7f0102980 2024-04-22T13:24:08.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.656+0000 7fa7f5e6d640 1 -- 172.21.15.12:0/4094535981 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7f01a0110 con 0x7fa7f0102980 2024-04-22T13:24:08.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.656+0000 7fa7ecff9640 1 -- 172.21.15.12:0/4094535981 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7e8005020 con 0x7fa7f0102980 2024-04-22T13:24:08.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.657+0000 7fa7f5e6d640 1 -- 172.21.15.12:0/4094535981 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7f01a06b0 con 0x7fa7f0102980 2024-04-22T13:24:08.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.657+0000 7fa7ecff9640 1 -- 172.21.15.12:0/4094535981 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8010690 con 0x7fa7f0102980 2024-04-22T13:24:08.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.657+0000 7fa7f5e6d640 1 -- 172.21.15.12:0/4094535981 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7f0068de0 con 0x7fa7f0102980 2024-04-22T13:24:08.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.659+0000 7fa7ecff9640 1 -- 172.21.15.12:0/4094535981 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fa7e8005510 con 0x7fa7f0102980 2024-04-22T13:24:08.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.660+0000 7fa7ecff9640 1 --2- 172.21.15.12:0/4094535981 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa7d80a39b0 0x7fa7d80a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:08.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.660+0000 7fa7ef7fe640 1 --2- 172.21.15.12:0/4094535981 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa7d80a39b0 0x7fa7d80a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:08.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.660+0000 7fa7ecff9640 1 -- 172.21.15.12:0/4094535981 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fa7e80d0830 con 0x7fa7f0102980 2024-04-22T13:24:08.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.660+0000 7fa7ef7fe640 1 --2- 172.21.15.12:0/4094535981 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa7d80a39b0 0x7fa7d80a5e70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0009990 tx=0x7fa7e0009110 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:08.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.664+0000 7fa7ecff9640 1 -- 172.21.15.12:0/4094535981 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa7e8053ff0 con 0x7fa7f0102980 2024-04-22T13:24:08.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.693+0000 7fa7ecff9640 1 -- 172.21.15.12:0/4094535981 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7fa7e80993a0 con 0x7fa7f0102980 2024-04-22T13:24:08.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:08 smithi012 ceph-mon[28265]: [22/Apr/2024:13:24:06] ENGINE Serving on https://172.21.15.12:7150 2024-04-22T13:24:08.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:08 smithi012 ceph-mon[28265]: [22/Apr/2024:13:24:06] ENGINE Bus STARTED 2024-04-22T13:24:08.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:08 smithi012 ceph-mon[28265]: from='client.25001 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:24:08.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:08 smithi012 ceph-mon[28265]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:08.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:08 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1518996181' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:24:08.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:08 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3069348292' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:24:08.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:08 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:08.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:08 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:08.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:08 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:08.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:08 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:08.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:08 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:08.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:08 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:08.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:08 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/916862778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:24:08.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.866+0000 7fa7f5e6d640 1 -- 172.21.15.12:0/4094535981 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7f01a09d0 con 0x7fa7f0102980 2024-04-22T13:24:08.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.869+0000 7fa7ecff9640 1 -- 172.21.15.12:0/4094535981 <== mon.0 v2:172.21.15.12:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22997 (secure 0 0 0) 0x7fa7e80a38d0 con 0x7fa7f0102980 2024-04-22T13:24:08.866 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:24:08.867 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":90,"fsid":"0aede5ea-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:14:09.362918+0000","modified":"2024-04-22T13:24:06.155595+0000","last_up_change":"2024-04-22T13:21:15.263393+0000","last_in_change":"2024-04-22T13:20:54.174685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:17:32.372434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:22:02.768799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:22:03.578730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d9e2365-3a64-4b13-96fc-c217b1cee2ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6803","nonce":1191350632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6805","nonce":1191350632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6809","nonce":1191350632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6807","nonce":1191350632}]},"public_addr":"172.21.15.12:6803/1191350632","cluster_addr":"172.21.15.12:6805/1191350632","heartbeat_back_addr":"172.21.15.12:6809/1191350632","heartbeat_front_addr":"172.21.15.12:6807/1191350632","state":["exists","up"]},{"osd":1,"uuid":"1499c783-2bd7-48ce-a444-34add42bcf31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6801","nonce":417496465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6803","nonce":417496465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6807","nonce":417496465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6805","nonce":417496465}]},"public_addr":"172.21.15.111:6801/417496465","cluster_addr":"172.21.15.111:6803/417496465","heartbeat_back_addr":"172.21.15.111:6807/417496465","heartbeat_front_addr":"172.21.15.111:6805/417496465","state":["exists","up"]},{"osd":2,"uuid":"71a9ebc3-c622-434e-a6c2-f48358f384f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6801","nonce":1403153998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6803","nonce":1403153998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6807","nonce":1403153998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6805","nonce":1403153998}]},"public_addr":"172.21.15.134:6801/1403153998","cluster_addr":"172.21.15.134:6803/1403153998","heartbeat_back_addr":"172.21.15.134:6807/1403153998","heartbeat_front_addr":"172.21.15.134:6805/1403153998","state":["exists","up"]},{"osd":3,"uuid":"6dd00db2-a286-48ba-8666-56ffdd5f35ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6811","nonce":3359066681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6813","nonce":3359066681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6817","nonce":3359066681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6815","nonce":3359066681}]},"public_addr":"172.21.15.12:6811/3359066681","cluster_addr":"172.21.15.12:6813/3359066681","heartbeat_back_addr":"172.21.15.12:6817/3359066681","heartbeat_front_addr":"172.21.15.12:6815/3359066681","state":["exists","up"]},{"osd":4,"uuid":"1e29afd7-354c-4506-aaa0-b43eb1a19a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6809","nonce":2886349872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6811","nonce":2886349872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6815","nonce":2886349872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6813","nonce":2886349872}]},"public_addr":"172.21.15.111:6809/2886349872","cluster_addr":"172.21.15.111:6811/2886349872","heartbeat_back_addr":"172.21.15.111:6815/2886349872","heartbeat_front_addr":"172.21.15.111:6813/2886349872","state":["exists","up"]},{"osd":5,"uuid":"8f40f02d-0ba2-4842-a03d-2a1a04d1010d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6809","nonce":3071977602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6811","nonce":3071977602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6815","nonce":3071977602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6813","nonce":3071977602}]},"public_addr":"172.21.15.134:6809/3071977602","cluster_addr":"172.21.15.134:6811/3071977602","heartbeat_back_addr":"172.21.15.134:6815/3071977602","heartbeat_front_addr":"172.21.15.134:6813/3071977602","state":["exists","up"]},{"osd":6,"uuid":"bb0c17a9-6a8e-4be6-8660-f73cfa8200ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6819","nonce":3819619299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6821","nonce":3819619299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6825","nonce":3819619299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6823","nonce":3819619299}]},"public_addr":"172.21.15.12:6819/3819619299","cluster_addr":"172.21.15.12:6821/3819619299","heartbeat_back_addr":"172.21.15.12:6825/3819619299","heartbeat_front_addr":"172.21.15.12:6823/3819619299","state":["exists","up"]},{"osd":7,"uuid":"0a93d6af-569e-4e45-b697-9e978c1c1004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6817","nonce":3113856282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6819","nonce":3113856282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6823","nonce":3113856282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6821","nonce":3113856282}]},"public_addr":"172.21.15.111:6817/3113856282","cluster_addr":"172.21.15.111:6819/3113856282","heartbeat_back_addr":"172.21.15.111:6823/3113856282","heartbeat_front_addr":"172.21.15.111:6821/3113856282","state":["exists","up"]},{"osd":8,"uuid":"e21acc68-cf50-4f4f-b1ba-cf6c195b4955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6817","nonce":1689362191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6819","nonce":1689362191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6823","nonce":1689362191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6821","nonce":1689362191}]},"public_addr":"172.21.15.134:6817/1689362191","cluster_addr":"172.21.15.134:6819/1689362191","heartbeat_back_addr":"172.21.15.134:6823/1689362191","heartbeat_front_addr":"172.21.15.134:6821/1689362191","state":["exists","up"]},{"osd":9,"uuid":"fd745869-7f5f-48a8-b412-bb9c92b586e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6827","nonce":4292385779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6829","nonce":4292385779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6833","nonce":4292385779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6831","nonce":4292385779}]},"public_addr":"172.21.15.12:6827/4292385779","cluster_addr":"172.21.15.12:6829/4292385779","heartbeat_back_addr":"172.21.15.12:6833/4292385779","heartbeat_front_addr":"172.21.15.12:6831/4292385779","state":["exists","up"]},{"osd":10,"uuid":"798b1abd-f6c7-4a5f-8da0-fdae49e50d2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6825","nonce":3338566976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6827","nonce":3338566976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6831","nonce":3338566976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6829","nonce":3338566976}]},"public_addr":"172.21.15.111:6825/3338566976","cluster_addr":"172.21.15.111:6827/3338566976","heartbeat_back_addr":"172.21.15.111:6831/3338566976","heartbeat_front_addr":"172.21.15.111:6829/3338566976","state":["exists","up"]},{"osd":11,"uuid":"b18fc5f3-1049-4871-b060-e99290273cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6825","nonce":578640428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6827","nonce":578640428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6831","nonce":578640428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6829","nonce":578640428}]},"public_addr":"172.21.15.134:6825/578640428","cluster_addr":"172.21.15.134:6827/578640428","heartbeat_back_addr":"172.21.15.134:6831/578640428","heartbeat_front_addr":"172.21.15.134:6829/578640428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:16:40.828608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:05.052528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:28.297316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:53.496497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:17.947259+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:41.712894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:07.105945+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:32.129688+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:56.008516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:22.239867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:47.334692+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:12.301276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/3535299028":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/2837479598":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/3010129696":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/2593601362":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/2519382069":"2024-04-23T13:15:04.279990+0000","172.21.15.12:6801/2009831940":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/856202072":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/3509412187":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/1802873236":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/1007749062":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3588480095":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1904962176":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/1802873236":"2024-04-23T13:24:06.155564+0000","172.21.15.12:6800/2009831940":"2024-04-23T13:15:04.279990+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:24:08.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.871+0000 7fa7f5e6d640 1 -- 172.21.15.12:0/4094535981 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa7d80a39b0 msgr2=0x7fa7d80a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:08.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.871+0000 7fa7f5e6d640 1 --2- 172.21.15.12:0/4094535981 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa7d80a39b0 0x7fa7d80a5e70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0009990 tx=0x7fa7e0009110 comp rx=0 tx=0).stop 2024-04-22T13:24:08.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.871+0000 7fa7f5e6d640 1 -- 172.21.15.12:0/4094535981 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7f0102980 msgr2=0x7fa7f019f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:08.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.871+0000 7fa7f5e6d640 1 --2- 172.21.15.12:0/4094535981 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7f0102980 0x7fa7f019f7b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800eb10 tx=0x7fa7e800ef10 comp rx=0 tx=0).stop 2024-04-22T13:24:08.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.871+0000 7fa7f5e6d640 1 -- 172.21.15.12:0/4094535981 shutdown_connections 2024-04-22T13:24:08.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.871+0000 7fa7f5e6d640 1 --2- 172.21.15.12:0/4094535981 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa7d80a39b0 0x7fa7d80a5e70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:08.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.871+0000 7fa7f5e6d640 1 --2- 172.21.15.12:0/4094535981 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa7f0102980 0x7fa7f019f7b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:08.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.871+0000 7fa7f5e6d640 1 --2- 172.21.15.12:0/4094535981 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7f0101fc0 0x7fa7f0198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:08.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.871+0000 7fa7f5e6d640 1 --2- 172.21.15.12:0/4094535981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7f0100e70 0x7fa7f0198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:08.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.871+0000 7fa7f5e6d640 1 -- 172.21.15.12:0/4094535981 >> 172.21.15.12:0/4094535981 conn(0x7fa7f00fc3e0 msgr2=0x7fa7f00fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:08.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.871+0000 7fa7f5e6d640 1 -- 172.21.15.12:0/4094535981 shutdown_connections 2024-04-22T13:24:08.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.871+0000 7fa7f5e6d640 1 -- 172.21.15.12:0/4094535981 wait complete. 2024-04-22T13:24:08.881 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T13:17:32.372434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T13:22:02.768799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T13:22:03.578730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T13:24:08.881 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T13:24:08.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:08 smithi134 ceph-mon[31338]: [22/Apr/2024:13:24:06] ENGINE Serving on https://172.21.15.12:7150 2024-04-22T13:24:08.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:08 smithi134 ceph-mon[31338]: [22/Apr/2024:13:24:06] ENGINE Bus STARTED 2024-04-22T13:24:08.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:08 smithi134 ceph-mon[31338]: from='client.25001 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:24:08.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:08 smithi134 ceph-mon[31338]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:08.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:08 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1518996181' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:24:08.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:08 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/3069348292' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:24:08.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:08 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:08.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:08 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:08.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:08 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:08.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:08 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:08.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:08 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:08.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:08 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:08.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:08 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/916862778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:24:08.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:08 smithi111 ceph-mon[31361]: [22/Apr/2024:13:24:06] ENGINE Serving on https://172.21.15.12:7150 2024-04-22T13:24:08.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:08 smithi111 ceph-mon[31361]: [22/Apr/2024:13:24:06] ENGINE Bus STARTED 2024-04-22T13:24:08.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:08 smithi111 ceph-mon[31361]: from='client.25001 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:24:08.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:08 smithi111 ceph-mon[31361]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:08.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:08 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1518996181' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:24:08.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:08 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/3069348292' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:24:08.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:08 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:08.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:08 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:08.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:08 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:08.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:08 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:08.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:08 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:08.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:08 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:08.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:08 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/916862778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:24:08.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.993+0000 7ff74d350640 1 -- 172.21.15.12:0/3227806800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff748100e90 msgr2=0x7ff7481012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:08.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.993+0000 7ff74d350640 1 --2- 172.21.15.12:0/3227806800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff748100e90 0x7ff7481012b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff740009ca0 tx=0x7ff74002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:24:08.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.993+0000 7ff74d350640 1 -- 172.21.15.12:0/3227806800 shutdown_connections 2024-04-22T13:24:08.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.993+0000 7ff74d350640 1 --2- 172.21.15.12:0/3227806800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7481029a0 0x7ff7481092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:08.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.993+0000 7ff74d350640 1 --2- 172.21.15.12:0/3227806800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff748101fe0 0x7ff748102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:08.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.993+0000 7ff74d350640 1 --2- 172.21.15.12:0/3227806800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff748100e90 0x7ff7481012b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:08.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.993+0000 7ff74d350640 1 -- 172.21.15.12:0/3227806800 >> 172.21.15.12:0/3227806800 conn(0x7ff7480fc400 msgr2=0x7ff7480fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:08.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.993+0000 7ff74d350640 1 -- 172.21.15.12:0/3227806800 shutdown_connections 2024-04-22T13:24:08.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.993+0000 7ff74d350640 1 -- 172.21.15.12:0/3227806800 wait complete. 2024-04-22T13:24:08.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.994+0000 7ff74d350640 1 Processor -- start 2024-04-22T13:24:08.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.994+0000 7ff74d350640 1 -- start start 2024-04-22T13:24:08.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.994+0000 7ff74d350640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff748100e90 0x7ff748197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:08.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.994+0000 7ff74d350640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff748101fe0 0x7ff7481976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:08.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.994+0000 7ff74d350640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7481029a0 0x7ff74819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:08.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.994+0000 7ff74d350640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff74810c9e0 con 0x7ff7481029a0 2024-04-22T13:24:08.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.994+0000 7ff74d350640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff74810c860 con 0x7ff748100e90 2024-04-22T13:24:08.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.994+0000 7ff74d350640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff74810cb60 con 0x7ff748101fe0 2024-04-22T13:24:08.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.994+0000 7ff7477fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7481029a0 0x7ff74819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:08.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.994+0000 7ff7477fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7481029a0 0x7ff74819e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34956/0 (socket says 172.21.15.12:34956) 2024-04-22T13:24:08.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.994+0000 7ff7477fe640 1 -- 172.21.15.12:0/3442454521 learned_addr learned my addr 172.21.15.12:0/3442454521 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:08.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.994+0000 7ff746ffd640 1 --2- 172.21.15.12:0/3442454521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff748100e90 0x7ff748197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:08.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.995+0000 7ff7467fc640 1 --2- 172.21.15.12:0/3442454521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff748101fe0 0x7ff7481976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:08.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.995+0000 7ff7477fe640 1 -- 172.21.15.12:0/3442454521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff748101fe0 msgr2=0x7ff7481976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:08.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.995+0000 7ff7477fe640 1 --2- 172.21.15.12:0/3442454521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff748101fe0 0x7ff7481976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:08.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.995+0000 7ff7477fe640 1 -- 172.21.15.12:0/3442454521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff748100e90 msgr2=0x7ff748197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:08.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.995+0000 7ff7477fe640 1 --2- 172.21.15.12:0/3442454521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff748100e90 0x7ff748197160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:08.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.995+0000 7ff7477fe640 1 -- 172.21.15.12:0/3442454521 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74819ed80 con 0x7ff7481029a0 2024-04-22T13:24:08.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.995+0000 7ff746ffd640 1 --2- 172.21.15.12:0/3442454521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff748100e90 0x7ff748197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:24:08.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.995+0000 7ff7477fe640 1 --2- 172.21.15.12:0/3442454521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7481029a0 0x7ff74819e710 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00e9b0 tx=0x7ff73c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:08.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.995+0000 7ff72ffff640 1 -- 172.21.15.12:0/3442454521 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c00cd80 con 0x7ff7481029a0 2024-04-22T13:24:08.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.995+0000 7ff72ffff640 1 -- 172.21.15.12:0/3442454521 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff73c0035f0 con 0x7ff7481029a0 2024-04-22T13:24:08.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.995+0000 7ff74d350640 1 -- 172.21.15.12:0/3442454521 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff74819f070 con 0x7ff7481029a0 2024-04-22T13:24:08.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.995+0000 7ff72ffff640 1 -- 172.21.15.12:0/3442454521 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c010690 con 0x7ff7481029a0 2024-04-22T13:24:08.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.995+0000 7ff74d350640 1 -- 172.21.15.12:0/3442454521 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff74819f570 con 0x7ff7481029a0 2024-04-22T13:24:08.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.996+0000 7ff74d350640 1 -- 172.21.15.12:0/3442454521 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7481082c0 con 0x7ff7481029a0 2024-04-22T13:24:08.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.997+0000 7ff72ffff640 1 -- 172.21.15.12:0/3442454521 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7ff73c004020 con 0x7ff7481029a0 2024-04-22T13:24:08.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.998+0000 7ff72ffff640 1 --2- 172.21.15.12:0/3442454521 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff7300a3db0 0x7ff7300a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:08.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.998+0000 7ff746ffd640 1 --2- 172.21.15.12:0/3442454521 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff7300a3db0 0x7ff7300a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:08.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.998+0000 7ff72ffff640 1 -- 172.21.15.12:0/3442454521 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7ff73c0d0c30 con 0x7ff7481029a0 2024-04-22T13:24:08.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:08.999+0000 7ff746ffd640 1 --2- 172.21.15.12:0/3442454521 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff7300a3db0 0x7ff7300a6270 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff740005e30 tx=0x7ff740000f40 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:08.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.002+0000 7ff72ffff640 1 -- 172.21.15.12:0/3442454521 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff73c09d1b0 con 0x7ff7481029a0 2024-04-22T13:24:09.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.210+0000 7ff74d350640 1 -- 172.21.15.12:0/3442454521 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff74802d770 con 0x7ff7481029a0 2024-04-22T13:24:09.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.210+0000 7ff72ffff640 1 -- 172.21.15.12:0/3442454521 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7ff73c09cab0 con 0x7ff7481029a0 2024-04-22T13:24:09.207 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-22T13:24:09.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.212+0000 7ff74d350640 1 -- 172.21.15.12:0/3442454521 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff7300a3db0 msgr2=0x7ff7300a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:09.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.212+0000 7ff74d350640 1 --2- 172.21.15.12:0/3442454521 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff7300a3db0 0x7ff7300a6270 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff740005e30 tx=0x7ff740000f40 comp rx=0 tx=0).stop 2024-04-22T13:24:09.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.212+0000 7ff74d350640 1 -- 172.21.15.12:0/3442454521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7481029a0 msgr2=0x7ff74819e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:09.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.212+0000 7ff74d350640 1 --2- 172.21.15.12:0/3442454521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7481029a0 0x7ff74819e710 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00e9b0 tx=0x7ff73c00ee70 comp rx=0 tx=0).stop 2024-04-22T13:24:09.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.212+0000 7ff74d350640 1 -- 172.21.15.12:0/3442454521 shutdown_connections 2024-04-22T13:24:09.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.212+0000 7ff74d350640 1 --2- 172.21.15.12:0/3442454521 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff7300a3db0 0x7ff7300a6270 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:09.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.212+0000 7ff74d350640 1 --2- 172.21.15.12:0/3442454521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7481029a0 0x7ff74819e710 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:09.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.212+0000 7ff74d350640 1 --2- 172.21.15.12:0/3442454521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff748101fe0 0x7ff7481976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:09.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.212+0000 7ff74d350640 1 --2- 172.21.15.12:0/3442454521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff748100e90 0x7ff748197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:09.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.212+0000 7ff74d350640 1 -- 172.21.15.12:0/3442454521 >> 172.21.15.12:0/3442454521 conn(0x7ff7480fc400 msgr2=0x7ff748105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:09.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.212+0000 7ff74d350640 1 -- 172.21.15.12:0/3442454521 shutdown_connections 2024-04-22T13:24:09.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.212+0000 7ff74d350640 1 -- 172.21.15.12:0/3442454521 wait complete. 2024-04-22T13:24:09.221 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T13:24:09.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.330+0000 7fbf3c848640 1 -- 172.21.15.12:0/3935836068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf381029a0 msgr2=0x7fbf381092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:09.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.330+0000 7fbf3c848640 1 --2- 172.21.15.12:0/3935836068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf381029a0 0x7fbf381092c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009ca0 tx=0x7fbf3002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:24:09.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.330+0000 7fbf3c848640 1 -- 172.21.15.12:0/3935836068 shutdown_connections 2024-04-22T13:24:09.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.330+0000 7fbf3c848640 1 --2- 172.21.15.12:0/3935836068 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf381029a0 0x7fbf381092c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:09.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.330+0000 7fbf3c848640 1 --2- 172.21.15.12:0/3935836068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf38101fe0 0x7fbf38102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:09.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.330+0000 7fbf3c848640 1 --2- 172.21.15.12:0/3935836068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf38100e90 0x7fbf381012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:09.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.330+0000 7fbf3c848640 1 -- 172.21.15.12:0/3935836068 >> 172.21.15.12:0/3935836068 conn(0x7fbf380fc400 msgr2=0x7fbf380fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:09.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.330+0000 7fbf3c848640 1 -- 172.21.15.12:0/3935836068 shutdown_connections 2024-04-22T13:24:09.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.330+0000 7fbf3c848640 1 -- 172.21.15.12:0/3935836068 wait complete. 2024-04-22T13:24:09.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.331+0000 7fbf3c848640 1 Processor -- start 2024-04-22T13:24:09.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.331+0000 7fbf3c848640 1 -- start start 2024-04-22T13:24:09.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.331+0000 7fbf3c848640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf38100e90 0x7fbf381970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:09.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.331+0000 7fbf3c848640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf38101fe0 0x7fbf38197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:09.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.331+0000 7fbf3c848640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf381029a0 0x7fbf3819e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:09.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.331+0000 7fbf3c848640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3810c9e0 con 0x7fbf38100e90 2024-04-22T13:24:09.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.331+0000 7fbf3c848640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3810c860 con 0x7fbf381029a0 2024-04-22T13:24:09.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.331+0000 7fbf3c848640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbf3810cb60 con 0x7fbf38101fe0 2024-04-22T13:24:09.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.331+0000 7fbf3659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf38100e90 0x7fbf381970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:09.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.331+0000 7fbf3659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf38100e90 0x7fbf381970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34978/0 (socket says 172.21.15.12:34978) 2024-04-22T13:24:09.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.331+0000 7fbf3659c640 1 -- 172.21.15.12:0/2101818389 learned_addr learned my addr 172.21.15.12:0/2101818389 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:09.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.331+0000 7fbf36d9d640 1 --2- 172.21.15.12:0/2101818389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf381029a0 0x7fbf3819e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:09.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.331+0000 7fbf35d9b640 1 --2- 172.21.15.12:0/2101818389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf38101fe0 0x7fbf38197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:09.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.331+0000 7fbf3659c640 1 -- 172.21.15.12:0/2101818389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf38101fe0 msgr2=0x7fbf38197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:09.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.331+0000 7fbf3659c640 1 --2- 172.21.15.12:0/2101818389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf38101fe0 0x7fbf38197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:09.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.331+0000 7fbf3659c640 1 -- 172.21.15.12:0/2101818389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf381029a0 msgr2=0x7fbf3819e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:09.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.331+0000 7fbf3659c640 1 --2- 172.21.15.12:0/2101818389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf381029a0 0x7fbf3819e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:09.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.331+0000 7fbf3659c640 1 -- 172.21.15.12:0/2101818389 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf3819ed00 con 0x7fbf38100e90 2024-04-22T13:24:09.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.331+0000 7fbf36d9d640 1 --2- 172.21.15.12:0/2101818389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf381029a0 0x7fbf3819e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:24:09.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.332+0000 7fbf3659c640 1 --2- 172.21.15.12:0/2101818389 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf38100e90 0x7fbf381970e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c00e9f0 tx=0x7fbf2c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:09.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.332+0000 7fbf277fe640 1 -- 172.21.15.12:0/2101818389 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c00cd80 con 0x7fbf38100e90 2024-04-22T13:24:09.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.332+0000 7fbf277fe640 1 -- 172.21.15.12:0/2101818389 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf2c0108e0 con 0x7fbf38100e90 2024-04-22T13:24:09.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.332+0000 7fbf3c848640 1 -- 172.21.15.12:0/2101818389 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf3819eff0 con 0x7fbf38100e90 2024-04-22T13:24:09.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.332+0000 7fbf277fe640 1 -- 172.21.15.12:0/2101818389 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c018980 con 0x7fbf38100e90 2024-04-22T13:24:09.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.332+0000 7fbf3c848640 1 -- 172.21.15.12:0/2101818389 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf3819f450 con 0x7fbf38100e90 2024-04-22T13:24:09.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.333+0000 7fbf3c848640 1 -- 172.21.15.12:0/2101818389 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf38068d70 con 0x7fbf38100e90 2024-04-22T13:24:09.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.334+0000 7fbf277fe640 1 -- 172.21.15.12:0/2101818389 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7fbf2c010430 con 0x7fbf38100e90 2024-04-22T13:24:09.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.335+0000 7fbf277fe640 1 --2- 172.21.15.12:0/2101818389 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbf200a3db0 0x7fbf200a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:09.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.335+0000 7fbf35d9b640 1 --2- 172.21.15.12:0/2101818389 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbf200a3db0 0x7fbf200a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:09.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.335+0000 7fbf277fe640 1 -- 172.21.15.12:0/2101818389 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fbf2c0d0c30 con 0x7fbf38100e90 2024-04-22T13:24:09.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.336+0000 7fbf35d9b640 1 --2- 172.21.15.12:0/2101818389 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbf200a3db0 0x7fbf200a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbf381985f0 tx=0x7fbf28005f90 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:09.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.340+0000 7fbf277fe640 1 -- 172.21.15.12:0/2101818389 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbf2c09cb60 con 0x7fbf38100e90 2024-04-22T13:24:09.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.533+0000 7fbf3c848640 1 -- 172.21.15.12:0/2101818389 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbf3802d5f0 con 0x7fbf38100e90 2024-04-22T13:24:09.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.533+0000 7fbf277fe640 1 -- 172.21.15.12:0/2101818389 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7fbf2c0bd0f0 con 0x7fbf38100e90 2024-04-22T13:24:09.531 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-22T13:24:09.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.536+0000 7fbf3c848640 1 -- 172.21.15.12:0/2101818389 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbf200a3db0 msgr2=0x7fbf200a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:09.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.536+0000 7fbf3c848640 1 --2- 172.21.15.12:0/2101818389 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbf200a3db0 0x7fbf200a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbf381985f0 tx=0x7fbf28005f90 comp rx=0 tx=0).stop 2024-04-22T13:24:09.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.536+0000 7fbf3c848640 1 -- 172.21.15.12:0/2101818389 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf38100e90 msgr2=0x7fbf381970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:09.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.536+0000 7fbf3c848640 1 --2- 172.21.15.12:0/2101818389 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf38100e90 0x7fbf381970e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c00e9f0 tx=0x7fbf2c00eeb0 comp rx=0 tx=0).stop 2024-04-22T13:24:09.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.536+0000 7fbf3c848640 1 -- 172.21.15.12:0/2101818389 shutdown_connections 2024-04-22T13:24:09.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.536+0000 7fbf3c848640 1 --2- 172.21.15.12:0/2101818389 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbf200a3db0 0x7fbf200a6270 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:09.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.536+0000 7fbf3c848640 1 --2- 172.21.15.12:0/2101818389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf381029a0 0x7fbf3819e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:09.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.536+0000 7fbf3c848640 1 --2- 172.21.15.12:0/2101818389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf38101fe0 0x7fbf38197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:09.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.536+0000 7fbf3c848640 1 --2- 172.21.15.12:0/2101818389 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbf38100e90 0x7fbf381970e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:09.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.536+0000 7fbf3c848640 1 -- 172.21.15.12:0/2101818389 >> 172.21.15.12:0/2101818389 conn(0x7fbf380fc400 msgr2=0x7fbf381053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:09.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.536+0000 7fbf3c848640 1 -- 172.21.15.12:0/2101818389 shutdown_connections 2024-04-22T13:24:09.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.536+0000 7fbf3c848640 1 -- 172.21.15.12:0/2101818389 wait complete. 2024-04-22T13:24:09.545 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T13:24:09.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.657+0000 7fb5ca59f640 1 -- 172.21.15.12:0/1322297344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c4102910 msgr2=0x7fb5c4104d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:09.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.657+0000 7fb5ca59f640 1 --2- 172.21.15.12:0/1322297344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c4102910 0x7fb5c4104d00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4009900 tx=0x7fb5b402bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:09.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.657+0000 7fb5ca59f640 1 -- 172.21.15.12:0/1322297344 shutdown_connections 2024-04-22T13:24:09.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.657+0000 7fb5ca59f640 1 --2- 172.21.15.12:0/1322297344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5c4107b70 0x7fb5c4109fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:09.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.657+0000 7fb5ca59f640 1 --2- 172.21.15.12:0/1322297344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5c4105240 0x7fb5c4107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:09.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.657+0000 7fb5ca59f640 1 --2- 172.21.15.12:0/1322297344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c4102910 0x7fb5c4104d00 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:09.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.657+0000 7fb5ca59f640 1 -- 172.21.15.12:0/1322297344 >> 172.21.15.12:0/1322297344 conn(0x7fb5c40fc300 msgr2=0x7fb5c40fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:09.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.657+0000 7fb5ca59f640 1 -- 172.21.15.12:0/1322297344 shutdown_connections 2024-04-22T13:24:09.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.657+0000 7fb5ca59f640 1 -- 172.21.15.12:0/1322297344 wait complete. 2024-04-22T13:24:09.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.658+0000 7fb5ca59f640 1 Processor -- start 2024-04-22T13:24:09.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.658+0000 7fb5ca59f640 1 -- start start 2024-04-22T13:24:09.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.658+0000 7fb5ca59f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5c4102910 0x7fb5c4102280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:09.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.658+0000 7fb5ca59f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c4105240 0x7fb5c41008c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:09.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.658+0000 7fb5ca59f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5c4107b70 0x7fb5c4100e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:09.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.658+0000 7fb5ca59f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c410da40 con 0x7fb5c4105240 2024-04-22T13:24:09.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.658+0000 7fb5ca59f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c410d8c0 con 0x7fb5c4107b70 2024-04-22T13:24:09.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.658+0000 7fb5ca59f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c410dbc0 con 0x7fb5c4102910 2024-04-22T13:24:09.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.658+0000 7fb5c8d9c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c4105240 0x7fb5c41008c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:09.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.658+0000 7fb5c8d9c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c4105240 0x7fb5c41008c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34996/0 (socket says 172.21.15.12:34996) 2024-04-22T13:24:09.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.658+0000 7fb5c8d9c640 1 -- 172.21.15.12:0/1922322927 learned_addr learned my addr 172.21.15.12:0/1922322927 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:09.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.658+0000 7fb5c9d9e640 1 --2- 172.21.15.12:0/1922322927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5c4107b70 0x7fb5c4100e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:09.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.658+0000 7fb5c959d640 1 --2- 172.21.15.12:0/1922322927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5c4102910 0x7fb5c4102280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:09.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.659+0000 7fb5c8d9c640 1 -- 172.21.15.12:0/1922322927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5c4102910 msgr2=0x7fb5c4102280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:09.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.659+0000 7fb5c8d9c640 1 --2- 172.21.15.12:0/1922322927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5c4102910 0x7fb5c4102280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:09.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.659+0000 7fb5c8d9c640 1 -- 172.21.15.12:0/1922322927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5c4107b70 msgr2=0x7fb5c4100e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:09.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.659+0000 7fb5c8d9c640 1 --2- 172.21.15.12:0/1922322927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5c4107b70 0x7fb5c4100e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:09.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.659+0000 7fb5c8d9c640 1 -- 172.21.15.12:0/1922322927 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c4101690 con 0x7fb5c4105240 2024-04-22T13:24:09.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.659+0000 7fb5c9d9e640 1 --2- 172.21.15.12:0/1922322927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5c4107b70 0x7fb5c4100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:09.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.659+0000 7fb5c8d9c640 1 --2- 172.21.15.12:0/1922322927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c4105240 0x7fb5c41008c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000d740 tx=0x7fb5c000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:09.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.659+0000 7fb5ba7fc640 1 -- 172.21.15.12:0/1922322927 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0017070 con 0x7fb5c4105240 2024-04-22T13:24:09.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.659+0000 7fb5ba7fc640 1 -- 172.21.15.12:0/1922322927 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5c000de50 con 0x7fb5c4105240 2024-04-22T13:24:09.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.659+0000 7fb5ca59f640 1 -- 172.21.15.12:0/1922322927 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c41a2210 con 0x7fb5c4105240 2024-04-22T13:24:09.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.659+0000 7fb5ba7fc640 1 -- 172.21.15.12:0/1922322927 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0004ce0 con 0x7fb5c4105240 2024-04-22T13:24:09.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.659+0000 7fb5ca59f640 1 -- 172.21.15.12:0/1922322927 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c41a2760 con 0x7fb5c4105240 2024-04-22T13:24:09.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.660+0000 7fb5ca59f640 1 -- 172.21.15.12:0/1922322927 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5c40fddf0 con 0x7fb5c4105240 2024-04-22T13:24:09.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.662+0000 7fb5ba7fc640 1 -- 172.21.15.12:0/1922322927 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7fb5c0004e80 con 0x7fb5c4105240 2024-04-22T13:24:09.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.662+0000 7fb5ba7fc640 1 --2- 172.21.15.12:0/1922322927 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb5b00a3e30 0x7fb5b00a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:09.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.663+0000 7fb5ba7fc640 1 -- 172.21.15.12:0/1922322927 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fb5c00d0770 con 0x7fb5c4105240 2024-04-22T13:24:09.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.663+0000 7fb5c959d640 1 --2- 172.21.15.12:0/1922322927 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb5b00a3e30 0x7fb5b00a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:09.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.663+0000 7fb5c959d640 1 --2- 172.21.15.12:0/1922322927 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb5b00a3e30 0x7fb5b00a62f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4006010 tx=0x7fb5b4000bc0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:09.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.666+0000 7fb5ba7fc640 1 -- 172.21.15.12:0/1922322927 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb5c009cd70 con 0x7fb5c4105240 2024-04-22T13:24:09.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:09 smithi012 ceph-mon[28265]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:09.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:09 smithi012 ceph-mon[28265]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-22T13:24:09.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:09 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/4094535981' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:24:09.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:09 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3442454521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:24:09.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:09 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:09.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:09 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:09.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:09 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:09.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:09 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:09.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:09 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:09.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:09 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:09.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:09 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:09.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:09 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:09.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:09 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:09.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:09 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:09.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:09 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:09.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:09 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:09.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:09 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2101818389' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:24:09.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:09 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:09.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:09 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:09.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:09 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:09.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:09 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:09.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:09 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:09.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:09 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:09.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:09 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:09.839 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:09 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:24:09.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.861+0000 7fb5ca59f640 1 -- 172.21.15.12:0/1922322927 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb5c402d810 con 0x7fb5c4105240 2024-04-22T13:24:09.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.862+0000 7fb5ba7fc640 1 -- 172.21.15.12:0/1922322927 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fb5c009c670 con 0x7fb5c4105240 2024-04-22T13:24:09.859 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-22T13:24:09.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.864+0000 7fb5ca59f640 1 -- 172.21.15.12:0/1922322927 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb5b00a3e30 msgr2=0x7fb5b00a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:09.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.864+0000 7fb5ca59f640 1 --2- 172.21.15.12:0/1922322927 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb5b00a3e30 0x7fb5b00a62f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4006010 tx=0x7fb5b4000bc0 comp rx=0 tx=0).stop 2024-04-22T13:24:09.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.864+0000 7fb5ca59f640 1 -- 172.21.15.12:0/1922322927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c4105240 msgr2=0x7fb5c41008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:09.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.864+0000 7fb5ca59f640 1 --2- 172.21.15.12:0/1922322927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c4105240 0x7fb5c41008c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000d740 tx=0x7fb5c000dc00 comp rx=0 tx=0).stop 2024-04-22T13:24:09.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.864+0000 7fb5ca59f640 1 -- 172.21.15.12:0/1922322927 shutdown_connections 2024-04-22T13:24:09.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.864+0000 7fb5ca59f640 1 --2- 172.21.15.12:0/1922322927 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb5b00a3e30 0x7fb5b00a62f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:09.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.864+0000 7fb5ca59f640 1 --2- 172.21.15.12:0/1922322927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5c4107b70 0x7fb5c4100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:09.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.864+0000 7fb5ca59f640 1 --2- 172.21.15.12:0/1922322927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5c4105240 0x7fb5c41008c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:09.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.864+0000 7fb5ca59f640 1 --2- 172.21.15.12:0/1922322927 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5c4102910 0x7fb5c4102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:09.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.864+0000 7fb5ca59f640 1 -- 172.21.15.12:0/1922322927 >> 172.21.15.12:0/1922322927 conn(0x7fb5c40fc300 msgr2=0x7fb5c4103270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:09.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.864+0000 7fb5ca59f640 1 -- 172.21.15.12:0/1922322927 shutdown_connections 2024-04-22T13:24:09.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.864+0000 7fb5ca59f640 1 -- 172.21.15.12:0/1922322927 wait complete. 2024-04-22T13:24:09.872 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-22T13:24:09.873 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:24:09.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.979+0000 7f2d5cc26640 1 -- 172.21.15.12:0/1510492071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d58102040 msgr2=0x7f2d581024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:09.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.979+0000 7f2d5cc26640 1 --2- 172.21.15.12:0/1510492071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d58102040 0x7f2d581024c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c009900 tx=0x7f2d4c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:09.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.979+0000 7f2d5cc26640 1 -- 172.21.15.12:0/1510492071 shutdown_connections 2024-04-22T13:24:09.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.979+0000 7f2d5cc26640 1 --2- 172.21.15.12:0/1510492071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d58102a00 0x7f2d58109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:09.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.979+0000 7f2d5cc26640 1 --2- 172.21.15.12:0/1510492071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d58102040 0x7f2d581024c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:09.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.979+0000 7f2d5cc26640 1 --2- 172.21.15.12:0/1510492071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d58100e60 0x7f2d58101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:09.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.979+0000 7f2d5cc26640 1 -- 172.21.15.12:0/1510492071 >> 172.21.15.12:0/1510492071 conn(0x7f2d580fc420 msgr2=0x7f2d580fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:09.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.979+0000 7f2d5cc26640 1 -- 172.21.15.12:0/1510492071 shutdown_connections 2024-04-22T13:24:09.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.979+0000 7f2d5cc26640 1 -- 172.21.15.12:0/1510492071 wait complete. 2024-04-22T13:24:09.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:09 smithi111 ceph-mon[31361]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:09.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:09 smithi111 ceph-mon[31361]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-22T13:24:09.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:09 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/4094535981' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:24:09.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:09 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/3442454521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:24:09.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:09 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:09.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:09 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:09.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:09 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:09.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:09 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:09.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:09 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:09.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:09 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:09.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:09 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:09.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:09 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:09.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:09 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:09.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:09 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:09.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:09 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:09.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:09 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:09.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:09 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2101818389' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:24:09.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:09 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:09.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:09 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:09.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:09 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:09.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:09 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:09.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:09 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:09.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:09 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:09.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:09 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:09.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:09 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:24:09.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.980+0000 7f2d5cc26640 1 Processor -- start 2024-04-22T13:24:09.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.980+0000 7f2d5cc26640 1 -- start start 2024-04-22T13:24:09.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.980+0000 7f2d5cc26640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d58100e60 0x7f2d5806aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:09.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.980+0000 7f2d5cc26640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d58102040 0x7f2d5819f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:09.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.980+0000 7f2d5cc26640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d58102a00 0x7f2d581a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:09.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.980+0000 7f2d5cc26640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5810c9f0 con 0x7f2d58100e60 2024-04-22T13:24:09.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.980+0000 7f2d5cc26640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5810c870 con 0x7f2d58102a00 2024-04-22T13:24:09.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.980+0000 7f2d5cc26640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5810cb70 con 0x7f2d58102040 2024-04-22T13:24:09.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.981+0000 7f2d5659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d58100e60 0x7f2d5806aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:09.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.981+0000 7f2d5659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d58100e60 0x7f2d5806aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35006/0 (socket says 172.21.15.12:35006) 2024-04-22T13:24:09.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.981+0000 7f2d5659c640 1 -- 172.21.15.12:0/339579050 learned_addr learned my addr 172.21.15.12:0/339579050 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:09.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.981+0000 7f2d55d9b640 1 --2- 172.21.15.12:0/339579050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d58102040 0x7f2d5819f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:09.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.981+0000 7f2d56d9d640 1 --2- 172.21.15.12:0/339579050 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d58102a00 0x7f2d581a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:09.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.981+0000 7f2d5659c640 1 -- 172.21.15.12:0/339579050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d58102040 msgr2=0x7f2d5819f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:09.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.981+0000 7f2d5659c640 1 --2- 172.21.15.12:0/339579050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d58102040 0x7f2d5819f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:09.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.981+0000 7f2d5659c640 1 -- 172.21.15.12:0/339579050 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d58102a00 msgr2=0x7f2d581a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:09.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.981+0000 7f2d5659c640 1 --2- 172.21.15.12:0/339579050 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d58102a00 0x7f2d581a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:09.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.981+0000 7f2d5659c640 1 -- 172.21.15.12:0/339579050 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d581a2020 con 0x7f2d58100e60 2024-04-22T13:24:09.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.981+0000 7f2d5659c640 1 --2- 172.21.15.12:0/339579050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d58100e60 0x7f2d5806aeb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2d4800e9c0 tx=0x7f2d4800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:09.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.981+0000 7f2d477fe640 1 -- 172.21.15.12:0/339579050 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4800cd80 con 0x7f2d58100e60 2024-04-22T13:24:09.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.981+0000 7f2d5cc26640 1 -- 172.21.15.12:0/339579050 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d581a22b0 con 0x7f2d58100e60 2024-04-22T13:24:09.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.981+0000 7f2d477fe640 1 -- 172.21.15.12:0/339579050 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d480035f0 con 0x7f2d58100e60 2024-04-22T13:24:09.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.981+0000 7f2d477fe640 1 -- 172.21.15.12:0/339579050 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48010690 con 0x7f2d58100e60 2024-04-22T13:24:09.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.981+0000 7f2d5cc26640 1 -- 172.21.15.12:0/339579050 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d581a2820 con 0x7f2d58100e60 2024-04-22T13:24:09.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.982+0000 7f2d5cc26640 1 -- 172.21.15.12:0/339579050 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d58068550 con 0x7f2d58100e60 2024-04-22T13:24:09.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.984+0000 7f2d477fe640 1 -- 172.21.15.12:0/339579050 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f2d480037a0 con 0x7f2d58100e60 2024-04-22T13:24:09.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.984+0000 7f2d477fe640 1 --2- 172.21.15.12:0/339579050 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2d400a3de0 0x7f2d400a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:09.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.985+0000 7f2d477fe640 1 -- 172.21.15.12:0/339579050 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f2d480d0ea0 con 0x7f2d58100e60 2024-04-22T13:24:09.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.985+0000 7f2d55d9b640 1 --2- 172.21.15.12:0/339579050 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2d400a3de0 0x7f2d400a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:09.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.985+0000 7f2d55d9b640 1 --2- 172.21.15.12:0/339579050 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2d400a3de0 0x7f2d400a62a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c009230 tx=0x7f2d4c005fd0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:09.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:09.988+0000 7f2d477fe640 1 -- 172.21.15.12:0/339579050 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2d48053ff0 con 0x7f2d58100e60 2024-04-22T13:24:10.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:09 smithi134 ceph-mon[31338]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:10.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:09 smithi134 ceph-mon[31338]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-22T13:24:10.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:09 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/4094535981' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:24:10.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:09 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/3442454521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:24:10.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:09 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:10.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:09 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:10.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:09 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:10.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:09 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:10.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:09 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:10.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:09 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:10.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:09 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:10.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:09 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:10.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:09 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:10.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:09 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:10.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:09 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:10.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:09 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:10.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:09 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2101818389' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:24:10.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:09 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:10.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:09 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:10.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:09 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:10.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:09 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:10.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:09 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:10.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:09 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:24:10.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:09 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:10.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:09 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:24:10.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.222+0000 7f2d5cc26640 1 -- 172.21.15.12:0/339579050 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d58108200 con 0x7f2d58100e60 2024-04-22T13:24:10.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.223+0000 7f2d477fe640 1 -- 172.21.15.12:0/339579050 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7f2d4809d420 con 0x7f2d58100e60 2024-04-22T13:24:10.220 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:24:10.220 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:24:10.221 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:24:10.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.225+0000 7f2d5cc26640 1 -- 172.21.15.12:0/339579050 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2d400a3de0 msgr2=0x7f2d400a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:10.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.225+0000 7f2d5cc26640 1 --2- 172.21.15.12:0/339579050 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2d400a3de0 0x7f2d400a62a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c009230 tx=0x7f2d4c005fd0 comp rx=0 tx=0).stop 2024-04-22T13:24:10.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.225+0000 7f2d5cc26640 1 -- 172.21.15.12:0/339579050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d58100e60 msgr2=0x7f2d5806aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:10.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.225+0000 7f2d5cc26640 1 --2- 172.21.15.12:0/339579050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d58100e60 0x7f2d5806aeb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2d4800e9c0 tx=0x7f2d4800ee80 comp rx=0 tx=0).stop 2024-04-22T13:24:10.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.225+0000 7f2d5cc26640 1 -- 172.21.15.12:0/339579050 shutdown_connections 2024-04-22T13:24:10.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.225+0000 7f2d5cc26640 1 --2- 172.21.15.12:0/339579050 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2d400a3de0 0x7f2d400a62a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:10.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.225+0000 7f2d5cc26640 1 --2- 172.21.15.12:0/339579050 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d58102a00 0x7f2d581a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:10.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.225+0000 7f2d5cc26640 1 --2- 172.21.15.12:0/339579050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2d58102040 0x7f2d5819f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:10.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.225+0000 7f2d5cc26640 1 --2- 172.21.15.12:0/339579050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2d58100e60 0x7f2d5806aeb0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:10.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.225+0000 7f2d5cc26640 1 -- 172.21.15.12:0/339579050 >> 172.21.15.12:0/339579050 conn(0x7f2d580fc420 msgr2=0x7f2d581053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:10.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.225+0000 7f2d5cc26640 1 -- 172.21.15.12:0/339579050 shutdown_connections 2024-04-22T13:24:10.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.225+0000 7f2d5cc26640 1 -- 172.21.15.12:0/339579050 wait complete. 2024-04-22T13:24:10.234 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-22T13:24:10.234 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-22T13:24:10.235 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T13:24:10.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.355+0000 7f6fc26d2640 1 -- 172.21.15.12:0/3369424015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fbc101fc0 msgr2=0x7f6fbc102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:10.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.355+0000 7f6fc26d2640 1 --2- 172.21.15.12:0/3369424015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fbc101fc0 0x7f6fbc102440 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4009900 tx=0x7f6fa402bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:10.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.355+0000 7f6fc26d2640 1 -- 172.21.15.12:0/3369424015 shutdown_connections 2024-04-22T13:24:10.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.355+0000 7f6fc26d2640 1 --2- 172.21.15.12:0/3369424015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fbc102980 0x7f6fbc1092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:10.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.355+0000 7f6fc26d2640 1 --2- 172.21.15.12:0/3369424015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fbc101fc0 0x7f6fbc102440 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:10.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.355+0000 7f6fc26d2640 1 --2- 172.21.15.12:0/3369424015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6fbc100e70 0x7f6fbc101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:10.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.355+0000 7f6fc26d2640 1 -- 172.21.15.12:0/3369424015 >> 172.21.15.12:0/3369424015 conn(0x7f6fbc0fc3e0 msgr2=0x7f6fbc0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:10.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.355+0000 7f6fc26d2640 1 -- 172.21.15.12:0/3369424015 shutdown_connections 2024-04-22T13:24:10.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.355+0000 7f6fc26d2640 1 -- 172.21.15.12:0/3369424015 wait complete. 2024-04-22T13:24:10.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.356+0000 7f6fc26d2640 1 Processor -- start 2024-04-22T13:24:10.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.356+0000 7f6fc26d2640 1 -- start start 2024-04-22T13:24:10.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.356+0000 7f6fc26d2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6fbc100e70 0x7f6fbc197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:10.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.356+0000 7f6fc26d2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fbc101fc0 0x7f6fbc197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:10.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.356+0000 7f6fc26d2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fbc102980 0x7f6fbc19e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:10.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.356+0000 7f6fc26d2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6fbc10ca20 con 0x7f6fbc102980 2024-04-22T13:24:10.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.356+0000 7f6fc26d2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6fbc10c8a0 con 0x7f6fbc100e70 2024-04-22T13:24:10.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.356+0000 7f6fc26d2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6fbc10cba0 con 0x7f6fbc101fc0 2024-04-22T13:24:10.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.356+0000 7f6fc0c6f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fbc102980 0x7f6fbc19e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:10.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.356+0000 7f6fbbfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6fbc100e70 0x7f6fbc197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:10.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.356+0000 7f6fbbfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6fbc100e70 0x7f6fbc197150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:49706/0 (socket says 172.21.15.12:49706) 2024-04-22T13:24:10.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.356+0000 7f6fbbfff640 1 -- 172.21.15.12:0/427223630 learned_addr learned my addr 172.21.15.12:0/427223630 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:10.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.356+0000 7f6fbb7fe640 1 --2- 172.21.15.12:0/427223630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fbc101fc0 0x7f6fbc197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:10.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.356+0000 7f6fc0c6f640 1 -- 172.21.15.12:0/427223630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fbc101fc0 msgr2=0x7f6fbc197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:10.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.356+0000 7f6fc0c6f640 1 --2- 172.21.15.12:0/427223630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fbc101fc0 0x7f6fbc197690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:10.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.356+0000 7f6fc0c6f640 1 -- 172.21.15.12:0/427223630 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6fbc100e70 msgr2=0x7f6fbc197150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:10.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.356+0000 7f6fc0c6f640 1 --2- 172.21.15.12:0/427223630 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6fbc100e70 0x7f6fbc197150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:10.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.356+0000 7f6fc0c6f640 1 -- 172.21.15.12:0/427223630 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fbc19ed70 con 0x7f6fbc102980 2024-04-22T13:24:10.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.356+0000 7f6fbbfff640 1 --2- 172.21.15.12:0/427223630 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6fbc100e70 0x7f6fbc197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:24:10.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.357+0000 7f6fc0c6f640 1 --2- 172.21.15.12:0/427223630 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fbc102980 0x7f6fbc19e700 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6fb400db00 tx=0x7f6fb400e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:10.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.357+0000 7f6fb97fa640 1 -- 172.21.15.12:0/427223630 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb400e8f0 con 0x7f6fbc102980 2024-04-22T13:24:10.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.357+0000 7f6fb97fa640 1 -- 172.21.15.12:0/427223630 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fb40094c0 con 0x7f6fbc102980 2024-04-22T13:24:10.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.357+0000 7f6fc26d2640 1 -- 172.21.15.12:0/427223630 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fbc0690e0 con 0x7f6fbc102980 2024-04-22T13:24:10.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.357+0000 7f6fb97fa640 1 -- 172.21.15.12:0/427223630 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4016650 con 0x7f6fbc102980 2024-04-22T13:24:10.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.357+0000 7f6fc26d2640 1 -- 172.21.15.12:0/427223630 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fbc069600 con 0x7f6fbc102980 2024-04-22T13:24:10.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.358+0000 7f6fc26d2640 1 -- 172.21.15.12:0/427223630 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fbc068de0 con 0x7f6fbc102980 2024-04-22T13:24:10.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.359+0000 7f6fb97fa640 1 -- 172.21.15.12:0/427223630 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f6fb4009670 con 0x7f6fbc102980 2024-04-22T13:24:10.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.360+0000 7f6fb97fa640 1 --2- 172.21.15.12:0/427223630 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6fa80a3db0 0x7f6fa80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:10.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.360+0000 7f6fb97fa640 1 -- 172.21.15.12:0/427223630 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f6fb40d1f70 con 0x7f6fbc102980 2024-04-22T13:24:10.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.360+0000 7f6fbbfff640 1 --2- 172.21.15.12:0/427223630 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6fa80a3db0 0x7f6fa80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:10.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.361+0000 7f6fbbfff640 1 --2- 172.21.15.12:0/427223630 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6fa80a3db0 0x7f6fa80a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6fac005fd0 tx=0x7f6fac00b4a0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:10.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.364+0000 7f6fb97fa640 1 -- 172.21.15.12:0/427223630 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6fb409dfb0 con 0x7f6fbc102980 2024-04-22T13:24:10.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.572+0000 7f6fc26d2640 1 -- 172.21.15.12:0/427223630 --> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6fbc069920 con 0x7f6fa80a3db0 2024-04-22T13:24:10.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.580+0000 7f6fb97fa640 1 -- 172.21.15.12:0/427223630 <== mgr.24968 v2:172.21.15.12:6800/1615661596 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257781 (secure 0 0 0) 0x7f6fbc069920 con 0x7f6fa80a3db0 2024-04-22T13:24:10.580 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:24:10.582 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-22T13:24:10.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.583+0000 7f6fc26d2640 1 -- 172.21.15.12:0/427223630 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6fa80a3db0 msgr2=0x7f6fa80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:10.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.583+0000 7f6fc26d2640 1 --2- 172.21.15.12:0/427223630 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6fa80a3db0 0x7f6fa80a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6fac005fd0 tx=0x7f6fac00b4a0 comp rx=0 tx=0).stop 2024-04-22T13:24:10.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.583+0000 7f6fc26d2640 1 -- 172.21.15.12:0/427223630 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fbc102980 msgr2=0x7f6fbc19e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:10.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.583+0000 7f6fc26d2640 1 --2- 172.21.15.12:0/427223630 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fbc102980 0x7f6fbc19e700 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6fb400db00 tx=0x7f6fb400e0c0 comp rx=0 tx=0).stop 2024-04-22T13:24:10.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.583+0000 7f6fc26d2640 1 -- 172.21.15.12:0/427223630 shutdown_connections 2024-04-22T13:24:10.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.583+0000 7f6fc26d2640 1 --2- 172.21.15.12:0/427223630 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6fa80a3db0 0x7f6fa80a6270 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:10.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.583+0000 7f6fc26d2640 1 --2- 172.21.15.12:0/427223630 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fbc102980 0x7f6fbc19e700 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:10.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.583+0000 7f6fc26d2640 1 --2- 172.21.15.12:0/427223630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fbc101fc0 0x7f6fbc197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:10.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.583+0000 7f6fc26d2640 1 --2- 172.21.15.12:0/427223630 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6fbc100e70 0x7f6fbc197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:10.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.583+0000 7f6fc26d2640 1 -- 172.21.15.12:0/427223630 >> 172.21.15.12:0/427223630 conn(0x7f6fbc0fc3e0 msgr2=0x7f6fbc1053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:10.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.583+0000 7f6fc26d2640 1 -- 172.21.15.12:0/427223630 shutdown_connections 2024-04-22T13:24:10.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.583+0000 7f6fc26d2640 1 -- 172.21.15.12:0/427223630 wait complete. 2024-04-22T13:24:10.592 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-22T13:24:10.176283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189362,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1436,"num_read_kb":883,"num_write":551,"num_write_kb":4402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":937,"ondisk_log_size":937,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376920,"kb_used_data":15252,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124564904,"statfs":{"total":1151940427776,"available":1151554461696,"internally_reserved":0,"allocated":15618048,"data_stored":9218502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.991258"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418486+0000","last_change":"2024-04-22T13:22:05.254388+0000","last_active":"2024-04-22T13:24:06.418486+0000","last_peered":"2024-04-22T13:24:06.418486+0000","last_clean":"2024-04-22T13:24:06.418486+0000","last_became_active":"2024-04-22T13:22:05.254042+0000","last_became_peered":"2024-04-22T13:22:05.254042+0000","last_unstale":"2024-04-22T13:24:06.418486+0000","last_undegraded":"2024-04-22T13:24:06.418486+0000","last_fullsized":"2024-04-22T13:24:06.418486+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:07:56.139653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.412013+0000","last_change":"2024-04-22T13:22:05.247879+0000","last_active":"2024-04-22T13:24:06.412013+0000","last_peered":"2024-04-22T13:24:06.412013+0000","last_clean":"2024-04-22T13:24:06.412013+0000","last_became_active":"2024-04-22T13:22:05.247508+0000","last_became_peered":"2024-04-22T13:22:05.247508+0000","last_unstale":"2024-04-22T13:24:06.412013+0000","last_undegraded":"2024-04-22T13:24:06.412013+0000","last_fullsized":"2024-04-22T13:24:06.412013+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:55:51.928658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411868+0000","last_change":"2024-04-22T13:22:05.257970+0000","last_active":"2024-04-22T13:24:06.411868+0000","last_peered":"2024-04-22T13:24:06.411868+0000","last_clean":"2024-04-22T13:24:06.411868+0000","last_became_active":"2024-04-22T13:22:05.257652+0000","last_became_peered":"2024-04-22T13:22:05.257652+0000","last_unstale":"2024-04-22T13:24:06.411868+0000","last_undegraded":"2024-04-22T13:24:06.411868+0000","last_fullsized":"2024-04-22T13:24:06.411868+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:42:17.476349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411332+0000","last_change":"2024-04-22T13:22:05.249186+0000","last_active":"2024-04-22T13:24:06.411332+0000","last_peered":"2024-04-22T13:24:06.411332+0000","last_clean":"2024-04-22T13:24:06.411332+0000","last_became_active":"2024-04-22T13:22:05.248814+0000","last_became_peered":"2024-04-22T13:22:05.248814+0000","last_unstale":"2024-04-22T13:24:06.411332+0000","last_undegraded":"2024-04-22T13:24:06.411332+0000","last_fullsized":"2024-04-22T13:24:06.411332+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:57.450794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258738+0000","last_change":"2024-04-22T13:22:05.251953+0000","last_active":"2024-04-22T13:24:06.258738+0000","last_peered":"2024-04-22T13:24:06.258738+0000","last_clean":"2024-04-22T13:24:06.258738+0000","last_became_active":"2024-04-22T13:22:05.251547+0000","last_became_peered":"2024-04-22T13:22:05.251547+0000","last_unstale":"2024-04-22T13:24:06.258738+0000","last_undegraded":"2024-04-22T13:24:06.258738+0000","last_fullsized":"2024-04-22T13:24:06.258738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:36:34.809878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.266346+0000","last_change":"2024-04-22T13:23:05.510044+0000","last_active":"2024-04-22T13:24:06.266346+0000","last_peered":"2024-04-22T13:24:06.266346+0000","last_clean":"2024-04-22T13:24:06.266346+0000","last_became_active":"2024-04-22T13:23:05.509677+0000","last_became_peered":"2024-04-22T13:23:05.509677+0000","last_unstale":"2024-04-22T13:24:06.266346+0000","last_undegraded":"2024-04-22T13:24:06.266346+0000","last_fullsized":"2024-04-22T13:24:06.266346+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:46.979291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409036+0000","last_change":"2024-04-22T13:22:05.250418+0000","last_active":"2024-04-22T13:24:06.409036+0000","last_peered":"2024-04-22T13:24:06.409036+0000","last_clean":"2024-04-22T13:24:06.409036+0000","last_became_active":"2024-04-22T13:22:05.250097+0000","last_became_peered":"2024-04-22T13:22:05.250097+0000","last_unstale":"2024-04-22T13:24:06.409036+0000","last_undegraded":"2024-04-22T13:24:06.409036+0000","last_fullsized":"2024-04-22T13:24:06.409036+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:34:59.598950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258759+0000","last_change":"2024-04-22T13:22:05.244762+0000","last_active":"2024-04-22T13:24:06.258759+0000","last_peered":"2024-04-22T13:24:06.258759+0000","last_clean":"2024-04-22T13:24:06.258759+0000","last_became_active":"2024-04-22T13:22:05.244430+0000","last_became_peered":"2024-04-22T13:22:05.244430+0000","last_unstale":"2024-04-22T13:24:06.258759+0000","last_undegraded":"2024-04-22T13:24:06.258759+0000","last_fullsized":"2024-04-22T13:24:06.258759+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:33:01.582851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164321+0000","last_change":"2024-04-22T13:22:05.257596+0000","last_active":"2024-04-22T13:24:06.164321+0000","last_peered":"2024-04-22T13:24:06.164321+0000","last_clean":"2024-04-22T13:24:06.164321+0000","last_became_active":"2024-04-22T13:22:05.257277+0000","last_became_peered":"2024-04-22T13:22:05.257277+0000","last_unstale":"2024-04-22T13:24:06.164321+0000","last_undegraded":"2024-04-22T13:24:06.164321+0000","last_fullsized":"2024-04-22T13:24:06.164321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:09:44.715965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.266708+0000","last_change":"2024-04-22T13:22:05.252177+0000","last_active":"2024-04-22T13:24:06.266708+0000","last_peered":"2024-04-22T13:24:06.266708+0000","last_clean":"2024-04-22T13:24:06.266708+0000","last_became_active":"2024-04-22T13:22:05.251794+0000","last_became_peered":"2024-04-22T13:22:05.251794+0000","last_unstale":"2024-04-22T13:24:06.266708+0000","last_undegraded":"2024-04-22T13:24:06.266708+0000","last_fullsized":"2024-04-22T13:24:06.266708+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:08:59.500540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409476+0000","last_change":"2024-04-22T13:22:05.256209+0000","last_active":"2024-04-22T13:24:06.409476+0000","last_peered":"2024-04-22T13:24:06.409476+0000","last_clean":"2024-04-22T13:24:06.409476+0000","last_became_active":"2024-04-22T13:22:05.255504+0000","last_became_peered":"2024-04-22T13:22:05.255504+0000","last_unstale":"2024-04-22T13:24:06.409476+0000","last_undegraded":"2024-04-22T13:24:06.409476+0000","last_fullsized":"2024-04-22T13:24:06.409476+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:32:08.190921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.256744+0000","last_change":"2024-04-22T13:23:05.509837+0000","last_active":"2024-04-22T13:24:06.256744+0000","last_peered":"2024-04-22T13:24:06.256744+0000","last_clean":"2024-04-22T13:24:06.256744+0000","last_became_active":"2024-04-22T13:23:05.509421+0000","last_became_peered":"2024-04-22T13:23:05.509421+0000","last_unstale":"2024-04-22T13:24:06.256744+0000","last_undegraded":"2024-04-22T13:24:06.256744+0000","last_fullsized":"2024-04-22T13:24:06.256744+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:30:01.103373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162449+0000","last_change":"2024-04-22T13:22:05.246667+0000","last_active":"2024-04-22T13:24:06.162449+0000","last_peered":"2024-04-22T13:24:06.162449+0000","last_clean":"2024-04-22T13:24:06.162449+0000","last_became_active":"2024-04-22T13:22:05.246365+0000","last_became_peered":"2024-04-22T13:22:05.246365+0000","last_unstale":"2024-04-22T13:24:06.162449+0000","last_undegraded":"2024-04-22T13:24:06.162449+0000","last_fullsized":"2024-04-22T13:24:06.162449+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:41:55.533430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411527+0000","last_change":"2024-04-22T13:22:05.251982+0000","last_active":"2024-04-22T13:24:06.411527+0000","last_peered":"2024-04-22T13:24:06.411527+0000","last_clean":"2024-04-22T13:24:06.411527+0000","last_became_active":"2024-04-22T13:22:05.251575+0000","last_became_peered":"2024-04-22T13:22:05.251575+0000","last_unstale":"2024-04-22T13:24:06.411527+0000","last_undegraded":"2024-04-22T13:24:06.411527+0000","last_fullsized":"2024-04-22T13:24:06.411527+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:28:05.343140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411427+0000","last_change":"2024-04-22T13:22:05.251056+0000","last_active":"2024-04-22T13:24:06.411427+0000","last_peered":"2024-04-22T13:24:06.411427+0000","last_clean":"2024-04-22T13:24:06.411427+0000","last_became_active":"2024-04-22T13:22:05.250652+0000","last_became_peered":"2024-04-22T13:22:05.250652+0000","last_unstale":"2024-04-22T13:24:06.411427+0000","last_undegraded":"2024-04-22T13:24:06.411427+0000","last_fullsized":"2024-04-22T13:24:06.411427+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:08:28.629965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.163636+0000","last_change":"2024-04-22T13:22:05.256877+0000","last_active":"2024-04-22T13:24:06.163636+0000","last_peered":"2024-04-22T13:24:06.163636+0000","last_clean":"2024-04-22T13:24:06.163636+0000","last_became_active":"2024-04-22T13:22:05.256074+0000","last_became_peered":"2024-04-22T13:22:05.256074+0000","last_unstale":"2024-04-22T13:24:06.163636+0000","last_undegraded":"2024-04-22T13:24:06.163636+0000","last_fullsized":"2024-04-22T13:24:06.163636+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:08:53.607626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418592+0000","last_change":"2024-04-22T13:22:05.255598+0000","last_active":"2024-04-22T13:24:06.418592+0000","last_peered":"2024-04-22T13:24:06.418592+0000","last_clean":"2024-04-22T13:24:06.418592+0000","last_became_active":"2024-04-22T13:22:05.255239+0000","last_became_peered":"2024-04-22T13:22:05.255239+0000","last_unstale":"2024-04-22T13:24:06.418592+0000","last_undegraded":"2024-04-22T13:24:06.418592+0000","last_fullsized":"2024-04-22T13:24:06.418592+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:38:20.547882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.257085+0000","last_change":"2024-04-22T13:22:05.254232+0000","last_active":"2024-04-22T13:24:06.257085+0000","last_peered":"2024-04-22T13:24:06.257085+0000","last_clean":"2024-04-22T13:24:06.257085+0000","last_became_active":"2024-04-22T13:22:05.253907+0000","last_became_peered":"2024-04-22T13:22:05.253907+0000","last_unstale":"2024-04-22T13:24:06.257085+0000","last_undegraded":"2024-04-22T13:24:06.257085+0000","last_fullsized":"2024-04-22T13:24:06.257085+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:15:49.931112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.415196+0000","last_change":"2024-04-22T13:22:05.251518+0000","last_active":"2024-04-22T13:24:06.415196+0000","last_peered":"2024-04-22T13:24:06.415196+0000","last_clean":"2024-04-22T13:24:06.415196+0000","last_became_active":"2024-04-22T13:22:05.250478+0000","last_became_peered":"2024-04-22T13:22:05.250478+0000","last_unstale":"2024-04-22T13:24:06.415196+0000","last_undegraded":"2024-04-22T13:24:06.415196+0000","last_fullsized":"2024-04-22T13:24:06.415196+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:25:26.758692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'14","reported_seq":152,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.559609+0000","last_change":"2024-04-22T13:22:04.241365+0000","last_active":"2024-04-22T13:24:06.559609+0000","last_peered":"2024-04-22T13:24:06.559609+0000","last_clean":"2024-04-22T13:24:06.559609+0000","last_became_active":"2024-04-22T13:22:04.240822+0000","last_became_peered":"2024-04-22T13:22:04.240822+0000","last_unstale":"2024-04-22T13:24:06.559609+0000","last_undegraded":"2024-04-22T13:24:06.559609+0000","last_fullsized":"2024-04-22T13:24:06.559609+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:20:23.928534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.256956+0000","last_change":"2024-04-22T13:23:05.876262+0000","last_active":"2024-04-22T13:24:06.256956+0000","last_peered":"2024-04-22T13:24:06.256956+0000","last_clean":"2024-04-22T13:24:06.256956+0000","last_became_active":"2024-04-22T13:23:05.875872+0000","last_became_peered":"2024-04-22T13:23:05.875872+0000","last_unstale":"2024-04-22T13:24:06.256956+0000","last_undegraded":"2024-04-22T13:24:06.256956+0000","last_fullsized":"2024-04-22T13:24:06.256956+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:29:33.092887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259476+0000","last_change":"2024-04-22T13:22:04.248872+0000","last_active":"2024-04-22T13:24:06.259476+0000","last_peered":"2024-04-22T13:24:06.259476+0000","last_clean":"2024-04-22T13:24:06.259476+0000","last_became_active":"2024-04-22T13:22:04.248466+0000","last_became_peered":"2024-04-22T13:22:04.248466+0000","last_unstale":"2024-04-22T13:24:06.259476+0000","last_undegraded":"2024-04-22T13:24:06.259476+0000","last_fullsized":"2024-04-22T13:24:06.259476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:38:10.871602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.412391+0000","last_change":"2024-04-22T13:22:05.258005+0000","last_active":"2024-04-22T13:24:06.412391+0000","last_peered":"2024-04-22T13:24:06.412391+0000","last_clean":"2024-04-22T13:24:06.412391+0000","last_became_active":"2024-04-22T13:22:05.257615+0000","last_became_peered":"2024-04-22T13:22:05.257615+0000","last_unstale":"2024-04-22T13:24:06.412391+0000","last_undegraded":"2024-04-22T13:24:06.412391+0000","last_fullsized":"2024-04-22T13:24:06.412391+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:21:18.030530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259060+0000","last_change":"2024-04-22T13:22:04.246273+0000","last_active":"2024-04-22T13:24:06.259060+0000","last_peered":"2024-04-22T13:24:06.259060+0000","last_clean":"2024-04-22T13:24:06.259060+0000","last_became_active":"2024-04-22T13:22:04.245969+0000","last_became_peered":"2024-04-22T13:22:04.245969+0000","last_unstale":"2024-04-22T13:24:06.259060+0000","last_undegraded":"2024-04-22T13:24:06.259060+0000","last_fullsized":"2024-04-22T13:24:06.259060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:10.260192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.163964+0000","last_change":"2024-04-22T13:22:05.260299+0000","last_active":"2024-04-22T13:24:06.163964+0000","last_peered":"2024-04-22T13:24:06.163964+0000","last_clean":"2024-04-22T13:24:06.163964+0000","last_became_active":"2024-04-22T13:22:05.259634+0000","last_became_peered":"2024-04-22T13:22:05.259634+0000","last_unstale":"2024-04-22T13:24:06.163964+0000","last_undegraded":"2024-04-22T13:24:06.163964+0000","last_fullsized":"2024-04-22T13:24:06.163964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:16:44.421652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258970+0000","last_change":"2024-04-22T13:22:04.243843+0000","last_active":"2024-04-22T13:24:06.258970+0000","last_peered":"2024-04-22T13:24:06.258970+0000","last_clean":"2024-04-22T13:24:06.258970+0000","last_became_active":"2024-04-22T13:22:04.243465+0000","last_became_peered":"2024-04-22T13:22:04.243465+0000","last_unstale":"2024-04-22T13:24:06.258970+0000","last_undegraded":"2024-04-22T13:24:06.258970+0000","last_fullsized":"2024-04-22T13:24:06.258970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:48:36.431697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164717+0000","last_change":"2024-04-22T13:22:05.259789+0000","last_active":"2024-04-22T13:24:06.164717+0000","last_peered":"2024-04-22T13:24:06.164717+0000","last_clean":"2024-04-22T13:24:06.164717+0000","last_became_active":"2024-04-22T13:22:05.259395+0000","last_became_peered":"2024-04-22T13:22:05.259395+0000","last_unstale":"2024-04-22T13:24:06.164717+0000","last_undegraded":"2024-04-22T13:24:06.164717+0000","last_fullsized":"2024-04-22T13:24:06.164717+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:43:51.430561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":148,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.533918+0000","last_change":"2024-04-22T13:22:04.240615+0000","last_active":"2024-04-22T13:24:06.533918+0000","last_peered":"2024-04-22T13:24:06.533918+0000","last_clean":"2024-04-22T13:24:06.533918+0000","last_became_active":"2024-04-22T13:22:04.240248+0000","last_became_peered":"2024-04-22T13:22:04.240248+0000","last_unstale":"2024-04-22T13:24:06.533918+0000","last_undegraded":"2024-04-22T13:24:06.533918+0000","last_fullsized":"2024-04-22T13:24:06.533918+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:55:05.948979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258665+0000","last_change":"2024-04-22T13:22:05.253788+0000","last_active":"2024-04-22T13:24:06.258665+0000","last_peered":"2024-04-22T13:24:06.258665+0000","last_clean":"2024-04-22T13:24:06.258665+0000","last_became_active":"2024-04-22T13:22:05.252578+0000","last_became_peered":"2024-04-22T13:22:05.252578+0000","last_unstale":"2024-04-22T13:24:06.258665+0000","last_undegraded":"2024-04-22T13:24:06.258665+0000","last_fullsized":"2024-04-22T13:24:06.258665+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:04:11.769930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.266274+0000","last_change":"2024-04-22T13:22:04.241170+0000","last_active":"2024-04-22T13:24:06.266274+0000","last_peered":"2024-04-22T13:24:06.266274+0000","last_clean":"2024-04-22T13:24:06.266274+0000","last_became_active":"2024-04-22T13:22:04.240830+0000","last_became_peered":"2024-04-22T13:22:04.240830+0000","last_unstale":"2024-04-22T13:24:06.266274+0000","last_undegraded":"2024-04-22T13:24:06.266274+0000","last_fullsized":"2024-04-22T13:24:06.266274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:51:30.520613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164733+0000","last_change":"2024-04-22T13:22:05.237900+0000","last_active":"2024-04-22T13:24:06.164733+0000","last_peered":"2024-04-22T13:24:06.164733+0000","last_clean":"2024-04-22T13:24:06.164733+0000","last_became_active":"2024-04-22T13:22:05.237507+0000","last_became_peered":"2024-04-22T13:22:05.237507+0000","last_unstale":"2024-04-22T13:24:06.164733+0000","last_undegraded":"2024-04-22T13:24:06.164733+0000","last_fullsized":"2024-04-22T13:24:06.164733+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:26:24.307803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162328+0000","last_change":"2024-04-22T13:22:04.236078+0000","last_active":"2024-04-22T13:24:06.162328+0000","last_peered":"2024-04-22T13:24:06.162328+0000","last_clean":"2024-04-22T13:24:06.162328+0000","last_became_active":"2024-04-22T13:22:04.235710+0000","last_became_peered":"2024-04-22T13:22:04.235710+0000","last_unstale":"2024-04-22T13:24:06.162328+0000","last_undegraded":"2024-04-22T13:24:06.162328+0000","last_fullsized":"2024-04-22T13:24:06.162328+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:37:55.335237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258717+0000","last_change":"2024-04-22T13:22:05.243152+0000","last_active":"2024-04-22T13:24:06.258717+0000","last_peered":"2024-04-22T13:24:06.258717+0000","last_clean":"2024-04-22T13:24:06.258717+0000","last_became_active":"2024-04-22T13:22:05.242772+0000","last_became_peered":"2024-04-22T13:22:05.242772+0000","last_unstale":"2024-04-22T13:24:06.258717+0000","last_undegraded":"2024-04-22T13:24:06.258717+0000","last_fullsized":"2024-04-22T13:24:06.258717+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:33:42.359679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164258+0000","last_change":"2024-04-22T13:22:04.244381+0000","last_active":"2024-04-22T13:24:06.164258+0000","last_peered":"2024-04-22T13:24:06.164258+0000","last_clean":"2024-04-22T13:24:06.164258+0000","last_became_active":"2024-04-22T13:22:04.243746+0000","last_became_peered":"2024-04-22T13:22:04.243746+0000","last_unstale":"2024-04-22T13:24:06.164258+0000","last_undegraded":"2024-04-22T13:24:06.164258+0000","last_fullsized":"2024-04-22T13:24:06.164258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:15:36.019930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.256753+0000","last_change":"2024-04-22T13:22:05.245394+0000","last_active":"2024-04-22T13:24:06.256753+0000","last_peered":"2024-04-22T13:24:06.256753+0000","last_clean":"2024-04-22T13:24:06.256753+0000","last_became_active":"2024-04-22T13:22:05.243931+0000","last_became_peered":"2024-04-22T13:22:05.243931+0000","last_unstale":"2024-04-22T13:24:06.256753+0000","last_undegraded":"2024-04-22T13:24:06.256753+0000","last_fullsized":"2024-04-22T13:24:06.256753+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:11:10.790842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'6","reported_seq":260,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.533504+0000","last_change":"2024-04-22T13:22:04.242583+0000","last_active":"2024-04-22T13:24:06.533504+0000","last_peered":"2024-04-22T13:24:06.533504+0000","last_clean":"2024-04-22T13:24:06.533504+0000","last_became_active":"2024-04-22T13:22:04.242261+0000","last_became_peered":"2024-04-22T13:22:04.242261+0000","last_unstale":"2024-04-22T13:24:06.533504+0000","last_undegraded":"2024-04-22T13:24:06.533504+0000","last_fullsized":"2024-04-22T13:24:06.533504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:37:47.618573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":233,"num_read_kb":233,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164258+0000","last_change":"2024-04-22T13:22:05.255293+0000","last_active":"2024-04-22T13:24:06.164258+0000","last_peered":"2024-04-22T13:24:06.164258+0000","last_clean":"2024-04-22T13:24:06.164258+0000","last_became_active":"2024-04-22T13:22:05.253949+0000","last_became_peered":"2024-04-22T13:22:05.253949+0000","last_unstale":"2024-04-22T13:24:06.164258+0000","last_undegraded":"2024-04-22T13:24:06.164258+0000","last_fullsized":"2024-04-22T13:24:06.164258+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:41:08.754989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409503+0000","last_change":"2024-04-22T13:22:04.241529+0000","last_active":"2024-04-22T13:24:06.409503+0000","last_peered":"2024-04-22T13:24:06.409503+0000","last_clean":"2024-04-22T13:24:06.409503+0000","last_became_active":"2024-04-22T13:22:04.241164+0000","last_became_peered":"2024-04-22T13:22:04.241164+0000","last_unstale":"2024-04-22T13:24:06.409503+0000","last_undegraded":"2024-04-22T13:24:06.409503+0000","last_fullsized":"2024-04-22T13:24:06.409503+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:16:49.387181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.412225+0000","last_change":"2024-04-22T13:22:05.241623+0000","last_active":"2024-04-22T13:24:06.412225+0000","last_peered":"2024-04-22T13:24:06.412225+0000","last_clean":"2024-04-22T13:24:06.412225+0000","last_became_active":"2024-04-22T13:22:05.241150+0000","last_became_peered":"2024-04-22T13:22:05.241150+0000","last_unstale":"2024-04-22T13:24:06.412225+0000","last_undegraded":"2024-04-22T13:24:06.412225+0000","last_fullsized":"2024-04-22T13:24:06.412225+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:17:28.412366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164893+0000","last_change":"2024-04-22T13:22:04.247852+0000","last_active":"2024-04-22T13:24:06.164893+0000","last_peered":"2024-04-22T13:24:06.164893+0000","last_clean":"2024-04-22T13:24:06.164893+0000","last_became_active":"2024-04-22T13:22:04.247550+0000","last_became_peered":"2024-04-22T13:22:04.247550+0000","last_unstale":"2024-04-22T13:24:06.164893+0000","last_undegraded":"2024-04-22T13:24:06.164893+0000","last_fullsized":"2024-04-22T13:24:06.164893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:55:01.928435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164057+0000","last_change":"2024-04-22T13:22:05.260050+0000","last_active":"2024-04-22T13:24:06.164057+0000","last_peered":"2024-04-22T13:24:06.164057+0000","last_clean":"2024-04-22T13:24:06.164057+0000","last_became_active":"2024-04-22T13:22:05.259098+0000","last_became_peered":"2024-04-22T13:22:05.259098+0000","last_unstale":"2024-04-22T13:24:06.164057+0000","last_undegraded":"2024-04-22T13:24:06.164057+0000","last_fullsized":"2024-04-22T13:24:06.164057+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:05:06.245492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'36","reported_seq":305,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.534247+0000","last_change":"2024-04-22T13:22:04.240233+0000","last_active":"2024-04-22T13:24:06.534247+0000","last_peered":"2024-04-22T13:24:06.534247+0000","last_clean":"2024-04-22T13:24:06.534247+0000","last_became_active":"2024-04-22T13:22:04.239890+0000","last_became_peered":"2024-04-22T13:22:04.239890+0000","last_unstale":"2024-04-22T13:24:06.534247+0000","last_undegraded":"2024-04-22T13:24:06.534247+0000","last_fullsized":"2024-04-22T13:24:06.534247+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:04:21.547422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64727,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":248,"num_read_kb":188,"num_write":38,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258900+0000","last_change":"2024-04-22T13:22:05.257756+0000","last_active":"2024-04-22T13:24:06.258900+0000","last_peered":"2024-04-22T13:24:06.258900+0000","last_clean":"2024-04-22T13:24:06.258900+0000","last_became_active":"2024-04-22T13:22:05.257442+0000","last_became_peered":"2024-04-22T13:22:05.257442+0000","last_unstale":"2024-04-22T13:24:06.258900+0000","last_undegraded":"2024-04-22T13:24:06.258900+0000","last_fullsized":"2024-04-22T13:24:06.258900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:16:53.465689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.414923+0000","last_change":"2024-04-22T13:22:04.240815+0000","last_active":"2024-04-22T13:24:06.414923+0000","last_peered":"2024-04-22T13:24:06.414923+0000","last_clean":"2024-04-22T13:24:06.414923+0000","last_became_active":"2024-04-22T13:22:04.240218+0000","last_became_peered":"2024-04-22T13:22:04.240218+0000","last_unstale":"2024-04-22T13:24:06.414923+0000","last_undegraded":"2024-04-22T13:24:06.414923+0000","last_fullsized":"2024-04-22T13:24:06.414923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:29:29.148764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162257+0000","last_change":"2024-04-22T13:22:05.248797+0000","last_active":"2024-04-22T13:24:06.162257+0000","last_peered":"2024-04-22T13:24:06.162257+0000","last_clean":"2024-04-22T13:24:06.162257+0000","last_became_active":"2024-04-22T13:22:05.248435+0000","last_became_peered":"2024-04-22T13:22:05.248435+0000","last_unstale":"2024-04-22T13:24:06.162257+0000","last_undegraded":"2024-04-22T13:24:06.162257+0000","last_fullsized":"2024-04-22T13:24:06.162257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:32:31.970494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164389+0000","last_change":"2024-04-22T13:22:04.245303+0000","last_active":"2024-04-22T13:24:06.164389+0000","last_peered":"2024-04-22T13:24:06.164389+0000","last_clean":"2024-04-22T13:24:06.164389+0000","last_became_active":"2024-04-22T13:22:04.244903+0000","last_became_peered":"2024-04-22T13:22:04.244903+0000","last_unstale":"2024-04-22T13:24:06.164389+0000","last_undegraded":"2024-04-22T13:24:06.164389+0000","last_fullsized":"2024-04-22T13:24:06.164389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:34:32.174858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409264+0000","last_change":"2024-04-22T13:22:05.246318+0000","last_active":"2024-04-22T13:24:06.409264+0000","last_peered":"2024-04-22T13:24:06.409264+0000","last_clean":"2024-04-22T13:24:06.409264+0000","last_became_active":"2024-04-22T13:22:05.245899+0000","last_became_peered":"2024-04-22T13:22:05.245899+0000","last_unstale":"2024-04-22T13:24:06.409264+0000","last_undegraded":"2024-04-22T13:24:06.409264+0000","last_fullsized":"2024-04-22T13:24:06.409264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:27:42.789058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'8","reported_seq":148,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.591632+0000","last_change":"2024-04-22T13:22:04.240612+0000","last_active":"2024-04-22T13:24:06.591632+0000","last_peered":"2024-04-22T13:24:06.591632+0000","last_clean":"2024-04-22T13:24:06.591632+0000","last_became_active":"2024-04-22T13:22:04.240290+0000","last_became_peered":"2024-04-22T13:22:04.240290+0000","last_unstale":"2024-04-22T13:24:06.591632+0000","last_undegraded":"2024-04-22T13:24:06.591632+0000","last_fullsized":"2024-04-22T13:24:06.591632+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:00:50.285553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.408976+0000","last_change":"2024-04-22T13:22:05.253205+0000","last_active":"2024-04-22T13:24:06.408976+0000","last_peered":"2024-04-22T13:24:06.408976+0000","last_clean":"2024-04-22T13:24:06.408976+0000","last_became_active":"2024-04-22T13:22:05.252865+0000","last_became_peered":"2024-04-22T13:22:05.252865+0000","last_unstale":"2024-04-22T13:24:06.408976+0000","last_undegraded":"2024-04-22T13:24:06.408976+0000","last_fullsized":"2024-04-22T13:24:06.408976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:19:12.482371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258636+0000","last_change":"2024-04-22T13:22:04.248938+0000","last_active":"2024-04-22T13:24:06.258636+0000","last_peered":"2024-04-22T13:24:06.258636+0000","last_clean":"2024-04-22T13:24:06.258636+0000","last_became_active":"2024-04-22T13:22:04.248440+0000","last_became_peered":"2024-04-22T13:22:04.248440+0000","last_unstale":"2024-04-22T13:24:06.258636+0000","last_undegraded":"2024-04-22T13:24:06.258636+0000","last_fullsized":"2024-04-22T13:24:06.258636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:46:50.928067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164391+0000","last_change":"2024-04-22T13:22:05.254664+0000","last_active":"2024-04-22T13:24:06.164391+0000","last_peered":"2024-04-22T13:24:06.164391+0000","last_clean":"2024-04-22T13:24:06.164391+0000","last_became_active":"2024-04-22T13:22:05.254304+0000","last_became_peered":"2024-04-22T13:22:05.254304+0000","last_unstale":"2024-04-22T13:24:06.164391+0000","last_undegraded":"2024-04-22T13:24:06.164391+0000","last_fullsized":"2024-04-22T13:24:06.164391+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:58:49.345573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259033+0000","last_change":"2024-04-22T13:22:04.244128+0000","last_active":"2024-04-22T13:24:06.259033+0000","last_peered":"2024-04-22T13:24:06.259033+0000","last_clean":"2024-04-22T13:24:06.259033+0000","last_became_active":"2024-04-22T13:22:04.243738+0000","last_became_peered":"2024-04-22T13:22:04.243738+0000","last_unstale":"2024-04-22T13:24:06.259033+0000","last_undegraded":"2024-04-22T13:24:06.259033+0000","last_fullsized":"2024-04-22T13:24:06.259033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:23:52.220584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418387+0000","last_change":"2024-04-22T13:22:05.250910+0000","last_active":"2024-04-22T13:24:06.418387+0000","last_peered":"2024-04-22T13:24:06.418387+0000","last_clean":"2024-04-22T13:24:06.418387+0000","last_became_active":"2024-04-22T13:22:05.250410+0000","last_became_peered":"2024-04-22T13:22:05.250410+0000","last_unstale":"2024-04-22T13:24:06.418387+0000","last_undegraded":"2024-04-22T13:24:06.418387+0000","last_fullsized":"2024-04-22T13:24:06.418387+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:04:25.492677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.256761+0000","last_change":"2024-04-22T13:22:04.245889+0000","last_active":"2024-04-22T13:24:06.256761+0000","last_peered":"2024-04-22T13:24:06.256761+0000","last_clean":"2024-04-22T13:24:06.256761+0000","last_became_active":"2024-04-22T13:22:04.245474+0000","last_became_peered":"2024-04-22T13:22:04.245474+0000","last_unstale":"2024-04-22T13:24:06.256761+0000","last_undegraded":"2024-04-22T13:24:06.256761+0000","last_fullsized":"2024-04-22T13:24:06.256761+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:22:12.626950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.266594+0000","last_change":"2024-04-22T13:22:05.250978+0000","last_active":"2024-04-22T13:24:06.266594+0000","last_peered":"2024-04-22T13:24:06.266594+0000","last_clean":"2024-04-22T13:24:06.266594+0000","last_became_active":"2024-04-22T13:22:05.250463+0000","last_became_peered":"2024-04-22T13:22:05.250463+0000","last_unstale":"2024-04-22T13:24:06.266594+0000","last_undegraded":"2024-04-22T13:24:06.266594+0000","last_fullsized":"2024-04-22T13:24:06.266594+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:45:18.591072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'7","reported_seq":146,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.560650+0000","last_change":"2024-04-22T13:22:04.242503+0000","last_active":"2024-04-22T13:24:06.560650+0000","last_peered":"2024-04-22T13:24:06.560650+0000","last_clean":"2024-04-22T13:24:06.560650+0000","last_became_active":"2024-04-22T13:22:04.242143+0000","last_became_peered":"2024-04-22T13:22:04.242143+0000","last_unstale":"2024-04-22T13:24:06.560650+0000","last_undegraded":"2024-04-22T13:24:06.560650+0000","last_fullsized":"2024-04-22T13:24:06.560650+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:40.236377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.266483+0000","last_change":"2024-04-22T13:22:05.252387+0000","last_active":"2024-04-22T13:24:06.266483+0000","last_peered":"2024-04-22T13:24:06.266483+0000","last_clean":"2024-04-22T13:24:06.266483+0000","last_became_active":"2024-04-22T13:22:05.252033+0000","last_became_peered":"2024-04-22T13:22:05.252033+0000","last_unstale":"2024-04-22T13:24:06.266483+0000","last_undegraded":"2024-04-22T13:24:06.266483+0000","last_fullsized":"2024-04-22T13:24:06.266483+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:55:01.791283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164402+0000","last_change":"2024-04-22T13:22:05.248394+0000","last_active":"2024-04-22T13:24:06.164402+0000","last_peered":"2024-04-22T13:24:06.164402+0000","last_clean":"2024-04-22T13:24:06.164402+0000","last_became_active":"2024-04-22T13:22:05.248024+0000","last_became_peered":"2024-04-22T13:22:05.248024+0000","last_unstale":"2024-04-22T13:24:06.164402+0000","last_undegraded":"2024-04-22T13:24:06.164402+0000","last_fullsized":"2024-04-22T13:24:06.164402+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:49:36.556136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'7","reported_seq":145,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.590537+0000","last_change":"2024-04-22T13:22:04.247692+0000","last_active":"2024-04-22T13:24:06.590537+0000","last_peered":"2024-04-22T13:24:06.590537+0000","last_clean":"2024-04-22T13:24:06.590537+0000","last_became_active":"2024-04-22T13:22:04.247345+0000","last_became_peered":"2024-04-22T13:22:04.247345+0000","last_unstale":"2024-04-22T13:24:06.590537+0000","last_undegraded":"2024-04-22T13:24:06.590537+0000","last_fullsized":"2024-04-22T13:24:06.590537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:07:49.803248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259293+0000","last_change":"2024-04-22T13:22:05.241022+0000","last_active":"2024-04-22T13:24:06.259293+0000","last_peered":"2024-04-22T13:24:06.259293+0000","last_clean":"2024-04-22T13:24:06.259293+0000","last_became_active":"2024-04-22T13:22:05.240635+0000","last_became_peered":"2024-04-22T13:22:05.240635+0000","last_unstale":"2024-04-22T13:24:06.259293+0000","last_undegraded":"2024-04-22T13:24:06.259293+0000","last_fullsized":"2024-04-22T13:24:06.259293+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:50:02.012260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411229+0000","last_change":"2024-04-22T13:22:04.236817+0000","last_active":"2024-04-22T13:24:06.411229+0000","last_peered":"2024-04-22T13:24:06.411229+0000","last_clean":"2024-04-22T13:24:06.411229+0000","last_became_active":"2024-04-22T13:22:04.236484+0000","last_became_peered":"2024-04-22T13:22:04.236484+0000","last_unstale":"2024-04-22T13:24:06.411229+0000","last_undegraded":"2024-04-22T13:24:06.411229+0000","last_fullsized":"2024-04-22T13:24:06.411229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:27:41.053327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":357,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409048+0000","last_change":"2024-04-22T13:20:53.094123+0000","last_active":"2024-04-22T13:24:06.409048+0000","last_peered":"2024-04-22T13:24:06.409048+0000","last_clean":"2024-04-22T13:24:06.409048+0000","last_became_active":"2024-04-22T13:20:53.093664+0000","last_became_peered":"2024-04-22T13:20:53.093664+0000","last_unstale":"2024-04-22T13:24:06.409048+0000","last_undegraded":"2024-04-22T13:24:06.409048+0000","last_fullsized":"2024-04-22T13:24:06.409048+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:17:33.201786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:17:33.201786+0000","last_clean_scrub_stamp":"2024-04-22T13:17:33.201786+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:46:58.986399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162213+0000","last_change":"2024-04-22T13:22:05.247766+0000","last_active":"2024-04-22T13:24:06.162213+0000","last_peered":"2024-04-22T13:24:06.162213+0000","last_clean":"2024-04-22T13:24:06.162213+0000","last_became_active":"2024-04-22T13:22:05.247362+0000","last_became_peered":"2024-04-22T13:22:05.247362+0000","last_unstale":"2024-04-22T13:24:06.162213+0000","last_undegraded":"2024-04-22T13:24:06.162213+0000","last_fullsized":"2024-04-22T13:24:06.162213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:25:00.126638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409239+0000","last_change":"2024-04-22T13:22:04.243493+0000","last_active":"2024-04-22T13:24:06.409239+0000","last_peered":"2024-04-22T13:24:06.409239+0000","last_clean":"2024-04-22T13:24:06.409239+0000","last_became_active":"2024-04-22T13:22:04.243190+0000","last_became_peered":"2024-04-22T13:22:04.243190+0000","last_unstale":"2024-04-22T13:24:06.409239+0000","last_undegraded":"2024-04-22T13:24:06.409239+0000","last_fullsized":"2024-04-22T13:24:06.409239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:15:09.025152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258892+0000","last_change":"2024-04-22T13:22:04.249222+0000","last_active":"2024-04-22T13:24:06.258892+0000","last_peered":"2024-04-22T13:24:06.258892+0000","last_clean":"2024-04-22T13:24:06.258892+0000","last_became_active":"2024-04-22T13:22:04.248926+0000","last_became_peered":"2024-04-22T13:22:04.248926+0000","last_unstale":"2024-04-22T13:24:06.258892+0000","last_undegraded":"2024-04-22T13:24:06.258892+0000","last_fullsized":"2024-04-22T13:24:06.258892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:33:50.168426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162433+0000","last_change":"2024-04-22T13:22:05.248199+0000","last_active":"2024-04-22T13:24:06.162433+0000","last_peered":"2024-04-22T13:24:06.162433+0000","last_clean":"2024-04-22T13:24:06.162433+0000","last_became_active":"2024-04-22T13:22:05.247873+0000","last_became_peered":"2024-04-22T13:22:05.247873+0000","last_unstale":"2024-04-22T13:24:06.162433+0000","last_undegraded":"2024-04-22T13:24:06.162433+0000","last_fullsized":"2024-04-22T13:24:06.162433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:40:29.353901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'48","reported_seq":92,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.370574+0000","last_change":"2024-04-22T13:22:04.243474+0000","last_active":"2024-04-22T13:24:06.370574+0000","last_peered":"2024-04-22T13:24:06.370574+0000","last_clean":"2024-04-22T13:24:06.370574+0000","last_became_active":"2024-04-22T13:22:04.243151+0000","last_became_peered":"2024-04-22T13:22:04.243151+0000","last_unstale":"2024-04-22T13:24:06.370574+0000","last_undegraded":"2024-04-22T13:24:06.370574+0000","last_fullsized":"2024-04-22T13:24:06.370574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:18:25.140715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259264+0000","last_change":"2024-04-22T13:22:05.243120+0000","last_active":"2024-04-22T13:24:06.259264+0000","last_peered":"2024-04-22T13:24:06.259264+0000","last_clean":"2024-04-22T13:24:06.259264+0000","last_became_active":"2024-04-22T13:22:05.242726+0000","last_became_peered":"2024-04-22T13:22:05.242726+0000","last_unstale":"2024-04-22T13:24:06.259264+0000","last_undegraded":"2024-04-22T13:24:06.259264+0000","last_fullsized":"2024-04-22T13:24:06.259264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:06:16.087884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411424+0000","last_change":"2024-04-22T13:22:04.239672+0000","last_active":"2024-04-22T13:24:06.411424+0000","last_peered":"2024-04-22T13:24:06.411424+0000","last_clean":"2024-04-22T13:24:06.411424+0000","last_became_active":"2024-04-22T13:22:04.238776+0000","last_became_peered":"2024-04-22T13:22:04.238776+0000","last_unstale":"2024-04-22T13:24:06.411424+0000","last_undegraded":"2024-04-22T13:24:06.411424+0000","last_fullsized":"2024-04-22T13:24:06.411424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:43:21.752547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.412035+0000","last_change":"2024-04-22T13:22:05.242089+0000","last_active":"2024-04-22T13:24:06.412035+0000","last_peered":"2024-04-22T13:24:06.412035+0000","last_clean":"2024-04-22T13:24:06.412035+0000","last_became_active":"2024-04-22T13:22:05.241681+0000","last_became_peered":"2024-04-22T13:22:05.241681+0000","last_unstale":"2024-04-22T13:24:06.412035+0000","last_undegraded":"2024-04-22T13:24:06.412035+0000","last_fullsized":"2024-04-22T13:24:06.412035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:58:49.906188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.163728+0000","last_change":"2024-04-22T13:23:05.513123+0000","last_active":"2024-04-22T13:24:06.163728+0000","last_peered":"2024-04-22T13:24:06.163728+0000","last_clean":"2024-04-22T13:24:06.163728+0000","last_became_active":"2024-04-22T13:23:05.509367+0000","last_became_peered":"2024-04-22T13:23:05.509367+0000","last_unstale":"2024-04-22T13:24:06.163728+0000","last_undegraded":"2024-04-22T13:24:06.163728+0000","last_fullsized":"2024-04-22T13:24:06.163728+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:28:41.770957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258638+0000","last_change":"2024-04-22T13:22:05.250120+0000","last_active":"2024-04-22T13:24:06.258638+0000","last_peered":"2024-04-22T13:24:06.258638+0000","last_clean":"2024-04-22T13:24:06.258638+0000","last_became_active":"2024-04-22T13:22:05.247936+0000","last_became_peered":"2024-04-22T13:22:05.247936+0000","last_unstale":"2024-04-22T13:24:06.258638+0000","last_undegraded":"2024-04-22T13:24:06.258638+0000","last_fullsized":"2024-04-22T13:24:06.258638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:27:38.437918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258902+0000","last_change":"2024-04-22T13:22:04.244026+0000","last_active":"2024-04-22T13:24:06.258902+0000","last_peered":"2024-04-22T13:24:06.258902+0000","last_clean":"2024-04-22T13:24:06.258902+0000","last_became_active":"2024-04-22T13:22:04.243518+0000","last_became_peered":"2024-04-22T13:22:04.243518+0000","last_unstale":"2024-04-22T13:24:06.258902+0000","last_undegraded":"2024-04-22T13:24:06.258902+0000","last_fullsized":"2024-04-22T13:24:06.258902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:54:48.791874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418220+0000","last_change":"2024-04-22T13:22:05.252674+0000","last_active":"2024-04-22T13:24:06.418220+0000","last_peered":"2024-04-22T13:24:06.418220+0000","last_clean":"2024-04-22T13:24:06.418220+0000","last_became_active":"2024-04-22T13:22:05.251087+0000","last_became_peered":"2024-04-22T13:22:05.251087+0000","last_unstale":"2024-04-22T13:24:06.418220+0000","last_undegraded":"2024-04-22T13:24:06.418220+0000","last_fullsized":"2024-04-22T13:24:06.418220+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:20:06.873412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.414640+0000","last_change":"2024-04-22T13:22:04.244409+0000","last_active":"2024-04-22T13:24:06.414640+0000","last_peered":"2024-04-22T13:24:06.414640+0000","last_clean":"2024-04-22T13:24:06.414640+0000","last_became_active":"2024-04-22T13:22:04.244104+0000","last_became_peered":"2024-04-22T13:22:04.244104+0000","last_unstale":"2024-04-22T13:24:06.414640+0000","last_undegraded":"2024-04-22T13:24:06.414640+0000","last_fullsized":"2024-04-22T13:24:06.414640+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:39:14.714755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259066+0000","last_change":"2024-04-22T13:22:05.245730+0000","last_active":"2024-04-22T13:24:06.259066+0000","last_peered":"2024-04-22T13:24:06.259066+0000","last_clean":"2024-04-22T13:24:06.259066+0000","last_became_active":"2024-04-22T13:22:05.245366+0000","last_became_peered":"2024-04-22T13:22:05.245366+0000","last_unstale":"2024-04-22T13:24:06.259066+0000","last_undegraded":"2024-04-22T13:24:06.259066+0000","last_fullsized":"2024-04-22T13:24:06.259066+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:31.935258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418212+0000","last_change":"2024-04-22T13:23:05.514197+0000","last_active":"2024-04-22T13:24:06.418212+0000","last_peered":"2024-04-22T13:24:06.418212+0000","last_clean":"2024-04-22T13:24:06.418212+0000","last_became_active":"2024-04-22T13:23:05.509431+0000","last_became_peered":"2024-04-22T13:23:05.509431+0000","last_unstale":"2024-04-22T13:24:06.418212+0000","last_undegraded":"2024-04-22T13:24:06.418212+0000","last_fullsized":"2024-04-22T13:24:06.418212+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:43:59.825567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409273+0000","last_change":"2024-04-22T13:22:05.242146+0000","last_active":"2024-04-22T13:24:06.409273+0000","last_peered":"2024-04-22T13:24:06.409273+0000","last_clean":"2024-04-22T13:24:06.409273+0000","last_became_active":"2024-04-22T13:22:05.241802+0000","last_became_peered":"2024-04-22T13:22:05.241802+0000","last_unstale":"2024-04-22T13:24:06.409273+0000","last_undegraded":"2024-04-22T13:24:06.409273+0000","last_fullsized":"2024-04-22T13:24:06.409273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:49:49.002123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'7","reported_seq":147,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.562849+0000","last_change":"2024-04-22T13:22:04.248884+0000","last_active":"2024-04-22T13:24:06.562849+0000","last_peered":"2024-04-22T13:24:06.562849+0000","last_clean":"2024-04-22T13:24:06.562849+0000","last_became_active":"2024-04-22T13:22:04.248393+0000","last_became_peered":"2024-04-22T13:22:04.248393+0000","last_unstale":"2024-04-22T13:24:06.562849+0000","last_undegraded":"2024-04-22T13:24:06.562849+0000","last_fullsized":"2024-04-22T13:24:06.562849+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:36:14.687457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1869,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409545+0000","last_change":"2024-04-22T13:22:05.256881+0000","last_active":"2024-04-22T13:24:06.409545+0000","last_peered":"2024-04-22T13:24:06.409545+0000","last_clean":"2024-04-22T13:24:06.409545+0000","last_became_active":"2024-04-22T13:22:05.255967+0000","last_became_peered":"2024-04-22T13:22:05.255967+0000","last_unstale":"2024-04-22T13:24:06.409545+0000","last_undegraded":"2024-04-22T13:24:06.409545+0000","last_fullsized":"2024-04-22T13:24:06.409545+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:24:21.137344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'41","reported_seq":70,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.367821+0000","last_change":"2024-04-22T13:22:04.247303+0000","last_active":"2024-04-22T13:24:06.367821+0000","last_peered":"2024-04-22T13:24:06.367821+0000","last_clean":"2024-04-22T13:24:06.367821+0000","last_became_active":"2024-04-22T13:22:04.246742+0000","last_became_peered":"2024-04-22T13:22:04.246742+0000","last_unstale":"2024-04-22T13:24:06.367821+0000","last_undegraded":"2024-04-22T13:24:06.367821+0000","last_fullsized":"2024-04-22T13:24:06.367821+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:45:03.189042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409288+0000","last_change":"2024-04-22T13:22:05.251754+0000","last_active":"2024-04-22T13:24:06.409288+0000","last_peered":"2024-04-22T13:24:06.409288+0000","last_clean":"2024-04-22T13:24:06.409288+0000","last_became_active":"2024-04-22T13:22:05.251155+0000","last_became_peered":"2024-04-22T13:22:05.251155+0000","last_unstale":"2024-04-22T13:24:06.409288+0000","last_undegraded":"2024-04-22T13:24:06.409288+0000","last_fullsized":"2024-04-22T13:24:06.409288+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:08:46.682600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162485+0000","last_change":"2024-04-22T13:22:04.243382+0000","last_active":"2024-04-22T13:24:06.162485+0000","last_peered":"2024-04-22T13:24:06.162485+0000","last_clean":"2024-04-22T13:24:06.162485+0000","last_became_active":"2024-04-22T13:22:04.243055+0000","last_became_peered":"2024-04-22T13:22:04.243055+0000","last_unstale":"2024-04-22T13:24:06.162485+0000","last_undegraded":"2024-04-22T13:24:06.162485+0000","last_fullsized":"2024-04-22T13:24:06.162485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:19:58.379645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409542+0000","last_change":"2024-04-22T13:22:05.249527+0000","last_active":"2024-04-22T13:24:06.409542+0000","last_peered":"2024-04-22T13:24:06.409542+0000","last_clean":"2024-04-22T13:24:06.409542+0000","last_became_active":"2024-04-22T13:22:05.249207+0000","last_became_peered":"2024-04-22T13:22:05.249207+0000","last_unstale":"2024-04-22T13:24:06.409542+0000","last_undegraded":"2024-04-22T13:24:06.409542+0000","last_fullsized":"2024-04-22T13:24:06.409542+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:50:02.755288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418234+0000","last_change":"2024-04-22T13:23:05.508324+0000","last_active":"2024-04-22T13:24:06.418234+0000","last_peered":"2024-04-22T13:24:06.418234+0000","last_clean":"2024-04-22T13:24:06.418234+0000","last_became_active":"2024-04-22T13:23:05.507814+0000","last_became_peered":"2024-04-22T13:23:05.507814+0000","last_unstale":"2024-04-22T13:24:06.418234+0000","last_undegraded":"2024-04-22T13:24:06.418234+0000","last_fullsized":"2024-04-22T13:24:06.418234+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:46:20.523715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.414586+0000","last_change":"2024-04-22T13:23:05.875683+0000","last_active":"2024-04-22T13:24:06.414586+0000","last_peered":"2024-04-22T13:24:06.414586+0000","last_clean":"2024-04-22T13:24:06.414586+0000","last_became_active":"2024-04-22T13:23:05.875150+0000","last_became_peered":"2024-04-22T13:23:05.875150+0000","last_unstale":"2024-04-22T13:24:06.414586+0000","last_undegraded":"2024-04-22T13:24:06.414586+0000","last_fullsized":"2024-04-22T13:24:06.414586+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:12:47.592230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.414529+0000","last_change":"2024-04-22T13:22:05.247882+0000","last_active":"2024-04-22T13:24:06.414529+0000","last_peered":"2024-04-22T13:24:06.414529+0000","last_clean":"2024-04-22T13:24:06.414529+0000","last_became_active":"2024-04-22T13:22:05.247525+0000","last_became_peered":"2024-04-22T13:22:05.247525+0000","last_unstale":"2024-04-22T13:24:06.414529+0000","last_undegraded":"2024-04-22T13:24:06.414529+0000","last_fullsized":"2024-04-22T13:24:06.414529+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:46:04.349076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411567+0000","last_change":"2024-04-22T13:22:05.243716+0000","last_active":"2024-04-22T13:24:06.411567+0000","last_peered":"2024-04-22T13:24:06.411567+0000","last_clean":"2024-04-22T13:24:06.411567+0000","last_became_active":"2024-04-22T13:22:05.243397+0000","last_became_peered":"2024-04-22T13:22:05.243397+0000","last_unstale":"2024-04-22T13:24:06.411567+0000","last_undegraded":"2024-04-22T13:24:06.411567+0000","last_fullsized":"2024-04-22T13:24:06.411567+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:16:21.236339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162619+0000","last_change":"2024-04-22T13:22:05.246042+0000","last_active":"2024-04-22T13:24:06.162619+0000","last_peered":"2024-04-22T13:24:06.162619+0000","last_clean":"2024-04-22T13:24:06.162619+0000","last_became_active":"2024-04-22T13:22:05.245669+0000","last_became_peered":"2024-04-22T13:22:05.245669+0000","last_unstale":"2024-04-22T13:24:06.162619+0000","last_undegraded":"2024-04-22T13:24:06.162619+0000","last_fullsized":"2024-04-22T13:24:06.162619+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:49:05.539641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418321+0000","last_change":"2024-04-22T13:22:05.257009+0000","last_active":"2024-04-22T13:24:06.418321+0000","last_peered":"2024-04-22T13:24:06.418321+0000","last_clean":"2024-04-22T13:24:06.418321+0000","last_became_active":"2024-04-22T13:22:05.254327+0000","last_became_peered":"2024-04-22T13:22:05.254327+0000","last_unstale":"2024-04-22T13:24:06.418321+0000","last_undegraded":"2024-04-22T13:24:06.418321+0000","last_fullsized":"2024-04-22T13:24:06.418321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:29:12.482525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.415339+0000","last_change":"2024-04-22T13:22:05.260575+0000","last_active":"2024-04-22T13:24:06.415339+0000","last_peered":"2024-04-22T13:24:06.415339+0000","last_clean":"2024-04-22T13:24:06.415339+0000","last_became_active":"2024-04-22T13:22:05.260181+0000","last_became_peered":"2024-04-22T13:22:05.260181+0000","last_unstale":"2024-04-22T13:24:06.415339+0000","last_undegraded":"2024-04-22T13:24:06.415339+0000","last_fullsized":"2024-04-22T13:24:06.415339+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:49:20.064368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259116+0000","last_change":"2024-04-22T13:22:05.248199+0000","last_active":"2024-04-22T13:24:06.259116+0000","last_peered":"2024-04-22T13:24:06.259116+0000","last_clean":"2024-04-22T13:24:06.259116+0000","last_became_active":"2024-04-22T13:22:05.247858+0000","last_became_peered":"2024-04-22T13:22:05.247858+0000","last_unstale":"2024-04-22T13:24:06.259116+0000","last_undegraded":"2024-04-22T13:24:06.259116+0000","last_fullsized":"2024-04-22T13:24:06.259116+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:49:21.228105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164341+0000","last_change":"2024-04-22T13:22:05.239137+0000","last_active":"2024-04-22T13:24:06.164341+0000","last_peered":"2024-04-22T13:24:06.164341+0000","last_clean":"2024-04-22T13:24:06.164341+0000","last_became_active":"2024-04-22T13:22:05.238807+0000","last_became_peered":"2024-04-22T13:22:05.238807+0000","last_unstale":"2024-04-22T13:24:06.164341+0000","last_undegraded":"2024-04-22T13:24:06.164341+0000","last_fullsized":"2024-04-22T13:24:06.164341+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:02:48.587097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258756+0000","last_change":"2024-04-22T13:22:05.249080+0000","last_active":"2024-04-22T13:24:06.258756+0000","last_peered":"2024-04-22T13:24:06.258756+0000","last_clean":"2024-04-22T13:24:06.258756+0000","last_became_active":"2024-04-22T13:22:05.248733+0000","last_became_peered":"2024-04-22T13:22:05.248733+0000","last_unstale":"2024-04-22T13:24:06.258756+0000","last_undegraded":"2024-04-22T13:24:06.258756+0000","last_fullsized":"2024-04-22T13:24:06.258756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:58.735941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.163779+0000","last_change":"2024-04-22T13:22:05.255490+0000","last_active":"2024-04-22T13:24:06.163779+0000","last_peered":"2024-04-22T13:24:06.163779+0000","last_clean":"2024-04-22T13:24:06.163779+0000","last_became_active":"2024-04-22T13:22:05.253644+0000","last_became_peered":"2024-04-22T13:22:05.253644+0000","last_unstale":"2024-04-22T13:24:06.163779+0000","last_undegraded":"2024-04-22T13:24:06.163779+0000","last_fullsized":"2024-04-22T13:24:06.163779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:31:01.816182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162443+0000","last_change":"2024-04-22T13:22:05.240988+0000","last_active":"2024-04-22T13:24:06.162443+0000","last_peered":"2024-04-22T13:24:06.162443+0000","last_clean":"2024-04-22T13:24:06.162443+0000","last_became_active":"2024-04-22T13:22:05.240642+0000","last_became_peered":"2024-04-22T13:22:05.240642+0000","last_unstale":"2024-04-22T13:24:06.162443+0000","last_undegraded":"2024-04-22T13:24:06.162443+0000","last_fullsized":"2024-04-22T13:24:06.162443+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:41:29.001979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418222+0000","last_change":"2024-04-22T13:22:05.252212+0000","last_active":"2024-04-22T13:24:06.418222+0000","last_peered":"2024-04-22T13:24:06.418222+0000","last_clean":"2024-04-22T13:24:06.418222+0000","last_became_active":"2024-04-22T13:22:05.251852+0000","last_became_peered":"2024-04-22T13:22:05.251852+0000","last_unstale":"2024-04-22T13:24:06.418222+0000","last_undegraded":"2024-04-22T13:24:06.418222+0000","last_fullsized":"2024-04-22T13:24:06.418222+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:13:55.132501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":598760,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1300,"num_read_kb":764,"num_write":368,"num_write_kb":1462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2162688,"data_stored":1796280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":298,"ondisk_log_size":298,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318310,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":472759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579945,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28796,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716356,"statfs":{"total":95995035648,"available":95965548544,"internally_reserved":0,"allocated":1822720,"data_stored":1260339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743471,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28388,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716764,"statfs":{"total":95995035648,"available":95965966336,"internally_reserved":0,"allocated":1400832,"data_stored":865206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005108,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28384,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716768,"statfs":{"total":95995035648,"available":95965970432,"internally_reserved":0,"allocated":1396736,"data_stored":865584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234041,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32744,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712408,"statfs":{"total":95995035648,"available":95961505792,"internally_reserved":0,"allocated":1601536,"data_stored":1066591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430270,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":991232,"data_stored":472489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":538129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":473745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1077248,"data_stored":538657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117070,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36836,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708316,"statfs":{"total":95995035648,"available":95957315584,"internally_reserved":0,"allocated":1536000,"data_stored":996348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855433,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":991232,"data_stored":472491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691907,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28724,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716428,"statfs":{"total":95995035648,"available":95965622272,"internally_reserved":0,"allocated":1748992,"data_stored":1196164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":134811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":198986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T13:24:10.596 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T13:24:10.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.751+0000 7f2522b75640 1 -- 172.21.15.12:0/1404114492 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f251c0690e0 msgr2=0x7f251c069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:10.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.751+0000 7f2522b75640 1 --2- 172.21.15.12:0/1404114492 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f251c0690e0 0x7f251c069560 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2508009900 tx=0x7f250802bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:10.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.751+0000 7f2522b75640 1 -- 172.21.15.12:0/1404114492 shutdown_connections 2024-04-22T13:24:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.751+0000 7f2522b75640 1 --2- 172.21.15.12:0/1404114492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f251c069aa0 0x7f251c10ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.751+0000 7f2522b75640 1 --2- 172.21.15.12:0/1404114492 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f251c0690e0 0x7f251c069560 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.751+0000 7f2522b75640 1 --2- 172.21.15.12:0/1404114492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f251c106f50 0x7f251c107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.751+0000 7f2522b75640 1 -- 172.21.15.12:0/1404114492 >> 172.21.15.12:0/1404114492 conn(0x7f251c077e30 msgr2=0x7f251c078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.751+0000 7f2522b75640 1 -- 172.21.15.12:0/1404114492 shutdown_connections 2024-04-22T13:24:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.751+0000 7f2522b75640 1 -- 172.21.15.12:0/1404114492 wait complete. 2024-04-22T13:24:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.752+0000 7f2522b75640 1 Processor -- start 2024-04-22T13:24:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.752+0000 7f2522b75640 1 -- start start 2024-04-22T13:24:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.752+0000 7f2522b75640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f251c0690e0 0x7f251c1a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.752+0000 7f2522b75640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f251c069aa0 0x7f251c1a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.752+0000 7f2522b75640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f251c106f50 0x7f251c19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.752+0000 7f2522b75640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f251c112170 con 0x7f251c106f50 2024-04-22T13:24:10.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.752+0000 7f2522b75640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f251c111ff0 con 0x7f251c0690e0 2024-04-22T13:24:10.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.752+0000 7f2522b75640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f251c1122f0 con 0x7f251c069aa0 2024-04-22T13:24:10.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.752+0000 7f2521112640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f251c106f50 0x7f251c19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:10.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.752+0000 7f2521112640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f251c106f50 0x7f251c19a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35056/0 (socket says 172.21.15.12:35056) 2024-04-22T13:24:10.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.752+0000 7f2521112640 1 -- 172.21.15.12:0/3848925447 learned_addr learned my addr 172.21.15.12:0/3848925447 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:10.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.752+0000 7f2520911640 1 --2- 172.21.15.12:0/3848925447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f251c0690e0 0x7f251c1a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:10.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.752+0000 7f2521112640 1 -- 172.21.15.12:0/3848925447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f251c069aa0 msgr2=0x7f251c1a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:10.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.752+0000 7f2521112640 1 --2- 172.21.15.12:0/3848925447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f251c069aa0 0x7f251c1a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:10.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.752+0000 7f2521112640 1 -- 172.21.15.12:0/3848925447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f251c0690e0 msgr2=0x7f251c1a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:10.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.752+0000 7f2521112640 1 --2- 172.21.15.12:0/3848925447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f251c0690e0 0x7f251c1a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:10.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.752+0000 7f2521112640 1 -- 172.21.15.12:0/3848925447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f251c19aef0 con 0x7f251c106f50 2024-04-22T13:24:10.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.752+0000 7f2521112640 1 --2- 172.21.15.12:0/3848925447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f251c106f50 0x7f251c19a630 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f251400cce0 tx=0x7f251400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:10.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.753+0000 7f2519ffb640 1 -- 172.21.15.12:0/3848925447 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251400f450 con 0x7f251c106f50 2024-04-22T13:24:10.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.753+0000 7f2519ffb640 1 -- 172.21.15.12:0/3848925447 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f251400faa0 con 0x7f251c106f50 2024-04-22T13:24:10.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.753+0000 7f2522b75640 1 -- 172.21.15.12:0/3848925447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f251c19b1e0 con 0x7f251c106f50 2024-04-22T13:24:10.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.753+0000 7f2519ffb640 1 -- 172.21.15.12:0/3848925447 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2514018b70 con 0x7f251c106f50 2024-04-22T13:24:10.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.753+0000 7f2522b75640 1 -- 172.21.15.12:0/3848925447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f251c1a74f0 con 0x7f251c106f50 2024-04-22T13:24:10.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.754+0000 7f2522b75640 1 -- 172.21.15.12:0/3848925447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f251c103650 con 0x7f251c106f50 2024-04-22T13:24:10.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.758+0000 7f2519ffb640 1 -- 172.21.15.12:0/3848925447 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f2514018d10 con 0x7f251c106f50 2024-04-22T13:24:10.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.759+0000 7f2519ffb640 1 --2- 172.21.15.12:0/3848925447 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f250c0a3ec0 0x7f250c0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:10.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.759+0000 7f2520911640 1 --2- 172.21.15.12:0/3848925447 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f250c0a3ec0 0x7f250c0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:10.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.760+0000 7f2519ffb640 1 -- 172.21.15.12:0/3848925447 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f25140d0ee0 con 0x7f251c106f50 2024-04-22T13:24:10.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.760+0000 7f2520911640 1 --2- 172.21.15.12:0/3848925447 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f250c0a3ec0 0x7f250c0a6380 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2510009bd0 tx=0x7f2510005f90 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:10.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.766+0000 7f2519ffb640 1 -- 172.21.15.12:0/3848925447 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2514014030 con 0x7f251c106f50 2024-04-22T13:24:10.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.955+0000 7f2522b75640 1 -- 172.21.15.12:0/3848925447 --> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f251c100320 con 0x7f250c0a3ec0 2024-04-22T13:24:10.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.963+0000 7f2519ffb640 1 -- 172.21.15.12:0/3848925447 <== mgr.24968 v2:172.21.15.12:6800/1615661596 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257781 (secure 0 0 0) 0x7f2510004030 con 0x7f250c0a3ec0 2024-04-22T13:24:10.963 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:24:10.964 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-22T13:24:10.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.966+0000 7f2522b75640 1 -- 172.21.15.12:0/3848925447 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f250c0a3ec0 msgr2=0x7f250c0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:10.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.966+0000 7f2522b75640 1 --2- 172.21.15.12:0/3848925447 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f250c0a3ec0 0x7f250c0a6380 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2510009bd0 tx=0x7f2510005f90 comp rx=0 tx=0).stop 2024-04-22T13:24:10.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.966+0000 7f2522b75640 1 -- 172.21.15.12:0/3848925447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f251c106f50 msgr2=0x7f251c19a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:10.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.966+0000 7f2522b75640 1 --2- 172.21.15.12:0/3848925447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f251c106f50 0x7f251c19a630 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f251400cce0 tx=0x7f251400a620 comp rx=0 tx=0).stop 2024-04-22T13:24:10.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.967+0000 7f2522b75640 1 -- 172.21.15.12:0/3848925447 shutdown_connections 2024-04-22T13:24:10.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.967+0000 7f2522b75640 1 --2- 172.21.15.12:0/3848925447 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f250c0a3ec0 0x7f250c0a6380 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:10.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.967+0000 7f2522b75640 1 --2- 172.21.15.12:0/3848925447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f251c106f50 0x7f251c19a630 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:10.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.967+0000 7f2522b75640 1 --2- 172.21.15.12:0/3848925447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f251c069aa0 0x7f251c1a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:10.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.967+0000 7f2522b75640 1 --2- 172.21.15.12:0/3848925447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f251c0690e0 0x7f251c1a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:10.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.967+0000 7f2522b75640 1 -- 172.21.15.12:0/3848925447 >> 172.21.15.12:0/3848925447 conn(0x7f251c077e30 msgr2=0x7f251c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:10.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.967+0000 7f2522b75640 1 -- 172.21.15.12:0/3848925447 shutdown_connections 2024-04-22T13:24:10.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:10.967+0000 7f2522b75640 1 -- 172.21.15.12:0/3848925447 wait complete. 2024-04-22T13:24:10.974 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:10 smithi012 ceph-mon[28265]: Adjusting osd_memory_target on smithi111 to 176.2M 2024-04-22T13:24:10.974 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:10 smithi012 ceph-mon[28265]: Adjusting osd_memory_target on smithi134 to 176.2M 2024-04-22T13:24:10.974 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:10 smithi012 ceph-mon[28265]: Unable to set osd_memory_target on smithi111 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-22T13:24:10.974 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:10 smithi012 ceph-mon[28265]: Unable to set osd_memory_target on smithi134 to 184792473: error parsing value: Value '184792473' is below minimum 939524096 2024-04-22T13:24:10.974 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:10 smithi012 ceph-mon[28265]: Adjusting osd_memory_target on smithi012 to 177.2M 2024-04-22T13:24:10.974 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:10 smithi012 ceph-mon[28265]: Unable to set osd_memory_target on smithi012 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-04-22T13:24:10.974 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:10 smithi012 ceph-mon[28265]: Updating smithi012:/etc/ceph/ceph.conf 2024-04-22T13:24:10.974 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:10 smithi012 ceph-mon[28265]: Updating smithi111:/etc/ceph/ceph.conf 2024-04-22T13:24:10.975 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:10 smithi012 ceph-mon[28265]: Updating smithi134:/etc/ceph/ceph.conf 2024-04-22T13:24:10.975 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:10 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1922322927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:24:10.975 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:10 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/339579050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:10.976 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-22T13:24:10.176283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189362,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1436,"num_read_kb":883,"num_write":551,"num_write_kb":4402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":937,"ondisk_log_size":937,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376920,"kb_used_data":15252,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124564904,"statfs":{"total":1151940427776,"available":1151554461696,"internally_reserved":0,"allocated":15618048,"data_stored":9218502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.991258"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418486+0000","last_change":"2024-04-22T13:22:05.254388+0000","last_active":"2024-04-22T13:24:06.418486+0000","last_peered":"2024-04-22T13:24:06.418486+0000","last_clean":"2024-04-22T13:24:06.418486+0000","last_became_active":"2024-04-22T13:22:05.254042+0000","last_became_peered":"2024-04-22T13:22:05.254042+0000","last_unstale":"2024-04-22T13:24:06.418486+0000","last_undegraded":"2024-04-22T13:24:06.418486+0000","last_fullsized":"2024-04-22T13:24:06.418486+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:07:56.139653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.412013+0000","last_change":"2024-04-22T13:22:05.247879+0000","last_active":"2024-04-22T13:24:06.412013+0000","last_peered":"2024-04-22T13:24:06.412013+0000","last_clean":"2024-04-22T13:24:06.412013+0000","last_became_active":"2024-04-22T13:22:05.247508+0000","last_became_peered":"2024-04-22T13:22:05.247508+0000","last_unstale":"2024-04-22T13:24:06.412013+0000","last_undegraded":"2024-04-22T13:24:06.412013+0000","last_fullsized":"2024-04-22T13:24:06.412013+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:55:51.928658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411868+0000","last_change":"2024-04-22T13:22:05.257970+0000","last_active":"2024-04-22T13:24:06.411868+0000","last_peered":"2024-04-22T13:24:06.411868+0000","last_clean":"2024-04-22T13:24:06.411868+0000","last_became_active":"2024-04-22T13:22:05.257652+0000","last_became_peered":"2024-04-22T13:22:05.257652+0000","last_unstale":"2024-04-22T13:24:06.411868+0000","last_undegraded":"2024-04-22T13:24:06.411868+0000","last_fullsized":"2024-04-22T13:24:06.411868+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:42:17.476349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411332+0000","last_change":"2024-04-22T13:22:05.249186+0000","last_active":"2024-04-22T13:24:06.411332+0000","last_peered":"2024-04-22T13:24:06.411332+0000","last_clean":"2024-04-22T13:24:06.411332+0000","last_became_active":"2024-04-22T13:22:05.248814+0000","last_became_peered":"2024-04-22T13:22:05.248814+0000","last_unstale":"2024-04-22T13:24:06.411332+0000","last_undegraded":"2024-04-22T13:24:06.411332+0000","last_fullsized":"2024-04-22T13:24:06.411332+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:57.450794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258738+0000","last_change":"2024-04-22T13:22:05.251953+0000","last_active":"2024-04-22T13:24:06.258738+0000","last_peered":"2024-04-22T13:24:06.258738+0000","last_clean":"2024-04-22T13:24:06.258738+0000","last_became_active":"2024-04-22T13:22:05.251547+0000","last_became_peered":"2024-04-22T13:22:05.251547+0000","last_unstale":"2024-04-22T13:24:06.258738+0000","last_undegraded":"2024-04-22T13:24:06.258738+0000","last_fullsized":"2024-04-22T13:24:06.258738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:36:34.809878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.266346+0000","last_change":"2024-04-22T13:23:05.510044+0000","last_active":"2024-04-22T13:24:06.266346+0000","last_peered":"2024-04-22T13:24:06.266346+0000","last_clean":"2024-04-22T13:24:06.266346+0000","last_became_active":"2024-04-22T13:23:05.509677+0000","last_became_peered":"2024-04-22T13:23:05.509677+0000","last_unstale":"2024-04-22T13:24:06.266346+0000","last_undegraded":"2024-04-22T13:24:06.266346+0000","last_fullsized":"2024-04-22T13:24:06.266346+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:46.979291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409036+0000","last_change":"2024-04-22T13:22:05.250418+0000","last_active":"2024-04-22T13:24:06.409036+0000","last_peered":"2024-04-22T13:24:06.409036+0000","last_clean":"2024-04-22T13:24:06.409036+0000","last_became_active":"2024-04-22T13:22:05.250097+0000","last_became_peered":"2024-04-22T13:22:05.250097+0000","last_unstale":"2024-04-22T13:24:06.409036+0000","last_undegraded":"2024-04-22T13:24:06.409036+0000","last_fullsized":"2024-04-22T13:24:06.409036+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:34:59.598950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258759+0000","last_change":"2024-04-22T13:22:05.244762+0000","last_active":"2024-04-22T13:24:06.258759+0000","last_peered":"2024-04-22T13:24:06.258759+0000","last_clean":"2024-04-22T13:24:06.258759+0000","last_became_active":"2024-04-22T13:22:05.244430+0000","last_became_peered":"2024-04-22T13:22:05.244430+0000","last_unstale":"2024-04-22T13:24:06.258759+0000","last_undegraded":"2024-04-22T13:24:06.258759+0000","last_fullsized":"2024-04-22T13:24:06.258759+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:33:01.582851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164321+0000","last_change":"2024-04-22T13:22:05.257596+0000","last_active":"2024-04-22T13:24:06.164321+0000","last_peered":"2024-04-22T13:24:06.164321+0000","last_clean":"2024-04-22T13:24:06.164321+0000","last_became_active":"2024-04-22T13:22:05.257277+0000","last_became_peered":"2024-04-22T13:22:05.257277+0000","last_unstale":"2024-04-22T13:24:06.164321+0000","last_undegraded":"2024-04-22T13:24:06.164321+0000","last_fullsized":"2024-04-22T13:24:06.164321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:09:44.715965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.266708+0000","last_change":"2024-04-22T13:22:05.252177+0000","last_active":"2024-04-22T13:24:06.266708+0000","last_peered":"2024-04-22T13:24:06.266708+0000","last_clean":"2024-04-22T13:24:06.266708+0000","last_became_active":"2024-04-22T13:22:05.251794+0000","last_became_peered":"2024-04-22T13:22:05.251794+0000","last_unstale":"2024-04-22T13:24:06.266708+0000","last_undegraded":"2024-04-22T13:24:06.266708+0000","last_fullsized":"2024-04-22T13:24:06.266708+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:08:59.500540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409476+0000","last_change":"2024-04-22T13:22:05.256209+0000","last_active":"2024-04-22T13:24:06.409476+0000","last_peered":"2024-04-22T13:24:06.409476+0000","last_clean":"2024-04-22T13:24:06.409476+0000","last_became_active":"2024-04-22T13:22:05.255504+0000","last_became_peered":"2024-04-22T13:22:05.255504+0000","last_unstale":"2024-04-22T13:24:06.409476+0000","last_undegraded":"2024-04-22T13:24:06.409476+0000","last_fullsized":"2024-04-22T13:24:06.409476+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:32:08.190921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.256744+0000","last_change":"2024-04-22T13:23:05.509837+0000","last_active":"2024-04-22T13:24:06.256744+0000","last_peered":"2024-04-22T13:24:06.256744+0000","last_clean":"2024-04-22T13:24:06.256744+0000","last_became_active":"2024-04-22T13:23:05.509421+0000","last_became_peered":"2024-04-22T13:23:05.509421+0000","last_unstale":"2024-04-22T13:24:06.256744+0000","last_undegraded":"2024-04-22T13:24:06.256744+0000","last_fullsized":"2024-04-22T13:24:06.256744+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:30:01.103373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162449+0000","last_change":"2024-04-22T13:22:05.246667+0000","last_active":"2024-04-22T13:24:06.162449+0000","last_peered":"2024-04-22T13:24:06.162449+0000","last_clean":"2024-04-22T13:24:06.162449+0000","last_became_active":"2024-04-22T13:22:05.246365+0000","last_became_peered":"2024-04-22T13:22:05.246365+0000","last_unstale":"2024-04-22T13:24:06.162449+0000","last_undegraded":"2024-04-22T13:24:06.162449+0000","last_fullsized":"2024-04-22T13:24:06.162449+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:41:55.533430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411527+0000","last_change":"2024-04-22T13:22:05.251982+0000","last_active":"2024-04-22T13:24:06.411527+0000","last_peered":"2024-04-22T13:24:06.411527+0000","last_clean":"2024-04-22T13:24:06.411527+0000","last_became_active":"2024-04-22T13:22:05.251575+0000","last_became_peered":"2024-04-22T13:22:05.251575+0000","last_unstale":"2024-04-22T13:24:06.411527+0000","last_undegraded":"2024-04-22T13:24:06.411527+0000","last_fullsized":"2024-04-22T13:24:06.411527+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:28:05.343140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411427+0000","last_change":"2024-04-22T13:22:05.251056+0000","last_active":"2024-04-22T13:24:06.411427+0000","last_peered":"2024-04-22T13:24:06.411427+0000","last_clean":"2024-04-22T13:24:06.411427+0000","last_became_active":"2024-04-22T13:22:05.250652+0000","last_became_peered":"2024-04-22T13:22:05.250652+0000","last_unstale":"2024-04-22T13:24:06.411427+0000","last_undegraded":"2024-04-22T13:24:06.411427+0000","last_fullsized":"2024-04-22T13:24:06.411427+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:08:28.629965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.163636+0000","last_change":"2024-04-22T13:22:05.256877+0000","last_active":"2024-04-22T13:24:06.163636+0000","last_peered":"2024-04-22T13:24:06.163636+0000","last_clean":"2024-04-22T13:24:06.163636+0000","last_became_active":"2024-04-22T13:22:05.256074+0000","last_became_peered":"2024-04-22T13:22:05.256074+0000","last_unstale":"2024-04-22T13:24:06.163636+0000","last_undegraded":"2024-04-22T13:24:06.163636+0000","last_fullsized":"2024-04-22T13:24:06.163636+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:08:53.607626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418592+0000","last_change":"2024-04-22T13:22:05.255598+0000","last_active":"2024-04-22T13:24:06.418592+0000","last_peered":"2024-04-22T13:24:06.418592+0000","last_clean":"2024-04-22T13:24:06.418592+0000","last_became_active":"2024-04-22T13:22:05.255239+0000","last_became_peered":"2024-04-22T13:22:05.255239+0000","last_unstale":"2024-04-22T13:24:06.418592+0000","last_undegraded":"2024-04-22T13:24:06.418592+0000","last_fullsized":"2024-04-22T13:24:06.418592+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:38:20.547882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.257085+0000","last_change":"2024-04-22T13:22:05.254232+0000","last_active":"2024-04-22T13:24:06.257085+0000","last_peered":"2024-04-22T13:24:06.257085+0000","last_clean":"2024-04-22T13:24:06.257085+0000","last_became_active":"2024-04-22T13:22:05.253907+0000","last_became_peered":"2024-04-22T13:22:05.253907+0000","last_unstale":"2024-04-22T13:24:06.257085+0000","last_undegraded":"2024-04-22T13:24:06.257085+0000","last_fullsized":"2024-04-22T13:24:06.257085+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:15:49.931112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.415196+0000","last_change":"2024-04-22T13:22:05.251518+0000","last_active":"2024-04-22T13:24:06.415196+0000","last_peered":"2024-04-22T13:24:06.415196+0000","last_clean":"2024-04-22T13:24:06.415196+0000","last_became_active":"2024-04-22T13:22:05.250478+0000","last_became_peered":"2024-04-22T13:22:05.250478+0000","last_unstale":"2024-04-22T13:24:06.415196+0000","last_undegraded":"2024-04-22T13:24:06.415196+0000","last_fullsized":"2024-04-22T13:24:06.415196+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:25:26.758692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'14","reported_seq":152,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.559609+0000","last_change":"2024-04-22T13:22:04.241365+0000","last_active":"2024-04-22T13:24:06.559609+0000","last_peered":"2024-04-22T13:24:06.559609+0000","last_clean":"2024-04-22T13:24:06.559609+0000","last_became_active":"2024-04-22T13:22:04.240822+0000","last_became_peered":"2024-04-22T13:22:04.240822+0000","last_unstale":"2024-04-22T13:24:06.559609+0000","last_undegraded":"2024-04-22T13:24:06.559609+0000","last_fullsized":"2024-04-22T13:24:06.559609+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:20:23.928534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.256956+0000","last_change":"2024-04-22T13:23:05.876262+0000","last_active":"2024-04-22T13:24:06.256956+0000","last_peered":"2024-04-22T13:24:06.256956+0000","last_clean":"2024-04-22T13:24:06.256956+0000","last_became_active":"2024-04-22T13:23:05.875872+0000","last_became_peered":"2024-04-22T13:23:05.875872+0000","last_unstale":"2024-04-22T13:24:06.256956+0000","last_undegraded":"2024-04-22T13:24:06.256956+0000","last_fullsized":"2024-04-22T13:24:06.256956+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:29:33.092887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259476+0000","last_change":"2024-04-22T13:22:04.248872+0000","last_active":"2024-04-22T13:24:06.259476+0000","last_peered":"2024-04-22T13:24:06.259476+0000","last_clean":"2024-04-22T13:24:06.259476+0000","last_became_active":"2024-04-22T13:22:04.248466+0000","last_became_peered":"2024-04-22T13:22:04.248466+0000","last_unstale":"2024-04-22T13:24:06.259476+0000","last_undegraded":"2024-04-22T13:24:06.259476+0000","last_fullsized":"2024-04-22T13:24:06.259476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:38:10.871602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.412391+0000","last_change":"2024-04-22T13:22:05.258005+0000","last_active":"2024-04-22T13:24:06.412391+0000","last_peered":"2024-04-22T13:24:06.412391+0000","last_clean":"2024-04-22T13:24:06.412391+0000","last_became_active":"2024-04-22T13:22:05.257615+0000","last_became_peered":"2024-04-22T13:22:05.257615+0000","last_unstale":"2024-04-22T13:24:06.412391+0000","last_undegraded":"2024-04-22T13:24:06.412391+0000","last_fullsized":"2024-04-22T13:24:06.412391+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:21:18.030530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259060+0000","last_change":"2024-04-22T13:22:04.246273+0000","last_active":"2024-04-22T13:24:06.259060+0000","last_peered":"2024-04-22T13:24:06.259060+0000","last_clean":"2024-04-22T13:24:06.259060+0000","last_became_active":"2024-04-22T13:22:04.245969+0000","last_became_peered":"2024-04-22T13:22:04.245969+0000","last_unstale":"2024-04-22T13:24:06.259060+0000","last_undegraded":"2024-04-22T13:24:06.259060+0000","last_fullsized":"2024-04-22T13:24:06.259060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:10.260192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.163964+0000","last_change":"2024-04-22T13:22:05.260299+0000","last_active":"2024-04-22T13:24:06.163964+0000","last_peered":"2024-04-22T13:24:06.163964+0000","last_clean":"2024-04-22T13:24:06.163964+0000","last_became_active":"2024-04-22T13:22:05.259634+0000","last_became_peered":"2024-04-22T13:22:05.259634+0000","last_unstale":"2024-04-22T13:24:06.163964+0000","last_undegraded":"2024-04-22T13:24:06.163964+0000","last_fullsized":"2024-04-22T13:24:06.163964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:16:44.421652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258970+0000","last_change":"2024-04-22T13:22:04.243843+0000","last_active":"2024-04-22T13:24:06.258970+0000","last_peered":"2024-04-22T13:24:06.258970+0000","last_clean":"2024-04-22T13:24:06.258970+0000","last_became_active":"2024-04-22T13:22:04.243465+0000","last_became_peered":"2024-04-22T13:22:04.243465+0000","last_unstale":"2024-04-22T13:24:06.258970+0000","last_undegraded":"2024-04-22T13:24:06.258970+0000","last_fullsized":"2024-04-22T13:24:06.258970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:48:36.431697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164717+0000","last_change":"2024-04-22T13:22:05.259789+0000","last_active":"2024-04-22T13:24:06.164717+0000","last_peered":"2024-04-22T13:24:06.164717+0000","last_clean":"2024-04-22T13:24:06.164717+0000","last_became_active":"2024-04-22T13:22:05.259395+0000","last_became_peered":"2024-04-22T13:22:05.259395+0000","last_unstale":"2024-04-22T13:24:06.164717+0000","last_undegraded":"2024-04-22T13:24:06.164717+0000","last_fullsized":"2024-04-22T13:24:06.164717+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:43:51.430561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":148,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.533918+0000","last_change":"2024-04-22T13:22:04.240615+0000","last_active":"2024-04-22T13:24:06.533918+0000","last_peered":"2024-04-22T13:24:06.533918+0000","last_clean":"2024-04-22T13:24:06.533918+0000","last_became_active":"2024-04-22T13:22:04.240248+0000","last_became_peered":"2024-04-22T13:22:04.240248+0000","last_unstale":"2024-04-22T13:24:06.533918+0000","last_undegraded":"2024-04-22T13:24:06.533918+0000","last_fullsized":"2024-04-22T13:24:06.533918+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:55:05.948979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258665+0000","last_change":"2024-04-22T13:22:05.253788+0000","last_active":"2024-04-22T13:24:06.258665+0000","last_peered":"2024-04-22T13:24:06.258665+0000","last_clean":"2024-04-22T13:24:06.258665+0000","last_became_active":"2024-04-22T13:22:05.252578+0000","last_became_peered":"2024-04-22T13:22:05.252578+0000","last_unstale":"2024-04-22T13:24:06.258665+0000","last_undegraded":"2024-04-22T13:24:06.258665+0000","last_fullsized":"2024-04-22T13:24:06.258665+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:04:11.769930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.266274+0000","last_change":"2024-04-22T13:22:04.241170+0000","last_active":"2024-04-22T13:24:06.266274+0000","last_peered":"2024-04-22T13:24:06.266274+0000","last_clean":"2024-04-22T13:24:06.266274+0000","last_became_active":"2024-04-22T13:22:04.240830+0000","last_became_peered":"2024-04-22T13:22:04.240830+0000","last_unstale":"2024-04-22T13:24:06.266274+0000","last_undegraded":"2024-04-22T13:24:06.266274+0000","last_fullsized":"2024-04-22T13:24:06.266274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:51:30.520613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164733+0000","last_change":"2024-04-22T13:22:05.237900+0000","last_active":"2024-04-22T13:24:06.164733+0000","last_peered":"2024-04-22T13:24:06.164733+0000","last_clean":"2024-04-22T13:24:06.164733+0000","last_became_active":"2024-04-22T13:22:05.237507+0000","last_became_peered":"2024-04-22T13:22:05.237507+0000","last_unstale":"2024-04-22T13:24:06.164733+0000","last_undegraded":"2024-04-22T13:24:06.164733+0000","last_fullsized":"2024-04-22T13:24:06.164733+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:26:24.307803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162328+0000","last_change":"2024-04-22T13:22:04.236078+0000","last_active":"2024-04-22T13:24:06.162328+0000","last_peered":"2024-04-22T13:24:06.162328+0000","last_clean":"2024-04-22T13:24:06.162328+0000","last_became_active":"2024-04-22T13:22:04.235710+0000","last_became_peered":"2024-04-22T13:22:04.235710+0000","last_unstale":"2024-04-22T13:24:06.162328+0000","last_undegraded":"2024-04-22T13:24:06.162328+0000","last_fullsized":"2024-04-22T13:24:06.162328+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:37:55.335237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258717+0000","last_change":"2024-04-22T13:22:05.243152+0000","last_active":"2024-04-22T13:24:06.258717+0000","last_peered":"2024-04-22T13:24:06.258717+0000","last_clean":"2024-04-22T13:24:06.258717+0000","last_became_active":"2024-04-22T13:22:05.242772+0000","last_became_peered":"2024-04-22T13:22:05.242772+0000","last_unstale":"2024-04-22T13:24:06.258717+0000","last_undegraded":"2024-04-22T13:24:06.258717+0000","last_fullsized":"2024-04-22T13:24:06.258717+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:33:42.359679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164258+0000","last_change":"2024-04-22T13:22:04.244381+0000","last_active":"2024-04-22T13:24:06.164258+0000","last_peered":"2024-04-22T13:24:06.164258+0000","last_clean":"2024-04-22T13:24:06.164258+0000","last_became_active":"2024-04-22T13:22:04.243746+0000","last_became_peered":"2024-04-22T13:22:04.243746+0000","last_unstale":"2024-04-22T13:24:06.164258+0000","last_undegraded":"2024-04-22T13:24:06.164258+0000","last_fullsized":"2024-04-22T13:24:06.164258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:15:36.019930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.256753+0000","last_change":"2024-04-22T13:22:05.245394+0000","last_active":"2024-04-22T13:24:06.256753+0000","last_peered":"2024-04-22T13:24:06.256753+0000","last_clean":"2024-04-22T13:24:06.256753+0000","last_became_active":"2024-04-22T13:22:05.243931+0000","last_became_peered":"2024-04-22T13:22:05.243931+0000","last_unstale":"2024-04-22T13:24:06.256753+0000","last_undegraded":"2024-04-22T13:24:06.256753+0000","last_fullsized":"2024-04-22T13:24:06.256753+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:11:10.790842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'6","reported_seq":260,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.533504+0000","last_change":"2024-04-22T13:22:04.242583+0000","last_active":"2024-04-22T13:24:06.533504+0000","last_peered":"2024-04-22T13:24:06.533504+0000","last_clean":"2024-04-22T13:24:06.533504+0000","last_became_active":"2024-04-22T13:22:04.242261+0000","last_became_peered":"2024-04-22T13:22:04.242261+0000","last_unstale":"2024-04-22T13:24:06.533504+0000","last_undegraded":"2024-04-22T13:24:06.533504+0000","last_fullsized":"2024-04-22T13:24:06.533504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:37:47.618573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":233,"num_read_kb":233,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164258+0000","last_change":"2024-04-22T13:22:05.255293+0000","last_active":"2024-04-22T13:24:06.164258+0000","last_peered":"2024-04-22T13:24:06.164258+0000","last_clean":"2024-04-22T13:24:06.164258+0000","last_became_active":"2024-04-22T13:22:05.253949+0000","last_became_peered":"2024-04-22T13:22:05.253949+0000","last_unstale":"2024-04-22T13:24:06.164258+0000","last_undegraded":"2024-04-22T13:24:06.164258+0000","last_fullsized":"2024-04-22T13:24:06.164258+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:41:08.754989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409503+0000","last_change":"2024-04-22T13:22:04.241529+0000","last_active":"2024-04-22T13:24:06.409503+0000","last_peered":"2024-04-22T13:24:06.409503+0000","last_clean":"2024-04-22T13:24:06.409503+0000","last_became_active":"2024-04-22T13:22:04.241164+0000","last_became_peered":"2024-04-22T13:22:04.241164+0000","last_unstale":"2024-04-22T13:24:06.409503+0000","last_undegraded":"2024-04-22T13:24:06.409503+0000","last_fullsized":"2024-04-22T13:24:06.409503+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:16:49.387181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.412225+0000","last_change":"2024-04-22T13:22:05.241623+0000","last_active":"2024-04-22T13:24:06.412225+0000","last_peered":"2024-04-22T13:24:06.412225+0000","last_clean":"2024-04-22T13:24:06.412225+0000","last_became_active":"2024-04-22T13:22:05.241150+0000","last_became_peered":"2024-04-22T13:22:05.241150+0000","last_unstale":"2024-04-22T13:24:06.412225+0000","last_undegraded":"2024-04-22T13:24:06.412225+0000","last_fullsized":"2024-04-22T13:24:06.412225+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:17:28.412366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164893+0000","last_change":"2024-04-22T13:22:04.247852+0000","last_active":"2024-04-22T13:24:06.164893+0000","last_peered":"2024-04-22T13:24:06.164893+0000","last_clean":"2024-04-22T13:24:06.164893+0000","last_became_active":"2024-04-22T13:22:04.247550+0000","last_became_peered":"2024-04-22T13:22:04.247550+0000","last_unstale":"2024-04-22T13:24:06.164893+0000","last_undegraded":"2024-04-22T13:24:06.164893+0000","last_fullsized":"2024-04-22T13:24:06.164893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:55:01.928435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164057+0000","last_change":"2024-04-22T13:22:05.260050+0000","last_active":"2024-04-22T13:24:06.164057+0000","last_peered":"2024-04-22T13:24:06.164057+0000","last_clean":"2024-04-22T13:24:06.164057+0000","last_became_active":"2024-04-22T13:22:05.259098+0000","last_became_peered":"2024-04-22T13:22:05.259098+0000","last_unstale":"2024-04-22T13:24:06.164057+0000","last_undegraded":"2024-04-22T13:24:06.164057+0000","last_fullsized":"2024-04-22T13:24:06.164057+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:05:06.245492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'36","reported_seq":305,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.534247+0000","last_change":"2024-04-22T13:22:04.240233+0000","last_active":"2024-04-22T13:24:06.534247+0000","last_peered":"2024-04-22T13:24:06.534247+0000","last_clean":"2024-04-22T13:24:06.534247+0000","last_became_active":"2024-04-22T13:22:04.239890+0000","last_became_peered":"2024-04-22T13:22:04.239890+0000","last_unstale":"2024-04-22T13:24:06.534247+0000","last_undegraded":"2024-04-22T13:24:06.534247+0000","last_fullsized":"2024-04-22T13:24:06.534247+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:04:21.547422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64727,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":248,"num_read_kb":188,"num_write":38,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258900+0000","last_change":"2024-04-22T13:22:05.257756+0000","last_active":"2024-04-22T13:24:06.258900+0000","last_peered":"2024-04-22T13:24:06.258900+0000","last_clean":"2024-04-22T13:24:06.258900+0000","last_became_active":"2024-04-22T13:22:05.257442+0000","last_became_peered":"2024-04-22T13:22:05.257442+0000","last_unstale":"2024-04-22T13:24:06.258900+0000","last_undegraded":"2024-04-22T13:24:06.258900+0000","last_fullsized":"2024-04-22T13:24:06.258900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:16:53.465689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.414923+0000","last_change":"2024-04-22T13:22:04.240815+0000","last_active":"2024-04-22T13:24:06.414923+0000","last_peered":"2024-04-22T13:24:06.414923+0000","last_clean":"2024-04-22T13:24:06.414923+0000","last_became_active":"2024-04-22T13:22:04.240218+0000","last_became_peered":"2024-04-22T13:22:04.240218+0000","last_unstale":"2024-04-22T13:24:06.414923+0000","last_undegraded":"2024-04-22T13:24:06.414923+0000","last_fullsized":"2024-04-22T13:24:06.414923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:29:29.148764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162257+0000","last_change":"2024-04-22T13:22:05.248797+0000","last_active":"2024-04-22T13:24:06.162257+0000","last_peered":"2024-04-22T13:24:06.162257+0000","last_clean":"2024-04-22T13:24:06.162257+0000","last_became_active":"2024-04-22T13:22:05.248435+0000","last_became_peered":"2024-04-22T13:22:05.248435+0000","last_unstale":"2024-04-22T13:24:06.162257+0000","last_undegraded":"2024-04-22T13:24:06.162257+0000","last_fullsized":"2024-04-22T13:24:06.162257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:32:31.970494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164389+0000","last_change":"2024-04-22T13:22:04.245303+0000","last_active":"2024-04-22T13:24:06.164389+0000","last_peered":"2024-04-22T13:24:06.164389+0000","last_clean":"2024-04-22T13:24:06.164389+0000","last_became_active":"2024-04-22T13:22:04.244903+0000","last_became_peered":"2024-04-22T13:22:04.244903+0000","last_unstale":"2024-04-22T13:24:06.164389+0000","last_undegraded":"2024-04-22T13:24:06.164389+0000","last_fullsized":"2024-04-22T13:24:06.164389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:34:32.174858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409264+0000","last_change":"2024-04-22T13:22:05.246318+0000","last_active":"2024-04-22T13:24:06.409264+0000","last_peered":"2024-04-22T13:24:06.409264+0000","last_clean":"2024-04-22T13:24:06.409264+0000","last_became_active":"2024-04-22T13:22:05.245899+0000","last_became_peered":"2024-04-22T13:22:05.245899+0000","last_unstale":"2024-04-22T13:24:06.409264+0000","last_undegraded":"2024-04-22T13:24:06.409264+0000","last_fullsized":"2024-04-22T13:24:06.409264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:27:42.789058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'8","reported_seq":148,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.591632+0000","last_change":"2024-04-22T13:22:04.240612+0000","last_active":"2024-04-22T13:24:06.591632+0000","last_peered":"2024-04-22T13:24:06.591632+0000","last_clean":"2024-04-22T13:24:06.591632+0000","last_became_active":"2024-04-22T13:22:04.240290+0000","last_became_peered":"2024-04-22T13:22:04.240290+0000","last_unstale":"2024-04-22T13:24:06.591632+0000","last_undegraded":"2024-04-22T13:24:06.591632+0000","last_fullsized":"2024-04-22T13:24:06.591632+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:00:50.285553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.408976+0000","last_change":"2024-04-22T13:22:05.253205+0000","last_active":"2024-04-22T13:24:06.408976+0000","last_peered":"2024-04-22T13:24:06.408976+0000","last_clean":"2024-04-22T13:24:06.408976+0000","last_became_active":"2024-04-22T13:22:05.252865+0000","last_became_peered":"2024-04-22T13:22:05.252865+0000","last_unstale":"2024-04-22T13:24:06.408976+0000","last_undegraded":"2024-04-22T13:24:06.408976+0000","last_fullsized":"2024-04-22T13:24:06.408976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:19:12.482371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258636+0000","last_change":"2024-04-22T13:22:04.248938+0000","last_active":"2024-04-22T13:24:06.258636+0000","last_peered":"2024-04-22T13:24:06.258636+0000","last_clean":"2024-04-22T13:24:06.258636+0000","last_became_active":"2024-04-22T13:22:04.248440+0000","last_became_peered":"2024-04-22T13:22:04.248440+0000","last_unstale":"2024-04-22T13:24:06.258636+0000","last_undegraded":"2024-04-22T13:24:06.258636+0000","last_fullsized":"2024-04-22T13:24:06.258636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:46:50.928067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164391+0000","last_change":"2024-04-22T13:22:05.254664+0000","last_active":"2024-04-22T13:24:06.164391+0000","last_peered":"2024-04-22T13:24:06.164391+0000","last_clean":"2024-04-22T13:24:06.164391+0000","last_became_active":"2024-04-22T13:22:05.254304+0000","last_became_peered":"2024-04-22T13:22:05.254304+0000","last_unstale":"2024-04-22T13:24:06.164391+0000","last_undegraded":"2024-04-22T13:24:06.164391+0000","last_fullsized":"2024-04-22T13:24:06.164391+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:58:49.345573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259033+0000","last_change":"2024-04-22T13:22:04.244128+0000","last_active":"2024-04-22T13:24:06.259033+0000","last_peered":"2024-04-22T13:24:06.259033+0000","last_clean":"2024-04-22T13:24:06.259033+0000","last_became_active":"2024-04-22T13:22:04.243738+0000","last_became_peered":"2024-04-22T13:22:04.243738+0000","last_unstale":"2024-04-22T13:24:06.259033+0000","last_undegraded":"2024-04-22T13:24:06.259033+0000","last_fullsized":"2024-04-22T13:24:06.259033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:23:52.220584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418387+0000","last_change":"2024-04-22T13:22:05.250910+0000","last_active":"2024-04-22T13:24:06.418387+0000","last_peered":"2024-04-22T13:24:06.418387+0000","last_clean":"2024-04-22T13:24:06.418387+0000","last_became_active":"2024-04-22T13:22:05.250410+0000","last_became_peered":"2024-04-22T13:22:05.250410+0000","last_unstale":"2024-04-22T13:24:06.418387+0000","last_undegraded":"2024-04-22T13:24:06.418387+0000","last_fullsized":"2024-04-22T13:24:06.418387+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:04:25.492677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.256761+0000","last_change":"2024-04-22T13:22:04.245889+0000","last_active":"2024-04-22T13:24:06.256761+0000","last_peered":"2024-04-22T13:24:06.256761+0000","last_clean":"2024-04-22T13:24:06.256761+0000","last_became_active":"2024-04-22T13:22:04.245474+0000","last_became_peered":"2024-04-22T13:22:04.245474+0000","last_unstale":"2024-04-22T13:24:06.256761+0000","last_undegraded":"2024-04-22T13:24:06.256761+0000","last_fullsized":"2024-04-22T13:24:06.256761+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:22:12.626950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.266594+0000","last_change":"2024-04-22T13:22:05.250978+0000","last_active":"2024-04-22T13:24:06.266594+0000","last_peered":"2024-04-22T13:24:06.266594+0000","last_clean":"2024-04-22T13:24:06.266594+0000","last_became_active":"2024-04-22T13:22:05.250463+0000","last_became_peered":"2024-04-22T13:22:05.250463+0000","last_unstale":"2024-04-22T13:24:06.266594+0000","last_undegraded":"2024-04-22T13:24:06.266594+0000","last_fullsized":"2024-04-22T13:24:06.266594+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:45:18.591072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'7","reported_seq":146,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.560650+0000","last_change":"2024-04-22T13:22:04.242503+0000","last_active":"2024-04-22T13:24:06.560650+0000","last_peered":"2024-04-22T13:24:06.560650+0000","last_clean":"2024-04-22T13:24:06.560650+0000","last_became_active":"2024-04-22T13:22:04.242143+0000","last_became_peered":"2024-04-22T13:22:04.242143+0000","last_unstale":"2024-04-22T13:24:06.560650+0000","last_undegraded":"2024-04-22T13:24:06.560650+0000","last_fullsized":"2024-04-22T13:24:06.560650+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:40.236377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.266483+0000","last_change":"2024-04-22T13:22:05.252387+0000","last_active":"2024-04-22T13:24:06.266483+0000","last_peered":"2024-04-22T13:24:06.266483+0000","last_clean":"2024-04-22T13:24:06.266483+0000","last_became_active":"2024-04-22T13:22:05.252033+0000","last_became_peered":"2024-04-22T13:22:05.252033+0000","last_unstale":"2024-04-22T13:24:06.266483+0000","last_undegraded":"2024-04-22T13:24:06.266483+0000","last_fullsized":"2024-04-22T13:24:06.266483+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:55:01.791283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164402+0000","last_change":"2024-04-22T13:22:05.248394+0000","last_active":"2024-04-22T13:24:06.164402+0000","last_peered":"2024-04-22T13:24:06.164402+0000","last_clean":"2024-04-22T13:24:06.164402+0000","last_became_active":"2024-04-22T13:22:05.248024+0000","last_became_peered":"2024-04-22T13:22:05.248024+0000","last_unstale":"2024-04-22T13:24:06.164402+0000","last_undegraded":"2024-04-22T13:24:06.164402+0000","last_fullsized":"2024-04-22T13:24:06.164402+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:49:36.556136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'7","reported_seq":145,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.590537+0000","last_change":"2024-04-22T13:22:04.247692+0000","last_active":"2024-04-22T13:24:06.590537+0000","last_peered":"2024-04-22T13:24:06.590537+0000","last_clean":"2024-04-22T13:24:06.590537+0000","last_became_active":"2024-04-22T13:22:04.247345+0000","last_became_peered":"2024-04-22T13:22:04.247345+0000","last_unstale":"2024-04-22T13:24:06.590537+0000","last_undegraded":"2024-04-22T13:24:06.590537+0000","last_fullsized":"2024-04-22T13:24:06.590537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:07:49.803248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259293+0000","last_change":"2024-04-22T13:22:05.241022+0000","last_active":"2024-04-22T13:24:06.259293+0000","last_peered":"2024-04-22T13:24:06.259293+0000","last_clean":"2024-04-22T13:24:06.259293+0000","last_became_active":"2024-04-22T13:22:05.240635+0000","last_became_peered":"2024-04-22T13:22:05.240635+0000","last_unstale":"2024-04-22T13:24:06.259293+0000","last_undegraded":"2024-04-22T13:24:06.259293+0000","last_fullsized":"2024-04-22T13:24:06.259293+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:50:02.012260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411229+0000","last_change":"2024-04-22T13:22:04.236817+0000","last_active":"2024-04-22T13:24:06.411229+0000","last_peered":"2024-04-22T13:24:06.411229+0000","last_clean":"2024-04-22T13:24:06.411229+0000","last_became_active":"2024-04-22T13:22:04.236484+0000","last_became_peered":"2024-04-22T13:22:04.236484+0000","last_unstale":"2024-04-22T13:24:06.411229+0000","last_undegraded":"2024-04-22T13:24:06.411229+0000","last_fullsized":"2024-04-22T13:24:06.411229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:27:41.053327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":357,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409048+0000","last_change":"2024-04-22T13:20:53.094123+0000","last_active":"2024-04-22T13:24:06.409048+0000","last_peered":"2024-04-22T13:24:06.409048+0000","last_clean":"2024-04-22T13:24:06.409048+0000","last_became_active":"2024-04-22T13:20:53.093664+0000","last_became_peered":"2024-04-22T13:20:53.093664+0000","last_unstale":"2024-04-22T13:24:06.409048+0000","last_undegraded":"2024-04-22T13:24:06.409048+0000","last_fullsized":"2024-04-22T13:24:06.409048+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:17:33.201786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:17:33.201786+0000","last_clean_scrub_stamp":"2024-04-22T13:17:33.201786+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:46:58.986399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162213+0000","last_change":"2024-04-22T13:22:05.247766+0000","last_active":"2024-04-22T13:24:06.162213+0000","last_peered":"2024-04-22T13:24:06.162213+0000","last_clean":"2024-04-22T13:24:06.162213+0000","last_became_active":"2024-04-22T13:22:05.247362+0000","last_became_peered":"2024-04-22T13:22:05.247362+0000","last_unstale":"2024-04-22T13:24:06.162213+0000","last_undegraded":"2024-04-22T13:24:06.162213+0000","last_fullsized":"2024-04-22T13:24:06.162213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:25:00.126638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409239+0000","last_change":"2024-04-22T13:22:04.243493+0000","last_active":"2024-04-22T13:24:06.409239+0000","last_peered":"2024-04-22T13:24:06.409239+0000","last_clean":"2024-04-22T13:24:06.409239+0000","last_became_active":"2024-04-22T13:22:04.243190+0000","last_became_peered":"2024-04-22T13:22:04.243190+0000","last_unstale":"2024-04-22T13:24:06.409239+0000","last_undegraded":"2024-04-22T13:24:06.409239+0000","last_fullsized":"2024-04-22T13:24:06.409239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:15:09.025152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258892+0000","last_change":"2024-04-22T13:22:04.249222+0000","last_active":"2024-04-22T13:24:06.258892+0000","last_peered":"2024-04-22T13:24:06.258892+0000","last_clean":"2024-04-22T13:24:06.258892+0000","last_became_active":"2024-04-22T13:22:04.248926+0000","last_became_peered":"2024-04-22T13:22:04.248926+0000","last_unstale":"2024-04-22T13:24:06.258892+0000","last_undegraded":"2024-04-22T13:24:06.258892+0000","last_fullsized":"2024-04-22T13:24:06.258892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:33:50.168426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162433+0000","last_change":"2024-04-22T13:22:05.248199+0000","last_active":"2024-04-22T13:24:06.162433+0000","last_peered":"2024-04-22T13:24:06.162433+0000","last_clean":"2024-04-22T13:24:06.162433+0000","last_became_active":"2024-04-22T13:22:05.247873+0000","last_became_peered":"2024-04-22T13:22:05.247873+0000","last_unstale":"2024-04-22T13:24:06.162433+0000","last_undegraded":"2024-04-22T13:24:06.162433+0000","last_fullsized":"2024-04-22T13:24:06.162433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:40:29.353901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'48","reported_seq":92,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.370574+0000","last_change":"2024-04-22T13:22:04.243474+0000","last_active":"2024-04-22T13:24:06.370574+0000","last_peered":"2024-04-22T13:24:06.370574+0000","last_clean":"2024-04-22T13:24:06.370574+0000","last_became_active":"2024-04-22T13:22:04.243151+0000","last_became_peered":"2024-04-22T13:22:04.243151+0000","last_unstale":"2024-04-22T13:24:06.370574+0000","last_undegraded":"2024-04-22T13:24:06.370574+0000","last_fullsized":"2024-04-22T13:24:06.370574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:18:25.140715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259264+0000","last_change":"2024-04-22T13:22:05.243120+0000","last_active":"2024-04-22T13:24:06.259264+0000","last_peered":"2024-04-22T13:24:06.259264+0000","last_clean":"2024-04-22T13:24:06.259264+0000","last_became_active":"2024-04-22T13:22:05.242726+0000","last_became_peered":"2024-04-22T13:22:05.242726+0000","last_unstale":"2024-04-22T13:24:06.259264+0000","last_undegraded":"2024-04-22T13:24:06.259264+0000","last_fullsized":"2024-04-22T13:24:06.259264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:06:16.087884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411424+0000","last_change":"2024-04-22T13:22:04.239672+0000","last_active":"2024-04-22T13:24:06.411424+0000","last_peered":"2024-04-22T13:24:06.411424+0000","last_clean":"2024-04-22T13:24:06.411424+0000","last_became_active":"2024-04-22T13:22:04.238776+0000","last_became_peered":"2024-04-22T13:22:04.238776+0000","last_unstale":"2024-04-22T13:24:06.411424+0000","last_undegraded":"2024-04-22T13:24:06.411424+0000","last_fullsized":"2024-04-22T13:24:06.411424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:43:21.752547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.412035+0000","last_change":"2024-04-22T13:22:05.242089+0000","last_active":"2024-04-22T13:24:06.412035+0000","last_peered":"2024-04-22T13:24:06.412035+0000","last_clean":"2024-04-22T13:24:06.412035+0000","last_became_active":"2024-04-22T13:22:05.241681+0000","last_became_peered":"2024-04-22T13:22:05.241681+0000","last_unstale":"2024-04-22T13:24:06.412035+0000","last_undegraded":"2024-04-22T13:24:06.412035+0000","last_fullsized":"2024-04-22T13:24:06.412035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:58:49.906188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.163728+0000","last_change":"2024-04-22T13:23:05.513123+0000","last_active":"2024-04-22T13:24:06.163728+0000","last_peered":"2024-04-22T13:24:06.163728+0000","last_clean":"2024-04-22T13:24:06.163728+0000","last_became_active":"2024-04-22T13:23:05.509367+0000","last_became_peered":"2024-04-22T13:23:05.509367+0000","last_unstale":"2024-04-22T13:24:06.163728+0000","last_undegraded":"2024-04-22T13:24:06.163728+0000","last_fullsized":"2024-04-22T13:24:06.163728+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:28:41.770957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258638+0000","last_change":"2024-04-22T13:22:05.250120+0000","last_active":"2024-04-22T13:24:06.258638+0000","last_peered":"2024-04-22T13:24:06.258638+0000","last_clean":"2024-04-22T13:24:06.258638+0000","last_became_active":"2024-04-22T13:22:05.247936+0000","last_became_peered":"2024-04-22T13:22:05.247936+0000","last_unstale":"2024-04-22T13:24:06.258638+0000","last_undegraded":"2024-04-22T13:24:06.258638+0000","last_fullsized":"2024-04-22T13:24:06.258638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:27:38.437918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258902+0000","last_change":"2024-04-22T13:22:04.244026+0000","last_active":"2024-04-22T13:24:06.258902+0000","last_peered":"2024-04-22T13:24:06.258902+0000","last_clean":"2024-04-22T13:24:06.258902+0000","last_became_active":"2024-04-22T13:22:04.243518+0000","last_became_peered":"2024-04-22T13:22:04.243518+0000","last_unstale":"2024-04-22T13:24:06.258902+0000","last_undegraded":"2024-04-22T13:24:06.258902+0000","last_fullsized":"2024-04-22T13:24:06.258902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:54:48.791874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418220+0000","last_change":"2024-04-22T13:22:05.252674+0000","last_active":"2024-04-22T13:24:06.418220+0000","last_peered":"2024-04-22T13:24:06.418220+0000","last_clean":"2024-04-22T13:24:06.418220+0000","last_became_active":"2024-04-22T13:22:05.251087+0000","last_became_peered":"2024-04-22T13:22:05.251087+0000","last_unstale":"2024-04-22T13:24:06.418220+0000","last_undegraded":"2024-04-22T13:24:06.418220+0000","last_fullsized":"2024-04-22T13:24:06.418220+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:20:06.873412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.414640+0000","last_change":"2024-04-22T13:22:04.244409+0000","last_active":"2024-04-22T13:24:06.414640+0000","last_peered":"2024-04-22T13:24:06.414640+0000","last_clean":"2024-04-22T13:24:06.414640+0000","last_became_active":"2024-04-22T13:22:04.244104+0000","last_became_peered":"2024-04-22T13:22:04.244104+0000","last_unstale":"2024-04-22T13:24:06.414640+0000","last_undegraded":"2024-04-22T13:24:06.414640+0000","last_fullsized":"2024-04-22T13:24:06.414640+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:39:14.714755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259066+0000","last_change":"2024-04-22T13:22:05.245730+0000","last_active":"2024-04-22T13:24:06.259066+0000","last_peered":"2024-04-22T13:24:06.259066+0000","last_clean":"2024-04-22T13:24:06.259066+0000","last_became_active":"2024-04-22T13:22:05.245366+0000","last_became_peered":"2024-04-22T13:22:05.245366+0000","last_unstale":"2024-04-22T13:24:06.259066+0000","last_undegraded":"2024-04-22T13:24:06.259066+0000","last_fullsized":"2024-04-22T13:24:06.259066+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:31.935258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418212+0000","last_change":"2024-04-22T13:23:05.514197+0000","last_active":"2024-04-22T13:24:06.418212+0000","last_peered":"2024-04-22T13:24:06.418212+0000","last_clean":"2024-04-22T13:24:06.418212+0000","last_became_active":"2024-04-22T13:23:05.509431+0000","last_became_peered":"2024-04-22T13:23:05.509431+0000","last_unstale":"2024-04-22T13:24:06.418212+0000","last_undegraded":"2024-04-22T13:24:06.418212+0000","last_fullsized":"2024-04-22T13:24:06.418212+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:43:59.825567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409273+0000","last_change":"2024-04-22T13:22:05.242146+0000","last_active":"2024-04-22T13:24:06.409273+0000","last_peered":"2024-04-22T13:24:06.409273+0000","last_clean":"2024-04-22T13:24:06.409273+0000","last_became_active":"2024-04-22T13:22:05.241802+0000","last_became_peered":"2024-04-22T13:22:05.241802+0000","last_unstale":"2024-04-22T13:24:06.409273+0000","last_undegraded":"2024-04-22T13:24:06.409273+0000","last_fullsized":"2024-04-22T13:24:06.409273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:49:49.002123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'7","reported_seq":147,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.562849+0000","last_change":"2024-04-22T13:22:04.248884+0000","last_active":"2024-04-22T13:24:06.562849+0000","last_peered":"2024-04-22T13:24:06.562849+0000","last_clean":"2024-04-22T13:24:06.562849+0000","last_became_active":"2024-04-22T13:22:04.248393+0000","last_became_peered":"2024-04-22T13:22:04.248393+0000","last_unstale":"2024-04-22T13:24:06.562849+0000","last_undegraded":"2024-04-22T13:24:06.562849+0000","last_fullsized":"2024-04-22T13:24:06.562849+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:36:14.687457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1869,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409545+0000","last_change":"2024-04-22T13:22:05.256881+0000","last_active":"2024-04-22T13:24:06.409545+0000","last_peered":"2024-04-22T13:24:06.409545+0000","last_clean":"2024-04-22T13:24:06.409545+0000","last_became_active":"2024-04-22T13:22:05.255967+0000","last_became_peered":"2024-04-22T13:22:05.255967+0000","last_unstale":"2024-04-22T13:24:06.409545+0000","last_undegraded":"2024-04-22T13:24:06.409545+0000","last_fullsized":"2024-04-22T13:24:06.409545+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:24:21.137344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'41","reported_seq":70,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.367821+0000","last_change":"2024-04-22T13:22:04.247303+0000","last_active":"2024-04-22T13:24:06.367821+0000","last_peered":"2024-04-22T13:24:06.367821+0000","last_clean":"2024-04-22T13:24:06.367821+0000","last_became_active":"2024-04-22T13:22:04.246742+0000","last_became_peered":"2024-04-22T13:22:04.246742+0000","last_unstale":"2024-04-22T13:24:06.367821+0000","last_undegraded":"2024-04-22T13:24:06.367821+0000","last_fullsized":"2024-04-22T13:24:06.367821+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:45:03.189042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409288+0000","last_change":"2024-04-22T13:22:05.251754+0000","last_active":"2024-04-22T13:24:06.409288+0000","last_peered":"2024-04-22T13:24:06.409288+0000","last_clean":"2024-04-22T13:24:06.409288+0000","last_became_active":"2024-04-22T13:22:05.251155+0000","last_became_peered":"2024-04-22T13:22:05.251155+0000","last_unstale":"2024-04-22T13:24:06.409288+0000","last_undegraded":"2024-04-22T13:24:06.409288+0000","last_fullsized":"2024-04-22T13:24:06.409288+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:08:46.682600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162485+0000","last_change":"2024-04-22T13:22:04.243382+0000","last_active":"2024-04-22T13:24:06.162485+0000","last_peered":"2024-04-22T13:24:06.162485+0000","last_clean":"2024-04-22T13:24:06.162485+0000","last_became_active":"2024-04-22T13:22:04.243055+0000","last_became_peered":"2024-04-22T13:22:04.243055+0000","last_unstale":"2024-04-22T13:24:06.162485+0000","last_undegraded":"2024-04-22T13:24:06.162485+0000","last_fullsized":"2024-04-22T13:24:06.162485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:19:58.379645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409542+0000","last_change":"2024-04-22T13:22:05.249527+0000","last_active":"2024-04-22T13:24:06.409542+0000","last_peered":"2024-04-22T13:24:06.409542+0000","last_clean":"2024-04-22T13:24:06.409542+0000","last_became_active":"2024-04-22T13:22:05.249207+0000","last_became_peered":"2024-04-22T13:22:05.249207+0000","last_unstale":"2024-04-22T13:24:06.409542+0000","last_undegraded":"2024-04-22T13:24:06.409542+0000","last_fullsized":"2024-04-22T13:24:06.409542+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:50:02.755288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418234+0000","last_change":"2024-04-22T13:23:05.508324+0000","last_active":"2024-04-22T13:24:06.418234+0000","last_peered":"2024-04-22T13:24:06.418234+0000","last_clean":"2024-04-22T13:24:06.418234+0000","last_became_active":"2024-04-22T13:23:05.507814+0000","last_became_peered":"2024-04-22T13:23:05.507814+0000","last_unstale":"2024-04-22T13:24:06.418234+0000","last_undegraded":"2024-04-22T13:24:06.418234+0000","last_fullsized":"2024-04-22T13:24:06.418234+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:46:20.523715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.414586+0000","last_change":"2024-04-22T13:23:05.875683+0000","last_active":"2024-04-22T13:24:06.414586+0000","last_peered":"2024-04-22T13:24:06.414586+0000","last_clean":"2024-04-22T13:24:06.414586+0000","last_became_active":"2024-04-22T13:23:05.875150+0000","last_became_peered":"2024-04-22T13:23:05.875150+0000","last_unstale":"2024-04-22T13:24:06.414586+0000","last_undegraded":"2024-04-22T13:24:06.414586+0000","last_fullsized":"2024-04-22T13:24:06.414586+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:12:47.592230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.414529+0000","last_change":"2024-04-22T13:22:05.247882+0000","last_active":"2024-04-22T13:24:06.414529+0000","last_peered":"2024-04-22T13:24:06.414529+0000","last_clean":"2024-04-22T13:24:06.414529+0000","last_became_active":"2024-04-22T13:22:05.247525+0000","last_became_peered":"2024-04-22T13:22:05.247525+0000","last_unstale":"2024-04-22T13:24:06.414529+0000","last_undegraded":"2024-04-22T13:24:06.414529+0000","last_fullsized":"2024-04-22T13:24:06.414529+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:46:04.349076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411567+0000","last_change":"2024-04-22T13:22:05.243716+0000","last_active":"2024-04-22T13:24:06.411567+0000","last_peered":"2024-04-22T13:24:06.411567+0000","last_clean":"2024-04-22T13:24:06.411567+0000","last_became_active":"2024-04-22T13:22:05.243397+0000","last_became_peered":"2024-04-22T13:22:05.243397+0000","last_unstale":"2024-04-22T13:24:06.411567+0000","last_undegraded":"2024-04-22T13:24:06.411567+0000","last_fullsized":"2024-04-22T13:24:06.411567+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:16:21.236339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162619+0000","last_change":"2024-04-22T13:22:05.246042+0000","last_active":"2024-04-22T13:24:06.162619+0000","last_peered":"2024-04-22T13:24:06.162619+0000","last_clean":"2024-04-22T13:24:06.162619+0000","last_became_active":"2024-04-22T13:22:05.245669+0000","last_became_peered":"2024-04-22T13:22:05.245669+0000","last_unstale":"2024-04-22T13:24:06.162619+0000","last_undegraded":"2024-04-22T13:24:06.162619+0000","last_fullsized":"2024-04-22T13:24:06.162619+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:49:05.539641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418321+0000","last_change":"2024-04-22T13:22:05.257009+0000","last_active":"2024-04-22T13:24:06.418321+0000","last_peered":"2024-04-22T13:24:06.418321+0000","last_clean":"2024-04-22T13:24:06.418321+0000","last_became_active":"2024-04-22T13:22:05.254327+0000","last_became_peered":"2024-04-22T13:22:05.254327+0000","last_unstale":"2024-04-22T13:24:06.418321+0000","last_undegraded":"2024-04-22T13:24:06.418321+0000","last_fullsized":"2024-04-22T13:24:06.418321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:29:12.482525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.415339+0000","last_change":"2024-04-22T13:22:05.260575+0000","last_active":"2024-04-22T13:24:06.415339+0000","last_peered":"2024-04-22T13:24:06.415339+0000","last_clean":"2024-04-22T13:24:06.415339+0000","last_became_active":"2024-04-22T13:22:05.260181+0000","last_became_peered":"2024-04-22T13:22:05.260181+0000","last_unstale":"2024-04-22T13:24:06.415339+0000","last_undegraded":"2024-04-22T13:24:06.415339+0000","last_fullsized":"2024-04-22T13:24:06.415339+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:49:20.064368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259116+0000","last_change":"2024-04-22T13:22:05.248199+0000","last_active":"2024-04-22T13:24:06.259116+0000","last_peered":"2024-04-22T13:24:06.259116+0000","last_clean":"2024-04-22T13:24:06.259116+0000","last_became_active":"2024-04-22T13:22:05.247858+0000","last_became_peered":"2024-04-22T13:22:05.247858+0000","last_unstale":"2024-04-22T13:24:06.259116+0000","last_undegraded":"2024-04-22T13:24:06.259116+0000","last_fullsized":"2024-04-22T13:24:06.259116+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:49:21.228105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164341+0000","last_change":"2024-04-22T13:22:05.239137+0000","last_active":"2024-04-22T13:24:06.164341+0000","last_peered":"2024-04-22T13:24:06.164341+0000","last_clean":"2024-04-22T13:24:06.164341+0000","last_became_active":"2024-04-22T13:22:05.238807+0000","last_became_peered":"2024-04-22T13:22:05.238807+0000","last_unstale":"2024-04-22T13:24:06.164341+0000","last_undegraded":"2024-04-22T13:24:06.164341+0000","last_fullsized":"2024-04-22T13:24:06.164341+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:02:48.587097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258756+0000","last_change":"2024-04-22T13:22:05.249080+0000","last_active":"2024-04-22T13:24:06.258756+0000","last_peered":"2024-04-22T13:24:06.258756+0000","last_clean":"2024-04-22T13:24:06.258756+0000","last_became_active":"2024-04-22T13:22:05.248733+0000","last_became_peered":"2024-04-22T13:22:05.248733+0000","last_unstale":"2024-04-22T13:24:06.258756+0000","last_undegraded":"2024-04-22T13:24:06.258756+0000","last_fullsized":"2024-04-22T13:24:06.258756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:58.735941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.163779+0000","last_change":"2024-04-22T13:22:05.255490+0000","last_active":"2024-04-22T13:24:06.163779+0000","last_peered":"2024-04-22T13:24:06.163779+0000","last_clean":"2024-04-22T13:24:06.163779+0000","last_became_active":"2024-04-22T13:22:05.253644+0000","last_became_peered":"2024-04-22T13:22:05.253644+0000","last_unstale":"2024-04-22T13:24:06.163779+0000","last_undegraded":"2024-04-22T13:24:06.163779+0000","last_fullsized":"2024-04-22T13:24:06.163779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:31:01.816182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162443+0000","last_change":"2024-04-22T13:22:05.240988+0000","last_active":"2024-04-22T13:24:06.162443+0000","last_peered":"2024-04-22T13:24:06.162443+0000","last_clean":"2024-04-22T13:24:06.162443+0000","last_became_active":"2024-04-22T13:22:05.240642+0000","last_became_peered":"2024-04-22T13:22:05.240642+0000","last_unstale":"2024-04-22T13:24:06.162443+0000","last_undegraded":"2024-04-22T13:24:06.162443+0000","last_fullsized":"2024-04-22T13:24:06.162443+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:41:29.001979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418222+0000","last_change":"2024-04-22T13:22:05.252212+0000","last_active":"2024-04-22T13:24:06.418222+0000","last_peered":"2024-04-22T13:24:06.418222+0000","last_clean":"2024-04-22T13:24:06.418222+0000","last_became_active":"2024-04-22T13:22:05.251852+0000","last_became_peered":"2024-04-22T13:22:05.251852+0000","last_unstale":"2024-04-22T13:24:06.418222+0000","last_undegraded":"2024-04-22T13:24:06.418222+0000","last_fullsized":"2024-04-22T13:24:06.418222+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:13:55.132501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":598760,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1300,"num_read_kb":764,"num_write":368,"num_write_kb":1462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2162688,"data_stored":1796280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":298,"ondisk_log_size":298,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318310,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":472759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579945,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28796,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716356,"statfs":{"total":95995035648,"available":95965548544,"internally_reserved":0,"allocated":1822720,"data_stored":1260339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743471,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28388,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716764,"statfs":{"total":95995035648,"available":95965966336,"internally_reserved":0,"allocated":1400832,"data_stored":865206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005108,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28384,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716768,"statfs":{"total":95995035648,"available":95965970432,"internally_reserved":0,"allocated":1396736,"data_stored":865584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234041,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32744,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712408,"statfs":{"total":95995035648,"available":95961505792,"internally_reserved":0,"allocated":1601536,"data_stored":1066591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430270,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":991232,"data_stored":472489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":538129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":473745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1077248,"data_stored":538657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117070,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36836,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708316,"statfs":{"total":95995035648,"available":95957315584,"internally_reserved":0,"allocated":1536000,"data_stored":996348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855433,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":991232,"data_stored":472491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691907,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28724,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716428,"statfs":{"total":95995035648,"available":95965622272,"internally_reserved":0,"allocated":1748992,"data_stored":1196164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":134811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":198986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T13:24:10.979 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-22T13:24:10.979 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T13:24:10.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:10 smithi111 ceph-mon[31361]: Adjusting osd_memory_target on smithi111 to 176.2M 2024-04-22T13:24:10.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:10 smithi111 ceph-mon[31361]: Adjusting osd_memory_target on smithi134 to 176.2M 2024-04-22T13:24:10.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:10 smithi111 ceph-mon[31361]: Unable to set osd_memory_target on smithi111 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-22T13:24:10.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:10 smithi111 ceph-mon[31361]: Unable to set osd_memory_target on smithi134 to 184792473: error parsing value: Value '184792473' is below minimum 939524096 2024-04-22T13:24:10.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:10 smithi111 ceph-mon[31361]: Adjusting osd_memory_target on smithi012 to 177.2M 2024-04-22T13:24:10.980 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:10 smithi111 ceph-mon[31361]: Unable to set osd_memory_target on smithi012 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-04-22T13:24:10.981 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:10 smithi111 ceph-mon[31361]: Updating smithi012:/etc/ceph/ceph.conf 2024-04-22T13:24:10.981 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:10 smithi111 ceph-mon[31361]: Updating smithi111:/etc/ceph/ceph.conf 2024-04-22T13:24:10.981 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:10 smithi111 ceph-mon[31361]: Updating smithi134:/etc/ceph/ceph.conf 2024-04-22T13:24:10.981 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:10 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1922322927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:24:10.981 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:10 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/339579050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:11.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.122+0000 7fa015f6d640 1 -- 172.21.15.12:0/3207729262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa010107270 msgr2=0x7fa01010ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:11.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.122+0000 7fa015f6d640 1 --2- 172.21.15.12:0/3207729262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa010107270 0x7fa01010ee40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa008009900 tx=0x7fa00802bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:11.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.122+0000 7fa015f6d640 1 -- 172.21.15.12:0/3207729262 shutdown_connections 2024-04-22T13:24:11.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.122+0000 7fa015f6d640 1 --2- 172.21.15.12:0/3207729262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa010107270 0x7fa01010ee40 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:11.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.122+0000 7fa015f6d640 1 --2- 172.21.15.12:0/3207729262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa010102b50 0x7fa010106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:11.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.122+0000 7fa015f6d640 1 --2- 172.21.15.12:0/3207729262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0101021a0 0x7fa010102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:11.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.122+0000 7fa015f6d640 1 -- 172.21.15.12:0/3207729262 >> 172.21.15.12:0/3207729262 conn(0x7fa0100793a0 msgr2=0x7fa0100797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:11.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.122+0000 7fa015f6d640 1 -- 172.21.15.12:0/3207729262 shutdown_connections 2024-04-22T13:24:11.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.122+0000 7fa015f6d640 1 -- 172.21.15.12:0/3207729262 wait complete. 2024-04-22T13:24:11.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.122+0000 7fa015f6d640 1 Processor -- start 2024-04-22T13:24:11.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.122+0000 7fa015f6d640 1 -- start start 2024-04-22T13:24:11.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.123+0000 7fa015f6d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0101021a0 0x7fa0101986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:11.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.123+0000 7fa015f6d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa010107270 0x7fa010198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:11.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.123+0000 7fa015f6d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa01019cf70 0x7fa01019d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:11.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.123+0000 7fa015f6d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa010112230 con 0x7fa01019cf70 2024-04-22T13:24:11.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.123+0000 7fa015f6d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa0101120b0 con 0x7fa0101021a0 2024-04-22T13:24:11.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.123+0000 7fa015f6d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa0101123b0 con 0x7fa010107270 2024-04-22T13:24:11.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.123+0000 7fa00ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa01019cf70 0x7fa01019d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:11.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.123+0000 7fa00ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa01019cf70 0x7fa01019d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35078/0 (socket says 172.21.15.12:35078) 2024-04-22T13:24:11.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.123+0000 7fa00ffff640 1 -- 172.21.15.12:0/2888575040 learned_addr learned my addr 172.21.15.12:0/2888575040 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:11.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.123+0000 7fa00f7fe640 1 --2- 172.21.15.12:0/2888575040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0101021a0 0x7fa0101986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:11.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.123+0000 7fa00effd640 1 --2- 172.21.15.12:0/2888575040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa010107270 0x7fa010198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:11.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.123+0000 7fa00ffff640 1 -- 172.21.15.12:0/2888575040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa010107270 msgr2=0x7fa010198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:11.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.123+0000 7fa00ffff640 1 --2- 172.21.15.12:0/2888575040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa010107270 0x7fa010198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:11.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.123+0000 7fa00ffff640 1 -- 172.21.15.12:0/2888575040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0101021a0 msgr2=0x7fa0101986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:11.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.123+0000 7fa00ffff640 1 --2- 172.21.15.12:0/2888575040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0101021a0 0x7fa0101986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:11.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.123+0000 7fa00ffff640 1 -- 172.21.15.12:0/2888575040 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa01019da60 con 0x7fa01019cf70 2024-04-22T13:24:11.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.123+0000 7fa00effd640 1 --2- 172.21.15.12:0/2888575040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa010107270 0x7fa010198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:11.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.123+0000 7fa00ffff640 1 --2- 172.21.15.12:0/2888575040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa01019cf70 0x7fa01019d420 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa00802c170 tx=0x7fa0080045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:11.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.123+0000 7fa00cff9640 1 -- 172.21.15.12:0/2888575040 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa008043020 con 0x7fa01019cf70 2024-04-22T13:24:11.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.124+0000 7fa015f6d640 1 -- 172.21.15.12:0/2888575040 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa01019dcf0 con 0x7fa01019cf70 2024-04-22T13:24:11.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.124+0000 7fa00cff9640 1 -- 172.21.15.12:0/2888575040 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa00802cb30 con 0x7fa01019cf70 2024-04-22T13:24:11.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.124+0000 7fa00cff9640 1 -- 172.21.15.12:0/2888575040 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa008003a10 con 0x7fa01019cf70 2024-04-22T13:24:11.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.124+0000 7fa015f6d640 1 -- 172.21.15.12:0/2888575040 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0101a54f0 con 0x7fa01019cf70 2024-04-22T13:24:11.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.125+0000 7fa015f6d640 1 -- 172.21.15.12:0/2888575040 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa010069230 con 0x7fa01019cf70 2024-04-22T13:24:11.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.126+0000 7fa00cff9640 1 -- 172.21.15.12:0/2888575040 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fa008046020 con 0x7fa01019cf70 2024-04-22T13:24:11.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.126+0000 7fa00cff9640 1 --2- 172.21.15.12:0/2888575040 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f9ff80a3e40 0x7f9ff80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:11.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.126+0000 7fa00f7fe640 1 --2- 172.21.15.12:0/2888575040 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f9ff80a3e40 0x7f9ff80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:11.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.127+0000 7fa00cff9640 1 -- 172.21.15.12:0/2888575040 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fa0080f2f30 con 0x7fa01019cf70 2024-04-22T13:24:11.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.127+0000 7fa00f7fe640 1 --2- 172.21.15.12:0/2888575040 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f9ff80a3e40 0x7f9ff80a6300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa000005fd0 tx=0x7fa000009500 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:11.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.130+0000 7fa00cff9640 1 -- 172.21.15.12:0/2888575040 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa00804d020 con 0x7fa01019cf70 2024-04-22T13:24:11.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:10 smithi134 ceph-mon[31338]: Adjusting osd_memory_target on smithi111 to 176.2M 2024-04-22T13:24:11.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:10 smithi134 ceph-mon[31338]: Adjusting osd_memory_target on smithi134 to 176.2M 2024-04-22T13:24:11.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:10 smithi134 ceph-mon[31338]: Unable to set osd_memory_target on smithi111 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-22T13:24:11.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:10 smithi134 ceph-mon[31338]: Unable to set osd_memory_target on smithi134 to 184792473: error parsing value: Value '184792473' is below minimum 939524096 2024-04-22T13:24:11.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:10 smithi134 ceph-mon[31338]: Adjusting osd_memory_target on smithi012 to 177.2M 2024-04-22T13:24:11.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:10 smithi134 ceph-mon[31338]: Unable to set osd_memory_target on smithi012 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-04-22T13:24:11.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:10 smithi134 ceph-mon[31338]: Updating smithi012:/etc/ceph/ceph.conf 2024-04-22T13:24:11.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:10 smithi134 ceph-mon[31338]: Updating smithi111:/etc/ceph/ceph.conf 2024-04-22T13:24:11.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:10 smithi134 ceph-mon[31338]: Updating smithi134:/etc/ceph/ceph.conf 2024-04-22T13:24:11.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:10 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1922322927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:24:11.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:10 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/339579050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:11.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.312+0000 7fa015f6d640 1 -- 172.21.15.12:0/2888575040 --> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa010102b50 con 0x7f9ff80a3e40 2024-04-22T13:24:11.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.319+0000 7fa00cff9640 1 -- 172.21.15.12:0/2888575040 <== mgr.24968 v2:172.21.15.12:6800/1615661596 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257781 (secure 0 0 0) 0x7fa010102b50 con 0x7f9ff80a3e40 2024-04-22T13:24:11.319 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:24:11.321 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-22T13:24:11.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.322+0000 7fa015f6d640 1 -- 172.21.15.12:0/2888575040 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f9ff80a3e40 msgr2=0x7f9ff80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:11.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.322+0000 7fa015f6d640 1 --2- 172.21.15.12:0/2888575040 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f9ff80a3e40 0x7f9ff80a6300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa000005fd0 tx=0x7fa000009500 comp rx=0 tx=0).stop 2024-04-22T13:24:11.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.322+0000 7fa015f6d640 1 -- 172.21.15.12:0/2888575040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa01019cf70 msgr2=0x7fa01019d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:11.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.322+0000 7fa015f6d640 1 --2- 172.21.15.12:0/2888575040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa01019cf70 0x7fa01019d420 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa00802c170 tx=0x7fa0080045e0 comp rx=0 tx=0).stop 2024-04-22T13:24:11.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.322+0000 7fa015f6d640 1 -- 172.21.15.12:0/2888575040 shutdown_connections 2024-04-22T13:24:11.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.322+0000 7fa015f6d640 1 --2- 172.21.15.12:0/2888575040 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f9ff80a3e40 0x7f9ff80a6300 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:11.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.322+0000 7fa015f6d640 1 --2- 172.21.15.12:0/2888575040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa01019cf70 0x7fa01019d420 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:11.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.322+0000 7fa015f6d640 1 --2- 172.21.15.12:0/2888575040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa010107270 0x7fa010198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:11.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.322+0000 7fa015f6d640 1 --2- 172.21.15.12:0/2888575040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0101021a0 0x7fa0101986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:11.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.322+0000 7fa015f6d640 1 -- 172.21.15.12:0/2888575040 >> 172.21.15.12:0/2888575040 conn(0x7fa0100793a0 msgr2=0x7fa0100ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:11.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.323+0000 7fa015f6d640 1 -- 172.21.15.12:0/2888575040 shutdown_connections 2024-04-22T13:24:11.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.323+0000 7fa015f6d640 1 -- 172.21.15.12:0/2888575040 wait complete. 2024-04-22T13:24:11.331 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-22T13:24:10.176283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189362,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1436,"num_read_kb":883,"num_write":551,"num_write_kb":4402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":937,"ondisk_log_size":937,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376920,"kb_used_data":15252,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124564904,"statfs":{"total":1151940427776,"available":1151554461696,"internally_reserved":0,"allocated":15618048,"data_stored":9218502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.991258"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418486+0000","last_change":"2024-04-22T13:22:05.254388+0000","last_active":"2024-04-22T13:24:06.418486+0000","last_peered":"2024-04-22T13:24:06.418486+0000","last_clean":"2024-04-22T13:24:06.418486+0000","last_became_active":"2024-04-22T13:22:05.254042+0000","last_became_peered":"2024-04-22T13:22:05.254042+0000","last_unstale":"2024-04-22T13:24:06.418486+0000","last_undegraded":"2024-04-22T13:24:06.418486+0000","last_fullsized":"2024-04-22T13:24:06.418486+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:07:56.139653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.412013+0000","last_change":"2024-04-22T13:22:05.247879+0000","last_active":"2024-04-22T13:24:06.412013+0000","last_peered":"2024-04-22T13:24:06.412013+0000","last_clean":"2024-04-22T13:24:06.412013+0000","last_became_active":"2024-04-22T13:22:05.247508+0000","last_became_peered":"2024-04-22T13:22:05.247508+0000","last_unstale":"2024-04-22T13:24:06.412013+0000","last_undegraded":"2024-04-22T13:24:06.412013+0000","last_fullsized":"2024-04-22T13:24:06.412013+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:55:51.928658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411868+0000","last_change":"2024-04-22T13:22:05.257970+0000","last_active":"2024-04-22T13:24:06.411868+0000","last_peered":"2024-04-22T13:24:06.411868+0000","last_clean":"2024-04-22T13:24:06.411868+0000","last_became_active":"2024-04-22T13:22:05.257652+0000","last_became_peered":"2024-04-22T13:22:05.257652+0000","last_unstale":"2024-04-22T13:24:06.411868+0000","last_undegraded":"2024-04-22T13:24:06.411868+0000","last_fullsized":"2024-04-22T13:24:06.411868+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:42:17.476349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411332+0000","last_change":"2024-04-22T13:22:05.249186+0000","last_active":"2024-04-22T13:24:06.411332+0000","last_peered":"2024-04-22T13:24:06.411332+0000","last_clean":"2024-04-22T13:24:06.411332+0000","last_became_active":"2024-04-22T13:22:05.248814+0000","last_became_peered":"2024-04-22T13:22:05.248814+0000","last_unstale":"2024-04-22T13:24:06.411332+0000","last_undegraded":"2024-04-22T13:24:06.411332+0000","last_fullsized":"2024-04-22T13:24:06.411332+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:57.450794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258738+0000","last_change":"2024-04-22T13:22:05.251953+0000","last_active":"2024-04-22T13:24:06.258738+0000","last_peered":"2024-04-22T13:24:06.258738+0000","last_clean":"2024-04-22T13:24:06.258738+0000","last_became_active":"2024-04-22T13:22:05.251547+0000","last_became_peered":"2024-04-22T13:22:05.251547+0000","last_unstale":"2024-04-22T13:24:06.258738+0000","last_undegraded":"2024-04-22T13:24:06.258738+0000","last_fullsized":"2024-04-22T13:24:06.258738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:36:34.809878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.266346+0000","last_change":"2024-04-22T13:23:05.510044+0000","last_active":"2024-04-22T13:24:06.266346+0000","last_peered":"2024-04-22T13:24:06.266346+0000","last_clean":"2024-04-22T13:24:06.266346+0000","last_became_active":"2024-04-22T13:23:05.509677+0000","last_became_peered":"2024-04-22T13:23:05.509677+0000","last_unstale":"2024-04-22T13:24:06.266346+0000","last_undegraded":"2024-04-22T13:24:06.266346+0000","last_fullsized":"2024-04-22T13:24:06.266346+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:46.979291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409036+0000","last_change":"2024-04-22T13:22:05.250418+0000","last_active":"2024-04-22T13:24:06.409036+0000","last_peered":"2024-04-22T13:24:06.409036+0000","last_clean":"2024-04-22T13:24:06.409036+0000","last_became_active":"2024-04-22T13:22:05.250097+0000","last_became_peered":"2024-04-22T13:22:05.250097+0000","last_unstale":"2024-04-22T13:24:06.409036+0000","last_undegraded":"2024-04-22T13:24:06.409036+0000","last_fullsized":"2024-04-22T13:24:06.409036+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:34:59.598950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258759+0000","last_change":"2024-04-22T13:22:05.244762+0000","last_active":"2024-04-22T13:24:06.258759+0000","last_peered":"2024-04-22T13:24:06.258759+0000","last_clean":"2024-04-22T13:24:06.258759+0000","last_became_active":"2024-04-22T13:22:05.244430+0000","last_became_peered":"2024-04-22T13:22:05.244430+0000","last_unstale":"2024-04-22T13:24:06.258759+0000","last_undegraded":"2024-04-22T13:24:06.258759+0000","last_fullsized":"2024-04-22T13:24:06.258759+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:33:01.582851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164321+0000","last_change":"2024-04-22T13:22:05.257596+0000","last_active":"2024-04-22T13:24:06.164321+0000","last_peered":"2024-04-22T13:24:06.164321+0000","last_clean":"2024-04-22T13:24:06.164321+0000","last_became_active":"2024-04-22T13:22:05.257277+0000","last_became_peered":"2024-04-22T13:22:05.257277+0000","last_unstale":"2024-04-22T13:24:06.164321+0000","last_undegraded":"2024-04-22T13:24:06.164321+0000","last_fullsized":"2024-04-22T13:24:06.164321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:09:44.715965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.266708+0000","last_change":"2024-04-22T13:22:05.252177+0000","last_active":"2024-04-22T13:24:06.266708+0000","last_peered":"2024-04-22T13:24:06.266708+0000","last_clean":"2024-04-22T13:24:06.266708+0000","last_became_active":"2024-04-22T13:22:05.251794+0000","last_became_peered":"2024-04-22T13:22:05.251794+0000","last_unstale":"2024-04-22T13:24:06.266708+0000","last_undegraded":"2024-04-22T13:24:06.266708+0000","last_fullsized":"2024-04-22T13:24:06.266708+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:08:59.500540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409476+0000","last_change":"2024-04-22T13:22:05.256209+0000","last_active":"2024-04-22T13:24:06.409476+0000","last_peered":"2024-04-22T13:24:06.409476+0000","last_clean":"2024-04-22T13:24:06.409476+0000","last_became_active":"2024-04-22T13:22:05.255504+0000","last_became_peered":"2024-04-22T13:22:05.255504+0000","last_unstale":"2024-04-22T13:24:06.409476+0000","last_undegraded":"2024-04-22T13:24:06.409476+0000","last_fullsized":"2024-04-22T13:24:06.409476+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:32:08.190921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.256744+0000","last_change":"2024-04-22T13:23:05.509837+0000","last_active":"2024-04-22T13:24:06.256744+0000","last_peered":"2024-04-22T13:24:06.256744+0000","last_clean":"2024-04-22T13:24:06.256744+0000","last_became_active":"2024-04-22T13:23:05.509421+0000","last_became_peered":"2024-04-22T13:23:05.509421+0000","last_unstale":"2024-04-22T13:24:06.256744+0000","last_undegraded":"2024-04-22T13:24:06.256744+0000","last_fullsized":"2024-04-22T13:24:06.256744+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:30:01.103373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162449+0000","last_change":"2024-04-22T13:22:05.246667+0000","last_active":"2024-04-22T13:24:06.162449+0000","last_peered":"2024-04-22T13:24:06.162449+0000","last_clean":"2024-04-22T13:24:06.162449+0000","last_became_active":"2024-04-22T13:22:05.246365+0000","last_became_peered":"2024-04-22T13:22:05.246365+0000","last_unstale":"2024-04-22T13:24:06.162449+0000","last_undegraded":"2024-04-22T13:24:06.162449+0000","last_fullsized":"2024-04-22T13:24:06.162449+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:41:55.533430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411527+0000","last_change":"2024-04-22T13:22:05.251982+0000","last_active":"2024-04-22T13:24:06.411527+0000","last_peered":"2024-04-22T13:24:06.411527+0000","last_clean":"2024-04-22T13:24:06.411527+0000","last_became_active":"2024-04-22T13:22:05.251575+0000","last_became_peered":"2024-04-22T13:22:05.251575+0000","last_unstale":"2024-04-22T13:24:06.411527+0000","last_undegraded":"2024-04-22T13:24:06.411527+0000","last_fullsized":"2024-04-22T13:24:06.411527+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:28:05.343140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411427+0000","last_change":"2024-04-22T13:22:05.251056+0000","last_active":"2024-04-22T13:24:06.411427+0000","last_peered":"2024-04-22T13:24:06.411427+0000","last_clean":"2024-04-22T13:24:06.411427+0000","last_became_active":"2024-04-22T13:22:05.250652+0000","last_became_peered":"2024-04-22T13:22:05.250652+0000","last_unstale":"2024-04-22T13:24:06.411427+0000","last_undegraded":"2024-04-22T13:24:06.411427+0000","last_fullsized":"2024-04-22T13:24:06.411427+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:08:28.629965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.163636+0000","last_change":"2024-04-22T13:22:05.256877+0000","last_active":"2024-04-22T13:24:06.163636+0000","last_peered":"2024-04-22T13:24:06.163636+0000","last_clean":"2024-04-22T13:24:06.163636+0000","last_became_active":"2024-04-22T13:22:05.256074+0000","last_became_peered":"2024-04-22T13:22:05.256074+0000","last_unstale":"2024-04-22T13:24:06.163636+0000","last_undegraded":"2024-04-22T13:24:06.163636+0000","last_fullsized":"2024-04-22T13:24:06.163636+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:08:53.607626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418592+0000","last_change":"2024-04-22T13:22:05.255598+0000","last_active":"2024-04-22T13:24:06.418592+0000","last_peered":"2024-04-22T13:24:06.418592+0000","last_clean":"2024-04-22T13:24:06.418592+0000","last_became_active":"2024-04-22T13:22:05.255239+0000","last_became_peered":"2024-04-22T13:22:05.255239+0000","last_unstale":"2024-04-22T13:24:06.418592+0000","last_undegraded":"2024-04-22T13:24:06.418592+0000","last_fullsized":"2024-04-22T13:24:06.418592+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:38:20.547882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.257085+0000","last_change":"2024-04-22T13:22:05.254232+0000","last_active":"2024-04-22T13:24:06.257085+0000","last_peered":"2024-04-22T13:24:06.257085+0000","last_clean":"2024-04-22T13:24:06.257085+0000","last_became_active":"2024-04-22T13:22:05.253907+0000","last_became_peered":"2024-04-22T13:22:05.253907+0000","last_unstale":"2024-04-22T13:24:06.257085+0000","last_undegraded":"2024-04-22T13:24:06.257085+0000","last_fullsized":"2024-04-22T13:24:06.257085+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:15:49.931112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.415196+0000","last_change":"2024-04-22T13:22:05.251518+0000","last_active":"2024-04-22T13:24:06.415196+0000","last_peered":"2024-04-22T13:24:06.415196+0000","last_clean":"2024-04-22T13:24:06.415196+0000","last_became_active":"2024-04-22T13:22:05.250478+0000","last_became_peered":"2024-04-22T13:22:05.250478+0000","last_unstale":"2024-04-22T13:24:06.415196+0000","last_undegraded":"2024-04-22T13:24:06.415196+0000","last_fullsized":"2024-04-22T13:24:06.415196+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:25:26.758692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'14","reported_seq":152,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.559609+0000","last_change":"2024-04-22T13:22:04.241365+0000","last_active":"2024-04-22T13:24:06.559609+0000","last_peered":"2024-04-22T13:24:06.559609+0000","last_clean":"2024-04-22T13:24:06.559609+0000","last_became_active":"2024-04-22T13:22:04.240822+0000","last_became_peered":"2024-04-22T13:22:04.240822+0000","last_unstale":"2024-04-22T13:24:06.559609+0000","last_undegraded":"2024-04-22T13:24:06.559609+0000","last_fullsized":"2024-04-22T13:24:06.559609+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:20:23.928534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.256956+0000","last_change":"2024-04-22T13:23:05.876262+0000","last_active":"2024-04-22T13:24:06.256956+0000","last_peered":"2024-04-22T13:24:06.256956+0000","last_clean":"2024-04-22T13:24:06.256956+0000","last_became_active":"2024-04-22T13:23:05.875872+0000","last_became_peered":"2024-04-22T13:23:05.875872+0000","last_unstale":"2024-04-22T13:24:06.256956+0000","last_undegraded":"2024-04-22T13:24:06.256956+0000","last_fullsized":"2024-04-22T13:24:06.256956+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:29:33.092887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259476+0000","last_change":"2024-04-22T13:22:04.248872+0000","last_active":"2024-04-22T13:24:06.259476+0000","last_peered":"2024-04-22T13:24:06.259476+0000","last_clean":"2024-04-22T13:24:06.259476+0000","last_became_active":"2024-04-22T13:22:04.248466+0000","last_became_peered":"2024-04-22T13:22:04.248466+0000","last_unstale":"2024-04-22T13:24:06.259476+0000","last_undegraded":"2024-04-22T13:24:06.259476+0000","last_fullsized":"2024-04-22T13:24:06.259476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:38:10.871602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.412391+0000","last_change":"2024-04-22T13:22:05.258005+0000","last_active":"2024-04-22T13:24:06.412391+0000","last_peered":"2024-04-22T13:24:06.412391+0000","last_clean":"2024-04-22T13:24:06.412391+0000","last_became_active":"2024-04-22T13:22:05.257615+0000","last_became_peered":"2024-04-22T13:22:05.257615+0000","last_unstale":"2024-04-22T13:24:06.412391+0000","last_undegraded":"2024-04-22T13:24:06.412391+0000","last_fullsized":"2024-04-22T13:24:06.412391+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:21:18.030530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259060+0000","last_change":"2024-04-22T13:22:04.246273+0000","last_active":"2024-04-22T13:24:06.259060+0000","last_peered":"2024-04-22T13:24:06.259060+0000","last_clean":"2024-04-22T13:24:06.259060+0000","last_became_active":"2024-04-22T13:22:04.245969+0000","last_became_peered":"2024-04-22T13:22:04.245969+0000","last_unstale":"2024-04-22T13:24:06.259060+0000","last_undegraded":"2024-04-22T13:24:06.259060+0000","last_fullsized":"2024-04-22T13:24:06.259060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:10.260192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.163964+0000","last_change":"2024-04-22T13:22:05.260299+0000","last_active":"2024-04-22T13:24:06.163964+0000","last_peered":"2024-04-22T13:24:06.163964+0000","last_clean":"2024-04-22T13:24:06.163964+0000","last_became_active":"2024-04-22T13:22:05.259634+0000","last_became_peered":"2024-04-22T13:22:05.259634+0000","last_unstale":"2024-04-22T13:24:06.163964+0000","last_undegraded":"2024-04-22T13:24:06.163964+0000","last_fullsized":"2024-04-22T13:24:06.163964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:16:44.421652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258970+0000","last_change":"2024-04-22T13:22:04.243843+0000","last_active":"2024-04-22T13:24:06.258970+0000","last_peered":"2024-04-22T13:24:06.258970+0000","last_clean":"2024-04-22T13:24:06.258970+0000","last_became_active":"2024-04-22T13:22:04.243465+0000","last_became_peered":"2024-04-22T13:22:04.243465+0000","last_unstale":"2024-04-22T13:24:06.258970+0000","last_undegraded":"2024-04-22T13:24:06.258970+0000","last_fullsized":"2024-04-22T13:24:06.258970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:48:36.431697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164717+0000","last_change":"2024-04-22T13:22:05.259789+0000","last_active":"2024-04-22T13:24:06.164717+0000","last_peered":"2024-04-22T13:24:06.164717+0000","last_clean":"2024-04-22T13:24:06.164717+0000","last_became_active":"2024-04-22T13:22:05.259395+0000","last_became_peered":"2024-04-22T13:22:05.259395+0000","last_unstale":"2024-04-22T13:24:06.164717+0000","last_undegraded":"2024-04-22T13:24:06.164717+0000","last_fullsized":"2024-04-22T13:24:06.164717+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:43:51.430561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":148,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.533918+0000","last_change":"2024-04-22T13:22:04.240615+0000","last_active":"2024-04-22T13:24:06.533918+0000","last_peered":"2024-04-22T13:24:06.533918+0000","last_clean":"2024-04-22T13:24:06.533918+0000","last_became_active":"2024-04-22T13:22:04.240248+0000","last_became_peered":"2024-04-22T13:22:04.240248+0000","last_unstale":"2024-04-22T13:24:06.533918+0000","last_undegraded":"2024-04-22T13:24:06.533918+0000","last_fullsized":"2024-04-22T13:24:06.533918+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:55:05.948979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258665+0000","last_change":"2024-04-22T13:22:05.253788+0000","last_active":"2024-04-22T13:24:06.258665+0000","last_peered":"2024-04-22T13:24:06.258665+0000","last_clean":"2024-04-22T13:24:06.258665+0000","last_became_active":"2024-04-22T13:22:05.252578+0000","last_became_peered":"2024-04-22T13:22:05.252578+0000","last_unstale":"2024-04-22T13:24:06.258665+0000","last_undegraded":"2024-04-22T13:24:06.258665+0000","last_fullsized":"2024-04-22T13:24:06.258665+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:04:11.769930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.266274+0000","last_change":"2024-04-22T13:22:04.241170+0000","last_active":"2024-04-22T13:24:06.266274+0000","last_peered":"2024-04-22T13:24:06.266274+0000","last_clean":"2024-04-22T13:24:06.266274+0000","last_became_active":"2024-04-22T13:22:04.240830+0000","last_became_peered":"2024-04-22T13:22:04.240830+0000","last_unstale":"2024-04-22T13:24:06.266274+0000","last_undegraded":"2024-04-22T13:24:06.266274+0000","last_fullsized":"2024-04-22T13:24:06.266274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:51:30.520613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164733+0000","last_change":"2024-04-22T13:22:05.237900+0000","last_active":"2024-04-22T13:24:06.164733+0000","last_peered":"2024-04-22T13:24:06.164733+0000","last_clean":"2024-04-22T13:24:06.164733+0000","last_became_active":"2024-04-22T13:22:05.237507+0000","last_became_peered":"2024-04-22T13:22:05.237507+0000","last_unstale":"2024-04-22T13:24:06.164733+0000","last_undegraded":"2024-04-22T13:24:06.164733+0000","last_fullsized":"2024-04-22T13:24:06.164733+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:26:24.307803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162328+0000","last_change":"2024-04-22T13:22:04.236078+0000","last_active":"2024-04-22T13:24:06.162328+0000","last_peered":"2024-04-22T13:24:06.162328+0000","last_clean":"2024-04-22T13:24:06.162328+0000","last_became_active":"2024-04-22T13:22:04.235710+0000","last_became_peered":"2024-04-22T13:22:04.235710+0000","last_unstale":"2024-04-22T13:24:06.162328+0000","last_undegraded":"2024-04-22T13:24:06.162328+0000","last_fullsized":"2024-04-22T13:24:06.162328+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:37:55.335237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258717+0000","last_change":"2024-04-22T13:22:05.243152+0000","last_active":"2024-04-22T13:24:06.258717+0000","last_peered":"2024-04-22T13:24:06.258717+0000","last_clean":"2024-04-22T13:24:06.258717+0000","last_became_active":"2024-04-22T13:22:05.242772+0000","last_became_peered":"2024-04-22T13:22:05.242772+0000","last_unstale":"2024-04-22T13:24:06.258717+0000","last_undegraded":"2024-04-22T13:24:06.258717+0000","last_fullsized":"2024-04-22T13:24:06.258717+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:33:42.359679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164258+0000","last_change":"2024-04-22T13:22:04.244381+0000","last_active":"2024-04-22T13:24:06.164258+0000","last_peered":"2024-04-22T13:24:06.164258+0000","last_clean":"2024-04-22T13:24:06.164258+0000","last_became_active":"2024-04-22T13:22:04.243746+0000","last_became_peered":"2024-04-22T13:22:04.243746+0000","last_unstale":"2024-04-22T13:24:06.164258+0000","last_undegraded":"2024-04-22T13:24:06.164258+0000","last_fullsized":"2024-04-22T13:24:06.164258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:15:36.019930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.256753+0000","last_change":"2024-04-22T13:22:05.245394+0000","last_active":"2024-04-22T13:24:06.256753+0000","last_peered":"2024-04-22T13:24:06.256753+0000","last_clean":"2024-04-22T13:24:06.256753+0000","last_became_active":"2024-04-22T13:22:05.243931+0000","last_became_peered":"2024-04-22T13:22:05.243931+0000","last_unstale":"2024-04-22T13:24:06.256753+0000","last_undegraded":"2024-04-22T13:24:06.256753+0000","last_fullsized":"2024-04-22T13:24:06.256753+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:11:10.790842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'6","reported_seq":260,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.533504+0000","last_change":"2024-04-22T13:22:04.242583+0000","last_active":"2024-04-22T13:24:06.533504+0000","last_peered":"2024-04-22T13:24:06.533504+0000","last_clean":"2024-04-22T13:24:06.533504+0000","last_became_active":"2024-04-22T13:22:04.242261+0000","last_became_peered":"2024-04-22T13:22:04.242261+0000","last_unstale":"2024-04-22T13:24:06.533504+0000","last_undegraded":"2024-04-22T13:24:06.533504+0000","last_fullsized":"2024-04-22T13:24:06.533504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:37:47.618573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":233,"num_read_kb":233,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164258+0000","last_change":"2024-04-22T13:22:05.255293+0000","last_active":"2024-04-22T13:24:06.164258+0000","last_peered":"2024-04-22T13:24:06.164258+0000","last_clean":"2024-04-22T13:24:06.164258+0000","last_became_active":"2024-04-22T13:22:05.253949+0000","last_became_peered":"2024-04-22T13:22:05.253949+0000","last_unstale":"2024-04-22T13:24:06.164258+0000","last_undegraded":"2024-04-22T13:24:06.164258+0000","last_fullsized":"2024-04-22T13:24:06.164258+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:41:08.754989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409503+0000","last_change":"2024-04-22T13:22:04.241529+0000","last_active":"2024-04-22T13:24:06.409503+0000","last_peered":"2024-04-22T13:24:06.409503+0000","last_clean":"2024-04-22T13:24:06.409503+0000","last_became_active":"2024-04-22T13:22:04.241164+0000","last_became_peered":"2024-04-22T13:22:04.241164+0000","last_unstale":"2024-04-22T13:24:06.409503+0000","last_undegraded":"2024-04-22T13:24:06.409503+0000","last_fullsized":"2024-04-22T13:24:06.409503+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:16:49.387181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.412225+0000","last_change":"2024-04-22T13:22:05.241623+0000","last_active":"2024-04-22T13:24:06.412225+0000","last_peered":"2024-04-22T13:24:06.412225+0000","last_clean":"2024-04-22T13:24:06.412225+0000","last_became_active":"2024-04-22T13:22:05.241150+0000","last_became_peered":"2024-04-22T13:22:05.241150+0000","last_unstale":"2024-04-22T13:24:06.412225+0000","last_undegraded":"2024-04-22T13:24:06.412225+0000","last_fullsized":"2024-04-22T13:24:06.412225+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:17:28.412366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164893+0000","last_change":"2024-04-22T13:22:04.247852+0000","last_active":"2024-04-22T13:24:06.164893+0000","last_peered":"2024-04-22T13:24:06.164893+0000","last_clean":"2024-04-22T13:24:06.164893+0000","last_became_active":"2024-04-22T13:22:04.247550+0000","last_became_peered":"2024-04-22T13:22:04.247550+0000","last_unstale":"2024-04-22T13:24:06.164893+0000","last_undegraded":"2024-04-22T13:24:06.164893+0000","last_fullsized":"2024-04-22T13:24:06.164893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:55:01.928435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164057+0000","last_change":"2024-04-22T13:22:05.260050+0000","last_active":"2024-04-22T13:24:06.164057+0000","last_peered":"2024-04-22T13:24:06.164057+0000","last_clean":"2024-04-22T13:24:06.164057+0000","last_became_active":"2024-04-22T13:22:05.259098+0000","last_became_peered":"2024-04-22T13:22:05.259098+0000","last_unstale":"2024-04-22T13:24:06.164057+0000","last_undegraded":"2024-04-22T13:24:06.164057+0000","last_fullsized":"2024-04-22T13:24:06.164057+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:05:06.245492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'36","reported_seq":305,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.534247+0000","last_change":"2024-04-22T13:22:04.240233+0000","last_active":"2024-04-22T13:24:06.534247+0000","last_peered":"2024-04-22T13:24:06.534247+0000","last_clean":"2024-04-22T13:24:06.534247+0000","last_became_active":"2024-04-22T13:22:04.239890+0000","last_became_peered":"2024-04-22T13:22:04.239890+0000","last_unstale":"2024-04-22T13:24:06.534247+0000","last_undegraded":"2024-04-22T13:24:06.534247+0000","last_fullsized":"2024-04-22T13:24:06.534247+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:04:21.547422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64727,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":248,"num_read_kb":188,"num_write":38,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258900+0000","last_change":"2024-04-22T13:22:05.257756+0000","last_active":"2024-04-22T13:24:06.258900+0000","last_peered":"2024-04-22T13:24:06.258900+0000","last_clean":"2024-04-22T13:24:06.258900+0000","last_became_active":"2024-04-22T13:22:05.257442+0000","last_became_peered":"2024-04-22T13:22:05.257442+0000","last_unstale":"2024-04-22T13:24:06.258900+0000","last_undegraded":"2024-04-22T13:24:06.258900+0000","last_fullsized":"2024-04-22T13:24:06.258900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:16:53.465689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.414923+0000","last_change":"2024-04-22T13:22:04.240815+0000","last_active":"2024-04-22T13:24:06.414923+0000","last_peered":"2024-04-22T13:24:06.414923+0000","last_clean":"2024-04-22T13:24:06.414923+0000","last_became_active":"2024-04-22T13:22:04.240218+0000","last_became_peered":"2024-04-22T13:22:04.240218+0000","last_unstale":"2024-04-22T13:24:06.414923+0000","last_undegraded":"2024-04-22T13:24:06.414923+0000","last_fullsized":"2024-04-22T13:24:06.414923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:29:29.148764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162257+0000","last_change":"2024-04-22T13:22:05.248797+0000","last_active":"2024-04-22T13:24:06.162257+0000","last_peered":"2024-04-22T13:24:06.162257+0000","last_clean":"2024-04-22T13:24:06.162257+0000","last_became_active":"2024-04-22T13:22:05.248435+0000","last_became_peered":"2024-04-22T13:22:05.248435+0000","last_unstale":"2024-04-22T13:24:06.162257+0000","last_undegraded":"2024-04-22T13:24:06.162257+0000","last_fullsized":"2024-04-22T13:24:06.162257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:32:31.970494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164389+0000","last_change":"2024-04-22T13:22:04.245303+0000","last_active":"2024-04-22T13:24:06.164389+0000","last_peered":"2024-04-22T13:24:06.164389+0000","last_clean":"2024-04-22T13:24:06.164389+0000","last_became_active":"2024-04-22T13:22:04.244903+0000","last_became_peered":"2024-04-22T13:22:04.244903+0000","last_unstale":"2024-04-22T13:24:06.164389+0000","last_undegraded":"2024-04-22T13:24:06.164389+0000","last_fullsized":"2024-04-22T13:24:06.164389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:34:32.174858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409264+0000","last_change":"2024-04-22T13:22:05.246318+0000","last_active":"2024-04-22T13:24:06.409264+0000","last_peered":"2024-04-22T13:24:06.409264+0000","last_clean":"2024-04-22T13:24:06.409264+0000","last_became_active":"2024-04-22T13:22:05.245899+0000","last_became_peered":"2024-04-22T13:22:05.245899+0000","last_unstale":"2024-04-22T13:24:06.409264+0000","last_undegraded":"2024-04-22T13:24:06.409264+0000","last_fullsized":"2024-04-22T13:24:06.409264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:27:42.789058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'8","reported_seq":148,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.591632+0000","last_change":"2024-04-22T13:22:04.240612+0000","last_active":"2024-04-22T13:24:06.591632+0000","last_peered":"2024-04-22T13:24:06.591632+0000","last_clean":"2024-04-22T13:24:06.591632+0000","last_became_active":"2024-04-22T13:22:04.240290+0000","last_became_peered":"2024-04-22T13:22:04.240290+0000","last_unstale":"2024-04-22T13:24:06.591632+0000","last_undegraded":"2024-04-22T13:24:06.591632+0000","last_fullsized":"2024-04-22T13:24:06.591632+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:00:50.285553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.408976+0000","last_change":"2024-04-22T13:22:05.253205+0000","last_active":"2024-04-22T13:24:06.408976+0000","last_peered":"2024-04-22T13:24:06.408976+0000","last_clean":"2024-04-22T13:24:06.408976+0000","last_became_active":"2024-04-22T13:22:05.252865+0000","last_became_peered":"2024-04-22T13:22:05.252865+0000","last_unstale":"2024-04-22T13:24:06.408976+0000","last_undegraded":"2024-04-22T13:24:06.408976+0000","last_fullsized":"2024-04-22T13:24:06.408976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:19:12.482371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258636+0000","last_change":"2024-04-22T13:22:04.248938+0000","last_active":"2024-04-22T13:24:06.258636+0000","last_peered":"2024-04-22T13:24:06.258636+0000","last_clean":"2024-04-22T13:24:06.258636+0000","last_became_active":"2024-04-22T13:22:04.248440+0000","last_became_peered":"2024-04-22T13:22:04.248440+0000","last_unstale":"2024-04-22T13:24:06.258636+0000","last_undegraded":"2024-04-22T13:24:06.258636+0000","last_fullsized":"2024-04-22T13:24:06.258636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:46:50.928067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164391+0000","last_change":"2024-04-22T13:22:05.254664+0000","last_active":"2024-04-22T13:24:06.164391+0000","last_peered":"2024-04-22T13:24:06.164391+0000","last_clean":"2024-04-22T13:24:06.164391+0000","last_became_active":"2024-04-22T13:22:05.254304+0000","last_became_peered":"2024-04-22T13:22:05.254304+0000","last_unstale":"2024-04-22T13:24:06.164391+0000","last_undegraded":"2024-04-22T13:24:06.164391+0000","last_fullsized":"2024-04-22T13:24:06.164391+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:58:49.345573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259033+0000","last_change":"2024-04-22T13:22:04.244128+0000","last_active":"2024-04-22T13:24:06.259033+0000","last_peered":"2024-04-22T13:24:06.259033+0000","last_clean":"2024-04-22T13:24:06.259033+0000","last_became_active":"2024-04-22T13:22:04.243738+0000","last_became_peered":"2024-04-22T13:22:04.243738+0000","last_unstale":"2024-04-22T13:24:06.259033+0000","last_undegraded":"2024-04-22T13:24:06.259033+0000","last_fullsized":"2024-04-22T13:24:06.259033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:23:52.220584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418387+0000","last_change":"2024-04-22T13:22:05.250910+0000","last_active":"2024-04-22T13:24:06.418387+0000","last_peered":"2024-04-22T13:24:06.418387+0000","last_clean":"2024-04-22T13:24:06.418387+0000","last_became_active":"2024-04-22T13:22:05.250410+0000","last_became_peered":"2024-04-22T13:22:05.250410+0000","last_unstale":"2024-04-22T13:24:06.418387+0000","last_undegraded":"2024-04-22T13:24:06.418387+0000","last_fullsized":"2024-04-22T13:24:06.418387+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:04:25.492677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.256761+0000","last_change":"2024-04-22T13:22:04.245889+0000","last_active":"2024-04-22T13:24:06.256761+0000","last_peered":"2024-04-22T13:24:06.256761+0000","last_clean":"2024-04-22T13:24:06.256761+0000","last_became_active":"2024-04-22T13:22:04.245474+0000","last_became_peered":"2024-04-22T13:22:04.245474+0000","last_unstale":"2024-04-22T13:24:06.256761+0000","last_undegraded":"2024-04-22T13:24:06.256761+0000","last_fullsized":"2024-04-22T13:24:06.256761+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:22:12.626950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.266594+0000","last_change":"2024-04-22T13:22:05.250978+0000","last_active":"2024-04-22T13:24:06.266594+0000","last_peered":"2024-04-22T13:24:06.266594+0000","last_clean":"2024-04-22T13:24:06.266594+0000","last_became_active":"2024-04-22T13:22:05.250463+0000","last_became_peered":"2024-04-22T13:22:05.250463+0000","last_unstale":"2024-04-22T13:24:06.266594+0000","last_undegraded":"2024-04-22T13:24:06.266594+0000","last_fullsized":"2024-04-22T13:24:06.266594+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:45:18.591072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'7","reported_seq":146,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.560650+0000","last_change":"2024-04-22T13:22:04.242503+0000","last_active":"2024-04-22T13:24:06.560650+0000","last_peered":"2024-04-22T13:24:06.560650+0000","last_clean":"2024-04-22T13:24:06.560650+0000","last_became_active":"2024-04-22T13:22:04.242143+0000","last_became_peered":"2024-04-22T13:22:04.242143+0000","last_unstale":"2024-04-22T13:24:06.560650+0000","last_undegraded":"2024-04-22T13:24:06.560650+0000","last_fullsized":"2024-04-22T13:24:06.560650+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:40.236377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.266483+0000","last_change":"2024-04-22T13:22:05.252387+0000","last_active":"2024-04-22T13:24:06.266483+0000","last_peered":"2024-04-22T13:24:06.266483+0000","last_clean":"2024-04-22T13:24:06.266483+0000","last_became_active":"2024-04-22T13:22:05.252033+0000","last_became_peered":"2024-04-22T13:22:05.252033+0000","last_unstale":"2024-04-22T13:24:06.266483+0000","last_undegraded":"2024-04-22T13:24:06.266483+0000","last_fullsized":"2024-04-22T13:24:06.266483+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:55:01.791283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164402+0000","last_change":"2024-04-22T13:22:05.248394+0000","last_active":"2024-04-22T13:24:06.164402+0000","last_peered":"2024-04-22T13:24:06.164402+0000","last_clean":"2024-04-22T13:24:06.164402+0000","last_became_active":"2024-04-22T13:22:05.248024+0000","last_became_peered":"2024-04-22T13:22:05.248024+0000","last_unstale":"2024-04-22T13:24:06.164402+0000","last_undegraded":"2024-04-22T13:24:06.164402+0000","last_fullsized":"2024-04-22T13:24:06.164402+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:49:36.556136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'7","reported_seq":145,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.590537+0000","last_change":"2024-04-22T13:22:04.247692+0000","last_active":"2024-04-22T13:24:06.590537+0000","last_peered":"2024-04-22T13:24:06.590537+0000","last_clean":"2024-04-22T13:24:06.590537+0000","last_became_active":"2024-04-22T13:22:04.247345+0000","last_became_peered":"2024-04-22T13:22:04.247345+0000","last_unstale":"2024-04-22T13:24:06.590537+0000","last_undegraded":"2024-04-22T13:24:06.590537+0000","last_fullsized":"2024-04-22T13:24:06.590537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:07:49.803248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":116,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259293+0000","last_change":"2024-04-22T13:22:05.241022+0000","last_active":"2024-04-22T13:24:06.259293+0000","last_peered":"2024-04-22T13:24:06.259293+0000","last_clean":"2024-04-22T13:24:06.259293+0000","last_became_active":"2024-04-22T13:22:05.240635+0000","last_became_peered":"2024-04-22T13:22:05.240635+0000","last_unstale":"2024-04-22T13:24:06.259293+0000","last_undegraded":"2024-04-22T13:24:06.259293+0000","last_fullsized":"2024-04-22T13:24:06.259293+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:50:02.012260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411229+0000","last_change":"2024-04-22T13:22:04.236817+0000","last_active":"2024-04-22T13:24:06.411229+0000","last_peered":"2024-04-22T13:24:06.411229+0000","last_clean":"2024-04-22T13:24:06.411229+0000","last_became_active":"2024-04-22T13:22:04.236484+0000","last_became_peered":"2024-04-22T13:22:04.236484+0000","last_unstale":"2024-04-22T13:24:06.411229+0000","last_undegraded":"2024-04-22T13:24:06.411229+0000","last_fullsized":"2024-04-22T13:24:06.411229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:27:41.053327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":357,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409048+0000","last_change":"2024-04-22T13:20:53.094123+0000","last_active":"2024-04-22T13:24:06.409048+0000","last_peered":"2024-04-22T13:24:06.409048+0000","last_clean":"2024-04-22T13:24:06.409048+0000","last_became_active":"2024-04-22T13:20:53.093664+0000","last_became_peered":"2024-04-22T13:20:53.093664+0000","last_unstale":"2024-04-22T13:24:06.409048+0000","last_undegraded":"2024-04-22T13:24:06.409048+0000","last_fullsized":"2024-04-22T13:24:06.409048+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:17:33.201786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:17:33.201786+0000","last_clean_scrub_stamp":"2024-04-22T13:17:33.201786+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:46:58.986399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162213+0000","last_change":"2024-04-22T13:22:05.247766+0000","last_active":"2024-04-22T13:24:06.162213+0000","last_peered":"2024-04-22T13:24:06.162213+0000","last_clean":"2024-04-22T13:24:06.162213+0000","last_became_active":"2024-04-22T13:22:05.247362+0000","last_became_peered":"2024-04-22T13:22:05.247362+0000","last_unstale":"2024-04-22T13:24:06.162213+0000","last_undegraded":"2024-04-22T13:24:06.162213+0000","last_fullsized":"2024-04-22T13:24:06.162213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:25:00.126638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409239+0000","last_change":"2024-04-22T13:22:04.243493+0000","last_active":"2024-04-22T13:24:06.409239+0000","last_peered":"2024-04-22T13:24:06.409239+0000","last_clean":"2024-04-22T13:24:06.409239+0000","last_became_active":"2024-04-22T13:22:04.243190+0000","last_became_peered":"2024-04-22T13:22:04.243190+0000","last_unstale":"2024-04-22T13:24:06.409239+0000","last_undegraded":"2024-04-22T13:24:06.409239+0000","last_fullsized":"2024-04-22T13:24:06.409239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:15:09.025152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258892+0000","last_change":"2024-04-22T13:22:04.249222+0000","last_active":"2024-04-22T13:24:06.258892+0000","last_peered":"2024-04-22T13:24:06.258892+0000","last_clean":"2024-04-22T13:24:06.258892+0000","last_became_active":"2024-04-22T13:22:04.248926+0000","last_became_peered":"2024-04-22T13:22:04.248926+0000","last_unstale":"2024-04-22T13:24:06.258892+0000","last_undegraded":"2024-04-22T13:24:06.258892+0000","last_fullsized":"2024-04-22T13:24:06.258892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:33:50.168426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162433+0000","last_change":"2024-04-22T13:22:05.248199+0000","last_active":"2024-04-22T13:24:06.162433+0000","last_peered":"2024-04-22T13:24:06.162433+0000","last_clean":"2024-04-22T13:24:06.162433+0000","last_became_active":"2024-04-22T13:22:05.247873+0000","last_became_peered":"2024-04-22T13:22:05.247873+0000","last_unstale":"2024-04-22T13:24:06.162433+0000","last_undegraded":"2024-04-22T13:24:06.162433+0000","last_fullsized":"2024-04-22T13:24:06.162433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:40:29.353901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'48","reported_seq":92,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.370574+0000","last_change":"2024-04-22T13:22:04.243474+0000","last_active":"2024-04-22T13:24:06.370574+0000","last_peered":"2024-04-22T13:24:06.370574+0000","last_clean":"2024-04-22T13:24:06.370574+0000","last_became_active":"2024-04-22T13:22:04.243151+0000","last_became_peered":"2024-04-22T13:22:04.243151+0000","last_unstale":"2024-04-22T13:24:06.370574+0000","last_undegraded":"2024-04-22T13:24:06.370574+0000","last_fullsized":"2024-04-22T13:24:06.370574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:18:25.140715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259264+0000","last_change":"2024-04-22T13:22:05.243120+0000","last_active":"2024-04-22T13:24:06.259264+0000","last_peered":"2024-04-22T13:24:06.259264+0000","last_clean":"2024-04-22T13:24:06.259264+0000","last_became_active":"2024-04-22T13:22:05.242726+0000","last_became_peered":"2024-04-22T13:22:05.242726+0000","last_unstale":"2024-04-22T13:24:06.259264+0000","last_undegraded":"2024-04-22T13:24:06.259264+0000","last_fullsized":"2024-04-22T13:24:06.259264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:06:16.087884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411424+0000","last_change":"2024-04-22T13:22:04.239672+0000","last_active":"2024-04-22T13:24:06.411424+0000","last_peered":"2024-04-22T13:24:06.411424+0000","last_clean":"2024-04-22T13:24:06.411424+0000","last_became_active":"2024-04-22T13:22:04.238776+0000","last_became_peered":"2024-04-22T13:22:04.238776+0000","last_unstale":"2024-04-22T13:24:06.411424+0000","last_undegraded":"2024-04-22T13:24:06.411424+0000","last_fullsized":"2024-04-22T13:24:06.411424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:43:21.752547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.412035+0000","last_change":"2024-04-22T13:22:05.242089+0000","last_active":"2024-04-22T13:24:06.412035+0000","last_peered":"2024-04-22T13:24:06.412035+0000","last_clean":"2024-04-22T13:24:06.412035+0000","last_became_active":"2024-04-22T13:22:05.241681+0000","last_became_peered":"2024-04-22T13:22:05.241681+0000","last_unstale":"2024-04-22T13:24:06.412035+0000","last_undegraded":"2024-04-22T13:24:06.412035+0000","last_fullsized":"2024-04-22T13:24:06.412035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:58:49.906188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.163728+0000","last_change":"2024-04-22T13:23:05.513123+0000","last_active":"2024-04-22T13:24:06.163728+0000","last_peered":"2024-04-22T13:24:06.163728+0000","last_clean":"2024-04-22T13:24:06.163728+0000","last_became_active":"2024-04-22T13:23:05.509367+0000","last_became_peered":"2024-04-22T13:23:05.509367+0000","last_unstale":"2024-04-22T13:24:06.163728+0000","last_undegraded":"2024-04-22T13:24:06.163728+0000","last_fullsized":"2024-04-22T13:24:06.163728+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:28:41.770957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258638+0000","last_change":"2024-04-22T13:22:05.250120+0000","last_active":"2024-04-22T13:24:06.258638+0000","last_peered":"2024-04-22T13:24:06.258638+0000","last_clean":"2024-04-22T13:24:06.258638+0000","last_became_active":"2024-04-22T13:22:05.247936+0000","last_became_peered":"2024-04-22T13:22:05.247936+0000","last_unstale":"2024-04-22T13:24:06.258638+0000","last_undegraded":"2024-04-22T13:24:06.258638+0000","last_fullsized":"2024-04-22T13:24:06.258638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:27:38.437918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258902+0000","last_change":"2024-04-22T13:22:04.244026+0000","last_active":"2024-04-22T13:24:06.258902+0000","last_peered":"2024-04-22T13:24:06.258902+0000","last_clean":"2024-04-22T13:24:06.258902+0000","last_became_active":"2024-04-22T13:22:04.243518+0000","last_became_peered":"2024-04-22T13:22:04.243518+0000","last_unstale":"2024-04-22T13:24:06.258902+0000","last_undegraded":"2024-04-22T13:24:06.258902+0000","last_fullsized":"2024-04-22T13:24:06.258902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:54:48.791874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418220+0000","last_change":"2024-04-22T13:22:05.252674+0000","last_active":"2024-04-22T13:24:06.418220+0000","last_peered":"2024-04-22T13:24:06.418220+0000","last_clean":"2024-04-22T13:24:06.418220+0000","last_became_active":"2024-04-22T13:22:05.251087+0000","last_became_peered":"2024-04-22T13:22:05.251087+0000","last_unstale":"2024-04-22T13:24:06.418220+0000","last_undegraded":"2024-04-22T13:24:06.418220+0000","last_fullsized":"2024-04-22T13:24:06.418220+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:20:06.873412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.414640+0000","last_change":"2024-04-22T13:22:04.244409+0000","last_active":"2024-04-22T13:24:06.414640+0000","last_peered":"2024-04-22T13:24:06.414640+0000","last_clean":"2024-04-22T13:24:06.414640+0000","last_became_active":"2024-04-22T13:22:04.244104+0000","last_became_peered":"2024-04-22T13:22:04.244104+0000","last_unstale":"2024-04-22T13:24:06.414640+0000","last_undegraded":"2024-04-22T13:24:06.414640+0000","last_fullsized":"2024-04-22T13:24:06.414640+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:39:14.714755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259066+0000","last_change":"2024-04-22T13:22:05.245730+0000","last_active":"2024-04-22T13:24:06.259066+0000","last_peered":"2024-04-22T13:24:06.259066+0000","last_clean":"2024-04-22T13:24:06.259066+0000","last_became_active":"2024-04-22T13:22:05.245366+0000","last_became_peered":"2024-04-22T13:22:05.245366+0000","last_unstale":"2024-04-22T13:24:06.259066+0000","last_undegraded":"2024-04-22T13:24:06.259066+0000","last_fullsized":"2024-04-22T13:24:06.259066+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:31.935258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418212+0000","last_change":"2024-04-22T13:23:05.514197+0000","last_active":"2024-04-22T13:24:06.418212+0000","last_peered":"2024-04-22T13:24:06.418212+0000","last_clean":"2024-04-22T13:24:06.418212+0000","last_became_active":"2024-04-22T13:23:05.509431+0000","last_became_peered":"2024-04-22T13:23:05.509431+0000","last_unstale":"2024-04-22T13:24:06.418212+0000","last_undegraded":"2024-04-22T13:24:06.418212+0000","last_fullsized":"2024-04-22T13:24:06.418212+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:43:59.825567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409273+0000","last_change":"2024-04-22T13:22:05.242146+0000","last_active":"2024-04-22T13:24:06.409273+0000","last_peered":"2024-04-22T13:24:06.409273+0000","last_clean":"2024-04-22T13:24:06.409273+0000","last_became_active":"2024-04-22T13:22:05.241802+0000","last_became_peered":"2024-04-22T13:22:05.241802+0000","last_unstale":"2024-04-22T13:24:06.409273+0000","last_undegraded":"2024-04-22T13:24:06.409273+0000","last_fullsized":"2024-04-22T13:24:06.409273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:49:49.002123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'7","reported_seq":147,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.562849+0000","last_change":"2024-04-22T13:22:04.248884+0000","last_active":"2024-04-22T13:24:06.562849+0000","last_peered":"2024-04-22T13:24:06.562849+0000","last_clean":"2024-04-22T13:24:06.562849+0000","last_became_active":"2024-04-22T13:22:04.248393+0000","last_became_peered":"2024-04-22T13:22:04.248393+0000","last_unstale":"2024-04-22T13:24:06.562849+0000","last_undegraded":"2024-04-22T13:24:06.562849+0000","last_fullsized":"2024-04-22T13:24:06.562849+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:36:14.687457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1869,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409545+0000","last_change":"2024-04-22T13:22:05.256881+0000","last_active":"2024-04-22T13:24:06.409545+0000","last_peered":"2024-04-22T13:24:06.409545+0000","last_clean":"2024-04-22T13:24:06.409545+0000","last_became_active":"2024-04-22T13:22:05.255967+0000","last_became_peered":"2024-04-22T13:22:05.255967+0000","last_unstale":"2024-04-22T13:24:06.409545+0000","last_undegraded":"2024-04-22T13:24:06.409545+0000","last_fullsized":"2024-04-22T13:24:06.409545+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:24:21.137344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'41","reported_seq":70,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.367821+0000","last_change":"2024-04-22T13:22:04.247303+0000","last_active":"2024-04-22T13:24:06.367821+0000","last_peered":"2024-04-22T13:24:06.367821+0000","last_clean":"2024-04-22T13:24:06.367821+0000","last_became_active":"2024-04-22T13:22:04.246742+0000","last_became_peered":"2024-04-22T13:22:04.246742+0000","last_unstale":"2024-04-22T13:24:06.367821+0000","last_undegraded":"2024-04-22T13:24:06.367821+0000","last_fullsized":"2024-04-22T13:24:06.367821+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:45:03.189042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409288+0000","last_change":"2024-04-22T13:22:05.251754+0000","last_active":"2024-04-22T13:24:06.409288+0000","last_peered":"2024-04-22T13:24:06.409288+0000","last_clean":"2024-04-22T13:24:06.409288+0000","last_became_active":"2024-04-22T13:22:05.251155+0000","last_became_peered":"2024-04-22T13:22:05.251155+0000","last_unstale":"2024-04-22T13:24:06.409288+0000","last_undegraded":"2024-04-22T13:24:06.409288+0000","last_fullsized":"2024-04-22T13:24:06.409288+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:08:46.682600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162485+0000","last_change":"2024-04-22T13:22:04.243382+0000","last_active":"2024-04-22T13:24:06.162485+0000","last_peered":"2024-04-22T13:24:06.162485+0000","last_clean":"2024-04-22T13:24:06.162485+0000","last_became_active":"2024-04-22T13:22:04.243055+0000","last_became_peered":"2024-04-22T13:22:04.243055+0000","last_unstale":"2024-04-22T13:24:06.162485+0000","last_undegraded":"2024-04-22T13:24:06.162485+0000","last_fullsized":"2024-04-22T13:24:06.162485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:19:58.379645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409542+0000","last_change":"2024-04-22T13:22:05.249527+0000","last_active":"2024-04-22T13:24:06.409542+0000","last_peered":"2024-04-22T13:24:06.409542+0000","last_clean":"2024-04-22T13:24:06.409542+0000","last_became_active":"2024-04-22T13:22:05.249207+0000","last_became_peered":"2024-04-22T13:22:05.249207+0000","last_unstale":"2024-04-22T13:24:06.409542+0000","last_undegraded":"2024-04-22T13:24:06.409542+0000","last_fullsized":"2024-04-22T13:24:06.409542+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:50:02.755288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418234+0000","last_change":"2024-04-22T13:23:05.508324+0000","last_active":"2024-04-22T13:24:06.418234+0000","last_peered":"2024-04-22T13:24:06.418234+0000","last_clean":"2024-04-22T13:24:06.418234+0000","last_became_active":"2024-04-22T13:23:05.507814+0000","last_became_peered":"2024-04-22T13:23:05.507814+0000","last_unstale":"2024-04-22T13:24:06.418234+0000","last_undegraded":"2024-04-22T13:24:06.418234+0000","last_fullsized":"2024-04-22T13:24:06.418234+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:46:20.523715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.414586+0000","last_change":"2024-04-22T13:23:05.875683+0000","last_active":"2024-04-22T13:24:06.414586+0000","last_peered":"2024-04-22T13:24:06.414586+0000","last_clean":"2024-04-22T13:24:06.414586+0000","last_became_active":"2024-04-22T13:23:05.875150+0000","last_became_peered":"2024-04-22T13:23:05.875150+0000","last_unstale":"2024-04-22T13:24:06.414586+0000","last_undegraded":"2024-04-22T13:24:06.414586+0000","last_fullsized":"2024-04-22T13:24:06.414586+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:12:47.592230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.414529+0000","last_change":"2024-04-22T13:22:05.247882+0000","last_active":"2024-04-22T13:24:06.414529+0000","last_peered":"2024-04-22T13:24:06.414529+0000","last_clean":"2024-04-22T13:24:06.414529+0000","last_became_active":"2024-04-22T13:22:05.247525+0000","last_became_peered":"2024-04-22T13:22:05.247525+0000","last_unstale":"2024-04-22T13:24:06.414529+0000","last_undegraded":"2024-04-22T13:24:06.414529+0000","last_fullsized":"2024-04-22T13:24:06.414529+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:46:04.349076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411567+0000","last_change":"2024-04-22T13:22:05.243716+0000","last_active":"2024-04-22T13:24:06.411567+0000","last_peered":"2024-04-22T13:24:06.411567+0000","last_clean":"2024-04-22T13:24:06.411567+0000","last_became_active":"2024-04-22T13:22:05.243397+0000","last_became_peered":"2024-04-22T13:22:05.243397+0000","last_unstale":"2024-04-22T13:24:06.411567+0000","last_undegraded":"2024-04-22T13:24:06.411567+0000","last_fullsized":"2024-04-22T13:24:06.411567+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:16:21.236339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162619+0000","last_change":"2024-04-22T13:22:05.246042+0000","last_active":"2024-04-22T13:24:06.162619+0000","last_peered":"2024-04-22T13:24:06.162619+0000","last_clean":"2024-04-22T13:24:06.162619+0000","last_became_active":"2024-04-22T13:22:05.245669+0000","last_became_peered":"2024-04-22T13:22:05.245669+0000","last_unstale":"2024-04-22T13:24:06.162619+0000","last_undegraded":"2024-04-22T13:24:06.162619+0000","last_fullsized":"2024-04-22T13:24:06.162619+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:49:05.539641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418321+0000","last_change":"2024-04-22T13:22:05.257009+0000","last_active":"2024-04-22T13:24:06.418321+0000","last_peered":"2024-04-22T13:24:06.418321+0000","last_clean":"2024-04-22T13:24:06.418321+0000","last_became_active":"2024-04-22T13:22:05.254327+0000","last_became_peered":"2024-04-22T13:22:05.254327+0000","last_unstale":"2024-04-22T13:24:06.418321+0000","last_undegraded":"2024-04-22T13:24:06.418321+0000","last_fullsized":"2024-04-22T13:24:06.418321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:29:12.482525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.415339+0000","last_change":"2024-04-22T13:22:05.260575+0000","last_active":"2024-04-22T13:24:06.415339+0000","last_peered":"2024-04-22T13:24:06.415339+0000","last_clean":"2024-04-22T13:24:06.415339+0000","last_became_active":"2024-04-22T13:22:05.260181+0000","last_became_peered":"2024-04-22T13:22:05.260181+0000","last_unstale":"2024-04-22T13:24:06.415339+0000","last_undegraded":"2024-04-22T13:24:06.415339+0000","last_fullsized":"2024-04-22T13:24:06.415339+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:49:20.064368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259116+0000","last_change":"2024-04-22T13:22:05.248199+0000","last_active":"2024-04-22T13:24:06.259116+0000","last_peered":"2024-04-22T13:24:06.259116+0000","last_clean":"2024-04-22T13:24:06.259116+0000","last_became_active":"2024-04-22T13:22:05.247858+0000","last_became_peered":"2024-04-22T13:22:05.247858+0000","last_unstale":"2024-04-22T13:24:06.259116+0000","last_undegraded":"2024-04-22T13:24:06.259116+0000","last_fullsized":"2024-04-22T13:24:06.259116+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:49:21.228105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164341+0000","last_change":"2024-04-22T13:22:05.239137+0000","last_active":"2024-04-22T13:24:06.164341+0000","last_peered":"2024-04-22T13:24:06.164341+0000","last_clean":"2024-04-22T13:24:06.164341+0000","last_became_active":"2024-04-22T13:22:05.238807+0000","last_became_peered":"2024-04-22T13:22:05.238807+0000","last_unstale":"2024-04-22T13:24:06.164341+0000","last_undegraded":"2024-04-22T13:24:06.164341+0000","last_fullsized":"2024-04-22T13:24:06.164341+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:02:48.587097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258756+0000","last_change":"2024-04-22T13:22:05.249080+0000","last_active":"2024-04-22T13:24:06.258756+0000","last_peered":"2024-04-22T13:24:06.258756+0000","last_clean":"2024-04-22T13:24:06.258756+0000","last_became_active":"2024-04-22T13:22:05.248733+0000","last_became_peered":"2024-04-22T13:22:05.248733+0000","last_unstale":"2024-04-22T13:24:06.258756+0000","last_undegraded":"2024-04-22T13:24:06.258756+0000","last_fullsized":"2024-04-22T13:24:06.258756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:58.735941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.163779+0000","last_change":"2024-04-22T13:22:05.255490+0000","last_active":"2024-04-22T13:24:06.163779+0000","last_peered":"2024-04-22T13:24:06.163779+0000","last_clean":"2024-04-22T13:24:06.163779+0000","last_became_active":"2024-04-22T13:22:05.253644+0000","last_became_peered":"2024-04-22T13:22:05.253644+0000","last_unstale":"2024-04-22T13:24:06.163779+0000","last_undegraded":"2024-04-22T13:24:06.163779+0000","last_fullsized":"2024-04-22T13:24:06.163779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:31:01.816182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162443+0000","last_change":"2024-04-22T13:22:05.240988+0000","last_active":"2024-04-22T13:24:06.162443+0000","last_peered":"2024-04-22T13:24:06.162443+0000","last_clean":"2024-04-22T13:24:06.162443+0000","last_became_active":"2024-04-22T13:22:05.240642+0000","last_became_peered":"2024-04-22T13:22:05.240642+0000","last_unstale":"2024-04-22T13:24:06.162443+0000","last_undegraded":"2024-04-22T13:24:06.162443+0000","last_fullsized":"2024-04-22T13:24:06.162443+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:41:29.001979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418222+0000","last_change":"2024-04-22T13:22:05.252212+0000","last_active":"2024-04-22T13:24:06.418222+0000","last_peered":"2024-04-22T13:24:06.418222+0000","last_clean":"2024-04-22T13:24:06.418222+0000","last_became_active":"2024-04-22T13:22:05.251852+0000","last_became_peered":"2024-04-22T13:22:05.251852+0000","last_unstale":"2024-04-22T13:24:06.418222+0000","last_undegraded":"2024-04-22T13:24:06.418222+0000","last_fullsized":"2024-04-22T13:24:06.418222+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:13:55.132501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":598760,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1300,"num_read_kb":764,"num_write":368,"num_write_kb":1462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2162688,"data_stored":1796280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":298,"ondisk_log_size":298,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318310,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":472759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579945,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28796,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716356,"statfs":{"total":95995035648,"available":95965548544,"internally_reserved":0,"allocated":1822720,"data_stored":1260339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743471,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28388,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716764,"statfs":{"total":95995035648,"available":95965966336,"internally_reserved":0,"allocated":1400832,"data_stored":865206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005108,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28384,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716768,"statfs":{"total":95995035648,"available":95965970432,"internally_reserved":0,"allocated":1396736,"data_stored":865584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234041,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32744,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712408,"statfs":{"total":95995035648,"available":95961505792,"internally_reserved":0,"allocated":1601536,"data_stored":1066591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430270,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":991232,"data_stored":472489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":538129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":473745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1077248,"data_stored":538657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117070,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36836,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708316,"statfs":{"total":95995035648,"available":95957315584,"internally_reserved":0,"allocated":1536000,"data_stored":996348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855433,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":991232,"data_stored":472491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691907,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28724,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716428,"statfs":{"total":95995035648,"available":95965622272,"internally_reserved":0,"allocated":1748992,"data_stored":1196164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":134811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":198986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T13:24:11.335 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:24:11.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.476+0000 7fc8782a8640 1 -- 172.21.15.12:0/3551855691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc870106f10 msgr2=0x7fc8701072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:11.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.476+0000 7fc8782a8640 1 --2- 172.21.15.12:0/3551855691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc870106f10 0x7fc8701072f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009900 tx=0x7fc86c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:11.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.476+0000 7fc8782a8640 1 -- 172.21.15.12:0/3551855691 shutdown_connections 2024-04-22T13:24:11.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.476+0000 7fc8782a8640 1 --2- 172.21.15.12:0/3551855691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc870101ad0 0x7fc87010dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:11.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.476+0000 7fc8782a8640 1 --2- 172.21.15.12:0/3551855691 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc870101130 0x7fc870101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:11.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.476+0000 7fc8782a8640 1 --2- 172.21.15.12:0/3551855691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc870106f10 0x7fc8701072f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:11.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.476+0000 7fc8782a8640 1 -- 172.21.15.12:0/3551855691 >> 172.21.15.12:0/3551855691 conn(0x7fc870077e30 msgr2=0x7fc870078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:11.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.476+0000 7fc8782a8640 1 -- 172.21.15.12:0/3551855691 shutdown_connections 2024-04-22T13:24:11.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.476+0000 7fc8782a8640 1 -- 172.21.15.12:0/3551855691 wait complete. 2024-04-22T13:24:11.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.477+0000 7fc8782a8640 1 Processor -- start 2024-04-22T13:24:11.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.477+0000 7fc8782a8640 1 -- start start 2024-04-22T13:24:11.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.477+0000 7fc8782a8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc870101130 0x7fc870102b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:11.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.477+0000 7fc8782a8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc870101ad0 0x7fc870103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:11.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.477+0000 7fc8782a8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc870106f10 0x7fc8701035d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:11.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.477+0000 7fc8782a8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc8701110e0 con 0x7fc870101130 2024-04-22T13:24:11.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.477+0000 7fc8782a8640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc870110f60 con 0x7fc870106f10 2024-04-22T13:24:11.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.477+0000 7fc8782a8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc870111260 con 0x7fc870101ad0 2024-04-22T13:24:11.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.477+0000 7fc876044640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc870101130 0x7fc870102b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:11.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.477+0000 7fc876044640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc870101130 0x7fc870102b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35104/0 (socket says 172.21.15.12:35104) 2024-04-22T13:24:11.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.477+0000 7fc876044640 1 -- 172.21.15.12:0/1691260437 learned_addr learned my addr 172.21.15.12:0/1691260437 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:11.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.477+0000 7fc875843640 1 --2- 172.21.15.12:0/1691260437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc870101ad0 0x7fc870103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:11.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.477+0000 7fc876845640 1 --2- 172.21.15.12:0/1691260437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc870106f10 0x7fc8701035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:11.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.477+0000 7fc876044640 1 -- 172.21.15.12:0/1691260437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc870101ad0 msgr2=0x7fc870103090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:11.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.477+0000 7fc876044640 1 --2- 172.21.15.12:0/1691260437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc870101ad0 0x7fc870103090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:11.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.477+0000 7fc876044640 1 -- 172.21.15.12:0/1691260437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc870106f10 msgr2=0x7fc8701035d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:11.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.477+0000 7fc876044640 1 --2- 172.21.15.12:0/1691260437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc870106f10 0x7fc8701035d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:11.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.477+0000 7fc876044640 1 -- 172.21.15.12:0/1691260437 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8701a6bc0 con 0x7fc870101130 2024-04-22T13:24:11.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.477+0000 7fc875843640 1 --2- 172.21.15.12:0/1691260437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc870101ad0 0x7fc870103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:11.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.477+0000 7fc876845640 1 --2- 172.21.15.12:0/1691260437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc870106f10 0x7fc8701035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:11.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.477+0000 7fc876044640 1 --2- 172.21.15.12:0/1691260437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc870101130 0x7fc870102b50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc86c005940 tx=0x7fc86c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:11.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.478+0000 7fc8677fe640 1 -- 172.21.15.12:0/1691260437 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c043020 con 0x7fc870101130 2024-04-22T13:24:11.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.478+0000 7fc8677fe640 1 -- 172.21.15.12:0/1691260437 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc86c02caf0 con 0x7fc870101130 2024-04-22T13:24:11.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.478+0000 7fc8782a8640 1 -- 172.21.15.12:0/1691260437 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8701a6e50 con 0x7fc870101130 2024-04-22T13:24:11.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.478+0000 7fc8677fe640 1 -- 172.21.15.12:0/1691260437 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c003af0 con 0x7fc870101130 2024-04-22T13:24:11.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.478+0000 7fc8782a8640 1 -- 172.21.15.12:0/1691260437 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8701a7310 con 0x7fc870101130 2024-04-22T13:24:11.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.479+0000 7fc8782a8640 1 -- 172.21.15.12:0/1691260437 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc870069230 con 0x7fc870101130 2024-04-22T13:24:11.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.480+0000 7fc8677fe640 1 -- 172.21.15.12:0/1691260437 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fc86c046020 con 0x7fc870101130 2024-04-22T13:24:11.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.481+0000 7fc8677fe640 1 --2- 172.21.15.12:0/1691260437 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc85c0a3e40 0x7fc85c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:11.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.481+0000 7fc8677fe640 1 -- 172.21.15.12:0/1691260437 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fc86c0f3210 con 0x7fc870101130 2024-04-22T13:24:11.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.481+0000 7fc875843640 1 --2- 172.21.15.12:0/1691260437 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc85c0a3e40 0x7fc85c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:11.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.482+0000 7fc875843640 1 --2- 172.21.15.12:0/1691260437 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc85c0a3e40 0x7fc85c0a6300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc8701045e0 tx=0x7fc86000a5c0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:11.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.485+0000 7fc8677fe640 1 -- 172.21.15.12:0/1691260437 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc86c04d020 con 0x7fc870101130 2024-04-22T13:24:11.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.673+0000 7fc8782a8640 1 -- 172.21.15.12:0/1691260437 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8701a74e0 con 0x7fc870101130 2024-04-22T13:24:11.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.674+0000 7fc8677fe640 1 -- 172.21.15.12:0/1691260437 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22997 (secure 0 0 0) 0x7fc86c0bf7c0 con 0x7fc870101130 2024-04-22T13:24:11.672 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:24:11.672 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":90,"fsid":"0aede5ea-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:14:09.362918+0000","modified":"2024-04-22T13:24:06.155595+0000","last_up_change":"2024-04-22T13:21:15.263393+0000","last_in_change":"2024-04-22T13:20:54.174685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:17:32.372434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:22:02.768799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:22:03.578730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d9e2365-3a64-4b13-96fc-c217b1cee2ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6803","nonce":1191350632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6805","nonce":1191350632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6809","nonce":1191350632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6807","nonce":1191350632}]},"public_addr":"172.21.15.12:6803/1191350632","cluster_addr":"172.21.15.12:6805/1191350632","heartbeat_back_addr":"172.21.15.12:6809/1191350632","heartbeat_front_addr":"172.21.15.12:6807/1191350632","state":["exists","up"]},{"osd":1,"uuid":"1499c783-2bd7-48ce-a444-34add42bcf31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6801","nonce":417496465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6803","nonce":417496465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6807","nonce":417496465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6805","nonce":417496465}]},"public_addr":"172.21.15.111:6801/417496465","cluster_addr":"172.21.15.111:6803/417496465","heartbeat_back_addr":"172.21.15.111:6807/417496465","heartbeat_front_addr":"172.21.15.111:6805/417496465","state":["exists","up"]},{"osd":2,"uuid":"71a9ebc3-c622-434e-a6c2-f48358f384f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6801","nonce":1403153998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6803","nonce":1403153998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6807","nonce":1403153998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6805","nonce":1403153998}]},"public_addr":"172.21.15.134:6801/1403153998","cluster_addr":"172.21.15.134:6803/1403153998","heartbeat_back_addr":"172.21.15.134:6807/1403153998","heartbeat_front_addr":"172.21.15.134:6805/1403153998","state":["exists","up"]},{"osd":3,"uuid":"6dd00db2-a286-48ba-8666-56ffdd5f35ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6811","nonce":3359066681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6813","nonce":3359066681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6817","nonce":3359066681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6815","nonce":3359066681}]},"public_addr":"172.21.15.12:6811/3359066681","cluster_addr":"172.21.15.12:6813/3359066681","heartbeat_back_addr":"172.21.15.12:6817/3359066681","heartbeat_front_addr":"172.21.15.12:6815/3359066681","state":["exists","up"]},{"osd":4,"uuid":"1e29afd7-354c-4506-aaa0-b43eb1a19a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6809","nonce":2886349872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6811","nonce":2886349872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6815","nonce":2886349872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6813","nonce":2886349872}]},"public_addr":"172.21.15.111:6809/2886349872","cluster_addr":"172.21.15.111:6811/2886349872","heartbeat_back_addr":"172.21.15.111:6815/2886349872","heartbeat_front_addr":"172.21.15.111:6813/2886349872","state":["exists","up"]},{"osd":5,"uuid":"8f40f02d-0ba2-4842-a03d-2a1a04d1010d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6809","nonce":3071977602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6811","nonce":3071977602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6815","nonce":3071977602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6813","nonce":3071977602}]},"public_addr":"172.21.15.134:6809/3071977602","cluster_addr":"172.21.15.134:6811/3071977602","heartbeat_back_addr":"172.21.15.134:6815/3071977602","heartbeat_front_addr":"172.21.15.134:6813/3071977602","state":["exists","up"]},{"osd":6,"uuid":"bb0c17a9-6a8e-4be6-8660-f73cfa8200ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6819","nonce":3819619299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6821","nonce":3819619299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6825","nonce":3819619299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6823","nonce":3819619299}]},"public_addr":"172.21.15.12:6819/3819619299","cluster_addr":"172.21.15.12:6821/3819619299","heartbeat_back_addr":"172.21.15.12:6825/3819619299","heartbeat_front_addr":"172.21.15.12:6823/3819619299","state":["exists","up"]},{"osd":7,"uuid":"0a93d6af-569e-4e45-b697-9e978c1c1004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6817","nonce":3113856282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6819","nonce":3113856282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6823","nonce":3113856282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6821","nonce":3113856282}]},"public_addr":"172.21.15.111:6817/3113856282","cluster_addr":"172.21.15.111:6819/3113856282","heartbeat_back_addr":"172.21.15.111:6823/3113856282","heartbeat_front_addr":"172.21.15.111:6821/3113856282","state":["exists","up"]},{"osd":8,"uuid":"e21acc68-cf50-4f4f-b1ba-cf6c195b4955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6817","nonce":1689362191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6819","nonce":1689362191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6823","nonce":1689362191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6821","nonce":1689362191}]},"public_addr":"172.21.15.134:6817/1689362191","cluster_addr":"172.21.15.134:6819/1689362191","heartbeat_back_addr":"172.21.15.134:6823/1689362191","heartbeat_front_addr":"172.21.15.134:6821/1689362191","state":["exists","up"]},{"osd":9,"uuid":"fd745869-7f5f-48a8-b412-bb9c92b586e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6827","nonce":4292385779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6829","nonce":4292385779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6833","nonce":4292385779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6831","nonce":4292385779}]},"public_addr":"172.21.15.12:6827/4292385779","cluster_addr":"172.21.15.12:6829/4292385779","heartbeat_back_addr":"172.21.15.12:6833/4292385779","heartbeat_front_addr":"172.21.15.12:6831/4292385779","state":["exists","up"]},{"osd":10,"uuid":"798b1abd-f6c7-4a5f-8da0-fdae49e50d2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6825","nonce":3338566976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6827","nonce":3338566976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6831","nonce":3338566976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6829","nonce":3338566976}]},"public_addr":"172.21.15.111:6825/3338566976","cluster_addr":"172.21.15.111:6827/3338566976","heartbeat_back_addr":"172.21.15.111:6831/3338566976","heartbeat_front_addr":"172.21.15.111:6829/3338566976","state":["exists","up"]},{"osd":11,"uuid":"b18fc5f3-1049-4871-b060-e99290273cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6825","nonce":578640428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6827","nonce":578640428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6831","nonce":578640428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6829","nonce":578640428}]},"public_addr":"172.21.15.134:6825/578640428","cluster_addr":"172.21.15.134:6827/578640428","heartbeat_back_addr":"172.21.15.134:6831/578640428","heartbeat_front_addr":"172.21.15.134:6829/578640428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:16:40.828608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:05.052528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:28.297316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:53.496497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:17.947259+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:41.712894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:07.105945+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:32.129688+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:56.008516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:22.239867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:47.334692+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:12.301276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/3535299028":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/2837479598":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/3010129696":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/2593601362":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/2519382069":"2024-04-23T13:15:04.279990+0000","172.21.15.12:6801/2009831940":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/856202072":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/3509412187":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/1802873236":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/1007749062":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3588480095":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1904962176":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/1802873236":"2024-04-23T13:24:06.155564+0000","172.21.15.12:6800/2009831940":"2024-04-23T13:15:04.279990+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:24:11.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.677+0000 7fc8782a8640 1 -- 172.21.15.12:0/1691260437 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc85c0a3e40 msgr2=0x7fc85c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:11.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.677+0000 7fc8782a8640 1 --2- 172.21.15.12:0/1691260437 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc85c0a3e40 0x7fc85c0a6300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc8701045e0 tx=0x7fc86000a5c0 comp rx=0 tx=0).stop 2024-04-22T13:24:11.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.677+0000 7fc8782a8640 1 -- 172.21.15.12:0/1691260437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc870101130 msgr2=0x7fc870102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:11.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.677+0000 7fc8782a8640 1 --2- 172.21.15.12:0/1691260437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc870101130 0x7fc870102b50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc86c005940 tx=0x7fc86c0045e0 comp rx=0 tx=0).stop 2024-04-22T13:24:11.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.677+0000 7fc8782a8640 1 -- 172.21.15.12:0/1691260437 shutdown_connections 2024-04-22T13:24:11.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.677+0000 7fc8782a8640 1 --2- 172.21.15.12:0/1691260437 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc85c0a3e40 0x7fc85c0a6300 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:11.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.677+0000 7fc8782a8640 1 --2- 172.21.15.12:0/1691260437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc870106f10 0x7fc8701035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:11.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.677+0000 7fc8782a8640 1 --2- 172.21.15.12:0/1691260437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc870101ad0 0x7fc870103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:11.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.677+0000 7fc8782a8640 1 --2- 172.21.15.12:0/1691260437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc870101130 0x7fc870102b50 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:11.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.677+0000 7fc8782a8640 1 -- 172.21.15.12:0/1691260437 >> 172.21.15.12:0/1691260437 conn(0x7fc870077e30 msgr2=0x7fc87006a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:11.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.677+0000 7fc8782a8640 1 -- 172.21.15.12:0/1691260437 shutdown_connections 2024-04-22T13:24:11.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.677+0000 7fc8782a8640 1 -- 172.21.15.12:0/1691260437 wait complete. 2024-04-22T13:24:11.685 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T13:17:32.372434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T13:22:02.768799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T13:22:03.578730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T13:24:11.685 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T13:24:11.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.793+0000 7ff5d9f0e640 1 -- 172.21.15.12:0/2090296789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d406b1f0 msgr2=0x7ff5d4106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:11.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.793+0000 7ff5d9f0e640 1 --2- 172.21.15.12:0/2090296789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d406b1f0 0x7ff5d4106b90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8009900 tx=0x7ff5c802bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:11.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.793+0000 7ff5d9f0e640 1 -- 172.21.15.12:0/2090296789 shutdown_connections 2024-04-22T13:24:11.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.793+0000 7ff5d9f0e640 1 --2- 172.21.15.12:0/2090296789 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5d4107270 0x7ff5d410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:11.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.793+0000 7ff5d9f0e640 1 --2- 172.21.15.12:0/2090296789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d406b1f0 0x7ff5d4106b90 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:11.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.793+0000 7ff5d9f0e640 1 --2- 172.21.15.12:0/2090296789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5d406a840 0x7ff5d406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:11.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.793+0000 7ff5d9f0e640 1 -- 172.21.15.12:0/2090296789 >> 172.21.15.12:0/2090296789 conn(0x7ff5d4077eb0 msgr2=0x7ff5d40782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:11.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.793+0000 7ff5d9f0e640 1 -- 172.21.15.12:0/2090296789 shutdown_connections 2024-04-22T13:24:11.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.793+0000 7ff5d9f0e640 1 -- 172.21.15.12:0/2090296789 wait complete. 2024-04-22T13:24:11.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.794+0000 7ff5d9f0e640 1 Processor -- start 2024-04-22T13:24:11.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.794+0000 7ff5d9f0e640 1 -- start start 2024-04-22T13:24:11.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.794+0000 7ff5d9f0e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d406a840 0x7ff5d41986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:11.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.794+0000 7ff5d9f0e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5d4107270 0x7ff5d4198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:11.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.794+0000 7ff5d9f0e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5d419cf90 0x7ff5d419d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:11.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.794+0000 7ff5d9f0e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d41121b0 con 0x7ff5d406a840 2024-04-22T13:24:11.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.794+0000 7ff5d9f0e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d4112030 con 0x7ff5d419cf90 2024-04-22T13:24:11.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.794+0000 7ff5d9f0e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d4112330 con 0x7ff5d4107270 2024-04-22T13:24:11.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.794+0000 7ff5d37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d406a840 0x7ff5d41986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:11.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.794+0000 7ff5d37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d406a840 0x7ff5d41986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59126/0 (socket says 172.21.15.12:59126) 2024-04-22T13:24:11.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.794+0000 7ff5d37fe640 1 -- 172.21.15.12:0/1006681116 learned_addr learned my addr 172.21.15.12:0/1006681116 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:11.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.794+0000 7ff5d2ffd640 1 --2- 172.21.15.12:0/1006681116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5d4107270 0x7ff5d4198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:11.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.794+0000 7ff5d3fff640 1 --2- 172.21.15.12:0/1006681116 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5d419cf90 0x7ff5d419d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:11.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.795+0000 7ff5d37fe640 1 -- 172.21.15.12:0/1006681116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5d4107270 msgr2=0x7ff5d4198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:11.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.795+0000 7ff5d37fe640 1 --2- 172.21.15.12:0/1006681116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5d4107270 0x7ff5d4198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:11.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.795+0000 7ff5d37fe640 1 -- 172.21.15.12:0/1006681116 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5d419cf90 msgr2=0x7ff5d419d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:11.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.795+0000 7ff5d37fe640 1 --2- 172.21.15.12:0/1006681116 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5d419cf90 0x7ff5d419d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:11.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.795+0000 7ff5d37fe640 1 -- 172.21.15.12:0/1006681116 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d419da80 con 0x7ff5d406a840 2024-04-22T13:24:11.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.795+0000 7ff5d2ffd640 1 --2- 172.21.15.12:0/1006681116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5d4107270 0x7ff5d4198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:11.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.795+0000 7ff5d3fff640 1 --2- 172.21.15.12:0/1006681116 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5d419cf90 0x7ff5d419d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:11.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.795+0000 7ff5d37fe640 1 --2- 172.21.15.12:0/1006681116 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d406a840 0x7ff5d41986d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff5c400c920 tx=0x7ff5c400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:11.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.795+0000 7ff5d0ff9640 1 -- 172.21.15.12:0/1006681116 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4007c70 con 0x7ff5d406a840 2024-04-22T13:24:11.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.795+0000 7ff5d9f0e640 1 -- 172.21.15.12:0/1006681116 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5d419dd70 con 0x7ff5d406a840 2024-04-22T13:24:11.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.795+0000 7ff5d0ff9640 1 -- 172.21.15.12:0/1006681116 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5c4007e10 con 0x7ff5d406a840 2024-04-22T13:24:11.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.795+0000 7ff5d9f0e640 1 -- 172.21.15.12:0/1006681116 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5d41a5580 con 0x7ff5d406a840 2024-04-22T13:24:11.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.795+0000 7ff5d0ff9640 1 -- 172.21.15.12:0/1006681116 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4018820 con 0x7ff5d406a840 2024-04-22T13:24:11.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.796+0000 7ff5d9f0e640 1 -- 172.21.15.12:0/1006681116 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5d40691c0 con 0x7ff5d406a840 2024-04-22T13:24:11.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.797+0000 7ff5d0ff9640 1 -- 172.21.15.12:0/1006681116 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7ff5c400f450 con 0x7ff5d406a840 2024-04-22T13:24:11.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.798+0000 7ff5d0ff9640 1 --2- 172.21.15.12:0/1006681116 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff5bc0a3ec0 0x7ff5bc0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:11.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.798+0000 7ff5d0ff9640 1 -- 172.21.15.12:0/1006681116 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7ff5c40d0eb0 con 0x7ff5d406a840 2024-04-22T13:24:11.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.798+0000 7ff5d2ffd640 1 --2- 172.21.15.12:0/1006681116 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff5bc0a3ec0 0x7ff5bc0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:11.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.798+0000 7ff5d2ffd640 1 --2- 172.21.15.12:0/1006681116 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff5bc0a3ec0 0x7ff5bc0a6380 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8006010 tx=0x7ff5c8005a20 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:11.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.802+0000 7ff5d0ff9640 1 -- 172.21.15.12:0/1006681116 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff5c40ffff0 con 0x7ff5d406a840 2024-04-22T13:24:11.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:11 smithi111 ceph-mon[31361]: Updating smithi111:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:24:11.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:11 smithi111 ceph-mon[31361]: Updating smithi134:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:24:11.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:11 smithi111 ceph-mon[31361]: Updating smithi012:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:24:11.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:11 smithi111 ceph-mon[31361]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:24:11.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:11 smithi111 ceph-mon[31361]: Updating smithi111:/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:24:11.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:11 smithi111 ceph-mon[31361]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:11.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:11 smithi111 ceph-mon[31361]: Updating smithi012:/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:24:11.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:11 smithi111 ceph-mon[31361]: Updating smithi134:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T13:24:11.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:11 smithi111 ceph-mon[31361]: Updating smithi012:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T13:24:11.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:11 smithi111 ceph-mon[31361]: Updating smithi111:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T13:24:11.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:11 smithi111 ceph-mon[31361]: from='client.15495 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:24:11.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:11 smithi111 ceph-mon[31361]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-22T13:24:11.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:11 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:11.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:11 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:11.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:11 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:11.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:11 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:11.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:11 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:11.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:11 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:11.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:11 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:11.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:11 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:24:11.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:11 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:11.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:11 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:24:11.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:11 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:11.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:11 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1691260437' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:24:11.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.987+0000 7ff5d9f0e640 1 -- 172.21.15.12:0/1006681116 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff5d410a4c0 con 0x7ff5d406a840 2024-04-22T13:24:11.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.988+0000 7ff5d0ff9640 1 -- 172.21.15.12:0/1006681116 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7ff5c4014030 con 0x7ff5d406a840 2024-04-22T13:24:11.985 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-22T13:24:11.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.989+0000 7ff5d9f0e640 1 -- 172.21.15.12:0/1006681116 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff5bc0a3ec0 msgr2=0x7ff5bc0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:11.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.989+0000 7ff5d9f0e640 1 --2- 172.21.15.12:0/1006681116 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff5bc0a3ec0 0x7ff5bc0a6380 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8006010 tx=0x7ff5c8005a20 comp rx=0 tx=0).stop 2024-04-22T13:24:11.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.989+0000 7ff5d9f0e640 1 -- 172.21.15.12:0/1006681116 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d406a840 msgr2=0x7ff5d41986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:11.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.989+0000 7ff5d9f0e640 1 --2- 172.21.15.12:0/1006681116 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d406a840 0x7ff5d41986d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff5c400c920 tx=0x7ff5c400cde0 comp rx=0 tx=0).stop 2024-04-22T13:24:11.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.989+0000 7ff5d9f0e640 1 -- 172.21.15.12:0/1006681116 shutdown_connections 2024-04-22T13:24:11.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.989+0000 7ff5d9f0e640 1 --2- 172.21.15.12:0/1006681116 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff5bc0a3ec0 0x7ff5bc0a6380 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:11.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.989+0000 7ff5d9f0e640 1 --2- 172.21.15.12:0/1006681116 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5d419cf90 0x7ff5d419d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:11.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.989+0000 7ff5d9f0e640 1 --2- 172.21.15.12:0/1006681116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5d4107270 0x7ff5d4198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:11.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.989+0000 7ff5d9f0e640 1 --2- 172.21.15.12:0/1006681116 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d406a840 0x7ff5d41986d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:11.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.989+0000 7ff5d9f0e640 1 -- 172.21.15.12:0/1006681116 >> 172.21.15.12:0/1006681116 conn(0x7ff5d4077eb0 msgr2=0x7ff5d40fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:11.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.990+0000 7ff5d9f0e640 1 -- 172.21.15.12:0/1006681116 shutdown_connections 2024-04-22T13:24:11.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:11.990+0000 7ff5d9f0e640 1 -- 172.21.15.12:0/1006681116 wait complete. 2024-04-22T13:24:11.997 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:11 smithi012 ceph-mon[28265]: Updating smithi111:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:24:11.997 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:11 smithi012 ceph-mon[28265]: Updating smithi134:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:24:11.997 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:11 smithi012 ceph-mon[28265]: Updating smithi012:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:24:11.998 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:11 smithi012 ceph-mon[28265]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:24:11.998 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:11 smithi012 ceph-mon[28265]: Updating smithi111:/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:24:11.998 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:11 smithi012 ceph-mon[28265]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:11.998 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:11 smithi012 ceph-mon[28265]: Updating smithi012:/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:24:11.998 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:11 smithi012 ceph-mon[28265]: Updating smithi134:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T13:24:11.998 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:11 smithi012 ceph-mon[28265]: Updating smithi012:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T13:24:11.998 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:11 smithi012 ceph-mon[28265]: Updating smithi111:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T13:24:11.998 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:11 smithi012 ceph-mon[28265]: from='client.15495 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:24:11.998 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:11 smithi012 ceph-mon[28265]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-22T13:24:11.998 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:11 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:11.998 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:11 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:11.998 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:11 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:11.998 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:11 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:11.998 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:11 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:11.999 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:11 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:11.999 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:11 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:11.999 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:11 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:24:11.999 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:11 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:11.999 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:11 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:24:11.999 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:11 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:11.999 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:11 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1691260437' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:24:12.000 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T13:24:12.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.107+0000 7fdc28a43640 1 -- 172.21.15.12:0/4126210858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc24101090 msgr2=0x7fdc24101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:12.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.107+0000 7fdc28a43640 1 --2- 172.21.15.12:0/4126210858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc24101090 0x7fdc24101470 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009900 tx=0x7fdc1402bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:12.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.108+0000 7fdc28a43640 1 -- 172.21.15.12:0/4126210858 shutdown_connections 2024-04-22T13:24:12.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.108+0000 7fdc28a43640 1 --2- 172.21.15.12:0/4126210858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc24106160 0x7fdc2410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:12.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.108+0000 7fdc28a43640 1 --2- 172.21.15.12:0/4126210858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc24101a40 0x7fdc24105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:12.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.108+0000 7fdc28a43640 1 --2- 172.21.15.12:0/4126210858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc24101090 0x7fdc24101470 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:12.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.108+0000 7fdc28a43640 1 -- 172.21.15.12:0/4126210858 >> 172.21.15.12:0/4126210858 conn(0x7fdc24077f50 msgr2=0x7fdc24078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:12.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.108+0000 7fdc28a43640 1 -- 172.21.15.12:0/4126210858 shutdown_connections 2024-04-22T13:24:12.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.108+0000 7fdc28a43640 1 -- 172.21.15.12:0/4126210858 wait complete. 2024-04-22T13:24:12.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.108+0000 7fdc28a43640 1 Processor -- start 2024-04-22T13:24:12.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.108+0000 7fdc28a43640 1 -- start start 2024-04-22T13:24:12.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.109+0000 7fdc28a43640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc24101090 0x7fdc2410e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:12.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.109+0000 7fdc28a43640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc24106160 0x7fdc2410ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:12.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.109+0000 7fdc28a43640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc2410ef40 0x7fdc241a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:12.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.109+0000 7fdc28a43640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdc24112230 con 0x7fdc2410ef40 2024-04-22T13:24:12.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.109+0000 7fdc28a43640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdc241120b0 con 0x7fdc24101090 2024-04-22T13:24:12.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.109+0000 7fdc28a43640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdc241123b0 con 0x7fdc24106160 2024-04-22T13:24:12.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.109+0000 7fdc22d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc2410ef40 0x7fdc241a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:12.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.109+0000 7fdc22d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc2410ef40 0x7fdc241a69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59146/0 (socket says 172.21.15.12:59146) 2024-04-22T13:24:12.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.109+0000 7fdc22d9d640 1 -- 172.21.15.12:0/3300507423 learned_addr learned my addr 172.21.15.12:0/3300507423 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:12.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.109+0000 7fdc21d9b640 1 --2- 172.21.15.12:0/3300507423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc24106160 0x7fdc2410ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:12.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.109+0000 7fdc2259c640 1 --2- 172.21.15.12:0/3300507423 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc24101090 0x7fdc2410e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:12.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.109+0000 7fdc22d9d640 1 -- 172.21.15.12:0/3300507423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc24106160 msgr2=0x7fdc2410ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:12.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.109+0000 7fdc22d9d640 1 --2- 172.21.15.12:0/3300507423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc24106160 0x7fdc2410ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:12.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.109+0000 7fdc22d9d640 1 -- 172.21.15.12:0/3300507423 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc24101090 msgr2=0x7fdc2410e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:12.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.109+0000 7fdc22d9d640 1 --2- 172.21.15.12:0/3300507423 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc24101090 0x7fdc2410e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:12.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.109+0000 7fdc22d9d640 1 -- 172.21.15.12:0/3300507423 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc241a70c0 con 0x7fdc2410ef40 2024-04-22T13:24:12.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.109+0000 7fdc21d9b640 1 --2- 172.21.15.12:0/3300507423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc24106160 0x7fdc2410ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:12.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.109+0000 7fdc22d9d640 1 --2- 172.21.15.12:0/3300507423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc2410ef40 0x7fdc241a69c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00cad0 tx=0x7fdc1c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:12.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.109+0000 7fdc137fe640 1 -- 172.21.15.12:0/3300507423 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c00ae20 con 0x7fdc2410ef40 2024-04-22T13:24:12.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.109+0000 7fdc28a43640 1 -- 172.21.15.12:0/3300507423 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc241a73b0 con 0x7fdc2410ef40 2024-04-22T13:24:12.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.109+0000 7fdc137fe640 1 -- 172.21.15.12:0/3300507423 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc1c005020 con 0x7fdc2410ef40 2024-04-22T13:24:12.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.109+0000 7fdc137fe640 1 -- 172.21.15.12:0/3300507423 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c00f6b0 con 0x7fdc2410ef40 2024-04-22T13:24:12.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.109+0000 7fdc28a43640 1 -- 172.21.15.12:0/3300507423 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc241a78b0 con 0x7fdc2410ef40 2024-04-22T13:24:12.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.110+0000 7fdc28a43640 1 -- 172.21.15.12:0/3300507423 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc24068d70 con 0x7fdc2410ef40 2024-04-22T13:24:12.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.111+0000 7fdc137fe640 1 -- 172.21.15.12:0/3300507423 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fdc1c0051d0 con 0x7fdc2410ef40 2024-04-22T13:24:12.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.112+0000 7fdc137fe640 1 --2- 172.21.15.12:0/3300507423 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fdc0c0a3e70 0x7fdc0c0a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:12.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.112+0000 7fdc2259c640 1 --2- 172.21.15.12:0/3300507423 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fdc0c0a3e70 0x7fdc0c0a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:12.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.112+0000 7fdc137fe640 1 -- 172.21.15.12:0/3300507423 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fdc1c0d0c00 con 0x7fdc2410ef40 2024-04-22T13:24:12.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.113+0000 7fdc2259c640 1 --2- 172.21.15.12:0/3300507423 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fdc0c0a3e70 0x7fdc0c0a6330 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdc14036040 tx=0x7fdc14000bc0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:12.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.117+0000 7fdc137fe640 1 -- 172.21.15.12:0/3300507423 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdc1c014030 con 0x7fdc2410ef40 2024-04-22T13:24:12.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:11 smithi134 ceph-mon[31338]: Updating smithi111:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:24:12.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:11 smithi134 ceph-mon[31338]: Updating smithi134:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:24:12.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:11 smithi134 ceph-mon[31338]: Updating smithi012:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-22T13:24:12.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:11 smithi134 ceph-mon[31338]: Updating smithi134:/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:24:12.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:11 smithi134 ceph-mon[31338]: Updating smithi111:/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:24:12.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:11 smithi134 ceph-mon[31338]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-22T13:24:12.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:11 smithi134 ceph-mon[31338]: Updating smithi012:/etc/ceph/ceph.client.admin.keyring 2024-04-22T13:24:12.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:11 smithi134 ceph-mon[31338]: Updating smithi134:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T13:24:12.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:11 smithi134 ceph-mon[31338]: Updating smithi012:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T13:24:12.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:11 smithi134 ceph-mon[31338]: Updating smithi111:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-22T13:24:12.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:11 smithi134 ceph-mon[31338]: from='client.15495 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:24:12.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:11 smithi134 ceph-mon[31338]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-22T13:24:12.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:11 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:12.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:11 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:12.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:11 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:12.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:11 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:12.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:11 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:12.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:11 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:12.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:11 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:12.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:11 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:24:12.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:11 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:24:12.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:11 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:24:12.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:11 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:24:12.179 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:11 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1691260437' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:24:12.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.305+0000 7fdc28a43640 1 -- 172.21.15.12:0/3300507423 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdc24101a40 con 0x7fdc2410ef40 2024-04-22T13:24:12.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.305+0000 7fdc137fe640 1 -- 172.21.15.12:0/3300507423 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7fdc1c09d040 con 0x7fdc2410ef40 2024-04-22T13:24:12.303 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-22T13:24:12.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.307+0000 7fdc28a43640 1 -- 172.21.15.12:0/3300507423 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fdc0c0a3e70 msgr2=0x7fdc0c0a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:12.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.307+0000 7fdc28a43640 1 --2- 172.21.15.12:0/3300507423 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fdc0c0a3e70 0x7fdc0c0a6330 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdc14036040 tx=0x7fdc14000bc0 comp rx=0 tx=0).stop 2024-04-22T13:24:12.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.307+0000 7fdc28a43640 1 -- 172.21.15.12:0/3300507423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc2410ef40 msgr2=0x7fdc241a69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:12.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.307+0000 7fdc28a43640 1 --2- 172.21.15.12:0/3300507423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc2410ef40 0x7fdc241a69c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00cad0 tx=0x7fdc1c00cf90 comp rx=0 tx=0).stop 2024-04-22T13:24:12.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.307+0000 7fdc28a43640 1 -- 172.21.15.12:0/3300507423 shutdown_connections 2024-04-22T13:24:12.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.307+0000 7fdc28a43640 1 --2- 172.21.15.12:0/3300507423 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fdc0c0a3e70 0x7fdc0c0a6330 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:12.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.307+0000 7fdc28a43640 1 --2- 172.21.15.12:0/3300507423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdc2410ef40 0x7fdc241a69c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:12.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.307+0000 7fdc28a43640 1 --2- 172.21.15.12:0/3300507423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc24106160 0x7fdc2410ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:12.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.307+0000 7fdc28a43640 1 --2- 172.21.15.12:0/3300507423 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc24101090 0x7fdc2410e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:12.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.307+0000 7fdc28a43640 1 -- 172.21.15.12:0/3300507423 >> 172.21.15.12:0/3300507423 conn(0x7fdc24077f50 msgr2=0x7fdc240ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:12.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.307+0000 7fdc28a43640 1 -- 172.21.15.12:0/3300507423 shutdown_connections 2024-04-22T13:24:12.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.307+0000 7fdc28a43640 1 -- 172.21.15.12:0/3300507423 wait complete. 2024-04-22T13:24:12.315 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T13:24:12.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.424+0000 7fdb0ffb6640 1 -- 172.21.15.12:0/3718566271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb08107260 msgr2=0x7fdb0810ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:12.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.424+0000 7fdb0ffb6640 1 --2- 172.21.15.12:0/3718566271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb08107260 0x7fdb0810ee30 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fdb00009ca0 tx=0x7fdb0002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:24:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.424+0000 7fdb0ffb6640 1 -- 172.21.15.12:0/3718566271 shutdown_connections 2024-04-22T13:24:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.424+0000 7fdb0ffb6640 1 --2- 172.21.15.12:0/3718566271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb08107260 0x7fdb0810ee30 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.424+0000 7fdb0ffb6640 1 --2- 172.21.15.12:0/3718566271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb08102b40 0x7fdb08106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.424+0000 7fdb0ffb6640 1 --2- 172.21.15.12:0/3718566271 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb08102190 0x7fdb08102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.424+0000 7fdb0ffb6640 1 -- 172.21.15.12:0/3718566271 >> 172.21.15.12:0/3718566271 conn(0x7fdb080fd990 msgr2=0x7fdb080ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.424+0000 7fdb0ffb6640 1 -- 172.21.15.12:0/3718566271 shutdown_connections 2024-04-22T13:24:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.424+0000 7fdb0ffb6640 1 -- 172.21.15.12:0/3718566271 wait complete. 2024-04-22T13:24:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.425+0000 7fdb0ffb6640 1 Processor -- start 2024-04-22T13:24:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.425+0000 7fdb0ffb6640 1 -- start start 2024-04-22T13:24:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.425+0000 7fdb0ffb6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb08102190 0x7fdb08198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.425+0000 7fdb0ffb6640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb08107260 0x7fdb08198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.425+0000 7fdb0ffb6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb0819cf50 0x7fdb0819d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.425+0000 7fdb0ffb6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdb081121b0 con 0x7fdb0819cf50 2024-04-22T13:24:12.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.425+0000 7fdb0ffb6640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdb08112030 con 0x7fdb08107260 2024-04-22T13:24:12.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.425+0000 7fdb0ffb6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdb08112330 con 0x7fdb08102190 2024-04-22T13:24:12.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.425+0000 7fdb0e553640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb0819cf50 0x7fdb0819d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:12.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.425+0000 7fdb0e553640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb0819cf50 0x7fdb0819d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59158/0 (socket says 172.21.15.12:59158) 2024-04-22T13:24:12.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.425+0000 7fdb0e553640 1 -- 172.21.15.12:0/1699363090 learned_addr learned my addr 172.21.15.12:0/1699363090 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:12.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.426+0000 7fdb0d551640 1 --2- 172.21.15.12:0/1699363090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb08107260 0x7fdb08198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:12.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.426+0000 7fdb0e553640 1 -- 172.21.15.12:0/1699363090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb08102190 msgr2=0x7fdb08198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:12.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.426+0000 7fdb0e553640 1 --2- 172.21.15.12:0/1699363090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb08102190 0x7fdb08198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:12.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.426+0000 7fdb0e553640 1 -- 172.21.15.12:0/1699363090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb08107260 msgr2=0x7fdb08198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:12.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.426+0000 7fdb0e553640 1 --2- 172.21.15.12:0/1699363090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb08107260 0x7fdb08198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:12.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.426+0000 7fdb0e553640 1 -- 172.21.15.12:0/1699363090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb0819da40 con 0x7fdb0819cf50 2024-04-22T13:24:12.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.426+0000 7fdb0d551640 1 --2- 172.21.15.12:0/1699363090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb08107260 0x7fdb08198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:12.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.426+0000 7fdb0e553640 1 --2- 172.21.15.12:0/1699363090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb0819cf50 0x7fdb0819d400 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdb00005e30 tx=0x7fdb00034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:12.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.426+0000 7fdafeffd640 1 -- 172.21.15.12:0/1699363090 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb00039040 con 0x7fdb0819cf50 2024-04-22T13:24:12.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.426+0000 7fdafeffd640 1 -- 172.21.15.12:0/1699363090 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb00044070 con 0x7fdb0819cf50 2024-04-22T13:24:12.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.426+0000 7fdb0ffb6640 1 -- 172.21.15.12:0/1699363090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb0819dcd0 con 0x7fdb0819cf50 2024-04-22T13:24:12.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.426+0000 7fdafeffd640 1 -- 172.21.15.12:0/1699363090 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb00003e60 con 0x7fdb0819cf50 2024-04-22T13:24:12.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.426+0000 7fdb0ffb6640 1 -- 172.21.15.12:0/1699363090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb081a5490 con 0x7fdb0819cf50 2024-04-22T13:24:12.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.427+0000 7fdb0ffb6640 1 -- 172.21.15.12:0/1699363090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb080691c0 con 0x7fdb0819cf50 2024-04-22T13:24:12.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.428+0000 7fdafeffd640 1 -- 172.21.15.12:0/1699363090 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fdb000324c0 con 0x7fdb0819cf50 2024-04-22T13:24:12.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.429+0000 7fdafeffd640 1 --2- 172.21.15.12:0/1699363090 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fdaf40a3e40 0x7fdaf40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:12.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.429+0000 7fdb0dd52640 1 --2- 172.21.15.12:0/1699363090 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fdaf40a3e40 0x7fdaf40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:12.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.429+0000 7fdafeffd640 1 -- 172.21.15.12:0/1699363090 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fdb000f3730 con 0x7fdb0819cf50 2024-04-22T13:24:12.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.429+0000 7fdb0dd52640 1 --2- 172.21.15.12:0/1699363090 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fdaf40a3e40 0x7fdaf40a6300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdb04005fd0 tx=0x7fdb04005850 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:12.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.432+0000 7fdafeffd640 1 -- 172.21.15.12:0/1699363090 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdb000bfce0 con 0x7fdb0819cf50 2024-04-22T13:24:12.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.617+0000 7fdb0ffb6640 1 -- 172.21.15.12:0/1699363090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdb08102b40 con 0x7fdb0819cf50 2024-04-22T13:24:12.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.618+0000 7fdafeffd640 1 -- 172.21.15.12:0/1699363090 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7fdb00051e10 con 0x7fdb0819cf50 2024-04-22T13:24:12.615 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-22T13:24:12.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.620+0000 7fdb0ffb6640 1 -- 172.21.15.12:0/1699363090 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fdaf40a3e40 msgr2=0x7fdaf40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:12.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.620+0000 7fdb0ffb6640 1 --2- 172.21.15.12:0/1699363090 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fdaf40a3e40 0x7fdaf40a6300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdb04005fd0 tx=0x7fdb04005850 comp rx=0 tx=0).stop 2024-04-22T13:24:12.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.620+0000 7fdb0ffb6640 1 -- 172.21.15.12:0/1699363090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb0819cf50 msgr2=0x7fdb0819d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:12.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.620+0000 7fdb0ffb6640 1 --2- 172.21.15.12:0/1699363090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb0819cf50 0x7fdb0819d400 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdb00005e30 tx=0x7fdb00034760 comp rx=0 tx=0).stop 2024-04-22T13:24:12.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.620+0000 7fdb0ffb6640 1 -- 172.21.15.12:0/1699363090 shutdown_connections 2024-04-22T13:24:12.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.620+0000 7fdb0ffb6640 1 --2- 172.21.15.12:0/1699363090 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fdaf40a3e40 0x7fdaf40a6300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:12.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.620+0000 7fdb0ffb6640 1 --2- 172.21.15.12:0/1699363090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdb0819cf50 0x7fdb0819d400 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:12.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.620+0000 7fdb0ffb6640 1 --2- 172.21.15.12:0/1699363090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb08107260 0x7fdb08198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:12.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.620+0000 7fdb0ffb6640 1 --2- 172.21.15.12:0/1699363090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb08102190 0x7fdb08198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:12.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.620+0000 7fdb0ffb6640 1 -- 172.21.15.12:0/1699363090 >> 172.21.15.12:0/1699363090 conn(0x7fdb080fd990 msgr2=0x7fdb080ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:12.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.620+0000 7fdb0ffb6640 1 -- 172.21.15.12:0/1699363090 shutdown_connections 2024-04-22T13:24:12.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.620+0000 7fdb0ffb6640 1 -- 172.21.15.12:0/1699363090 wait complete. 2024-04-22T13:24:12.628 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:24:12.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.733+0000 7fa4c4432640 1 -- 172.21.15.12:0/3860280509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4bc101a60 msgr2=0x7fa4bc105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:12.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.733+0000 7fa4c4432640 1 --2- 172.21.15.12:0/3860280509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4bc101a60 0x7fa4bc105aa0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac009900 tx=0x7fa4ac02bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:12.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.734+0000 7fa4c4432640 1 -- 172.21.15.12:0/3860280509 shutdown_connections 2024-04-22T13:24:12.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.734+0000 7fa4c4432640 1 --2- 172.21.15.12:0/3860280509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4bc106180 0x7fa4bc10dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:12.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.734+0000 7fa4c4432640 1 --2- 172.21.15.12:0/3860280509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4bc101a60 0x7fa4bc105aa0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:12.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.734+0000 7fa4c4432640 1 --2- 172.21.15.12:0/3860280509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4bc1010b0 0x7fa4bc101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:12.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.734+0000 7fa4c4432640 1 -- 172.21.15.12:0/3860280509 >> 172.21.15.12:0/3860280509 conn(0x7fa4bc0771d0 msgr2=0x7fa4bc0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:12.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.734+0000 7fa4c4432640 1 -- 172.21.15.12:0/3860280509 shutdown_connections 2024-04-22T13:24:12.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.734+0000 7fa4c4432640 1 -- 172.21.15.12:0/3860280509 wait complete. 2024-04-22T13:24:12.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.734+0000 7fa4c4432640 1 Processor -- start 2024-04-22T13:24:12.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.734+0000 7fa4c4432640 1 -- start start 2024-04-22T13:24:12.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.734+0000 7fa4c4432640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4bc1010b0 0x7fa4bc10e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:12.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.734+0000 7fa4c4432640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4bc106180 0x7fa4bc10ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:12.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.734+0000 7fa4c4432640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4bc10ef60 0x7fa4bc1a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:12.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.734+0000 7fa4c4432640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa4bc1121f0 con 0x7fa4bc1010b0 2024-04-22T13:24:12.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.734+0000 7fa4c4432640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa4bc112070 con 0x7fa4bc10ef60 2024-04-22T13:24:12.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.734+0000 7fa4c4432640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa4bc112370 con 0x7fa4bc106180 2024-04-22T13:24:12.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.735+0000 7fa4c21ce640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4bc1010b0 0x7fa4bc10e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:12.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.735+0000 7fa4c21ce640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4bc1010b0 0x7fa4bc10e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59168/0 (socket says 172.21.15.12:59168) 2024-04-22T13:24:12.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.735+0000 7fa4c21ce640 1 -- 172.21.15.12:0/1597149249 learned_addr learned my addr 172.21.15.12:0/1597149249 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:12.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.735+0000 7fa4c21ce640 1 -- 172.21.15.12:0/1597149249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4bc106180 msgr2=0x7fa4bc10ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:12.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.735+0000 7fa4c21ce640 1 --2- 172.21.15.12:0/1597149249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4bc106180 0x7fa4bc10ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:12.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.735+0000 7fa4c21ce640 1 -- 172.21.15.12:0/1597149249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4bc10ef60 msgr2=0x7fa4bc1a68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:12.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.735+0000 7fa4c21ce640 1 --2- 172.21.15.12:0/1597149249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4bc10ef60 0x7fa4bc1a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:12.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.735+0000 7fa4c21ce640 1 -- 172.21.15.12:0/1597149249 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4bc1a6fe0 con 0x7fa4bc1010b0 2024-04-22T13:24:12.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.735+0000 7fa4c21ce640 1 --2- 172.21.15.12:0/1597149249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4bc1010b0 0x7fa4bc10e4e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa4b800c970 tx=0x7fa4b800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:12.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.735+0000 7fa4b37fe640 1 -- 172.21.15.12:0/1597149249 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8007bb0 con 0x7fa4bc1010b0 2024-04-22T13:24:12.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.735+0000 7fa4b37fe640 1 -- 172.21.15.12:0/1597149249 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4b8007d50 con 0x7fa4bc1010b0 2024-04-22T13:24:12.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.735+0000 7fa4b37fe640 1 -- 172.21.15.12:0/1597149249 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b800f450 con 0x7fa4bc1010b0 2024-04-22T13:24:12.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.735+0000 7fa4c4432640 1 -- 172.21.15.12:0/1597149249 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4bc1a72d0 con 0x7fa4bc1010b0 2024-04-22T13:24:12.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.735+0000 7fa4c4432640 1 -- 172.21.15.12:0/1597149249 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4bc1a7820 con 0x7fa4bc1010b0 2024-04-22T13:24:12.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.736+0000 7fa4c4432640 1 -- 172.21.15.12:0/1597149249 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4bc069230 con 0x7fa4bc1010b0 2024-04-22T13:24:12.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.738+0000 7fa4b37fe640 1 -- 172.21.15.12:0/1597149249 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fa4b8004ee0 con 0x7fa4bc1010b0 2024-04-22T13:24:12.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.739+0000 7fa4b37fe640 1 --2- 172.21.15.12:0/1597149249 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa4a80a3ef0 0x7fa4a80a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:12.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.739+0000 7fa4c19cd640 1 --2- 172.21.15.12:0/1597149249 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa4a80a3ef0 0x7fa4a80a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:12.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.739+0000 7fa4b37fe640 1 -- 172.21.15.12:0/1597149249 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fa4b80d0c10 con 0x7fa4bc1010b0 2024-04-22T13:24:12.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.739+0000 7fa4c19cd640 1 --2- 172.21.15.12:0/1597149249 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa4a80a3ef0 0x7fa4a80a63b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0096c0 tx=0x7fa4ac000bc0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:12.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.743+0000 7fa4b37fe640 1 -- 172.21.15.12:0/1597149249 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa4b8014030 con 0x7fa4bc1010b0 2024-04-22T13:24:12.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.968+0000 7fa4c4432640 1 -- 172.21.15.12:0/1597149249 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4bc10fa10 con 0x7fa4bc1010b0 2024-04-22T13:24:12.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.968+0000 7fa4b37fe640 1 -- 172.21.15.12:0/1597149249 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7fa4b809d0d0 con 0x7fa4bc1010b0 2024-04-22T13:24:12.966 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:24:12.966 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:24:12.966 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:24:12.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.970+0000 7fa4c4432640 1 -- 172.21.15.12:0/1597149249 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa4a80a3ef0 msgr2=0x7fa4a80a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:12.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.970+0000 7fa4c4432640 1 --2- 172.21.15.12:0/1597149249 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa4a80a3ef0 0x7fa4a80a63b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0096c0 tx=0x7fa4ac000bc0 comp rx=0 tx=0).stop 2024-04-22T13:24:12.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.970+0000 7fa4c4432640 1 -- 172.21.15.12:0/1597149249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4bc1010b0 msgr2=0x7fa4bc10e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:12.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.970+0000 7fa4c4432640 1 --2- 172.21.15.12:0/1597149249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4bc1010b0 0x7fa4bc10e4e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa4b800c970 tx=0x7fa4b800ce30 comp rx=0 tx=0).stop 2024-04-22T13:24:12.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.970+0000 7fa4c4432640 1 -- 172.21.15.12:0/1597149249 shutdown_connections 2024-04-22T13:24:12.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.970+0000 7fa4c4432640 1 --2- 172.21.15.12:0/1597149249 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa4a80a3ef0 0x7fa4a80a63b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:12.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.970+0000 7fa4c4432640 1 --2- 172.21.15.12:0/1597149249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4bc10ef60 0x7fa4bc1a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:12.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.970+0000 7fa4c4432640 1 --2- 172.21.15.12:0/1597149249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4bc106180 0x7fa4bc10ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:12.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.970+0000 7fa4c4432640 1 --2- 172.21.15.12:0/1597149249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa4bc1010b0 0x7fa4bc10e4e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:12.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.970+0000 7fa4c4432640 1 -- 172.21.15.12:0/1597149249 >> 172.21.15.12:0/1597149249 conn(0x7fa4bc0771d0 msgr2=0x7fa4bc0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:12.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.970+0000 7fa4c4432640 1 -- 172.21.15.12:0/1597149249 shutdown_connections 2024-04-22T13:24:12.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:12.970+0000 7fa4c4432640 1 -- 172.21.15.12:0/1597149249 wait complete. 2024-04-22T13:24:12.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:12 smithi111 ceph-mon[31361]: from='client.15501 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:24:12.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:12 smithi111 ceph-mon[31361]: from='client.15507 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:24:12.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:12 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1006681116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:24:12.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:12 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/3300507423' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:24:12.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:12 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1699363090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:24:12.978 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:12 smithi012 ceph-mon[28265]: from='client.15501 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:24:12.978 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:12 smithi012 ceph-mon[28265]: from='client.15507 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:24:12.978 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:12 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1006681116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:24:12.978 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:12 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3300507423' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:24:12.978 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:12 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1699363090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:24:12.979 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:24:13.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.084+0000 7f350116d640 1 -- 172.21.15.12:0/1102733351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34fc102b50 msgr2=0x7f34fc106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:13.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.084+0000 7f350116d640 1 --2- 172.21.15.12:0/1102733351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34fc102b50 0x7f34fc106b90 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f34f0009900 tx=0x7f34f002bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:13.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.084+0000 7f350116d640 1 -- 172.21.15.12:0/1102733351 shutdown_connections 2024-04-22T13:24:13.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.084+0000 7f350116d640 1 --2- 172.21.15.12:0/1102733351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34fc107270 0x7f34fc10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.084+0000 7f350116d640 1 --2- 172.21.15.12:0/1102733351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34fc102b50 0x7f34fc106b90 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.084+0000 7f350116d640 1 --2- 172.21.15.12:0/1102733351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34fc1021a0 0x7f34fc102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.084+0000 7f350116d640 1 -- 172.21.15.12:0/1102733351 >> 172.21.15.12:0/1102733351 conn(0x7f34fc0793a0 msgr2=0x7f34fc0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:13.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.084+0000 7f350116d640 1 -- 172.21.15.12:0/1102733351 shutdown_connections 2024-04-22T13:24:13.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.084+0000 7f350116d640 1 -- 172.21.15.12:0/1102733351 wait complete. 2024-04-22T13:24:13.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.084+0000 7f350116d640 1 Processor -- start 2024-04-22T13:24:13.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.085+0000 7f350116d640 1 -- start start 2024-04-22T13:24:13.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.085+0000 7f350116d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34fc1021a0 0x7f34fc198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.085+0000 7f350116d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34fc107270 0x7f34fc198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.085+0000 7f350116d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34fc19cf20 0x7f34fc19d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.085+0000 7f350116d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc1121e0 con 0x7f34fc1021a0 2024-04-22T13:24:13.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.085+0000 7f350116d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc112060 con 0x7f34fc19cf20 2024-04-22T13:24:13.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.085+0000 7f350116d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc112360 con 0x7f34fc107270 2024-04-22T13:24:13.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.085+0000 7f34fad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34fc1021a0 0x7f34fc198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.085+0000 7f34fad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34fc1021a0 0x7f34fc198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59186/0 (socket says 172.21.15.12:59186) 2024-04-22T13:24:13.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.085+0000 7f34fad9d640 1 -- 172.21.15.12:0/1381871018 learned_addr learned my addr 172.21.15.12:0/1381871018 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:13.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.085+0000 7f34fb59e640 1 --2- 172.21.15.12:0/1381871018 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34fc19cf20 0x7f34fc19d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.085+0000 7f34fa59c640 1 --2- 172.21.15.12:0/1381871018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34fc107270 0x7f34fc198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.085+0000 7f34fad9d640 1 -- 172.21.15.12:0/1381871018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34fc107270 msgr2=0x7f34fc198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:13.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.085+0000 7f34fad9d640 1 --2- 172.21.15.12:0/1381871018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34fc107270 0x7f34fc198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.085+0000 7f34fad9d640 1 -- 172.21.15.12:0/1381871018 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34fc19cf20 msgr2=0x7f34fc19d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:13.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.085+0000 7f34fad9d640 1 --2- 172.21.15.12:0/1381871018 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34fc19cf20 0x7f34fc19d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.085+0000 7f34fad9d640 1 -- 172.21.15.12:0/1381871018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34fc19da10 con 0x7f34fc1021a0 2024-04-22T13:24:13.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.085+0000 7f34fa59c640 1 --2- 172.21.15.12:0/1381871018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34fc107270 0x7f34fc198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:13.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.086+0000 7f34fad9d640 1 --2- 172.21.15.12:0/1381871018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34fc1021a0 0x7f34fc198660 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f34ec00c920 tx=0x7f34ec00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:13.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.086+0000 7f34e3fff640 1 -- 172.21.15.12:0/1381871018 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec007c70 con 0x7f34fc1021a0 2024-04-22T13:24:13.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.086+0000 7f34e3fff640 1 -- 172.21.15.12:0/1381871018 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34ec007e10 con 0x7f34fc1021a0 2024-04-22T13:24:13.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.086+0000 7f350116d640 1 -- 172.21.15.12:0/1381871018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34fc19dd00 con 0x7f34fc1021a0 2024-04-22T13:24:13.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.086+0000 7f34e3fff640 1 -- 172.21.15.12:0/1381871018 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec018820 con 0x7f34fc1021a0 2024-04-22T13:24:13.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.086+0000 7f350116d640 1 -- 172.21.15.12:0/1381871018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34fc1a54c0 con 0x7f34fc1021a0 2024-04-22T13:24:13.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.087+0000 7f350116d640 1 -- 172.21.15.12:0/1381871018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34fc069230 con 0x7f34fc1021a0 2024-04-22T13:24:13.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.088+0000 7f34e3fff640 1 -- 172.21.15.12:0/1381871018 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f34ec00f450 con 0x7f34fc1021a0 2024-04-22T13:24:13.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.088+0000 7f34e3fff640 1 --2- 172.21.15.12:0/1381871018 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f34e40a3ec0 0x7f34e40a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.089+0000 7f34fa59c640 1 --2- 172.21.15.12:0/1381871018 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f34e40a3ec0 0x7f34e40a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.089+0000 7f34e3fff640 1 -- 172.21.15.12:0/1381871018 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f34ec0d0be0 con 0x7f34fc1021a0 2024-04-22T13:24:13.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.089+0000 7f34fa59c640 1 --2- 172.21.15.12:0/1381871018 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f34e40a3ec0 0x7f34e40a6380 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f34f002c240 tx=0x7f34f0005fd0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:13.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.093+0000 7f34e3fff640 1 -- 172.21.15.12:0/1381871018 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f34ec014030 con 0x7f34fc1021a0 2024-04-22T13:24:13.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:12 smithi134 ceph-mon[31338]: from='client.15501 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:24:13.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:12 smithi134 ceph-mon[31338]: from='client.15507 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:24:13.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:12 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1006681116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:24:13.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:12 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/3300507423' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:24:13.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:12 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1699363090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:24:13.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.284+0000 7f350116d640 1 -- 172.21.15.12:0/1381871018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34fc1999b0 con 0x7f34fc1021a0 2024-04-22T13:24:13.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.286+0000 7f34e3fff640 1 -- 172.21.15.12:0/1381871018 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22997 (secure 0 0 0) 0x7f34ec09d020 con 0x7f34fc1021a0 2024-04-22T13:24:13.283 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:24:13.283 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":90,"fsid":"0aede5ea-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:14:09.362918+0000","modified":"2024-04-22T13:24:06.155595+0000","last_up_change":"2024-04-22T13:21:15.263393+0000","last_in_change":"2024-04-22T13:20:54.174685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:17:32.372434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:22:02.768799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:22:03.578730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d9e2365-3a64-4b13-96fc-c217b1cee2ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6803","nonce":1191350632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6805","nonce":1191350632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6809","nonce":1191350632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6807","nonce":1191350632}]},"public_addr":"172.21.15.12:6803/1191350632","cluster_addr":"172.21.15.12:6805/1191350632","heartbeat_back_addr":"172.21.15.12:6809/1191350632","heartbeat_front_addr":"172.21.15.12:6807/1191350632","state":["exists","up"]},{"osd":1,"uuid":"1499c783-2bd7-48ce-a444-34add42bcf31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6801","nonce":417496465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6803","nonce":417496465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6807","nonce":417496465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6805","nonce":417496465}]},"public_addr":"172.21.15.111:6801/417496465","cluster_addr":"172.21.15.111:6803/417496465","heartbeat_back_addr":"172.21.15.111:6807/417496465","heartbeat_front_addr":"172.21.15.111:6805/417496465","state":["exists","up"]},{"osd":2,"uuid":"71a9ebc3-c622-434e-a6c2-f48358f384f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6801","nonce":1403153998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6803","nonce":1403153998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6807","nonce":1403153998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6805","nonce":1403153998}]},"public_addr":"172.21.15.134:6801/1403153998","cluster_addr":"172.21.15.134:6803/1403153998","heartbeat_back_addr":"172.21.15.134:6807/1403153998","heartbeat_front_addr":"172.21.15.134:6805/1403153998","state":["exists","up"]},{"osd":3,"uuid":"6dd00db2-a286-48ba-8666-56ffdd5f35ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6811","nonce":3359066681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6813","nonce":3359066681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6817","nonce":3359066681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6815","nonce":3359066681}]},"public_addr":"172.21.15.12:6811/3359066681","cluster_addr":"172.21.15.12:6813/3359066681","heartbeat_back_addr":"172.21.15.12:6817/3359066681","heartbeat_front_addr":"172.21.15.12:6815/3359066681","state":["exists","up"]},{"osd":4,"uuid":"1e29afd7-354c-4506-aaa0-b43eb1a19a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6809","nonce":2886349872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6811","nonce":2886349872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6815","nonce":2886349872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6813","nonce":2886349872}]},"public_addr":"172.21.15.111:6809/2886349872","cluster_addr":"172.21.15.111:6811/2886349872","heartbeat_back_addr":"172.21.15.111:6815/2886349872","heartbeat_front_addr":"172.21.15.111:6813/2886349872","state":["exists","up"]},{"osd":5,"uuid":"8f40f02d-0ba2-4842-a03d-2a1a04d1010d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6809","nonce":3071977602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6811","nonce":3071977602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6815","nonce":3071977602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6813","nonce":3071977602}]},"public_addr":"172.21.15.134:6809/3071977602","cluster_addr":"172.21.15.134:6811/3071977602","heartbeat_back_addr":"172.21.15.134:6815/3071977602","heartbeat_front_addr":"172.21.15.134:6813/3071977602","state":["exists","up"]},{"osd":6,"uuid":"bb0c17a9-6a8e-4be6-8660-f73cfa8200ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6819","nonce":3819619299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6821","nonce":3819619299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6825","nonce":3819619299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6823","nonce":3819619299}]},"public_addr":"172.21.15.12:6819/3819619299","cluster_addr":"172.21.15.12:6821/3819619299","heartbeat_back_addr":"172.21.15.12:6825/3819619299","heartbeat_front_addr":"172.21.15.12:6823/3819619299","state":["exists","up"]},{"osd":7,"uuid":"0a93d6af-569e-4e45-b697-9e978c1c1004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6817","nonce":3113856282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6819","nonce":3113856282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6823","nonce":3113856282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6821","nonce":3113856282}]},"public_addr":"172.21.15.111:6817/3113856282","cluster_addr":"172.21.15.111:6819/3113856282","heartbeat_back_addr":"172.21.15.111:6823/3113856282","heartbeat_front_addr":"172.21.15.111:6821/3113856282","state":["exists","up"]},{"osd":8,"uuid":"e21acc68-cf50-4f4f-b1ba-cf6c195b4955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6817","nonce":1689362191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6819","nonce":1689362191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6823","nonce":1689362191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6821","nonce":1689362191}]},"public_addr":"172.21.15.134:6817/1689362191","cluster_addr":"172.21.15.134:6819/1689362191","heartbeat_back_addr":"172.21.15.134:6823/1689362191","heartbeat_front_addr":"172.21.15.134:6821/1689362191","state":["exists","up"]},{"osd":9,"uuid":"fd745869-7f5f-48a8-b412-bb9c92b586e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6827","nonce":4292385779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6829","nonce":4292385779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6833","nonce":4292385779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6831","nonce":4292385779}]},"public_addr":"172.21.15.12:6827/4292385779","cluster_addr":"172.21.15.12:6829/4292385779","heartbeat_back_addr":"172.21.15.12:6833/4292385779","heartbeat_front_addr":"172.21.15.12:6831/4292385779","state":["exists","up"]},{"osd":10,"uuid":"798b1abd-f6c7-4a5f-8da0-fdae49e50d2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6825","nonce":3338566976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6827","nonce":3338566976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6831","nonce":3338566976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6829","nonce":3338566976}]},"public_addr":"172.21.15.111:6825/3338566976","cluster_addr":"172.21.15.111:6827/3338566976","heartbeat_back_addr":"172.21.15.111:6831/3338566976","heartbeat_front_addr":"172.21.15.111:6829/3338566976","state":["exists","up"]},{"osd":11,"uuid":"b18fc5f3-1049-4871-b060-e99290273cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6825","nonce":578640428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6827","nonce":578640428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6831","nonce":578640428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6829","nonce":578640428}]},"public_addr":"172.21.15.134:6825/578640428","cluster_addr":"172.21.15.134:6827/578640428","heartbeat_back_addr":"172.21.15.134:6831/578640428","heartbeat_front_addr":"172.21.15.134:6829/578640428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:16:40.828608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:05.052528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:28.297316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:53.496497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:17.947259+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:41.712894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:07.105945+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:32.129688+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:56.008516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:22.239867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:47.334692+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:12.301276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/3535299028":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/2837479598":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/3010129696":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/2593601362":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/2519382069":"2024-04-23T13:15:04.279990+0000","172.21.15.12:6801/2009831940":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/856202072":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/3509412187":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/1802873236":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/1007749062":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3588480095":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1904962176":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/1802873236":"2024-04-23T13:24:06.155564+0000","172.21.15.12:6800/2009831940":"2024-04-23T13:15:04.279990+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:24:13.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.288+0000 7f350116d640 1 -- 172.21.15.12:0/1381871018 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f34e40a3ec0 msgr2=0x7f34e40a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:13.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.288+0000 7f350116d640 1 --2- 172.21.15.12:0/1381871018 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f34e40a3ec0 0x7f34e40a6380 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f34f002c240 tx=0x7f34f0005fd0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.288+0000 7f350116d640 1 -- 172.21.15.12:0/1381871018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34fc1021a0 msgr2=0x7f34fc198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:13.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.288+0000 7f350116d640 1 --2- 172.21.15.12:0/1381871018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34fc1021a0 0x7f34fc198660 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f34ec00c920 tx=0x7f34ec00cde0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.288+0000 7f350116d640 1 -- 172.21.15.12:0/1381871018 shutdown_connections 2024-04-22T13:24:13.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.288+0000 7f350116d640 1 --2- 172.21.15.12:0/1381871018 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f34e40a3ec0 0x7f34e40a6380 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.288+0000 7f350116d640 1 --2- 172.21.15.12:0/1381871018 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34fc19cf20 0x7f34fc19d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.288+0000 7f350116d640 1 --2- 172.21.15.12:0/1381871018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34fc107270 0x7f34fc198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.288+0000 7f350116d640 1 --2- 172.21.15.12:0/1381871018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f34fc1021a0 0x7f34fc198660 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.288+0000 7f350116d640 1 -- 172.21.15.12:0/1381871018 >> 172.21.15.12:0/1381871018 conn(0x7f34fc0793a0 msgr2=0x7f34fc0ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:13.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.288+0000 7f350116d640 1 -- 172.21.15.12:0/1381871018 shutdown_connections 2024-04-22T13:24:13.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.288+0000 7f350116d640 1 -- 172.21.15.12:0/1381871018 wait complete. 2024-04-22T13:24:13.297 DEBUG:tasks.fwd_scrub:Yielding 2024-04-22T13:24:13.297 INFO:teuthology.run_tasks:Running task exec... 2024-04-22T13:24:13.306 INFO:teuthology.task.exec:Executing custom commands... 2024-04-22T13:24:13.306 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-04-22T13:24:13.307 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-22T13:24:13.307 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-22T13:24:13.307 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:24:13.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:24:13.307 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:24:13.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.415+0000 7f3d0fc9a640 1 -- 172.21.15.12:0/2076669634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d0810e910 msgr2=0x7f3d08110d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:13.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.415+0000 7f3d0fc9a640 1 --2- 172.21.15.12:0/2076669634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d0810e910 0x7f3d08110d00 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009ca0 tx=0x7f3d0002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.415+0000 7f3d0fc9a640 1 -- 172.21.15.12:0/2076669634 shutdown_connections 2024-04-22T13:24:13.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.415+0000 7f3d0fc9a640 1 --2- 172.21.15.12:0/2076669634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d0810e910 0x7f3d08110d00 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.415+0000 7f3d0fc9a640 1 --2- 172.21.15.12:0/2076669634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d08101680 0x7f3d0810e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.415+0000 7f3d0fc9a640 1 --2- 172.21.15.12:0/2076669634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d08100d60 0x7f3d08101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.415+0000 7f3d0fc9a640 1 -- 172.21.15.12:0/2076669634 >> 172.21.15.12:0/2076669634 conn(0x7f3d08077fd0 msgr2=0x7f3d080783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:13.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.415+0000 7f3d0fc9a640 1 -- 172.21.15.12:0/2076669634 shutdown_connections 2024-04-22T13:24:13.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.415+0000 7fef0c026640 1 -- 172.21.15.12:0/988867141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef04106160 msgr2=0x7fef0410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:13.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.415+0000 7fef0c026640 1 --2- 172.21.15.12:0/988867141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef04106160 0x7fef0410dd30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7feefc009ca0 tx=0x7feefc02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.415+0000 7f3d0fc9a640 1 -- 172.21.15.12:0/2076669634 wait complete. 2024-04-22T13:24:13.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.415+0000 7fef0c026640 1 -- 172.21.15.12:0/988867141 shutdown_connections 2024-04-22T13:24:13.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.415+0000 7fef0c026640 1 --2- 172.21.15.12:0/988867141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef04106160 0x7fef0410dd30 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.415+0000 7fef0c026640 1 --2- 172.21.15.12:0/988867141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef04101a40 0x7fef04105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.415+0000 7fef0c026640 1 --2- 172.21.15.12:0/988867141 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef04101090 0x7fef04101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.415+0000 7fef0c026640 1 -- 172.21.15.12:0/988867141 >> 172.21.15.12:0/988867141 conn(0x7fef04077f50 msgr2=0x7fef04078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:13.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.416+0000 7fef0c026640 1 -- 172.21.15.12:0/988867141 shutdown_connections 2024-04-22T13:24:13.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.416+0000 7fef0c026640 1 -- 172.21.15.12:0/988867141 wait complete. 2024-04-22T13:24:13.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.416+0000 7f3d0fc9a640 1 Processor -- start 2024-04-22T13:24:13.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.416+0000 7f3d0fc9a640 1 -- start start 2024-04-22T13:24:13.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.416+0000 7fef0c026640 1 Processor -- start 2024-04-22T13:24:13.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.416+0000 7f3d0fc9a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d08100d60 0x7f3d081a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.416+0000 7f3d0fc9a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d08101680 0x7f3d081a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.416+0000 7f3d0fc9a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d0810e910 0x7f3d0819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.416+0000 7f3d0fc9a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3d081143d0 con 0x7f3d0810e910 2024-04-22T13:24:13.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.416+0000 7f3d0fc9a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3d08114250 con 0x7f3d08101680 2024-04-22T13:24:13.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.416+0000 7f3d0fc9a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3d08114550 con 0x7f3d08100d60 2024-04-22T13:24:13.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.416+0000 7fef0c026640 1 -- start start 2024-04-22T13:24:13.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.416+0000 7f3d0e237640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d0810e910 0x7f3d0819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.416+0000 7f3d0e237640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d0810e910 0x7f3d0819a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59228/0 (socket says 172.21.15.12:59228) 2024-04-22T13:24:13.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.416+0000 7f3d0e237640 1 -- 172.21.15.12:0/3392064645 learned_addr learned my addr 172.21.15.12:0/3392064645 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:13.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.416+0000 7fef0c026640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef04101090 0x7fef04102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.416+0000 7fef0c026640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef04106160 0x7fef04103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.416+0000 7fef0c026640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef041035a0 0x7fef041a68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.416+0000 7fef0c026640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fef041110b0 con 0x7fef04106160 2024-04-22T13:24:13.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.417+0000 7fef0c026640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fef04110f30 con 0x7fef04101090 2024-04-22T13:24:13.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.417+0000 7fef0c026640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fef04111230 con 0x7fef041035a0 2024-04-22T13:24:13.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.417+0000 7f3d0d235640 1 --2- 172.21.15.12:0/3392064645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d08101680 0x7f3d081a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.417+0000 7f3d0da36640 1 --2- 172.21.15.12:0/3392064645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d08100d60 0x7f3d081a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.417+0000 7f3d0e237640 1 -- 172.21.15.12:0/3392064645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d08100d60 msgr2=0x7f3d081a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:13.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.417+0000 7f3d0e237640 1 --2- 172.21.15.12:0/3392064645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d08100d60 0x7f3d081a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.417+0000 7f3d0e237640 1 -- 172.21.15.12:0/3392064645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d08101680 msgr2=0x7f3d081a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:13.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.417+0000 7f3d0e237640 1 --2- 172.21.15.12:0/3392064645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d08101680 0x7f3d081a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.417+0000 7f3d0e237640 1 -- 172.21.15.12:0/3392064645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0819af60 con 0x7f3d0810e910 2024-04-22T13:24:13.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.417+0000 7fef09dc2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef04101090 0x7fef04102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.417+0000 7fef095c1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef04106160 0x7fef04103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.417+0000 7fef09dc2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef04101090 0x7fef04102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:41446/0 (socket says 172.21.15.12:41446) 2024-04-22T13:24:13.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.417+0000 7fef09dc2640 1 -- 172.21.15.12:0/1992394604 learned_addr learned my addr 172.21.15.12:0/1992394604 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:13.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.417+0000 7f3d0da36640 1 --2- 172.21.15.12:0/3392064645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d08100d60 0x7f3d081a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:13.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.417+0000 7fef0a5c3640 1 --2- 172.21.15.12:0/1992394604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef041035a0 0x7fef041a68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.417+0000 7f3d0e237640 1 --2- 172.21.15.12:0/3392064645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d0810e910 0x7f3d0819a790 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3d0002c700 tx=0x7f3d00034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:13.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.417+0000 7fef095c1640 1 -- 172.21.15.12:0/1992394604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef041035a0 msgr2=0x7fef041a68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:13.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.417+0000 7fef095c1640 1 --2- 172.21.15.12:0/1992394604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef041035a0 0x7fef041a68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.417+0000 7fef095c1640 1 -- 172.21.15.12:0/1992394604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef04101090 msgr2=0x7fef04102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:13.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.417+0000 7fef095c1640 1 --2- 172.21.15.12:0/1992394604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef04101090 0x7fef04102b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.417+0000 7fef095c1640 1 -- 172.21.15.12:0/1992394604 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef041a6ff0 con 0x7fef04106160 2024-04-22T13:24:13.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.417+0000 7fef09dc2640 1 --2- 172.21.15.12:0/1992394604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef04101090 0x7fef04102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:24:13.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.417+0000 7f3cfeffd640 1 -- 172.21.15.12:0/3392064645 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d00041020 con 0x7f3d0810e910 2024-04-22T13:24:13.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.417+0000 7fef095c1640 1 --2- 172.21.15.12:0/1992394604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef04106160 0x7fef04103060 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7feef400e9c0 tx=0x7feef400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:13.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.417+0000 7f3cfeffd640 1 -- 172.21.15.12:0/3392064645 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d00034ca0 con 0x7f3d0810e910 2024-04-22T13:24:13.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.417+0000 7f3d0fc9a640 1 -- 172.21.15.12:0/3392064645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d0819b1f0 con 0x7f3d0810e910 2024-04-22T13:24:13.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.417+0000 7f3cfeffd640 1 -- 172.21.15.12:0/3392064645 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d00039040 con 0x7f3d0810e910 2024-04-22T13:24:13.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.417+0000 7f3d0fc9a640 1 -- 172.21.15.12:0/3392064645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d081a7470 con 0x7f3d0810e910 2024-04-22T13:24:13.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.418+0000 7feefaffd640 1 -- 172.21.15.12:0/1992394604 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef400ccc0 con 0x7fef04106160 2024-04-22T13:24:13.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.418+0000 7feefaffd640 1 -- 172.21.15.12:0/1992394604 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feef40108e0 con 0x7fef04106160 2024-04-22T13:24:13.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.418+0000 7fef0c026640 1 -- 172.21.15.12:0/1992394604 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef041a72e0 con 0x7fef04106160 2024-04-22T13:24:13.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.418+0000 7feefaffd640 1 -- 172.21.15.12:0/1992394604 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef40187c0 con 0x7fef04106160 2024-04-22T13:24:13.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.418+0000 7fef0c026640 1 -- 172.21.15.12:0/1992394604 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef041a7830 con 0x7fef04106160 2024-04-22T13:24:13.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.419+0000 7f3d0fc9a640 1 -- 172.21.15.12:0/3392064645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d08069e00 con 0x7f3d0810e910 2024-04-22T13:24:13.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.420+0000 7f3cfeffd640 1 -- 172.21.15.12:0/3392064645 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f3d00044020 con 0x7f3d0810e910 2024-04-22T13:24:13.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.420+0000 7f3cfeffd640 1 --2- 172.21.15.12:0/3392064645 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3cf40a3f60 0x7f3cf40a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.420+0000 7f3cfeffd640 1 -- 172.21.15.12:0/3392064645 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f3d000f2730 con 0x7f3d0810e910 2024-04-22T13:24:13.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.420+0000 7f3d0da36640 1 --2- 172.21.15.12:0/3392064645 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3cf40a3f60 0x7f3cf40a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.421+0000 7feefaffd640 1 -- 172.21.15.12:0/1992394604 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7feef4010430 con 0x7fef04106160 2024-04-22T13:24:13.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.421+0000 7fef0ec0f740 1 Processor -- start 2024-04-22T13:24:13.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.421+0000 7fef0ec0f740 1 -- start start 2024-04-22T13:24:13.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.421+0000 7f3d0da36640 1 --2- 172.21.15.12:0/3392064645 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3cf40a3f60 0x7f3cf40a6420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3d04009b50 tx=0x7f3d04005f90 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:13.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.421+0000 7fef0ec0f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f6ab538e0 0x559f6ab53cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.421+0000 7fef0ec0f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f6ab54290 0x559f6ab5c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.421+0000 7fef0ec0f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f6ab5ce10 0x559f6ab5f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.421+0000 7fef0ec0f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559f6aab7f00 con 0x559f6ab5ce10 2024-04-22T13:24:13.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.421+0000 7fef0ec0f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559f6aa55590 con 0x559f6ab54290 2024-04-22T13:24:13.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.421+0000 7fef0ec0f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559f6aab3ec0 con 0x559f6ab538e0 2024-04-22T13:24:13.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.421+0000 7fef095c1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f6ab5ce10 0x559f6ab5f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.421+0000 7fef09dc2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f6ab54290 0x559f6ab5c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.421+0000 7fef095c1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f6ab5ce10 0x559f6ab5f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59246/0 (socket says 172.21.15.12:59246) 2024-04-22T13:24:13.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.421+0000 7fef095c1640 1 -- 172.21.15.12:0/4217184835 learned_addr learned my addr 172.21.15.12:0/4217184835 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:13.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.421+0000 7fef0a5c3640 1 --2- 172.21.15.12:0/4217184835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f6ab538e0 0x559f6ab53cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.422+0000 7feefaffd640 1 --2- 172.21.15.12:0/1992394604 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7feef00a3fc0 0x7feef00a6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.422+0000 7fef095c1640 1 -- 172.21.15.12:0/4217184835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f6ab538e0 msgr2=0x559f6ab53cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:13.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.422+0000 7fef095c1640 1 --2- 172.21.15.12:0/4217184835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f6ab538e0 0x559f6ab53cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.422+0000 7fef095c1640 1 -- 172.21.15.12:0/4217184835 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f6ab54290 msgr2=0x559f6ab5c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:13.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.422+0000 7fef095c1640 1 --2- 172.21.15.12:0/4217184835 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f6ab54290 0x559f6ab5c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.422+0000 7fef095c1640 1 -- 172.21.15.12:0/4217184835 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f6a8927e0 con 0x559f6ab5ce10 2024-04-22T13:24:13.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.422+0000 7fef09dc2640 1 --2- 172.21.15.12:0/4217184835 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f6ab54290 0x559f6ab5c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:24:13.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.422+0000 7fef0a5c3640 1 --2- 172.21.15.12:0/1992394604 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7feef00a3fc0 0x7feef00a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.422+0000 7feefaffd640 1 -- 172.21.15.12:0/1992394604 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7feef40d0b60 con 0x7fef04106160 2024-04-22T13:24:13.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.422+0000 7fef095c1640 1 --2- 172.21.15.12:0/4217184835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f6ab5ce10 0x559f6ab5f260 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7feef40ce290 tx=0x7feef40670e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de44be936d730c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:13.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.422+0000 7fef0a5c3640 1 --2- 172.21.15.12:0/1992394604 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7feef00a3fc0 0x7feef00a6480 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feefc005e30 tx=0x7feefc02e040 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:13.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.425+0000 7feeeffff640 1 -- 172.21.15.12:0/4217184835 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef409c310 con 0x559f6ab5ce10 2024-04-22T13:24:13.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.426+0000 7feeeffff640 1 -- 172.21.15.12:0/4217184835 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feef4026050 con 0x559f6ab5ce10 2024-04-22T13:24:13.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.426+0000 7feeeffff640 1 -- 172.21.15.12:0/4217184835 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef409c9b0 con 0x559f6ab5ce10 2024-04-22T13:24:13.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.426+0000 7f3cfeffd640 1 -- 172.21.15.12:0/3392064645 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3d0004b020 con 0x7f3d0810e910 2024-04-22T13:24:13.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.426+0000 7fef0ec0f740 1 -- 172.21.15.12:0/4217184835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f6ab5ce10 msgr2=0x559f6ab5f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:13.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.426+0000 7fef0ec0f740 1 --2- 172.21.15.12:0/4217184835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f6ab5ce10 0x559f6ab5f260 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7feef40ce290 tx=0x7feef40670e0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.427+0000 7fef0ec0f740 1 -- 172.21.15.12:0/4217184835 shutdown_connections 2024-04-22T13:24:13.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.427+0000 7fef0ec0f740 1 --2- 172.21.15.12:0/4217184835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f6ab5ce10 0x559f6ab5f260 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.427+0000 7fef0ec0f740 1 --2- 172.21.15.12:0/4217184835 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f6ab54290 0x559f6ab5c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.427+0000 7fef0ec0f740 1 --2- 172.21.15.12:0/4217184835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f6ab538e0 0x559f6ab53cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.427+0000 7fef0ec0f740 1 -- 172.21.15.12:0/4217184835 >> 172.21.15.12:0/4217184835 conn(0x559f6ab4dfb0 msgr2=0x559f6ab4e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:13.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.427+0000 7fef0ec0f740 1 -- 172.21.15.12:0/4217184835 shutdown_connections 2024-04-22T13:24:13.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.427+0000 7fef0ec0f740 1 -- 172.21.15.12:0/4217184835 wait complete. 2024-04-22T13:24:13.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.427+0000 7fef0ec0f740 1 -- start start 2024-04-22T13:24:13.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.427+0000 7fef0ec0f740 1 Processor -- start 2024-04-22T13:24:13.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.427+0000 7fef0ec0f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f6ab538e0 0x559f6ab59e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.428+0000 7fef0ec0f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f6ab54290 0x559f6ab5a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.428+0000 7fef0ec0f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f6ab5ce10 0x559f6ab5a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.428+0000 7fef0ec0f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559f6ab5fd20 con 0x559f6ab538e0 2024-04-22T13:24:13.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.428+0000 7fef0ec0f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559f6aa71580 con 0x559f6ab5ce10 2024-04-22T13:24:13.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.428+0000 7fef0ec0f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559f6ab5fea0 con 0x559f6ab54290 2024-04-22T13:24:13.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.428+0000 7fef0ec0f740 10 client.0 _ll_register_callbacks cb 0x559f6aa77650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:13.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.428+0000 7fef09dc2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f6ab538e0 0x559f6ab59e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.428+0000 7fef09dc2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f6ab538e0 0x559f6ab59e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59256/0 (socket says 172.21.15.12:59256) 2024-04-22T13:24:13.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.428+0000 7fef09dc2640 1 -- 172.21.15.12:0/2112612275 learned_addr learned my addr 172.21.15.12:0/2112612275 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:13.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.428+0000 7fef095c1640 1 --2- 172.21.15.12:0/2112612275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f6ab54290 0x559f6ab5a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.428+0000 7fef0a5c3640 1 --2- 172.21.15.12:0/2112612275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f6ab5ce10 0x559f6ab5a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.428+0000 7fef09dc2640 1 -- 172.21.15.12:0/2112612275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f6ab54290 msgr2=0x559f6ab5a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:13.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.428+0000 7fef09dc2640 1 --2- 172.21.15.12:0/2112612275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f6ab54290 0x559f6ab5a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.428+0000 7fef09dc2640 1 -- 172.21.15.12:0/2112612275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f6ab5ce10 msgr2=0x559f6ab5a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:13.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.428+0000 7fef09dc2640 1 --2- 172.21.15.12:0/2112612275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f6ab5ce10 0x559f6ab5a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.428+0000 7fef09dc2640 1 -- 172.21.15.12:0/2112612275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f6a8f2f70 con 0x559f6ab538e0 2024-04-22T13:24:13.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.428+0000 7fef0a5c3640 1 --2- 172.21.15.12:0/2112612275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f6ab5ce10 0x559f6ab5a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:13.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.428+0000 7fef09dc2640 1 --2- 172.21.15.12:0/2112612275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f6ab538e0 0x559f6ab59e10 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fef00012d20 tx=0x7fef0000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17805c83fa72a3ea server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:13.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.428+0000 7feeed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:13.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.428+0000 7feeed7fa640 1 -- 172.21.15.12:0/2112612275 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef00004020 con 0x559f6ab538e0 2024-04-22T13:24:13.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.428+0000 7feeed7fa640 1 -- 172.21.15.12:0/2112612275 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fef000041c0 con 0x559f6ab538e0 2024-04-22T13:24:13.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.428+0000 7fef0ec0f740 1 -- 172.21.15.12:0/2112612275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feef4026050 con 0x559f6ab538e0 2024-04-22T13:24:13.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.429+0000 7feeed7fa640 1 -- 172.21.15.12:0/2112612275 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fef000043c0 con 0x559f6ab538e0 2024-04-22T13:24:13.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.429+0000 7fef0ec0f740 10 client.15561 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:13.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.429+0000 7fef0ec0f740 1 -- 172.21.15.12:0/2112612275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559f6a78a170 con 0x559f6ab538e0 2024-04-22T13:24:13.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.429+0000 7feeed7fa640 1 -- 172.21.15.12:0/2112612275 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feef4026050 con 0x559f6ab538e0 2024-04-22T13:24:13.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.429+0000 7feeed7fa640 1 -- 172.21.15.12:0/2112612275 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fef0001c470 con 0x559f6ab538e0 2024-04-22T13:24:13.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.429+0000 7fef0ec0f740 10 client.15561 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:13.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.429+0000 7fef0ec0f740 10 client.15561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:24:13.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.429+0000 7fef0ec0f740 20 client.15561 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:13.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.429+0000 7fef0ec0f740 1 --2- 172.21.15.12:0/2112612275 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559f6ab6d2c0 0x559f6ab6f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.429+0000 7fef0ec0f740 4 client.15561 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:24:13.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.429+0000 7fef0ec0f740 1 -- 172.21.15.12:0/2112612275 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f6aac5920 con 0x559f6ab6d2c0 2024-04-22T13:24:13.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.429+0000 7fef095c1640 1 --2- 172.21.15.12:0/2112612275 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559f6ab6d2c0 0x559f6ab6f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.430+0000 7feeed7fa640 10 client.15561 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:13.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.430+0000 7fef095c1640 1 --2- 172.21.15.12:0/2112612275 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559f6ab6d2c0 0x559f6ab6f760 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bceb70a916b81c79 server_cookie=e411acd0788e61d9 in_seq=0 out_seq=0 2024-04-22T13:24:13.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.431+0000 7feeed7fa640 1 -- 172.21.15.12:0/2112612275 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559f6aac5920 con 0x559f6ab6d2c0 2024-04-22T13:24:13.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.431+0000 7feeed7fa640 10 client.15561 handle_command_reply: tid=0 2024-04-22T13:24:13.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.431+0000 7fef0ec0f740 1 client.15561 shutdown 2024-04-22T13:24:13.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.431+0000 7fef0ec0f740 1 -- 172.21.15.12:0/2112612275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f6ab538e0 msgr2=0x559f6ab59e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:13.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.431+0000 7fef0ec0f740 1 --2- 172.21.15.12:0/2112612275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f6ab538e0 0x559f6ab59e10 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fef00012d20 tx=0x7fef0000f790 comp rx=0 tx=0).stop 2024-04-22T13:24:13.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.431+0000 7fef0ec0f740 1 -- 172.21.15.12:0/2112612275 shutdown_connections 2024-04-22T13:24:13.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.431+0000 7fef0ec0f740 1 --2- 172.21.15.12:0/2112612275 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559f6ab6d2c0 0x559f6ab6f760 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.431+0000 7fef0ec0f740 1 --2- 172.21.15.12:0/2112612275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f6ab5ce10 0x559f6ab5a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.431+0000 7feeed7fa640 0 client.15561 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:13.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.431+0000 7fef0ec0f740 1 --2- 172.21.15.12:0/2112612275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f6ab54290 0x559f6ab5a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.431+0000 7fef0ec0f740 1 --2- 172.21.15.12:0/2112612275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f6ab538e0 0x559f6ab59e10 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.431+0000 7fef0ec0f740 1 -- 172.21.15.12:0/2112612275 >> 172.21.15.12:0/2112612275 conn(0x559f6ab4dfb0 msgr2=0x559f6ab4fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:13.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.431+0000 7fef0ec0f740 1 -- 172.21.15.12:0/2112612275 shutdown_connections 2024-04-22T13:24:13.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.431+0000 7fef0ec0f740 1 -- 172.21.15.12:0/2112612275 wait complete. 2024-04-22T13:24:13.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.431+0000 7fef0ec0f740 20 client.15561 trim_cache size 0 max 16384 2024-04-22T13:24:13.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.465+0000 7fef0ec0f740 1 Processor -- start 2024-04-22T13:24:13.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.465+0000 7fef0ec0f740 1 -- start start 2024-04-22T13:24:13.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.465+0000 7fef0ec0f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f6ab538e0 0x559f6ab5c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.465+0000 7fef0ec0f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f6ab54290 0x559f6ab68be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.465+0000 7fef0ec0f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f6ab5ce10 0x559f6ab58f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.465+0000 7fef0ec0f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fef041afd20 con 0x559f6ab538e0 2024-04-22T13:24:13.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.465+0000 7fef0ec0f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fef041b1860 con 0x559f6ab5ce10 2024-04-22T13:24:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.465+0000 7fef0ec0f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fef041b03f0 con 0x559f6ab54290 2024-04-22T13:24:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.465+0000 7fef09dc2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f6ab538e0 0x559f6ab5c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.465+0000 7fef09dc2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f6ab538e0 0x559f6ab5c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59270/0 (socket says 172.21.15.12:59270) 2024-04-22T13:24:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.465+0000 7fef09dc2640 1 -- 172.21.15.12:0/2557492418 learned_addr learned my addr 172.21.15.12:0/2557492418 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.465+0000 7fef0a5c3640 1 --2- 172.21.15.12:0/2557492418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f6ab5ce10 0x559f6ab58f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.465+0000 7fef095c1640 1 --2- 172.21.15.12:0/2557492418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f6ab54290 0x559f6ab68be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.465+0000 7fef0a5c3640 1 -- 172.21.15.12:0/2557492418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f6ab54290 msgr2=0x559f6ab68be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.465+0000 7fef0a5c3640 1 --2- 172.21.15.12:0/2557492418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f6ab54290 0x559f6ab68be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.465+0000 7fef0a5c3640 1 -- 172.21.15.12:0/2557492418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f6ab538e0 msgr2=0x559f6ab5c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.465+0000 7fef0a5c3640 1 --2- 172.21.15.12:0/2557492418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f6ab538e0 0x559f6ab5c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.465+0000 7fef0a5c3640 1 -- 172.21.15.12:0/2557492418 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f6ab52c20 con 0x559f6ab5ce10 2024-04-22T13:24:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.465+0000 7fef09dc2640 1 --2- 172.21.15.12:0/2557492418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f6ab538e0 0x559f6ab5c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:24:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.466+0000 7fef0a5c3640 1 --2- 172.21.15.12:0/2557492418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f6ab5ce10 0x559f6ab58f90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feefc009bf0 tx=0x7feefc040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14d8f7090d215aa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:13.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.466+0000 7feeef7fe640 1 -- 172.21.15.12:0/2557492418 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc050070 con 0x559f6ab5ce10 2024-04-22T13:24:13.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.466+0000 7feeef7fe640 1 -- 172.21.15.12:0/2557492418 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feefc045040 con 0x559f6ab5ce10 2024-04-22T13:24:13.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.466+0000 7feeef7fe640 1 -- 172.21.15.12:0/2557492418 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc039040 con 0x559f6ab5ce10 2024-04-22T13:24:13.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.467+0000 7fef0ec0f740 1 -- 172.21.15.12:0/2557492418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f6ab5ce10 msgr2=0x559f6ab58f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:13.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.467+0000 7fef0ec0f740 1 --2- 172.21.15.12:0/2557492418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f6ab5ce10 0x559f6ab58f90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feefc009bf0 tx=0x7feefc040000 comp rx=0 tx=0).stop 2024-04-22T13:24:13.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.467+0000 7fef0ec0f740 1 -- 172.21.15.12:0/2557492418 shutdown_connections 2024-04-22T13:24:13.469 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.467+0000 7fef0ec0f740 1 --2- 172.21.15.12:0/2557492418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f6ab5ce10 0x559f6ab58f90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.467+0000 7fef0ec0f740 1 --2- 172.21.15.12:0/2557492418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f6ab54290 0x559f6ab68be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.467+0000 7fef0ec0f740 1 --2- 172.21.15.12:0/2557492418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f6ab538e0 0x559f6ab5c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.467+0000 7fef0ec0f740 1 -- 172.21.15.12:0/2557492418 >> 172.21.15.12:0/2557492418 conn(0x559f6ab4dfb0 msgr2=0x559f6ab4c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:13.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.467+0000 7fef0ec0f740 1 -- 172.21.15.12:0/2557492418 shutdown_connections 2024-04-22T13:24:13.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.467+0000 7fef0ec0f740 1 -- 172.21.15.12:0/2557492418 wait complete. 2024-04-22T13:24:13.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.467+0000 7fef0ec0f740 1 -- start start 2024-04-22T13:24:13.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.467+0000 7fef0ec0f740 1 Processor -- start 2024-04-22T13:24:13.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.467+0000 7fef0ec0f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f6ab538e0 0x559f6ab56ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.467+0000 7fef0ec0f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f6ab54290 0x559f6ab570e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.467+0000 7fef0ec0f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f6ab5ce10 0x559f6ab57620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.467+0000 7fef0ec0f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fef041b1190 con 0x559f6ab5ce10 2024-04-22T13:24:13.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.467+0000 7fef0ec0f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fef041b0ac0 con 0x559f6ab54290 2024-04-22T13:24:13.470 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.467+0000 7fef0ec0f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fef041add20 con 0x559f6ab538e0 2024-04-22T13:24:13.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.467+0000 7fef0ec0f740 10 client.0 _ll_register_callbacks cb 0x7fef00009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:13.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.468+0000 7fef0a5c3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f6ab5ce10 0x559f6ab57620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.468+0000 7fef0a5c3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f6ab5ce10 0x559f6ab57620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59274/0 (socket says 172.21.15.12:59274) 2024-04-22T13:24:13.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.468+0000 7fef0a5c3640 1 -- 172.21.15.12:0/959883813 learned_addr learned my addr 172.21.15.12:0/959883813 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:13.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.468+0000 7fef09dc2640 1 --2- 172.21.15.12:0/959883813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f6ab538e0 0x559f6ab56ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.468+0000 7fef095c1640 1 --2- 172.21.15.12:0/959883813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f6ab54290 0x559f6ab570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.468+0000 7fef0a5c3640 1 -- 172.21.15.12:0/959883813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f6ab538e0 msgr2=0x559f6ab56ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:13.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.468+0000 7fef0a5c3640 1 --2- 172.21.15.12:0/959883813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f6ab538e0 0x559f6ab56ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.468+0000 7fef0a5c3640 1 -- 172.21.15.12:0/959883813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f6ab54290 msgr2=0x559f6ab570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:13.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.468+0000 7fef0a5c3640 1 --2- 172.21.15.12:0/959883813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f6ab54290 0x559f6ab570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.468+0000 7fef0a5c3640 1 -- 172.21.15.12:0/959883813 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f6abd28f0 con 0x559f6ab5ce10 2024-04-22T13:24:13.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.468+0000 7fef095c1640 1 --2- 172.21.15.12:0/959883813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f6ab54290 0x559f6ab570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:13.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.468+0000 7fef0a5c3640 1 --2- 172.21.15.12:0/959883813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f6ab5ce10 0x559f6ab57620 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7feefc034d70 tx=0x7feefc034da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a8aa42e7e1e79f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:13.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.468+0000 7feeed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:13.471 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.468+0000 7feeed7fa640 1 -- 172.21.15.12:0/959883813 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc05c020 con 0x559f6ab5ce10 2024-04-22T13:24:13.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.468+0000 7feeed7fa640 1 -- 172.21.15.12:0/959883813 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7feefc034440 con 0x559f6ab5ce10 2024-04-22T13:24:13.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.468+0000 7fef0ec0f740 1 -- 172.21.15.12:0/959883813 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feefc045040 con 0x559f6ab5ce10 2024-04-22T13:24:13.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.468+0000 7feeed7fa640 1 -- 172.21.15.12:0/959883813 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feefc067050 con 0x559f6ab5ce10 2024-04-22T13:24:13.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.469+0000 7fef0ec0f740 10 client.15567 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:13.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.469+0000 7fef0ec0f740 1 -- 172.21.15.12:0/959883813 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559f6abd3c30 con 0x559f6ab5ce10 2024-04-22T13:24:13.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.469+0000 7feeed7fa640 1 -- 172.21.15.12:0/959883813 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feefc045040 con 0x559f6ab5ce10 2024-04-22T13:24:13.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.469+0000 7feeed7fa640 1 -- 172.21.15.12:0/959883813 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7feefc040b30 con 0x559f6ab5ce10 2024-04-22T13:24:13.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.469+0000 7fef0ec0f740 10 client.15567 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:13.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.469+0000 7fef0ec0f740 10 client.15567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:24:13.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.469+0000 7fef0ec0f740 20 client.15567 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:13.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.469+0000 7fef0ec0f740 1 --2- 172.21.15.12:0/959883813 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559f6abd43c0 0x559f6abd67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.469+0000 7fef0ec0f740 4 client.15567 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:24:13.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.469+0000 7fef0ec0f740 1 -- 172.21.15.12:0/959883813 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559f6aabf790 con 0x559f6abd43c0 2024-04-22T13:24:13.472 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.469+0000 7fef09dc2640 1 --2- 172.21.15.12:0/959883813 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559f6abd43c0 0x559f6abd67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.470+0000 7fef09dc2640 1 --2- 172.21.15.12:0/959883813 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559f6abd43c0 0x559f6abd67e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc50eeac998af77d server_cookie=ec69d03ffc2ab723 in_seq=0 out_seq=0 2024-04-22T13:24:13.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.470+0000 7feeed7fa640 10 client.15567 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:13.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.471+0000 7feeed7fa640 1 -- 172.21.15.12:0/959883813 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559f6aabf790 con 0x559f6abd43c0 2024-04-22T13:24:13.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.471+0000 7feeed7fa640 10 client.15567 handle_command_reply: tid=0 2024-04-22T13:24:13.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.471+0000 7fef0ec0f740 1 client.15567 shutdown 2024-04-22T13:24:13.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.471+0000 7fef0ec0f740 1 -- 172.21.15.12:0/959883813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f6ab5ce10 msgr2=0x559f6ab57620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:13.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.471+0000 7fef0ec0f740 1 --2- 172.21.15.12:0/959883813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f6ab5ce10 0x559f6ab57620 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7feefc034d70 tx=0x7feefc034da0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.471+0000 7fef0ec0f740 1 -- 172.21.15.12:0/959883813 shutdown_connections 2024-04-22T13:24:13.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.471+0000 7fef0ec0f740 1 --2- 172.21.15.12:0/959883813 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559f6abd43c0 0x559f6abd67e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.471+0000 7fef0ec0f740 1 --2- 172.21.15.12:0/959883813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f6ab5ce10 0x559f6ab57620 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.471+0000 7feeed7fa640 0 client.15567 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:13.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.471+0000 7fef0ec0f740 1 --2- 172.21.15.12:0/959883813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f6ab54290 0x559f6ab570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.471+0000 7fef0ec0f740 1 --2- 172.21.15.12:0/959883813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f6ab538e0 0x559f6ab56ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.471+0000 7fef0ec0f740 1 -- 172.21.15.12:0/959883813 >> 172.21.15.12:0/959883813 conn(0x559f6ab4dfb0 msgr2=0x559f6ab58e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:13.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.471+0000 7fef0ec0f740 1 -- 172.21.15.12:0/959883813 shutdown_connections 2024-04-22T13:24:13.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.471+0000 7fef0ec0f740 1 -- 172.21.15.12:0/959883813 wait complete. 2024-04-22T13:24:13.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.471+0000 7fef0ec0f740 20 client.15567 trim_cache size 0 max 16384 2024-04-22T13:24:13.474 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:24:13.474 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-04-22T13:24:13.474 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "aa5ba72d-11cb-489a-91fe-1b2fb60be456", 2024-04-22T13:24:13.474 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-04-22T13:24:13.474 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:24:13.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.472+0000 7fef0c026640 1 -- 172.21.15.12:0/1992394604 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7feef00a3fc0 msgr2=0x7feef00a6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:13.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.472+0000 7fef0c026640 1 --2- 172.21.15.12:0/1992394604 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7feef00a3fc0 0x7feef00a6480 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feefc005e30 tx=0x7feefc02e040 comp rx=0 tx=0).stop 2024-04-22T13:24:13.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.472+0000 7fef0c026640 1 -- 172.21.15.12:0/1992394604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef04106160 msgr2=0x7fef04103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:13.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.472+0000 7fef0c026640 1 --2- 172.21.15.12:0/1992394604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef04106160 0x7fef04103060 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7feef400e9c0 tx=0x7feef400ee80 comp rx=0 tx=0).stop 2024-04-22T13:24:13.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.472+0000 7fef0c026640 1 -- 172.21.15.12:0/1992394604 shutdown_connections 2024-04-22T13:24:13.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.472+0000 7fef0c026640 1 --2- 172.21.15.12:0/1992394604 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7feef00a3fc0 0x7feef00a6480 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.472+0000 7fef0c026640 1 --2- 172.21.15.12:0/1992394604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef041035a0 0x7fef041a68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.472+0000 7fef0c026640 1 --2- 172.21.15.12:0/1992394604 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef04106160 0x7fef04103060 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.472+0000 7fef0c026640 1 --2- 172.21.15.12:0/1992394604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef04101090 0x7fef04102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.472+0000 7fef0c026640 1 -- 172.21.15.12:0/1992394604 >> 172.21.15.12:0/1992394604 conn(0x7fef04077f50 msgr2=0x7fef040694c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:13.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.472+0000 7fef0c026640 1 -- 172.21.15.12:0/1992394604 shutdown_connections 2024-04-22T13:24:13.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.472+0000 7fef0c026640 1 -- 172.21.15.12:0/1992394604 wait complete. 2024-04-22T13:24:13.480 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:24:13.481 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:24:13.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.589+0000 7fd624f52640 1 -- 172.21.15.12:0/1299140386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd62010c500 msgr2=0x7fd62010e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:13.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.589+0000 7fd624f52640 1 --2- 172.21.15.12:0/1299140386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd62010c500 0x7fd62010e9a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd618009ca0 tx=0x7fd61802c230 comp rx=0 tx=0).stop 2024-04-22T13:24:13.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.589+0000 7fd624f52640 1 -- 172.21.15.12:0/1299140386 shutdown_connections 2024-04-22T13:24:13.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.589+0000 7fd624f52640 1 --2- 172.21.15.12:0/1299140386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd62010c500 0x7fd62010e9a0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.589+0000 7fd624f52640 1 --2- 172.21.15.12:0/1299140386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd620103ae0 0x7fd62010bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.589+0000 7fd624f52640 1 --2- 172.21.15.12:0/1299140386 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6201031c0 0x7fd6201035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.589+0000 7fd624f52640 1 -- 172.21.15.12:0/1299140386 >> 172.21.15.12:0/1299140386 conn(0x7fd620078000 msgr2=0x7fd620078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:13.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.589+0000 7fd624f52640 1 -- 172.21.15.12:0/1299140386 shutdown_connections 2024-04-22T13:24:13.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.589+0000 7fd624f52640 1 -- 172.21.15.12:0/1299140386 wait complete. 2024-04-22T13:24:13.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.590+0000 7fd624f52640 1 Processor -- start 2024-04-22T13:24:13.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.590+0000 7fd624f52640 1 -- start start 2024-04-22T13:24:13.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.590+0000 7fd624f52640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6201031c0 0x7fd620198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.590+0000 7fd624f52640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd620103ae0 0x7fd620198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.590+0000 7fd624f52640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd62010c500 0x7fd62019cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.590+0000 7fd624f52640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd620112170 con 0x7fd620103ae0 2024-04-22T13:24:13.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.590+0000 7fd624f52640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd620111ff0 con 0x7fd6201031c0 2024-04-22T13:24:13.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.590+0000 7fd624f52640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd6201122f0 con 0x7fd62010c500 2024-04-22T13:24:13.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.590+0000 7fd617fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd620103ae0 0x7fd620198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.590+0000 7fd617fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd620103ae0 0x7fd620198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59286/0 (socket says 172.21.15.12:59286) 2024-04-22T13:24:13.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.590+0000 7fd617fff640 1 -- 172.21.15.12:0/3020388294 learned_addr learned my addr 172.21.15.12:0/3020388294 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:13.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.590+0000 7fd61e59c640 1 --2- 172.21.15.12:0/3020388294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6201031c0 0x7fd620198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.590+0000 7fd61ed9d640 1 --2- 172.21.15.12:0/3020388294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd62010c500 0x7fd62019cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.590+0000 7fd617fff640 1 -- 172.21.15.12:0/3020388294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd62010c500 msgr2=0x7fd62019cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:13.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.590+0000 7fd617fff640 1 --2- 172.21.15.12:0/3020388294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd62010c500 0x7fd62019cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.590+0000 7fd617fff640 1 -- 172.21.15.12:0/3020388294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6201031c0 msgr2=0x7fd620198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:13.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.590+0000 7fd617fff640 1 --2- 172.21.15.12:0/3020388294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6201031c0 0x7fd620198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.590+0000 7fd617fff640 1 -- 172.21.15.12:0/3020388294 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd62019d660 con 0x7fd620103ae0 2024-04-22T13:24:13.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.590+0000 7fd61ed9d640 1 --2- 172.21.15.12:0/3020388294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd62010c500 0x7fd62019cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:13.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.591+0000 7fd617fff640 1 --2- 172.21.15.12:0/3020388294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd620103ae0 0x7fd620198bb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd60800d950 tx=0x7fd60800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:13.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.591+0000 7fd6177fe640 1 -- 172.21.15.12:0/3020388294 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd608017030 con 0x7fd620103ae0 2024-04-22T13:24:13.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.591+0000 7fd624f52640 1 -- 172.21.15.12:0/3020388294 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd62019d950 con 0x7fd620103ae0 2024-04-22T13:24:13.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.591+0000 7fd6177fe640 1 -- 172.21.15.12:0/3020388294 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd608004020 con 0x7fd620103ae0 2024-04-22T13:24:13.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.591+0000 7fd6177fe640 1 -- 172.21.15.12:0/3020388294 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6080047a0 con 0x7fd620103ae0 2024-04-22T13:24:13.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.591+0000 7fd624f52640 1 -- 172.21.15.12:0/3020388294 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6201a51e0 con 0x7fd620103ae0 2024-04-22T13:24:13.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.593+0000 7fd6177fe640 1 -- 172.21.15.12:0/3020388294 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fd608009a30 con 0x7fd620103ae0 2024-04-22T13:24:13.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.593+0000 7fd6177fe640 1 --2- 172.21.15.12:0/3020388294 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd60c0a3e40 0x7fd60c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.594+0000 7fd6177fe640 1 -- 172.21.15.12:0/3020388294 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fd6080d0410 con 0x7fd620103ae0 2024-04-22T13:24:13.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.594+0000 7fd61e59c640 1 --2- 172.21.15.12:0/3020388294 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd60c0a3e40 0x7fd60c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.594+0000 7fd61e59c640 1 --2- 172.21.15.12:0/3020388294 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd60c0a3e40 0x7fd60c0a6300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd610009bd0 tx=0x7fd610005f90 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:13.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.594+0000 7fd627c27740 1 Processor -- start 2024-04-22T13:24:13.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.595+0000 7fd627c27740 1 -- start start 2024-04-22T13:24:13.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.595+0000 7fd627c27740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14e25c580 0x55c14e25c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.595+0000 7fd627c27740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14e25cea0 0x55c14e265540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.595+0000 7fd627c27740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14e265a80 0x55c14e267e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.595+0000 7fd627c27740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c14e1c11e0 con 0x55c14e265a80 2024-04-22T13:24:13.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.595+0000 7fd627c27740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c14e15e390 con 0x55c14e25c580 2024-04-22T13:24:13.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.595+0000 7fd627c27740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c14e1bd1a0 con 0x55c14e25cea0 2024-04-22T13:24:13.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.595+0000 7fd61ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14e265a80 0x55c14e267e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.595+0000 7fd61ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14e265a80 0x55c14e267e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59298/0 (socket says 172.21.15.12:59298) 2024-04-22T13:24:13.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.595+0000 7fd61ed9d640 1 -- 172.21.15.12:0/3630754055 learned_addr learned my addr 172.21.15.12:0/3630754055 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:13.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.595+0000 7fd617fff640 1 --2- 172.21.15.12:0/3630754055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14e25cea0 0x55c14e265540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.595+0000 7fd61e59c640 1 --2- 172.21.15.12:0/3630754055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14e25c580 0x55c14e25c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.595+0000 7fd61ed9d640 1 -- 172.21.15.12:0/3630754055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14e25cea0 msgr2=0x55c14e265540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:13.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.595+0000 7fd61ed9d640 1 --2- 172.21.15.12:0/3630754055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14e25cea0 0x55c14e265540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.595+0000 7fd61ed9d640 1 -- 172.21.15.12:0/3630754055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14e25c580 msgr2=0x55c14e25c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:13.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.595+0000 7fd61ed9d640 1 --2- 172.21.15.12:0/3630754055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14e25c580 0x55c14e25c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.595+0000 7fd61ed9d640 1 -- 172.21.15.12:0/3630754055 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c14df9d7d0 con 0x55c14e265a80 2024-04-22T13:24:13.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.595+0000 7fd61e59c640 1 --2- 172.21.15.12:0/3630754055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14e25c580 0x55c14e25c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:13.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.595+0000 7fd617fff640 1 --2- 172.21.15.12:0/3630754055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14e25cea0 0x55c14e265540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:13.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.596+0000 7fd61ed9d640 1 --2- 172.21.15.12:0/3630754055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14e265a80 0x55c14e267e70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd618009ca0 tx=0x7fd618005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a68eff3f5486bd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:13.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.596+0000 7fd614ff9640 1 -- 172.21.15.12:0/3630754055 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd618043020 con 0x55c14e265a80 2024-04-22T13:24:13.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.596+0000 7fd614ff9640 1 -- 172.21.15.12:0/3630754055 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd618003e20 con 0x55c14e265a80 2024-04-22T13:24:13.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.596+0000 7fd614ff9640 1 -- 172.21.15.12:0/3630754055 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd618039040 con 0x55c14e265a80 2024-04-22T13:24:13.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.597+0000 7fd627c27740 1 -- 172.21.15.12:0/3630754055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14e265a80 msgr2=0x55c14e267e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:13.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.597+0000 7fd627c27740 1 --2- 172.21.15.12:0/3630754055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14e265a80 0x55c14e267e70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd618009ca0 tx=0x7fd618005fa0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.597+0000 7fd627c27740 1 -- 172.21.15.12:0/3630754055 shutdown_connections 2024-04-22T13:24:13.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.597+0000 7fd627c27740 1 --2- 172.21.15.12:0/3630754055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14e265a80 0x55c14e267e70 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.597+0000 7fd627c27740 1 --2- 172.21.15.12:0/3630754055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14e25cea0 0x55c14e265540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.597+0000 7fd627c27740 1 --2- 172.21.15.12:0/3630754055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14e25c580 0x55c14e25c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.597+0000 7fd627c27740 1 -- 172.21.15.12:0/3630754055 >> 172.21.15.12:0/3630754055 conn(0x55c14e256cb0 msgr2=0x55c14e257090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:13.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.597+0000 7fd627c27740 1 -- 172.21.15.12:0/3630754055 shutdown_connections 2024-04-22T13:24:13.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.597+0000 7fd627c27740 1 -- 172.21.15.12:0/3630754055 wait complete. 2024-04-22T13:24:13.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.597+0000 7fd627c27740 1 -- start start 2024-04-22T13:24:13.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.597+0000 7fd627c27740 1 Processor -- start 2024-04-22T13:24:13.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.598+0000 7fd627c27740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14e25c580 0x55c14e262a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.598+0000 7fd627c27740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14e25cea0 0x55c14e262f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.598+0000 7fd627c27740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14e265a80 0x55c14e2634d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.598+0000 7fd627c27740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c14e268810 con 0x55c14e25c580 2024-04-22T13:24:13.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.598+0000 7fd627c27740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c14e17a2f0 con 0x55c14e25cea0 2024-04-22T13:24:13.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.598+0000 7fd627c27740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c14e268990 con 0x55c14e265a80 2024-04-22T13:24:13.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.598+0000 7fd627c27740 10 client.0 _ll_register_callbacks cb 0x55c14e187580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:13.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.598+0000 7fd61e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14e25c580 0x55c14e262a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.598+0000 7fd61e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14e25c580 0x55c14e262a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59310/0 (socket says 172.21.15.12:59310) 2024-04-22T13:24:13.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.598+0000 7fd61e59c640 1 -- 172.21.15.12:0/2159437525 learned_addr learned my addr 172.21.15.12:0/2159437525 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:13.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.598+0000 7fd617fff640 1 --2- 172.21.15.12:0/2159437525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14e25cea0 0x55c14e262f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.598+0000 7fd61ed9d640 1 --2- 172.21.15.12:0/2159437525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14e265a80 0x55c14e2634d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.598+0000 7fd61e59c640 1 -- 172.21.15.12:0/2159437525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14e265a80 msgr2=0x55c14e2634d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:13.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.598+0000 7fd61e59c640 1 --2- 172.21.15.12:0/2159437525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14e265a80 0x55c14e2634d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.598+0000 7fd61e59c640 1 -- 172.21.15.12:0/2159437525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14e25cea0 msgr2=0x55c14e262f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:13.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.598+0000 7fd61e59c640 1 --2- 172.21.15.12:0/2159437525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14e25cea0 0x55c14e262f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.598+0000 7fd61e59c640 1 -- 172.21.15.12:0/2159437525 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c14ded33f0 con 0x55c14e25c580 2024-04-22T13:24:13.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.598+0000 7fd617fff640 1 --2- 172.21.15.12:0/2159437525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14e25cea0 0x55c14e262f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:13.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.598+0000 7fd61ed9d640 1 --2- 172.21.15.12:0/2159437525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14e265a80 0x55c14e2634d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:13.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.598+0000 7fd61e59c640 1 --2- 172.21.15.12:0/2159437525 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14e25c580 0x55c14e262a50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd61001ed10 tx=0x7fd610004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=197e6fe191cd177c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:13.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.598+0000 7fd605ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:13.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.599+0000 7fd605ffb640 1 -- 172.21.15.12:0/2159437525 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61000e930 con 0x55c14e25c580 2024-04-22T13:24:13.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.599+0000 7fd605ffb640 1 -- 172.21.15.12:0/2159437525 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fd61000ead0 con 0x55c14e25c580 2024-04-22T13:24:13.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.599+0000 7fd627c27740 1 -- 172.21.15.12:0/2159437525 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd618003e20 con 0x55c14e25c580 2024-04-22T13:24:13.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.599+0000 7fd605ffb640 1 -- 172.21.15.12:0/2159437525 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd61000ecd0 con 0x55c14e25c580 2024-04-22T13:24:13.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.599+0000 7fd627c27740 10 client.15579 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:13.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.599+0000 7fd627c27740 1 -- 172.21.15.12:0/2159437525 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c14de930f0 con 0x55c14e25c580 2024-04-22T13:24:13.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.599+0000 7fd605ffb640 1 -- 172.21.15.12:0/2159437525 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd618003e20 con 0x55c14e25c580 2024-04-22T13:24:13.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.599+0000 7fd605ffb640 1 -- 172.21.15.12:0/2159437525 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd610025020 con 0x55c14e25c580 2024-04-22T13:24:13.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.599+0000 7fd627c27740 10 client.15579 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:13.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.599+0000 7fd627c27740 10 client.15579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:24:13.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.599+0000 7fd627c27740 20 client.15579 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:13.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.599+0000 7fd627c27740 1 --2- 172.21.15.12:0/2159437525 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c14e275990 0x55c14e277e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.599+0000 7fd627c27740 4 client.15579 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:24:13.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.599+0000 7fd627c27740 1 -- 172.21.15.12:0/2159437525 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c14e1cec00 con 0x55c14e275990 2024-04-22T13:24:13.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.599+0000 7fd617fff640 1 --2- 172.21.15.12:0/2159437525 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c14e275990 0x55c14e277e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.600+0000 7fd617fff640 1 --2- 172.21.15.12:0/2159437525 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c14e275990 0x55c14e277e10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7024bb33366c395 server_cookie=a1f157a0cf9d2932 in_seq=0 out_seq=0 2024-04-22T13:24:13.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.600+0000 7fd605ffb640 10 client.15579 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:13.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.600+0000 7fd605ffb640 1 -- 172.21.15.12:0/2159437525 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c14e1cec00 con 0x55c14e275990 2024-04-22T13:24:13.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.600+0000 7fd605ffb640 10 client.15579 handle_command_reply: tid=0 2024-04-22T13:24:13.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.600+0000 7fd627c27740 1 client.15579 shutdown 2024-04-22T13:24:13.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.601+0000 7fd627c27740 1 -- 172.21.15.12:0/2159437525 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14e25c580 msgr2=0x55c14e262a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:13.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.601+0000 7fd627c27740 1 --2- 172.21.15.12:0/2159437525 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14e25c580 0x55c14e262a50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd61001ed10 tx=0x7fd610004730 comp rx=0 tx=0).stop 2024-04-22T13:24:13.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.601+0000 7fd627c27740 1 -- 172.21.15.12:0/2159437525 shutdown_connections 2024-04-22T13:24:13.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.601+0000 7fd627c27740 1 --2- 172.21.15.12:0/2159437525 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c14e275990 0x55c14e277e10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.601+0000 7fd627c27740 1 --2- 172.21.15.12:0/2159437525 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14e265a80 0x55c14e2634d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.601+0000 7fd627c27740 1 --2- 172.21.15.12:0/2159437525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14e25cea0 0x55c14e262f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.601+0000 7fd605ffb640 0 client.15579 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:13.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.601+0000 7fd627c27740 1 --2- 172.21.15.12:0/2159437525 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14e25c580 0x55c14e262a50 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.601+0000 7fd627c27740 1 -- 172.21.15.12:0/2159437525 >> 172.21.15.12:0/2159437525 conn(0x55c14e256cb0 msgr2=0x55c14e258a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:13.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.601+0000 7fd627c27740 1 -- 172.21.15.12:0/2159437525 shutdown_connections 2024-04-22T13:24:13.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.601+0000 7fd627c27740 1 -- 172.21.15.12:0/2159437525 wait complete. 2024-04-22T13:24:13.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.601+0000 7fd627c27740 20 client.15579 trim_cache size 0 max 16384 2024-04-22T13:24:13.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.618+0000 7fd627c27740 1 Processor -- start 2024-04-22T13:24:13.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.618+0000 7fd627c27740 1 -- start start 2024-04-22T13:24:13.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.619+0000 7fd627c27740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14e25c580 0x55c14e265180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.619+0000 7fd627c27740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14e25cea0 0x55c14e271700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.619+0000 7fd627c27740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14e265a80 0x55c14e27d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.619+0000 7fd627c27740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd6201ada70 con 0x55c14e265a80 2024-04-22T13:24:13.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.619+0000 7fd627c27740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd6201af5b0 con 0x55c14e25c580 2024-04-22T13:24:13.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.619+0000 7fd627c27740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd6201ae140 con 0x55c14e25cea0 2024-04-22T13:24:13.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.619+0000 7fd61ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14e265a80 0x55c14e27d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.619+0000 7fd61ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14e265a80 0x55c14e27d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59314/0 (socket says 172.21.15.12:59314) 2024-04-22T13:24:13.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.619+0000 7fd61ed9d640 1 -- 172.21.15.12:0/1058468143 learned_addr learned my addr 172.21.15.12:0/1058468143 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:13.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.619+0000 7fd617fff640 1 --2- 172.21.15.12:0/1058468143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14e25cea0 0x55c14e271700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.619+0000 7fd61ed9d640 1 -- 172.21.15.12:0/1058468143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14e25cea0 msgr2=0x55c14e271700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:13.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.619+0000 7fd61ed9d640 1 --2- 172.21.15.12:0/1058468143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14e25cea0 0x55c14e271700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.619+0000 7fd61ed9d640 1 -- 172.21.15.12:0/1058468143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14e25c580 msgr2=0x55c14e265180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:13.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.619+0000 7fd61e59c640 1 --2- 172.21.15.12:0/1058468143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14e25c580 0x55c14e265180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.619+0000 7fd61ed9d640 1 --2- 172.21.15.12:0/1058468143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14e25c580 0x55c14e265180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.619+0000 7fd61ed9d640 1 -- 172.21.15.12:0/1058468143 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c14e256ae0 con 0x55c14e265a80 2024-04-22T13:24:13.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.619+0000 7fd61e59c640 1 --2- 172.21.15.12:0/1058468143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14e25c580 0x55c14e265180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:13.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.619+0000 7fd61ed9d640 1 --2- 172.21.15.12:0/1058468143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14e265a80 0x55c14e27d6e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd618009bf0 tx=0x7fd6180095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9976d69a2c6b0cc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:13.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.619+0000 7fd607fff640 1 -- 172.21.15.12:0/1058468143 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61804d070 con 0x55c14e265a80 2024-04-22T13:24:13.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.619+0000 7fd607fff640 1 -- 172.21.15.12:0/1058468143 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd618003e20 con 0x55c14e265a80 2024-04-22T13:24:13.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.619+0000 7fd607fff640 1 -- 172.21.15.12:0/1058468143 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd618043070 con 0x55c14e265a80 2024-04-22T13:24:13.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.620+0000 7fd627c27740 1 -- 172.21.15.12:0/1058468143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14e265a80 msgr2=0x55c14e27d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:13.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.620+0000 7fd627c27740 1 --2- 172.21.15.12:0/1058468143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14e265a80 0x55c14e27d6e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd618009bf0 tx=0x7fd6180095a0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.620+0000 7fd627c27740 1 -- 172.21.15.12:0/1058468143 shutdown_connections 2024-04-22T13:24:13.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.620+0000 7fd627c27740 1 --2- 172.21.15.12:0/1058468143 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14e265a80 0x55c14e27d6e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.620+0000 7fd627c27740 1 --2- 172.21.15.12:0/1058468143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14e25cea0 0x55c14e271700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.620+0000 7fd627c27740 1 --2- 172.21.15.12:0/1058468143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14e25c580 0x55c14e265180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.620+0000 7fd627c27740 1 -- 172.21.15.12:0/1058468143 >> 172.21.15.12:0/1058468143 conn(0x55c14e256cb0 msgr2=0x55c14e25dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:13.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.620+0000 7fd627c27740 1 -- 172.21.15.12:0/1058468143 shutdown_connections 2024-04-22T13:24:13.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.620+0000 7fd627c27740 1 -- 172.21.15.12:0/1058468143 wait complete. 2024-04-22T13:24:13.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.621+0000 7fd627c27740 1 -- start start 2024-04-22T13:24:13.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.621+0000 7fd627c27740 1 Processor -- start 2024-04-22T13:24:13.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.621+0000 7fd627c27740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14e25c580 0x55c14e26dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.621+0000 7fd627c27740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14e25cea0 0x55c14e269320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.621+0000 7fd627c27740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14e265a80 0x55c14e269860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.621+0000 7fd627c27740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd6201aeee0 con 0x55c14e25c580 2024-04-22T13:24:13.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.621+0000 7fd627c27740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd6201ae810 con 0x55c14e265a80 2024-04-22T13:24:13.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.621+0000 7fd627c27740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd6201aba50 con 0x55c14e25cea0 2024-04-22T13:24:13.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.621+0000 7fd627c27740 10 client.0 _ll_register_callbacks cb 0x7fd610009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:13.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.621+0000 7fd61e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14e25c580 0x55c14e26dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.621+0000 7fd61e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14e25c580 0x55c14e26dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59330/0 (socket says 172.21.15.12:59330) 2024-04-22T13:24:13.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.621+0000 7fd61e59c640 1 -- 172.21.15.12:0/2096282400 learned_addr learned my addr 172.21.15.12:0/2096282400 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:13.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.621+0000 7fd617fff640 1 --2- 172.21.15.12:0/2096282400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14e25cea0 0x55c14e269320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.622+0000 7fd61ed9d640 1 --2- 172.21.15.12:0/2096282400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14e265a80 0x55c14e269860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.622+0000 7fd61e59c640 1 -- 172.21.15.12:0/2096282400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14e25cea0 msgr2=0x55c14e269320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:13.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.622+0000 7fd61e59c640 1 --2- 172.21.15.12:0/2096282400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14e25cea0 0x55c14e269320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.622+0000 7fd61e59c640 1 -- 172.21.15.12:0/2096282400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14e265a80 msgr2=0x55c14e269860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:13.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.622+0000 7fd61e59c640 1 --2- 172.21.15.12:0/2096282400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14e265a80 0x55c14e269860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.622+0000 7fd61e59c640 1 -- 172.21.15.12:0/2096282400 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c14e26a090 con 0x55c14e25c580 2024-04-22T13:24:13.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.622+0000 7fd61ed9d640 1 --2- 172.21.15.12:0/2096282400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14e265a80 0x55c14e269860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:13.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.622+0000 7fd61e59c640 1 --2- 172.21.15.12:0/2096282400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14e25c580 0x55c14e26dfb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd61000ea30 tx=0x7fd61000ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72eee28676fa0c84 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:13.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.622+0000 7fd605ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:13.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.622+0000 7fd605ffb640 1 -- 172.21.15.12:0/2096282400 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd610005b90 con 0x55c14e25c580 2024-04-22T13:24:13.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.622+0000 7fd605ffb640 1 -- 172.21.15.12:0/2096282400 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fd610009040 con 0x55c14e25c580 2024-04-22T13:24:13.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.622+0000 7fd627c27740 1 -- 172.21.15.12:0/2096282400 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd618003e20 con 0x55c14e25c580 2024-04-22T13:24:13.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.622+0000 7fd605ffb640 1 -- 172.21.15.12:0/2096282400 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd61001e820 con 0x55c14e25c580 2024-04-22T13:24:13.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.622+0000 7fd627c27740 10 client.15585 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:13.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.622+0000 7fd627c27740 1 -- 172.21.15.12:0/2096282400 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c14e2dbf20 con 0x55c14e25c580 2024-04-22T13:24:13.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.622+0000 7fd605ffb640 1 -- 172.21.15.12:0/2096282400 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd618003e20 con 0x55c14e25c580 2024-04-22T13:24:13.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.623+0000 7fd605ffb640 1 -- 172.21.15.12:0/2096282400 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd610025040 con 0x55c14e25c580 2024-04-22T13:24:13.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.623+0000 7fd627c27740 10 client.15585 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:13.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.623+0000 7fd627c27740 10 client.15585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:24:13.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.623+0000 7fd627c27740 20 client.15585 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:13.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.623+0000 7fd627c27740 1 --2- 172.21.15.12:0/2096282400 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c14e275990 0x55c14e2de6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.623+0000 7fd627c27740 4 client.15585 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:24:13.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.623+0000 7fd627c27740 1 -- 172.21.15.12:0/2096282400 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c14e1c8a70 con 0x55c14e275990 2024-04-22T13:24:13.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.623+0000 7fd617fff640 1 --2- 172.21.15.12:0/2096282400 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c14e275990 0x55c14e2de6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.623+0000 7fd605ffb640 10 client.15585 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:13.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.623+0000 7fd617fff640 1 --2- 172.21.15.12:0/2096282400 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c14e275990 0x55c14e2de6e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97ffdfdcdf873457 server_cookie=3d6ab3e55ae73323 in_seq=0 out_seq=0 2024-04-22T13:24:13.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.623+0000 7fd605ffb640 1 -- 172.21.15.12:0/2096282400 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c14e1c8a70 con 0x55c14e275990 2024-04-22T13:24:13.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.623+0000 7fd605ffb640 10 client.15585 handle_command_reply: tid=0 2024-04-22T13:24:13.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.623+0000 7fd627c27740 1 client.15585 shutdown 2024-04-22T13:24:13.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.624+0000 7fd627c27740 1 -- 172.21.15.12:0/2096282400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14e25c580 msgr2=0x55c14e26dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:13.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.624+0000 7fd627c27740 1 --2- 172.21.15.12:0/2096282400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14e25c580 0x55c14e26dfb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd61000ea30 tx=0x7fd61000ea60 comp rx=0 tx=0).stop 2024-04-22T13:24:13.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.624+0000 7fd627c27740 1 -- 172.21.15.12:0/2096282400 shutdown_connections 2024-04-22T13:24:13.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.624+0000 7fd627c27740 1 --2- 172.21.15.12:0/2096282400 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c14e275990 0x55c14e2de6e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.624+0000 7fd627c27740 1 --2- 172.21.15.12:0/2096282400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14e265a80 0x55c14e269860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.624+0000 7fd627c27740 1 --2- 172.21.15.12:0/2096282400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14e25cea0 0x55c14e269320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.624+0000 7fd605ffb640 0 client.15585 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:13.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.624+0000 7fd627c27740 1 --2- 172.21.15.12:0/2096282400 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14e25c580 0x55c14e26dfb0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.624+0000 7fd627c27740 1 -- 172.21.15.12:0/2096282400 >> 172.21.15.12:0/2096282400 conn(0x55c14e256cb0 msgr2=0x55c14e257ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:13.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.624+0000 7fd627c27740 1 -- 172.21.15.12:0/2096282400 shutdown_connections 2024-04-22T13:24:13.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.624+0000 7fd627c27740 1 -- 172.21.15.12:0/2096282400 wait complete. 2024-04-22T13:24:13.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.624+0000 7fd627c27740 20 client.15585 trim_cache size 0 max 16384 2024-04-22T13:24:13.629 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:24:13.629 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:24:13.629 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-04-22T13:24:13.629 INFO:teuthology.orchestra.run.smithi012.stdout: "aa5ba72d-11cb-489a-91fe-1b2fb60be456": { 2024-04-22T13:24:13.629 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-04-22T13:24:13.630 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "aa5ba72d-11cb-489a-91fe-1b2fb60be456", 2024-04-22T13:24:13.630 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-04-22T13:24:13.630 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:24:13.630 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:24:13.630 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:24:13.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.624+0000 7fd624f52640 1 -- 172.21.15.12:0/3020388294 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd60c0a3e40 msgr2=0x7fd60c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:13.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.624+0000 7fd624f52640 1 --2- 172.21.15.12:0/3020388294 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd60c0a3e40 0x7fd60c0a6300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd610009bd0 tx=0x7fd610005f90 comp rx=0 tx=0).stop 2024-04-22T13:24:13.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.624+0000 7fd624f52640 1 -- 172.21.15.12:0/3020388294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd620103ae0 msgr2=0x7fd620198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:13.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.624+0000 7fd624f52640 1 --2- 172.21.15.12:0/3020388294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd620103ae0 0x7fd620198bb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd60800d950 tx=0x7fd60800de10 comp rx=0 tx=0).stop 2024-04-22T13:24:13.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.624+0000 7fd624f52640 1 -- 172.21.15.12:0/3020388294 shutdown_connections 2024-04-22T13:24:13.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.624+0000 7fd624f52640 1 --2- 172.21.15.12:0/3020388294 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd60c0a3e40 0x7fd60c0a6300 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.624+0000 7fd624f52640 1 --2- 172.21.15.12:0/3020388294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd62010c500 0x7fd62019cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.624+0000 7fd624f52640 1 --2- 172.21.15.12:0/3020388294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd620103ae0 0x7fd620198bb0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.624+0000 7fd624f52640 1 --2- 172.21.15.12:0/3020388294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6201031c0 0x7fd620198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.624+0000 7fd624f52640 1 -- 172.21.15.12:0/3020388294 >> 172.21.15.12:0/3020388294 conn(0x7fd620078000 msgr2=0x7fd6200fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:13.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.625+0000 7fd624f52640 1 -- 172.21.15.12:0/3020388294 shutdown_connections 2024-04-22T13:24:13.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.625+0000 7fd624f52640 1 -- 172.21.15.12:0/3020388294 wait complete. 2024-04-22T13:24:13.632 INFO:tasks.cephfs.filesystem:scrub status for tag:aa5ba72d-11cb-489a-91fe-1b2fb60be456 - {'path': '/', 'tag': 'aa5ba72d-11cb-489a-91fe-1b2fb60be456', 'options': 'recursive,force'} 2024-04-22T13:24:13.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.663+0000 7f3d0fc9a640 1 -- 172.21.15.12:0/3392064645 --> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f3d08109b10 con 0x7f3cf40a3f60 2024-04-22T13:24:13.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.734+0000 7f3cfeffd640 1 -- 172.21.15.12:0/3392064645 <== mgr.24968 v2:172.21.15.12:6800/1615661596 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3d04010b70 con 0x7f3cf40a3f60 2024-04-22T13:24:13.731 INFO:teuthology.orchestra.run.smithi012.stdout:Schedule set for path / 2024-04-22T13:24:13.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.736+0000 7f3d0fc9a640 1 -- 172.21.15.12:0/3392064645 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3cf40a3f60 msgr2=0x7f3cf40a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:13.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.736+0000 7f3d0fc9a640 1 --2- 172.21.15.12:0/3392064645 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3cf40a3f60 0x7f3cf40a6420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3d04009b50 tx=0x7f3d04005f90 comp rx=0 tx=0).stop 2024-04-22T13:24:13.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.736+0000 7f3d0fc9a640 1 -- 172.21.15.12:0/3392064645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d0810e910 msgr2=0x7f3d0819a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:13.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.736+0000 7f3d0fc9a640 1 --2- 172.21.15.12:0/3392064645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d0810e910 0x7f3d0819a790 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3d0002c700 tx=0x7f3d00034670 comp rx=0 tx=0).stop 2024-04-22T13:24:13.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.736+0000 7f3d0fc9a640 1 -- 172.21.15.12:0/3392064645 shutdown_connections 2024-04-22T13:24:13.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.736+0000 7f3d0fc9a640 1 --2- 172.21.15.12:0/3392064645 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3cf40a3f60 0x7f3cf40a6420 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.736+0000 7f3d0fc9a640 1 --2- 172.21.15.12:0/3392064645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3d0810e910 0x7f3d0819a790 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.736+0000 7f3d0fc9a640 1 --2- 172.21.15.12:0/3392064645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d08101680 0x7f3d081a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.736+0000 7f3d0fc9a640 1 --2- 172.21.15.12:0/3392064645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d08100d60 0x7f3d081a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.736+0000 7f3d0fc9a640 1 -- 172.21.15.12:0/3392064645 >> 172.21.15.12:0/3392064645 conn(0x7f3d08077fd0 msgr2=0x7f3d0810fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:13.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.736+0000 7f3d0fc9a640 1 -- 172.21.15.12:0/3392064645 shutdown_connections 2024-04-22T13:24:13.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.736+0000 7f3d0fc9a640 1 -- 172.21.15.12:0/3392064645 wait complete. 2024-04-22T13:24:13.744 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-22T13:24:13.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.893+0000 7f4a7a106640 1 -- 172.21.15.12:0/1025019373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a74106fb0 msgr2=0x7f4a74107390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:13.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.893+0000 7f4a7a106640 1 --2- 172.21.15.12:0/1025019373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a74106fb0 0x7f4a74107390 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4a64009900 tx=0x7f4a6402bc80 comp rx=0 tx=0).stop 2024-04-22T13:24:13.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.893+0000 7f4a7a106640 1 -- 172.21.15.12:0/1025019373 shutdown_connections 2024-04-22T13:24:13.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.893+0000 7f4a7a106640 1 --2- 172.21.15.12:0/1025019373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a74101b60 0x7f4a7410dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.893+0000 7f4a7a106640 1 --2- 172.21.15.12:0/1025019373 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a741011a0 0x7f4a74101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.893+0000 7f4a7a106640 1 --2- 172.21.15.12:0/1025019373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a74106fb0 0x7f4a74107390 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.893+0000 7f4a7a106640 1 -- 172.21.15.12:0/1025019373 >> 172.21.15.12:0/1025019373 conn(0x7f4a74076c50 msgr2=0x7f4a74077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:13.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.893+0000 7f4a7a106640 1 -- 172.21.15.12:0/1025019373 shutdown_connections 2024-04-22T13:24:13.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.893+0000 7f4a7a106640 1 -- 172.21.15.12:0/1025019373 wait complete. 2024-04-22T13:24:13.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.894+0000 7f4a7a106640 1 Processor -- start 2024-04-22T13:24:13.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.894+0000 7f4a7a106640 1 -- start start 2024-04-22T13:24:13.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.894+0000 7f4a7a106640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a741011a0 0x7f4a74198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.894+0000 7f4a7a106640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a74101b60 0x7f4a74198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.894+0000 7f4a7a106640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a74106fb0 0x7f4a7419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.894+0000 7f4a7a106640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4a741110e0 con 0x7f4a741011a0 2024-04-22T13:24:13.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.894+0000 7f4a7a106640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4a74110f60 con 0x7f4a74106fb0 2024-04-22T13:24:13.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.894+0000 7f4a7a106640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4a74111260 con 0x7f4a74101b60 2024-04-22T13:24:13.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.894+0000 7f4a737fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a741011a0 0x7f4a74198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.894+0000 7f4a737fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a741011a0 0x7f4a74198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59342/0 (socket says 172.21.15.12:59342) 2024-04-22T13:24:13.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.894+0000 7f4a737fe640 1 -- 172.21.15.12:0/76799859 learned_addr learned my addr 172.21.15.12:0/76799859 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:13.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.894+0000 7f4a737fe640 1 -- 172.21.15.12:0/76799859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a74101b60 msgr2=0x7f4a74198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:13.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.894+0000 7f4a737fe640 1 --2- 172.21.15.12:0/76799859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a74101b60 0x7f4a74198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.894+0000 7f4a737fe640 1 -- 172.21.15.12:0/76799859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a74106fb0 msgr2=0x7f4a7419ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:13.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.894+0000 7f4a737fe640 1 --2- 172.21.15.12:0/76799859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a74106fb0 0x7f4a7419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:13.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.894+0000 7f4a737fe640 1 -- 172.21.15.12:0/76799859 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a7419d600 con 0x7f4a741011a0 2024-04-22T13:24:13.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.895+0000 7f4a737fe640 1 --2- 172.21.15.12:0/76799859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a741011a0 0x7f4a74198610 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4a64000c00 tx=0x7f4a6402c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:13.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.895+0000 7f4a70ff9640 1 -- 172.21.15.12:0/76799859 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a64043020 con 0x7f4a741011a0 2024-04-22T13:24:13.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.895+0000 7f4a70ff9640 1 -- 172.21.15.12:0/76799859 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a6402cba0 con 0x7f4a741011a0 2024-04-22T13:24:13.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.895+0000 7f4a7a106640 1 -- 172.21.15.12:0/76799859 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a7419d890 con 0x7f4a741011a0 2024-04-22T13:24:13.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.895+0000 7f4a70ff9640 1 -- 172.21.15.12:0/76799859 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a64003ac0 con 0x7f4a741011a0 2024-04-22T13:24:13.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.895+0000 7f4a7a106640 1 -- 172.21.15.12:0/76799859 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a741a5180 con 0x7f4a741011a0 2024-04-22T13:24:13.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.896+0000 7f4a7a106640 1 -- 172.21.15.12:0/76799859 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a74102550 con 0x7f4a741011a0 2024-04-22T13:24:13.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.897+0000 7f4a70ff9640 1 -- 172.21.15.12:0/76799859 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f4a64046020 con 0x7f4a741011a0 2024-04-22T13:24:13.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.898+0000 7f4a70ff9640 1 --2- 172.21.15.12:0/76799859 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4a600a3ee0 0x7f4a600a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:13.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.898+0000 7f4a70ff9640 1 -- 172.21.15.12:0/76799859 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f4a640f2e90 con 0x7f4a741011a0 2024-04-22T13:24:13.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.898+0000 7f4a72ffd640 1 --2- 172.21.15.12:0/76799859 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4a600a3ee0 0x7f4a600a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:13.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.898+0000 7f4a72ffd640 1 --2- 172.21.15.12:0/76799859 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4a600a3ee0 0x7f4a600a63a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c005fd0 tx=0x7f4a5c005850 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:13.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:13.901+0000 7f4a70ff9640 1 -- 172.21.15.12:0/76799859 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4a6404d020 con 0x7f4a741011a0 2024-04-22T13:24:13.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:13 smithi111 ceph-mon[31361]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-22T13:24:13.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:13 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1597149249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:13.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:13 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1381871018' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:24:14.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:13 smithi012 ceph-mon[28265]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-22T13:24:14.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:13 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1597149249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:14.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:13 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1381871018' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:24:14.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.114+0000 7f4a7a106640 1 -- 172.21.15.12:0/76799859 --> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f4a74104c50 con 0x7f4a600a3ee0 2024-04-22T13:24:14.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.177+0000 7f4a70ff9640 1 -- 172.21.15.12:0/76799859 <== mgr.24968 v2:172.21.15.12:6800/1615661596 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f4a74104c50 con 0x7f4a600a3ee0 2024-04-22T13:24:14.174 INFO:teuthology.orchestra.run.smithi012.stdout:Retention added to path / 2024-04-22T13:24:14.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.179+0000 7f4a7a106640 1 -- 172.21.15.12:0/76799859 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4a600a3ee0 msgr2=0x7f4a600a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:14.176 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.179+0000 7f4a7a106640 1 --2- 172.21.15.12:0/76799859 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4a600a3ee0 0x7f4a600a63a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c005fd0 tx=0x7f4a5c005850 comp rx=0 tx=0).stop 2024-04-22T13:24:14.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.179+0000 7f4a7a106640 1 -- 172.21.15.12:0/76799859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a741011a0 msgr2=0x7f4a74198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:14.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.179+0000 7f4a7a106640 1 --2- 172.21.15.12:0/76799859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a741011a0 0x7f4a74198610 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4a64000c00 tx=0x7f4a6402c6b0 comp rx=0 tx=0).stop 2024-04-22T13:24:14.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.179+0000 7f4a7a106640 1 -- 172.21.15.12:0/76799859 shutdown_connections 2024-04-22T13:24:14.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.179+0000 7f4a7a106640 1 --2- 172.21.15.12:0/76799859 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4a600a3ee0 0x7f4a600a63a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:14.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.179+0000 7f4a7a106640 1 --2- 172.21.15.12:0/76799859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a74106fb0 0x7f4a7419ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:14.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.179+0000 7f4a7a106640 1 --2- 172.21.15.12:0/76799859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a74101b60 0x7f4a74198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:14.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.179+0000 7f4a7a106640 1 --2- 172.21.15.12:0/76799859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4a741011a0 0x7f4a74198610 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:14.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.179+0000 7f4a7a106640 1 -- 172.21.15.12:0/76799859 >> 172.21.15.12:0/76799859 conn(0x7f4a74076c50 msgr2=0x7f4a74100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:14.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.179+0000 7f4a7a106640 1 -- 172.21.15.12:0/76799859 shutdown_connections 2024-04-22T13:24:14.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.179+0000 7f4a7a106640 1 -- 172.21.15.12:0/76799859 wait complete. 2024-04-22T13:24:14.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:13 smithi134 ceph-mon[31338]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-22T13:24:14.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:13 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1597149249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:14.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:13 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1381871018' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:24:14.187 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-22T13:24:14.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.292+0000 7fe09d1e6640 1 -- 172.21.15.12:0/1236220873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0981061b0 msgr2=0x7fe09810dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:14.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.292+0000 7fe09d1e6640 1 --2- 172.21.15.12:0/1236220873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0981061b0 0x7fe09810dd80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe090009ca0 tx=0x7fe09002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:24:14.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.292+0000 7fe09d1e6640 1 -- 172.21.15.12:0/1236220873 shutdown_connections 2024-04-22T13:24:14.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.292+0000 7fe09d1e6640 1 --2- 172.21.15.12:0/1236220873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe0981061b0 0x7fe09810dd80 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:14.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.292+0000 7fe09d1e6640 1 --2- 172.21.15.12:0/1236220873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe098101a90 0x7fe098105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:14.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.292+0000 7fe09d1e6640 1 --2- 172.21.15.12:0/1236220873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0981010e0 0x7fe0981014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:14.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.292+0000 7fe09d1e6640 1 -- 172.21.15.12:0/1236220873 >> 172.21.15.12:0/1236220873 conn(0x7fe0980771d0 msgr2=0x7fe0980775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:14.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.292+0000 7fe09d1e6640 1 -- 172.21.15.12:0/1236220873 shutdown_connections 2024-04-22T13:24:14.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.292+0000 7fe09d1e6640 1 -- 172.21.15.12:0/1236220873 wait complete. 2024-04-22T13:24:14.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.293+0000 7fe09d1e6640 1 Processor -- start 2024-04-22T13:24:14.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.293+0000 7fe09d1e6640 1 -- start start 2024-04-22T13:24:14.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.293+0000 7fe09d1e6640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0981010e0 0x7fe0981986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:14.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.293+0000 7fe09d1e6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0981061b0 0x7fe098198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:14.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.294+0000 7fe09d1e6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe09819cf20 0x7fe09819d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:14.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.294+0000 7fe09d1e6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe098112230 con 0x7fe09819cf20 2024-04-22T13:24:14.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.294+0000 7fe09d1e6640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981120b0 con 0x7fe0981010e0 2024-04-22T13:24:14.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.294+0000 7fe09d1e6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981123b0 con 0x7fe0981061b0 2024-04-22T13:24:14.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.294+0000 7fe09759e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe09819cf20 0x7fe09819d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:14.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.294+0000 7fe09759e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe09819cf20 0x7fe09819d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59368/0 (socket says 172.21.15.12:59368) 2024-04-22T13:24:14.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.294+0000 7fe09759e640 1 -- 172.21.15.12:0/3154472538 learned_addr learned my addr 172.21.15.12:0/3154472538 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:14.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.294+0000 7fe09659c640 1 --2- 172.21.15.12:0/3154472538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0981061b0 0x7fe098198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:14.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.294+0000 7fe096d9d640 1 --2- 172.21.15.12:0/3154472538 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0981010e0 0x7fe0981986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:14.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.294+0000 7fe09759e640 1 -- 172.21.15.12:0/3154472538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0981061b0 msgr2=0x7fe098198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:14.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.294+0000 7fe09759e640 1 --2- 172.21.15.12:0/3154472538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0981061b0 0x7fe098198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:14.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.294+0000 7fe09759e640 1 -- 172.21.15.12:0/3154472538 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0981010e0 msgr2=0x7fe0981986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:14.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.294+0000 7fe09759e640 1 --2- 172.21.15.12:0/3154472538 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0981010e0 0x7fe0981986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:14.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.294+0000 7fe09759e640 1 -- 172.21.15.12:0/3154472538 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe09819da10 con 0x7fe09819cf20 2024-04-22T13:24:14.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.294+0000 7fe096d9d640 1 --2- 172.21.15.12:0/3154472538 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0981010e0 0x7fe0981986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:14.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.294+0000 7fe09759e640 1 --2- 172.21.15.12:0/3154472538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe09819cf20 0x7fe09819d3d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe090005e30 tx=0x7fe090034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:14.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.294+0000 7fe07ffff640 1 -- 172.21.15.12:0/3154472538 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe090039040 con 0x7fe09819cf20 2024-04-22T13:24:14.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.294+0000 7fe09d1e6640 1 -- 172.21.15.12:0/3154472538 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe09819dca0 con 0x7fe09819cf20 2024-04-22T13:24:14.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.294+0000 7fe07ffff640 1 -- 172.21.15.12:0/3154472538 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe090044070 con 0x7fe09819cf20 2024-04-22T13:24:14.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.294+0000 7fe07ffff640 1 -- 172.21.15.12:0/3154472538 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe090003e60 con 0x7fe09819cf20 2024-04-22T13:24:14.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.294+0000 7fe09d1e6640 1 -- 172.21.15.12:0/3154472538 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0981a54c0 con 0x7fe09819cf20 2024-04-22T13:24:14.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.295+0000 7fe09d1e6640 1 -- 172.21.15.12:0/3154472538 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe09804fce0 con 0x7fe09819cf20 2024-04-22T13:24:14.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.297+0000 7fe07ffff640 1 -- 172.21.15.12:0/3154472538 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fe0900324c0 con 0x7fe09819cf20 2024-04-22T13:24:14.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.298+0000 7fe07ffff640 1 --2- 172.21.15.12:0/3154472538 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe0800a3ee0 0x7fe0800a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:14.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.298+0000 7fe07ffff640 1 -- 172.21.15.12:0/3154472538 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fe0900f3660 con 0x7fe09819cf20 2024-04-22T13:24:14.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.298+0000 7fe096d9d640 1 --2- 172.21.15.12:0/3154472538 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe0800a3ee0 0x7fe0800a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:14.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.298+0000 7fe096d9d640 1 --2- 172.21.15.12:0/3154472538 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe0800a3ee0 0x7fe0800a63a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe088009b40 tx=0x7fe088005e50 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:14.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.301+0000 7fe07ffff640 1 -- 172.21.15.12:0/3154472538 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe0900bfc10 con 0x7fe09819cf20 2024-04-22T13:24:14.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.510+0000 7fe09d1e6640 1 -- 172.21.15.12:0/3154472538 --> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fe098101b30 con 0x7fe0800a3ee0 2024-04-22T13:24:14.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.518+0000 7fe07ffff640 1 -- 172.21.15.12:0/3154472538 <== mgr.24968 v2:172.21.15.12:6800/1615661596 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fe098101b30 con 0x7fe0800a3ee0 2024-04-22T13:24:14.515 INFO:teuthology.orchestra.run.smithi012.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-22T00:00:00", "created": "2024-04-22T13:24:13", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-22T13:24:14.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.520+0000 7fe09d1e6640 1 -- 172.21.15.12:0/3154472538 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe0800a3ee0 msgr2=0x7fe0800a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.520+0000 7fe09d1e6640 1 --2- 172.21.15.12:0/3154472538 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe0800a3ee0 0x7fe0800a63a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe088009b40 tx=0x7fe088005e50 comp rx=0 tx=0).stop 2024-04-22T13:24:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.520+0000 7fe09d1e6640 1 -- 172.21.15.12:0/3154472538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe09819cf20 msgr2=0x7fe09819d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.520+0000 7fe09d1e6640 1 --2- 172.21.15.12:0/3154472538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe09819cf20 0x7fe09819d3d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe090005e30 tx=0x7fe090034760 comp rx=0 tx=0).stop 2024-04-22T13:24:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.520+0000 7fe09d1e6640 1 -- 172.21.15.12:0/3154472538 shutdown_connections 2024-04-22T13:24:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.520+0000 7fe09d1e6640 1 --2- 172.21.15.12:0/3154472538 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe0800a3ee0 0x7fe0800a63a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.520+0000 7fe09d1e6640 1 --2- 172.21.15.12:0/3154472538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe09819cf20 0x7fe09819d3d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.520+0000 7fe09d1e6640 1 --2- 172.21.15.12:0/3154472538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0981061b0 0x7fe098198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.520+0000 7fe09d1e6640 1 --2- 172.21.15.12:0/3154472538 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0981010e0 0x7fe0981986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.520+0000 7fe09d1e6640 1 -- 172.21.15.12:0/3154472538 >> 172.21.15.12:0/3154472538 conn(0x7fe0980771d0 msgr2=0x7fe0980feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.520+0000 7fe09d1e6640 1 -- 172.21.15.12:0/3154472538 shutdown_connections 2024-04-22T13:24:14.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.520+0000 7fe09d1e6640 1 -- 172.21.15.12:0/3154472538 wait complete. 2024-04-22T13:24:14.528 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-22T13:24:14.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.634+0000 7f3f1cbca640 1 -- 172.21.15.12:0/1086728834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f18107290 msgr2=0x7f3f1810ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:14.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.634+0000 7f3f1cbca640 1 --2- 172.21.15.12:0/1086728834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f18107290 0x7f3f1810ee60 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3f10009ca0 tx=0x7f3f1002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:24:14.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.635+0000 7f3f1cbca640 1 -- 172.21.15.12:0/1086728834 shutdown_connections 2024-04-22T13:24:14.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.635+0000 7f3f1cbca640 1 --2- 172.21.15.12:0/1086728834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f18107290 0x7f3f1810ee60 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:14.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.635+0000 7f3f1cbca640 1 --2- 172.21.15.12:0/1086728834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f18102b70 0x7f3f18106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:14.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.635+0000 7f3f1cbca640 1 --2- 172.21.15.12:0/1086728834 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f181021c0 0x7f3f181025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:14.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.635+0000 7f3f1cbca640 1 -- 172.21.15.12:0/1086728834 >> 172.21.15.12:0/1086728834 conn(0x7f3f180fda20 msgr2=0x7f3f180ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:14.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.635+0000 7f3f1cbca640 1 -- 172.21.15.12:0/1086728834 shutdown_connections 2024-04-22T13:24:14.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.635+0000 7f3f1cbca640 1 -- 172.21.15.12:0/1086728834 wait complete. 2024-04-22T13:24:14.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.635+0000 7f3f1cbca640 1 Processor -- start 2024-04-22T13:24:14.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.635+0000 7f3f1cbca640 1 -- start start 2024-04-22T13:24:14.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.635+0000 7f3f1cbca640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f181021c0 0x7f3f18198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:14.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.635+0000 7f3f1cbca640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f18107290 0x7f3f18198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:14.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.635+0000 7f3f1cbca640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f1819cf90 0x7f3f1819d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:14.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.636+0000 7f3f1cbca640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3f18112200 con 0x7f3f1819cf90 2024-04-22T13:24:14.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.636+0000 7f3f1cbca640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3f18112080 con 0x7f3f18107290 2024-04-22T13:24:14.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.636+0000 7f3f1cbca640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3f18112380 con 0x7f3f181021c0 2024-04-22T13:24:14.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.636+0000 7f3f16d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f1819cf90 0x7f3f1819d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:14.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.636+0000 7f3f16d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f1819cf90 0x7f3f1819d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59380/0 (socket says 172.21.15.12:59380) 2024-04-22T13:24:14.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.636+0000 7f3f16d9d640 1 -- 172.21.15.12:0/423375454 learned_addr learned my addr 172.21.15.12:0/423375454 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:14.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.636+0000 7f3f15d9b640 1 --2- 172.21.15.12:0/423375454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f18107290 0x7f3f18198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:14.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.636+0000 7f3f16d9d640 1 -- 172.21.15.12:0/423375454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f181021c0 msgr2=0x7f3f18198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:14.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.636+0000 7f3f1659c640 1 --2- 172.21.15.12:0/423375454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f181021c0 0x7f3f18198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:14.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.636+0000 7f3f16d9d640 1 --2- 172.21.15.12:0/423375454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f181021c0 0x7f3f18198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:14.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.636+0000 7f3f16d9d640 1 -- 172.21.15.12:0/423375454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f18107290 msgr2=0x7f3f18198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:14.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.636+0000 7f3f16d9d640 1 --2- 172.21.15.12:0/423375454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f18107290 0x7f3f18198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:14.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.636+0000 7f3f16d9d640 1 -- 172.21.15.12:0/423375454 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f1819da80 con 0x7f3f1819cf90 2024-04-22T13:24:14.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.636+0000 7f3f15d9b640 1 --2- 172.21.15.12:0/423375454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f18107290 0x7f3f18198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:14.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.636+0000 7f3f1659c640 1 --2- 172.21.15.12:0/423375454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f181021c0 0x7f3f18198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:14.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.636+0000 7f3f16d9d640 1 --2- 172.21.15.12:0/423375454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f1819cf90 0x7f3f1819d440 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3f10005e30 tx=0x7f3f10034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:14.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.636+0000 7f3f077fe640 1 -- 172.21.15.12:0/423375454 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f10039040 con 0x7f3f1819cf90 2024-04-22T13:24:14.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.636+0000 7f3f1cbca640 1 -- 172.21.15.12:0/423375454 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f1819dd10 con 0x7f3f1819cf90 2024-04-22T13:24:14.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.636+0000 7f3f077fe640 1 -- 172.21.15.12:0/423375454 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f10044070 con 0x7f3f1819cf90 2024-04-22T13:24:14.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.636+0000 7f3f077fe640 1 -- 172.21.15.12:0/423375454 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f10003e60 con 0x7f3f1819cf90 2024-04-22T13:24:14.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.636+0000 7f3f1cbca640 1 -- 172.21.15.12:0/423375454 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f181a5530 con 0x7f3f1819cf90 2024-04-22T13:24:14.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.637+0000 7f3f1cbca640 1 -- 172.21.15.12:0/423375454 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f1804fce0 con 0x7f3f1819cf90 2024-04-22T13:24:14.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.639+0000 7f3f077fe640 1 -- 172.21.15.12:0/423375454 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f3f100324c0 con 0x7f3f1819cf90 2024-04-22T13:24:14.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.639+0000 7f3f077fe640 1 --2- 172.21.15.12:0/423375454 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3f000a3ee0 0x7f3f000a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:14.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.639+0000 7f3f1659c640 1 --2- 172.21.15.12:0/423375454 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3f000a3ee0 0x7f3f000a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:14.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.639+0000 7f3f077fe640 1 -- 172.21.15.12:0/423375454 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f3f100f3730 con 0x7f3f1819cf90 2024-04-22T13:24:14.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.640+0000 7f3f1659c640 1 --2- 172.21.15.12:0/423375454 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3f000a3ee0 0x7f3f000a63a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3f080058c0 tx=0x7f3f08005d30 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:14.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.643+0000 7f3f077fe640 1 -- 172.21.15.12:0/423375454 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3f100bfce0 con 0x7f3f1819cf90 2024-04-22T13:24:14.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:14 smithi012 ceph-mon[28265]: scrub queued for path: / 2024-04-22T13:24:14.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:14 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:24:14.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:14 smithi012 ceph-mon[28265]: scrub summary: active paths [/] 2024-04-22T13:24:14.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:14 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:24:14.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:14 smithi012 ceph-mon[28265]: from='client.15552 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:24:14.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.852+0000 7f3f1cbca640 1 -- 172.21.15.12:0/423375454 --> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f3f1810a4f0 con 0x7f3f000a3ee0 2024-04-22T13:24:14.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.860+0000 7f3f077fe640 1 -- 172.21.15.12:0/423375454 <== mgr.24968 v2:172.21.15.12:6800/1615661596 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f3f1810a4f0 con 0x7f3f000a3ee0 2024-04-22T13:24:14.857 INFO:teuthology.orchestra.run.smithi012.stdout:/ 1m 6m3h 2024-04-22T13:24:14.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.862+0000 7f3f1cbca640 1 -- 172.21.15.12:0/423375454 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3f000a3ee0 msgr2=0x7f3f000a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:14.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.862+0000 7f3f1cbca640 1 --2- 172.21.15.12:0/423375454 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3f000a3ee0 0x7f3f000a63a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3f080058c0 tx=0x7f3f08005d30 comp rx=0 tx=0).stop 2024-04-22T13:24:14.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.862+0000 7f3f1cbca640 1 -- 172.21.15.12:0/423375454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f1819cf90 msgr2=0x7f3f1819d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:14.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.862+0000 7f3f1cbca640 1 --2- 172.21.15.12:0/423375454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f1819cf90 0x7f3f1819d440 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3f10005e30 tx=0x7f3f10034760 comp rx=0 tx=0).stop 2024-04-22T13:24:14.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.862+0000 7f3f1cbca640 1 -- 172.21.15.12:0/423375454 shutdown_connections 2024-04-22T13:24:14.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.862+0000 7f3f1cbca640 1 --2- 172.21.15.12:0/423375454 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3f000a3ee0 0x7f3f000a63a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:14.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.862+0000 7f3f1cbca640 1 --2- 172.21.15.12:0/423375454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f1819cf90 0x7f3f1819d440 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:14.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.862+0000 7f3f1cbca640 1 --2- 172.21.15.12:0/423375454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f18107290 0x7f3f18198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:14.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.862+0000 7f3f1cbca640 1 --2- 172.21.15.12:0/423375454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f181021c0 0x7f3f18198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:14.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.862+0000 7f3f1cbca640 1 -- 172.21.15.12:0/423375454 >> 172.21.15.12:0/423375454 conn(0x7f3f180fda20 msgr2=0x7f3f180fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:14.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.862+0000 7f3f1cbca640 1 -- 172.21.15.12:0/423375454 shutdown_connections 2024-04-22T13:24:14.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:14.862+0000 7f3f1cbca640 1 -- 172.21.15.12:0/423375454 wait complete. 2024-04-22T13:24:14.870 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-22T13:24:14.896 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-22T13:24:14.907 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-22T13:24:14.918 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-04-22T13:24:14.918 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-22T13:24:14.938 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-22T13:24:14.949 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:24:14.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:14 smithi111 ceph-mon[31361]: scrub queued for path: / 2024-04-22T13:24:14.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:14 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:24:14.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:14 smithi111 ceph-mon[31361]: scrub summary: active paths [/] 2024-04-22T13:24:14.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:14 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:24:14.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:14 smithi111 ceph-mon[31361]: from='client.15552 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:24:15.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.053+0000 7fd1b4f8e640 1 -- 172.21.15.12:0/2537086559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1b006b280 msgr2=0x7fd1b0106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:15.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.053+0000 7fd1b4f8e640 1 --2- 172.21.15.12:0/2537086559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1b006b280 0x7fd1b0106b80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4009900 tx=0x7fd1a402bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:15.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.054+0000 7fd1b4f8e640 1 -- 172.21.15.12:0/2537086559 shutdown_connections 2024-04-22T13:24:15.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.054+0000 7fd1b4f8e640 1 --2- 172.21.15.12:0/2537086559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1b0107260 0x7fd1b010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.054+0000 7fd1b4f8e640 1 --2- 172.21.15.12:0/2537086559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1b006b280 0x7fd1b0106b80 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.054+0000 7fd1b4f8e640 1 --2- 172.21.15.12:0/2537086559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1b006a8d0 0x7fd1b006acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.054+0000 7fd1b4f8e640 1 -- 172.21.15.12:0/2537086559 >> 172.21.15.12:0/2537086559 conn(0x7fd1b0077c20 msgr2=0x7fd1b0078030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:15.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.054+0000 7fd1b4f8e640 1 -- 172.21.15.12:0/2537086559 shutdown_connections 2024-04-22T13:24:15.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.054+0000 7fd1b4f8e640 1 -- 172.21.15.12:0/2537086559 wait complete. 2024-04-22T13:24:15.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.054+0000 7fd1b4f8e640 1 Processor -- start 2024-04-22T13:24:15.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.054+0000 7fd1b4f8e640 1 -- start start 2024-04-22T13:24:15.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.054+0000 7fd1b4f8e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1b006a8d0 0x7fd1b0198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:15.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.054+0000 7fd1b4f8e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1b0107260 0x7fd1b0198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:15.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.054+0000 7fd1b4f8e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1b019cfe0 0x7fd1b019d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:15.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.055+0000 7fd1b4f8e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b0112200 con 0x7fd1b006a8d0 2024-04-22T13:24:15.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.055+0000 7fd1b4f8e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b0112080 con 0x7fd1b0107260 2024-04-22T13:24:15.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.055+0000 7fd1b4f8e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b0112380 con 0x7fd1b019cfe0 2024-04-22T13:24:15.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.055+0000 7fd1ae59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1b006a8d0 0x7fd1b0198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:15.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.055+0000 7fd1ae59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1b006a8d0 0x7fd1b0198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59396/0 (socket says 172.21.15.12:59396) 2024-04-22T13:24:15.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.055+0000 7fd1ae59c640 1 -- 172.21.15.12:0/2251512047 learned_addr learned my addr 172.21.15.12:0/2251512047 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:15.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.055+0000 7fd1aed9d640 1 --2- 172.21.15.12:0/2251512047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1b019cfe0 0x7fd1b019d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:15.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.055+0000 7fd1add9b640 1 --2- 172.21.15.12:0/2251512047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1b0107260 0x7fd1b0198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:15.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.055+0000 7fd1ae59c640 1 -- 172.21.15.12:0/2251512047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1b019cfe0 msgr2=0x7fd1b019d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:15.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.055+0000 7fd1ae59c640 1 --2- 172.21.15.12:0/2251512047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1b019cfe0 0x7fd1b019d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.055+0000 7fd1ae59c640 1 -- 172.21.15.12:0/2251512047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1b0107260 msgr2=0x7fd1b0198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:15.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.055+0000 7fd1ae59c640 1 --2- 172.21.15.12:0/2251512047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1b0107260 0x7fd1b0198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.055+0000 7fd1ae59c640 1 -- 172.21.15.12:0/2251512047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b019dad0 con 0x7fd1b006a8d0 2024-04-22T13:24:15.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.055+0000 7fd1add9b640 1 --2- 172.21.15.12:0/2251512047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1b0107260 0x7fd1b0198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:15.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.055+0000 7fd1ae59c640 1 --2- 172.21.15.12:0/2251512047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1b006a8d0 0x7fd1b0198720 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd1a000c920 tx=0x7fd1a000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:15.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.055+0000 7fd19f7fe640 1 -- 172.21.15.12:0/2251512047 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a0007c70 con 0x7fd1b006a8d0 2024-04-22T13:24:15.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.055+0000 7fd1b4f8e640 1 -- 172.21.15.12:0/2251512047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1b019ddc0 con 0x7fd1b006a8d0 2024-04-22T13:24:15.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.055+0000 7fd19f7fe640 1 -- 172.21.15.12:0/2251512047 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1a0007e10 con 0x7fd1b006a8d0 2024-04-22T13:24:15.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.055+0000 7fd19f7fe640 1 -- 172.21.15.12:0/2251512047 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a0018820 con 0x7fd1b006a8d0 2024-04-22T13:24:15.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.055+0000 7fd1b4f8e640 1 -- 172.21.15.12:0/2251512047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1b01a5600 con 0x7fd1b006a8d0 2024-04-22T13:24:15.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.057+0000 7fd1b4f8e640 1 -- 172.21.15.12:0/2251512047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1b0069230 con 0x7fd1b006a8d0 2024-04-22T13:24:15.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.058+0000 7fd19f7fe640 1 -- 172.21.15.12:0/2251512047 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fd1a000f450 con 0x7fd1b006a8d0 2024-04-22T13:24:15.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.058+0000 7fd19f7fe640 1 --2- 172.21.15.12:0/2251512047 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd1980a3e40 0x7fd1980a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:15.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.058+0000 7fd1add9b640 1 --2- 172.21.15.12:0/2251512047 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd1980a3e40 0x7fd1980a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:15.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.058+0000 7fd19f7fe640 1 -- 172.21.15.12:0/2251512047 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fd1a00d0c30 con 0x7fd1b006a8d0 2024-04-22T13:24:15.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.059+0000 7fd1add9b640 1 --2- 172.21.15.12:0/2251512047 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd1980a3e40 0x7fd1980a6300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4036040 tx=0x7fd1a4000bc0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:15.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.063+0000 7fd19f7fe640 1 -- 172.21.15.12:0/2251512047 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd1a0014030 con 0x7fd1b006a8d0 2024-04-22T13:24:15.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:14 smithi134 ceph-mon[31338]: scrub queued for path: / 2024-04-22T13:24:15.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:14 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:24:15.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:14 smithi134 ceph-mon[31338]: scrub summary: active paths [/] 2024-04-22T13:24:15.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:14 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:24:15.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:14 smithi134 ceph-mon[31338]: from='client.15552 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:24:15.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.250+0000 7fd1b4f8e640 1 -- 172.21.15.12:0/2251512047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd1b010a4c0 con 0x7fd1b006a8d0 2024-04-22T13:24:15.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.252+0000 7fd19f7fe640 1 -- 172.21.15.12:0/2251512047 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22997 (secure 0 0 0) 0x7fd1a009d0d0 con 0x7fd1b006a8d0 2024-04-22T13:24:15.249 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:24:15.249 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":90,"fsid":"0aede5ea-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:14:09.362918+0000","modified":"2024-04-22T13:24:06.155595+0000","last_up_change":"2024-04-22T13:21:15.263393+0000","last_in_change":"2024-04-22T13:20:54.174685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:17:32.372434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:22:02.768799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:22:03.578730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d9e2365-3a64-4b13-96fc-c217b1cee2ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6803","nonce":1191350632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6805","nonce":1191350632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6809","nonce":1191350632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6807","nonce":1191350632}]},"public_addr":"172.21.15.12:6803/1191350632","cluster_addr":"172.21.15.12:6805/1191350632","heartbeat_back_addr":"172.21.15.12:6809/1191350632","heartbeat_front_addr":"172.21.15.12:6807/1191350632","state":["exists","up"]},{"osd":1,"uuid":"1499c783-2bd7-48ce-a444-34add42bcf31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6801","nonce":417496465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6803","nonce":417496465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6807","nonce":417496465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6805","nonce":417496465}]},"public_addr":"172.21.15.111:6801/417496465","cluster_addr":"172.21.15.111:6803/417496465","heartbeat_back_addr":"172.21.15.111:6807/417496465","heartbeat_front_addr":"172.21.15.111:6805/417496465","state":["exists","up"]},{"osd":2,"uuid":"71a9ebc3-c622-434e-a6c2-f48358f384f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6801","nonce":1403153998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6803","nonce":1403153998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6807","nonce":1403153998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6805","nonce":1403153998}]},"public_addr":"172.21.15.134:6801/1403153998","cluster_addr":"172.21.15.134:6803/1403153998","heartbeat_back_addr":"172.21.15.134:6807/1403153998","heartbeat_front_addr":"172.21.15.134:6805/1403153998","state":["exists","up"]},{"osd":3,"uuid":"6dd00db2-a286-48ba-8666-56ffdd5f35ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6811","nonce":3359066681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6813","nonce":3359066681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6817","nonce":3359066681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6815","nonce":3359066681}]},"public_addr":"172.21.15.12:6811/3359066681","cluster_addr":"172.21.15.12:6813/3359066681","heartbeat_back_addr":"172.21.15.12:6817/3359066681","heartbeat_front_addr":"172.21.15.12:6815/3359066681","state":["exists","up"]},{"osd":4,"uuid":"1e29afd7-354c-4506-aaa0-b43eb1a19a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6809","nonce":2886349872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6811","nonce":2886349872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6815","nonce":2886349872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6813","nonce":2886349872}]},"public_addr":"172.21.15.111:6809/2886349872","cluster_addr":"172.21.15.111:6811/2886349872","heartbeat_back_addr":"172.21.15.111:6815/2886349872","heartbeat_front_addr":"172.21.15.111:6813/2886349872","state":["exists","up"]},{"osd":5,"uuid":"8f40f02d-0ba2-4842-a03d-2a1a04d1010d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6809","nonce":3071977602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6811","nonce":3071977602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6815","nonce":3071977602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6813","nonce":3071977602}]},"public_addr":"172.21.15.134:6809/3071977602","cluster_addr":"172.21.15.134:6811/3071977602","heartbeat_back_addr":"172.21.15.134:6815/3071977602","heartbeat_front_addr":"172.21.15.134:6813/3071977602","state":["exists","up"]},{"osd":6,"uuid":"bb0c17a9-6a8e-4be6-8660-f73cfa8200ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6819","nonce":3819619299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6821","nonce":3819619299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6825","nonce":3819619299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6823","nonce":3819619299}]},"public_addr":"172.21.15.12:6819/3819619299","cluster_addr":"172.21.15.12:6821/3819619299","heartbeat_back_addr":"172.21.15.12:6825/3819619299","heartbeat_front_addr":"172.21.15.12:6823/3819619299","state":["exists","up"]},{"osd":7,"uuid":"0a93d6af-569e-4e45-b697-9e978c1c1004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6817","nonce":3113856282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6819","nonce":3113856282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6823","nonce":3113856282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6821","nonce":3113856282}]},"public_addr":"172.21.15.111:6817/3113856282","cluster_addr":"172.21.15.111:6819/3113856282","heartbeat_back_addr":"172.21.15.111:6823/3113856282","heartbeat_front_addr":"172.21.15.111:6821/3113856282","state":["exists","up"]},{"osd":8,"uuid":"e21acc68-cf50-4f4f-b1ba-cf6c195b4955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6817","nonce":1689362191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6819","nonce":1689362191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6823","nonce":1689362191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6821","nonce":1689362191}]},"public_addr":"172.21.15.134:6817/1689362191","cluster_addr":"172.21.15.134:6819/1689362191","heartbeat_back_addr":"172.21.15.134:6823/1689362191","heartbeat_front_addr":"172.21.15.134:6821/1689362191","state":["exists","up"]},{"osd":9,"uuid":"fd745869-7f5f-48a8-b412-bb9c92b586e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6827","nonce":4292385779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6829","nonce":4292385779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6833","nonce":4292385779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6831","nonce":4292385779}]},"public_addr":"172.21.15.12:6827/4292385779","cluster_addr":"172.21.15.12:6829/4292385779","heartbeat_back_addr":"172.21.15.12:6833/4292385779","heartbeat_front_addr":"172.21.15.12:6831/4292385779","state":["exists","up"]},{"osd":10,"uuid":"798b1abd-f6c7-4a5f-8da0-fdae49e50d2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6825","nonce":3338566976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6827","nonce":3338566976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6831","nonce":3338566976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6829","nonce":3338566976}]},"public_addr":"172.21.15.111:6825/3338566976","cluster_addr":"172.21.15.111:6827/3338566976","heartbeat_back_addr":"172.21.15.111:6831/3338566976","heartbeat_front_addr":"172.21.15.111:6829/3338566976","state":["exists","up"]},{"osd":11,"uuid":"b18fc5f3-1049-4871-b060-e99290273cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6825","nonce":578640428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6827","nonce":578640428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6831","nonce":578640428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6829","nonce":578640428}]},"public_addr":"172.21.15.134:6825/578640428","cluster_addr":"172.21.15.134:6827/578640428","heartbeat_back_addr":"172.21.15.134:6831/578640428","heartbeat_front_addr":"172.21.15.134:6829/578640428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:16:40.828608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:05.052528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:28.297316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:53.496497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:17.947259+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:41.712894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:07.105945+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:32.129688+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:56.008516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:22.239867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:47.334692+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:12.301276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/3535299028":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/2837479598":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/3010129696":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/2593601362":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/2519382069":"2024-04-23T13:15:04.279990+0000","172.21.15.12:6801/2009831940":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/856202072":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/3509412187":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/1802873236":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/1007749062":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3588480095":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1904962176":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/1802873236":"2024-04-23T13:24:06.155564+0000","172.21.15.12:6800/2009831940":"2024-04-23T13:15:04.279990+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:24:15.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.254+0000 7fd1b4f8e640 1 -- 172.21.15.12:0/2251512047 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd1980a3e40 msgr2=0x7fd1980a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:15.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.254+0000 7fd1b4f8e640 1 --2- 172.21.15.12:0/2251512047 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd1980a3e40 0x7fd1980a6300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4036040 tx=0x7fd1a4000bc0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.254+0000 7fd1b4f8e640 1 -- 172.21.15.12:0/2251512047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1b006a8d0 msgr2=0x7fd1b0198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:15.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.254+0000 7fd1b4f8e640 1 --2- 172.21.15.12:0/2251512047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1b006a8d0 0x7fd1b0198720 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd1a000c920 tx=0x7fd1a000cde0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.254+0000 7fd1b4f8e640 1 -- 172.21.15.12:0/2251512047 shutdown_connections 2024-04-22T13:24:15.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.254+0000 7fd1b4f8e640 1 --2- 172.21.15.12:0/2251512047 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd1980a3e40 0x7fd1980a6300 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.254+0000 7fd1b4f8e640 1 --2- 172.21.15.12:0/2251512047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1b019cfe0 0x7fd1b019d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.254+0000 7fd1b4f8e640 1 --2- 172.21.15.12:0/2251512047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1b0107260 0x7fd1b0198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.254+0000 7fd1b4f8e640 1 --2- 172.21.15.12:0/2251512047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd1b006a8d0 0x7fd1b0198720 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.254+0000 7fd1b4f8e640 1 -- 172.21.15.12:0/2251512047 >> 172.21.15.12:0/2251512047 conn(0x7fd1b0077c20 msgr2=0x7fd1b00ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:15.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.254+0000 7fd1b4f8e640 1 -- 172.21.15.12:0/2251512047 shutdown_connections 2024-04-22T13:24:15.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.254+0000 7fd1b4f8e640 1 -- 172.21.15.12:0/2251512047 wait complete. 2024-04-22T13:24:15.263 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T13:17:32.372434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T13:22:02.768799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T13:22:03.578730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T13:24:15.263 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T13:24:15.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.370+0000 7f5fc5eeb640 1 -- 172.21.15.12:0/4082464922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fc0101a40 msgr2=0x7f5fc0105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:15.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.370+0000 7f5fc5eeb640 1 --2- 172.21.15.12:0/4082464922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fc0101a40 0x7f5fc0105a80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4009900 tx=0x7f5fb402bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:15.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.370+0000 7f5fc5eeb640 1 -- 172.21.15.12:0/4082464922 shutdown_connections 2024-04-22T13:24:15.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.370+0000 7f5fc5eeb640 1 --2- 172.21.15.12:0/4082464922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fc0106160 0x7f5fc010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.370+0000 7f5fc5eeb640 1 --2- 172.21.15.12:0/4082464922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fc0101a40 0x7f5fc0105a80 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.370+0000 7f5fc5eeb640 1 --2- 172.21.15.12:0/4082464922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fc0101090 0x7f5fc0101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.370+0000 7f5fc5eeb640 1 -- 172.21.15.12:0/4082464922 >> 172.21.15.12:0/4082464922 conn(0x7f5fc0077f50 msgr2=0x7f5fc0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:15.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.370+0000 7f5fc5eeb640 1 -- 172.21.15.12:0/4082464922 shutdown_connections 2024-04-22T13:24:15.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.370+0000 7f5fc5eeb640 1 -- 172.21.15.12:0/4082464922 wait complete. 2024-04-22T13:24:15.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.371+0000 7f5fc5eeb640 1 Processor -- start 2024-04-22T13:24:15.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.371+0000 7f5fc5eeb640 1 -- start start 2024-04-22T13:24:15.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.371+0000 7f5fc5eeb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fc0101090 0x7f5fc019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:15.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.371+0000 7f5fc5eeb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fc0106160 0x7f5fc019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:15.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.371+0000 7f5fc5eeb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fc0198590 0x7f5fc0198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:15.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.371+0000 7f5fc5eeb640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc0111080 con 0x7f5fc0106160 2024-04-22T13:24:15.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.371+0000 7f5fc5eeb640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc0110f00 con 0x7f5fc0198590 2024-04-22T13:24:15.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.371+0000 7f5fc5eeb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc0111200 con 0x7f5fc0101090 2024-04-22T13:24:15.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.371+0000 7f5fbeffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fc0106160 0x7f5fc019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:15.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.371+0000 7f5fbeffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fc0106160 0x7f5fc019e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59414/0 (socket says 172.21.15.12:59414) 2024-04-22T13:24:15.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.372+0000 7f5fbeffd640 1 -- 172.21.15.12:0/1391668274 learned_addr learned my addr 172.21.15.12:0/1391668274 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:15.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.372+0000 7f5fbf7fe640 1 --2- 172.21.15.12:0/1391668274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fc0101090 0x7f5fc019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:15.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.372+0000 7f5fbffff640 1 --2- 172.21.15.12:0/1391668274 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fc0198590 0x7f5fc0198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:15.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.372+0000 7f5fbeffd640 1 -- 172.21.15.12:0/1391668274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fc0101090 msgr2=0x7f5fc019e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:15.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.372+0000 7f5fbeffd640 1 --2- 172.21.15.12:0/1391668274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fc0101090 0x7f5fc019e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.372+0000 7f5fbeffd640 1 -- 172.21.15.12:0/1391668274 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fc0198590 msgr2=0x7f5fc0198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:15.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.372+0000 7f5fbeffd640 1 --2- 172.21.15.12:0/1391668274 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fc0198590 0x7f5fc0198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.372+0000 7f5fbeffd640 1 -- 172.21.15.12:0/1391668274 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc01990b0 con 0x7f5fc0106160 2024-04-22T13:24:15.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.372+0000 7f5fbffff640 1 --2- 172.21.15.12:0/1391668274 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fc0198590 0x7f5fc0198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:15.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.372+0000 7f5fbeffd640 1 --2- 172.21.15.12:0/1391668274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fc0106160 0x7f5fc019e920 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4036040 tx=0x7f5fb402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:15.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.372+0000 7f5fbcff9640 1 -- 172.21.15.12:0/1391668274 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4043020 con 0x7f5fc0106160 2024-04-22T13:24:15.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.372+0000 7f5fbcff9640 1 -- 172.21.15.12:0/1391668274 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fb402cbe0 con 0x7f5fc0106160 2024-04-22T13:24:15.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.372+0000 7f5fc5eeb640 1 -- 172.21.15.12:0/1391668274 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fc0199340 con 0x7f5fc0106160 2024-04-22T13:24:15.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.372+0000 7f5fbcff9640 1 -- 172.21.15.12:0/1391668274 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4003bf0 con 0x7f5fc0106160 2024-04-22T13:24:15.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.372+0000 7f5fc5eeb640 1 -- 172.21.15.12:0/1391668274 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fc01a5590 con 0x7f5fc0106160 2024-04-22T13:24:15.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.373+0000 7f5fc5eeb640 1 -- 172.21.15.12:0/1391668274 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fc0068d70 con 0x7f5fc0106160 2024-04-22T13:24:15.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.374+0000 7f5fbcff9640 1 -- 172.21.15.12:0/1391668274 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f5fb4046020 con 0x7f5fc0106160 2024-04-22T13:24:15.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.375+0000 7f5fbcff9640 1 --2- 172.21.15.12:0/1391668274 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5fa80a3e40 0x7f5fa80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:15.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.375+0000 7f5fbcff9640 1 -- 172.21.15.12:0/1391668274 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f5fb40f2c50 con 0x7f5fc0106160 2024-04-22T13:24:15.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.375+0000 7f5fbf7fe640 1 --2- 172.21.15.12:0/1391668274 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5fa80a3e40 0x7f5fa80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:15.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.375+0000 7f5fbf7fe640 1 --2- 172.21.15.12:0/1391668274 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5fa80a3e40 0x7f5fa80a6300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00058c0 tx=0x7f5fb000a4c0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:15.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.379+0000 7f5fbcff9640 1 -- 172.21.15.12:0/1391668274 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5fb40f8020 con 0x7f5fc0106160 2024-04-22T13:24:15.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.566+0000 7f5fc5eeb640 1 -- 172.21.15.12:0/1391668274 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5fc0101a40 con 0x7f5fc0106160 2024-04-22T13:24:15.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.567+0000 7f5fbcff9640 1 -- 172.21.15.12:0/1391668274 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f5fb404d020 con 0x7f5fc0106160 2024-04-22T13:24:15.564 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-22T13:24:15.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.568+0000 7f5fc5eeb640 1 -- 172.21.15.12:0/1391668274 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5fa80a3e40 msgr2=0x7f5fa80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:15.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.568+0000 7f5fc5eeb640 1 --2- 172.21.15.12:0/1391668274 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5fa80a3e40 0x7f5fa80a6300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5fb00058c0 tx=0x7f5fb000a4c0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.569+0000 7f5fc5eeb640 1 -- 172.21.15.12:0/1391668274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fc0106160 msgr2=0x7f5fc019e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:15.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.569+0000 7f5fc5eeb640 1 --2- 172.21.15.12:0/1391668274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fc0106160 0x7f5fc019e920 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4036040 tx=0x7f5fb402c7e0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.569+0000 7f5fc5eeb640 1 -- 172.21.15.12:0/1391668274 shutdown_connections 2024-04-22T13:24:15.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.569+0000 7f5fc5eeb640 1 --2- 172.21.15.12:0/1391668274 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5fa80a3e40 0x7f5fa80a6300 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.569+0000 7f5fc5eeb640 1 --2- 172.21.15.12:0/1391668274 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fc0198590 0x7f5fc0198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.569+0000 7f5fc5eeb640 1 --2- 172.21.15.12:0/1391668274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fc0106160 0x7f5fc019e920 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.569+0000 7f5fc5eeb640 1 --2- 172.21.15.12:0/1391668274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fc0101090 0x7f5fc019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.569+0000 7f5fc5eeb640 1 -- 172.21.15.12:0/1391668274 >> 172.21.15.12:0/1391668274 conn(0x7f5fc0077f50 msgr2=0x7f5fc00694e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:15.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.569+0000 7f5fc5eeb640 1 -- 172.21.15.12:0/1391668274 shutdown_connections 2024-04-22T13:24:15.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.569+0000 7f5fc5eeb640 1 -- 172.21.15.12:0/1391668274 wait complete. 2024-04-22T13:24:15.577 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T13:24:15.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.684+0000 7f3379ac8640 1 -- 172.21.15.12:0/2248951306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3374102b40 msgr2=0x7f3374106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:15.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.684+0000 7f3379ac8640 1 --2- 172.21.15.12:0/2248951306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3374102b40 0x7f3374106b80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3368009900 tx=0x7f336802bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:15.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.684+0000 7f3379ac8640 1 -- 172.21.15.12:0/2248951306 shutdown_connections 2024-04-22T13:24:15.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.684+0000 7f3379ac8640 1 --2- 172.21.15.12:0/2248951306 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3374107260 0x7f337410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.684+0000 7f3379ac8640 1 --2- 172.21.15.12:0/2248951306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3374102b40 0x7f3374106b80 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.684+0000 7f3379ac8640 1 --2- 172.21.15.12:0/2248951306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3374102190 0x7f3374102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.684+0000 7f3379ac8640 1 -- 172.21.15.12:0/2248951306 >> 172.21.15.12:0/2248951306 conn(0x7f33740fd990 msgr2=0x7f33740ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:15.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.685+0000 7f3379ac8640 1 -- 172.21.15.12:0/2248951306 shutdown_connections 2024-04-22T13:24:15.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.685+0000 7f3379ac8640 1 -- 172.21.15.12:0/2248951306 wait complete. 2024-04-22T13:24:15.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.685+0000 7f3379ac8640 1 Processor -- start 2024-04-22T13:24:15.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.685+0000 7f3379ac8640 1 -- start start 2024-04-22T13:24:15.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.686+0000 7f3379ac8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3374102190 0x7f33741a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:15.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.686+0000 7f3379ac8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3374107260 0x7f33741a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:15.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.686+0000 7f3379ac8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f337419a810 0x7f337419acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:15.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.686+0000 7f3379ac8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f33741121d0 con 0x7f337419a810 2024-04-22T13:24:15.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.686+0000 7f3379ac8640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3374112050 con 0x7f3374107260 2024-04-22T13:24:15.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.686+0000 7f3379ac8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3374112350 con 0x7f3374102190 2024-04-22T13:24:15.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.686+0000 7f3373fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f337419a810 0x7f337419acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:15.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.686+0000 7f3373fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f337419a810 0x7f337419acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59422/0 (socket says 172.21.15.12:59422) 2024-04-22T13:24:15.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.686+0000 7f3373fff640 1 -- 172.21.15.12:0/3993057345 learned_addr learned my addr 172.21.15.12:0/3993057345 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:15.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.686+0000 7f33737fe640 1 --2- 172.21.15.12:0/3993057345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3374102190 0x7f33741a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:15.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.686+0000 7f3372ffd640 1 --2- 172.21.15.12:0/3993057345 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3374107260 0x7f33741a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:15.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.686+0000 7f3373fff640 1 -- 172.21.15.12:0/3993057345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3374102190 msgr2=0x7f33741a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:15.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.686+0000 7f3373fff640 1 --2- 172.21.15.12:0/3993057345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3374102190 0x7f33741a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.686+0000 7f3373fff640 1 -- 172.21.15.12:0/3993057345 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3374107260 msgr2=0x7f33741a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:15.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.686+0000 7f3373fff640 1 --2- 172.21.15.12:0/3993057345 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3374107260 0x7f33741a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.686+0000 7f3373fff640 1 -- 172.21.15.12:0/3993057345 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f337419b3a0 con 0x7f337419a810 2024-04-22T13:24:15.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.686+0000 7f3373fff640 1 --2- 172.21.15.12:0/3993057345 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f337419a810 0x7f337419acc0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f336c00cad0 tx=0x7f336c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:15.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.686+0000 7f3370ff9640 1 -- 172.21.15.12:0/3993057345 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c00ae20 con 0x7f337419a810 2024-04-22T13:24:15.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.686+0000 7f3379ac8640 1 -- 172.21.15.12:0/3993057345 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f337419b690 con 0x7f337419a810 2024-04-22T13:24:15.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.686+0000 7f3370ff9640 1 -- 172.21.15.12:0/3993057345 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f336c005020 con 0x7f337419a810 2024-04-22T13:24:15.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.686+0000 7f3370ff9640 1 -- 172.21.15.12:0/3993057345 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c00f6b0 con 0x7f337419a810 2024-04-22T13:24:15.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.687+0000 7f3379ac8640 1 -- 172.21.15.12:0/3993057345 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33741a77a0 con 0x7f337419a810 2024-04-22T13:24:15.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.687+0000 7f3379ac8640 1 -- 172.21.15.12:0/3993057345 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3374103660 con 0x7f337419a810 2024-04-22T13:24:15.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.689+0000 7f3370ff9640 1 -- 172.21.15.12:0/3993057345 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f336c0051d0 con 0x7f337419a810 2024-04-22T13:24:15.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.689+0000 7f3370ff9640 1 --2- 172.21.15.12:0/3993057345 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f335c0a3e40 0x7f335c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:15.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.689+0000 7f3370ff9640 1 -- 172.21.15.12:0/3993057345 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f336c0d0a10 con 0x7f337419a810 2024-04-22T13:24:15.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.689+0000 7f33737fe640 1 --2- 172.21.15.12:0/3993057345 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f335c0a3e40 0x7f335c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:15.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.690+0000 7f33737fe640 1 --2- 172.21.15.12:0/3993057345 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f335c0a3e40 0x7f335c0a6300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f336400bbe0 tx=0x7f336400b390 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:15.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.694+0000 7f3370ff9640 1 -- 172.21.15.12:0/3993057345 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f337419b690 con 0x7f337419a810 2024-04-22T13:24:15.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:15 smithi012 ceph-mon[28265]: from='client.15591 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:24:15.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:15 smithi012 ceph-mon[28265]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.3 KiB/s wr, 26 op/s 2024-04-22T13:24:15.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:15 smithi012 ceph-mon[28265]: from='client.15597 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:24:15.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:15 smithi012 ceph-mon[28265]: scrub summary: idle 2024-04-22T13:24:15.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:15 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2251512047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:24:15.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:15 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1391668274' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:24:15.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.881+0000 7f3379ac8640 1 -- 172.21.15.12:0/3993057345 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3374102c40 con 0x7f337419a810 2024-04-22T13:24:15.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.881+0000 7f3370ff9640 1 -- 172.21.15.12:0/3993057345 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f336c014030 con 0x7f337419a810 2024-04-22T13:24:15.879 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-22T13:24:15.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.883+0000 7f3379ac8640 1 -- 172.21.15.12:0/3993057345 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f335c0a3e40 msgr2=0x7f335c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:15.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.883+0000 7f3379ac8640 1 --2- 172.21.15.12:0/3993057345 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f335c0a3e40 0x7f335c0a6300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f336400bbe0 tx=0x7f336400b390 comp rx=0 tx=0).stop 2024-04-22T13:24:15.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.883+0000 7f3379ac8640 1 -- 172.21.15.12:0/3993057345 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f337419a810 msgr2=0x7f337419acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:15.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.883+0000 7f3379ac8640 1 --2- 172.21.15.12:0/3993057345 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f337419a810 0x7f337419acc0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f336c00cad0 tx=0x7f336c00cf90 comp rx=0 tx=0).stop 2024-04-22T13:24:15.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.883+0000 7f3379ac8640 1 -- 172.21.15.12:0/3993057345 shutdown_connections 2024-04-22T13:24:15.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.883+0000 7f3379ac8640 1 --2- 172.21.15.12:0/3993057345 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f335c0a3e40 0x7f335c0a6300 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.883+0000 7f3379ac8640 1 --2- 172.21.15.12:0/3993057345 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f337419a810 0x7f337419acc0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.883+0000 7f3379ac8640 1 --2- 172.21.15.12:0/3993057345 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3374107260 0x7f33741a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.883+0000 7f3379ac8640 1 --2- 172.21.15.12:0/3993057345 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3374102190 0x7f33741a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.883+0000 7f3379ac8640 1 -- 172.21.15.12:0/3993057345 >> 172.21.15.12:0/3993057345 conn(0x7f33740fd990 msgr2=0x7f33740fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:15.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.884+0000 7f3379ac8640 1 -- 172.21.15.12:0/3993057345 shutdown_connections 2024-04-22T13:24:15.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.884+0000 7f3379ac8640 1 -- 172.21.15.12:0/3993057345 wait complete. 2024-04-22T13:24:15.892 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T13:24:15.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:15 smithi111 ceph-mon[31361]: from='client.15591 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:24:15.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:15 smithi111 ceph-mon[31361]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.3 KiB/s wr, 26 op/s 2024-04-22T13:24:15.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:15 smithi111 ceph-mon[31361]: from='client.15597 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:24:15.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:15 smithi111 ceph-mon[31361]: scrub summary: idle 2024-04-22T13:24:15.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:15 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2251512047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:24:15.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:15 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1391668274' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:24:15.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.999+0000 7f014d5c7640 1 -- 172.21.15.12:0/2298564151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0148100d50 msgr2=0x7f0148101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:15.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.999+0000 7f014d5c7640 1 --2- 172.21.15.12:0/2298564151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0148100d50 0x7f0148101130 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0138009ca0 tx=0x7f013802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.999+0000 7f014d5c7640 1 -- 172.21.15.12:0/2298564151 shutdown_connections 2024-04-22T13:24:15.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.999+0000 7f014d5c7640 1 --2- 172.21.15.12:0/2298564151 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f014810e900 0x7f0148110cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:15.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.999+0000 7f014d5c7640 1 --2- 172.21.15.12:0/2298564151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0148101670 0x7f014810e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.999+0000 7f014d5c7640 1 --2- 172.21.15.12:0/2298564151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0148100d50 0x7f0148101130 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:15.999+0000 7f014d5c7640 1 -- 172.21.15.12:0/2298564151 >> 172.21.15.12:0/2298564151 conn(0x7f0148077f50 msgr2=0x7f0148078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.000+0000 7f014d5c7640 1 -- 172.21.15.12:0/2298564151 shutdown_connections 2024-04-22T13:24:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.000+0000 7f014d5c7640 1 -- 172.21.15.12:0/2298564151 wait complete. 2024-04-22T13:24:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.000+0000 7f014d5c7640 1 Processor -- start 2024-04-22T13:24:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.000+0000 7f014d5c7640 1 -- start start 2024-04-22T13:24:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.000+0000 7f014d5c7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0148100d50 0x7f01481a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.000+0000 7f014d5c7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0148101670 0x7f01481a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.000+0000 7f014d5c7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f014810e900 0x7f014819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.000+0000 7f014d5c7640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f01481143d0 con 0x7f0148101670 2024-04-22T13:24:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.000+0000 7f014d5c7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0148114250 con 0x7f0148100d50 2024-04-22T13:24:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.000+0000 7f014d5c7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0148114550 con 0x7f014810e900 2024-04-22T13:24:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.001+0000 7f01467fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0148101670 0x7f01481a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:16.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.001+0000 7f01467fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0148101670 0x7f01481a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59436/0 (socket says 172.21.15.12:59436) 2024-04-22T13:24:16.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.001+0000 7f01467fc640 1 -- 172.21.15.12:0/1316559422 learned_addr learned my addr 172.21.15.12:0/1316559422 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:16.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.001+0000 7f01477fe640 1 --2- 172.21.15.12:0/1316559422 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f014810e900 0x7f014819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:16.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.001+0000 7f0146ffd640 1 --2- 172.21.15.12:0/1316559422 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0148100d50 0x7f01481a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:16.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.001+0000 7f01467fc640 1 -- 172.21.15.12:0/1316559422 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f014810e900 msgr2=0x7f014819a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:16.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.001+0000 7f01467fc640 1 --2- 172.21.15.12:0/1316559422 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f014810e900 0x7f014819a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.001+0000 7f01467fc640 1 -- 172.21.15.12:0/1316559422 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0148100d50 msgr2=0x7f01481a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:16.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.001+0000 7f01467fc640 1 --2- 172.21.15.12:0/1316559422 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0148100d50 0x7f01481a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.001+0000 7f01467fc640 1 -- 172.21.15.12:0/1316559422 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f014819af80 con 0x7f0148101670 2024-04-22T13:24:16.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.001+0000 7f01477fe640 1 --2- 172.21.15.12:0/1316559422 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f014810e900 0x7f014819a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:16.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.001+0000 7f0146ffd640 1 --2- 172.21.15.12:0/1316559422 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0148100d50 0x7f01481a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:16.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.001+0000 7f01467fc640 1 --2- 172.21.15.12:0/1316559422 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0148101670 0x7f01481a0b80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f013c00d770 tx=0x7f013c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:16.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.001+0000 7f012ffff640 1 -- 172.21.15.12:0/1316559422 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c017070 con 0x7f0148101670 2024-04-22T13:24:16.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.001+0000 7f014d5c7640 1 -- 172.21.15.12:0/1316559422 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f014819b270 con 0x7f0148101670 2024-04-22T13:24:16.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.001+0000 7f012ffff640 1 -- 172.21.15.12:0/1316559422 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f013c0054d0 con 0x7f0148101670 2024-04-22T13:24:16.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.001+0000 7f012ffff640 1 -- 172.21.15.12:0/1316559422 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013c004c30 con 0x7f0148101670 2024-04-22T13:24:16.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.001+0000 7f014d5c7640 1 -- 172.21.15.12:0/1316559422 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01481a7490 con 0x7f0148101670 2024-04-22T13:24:16.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.002+0000 7f014d5c7640 1 -- 172.21.15.12:0/1316559422 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01480698e0 con 0x7f0148101670 2024-04-22T13:24:16.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.004+0000 7f012ffff640 1 -- 172.21.15.12:0/1316559422 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f013c004dd0 con 0x7f0148101670 2024-04-22T13:24:16.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.004+0000 7f012ffff640 1 --2- 172.21.15.12:0/1316559422 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f01300a3e40 0x7f01300a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:16.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.005+0000 7f0146ffd640 1 --2- 172.21.15.12:0/1316559422 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f01300a3e40 0x7f01300a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:16.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.005+0000 7f012ffff640 1 -- 172.21.15.12:0/1316559422 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f013c0d0a80 con 0x7f0148101670 2024-04-22T13:24:16.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.005+0000 7f0146ffd640 1 --2- 172.21.15.12:0/1316559422 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f01300a3e40 0x7f01300a6300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0138006010 tx=0x7f0138000f40 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:16.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.008+0000 7f012ffff640 1 -- 172.21.15.12:0/1316559422 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f013c09cec0 con 0x7f0148101670 2024-04-22T13:24:16.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:15 smithi134 ceph-mon[31338]: from='client.15591 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:24:16.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:15 smithi134 ceph-mon[31338]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.3 KiB/s wr, 26 op/s 2024-04-22T13:24:16.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:15 smithi134 ceph-mon[31338]: from='client.15597 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:24:16.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:15 smithi134 ceph-mon[31338]: scrub summary: idle 2024-04-22T13:24:16.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:15 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2251512047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:24:16.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:15 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1391668274' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:24:16.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.196+0000 7f014d5c7640 1 -- 172.21.15.12:0/1316559422 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f014802d5c0 con 0x7f0148101670 2024-04-22T13:24:16.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.196+0000 7f012ffff640 1 -- 172.21.15.12:0/1316559422 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f013c09c7c0 con 0x7f0148101670 2024-04-22T13:24:16.194 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-22T13:24:16.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.198+0000 7f014d5c7640 1 -- 172.21.15.12:0/1316559422 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f01300a3e40 msgr2=0x7f01300a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:16.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.198+0000 7f014d5c7640 1 --2- 172.21.15.12:0/1316559422 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f01300a3e40 0x7f01300a6300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0138006010 tx=0x7f0138000f40 comp rx=0 tx=0).stop 2024-04-22T13:24:16.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.198+0000 7f014d5c7640 1 -- 172.21.15.12:0/1316559422 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0148101670 msgr2=0x7f01481a0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:16.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.198+0000 7f014d5c7640 1 --2- 172.21.15.12:0/1316559422 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0148101670 0x7f01481a0b80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f013c00d770 tx=0x7f013c00dc30 comp rx=0 tx=0).stop 2024-04-22T13:24:16.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.198+0000 7f014d5c7640 1 -- 172.21.15.12:0/1316559422 shutdown_connections 2024-04-22T13:24:16.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.198+0000 7f014d5c7640 1 --2- 172.21.15.12:0/1316559422 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f01300a3e40 0x7f01300a6300 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.198+0000 7f014d5c7640 1 --2- 172.21.15.12:0/1316559422 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f014810e900 0x7f014819a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.198+0000 7f014d5c7640 1 --2- 172.21.15.12:0/1316559422 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0148101670 0x7f01481a0b80 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.198+0000 7f014d5c7640 1 --2- 172.21.15.12:0/1316559422 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0148100d50 0x7f01481a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.198+0000 7f014d5c7640 1 -- 172.21.15.12:0/1316559422 >> 172.21.15.12:0/1316559422 conn(0x7f0148077f50 msgr2=0x7f0148110080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:16.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.198+0000 7f014d5c7640 1 -- 172.21.15.12:0/1316559422 shutdown_connections 2024-04-22T13:24:16.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.198+0000 7f014d5c7640 1 -- 172.21.15.12:0/1316559422 wait complete. 2024-04-22T13:24:16.207 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:24:16.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.311+0000 7f485cbd5640 1 -- 172.21.15.12:0/2666564444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4858103af0 msgr2=0x7f485810bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:16.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.311+0000 7f485cbd5640 1 --2- 172.21.15.12:0/2666564444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4858103af0 0x7f485810bfd0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f484c009900 tx=0x7f484c02bcd0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.311+0000 7f485cbd5640 1 -- 172.21.15.12:0/2666564444 shutdown_connections 2024-04-22T13:24:16.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.311+0000 7f485cbd5640 1 --2- 172.21.15.12:0/2666564444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f485810c5a0 0x7f485810e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.311+0000 7f485cbd5640 1 --2- 172.21.15.12:0/2666564444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4858103af0 0x7f485810bfd0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.311+0000 7f485cbd5640 1 --2- 172.21.15.12:0/2666564444 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48581031d0 0x7f48581035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.311+0000 7f485cbd5640 1 -- 172.21.15.12:0/2666564444 >> 172.21.15.12:0/2666564444 conn(0x7f48580771d0 msgr2=0x7f48580775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:16.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.312+0000 7f485cbd5640 1 -- 172.21.15.12:0/2666564444 shutdown_connections 2024-04-22T13:24:16.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.312+0000 7f485cbd5640 1 -- 172.21.15.12:0/2666564444 wait complete. 2024-04-22T13:24:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.312+0000 7f485cbd5640 1 Processor -- start 2024-04-22T13:24:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.312+0000 7f485cbd5640 1 -- start start 2024-04-22T13:24:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.312+0000 7f485cbd5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48581031d0 0x7f485819e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.312+0000 7f485cbd5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4858103af0 0x7f485819e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.312+0000 7f485cbd5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f485810c5a0 0x7f4858198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.312+0000 7f485cbd5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4858112240 con 0x7f4858103af0 2024-04-22T13:24:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.312+0000 7f485cbd5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f48581120c0 con 0x7f485810c5a0 2024-04-22T13:24:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.312+0000 7f485cbd5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f48581123c0 con 0x7f48581031d0 2024-04-22T13:24:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.313+0000 7f4855d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4858103af0 0x7f485819e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.313+0000 7f4855d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4858103af0 0x7f485819e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59466/0 (socket says 172.21.15.12:59466) 2024-04-22T13:24:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.313+0000 7f4855d9b640 1 -- 172.21.15.12:0/1230942755 learned_addr learned my addr 172.21.15.12:0/1230942755 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.313+0000 7f485659c640 1 --2- 172.21.15.12:0/1230942755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48581031d0 0x7f485819e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.313+0000 7f4856d9d640 1 --2- 172.21.15.12:0/1230942755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f485810c5a0 0x7f4858198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:16.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.313+0000 7f4855d9b640 1 -- 172.21.15.12:0/1230942755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48581031d0 msgr2=0x7f485819e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.313+0000 7f4855d9b640 1 --2- 172.21.15.12:0/1230942755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48581031d0 0x7f485819e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.313+0000 7f4855d9b640 1 -- 172.21.15.12:0/1230942755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f485810c5a0 msgr2=0x7f4858198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.313+0000 7f4855d9b640 1 --2- 172.21.15.12:0/1230942755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f485810c5a0 0x7f4858198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.313+0000 7f4855d9b640 1 -- 172.21.15.12:0/1230942755 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4858198d80 con 0x7f4858103af0 2024-04-22T13:24:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.313+0000 7f4855d9b640 1 --2- 172.21.15.12:0/1230942755 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4858103af0 0x7f485819e990 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f484c005940 tx=0x7f484c02d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.313+0000 7f48477fe640 1 -- 172.21.15.12:0/1230942755 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c043020 con 0x7f4858103af0 2024-04-22T13:24:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.313+0000 7f485cbd5640 1 -- 172.21.15.12:0/1230942755 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4858199010 con 0x7f4858103af0 2024-04-22T13:24:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.313+0000 7f48477fe640 1 -- 172.21.15.12:0/1230942755 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f484c02dc90 con 0x7f4858103af0 2024-04-22T13:24:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.313+0000 7f48477fe640 1 -- 172.21.15.12:0/1230942755 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c003970 con 0x7f4858103af0 2024-04-22T13:24:16.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.313+0000 7f485cbd5640 1 -- 172.21.15.12:0/1230942755 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48581a5310 con 0x7f4858103af0 2024-04-22T13:24:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.314+0000 7f485cbd5640 1 -- 172.21.15.12:0/1230942755 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4858069230 con 0x7f4858103af0 2024-04-22T13:24:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.315+0000 7f48477fe640 1 -- 172.21.15.12:0/1230942755 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f484c046020 con 0x7f4858103af0 2024-04-22T13:24:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.316+0000 7f48477fe640 1 --2- 172.21.15.12:0/1230942755 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f48400a3e40 0x7f48400a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:16.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.316+0000 7f48477fe640 1 -- 172.21.15.12:0/1230942755 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f484c0f2b60 con 0x7f4858103af0 2024-04-22T13:24:16.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.316+0000 7f485659c640 1 --2- 172.21.15.12:0/1230942755 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f48400a3e40 0x7f48400a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:16.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.317+0000 7f485659c640 1 --2- 172.21.15.12:0/1230942755 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f48400a3e40 0x7f48400a6300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4848009990 tx=0x7f4848009110 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:16.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.321+0000 7f48477fe640 1 -- 172.21.15.12:0/1230942755 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f484c04d020 con 0x7f4858103af0 2024-04-22T13:24:16.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.508+0000 7f485cbd5640 1 -- 172.21.15.12:0/1230942755 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4858107690 con 0x7f4858103af0 2024-04-22T13:24:16.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.510+0000 7f48477fe640 1 -- 172.21.15.12:0/1230942755 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22997 (secure 0 0 0) 0x7f484c0bf110 con 0x7f4858103af0 2024-04-22T13:24:16.507 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:24:16.507 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":90,"fsid":"0aede5ea-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:14:09.362918+0000","modified":"2024-04-22T13:24:06.155595+0000","last_up_change":"2024-04-22T13:21:15.263393+0000","last_in_change":"2024-04-22T13:20:54.174685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:17:32.372434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:22:02.768799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:22:03.578730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d9e2365-3a64-4b13-96fc-c217b1cee2ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6803","nonce":1191350632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6805","nonce":1191350632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6809","nonce":1191350632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6807","nonce":1191350632}]},"public_addr":"172.21.15.12:6803/1191350632","cluster_addr":"172.21.15.12:6805/1191350632","heartbeat_back_addr":"172.21.15.12:6809/1191350632","heartbeat_front_addr":"172.21.15.12:6807/1191350632","state":["exists","up"]},{"osd":1,"uuid":"1499c783-2bd7-48ce-a444-34add42bcf31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6801","nonce":417496465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6803","nonce":417496465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6807","nonce":417496465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6805","nonce":417496465}]},"public_addr":"172.21.15.111:6801/417496465","cluster_addr":"172.21.15.111:6803/417496465","heartbeat_back_addr":"172.21.15.111:6807/417496465","heartbeat_front_addr":"172.21.15.111:6805/417496465","state":["exists","up"]},{"osd":2,"uuid":"71a9ebc3-c622-434e-a6c2-f48358f384f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6801","nonce":1403153998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6803","nonce":1403153998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6807","nonce":1403153998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6805","nonce":1403153998}]},"public_addr":"172.21.15.134:6801/1403153998","cluster_addr":"172.21.15.134:6803/1403153998","heartbeat_back_addr":"172.21.15.134:6807/1403153998","heartbeat_front_addr":"172.21.15.134:6805/1403153998","state":["exists","up"]},{"osd":3,"uuid":"6dd00db2-a286-48ba-8666-56ffdd5f35ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6811","nonce":3359066681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6813","nonce":3359066681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6817","nonce":3359066681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6815","nonce":3359066681}]},"public_addr":"172.21.15.12:6811/3359066681","cluster_addr":"172.21.15.12:6813/3359066681","heartbeat_back_addr":"172.21.15.12:6817/3359066681","heartbeat_front_addr":"172.21.15.12:6815/3359066681","state":["exists","up"]},{"osd":4,"uuid":"1e29afd7-354c-4506-aaa0-b43eb1a19a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6809","nonce":2886349872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6811","nonce":2886349872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6815","nonce":2886349872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6813","nonce":2886349872}]},"public_addr":"172.21.15.111:6809/2886349872","cluster_addr":"172.21.15.111:6811/2886349872","heartbeat_back_addr":"172.21.15.111:6815/2886349872","heartbeat_front_addr":"172.21.15.111:6813/2886349872","state":["exists","up"]},{"osd":5,"uuid":"8f40f02d-0ba2-4842-a03d-2a1a04d1010d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6809","nonce":3071977602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6811","nonce":3071977602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6815","nonce":3071977602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6813","nonce":3071977602}]},"public_addr":"172.21.15.134:6809/3071977602","cluster_addr":"172.21.15.134:6811/3071977602","heartbeat_back_addr":"172.21.15.134:6815/3071977602","heartbeat_front_addr":"172.21.15.134:6813/3071977602","state":["exists","up"]},{"osd":6,"uuid":"bb0c17a9-6a8e-4be6-8660-f73cfa8200ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6819","nonce":3819619299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6821","nonce":3819619299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6825","nonce":3819619299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6823","nonce":3819619299}]},"public_addr":"172.21.15.12:6819/3819619299","cluster_addr":"172.21.15.12:6821/3819619299","heartbeat_back_addr":"172.21.15.12:6825/3819619299","heartbeat_front_addr":"172.21.15.12:6823/3819619299","state":["exists","up"]},{"osd":7,"uuid":"0a93d6af-569e-4e45-b697-9e978c1c1004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6817","nonce":3113856282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6819","nonce":3113856282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6823","nonce":3113856282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6821","nonce":3113856282}]},"public_addr":"172.21.15.111:6817/3113856282","cluster_addr":"172.21.15.111:6819/3113856282","heartbeat_back_addr":"172.21.15.111:6823/3113856282","heartbeat_front_addr":"172.21.15.111:6821/3113856282","state":["exists","up"]},{"osd":8,"uuid":"e21acc68-cf50-4f4f-b1ba-cf6c195b4955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6817","nonce":1689362191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6819","nonce":1689362191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6823","nonce":1689362191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6821","nonce":1689362191}]},"public_addr":"172.21.15.134:6817/1689362191","cluster_addr":"172.21.15.134:6819/1689362191","heartbeat_back_addr":"172.21.15.134:6823/1689362191","heartbeat_front_addr":"172.21.15.134:6821/1689362191","state":["exists","up"]},{"osd":9,"uuid":"fd745869-7f5f-48a8-b412-bb9c92b586e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6827","nonce":4292385779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6829","nonce":4292385779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6833","nonce":4292385779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6831","nonce":4292385779}]},"public_addr":"172.21.15.12:6827/4292385779","cluster_addr":"172.21.15.12:6829/4292385779","heartbeat_back_addr":"172.21.15.12:6833/4292385779","heartbeat_front_addr":"172.21.15.12:6831/4292385779","state":["exists","up"]},{"osd":10,"uuid":"798b1abd-f6c7-4a5f-8da0-fdae49e50d2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6825","nonce":3338566976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6827","nonce":3338566976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6831","nonce":3338566976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6829","nonce":3338566976}]},"public_addr":"172.21.15.111:6825/3338566976","cluster_addr":"172.21.15.111:6827/3338566976","heartbeat_back_addr":"172.21.15.111:6831/3338566976","heartbeat_front_addr":"172.21.15.111:6829/3338566976","state":["exists","up"]},{"osd":11,"uuid":"b18fc5f3-1049-4871-b060-e99290273cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6825","nonce":578640428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6827","nonce":578640428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6831","nonce":578640428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6829","nonce":578640428}]},"public_addr":"172.21.15.134:6825/578640428","cluster_addr":"172.21.15.134:6827/578640428","heartbeat_back_addr":"172.21.15.134:6831/578640428","heartbeat_front_addr":"172.21.15.134:6829/578640428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:16:40.828608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:05.052528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:28.297316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:53.496497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:17.947259+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:41.712894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:07.105945+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:32.129688+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:56.008516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:22.239867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:47.334692+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:12.301276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/3535299028":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/2837479598":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/3010129696":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/2593601362":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/2519382069":"2024-04-23T13:15:04.279990+0000","172.21.15.12:6801/2009831940":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/856202072":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/3509412187":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/1802873236":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/1007749062":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3588480095":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1904962176":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/1802873236":"2024-04-23T13:24:06.155564+0000","172.21.15.12:6800/2009831940":"2024-04-23T13:15:04.279990+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:24:16.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.512+0000 7f485cbd5640 1 -- 172.21.15.12:0/1230942755 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f48400a3e40 msgr2=0x7f48400a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:16.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.512+0000 7f485cbd5640 1 --2- 172.21.15.12:0/1230942755 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f48400a3e40 0x7f48400a6300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4848009990 tx=0x7f4848009110 comp rx=0 tx=0).stop 2024-04-22T13:24:16.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.512+0000 7f485cbd5640 1 -- 172.21.15.12:0/1230942755 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4858103af0 msgr2=0x7f485819e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:16.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.512+0000 7f485cbd5640 1 --2- 172.21.15.12:0/1230942755 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4858103af0 0x7f485819e990 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f484c005940 tx=0x7f484c02d890 comp rx=0 tx=0).stop 2024-04-22T13:24:16.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.512+0000 7f485cbd5640 1 -- 172.21.15.12:0/1230942755 shutdown_connections 2024-04-22T13:24:16.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.512+0000 7f485cbd5640 1 --2- 172.21.15.12:0/1230942755 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f48400a3e40 0x7f48400a6300 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.512+0000 7f485cbd5640 1 --2- 172.21.15.12:0/1230942755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f485810c5a0 0x7f4858198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.512+0000 7f485cbd5640 1 --2- 172.21.15.12:0/1230942755 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4858103af0 0x7f485819e990 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.512+0000 7f485cbd5640 1 --2- 172.21.15.12:0/1230942755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f48581031d0 0x7f485819e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.512+0000 7f485cbd5640 1 -- 172.21.15.12:0/1230942755 >> 172.21.15.12:0/1230942755 conn(0x7f48580771d0 msgr2=0x7f48580fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:16.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.512+0000 7f485cbd5640 1 -- 172.21.15.12:0/1230942755 shutdown_connections 2024-04-22T13:24:16.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.512+0000 7f485cbd5640 1 -- 172.21.15.12:0/1230942755 wait complete. 2024-04-22T13:24:16.521 INFO:tasks.quiescer.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T13:17:32.372434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T13:22:02.768799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T13:22:03.578730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T13:24:16.521 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T13:24:16.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.630+0000 7f5664c9a640 1 -- 172.21.15.12:0/2948426072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5660107260 msgr2=0x7f566010ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:16.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.630+0000 7f5664c9a640 1 --2- 172.21.15.12:0/2948426072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5660107260 0x7f566010ee30 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5658009ca0 tx=0x7f565802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.630+0000 7f5664c9a640 1 -- 172.21.15.12:0/2948426072 shutdown_connections 2024-04-22T13:24:16.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.630+0000 7f5664c9a640 1 --2- 172.21.15.12:0/2948426072 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5660107260 0x7f566010ee30 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.630+0000 7f5664c9a640 1 --2- 172.21.15.12:0/2948426072 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5660102b40 0x7f5660106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.630+0000 7f5664c9a640 1 --2- 172.21.15.12:0/2948426072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5660102190 0x7f5660102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.630+0000 7f5664c9a640 1 -- 172.21.15.12:0/2948426072 >> 172.21.15.12:0/2948426072 conn(0x7f56600fd990 msgr2=0x7f56600ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:16.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.630+0000 7f5664c9a640 1 -- 172.21.15.12:0/2948426072 shutdown_connections 2024-04-22T13:24:16.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.630+0000 7f5664c9a640 1 -- 172.21.15.12:0/2948426072 wait complete. 2024-04-22T13:24:16.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.631+0000 7f5664c9a640 1 Processor -- start 2024-04-22T13:24:16.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.631+0000 7f5664c9a640 1 -- start start 2024-04-22T13:24:16.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.631+0000 7f5664c9a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5660102190 0x7f5660198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:16.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.631+0000 7f5664c9a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5660107260 0x7f5660198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:16.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.631+0000 7f5664c9a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f566019cf50 0x7f566019d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:16.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.631+0000 7f5664c9a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f56601121d0 con 0x7f5660107260 2024-04-22T13:24:16.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.631+0000 7f5664c9a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5660112050 con 0x7f566019cf50 2024-04-22T13:24:16.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.631+0000 7f5664c9a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5660112350 con 0x7f5660102190 2024-04-22T13:24:16.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.631+0000 7f5657fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5660107260 0x7f5660198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:16.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.631+0000 7f5657fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5660107260 0x7f5660198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59472/0 (socket says 172.21.15.12:59472) 2024-04-22T13:24:16.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.631+0000 7f5657fff640 1 -- 172.21.15.12:0/3844291435 learned_addr learned my addr 172.21.15.12:0/3844291435 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:16.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.631+0000 7f565ed9d640 1 --2- 172.21.15.12:0/3844291435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f566019cf50 0x7f566019d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:16.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.631+0000 7f565e59c640 1 --2- 172.21.15.12:0/3844291435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5660102190 0x7f5660198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:16.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.631+0000 7f5657fff640 1 -- 172.21.15.12:0/3844291435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5660102190 msgr2=0x7f5660198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:16.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.631+0000 7f5657fff640 1 --2- 172.21.15.12:0/3844291435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5660102190 0x7f5660198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.631+0000 7f5657fff640 1 -- 172.21.15.12:0/3844291435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f566019cf50 msgr2=0x7f566019d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:16.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.631+0000 7f5657fff640 1 --2- 172.21.15.12:0/3844291435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f566019cf50 0x7f566019d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.631+0000 7f5657fff640 1 -- 172.21.15.12:0/3844291435 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566019da40 con 0x7f5660107260 2024-04-22T13:24:16.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.631+0000 7f5657fff640 1 --2- 172.21.15.12:0/3844291435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5660107260 0x7f5660198bd0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f564800c970 tx=0x7f564800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:16.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.632+0000 7f56577fe640 1 -- 172.21.15.12:0/3844291435 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5648014070 con 0x7f5660107260 2024-04-22T13:24:16.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.632+0000 7f56577fe640 1 -- 172.21.15.12:0/3844291435 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5648004440 con 0x7f5660107260 2024-04-22T13:24:16.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.632+0000 7f56577fe640 1 -- 172.21.15.12:0/3844291435 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5648003e50 con 0x7f5660107260 2024-04-22T13:24:16.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.632+0000 7f5664c9a640 1 -- 172.21.15.12:0/3844291435 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f566019dd30 con 0x7f5660107260 2024-04-22T13:24:16.632 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.632+0000 7f5664c9a640 1 -- 172.21.15.12:0/3844291435 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56601a54d0 con 0x7f5660107260 2024-04-22T13:24:16.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.633+0000 7f5664c9a640 1 -- 172.21.15.12:0/3844291435 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56600691c0 con 0x7f5660107260 2024-04-22T13:24:16.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.634+0000 7f56577fe640 1 -- 172.21.15.12:0/3844291435 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f5648017070 con 0x7f5660107260 2024-04-22T13:24:16.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.634+0000 7f56577fe640 1 --2- 172.21.15.12:0/3844291435 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f564c0a3fe0 0x7f564c0a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:16.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.635+0000 7f565e59c640 1 --2- 172.21.15.12:0/3844291435 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f564c0a3fe0 0x7f564c0a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:16.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.635+0000 7f56577fe640 1 -- 172.21.15.12:0/3844291435 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f56480d0820 con 0x7f5660107260 2024-04-22T13:24:16.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.635+0000 7f565e59c640 1 --2- 172.21.15.12:0/3844291435 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f564c0a3fe0 0x7f564c0a64a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f56500058c0 tx=0x7f565000a250 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:16.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.639+0000 7f56577fe640 1 -- 172.21.15.12:0/3844291435 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f564801e020 con 0x7f5660107260 2024-04-22T13:24:16.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.830+0000 7f5664c9a640 1 -- 172.21.15.12:0/3844291435 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5660102b40 con 0x7f5660107260 2024-04-22T13:24:16.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.831+0000 7f56577fe640 1 -- 172.21.15.12:0/3844291435 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f564809cc60 con 0x7f5660107260 2024-04-22T13:24:16.828 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-22T13:24:16.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.833+0000 7f5664c9a640 1 -- 172.21.15.12:0/3844291435 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f564c0a3fe0 msgr2=0x7f564c0a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:16.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.833+0000 7f5664c9a640 1 --2- 172.21.15.12:0/3844291435 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f564c0a3fe0 0x7f564c0a64a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f56500058c0 tx=0x7f565000a250 comp rx=0 tx=0).stop 2024-04-22T13:24:16.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.833+0000 7f5664c9a640 1 -- 172.21.15.12:0/3844291435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5660107260 msgr2=0x7f5660198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:16.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.833+0000 7f5664c9a640 1 --2- 172.21.15.12:0/3844291435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5660107260 0x7f5660198bd0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f564800c970 tx=0x7f564800ce30 comp rx=0 tx=0).stop 2024-04-22T13:24:16.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.833+0000 7f5664c9a640 1 -- 172.21.15.12:0/3844291435 shutdown_connections 2024-04-22T13:24:16.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.833+0000 7f5664c9a640 1 --2- 172.21.15.12:0/3844291435 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f564c0a3fe0 0x7f564c0a64a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.833+0000 7f5664c9a640 1 --2- 172.21.15.12:0/3844291435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f566019cf50 0x7f566019d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.833+0000 7f5664c9a640 1 --2- 172.21.15.12:0/3844291435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5660107260 0x7f5660198bd0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.833+0000 7f5664c9a640 1 --2- 172.21.15.12:0/3844291435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5660102190 0x7f5660198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.833+0000 7f5664c9a640 1 -- 172.21.15.12:0/3844291435 >> 172.21.15.12:0/3844291435 conn(0x7f56600fd990 msgr2=0x7f56600ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:16.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.833+0000 7f5664c9a640 1 -- 172.21.15.12:0/3844291435 shutdown_connections 2024-04-22T13:24:16.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.833+0000 7f5664c9a640 1 -- 172.21.15.12:0/3844291435 wait complete. 2024-04-22T13:24:16.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:16 smithi012 ceph-mon[28265]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:24:16.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:16 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3993057345' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:24:16.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:16 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1316559422' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:24:16.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:16 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1230942755' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:24:16.841 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T13:24:16.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.950+0000 7f9f62626640 1 -- 172.21.15.12:0/891125233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f5c106160 msgr2=0x7f9f5c10dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:16.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.950+0000 7f9f62626640 1 --2- 172.21.15.12:0/891125233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f5c106160 0x7f9f5c10dd30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9f54009ca0 tx=0x7f9f5402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.950+0000 7f9f62626640 1 -- 172.21.15.12:0/891125233 shutdown_connections 2024-04-22T13:24:16.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.950+0000 7f9f62626640 1 --2- 172.21.15.12:0/891125233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f5c106160 0x7f9f5c10dd30 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.950+0000 7f9f62626640 1 --2- 172.21.15.12:0/891125233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f5c101a40 0x7f9f5c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.950+0000 7f9f62626640 1 --2- 172.21.15.12:0/891125233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f5c101090 0x7f9f5c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.950+0000 7f9f62626640 1 -- 172.21.15.12:0/891125233 >> 172.21.15.12:0/891125233 conn(0x7f9f5c077f50 msgr2=0x7f9f5c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:16.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.950+0000 7f9f62626640 1 -- 172.21.15.12:0/891125233 shutdown_connections 2024-04-22T13:24:16.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.950+0000 7f9f62626640 1 -- 172.21.15.12:0/891125233 wait complete. 2024-04-22T13:24:16.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.950+0000 7f9f62626640 1 Processor -- start 2024-04-22T13:24:16.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.950+0000 7f9f62626640 1 -- start start 2024-04-22T13:24:16.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.951+0000 7f9f62626640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f5c101090 0x7f9f5c198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:16.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.951+0000 7f9f62626640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f5c106160 0x7f9f5c198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:16.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.951+0000 7f9f62626640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f5c19cff0 0x7f9f5c19d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:16.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.951+0000 7f9f62626640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c1110b0 con 0x7f9f5c19cff0 2024-04-22T13:24:16.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.951+0000 7f9f62626640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c110f30 con 0x7f9f5c101090 2024-04-22T13:24:16.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.951+0000 7f9f62626640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c111230 con 0x7f9f5c106160 2024-04-22T13:24:16.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.951+0000 7f9f60bc3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f5c19cff0 0x7f9f5c19d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:16.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.951+0000 7f9f60bc3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f5c19cff0 0x7f9f5c19d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59484/0 (socket says 172.21.15.12:59484) 2024-04-22T13:24:16.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.951+0000 7f9f60bc3640 1 -- 172.21.15.12:0/2665104165 learned_addr learned my addr 172.21.15.12:0/2665104165 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:16.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.951+0000 7f9f5bfff640 1 --2- 172.21.15.12:0/2665104165 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f5c101090 0x7f9f5c198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:16.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.951+0000 7f9f5b7fe640 1 --2- 172.21.15.12:0/2665104165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f5c106160 0x7f9f5c198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:16.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.951+0000 7f9f60bc3640 1 -- 172.21.15.12:0/2665104165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f5c106160 msgr2=0x7f9f5c198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:16.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.951+0000 7f9f60bc3640 1 --2- 172.21.15.12:0/2665104165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f5c106160 0x7f9f5c198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.951+0000 7f9f60bc3640 1 -- 172.21.15.12:0/2665104165 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f5c101090 msgr2=0x7f9f5c198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:16.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.951+0000 7f9f60bc3640 1 --2- 172.21.15.12:0/2665104165 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f5c101090 0x7f9f5c198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:16.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.951+0000 7f9f60bc3640 1 -- 172.21.15.12:0/2665104165 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5c19dae0 con 0x7f9f5c19cff0 2024-04-22T13:24:16.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.951+0000 7f9f60bc3640 1 --2- 172.21.15.12:0/2665104165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f5c19cff0 0x7f9f5c19d4a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9f54005e30 tx=0x7f9f54034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:16.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.952+0000 7f9f597fa640 1 -- 172.21.15.12:0/2665104165 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f54039040 con 0x7f9f5c19cff0 2024-04-22T13:24:16.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.952+0000 7f9f597fa640 1 -- 172.21.15.12:0/2665104165 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f54044070 con 0x7f9f5c19cff0 2024-04-22T13:24:16.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.952+0000 7f9f597fa640 1 -- 172.21.15.12:0/2665104165 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f54003e60 con 0x7f9f5c19cff0 2024-04-22T13:24:16.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.952+0000 7f9f62626640 1 -- 172.21.15.12:0/2665104165 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f5c19dd70 con 0x7f9f5c19cff0 2024-04-22T13:24:16.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.952+0000 7f9f62626640 1 -- 172.21.15.12:0/2665104165 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f5c1a5530 con 0x7f9f5c19cff0 2024-04-22T13:24:16.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.953+0000 7f9f62626640 1 -- 172.21.15.12:0/2665104165 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f5c068d70 con 0x7f9f5c19cff0 2024-04-22T13:24:16.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.954+0000 7f9f597fa640 1 -- 172.21.15.12:0/2665104165 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f9f540324c0 con 0x7f9f5c19cff0 2024-04-22T13:24:16.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.954+0000 7f9f597fa640 1 --2- 172.21.15.12:0/2665104165 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f9f440a3e40 0x7f9f440a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:16.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.955+0000 7f9f5bfff640 1 --2- 172.21.15.12:0/2665104165 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f9f440a3e40 0x7f9f440a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:16.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.955+0000 7f9f597fa640 1 -- 172.21.15.12:0/2665104165 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f9f540f35e0 con 0x7f9f5c19cff0 2024-04-22T13:24:16.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.955+0000 7f9f5bfff640 1 --2- 172.21.15.12:0/2665104165 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f9f440a3e40 0x7f9f440a6300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c005d00 tx=0x7f9f4c00b410 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:16.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:16.958+0000 7f9f597fa640 1 -- 172.21.15.12:0/2665104165 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9f540bfb90 con 0x7f9f5c19cff0 2024-04-22T13:24:16.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:16 smithi111 ceph-mon[31361]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:24:16.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:16 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/3993057345' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:24:16.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:16 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1316559422' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:24:16.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:16 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1230942755' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:24:17.079 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:16 smithi134 ceph-mon[31338]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:24:17.079 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:16 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/3993057345' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:24:17.079 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:16 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1316559422' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:24:17.079 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:16 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1230942755' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:24:17.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.142+0000 7f9f62626640 1 -- 172.21.15.12:0/2665104165 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9f5c101a40 con 0x7f9f5c19cff0 2024-04-22T13:24:17.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.143+0000 7f9f597fa640 1 -- 172.21.15.12:0/2665104165 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f9f540bf490 con 0x7f9f5c19cff0 2024-04-22T13:24:17.140 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-22T13:24:17.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.144+0000 7f9f62626640 1 -- 172.21.15.12:0/2665104165 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f9f440a3e40 msgr2=0x7f9f440a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:17.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.144+0000 7f9f62626640 1 --2- 172.21.15.12:0/2665104165 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f9f440a3e40 0x7f9f440a6300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c005d00 tx=0x7f9f4c00b410 comp rx=0 tx=0).stop 2024-04-22T13:24:17.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.144+0000 7f9f62626640 1 -- 172.21.15.12:0/2665104165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f5c19cff0 msgr2=0x7f9f5c19d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:17.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.144+0000 7f9f62626640 1 --2- 172.21.15.12:0/2665104165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f5c19cff0 0x7f9f5c19d4a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9f54005e30 tx=0x7f9f54034760 comp rx=0 tx=0).stop 2024-04-22T13:24:17.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.145+0000 7f9f62626640 1 -- 172.21.15.12:0/2665104165 shutdown_connections 2024-04-22T13:24:17.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.145+0000 7f9f62626640 1 --2- 172.21.15.12:0/2665104165 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f9f440a3e40 0x7f9f440a6300 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:17.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.145+0000 7f9f62626640 1 --2- 172.21.15.12:0/2665104165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9f5c19cff0 0x7f9f5c19d4a0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:17.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.145+0000 7f9f62626640 1 --2- 172.21.15.12:0/2665104165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9f5c106160 0x7f9f5c198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:17.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.145+0000 7f9f62626640 1 --2- 172.21.15.12:0/2665104165 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9f5c101090 0x7f9f5c198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:17.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.145+0000 7f9f62626640 1 -- 172.21.15.12:0/2665104165 >> 172.21.15.12:0/2665104165 conn(0x7f9f5c077f50 msgr2=0x7f9f5c1000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:17.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.145+0000 7f9f62626640 1 -- 172.21.15.12:0/2665104165 shutdown_connections 2024-04-22T13:24:17.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.145+0000 7f9f62626640 1 -- 172.21.15.12:0/2665104165 wait complete. 2024-04-22T13:24:17.153 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T13:24:17.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.261+0000 7f6f7823f640 1 -- 172.21.15.12:0/1796951918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f70102190 msgr2=0x7f6f70102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:17.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.261+0000 7f6f7823f640 1 --2- 172.21.15.12:0/1796951918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f70102190 0x7f6f70102570 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c009900 tx=0x7f6f6c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:17.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.262+0000 7f6f7823f640 1 -- 172.21.15.12:0/1796951918 shutdown_connections 2024-04-22T13:24:17.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.262+0000 7f6f7823f640 1 --2- 172.21.15.12:0/1796951918 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f70107260 0x7f6f7010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:17.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.262+0000 7f6f7823f640 1 --2- 172.21.15.12:0/1796951918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f70102b40 0x7f6f70106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:17.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.262+0000 7f6f7823f640 1 --2- 172.21.15.12:0/1796951918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f70102190 0x7f6f70102570 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:17.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.262+0000 7f6f7823f640 1 -- 172.21.15.12:0/1796951918 >> 172.21.15.12:0/1796951918 conn(0x7f6f700793a0 msgr2=0x7f6f700797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:17.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.262+0000 7f6f7823f640 1 -- 172.21.15.12:0/1796951918 shutdown_connections 2024-04-22T13:24:17.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.262+0000 7f6f7823f640 1 -- 172.21.15.12:0/1796951918 wait complete. 2024-04-22T13:24:17.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.262+0000 7f6f7823f640 1 Processor -- start 2024-04-22T13:24:17.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.262+0000 7f6f7823f640 1 -- start start 2024-04-22T13:24:17.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.263+0000 7f6f7823f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f70102190 0x7f6f7019e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:17.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.263+0000 7f6f7823f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f70107260 0x7f6f7019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:17.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.263+0000 7f6f7823f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f701985d0 0x7f6f70198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:17.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.263+0000 7f6f7823f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6f70112210 con 0x7f6f701985d0 2024-04-22T13:24:17.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.263+0000 7f6f7823f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6f70112090 con 0x7f6f70107260 2024-04-22T13:24:17.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.263+0000 7f6f7823f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6f70112390 con 0x7f6f70102190 2024-04-22T13:24:17.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.263+0000 7f6f767dc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f701985d0 0x7f6f70198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:17.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.263+0000 7f6f767dc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f701985d0 0x7f6f70198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59488/0 (socket says 172.21.15.12:59488) 2024-04-22T13:24:17.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.263+0000 7f6f75fdb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f70102190 0x7f6f7019e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:17.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.263+0000 7f6f767dc640 1 -- 172.21.15.12:0/19074878 learned_addr learned my addr 172.21.15.12:0/19074878 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:17.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.263+0000 7f6f757da640 1 --2- 172.21.15.12:0/19074878 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f70107260 0x7f6f7019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:17.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.263+0000 7f6f767dc640 1 -- 172.21.15.12:0/19074878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f70102190 msgr2=0x7f6f7019e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:17.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.263+0000 7f6f767dc640 1 --2- 172.21.15.12:0/19074878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f70102190 0x7f6f7019e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:17.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.263+0000 7f6f767dc640 1 -- 172.21.15.12:0/19074878 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f70107260 msgr2=0x7f6f7019e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:17.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.263+0000 7f6f767dc640 1 --2- 172.21.15.12:0/19074878 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f70107260 0x7f6f7019e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:17.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.263+0000 7f6f767dc640 1 -- 172.21.15.12:0/19074878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f701990f0 con 0x7f6f701985d0 2024-04-22T13:24:17.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.263+0000 7f6f757da640 1 --2- 172.21.15.12:0/19074878 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f70107260 0x7f6f7019e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:17.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.263+0000 7f6f767dc640 1 --2- 172.21.15.12:0/19074878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f701985d0 0x7f6f70198a80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6f68009fd0 tx=0x7f6f6800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:17.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.263+0000 7f6f66ffd640 1 -- 172.21.15.12:0/19074878 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f68017070 con 0x7f6f701985d0 2024-04-22T13:24:17.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.263+0000 7f6f7823f640 1 -- 172.21.15.12:0/19074878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f701993e0 con 0x7f6f701985d0 2024-04-22T13:24:17.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.263+0000 7f6f66ffd640 1 -- 172.21.15.12:0/19074878 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f68003e20 con 0x7f6f701985d0 2024-04-22T13:24:17.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.263+0000 7f6f66ffd640 1 -- 172.21.15.12:0/19074878 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f68010460 con 0x7f6f701985d0 2024-04-22T13:24:17.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.263+0000 7f6f7823f640 1 -- 172.21.15.12:0/19074878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f701a5630 con 0x7f6f701985d0 2024-04-22T13:24:17.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.264+0000 7f6f7823f640 1 -- 172.21.15.12:0/19074878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f700691c0 con 0x7f6f701985d0 2024-04-22T13:24:17.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.266+0000 7f6f66ffd640 1 -- 172.21.15.12:0/19074878 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f6f68003970 con 0x7f6f701985d0 2024-04-22T13:24:17.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.267+0000 7f6f66ffd640 1 --2- 172.21.15.12:0/19074878 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6f5c0a3e40 0x7f6f5c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:17.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.267+0000 7f6f66ffd640 1 -- 172.21.15.12:0/19074878 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f6f680d07b0 con 0x7f6f701985d0 2024-04-22T13:24:17.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.267+0000 7f6f75fdb640 1 --2- 172.21.15.12:0/19074878 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6f5c0a3e40 0x7f6f5c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:17.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.268+0000 7f6f75fdb640 1 --2- 172.21.15.12:0/19074878 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6f5c0a3e40 0x7f6f5c0a6300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c005ab0 tx=0x7f6f6c005a20 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:17.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.271+0000 7f6f66ffd640 1 -- 172.21.15.12:0/19074878 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6f6809cbf0 con 0x7f6f701985d0 2024-04-22T13:24:17.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.456+0000 7f6f7823f640 1 -- 172.21.15.12:0/19074878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6f70102b40 con 0x7f6f701985d0 2024-04-22T13:24:17.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.457+0000 7f6f66ffd640 1 -- 172.21.15.12:0/19074878 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f6f6809c4f0 con 0x7f6f701985d0 2024-04-22T13:24:17.454 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-22T13:24:17.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.459+0000 7f6f7823f640 1 -- 172.21.15.12:0/19074878 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6f5c0a3e40 msgr2=0x7f6f5c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:17.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.459+0000 7f6f7823f640 1 --2- 172.21.15.12:0/19074878 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6f5c0a3e40 0x7f6f5c0a6300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c005ab0 tx=0x7f6f6c005a20 comp rx=0 tx=0).stop 2024-04-22T13:24:17.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.459+0000 7f6f7823f640 1 -- 172.21.15.12:0/19074878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f701985d0 msgr2=0x7f6f70198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:17.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.459+0000 7f6f7823f640 1 --2- 172.21.15.12:0/19074878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f701985d0 0x7f6f70198a80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6f68009fd0 tx=0x7f6f6800b260 comp rx=0 tx=0).stop 2024-04-22T13:24:17.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.459+0000 7f6f7823f640 1 -- 172.21.15.12:0/19074878 shutdown_connections 2024-04-22T13:24:17.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.459+0000 7f6f7823f640 1 --2- 172.21.15.12:0/19074878 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6f5c0a3e40 0x7f6f5c0a6300 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:17.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.459+0000 7f6f7823f640 1 --2- 172.21.15.12:0/19074878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f701985d0 0x7f6f70198a80 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:17.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.459+0000 7f6f7823f640 1 --2- 172.21.15.12:0/19074878 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f70107260 0x7f6f7019e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:17.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.459+0000 7f6f7823f640 1 --2- 172.21.15.12:0/19074878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f70102190 0x7f6f7019e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:17.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.459+0000 7f6f7823f640 1 -- 172.21.15.12:0/19074878 >> 172.21.15.12:0/19074878 conn(0x7f6f700793a0 msgr2=0x7f6f700fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:17.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.459+0000 7f6f7823f640 1 -- 172.21.15.12:0/19074878 shutdown_connections 2024-04-22T13:24:17.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.459+0000 7f6f7823f640 1 -- 172.21.15.12:0/19074878 wait complete. 2024-04-22T13:24:17.467 INFO:tasks.quiescer.ceph_manager:waiting for clean 2024-04-22T13:24:17.467 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T13:24:17.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.572+0000 7f65cadef640 1 -- 172.21.15.12:0/3074061489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65c41010b0 msgr2=0x7f65c4101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:17.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.572+0000 7f65cadef640 1 --2- 172.21.15.12:0/3074061489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65c41010b0 0x7f65c4101490 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009900 tx=0x7f65b802bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:17.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.572+0000 7f65cadef640 1 -- 172.21.15.12:0/3074061489 shutdown_connections 2024-04-22T13:24:17.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.572+0000 7f65cadef640 1 --2- 172.21.15.12:0/3074061489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65c4106180 0x7f65c410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:17.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.572+0000 7f65cadef640 1 --2- 172.21.15.12:0/3074061489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65c4101a60 0x7f65c4105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:17.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.572+0000 7f65cadef640 1 --2- 172.21.15.12:0/3074061489 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65c41010b0 0x7f65c4101490 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:17.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.572+0000 7f65cadef640 1 -- 172.21.15.12:0/3074061489 >> 172.21.15.12:0/3074061489 conn(0x7f65c40771d0 msgr2=0x7f65c40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:17.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.572+0000 7f65cadef640 1 -- 172.21.15.12:0/3074061489 shutdown_connections 2024-04-22T13:24:17.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.572+0000 7f65cadef640 1 -- 172.21.15.12:0/3074061489 wait complete. 2024-04-22T13:24:17.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.573+0000 7f65cadef640 1 Processor -- start 2024-04-22T13:24:17.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.573+0000 7f65cadef640 1 -- start start 2024-04-22T13:24:17.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.573+0000 7f65cadef640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65c41010b0 0x7f65c41986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:17.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.573+0000 7f65cadef640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65c4106180 0x7f65c4198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:17.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.573+0000 7f65cadef640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65c419cfb0 0x7f65c419d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:17.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.573+0000 7f65cadef640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f65c41110e0 con 0x7f65c4106180 2024-04-22T13:24:17.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.573+0000 7f65cadef640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f65c4110f60 con 0x7f65c419cfb0 2024-04-22T13:24:17.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.573+0000 7f65cadef640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f65c4111260 con 0x7f65c41010b0 2024-04-22T13:24:17.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.573+0000 7f65c3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65c4106180 0x7f65c4198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:17.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.573+0000 7f65c3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65c4106180 0x7f65c4198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59496/0 (socket says 172.21.15.12:59496) 2024-04-22T13:24:17.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.573+0000 7f65c3fff640 1 -- 172.21.15.12:0/3288935746 learned_addr learned my addr 172.21.15.12:0/3288935746 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:17.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.573+0000 7f65c8b8b640 1 --2- 172.21.15.12:0/3288935746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65c41010b0 0x7f65c41986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:17.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.573+0000 7f65c938c640 1 --2- 172.21.15.12:0/3288935746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65c419cfb0 0x7f65c419d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:17.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.574+0000 7f65c3fff640 1 -- 172.21.15.12:0/3288935746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65c41010b0 msgr2=0x7f65c41986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:17.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.574+0000 7f65c3fff640 1 --2- 172.21.15.12:0/3288935746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65c41010b0 0x7f65c41986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:17.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.574+0000 7f65c3fff640 1 -- 172.21.15.12:0/3288935746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65c419cfb0 msgr2=0x7f65c419d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:17.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.574+0000 7f65c3fff640 1 --2- 172.21.15.12:0/3288935746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65c419cfb0 0x7f65c419d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:17.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.574+0000 7f65c3fff640 1 -- 172.21.15.12:0/3288935746 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c419daa0 con 0x7f65c4106180 2024-04-22T13:24:17.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.574+0000 7f65c3fff640 1 --2- 172.21.15.12:0/3288935746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65c4106180 0x7f65c4198c30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f65b000d740 tx=0x7f65b000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:17.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.574+0000 7f65c1ffb640 1 -- 172.21.15.12:0/3288935746 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b0017070 con 0x7f65c4106180 2024-04-22T13:24:17.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.574+0000 7f65c1ffb640 1 -- 172.21.15.12:0/3288935746 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65b000de50 con 0x7f65c4106180 2024-04-22T13:24:17.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.574+0000 7f65cadef640 1 -- 172.21.15.12:0/3288935746 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c419dd90 con 0x7f65c4106180 2024-04-22T13:24:17.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.574+0000 7f65c1ffb640 1 -- 172.21.15.12:0/3288935746 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b0004ce0 con 0x7f65c4106180 2024-04-22T13:24:17.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.574+0000 7f65cadef640 1 -- 172.21.15.12:0/3288935746 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c41a5530 con 0x7f65c4106180 2024-04-22T13:24:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.575+0000 7f65cadef640 1 -- 172.21.15.12:0/3288935746 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65c4069230 con 0x7f65c4106180 2024-04-22T13:24:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.576+0000 7f65c1ffb640 1 -- 172.21.15.12:0/3288935746 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f65b0004e80 con 0x7f65c4106180 2024-04-22T13:24:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.577+0000 7f65c1ffb640 1 --2- 172.21.15.12:0/3288935746 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f65b40a3e40 0x7f65b40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.577+0000 7f65c8b8b640 1 --2- 172.21.15.12:0/3288935746 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f65b40a3e40 0x7f65b40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.577+0000 7f65c1ffb640 1 -- 172.21.15.12:0/3288935746 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f65b00d0780 con 0x7f65c4106180 2024-04-22T13:24:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.578+0000 7f65c8b8b640 1 --2- 172.21.15.12:0/3288935746 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f65b40a3e40 0x7f65b40a6300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f65b8006010 tx=0x7f65b8005e40 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:17.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.581+0000 7f65c1ffb640 1 -- 172.21.15.12:0/3288935746 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f65b009c5d0 con 0x7f65c4106180 2024-04-22T13:24:17.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.764+0000 7f65cadef640 1 -- 172.21.15.12:0/3288935746 --> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f65c4101a60 con 0x7f65b40a3e40 2024-04-22T13:24:17.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.772+0000 7f65c1ffb640 1 -- 172.21.15.12:0/3288935746 <== mgr.24968 v2:172.21.15.12:6800/1615661596 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257790 (secure 0 0 0) 0x7f65c4101a60 con 0x7f65b40a3e40 2024-04-22T13:24:17.772 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:24:17.773 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-22T13:24:17.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.775+0000 7f65cadef640 1 -- 172.21.15.12:0/3288935746 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f65b40a3e40 msgr2=0x7f65b40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:17.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.775+0000 7f65cadef640 1 --2- 172.21.15.12:0/3288935746 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f65b40a3e40 0x7f65b40a6300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f65b8006010 tx=0x7f65b8005e40 comp rx=0 tx=0).stop 2024-04-22T13:24:17.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.775+0000 7f65cadef640 1 -- 172.21.15.12:0/3288935746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65c4106180 msgr2=0x7f65c4198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:17.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.775+0000 7f65cadef640 1 --2- 172.21.15.12:0/3288935746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65c4106180 0x7f65c4198c30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f65b000d740 tx=0x7f65b000dc00 comp rx=0 tx=0).stop 2024-04-22T13:24:17.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.775+0000 7f65cadef640 1 -- 172.21.15.12:0/3288935746 shutdown_connections 2024-04-22T13:24:17.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.775+0000 7f65cadef640 1 --2- 172.21.15.12:0/3288935746 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f65b40a3e40 0x7f65b40a6300 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:17.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.775+0000 7f65cadef640 1 --2- 172.21.15.12:0/3288935746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65c419cfb0 0x7f65c419d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:17.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.775+0000 7f65cadef640 1 --2- 172.21.15.12:0/3288935746 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65c4106180 0x7f65c4198c30 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:17.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.775+0000 7f65cadef640 1 --2- 172.21.15.12:0/3288935746 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65c41010b0 0x7f65c41986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:17.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.775+0000 7f65cadef640 1 -- 172.21.15.12:0/3288935746 >> 172.21.15.12:0/3288935746 conn(0x7f65c40771d0 msgr2=0x7f65c406aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:17.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.775+0000 7f65cadef640 1 -- 172.21.15.12:0/3288935746 shutdown_connections 2024-04-22T13:24:17.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.775+0000 7f65cadef640 1 -- 172.21.15.12:0/3288935746 wait complete. 2024-04-22T13:24:17.782 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:17 smithi012 ceph-mon[28265]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1 KiB/s wr, 20 op/s 2024-04-22T13:24:17.782 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:17 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3844291435' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:24:17.782 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:17 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2665104165' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:24:17.783 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:17 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/19074878' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:24:17.783 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-04-22T13:24:16.177526+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1193446,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1592,"num_read_kb":1126,"num_write":580,"num_write_kb":4411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":955,"ondisk_log_size":955,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381080,"kb_used_data":15264,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124560744,"statfs":{"total":1151940427776,"available":1151550201856,"internally_reserved":0,"allocated":15630336,"data_stored":9230754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4084,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":156,"num_read_kb":243,"num_write":29,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.992501"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418512+0000","last_change":"2024-04-22T13:22:05.254388+0000","last_active":"2024-04-22T13:24:06.418512+0000","last_peered":"2024-04-22T13:24:06.418512+0000","last_clean":"2024-04-22T13:24:06.418512+0000","last_became_active":"2024-04-22T13:22:05.254042+0000","last_became_peered":"2024-04-22T13:22:05.254042+0000","last_unstale":"2024-04-22T13:24:06.418512+0000","last_undegraded":"2024-04-22T13:24:06.418512+0000","last_fullsized":"2024-04-22T13:24:06.418512+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:07:56.139653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.412069+0000","last_change":"2024-04-22T13:22:05.247879+0000","last_active":"2024-04-22T13:24:06.412069+0000","last_peered":"2024-04-22T13:24:06.412069+0000","last_clean":"2024-04-22T13:24:06.412069+0000","last_became_active":"2024-04-22T13:22:05.247508+0000","last_became_peered":"2024-04-22T13:22:05.247508+0000","last_unstale":"2024-04-22T13:24:06.412069+0000","last_undegraded":"2024-04-22T13:24:06.412069+0000","last_fullsized":"2024-04-22T13:24:06.412069+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:55:51.928658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411912+0000","last_change":"2024-04-22T13:22:05.257970+0000","last_active":"2024-04-22T13:24:06.411912+0000","last_peered":"2024-04-22T13:24:06.411912+0000","last_clean":"2024-04-22T13:24:06.411912+0000","last_became_active":"2024-04-22T13:22:05.257652+0000","last_became_peered":"2024-04-22T13:22:05.257652+0000","last_unstale":"2024-04-22T13:24:06.411912+0000","last_undegraded":"2024-04-22T13:24:06.411912+0000","last_fullsized":"2024-04-22T13:24:06.411912+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:42:17.476349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411332+0000","last_change":"2024-04-22T13:22:05.249186+0000","last_active":"2024-04-22T13:24:06.411332+0000","last_peered":"2024-04-22T13:24:06.411332+0000","last_clean":"2024-04-22T13:24:06.411332+0000","last_became_active":"2024-04-22T13:22:05.248814+0000","last_became_peered":"2024-04-22T13:22:05.248814+0000","last_unstale":"2024-04-22T13:24:06.411332+0000","last_undegraded":"2024-04-22T13:24:06.411332+0000","last_fullsized":"2024-04-22T13:24:06.411332+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:57.450794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258738+0000","last_change":"2024-04-22T13:22:05.251953+0000","last_active":"2024-04-22T13:24:06.258738+0000","last_peered":"2024-04-22T13:24:06.258738+0000","last_clean":"2024-04-22T13:24:06.258738+0000","last_became_active":"2024-04-22T13:22:05.251547+0000","last_became_peered":"2024-04-22T13:22:05.251547+0000","last_unstale":"2024-04-22T13:24:06.258738+0000","last_undegraded":"2024-04-22T13:24:06.258738+0000","last_fullsized":"2024-04-22T13:24:06.258738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:36:34.809878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.266346+0000","last_change":"2024-04-22T13:23:05.510044+0000","last_active":"2024-04-22T13:24:06.266346+0000","last_peered":"2024-04-22T13:24:06.266346+0000","last_clean":"2024-04-22T13:24:06.266346+0000","last_became_active":"2024-04-22T13:23:05.509677+0000","last_became_peered":"2024-04-22T13:23:05.509677+0000","last_unstale":"2024-04-22T13:24:06.266346+0000","last_undegraded":"2024-04-22T13:24:06.266346+0000","last_fullsized":"2024-04-22T13:24:06.266346+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:46.979291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409036+0000","last_change":"2024-04-22T13:22:05.250418+0000","last_active":"2024-04-22T13:24:06.409036+0000","last_peered":"2024-04-22T13:24:06.409036+0000","last_clean":"2024-04-22T13:24:06.409036+0000","last_became_active":"2024-04-22T13:22:05.250097+0000","last_became_peered":"2024-04-22T13:22:05.250097+0000","last_unstale":"2024-04-22T13:24:06.409036+0000","last_undegraded":"2024-04-22T13:24:06.409036+0000","last_fullsized":"2024-04-22T13:24:06.409036+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:34:59.598950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258759+0000","last_change":"2024-04-22T13:22:05.244762+0000","last_active":"2024-04-22T13:24:06.258759+0000","last_peered":"2024-04-22T13:24:06.258759+0000","last_clean":"2024-04-22T13:24:06.258759+0000","last_became_active":"2024-04-22T13:22:05.244430+0000","last_became_peered":"2024-04-22T13:22:05.244430+0000","last_unstale":"2024-04-22T13:24:06.258759+0000","last_undegraded":"2024-04-22T13:24:06.258759+0000","last_fullsized":"2024-04-22T13:24:06.258759+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:33:01.582851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164321+0000","last_change":"2024-04-22T13:22:05.257596+0000","last_active":"2024-04-22T13:24:06.164321+0000","last_peered":"2024-04-22T13:24:06.164321+0000","last_clean":"2024-04-22T13:24:06.164321+0000","last_became_active":"2024-04-22T13:22:05.257277+0000","last_became_peered":"2024-04-22T13:22:05.257277+0000","last_unstale":"2024-04-22T13:24:06.164321+0000","last_undegraded":"2024-04-22T13:24:06.164321+0000","last_fullsized":"2024-04-22T13:24:06.164321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:09:44.715965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.266708+0000","last_change":"2024-04-22T13:22:05.252177+0000","last_active":"2024-04-22T13:24:06.266708+0000","last_peered":"2024-04-22T13:24:06.266708+0000","last_clean":"2024-04-22T13:24:06.266708+0000","last_became_active":"2024-04-22T13:22:05.251794+0000","last_became_peered":"2024-04-22T13:22:05.251794+0000","last_unstale":"2024-04-22T13:24:06.266708+0000","last_undegraded":"2024-04-22T13:24:06.266708+0000","last_fullsized":"2024-04-22T13:24:06.266708+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:08:59.500540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409476+0000","last_change":"2024-04-22T13:22:05.256209+0000","last_active":"2024-04-22T13:24:06.409476+0000","last_peered":"2024-04-22T13:24:06.409476+0000","last_clean":"2024-04-22T13:24:06.409476+0000","last_became_active":"2024-04-22T13:22:05.255504+0000","last_became_peered":"2024-04-22T13:22:05.255504+0000","last_unstale":"2024-04-22T13:24:06.409476+0000","last_undegraded":"2024-04-22T13:24:06.409476+0000","last_fullsized":"2024-04-22T13:24:06.409476+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:32:08.190921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.256744+0000","last_change":"2024-04-22T13:23:05.509837+0000","last_active":"2024-04-22T13:24:06.256744+0000","last_peered":"2024-04-22T13:24:06.256744+0000","last_clean":"2024-04-22T13:24:06.256744+0000","last_became_active":"2024-04-22T13:23:05.509421+0000","last_became_peered":"2024-04-22T13:23:05.509421+0000","last_unstale":"2024-04-22T13:24:06.256744+0000","last_undegraded":"2024-04-22T13:24:06.256744+0000","last_fullsized":"2024-04-22T13:24:06.256744+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:30:01.103373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162449+0000","last_change":"2024-04-22T13:22:05.246667+0000","last_active":"2024-04-22T13:24:06.162449+0000","last_peered":"2024-04-22T13:24:06.162449+0000","last_clean":"2024-04-22T13:24:06.162449+0000","last_became_active":"2024-04-22T13:22:05.246365+0000","last_became_peered":"2024-04-22T13:22:05.246365+0000","last_unstale":"2024-04-22T13:24:06.162449+0000","last_undegraded":"2024-04-22T13:24:06.162449+0000","last_fullsized":"2024-04-22T13:24:06.162449+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:41:55.533430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411527+0000","last_change":"2024-04-22T13:22:05.251982+0000","last_active":"2024-04-22T13:24:06.411527+0000","last_peered":"2024-04-22T13:24:06.411527+0000","last_clean":"2024-04-22T13:24:06.411527+0000","last_became_active":"2024-04-22T13:22:05.251575+0000","last_became_peered":"2024-04-22T13:22:05.251575+0000","last_unstale":"2024-04-22T13:24:06.411527+0000","last_undegraded":"2024-04-22T13:24:06.411527+0000","last_fullsized":"2024-04-22T13:24:06.411527+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:28:05.343140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411427+0000","last_change":"2024-04-22T13:22:05.251056+0000","last_active":"2024-04-22T13:24:06.411427+0000","last_peered":"2024-04-22T13:24:06.411427+0000","last_clean":"2024-04-22T13:24:06.411427+0000","last_became_active":"2024-04-22T13:22:05.250652+0000","last_became_peered":"2024-04-22T13:22:05.250652+0000","last_unstale":"2024-04-22T13:24:06.411427+0000","last_undegraded":"2024-04-22T13:24:06.411427+0000","last_fullsized":"2024-04-22T13:24:06.411427+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:08:28.629965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.163636+0000","last_change":"2024-04-22T13:22:05.256877+0000","last_active":"2024-04-22T13:24:06.163636+0000","last_peered":"2024-04-22T13:24:06.163636+0000","last_clean":"2024-04-22T13:24:06.163636+0000","last_became_active":"2024-04-22T13:22:05.256074+0000","last_became_peered":"2024-04-22T13:22:05.256074+0000","last_unstale":"2024-04-22T13:24:06.163636+0000","last_undegraded":"2024-04-22T13:24:06.163636+0000","last_fullsized":"2024-04-22T13:24:06.163636+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:08:53.607626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418592+0000","last_change":"2024-04-22T13:22:05.255598+0000","last_active":"2024-04-22T13:24:06.418592+0000","last_peered":"2024-04-22T13:24:06.418592+0000","last_clean":"2024-04-22T13:24:06.418592+0000","last_became_active":"2024-04-22T13:22:05.255239+0000","last_became_peered":"2024-04-22T13:22:05.255239+0000","last_unstale":"2024-04-22T13:24:06.418592+0000","last_undegraded":"2024-04-22T13:24:06.418592+0000","last_fullsized":"2024-04-22T13:24:06.418592+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:38:20.547882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.257085+0000","last_change":"2024-04-22T13:22:05.254232+0000","last_active":"2024-04-22T13:24:06.257085+0000","last_peered":"2024-04-22T13:24:06.257085+0000","last_clean":"2024-04-22T13:24:06.257085+0000","last_became_active":"2024-04-22T13:22:05.253907+0000","last_became_peered":"2024-04-22T13:22:05.253907+0000","last_unstale":"2024-04-22T13:24:06.257085+0000","last_undegraded":"2024-04-22T13:24:06.257085+0000","last_fullsized":"2024-04-22T13:24:06.257085+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:15:49.931112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.415196+0000","last_change":"2024-04-22T13:22:05.251518+0000","last_active":"2024-04-22T13:24:06.415196+0000","last_peered":"2024-04-22T13:24:06.415196+0000","last_clean":"2024-04-22T13:24:06.415196+0000","last_became_active":"2024-04-22T13:22:05.250478+0000","last_became_peered":"2024-04-22T13:22:05.250478+0000","last_unstale":"2024-04-22T13:24:06.415196+0000","last_undegraded":"2024-04-22T13:24:06.415196+0000","last_fullsized":"2024-04-22T13:24:06.415196+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:25:26.758692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'14","reported_seq":157,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:10.565206+0000","last_change":"2024-04-22T13:22:04.241365+0000","last_active":"2024-04-22T13:24:10.565206+0000","last_peered":"2024-04-22T13:24:10.565206+0000","last_clean":"2024-04-22T13:24:10.565206+0000","last_became_active":"2024-04-22T13:22:04.240822+0000","last_became_peered":"2024-04-22T13:22:04.240822+0000","last_unstale":"2024-04-22T13:24:10.565206+0000","last_undegraded":"2024-04-22T13:24:10.565206+0000","last_fullsized":"2024-04-22T13:24:10.565206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:20:23.928534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.256956+0000","last_change":"2024-04-22T13:23:05.876262+0000","last_active":"2024-04-22T13:24:06.256956+0000","last_peered":"2024-04-22T13:24:06.256956+0000","last_clean":"2024-04-22T13:24:06.256956+0000","last_became_active":"2024-04-22T13:23:05.875872+0000","last_became_peered":"2024-04-22T13:23:05.875872+0000","last_unstale":"2024-04-22T13:24:06.256956+0000","last_undegraded":"2024-04-22T13:24:06.256956+0000","last_fullsized":"2024-04-22T13:24:06.256956+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:29:33.092887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259532+0000","last_change":"2024-04-22T13:22:04.248872+0000","last_active":"2024-04-22T13:24:06.259532+0000","last_peered":"2024-04-22T13:24:06.259532+0000","last_clean":"2024-04-22T13:24:06.259532+0000","last_became_active":"2024-04-22T13:22:04.248466+0000","last_became_peered":"2024-04-22T13:22:04.248466+0000","last_unstale":"2024-04-22T13:24:06.259532+0000","last_undegraded":"2024-04-22T13:24:06.259532+0000","last_fullsized":"2024-04-22T13:24:06.259532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:38:10.871602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.412438+0000","last_change":"2024-04-22T13:22:05.258005+0000","last_active":"2024-04-22T13:24:06.412438+0000","last_peered":"2024-04-22T13:24:06.412438+0000","last_clean":"2024-04-22T13:24:06.412438+0000","last_became_active":"2024-04-22T13:22:05.257615+0000","last_became_peered":"2024-04-22T13:22:05.257615+0000","last_unstale":"2024-04-22T13:24:06.412438+0000","last_undegraded":"2024-04-22T13:24:06.412438+0000","last_fullsized":"2024-04-22T13:24:06.412438+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:21:18.030530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259060+0000","last_change":"2024-04-22T13:22:04.246273+0000","last_active":"2024-04-22T13:24:06.259060+0000","last_peered":"2024-04-22T13:24:06.259060+0000","last_clean":"2024-04-22T13:24:06.259060+0000","last_became_active":"2024-04-22T13:22:04.245969+0000","last_became_peered":"2024-04-22T13:22:04.245969+0000","last_unstale":"2024-04-22T13:24:06.259060+0000","last_undegraded":"2024-04-22T13:24:06.259060+0000","last_fullsized":"2024-04-22T13:24:06.259060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:10.260192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.163964+0000","last_change":"2024-04-22T13:22:05.260299+0000","last_active":"2024-04-22T13:24:06.163964+0000","last_peered":"2024-04-22T13:24:06.163964+0000","last_clean":"2024-04-22T13:24:06.163964+0000","last_became_active":"2024-04-22T13:22:05.259634+0000","last_became_peered":"2024-04-22T13:22:05.259634+0000","last_unstale":"2024-04-22T13:24:06.163964+0000","last_undegraded":"2024-04-22T13:24:06.163964+0000","last_fullsized":"2024-04-22T13:24:06.163964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:16:44.421652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258970+0000","last_change":"2024-04-22T13:22:04.243843+0000","last_active":"2024-04-22T13:24:06.258970+0000","last_peered":"2024-04-22T13:24:06.258970+0000","last_clean":"2024-04-22T13:24:06.258970+0000","last_became_active":"2024-04-22T13:22:04.243465+0000","last_became_peered":"2024-04-22T13:22:04.243465+0000","last_unstale":"2024-04-22T13:24:06.258970+0000","last_undegraded":"2024-04-22T13:24:06.258970+0000","last_fullsized":"2024-04-22T13:24:06.258970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:48:36.431697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164717+0000","last_change":"2024-04-22T13:22:05.259789+0000","last_active":"2024-04-22T13:24:06.164717+0000","last_peered":"2024-04-22T13:24:06.164717+0000","last_clean":"2024-04-22T13:24:06.164717+0000","last_became_active":"2024-04-22T13:22:05.259395+0000","last_became_peered":"2024-04-22T13:22:05.259395+0000","last_unstale":"2024-04-22T13:24:06.164717+0000","last_undegraded":"2024-04-22T13:24:06.164717+0000","last_fullsized":"2024-04-22T13:24:06.164717+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:43:51.430561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'7","reported_seq":155,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:10.539561+0000","last_change":"2024-04-22T13:22:04.240615+0000","last_active":"2024-04-22T13:24:10.539561+0000","last_peered":"2024-04-22T13:24:10.539561+0000","last_clean":"2024-04-22T13:24:10.539561+0000","last_became_active":"2024-04-22T13:22:04.240248+0000","last_became_peered":"2024-04-22T13:22:04.240248+0000","last_unstale":"2024-04-22T13:24:10.539561+0000","last_undegraded":"2024-04-22T13:24:10.539561+0000","last_fullsized":"2024-04-22T13:24:10.539561+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:55:05.948979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1404,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":127,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258665+0000","last_change":"2024-04-22T13:22:05.253788+0000","last_active":"2024-04-22T13:24:06.258665+0000","last_peered":"2024-04-22T13:24:06.258665+0000","last_clean":"2024-04-22T13:24:06.258665+0000","last_became_active":"2024-04-22T13:22:05.252578+0000","last_became_peered":"2024-04-22T13:22:05.252578+0000","last_unstale":"2024-04-22T13:24:06.258665+0000","last_undegraded":"2024-04-22T13:24:06.258665+0000","last_fullsized":"2024-04-22T13:24:06.258665+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:04:11.769930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.266274+0000","last_change":"2024-04-22T13:22:04.241170+0000","last_active":"2024-04-22T13:24:06.266274+0000","last_peered":"2024-04-22T13:24:06.266274+0000","last_clean":"2024-04-22T13:24:06.266274+0000","last_became_active":"2024-04-22T13:22:04.240830+0000","last_became_peered":"2024-04-22T13:22:04.240830+0000","last_unstale":"2024-04-22T13:24:06.266274+0000","last_undegraded":"2024-04-22T13:24:06.266274+0000","last_fullsized":"2024-04-22T13:24:06.266274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:51:30.520613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164733+0000","last_change":"2024-04-22T13:22:05.237900+0000","last_active":"2024-04-22T13:24:06.164733+0000","last_peered":"2024-04-22T13:24:06.164733+0000","last_clean":"2024-04-22T13:24:06.164733+0000","last_became_active":"2024-04-22T13:22:05.237507+0000","last_became_peered":"2024-04-22T13:22:05.237507+0000","last_unstale":"2024-04-22T13:24:06.164733+0000","last_undegraded":"2024-04-22T13:24:06.164733+0000","last_fullsized":"2024-04-22T13:24:06.164733+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:26:24.307803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162328+0000","last_change":"2024-04-22T13:22:04.236078+0000","last_active":"2024-04-22T13:24:06.162328+0000","last_peered":"2024-04-22T13:24:06.162328+0000","last_clean":"2024-04-22T13:24:06.162328+0000","last_became_active":"2024-04-22T13:22:04.235710+0000","last_became_peered":"2024-04-22T13:22:04.235710+0000","last_unstale":"2024-04-22T13:24:06.162328+0000","last_undegraded":"2024-04-22T13:24:06.162328+0000","last_fullsized":"2024-04-22T13:24:06.162328+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:37:55.335237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258717+0000","last_change":"2024-04-22T13:22:05.243152+0000","last_active":"2024-04-22T13:24:06.258717+0000","last_peered":"2024-04-22T13:24:06.258717+0000","last_clean":"2024-04-22T13:24:06.258717+0000","last_became_active":"2024-04-22T13:22:05.242772+0000","last_became_peered":"2024-04-22T13:22:05.242772+0000","last_unstale":"2024-04-22T13:24:06.258717+0000","last_undegraded":"2024-04-22T13:24:06.258717+0000","last_fullsized":"2024-04-22T13:24:06.258717+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:33:42.359679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164258+0000","last_change":"2024-04-22T13:22:04.244381+0000","last_active":"2024-04-22T13:24:06.164258+0000","last_peered":"2024-04-22T13:24:06.164258+0000","last_clean":"2024-04-22T13:24:06.164258+0000","last_became_active":"2024-04-22T13:22:04.243746+0000","last_became_peered":"2024-04-22T13:22:04.243746+0000","last_unstale":"2024-04-22T13:24:06.164258+0000","last_undegraded":"2024-04-22T13:24:06.164258+0000","last_fullsized":"2024-04-22T13:24:06.164258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:15:36.019930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.256753+0000","last_change":"2024-04-22T13:22:05.245394+0000","last_active":"2024-04-22T13:24:06.256753+0000","last_peered":"2024-04-22T13:24:06.256753+0000","last_clean":"2024-04-22T13:24:06.256753+0000","last_became_active":"2024-04-22T13:22:05.243931+0000","last_became_peered":"2024-04-22T13:22:05.243931+0000","last_unstale":"2024-04-22T13:24:06.256753+0000","last_undegraded":"2024-04-22T13:24:06.256753+0000","last_fullsized":"2024-04-22T13:24:06.256753+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:11:10.790842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'6","reported_seq":270,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:11.540611+0000","last_change":"2024-04-22T13:22:04.242583+0000","last_active":"2024-04-22T13:24:11.540611+0000","last_peered":"2024-04-22T13:24:11.540611+0000","last_clean":"2024-04-22T13:24:11.540611+0000","last_became_active":"2024-04-22T13:22:04.242261+0000","last_became_peered":"2024-04-22T13:22:04.242261+0000","last_unstale":"2024-04-22T13:24:11.540611+0000","last_undegraded":"2024-04-22T13:24:11.540611+0000","last_fullsized":"2024-04-22T13:24:11.540611+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:37:47.618573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":243,"num_read_kb":243,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164258+0000","last_change":"2024-04-22T13:22:05.255293+0000","last_active":"2024-04-22T13:24:06.164258+0000","last_peered":"2024-04-22T13:24:06.164258+0000","last_clean":"2024-04-22T13:24:06.164258+0000","last_became_active":"2024-04-22T13:22:05.253949+0000","last_became_peered":"2024-04-22T13:22:05.253949+0000","last_unstale":"2024-04-22T13:24:06.164258+0000","last_undegraded":"2024-04-22T13:24:06.164258+0000","last_fullsized":"2024-04-22T13:24:06.164258+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:41:08.754989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409503+0000","last_change":"2024-04-22T13:22:04.241529+0000","last_active":"2024-04-22T13:24:06.409503+0000","last_peered":"2024-04-22T13:24:06.409503+0000","last_clean":"2024-04-22T13:24:06.409503+0000","last_became_active":"2024-04-22T13:22:04.241164+0000","last_became_peered":"2024-04-22T13:22:04.241164+0000","last_unstale":"2024-04-22T13:24:06.409503+0000","last_undegraded":"2024-04-22T13:24:06.409503+0000","last_fullsized":"2024-04-22T13:24:06.409503+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:16:49.387181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.412225+0000","last_change":"2024-04-22T13:22:05.241623+0000","last_active":"2024-04-22T13:24:06.412225+0000","last_peered":"2024-04-22T13:24:06.412225+0000","last_clean":"2024-04-22T13:24:06.412225+0000","last_became_active":"2024-04-22T13:22:05.241150+0000","last_became_peered":"2024-04-22T13:22:05.241150+0000","last_unstale":"2024-04-22T13:24:06.412225+0000","last_undegraded":"2024-04-22T13:24:06.412225+0000","last_fullsized":"2024-04-22T13:24:06.412225+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:17:28.412366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164893+0000","last_change":"2024-04-22T13:22:04.247852+0000","last_active":"2024-04-22T13:24:06.164893+0000","last_peered":"2024-04-22T13:24:06.164893+0000","last_clean":"2024-04-22T13:24:06.164893+0000","last_became_active":"2024-04-22T13:22:04.247550+0000","last_became_peered":"2024-04-22T13:22:04.247550+0000","last_unstale":"2024-04-22T13:24:06.164893+0000","last_undegraded":"2024-04-22T13:24:06.164893+0000","last_fullsized":"2024-04-22T13:24:06.164893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:55:01.928435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164057+0000","last_change":"2024-04-22T13:22:05.260050+0000","last_active":"2024-04-22T13:24:06.164057+0000","last_peered":"2024-04-22T13:24:06.164057+0000","last_clean":"2024-04-22T13:24:06.164057+0000","last_became_active":"2024-04-22T13:22:05.259098+0000","last_became_peered":"2024-04-22T13:22:05.259098+0000","last_unstale":"2024-04-22T13:24:06.164057+0000","last_undegraded":"2024-04-22T13:24:06.164057+0000","last_fullsized":"2024-04-22T13:24:06.164057+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:05:06.245492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'38","reported_seq":319,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:11.540734+0000","last_change":"2024-04-22T13:22:04.240233+0000","last_active":"2024-04-22T13:24:11.540734+0000","last_peered":"2024-04-22T13:24:11.540734+0000","last_clean":"2024-04-22T13:24:11.540734+0000","last_became_active":"2024-04-22T13:22:04.239890+0000","last_became_peered":"2024-04-22T13:22:04.239890+0000","last_unstale":"2024-04-22T13:24:11.540734+0000","last_undegraded":"2024-04-22T13:24:11.540734+0000","last_fullsized":"2024-04-22T13:24:11.540734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:04:21.547422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67638,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":260,"num_read_kb":197,"num_write":40,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258900+0000","last_change":"2024-04-22T13:22:05.257756+0000","last_active":"2024-04-22T13:24:06.258900+0000","last_peered":"2024-04-22T13:24:06.258900+0000","last_clean":"2024-04-22T13:24:06.258900+0000","last_became_active":"2024-04-22T13:22:05.257442+0000","last_became_peered":"2024-04-22T13:22:05.257442+0000","last_unstale":"2024-04-22T13:24:06.258900+0000","last_undegraded":"2024-04-22T13:24:06.258900+0000","last_fullsized":"2024-04-22T13:24:06.258900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:16:53.465689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.414923+0000","last_change":"2024-04-22T13:22:04.240815+0000","last_active":"2024-04-22T13:24:06.414923+0000","last_peered":"2024-04-22T13:24:06.414923+0000","last_clean":"2024-04-22T13:24:06.414923+0000","last_became_active":"2024-04-22T13:22:04.240218+0000","last_became_peered":"2024-04-22T13:22:04.240218+0000","last_unstale":"2024-04-22T13:24:06.414923+0000","last_undegraded":"2024-04-22T13:24:06.414923+0000","last_fullsized":"2024-04-22T13:24:06.414923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:29:29.148764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162257+0000","last_change":"2024-04-22T13:22:05.248797+0000","last_active":"2024-04-22T13:24:06.162257+0000","last_peered":"2024-04-22T13:24:06.162257+0000","last_clean":"2024-04-22T13:24:06.162257+0000","last_became_active":"2024-04-22T13:22:05.248435+0000","last_became_peered":"2024-04-22T13:22:05.248435+0000","last_unstale":"2024-04-22T13:24:06.162257+0000","last_undegraded":"2024-04-22T13:24:06.162257+0000","last_fullsized":"2024-04-22T13:24:06.162257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:32:31.970494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164389+0000","last_change":"2024-04-22T13:22:04.245303+0000","last_active":"2024-04-22T13:24:06.164389+0000","last_peered":"2024-04-22T13:24:06.164389+0000","last_clean":"2024-04-22T13:24:06.164389+0000","last_became_active":"2024-04-22T13:22:04.244903+0000","last_became_peered":"2024-04-22T13:22:04.244903+0000","last_unstale":"2024-04-22T13:24:06.164389+0000","last_undegraded":"2024-04-22T13:24:06.164389+0000","last_fullsized":"2024-04-22T13:24:06.164389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:34:32.174858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409264+0000","last_change":"2024-04-22T13:22:05.246318+0000","last_active":"2024-04-22T13:24:06.409264+0000","last_peered":"2024-04-22T13:24:06.409264+0000","last_clean":"2024-04-22T13:24:06.409264+0000","last_became_active":"2024-04-22T13:22:05.245899+0000","last_became_peered":"2024-04-22T13:22:05.245899+0000","last_unstale":"2024-04-22T13:24:06.409264+0000","last_undegraded":"2024-04-22T13:24:06.409264+0000","last_fullsized":"2024-04-22T13:24:06.409264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:27:42.789058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'9","reported_seq":155,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:10.597760+0000","last_change":"2024-04-22T13:22:04.240612+0000","last_active":"2024-04-22T13:24:10.597760+0000","last_peered":"2024-04-22T13:24:10.597760+0000","last_clean":"2024-04-22T13:24:10.597760+0000","last_became_active":"2024-04-22T13:22:04.240290+0000","last_became_peered":"2024-04-22T13:22:04.240290+0000","last_unstale":"2024-04-22T13:24:10.597760+0000","last_undegraded":"2024-04-22T13:24:10.597760+0000","last_fullsized":"2024-04-22T13:24:10.597760+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:00:50.285553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1902,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":126,"num_read_kb":6,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.408976+0000","last_change":"2024-04-22T13:22:05.253205+0000","last_active":"2024-04-22T13:24:06.408976+0000","last_peered":"2024-04-22T13:24:06.408976+0000","last_clean":"2024-04-22T13:24:06.408976+0000","last_became_active":"2024-04-22T13:22:05.252865+0000","last_became_peered":"2024-04-22T13:22:05.252865+0000","last_unstale":"2024-04-22T13:24:06.408976+0000","last_undegraded":"2024-04-22T13:24:06.408976+0000","last_fullsized":"2024-04-22T13:24:06.408976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:19:12.482371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258636+0000","last_change":"2024-04-22T13:22:04.248938+0000","last_active":"2024-04-22T13:24:06.258636+0000","last_peered":"2024-04-22T13:24:06.258636+0000","last_clean":"2024-04-22T13:24:06.258636+0000","last_became_active":"2024-04-22T13:22:04.248440+0000","last_became_peered":"2024-04-22T13:22:04.248440+0000","last_unstale":"2024-04-22T13:24:06.258636+0000","last_undegraded":"2024-04-22T13:24:06.258636+0000","last_fullsized":"2024-04-22T13:24:06.258636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:46:50.928067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164391+0000","last_change":"2024-04-22T13:22:05.254664+0000","last_active":"2024-04-22T13:24:06.164391+0000","last_peered":"2024-04-22T13:24:06.164391+0000","last_clean":"2024-04-22T13:24:06.164391+0000","last_became_active":"2024-04-22T13:22:05.254304+0000","last_became_peered":"2024-04-22T13:22:05.254304+0000","last_unstale":"2024-04-22T13:24:06.164391+0000","last_undegraded":"2024-04-22T13:24:06.164391+0000","last_fullsized":"2024-04-22T13:24:06.164391+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:58:49.345573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259033+0000","last_change":"2024-04-22T13:22:04.244128+0000","last_active":"2024-04-22T13:24:06.259033+0000","last_peered":"2024-04-22T13:24:06.259033+0000","last_clean":"2024-04-22T13:24:06.259033+0000","last_became_active":"2024-04-22T13:22:04.243738+0000","last_became_peered":"2024-04-22T13:22:04.243738+0000","last_unstale":"2024-04-22T13:24:06.259033+0000","last_undegraded":"2024-04-22T13:24:06.259033+0000","last_fullsized":"2024-04-22T13:24:06.259033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:23:52.220584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418387+0000","last_change":"2024-04-22T13:22:05.250910+0000","last_active":"2024-04-22T13:24:06.418387+0000","last_peered":"2024-04-22T13:24:06.418387+0000","last_clean":"2024-04-22T13:24:06.418387+0000","last_became_active":"2024-04-22T13:22:05.250410+0000","last_became_peered":"2024-04-22T13:22:05.250410+0000","last_unstale":"2024-04-22T13:24:06.418387+0000","last_undegraded":"2024-04-22T13:24:06.418387+0000","last_fullsized":"2024-04-22T13:24:06.418387+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:04:25.492677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.256811+0000","last_change":"2024-04-22T13:22:04.245889+0000","last_active":"2024-04-22T13:24:06.256811+0000","last_peered":"2024-04-22T13:24:06.256811+0000","last_clean":"2024-04-22T13:24:06.256811+0000","last_became_active":"2024-04-22T13:22:04.245474+0000","last_became_peered":"2024-04-22T13:22:04.245474+0000","last_unstale":"2024-04-22T13:24:06.256811+0000","last_undegraded":"2024-04-22T13:24:06.256811+0000","last_fullsized":"2024-04-22T13:24:06.256811+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:22:12.626950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.266594+0000","last_change":"2024-04-22T13:22:05.250978+0000","last_active":"2024-04-22T13:24:06.266594+0000","last_peered":"2024-04-22T13:24:06.266594+0000","last_clean":"2024-04-22T13:24:06.266594+0000","last_became_active":"2024-04-22T13:22:05.250463+0000","last_became_peered":"2024-04-22T13:22:05.250463+0000","last_unstale":"2024-04-22T13:24:06.266594+0000","last_undegraded":"2024-04-22T13:24:06.266594+0000","last_fullsized":"2024-04-22T13:24:06.266594+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:45:18.591072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'7","reported_seq":151,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:10.565681+0000","last_change":"2024-04-22T13:22:04.242503+0000","last_active":"2024-04-22T13:24:10.565681+0000","last_peered":"2024-04-22T13:24:10.565681+0000","last_clean":"2024-04-22T13:24:10.565681+0000","last_became_active":"2024-04-22T13:22:04.242143+0000","last_became_peered":"2024-04-22T13:22:04.242143+0000","last_unstale":"2024-04-22T13:24:10.565681+0000","last_undegraded":"2024-04-22T13:24:10.565681+0000","last_fullsized":"2024-04-22T13:24:10.565681+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:40.236377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.266578+0000","last_change":"2024-04-22T13:22:05.252387+0000","last_active":"2024-04-22T13:24:06.266578+0000","last_peered":"2024-04-22T13:24:06.266578+0000","last_clean":"2024-04-22T13:24:06.266578+0000","last_became_active":"2024-04-22T13:22:05.252033+0000","last_became_peered":"2024-04-22T13:22:05.252033+0000","last_unstale":"2024-04-22T13:24:06.266578+0000","last_undegraded":"2024-04-22T13:24:06.266578+0000","last_fullsized":"2024-04-22T13:24:06.266578+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:55:01.791283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164456+0000","last_change":"2024-04-22T13:22:05.248394+0000","last_active":"2024-04-22T13:24:06.164456+0000","last_peered":"2024-04-22T13:24:06.164456+0000","last_clean":"2024-04-22T13:24:06.164456+0000","last_became_active":"2024-04-22T13:22:05.248024+0000","last_became_peered":"2024-04-22T13:22:05.248024+0000","last_unstale":"2024-04-22T13:24:06.164456+0000","last_undegraded":"2024-04-22T13:24:06.164456+0000","last_fullsized":"2024-04-22T13:24:06.164456+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:49:36.556136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'7","reported_seq":150,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:10.597275+0000","last_change":"2024-04-22T13:22:04.247692+0000","last_active":"2024-04-22T13:24:10.597275+0000","last_peered":"2024-04-22T13:24:10.597275+0000","last_clean":"2024-04-22T13:24:10.597275+0000","last_became_active":"2024-04-22T13:22:04.247345+0000","last_became_peered":"2024-04-22T13:22:04.247345+0000","last_unstale":"2024-04-22T13:24:10.597275+0000","last_undegraded":"2024-04-22T13:24:10.597275+0000","last_fullsized":"2024-04-22T13:24:10.597275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:07:49.803248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259343+0000","last_change":"2024-04-22T13:22:05.241022+0000","last_active":"2024-04-22T13:24:06.259343+0000","last_peered":"2024-04-22T13:24:06.259343+0000","last_clean":"2024-04-22T13:24:06.259343+0000","last_became_active":"2024-04-22T13:22:05.240635+0000","last_became_peered":"2024-04-22T13:22:05.240635+0000","last_unstale":"2024-04-22T13:24:06.259343+0000","last_undegraded":"2024-04-22T13:24:06.259343+0000","last_fullsized":"2024-04-22T13:24:06.259343+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:50:02.012260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411284+0000","last_change":"2024-04-22T13:22:04.236817+0000","last_active":"2024-04-22T13:24:06.411284+0000","last_peered":"2024-04-22T13:24:06.411284+0000","last_clean":"2024-04-22T13:24:06.411284+0000","last_became_active":"2024-04-22T13:22:04.236484+0000","last_became_peered":"2024-04-22T13:22:04.236484+0000","last_unstale":"2024-04-22T13:24:06.411284+0000","last_undegraded":"2024-04-22T13:24:06.411284+0000","last_fullsized":"2024-04-22T13:24:06.411284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:27:41.053327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"90'148","reported_seq":401,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:08.291461+0000","last_change":"2024-04-22T13:20:53.094123+0000","last_active":"2024-04-22T13:24:08.291461+0000","last_peered":"2024-04-22T13:24:08.291461+0000","last_clean":"2024-04-22T13:24:08.291461+0000","last_became_active":"2024-04-22T13:20:53.093664+0000","last_became_peered":"2024-04-22T13:20:53.093664+0000","last_unstale":"2024-04-22T13:24:08.291461+0000","last_undegraded":"2024-04-22T13:24:08.291461+0000","last_fullsized":"2024-04-22T13:24:08.291461+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:17:33.201786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:17:33.201786+0000","last_clean_scrub_stamp":"2024-04-22T13:17:33.201786+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:46:58.986399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162213+0000","last_change":"2024-04-22T13:22:05.247766+0000","last_active":"2024-04-22T13:24:06.162213+0000","last_peered":"2024-04-22T13:24:06.162213+0000","last_clean":"2024-04-22T13:24:06.162213+0000","last_became_active":"2024-04-22T13:22:05.247362+0000","last_became_peered":"2024-04-22T13:22:05.247362+0000","last_unstale":"2024-04-22T13:24:06.162213+0000","last_undegraded":"2024-04-22T13:24:06.162213+0000","last_fullsized":"2024-04-22T13:24:06.162213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:25:00.126638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409239+0000","last_change":"2024-04-22T13:22:04.243493+0000","last_active":"2024-04-22T13:24:06.409239+0000","last_peered":"2024-04-22T13:24:06.409239+0000","last_clean":"2024-04-22T13:24:06.409239+0000","last_became_active":"2024-04-22T13:22:04.243190+0000","last_became_peered":"2024-04-22T13:22:04.243190+0000","last_unstale":"2024-04-22T13:24:06.409239+0000","last_undegraded":"2024-04-22T13:24:06.409239+0000","last_fullsized":"2024-04-22T13:24:06.409239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:15:09.025152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258892+0000","last_change":"2024-04-22T13:22:04.249222+0000","last_active":"2024-04-22T13:24:06.258892+0000","last_peered":"2024-04-22T13:24:06.258892+0000","last_clean":"2024-04-22T13:24:06.258892+0000","last_became_active":"2024-04-22T13:22:04.248926+0000","last_became_peered":"2024-04-22T13:22:04.248926+0000","last_unstale":"2024-04-22T13:24:06.258892+0000","last_undegraded":"2024-04-22T13:24:06.258892+0000","last_fullsized":"2024-04-22T13:24:06.258892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:33:50.168426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162488+0000","last_change":"2024-04-22T13:22:05.248199+0000","last_active":"2024-04-22T13:24:06.162488+0000","last_peered":"2024-04-22T13:24:06.162488+0000","last_clean":"2024-04-22T13:24:06.162488+0000","last_became_active":"2024-04-22T13:22:05.247873+0000","last_became_peered":"2024-04-22T13:22:05.247873+0000","last_unstale":"2024-04-22T13:24:06.162488+0000","last_undegraded":"2024-04-22T13:24:06.162488+0000","last_fullsized":"2024-04-22T13:24:06.162488+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:40:29.353901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'49","reported_seq":94,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:07.392889+0000","last_change":"2024-04-22T13:22:04.243474+0000","last_active":"2024-04-22T13:24:07.392889+0000","last_peered":"2024-04-22T13:24:07.392889+0000","last_clean":"2024-04-22T13:24:07.392889+0000","last_became_active":"2024-04-22T13:22:04.243151+0000","last_became_peered":"2024-04-22T13:22:04.243151+0000","last_unstale":"2024-04-22T13:24:07.392889+0000","last_undegraded":"2024-04-22T13:24:07.392889+0000","last_fullsized":"2024-04-22T13:24:07.392889+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:18:25.140715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":86,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259264+0000","last_change":"2024-04-22T13:22:05.243120+0000","last_active":"2024-04-22T13:24:06.259264+0000","last_peered":"2024-04-22T13:24:06.259264+0000","last_clean":"2024-04-22T13:24:06.259264+0000","last_became_active":"2024-04-22T13:22:05.242726+0000","last_became_peered":"2024-04-22T13:22:05.242726+0000","last_unstale":"2024-04-22T13:24:06.259264+0000","last_undegraded":"2024-04-22T13:24:06.259264+0000","last_fullsized":"2024-04-22T13:24:06.259264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:06:16.087884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411424+0000","last_change":"2024-04-22T13:22:04.239672+0000","last_active":"2024-04-22T13:24:06.411424+0000","last_peered":"2024-04-22T13:24:06.411424+0000","last_clean":"2024-04-22T13:24:06.411424+0000","last_became_active":"2024-04-22T13:22:04.238776+0000","last_became_peered":"2024-04-22T13:22:04.238776+0000","last_unstale":"2024-04-22T13:24:06.411424+0000","last_undegraded":"2024-04-22T13:24:06.411424+0000","last_fullsized":"2024-04-22T13:24:06.411424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:43:21.752547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.412035+0000","last_change":"2024-04-22T13:22:05.242089+0000","last_active":"2024-04-22T13:24:06.412035+0000","last_peered":"2024-04-22T13:24:06.412035+0000","last_clean":"2024-04-22T13:24:06.412035+0000","last_became_active":"2024-04-22T13:22:05.241681+0000","last_became_peered":"2024-04-22T13:22:05.241681+0000","last_unstale":"2024-04-22T13:24:06.412035+0000","last_undegraded":"2024-04-22T13:24:06.412035+0000","last_fullsized":"2024-04-22T13:24:06.412035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:58:49.906188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.163728+0000","last_change":"2024-04-22T13:23:05.513123+0000","last_active":"2024-04-22T13:24:06.163728+0000","last_peered":"2024-04-22T13:24:06.163728+0000","last_clean":"2024-04-22T13:24:06.163728+0000","last_became_active":"2024-04-22T13:23:05.509367+0000","last_became_peered":"2024-04-22T13:23:05.509367+0000","last_unstale":"2024-04-22T13:24:06.163728+0000","last_undegraded":"2024-04-22T13:24:06.163728+0000","last_fullsized":"2024-04-22T13:24:06.163728+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:28:41.770957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258638+0000","last_change":"2024-04-22T13:22:05.250120+0000","last_active":"2024-04-22T13:24:06.258638+0000","last_peered":"2024-04-22T13:24:06.258638+0000","last_clean":"2024-04-22T13:24:06.258638+0000","last_became_active":"2024-04-22T13:22:05.247936+0000","last_became_peered":"2024-04-22T13:22:05.247936+0000","last_unstale":"2024-04-22T13:24:06.258638+0000","last_undegraded":"2024-04-22T13:24:06.258638+0000","last_fullsized":"2024-04-22T13:24:06.258638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:27:38.437918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258902+0000","last_change":"2024-04-22T13:22:04.244026+0000","last_active":"2024-04-22T13:24:06.258902+0000","last_peered":"2024-04-22T13:24:06.258902+0000","last_clean":"2024-04-22T13:24:06.258902+0000","last_became_active":"2024-04-22T13:22:04.243518+0000","last_became_peered":"2024-04-22T13:22:04.243518+0000","last_unstale":"2024-04-22T13:24:06.258902+0000","last_undegraded":"2024-04-22T13:24:06.258902+0000","last_fullsized":"2024-04-22T13:24:06.258902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:54:48.791874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418220+0000","last_change":"2024-04-22T13:22:05.252674+0000","last_active":"2024-04-22T13:24:06.418220+0000","last_peered":"2024-04-22T13:24:06.418220+0000","last_clean":"2024-04-22T13:24:06.418220+0000","last_became_active":"2024-04-22T13:22:05.251087+0000","last_became_peered":"2024-04-22T13:22:05.251087+0000","last_unstale":"2024-04-22T13:24:06.418220+0000","last_undegraded":"2024-04-22T13:24:06.418220+0000","last_fullsized":"2024-04-22T13:24:06.418220+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:20:06.873412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.414693+0000","last_change":"2024-04-22T13:22:04.244409+0000","last_active":"2024-04-22T13:24:06.414693+0000","last_peered":"2024-04-22T13:24:06.414693+0000","last_clean":"2024-04-22T13:24:06.414693+0000","last_became_active":"2024-04-22T13:22:04.244104+0000","last_became_peered":"2024-04-22T13:22:04.244104+0000","last_unstale":"2024-04-22T13:24:06.414693+0000","last_undegraded":"2024-04-22T13:24:06.414693+0000","last_fullsized":"2024-04-22T13:24:06.414693+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:39:14.714755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259066+0000","last_change":"2024-04-22T13:22:05.245730+0000","last_active":"2024-04-22T13:24:06.259066+0000","last_peered":"2024-04-22T13:24:06.259066+0000","last_clean":"2024-04-22T13:24:06.259066+0000","last_became_active":"2024-04-22T13:22:05.245366+0000","last_became_peered":"2024-04-22T13:22:05.245366+0000","last_unstale":"2024-04-22T13:24:06.259066+0000","last_undegraded":"2024-04-22T13:24:06.259066+0000","last_fullsized":"2024-04-22T13:24:06.259066+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:31.935258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418212+0000","last_change":"2024-04-22T13:23:05.514197+0000","last_active":"2024-04-22T13:24:06.418212+0000","last_peered":"2024-04-22T13:24:06.418212+0000","last_clean":"2024-04-22T13:24:06.418212+0000","last_became_active":"2024-04-22T13:23:05.509431+0000","last_became_peered":"2024-04-22T13:23:05.509431+0000","last_unstale":"2024-04-22T13:24:06.418212+0000","last_undegraded":"2024-04-22T13:24:06.418212+0000","last_fullsized":"2024-04-22T13:24:06.418212+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:43:59.825567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409273+0000","last_change":"2024-04-22T13:22:05.242146+0000","last_active":"2024-04-22T13:24:06.409273+0000","last_peered":"2024-04-22T13:24:06.409273+0000","last_clean":"2024-04-22T13:24:06.409273+0000","last_became_active":"2024-04-22T13:22:05.241802+0000","last_became_peered":"2024-04-22T13:22:05.241802+0000","last_unstale":"2024-04-22T13:24:06.409273+0000","last_undegraded":"2024-04-22T13:24:06.409273+0000","last_fullsized":"2024-04-22T13:24:06.409273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:49:49.002123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'8","reported_seq":154,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:10.568340+0000","last_change":"2024-04-22T13:22:04.248884+0000","last_active":"2024-04-22T13:24:10.568340+0000","last_peered":"2024-04-22T13:24:10.568340+0000","last_clean":"2024-04-22T13:24:10.568340+0000","last_became_active":"2024-04-22T13:22:04.248393+0000","last_became_peered":"2024-04-22T13:22:04.248393+0000","last_unstale":"2024-04-22T13:24:10.568340+0000","last_undegraded":"2024-04-22T13:24:10.568340+0000","last_fullsized":"2024-04-22T13:24:10.568340+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:36:14.687457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2260,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":125,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409545+0000","last_change":"2024-04-22T13:22:05.256881+0000","last_active":"2024-04-22T13:24:06.409545+0000","last_peered":"2024-04-22T13:24:06.409545+0000","last_clean":"2024-04-22T13:24:06.409545+0000","last_became_active":"2024-04-22T13:22:05.255967+0000","last_became_peered":"2024-04-22T13:22:05.255967+0000","last_unstale":"2024-04-22T13:24:06.409545+0000","last_undegraded":"2024-04-22T13:24:06.409545+0000","last_fullsized":"2024-04-22T13:24:06.409545+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:24:21.137344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'41","reported_seq":70,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.367821+0000","last_change":"2024-04-22T13:22:04.247303+0000","last_active":"2024-04-22T13:24:06.367821+0000","last_peered":"2024-04-22T13:24:06.367821+0000","last_clean":"2024-04-22T13:24:06.367821+0000","last_became_active":"2024-04-22T13:22:04.246742+0000","last_became_peered":"2024-04-22T13:22:04.246742+0000","last_unstale":"2024-04-22T13:24:06.367821+0000","last_undegraded":"2024-04-22T13:24:06.367821+0000","last_fullsized":"2024-04-22T13:24:06.367821+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:45:03.189042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409288+0000","last_change":"2024-04-22T13:22:05.251754+0000","last_active":"2024-04-22T13:24:06.409288+0000","last_peered":"2024-04-22T13:24:06.409288+0000","last_clean":"2024-04-22T13:24:06.409288+0000","last_became_active":"2024-04-22T13:22:05.251155+0000","last_became_peered":"2024-04-22T13:22:05.251155+0000","last_unstale":"2024-04-22T13:24:06.409288+0000","last_undegraded":"2024-04-22T13:24:06.409288+0000","last_fullsized":"2024-04-22T13:24:06.409288+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:08:46.682600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162485+0000","last_change":"2024-04-22T13:22:04.243382+0000","last_active":"2024-04-22T13:24:06.162485+0000","last_peered":"2024-04-22T13:24:06.162485+0000","last_clean":"2024-04-22T13:24:06.162485+0000","last_became_active":"2024-04-22T13:22:04.243055+0000","last_became_peered":"2024-04-22T13:22:04.243055+0000","last_unstale":"2024-04-22T13:24:06.162485+0000","last_undegraded":"2024-04-22T13:24:06.162485+0000","last_fullsized":"2024-04-22T13:24:06.162485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:19:58.379645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409542+0000","last_change":"2024-04-22T13:22:05.249527+0000","last_active":"2024-04-22T13:24:06.409542+0000","last_peered":"2024-04-22T13:24:06.409542+0000","last_clean":"2024-04-22T13:24:06.409542+0000","last_became_active":"2024-04-22T13:22:05.249207+0000","last_became_peered":"2024-04-22T13:22:05.249207+0000","last_unstale":"2024-04-22T13:24:06.409542+0000","last_undegraded":"2024-04-22T13:24:06.409542+0000","last_fullsized":"2024-04-22T13:24:06.409542+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:50:02.755288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418285+0000","last_change":"2024-04-22T13:23:05.508324+0000","last_active":"2024-04-22T13:24:06.418285+0000","last_peered":"2024-04-22T13:24:06.418285+0000","last_clean":"2024-04-22T13:24:06.418285+0000","last_became_active":"2024-04-22T13:23:05.507814+0000","last_became_peered":"2024-04-22T13:23:05.507814+0000","last_unstale":"2024-04-22T13:24:06.418285+0000","last_undegraded":"2024-04-22T13:24:06.418285+0000","last_fullsized":"2024-04-22T13:24:06.418285+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:46:20.523715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.414640+0000","last_change":"2024-04-22T13:23:05.875683+0000","last_active":"2024-04-22T13:24:06.414640+0000","last_peered":"2024-04-22T13:24:06.414640+0000","last_clean":"2024-04-22T13:24:06.414640+0000","last_became_active":"2024-04-22T13:23:05.875150+0000","last_became_peered":"2024-04-22T13:23:05.875150+0000","last_unstale":"2024-04-22T13:24:06.414640+0000","last_undegraded":"2024-04-22T13:24:06.414640+0000","last_fullsized":"2024-04-22T13:24:06.414640+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:12:47.592230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.414612+0000","last_change":"2024-04-22T13:22:05.247882+0000","last_active":"2024-04-22T13:24:06.414612+0000","last_peered":"2024-04-22T13:24:06.414612+0000","last_clean":"2024-04-22T13:24:06.414612+0000","last_became_active":"2024-04-22T13:22:05.247525+0000","last_became_peered":"2024-04-22T13:22:05.247525+0000","last_unstale":"2024-04-22T13:24:06.414612+0000","last_undegraded":"2024-04-22T13:24:06.414612+0000","last_fullsized":"2024-04-22T13:24:06.414612+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:46:04.349076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411567+0000","last_change":"2024-04-22T13:22:05.243716+0000","last_active":"2024-04-22T13:24:06.411567+0000","last_peered":"2024-04-22T13:24:06.411567+0000","last_clean":"2024-04-22T13:24:06.411567+0000","last_became_active":"2024-04-22T13:22:05.243397+0000","last_became_peered":"2024-04-22T13:22:05.243397+0000","last_unstale":"2024-04-22T13:24:06.411567+0000","last_undegraded":"2024-04-22T13:24:06.411567+0000","last_fullsized":"2024-04-22T13:24:06.411567+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:16:21.236339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162647+0000","last_change":"2024-04-22T13:22:05.246042+0000","last_active":"2024-04-22T13:24:06.162647+0000","last_peered":"2024-04-22T13:24:06.162647+0000","last_clean":"2024-04-22T13:24:06.162647+0000","last_became_active":"2024-04-22T13:22:05.245669+0000","last_became_peered":"2024-04-22T13:22:05.245669+0000","last_unstale":"2024-04-22T13:24:06.162647+0000","last_undegraded":"2024-04-22T13:24:06.162647+0000","last_fullsized":"2024-04-22T13:24:06.162647+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:49:05.539641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418321+0000","last_change":"2024-04-22T13:22:05.257009+0000","last_active":"2024-04-22T13:24:06.418321+0000","last_peered":"2024-04-22T13:24:06.418321+0000","last_clean":"2024-04-22T13:24:06.418321+0000","last_became_active":"2024-04-22T13:22:05.254327+0000","last_became_peered":"2024-04-22T13:22:05.254327+0000","last_unstale":"2024-04-22T13:24:06.418321+0000","last_undegraded":"2024-04-22T13:24:06.418321+0000","last_fullsized":"2024-04-22T13:24:06.418321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:29:12.482525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.415339+0000","last_change":"2024-04-22T13:22:05.260575+0000","last_active":"2024-04-22T13:24:06.415339+0000","last_peered":"2024-04-22T13:24:06.415339+0000","last_clean":"2024-04-22T13:24:06.415339+0000","last_became_active":"2024-04-22T13:22:05.260181+0000","last_became_peered":"2024-04-22T13:22:05.260181+0000","last_unstale":"2024-04-22T13:24:06.415339+0000","last_undegraded":"2024-04-22T13:24:06.415339+0000","last_fullsized":"2024-04-22T13:24:06.415339+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:49:20.064368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259116+0000","last_change":"2024-04-22T13:22:05.248199+0000","last_active":"2024-04-22T13:24:06.259116+0000","last_peered":"2024-04-22T13:24:06.259116+0000","last_clean":"2024-04-22T13:24:06.259116+0000","last_became_active":"2024-04-22T13:22:05.247858+0000","last_became_peered":"2024-04-22T13:22:05.247858+0000","last_unstale":"2024-04-22T13:24:06.259116+0000","last_undegraded":"2024-04-22T13:24:06.259116+0000","last_fullsized":"2024-04-22T13:24:06.259116+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:49:21.228105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164341+0000","last_change":"2024-04-22T13:22:05.239137+0000","last_active":"2024-04-22T13:24:06.164341+0000","last_peered":"2024-04-22T13:24:06.164341+0000","last_clean":"2024-04-22T13:24:06.164341+0000","last_became_active":"2024-04-22T13:22:05.238807+0000","last_became_peered":"2024-04-22T13:22:05.238807+0000","last_unstale":"2024-04-22T13:24:06.164341+0000","last_undegraded":"2024-04-22T13:24:06.164341+0000","last_fullsized":"2024-04-22T13:24:06.164341+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:02:48.587097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258756+0000","last_change":"2024-04-22T13:22:05.249080+0000","last_active":"2024-04-22T13:24:06.258756+0000","last_peered":"2024-04-22T13:24:06.258756+0000","last_clean":"2024-04-22T13:24:06.258756+0000","last_became_active":"2024-04-22T13:22:05.248733+0000","last_became_peered":"2024-04-22T13:22:05.248733+0000","last_unstale":"2024-04-22T13:24:06.258756+0000","last_undegraded":"2024-04-22T13:24:06.258756+0000","last_fullsized":"2024-04-22T13:24:06.258756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:58.735941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.163779+0000","last_change":"2024-04-22T13:22:05.255490+0000","last_active":"2024-04-22T13:24:06.163779+0000","last_peered":"2024-04-22T13:24:06.163779+0000","last_clean":"2024-04-22T13:24:06.163779+0000","last_became_active":"2024-04-22T13:22:05.253644+0000","last_became_peered":"2024-04-22T13:22:05.253644+0000","last_unstale":"2024-04-22T13:24:06.163779+0000","last_undegraded":"2024-04-22T13:24:06.163779+0000","last_fullsized":"2024-04-22T13:24:06.163779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:31:01.816182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162443+0000","last_change":"2024-04-22T13:22:05.240988+0000","last_active":"2024-04-22T13:24:06.162443+0000","last_peered":"2024-04-22T13:24:06.162443+0000","last_clean":"2024-04-22T13:24:06.162443+0000","last_became_active":"2024-04-22T13:22:05.240642+0000","last_became_peered":"2024-04-22T13:22:05.240642+0000","last_unstale":"2024-04-22T13:24:06.162443+0000","last_undegraded":"2024-04-22T13:24:06.162443+0000","last_fullsized":"2024-04-22T13:24:06.162443+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:41:29.001979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418222+0000","last_change":"2024-04-22T13:22:05.252212+0000","last_active":"2024-04-22T13:24:06.418222+0000","last_peered":"2024-04-22T13:24:06.418222+0000","last_clean":"2024-04-22T13:24:06.418222+0000","last_became_active":"2024-04-22T13:22:05.251852+0000","last_became_peered":"2024-04-22T13:22:05.251852+0000","last_unstale":"2024-04-22T13:24:06.418222+0000","last_undegraded":"2024-04-22T13:24:06.418222+0000","last_fullsized":"2024-04-22T13:24:06.418222+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:13:55.132501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602844,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1355,"num_read_kb":796,"num_write":377,"num_write_kb":1471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1808532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":305,"ondisk_log_size":305,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318311,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":473150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579946,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28800,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716352,"statfs":{"total":95995035648,"available":95965544448,"internally_reserved":0,"allocated":1826816,"data_stored":1263641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743472,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28388,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716764,"statfs":{"total":95995035648,"available":95965966336,"internally_reserved":0,"allocated":1400832,"data_stored":865206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005109,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28384,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716768,"statfs":{"total":95995035648,"available":95965970432,"internally_reserved":0,"allocated":1396736,"data_stored":865584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234042,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36900,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708252,"statfs":{"total":95995035648,"available":95957250048,"internally_reserved":0,"allocated":1601536,"data_stored":1067373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430271,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":991232,"data_stored":472489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":541040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":474136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1081344,"data_stored":542350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117071,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36836,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708316,"statfs":{"total":95995035648,"available":95957315584,"internally_reserved":0,"allocated":1536000,"data_stored":996348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855434,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":991232,"data_stored":472882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691908,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28724,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716428,"statfs":{"total":95995035648,"available":95965622272,"internally_reserved":0,"allocated":1748992,"data_stored":1196555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":202288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T13:24:17.788 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T13:24:17.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.930+0000 7fc226a9a640 1 -- 172.21.15.12:0/1271487966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2201021a0 msgr2=0x7fc220102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:17.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.930+0000 7fc226a9a640 1 --2- 172.21.15.12:0/1271487966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2201021a0 0x7fc220102580 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc214009900 tx=0x7fc21402bc50 comp rx=0 tx=0).stop 2024-04-22T13:24:17.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.930+0000 7fc226a9a640 1 -- 172.21.15.12:0/1271487966 shutdown_connections 2024-04-22T13:24:17.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.930+0000 7fc226a9a640 1 --2- 172.21.15.12:0/1271487966 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc220107270 0x7fc22010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:17.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.930+0000 7fc226a9a640 1 --2- 172.21.15.12:0/1271487966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc220102b50 0x7fc220106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:17.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.930+0000 7fc226a9a640 1 --2- 172.21.15.12:0/1271487966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2201021a0 0x7fc220102580 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:17.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.930+0000 7fc226a9a640 1 -- 172.21.15.12:0/1271487966 >> 172.21.15.12:0/1271487966 conn(0x7fc2200793a0 msgr2=0x7fc2200797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:17.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.930+0000 7fc226a9a640 1 -- 172.21.15.12:0/1271487966 shutdown_connections 2024-04-22T13:24:17.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.930+0000 7fc226a9a640 1 -- 172.21.15.12:0/1271487966 wait complete. 2024-04-22T13:24:17.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.930+0000 7fc226a9a640 1 Processor -- start 2024-04-22T13:24:17.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.931+0000 7fc226a9a640 1 -- start start 2024-04-22T13:24:17.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.931+0000 7fc226a9a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2201021a0 0x7fc220074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:17.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.931+0000 7fc226a9a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc220107270 0x7fc220074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:17.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.931+0000 7fc226a9a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc220074cd0 0x7fc2201a6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:17.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.931+0000 7fc226a9a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc220112280 con 0x7fc220074cd0 2024-04-22T13:24:17.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.931+0000 7fc226a9a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc220112100 con 0x7fc220107270 2024-04-22T13:24:17.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.931+0000 7fc226a9a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc220112400 con 0x7fc2201021a0 2024-04-22T13:24:17.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.931+0000 7fc225037640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc220074cd0 0x7fc2201a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:17.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.931+0000 7fc225037640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc220074cd0 0x7fc2201a6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59510/0 (socket says 172.21.15.12:59510) 2024-04-22T13:24:17.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.931+0000 7fc225037640 1 -- 172.21.15.12:0/3089178520 learned_addr learned my addr 172.21.15.12:0/3089178520 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:17.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.931+0000 7fc224836640 1 --2- 172.21.15.12:0/3089178520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2201021a0 0x7fc220074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:17.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.931+0000 7fc21ffff640 1 --2- 172.21.15.12:0/3089178520 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc220107270 0x7fc220074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:17.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.931+0000 7fc225037640 1 -- 172.21.15.12:0/3089178520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2201021a0 msgr2=0x7fc220074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:17.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.931+0000 7fc225037640 1 --2- 172.21.15.12:0/3089178520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2201021a0 0x7fc220074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:17.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.931+0000 7fc225037640 1 -- 172.21.15.12:0/3089178520 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc220107270 msgr2=0x7fc220074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:17.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.931+0000 7fc225037640 1 --2- 172.21.15.12:0/3089178520 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc220107270 0x7fc220074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:17.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.931+0000 7fc225037640 1 -- 172.21.15.12:0/3089178520 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2201a7050 con 0x7fc220074cd0 2024-04-22T13:24:17.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.931+0000 7fc21ffff640 1 --2- 172.21.15.12:0/3089178520 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc220107270 0x7fc220074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:17.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.931+0000 7fc225037640 1 --2- 172.21.15.12:0/3089178520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc220074cd0 0x7fc2201a6950 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc21800eaa0 tx=0x7fc21800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:17.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.931+0000 7fc21dffb640 1 -- 172.21.15.12:0/3089178520 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21800cdb0 con 0x7fc220074cd0 2024-04-22T13:24:17.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.932+0000 7fc21dffb640 1 -- 172.21.15.12:0/3089178520 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc218005020 con 0x7fc220074cd0 2024-04-22T13:24:17.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.932+0000 7fc226a9a640 1 -- 172.21.15.12:0/3089178520 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2201a7340 con 0x7fc220074cd0 2024-04-22T13:24:17.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.932+0000 7fc21dffb640 1 -- 172.21.15.12:0/3089178520 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2180116d0 con 0x7fc220074cd0 2024-04-22T13:24:17.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.932+0000 7fc226a9a640 1 -- 172.21.15.12:0/3089178520 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2201a7890 con 0x7fc220074cd0 2024-04-22T13:24:17.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.933+0000 7fc226a9a640 1 -- 172.21.15.12:0/3089178520 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc220069230 con 0x7fc220074cd0 2024-04-22T13:24:17.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.934+0000 7fc21dffb640 1 -- 172.21.15.12:0/3089178520 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fc2180051d0 con 0x7fc220074cd0 2024-04-22T13:24:17.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.934+0000 7fc21dffb640 1 --2- 172.21.15.12:0/3089178520 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc2100a3ef0 0x7fc2100a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:17.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.935+0000 7fc224836640 1 --2- 172.21.15.12:0/3089178520 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc2100a3ef0 0x7fc2100a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:17.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.935+0000 7fc21dffb640 1 -- 172.21.15.12:0/3089178520 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fc2180d0c90 con 0x7fc220074cd0 2024-04-22T13:24:17.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.935+0000 7fc224836640 1 --2- 172.21.15.12:0/3089178520 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc2100a3ef0 0x7fc2100a63b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc214006010 tx=0x7fc214000bc0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:17.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:17.939+0000 7fc21dffb640 1 -- 172.21.15.12:0/3089178520 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc218016030 con 0x7fc220074cd0 2024-04-22T13:24:17.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:17 smithi111 ceph-mon[31361]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1 KiB/s wr, 20 op/s 2024-04-22T13:24:17.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:17 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/3844291435' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:24:17.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:17 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2665104165' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:24:17.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:17 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/19074878' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:24:18.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.123+0000 7fc226a9a640 1 -- 172.21.15.12:0/3089178520 --> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc220102b50 con 0x7fc2100a3ef0 2024-04-22T13:24:18.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.130+0000 7fc21dffb640 1 -- 172.21.15.12:0/3089178520 <== mgr.24968 v2:172.21.15.12:6800/1615661596 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257790 (secure 0 0 0) 0x7fc220102b50 con 0x7fc2100a3ef0 2024-04-22T13:24:18.130 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:24:18.131 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-22T13:24:18.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.133+0000 7fc226a9a640 1 -- 172.21.15.12:0/3089178520 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc2100a3ef0 msgr2=0x7fc2100a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:18.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.133+0000 7fc226a9a640 1 --2- 172.21.15.12:0/3089178520 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc2100a3ef0 0x7fc2100a63b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc214006010 tx=0x7fc214000bc0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.133+0000 7fc226a9a640 1 -- 172.21.15.12:0/3089178520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc220074cd0 msgr2=0x7fc2201a6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:18.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.133+0000 7fc226a9a640 1 --2- 172.21.15.12:0/3089178520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc220074cd0 0x7fc2201a6950 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc21800eaa0 tx=0x7fc21800ef60 comp rx=0 tx=0).stop 2024-04-22T13:24:18.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.133+0000 7fc226a9a640 1 -- 172.21.15.12:0/3089178520 shutdown_connections 2024-04-22T13:24:18.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.133+0000 7fc226a9a640 1 --2- 172.21.15.12:0/3089178520 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc2100a3ef0 0x7fc2100a63b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.133+0000 7fc226a9a640 1 --2- 172.21.15.12:0/3089178520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc220074cd0 0x7fc2201a6950 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.133+0000 7fc226a9a640 1 --2- 172.21.15.12:0/3089178520 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc220107270 0x7fc220074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.133+0000 7fc226a9a640 1 --2- 172.21.15.12:0/3089178520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2201021a0 0x7fc220074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.133+0000 7fc226a9a640 1 -- 172.21.15.12:0/3089178520 >> 172.21.15.12:0/3089178520 conn(0x7fc2200793a0 msgr2=0x7fc2200ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:18.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.133+0000 7fc226a9a640 1 -- 172.21.15.12:0/3089178520 shutdown_connections 2024-04-22T13:24:18.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.133+0000 7fc226a9a640 1 -- 172.21.15.12:0/3089178520 wait complete. 2024-04-22T13:24:18.141 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-04-22T13:24:16.177526+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1193446,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1592,"num_read_kb":1126,"num_write":580,"num_write_kb":4411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":955,"ondisk_log_size":955,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381080,"kb_used_data":15264,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124560744,"statfs":{"total":1151940427776,"available":1151550201856,"internally_reserved":0,"allocated":15630336,"data_stored":9230754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4084,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":156,"num_read_kb":243,"num_write":29,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.992501"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418512+0000","last_change":"2024-04-22T13:22:05.254388+0000","last_active":"2024-04-22T13:24:06.418512+0000","last_peered":"2024-04-22T13:24:06.418512+0000","last_clean":"2024-04-22T13:24:06.418512+0000","last_became_active":"2024-04-22T13:22:05.254042+0000","last_became_peered":"2024-04-22T13:22:05.254042+0000","last_unstale":"2024-04-22T13:24:06.418512+0000","last_undegraded":"2024-04-22T13:24:06.418512+0000","last_fullsized":"2024-04-22T13:24:06.418512+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:07:56.139653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.412069+0000","last_change":"2024-04-22T13:22:05.247879+0000","last_active":"2024-04-22T13:24:06.412069+0000","last_peered":"2024-04-22T13:24:06.412069+0000","last_clean":"2024-04-22T13:24:06.412069+0000","last_became_active":"2024-04-22T13:22:05.247508+0000","last_became_peered":"2024-04-22T13:22:05.247508+0000","last_unstale":"2024-04-22T13:24:06.412069+0000","last_undegraded":"2024-04-22T13:24:06.412069+0000","last_fullsized":"2024-04-22T13:24:06.412069+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:55:51.928658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411912+0000","last_change":"2024-04-22T13:22:05.257970+0000","last_active":"2024-04-22T13:24:06.411912+0000","last_peered":"2024-04-22T13:24:06.411912+0000","last_clean":"2024-04-22T13:24:06.411912+0000","last_became_active":"2024-04-22T13:22:05.257652+0000","last_became_peered":"2024-04-22T13:22:05.257652+0000","last_unstale":"2024-04-22T13:24:06.411912+0000","last_undegraded":"2024-04-22T13:24:06.411912+0000","last_fullsized":"2024-04-22T13:24:06.411912+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:42:17.476349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411332+0000","last_change":"2024-04-22T13:22:05.249186+0000","last_active":"2024-04-22T13:24:06.411332+0000","last_peered":"2024-04-22T13:24:06.411332+0000","last_clean":"2024-04-22T13:24:06.411332+0000","last_became_active":"2024-04-22T13:22:05.248814+0000","last_became_peered":"2024-04-22T13:22:05.248814+0000","last_unstale":"2024-04-22T13:24:06.411332+0000","last_undegraded":"2024-04-22T13:24:06.411332+0000","last_fullsized":"2024-04-22T13:24:06.411332+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:57.450794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258738+0000","last_change":"2024-04-22T13:22:05.251953+0000","last_active":"2024-04-22T13:24:06.258738+0000","last_peered":"2024-04-22T13:24:06.258738+0000","last_clean":"2024-04-22T13:24:06.258738+0000","last_became_active":"2024-04-22T13:22:05.251547+0000","last_became_peered":"2024-04-22T13:22:05.251547+0000","last_unstale":"2024-04-22T13:24:06.258738+0000","last_undegraded":"2024-04-22T13:24:06.258738+0000","last_fullsized":"2024-04-22T13:24:06.258738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:36:34.809878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.266346+0000","last_change":"2024-04-22T13:23:05.510044+0000","last_active":"2024-04-22T13:24:06.266346+0000","last_peered":"2024-04-22T13:24:06.266346+0000","last_clean":"2024-04-22T13:24:06.266346+0000","last_became_active":"2024-04-22T13:23:05.509677+0000","last_became_peered":"2024-04-22T13:23:05.509677+0000","last_unstale":"2024-04-22T13:24:06.266346+0000","last_undegraded":"2024-04-22T13:24:06.266346+0000","last_fullsized":"2024-04-22T13:24:06.266346+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:46.979291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409036+0000","last_change":"2024-04-22T13:22:05.250418+0000","last_active":"2024-04-22T13:24:06.409036+0000","last_peered":"2024-04-22T13:24:06.409036+0000","last_clean":"2024-04-22T13:24:06.409036+0000","last_became_active":"2024-04-22T13:22:05.250097+0000","last_became_peered":"2024-04-22T13:22:05.250097+0000","last_unstale":"2024-04-22T13:24:06.409036+0000","last_undegraded":"2024-04-22T13:24:06.409036+0000","last_fullsized":"2024-04-22T13:24:06.409036+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:34:59.598950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258759+0000","last_change":"2024-04-22T13:22:05.244762+0000","last_active":"2024-04-22T13:24:06.258759+0000","last_peered":"2024-04-22T13:24:06.258759+0000","last_clean":"2024-04-22T13:24:06.258759+0000","last_became_active":"2024-04-22T13:22:05.244430+0000","last_became_peered":"2024-04-22T13:22:05.244430+0000","last_unstale":"2024-04-22T13:24:06.258759+0000","last_undegraded":"2024-04-22T13:24:06.258759+0000","last_fullsized":"2024-04-22T13:24:06.258759+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:33:01.582851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164321+0000","last_change":"2024-04-22T13:22:05.257596+0000","last_active":"2024-04-22T13:24:06.164321+0000","last_peered":"2024-04-22T13:24:06.164321+0000","last_clean":"2024-04-22T13:24:06.164321+0000","last_became_active":"2024-04-22T13:22:05.257277+0000","last_became_peered":"2024-04-22T13:22:05.257277+0000","last_unstale":"2024-04-22T13:24:06.164321+0000","last_undegraded":"2024-04-22T13:24:06.164321+0000","last_fullsized":"2024-04-22T13:24:06.164321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:09:44.715965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.266708+0000","last_change":"2024-04-22T13:22:05.252177+0000","last_active":"2024-04-22T13:24:06.266708+0000","last_peered":"2024-04-22T13:24:06.266708+0000","last_clean":"2024-04-22T13:24:06.266708+0000","last_became_active":"2024-04-22T13:22:05.251794+0000","last_became_peered":"2024-04-22T13:22:05.251794+0000","last_unstale":"2024-04-22T13:24:06.266708+0000","last_undegraded":"2024-04-22T13:24:06.266708+0000","last_fullsized":"2024-04-22T13:24:06.266708+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:08:59.500540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409476+0000","last_change":"2024-04-22T13:22:05.256209+0000","last_active":"2024-04-22T13:24:06.409476+0000","last_peered":"2024-04-22T13:24:06.409476+0000","last_clean":"2024-04-22T13:24:06.409476+0000","last_became_active":"2024-04-22T13:22:05.255504+0000","last_became_peered":"2024-04-22T13:22:05.255504+0000","last_unstale":"2024-04-22T13:24:06.409476+0000","last_undegraded":"2024-04-22T13:24:06.409476+0000","last_fullsized":"2024-04-22T13:24:06.409476+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:32:08.190921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.256744+0000","last_change":"2024-04-22T13:23:05.509837+0000","last_active":"2024-04-22T13:24:06.256744+0000","last_peered":"2024-04-22T13:24:06.256744+0000","last_clean":"2024-04-22T13:24:06.256744+0000","last_became_active":"2024-04-22T13:23:05.509421+0000","last_became_peered":"2024-04-22T13:23:05.509421+0000","last_unstale":"2024-04-22T13:24:06.256744+0000","last_undegraded":"2024-04-22T13:24:06.256744+0000","last_fullsized":"2024-04-22T13:24:06.256744+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:30:01.103373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162449+0000","last_change":"2024-04-22T13:22:05.246667+0000","last_active":"2024-04-22T13:24:06.162449+0000","last_peered":"2024-04-22T13:24:06.162449+0000","last_clean":"2024-04-22T13:24:06.162449+0000","last_became_active":"2024-04-22T13:22:05.246365+0000","last_became_peered":"2024-04-22T13:22:05.246365+0000","last_unstale":"2024-04-22T13:24:06.162449+0000","last_undegraded":"2024-04-22T13:24:06.162449+0000","last_fullsized":"2024-04-22T13:24:06.162449+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:41:55.533430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411527+0000","last_change":"2024-04-22T13:22:05.251982+0000","last_active":"2024-04-22T13:24:06.411527+0000","last_peered":"2024-04-22T13:24:06.411527+0000","last_clean":"2024-04-22T13:24:06.411527+0000","last_became_active":"2024-04-22T13:22:05.251575+0000","last_became_peered":"2024-04-22T13:22:05.251575+0000","last_unstale":"2024-04-22T13:24:06.411527+0000","last_undegraded":"2024-04-22T13:24:06.411527+0000","last_fullsized":"2024-04-22T13:24:06.411527+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:28:05.343140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411427+0000","last_change":"2024-04-22T13:22:05.251056+0000","last_active":"2024-04-22T13:24:06.411427+0000","last_peered":"2024-04-22T13:24:06.411427+0000","last_clean":"2024-04-22T13:24:06.411427+0000","last_became_active":"2024-04-22T13:22:05.250652+0000","last_became_peered":"2024-04-22T13:22:05.250652+0000","last_unstale":"2024-04-22T13:24:06.411427+0000","last_undegraded":"2024-04-22T13:24:06.411427+0000","last_fullsized":"2024-04-22T13:24:06.411427+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:08:28.629965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.163636+0000","last_change":"2024-04-22T13:22:05.256877+0000","last_active":"2024-04-22T13:24:06.163636+0000","last_peered":"2024-04-22T13:24:06.163636+0000","last_clean":"2024-04-22T13:24:06.163636+0000","last_became_active":"2024-04-22T13:22:05.256074+0000","last_became_peered":"2024-04-22T13:22:05.256074+0000","last_unstale":"2024-04-22T13:24:06.163636+0000","last_undegraded":"2024-04-22T13:24:06.163636+0000","last_fullsized":"2024-04-22T13:24:06.163636+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:08:53.607626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418592+0000","last_change":"2024-04-22T13:22:05.255598+0000","last_active":"2024-04-22T13:24:06.418592+0000","last_peered":"2024-04-22T13:24:06.418592+0000","last_clean":"2024-04-22T13:24:06.418592+0000","last_became_active":"2024-04-22T13:22:05.255239+0000","last_became_peered":"2024-04-22T13:22:05.255239+0000","last_unstale":"2024-04-22T13:24:06.418592+0000","last_undegraded":"2024-04-22T13:24:06.418592+0000","last_fullsized":"2024-04-22T13:24:06.418592+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:38:20.547882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.257085+0000","last_change":"2024-04-22T13:22:05.254232+0000","last_active":"2024-04-22T13:24:06.257085+0000","last_peered":"2024-04-22T13:24:06.257085+0000","last_clean":"2024-04-22T13:24:06.257085+0000","last_became_active":"2024-04-22T13:22:05.253907+0000","last_became_peered":"2024-04-22T13:22:05.253907+0000","last_unstale":"2024-04-22T13:24:06.257085+0000","last_undegraded":"2024-04-22T13:24:06.257085+0000","last_fullsized":"2024-04-22T13:24:06.257085+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:15:49.931112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.415196+0000","last_change":"2024-04-22T13:22:05.251518+0000","last_active":"2024-04-22T13:24:06.415196+0000","last_peered":"2024-04-22T13:24:06.415196+0000","last_clean":"2024-04-22T13:24:06.415196+0000","last_became_active":"2024-04-22T13:22:05.250478+0000","last_became_peered":"2024-04-22T13:22:05.250478+0000","last_unstale":"2024-04-22T13:24:06.415196+0000","last_undegraded":"2024-04-22T13:24:06.415196+0000","last_fullsized":"2024-04-22T13:24:06.415196+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:25:26.758692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'14","reported_seq":157,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:10.565206+0000","last_change":"2024-04-22T13:22:04.241365+0000","last_active":"2024-04-22T13:24:10.565206+0000","last_peered":"2024-04-22T13:24:10.565206+0000","last_clean":"2024-04-22T13:24:10.565206+0000","last_became_active":"2024-04-22T13:22:04.240822+0000","last_became_peered":"2024-04-22T13:22:04.240822+0000","last_unstale":"2024-04-22T13:24:10.565206+0000","last_undegraded":"2024-04-22T13:24:10.565206+0000","last_fullsized":"2024-04-22T13:24:10.565206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:20:23.928534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.256956+0000","last_change":"2024-04-22T13:23:05.876262+0000","last_active":"2024-04-22T13:24:06.256956+0000","last_peered":"2024-04-22T13:24:06.256956+0000","last_clean":"2024-04-22T13:24:06.256956+0000","last_became_active":"2024-04-22T13:23:05.875872+0000","last_became_peered":"2024-04-22T13:23:05.875872+0000","last_unstale":"2024-04-22T13:24:06.256956+0000","last_undegraded":"2024-04-22T13:24:06.256956+0000","last_fullsized":"2024-04-22T13:24:06.256956+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:29:33.092887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259532+0000","last_change":"2024-04-22T13:22:04.248872+0000","last_active":"2024-04-22T13:24:06.259532+0000","last_peered":"2024-04-22T13:24:06.259532+0000","last_clean":"2024-04-22T13:24:06.259532+0000","last_became_active":"2024-04-22T13:22:04.248466+0000","last_became_peered":"2024-04-22T13:22:04.248466+0000","last_unstale":"2024-04-22T13:24:06.259532+0000","last_undegraded":"2024-04-22T13:24:06.259532+0000","last_fullsized":"2024-04-22T13:24:06.259532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:38:10.871602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.412438+0000","last_change":"2024-04-22T13:22:05.258005+0000","last_active":"2024-04-22T13:24:06.412438+0000","last_peered":"2024-04-22T13:24:06.412438+0000","last_clean":"2024-04-22T13:24:06.412438+0000","last_became_active":"2024-04-22T13:22:05.257615+0000","last_became_peered":"2024-04-22T13:22:05.257615+0000","last_unstale":"2024-04-22T13:24:06.412438+0000","last_undegraded":"2024-04-22T13:24:06.412438+0000","last_fullsized":"2024-04-22T13:24:06.412438+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:21:18.030530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259060+0000","last_change":"2024-04-22T13:22:04.246273+0000","last_active":"2024-04-22T13:24:06.259060+0000","last_peered":"2024-04-22T13:24:06.259060+0000","last_clean":"2024-04-22T13:24:06.259060+0000","last_became_active":"2024-04-22T13:22:04.245969+0000","last_became_peered":"2024-04-22T13:22:04.245969+0000","last_unstale":"2024-04-22T13:24:06.259060+0000","last_undegraded":"2024-04-22T13:24:06.259060+0000","last_fullsized":"2024-04-22T13:24:06.259060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:10.260192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.163964+0000","last_change":"2024-04-22T13:22:05.260299+0000","last_active":"2024-04-22T13:24:06.163964+0000","last_peered":"2024-04-22T13:24:06.163964+0000","last_clean":"2024-04-22T13:24:06.163964+0000","last_became_active":"2024-04-22T13:22:05.259634+0000","last_became_peered":"2024-04-22T13:22:05.259634+0000","last_unstale":"2024-04-22T13:24:06.163964+0000","last_undegraded":"2024-04-22T13:24:06.163964+0000","last_fullsized":"2024-04-22T13:24:06.163964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:16:44.421652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258970+0000","last_change":"2024-04-22T13:22:04.243843+0000","last_active":"2024-04-22T13:24:06.258970+0000","last_peered":"2024-04-22T13:24:06.258970+0000","last_clean":"2024-04-22T13:24:06.258970+0000","last_became_active":"2024-04-22T13:22:04.243465+0000","last_became_peered":"2024-04-22T13:22:04.243465+0000","last_unstale":"2024-04-22T13:24:06.258970+0000","last_undegraded":"2024-04-22T13:24:06.258970+0000","last_fullsized":"2024-04-22T13:24:06.258970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:48:36.431697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164717+0000","last_change":"2024-04-22T13:22:05.259789+0000","last_active":"2024-04-22T13:24:06.164717+0000","last_peered":"2024-04-22T13:24:06.164717+0000","last_clean":"2024-04-22T13:24:06.164717+0000","last_became_active":"2024-04-22T13:22:05.259395+0000","last_became_peered":"2024-04-22T13:22:05.259395+0000","last_unstale":"2024-04-22T13:24:06.164717+0000","last_undegraded":"2024-04-22T13:24:06.164717+0000","last_fullsized":"2024-04-22T13:24:06.164717+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:43:51.430561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'7","reported_seq":155,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:10.539561+0000","last_change":"2024-04-22T13:22:04.240615+0000","last_active":"2024-04-22T13:24:10.539561+0000","last_peered":"2024-04-22T13:24:10.539561+0000","last_clean":"2024-04-22T13:24:10.539561+0000","last_became_active":"2024-04-22T13:22:04.240248+0000","last_became_peered":"2024-04-22T13:22:04.240248+0000","last_unstale":"2024-04-22T13:24:10.539561+0000","last_undegraded":"2024-04-22T13:24:10.539561+0000","last_fullsized":"2024-04-22T13:24:10.539561+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:55:05.948979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1404,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":127,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258665+0000","last_change":"2024-04-22T13:22:05.253788+0000","last_active":"2024-04-22T13:24:06.258665+0000","last_peered":"2024-04-22T13:24:06.258665+0000","last_clean":"2024-04-22T13:24:06.258665+0000","last_became_active":"2024-04-22T13:22:05.252578+0000","last_became_peered":"2024-04-22T13:22:05.252578+0000","last_unstale":"2024-04-22T13:24:06.258665+0000","last_undegraded":"2024-04-22T13:24:06.258665+0000","last_fullsized":"2024-04-22T13:24:06.258665+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:04:11.769930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.266274+0000","last_change":"2024-04-22T13:22:04.241170+0000","last_active":"2024-04-22T13:24:06.266274+0000","last_peered":"2024-04-22T13:24:06.266274+0000","last_clean":"2024-04-22T13:24:06.266274+0000","last_became_active":"2024-04-22T13:22:04.240830+0000","last_became_peered":"2024-04-22T13:22:04.240830+0000","last_unstale":"2024-04-22T13:24:06.266274+0000","last_undegraded":"2024-04-22T13:24:06.266274+0000","last_fullsized":"2024-04-22T13:24:06.266274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:51:30.520613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164733+0000","last_change":"2024-04-22T13:22:05.237900+0000","last_active":"2024-04-22T13:24:06.164733+0000","last_peered":"2024-04-22T13:24:06.164733+0000","last_clean":"2024-04-22T13:24:06.164733+0000","last_became_active":"2024-04-22T13:22:05.237507+0000","last_became_peered":"2024-04-22T13:22:05.237507+0000","last_unstale":"2024-04-22T13:24:06.164733+0000","last_undegraded":"2024-04-22T13:24:06.164733+0000","last_fullsized":"2024-04-22T13:24:06.164733+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:26:24.307803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162328+0000","last_change":"2024-04-22T13:22:04.236078+0000","last_active":"2024-04-22T13:24:06.162328+0000","last_peered":"2024-04-22T13:24:06.162328+0000","last_clean":"2024-04-22T13:24:06.162328+0000","last_became_active":"2024-04-22T13:22:04.235710+0000","last_became_peered":"2024-04-22T13:22:04.235710+0000","last_unstale":"2024-04-22T13:24:06.162328+0000","last_undegraded":"2024-04-22T13:24:06.162328+0000","last_fullsized":"2024-04-22T13:24:06.162328+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:37:55.335237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258717+0000","last_change":"2024-04-22T13:22:05.243152+0000","last_active":"2024-04-22T13:24:06.258717+0000","last_peered":"2024-04-22T13:24:06.258717+0000","last_clean":"2024-04-22T13:24:06.258717+0000","last_became_active":"2024-04-22T13:22:05.242772+0000","last_became_peered":"2024-04-22T13:22:05.242772+0000","last_unstale":"2024-04-22T13:24:06.258717+0000","last_undegraded":"2024-04-22T13:24:06.258717+0000","last_fullsized":"2024-04-22T13:24:06.258717+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:33:42.359679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164258+0000","last_change":"2024-04-22T13:22:04.244381+0000","last_active":"2024-04-22T13:24:06.164258+0000","last_peered":"2024-04-22T13:24:06.164258+0000","last_clean":"2024-04-22T13:24:06.164258+0000","last_became_active":"2024-04-22T13:22:04.243746+0000","last_became_peered":"2024-04-22T13:22:04.243746+0000","last_unstale":"2024-04-22T13:24:06.164258+0000","last_undegraded":"2024-04-22T13:24:06.164258+0000","last_fullsized":"2024-04-22T13:24:06.164258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:15:36.019930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.256753+0000","last_change":"2024-04-22T13:22:05.245394+0000","last_active":"2024-04-22T13:24:06.256753+0000","last_peered":"2024-04-22T13:24:06.256753+0000","last_clean":"2024-04-22T13:24:06.256753+0000","last_became_active":"2024-04-22T13:22:05.243931+0000","last_became_peered":"2024-04-22T13:22:05.243931+0000","last_unstale":"2024-04-22T13:24:06.256753+0000","last_undegraded":"2024-04-22T13:24:06.256753+0000","last_fullsized":"2024-04-22T13:24:06.256753+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:11:10.790842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'6","reported_seq":270,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:11.540611+0000","last_change":"2024-04-22T13:22:04.242583+0000","last_active":"2024-04-22T13:24:11.540611+0000","last_peered":"2024-04-22T13:24:11.540611+0000","last_clean":"2024-04-22T13:24:11.540611+0000","last_became_active":"2024-04-22T13:22:04.242261+0000","last_became_peered":"2024-04-22T13:22:04.242261+0000","last_unstale":"2024-04-22T13:24:11.540611+0000","last_undegraded":"2024-04-22T13:24:11.540611+0000","last_fullsized":"2024-04-22T13:24:11.540611+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:37:47.618573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":243,"num_read_kb":243,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164258+0000","last_change":"2024-04-22T13:22:05.255293+0000","last_active":"2024-04-22T13:24:06.164258+0000","last_peered":"2024-04-22T13:24:06.164258+0000","last_clean":"2024-04-22T13:24:06.164258+0000","last_became_active":"2024-04-22T13:22:05.253949+0000","last_became_peered":"2024-04-22T13:22:05.253949+0000","last_unstale":"2024-04-22T13:24:06.164258+0000","last_undegraded":"2024-04-22T13:24:06.164258+0000","last_fullsized":"2024-04-22T13:24:06.164258+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:41:08.754989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409503+0000","last_change":"2024-04-22T13:22:04.241529+0000","last_active":"2024-04-22T13:24:06.409503+0000","last_peered":"2024-04-22T13:24:06.409503+0000","last_clean":"2024-04-22T13:24:06.409503+0000","last_became_active":"2024-04-22T13:22:04.241164+0000","last_became_peered":"2024-04-22T13:22:04.241164+0000","last_unstale":"2024-04-22T13:24:06.409503+0000","last_undegraded":"2024-04-22T13:24:06.409503+0000","last_fullsized":"2024-04-22T13:24:06.409503+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:16:49.387181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.412225+0000","last_change":"2024-04-22T13:22:05.241623+0000","last_active":"2024-04-22T13:24:06.412225+0000","last_peered":"2024-04-22T13:24:06.412225+0000","last_clean":"2024-04-22T13:24:06.412225+0000","last_became_active":"2024-04-22T13:22:05.241150+0000","last_became_peered":"2024-04-22T13:22:05.241150+0000","last_unstale":"2024-04-22T13:24:06.412225+0000","last_undegraded":"2024-04-22T13:24:06.412225+0000","last_fullsized":"2024-04-22T13:24:06.412225+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:17:28.412366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164893+0000","last_change":"2024-04-22T13:22:04.247852+0000","last_active":"2024-04-22T13:24:06.164893+0000","last_peered":"2024-04-22T13:24:06.164893+0000","last_clean":"2024-04-22T13:24:06.164893+0000","last_became_active":"2024-04-22T13:22:04.247550+0000","last_became_peered":"2024-04-22T13:22:04.247550+0000","last_unstale":"2024-04-22T13:24:06.164893+0000","last_undegraded":"2024-04-22T13:24:06.164893+0000","last_fullsized":"2024-04-22T13:24:06.164893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:55:01.928435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164057+0000","last_change":"2024-04-22T13:22:05.260050+0000","last_active":"2024-04-22T13:24:06.164057+0000","last_peered":"2024-04-22T13:24:06.164057+0000","last_clean":"2024-04-22T13:24:06.164057+0000","last_became_active":"2024-04-22T13:22:05.259098+0000","last_became_peered":"2024-04-22T13:22:05.259098+0000","last_unstale":"2024-04-22T13:24:06.164057+0000","last_undegraded":"2024-04-22T13:24:06.164057+0000","last_fullsized":"2024-04-22T13:24:06.164057+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:05:06.245492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'38","reported_seq":319,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:11.540734+0000","last_change":"2024-04-22T13:22:04.240233+0000","last_active":"2024-04-22T13:24:11.540734+0000","last_peered":"2024-04-22T13:24:11.540734+0000","last_clean":"2024-04-22T13:24:11.540734+0000","last_became_active":"2024-04-22T13:22:04.239890+0000","last_became_peered":"2024-04-22T13:22:04.239890+0000","last_unstale":"2024-04-22T13:24:11.540734+0000","last_undegraded":"2024-04-22T13:24:11.540734+0000","last_fullsized":"2024-04-22T13:24:11.540734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:04:21.547422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67638,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":260,"num_read_kb":197,"num_write":40,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258900+0000","last_change":"2024-04-22T13:22:05.257756+0000","last_active":"2024-04-22T13:24:06.258900+0000","last_peered":"2024-04-22T13:24:06.258900+0000","last_clean":"2024-04-22T13:24:06.258900+0000","last_became_active":"2024-04-22T13:22:05.257442+0000","last_became_peered":"2024-04-22T13:22:05.257442+0000","last_unstale":"2024-04-22T13:24:06.258900+0000","last_undegraded":"2024-04-22T13:24:06.258900+0000","last_fullsized":"2024-04-22T13:24:06.258900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:16:53.465689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.414923+0000","last_change":"2024-04-22T13:22:04.240815+0000","last_active":"2024-04-22T13:24:06.414923+0000","last_peered":"2024-04-22T13:24:06.414923+0000","last_clean":"2024-04-22T13:24:06.414923+0000","last_became_active":"2024-04-22T13:22:04.240218+0000","last_became_peered":"2024-04-22T13:22:04.240218+0000","last_unstale":"2024-04-22T13:24:06.414923+0000","last_undegraded":"2024-04-22T13:24:06.414923+0000","last_fullsized":"2024-04-22T13:24:06.414923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:29:29.148764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162257+0000","last_change":"2024-04-22T13:22:05.248797+0000","last_active":"2024-04-22T13:24:06.162257+0000","last_peered":"2024-04-22T13:24:06.162257+0000","last_clean":"2024-04-22T13:24:06.162257+0000","last_became_active":"2024-04-22T13:22:05.248435+0000","last_became_peered":"2024-04-22T13:22:05.248435+0000","last_unstale":"2024-04-22T13:24:06.162257+0000","last_undegraded":"2024-04-22T13:24:06.162257+0000","last_fullsized":"2024-04-22T13:24:06.162257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:32:31.970494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164389+0000","last_change":"2024-04-22T13:22:04.245303+0000","last_active":"2024-04-22T13:24:06.164389+0000","last_peered":"2024-04-22T13:24:06.164389+0000","last_clean":"2024-04-22T13:24:06.164389+0000","last_became_active":"2024-04-22T13:22:04.244903+0000","last_became_peered":"2024-04-22T13:22:04.244903+0000","last_unstale":"2024-04-22T13:24:06.164389+0000","last_undegraded":"2024-04-22T13:24:06.164389+0000","last_fullsized":"2024-04-22T13:24:06.164389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:34:32.174858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409264+0000","last_change":"2024-04-22T13:22:05.246318+0000","last_active":"2024-04-22T13:24:06.409264+0000","last_peered":"2024-04-22T13:24:06.409264+0000","last_clean":"2024-04-22T13:24:06.409264+0000","last_became_active":"2024-04-22T13:22:05.245899+0000","last_became_peered":"2024-04-22T13:22:05.245899+0000","last_unstale":"2024-04-22T13:24:06.409264+0000","last_undegraded":"2024-04-22T13:24:06.409264+0000","last_fullsized":"2024-04-22T13:24:06.409264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:27:42.789058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'9","reported_seq":155,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:10.597760+0000","last_change":"2024-04-22T13:22:04.240612+0000","last_active":"2024-04-22T13:24:10.597760+0000","last_peered":"2024-04-22T13:24:10.597760+0000","last_clean":"2024-04-22T13:24:10.597760+0000","last_became_active":"2024-04-22T13:22:04.240290+0000","last_became_peered":"2024-04-22T13:22:04.240290+0000","last_unstale":"2024-04-22T13:24:10.597760+0000","last_undegraded":"2024-04-22T13:24:10.597760+0000","last_fullsized":"2024-04-22T13:24:10.597760+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:00:50.285553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1902,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":126,"num_read_kb":6,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.408976+0000","last_change":"2024-04-22T13:22:05.253205+0000","last_active":"2024-04-22T13:24:06.408976+0000","last_peered":"2024-04-22T13:24:06.408976+0000","last_clean":"2024-04-22T13:24:06.408976+0000","last_became_active":"2024-04-22T13:22:05.252865+0000","last_became_peered":"2024-04-22T13:22:05.252865+0000","last_unstale":"2024-04-22T13:24:06.408976+0000","last_undegraded":"2024-04-22T13:24:06.408976+0000","last_fullsized":"2024-04-22T13:24:06.408976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:19:12.482371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258636+0000","last_change":"2024-04-22T13:22:04.248938+0000","last_active":"2024-04-22T13:24:06.258636+0000","last_peered":"2024-04-22T13:24:06.258636+0000","last_clean":"2024-04-22T13:24:06.258636+0000","last_became_active":"2024-04-22T13:22:04.248440+0000","last_became_peered":"2024-04-22T13:22:04.248440+0000","last_unstale":"2024-04-22T13:24:06.258636+0000","last_undegraded":"2024-04-22T13:24:06.258636+0000","last_fullsized":"2024-04-22T13:24:06.258636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:46:50.928067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164391+0000","last_change":"2024-04-22T13:22:05.254664+0000","last_active":"2024-04-22T13:24:06.164391+0000","last_peered":"2024-04-22T13:24:06.164391+0000","last_clean":"2024-04-22T13:24:06.164391+0000","last_became_active":"2024-04-22T13:22:05.254304+0000","last_became_peered":"2024-04-22T13:22:05.254304+0000","last_unstale":"2024-04-22T13:24:06.164391+0000","last_undegraded":"2024-04-22T13:24:06.164391+0000","last_fullsized":"2024-04-22T13:24:06.164391+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:58:49.345573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259033+0000","last_change":"2024-04-22T13:22:04.244128+0000","last_active":"2024-04-22T13:24:06.259033+0000","last_peered":"2024-04-22T13:24:06.259033+0000","last_clean":"2024-04-22T13:24:06.259033+0000","last_became_active":"2024-04-22T13:22:04.243738+0000","last_became_peered":"2024-04-22T13:22:04.243738+0000","last_unstale":"2024-04-22T13:24:06.259033+0000","last_undegraded":"2024-04-22T13:24:06.259033+0000","last_fullsized":"2024-04-22T13:24:06.259033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:23:52.220584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418387+0000","last_change":"2024-04-22T13:22:05.250910+0000","last_active":"2024-04-22T13:24:06.418387+0000","last_peered":"2024-04-22T13:24:06.418387+0000","last_clean":"2024-04-22T13:24:06.418387+0000","last_became_active":"2024-04-22T13:22:05.250410+0000","last_became_peered":"2024-04-22T13:22:05.250410+0000","last_unstale":"2024-04-22T13:24:06.418387+0000","last_undegraded":"2024-04-22T13:24:06.418387+0000","last_fullsized":"2024-04-22T13:24:06.418387+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:04:25.492677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.256811+0000","last_change":"2024-04-22T13:22:04.245889+0000","last_active":"2024-04-22T13:24:06.256811+0000","last_peered":"2024-04-22T13:24:06.256811+0000","last_clean":"2024-04-22T13:24:06.256811+0000","last_became_active":"2024-04-22T13:22:04.245474+0000","last_became_peered":"2024-04-22T13:22:04.245474+0000","last_unstale":"2024-04-22T13:24:06.256811+0000","last_undegraded":"2024-04-22T13:24:06.256811+0000","last_fullsized":"2024-04-22T13:24:06.256811+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:22:12.626950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.266594+0000","last_change":"2024-04-22T13:22:05.250978+0000","last_active":"2024-04-22T13:24:06.266594+0000","last_peered":"2024-04-22T13:24:06.266594+0000","last_clean":"2024-04-22T13:24:06.266594+0000","last_became_active":"2024-04-22T13:22:05.250463+0000","last_became_peered":"2024-04-22T13:22:05.250463+0000","last_unstale":"2024-04-22T13:24:06.266594+0000","last_undegraded":"2024-04-22T13:24:06.266594+0000","last_fullsized":"2024-04-22T13:24:06.266594+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:45:18.591072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'7","reported_seq":151,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:10.565681+0000","last_change":"2024-04-22T13:22:04.242503+0000","last_active":"2024-04-22T13:24:10.565681+0000","last_peered":"2024-04-22T13:24:10.565681+0000","last_clean":"2024-04-22T13:24:10.565681+0000","last_became_active":"2024-04-22T13:22:04.242143+0000","last_became_peered":"2024-04-22T13:22:04.242143+0000","last_unstale":"2024-04-22T13:24:10.565681+0000","last_undegraded":"2024-04-22T13:24:10.565681+0000","last_fullsized":"2024-04-22T13:24:10.565681+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:40.236377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.266578+0000","last_change":"2024-04-22T13:22:05.252387+0000","last_active":"2024-04-22T13:24:06.266578+0000","last_peered":"2024-04-22T13:24:06.266578+0000","last_clean":"2024-04-22T13:24:06.266578+0000","last_became_active":"2024-04-22T13:22:05.252033+0000","last_became_peered":"2024-04-22T13:22:05.252033+0000","last_unstale":"2024-04-22T13:24:06.266578+0000","last_undegraded":"2024-04-22T13:24:06.266578+0000","last_fullsized":"2024-04-22T13:24:06.266578+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:55:01.791283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164456+0000","last_change":"2024-04-22T13:22:05.248394+0000","last_active":"2024-04-22T13:24:06.164456+0000","last_peered":"2024-04-22T13:24:06.164456+0000","last_clean":"2024-04-22T13:24:06.164456+0000","last_became_active":"2024-04-22T13:22:05.248024+0000","last_became_peered":"2024-04-22T13:22:05.248024+0000","last_unstale":"2024-04-22T13:24:06.164456+0000","last_undegraded":"2024-04-22T13:24:06.164456+0000","last_fullsized":"2024-04-22T13:24:06.164456+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:49:36.556136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'7","reported_seq":150,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:10.597275+0000","last_change":"2024-04-22T13:22:04.247692+0000","last_active":"2024-04-22T13:24:10.597275+0000","last_peered":"2024-04-22T13:24:10.597275+0000","last_clean":"2024-04-22T13:24:10.597275+0000","last_became_active":"2024-04-22T13:22:04.247345+0000","last_became_peered":"2024-04-22T13:22:04.247345+0000","last_unstale":"2024-04-22T13:24:10.597275+0000","last_undegraded":"2024-04-22T13:24:10.597275+0000","last_fullsized":"2024-04-22T13:24:10.597275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:07:49.803248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259343+0000","last_change":"2024-04-22T13:22:05.241022+0000","last_active":"2024-04-22T13:24:06.259343+0000","last_peered":"2024-04-22T13:24:06.259343+0000","last_clean":"2024-04-22T13:24:06.259343+0000","last_became_active":"2024-04-22T13:22:05.240635+0000","last_became_peered":"2024-04-22T13:22:05.240635+0000","last_unstale":"2024-04-22T13:24:06.259343+0000","last_undegraded":"2024-04-22T13:24:06.259343+0000","last_fullsized":"2024-04-22T13:24:06.259343+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:50:02.012260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411284+0000","last_change":"2024-04-22T13:22:04.236817+0000","last_active":"2024-04-22T13:24:06.411284+0000","last_peered":"2024-04-22T13:24:06.411284+0000","last_clean":"2024-04-22T13:24:06.411284+0000","last_became_active":"2024-04-22T13:22:04.236484+0000","last_became_peered":"2024-04-22T13:22:04.236484+0000","last_unstale":"2024-04-22T13:24:06.411284+0000","last_undegraded":"2024-04-22T13:24:06.411284+0000","last_fullsized":"2024-04-22T13:24:06.411284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:27:41.053327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"90'148","reported_seq":401,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:08.291461+0000","last_change":"2024-04-22T13:20:53.094123+0000","last_active":"2024-04-22T13:24:08.291461+0000","last_peered":"2024-04-22T13:24:08.291461+0000","last_clean":"2024-04-22T13:24:08.291461+0000","last_became_active":"2024-04-22T13:20:53.093664+0000","last_became_peered":"2024-04-22T13:20:53.093664+0000","last_unstale":"2024-04-22T13:24:08.291461+0000","last_undegraded":"2024-04-22T13:24:08.291461+0000","last_fullsized":"2024-04-22T13:24:08.291461+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:17:33.201786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:17:33.201786+0000","last_clean_scrub_stamp":"2024-04-22T13:17:33.201786+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:46:58.986399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162213+0000","last_change":"2024-04-22T13:22:05.247766+0000","last_active":"2024-04-22T13:24:06.162213+0000","last_peered":"2024-04-22T13:24:06.162213+0000","last_clean":"2024-04-22T13:24:06.162213+0000","last_became_active":"2024-04-22T13:22:05.247362+0000","last_became_peered":"2024-04-22T13:22:05.247362+0000","last_unstale":"2024-04-22T13:24:06.162213+0000","last_undegraded":"2024-04-22T13:24:06.162213+0000","last_fullsized":"2024-04-22T13:24:06.162213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:25:00.126638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409239+0000","last_change":"2024-04-22T13:22:04.243493+0000","last_active":"2024-04-22T13:24:06.409239+0000","last_peered":"2024-04-22T13:24:06.409239+0000","last_clean":"2024-04-22T13:24:06.409239+0000","last_became_active":"2024-04-22T13:22:04.243190+0000","last_became_peered":"2024-04-22T13:22:04.243190+0000","last_unstale":"2024-04-22T13:24:06.409239+0000","last_undegraded":"2024-04-22T13:24:06.409239+0000","last_fullsized":"2024-04-22T13:24:06.409239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:15:09.025152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258892+0000","last_change":"2024-04-22T13:22:04.249222+0000","last_active":"2024-04-22T13:24:06.258892+0000","last_peered":"2024-04-22T13:24:06.258892+0000","last_clean":"2024-04-22T13:24:06.258892+0000","last_became_active":"2024-04-22T13:22:04.248926+0000","last_became_peered":"2024-04-22T13:22:04.248926+0000","last_unstale":"2024-04-22T13:24:06.258892+0000","last_undegraded":"2024-04-22T13:24:06.258892+0000","last_fullsized":"2024-04-22T13:24:06.258892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:33:50.168426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162488+0000","last_change":"2024-04-22T13:22:05.248199+0000","last_active":"2024-04-22T13:24:06.162488+0000","last_peered":"2024-04-22T13:24:06.162488+0000","last_clean":"2024-04-22T13:24:06.162488+0000","last_became_active":"2024-04-22T13:22:05.247873+0000","last_became_peered":"2024-04-22T13:22:05.247873+0000","last_unstale":"2024-04-22T13:24:06.162488+0000","last_undegraded":"2024-04-22T13:24:06.162488+0000","last_fullsized":"2024-04-22T13:24:06.162488+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:40:29.353901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'49","reported_seq":94,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:07.392889+0000","last_change":"2024-04-22T13:22:04.243474+0000","last_active":"2024-04-22T13:24:07.392889+0000","last_peered":"2024-04-22T13:24:07.392889+0000","last_clean":"2024-04-22T13:24:07.392889+0000","last_became_active":"2024-04-22T13:22:04.243151+0000","last_became_peered":"2024-04-22T13:22:04.243151+0000","last_unstale":"2024-04-22T13:24:07.392889+0000","last_undegraded":"2024-04-22T13:24:07.392889+0000","last_fullsized":"2024-04-22T13:24:07.392889+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:18:25.140715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":86,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259264+0000","last_change":"2024-04-22T13:22:05.243120+0000","last_active":"2024-04-22T13:24:06.259264+0000","last_peered":"2024-04-22T13:24:06.259264+0000","last_clean":"2024-04-22T13:24:06.259264+0000","last_became_active":"2024-04-22T13:22:05.242726+0000","last_became_peered":"2024-04-22T13:22:05.242726+0000","last_unstale":"2024-04-22T13:24:06.259264+0000","last_undegraded":"2024-04-22T13:24:06.259264+0000","last_fullsized":"2024-04-22T13:24:06.259264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:06:16.087884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411424+0000","last_change":"2024-04-22T13:22:04.239672+0000","last_active":"2024-04-22T13:24:06.411424+0000","last_peered":"2024-04-22T13:24:06.411424+0000","last_clean":"2024-04-22T13:24:06.411424+0000","last_became_active":"2024-04-22T13:22:04.238776+0000","last_became_peered":"2024-04-22T13:22:04.238776+0000","last_unstale":"2024-04-22T13:24:06.411424+0000","last_undegraded":"2024-04-22T13:24:06.411424+0000","last_fullsized":"2024-04-22T13:24:06.411424+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:43:21.752547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.412035+0000","last_change":"2024-04-22T13:22:05.242089+0000","last_active":"2024-04-22T13:24:06.412035+0000","last_peered":"2024-04-22T13:24:06.412035+0000","last_clean":"2024-04-22T13:24:06.412035+0000","last_became_active":"2024-04-22T13:22:05.241681+0000","last_became_peered":"2024-04-22T13:22:05.241681+0000","last_unstale":"2024-04-22T13:24:06.412035+0000","last_undegraded":"2024-04-22T13:24:06.412035+0000","last_fullsized":"2024-04-22T13:24:06.412035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:58:49.906188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.163728+0000","last_change":"2024-04-22T13:23:05.513123+0000","last_active":"2024-04-22T13:24:06.163728+0000","last_peered":"2024-04-22T13:24:06.163728+0000","last_clean":"2024-04-22T13:24:06.163728+0000","last_became_active":"2024-04-22T13:23:05.509367+0000","last_became_peered":"2024-04-22T13:23:05.509367+0000","last_unstale":"2024-04-22T13:24:06.163728+0000","last_undegraded":"2024-04-22T13:24:06.163728+0000","last_fullsized":"2024-04-22T13:24:06.163728+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:28:41.770957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258638+0000","last_change":"2024-04-22T13:22:05.250120+0000","last_active":"2024-04-22T13:24:06.258638+0000","last_peered":"2024-04-22T13:24:06.258638+0000","last_clean":"2024-04-22T13:24:06.258638+0000","last_became_active":"2024-04-22T13:22:05.247936+0000","last_became_peered":"2024-04-22T13:22:05.247936+0000","last_unstale":"2024-04-22T13:24:06.258638+0000","last_undegraded":"2024-04-22T13:24:06.258638+0000","last_fullsized":"2024-04-22T13:24:06.258638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:27:38.437918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258902+0000","last_change":"2024-04-22T13:22:04.244026+0000","last_active":"2024-04-22T13:24:06.258902+0000","last_peered":"2024-04-22T13:24:06.258902+0000","last_clean":"2024-04-22T13:24:06.258902+0000","last_became_active":"2024-04-22T13:22:04.243518+0000","last_became_peered":"2024-04-22T13:22:04.243518+0000","last_unstale":"2024-04-22T13:24:06.258902+0000","last_undegraded":"2024-04-22T13:24:06.258902+0000","last_fullsized":"2024-04-22T13:24:06.258902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:54:48.791874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418220+0000","last_change":"2024-04-22T13:22:05.252674+0000","last_active":"2024-04-22T13:24:06.418220+0000","last_peered":"2024-04-22T13:24:06.418220+0000","last_clean":"2024-04-22T13:24:06.418220+0000","last_became_active":"2024-04-22T13:22:05.251087+0000","last_became_peered":"2024-04-22T13:22:05.251087+0000","last_unstale":"2024-04-22T13:24:06.418220+0000","last_undegraded":"2024-04-22T13:24:06.418220+0000","last_fullsized":"2024-04-22T13:24:06.418220+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:20:06.873412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.414693+0000","last_change":"2024-04-22T13:22:04.244409+0000","last_active":"2024-04-22T13:24:06.414693+0000","last_peered":"2024-04-22T13:24:06.414693+0000","last_clean":"2024-04-22T13:24:06.414693+0000","last_became_active":"2024-04-22T13:22:04.244104+0000","last_became_peered":"2024-04-22T13:22:04.244104+0000","last_unstale":"2024-04-22T13:24:06.414693+0000","last_undegraded":"2024-04-22T13:24:06.414693+0000","last_fullsized":"2024-04-22T13:24:06.414693+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:39:14.714755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259066+0000","last_change":"2024-04-22T13:22:05.245730+0000","last_active":"2024-04-22T13:24:06.259066+0000","last_peered":"2024-04-22T13:24:06.259066+0000","last_clean":"2024-04-22T13:24:06.259066+0000","last_became_active":"2024-04-22T13:22:05.245366+0000","last_became_peered":"2024-04-22T13:22:05.245366+0000","last_unstale":"2024-04-22T13:24:06.259066+0000","last_undegraded":"2024-04-22T13:24:06.259066+0000","last_fullsized":"2024-04-22T13:24:06.259066+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:31.935258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418212+0000","last_change":"2024-04-22T13:23:05.514197+0000","last_active":"2024-04-22T13:24:06.418212+0000","last_peered":"2024-04-22T13:24:06.418212+0000","last_clean":"2024-04-22T13:24:06.418212+0000","last_became_active":"2024-04-22T13:23:05.509431+0000","last_became_peered":"2024-04-22T13:23:05.509431+0000","last_unstale":"2024-04-22T13:24:06.418212+0000","last_undegraded":"2024-04-22T13:24:06.418212+0000","last_fullsized":"2024-04-22T13:24:06.418212+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:43:59.825567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409273+0000","last_change":"2024-04-22T13:22:05.242146+0000","last_active":"2024-04-22T13:24:06.409273+0000","last_peered":"2024-04-22T13:24:06.409273+0000","last_clean":"2024-04-22T13:24:06.409273+0000","last_became_active":"2024-04-22T13:22:05.241802+0000","last_became_peered":"2024-04-22T13:22:05.241802+0000","last_unstale":"2024-04-22T13:24:06.409273+0000","last_undegraded":"2024-04-22T13:24:06.409273+0000","last_fullsized":"2024-04-22T13:24:06.409273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:49:49.002123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'8","reported_seq":154,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:10.568340+0000","last_change":"2024-04-22T13:22:04.248884+0000","last_active":"2024-04-22T13:24:10.568340+0000","last_peered":"2024-04-22T13:24:10.568340+0000","last_clean":"2024-04-22T13:24:10.568340+0000","last_became_active":"2024-04-22T13:22:04.248393+0000","last_became_peered":"2024-04-22T13:22:04.248393+0000","last_unstale":"2024-04-22T13:24:10.568340+0000","last_undegraded":"2024-04-22T13:24:10.568340+0000","last_fullsized":"2024-04-22T13:24:10.568340+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:36:14.687457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2260,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":125,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409545+0000","last_change":"2024-04-22T13:22:05.256881+0000","last_active":"2024-04-22T13:24:06.409545+0000","last_peered":"2024-04-22T13:24:06.409545+0000","last_clean":"2024-04-22T13:24:06.409545+0000","last_became_active":"2024-04-22T13:22:05.255967+0000","last_became_peered":"2024-04-22T13:22:05.255967+0000","last_unstale":"2024-04-22T13:24:06.409545+0000","last_undegraded":"2024-04-22T13:24:06.409545+0000","last_fullsized":"2024-04-22T13:24:06.409545+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:24:21.137344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'41","reported_seq":70,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.367821+0000","last_change":"2024-04-22T13:22:04.247303+0000","last_active":"2024-04-22T13:24:06.367821+0000","last_peered":"2024-04-22T13:24:06.367821+0000","last_clean":"2024-04-22T13:24:06.367821+0000","last_became_active":"2024-04-22T13:22:04.246742+0000","last_became_peered":"2024-04-22T13:22:04.246742+0000","last_unstale":"2024-04-22T13:24:06.367821+0000","last_undegraded":"2024-04-22T13:24:06.367821+0000","last_fullsized":"2024-04-22T13:24:06.367821+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:45:03.189042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409288+0000","last_change":"2024-04-22T13:22:05.251754+0000","last_active":"2024-04-22T13:24:06.409288+0000","last_peered":"2024-04-22T13:24:06.409288+0000","last_clean":"2024-04-22T13:24:06.409288+0000","last_became_active":"2024-04-22T13:22:05.251155+0000","last_became_peered":"2024-04-22T13:22:05.251155+0000","last_unstale":"2024-04-22T13:24:06.409288+0000","last_undegraded":"2024-04-22T13:24:06.409288+0000","last_fullsized":"2024-04-22T13:24:06.409288+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:08:46.682600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162485+0000","last_change":"2024-04-22T13:22:04.243382+0000","last_active":"2024-04-22T13:24:06.162485+0000","last_peered":"2024-04-22T13:24:06.162485+0000","last_clean":"2024-04-22T13:24:06.162485+0000","last_became_active":"2024-04-22T13:22:04.243055+0000","last_became_peered":"2024-04-22T13:22:04.243055+0000","last_unstale":"2024-04-22T13:24:06.162485+0000","last_undegraded":"2024-04-22T13:24:06.162485+0000","last_fullsized":"2024-04-22T13:24:06.162485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:19:58.379645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409542+0000","last_change":"2024-04-22T13:22:05.249527+0000","last_active":"2024-04-22T13:24:06.409542+0000","last_peered":"2024-04-22T13:24:06.409542+0000","last_clean":"2024-04-22T13:24:06.409542+0000","last_became_active":"2024-04-22T13:22:05.249207+0000","last_became_peered":"2024-04-22T13:22:05.249207+0000","last_unstale":"2024-04-22T13:24:06.409542+0000","last_undegraded":"2024-04-22T13:24:06.409542+0000","last_fullsized":"2024-04-22T13:24:06.409542+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:50:02.755288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418285+0000","last_change":"2024-04-22T13:23:05.508324+0000","last_active":"2024-04-22T13:24:06.418285+0000","last_peered":"2024-04-22T13:24:06.418285+0000","last_clean":"2024-04-22T13:24:06.418285+0000","last_became_active":"2024-04-22T13:23:05.507814+0000","last_became_peered":"2024-04-22T13:23:05.507814+0000","last_unstale":"2024-04-22T13:24:06.418285+0000","last_undegraded":"2024-04-22T13:24:06.418285+0000","last_fullsized":"2024-04-22T13:24:06.418285+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:46:20.523715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.414640+0000","last_change":"2024-04-22T13:23:05.875683+0000","last_active":"2024-04-22T13:24:06.414640+0000","last_peered":"2024-04-22T13:24:06.414640+0000","last_clean":"2024-04-22T13:24:06.414640+0000","last_became_active":"2024-04-22T13:23:05.875150+0000","last_became_peered":"2024-04-22T13:23:05.875150+0000","last_unstale":"2024-04-22T13:24:06.414640+0000","last_undegraded":"2024-04-22T13:24:06.414640+0000","last_fullsized":"2024-04-22T13:24:06.414640+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:12:47.592230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.414612+0000","last_change":"2024-04-22T13:22:05.247882+0000","last_active":"2024-04-22T13:24:06.414612+0000","last_peered":"2024-04-22T13:24:06.414612+0000","last_clean":"2024-04-22T13:24:06.414612+0000","last_became_active":"2024-04-22T13:22:05.247525+0000","last_became_peered":"2024-04-22T13:22:05.247525+0000","last_unstale":"2024-04-22T13:24:06.414612+0000","last_undegraded":"2024-04-22T13:24:06.414612+0000","last_fullsized":"2024-04-22T13:24:06.414612+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:46:04.349076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411567+0000","last_change":"2024-04-22T13:22:05.243716+0000","last_active":"2024-04-22T13:24:06.411567+0000","last_peered":"2024-04-22T13:24:06.411567+0000","last_clean":"2024-04-22T13:24:06.411567+0000","last_became_active":"2024-04-22T13:22:05.243397+0000","last_became_peered":"2024-04-22T13:22:05.243397+0000","last_unstale":"2024-04-22T13:24:06.411567+0000","last_undegraded":"2024-04-22T13:24:06.411567+0000","last_fullsized":"2024-04-22T13:24:06.411567+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:16:21.236339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162647+0000","last_change":"2024-04-22T13:22:05.246042+0000","last_active":"2024-04-22T13:24:06.162647+0000","last_peered":"2024-04-22T13:24:06.162647+0000","last_clean":"2024-04-22T13:24:06.162647+0000","last_became_active":"2024-04-22T13:22:05.245669+0000","last_became_peered":"2024-04-22T13:22:05.245669+0000","last_unstale":"2024-04-22T13:24:06.162647+0000","last_undegraded":"2024-04-22T13:24:06.162647+0000","last_fullsized":"2024-04-22T13:24:06.162647+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:49:05.539641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418321+0000","last_change":"2024-04-22T13:22:05.257009+0000","last_active":"2024-04-22T13:24:06.418321+0000","last_peered":"2024-04-22T13:24:06.418321+0000","last_clean":"2024-04-22T13:24:06.418321+0000","last_became_active":"2024-04-22T13:22:05.254327+0000","last_became_peered":"2024-04-22T13:22:05.254327+0000","last_unstale":"2024-04-22T13:24:06.418321+0000","last_undegraded":"2024-04-22T13:24:06.418321+0000","last_fullsized":"2024-04-22T13:24:06.418321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:29:12.482525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.415339+0000","last_change":"2024-04-22T13:22:05.260575+0000","last_active":"2024-04-22T13:24:06.415339+0000","last_peered":"2024-04-22T13:24:06.415339+0000","last_clean":"2024-04-22T13:24:06.415339+0000","last_became_active":"2024-04-22T13:22:05.260181+0000","last_became_peered":"2024-04-22T13:22:05.260181+0000","last_unstale":"2024-04-22T13:24:06.415339+0000","last_undegraded":"2024-04-22T13:24:06.415339+0000","last_fullsized":"2024-04-22T13:24:06.415339+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:49:20.064368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259116+0000","last_change":"2024-04-22T13:22:05.248199+0000","last_active":"2024-04-22T13:24:06.259116+0000","last_peered":"2024-04-22T13:24:06.259116+0000","last_clean":"2024-04-22T13:24:06.259116+0000","last_became_active":"2024-04-22T13:22:05.247858+0000","last_became_peered":"2024-04-22T13:22:05.247858+0000","last_unstale":"2024-04-22T13:24:06.259116+0000","last_undegraded":"2024-04-22T13:24:06.259116+0000","last_fullsized":"2024-04-22T13:24:06.259116+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:49:21.228105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164341+0000","last_change":"2024-04-22T13:22:05.239137+0000","last_active":"2024-04-22T13:24:06.164341+0000","last_peered":"2024-04-22T13:24:06.164341+0000","last_clean":"2024-04-22T13:24:06.164341+0000","last_became_active":"2024-04-22T13:22:05.238807+0000","last_became_peered":"2024-04-22T13:22:05.238807+0000","last_unstale":"2024-04-22T13:24:06.164341+0000","last_undegraded":"2024-04-22T13:24:06.164341+0000","last_fullsized":"2024-04-22T13:24:06.164341+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:02:48.587097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258756+0000","last_change":"2024-04-22T13:22:05.249080+0000","last_active":"2024-04-22T13:24:06.258756+0000","last_peered":"2024-04-22T13:24:06.258756+0000","last_clean":"2024-04-22T13:24:06.258756+0000","last_became_active":"2024-04-22T13:22:05.248733+0000","last_became_peered":"2024-04-22T13:22:05.248733+0000","last_unstale":"2024-04-22T13:24:06.258756+0000","last_undegraded":"2024-04-22T13:24:06.258756+0000","last_fullsized":"2024-04-22T13:24:06.258756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:58.735941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.163779+0000","last_change":"2024-04-22T13:22:05.255490+0000","last_active":"2024-04-22T13:24:06.163779+0000","last_peered":"2024-04-22T13:24:06.163779+0000","last_clean":"2024-04-22T13:24:06.163779+0000","last_became_active":"2024-04-22T13:22:05.253644+0000","last_became_peered":"2024-04-22T13:22:05.253644+0000","last_unstale":"2024-04-22T13:24:06.163779+0000","last_undegraded":"2024-04-22T13:24:06.163779+0000","last_fullsized":"2024-04-22T13:24:06.163779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:31:01.816182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162443+0000","last_change":"2024-04-22T13:22:05.240988+0000","last_active":"2024-04-22T13:24:06.162443+0000","last_peered":"2024-04-22T13:24:06.162443+0000","last_clean":"2024-04-22T13:24:06.162443+0000","last_became_active":"2024-04-22T13:22:05.240642+0000","last_became_peered":"2024-04-22T13:22:05.240642+0000","last_unstale":"2024-04-22T13:24:06.162443+0000","last_undegraded":"2024-04-22T13:24:06.162443+0000","last_fullsized":"2024-04-22T13:24:06.162443+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:41:29.001979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418222+0000","last_change":"2024-04-22T13:22:05.252212+0000","last_active":"2024-04-22T13:24:06.418222+0000","last_peered":"2024-04-22T13:24:06.418222+0000","last_clean":"2024-04-22T13:24:06.418222+0000","last_became_active":"2024-04-22T13:22:05.251852+0000","last_became_peered":"2024-04-22T13:22:05.251852+0000","last_unstale":"2024-04-22T13:24:06.418222+0000","last_undegraded":"2024-04-22T13:24:06.418222+0000","last_fullsized":"2024-04-22T13:24:06.418222+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:13:55.132501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602844,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1355,"num_read_kb":796,"num_write":377,"num_write_kb":1471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1808532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":305,"ondisk_log_size":305,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318311,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":473150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579946,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28800,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716352,"statfs":{"total":95995035648,"available":95965544448,"internally_reserved":0,"allocated":1826816,"data_stored":1263641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743472,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28388,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716764,"statfs":{"total":95995035648,"available":95965966336,"internally_reserved":0,"allocated":1400832,"data_stored":865206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005109,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28384,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716768,"statfs":{"total":95995035648,"available":95965970432,"internally_reserved":0,"allocated":1396736,"data_stored":865584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234042,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36900,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708252,"statfs":{"total":95995035648,"available":95957250048,"internally_reserved":0,"allocated":1601536,"data_stored":1067373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430271,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":991232,"data_stored":472489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":541040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":474136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1081344,"data_stored":542350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117071,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36836,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708316,"statfs":{"total":95995035648,"available":95957315584,"internally_reserved":0,"allocated":1536000,"data_stored":996348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855434,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":991232,"data_stored":472882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691908,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28724,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716428,"statfs":{"total":95995035648,"available":95965622272,"internally_reserved":0,"allocated":1748992,"data_stored":1196555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":202288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T13:24:18.145 INFO:tasks.quiescer.ceph_manager:clean! 2024-04-22T13:24:18.145 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T13:24:18.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:17 smithi134 ceph-mon[31338]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1 KiB/s wr, 20 op/s 2024-04-22T13:24:18.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:17 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/3844291435' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:24:18.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:17 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2665104165' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:24:18.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:17 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/19074878' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:24:18.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.286+0000 7fe79477e640 1 -- 172.21.15.12:0/242412370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe78c106180 msgr2=0x7fe78c10dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:18.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.286+0000 7fe79477e640 1 --2- 172.21.15.12:0/242412370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe78c106180 0x7fe78c10dd50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe784009ca0 tx=0x7fe78402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.286+0000 7fe79477e640 1 -- 172.21.15.12:0/242412370 shutdown_connections 2024-04-22T13:24:18.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.286+0000 7fe79477e640 1 --2- 172.21.15.12:0/242412370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe78c106180 0x7fe78c10dd50 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.286+0000 7fe79477e640 1 --2- 172.21.15.12:0/242412370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe78c101a60 0x7fe78c105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.286+0000 7fe79477e640 1 --2- 172.21.15.12:0/242412370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe78c1010b0 0x7fe78c101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.287+0000 7fe79477e640 1 -- 172.21.15.12:0/242412370 >> 172.21.15.12:0/242412370 conn(0x7fe78c0771d0 msgr2=0x7fe78c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:18.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.287+0000 7fe79477e640 1 -- 172.21.15.12:0/242412370 shutdown_connections 2024-04-22T13:24:18.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.287+0000 7fe79477e640 1 -- 172.21.15.12:0/242412370 wait complete. 2024-04-22T13:24:18.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.287+0000 7fe79477e640 1 Processor -- start 2024-04-22T13:24:18.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.287+0000 7fe79477e640 1 -- start start 2024-04-22T13:24:18.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.287+0000 7fe79477e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe78c1010b0 0x7fe78c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:18.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.287+0000 7fe79477e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe78c106180 0x7fe78c198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:18.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.287+0000 7fe79477e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe78c19cfb0 0x7fe78c19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:18.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.287+0000 7fe79477e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c112200 con 0x7fe78c19cfb0 2024-04-22T13:24:18.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.287+0000 7fe79477e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c112080 con 0x7fe78c1010b0 2024-04-22T13:24:18.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.287+0000 7fe79477e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe78c112380 con 0x7fe78c106180 2024-04-22T13:24:18.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.288+0000 7fe792d1b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe78c19cfb0 0x7fe78c19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.288+0000 7fe792d1b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe78c19cfb0 0x7fe78c19d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59534/0 (socket says 172.21.15.12:59534) 2024-04-22T13:24:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.288+0000 7fe792d1b640 1 -- 172.21.15.12:0/2394535361 learned_addr learned my addr 172.21.15.12:0/2394535361 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.288+0000 7fe79251a640 1 --2- 172.21.15.12:0/2394535361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe78c1010b0 0x7fe78c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.288+0000 7fe791d19640 1 --2- 172.21.15.12:0/2394535361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe78c106180 0x7fe78c198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.288+0000 7fe792d1b640 1 -- 172.21.15.12:0/2394535361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe78c106180 msgr2=0x7fe78c198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.288+0000 7fe792d1b640 1 --2- 172.21.15.12:0/2394535361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe78c106180 0x7fe78c198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.288+0000 7fe792d1b640 1 -- 172.21.15.12:0/2394535361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe78c1010b0 msgr2=0x7fe78c198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.288+0000 7fe792d1b640 1 --2- 172.21.15.12:0/2394535361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe78c1010b0 0x7fe78c198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.288+0000 7fe792d1b640 1 -- 172.21.15.12:0/2394535361 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe78c19daa0 con 0x7fe78c19cfb0 2024-04-22T13:24:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.288+0000 7fe79251a640 1 --2- 172.21.15.12:0/2394535361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe78c1010b0 0x7fe78c198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.288+0000 7fe792d1b640 1 --2- 172.21.15.12:0/2394535361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe78c19cfb0 0x7fe78c19d460 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe784005e30 tx=0x7fe784034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.288+0000 7fe7837fe640 1 -- 172.21.15.12:0/2394535361 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe784041020 con 0x7fe78c19cfb0 2024-04-22T13:24:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.288+0000 7fe7837fe640 1 -- 172.21.15.12:0/2394535361 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe784032970 con 0x7fe78c19cfb0 2024-04-22T13:24:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.288+0000 7fe79477e640 1 -- 172.21.15.12:0/2394535361 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe78c19dd30 con 0x7fe78c19cfb0 2024-04-22T13:24:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.288+0000 7fe7837fe640 1 -- 172.21.15.12:0/2394535361 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe784039040 con 0x7fe78c19cfb0 2024-04-22T13:24:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.288+0000 7fe79477e640 1 -- 172.21.15.12:0/2394535361 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe78c1a5550 con 0x7fe78c19cfb0 2024-04-22T13:24:18.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.289+0000 7fe79477e640 1 -- 172.21.15.12:0/2394535361 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe78c069230 con 0x7fe78c19cfb0 2024-04-22T13:24:18.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.290+0000 7fe7837fe640 1 -- 172.21.15.12:0/2394535361 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fe784044020 con 0x7fe78c19cfb0 2024-04-22T13:24:18.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.291+0000 7fe7837fe640 1 --2- 172.21.15.12:0/2394535361 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe7780a3ec0 0x7fe7780a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:18.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.291+0000 7fe79251a640 1 --2- 172.21.15.12:0/2394535361 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe7780a3ec0 0x7fe7780a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:18.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.291+0000 7fe7837fe640 1 -- 172.21.15.12:0/2394535361 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fe7840f2730 con 0x7fe78c19cfb0 2024-04-22T13:24:18.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.291+0000 7fe79251a640 1 --2- 172.21.15.12:0/2394535361 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe7780a3ec0 0x7fe7780a6380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe7880058c0 tx=0x7fe788005850 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:18.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.296+0000 7fe7837fe640 1 -- 172.21.15.12:0/2394535361 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe78404b020 con 0x7fe78c19cfb0 2024-04-22T13:24:18.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.479+0000 7fe79477e640 1 -- 172.21.15.12:0/2394535361 --> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe78c101a60 con 0x7fe7780a3ec0 2024-04-22T13:24:18.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.486+0000 7fe7837fe640 1 -- 172.21.15.12:0/2394535361 <== mgr.24968 v2:172.21.15.12:6800/1615661596 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257809 (secure 0 0 0) 0x7fe78c101a60 con 0x7fe7780a3ec0 2024-04-22T13:24:18.486 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:24:18.487 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-22T13:24:18.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.489+0000 7fe79477e640 1 -- 172.21.15.12:0/2394535361 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe7780a3ec0 msgr2=0x7fe7780a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:18.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.489+0000 7fe79477e640 1 --2- 172.21.15.12:0/2394535361 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe7780a3ec0 0x7fe7780a6380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe7880058c0 tx=0x7fe788005850 comp rx=0 tx=0).stop 2024-04-22T13:24:18.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.489+0000 7fe79477e640 1 -- 172.21.15.12:0/2394535361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe78c19cfb0 msgr2=0x7fe78c19d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:18.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.489+0000 7fe79477e640 1 --2- 172.21.15.12:0/2394535361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe78c19cfb0 0x7fe78c19d460 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe784005e30 tx=0x7fe784034670 comp rx=0 tx=0).stop 2024-04-22T13:24:18.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.489+0000 7fe79477e640 1 -- 172.21.15.12:0/2394535361 shutdown_connections 2024-04-22T13:24:18.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.489+0000 7fe79477e640 1 --2- 172.21.15.12:0/2394535361 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe7780a3ec0 0x7fe7780a6380 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.489+0000 7fe79477e640 1 --2- 172.21.15.12:0/2394535361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe78c19cfb0 0x7fe78c19d460 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.489+0000 7fe79477e640 1 --2- 172.21.15.12:0/2394535361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe78c106180 0x7fe78c198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.489+0000 7fe79477e640 1 --2- 172.21.15.12:0/2394535361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe78c1010b0 0x7fe78c198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.489+0000 7fe79477e640 1 -- 172.21.15.12:0/2394535361 >> 172.21.15.12:0/2394535361 conn(0x7fe78c0771d0 msgr2=0x7fe78c0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:18.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.489+0000 7fe79477e640 1 -- 172.21.15.12:0/2394535361 shutdown_connections 2024-04-22T13:24:18.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.489+0000 7fe79477e640 1 -- 172.21.15.12:0/2394535361 wait complete. 2024-04-22T13:24:18.497 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-04-22T13:24:18.177845+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1390078,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1897,"num_read_kb":1383,"num_write":706,"num_write_kb":5325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1034,"ondisk_log_size":1034,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382008,"kb_used_data":15840,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124559816,"statfs":{"total":1151940427776,"available":1151549251584,"internally_reserved":0,"allocated":16220160,"data_stored":9820650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":200716,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":461,"num_read_kb":500,"num_write":155,"num_write_kb":923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.992821"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418512+0000","last_change":"2024-04-22T13:22:05.254388+0000","last_active":"2024-04-22T13:24:06.418512+0000","last_peered":"2024-04-22T13:24:06.418512+0000","last_clean":"2024-04-22T13:24:06.418512+0000","last_became_active":"2024-04-22T13:22:05.254042+0000","last_became_peered":"2024-04-22T13:22:05.254042+0000","last_unstale":"2024-04-22T13:24:06.418512+0000","last_undegraded":"2024-04-22T13:24:06.418512+0000","last_fullsized":"2024-04-22T13:24:06.418512+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:07:56.139653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.412069+0000","last_change":"2024-04-22T13:22:05.247879+0000","last_active":"2024-04-22T13:24:06.412069+0000","last_peered":"2024-04-22T13:24:06.412069+0000","last_clean":"2024-04-22T13:24:06.412069+0000","last_became_active":"2024-04-22T13:22:05.247508+0000","last_became_peered":"2024-04-22T13:22:05.247508+0000","last_unstale":"2024-04-22T13:24:06.412069+0000","last_undegraded":"2024-04-22T13:24:06.412069+0000","last_fullsized":"2024-04-22T13:24:06.412069+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:55:51.928658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411912+0000","last_change":"2024-04-22T13:22:05.257970+0000","last_active":"2024-04-22T13:24:06.411912+0000","last_peered":"2024-04-22T13:24:06.411912+0000","last_clean":"2024-04-22T13:24:06.411912+0000","last_became_active":"2024-04-22T13:22:05.257652+0000","last_became_peered":"2024-04-22T13:22:05.257652+0000","last_unstale":"2024-04-22T13:24:06.411912+0000","last_undegraded":"2024-04-22T13:24:06.411912+0000","last_fullsized":"2024-04-22T13:24:06.411912+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:42:17.476349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411332+0000","last_change":"2024-04-22T13:22:05.249186+0000","last_active":"2024-04-22T13:24:06.411332+0000","last_peered":"2024-04-22T13:24:06.411332+0000","last_clean":"2024-04-22T13:24:06.411332+0000","last_became_active":"2024-04-22T13:22:05.248814+0000","last_became_peered":"2024-04-22T13:22:05.248814+0000","last_unstale":"2024-04-22T13:24:06.411332+0000","last_undegraded":"2024-04-22T13:24:06.411332+0000","last_fullsized":"2024-04-22T13:24:06.411332+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:57.450794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258738+0000","last_change":"2024-04-22T13:22:05.251953+0000","last_active":"2024-04-22T13:24:06.258738+0000","last_peered":"2024-04-22T13:24:06.258738+0000","last_clean":"2024-04-22T13:24:06.258738+0000","last_became_active":"2024-04-22T13:22:05.251547+0000","last_became_peered":"2024-04-22T13:22:05.251547+0000","last_unstale":"2024-04-22T13:24:06.258738+0000","last_undegraded":"2024-04-22T13:24:06.258738+0000","last_fullsized":"2024-04-22T13:24:06.258738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:36:34.809878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.266346+0000","last_change":"2024-04-22T13:23:05.510044+0000","last_active":"2024-04-22T13:24:06.266346+0000","last_peered":"2024-04-22T13:24:06.266346+0000","last_clean":"2024-04-22T13:24:06.266346+0000","last_became_active":"2024-04-22T13:23:05.509677+0000","last_became_peered":"2024-04-22T13:23:05.509677+0000","last_unstale":"2024-04-22T13:24:06.266346+0000","last_undegraded":"2024-04-22T13:24:06.266346+0000","last_fullsized":"2024-04-22T13:24:06.266346+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:59:46.979291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409036+0000","last_change":"2024-04-22T13:22:05.250418+0000","last_active":"2024-04-22T13:24:06.409036+0000","last_peered":"2024-04-22T13:24:06.409036+0000","last_clean":"2024-04-22T13:24:06.409036+0000","last_became_active":"2024-04-22T13:22:05.250097+0000","last_became_peered":"2024-04-22T13:22:05.250097+0000","last_unstale":"2024-04-22T13:24:06.409036+0000","last_undegraded":"2024-04-22T13:24:06.409036+0000","last_fullsized":"2024-04-22T13:24:06.409036+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:34:59.598950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258759+0000","last_change":"2024-04-22T13:22:05.244762+0000","last_active":"2024-04-22T13:24:06.258759+0000","last_peered":"2024-04-22T13:24:06.258759+0000","last_clean":"2024-04-22T13:24:06.258759+0000","last_became_active":"2024-04-22T13:22:05.244430+0000","last_became_peered":"2024-04-22T13:22:05.244430+0000","last_unstale":"2024-04-22T13:24:06.258759+0000","last_undegraded":"2024-04-22T13:24:06.258759+0000","last_fullsized":"2024-04-22T13:24:06.258759+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:33:01.582851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164321+0000","last_change":"2024-04-22T13:22:05.257596+0000","last_active":"2024-04-22T13:24:06.164321+0000","last_peered":"2024-04-22T13:24:06.164321+0000","last_clean":"2024-04-22T13:24:06.164321+0000","last_became_active":"2024-04-22T13:22:05.257277+0000","last_became_peered":"2024-04-22T13:22:05.257277+0000","last_unstale":"2024-04-22T13:24:06.164321+0000","last_undegraded":"2024-04-22T13:24:06.164321+0000","last_fullsized":"2024-04-22T13:24:06.164321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:09:44.715965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.266708+0000","last_change":"2024-04-22T13:22:05.252177+0000","last_active":"2024-04-22T13:24:06.266708+0000","last_peered":"2024-04-22T13:24:06.266708+0000","last_clean":"2024-04-22T13:24:06.266708+0000","last_became_active":"2024-04-22T13:22:05.251794+0000","last_became_peered":"2024-04-22T13:22:05.251794+0000","last_unstale":"2024-04-22T13:24:06.266708+0000","last_undegraded":"2024-04-22T13:24:06.266708+0000","last_fullsized":"2024-04-22T13:24:06.266708+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:08:59.500540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409476+0000","last_change":"2024-04-22T13:22:05.256209+0000","last_active":"2024-04-22T13:24:06.409476+0000","last_peered":"2024-04-22T13:24:06.409476+0000","last_clean":"2024-04-22T13:24:06.409476+0000","last_became_active":"2024-04-22T13:22:05.255504+0000","last_became_peered":"2024-04-22T13:22:05.255504+0000","last_unstale":"2024-04-22T13:24:06.409476+0000","last_undegraded":"2024-04-22T13:24:06.409476+0000","last_fullsized":"2024-04-22T13:24:06.409476+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:32:08.190921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.256744+0000","last_change":"2024-04-22T13:23:05.509837+0000","last_active":"2024-04-22T13:24:06.256744+0000","last_peered":"2024-04-22T13:24:06.256744+0000","last_clean":"2024-04-22T13:24:06.256744+0000","last_became_active":"2024-04-22T13:23:05.509421+0000","last_became_peered":"2024-04-22T13:23:05.509421+0000","last_unstale":"2024-04-22T13:24:06.256744+0000","last_undegraded":"2024-04-22T13:24:06.256744+0000","last_fullsized":"2024-04-22T13:24:06.256744+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:30:01.103373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162449+0000","last_change":"2024-04-22T13:22:05.246667+0000","last_active":"2024-04-22T13:24:06.162449+0000","last_peered":"2024-04-22T13:24:06.162449+0000","last_clean":"2024-04-22T13:24:06.162449+0000","last_became_active":"2024-04-22T13:22:05.246365+0000","last_became_peered":"2024-04-22T13:22:05.246365+0000","last_unstale":"2024-04-22T13:24:06.162449+0000","last_undegraded":"2024-04-22T13:24:06.162449+0000","last_fullsized":"2024-04-22T13:24:06.162449+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:41:55.533430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411527+0000","last_change":"2024-04-22T13:22:05.251982+0000","last_active":"2024-04-22T13:24:06.411527+0000","last_peered":"2024-04-22T13:24:06.411527+0000","last_clean":"2024-04-22T13:24:06.411527+0000","last_became_active":"2024-04-22T13:22:05.251575+0000","last_became_peered":"2024-04-22T13:22:05.251575+0000","last_unstale":"2024-04-22T13:24:06.411527+0000","last_undegraded":"2024-04-22T13:24:06.411527+0000","last_fullsized":"2024-04-22T13:24:06.411527+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:28:05.343140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411427+0000","last_change":"2024-04-22T13:22:05.251056+0000","last_active":"2024-04-22T13:24:06.411427+0000","last_peered":"2024-04-22T13:24:06.411427+0000","last_clean":"2024-04-22T13:24:06.411427+0000","last_became_active":"2024-04-22T13:22:05.250652+0000","last_became_peered":"2024-04-22T13:22:05.250652+0000","last_unstale":"2024-04-22T13:24:06.411427+0000","last_undegraded":"2024-04-22T13:24:06.411427+0000","last_fullsized":"2024-04-22T13:24:06.411427+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:08:28.629965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.163636+0000","last_change":"2024-04-22T13:22:05.256877+0000","last_active":"2024-04-22T13:24:06.163636+0000","last_peered":"2024-04-22T13:24:06.163636+0000","last_clean":"2024-04-22T13:24:06.163636+0000","last_became_active":"2024-04-22T13:22:05.256074+0000","last_became_peered":"2024-04-22T13:22:05.256074+0000","last_unstale":"2024-04-22T13:24:06.163636+0000","last_undegraded":"2024-04-22T13:24:06.163636+0000","last_fullsized":"2024-04-22T13:24:06.163636+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:08:53.607626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418592+0000","last_change":"2024-04-22T13:22:05.255598+0000","last_active":"2024-04-22T13:24:06.418592+0000","last_peered":"2024-04-22T13:24:06.418592+0000","last_clean":"2024-04-22T13:24:06.418592+0000","last_became_active":"2024-04-22T13:22:05.255239+0000","last_became_peered":"2024-04-22T13:22:05.255239+0000","last_unstale":"2024-04-22T13:24:06.418592+0000","last_undegraded":"2024-04-22T13:24:06.418592+0000","last_fullsized":"2024-04-22T13:24:06.418592+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:38:20.547882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.257085+0000","last_change":"2024-04-22T13:22:05.254232+0000","last_active":"2024-04-22T13:24:06.257085+0000","last_peered":"2024-04-22T13:24:06.257085+0000","last_clean":"2024-04-22T13:24:06.257085+0000","last_became_active":"2024-04-22T13:22:05.253907+0000","last_became_peered":"2024-04-22T13:22:05.253907+0000","last_unstale":"2024-04-22T13:24:06.257085+0000","last_undegraded":"2024-04-22T13:24:06.257085+0000","last_fullsized":"2024-04-22T13:24:06.257085+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:15:49.931112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.415196+0000","last_change":"2024-04-22T13:22:05.251518+0000","last_active":"2024-04-22T13:24:06.415196+0000","last_peered":"2024-04-22T13:24:06.415196+0000","last_clean":"2024-04-22T13:24:06.415196+0000","last_became_active":"2024-04-22T13:22:05.250478+0000","last_became_peered":"2024-04-22T13:22:05.250478+0000","last_unstale":"2024-04-22T13:24:06.415196+0000","last_undegraded":"2024-04-22T13:24:06.415196+0000","last_fullsized":"2024-04-22T13:24:06.415196+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:25:26.758692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'14","reported_seq":163,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:15.571730+0000","last_change":"2024-04-22T13:22:04.241365+0000","last_active":"2024-04-22T13:24:15.571730+0000","last_peered":"2024-04-22T13:24:15.571730+0000","last_clean":"2024-04-22T13:24:15.571730+0000","last_became_active":"2024-04-22T13:22:04.240822+0000","last_became_peered":"2024-04-22T13:22:04.240822+0000","last_unstale":"2024-04-22T13:24:15.571730+0000","last_undegraded":"2024-04-22T13:24:15.571730+0000","last_fullsized":"2024-04-22T13:24:15.571730+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:20:23.928534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":127,"num_read_kb":127,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.256956+0000","last_change":"2024-04-22T13:23:05.876262+0000","last_active":"2024-04-22T13:24:06.256956+0000","last_peered":"2024-04-22T13:24:06.256956+0000","last_clean":"2024-04-22T13:24:06.256956+0000","last_became_active":"2024-04-22T13:23:05.875872+0000","last_became_peered":"2024-04-22T13:23:05.875872+0000","last_unstale":"2024-04-22T13:24:06.256956+0000","last_undegraded":"2024-04-22T13:24:06.256956+0000","last_fullsized":"2024-04-22T13:24:06.256956+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:29:33.092887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259532+0000","last_change":"2024-04-22T13:22:04.248872+0000","last_active":"2024-04-22T13:24:06.259532+0000","last_peered":"2024-04-22T13:24:06.259532+0000","last_clean":"2024-04-22T13:24:06.259532+0000","last_became_active":"2024-04-22T13:22:04.248466+0000","last_became_peered":"2024-04-22T13:22:04.248466+0000","last_unstale":"2024-04-22T13:24:06.259532+0000","last_undegraded":"2024-04-22T13:24:06.259532+0000","last_fullsized":"2024-04-22T13:24:06.259532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:38:10.871602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.412438+0000","last_change":"2024-04-22T13:22:05.258005+0000","last_active":"2024-04-22T13:24:06.412438+0000","last_peered":"2024-04-22T13:24:06.412438+0000","last_clean":"2024-04-22T13:24:06.412438+0000","last_became_active":"2024-04-22T13:22:05.257615+0000","last_became_peered":"2024-04-22T13:22:05.257615+0000","last_unstale":"2024-04-22T13:24:06.412438+0000","last_undegraded":"2024-04-22T13:24:06.412438+0000","last_fullsized":"2024-04-22T13:24:06.412438+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:21:18.030530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259060+0000","last_change":"2024-04-22T13:22:04.246273+0000","last_active":"2024-04-22T13:24:06.259060+0000","last_peered":"2024-04-22T13:24:06.259060+0000","last_clean":"2024-04-22T13:24:06.259060+0000","last_became_active":"2024-04-22T13:22:04.245969+0000","last_became_peered":"2024-04-22T13:22:04.245969+0000","last_unstale":"2024-04-22T13:24:06.259060+0000","last_undegraded":"2024-04-22T13:24:06.259060+0000","last_fullsized":"2024-04-22T13:24:06.259060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:32:10.260192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.163964+0000","last_change":"2024-04-22T13:22:05.260299+0000","last_active":"2024-04-22T13:24:06.163964+0000","last_peered":"2024-04-22T13:24:06.163964+0000","last_clean":"2024-04-22T13:24:06.163964+0000","last_became_active":"2024-04-22T13:22:05.259634+0000","last_became_peered":"2024-04-22T13:22:05.259634+0000","last_unstale":"2024-04-22T13:24:06.163964+0000","last_undegraded":"2024-04-22T13:24:06.163964+0000","last_fullsized":"2024-04-22T13:24:06.163964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:16:44.421652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258970+0000","last_change":"2024-04-22T13:22:04.243843+0000","last_active":"2024-04-22T13:24:06.258970+0000","last_peered":"2024-04-22T13:24:06.258970+0000","last_clean":"2024-04-22T13:24:06.258970+0000","last_became_active":"2024-04-22T13:22:04.243465+0000","last_became_peered":"2024-04-22T13:22:04.243465+0000","last_unstale":"2024-04-22T13:24:06.258970+0000","last_undegraded":"2024-04-22T13:24:06.258970+0000","last_fullsized":"2024-04-22T13:24:06.258970+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:48:36.431697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164717+0000","last_change":"2024-04-22T13:22:05.259789+0000","last_active":"2024-04-22T13:24:06.164717+0000","last_peered":"2024-04-22T13:24:06.164717+0000","last_clean":"2024-04-22T13:24:06.164717+0000","last_became_active":"2024-04-22T13:22:05.259395+0000","last_became_peered":"2024-04-22T13:22:05.259395+0000","last_unstale":"2024-04-22T13:24:06.164717+0000","last_undegraded":"2024-04-22T13:24:06.164717+0000","last_fullsized":"2024-04-22T13:24:06.164717+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:43:51.430561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'7","reported_seq":160,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:15.547006+0000","last_change":"2024-04-22T13:22:04.240615+0000","last_active":"2024-04-22T13:24:15.547006+0000","last_peered":"2024-04-22T13:24:15.547006+0000","last_clean":"2024-04-22T13:24:15.547006+0000","last_became_active":"2024-04-22T13:22:04.240248+0000","last_became_peered":"2024-04-22T13:22:04.240248+0000","last_unstale":"2024-04-22T13:24:15.547006+0000","last_undegraded":"2024-04-22T13:24:15.547006+0000","last_fullsized":"2024-04-22T13:24:15.547006+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:55:05.948979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1404,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":132,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258665+0000","last_change":"2024-04-22T13:22:05.253788+0000","last_active":"2024-04-22T13:24:06.258665+0000","last_peered":"2024-04-22T13:24:06.258665+0000","last_clean":"2024-04-22T13:24:06.258665+0000","last_became_active":"2024-04-22T13:22:05.252578+0000","last_became_peered":"2024-04-22T13:22:05.252578+0000","last_unstale":"2024-04-22T13:24:06.258665+0000","last_undegraded":"2024-04-22T13:24:06.258665+0000","last_fullsized":"2024-04-22T13:24:06.258665+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:04:11.769930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.266274+0000","last_change":"2024-04-22T13:22:04.241170+0000","last_active":"2024-04-22T13:24:06.266274+0000","last_peered":"2024-04-22T13:24:06.266274+0000","last_clean":"2024-04-22T13:24:06.266274+0000","last_became_active":"2024-04-22T13:22:04.240830+0000","last_became_peered":"2024-04-22T13:22:04.240830+0000","last_unstale":"2024-04-22T13:24:06.266274+0000","last_undegraded":"2024-04-22T13:24:06.266274+0000","last_fullsized":"2024-04-22T13:24:06.266274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:51:30.520613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164733+0000","last_change":"2024-04-22T13:22:05.237900+0000","last_active":"2024-04-22T13:24:06.164733+0000","last_peered":"2024-04-22T13:24:06.164733+0000","last_clean":"2024-04-22T13:24:06.164733+0000","last_became_active":"2024-04-22T13:22:05.237507+0000","last_became_peered":"2024-04-22T13:22:05.237507+0000","last_unstale":"2024-04-22T13:24:06.164733+0000","last_undegraded":"2024-04-22T13:24:06.164733+0000","last_fullsized":"2024-04-22T13:24:06.164733+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:26:24.307803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162328+0000","last_change":"2024-04-22T13:22:04.236078+0000","last_active":"2024-04-22T13:24:06.162328+0000","last_peered":"2024-04-22T13:24:06.162328+0000","last_clean":"2024-04-22T13:24:06.162328+0000","last_became_active":"2024-04-22T13:22:04.235710+0000","last_became_peered":"2024-04-22T13:22:04.235710+0000","last_unstale":"2024-04-22T13:24:06.162328+0000","last_undegraded":"2024-04-22T13:24:06.162328+0000","last_fullsized":"2024-04-22T13:24:06.162328+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:37:55.335237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258717+0000","last_change":"2024-04-22T13:22:05.243152+0000","last_active":"2024-04-22T13:24:06.258717+0000","last_peered":"2024-04-22T13:24:06.258717+0000","last_clean":"2024-04-22T13:24:06.258717+0000","last_became_active":"2024-04-22T13:22:05.242772+0000","last_became_peered":"2024-04-22T13:22:05.242772+0000","last_unstale":"2024-04-22T13:24:06.258717+0000","last_undegraded":"2024-04-22T13:24:06.258717+0000","last_fullsized":"2024-04-22T13:24:06.258717+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:33:42.359679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164258+0000","last_change":"2024-04-22T13:22:04.244381+0000","last_active":"2024-04-22T13:24:06.164258+0000","last_peered":"2024-04-22T13:24:06.164258+0000","last_clean":"2024-04-22T13:24:06.164258+0000","last_became_active":"2024-04-22T13:22:04.243746+0000","last_became_peered":"2024-04-22T13:22:04.243746+0000","last_unstale":"2024-04-22T13:24:06.164258+0000","last_undegraded":"2024-04-22T13:24:06.164258+0000","last_fullsized":"2024-04-22T13:24:06.164258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:15:36.019930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.256753+0000","last_change":"2024-04-22T13:22:05.245394+0000","last_active":"2024-04-22T13:24:06.256753+0000","last_peered":"2024-04-22T13:24:06.256753+0000","last_clean":"2024-04-22T13:24:06.256753+0000","last_became_active":"2024-04-22T13:22:05.243931+0000","last_became_peered":"2024-04-22T13:22:05.243931+0000","last_unstale":"2024-04-22T13:24:06.256753+0000","last_undegraded":"2024-04-22T13:24:06.256753+0000","last_fullsized":"2024-04-22T13:24:06.256753+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:11:10.790842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'6","reported_seq":280,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:16.547961+0000","last_change":"2024-04-22T13:22:04.242583+0000","last_active":"2024-04-22T13:24:16.547961+0000","last_peered":"2024-04-22T13:24:16.547961+0000","last_clean":"2024-04-22T13:24:16.547961+0000","last_became_active":"2024-04-22T13:22:04.242261+0000","last_became_peered":"2024-04-22T13:22:04.242261+0000","last_unstale":"2024-04-22T13:24:16.547961+0000","last_undegraded":"2024-04-22T13:24:16.547961+0000","last_fullsized":"2024-04-22T13:24:16.547961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:37:47.618573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":253,"num_read_kb":253,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164258+0000","last_change":"2024-04-22T13:22:05.255293+0000","last_active":"2024-04-22T13:24:06.164258+0000","last_peered":"2024-04-22T13:24:06.164258+0000","last_clean":"2024-04-22T13:24:06.164258+0000","last_became_active":"2024-04-22T13:22:05.253949+0000","last_became_peered":"2024-04-22T13:22:05.253949+0000","last_unstale":"2024-04-22T13:24:06.164258+0000","last_undegraded":"2024-04-22T13:24:06.164258+0000","last_fullsized":"2024-04-22T13:24:06.164258+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:41:08.754989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409503+0000","last_change":"2024-04-22T13:22:04.241529+0000","last_active":"2024-04-22T13:24:06.409503+0000","last_peered":"2024-04-22T13:24:06.409503+0000","last_clean":"2024-04-22T13:24:06.409503+0000","last_became_active":"2024-04-22T13:22:04.241164+0000","last_became_peered":"2024-04-22T13:22:04.241164+0000","last_unstale":"2024-04-22T13:24:06.409503+0000","last_undegraded":"2024-04-22T13:24:06.409503+0000","last_fullsized":"2024-04-22T13:24:06.409503+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:16:49.387181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.412225+0000","last_change":"2024-04-22T13:22:05.241623+0000","last_active":"2024-04-22T13:24:06.412225+0000","last_peered":"2024-04-22T13:24:06.412225+0000","last_clean":"2024-04-22T13:24:06.412225+0000","last_became_active":"2024-04-22T13:22:05.241150+0000","last_became_peered":"2024-04-22T13:22:05.241150+0000","last_unstale":"2024-04-22T13:24:06.412225+0000","last_undegraded":"2024-04-22T13:24:06.412225+0000","last_fullsized":"2024-04-22T13:24:06.412225+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:17:28.412366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164893+0000","last_change":"2024-04-22T13:22:04.247852+0000","last_active":"2024-04-22T13:24:06.164893+0000","last_peered":"2024-04-22T13:24:06.164893+0000","last_clean":"2024-04-22T13:24:06.164893+0000","last_became_active":"2024-04-22T13:22:04.247550+0000","last_became_peered":"2024-04-22T13:22:04.247550+0000","last_unstale":"2024-04-22T13:24:06.164893+0000","last_undegraded":"2024-04-22T13:24:06.164893+0000","last_fullsized":"2024-04-22T13:24:06.164893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:55:01.928435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164057+0000","last_change":"2024-04-22T13:22:05.260050+0000","last_active":"2024-04-22T13:24:06.164057+0000","last_peered":"2024-04-22T13:24:06.164057+0000","last_clean":"2024-04-22T13:24:06.164057+0000","last_became_active":"2024-04-22T13:22:05.259098+0000","last_became_peered":"2024-04-22T13:22:05.259098+0000","last_unstale":"2024-04-22T13:24:06.164057+0000","last_undegraded":"2024-04-22T13:24:06.164057+0000","last_fullsized":"2024-04-22T13:24:06.164057+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:05:06.245492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'38","reported_seq":330,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:16.547905+0000","last_change":"2024-04-22T13:22:04.240233+0000","last_active":"2024-04-22T13:24:16.547905+0000","last_peered":"2024-04-22T13:24:16.547905+0000","last_clean":"2024-04-22T13:24:16.547905+0000","last_became_active":"2024-04-22T13:22:04.239890+0000","last_became_peered":"2024-04-22T13:22:04.239890+0000","last_unstale":"2024-04-22T13:24:16.547905+0000","last_undegraded":"2024-04-22T13:24:16.547905+0000","last_fullsized":"2024-04-22T13:24:16.547905+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:04:21.547422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67638,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":271,"num_read_kb":203,"num_write":40,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258900+0000","last_change":"2024-04-22T13:22:05.257756+0000","last_active":"2024-04-22T13:24:06.258900+0000","last_peered":"2024-04-22T13:24:06.258900+0000","last_clean":"2024-04-22T13:24:06.258900+0000","last_became_active":"2024-04-22T13:22:05.257442+0000","last_became_peered":"2024-04-22T13:22:05.257442+0000","last_unstale":"2024-04-22T13:24:06.258900+0000","last_undegraded":"2024-04-22T13:24:06.258900+0000","last_fullsized":"2024-04-22T13:24:06.258900+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:16:53.465689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.414923+0000","last_change":"2024-04-22T13:22:04.240815+0000","last_active":"2024-04-22T13:24:06.414923+0000","last_peered":"2024-04-22T13:24:06.414923+0000","last_clean":"2024-04-22T13:24:06.414923+0000","last_became_active":"2024-04-22T13:22:04.240218+0000","last_became_peered":"2024-04-22T13:22:04.240218+0000","last_unstale":"2024-04-22T13:24:06.414923+0000","last_undegraded":"2024-04-22T13:24:06.414923+0000","last_fullsized":"2024-04-22T13:24:06.414923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:29:29.148764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162257+0000","last_change":"2024-04-22T13:22:05.248797+0000","last_active":"2024-04-22T13:24:06.162257+0000","last_peered":"2024-04-22T13:24:06.162257+0000","last_clean":"2024-04-22T13:24:06.162257+0000","last_became_active":"2024-04-22T13:22:05.248435+0000","last_became_peered":"2024-04-22T13:22:05.248435+0000","last_unstale":"2024-04-22T13:24:06.162257+0000","last_undegraded":"2024-04-22T13:24:06.162257+0000","last_fullsized":"2024-04-22T13:24:06.162257+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:32:31.970494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164389+0000","last_change":"2024-04-22T13:22:04.245303+0000","last_active":"2024-04-22T13:24:06.164389+0000","last_peered":"2024-04-22T13:24:06.164389+0000","last_clean":"2024-04-22T13:24:06.164389+0000","last_became_active":"2024-04-22T13:22:04.244903+0000","last_became_peered":"2024-04-22T13:22:04.244903+0000","last_unstale":"2024-04-22T13:24:06.164389+0000","last_undegraded":"2024-04-22T13:24:06.164389+0000","last_fullsized":"2024-04-22T13:24:06.164389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T13:34:32.174858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409264+0000","last_change":"2024-04-22T13:22:05.246318+0000","last_active":"2024-04-22T13:24:06.409264+0000","last_peered":"2024-04-22T13:24:06.409264+0000","last_clean":"2024-04-22T13:24:06.409264+0000","last_became_active":"2024-04-22T13:22:05.245899+0000","last_became_peered":"2024-04-22T13:22:05.245899+0000","last_unstale":"2024-04-22T13:24:06.409264+0000","last_undegraded":"2024-04-22T13:24:06.409264+0000","last_fullsized":"2024-04-22T13:24:06.409264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:27:42.789058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'9","reported_seq":160,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:15.606695+0000","last_change":"2024-04-22T13:22:04.240612+0000","last_active":"2024-04-22T13:24:15.606695+0000","last_peered":"2024-04-22T13:24:15.606695+0000","last_clean":"2024-04-22T13:24:15.606695+0000","last_became_active":"2024-04-22T13:22:04.240290+0000","last_became_peered":"2024-04-22T13:22:04.240290+0000","last_unstale":"2024-04-22T13:24:15.606695+0000","last_undegraded":"2024-04-22T13:24:15.606695+0000","last_fullsized":"2024-04-22T13:24:15.606695+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:00:50.285553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1902,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":131,"num_read_kb":6,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.408976+0000","last_change":"2024-04-22T13:22:05.253205+0000","last_active":"2024-04-22T13:24:06.408976+0000","last_peered":"2024-04-22T13:24:06.408976+0000","last_clean":"2024-04-22T13:24:06.408976+0000","last_became_active":"2024-04-22T13:22:05.252865+0000","last_became_peered":"2024-04-22T13:22:05.252865+0000","last_unstale":"2024-04-22T13:24:06.408976+0000","last_undegraded":"2024-04-22T13:24:06.408976+0000","last_fullsized":"2024-04-22T13:24:06.408976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:19:12.482371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258636+0000","last_change":"2024-04-22T13:22:04.248938+0000","last_active":"2024-04-22T13:24:06.258636+0000","last_peered":"2024-04-22T13:24:06.258636+0000","last_clean":"2024-04-22T13:24:06.258636+0000","last_became_active":"2024-04-22T13:22:04.248440+0000","last_became_peered":"2024-04-22T13:22:04.248440+0000","last_unstale":"2024-04-22T13:24:06.258636+0000","last_undegraded":"2024-04-22T13:24:06.258636+0000","last_fullsized":"2024-04-22T13:24:06.258636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:46:50.928067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164391+0000","last_change":"2024-04-22T13:22:05.254664+0000","last_active":"2024-04-22T13:24:06.164391+0000","last_peered":"2024-04-22T13:24:06.164391+0000","last_clean":"2024-04-22T13:24:06.164391+0000","last_became_active":"2024-04-22T13:22:05.254304+0000","last_became_peered":"2024-04-22T13:22:05.254304+0000","last_unstale":"2024-04-22T13:24:06.164391+0000","last_undegraded":"2024-04-22T13:24:06.164391+0000","last_fullsized":"2024-04-22T13:24:06.164391+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:58:49.345573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259033+0000","last_change":"2024-04-22T13:22:04.244128+0000","last_active":"2024-04-22T13:24:06.259033+0000","last_peered":"2024-04-22T13:24:06.259033+0000","last_clean":"2024-04-22T13:24:06.259033+0000","last_became_active":"2024-04-22T13:22:04.243738+0000","last_became_peered":"2024-04-22T13:22:04.243738+0000","last_unstale":"2024-04-22T13:24:06.259033+0000","last_undegraded":"2024-04-22T13:24:06.259033+0000","last_fullsized":"2024-04-22T13:24:06.259033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:23:52.220584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418387+0000","last_change":"2024-04-22T13:22:05.250910+0000","last_active":"2024-04-22T13:24:06.418387+0000","last_peered":"2024-04-22T13:24:06.418387+0000","last_clean":"2024-04-22T13:24:06.418387+0000","last_became_active":"2024-04-22T13:22:05.250410+0000","last_became_peered":"2024-04-22T13:22:05.250410+0000","last_unstale":"2024-04-22T13:24:06.418387+0000","last_undegraded":"2024-04-22T13:24:06.418387+0000","last_fullsized":"2024-04-22T13:24:06.418387+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:04:25.492677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.256811+0000","last_change":"2024-04-22T13:22:04.245889+0000","last_active":"2024-04-22T13:24:06.256811+0000","last_peered":"2024-04-22T13:24:06.256811+0000","last_clean":"2024-04-22T13:24:06.256811+0000","last_became_active":"2024-04-22T13:22:04.245474+0000","last_became_peered":"2024-04-22T13:22:04.245474+0000","last_unstale":"2024-04-22T13:24:06.256811+0000","last_undegraded":"2024-04-22T13:24:06.256811+0000","last_fullsized":"2024-04-22T13:24:06.256811+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:22:12.626950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.266676+0000","last_change":"2024-04-22T13:22:05.250978+0000","last_active":"2024-04-22T13:24:06.266676+0000","last_peered":"2024-04-22T13:24:06.266676+0000","last_clean":"2024-04-22T13:24:06.266676+0000","last_became_active":"2024-04-22T13:22:05.250463+0000","last_became_peered":"2024-04-22T13:22:05.250463+0000","last_unstale":"2024-04-22T13:24:06.266676+0000","last_undegraded":"2024-04-22T13:24:06.266676+0000","last_fullsized":"2024-04-22T13:24:06.266676+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:45:18.591072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'7","reported_seq":156,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:15.572397+0000","last_change":"2024-04-22T13:22:04.242503+0000","last_active":"2024-04-22T13:24:15.572397+0000","last_peered":"2024-04-22T13:24:15.572397+0000","last_clean":"2024-04-22T13:24:15.572397+0000","last_became_active":"2024-04-22T13:22:04.242143+0000","last_became_peered":"2024-04-22T13:22:04.242143+0000","last_unstale":"2024-04-22T13:24:15.572397+0000","last_undegraded":"2024-04-22T13:24:15.572397+0000","last_fullsized":"2024-04-22T13:24:15.572397+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:40.236377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":127,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.266578+0000","last_change":"2024-04-22T13:22:05.252387+0000","last_active":"2024-04-22T13:24:06.266578+0000","last_peered":"2024-04-22T13:24:06.266578+0000","last_clean":"2024-04-22T13:24:06.266578+0000","last_became_active":"2024-04-22T13:22:05.252033+0000","last_became_peered":"2024-04-22T13:22:05.252033+0000","last_unstale":"2024-04-22T13:24:06.266578+0000","last_undegraded":"2024-04-22T13:24:06.266578+0000","last_fullsized":"2024-04-22T13:24:06.266578+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:55:01.791283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164456+0000","last_change":"2024-04-22T13:22:05.248394+0000","last_active":"2024-04-22T13:24:06.164456+0000","last_peered":"2024-04-22T13:24:06.164456+0000","last_clean":"2024-04-22T13:24:06.164456+0000","last_became_active":"2024-04-22T13:22:05.248024+0000","last_became_peered":"2024-04-22T13:22:05.248024+0000","last_unstale":"2024-04-22T13:24:06.164456+0000","last_undegraded":"2024-04-22T13:24:06.164456+0000","last_fullsized":"2024-04-22T13:24:06.164456+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:49:36.556136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'7","reported_seq":155,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:15.605806+0000","last_change":"2024-04-22T13:22:04.247692+0000","last_active":"2024-04-22T13:24:15.605806+0000","last_peered":"2024-04-22T13:24:15.605806+0000","last_clean":"2024-04-22T13:24:15.605806+0000","last_became_active":"2024-04-22T13:22:04.247345+0000","last_became_peered":"2024-04-22T13:22:04.247345+0000","last_unstale":"2024-04-22T13:24:15.605806+0000","last_undegraded":"2024-04-22T13:24:15.605806+0000","last_fullsized":"2024-04-22T13:24:15.605806+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:07:49.803248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259343+0000","last_change":"2024-04-22T13:22:05.241022+0000","last_active":"2024-04-22T13:24:06.259343+0000","last_peered":"2024-04-22T13:24:06.259343+0000","last_clean":"2024-04-22T13:24:06.259343+0000","last_became_active":"2024-04-22T13:22:05.240635+0000","last_became_peered":"2024-04-22T13:22:05.240635+0000","last_unstale":"2024-04-22T13:24:06.259343+0000","last_undegraded":"2024-04-22T13:24:06.259343+0000","last_fullsized":"2024-04-22T13:24:06.259343+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:50:02.012260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411284+0000","last_change":"2024-04-22T13:22:04.236817+0000","last_active":"2024-04-22T13:24:06.411284+0000","last_peered":"2024-04-22T13:24:06.411284+0000","last_clean":"2024-04-22T13:24:06.411284+0000","last_became_active":"2024-04-22T13:22:04.236484+0000","last_became_peered":"2024-04-22T13:22:04.236484+0000","last_unstale":"2024-04-22T13:24:06.411284+0000","last_undegraded":"2024-04-22T13:24:06.411284+0000","last_fullsized":"2024-04-22T13:24:06.411284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:27:41.053327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"90'148","reported_seq":401,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:08.291461+0000","last_change":"2024-04-22T13:20:53.094123+0000","last_active":"2024-04-22T13:24:08.291461+0000","last_peered":"2024-04-22T13:24:08.291461+0000","last_clean":"2024-04-22T13:24:08.291461+0000","last_became_active":"2024-04-22T13:20:53.093664+0000","last_became_peered":"2024-04-22T13:20:53.093664+0000","last_unstale":"2024-04-22T13:24:08.291461+0000","last_undegraded":"2024-04-22T13:24:08.291461+0000","last_fullsized":"2024-04-22T13:24:08.291461+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:17:33.201786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:17:33.201786+0000","last_clean_scrub_stamp":"2024-04-22T13:17:33.201786+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:46:58.986399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162213+0000","last_change":"2024-04-22T13:22:05.247766+0000","last_active":"2024-04-22T13:24:06.162213+0000","last_peered":"2024-04-22T13:24:06.162213+0000","last_clean":"2024-04-22T13:24:06.162213+0000","last_became_active":"2024-04-22T13:22:05.247362+0000","last_became_peered":"2024-04-22T13:22:05.247362+0000","last_unstale":"2024-04-22T13:24:06.162213+0000","last_undegraded":"2024-04-22T13:24:06.162213+0000","last_fullsized":"2024-04-22T13:24:06.162213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:25:00.126638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409239+0000","last_change":"2024-04-22T13:22:04.243493+0000","last_active":"2024-04-22T13:24:06.409239+0000","last_peered":"2024-04-22T13:24:06.409239+0000","last_clean":"2024-04-22T13:24:06.409239+0000","last_became_active":"2024-04-22T13:22:04.243190+0000","last_became_peered":"2024-04-22T13:22:04.243190+0000","last_unstale":"2024-04-22T13:24:06.409239+0000","last_undegraded":"2024-04-22T13:24:06.409239+0000","last_fullsized":"2024-04-22T13:24:06.409239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:15:09.025152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258942+0000","last_change":"2024-04-22T13:22:04.249222+0000","last_active":"2024-04-22T13:24:06.258942+0000","last_peered":"2024-04-22T13:24:06.258942+0000","last_clean":"2024-04-22T13:24:06.258942+0000","last_became_active":"2024-04-22T13:22:04.248926+0000","last_became_peered":"2024-04-22T13:22:04.248926+0000","last_unstale":"2024-04-22T13:24:06.258942+0000","last_undegraded":"2024-04-22T13:24:06.258942+0000","last_fullsized":"2024-04-22T13:24:06.258942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:33:50.168426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162488+0000","last_change":"2024-04-22T13:22:05.248199+0000","last_active":"2024-04-22T13:24:06.162488+0000","last_peered":"2024-04-22T13:24:06.162488+0000","last_clean":"2024-04-22T13:24:06.162488+0000","last_became_active":"2024-04-22T13:22:05.247873+0000","last_became_peered":"2024-04-22T13:22:05.247873+0000","last_unstale":"2024-04-22T13:24:06.162488+0000","last_undegraded":"2024-04-22T13:24:06.162488+0000","last_fullsized":"2024-04-22T13:24:06.162488+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:40:29.353901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'102","reported_seq":216,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:14.859456+0000","last_change":"2024-04-22T13:22:04.243474+0000","last_active":"2024-04-22T13:24:14.859456+0000","last_peered":"2024-04-22T13:24:14.859456+0000","last_clean":"2024-04-22T13:24:14.859456+0000","last_became_active":"2024-04-22T13:22:04.243151+0000","last_became_peered":"2024-04-22T13:22:04.243151+0000","last_unstale":"2024-04-22T13:24:14.859456+0000","last_undegraded":"2024-04-22T13:24:14.859456+0000","last_fullsized":"2024-04-22T13:24:14.859456+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:18:25.140715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":185,"num_write_kb":1285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259312+0000","last_change":"2024-04-22T13:22:05.243120+0000","last_active":"2024-04-22T13:24:06.259312+0000","last_peered":"2024-04-22T13:24:06.259312+0000","last_clean":"2024-04-22T13:24:06.259312+0000","last_became_active":"2024-04-22T13:22:05.242726+0000","last_became_peered":"2024-04-22T13:22:05.242726+0000","last_unstale":"2024-04-22T13:24:06.259312+0000","last_undegraded":"2024-04-22T13:24:06.259312+0000","last_fullsized":"2024-04-22T13:24:06.259312+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:06:16.087884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411472+0000","last_change":"2024-04-22T13:22:04.239672+0000","last_active":"2024-04-22T13:24:06.411472+0000","last_peered":"2024-04-22T13:24:06.411472+0000","last_clean":"2024-04-22T13:24:06.411472+0000","last_became_active":"2024-04-22T13:22:04.238776+0000","last_became_peered":"2024-04-22T13:22:04.238776+0000","last_unstale":"2024-04-22T13:24:06.411472+0000","last_undegraded":"2024-04-22T13:24:06.411472+0000","last_fullsized":"2024-04-22T13:24:06.411472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:43:21.752547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.412085+0000","last_change":"2024-04-22T13:22:05.242089+0000","last_active":"2024-04-22T13:24:06.412085+0000","last_peered":"2024-04-22T13:24:06.412085+0000","last_clean":"2024-04-22T13:24:06.412085+0000","last_became_active":"2024-04-22T13:22:05.241681+0000","last_became_peered":"2024-04-22T13:22:05.241681+0000","last_unstale":"2024-04-22T13:24:06.412085+0000","last_undegraded":"2024-04-22T13:24:06.412085+0000","last_fullsized":"2024-04-22T13:24:06.412085+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:58:49.906188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.163790+0000","last_change":"2024-04-22T13:23:05.513123+0000","last_active":"2024-04-22T13:24:06.163790+0000","last_peered":"2024-04-22T13:24:06.163790+0000","last_clean":"2024-04-22T13:24:06.163790+0000","last_became_active":"2024-04-22T13:23:05.509367+0000","last_became_peered":"2024-04-22T13:23:05.509367+0000","last_unstale":"2024-04-22T13:24:06.163790+0000","last_undegraded":"2024-04-22T13:24:06.163790+0000","last_fullsized":"2024-04-22T13:24:06.163790+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:28:41.770957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258703+0000","last_change":"2024-04-22T13:22:05.250120+0000","last_active":"2024-04-22T13:24:06.258703+0000","last_peered":"2024-04-22T13:24:06.258703+0000","last_clean":"2024-04-22T13:24:06.258703+0000","last_became_active":"2024-04-22T13:22:05.247936+0000","last_became_peered":"2024-04-22T13:22:05.247936+0000","last_unstale":"2024-04-22T13:24:06.258703+0000","last_undegraded":"2024-04-22T13:24:06.258703+0000","last_fullsized":"2024-04-22T13:24:06.258703+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:27:38.437918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258959+0000","last_change":"2024-04-22T13:22:04.244026+0000","last_active":"2024-04-22T13:24:06.258959+0000","last_peered":"2024-04-22T13:24:06.258959+0000","last_clean":"2024-04-22T13:24:06.258959+0000","last_became_active":"2024-04-22T13:22:04.243518+0000","last_became_peered":"2024-04-22T13:22:04.243518+0000","last_unstale":"2024-04-22T13:24:06.258959+0000","last_undegraded":"2024-04-22T13:24:06.258959+0000","last_fullsized":"2024-04-22T13:24:06.258959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:54:48.791874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418285+0000","last_change":"2024-04-22T13:22:05.252674+0000","last_active":"2024-04-22T13:24:06.418285+0000","last_peered":"2024-04-22T13:24:06.418285+0000","last_clean":"2024-04-22T13:24:06.418285+0000","last_became_active":"2024-04-22T13:22:05.251087+0000","last_became_peered":"2024-04-22T13:22:05.251087+0000","last_unstale":"2024-04-22T13:24:06.418285+0000","last_undegraded":"2024-04-22T13:24:06.418285+0000","last_fullsized":"2024-04-22T13:24:06.418285+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:20:06.873412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.414693+0000","last_change":"2024-04-22T13:22:04.244409+0000","last_active":"2024-04-22T13:24:06.414693+0000","last_peered":"2024-04-22T13:24:06.414693+0000","last_clean":"2024-04-22T13:24:06.414693+0000","last_became_active":"2024-04-22T13:22:04.244104+0000","last_became_peered":"2024-04-22T13:22:04.244104+0000","last_unstale":"2024-04-22T13:24:06.414693+0000","last_undegraded":"2024-04-22T13:24:06.414693+0000","last_fullsized":"2024-04-22T13:24:06.414693+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T14:39:14.714755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259066+0000","last_change":"2024-04-22T13:22:05.245730+0000","last_active":"2024-04-22T13:24:06.259066+0000","last_peered":"2024-04-22T13:24:06.259066+0000","last_clean":"2024-04-22T13:24:06.259066+0000","last_became_active":"2024-04-22T13:22:05.245366+0000","last_became_peered":"2024-04-22T13:22:05.245366+0000","last_unstale":"2024-04-22T13:24:06.259066+0000","last_undegraded":"2024-04-22T13:24:06.259066+0000","last_fullsized":"2024-04-22T13:24:06.259066+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:17:31.935258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418212+0000","last_change":"2024-04-22T13:23:05.514197+0000","last_active":"2024-04-22T13:24:06.418212+0000","last_peered":"2024-04-22T13:24:06.418212+0000","last_clean":"2024-04-22T13:24:06.418212+0000","last_became_active":"2024-04-22T13:23:05.509431+0000","last_became_peered":"2024-04-22T13:23:05.509431+0000","last_unstale":"2024-04-22T13:24:06.418212+0000","last_undegraded":"2024-04-22T13:24:06.418212+0000","last_fullsized":"2024-04-22T13:24:06.418212+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:43:59.825567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409326+0000","last_change":"2024-04-22T13:22:05.242146+0000","last_active":"2024-04-22T13:24:06.409326+0000","last_peered":"2024-04-22T13:24:06.409326+0000","last_clean":"2024-04-22T13:24:06.409326+0000","last_became_active":"2024-04-22T13:22:05.241802+0000","last_became_peered":"2024-04-22T13:22:05.241802+0000","last_unstale":"2024-04-22T13:24:06.409326+0000","last_undegraded":"2024-04-22T13:24:06.409326+0000","last_fullsized":"2024-04-22T13:24:06.409326+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:49:49.002123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'8","reported_seq":159,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:15.574942+0000","last_change":"2024-04-22T13:22:04.248884+0000","last_active":"2024-04-22T13:24:15.574942+0000","last_peered":"2024-04-22T13:24:15.574942+0000","last_clean":"2024-04-22T13:24:15.574942+0000","last_became_active":"2024-04-22T13:22:04.248393+0000","last_became_peered":"2024-04-22T13:22:04.248393+0000","last_unstale":"2024-04-22T13:24:15.574942+0000","last_undegraded":"2024-04-22T13:24:15.574942+0000","last_fullsized":"2024-04-22T13:24:15.574942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:36:14.687457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2260,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":130,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409545+0000","last_change":"2024-04-22T13:22:05.256881+0000","last_active":"2024-04-22T13:24:06.409545+0000","last_peered":"2024-04-22T13:24:06.409545+0000","last_clean":"2024-04-22T13:24:06.409545+0000","last_became_active":"2024-04-22T13:22:05.255967+0000","last_became_peered":"2024-04-22T13:22:05.255967+0000","last_unstale":"2024-04-22T13:24:06.409545+0000","last_undegraded":"2024-04-22T13:24:06.409545+0000","last_fullsized":"2024-04-22T13:24:06.409545+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T18:24:21.137344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'67","reported_seq":120,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:14.858805+0000","last_change":"2024-04-22T13:22:04.247303+0000","last_active":"2024-04-22T13:24:14.858805+0000","last_peered":"2024-04-22T13:24:14.858805+0000","last_clean":"2024-04-22T13:24:14.858805+0000","last_became_active":"2024-04-22T13:22:04.246742+0000","last_became_peered":"2024-04-22T13:22:04.246742+0000","last_unstale":"2024-04-22T13:24:14.858805+0000","last_undegraded":"2024-04-22T13:24:14.858805+0000","last_fullsized":"2024-04-22T13:24:14.858805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:45:03.189042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409338+0000","last_change":"2024-04-22T13:22:05.251754+0000","last_active":"2024-04-22T13:24:06.409338+0000","last_peered":"2024-04-22T13:24:06.409338+0000","last_clean":"2024-04-22T13:24:06.409338+0000","last_became_active":"2024-04-22T13:22:05.251155+0000","last_became_peered":"2024-04-22T13:22:05.251155+0000","last_unstale":"2024-04-22T13:24:06.409338+0000","last_undegraded":"2024-04-22T13:24:06.409338+0000","last_fullsized":"2024-04-22T13:24:06.409338+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:08:46.682600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162485+0000","last_change":"2024-04-22T13:22:04.243382+0000","last_active":"2024-04-22T13:24:06.162485+0000","last_peered":"2024-04-22T13:24:06.162485+0000","last_clean":"2024-04-22T13:24:06.162485+0000","last_became_active":"2024-04-22T13:22:04.243055+0000","last_became_peered":"2024-04-22T13:22:04.243055+0000","last_unstale":"2024-04-22T13:24:06.162485+0000","last_undegraded":"2024-04-22T13:24:06.162485+0000","last_fullsized":"2024-04-22T13:24:06.162485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:03.211127+0000","last_clean_scrub_stamp":"2024-04-22T13:22:03.211127+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:19:58.379645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.409542+0000","last_change":"2024-04-22T13:22:05.249527+0000","last_active":"2024-04-22T13:24:06.409542+0000","last_peered":"2024-04-22T13:24:06.409542+0000","last_clean":"2024-04-22T13:24:06.409542+0000","last_became_active":"2024-04-22T13:22:05.249207+0000","last_became_peered":"2024-04-22T13:22:05.249207+0000","last_unstale":"2024-04-22T13:24:06.409542+0000","last_undegraded":"2024-04-22T13:24:06.409542+0000","last_fullsized":"2024-04-22T13:24:06.409542+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:50:02.755288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418285+0000","last_change":"2024-04-22T13:23:05.508324+0000","last_active":"2024-04-22T13:24:06.418285+0000","last_peered":"2024-04-22T13:24:06.418285+0000","last_clean":"2024-04-22T13:24:06.418285+0000","last_became_active":"2024-04-22T13:23:05.507814+0000","last_became_peered":"2024-04-22T13:23:05.507814+0000","last_unstale":"2024-04-22T13:24:06.418285+0000","last_undegraded":"2024-04-22T13:24:06.418285+0000","last_fullsized":"2024-04-22T13:24:06.418285+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:46:20.523715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.414640+0000","last_change":"2024-04-22T13:23:05.875683+0000","last_active":"2024-04-22T13:24:06.414640+0000","last_peered":"2024-04-22T13:24:06.414640+0000","last_clean":"2024-04-22T13:24:06.414640+0000","last_became_active":"2024-04-22T13:23:05.875150+0000","last_became_peered":"2024-04-22T13:23:05.875150+0000","last_unstale":"2024-04-22T13:24:06.414640+0000","last_undegraded":"2024-04-22T13:24:06.414640+0000","last_fullsized":"2024-04-22T13:24:06.414640+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:12:47.592230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.414612+0000","last_change":"2024-04-22T13:22:05.247882+0000","last_active":"2024-04-22T13:24:06.414612+0000","last_peered":"2024-04-22T13:24:06.414612+0000","last_clean":"2024-04-22T13:24:06.414612+0000","last_became_active":"2024-04-22T13:22:05.247525+0000","last_became_peered":"2024-04-22T13:22:05.247525+0000","last_unstale":"2024-04-22T13:24:06.414612+0000","last_undegraded":"2024-04-22T13:24:06.414612+0000","last_fullsized":"2024-04-22T13:24:06.414612+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T17:46:04.349076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.411618+0000","last_change":"2024-04-22T13:22:05.243716+0000","last_active":"2024-04-22T13:24:06.411618+0000","last_peered":"2024-04-22T13:24:06.411618+0000","last_clean":"2024-04-22T13:24:06.411618+0000","last_became_active":"2024-04-22T13:22:05.243397+0000","last_became_peered":"2024-04-22T13:22:05.243397+0000","last_unstale":"2024-04-22T13:24:06.411618+0000","last_undegraded":"2024-04-22T13:24:06.411618+0000","last_fullsized":"2024-04-22T13:24:06.411618+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:16:21.236339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162647+0000","last_change":"2024-04-22T13:22:05.246042+0000","last_active":"2024-04-22T13:24:06.162647+0000","last_peered":"2024-04-22T13:24:06.162647+0000","last_clean":"2024-04-22T13:24:06.162647+0000","last_became_active":"2024-04-22T13:22:05.245669+0000","last_became_peered":"2024-04-22T13:22:05.245669+0000","last_unstale":"2024-04-22T13:24:06.162647+0000","last_undegraded":"2024-04-22T13:24:06.162647+0000","last_fullsized":"2024-04-22T13:24:06.162647+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:49:05.539641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418387+0000","last_change":"2024-04-22T13:22:05.257009+0000","last_active":"2024-04-22T13:24:06.418387+0000","last_peered":"2024-04-22T13:24:06.418387+0000","last_clean":"2024-04-22T13:24:06.418387+0000","last_became_active":"2024-04-22T13:22:05.254327+0000","last_became_peered":"2024-04-22T13:22:05.254327+0000","last_unstale":"2024-04-22T13:24:06.418387+0000","last_undegraded":"2024-04-22T13:24:06.418387+0000","last_fullsized":"2024-04-22T13:24:06.418387+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:29:12.482525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.415396+0000","last_change":"2024-04-22T13:22:05.260575+0000","last_active":"2024-04-22T13:24:06.415396+0000","last_peered":"2024-04-22T13:24:06.415396+0000","last_clean":"2024-04-22T13:24:06.415396+0000","last_became_active":"2024-04-22T13:22:05.260181+0000","last_became_peered":"2024-04-22T13:22:05.260181+0000","last_unstale":"2024-04-22T13:24:06.415396+0000","last_undegraded":"2024-04-22T13:24:06.415396+0000","last_fullsized":"2024-04-22T13:24:06.415396+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:49:20.064368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.259185+0000","last_change":"2024-04-22T13:22:05.248199+0000","last_active":"2024-04-22T13:24:06.259185+0000","last_peered":"2024-04-22T13:24:06.259185+0000","last_clean":"2024-04-22T13:24:06.259185+0000","last_became_active":"2024-04-22T13:22:05.247858+0000","last_became_peered":"2024-04-22T13:22:05.247858+0000","last_unstale":"2024-04-22T13:24:06.259185+0000","last_undegraded":"2024-04-22T13:24:06.259185+0000","last_fullsized":"2024-04-22T13:24:06.259185+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:49:21.228105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.164341+0000","last_change":"2024-04-22T13:22:05.239137+0000","last_active":"2024-04-22T13:24:06.164341+0000","last_peered":"2024-04-22T13:24:06.164341+0000","last_clean":"2024-04-22T13:24:06.164341+0000","last_became_active":"2024-04-22T13:22:05.238807+0000","last_became_peered":"2024-04-22T13:22:05.238807+0000","last_unstale":"2024-04-22T13:24:06.164341+0000","last_undegraded":"2024-04-22T13:24:06.164341+0000","last_fullsized":"2024-04-22T13:24:06.164341+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T15:02:48.587097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.258756+0000","last_change":"2024-04-22T13:22:05.249080+0000","last_active":"2024-04-22T13:24:06.258756+0000","last_peered":"2024-04-22T13:24:06.258756+0000","last_clean":"2024-04-22T13:24:06.258756+0000","last_became_active":"2024-04-22T13:22:05.248733+0000","last_became_peered":"2024-04-22T13:22:05.248733+0000","last_unstale":"2024-04-22T13:24:06.258756+0000","last_undegraded":"2024-04-22T13:24:06.258756+0000","last_fullsized":"2024-04-22T13:24:06.258756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:58.735941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.163779+0000","last_change":"2024-04-22T13:22:05.255490+0000","last_active":"2024-04-22T13:24:06.163779+0000","last_peered":"2024-04-22T13:24:06.163779+0000","last_clean":"2024-04-22T13:24:06.163779+0000","last_became_active":"2024-04-22T13:22:05.253644+0000","last_became_peered":"2024-04-22T13:22:05.253644+0000","last_unstale":"2024-04-22T13:24:06.163779+0000","last_undegraded":"2024-04-22T13:24:06.163779+0000","last_fullsized":"2024-04-22T13:24:06.163779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:31:01.816182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.162443+0000","last_change":"2024-04-22T13:22:05.240988+0000","last_active":"2024-04-22T13:24:06.162443+0000","last_peered":"2024-04-22T13:24:06.162443+0000","last_clean":"2024-04-22T13:24:06.162443+0000","last_became_active":"2024-04-22T13:22:05.240642+0000","last_became_peered":"2024-04-22T13:22:05.240642+0000","last_unstale":"2024-04-22T13:24:06.162443+0000","last_undegraded":"2024-04-22T13:24:06.162443+0000","last_fullsized":"2024-04-22T13:24:06.162443+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T16:41:29.001979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-22T13:24:06.418222+0000","last_change":"2024-04-22T13:22:05.252212+0000","last_active":"2024-04-22T13:24:06.418222+0000","last_peered":"2024-04-22T13:24:06.418222+0000","last_clean":"2024-04-22T13:24:06.418222+0000","last_became_active":"2024-04-22T13:22:05.251852+0000","last_became_peered":"2024-04-22T13:22:05.251852+0000","last_unstale":"2024-04-22T13:24:06.418222+0000","last_undegraded":"2024-04-22T13:24:06.418222+0000","last_fullsized":"2024-04-22T13:24:06.418222+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T13:22:04.218431+0000","last_clean_scrub_stamp":"2024-04-22T13:22:04.218431+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:13:55.132501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":799476,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1660,"num_read_kb":1053,"num_write":503,"num_write_kb":2385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2764800,"data_stored":2398428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":384,"ondisk_log_size":384,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318312,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":473150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579947,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29112,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716040,"statfs":{"total":95995035648,"available":95965224960,"internally_reserved":0,"allocated":2023424,"data_stored":1460273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743473,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28384,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716768,"statfs":{"total":95995035648,"available":95965970432,"internally_reserved":0,"allocated":1400832,"data_stored":865206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005110,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28380,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716772,"statfs":{"total":95995035648,"available":95965974528,"internally_reserved":0,"allocated":1396736,"data_stored":865584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234043,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36900,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708252,"statfs":{"total":95995035648,"available":95957250048,"internally_reserved":0,"allocated":1601536,"data_stored":1067373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430272,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":991232,"data_stored":472489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":541040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":474136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1081344,"data_stored":542350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117072,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37148,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708004,"statfs":{"total":95995035648,"available":95956996096,"internally_reserved":0,"allocated":1732608,"data_stored":1192980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855435,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":991232,"data_stored":472882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691909,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29036,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716116,"statfs":{"total":95995035648,"available":95965302784,"internally_reserved":0,"allocated":1945600,"data_stored":1393187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":331834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":394716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":398920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T13:24:18.501 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:24:18.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.643+0000 7f2517375640 1 -- 172.21.15.12:0/3689411242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2510101ad0 msgr2=0x7f251010dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:18.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.643+0000 7f2517375640 1 --2- 172.21.15.12:0/3689411242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2510101ad0 0x7f251010dca0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2508009ca0 tx=0x7f250802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.643+0000 7f2517375640 1 -- 172.21.15.12:0/3689411242 shutdown_connections 2024-04-22T13:24:18.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.643+0000 7f2517375640 1 --2- 172.21.15.12:0/3689411242 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2510101ad0 0x7f251010dca0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.643+0000 7f2517375640 1 --2- 172.21.15.12:0/3689411242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2510101130 0x7f2510101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.643+0000 7f2517375640 1 --2- 172.21.15.12:0/3689411242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2510106f00 0x7f25101072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.643+0000 7f2517375640 1 -- 172.21.15.12:0/3689411242 >> 172.21.15.12:0/3689411242 conn(0x7f2510077e30 msgr2=0x7f2510078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:18.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.643+0000 7f2517375640 1 -- 172.21.15.12:0/3689411242 shutdown_connections 2024-04-22T13:24:18.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.643+0000 7f2517375640 1 -- 172.21.15.12:0/3689411242 wait complete. 2024-04-22T13:24:18.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.644+0000 7f2517375640 1 Processor -- start 2024-04-22T13:24:18.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.644+0000 7f2517375640 1 -- start start 2024-04-22T13:24:18.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.644+0000 7f2517375640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2510101130 0x7f2510198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:18.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.644+0000 7f2517375640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2510101ad0 0x7f2510198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:18.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.644+0000 7f2517375640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2510106f00 0x7f251019cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:18.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.644+0000 7f2517375640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f25101110e0 con 0x7f2510106f00 2024-04-22T13:24:18.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.644+0000 7f2517375640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2510110f60 con 0x7f2510101ad0 2024-04-22T13:24:18.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.644+0000 7f2517375640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2510111260 con 0x7f2510101130 2024-04-22T13:24:18.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.644+0000 7f2515912640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2510106f00 0x7f251019cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:18.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.644+0000 7f2515912640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2510106f00 0x7f251019cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59548/0 (socket says 172.21.15.12:59548) 2024-04-22T13:24:18.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.644+0000 7f2515912640 1 -- 172.21.15.12:0/2001777862 learned_addr learned my addr 172.21.15.12:0/2001777862 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:18.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.644+0000 7f2514910640 1 --2- 172.21.15.12:0/2001777862 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2510101ad0 0x7f2510198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:18.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.645+0000 7f2515111640 1 --2- 172.21.15.12:0/2001777862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2510101130 0x7f2510198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:18.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.645+0000 7f2515912640 1 -- 172.21.15.12:0/2001777862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2510101130 msgr2=0x7f2510198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:18.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.645+0000 7f2515912640 1 --2- 172.21.15.12:0/2001777862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2510101130 0x7f2510198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.645+0000 7f2515912640 1 -- 172.21.15.12:0/2001777862 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2510101ad0 msgr2=0x7f2510198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:18.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.645+0000 7f2515912640 1 --2- 172.21.15.12:0/2001777862 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2510101ad0 0x7f2510198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.645+0000 7f2515912640 1 -- 172.21.15.12:0/2001777862 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f251019d6c0 con 0x7f2510106f00 2024-04-22T13:24:18.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.645+0000 7f2515111640 1 --2- 172.21.15.12:0/2001777862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2510101130 0x7f2510198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:18.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.645+0000 7f2515912640 1 --2- 172.21.15.12:0/2001777862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2510106f00 0x7f251019cf90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2508005e30 tx=0x7f2508034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:18.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.645+0000 7f25067fc640 1 -- 172.21.15.12:0/2001777862 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508041020 con 0x7f2510106f00 2024-04-22T13:24:18.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.645+0000 7f25067fc640 1 -- 172.21.15.12:0/2001777862 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2508032970 con 0x7f2510106f00 2024-04-22T13:24:18.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.645+0000 7f2517375640 1 -- 172.21.15.12:0/2001777862 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f251019d950 con 0x7f2510106f00 2024-04-22T13:24:18.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.645+0000 7f25067fc640 1 -- 172.21.15.12:0/2001777862 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508039040 con 0x7f2510106f00 2024-04-22T13:24:18.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.645+0000 7f2517375640 1 -- 172.21.15.12:0/2001777862 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25101a5240 con 0x7f2510106f00 2024-04-22T13:24:18.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.646+0000 7f2517375640 1 -- 172.21.15.12:0/2001777862 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2510069230 con 0x7f2510106f00 2024-04-22T13:24:18.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.647+0000 7f25067fc640 1 -- 172.21.15.12:0/2001777862 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f2508044020 con 0x7f2510106f00 2024-04-22T13:24:18.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.648+0000 7f25067fc640 1 --2- 172.21.15.12:0/2001777862 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f24fc0a3ec0 0x7f24fc0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:18.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.648+0000 7f25067fc640 1 -- 172.21.15.12:0/2001777862 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f25080f2b10 con 0x7f2510106f00 2024-04-22T13:24:18.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.648+0000 7f2515111640 1 --2- 172.21.15.12:0/2001777862 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f24fc0a3ec0 0x7f24fc0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:18.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.648+0000 7f2515111640 1 --2- 172.21.15.12:0/2001777862 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f24fc0a3ec0 0x7f24fc0a6380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f250c005fd0 tx=0x7f250c00b5d0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:18.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.651+0000 7f25067fc640 1 -- 172.21.15.12:0/2001777862 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f250804b020 con 0x7f2510106f00 2024-04-22T13:24:18.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:18 smithi012 ceph-mon[28265]: from='client.15657 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:24:18.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.877+0000 7f2517375640 1 -- 172.21.15.12:0/2001777862 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f25101a5470 con 0x7f2510106f00 2024-04-22T13:24:18.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.878+0000 7f25067fc640 1 -- 172.21.15.12:0/2001777862 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7f25080bf060 con 0x7f2510106f00 2024-04-22T13:24:18.875 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:24:18.875 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:24:18.875 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:24:18.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.880+0000 7f2517375640 1 -- 172.21.15.12:0/2001777862 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f24fc0a3ec0 msgr2=0x7f24fc0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:18.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.880+0000 7f2517375640 1 --2- 172.21.15.12:0/2001777862 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f24fc0a3ec0 0x7f24fc0a6380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f250c005fd0 tx=0x7f250c00b5d0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.880+0000 7f2517375640 1 -- 172.21.15.12:0/2001777862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2510106f00 msgr2=0x7f251019cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:18.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.880+0000 7f2517375640 1 --2- 172.21.15.12:0/2001777862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2510106f00 0x7f251019cf90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2508005e30 tx=0x7f2508034670 comp rx=0 tx=0).stop 2024-04-22T13:24:18.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.880+0000 7f2517375640 1 -- 172.21.15.12:0/2001777862 shutdown_connections 2024-04-22T13:24:18.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.880+0000 7f2517375640 1 --2- 172.21.15.12:0/2001777862 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f24fc0a3ec0 0x7f24fc0a6380 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.880+0000 7f2517375640 1 --2- 172.21.15.12:0/2001777862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2510106f00 0x7f251019cf90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.880+0000 7f2517375640 1 --2- 172.21.15.12:0/2001777862 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2510101ad0 0x7f2510198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.880+0000 7f2517375640 1 --2- 172.21.15.12:0/2001777862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2510101130 0x7f2510198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.880+0000 7f2517375640 1 -- 172.21.15.12:0/2001777862 >> 172.21.15.12:0/2001777862 conn(0x7f2510077e30 msgr2=0x7f251006a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:18.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.880+0000 7f2517375640 1 -- 172.21.15.12:0/2001777862 shutdown_connections 2024-04-22T13:24:18.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.880+0000 7f2517375640 1 -- 172.21.15.12:0/2001777862 wait complete. 2024-04-22T13:24:18.888 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:24:18.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:18 smithi111 ceph-mon[31361]: from='client.15657 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:24:18.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.993+0000 7fa2d0b2b640 1 -- 172.21.15.12:0/1520055091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2cc1031d0 msgr2=0x7fa2cc1035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:18.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.993+0000 7fa2d0b2b640 1 --2- 172.21.15.12:0/1520055091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2cc1031d0 0x7fa2cc1035b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009900 tx=0x7fa2bc02bcd0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.993+0000 7fa2d0b2b640 1 -- 172.21.15.12:0/1520055091 shutdown_connections 2024-04-22T13:24:18.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.993+0000 7fa2d0b2b640 1 --2- 172.21.15.12:0/1520055091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2cc10c5a0 0x7fa2cc10e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.993+0000 7fa2d0b2b640 1 --2- 172.21.15.12:0/1520055091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2cc103af0 0x7fa2cc10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.993+0000 7fa2d0b2b640 1 --2- 172.21.15.12:0/1520055091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2cc1031d0 0x7fa2cc1035b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.993+0000 7fa2d0b2b640 1 -- 172.21.15.12:0/1520055091 >> 172.21.15.12:0/1520055091 conn(0x7fa2cc0771d0 msgr2=0x7fa2cc0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:18.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.993+0000 7fa2d0b2b640 1 -- 172.21.15.12:0/1520055091 shutdown_connections 2024-04-22T13:24:18.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.993+0000 7fa2d0b2b640 1 -- 172.21.15.12:0/1520055091 wait complete. 2024-04-22T13:24:18.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.994+0000 7fa2d0b2b640 1 Processor -- start 2024-04-22T13:24:18.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.994+0000 7fa2d0b2b640 1 -- start start 2024-04-22T13:24:18.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.994+0000 7fa2d0b2b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2cc1031d0 0x7fa2cc198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:18.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.994+0000 7fa2d0b2b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2cc103af0 0x7fa2cc198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:18.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.994+0000 7fa2d0b2b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2cc10c5a0 0x7fa2cc19d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:18.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.994+0000 7fa2d0b2b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc1121f0 con 0x7fa2cc1031d0 2024-04-22T13:24:18.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.994+0000 7fa2d0b2b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc112070 con 0x7fa2cc103af0 2024-04-22T13:24:18.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.994+0000 7fa2d0b2b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc112370 con 0x7fa2cc10c5a0 2024-04-22T13:24:18.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.994+0000 7fa2ca59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2cc1031d0 0x7fa2cc198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:18.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.994+0000 7fa2ca59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2cc1031d0 0x7fa2cc198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59556/0 (socket says 172.21.15.12:59556) 2024-04-22T13:24:18.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.994+0000 7fa2ca59c640 1 -- 172.21.15.12:0/977482418 learned_addr learned my addr 172.21.15.12:0/977482418 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:18.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.994+0000 7fa2c9d9b640 1 --2- 172.21.15.12:0/977482418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2cc103af0 0x7fa2cc198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:18.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.994+0000 7fa2ca59c640 1 -- 172.21.15.12:0/977482418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2cc10c5a0 msgr2=0x7fa2cc19d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:18.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.994+0000 7fa2cad9d640 1 --2- 172.21.15.12:0/977482418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2cc10c5a0 0x7fa2cc19d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:18.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.994+0000 7fa2ca59c640 1 --2- 172.21.15.12:0/977482418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2cc10c5a0 0x7fa2cc19d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.994+0000 7fa2ca59c640 1 -- 172.21.15.12:0/977482418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2cc103af0 msgr2=0x7fa2cc198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:18.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.994+0000 7fa2ca59c640 1 --2- 172.21.15.12:0/977482418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2cc103af0 0x7fa2cc198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:18.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.994+0000 7fa2ca59c640 1 -- 172.21.15.12:0/977482418 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2cc19d740 con 0x7fa2cc1031d0 2024-04-22T13:24:18.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.994+0000 7fa2c9d9b640 1 --2- 172.21.15.12:0/977482418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2cc103af0 0x7fa2cc198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:18.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.995+0000 7fa2ca59c640 1 --2- 172.21.15.12:0/977482418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2cc1031d0 0x7fa2cc198750 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc005940 tx=0x7fa2bc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:18.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.995+0000 7fa2bb7fe640 1 -- 172.21.15.12:0/977482418 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc043020 con 0x7fa2cc1031d0 2024-04-22T13:24:18.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.995+0000 7fa2d0b2b640 1 -- 172.21.15.12:0/977482418 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2cc19d9d0 con 0x7fa2cc1031d0 2024-04-22T13:24:18.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.995+0000 7fa2bb7fe640 1 -- 172.21.15.12:0/977482418 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2bc02dba0 con 0x7fa2cc1031d0 2024-04-22T13:24:18.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.995+0000 7fa2bb7fe640 1 -- 172.21.15.12:0/977482418 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc003930 con 0x7fa2cc1031d0 2024-04-22T13:24:18.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.995+0000 7fa2d0b2b640 1 -- 172.21.15.12:0/977482418 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2cc1a52c0 con 0x7fa2cc1031d0 2024-04-22T13:24:18.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.996+0000 7fa2d0b2b640 1 -- 172.21.15.12:0/977482418 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2cc069230 con 0x7fa2cc1031d0 2024-04-22T13:24:18.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.997+0000 7fa2bb7fe640 1 -- 172.21.15.12:0/977482418 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fa2bc046020 con 0x7fa2cc1031d0 2024-04-22T13:24:18.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.998+0000 7fa2bb7fe640 1 --2- 172.21.15.12:0/977482418 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa2b40a3e40 0x7fa2b40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:18.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.998+0000 7fa2c9d9b640 1 --2- 172.21.15.12:0/977482418 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa2b40a3e40 0x7fa2b40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:18.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.998+0000 7fa2bb7fe640 1 -- 172.21.15.12:0/977482418 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fa2bc0f3180 con 0x7fa2cc1031d0 2024-04-22T13:24:18.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:18.998+0000 7fa2c9d9b640 1 --2- 172.21.15.12:0/977482418 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa2b40a3e40 0x7fa2b40a6300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc10e360 tx=0x7fa2c0005f90 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:18.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.001+0000 7fa2bb7fe640 1 -- 172.21.15.12:0/977482418 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa2bc04d020 con 0x7fa2cc1031d0 2024-04-22T13:24:19.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:18 smithi134 ceph-mon[31338]: from='client.15657 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:24:19.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.188+0000 7fa2d0b2b640 1 -- 172.21.15.12:0/977482418 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa2cc107690 con 0x7fa2cc1031d0 2024-04-22T13:24:19.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.189+0000 7fa2bb7fe640 1 -- 172.21.15.12:0/977482418 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22997 (secure 0 0 0) 0x7fa2bc0bf6d0 con 0x7fa2cc1031d0 2024-04-22T13:24:19.187 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:24:19.187 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":90,"fsid":"0aede5ea-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:14:09.362918+0000","modified":"2024-04-22T13:24:06.155595+0000","last_up_change":"2024-04-22T13:21:15.263393+0000","last_in_change":"2024-04-22T13:20:54.174685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:17:32.372434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:22:02.768799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:22:03.578730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d9e2365-3a64-4b13-96fc-c217b1cee2ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6803","nonce":1191350632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6805","nonce":1191350632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6809","nonce":1191350632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6807","nonce":1191350632}]},"public_addr":"172.21.15.12:6803/1191350632","cluster_addr":"172.21.15.12:6805/1191350632","heartbeat_back_addr":"172.21.15.12:6809/1191350632","heartbeat_front_addr":"172.21.15.12:6807/1191350632","state":["exists","up"]},{"osd":1,"uuid":"1499c783-2bd7-48ce-a444-34add42bcf31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6801","nonce":417496465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6803","nonce":417496465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6807","nonce":417496465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6805","nonce":417496465}]},"public_addr":"172.21.15.111:6801/417496465","cluster_addr":"172.21.15.111:6803/417496465","heartbeat_back_addr":"172.21.15.111:6807/417496465","heartbeat_front_addr":"172.21.15.111:6805/417496465","state":["exists","up"]},{"osd":2,"uuid":"71a9ebc3-c622-434e-a6c2-f48358f384f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6801","nonce":1403153998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6803","nonce":1403153998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6807","nonce":1403153998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6805","nonce":1403153998}]},"public_addr":"172.21.15.134:6801/1403153998","cluster_addr":"172.21.15.134:6803/1403153998","heartbeat_back_addr":"172.21.15.134:6807/1403153998","heartbeat_front_addr":"172.21.15.134:6805/1403153998","state":["exists","up"]},{"osd":3,"uuid":"6dd00db2-a286-48ba-8666-56ffdd5f35ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6811","nonce":3359066681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6813","nonce":3359066681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6817","nonce":3359066681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6815","nonce":3359066681}]},"public_addr":"172.21.15.12:6811/3359066681","cluster_addr":"172.21.15.12:6813/3359066681","heartbeat_back_addr":"172.21.15.12:6817/3359066681","heartbeat_front_addr":"172.21.15.12:6815/3359066681","state":["exists","up"]},{"osd":4,"uuid":"1e29afd7-354c-4506-aaa0-b43eb1a19a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6809","nonce":2886349872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6811","nonce":2886349872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6815","nonce":2886349872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6813","nonce":2886349872}]},"public_addr":"172.21.15.111:6809/2886349872","cluster_addr":"172.21.15.111:6811/2886349872","heartbeat_back_addr":"172.21.15.111:6815/2886349872","heartbeat_front_addr":"172.21.15.111:6813/2886349872","state":["exists","up"]},{"osd":5,"uuid":"8f40f02d-0ba2-4842-a03d-2a1a04d1010d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6809","nonce":3071977602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6811","nonce":3071977602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6815","nonce":3071977602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6813","nonce":3071977602}]},"public_addr":"172.21.15.134:6809/3071977602","cluster_addr":"172.21.15.134:6811/3071977602","heartbeat_back_addr":"172.21.15.134:6815/3071977602","heartbeat_front_addr":"172.21.15.134:6813/3071977602","state":["exists","up"]},{"osd":6,"uuid":"bb0c17a9-6a8e-4be6-8660-f73cfa8200ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6819","nonce":3819619299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6821","nonce":3819619299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6825","nonce":3819619299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6823","nonce":3819619299}]},"public_addr":"172.21.15.12:6819/3819619299","cluster_addr":"172.21.15.12:6821/3819619299","heartbeat_back_addr":"172.21.15.12:6825/3819619299","heartbeat_front_addr":"172.21.15.12:6823/3819619299","state":["exists","up"]},{"osd":7,"uuid":"0a93d6af-569e-4e45-b697-9e978c1c1004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6817","nonce":3113856282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6819","nonce":3113856282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6823","nonce":3113856282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6821","nonce":3113856282}]},"public_addr":"172.21.15.111:6817/3113856282","cluster_addr":"172.21.15.111:6819/3113856282","heartbeat_back_addr":"172.21.15.111:6823/3113856282","heartbeat_front_addr":"172.21.15.111:6821/3113856282","state":["exists","up"]},{"osd":8,"uuid":"e21acc68-cf50-4f4f-b1ba-cf6c195b4955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6817","nonce":1689362191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6819","nonce":1689362191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6823","nonce":1689362191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6821","nonce":1689362191}]},"public_addr":"172.21.15.134:6817/1689362191","cluster_addr":"172.21.15.134:6819/1689362191","heartbeat_back_addr":"172.21.15.134:6823/1689362191","heartbeat_front_addr":"172.21.15.134:6821/1689362191","state":["exists","up"]},{"osd":9,"uuid":"fd745869-7f5f-48a8-b412-bb9c92b586e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6827","nonce":4292385779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6829","nonce":4292385779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6833","nonce":4292385779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6831","nonce":4292385779}]},"public_addr":"172.21.15.12:6827/4292385779","cluster_addr":"172.21.15.12:6829/4292385779","heartbeat_back_addr":"172.21.15.12:6833/4292385779","heartbeat_front_addr":"172.21.15.12:6831/4292385779","state":["exists","up"]},{"osd":10,"uuid":"798b1abd-f6c7-4a5f-8da0-fdae49e50d2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6825","nonce":3338566976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6827","nonce":3338566976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6831","nonce":3338566976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6829","nonce":3338566976}]},"public_addr":"172.21.15.111:6825/3338566976","cluster_addr":"172.21.15.111:6827/3338566976","heartbeat_back_addr":"172.21.15.111:6831/3338566976","heartbeat_front_addr":"172.21.15.111:6829/3338566976","state":["exists","up"]},{"osd":11,"uuid":"b18fc5f3-1049-4871-b060-e99290273cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6825","nonce":578640428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6827","nonce":578640428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6831","nonce":578640428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6829","nonce":578640428}]},"public_addr":"172.21.15.134:6825/578640428","cluster_addr":"172.21.15.134:6827/578640428","heartbeat_back_addr":"172.21.15.134:6831/578640428","heartbeat_front_addr":"172.21.15.134:6829/578640428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:16:40.828608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:05.052528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:28.297316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:53.496497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:17.947259+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:41.712894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:07.105945+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:32.129688+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:56.008516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:22.239867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:47.334692+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:12.301276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/3535299028":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/2837479598":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/3010129696":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/2593601362":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/2519382069":"2024-04-23T13:15:04.279990+0000","172.21.15.12:6801/2009831940":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/856202072":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/3509412187":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/1802873236":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/1007749062":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3588480095":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1904962176":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/1802873236":"2024-04-23T13:24:06.155564+0000","172.21.15.12:6800/2009831940":"2024-04-23T13:15:04.279990+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:24:19.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.191+0000 7fa2d0b2b640 1 -- 172.21.15.12:0/977482418 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa2b40a3e40 msgr2=0x7fa2b40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:19.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.191+0000 7fa2d0b2b640 1 --2- 172.21.15.12:0/977482418 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa2b40a3e40 0x7fa2b40a6300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc10e360 tx=0x7fa2c0005f90 comp rx=0 tx=0).stop 2024-04-22T13:24:19.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.191+0000 7fa2d0b2b640 1 -- 172.21.15.12:0/977482418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2cc1031d0 msgr2=0x7fa2cc198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:19.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.191+0000 7fa2d0b2b640 1 --2- 172.21.15.12:0/977482418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2cc1031d0 0x7fa2cc198750 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc005940 tx=0x7fa2bc0045e0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.191+0000 7fa2d0b2b640 1 -- 172.21.15.12:0/977482418 shutdown_connections 2024-04-22T13:24:19.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.191+0000 7fa2d0b2b640 1 --2- 172.21.15.12:0/977482418 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa2b40a3e40 0x7fa2b40a6300 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.191+0000 7fa2d0b2b640 1 --2- 172.21.15.12:0/977482418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa2cc10c5a0 0x7fa2cc19d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.191+0000 7fa2d0b2b640 1 --2- 172.21.15.12:0/977482418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa2cc103af0 0x7fa2cc198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.191+0000 7fa2d0b2b640 1 --2- 172.21.15.12:0/977482418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa2cc1031d0 0x7fa2cc198750 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.191+0000 7fa2d0b2b640 1 -- 172.21.15.12:0/977482418 >> 172.21.15.12:0/977482418 conn(0x7fa2cc0771d0 msgr2=0x7fa2cc0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:19.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.191+0000 7fa2d0b2b640 1 -- 172.21.15.12:0/977482418 shutdown_connections 2024-04-22T13:24:19.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.191+0000 7fa2d0b2b640 1 -- 172.21.15.12:0/977482418 wait complete. 2024-04-22T13:24:19.200 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T13:17:32.372434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T13:22:02.768799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T13:22:03.578730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T13:24:19.200 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T13:24:19.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.308+0000 7fe7693c7640 1 -- 172.21.15.12:0/3142138365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe764102190 msgr2=0x7fe764102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:19.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.308+0000 7fe7693c7640 1 --2- 172.21.15.12:0/3142138365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe764102190 0x7fe764102570 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe754009900 tx=0x7fe75402bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:19.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.308+0000 7fe7693c7640 1 -- 172.21.15.12:0/3142138365 shutdown_connections 2024-04-22T13:24:19.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.308+0000 7fe7693c7640 1 --2- 172.21.15.12:0/3142138365 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe764107260 0x7fe76410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.308+0000 7fe7693c7640 1 --2- 172.21.15.12:0/3142138365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe764102b40 0x7fe764106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.308+0000 7fe7693c7640 1 --2- 172.21.15.12:0/3142138365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe764102190 0x7fe764102570 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.308+0000 7fe7693c7640 1 -- 172.21.15.12:0/3142138365 >> 172.21.15.12:0/3142138365 conn(0x7fe7640fd990 msgr2=0x7fe7640ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:19.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.308+0000 7fe7693c7640 1 -- 172.21.15.12:0/3142138365 shutdown_connections 2024-04-22T13:24:19.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.308+0000 7fe7693c7640 1 -- 172.21.15.12:0/3142138365 wait complete. 2024-04-22T13:24:19.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.309+0000 7fe7693c7640 1 Processor -- start 2024-04-22T13:24:19.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.309+0000 7fe7693c7640 1 -- start start 2024-04-22T13:24:19.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.309+0000 7fe7693c7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe764102190 0x7fe76419e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:19.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.309+0000 7fe7693c7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe764107260 0x7fe76419e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:19.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.309+0000 7fe7693c7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe764198610 0x7fe764198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:19.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.309+0000 7fe7693c7640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe7641121c0 con 0x7fe764102190 2024-04-22T13:24:19.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.309+0000 7fe7693c7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe764112040 con 0x7fe764107260 2024-04-22T13:24:19.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.309+0000 7fe7693c7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe764112340 con 0x7fe764198610 2024-04-22T13:24:19.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.309+0000 7fe762ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe764102190 0x7fe76419e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:19.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.309+0000 7fe762ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe764102190 0x7fe76419e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59568/0 (socket says 172.21.15.12:59568) 2024-04-22T13:24:19.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.309+0000 7fe762ffd640 1 -- 172.21.15.12:0/2783851062 learned_addr learned my addr 172.21.15.12:0/2783851062 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:19.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.309+0000 7fe75bfff640 1 --2- 172.21.15.12:0/2783851062 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe764107260 0x7fe76419e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:19.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.310+0000 7fe7637fe640 1 --2- 172.21.15.12:0/2783851062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe764198610 0x7fe764198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:19.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.310+0000 7fe762ffd640 1 -- 172.21.15.12:0/2783851062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe764198610 msgr2=0x7fe764198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:19.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.310+0000 7fe762ffd640 1 --2- 172.21.15.12:0/2783851062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe764198610 0x7fe764198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.310+0000 7fe762ffd640 1 -- 172.21.15.12:0/2783851062 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe764107260 msgr2=0x7fe76419e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:19.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.310+0000 7fe762ffd640 1 --2- 172.21.15.12:0/2783851062 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe764107260 0x7fe76419e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.310+0000 7fe762ffd640 1 -- 172.21.15.12:0/2783851062 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe764199130 con 0x7fe764102190 2024-04-22T13:24:19.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.310+0000 7fe7637fe640 1 --2- 172.21.15.12:0/2783851062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe764198610 0x7fe764198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:19.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.310+0000 7fe762ffd640 1 --2- 172.21.15.12:0/2783851062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe764102190 0x7fe76419e410 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe754005940 tx=0x7fe75402c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:19.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.310+0000 7fe760ff9640 1 -- 172.21.15.12:0/2783851062 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe754043020 con 0x7fe764102190 2024-04-22T13:24:19.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.310+0000 7fe760ff9640 1 -- 172.21.15.12:0/2783851062 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe75402ca70 con 0x7fe764102190 2024-04-22T13:24:19.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.310+0000 7fe7693c7640 1 -- 172.21.15.12:0/2783851062 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7641993c0 con 0x7fe764102190 2024-04-22T13:24:19.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.310+0000 7fe760ff9640 1 -- 172.21.15.12:0/2783851062 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7540039f0 con 0x7fe764102190 2024-04-22T13:24:19.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.310+0000 7fe7693c7640 1 -- 172.21.15.12:0/2783851062 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7641a5610 con 0x7fe764102190 2024-04-22T13:24:19.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.311+0000 7fe7693c7640 1 -- 172.21.15.12:0/2783851062 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7640691c0 con 0x7fe764102190 2024-04-22T13:24:19.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.312+0000 7fe760ff9640 1 -- 172.21.15.12:0/2783851062 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fe754046020 con 0x7fe764102190 2024-04-22T13:24:19.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.313+0000 7fe760ff9640 1 --2- 172.21.15.12:0/2783851062 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe7500a3ec0 0x7fe7500a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:19.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.313+0000 7fe75bfff640 1 --2- 172.21.15.12:0/2783851062 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe7500a3ec0 0x7fe7500a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:19.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.313+0000 7fe760ff9640 1 -- 172.21.15.12:0/2783851062 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fe7540f29c0 con 0x7fe764102190 2024-04-22T13:24:19.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.313+0000 7fe75bfff640 1 --2- 172.21.15.12:0/2783851062 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe7500a3ec0 0x7fe7500a6380 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe7640fdeb0 tx=0x7fe74c005f90 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:19.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.317+0000 7fe760ff9640 1 -- 172.21.15.12:0/2783851062 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe7540f4e20 con 0x7fe764102190 2024-04-22T13:24:19.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.504+0000 7fe7693c7640 1 -- 172.21.15.12:0/2783851062 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe76410a490 con 0x7fe764102190 2024-04-22T13:24:19.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.504+0000 7fe760ff9640 1 -- 172.21.15.12:0/2783851062 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7fe75404d020 con 0x7fe764102190 2024-04-22T13:24:19.502 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-22T13:24:19.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.506+0000 7fe7693c7640 1 -- 172.21.15.12:0/2783851062 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe7500a3ec0 msgr2=0x7fe7500a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:19.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.506+0000 7fe7693c7640 1 --2- 172.21.15.12:0/2783851062 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe7500a3ec0 0x7fe7500a6380 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe7640fdeb0 tx=0x7fe74c005f90 comp rx=0 tx=0).stop 2024-04-22T13:24:19.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.506+0000 7fe7693c7640 1 -- 172.21.15.12:0/2783851062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe764102190 msgr2=0x7fe76419e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:19.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.506+0000 7fe7693c7640 1 --2- 172.21.15.12:0/2783851062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe764102190 0x7fe76419e410 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe754005940 tx=0x7fe75402c5f0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.506+0000 7fe7693c7640 1 -- 172.21.15.12:0/2783851062 shutdown_connections 2024-04-22T13:24:19.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.506+0000 7fe7693c7640 1 --2- 172.21.15.12:0/2783851062 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe7500a3ec0 0x7fe7500a6380 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.506+0000 7fe7693c7640 1 --2- 172.21.15.12:0/2783851062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe764198610 0x7fe764198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.506+0000 7fe7693c7640 1 --2- 172.21.15.12:0/2783851062 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe764107260 0x7fe76419e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.506+0000 7fe7693c7640 1 --2- 172.21.15.12:0/2783851062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe764102190 0x7fe76419e410 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.506+0000 7fe7693c7640 1 -- 172.21.15.12:0/2783851062 >> 172.21.15.12:0/2783851062 conn(0x7fe7640fd990 msgr2=0x7fe7640fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:19.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.506+0000 7fe7693c7640 1 -- 172.21.15.12:0/2783851062 shutdown_connections 2024-04-22T13:24:19.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.506+0000 7fe7693c7640 1 -- 172.21.15.12:0/2783851062 wait complete. 2024-04-22T13:24:19.515 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T13:24:19.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.623+0000 7f68f83f4640 1 -- 172.21.15.12:0/1059934814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68f0101a50 msgr2=0x7f68f0105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:19.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.623+0000 7f68f83f4640 1 --2- 172.21.15.12:0/1059934814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68f0101a50 0x7f68f0105a90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f68e0009900 tx=0x7f68e002bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.623+0000 7f68f83f4640 1 -- 172.21.15.12:0/1059934814 shutdown_connections 2024-04-22T13:24:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.623+0000 7f68f83f4640 1 --2- 172.21.15.12:0/1059934814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68f0106170 0x7f68f010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.623+0000 7f68f83f4640 1 --2- 172.21.15.12:0/1059934814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68f0101a50 0x7f68f0105a90 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.623+0000 7f68f83f4640 1 --2- 172.21.15.12:0/1059934814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68f01010a0 0x7f68f0101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.623+0000 7f68f83f4640 1 -- 172.21.15.12:0/1059934814 >> 172.21.15.12:0/1059934814 conn(0x7f68f0077eb0 msgr2=0x7f68f00782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.623+0000 7f68f83f4640 1 -- 172.21.15.12:0/1059934814 shutdown_connections 2024-04-22T13:24:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.623+0000 7f68f83f4640 1 -- 172.21.15.12:0/1059934814 wait complete. 2024-04-22T13:24:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.623+0000 7f68f83f4640 1 Processor -- start 2024-04-22T13:24:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.623+0000 7f68f83f4640 1 -- start start 2024-04-22T13:24:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.624+0000 7f68f83f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68f01010a0 0x7f68f0197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.624+0000 7f68f83f4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68f0106170 0x7f68f0197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:19.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.624+0000 7f68f83f4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68f019bf10 0x7f68f019c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.624+0000 7f68f83f4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f68f01110b0 con 0x7f68f019bf10 2024-04-22T13:24:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.624+0000 7f68f83f4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f68f0110f30 con 0x7f68f0106170 2024-04-22T13:24:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.624+0000 7f68f83f4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f68f0111230 con 0x7f68f01010a0 2024-04-22T13:24:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.624+0000 7f68f6991640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68f019bf10 0x7f68f019c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.624+0000 7f68f6991640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68f019bf10 0x7f68f019c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59578/0 (socket says 172.21.15.12:59578) 2024-04-22T13:24:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.624+0000 7f68f6991640 1 -- 172.21.15.12:0/1895363256 learned_addr learned my addr 172.21.15.12:0/1895363256 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.624+0000 7f68f598f640 1 --2- 172.21.15.12:0/1895363256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68f0106170 0x7f68f0197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.624+0000 7f68f6190640 1 --2- 172.21.15.12:0/1895363256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68f01010a0 0x7f68f0197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.624+0000 7f68f6991640 1 -- 172.21.15.12:0/1895363256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68f01010a0 msgr2=0x7f68f0197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.624+0000 7f68f6991640 1 --2- 172.21.15.12:0/1895363256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68f01010a0 0x7f68f0197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.624+0000 7f68f6991640 1 -- 172.21.15.12:0/1895363256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68f0106170 msgr2=0x7f68f0197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:19.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.624+0000 7f68f6991640 1 --2- 172.21.15.12:0/1895363256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68f0106170 0x7f68f0197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.624+0000 7f68f6991640 1 -- 172.21.15.12:0/1895363256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68f019ca00 con 0x7f68f019bf10 2024-04-22T13:24:19.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.624+0000 7f68f6190640 1 --2- 172.21.15.12:0/1895363256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68f01010a0 0x7f68f0197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:19.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.624+0000 7f68f598f640 1 --2- 172.21.15.12:0/1895363256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68f0106170 0x7f68f0197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:19.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.624+0000 7f68f6991640 1 --2- 172.21.15.12:0/1895363256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68f019bf10 0x7f68f019c3c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f68e800cad0 tx=0x7f68e800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:19.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.624+0000 7f68e77fe640 1 -- 172.21.15.12:0/1895363256 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e800ae20 con 0x7f68f019bf10 2024-04-22T13:24:19.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.624+0000 7f68e77fe640 1 -- 172.21.15.12:0/1895363256 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68e8005020 con 0x7f68f019bf10 2024-04-22T13:24:19.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.624+0000 7f68f83f4640 1 -- 172.21.15.12:0/1895363256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68f019ccf0 con 0x7f68f019bf10 2024-04-22T13:24:19.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.624+0000 7f68e77fe640 1 -- 172.21.15.12:0/1895363256 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e800f6b0 con 0x7f68f019bf10 2024-04-22T13:24:19.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.625+0000 7f68f83f4640 1 -- 172.21.15.12:0/1895363256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68f01a4460 con 0x7f68f019bf10 2024-04-22T13:24:19.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.625+0000 7f68f83f4640 1 -- 172.21.15.12:0/1895363256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68f00691c0 con 0x7f68f019bf10 2024-04-22T13:24:19.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.627+0000 7f68e77fe640 1 -- 172.21.15.12:0/1895363256 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f68e80051d0 con 0x7f68f019bf10 2024-04-22T13:24:19.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.627+0000 7f68e77fe640 1 --2- 172.21.15.12:0/1895363256 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f68dc0a3e40 0x7f68dc0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:19.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.627+0000 7f68f6190640 1 --2- 172.21.15.12:0/1895363256 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f68dc0a3e40 0x7f68dc0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:19.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.627+0000 7f68e77fe640 1 -- 172.21.15.12:0/1895363256 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f68e80d0ec0 con 0x7f68f019bf10 2024-04-22T13:24:19.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.628+0000 7f68f6190640 1 --2- 172.21.15.12:0/1895363256 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f68dc0a3e40 0x7f68dc0a6300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009b40 tx=0x7f68ec005e50 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:19.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.631+0000 7f68e77fe640 1 -- 172.21.15.12:0/1895363256 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f68e8014030 con 0x7f68f019bf10 2024-04-22T13:24:19.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.817+0000 7f68f83f4640 1 -- 172.21.15.12:0/1895363256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f68f0101a50 con 0x7f68f019bf10 2024-04-22T13:24:19.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.818+0000 7f68e77fe640 1 -- 172.21.15.12:0/1895363256 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f68e80bbe20 con 0x7f68f019bf10 2024-04-22T13:24:19.815 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-22T13:24:19.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.820+0000 7f68f83f4640 1 -- 172.21.15.12:0/1895363256 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f68dc0a3e40 msgr2=0x7f68dc0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:19.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.820+0000 7f68f83f4640 1 --2- 172.21.15.12:0/1895363256 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f68dc0a3e40 0x7f68dc0a6300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009b40 tx=0x7f68ec005e50 comp rx=0 tx=0).stop 2024-04-22T13:24:19.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.820+0000 7f68f83f4640 1 -- 172.21.15.12:0/1895363256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68f019bf10 msgr2=0x7f68f019c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:19.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.820+0000 7f68f83f4640 1 --2- 172.21.15.12:0/1895363256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68f019bf10 0x7f68f019c3c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f68e800cad0 tx=0x7f68e800cf90 comp rx=0 tx=0).stop 2024-04-22T13:24:19.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.820+0000 7f68f83f4640 1 -- 172.21.15.12:0/1895363256 shutdown_connections 2024-04-22T13:24:19.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.820+0000 7f68f83f4640 1 --2- 172.21.15.12:0/1895363256 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f68dc0a3e40 0x7f68dc0a6300 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.820+0000 7f68f83f4640 1 --2- 172.21.15.12:0/1895363256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68f019bf10 0x7f68f019c3c0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.820+0000 7f68f83f4640 1 --2- 172.21.15.12:0/1895363256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68f0106170 0x7f68f0197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.820+0000 7f68f83f4640 1 --2- 172.21.15.12:0/1895363256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68f01010a0 0x7f68f0197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.820+0000 7f68f83f4640 1 -- 172.21.15.12:0/1895363256 >> 172.21.15.12:0/1895363256 conn(0x7f68f0077eb0 msgr2=0x7f68f00ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.820+0000 7f68f83f4640 1 -- 172.21.15.12:0/1895363256 shutdown_connections 2024-04-22T13:24:19.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.820+0000 7f68f83f4640 1 -- 172.21.15.12:0/1895363256 wait complete. 2024-04-22T13:24:19.827 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:19 smithi012 ceph-mon[28265]: from='client.15663 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:24:19.827 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:19 smithi012 ceph-mon[28265]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 56 op/s 2024-04-22T13:24:19.827 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:19 smithi012 ceph-mon[28265]: from='client.15669 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:24:19.827 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:19 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2001777862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:19.827 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:19 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/977482418' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:24:19.827 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:19 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2783851062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:24:19.828 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T13:24:19.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.936+0000 7f0998e8c640 1 -- 172.21.15.12:0/2881029526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0994106f30 msgr2=0x7f0994107310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:19.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.936+0000 7f0998e8c640 1 --2- 172.21.15.12:0/2881029526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0994106f30 0x7f0994107310 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0984009930 tx=0x7f098402bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:19.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.936+0000 7f0998e8c640 1 -- 172.21.15.12:0/2881029526 shutdown_connections 2024-04-22T13:24:19.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.936+0000 7f0998e8c640 1 --2- 172.21.15.12:0/2881029526 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0994100e10 0x7f099410ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.937+0000 7f0998e8c640 1 --2- 172.21.15.12:0/2881029526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0994100450 0x7f09941008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.937+0000 7f0998e8c640 1 --2- 172.21.15.12:0/2881029526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0994106f30 0x7f0994107310 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.937+0000 7f0998e8c640 1 -- 172.21.15.12:0/2881029526 >> 172.21.15.12:0/2881029526 conn(0x7f0994077410 msgr2=0x7f09940fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:19.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.937+0000 7f0998e8c640 1 -- 172.21.15.12:0/2881029526 shutdown_connections 2024-04-22T13:24:19.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.937+0000 7f0998e8c640 1 -- 172.21.15.12:0/2881029526 wait complete. 2024-04-22T13:24:19.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.937+0000 7f0998e8c640 1 Processor -- start 2024-04-22T13:24:19.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.937+0000 7f0998e8c640 1 -- start start 2024-04-22T13:24:19.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.937+0000 7f0998e8c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0994100450 0x7f099419e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:19.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.937+0000 7f0998e8c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0994100e10 0x7f099419eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:19.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.937+0000 7f0998e8c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0994106f30 0x7f0994198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:19.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.937+0000 7f0998e8c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f09941122e0 con 0x7f0994106f30 2024-04-22T13:24:19.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.937+0000 7f0998e8c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0994112160 con 0x7f0994100450 2024-04-22T13:24:19.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.937+0000 7f0998e8c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0994112460 con 0x7f0994100e10 2024-04-22T13:24:19.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.938+0000 7f0992d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0994106f30 0x7f0994198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:19.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.938+0000 7f0992d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0994106f30 0x7f0994198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59598/0 (socket says 172.21.15.12:59598) 2024-04-22T13:24:19.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.938+0000 7f0992d9d640 1 -- 172.21.15.12:0/2532677773 learned_addr learned my addr 172.21.15.12:0/2532677773 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:19.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.938+0000 7f099259c640 1 --2- 172.21.15.12:0/2532677773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0994100450 0x7f099419e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:19.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.938+0000 7f0991d9b640 1 --2- 172.21.15.12:0/2532677773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0994100e10 0x7f099419eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:19.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.938+0000 7f0992d9d640 1 -- 172.21.15.12:0/2532677773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0994100e10 msgr2=0x7f099419eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:19.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.938+0000 7f0992d9d640 1 --2- 172.21.15.12:0/2532677773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0994100e10 0x7f099419eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.938+0000 7f0992d9d640 1 -- 172.21.15.12:0/2532677773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0994100450 msgr2=0x7f099419e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:19.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.938+0000 7f0992d9d640 1 --2- 172.21.15.12:0/2532677773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0994100450 0x7f099419e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:19.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.938+0000 7f0992d9d640 1 -- 172.21.15.12:0/2532677773 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0994198f00 con 0x7f0994106f30 2024-04-22T13:24:19.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.938+0000 7f0992d9d640 1 --2- 172.21.15.12:0/2532677773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0994106f30 0x7f0994198640 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f098c00eaa0 tx=0x7f098c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:19.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.938+0000 7f09837fe640 1 -- 172.21.15.12:0/2532677773 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c00cdb0 con 0x7f0994106f30 2024-04-22T13:24:19.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.938+0000 7f0998e8c640 1 -- 172.21.15.12:0/2532677773 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09941991f0 con 0x7f0994106f30 2024-04-22T13:24:19.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.938+0000 7f09837fe640 1 -- 172.21.15.12:0/2532677773 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f098c005020 con 0x7f0994106f30 2024-04-22T13:24:19.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.938+0000 7f0998e8c640 1 -- 172.21.15.12:0/2532677773 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09941a5500 con 0x7f0994106f30 2024-04-22T13:24:19.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.938+0000 7f09837fe640 1 -- 172.21.15.12:0/2532677773 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c0116d0 con 0x7f0994106f30 2024-04-22T13:24:19.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.939+0000 7f0998e8c640 1 -- 172.21.15.12:0/2532677773 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0994103770 con 0x7f0994106f30 2024-04-22T13:24:19.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.941+0000 7f09837fe640 1 -- 172.21.15.12:0/2532677773 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f098c0054a0 con 0x7f0994106f30 2024-04-22T13:24:19.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.941+0000 7f09837fe640 1 --2- 172.21.15.12:0/2532677773 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f097c0a3e40 0x7f097c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:19.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.941+0000 7f09837fe640 1 -- 172.21.15.12:0/2532677773 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f098c0d0f60 con 0x7f0994106f30 2024-04-22T13:24:19.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.941+0000 7f099259c640 1 --2- 172.21.15.12:0/2532677773 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f097c0a3e40 0x7f097c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:19.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.942+0000 7f099259c640 1 --2- 172.21.15.12:0/2532677773 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f097c0a3e40 0x7f097c0a6300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0984009900 tx=0x7f0984005a20 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:19.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:19.945+0000 7f09837fe640 1 -- 172.21.15.12:0/2532677773 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f098c016030 con 0x7f0994106f30 2024-04-22T13:24:19.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:19 smithi111 ceph-mon[31361]: from='client.15663 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:24:19.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:19 smithi111 ceph-mon[31361]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 56 op/s 2024-04-22T13:24:19.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:19 smithi111 ceph-mon[31361]: from='client.15669 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:24:19.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:19 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2001777862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:19.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:19 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/977482418' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:24:19.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:19 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2783851062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:24:20.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.131+0000 7f0998e8c640 1 -- 172.21.15.12:0/2532677773 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f099410a410 con 0x7f0994106f30 2024-04-22T13:24:20.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.131+0000 7f09837fe640 1 -- 172.21.15.12:0/2532677773 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f098c09cdb0 con 0x7f0994106f30 2024-04-22T13:24:20.129 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-22T13:24:20.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.133+0000 7f0998e8c640 1 -- 172.21.15.12:0/2532677773 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f097c0a3e40 msgr2=0x7f097c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:20.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.133+0000 7f0998e8c640 1 --2- 172.21.15.12:0/2532677773 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f097c0a3e40 0x7f097c0a6300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0984009900 tx=0x7f0984005a20 comp rx=0 tx=0).stop 2024-04-22T13:24:20.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.133+0000 7f0998e8c640 1 -- 172.21.15.12:0/2532677773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0994106f30 msgr2=0x7f0994198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:20.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.133+0000 7f0998e8c640 1 --2- 172.21.15.12:0/2532677773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0994106f30 0x7f0994198640 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f098c00eaa0 tx=0x7f098c00ef60 comp rx=0 tx=0).stop 2024-04-22T13:24:20.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.133+0000 7f0998e8c640 1 -- 172.21.15.12:0/2532677773 shutdown_connections 2024-04-22T13:24:20.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.133+0000 7f0998e8c640 1 --2- 172.21.15.12:0/2532677773 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f097c0a3e40 0x7f097c0a6300 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:20.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.133+0000 7f0998e8c640 1 --2- 172.21.15.12:0/2532677773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0994106f30 0x7f0994198640 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:20.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.133+0000 7f0998e8c640 1 --2- 172.21.15.12:0/2532677773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0994100e10 0x7f099419eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:20.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.133+0000 7f0998e8c640 1 --2- 172.21.15.12:0/2532677773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0994100450 0x7f099419e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:20.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.133+0000 7f0998e8c640 1 -- 172.21.15.12:0/2532677773 >> 172.21.15.12:0/2532677773 conn(0x7f0994077410 msgr2=0x7f09941051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:20.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.133+0000 7f0998e8c640 1 -- 172.21.15.12:0/2532677773 shutdown_connections 2024-04-22T13:24:20.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.133+0000 7f0998e8c640 1 -- 172.21.15.12:0/2532677773 wait complete. 2024-04-22T13:24:20.141 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:24:20.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:19 smithi134 ceph-mon[31338]: from='client.15663 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:24:20.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:19 smithi134 ceph-mon[31338]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 56 op/s 2024-04-22T13:24:20.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:19 smithi134 ceph-mon[31338]: from='client.15669 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-22T13:24:20.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:19 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2001777862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:20.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:19 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/977482418' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:24:20.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:19 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2783851062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:24:20.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.247+0000 7faadeb1e640 1 -- 172.21.15.12:0/3397783501 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faad8069aa0 msgr2=0x7faad810edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:20.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.247+0000 7faadeb1e640 1 --2- 172.21.15.12:0/3397783501 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faad8069aa0 0x7faad810edd0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7faad0009cd0 tx=0x7faad002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:24:20.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.247+0000 7faadeb1e640 1 -- 172.21.15.12:0/3397783501 shutdown_connections 2024-04-22T13:24:20.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.247+0000 7faadeb1e640 1 --2- 172.21.15.12:0/3397783501 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faad8069aa0 0x7faad810edd0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:20.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.247+0000 7faadeb1e640 1 --2- 172.21.15.12:0/3397783501 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faad80690e0 0x7faad8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:20.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.247+0000 7faadeb1e640 1 --2- 172.21.15.12:0/3397783501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faad8108030 0x7faad8108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:20.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.247+0000 7faadeb1e640 1 -- 172.21.15.12:0/3397783501 >> 172.21.15.12:0/3397783501 conn(0x7faad8077e30 msgr2=0x7faad8078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:20.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.247+0000 7faadeb1e640 1 -- 172.21.15.12:0/3397783501 shutdown_connections 2024-04-22T13:24:20.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.247+0000 7faadeb1e640 1 -- 172.21.15.12:0/3397783501 wait complete. 2024-04-22T13:24:20.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.247+0000 7faadeb1e640 1 Processor -- start 2024-04-22T13:24:20.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.247+0000 7faadeb1e640 1 -- start start 2024-04-22T13:24:20.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.247+0000 7faadeb1e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faad80690e0 0x7faad81986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:20.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.247+0000 7faadeb1e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faad8069aa0 0x7faad8198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:20.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.247+0000 7faadeb1e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faad8108030 0x7faad819cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:20.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.247+0000 7faadeb1e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7faad8112160 con 0x7faad80690e0 2024-04-22T13:24:20.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.247+0000 7faadeb1e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faad8111fe0 con 0x7faad8069aa0 2024-04-22T13:24:20.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.247+0000 7faadeb1e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faad81122e0 con 0x7faad8108030 2024-04-22T13:24:20.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.248+0000 7faadc8ba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faad80690e0 0x7faad81986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:20.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.248+0000 7faadc8ba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faad80690e0 0x7faad81986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59606/0 (socket says 172.21.15.12:59606) 2024-04-22T13:24:20.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.248+0000 7faadc8ba640 1 -- 172.21.15.12:0/1000769866 learned_addr learned my addr 172.21.15.12:0/1000769866 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:20.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.248+0000 7faad7fff640 1 --2- 172.21.15.12:0/1000769866 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faad8069aa0 0x7faad8198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:20.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.248+0000 7faadd0bb640 1 --2- 172.21.15.12:0/1000769866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faad8108030 0x7faad819cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:20.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.248+0000 7faadc8ba640 1 -- 172.21.15.12:0/1000769866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faad8108030 msgr2=0x7faad819cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:20.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.248+0000 7faadc8ba640 1 --2- 172.21.15.12:0/1000769866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faad8108030 0x7faad819cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:20.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.248+0000 7faadc8ba640 1 -- 172.21.15.12:0/1000769866 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faad8069aa0 msgr2=0x7faad8198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:20.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.248+0000 7faadc8ba640 1 --2- 172.21.15.12:0/1000769866 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faad8069aa0 0x7faad8198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:20.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.248+0000 7faadc8ba640 1 -- 172.21.15.12:0/1000769866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad819d690 con 0x7faad80690e0 2024-04-22T13:24:20.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.248+0000 7faadc8ba640 1 --2- 172.21.15.12:0/1000769866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faad80690e0 0x7faad81986a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7faacc00e9f0 tx=0x7faacc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:20.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.248+0000 7faad5ffb640 1 -- 172.21.15.12:0/1000769866 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc00cd80 con 0x7faad80690e0 2024-04-22T13:24:20.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.248+0000 7faadeb1e640 1 -- 172.21.15.12:0/1000769866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faad819d980 con 0x7faad80690e0 2024-04-22T13:24:20.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.248+0000 7faad5ffb640 1 -- 172.21.15.12:0/1000769866 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faacc0054d0 con 0x7faad80690e0 2024-04-22T13:24:20.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.248+0000 7faad5ffb640 1 -- 172.21.15.12:0/1000769866 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc010690 con 0x7faad80690e0 2024-04-22T13:24:20.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.249+0000 7faadeb1e640 1 -- 172.21.15.12:0/1000769866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faad81a5210 con 0x7faad80690e0 2024-04-22T13:24:20.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.250+0000 7faadeb1e640 1 -- 172.21.15.12:0/1000769866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faad8068de0 con 0x7faad80690e0 2024-04-22T13:24:20.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.251+0000 7faad5ffb640 1 -- 172.21.15.12:0/1000769866 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7faacc004e90 con 0x7faad80690e0 2024-04-22T13:24:20.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.251+0000 7faad5ffb640 1 --2- 172.21.15.12:0/1000769866 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7faac00a3e40 0x7faac00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:20.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.251+0000 7faad5ffb640 1 -- 172.21.15.12:0/1000769866 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7faacc0d0e50 con 0x7faad80690e0 2024-04-22T13:24:20.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.251+0000 7faad7fff640 1 --2- 172.21.15.12:0/1000769866 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7faac00a3e40 0x7faac00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:20.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.252+0000 7faad7fff640 1 --2- 172.21.15.12:0/1000769866 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7faac00a3e40 0x7faac00a6300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faad8107750 tx=0x7faac8005f90 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:20.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.256+0000 7faad5ffb640 1 -- 172.21.15.12:0/1000769866 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faacc09d2f0 con 0x7faad80690e0 2024-04-22T13:24:20.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.483+0000 7faadeb1e640 1 -- 172.21.15.12:0/1000769866 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faad81999f0 con 0x7faad80690e0 2024-04-22T13:24:20.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.483+0000 7faad5ffb640 1 -- 172.21.15.12:0/1000769866 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7faacc09cbf0 con 0x7faad80690e0 2024-04-22T13:24:20.481 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:24:20.481 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:24:20.481 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:24:20.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.485+0000 7faadeb1e640 1 -- 172.21.15.12:0/1000769866 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7faac00a3e40 msgr2=0x7faac00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:20.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.485+0000 7faadeb1e640 1 --2- 172.21.15.12:0/1000769866 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7faac00a3e40 0x7faac00a6300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faad8107750 tx=0x7faac8005f90 comp rx=0 tx=0).stop 2024-04-22T13:24:20.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.485+0000 7faadeb1e640 1 -- 172.21.15.12:0/1000769866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faad80690e0 msgr2=0x7faad81986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:20.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.485+0000 7faadeb1e640 1 --2- 172.21.15.12:0/1000769866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faad80690e0 0x7faad81986a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7faacc00e9f0 tx=0x7faacc00edf0 comp rx=0 tx=0).stop 2024-04-22T13:24:20.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.485+0000 7faadeb1e640 1 -- 172.21.15.12:0/1000769866 shutdown_connections 2024-04-22T13:24:20.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.485+0000 7faadeb1e640 1 --2- 172.21.15.12:0/1000769866 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7faac00a3e40 0x7faac00a6300 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:20.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.485+0000 7faadeb1e640 1 --2- 172.21.15.12:0/1000769866 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faad8108030 0x7faad819cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:20.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.485+0000 7faadeb1e640 1 --2- 172.21.15.12:0/1000769866 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faad8069aa0 0x7faad8198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:20.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.485+0000 7faadeb1e640 1 --2- 172.21.15.12:0/1000769866 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faad80690e0 0x7faad81986a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:20.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.485+0000 7faadeb1e640 1 -- 172.21.15.12:0/1000769866 >> 172.21.15.12:0/1000769866 conn(0x7faad8077e30 msgr2=0x7faad80fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:20.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.486+0000 7faadeb1e640 1 -- 172.21.15.12:0/1000769866 shutdown_connections 2024-04-22T13:24:20.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.486+0000 7faadeb1e640 1 -- 172.21.15.12:0/1000769866 wait complete. 2024-04-22T13:24:20.494 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:24:20.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.599+0000 7fd175274640 1 -- 172.21.15.12:0/29116695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd170101ad0 msgr2=0x7fd17010dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:20.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.599+0000 7fd175274640 1 --2- 172.21.15.12:0/29116695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd170101ad0 0x7fd17010dcb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd160009900 tx=0x7fd16002bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:20.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.599+0000 7fd175274640 1 -- 172.21.15.12:0/29116695 shutdown_connections 2024-04-22T13:24:20.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.599+0000 7fd175274640 1 --2- 172.21.15.12:0/29116695 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd170101ad0 0x7fd17010dcb0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:20.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.599+0000 7fd175274640 1 --2- 172.21.15.12:0/29116695 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd170101130 0x7fd170101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:20.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.599+0000 7fd175274640 1 --2- 172.21.15.12:0/29116695 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd170106f10 0x7fd1701072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:20.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.599+0000 7fd175274640 1 -- 172.21.15.12:0/29116695 >> 172.21.15.12:0/29116695 conn(0x7fd170077e30 msgr2=0x7fd170078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:20.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.599+0000 7fd175274640 1 -- 172.21.15.12:0/29116695 shutdown_connections 2024-04-22T13:24:20.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.599+0000 7fd175274640 1 -- 172.21.15.12:0/29116695 wait complete. 2024-04-22T13:24:20.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.600+0000 7fd175274640 1 Processor -- start 2024-04-22T13:24:20.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.600+0000 7fd175274640 1 -- start start 2024-04-22T13:24:20.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.600+0000 7fd175274640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd170101130 0x7fd170197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:20.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.600+0000 7fd175274640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd170101ad0 0x7fd170197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:20.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.600+0000 7fd175274640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd170106f10 0x7fd17019bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:20.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.600+0000 7fd175274640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd1701110d0 con 0x7fd170101ad0 2024-04-22T13:24:20.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.600+0000 7fd175274640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd170110f50 con 0x7fd170106f10 2024-04-22T13:24:20.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.600+0000 7fd175274640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd170111250 con 0x7fd170101130 2024-04-22T13:24:20.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.600+0000 7fd16e7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd170101ad0 0x7fd170197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:20.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.600+0000 7fd16e7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd170101ad0 0x7fd170197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59614/0 (socket says 172.21.15.12:59614) 2024-04-22T13:24:20.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.600+0000 7fd16e7fc640 1 -- 172.21.15.12:0/2042523652 learned_addr learned my addr 172.21.15.12:0/2042523652 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:20.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.600+0000 7fd16f7fe640 1 --2- 172.21.15.12:0/2042523652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd170106f10 0x7fd17019bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:20.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.600+0000 7fd16effd640 1 --2- 172.21.15.12:0/2042523652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd170101130 0x7fd170197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:20.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.600+0000 7fd16e7fc640 1 -- 172.21.15.12:0/2042523652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd170101130 msgr2=0x7fd170197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:20.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.600+0000 7fd16e7fc640 1 --2- 172.21.15.12:0/2042523652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd170101130 0x7fd170197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:20.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.600+0000 7fd16e7fc640 1 -- 172.21.15.12:0/2042523652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd170106f10 msgr2=0x7fd17019bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:20.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.600+0000 7fd16e7fc640 1 --2- 172.21.15.12:0/2042523652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd170106f10 0x7fd17019bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:20.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.600+0000 7fd16e7fc640 1 -- 172.21.15.12:0/2042523652 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17019c620 con 0x7fd170101ad0 2024-04-22T13:24:20.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.601+0000 7fd16e7fc640 1 --2- 172.21.15.12:0/2042523652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd170101ad0 0x7fd170197b70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd15800c920 tx=0x7fd15800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:20.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.601+0000 7fd167fff640 1 -- 172.21.15.12:0/2042523652 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd158007c70 con 0x7fd170101ad0 2024-04-22T13:24:20.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.601+0000 7fd175274640 1 -- 172.21.15.12:0/2042523652 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd17019c910 con 0x7fd170101ad0 2024-04-22T13:24:20.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.601+0000 7fd167fff640 1 -- 172.21.15.12:0/2042523652 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd158007e10 con 0x7fd170101ad0 2024-04-22T13:24:20.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.601+0000 7fd167fff640 1 -- 172.21.15.12:0/2042523652 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd158018820 con 0x7fd170101ad0 2024-04-22T13:24:20.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.601+0000 7fd175274640 1 -- 172.21.15.12:0/2042523652 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1701a41a0 con 0x7fd170101ad0 2024-04-22T13:24:20.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.602+0000 7fd175274640 1 -- 172.21.15.12:0/2042523652 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd170069230 con 0x7fd170101ad0 2024-04-22T13:24:20.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.603+0000 7fd167fff640 1 -- 172.21.15.12:0/2042523652 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fd15800f450 con 0x7fd170101ad0 2024-04-22T13:24:20.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.603+0000 7fd167fff640 1 --2- 172.21.15.12:0/2042523652 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd15c0a3e40 0x7fd15c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:20.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.604+0000 7fd16effd640 1 --2- 172.21.15.12:0/2042523652 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd15c0a3e40 0x7fd15c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:20.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.604+0000 7fd167fff640 1 -- 172.21.15.12:0/2042523652 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fd1580d0b90 con 0x7fd170101ad0 2024-04-22T13:24:20.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.604+0000 7fd16effd640 1 --2- 172.21.15.12:0/2042523652 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd15c0a3e40 0x7fd15c0a6300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd168005fd0 tx=0x7fd168009500 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:20.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.608+0000 7fd167fff640 1 -- 172.21.15.12:0/2042523652 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd158014030 con 0x7fd170101ad0 2024-04-22T13:24:20.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.798+0000 7fd175274640 1 -- 172.21.15.12:0/2042523652 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd170109370 con 0x7fd170101ad0 2024-04-22T13:24:20.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.800+0000 7fd167fff640 1 -- 172.21.15.12:0/2042523652 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22997 (secure 0 0 0) 0x7fd15809d030 con 0x7fd170101ad0 2024-04-22T13:24:20.797 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:24:20.798 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":90,"fsid":"0aede5ea-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:14:09.362918+0000","modified":"2024-04-22T13:24:06.155595+0000","last_up_change":"2024-04-22T13:21:15.263393+0000","last_in_change":"2024-04-22T13:20:54.174685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:17:32.372434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:22:02.768799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:22:03.578730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d9e2365-3a64-4b13-96fc-c217b1cee2ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6803","nonce":1191350632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6805","nonce":1191350632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6809","nonce":1191350632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6807","nonce":1191350632}]},"public_addr":"172.21.15.12:6803/1191350632","cluster_addr":"172.21.15.12:6805/1191350632","heartbeat_back_addr":"172.21.15.12:6809/1191350632","heartbeat_front_addr":"172.21.15.12:6807/1191350632","state":["exists","up"]},{"osd":1,"uuid":"1499c783-2bd7-48ce-a444-34add42bcf31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6801","nonce":417496465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6803","nonce":417496465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6807","nonce":417496465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6805","nonce":417496465}]},"public_addr":"172.21.15.111:6801/417496465","cluster_addr":"172.21.15.111:6803/417496465","heartbeat_back_addr":"172.21.15.111:6807/417496465","heartbeat_front_addr":"172.21.15.111:6805/417496465","state":["exists","up"]},{"osd":2,"uuid":"71a9ebc3-c622-434e-a6c2-f48358f384f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6801","nonce":1403153998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6803","nonce":1403153998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6807","nonce":1403153998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6805","nonce":1403153998}]},"public_addr":"172.21.15.134:6801/1403153998","cluster_addr":"172.21.15.134:6803/1403153998","heartbeat_back_addr":"172.21.15.134:6807/1403153998","heartbeat_front_addr":"172.21.15.134:6805/1403153998","state":["exists","up"]},{"osd":3,"uuid":"6dd00db2-a286-48ba-8666-56ffdd5f35ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6811","nonce":3359066681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6813","nonce":3359066681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6817","nonce":3359066681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6815","nonce":3359066681}]},"public_addr":"172.21.15.12:6811/3359066681","cluster_addr":"172.21.15.12:6813/3359066681","heartbeat_back_addr":"172.21.15.12:6817/3359066681","heartbeat_front_addr":"172.21.15.12:6815/3359066681","state":["exists","up"]},{"osd":4,"uuid":"1e29afd7-354c-4506-aaa0-b43eb1a19a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6809","nonce":2886349872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6811","nonce":2886349872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6815","nonce":2886349872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6813","nonce":2886349872}]},"public_addr":"172.21.15.111:6809/2886349872","cluster_addr":"172.21.15.111:6811/2886349872","heartbeat_back_addr":"172.21.15.111:6815/2886349872","heartbeat_front_addr":"172.21.15.111:6813/2886349872","state":["exists","up"]},{"osd":5,"uuid":"8f40f02d-0ba2-4842-a03d-2a1a04d1010d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6809","nonce":3071977602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6811","nonce":3071977602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6815","nonce":3071977602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6813","nonce":3071977602}]},"public_addr":"172.21.15.134:6809/3071977602","cluster_addr":"172.21.15.134:6811/3071977602","heartbeat_back_addr":"172.21.15.134:6815/3071977602","heartbeat_front_addr":"172.21.15.134:6813/3071977602","state":["exists","up"]},{"osd":6,"uuid":"bb0c17a9-6a8e-4be6-8660-f73cfa8200ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6819","nonce":3819619299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6821","nonce":3819619299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6825","nonce":3819619299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6823","nonce":3819619299}]},"public_addr":"172.21.15.12:6819/3819619299","cluster_addr":"172.21.15.12:6821/3819619299","heartbeat_back_addr":"172.21.15.12:6825/3819619299","heartbeat_front_addr":"172.21.15.12:6823/3819619299","state":["exists","up"]},{"osd":7,"uuid":"0a93d6af-569e-4e45-b697-9e978c1c1004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6817","nonce":3113856282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6819","nonce":3113856282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6823","nonce":3113856282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6821","nonce":3113856282}]},"public_addr":"172.21.15.111:6817/3113856282","cluster_addr":"172.21.15.111:6819/3113856282","heartbeat_back_addr":"172.21.15.111:6823/3113856282","heartbeat_front_addr":"172.21.15.111:6821/3113856282","state":["exists","up"]},{"osd":8,"uuid":"e21acc68-cf50-4f4f-b1ba-cf6c195b4955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6817","nonce":1689362191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6819","nonce":1689362191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6823","nonce":1689362191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6821","nonce":1689362191}]},"public_addr":"172.21.15.134:6817/1689362191","cluster_addr":"172.21.15.134:6819/1689362191","heartbeat_back_addr":"172.21.15.134:6823/1689362191","heartbeat_front_addr":"172.21.15.134:6821/1689362191","state":["exists","up"]},{"osd":9,"uuid":"fd745869-7f5f-48a8-b412-bb9c92b586e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6827","nonce":4292385779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6829","nonce":4292385779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6833","nonce":4292385779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6831","nonce":4292385779}]},"public_addr":"172.21.15.12:6827/4292385779","cluster_addr":"172.21.15.12:6829/4292385779","heartbeat_back_addr":"172.21.15.12:6833/4292385779","heartbeat_front_addr":"172.21.15.12:6831/4292385779","state":["exists","up"]},{"osd":10,"uuid":"798b1abd-f6c7-4a5f-8da0-fdae49e50d2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6825","nonce":3338566976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6827","nonce":3338566976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6831","nonce":3338566976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6829","nonce":3338566976}]},"public_addr":"172.21.15.111:6825/3338566976","cluster_addr":"172.21.15.111:6827/3338566976","heartbeat_back_addr":"172.21.15.111:6831/3338566976","heartbeat_front_addr":"172.21.15.111:6829/3338566976","state":["exists","up"]},{"osd":11,"uuid":"b18fc5f3-1049-4871-b060-e99290273cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6825","nonce":578640428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6827","nonce":578640428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6831","nonce":578640428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6829","nonce":578640428}]},"public_addr":"172.21.15.134:6825/578640428","cluster_addr":"172.21.15.134:6827/578640428","heartbeat_back_addr":"172.21.15.134:6831/578640428","heartbeat_front_addr":"172.21.15.134:6829/578640428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:16:40.828608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:05.052528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:28.297316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:53.496497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:17.947259+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:41.712894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:07.105945+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:32.129688+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:56.008516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:22.239867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:47.334692+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:12.301276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/3535299028":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/2837479598":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/3010129696":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/2593601362":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/2519382069":"2024-04-23T13:15:04.279990+0000","172.21.15.12:6801/2009831940":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/856202072":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/3509412187":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/1802873236":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/1007749062":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3588480095":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1904962176":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/1802873236":"2024-04-23T13:24:06.155564+0000","172.21.15.12:6800/2009831940":"2024-04-23T13:15:04.279990+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:24:20.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.802+0000 7fd175274640 1 -- 172.21.15.12:0/2042523652 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd15c0a3e40 msgr2=0x7fd15c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:20.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.802+0000 7fd175274640 1 --2- 172.21.15.12:0/2042523652 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd15c0a3e40 0x7fd15c0a6300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd168005fd0 tx=0x7fd168009500 comp rx=0 tx=0).stop 2024-04-22T13:24:20.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.802+0000 7fd175274640 1 -- 172.21.15.12:0/2042523652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd170101ad0 msgr2=0x7fd170197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:20.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.802+0000 7fd175274640 1 --2- 172.21.15.12:0/2042523652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd170101ad0 0x7fd170197b70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd15800c920 tx=0x7fd15800cde0 comp rx=0 tx=0).stop 2024-04-22T13:24:20.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.802+0000 7fd175274640 1 -- 172.21.15.12:0/2042523652 shutdown_connections 2024-04-22T13:24:20.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.802+0000 7fd175274640 1 --2- 172.21.15.12:0/2042523652 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd15c0a3e40 0x7fd15c0a6300 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:20.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.802+0000 7fd175274640 1 --2- 172.21.15.12:0/2042523652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd170106f10 0x7fd17019bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:20.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.802+0000 7fd175274640 1 --2- 172.21.15.12:0/2042523652 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd170101ad0 0x7fd170197b70 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:20.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.802+0000 7fd175274640 1 --2- 172.21.15.12:0/2042523652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd170101130 0x7fd170197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:20.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.802+0000 7fd175274640 1 -- 172.21.15.12:0/2042523652 >> 172.21.15.12:0/2042523652 conn(0x7fd170077e30 msgr2=0x7fd1700ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:20.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.802+0000 7fd175274640 1 -- 172.21.15.12:0/2042523652 shutdown_connections 2024-04-22T13:24:20.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.802+0000 7fd175274640 1 -- 172.21.15.12:0/2042523652 wait complete. 2024-04-22T13:24:20.809 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:20 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1895363256' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:24:20.809 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:20 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2532677773' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:24:20.809 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:20 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1000769866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:20.811 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with random seed 616281 2024-04-22T13:24:20.811 DEBUG:tasks.quiescer:Yielding 2024-04-22T13:24:20.811 INFO:teuthology.run_tasks:Running task workunit... 2024-04-22T13:24:20.826 INFO:tasks.workunit:Pulling workunits from ref d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-22T13:24:20.826 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-22T13:24:20.826 INFO:tasks.workunit:timeout=3h 2024-04-22T13:24:20.826 INFO:tasks.workunit:cleanup=True 2024-04-22T13:24:20.827 DEBUG:teuthology.orchestra.run.smithi012:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-22T13:24:20.827 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:24:20.842 INFO:teuthology.orchestra.run.smithi012.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-22T13:24:20.843 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-22T13:24:20.843 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-22T13:24:20.843 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T13:24:20.843 INFO:teuthology.orchestra.run.smithi012.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-22T13:24:20.843 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-04-22 13:22:12.491588957 +0000 2024-04-22T13:24:20.843 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-04-22 13:22:12.491588957 +0000 2024-04-22T13:24:20.843 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-04-22 13:24:03.888887608 +0000 2024-04-22T13:24:20.843 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: 2024-04-22 13:22:12.491588957 +0000 2024-04-22T13:24:20.843 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-22T13:24:20.843 DEBUG:teuthology.orchestra.run.smithi012:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-22T13:24:20.916 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-22T13:24:20.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.932+0000 7f05f792c640 1 -- 172.21.15.12:0/532986275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05f0106180 msgr2=0x7f05f010dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:20.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.932+0000 7f05f792c640 1 --2- 172.21.15.12:0/532986275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05f0106180 0x7f05f010dd50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f05e0009ca0 tx=0x7f05e002dfe0 comp rx=0 tx=0).stop 2024-04-22T13:24:20.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.933+0000 7f05f792c640 1 -- 172.21.15.12:0/532986275 shutdown_connections 2024-04-22T13:24:20.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.933+0000 7f05f792c640 1 --2- 172.21.15.12:0/532986275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05f0106180 0x7f05f010dd50 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:20.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.933+0000 7f05f792c640 1 --2- 172.21.15.12:0/532986275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05f0101a60 0x7f05f0105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:20.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.933+0000 7f05f792c640 1 --2- 172.21.15.12:0/532986275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05f01010b0 0x7f05f0101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:20.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.933+0000 7f05f792c640 1 -- 172.21.15.12:0/532986275 >> 172.21.15.12:0/532986275 conn(0x7f05f00771d0 msgr2=0x7f05f00775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:20.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.933+0000 7f05f792c640 1 -- 172.21.15.12:0/532986275 shutdown_connections 2024-04-22T13:24:20.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.933+0000 7f05f792c640 1 -- 172.21.15.12:0/532986275 wait complete. 2024-04-22T13:24:20.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.933+0000 7f05f792c640 1 Processor -- start 2024-04-22T13:24:20.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.933+0000 7f05f792c640 1 -- start start 2024-04-22T13:24:20.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.933+0000 7f05f792c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05f01010b0 0x7f05f01964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:20.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.933+0000 7f05f792c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05f0106180 0x7f05f01969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:20.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.933+0000 7f05f792c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05f019ad60 0x7f05f019b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:20.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.933+0000 7f05f792c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f05f010ffe0 con 0x7f05f01010b0 2024-04-22T13:24:20.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.933+0000 7f05f792c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f05f010fe60 con 0x7f05f019ad60 2024-04-22T13:24:20.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.933+0000 7f05f792c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f05f0110160 con 0x7f05f0106180 2024-04-22T13:24:20.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.934+0000 7f05f56c8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05f01010b0 0x7f05f01964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:20.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.934+0000 7f05f56c8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05f01010b0 0x7f05f01964a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59642/0 (socket says 172.21.15.12:59642) 2024-04-22T13:24:20.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.934+0000 7f05f56c8640 1 -- 172.21.15.12:0/244274738 learned_addr learned my addr 172.21.15.12:0/244274738 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:20.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.934+0000 7f05f5ec9640 1 --2- 172.21.15.12:0/244274738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05f019ad60 0x7f05f019b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:20.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.934+0000 7f05f4ec7640 1 --2- 172.21.15.12:0/244274738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05f0106180 0x7f05f01969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:20.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.934+0000 7f05f56c8640 1 -- 172.21.15.12:0/244274738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05f0106180 msgr2=0x7f05f01969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:20.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.934+0000 7f05f56c8640 1 --2- 172.21.15.12:0/244274738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05f0106180 0x7f05f01969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:20.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.934+0000 7f05f56c8640 1 -- 172.21.15.12:0/244274738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05f019ad60 msgr2=0x7f05f019b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:20.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.934+0000 7f05f56c8640 1 --2- 172.21.15.12:0/244274738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05f019ad60 0x7f05f019b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:20.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.934+0000 7f05f56c8640 1 -- 172.21.15.12:0/244274738 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f019b850 con 0x7f05f01010b0 2024-04-22T13:24:20.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.934+0000 7f05f4ec7640 1 --2- 172.21.15.12:0/244274738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05f0106180 0x7f05f01969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:20.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.934+0000 7f05f56c8640 1 --2- 172.21.15.12:0/244274738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05f01010b0 0x7f05f01964a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f05e800e9f0 tx=0x7f05e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:20.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.934+0000 7f05ee7fc640 1 -- 172.21.15.12:0/244274738 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e800cd80 con 0x7f05f01010b0 2024-04-22T13:24:20.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.934+0000 7f05ee7fc640 1 -- 172.21.15.12:0/244274738 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05e80108e0 con 0x7f05f01010b0 2024-04-22T13:24:20.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.934+0000 7f05ee7fc640 1 -- 172.21.15.12:0/244274738 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8018980 con 0x7f05f01010b0 2024-04-22T13:24:20.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.934+0000 7f05f792c640 1 -- 172.21.15.12:0/244274738 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f019bb40 con 0x7f05f01010b0 2024-04-22T13:24:20.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.934+0000 7f05f792c640 1 -- 172.21.15.12:0/244274738 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f01a3330 con 0x7f05f01010b0 2024-04-22T13:24:20.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.935+0000 7f05f792c640 1 -- 172.21.15.12:0/244274738 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05f0069230 con 0x7f05f01010b0 2024-04-22T13:24:20.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.937+0000 7f05ee7fc640 1 -- 172.21.15.12:0/244274738 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f05e8010430 con 0x7f05f01010b0 2024-04-22T13:24:20.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.938+0000 7f05ee7fc640 1 --2- 172.21.15.12:0/244274738 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f05d80a3e40 0x7f05d80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:20.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.938+0000 7f05f4ec7640 1 --2- 172.21.15.12:0/244274738 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f05d80a3e40 0x7f05d80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:20.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.938+0000 7f05ee7fc640 1 -- 172.21.15.12:0/244274738 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f05e80d1350 con 0x7f05f01010b0 2024-04-22T13:24:20.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.938+0000 7f05f4ec7640 1 --2- 172.21.15.12:0/244274738 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f05d80a3e40 0x7f05d80a6300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f05f0100680 tx=0x7f05e4005f90 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:20.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:20.941+0000 7f05ee7fc640 1 -- 172.21.15.12:0/244274738 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f05e809d790 con 0x7f05f01010b0 2024-04-22T13:24:20.946 INFO:tasks.workunit.client.0.smithi012.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-22T13:24:20.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:20 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1895363256' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:24:20.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:20 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2532677773' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:24:20.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:20 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1000769866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:21.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.167+0000 7f05f792c640 1 -- 172.21.15.12:0/244274738 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05f01977f0 con 0x7f05f01010b0 2024-04-22T13:24:21.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.168+0000 7f05ee7fc640 1 -- 172.21.15.12:0/244274738 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7f05e809d090 con 0x7f05f01010b0 2024-04-22T13:24:21.166 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:24:21.166 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:24:21.166 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:24:21.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.170+0000 7f05f792c640 1 -- 172.21.15.12:0/244274738 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f05d80a3e40 msgr2=0x7f05d80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.170+0000 7f05f792c640 1 --2- 172.21.15.12:0/244274738 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f05d80a3e40 0x7f05d80a6300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f05f0100680 tx=0x7f05e4005f90 comp rx=0 tx=0).stop 2024-04-22T13:24:21.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.170+0000 7f05f792c640 1 -- 172.21.15.12:0/244274738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05f01010b0 msgr2=0x7f05f01964a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.170+0000 7f05f792c640 1 --2- 172.21.15.12:0/244274738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05f01010b0 0x7f05f01964a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f05e800e9f0 tx=0x7f05e800eeb0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.170+0000 7f05f792c640 1 -- 172.21.15.12:0/244274738 shutdown_connections 2024-04-22T13:24:21.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.170+0000 7f05f792c640 1 --2- 172.21.15.12:0/244274738 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f05d80a3e40 0x7f05d80a6300 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.170+0000 7f05f792c640 1 --2- 172.21.15.12:0/244274738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f05f019ad60 0x7f05f019b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.170+0000 7f05f792c640 1 --2- 172.21.15.12:0/244274738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05f0106180 0x7f05f01969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.170+0000 7f05f792c640 1 --2- 172.21.15.12:0/244274738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f05f01010b0 0x7f05f01964a0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.170+0000 7f05f792c640 1 -- 172.21.15.12:0/244274738 >> 172.21.15.12:0/244274738 conn(0x7f05f00771d0 msgr2=0x7f05f00fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:21.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.171+0000 7f05f792c640 1 -- 172.21.15.12:0/244274738 shutdown_connections 2024-04-22T13:24:21.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.171+0000 7f05f792c640 1 -- 172.21.15.12:0/244274738 wait complete. 2024-04-22T13:24:21.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:20 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1895363256' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:24:21.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:20 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2532677773' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:24:21.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:20 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1000769866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:21.180 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-22T13:22:04.602553+0000', 'modified': '2024-04-22T13:24:04.600025+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24509, 'mds_1': 24484, 'mds_2': 24448, 'mds_3': 24443, 'mds_4': 24407}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14631': {'gid': 14631, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.12:6839/4083188176', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6838', 'nonce': 4083188176}, {'type': 'v1', 'addr': '172.21.15.12:6839', 'nonce': 4083188176}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14643': {'gid': 14643, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.12:6841/763294965', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6840', 'nonce': 763294965}, {'type': 'v1', 'addr': '172.21.15.12:6841', 'nonce': 763294965}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24406': {'gid': 24406, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6833/4235830781', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6832', 'nonce': 4235830781}, {'type': 'v1', 'addr': '172.21.15.134:6833', 'nonce': 4235830781}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'k', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.111:6833/1902687060', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6832', 'nonce': 1902687060}, {'type': 'v1', 'addr': '172.21.15.111:6833', 'nonce': 1902687060}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'e', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.111:6835/2665640170', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6834', 'nonce': 2665640170}, {'type': 'v1', 'addr': '172.21.15.111:6835', 'nonce': 2665640170}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'f', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.134:6835/3927126579', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 3927126579}, {'type': 'v1', 'addr': '172.21.15.134:6835', 'nonce': 3927126579}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.134:6837/3631577545', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6836', 'nonce': 3631577545}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 3631577545}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.111:6837/4218407470', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6836', 'nonce': 4218407470}, {'type': 'v1', 'addr': '172.21.15.111:6837', 'nonce': 4218407470}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.134:6839/1488513960', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6838', 'nonce': 1488513960}, {'type': 'v1', 'addr': '172.21.15.134:6839', 'nonce': 1488513960}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24509': {'gid': 24509, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.111:6839/2126656770', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6838', 'nonce': 2126656770}, {'type': 'v1', 'addr': '172.21.15.111:6839', 'nonce': 2126656770}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24509, 'qdb_cluster': [24407, 24484, 24443, 24448, 24509]} 2024-04-22T13:24:21.180 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-22T13:24:21.180 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-22T13:24:21.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.332+0000 7f14ee7f4640 1 -- 172.21.15.12:0/2246231220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14e8103af0 msgr2=0x7f14e810bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.332+0000 7f14ee7f4640 1 --2- 172.21.15.12:0/2246231220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14e8103af0 0x7f14e810bfd0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009900 tx=0x7f14dc02bd60 comp rx=0 tx=0).stop 2024-04-22T13:24:21.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.333+0000 7f14ee7f4640 1 -- 172.21.15.12:0/2246231220 shutdown_connections 2024-04-22T13:24:21.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.333+0000 7f14ee7f4640 1 --2- 172.21.15.12:0/2246231220 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14e810c510 0x7f14e810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.333+0000 7f14ee7f4640 1 --2- 172.21.15.12:0/2246231220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14e8103af0 0x7f14e810bfd0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.333+0000 7f14ee7f4640 1 --2- 172.21.15.12:0/2246231220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14e81031d0 0x7f14e81035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.333+0000 7f14ee7f4640 1 -- 172.21.15.12:0/2246231220 >> 172.21.15.12:0/2246231220 conn(0x7f14e8077f50 msgr2=0x7f14e8078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:21.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.333+0000 7f14ee7f4640 1 -- 172.21.15.12:0/2246231220 shutdown_connections 2024-04-22T13:24:21.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.333+0000 7f14ee7f4640 1 -- 172.21.15.12:0/2246231220 wait complete. 2024-04-22T13:24:21.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.333+0000 7f14ee7f4640 1 Processor -- start 2024-04-22T13:24:21.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.333+0000 7f14ee7f4640 1 -- start start 2024-04-22T13:24:21.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.334+0000 7f14ee7f4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14e81031d0 0x7f14e8198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.334+0000 7f14ee7f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14e8103af0 0x7f14e8198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.334+0000 7f14ee7f4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14e810c510 0x7f14e819d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.334+0000 7f14ee7f4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f14e81121a0 con 0x7f14e81031d0 2024-04-22T13:24:21.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.334+0000 7f14ee7f4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f14e8112020 con 0x7f14e810c510 2024-04-22T13:24:21.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.334+0000 7f14ee7f4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f14e8112320 con 0x7f14e8103af0 2024-04-22T13:24:21.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.334+0000 7f14e7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14e81031d0 0x7f14e8198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.334+0000 7f14e7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14e81031d0 0x7f14e8198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59668/0 (socket says 172.21.15.12:59668) 2024-04-22T13:24:21.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.334+0000 7f14e7fff640 1 -- 172.21.15.12:0/1436481121 learned_addr learned my addr 172.21.15.12:0/1436481121 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:21.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.334+0000 7f14ecd91640 1 --2- 172.21.15.12:0/1436481121 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14e810c510 0x7f14e819d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.334+0000 7f14e77fe640 1 --2- 172.21.15.12:0/1436481121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14e8103af0 0x7f14e8198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.334+0000 7f14e7fff640 1 -- 172.21.15.12:0/1436481121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14e8103af0 msgr2=0x7f14e8198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.334+0000 7f14e7fff640 1 --2- 172.21.15.12:0/1436481121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14e8103af0 0x7f14e8198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.334+0000 7f14e7fff640 1 -- 172.21.15.12:0/1436481121 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14e810c510 msgr2=0x7f14e819d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.334+0000 7f14e7fff640 1 --2- 172.21.15.12:0/1436481121 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14e810c510 0x7f14e819d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.334+0000 7f14e7fff640 1 -- 172.21.15.12:0/1436481121 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e819d750 con 0x7f14e81031d0 2024-04-22T13:24:21.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.335+0000 7f14e7fff640 1 --2- 172.21.15.12:0/1436481121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14e81031d0 0x7f14e8198760 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f14d800c970 tx=0x7f14d800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:21.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.335+0000 7f14e57fa640 1 -- 172.21.15.12:0/1436481121 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d8007bb0 con 0x7f14e81031d0 2024-04-22T13:24:21.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.335+0000 7f14ee7f4640 1 -- 172.21.15.12:0/1436481121 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14e819da40 con 0x7f14e81031d0 2024-04-22T13:24:21.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.335+0000 7f14e57fa640 1 -- 172.21.15.12:0/1436481121 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14d8007d50 con 0x7f14e81031d0 2024-04-22T13:24:21.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.335+0000 7f14e57fa640 1 -- 172.21.15.12:0/1436481121 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d800f450 con 0x7f14e81031d0 2024-04-22T13:24:21.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.335+0000 7f14ee7f4640 1 -- 172.21.15.12:0/1436481121 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14e81a52d0 con 0x7f14e81031d0 2024-04-22T13:24:21.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.337+0000 7f14e57fa640 1 -- 172.21.15.12:0/1436481121 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f14d8004f20 con 0x7f14e81031d0 2024-04-22T13:24:21.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.338+0000 7f14e57fa640 1 --2- 172.21.15.12:0/1436481121 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f14d00a3e40 0x7f14d00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.338+0000 7f14e77fe640 1 --2- 172.21.15.12:0/1436481121 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f14d00a3e40 0x7f14d00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.338+0000 7f14e57fa640 1 -- 172.21.15.12:0/1436481121 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f14d80d0710 con 0x7f14e81031d0 2024-04-22T13:24:21.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.338+0000 7f14f13e9740 1 Processor -- start 2024-04-22T13:24:21.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.338+0000 7f14e77fe640 1 --2- 172.21.15.12:0/1436481121 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f14d00a3e40 0x7f14d00a6300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f14dc006010 tx=0x7f14dc005e40 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:21.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.339+0000 7f14f13e9740 1 -- start start 2024-04-22T13:24:21.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.339+0000 7f14f13e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c908ef7b0 0x556c908efb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.339+0000 7f14f13e9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c908f0160 0x556c908f8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.339+0000 7f14f13e9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c908f8ce0 0x556c908fb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.339+0000 7f14f13e9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556c90853b10 con 0x556c908f8ce0 2024-04-22T13:24:21.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.339+0000 7f14f13e9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556c907f1240 con 0x556c908f0160 2024-04-22T13:24:21.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.339+0000 7f14f13e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556c9084fad0 con 0x556c908ef7b0 2024-04-22T13:24:21.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.339+0000 7f14ecd91640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c908f8ce0 0x556c908fb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.339+0000 7f14e7fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c908ef7b0 0x556c908efb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.339+0000 7f14e7fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c908ef7b0 0x556c908efb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.12:38176/0 (socket says 172.21.15.12:38176) 2024-04-22T13:24:21.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.339+0000 7f14e77fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c908f0160 0x556c908f8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.339+0000 7f14ecd91640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c908f8ce0 0x556c908fb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59670/0 (socket says 172.21.15.12:59670) 2024-04-22T13:24:21.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.339+0000 7f14e7fff640 1 -- 172.21.15.12:0/2021554585 learned_addr learned my addr 172.21.15.12:0/2021554585 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:21.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.339+0000 7f14e7fff640 1 -- 172.21.15.12:0/2021554585 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c908f0160 msgr2=0x556c908f8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.339+0000 7f14e7fff640 1 --2- 172.21.15.12:0/2021554585 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c908f0160 0x556c908f8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.339+0000 7f14e7fff640 1 -- 172.21.15.12:0/2021554585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c908f8ce0 msgr2=0x556c908fb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.339+0000 7f14e7fff640 1 --2- 172.21.15.12:0/2021554585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c908f8ce0 0x556c908fb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.339+0000 7f14e7fff640 1 -- 172.21.15.12:0/2021554585 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c905662d0 con 0x556c908ef7b0 2024-04-22T13:24:21.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.339+0000 7f14ecd91640 1 --2- 172.21.15.12:0/2021554585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c908f8ce0 0x556c908fb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:24:21.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.339+0000 7f14e77fe640 1 --2- 172.21.15.12:0/2021554585 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c908f0160 0x556c908f8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:24:21.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.340+0000 7f14e7fff640 1 --2- 172.21.15.12:0/2021554585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c908ef7b0 0x556c908efb90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f14d800c940 tx=0x7f14d809c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88ef74e21e09c755 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:21.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.340+0000 7f14ce7fc640 1 -- 172.21.15.12:0/2021554585 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d80b5070 con 0x556c908ef7b0 2024-04-22T13:24:21.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.340+0000 7f14ce7fc640 1 -- 172.21.15.12:0/2021554585 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14d80a5120 con 0x556c908ef7b0 2024-04-22T13:24:21.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.340+0000 7f14ce7fc640 1 -- 172.21.15.12:0/2021554585 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d801d040 con 0x556c908ef7b0 2024-04-22T13:24:21.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.341+0000 7f14f13e9740 1 -- 172.21.15.12:0/2021554585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c908ef7b0 msgr2=0x556c908efb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.341+0000 7f14f13e9740 1 --2- 172.21.15.12:0/2021554585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c908ef7b0 0x556c908efb90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f14d800c940 tx=0x7f14d809c460 comp rx=0 tx=0).stop 2024-04-22T13:24:21.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.341+0000 7f14f13e9740 1 -- 172.21.15.12:0/2021554585 shutdown_connections 2024-04-22T13:24:21.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.341+0000 7f14f13e9740 1 --2- 172.21.15.12:0/2021554585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c908f8ce0 0x556c908fb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.341+0000 7f14f13e9740 1 --2- 172.21.15.12:0/2021554585 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c908f0160 0x556c908f8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.341+0000 7f14f13e9740 1 --2- 172.21.15.12:0/2021554585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c908ef7b0 0x556c908efb90 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.341+0000 7f14f13e9740 1 -- 172.21.15.12:0/2021554585 >> 172.21.15.12:0/2021554585 conn(0x556c908e9a60 msgr2=0x556c908e9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:21.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.341+0000 7f14f13e9740 1 -- 172.21.15.12:0/2021554585 shutdown_connections 2024-04-22T13:24:21.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.341+0000 7f14f13e9740 1 -- 172.21.15.12:0/2021554585 wait complete. 2024-04-22T13:24:21.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.341+0000 7f14f13e9740 1 -- start start 2024-04-22T13:24:21.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.342+0000 7f14f13e9740 1 Processor -- start 2024-04-22T13:24:21.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.342+0000 7f14f13e9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c908ef7b0 0x556c908f5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.342+0000 7f14f13e9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c908f0160 0x556c908f62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.342+0000 7f14f13e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c908f8ce0 0x556c908f67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.342+0000 7f14f13e9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556c908fbc50 con 0x556c908f0160 2024-04-22T13:24:21.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.342+0000 7f14f13e9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556c907dc190 con 0x556c908ef7b0 2024-04-22T13:24:21.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.342+0000 7f14f13e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556c908fbdd0 con 0x556c908f8ce0 2024-04-22T13:24:21.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.342+0000 7f14f13e9740 10 client.0 _ll_register_callbacks cb 0x556c907d78b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:21.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.342+0000 7f14e77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c908f0160 0x556c908f62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.342+0000 7f14e77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c908f0160 0x556c908f62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59682/0 (socket says 172.21.15.12:59682) 2024-04-22T13:24:21.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.342+0000 7f14e77fe640 1 -- 172.21.15.12:0/3581819360 learned_addr learned my addr 172.21.15.12:0/3581819360 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:21.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.342+0000 7f14ecd91640 1 --2- 172.21.15.12:0/3581819360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c908f8ce0 0x556c908f67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.342+0000 7f14e7fff640 1 --2- 172.21.15.12:0/3581819360 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c908ef7b0 0x556c908f5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.343+0000 7f14e7fff640 1 -- 172.21.15.12:0/3581819360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c908f8ce0 msgr2=0x556c908f67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.343+0000 7f14e7fff640 1 --2- 172.21.15.12:0/3581819360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c908f8ce0 0x556c908f67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.343+0000 7f14e7fff640 1 -- 172.21.15.12:0/3581819360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c908f0160 msgr2=0x556c908f62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.343+0000 7f14e7fff640 1 --2- 172.21.15.12:0/3581819360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c908f0160 0x556c908f62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.343+0000 7f14e7fff640 1 -- 172.21.15.12:0/3581819360 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c906306b0 con 0x556c908ef7b0 2024-04-22T13:24:21.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.343+0000 7f14ecd91640 1 --2- 172.21.15.12:0/3581819360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c908f8ce0 0x556c908f67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:24:21.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.343+0000 7f14e77fe640 1 --2- 172.21.15.12:0/3581819360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c908f0160 0x556c908f62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:24:21.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.343+0000 7f14e7fff640 1 --2- 172.21.15.12:0/3581819360 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c908ef7b0 0x556c908f5d70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x556c908fb920 tx=0x7f14d80048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59fb7922b1af5004 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:21.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.343+0000 7f14c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-22T13:24:21.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.343+0000 7f14c77fe640 1 -- 172.21.15.12:0/3581819360 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d80b5030 con 0x556c908ef7b0 2024-04-22T13:24:21.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.343+0000 7f14f13e9740 1 -- 172.21.15.12:0/3581819360 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14d80a5120 con 0x556c908ef7b0 2024-04-22T13:24:21.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.343+0000 7f14c77fe640 1 -- 172.21.15.12:0/3581819360 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f14d80bb2a0 con 0x556c908ef7b0 2024-04-22T13:24:21.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.343+0000 7f14c77fe640 1 -- 172.21.15.12:0/3581819360 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f14d801d070 con 0x556c908ef7b0 2024-04-22T13:24:21.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.343+0000 7f14f13e9740 10 client.25259 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:21.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.343+0000 7f14f13e9740 1 -- 172.21.15.12:0/3581819360 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556c90907fe0 con 0x556c908ef7b0 2024-04-22T13:24:21.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.343+0000 7f14c77fe640 1 -- 172.21.15.12:0/3581819360 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14d80a5120 con 0x556c908ef7b0 2024-04-22T13:24:21.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.344+0000 7f14c77fe640 1 -- 172.21.15.12:0/3581819360 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f14d809b420 con 0x556c908ef7b0 2024-04-22T13:24:21.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.344+0000 7f14f13e9740 10 client.25259 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:21.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.344+0000 7f14f13e9740 10 client.25259 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-22T13:24:21.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.344+0000 7f14f13e9740 20 client.25259 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:21.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.344+0000 7f14f13e9740 1 --2- 172.21.15.12:0/3581819360 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x556c9090a110 0x556c9090b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.344+0000 7f14f13e9740 4 client.25259 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:24:21.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.344+0000 7f14f13e9740 1 -- 172.21.15.12:0/3581819360 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c90861530 con 0x556c9090a110 2024-04-22T13:24:21.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.344+0000 7f14e77fe640 1 --2- 172.21.15.12:0/3581819360 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x556c9090a110 0x556c9090b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.344+0000 7f14c77fe640 10 client.25259 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:24:21.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.344+0000 7f14e77fe640 1 --2- 172.21.15.12:0/3581819360 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x556c9090a110 0x556c9090b580 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=324f23cd8a3338ce server_cookie=c221e001f36edc8d in_seq=0 out_seq=0 2024-04-22T13:24:21.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.345+0000 7f14c77fe640 1 -- 172.21.15.12:0/3581819360 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556c90861530 con 0x556c9090a110 2024-04-22T13:24:21.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.345+0000 7f14c77fe640 10 client.25259 handle_command_reply: tid=0 2024-04-22T13:24:21.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.345+0000 7f14f13e9740 1 client.25259 shutdown 2024-04-22T13:24:21.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.345+0000 7f14f13e9740 1 -- 172.21.15.12:0/3581819360 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c908ef7b0 msgr2=0x556c908f5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.345+0000 7f14f13e9740 1 --2- 172.21.15.12:0/3581819360 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c908ef7b0 0x556c908f5d70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x556c908fb920 tx=0x7f14d80048f0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.345+0000 7f14f13e9740 1 -- 172.21.15.12:0/3581819360 shutdown_connections 2024-04-22T13:24:21.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.345+0000 7f14f13e9740 1 --2- 172.21.15.12:0/3581819360 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x556c9090a110 0x556c9090b580 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.345+0000 7f14f13e9740 1 --2- 172.21.15.12:0/3581819360 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c908f8ce0 0x556c908f67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.345+0000 7f14c77fe640 0 client.25259 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:24:21.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.345+0000 7f14f13e9740 1 --2- 172.21.15.12:0/3581819360 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c908f0160 0x556c908f62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.345+0000 7f14f13e9740 1 --2- 172.21.15.12:0/3581819360 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c908ef7b0 0x556c908f5d70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.345+0000 7f14f13e9740 1 -- 172.21.15.12:0/3581819360 >> 172.21.15.12:0/3581819360 conn(0x556c908e9a60 msgr2=0x556c908ebbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:21.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.345+0000 7f14f13e9740 1 -- 172.21.15.12:0/3581819360 shutdown_connections 2024-04-22T13:24:21.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.345+0000 7f14f13e9740 1 -- 172.21.15.12:0/3581819360 wait complete. 2024-04-22T13:24:21.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.346+0000 7f14f13e9740 20 client.25259 trim_cache size 0 max 16384 2024-04-22T13:24:21.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.381+0000 7f14f13e9740 1 Processor -- start 2024-04-22T13:24:21.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.381+0000 7f14f13e9740 1 -- start start 2024-04-22T13:24:21.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.381+0000 7f14f13e9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c908ef7b0 0x556c908f84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.381+0000 7f14f13e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c908f0160 0x556c908f0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.381+0000 7f14f13e9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c908f8ce0 0x556c908e4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.381+0000 7f14f13e9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556c909098c0 con 0x556c908ef7b0 2024-04-22T13:24:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.381+0000 7f14f13e9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556c90909f90 con 0x556c908f8ce0 2024-04-22T13:24:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.381+0000 7f14f13e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556c9090a660 con 0x556c908f0160 2024-04-22T13:24:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.382+0000 7f14e7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c908ef7b0 0x556c908f84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.382+0000 7f14e7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c908ef7b0 0x556c908f84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59694/0 (socket says 172.21.15.12:59694) 2024-04-22T13:24:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.382+0000 7f14e7fff640 1 -- 172.21.15.12:0/2307441362 learned_addr learned my addr 172.21.15.12:0/2307441362 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.382+0000 7f14ecd91640 1 --2- 172.21.15.12:0/2307441362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c908f8ce0 0x556c908e4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.382+0000 7f14e77fe640 1 --2- 172.21.15.12:0/2307441362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c908f0160 0x556c908f0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.382+0000 7f14e7fff640 1 -- 172.21.15.12:0/2307441362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c908f0160 msgr2=0x556c908f0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.382+0000 7f14e7fff640 1 --2- 172.21.15.12:0/2307441362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c908f0160 0x556c908f0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.382+0000 7f14e7fff640 1 -- 172.21.15.12:0/2307441362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c908f8ce0 msgr2=0x556c908e4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.382+0000 7f14e7fff640 1 --2- 172.21.15.12:0/2307441362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c908f8ce0 0x556c908e4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.382+0000 7f14e7fff640 1 -- 172.21.15.12:0/2307441362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c908fbd70 con 0x556c908ef7b0 2024-04-22T13:24:21.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.382+0000 7f14ecd91640 1 --2- 172.21.15.12:0/2307441362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c908f8ce0 0x556c908e4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:24:21.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.382+0000 7f14e7fff640 1 --2- 172.21.15.12:0/2307441362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c908ef7b0 0x556c908f84a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f14d80ce4b0 tx=0x7f14d8005210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e831faf7b1e2f25a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:21.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.382+0000 7f14cdffb640 1 -- 172.21.15.12:0/2307441362 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d80b5030 con 0x556c908ef7b0 2024-04-22T13:24:21.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.382+0000 7f14cdffb640 1 -- 172.21.15.12:0/2307441362 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14d80cdc70 con 0x556c908ef7b0 2024-04-22T13:24:21.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.382+0000 7f14cdffb640 1 -- 172.21.15.12:0/2307441362 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d80ce270 con 0x556c908ef7b0 2024-04-22T13:24:21.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.383+0000 7f14f13e9740 1 -- 172.21.15.12:0/2307441362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c908ef7b0 msgr2=0x556c908f84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.383+0000 7f14f13e9740 1 --2- 172.21.15.12:0/2307441362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c908ef7b0 0x556c908f84a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f14d80ce4b0 tx=0x7f14d8005210 comp rx=0 tx=0).stop 2024-04-22T13:24:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.383+0000 7f14f13e9740 1 -- 172.21.15.12:0/2307441362 shutdown_connections 2024-04-22T13:24:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.383+0000 7f14f13e9740 1 --2- 172.21.15.12:0/2307441362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c908f8ce0 0x556c908e4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.384+0000 7f14f13e9740 1 --2- 172.21.15.12:0/2307441362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c908f0160 0x556c908f0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.384+0000 7f14f13e9740 1 --2- 172.21.15.12:0/2307441362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c908ef7b0 0x556c908f84a0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.384+0000 7f14f13e9740 1 -- 172.21.15.12:0/2307441362 >> 172.21.15.12:0/2307441362 conn(0x556c908e9a60 msgr2=0x556c908e8a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.384+0000 7f14f13e9740 1 -- 172.21.15.12:0/2307441362 shutdown_connections 2024-04-22T13:24:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.384+0000 7f14f13e9740 1 -- 172.21.15.12:0/2307441362 wait complete. 2024-04-22T13:24:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.384+0000 7f14f13e9740 1 -- start start 2024-04-22T13:24:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.384+0000 7f14f13e9740 1 Processor -- start 2024-04-22T13:24:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.385+0000 7f14f13e9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c908ef7b0 0x556c908f2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.385+0000 7f14f13e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c908f0160 0x556c908f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.385+0000 7f14f13e9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c908f8ce0 0x556c909127d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.385+0000 7f14f13e9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556c9090b400 con 0x556c908ef7b0 2024-04-22T13:24:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.385+0000 7f14f13e9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556c9090ad30 con 0x556c908f8ce0 2024-04-22T13:24:21.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.385+0000 7f14f13e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556c9090bad0 con 0x556c908f0160 2024-04-22T13:24:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.385+0000 7f14f13e9740 10 client.0 _ll_register_callbacks cb 0x7f14d80b3070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.385+0000 7f14e7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c908ef7b0 0x556c908f2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.385+0000 7f14e7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c908ef7b0 0x556c908f2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59710/0 (socket says 172.21.15.12:59710) 2024-04-22T13:24:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.385+0000 7f14e7fff640 1 -- 172.21.15.12:0/2725040050 learned_addr learned my addr 172.21.15.12:0/2725040050 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.385+0000 7f14ecd91640 1 --2- 172.21.15.12:0/2725040050 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c908f8ce0 0x556c909127d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.385+0000 7f14e77fe640 1 --2- 172.21.15.12:0/2725040050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c908f0160 0x556c908f2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.385+0000 7f14e7fff640 1 -- 172.21.15.12:0/2725040050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c908f0160 msgr2=0x556c908f2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.385+0000 7f14e7fff640 1 --2- 172.21.15.12:0/2725040050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c908f0160 0x556c908f2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.385+0000 7f14e7fff640 1 -- 172.21.15.12:0/2725040050 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c908f8ce0 msgr2=0x556c909127d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.385+0000 7f14e7fff640 1 --2- 172.21.15.12:0/2725040050 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c908f8ce0 0x556c909127d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.385+0000 7f14e7fff640 1 -- 172.21.15.12:0/2725040050 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c90912e10 con 0x556c908ef7b0 2024-04-22T13:24:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.386+0000 7f14e7fff640 1 --2- 172.21.15.12:0/2725040050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c908ef7b0 0x556c908f2830 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f14d80a4bf0 tx=0x7f14d80bb330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78ce5c1a84f798c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.386+0000 7f14c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.386+0000 7f14c77fe640 1 -- 172.21.15.12:0/2725040050 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d80b5040 con 0x556c908ef7b0 2024-04-22T13:24:21.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.386+0000 7f14c77fe640 1 -- 172.21.15.12:0/2725040050 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f14d809bbc0 con 0x556c908ef7b0 2024-04-22T13:24:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.386+0000 7f14f13e9740 1 -- 172.21.15.12:0/2725040050 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14d80cdc70 con 0x556c908ef7b0 2024-04-22T13:24:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.386+0000 7f14c77fe640 1 -- 172.21.15.12:0/2725040050 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f14d80a5120 con 0x556c908ef7b0 2024-04-22T13:24:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.386+0000 7f14f13e9740 10 client.15735 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.386+0000 7f14f13e9740 1 -- 172.21.15.12:0/2725040050 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556c90912e10 con 0x556c908ef7b0 2024-04-22T13:24:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.386+0000 7f14c77fe640 1 -- 172.21.15.12:0/2725040050 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14d80cdc70 con 0x556c908ef7b0 2024-04-22T13:24:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.386+0000 7f14c77fe640 1 -- 172.21.15.12:0/2725040050 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f14d80d0110 con 0x556c908ef7b0 2024-04-22T13:24:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.386+0000 7f14f13e9740 10 client.15735 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.386+0000 7f14f13e9740 10 client.15735 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-22T13:24:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.386+0000 7f14f13e9740 20 client.15735 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.386+0000 7f14f13e9740 1 --2- 172.21.15.12:0/2725040050 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x556c90974220 0x556c90972f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.386+0000 7f14f13e9740 4 client.15735 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T13:24:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.386+0000 7f14f13e9740 1 -- 172.21.15.12:0/2725040050 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "status"}) -- 0x556c9085b3a0 con 0x556c90974220 2024-04-22T13:24:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.387+0000 7f14e77fe640 1 --2- 172.21.15.12:0/2725040050 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x556c90974220 0x556c90972f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.387+0000 7f14c77fe640 10 client.15735 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:24:21.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.387+0000 7f14e77fe640 1 --2- 172.21.15.12:0/2725040050 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x556c90974220 0x556c90972f30 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c09eb859663a2acb server_cookie=cf044f140bb97c9e in_seq=0 out_seq=0 2024-04-22T13:24:21.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.387+0000 7f14c77fe640 1 -- 172.21.15.12:0/2725040050 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556c9085b3a0 con 0x556c90974220 2024-04-22T13:24:21.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.387+0000 7f14c77fe640 10 client.15735 handle_command_reply: tid=0 2024-04-22T13:24:21.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.387+0000 7f14f13e9740 1 client.15735 shutdown 2024-04-22T13:24:21.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.388+0000 7f14f13e9740 1 -- 172.21.15.12:0/2725040050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c908ef7b0 msgr2=0x556c908f2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.388+0000 7f14f13e9740 1 --2- 172.21.15.12:0/2725040050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c908ef7b0 0x556c908f2830 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f14d80a4bf0 tx=0x7f14d80bb330 comp rx=0 tx=0).stop 2024-04-22T13:24:21.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.388+0000 7f14f13e9740 1 -- 172.21.15.12:0/2725040050 shutdown_connections 2024-04-22T13:24:21.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.388+0000 7f14f13e9740 1 --2- 172.21.15.12:0/2725040050 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x556c90974220 0x556c90972f30 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.388+0000 7f14f13e9740 1 --2- 172.21.15.12:0/2725040050 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556c908f8ce0 0x556c909127d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.388+0000 7f14f13e9740 1 --2- 172.21.15.12:0/2725040050 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556c908f0160 0x556c908f2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.388+0000 7f14f13e9740 1 --2- 172.21.15.12:0/2725040050 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556c908ef7b0 0x556c908f2830 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.388+0000 7f14f13e9740 1 -- 172.21.15.12:0/2725040050 >> 172.21.15.12:0/2725040050 conn(0x556c908e9a60 msgr2=0x556c908e8a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:21.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.388+0000 7f14c77fe640 0 client.15735 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:24:21.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.388+0000 7f14f13e9740 1 -- 172.21.15.12:0/2725040050 shutdown_connections 2024-04-22T13:24:21.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.388+0000 7f14f13e9740 1 -- 172.21.15.12:0/2725040050 wait complete. 2024-04-22T13:24:21.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.388+0000 7f14f13e9740 20 client.15735 trim_cache size 0 max 16384 2024-04-22T13:24:21.391 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:24:21.391 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "0aede5ea-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:24:21.391 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 4, 2024-04-22T13:24:21.391 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24407, 2024-04-22T13:24:21.391 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-04-22T13:24:21.391 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-04-22T13:24:21.391 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-04-22T13:24:21.391 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 129.99633682300001, 2024-04-22T13:24:21.391 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 28, 2024-04-22T13:24:21.391 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 90, 2024-04-22T13:24:21.392 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 89, 2024-04-22T13:24:21.392 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 169.335962507 2024-04-22T13:24:21.392 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:24:21.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.388+0000 7f14ee7f4640 1 -- 172.21.15.12:0/1436481121 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f14d00a3e40 msgr2=0x7f14d00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.388+0000 7f14ee7f4640 1 --2- 172.21.15.12:0/1436481121 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f14d00a3e40 0x7f14d00a6300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f14dc006010 tx=0x7f14dc005e40 comp rx=0 tx=0).stop 2024-04-22T13:24:21.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.388+0000 7f14ee7f4640 1 -- 172.21.15.12:0/1436481121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14e81031d0 msgr2=0x7f14e8198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.388+0000 7f14ee7f4640 1 --2- 172.21.15.12:0/1436481121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14e81031d0 0x7f14e8198760 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f14d800c970 tx=0x7f14d800ce30 comp rx=0 tx=0).stop 2024-04-22T13:24:21.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.389+0000 7f14ee7f4640 1 -- 172.21.15.12:0/1436481121 shutdown_connections 2024-04-22T13:24:21.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.389+0000 7f14ee7f4640 1 --2- 172.21.15.12:0/1436481121 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f14d00a3e40 0x7f14d00a6300 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.389+0000 7f14ee7f4640 1 --2- 172.21.15.12:0/1436481121 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14e810c510 0x7f14e819d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.389+0000 7f14ee7f4640 1 --2- 172.21.15.12:0/1436481121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f14e8103af0 0x7f14e8198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.389+0000 7f14ee7f4640 1 --2- 172.21.15.12:0/1436481121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f14e81031d0 0x7f14e8198760 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.389+0000 7f14ee7f4640 1 -- 172.21.15.12:0/1436481121 >> 172.21.15.12:0/1436481121 conn(0x7f14e8077f50 msgr2=0x7f14e80fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:21.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.389+0000 7f14ee7f4640 1 -- 172.21.15.12:0/1436481121 shutdown_connections 2024-04-22T13:24:21.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.389+0000 7f14ee7f4640 1 -- 172.21.15.12:0/1436481121 wait complete. 2024-04-22T13:24:21.398 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-22T13:24:21.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.508+0000 7f4fa790d640 1 -- 172.21.15.12:0/3702751163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fa0101090 msgr2=0x7f4fa0101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.508+0000 7f4fa790d640 1 --2- 172.21.15.12:0/3702751163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fa0101090 0x7f4fa0101470 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c009900 tx=0x7f4f9c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:21.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.508+0000 7f4fa790d640 1 -- 172.21.15.12:0/3702751163 shutdown_connections 2024-04-22T13:24:21.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.508+0000 7f4fa790d640 1 --2- 172.21.15.12:0/3702751163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fa0106160 0x7f4fa010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.508+0000 7f4fa790d640 1 --2- 172.21.15.12:0/3702751163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4fa0101a40 0x7f4fa0105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.508+0000 7f4fa790d640 1 --2- 172.21.15.12:0/3702751163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fa0101090 0x7f4fa0101470 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.508+0000 7f4fa790d640 1 -- 172.21.15.12:0/3702751163 >> 172.21.15.12:0/3702751163 conn(0x7f4fa0077f50 msgr2=0x7f4fa0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:21.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.509+0000 7f4fa790d640 1 -- 172.21.15.12:0/3702751163 shutdown_connections 2024-04-22T13:24:21.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.509+0000 7f4fa790d640 1 -- 172.21.15.12:0/3702751163 wait complete. 2024-04-22T13:24:21.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.509+0000 7f4fa790d640 1 Processor -- start 2024-04-22T13:24:21.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.509+0000 7f4fa790d640 1 -- start start 2024-04-22T13:24:21.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.509+0000 7f4fa790d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fa0101090 0x7f4fa01975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.509+0000 7f4fa790d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4fa0106160 0x7f4fa0197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.509+0000 7f4fa790d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fa019be60 0x7f4fa019c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.509+0000 7f4fa790d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa01110a0 con 0x7f4fa019be60 2024-04-22T13:24:21.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.509+0000 7f4fa790d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa0110f20 con 0x7f4fa0106160 2024-04-22T13:24:21.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.509+0000 7f4fa790d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa0111220 con 0x7f4fa0101090 2024-04-22T13:24:21.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.510+0000 7f4fa5eaa640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fa019be60 0x7f4fa019c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.510+0000 7f4fa5eaa640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fa019be60 0x7f4fa019c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59734/0 (socket says 172.21.15.12:59734) 2024-04-22T13:24:21.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.510+0000 7f4fa5eaa640 1 -- 172.21.15.12:0/557987192 learned_addr learned my addr 172.21.15.12:0/557987192 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:21.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.510+0000 7f4fa4ea8640 1 --2- 172.21.15.12:0/557987192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4fa0106160 0x7f4fa0197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.510+0000 7f4fa56a9640 1 --2- 172.21.15.12:0/557987192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fa0101090 0x7f4fa01975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.510+0000 7f4fa5eaa640 1 -- 172.21.15.12:0/557987192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fa0101090 msgr2=0x7f4fa01975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.510+0000 7f4fa5eaa640 1 --2- 172.21.15.12:0/557987192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fa0101090 0x7f4fa01975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.510+0000 7f4fa5eaa640 1 -- 172.21.15.12:0/557987192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4fa0106160 msgr2=0x7f4fa0197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.510+0000 7f4fa5eaa640 1 --2- 172.21.15.12:0/557987192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4fa0106160 0x7f4fa0197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.510+0000 7f4fa5eaa640 1 -- 172.21.15.12:0/557987192 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa019c950 con 0x7f4fa019be60 2024-04-22T13:24:21.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.510+0000 7f4fa4ea8640 1 --2- 172.21.15.12:0/557987192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4fa0106160 0x7f4fa0197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:21.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.510+0000 7f4fa5eaa640 1 --2- 172.21.15.12:0/557987192 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fa019be60 0x7f4fa019c310 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800cce0 tx=0x7f4f9800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:21.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.510+0000 7f4f967fc640 1 -- 172.21.15.12:0/557987192 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9800f450 con 0x7f4fa019be60 2024-04-22T13:24:21.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.510+0000 7f4f967fc640 1 -- 172.21.15.12:0/557987192 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f9800faa0 con 0x7f4fa019be60 2024-04-22T13:24:21.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.510+0000 7f4fa790d640 1 -- 172.21.15.12:0/557987192 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fa019cc40 con 0x7f4fa019be60 2024-04-22T13:24:21.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.510+0000 7f4f967fc640 1 -- 172.21.15.12:0/557987192 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f98018b70 con 0x7f4fa019be60 2024-04-22T13:24:21.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.510+0000 7f4fa790d640 1 -- 172.21.15.12:0/557987192 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fa01a43e0 con 0x7f4fa019be60 2024-04-22T13:24:21.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.513+0000 7f4f967fc640 1 -- 172.21.15.12:0/557987192 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f4f98018d10 con 0x7f4fa019be60 2024-04-22T13:24:21.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.513+0000 7f4f967fc640 1 --2- 172.21.15.12:0/557987192 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4f8c0a3e40 0x7f4f8c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.514+0000 7f4fa56a9640 1 --2- 172.21.15.12:0/557987192 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4f8c0a3e40 0x7f4f8c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.514+0000 7f4f967fc640 1 -- 172.21.15.12:0/557987192 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f4f980d0f70 con 0x7f4fa019be60 2024-04-22T13:24:21.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.514+0000 7f4fa56a9640 1 --2- 172.21.15.12:0/557987192 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4f8c0a3e40 0x7f4f8c0a6300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c000c00 tx=0x7f4f9c004040 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:21.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.514+0000 7f4faa5d0740 1 Processor -- start 2024-04-22T13:24:21.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.514+0000 7f4faa5d0740 1 -- start start 2024-04-22T13:24:21.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.515+0000 7f4faa5d0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561994eaf610 0x561994eaf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.515+0000 7f4faa5d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561994eaffc0 0x561994eb6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.515+0000 7f4faa5d0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561994eb6f20 0x561994eb73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.515+0000 7f4faa5d0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561994e13b10 con 0x561994eb6f20 2024-04-22T13:24:21.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.515+0000 7f4faa5d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561994db1240 con 0x561994eaffc0 2024-04-22T13:24:21.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.515+0000 7f4faa5d0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561994e0fad0 con 0x561994eaf610 2024-04-22T13:24:21.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.515+0000 7f4fa5eaa640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561994eb6f20 0x561994eb73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.515+0000 7f4fa5eaa640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561994eb6f20 0x561994eb73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59738/0 (socket says 172.21.15.12:59738) 2024-04-22T13:24:21.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.515+0000 7f4fa5eaa640 1 -- 172.21.15.12:0/2504928228 learned_addr learned my addr 172.21.15.12:0/2504928228 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:21.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.515+0000 7f4fa4ea8640 1 --2- 172.21.15.12:0/2504928228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561994eaffc0 0x561994eb6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.515+0000 7f4fa56a9640 1 --2- 172.21.15.12:0/2504928228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561994eaf610 0x561994eaf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.515+0000 7f4fa5eaa640 1 -- 172.21.15.12:0/2504928228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561994eaf610 msgr2=0x561994eaf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.515+0000 7f4fa5eaa640 1 --2- 172.21.15.12:0/2504928228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561994eaf610 0x561994eaf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.515+0000 7f4fa5eaa640 1 -- 172.21.15.12:0/2504928228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561994eaffc0 msgr2=0x561994eb6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.515+0000 7f4fa5eaa640 1 --2- 172.21.15.12:0/2504928228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561994eaffc0 0x561994eb6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.515+0000 7f4fa5eaa640 1 -- 172.21.15.12:0/2504928228 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561994b262d0 con 0x561994eb6f20 2024-04-22T13:24:21.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.515+0000 7f4fa4ea8640 1 --2- 172.21.15.12:0/2504928228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561994eaffc0 0x561994eb6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:24:21.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.515+0000 7f4fa5eaa640 1 --2- 172.21.15.12:0/2504928228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561994eb6f20 0x561994eb73a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4f98099110 tx=0x7f4f9800af30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a8a20845a9190ad server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:21.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.515+0000 7f4f8b7fe640 1 -- 172.21.15.12:0/2504928228 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f980b6070 con 0x561994eb6f20 2024-04-22T13:24:21.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.516+0000 7f4f8b7fe640 1 -- 172.21.15.12:0/2504928228 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f980b6210 con 0x561994eb6f20 2024-04-22T13:24:21.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.516+0000 7f4f8b7fe640 1 -- 172.21.15.12:0/2504928228 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f98099320 con 0x561994eb6f20 2024-04-22T13:24:21.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.516+0000 7f4faa5d0740 1 -- 172.21.15.12:0/2504928228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561994eb6f20 msgr2=0x561994eb73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.516+0000 7f4faa5d0740 1 --2- 172.21.15.12:0/2504928228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561994eb6f20 0x561994eb73a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4f98099110 tx=0x7f4f9800af30 comp rx=0 tx=0).stop 2024-04-22T13:24:21.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.516+0000 7f4faa5d0740 1 -- 172.21.15.12:0/2504928228 shutdown_connections 2024-04-22T13:24:21.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.516+0000 7f4faa5d0740 1 --2- 172.21.15.12:0/2504928228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561994eb6f20 0x561994eb73a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.516+0000 7f4faa5d0740 1 --2- 172.21.15.12:0/2504928228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561994eaffc0 0x561994eb6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.516+0000 7f4faa5d0740 1 --2- 172.21.15.12:0/2504928228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561994eaf610 0x561994eaf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.516+0000 7f4faa5d0740 1 -- 172.21.15.12:0/2504928228 >> 172.21.15.12:0/2504928228 conn(0x561994ea9a60 msgr2=0x561994ea9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:21.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.516+0000 7f4faa5d0740 1 -- 172.21.15.12:0/2504928228 shutdown_connections 2024-04-22T13:24:21.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.516+0000 7f4faa5d0740 1 -- 172.21.15.12:0/2504928228 wait complete. 2024-04-22T13:24:21.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.517+0000 7f4faa5d0740 1 -- start start 2024-04-22T13:24:21.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.517+0000 7f4faa5d0740 1 Processor -- start 2024-04-22T13:24:21.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.517+0000 7f4faa5d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561994eaf610 0x561994ebaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.517+0000 7f4faa5d0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561994eaffc0 0x561994ebb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.517+0000 7f4faa5d0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561994eb6f20 0x561994ebb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.517+0000 7f4faa5d0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561994ebdea0 con 0x561994eaffc0 2024-04-22T13:24:21.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.517+0000 7f4faa5d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561994d9c190 con 0x561994eaf610 2024-04-22T13:24:21.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.517+0000 7f4faa5d0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561994ebe020 con 0x561994eb6f20 2024-04-22T13:24:21.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.517+0000 7f4faa5d0740 10 client.0 _ll_register_callbacks cb 0x561994d978b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:21.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.518+0000 7f4fa4ea8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561994eaffc0 0x561994ebb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.518+0000 7f4fa4ea8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561994eaffc0 0x561994ebb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59752/0 (socket says 172.21.15.12:59752) 2024-04-22T13:24:21.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.518+0000 7f4fa4ea8640 1 -- 172.21.15.12:0/3230108319 learned_addr learned my addr 172.21.15.12:0/3230108319 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:21.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.518+0000 7f4fa56a9640 1 --2- 172.21.15.12:0/3230108319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561994eaf610 0x561994ebaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.518+0000 7f4fa5eaa640 1 --2- 172.21.15.12:0/3230108319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561994eb6f20 0x561994ebb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.518+0000 7f4fa4ea8640 1 -- 172.21.15.12:0/3230108319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561994eb6f20 msgr2=0x561994ebb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.518+0000 7f4fa4ea8640 1 --2- 172.21.15.12:0/3230108319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561994eb6f20 0x561994ebb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.518+0000 7f4fa4ea8640 1 -- 172.21.15.12:0/3230108319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561994eaf610 msgr2=0x561994ebaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.518+0000 7f4fa4ea8640 1 --2- 172.21.15.12:0/3230108319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561994eaf610 0x561994ebaac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.518+0000 7f4fa4ea8640 1 -- 172.21.15.12:0/3230108319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561994bf06b0 con 0x561994eaffc0 2024-04-22T13:24:21.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.518+0000 7f4fa4ea8640 1 --2- 172.21.15.12:0/3230108319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561994eaffc0 0x561994ebb000 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4fa006b6e0 tx=0x7f4f90012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd40d2f180b1f156 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:21.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.518+0000 7f4f88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:21.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.518+0000 7f4f88ff9640 1 -- 172.21.15.12:0/3230108319 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f90004230 con 0x561994eaffc0 2024-04-22T13:24:21.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.518+0000 7f4f88ff9640 1 -- 172.21.15.12:0/3230108319 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f4f900043d0 con 0x561994eaffc0 2024-04-22T13:24:21.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.518+0000 7f4faa5d0740 1 -- 172.21.15.12:0/3230108319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561994e21530 con 0x561994eaffc0 2024-04-22T13:24:21.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.518+0000 7f4f88ff9640 1 -- 172.21.15.12:0/3230108319 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4f900106a0 con 0x561994eaffc0 2024-04-22T13:24:21.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.519+0000 7f4faa5d0740 10 client.15747 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:21.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.519+0000 7f4faa5d0740 1 -- 172.21.15.12:0/3230108319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561994eca2c0 con 0x561994eaffc0 2024-04-22T13:24:21.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.519+0000 7f4f88ff9640 1 -- 172.21.15.12:0/3230108319 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561994e21530 con 0x561994eaffc0 2024-04-22T13:24:21.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.519+0000 7f4f88ff9640 1 -- 172.21.15.12:0/3230108319 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4f9001b030 con 0x561994eaffc0 2024-04-22T13:24:21.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.519+0000 7f4faa5d0740 10 client.15747 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:21.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.519+0000 7f4faa5d0740 10 client.15747 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-22T13:24:21.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.519+0000 7f4faa5d0740 20 client.15747 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:21.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.519+0000 7f4faa5d0740 1 --2- 172.21.15.12:0/3230108319 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x561994ecc880 0x561994ecdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.519+0000 7f4faa5d0740 4 client.15747 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:24:21.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.519+0000 7f4faa5d0740 1 -- 172.21.15.12:0/3230108319 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561994e1b3a0 con 0x561994ecc880 2024-04-22T13:24:21.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.519+0000 7f4fa56a9640 1 --2- 172.21.15.12:0/3230108319 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x561994ecc880 0x561994ecdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.519+0000 7f4f88ff9640 10 client.15747 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:24:21.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.519+0000 7f4fa56a9640 1 --2- 172.21.15.12:0/3230108319 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x561994ecc880 0x561994ecdcf0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c63b9d41ceb81a4 server_cookie=9497e8f3302054da in_seq=0 out_seq=0 2024-04-22T13:24:21.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.520+0000 7f4f88ff9640 1 -- 172.21.15.12:0/3230108319 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561994e1b3a0 con 0x561994ecc880 2024-04-22T13:24:21.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.520+0000 7f4f88ff9640 10 client.15747 handle_command_reply: tid=0 2024-04-22T13:24:21.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.520+0000 7f4faa5d0740 1 client.15747 shutdown 2024-04-22T13:24:21.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.520+0000 7f4faa5d0740 1 -- 172.21.15.12:0/3230108319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561994eaffc0 msgr2=0x561994ebb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.520+0000 7f4faa5d0740 1 --2- 172.21.15.12:0/3230108319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561994eaffc0 0x561994ebb000 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4fa006b6e0 tx=0x7f4f90012820 comp rx=0 tx=0).stop 2024-04-22T13:24:21.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.520+0000 7f4faa5d0740 1 -- 172.21.15.12:0/3230108319 shutdown_connections 2024-04-22T13:24:21.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.520+0000 7f4faa5d0740 1 --2- 172.21.15.12:0/3230108319 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x561994ecc880 0x561994ecdcf0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.520+0000 7f4faa5d0740 1 --2- 172.21.15.12:0/3230108319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561994eb6f20 0x561994ebb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.520+0000 7f4faa5d0740 1 --2- 172.21.15.12:0/3230108319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561994eaffc0 0x561994ebb000 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.520+0000 7f4faa5d0740 1 --2- 172.21.15.12:0/3230108319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561994eaf610 0x561994ebaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.520+0000 7f4f88ff9640 0 client.15747 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:24:21.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.520+0000 7f4faa5d0740 1 -- 172.21.15.12:0/3230108319 >> 172.21.15.12:0/3230108319 conn(0x561994ea9a60 msgr2=0x561994eabbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:21.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.520+0000 7f4faa5d0740 1 -- 172.21.15.12:0/3230108319 shutdown_connections 2024-04-22T13:24:21.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.521+0000 7f4faa5d0740 1 -- 172.21.15.12:0/3230108319 wait complete. 2024-04-22T13:24:21.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.521+0000 7f4faa5d0740 20 client.15747 trim_cache size 0 max 16384 2024-04-22T13:24:21.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.557+0000 7f4faa5d0740 1 Processor -- start 2024-04-22T13:24:21.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.557+0000 7f4faa5d0740 1 -- start start 2024-04-22T13:24:21.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.557+0000 7f4faa5d0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561994eaf610 0x561994ebd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.557+0000 7f4faa5d0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561994eaffc0 0x561994ec7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.557+0000 7f4faa5d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561994eb6f20 0x561994ebb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.557+0000 7f4faa5d0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561994ecc030 con 0x561994eaf610 2024-04-22T13:24:21.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.557+0000 7f4faa5d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561994ecc700 con 0x561994eb6f20 2024-04-22T13:24:21.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.557+0000 7f4faa5d0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561994eccdd0 con 0x561994eaffc0 2024-04-22T13:24:21.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.557+0000 7f4fa56a9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561994eaf610 0x561994ebd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.557+0000 7f4fa56a9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561994eaf610 0x561994ebd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59754/0 (socket says 172.21.15.12:59754) 2024-04-22T13:24:21.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.557+0000 7f4fa56a9640 1 -- 172.21.15.12:0/1126627650 learned_addr learned my addr 172.21.15.12:0/1126627650 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:21.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.558+0000 7f4fa4ea8640 1 --2- 172.21.15.12:0/1126627650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561994eaffc0 0x561994ec7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.558+0000 7f4fa5eaa640 1 --2- 172.21.15.12:0/1126627650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561994eb6f20 0x561994ebb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.558+0000 7f4fa56a9640 1 -- 172.21.15.12:0/1126627650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561994eaffc0 msgr2=0x561994ec7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.558+0000 7f4fa56a9640 1 --2- 172.21.15.12:0/1126627650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561994eaffc0 0x561994ec7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.558+0000 7f4fa56a9640 1 -- 172.21.15.12:0/1126627650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561994eb6f20 msgr2=0x561994ebb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.558+0000 7f4fa56a9640 1 --2- 172.21.15.12:0/1126627650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561994eb6f20 0x561994ebb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.558+0000 7f4fa56a9640 1 -- 172.21.15.12:0/1126627650 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561994eb61a0 con 0x561994eaf610 2024-04-22T13:24:21.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.558+0000 7f4fa5eaa640 1 --2- 172.21.15.12:0/1126627650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561994eb6f20 0x561994ebb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:21.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.558+0000 7f4fa4ea8640 1 --2- 172.21.15.12:0/1126627650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561994eaffc0 0x561994ec7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:21.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.558+0000 7f4fa56a9640 1 --2- 172.21.15.12:0/1126627650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561994eaf610 0x561994ebd1f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c02c170 tx=0x7f4f9c041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d99d7eec824b6f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:21.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.558+0000 7f4f8affd640 1 -- 172.21.15.12:0/1126627650 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c02c9e0 con 0x561994eaf610 2024-04-22T13:24:21.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.558+0000 7f4f8affd640 1 -- 172.21.15.12:0/1126627650 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f9c059070 con 0x561994eaf610 2024-04-22T13:24:21.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.558+0000 7f4f8affd640 1 -- 172.21.15.12:0/1126627650 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c053770 con 0x561994eaf610 2024-04-22T13:24:21.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.559+0000 7f4faa5d0740 1 -- 172.21.15.12:0/1126627650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561994eaf610 msgr2=0x561994ebd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.559+0000 7f4faa5d0740 1 --2- 172.21.15.12:0/1126627650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561994eaf610 0x561994ebd1f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c02c170 tx=0x7f4f9c041000 comp rx=0 tx=0).stop 2024-04-22T13:24:21.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.559+0000 7f4faa5d0740 1 -- 172.21.15.12:0/1126627650 shutdown_connections 2024-04-22T13:24:21.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.559+0000 7f4faa5d0740 1 --2- 172.21.15.12:0/1126627650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561994eb6f20 0x561994ebb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.559+0000 7f4faa5d0740 1 --2- 172.21.15.12:0/1126627650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561994eaffc0 0x561994ec7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.559+0000 7f4faa5d0740 1 --2- 172.21.15.12:0/1126627650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561994eaf610 0x561994ebd1f0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.559+0000 7f4faa5d0740 1 -- 172.21.15.12:0/1126627650 >> 172.21.15.12:0/1126627650 conn(0x561994ea9a60 msgr2=0x561994eb4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:21.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.559+0000 7f4faa5d0740 1 -- 172.21.15.12:0/1126627650 shutdown_connections 2024-04-22T13:24:21.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.559+0000 7f4faa5d0740 1 -- 172.21.15.12:0/1126627650 wait complete. 2024-04-22T13:24:21.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.559+0000 7f4faa5d0740 1 -- start start 2024-04-22T13:24:21.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.559+0000 7f4faa5d0740 1 Processor -- start 2024-04-22T13:24:21.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.560+0000 7f4faa5d0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561994eaf610 0x561994eb7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.560+0000 7f4faa5d0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561994eaffc0 0x561994eb7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.560+0000 7f4faa5d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561994eb6f20 0x561994eb83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.560+0000 7f4faa5d0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561994ecdb70 con 0x561994eaf610 2024-04-22T13:24:21.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.560+0000 7f4faa5d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561994ecd4a0 con 0x561994eb6f20 2024-04-22T13:24:21.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.560+0000 7f4faa5d0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561994ece240 con 0x561994eaffc0 2024-04-22T13:24:21.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.560+0000 7f4faa5d0740 10 client.0 _ll_register_callbacks cb 0x7f4f9000b790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.560+0000 7f4fa56a9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561994eaf610 0x561994eb7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.560+0000 7f4fa56a9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561994eaf610 0x561994eb7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:59760/0 (socket says 172.21.15.12:59760) 2024-04-22T13:24:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.560+0000 7f4fa56a9640 1 -- 172.21.15.12:0/3686108017 learned_addr learned my addr 172.21.15.12:0/3686108017 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.560+0000 7f4fa5eaa640 1 --2- 172.21.15.12:0/3686108017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561994eb6f20 0x561994eb83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.560+0000 7f4fa4ea8640 1 --2- 172.21.15.12:0/3686108017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561994eaffc0 0x561994eb7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.560+0000 7f4fa56a9640 1 -- 172.21.15.12:0/3686108017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561994eaffc0 msgr2=0x561994eb7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.560+0000 7f4fa56a9640 1 --2- 172.21.15.12:0/3686108017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561994eaffc0 0x561994eb7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.560+0000 7f4fa56a9640 1 -- 172.21.15.12:0/3686108017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561994eb6f20 msgr2=0x561994eb83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.560+0000 7f4fa56a9640 1 --2- 172.21.15.12:0/3686108017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561994eb6f20 0x561994eb83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.560+0000 7f4fa56a9640 1 -- 172.21.15.12:0/3686108017 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561994ed4250 con 0x561994eaf610 2024-04-22T13:24:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.560+0000 7f4fa56a9640 1 --2- 172.21.15.12:0/3686108017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561994eaf610 0x561994eb7960 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c053410 tx=0x7f4f9c02ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78167e513925fb7c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.560+0000 7f4f88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.561+0000 7f4f88ff9640 1 -- 172.21.15.12:0/3686108017 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c0536c0 con 0x561994eaf610 2024-04-22T13:24:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.561+0000 7f4f88ff9640 1 -- 172.21.15.12:0/3686108017 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f4f9c053860 con 0x561994eaf610 2024-04-22T13:24:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.561+0000 7f4faa5d0740 1 -- 172.21.15.12:0/3686108017 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f9c059070 con 0x561994eaf610 2024-04-22T13:24:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.561+0000 7f4f88ff9640 1 -- 172.21.15.12:0/3686108017 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4f9c06b050 con 0x561994eaf610 2024-04-22T13:24:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.561+0000 7f4faa5d0740 10 client.15753 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.561+0000 7f4faa5d0740 1 -- 172.21.15.12:0/3686108017 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561994f32ad0 con 0x561994eaf610 2024-04-22T13:24:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.561+0000 7f4f88ff9640 1 -- 172.21.15.12:0/3686108017 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f9c059070 con 0x561994eaf610 2024-04-22T13:24:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.561+0000 7f4f88ff9640 1 -- 172.21.15.12:0/3686108017 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4f9c05d550 con 0x561994eaf610 2024-04-22T13:24:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.561+0000 7f4faa5d0740 10 client.15753 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.561+0000 7f4faa5d0740 10 client.15753 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-22T13:24:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.561+0000 7f4faa5d0740 20 client.15753 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.561+0000 7f4faa5d0740 1 --2- 172.21.15.12:0/3686108017 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x561994f372c0 0x561994f35fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.561+0000 7f4faa5d0740 4 client.15753 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T13:24:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.561+0000 7f4faa5d0740 1 -- 172.21.15.12:0/3686108017 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "status"}) -- 0x561994d97960 con 0x561994f372c0 2024-04-22T13:24:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.562+0000 7f4fa4ea8640 1 --2- 172.21.15.12:0/3686108017 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x561994f372c0 0x561994f35fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.562+0000 7f4fa4ea8640 1 --2- 172.21.15.12:0/3686108017 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x561994f372c0 0x561994f35fd0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a302e5921a52c69a server_cookie=5e2295ed68b9464f in_seq=0 out_seq=0 2024-04-22T13:24:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.562+0000 7f4f88ff9640 10 client.15753 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:24:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.562+0000 7f4f88ff9640 1 -- 172.21.15.12:0/3686108017 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x561994d97960 con 0x561994f372c0 2024-04-22T13:24:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.562+0000 7f4f88ff9640 10 client.15753 handle_command_reply: tid=0 2024-04-22T13:24:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.562+0000 7f4faa5d0740 1 client.15753 shutdown 2024-04-22T13:24:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.562+0000 7f4faa5d0740 1 -- 172.21.15.12:0/3686108017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561994eaf610 msgr2=0x561994eb7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.562+0000 7f4faa5d0740 1 --2- 172.21.15.12:0/3686108017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561994eaf610 0x561994eb7960 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c053410 tx=0x7f4f9c02ca60 comp rx=0 tx=0).stop 2024-04-22T13:24:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.562+0000 7f4faa5d0740 1 -- 172.21.15.12:0/3686108017 shutdown_connections 2024-04-22T13:24:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.562+0000 7f4faa5d0740 1 --2- 172.21.15.12:0/3686108017 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x561994f372c0 0x561994f35fd0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.562+0000 7f4faa5d0740 1 --2- 172.21.15.12:0/3686108017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561994eb6f20 0x561994eb83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.562+0000 7f4f88ff9640 0 client.15753 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:24:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.562+0000 7f4faa5d0740 1 --2- 172.21.15.12:0/3686108017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561994eaffc0 0x561994eb7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.562+0000 7f4faa5d0740 1 --2- 172.21.15.12:0/3686108017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561994eaf610 0x561994eb7960 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.562+0000 7f4faa5d0740 1 -- 172.21.15.12:0/3686108017 >> 172.21.15.12:0/3686108017 conn(0x561994ea9a60 msgr2=0x561994eb92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.563+0000 7f4faa5d0740 1 -- 172.21.15.12:0/3686108017 shutdown_connections 2024-04-22T13:24:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.563+0000 7f4faa5d0740 1 -- 172.21.15.12:0/3686108017 wait complete. 2024-04-22T13:24:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.563+0000 7f4faa5d0740 20 client.15753 trim_cache size 0 max 16384 2024-04-22T13:24:21.566 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:24:21.566 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "0aede5ea-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:24:21.566 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 3, 2024-04-22T13:24:21.566 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24443, 2024-04-22T13:24:21.566 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-04-22T13:24:21.566 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-04-22T13:24:21.566 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-04-22T13:24:21.566 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 131.179881897, 2024-04-22T13:24:21.566 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 28, 2024-04-22T13:24:21.566 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 90, 2024-04-22T13:24:21.566 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 89, 2024-04-22T13:24:21.566 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 161.24924462000001 2024-04-22T13:24:21.567 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:24:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.563+0000 7f4fa790d640 1 -- 172.21.15.12:0/557987192 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4f8c0a3e40 msgr2=0x7f4f8c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.563+0000 7f4fa790d640 1 --2- 172.21.15.12:0/557987192 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4f8c0a3e40 0x7f4f8c0a6300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c000c00 tx=0x7f4f9c004040 comp rx=0 tx=0).stop 2024-04-22T13:24:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.563+0000 7f4fa790d640 1 -- 172.21.15.12:0/557987192 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fa019be60 msgr2=0x7f4fa019c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.563+0000 7f4fa790d640 1 --2- 172.21.15.12:0/557987192 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fa019be60 0x7f4fa019c310 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800cce0 tx=0x7f4f9800a620 comp rx=0 tx=0).stop 2024-04-22T13:24:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.563+0000 7f4fa790d640 1 -- 172.21.15.12:0/557987192 shutdown_connections 2024-04-22T13:24:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.563+0000 7f4fa790d640 1 --2- 172.21.15.12:0/557987192 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4f8c0a3e40 0x7f4f8c0a6300 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.563+0000 7f4fa790d640 1 --2- 172.21.15.12:0/557987192 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4fa019be60 0x7f4fa019c310 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.563+0000 7f4fa790d640 1 --2- 172.21.15.12:0/557987192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4fa0106160 0x7f4fa0197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.563+0000 7f4fa790d640 1 --2- 172.21.15.12:0/557987192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fa0101090 0x7f4fa01975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.563+0000 7f4fa790d640 1 -- 172.21.15.12:0/557987192 >> 172.21.15.12:0/557987192 conn(0x7f4fa0077f50 msgr2=0x7f4fa00fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.564+0000 7f4fa790d640 1 -- 172.21.15.12:0/557987192 shutdown_connections 2024-04-22T13:24:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.564+0000 7f4fa790d640 1 -- 172.21.15.12:0/557987192 wait complete. 2024-04-22T13:24:21.575 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-04-22T13:24:21.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.698+0000 7f6d7066d640 1 -- 172.21.15.12:0/2121858547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d6810c510 msgr2=0x7f6d6810e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.698+0000 7f6d7066d640 1 --2- 172.21.15.12:0/2121858547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d6810c510 0x7f6d6810e9b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6d60009ca0 tx=0x7f6d6002c230 comp rx=0 tx=0).stop 2024-04-22T13:24:21.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.698+0000 7f6d7066d640 1 -- 172.21.15.12:0/2121858547 shutdown_connections 2024-04-22T13:24:21.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.698+0000 7f6d7066d640 1 --2- 172.21.15.12:0/2121858547 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d6810c510 0x7f6d6810e9b0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.698+0000 7f6d7066d640 1 --2- 172.21.15.12:0/2121858547 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d68103af0 0x7f6d6810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.698+0000 7f6d7066d640 1 --2- 172.21.15.12:0/2121858547 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d681031d0 0x7f6d681035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.698+0000 7f6d7066d640 1 -- 172.21.15.12:0/2121858547 >> 172.21.15.12:0/2121858547 conn(0x7f6d68077f50 msgr2=0x7f6d68078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:21.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.698+0000 7f6d7066d640 1 -- 172.21.15.12:0/2121858547 shutdown_connections 2024-04-22T13:24:21.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.698+0000 7f6d7066d640 1 -- 172.21.15.12:0/2121858547 wait complete. 2024-04-22T13:24:21.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.699+0000 7f6d7066d640 1 Processor -- start 2024-04-22T13:24:21.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.699+0000 7f6d7066d640 1 -- start start 2024-04-22T13:24:21.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.699+0000 7f6d7066d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d681031d0 0x7f6d68100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.699+0000 7f6d7066d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d68103af0 0x7f6d68101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.699+0000 7f6d7066d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d6810c510 0x7f6d681018a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.699+0000 7f6d7066d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6d68112180 con 0x7f6d6810c510 2024-04-22T13:24:21.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.699+0000 7f6d7066d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6d68112000 con 0x7f6d681031d0 2024-04-22T13:24:21.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.699+0000 7f6d7066d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6d68112300 con 0x7f6d68103af0 2024-04-22T13:24:21.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.699+0000 7f6d6ec0a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d6810c510 0x7f6d681018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.699+0000 7f6d6e409640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d681031d0 0x7f6d68100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.699+0000 7f6d6ec0a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d6810c510 0x7f6d681018a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45028/0 (socket says 172.21.15.12:45028) 2024-04-22T13:24:21.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.699+0000 7f6d6ec0a640 1 -- 172.21.15.12:0/3274028073 learned_addr learned my addr 172.21.15.12:0/3274028073 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:21.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.700+0000 7f6d6dc08640 1 --2- 172.21.15.12:0/3274028073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d68103af0 0x7f6d68101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.700+0000 7f6d6e409640 1 -- 172.21.15.12:0/3274028073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d68103af0 msgr2=0x7f6d68101360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.700+0000 7f6d6e409640 1 --2- 172.21.15.12:0/3274028073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d68103af0 0x7f6d68101360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.700+0000 7f6d6e409640 1 -- 172.21.15.12:0/3274028073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d6810c510 msgr2=0x7f6d681018a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.700+0000 7f6d6e409640 1 --2- 172.21.15.12:0/3274028073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d6810c510 0x7f6d681018a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.700+0000 7f6d6e409640 1 -- 172.21.15.12:0/3274028073 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d681a6ba0 con 0x7f6d681031d0 2024-04-22T13:24:21.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.700+0000 7f6d6ec0a640 1 --2- 172.21.15.12:0/3274028073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d6810c510 0x7f6d681018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:24:21.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.700+0000 7f6d6e409640 1 --2- 172.21.15.12:0/3274028073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d681031d0 0x7f6d68100e20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400d6d0 tx=0x7f6d6400db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:21.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.700+0000 7f6d5f7fe640 1 -- 172.21.15.12:0/3274028073 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d64017070 con 0x7f6d681031d0 2024-04-22T13:24:21.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.700+0000 7f6d7066d640 1 -- 172.21.15.12:0/3274028073 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d681a6e90 con 0x7f6d681031d0 2024-04-22T13:24:21.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.700+0000 7f6d5f7fe640 1 -- 172.21.15.12:0/3274028073 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d64009820 con 0x7f6d681031d0 2024-04-22T13:24:21.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.700+0000 7f6d5f7fe640 1 -- 172.21.15.12:0/3274028073 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d64004c40 con 0x7f6d681031d0 2024-04-22T13:24:21.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.700+0000 7f6d7066d640 1 -- 172.21.15.12:0/3274028073 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d681a73a0 con 0x7f6d681031d0 2024-04-22T13:24:21.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.703+0000 7f6d73267740 1 Processor -- start 2024-04-22T13:24:21.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.703+0000 7f6d5f7fe640 1 -- 172.21.15.12:0/3274028073 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f6d64005220 con 0x7f6d681031d0 2024-04-22T13:24:21.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.704+0000 7f6d73267740 1 -- start start 2024-04-22T13:24:21.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.704+0000 7f6d73267740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe4ba8c7b0 0x55fe4ba8cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.704+0000 7f6d73267740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe4ba8d160 0x55fe4ba95710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.704+0000 7f6d73267740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe4ba95ce0 0x55fe4ba98130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.704+0000 7f6d73267740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fe4b9f0b10 con 0x55fe4ba95ce0 2024-04-22T13:24:21.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.704+0000 7f6d73267740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fe4b98e240 con 0x55fe4ba8d160 2024-04-22T13:24:21.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.704+0000 7f6d73267740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fe4b9ecad0 con 0x55fe4ba8c7b0 2024-04-22T13:24:21.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.704+0000 7f6d6dc08640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe4ba95ce0 0x55fe4ba98130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.704+0000 7f6d6dc08640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe4ba95ce0 0x55fe4ba98130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45038/0 (socket says 172.21.15.12:45038) 2024-04-22T13:24:21.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.704+0000 7f6d6dc08640 1 -- 172.21.15.12:0/3496655935 learned_addr learned my addr 172.21.15.12:0/3496655935 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:21.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.704+0000 7f6d6ec0a640 1 --2- 172.21.15.12:0/3496655935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe4ba8c7b0 0x55fe4ba8cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.704+0000 7f6d6e409640 1 --2- 172.21.15.12:0/3496655935 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe4ba8d160 0x55fe4ba95710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.704+0000 7f6d5f7fe640 1 --2- 172.21.15.12:0/3274028073 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6d540a3e70 0x7f6d540a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.704+0000 7f6d6dc08640 1 -- 172.21.15.12:0/3496655935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe4ba8c7b0 msgr2=0x55fe4ba8cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.704+0000 7f6d6dc08640 1 --2- 172.21.15.12:0/3496655935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe4ba8c7b0 0x55fe4ba8cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.704+0000 7f6d6dc08640 1 -- 172.21.15.12:0/3496655935 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe4ba8d160 msgr2=0x55fe4ba95710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.704+0000 7f6d6dc08640 1 --2- 172.21.15.12:0/3496655935 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe4ba8d160 0x55fe4ba95710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.704+0000 7f6d6dc08640 1 -- 172.21.15.12:0/3496655935 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe4b7032d0 con 0x55fe4ba95ce0 2024-04-22T13:24:21.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.704+0000 7f6d6ec0a640 1 --2- 172.21.15.12:0/3496655935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe4ba8c7b0 0x55fe4ba8cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:21.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.704+0000 7f6d6ec0a640 1 --2- 172.21.15.12:0/3274028073 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6d540a3e70 0x7f6d540a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.704+0000 7f6d5f7fe640 1 -- 172.21.15.12:0/3274028073 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f6d640d04a0 con 0x7f6d681031d0 2024-04-22T13:24:21.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.705+0000 7f6d6dc08640 1 --2- 172.21.15.12:0/3496655935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe4ba95ce0 0x55fe4ba98130 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009c10 tx=0x7f6d58011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acad8b5b387c979e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:21.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.705+0000 7f6d5cbfe640 1 -- 172.21.15.12:0/3496655935 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d58004230 con 0x55fe4ba95ce0 2024-04-22T13:24:21.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.705+0000 7f6d5cbfe640 1 -- 172.21.15.12:0/3496655935 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d580043d0 con 0x55fe4ba95ce0 2024-04-22T13:24:21.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.705+0000 7f6d5cbfe640 1 -- 172.21.15.12:0/3496655935 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d58004ce0 con 0x55fe4ba95ce0 2024-04-22T13:24:21.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.706+0000 7f6d6ec0a640 1 --2- 172.21.15.12:0/3274028073 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6d540a3e70 0x7f6d540a6330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6d60009a60 tx=0x7f6d6002e040 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:21.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.706+0000 7f6d73267740 1 -- 172.21.15.12:0/3496655935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe4ba95ce0 msgr2=0x55fe4ba98130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.706+0000 7f6d73267740 1 --2- 172.21.15.12:0/3496655935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe4ba95ce0 0x55fe4ba98130 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009c10 tx=0x7f6d58011810 comp rx=0 tx=0).stop 2024-04-22T13:24:21.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.706+0000 7f6d73267740 1 -- 172.21.15.12:0/3496655935 shutdown_connections 2024-04-22T13:24:21.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.706+0000 7f6d73267740 1 --2- 172.21.15.12:0/3496655935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe4ba95ce0 0x55fe4ba98130 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.706+0000 7f6d73267740 1 --2- 172.21.15.12:0/3496655935 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe4ba8d160 0x55fe4ba95710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.706+0000 7f6d73267740 1 --2- 172.21.15.12:0/3496655935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe4ba8c7b0 0x55fe4ba8cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.706+0000 7f6d73267740 1 -- 172.21.15.12:0/3496655935 >> 172.21.15.12:0/3496655935 conn(0x55fe4ba86a60 msgr2=0x55fe4ba86e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:21.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.706+0000 7f6d73267740 1 -- 172.21.15.12:0/3496655935 shutdown_connections 2024-04-22T13:24:21.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.706+0000 7f6d73267740 1 -- 172.21.15.12:0/3496655935 wait complete. 2024-04-22T13:24:21.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.706+0000 7f6d73267740 1 -- start start 2024-04-22T13:24:21.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.707+0000 7f6d73267740 1 Processor -- start 2024-04-22T13:24:21.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.707+0000 7f6d73267740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe4ba8c7b0 0x55fe4ba92c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.707+0000 7f6d73267740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe4ba8d160 0x55fe4ba93180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.707+0000 7f6d73267740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe4ba95ce0 0x55fe4ba936c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.707+0000 7f6d73267740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fe4ba98c50 con 0x55fe4ba95ce0 2024-04-22T13:24:21.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.707+0000 7f6d73267740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fe4b979190 con 0x55fe4ba8c7b0 2024-04-22T13:24:21.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.707+0000 7f6d73267740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fe4ba98dd0 con 0x55fe4ba8d160 2024-04-22T13:24:21.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.707+0000 7f6d73267740 10 client.0 _ll_register_callbacks cb 0x55fe4b9748b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:21.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.707+0000 7f6d6ec0a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe4ba95ce0 0x55fe4ba936c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.707+0000 7f6d6ec0a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe4ba95ce0 0x55fe4ba936c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45048/0 (socket says 172.21.15.12:45048) 2024-04-22T13:24:21.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.707+0000 7f6d6e409640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe4ba8c7b0 0x55fe4ba92c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.707+0000 7f6d6ec0a640 1 -- 172.21.15.12:0/78963692 learned_addr learned my addr 172.21.15.12:0/78963692 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:21.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.707+0000 7f6d6dc08640 1 --2- 172.21.15.12:0/78963692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe4ba8d160 0x55fe4ba93180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.707+0000 7f6d6ec0a640 1 -- 172.21.15.12:0/78963692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe4ba8d160 msgr2=0x55fe4ba93180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.707+0000 7f6d6ec0a640 1 --2- 172.21.15.12:0/78963692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe4ba8d160 0x55fe4ba93180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.707+0000 7f6d6ec0a640 1 -- 172.21.15.12:0/78963692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe4ba8c7b0 msgr2=0x55fe4ba92c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.707+0000 7f6d6ec0a640 1 --2- 172.21.15.12:0/78963692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe4ba8c7b0 0x55fe4ba92c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.707+0000 7f6d6ec0a640 1 -- 172.21.15.12:0/78963692 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe4b7cd6b0 con 0x55fe4ba95ce0 2024-04-22T13:24:21.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.707+0000 7f6d6e409640 1 --2- 172.21.15.12:0/78963692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe4ba8c7b0 0x55fe4ba92c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:24:21.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.708+0000 7f6d6ec0a640 1 --2- 172.21.15.12:0/78963692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe4ba95ce0 0x55fe4ba936c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6d60006010 tx=0x7f6d60005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b861b2234b9e7d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:21.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.708+0000 7f6d51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:21.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.708+0000 7f6d51ffb640 1 -- 172.21.15.12:0/78963692 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d60039040 con 0x55fe4ba95ce0 2024-04-22T13:24:21.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.708+0000 7f6d51ffb640 1 -- 172.21.15.12:0/78963692 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f6d60032850 con 0x55fe4ba95ce0 2024-04-22T13:24:21.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.708+0000 7f6d73267740 1 -- 172.21.15.12:0/78963692 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d580043d0 con 0x55fe4ba95ce0 2024-04-22T13:24:21.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.708+0000 7f6d51ffb640 1 -- 172.21.15.12:0/78963692 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6d60032a50 con 0x55fe4ba95ce0 2024-04-22T13:24:21.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.708+0000 7f6d73267740 10 client.15765 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:21.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.708+0000 7f6d73267740 1 -- 172.21.15.12:0/78963692 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fe4baa4ea0 con 0x55fe4ba95ce0 2024-04-22T13:24:21.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.708+0000 7f6d51ffb640 1 -- 172.21.15.12:0/78963692 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d580043d0 con 0x55fe4ba95ce0 2024-04-22T13:24:21.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.708+0000 7f6d51ffb640 1 -- 172.21.15.12:0/78963692 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6d6004d450 con 0x55fe4ba95ce0 2024-04-22T13:24:21.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.709+0000 7f6d73267740 10 client.15765 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:21.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.709+0000 7f6d73267740 10 client.15765 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-22T13:24:21.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.709+0000 7f6d73267740 20 client.15765 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:21.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.709+0000 7f6d73267740 1 --2- 172.21.15.12:0/78963692 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55fe4baa7570 0x55fe4baa89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.709+0000 7f6d73267740 4 client.15765 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:24:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.709+0000 7f6d73267740 1 -- 172.21.15.12:0/78963692 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe4b9fe530 con 0x55fe4baa7570 2024-04-22T13:24:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.709+0000 7f6d6e409640 1 --2- 172.21.15.12:0/78963692 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55fe4baa7570 0x55fe4baa89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.709+0000 7f6d51ffb640 10 client.15765 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:24:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.709+0000 7f6d6e409640 1 --2- 172.21.15.12:0/78963692 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55fe4baa7570 0x55fe4baa89e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c6dfe90990fc95bb server_cookie=4a5178172c01ce7a in_seq=0 out_seq=0 2024-04-22T13:24:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.710+0000 7f6d51ffb640 1 -- 172.21.15.12:0/78963692 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fe4b9fe530 con 0x55fe4baa7570 2024-04-22T13:24:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.710+0000 7f6d51ffb640 10 client.15765 handle_command_reply: tid=0 2024-04-22T13:24:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.710+0000 7f6d73267740 1 client.15765 shutdown 2024-04-22T13:24:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.710+0000 7f6d73267740 1 -- 172.21.15.12:0/78963692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe4ba95ce0 msgr2=0x55fe4ba936c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.710+0000 7f6d73267740 1 --2- 172.21.15.12:0/78963692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe4ba95ce0 0x55fe4ba936c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6d60006010 tx=0x7f6d60005d80 comp rx=0 tx=0).stop 2024-04-22T13:24:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.710+0000 7f6d73267740 1 -- 172.21.15.12:0/78963692 shutdown_connections 2024-04-22T13:24:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.710+0000 7f6d73267740 1 --2- 172.21.15.12:0/78963692 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55fe4baa7570 0x55fe4baa89e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.710+0000 7f6d73267740 1 --2- 172.21.15.12:0/78963692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe4ba95ce0 0x55fe4ba936c0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.710+0000 7f6d51ffb640 0 client.15765 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:24:21.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.710+0000 7f6d73267740 1 --2- 172.21.15.12:0/78963692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe4ba8d160 0x55fe4ba93180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.710+0000 7f6d73267740 1 --2- 172.21.15.12:0/78963692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe4ba8c7b0 0x55fe4ba92c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.710+0000 7f6d73267740 1 -- 172.21.15.12:0/78963692 >> 172.21.15.12:0/78963692 conn(0x55fe4ba86a60 msgr2=0x55fe4ba88bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:21.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.710+0000 7f6d73267740 1 -- 172.21.15.12:0/78963692 shutdown_connections 2024-04-22T13:24:21.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.710+0000 7f6d73267740 1 -- 172.21.15.12:0/78963692 wait complete. 2024-04-22T13:24:21.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.710+0000 7f6d73267740 20 client.15765 trim_cache size 0 max 16384 2024-04-22T13:24:21.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.729+0000 7f6d73267740 1 Processor -- start 2024-04-22T13:24:21.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.729+0000 7f6d73267740 1 -- start start 2024-04-22T13:24:21.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.729+0000 7f6d73267740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe4ba8c7b0 0x55fe4ba95370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.729+0000 7f6d73267740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe4ba8d160 0x55fe4baa25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.729+0000 7f6d73267740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe4ba95ce0 0x55fe4ba91510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.729+0000 7f6d73267740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fe4baa6d20 con 0x55fe4ba95ce0 2024-04-22T13:24:21.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.729+0000 7f6d73267740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fe4baa73f0 con 0x55fe4ba8c7b0 2024-04-22T13:24:21.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.729+0000 7f6d73267740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fe4baa7ac0 con 0x55fe4ba8d160 2024-04-22T13:24:21.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.729+0000 7f6d6ec0a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe4ba95ce0 0x55fe4ba91510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.730+0000 7f6d6ec0a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe4ba95ce0 0x55fe4ba91510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45058/0 (socket says 172.21.15.12:45058) 2024-04-22T13:24:21.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.730+0000 7f6d6ec0a640 1 -- 172.21.15.12:0/2389075957 learned_addr learned my addr 172.21.15.12:0/2389075957 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:21.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.730+0000 7f6d6dc08640 1 --2- 172.21.15.12:0/2389075957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe4ba8d160 0x55fe4baa25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.730+0000 7f6d6e409640 1 --2- 172.21.15.12:0/2389075957 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe4ba8c7b0 0x55fe4ba95370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.730+0000 7f6d6ec0a640 1 -- 172.21.15.12:0/2389075957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe4ba8d160 msgr2=0x55fe4baa25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.730+0000 7f6d6ec0a640 1 --2- 172.21.15.12:0/2389075957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe4ba8d160 0x55fe4baa25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.730+0000 7f6d6ec0a640 1 -- 172.21.15.12:0/2389075957 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe4ba8c7b0 msgr2=0x55fe4ba95370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.730+0000 7f6d6ec0a640 1 --2- 172.21.15.12:0/2389075957 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe4ba8c7b0 0x55fe4ba95370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.730+0000 7f6d6ec0a640 1 -- 172.21.15.12:0/2389075957 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe4ba8e800 con 0x55fe4ba95ce0 2024-04-22T13:24:21.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.730+0000 7f6d6ec0a640 1 --2- 172.21.15.12:0/2389075957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe4ba95ce0 0x55fe4ba91510 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6d60009dc0 tx=0x7f6d60009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1384ab3a899188 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:21.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.732+0000 7f6d53fff640 1 -- 172.21.15.12:0/2389075957 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d60039040 con 0x55fe4ba95ce0 2024-04-22T13:24:21.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.732+0000 7f6d53fff640 1 -- 172.21.15.12:0/2389075957 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d600323d0 con 0x55fe4ba95ce0 2024-04-22T13:24:21.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.732+0000 7f6d53fff640 1 -- 172.21.15.12:0/2389075957 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6004d450 con 0x55fe4ba95ce0 2024-04-22T13:24:21.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.733+0000 7f6d73267740 1 -- 172.21.15.12:0/2389075957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe4ba95ce0 msgr2=0x55fe4ba91510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.733+0000 7f6d73267740 1 --2- 172.21.15.12:0/2389075957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe4ba95ce0 0x55fe4ba91510 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6d60009dc0 tx=0x7f6d60009b20 comp rx=0 tx=0).stop 2024-04-22T13:24:21.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.733+0000 7f6d73267740 1 -- 172.21.15.12:0/2389075957 shutdown_connections 2024-04-22T13:24:21.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.733+0000 7f6d73267740 1 --2- 172.21.15.12:0/2389075957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe4ba95ce0 0x55fe4ba91510 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.733+0000 7f6d73267740 1 --2- 172.21.15.12:0/2389075957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe4ba8d160 0x55fe4baa25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.733+0000 7f6d73267740 1 --2- 172.21.15.12:0/2389075957 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe4ba8c7b0 0x55fe4ba95370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.733+0000 7f6d73267740 1 -- 172.21.15.12:0/2389075957 >> 172.21.15.12:0/2389075957 conn(0x55fe4ba86a60 msgr2=0x55fe4ba81800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:21.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.733+0000 7f6d73267740 1 -- 172.21.15.12:0/2389075957 shutdown_connections 2024-04-22T13:24:21.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.733+0000 7f6d73267740 1 -- 172.21.15.12:0/2389075957 wait complete. 2024-04-22T13:24:21.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.733+0000 7f6d73267740 1 -- start start 2024-04-22T13:24:21.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.734+0000 7f6d73267740 1 Processor -- start 2024-04-22T13:24:21.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.734+0000 7f6d73267740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe4ba8c7b0 0x55fe4ba8f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.734+0000 7f6d73267740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe4ba8d160 0x55fe4ba8fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.734+0000 7f6d73267740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe4ba95ce0 0x55fe4ba90240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.734+0000 7f6d73267740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fe4baa8860 con 0x55fe4ba95ce0 2024-04-22T13:24:21.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.734+0000 7f6d73267740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fe4baa8190 con 0x55fe4ba8c7b0 2024-04-22T13:24:21.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.734+0000 7f6d73267740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fe4baa8f30 con 0x55fe4ba8d160 2024-04-22T13:24:21.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.734+0000 7f6d73267740 10 client.0 _ll_register_callbacks cb 0x7f6d58005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:21.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.734+0000 7f6d6ec0a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe4ba95ce0 0x55fe4ba90240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.734+0000 7f6d6ec0a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe4ba95ce0 0x55fe4ba90240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45060/0 (socket says 172.21.15.12:45060) 2024-04-22T13:24:21.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.734+0000 7f6d6dc08640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe4ba8d160 0x55fe4ba8fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.734+0000 7f6d6ec0a640 1 -- 172.21.15.12:0/1939848864 learned_addr learned my addr 172.21.15.12:0/1939848864 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:21.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.734+0000 7f6d6e409640 1 --2- 172.21.15.12:0/1939848864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe4ba8c7b0 0x55fe4ba8f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.734+0000 7f6d6ec0a640 1 -- 172.21.15.12:0/1939848864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe4ba8d160 msgr2=0x55fe4ba8fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.734+0000 7f6d6ec0a640 1 --2- 172.21.15.12:0/1939848864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe4ba8d160 0x55fe4ba8fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.734+0000 7f6d6ec0a640 1 -- 172.21.15.12:0/1939848864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe4ba8c7b0 msgr2=0x55fe4ba8f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.734+0000 7f6d6ec0a640 1 --2- 172.21.15.12:0/1939848864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe4ba8c7b0 0x55fe4ba8f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.734+0000 7f6d6ec0a640 1 -- 172.21.15.12:0/1939848864 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe4baaed70 con 0x55fe4ba95ce0 2024-04-22T13:24:21.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.734+0000 7f6d6dc08640 1 --2- 172.21.15.12:0/1939848864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe4ba8d160 0x55fe4ba8fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:24:21.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.734+0000 7f6d6e409640 1 --2- 172.21.15.12:0/1939848864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe4ba8c7b0 0x55fe4ba8f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:24:21.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.735+0000 7f6d6ec0a640 1 --2- 172.21.15.12:0/1939848864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe4ba95ce0 0x55fe4ba90240 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f6d60037040 tx=0x7f6d60005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc0addd88ab4bbb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:21.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.735+0000 7f6d51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:21.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.735+0000 7f6d51ffb640 1 -- 172.21.15.12:0/1939848864 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6006d060 con 0x55fe4ba95ce0 2024-04-22T13:24:21.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.735+0000 7f6d73267740 1 -- 172.21.15.12:0/1939848864 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d600323d0 con 0x55fe4ba95ce0 2024-04-22T13:24:21.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.735+0000 7f6d51ffb640 1 -- 172.21.15.12:0/1939848864 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f6d6004d660 con 0x55fe4ba95ce0 2024-04-22T13:24:21.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.735+0000 7f6d51ffb640 1 -- 172.21.15.12:0/1939848864 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6d60039040 con 0x55fe4ba95ce0 2024-04-22T13:24:21.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.735+0000 7f6d73267740 10 client.15771 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:21.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.735+0000 7f6d73267740 1 -- 172.21.15.12:0/1939848864 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fe4bb0d780 con 0x55fe4ba95ce0 2024-04-22T13:24:21.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.735+0000 7f6d51ffb640 1 -- 172.21.15.12:0/1939848864 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d600323d0 con 0x55fe4ba95ce0 2024-04-22T13:24:21.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.735+0000 7f6d51ffb640 1 -- 172.21.15.12:0/1939848864 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6d60072790 con 0x55fe4ba95ce0 2024-04-22T13:24:21.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.735+0000 7f6d73267740 10 client.15771 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:21.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.735+0000 7f6d73267740 10 client.15771 resolve_mds: resolved name 'f' to daemon mds.f 2024-04-22T13:24:21.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.735+0000 7f6d73267740 20 client.15771 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:21.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.735+0000 7f6d73267740 1 --2- 172.21.15.12:0/1939848864 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55fe4bb11f60 0x55fe4bb10c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.735+0000 7f6d73267740 4 client.15771 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T13:24:21.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.735+0000 7f6d73267740 1 -- 172.21.15.12:0/1939848864 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "status"}) -- 0x55fe4b9f83a0 con 0x55fe4bb11f60 2024-04-22T13:24:21.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.736+0000 7f6d6e409640 1 --2- 172.21.15.12:0/1939848864 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55fe4bb11f60 0x55fe4bb10c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.736+0000 7f6d6e409640 1 --2- 172.21.15.12:0/1939848864 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55fe4bb11f60 0x55fe4bb10c70 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=473f6f628e56e9a7 server_cookie=edc658123a6a713c in_seq=0 out_seq=0 2024-04-22T13:24:21.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.736+0000 7f6d51ffb640 10 client.15771 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:24:21.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.736+0000 7f6d51ffb640 1 -- 172.21.15.12:0/1939848864 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55fe4b9f83a0 con 0x55fe4bb11f60 2024-04-22T13:24:21.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.736+0000 7f6d51ffb640 10 client.15771 handle_command_reply: tid=0 2024-04-22T13:24:21.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.736+0000 7f6d73267740 1 client.15771 shutdown 2024-04-22T13:24:21.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.736+0000 7f6d73267740 1 -- 172.21.15.12:0/1939848864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe4ba95ce0 msgr2=0x55fe4ba90240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.736+0000 7f6d73267740 1 --2- 172.21.15.12:0/1939848864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe4ba95ce0 0x55fe4ba90240 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f6d60037040 tx=0x7f6d60005cf0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.737+0000 7f6d73267740 1 -- 172.21.15.12:0/1939848864 shutdown_connections 2024-04-22T13:24:21.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.737+0000 7f6d73267740 1 --2- 172.21.15.12:0/1939848864 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55fe4bb11f60 0x55fe4bb10c70 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.737+0000 7f6d73267740 1 --2- 172.21.15.12:0/1939848864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe4ba95ce0 0x55fe4ba90240 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.737+0000 7f6d73267740 1 --2- 172.21.15.12:0/1939848864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe4ba8d160 0x55fe4ba8fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.737+0000 7f6d73267740 1 --2- 172.21.15.12:0/1939848864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe4ba8c7b0 0x55fe4ba8f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.737+0000 7f6d73267740 1 -- 172.21.15.12:0/1939848864 >> 172.21.15.12:0/1939848864 conn(0x55fe4ba86a60 msgr2=0x55fe4ba913a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:21.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.737+0000 7f6d51ffb640 0 client.15771 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:24:21.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.737+0000 7f6d73267740 1 -- 172.21.15.12:0/1939848864 shutdown_connections 2024-04-22T13:24:21.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.737+0000 7f6d73267740 1 -- 172.21.15.12:0/1939848864 wait complete. 2024-04-22T13:24:21.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.737+0000 7f6d73267740 20 client.15771 trim_cache size 0 max 16384 2024-04-22T13:24:21.739 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:24:21.739 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "0aede5ea-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:24:21.739 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 2, 2024-04-22T13:24:21.739 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24448, 2024-04-22T13:24:21.740 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-04-22T13:24:21.740 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-04-22T13:24:21.740 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-04-22T13:24:21.740 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 132.36296190600001, 2024-04-22T13:24:21.740 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 28, 2024-04-22T13:24:21.740 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 90, 2024-04-22T13:24:21.740 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 89, 2024-04-22T13:24:21.740 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 158.81069463200001 2024-04-22T13:24:21.740 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:24:21.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.737+0000 7f6d7066d640 1 -- 172.21.15.12:0/3274028073 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6d540a3e70 msgr2=0x7f6d540a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.737+0000 7f6d7066d640 1 --2- 172.21.15.12:0/3274028073 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6d540a3e70 0x7f6d540a6330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6d60009a60 tx=0x7f6d6002e040 comp rx=0 tx=0).stop 2024-04-22T13:24:21.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.737+0000 7f6d7066d640 1 -- 172.21.15.12:0/3274028073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d681031d0 msgr2=0x7f6d68100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.737+0000 7f6d7066d640 1 --2- 172.21.15.12:0/3274028073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d681031d0 0x7f6d68100e20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400d6d0 tx=0x7f6d6400db90 comp rx=0 tx=0).stop 2024-04-22T13:24:21.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.737+0000 7f6d7066d640 1 -- 172.21.15.12:0/3274028073 shutdown_connections 2024-04-22T13:24:21.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.737+0000 7f6d7066d640 1 --2- 172.21.15.12:0/3274028073 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6d540a3e70 0x7f6d540a6330 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.737+0000 7f6d7066d640 1 --2- 172.21.15.12:0/3274028073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d6810c510 0x7f6d681018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.737+0000 7f6d7066d640 1 --2- 172.21.15.12:0/3274028073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d68103af0 0x7f6d68101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.737+0000 7f6d7066d640 1 --2- 172.21.15.12:0/3274028073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d681031d0 0x7f6d68100e20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.737+0000 7f6d7066d640 1 -- 172.21.15.12:0/3274028073 >> 172.21.15.12:0/3274028073 conn(0x7f6d68077f50 msgr2=0x7f6d680fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:21.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.737+0000 7f6d7066d640 1 -- 172.21.15.12:0/3274028073 shutdown_connections 2024-04-22T13:24:21.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.737+0000 7f6d7066d640 1 -- 172.21.15.12:0/3274028073 wait complete. 2024-04-22T13:24:21.745 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-22T13:24:21.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.852+0000 7fc288a93640 1 -- 172.21.15.12:0/1847768505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2841021a0 msgr2=0x7fc284102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.852+0000 7fc288a93640 1 --2- 172.21.15.12:0/1847768505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2841021a0 0x7fc284102580 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc274009900 tx=0x7fc27402bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:21.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.852+0000 7fc288a93640 1 -- 172.21.15.12:0/1847768505 shutdown_connections 2024-04-22T13:24:21.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.852+0000 7fc288a93640 1 --2- 172.21.15.12:0/1847768505 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc284107270 0x7fc28410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.852+0000 7fc288a93640 1 --2- 172.21.15.12:0/1847768505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc284102b50 0x7fc284106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.852+0000 7fc288a93640 1 --2- 172.21.15.12:0/1847768505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2841021a0 0x7fc284102580 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.852+0000 7fc288a93640 1 -- 172.21.15.12:0/1847768505 >> 172.21.15.12:0/1847768505 conn(0x7fc284079350 msgr2=0x7fc284079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:21.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.852+0000 7fc288a93640 1 -- 172.21.15.12:0/1847768505 shutdown_connections 2024-04-22T13:24:21.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.852+0000 7fc288a93640 1 -- 172.21.15.12:0/1847768505 wait complete. 2024-04-22T13:24:21.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.853+0000 7fc288a93640 1 Processor -- start 2024-04-22T13:24:21.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.853+0000 7fc288a93640 1 -- start start 2024-04-22T13:24:21.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.853+0000 7fc288a93640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2841021a0 0x7fc28419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.853+0000 7fc288a93640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc284107270 0x7fc28419e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.853+0000 7fc288a93640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc284198590 0x7fc284198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.853+0000 7fc288a93640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc2841121b0 con 0x7fc2841021a0 2024-04-22T13:24:21.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.853+0000 7fc288a93640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc284112030 con 0x7fc284198590 2024-04-22T13:24:21.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.853+0000 7fc288a93640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc284112330 con 0x7fc284107270 2024-04-22T13:24:21.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.853+0000 7fc28259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2841021a0 0x7fc28419e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.854+0000 7fc28259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2841021a0 0x7fc28419e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45070/0 (socket says 172.21.15.12:45070) 2024-04-22T13:24:21.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.854+0000 7fc28259c640 1 -- 172.21.15.12:0/634675691 learned_addr learned my addr 172.21.15.12:0/634675691 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:21.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.854+0000 7fc282d9d640 1 --2- 172.21.15.12:0/634675691 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc284198590 0x7fc284198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.854+0000 7fc281d9b640 1 --2- 172.21.15.12:0/634675691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc284107270 0x7fc28419e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.854+0000 7fc28259c640 1 -- 172.21.15.12:0/634675691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc284107270 msgr2=0x7fc28419e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.854+0000 7fc28259c640 1 --2- 172.21.15.12:0/634675691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc284107270 0x7fc28419e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.854+0000 7fc28259c640 1 -- 172.21.15.12:0/634675691 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc284198590 msgr2=0x7fc284198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.854+0000 7fc28259c640 1 --2- 172.21.15.12:0/634675691 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc284198590 0x7fc284198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.854+0000 7fc28259c640 1 -- 172.21.15.12:0/634675691 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2841990b0 con 0x7fc2841021a0 2024-04-22T13:24:21.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.854+0000 7fc28259c640 1 --2- 172.21.15.12:0/634675691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2841021a0 0x7fc28419e3e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc274005940 tx=0x7fc2740045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:21.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.854+0000 7fc2737fe640 1 -- 172.21.15.12:0/634675691 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc274043020 con 0x7fc2841021a0 2024-04-22T13:24:21.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.854+0000 7fc288a93640 1 -- 172.21.15.12:0/634675691 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc284199340 con 0x7fc2841021a0 2024-04-22T13:24:21.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.854+0000 7fc2737fe640 1 -- 172.21.15.12:0/634675691 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc27402caf0 con 0x7fc2841021a0 2024-04-22T13:24:21.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.854+0000 7fc2737fe640 1 -- 172.21.15.12:0/634675691 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc274003af0 con 0x7fc2841021a0 2024-04-22T13:24:21.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.854+0000 7fc288a93640 1 -- 172.21.15.12:0/634675691 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2841a55e0 con 0x7fc2841021a0 2024-04-22T13:24:21.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.857+0000 7fc28b767740 1 Processor -- start 2024-04-22T13:24:21.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.857+0000 7fc28b767740 1 -- start start 2024-04-22T13:24:21.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.857+0000 7fc28b767740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570d81867b0 0x5570d8186b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.857+0000 7fc28b767740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570d8187160 0x5570d818f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.857+0000 7fc28b767740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570d818fce0 0x5570d8192130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.857+0000 7fc28b767740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5570d80eab10 con 0x5570d8187160 2024-04-22T13:24:21.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.857+0000 7fc28b767740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5570d8088240 con 0x5570d81867b0 2024-04-22T13:24:21.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.857+0000 7fc28b767740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5570d80e6ad0 con 0x5570d818fce0 2024-04-22T13:24:21.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.857+0000 7fc282d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570d81867b0 0x5570d8186b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.857+0000 7fc282d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570d81867b0 0x5570d8186b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:42728/0 (socket says 172.21.15.12:42728) 2024-04-22T13:24:21.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.857+0000 7fc282d9d640 1 -- 172.21.15.12:0/1060362521 learned_addr learned my addr 172.21.15.12:0/1060362521 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:21.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.857+0000 7fc281d9b640 1 --2- 172.21.15.12:0/1060362521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570d818fce0 0x5570d8192130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.857+0000 7fc2737fe640 1 -- 172.21.15.12:0/634675691 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fc274046020 con 0x7fc2841021a0 2024-04-22T13:24:21.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.858+0000 7fc28259c640 1 --2- 172.21.15.12:0/1060362521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570d8187160 0x5570d818f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.858+0000 7fc282d9d640 1 -- 172.21.15.12:0/1060362521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570d818fce0 msgr2=0x5570d8192130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.858+0000 7fc282d9d640 1 --2- 172.21.15.12:0/1060362521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570d818fce0 0x5570d8192130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.858+0000 7fc282d9d640 1 -- 172.21.15.12:0/1060362521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570d8187160 msgr2=0x5570d818f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.858+0000 7fc282d9d640 1 --2- 172.21.15.12:0/1060362521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570d8187160 0x5570d818f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.858+0000 7fc282d9d640 1 -- 172.21.15.12:0/1060362521 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570d7dfd2d0 con 0x5570d81867b0 2024-04-22T13:24:21.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.858+0000 7fc281d9b640 1 --2- 172.21.15.12:0/1060362521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570d818fce0 0x5570d8192130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:24:21.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.858+0000 7fc28259c640 1 --2- 172.21.15.12:0/1060362521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570d8187160 0x5570d818f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:21.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.858+0000 7fc282d9d640 1 --2- 172.21.15.12:0/1060362521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570d81867b0 0x5570d8186b90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc27c009f40 tx=0x7fc27c010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18eccc65610b610f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:21.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.858+0000 7fc2737fe640 1 --2- 172.21.15.12:0/634675691 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc26c0a3e40 0x7fc26c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.858+0000 7fc270ff9640 1 -- 172.21.15.12:0/1060362521 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c003b70 con 0x5570d81867b0 2024-04-22T13:24:21.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.858+0000 7fc270ff9640 1 -- 172.21.15.12:0/1060362521 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc27c003d10 con 0x5570d81867b0 2024-04-22T13:24:21.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.858+0000 7fc270ff9640 1 -- 172.21.15.12:0/1060362521 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c011960 con 0x5570d81867b0 2024-04-22T13:24:21.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.858+0000 7fc28259c640 1 --2- 172.21.15.12:0/634675691 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc26c0a3e40 0x7fc26c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.858+0000 7fc2737fe640 1 -- 172.21.15.12:0/634675691 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fc2740f2b80 con 0x7fc2841021a0 2024-04-22T13:24:21.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.859+0000 7fc28b767740 1 -- 172.21.15.12:0/1060362521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570d81867b0 msgr2=0x5570d8186b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.859+0000 7fc28b767740 1 --2- 172.21.15.12:0/1060362521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570d81867b0 0x5570d8186b90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc27c009f40 tx=0x7fc27c010860 comp rx=0 tx=0).stop 2024-04-22T13:24:21.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.859+0000 7fc28259c640 1 --2- 172.21.15.12:0/634675691 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc26c0a3e40 0x7fc26c0a6300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc2740f0310 tx=0x7fc27402bc30 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:21.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.859+0000 7fc28b767740 1 -- 172.21.15.12:0/1060362521 shutdown_connections 2024-04-22T13:24:21.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.859+0000 7fc28b767740 1 --2- 172.21.15.12:0/1060362521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570d818fce0 0x5570d8192130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.859+0000 7fc28b767740 1 --2- 172.21.15.12:0/1060362521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570d8187160 0x5570d818f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.859+0000 7fc28b767740 1 --2- 172.21.15.12:0/1060362521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570d81867b0 0x5570d8186b90 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.859+0000 7fc28b767740 1 -- 172.21.15.12:0/1060362521 >> 172.21.15.12:0/1060362521 conn(0x5570d8180a60 msgr2=0x5570d8180e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:21.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.859+0000 7fc28b767740 1 -- 172.21.15.12:0/1060362521 shutdown_connections 2024-04-22T13:24:21.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.859+0000 7fc28b767740 1 -- 172.21.15.12:0/1060362521 wait complete. 2024-04-22T13:24:21.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.859+0000 7fc28b767740 1 -- start start 2024-04-22T13:24:21.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.860+0000 7fc28b767740 1 Processor -- start 2024-04-22T13:24:21.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.860+0000 7fc28b767740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570d81867b0 0x5570d818ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.860+0000 7fc28b767740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570d8187160 0x5570d818d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.860+0000 7fc28b767740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570d818fce0 0x5570d818d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.860+0000 7fc28b767740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5570d8192c50 con 0x5570d81867b0 2024-04-22T13:24:21.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.860+0000 7fc28b767740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5570d8073190 con 0x5570d818fce0 2024-04-22T13:24:21.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.860+0000 7fc28b767740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5570d8192dd0 con 0x5570d8187160 2024-04-22T13:24:21.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.860+0000 7fc28b767740 10 client.0 _ll_register_callbacks cb 0x5570d806e8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:21.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.860+0000 7fc282d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570d81867b0 0x5570d818ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.860+0000 7fc282d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570d81867b0 0x5570d818ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45096/0 (socket says 172.21.15.12:45096) 2024-04-22T13:24:21.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.860+0000 7fc282d9d640 1 -- 172.21.15.12:0/2379698919 learned_addr learned my addr 172.21.15.12:0/2379698919 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:21.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.860+0000 7fc281d9b640 1 --2- 172.21.15.12:0/2379698919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570d8187160 0x5570d818d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.860+0000 7fc282d9d640 1 --2- 172.21.15.12:0/2379698919 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570d818fce0 0x5570d818d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.860+0000 7fc282d9d640 1 -- 172.21.15.12:0/2379698919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570d8187160 msgr2=0x5570d818d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.860+0000 7fc282d9d640 1 --2- 172.21.15.12:0/2379698919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570d8187160 0x5570d818d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.860+0000 7fc282d9d640 1 -- 172.21.15.12:0/2379698919 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570d818fce0 msgr2=0x5570d818d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.860+0000 7fc282d9d640 1 --2- 172.21.15.12:0/2379698919 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570d818fce0 0x5570d818d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.860+0000 7fc282d9d640 1 -- 172.21.15.12:0/2379698919 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570d7ec76b0 con 0x5570d81867b0 2024-04-22T13:24:21.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.861+0000 7fc282d9d640 1 --2- 172.21.15.12:0/2379698919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570d81867b0 0x5570d818ccd0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc27c009f10 tx=0x7fc27c0098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8648f934a8a03bf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:21.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.861+0000 7fc269ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:21.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.861+0000 7fc269ffb640 1 -- 172.21.15.12:0/2379698919 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c004580 con 0x5570d81867b0 2024-04-22T13:24:21.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.861+0000 7fc269ffb640 1 -- 172.21.15.12:0/2379698919 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fc27c011680 con 0x5570d81867b0 2024-04-22T13:24:21.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.861+0000 7fc28b767740 1 -- 172.21.15.12:0/2379698919 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc27c003d10 con 0x5570d81867b0 2024-04-22T13:24:21.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.861+0000 7fc269ffb640 1 -- 172.21.15.12:0/2379698919 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc27c003b40 con 0x5570d81867b0 2024-04-22T13:24:21.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.861+0000 7fc28b767740 10 client.15780 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:21.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.861+0000 7fc28b767740 1 -- 172.21.15.12:0/2379698919 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5570d819eeb0 con 0x5570d81867b0 2024-04-22T13:24:21.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.861+0000 7fc269ffb640 1 -- 172.21.15.12:0/2379698919 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc27c003d10 con 0x5570d81867b0 2024-04-22T13:24:21.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.861+0000 7fc269ffb640 1 -- 172.21.15.12:0/2379698919 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc27c0055e0 con 0x5570d81867b0 2024-04-22T13:24:21.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.861+0000 7fc28b767740 10 client.15780 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:21.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.861+0000 7fc28b767740 10 client.15780 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-22T13:24:21.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.862+0000 7fc28b767740 20 client.15780 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:21.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.862+0000 7fc28b767740 1 --2- 172.21.15.12:0/2379698919 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5570d81a1070 0x5570d81a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.862+0000 7fc28b767740 4 client.15780 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:24:21.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.862+0000 7fc28b767740 1 -- 172.21.15.12:0/2379698919 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570d80f8530 con 0x5570d81a1070 2024-04-22T13:24:21.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.862+0000 7fc281d9b640 1 --2- 172.21.15.12:0/2379698919 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5570d81a1070 0x5570d81a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.862+0000 7fc281d9b640 1 --2- 172.21.15.12:0/2379698919 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5570d81a1070 0x5570d81a24e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9b3b8fabd9bef67e server_cookie=a0044011ee062ae2 in_seq=0 out_seq=0 2024-04-22T13:24:21.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.862+0000 7fc269ffb640 10 client.15780 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:24:21.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.863+0000 7fc269ffb640 1 -- 172.21.15.12:0/2379698919 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5570d80f8530 con 0x5570d81a1070 2024-04-22T13:24:21.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.863+0000 7fc269ffb640 10 client.15780 handle_command_reply: tid=0 2024-04-22T13:24:21.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.863+0000 7fc28b767740 1 client.15780 shutdown 2024-04-22T13:24:21.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.863+0000 7fc28b767740 1 -- 172.21.15.12:0/2379698919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570d81867b0 msgr2=0x5570d818ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.863+0000 7fc28b767740 1 --2- 172.21.15.12:0/2379698919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570d81867b0 0x5570d818ccd0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc27c009f10 tx=0x7fc27c0098e0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.863+0000 7fc28b767740 1 -- 172.21.15.12:0/2379698919 shutdown_connections 2024-04-22T13:24:21.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.863+0000 7fc28b767740 1 --2- 172.21.15.12:0/2379698919 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5570d81a1070 0x5570d81a24e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.863+0000 7fc28b767740 1 --2- 172.21.15.12:0/2379698919 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570d818fce0 0x5570d818d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.863+0000 7fc28b767740 1 --2- 172.21.15.12:0/2379698919 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570d8187160 0x5570d818d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.863+0000 7fc28b767740 1 --2- 172.21.15.12:0/2379698919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570d81867b0 0x5570d818ccd0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.863+0000 7fc28b767740 1 -- 172.21.15.12:0/2379698919 >> 172.21.15.12:0/2379698919 conn(0x5570d8180a60 msgr2=0x5570d8182bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:21.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.863+0000 7fc269ffb640 0 client.15780 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:24:21.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.863+0000 7fc28b767740 1 -- 172.21.15.12:0/2379698919 shutdown_connections 2024-04-22T13:24:21.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.863+0000 7fc28b767740 1 -- 172.21.15.12:0/2379698919 wait complete. 2024-04-22T13:24:21.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.863+0000 7fc28b767740 20 client.15780 trim_cache size 0 max 16384 2024-04-22T13:24:21.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.883+0000 7fc28b767740 1 Processor -- start 2024-04-22T13:24:21.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.883+0000 7fc28b767740 1 -- start start 2024-04-22T13:24:21.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.883+0000 7fc28b767740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570d81867b0 0x5570d81a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.883+0000 7fc28b767740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570d8187160 0x5570d818f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.883+0000 7fc28b767740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570d818fce0 0x5570d817b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.883+0000 7fc28b767740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5570d81a0820 con 0x5570d818fce0 2024-04-22T13:24:21.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.883+0000 7fc28b767740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5570d81a0ef0 con 0x5570d8187160 2024-04-22T13:24:21.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.883+0000 7fc28b767740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5570d81a15c0 con 0x5570d81867b0 2024-04-22T13:24:21.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.883+0000 7fc282d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570d818fce0 0x5570d817b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.883+0000 7fc282d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570d818fce0 0x5570d817b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45100/0 (socket says 172.21.15.12:45100) 2024-04-22T13:24:21.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.883+0000 7fc282d9d640 1 -- 172.21.15.12:0/2424358765 learned_addr learned my addr 172.21.15.12:0/2424358765 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:21.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.884+0000 7fc281d9b640 1 --2- 172.21.15.12:0/2424358765 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570d8187160 0x5570d818f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.884+0000 7fc282d9d640 1 --2- 172.21.15.12:0/2424358765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570d81867b0 0x5570d81a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.884+0000 7fc282d9d640 1 -- 172.21.15.12:0/2424358765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570d81867b0 msgr2=0x5570d81a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.884+0000 7fc282d9d640 1 --2- 172.21.15.12:0/2424358765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570d81867b0 0x5570d81a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.884+0000 7fc282d9d640 1 -- 172.21.15.12:0/2424358765 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570d8187160 msgr2=0x5570d818f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.884+0000 7fc282d9d640 1 --2- 172.21.15.12:0/2424358765 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570d8187160 0x5570d818f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.884+0000 7fc282d9d640 1 -- 172.21.15.12:0/2424358765 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570d81817f0 con 0x5570d818fce0 2024-04-22T13:24:21.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.884+0000 7fc281d9b640 1 --2- 172.21.15.12:0/2424358765 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570d8187160 0x5570d818f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:24:21.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.884+0000 7fc282d9d640 1 --2- 172.21.15.12:0/2424358765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570d818fce0 0x5570d817b800 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc27c004800 tx=0x7fc27c01e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5c4d779a55990a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:21.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.884+0000 7fc26bfff640 1 -- 172.21.15.12:0/2424358765 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c003ce0 con 0x5570d818fce0 2024-04-22T13:24:21.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.884+0000 7fc26bfff640 1 -- 172.21.15.12:0/2424358765 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc27c020020 con 0x5570d818fce0 2024-04-22T13:24:21.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.884+0000 7fc26bfff640 1 -- 172.21.15.12:0/2424358765 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c005000 con 0x5570d818fce0 2024-04-22T13:24:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.885+0000 7fc28b767740 1 -- 172.21.15.12:0/2424358765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570d818fce0 msgr2=0x5570d817b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.885+0000 7fc28b767740 1 --2- 172.21.15.12:0/2424358765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570d818fce0 0x5570d817b800 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc27c004800 tx=0x7fc27c01e040 comp rx=0 tx=0).stop 2024-04-22T13:24:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.885+0000 7fc28b767740 1 -- 172.21.15.12:0/2424358765 shutdown_connections 2024-04-22T13:24:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.885+0000 7fc28b767740 1 --2- 172.21.15.12:0/2424358765 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570d818fce0 0x5570d817b800 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.885+0000 7fc28b767740 1 --2- 172.21.15.12:0/2424358765 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570d8187160 0x5570d818f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.885+0000 7fc28b767740 1 --2- 172.21.15.12:0/2424358765 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570d81867b0 0x5570d81a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.885+0000 7fc28b767740 1 -- 172.21.15.12:0/2424358765 >> 172.21.15.12:0/2424358765 conn(0x5570d8180a60 msgr2=0x5570d8187540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.885+0000 7fc28b767740 1 -- 172.21.15.12:0/2424358765 shutdown_connections 2024-04-22T13:24:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.885+0000 7fc28b767740 1 -- 172.21.15.12:0/2424358765 wait complete. 2024-04-22T13:24:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.886+0000 7fc28b767740 1 -- start start 2024-04-22T13:24:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.886+0000 7fc28b767740 1 Processor -- start 2024-04-22T13:24:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.886+0000 7fc28b767740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570d81867b0 0x5570d81aac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.886+0000 7fc28b767740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570d8187160 0x5570d8189940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.886+0000 7fc28b767740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570d818fce0 0x5570d8189fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.886+0000 7fc28b767740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5570d81a2360 con 0x5570d818fce0 2024-04-22T13:24:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.886+0000 7fc28b767740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5570d81a1c90 con 0x5570d81867b0 2024-04-22T13:24:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.886+0000 7fc28b767740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5570d81a2a30 con 0x5570d8187160 2024-04-22T13:24:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.886+0000 7fc28b767740 10 client.0 _ll_register_callbacks cb 0x7fc2780092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.886+0000 7fc282d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570d818fce0 0x5570d8189fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.886+0000 7fc282d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570d818fce0 0x5570d8189fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45106/0 (socket says 172.21.15.12:45106) 2024-04-22T13:24:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.886+0000 7fc282d9d640 1 -- 172.21.15.12:0/272143429 learned_addr learned my addr 172.21.15.12:0/272143429 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.886+0000 7fc282d9d640 1 --2- 172.21.15.12:0/272143429 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570d81867b0 0x5570d81aac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.886+0000 7fc281d9b640 1 --2- 172.21.15.12:0/272143429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570d8187160 0x5570d8189940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.887+0000 7fc282d9d640 1 -- 172.21.15.12:0/272143429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570d8187160 msgr2=0x5570d8189940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.887+0000 7fc282d9d640 1 --2- 172.21.15.12:0/272143429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570d8187160 0x5570d8189940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.887+0000 7fc282d9d640 1 -- 172.21.15.12:0/272143429 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570d81867b0 msgr2=0x5570d81aac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:21.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.887+0000 7fc282d9d640 1 --2- 172.21.15.12:0/272143429 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570d81867b0 0x5570d81aac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.887+0000 7fc282d9d640 1 -- 172.21.15.12:0/272143429 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570d81a8760 con 0x5570d818fce0 2024-04-22T13:24:21.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.887+0000 7fc282d9d640 1 --2- 172.21.15.12:0/272143429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570d818fce0 0x5570d8189fc0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc27c004f50 tx=0x7fc27c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d50a08c1b98fa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:21.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.887+0000 7fc269ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:21.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.887+0000 7fc269ffb640 1 -- 172.21.15.12:0/272143429 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c043040 con 0x5570d818fce0 2024-04-22T13:24:21.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.887+0000 7fc269ffb640 1 -- 172.21.15.12:0/272143429 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fc27c004480 con 0x5570d818fce0 2024-04-22T13:24:21.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.887+0000 7fc28b767740 1 -- 172.21.15.12:0/272143429 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc27c020020 con 0x5570d818fce0 2024-04-22T13:24:21.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.887+0000 7fc269ffb640 1 -- 172.21.15.12:0/272143429 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc27c005000 con 0x5570d818fce0 2024-04-22T13:24:21.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.887+0000 7fc28b767740 10 client.15786 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:21.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.887+0000 7fc28b767740 1 -- 172.21.15.12:0/272143429 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5570d8207170 con 0x5570d818fce0 2024-04-22T13:24:21.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.887+0000 7fc269ffb640 1 -- 172.21.15.12:0/272143429 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc27c020020 con 0x5570d818fce0 2024-04-22T13:24:21.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.888+0000 7fc269ffb640 1 -- 172.21.15.12:0/272143429 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc27c02c6b0 con 0x5570d818fce0 2024-04-22T13:24:21.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.888+0000 7fc28b767740 10 client.15786 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:21.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.888+0000 7fc28b767740 10 client.15786 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-22T13:24:21.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.888+0000 7fc28b767740 20 client.15786 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:21.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.888+0000 7fc28b767740 1 --2- 172.21.15.12:0/272143429 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5570d820b9f0 0x5570d820a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:21.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.888+0000 7fc28b767740 4 client.15786 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T13:24:21.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.888+0000 7fc28b767740 1 -- 172.21.15.12:0/272143429 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "status"}) -- 0x5570d80f23a0 con 0x5570d820b9f0 2024-04-22T13:24:21.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.888+0000 7fc281d9b640 1 --2- 172.21.15.12:0/272143429 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5570d820b9f0 0x5570d820a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:21.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.889+0000 7fc269ffb640 10 client.15786 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:24:21.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.889+0000 7fc281d9b640 1 --2- 172.21.15.12:0/272143429 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5570d820b9f0 0x5570d820a700 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=70fe0ba629670c21 server_cookie=56f012707ad645cb in_seq=0 out_seq=0 2024-04-22T13:24:21.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.889+0000 7fc269ffb640 1 -- 172.21.15.12:0/272143429 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5570d80f23a0 con 0x5570d820b9f0 2024-04-22T13:24:21.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.889+0000 7fc269ffb640 10 client.15786 handle_command_reply: tid=0 2024-04-22T13:24:21.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.889+0000 7fc28b767740 1 client.15786 shutdown 2024-04-22T13:24:21.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.889+0000 7fc28b767740 1 -- 172.21.15.12:0/272143429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570d818fce0 msgr2=0x5570d8189fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.889+0000 7fc28b767740 1 --2- 172.21.15.12:0/272143429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570d818fce0 0x5570d8189fc0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc27c004f50 tx=0x7fc27c009bf0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.889+0000 7fc28b767740 1 -- 172.21.15.12:0/272143429 shutdown_connections 2024-04-22T13:24:21.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.889+0000 7fc28b767740 1 --2- 172.21.15.12:0/272143429 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5570d820b9f0 0x5570d820a700 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.889+0000 7fc28b767740 1 --2- 172.21.15.12:0/272143429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570d818fce0 0x5570d8189fc0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.889+0000 7fc28b767740 1 --2- 172.21.15.12:0/272143429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570d8187160 0x5570d8189940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.889+0000 7fc269ffb640 0 client.15786 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:24:21.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.889+0000 7fc28b767740 1 --2- 172.21.15.12:0/272143429 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570d81867b0 0x5570d81aac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.889+0000 7fc28b767740 1 -- 172.21.15.12:0/272143429 >> 172.21.15.12:0/272143429 conn(0x5570d8180a60 msgr2=0x5570d817b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:21.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.889+0000 7fc28b767740 1 -- 172.21.15.12:0/272143429 shutdown_connections 2024-04-22T13:24:21.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.889+0000 7fc28b767740 1 -- 172.21.15.12:0/272143429 wait complete. 2024-04-22T13:24:21.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.889+0000 7fc28b767740 20 client.15786 trim_cache size 0 max 16384 2024-04-22T13:24:21.893 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:24:21.893 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "0aede5ea-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:24:21.893 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 1, 2024-04-22T13:24:21.893 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24484, 2024-04-22T13:24:21.893 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-04-22T13:24:21.893 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-04-22T13:24:21.893 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-04-22T13:24:21.893 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 133.501705915, 2024-04-22T13:24:21.893 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 28, 2024-04-22T13:24:21.893 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 90, 2024-04-22T13:24:21.893 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 89, 2024-04-22T13:24:21.893 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 150.87430137300001 2024-04-22T13:24:21.894 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:24:21.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.890+0000 7fc288a93640 1 -- 172.21.15.12:0/634675691 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc26c0a3e40 msgr2=0x7fc26c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.890+0000 7fc288a93640 1 --2- 172.21.15.12:0/634675691 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc26c0a3e40 0x7fc26c0a6300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc2740f0310 tx=0x7fc27402bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.890+0000 7fc288a93640 1 -- 172.21.15.12:0/634675691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2841021a0 msgr2=0x7fc28419e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.890+0000 7fc288a93640 1 --2- 172.21.15.12:0/634675691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2841021a0 0x7fc28419e3e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc274005940 tx=0x7fc2740045e0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.890+0000 7fc288a93640 1 -- 172.21.15.12:0/634675691 shutdown_connections 2024-04-22T13:24:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.890+0000 7fc288a93640 1 --2- 172.21.15.12:0/634675691 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc26c0a3e40 0x7fc26c0a6300 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.890+0000 7fc288a93640 1 --2- 172.21.15.12:0/634675691 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc284198590 0x7fc284198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.890+0000 7fc288a93640 1 --2- 172.21.15.12:0/634675691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc284107270 0x7fc28419e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.890+0000 7fc288a93640 1 --2- 172.21.15.12:0/634675691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc2841021a0 0x7fc28419e3e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.890+0000 7fc288a93640 1 -- 172.21.15.12:0/634675691 >> 172.21.15.12:0/634675691 conn(0x7fc284079350 msgr2=0x7fc284100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.890+0000 7fc288a93640 1 -- 172.21.15.12:0/634675691 shutdown_connections 2024-04-22T13:24:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:21.890+0000 7fc288a93640 1 -- 172.21.15.12:0/634675691 wait complete. 2024-04-22T13:24:21.898 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-22T13:24:21.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:21 smithi111 ceph-mon[31361]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-22T13:24:21.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:21 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2042523652' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:24:21.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:21 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/244274738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:22.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.004+0000 7fa11939d640 1 -- 172.21.15.12:0/4126924552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1141010a0 msgr2=0x7fa114101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:22.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.004+0000 7fa11939d640 1 --2- 172.21.15.12:0/4126924552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1141010a0 0x7fa114101480 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa104009860 tx=0x7fa10402bb70 comp rx=0 tx=0).stop 2024-04-22T13:24:22.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.004+0000 7fa11939d640 1 -- 172.21.15.12:0/4126924552 shutdown_connections 2024-04-22T13:24:22.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.004+0000 7fa11939d640 1 --2- 172.21.15.12:0/4126924552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa114106170 0x7fa11410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.004+0000 7fa11939d640 1 --2- 172.21.15.12:0/4126924552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa114101a50 0x7fa114105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.004+0000 7fa11939d640 1 --2- 172.21.15.12:0/4126924552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1141010a0 0x7fa114101480 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.004+0000 7fa11939d640 1 -- 172.21.15.12:0/4126924552 >> 172.21.15.12:0/4126924552 conn(0x7fa114077eb0 msgr2=0x7fa1140782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.004+0000 7fa11939d640 1 -- 172.21.15.12:0/4126924552 shutdown_connections 2024-04-22T13:24:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.004+0000 7fa11939d640 1 -- 172.21.15.12:0/4126924552 wait complete. 2024-04-22T13:24:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.005+0000 7fa11939d640 1 Processor -- start 2024-04-22T13:24:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.005+0000 7fa11939d640 1 -- start start 2024-04-22T13:24:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.005+0000 7fa11939d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1141010a0 0x7fa114102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.005+0000 7fa11939d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa114106170 0x7fa114103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.005+0000 7fa11939d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1141035b0 0x7fa1141a6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.005+0000 7fa11939d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa114111060 con 0x7fa1141010a0 2024-04-22T13:24:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.005+0000 7fa11939d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa114110ee0 con 0x7fa1141035b0 2024-04-22T13:24:22.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.005+0000 7fa11939d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa1141111e0 con 0x7fa114106170 2024-04-22T13:24:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.005+0000 7fa112ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1141010a0 0x7fa114102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.005+0000 7fa112ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1141010a0 0x7fa114102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45126/0 (socket says 172.21.15.12:45126) 2024-04-22T13:24:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.005+0000 7fa112ffd640 1 -- 172.21.15.12:0/3696440620 learned_addr learned my addr 172.21.15.12:0/3696440620 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.005+0000 7fa1137fe640 1 --2- 172.21.15.12:0/3696440620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1141035b0 0x7fa1141a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.005+0000 7fa10a7fc640 1 --2- 172.21.15.12:0/3696440620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa114106170 0x7fa114103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.005+0000 7fa112ffd640 1 -- 172.21.15.12:0/3696440620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa114106170 msgr2=0x7fa114103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.005+0000 7fa112ffd640 1 --2- 172.21.15.12:0/3696440620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa114106170 0x7fa114103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.005+0000 7fa112ffd640 1 -- 172.21.15.12:0/3696440620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1141035b0 msgr2=0x7fa1141a6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.006+0000 7fa112ffd640 1 --2- 172.21.15.12:0/3696440620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1141035b0 0x7fa1141a6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.006+0000 7fa112ffd640 1 -- 172.21.15.12:0/3696440620 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1141a6fe0 con 0x7fa1141010a0 2024-04-22T13:24:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.006+0000 7fa1137fe640 1 --2- 172.21.15.12:0/3696440620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1141035b0 0x7fa1141a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:24:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.006+0000 7fa112ffd640 1 --2- 172.21.15.12:0/3696440620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1141010a0 0x7fa114102b30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa104037040 tx=0x7fa104035a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.006+0000 7fa110ff9640 1 -- 172.21.15.12:0/3696440620 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa104041020 con 0x7fa1141010a0 2024-04-22T13:24:22.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.006+0000 7fa11939d640 1 -- 172.21.15.12:0/3696440620 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1141a7270 con 0x7fa1141010a0 2024-04-22T13:24:22.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.006+0000 7fa110ff9640 1 -- 172.21.15.12:0/3696440620 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa104035e30 con 0x7fa1141010a0 2024-04-22T13:24:22.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.006+0000 7fa110ff9640 1 -- 172.21.15.12:0/3696440620 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa104039040 con 0x7fa1141010a0 2024-04-22T13:24:22.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.006+0000 7fa11939d640 1 -- 172.21.15.12:0/3696440620 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1141a7760 con 0x7fa1141010a0 2024-04-22T13:24:22.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.008+0000 7fa110ff9640 1 -- 172.21.15.12:0/3696440620 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fa104044020 con 0x7fa1141010a0 2024-04-22T13:24:22.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.009+0000 7fa11bf8c740 1 Processor -- start 2024-04-22T13:24:22.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.009+0000 7fa11bf8c740 1 -- start start 2024-04-22T13:24:22.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.009+0000 7fa11bf8c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca7139e7b0 0x55ca7139eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:22.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.009+0000 7fa11bf8c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ca7139f160 0x55ca713a7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:22.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.009+0000 7fa11bf8c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca713a7ce0 0x55ca713aa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:22.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.009+0000 7fa11bf8c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ca71302b10 con 0x55ca7139e7b0 2024-04-22T13:24:22.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.009+0000 7fa11bf8c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ca712a0240 con 0x55ca7139f160 2024-04-22T13:24:22.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.009+0000 7fa11bf8c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ca712fead0 con 0x55ca713a7ce0 2024-04-22T13:24:22.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.009+0000 7fa1137fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca7139e7b0 0x55ca7139eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:22.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.009+0000 7fa1137fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca7139e7b0 0x55ca7139eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45132/0 (socket says 172.21.15.12:45132) 2024-04-22T13:24:22.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.009+0000 7fa110ff9640 1 --2- 172.21.15.12:0/3696440620 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa1000a3e40 0x7fa1000a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:22.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.009+0000 7fa1137fe640 1 -- 172.21.15.12:0/2252094634 learned_addr learned my addr 172.21.15.12:0/2252094634 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:22.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.009+0000 7fa10a7fc640 1 --2- 172.21.15.12:0/2252094634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca713a7ce0 0x55ca713aa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:22.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.009+0000 7fa112ffd640 1 --2- 172.21.15.12:0/2252094634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ca7139f160 0x55ca713a7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:22.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.009+0000 7fa1137fe640 1 --2- 172.21.15.12:0/3696440620 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa1000a3e40 0x7fa1000a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:22.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.009+0000 7fa110ff9640 1 -- 172.21.15.12:0/3696440620 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fa1040f2340 con 0x7fa1141010a0 2024-04-22T13:24:22.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.010+0000 7fa1137fe640 1 -- 172.21.15.12:0/2252094634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca713a7ce0 msgr2=0x55ca713aa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:22.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.010+0000 7fa1137fe640 1 --2- 172.21.15.12:0/2252094634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca713a7ce0 0x55ca713aa130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.010+0000 7fa1137fe640 1 -- 172.21.15.12:0/2252094634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ca7139f160 msgr2=0x55ca713a7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:22.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.010+0000 7fa1137fe640 1 --2- 172.21.15.12:0/2252094634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ca7139f160 0x55ca713a7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.010+0000 7fa1137fe640 1 -- 172.21.15.12:0/2252094634 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca710152d0 con 0x55ca7139e7b0 2024-04-22T13:24:22.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.010+0000 7fa10a7fc640 1 --2- 172.21.15.12:0/2252094634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca713a7ce0 0x55ca713aa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:24:22.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.010+0000 7fa1137fe640 1 --2- 172.21.15.12:0/2252094634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca7139e7b0 0x55ca7139eb90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa10c009a60 tx=0x7fa10c00c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb83da47ecfbe6f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:22.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.010+0000 7fa1137fe640 1 --2- 172.21.15.12:0/3696440620 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa1000a3e40 0x7fa1000a6300 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa10c023000 tx=0x7fa10c01f000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:22.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.010+0000 7fa1097fa640 1 -- 172.21.15.12:0/2252094634 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10c022020 con 0x55ca7139e7b0 2024-04-22T13:24:22.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.010+0000 7fa1097fa640 1 -- 172.21.15.12:0/2252094634 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa10c01e480 con 0x55ca7139e7b0 2024-04-22T13:24:22.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.010+0000 7fa1097fa640 1 -- 172.21.15.12:0/2252094634 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10c017040 con 0x55ca7139e7b0 2024-04-22T13:24:22.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.011+0000 7fa11bf8c740 1 -- 172.21.15.12:0/2252094634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca7139e7b0 msgr2=0x55ca7139eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:22.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.011+0000 7fa11bf8c740 1 --2- 172.21.15.12:0/2252094634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca7139e7b0 0x55ca7139eb90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa10c009a60 tx=0x7fa10c00c740 comp rx=0 tx=0).stop 2024-04-22T13:24:22.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.011+0000 7fa11bf8c740 1 -- 172.21.15.12:0/2252094634 shutdown_connections 2024-04-22T13:24:22.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.011+0000 7fa11bf8c740 1 --2- 172.21.15.12:0/2252094634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca713a7ce0 0x55ca713aa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.011+0000 7fa11bf8c740 1 --2- 172.21.15.12:0/2252094634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ca7139f160 0x55ca713a7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.011+0000 7fa11bf8c740 1 --2- 172.21.15.12:0/2252094634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca7139e7b0 0x55ca7139eb90 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.011+0000 7fa11bf8c740 1 -- 172.21.15.12:0/2252094634 >> 172.21.15.12:0/2252094634 conn(0x55ca71398a60 msgr2=0x55ca71398e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:22.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.011+0000 7fa11bf8c740 1 -- 172.21.15.12:0/2252094634 shutdown_connections 2024-04-22T13:24:22.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.011+0000 7fa11bf8c740 1 -- 172.21.15.12:0/2252094634 wait complete. 2024-04-22T13:24:22.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.011+0000 7fa11bf8c740 1 -- start start 2024-04-22T13:24:22.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.011+0000 7fa11bf8c740 1 Processor -- start 2024-04-22T13:24:22.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.011+0000 7fa11bf8c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ca7139e7b0 0x55ca713a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:22.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.011+0000 7fa11bf8c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca7139f160 0x55ca713a5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:22.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.011+0000 7fa11bf8c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca713a7ce0 0x55ca713a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:22.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.011+0000 7fa11bf8c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ca713aac50 con 0x55ca7139f160 2024-04-22T13:24:22.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.011+0000 7fa11bf8c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ca7128b190 con 0x55ca7139e7b0 2024-04-22T13:24:22.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.011+0000 7fa11bf8c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ca713aadd0 con 0x55ca713a7ce0 2024-04-22T13:24:22.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.012+0000 7fa11bf8c740 10 client.0 _ll_register_callbacks cb 0x55ca712868b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:22.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.012+0000 7fa10a7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca7139f160 0x55ca713a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:22.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.012+0000 7fa10a7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca7139f160 0x55ca713a5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45146/0 (socket says 172.21.15.12:45146) 2024-04-22T13:24:22.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.012+0000 7fa10a7fc640 1 -- 172.21.15.12:0/2728290937 learned_addr learned my addr 172.21.15.12:0/2728290937 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:22.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.012+0000 7fa112ffd640 1 --2- 172.21.15.12:0/2728290937 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ca7139e7b0 0x55ca713a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:22.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.012+0000 7fa1137fe640 1 --2- 172.21.15.12:0/2728290937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca713a7ce0 0x55ca713a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:22.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.012+0000 7fa10a7fc640 1 -- 172.21.15.12:0/2728290937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca713a7ce0 msgr2=0x55ca713a57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:22.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.012+0000 7fa10a7fc640 1 --2- 172.21.15.12:0/2728290937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca713a7ce0 0x55ca713a57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.012+0000 7fa10a7fc640 1 -- 172.21.15.12:0/2728290937 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ca7139e7b0 msgr2=0x55ca713a4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:22.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.012+0000 7fa10a7fc640 1 --2- 172.21.15.12:0/2728290937 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ca7139e7b0 0x55ca713a4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.012+0000 7fa10a7fc640 1 -- 172.21.15.12:0/2728290937 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca710df6b0 con 0x55ca7139f160 2024-04-22T13:24:22.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.012+0000 7fa112ffd640 1 --2- 172.21.15.12:0/2728290937 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ca7139e7b0 0x55ca713a4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:24:22.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.012+0000 7fa10a7fc640 1 --2- 172.21.15.12:0/2728290937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca7139f160 0x55ca713a5260 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00bec0 tx=0x7fa0fc012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606927b92dbad19d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:22.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.012+0000 7fa0fa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:22.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.012+0000 7fa0fa7fc640 1 -- 172.21.15.12:0/2728290937 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc004230 con 0x55ca7139f160 2024-04-22T13:24:22.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.012+0000 7fa0fa7fc640 1 -- 172.21.15.12:0/2728290937 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fa0fc0043d0 con 0x55ca7139f160 2024-04-22T13:24:22.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.012+0000 7fa11bf8c740 1 -- 172.21.15.12:0/2728290937 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa10c01e480 con 0x55ca7139f160 2024-04-22T13:24:22.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.013+0000 7fa0fa7fc640 1 -- 172.21.15.12:0/2728290937 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa0fc010760 con 0x55ca7139f160 2024-04-22T13:24:22.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.013+0000 7fa11bf8c740 10 client.15798 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:22.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.013+0000 7fa11bf8c740 1 -- 172.21.15.12:0/2728290937 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ca713b6fe0 con 0x55ca7139f160 2024-04-22T13:24:22.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.013+0000 7fa0fa7fc640 1 -- 172.21.15.12:0/2728290937 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa10c01e480 con 0x55ca7139f160 2024-04-22T13:24:22.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.013+0000 7fa0fa7fc640 1 -- 172.21.15.12:0/2728290937 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa0fc003bc0 con 0x55ca7139f160 2024-04-22T13:24:22.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.013+0000 7fa11bf8c740 10 client.15798 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:22.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.013+0000 7fa11bf8c740 10 client.15798 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-22T13:24:22.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.013+0000 7fa11bf8c740 20 client.15798 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:22.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.013+0000 7fa11bf8c740 1 --2- 172.21.15.12:0/2728290937 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55ca713b95c0 0x55ca713baa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:22.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.013+0000 7fa11bf8c740 4 client.15798 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:24:22.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.013+0000 7fa11bf8c740 1 -- 172.21.15.12:0/2728290937 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca71310530 con 0x55ca713b95c0 2024-04-22T13:24:22.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.013+0000 7fa112ffd640 1 --2- 172.21.15.12:0/2728290937 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55ca713b95c0 0x55ca713baa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:22.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.014+0000 7fa0fa7fc640 10 client.15798 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:22.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.014+0000 7fa112ffd640 1 --2- 172.21.15.12:0/2728290937 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55ca713b95c0 0x55ca713baa30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7306ce35bb7efbd8 server_cookie=787f04c8d42c00f2 in_seq=0 out_seq=0 2024-04-22T13:24:22.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.014+0000 7fa0fa7fc640 1 -- 172.21.15.12:0/2728290937 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ca71310530 con 0x55ca713b95c0 2024-04-22T13:24:22.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.014+0000 7fa0fa7fc640 10 client.15798 handle_command_reply: tid=0 2024-04-22T13:24:22.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.014+0000 7fa11bf8c740 1 client.15798 shutdown 2024-04-22T13:24:22.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.015+0000 7fa11bf8c740 1 -- 172.21.15.12:0/2728290937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca7139f160 msgr2=0x55ca713a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:22.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.015+0000 7fa11bf8c740 1 --2- 172.21.15.12:0/2728290937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca7139f160 0x55ca713a5260 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00bec0 tx=0x7fa0fc012820 comp rx=0 tx=0).stop 2024-04-22T13:24:22.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.015+0000 7fa11bf8c740 1 -- 172.21.15.12:0/2728290937 shutdown_connections 2024-04-22T13:24:22.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.015+0000 7fa11bf8c740 1 --2- 172.21.15.12:0/2728290937 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55ca713b95c0 0x55ca713baa30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.015+0000 7fa11bf8c740 1 --2- 172.21.15.12:0/2728290937 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca713a7ce0 0x55ca713a57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.015+0000 7fa0fa7fc640 0 client.15798 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:22.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.015+0000 7fa11bf8c740 1 --2- 172.21.15.12:0/2728290937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca7139f160 0x55ca713a5260 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.015+0000 7fa11bf8c740 1 --2- 172.21.15.12:0/2728290937 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ca7139e7b0 0x55ca713a4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.015+0000 7fa11bf8c740 1 -- 172.21.15.12:0/2728290937 >> 172.21.15.12:0/2728290937 conn(0x55ca71398a60 msgr2=0x55ca7139abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:22.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.015+0000 7fa11bf8c740 1 -- 172.21.15.12:0/2728290937 shutdown_connections 2024-04-22T13:24:22.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.015+0000 7fa11bf8c740 1 -- 172.21.15.12:0/2728290937 wait complete. 2024-04-22T13:24:22.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.015+0000 7fa11bf8c740 20 client.15798 trim_cache size 0 max 16384 2024-04-22T13:24:22.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.033+0000 7fa11bf8c740 1 Processor -- start 2024-04-22T13:24:22.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.033+0000 7fa11bf8c740 1 -- start start 2024-04-22T13:24:22.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.033+0000 7fa11bf8c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca71398a60 0x55ca713a7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:22.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.033+0000 7fa11bf8c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ca7139e7b0 0x55ca713b8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:22.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.033+0000 7fa11bf8c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca7139f160 0x55ca713a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:22.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.033+0000 7fa11bf8c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ca713b8d70 con 0x55ca7139f160 2024-04-22T13:24:22.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.033+0000 7fa11bf8c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ca713b9440 con 0x55ca7139e7b0 2024-04-22T13:24:22.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.033+0000 7fa11bf8c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ca713b9b10 con 0x55ca71398a60 2024-04-22T13:24:22.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.033+0000 7fa1137fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca7139f160 0x55ca713a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:22.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.034+0000 7fa1137fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca7139f160 0x55ca713a55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45150/0 (socket says 172.21.15.12:45150) 2024-04-22T13:24:22.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.034+0000 7fa1137fe640 1 -- 172.21.15.12:0/2013525514 learned_addr learned my addr 172.21.15.12:0/2013525514 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:22.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.034+0000 7fa10a7fc640 1 --2- 172.21.15.12:0/2013525514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ca7139e7b0 0x55ca713b8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:22.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.034+0000 7fa112ffd640 1 --2- 172.21.15.12:0/2013525514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca71398a60 0x55ca713a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:22.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.034+0000 7fa1137fe640 1 -- 172.21.15.12:0/2013525514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca71398a60 msgr2=0x55ca713a7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:22.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.034+0000 7fa1137fe640 1 --2- 172.21.15.12:0/2013525514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca71398a60 0x55ca713a7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.034+0000 7fa1137fe640 1 -- 172.21.15.12:0/2013525514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ca7139e7b0 msgr2=0x55ca713b8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:22.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.034+0000 7fa1137fe640 1 --2- 172.21.15.12:0/2013525514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ca7139e7b0 0x55ca713b8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.034+0000 7fa1137fe640 1 -- 172.21.15.12:0/2013525514 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca71392d80 con 0x55ca7139f160 2024-04-22T13:24:22.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.034+0000 7fa10a7fc640 1 --2- 172.21.15.12:0/2013525514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ca7139e7b0 0x55ca713b8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:22.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.034+0000 7fa1137fe640 1 --2- 172.21.15.12:0/2013525514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca7139f160 0x55ca713a55c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa10c005c10 tx=0x7fa10c005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626e43ae60834781 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:22.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.034+0000 7fa108ff9640 1 -- 172.21.15.12:0/2013525514 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10c022040 con 0x55ca7139f160 2024-04-22T13:24:22.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.034+0000 7fa108ff9640 1 -- 172.21.15.12:0/2013525514 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa10c00e7c0 con 0x55ca7139f160 2024-04-22T13:24:22.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.034+0000 7fa108ff9640 1 -- 172.21.15.12:0/2013525514 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10c01ea70 con 0x55ca7139f160 2024-04-22T13:24:22.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.035+0000 7fa11bf8c740 1 -- 172.21.15.12:0/2013525514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca7139f160 msgr2=0x55ca713a55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:22.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.035+0000 7fa11bf8c740 1 --2- 172.21.15.12:0/2013525514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca7139f160 0x55ca713a55c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa10c005c10 tx=0x7fa10c005c40 comp rx=0 tx=0).stop 2024-04-22T13:24:22.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.035+0000 7fa11bf8c740 1 -- 172.21.15.12:0/2013525514 shutdown_connections 2024-04-22T13:24:22.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.035+0000 7fa11bf8c740 1 --2- 172.21.15.12:0/2013525514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca7139f160 0x55ca713a55c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.035+0000 7fa11bf8c740 1 --2- 172.21.15.12:0/2013525514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ca7139e7b0 0x55ca713b8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.035+0000 7fa11bf8c740 1 --2- 172.21.15.12:0/2013525514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca71398a60 0x55ca713a7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.035+0000 7fa11bf8c740 1 -- 172.21.15.12:0/2013525514 >> 172.21.15.12:0/2013525514 conn(0x55ca7139f8b0 msgr2=0x55ca713a35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:22.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.035+0000 7fa11bf8c740 1 -- 172.21.15.12:0/2013525514 shutdown_connections 2024-04-22T13:24:22.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.035+0000 7fa11bf8c740 1 -- 172.21.15.12:0/2013525514 wait complete. 2024-04-22T13:24:22.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.035+0000 7fa11bf8c740 1 -- start start 2024-04-22T13:24:22.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.035+0000 7fa11bf8c740 1 Processor -- start 2024-04-22T13:24:22.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.036+0000 7fa11bf8c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca71398a60 0x55ca713bc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:22.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.036+0000 7fa11bf8c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ca7139e7b0 0x55ca713bcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:22.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.036+0000 7fa11bf8c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca7139f160 0x55ca713bd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:22.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.036+0000 7fa11bf8c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ca713ba8b0 con 0x55ca71398a60 2024-04-22T13:24:22.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.036+0000 7fa11bf8c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ca713ba1e0 con 0x55ca7139e7b0 2024-04-22T13:24:22.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.036+0000 7fa11bf8c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ca713baf80 con 0x55ca7139f160 2024-04-22T13:24:22.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.036+0000 7fa11bf8c740 10 client.0 _ll_register_callbacks cb 0x7fa0fc00b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:22.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.036+0000 7fa112ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca71398a60 0x55ca713bc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:22.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.036+0000 7fa112ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca71398a60 0x55ca713bc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45166/0 (socket says 172.21.15.12:45166) 2024-04-22T13:24:22.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.036+0000 7fa112ffd640 1 -- 172.21.15.12:0/1486613703 learned_addr learned my addr 172.21.15.12:0/1486613703 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:22.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.036+0000 7fa10a7fc640 1 --2- 172.21.15.12:0/1486613703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ca7139e7b0 0x55ca713bcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:22.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.036+0000 7fa1137fe640 1 --2- 172.21.15.12:0/1486613703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca7139f160 0x55ca713bd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:22.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.036+0000 7fa112ffd640 1 -- 172.21.15.12:0/1486613703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca7139f160 msgr2=0x55ca713bd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:22.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.036+0000 7fa112ffd640 1 --2- 172.21.15.12:0/1486613703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca7139f160 0x55ca713bd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.036+0000 7fa112ffd640 1 -- 172.21.15.12:0/1486613703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ca7139e7b0 msgr2=0x55ca713bcc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:22.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.036+0000 7fa112ffd640 1 --2- 172.21.15.12:0/1486613703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ca7139e7b0 0x55ca713bcc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.036+0000 7fa112ffd640 1 -- 172.21.15.12:0/1486613703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca713c0f20 con 0x55ca71398a60 2024-04-22T13:24:22.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.036+0000 7fa1137fe640 1 --2- 172.21.15.12:0/1486613703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca7139f160 0x55ca713bd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:22.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.036+0000 7fa112ffd640 1 --2- 172.21.15.12:0/1486613703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca71398a60 0x55ca713bc730 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa10404dff0 tx=0x7fa10408da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7aa14799fd486a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:22.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.036+0000 7fa0fa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:22.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.036+0000 7fa0fa7fc640 1 -- 172.21.15.12:0/1486613703 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1040be190 con 0x55ca71398a60 2024-04-22T13:24:22.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.037+0000 7fa0fa7fc640 1 -- 172.21.15.12:0/1486613703 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fa1040035f0 con 0x55ca71398a60 2024-04-22T13:24:22.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.037+0000 7fa11bf8c740 1 -- 172.21.15.12:0/1486613703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa10c00e7c0 con 0x55ca71398a60 2024-04-22T13:24:22.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.037+0000 7fa0fa7fc640 1 -- 172.21.15.12:0/1486613703 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa1040c2540 con 0x55ca71398a60 2024-04-22T13:24:22.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.037+0000 7fa11bf8c740 10 client.15804 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:22.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.037+0000 7fa11bf8c740 1 -- 172.21.15.12:0/1486613703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ca7141f8a0 con 0x55ca71398a60 2024-04-22T13:24:22.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.037+0000 7fa0fa7fc640 1 -- 172.21.15.12:0/1486613703 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa10c00e7c0 con 0x55ca71398a60 2024-04-22T13:24:22.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.037+0000 7fa0fa7fc640 1 -- 172.21.15.12:0/1486613703 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa10403c030 con 0x55ca71398a60 2024-04-22T13:24:22.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.037+0000 7fa11bf8c740 10 client.15804 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:22.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.037+0000 7fa11bf8c740 10 client.15804 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-22T13:24:22.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.037+0000 7fa11bf8c740 20 client.15804 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:22.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.037+0000 7fa11bf8c740 1 --2- 172.21.15.12:0/1486613703 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55ca713a7ce0 0x55ca71423150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:22.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.037+0000 7fa11bf8c740 4 client.15804 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-22T13:24:22.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.037+0000 7fa11bf8c740 1 -- 172.21.15.12:0/1486613703 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "status"}) -- 0x55ca7130a3a0 con 0x55ca713a7ce0 2024-04-22T13:24:22.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.037+0000 7fa10a7fc640 1 --2- 172.21.15.12:0/1486613703 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55ca713a7ce0 0x55ca71423150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:22.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.038+0000 7fa0fa7fc640 10 client.15804 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:22.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.038+0000 7fa10a7fc640 1 --2- 172.21.15.12:0/1486613703 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55ca713a7ce0 0x55ca71423150 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3e7adde6ef3917c server_cookie=1368766810c9c79f in_seq=0 out_seq=0 2024-04-22T13:24:22.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.038+0000 7fa0fa7fc640 1 -- 172.21.15.12:0/1486613703 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ca7130a3a0 con 0x55ca713a7ce0 2024-04-22T13:24:22.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.038+0000 7fa0fa7fc640 10 client.15804 handle_command_reply: tid=0 2024-04-22T13:24:22.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.038+0000 7fa11bf8c740 1 client.15804 shutdown 2024-04-22T13:24:22.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.038+0000 7fa11bf8c740 1 -- 172.21.15.12:0/1486613703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca71398a60 msgr2=0x55ca713bc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:22.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.038+0000 7fa11bf8c740 1 --2- 172.21.15.12:0/1486613703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca71398a60 0x55ca713bc730 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa10404dff0 tx=0x7fa10408da70 comp rx=0 tx=0).stop 2024-04-22T13:24:22.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.038+0000 7fa11bf8c740 1 -- 172.21.15.12:0/1486613703 shutdown_connections 2024-04-22T13:24:22.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.038+0000 7fa11bf8c740 1 --2- 172.21.15.12:0/1486613703 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55ca713a7ce0 0x55ca71423150 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.038+0000 7fa11bf8c740 1 --2- 172.21.15.12:0/1486613703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ca7139f160 0x55ca713bd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.038+0000 7fa0fa7fc640 0 client.15804 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:22.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.038+0000 7fa11bf8c740 1 --2- 172.21.15.12:0/1486613703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ca7139e7b0 0x55ca713bcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.038+0000 7fa11bf8c740 1 --2- 172.21.15.12:0/1486613703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca71398a60 0x55ca713bc730 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.038+0000 7fa11bf8c740 1 -- 172.21.15.12:0/1486613703 >> 172.21.15.12:0/1486613703 conn(0x55ca7139f8b0 msgr2=0x55ca71397170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:22.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.038+0000 7fa11bf8c740 1 -- 172.21.15.12:0/1486613703 shutdown_connections 2024-04-22T13:24:22.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.038+0000 7fa11bf8c740 1 -- 172.21.15.12:0/1486613703 wait complete. 2024-04-22T13:24:22.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.038+0000 7fa11bf8c740 20 client.15804 trim_cache size 0 max 16384 2024-04-22T13:24:22.042 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:24:22.042 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_fsid": "0aede5ea-00aa-11ef-bc93-c7b262605968", 2024-04-22T13:24:22.042 INFO:teuthology.orchestra.run.smithi012.stdout: "whoami": 0, 2024-04-22T13:24:22.042 INFO:teuthology.orchestra.run.smithi012.stdout: "id": 24509, 2024-04-22T13:24:22.042 INFO:teuthology.orchestra.run.smithi012.stdout: "want_state": "up:active", 2024-04-22T13:24:22.042 INFO:teuthology.orchestra.run.smithi012.stdout: "state": "up:active", 2024-04-22T13:24:22.043 INFO:teuthology.orchestra.run.smithi012.stdout: "fs_name": "cephfs", 2024-04-22T13:24:22.043 INFO:teuthology.orchestra.run.smithi012.stdout: "rank_uptime": 136.67089107199999, 2024-04-22T13:24:22.043 INFO:teuthology.orchestra.run.smithi012.stdout: "mdsmap_epoch": 28, 2024-04-22T13:24:22.043 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch": 90, 2024-04-22T13:24:22.043 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_epoch_barrier": 89, 2024-04-22T13:24:22.043 INFO:teuthology.orchestra.run.smithi012.stdout: "uptime": 142.31829789700001 2024-04-22T13:24:22.043 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:24:22.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.039+0000 7fa11939d640 1 -- 172.21.15.12:0/3696440620 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa1000a3e40 msgr2=0x7fa1000a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:22.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.039+0000 7fa11939d640 1 --2- 172.21.15.12:0/3696440620 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa1000a3e40 0x7fa1000a6300 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa10c023000 tx=0x7fa10c01f000 comp rx=0 tx=0).stop 2024-04-22T13:24:22.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.039+0000 7fa11939d640 1 -- 172.21.15.12:0/3696440620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1141010a0 msgr2=0x7fa114102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:22.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.039+0000 7fa11939d640 1 --2- 172.21.15.12:0/3696440620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1141010a0 0x7fa114102b30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa104037040 tx=0x7fa104035a40 comp rx=0 tx=0).stop 2024-04-22T13:24:22.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.039+0000 7fa11939d640 1 -- 172.21.15.12:0/3696440620 shutdown_connections 2024-04-22T13:24:22.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.039+0000 7fa11939d640 1 --2- 172.21.15.12:0/3696440620 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa1000a3e40 0x7fa1000a6300 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.039+0000 7fa11939d640 1 --2- 172.21.15.12:0/3696440620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1141035b0 0x7fa1141a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.039+0000 7fa11939d640 1 --2- 172.21.15.12:0/3696440620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa114106170 0x7fa114103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.039+0000 7fa11939d640 1 --2- 172.21.15.12:0/3696440620 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa1141010a0 0x7fa114102b30 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:22.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.039+0000 7fa11939d640 1 -- 172.21.15.12:0/3696440620 >> 172.21.15.12:0/3696440620 conn(0x7fa114077eb0 msgr2=0x7fa11406a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:22.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.039+0000 7fa11939d640 1 -- 172.21.15.12:0/3696440620 shutdown_connections 2024-04-22T13:24:22.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:22.039+0000 7fa11939d640 1 -- 172.21.15.12:0/3696440620 wait complete. 2024-04-22T13:24:22.046 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:21 smithi012 ceph-mon[28265]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-22T13:24:22.046 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:21 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2042523652' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:24:22.046 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:21 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/244274738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:22.047 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-04-22T13:24:22.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:21 smithi134 ceph-mon[31338]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-22T13:24:22.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:21 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2042523652' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:24:22.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:21 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/244274738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:24.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:23 smithi012 ceph-mon[28265]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 52 op/s 2024-04-22T13:24:24.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:23 smithi134 ceph-mon[31338]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 52 op/s 2024-04-22T13:24:24.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:23 smithi111 ceph-mon[31361]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 52 op/s 2024-04-22T13:24:26.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:25 smithi012 ceph-mon[28265]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 78 KiB/s wr, 55 op/s 2024-04-22T13:24:26.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:25 smithi134 ceph-mon[31338]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 78 KiB/s wr, 55 op/s 2024-04-22T13:24:26.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:25 smithi111 ceph-mon[31361]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 78 KiB/s wr, 55 op/s 2024-04-22T13:24:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.082+0000 7f02210ba640 1 -- 172.21.15.12:0/856927380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f021c1061b0 msgr2=0x7f021c10dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.082+0000 7f02210ba640 1 --2- 172.21.15.12:0/856927380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f021c1061b0 0x7f021c10dd80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0214009ca0 tx=0x7f021402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.083+0000 7f02210ba640 1 -- 172.21.15.12:0/856927380 shutdown_connections 2024-04-22T13:24:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.083+0000 7f02210ba640 1 --2- 172.21.15.12:0/856927380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f021c1061b0 0x7f021c10dd80 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.083+0000 7f02210ba640 1 --2- 172.21.15.12:0/856927380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f021c101a90 0x7f021c105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.083+0000 7f02210ba640 1 --2- 172.21.15.12:0/856927380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f021c1010e0 0x7f021c1014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.083+0000 7f02210ba640 1 -- 172.21.15.12:0/856927380 >> 172.21.15.12:0/856927380 conn(0x7f021c0771d0 msgr2=0x7f021c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.083+0000 7f02210ba640 1 -- 172.21.15.12:0/856927380 shutdown_connections 2024-04-22T13:24:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.083+0000 7f02210ba640 1 -- 172.21.15.12:0/856927380 wait complete. 2024-04-22T13:24:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.083+0000 7f02210ba640 1 Processor -- start 2024-04-22T13:24:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.084+0000 7f02210ba640 1 -- start start 2024-04-22T13:24:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.084+0000 7f02210ba640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f021c1010e0 0x7f021c1994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.084+0000 7f02210ba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f021c1061b0 0x7f021c1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.084+0000 7f02210ba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f021c199f70 0x7f021c1a4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.084+0000 7f02210ba640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f021c111100 con 0x7f021c199f70 2024-04-22T13:24:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.084+0000 7f02210ba640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f021c110f80 con 0x7f021c1010e0 2024-04-22T13:24:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.084+0000 7f02210ba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f021c111280 con 0x7f021c1061b0 2024-04-22T13:24:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.084+0000 7f021b59e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f021c199f70 0x7f021c1a4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.084+0000 7f021b59e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f021c199f70 0x7f021c1a4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45180/0 (socket says 172.21.15.12:45180) 2024-04-22T13:24:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.084+0000 7f021b59e640 1 -- 172.21.15.12:0/2525607185 learned_addr learned my addr 172.21.15.12:0/2525607185 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.084+0000 7f021ad9d640 1 --2- 172.21.15.12:0/2525607185 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f021c1010e0 0x7f021c1994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.084+0000 7f021a59c640 1 --2- 172.21.15.12:0/2525607185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f021c1061b0 0x7f021c1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.084+0000 7f021b59e640 1 -- 172.21.15.12:0/2525607185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f021c1061b0 msgr2=0x7f021c1999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.084+0000 7f021b59e640 1 --2- 172.21.15.12:0/2525607185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f021c1061b0 0x7f021c1999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.084+0000 7f021b59e640 1 -- 172.21.15.12:0/2525607185 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f021c1010e0 msgr2=0x7f021c1994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.084+0000 7f021b59e640 1 --2- 172.21.15.12:0/2525607185 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f021c1010e0 0x7f021c1994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.084+0000 7f021b59e640 1 -- 172.21.15.12:0/2525607185 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f021c1a4cf0 con 0x7f021c199f70 2024-04-22T13:24:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.085+0000 7f021b59e640 1 --2- 172.21.15.12:0/2525607185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f021c199f70 0x7f021c1a4520 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0214005e30 tx=0x7f0214034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.085+0000 7f0203fff640 1 -- 172.21.15.12:0/2525607185 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0214039040 con 0x7f021c199f70 2024-04-22T13:24:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.085+0000 7f02210ba640 1 -- 172.21.15.12:0/2525607185 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f021c1a4f80 con 0x7f021c199f70 2024-04-22T13:24:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.085+0000 7f0203fff640 1 -- 172.21.15.12:0/2525607185 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0214044070 con 0x7f021c199f70 2024-04-22T13:24:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.085+0000 7f0203fff640 1 -- 172.21.15.12:0/2525607185 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0214003e60 con 0x7f021c199f70 2024-04-22T13:24:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.085+0000 7f02210ba640 1 -- 172.21.15.12:0/2525607185 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f021c1a5440 con 0x7f021c199f70 2024-04-22T13:24:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.087+0000 7f0203fff640 1 -- 172.21.15.12:0/2525607185 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f02140324c0 con 0x7f021c199f70 2024-04-22T13:24:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.088+0000 7f0223d86740 1 Processor -- start 2024-04-22T13:24:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.088+0000 7f0223d86740 1 -- start start 2024-04-22T13:24:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.088+0000 7f0203fff640 1 --2- 172.21.15.12:0/2525607185 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f02040a3ee0 0x7f02040a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.088+0000 7f0223d86740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3afc3e8d0 0x55c3afc3d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.088+0000 7f0223d86740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3afc3db60 0x55c3afc3dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.088+0000 7f0223d86740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3afc470e0 0x55c3afc494d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.088+0000 7f0223d86740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c3afba28d0 con 0x55c3afc3db60 2024-04-22T13:24:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.088+0000 7f0223d86740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c3afb3fe80 con 0x55c3afc470e0 2024-04-22T13:24:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.088+0000 7f0223d86740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c3afb9e890 con 0x55c3afc3e8d0 2024-04-22T13:24:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.088+0000 7f0203fff640 1 -- 172.21.15.12:0/2525607185 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f02140f2b80 con 0x7f021c199f70 2024-04-22T13:24:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.088+0000 7f021a59c640 1 --2- 172.21.15.12:0/2525607185 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f02040a3ee0 0x7f02040a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.088+0000 7f021a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3afc3db60 0x55c3afc3dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.088+0000 7f021a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3afc3db60 0x55c3afc3dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45192/0 (socket says 172.21.15.12:45192) 2024-04-22T13:24:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.088+0000 7f021a59c640 1 -- 172.21.15.12:0/1091713502 learned_addr learned my addr 172.21.15.12:0/1091713502 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.088+0000 7f021b59e640 1 --2- 172.21.15.12:0/1091713502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3afc470e0 0x55c3afc494d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.088+0000 7f021ad9d640 1 --2- 172.21.15.12:0/1091713502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3afc3e8d0 0x55c3afc3d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.089+0000 7f021a59c640 1 -- 172.21.15.12:0/1091713502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3afc3e8d0 msgr2=0x55c3afc3d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.089+0000 7f021a59c640 1 --2- 172.21.15.12:0/1091713502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3afc3e8d0 0x55c3afc3d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.088 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.089+0000 7f021a59c640 1 -- 172.21.15.12:0/1091713502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3afc470e0 msgr2=0x55c3afc494d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.089+0000 7f021a59c640 1 --2- 172.21.15.12:0/1091713502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3afc470e0 0x55c3afc494d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.089+0000 7f021a59c640 1 -- 172.21.15.12:0/1091713502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3afb61320 con 0x55c3afc3db60 2024-04-22T13:24:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.089+0000 7f021ad9d640 1 --2- 172.21.15.12:0/1091713502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3afc3e8d0 0x55c3afc3d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:24:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.089+0000 7f021b59e640 1 --2- 172.21.15.12:0/1091713502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3afc470e0 0x55c3afc494d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:24:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.089+0000 7f021a59c640 1 --2- 172.21.15.12:0/2525607185 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f02040a3ee0 0x7f02040a63a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f02100128c0 tx=0x7f0210005cf0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.089+0000 7f021a59c640 1 --2- 172.21.15.12:0/1091713502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3afc3db60 0x55c3afc3dfe0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f021000f730 tx=0x7f021001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26149fa88faa0a91 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.089+0000 7f02017fa640 1 -- 172.21.15.12:0/1091713502 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0210024070 con 0x55c3afc3db60 2024-04-22T13:24:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.089+0000 7f02017fa640 1 -- 172.21.15.12:0/1091713502 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f021000fdf0 con 0x55c3afc3db60 2024-04-22T13:24:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.089+0000 7f02017fa640 1 -- 172.21.15.12:0/1091713502 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0210004d60 con 0x55c3afc3db60 2024-04-22T13:24:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.090+0000 7f0223d86740 1 -- 172.21.15.12:0/1091713502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3afc3db60 msgr2=0x55c3afc3dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.090+0000 7f0223d86740 1 --2- 172.21.15.12:0/1091713502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3afc3db60 0x55c3afc3dfe0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f021000f730 tx=0x7f021001c000 comp rx=0 tx=0).stop 2024-04-22T13:24:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.090+0000 7f0223d86740 1 -- 172.21.15.12:0/1091713502 shutdown_connections 2024-04-22T13:24:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.090+0000 7f0223d86740 1 --2- 172.21.15.12:0/1091713502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3afc470e0 0x55c3afc494d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.090+0000 7f0223d86740 1 --2- 172.21.15.12:0/1091713502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3afc3db60 0x55c3afc3dfe0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.090+0000 7f0223d86740 1 --2- 172.21.15.12:0/1091713502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3afc3e8d0 0x55c3afc3d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.090+0000 7f0223d86740 1 -- 172.21.15.12:0/1091713502 >> 172.21.15.12:0/1091713502 conn(0x55c3afc04c90 msgr2=0x55c3afc38850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.090+0000 7f0223d86740 1 -- 172.21.15.12:0/1091713502 shutdown_connections 2024-04-22T13:24:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.090+0000 7f0223d86740 1 -- 172.21.15.12:0/1091713502 wait complete. 2024-04-22T13:24:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.090+0000 7f0223d86740 1 -- start start 2024-04-22T13:24:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.090+0000 7f0223d86740 1 Processor -- start 2024-04-22T13:24:28.090 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.091+0000 7f0223d86740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3afc3db60 0x55c3afc46ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:28.091 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:27 smithi012 ceph-mon[28265]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-04-22T13:24:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.091+0000 7f0223d86740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3afc3e8d0 0x55c3afc43e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.091+0000 7f0223d86740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3afc470e0 0x55c3afc44450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.091+0000 7f0223d86740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c3afc49db0 con 0x55c3afc3e8d0 2024-04-22T13:24:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.091+0000 7f0223d86740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c3afb2aeb0 con 0x55c3afc3db60 2024-04-22T13:24:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.091+0000 7f0223d86740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c3afc49f30 con 0x55c3afc470e0 2024-04-22T13:24:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.091+0000 7f0223d86740 10 client.0 _ll_register_callbacks cb 0x55c3afb57b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.091+0000 7f021a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3afc3e8d0 0x55c3afc43e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.091+0000 7f021a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3afc3e8d0 0x55c3afc43e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45206/0 (socket says 172.21.15.12:45206) 2024-04-22T13:24:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.091+0000 7f021a59c640 1 -- 172.21.15.12:0/4106348244 learned_addr learned my addr 172.21.15.12:0/4106348244 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.091+0000 7f021ad9d640 1 --2- 172.21.15.12:0/4106348244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3afc3db60 0x55c3afc46ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.091+0000 7f021b59e640 1 --2- 172.21.15.12:0/4106348244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3afc470e0 0x55c3afc44450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.091+0000 7f021a59c640 1 -- 172.21.15.12:0/4106348244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3afc470e0 msgr2=0x55c3afc44450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.091+0000 7f021a59c640 1 --2- 172.21.15.12:0/4106348244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3afc470e0 0x55c3afc44450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.091+0000 7f021a59c640 1 -- 172.21.15.12:0/4106348244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3afc3db60 msgr2=0x55c3afc46ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.091+0000 7f021a59c640 1 --2- 172.21.15.12:0/4106348244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3afc3db60 0x55c3afc46ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.091+0000 7f021a59c640 1 -- 172.21.15.12:0/4106348244 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3af8b5210 con 0x55c3afc3e8d0 2024-04-22T13:24:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.091+0000 7f021ad9d640 1 --2- 172.21.15.12:0/4106348244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3afc3db60 0x55c3afc46ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.091+0000 7f021a59c640 1 --2- 172.21.15.12:0/4106348244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3afc3e8d0 0x55c3afc43e40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f021000f780 tx=0x7f0210004850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8cdecab4bdbd7a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.091+0000 7f01feff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.091+0000 7f01feff5640 1 -- 172.21.15.12:0/4106348244 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0210024040 con 0x55c3afc3e8d0 2024-04-22T13:24:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.092+0000 7f01feff5640 1 -- 172.21.15.12:0/4106348244 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f021000db40 con 0x55c3afc3e8d0 2024-04-22T13:24:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.092+0000 7f0223d86740 1 -- 172.21.15.12:0/4106348244 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f021000fdf0 con 0x55c3afc3e8d0 2024-04-22T13:24:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.092+0000 7f01feff5640 1 -- 172.21.15.12:0/4106348244 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f021000dd40 con 0x55c3afc3e8d0 2024-04-22T13:24:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.092+0000 7f0223d86740 10 client.15816 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.092+0000 7f0223d86740 1 -- 172.21.15.12:0/4106348244 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c3af981620 con 0x55c3afc3e8d0 2024-04-22T13:24:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.092+0000 7f01feff5640 1 -- 172.21.15.12:0/4106348244 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f021000fdf0 con 0x55c3afc3e8d0 2024-04-22T13:24:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.092+0000 7f01feff5640 1 -- 172.21.15.12:0/4106348244 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f021001f070 con 0x55c3afc3e8d0 2024-04-22T13:24:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.092+0000 7f0223d86740 10 client.15816 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.092+0000 7f0223d86740 10 client.15816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:24:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.092+0000 7f0223d86740 20 client.15816 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.092+0000 7f0223d86740 1 --2- 172.21.15.12:0/4106348244 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c3afc57050 0x55c3afc59440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.092+0000 7f0223d86740 4 client.15816 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:24:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.092+0000 7f0223d86740 1 -- 172.21.15.12:0/4106348244 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3afbb6880 con 0x55c3afc57050 2024-04-22T13:24:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.092+0000 7f021ad9d640 1 --2- 172.21.15.12:0/4106348244 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c3afc57050 0x55c3afc59440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.093+0000 7f021ad9d640 1 --2- 172.21.15.12:0/4106348244 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c3afc57050 0x55c3afc59440 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1044225e91489d27 server_cookie=3e085d4736275019 in_seq=0 out_seq=0 2024-04-22T13:24:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.093+0000 7f01feff5640 10 client.15816 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.094+0000 7f01feff5640 1 -- 172.21.15.12:0/4106348244 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c3afbb6880 con 0x55c3afc57050 2024-04-22T13:24:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.094+0000 7f01feff5640 10 client.15816 handle_command_reply: tid=0 2024-04-22T13:24:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.094+0000 7f0223d86740 1 client.15816 shutdown 2024-04-22T13:24:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.094+0000 7f0223d86740 1 -- 172.21.15.12:0/4106348244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3afc3e8d0 msgr2=0x55c3afc43e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.094+0000 7f0223d86740 1 --2- 172.21.15.12:0/4106348244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3afc3e8d0 0x55c3afc43e40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f021000f780 tx=0x7f0210004850 comp rx=0 tx=0).stop 2024-04-22T13:24:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.094+0000 7f0223d86740 1 -- 172.21.15.12:0/4106348244 shutdown_connections 2024-04-22T13:24:28.097 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.094+0000 7f0223d86740 1 --2- 172.21.15.12:0/4106348244 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c3afc57050 0x55c3afc59440 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.094+0000 7f0223d86740 1 --2- 172.21.15.12:0/4106348244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3afc470e0 0x55c3afc44450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.094+0000 7f0223d86740 1 --2- 172.21.15.12:0/4106348244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3afc3e8d0 0x55c3afc43e40 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.094+0000 7f01feff5640 0 client.15816 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.094+0000 7f0223d86740 1 --2- 172.21.15.12:0/4106348244 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3afc3db60 0x55c3afc46ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.094+0000 7f0223d86740 1 -- 172.21.15.12:0/4106348244 >> 172.21.15.12:0/4106348244 conn(0x55c3afc04c90 msgr2=0x55c3afc3a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.094+0000 7f0223d86740 1 -- 172.21.15.12:0/4106348244 shutdown_connections 2024-04-22T13:24:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.094+0000 7f0223d86740 1 -- 172.21.15.12:0/4106348244 wait complete. 2024-04-22T13:24:28.098 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.094+0000 7f0223d86740 20 client.15816 trim_cache size 0 max 16384 2024-04-22T13:24:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.109+0000 7f0223d86740 1 Processor -- start 2024-04-22T13:24:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.109+0000 7f0223d86740 1 -- start start 2024-04-22T13:24:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.109+0000 7f0223d86740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3afc3db60 0x55c3afc52d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.109+0000 7f0223d86740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3afc3e8d0 0x55c3afc3f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.109+0000 7f0223d86740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3afc470e0 0x55c3afc46570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.109+0000 7f0223d86740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f021c1ad970 con 0x55c3afc3e8d0 2024-04-22T13:24:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.109+0000 7f0223d86740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f021c1af4b0 con 0x55c3afc470e0 2024-04-22T13:24:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.109+0000 7f0223d86740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f021c1ae040 con 0x55c3afc3db60 2024-04-22T13:24:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.109+0000 7f021a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3afc3e8d0 0x55c3afc3f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.109+0000 7f021a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3afc3e8d0 0x55c3afc3f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45220/0 (socket says 172.21.15.12:45220) 2024-04-22T13:24:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.109+0000 7f021a59c640 1 -- 172.21.15.12:0/3572106410 learned_addr learned my addr 172.21.15.12:0/3572106410 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.109+0000 7f021b59e640 1 --2- 172.21.15.12:0/3572106410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3afc470e0 0x55c3afc46570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.110+0000 7f021a59c640 1 -- 172.21.15.12:0/3572106410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3afc3db60 msgr2=0x55c3afc52d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.110+0000 7f021a59c640 1 --2- 172.21.15.12:0/3572106410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3afc3db60 0x55c3afc52d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.110+0000 7f021a59c640 1 -- 172.21.15.12:0/3572106410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3afc470e0 msgr2=0x55c3afc46570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.110+0000 7f021a59c640 1 --2- 172.21.15.12:0/3572106410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3afc470e0 0x55c3afc46570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.110+0000 7f021a59c640 1 -- 172.21.15.12:0/3572106410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3afc3ce30 con 0x55c3afc3e8d0 2024-04-22T13:24:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.110+0000 7f021b59e640 1 --2- 172.21.15.12:0/3572106410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3afc470e0 0x55c3afc46570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.110+0000 7f021a59c640 1 --2- 172.21.15.12:0/3572106410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3afc3e8d0 0x55c3afc3f5d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f021000f750 tx=0x7f021000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d0b8e866f8e03 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.110+0000 7f0200ff9640 1 -- 172.21.15.12:0/3572106410 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021003d070 con 0x55c3afc3e8d0 2024-04-22T13:24:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.110+0000 7f0200ff9640 1 -- 172.21.15.12:0/3572106410 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0210012450 con 0x55c3afc3e8d0 2024-04-22T13:24:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.110+0000 7f0200ff9640 1 -- 172.21.15.12:0/3572106410 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0210020690 con 0x55c3afc3e8d0 2024-04-22T13:24:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.110+0000 7f0223d86740 1 -- 172.21.15.12:0/3572106410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3afc3e8d0 msgr2=0x55c3afc3f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.110+0000 7f0223d86740 1 --2- 172.21.15.12:0/3572106410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3afc3e8d0 0x55c3afc3f5d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f021000f750 tx=0x7f021000be10 comp rx=0 tx=0).stop 2024-04-22T13:24:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.111+0000 7f0223d86740 1 -- 172.21.15.12:0/3572106410 shutdown_connections 2024-04-22T13:24:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.111+0000 7f0223d86740 1 --2- 172.21.15.12:0/3572106410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3afc470e0 0x55c3afc46570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.111+0000 7f0223d86740 1 --2- 172.21.15.12:0/3572106410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3afc3e8d0 0x55c3afc3f5d0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.111+0000 7f0223d86740 1 --2- 172.21.15.12:0/3572106410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3afc3db60 0x55c3afc52d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.111+0000 7f0223d86740 1 -- 172.21.15.12:0/3572106410 >> 172.21.15.12:0/3572106410 conn(0x55c3afc04c90 msgr2=0x55c3afc3eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.111+0000 7f0223d86740 1 -- 172.21.15.12:0/3572106410 shutdown_connections 2024-04-22T13:24:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.111+0000 7f0223d86740 1 -- 172.21.15.12:0/3572106410 wait complete. 2024-04-22T13:24:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.111+0000 7f0223d86740 1 -- start start 2024-04-22T13:24:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.111+0000 7f0223d86740 1 Processor -- start 2024-04-22T13:24:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.111+0000 7f0223d86740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3afc3db60 0x55c3afc4f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.111+0000 7f0223d86740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3afc3e8d0 0x55c3afc41370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.111+0000 7f0223d86740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3afc470e0 0x55c3afc419a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.111+0000 7f0223d86740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f021c1aede0 con 0x55c3afc3db60 2024-04-22T13:24:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.111+0000 7f0223d86740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f021c1ae710 con 0x55c3afc3e8d0 2024-04-22T13:24:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.111+0000 7f0223d86740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f021c1ab950 con 0x55c3afc470e0 2024-04-22T13:24:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.111+0000 7f0223d86740 10 client.0 _ll_register_callbacks cb 0x7f0214005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.111+0000 7f021ad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3afc3db60 0x55c3afc4f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.111+0000 7f021ad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3afc3db60 0x55c3afc4f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45236/0 (socket says 172.21.15.12:45236) 2024-04-22T13:24:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.111+0000 7f021ad9d640 1 -- 172.21.15.12:0/3765200930 learned_addr learned my addr 172.21.15.12:0/3765200930 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.111+0000 7f021a59c640 1 --2- 172.21.15.12:0/3765200930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3afc3e8d0 0x55c3afc41370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.112+0000 7f021ad9d640 1 -- 172.21.15.12:0/3765200930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3afc470e0 msgr2=0x55c3afc419a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.112+0000 7f021ad9d640 1 --2- 172.21.15.12:0/3765200930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3afc470e0 0x55c3afc419a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.112+0000 7f021ad9d640 1 -- 172.21.15.12:0/3765200930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3afc3e8d0 msgr2=0x55c3afc41370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.112+0000 7f021ad9d640 1 --2- 172.21.15.12:0/3765200930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3afc3e8d0 0x55c3afc41370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.112+0000 7f021ad9d640 1 -- 172.21.15.12:0/3765200930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3afc33f30 con 0x55c3afc3db60 2024-04-22T13:24:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.112+0000 7f021ad9d640 1 --2- 172.21.15.12:0/3765200930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3afc3db60 0x55c3afc4f6e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x55c3afc59aa0 tx=0x7f020c00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704986b895331641 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.112+0000 7f01feff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.112+0000 7f01feff5640 1 -- 172.21.15.12:0/3765200930 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020c005020 con 0x55c3afc3db60 2024-04-22T13:24:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.112+0000 7f01feff5640 1 -- 172.21.15.12:0/3765200930 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f020c0051c0 con 0x55c3afc3db60 2024-04-22T13:24:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.112+0000 7f0223d86740 1 -- 172.21.15.12:0/3765200930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0210012450 con 0x55c3afc3db60 2024-04-22T13:24:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.112+0000 7f01feff5640 1 -- 172.21.15.12:0/3765200930 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f020c02e050 con 0x55c3afc3db60 2024-04-22T13:24:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.112+0000 7f0223d86740 10 client.15822 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.112+0000 7f0223d86740 1 -- 172.21.15.12:0/3765200930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c3afcbd580 con 0x55c3afc3db60 2024-04-22T13:24:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.112+0000 7f01feff5640 1 -- 172.21.15.12:0/3765200930 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0210012450 con 0x55c3afc3db60 2024-04-22T13:24:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.113+0000 7f01feff5640 1 -- 172.21.15.12:0/3765200930 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f020c01f180 con 0x55c3afc3db60 2024-04-22T13:24:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.113+0000 7f0223d86740 10 client.15822 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.113+0000 7f0223d86740 10 client.15822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:24:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.113+0000 7f0223d86740 20 client.15822 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.113+0000 7f0223d86740 1 --2- 172.21.15.12:0/3765200930 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c3afcbdd80 0x55c3afcc0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.113+0000 7f0223d86740 4 client.15822 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-22T13:24:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.113+0000 7f0223d86740 1 -- 172.21.15.12:0/3765200930 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c3afb423d0 con 0x55c3afcbdd80 2024-04-22T13:24:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.113+0000 7f021a59c640 1 --2- 172.21.15.12:0/3765200930 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c3afcbdd80 0x55c3afcc0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.113+0000 7f021a59c640 1 --2- 172.21.15.12:0/3765200930 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c3afcbdd80 0x55c3afcc0170 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16eaa0782ff0657e server_cookie=41769222990f2f38 in_seq=0 out_seq=0 2024-04-22T13:24:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.113+0000 7f01feff5640 10 client.15822 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.125+0000 7f01feff5640 1 -- 172.21.15.12:0/3765200930 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c3afb423d0 con 0x55c3afcbdd80 2024-04-22T13:24:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.125+0000 7f01feff5640 10 client.15822 handle_command_reply: tid=0 2024-04-22T13:24:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.126+0000 7f0223d86740 1 client.15822 shutdown 2024-04-22T13:24:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.126+0000 7f0223d86740 1 -- 172.21.15.12:0/3765200930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3afc3db60 msgr2=0x55c3afc4f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.126+0000 7f0223d86740 1 --2- 172.21.15.12:0/3765200930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3afc3db60 0x55c3afc4f6e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x55c3afc59aa0 tx=0x7f020c00bed0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.126+0000 7f0223d86740 1 -- 172.21.15.12:0/3765200930 shutdown_connections 2024-04-22T13:24:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.126+0000 7f0223d86740 1 --2- 172.21.15.12:0/3765200930 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c3afcbdd80 0x55c3afcc0170 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.126+0000 7f0223d86740 1 --2- 172.21.15.12:0/3765200930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3afc470e0 0x55c3afc419a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.126+0000 7f0223d86740 1 --2- 172.21.15.12:0/3765200930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3afc3e8d0 0x55c3afc41370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.126+0000 7f0223d86740 1 --2- 172.21.15.12:0/3765200930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c3afc3db60 0x55c3afc4f6e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.126+0000 7f01feff5640 0 client.15822 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.126+0000 7f0223d86740 1 -- 172.21.15.12:0/3765200930 >> 172.21.15.12:0/3765200930 conn(0x55c3afc04c90 msgr2=0x55c3afc37040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.126+0000 7f0223d86740 1 -- 172.21.15.12:0/3765200930 shutdown_connections 2024-04-22T13:24:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.126+0000 7f0223d86740 1 -- 172.21.15.12:0/3765200930 wait complete. 2024-04-22T13:24:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.126+0000 7f0223d86740 20 client.15822 trim_cache size 0 max 16384 2024-04-22T13:24:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-04-22T13:24:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-04-22T13:24:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-04-22T13:24:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-04-22T13:24:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.126+0000 7f02210ba640 1 -- 172.21.15.12:0/2525607185 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f02040a3ee0 msgr2=0x7f02040a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.126+0000 7f02210ba640 1 --2- 172.21.15.12:0/2525607185 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f02040a3ee0 0x7f02040a63a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f02100128c0 tx=0x7f0210005cf0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.126+0000 7f02210ba640 1 -- 172.21.15.12:0/2525607185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f021c199f70 msgr2=0x7f021c1a4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.126+0000 7f02210ba640 1 --2- 172.21.15.12:0/2525607185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f021c199f70 0x7f021c1a4520 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0214005e30 tx=0x7f0214034760 comp rx=0 tx=0).stop 2024-04-22T13:24:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.126+0000 7f02210ba640 1 -- 172.21.15.12:0/2525607185 shutdown_connections 2024-04-22T13:24:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.126+0000 7f02210ba640 1 --2- 172.21.15.12:0/2525607185 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f02040a3ee0 0x7f02040a63a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.126+0000 7f02210ba640 1 --2- 172.21.15.12:0/2525607185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f021c199f70 0x7f021c1a4520 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.126+0000 7f02210ba640 1 --2- 172.21.15.12:0/2525607185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f021c1061b0 0x7f021c1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.126+0000 7f02210ba640 1 --2- 172.21.15.12:0/2525607185 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f021c1010e0 0x7f021c1994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.127+0000 7f02210ba640 1 -- 172.21.15.12:0/2525607185 >> 172.21.15.12:0/2525607185 conn(0x7f021c0771d0 msgr2=0x7f021c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.127+0000 7f02210ba640 1 -- 172.21.15.12:0/2525607185 shutdown_connections 2024-04-22T13:24:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:28.127+0000 7f02210ba640 1 -- 172.21.15.12:0/2525607185 wait complete. 2024-04-22T13:24:28.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:27 smithi134 ceph-mon[31338]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-04-22T13:24:28.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:27 smithi111 ceph-mon[31361]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-04-22T13:24:30.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:29 smithi012 ceph-mon[28265]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-04-22T13:24:30.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:29 smithi134 ceph-mon[31338]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-04-22T13:24:30.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:29 smithi111 ceph-mon[31361]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-04-22T13:24:32.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:31 smithi012 ceph-mon[28265]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 9 op/s 2024-04-22T13:24:32.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:31 smithi134 ceph-mon[31338]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 9 op/s 2024-04-22T13:24:32.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:31 smithi111 ceph-mon[31361]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 9 op/s 2024-04-22T13:24:34.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:33 smithi012 ceph-mon[28265]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-04-22T13:24:34.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:33 smithi134 ceph-mon[31338]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-04-22T13:24:34.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:33 smithi111 ceph-mon[31361]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-04-22T13:24:36.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:35 smithi012 ceph-mon[28265]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.4 KiB/s wr, 16 op/s 2024-04-22T13:24:36.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:35 smithi134 ceph-mon[31338]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.4 KiB/s wr, 16 op/s 2024-04-22T13:24:36.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:35 smithi111 ceph-mon[31361]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.4 KiB/s wr, 16 op/s 2024-04-22T13:24:37.047 INFO:tasks.quiescer.fs.[cephfs]:Total duration (50.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-22T13:24:37.048 INFO:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.35 2024-04-22T13:24:37.048 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:24:37.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.167+0000 7fbacc975640 1 -- 172.21.15.12:0/2967583354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbac81031d0 msgr2=0x7fbac81035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:37.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.167+0000 7fbacc975640 1 --2- 172.21.15.12:0/2967583354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbac81031d0 0x7fbac81035b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbab8009900 tx=0x7fbab802bcd0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.168+0000 7fbacc975640 1 -- 172.21.15.12:0/2967583354 shutdown_connections 2024-04-22T13:24:37.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.168+0000 7fbacc975640 1 --2- 172.21.15.12:0/2967583354 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbac810c5a0 0x7fbac810e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.168+0000 7fbacc975640 1 --2- 172.21.15.12:0/2967583354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbac8103af0 0x7fbac810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.168+0000 7fbacc975640 1 --2- 172.21.15.12:0/2967583354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbac81031d0 0x7fbac81035b0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.168+0000 7fbacc975640 1 -- 172.21.15.12:0/2967583354 >> 172.21.15.12:0/2967583354 conn(0x7fbac80771d0 msgr2=0x7fbac80775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.168+0000 7fbacc975640 1 -- 172.21.15.12:0/2967583354 shutdown_connections 2024-04-22T13:24:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.168+0000 7fbacc975640 1 -- 172.21.15.12:0/2967583354 wait complete. 2024-04-22T13:24:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.168+0000 7fbacc975640 1 Processor -- start 2024-04-22T13:24:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.168+0000 7fbacc975640 1 -- start start 2024-04-22T13:24:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.169+0000 7fbacc975640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbac81031d0 0x7fbac81a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.169+0000 7fbacc975640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbac8103af0 0x7fbac81a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.169+0000 7fbacc975640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbac810c5a0 0x7fbac819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.169+0000 7fbacc975640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbac81121f0 con 0x7fbac8103af0 2024-04-22T13:24:37.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.169+0000 7fbacc975640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbac8112070 con 0x7fbac810c5a0 2024-04-22T13:24:37.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.169+0000 7fbacc975640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbac8112370 con 0x7fbac81031d0 2024-04-22T13:24:37.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.169+0000 7fbabffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbac8103af0 0x7fbac81a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:37.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.169+0000 7fbabffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbac8103af0 0x7fbac81a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44130/0 (socket says 172.21.15.12:44130) 2024-04-22T13:24:37.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.169+0000 7fbabffff640 1 -- 172.21.15.12:0/156704859 learned_addr learned my addr 172.21.15.12:0/156704859 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:37.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.169+0000 7fbac659c640 1 --2- 172.21.15.12:0/156704859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbac81031d0 0x7fbac81a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:37.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.169+0000 7fbac6d9d640 1 --2- 172.21.15.12:0/156704859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbac810c5a0 0x7fbac819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:37.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.169+0000 7fbabffff640 1 -- 172.21.15.12:0/156704859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbac81031d0 msgr2=0x7fbac81a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:37.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.169+0000 7fbabffff640 1 --2- 172.21.15.12:0/156704859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbac81031d0 0x7fbac81a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.169+0000 7fbabffff640 1 -- 172.21.15.12:0/156704859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbac810c5a0 msgr2=0x7fbac819a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:37.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.169+0000 7fbabffff640 1 --2- 172.21.15.12:0/156704859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbac810c5a0 0x7fbac819a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.169+0000 7fbabffff640 1 -- 172.21.15.12:0/156704859 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac819b000 con 0x7fbac8103af0 2024-04-22T13:24:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.169+0000 7fbac659c640 1 --2- 172.21.15.12:0/156704859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbac81031d0 0x7fbac81a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.169+0000 7fbabffff640 1 --2- 172.21.15.12:0/156704859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbac8103af0 0x7fbac81a0bb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbab000d770 tx=0x7fbab000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.169+0000 7fbabf7fe640 1 -- 172.21.15.12:0/156704859 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab0017070 con 0x7fbac8103af0 2024-04-22T13:24:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.169+0000 7fbacc975640 1 -- 172.21.15.12:0/156704859 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbac819b2f0 con 0x7fbac8103af0 2024-04-22T13:24:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.170+0000 7fbabf7fe640 1 -- 172.21.15.12:0/156704859 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbab00054d0 con 0x7fbac8103af0 2024-04-22T13:24:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.170+0000 7fbabf7fe640 1 -- 172.21.15.12:0/156704859 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab0004c30 con 0x7fbac8103af0 2024-04-22T13:24:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.170+0000 7fbacc975640 1 -- 172.21.15.12:0/156704859 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbac81a74c0 con 0x7fbac8103af0 2024-04-22T13:24:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.171+0000 7fbacc975640 1 -- 172.21.15.12:0/156704859 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbac806ae60 con 0x7fbac8103af0 2024-04-22T13:24:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.172+0000 7fbabf7fe640 1 -- 172.21.15.12:0/156704859 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fbab0004dd0 con 0x7fbac8103af0 2024-04-22T13:24:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.172+0000 7fbabf7fe640 1 --2- 172.21.15.12:0/156704859 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbab40a3ec0 0x7fbab40a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.173+0000 7fbac659c640 1 --2- 172.21.15.12:0/156704859 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbab40a3ec0 0x7fbab40a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.173+0000 7fbabf7fe640 1 -- 172.21.15.12:0/156704859 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fbab00d0990 con 0x7fbac8103af0 2024-04-22T13:24:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.173+0000 7fbac659c640 1 --2- 172.21.15.12:0/156704859 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbab40a3ec0 0x7fbab40a6380 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbab8005ab0 tx=0x7fbab8005a20 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:37.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.177+0000 7fbabf7fe640 1 -- 172.21.15.12:0/156704859 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbab009ceb0 con 0x7fbac8103af0 2024-04-22T13:24:37.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.409+0000 7fbacc975640 1 -- 172.21.15.12:0/156704859 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbac81a76f0 con 0x7fbac8103af0 2024-04-22T13:24:37.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.409+0000 7fbabf7fe640 1 -- 172.21.15.12:0/156704859 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7fbab009c7b0 con 0x7fbac8103af0 2024-04-22T13:24:37.406 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:24:37.406 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:24:37.406 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:24:37.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.411+0000 7fbacc975640 1 -- 172.21.15.12:0/156704859 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbab40a3ec0 msgr2=0x7fbab40a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:37.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.411+0000 7fbacc975640 1 --2- 172.21.15.12:0/156704859 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbab40a3ec0 0x7fbab40a6380 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbab8005ab0 tx=0x7fbab8005a20 comp rx=0 tx=0).stop 2024-04-22T13:24:37.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.412+0000 7fbacc975640 1 -- 172.21.15.12:0/156704859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbac8103af0 msgr2=0x7fbac81a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:37.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.412+0000 7fbacc975640 1 --2- 172.21.15.12:0/156704859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbac8103af0 0x7fbac81a0bb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbab000d770 tx=0x7fbab000dc30 comp rx=0 tx=0).stop 2024-04-22T13:24:37.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.412+0000 7fbacc975640 1 -- 172.21.15.12:0/156704859 shutdown_connections 2024-04-22T13:24:37.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.412+0000 7fbacc975640 1 --2- 172.21.15.12:0/156704859 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbab40a3ec0 0x7fbab40a6380 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.412+0000 7fbacc975640 1 --2- 172.21.15.12:0/156704859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbac810c5a0 0x7fbac819a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.412+0000 7fbacc975640 1 --2- 172.21.15.12:0/156704859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbac8103af0 0x7fbac81a0bb0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.412+0000 7fbacc975640 1 --2- 172.21.15.12:0/156704859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbac81031d0 0x7fbac81a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.412+0000 7fbacc975640 1 -- 172.21.15.12:0/156704859 >> 172.21.15.12:0/156704859 conn(0x7fbac80771d0 msgr2=0x7fbac8100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:37.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.412+0000 7fbacc975640 1 -- 172.21.15.12:0/156704859 shutdown_connections 2024-04-22T13:24:37.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.412+0000 7fbacc975640 1 -- 172.21.15.12:0/156704859 wait complete. 2024-04-22T13:24:37.419 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db / --timeout 60 --expiration 85.35 --await' 2024-04-22T13:24:37.419 DEBUG:teuthology.orchestra.run.smithi012:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24509 quiesce db / --timeout 60 --expiration 85.35 --await 2024-04-22T13:24:37.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.534+0000 7f24ffea8640 1 -- 172.21.15.12:0/1334248258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24f8103b50 msgr2=0x7f24f810c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:37.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.534+0000 7f24ffea8640 1 --2- 172.21.15.12:0/1334248258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24f8103b50 0x7f24f810c030 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009900 tx=0x7f24e802bcd0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.534+0000 7f24ffea8640 1 -- 172.21.15.12:0/1334248258 shutdown_connections 2024-04-22T13:24:37.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.534+0000 7f24ffea8640 1 --2- 172.21.15.12:0/1334248258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24f810c570 0x7f24f810e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.534+0000 7f24ffea8640 1 --2- 172.21.15.12:0/1334248258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24f8103b50 0x7f24f810c030 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.534+0000 7f24ffea8640 1 --2- 172.21.15.12:0/1334248258 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24f8103230 0x7f24f8103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.534+0000 7f24ffea8640 1 -- 172.21.15.12:0/1334248258 >> 172.21.15.12:0/1334248258 conn(0x7f24f8077fb0 msgr2=0x7f24f80783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:37.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.534+0000 7f24ffea8640 1 -- 172.21.15.12:0/1334248258 shutdown_connections 2024-04-22T13:24:37.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.534+0000 7f24ffea8640 1 -- 172.21.15.12:0/1334248258 wait complete. 2024-04-22T13:24:37.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.535+0000 7f24ffea8640 1 Processor -- start 2024-04-22T13:24:37.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.535+0000 7f24ffea8640 1 -- start start 2024-04-22T13:24:37.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.535+0000 7f24ffea8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24f8103230 0x7f24f81986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:37.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.535+0000 7f24ffea8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24f8103b50 0x7f24f8198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:37.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.535+0000 7f24ffea8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24f810c570 0x7f24f819cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:37.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.535+0000 7f24ffea8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f24f81121b0 con 0x7f24f8103230 2024-04-22T13:24:37.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.535+0000 7f24ffea8640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f24f8112030 con 0x7f24f8103b50 2024-04-22T13:24:37.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.535+0000 7f24ffea8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f24f8112330 con 0x7f24f810c570 2024-04-22T13:24:37.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.535+0000 7f24fdc44640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24f8103230 0x7f24f81986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:37.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.535+0000 7f24fdc44640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24f8103230 0x7f24f81986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44152/0 (socket says 172.21.15.12:44152) 2024-04-22T13:24:37.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.535+0000 7f24fdc44640 1 -- 172.21.15.12:0/4284670592 learned_addr learned my addr 172.21.15.12:0/4284670592 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:37.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.535+0000 7f24fe445640 1 --2- 172.21.15.12:0/4284670592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24f810c570 0x7f24f819cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:37.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.536+0000 7f24fd443640 1 --2- 172.21.15.12:0/4284670592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24f8103b50 0x7f24f8198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:37.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.536+0000 7f24fdc44640 1 -- 172.21.15.12:0/4284670592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24f810c570 msgr2=0x7f24f819cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:37.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.536+0000 7f24fdc44640 1 --2- 172.21.15.12:0/4284670592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24f810c570 0x7f24f819cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.536+0000 7f24fdc44640 1 -- 172.21.15.12:0/4284670592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24f8103b50 msgr2=0x7f24f8198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:37.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.536+0000 7f24fdc44640 1 --2- 172.21.15.12:0/4284670592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24f8103b50 0x7f24f8198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.536+0000 7f24fdc44640 1 -- 172.21.15.12:0/4284670592 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f819d6c0 con 0x7f24f8103230 2024-04-22T13:24:37.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.536+0000 7f24fd443640 1 --2- 172.21.15.12:0/4284670592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24f8103b50 0x7f24f8198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:37.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.536+0000 7f24fdc44640 1 --2- 172.21.15.12:0/4284670592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24f8103230 0x7f24f81986d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f24f400d950 tx=0x7f24f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:37.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.536+0000 7f24eeffd640 1 -- 172.21.15.12:0/4284670592 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f4017030 con 0x7f24f8103230 2024-04-22T13:24:37.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.536+0000 7f24ffea8640 1 -- 172.21.15.12:0/4284670592 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24f819d9b0 con 0x7f24f8103230 2024-04-22T13:24:37.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.536+0000 7f24eeffd640 1 -- 172.21.15.12:0/4284670592 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24f4004020 con 0x7f24f8103230 2024-04-22T13:24:37.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.536+0000 7f24eeffd640 1 -- 172.21.15.12:0/4284670592 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f40047a0 con 0x7f24f8103230 2024-04-22T13:24:37.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.536+0000 7f24ffea8640 1 -- 172.21.15.12:0/4284670592 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24f81a5240 con 0x7f24f8103230 2024-04-22T13:24:37.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.538+0000 7f24eeffd640 1 -- 172.21.15.12:0/4284670592 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f24f4009a30 con 0x7f24f8103230 2024-04-22T13:24:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.538+0000 7f2502b7b740 1 Processor -- start 2024-04-22T13:24:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.539+0000 7f2502b7b740 1 -- start start 2024-04-22T13:24:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.539+0000 7f2502b7b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639b9c9c6f0 0x5639b9c9cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.539+0000 7f2502b7b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5639b9c9d010 0x5639b9ca9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.539+0000 7f2502b7b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639b9caa2d0 0x5639b9cac6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.539+0000 7f2502b7b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5639b9bfd0b0 con 0x5639b9c9d010 2024-04-22T13:24:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.539+0000 7f2502b7b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5639b9bae680 con 0x5639b9c9c6f0 2024-04-22T13:24:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.539+0000 7f2502b7b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5639b9bbac60 con 0x5639b9caa2d0 2024-04-22T13:24:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.539+0000 7f24eeffd640 1 --2- 172.21.15.12:0/4284670592 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f24e40a3e40 0x7f24e40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.539+0000 7f24fdc44640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5639b9c9d010 0x5639b9ca9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.539+0000 7f24fdc44640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5639b9c9d010 0x5639b9ca9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44156/0 (socket says 172.21.15.12:44156) 2024-04-22T13:24:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.539+0000 7f24fdc44640 1 -- 172.21.15.12:0/3764333672 learned_addr learned my addr 172.21.15.12:0/3764333672 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.539+0000 7f24fe445640 1 --2- 172.21.15.12:0/3764333672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639b9c9c6f0 0x5639b9c9cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:37.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.539+0000 7f24fe445640 1 --2- 172.21.15.12:0/4284670592 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f24e40a3e40 0x7f24e40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.539+0000 7f24fd443640 1 --2- 172.21.15.12:0/3764333672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639b9caa2d0 0x5639b9cac6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.539+0000 7f24eeffd640 1 -- 172.21.15.12:0/4284670592 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f24f40d04e0 con 0x7f24f8103230 2024-04-22T13:24:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.539+0000 7f24fdc44640 1 -- 172.21.15.12:0/3764333672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639b9caa2d0 msgr2=0x5639b9cac6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.540+0000 7f24fdc44640 1 --2- 172.21.15.12:0/3764333672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639b9caa2d0 0x5639b9cac6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.540+0000 7f24fdc44640 1 -- 172.21.15.12:0/3764333672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639b9c9c6f0 msgr2=0x5639b9c9cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.540+0000 7f24fdc44640 1 --2- 172.21.15.12:0/3764333672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639b9c9c6f0 0x5639b9c9cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.540+0000 7f24fdc44640 1 -- 172.21.15.12:0/3764333672 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639b99dfc90 con 0x5639b9c9d010 2024-04-22T13:24:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.540+0000 7f24fd443640 1 --2- 172.21.15.12:0/3764333672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639b9caa2d0 0x5639b9cac6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.540+0000 7f24fdc44640 1 --2- 172.21.15.12:0/3764333672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5639b9c9d010 0x5639b9ca9d90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f24f40cdbc0 tx=0x7f24f4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6655aab96b4c05b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.540+0000 7f24fe445640 1 --2- 172.21.15.12:0/4284670592 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f24e40a3e40 0x7f24e40a6300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f24f0007740 tx=0x7f24f00079e0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.540+0000 7f24e3fff640 1 -- 172.21.15.12:0/3764333672 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f409c980 con 0x5639b9c9d010 2024-04-22T13:24:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.540+0000 7f24e3fff640 1 -- 172.21.15.12:0/3764333672 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24f409c280 con 0x5639b9c9d010 2024-04-22T13:24:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.540+0000 7f24e3fff640 1 -- 172.21.15.12:0/3764333672 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f40988f0 con 0x5639b9c9d010 2024-04-22T13:24:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.541+0000 7f2502b7b740 1 -- 172.21.15.12:0/3764333672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5639b9c9d010 msgr2=0x5639b9ca9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.541+0000 7f2502b7b740 1 --2- 172.21.15.12:0/3764333672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5639b9c9d010 0x5639b9ca9d90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f24f40cdbc0 tx=0x7f24f4000bc0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.541+0000 7f2502b7b740 1 -- 172.21.15.12:0/3764333672 shutdown_connections 2024-04-22T13:24:37.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.541+0000 7f2502b7b740 1 --2- 172.21.15.12:0/3764333672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639b9caa2d0 0x5639b9cac6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.541+0000 7f2502b7b740 1 --2- 172.21.15.12:0/3764333672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5639b9c9d010 0x5639b9ca9d90 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.541+0000 7f2502b7b740 1 --2- 172.21.15.12:0/3764333672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639b9c9c6f0 0x5639b9c9cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.541+0000 7f2502b7b740 1 -- 172.21.15.12:0/3764333672 >> 172.21.15.12:0/3764333672 conn(0x5639b9c970f0 msgr2=0x5639b9c974d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:37.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.541+0000 7f2502b7b740 1 -- 172.21.15.12:0/3764333672 shutdown_connections 2024-04-22T13:24:37.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.541+0000 7f2502b7b740 1 -- 172.21.15.12:0/3764333672 wait complete. 2024-04-22T13:24:37.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.541+0000 7f2502b7b740 1 -- start start 2024-04-22T13:24:37.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.541+0000 7f2502b7b740 1 Processor -- start 2024-04-22T13:24:37.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.542+0000 7f2502b7b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639b9c9c6f0 0x5639b9ca7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:37.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.542+0000 7f2502b7b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5639b9c9d010 0x5639b9ca78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:37.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.542+0000 7f2502b7b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639b9caa2d0 0x5639b9ca7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:37.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.542+0000 7f2502b7b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5639b9c00400 con 0x5639b9c9d010 2024-04-22T13:24:37.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.542+0000 7f2502b7b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5639b9bbc840 con 0x5639b9caa2d0 2024-04-22T13:24:37.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.542+0000 7f2502b7b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5639b9c05720 con 0x5639b9c9c6f0 2024-04-22T13:24:37.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.542+0000 7f2502b7b740 10 client.0 _ll_register_callbacks cb 0x5639b9c0dea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:37.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.542+0000 7f24fd443640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5639b9c9d010 0x5639b9ca78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:37.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.542+0000 7f24fd443640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5639b9c9d010 0x5639b9ca78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44158/0 (socket says 172.21.15.12:44158) 2024-04-22T13:24:37.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.542+0000 7f24fd443640 1 -- 172.21.15.12:0/29868638 learned_addr learned my addr 172.21.15.12:0/29868638 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:37.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.542+0000 7f24fdc44640 1 --2- 172.21.15.12:0/29868638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639b9c9c6f0 0x5639b9ca7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:37.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.542+0000 7f24fe445640 1 --2- 172.21.15.12:0/29868638 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639b9caa2d0 0x5639b9ca7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:37.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.542+0000 7f24fd443640 1 -- 172.21.15.12:0/29868638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639b9c9c6f0 msgr2=0x5639b9ca7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:37.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.542+0000 7f24fd443640 1 --2- 172.21.15.12:0/29868638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639b9c9c6f0 0x5639b9ca7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.542+0000 7f24fd443640 1 -- 172.21.15.12:0/29868638 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639b9caa2d0 msgr2=0x5639b9ca7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:37.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.542+0000 7f24fd443640 1 --2- 172.21.15.12:0/29868638 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639b9caa2d0 0x5639b9ca7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.542+0000 7f24fd443640 1 -- 172.21.15.12:0/29868638 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639b9a40420 con 0x5639b9c9d010 2024-04-22T13:24:37.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.542+0000 7f24fdc44640 1 --2- 172.21.15.12:0/29868638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639b9c9c6f0 0x5639b9ca7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:37.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.542+0000 7f24fe445640 1 --2- 172.21.15.12:0/29868638 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639b9caa2d0 0x5639b9ca7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:37.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.542+0000 7f24fd443640 1 --2- 172.21.15.12:0/29868638 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5639b9c9d010 0x5639b9ca78a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f24e8000c00 tx=0x7f24e80052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6676defe727a626a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:37.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.542+0000 7f24e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:37.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.542+0000 7f24e17fa640 1 -- 172.21.15.12:0/29868638 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8042030 con 0x5639b9c9d010 2024-04-22T13:24:37.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.542+0000 7f24e17fa640 1 -- 172.21.15.12:0/29868638 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f24e802c880 con 0x5639b9c9d010 2024-04-22T13:24:37.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.542+0000 7f2502b7b740 1 -- 172.21.15.12:0/29868638 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24f409c280 con 0x5639b9c9d010 2024-04-22T13:24:37.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.543+0000 7f24e17fa640 1 -- 172.21.15.12:0/29868638 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f24e804b020 con 0x5639b9c9d010 2024-04-22T13:24:37.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.543+0000 7f2502b7b740 10 client.15840 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:37.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.543+0000 7f2502b7b740 1 -- 172.21.15.12:0/29868638 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5639b98d3580 con 0x5639b9c9d010 2024-04-22T13:24:37.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.543+0000 7f24e17fa640 1 -- 172.21.15.12:0/29868638 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24f409c280 con 0x5639b9c9d010 2024-04-22T13:24:37.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.543+0000 7f24e17fa640 1 -- 172.21.15.12:0/29868638 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f24e8004cc0 con 0x5639b9c9d010 2024-04-22T13:24:37.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.543+0000 7f2502b7b740 10 client.15840 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:37.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.543+0000 7f2502b7b740 10 client.15840 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-22T13:24:37.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.543+0000 7f2502b7b740 20 client.15840 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:37.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.543+0000 7f2502b7b740 1 --2- 172.21.15.12:0/29868638 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5639b9cb9f70 0x5639b9cbc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:37.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.543+0000 7f2502b7b740 4 client.15840 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:24:37.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.543+0000 7f2502b7b740 1 -- 172.21.15.12:0/29868638 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639b9c0eb10 con 0x5639b9cb9f70 2024-04-22T13:24:37.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.543+0000 7f24fdc44640 1 --2- 172.21.15.12:0/29868638 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5639b9cb9f70 0x5639b9cbc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:37.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.544+0000 7f24e17fa640 10 client.15840 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:37.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.544+0000 7f24fdc44640 1 --2- 172.21.15.12:0/29868638 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5639b9cb9f70 0x5639b9cbc3e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=221e07fc3ac4e923 server_cookie=e8b38e9f25e8236 in_seq=0 out_seq=0 2024-04-22T13:24:37.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.544+0000 7f24e17fa640 1 -- 172.21.15.12:0/29868638 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5639b9c0eb10 con 0x5639b9cb9f70 2024-04-22T13:24:37.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.544+0000 7f24e17fa640 10 client.15840 handle_command_reply: tid=0 2024-04-22T13:24:37.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.544+0000 7f2502b7b740 1 client.15840 shutdown 2024-04-22T13:24:37.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.544+0000 7f2502b7b740 1 -- 172.21.15.12:0/29868638 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5639b9c9d010 msgr2=0x5639b9ca78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:37.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.544+0000 7f2502b7b740 1 --2- 172.21.15.12:0/29868638 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5639b9c9d010 0x5639b9ca78a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f24e8000c00 tx=0x7f24e80052f0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.545+0000 7f2502b7b740 1 -- 172.21.15.12:0/29868638 shutdown_connections 2024-04-22T13:24:37.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.545+0000 7f2502b7b740 1 --2- 172.21.15.12:0/29868638 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5639b9cb9f70 0x5639b9cbc3e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.545+0000 7f2502b7b740 1 --2- 172.21.15.12:0/29868638 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639b9caa2d0 0x5639b9ca7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.545+0000 7f2502b7b740 1 --2- 172.21.15.12:0/29868638 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5639b9c9d010 0x5639b9ca78a0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.545+0000 7f2502b7b740 1 --2- 172.21.15.12:0/29868638 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639b9c9c6f0 0x5639b9ca7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.545+0000 7f24e17fa640 0 client.15840 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:37.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.545+0000 7f2502b7b740 1 -- 172.21.15.12:0/29868638 >> 172.21.15.12:0/29868638 conn(0x5639b9c970f0 msgr2=0x5639b9c98e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:37.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.545+0000 7f2502b7b740 1 -- 172.21.15.12:0/29868638 shutdown_connections 2024-04-22T13:24:37.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.545+0000 7f2502b7b740 1 -- 172.21.15.12:0/29868638 wait complete. 2024-04-22T13:24:37.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.545+0000 7f2502b7b740 20 client.15840 trim_cache size 0 max 16384 2024-04-22T13:24:37.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.562+0000 7f2502b7b740 1 Processor -- start 2024-04-22T13:24:37.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.562+0000 7f2502b7b740 1 -- start start 2024-04-22T13:24:37.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.562+0000 7f2502b7b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639b9c9c6f0 0x5639b9cb5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:37.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.562+0000 7f2502b7b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639b9c9d010 0x5639b9ca7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:37.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.562+0000 7f2502b7b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5639b9caa2d0 0x5639b9cc1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:37.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.562+0000 7f2502b7b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f24f81ad3b0 con 0x5639b9caa2d0 2024-04-22T13:24:37.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.562+0000 7f2502b7b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f24f81abf40 con 0x5639b9c9c6f0 2024-04-22T13:24:37.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.562+0000 7f2502b7b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f24f81ac610 con 0x5639b9c9d010 2024-04-22T13:24:37.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.562+0000 7f24fe445640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5639b9caa2d0 0x5639b9cc1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:37.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.562+0000 7f24fe445640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5639b9caa2d0 0x5639b9cc1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44162/0 (socket says 172.21.15.12:44162) 2024-04-22T13:24:37.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.562+0000 7f24fe445640 1 -- 172.21.15.12:0/630893794 learned_addr learned my addr 172.21.15.12:0/630893794 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:37.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.562+0000 7f24fdc44640 1 --2- 172.21.15.12:0/630893794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639b9c9c6f0 0x5639b9cb5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:37.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.562+0000 7f24fe445640 1 -- 172.21.15.12:0/630893794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639b9c9d010 msgr2=0x5639b9ca7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:37.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.562+0000 7f24fe445640 1 --2- 172.21.15.12:0/630893794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639b9c9d010 0x5639b9ca7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.562+0000 7f24fe445640 1 -- 172.21.15.12:0/630893794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639b9c9c6f0 msgr2=0x5639b9cb5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:37.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.562+0000 7f24fe445640 1 --2- 172.21.15.12:0/630893794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639b9c9c6f0 0x5639b9cb5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.563+0000 7f24fe445640 1 -- 172.21.15.12:0/630893794 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639b9c96950 con 0x5639b9caa2d0 2024-04-22T13:24:37.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.563+0000 7f24fdc44640 1 --2- 172.21.15.12:0/630893794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639b9c9c6f0 0x5639b9cb5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:37.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.563+0000 7f24fe445640 1 --2- 172.21.15.12:0/630893794 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5639b9caa2d0 0x5639b9cc1cf0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f24f0014870 tx=0x7f24f001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=217fb71c099ee7c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:37.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.563+0000 7f24e37fe640 1 -- 172.21.15.12:0/630893794 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f0004d20 con 0x5639b9caa2d0 2024-04-22T13:24:37.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.563+0000 7f24e37fe640 1 -- 172.21.15.12:0/630893794 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24f00046f0 con 0x5639b9caa2d0 2024-04-22T13:24:37.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.563+0000 7f24e37fe640 1 -- 172.21.15.12:0/630893794 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f0005800 con 0x5639b9caa2d0 2024-04-22T13:24:37.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.563+0000 7f2502b7b740 1 -- 172.21.15.12:0/630893794 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5639b9caa2d0 msgr2=0x5639b9cc1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:37.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.563+0000 7f2502b7b740 1 --2- 172.21.15.12:0/630893794 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5639b9caa2d0 0x5639b9cc1cf0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f24f0014870 tx=0x7f24f001e000 comp rx=0 tx=0).stop 2024-04-22T13:24:37.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.564+0000 7f2502b7b740 1 -- 172.21.15.12:0/630893794 shutdown_connections 2024-04-22T13:24:37.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.564+0000 7f2502b7b740 1 --2- 172.21.15.12:0/630893794 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5639b9caa2d0 0x5639b9cc1cf0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.564+0000 7f2502b7b740 1 --2- 172.21.15.12:0/630893794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639b9c9d010 0x5639b9ca7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.564+0000 7f2502b7b740 1 --2- 172.21.15.12:0/630893794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639b9c9c6f0 0x5639b9cb5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.564+0000 7f2502b7b740 1 -- 172.21.15.12:0/630893794 >> 172.21.15.12:0/630893794 conn(0x5639b9c970f0 msgr2=0x5639b9ca7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:37.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.564+0000 7f2502b7b740 1 -- 172.21.15.12:0/630893794 shutdown_connections 2024-04-22T13:24:37.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.564+0000 7f2502b7b740 1 -- 172.21.15.12:0/630893794 wait complete. 2024-04-22T13:24:37.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.564+0000 7f2502b7b740 1 -- start start 2024-04-22T13:24:37.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.564+0000 7f2502b7b740 1 Processor -- start 2024-04-22T13:24:37.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.564+0000 7f2502b7b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639b9c9c6f0 0x5639b9caf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:37.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.564+0000 7f2502b7b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5639b9c9d010 0x5639b9caf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:37.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.564+0000 7f2502b7b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639b9caa2d0 0x5639b9cafb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:37.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.564+0000 7f2502b7b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f24f81a9870 con 0x5639b9c9d010 2024-04-22T13:24:37.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.564+0000 7f2502b7b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f24f81acce0 con 0x5639b9c9c6f0 2024-04-22T13:24:37.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.564+0000 7f2502b7b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f24f81a9ec0 con 0x5639b9caa2d0 2024-04-22T13:24:37.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.564+0000 7f2502b7b740 10 client.0 _ll_register_callbacks cb 0x7f24e8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:37.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.564+0000 7f24fd443640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5639b9c9d010 0x5639b9caf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:37.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.565+0000 7f24fd443640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5639b9c9d010 0x5639b9caf650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44174/0 (socket says 172.21.15.12:44174) 2024-04-22T13:24:37.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.565+0000 7f24fd443640 1 -- 172.21.15.12:0/4241293696 learned_addr learned my addr 172.21.15.12:0/4241293696 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:37.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.565+0000 7f24fe445640 1 --2- 172.21.15.12:0/4241293696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639b9caa2d0 0x5639b9cafb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:37.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.565+0000 7f24fdc44640 1 --2- 172.21.15.12:0/4241293696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639b9c9c6f0 0x5639b9caf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:37.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.565+0000 7f24fd443640 1 -- 172.21.15.12:0/4241293696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639b9caa2d0 msgr2=0x5639b9cafb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:37.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.565+0000 7f24fd443640 1 --2- 172.21.15.12:0/4241293696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639b9caa2d0 0x5639b9cafb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.565+0000 7f24fd443640 1 -- 172.21.15.12:0/4241293696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639b9c9c6f0 msgr2=0x5639b9caf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:37.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.565+0000 7f24fd443640 1 --2- 172.21.15.12:0/4241293696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639b9c9c6f0 0x5639b9caf110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.565+0000 7f24fd443640 1 -- 172.21.15.12:0/4241293696 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639b9d20670 con 0x5639b9c9d010 2024-04-22T13:24:37.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.565+0000 7f24fd443640 1 --2- 172.21.15.12:0/4241293696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5639b9c9d010 0x5639b9caf650 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009850 tx=0x7f24e8005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33a903dc6c1378d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:37.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.565+0000 7f24e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:37.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.565+0000 7f24e17fa640 1 -- 172.21.15.12:0/4241293696 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8055060 con 0x5639b9c9d010 2024-04-22T13:24:37.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.565+0000 7f2502b7b740 1 -- 172.21.15.12:0/4241293696 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24f00046f0 con 0x5639b9c9d010 2024-04-22T13:24:37.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.565+0000 7f24e17fa640 1 -- 172.21.15.12:0/4241293696 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f24e802c880 con 0x5639b9c9d010 2024-04-22T13:24:37.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.565+0000 7f24e17fa640 1 -- 172.21.15.12:0/4241293696 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f24e8042020 con 0x5639b9c9d010 2024-04-22T13:24:37.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.565+0000 7f2502b7b740 10 client.15846 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:37.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.565+0000 7f2502b7b740 1 -- 172.21.15.12:0/4241293696 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5639b9d218c0 con 0x5639b9c9d010 2024-04-22T13:24:37.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.565+0000 7f24e17fa640 1 -- 172.21.15.12:0/4241293696 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24f00046f0 con 0x5639b9c9d010 2024-04-22T13:24:37.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.566+0000 7f24e17fa640 1 -- 172.21.15.12:0/4241293696 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f24e8004bf0 con 0x5639b9c9d010 2024-04-22T13:24:37.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.566+0000 7f2502b7b740 10 client.15846 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:37.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.566+0000 7f2502b7b740 10 client.15846 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-22T13:24:37.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.566+0000 7f2502b7b740 20 client.15846 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:37.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.566+0000 7f2502b7b740 1 --2- 172.21.15.12:0/4241293696 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5639b9d22130 0x5639b9d24550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:37.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.566+0000 7f2502b7b740 4 client.15846 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.35, "await": true}] 2024-04-22T13:24:37.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.566+0000 7f2502b7b740 1 -- 172.21.15.12:0/4241293696 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.35, "await": true}) -- 0x5639b9c08980 con 0x5639b9d22130 2024-04-22T13:24:37.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.566+0000 7f24fdc44640 1 --2- 172.21.15.12:0/4241293696 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5639b9d22130 0x5639b9d24550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:37.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.566+0000 7f24fdc44640 1 --2- 172.21.15.12:0/4241293696 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5639b9d22130 0x5639b9d24550 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8753c6defd31fe8 server_cookie=5f04a4f457c2343d in_seq=0 out_seq=0 2024-04-22T13:24:37.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.566+0000 7f24e17fa640 10 client.15846 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:37.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.579+0000 7f24e17fa640 1 -- 172.21.15.12:0/4241293696 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x7f24f4014260 con 0x5639b9d22130 2024-04-22T13:24:37.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.579+0000 7f24e17fa640 10 client.15846 handle_command_reply: tid=0 2024-04-22T13:24:37.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.579+0000 7f2502b7b740 1 client.15846 shutdown 2024-04-22T13:24:37.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.580+0000 7f2502b7b740 1 -- 172.21.15.12:0/4241293696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5639b9c9d010 msgr2=0x5639b9caf650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:37.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.580+0000 7f2502b7b740 1 --2- 172.21.15.12:0/4241293696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5639b9c9d010 0x5639b9caf650 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009850 tx=0x7f24e8005770 comp rx=0 tx=0).stop 2024-04-22T13:24:37.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.580+0000 7f2502b7b740 1 -- 172.21.15.12:0/4241293696 shutdown_connections 2024-04-22T13:24:37.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.580+0000 7f2502b7b740 1 --2- 172.21.15.12:0/4241293696 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5639b9d22130 0x5639b9d24550 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.580+0000 7f2502b7b740 1 --2- 172.21.15.12:0/4241293696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5639b9caa2d0 0x5639b9cafb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.580+0000 7f2502b7b740 1 --2- 172.21.15.12:0/4241293696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5639b9c9d010 0x5639b9caf650 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.580+0000 7f2502b7b740 1 --2- 172.21.15.12:0/4241293696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5639b9c9c6f0 0x5639b9caf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.580+0000 7f24e17fa640 0 client.15846 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:37.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.580+0000 7f2502b7b740 1 -- 172.21.15.12:0/4241293696 >> 172.21.15.12:0/4241293696 conn(0x5639b9c970f0 msgr2=0x5639b9c961d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:37.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.580+0000 7f2502b7b740 1 -- 172.21.15.12:0/4241293696 shutdown_connections 2024-04-22T13:24:37.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.580+0000 7f2502b7b740 1 -- 172.21.15.12:0/4241293696 wait complete. 2024-04-22T13:24:37.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.580+0000 7f2502b7b740 20 client.15846 trim_cache size 0 max 16384 2024-04-22T13:24:37.577 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:24:37.578 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 28, 2024-04-22T13:24:37.578 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24509, 2024-04-22T13:24:37.578 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 2, 2024-04-22T13:24:37.578 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-04-22T13:24:37.578 INFO:teuthology.orchestra.run.smithi012.stdout: "fab6558f": { 2024-04-22T13:24:37.578 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 2, 2024-04-22T13:24:37.578 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-04-22T13:24:37.578 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-04-22T13:24:37.578 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-04-22T13:24:37.578 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-04-22T13:24:37.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:24:37.578 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 60.0, 2024-04-22T13:24:37.578 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 85.3, 2024-04-22T13:24:37.579 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-04-22T13:24:37.579 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-04-22T13:24:37.579 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-04-22T13:24:37.579 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-04-22T13:24:37.579 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-04-22T13:24:37.579 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-04-22T13:24:37.579 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:24:37.579 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:24:37.579 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:24:37.579 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:24:37.579 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:24:37.579 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:24:37.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.580+0000 7f24ffea8640 1 -- 172.21.15.12:0/4284670592 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f24e40a3e40 msgr2=0x7f24e40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:37.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.580+0000 7f24ffea8640 1 --2- 172.21.15.12:0/4284670592 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f24e40a3e40 0x7f24e40a6300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f24f0007740 tx=0x7f24f00079e0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.580+0000 7f24ffea8640 1 -- 172.21.15.12:0/4284670592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24f8103230 msgr2=0x7f24f81986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:37.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.580+0000 7f24ffea8640 1 --2- 172.21.15.12:0/4284670592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24f8103230 0x7f24f81986d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f24f400d950 tx=0x7f24f400de10 comp rx=0 tx=0).stop 2024-04-22T13:24:37.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.580+0000 7f24ffea8640 1 -- 172.21.15.12:0/4284670592 shutdown_connections 2024-04-22T13:24:37.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.580+0000 7f24ffea8640 1 --2- 172.21.15.12:0/4284670592 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f24e40a3e40 0x7f24e40a6300 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.580+0000 7f24ffea8640 1 --2- 172.21.15.12:0/4284670592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24f810c570 0x7f24f819cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.580+0000 7f24ffea8640 1 --2- 172.21.15.12:0/4284670592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f24f8103b50 0x7f24f8198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.580+0000 7f24ffea8640 1 --2- 172.21.15.12:0/4284670592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f24f8103230 0x7f24f81986d0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:37.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.580+0000 7f24ffea8640 1 -- 172.21.15.12:0/4284670592 >> 172.21.15.12:0/4284670592 conn(0x7f24f8077fb0 msgr2=0x7f24f80fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:37.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.580+0000 7f24ffea8640 1 -- 172.21.15.12:0/4284670592 shutdown_connections 2024-04-22T13:24:37.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:37.581+0000 7f24ffea8640 1 -- 172.21.15.12:0/4284670592 wait complete. 2024-04-22T13:24:37.587 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced, set_id: fab6558f, quiesce took 0.5 seconds. Will release after: 25.35 2024-04-22T13:24:38.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:37 smithi012 ceph-mon[28265]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-22T13:24:38.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:37 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/156704859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:38.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:37 smithi134 ceph-mon[31338]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-22T13:24:38.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:37 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/156704859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:38.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:37 smithi111 ceph-mon[31361]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-22T13:24:38.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:37 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/156704859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:40.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:39 smithi012 ceph-mon[28265]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 16 op/s 2024-04-22T13:24:40.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:39 smithi134 ceph-mon[31338]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 16 op/s 2024-04-22T13:24:40.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:39 smithi111 ceph-mon[31361]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 16 op/s 2024-04-22T13:24:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.227+0000 7f33c2c43640 1 -- 172.21.15.12:0/1263314907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33bc103200 msgr2=0x7f33bc1035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.227+0000 7f33c2c43640 1 --2- 172.21.15.12:0/1263314907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33bc103200 0x7f33bc1035e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009900 tx=0x7f33b002bcd0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.227+0000 7f33c2c43640 1 -- 172.21.15.12:0/1263314907 shutdown_connections 2024-04-22T13:24:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.227+0000 7f33c2c43640 1 --2- 172.21.15.12:0/1263314907 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33bc10c5d0 0x7f33bc10ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.227+0000 7f33c2c43640 1 --2- 172.21.15.12:0/1263314907 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33bc103b20 0x7f33bc10c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.227+0000 7f33c2c43640 1 --2- 172.21.15.12:0/1263314907 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33bc103200 0x7f33bc1035e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.227+0000 7f33c2c43640 1 -- 172.21.15.12:0/1263314907 >> 172.21.15.12:0/1263314907 conn(0x7f33bc0771d0 msgr2=0x7f33bc0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.227+0000 7f33c2c43640 1 -- 172.21.15.12:0/1263314907 shutdown_connections 2024-04-22T13:24:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.227+0000 7f33c2c43640 1 -- 172.21.15.12:0/1263314907 wait complete. 2024-04-22T13:24:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.227+0000 7f33c2c43640 1 Processor -- start 2024-04-22T13:24:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.227+0000 7f33c2c43640 1 -- start start 2024-04-22T13:24:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.228+0000 7f33c2c43640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33bc103200 0x7f33bc1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.228+0000 7f33c2c43640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33bc103b20 0x7f33bc1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.228+0000 7f33c2c43640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33bc10c5d0 0x7f33bc19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.228+0000 7f33c2c43640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc112200 con 0x7f33bc103200 2024-04-22T13:24:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.228+0000 7f33c2c43640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc112080 con 0x7f33bc10c5d0 2024-04-22T13:24:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.228+0000 7f33c2c43640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc112380 con 0x7f33bc103b20 2024-04-22T13:24:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.228+0000 7f33c09df640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33bc103200 0x7f33bc1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.228+0000 7f33c09df640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33bc103200 0x7f33bc1a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44186/0 (socket says 172.21.15.12:44186) 2024-04-22T13:24:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.228+0000 7f33c09df640 1 -- 172.21.15.12:0/3838408109 learned_addr learned my addr 172.21.15.12:0/3838408109 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.228+0000 7f33c11e0640 1 --2- 172.21.15.12:0/3838408109 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33bc10c5d0 0x7f33bc19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:41.226 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.228+0000 7f33c09df640 1 -- 172.21.15.12:0/3838408109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33bc103b20 msgr2=0x7f33bc1a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.228+0000 7f33c09df640 1 --2- 172.21.15.12:0/3838408109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33bc103b20 0x7f33bc1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.228+0000 7f33c09df640 1 -- 172.21.15.12:0/3838408109 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33bc10c5d0 msgr2=0x7f33bc19a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.228+0000 7f33c09df640 1 --2- 172.21.15.12:0/3838408109 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33bc10c5d0 0x7f33bc19a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.228+0000 7f33c09df640 1 -- 172.21.15.12:0/3838408109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33bc19b000 con 0x7f33bc103200 2024-04-22T13:24:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.228+0000 7f33c09df640 1 --2- 172.21.15.12:0/3838408109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33bc103200 0x7f33bc1a0680 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f33b0005940 tx=0x7f33b002d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.229+0000 7f33b9ffb640 1 -- 172.21.15.12:0/3838408109 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b0043020 con 0x7f33bc103200 2024-04-22T13:24:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.229+0000 7f33b9ffb640 1 -- 172.21.15.12:0/3838408109 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33b002dc90 con 0x7f33bc103200 2024-04-22T13:24:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.229+0000 7f33c2c43640 1 -- 172.21.15.12:0/3838408109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33bc19b290 con 0x7f33bc103200 2024-04-22T13:24:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.229+0000 7f33b9ffb640 1 -- 172.21.15.12:0/3838408109 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b0003ab0 con 0x7f33bc103200 2024-04-22T13:24:41.227 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.229+0000 7f33c2c43640 1 -- 172.21.15.12:0/3838408109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33bc1a7600 con 0x7f33bc103200 2024-04-22T13:24:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.231+0000 7f33b9ffb640 1 -- 172.21.15.12:0/3838408109 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f33b0046020 con 0x7f33bc103200 2024-04-22T13:24:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.231+0000 7f33c5836740 1 Processor -- start 2024-04-22T13:24:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.231+0000 7f33c5836740 1 -- start start 2024-04-22T13:24:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.232+0000 7f33c5836740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557bf5cf8d0 0x5557bf5ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.232+0000 7f33c5836740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557bf5ceb60 0x5557bf5cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.232+0000 7f33c5836740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bf5d80e0 0x5557bf5da4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:41.233 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.232+0000 7f33c5836740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5557bf5338d0 con 0x5557bf5d80e0 2024-04-22T13:24:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.232+0000 7f33c5836740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5557bf4d0e80 con 0x5557bf5ceb60 2024-04-22T13:24:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.232+0000 7f33c5836740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5557bf52f890 con 0x5557bf5cf8d0 2024-04-22T13:24:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.232+0000 7f33bbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bf5d80e0 0x5557bf5da4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.232+0000 7f33bbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bf5d80e0 0x5557bf5da4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44192/0 (socket says 172.21.15.12:44192) 2024-04-22T13:24:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.232+0000 7f33bbfff640 1 -- 172.21.15.12:0/585234566 learned_addr learned my addr 172.21.15.12:0/585234566 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.232+0000 7f33c09df640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557bf5ceb60 0x5557bf5cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.232+0000 7f33b9ffb640 1 --2- 172.21.15.12:0/3838408109 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f33ac0a3ee0 0x7f33ac0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.232+0000 7f33c11e0640 1 --2- 172.21.15.12:0/585234566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557bf5cf8d0 0x5557bf5ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.232+0000 7f33bbfff640 1 -- 172.21.15.12:0/585234566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557bf5cf8d0 msgr2=0x5557bf5ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.232+0000 7f33bbfff640 1 --2- 172.21.15.12:0/585234566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557bf5cf8d0 0x5557bf5ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.232+0000 7f33bbfff640 1 -- 172.21.15.12:0/585234566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557bf5ceb60 msgr2=0x5557bf5cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.232+0000 7f33bbfff640 1 --2- 172.21.15.12:0/585234566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557bf5ceb60 0x5557bf5cefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.232+0000 7f33bbfff640 1 -- 172.21.15.12:0/585234566 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557bf4f2320 con 0x5557bf5d80e0 2024-04-22T13:24:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.232+0000 7f33c11e0640 1 --2- 172.21.15.12:0/585234566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557bf5cf8d0 0x5557bf5ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:41.234 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.232+0000 7f33b9ffb640 1 -- 172.21.15.12:0/3838408109 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f33b00f2a90 con 0x7f33bc103200 2024-04-22T13:24:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.232+0000 7f33c11e0640 1 --2- 172.21.15.12:0/3838408109 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f33ac0a3ee0 0x7f33ac0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.232+0000 7f33bbfff640 1 --2- 172.21.15.12:0/585234566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bf5d80e0 0x5557bf5da4d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f33a800aa40 tx=0x7f33a800af00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ec0fa2b84006f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.233+0000 7f33a6ffd640 1 -- 172.21.15.12:0/585234566 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a800ce40 con 0x5557bf5d80e0 2024-04-22T13:24:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.233+0000 7f33a6ffd640 1 -- 172.21.15.12:0/585234566 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33a80035f0 con 0x5557bf5d80e0 2024-04-22T13:24:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.233+0000 7f33a6ffd640 1 -- 172.21.15.12:0/585234566 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a800f690 con 0x5557bf5d80e0 2024-04-22T13:24:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.233+0000 7f33c11e0640 1 --2- 172.21.15.12:0/3838408109 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f33ac0a3ee0 0x7f33ac0a63a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f33bc06b860 tx=0x7f33b4004a40 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.233+0000 7f33c5836740 1 -- 172.21.15.12:0/585234566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bf5d80e0 msgr2=0x5557bf5da4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.233+0000 7f33c5836740 1 --2- 172.21.15.12:0/585234566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bf5d80e0 0x5557bf5da4d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f33a800aa40 tx=0x7f33a800af00 comp rx=0 tx=0).stop 2024-04-22T13:24:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.233+0000 7f33c5836740 1 -- 172.21.15.12:0/585234566 shutdown_connections 2024-04-22T13:24:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.233+0000 7f33c5836740 1 --2- 172.21.15.12:0/585234566 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bf5d80e0 0x5557bf5da4d0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.233+0000 7f33c5836740 1 --2- 172.21.15.12:0/585234566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557bf5ceb60 0x5557bf5cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.233+0000 7f33c5836740 1 --2- 172.21.15.12:0/585234566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557bf5cf8d0 0x5557bf5ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.233+0000 7f33c5836740 1 -- 172.21.15.12:0/585234566 >> 172.21.15.12:0/585234566 conn(0x5557bf595c90 msgr2=0x5557bf5c9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.233+0000 7f33c5836740 1 -- 172.21.15.12:0/585234566 shutdown_connections 2024-04-22T13:24:41.235 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.234+0000 7f33c5836740 1 -- 172.21.15.12:0/585234566 wait complete. 2024-04-22T13:24:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.234+0000 7f33c5836740 1 -- start start 2024-04-22T13:24:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.234+0000 7f33c5836740 1 Processor -- start 2024-04-22T13:24:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.234+0000 7f33c5836740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557bf5ceb60 0x5557bf5d79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.234+0000 7f33c5836740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557bf5cf8d0 0x5557bf5d4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.234+0000 7f33c5836740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bf5d80e0 0x5557bf5d52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.234+0000 7f33c5836740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5557bf5dae10 con 0x5557bf5d80e0 2024-04-22T13:24:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.234+0000 7f33c5836740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5557bf4bbeb0 con 0x5557bf5cf8d0 2024-04-22T13:24:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.234+0000 7f33c5836740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5557bf5daf90 con 0x5557bf5ceb60 2024-04-22T13:24:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.235+0000 7f33c5836740 10 client.0 _ll_register_callbacks cb 0x5557bf4e8b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.235+0000 7f33c11e0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bf5d80e0 0x5557bf5d52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.235+0000 7f33c11e0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bf5d80e0 0x5557bf5d52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44208/0 (socket says 172.21.15.12:44208) 2024-04-22T13:24:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.235+0000 7f33c11e0640 1 -- 172.21.15.12:0/1920487600 learned_addr learned my addr 172.21.15.12:0/1920487600 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.235+0000 7f33c09df640 1 --2- 172.21.15.12:0/1920487600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557bf5ceb60 0x5557bf5d79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.235+0000 7f33bbfff640 1 --2- 172.21.15.12:0/1920487600 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557bf5cf8d0 0x5557bf5d4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.235+0000 7f33c11e0640 1 -- 172.21.15.12:0/1920487600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557bf5ceb60 msgr2=0x5557bf5d79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.235+0000 7f33c11e0640 1 --2- 172.21.15.12:0/1920487600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557bf5ceb60 0x5557bf5d79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.235+0000 7f33c11e0640 1 -- 172.21.15.12:0/1920487600 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557bf5cf8d0 msgr2=0x5557bf5d4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.235+0000 7f33c11e0640 1 --2- 172.21.15.12:0/1920487600 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557bf5cf8d0 0x5557bf5d4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.235+0000 7f33c11e0640 1 -- 172.21.15.12:0/1920487600 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557bf246210 con 0x5557bf5d80e0 2024-04-22T13:24:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.235+0000 7f33c11e0640 1 --2- 172.21.15.12:0/1920487600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bf5d80e0 0x5557bf5d52b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f33b4009fd0 tx=0x7f33b4009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869eb87b224a9a2f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.235+0000 7f339bfff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.235+0000 7f339bfff640 1 -- 172.21.15.12:0/1920487600 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b4023070 con 0x5557bf5d80e0 2024-04-22T13:24:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.235+0000 7f339bfff640 1 -- 172.21.15.12:0/1920487600 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f33b400ece0 con 0x5557bf5d80e0 2024-04-22T13:24:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.235+0000 7f33c5836740 1 -- 172.21.15.12:0/1920487600 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33a80035f0 con 0x5557bf5d80e0 2024-04-22T13:24:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.236+0000 7f339bfff640 1 -- 172.21.15.12:0/1920487600 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f33b402f050 con 0x5557bf5d80e0 2024-04-22T13:24:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.236+0000 7f33c5836740 10 client.15858 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.236+0000 7f33c5836740 1 -- 172.21.15.12:0/1920487600 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5557bf312620 con 0x5557bf5d80e0 2024-04-22T13:24:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.236+0000 7f339bfff640 1 -- 172.21.15.12:0/1920487600 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33a80035f0 con 0x5557bf5d80e0 2024-04-22T13:24:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.236+0000 7f339bfff640 1 -- 172.21.15.12:0/1920487600 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f33b400f040 con 0x5557bf5d80e0 2024-04-22T13:24:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.236+0000 7f33c5836740 10 client.15858 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.236+0000 7f33c5836740 10 client.15858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:24:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.236+0000 7f33c5836740 20 client.15858 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.236+0000 7f33c5836740 1 --2- 172.21.15.12:0/1920487600 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5557bf5e8080 0x5557bf5ea4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.236+0000 7f33c5836740 4 client.15858 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:24:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.236+0000 7f33c5836740 1 -- 172.21.15.12:0/1920487600 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557bf547880 con 0x5557bf5e8080 2024-04-22T13:24:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.236+0000 7f33c09df640 1 --2- 172.21.15.12:0/1920487600 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5557bf5e8080 0x5557bf5ea4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.237+0000 7f33c09df640 1 --2- 172.21.15.12:0/1920487600 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5557bf5e8080 0x5557bf5ea4b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ebff8ebbfdf29d3 server_cookie=8dc6c79ab39c1f21 in_seq=0 out_seq=0 2024-04-22T13:24:41.238 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.237+0000 7f339bfff640 10 client.15858 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.237+0000 7f339bfff640 1 -- 172.21.15.12:0/1920487600 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5557bf547880 con 0x5557bf5e8080 2024-04-22T13:24:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.237+0000 7f339bfff640 10 client.15858 handle_command_reply: tid=0 2024-04-22T13:24:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.238+0000 7f33c5836740 1 client.15858 shutdown 2024-04-22T13:24:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.238+0000 7f33c5836740 1 -- 172.21.15.12:0/1920487600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bf5d80e0 msgr2=0x5557bf5d52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.238+0000 7f33c5836740 1 --2- 172.21.15.12:0/1920487600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bf5d80e0 0x5557bf5d52b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f33b4009fd0 tx=0x7f33b4009600 comp rx=0 tx=0).stop 2024-04-22T13:24:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.238+0000 7f33c5836740 1 -- 172.21.15.12:0/1920487600 shutdown_connections 2024-04-22T13:24:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.238+0000 7f33c5836740 1 --2- 172.21.15.12:0/1920487600 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5557bf5e8080 0x5557bf5ea4b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.238+0000 7f33c5836740 1 --2- 172.21.15.12:0/1920487600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bf5d80e0 0x5557bf5d52b0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.238+0000 7f33c5836740 1 --2- 172.21.15.12:0/1920487600 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557bf5cf8d0 0x5557bf5d4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.238+0000 7f339bfff640 0 client.15858 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.238+0000 7f33c5836740 1 --2- 172.21.15.12:0/1920487600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557bf5ceb60 0x5557bf5d79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.238+0000 7f33c5836740 1 -- 172.21.15.12:0/1920487600 >> 172.21.15.12:0/1920487600 conn(0x5557bf595c90 msgr2=0x5557bf5cb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.238+0000 7f33c5836740 1 -- 172.21.15.12:0/1920487600 shutdown_connections 2024-04-22T13:24:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.238+0000 7f33c5836740 1 -- 172.21.15.12:0/1920487600 wait complete. 2024-04-22T13:24:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.238+0000 7f33c5836740 20 client.15858 trim_cache size 0 max 16384 2024-04-22T13:24:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.253+0000 7f33c5836740 1 Processor -- start 2024-04-22T13:24:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.253+0000 7f33c5836740 1 -- start start 2024-04-22T13:24:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.253+0000 7f33c5836740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557bf5ceb60 0x5557bf5e3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.253+0000 7f33c5836740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557bf5cf8d0 0x5557bf5d74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.253+0000 7f33c5836740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bf5d80e0 0x5557bf5d79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.253+0000 7f33c5836740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc1afb70 con 0x5557bf5d80e0 2024-04-22T13:24:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.253+0000 7f33c5836740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc1b16b0 con 0x5557bf5ceb60 2024-04-22T13:24:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.253+0000 7f33c5836740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc1b0240 con 0x5557bf5cf8d0 2024-04-22T13:24:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.253+0000 7f33c11e0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bf5d80e0 0x5557bf5d79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.253+0000 7f33c11e0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bf5d80e0 0x5557bf5d79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44212/0 (socket says 172.21.15.12:44212) 2024-04-22T13:24:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.253+0000 7f33c11e0640 1 -- 172.21.15.12:0/3884056402 learned_addr learned my addr 172.21.15.12:0/3884056402 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.253+0000 7f33bbfff640 1 --2- 172.21.15.12:0/3884056402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557bf5cf8d0 0x5557bf5d74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.253+0000 7f33c09df640 1 --2- 172.21.15.12:0/3884056402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557bf5ceb60 0x5557bf5e3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.253+0000 7f33c11e0640 1 -- 172.21.15.12:0/3884056402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557bf5cf8d0 msgr2=0x5557bf5d74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.253+0000 7f33c11e0640 1 --2- 172.21.15.12:0/3884056402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557bf5cf8d0 0x5557bf5d74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.253+0000 7f33c11e0640 1 -- 172.21.15.12:0/3884056402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557bf5ceb60 msgr2=0x5557bf5e3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.253+0000 7f33c11e0640 1 --2- 172.21.15.12:0/3884056402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557bf5ceb60 0x5557bf5e3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.253+0000 7f33c11e0640 1 -- 172.21.15.12:0/3884056402 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557bf5cde30 con 0x5557bf5d80e0 2024-04-22T13:24:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.253+0000 7f33c09df640 1 --2- 172.21.15.12:0/3884056402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557bf5ceb60 0x5557bf5e3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.254+0000 7f33c11e0640 1 --2- 172.21.15.12:0/3884056402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bf5d80e0 0x5557bf5d79e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f33b4005d70 tx=0x7f33b4005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8cc9b881ec16be server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.254+0000 7f33a67fc640 1 -- 172.21.15.12:0/3884056402 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b4023070 con 0x5557bf5d80e0 2024-04-22T13:24:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.254+0000 7f33a67fc640 1 -- 172.21.15.12:0/3884056402 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33b4012e50 con 0x5557bf5d80e0 2024-04-22T13:24:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.254+0000 7f33a67fc640 1 -- 172.21.15.12:0/3884056402 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b401f6b0 con 0x5557bf5d80e0 2024-04-22T13:24:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.254+0000 7f33c5836740 1 -- 172.21.15.12:0/3884056402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bf5d80e0 msgr2=0x5557bf5d79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.254+0000 7f33c5836740 1 --2- 172.21.15.12:0/3884056402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bf5d80e0 0x5557bf5d79e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f33b4005d70 tx=0x7f33b4005ac0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.254+0000 7f33c5836740 1 -- 172.21.15.12:0/3884056402 shutdown_connections 2024-04-22T13:24:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.254+0000 7f33c5836740 1 --2- 172.21.15.12:0/3884056402 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bf5d80e0 0x5557bf5d79e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.254+0000 7f33c5836740 1 --2- 172.21.15.12:0/3884056402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557bf5cf8d0 0x5557bf5d74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.254+0000 7f33c5836740 1 --2- 172.21.15.12:0/3884056402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557bf5ceb60 0x5557bf5e3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.254+0000 7f33c5836740 1 -- 172.21.15.12:0/3884056402 >> 172.21.15.12:0/3884056402 conn(0x5557bf595c90 msgr2=0x5557bf5c9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.254+0000 7f33c5836740 1 -- 172.21.15.12:0/3884056402 shutdown_connections 2024-04-22T13:24:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.254+0000 7f33c5836740 1 -- 172.21.15.12:0/3884056402 wait complete. 2024-04-22T13:24:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.255+0000 7f33c5836740 1 -- start start 2024-04-22T13:24:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.255+0000 7f33c5836740 1 Processor -- start 2024-04-22T13:24:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.255+0000 7f33c5836740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557bf5ceb60 0x5557bf5d2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.255+0000 7f33c5836740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bf5cf8d0 0x5557bf5d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.255+0000 7f33c5836740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557bf5d80e0 0x5557bf64d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.255+0000 7f33c5836740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc1b0fe0 con 0x5557bf5cf8d0 2024-04-22T13:24:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.255+0000 7f33c5836740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc1b0910 con 0x5557bf5d80e0 2024-04-22T13:24:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.255+0000 7f33c5836740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc1adb50 con 0x5557bf5ceb60 2024-04-22T13:24:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.255+0000 7f33c5836740 10 client.0 _ll_register_callbacks cb 0x7f33b4009980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.255+0000 7f33bbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bf5cf8d0 0x5557bf5d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.255+0000 7f33bbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bf5cf8d0 0x5557bf5d26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44222/0 (socket says 172.21.15.12:44222) 2024-04-22T13:24:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.255+0000 7f33bbfff640 1 -- 172.21.15.12:0/2221507862 learned_addr learned my addr 172.21.15.12:0/2221507862 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.256+0000 7f33c09df640 1 --2- 172.21.15.12:0/2221507862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557bf5ceb60 0x5557bf5d2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.256+0000 7f33bbfff640 1 -- 172.21.15.12:0/2221507862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557bf5ceb60 msgr2=0x5557bf5d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.256+0000 7f33bbfff640 1 --2- 172.21.15.12:0/2221507862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557bf5ceb60 0x5557bf5d2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.256+0000 7f33bbfff640 1 -- 172.21.15.12:0/2221507862 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557bf5d80e0 msgr2=0x5557bf64d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.256+0000 7f33c11e0640 1 --2- 172.21.15.12:0/2221507862 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557bf5d80e0 0x5557bf64d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.256+0000 7f33bbfff640 1 --2- 172.21.15.12:0/2221507862 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557bf5d80e0 0x5557bf64d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.256+0000 7f33bbfff640 1 -- 172.21.15.12:0/2221507862 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557bf64db70 con 0x5557bf5cf8d0 2024-04-22T13:24:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.256+0000 7f33bbfff640 1 --2- 172.21.15.12:0/2221507862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bf5cf8d0 0x5557bf5d26a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x5557bf5cf600 tx=0x7f33a8004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8cc0ce9d59a4463 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.256+0000 7f339bfff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.256+0000 7f339bfff640 1 -- 172.21.15.12:0/2221507862 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a8005220 con 0x5557bf5cf8d0 2024-04-22T13:24:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.256+0000 7f339bfff640 1 -- 172.21.15.12:0/2221507862 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f33a80053c0 con 0x5557bf5cf8d0 2024-04-22T13:24:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.256+0000 7f33c5836740 1 -- 172.21.15.12:0/2221507862 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33b4012e50 con 0x5557bf5cf8d0 2024-04-22T13:24:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.256+0000 7f339bfff640 1 -- 172.21.15.12:0/2221507862 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f33a80055c0 con 0x5557bf5cf8d0 2024-04-22T13:24:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.256+0000 7f33c5836740 10 client.15864 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.256+0000 7f33c5836740 1 -- 172.21.15.12:0/2221507862 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5557bf64edc0 con 0x5557bf5cf8d0 2024-04-22T13:24:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.257+0000 7f339bfff640 1 -- 172.21.15.12:0/2221507862 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33b4012e50 con 0x5557bf5cf8d0 2024-04-22T13:24:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.257+0000 7f339bfff640 1 -- 172.21.15.12:0/2221507862 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f33a8023070 con 0x5557bf5cf8d0 2024-04-22T13:24:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.257+0000 7f33c5836740 10 client.15864 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.257+0000 7f33c5836740 10 client.15864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:24:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.257+0000 7f33c5836740 20 client.15864 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.257+0000 7f33c5836740 1 --2- 172.21.15.12:0/2221507862 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5557bf64f1a0 0x5557bf651590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.257+0000 7f33c5836740 4 client.15864 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-22T13:24:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.257+0000 7f33c5836740 1 -- 172.21.15.12:0/2221507862 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5557bf5d5640 con 0x5557bf64f1a0 2024-04-22T13:24:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.257+0000 7f33c09df640 1 --2- 172.21.15.12:0/2221507862 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5557bf64f1a0 0x5557bf651590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.257+0000 7f339bfff640 10 client.15864 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.257+0000 7f33c09df640 1 --2- 172.21.15.12:0/2221507862 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5557bf64f1a0 0x5557bf651590 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dbb5d1db8d0074b server_cookie=58afe44724a9f4bd in_seq=0 out_seq=0 2024-04-22T13:24:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.264+0000 7f339bfff640 1 -- 172.21.15.12:0/2221507862 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5557bf5d5640 con 0x5557bf64f1a0 2024-04-22T13:24:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.264+0000 7f339bfff640 10 client.15864 handle_command_reply: tid=0 2024-04-22T13:24:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.264+0000 7f33c5836740 1 client.15864 shutdown 2024-04-22T13:24:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.264+0000 7f33c5836740 1 -- 172.21.15.12:0/2221507862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bf5cf8d0 msgr2=0x5557bf5d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.264+0000 7f33c5836740 1 --2- 172.21.15.12:0/2221507862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bf5cf8d0 0x5557bf5d26a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x5557bf5cf600 tx=0x7f33a8004230 comp rx=0 tx=0).stop 2024-04-22T13:24:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.264+0000 7f33c5836740 1 -- 172.21.15.12:0/2221507862 shutdown_connections 2024-04-22T13:24:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.264+0000 7f33c5836740 1 --2- 172.21.15.12:0/2221507862 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5557bf64f1a0 0x5557bf651590 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.264+0000 7f33c5836740 1 --2- 172.21.15.12:0/2221507862 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5557bf5d80e0 0x5557bf64d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.264+0000 7f33c5836740 1 --2- 172.21.15.12:0/2221507862 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5557bf5cf8d0 0x5557bf5d26a0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.264+0000 7f339bfff640 0 client.15864 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.264+0000 7f33c5836740 1 --2- 172.21.15.12:0/2221507862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5557bf5ceb60 0x5557bf5d2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.264+0000 7f33c5836740 1 -- 172.21.15.12:0/2221507862 >> 172.21.15.12:0/2221507862 conn(0x5557bf595c90 msgr2=0x5557bf5c8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.265+0000 7f33c5836740 1 -- 172.21.15.12:0/2221507862 shutdown_connections 2024-04-22T13:24:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.265+0000 7f33c5836740 1 -- 172.21.15.12:0/2221507862 wait complete. 2024-04-22T13:24:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.265+0000 7f33c5836740 20 client.15864 trim_cache size 0 max 16384 2024-04-22T13:24:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-04-22T13:24:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-04-22T13:24:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-04-22T13:24:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-04-22T13:24:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.265+0000 7f33c2c43640 1 -- 172.21.15.12:0/3838408109 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f33ac0a3ee0 msgr2=0x7f33ac0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.265+0000 7f33c2c43640 1 --2- 172.21.15.12:0/3838408109 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f33ac0a3ee0 0x7f33ac0a63a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f33bc06b860 tx=0x7f33b4004a40 comp rx=0 tx=0).stop 2024-04-22T13:24:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.265+0000 7f33c2c43640 1 -- 172.21.15.12:0/3838408109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33bc103200 msgr2=0x7f33bc1a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.265+0000 7f33c2c43640 1 --2- 172.21.15.12:0/3838408109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33bc103200 0x7f33bc1a0680 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f33b0005940 tx=0x7f33b002d890 comp rx=0 tx=0).stop 2024-04-22T13:24:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.265+0000 7f33c2c43640 1 -- 172.21.15.12:0/3838408109 shutdown_connections 2024-04-22T13:24:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.265+0000 7f33c2c43640 1 --2- 172.21.15.12:0/3838408109 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f33ac0a3ee0 0x7f33ac0a63a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.265+0000 7f33c2c43640 1 --2- 172.21.15.12:0/3838408109 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f33bc10c5d0 0x7f33bc19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.265+0000 7f33c2c43640 1 --2- 172.21.15.12:0/3838408109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33bc103b20 0x7f33bc1a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.265+0000 7f33c2c43640 1 --2- 172.21.15.12:0/3838408109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f33bc103200 0x7f33bc1a0680 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.265+0000 7f33c2c43640 1 -- 172.21.15.12:0/3838408109 >> 172.21.15.12:0/3838408109 conn(0x7f33bc0771d0 msgr2=0x7f33bc100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.266+0000 7f33c2c43640 1 -- 172.21.15.12:0/3838408109 shutdown_connections 2024-04-22T13:24:41.264 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:41.266+0000 7f33c2c43640 1 -- 172.21.15.12:0/3838408109 wait complete. 2024-04-22T13:24:42.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:41 smithi012 ceph-mon[28265]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-22T13:24:42.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:41 smithi134 ceph-mon[31338]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-22T13:24:42.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:41 smithi111 ceph-mon[31361]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-22T13:24:43.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:24:43.634 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:24:43.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.755+0000 7fd208567640 1 -- 172.21.15.12:0/234697649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd200108060 msgr2=0x7fd200108440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:43.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.755+0000 7fd208567640 1 --2- 172.21.15.12:0/234697649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd200108060 0x7fd200108440 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009930 tx=0x7fd1fc02bc20 comp rx=0 tx=0).stop 2024-04-22T13:24:43.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.755+0000 7fd208567640 1 -- 172.21.15.12:0/234697649 shutdown_connections 2024-04-22T13:24:43.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.755+0000 7fd208567640 1 --2- 172.21.15.12:0/234697649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd20006af70 0x7fd20010ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.755+0000 7fd208567640 1 --2- 172.21.15.12:0/234697649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd20006a5b0 0x7fd20006aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.755+0000 7fd208567640 1 --2- 172.21.15.12:0/234697649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd200108060 0x7fd200108440 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.755+0000 7fd208567640 1 -- 172.21.15.12:0/234697649 >> 172.21.15.12:0/234697649 conn(0x7fd200077c40 msgr2=0x7fd200078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:43.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.755+0000 7fd208567640 1 -- 172.21.15.12:0/234697649 shutdown_connections 2024-04-22T13:24:43.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.755+0000 7fd208567640 1 -- 172.21.15.12:0/234697649 wait complete. 2024-04-22T13:24:43.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.756+0000 7fd208567640 1 Processor -- start 2024-04-22T13:24:43.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.756+0000 7fd208567640 1 -- start start 2024-04-22T13:24:43.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.756+0000 7fd208567640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd20006a5b0 0x7fd200101170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:43.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.756+0000 7fd208567640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd20006af70 0x7fd2001016b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:43.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.756+0000 7fd208567640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd200108060 0x7fd200101bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:43.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.756+0000 7fd208567640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd2001121a0 con 0x7fd20006a5b0 2024-04-22T13:24:43.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.756+0000 7fd208567640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd200112020 con 0x7fd200108060 2024-04-22T13:24:43.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.756+0000 7fd208567640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd200112320 con 0x7fd20006af70 2024-04-22T13:24:43.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.756+0000 7fd206303640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd20006a5b0 0x7fd200101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:43.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.756+0000 7fd206303640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd20006a5b0 0x7fd200101170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50056/0 (socket says 172.21.15.12:50056) 2024-04-22T13:24:43.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.756+0000 7fd206303640 1 -- 172.21.15.12:0/4156361186 learned_addr learned my addr 172.21.15.12:0/4156361186 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:43.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.756+0000 7fd206303640 1 -- 172.21.15.12:0/4156361186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd20006af70 msgr2=0x7fd2001016b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:43.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.756+0000 7fd206303640 1 --2- 172.21.15.12:0/4156361186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd20006af70 0x7fd2001016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.756+0000 7fd206303640 1 -- 172.21.15.12:0/4156361186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd200108060 msgr2=0x7fd200101bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:43.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.756+0000 7fd206303640 1 --2- 172.21.15.12:0/4156361186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd200108060 0x7fd200101bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.756+0000 7fd206303640 1 -- 172.21.15.12:0/4156361186 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2001a6b20 con 0x7fd20006a5b0 2024-04-22T13:24:43.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.757+0000 7fd206303640 1 --2- 172.21.15.12:0/4156361186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd20006a5b0 0x7fd200101170 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc005940 tx=0x7fd1fc02ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:43.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.757+0000 7fd1f77fe640 1 -- 172.21.15.12:0/4156361186 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc043020 con 0x7fd20006a5b0 2024-04-22T13:24:43.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.757+0000 7fd208567640 1 -- 172.21.15.12:0/4156361186 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2001a6db0 con 0x7fd20006a5b0 2024-04-22T13:24:43.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.757+0000 7fd1f77fe640 1 -- 172.21.15.12:0/4156361186 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1fc02cc70 con 0x7fd20006a5b0 2024-04-22T13:24:43.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.757+0000 7fd1f77fe640 1 -- 172.21.15.12:0/4156361186 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc003c70 con 0x7fd20006a5b0 2024-04-22T13:24:43.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.757+0000 7fd208567640 1 -- 172.21.15.12:0/4156361186 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2001a7270 con 0x7fd20006a5b0 2024-04-22T13:24:43.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.759+0000 7fd1f77fe640 1 -- 172.21.15.12:0/4156361186 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fd1fc046020 con 0x7fd20006a5b0 2024-04-22T13:24:43.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.760+0000 7fd20b237740 1 Processor -- start 2024-04-22T13:24:43.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.760+0000 7fd20b237740 1 -- start start 2024-04-22T13:24:43.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.760+0000 7fd20b237740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e7b0fc1580 0x55e7b0fc1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:43.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.760+0000 7fd20b237740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e7b0fc1ea0 0x55e7b0fca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:43.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.760+0000 7fd20b237740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b0fcaa80 0x55e7b0fcce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:43.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.760+0000 7fd20b237740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e7b0f261e0 con 0x55e7b0fc1580 2024-04-22T13:24:43.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.760+0000 7fd20b237740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e7b0ec3390 con 0x55e7b0fc1ea0 2024-04-22T13:24:43.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.760+0000 7fd20b237740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e7b0f221a0 con 0x55e7b0fcaa80 2024-04-22T13:24:43.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.760+0000 7fd1f77fe640 1 --2- 172.21.15.12:0/4156361186 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd1ec0a3e40 0x7fd1ec0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:43.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.760+0000 7fd206b04640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e7b0fc1580 0x55e7b0fc1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:43.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.760+0000 7fd206b04640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e7b0fc1580 0x55e7b0fc1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50064/0 (socket says 172.21.15.12:50064) 2024-04-22T13:24:43.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.760+0000 7fd206b04640 1 -- 172.21.15.12:0/3083173467 learned_addr learned my addr 172.21.15.12:0/3083173467 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:43.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.760+0000 7fd206b04640 1 --2- 172.21.15.12:0/4156361186 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd1ec0a3e40 0x7fd1ec0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:43.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.761+0000 7fd206303640 1 --2- 172.21.15.12:0/3083173467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e7b0fc1ea0 0x55e7b0fca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:43.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.761+0000 7fd205b02640 1 --2- 172.21.15.12:0/3083173467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b0fcaa80 0x55e7b0fcce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:43.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.761+0000 7fd1f77fe640 1 -- 172.21.15.12:0/4156361186 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fd1fc0f2c60 con 0x7fd20006a5b0 2024-04-22T13:24:43.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.761+0000 7fd206b04640 1 -- 172.21.15.12:0/3083173467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b0fcaa80 msgr2=0x55e7b0fcce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:43.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.761+0000 7fd206b04640 1 --2- 172.21.15.12:0/3083173467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b0fcaa80 0x55e7b0fcce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.761+0000 7fd206b04640 1 -- 172.21.15.12:0/3083173467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e7b0fc1ea0 msgr2=0x55e7b0fca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:43.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.761+0000 7fd206b04640 1 --2- 172.21.15.12:0/3083173467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e7b0fc1ea0 0x55e7b0fca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.761+0000 7fd206b04640 1 -- 172.21.15.12:0/3083173467 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7b0d027d0 con 0x55e7b0fc1580 2024-04-22T13:24:43.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.761+0000 7fd206b04640 1 --2- 172.21.15.12:0/3083173467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e7b0fc1580 0x55e7b0fc1960 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8013c70 tx=0x7fd1f8011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2301e4b316945bf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:43.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.761+0000 7fd206b04640 1 --2- 172.21.15.12:0/4156361186 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd1ec0a3e40 0x7fd1ec0a6300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8011c50 tx=0x7fd1f801a000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:43.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.761+0000 7fd1f4bfe640 1 -- 172.21.15.12:0/3083173467 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8020070 con 0x55e7b0fc1580 2024-04-22T13:24:43.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.761+0000 7fd1f4bfe640 1 -- 172.21.15.12:0/3083173467 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1f8009a70 con 0x55e7b0fc1580 2024-04-22T13:24:43.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.761+0000 7fd1f4bfe640 1 -- 172.21.15.12:0/3083173467 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8003bd0 con 0x55e7b0fc1580 2024-04-22T13:24:43.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.762+0000 7fd20b237740 1 -- 172.21.15.12:0/3083173467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e7b0fc1580 msgr2=0x55e7b0fc1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:43.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.762+0000 7fd20b237740 1 --2- 172.21.15.12:0/3083173467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e7b0fc1580 0x55e7b0fc1960 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8013c70 tx=0x7fd1f8011750 comp rx=0 tx=0).stop 2024-04-22T13:24:43.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.762+0000 7fd20b237740 1 -- 172.21.15.12:0/3083173467 shutdown_connections 2024-04-22T13:24:43.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.762+0000 7fd20b237740 1 --2- 172.21.15.12:0/3083173467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b0fcaa80 0x55e7b0fcce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.762+0000 7fd20b237740 1 --2- 172.21.15.12:0/3083173467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e7b0fc1ea0 0x55e7b0fca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.762+0000 7fd20b237740 1 --2- 172.21.15.12:0/3083173467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e7b0fc1580 0x55e7b0fc1960 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.762+0000 7fd20b237740 1 -- 172.21.15.12:0/3083173467 >> 172.21.15.12:0/3083173467 conn(0x55e7b0fbbcb0 msgr2=0x55e7b0fbc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:43.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.762+0000 7fd20b237740 1 -- 172.21.15.12:0/3083173467 shutdown_connections 2024-04-22T13:24:43.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.762+0000 7fd20b237740 1 -- 172.21.15.12:0/3083173467 wait complete. 2024-04-22T13:24:43.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.763+0000 7fd20b237740 1 -- start start 2024-04-22T13:24:43.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.763+0000 7fd20b237740 1 Processor -- start 2024-04-22T13:24:43.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.763+0000 7fd20b237740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e7b0fc1580 0x55e7b0fc7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:43.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.763+0000 7fd20b237740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b0fc1ea0 0x55e7b0fc8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:43.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.763+0000 7fd20b237740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e7b0fcaa80 0x55e7b0fc85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:43.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.763+0000 7fd20b237740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e7b0fcd810 con 0x55e7b0fc1580 2024-04-22T13:24:43.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.763+0000 7fd20b237740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e7b0edf2f0 con 0x55e7b0fcaa80 2024-04-22T13:24:43.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.763+0000 7fd20b237740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e7b0fcd990 con 0x55e7b0fc1ea0 2024-04-22T13:24:43.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.763+0000 7fd20b237740 10 client.0 _ll_register_callbacks cb 0x55e7b0eec580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:43.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.763+0000 7fd206303640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e7b0fc1580 0x55e7b0fc7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:43.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.763+0000 7fd206303640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e7b0fc1580 0x55e7b0fc7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50074/0 (socket says 172.21.15.12:50074) 2024-04-22T13:24:43.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.763+0000 7fd206303640 1 -- 172.21.15.12:0/3540978665 learned_addr learned my addr 172.21.15.12:0/3540978665 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:43.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.763+0000 7fd205b02640 1 --2- 172.21.15.12:0/3540978665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b0fc1ea0 0x55e7b0fc8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:43.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.763+0000 7fd206b04640 1 --2- 172.21.15.12:0/3540978665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e7b0fcaa80 0x55e7b0fc85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:43.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.763+0000 7fd206303640 1 -- 172.21.15.12:0/3540978665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b0fc1ea0 msgr2=0x55e7b0fc8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:43.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.763+0000 7fd206303640 1 --2- 172.21.15.12:0/3540978665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b0fc1ea0 0x55e7b0fc8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.764+0000 7fd206303640 1 -- 172.21.15.12:0/3540978665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e7b0fcaa80 msgr2=0x55e7b0fc85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:43.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.764+0000 7fd206303640 1 --2- 172.21.15.12:0/3540978665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e7b0fcaa80 0x55e7b0fc85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.764+0000 7fd206303640 1 -- 172.21.15.12:0/3540978665 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7b0c383f0 con 0x55e7b0fc1580 2024-04-22T13:24:43.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.764+0000 7fd206303640 1 --2- 172.21.15.12:0/3540978665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e7b0fc1580 0x55e7b0fc7b20 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc005ec0 tx=0x7fd1fc02bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c430782f8389382 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:43.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.764+0000 7fd1e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:43.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.766+0000 7fd1e9ffb640 1 -- 172.21.15.12:0/3540978665 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc0035f0 con 0x55e7b0fc1580 2024-04-22T13:24:43.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.766+0000 7fd1e9ffb640 1 -- 172.21.15.12:0/3540978665 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fd1fc0b92c0 con 0x55e7b0fc1580 2024-04-22T13:24:43.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.766+0000 7fd20b237740 1 -- 172.21.15.12:0/3540978665 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e7b0f33c00 con 0x55e7b0fc1580 2024-04-22T13:24:43.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.766+0000 7fd1e9ffb640 1 -- 172.21.15.12:0/3540978665 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd1fc0bf1b0 con 0x55e7b0fc1580 2024-04-22T13:24:43.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.766+0000 7fd20b237740 10 client.15876 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:43.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.766+0000 7fd20b237740 1 -- 172.21.15.12:0/3540978665 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e7b0bf80f0 con 0x55e7b0fc1580 2024-04-22T13:24:43.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.766+0000 7fd1e9ffb640 1 -- 172.21.15.12:0/3540978665 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e7b0f33c00 con 0x55e7b0fc1580 2024-04-22T13:24:43.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.766+0000 7fd1e9ffb640 1 -- 172.21.15.12:0/3540978665 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd1fc03ac80 con 0x55e7b0fc1580 2024-04-22T13:24:43.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.766+0000 7fd20b237740 10 client.15876 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:43.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.766+0000 7fd20b237740 10 client.15876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:24:43.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.767+0000 7fd20b237740 20 client.15876 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:43.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.767+0000 7fd20b237740 1 --2- 172.21.15.12:0/3540978665 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e7b0fdaa10 0x55e7b0fdce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:43.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.767+0000 7fd20b237740 4 client.15876 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:24:43.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.767+0000 7fd20b237740 1 -- 172.21.15.12:0/3540978665 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7b0f2da70 con 0x55e7b0fdaa10 2024-04-22T13:24:43.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.767+0000 7fd205b02640 1 --2- 172.21.15.12:0/3540978665 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e7b0fdaa10 0x55e7b0fdce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:43.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.767+0000 7fd1e9ffb640 10 client.15876 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:43.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.767+0000 7fd205b02640 1 --2- 172.21.15.12:0/3540978665 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e7b0fdaa10 0x55e7b0fdce70 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea1f9044d3f68396 server_cookie=5e0792bd0e48ca54 in_seq=0 out_seq=0 2024-04-22T13:24:43.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.768+0000 7fd1e9ffb640 1 -- 172.21.15.12:0/3540978665 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e7b0f2da70 con 0x55e7b0fdaa10 2024-04-22T13:24:43.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.768+0000 7fd1e9ffb640 10 client.15876 handle_command_reply: tid=0 2024-04-22T13:24:43.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.768+0000 7fd20b237740 1 client.15876 shutdown 2024-04-22T13:24:43.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.768+0000 7fd20b237740 1 -- 172.21.15.12:0/3540978665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e7b0fc1580 msgr2=0x55e7b0fc7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:43.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.768+0000 7fd20b237740 1 --2- 172.21.15.12:0/3540978665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e7b0fc1580 0x55e7b0fc7b20 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc005ec0 tx=0x7fd1fc02bc20 comp rx=0 tx=0).stop 2024-04-22T13:24:43.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.768+0000 7fd20b237740 1 -- 172.21.15.12:0/3540978665 shutdown_connections 2024-04-22T13:24:43.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.768+0000 7fd20b237740 1 --2- 172.21.15.12:0/3540978665 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e7b0fdaa10 0x55e7b0fdce70 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.768+0000 7fd20b237740 1 --2- 172.21.15.12:0/3540978665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e7b0fcaa80 0x55e7b0fc85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.768+0000 7fd20b237740 1 --2- 172.21.15.12:0/3540978665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b0fc1ea0 0x55e7b0fc8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.768+0000 7fd20b237740 1 --2- 172.21.15.12:0/3540978665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e7b0fc1580 0x55e7b0fc7b20 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.768+0000 7fd20b237740 1 -- 172.21.15.12:0/3540978665 >> 172.21.15.12:0/3540978665 conn(0x55e7b0fbbcb0 msgr2=0x55e7b0fbda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:43.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.768+0000 7fd1e9ffb640 0 client.15876 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:43.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.768+0000 7fd20b237740 1 -- 172.21.15.12:0/3540978665 shutdown_connections 2024-04-22T13:24:43.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.768+0000 7fd20b237740 1 -- 172.21.15.12:0/3540978665 wait complete. 2024-04-22T13:24:43.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.768+0000 7fd20b237740 20 client.15876 trim_cache size 0 max 16384 2024-04-22T13:24:43.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.786+0000 7fd20b237740 1 Processor -- start 2024-04-22T13:24:43.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.786+0000 7fd20b237740 1 -- start start 2024-04-22T13:24:43.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.786+0000 7fd20b237740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e7b0fc1580 0x55e7b0fca250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:43.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.786+0000 7fd20b237740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b0fc1ea0 0x55e7b0fd66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:43.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.786+0000 7fd20b237740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e7b0fcaa80 0x55e7b0fc6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:43.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.786+0000 7fd20b237740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd2001afb20 con 0x55e7b0fc1580 2024-04-22T13:24:43.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.786+0000 7fd20b237740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd2001b1660 con 0x55e7b0fcaa80 2024-04-22T13:24:43.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.786+0000 7fd20b237740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd2001b01f0 con 0x55e7b0fc1ea0 2024-04-22T13:24:43.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.786+0000 7fd206303640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e7b0fc1580 0x55e7b0fca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:43.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.786+0000 7fd206303640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e7b0fc1580 0x55e7b0fca250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50082/0 (socket says 172.21.15.12:50082) 2024-04-22T13:24:43.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.786+0000 7fd206303640 1 -- 172.21.15.12:0/1490942434 learned_addr learned my addr 172.21.15.12:0/1490942434 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:43.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.786+0000 7fd206303640 1 -- 172.21.15.12:0/1490942434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b0fc1ea0 msgr2=0x55e7b0fd66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:43.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.787+0000 7fd206303640 1 --2- 172.21.15.12:0/1490942434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b0fc1ea0 0x55e7b0fd66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.787+0000 7fd206303640 1 -- 172.21.15.12:0/1490942434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e7b0fcaa80 msgr2=0x55e7b0fc6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:43.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.787+0000 7fd206303640 1 --2- 172.21.15.12:0/1490942434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e7b0fcaa80 0x55e7b0fc6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.787+0000 7fd206303640 1 -- 172.21.15.12:0/1490942434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7b0fc1cd0 con 0x55e7b0fc1580 2024-04-22T13:24:43.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.787+0000 7fd206303640 1 --2- 172.21.15.12:0/1490942434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e7b0fc1580 0x55e7b0fca250 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc000c00 tx=0x7fd1fc0c7290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0906cb8d1a31af server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:43.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.787+0000 7fd1ebfff640 1 -- 172.21.15.12:0/1490942434 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc087070 con 0x55e7b0fc1580 2024-04-22T13:24:43.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.787+0000 7fd1ebfff640 1 -- 172.21.15.12:0/1490942434 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1fc0d8020 con 0x55e7b0fc1580 2024-04-22T13:24:43.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.787+0000 7fd1ebfff640 1 -- 172.21.15.12:0/1490942434 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc003e10 con 0x55e7b0fc1580 2024-04-22T13:24:43.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.787+0000 7fd20b237740 1 -- 172.21.15.12:0/1490942434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e7b0fc1580 msgr2=0x55e7b0fca250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:43.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.787+0000 7fd20b237740 1 --2- 172.21.15.12:0/1490942434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e7b0fc1580 0x55e7b0fca250 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc000c00 tx=0x7fd1fc0c7290 comp rx=0 tx=0).stop 2024-04-22T13:24:43.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.788+0000 7fd20b237740 1 -- 172.21.15.12:0/1490942434 shutdown_connections 2024-04-22T13:24:43.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.788+0000 7fd20b237740 1 --2- 172.21.15.12:0/1490942434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e7b0fcaa80 0x55e7b0fc6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.788+0000 7fd20b237740 1 --2- 172.21.15.12:0/1490942434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b0fc1ea0 0x55e7b0fd66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.788+0000 7fd20b237740 1 --2- 172.21.15.12:0/1490942434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e7b0fc1580 0x55e7b0fca250 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.788+0000 7fd20b237740 1 -- 172.21.15.12:0/1490942434 >> 172.21.15.12:0/1490942434 conn(0x55e7b0fbbcb0 msgr2=0x55e7b0fbad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:43.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.788+0000 7fd20b237740 1 -- 172.21.15.12:0/1490942434 shutdown_connections 2024-04-22T13:24:43.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.788+0000 7fd20b237740 1 -- 172.21.15.12:0/1490942434 wait complete. 2024-04-22T13:24:43.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.788+0000 7fd20b237740 1 -- start start 2024-04-22T13:24:43.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.788+0000 7fd20b237740 1 Processor -- start 2024-04-22T13:24:43.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.788+0000 7fd20b237740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e7b0fc1580 0x55e7b0fcec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:43.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.788+0000 7fd20b237740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e7b0fc1ea0 0x55e7b0fcf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:43.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.788+0000 7fd20b237740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b0fcaa80 0x55e7b103ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:43.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.788+0000 7fd20b237740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd2001b0f90 con 0x55e7b0fc1580 2024-04-22T13:24:43.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.788+0000 7fd20b237740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd2001b08c0 con 0x55e7b0fc1ea0 2024-04-22T13:24:43.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.788+0000 7fd20b237740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd2001adb00 con 0x55e7b0fcaa80 2024-04-22T13:24:43.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.788+0000 7fd20b237740 10 client.0 _ll_register_callbacks cb 0x7fd1fc0057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:43.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.788+0000 7fd206303640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e7b0fc1580 0x55e7b0fcec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:43.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.788+0000 7fd206303640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e7b0fc1580 0x55e7b0fcec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50088/0 (socket says 172.21.15.12:50088) 2024-04-22T13:24:43.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.788+0000 7fd206303640 1 -- 172.21.15.12:0/1515278257 learned_addr learned my addr 172.21.15.12:0/1515278257 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:43.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.789+0000 7fd205b02640 1 --2- 172.21.15.12:0/1515278257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e7b0fc1ea0 0x55e7b0fcf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:43.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.789+0000 7fd206303640 1 -- 172.21.15.12:0/1515278257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b0fcaa80 msgr2=0x55e7b103ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:43.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.789+0000 7fd206303640 1 --2- 172.21.15.12:0/1515278257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b0fcaa80 0x55e7b103ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.789+0000 7fd206303640 1 -- 172.21.15.12:0/1515278257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e7b0fc1ea0 msgr2=0x55e7b0fcf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:43.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.789+0000 7fd206303640 1 --2- 172.21.15.12:0/1515278257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e7b0fc1ea0 0x55e7b0fcf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.789+0000 7fd206303640 1 -- 172.21.15.12:0/1515278257 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7b10404f0 con 0x55e7b0fc1580 2024-04-22T13:24:43.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.789+0000 7fd205b02640 1 --2- 172.21.15.12:0/1515278257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e7b0fc1ea0 0x55e7b0fcf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:43.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.789+0000 7fd206303640 1 --2- 172.21.15.12:0/1515278257 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e7b0fc1580 0x55e7b0fcec50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc006010 tx=0x7fd1fc0bae10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1644ec39b5098c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:43.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.789+0000 7fd1e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:43.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.789+0000 7fd1e9ffb640 1 -- 172.21.15.12:0/1515278257 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc087020 con 0x55e7b0fc1580 2024-04-22T13:24:43.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.789+0000 7fd1e9ffb640 1 -- 172.21.15.12:0/1515278257 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fd1fc0be100 con 0x55e7b0fc1580 2024-04-22T13:24:43.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.789+0000 7fd20b237740 1 -- 172.21.15.12:0/1515278257 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1fc0d8020 con 0x55e7b0fc1580 2024-04-22T13:24:43.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.789+0000 7fd1e9ffb640 1 -- 172.21.15.12:0/1515278257 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd1fc0beab0 con 0x55e7b0fc1580 2024-04-22T13:24:43.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.789+0000 7fd20b237740 10 client.15882 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:43.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.789+0000 7fd20b237740 1 -- 172.21.15.12:0/1515278257 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e7b10404f0 con 0x55e7b0fc1580 2024-04-22T13:24:43.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.789+0000 7fd1e9ffb640 1 -- 172.21.15.12:0/1515278257 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1fc0d8020 con 0x55e7b0fc1580 2024-04-22T13:24:43.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.789+0000 7fd1e9ffb640 1 -- 172.21.15.12:0/1515278257 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd1fc0892a0 con 0x55e7b0fc1580 2024-04-22T13:24:43.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.790+0000 7fd20b237740 10 client.15882 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:43.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.790+0000 7fd20b237740 10 client.15882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:24:43.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.790+0000 7fd20b237740 20 client.15882 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:43.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.790+0000 7fd20b237740 1 --2- 172.21.15.12:0/1515278257 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e7b1041980 0x55e7b1043d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:43.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.790+0000 7fd20b237740 4 client.15882 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:24:43.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.790+0000 7fd20b237740 1 -- 172.21.15.12:0/1515278257 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e7b0ea9590 con 0x55e7b1041980 2024-04-22T13:24:43.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.790+0000 7fd205b02640 1 --2- 172.21.15.12:0/1515278257 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e7b1041980 0x55e7b1043d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:43.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.790+0000 7fd205b02640 1 --2- 172.21.15.12:0/1515278257 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e7b1041980 0x55e7b1043d70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff80a832b1468ff6 server_cookie=c8f52a2a8b754e68 in_seq=0 out_seq=0 2024-04-22T13:24:43.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.790+0000 7fd1e9ffb640 10 client.15882 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:43.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.791+0000 7fd1e9ffb640 1 -- 172.21.15.12:0/1515278257 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e7b0ea9590 con 0x55e7b1041980 2024-04-22T13:24:43.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.791+0000 7fd1e9ffb640 10 client.15882 handle_command_reply: tid=0 2024-04-22T13:24:43.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.791+0000 7fd20b237740 1 client.15882 shutdown 2024-04-22T13:24:43.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.791+0000 7fd20b237740 1 -- 172.21.15.12:0/1515278257 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e7b0fc1580 msgr2=0x55e7b0fcec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:43.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.791+0000 7fd20b237740 1 --2- 172.21.15.12:0/1515278257 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e7b0fc1580 0x55e7b0fcec50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc006010 tx=0x7fd1fc0bae10 comp rx=0 tx=0).stop 2024-04-22T13:24:43.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.791+0000 7fd20b237740 1 -- 172.21.15.12:0/1515278257 shutdown_connections 2024-04-22T13:24:43.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.791+0000 7fd20b237740 1 --2- 172.21.15.12:0/1515278257 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e7b1041980 0x55e7b1043d70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.791+0000 7fd20b237740 1 --2- 172.21.15.12:0/1515278257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e7b0fcaa80 0x55e7b103ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.791+0000 7fd20b237740 1 --2- 172.21.15.12:0/1515278257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e7b0fc1ea0 0x55e7b0fcf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.791+0000 7fd1e9ffb640 0 client.15882 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:43.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.791+0000 7fd20b237740 1 --2- 172.21.15.12:0/1515278257 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e7b0fc1580 0x55e7b0fcec50 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.791+0000 7fd20b237740 1 -- 172.21.15.12:0/1515278257 >> 172.21.15.12:0/1515278257 conn(0x55e7b0fbbcb0 msgr2=0x55e7b0fc7ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:43.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.791+0000 7fd20b237740 1 -- 172.21.15.12:0/1515278257 shutdown_connections 2024-04-22T13:24:43.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.791+0000 7fd20b237740 1 -- 172.21.15.12:0/1515278257 wait complete. 2024-04-22T13:24:43.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.791+0000 7fd20b237740 20 client.15882 trim_cache size 0 max 16384 2024-04-22T13:24:43.795 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:24:43.795 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-04-22T13:24:43.795 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-04-22T13:24:43.795 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:24:43.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.791+0000 7fd208567640 1 -- 172.21.15.12:0/4156361186 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd1ec0a3e40 msgr2=0x7fd1ec0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:43.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.791+0000 7fd208567640 1 --2- 172.21.15.12:0/4156361186 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd1ec0a3e40 0x7fd1ec0a6300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8011c50 tx=0x7fd1f801a000 comp rx=0 tx=0).stop 2024-04-22T13:24:43.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.791+0000 7fd208567640 1 -- 172.21.15.12:0/4156361186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd20006a5b0 msgr2=0x7fd200101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:43.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.791+0000 7fd208567640 1 --2- 172.21.15.12:0/4156361186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd20006a5b0 0x7fd200101170 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc005940 tx=0x7fd1fc02ca80 comp rx=0 tx=0).stop 2024-04-22T13:24:43.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.791+0000 7fd208567640 1 -- 172.21.15.12:0/4156361186 shutdown_connections 2024-04-22T13:24:43.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.791+0000 7fd208567640 1 --2- 172.21.15.12:0/4156361186 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd1ec0a3e40 0x7fd1ec0a6300 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.791+0000 7fd208567640 1 --2- 172.21.15.12:0/4156361186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd200108060 0x7fd200101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.791+0000 7fd208567640 1 --2- 172.21.15.12:0/4156361186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd20006af70 0x7fd2001016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.791+0000 7fd208567640 1 --2- 172.21.15.12:0/4156361186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd20006a5b0 0x7fd200101170 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.791+0000 7fd208567640 1 -- 172.21.15.12:0/4156361186 >> 172.21.15.12:0/4156361186 conn(0x7fd200077c40 msgr2=0x7fd2000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:43.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.791+0000 7fd208567640 1 -- 172.21.15.12:0/4156361186 shutdown_connections 2024-04-22T13:24:43.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.792+0000 7fd208567640 1 -- 172.21.15.12:0/4156361186 wait complete. 2024-04-22T13:24:43.798 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:24:43.799 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:24:43.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.905+0000 7ff036248640 1 -- 172.21.15.12:0/3084414912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff030101a60 msgr2=0x7ff030105aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:43.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.905+0000 7ff036248640 1 --2- 172.21.15.12:0/3084414912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff030101a60 0x7ff030105aa0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff024009900 tx=0x7ff02402bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:43.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.905+0000 7ff036248640 1 -- 172.21.15.12:0/3084414912 shutdown_connections 2024-04-22T13:24:43.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.905+0000 7ff036248640 1 --2- 172.21.15.12:0/3084414912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff030106180 0x7ff03010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.905+0000 7ff036248640 1 --2- 172.21.15.12:0/3084414912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff030101a60 0x7ff030105aa0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.905+0000 7ff036248640 1 --2- 172.21.15.12:0/3084414912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0301010b0 0x7ff030101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.905+0000 7ff036248640 1 -- 172.21.15.12:0/3084414912 >> 172.21.15.12:0/3084414912 conn(0x7ff0300771d0 msgr2=0x7ff0300775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:43.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.905+0000 7ff036248640 1 -- 172.21.15.12:0/3084414912 shutdown_connections 2024-04-22T13:24:43.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.905+0000 7ff036248640 1 -- 172.21.15.12:0/3084414912 wait complete. 2024-04-22T13:24:43.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.905+0000 7ff036248640 1 Processor -- start 2024-04-22T13:24:43.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.906+0000 7ff036248640 1 -- start start 2024-04-22T13:24:43.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.906+0000 7ff036248640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0301010b0 0x7ff030198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:43.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.906+0000 7ff036248640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff030106180 0x7ff030198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:43.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.906+0000 7ff036248640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff03019cfc0 0x7ff03019d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:43.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.906+0000 7ff036248640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff030112200 con 0x7ff030106180 2024-04-22T13:24:43.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.906+0000 7ff036248640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff030112080 con 0x7ff03019cfc0 2024-04-22T13:24:43.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.906+0000 7ff036248640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff030112380 con 0x7ff0301010b0 2024-04-22T13:24:43.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.906+0000 7ff02effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff030106180 0x7ff030198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:43.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.906+0000 7ff02effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff030106180 0x7ff030198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50112/0 (socket says 172.21.15.12:50112) 2024-04-22T13:24:43.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.906+0000 7ff02effd640 1 -- 172.21.15.12:0/2358171320 learned_addr learned my addr 172.21.15.12:0/2358171320 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:43.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.906+0000 7ff02f7fe640 1 --2- 172.21.15.12:0/2358171320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0301010b0 0x7ff030198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:43.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.906+0000 7ff02ffff640 1 --2- 172.21.15.12:0/2358171320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff03019cfc0 0x7ff03019d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:43.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.906+0000 7ff02effd640 1 -- 172.21.15.12:0/2358171320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0301010b0 msgr2=0x7ff030198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:43.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.906+0000 7ff02effd640 1 --2- 172.21.15.12:0/2358171320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0301010b0 0x7ff030198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.906+0000 7ff02effd640 1 -- 172.21.15.12:0/2358171320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff03019cfc0 msgr2=0x7ff03019d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:43.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.906+0000 7ff02effd640 1 --2- 172.21.15.12:0/2358171320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff03019cfc0 0x7ff03019d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:43.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.906+0000 7ff02effd640 1 -- 172.21.15.12:0/2358171320 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff03019dab0 con 0x7ff030106180 2024-04-22T13:24:43.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.906+0000 7ff02f7fe640 1 --2- 172.21.15.12:0/2358171320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0301010b0 0x7ff030198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:24:43.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.907+0000 7ff02effd640 1 --2- 172.21.15.12:0/2358171320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff030106180 0x7ff030198c40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff024036040 tx=0x7ff02402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:43.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.907+0000 7ff02cff9640 1 -- 172.21.15.12:0/2358171320 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff024043020 con 0x7ff030106180 2024-04-22T13:24:43.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.907+0000 7ff02cff9640 1 -- 172.21.15.12:0/2358171320 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff02402cbe0 con 0x7ff030106180 2024-04-22T13:24:43.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.907+0000 7ff036248640 1 -- 172.21.15.12:0/2358171320 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff03019dd40 con 0x7ff030106180 2024-04-22T13:24:43.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.907+0000 7ff02cff9640 1 -- 172.21.15.12:0/2358171320 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff024003bf0 con 0x7ff030106180 2024-04-22T13:24:43.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.907+0000 7ff036248640 1 -- 172.21.15.12:0/2358171320 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0301a5560 con 0x7ff030106180 2024-04-22T13:24:43.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.908+0000 7ff036248640 1 -- 172.21.15.12:0/2358171320 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff030069230 con 0x7ff030106180 2024-04-22T13:24:43.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.909+0000 7ff02cff9640 1 -- 172.21.15.12:0/2358171320 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7ff024046020 con 0x7ff030106180 2024-04-22T13:24:43.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.910+0000 7ff02cff9640 1 --2- 172.21.15.12:0/2358171320 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff0180a3e40 0x7ff0180a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:43.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.910+0000 7ff02f7fe640 1 --2- 172.21.15.12:0/2358171320 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff0180a3e40 0x7ff0180a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:43.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.910+0000 7ff02cff9640 1 -- 172.21.15.12:0/2358171320 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7ff0240f3120 con 0x7ff030106180 2024-04-22T13:24:43.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.910+0000 7ff02f7fe640 1 --2- 172.21.15.12:0/2358171320 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff0180a3e40 0x7ff0180a6300 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff02000bfd0 tx=0x7ff02000b390 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:43.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:43.914+0000 7ff02cff9640 1 -- 172.21.15.12:0/2358171320 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff02404d020 con 0x7ff030106180 2024-04-22T13:24:44.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:43 smithi012 ceph-mon[28265]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.8 KiB/s wr, 18 op/s 2024-04-22T13:24:44.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.140+0000 7ff036248640 1 -- 172.21.15.12:0/2358171320 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff030199a50 con 0x7ff030106180 2024-04-22T13:24:44.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.140+0000 7ff02cff9640 1 -- 172.21.15.12:0/2358171320 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7ff030199a50 con 0x7ff030106180 2024-04-22T13:24:44.137 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:24:44.137 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:24:44.137 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:24:44.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.142+0000 7ff036248640 1 -- 172.21.15.12:0/2358171320 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff0180a3e40 msgr2=0x7ff0180a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.142+0000 7ff036248640 1 --2- 172.21.15.12:0/2358171320 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff0180a3e40 0x7ff0180a6300 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff02000bfd0 tx=0x7ff02000b390 comp rx=0 tx=0).stop 2024-04-22T13:24:44.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.142+0000 7ff036248640 1 -- 172.21.15.12:0/2358171320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff030106180 msgr2=0x7ff030198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.142+0000 7ff036248640 1 --2- 172.21.15.12:0/2358171320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff030106180 0x7ff030198c40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff024036040 tx=0x7ff02402c7e0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.142+0000 7ff036248640 1 -- 172.21.15.12:0/2358171320 shutdown_connections 2024-04-22T13:24:44.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.142+0000 7ff036248640 1 --2- 172.21.15.12:0/2358171320 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff0180a3e40 0x7ff0180a6300 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.142+0000 7ff036248640 1 --2- 172.21.15.12:0/2358171320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff03019cfc0 0x7ff03019d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.142+0000 7ff036248640 1 --2- 172.21.15.12:0/2358171320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff030106180 0x7ff030198c40 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.142+0000 7ff036248640 1 --2- 172.21.15.12:0/2358171320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0301010b0 0x7ff030198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.142+0000 7ff036248640 1 -- 172.21.15.12:0/2358171320 >> 172.21.15.12:0/2358171320 conn(0x7ff0300771d0 msgr2=0x7ff0300fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:44.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.143+0000 7ff036248640 1 -- 172.21.15.12:0/2358171320 shutdown_connections 2024-04-22T13:24:44.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.143+0000 7ff036248640 1 -- 172.21.15.12:0/2358171320 wait complete. 2024-04-22T13:24:44.150 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:24:44.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:43 smithi134 ceph-mon[31338]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.8 KiB/s wr, 18 op/s 2024-04-22T13:24:44.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:43 smithi111 ceph-mon[31361]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.8 KiB/s wr, 18 op/s 2024-04-22T13:24:44.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.259+0000 7efd71ff4640 1 -- 172.21.15.12:0/1870444317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd6c102b40 msgr2=0x7efd6c106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.259+0000 7efd71ff4640 1 --2- 172.21.15.12:0/1870444317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd6c102b40 0x7efd6c106b80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7efd54009900 tx=0x7efd5402bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:44.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.259+0000 7efd71ff4640 1 -- 172.21.15.12:0/1870444317 shutdown_connections 2024-04-22T13:24:44.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.259+0000 7efd71ff4640 1 --2- 172.21.15.12:0/1870444317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd6c107260 0x7efd6c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.259+0000 7efd71ff4640 1 --2- 172.21.15.12:0/1870444317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd6c102b40 0x7efd6c106b80 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.259+0000 7efd71ff4640 1 --2- 172.21.15.12:0/1870444317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd6c102190 0x7efd6c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.259+0000 7efd71ff4640 1 -- 172.21.15.12:0/1870444317 >> 172.21.15.12:0/1870444317 conn(0x7efd6c0fd990 msgr2=0x7efd6c0ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:44.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.259+0000 7efd71ff4640 1 -- 172.21.15.12:0/1870444317 shutdown_connections 2024-04-22T13:24:44.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.259+0000 7efd71ff4640 1 -- 172.21.15.12:0/1870444317 wait complete. 2024-04-22T13:24:44.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.260+0000 7efd71ff4640 1 Processor -- start 2024-04-22T13:24:44.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.260+0000 7efd71ff4640 1 -- start start 2024-04-22T13:24:44.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.260+0000 7efd71ff4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd6c102190 0x7efd6c074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.260+0000 7efd71ff4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd6c107260 0x7efd6c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.260+0000 7efd71ff4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd6c074cd0 0x7efd6c1a6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.260+0000 7efd71ff4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c1121a0 con 0x7efd6c107260 2024-04-22T13:24:44.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.260+0000 7efd71ff4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c112020 con 0x7efd6c102190 2024-04-22T13:24:44.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.260+0000 7efd71ff4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c112320 con 0x7efd6c074cd0 2024-04-22T13:24:44.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.260+0000 7efd63fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd6c107260 0x7efd6c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.260+0000 7efd63fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd6c107260 0x7efd6c074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50122/0 (socket says 172.21.15.12:50122) 2024-04-22T13:24:44.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.260+0000 7efd63fff640 1 -- 172.21.15.12:0/1133523262 learned_addr learned my addr 172.21.15.12:0/1133523262 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:44.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.260+0000 7efd6b7fe640 1 --2- 172.21.15.12:0/1133523262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd6c102190 0x7efd6c074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.260+0000 7efd6bfff640 1 --2- 172.21.15.12:0/1133523262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd6c074cd0 0x7efd6c1a6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.260+0000 7efd63fff640 1 -- 172.21.15.12:0/1133523262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd6c074cd0 msgr2=0x7efd6c1a6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.260+0000 7efd63fff640 1 --2- 172.21.15.12:0/1133523262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd6c074cd0 0x7efd6c1a6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.260+0000 7efd63fff640 1 -- 172.21.15.12:0/1133523262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd6c102190 msgr2=0x7efd6c074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.260+0000 7efd63fff640 1 --2- 172.21.15.12:0/1133523262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd6c102190 0x7efd6c074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.260+0000 7efd63fff640 1 -- 172.21.15.12:0/1133523262 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd6c1a7060 con 0x7efd6c107260 2024-04-22T13:24:44.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.260+0000 7efd6b7fe640 1 --2- 172.21.15.12:0/1133523262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd6c102190 0x7efd6c074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:44.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.261+0000 7efd63fff640 1 --2- 172.21.15.12:0/1133523262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd6c107260 0x7efd6c074790 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7efd54036040 tx=0x7efd5402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:44.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.261+0000 7efd697fa640 1 -- 172.21.15.12:0/1133523262 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54043020 con 0x7efd6c107260 2024-04-22T13:24:44.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.261+0000 7efd697fa640 1 -- 172.21.15.12:0/1133523262 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd5402cbe0 con 0x7efd6c107260 2024-04-22T13:24:44.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.261+0000 7efd71ff4640 1 -- 172.21.15.12:0/1133523262 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd6c1a72f0 con 0x7efd6c107260 2024-04-22T13:24:44.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.261+0000 7efd697fa640 1 -- 172.21.15.12:0/1133523262 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54003bf0 con 0x7efd6c107260 2024-04-22T13:24:44.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.261+0000 7efd71ff4640 1 -- 172.21.15.12:0/1133523262 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd6c1a7890 con 0x7efd6c107260 2024-04-22T13:24:44.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.263+0000 7efd697fa640 1 -- 172.21.15.12:0/1133523262 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7efd54046020 con 0x7efd6c107260 2024-04-22T13:24:44.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.263+0000 7efd74bec740 1 Processor -- start 2024-04-22T13:24:44.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.263+0000 7efd74bec740 1 -- start start 2024-04-22T13:24:44.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.264+0000 7efd74bec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56439ad5d4d0 0x56439ad5d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.264+0000 7efd74bec740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56439ad5de80 0x56439ad66430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.264+0000 7efd74bec740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56439ad66a00 0x56439ad68e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.264+0000 7efd74bec740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56439acc21f0 con 0x56439ad66a00 2024-04-22T13:24:44.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.264+0000 7efd74bec740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56439ac5f4b0 con 0x56439ad5de80 2024-04-22T13:24:44.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.264+0000 7efd74bec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56439acbe1b0 con 0x56439ad5d4d0 2024-04-22T13:24:44.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.264+0000 7efd63fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56439ad66a00 0x56439ad68e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.264+0000 7efd63fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56439ad66a00 0x56439ad68e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50132/0 (socket says 172.21.15.12:50132) 2024-04-22T13:24:44.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.264+0000 7efd63fff640 1 -- 172.21.15.12:0/1635498381 learned_addr learned my addr 172.21.15.12:0/1635498381 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:44.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.264+0000 7efd6b7fe640 1 --2- 172.21.15.12:0/1635498381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56439ad5de80 0x56439ad66430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.264+0000 7efd697fa640 1 --2- 172.21.15.12:0/1133523262 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7efd580a3ef0 0x7efd580a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.264+0000 7efd6bfff640 1 --2- 172.21.15.12:0/1635498381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56439ad5d4d0 0x56439ad5d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.264+0000 7efd6bfff640 1 --2- 172.21.15.12:0/1133523262 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7efd580a3ef0 0x7efd580a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.264+0000 7efd697fa640 1 -- 172.21.15.12:0/1133523262 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7efd540f2be0 con 0x7efd6c107260 2024-04-22T13:24:44.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.264+0000 7efd63fff640 1 -- 172.21.15.12:0/1635498381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56439ad5d4d0 msgr2=0x56439ad5d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.264+0000 7efd63fff640 1 --2- 172.21.15.12:0/1635498381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56439ad5d4d0 0x56439ad5d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.264+0000 7efd63fff640 1 -- 172.21.15.12:0/1635498381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56439ad5de80 msgr2=0x56439ad66430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.264+0000 7efd63fff640 1 --2- 172.21.15.12:0/1635498381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56439ad5de80 0x56439ad66430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.264+0000 7efd63fff640 1 -- 172.21.15.12:0/1635498381 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56439ac511b0 con 0x56439ad66a00 2024-04-22T13:24:44.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.264+0000 7efd6bfff640 1 --2- 172.21.15.12:0/1635498381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56439ad5d4d0 0x56439ad5d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:44.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.264+0000 7efd6b7fe640 1 --2- 172.21.15.12:0/1635498381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56439ad5de80 0x56439ad66430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:44.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.264+0000 7efd63fff640 1 --2- 172.21.15.12:0/1635498381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56439ad66a00 0x56439ad68e50 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7efd540f0250 tx=0x7efd5402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54a63d5f89a2502 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:44.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.265+0000 7efd61ffb640 1 -- 172.21.15.12:0/1635498381 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd540be800 con 0x56439ad66a00 2024-04-22T13:24:44.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.265+0000 7efd6bfff640 1 --2- 172.21.15.12:0/1133523262 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7efd580a3ef0 0x7efd580a63b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efd6400afa0 tx=0x7efd64005c80 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:44.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.265+0000 7efd61ffb640 1 -- 172.21.15.12:0/1635498381 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd5403c350 con 0x56439ad66a00 2024-04-22T13:24:44.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.265+0000 7efd61ffb640 1 -- 172.21.15.12:0/1635498381 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd540be390 con 0x56439ad66a00 2024-04-22T13:24:44.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.265+0000 7efd74bec740 1 -- 172.21.15.12:0/1635498381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56439ad66a00 msgr2=0x56439ad68e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.265+0000 7efd74bec740 1 --2- 172.21.15.12:0/1635498381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56439ad66a00 0x56439ad68e50 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7efd540f0250 tx=0x7efd5402bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:44.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.265+0000 7efd74bec740 1 -- 172.21.15.12:0/1635498381 shutdown_connections 2024-04-22T13:24:44.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.265+0000 7efd74bec740 1 --2- 172.21.15.12:0/1635498381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56439ad66a00 0x56439ad68e50 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.265+0000 7efd74bec740 1 --2- 172.21.15.12:0/1635498381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56439ad5de80 0x56439ad66430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.265+0000 7efd74bec740 1 --2- 172.21.15.12:0/1635498381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56439ad5d4d0 0x56439ad5d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.265+0000 7efd74bec740 1 -- 172.21.15.12:0/1635498381 >> 172.21.15.12:0/1635498381 conn(0x56439ad57ba0 msgr2=0x56439ad57f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:44.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.265+0000 7efd74bec740 1 -- 172.21.15.12:0/1635498381 shutdown_connections 2024-04-22T13:24:44.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.265+0000 7efd74bec740 1 -- 172.21.15.12:0/1635498381 wait complete. 2024-04-22T13:24:44.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.266+0000 7efd74bec740 1 -- start start 2024-04-22T13:24:44.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.266+0000 7efd74bec740 1 Processor -- start 2024-04-22T13:24:44.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.266+0000 7efd74bec740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56439ad5d4d0 0x56439ad66360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.266+0000 7efd74bec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56439ad5de80 0x56439ad636f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.266+0000 7efd74bec740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56439ad66a00 0x56439ad63d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.266+0000 7efd74bec740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56439ad69730 con 0x56439ad5d4d0 2024-04-22T13:24:44.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.266+0000 7efd74bec740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56439ac7b500 con 0x56439ad66a00 2024-04-22T13:24:44.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.266+0000 7efd74bec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56439ad698b0 con 0x56439ad5de80 2024-04-22T13:24:44.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.266+0000 7efd74bec740 10 client.0 _ll_register_callbacks cb 0x56439ac49b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:44.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.266+0000 7efd6b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56439ad5d4d0 0x56439ad66360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.266+0000 7efd6b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56439ad5d4d0 0x56439ad66360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50140/0 (socket says 172.21.15.12:50140) 2024-04-22T13:24:44.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.266+0000 7efd6b7fe640 1 -- 172.21.15.12:0/3942185031 learned_addr learned my addr 172.21.15.12:0/3942185031 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:44.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.266+0000 7efd6bfff640 1 --2- 172.21.15.12:0/3942185031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56439ad66a00 0x56439ad63d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.266+0000 7efd63fff640 1 --2- 172.21.15.12:0/3942185031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56439ad5de80 0x56439ad636f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.266+0000 7efd6b7fe640 1 -- 172.21.15.12:0/3942185031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56439ad5de80 msgr2=0x56439ad636f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.266+0000 7efd6b7fe640 1 --2- 172.21.15.12:0/3942185031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56439ad5de80 0x56439ad636f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.266+0000 7efd6b7fe640 1 -- 172.21.15.12:0/3942185031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56439ad66a00 msgr2=0x56439ad63d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.266+0000 7efd6b7fe640 1 --2- 172.21.15.12:0/3942185031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56439ad66a00 0x56439ad63d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.267+0000 7efd6b7fe640 1 -- 172.21.15.12:0/3942185031 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56439a9d43f0 con 0x56439ad5d4d0 2024-04-22T13:24:44.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.267+0000 7efd6b7fe640 1 --2- 172.21.15.12:0/3942185031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56439ad5d4d0 0x56439ad66360 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7efd5c00bf80 tx=0x7efd5c011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc06582958f62aad server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:44.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.267+0000 7efd4affd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:44.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.267+0000 7efd4affd640 1 -- 172.21.15.12:0/3942185031 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c005220 con 0x56439ad5d4d0 2024-04-22T13:24:44.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.267+0000 7efd4affd640 1 -- 172.21.15.12:0/3942185031 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7efd5c0053c0 con 0x56439ad5d4d0 2024-04-22T13:24:44.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.267+0000 7efd74bec740 1 -- 172.21.15.12:0/3942185031 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd5403c350 con 0x56439ad5d4d0 2024-04-22T13:24:44.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.267+0000 7efd4affd640 1 -- 172.21.15.12:0/3942185031 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efd5c0055c0 con 0x56439ad5d4d0 2024-04-22T13:24:44.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.267+0000 7efd74bec740 10 client.15900 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:44.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.267+0000 7efd74bec740 1 -- 172.21.15.12:0/3942185031 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56439aa9e7d0 con 0x56439ad5d4d0 2024-04-22T13:24:44.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.267+0000 7efd4affd640 1 -- 172.21.15.12:0/3942185031 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd5403c350 con 0x56439ad5d4d0 2024-04-22T13:24:44.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.267+0000 7efd4affd640 1 -- 172.21.15.12:0/3942185031 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7efd5c012a70 con 0x56439ad5d4d0 2024-04-22T13:24:44.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.267+0000 7efd74bec740 10 client.15900 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:44.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.267+0000 7efd74bec740 10 client.15900 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:24:44.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.267+0000 7efd74bec740 20 client.15900 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:44.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.267+0000 7efd74bec740 1 --2- 172.21.15.12:0/3942185031 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x56439ad76960 0x56439ad78d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.267+0000 7efd74bec740 4 client.15900 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:24:44.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.267+0000 7efd74bec740 1 -- 172.21.15.12:0/3942185031 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56439ac61930 con 0x56439ad76960 2024-04-22T13:24:44.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.268+0000 7efd63fff640 1 --2- 172.21.15.12:0/3942185031 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x56439ad76960 0x56439ad78d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.268+0000 7efd63fff640 1 --2- 172.21.15.12:0/3942185031 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x56439ad76960 0x56439ad78d70 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=22777a221a095a7b server_cookie=e7172333a3b74690 in_seq=0 out_seq=0 2024-04-22T13:24:44.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.268+0000 7efd4affd640 10 client.15900 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:24:44.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.269+0000 7efd4affd640 1 -- 172.21.15.12:0/3942185031 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56439ac61930 con 0x56439ad76960 2024-04-22T13:24:44.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.269+0000 7efd4affd640 10 client.15900 handle_command_reply: tid=0 2024-04-22T13:24:44.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.269+0000 7efd74bec740 1 client.15900 shutdown 2024-04-22T13:24:44.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.269+0000 7efd74bec740 1 -- 172.21.15.12:0/3942185031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56439ad5d4d0 msgr2=0x56439ad66360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.269+0000 7efd74bec740 1 --2- 172.21.15.12:0/3942185031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56439ad5d4d0 0x56439ad66360 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7efd5c00bf80 tx=0x7efd5c011870 comp rx=0 tx=0).stop 2024-04-22T13:24:44.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.269+0000 7efd74bec740 1 -- 172.21.15.12:0/3942185031 shutdown_connections 2024-04-22T13:24:44.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.269+0000 7efd74bec740 1 --2- 172.21.15.12:0/3942185031 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x56439ad76960 0x56439ad78d70 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.269+0000 7efd74bec740 1 --2- 172.21.15.12:0/3942185031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56439ad66a00 0x56439ad63d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.269+0000 7efd74bec740 1 --2- 172.21.15.12:0/3942185031 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56439ad5de80 0x56439ad636f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.269+0000 7efd74bec740 1 --2- 172.21.15.12:0/3942185031 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56439ad5d4d0 0x56439ad66360 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.269+0000 7efd4affd640 0 client.15900 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:24:44.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.269+0000 7efd74bec740 1 -- 172.21.15.12:0/3942185031 >> 172.21.15.12:0/3942185031 conn(0x56439ad57ba0 msgr2=0x56439ad598f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:44.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.269+0000 7efd74bec740 1 -- 172.21.15.12:0/3942185031 shutdown_connections 2024-04-22T13:24:44.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.269+0000 7efd74bec740 1 -- 172.21.15.12:0/3942185031 wait complete. 2024-04-22T13:24:44.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.269+0000 7efd74bec740 20 client.15900 trim_cache size 0 max 16384 2024-04-22T13:24:44.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.282+0000 7efd74bec740 1 Processor -- start 2024-04-22T13:24:44.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.282+0000 7efd74bec740 1 -- start start 2024-04-22T13:24:44.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.283+0000 7efd74bec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56439ad5d4d0 0x56439ad724e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.283+0000 7efd74bec740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56439ad5de80 0x56439ad635f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.283+0000 7efd74bec740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56439ad66a00 0x56439ad65e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.283+0000 7efd74bec740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c1afd60 con 0x56439ad5de80 2024-04-22T13:24:44.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.283+0000 7efd74bec740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c1b18a0 con 0x56439ad66a00 2024-04-22T13:24:44.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.283+0000 7efd74bec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c1b0430 con 0x56439ad5d4d0 2024-04-22T13:24:44.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.283+0000 7efd63fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56439ad5de80 0x56439ad635f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.283+0000 7efd63fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56439ad5de80 0x56439ad635f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50148/0 (socket says 172.21.15.12:50148) 2024-04-22T13:24:44.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.283+0000 7efd63fff640 1 -- 172.21.15.12:0/455169459 learned_addr learned my addr 172.21.15.12:0/455169459 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:44.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.283+0000 7efd6b7fe640 1 --2- 172.21.15.12:0/455169459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56439ad5d4d0 0x56439ad724e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.283+0000 7efd6bfff640 1 --2- 172.21.15.12:0/455169459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56439ad66a00 0x56439ad65e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.283+0000 7efd63fff640 1 -- 172.21.15.12:0/455169459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56439ad5d4d0 msgr2=0x56439ad724e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.283+0000 7efd63fff640 1 --2- 172.21.15.12:0/455169459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56439ad5d4d0 0x56439ad724e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.283+0000 7efd63fff640 1 -- 172.21.15.12:0/455169459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56439ad66a00 msgr2=0x56439ad65e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.283+0000 7efd63fff640 1 --2- 172.21.15.12:0/455169459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56439ad66a00 0x56439ad65e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.283+0000 7efd63fff640 1 -- 172.21.15.12:0/455169459 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56439ad61c40 con 0x56439ad5de80 2024-04-22T13:24:44.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.283+0000 7efd63fff640 1 --2- 172.21.15.12:0/455169459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56439ad5de80 0x56439ad635f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7efd540bcce0 tx=0x7efd540036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b7348b92dc4e8a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:44.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.283+0000 7efd617fa640 1 -- 172.21.15.12:0/455169459 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd540be570 con 0x56439ad5de80 2024-04-22T13:24:44.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.283+0000 7efd617fa640 1 -- 172.21.15.12:0/455169459 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd5403c350 con 0x56439ad5de80 2024-04-22T13:24:44.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.283+0000 7efd617fa640 1 -- 172.21.15.12:0/455169459 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd540bea30 con 0x56439ad5de80 2024-04-22T13:24:44.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.284+0000 7efd74bec740 1 -- 172.21.15.12:0/455169459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56439ad5de80 msgr2=0x56439ad635f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.284+0000 7efd74bec740 1 --2- 172.21.15.12:0/455169459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56439ad5de80 0x56439ad635f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7efd540bcce0 tx=0x7efd540036f0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.284+0000 7efd74bec740 1 -- 172.21.15.12:0/455169459 shutdown_connections 2024-04-22T13:24:44.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.284+0000 7efd74bec740 1 --2- 172.21.15.12:0/455169459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56439ad66a00 0x56439ad65e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.284+0000 7efd74bec740 1 --2- 172.21.15.12:0/455169459 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56439ad5de80 0x56439ad635f0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.284+0000 7efd74bec740 1 --2- 172.21.15.12:0/455169459 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56439ad5d4d0 0x56439ad724e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.284+0000 7efd74bec740 1 -- 172.21.15.12:0/455169459 >> 172.21.15.12:0/455169459 conn(0x56439ad57ba0 msgr2=0x56439ad56c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:44.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.284+0000 7efd74bec740 1 -- 172.21.15.12:0/455169459 shutdown_connections 2024-04-22T13:24:44.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.284+0000 7efd74bec740 1 -- 172.21.15.12:0/455169459 wait complete. 2024-04-22T13:24:44.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.284+0000 7efd74bec740 1 -- start start 2024-04-22T13:24:44.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.284+0000 7efd74bec740 1 Processor -- start 2024-04-22T13:24:44.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.285+0000 7efd74bec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56439ad5d4d0 0x56439ad6aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.285+0000 7efd74bec740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56439ad5de80 0x56439ad6b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.285+0000 7efd74bec740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56439ad66a00 0x56439addbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.285+0000 7efd74bec740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c1b11d0 con 0x56439ad66a00 2024-04-22T13:24:44.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.285+0000 7efd74bec740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c1b0b00 con 0x56439ad5de80 2024-04-22T13:24:44.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.285+0000 7efd74bec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c1add60 con 0x56439ad5d4d0 2024-04-22T13:24:44.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.285+0000 7efd74bec740 10 client.0 _ll_register_callbacks cb 0x7efd5c00bb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:44.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.285+0000 7efd6bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56439ad66a00 0x56439addbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.285+0000 7efd6bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56439ad66a00 0x56439addbeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50164/0 (socket says 172.21.15.12:50164) 2024-04-22T13:24:44.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.285+0000 7efd6bfff640 1 -- 172.21.15.12:0/450592168 learned_addr learned my addr 172.21.15.12:0/450592168 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:44.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.285+0000 7efd6b7fe640 1 --2- 172.21.15.12:0/450592168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56439ad5d4d0 0x56439ad6aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.285+0000 7efd63fff640 1 --2- 172.21.15.12:0/450592168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56439ad5de80 0x56439ad6b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.285+0000 7efd6bfff640 1 -- 172.21.15.12:0/450592168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56439ad5d4d0 msgr2=0x56439ad6aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.285+0000 7efd6bfff640 1 --2- 172.21.15.12:0/450592168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56439ad5d4d0 0x56439ad6aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.285+0000 7efd6bfff640 1 -- 172.21.15.12:0/450592168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56439ad5de80 msgr2=0x56439ad6b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.285+0000 7efd6bfff640 1 --2- 172.21.15.12:0/450592168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56439ad5de80 0x56439ad6b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.285+0000 7efd6bfff640 1 -- 172.21.15.12:0/450592168 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56439addc3f0 con 0x56439ad66a00 2024-04-22T13:24:44.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.285+0000 7efd6bfff640 1 --2- 172.21.15.12:0/450592168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56439ad66a00 0x56439addbeb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7efd6401ea90 tx=0x7efd64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82defd1beffc8e3b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:44.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.285+0000 7efd4affd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:44.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.286+0000 7efd4affd640 1 -- 172.21.15.12:0/450592168 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6400fe40 con 0x56439ad66a00 2024-04-22T13:24:44.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.286+0000 7efd4affd640 1 -- 172.21.15.12:0/450592168 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7efd6401f430 con 0x56439ad66a00 2024-04-22T13:24:44.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.286+0000 7efd74bec740 1 -- 172.21.15.12:0/450592168 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd5403c350 con 0x56439ad66a00 2024-04-22T13:24:44.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.286+0000 7efd4affd640 1 -- 172.21.15.12:0/450592168 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efd6401f630 con 0x56439ad66a00 2024-04-22T13:24:44.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.286+0000 7efd74bec740 10 client.15906 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:44.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.286+0000 7efd74bec740 1 -- 172.21.15.12:0/450592168 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56439addd6a0 con 0x56439ad66a00 2024-04-22T13:24:44.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.286+0000 7efd4affd640 1 -- 172.21.15.12:0/450592168 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd5403c350 con 0x56439ad66a00 2024-04-22T13:24:44.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.286+0000 7efd4affd640 1 -- 172.21.15.12:0/450592168 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7efd64027880 con 0x56439ad66a00 2024-04-22T13:24:44.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.286+0000 7efd74bec740 10 client.15906 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:44.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.286+0000 7efd74bec740 10 client.15906 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:24:44.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.286+0000 7efd74bec740 20 client.15906 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:44.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.286+0000 7efd74bec740 1 --2- 172.21.15.12:0/450592168 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x56439addde10 0x56439ade0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.286+0000 7efd74bec740 4 client.15906 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:24:44.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.286+0000 7efd74bec740 1 -- 172.21.15.12:0/450592168 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56439accfc10 con 0x56439addde10 2024-04-22T13:24:44.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.286+0000 7efd6b7fe640 1 --2- 172.21.15.12:0/450592168 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x56439addde10 0x56439ade0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.287+0000 7efd4affd640 10 client.15906 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:24:44.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.287+0000 7efd6b7fe640 1 --2- 172.21.15.12:0/450592168 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x56439addde10 0x56439ade0230 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1d5b98695b786ac3 server_cookie=eaa31cd94aed248a in_seq=0 out_seq=0 2024-04-22T13:24:44.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.287+0000 7efd4affd640 1 -- 172.21.15.12:0/450592168 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56439accfc10 con 0x56439addde10 2024-04-22T13:24:44.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.287+0000 7efd4affd640 10 client.15906 handle_command_reply: tid=0 2024-04-22T13:24:44.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.287+0000 7efd74bec740 1 client.15906 shutdown 2024-04-22T13:24:44.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.287+0000 7efd74bec740 1 -- 172.21.15.12:0/450592168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56439ad66a00 msgr2=0x56439addbeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.287+0000 7efd74bec740 1 --2- 172.21.15.12:0/450592168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56439ad66a00 0x56439addbeb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7efd6401ea90 tx=0x7efd64004a40 comp rx=0 tx=0).stop 2024-04-22T13:24:44.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.287+0000 7efd74bec740 1 -- 172.21.15.12:0/450592168 shutdown_connections 2024-04-22T13:24:44.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.287+0000 7efd74bec740 1 --2- 172.21.15.12:0/450592168 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x56439addde10 0x56439ade0230 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.287+0000 7efd74bec740 1 --2- 172.21.15.12:0/450592168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56439ad66a00 0x56439addbeb0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.287+0000 7efd4affd640 0 client.15906 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:24:44.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.287+0000 7efd74bec740 1 --2- 172.21.15.12:0/450592168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56439ad5de80 0x56439ad6b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.287+0000 7efd74bec740 1 --2- 172.21.15.12:0/450592168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56439ad5d4d0 0x56439ad6aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.287+0000 7efd74bec740 1 -- 172.21.15.12:0/450592168 >> 172.21.15.12:0/450592168 conn(0x56439ad57ba0 msgr2=0x56439ad5e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:44.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.287+0000 7efd74bec740 1 -- 172.21.15.12:0/450592168 shutdown_connections 2024-04-22T13:24:44.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.287+0000 7efd74bec740 1 -- 172.21.15.12:0/450592168 wait complete. 2024-04-22T13:24:44.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.287+0000 7efd74bec740 20 client.15906 trim_cache size 0 max 16384 2024-04-22T13:24:44.290 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:24:44.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.288+0000 7efd71ff4640 1 -- 172.21.15.12:0/1133523262 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7efd580a3ef0 msgr2=0x7efd580a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.288+0000 7efd71ff4640 1 --2- 172.21.15.12:0/1133523262 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7efd580a3ef0 0x7efd580a63b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efd6400afa0 tx=0x7efd64005c80 comp rx=0 tx=0).stop 2024-04-22T13:24:44.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.288+0000 7efd71ff4640 1 -- 172.21.15.12:0/1133523262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd6c107260 msgr2=0x7efd6c074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.288+0000 7efd71ff4640 1 --2- 172.21.15.12:0/1133523262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd6c107260 0x7efd6c074790 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7efd54036040 tx=0x7efd5402c7e0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.288+0000 7efd71ff4640 1 -- 172.21.15.12:0/1133523262 shutdown_connections 2024-04-22T13:24:44.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.288+0000 7efd71ff4640 1 --2- 172.21.15.12:0/1133523262 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7efd580a3ef0 0x7efd580a63b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.288+0000 7efd71ff4640 1 --2- 172.21.15.12:0/1133523262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd6c074cd0 0x7efd6c1a6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.288+0000 7efd71ff4640 1 --2- 172.21.15.12:0/1133523262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd6c107260 0x7efd6c074790 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.288+0000 7efd71ff4640 1 --2- 172.21.15.12:0/1133523262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd6c102190 0x7efd6c074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.288+0000 7efd71ff4640 1 -- 172.21.15.12:0/1133523262 >> 172.21.15.12:0/1133523262 conn(0x7efd6c0fd990 msgr2=0x7efd6c0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:44.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.288+0000 7efd71ff4640 1 -- 172.21.15.12:0/1133523262 shutdown_connections 2024-04-22T13:24:44.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.288+0000 7efd71ff4640 1 -- 172.21.15.12:0/1133523262 wait complete. 2024-04-22T13:24:44.295 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:24:44.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.402+0000 7fc5b151e640 1 -- 172.21.15.12:0/1916286372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5ac06a840 msgr2=0x7fc5ac06ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.402+0000 7fc5b151e640 1 --2- 172.21.15.12:0/1916286372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5ac06a840 0x7fc5ac06ac20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009900 tx=0x7fc59c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:44.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.402+0000 7fc5b151e640 1 -- 172.21.15.12:0/1916286372 shutdown_connections 2024-04-22T13:24:44.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.402+0000 7fc5b151e640 1 --2- 172.21.15.12:0/1916286372 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5ac107270 0x7fc5ac10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.402+0000 7fc5b151e640 1 --2- 172.21.15.12:0/1916286372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5ac06b1f0 0x7fc5ac106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.402+0000 7fc5b151e640 1 --2- 172.21.15.12:0/1916286372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5ac06a840 0x7fc5ac06ac20 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.402+0000 7fc5b151e640 1 -- 172.21.15.12:0/1916286372 >> 172.21.15.12:0/1916286372 conn(0x7fc5ac077eb0 msgr2=0x7fc5ac0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:44.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.402+0000 7fc5b151e640 1 -- 172.21.15.12:0/1916286372 shutdown_connections 2024-04-22T13:24:44.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.402+0000 7fc5b151e640 1 -- 172.21.15.12:0/1916286372 wait complete. 2024-04-22T13:24:44.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.403+0000 7fc5b151e640 1 Processor -- start 2024-04-22T13:24:44.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.403+0000 7fc5b151e640 1 -- start start 2024-04-22T13:24:44.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.403+0000 7fc5b151e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5ac06a840 0x7fc5ac1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.403+0000 7fc5b151e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5ac107270 0x7fc5ac198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.403+0000 7fc5b151e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5ac19cf60 0x7fc5ac19d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.403+0000 7fc5b151e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac1121b0 con 0x7fc5ac06a840 2024-04-22T13:24:44.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.403+0000 7fc5b151e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac112030 con 0x7fc5ac107270 2024-04-22T13:24:44.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.403+0000 7fc5b151e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac112330 con 0x7fc5ac19cf60 2024-04-22T13:24:44.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.403+0000 7fc5aaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5ac06a840 0x7fc5ac1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.403+0000 7fc5aaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5ac06a840 0x7fc5ac1986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50176/0 (socket says 172.21.15.12:50176) 2024-04-22T13:24:44.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.403+0000 7fc5aaffd640 1 -- 172.21.15.12:0/475351424 learned_addr learned my addr 172.21.15.12:0/475351424 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:44.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.403+0000 7fc5a3fff640 1 --2- 172.21.15.12:0/475351424 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5ac107270 0x7fc5ac198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.403+0000 7fc5ab7fe640 1 --2- 172.21.15.12:0/475351424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5ac19cf60 0x7fc5ac19d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.404+0000 7fc5aaffd640 1 -- 172.21.15.12:0/475351424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5ac19cf60 msgr2=0x7fc5ac19d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.404+0000 7fc5aaffd640 1 --2- 172.21.15.12:0/475351424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5ac19cf60 0x7fc5ac19d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.404+0000 7fc5aaffd640 1 -- 172.21.15.12:0/475351424 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5ac107270 msgr2=0x7fc5ac198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.404+0000 7fc5aaffd640 1 --2- 172.21.15.12:0/475351424 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5ac107270 0x7fc5ac198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.404+0000 7fc5aaffd640 1 -- 172.21.15.12:0/475351424 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5ac19da50 con 0x7fc5ac06a840 2024-04-22T13:24:44.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.404+0000 7fc5aaffd640 1 --2- 172.21.15.12:0/475351424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5ac06a840 0x7fc5ac1986f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc59c005940 tx=0x7fc59c02c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:44.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.404+0000 7fc5a8ff9640 1 -- 172.21.15.12:0/475351424 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c043020 con 0x7fc5ac06a840 2024-04-22T13:24:44.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.404+0000 7fc5b151e640 1 -- 172.21.15.12:0/475351424 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5ac19dce0 con 0x7fc5ac06a840 2024-04-22T13:24:44.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.404+0000 7fc5a8ff9640 1 -- 172.21.15.12:0/475351424 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc59c02cbe0 con 0x7fc5ac06a840 2024-04-22T13:24:44.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.404+0000 7fc5a8ff9640 1 -- 172.21.15.12:0/475351424 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c003bf0 con 0x7fc5ac06a840 2024-04-22T13:24:44.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.404+0000 7fc5b151e640 1 -- 172.21.15.12:0/475351424 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5ac1a54e0 con 0x7fc5ac06a840 2024-04-22T13:24:44.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.406+0000 7fc5a8ff9640 1 -- 172.21.15.12:0/475351424 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fc59c046020 con 0x7fc5ac06a840 2024-04-22T13:24:44.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.406+0000 7fc5b41df740 1 Processor -- start 2024-04-22T13:24:44.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.406+0000 7fc5b41df740 1 -- start start 2024-04-22T13:24:44.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.407+0000 7fc5b41df740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc7df703b0 0x55cc7df70790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.407+0000 7fc5b41df740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc7df70d60 0x55cc7df79890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.407+0000 7fc5b41df740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc7df79e60 0x55cc7df7a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.407+0000 7fc5b41df740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55cc7ded51f0 con 0x55cc7df79e60 2024-04-22T13:24:44.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.407+0000 7fc5b41df740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55cc7de724b0 con 0x55cc7df70d60 2024-04-22T13:24:44.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.407+0000 7fc5b41df740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cc7ded11b0 con 0x55cc7df703b0 2024-04-22T13:24:44.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.407+0000 7fc5a8ff9640 1 --2- 172.21.15.12:0/475351424 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc5980a3e40 0x7fc5980a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.407+0000 7fc5a3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc7df79e60 0x55cc7df7a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.407+0000 7fc5a3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc7df79e60 0x55cc7df7a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50182/0 (socket says 172.21.15.12:50182) 2024-04-22T13:24:44.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.407+0000 7fc5a3fff640 1 -- 172.21.15.12:0/2102240808 learned_addr learned my addr 172.21.15.12:0/2102240808 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:44.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.407+0000 7fc5ab7fe640 1 --2- 172.21.15.12:0/2102240808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc7df703b0 0x55cc7df70790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.407+0000 7fc5aaffd640 1 --2- 172.21.15.12:0/2102240808 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc7df70d60 0x55cc7df79890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.407+0000 7fc5ab7fe640 1 --2- 172.21.15.12:0/475351424 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc5980a3e40 0x7fc5980a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.407+0000 7fc5a8ff9640 1 -- 172.21.15.12:0/475351424 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fc59c0f2bd0 con 0x7fc5ac06a840 2024-04-22T13:24:44.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.407+0000 7fc5a3fff640 1 -- 172.21.15.12:0/2102240808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc7df703b0 msgr2=0x55cc7df70790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.407+0000 7fc5a3fff640 1 --2- 172.21.15.12:0/2102240808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc7df703b0 0x55cc7df70790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.407+0000 7fc5a3fff640 1 -- 172.21.15.12:0/2102240808 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc7df70d60 msgr2=0x55cc7df79890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.407+0000 7fc5a3fff640 1 --2- 172.21.15.12:0/2102240808 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc7df70d60 0x55cc7df79890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.407+0000 7fc5a3fff640 1 -- 172.21.15.12:0/2102240808 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc7de641b0 con 0x55cc7df79e60 2024-04-22T13:24:44.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.407+0000 7fc5aaffd640 1 --2- 172.21.15.12:0/2102240808 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc7df70d60 0x55cc7df79890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:44.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.408+0000 7fc5a3fff640 1 --2- 172.21.15.12:0/2102240808 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc7df79e60 0x55cc7df7a2c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc59400bc70 tx=0x7fc5940107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be0f411ebebe801b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:44.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.408+0000 7fc5ab7fe640 1 --2- 172.21.15.12:0/475351424 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc5980a3e40 0x7fc5980a6300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4013b10 tx=0x7fc5a4004ab0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:44.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.408+0000 7fc5a17fa640 1 -- 172.21.15.12:0/2102240808 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594005220 con 0x55cc7df79e60 2024-04-22T13:24:44.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.408+0000 7fc5a17fa640 1 -- 172.21.15.12:0/2102240808 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc59400be20 con 0x55cc7df79e60 2024-04-22T13:24:44.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.408+0000 7fc5a17fa640 1 -- 172.21.15.12:0/2102240808 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594011680 con 0x55cc7df79e60 2024-04-22T13:24:44.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.408+0000 7fc5b41df740 1 -- 172.21.15.12:0/2102240808 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc7df79e60 msgr2=0x55cc7df7a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.408+0000 7fc5b41df740 1 --2- 172.21.15.12:0/2102240808 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc7df79e60 0x55cc7df7a2c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc59400bc70 tx=0x7fc5940107a0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.409+0000 7fc5b41df740 1 -- 172.21.15.12:0/2102240808 shutdown_connections 2024-04-22T13:24:44.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.409+0000 7fc5b41df740 1 --2- 172.21.15.12:0/2102240808 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc7df79e60 0x55cc7df7a2c0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.409+0000 7fc5b41df740 1 --2- 172.21.15.12:0/2102240808 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc7df70d60 0x55cc7df79890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.409+0000 7fc5b41df740 1 --2- 172.21.15.12:0/2102240808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc7df703b0 0x55cc7df70790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.409+0000 7fc5b41df740 1 -- 172.21.15.12:0/2102240808 >> 172.21.15.12:0/2102240808 conn(0x55cc7df6aba0 msgr2=0x55cc7df6af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:44.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.409+0000 7fc5b41df740 1 -- 172.21.15.12:0/2102240808 shutdown_connections 2024-04-22T13:24:44.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.409+0000 7fc5b41df740 1 -- 172.21.15.12:0/2102240808 wait complete. 2024-04-22T13:24:44.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.409+0000 7fc5b41df740 1 -- start start 2024-04-22T13:24:44.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.409+0000 7fc5b41df740 1 Processor -- start 2024-04-22T13:24:44.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.409+0000 7fc5b41df740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc7df703b0 0x55cc7df80290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.409+0000 7fc5b41df740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc7df70d60 0x55cc7df7d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.409+0000 7fc5b41df740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc7df79e60 0x55cc7df7db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.409+0000 7fc5b41df740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55cc7df80da0 con 0x55cc7df70d60 2024-04-22T13:24:44.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.409+0000 7fc5b41df740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55cc7de8e500 con 0x55cc7df79e60 2024-04-22T13:24:44.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.409+0000 7fc5b41df740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cc7df80f20 con 0x55cc7df703b0 2024-04-22T13:24:44.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.409+0000 7fc5b41df740 10 client.0 _ll_register_callbacks cb 0x55cc7de5cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:44.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.409+0000 7fc5a3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc7df70d60 0x55cc7df7d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.409+0000 7fc5a3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc7df70d60 0x55cc7df7d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50186/0 (socket says 172.21.15.12:50186) 2024-04-22T13:24:44.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.409+0000 7fc5a3fff640 1 -- 172.21.15.12:0/699250916 learned_addr learned my addr 172.21.15.12:0/699250916 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:44.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.410+0000 7fc5ab7fe640 1 --2- 172.21.15.12:0/699250916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc7df79e60 0x55cc7df7db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.410+0000 7fc5aaffd640 1 --2- 172.21.15.12:0/699250916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc7df703b0 0x55cc7df80290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.410+0000 7fc5a3fff640 1 -- 172.21.15.12:0/699250916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc7df703b0 msgr2=0x55cc7df80290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.410+0000 7fc5a3fff640 1 --2- 172.21.15.12:0/699250916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc7df703b0 0x55cc7df80290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.410+0000 7fc5a3fff640 1 -- 172.21.15.12:0/699250916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc7df79e60 msgr2=0x55cc7df7db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.410+0000 7fc5a3fff640 1 --2- 172.21.15.12:0/699250916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc7df79e60 0x55cc7df7db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.410+0000 7fc5a3fff640 1 -- 172.21.15.12:0/699250916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc7dbe73f0 con 0x55cc7df70d60 2024-04-22T13:24:44.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.410+0000 7fc5aaffd640 1 --2- 172.21.15.12:0/699250916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc7df703b0 0x55cc7df80290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:44.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.410+0000 7fc5ab7fe640 1 --2- 172.21.15.12:0/699250916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc7df79e60 0x55cc7df7db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:44.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.410+0000 7fc5a3fff640 1 --2- 172.21.15.12:0/699250916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc7df70d60 0x55cc7df7d620 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc594011010 tx=0x7fc5940042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd235266ff7d9ef server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:44.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.410+0000 7fc5927fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:44.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.410+0000 7fc5927fc640 1 -- 172.21.15.12:0/699250916 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59402a060 con 0x55cc7df70d60 2024-04-22T13:24:44.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.410+0000 7fc5927fc640 1 -- 172.21.15.12:0/699250916 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fc594005140 con 0x55cc7df70d60 2024-04-22T13:24:44.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.410+0000 7fc5b41df740 1 -- 172.21.15.12:0/699250916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc59400be20 con 0x55cc7df70d60 2024-04-22T13:24:44.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.410+0000 7fc5927fc640 1 -- 172.21.15.12:0/699250916 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc594005340 con 0x55cc7df70d60 2024-04-22T13:24:44.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.410+0000 7fc5b41df740 10 client.15918 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:44.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.410+0000 7fc5b41df740 1 -- 172.21.15.12:0/699250916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cc7dcb17d0 con 0x55cc7df70d60 2024-04-22T13:24:44.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.410+0000 7fc5927fc640 1 -- 172.21.15.12:0/699250916 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc59400be20 con 0x55cc7df70d60 2024-04-22T13:24:44.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.411+0000 7fc5927fc640 1 -- 172.21.15.12:0/699250916 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc594003960 con 0x55cc7df70d60 2024-04-22T13:24:44.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.411+0000 7fc5b41df740 10 client.15918 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:44.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.411+0000 7fc5b41df740 10 client.15918 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:24:44.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.411+0000 7fc5b41df740 20 client.15918 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:44.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.411+0000 7fc5b41df740 1 --2- 172.21.15.12:0/699250916 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55cc7df8df80 0x55cc7df90370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.411+0000 7fc5b41df740 4 client.15918 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:24:44.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.411+0000 7fc5b41df740 1 -- 172.21.15.12:0/699250916 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc7de74930 con 0x55cc7df8df80 2024-04-22T13:24:44.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.411+0000 7fc5aaffd640 1 --2- 172.21.15.12:0/699250916 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55cc7df8df80 0x55cc7df90370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.411+0000 7fc5aaffd640 1 --2- 172.21.15.12:0/699250916 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55cc7df8df80 0x55cc7df90370 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b6e6624ec2e9a0a3 server_cookie=4d6387054cde2e12 in_seq=0 out_seq=0 2024-04-22T13:24:44.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.411+0000 7fc5927fc640 10 client.15918 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:24:44.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.412+0000 7fc5927fc640 1 -- 172.21.15.12:0/699250916 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cc7de74930 con 0x55cc7df8df80 2024-04-22T13:24:44.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.412+0000 7fc5927fc640 10 client.15918 handle_command_reply: tid=0 2024-04-22T13:24:44.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.412+0000 7fc5b41df740 1 client.15918 shutdown 2024-04-22T13:24:44.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.412+0000 7fc5b41df740 1 -- 172.21.15.12:0/699250916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc7df70d60 msgr2=0x55cc7df7d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.412+0000 7fc5b41df740 1 --2- 172.21.15.12:0/699250916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc7df70d60 0x55cc7df7d620 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc594011010 tx=0x7fc5940042e0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.412+0000 7fc5b41df740 1 -- 172.21.15.12:0/699250916 shutdown_connections 2024-04-22T13:24:44.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.412+0000 7fc5b41df740 1 --2- 172.21.15.12:0/699250916 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55cc7df8df80 0x55cc7df90370 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.412+0000 7fc5b41df740 1 --2- 172.21.15.12:0/699250916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc7df79e60 0x55cc7df7db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.412+0000 7fc5b41df740 1 --2- 172.21.15.12:0/699250916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc7df70d60 0x55cc7df7d620 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.412+0000 7fc5927fc640 0 client.15918 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:24:44.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.412+0000 7fc5b41df740 1 --2- 172.21.15.12:0/699250916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc7df703b0 0x55cc7df80290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.412+0000 7fc5b41df740 1 -- 172.21.15.12:0/699250916 >> 172.21.15.12:0/699250916 conn(0x55cc7df6aba0 msgr2=0x55cc7df6c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:44.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.412+0000 7fc5b41df740 1 -- 172.21.15.12:0/699250916 shutdown_connections 2024-04-22T13:24:44.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.412+0000 7fc5b41df740 1 -- 172.21.15.12:0/699250916 wait complete. 2024-04-22T13:24:44.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.413+0000 7fc5b41df740 20 client.15918 trim_cache size 0 max 16384 2024-04-22T13:24:44.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.426+0000 7fc5b41df740 1 Processor -- start 2024-04-22T13:24:44.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.426+0000 7fc5b41df740 1 -- start start 2024-04-22T13:24:44.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.426+0000 7fc5b41df740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc7df703b0 0x55cc7df89ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.426+0000 7fc5b41df740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc7df70d60 0x55cc7df7fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.426+0000 7fc5b41df740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc7df79e60 0x55cc7df80290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.426+0000 7fc5b41df740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac1ab810 con 0x55cc7df703b0 2024-04-22T13:24:44.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.426+0000 7fc5b41df740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac1ad350 con 0x55cc7df79e60 2024-04-22T13:24:44.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.426+0000 7fc5b41df740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac1abee0 con 0x55cc7df70d60 2024-04-22T13:24:44.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.426+0000 7fc5aaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc7df703b0 0x55cc7df89ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.426+0000 7fc5aaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc7df703b0 0x55cc7df89ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50190/0 (socket says 172.21.15.12:50190) 2024-04-22T13:24:44.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.426+0000 7fc5aaffd640 1 -- 172.21.15.12:0/3108646707 learned_addr learned my addr 172.21.15.12:0/3108646707 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:44.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.426+0000 7fc5ab7fe640 1 --2- 172.21.15.12:0/3108646707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc7df79e60 0x55cc7df80290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.426+0000 7fc5aaffd640 1 -- 172.21.15.12:0/3108646707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc7df70d60 msgr2=0x55cc7df7fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.426+0000 7fc5aaffd640 1 --2- 172.21.15.12:0/3108646707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc7df70d60 0x55cc7df7fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.426+0000 7fc5aaffd640 1 -- 172.21.15.12:0/3108646707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc7df79e60 msgr2=0x55cc7df80290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.426+0000 7fc5aaffd640 1 --2- 172.21.15.12:0/3108646707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc7df79e60 0x55cc7df80290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.426+0000 7fc5aaffd640 1 -- 172.21.15.12:0/3108646707 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc7df6f580 con 0x55cc7df703b0 2024-04-22T13:24:44.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.426+0000 7fc5ab7fe640 1 --2- 172.21.15.12:0/3108646707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc7df79e60 0x55cc7df80290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:44.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.427+0000 7fc5aaffd640 1 --2- 172.21.15.12:0/3108646707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc7df703b0 0x55cc7df89ca0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc59c005ec0 tx=0x7fc59c091570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db5afc20f76bc72 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:44.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.427+0000 7fc5a0ff9640 1 -- 172.21.15.12:0/3108646707 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c0bf180 con 0x55cc7df703b0 2024-04-22T13:24:44.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.427+0000 7fc5a0ff9640 1 -- 172.21.15.12:0/3108646707 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc59c09be50 con 0x55cc7df703b0 2024-04-22T13:24:44.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.427+0000 7fc5a0ff9640 1 -- 172.21.15.12:0/3108646707 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c0c76f0 con 0x55cc7df703b0 2024-04-22T13:24:44.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.427+0000 7fc5b41df740 1 -- 172.21.15.12:0/3108646707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc7df703b0 msgr2=0x55cc7df89ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.427+0000 7fc5b41df740 1 --2- 172.21.15.12:0/3108646707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc7df703b0 0x55cc7df89ca0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc59c005ec0 tx=0x7fc59c091570 comp rx=0 tx=0).stop 2024-04-22T13:24:44.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.427+0000 7fc5b41df740 1 -- 172.21.15.12:0/3108646707 shutdown_connections 2024-04-22T13:24:44.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.427+0000 7fc5b41df740 1 --2- 172.21.15.12:0/3108646707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc7df79e60 0x55cc7df80290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.427+0000 7fc5b41df740 1 --2- 172.21.15.12:0/3108646707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc7df70d60 0x55cc7df7fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.427+0000 7fc5b41df740 1 --2- 172.21.15.12:0/3108646707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc7df703b0 0x55cc7df89ca0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.427+0000 7fc5b41df740 1 -- 172.21.15.12:0/3108646707 >> 172.21.15.12:0/3108646707 conn(0x55cc7df6aba0 msgr2=0x55cc7df69c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:44.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.428+0000 7fc5b41df740 1 -- 172.21.15.12:0/3108646707 shutdown_connections 2024-04-22T13:24:44.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.428+0000 7fc5b41df740 1 -- 172.21.15.12:0/3108646707 wait complete. 2024-04-22T13:24:44.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.428+0000 7fc5b41df740 1 -- start start 2024-04-22T13:24:44.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.428+0000 7fc5b41df740 1 Processor -- start 2024-04-22T13:24:44.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.428+0000 7fc5b41df740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc7df703b0 0x55cc7df7ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.428+0000 7fc5b41df740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc7df70d60 0x55cc7df7f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.428+0000 7fc5b41df740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc7df79e60 0x55cc7df851b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.428+0000 7fc5b41df740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac1acc80 con 0x55cc7df703b0 2024-04-22T13:24:44.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.428+0000 7fc5b41df740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac1ac5b0 con 0x55cc7df70d60 2024-04-22T13:24:44.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.428+0000 7fc5b41df740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac1a9810 con 0x55cc7df79e60 2024-04-22T13:24:44.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.428+0000 7fc5b41df740 10 client.0 _ll_register_callbacks cb 0x7fc59400b930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:44.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.428+0000 7fc5aaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc7df703b0 0x55cc7df7ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.428+0000 7fc5aaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc7df703b0 0x55cc7df7ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50206/0 (socket says 172.21.15.12:50206) 2024-04-22T13:24:44.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.428+0000 7fc5aaffd640 1 -- 172.21.15.12:0/2838126227 learned_addr learned my addr 172.21.15.12:0/2838126227 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:44.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.428+0000 7fc5a3fff640 1 --2- 172.21.15.12:0/2838126227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc7df70d60 0x55cc7df7f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.428+0000 7fc5aaffd640 1 -- 172.21.15.12:0/2838126227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc7df79e60 msgr2=0x55cc7df851b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.428+0000 7fc5aaffd640 1 --2- 172.21.15.12:0/2838126227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc7df79e60 0x55cc7df851b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.428+0000 7fc5aaffd640 1 -- 172.21.15.12:0/2838126227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc7df70d60 msgr2=0x55cc7df7f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.428+0000 7fc5aaffd640 1 --2- 172.21.15.12:0/2838126227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc7df70d60 0x55cc7df7f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.428+0000 7fc5aaffd640 1 -- 172.21.15.12:0/2838126227 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc7df85770 con 0x55cc7df703b0 2024-04-22T13:24:44.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.429+0000 7fc5aaffd640 1 --2- 172.21.15.12:0/2838126227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc7df703b0 0x55cc7df7ef10 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0be780 tx=0x7fc59c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad68de60ca52772 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:44.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.429+0000 7fc5927fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:44.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.429+0000 7fc5927fc640 1 -- 172.21.15.12:0/2838126227 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c0bf180 con 0x55cc7df703b0 2024-04-22T13:24:44.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.429+0000 7fc5927fc640 1 -- 172.21.15.12:0/2838126227 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fc59c0054d0 con 0x55cc7df703b0 2024-04-22T13:24:44.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.429+0000 7fc5b41df740 1 -- 172.21.15.12:0/2838126227 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc59c09be50 con 0x55cc7df703b0 2024-04-22T13:24:44.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.429+0000 7fc5927fc640 1 -- 172.21.15.12:0/2838126227 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc59c08d070 con 0x55cc7df703b0 2024-04-22T13:24:44.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.429+0000 7fc5b41df740 10 client.15924 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:44.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.429+0000 7fc5b41df740 1 -- 172.21.15.12:0/2838126227 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cc7df869e0 con 0x55cc7df703b0 2024-04-22T13:24:44.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.429+0000 7fc5927fc640 1 -- 172.21.15.12:0/2838126227 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc59c09be50 con 0x55cc7df703b0 2024-04-22T13:24:44.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.429+0000 7fc5927fc640 1 -- 172.21.15.12:0/2838126227 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc59c0f04d0 con 0x55cc7df703b0 2024-04-22T13:24:44.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.429+0000 7fc5b41df740 10 client.15924 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:44.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.429+0000 7fc5b41df740 10 client.15924 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:24:44.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.429+0000 7fc5b41df740 20 client.15924 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:44.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.429+0000 7fc5b41df740 1 --2- 172.21.15.12:0/2838126227 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55cc7dff4590 0x55cc7dff69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.429+0000 7fc5b41df740 4 client.15924 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:24:44.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.429+0000 7fc5b41df740 1 -- 172.21.15.12:0/2838126227 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cc7dee2c10 con 0x55cc7dff4590 2024-04-22T13:24:44.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.430+0000 7fc5a3fff640 1 --2- 172.21.15.12:0/2838126227 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55cc7dff4590 0x55cc7dff69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.430+0000 7fc5a3fff640 1 --2- 172.21.15.12:0/2838126227 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55cc7dff4590 0x55cc7dff69b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a247d00b283a77d4 server_cookie=8ecb16d33365b50 in_seq=0 out_seq=0 2024-04-22T13:24:44.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.430+0000 7fc5927fc640 10 client.15924 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:24:44.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.430+0000 7fc5927fc640 1 -- 172.21.15.12:0/2838126227 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cc7dee2c10 con 0x55cc7dff4590 2024-04-22T13:24:44.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.430+0000 7fc5927fc640 10 client.15924 handle_command_reply: tid=0 2024-04-22T13:24:44.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.430+0000 7fc5b41df740 1 client.15924 shutdown 2024-04-22T13:24:44.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.430+0000 7fc5b41df740 1 -- 172.21.15.12:0/2838126227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc7df703b0 msgr2=0x55cc7df7ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.430+0000 7fc5b41df740 1 --2- 172.21.15.12:0/2838126227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc7df703b0 0x55cc7df7ef10 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0be780 tx=0x7fc59c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:44.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.430+0000 7fc5b41df740 1 -- 172.21.15.12:0/2838126227 shutdown_connections 2024-04-22T13:24:44.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.430+0000 7fc5b41df740 1 --2- 172.21.15.12:0/2838126227 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55cc7dff4590 0x55cc7dff69b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.430+0000 7fc5b41df740 1 --2- 172.21.15.12:0/2838126227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc7df79e60 0x55cc7df851b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.430+0000 7fc5b41df740 1 --2- 172.21.15.12:0/2838126227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc7df70d60 0x55cc7df7f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.430+0000 7fc5927fc640 0 client.15924 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:24:44.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.430+0000 7fc5b41df740 1 --2- 172.21.15.12:0/2838126227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cc7df703b0 0x55cc7df7ef10 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.430+0000 7fc5b41df740 1 -- 172.21.15.12:0/2838126227 >> 172.21.15.12:0/2838126227 conn(0x55cc7df6aba0 msgr2=0x55cc7df69370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:44.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.430+0000 7fc5b41df740 1 -- 172.21.15.12:0/2838126227 shutdown_connections 2024-04-22T13:24:44.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.430+0000 7fc5b41df740 1 -- 172.21.15.12:0/2838126227 wait complete. 2024-04-22T13:24:44.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.430+0000 7fc5b41df740 20 client.15924 trim_cache size 0 max 16384 2024-04-22T13:24:44.433 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:24:44.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.431+0000 7fc5b151e640 1 -- 172.21.15.12:0/475351424 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc5980a3e40 msgr2=0x7fc5980a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.431+0000 7fc5b151e640 1 --2- 172.21.15.12:0/475351424 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc5980a3e40 0x7fc5980a6300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4013b10 tx=0x7fc5a4004ab0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.431+0000 7fc5b151e640 1 -- 172.21.15.12:0/475351424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5ac06a840 msgr2=0x7fc5ac1986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.431+0000 7fc5b151e640 1 --2- 172.21.15.12:0/475351424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5ac06a840 0x7fc5ac1986f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc59c005940 tx=0x7fc59c02c7e0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.431+0000 7fc5b151e640 1 -- 172.21.15.12:0/475351424 shutdown_connections 2024-04-22T13:24:44.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.431+0000 7fc5b151e640 1 --2- 172.21.15.12:0/475351424 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc5980a3e40 0x7fc5980a6300 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.431+0000 7fc5b151e640 1 --2- 172.21.15.12:0/475351424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5ac19cf60 0x7fc5ac19d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.431+0000 7fc5b151e640 1 --2- 172.21.15.12:0/475351424 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5ac107270 0x7fc5ac198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.431+0000 7fc5b151e640 1 --2- 172.21.15.12:0/475351424 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5ac06a840 0x7fc5ac1986f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.431+0000 7fc5b151e640 1 -- 172.21.15.12:0/475351424 >> 172.21.15.12:0/475351424 conn(0x7fc5ac077eb0 msgr2=0x7fc5ac100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:44.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.431+0000 7fc5b151e640 1 -- 172.21.15.12:0/475351424 shutdown_connections 2024-04-22T13:24:44.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.431+0000 7fc5b151e640 1 -- 172.21.15.12:0/475351424 wait complete. 2024-04-22T13:24:44.438 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:24:44.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.546+0000 7f5fb15b2640 1 -- 172.21.15.12:0/3995615591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fac106160 msgr2=0x7f5fac10dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.546+0000 7f5fb15b2640 1 --2- 172.21.15.12:0/3995615591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fac106160 0x7f5fac10dd30 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009ca0 tx=0x7f5f9c02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.546+0000 7f5fb15b2640 1 -- 172.21.15.12:0/3995615591 shutdown_connections 2024-04-22T13:24:44.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.546+0000 7f5fb15b2640 1 --2- 172.21.15.12:0/3995615591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fac106160 0x7f5fac10dd30 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.546+0000 7f5fb15b2640 1 --2- 172.21.15.12:0/3995615591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fac101a40 0x7f5fac105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.546+0000 7f5fb15b2640 1 --2- 172.21.15.12:0/3995615591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fac101090 0x7f5fac101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.546+0000 7f5fb15b2640 1 -- 172.21.15.12:0/3995615591 >> 172.21.15.12:0/3995615591 conn(0x7f5fac077f50 msgr2=0x7f5fac078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:44.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.546+0000 7f5fb15b2640 1 -- 172.21.15.12:0/3995615591 shutdown_connections 2024-04-22T13:24:44.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.546+0000 7f5fb15b2640 1 -- 172.21.15.12:0/3995615591 wait complete. 2024-04-22T13:24:44.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.547+0000 7f5fb15b2640 1 Processor -- start 2024-04-22T13:24:44.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.547+0000 7f5fb15b2640 1 -- start start 2024-04-22T13:24:44.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.547+0000 7f5fb15b2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fac101090 0x7f5fac102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.547+0000 7f5fb15b2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fac106160 0x7f5fac103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.547+0000 7f5fb15b2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fac1035a0 0x7f5fac1a68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.547+0000 7f5fb15b2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac111120 con 0x7f5fac101090 2024-04-22T13:24:44.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.547+0000 7f5fb15b2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac110fa0 con 0x7f5fac1035a0 2024-04-22T13:24:44.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.547+0000 7f5fb15b2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac1112a0 con 0x7f5fac106160 2024-04-22T13:24:44.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.547+0000 7f5faaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fac101090 0x7f5fac102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.547+0000 7f5faaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fac101090 0x7f5fac102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50232/0 (socket says 172.21.15.12:50232) 2024-04-22T13:24:44.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.547+0000 7f5faaffd640 1 -- 172.21.15.12:0/1489053494 learned_addr learned my addr 172.21.15.12:0/1489053494 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:44.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.547+0000 7f5faa7fc640 1 --2- 172.21.15.12:0/1489053494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fac106160 0x7f5fac103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.547+0000 7f5fab7fe640 1 --2- 172.21.15.12:0/1489053494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fac1035a0 0x7f5fac1a68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.547+0000 7f5faaffd640 1 -- 172.21.15.12:0/1489053494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fac106160 msgr2=0x7f5fac103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.547+0000 7f5faaffd640 1 --2- 172.21.15.12:0/1489053494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fac106160 0x7f5fac103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.547+0000 7f5faaffd640 1 -- 172.21.15.12:0/1489053494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fac1035a0 msgr2=0x7f5fac1a68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.547+0000 7f5faaffd640 1 --2- 172.21.15.12:0/1489053494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fac1035a0 0x7f5fac1a68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.547+0000 7f5faaffd640 1 -- 172.21.15.12:0/1489053494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fac1a6fd0 con 0x7f5fac101090 2024-04-22T13:24:44.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.547+0000 7f5fab7fe640 1 --2- 172.21.15.12:0/1489053494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fac1035a0 0x7f5fac1a68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:44.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.547+0000 7f5faa7fc640 1 --2- 172.21.15.12:0/1489053494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fac106160 0x7f5fac103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:44.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.548+0000 7f5faaffd640 1 --2- 172.21.15.12:0/1489053494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fac101090 0x7f5fac102b20 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400c920 tx=0x7f5fa400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:44.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.548+0000 7f5fa3fff640 1 -- 172.21.15.12:0/1489053494 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4007c70 con 0x7f5fac101090 2024-04-22T13:24:44.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.548+0000 7f5fb15b2640 1 -- 172.21.15.12:0/1489053494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fac1a72c0 con 0x7f5fac101090 2024-04-22T13:24:44.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.548+0000 7f5fa3fff640 1 -- 172.21.15.12:0/1489053494 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fa4007e10 con 0x7f5fac101090 2024-04-22T13:24:44.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.548+0000 7f5fa3fff640 1 -- 172.21.15.12:0/1489053494 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4005220 con 0x7f5fac101090 2024-04-22T13:24:44.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.548+0000 7f5fb15b2640 1 -- 172.21.15.12:0/1489053494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fac1a7810 con 0x7f5fac101090 2024-04-22T13:24:44.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.550+0000 7f5fa3fff640 1 -- 172.21.15.12:0/1489053494 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f5fa40053c0 con 0x7f5fac101090 2024-04-22T13:24:44.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.550+0000 7f5fb41a0740 1 Processor -- start 2024-04-22T13:24:44.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.550+0000 7f5fb41a0740 1 -- start start 2024-04-22T13:24:44.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.551+0000 7f5fb41a0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf5fa9d4d0 0x55cf5fa9d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.551+0000 7f5fb41a0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5fa9de80 0x55cf5faa6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.551+0000 7f5fb41a0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf5faa6a00 0x55cf5faa8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.551+0000 7f5fb41a0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55cf5fa021f0 con 0x55cf5faa6a00 2024-04-22T13:24:44.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.551+0000 7f5fb41a0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55cf5f99f4b0 con 0x55cf5fa9de80 2024-04-22T13:24:44.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.551+0000 7f5fb41a0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cf5f9fe1b0 con 0x55cf5fa9d4d0 2024-04-22T13:24:44.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.551+0000 7f5fa3fff640 1 --2- 172.21.15.12:0/1489053494 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5f980a3e70 0x7f5f980a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.551+0000 7f5faa7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf5faa6a00 0x55cf5faa8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.551+0000 7f5faa7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf5faa6a00 0x55cf5faa8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50246/0 (socket says 172.21.15.12:50246) 2024-04-22T13:24:44.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.551+0000 7f5faa7fc640 1 -- 172.21.15.12:0/2556290048 learned_addr learned my addr 172.21.15.12:0/2556290048 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.551+0000 7f5faaffd640 1 --2- 172.21.15.12:0/2556290048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5fa9de80 0x55cf5faa6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.551+0000 7f5fab7fe640 1 --2- 172.21.15.12:0/1489053494 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5f980a3e70 0x7f5f980a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.551+0000 7f5fa3fff640 1 -- 172.21.15.12:0/1489053494 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f5fa40d0c10 con 0x7f5fac101090 2024-04-22T13:24:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.551+0000 7f5fab7fe640 1 --2- 172.21.15.12:0/2556290048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf5fa9d4d0 0x55cf5fa9d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.551+0000 7f5faa7fc640 1 -- 172.21.15.12:0/2556290048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf5fa9d4d0 msgr2=0x55cf5fa9d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.551+0000 7f5faa7fc640 1 --2- 172.21.15.12:0/2556290048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf5fa9d4d0 0x55cf5fa9d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.551+0000 7f5faa7fc640 1 -- 172.21.15.12:0/2556290048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5fa9de80 msgr2=0x55cf5faa6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.551+0000 7f5faa7fc640 1 --2- 172.21.15.12:0/2556290048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5fa9de80 0x55cf5faa6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.551+0000 7f5faa7fc640 1 -- 172.21.15.12:0/2556290048 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf5f9911b0 con 0x55cf5faa6a00 2024-04-22T13:24:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.551+0000 7f5faa7fc640 1 --2- 172.21.15.12:0/2556290048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf5faa6a00 0x55cf5faa8e50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f5f9400a9a0 tx=0x7f5f9400ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305d040593f3740 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.552+0000 7f5fab7fe640 1 --2- 172.21.15.12:0/1489053494 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5f980a3e70 0x7f5f980a6330 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c005e30 tx=0x7f5f9c004a80 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.552+0000 7f5fa17fa640 1 -- 172.21.15.12:0/2556290048 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f94015070 con 0x55cf5faa6a00 2024-04-22T13:24:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.552+0000 7f5fa17fa640 1 -- 172.21.15.12:0/2556290048 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f9400cc00 con 0x55cf5faa6a00 2024-04-22T13:24:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.552+0000 7f5fa17fa640 1 -- 172.21.15.12:0/2556290048 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f940105b0 con 0x55cf5faa6a00 2024-04-22T13:24:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.552+0000 7f5fb41a0740 1 -- 172.21.15.12:0/2556290048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf5faa6a00 msgr2=0x55cf5faa8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.552+0000 7f5fb41a0740 1 --2- 172.21.15.12:0/2556290048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf5faa6a00 0x55cf5faa8e50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f5f9400a9a0 tx=0x7f5f9400ae60 comp rx=0 tx=0).stop 2024-04-22T13:24:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.552+0000 7f5fb41a0740 1 -- 172.21.15.12:0/2556290048 shutdown_connections 2024-04-22T13:24:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.552+0000 7f5fb41a0740 1 --2- 172.21.15.12:0/2556290048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf5faa6a00 0x55cf5faa8e50 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.552+0000 7f5fb41a0740 1 --2- 172.21.15.12:0/2556290048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5fa9de80 0x55cf5faa6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.552+0000 7f5fb41a0740 1 --2- 172.21.15.12:0/2556290048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf5fa9d4d0 0x55cf5fa9d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.552+0000 7f5fb41a0740 1 -- 172.21.15.12:0/2556290048 >> 172.21.15.12:0/2556290048 conn(0x55cf5fa97ba0 msgr2=0x55cf5fa97f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.552+0000 7f5fb41a0740 1 -- 172.21.15.12:0/2556290048 shutdown_connections 2024-04-22T13:24:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.553+0000 7f5fb41a0740 1 -- 172.21.15.12:0/2556290048 wait complete. 2024-04-22T13:24:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.553+0000 7f5fb41a0740 1 -- start start 2024-04-22T13:24:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.553+0000 7f5fb41a0740 1 Processor -- start 2024-04-22T13:24:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.553+0000 7f5fb41a0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf5fa9d4d0 0x55cf5faa62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.553+0000 7f5fb41a0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5fa9de80 0x55cf5faa3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.553+0000 7f5fb41a0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf5faa6a00 0x55cf5faa3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.553+0000 7f5fb41a0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55cf5faa9970 con 0x55cf5fa9d4d0 2024-04-22T13:24:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.553+0000 7f5fb41a0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55cf5f9bb500 con 0x55cf5fa9de80 2024-04-22T13:24:44.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.553+0000 7f5fb41a0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cf5faa9af0 con 0x55cf5faa6a00 2024-04-22T13:24:44.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.553+0000 7f5faaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf5fa9d4d0 0x55cf5faa62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.553+0000 7f5fb41a0740 10 client.0 _ll_register_callbacks cb 0x55cf5f989b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:44.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.553+0000 7f5faaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf5fa9d4d0 0x55cf5faa62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50250/0 (socket says 172.21.15.12:50250) 2024-04-22T13:24:44.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.553+0000 7f5faaffd640 1 -- 172.21.15.12:0/640281660 learned_addr learned my addr 172.21.15.12:0/640281660 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:44.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.553+0000 7f5fab7fe640 1 --2- 172.21.15.12:0/640281660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf5faa6a00 0x55cf5faa3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.553+0000 7f5faa7fc640 1 --2- 172.21.15.12:0/640281660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5fa9de80 0x55cf5faa3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.553+0000 7f5faaffd640 1 -- 172.21.15.12:0/640281660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf5faa6a00 msgr2=0x55cf5faa3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.554+0000 7f5faaffd640 1 --2- 172.21.15.12:0/640281660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf5faa6a00 0x55cf5faa3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.554+0000 7f5faaffd640 1 -- 172.21.15.12:0/640281660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5fa9de80 msgr2=0x55cf5faa3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.554+0000 7f5faaffd640 1 --2- 172.21.15.12:0/640281660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5fa9de80 0x55cf5faa3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.554+0000 7f5faaffd640 1 -- 172.21.15.12:0/640281660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf5f7143f0 con 0x55cf5fa9d4d0 2024-04-22T13:24:44.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.554+0000 7f5faaffd640 1 --2- 172.21.15.12:0/640281660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf5fa9d4d0 0x55cf5faa62b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4000c00 tx=0x7f5fa400fe60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3971dec1e17f1f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:44.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.554+0000 7f5f927fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:44.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.554+0000 7f5f927fc640 1 -- 172.21.15.12:0/640281660 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa409c9b0 con 0x55cf5fa9d4d0 2024-04-22T13:24:44.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.554+0000 7f5f927fc640 1 -- 172.21.15.12:0/640281660 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f5fa40a5190 con 0x55cf5fa9d4d0 2024-04-22T13:24:44.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.554+0000 7f5fb41a0740 1 -- 172.21.15.12:0/640281660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f9400cc00 con 0x55cf5fa9d4d0 2024-04-22T13:24:44.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.554+0000 7f5f927fc640 1 -- 172.21.15.12:0/640281660 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5fa40a0d60 con 0x55cf5fa9d4d0 2024-04-22T13:24:44.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.554+0000 7f5fb41a0740 10 client.15936 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:44.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.554+0000 7f5fb41a0740 1 -- 172.21.15.12:0/640281660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cf5f7de7d0 con 0x55cf5fa9d4d0 2024-04-22T13:24:44.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.554+0000 7f5f927fc640 1 -- 172.21.15.12:0/640281660 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f9400cc00 con 0x55cf5fa9d4d0 2024-04-22T13:24:44.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.554+0000 7f5f927fc640 1 -- 172.21.15.12:0/640281660 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5fa409dce0 con 0x55cf5fa9d4d0 2024-04-22T13:24:44.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.554+0000 7f5fb41a0740 10 client.15936 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:44.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.554+0000 7f5fb41a0740 10 client.15936 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:24:44.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.554+0000 7f5fb41a0740 20 client.15936 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:44.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.554+0000 7f5fb41a0740 1 --2- 172.21.15.12:0/640281660 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55cf5fab6bb0 0x55cf5fab9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.554+0000 7f5fb41a0740 4 client.15936 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:24:44.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.554+0000 7f5fb41a0740 1 -- 172.21.15.12:0/640281660 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf5f9a1930 con 0x55cf5fab6bb0 2024-04-22T13:24:44.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.555+0000 7f5faa7fc640 1 --2- 172.21.15.12:0/640281660 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55cf5fab6bb0 0x55cf5fab9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.555+0000 7f5faa7fc640 1 --2- 172.21.15.12:0/640281660 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55cf5fab6bb0 0x55cf5fab9020 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=83d0f412074f5e4 server_cookie=cc6eaa57b92d319 in_seq=0 out_seq=0 2024-04-22T13:24:44.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.555+0000 7f5f927fc640 10 client.15936 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:24:44.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.556+0000 7f5f927fc640 1 -- 172.21.15.12:0/640281660 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cf5f9a1930 con 0x55cf5fab6bb0 2024-04-22T13:24:44.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.556+0000 7f5f927fc640 10 client.15936 handle_command_reply: tid=0 2024-04-22T13:24:44.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.556+0000 7f5fb41a0740 1 client.15936 shutdown 2024-04-22T13:24:44.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.556+0000 7f5fb41a0740 1 -- 172.21.15.12:0/640281660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf5fa9d4d0 msgr2=0x55cf5faa62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.556+0000 7f5fb41a0740 1 --2- 172.21.15.12:0/640281660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf5fa9d4d0 0x55cf5faa62b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4000c00 tx=0x7f5fa400fe60 comp rx=0 tx=0).stop 2024-04-22T13:24:44.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.556+0000 7f5fb41a0740 1 -- 172.21.15.12:0/640281660 shutdown_connections 2024-04-22T13:24:44.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.556+0000 7f5fb41a0740 1 --2- 172.21.15.12:0/640281660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf5faa6a00 0x55cf5faa3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.556+0000 7f5fb41a0740 1 --2- 172.21.15.12:0/640281660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5fa9de80 0x55cf5faa3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.556+0000 7f5fb41a0740 1 --2- 172.21.15.12:0/640281660 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55cf5fab6bb0 0x55cf5fab9020 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.556+0000 7f5fb41a0740 1 --2- 172.21.15.12:0/640281660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf5fa9d4d0 0x55cf5faa62b0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.556+0000 7f5f927fc640 0 client.15936 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:24:44.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.556+0000 7f5fb41a0740 1 -- 172.21.15.12:0/640281660 >> 172.21.15.12:0/640281660 conn(0x55cf5fa97ba0 msgr2=0x55cf5fa998f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:44.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.556+0000 7f5fb41a0740 1 -- 172.21.15.12:0/640281660 shutdown_connections 2024-04-22T13:24:44.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.556+0000 7f5fb41a0740 1 -- 172.21.15.12:0/640281660 wait complete. 2024-04-22T13:24:44.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.556+0000 7f5fb41a0740 20 client.15936 trim_cache size 0 max 16384 2024-04-22T13:24:44.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.569+0000 7f5fb41a0740 1 Processor -- start 2024-04-22T13:24:44.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.569+0000 7f5fb41a0740 1 -- start start 2024-04-22T13:24:44.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.570+0000 7f5fb41a0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf5fa9d4d0 0x55cf5fab2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.570+0000 7f5fb41a0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5fa9de80 0x55cf5fa98f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.570+0000 7f5fb41a0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf5faa6a00 0x55cf5faa5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.570+0000 7f5fb41a0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac1afb50 con 0x55cf5fa9d4d0 2024-04-22T13:24:44.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.570+0000 7f5fb41a0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac1b1690 con 0x55cf5fa9de80 2024-04-22T13:24:44.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.570+0000 7f5fb41a0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac1b0220 con 0x55cf5faa6a00 2024-04-22T13:24:44.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.570+0000 7f5faaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf5fa9d4d0 0x55cf5fab2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.570+0000 7f5faaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf5fa9d4d0 0x55cf5fab2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50252/0 (socket says 172.21.15.12:50252) 2024-04-22T13:24:44.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.570+0000 7f5faaffd640 1 -- 172.21.15.12:0/2658378015 learned_addr learned my addr 172.21.15.12:0/2658378015 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:44.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.570+0000 7f5faa7fc640 1 --2- 172.21.15.12:0/2658378015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5fa9de80 0x55cf5fa98f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.570+0000 7f5fab7fe640 1 --2- 172.21.15.12:0/2658378015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf5faa6a00 0x55cf5faa5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.570+0000 7f5faaffd640 1 -- 172.21.15.12:0/2658378015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf5faa6a00 msgr2=0x55cf5faa5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.570+0000 7f5faaffd640 1 --2- 172.21.15.12:0/2658378015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf5faa6a00 0x55cf5faa5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.570+0000 7f5faaffd640 1 -- 172.21.15.12:0/2658378015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5fa9de80 msgr2=0x55cf5fa98f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.570+0000 7f5faaffd640 1 --2- 172.21.15.12:0/2658378015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5fa9de80 0x55cf5fa98f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.570+0000 7f5faaffd640 1 -- 172.21.15.12:0/2658378015 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf5faa1b90 con 0x55cf5fa9d4d0 2024-04-22T13:24:44.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.570+0000 7f5faaffd640 1 --2- 172.21.15.12:0/2658378015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf5fa9d4d0 0x55cf5fab2830 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40ce9b0 tx=0x7f5fa4013e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=674309c5a76295c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:44.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.570+0000 7f5fa0ff9640 1 -- 172.21.15.12:0/2658378015 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa40ce170 con 0x55cf5fa9d4d0 2024-04-22T13:24:44.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.571+0000 7f5fa0ff9640 1 -- 172.21.15.12:0/2658378015 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fa409c9b0 con 0x55cf5fa9d4d0 2024-04-22T13:24:44.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.571+0000 7f5fa0ff9640 1 -- 172.21.15.12:0/2658378015 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa40972e0 con 0x55cf5fa9d4d0 2024-04-22T13:24:44.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.571+0000 7f5fb41a0740 1 -- 172.21.15.12:0/2658378015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf5fa9d4d0 msgr2=0x55cf5fab2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.571+0000 7f5fb41a0740 1 --2- 172.21.15.12:0/2658378015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf5fa9d4d0 0x55cf5fab2830 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40ce9b0 tx=0x7f5fa4013e00 comp rx=0 tx=0).stop 2024-04-22T13:24:44.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.571+0000 7f5fb41a0740 1 -- 172.21.15.12:0/2658378015 shutdown_connections 2024-04-22T13:24:44.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.571+0000 7f5fb41a0740 1 --2- 172.21.15.12:0/2658378015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf5faa6a00 0x55cf5faa5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.571+0000 7f5fb41a0740 1 --2- 172.21.15.12:0/2658378015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5fa9de80 0x55cf5fa98f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.571+0000 7f5fb41a0740 1 --2- 172.21.15.12:0/2658378015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf5fa9d4d0 0x55cf5fab2830 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.571+0000 7f5fb41a0740 1 -- 172.21.15.12:0/2658378015 >> 172.21.15.12:0/2658378015 conn(0x55cf5fa97ba0 msgr2=0x55cf5fa96c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:44.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.571+0000 7f5fb41a0740 1 -- 172.21.15.12:0/2658378015 shutdown_connections 2024-04-22T13:24:44.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.571+0000 7f5fb41a0740 1 -- 172.21.15.12:0/2658378015 wait complete. 2024-04-22T13:24:44.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.572+0000 7f5fb41a0740 1 -- start start 2024-04-22T13:24:44.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.572+0000 7f5fb41a0740 1 Processor -- start 2024-04-22T13:24:44.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.572+0000 7f5fb41a0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf5fa9d4d0 0x55cf5faaef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.572+0000 7f5fb41a0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf5fa9de80 0x55cf5fa9fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.572+0000 7f5fb41a0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5faa6a00 0x55cf5faa03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.572+0000 7f5fb41a0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac1b0fc0 con 0x55cf5fa9d4d0 2024-04-22T13:24:44.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.572+0000 7f5fb41a0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac1b08f0 con 0x55cf5faa6a00 2024-04-22T13:24:44.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.572+0000 7f5fb41a0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac1adb30 con 0x55cf5fa9de80 2024-04-22T13:24:44.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.572+0000 7f5fb41a0740 10 client.0 _ll_register_callbacks cb 0x7f5fa409c570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:44.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.572+0000 7f5faaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf5fa9d4d0 0x55cf5faaef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.572+0000 7f5faaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf5fa9d4d0 0x55cf5faaef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50268/0 (socket says 172.21.15.12:50268) 2024-04-22T13:24:44.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.572+0000 7f5faaffd640 1 -- 172.21.15.12:0/2161734222 learned_addr learned my addr 172.21.15.12:0/2161734222 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:44.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.572+0000 7f5faa7fc640 1 --2- 172.21.15.12:0/2161734222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf5fa9de80 0x55cf5fa9fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.572+0000 7f5fab7fe640 1 --2- 172.21.15.12:0/2161734222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5faa6a00 0x55cf5faa03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.572+0000 7f5faaffd640 1 -- 172.21.15.12:0/2161734222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf5fa9de80 msgr2=0x55cf5fa9fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.572+0000 7f5faaffd640 1 --2- 172.21.15.12:0/2161734222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf5fa9de80 0x55cf5fa9fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.572+0000 7f5faaffd640 1 -- 172.21.15.12:0/2161734222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5faa6a00 msgr2=0x55cf5faa03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.572+0000 7f5faaffd640 1 --2- 172.21.15.12:0/2161734222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5faa6a00 0x55cf5faa03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.572+0000 7f5faaffd640 1 -- 172.21.15.12:0/2161734222 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf5faa0a10 con 0x55cf5fa9d4d0 2024-04-22T13:24:44.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.572+0000 7f5faaffd640 1 --2- 172.21.15.12:0/2161734222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf5fa9d4d0 0x55cf5faaef30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5fa409c940 tx=0x7f5fa40ce8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=740026049b810a08 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:44.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.572+0000 7f5f927fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:44.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.573+0000 7f5f927fc640 1 -- 172.21.15.12:0/2161734222 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4067050 con 0x55cf5fa9d4d0 2024-04-22T13:24:44.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.573+0000 7f5fb41a0740 1 -- 172.21.15.12:0/2161734222 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fa409c9b0 con 0x55cf5fa9d4d0 2024-04-22T13:24:44.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.573+0000 7f5f927fc640 1 -- 172.21.15.12:0/2161734222 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f5fa4097280 con 0x55cf5fa9d4d0 2024-04-22T13:24:44.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.573+0000 7f5f927fc640 1 -- 172.21.15.12:0/2161734222 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5fa40ce170 con 0x55cf5fa9d4d0 2024-04-22T13:24:44.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.573+0000 7f5fb41a0740 10 client.15942 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:44.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.573+0000 7f5fb41a0740 1 -- 172.21.15.12:0/2161734222 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cf5fb1cf30 con 0x55cf5fa9d4d0 2024-04-22T13:24:44.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.573+0000 7f5f927fc640 1 -- 172.21.15.12:0/2161734222 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fa409c9b0 con 0x55cf5fa9d4d0 2024-04-22T13:24:44.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.573+0000 7f5f927fc640 1 -- 172.21.15.12:0/2161734222 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5fa404d060 con 0x55cf5fa9d4d0 2024-04-22T13:24:44.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.573+0000 7f5fb41a0740 10 client.15942 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:44.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.573+0000 7f5fb41a0740 10 client.15942 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:24:44.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.573+0000 7f5fb41a0740 20 client.15942 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:44.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.573+0000 7f5fb41a0740 1 --2- 172.21.15.12:0/2161734222 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55cf5fb1d400 0x55cf5fb1f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.573+0000 7f5fb41a0740 4 client.15942 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:24:44.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.573+0000 7f5fb41a0740 1 -- 172.21.15.12:0/2161734222 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cf5fa0fc10 con 0x55cf5fb1d400 2024-04-22T13:24:44.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.573+0000 7f5faa7fc640 1 --2- 172.21.15.12:0/2161734222 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55cf5fb1d400 0x55cf5fb1f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.574+0000 7f5faa7fc640 1 --2- 172.21.15.12:0/2161734222 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55cf5fb1d400 0x55cf5fb1f7f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f6087e6c7729df5e server_cookie=8095f452e76126c9 in_seq=0 out_seq=0 2024-04-22T13:24:44.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.574+0000 7f5f927fc640 10 client.15942 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:24:44.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.574+0000 7f5f927fc640 1 -- 172.21.15.12:0/2161734222 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cf5fa0fc10 con 0x55cf5fb1d400 2024-04-22T13:24:44.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.574+0000 7f5f927fc640 10 client.15942 handle_command_reply: tid=0 2024-04-22T13:24:44.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.574+0000 7f5fb41a0740 1 client.15942 shutdown 2024-04-22T13:24:44.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.574+0000 7f5fb41a0740 1 -- 172.21.15.12:0/2161734222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf5fa9d4d0 msgr2=0x55cf5faaef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.574+0000 7f5fb41a0740 1 --2- 172.21.15.12:0/2161734222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf5fa9d4d0 0x55cf5faaef30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5fa409c940 tx=0x7f5fa40ce8f0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.574+0000 7f5fb41a0740 1 -- 172.21.15.12:0/2161734222 shutdown_connections 2024-04-22T13:24:44.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.574+0000 7f5fb41a0740 1 --2- 172.21.15.12:0/2161734222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf5faa6a00 0x55cf5faa03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.574+0000 7f5fb41a0740 1 --2- 172.21.15.12:0/2161734222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf5fa9de80 0x55cf5fa9fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.574+0000 7f5fb41a0740 1 --2- 172.21.15.12:0/2161734222 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55cf5fb1d400 0x55cf5fb1f7f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.574+0000 7f5fb41a0740 1 --2- 172.21.15.12:0/2161734222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf5fa9d4d0 0x55cf5faaef30 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.574+0000 7f5f927fc640 0 client.15942 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:24:44.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.574+0000 7f5fb41a0740 1 -- 172.21.15.12:0/2161734222 >> 172.21.15.12:0/2161734222 conn(0x55cf5fa97ba0 msgr2=0x55cf5fa9e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:44.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.574+0000 7f5fb41a0740 1 -- 172.21.15.12:0/2161734222 shutdown_connections 2024-04-22T13:24:44.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.574+0000 7f5fb41a0740 1 -- 172.21.15.12:0/2161734222 wait complete. 2024-04-22T13:24:44.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.574+0000 7f5fb41a0740 20 client.15942 trim_cache size 0 max 16384 2024-04-22T13:24:44.577 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:24:44.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.575+0000 7f5fb15b2640 1 -- 172.21.15.12:0/1489053494 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5f980a3e70 msgr2=0x7f5f980a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.575+0000 7f5fb15b2640 1 --2- 172.21.15.12:0/1489053494 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5f980a3e70 0x7f5f980a6330 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c005e30 tx=0x7f5f9c004a80 comp rx=0 tx=0).stop 2024-04-22T13:24:44.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.575+0000 7f5fb15b2640 1 -- 172.21.15.12:0/1489053494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fac101090 msgr2=0x7f5fac102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.575+0000 7f5fb15b2640 1 --2- 172.21.15.12:0/1489053494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fac101090 0x7f5fac102b20 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400c920 tx=0x7f5fa400cde0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.575+0000 7f5fb15b2640 1 -- 172.21.15.12:0/1489053494 shutdown_connections 2024-04-22T13:24:44.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.575+0000 7f5fb15b2640 1 --2- 172.21.15.12:0/1489053494 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5f980a3e70 0x7f5f980a6330 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.575+0000 7f5fb15b2640 1 --2- 172.21.15.12:0/1489053494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fac1035a0 0x7f5fac1a68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.575+0000 7f5fb15b2640 1 --2- 172.21.15.12:0/1489053494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fac106160 0x7f5fac103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.575+0000 7f5fb15b2640 1 --2- 172.21.15.12:0/1489053494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fac101090 0x7f5fac102b20 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.575+0000 7f5fb15b2640 1 -- 172.21.15.12:0/1489053494 >> 172.21.15.12:0/1489053494 conn(0x7f5fac077f50 msgr2=0x7f5fac069460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:44.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.575+0000 7f5fb15b2640 1 -- 172.21.15.12:0/1489053494 shutdown_connections 2024-04-22T13:24:44.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.575+0000 7f5fb15b2640 1 -- 172.21.15.12:0/1489053494 wait complete. 2024-04-22T13:24:44.582 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:24:44.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.689+0000 7fcddec4e640 1 -- 172.21.15.12:0/1179017790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdd8102b40 msgr2=0x7fcdd8106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.689+0000 7fcddec4e640 1 --2- 172.21.15.12:0/1179017790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdd8102b40 0x7fcdd8106b80 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009900 tx=0x7fcdc802bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:44.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.689+0000 7fcddec4e640 1 -- 172.21.15.12:0/1179017790 shutdown_connections 2024-04-22T13:24:44.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.689+0000 7fcddec4e640 1 --2- 172.21.15.12:0/1179017790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdd8107260 0x7fcdd810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.689+0000 7fcddec4e640 1 --2- 172.21.15.12:0/1179017790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdd8102b40 0x7fcdd8106b80 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.689+0000 7fcddec4e640 1 --2- 172.21.15.12:0/1179017790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdd8102190 0x7fcdd8102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.689+0000 7fcddec4e640 1 -- 172.21.15.12:0/1179017790 >> 172.21.15.12:0/1179017790 conn(0x7fcdd80fd990 msgr2=0x7fcdd80ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:44.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.689+0000 7fcddec4e640 1 -- 172.21.15.12:0/1179017790 shutdown_connections 2024-04-22T13:24:44.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.689+0000 7fcddec4e640 1 -- 172.21.15.12:0/1179017790 wait complete. 2024-04-22T13:24:44.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.690+0000 7fcddec4e640 1 Processor -- start 2024-04-22T13:24:44.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.690+0000 7fcddec4e640 1 -- start start 2024-04-22T13:24:44.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.690+0000 7fcddec4e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdd8102190 0x7fcdd8198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.690+0000 7fcddec4e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdd8107260 0x7fcdd8198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.690+0000 7fcddec4e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdd819d020 0x7fcdd819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.690+0000 7fcddec4e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd8112180 con 0x7fcdd8107260 2024-04-22T13:24:44.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.690+0000 7fcddec4e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd8112000 con 0x7fcdd819d020 2024-04-22T13:24:44.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.690+0000 7fcddec4e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd8112300 con 0x7fcdd8102190 2024-04-22T13:24:44.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.690+0000 7fcdcffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdd8107260 0x7fcdd8198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.690+0000 7fcdcffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdd8107260 0x7fcdd8198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50286/0 (socket says 172.21.15.12:50286) 2024-04-22T13:24:44.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.690+0000 7fcdcffff640 1 -- 172.21.15.12:0/1616080650 learned_addr learned my addr 172.21.15.12:0/1616080650 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:44.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.691+0000 7fcddc9ea640 1 --2- 172.21.15.12:0/1616080650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdd8102190 0x7fcdd8198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.691+0000 7fcddd1eb640 1 --2- 172.21.15.12:0/1616080650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdd819d020 0x7fcdd819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.691+0000 7fcdcffff640 1 -- 172.21.15.12:0/1616080650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdd8102190 msgr2=0x7fcdd8198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.691+0000 7fcdcffff640 1 --2- 172.21.15.12:0/1616080650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdd8102190 0x7fcdd8198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.691+0000 7fcdcffff640 1 -- 172.21.15.12:0/1616080650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdd819d020 msgr2=0x7fcdd819d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.691+0000 7fcdcffff640 1 --2- 172.21.15.12:0/1616080650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdd819d020 0x7fcdd819d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.691+0000 7fcdcffff640 1 -- 172.21.15.12:0/1616080650 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd819db10 con 0x7fcdd8107260 2024-04-22T13:24:44.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.691+0000 7fcdcffff640 1 --2- 172.21.15.12:0/1616080650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdd8107260 0x7fcdd8198ca0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8036040 tx=0x7fcdc80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:44.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.691+0000 7fcdcdffb640 1 -- 172.21.15.12:0/1616080650 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8043020 con 0x7fcdd8107260 2024-04-22T13:24:44.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.691+0000 7fcdcdffb640 1 -- 172.21.15.12:0/1616080650 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdc802cb70 con 0x7fcdd8107260 2024-04-22T13:24:44.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.691+0000 7fcddec4e640 1 -- 172.21.15.12:0/1616080650 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd819dda0 con 0x7fcdd8107260 2024-04-22T13:24:44.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.691+0000 7fcdcdffb640 1 -- 172.21.15.12:0/1616080650 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8003bf0 con 0x7fcdd8107260 2024-04-22T13:24:44.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.691+0000 7fcddec4e640 1 -- 172.21.15.12:0/1616080650 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd81a55c0 con 0x7fcdd8107260 2024-04-22T13:24:44.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.693+0000 7fcdcdffb640 1 -- 172.21.15.12:0/1616080650 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fcdc8046020 con 0x7fcdd8107260 2024-04-22T13:24:44.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.694+0000 7fcde1841740 1 Processor -- start 2024-04-22T13:24:44.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.694+0000 7fcde1841740 1 -- start start 2024-04-22T13:24:44.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.694+0000 7fcde1841740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdba1034d0 0x55bdba1038b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.694+0000 7fcde1841740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bdba103e80 0x55bdba10c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.694+0000 7fcde1841740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba10ca00 0x55bdba10ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.694+0000 7fcde1841740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55bdba0681f0 con 0x55bdba1034d0 2024-04-22T13:24:44.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.694+0000 7fcde1841740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55bdba0054b0 con 0x55bdba103e80 2024-04-22T13:24:44.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.694+0000 7fcde1841740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bdba0641b0 con 0x55bdba10ca00 2024-04-22T13:24:44.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.694+0000 7fcddd1eb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdba1034d0 0x55bdba1038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.694+0000 7fcddd1eb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdba1034d0 0x55bdba1038b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50300/0 (socket says 172.21.15.12:50300) 2024-04-22T13:24:44.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.694+0000 7fcddd1eb640 1 -- 172.21.15.12:0/1494812465 learned_addr learned my addr 172.21.15.12:0/1494812465 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:44.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.694+0000 7fcdcdffb640 1 --2- 172.21.15.12:0/1616080650 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fcdc00a3e40 0x7fcdc00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.694+0000 7fcddc9ea640 1 --2- 172.21.15.12:0/1494812465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bdba103e80 0x55bdba10c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.694+0000 7fcdcffff640 1 --2- 172.21.15.12:0/1494812465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba10ca00 0x55bdba10ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.694+0000 7fcddd1eb640 1 --2- 172.21.15.12:0/1616080650 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fcdc00a3e40 0x7fcdc00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.694+0000 7fcdcdffb640 1 -- 172.21.15.12:0/1616080650 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fcdc80f2b00 con 0x7fcdd8107260 2024-04-22T13:24:44.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.694+0000 7fcddd1eb640 1 -- 172.21.15.12:0/1494812465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba10ca00 msgr2=0x55bdba10ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.694+0000 7fcddd1eb640 1 --2- 172.21.15.12:0/1494812465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba10ca00 0x55bdba10ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.694+0000 7fcddd1eb640 1 -- 172.21.15.12:0/1494812465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bdba103e80 msgr2=0x55bdba10c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.694+0000 7fcddd1eb640 1 --2- 172.21.15.12:0/1494812465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bdba103e80 0x55bdba10c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.695+0000 7fcddd1eb640 1 -- 172.21.15.12:0/1494812465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdb9ff71b0 con 0x55bdba1034d0 2024-04-22T13:24:44.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.695+0000 7fcdcffff640 1 --2- 172.21.15.12:0/1494812465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba10ca00 0x55bdba10ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:44.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.695+0000 7fcddc9ea640 1 --2- 172.21.15.12:0/1494812465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bdba103e80 0x55bdba10c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:24:44.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.695+0000 7fcddd1eb640 1 --2- 172.21.15.12:0/1494812465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdba1034d0 0x55bdba1038b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fcdd00129f0 tx=0x7fcdd0012eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73977670eb65ce7a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:44.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.695+0000 7fcdbeffd640 1 -- 172.21.15.12:0/1494812465 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0023070 con 0x55bdba1034d0 2024-04-22T13:24:44.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.695+0000 7fcdbeffd640 1 -- 172.21.15.12:0/1494812465 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdd00046e0 con 0x55bdba1034d0 2024-04-22T13:24:44.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.695+0000 7fcdbeffd640 1 -- 172.21.15.12:0/1494812465 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd001f940 con 0x55bdba1034d0 2024-04-22T13:24:44.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.695+0000 7fcde1841740 1 -- 172.21.15.12:0/1494812465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdba1034d0 msgr2=0x55bdba1038b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.695+0000 7fcde1841740 1 --2- 172.21.15.12:0/1494812465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdba1034d0 0x55bdba1038b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fcdd00129f0 tx=0x7fcdd0012eb0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.695+0000 7fcddd1eb640 1 --2- 172.21.15.12:0/1616080650 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fcdc00a3e40 0x7fcdc00a6300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0022000 tx=0x7fcdd0021000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:44.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.696+0000 7fcde1841740 1 -- 172.21.15.12:0/1494812465 shutdown_connections 2024-04-22T13:24:44.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.696+0000 7fcde1841740 1 --2- 172.21.15.12:0/1494812465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba10ca00 0x55bdba10ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.696+0000 7fcde1841740 1 --2- 172.21.15.12:0/1494812465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bdba103e80 0x55bdba10c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.696+0000 7fcde1841740 1 --2- 172.21.15.12:0/1494812465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdba1034d0 0x55bdba1038b0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.696+0000 7fcde1841740 1 -- 172.21.15.12:0/1494812465 >> 172.21.15.12:0/1494812465 conn(0x55bdba0fdba0 msgr2=0x55bdba0fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:44.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.696+0000 7fcde1841740 1 -- 172.21.15.12:0/1494812465 shutdown_connections 2024-04-22T13:24:44.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.696+0000 7fcde1841740 1 -- 172.21.15.12:0/1494812465 wait complete. 2024-04-22T13:24:44.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.696+0000 7fcde1841740 1 -- start start 2024-04-22T13:24:44.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.696+0000 7fcde1841740 1 Processor -- start 2024-04-22T13:24:44.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.696+0000 7fcde1841740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bdba1034d0 0x55bdba10c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.696+0000 7fcde1841740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba103e80 0x55bdba1096f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.696+0000 7fcde1841740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdba10ca00 0x55bdba109d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.696+0000 7fcde1841740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55bdba10f970 con 0x55bdba10ca00 2024-04-22T13:24:44.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.696+0000 7fcde1841740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55bdba021500 con 0x55bdba1034d0 2024-04-22T13:24:44.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.696+0000 7fcde1841740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bdba10faf0 con 0x55bdba103e80 2024-04-22T13:24:44.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.697+0000 7fcde1841740 10 client.0 _ll_register_callbacks cb 0x55bdb9fefb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:44.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.697+0000 7fcddd1eb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdba10ca00 0x55bdba109d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.697+0000 7fcddd1eb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdba10ca00 0x55bdba109d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50310/0 (socket says 172.21.15.12:50310) 2024-04-22T13:24:44.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.697+0000 7fcddd1eb640 1 -- 172.21.15.12:0/3658979057 learned_addr learned my addr 172.21.15.12:0/3658979057 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:44.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.697+0000 7fcddc9ea640 1 --2- 172.21.15.12:0/3658979057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bdba1034d0 0x55bdba10c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.697+0000 7fcdcffff640 1 --2- 172.21.15.12:0/3658979057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba103e80 0x55bdba1096f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.697+0000 7fcddd1eb640 1 -- 172.21.15.12:0/3658979057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba103e80 msgr2=0x55bdba1096f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.697+0000 7fcddd1eb640 1 --2- 172.21.15.12:0/3658979057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba103e80 0x55bdba1096f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.697+0000 7fcddd1eb640 1 -- 172.21.15.12:0/3658979057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bdba1034d0 msgr2=0x55bdba10c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.697+0000 7fcddd1eb640 1 --2- 172.21.15.12:0/3658979057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bdba1034d0 0x55bdba10c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.697+0000 7fcddd1eb640 1 -- 172.21.15.12:0/3658979057 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdb9d7a3f0 con 0x55bdba10ca00 2024-04-22T13:24:44.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.697+0000 7fcddc9ea640 1 --2- 172.21.15.12:0/3658979057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bdba1034d0 0x55bdba10c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:44.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.697+0000 7fcddd1eb640 1 --2- 172.21.15.12:0/3658979057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdba10ca00 0x55bdba109d00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0009530 tx=0x7fcdd000e7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b3737ceff866296 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:44.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.697+0000 7fcdb7fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:44.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.697+0000 7fcdb7fff640 1 -- 172.21.15.12:0/3658979057 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0023030 con 0x55bdba10ca00 2024-04-22T13:24:44.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.697+0000 7fcdb7fff640 1 -- 172.21.15.12:0/3658979057 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fcdd0004ca0 con 0x55bdba10ca00 2024-04-22T13:24:44.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.697+0000 7fcde1841740 1 -- 172.21.15.12:0/3658979057 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdd00046e0 con 0x55bdba10ca00 2024-04-22T13:24:44.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.697+0000 7fcdb7fff640 1 -- 172.21.15.12:0/3658979057 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcdd0003ec0 con 0x55bdba10ca00 2024-04-22T13:24:44.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.697+0000 7fcde1841740 10 client.15954 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:44.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.697+0000 7fcde1841740 1 -- 172.21.15.12:0/3658979057 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bdb9e447d0 con 0x55bdba10ca00 2024-04-22T13:24:44.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.698+0000 7fcdb7fff640 1 -- 172.21.15.12:0/3658979057 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdd00046e0 con 0x55bdba10ca00 2024-04-22T13:24:44.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.698+0000 7fcdb7fff640 1 -- 172.21.15.12:0/3658979057 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fcdd002e020 con 0x55bdba10ca00 2024-04-22T13:24:44.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.698+0000 7fcde1841740 10 client.15954 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:44.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.698+0000 7fcde1841740 10 client.15954 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:24:44.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.698+0000 7fcde1841740 20 client.15954 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:44.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.698+0000 7fcde1841740 1 --2- 172.21.15.12:0/3658979057 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55bdba11cb10 0x55bdba11ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.698+0000 7fcde1841740 4 client.15954 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:24:44.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.698+0000 7fcde1841740 1 -- 172.21.15.12:0/3658979057 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdba007930 con 0x55bdba11cb10 2024-04-22T13:24:44.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.698+0000 7fcddc9ea640 1 --2- 172.21.15.12:0/3658979057 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55bdba11cb10 0x55bdba11ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.698+0000 7fcddc9ea640 1 --2- 172.21.15.12:0/3658979057 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55bdba11cb10 0x55bdba11ef00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e5c80c674a30089b server_cookie=9f6edf80526dcbba in_seq=0 out_seq=0 2024-04-22T13:24:44.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.698+0000 7fcdb7fff640 10 client.15954 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:24:44.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.699+0000 7fcdb7fff640 1 -- 172.21.15.12:0/3658979057 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bdba007930 con 0x55bdba11cb10 2024-04-22T13:24:44.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.699+0000 7fcdb7fff640 10 client.15954 handle_command_reply: tid=0 2024-04-22T13:24:44.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.699+0000 7fcde1841740 1 client.15954 shutdown 2024-04-22T13:24:44.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.699+0000 7fcde1841740 1 -- 172.21.15.12:0/3658979057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdba10ca00 msgr2=0x55bdba109d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.699+0000 7fcde1841740 1 --2- 172.21.15.12:0/3658979057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdba10ca00 0x55bdba109d00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0009530 tx=0x7fcdd000e7d0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.699+0000 7fcde1841740 1 -- 172.21.15.12:0/3658979057 shutdown_connections 2024-04-22T13:24:44.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.699+0000 7fcde1841740 1 --2- 172.21.15.12:0/3658979057 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55bdba11cb10 0x55bdba11ef00 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.699+0000 7fcde1841740 1 --2- 172.21.15.12:0/3658979057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdba10ca00 0x55bdba109d00 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.699+0000 7fcde1841740 1 --2- 172.21.15.12:0/3658979057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba103e80 0x55bdba1096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.699+0000 7fcde1841740 1 --2- 172.21.15.12:0/3658979057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bdba1034d0 0x55bdba10c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.699+0000 7fcde1841740 1 -- 172.21.15.12:0/3658979057 >> 172.21.15.12:0/3658979057 conn(0x55bdba0fdba0 msgr2=0x55bdba0ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:44.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.699+0000 7fcdb7fff640 0 client.15954 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:24:44.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.699+0000 7fcde1841740 1 -- 172.21.15.12:0/3658979057 shutdown_connections 2024-04-22T13:24:44.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.699+0000 7fcde1841740 1 -- 172.21.15.12:0/3658979057 wait complete. 2024-04-22T13:24:44.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.700+0000 7fcde1841740 20 client.15954 trim_cache size 0 max 16384 2024-04-22T13:24:44.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.713+0000 7fcde1841740 1 Processor -- start 2024-04-22T13:24:44.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.713+0000 7fcde1841740 1 -- start start 2024-04-22T13:24:44.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.713+0000 7fcde1841740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bdba1034d0 0x55bdba118860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.713+0000 7fcde1841740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdba103e80 0x55bdba0ff630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.713+0000 7fcde1841740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba10ca00 0x55bdba10be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.713+0000 7fcde1841740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd81ada20 con 0x55bdba103e80 2024-04-22T13:24:44.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.713+0000 7fcde1841740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd81af560 con 0x55bdba1034d0 2024-04-22T13:24:44.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.713+0000 7fcde1841740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd81ae0f0 con 0x55bdba10ca00 2024-04-22T13:24:44.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.713+0000 7fcdcffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdba103e80 0x55bdba0ff630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.713+0000 7fcdcffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdba103e80 0x55bdba0ff630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50322/0 (socket says 172.21.15.12:50322) 2024-04-22T13:24:44.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.713+0000 7fcdcffff640 1 -- 172.21.15.12:0/1611193912 learned_addr learned my addr 172.21.15.12:0/1611193912 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:44.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.713+0000 7fcddd1eb640 1 --2- 172.21.15.12:0/1611193912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba10ca00 0x55bdba10be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.713+0000 7fcddc9ea640 1 --2- 172.21.15.12:0/1611193912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bdba1034d0 0x55bdba118860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.713+0000 7fcdcffff640 1 -- 172.21.15.12:0/1611193912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba10ca00 msgr2=0x55bdba10be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.713+0000 7fcdcffff640 1 --2- 172.21.15.12:0/1611193912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba10ca00 0x55bdba10be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.713+0000 7fcdcffff640 1 -- 172.21.15.12:0/1611193912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bdba1034d0 msgr2=0x55bdba118860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.713+0000 7fcdcffff640 1 --2- 172.21.15.12:0/1611193912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bdba1034d0 0x55bdba118860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.713+0000 7fcdcffff640 1 -- 172.21.15.12:0/1611193912 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdba103c20 con 0x55bdba103e80 2024-04-22T13:24:44.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.714+0000 7fcdcffff640 1 --2- 172.21.15.12:0/1611193912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdba103e80 0x55bdba0ff630 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8005ec0 tx=0x7fcdc80f5010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997d8d9c1dc46dba server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:44.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.714+0000 7fcdbe7fc640 1 -- 172.21.15.12:0/1611193912 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc80be780 con 0x55bdba103e80 2024-04-22T13:24:44.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.714+0000 7fcdbe7fc640 1 -- 172.21.15.12:0/1611193912 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdc80bf0b0 con 0x55bdba103e80 2024-04-22T13:24:44.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.714+0000 7fcdbe7fc640 1 -- 172.21.15.12:0/1611193912 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc80be310 con 0x55bdba103e80 2024-04-22T13:24:44.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.714+0000 7fcde1841740 1 -- 172.21.15.12:0/1611193912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdba103e80 msgr2=0x55bdba0ff630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.714+0000 7fcde1841740 1 --2- 172.21.15.12:0/1611193912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdba103e80 0x55bdba0ff630 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8005ec0 tx=0x7fcdc80f5010 comp rx=0 tx=0).stop 2024-04-22T13:24:44.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.714+0000 7fcde1841740 1 -- 172.21.15.12:0/1611193912 shutdown_connections 2024-04-22T13:24:44.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.714+0000 7fcde1841740 1 --2- 172.21.15.12:0/1611193912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba10ca00 0x55bdba10be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.714+0000 7fcde1841740 1 --2- 172.21.15.12:0/1611193912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdba103e80 0x55bdba0ff630 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.714+0000 7fcde1841740 1 --2- 172.21.15.12:0/1611193912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bdba1034d0 0x55bdba118860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.714+0000 7fcde1841740 1 -- 172.21.15.12:0/1611193912 >> 172.21.15.12:0/1611193912 conn(0x55bdba0fdba0 msgr2=0x55bdba0fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:44.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.714+0000 7fcde1841740 1 -- 172.21.15.12:0/1611193912 shutdown_connections 2024-04-22T13:24:44.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.714+0000 7fcde1841740 1 -- 172.21.15.12:0/1611193912 wait complete. 2024-04-22T13:24:44.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.715+0000 7fcde1841740 1 -- start start 2024-04-22T13:24:44.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.715+0000 7fcde1841740 1 Processor -- start 2024-04-22T13:24:44.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.715+0000 7fcde1841740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdba1034d0 0x55bdba114d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.715+0000 7fcde1841740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bdba103e80 0x55bdba1152b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.715+0000 7fcde1841740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba10ca00 0x55bdba105f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.715+0000 7fcde1841740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd81aee90 con 0x55bdba1034d0 2024-04-22T13:24:44.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.715+0000 7fcde1841740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd81ae7c0 con 0x55bdba103e80 2024-04-22T13:24:44.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.715+0000 7fcde1841740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd81aba20 con 0x55bdba10ca00 2024-04-22T13:24:44.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.715+0000 7fcde1841740 10 client.0 _ll_register_callbacks cb 0x7fcdd001fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:44.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.715+0000 7fcddc9ea640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdba1034d0 0x55bdba114d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.715+0000 7fcddc9ea640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdba1034d0 0x55bdba114d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50334/0 (socket says 172.21.15.12:50334) 2024-04-22T13:24:44.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.715+0000 7fcddc9ea640 1 -- 172.21.15.12:0/1039866073 learned_addr learned my addr 172.21.15.12:0/1039866073 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:44.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.715+0000 7fcdcffff640 1 --2- 172.21.15.12:0/1039866073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bdba103e80 0x55bdba1152b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.716+0000 7fcddd1eb640 1 --2- 172.21.15.12:0/1039866073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba10ca00 0x55bdba105f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.716+0000 7fcddc9ea640 1 -- 172.21.15.12:0/1039866073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba10ca00 msgr2=0x55bdba105f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.716+0000 7fcddc9ea640 1 --2- 172.21.15.12:0/1039866073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba10ca00 0x55bdba105f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.716+0000 7fcddc9ea640 1 -- 172.21.15.12:0/1039866073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bdba103e80 msgr2=0x55bdba1152b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.716+0000 7fcddc9ea640 1 --2- 172.21.15.12:0/1039866073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bdba103e80 0x55bdba1152b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.716+0000 7fcddc9ea640 1 -- 172.21.15.12:0/1039866073 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdba106720 con 0x55bdba1034d0 2024-04-22T13:24:44.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.716+0000 7fcddd1eb640 1 --2- 172.21.15.12:0/1039866073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba10ca00 0x55bdba105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:44.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.716+0000 7fcddc9ea640 1 --2- 172.21.15.12:0/1039866073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdba1034d0 0x55bdba114d70 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4000c00 tx=0x7fcdd4011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b92a96f0546058 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:44.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.716+0000 7fcdb7fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:44.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.716+0000 7fcdb7fff640 1 -- 172.21.15.12:0/1039866073 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4005220 con 0x55bdba1034d0 2024-04-22T13:24:44.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.716+0000 7fcdb7fff640 1 -- 172.21.15.12:0/1039866073 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fcdd40053c0 con 0x55bdba1034d0 2024-04-22T13:24:44.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.716+0000 7fcde1841740 1 -- 172.21.15.12:0/1039866073 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdc80bf0b0 con 0x55bdba1034d0 2024-04-22T13:24:44.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.716+0000 7fcdb7fff640 1 -- 172.21.15.12:0/1039866073 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcdd402f050 con 0x55bdba1034d0 2024-04-22T13:24:44.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.716+0000 7fcde1841740 10 client.15960 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:44.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.716+0000 7fcde1841740 1 -- 172.21.15.12:0/1039866073 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bdba182bf0 con 0x55bdba1034d0 2024-04-22T13:24:44.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.716+0000 7fcdb7fff640 1 -- 172.21.15.12:0/1039866073 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdc80bf0b0 con 0x55bdba1034d0 2024-04-22T13:24:44.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.716+0000 7fcdb7fff640 1 -- 172.21.15.12:0/1039866073 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fcdd401f440 con 0x55bdba1034d0 2024-04-22T13:24:44.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.716+0000 7fcde1841740 10 client.15960 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:44.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.716+0000 7fcde1841740 10 client.15960 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:24:44.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.716+0000 7fcde1841740 20 client.15960 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:44.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.716+0000 7fcde1841740 1 --2- 172.21.15.12:0/1039866073 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55bdba183310 0x55bdba185700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.717+0000 7fcde1841740 4 client.15960 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:24:44.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.717+0000 7fcde1841740 1 -- 172.21.15.12:0/1039866073 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bdba075c10 con 0x55bdba183310 2024-04-22T13:24:44.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.717+0000 7fcdcffff640 1 --2- 172.21.15.12:0/1039866073 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55bdba183310 0x55bdba185700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.717+0000 7fcdcffff640 1 --2- 172.21.15.12:0/1039866073 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55bdba183310 0x55bdba185700 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=880c9433437b46d6 server_cookie=a2fc16519c4cdc6e in_seq=0 out_seq=0 2024-04-22T13:24:44.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.717+0000 7fcdb7fff640 10 client.15960 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:24:44.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.717+0000 7fcdb7fff640 1 -- 172.21.15.12:0/1039866073 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bdba075c10 con 0x55bdba183310 2024-04-22T13:24:44.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.717+0000 7fcdb7fff640 10 client.15960 handle_command_reply: tid=0 2024-04-22T13:24:44.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.717+0000 7fcde1841740 1 client.15960 shutdown 2024-04-22T13:24:44.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.718+0000 7fcde1841740 1 -- 172.21.15.12:0/1039866073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdba1034d0 msgr2=0x55bdba114d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.718+0000 7fcde1841740 1 --2- 172.21.15.12:0/1039866073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdba1034d0 0x55bdba114d70 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4000c00 tx=0x7fcdd4011870 comp rx=0 tx=0).stop 2024-04-22T13:24:44.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.718+0000 7fcde1841740 1 -- 172.21.15.12:0/1039866073 shutdown_connections 2024-04-22T13:24:44.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.718+0000 7fcde1841740 1 --2- 172.21.15.12:0/1039866073 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55bdba183310 0x55bdba185700 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.718+0000 7fcde1841740 1 --2- 172.21.15.12:0/1039866073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bdba10ca00 0x55bdba105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.718+0000 7fcdb7fff640 0 client.15960 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:24:44.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.718+0000 7fcde1841740 1 --2- 172.21.15.12:0/1039866073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bdba103e80 0x55bdba1152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.718+0000 7fcde1841740 1 --2- 172.21.15.12:0/1039866073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bdba1034d0 0x55bdba114d70 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.718+0000 7fcde1841740 1 -- 172.21.15.12:0/1039866073 >> 172.21.15.12:0/1039866073 conn(0x55bdba0fdba0 msgr2=0x55bdba0ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:44.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.718+0000 7fcde1841740 1 -- 172.21.15.12:0/1039866073 shutdown_connections 2024-04-22T13:24:44.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.718+0000 7fcde1841740 1 -- 172.21.15.12:0/1039866073 wait complete. 2024-04-22T13:24:44.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.718+0000 7fcde1841740 20 client.15960 trim_cache size 0 max 16384 2024-04-22T13:24:44.770 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:24:44.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.718+0000 7fcddec4e640 1 -- 172.21.15.12:0/1616080650 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fcdc00a3e40 msgr2=0x7fcdc00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.718+0000 7fcddec4e640 1 --2- 172.21.15.12:0/1616080650 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fcdc00a3e40 0x7fcdc00a6300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0022000 tx=0x7fcdd0021000 comp rx=0 tx=0).stop 2024-04-22T13:24:44.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.718+0000 7fcddec4e640 1 -- 172.21.15.12:0/1616080650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdd8107260 msgr2=0x7fcdd8198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.718+0000 7fcddec4e640 1 --2- 172.21.15.12:0/1616080650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdd8107260 0x7fcdd8198ca0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8036040 tx=0x7fcdc80045e0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.718+0000 7fcddec4e640 1 -- 172.21.15.12:0/1616080650 shutdown_connections 2024-04-22T13:24:44.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.718+0000 7fcddec4e640 1 --2- 172.21.15.12:0/1616080650 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fcdc00a3e40 0x7fcdc00a6300 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.718+0000 7fcddec4e640 1 --2- 172.21.15.12:0/1616080650 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdd819d020 0x7fcdd819d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.718+0000 7fcddec4e640 1 --2- 172.21.15.12:0/1616080650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcdd8107260 0x7fcdd8198ca0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.718+0000 7fcddec4e640 1 --2- 172.21.15.12:0/1616080650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdd8102190 0x7fcdd8198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.718+0000 7fcddec4e640 1 -- 172.21.15.12:0/1616080650 >> 172.21.15.12:0/1616080650 conn(0x7fcdd80fd990 msgr2=0x7fcdd80fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:44.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.718+0000 7fcddec4e640 1 -- 172.21.15.12:0/1616080650 shutdown_connections 2024-04-22T13:24:44.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.718+0000 7fcddec4e640 1 -- 172.21.15.12:0/1616080650 wait complete. 2024-04-22T13:24:44.772 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:24:44.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.880+0000 7f8ecdf67640 1 -- 172.21.15.12:0/1309637854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ec810c510 msgr2=0x7f8ec810e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.880+0000 7f8ecdf67640 1 --2- 172.21.15.12:0/1309637854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ec810c510 0x7f8ec810e9b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009ca0 tx=0x7f8ec002c230 comp rx=0 tx=0).stop 2024-04-22T13:24:44.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.880+0000 7f8ecdf67640 1 -- 172.21.15.12:0/1309637854 shutdown_connections 2024-04-22T13:24:44.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.880+0000 7f8ecdf67640 1 --2- 172.21.15.12:0/1309637854 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ec810c510 0x7f8ec810e9b0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.880+0000 7f8ecdf67640 1 --2- 172.21.15.12:0/1309637854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ec8103af0 0x7f8ec810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.880+0000 7f8ecdf67640 1 --2- 172.21.15.12:0/1309637854 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ec81031d0 0x7f8ec81035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.880+0000 7f8ecdf67640 1 -- 172.21.15.12:0/1309637854 >> 172.21.15.12:0/1309637854 conn(0x7f8ec8077f50 msgr2=0x7f8ec8078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:44.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.880+0000 7f8ecdf67640 1 -- 172.21.15.12:0/1309637854 shutdown_connections 2024-04-22T13:24:44.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.880+0000 7f8ecdf67640 1 -- 172.21.15.12:0/1309637854 wait complete. 2024-04-22T13:24:44.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.881+0000 7f8ecdf67640 1 Processor -- start 2024-04-22T13:24:44.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.881+0000 7f8ecdf67640 1 -- start start 2024-04-22T13:24:44.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.881+0000 7f8ecdf67640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ec81031d0 0x7f8ec8198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.881+0000 7f8ecdf67640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ec8103af0 0x7f8ec8198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.881+0000 7f8ecdf67640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ec810c510 0x7f8ec819cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.881+0000 7f8ecdf67640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec8112180 con 0x7f8ec8103af0 2024-04-22T13:24:44.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.881+0000 7f8ecdf67640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec8112000 con 0x7f8ec81031d0 2024-04-22T13:24:44.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.882+0000 7f8ecdf67640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec8112300 con 0x7f8ec810c510 2024-04-22T13:24:44.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.882+0000 7f8ec6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ec8103af0 0x7f8ec8198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.882+0000 7f8ec6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ec8103af0 0x7f8ec8198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50356/0 (socket says 172.21.15.12:50356) 2024-04-22T13:24:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.882+0000 7f8ec6ffd640 1 -- 172.21.15.12:0/2403616701 learned_addr learned my addr 172.21.15.12:0/2403616701 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.882+0000 7f8ec77fe640 1 --2- 172.21.15.12:0/2403616701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ec81031d0 0x7f8ec8198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.882+0000 7f8ec7fff640 1 --2- 172.21.15.12:0/2403616701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ec810c510 0x7f8ec819cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.882+0000 7f8ec6ffd640 1 -- 172.21.15.12:0/2403616701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ec810c510 msgr2=0x7f8ec819cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.882+0000 7f8ec6ffd640 1 --2- 172.21.15.12:0/2403616701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ec810c510 0x7f8ec819cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.882+0000 7f8ec6ffd640 1 -- 172.21.15.12:0/2403616701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ec81031d0 msgr2=0x7f8ec8198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.882+0000 7f8ec6ffd640 1 --2- 172.21.15.12:0/2403616701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ec81031d0 0x7f8ec8198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.882+0000 7f8ec6ffd640 1 -- 172.21.15.12:0/2403616701 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec819d700 con 0x7f8ec8103af0 2024-04-22T13:24:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.882+0000 7f8ec7fff640 1 --2- 172.21.15.12:0/2403616701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ec810c510 0x7f8ec819cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.882+0000 7f8ec6ffd640 1 --2- 172.21.15.12:0/2403616701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ec8103af0 0x7f8ec8198ca0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00e9f0 tx=0x7f8ebc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.883+0000 7f8ec4ff9640 1 -- 172.21.15.12:0/2403616701 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc00cd80 con 0x7f8ec8103af0 2024-04-22T13:24:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.883+0000 7f8ec4ff9640 1 -- 172.21.15.12:0/2403616701 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ebc0108e0 con 0x7f8ec8103af0 2024-04-22T13:24:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.883+0000 7f8ecdf67640 1 -- 172.21.15.12:0/2403616701 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ec819d9f0 con 0x7f8ec8103af0 2024-04-22T13:24:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.883+0000 7f8ec4ff9640 1 -- 172.21.15.12:0/2403616701 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc018980 con 0x7f8ec8103af0 2024-04-22T13:24:44.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.883+0000 7f8ecdf67640 1 -- 172.21.15.12:0/2403616701 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ec81a5280 con 0x7f8ec8103af0 2024-04-22T13:24:44.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.885+0000 7f8ec4ff9640 1 -- 172.21.15.12:0/2403616701 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f8ebc010430 con 0x7f8ec8103af0 2024-04-22T13:24:44.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.885+0000 7f8ed0c3a740 1 Processor -- start 2024-04-22T13:24:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.885+0000 7f8ed0c3a740 1 -- start start 2024-04-22T13:24:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.885+0000 7f8ec4ff9640 1 --2- 172.21.15.12:0/2403616701 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f8eb00a3e40 0x7f8eb00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.885+0000 7f8ed0c3a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f3e0f54d0 0x560f3e0f58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.885+0000 7f8ed0c3a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f3e0f5e80 0x560f3e0fe430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.885+0000 7f8ed0c3a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f3e0fea00 0x560f3e100e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.885+0000 7f8ed0c3a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560f3e05a1f0 con 0x560f3e0f5e80 2024-04-22T13:24:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.885+0000 7f8ed0c3a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560f3dff74b0 con 0x560f3e0f54d0 2024-04-22T13:24:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.885+0000 7f8ed0c3a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560f3e0561b0 con 0x560f3e0fea00 2024-04-22T13:24:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.886+0000 7f8ec6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f3e0f5e80 0x560f3e0fe430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.886+0000 7f8ec7fff640 1 --2- 172.21.15.12:0/2403616701 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f8eb00a3e40 0x7f8eb00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.886+0000 7f8ec6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f3e0f5e80 0x560f3e0fe430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50368/0 (socket says 172.21.15.12:50368) 2024-04-22T13:24:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.886+0000 7f8ec6ffd640 1 -- 172.21.15.12:0/1716355945 learned_addr learned my addr 172.21.15.12:0/1716355945 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.886+0000 7f8ec4ff9640 1 -- 172.21.15.12:0/2403616701 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f8ebc0d0dd0 con 0x7f8ec8103af0 2024-04-22T13:24:44.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.886+0000 7f8ec77fe640 1 --2- 172.21.15.12:0/1716355945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f3e0f54d0 0x560f3e0f58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.886+0000 7f8ec7fff640 1 --2- 172.21.15.12:0/1716355945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f3e0fea00 0x560f3e100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.886+0000 7f8ec6ffd640 1 -- 172.21.15.12:0/1716355945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f3e0fea00 msgr2=0x560f3e100e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.886+0000 7f8ec6ffd640 1 --2- 172.21.15.12:0/1716355945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f3e0fea00 0x560f3e100e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.886+0000 7f8ec6ffd640 1 -- 172.21.15.12:0/1716355945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f3e0f54d0 msgr2=0x560f3e0f58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.886+0000 7f8ec6ffd640 1 --2- 172.21.15.12:0/1716355945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f3e0f54d0 0x560f3e0f58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.886+0000 7f8ec6ffd640 1 -- 172.21.15.12:0/1716355945 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f3dfe91b0 con 0x560f3e0f5e80 2024-04-22T13:24:44.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.886+0000 7f8ec7fff640 1 --2- 172.21.15.12:0/2403616701 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f8eb00a3e40 0x7f8eb00a6300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0006010 tx=0x7f8ec0032500 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:44.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.886+0000 7f8ec6ffd640 1 --2- 172.21.15.12:0/1716355945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f3e0f5e80 0x560f3e0fe430 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc098f70 tx=0x7f8ebc00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ceecf14913dc7b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:44.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.886+0000 7f8eadffb640 1 -- 172.21.15.12:0/1716355945 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc026040 con 0x560f3e0f5e80 2024-04-22T13:24:44.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.886+0000 7f8eadffb640 1 -- 172.21.15.12:0/1716355945 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ebc010720 con 0x560f3e0f5e80 2024-04-22T13:24:44.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.886+0000 7f8eadffb640 1 -- 172.21.15.12:0/1716355945 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc0b5570 con 0x560f3e0f5e80 2024-04-22T13:24:44.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.887+0000 7f8ed0c3a740 1 -- 172.21.15.12:0/1716355945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f3e0f5e80 msgr2=0x560f3e0fe430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.887+0000 7f8ed0c3a740 1 --2- 172.21.15.12:0/1716355945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f3e0f5e80 0x560f3e0fe430 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc098f70 tx=0x7f8ebc00cf20 comp rx=0 tx=0).stop 2024-04-22T13:24:44.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.887+0000 7f8ed0c3a740 1 -- 172.21.15.12:0/1716355945 shutdown_connections 2024-04-22T13:24:44.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.887+0000 7f8ed0c3a740 1 --2- 172.21.15.12:0/1716355945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f3e0fea00 0x560f3e100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.887+0000 7f8ed0c3a740 1 --2- 172.21.15.12:0/1716355945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f3e0f5e80 0x560f3e0fe430 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.887+0000 7f8ed0c3a740 1 --2- 172.21.15.12:0/1716355945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f3e0f54d0 0x560f3e0f58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.887+0000 7f8ed0c3a740 1 -- 172.21.15.12:0/1716355945 >> 172.21.15.12:0/1716355945 conn(0x560f3e0efba0 msgr2=0x560f3e0eff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:44.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.887+0000 7f8ed0c3a740 1 -- 172.21.15.12:0/1716355945 shutdown_connections 2024-04-22T13:24:44.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.887+0000 7f8ed0c3a740 1 -- 172.21.15.12:0/1716355945 wait complete. 2024-04-22T13:24:44.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.887+0000 7f8ed0c3a740 1 -- start start 2024-04-22T13:24:44.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.888+0000 7f8ed0c3a740 1 Processor -- start 2024-04-22T13:24:44.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.888+0000 7f8ed0c3a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f3e0f54d0 0x560f3e0fe360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.888+0000 7f8ed0c3a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f3e0f5e80 0x560f3e0fb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.888+0000 7f8ed0c3a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f3e0fea00 0x560f3e0fbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.888+0000 7f8ed0c3a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560f3e101970 con 0x560f3e0fea00 2024-04-22T13:24:44.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.888+0000 7f8ed0c3a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560f3e013500 con 0x560f3e0f54d0 2024-04-22T13:24:44.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.888+0000 7f8ed0c3a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560f3e101af0 con 0x560f3e0f5e80 2024-04-22T13:24:44.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.888+0000 7f8ed0c3a740 10 client.0 _ll_register_callbacks cb 0x560f3dfe1b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:44.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.888+0000 7f8ec7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f3e0fea00 0x560f3e0fbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.888+0000 7f8ec7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f3e0fea00 0x560f3e0fbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50370/0 (socket says 172.21.15.12:50370) 2024-04-22T13:24:44.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.888+0000 7f8ec7fff640 1 -- 172.21.15.12:0/2672782240 learned_addr learned my addr 172.21.15.12:0/2672782240 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:44.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.888+0000 7f8ec6ffd640 1 --2- 172.21.15.12:0/2672782240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f3e0f5e80 0x560f3e0fb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.888+0000 7f8ec77fe640 1 --2- 172.21.15.12:0/2672782240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f3e0f54d0 0x560f3e0fe360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.888+0000 7f8ec7fff640 1 -- 172.21.15.12:0/2672782240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f3e0f5e80 msgr2=0x560f3e0fb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.888+0000 7f8ec7fff640 1 --2- 172.21.15.12:0/2672782240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f3e0f5e80 0x560f3e0fb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.888+0000 7f8ec7fff640 1 -- 172.21.15.12:0/2672782240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f3e0f54d0 msgr2=0x560f3e0fe360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.888+0000 7f8ec7fff640 1 --2- 172.21.15.12:0/2672782240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f3e0f54d0 0x560f3e0fe360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.888+0000 7f8ec7fff640 1 -- 172.21.15.12:0/2672782240 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f3dd6c3f0 con 0x560f3e0fea00 2024-04-22T13:24:44.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.889+0000 7f8ec7fff640 1 --2- 172.21.15.12:0/2672782240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f3e0fea00 0x560f3e0fbd00 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0036000 tx=0x7f8ec002c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f11ec5f8748f3f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:44.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.889+0000 7f8ea6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:44.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.889+0000 7f8ea6ffd640 1 -- 172.21.15.12:0/2672782240 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec00349c0 con 0x560f3e0fea00 2024-04-22T13:24:44.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.889+0000 7f8ea6ffd640 1 -- 172.21.15.12:0/2672782240 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f8ec0034b60 con 0x560f3e0fea00 2024-04-22T13:24:44.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.889+0000 7f8ed0c3a740 1 -- 172.21.15.12:0/2672782240 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560f3dff9930 con 0x560f3e0fea00 2024-04-22T13:24:44.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.889+0000 7f8ea6ffd640 1 -- 172.21.15.12:0/2672782240 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8ec005a050 con 0x560f3e0fea00 2024-04-22T13:24:44.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.889+0000 7f8ed0c3a740 10 client.15972 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:44.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.889+0000 7f8ed0c3a740 1 -- 172.21.15.12:0/2672782240 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f3de367d0 con 0x560f3e0fea00 2024-04-22T13:24:44.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.889+0000 7f8ea6ffd640 1 -- 172.21.15.12:0/2672782240 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560f3dff9930 con 0x560f3e0fea00 2024-04-22T13:24:44.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.889+0000 7f8ea6ffd640 1 -- 172.21.15.12:0/2672782240 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8ec0041410 con 0x560f3e0fea00 2024-04-22T13:24:44.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.889+0000 7f8ed0c3a740 10 client.15972 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:44.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.889+0000 7f8ed0c3a740 10 client.15972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:24:44.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.889+0000 7f8ed0c3a740 20 client.15972 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:44.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.889+0000 7f8ed0c3a740 1 --2- 172.21.15.12:0/2672782240 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560f3e10ec50 0x560f3e111080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.889+0000 7f8ed0c3a740 4 client.15972 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:24:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.889+0000 7f8ed0c3a740 1 -- 172.21.15.12:0/2672782240 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f3e067c10 con 0x560f3e10ec50 2024-04-22T13:24:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.890+0000 7f8ec77fe640 1 --2- 172.21.15.12:0/2672782240 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560f3e10ec50 0x560f3e111080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.890+0000 7f8ec77fe640 1 --2- 172.21.15.12:0/2672782240 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560f3e10ec50 0x560f3e111080 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5993c6b2839a39 server_cookie=5ff569c5d2124e43 in_seq=0 out_seq=0 2024-04-22T13:24:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.890+0000 7f8ea6ffd640 10 client.15972 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.891+0000 7f8ea6ffd640 1 -- 172.21.15.12:0/2672782240 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560f3e067c10 con 0x560f3e10ec50 2024-04-22T13:24:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.891+0000 7f8ea6ffd640 10 client.15972 handle_command_reply: tid=0 2024-04-22T13:24:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.891+0000 7f8ed0c3a740 1 client.15972 shutdown 2024-04-22T13:24:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.891+0000 7f8ed0c3a740 1 -- 172.21.15.12:0/2672782240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f3e0fea00 msgr2=0x560f3e0fbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.891+0000 7f8ed0c3a740 1 --2- 172.21.15.12:0/2672782240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f3e0fea00 0x560f3e0fbd00 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0036000 tx=0x7f8ec002c230 comp rx=0 tx=0).stop 2024-04-22T13:24:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.891+0000 7f8ed0c3a740 1 -- 172.21.15.12:0/2672782240 shutdown_connections 2024-04-22T13:24:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.891+0000 7f8ed0c3a740 1 --2- 172.21.15.12:0/2672782240 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560f3e10ec50 0x560f3e111080 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.891+0000 7f8ed0c3a740 1 --2- 172.21.15.12:0/2672782240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f3e0fea00 0x560f3e0fbd00 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.891+0000 7f8ed0c3a740 1 --2- 172.21.15.12:0/2672782240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f3e0f5e80 0x560f3e0fb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.891+0000 7f8ed0c3a740 1 --2- 172.21.15.12:0/2672782240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f3e0f54d0 0x560f3e0fe360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.891+0000 7f8ea6ffd640 0 client.15972 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:44.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.891+0000 7f8ed0c3a740 1 -- 172.21.15.12:0/2672782240 >> 172.21.15.12:0/2672782240 conn(0x560f3e0efba0 msgr2=0x560f3e0f18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:44.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.891+0000 7f8ed0c3a740 1 -- 172.21.15.12:0/2672782240 shutdown_connections 2024-04-22T13:24:44.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.891+0000 7f8ed0c3a740 1 -- 172.21.15.12:0/2672782240 wait complete. 2024-04-22T13:24:44.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.891+0000 7f8ed0c3a740 20 client.15972 trim_cache size 0 max 16384 2024-04-22T13:24:44.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.905+0000 7f8ed0c3a740 1 Processor -- start 2024-04-22T13:24:44.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.905+0000 7f8ed0c3a740 1 -- start start 2024-04-22T13:24:44.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.905+0000 7f8ed0c3a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f3e0f54d0 0x560f3e10a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.905+0000 7f8ed0c3a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f3e0f5e80 0x560f3e0fde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.905+0000 7f8ed0c3a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f3e0fea00 0x560f3e0fe360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.905+0000 7f8ed0c3a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec81adb00 con 0x560f3e0f54d0 2024-04-22T13:24:44.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.905+0000 7f8ed0c3a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec81af640 con 0x560f3e0fea00 2024-04-22T13:24:44.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.905+0000 7f8ed0c3a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec81ae1d0 con 0x560f3e0f5e80 2024-04-22T13:24:44.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.905+0000 7f8ec77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f3e0f54d0 0x560f3e10a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.905+0000 7f8ec77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f3e0f54d0 0x560f3e10a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50382/0 (socket says 172.21.15.12:50382) 2024-04-22T13:24:44.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.905+0000 7f8ec77fe640 1 -- 172.21.15.12:0/217222232 learned_addr learned my addr 172.21.15.12:0/217222232 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:44.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.905+0000 7f8ec77fe640 1 -- 172.21.15.12:0/217222232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f3e0f5e80 msgr2=0x560f3e0fde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.905+0000 7f8ec77fe640 1 --2- 172.21.15.12:0/217222232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f3e0f5e80 0x560f3e0fde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.905+0000 7f8ec77fe640 1 -- 172.21.15.12:0/217222232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f3e0fea00 msgr2=0x560f3e0fe360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.905+0000 7f8ec77fe640 1 --2- 172.21.15.12:0/217222232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f3e0fea00 0x560f3e0fe360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.905+0000 7f8ec77fe640 1 -- 172.21.15.12:0/217222232 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f3e0f5c20 con 0x560f3e0f54d0 2024-04-22T13:24:44.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.905+0000 7f8ec77fe640 1 --2- 172.21.15.12:0/217222232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f3e0f54d0 0x560f3e10a830 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8000c00 tx=0x7f8eb800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3925b1074e7cc096 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:44.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.905+0000 7f8ead7fa640 1 -- 172.21.15.12:0/217222232 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb8003dc0 con 0x560f3e0f54d0 2024-04-22T13:24:44.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.905+0000 7f8ead7fa640 1 -- 172.21.15.12:0/217222232 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eb8023030 con 0x560f3e0f54d0 2024-04-22T13:24:44.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.905+0000 7f8ead7fa640 1 -- 172.21.15.12:0/217222232 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb801f0f0 con 0x560f3e0f54d0 2024-04-22T13:24:44.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.906+0000 7f8ed0c3a740 1 -- 172.21.15.12:0/217222232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f3e0f54d0 msgr2=0x560f3e10a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.906+0000 7f8ed0c3a740 1 --2- 172.21.15.12:0/217222232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f3e0f54d0 0x560f3e10a830 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8000c00 tx=0x7f8eb800b390 comp rx=0 tx=0).stop 2024-04-22T13:24:44.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.906+0000 7f8ed0c3a740 1 -- 172.21.15.12:0/217222232 shutdown_connections 2024-04-22T13:24:44.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.906+0000 7f8ed0c3a740 1 --2- 172.21.15.12:0/217222232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f3e0fea00 0x560f3e0fe360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.906+0000 7f8ed0c3a740 1 --2- 172.21.15.12:0/217222232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f3e0f5e80 0x560f3e0fde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.906+0000 7f8ed0c3a740 1 --2- 172.21.15.12:0/217222232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f3e0f54d0 0x560f3e10a830 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.906+0000 7f8ed0c3a740 1 -- 172.21.15.12:0/217222232 >> 172.21.15.12:0/217222232 conn(0x560f3e0efba0 msgr2=0x560f3e0eec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:44.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.906+0000 7f8ed0c3a740 1 -- 172.21.15.12:0/217222232 shutdown_connections 2024-04-22T13:24:44.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.906+0000 7f8ed0c3a740 1 -- 172.21.15.12:0/217222232 wait complete. 2024-04-22T13:24:44.904 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.906+0000 7f8ed0c3a740 1 -- start start 2024-04-22T13:24:44.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.907+0000 7f8ed0c3a740 1 Processor -- start 2024-04-22T13:24:44.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.907+0000 7f8ed0c3a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f3e0f54d0 0x560f3e1027c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.907+0000 7f8ed0c3a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f3e0f5e80 0x560f3e102d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.907+0000 7f8ed0c3a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f3e0fea00 0x560f3e103240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.907+0000 7f8ed0c3a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec81aef70 con 0x560f3e0f54d0 2024-04-22T13:24:44.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.907+0000 7f8ed0c3a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec81ae8a0 con 0x560f3e0f5e80 2024-04-22T13:24:44.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.907+0000 7f8ed0c3a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec81abae0 con 0x560f3e0fea00 2024-04-22T13:24:44.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.907+0000 7f8ed0c3a740 10 client.0 _ll_register_callbacks cb 0x7f8ec0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:44.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.907+0000 7f8ec77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f3e0f54d0 0x560f3e1027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.907+0000 7f8ec77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f3e0f54d0 0x560f3e1027c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50398/0 (socket says 172.21.15.12:50398) 2024-04-22T13:24:44.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.907+0000 7f8ec77fe640 1 -- 172.21.15.12:0/2286323462 learned_addr learned my addr 172.21.15.12:0/2286323462 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:44.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.907+0000 7f8ec7fff640 1 --2- 172.21.15.12:0/2286323462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f3e0fea00 0x560f3e103240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.907+0000 7f8ec6ffd640 1 --2- 172.21.15.12:0/2286323462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f3e0f5e80 0x560f3e102d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.907+0000 7f8ec77fe640 1 -- 172.21.15.12:0/2286323462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f3e0fea00 msgr2=0x560f3e103240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.907+0000 7f8ec77fe640 1 --2- 172.21.15.12:0/2286323462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f3e0fea00 0x560f3e103240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.907+0000 7f8ec77fe640 1 -- 172.21.15.12:0/2286323462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f3e0f5e80 msgr2=0x560f3e102d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:44.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.907+0000 7f8ec77fe640 1 --2- 172.21.15.12:0/2286323462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f3e0f5e80 0x560f3e102d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.907+0000 7f8ec77fe640 1 -- 172.21.15.12:0/2286323462 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f3e1741f0 con 0x560f3e0f54d0 2024-04-22T13:24:44.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.907+0000 7f8ec7fff640 1 --2- 172.21.15.12:0/2286323462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f3e0fea00 0x560f3e103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:44.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.907+0000 7f8ec6ffd640 1 --2- 172.21.15.12:0/2286323462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f3e0f5e80 0x560f3e102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:44.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.907+0000 7f8ec77fe640 1 --2- 172.21.15.12:0/2286323462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f3e0f54d0 0x560f3e1027c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8003c70 tx=0x7f8eb8003ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=250507c4c66e7592 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:44.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.907+0000 7f8ea6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:44.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.908+0000 7f8ea6ffd640 1 -- 172.21.15.12:0/2286323462 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb802c070 con 0x560f3e0f54d0 2024-04-22T13:24:44.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.908+0000 7f8ea6ffd640 1 -- 172.21.15.12:0/2286323462 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f8eb8004af0 con 0x560f3e0f54d0 2024-04-22T13:24:44.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.908+0000 7f8ed0c3a740 1 -- 172.21.15.12:0/2286323462 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8eb8023030 con 0x560f3e0f54d0 2024-04-22T13:24:44.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.908+0000 7f8ea6ffd640 1 -- 172.21.15.12:0/2286323462 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8eb8038050 con 0x560f3e0f54d0 2024-04-22T13:24:44.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.908+0000 7f8ed0c3a740 10 client.15978 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:44.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.908+0000 7f8ed0c3a740 1 -- 172.21.15.12:0/2286323462 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f3e1754b0 con 0x560f3e0f54d0 2024-04-22T13:24:44.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.908+0000 7f8ea6ffd640 1 -- 172.21.15.12:0/2286323462 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eb8023030 con 0x560f3e0f54d0 2024-04-22T13:24:44.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.908+0000 7f8ea6ffd640 1 -- 172.21.15.12:0/2286323462 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8eb8005150 con 0x560f3e0f54d0 2024-04-22T13:24:44.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.908+0000 7f8ed0c3a740 10 client.15978 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:44.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.908+0000 7f8ed0c3a740 10 client.15978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:24:44.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.908+0000 7f8ed0c3a740 20 client.15978 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:44.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.908+0000 7f8ed0c3a740 1 --2- 172.21.15.12:0/2286323462 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560f3e175b40 0x560f3e177f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:44.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.908+0000 7f8ed0c3a740 4 client.15978 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:24:44.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.908+0000 7f8ed0c3a740 1 -- 172.21.15.12:0/2286323462 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560f3e061a80 con 0x560f3e175b40 2024-04-22T13:24:44.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.909+0000 7f8ec6ffd640 1 --2- 172.21.15.12:0/2286323462 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560f3e175b40 0x560f3e177f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:44.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.909+0000 7f8ec6ffd640 1 --2- 172.21.15.12:0/2286323462 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560f3e175b40 0x560f3e177f60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e62fadc86fb993b5 server_cookie=e092cc4908fc2098 in_seq=0 out_seq=0 2024-04-22T13:24:44.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.909+0000 7f8ea6ffd640 10 client.15978 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:44.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.909+0000 7f8ea6ffd640 1 -- 172.21.15.12:0/2286323462 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560f3e061a80 con 0x560f3e175b40 2024-04-22T13:24:44.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.909+0000 7f8ea6ffd640 10 client.15978 handle_command_reply: tid=0 2024-04-22T13:24:44.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.909+0000 7f8ed0c3a740 1 client.15978 shutdown 2024-04-22T13:24:44.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.909+0000 7f8ed0c3a740 1 -- 172.21.15.12:0/2286323462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f3e0f54d0 msgr2=0x560f3e1027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.909+0000 7f8ed0c3a740 1 --2- 172.21.15.12:0/2286323462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f3e0f54d0 0x560f3e1027c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8003c70 tx=0x7f8eb8003ca0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.909+0000 7f8ed0c3a740 1 -- 172.21.15.12:0/2286323462 shutdown_connections 2024-04-22T13:24:44.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.909+0000 7f8ed0c3a740 1 --2- 172.21.15.12:0/2286323462 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560f3e175b40 0x560f3e177f60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.909+0000 7f8ed0c3a740 1 --2- 172.21.15.12:0/2286323462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560f3e0fea00 0x560f3e103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.909+0000 7f8ed0c3a740 1 --2- 172.21.15.12:0/2286323462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560f3e0f5e80 0x560f3e102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.909+0000 7f8ea6ffd640 0 client.15978 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:44.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.909+0000 7f8ed0c3a740 1 --2- 172.21.15.12:0/2286323462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f3e0f54d0 0x560f3e1027c0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.909+0000 7f8ed0c3a740 1 -- 172.21.15.12:0/2286323462 >> 172.21.15.12:0/2286323462 conn(0x560f3e0efba0 msgr2=0x560f3e0ee370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:44.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.909+0000 7f8ed0c3a740 1 -- 172.21.15.12:0/2286323462 shutdown_connections 2024-04-22T13:24:44.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.909+0000 7f8ed0c3a740 1 -- 172.21.15.12:0/2286323462 wait complete. 2024-04-22T13:24:44.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.909+0000 7f8ed0c3a740 20 client.15978 trim_cache size 0 max 16384 2024-04-22T13:24:44.912 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:24:44.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.910+0000 7f8ecdf67640 1 -- 172.21.15.12:0/2403616701 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f8eb00a3e40 msgr2=0x7f8eb00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.910+0000 7f8ecdf67640 1 --2- 172.21.15.12:0/2403616701 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f8eb00a3e40 0x7f8eb00a6300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0006010 tx=0x7f8ec0032500 comp rx=0 tx=0).stop 2024-04-22T13:24:44.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.910+0000 7f8ecdf67640 1 -- 172.21.15.12:0/2403616701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ec8103af0 msgr2=0x7f8ec8198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:44.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.910+0000 7f8ecdf67640 1 --2- 172.21.15.12:0/2403616701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ec8103af0 0x7f8ec8198ca0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00e9f0 tx=0x7f8ebc00eeb0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.910+0000 7f8ecdf67640 1 -- 172.21.15.12:0/2403616701 shutdown_connections 2024-04-22T13:24:44.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.910+0000 7f8ecdf67640 1 --2- 172.21.15.12:0/2403616701 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f8eb00a3e40 0x7f8eb00a6300 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.910+0000 7f8ecdf67640 1 --2- 172.21.15.12:0/2403616701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ec810c510 0x7f8ec819cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.910+0000 7f8ecdf67640 1 --2- 172.21.15.12:0/2403616701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8ec8103af0 0x7f8ec8198ca0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.910+0000 7f8ecdf67640 1 --2- 172.21.15.12:0/2403616701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ec81031d0 0x7f8ec8198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:44.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.910+0000 7f8ecdf67640 1 -- 172.21.15.12:0/2403616701 >> 172.21.15.12:0/2403616701 conn(0x7f8ec8077f50 msgr2=0x7f8ec80fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:44.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.910+0000 7f8ecdf67640 1 -- 172.21.15.12:0/2403616701 shutdown_connections 2024-04-22T13:24:44.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:44.910+0000 7f8ecdf67640 1 -- 172.21.15.12:0/2403616701 wait complete. 2024-04-22T13:24:45.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:44 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2358171320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:45.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:44 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2358171320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:45.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:44 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2358171320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:24:45.917 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:24:45.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:24:45.918 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:24:46.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.041+0000 7fa64ae81640 1 -- 172.21.15.12:0/2588513536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa64410c580 msgr2=0x7fa64410e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:46.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.041+0000 7fa64ae81640 1 --2- 172.21.15.12:0/2588513536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa64410c580 0x7fa64410e9b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa62c009900 tx=0x7fa62c02bcd0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.042+0000 7fa64ae81640 1 -- 172.21.15.12:0/2588513536 shutdown_connections 2024-04-22T13:24:46.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.042+0000 7fa64ae81640 1 --2- 172.21.15.12:0/2588513536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa64410c580 0x7fa64410e9b0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.042+0000 7fa64ae81640 1 --2- 172.21.15.12:0/2588513536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa644103ad0 0x7fa64410bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.042+0000 7fa64ae81640 1 --2- 172.21.15.12:0/2588513536 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6441031b0 0x7fa644103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.042+0000 7fa64ae81640 1 -- 172.21.15.12:0/2588513536 >> 172.21.15.12:0/2588513536 conn(0x7fa6440771d0 msgr2=0x7fa6440775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:46.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.042+0000 7fa64ae81640 1 -- 172.21.15.12:0/2588513536 shutdown_connections 2024-04-22T13:24:46.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.042+0000 7fa64ae81640 1 -- 172.21.15.12:0/2588513536 wait complete. 2024-04-22T13:24:46.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.042+0000 7fa64ae81640 1 Processor -- start 2024-04-22T13:24:46.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.042+0000 7fa64ae81640 1 -- start start 2024-04-22T13:24:46.041 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.042+0000 7fa64ae81640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6441031b0 0x7fa644198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.042+0000 7fa64ae81640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa644103ad0 0x7fa644198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.042+0000 7fa64ae81640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa64410c580 0x7fa64419cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.042+0000 7fa64ae81640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa6441121a0 con 0x7fa64410c580 2024-04-22T13:24:46.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.042+0000 7fa64ae81640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa644112020 con 0x7fa6441031b0 2024-04-22T13:24:46.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.042+0000 7fa64ae81640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa644112320 con 0x7fa644103ad0 2024-04-22T13:24:46.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.043+0000 7fa64941e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa64410c580 0x7fa64419cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.043+0000 7fa64941e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa64410c580 0x7fa64419cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50428/0 (socket says 172.21.15.12:50428) 2024-04-22T13:24:46.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.043+0000 7fa64941e640 1 -- 172.21.15.12:0/1711566438 learned_addr learned my addr 172.21.15.12:0/1711566438 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:46.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.043+0000 7fa648c1d640 1 --2- 172.21.15.12:0/1711566438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6441031b0 0x7fa644198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.043+0000 7fa64941e640 1 -- 172.21.15.12:0/1711566438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa644103ad0 msgr2=0x7fa644198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:46.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.043+0000 7fa643fff640 1 --2- 172.21.15.12:0/1711566438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa644103ad0 0x7fa644198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.043+0000 7fa64941e640 1 --2- 172.21.15.12:0/1711566438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa644103ad0 0x7fa644198cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.043+0000 7fa64941e640 1 -- 172.21.15.12:0/1711566438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6441031b0 msgr2=0x7fa644198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:46.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.043+0000 7fa64941e640 1 --2- 172.21.15.12:0/1711566438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6441031b0 0x7fa644198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.043+0000 7fa64941e640 1 -- 172.21.15.12:0/1711566438 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa64419d720 con 0x7fa64410c580 2024-04-22T13:24:46.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.043+0000 7fa64941e640 1 --2- 172.21.15.12:0/1711566438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa64410c580 0x7fa64419cff0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa62c005940 tx=0x7fa62c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:46.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.043+0000 7fa641ffb640 1 -- 172.21.15.12:0/1711566438 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c043020 con 0x7fa64410c580 2024-04-22T13:24:46.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.043+0000 7fa641ffb640 1 -- 172.21.15.12:0/1711566438 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa62c02dc20 con 0x7fa64410c580 2024-04-22T13:24:46.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.043+0000 7fa64ae81640 1 -- 172.21.15.12:0/1711566438 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa64419d9b0 con 0x7fa64410c580 2024-04-22T13:24:46.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.043+0000 7fa641ffb640 1 -- 172.21.15.12:0/1711566438 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c003ab0 con 0x7fa64410c580 2024-04-22T13:24:46.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.044+0000 7fa64ae81640 1 -- 172.21.15.12:0/1711566438 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6441a52a0 con 0x7fa64410c580 2024-04-22T13:24:46.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.046+0000 7fa641ffb640 1 -- 172.21.15.12:0/1711566438 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fa62c046020 con 0x7fa64410c580 2024-04-22T13:24:46.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.046+0000 7fa64da79740 1 Processor -- start 2024-04-22T13:24:46.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.046+0000 7fa64da79740 1 -- start start 2024-04-22T13:24:46.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.047+0000 7fa64da79740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56145c4598e0 0x56145c459cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.047+0000 7fa64da79740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56145c45a290 0x56145c462840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.047+0000 7fa64da79740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56145c462e10 0x56145c465260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.047+0000 7fa64da79740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56145c3bdf00 con 0x56145c4598e0 2024-04-22T13:24:46.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.047+0000 7fa64da79740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56145c35b590 con 0x56145c45a290 2024-04-22T13:24:46.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.047+0000 7fa641ffb640 1 --2- 172.21.15.12:0/1711566438 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa6300a3e40 0x7fa6300a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.047+0000 7fa64da79740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56145c3b9ec0 con 0x56145c462e10 2024-04-22T13:24:46.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.047+0000 7fa643fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56145c4598e0 0x56145c459cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.047+0000 7fa643fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56145c4598e0 0x56145c459cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50444/0 (socket says 172.21.15.12:50444) 2024-04-22T13:24:46.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.047+0000 7fa643fff640 1 -- 172.21.15.12:0/3029887314 learned_addr learned my addr 172.21.15.12:0/3029887314 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:46.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.047+0000 7fa64941e640 1 --2- 172.21.15.12:0/1711566438 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa6300a3e40 0x7fa6300a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.047+0000 7fa648c1d640 1 --2- 172.21.15.12:0/3029887314 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56145c45a290 0x56145c462840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.047+0000 7fa641ffb640 1 -- 172.21.15.12:0/1711566438 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fa62c0f2af0 con 0x7fa64410c580 2024-04-22T13:24:46.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.047+0000 7fa643fff640 1 --2- 172.21.15.12:0/3029887314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56145c462e10 0x56145c465260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.047+0000 7fa643fff640 1 -- 172.21.15.12:0/3029887314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56145c462e10 msgr2=0x56145c465260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:46.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.047+0000 7fa643fff640 1 --2- 172.21.15.12:0/3029887314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56145c462e10 0x56145c465260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.047+0000 7fa643fff640 1 -- 172.21.15.12:0/3029887314 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56145c45a290 msgr2=0x56145c462840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:46.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.047+0000 7fa643fff640 1 --2- 172.21.15.12:0/3029887314 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56145c45a290 0x56145c462840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.047+0000 7fa643fff640 1 -- 172.21.15.12:0/3029887314 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56145c1987e0 con 0x56145c4598e0 2024-04-22T13:24:46.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.047+0000 7fa643fff640 1 --2- 172.21.15.12:0/3029887314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56145c4598e0 0x56145c459cc0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa634010310 tx=0x7fa634009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de7f077151a7270 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:46.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.047+0000 7fa64941e640 1 --2- 172.21.15.12:0/1711566438 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa6300a3e40 0x7fa6300a6300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa62c0f0220 tx=0x7fa62c0a9040 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:46.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.047+0000 7fa63affd640 1 -- 172.21.15.12:0/3029887314 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa634013600 con 0x56145c4598e0 2024-04-22T13:24:46.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.048+0000 7fa63affd640 1 -- 172.21.15.12:0/3029887314 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa63401b070 con 0x56145c4598e0 2024-04-22T13:24:46.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.048+0000 7fa63affd640 1 -- 172.21.15.12:0/3029887314 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6340128e0 con 0x56145c4598e0 2024-04-22T13:24:46.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.048+0000 7fa64da79740 1 -- 172.21.15.12:0/3029887314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56145c4598e0 msgr2=0x56145c459cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:46.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.048+0000 7fa64da79740 1 --2- 172.21.15.12:0/3029887314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56145c4598e0 0x56145c459cc0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa634010310 tx=0x7fa634009500 comp rx=0 tx=0).stop 2024-04-22T13:24:46.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.048+0000 7fa64da79740 1 -- 172.21.15.12:0/3029887314 shutdown_connections 2024-04-22T13:24:46.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.048+0000 7fa64da79740 1 --2- 172.21.15.12:0/3029887314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56145c462e10 0x56145c465260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.048+0000 7fa64da79740 1 --2- 172.21.15.12:0/3029887314 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56145c45a290 0x56145c462840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.048+0000 7fa64da79740 1 --2- 172.21.15.12:0/3029887314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56145c4598e0 0x56145c459cc0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.048+0000 7fa64da79740 1 -- 172.21.15.12:0/3029887314 >> 172.21.15.12:0/3029887314 conn(0x56145c453fb0 msgr2=0x56145c454390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:46.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.048+0000 7fa64da79740 1 -- 172.21.15.12:0/3029887314 shutdown_connections 2024-04-22T13:24:46.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.048+0000 7fa64da79740 1 -- 172.21.15.12:0/3029887314 wait complete. 2024-04-22T13:24:46.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.049+0000 7fa64da79740 1 -- start start 2024-04-22T13:24:46.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.049+0000 7fa64da79740 1 Processor -- start 2024-04-22T13:24:46.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.049+0000 7fa64da79740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56145c4598e0 0x56145c45fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.049+0000 7fa64da79740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56145c45a290 0x56145c460350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.049+0000 7fa64da79740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56145c462e10 0x56145c460890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.049+0000 7fa64da79740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56145c465d80 con 0x56145c45a290 2024-04-22T13:24:46.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.049+0000 7fa64da79740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56145c377580 con 0x56145c4598e0 2024-04-22T13:24:46.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.049+0000 7fa64da79740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56145c465f00 con 0x56145c462e10 2024-04-22T13:24:46.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.049+0000 7fa64da79740 10 client.0 _ll_register_callbacks cb 0x56145c37d650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:46.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.049+0000 7fa648c1d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56145c45a290 0x56145c460350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.049+0000 7fa648c1d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56145c45a290 0x56145c460350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50460/0 (socket says 172.21.15.12:50460) 2024-04-22T13:24:46.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.049+0000 7fa648c1d640 1 -- 172.21.15.12:0/1713595892 learned_addr learned my addr 172.21.15.12:0/1713595892 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:46.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.049+0000 7fa643fff640 1 --2- 172.21.15.12:0/1713595892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56145c4598e0 0x56145c45fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.050+0000 7fa643fff640 1 --2- 172.21.15.12:0/1713595892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56145c462e10 0x56145c460890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.050+0000 7fa648c1d640 1 -- 172.21.15.12:0/1713595892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56145c462e10 msgr2=0x56145c460890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:46.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.050+0000 7fa648c1d640 1 --2- 172.21.15.12:0/1713595892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56145c462e10 0x56145c460890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.050+0000 7fa648c1d640 1 -- 172.21.15.12:0/1713595892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56145c4598e0 msgr2=0x56145c45fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:46.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.050+0000 7fa648c1d640 1 --2- 172.21.15.12:0/1713595892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56145c4598e0 0x56145c45fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.050+0000 7fa648c1d640 1 -- 172.21.15.12:0/1713595892 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56145c1f8f70 con 0x56145c45a290 2024-04-22T13:24:46.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.050+0000 7fa648c1d640 1 --2- 172.21.15.12:0/1713595892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56145c45a290 0x56145c460350 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa63c009fc0 tx=0x7fa63c0044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dbbbd818a3f6244 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:46.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.050+0000 7fa623fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:46.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.050+0000 7fa623fff640 1 -- 172.21.15.12:0/1713595892 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c004d20 con 0x56145c45a290 2024-04-22T13:24:46.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.050+0000 7fa623fff640 1 -- 172.21.15.12:0/1713595892 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fa63c013970 con 0x56145c45a290 2024-04-22T13:24:46.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.050+0000 7fa64da79740 1 -- 172.21.15.12:0/1713595892 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa63401b070 con 0x56145c45a290 2024-04-22T13:24:46.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.050+0000 7fa623fff640 1 -- 172.21.15.12:0/1713595892 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa63c013b70 con 0x56145c45a290 2024-04-22T13:24:46.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.050+0000 7fa64da79740 10 client.15990 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:46.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.050+0000 7fa64da79740 1 -- 172.21.15.12:0/1713595892 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56145c090170 con 0x56145c45a290 2024-04-22T13:24:46.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.050+0000 7fa623fff640 1 -- 172.21.15.12:0/1713595892 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa63401b070 con 0x56145c45a290 2024-04-22T13:24:46.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.051+0000 7fa623fff640 1 -- 172.21.15.12:0/1713595892 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa63c01a030 con 0x56145c45a290 2024-04-22T13:24:46.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.051+0000 7fa64da79740 10 client.15990 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:46.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.051+0000 7fa64da79740 10 client.15990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:24:46.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.051+0000 7fa64da79740 20 client.15990 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:46.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.051+0000 7fa64da79740 1 --2- 172.21.15.12:0/1713595892 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56145c473350 0x56145c4757d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.051+0000 7fa64da79740 4 client.15990 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:24:46.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.051+0000 7fa64da79740 1 -- 172.21.15.12:0/1713595892 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56145c3cb920 con 0x56145c473350 2024-04-22T13:24:46.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.051+0000 7fa643fff640 1 --2- 172.21.15.12:0/1713595892 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56145c473350 0x56145c4757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.051+0000 7fa623fff640 10 client.15990 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:46.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.051+0000 7fa643fff640 1 --2- 172.21.15.12:0/1713595892 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56145c473350 0x56145c4757d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6693df69524dba5 server_cookie=a743940a6e4ce34c in_seq=0 out_seq=0 2024-04-22T13:24:46.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.052+0000 7fa623fff640 1 -- 172.21.15.12:0/1713595892 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56145c3cb920 con 0x56145c473350 2024-04-22T13:24:46.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.052+0000 7fa623fff640 10 client.15990 handle_command_reply: tid=0 2024-04-22T13:24:46.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.052+0000 7fa64da79740 1 client.15990 shutdown 2024-04-22T13:24:46.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.052+0000 7fa64da79740 1 -- 172.21.15.12:0/1713595892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56145c45a290 msgr2=0x56145c460350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:46.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.052+0000 7fa64da79740 1 --2- 172.21.15.12:0/1713595892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56145c45a290 0x56145c460350 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa63c009fc0 tx=0x7fa63c0044d0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.053+0000 7fa64da79740 1 -- 172.21.15.12:0/1713595892 shutdown_connections 2024-04-22T13:24:46.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.053+0000 7fa64da79740 1 --2- 172.21.15.12:0/1713595892 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56145c473350 0x56145c4757d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.053+0000 7fa64da79740 1 --2- 172.21.15.12:0/1713595892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56145c462e10 0x56145c460890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.053+0000 7fa623fff640 0 client.15990 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:46.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.053+0000 7fa64da79740 1 --2- 172.21.15.12:0/1713595892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56145c45a290 0x56145c460350 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.053+0000 7fa64da79740 1 --2- 172.21.15.12:0/1713595892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56145c4598e0 0x56145c45fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.053+0000 7fa64da79740 1 -- 172.21.15.12:0/1713595892 >> 172.21.15.12:0/1713595892 conn(0x56145c453fb0 msgr2=0x56145c455d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:46.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.053+0000 7fa64da79740 1 -- 172.21.15.12:0/1713595892 shutdown_connections 2024-04-22T13:24:46.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.053+0000 7fa64da79740 1 -- 172.21.15.12:0/1713595892 wait complete. 2024-04-22T13:24:46.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.053+0000 7fa64da79740 20 client.15990 trim_cache size 0 max 16384 2024-04-22T13:24:46.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.070+0000 7fa64da79740 1 Processor -- start 2024-04-22T13:24:46.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.071+0000 7fa64da79740 1 -- start start 2024-04-22T13:24:46.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.071+0000 7fa64da79740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56145c4598e0 0x56145c462540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.071+0000 7fa64da79740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56145c45a290 0x56145c45ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.071+0000 7fa64da79740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56145c462e10 0x56145c46ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.071+0000 7fa64da79740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa6441adac0 con 0x56145c45a290 2024-04-22T13:24:46.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.071+0000 7fa64da79740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa6441af600 con 0x56145c4598e0 2024-04-22T13:24:46.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.071+0000 7fa64da79740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa6441ae190 con 0x56145c462e10 2024-04-22T13:24:46.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.071+0000 7fa648c1d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56145c45a290 0x56145c45ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.071+0000 7fa648c1d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56145c45a290 0x56145c45ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50464/0 (socket says 172.21.15.12:50464) 2024-04-22T13:24:46.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.071+0000 7fa648c1d640 1 -- 172.21.15.12:0/1139652692 learned_addr learned my addr 172.21.15.12:0/1139652692 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:46.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.071+0000 7fa643fff640 1 --2- 172.21.15.12:0/1139652692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56145c4598e0 0x56145c462540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.071+0000 7fa648c1d640 1 -- 172.21.15.12:0/1139652692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56145c462e10 msgr2=0x56145c46ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:46.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.071+0000 7fa648c1d640 1 --2- 172.21.15.12:0/1139652692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56145c462e10 0x56145c46ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.071+0000 7fa648c1d640 1 -- 172.21.15.12:0/1139652692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56145c4598e0 msgr2=0x56145c462540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:46.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.071+0000 7fa648c1d640 1 --2- 172.21.15.12:0/1139652692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56145c4598e0 0x56145c462540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.071+0000 7fa648c1d640 1 -- 172.21.15.12:0/1139652692 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56145c45e360 con 0x56145c45a290 2024-04-22T13:24:46.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.071+0000 7fa643fff640 1 --2- 172.21.15.12:0/1139652692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56145c4598e0 0x56145c462540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:46.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.071+0000 7fa648c1d640 1 --2- 172.21.15.12:0/1139652692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56145c45a290 0x56145c45ed50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa63c002230 tx=0x7fa63c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e2844d912dff66e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:46.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.071+0000 7fa63a7fc640 1 -- 172.21.15.12:0/1139652692 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c01b830 con 0x56145c45a290 2024-04-22T13:24:46.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.071+0000 7fa63a7fc640 1 -- 172.21.15.12:0/1139652692 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa63c01ba00 con 0x56145c45a290 2024-04-22T13:24:46.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.071+0000 7fa63a7fc640 1 -- 172.21.15.12:0/1139652692 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c025860 con 0x56145c45a290 2024-04-22T13:24:46.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.072+0000 7fa64da79740 1 -- 172.21.15.12:0/1139652692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56145c45a290 msgr2=0x56145c45ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:46.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.072+0000 7fa64da79740 1 --2- 172.21.15.12:0/1139652692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56145c45a290 0x56145c45ed50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa63c002230 tx=0x7fa63c005fd0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.072+0000 7fa64da79740 1 -- 172.21.15.12:0/1139652692 shutdown_connections 2024-04-22T13:24:46.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.072+0000 7fa64da79740 1 --2- 172.21.15.12:0/1139652692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56145c462e10 0x56145c46ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.072+0000 7fa64da79740 1 --2- 172.21.15.12:0/1139652692 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56145c45a290 0x56145c45ed50 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.072+0000 7fa64da79740 1 --2- 172.21.15.12:0/1139652692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56145c4598e0 0x56145c462540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.072+0000 7fa64da79740 1 -- 172.21.15.12:0/1139652692 >> 172.21.15.12:0/1139652692 conn(0x56145c453fb0 msgr2=0x56145c460660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:46.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.072+0000 7fa64da79740 1 -- 172.21.15.12:0/1139652692 shutdown_connections 2024-04-22T13:24:46.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.072+0000 7fa64da79740 1 -- 172.21.15.12:0/1139652692 wait complete. 2024-04-22T13:24:46.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.072+0000 7fa64da79740 1 -- start start 2024-04-22T13:24:46.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.073+0000 7fa64da79740 1 Processor -- start 2024-04-22T13:24:46.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.073+0000 7fa64da79740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56145c4598e0 0x56145c46b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.073+0000 7fa64da79740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56145c45a290 0x56145c46b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.073+0000 7fa64da79740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56145c462e10 0x56145c468150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.073+0000 7fa64da79740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa6441aef30 con 0x56145c45a290 2024-04-22T13:24:46.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.073+0000 7fa64da79740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa6441ae860 con 0x56145c462e10 2024-04-22T13:24:46.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.073+0000 7fa64da79740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa6441abaa0 con 0x56145c4598e0 2024-04-22T13:24:46.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.073+0000 7fa64da79740 10 client.0 _ll_register_callbacks cb 0x7fa63c009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:46.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.073+0000 7fa648c1d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56145c45a290 0x56145c46b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.073+0000 7fa648c1d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56145c45a290 0x56145c46b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50478/0 (socket says 172.21.15.12:50478) 2024-04-22T13:24:46.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.073+0000 7fa648c1d640 1 -- 172.21.15.12:0/38410590 learned_addr learned my addr 172.21.15.12:0/38410590 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:46.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.073+0000 7fa643fff640 1 --2- 172.21.15.12:0/38410590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56145c4598e0 0x56145c46b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.073+0000 7fa643fff640 1 --2- 172.21.15.12:0/38410590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56145c462e10 0x56145c468150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.073+0000 7fa648c1d640 1 -- 172.21.15.12:0/38410590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56145c4598e0 msgr2=0x56145c46b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:46.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.073+0000 7fa648c1d640 1 --2- 172.21.15.12:0/38410590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56145c4598e0 0x56145c46b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.073+0000 7fa648c1d640 1 -- 172.21.15.12:0/38410590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56145c462e10 msgr2=0x56145c468150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:46.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.073+0000 7fa648c1d640 1 --2- 172.21.15.12:0/38410590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56145c462e10 0x56145c468150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.073+0000 7fa648c1d640 1 -- 172.21.15.12:0/38410590 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56145c468950 con 0x56145c45a290 2024-04-22T13:24:46.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.074+0000 7fa648c1d640 1 --2- 172.21.15.12:0/38410590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56145c45a290 0x56145c46b6c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa63c003d90 tx=0x7fa63c01be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead2fedd596cc789 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:46.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.074+0000 7fa623fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:46.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.074+0000 7fa623fff640 1 -- 172.21.15.12:0/38410590 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c013040 con 0x56145c45a290 2024-04-22T13:24:46.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.074+0000 7fa623fff640 1 -- 172.21.15.12:0/38410590 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fa63c0131e0 con 0x56145c45a290 2024-04-22T13:24:46.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.074+0000 7fa64da79740 1 -- 172.21.15.12:0/38410590 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa63c01ba00 con 0x56145c45a290 2024-04-22T13:24:46.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.074+0000 7fa623fff640 1 -- 172.21.15.12:0/38410590 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa63c0133e0 con 0x56145c45a290 2024-04-22T13:24:46.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.074+0000 7fa64da79740 10 client.15996 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:46.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.074+0000 7fa64da79740 1 -- 172.21.15.12:0/38410590 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56145c4da310 con 0x56145c45a290 2024-04-22T13:24:46.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.074+0000 7fa623fff640 1 -- 172.21.15.12:0/38410590 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa63c01ba00 con 0x56145c45a290 2024-04-22T13:24:46.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.074+0000 7fa623fff640 1 -- 172.21.15.12:0/38410590 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa63c01a030 con 0x56145c45a290 2024-04-22T13:24:46.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.074+0000 7fa64da79740 10 client.15996 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:46.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.074+0000 7fa64da79740 10 client.15996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:24:46.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.074+0000 7fa64da79740 20 client.15996 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:46.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.074+0000 7fa64da79740 1 --2- 172.21.15.12:0/38410590 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56145c4da980 0x56145c4dcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.074+0000 7fa64da79740 4 client.15996 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:24:46.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.074+0000 7fa64da79740 1 -- 172.21.15.12:0/38410590 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56145c3c5790 con 0x56145c4da980 2024-04-22T13:24:46.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.075+0000 7fa643fff640 1 --2- 172.21.15.12:0/38410590 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56145c4da980 0x56145c4dcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.075+0000 7fa643fff640 1 --2- 172.21.15.12:0/38410590 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56145c4da980 0x56145c4dcda0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e2f604f77fae618 server_cookie=348271d0edf39293 in_seq=0 out_seq=0 2024-04-22T13:24:46.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.075+0000 7fa623fff640 10 client.15996 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:46.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.076+0000 7fa623fff640 1 -- 172.21.15.12:0/38410590 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56145c3c5790 con 0x56145c4da980 2024-04-22T13:24:46.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.076+0000 7fa623fff640 10 client.15996 handle_command_reply: tid=0 2024-04-22T13:24:46.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.076+0000 7fa64da79740 1 client.15996 shutdown 2024-04-22T13:24:46.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.076+0000 7fa64da79740 1 -- 172.21.15.12:0/38410590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56145c45a290 msgr2=0x56145c46b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:46.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.076+0000 7fa64da79740 1 --2- 172.21.15.12:0/38410590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56145c45a290 0x56145c46b6c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa63c003d90 tx=0x7fa63c01be60 comp rx=0 tx=0).stop 2024-04-22T13:24:46.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.076+0000 7fa64da79740 1 -- 172.21.15.12:0/38410590 shutdown_connections 2024-04-22T13:24:46.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.076+0000 7fa64da79740 1 --2- 172.21.15.12:0/38410590 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56145c4da980 0x56145c4dcda0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.076+0000 7fa64da79740 1 --2- 172.21.15.12:0/38410590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56145c462e10 0x56145c468150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.076+0000 7fa64da79740 1 --2- 172.21.15.12:0/38410590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56145c45a290 0x56145c46b6c0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.076+0000 7fa64da79740 1 --2- 172.21.15.12:0/38410590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56145c4598e0 0x56145c46b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.076+0000 7fa623fff640 0 client.15996 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:46.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.076+0000 7fa64da79740 1 -- 172.21.15.12:0/38410590 >> 172.21.15.12:0/38410590 conn(0x56145c453fb0 msgr2=0x56145c4552b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:46.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.077+0000 7fa64da79740 1 -- 172.21.15.12:0/38410590 shutdown_connections 2024-04-22T13:24:46.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.077+0000 7fa64da79740 1 -- 172.21.15.12:0/38410590 wait complete. 2024-04-22T13:24:46.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.077+0000 7fa64da79740 20 client.15996 trim_cache size 0 max 16384 2024-04-22T13:24:46.080 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:24:46.080 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-04-22T13:24:46.080 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "73d47bd3-64e7-497b-a713-8289d23d8d60", 2024-04-22T13:24:46.080 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-04-22T13:24:46.080 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:24:46.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.077+0000 7fa64ae81640 1 -- 172.21.15.12:0/1711566438 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa6300a3e40 msgr2=0x7fa6300a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:46.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.077+0000 7fa64ae81640 1 --2- 172.21.15.12:0/1711566438 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa6300a3e40 0x7fa6300a6300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa62c0f0220 tx=0x7fa62c0a9040 comp rx=0 tx=0).stop 2024-04-22T13:24:46.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.077+0000 7fa64ae81640 1 -- 172.21.15.12:0/1711566438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa64410c580 msgr2=0x7fa64419cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:46.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.077+0000 7fa64ae81640 1 --2- 172.21.15.12:0/1711566438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa64410c580 0x7fa64419cff0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa62c005940 tx=0x7fa62c0045e0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.077+0000 7fa64ae81640 1 -- 172.21.15.12:0/1711566438 shutdown_connections 2024-04-22T13:24:46.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.077+0000 7fa64ae81640 1 --2- 172.21.15.12:0/1711566438 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa6300a3e40 0x7fa6300a6300 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.077+0000 7fa64ae81640 1 --2- 172.21.15.12:0/1711566438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa64410c580 0x7fa64419cff0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.077+0000 7fa64ae81640 1 --2- 172.21.15.12:0/1711566438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa644103ad0 0x7fa644198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.077+0000 7fa64ae81640 1 --2- 172.21.15.12:0/1711566438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6441031b0 0x7fa644198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.077+0000 7fa64ae81640 1 -- 172.21.15.12:0/1711566438 >> 172.21.15.12:0/1711566438 conn(0x7fa6440771d0 msgr2=0x7fa6440fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:46.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.077+0000 7fa64ae81640 1 -- 172.21.15.12:0/1711566438 shutdown_connections 2024-04-22T13:24:46.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.077+0000 7fa64ae81640 1 -- 172.21.15.12:0/1711566438 wait complete. 2024-04-22T13:24:46.083 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:45 smithi012 ceph-mon[28265]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.7 KiB/s wr, 16 op/s 2024-04-22T13:24:46.085 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:24:46.085 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:24:46.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:45 smithi134 ceph-mon[31338]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.7 KiB/s wr, 16 op/s 2024-04-22T13:24:46.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.193+0000 7f0764767640 1 -- 172.21.15.12:0/3834131741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f075c10c500 msgr2=0x7f075c10e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:46.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.193+0000 7f0764767640 1 --2- 172.21.15.12:0/3834131741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f075c10c500 0x7f075c10e9a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0754009ca0 tx=0x7f075402c230 comp rx=0 tx=0).stop 2024-04-22T13:24:46.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.193+0000 7f0764767640 1 -- 172.21.15.12:0/3834131741 shutdown_connections 2024-04-22T13:24:46.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.193+0000 7f0764767640 1 --2- 172.21.15.12:0/3834131741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f075c10c500 0x7f075c10e9a0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.193+0000 7f0764767640 1 --2- 172.21.15.12:0/3834131741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f075c103ae0 0x7f075c10bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.193+0000 7f0764767640 1 --2- 172.21.15.12:0/3834131741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f075c1031c0 0x7f075c1035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.193+0000 7f0764767640 1 -- 172.21.15.12:0/3834131741 >> 172.21.15.12:0/3834131741 conn(0x7f075c078000 msgr2=0x7f075c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:46.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.193+0000 7f0764767640 1 -- 172.21.15.12:0/3834131741 shutdown_connections 2024-04-22T13:24:46.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.193+0000 7f0764767640 1 -- 172.21.15.12:0/3834131741 wait complete. 2024-04-22T13:24:46.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.194+0000 7f0764767640 1 Processor -- start 2024-04-22T13:24:46.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.194+0000 7f0764767640 1 -- start start 2024-04-22T13:24:46.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.194+0000 7f0764767640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f075c1031c0 0x7f075c198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.194+0000 7f0764767640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f075c103ae0 0x7f075c198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.194+0000 7f0764767640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f075c10c500 0x7f075c19ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.194+0000 7f0764767640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f075c112170 con 0x7f075c10c500 2024-04-22T13:24:46.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.194+0000 7f0764767640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f075c111ff0 con 0x7f075c1031c0 2024-04-22T13:24:46.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.194+0000 7f0764767640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f075c1122f0 con 0x7f075c103ae0 2024-04-22T13:24:46.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.194+0000 7f0762d04640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f075c10c500 0x7f075c19ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.194+0000 7f0762d04640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f075c10c500 0x7f075c19ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50496/0 (socket says 172.21.15.12:50496) 2024-04-22T13:24:46.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.194+0000 7f0762d04640 1 -- 172.21.15.12:0/921796246 learned_addr learned my addr 172.21.15.12:0/921796246 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:46.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.194+0000 7f0761d02640 1 --2- 172.21.15.12:0/921796246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f075c103ae0 0x7f075c198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.194+0000 7f0762503640 1 --2- 172.21.15.12:0/921796246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f075c1031c0 0x7f075c198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.194+0000 7f0762d04640 1 -- 172.21.15.12:0/921796246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f075c103ae0 msgr2=0x7f075c198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:46.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.194+0000 7f0762d04640 1 --2- 172.21.15.12:0/921796246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f075c103ae0 0x7f075c198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.194+0000 7f0762d04640 1 -- 172.21.15.12:0/921796246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f075c1031c0 msgr2=0x7f075c198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:46.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.194+0000 7f0762d04640 1 --2- 172.21.15.12:0/921796246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f075c1031c0 0x7f075c198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.194+0000 7f0762d04640 1 -- 172.21.15.12:0/921796246 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075c19d600 con 0x7f075c10c500 2024-04-22T13:24:46.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.194+0000 7f0762503640 1 --2- 172.21.15.12:0/921796246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f075c1031c0 0x7f075c198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:46.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.194+0000 7f0762d04640 1 --2- 172.21.15.12:0/921796246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f075c10c500 0x7f075c19ced0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f075402c840 tx=0x7f0754003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:46.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.195+0000 7f07537fe640 1 -- 172.21.15.12:0/921796246 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0754039040 con 0x7f075c10c500 2024-04-22T13:24:46.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.195+0000 7f07537fe640 1 -- 172.21.15.12:0/921796246 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0754044070 con 0x7f075c10c500 2024-04-22T13:24:46.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.195+0000 7f0764767640 1 -- 172.21.15.12:0/921796246 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f075c19d890 con 0x7f075c10c500 2024-04-22T13:24:46.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.195+0000 7f07537fe640 1 -- 172.21.15.12:0/921796246 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07540328a0 con 0x7f075c10c500 2024-04-22T13:24:46.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.195+0000 7f0764767640 1 -- 172.21.15.12:0/921796246 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f075c1a5180 con 0x7f075c10c500 2024-04-22T13:24:46.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.198+0000 7f0767438740 1 Processor -- start 2024-04-22T13:24:46.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.198+0000 7f07537fe640 1 -- 172.21.15.12:0/921796246 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f0754032a40 con 0x7f075c10c500 2024-04-22T13:24:46.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.198+0000 7f0767438740 1 -- start start 2024-04-22T13:24:46.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.198+0000 7f0767438740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a313fd580 0x564a313fd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.198+0000 7f0767438740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564a313fdea0 0x564a31406540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.198+0000 7f0767438740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a31406a80 0x564a31408e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.198+0000 7f0767438740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564a313621e0 con 0x564a31406a80 2024-04-22T13:24:46.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.198+0000 7f0767438740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564a312ff390 con 0x564a313fdea0 2024-04-22T13:24:46.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.198+0000 7f0767438740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564a3135e1a0 con 0x564a313fd580 2024-04-22T13:24:46.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.198+0000 7f0761d02640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a31406a80 0x564a31408e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.198+0000 7f0761d02640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a31406a80 0x564a31408e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50508/0 (socket says 172.21.15.12:50508) 2024-04-22T13:24:46.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.198+0000 7f0761d02640 1 -- 172.21.15.12:0/877525433 learned_addr learned my addr 172.21.15.12:0/877525433 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:46.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.198+0000 7f07537fe640 1 --2- 172.21.15.12:0/921796246 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f07480a3e40 0x7f07480a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.198+0000 7f0762503640 1 --2- 172.21.15.12:0/877525433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564a313fdea0 0x564a31406540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.198+0000 7f0761d02640 1 --2- 172.21.15.12:0/877525433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a313fd580 0x564a313fd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.199+0000 7f0762d04640 1 --2- 172.21.15.12:0/921796246 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f07480a3e40 0x7f07480a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.199+0000 7f0761d02640 1 -- 172.21.15.12:0/877525433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a313fd580 msgr2=0x564a313fd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:46.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.199+0000 7f0761d02640 1 --2- 172.21.15.12:0/877525433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a313fd580 0x564a313fd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.199+0000 7f0761d02640 1 -- 172.21.15.12:0/877525433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564a313fdea0 msgr2=0x564a31406540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:46.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.199+0000 7f0761d02640 1 --2- 172.21.15.12:0/877525433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564a313fdea0 0x564a31406540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.199+0000 7f0761d02640 1 -- 172.21.15.12:0/877525433 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a3113e7d0 con 0x564a31406a80 2024-04-22T13:24:46.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.199+0000 7f07537fe640 1 -- 172.21.15.12:0/921796246 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f07540f2ea0 con 0x7f075c10c500 2024-04-22T13:24:46.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.199+0000 7f0761d02640 1 --2- 172.21.15.12:0/877525433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a31406a80 0x564a31408e70 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f074c00af90 tx=0x7f074c00a790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfcc247487ad8cd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:46.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.199+0000 7f0762d04640 1 --2- 172.21.15.12:0/921796246 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f07480a3e40 0x7f07480a6300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f07540f0630 tx=0x7f075402c230 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:46.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.199+0000 7f0750ff9640 1 -- 172.21.15.12:0/877525433 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c00f910 con 0x564a31406a80 2024-04-22T13:24:46.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.199+0000 7f0750ff9640 1 -- 172.21.15.12:0/877525433 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f074c01b390 con 0x564a31406a80 2024-04-22T13:24:46.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.199+0000 7f0750ff9640 1 -- 172.21.15.12:0/877525433 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c00d5a0 con 0x564a31406a80 2024-04-22T13:24:46.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.200+0000 7f0767438740 1 -- 172.21.15.12:0/877525433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a31406a80 msgr2=0x564a31408e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:46.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.200+0000 7f0767438740 1 --2- 172.21.15.12:0/877525433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a31406a80 0x564a31408e70 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f074c00af90 tx=0x7f074c00a790 comp rx=0 tx=0).stop 2024-04-22T13:24:46.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.200+0000 7f0767438740 1 -- 172.21.15.12:0/877525433 shutdown_connections 2024-04-22T13:24:46.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.200+0000 7f0767438740 1 --2- 172.21.15.12:0/877525433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a31406a80 0x564a31408e70 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.200+0000 7f0767438740 1 --2- 172.21.15.12:0/877525433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564a313fdea0 0x564a31406540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.200+0000 7f0767438740 1 --2- 172.21.15.12:0/877525433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a313fd580 0x564a313fd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.200+0000 7f0767438740 1 -- 172.21.15.12:0/877525433 >> 172.21.15.12:0/877525433 conn(0x564a313f7cb0 msgr2=0x564a313f8090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:46.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.200+0000 7f0767438740 1 -- 172.21.15.12:0/877525433 shutdown_connections 2024-04-22T13:24:46.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.200+0000 7f0767438740 1 -- 172.21.15.12:0/877525433 wait complete. 2024-04-22T13:24:46.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.200+0000 7f0767438740 1 -- start start 2024-04-22T13:24:46.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.200+0000 7f0767438740 1 Processor -- start 2024-04-22T13:24:46.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.201+0000 7f0767438740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564a313fd580 0x564a31403b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.201+0000 7f0767438740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a313fdea0 0x564a31404080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.201+0000 7f0767438740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a31406a80 0x564a314045c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.201+0000 7f0767438740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564a31409870 con 0x564a31406a80 2024-04-22T13:24:46.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.201+0000 7f0767438740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564a3131b2f0 con 0x564a313fd580 2024-04-22T13:24:46.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.201+0000 7f0767438740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564a314099f0 con 0x564a313fdea0 2024-04-22T13:24:46.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.201+0000 7f0767438740 10 client.0 _ll_register_callbacks cb 0x564a31328580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:46.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.201+0000 7f0761d02640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a31406a80 0x564a314045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.201+0000 7f0761d02640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a31406a80 0x564a314045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50524/0 (socket says 172.21.15.12:50524) 2024-04-22T13:24:46.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.201+0000 7f0761d02640 1 -- 172.21.15.12:0/5731872 learned_addr learned my addr 172.21.15.12:0/5731872 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:46.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.201+0000 7f0762503640 1 --2- 172.21.15.12:0/5731872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a313fdea0 0x564a31404080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.201+0000 7f0761d02640 1 --2- 172.21.15.12:0/5731872 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564a313fd580 0x564a31403b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.201+0000 7f0761d02640 1 -- 172.21.15.12:0/5731872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a313fdea0 msgr2=0x564a31404080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:46.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.201+0000 7f0761d02640 1 --2- 172.21.15.12:0/5731872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a313fdea0 0x564a31404080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.201+0000 7f0761d02640 1 -- 172.21.15.12:0/5731872 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564a313fd580 msgr2=0x564a31403b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:46.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.201+0000 7f0761d02640 1 --2- 172.21.15.12:0/5731872 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564a313fd580 0x564a31403b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.201+0000 7f0761d02640 1 -- 172.21.15.12:0/5731872 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a310743f0 con 0x564a31406a80 2024-04-22T13:24:46.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.201+0000 7f0761d02640 1 --2- 172.21.15.12:0/5731872 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a31406a80 0x564a314045c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f074c00a550 tx=0x7f074c01bc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e4bea0cfad8ba6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:46.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.201+0000 7f0745ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:46.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.201+0000 7f0745ffb640 1 -- 172.21.15.12:0/5731872 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c02d060 con 0x564a31406a80 2024-04-22T13:24:46.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.201+0000 7f0745ffb640 1 -- 172.21.15.12:0/5731872 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f074c00fdb0 con 0x564a31406a80 2024-04-22T13:24:46.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.201+0000 7f0767438740 1 -- 172.21.15.12:0/5731872 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f074c01b390 con 0x564a31406a80 2024-04-22T13:24:46.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.202+0000 7f0745ffb640 1 -- 172.21.15.12:0/5731872 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f074c005d60 con 0x564a31406a80 2024-04-22T13:24:46.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.202+0000 7f0767438740 10 client.16008 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:46.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.202+0000 7f0767438740 1 -- 172.21.15.12:0/5731872 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564a310340f0 con 0x564a31406a80 2024-04-22T13:24:46.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.202+0000 7f0745ffb640 1 -- 172.21.15.12:0/5731872 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f074c01b390 con 0x564a31406a80 2024-04-22T13:24:46.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.202+0000 7f0745ffb640 1 -- 172.21.15.12:0/5731872 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f074c01b070 con 0x564a31406a80 2024-04-22T13:24:46.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.202+0000 7f0767438740 10 client.16008 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:46.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.202+0000 7f0767438740 10 client.16008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:24:46.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.202+0000 7f0767438740 20 client.16008 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:46.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.202+0000 7f0767438740 1 --2- 172.21.15.12:0/5731872 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x564a314169c0 0x564a31418db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.202+0000 7f0767438740 4 client.16008 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:24:46.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.202+0000 7f0767438740 1 -- 172.21.15.12:0/5731872 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a3136fc00 con 0x564a314169c0 2024-04-22T13:24:46.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.202+0000 7f0762503640 1 --2- 172.21.15.12:0/5731872 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x564a314169c0 0x564a31418db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.203+0000 7f0745ffb640 10 client.16008 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:46.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.203+0000 7f0762503640 1 --2- 172.21.15.12:0/5731872 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x564a314169c0 0x564a31418db0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1685f1808d4ce966 server_cookie=2bd6c1c5913cc3db in_seq=0 out_seq=0 2024-04-22T13:24:46.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.203+0000 7f0745ffb640 1 -- 172.21.15.12:0/5731872 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564a3136fc00 con 0x564a314169c0 2024-04-22T13:24:46.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.203+0000 7f0745ffb640 10 client.16008 handle_command_reply: tid=0 2024-04-22T13:24:46.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.203+0000 7f0767438740 1 client.16008 shutdown 2024-04-22T13:24:46.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.204+0000 7f0767438740 1 -- 172.21.15.12:0/5731872 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a31406a80 msgr2=0x564a314045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:46.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.204+0000 7f0767438740 1 --2- 172.21.15.12:0/5731872 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a31406a80 0x564a314045c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f074c00a550 tx=0x7f074c01bc90 comp rx=0 tx=0).stop 2024-04-22T13:24:46.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.204+0000 7f0767438740 1 -- 172.21.15.12:0/5731872 shutdown_connections 2024-04-22T13:24:46.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.204+0000 7f0767438740 1 --2- 172.21.15.12:0/5731872 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x564a314169c0 0x564a31418db0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.204+0000 7f0767438740 1 --2- 172.21.15.12:0/5731872 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a31406a80 0x564a314045c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.204+0000 7f0767438740 1 --2- 172.21.15.12:0/5731872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a313fdea0 0x564a31404080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.204+0000 7f0767438740 1 --2- 172.21.15.12:0/5731872 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564a313fd580 0x564a31403b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.204+0000 7f0745ffb640 0 client.16008 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:46.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.204+0000 7f0767438740 1 -- 172.21.15.12:0/5731872 >> 172.21.15.12:0/5731872 conn(0x564a313f7cb0 msgr2=0x564a313f9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:46.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.204+0000 7f0767438740 1 -- 172.21.15.12:0/5731872 shutdown_connections 2024-04-22T13:24:46.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.204+0000 7f0767438740 1 -- 172.21.15.12:0/5731872 wait complete. 2024-04-22T13:24:46.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.204+0000 7f0767438740 20 client.16008 trim_cache size 0 max 16384 2024-04-22T13:24:46.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.221+0000 7f0767438740 1 Processor -- start 2024-04-22T13:24:46.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.221+0000 7f0767438740 1 -- start start 2024-04-22T13:24:46.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.222+0000 7f0767438740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564a313f7cb0 0x564a31406270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.222+0000 7f0767438740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a313fd580 0x564a31412760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.222+0000 7f0767438740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a313fdea0 0x564a3141e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.222+0000 7f0767438740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f075c1ada40 con 0x564a313fdea0 2024-04-22T13:24:46.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.222+0000 7f0767438740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f075c1af580 con 0x564a313f7cb0 2024-04-22T13:24:46.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.222+0000 7f0767438740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f075c1ae110 con 0x564a313fd580 2024-04-22T13:24:46.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.222+0000 7f0761d02640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a313fdea0 0x564a3141e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.222+0000 7f0761d02640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a313fdea0 0x564a3141e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50532/0 (socket says 172.21.15.12:50532) 2024-04-22T13:24:46.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.222+0000 7f0761d02640 1 -- 172.21.15.12:0/3174260116 learned_addr learned my addr 172.21.15.12:0/3174260116 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:46.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.222+0000 7f0761d02640 1 --2- 172.21.15.12:0/3174260116 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564a313f7cb0 0x564a31406270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.222+0000 7f0762503640 1 --2- 172.21.15.12:0/3174260116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a313fd580 0x564a31412760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.222+0000 7f0761d02640 1 -- 172.21.15.12:0/3174260116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a313fd580 msgr2=0x564a31412760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:46.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.222+0000 7f0761d02640 1 --2- 172.21.15.12:0/3174260116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a313fd580 0x564a31412760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.222+0000 7f0761d02640 1 -- 172.21.15.12:0/3174260116 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564a313f7cb0 msgr2=0x564a31406270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:46.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.222+0000 7f0761d02640 1 --2- 172.21.15.12:0/3174260116 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564a313f7cb0 0x564a31406270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.222+0000 7f0761d02640 1 -- 172.21.15.12:0/3174260116 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a313f8ee0 con 0x564a313fdea0 2024-04-22T13:24:46.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.222+0000 7f0762503640 1 --2- 172.21.15.12:0/3174260116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a313fd580 0x564a31412760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:46.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.222+0000 7f0761d02640 1 --2- 172.21.15.12:0/3174260116 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a313fdea0 0x564a3141e4f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f074c031660 tx=0x7f074c00a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d88dbef9224be server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:46.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.222+0000 7f0747fff640 1 -- 172.21.15.12:0/3174260116 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c02d070 con 0x564a313fdea0 2024-04-22T13:24:46.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.222+0000 7f0747fff640 1 -- 172.21.15.12:0/3174260116 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f074c00ab50 con 0x564a313fdea0 2024-04-22T13:24:46.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.222+0000 7f0747fff640 1 -- 172.21.15.12:0/3174260116 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c00fbb0 con 0x564a313fdea0 2024-04-22T13:24:46.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.223+0000 7f0767438740 1 -- 172.21.15.12:0/3174260116 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a313fdea0 msgr2=0x564a3141e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:46.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.223+0000 7f0767438740 1 --2- 172.21.15.12:0/3174260116 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a313fdea0 0x564a3141e4f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f074c031660 tx=0x7f074c00a400 comp rx=0 tx=0).stop 2024-04-22T13:24:46.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.223+0000 7f0767438740 1 -- 172.21.15.12:0/3174260116 shutdown_connections 2024-04-22T13:24:46.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.223+0000 7f0767438740 1 --2- 172.21.15.12:0/3174260116 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a313fdea0 0x564a3141e4f0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.223+0000 7f0767438740 1 --2- 172.21.15.12:0/3174260116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a313fd580 0x564a31412760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.223+0000 7f0767438740 1 --2- 172.21.15.12:0/3174260116 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564a313f7cb0 0x564a31406270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.223+0000 7f0767438740 1 -- 172.21.15.12:0/3174260116 >> 172.21.15.12:0/3174260116 conn(0x564a31402780 msgr2=0x564a313f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:46.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.223+0000 7f0767438740 1 -- 172.21.15.12:0/3174260116 shutdown_connections 2024-04-22T13:24:46.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.223+0000 7f0767438740 1 -- 172.21.15.12:0/3174260116 wait complete. 2024-04-22T13:24:46.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.223+0000 7f0767438740 1 -- start start 2024-04-22T13:24:46.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.223+0000 7f0767438740 1 Processor -- start 2024-04-22T13:24:46.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.224+0000 7f0767438740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564a313f7cb0 0x564a31475060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.224+0000 7f0767438740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a313fd580 0x564a314755a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.224+0000 7f0767438740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a313fdea0 0x564a3140afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.224+0000 7f0767438740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f075c1aeeb0 con 0x564a313fd580 2024-04-22T13:24:46.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.224+0000 7f0767438740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f075c1ae7e0 con 0x564a313f7cb0 2024-04-22T13:24:46.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.224+0000 7f0767438740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f075c1aba20 con 0x564a313fdea0 2024-04-22T13:24:46.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.224+0000 7f0767438740 10 client.0 _ll_register_callbacks cb 0x7f075800b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:46.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.224+0000 7f0762503640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a313fd580 0x564a314755a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.224+0000 7f0762503640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a313fd580 0x564a314755a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50542/0 (socket says 172.21.15.12:50542) 2024-04-22T13:24:46.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.224+0000 7f0762503640 1 -- 172.21.15.12:0/1269889540 learned_addr learned my addr 172.21.15.12:0/1269889540 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:46.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.224+0000 7f0761d02640 1 --2- 172.21.15.12:0/1269889540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a313fdea0 0x564a3140afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.224+0000 7f0762503640 1 -- 172.21.15.12:0/1269889540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a313fdea0 msgr2=0x564a3140afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:46.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.224+0000 7f0762503640 1 --2- 172.21.15.12:0/1269889540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a313fdea0 0x564a3140afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.224+0000 7f0762503640 1 -- 172.21.15.12:0/1269889540 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564a313f7cb0 msgr2=0x564a31475060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:46.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.224+0000 7f0762503640 1 --2- 172.21.15.12:0/1269889540 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564a313f7cb0 0x564a31475060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.224+0000 7f0762503640 1 -- 172.21.15.12:0/1269889540 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a3140b6b0 con 0x564a313fd580 2024-04-22T13:24:46.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.224+0000 7f0761d02640 1 --2- 172.21.15.12:0/1269889540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a313fdea0 0x564a3140afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:24:46.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.224+0000 7f0762503640 1 --2- 172.21.15.12:0/1269889540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a313fd580 0x564a314755a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x564a313fd050 tx=0x7f075800b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c1272a34dff8eae server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:46.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.224+0000 7f0745ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:46.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.224+0000 7f0745ffb640 1 -- 172.21.15.12:0/1269889540 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075801d470 con 0x564a313fd580 2024-04-22T13:24:46.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.224+0000 7f0745ffb640 1 -- 172.21.15.12:0/1269889540 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f075801d610 con 0x564a313fd580 2024-04-22T13:24:46.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.224+0000 7f0767438740 1 -- 172.21.15.12:0/1269889540 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f074c00ab50 con 0x564a313fd580 2024-04-22T13:24:46.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.224+0000 7f0745ffb640 1 -- 172.21.15.12:0/1269889540 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f075802c050 con 0x564a313fd580 2024-04-22T13:24:46.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.224+0000 7f0767438740 10 client.16014 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:46.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.224+0000 7f0767438740 1 -- 172.21.15.12:0/1269889540 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564a3147d940 con 0x564a313fd580 2024-04-22T13:24:46.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.225+0000 7f0745ffb640 1 -- 172.21.15.12:0/1269889540 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f074c00ab50 con 0x564a313fd580 2024-04-22T13:24:46.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.225+0000 7f0745ffb640 1 -- 172.21.15.12:0/1269889540 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f075801c3d0 con 0x564a313fd580 2024-04-22T13:24:46.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.225+0000 7f0767438740 10 client.16014 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:46.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.225+0000 7f0767438740 10 client.16014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:24:46.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.225+0000 7f0767438740 20 client.16014 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:46.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.225+0000 7f0767438740 1 --2- 172.21.15.12:0/1269889540 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x564a31406a80 0x564a31480130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:46.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.225+0000 7f0767438740 4 client.16014 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:24:46.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.225+0000 7f0767438740 1 -- 172.21.15.12:0/1269889540 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564a31369a70 con 0x564a31406a80 2024-04-22T13:24:46.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.225+0000 7f0761d02640 1 --2- 172.21.15.12:0/1269889540 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x564a31406a80 0x564a31480130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:46.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.226+0000 7f0745ffb640 10 client.16014 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:46.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.226+0000 7f0761d02640 1 --2- 172.21.15.12:0/1269889540 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x564a31406a80 0x564a31480130 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a01892d75a41288e server_cookie=494118d0fc857127 in_seq=0 out_seq=0 2024-04-22T13:24:46.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.226+0000 7f0745ffb640 1 -- 172.21.15.12:0/1269889540 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564a31369a70 con 0x564a31406a80 2024-04-22T13:24:46.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.226+0000 7f0745ffb640 10 client.16014 handle_command_reply: tid=0 2024-04-22T13:24:46.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.226+0000 7f0767438740 1 client.16014 shutdown 2024-04-22T13:24:46.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.226+0000 7f0767438740 1 -- 172.21.15.12:0/1269889540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a313fd580 msgr2=0x564a314755a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:46.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.226+0000 7f0767438740 1 --2- 172.21.15.12:0/1269889540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a313fd580 0x564a314755a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x564a313fd050 tx=0x7f075800b730 comp rx=0 tx=0).stop 2024-04-22T13:24:46.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.226+0000 7f0767438740 1 -- 172.21.15.12:0/1269889540 shutdown_connections 2024-04-22T13:24:46.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.226+0000 7f0767438740 1 --2- 172.21.15.12:0/1269889540 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x564a31406a80 0x564a31480130 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.226+0000 7f0767438740 1 --2- 172.21.15.12:0/1269889540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564a313fdea0 0x564a3140afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.226+0000 7f0745ffb640 0 client.16014 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:46.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.226+0000 7f0767438740 1 --2- 172.21.15.12:0/1269889540 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564a313fd580 0x564a314755a0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.226+0000 7f0767438740 1 --2- 172.21.15.12:0/1269889540 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564a313f7cb0 0x564a31475060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.226+0000 7f0767438740 1 -- 172.21.15.12:0/1269889540 >> 172.21.15.12:0/1269889540 conn(0x564a31402780 msgr2=0x564a313f64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:46.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.226+0000 7f0767438740 1 -- 172.21.15.12:0/1269889540 shutdown_connections 2024-04-22T13:24:46.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.226+0000 7f0767438740 1 -- 172.21.15.12:0/1269889540 wait complete. 2024-04-22T13:24:46.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.226+0000 7f0767438740 20 client.16014 trim_cache size 0 max 16384 2024-04-22T13:24:46.229 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:24:46.230 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:24:46.230 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-04-22T13:24:46.230 INFO:teuthology.orchestra.run.smithi012.stdout: "73d47bd3-64e7-497b-a713-8289d23d8d60": { 2024-04-22T13:24:46.230 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-04-22T13:24:46.230 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "73d47bd3-64e7-497b-a713-8289d23d8d60", 2024-04-22T13:24:46.230 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-04-22T13:24:46.230 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:24:46.230 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:24:46.230 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:24:46.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.227+0000 7f0764767640 1 -- 172.21.15.12:0/921796246 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f07480a3e40 msgr2=0x7f07480a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:46.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.227+0000 7f0764767640 1 --2- 172.21.15.12:0/921796246 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f07480a3e40 0x7f07480a6300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f07540f0630 tx=0x7f075402c230 comp rx=0 tx=0).stop 2024-04-22T13:24:46.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.227+0000 7f0764767640 1 -- 172.21.15.12:0/921796246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f075c10c500 msgr2=0x7f075c19ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:46.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.227+0000 7f0764767640 1 --2- 172.21.15.12:0/921796246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f075c10c500 0x7f075c19ced0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f075402c840 tx=0x7f0754003c20 comp rx=0 tx=0).stop 2024-04-22T13:24:46.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.227+0000 7f0764767640 1 -- 172.21.15.12:0/921796246 shutdown_connections 2024-04-22T13:24:46.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.227+0000 7f0764767640 1 --2- 172.21.15.12:0/921796246 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f07480a3e40 0x7f07480a6300 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.227+0000 7f0764767640 1 --2- 172.21.15.12:0/921796246 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f075c10c500 0x7f075c19ced0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.227+0000 7f0764767640 1 --2- 172.21.15.12:0/921796246 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f075c103ae0 0x7f075c198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.227+0000 7f0764767640 1 --2- 172.21.15.12:0/921796246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f075c1031c0 0x7f075c198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:46.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.227+0000 7f0764767640 1 -- 172.21.15.12:0/921796246 >> 172.21.15.12:0/921796246 conn(0x7f075c078000 msgr2=0x7f075c0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:46.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.227+0000 7f0764767640 1 -- 172.21.15.12:0/921796246 shutdown_connections 2024-04-22T13:24:46.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:24:46.227+0000 7f0764767640 1 -- 172.21.15.12:0/921796246 wait complete. 2024-04-22T13:24:46.232 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:45 smithi111 ceph-mon[31361]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.7 KiB/s wr, 16 op/s 2024-04-22T13:24:46.233 INFO:tasks.cephfs.filesystem:scrub status for tag:73d47bd3-64e7-497b-a713-8289d23d8d60 - {'path': '/', 'tag': '73d47bd3-64e7-497b-a713-8289d23d8d60', 'options': 'recursive,force'} 2024-04-22T13:24:47.079 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:46 smithi134 ceph-mon[31338]: scrub queued for path: / 2024-04-22T13:24:47.079 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:46 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:24:47.079 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:46 smithi134 ceph-mon[31338]: scrub summary: active paths [/] 2024-04-22T13:24:47.079 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:46 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:24:47.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:46 smithi012 ceph-mon[28265]: scrub queued for path: / 2024-04-22T13:24:47.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:46 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:24:47.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:46 smithi012 ceph-mon[28265]: scrub summary: active paths [/] 2024-04-22T13:24:47.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:46 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:24:47.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:46 smithi111 ceph-mon[31361]: scrub queued for path: / 2024-04-22T13:24:47.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:46 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:24:47.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:46 smithi111 ceph-mon[31361]: scrub summary: active paths [/] 2024-04-22T13:24:47.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:46 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:24:48.090 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:47 smithi012 ceph-mon[28265]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-22T13:24:48.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:47 smithi134 ceph-mon[31338]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-22T13:24:48.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:47 smithi111 ceph-mon[31361]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-22T13:24:50.029 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:49 smithi111 ceph-mon[31361]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-04-22T13:24:50.029 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:49 smithi111 ceph-mon[31361]: scrub summary: idle 2024-04-22T13:24:50.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:49 smithi012 ceph-mon[28265]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-04-22T13:24:50.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:49 smithi012 ceph-mon[28265]: scrub summary: idle 2024-04-22T13:24:50.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:49 smithi134 ceph-mon[31338]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-04-22T13:24:50.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:49 smithi134 ceph-mon[31338]: scrub summary: idle 2024-04-22T13:24:52.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:51 smithi012 ceph-mon[28265]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-04-22T13:24:52.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:51 smithi134 ceph-mon[31338]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-04-22T13:24:52.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:51 smithi111 ceph-mon[31361]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-04-22T13:24:54.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:53 smithi012 ceph-mon[28265]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.3 KiB/s wr, 15 op/s 2024-04-22T13:24:54.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:53 smithi134 ceph-mon[31338]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.3 KiB/s wr, 15 op/s 2024-04-22T13:24:54.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:53 smithi111 ceph-mon[31361]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.3 KiB/s wr, 15 op/s 2024-04-22T13:24:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.366+0000 7f3e984b3640 1 -- 172.21.15.12:0/3593783073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e901043a0 msgr2=0x7f3e90104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.366+0000 7f3e984b3640 1 --2- 172.21.15.12:0/3593783073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e901043a0 0x7f3e90104780 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009900 tx=0x7f3e8c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:24:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.367+0000 7f3e984b3640 1 -- 172.21.15.12:0/3593783073 shutdown_connections 2024-04-22T13:24:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.367+0000 7f3e984b3640 1 --2- 172.21.15.12:0/3593783073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e90109470 0x7f3e90111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.367+0000 7f3e984b3640 1 --2- 172.21.15.12:0/3593783073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e90104d50 0x7f3e90108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.363 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.367+0000 7f3e984b3640 1 --2- 172.21.15.12:0/3593783073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e901043a0 0x7f3e90104780 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.367+0000 7f3e984b3640 1 -- 172.21.15.12:0/3593783073 >> 172.21.15.12:0/3593783073 conn(0x7f3e900ffc40 msgr2=0x7f3e90102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.367+0000 7f3e984b3640 1 -- 172.21.15.12:0/3593783073 shutdown_connections 2024-04-22T13:24:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.367+0000 7f3e984b3640 1 -- 172.21.15.12:0/3593783073 wait complete. 2024-04-22T13:24:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.367+0000 7f3e984b3640 1 Processor -- start 2024-04-22T13:24:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.367+0000 7f3e984b3640 1 -- start start 2024-04-22T13:24:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.368+0000 7f3e984b3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e901043a0 0x7f3e901a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.368+0000 7f3e984b3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e90109470 0x7f3e901a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.368+0000 7f3e984b3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e9019a690 0x7f3e9019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.368+0000 7f3e984b3640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3e901143b0 con 0x7f3e9019a690 2024-04-22T13:24:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.368+0000 7f3e984b3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3e90114230 con 0x7f3e901043a0 2024-04-22T13:24:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.368+0000 7f3e984b3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3e90114530 con 0x7f3e90109470 2024-04-22T13:24:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.368+0000 7f3e96a50640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e9019a690 0x7f3e9019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.368+0000 7f3e96a50640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e9019a690 0x7f3e9019ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44182/0 (socket says 172.21.15.12:44182) 2024-04-22T13:24:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.368+0000 7f3e96a50640 1 -- 172.21.15.12:0/1718816832 learned_addr learned my addr 172.21.15.12:0/1718816832 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.368+0000 7f3e95a4e640 1 --2- 172.21.15.12:0/1718816832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e90109470 0x7f3e901a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.368+0000 7f3e9624f640 1 --2- 172.21.15.12:0/1718816832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e901043a0 0x7f3e901a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.368+0000 7f3e96a50640 1 -- 172.21.15.12:0/1718816832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e90109470 msgr2=0x7f3e901a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.368+0000 7f3e96a50640 1 --2- 172.21.15.12:0/1718816832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e90109470 0x7f3e901a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.368+0000 7f3e96a50640 1 -- 172.21.15.12:0/1718816832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e901043a0 msgr2=0x7f3e901a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.368+0000 7f3e96a50640 1 --2- 172.21.15.12:0/1718816832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e901043a0 0x7f3e901a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.368+0000 7f3e96a50640 1 -- 172.21.15.12:0/1718816832 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e9019b290 con 0x7f3e9019a690 2024-04-22T13:24:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.368+0000 7f3e96a50640 1 --2- 172.21.15.12:0/1718816832 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e9019a690 0x7f3e9019ab10 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3e8800cce0 tx=0x7f3e8800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.368+0000 7f3e877fe640 1 -- 172.21.15.12:0/1718816832 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8800f450 con 0x7f3e9019a690 2024-04-22T13:24:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.369+0000 7f3e877fe640 1 -- 172.21.15.12:0/1718816832 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e8800faa0 con 0x7f3e9019a690 2024-04-22T13:24:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.369+0000 7f3e984b3640 1 -- 172.21.15.12:0/1718816832 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e901a7320 con 0x7f3e9019a690 2024-04-22T13:24:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.369+0000 7f3e877fe640 1 -- 172.21.15.12:0/1718816832 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e88018b70 con 0x7f3e9019a690 2024-04-22T13:24:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.369+0000 7f3e984b3640 1 -- 172.21.15.12:0/1718816832 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e901a76d0 con 0x7f3e9019a690 2024-04-22T13:24:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.371+0000 7f3e877fe640 1 -- 172.21.15.12:0/1718816832 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f3e88018d10 con 0x7f3e9019a690 2024-04-22T13:24:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.371+0000 7f3e9b17c740 1 Processor -- start 2024-04-22T13:24:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.372+0000 7f3e9b17c740 1 -- start start 2024-04-22T13:24:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.372+0000 7f3e9b17c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f82198d0 0x5565f8218620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.372+0000 7f3e9b17c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565f8218b60 0x5565f8218fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.372+0000 7f3e9b17c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565f82220e0 0x5565f82244d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.372+0000 7f3e9b17c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5565f817d8d0 con 0x5565f8218b60 2024-04-22T13:24:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.372+0000 7f3e9b17c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5565f811ae80 con 0x5565f82220e0 2024-04-22T13:24:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.372+0000 7f3e9b17c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5565f8179890 con 0x5565f82198d0 2024-04-22T13:24:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.372+0000 7f3e877fe640 1 --2- 172.21.15.12:0/1718816832 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3e7c0a3ee0 0x7f3e7c0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.372+0000 7f3e9624f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565f8218b60 0x5565f8218fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.372+0000 7f3e9624f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565f8218b60 0x5565f8218fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44188/0 (socket says 172.21.15.12:44188) 2024-04-22T13:24:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.372+0000 7f3e9624f640 1 -- 172.21.15.12:0/1678887083 learned_addr learned my addr 172.21.15.12:0/1678887083 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.372+0000 7f3e96a50640 1 --2- 172.21.15.12:0/1718816832 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3e7c0a3ee0 0x7f3e7c0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.372+0000 7f3e95a4e640 1 --2- 172.21.15.12:0/1678887083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f82198d0 0x5565f8218620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.372+0000 7f3e877fe640 1 -- 172.21.15.12:0/1718816832 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f3e880d0fa0 con 0x7f3e9019a690 2024-04-22T13:24:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.372+0000 7f3e95a4e640 1 --2- 172.21.15.12:0/1678887083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565f82220e0 0x5565f82244d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.372+0000 7f3e9624f640 1 -- 172.21.15.12:0/1678887083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f82198d0 msgr2=0x5565f8218620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.372+0000 7f3e9624f640 1 --2- 172.21.15.12:0/1678887083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f82198d0 0x5565f8218620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.372+0000 7f3e9624f640 1 -- 172.21.15.12:0/1678887083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565f82220e0 msgr2=0x5565f82244d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.372+0000 7f3e9624f640 1 --2- 172.21.15.12:0/1678887083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565f82220e0 0x5565f82244d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.372+0000 7f3e9624f640 1 -- 172.21.15.12:0/1678887083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565f813c320 con 0x5565f8218b60 2024-04-22T13:24:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.372+0000 7f3e95a4e640 1 --2- 172.21.15.12:0/1678887083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f82198d0 0x5565f8218620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.373+0000 7f3e9624f640 1 --2- 172.21.15.12:0/1678887083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565f8218b60 0x5565f8218fe0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009900 tx=0x7f3e8c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8015bf0b270e291b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.373+0000 7f3e96a50640 1 --2- 172.21.15.12:0/1718816832 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3e7c0a3ee0 0x7f3e7c0a63a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3e880ce5c0 tx=0x7f3e88092170 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.373+0000 7f3e84bfe640 1 -- 172.21.15.12:0/1678887083 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c044020 con 0x5565f8218b60 2024-04-22T13:24:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.373+0000 7f3e84bfe640 1 -- 172.21.15.12:0/1678887083 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e8c02cc90 con 0x5565f8218b60 2024-04-22T13:24:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.373+0000 7f3e84bfe640 1 -- 172.21.15.12:0/1678887083 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c0046c0 con 0x5565f8218b60 2024-04-22T13:24:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.374+0000 7f3e9b17c740 1 -- 172.21.15.12:0/1678887083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565f8218b60 msgr2=0x5565f8218fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.374+0000 7f3e9b17c740 1 --2- 172.21.15.12:0/1678887083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565f8218b60 0x5565f8218fe0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009900 tx=0x7f3e8c02c650 comp rx=0 tx=0).stop 2024-04-22T13:24:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.374+0000 7f3e9b17c740 1 -- 172.21.15.12:0/1678887083 shutdown_connections 2024-04-22T13:24:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.374+0000 7f3e9b17c740 1 --2- 172.21.15.12:0/1678887083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565f82220e0 0x5565f82244d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.374+0000 7f3e9b17c740 1 --2- 172.21.15.12:0/1678887083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565f8218b60 0x5565f8218fe0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.374+0000 7f3e9b17c740 1 --2- 172.21.15.12:0/1678887083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f82198d0 0x5565f8218620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.374+0000 7f3e9b17c740 1 -- 172.21.15.12:0/1678887083 >> 172.21.15.12:0/1678887083 conn(0x5565f81dfc90 msgr2=0x5565f8213850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.374+0000 7f3e9b17c740 1 -- 172.21.15.12:0/1678887083 shutdown_connections 2024-04-22T13:24:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.374+0000 7f3e9b17c740 1 -- 172.21.15.12:0/1678887083 wait complete. 2024-04-22T13:24:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.374+0000 7f3e9b17c740 1 -- start start 2024-04-22T13:24:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.374+0000 7f3e9b17c740 1 Processor -- start 2024-04-22T13:24:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.375+0000 7f3e9b17c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8218b60 0x5565f8221a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.375+0000 7f3e9b17c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565f82198d0 0x5565f821ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.375+0000 7f3e9b17c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565f82220e0 0x5565f821f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.375+0000 7f3e9b17c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5565f8224db0 con 0x5565f82220e0 2024-04-22T13:24:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.375+0000 7f3e9b17c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5565f8105eb0 con 0x5565f82198d0 2024-04-22T13:24:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.375+0000 7f3e9b17c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5565f8224f30 con 0x5565f8218b60 2024-04-22T13:24:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.375+0000 7f3e9b17c740 10 client.0 _ll_register_callbacks cb 0x5565f8132b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.375+0000 7f3e95a4e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565f82220e0 0x5565f821f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.375+0000 7f3e95a4e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565f82220e0 0x5565f821f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44194/0 (socket says 172.21.15.12:44194) 2024-04-22T13:24:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.375+0000 7f3e95a4e640 1 -- 172.21.15.12:0/3748781804 learned_addr learned my addr 172.21.15.12:0/3748781804 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.375+0000 7f3e95a4e640 1 --2- 172.21.15.12:0/3748781804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8218b60 0x5565f8221a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.375+0000 7f3e9624f640 1 --2- 172.21.15.12:0/3748781804 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565f82198d0 0x5565f821ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.375+0000 7f3e95a4e640 1 -- 172.21.15.12:0/3748781804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8218b60 msgr2=0x5565f8221a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.375+0000 7f3e95a4e640 1 --2- 172.21.15.12:0/3748781804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8218b60 0x5565f8221a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.375+0000 7f3e95a4e640 1 -- 172.21.15.12:0/3748781804 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565f82198d0 msgr2=0x5565f821ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.375+0000 7f3e95a4e640 1 --2- 172.21.15.12:0/3748781804 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565f82198d0 0x5565f821ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.375+0000 7f3e95a4e640 1 -- 172.21.15.12:0/3748781804 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565f7e90210 con 0x5565f82220e0 2024-04-22T13:24:54.377 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.375+0000 7f3e95a4e640 1 --2- 172.21.15.12:0/3748781804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565f82220e0 0x5565f821f340 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3e8000ce70 tx=0x7f3e8000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1864863a7df2e42 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.375+0000 7f3e79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.375+0000 7f3e79ffb640 1 -- 172.21.15.12:0/3748781804 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e800142d0 con 0x5565f82220e0 2024-04-22T13:24:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.375+0000 7f3e79ffb640 1 -- 172.21.15.12:0/3748781804 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f3e80004030 con 0x5565f82220e0 2024-04-22T13:24:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.375+0000 7f3e9b17c740 1 -- 172.21.15.12:0/3748781804 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e8c02cc90 con 0x5565f82220e0 2024-04-22T13:24:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.376+0000 7f3e79ffb640 1 -- 172.21.15.12:0/3748781804 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3e80004230 con 0x5565f82220e0 2024-04-22T13:24:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.376+0000 7f3e9b17c740 10 client.16026 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.376+0000 7f3e9b17c740 1 -- 172.21.15.12:0/3748781804 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5565f7f5c620 con 0x5565f82220e0 2024-04-22T13:24:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.376+0000 7f3e79ffb640 1 -- 172.21.15.12:0/3748781804 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e8c02cc90 con 0x5565f82220e0 2024-04-22T13:24:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.376+0000 7f3e79ffb640 1 -- 172.21.15.12:0/3748781804 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3e800035f0 con 0x5565f82220e0 2024-04-22T13:24:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.376+0000 7f3e9b17c740 10 client.16026 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.376+0000 7f3e9b17c740 10 client.16026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:24:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.376+0000 7f3e9b17c740 20 client.16026 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.376+0000 7f3e9b17c740 1 --2- 172.21.15.12:0/3748781804 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5565f82320e0 0x5565f82344d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.376+0000 7f3e9b17c740 4 client.16026 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:24:54.378 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.376+0000 7f3e9b17c740 1 -- 172.21.15.12:0/3748781804 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565f8191880 con 0x5565f82320e0 2024-04-22T13:24:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.376+0000 7f3e9624f640 1 --2- 172.21.15.12:0/3748781804 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5565f82320e0 0x5565f82344d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.377+0000 7f3e9624f640 1 --2- 172.21.15.12:0/3748781804 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5565f82320e0 0x5565f82344d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2f24c9cf741e540 server_cookie=eb0abf6f711527f6 in_seq=0 out_seq=0 2024-04-22T13:24:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.377+0000 7f3e79ffb640 10 client.16026 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.377+0000 7f3e79ffb640 1 -- 172.21.15.12:0/3748781804 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5565f8191880 con 0x5565f82320e0 2024-04-22T13:24:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.377+0000 7f3e79ffb640 10 client.16026 handle_command_reply: tid=0 2024-04-22T13:24:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.378+0000 7f3e9b17c740 1 client.16026 shutdown 2024-04-22T13:24:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.378+0000 7f3e9b17c740 1 -- 172.21.15.12:0/3748781804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565f82220e0 msgr2=0x5565f821f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.378+0000 7f3e9b17c740 1 --2- 172.21.15.12:0/3748781804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565f82220e0 0x5565f821f340 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3e8000ce70 tx=0x7f3e8000c970 comp rx=0 tx=0).stop 2024-04-22T13:24:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.378+0000 7f3e9b17c740 1 -- 172.21.15.12:0/3748781804 shutdown_connections 2024-04-22T13:24:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.378+0000 7f3e9b17c740 1 --2- 172.21.15.12:0/3748781804 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5565f82320e0 0x5565f82344d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.378+0000 7f3e9b17c740 1 --2- 172.21.15.12:0/3748781804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565f82220e0 0x5565f821f340 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.378+0000 7f3e9b17c740 1 --2- 172.21.15.12:0/3748781804 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565f82198d0 0x5565f821ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.378+0000 7f3e9b17c740 1 --2- 172.21.15.12:0/3748781804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8218b60 0x5565f8221a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.378+0000 7f3e9b17c740 1 -- 172.21.15.12:0/3748781804 >> 172.21.15.12:0/3748781804 conn(0x5565f81dfc90 msgr2=0x5565f82155a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.378+0000 7f3e79ffb640 0 client.16026 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:54.379 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.378+0000 7f3e9b17c740 1 -- 172.21.15.12:0/3748781804 shutdown_connections 2024-04-22T13:24:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.378+0000 7f3e9b17c740 1 -- 172.21.15.12:0/3748781804 wait complete. 2024-04-22T13:24:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.378+0000 7f3e9b17c740 20 client.16026 trim_cache size 0 max 16384 2024-04-22T13:24:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.393+0000 7f3e9b17c740 1 Processor -- start 2024-04-22T13:24:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.393+0000 7f3e9b17c740 1 -- start start 2024-04-22T13:24:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.393+0000 7f3e9b17c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8218b60 0x5565f822dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.393+0000 7f3e9b17c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565f82198d0 0x5565f821e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.393+0000 7f3e9b17c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565f82220e0 0x5565f8221510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.393+0000 7f3e9b17c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3e901afb80 con 0x5565f82220e0 2024-04-22T13:24:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.393+0000 7f3e9b17c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3e901b16c0 con 0x5565f82198d0 2024-04-22T13:24:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.393+0000 7f3e9b17c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3e901b0250 con 0x5565f8218b60 2024-04-22T13:24:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.393+0000 7f3e95a4e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565f82220e0 0x5565f8221510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.393+0000 7f3e95a4e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565f82220e0 0x5565f8221510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44202/0 (socket says 172.21.15.12:44202) 2024-04-22T13:24:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.393+0000 7f3e95a4e640 1 -- 172.21.15.12:0/2827518637 learned_addr learned my addr 172.21.15.12:0/2827518637 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.393+0000 7f3e9624f640 1 --2- 172.21.15.12:0/2827518637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565f82198d0 0x5565f821e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.393+0000 7f3e95a4e640 1 --2- 172.21.15.12:0/2827518637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8218b60 0x5565f822dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.394+0000 7f3e95a4e640 1 -- 172.21.15.12:0/2827518637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8218b60 msgr2=0x5565f822dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.394+0000 7f3e95a4e640 1 --2- 172.21.15.12:0/2827518637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8218b60 0x5565f822dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.394+0000 7f3e95a4e640 1 -- 172.21.15.12:0/2827518637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565f82198d0 msgr2=0x5565f821e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.394+0000 7f3e95a4e640 1 --2- 172.21.15.12:0/2827518637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565f82198d0 0x5565f821e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.394+0000 7f3e95a4e640 1 -- 172.21.15.12:0/2827518637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565f821a8f0 con 0x5565f82220e0 2024-04-22T13:24:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.394+0000 7f3e9624f640 1 --2- 172.21.15.12:0/2827518637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565f82198d0 0x5565f821e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:24:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.394+0000 7f3e95a4e640 1 --2- 172.21.15.12:0/2827518637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565f82220e0 0x5565f8221510 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3e80001eb0 tx=0x7f3e800045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15644b8d01a7555a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.394+0000 7f3e7bfff640 1 -- 172.21.15.12:0/2827518637 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e800142d0 con 0x5565f82220e0 2024-04-22T13:24:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.394+0000 7f3e7bfff640 1 -- 172.21.15.12:0/2827518637 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e80011ca0 con 0x5565f82220e0 2024-04-22T13:24:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.394+0000 7f3e7bfff640 1 -- 172.21.15.12:0/2827518637 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8000fab0 con 0x5565f82220e0 2024-04-22T13:24:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.394+0000 7f3e9b17c740 1 -- 172.21.15.12:0/2827518637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565f82220e0 msgr2=0x5565f8221510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.394+0000 7f3e9b17c740 1 --2- 172.21.15.12:0/2827518637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565f82220e0 0x5565f8221510 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3e80001eb0 tx=0x7f3e800045d0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.395+0000 7f3e9b17c740 1 -- 172.21.15.12:0/2827518637 shutdown_connections 2024-04-22T13:24:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.395+0000 7f3e9b17c740 1 --2- 172.21.15.12:0/2827518637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565f82220e0 0x5565f8221510 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.395+0000 7f3e9b17c740 1 --2- 172.21.15.12:0/2827518637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565f82198d0 0x5565f821e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.395+0000 7f3e9b17c740 1 --2- 172.21.15.12:0/2827518637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f8218b60 0x5565f822dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.395+0000 7f3e9b17c740 1 -- 172.21.15.12:0/2827518637 >> 172.21.15.12:0/2827518637 conn(0x5565f81dfc90 msgr2=0x5565f8213380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.395+0000 7f3e9b17c740 1 -- 172.21.15.12:0/2827518637 shutdown_connections 2024-04-22T13:24:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.395+0000 7f3e9b17c740 1 -- 172.21.15.12:0/2827518637 wait complete. 2024-04-22T13:24:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.395+0000 7f3e9b17c740 1 -- start start 2024-04-22T13:24:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.395+0000 7f3e9b17c740 1 Processor -- start 2024-04-22T13:24:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.395+0000 7f3e9b17c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565f8218b60 0x5565f822a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.395+0000 7f3e9b17c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f82198d0 0x5565f822a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.395+0000 7f3e9b17c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565f82220e0 0x5565f821b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.395+0000 7f3e9b17c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3e901b0ff0 con 0x5565f8218b60 2024-04-22T13:24:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.395+0000 7f3e9b17c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3e901b0920 con 0x5565f82220e0 2024-04-22T13:24:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.395+0000 7f3e9b17c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3e901adb80 con 0x5565f82198d0 2024-04-22T13:24:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.395+0000 7f3e9b17c740 10 client.0 _ll_register_callbacks cb 0x7f3e80014a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:24:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.395+0000 7f3e95a4e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565f8218b60 0x5565f822a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.395+0000 7f3e95a4e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565f8218b60 0x5565f822a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44210/0 (socket says 172.21.15.12:44210) 2024-04-22T13:24:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.395+0000 7f3e95a4e640 1 -- 172.21.15.12:0/2601967554 learned_addr learned my addr 172.21.15.12:0/2601967554 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:24:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.396+0000 7f3e9624f640 1 --2- 172.21.15.12:0/2601967554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f82198d0 0x5565f822a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.396+0000 7f3e95a4e640 1 -- 172.21.15.12:0/2601967554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f82198d0 msgr2=0x5565f822a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.396+0000 7f3e95a4e640 1 --2- 172.21.15.12:0/2601967554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f82198d0 0x5565f822a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.396+0000 7f3e95a4e640 1 -- 172.21.15.12:0/2601967554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565f82220e0 msgr2=0x5565f821b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:24:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.396+0000 7f3e95a4e640 1 --2- 172.21.15.12:0/2601967554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565f82220e0 0x5565f821b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.396+0000 7f3e95a4e640 1 -- 172.21.15.12:0/2601967554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565f821be60 con 0x5565f8218b60 2024-04-22T13:24:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.396+0000 7f3e95a4e640 1 --2- 172.21.15.12:0/2601967554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565f8218b60 0x5565f822a1c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3e80014db0 tx=0x7f3e80004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2b19b4fe9d44e25 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:24:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.396+0000 7f3e79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:24:54.396 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.396+0000 7f3e79ffb640 1 -- 172.21.15.12:0/2601967554 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e800142d0 con 0x5565f8218b60 2024-04-22T13:24:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.396+0000 7f3e79ffb640 1 -- 172.21.15.12:0/2601967554 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f3e80005040 con 0x5565f8218b60 2024-04-22T13:24:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.396+0000 7f3e9b17c740 1 -- 172.21.15.12:0/2601967554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e80011ca0 con 0x5565f8218b60 2024-04-22T13:24:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.396+0000 7f3e79ffb640 1 -- 172.21.15.12:0/2601967554 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3e8003c030 con 0x5565f8218b60 2024-04-22T13:24:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.396+0000 7f3e9b17c740 10 client.16032 fetch_fsmap learned FSMap version 28 2024-04-22T13:24:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.396+0000 7f3e9b17c740 1 -- 172.21.15.12:0/2601967554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5565f82981f0 con 0x5565f8218b60 2024-04-22T13:24:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.396+0000 7f3e79ffb640 1 -- 172.21.15.12:0/2601967554 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e80011ca0 con 0x5565f8218b60 2024-04-22T13:24:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.397+0000 7f3e79ffb640 1 -- 172.21.15.12:0/2601967554 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3e80004680 con 0x5565f8218b60 2024-04-22T13:24:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.397+0000 7f3e9b17c740 10 client.16032 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:24:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.397+0000 7f3e9b17c740 10 client.16032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:24:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.397+0000 7f3e9b17c740 20 client.16032 populate_metadata read hostname 'smithi012' 2024-04-22T13:24:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.397+0000 7f3e9b17c740 1 --2- 172.21.15.12:0/2601967554 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5565f8298940 0x5565f829ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:24:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.397+0000 7f3e9b17c740 4 client.16032 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-22T13:24:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.397+0000 7f3e9b17c740 1 -- 172.21.15.12:0/2601967554 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5565f811d3d0 con 0x5565f8298940 2024-04-22T13:24:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.397+0000 7f3e9624f640 1 --2- 172.21.15.12:0/2601967554 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5565f8298940 0x5565f829ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:24:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.397+0000 7f3e9624f640 1 --2- 172.21.15.12:0/2601967554 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5565f8298940 0x5565f829ad60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2123a116cfbbfb0 server_cookie=e2fb3c81ac997ce5 in_seq=0 out_seq=0 2024-04-22T13:24:54.397 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.397+0000 7f3e79ffb640 10 client.16032 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:54.398 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.401+0000 7f3e79ffb640 1 -- 172.21.15.12:0/2601967554 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5565f811d3d0 con 0x5565f8298940 2024-04-22T13:24:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.401+0000 7f3e79ffb640 10 client.16032 handle_command_reply: tid=0 2024-04-22T13:24:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.401+0000 7f3e9b17c740 1 client.16032 shutdown 2024-04-22T13:24:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.401+0000 7f3e9b17c740 1 -- 172.21.15.12:0/2601967554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565f8218b60 msgr2=0x5565f822a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.401+0000 7f3e9b17c740 1 --2- 172.21.15.12:0/2601967554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565f8218b60 0x5565f822a1c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3e80014db0 tx=0x7f3e80004440 comp rx=0 tx=0).stop 2024-04-22T13:24:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.401+0000 7f3e9b17c740 1 -- 172.21.15.12:0/2601967554 shutdown_connections 2024-04-22T13:24:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.401+0000 7f3e9b17c740 1 --2- 172.21.15.12:0/2601967554 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5565f8298940 0x5565f829ad60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.401+0000 7f3e9b17c740 1 --2- 172.21.15.12:0/2601967554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5565f82220e0 0x5565f821b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.401+0000 7f3e9b17c740 1 --2- 172.21.15.12:0/2601967554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5565f82198d0 0x5565f822a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.401+0000 7f3e9b17c740 1 --2- 172.21.15.12:0/2601967554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5565f8218b60 0x5565f822a1c0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.401+0000 7f3e79ffb640 0 client.16032 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:24:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.401+0000 7f3e9b17c740 1 -- 172.21.15.12:0/2601967554 >> 172.21.15.12:0/2601967554 conn(0x5565f81dfc90 msgr2=0x5565f821d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.401+0000 7f3e9b17c740 1 -- 172.21.15.12:0/2601967554 shutdown_connections 2024-04-22T13:24:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.401+0000 7f3e9b17c740 1 -- 172.21.15.12:0/2601967554 wait complete. 2024-04-22T13:24:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.401+0000 7f3e9b17c740 20 client.16032 trim_cache size 0 max 16384 2024-04-22T13:24:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-04-22T13:24:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-04-22T13:24:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-04-22T13:24:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-04-22T13:24:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.402+0000 7f3e984b3640 1 -- 172.21.15.12:0/1718816832 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3e7c0a3ee0 msgr2=0x7f3e7c0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.402+0000 7f3e984b3640 1 --2- 172.21.15.12:0/1718816832 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3e7c0a3ee0 0x7f3e7c0a63a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3e880ce5c0 tx=0x7f3e88092170 comp rx=0 tx=0).stop 2024-04-22T13:24:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.402+0000 7f3e984b3640 1 -- 172.21.15.12:0/1718816832 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e9019a690 msgr2=0x7f3e9019ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:24:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.402+0000 7f3e984b3640 1 --2- 172.21.15.12:0/1718816832 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e9019a690 0x7f3e9019ab10 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3e8800cce0 tx=0x7f3e8800a620 comp rx=0 tx=0).stop 2024-04-22T13:24:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.402+0000 7f3e984b3640 1 -- 172.21.15.12:0/1718816832 shutdown_connections 2024-04-22T13:24:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.402+0000 7f3e984b3640 1 --2- 172.21.15.12:0/1718816832 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3e7c0a3ee0 0x7f3e7c0a63a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.402+0000 7f3e984b3640 1 --2- 172.21.15.12:0/1718816832 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e9019a690 0x7f3e9019ab10 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.402+0000 7f3e984b3640 1 --2- 172.21.15.12:0/1718816832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e90109470 0x7f3e901a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.402+0000 7f3e984b3640 1 --2- 172.21.15.12:0/1718816832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e901043a0 0x7f3e901a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:24:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.402+0000 7f3e984b3640 1 -- 172.21.15.12:0/1718816832 >> 172.21.15.12:0/1718816832 conn(0x7f3e900ffc40 msgr2=0x7f3e90101f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:24:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.402+0000 7f3e984b3640 1 -- 172.21.15.12:0/1718816832 shutdown_connections 2024-04-22T13:24:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:24:54.402+0000 7f3e984b3640 1 -- 172.21.15.12:0/1718816832 wait complete. 2024-04-22T13:24:56.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:55 smithi012 ceph-mon[28265]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-22T13:24:56.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:55 smithi134 ceph-mon[31338]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-22T13:24:56.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:55 smithi111 ceph-mon[31361]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-22T13:24:58.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:57 smithi012 ceph-mon[28265]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T13:24:58.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:57 smithi134 ceph-mon[31338]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T13:24:58.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:57 smithi111 ceph-mon[31361]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T13:25:00.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:24:59 smithi012 ceph-mon[28265]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-22T13:25:00.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:24:59 smithi134 ceph-mon[31338]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-22T13:25:00.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:24:59 smithi111 ceph-mon[31361]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-22T13:25:00.587 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:25:00 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:25:00.189+0000 7fdc07ca5700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:25:00.587 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:25:00 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:25:00.189+0000 7fdc07ca5700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:25:00.587 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:25:00 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:25:00.189+0000 7fdc07ca5700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:25:00.587 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:25:00 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:25:00.189+0000 7fdc07ca5700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:25:00.587 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:25:00 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:25:00.189+0000 7fdc07ca5700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-22T13:25:02.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:01 smithi012 ceph-mon[28265]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-22T13:25:02.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:01 smithi134 ceph-mon[31338]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-22T13:25:02.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:01 smithi111 ceph-mon[31361]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-22T13:25:02.938 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:25:03.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.051+0000 7ff352021640 1 -- 172.21.15.12:0/2793586731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff34c069aa0 msgr2=0x7ff34c10ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:03.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.051+0000 7ff352021640 1 --2- 172.21.15.12:0/2793586731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff34c069aa0 0x7ff34c10ede0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff344009ca0 tx=0x7ff34402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.051+0000 7ff352021640 1 -- 172.21.15.12:0/2793586731 shutdown_connections 2024-04-22T13:25:03.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.051+0000 7ff352021640 1 --2- 172.21.15.12:0/2793586731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff34c069aa0 0x7ff34c10ede0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.051+0000 7ff352021640 1 --2- 172.21.15.12:0/2793586731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff34c0690e0 0x7ff34c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.051+0000 7ff352021640 1 --2- 172.21.15.12:0/2793586731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff34c106f50 0x7ff34c107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.051+0000 7ff352021640 1 -- 172.21.15.12:0/2793586731 >> 172.21.15.12:0/2793586731 conn(0x7ff34c077e30 msgr2=0x7ff34c078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:03.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.051+0000 7ff352021640 1 -- 172.21.15.12:0/2793586731 shutdown_connections 2024-04-22T13:25:03.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.051+0000 7ff352021640 1 -- 172.21.15.12:0/2793586731 wait complete. 2024-04-22T13:25:03.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.052+0000 7ff352021640 1 Processor -- start 2024-04-22T13:25:03.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.052+0000 7ff352021640 1 -- start start 2024-04-22T13:25:03.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.052+0000 7ff352021640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff34c0690e0 0x7ff34c101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.052+0000 7ff352021640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff34c069aa0 0x7ff34c1017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.052+0000 7ff352021640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff34c106f50 0x7ff34c101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.052+0000 7ff352021640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c112170 con 0x7ff34c0690e0 2024-04-22T13:25:03.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.052+0000 7ff352021640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c111ff0 con 0x7ff34c106f50 2024-04-22T13:25:03.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.052+0000 7ff352021640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c1122f0 con 0x7ff34c069aa0 2024-04-22T13:25:03.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.052+0000 7ff34b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff34c0690e0 0x7ff34c101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.052+0000 7ff34b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff34c0690e0 0x7ff34c101270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40106/0 (socket says 172.21.15.12:40106) 2024-04-22T13:25:03.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.052+0000 7ff34b7fe640 1 -- 172.21.15.12:0/1616960414 learned_addr learned my addr 172.21.15.12:0/1616960414 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:03.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.052+0000 7ff34bfff640 1 --2- 172.21.15.12:0/1616960414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff34c106f50 0x7ff34c101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.053+0000 7ff34affd640 1 --2- 172.21.15.12:0/1616960414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff34c069aa0 0x7ff34c1017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.053+0000 7ff34b7fe640 1 -- 172.21.15.12:0/1616960414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff34c069aa0 msgr2=0x7ff34c1017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:03.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.053+0000 7ff34b7fe640 1 --2- 172.21.15.12:0/1616960414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff34c069aa0 0x7ff34c1017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.053+0000 7ff34b7fe640 1 -- 172.21.15.12:0/1616960414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff34c106f50 msgr2=0x7ff34c101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:03.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.053+0000 7ff34b7fe640 1 --2- 172.21.15.12:0/1616960414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff34c106f50 0x7ff34c101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.053+0000 7ff34b7fe640 1 -- 172.21.15.12:0/1616960414 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34c1a6b50 con 0x7ff34c0690e0 2024-04-22T13:25:03.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.053+0000 7ff34b7fe640 1 --2- 172.21.15.12:0/1616960414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff34c0690e0 0x7ff34c101270 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00d770 tx=0x7ff33c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:03.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.053+0000 7ff348ff9640 1 -- 172.21.15.12:0/1616960414 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c017070 con 0x7ff34c0690e0 2024-04-22T13:25:03.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.053+0000 7ff348ff9640 1 -- 172.21.15.12:0/1616960414 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff33c0054d0 con 0x7ff34c0690e0 2024-04-22T13:25:03.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.053+0000 7ff352021640 1 -- 172.21.15.12:0/1616960414 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff34c1a6e40 con 0x7ff34c0690e0 2024-04-22T13:25:03.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.053+0000 7ff348ff9640 1 -- 172.21.15.12:0/1616960414 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c004c30 con 0x7ff34c0690e0 2024-04-22T13:25:03.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.053+0000 7ff352021640 1 -- 172.21.15.12:0/1616960414 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff34c1a72b0 con 0x7ff34c0690e0 2024-04-22T13:25:03.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.054+0000 7ff352021640 1 -- 172.21.15.12:0/1616960414 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff34c068de0 con 0x7ff34c0690e0 2024-04-22T13:25:03.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.055+0000 7ff348ff9640 1 -- 172.21.15.12:0/1616960414 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7ff33c004df0 con 0x7ff34c0690e0 2024-04-22T13:25:03.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.056+0000 7ff348ff9640 1 --2- 172.21.15.12:0/1616960414 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff3340a3ed0 0x7ff3340a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.056+0000 7ff348ff9640 1 -- 172.21.15.12:0/1616960414 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7ff33c0d0440 con 0x7ff34c0690e0 2024-04-22T13:25:03.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.056+0000 7ff34affd640 1 --2- 172.21.15.12:0/1616960414 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff3340a3ed0 0x7ff3340a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.057+0000 7ff34affd640 1 --2- 172.21.15.12:0/1616960414 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff3340a3ed0 0x7ff3340a6390 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff34c102d00 tx=0x7ff34000b390 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:03.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.061+0000 7ff348ff9640 1 -- 172.21.15.12:0/1616960414 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff33c09c2a0 con 0x7ff34c0690e0 2024-04-22T13:25:03.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.299+0000 7ff352021640 1 -- 172.21.15.12:0/1616960414 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff34c02d5c0 con 0x7ff34c0690e0 2024-04-22T13:25:03.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.300+0000 7ff348ff9640 1 -- 172.21.15.12:0/1616960414 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7ff33c0a05a0 con 0x7ff34c0690e0 2024-04-22T13:25:03.296 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:25:03.296 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:25:03.296 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:25:03.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.302+0000 7ff352021640 1 -- 172.21.15.12:0/1616960414 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff3340a3ed0 msgr2=0x7ff3340a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:03.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.302+0000 7ff352021640 1 --2- 172.21.15.12:0/1616960414 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff3340a3ed0 0x7ff3340a6390 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff34c102d00 tx=0x7ff34000b390 comp rx=0 tx=0).stop 2024-04-22T13:25:03.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.302+0000 7ff352021640 1 -- 172.21.15.12:0/1616960414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff34c0690e0 msgr2=0x7ff34c101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:03.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.302+0000 7ff352021640 1 --2- 172.21.15.12:0/1616960414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff34c0690e0 0x7ff34c101270 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff33c00d770 tx=0x7ff33c00dc30 comp rx=0 tx=0).stop 2024-04-22T13:25:03.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.302+0000 7ff352021640 1 -- 172.21.15.12:0/1616960414 shutdown_connections 2024-04-22T13:25:03.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.302+0000 7ff352021640 1 --2- 172.21.15.12:0/1616960414 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff3340a3ed0 0x7ff3340a6390 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.302+0000 7ff352021640 1 --2- 172.21.15.12:0/1616960414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff34c106f50 0x7ff34c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.302+0000 7ff352021640 1 --2- 172.21.15.12:0/1616960414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff34c069aa0 0x7ff34c1017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.302+0000 7ff352021640 1 --2- 172.21.15.12:0/1616960414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff34c0690e0 0x7ff34c101270 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.302+0000 7ff352021640 1 -- 172.21.15.12:0/1616960414 >> 172.21.15.12:0/1616960414 conn(0x7ff34c077e30 msgr2=0x7ff34c0fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:03.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.302+0000 7ff352021640 1 -- 172.21.15.12:0/1616960414 shutdown_connections 2024-04-22T13:25:03.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.302+0000 7ff352021640 1 -- 172.21.15.12:0/1616960414 wait complete. 2024-04-22T13:25:03.309 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db --set-id fab6558f --release --await' 2024-04-22T13:25:03.310 DEBUG:teuthology.orchestra.run.smithi012:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24509 quiesce db --set-id fab6558f --release --await 2024-04-22T13:25:03.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.425+0000 7f1916fef640 1 -- 172.21.15.12:0/3167692022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19101031d0 msgr2=0x7f19101035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:03.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.425+0000 7f1916fef640 1 --2- 172.21.15.12:0/3167692022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19101031d0 0x7f19101035b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1904009900 tx=0x7f190402bd60 comp rx=0 tx=0).stop 2024-04-22T13:25:03.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.425+0000 7f1916fef640 1 -- 172.21.15.12:0/3167692022 shutdown_connections 2024-04-22T13:25:03.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.425+0000 7f1916fef640 1 --2- 172.21.15.12:0/3167692022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f191010c510 0x7f191010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.425+0000 7f1916fef640 1 --2- 172.21.15.12:0/3167692022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1910103af0 0x7f191010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.425+0000 7f1916fef640 1 --2- 172.21.15.12:0/3167692022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19101031d0 0x7f19101035b0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.425+0000 7f1916fef640 1 -- 172.21.15.12:0/3167692022 >> 172.21.15.12:0/3167692022 conn(0x7f1910077f50 msgr2=0x7f1910078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:03.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.425+0000 7f1916fef640 1 -- 172.21.15.12:0/3167692022 shutdown_connections 2024-04-22T13:25:03.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.425+0000 7f1916fef640 1 -- 172.21.15.12:0/3167692022 wait complete. 2024-04-22T13:25:03.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.426+0000 7f1916fef640 1 Processor -- start 2024-04-22T13:25:03.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.426+0000 7f1916fef640 1 -- start start 2024-04-22T13:25:03.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.426+0000 7f1916fef640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19101031d0 0x7f191019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.426+0000 7f1916fef640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1910103af0 0x7f191019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.426+0000 7f1916fef640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f191010c510 0x7f1910198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.426+0000 7f1916fef640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1910112180 con 0x7f19101031d0 2024-04-22T13:25:03.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.426+0000 7f1916fef640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1910112000 con 0x7f1910103af0 2024-04-22T13:25:03.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.426+0000 7f1916fef640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1910112300 con 0x7f191010c510 2024-04-22T13:25:03.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.426+0000 7f1914d8b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19101031d0 0x7f191019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.426+0000 7f1914d8b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19101031d0 0x7f191019e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40128/0 (socket says 172.21.15.12:40128) 2024-04-22T13:25:03.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.426+0000 7f1914d8b640 1 -- 172.21.15.12:0/3454676339 learned_addr learned my addr 172.21.15.12:0/3454676339 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:03.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.426+0000 7f190ffff640 1 --2- 172.21.15.12:0/3454676339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1910103af0 0x7f191019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.427+0000 7f191558c640 1 --2- 172.21.15.12:0/3454676339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f191010c510 0x7f1910198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.427+0000 7f1914d8b640 1 -- 172.21.15.12:0/3454676339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f191010c510 msgr2=0x7f1910198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:03.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.427+0000 7f1914d8b640 1 --2- 172.21.15.12:0/3454676339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f191010c510 0x7f1910198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.427+0000 7f1914d8b640 1 -- 172.21.15.12:0/3454676339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1910103af0 msgr2=0x7f191019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:03.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.427+0000 7f1914d8b640 1 --2- 172.21.15.12:0/3454676339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1910103af0 0x7f191019e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.427+0000 7f1914d8b640 1 -- 172.21.15.12:0/3454676339 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1910198d10 con 0x7f19101031d0 2024-04-22T13:25:03.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.427+0000 7f1914d8b640 1 --2- 172.21.15.12:0/3454676339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19101031d0 0x7f191019e3e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1904005940 tx=0x7f190402d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:03.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.427+0000 7f190dffb640 1 -- 172.21.15.12:0/3454676339 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1904043020 con 0x7f19101031d0 2024-04-22T13:25:03.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.427+0000 7f1916fef640 1 -- 172.21.15.12:0/3454676339 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1910198fa0 con 0x7f19101031d0 2024-04-22T13:25:03.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.427+0000 7f190dffb640 1 -- 172.21.15.12:0/3454676339 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f190402dd20 con 0x7f19101031d0 2024-04-22T13:25:03.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.427+0000 7f190dffb640 1 -- 172.21.15.12:0/3454676339 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1904003b80 con 0x7f19101031d0 2024-04-22T13:25:03.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.427+0000 7f1916fef640 1 -- 172.21.15.12:0/3454676339 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19101a52a0 con 0x7f19101031d0 2024-04-22T13:25:03.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.430+0000 7f190dffb640 1 -- 172.21.15.12:0/3454676339 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f1904046020 con 0x7f19101031d0 2024-04-22T13:25:03.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.430+0000 7f1919bd6740 1 Processor -- start 2024-04-22T13:25:03.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.430+0000 7f1919bd6740 1 -- start start 2024-04-22T13:25:03.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.431+0000 7f1919bd6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d600c9acf0 0x55d600c9b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.431+0000 7f1919bd6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d600c9b6a0 0x55d600ca86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.431+0000 7f1919bd6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d600ca8c20 0x55d600cab010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.431+0000 7f1919bd6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d600c00500 con 0x55d600c9b6a0 2024-04-22T13:25:03.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.431+0000 7f1919bd6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d600ba66a0 con 0x55d600ca8c20 2024-04-22T13:25:03.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.431+0000 7f1919bd6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d600bfc4c0 con 0x55d600c9acf0 2024-04-22T13:25:03.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.431+0000 7f1914d8b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d600c9b6a0 0x55d600ca86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.431+0000 7f1914d8b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d600c9b6a0 0x55d600ca86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40132/0 (socket says 172.21.15.12:40132) 2024-04-22T13:25:03.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.431+0000 7f1914d8b640 1 -- 172.21.15.12:0/1155117162 learned_addr learned my addr 172.21.15.12:0/1155117162 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:03.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.431+0000 7f190dffb640 1 --2- 172.21.15.12:0/3454676339 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f19000a3ed0 0x7f19000a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.431+0000 7f191558c640 1 --2- 172.21.15.12:0/1155117162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d600c9acf0 0x55d600c9b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.431+0000 7f190ffff640 1 --2- 172.21.15.12:0/1155117162 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d600ca8c20 0x55d600cab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.431+0000 7f191558c640 1 --2- 172.21.15.12:0/3454676339 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f19000a3ed0 0x7f19000a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.431+0000 7f190dffb640 1 -- 172.21.15.12:0/3454676339 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f19040f2c00 con 0x7f19101031d0 2024-04-22T13:25:03.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.431+0000 7f1914d8b640 1 -- 172.21.15.12:0/1155117162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d600c9acf0 msgr2=0x55d600c9b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:03.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.431+0000 7f1914d8b640 1 --2- 172.21.15.12:0/1155117162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d600c9acf0 0x55d600c9b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.431+0000 7f1914d8b640 1 -- 172.21.15.12:0/1155117162 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d600ca8c20 msgr2=0x55d600cab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:03.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.431+0000 7f1914d8b640 1 --2- 172.21.15.12:0/1155117162 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d600ca8c20 0x55d600cab010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.431+0000 7f1914d8b640 1 -- 172.21.15.12:0/1155117162 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d600bc55e0 con 0x55d600c9b6a0 2024-04-22T13:25:03.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.431+0000 7f1914d8b640 1 --2- 172.21.15.12:0/1155117162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d600c9b6a0 0x55d600ca86e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f19040be7c0 tx=0x7f190402bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a48ede91ad33808b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:03.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.432+0000 7f191558c640 1 --2- 172.21.15.12:0/3454676339 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f19000a3ed0 0x7f19000a6390 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1908005c40 tx=0x7f1908005bd0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:03.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.432+0000 7f18faffd640 1 -- 172.21.15.12:0/1155117162 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19040be420 con 0x55d600c9b6a0 2024-04-22T13:25:03.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.432+0000 7f18faffd640 1 -- 172.21.15.12:0/1155117162 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19040bf1c0 con 0x55d600c9b6a0 2024-04-22T13:25:03.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.432+0000 7f18faffd640 1 -- 172.21.15.12:0/1155117162 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19040be420 con 0x55d600c9b6a0 2024-04-22T13:25:03.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.432+0000 7f1919bd6740 1 -- 172.21.15.12:0/1155117162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d600c9b6a0 msgr2=0x55d600ca86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:03.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.432+0000 7f1919bd6740 1 --2- 172.21.15.12:0/1155117162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d600c9b6a0 0x55d600ca86e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f19040be7c0 tx=0x7f190402bd60 comp rx=0 tx=0).stop 2024-04-22T13:25:03.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.432+0000 7f1919bd6740 1 -- 172.21.15.12:0/1155117162 shutdown_connections 2024-04-22T13:25:03.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.432+0000 7f1919bd6740 1 --2- 172.21.15.12:0/1155117162 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d600ca8c20 0x55d600cab010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.433+0000 7f1919bd6740 1 --2- 172.21.15.12:0/1155117162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d600c9b6a0 0x55d600ca86e0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.433+0000 7f1919bd6740 1 --2- 172.21.15.12:0/1155117162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d600c9acf0 0x55d600c9b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.433+0000 7f1919bd6740 1 -- 172.21.15.12:0/1155117162 >> 172.21.15.12:0/1155117162 conn(0x55d600c95a40 msgr2=0x55d600c95e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:03.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.433+0000 7f1919bd6740 1 -- 172.21.15.12:0/1155117162 shutdown_connections 2024-04-22T13:25:03.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.433+0000 7f1919bd6740 1 -- 172.21.15.12:0/1155117162 wait complete. 2024-04-22T13:25:03.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.433+0000 7f1919bd6740 1 -- start start 2024-04-22T13:25:03.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.433+0000 7f1919bd6740 1 Processor -- start 2024-04-22T13:25:03.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.433+0000 7f1919bd6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d600c9acf0 0x55d600ca8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.433+0000 7f1919bd6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d600c9b6a0 0x55d600ca59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.433+0000 7f1919bd6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d600ca8c20 0x55d600ca6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.433+0000 7f1919bd6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d600cab7a0 con 0x55d600ca8c20 2024-04-22T13:25:03.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.433+0000 7f1919bd6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d600bb9a60 con 0x55d600c9acf0 2024-04-22T13:25:03.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.433+0000 7f1919bd6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d600cab920 con 0x55d600c9b6a0 2024-04-22T13:25:03.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.433+0000 7f1919bd6740 10 client.0 _ll_register_callbacks cb 0x55d600912110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:03.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.433+0000 7f191558c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d600ca8c20 0x55d600ca6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.434+0000 7f191558c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d600ca8c20 0x55d600ca6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40136/0 (socket says 172.21.15.12:40136) 2024-04-22T13:25:03.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.434+0000 7f191558c640 1 -- 172.21.15.12:0/1691246197 learned_addr learned my addr 172.21.15.12:0/1691246197 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:03.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.434+0000 7f190ffff640 1 --2- 172.21.15.12:0/1691246197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d600c9b6a0 0x55d600ca59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.434+0000 7f1914d8b640 1 --2- 172.21.15.12:0/1691246197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d600c9acf0 0x55d600ca8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.434+0000 7f191558c640 1 -- 172.21.15.12:0/1691246197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d600c9b6a0 msgr2=0x55d600ca59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:03.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.434+0000 7f191558c640 1 --2- 172.21.15.12:0/1691246197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d600c9b6a0 0x55d600ca59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.434+0000 7f191558c640 1 -- 172.21.15.12:0/1691246197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d600c9acf0 msgr2=0x55d600ca8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:03.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.434+0000 7f191558c640 1 --2- 172.21.15.12:0/1691246197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d600c9acf0 0x55d600ca8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.434+0000 7f191558c640 1 -- 172.21.15.12:0/1691246197 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d600912920 con 0x55d600ca8c20 2024-04-22T13:25:03.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.434+0000 7f1914d8b640 1 --2- 172.21.15.12:0/1691246197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d600c9acf0 0x55d600ca8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:03.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.434+0000 7f191558c640 1 --2- 172.21.15.12:0/1691246197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d600ca8c20 0x55d600ca6120 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1908014010 tx=0x7f1908004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e3ea698cd98940 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:03.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.434+0000 7f18effff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:03.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.434+0000 7f18effff640 1 -- 172.21.15.12:0/1691246197 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1908024030 con 0x55d600ca8c20 2024-04-22T13:25:03.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.434+0000 7f18effff640 1 -- 172.21.15.12:0/1691246197 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f1908020740 con 0x55d600ca8c20 2024-04-22T13:25:03.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.434+0000 7f1919bd6740 1 -- 172.21.15.12:0/1691246197 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19040bf1c0 con 0x55d600ca8c20 2024-04-22T13:25:03.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.434+0000 7f18effff640 1 -- 172.21.15.12:0/1691246197 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f190802d020 con 0x55d600ca8c20 2024-04-22T13:25:03.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.434+0000 7f1919bd6740 10 client.16056 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:03.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.434+0000 7f1919bd6740 1 -- 172.21.15.12:0/1691246197 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d6009ded30 con 0x55d600ca8c20 2024-04-22T13:25:03.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.435+0000 7f18effff640 1 -- 172.21.15.12:0/1691246197 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19040bf1c0 con 0x55d600ca8c20 2024-04-22T13:25:03.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.435+0000 7f18effff640 1 -- 172.21.15.12:0/1691246197 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f190801e040 con 0x55d600ca8c20 2024-04-22T13:25:03.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.435+0000 7f1919bd6740 10 client.16056 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:03.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.435+0000 7f1919bd6740 10 client.16056 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-22T13:25:03.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.435+0000 7f1919bd6740 20 client.16056 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:03.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.435+0000 7f1919bd6740 1 --2- 172.21.15.12:0/1691246197 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d600cb8a30 0x55d600cbaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.435+0000 7f1919bd6740 4 client.16056 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:25:03.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.435+0000 7f1919bd6740 1 -- 172.21.15.12:0/1691246197 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d600bd3a10 con 0x55d600cb8a30 2024-04-22T13:25:03.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.435+0000 7f1914d8b640 1 --2- 172.21.15.12:0/1691246197 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d600cb8a30 0x55d600cbaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.435+0000 7f18effff640 10 client.16056 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:03.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.435+0000 7f1914d8b640 1 --2- 172.21.15.12:0/1691246197 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d600cb8a30 0x55d600cbaed0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23dce822e8a5698e server_cookie=a499baa53ef518b9 in_seq=0 out_seq=0 2024-04-22T13:25:03.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.436+0000 7f18effff640 1 -- 172.21.15.12:0/1691246197 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d600bd3a10 con 0x55d600cb8a30 2024-04-22T13:25:03.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.436+0000 7f18effff640 10 client.16056 handle_command_reply: tid=0 2024-04-22T13:25:03.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.436+0000 7f1919bd6740 1 client.16056 shutdown 2024-04-22T13:25:03.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.436+0000 7f1919bd6740 1 -- 172.21.15.12:0/1691246197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d600ca8c20 msgr2=0x55d600ca6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:03.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.436+0000 7f1919bd6740 1 --2- 172.21.15.12:0/1691246197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d600ca8c20 0x55d600ca6120 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1908014010 tx=0x7f1908004a40 comp rx=0 tx=0).stop 2024-04-22T13:25:03.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.436+0000 7f1919bd6740 1 -- 172.21.15.12:0/1691246197 shutdown_connections 2024-04-22T13:25:03.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.436+0000 7f1919bd6740 1 --2- 172.21.15.12:0/1691246197 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d600cb8a30 0x55d600cbaed0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.436+0000 7f1919bd6740 1 --2- 172.21.15.12:0/1691246197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d600ca8c20 0x55d600ca6120 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.436+0000 7f1919bd6740 1 --2- 172.21.15.12:0/1691246197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d600c9b6a0 0x55d600ca59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.436+0000 7f1919bd6740 1 --2- 172.21.15.12:0/1691246197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d600c9acf0 0x55d600ca8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.436+0000 7f1919bd6740 1 -- 172.21.15.12:0/1691246197 >> 172.21.15.12:0/1691246197 conn(0x55d600c95a40 msgr2=0x55d600c97790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:03.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.436+0000 7f18effff640 0 client.16056 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:03.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.437+0000 7f1919bd6740 1 -- 172.21.15.12:0/1691246197 shutdown_connections 2024-04-22T13:25:03.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.437+0000 7f1919bd6740 1 -- 172.21.15.12:0/1691246197 wait complete. 2024-04-22T13:25:03.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.437+0000 7f1919bd6740 20 client.16056 trim_cache size 0 max 16384 2024-04-22T13:25:03.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.456+0000 7f1919bd6740 1 Processor -- start 2024-04-22T13:25:03.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.456+0000 7f1919bd6740 1 -- start start 2024-04-22T13:25:03.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.457+0000 7f1919bd6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d600c9acf0 0x55d600c975a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.457+0000 7f1919bd6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d600c9b6a0 0x55d600cc07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.457+0000 7f1919bd6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d600ca8c20 0x55d600cc0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.457+0000 7f1919bd6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f19101ab950 con 0x55d600c9b6a0 2024-04-22T13:25:03.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.457+0000 7f1919bd6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f19101ad490 con 0x55d600ca8c20 2024-04-22T13:25:03.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.457+0000 7f1919bd6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f19101ac020 con 0x55d600c9acf0 2024-04-22T13:25:03.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.457+0000 7f190ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d600c9b6a0 0x55d600cc07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.457+0000 7f190ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d600c9b6a0 0x55d600cc07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40140/0 (socket says 172.21.15.12:40140) 2024-04-22T13:25:03.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.457+0000 7f190ffff640 1 -- 172.21.15.12:0/1803251292 learned_addr learned my addr 172.21.15.12:0/1803251292 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:03.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.457+0000 7f1914d8b640 1 --2- 172.21.15.12:0/1803251292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d600c9acf0 0x55d600c975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.457+0000 7f191558c640 1 --2- 172.21.15.12:0/1803251292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d600ca8c20 0x55d600cc0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.457+0000 7f190ffff640 1 -- 172.21.15.12:0/1803251292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d600c9acf0 msgr2=0x55d600c975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:03.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.457+0000 7f190ffff640 1 --2- 172.21.15.12:0/1803251292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d600c9acf0 0x55d600c975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.457+0000 7f190ffff640 1 -- 172.21.15.12:0/1803251292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d600ca8c20 msgr2=0x55d600cc0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:03.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.457+0000 7f190ffff640 1 --2- 172.21.15.12:0/1803251292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d600ca8c20 0x55d600cc0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.457+0000 7f190ffff640 1 -- 172.21.15.12:0/1803251292 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d600c9a400 con 0x55d600c9b6a0 2024-04-22T13:25:03.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.457+0000 7f190ffff640 1 --2- 172.21.15.12:0/1803251292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d600c9b6a0 0x55d600cc07a0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f18fc005d00 tx=0x7f18fc012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8524404c89640fba server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:03.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.457+0000 7f18fa7fc640 1 -- 172.21.15.12:0/1803251292 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc01f070 con 0x55d600c9b6a0 2024-04-22T13:25:03.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.457+0000 7f18fa7fc640 1 -- 172.21.15.12:0/1803251292 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18fc0035f0 con 0x55d600c9b6a0 2024-04-22T13:25:03.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.458+0000 7f18fa7fc640 1 -- 172.21.15.12:0/1803251292 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc010730 con 0x55d600c9b6a0 2024-04-22T13:25:03.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.458+0000 7f1919bd6740 1 -- 172.21.15.12:0/1803251292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d600c9b6a0 msgr2=0x55d600cc07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:03.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.458+0000 7f1919bd6740 1 --2- 172.21.15.12:0/1803251292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d600c9b6a0 0x55d600cc07a0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f18fc005d00 tx=0x7f18fc012820 comp rx=0 tx=0).stop 2024-04-22T13:25:03.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.458+0000 7f1919bd6740 1 -- 172.21.15.12:0/1803251292 shutdown_connections 2024-04-22T13:25:03.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.458+0000 7f1919bd6740 1 --2- 172.21.15.12:0/1803251292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d600ca8c20 0x55d600cc0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.458+0000 7f1919bd6740 1 --2- 172.21.15.12:0/1803251292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d600c9b6a0 0x55d600cc07a0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.458+0000 7f1919bd6740 1 --2- 172.21.15.12:0/1803251292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d600c9acf0 0x55d600c975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.458+0000 7f1919bd6740 1 -- 172.21.15.12:0/1803251292 >> 172.21.15.12:0/1803251292 conn(0x55d600c95a40 msgr2=0x55d600c96e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:03.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.458+0000 7f1919bd6740 1 -- 172.21.15.12:0/1803251292 shutdown_connections 2024-04-22T13:25:03.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.458+0000 7f1919bd6740 1 -- 172.21.15.12:0/1803251292 wait complete. 2024-04-22T13:25:03.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.459+0000 7f1919bd6740 1 -- start start 2024-04-22T13:25:03.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.459+0000 7f1919bd6740 1 Processor -- start 2024-04-22T13:25:03.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.459+0000 7f1919bd6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d600c9acf0 0x55d600cb0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.459+0000 7f1919bd6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d600c9b6a0 0x55d600cab7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.459+0000 7f1919bd6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d600ca8c20 0x55d600cabce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.459+0000 7f1919bd6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f19101acdc0 con 0x55d600ca8c20 2024-04-22T13:25:03.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.459+0000 7f1919bd6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f19101ac6f0 con 0x55d600c9acf0 2024-04-22T13:25:03.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.459+0000 7f1919bd6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f19101a9930 con 0x55d600c9b6a0 2024-04-22T13:25:03.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.459+0000 7f1919bd6740 10 client.0 _ll_register_callbacks cb 0x7f1908009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:03.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.459+0000 7f191558c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d600ca8c20 0x55d600cabce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.459+0000 7f191558c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d600ca8c20 0x55d600cabce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40156/0 (socket says 172.21.15.12:40156) 2024-04-22T13:25:03.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.459+0000 7f191558c640 1 -- 172.21.15.12:0/4201848145 learned_addr learned my addr 172.21.15.12:0/4201848145 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:03.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.459+0000 7f1914d8b640 1 --2- 172.21.15.12:0/4201848145 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d600c9acf0 0x55d600cb0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.459+0000 7f190ffff640 1 --2- 172.21.15.12:0/4201848145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d600c9b6a0 0x55d600cab7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.459+0000 7f191558c640 1 -- 172.21.15.12:0/4201848145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d600c9b6a0 msgr2=0x55d600cab7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:03.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.459+0000 7f191558c640 1 --2- 172.21.15.12:0/4201848145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d600c9b6a0 0x55d600cab7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.460+0000 7f191558c640 1 -- 172.21.15.12:0/4201848145 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d600c9acf0 msgr2=0x55d600cb0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:03.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.460+0000 7f191558c640 1 --2- 172.21.15.12:0/4201848145 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d600c9acf0 0x55d600cb0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.460+0000 7f191558c640 1 -- 172.21.15.12:0/4201848145 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d600cac220 con 0x55d600ca8c20 2024-04-22T13:25:03.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.460+0000 7f1914d8b640 1 --2- 172.21.15.12:0/4201848145 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d600c9acf0 0x55d600cb0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:03.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.460+0000 7f191558c640 1 --2- 172.21.15.12:0/4201848145 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d600ca8c20 0x55d600cabce0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1908032b20 tx=0x7f190800de90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7427c1cdebd2e7e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:03.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.460+0000 7f18effff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:03.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.460+0000 7f18effff640 1 -- 172.21.15.12:0/4201848145 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1908037060 con 0x55d600ca8c20 2024-04-22T13:25:03.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.460+0000 7f18effff640 1 -- 172.21.15.12:0/4201848145 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f1908004cb0 con 0x55d600ca8c20 2024-04-22T13:25:03.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.460+0000 7f1919bd6740 1 -- 172.21.15.12:0/4201848145 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18fc0035f0 con 0x55d600ca8c20 2024-04-22T13:25:03.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.460+0000 7f18effff640 1 -- 172.21.15.12:0/4201848145 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1908024070 con 0x55d600ca8c20 2024-04-22T13:25:03.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.460+0000 7f1919bd6740 10 client.16062 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:03.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.460+0000 7f1919bd6740 1 -- 172.21.15.12:0/4201848145 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d600d1fd40 con 0x55d600ca8c20 2024-04-22T13:25:03.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.460+0000 7f18effff640 1 -- 172.21.15.12:0/4201848145 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18fc0035f0 con 0x55d600ca8c20 2024-04-22T13:25:03.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.460+0000 7f18effff640 1 -- 172.21.15.12:0/4201848145 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1908020ca0 con 0x55d600ca8c20 2024-04-22T13:25:03.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.461+0000 7f1919bd6740 10 client.16062 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:03.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.461+0000 7f1919bd6740 10 client.16062 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-22T13:25:03.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.461+0000 7f1919bd6740 20 client.16062 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:03.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.461+0000 7f1919bd6740 1 --2- 172.21.15.12:0/4201848145 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d600d20450 0x55d600d22870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.461+0000 7f1919bd6740 4 client.16062 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "fab6558f", "release": true, "await": true}] 2024-04-22T13:25:03.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.461+0000 7f1919bd6740 1 -- 172.21.15.12:0/4201848145 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "quiesce db", "set_id": "fab6558f", "release": true, "await": true}) -- 0x55d600c0df20 con 0x55d600d20450 2024-04-22T13:25:03.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.461+0000 7f1914d8b640 1 --2- 172.21.15.12:0/4201848145 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d600d20450 0x55d600d22870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.461+0000 7f18effff640 10 client.16062 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:03.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.461+0000 7f1914d8b640 1 --2- 172.21.15.12:0/4201848145 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d600d20450 0x55d600d22870 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cf2d52761469c54 server_cookie=308a32cf3ff39392 in_seq=0 out_seq=0 2024-04-22T13:25:03.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.463+0000 7f18effff640 1 -- 172.21.15.12:0/4201848145 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55d600c0df20 con 0x55d600d20450 2024-04-22T13:25:03.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.463+0000 7f18effff640 10 client.16062 handle_command_reply: tid=0 2024-04-22T13:25:03.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.463+0000 7f1919bd6740 1 client.16062 shutdown 2024-04-22T13:25:03.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.464+0000 7f1919bd6740 1 -- 172.21.15.12:0/4201848145 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d600ca8c20 msgr2=0x55d600cabce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:03.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.464+0000 7f1919bd6740 1 --2- 172.21.15.12:0/4201848145 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d600ca8c20 0x55d600cabce0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1908032b20 tx=0x7f190800de90 comp rx=0 tx=0).stop 2024-04-22T13:25:03.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.464+0000 7f1919bd6740 1 -- 172.21.15.12:0/4201848145 shutdown_connections 2024-04-22T13:25:03.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.464+0000 7f1919bd6740 1 --2- 172.21.15.12:0/4201848145 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d600d20450 0x55d600d22870 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.464+0000 7f1919bd6740 1 --2- 172.21.15.12:0/4201848145 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d600ca8c20 0x55d600cabce0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.464+0000 7f18effff640 0 client.16062 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:03.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.464+0000 7f1919bd6740 1 --2- 172.21.15.12:0/4201848145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d600c9b6a0 0x55d600cab7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.464+0000 7f1919bd6740 1 --2- 172.21.15.12:0/4201848145 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d600c9acf0 0x55d600cb0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.464+0000 7f1919bd6740 1 -- 172.21.15.12:0/4201848145 >> 172.21.15.12:0/4201848145 conn(0x55d600c95a40 msgr2=0x55d600c94b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:03.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.464+0000 7f1919bd6740 1 -- 172.21.15.12:0/4201848145 shutdown_connections 2024-04-22T13:25:03.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.464+0000 7f1919bd6740 1 -- 172.21.15.12:0/4201848145 wait complete. 2024-04-22T13:25:03.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.464+0000 7f1919bd6740 20 client.16062 trim_cache size 0 max 16384 2024-04-22T13:25:03.464 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:25:03.464 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 28, 2024-04-22T13:25:03.464 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24509, 2024-04-22T13:25:03.464 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 4, 2024-04-22T13:25:03.464 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-04-22T13:25:03.464 INFO:teuthology.orchestra.run.smithi012.stdout: "fab6558f": { 2024-04-22T13:25:03.464 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 4, 2024-04-22T13:25:03.464 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-04-22T13:25:03.465 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-04-22T13:25:03.465 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-04-22T13:25:03.465 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-04-22T13:25:03.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:25:03.465 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 60.0, 2024-04-22T13:25:03.465 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 85.3, 2024-04-22T13:25:03.465 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-04-22T13:25:03.465 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-04-22T13:25:03.465 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-04-22T13:25:03.465 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-04-22T13:25:03.465 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-04-22T13:25:03.465 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-04-22T13:25:03.465 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:25:03.465 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:25:03.466 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:25:03.466 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:25:03.466 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:25:03.466 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:25:03.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.464+0000 7f1916fef640 1 -- 172.21.15.12:0/3454676339 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f19000a3ed0 msgr2=0x7f19000a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:03.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.464+0000 7f1916fef640 1 --2- 172.21.15.12:0/3454676339 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f19000a3ed0 0x7f19000a6390 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1908005c40 tx=0x7f1908005bd0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.464+0000 7f1916fef640 1 -- 172.21.15.12:0/3454676339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19101031d0 msgr2=0x7f191019e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:03.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.464+0000 7f1916fef640 1 --2- 172.21.15.12:0/3454676339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19101031d0 0x7f191019e3e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1904005940 tx=0x7f190402d920 comp rx=0 tx=0).stop 2024-04-22T13:25:03.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.465+0000 7f1916fef640 1 -- 172.21.15.12:0/3454676339 shutdown_connections 2024-04-22T13:25:03.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.465+0000 7f1916fef640 1 --2- 172.21.15.12:0/3454676339 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f19000a3ed0 0x7f19000a6390 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.465+0000 7f1916fef640 1 --2- 172.21.15.12:0/3454676339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f191010c510 0x7f1910198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.465+0000 7f1916fef640 1 --2- 172.21.15.12:0/3454676339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1910103af0 0x7f191019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.465+0000 7f1916fef640 1 --2- 172.21.15.12:0/3454676339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19101031d0 0x7f191019e3e0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.465+0000 7f1916fef640 1 -- 172.21.15.12:0/3454676339 >> 172.21.15.12:0/3454676339 conn(0x7f1910077f50 msgr2=0x7f19100fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:03.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.465+0000 7f1916fef640 1 -- 172.21.15.12:0/3454676339 shutdown_connections 2024-04-22T13:25:03.467 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.465+0000 7f1916fef640 1 -- 172.21.15.12:0/3454676339 wait complete. 2024-04-22T13:25:03.471 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set_id: fab6558f, seconds elapsed: 26.4 2024-04-22T13:25:03.471 INFO:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.35 2024-04-22T13:25:03.471 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:25:03.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.579+0000 7f08ba3ef640 1 -- 172.21.15.12:0/3190668605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08b4105e30 msgr2=0x7f08b4106210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:03.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.579+0000 7f08ba3ef640 1 --2- 172.21.15.12:0/3190668605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08b4105e30 0x7f08b4106210 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f08a4009930 tx=0x7f08a402bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:03.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.579+0000 7f08ba3ef640 1 -- 172.21.15.12:0/3190668605 shutdown_connections 2024-04-22T13:25:03.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.579+0000 7f08ba3ef640 1 --2- 172.21.15.12:0/3190668605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08b4101b60 0x7f08b410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.579+0000 7f08ba3ef640 1 --2- 172.21.15.12:0/3190668605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08b41011a0 0x7f08b4101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.579+0000 7f08ba3ef640 1 --2- 172.21.15.12:0/3190668605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08b4105e30 0x7f08b4106210 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.579+0000 7f08ba3ef640 1 -- 172.21.15.12:0/3190668605 >> 172.21.15.12:0/3190668605 conn(0x7f08b4077e30 msgr2=0x7f08b4078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:03.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.579+0000 7f08ba3ef640 1 -- 172.21.15.12:0/3190668605 shutdown_connections 2024-04-22T13:25:03.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.579+0000 7f08ba3ef640 1 -- 172.21.15.12:0/3190668605 wait complete. 2024-04-22T13:25:03.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.579+0000 7f08ba3ef640 1 Processor -- start 2024-04-22T13:25:03.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.579+0000 7f08ba3ef640 1 -- start start 2024-04-22T13:25:03.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.580+0000 7f08ba3ef640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08b41011a0 0x7f08b4197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.580+0000 7f08ba3ef640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08b4101b60 0x7f08b4197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.580+0000 7f08ba3ef640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08b4105e30 0x7f08b419bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.580+0000 7f08ba3ef640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f08b41110e0 con 0x7f08b4105e30 2024-04-22T13:25:03.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.580+0000 7f08ba3ef640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f08b4110f60 con 0x7f08b4101b60 2024-04-22T13:25:03.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.580+0000 7f08ba3ef640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f08b4111260 con 0x7f08b41011a0 2024-04-22T13:25:03.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.580+0000 7f08b898c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08b4105e30 0x7f08b419bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.580+0000 7f08b898c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08b4105e30 0x7f08b419bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40170/0 (socket says 172.21.15.12:40170) 2024-04-22T13:25:03.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.580+0000 7f08b898c640 1 -- 172.21.15.12:0/2167594738 learned_addr learned my addr 172.21.15.12:0/2167594738 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:03.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.580+0000 7f08b3fff640 1 --2- 172.21.15.12:0/2167594738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08b41011a0 0x7f08b4197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.580+0000 7f08b37fe640 1 --2- 172.21.15.12:0/2167594738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08b4101b60 0x7f08b4197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.580+0000 7f08b898c640 1 -- 172.21.15.12:0/2167594738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08b41011a0 msgr2=0x7f08b4197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:03.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.580+0000 7f08b898c640 1 --2- 172.21.15.12:0/2167594738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08b41011a0 0x7f08b4197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.580+0000 7f08b898c640 1 -- 172.21.15.12:0/2167594738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08b4101b60 msgr2=0x7f08b4197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:03.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.580+0000 7f08b898c640 1 --2- 172.21.15.12:0/2167594738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08b4101b60 0x7f08b4197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.580+0000 7f08b898c640 1 -- 172.21.15.12:0/2167594738 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b419c630 con 0x7f08b4105e30 2024-04-22T13:25:03.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.581+0000 7f08b898c640 1 --2- 172.21.15.12:0/2167594738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08b4105e30 0x7f08b419bf00 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f08ac00eb10 tx=0x7f08ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:03.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.581+0000 7f08b17fa640 1 -- 172.21.15.12:0/2167594738 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac004b20 con 0x7f08b4105e30 2024-04-22T13:25:03.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.581+0000 7f08b17fa640 1 -- 172.21.15.12:0/2167594738 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08ac004cc0 con 0x7f08b4105e30 2024-04-22T13:25:03.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.581+0000 7f08ba3ef640 1 -- 172.21.15.12:0/2167594738 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b419c920 con 0x7f08b4105e30 2024-04-22T13:25:03.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.581+0000 7f08b17fa640 1 -- 172.21.15.12:0/2167594738 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac010670 con 0x7f08b4105e30 2024-04-22T13:25:03.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.581+0000 7f08ba3ef640 1 -- 172.21.15.12:0/2167594738 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b41a41b0 con 0x7f08b4105e30 2024-04-22T13:25:03.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.582+0000 7f08ba3ef640 1 -- 172.21.15.12:0/2167594738 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08b4102570 con 0x7f08b4105e30 2024-04-22T13:25:03.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.583+0000 7f08b17fa640 1 -- 172.21.15.12:0/2167594738 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f08ac005020 con 0x7f08b4105e30 2024-04-22T13:25:03.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.584+0000 7f08b17fa640 1 --2- 172.21.15.12:0/2167594738 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f089c0a3ed0 0x7f089c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.584+0000 7f08b3fff640 1 --2- 172.21.15.12:0/2167594738 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f089c0a3ed0 0x7f089c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.584+0000 7f08b17fa640 1 -- 172.21.15.12:0/2167594738 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f08ac0d1000 con 0x7f08b4105e30 2024-04-22T13:25:03.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.584+0000 7f08b3fff640 1 --2- 172.21.15.12:0/2167594738 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f089c0a3ed0 0x7f089c0a6390 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f08a4005ab0 tx=0x7f08a4005fd0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:03.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.588+0000 7f08b17fa640 1 -- 172.21.15.12:0/2167594738 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f08ac09d4b0 con 0x7f08b4105e30 2024-04-22T13:25:03.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.837+0000 7f08ba3ef640 1 -- 172.21.15.12:0/2167594738 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08b4109330 con 0x7f08b4105e30 2024-04-22T13:25:03.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.838+0000 7f08b17fa640 1 -- 172.21.15.12:0/2167594738 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7f08ac09cdb0 con 0x7f08b4105e30 2024-04-22T13:25:03.834 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:25:03.834 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:25:03.834 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:25:03.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.840+0000 7f08ba3ef640 1 -- 172.21.15.12:0/2167594738 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f089c0a3ed0 msgr2=0x7f089c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:03.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.840+0000 7f08ba3ef640 1 --2- 172.21.15.12:0/2167594738 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f089c0a3ed0 0x7f089c0a6390 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f08a4005ab0 tx=0x7f08a4005fd0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.840+0000 7f08ba3ef640 1 -- 172.21.15.12:0/2167594738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08b4105e30 msgr2=0x7f08b419bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:03.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.840+0000 7f08ba3ef640 1 --2- 172.21.15.12:0/2167594738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08b4105e30 0x7f08b419bf00 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f08ac00eb10 tx=0x7f08ac00c2d0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.840+0000 7f08ba3ef640 1 -- 172.21.15.12:0/2167594738 shutdown_connections 2024-04-22T13:25:03.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.840+0000 7f08ba3ef640 1 --2- 172.21.15.12:0/2167594738 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f089c0a3ed0 0x7f089c0a6390 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.840+0000 7f08ba3ef640 1 --2- 172.21.15.12:0/2167594738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08b4105e30 0x7f08b419bf00 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.840+0000 7f08ba3ef640 1 --2- 172.21.15.12:0/2167594738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08b4101b60 0x7f08b4197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.840+0000 7f08ba3ef640 1 --2- 172.21.15.12:0/2167594738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08b41011a0 0x7f08b4197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.840+0000 7f08ba3ef640 1 -- 172.21.15.12:0/2167594738 >> 172.21.15.12:0/2167594738 conn(0x7f08b4077e30 msgr2=0x7f08b40fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:03.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.840+0000 7f08ba3ef640 1 -- 172.21.15.12:0/2167594738 shutdown_connections 2024-04-22T13:25:03.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.840+0000 7f08ba3ef640 1 -- 172.21.15.12:0/2167594738 wait complete. 2024-04-22T13:25:03.847 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db / --timeout 60 --expiration 85.35 --await' 2024-04-22T13:25:03.847 DEBUG:teuthology.orchestra.run.smithi012:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24509 quiesce db / --timeout 60 --expiration 85.35 --await 2024-04-22T13:25:03.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.963+0000 7f3fdb821640 1 -- 172.21.15.12:0/1741759860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fd4101a50 msgr2=0x7f3fd4105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:03.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.963+0000 7f3fdb821640 1 --2- 172.21.15.12:0/1741759860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fd4101a50 0x7f3fd4105a90 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4009900 tx=0x7f3fc402bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:03.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.963+0000 7f3fdb821640 1 -- 172.21.15.12:0/1741759860 shutdown_connections 2024-04-22T13:25:03.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.963+0000 7f3fdb821640 1 --2- 172.21.15.12:0/1741759860 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3fd4106170 0x7f3fd410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.963+0000 7f3fdb821640 1 --2- 172.21.15.12:0/1741759860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fd4101a50 0x7f3fd4105a90 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.963+0000 7f3fdb821640 1 --2- 172.21.15.12:0/1741759860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fd41010a0 0x7f3fd4101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.963+0000 7f3fdb821640 1 -- 172.21.15.12:0/1741759860 >> 172.21.15.12:0/1741759860 conn(0x7f3fd4077f50 msgr2=0x7f3fd4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:03.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.964+0000 7f3fdb821640 1 -- 172.21.15.12:0/1741759860 shutdown_connections 2024-04-22T13:25:03.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.964+0000 7f3fdb821640 1 -- 172.21.15.12:0/1741759860 wait complete. 2024-04-22T13:25:03.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.964+0000 7f3fdb821640 1 Processor -- start 2024-04-22T13:25:03.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.964+0000 7f3fdb821640 1 -- start start 2024-04-22T13:25:03.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.964+0000 7f3fdb821640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3fd41010a0 0x7f3fd410e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.964+0000 7f3fdb821640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fd4106170 0x7f3fd410ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.964+0000 7f3fdb821640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fd410f150 0x7f3fd41a6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.964+0000 7f3fdb821640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd41121f0 con 0x7f3fd4106170 2024-04-22T13:25:03.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.964+0000 7f3fdb821640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd4112070 con 0x7f3fd41010a0 2024-04-22T13:25:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.964+0000 7f3fdb821640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd4112370 con 0x7f3fd410f150 2024-04-22T13:25:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.965+0000 7f3fd8dbc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fd4106170 0x7f3fd410ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.965+0000 7f3fd8dbc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fd4106170 0x7f3fd410ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40188/0 (socket says 172.21.15.12:40188) 2024-04-22T13:25:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.965+0000 7f3fd8dbc640 1 -- 172.21.15.12:0/1911297874 learned_addr learned my addr 172.21.15.12:0/1911297874 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.965+0000 7f3fd9dbe640 1 --2- 172.21.15.12:0/1911297874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fd410f150 0x7f3fd41a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.965+0000 7f3fd95bd640 1 --2- 172.21.15.12:0/1911297874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3fd41010a0 0x7f3fd410e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.965+0000 7f3fd8dbc640 1 -- 172.21.15.12:0/1911297874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fd410f150 msgr2=0x7f3fd41a6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.965+0000 7f3fd8dbc640 1 --2- 172.21.15.12:0/1911297874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fd410f150 0x7f3fd41a6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.965+0000 7f3fd8dbc640 1 -- 172.21.15.12:0/1911297874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3fd41010a0 msgr2=0x7f3fd410e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.965+0000 7f3fd8dbc640 1 --2- 172.21.15.12:0/1911297874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3fd41010a0 0x7f3fd410e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.965+0000 7f3fd8dbc640 1 -- 172.21.15.12:0/1911297874 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fd41a7000 con 0x7f3fd4106170 2024-04-22T13:25:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.965+0000 7f3fd9dbe640 1 --2- 172.21.15.12:0/1911297874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fd410f150 0x7f3fd41a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:25:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.965+0000 7f3fd8dbc640 1 --2- 172.21.15.12:0/1911297874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fd4106170 0x7f3fd410ec10 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4036040 tx=0x7f3fc40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.965+0000 7f3fca7fc640 1 -- 172.21.15.12:0/1911297874 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc4043020 con 0x7f3fd4106170 2024-04-22T13:25:03.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.965+0000 7f3fdb821640 1 -- 172.21.15.12:0/1911297874 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fd41a7290 con 0x7f3fd4106170 2024-04-22T13:25:03.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.965+0000 7f3fca7fc640 1 -- 172.21.15.12:0/1911297874 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fc402cb70 con 0x7f3fd4106170 2024-04-22T13:25:03.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.965+0000 7f3fca7fc640 1 -- 172.21.15.12:0/1911297874 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc4003bf0 con 0x7f3fd4106170 2024-04-22T13:25:03.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.965+0000 7f3fdb821640 1 -- 172.21.15.12:0/1911297874 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fd41a77e0 con 0x7f3fd4106170 2024-04-22T13:25:03.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.968+0000 7f3fca7fc640 1 -- 172.21.15.12:0/1911297874 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f3fc4046020 con 0x7f3fd4106170 2024-04-22T13:25:03.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.968+0000 7f3fde40b740 1 Processor -- start 2024-04-22T13:25:03.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.968+0000 7f3fde40b740 1 -- start start 2024-04-22T13:25:03.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.968+0000 7f3fde40b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571aa4769c0 0x5571aa476da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.968+0000 7f3fde40b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571aa4772e0 0x5571aa47f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.968+0000 7f3fde40b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571aa47fec0 0x5571aa4822b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.968+0000 7f3fde40b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5571aa3d70b0 con 0x5571aa4769c0 2024-04-22T13:25:03.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.968+0000 7f3fde40b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5571aa388680 con 0x5571aa47fec0 2024-04-22T13:25:03.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.968+0000 7f3fde40b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5571aa394c60 con 0x5571aa4772e0 2024-04-22T13:25:03.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.969+0000 7f3fd9dbe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571aa4769c0 0x5571aa476da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.969+0000 7f3fca7fc640 1 --2- 172.21.15.12:0/1911297874 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3fc00a3f00 0x7f3fc00a63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.969+0000 7f3fd9dbe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571aa4769c0 0x5571aa476da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40198/0 (socket says 172.21.15.12:40198) 2024-04-22T13:25:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.969+0000 7f3fd9dbe640 1 -- 172.21.15.12:0/1994448579 learned_addr learned my addr 172.21.15.12:0/1994448579 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.969+0000 7f3fd8dbc640 1 --2- 172.21.15.12:0/1994448579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571aa47fec0 0x5571aa4822b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.969+0000 7f3fd95bd640 1 --2- 172.21.15.12:0/1994448579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571aa4772e0 0x5571aa47f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.969+0000 7f3fd9dbe640 1 --2- 172.21.15.12:0/1911297874 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3fc00a3f00 0x7f3fc00a63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.969+0000 7f3fd9dbe640 1 -- 172.21.15.12:0/1994448579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571aa4772e0 msgr2=0x5571aa47f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.969+0000 7f3fd9dbe640 1 --2- 172.21.15.12:0/1994448579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571aa4772e0 0x5571aa47f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.969+0000 7f3fd9dbe640 1 -- 172.21.15.12:0/1994448579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571aa47fec0 msgr2=0x5571aa4822b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.969+0000 7f3fd9dbe640 1 --2- 172.21.15.12:0/1994448579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571aa47fec0 0x5571aa4822b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.969+0000 7f3fd9dbe640 1 -- 172.21.15.12:0/1994448579 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571aa1b9c90 con 0x5571aa4769c0 2024-04-22T13:25:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.969+0000 7f3fd8dbc640 1 --2- 172.21.15.12:0/1994448579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571aa47fec0 0x5571aa4822b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.969+0000 7f3fca7fc640 1 -- 172.21.15.12:0/1911297874 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f3fc40f2bc0 con 0x7f3fd4106170 2024-04-22T13:25:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.969+0000 7f3fd9dbe640 1 --2- 172.21.15.12:0/1994448579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571aa4769c0 0x5571aa476da0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc0139e0 tx=0x7f3fcc013ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4805404f8171eb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.969+0000 7f3fbf7fe640 1 -- 172.21.15.12:0/1994448579 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc01fb80 con 0x5571aa4769c0 2024-04-22T13:25:03.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.969+0000 7f3fbf7fe640 1 -- 172.21.15.12:0/1994448579 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fcc01fd20 con 0x5571aa4769c0 2024-04-22T13:25:03.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.969+0000 7f3fbf7fe640 1 -- 172.21.15.12:0/1994448579 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc0273c0 con 0x5571aa4769c0 2024-04-22T13:25:03.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.969+0000 7f3fd9dbe640 1 --2- 172.21.15.12:0/1911297874 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3fc00a3f00 0x7f3fc00a63c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc022000 tx=0x7f3fcc020000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:03.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.970+0000 7f3fde40b740 1 -- 172.21.15.12:0/1994448579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571aa4769c0 msgr2=0x5571aa476da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:03.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.970+0000 7f3fde40b740 1 --2- 172.21.15.12:0/1994448579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571aa4769c0 0x5571aa476da0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc0139e0 tx=0x7f3fcc013ea0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.970+0000 7f3fde40b740 1 -- 172.21.15.12:0/1994448579 shutdown_connections 2024-04-22T13:25:03.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.970+0000 7f3fde40b740 1 --2- 172.21.15.12:0/1994448579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571aa47fec0 0x5571aa4822b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.970+0000 7f3fde40b740 1 --2- 172.21.15.12:0/1994448579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571aa4772e0 0x5571aa47f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.970+0000 7f3fde40b740 1 --2- 172.21.15.12:0/1994448579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571aa4769c0 0x5571aa476da0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.970+0000 7f3fde40b740 1 -- 172.21.15.12:0/1994448579 >> 172.21.15.12:0/1994448579 conn(0x5571aa4710f0 msgr2=0x5571aa4714d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:03.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.971+0000 7f3fde40b740 1 -- 172.21.15.12:0/1994448579 shutdown_connections 2024-04-22T13:25:03.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.971+0000 7f3fde40b740 1 -- 172.21.15.12:0/1994448579 wait complete. 2024-04-22T13:25:03.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.971+0000 7f3fde40b740 1 -- start start 2024-04-22T13:25:03.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.971+0000 7f3fde40b740 1 Processor -- start 2024-04-22T13:25:03.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.972+0000 7f3fde40b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571aa4769c0 0x5571aa47cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.972+0000 7f3fde40b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571aa4772e0 0x5571aa47d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.972+0000 7f3fde40b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571aa47fec0 0x5571aa47d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.972+0000 7f3fde40b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5571aa3da400 con 0x5571aa47fec0 2024-04-22T13:25:03.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.972+0000 7f3fde40b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5571aa396840 con 0x5571aa4772e0 2024-04-22T13:25:03.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.972+0000 7f3fde40b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5571aa3df720 con 0x5571aa4769c0 2024-04-22T13:25:03.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.972+0000 7f3fde40b740 10 client.0 _ll_register_callbacks cb 0x5571aa3e7ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:03.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.972+0000 7f3fd9dbe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571aa47fec0 0x5571aa47d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.972+0000 7f3fd9dbe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571aa47fec0 0x5571aa47d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40208/0 (socket says 172.21.15.12:40208) 2024-04-22T13:25:03.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.972+0000 7f3fd9dbe640 1 -- 172.21.15.12:0/673992022 learned_addr learned my addr 172.21.15.12:0/673992022 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:03.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.972+0000 7f3fd95bd640 1 --2- 172.21.15.12:0/673992022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571aa4769c0 0x5571aa47cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.972+0000 7f3fd8dbc640 1 --2- 172.21.15.12:0/673992022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571aa4772e0 0x5571aa47d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.972+0000 7f3fd9dbe640 1 -- 172.21.15.12:0/673992022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571aa4769c0 msgr2=0x5571aa47cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:03.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.972+0000 7f3fd9dbe640 1 --2- 172.21.15.12:0/673992022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571aa4769c0 0x5571aa47cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.972+0000 7f3fd9dbe640 1 -- 172.21.15.12:0/673992022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571aa4772e0 msgr2=0x5571aa47d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:03.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.972+0000 7f3fd9dbe640 1 --2- 172.21.15.12:0/673992022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571aa4772e0 0x5571aa47d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.972+0000 7f3fd9dbe640 1 -- 172.21.15.12:0/673992022 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571aa21a420 con 0x5571aa47fec0 2024-04-22T13:25:03.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.972+0000 7f3fd95bd640 1 --2- 172.21.15.12:0/673992022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571aa4769c0 0x5571aa47cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:03.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.972+0000 7f3fd8dbc640 1 --2- 172.21.15.12:0/673992022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571aa4772e0 0x5571aa47d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:03.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.972+0000 7f3fd9dbe640 1 --2- 172.21.15.12:0/673992022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571aa47fec0 0x5571aa47d9a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc009fa0 tx=0x7f3fcc013620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eeba35f84b8c1b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:03.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.972+0000 7f3fbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:03.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.972+0000 7f3fbcff9640 1 -- 172.21.15.12:0/673992022 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc027440 con 0x5571aa47fec0 2024-04-22T13:25:03.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.972+0000 7f3fde40b740 1 -- 172.21.15.12:0/673992022 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fcc01fd20 con 0x5571aa47fec0 2024-04-22T13:25:03.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.972+0000 7f3fbcff9640 1 -- 172.21.15.12:0/673992022 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f3fcc003970 con 0x5571aa47fec0 2024-04-22T13:25:03.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.973+0000 7f3fbcff9640 1 -- 172.21.15.12:0/673992022 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3fcc01fb80 con 0x5571aa47fec0 2024-04-22T13:25:03.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.973+0000 7f3fde40b740 10 client.16080 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:03.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.973+0000 7f3fde40b740 1 -- 172.21.15.12:0/673992022 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5571aa0ad580 con 0x5571aa47fec0 2024-04-22T13:25:03.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.973+0000 7f3fbcff9640 1 -- 172.21.15.12:0/673992022 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fcc01fd20 con 0x5571aa47fec0 2024-04-22T13:25:03.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.973+0000 7f3fbcff9640 1 -- 172.21.15.12:0/673992022 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3fcc0277c0 con 0x5571aa47fec0 2024-04-22T13:25:03.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.973+0000 7f3fde40b740 10 client.16080 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:03.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.973+0000 7f3fde40b740 10 client.16080 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-22T13:25:03.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.973+0000 7f3fde40b740 20 client.16080 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:03.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.973+0000 7f3fde40b740 1 --2- 172.21.15.12:0/673992022 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5571aa48fad0 0x5571aa491ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:03.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.973+0000 7f3fde40b740 4 client.16080 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:25:03.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.973+0000 7f3fde40b740 1 -- 172.21.15.12:0/673992022 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571aa3e8b10 con 0x5571aa48fad0 2024-04-22T13:25:03.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.973+0000 7f3fd95bd640 1 --2- 172.21.15.12:0/673992022 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5571aa48fad0 0x5571aa491ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:03.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.974+0000 7f3fd95bd640 1 --2- 172.21.15.12:0/673992022 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5571aa48fad0 0x5571aa491ec0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50fbf183d56bb5d3 server_cookie=2701f5f8c76feea4 in_seq=0 out_seq=0 2024-04-22T13:25:03.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.974+0000 7f3fbcff9640 10 client.16080 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:03.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.975+0000 7f3fbcff9640 1 -- 172.21.15.12:0/673992022 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5571aa3e8b10 con 0x5571aa48fad0 2024-04-22T13:25:03.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.975+0000 7f3fbcff9640 10 client.16080 handle_command_reply: tid=0 2024-04-22T13:25:03.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.975+0000 7f3fde40b740 1 client.16080 shutdown 2024-04-22T13:25:03.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.975+0000 7f3fde40b740 1 -- 172.21.15.12:0/673992022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571aa47fec0 msgr2=0x5571aa47d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:03.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.975+0000 7f3fde40b740 1 --2- 172.21.15.12:0/673992022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571aa47fec0 0x5571aa47d9a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc009fa0 tx=0x7f3fcc013620 comp rx=0 tx=0).stop 2024-04-22T13:25:03.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.975+0000 7f3fde40b740 1 -- 172.21.15.12:0/673992022 shutdown_connections 2024-04-22T13:25:03.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.975+0000 7f3fde40b740 1 --2- 172.21.15.12:0/673992022 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5571aa48fad0 0x5571aa491ec0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.975+0000 7f3fde40b740 1 --2- 172.21.15.12:0/673992022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571aa47fec0 0x5571aa47d9a0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.975+0000 7f3fde40b740 1 --2- 172.21.15.12:0/673992022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571aa4772e0 0x5571aa47d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.975+0000 7f3fde40b740 1 --2- 172.21.15.12:0/673992022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571aa4769c0 0x5571aa47cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:03.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.975+0000 7f3fde40b740 1 -- 172.21.15.12:0/673992022 >> 172.21.15.12:0/673992022 conn(0x5571aa4710f0 msgr2=0x5571aa472e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:03.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.975+0000 7f3fbcff9640 0 client.16080 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:03.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.975+0000 7f3fde40b740 1 -- 172.21.15.12:0/673992022 shutdown_connections 2024-04-22T13:25:03.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.975+0000 7f3fde40b740 1 -- 172.21.15.12:0/673992022 wait complete. 2024-04-22T13:25:03.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:03.975+0000 7f3fde40b740 20 client.16080 trim_cache size 0 max 16384 2024-04-22T13:25:04.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.009+0000 7f3fde40b740 1 Processor -- start 2024-04-22T13:25:04.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.009+0000 7f3fde40b740 1 -- start start 2024-04-22T13:25:04.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.009+0000 7f3fde40b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571aa4769c0 0x5571aa47f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:04.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.009+0000 7f3fde40b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571aa4772e0 0x5571aa4784b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:04.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.009+0000 7f3fde40b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571aa47fec0 0x5571aa48b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:04.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.009+0000 7f3fde40b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd41b17b0 con 0x5571aa4772e0 2024-04-22T13:25:04.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.009+0000 7f3fde40b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd41b0340 con 0x5571aa4769c0 2024-04-22T13:25:04.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.009+0000 7f3fde40b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd41b0a10 con 0x5571aa47fec0 2024-04-22T13:25:04.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.010+0000 7f3fd8dbc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571aa4772e0 0x5571aa4784b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:04.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.010+0000 7f3fd8dbc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571aa4772e0 0x5571aa4784b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40214/0 (socket says 172.21.15.12:40214) 2024-04-22T13:25:04.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.010+0000 7f3fd8dbc640 1 -- 172.21.15.12:0/2543880369 learned_addr learned my addr 172.21.15.12:0/2543880369 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:04.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.010+0000 7f3fd95bd640 1 --2- 172.21.15.12:0/2543880369 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571aa4769c0 0x5571aa47f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:04.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.010+0000 7f3fd9dbe640 1 --2- 172.21.15.12:0/2543880369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571aa47fec0 0x5571aa48b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:04.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.010+0000 7f3fd8dbc640 1 -- 172.21.15.12:0/2543880369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571aa47fec0 msgr2=0x5571aa48b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:04.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.010+0000 7f3fd8dbc640 1 --2- 172.21.15.12:0/2543880369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571aa47fec0 0x5571aa48b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:04.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.010+0000 7f3fd8dbc640 1 -- 172.21.15.12:0/2543880369 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571aa4769c0 msgr2=0x5571aa47f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:04.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.010+0000 7f3fd8dbc640 1 --2- 172.21.15.12:0/2543880369 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571aa4769c0 0x5571aa47f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:04.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.010+0000 7f3fd8dbc640 1 -- 172.21.15.12:0/2543880369 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571aa476180 con 0x5571aa4772e0 2024-04-22T13:25:04.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.010+0000 7f3fd8dbc640 1 --2- 172.21.15.12:0/2543880369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571aa4772e0 0x5571aa4784b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3fc40bb450 tx=0x7f3fc4089080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73fd3c2a1d34be server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:04.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.010+0000 7f3fbeffd640 1 -- 172.21.15.12:0/2543880369 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc40bf180 con 0x5571aa4772e0 2024-04-22T13:25:04.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.010+0000 7f3fbeffd640 1 -- 172.21.15.12:0/2543880369 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fc40c76f0 con 0x5571aa4772e0 2024-04-22T13:25:04.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.010+0000 7f3fbeffd640 1 -- 172.21.15.12:0/2543880369 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc40bb0f0 con 0x5571aa4772e0 2024-04-22T13:25:04.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.011+0000 7f3fde40b740 1 -- 172.21.15.12:0/2543880369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571aa4772e0 msgr2=0x5571aa4784b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:04.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.011+0000 7f3fde40b740 1 --2- 172.21.15.12:0/2543880369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571aa4772e0 0x5571aa4784b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3fc40bb450 tx=0x7f3fc4089080 comp rx=0 tx=0).stop 2024-04-22T13:25:04.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.011+0000 7f3fde40b740 1 -- 172.21.15.12:0/2543880369 shutdown_connections 2024-04-22T13:25:04.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.011+0000 7f3fde40b740 1 --2- 172.21.15.12:0/2543880369 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571aa47fec0 0x5571aa48b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:04.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.011+0000 7f3fde40b740 1 --2- 172.21.15.12:0/2543880369 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571aa4772e0 0x5571aa4784b0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:04.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.011+0000 7f3fde40b740 1 --2- 172.21.15.12:0/2543880369 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571aa4769c0 0x5571aa47f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:04.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.011+0000 7f3fde40b740 1 -- 172.21.15.12:0/2543880369 >> 172.21.15.12:0/2543880369 conn(0x5571aa4710f0 msgr2=0x5571aa46f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:04.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.011+0000 7f3fde40b740 1 -- 172.21.15.12:0/2543880369 shutdown_connections 2024-04-22T13:25:04.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.011+0000 7f3fde40b740 1 -- 172.21.15.12:0/2543880369 wait complete. 2024-04-22T13:25:04.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.011+0000 7f3fde40b740 1 -- start start 2024-04-22T13:25:04.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.011+0000 7f3fde40b740 1 Processor -- start 2024-04-22T13:25:04.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.012+0000 7f3fde40b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571aa4769c0 0x5571aa479cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:04.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.012+0000 7f3fde40b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571aa4772e0 0x5571aa47a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:04.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.012+0000 7f3fde40b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571aa47fec0 0x5571aa47a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:04.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.012+0000 7f3fde40b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd41adc70 con 0x5571aa4769c0 2024-04-22T13:25:04.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.012+0000 7f3fde40b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd41b10e0 con 0x5571aa47fec0 2024-04-22T13:25:04.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.012+0000 7f3fde40b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd41ae2c0 con 0x5571aa4772e0 2024-04-22T13:25:04.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.012+0000 7f3fde40b740 10 client.0 _ll_register_callbacks cb 0x7f3fcc0137b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:04.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.012+0000 7f3fd95bd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571aa4769c0 0x5571aa479cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:04.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.012+0000 7f3fd95bd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571aa4769c0 0x5571aa479cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40228/0 (socket says 172.21.15.12:40228) 2024-04-22T13:25:04.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.012+0000 7f3fd95bd640 1 -- 172.21.15.12:0/3242238998 learned_addr learned my addr 172.21.15.12:0/3242238998 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:04.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.012+0000 7f3fd9dbe640 1 --2- 172.21.15.12:0/3242238998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571aa47fec0 0x5571aa47a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:04.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.012+0000 7f3fd8dbc640 1 --2- 172.21.15.12:0/3242238998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571aa4772e0 0x5571aa47a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:04.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.012+0000 7f3fd95bd640 1 -- 172.21.15.12:0/3242238998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571aa4772e0 msgr2=0x5571aa47a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:04.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.012+0000 7f3fd95bd640 1 --2- 172.21.15.12:0/3242238998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571aa4772e0 0x5571aa47a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:04.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.012+0000 7f3fd95bd640 1 -- 172.21.15.12:0/3242238998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571aa47fec0 msgr2=0x5571aa47a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:04.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.012+0000 7f3fd95bd640 1 --2- 172.21.15.12:0/3242238998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571aa47fec0 0x5571aa47a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:04.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.012+0000 7f3fd95bd640 1 -- 172.21.15.12:0/3242238998 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571aa4f4e60 con 0x5571aa4769c0 2024-04-22T13:25:04.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.012+0000 7f3fd95bd640 1 --2- 172.21.15.12:0/3242238998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571aa4769c0 0x5571aa479cd0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000f9a0 tx=0x7f3fd001bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db17bd587ac4764e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:04.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.012+0000 7f3fbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:04.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.013+0000 7f3fbcff9640 1 -- 172.21.15.12:0/3242238998 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd001b290 con 0x5571aa4769c0 2024-04-22T13:25:04.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.013+0000 7f3fbcff9640 1 -- 172.21.15.12:0/3242238998 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f3fd001b430 con 0x5571aa4769c0 2024-04-22T13:25:04.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.013+0000 7f3fde40b740 1 -- 172.21.15.12:0/3242238998 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fc40c76f0 con 0x5571aa4769c0 2024-04-22T13:25:04.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.013+0000 7f3fbcff9640 1 -- 172.21.15.12:0/3242238998 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3fd0028020 con 0x5571aa4769c0 2024-04-22T13:25:04.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.013+0000 7f3fde40b740 10 client.16086 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:04.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.013+0000 7f3fde40b740 1 -- 172.21.15.12:0/3242238998 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5571aa4f61a0 con 0x5571aa4769c0 2024-04-22T13:25:04.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.013+0000 7f3fbcff9640 1 -- 172.21.15.12:0/3242238998 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fc40c76f0 con 0x5571aa4769c0 2024-04-22T13:25:04.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.013+0000 7f3fbcff9640 1 -- 172.21.15.12:0/3242238998 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3fd002db50 con 0x5571aa4769c0 2024-04-22T13:25:04.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.013+0000 7f3fde40b740 10 client.16086 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:04.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.013+0000 7f3fde40b740 10 client.16086 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-22T13:25:04.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.013+0000 7f3fde40b740 20 client.16086 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:04.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.013+0000 7f3fde40b740 1 --2- 172.21.15.12:0/3242238998 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5571aa4f69f0 0x5571aa4f8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:04.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.013+0000 7f3fde40b740 4 client.16086 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.35, "await": true}] 2024-04-22T13:25:04.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.013+0000 7f3fde40b740 1 -- 172.21.15.12:0/3242238998 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.35, "await": true}) -- 0x5571aa4922b0 con 0x5571aa4f69f0 2024-04-22T13:25:04.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.013+0000 7f3fd8dbc640 1 --2- 172.21.15.12:0/3242238998 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5571aa4f69f0 0x5571aa4f8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:04.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.014+0000 7f3fd8dbc640 1 --2- 172.21.15.12:0/3242238998 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5571aa4f69f0 0x5571aa4f8e10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=225d3b802466598c server_cookie=a4c1146ded46e7fc in_seq=0 out_seq=0 2024-04-22T13:25:04.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.014+0000 7f3fbcff9640 10 client.16086 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:04.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.022+0000 7f3fbcff9640 1 -- 172.21.15.12:0/3242238998 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+587 (crc 0 0 0) 0x5571aa4922b0 con 0x5571aa4f69f0 2024-04-22T13:25:04.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.022+0000 7f3fbcff9640 10 client.16086 handle_command_reply: tid=0 2024-04-22T13:25:04.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.022+0000 7f3fde40b740 1 client.16086 shutdown 2024-04-22T13:25:04.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.022+0000 7f3fde40b740 1 -- 172.21.15.12:0/3242238998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571aa4769c0 msgr2=0x5571aa479cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:04.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.022+0000 7f3fde40b740 1 --2- 172.21.15.12:0/3242238998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571aa4769c0 0x5571aa479cd0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000f9a0 tx=0x7f3fd001bf90 comp rx=0 tx=0).stop 2024-04-22T13:25:04.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.022+0000 7f3fde40b740 1 -- 172.21.15.12:0/3242238998 shutdown_connections 2024-04-22T13:25:04.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.022+0000 7f3fde40b740 1 --2- 172.21.15.12:0/3242238998 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5571aa4f69f0 0x5571aa4f8e10 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:04.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.022+0000 7f3fde40b740 1 --2- 172.21.15.12:0/3242238998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571aa47fec0 0x5571aa47a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:04.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.022+0000 7f3fbcff9640 0 client.16086 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:04.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.022+0000 7f3fde40b740 1 --2- 172.21.15.12:0/3242238998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571aa4772e0 0x5571aa47a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:04.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.022+0000 7f3fde40b740 1 --2- 172.21.15.12:0/3242238998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571aa4769c0 0x5571aa479cd0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:04.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.022+0000 7f3fde40b740 1 -- 172.21.15.12:0/3242238998 >> 172.21.15.12:0/3242238998 conn(0x5571aa4710f0 msgr2=0x5571aa4701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:04.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.023+0000 7f3fde40b740 1 -- 172.21.15.12:0/3242238998 shutdown_connections 2024-04-22T13:25:04.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.023+0000 7f3fde40b740 1 -- 172.21.15.12:0/3242238998 wait complete. 2024-04-22T13:25:04.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.023+0000 7f3fde40b740 20 client.16086 trim_cache size 0 max 16384 2024-04-22T13:25:04.020 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:25:04.020 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 28, 2024-04-22T13:25:04.020 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24509, 2024-04-22T13:25:04.020 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 6, 2024-04-22T13:25:04.020 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-04-22T13:25:04.021 INFO:teuthology.orchestra.run.smithi012.stdout: "3ff6b7a": { 2024-04-22T13:25:04.021 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 6, 2024-04-22T13:25:04.021 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-04-22T13:25:04.021 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-04-22T13:25:04.021 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-04-22T13:25:04.021 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-04-22T13:25:04.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:25:04.021 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 60.0, 2024-04-22T13:25:04.021 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 85.3, 2024-04-22T13:25:04.021 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-04-22T13:25:04.021 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-04-22T13:25:04.021 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-04-22T13:25:04.021 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-04-22T13:25:04.021 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "QUIESCED", 2024-04-22T13:25:04.021 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-04-22T13:25:04.021 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:25:04.022 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:25:04.022 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:25:04.022 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:25:04.022 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:25:04.022 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:25:04.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.023+0000 7f3fdb821640 1 -- 172.21.15.12:0/1911297874 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3fc00a3f00 msgr2=0x7f3fc00a63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:04.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.023+0000 7f3fdb821640 1 --2- 172.21.15.12:0/1911297874 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3fc00a3f00 0x7f3fc00a63c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc022000 tx=0x7f3fcc020000 comp rx=0 tx=0).stop 2024-04-22T13:25:04.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.023+0000 7f3fdb821640 1 -- 172.21.15.12:0/1911297874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fd4106170 msgr2=0x7f3fd410ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:04.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.023+0000 7f3fdb821640 1 --2- 172.21.15.12:0/1911297874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fd4106170 0x7f3fd410ec10 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4036040 tx=0x7f3fc40045e0 comp rx=0 tx=0).stop 2024-04-22T13:25:04.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.023+0000 7f3fdb821640 1 -- 172.21.15.12:0/1911297874 shutdown_connections 2024-04-22T13:25:04.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.023+0000 7f3fdb821640 1 --2- 172.21.15.12:0/1911297874 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3fc00a3f00 0x7f3fc00a63c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:04.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.023+0000 7f3fdb821640 1 --2- 172.21.15.12:0/1911297874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fd410f150 0x7f3fd41a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:04.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.023+0000 7f3fdb821640 1 --2- 172.21.15.12:0/1911297874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3fd4106170 0x7f3fd410ec10 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:04.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.023+0000 7f3fdb821640 1 --2- 172.21.15.12:0/1911297874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3fd41010a0 0x7f3fd410e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:04.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.023+0000 7f3fdb821640 1 -- 172.21.15.12:0/1911297874 >> 172.21.15.12:0/1911297874 conn(0x7f3fd4077f50 msgr2=0x7f3fd40ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:04.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.023+0000 7f3fdb821640 1 -- 172.21.15.12:0/1911297874 shutdown_connections 2024-04-22T13:25:04.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:04.023+0000 7f3fdb821640 1 -- 172.21.15.12:0/1911297874 wait complete. 2024-04-22T13:25:04.030 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced, set_id: 3ff6b7a, quiesce took 0.6 seconds. Will release after: 25.35 2024-04-22T13:25:04.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:03 smithi012 ceph-mon[28265]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-22T13:25:04.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:03 smithi012 ceph-mon[28265]: mgrmap e25: x(active, since 56s), standbys: z, y 2024-04-22T13:25:04.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:03 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1616960414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:25:04.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:03 smithi134 ceph-mon[31338]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-22T13:25:04.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:03 smithi134 ceph-mon[31338]: mgrmap e25: x(active, since 56s), standbys: z, y 2024-04-22T13:25:04.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:03 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1616960414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:25:04.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:03 smithi111 ceph-mon[31361]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-22T13:25:04.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:03 smithi111 ceph-mon[31361]: mgrmap e25: x(active, since 56s), standbys: z, y 2024-04-22T13:25:04.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:03 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1616960414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:25:05.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:04 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2167594738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:25:05.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:04 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2167594738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:25:05.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:04 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2167594738' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:25:06.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:05 smithi012 ceph-mon[28265]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-22T13:25:06.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:05 smithi134 ceph-mon[31338]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-22T13:25:06.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:05 smithi111 ceph-mon[31361]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-22T13:25:07.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:06 smithi012 ceph-mon[28265]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-22T13:25:07.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:06 smithi134 ceph-mon[31338]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-22T13:25:07.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:06 smithi111 ceph-mon[31361]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-22T13:25:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.492+0000 7f6fed715640 1 -- 172.21.15.12:0/2352400162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fe81031c0 msgr2=0x7f6fe81035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.492+0000 7f6fed715640 1 --2- 172.21.15.12:0/2352400162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fe81031c0 0x7f6fe81035a0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8009900 tx=0x7f6fd802bcd0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.492+0000 7f6fed715640 1 -- 172.21.15.12:0/2352400162 shutdown_connections 2024-04-22T13:25:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.492+0000 7f6fed715640 1 --2- 172.21.15.12:0/2352400162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fe810c590 0x7f6fe810e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.492+0000 7f6fed715640 1 --2- 172.21.15.12:0/2352400162 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6fe8103ae0 0x7f6fe810bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.492+0000 7f6fed715640 1 --2- 172.21.15.12:0/2352400162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fe81031c0 0x7f6fe81035a0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.492+0000 7f6fed715640 1 -- 172.21.15.12:0/2352400162 >> 172.21.15.12:0/2352400162 conn(0x7f6fe8077fd0 msgr2=0x7f6fe80783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.492+0000 7f6fed715640 1 -- 172.21.15.12:0/2352400162 shutdown_connections 2024-04-22T13:25:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.492+0000 7f6fed715640 1 -- 172.21.15.12:0/2352400162 wait complete. 2024-04-22T13:25:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.493+0000 7f6fed715640 1 Processor -- start 2024-04-22T13:25:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.493+0000 7f6fed715640 1 -- start start 2024-04-22T13:25:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.493+0000 7f6fed715640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fe81031c0 0x7f6fe8100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.493+0000 7f6fed715640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6fe8103ae0 0x7f6fe8101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.493+0000 7f6fed715640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fe810c590 0x7f6fe8101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.493+0000 7f6fed715640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe81121d0 con 0x7f6fe81031c0 2024-04-22T13:25:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.493+0000 7f6fed715640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe8112050 con 0x7f6fe8103ae0 2024-04-22T13:25:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.493+0000 7f6fed715640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe8112350 con 0x7f6fe810c590 2024-04-22T13:25:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.493+0000 7f6fe6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fe81031c0 0x7f6fe8100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.493+0000 7f6fe6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fe81031c0 0x7f6fe8100e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40246/0 (socket says 172.21.15.12:40246) 2024-04-22T13:25:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.493+0000 7f6fe6ffd640 1 -- 172.21.15.12:0/1581062773 learned_addr learned my addr 172.21.15.12:0/1581062773 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.494+0000 7f6fe67fc640 1 --2- 172.21.15.12:0/1581062773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6fe8103ae0 0x7f6fe8101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.494+0000 7f6fe77fe640 1 --2- 172.21.15.12:0/1581062773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fe810c590 0x7f6fe8101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.494+0000 7f6fe6ffd640 1 -- 172.21.15.12:0/1581062773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fe810c590 msgr2=0x7f6fe8101890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:07.492 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.494+0000 7f6fe6ffd640 1 --2- 172.21.15.12:0/1581062773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fe810c590 0x7f6fe8101890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.494+0000 7f6fe6ffd640 1 -- 172.21.15.12:0/1581062773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6fe8103ae0 msgr2=0x7f6fe8101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.494+0000 7f6fe6ffd640 1 --2- 172.21.15.12:0/1581062773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6fe8103ae0 0x7f6fe8101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.494+0000 7f6fe6ffd640 1 -- 172.21.15.12:0/1581062773 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fe81a6b50 con 0x7f6fe81031c0 2024-04-22T13:25:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.494+0000 7f6fe67fc640 1 --2- 172.21.15.12:0/1581062773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6fe8103ae0 0x7f6fe8101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.494+0000 7f6fe6ffd640 1 --2- 172.21.15.12:0/1581062773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fe81031c0 0x7f6fe8100e10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8005940 tx=0x7f6fd802d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.494+0000 7f6fcffff640 1 -- 172.21.15.12:0/1581062773 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8043020 con 0x7f6fe81031c0 2024-04-22T13:25:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.494+0000 7f6fed715640 1 -- 172.21.15.12:0/1581062773 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fe81a6de0 con 0x7f6fe81031c0 2024-04-22T13:25:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.494+0000 7f6fcffff640 1 -- 172.21.15.12:0/1581062773 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fd802dc90 con 0x7f6fe81031c0 2024-04-22T13:25:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.494+0000 7f6fcffff640 1 -- 172.21.15.12:0/1581062773 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8003ab0 con 0x7f6fe81031c0 2024-04-22T13:25:07.493 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.494+0000 7f6fed715640 1 -- 172.21.15.12:0/1581062773 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fe81a72a0 con 0x7f6fe81031c0 2024-04-22T13:25:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.497+0000 7f6fcffff640 1 -- 172.21.15.12:0/1581062773 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f6fd8046020 con 0x7f6fe81031c0 2024-04-22T13:25:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.497+0000 7f6ff03e9740 1 Processor -- start 2024-04-22T13:25:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.497+0000 7f6ff03e9740 1 -- start start 2024-04-22T13:25:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.497+0000 7f6ff03e9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617a0dea8d0 0x5617a0de9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.497+0000 7f6ff03e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617a0de9b60 0x5617a0de9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.497+0000 7f6ff03e9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a0df30e0 0x5617a0df54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.497+0000 7f6ff03e9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5617a0d4e8d0 con 0x5617a0dea8d0 2024-04-22T13:25:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.497+0000 7f6ff03e9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5617a0cebe80 con 0x5617a0df30e0 2024-04-22T13:25:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.497+0000 7f6ff03e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5617a0d4a890 con 0x5617a0de9b60 2024-04-22T13:25:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.497+0000 7f6fe77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617a0dea8d0 0x5617a0de9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.497+0000 7f6fe77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617a0dea8d0 0x5617a0de9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40248/0 (socket says 172.21.15.12:40248) 2024-04-22T13:25:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.497+0000 7f6fe77fe640 1 -- 172.21.15.12:0/4001651764 learned_addr learned my addr 172.21.15.12:0/4001651764 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.497+0000 7f6fe6ffd640 1 --2- 172.21.15.12:0/4001651764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617a0de9b60 0x5617a0de9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.498+0000 7f6fcffff640 1 --2- 172.21.15.12:0/1581062773 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6fd00a3fb0 0x7f6fd00a6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.498+0000 7f6fe67fc640 1 --2- 172.21.15.12:0/4001651764 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a0df30e0 0x5617a0df54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.498+0000 7f6fe77fe640 1 -- 172.21.15.12:0/4001651764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617a0de9b60 msgr2=0x5617a0de9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.498+0000 7f6fe77fe640 1 --2- 172.21.15.12:0/4001651764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617a0de9b60 0x5617a0de9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.498+0000 7f6fe77fe640 1 -- 172.21.15.12:0/4001651764 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a0df30e0 msgr2=0x5617a0df54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.498+0000 7f6fe77fe640 1 --2- 172.21.15.12:0/4001651764 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a0df30e0 0x5617a0df54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.498+0000 7f6fe77fe640 1 -- 172.21.15.12:0/4001651764 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617a0d0d320 con 0x5617a0dea8d0 2024-04-22T13:25:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.498+0000 7f6fe77fe640 1 --2- 172.21.15.12:0/1581062773 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6fd00a3fb0 0x7f6fd00a6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.498+0000 7f6fcffff640 1 -- 172.21.15.12:0/1581062773 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f6fd80f2b30 con 0x7f6fe81031c0 2024-04-22T13:25:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.498+0000 7f6fe77fe640 1 --2- 172.21.15.12:0/4001651764 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617a0dea8d0 0x5617a0de9620 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0009eb0 tx=0x7f6fe000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c6305dc20fde3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.498+0000 7f6fcd7fa640 1 -- 172.21.15.12:0/4001651764 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe001a410 con 0x5617a0dea8d0 2024-04-22T13:25:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.498+0000 7f6fcd7fa640 1 -- 172.21.15.12:0/4001651764 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fe001aa60 con 0x5617a0dea8d0 2024-04-22T13:25:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.498+0000 7f6fcd7fa640 1 -- 172.21.15.12:0/4001651764 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe00106c0 con 0x5617a0dea8d0 2024-04-22T13:25:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.498+0000 7f6fe77fe640 1 --2- 172.21.15.12:0/1581062773 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6fd00a3fb0 0x7f6fd00a6470 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0020000 tx=0x7f6fe001d000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.499+0000 7f6ff03e9740 1 -- 172.21.15.12:0/4001651764 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617a0dea8d0 msgr2=0x5617a0de9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.499+0000 7f6ff03e9740 1 --2- 172.21.15.12:0/4001651764 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617a0dea8d0 0x5617a0de9620 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0009eb0 tx=0x7f6fe000f730 comp rx=0 tx=0).stop 2024-04-22T13:25:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.499+0000 7f6ff03e9740 1 -- 172.21.15.12:0/4001651764 shutdown_connections 2024-04-22T13:25:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.499+0000 7f6ff03e9740 1 --2- 172.21.15.12:0/4001651764 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a0df30e0 0x5617a0df54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.499+0000 7f6ff03e9740 1 --2- 172.21.15.12:0/4001651764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617a0de9b60 0x5617a0de9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.499+0000 7f6ff03e9740 1 --2- 172.21.15.12:0/4001651764 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617a0dea8d0 0x5617a0de9620 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.499+0000 7f6ff03e9740 1 -- 172.21.15.12:0/4001651764 >> 172.21.15.12:0/4001651764 conn(0x5617a0db0c90 msgr2=0x5617a0de4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.499+0000 7f6ff03e9740 1 -- 172.21.15.12:0/4001651764 shutdown_connections 2024-04-22T13:25:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.499+0000 7f6ff03e9740 1 -- 172.21.15.12:0/4001651764 wait complete. 2024-04-22T13:25:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.499+0000 7f6ff03e9740 1 -- start start 2024-04-22T13:25:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.499+0000 7f6ff03e9740 1 Processor -- start 2024-04-22T13:25:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.500+0000 7f6ff03e9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a0de9b60 0x5617a0df29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.500+0000 7f6ff03e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617a0dea8d0 0x5617a0defdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.500+0000 7f6ff03e9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617a0df30e0 0x5617a0df0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.500+0000 7f6ff03e9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5617a0df5e10 con 0x5617a0df30e0 2024-04-22T13:25:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.500+0000 7f6ff03e9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5617a0cd6eb0 con 0x5617a0de9b60 2024-04-22T13:25:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.500+0000 7f6ff03e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5617a0df5f90 con 0x5617a0dea8d0 2024-04-22T13:25:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.500+0000 7f6ff03e9740 10 client.0 _ll_register_callbacks cb 0x5617a0d03b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.500+0000 7f6fe77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617a0df30e0 0x5617a0df0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.500+0000 7f6fe77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617a0df30e0 0x5617a0df0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40258/0 (socket says 172.21.15.12:40258) 2024-04-22T13:25:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.500+0000 7f6fe77fe640 1 -- 172.21.15.12:0/3016571341 learned_addr learned my addr 172.21.15.12:0/3016571341 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.500+0000 7f6fe6ffd640 1 --2- 172.21.15.12:0/3016571341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a0de9b60 0x5617a0df29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.500+0000 7f6fe67fc640 1 --2- 172.21.15.12:0/3016571341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617a0dea8d0 0x5617a0defdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.500+0000 7f6fe77fe640 1 -- 172.21.15.12:0/3016571341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617a0dea8d0 msgr2=0x5617a0defdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.500+0000 7f6fe77fe640 1 --2- 172.21.15.12:0/3016571341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617a0dea8d0 0x5617a0defdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.500+0000 7f6fe77fe640 1 -- 172.21.15.12:0/3016571341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a0de9b60 msgr2=0x5617a0df29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.500+0000 7f6fe77fe640 1 --2- 172.21.15.12:0/3016571341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a0de9b60 0x5617a0df29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.500+0000 7f6fe77fe640 1 -- 172.21.15.12:0/3016571341 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617a0a61210 con 0x5617a0df30e0 2024-04-22T13:25:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.500+0000 7f6fe77fe640 1 --2- 172.21.15.12:0/3016571341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617a0df30e0 0x5617a0df0330 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0010680 tx=0x7f6fe00106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e5a80d99633d94 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.500+0000 7f6fcaff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.500+0000 7f6fcaff5640 1 -- 172.21.15.12:0/3016571341 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0033060 con 0x5617a0df30e0 2024-04-22T13:25:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.500+0000 7f6fcaff5640 1 -- 172.21.15.12:0/3016571341 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f6fe0010cb0 con 0x5617a0df30e0 2024-04-22T13:25:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.501+0000 7f6ff03e9740 1 -- 172.21.15.12:0/3016571341 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fe001aa60 con 0x5617a0df30e0 2024-04-22T13:25:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.501+0000 7f6fcaff5640 1 -- 172.21.15.12:0/3016571341 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6fe001a410 con 0x5617a0df30e0 2024-04-22T13:25:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.501+0000 7f6ff03e9740 10 client.16098 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.501+0000 7f6ff03e9740 1 -- 172.21.15.12:0/3016571341 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5617a0b2d620 con 0x5617a0df30e0 2024-04-22T13:25:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.501+0000 7f6fcaff5640 1 -- 172.21.15.12:0/3016571341 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fe001aa60 con 0x5617a0df30e0 2024-04-22T13:25:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.501+0000 7f6fcaff5640 1 -- 172.21.15.12:0/3016571341 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6fe0005050 con 0x5617a0df30e0 2024-04-22T13:25:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.501+0000 7f6ff03e9740 10 client.16098 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.501+0000 7f6ff03e9740 10 client.16098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:25:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.501+0000 7f6ff03e9740 20 client.16098 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.501+0000 7f6ff03e9740 1 --2- 172.21.15.12:0/3016571341 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5617a0e03050 0x5617a0e05440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.501+0000 7f6ff03e9740 4 client.16098 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:25:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.501+0000 7f6ff03e9740 1 -- 172.21.15.12:0/3016571341 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617a0d62880 con 0x5617a0e03050 2024-04-22T13:25:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.501+0000 7f6fe6ffd640 1 --2- 172.21.15.12:0/3016571341 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5617a0e03050 0x5617a0e05440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.502+0000 7f6fe6ffd640 1 --2- 172.21.15.12:0/3016571341 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5617a0e03050 0x5617a0e05440 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=896b9c7cca3149fd server_cookie=bb622678e411d760 in_seq=0 out_seq=0 2024-04-22T13:25:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.502+0000 7f6fcaff5640 10 client.16098 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.502+0000 7f6fcaff5640 1 -- 172.21.15.12:0/3016571341 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5617a0d62880 con 0x5617a0e03050 2024-04-22T13:25:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.502+0000 7f6fcaff5640 10 client.16098 handle_command_reply: tid=0 2024-04-22T13:25:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.502+0000 7f6ff03e9740 1 client.16098 shutdown 2024-04-22T13:25:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.503+0000 7f6ff03e9740 1 -- 172.21.15.12:0/3016571341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617a0df30e0 msgr2=0x5617a0df0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.503+0000 7f6ff03e9740 1 --2- 172.21.15.12:0/3016571341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617a0df30e0 0x5617a0df0330 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0010680 tx=0x7f6fe00106b0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.503+0000 7f6ff03e9740 1 -- 172.21.15.12:0/3016571341 shutdown_connections 2024-04-22T13:25:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.503+0000 7f6ff03e9740 1 --2- 172.21.15.12:0/3016571341 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5617a0e03050 0x5617a0e05440 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.503+0000 7f6ff03e9740 1 --2- 172.21.15.12:0/3016571341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617a0df30e0 0x5617a0df0330 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.503+0000 7f6fcaff5640 0 client.16098 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.503+0000 7f6ff03e9740 1 --2- 172.21.15.12:0/3016571341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617a0dea8d0 0x5617a0defdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.503+0000 7f6ff03e9740 1 --2- 172.21.15.12:0/3016571341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a0de9b60 0x5617a0df29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.503+0000 7f6ff03e9740 1 -- 172.21.15.12:0/3016571341 >> 172.21.15.12:0/3016571341 conn(0x5617a0db0c90 msgr2=0x5617a0de65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.503+0000 7f6ff03e9740 1 -- 172.21.15.12:0/3016571341 shutdown_connections 2024-04-22T13:25:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.503+0000 7f6ff03e9740 1 -- 172.21.15.12:0/3016571341 wait complete. 2024-04-22T13:25:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.503+0000 7f6ff03e9740 20 client.16098 trim_cache size 0 max 16384 2024-04-22T13:25:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.518+0000 7f6ff03e9740 1 Processor -- start 2024-04-22T13:25:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.518+0000 7f6ff03e9740 1 -- start start 2024-04-22T13:25:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.518+0000 7f6ff03e9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617a0de9b60 0x5617a0dfed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.518+0000 7f6ff03e9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a0dea8d0 0x5617a0def6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.518+0000 7f6ff03e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617a0df30e0 0x5617a0de45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.518+0000 7f6ff03e9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe81afb10 con 0x5617a0de9b60 2024-04-22T13:25:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.518+0000 7f6ff03e9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe81b1650 con 0x5617a0dea8d0 2024-04-22T13:25:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.518+0000 7f6ff03e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe81b01e0 con 0x5617a0df30e0 2024-04-22T13:25:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.518+0000 7f6fe6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617a0de9b60 0x5617a0dfed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.518+0000 7f6fe6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617a0de9b60 0x5617a0dfed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40266/0 (socket says 172.21.15.12:40266) 2024-04-22T13:25:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.518+0000 7f6fe6ffd640 1 -- 172.21.15.12:0/992425082 learned_addr learned my addr 172.21.15.12:0/992425082 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.518+0000 7f6fe6ffd640 1 -- 172.21.15.12:0/992425082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617a0df30e0 msgr2=0x5617a0de45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.518+0000 7f6fe6ffd640 1 --2- 172.21.15.12:0/992425082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617a0df30e0 0x5617a0de45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.518+0000 7f6fe6ffd640 1 -- 172.21.15.12:0/992425082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a0dea8d0 msgr2=0x5617a0def6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.518+0000 7f6fe6ffd640 1 --2- 172.21.15.12:0/992425082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a0dea8d0 0x5617a0def6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.518+0000 7f6fe6ffd640 1 -- 172.21.15.12:0/992425082 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617a0de8e30 con 0x5617a0de9b60 2024-04-22T13:25:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.518+0000 7f6fe6ffd640 1 --2- 172.21.15.12:0/992425082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617a0de9b60 0x5617a0dfed10 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f6fd80beaa0 tx=0x7f6fd80f4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8642347a2fffee86 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.518+0000 7f6fccff9640 1 -- 172.21.15.12:0/992425082 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd80be470 con 0x5617a0de9b60 2024-04-22T13:25:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.518+0000 7f6fccff9640 1 -- 172.21.15.12:0/992425082 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fd80f8040 con 0x5617a0de9b60 2024-04-22T13:25:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.518+0000 7f6fccff9640 1 -- 172.21.15.12:0/992425082 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd80c7780 con 0x5617a0de9b60 2024-04-22T13:25:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.519+0000 7f6ff03e9740 1 -- 172.21.15.12:0/992425082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617a0de9b60 msgr2=0x5617a0dfed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.519+0000 7f6ff03e9740 1 --2- 172.21.15.12:0/992425082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617a0de9b60 0x5617a0dfed10 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f6fd80beaa0 tx=0x7f6fd80f4040 comp rx=0 tx=0).stop 2024-04-22T13:25:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.519+0000 7f6ff03e9740 1 -- 172.21.15.12:0/992425082 shutdown_connections 2024-04-22T13:25:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.519+0000 7f6ff03e9740 1 --2- 172.21.15.12:0/992425082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617a0df30e0 0x5617a0de45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.519+0000 7f6ff03e9740 1 --2- 172.21.15.12:0/992425082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a0dea8d0 0x5617a0def6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.519+0000 7f6ff03e9740 1 --2- 172.21.15.12:0/992425082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617a0de9b60 0x5617a0dfed10 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.519+0000 7f6ff03e9740 1 -- 172.21.15.12:0/992425082 >> 172.21.15.12:0/992425082 conn(0x5617a0db0c90 msgr2=0x5617a0df14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.519+0000 7f6ff03e9740 1 -- 172.21.15.12:0/992425082 shutdown_connections 2024-04-22T13:25:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.519+0000 7f6ff03e9740 1 -- 172.21.15.12:0/992425082 wait complete. 2024-04-22T13:25:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.519+0000 7f6ff03e9740 1 -- start start 2024-04-22T13:25:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.519+0000 7f6ff03e9740 1 Processor -- start 2024-04-22T13:25:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.520+0000 7f6ff03e9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617a0de9b60 0x5617a0dfb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.520+0000 7f6ff03e9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a0dea8d0 0x5617a0dec450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.520+0000 7f6ff03e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617a0df30e0 0x5617a0dec990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.520+0000 7f6ff03e9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe81b0f80 con 0x5617a0de9b60 2024-04-22T13:25:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.520+0000 7f6ff03e9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe81b08b0 con 0x5617a0dea8d0 2024-04-22T13:25:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.520+0000 7f6ff03e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe81adaf0 con 0x5617a0df30e0 2024-04-22T13:25:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.520+0000 7f6ff03e9740 10 client.0 _ll_register_callbacks cb 0x7f6fd80bae30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.520+0000 7f6fe6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617a0de9b60 0x5617a0dfb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.520+0000 7f6fe6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617a0de9b60 0x5617a0dfb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:40274/0 (socket says 172.21.15.12:40274) 2024-04-22T13:25:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.520+0000 7f6fe6ffd640 1 -- 172.21.15.12:0/2707048665 learned_addr learned my addr 172.21.15.12:0/2707048665 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.520+0000 7f6fe77fe640 1 --2- 172.21.15.12:0/2707048665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617a0df30e0 0x5617a0dec990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.520+0000 7f6fe6ffd640 1 -- 172.21.15.12:0/2707048665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617a0df30e0 msgr2=0x5617a0dec990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.520+0000 7f6fe67fc640 1 --2- 172.21.15.12:0/2707048665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a0dea8d0 0x5617a0dec450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.520+0000 7f6fe6ffd640 1 --2- 172.21.15.12:0/2707048665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617a0df30e0 0x5617a0dec990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.520+0000 7f6fe6ffd640 1 -- 172.21.15.12:0/2707048665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a0dea8d0 msgr2=0x5617a0dec450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.520+0000 7f6fe6ffd640 1 --2- 172.21.15.12:0/2707048665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a0dea8d0 0x5617a0dec450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.520+0000 7f6fe6ffd640 1 -- 172.21.15.12:0/2707048665 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617a0deced0 con 0x5617a0de9b60 2024-04-22T13:25:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.520+0000 7f6fe67fc640 1 --2- 172.21.15.12:0/2707048665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a0dea8d0 0x5617a0dec450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:25:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.520+0000 7f6fe6ffd640 1 --2- 172.21.15.12:0/2707048665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617a0de9b60 0x5617a0dfb410 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8005820 tx=0x7f6fd80be470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c28d1622727723 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.520+0000 7f6fcaff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.520+0000 7f6fcaff5640 1 -- 172.21.15.12:0/2707048665 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8088060 con 0x5617a0de9b60 2024-04-22T13:25:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.520+0000 7f6fcaff5640 1 -- 172.21.15.12:0/2707048665 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f6fd8050d70 con 0x5617a0de9b60 2024-04-22T13:25:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.520+0000 7f6ff03e9740 1 -- 172.21.15.12:0/2707048665 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fd80f8040 con 0x5617a0de9b60 2024-04-22T13:25:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.521+0000 7f6fcaff5640 1 -- 172.21.15.12:0/2707048665 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6fd80bf210 con 0x5617a0de9b60 2024-04-22T13:25:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.521+0000 7f6ff03e9740 10 client.16104 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.521+0000 7f6ff03e9740 1 -- 172.21.15.12:0/2707048665 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5617a0e68fb0 con 0x5617a0de9b60 2024-04-22T13:25:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.521+0000 7f6fcaff5640 1 -- 172.21.15.12:0/2707048665 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fd80f8040 con 0x5617a0de9b60 2024-04-22T13:25:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.521+0000 7f6fcaff5640 1 -- 172.21.15.12:0/2707048665 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6fd8005340 con 0x5617a0de9b60 2024-04-22T13:25:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.521+0000 7f6ff03e9740 10 client.16104 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.521+0000 7f6ff03e9740 10 client.16104 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:25:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.521+0000 7f6ff03e9740 20 client.16104 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.521+0000 7f6ff03e9740 1 --2- 172.21.15.12:0/2707048665 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5617a0e69510 0x5617a0e6b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.521+0000 7f6ff03e9740 4 client.16104 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-22T13:25:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.521+0000 7f6ff03e9740 1 -- 172.21.15.12:0/2707048665 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5617a0cee3d0 con 0x5617a0e69510 2024-04-22T13:25:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.521+0000 7f6fe67fc640 1 --2- 172.21.15.12:0/2707048665 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5617a0e69510 0x5617a0e6b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.522+0000 7f6fe67fc640 1 --2- 172.21.15.12:0/2707048665 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5617a0e69510 0x5617a0e6b930 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6029d462b3c776db server_cookie=15b1f91857e9e27d in_seq=0 out_seq=0 2024-04-22T13:25:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.522+0000 7f6fcaff5640 10 client.16104 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.528+0000 7f6fcaff5640 1 -- 172.21.15.12:0/2707048665 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5617a0cee3d0 con 0x5617a0e69510 2024-04-22T13:25:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.528+0000 7f6fcaff5640 10 client.16104 handle_command_reply: tid=0 2024-04-22T13:25:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.528+0000 7f6ff03e9740 1 client.16104 shutdown 2024-04-22T13:25:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.528+0000 7f6ff03e9740 1 -- 172.21.15.12:0/2707048665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617a0de9b60 msgr2=0x5617a0dfb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.528+0000 7f6ff03e9740 1 --2- 172.21.15.12:0/2707048665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617a0de9b60 0x5617a0dfb410 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8005820 tx=0x7f6fd80be470 comp rx=0 tx=0).stop 2024-04-22T13:25:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.528+0000 7f6ff03e9740 1 -- 172.21.15.12:0/2707048665 shutdown_connections 2024-04-22T13:25:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.528+0000 7f6ff03e9740 1 --2- 172.21.15.12:0/2707048665 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5617a0e69510 0x5617a0e6b930 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.528+0000 7f6ff03e9740 1 --2- 172.21.15.12:0/2707048665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5617a0df30e0 0x5617a0dec990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.528+0000 7f6fcaff5640 0 client.16104 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.528+0000 7f6ff03e9740 1 --2- 172.21.15.12:0/2707048665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a0dea8d0 0x5617a0dec450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.528+0000 7f6ff03e9740 1 --2- 172.21.15.12:0/2707048665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5617a0de9b60 0x5617a0dfb410 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.528+0000 7f6ff03e9740 1 -- 172.21.15.12:0/2707048665 >> 172.21.15.12:0/2707048665 conn(0x5617a0db0c90 msgr2=0x5617a0de38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.528+0000 7f6ff03e9740 1 -- 172.21.15.12:0/2707048665 shutdown_connections 2024-04-22T13:25:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.528+0000 7f6ff03e9740 1 -- 172.21.15.12:0/2707048665 wait complete. 2024-04-22T13:25:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.528+0000 7f6ff03e9740 20 client.16104 trim_cache size 0 max 16384 2024-04-22T13:25:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-04-22T13:25:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-04-22T13:25:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-04-22T13:25:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-04-22T13:25:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.529+0000 7f6fed715640 1 -- 172.21.15.12:0/1581062773 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6fd00a3fb0 msgr2=0x7f6fd00a6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.529+0000 7f6fed715640 1 --2- 172.21.15.12:0/1581062773 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6fd00a3fb0 0x7f6fd00a6470 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0020000 tx=0x7f6fe001d000 comp rx=0 tx=0).stop 2024-04-22T13:25:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.529+0000 7f6fed715640 1 -- 172.21.15.12:0/1581062773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fe81031c0 msgr2=0x7f6fe8100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.529+0000 7f6fed715640 1 --2- 172.21.15.12:0/1581062773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fe81031c0 0x7f6fe8100e10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8005940 tx=0x7f6fd802d890 comp rx=0 tx=0).stop 2024-04-22T13:25:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.529+0000 7f6fed715640 1 -- 172.21.15.12:0/1581062773 shutdown_connections 2024-04-22T13:25:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.529+0000 7f6fed715640 1 --2- 172.21.15.12:0/1581062773 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6fd00a3fb0 0x7f6fd00a6470 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.529+0000 7f6fed715640 1 --2- 172.21.15.12:0/1581062773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6fe810c590 0x7f6fe8101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.529+0000 7f6fed715640 1 --2- 172.21.15.12:0/1581062773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6fe8103ae0 0x7f6fe8101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.529+0000 7f6fed715640 1 --2- 172.21.15.12:0/1581062773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6fe81031c0 0x7f6fe8100e10 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.529+0000 7f6fed715640 1 -- 172.21.15.12:0/1581062773 >> 172.21.15.12:0/1581062773 conn(0x7f6fe8077fd0 msgr2=0x7f6fe80fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.529+0000 7f6fed715640 1 -- 172.21.15.12:0/1581062773 shutdown_connections 2024-04-22T13:25:07.528 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:07.529+0000 7f6fed715640 1 -- 172.21.15.12:0/1581062773 wait complete. 2024-04-22T13:25:09.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:09 smithi012 ceph-mon[28265]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-22T13:25:09.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:09 smithi134 ceph-mon[31338]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-22T13:25:09.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:09 smithi111 ceph-mon[31361]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-22T13:25:11.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:11 smithi012 ceph-mon[28265]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-22T13:25:11.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:11 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:25:11.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:11 smithi134 ceph-mon[31338]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-22T13:25:11.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:11 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:25:11.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:11 smithi111 ceph-mon[31361]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-22T13:25:11.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:11 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:25:12.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:12 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:25:12.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:12 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:25:12.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:12 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:25:12.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:12 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:25:12.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:12 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:25:12.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:12 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:25:12.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:12 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:25:12.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:12 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:25:12.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:12 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:25:12.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:12 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:25:13.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:12 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:25:13.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:12 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:25:13.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:12 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:25:13.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:12 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:25:13.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:12 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:25:13.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:13 smithi134 ceph-mon[31338]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-04-22T13:25:13.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:13 smithi111 ceph-mon[31361]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-04-22T13:25:14.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:13 smithi012 ceph-mon[28265]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-04-22T13:25:15.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:15 smithi134 ceph-mon[31338]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-22T13:25:15.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:15 smithi111 ceph-mon[31361]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-22T13:25:16.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:15 smithi012 ceph-mon[28265]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-22T13:25:16.234 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:25:16.235 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:25:16.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.361+0000 7f18c7fff640 1 -- 172.21.15.12:0/3915680315 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c002cbe0 con 0x7f18d0101660 2024-04-22T13:25:16.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.362+0000 7f18d76d9640 1 -- 172.21.15.12:0/3915680315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18d0101660 msgr2=0x7f18d010e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:16.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.362+0000 7f18d76d9640 1 --2- 172.21.15.12:0/3915680315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18d0101660 0x7f18d010e3b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009900 tx=0x7f18c002bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:16.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.362+0000 7f18d76d9640 1 -- 172.21.15.12:0/3915680315 shutdown_connections 2024-04-22T13:25:16.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.362+0000 7f18d76d9640 1 --2- 172.21.15.12:0/3915680315 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18d010e8f0 0x7f18d0110ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.362+0000 7f18d76d9640 1 --2- 172.21.15.12:0/3915680315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18d0101660 0x7f18d010e3b0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.362+0000 7f18d76d9640 1 --2- 172.21.15.12:0/3915680315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18d0100d40 0x7f18d0101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.362+0000 7f18d76d9640 1 -- 172.21.15.12:0/3915680315 >> 172.21.15.12:0/3915680315 conn(0x7f18d0078000 msgr2=0x7f18d0078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:16.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.362+0000 7f18d76d9640 1 -- 172.21.15.12:0/3915680315 shutdown_connections 2024-04-22T13:25:16.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.362+0000 7f18d76d9640 1 -- 172.21.15.12:0/3915680315 wait complete. 2024-04-22T13:25:16.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.363+0000 7f18d76d9640 1 Processor -- start 2024-04-22T13:25:16.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.363+0000 7f18d76d9640 1 -- start start 2024-04-22T13:25:16.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.363+0000 7f18d76d9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18d0100d40 0x7f18d01a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:16.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.363+0000 7f18d76d9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18d0101660 0x7f18d01a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:16.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.363+0000 7f18d76d9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18d010e8f0 0x7f18d019a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:16.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.363+0000 7f18d76d9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f18d01143b0 con 0x7f18d010e8f0 2024-04-22T13:25:16.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.363+0000 7f18d76d9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f18d0114230 con 0x7f18d0101660 2024-04-22T13:25:16.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.363+0000 7f18d76d9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f18d0114530 con 0x7f18d0100d40 2024-04-22T13:25:16.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.363+0000 7f18d5c76640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18d010e8f0 0x7f18d019a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:16.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.363+0000 7f18d5c76640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18d010e8f0 0x7f18d019a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55482/0 (socket says 172.21.15.12:55482) 2024-04-22T13:25:16.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.363+0000 7f18d5c76640 1 -- 172.21.15.12:0/1539027788 learned_addr learned my addr 172.21.15.12:0/1539027788 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:16.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.363+0000 7f18d5475640 1 --2- 172.21.15.12:0/1539027788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18d0100d40 0x7f18d01a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:16.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.364+0000 7f18d4c74640 1 --2- 172.21.15.12:0/1539027788 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18d0101660 0x7f18d01a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:16.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.364+0000 7f18d5c76640 1 -- 172.21.15.12:0/1539027788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18d0100d40 msgr2=0x7f18d01a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:16.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.364+0000 7f18d5c76640 1 --2- 172.21.15.12:0/1539027788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18d0100d40 0x7f18d01a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.364+0000 7f18d5c76640 1 -- 172.21.15.12:0/1539027788 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18d0101660 msgr2=0x7f18d01a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:16.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.364+0000 7f18d5c76640 1 --2- 172.21.15.12:0/1539027788 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18d0101660 0x7f18d01a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.364+0000 7f18d5c76640 1 -- 172.21.15.12:0/1539027788 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d019b070 con 0x7f18d010e8f0 2024-04-22T13:25:16.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.364+0000 7f18d4c74640 1 --2- 172.21.15.12:0/1539027788 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18d0101660 0x7f18d01a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:16.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.364+0000 7f18d5c76640 1 --2- 172.21.15.12:0/1539027788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18d010e8f0 0x7f18d019a7b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f18c800e9d0 tx=0x7f18c800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:16.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.364+0000 7f18c67fc640 1 -- 172.21.15.12:0/1539027788 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c800cd80 con 0x7f18d010e8f0 2024-04-22T13:25:16.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.364+0000 7f18c67fc640 1 -- 172.21.15.12:0/1539027788 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18c8005020 con 0x7f18d010e8f0 2024-04-22T13:25:16.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.364+0000 7f18c67fc640 1 -- 172.21.15.12:0/1539027788 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c80116d0 con 0x7f18d010e8f0 2024-04-22T13:25:16.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.364+0000 7f18d76d9640 1 -- 172.21.15.12:0/1539027788 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18d019b360 con 0x7f18d010e8f0 2024-04-22T13:25:16.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.364+0000 7f18d76d9640 1 -- 172.21.15.12:0/1539027788 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18d01a7620 con 0x7f18d010e8f0 2024-04-22T13:25:16.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.367+0000 7f18da3a9740 1 Processor -- start 2024-04-22T13:25:16.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.367+0000 7f18c67fc640 1 -- 172.21.15.12:0/1539027788 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f18c80051d0 con 0x7f18d010e8f0 2024-04-22T13:25:16.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.367+0000 7f18da3a9740 1 -- start start 2024-04-22T13:25:16.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.367+0000 7f18da3a9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baaa39b580 0x55baaa39b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:16.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.367+0000 7f18da3a9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55baaa39bea0 0x55baaa3a4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:16.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.367+0000 7f18da3a9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55baaa3a4a80 0x55baaa3a6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:16.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.367+0000 7f18da3a9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55baaa3001e0 con 0x55baaa39bea0 2024-04-22T13:25:16.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.367+0000 7f18da3a9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55baaa29d390 con 0x55baaa3a4a80 2024-04-22T13:25:16.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.367+0000 7f18da3a9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55baaa2fc1a0 con 0x55baaa39b580 2024-04-22T13:25:16.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.368+0000 7f18d5475640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55baaa39bea0 0x55baaa3a4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:16.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.368+0000 7f18d5475640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55baaa39bea0 0x55baaa3a4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55496/0 (socket says 172.21.15.12:55496) 2024-04-22T13:25:16.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.368+0000 7f18d5475640 1 -- 172.21.15.12:0/1580784385 learned_addr learned my addr 172.21.15.12:0/1580784385 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:16.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.368+0000 7f18c67fc640 1 --2- 172.21.15.12:0/1539027788 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f18bc0a3f50 0x7f18bc0a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:16.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.368+0000 7f18d4c74640 1 --2- 172.21.15.12:0/1580784385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baaa39b580 0x55baaa39b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:16.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.368+0000 7f18d4c74640 1 --2- 172.21.15.12:0/1580784385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55baaa3a4a80 0x55baaa3a6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:16.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.368+0000 7f18d5c76640 1 --2- 172.21.15.12:0/1539027788 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f18bc0a3f50 0x7f18bc0a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:16.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.368+0000 7f18d5475640 1 -- 172.21.15.12:0/1580784385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baaa39b580 msgr2=0x55baaa39b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:16.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.368+0000 7f18d5475640 1 --2- 172.21.15.12:0/1580784385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baaa39b580 0x55baaa39b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.368+0000 7f18c67fc640 1 -- 172.21.15.12:0/1539027788 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f18c80d0bd0 con 0x7f18d010e8f0 2024-04-22T13:25:16.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.368+0000 7f18d5475640 1 -- 172.21.15.12:0/1580784385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55baaa3a4a80 msgr2=0x55baaa3a6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:16.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.368+0000 7f18d5475640 1 --2- 172.21.15.12:0/1580784385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55baaa3a4a80 0x55baaa3a6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.368+0000 7f18d5475640 1 -- 172.21.15.12:0/1580784385 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baaa0dc7d0 con 0x55baaa39bea0 2024-04-22T13:25:16.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.368+0000 7f18d5475640 1 --2- 172.21.15.12:0/1580784385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55baaa39bea0 0x55baaa3a4540 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f18cc00bd10 tx=0x7f18cc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4fa0864029503db server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:16.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.369+0000 7f18d5c76640 1 --2- 172.21.15.12:0/1539027788 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f18bc0a3f50 0x7f18bc0a6410 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f18c80ce190 tx=0x7f18c801eb40 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:16.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.369+0000 7f18bb7fe640 1 -- 172.21.15.12:0/1580784385 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc004030 con 0x55baaa39bea0 2024-04-22T13:25:16.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.369+0000 7f18bb7fe640 1 -- 172.21.15.12:0/1580784385 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18cc0035f0 con 0x55baaa39bea0 2024-04-22T13:25:16.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.369+0000 7f18bb7fe640 1 -- 172.21.15.12:0/1580784385 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc0116a0 con 0x55baaa39bea0 2024-04-22T13:25:16.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.369+0000 7f18da3a9740 1 -- 172.21.15.12:0/1580784385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55baaa39bea0 msgr2=0x55baaa3a4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:16.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.369+0000 7f18da3a9740 1 --2- 172.21.15.12:0/1580784385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55baaa39bea0 0x55baaa3a4540 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f18cc00bd10 tx=0x7f18cc0107a0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.369+0000 7f18da3a9740 1 -- 172.21.15.12:0/1580784385 shutdown_connections 2024-04-22T13:25:16.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.369+0000 7f18da3a9740 1 --2- 172.21.15.12:0/1580784385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55baaa3a4a80 0x55baaa3a6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.369+0000 7f18da3a9740 1 --2- 172.21.15.12:0/1580784385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55baaa39bea0 0x55baaa3a4540 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.369+0000 7f18da3a9740 1 --2- 172.21.15.12:0/1580784385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baaa39b580 0x55baaa39b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.369+0000 7f18da3a9740 1 -- 172.21.15.12:0/1580784385 >> 172.21.15.12:0/1580784385 conn(0x55baaa395cb0 msgr2=0x55baaa396090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:16.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.370+0000 7f18da3a9740 1 -- 172.21.15.12:0/1580784385 shutdown_connections 2024-04-22T13:25:16.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.370+0000 7f18da3a9740 1 -- 172.21.15.12:0/1580784385 wait complete. 2024-04-22T13:25:16.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.370+0000 7f18da3a9740 1 -- start start 2024-04-22T13:25:16.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.370+0000 7f18da3a9740 1 Processor -- start 2024-04-22T13:25:16.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.371+0000 7f18da3a9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55baaa39b580 0x55baaa3a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:16.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.371+0000 7f18da3a9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baaa39bea0 0x55baaa3a2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:16.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.371+0000 7f18da3a9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55baaa3a4a80 0x55baaa3a2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:16.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.371+0000 7f18da3a9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55baaa3a7870 con 0x55baaa3a4a80 2024-04-22T13:25:16.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.371+0000 7f18da3a9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55baaa2b92f0 con 0x55baaa39b580 2024-04-22T13:25:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.371+0000 7f18da3a9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55baaa3a79f0 con 0x55baaa39bea0 2024-04-22T13:25:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.371+0000 7f18d5475640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baaa39bea0 0x55baaa3a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.371+0000 7f18da3a9740 10 client.0 _ll_register_callbacks cb 0x55baaa2c6580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.371+0000 7f18d5475640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baaa39bea0 0x55baaa3a2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.12:54168/0 (socket says 172.21.15.12:54168) 2024-04-22T13:25:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.371+0000 7f18d5475640 1 -- 172.21.15.12:0/2180062090 learned_addr learned my addr 172.21.15.12:0/2180062090 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.371+0000 7f18d4c74640 1 --2- 172.21.15.12:0/2180062090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55baaa3a4a80 0x55baaa3a2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.371+0000 7f18d4c74640 1 --2- 172.21.15.12:0/2180062090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55baaa39b580 0x55baaa3a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.371+0000 7f18d5475640 1 -- 172.21.15.12:0/2180062090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55baaa39b580 msgr2=0x55baaa3a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.371+0000 7f18d5475640 1 --2- 172.21.15.12:0/2180062090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55baaa39b580 0x55baaa3a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.371+0000 7f18d5475640 1 -- 172.21.15.12:0/2180062090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55baaa3a4a80 msgr2=0x55baaa3a2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.371+0000 7f18d5475640 1 --2- 172.21.15.12:0/2180062090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55baaa3a4a80 0x55baaa3a2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.371+0000 7f18d5475640 1 -- 172.21.15.12:0/2180062090 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baaa0123f0 con 0x55baaa39bea0 2024-04-22T13:25:16.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.371+0000 7f18d4c74640 1 --2- 172.21.15.12:0/2180062090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55baaa3a4a80 0x55baaa3a2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:16.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.372+0000 7f18d5475640 1 --2- 172.21.15.12:0/2180062090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baaa39bea0 0x55baaa3a2010 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f18cc000c00 tx=0x7f18cc00b890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b98159aed06466a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:16.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.372+0000 7f18b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-22T13:25:16.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.372+0000 7f18b8ff9640 1 -- 172.21.15.12:0/2180062090 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc029060 con 0x55baaa39bea0 2024-04-22T13:25:16.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.372+0000 7f18da3a9740 1 -- 172.21.15.12:0/2180062090 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18cc0035f0 con 0x55baaa39bea0 2024-04-22T13:25:16.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.372+0000 7f18b8ff9640 1 -- 172.21.15.12:0/2180062090 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f18cc0039b0 con 0x55baaa39bea0 2024-04-22T13:25:16.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.372+0000 7f18b8ff9640 1 -- 172.21.15.12:0/2180062090 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f18cc003f40 con 0x55baaa39bea0 2024-04-22T13:25:16.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.372+0000 7f18da3a9740 10 client.25495 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:16.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.372+0000 7f18da3a9740 1 -- 172.21.15.12:0/2180062090 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55baa9fd20f0 con 0x55baaa39bea0 2024-04-22T13:25:16.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.373+0000 7f18b8ff9640 1 -- 172.21.15.12:0/2180062090 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18cc0035f0 con 0x55baaa39bea0 2024-04-22T13:25:16.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.373+0000 7f18b8ff9640 1 -- 172.21.15.12:0/2180062090 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f18cc0046c0 con 0x55baaa39bea0 2024-04-22T13:25:16.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.373+0000 7f18da3a9740 10 client.25495 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:16.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.373+0000 7f18da3a9740 10 client.25495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:25:16.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.373+0000 7f18da3a9740 20 client.25495 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:16.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.373+0000 7f18da3a9740 1 --2- 172.21.15.12:0/2180062090 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55baaa3b49d0 0x55baaa3b6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:16.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.373+0000 7f18da3a9740 4 client.25495 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:25:16.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.373+0000 7f18da3a9740 1 -- 172.21.15.12:0/2180062090 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55baaa30dc00 con 0x55baaa3b49d0 2024-04-22T13:25:16.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.373+0000 7f18d4c74640 1 --2- 172.21.15.12:0/2180062090 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55baaa3b49d0 0x55baaa3b6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:16.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.374+0000 7f18b8ff9640 10 client.25495 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:16.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.374+0000 7f18d4c74640 1 --2- 172.21.15.12:0/2180062090 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55baaa3b49d0 0x55baaa3b6dc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b731be8b68e53fed server_cookie=387c8ec0910e8262 in_seq=0 out_seq=0 2024-04-22T13:25:16.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.374+0000 7f18b8ff9640 1 -- 172.21.15.12:0/2180062090 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55baaa30dc00 con 0x55baaa3b49d0 2024-04-22T13:25:16.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.374+0000 7f18b8ff9640 10 client.25495 handle_command_reply: tid=0 2024-04-22T13:25:16.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.374+0000 7f18da3a9740 1 client.25495 shutdown 2024-04-22T13:25:16.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.374+0000 7f18da3a9740 1 -- 172.21.15.12:0/2180062090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baaa39bea0 msgr2=0x55baaa3a2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:16.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.374+0000 7f18da3a9740 1 --2- 172.21.15.12:0/2180062090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baaa39bea0 0x55baaa3a2010 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f18cc000c00 tx=0x7f18cc00b890 comp rx=0 tx=0).stop 2024-04-22T13:25:16.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.374+0000 7f18da3a9740 1 -- 172.21.15.12:0/2180062090 shutdown_connections 2024-04-22T13:25:16.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.374+0000 7f18da3a9740 1 --2- 172.21.15.12:0/2180062090 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55baaa3b49d0 0x55baaa3b6dc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.374+0000 7f18da3a9740 1 --2- 172.21.15.12:0/2180062090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55baaa3a4a80 0x55baaa3a2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.374+0000 7f18b8ff9640 0 client.25495 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:16.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.374+0000 7f18da3a9740 1 --2- 172.21.15.12:0/2180062090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baaa39bea0 0x55baaa3a2010 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.374+0000 7f18da3a9740 1 --2- 172.21.15.12:0/2180062090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55baaa39b580 0x55baaa3a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.374+0000 7f18da3a9740 1 -- 172.21.15.12:0/2180062090 >> 172.21.15.12:0/2180062090 conn(0x55baaa395cb0 msgr2=0x55baaa397a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:16.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.375+0000 7f18da3a9740 1 -- 172.21.15.12:0/2180062090 shutdown_connections 2024-04-22T13:25:16.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.375+0000 7f18da3a9740 1 -- 172.21.15.12:0/2180062090 wait complete. 2024-04-22T13:25:16.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.375+0000 7f18da3a9740 20 client.25495 trim_cache size 0 max 16384 2024-04-22T13:25:16.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.396+0000 7f18da3a9740 1 Processor -- start 2024-04-22T13:25:16.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.396+0000 7f18da3a9740 1 -- start start 2024-04-22T13:25:16.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.396+0000 7f18da3a9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baaa39b580 0x55baaa3b0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:16.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.396+0000 7f18da3a9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55baaa39bea0 0x55baaa3a4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:16.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.396+0000 7f18da3a9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55baaa3a4a80 0x55baaa396090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:16.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.396+0000 7f18da3a9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f18d01afd70 con 0x55baaa39bea0 2024-04-22T13:25:16.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.396+0000 7f18da3a9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f18d01b18b0 con 0x55baaa3a4a80 2024-04-22T13:25:16.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.396+0000 7f18da3a9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f18d01b0440 con 0x55baaa39b580 2024-04-22T13:25:16.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.396+0000 7f18d5475640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55baaa39bea0 0x55baaa3a4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:16.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.396+0000 7f18d5475640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55baaa39bea0 0x55baaa3a4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55520/0 (socket says 172.21.15.12:55520) 2024-04-22T13:25:16.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.396+0000 7f18d5475640 1 -- 172.21.15.12:0/2976402872 learned_addr learned my addr 172.21.15.12:0/2976402872 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:16.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.397+0000 7f18d5475640 1 -- 172.21.15.12:0/2976402872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baaa39b580 msgr2=0x55baaa3b0730 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-22T13:25:16.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.397+0000 7f18d4c74640 1 --2- 172.21.15.12:0/2976402872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baaa39b580 0x55baaa3b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:16.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.397+0000 7f18d5475640 1 --2- 172.21.15.12:0/2976402872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baaa39b580 0x55baaa3b0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.397+0000 7f18d5475640 1 -- 172.21.15.12:0/2976402872 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55baaa3a4a80 msgr2=0x55baaa396090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-22T13:25:16.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.397+0000 7f18d5475640 1 --2- 172.21.15.12:0/2976402872 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55baaa3a4a80 0x55baaa396090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.397+0000 7f18d5475640 1 -- 172.21.15.12:0/2976402872 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baaa39bcd0 con 0x55baaa39bea0 2024-04-22T13:25:16.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.397+0000 7f18d5475640 1 --2- 172.21.15.12:0/2976402872 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55baaa39bea0 0x55baaa3a4190 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f18cc004ac0 tx=0x7f18cc001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219630c710609446 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:16.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.398+0000 7f18baffd640 1 -- 172.21.15.12:0/2976402872 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc029050 con 0x55baaa39bea0 2024-04-22T13:25:16.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.398+0000 7f18baffd640 1 -- 172.21.15.12:0/2976402872 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18cc0035f0 con 0x55baaa39bea0 2024-04-22T13:25:16.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.398+0000 7f18baffd640 1 -- 172.21.15.12:0/2976402872 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc004340 con 0x55baaa39bea0 2024-04-22T13:25:16.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.399+0000 7f18da3a9740 1 -- 172.21.15.12:0/2976402872 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55baaa39bea0 msgr2=0x55baaa3a4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:16.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.399+0000 7f18da3a9740 1 --2- 172.21.15.12:0/2976402872 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55baaa39bea0 0x55baaa3a4190 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f18cc004ac0 tx=0x7f18cc001e70 comp rx=0 tx=0).stop 2024-04-22T13:25:16.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.399+0000 7f18da3a9740 1 -- 172.21.15.12:0/2976402872 shutdown_connections 2024-04-22T13:25:16.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.399+0000 7f18da3a9740 1 --2- 172.21.15.12:0/2976402872 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55baaa3a4a80 0x55baaa396090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.399+0000 7f18da3a9740 1 --2- 172.21.15.12:0/2976402872 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55baaa39bea0 0x55baaa3a4190 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.399+0000 7f18da3a9740 1 --2- 172.21.15.12:0/2976402872 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baaa39b580 0x55baaa3b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.399+0000 7f18da3a9740 1 -- 172.21.15.12:0/2976402872 >> 172.21.15.12:0/2976402872 conn(0x55baaa395cb0 msgr2=0x55baaa3944e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:16.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.399+0000 7f18da3a9740 1 -- 172.21.15.12:0/2976402872 shutdown_connections 2024-04-22T13:25:16.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.399+0000 7f18da3a9740 1 -- 172.21.15.12:0/2976402872 wait complete. 2024-04-22T13:25:16.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.399+0000 7f18da3a9740 1 -- start start 2024-04-22T13:25:16.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.400+0000 7f18da3a9740 1 Processor -- start 2024-04-22T13:25:16.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.400+0000 7f18da3a9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55baaa39b580 0x55baaa3acea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:16.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.400+0000 7f18da3a9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55baaa39bea0 0x55baaa39e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:16.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.401+0000 7f18da3a9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baaa3a4a80 0x55baaa39e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:16.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.401+0000 7f18da3a9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f18d01b11e0 con 0x55baaa39bea0 2024-04-22T13:25:16.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.401+0000 7f18d5475640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55baaa39bea0 0x55baaa39e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:16.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.401+0000 7f18da3a9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f18d01b0b10 con 0x55baaa39b580 2024-04-22T13:25:16.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.401+0000 7f18da3a9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f18d01add50 con 0x55baaa3a4a80 2024-04-22T13:25:16.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.401+0000 7f18d5475640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55baaa39bea0 0x55baaa39e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55530/0 (socket says 172.21.15.12:55530) 2024-04-22T13:25:16.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.401+0000 7f18d5475640 1 -- 172.21.15.12:0/3027989238 learned_addr learned my addr 172.21.15.12:0/3027989238 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:16.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.401+0000 7f18da3a9740 10 client.0 _ll_register_callbacks cb 0x7f18c002c930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:16.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.401+0000 7f18d4c74640 1 --2- 172.21.15.12:0/3027989238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55baaa39b580 0x55baaa3acea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:16.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.401+0000 7f18d5475640 1 -- 172.21.15.12:0/3027989238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baaa3a4a80 msgr2=0x55baaa39e840 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-22T13:25:16.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.401+0000 7f18d5475640 1 --2- 172.21.15.12:0/3027989238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baaa3a4a80 0x55baaa39e840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.401+0000 7f18d5475640 1 -- 172.21.15.12:0/3027989238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55baaa39b580 msgr2=0x55baaa3acea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:16.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.401+0000 7f18d5475640 1 --2- 172.21.15.12:0/3027989238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55baaa39b580 0x55baaa3acea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.401+0000 7f18d5475640 1 -- 172.21.15.12:0/3027989238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baaa39eeb0 con 0x55baaa39bea0 2024-04-22T13:25:16.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.401+0000 7f18d4c74640 1 --2- 172.21.15.12:0/3027989238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55baaa39b580 0x55baaa3acea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:25:16.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.401+0000 7f18d5475640 1 --2- 172.21.15.12:0/3027989238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55baaa39bea0 0x55baaa39e300 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f18cc004e00 tx=0x7f18cc005210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4707088d216a729 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:16.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.401+0000 7f18b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:16.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.401+0000 7f18b8ff9640 1 -- 172.21.15.12:0/3027989238 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc029070 con 0x55baaa39bea0 2024-04-22T13:25:16.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.401+0000 7f18b8ff9640 1 -- 172.21.15.12:0/3027989238 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f18cc0040e0 con 0x55baaa39bea0 2024-04-22T13:25:16.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.402+0000 7f18da3a9740 1 -- 172.21.15.12:0/3027989238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18cc0035f0 con 0x55baaa39bea0 2024-04-22T13:25:16.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.402+0000 7f18b8ff9640 1 -- 172.21.15.12:0/3027989238 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f18cc005590 con 0x55baaa39bea0 2024-04-22T13:25:16.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.402+0000 7f18da3a9740 10 client.16122 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:16.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.402+0000 7f18da3a9740 1 -- 172.21.15.12:0/3027989238 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55baaa41ab40 con 0x55baaa39bea0 2024-04-22T13:25:16.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.402+0000 7f18b8ff9640 1 -- 172.21.15.12:0/3027989238 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18cc0035f0 con 0x55baaa39bea0 2024-04-22T13:25:16.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.403+0000 7f18b8ff9640 1 -- 172.21.15.12:0/3027989238 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f18cc035070 con 0x55baaa39bea0 2024-04-22T13:25:16.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.403+0000 7f18da3a9740 10 client.16122 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:16.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.403+0000 7f18da3a9740 10 client.16122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:25:16.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.403+0000 7f18da3a9740 20 client.16122 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:16.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.403+0000 7f18da3a9740 1 --2- 172.21.15.12:0/3027989238 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55baaa41b220 0x55baaa41d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:16.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.403+0000 7f18da3a9740 4 client.16122 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:25:16.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.403+0000 7f18da3a9740 1 -- 172.21.15.12:0/3027989238 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55baaa307a70 con 0x55baaa41b220 2024-04-22T13:25:16.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.403+0000 7f18d4c74640 1 --2- 172.21.15.12:0/3027989238 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55baaa41b220 0x55baaa41d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:16.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.404+0000 7f18b8ff9640 10 client.16122 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:16.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.404+0000 7f18d4c74640 1 --2- 172.21.15.12:0/3027989238 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55baaa41b220 0x55baaa41d660 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4833a10dca57c70f server_cookie=987ffd6d93b0b21f in_seq=0 out_seq=0 2024-04-22T13:25:16.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.404+0000 7f18b8ff9640 1 -- 172.21.15.12:0/3027989238 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55baaa307a70 con 0x55baaa41b220 2024-04-22T13:25:16.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.404+0000 7f18b8ff9640 10 client.16122 handle_command_reply: tid=0 2024-04-22T13:25:16.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.404+0000 7f18da3a9740 1 client.16122 shutdown 2024-04-22T13:25:16.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.404+0000 7f18da3a9740 1 -- 172.21.15.12:0/3027989238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55baaa39bea0 msgr2=0x55baaa39e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:16.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.404+0000 7f18da3a9740 1 --2- 172.21.15.12:0/3027989238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55baaa39bea0 0x55baaa39e300 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f18cc004e00 tx=0x7f18cc005210 comp rx=0 tx=0).stop 2024-04-22T13:25:16.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.404+0000 7f18da3a9740 1 -- 172.21.15.12:0/3027989238 shutdown_connections 2024-04-22T13:25:16.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.404+0000 7f18da3a9740 1 --2- 172.21.15.12:0/3027989238 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55baaa41b220 0x55baaa41d660 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.404+0000 7f18da3a9740 1 --2- 172.21.15.12:0/3027989238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55baaa3a4a80 0x55baaa39e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.404+0000 7f18da3a9740 1 --2- 172.21.15.12:0/3027989238 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55baaa39bea0 0x55baaa39e300 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.404+0000 7f18da3a9740 1 --2- 172.21.15.12:0/3027989238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55baaa39b580 0x55baaa3acea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.404+0000 7f18da3a9740 1 -- 172.21.15.12:0/3027989238 >> 172.21.15.12:0/3027989238 conn(0x55baaa395cb0 msgr2=0x55baaa394d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:16.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.404+0000 7f18b8ff9640 0 client.16122 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:16.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.404+0000 7f18da3a9740 1 -- 172.21.15.12:0/3027989238 shutdown_connections 2024-04-22T13:25:16.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.404+0000 7f18da3a9740 1 -- 172.21.15.12:0/3027989238 wait complete. 2024-04-22T13:25:16.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.404+0000 7f18da3a9740 20 client.16122 trim_cache size 0 max 16384 2024-04-22T13:25:16.408 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:25:16.408 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-04-22T13:25:16.408 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-04-22T13:25:16.409 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:25:16.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.405+0000 7f18d76d9640 1 -- 172.21.15.12:0/1539027788 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f18bc0a3f50 msgr2=0x7f18bc0a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:16.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.405+0000 7f18d76d9640 1 --2- 172.21.15.12:0/1539027788 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f18bc0a3f50 0x7f18bc0a6410 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f18c80ce190 tx=0x7f18c801eb40 comp rx=0 tx=0).stop 2024-04-22T13:25:16.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.405+0000 7f18d76d9640 1 -- 172.21.15.12:0/1539027788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18d010e8f0 msgr2=0x7f18d019a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:16.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.405+0000 7f18d76d9640 1 --2- 172.21.15.12:0/1539027788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18d010e8f0 0x7f18d019a7b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f18c800e9d0 tx=0x7f18c800ee90 comp rx=0 tx=0).stop 2024-04-22T13:25:16.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.405+0000 7f18d76d9640 1 -- 172.21.15.12:0/1539027788 shutdown_connections 2024-04-22T13:25:16.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.405+0000 7f18d76d9640 1 --2- 172.21.15.12:0/1539027788 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f18bc0a3f50 0x7f18bc0a6410 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.405+0000 7f18d76d9640 1 --2- 172.21.15.12:0/1539027788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f18d010e8f0 0x7f18d019a7b0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.405+0000 7f18d76d9640 1 --2- 172.21.15.12:0/1539027788 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18d0101660 0x7f18d01a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.405+0000 7f18d76d9640 1 --2- 172.21.15.12:0/1539027788 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18d0100d40 0x7f18d01a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.405+0000 7f18d76d9640 1 -- 172.21.15.12:0/1539027788 >> 172.21.15.12:0/1539027788 conn(0x7f18d0078000 msgr2=0x7f18d010fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:16.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.405+0000 7f18d76d9640 1 -- 172.21.15.12:0/1539027788 shutdown_connections 2024-04-22T13:25:16.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.405+0000 7f18d76d9640 1 -- 172.21.15.12:0/1539027788 wait complete. 2024-04-22T13:25:16.413 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:25:16.414 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:25:16.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.538+0000 7f27bf2e0640 1 -- 172.21.15.12:0/2464884407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27b8107fd0 msgr2=0x7f27b81083b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:16.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.538+0000 7f27bf2e0640 1 --2- 172.21.15.12:0/2464884407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27b8107fd0 0x7f27b81083b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f27b4009930 tx=0x7f27b402bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:16.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.538+0000 7f27bf2e0640 1 -- 172.21.15.12:0/2464884407 shutdown_connections 2024-04-22T13:25:16.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.538+0000 7f27bf2e0640 1 --2- 172.21.15.12:0/2464884407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27b8069aa0 0x7f27b810ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.538+0000 7f27bf2e0640 1 --2- 172.21.15.12:0/2464884407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27b80690e0 0x7f27b8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.538+0000 7f27bf2e0640 1 --2- 172.21.15.12:0/2464884407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27b8107fd0 0x7f27b81083b0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.538+0000 7f27bf2e0640 1 -- 172.21.15.12:0/2464884407 >> 172.21.15.12:0/2464884407 conn(0x7f27b8077e30 msgr2=0x7f27b8078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:16.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.538+0000 7f27bf2e0640 1 -- 172.21.15.12:0/2464884407 shutdown_connections 2024-04-22T13:25:16.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.538+0000 7f27bf2e0640 1 -- 172.21.15.12:0/2464884407 wait complete. 2024-04-22T13:25:16.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.539+0000 7f27bf2e0640 1 Processor -- start 2024-04-22T13:25:16.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.539+0000 7f27bf2e0640 1 -- start start 2024-04-22T13:25:16.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.539+0000 7f27bf2e0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27b80690e0 0x7f27b8101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:16.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.539+0000 7f27bf2e0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27b8069aa0 0x7f27b81017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:16.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.539+0000 7f27bf2e0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27b8107fd0 0x7f27b8101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:16.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.539+0000 7f27bf2e0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f27b8112120 con 0x7f27b8107fd0 2024-04-22T13:25:16.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.539+0000 7f27bf2e0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f27b8111fa0 con 0x7f27b80690e0 2024-04-22T13:25:16.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.539+0000 7f27bd87d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27b8107fd0 0x7f27b8101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:16.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.539+0000 7f27bd87d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27b8107fd0 0x7f27b8101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55540/0 (socket says 172.21.15.12:55540) 2024-04-22T13:25:16.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.539+0000 7f27bd87d640 1 -- 172.21.15.12:0/4275890086 learned_addr learned my addr 172.21.15.12:0/4275890086 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:16.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.539+0000 7f27bf2e0640 1 -- 172.21.15.12:0/4275890086 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f27b81122a0 con 0x7f27b8069aa0 2024-04-22T13:25:16.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.539+0000 7f27bd07c640 1 --2- 172.21.15.12:0/4275890086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27b80690e0 0x7f27b8101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:16.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.539+0000 7f27bc87b640 1 --2- 172.21.15.12:0/4275890086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27b8069aa0 0x7f27b81017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:16.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.540+0000 7f27bd87d640 1 -- 172.21.15.12:0/4275890086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27b8069aa0 msgr2=0x7f27b81017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:16.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.540+0000 7f27bd87d640 1 --2- 172.21.15.12:0/4275890086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27b8069aa0 0x7f27b81017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.540+0000 7f27bd87d640 1 -- 172.21.15.12:0/4275890086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27b80690e0 msgr2=0x7f27b8101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:16.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.540+0000 7f27bd87d640 1 --2- 172.21.15.12:0/4275890086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27b80690e0 0x7f27b8101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.540+0000 7f27bd87d640 1 -- 172.21.15.12:0/4275890086 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b81a6bc0 con 0x7f27b8107fd0 2024-04-22T13:25:16.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.540+0000 7f27bd07c640 1 --2- 172.21.15.12:0/4275890086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27b80690e0 0x7f27b8101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:16.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.540+0000 7f27bd87d640 1 --2- 172.21.15.12:0/4275890086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27b8107fd0 0x7f27b8101cf0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f27b000eaa0 tx=0x7f27b000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:16.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.540+0000 7f27ae7fc640 1 -- 172.21.15.12:0/4275890086 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b000cdb0 con 0x7f27b8107fd0 2024-04-22T13:25:16.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.540+0000 7f27ae7fc640 1 -- 172.21.15.12:0/4275890086 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27b0005020 con 0x7f27b8107fd0 2024-04-22T13:25:16.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.540+0000 7f27bf2e0640 1 -- 172.21.15.12:0/4275890086 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27b81a6eb0 con 0x7f27b8107fd0 2024-04-22T13:25:16.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.540+0000 7f27bf2e0640 1 -- 172.21.15.12:0/4275890086 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27b81a73d0 con 0x7f27b8107fd0 2024-04-22T13:25:16.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.540+0000 7f27ae7fc640 1 -- 172.21.15.12:0/4275890086 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b00116d0 con 0x7f27b8107fd0 2024-04-22T13:25:16.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.542+0000 7f27bf2e0640 1 -- 172.21.15.12:0/4275890086 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27b8103630 con 0x7f27b8107fd0 2024-04-22T13:25:16.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.543+0000 7f27ae7fc640 1 -- 172.21.15.12:0/4275890086 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f27b00051d0 con 0x7f27b8107fd0 2024-04-22T13:25:16.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.544+0000 7f27ae7fc640 1 --2- 172.21.15.12:0/4275890086 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f27a40a3f50 0x7f27a40a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:16.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.545+0000 7f27bd07c640 1 --2- 172.21.15.12:0/4275890086 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f27a40a3f50 0x7f27a40a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:16.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.545+0000 7f27ae7fc640 1 -- 172.21.15.12:0/4275890086 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f27b00d0ac0 con 0x7f27b8107fd0 2024-04-22T13:25:16.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.545+0000 7f27bd07c640 1 --2- 172.21.15.12:0/4275890086 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f27a40a3f50 0x7f27a40a6410 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f27b4005ab0 tx=0x7f27b4005fd0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:16.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.553+0000 7f27ae7fc640 1 -- 172.21.15.12:0/4275890086 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f27b0016030 con 0x7f27b8107fd0 2024-04-22T13:25:16.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.855+0000 7f27bf2e0640 1 -- 172.21.15.12:0/4275890086 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27b8068de0 con 0x7f27b8107fd0 2024-04-22T13:25:16.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.855+0000 7f27ae7fc640 1 -- 172.21.15.12:0/4275890086 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7f27b8068de0 con 0x7f27b8107fd0 2024-04-22T13:25:16.852 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:25:16.852 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:25:16.852 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:25:16.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.858+0000 7f27bf2e0640 1 -- 172.21.15.12:0/4275890086 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f27a40a3f50 msgr2=0x7f27a40a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:16.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.858+0000 7f27bf2e0640 1 --2- 172.21.15.12:0/4275890086 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f27a40a3f50 0x7f27a40a6410 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f27b4005ab0 tx=0x7f27b4005fd0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.858+0000 7f27bf2e0640 1 -- 172.21.15.12:0/4275890086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27b8107fd0 msgr2=0x7f27b8101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:16.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.858+0000 7f27bf2e0640 1 --2- 172.21.15.12:0/4275890086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27b8107fd0 0x7f27b8101cf0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f27b000eaa0 tx=0x7f27b000ef60 comp rx=0 tx=0).stop 2024-04-22T13:25:16.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.859+0000 7f27bf2e0640 1 -- 172.21.15.12:0/4275890086 shutdown_connections 2024-04-22T13:25:16.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.859+0000 7f27bf2e0640 1 --2- 172.21.15.12:0/4275890086 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f27a40a3f50 0x7f27a40a6410 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.859+0000 7f27bf2e0640 1 --2- 172.21.15.12:0/4275890086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27b8107fd0 0x7f27b8101cf0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.859+0000 7f27bf2e0640 1 --2- 172.21.15.12:0/4275890086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27b8069aa0 0x7f27b81017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.859+0000 7f27bf2e0640 1 --2- 172.21.15.12:0/4275890086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27b80690e0 0x7f27b8101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:16.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.859+0000 7f27bf2e0640 1 -- 172.21.15.12:0/4275890086 >> 172.21.15.12:0/4275890086 conn(0x7f27b8077e30 msgr2=0x7f27b8100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:16.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.859+0000 7f27bf2e0640 1 -- 172.21.15.12:0/4275890086 shutdown_connections 2024-04-22T13:25:16.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:16.859+0000 7f27bf2e0640 1 -- 172.21.15.12:0/4275890086 wait complete. 2024-04-22T13:25:16.870 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:25:17.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.023+0000 7f7ee83ef640 1 -- 172.21.15.12:0/3692408767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ee0101a40 msgr2=0x7f7ee0105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.023+0000 7f7ee83ef640 1 --2- 172.21.15.12:0/3692408767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ee0101a40 0x7f7ee0105a80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0009900 tx=0x7f7ed002bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:17.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.023+0000 7f7ee83ef640 1 -- 172.21.15.12:0/3692408767 shutdown_connections 2024-04-22T13:25:17.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.023+0000 7f7ee83ef640 1 --2- 172.21.15.12:0/3692408767 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ee0106160 0x7f7ee010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.023+0000 7f7ee83ef640 1 --2- 172.21.15.12:0/3692408767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ee0101a40 0x7f7ee0105a80 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.023+0000 7f7ee83ef640 1 --2- 172.21.15.12:0/3692408767 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ee0101090 0x7f7ee0101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.023+0000 7f7ee83ef640 1 -- 172.21.15.12:0/3692408767 >> 172.21.15.12:0/3692408767 conn(0x7f7ee0077f50 msgr2=0x7f7ee0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.023+0000 7f7ee83ef640 1 -- 172.21.15.12:0/3692408767 shutdown_connections 2024-04-22T13:25:17.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.023+0000 7f7ee83ef640 1 -- 172.21.15.12:0/3692408767 wait complete. 2024-04-22T13:25:17.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.024+0000 7f7ee83ef640 1 Processor -- start 2024-04-22T13:25:17.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.024+0000 7f7ee83ef640 1 -- start start 2024-04-22T13:25:17.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.024+0000 7f7ee83ef640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ee0101090 0x7f7ee019e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.024+0000 7f7ee83ef640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ee0106160 0x7f7ee019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.024+0000 7f7ee83ef640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ee01985d0 0x7f7ee0198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.024+0000 7f7ee83ef640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee0111080 con 0x7f7ee0106160 2024-04-22T13:25:17.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.024+0000 7f7ee83ef640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee0110f00 con 0x7f7ee0101090 2024-04-22T13:25:17.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.024+0000 7f7ee83ef640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee0111200 con 0x7f7ee01985d0 2024-04-22T13:25:17.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.024+0000 7f7ee598a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ee0106160 0x7f7ee019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.024+0000 7f7ee598a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ee0106160 0x7f7ee019e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55548/0 (socket says 172.21.15.12:55548) 2024-04-22T13:25:17.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.024+0000 7f7ee598a640 1 -- 172.21.15.12:0/1665509120 learned_addr learned my addr 172.21.15.12:0/1665509120 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:17.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.025+0000 7f7ee618b640 1 --2- 172.21.15.12:0/1665509120 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ee0101090 0x7f7ee019e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.025+0000 7f7ee698c640 1 --2- 172.21.15.12:0/1665509120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ee01985d0 0x7f7ee0198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.025+0000 7f7ee598a640 1 -- 172.21.15.12:0/1665509120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ee01985d0 msgr2=0x7f7ee0198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.025+0000 7f7ee598a640 1 --2- 172.21.15.12:0/1665509120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ee01985d0 0x7f7ee0198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.025+0000 7f7ee598a640 1 -- 172.21.15.12:0/1665509120 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ee0101090 msgr2=0x7f7ee019e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.025+0000 7f7ee598a640 1 --2- 172.21.15.12:0/1665509120 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ee0101090 0x7f7ee019e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.025+0000 7f7ee598a640 1 -- 172.21.15.12:0/1665509120 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee01990f0 con 0x7f7ee0106160 2024-04-22T13:25:17.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.025+0000 7f7ee618b640 1 --2- 172.21.15.12:0/1665509120 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ee0101090 0x7f7ee019e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:17.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.025+0000 7f7ee598a640 1 --2- 172.21.15.12:0/1665509120 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ee0106160 0x7f7ee019e960 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0036040 tx=0x7f7ed00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.025+0000 7f7ed77fe640 1 -- 172.21.15.12:0/1665509120 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0043020 con 0x7f7ee0106160 2024-04-22T13:25:17.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.025+0000 7f7ee83ef640 1 -- 172.21.15.12:0/1665509120 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee0199380 con 0x7f7ee0106160 2024-04-22T13:25:17.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.025+0000 7f7ed77fe640 1 -- 172.21.15.12:0/1665509120 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ed002cb70 con 0x7f7ee0106160 2024-04-22T13:25:17.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.025+0000 7f7ed77fe640 1 -- 172.21.15.12:0/1665509120 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0003bf0 con 0x7f7ee0106160 2024-04-22T13:25:17.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.025+0000 7f7ee83ef640 1 -- 172.21.15.12:0/1665509120 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee01a5620 con 0x7f7ee0106160 2024-04-22T13:25:17.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.028+0000 7f7eeafd6740 1 Processor -- start 2024-04-22T13:25:17.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.028+0000 7f7eeafd6740 1 -- start start 2024-04-22T13:25:17.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.029+0000 7f7eeafd6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55630fa2b4d0 0x55630fa2b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.029+0000 7f7eeafd6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55630fa2be80 0x55630fa34430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.029+0000 7f7eeafd6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55630fa34a00 0x55630fa36e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.029+0000 7f7eeafd6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55630f9901f0 con 0x55630fa2be80 2024-04-22T13:25:17.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.029+0000 7f7eeafd6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55630f92d4b0 con 0x55630fa34a00 2024-04-22T13:25:17.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.029+0000 7f7eeafd6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55630f98c1b0 con 0x55630fa2b4d0 2024-04-22T13:25:17.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.029+0000 7f7ee618b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55630fa2be80 0x55630fa34430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.029+0000 7f7ee618b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55630fa2be80 0x55630fa34430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55556/0 (socket says 172.21.15.12:55556) 2024-04-22T13:25:17.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.029+0000 7f7ee618b640 1 -- 172.21.15.12:0/4260390176 learned_addr learned my addr 172.21.15.12:0/4260390176 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:17.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.029+0000 7f7ee698c640 1 --2- 172.21.15.12:0/4260390176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55630fa2b4d0 0x55630fa2b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.029+0000 7f7ed77fe640 1 -- 172.21.15.12:0/1665509120 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f7ed0046020 con 0x7f7ee0106160 2024-04-22T13:25:17.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.029+0000 7f7ee618b640 1 -- 172.21.15.12:0/4260390176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55630fa2b4d0 msgr2=0x55630fa2b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.029+0000 7f7ee618b640 1 --2- 172.21.15.12:0/4260390176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55630fa2b4d0 0x55630fa2b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.029+0000 7f7ee618b640 1 -- 172.21.15.12:0/4260390176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55630fa34a00 msgr2=0x55630fa36e50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T13:25:17.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.029+0000 7f7ee618b640 1 --2- 172.21.15.12:0/4260390176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55630fa34a00 0x55630fa36e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.029+0000 7f7ee618b640 1 -- 172.21.15.12:0/4260390176 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55630f91f1b0 con 0x55630fa2be80 2024-04-22T13:25:17.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.029+0000 7f7ee698c640 1 --2- 172.21.15.12:0/4260390176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55630fa2b4d0 0x55630fa2b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:17.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.030+0000 7f7ee618b640 1 --2- 172.21.15.12:0/4260390176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55630fa2be80 0x55630fa34430 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7edc00bcc0 tx=0x7f7edc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b9c2115e0103055 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.030+0000 7f7ed4bfe640 1 -- 172.21.15.12:0/4260390176 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7edc005020 con 0x55630fa2be80 2024-04-22T13:25:17.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.030+0000 7f7ed4bfe640 1 -- 172.21.15.12:0/4260390176 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7edc0047a0 con 0x55630fa2be80 2024-04-22T13:25:17.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.030+0000 7f7ed4bfe640 1 -- 172.21.15.12:0/4260390176 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7edc0116a0 con 0x55630fa2be80 2024-04-22T13:25:17.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.030+0000 7f7ed77fe640 1 --2- 172.21.15.12:0/1665509120 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7ecc0a3f50 0x7f7ecc0a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.030+0000 7f7ed77fe640 1 -- 172.21.15.12:0/1665509120 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f7ed00f2b60 con 0x7f7ee0106160 2024-04-22T13:25:17.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.031+0000 7f7eeafd6740 1 -- 172.21.15.12:0/4260390176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55630fa2be80 msgr2=0x55630fa34430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.031+0000 7f7eeafd6740 1 --2- 172.21.15.12:0/4260390176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55630fa2be80 0x55630fa34430 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7edc00bcc0 tx=0x7f7edc00f730 comp rx=0 tx=0).stop 2024-04-22T13:25:17.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.031+0000 7f7ee698c640 1 --2- 172.21.15.12:0/1665509120 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7ecc0a3f50 0x7f7ecc0a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.031+0000 7f7eeafd6740 1 -- 172.21.15.12:0/4260390176 shutdown_connections 2024-04-22T13:25:17.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.031+0000 7f7eeafd6740 1 --2- 172.21.15.12:0/4260390176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55630fa34a00 0x55630fa36e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.031+0000 7f7eeafd6740 1 --2- 172.21.15.12:0/4260390176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55630fa2be80 0x55630fa34430 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.031+0000 7f7eeafd6740 1 --2- 172.21.15.12:0/4260390176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55630fa2b4d0 0x55630fa2b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.031+0000 7f7eeafd6740 1 -- 172.21.15.12:0/4260390176 >> 172.21.15.12:0/4260390176 conn(0x55630fa25ba0 msgr2=0x55630fa25f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.031+0000 7f7eeafd6740 1 -- 172.21.15.12:0/4260390176 shutdown_connections 2024-04-22T13:25:17.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.031+0000 7f7eeafd6740 1 -- 172.21.15.12:0/4260390176 wait complete. 2024-04-22T13:25:17.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.031+0000 7f7ee698c640 1 --2- 172.21.15.12:0/1665509120 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7ecc0a3f50 0x7f7ecc0a6410 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8005fd0 tx=0x7f7ed8009500 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.031+0000 7f7eeafd6740 1 -- start start 2024-04-22T13:25:17.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.032+0000 7f7eeafd6740 1 Processor -- start 2024-04-22T13:25:17.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.032+0000 7f7eeafd6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55630fa2b4d0 0x55630fa34300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.032+0000 7f7eeafd6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55630fa2be80 0x55630fa31690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.032+0000 7f7eeafd6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55630fa34a00 0x55630fa31c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.032+0000 7f7eeafd6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55630fa37970 con 0x55630fa2be80 2024-04-22T13:25:17.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.032+0000 7f7eeafd6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55630f949500 con 0x55630fa2b4d0 2024-04-22T13:25:17.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.032+0000 7f7eeafd6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55630fa37af0 con 0x55630fa34a00 2024-04-22T13:25:17.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.032+0000 7f7eeafd6740 10 client.0 _ll_register_callbacks cb 0x55630f917b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:17.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.032+0000 7f7ee598a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55630fa2be80 0x55630fa31690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.032+0000 7f7ee598a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55630fa2be80 0x55630fa31690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55568/0 (socket says 172.21.15.12:55568) 2024-04-22T13:25:17.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.032+0000 7f7ee598a640 1 -- 172.21.15.12:0/2746533843 learned_addr learned my addr 172.21.15.12:0/2746533843 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:17.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.032+0000 7f7ee618b640 1 --2- 172.21.15.12:0/2746533843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55630fa2b4d0 0x55630fa34300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.032+0000 7f7ee698c640 1 --2- 172.21.15.12:0/2746533843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55630fa34a00 0x55630fa31c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.032+0000 7f7ee598a640 1 -- 172.21.15.12:0/2746533843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55630fa34a00 msgr2=0x55630fa31c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.032+0000 7f7ee598a640 1 --2- 172.21.15.12:0/2746533843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55630fa34a00 0x55630fa31c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.032+0000 7f7ee598a640 1 -- 172.21.15.12:0/2746533843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55630fa2b4d0 msgr2=0x55630fa34300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.032+0000 7f7ee598a640 1 --2- 172.21.15.12:0/2746533843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55630fa2b4d0 0x55630fa34300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.032+0000 7f7ee598a640 1 -- 172.21.15.12:0/2746533843 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55630f6a23f0 con 0x55630fa2be80 2024-04-22T13:25:17.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.032+0000 7f7ee698c640 1 --2- 172.21.15.12:0/2746533843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55630fa34a00 0x55630fa31c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:17.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.032+0000 7f7ee618b640 1 --2- 172.21.15.12:0/2746533843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55630fa2b4d0 0x55630fa34300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:25:17.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.033+0000 7f7ee598a640 1 --2- 172.21.15.12:0/2746533843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55630fa2be80 0x55630fa31690 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f7ed00f02c0 tx=0x7f7ed00badb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47cab69c7c0c2efa server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.033+0000 7f7ec9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:17.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.033+0000 7f7ec9ffb640 1 -- 172.21.15.12:0/2746533843 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed00bf120 con 0x55630fa2be80 2024-04-22T13:25:17.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.033+0000 7f7ec9ffb640 1 -- 172.21.15.12:0/2746533843 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f7ed00bec80 con 0x55630fa2be80 2024-04-22T13:25:17.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.033+0000 7f7eeafd6740 1 -- 172.21.15.12:0/2746533843 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7edc0047a0 con 0x55630fa2be80 2024-04-22T13:25:17.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.033+0000 7f7ec9ffb640 1 -- 172.21.15.12:0/2746533843 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7ed00bea20 con 0x55630fa2be80 2024-04-22T13:25:17.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.033+0000 7f7eeafd6740 10 client.16140 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:17.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.033+0000 7f7eeafd6740 1 -- 172.21.15.12:0/2746533843 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55630f76c7d0 con 0x55630fa2be80 2024-04-22T13:25:17.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.034+0000 7f7ec9ffb640 1 -- 172.21.15.12:0/2746533843 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7edc0047a0 con 0x55630fa2be80 2024-04-22T13:25:17.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.034+0000 7f7ec9ffb640 1 -- 172.21.15.12:0/2746533843 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7ed00dc050 con 0x55630fa2be80 2024-04-22T13:25:17.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.034+0000 7f7eeafd6740 10 client.16140 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:17.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.034+0000 7f7eeafd6740 10 client.16140 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:25:17.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.034+0000 7f7eeafd6740 20 client.16140 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:17.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.034+0000 7f7eeafd6740 1 --2- 172.21.15.12:0/2746533843 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55630fa4d5e0 0x55630fa4fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.034+0000 7f7eeafd6740 4 client.16140 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:25:17.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.034+0000 7f7eeafd6740 1 -- 172.21.15.12:0/2746533843 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55630f92f930 con 0x55630fa4d5e0 2024-04-22T13:25:17.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.034+0000 7f7ee618b640 1 --2- 172.21.15.12:0/2746533843 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55630fa4d5e0 0x55630fa4fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.035+0000 7f7ec9ffb640 10 client.16140 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:25:17.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.035+0000 7f7ee618b640 1 --2- 172.21.15.12:0/2746533843 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55630fa4d5e0 0x55630fa4fa40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f985615be1640aad server_cookie=17e4ec410efd9 in_seq=0 out_seq=0 2024-04-22T13:25:17.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.035+0000 7f7ec9ffb640 1 -- 172.21.15.12:0/2746533843 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55630f92f930 con 0x55630fa4d5e0 2024-04-22T13:25:17.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.035+0000 7f7ec9ffb640 10 client.16140 handle_command_reply: tid=0 2024-04-22T13:25:17.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.035+0000 7f7eeafd6740 1 client.16140 shutdown 2024-04-22T13:25:17.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.035+0000 7f7eeafd6740 1 -- 172.21.15.12:0/2746533843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55630fa2be80 msgr2=0x55630fa31690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.035+0000 7f7eeafd6740 1 --2- 172.21.15.12:0/2746533843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55630fa2be80 0x55630fa31690 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f7ed00f02c0 tx=0x7f7ed00badb0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.035+0000 7f7eeafd6740 1 -- 172.21.15.12:0/2746533843 shutdown_connections 2024-04-22T13:25:17.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.035+0000 7f7eeafd6740 1 --2- 172.21.15.12:0/2746533843 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55630fa4d5e0 0x55630fa4fa40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.035+0000 7f7eeafd6740 1 --2- 172.21.15.12:0/2746533843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55630fa34a00 0x55630fa31c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.035+0000 7f7eeafd6740 1 --2- 172.21.15.12:0/2746533843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55630fa2be80 0x55630fa31690 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.035+0000 7f7eeafd6740 1 --2- 172.21.15.12:0/2746533843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55630fa2b4d0 0x55630fa34300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.035+0000 7f7eeafd6740 1 -- 172.21.15.12:0/2746533843 >> 172.21.15.12:0/2746533843 conn(0x55630fa25ba0 msgr2=0x55630fa278f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.036+0000 7f7ec9ffb640 0 client.16140 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:25:17.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.036+0000 7f7eeafd6740 1 -- 172.21.15.12:0/2746533843 shutdown_connections 2024-04-22T13:25:17.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.036+0000 7f7eeafd6740 1 -- 172.21.15.12:0/2746533843 wait complete. 2024-04-22T13:25:17.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.036+0000 7f7eeafd6740 20 client.16140 trim_cache size 0 max 16384 2024-04-22T13:25:17.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.060+0000 7f7eeafd6740 1 Processor -- start 2024-04-22T13:25:17.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.060+0000 7f7eeafd6740 1 -- start start 2024-04-22T13:25:17.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.061+0000 7f7eeafd6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55630fa2b4d0 0x55630fa307f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.061+0000 7f7eeafd6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55630fa2be80 0x55630fa27700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.061+0000 7f7eeafd6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55630fa34a00 0x55630fa33dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.061+0000 7f7eeafd6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee01ab910 con 0x55630fa2be80 2024-04-22T13:25:17.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.061+0000 7f7eeafd6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee01ad450 con 0x55630fa34a00 2024-04-22T13:25:17.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.061+0000 7f7eeafd6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee01abfe0 con 0x55630fa2b4d0 2024-04-22T13:25:17.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.061+0000 7f7ee598a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55630fa2be80 0x55630fa27700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.061+0000 7f7ee598a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55630fa2be80 0x55630fa27700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55580/0 (socket says 172.21.15.12:55580) 2024-04-22T13:25:17.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.061+0000 7f7ee598a640 1 -- 172.21.15.12:0/2997536700 learned_addr learned my addr 172.21.15.12:0/2997536700 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:17.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.061+0000 7f7ee618b640 1 --2- 172.21.15.12:0/2997536700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55630fa2b4d0 0x55630fa307f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.061+0000 7f7ee698c640 1 --2- 172.21.15.12:0/2997536700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55630fa34a00 0x55630fa33dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.061+0000 7f7ee598a640 1 -- 172.21.15.12:0/2997536700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55630fa2b4d0 msgr2=0x55630fa307f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.061+0000 7f7ee598a640 1 --2- 172.21.15.12:0/2997536700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55630fa2b4d0 0x55630fa307f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.061+0000 7f7ee598a640 1 -- 172.21.15.12:0/2997536700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55630fa34a00 msgr2=0x55630fa33dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.061+0000 7f7ee598a640 1 --2- 172.21.15.12:0/2997536700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55630fa34a00 0x55630fa33dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.061+0000 7f7ee598a640 1 -- 172.21.15.12:0/2997536700 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55630fa2bc20 con 0x55630fa2be80 2024-04-22T13:25:17.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.061+0000 7f7ee598a640 1 --2- 172.21.15.12:0/2997536700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55630fa2be80 0x55630fa27700 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0005ec0 tx=0x7f7ed00bcf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6350d4f9925aeabd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.062+0000 7f7ecbfff640 1 -- 172.21.15.12:0/2997536700 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed00bf120 con 0x55630fa2be80 2024-04-22T13:25:17.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.062+0000 7f7ecbfff640 1 -- 172.21.15.12:0/2997536700 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ed00c7690 con 0x55630fa2be80 2024-04-22T13:25:17.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.062+0000 7f7ecbfff640 1 -- 172.21.15.12:0/2997536700 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed00ba770 con 0x55630fa2be80 2024-04-22T13:25:17.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.062+0000 7f7eeafd6740 1 -- 172.21.15.12:0/2997536700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55630fa2be80 msgr2=0x55630fa27700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.062+0000 7f7eeafd6740 1 --2- 172.21.15.12:0/2997536700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55630fa2be80 0x55630fa27700 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0005ec0 tx=0x7f7ed00bcf30 comp rx=0 tx=0).stop 2024-04-22T13:25:17.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.062+0000 7f7eeafd6740 1 -- 172.21.15.12:0/2997536700 shutdown_connections 2024-04-22T13:25:17.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.062+0000 7f7eeafd6740 1 --2- 172.21.15.12:0/2997536700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55630fa34a00 0x55630fa33dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.062+0000 7f7eeafd6740 1 --2- 172.21.15.12:0/2997536700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55630fa2be80 0x55630fa27700 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.062+0000 7f7eeafd6740 1 --2- 172.21.15.12:0/2997536700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55630fa2b4d0 0x55630fa307f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.062+0000 7f7eeafd6740 1 -- 172.21.15.12:0/2997536700 >> 172.21.15.12:0/2997536700 conn(0x55630fa25ba0 msgr2=0x55630fa26ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.062+0000 7f7eeafd6740 1 -- 172.21.15.12:0/2997536700 shutdown_connections 2024-04-22T13:25:17.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.063+0000 7f7eeafd6740 1 -- 172.21.15.12:0/2997536700 wait complete. 2024-04-22T13:25:17.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.063+0000 7f7eeafd6740 1 -- start start 2024-04-22T13:25:17.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.063+0000 7f7eeafd6740 1 Processor -- start 2024-04-22T13:25:17.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.063+0000 7f7eeafd6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55630fa2b4d0 0x55630fa45630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.063+0000 7f7eeafd6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55630fa34a00 0x55630fa45b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.063+0000 7f7eeafd6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55630fa2d980 0x55630fa2de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.063+0000 7f7eeafd6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee01acd80 con 0x55630fa34a00 2024-04-22T13:25:17.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.063+0000 7f7eeafd6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee01ac6b0 con 0x55630fa2b4d0 2024-04-22T13:25:17.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.063+0000 7f7eeafd6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee01a9930 con 0x55630fa2d980 2024-04-22T13:25:17.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.063+0000 7f7ee598a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55630fa34a00 0x55630fa45b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.063+0000 7f7ee598a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55630fa34a00 0x55630fa45b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55592/0 (socket says 172.21.15.12:55592) 2024-04-22T13:25:17.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.063+0000 7f7ee598a640 1 -- 172.21.15.12:0/3566662876 learned_addr learned my addr 172.21.15.12:0/3566662876 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:17.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.063+0000 7f7eeafd6740 10 client.0 _ll_register_callbacks cb 0x7f7ed00df0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:17.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.063+0000 7f7ee698c640 1 --2- 172.21.15.12:0/3566662876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55630fa2d980 0x55630fa2de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.063+0000 7f7ee618b640 1 --2- 172.21.15.12:0/3566662876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55630fa2b4d0 0x55630fa45630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.064+0000 7f7ee598a640 1 -- 172.21.15.12:0/3566662876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55630fa2d980 msgr2=0x55630fa2de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.064+0000 7f7ee598a640 1 --2- 172.21.15.12:0/3566662876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55630fa2d980 0x55630fa2de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.064+0000 7f7ee598a640 1 -- 172.21.15.12:0/3566662876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55630fa2b4d0 msgr2=0x55630fa45630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.064+0000 7f7ee598a640 1 --2- 172.21.15.12:0/3566662876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55630fa2b4d0 0x55630fa45630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.064+0000 7f7ee598a640 1 -- 172.21.15.12:0/3566662876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55630fa2e440 con 0x55630fa34a00 2024-04-22T13:25:17.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.064+0000 7f7ee598a640 1 --2- 172.21.15.12:0/3566662876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55630fa34a00 0x55630fa45b70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f7ed00f02c0 tx=0x7f7ed0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=902bfcfdaf2f3934 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.064+0000 7f7ec9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:17.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.064+0000 7f7ec9ffb640 1 -- 172.21.15.12:0/3566662876 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed00a2020 con 0x55630fa34a00 2024-04-22T13:25:17.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.064+0000 7f7ec9ffb640 1 -- 172.21.15.12:0/3566662876 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f7ed00be070 con 0x55630fa34a00 2024-04-22T13:25:17.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.064+0000 7f7eeafd6740 1 -- 172.21.15.12:0/3566662876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ed00c7690 con 0x55630fa34a00 2024-04-22T13:25:17.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.064+0000 7f7ec9ffb640 1 -- 172.21.15.12:0/3566662876 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7ed00bf120 con 0x55630fa34a00 2024-04-22T13:25:17.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.064+0000 7f7eeafd6740 10 client.16146 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:17.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.064+0000 7f7eeafd6740 1 -- 172.21.15.12:0/3566662876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55630fa2e440 con 0x55630fa34a00 2024-04-22T13:25:17.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.065+0000 7f7ec9ffb640 1 -- 172.21.15.12:0/3566662876 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ed00c7690 con 0x55630fa34a00 2024-04-22T13:25:17.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.065+0000 7f7ec9ffb640 1 -- 172.21.15.12:0/3566662876 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7ed00dc070 con 0x55630fa34a00 2024-04-22T13:25:17.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.065+0000 7f7eeafd6740 10 client.16146 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:17.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.065+0000 7f7eeafd6740 10 client.16146 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:25:17.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.065+0000 7f7eeafd6740 20 client.16146 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:17.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.065+0000 7f7eeafd6740 1 --2- 172.21.15.12:0/3566662876 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55630fab3c50 0x55630fab6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.065+0000 7f7eeafd6740 4 client.16146 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:25:17.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.065+0000 7f7eeafd6740 1 -- 172.21.15.12:0/3566662876 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55630f99dc10 con 0x55630fab3c50 2024-04-22T13:25:17.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.065+0000 7f7ee618b640 1 --2- 172.21.15.12:0/3566662876 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55630fab3c50 0x55630fab6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.066+0000 7f7ec9ffb640 10 client.16146 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:25:17.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.066+0000 7f7ee618b640 1 --2- 172.21.15.12:0/3566662876 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55630fab3c50 0x55630fab6070 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5cf16c8668539850 server_cookie=ac9b45d1599f214d in_seq=0 out_seq=0 2024-04-22T13:25:17.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.066+0000 7f7ec9ffb640 1 -- 172.21.15.12:0/3566662876 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55630f99dc10 con 0x55630fab3c50 2024-04-22T13:25:17.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.066+0000 7f7ec9ffb640 10 client.16146 handle_command_reply: tid=0 2024-04-22T13:25:17.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.066+0000 7f7eeafd6740 1 client.16146 shutdown 2024-04-22T13:25:17.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.066+0000 7f7eeafd6740 1 -- 172.21.15.12:0/3566662876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55630fa34a00 msgr2=0x55630fa45b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.066+0000 7f7eeafd6740 1 --2- 172.21.15.12:0/3566662876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55630fa34a00 0x55630fa45b70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f7ed00f02c0 tx=0x7f7ed0005fd0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.066+0000 7f7eeafd6740 1 -- 172.21.15.12:0/3566662876 shutdown_connections 2024-04-22T13:25:17.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.066+0000 7f7eeafd6740 1 --2- 172.21.15.12:0/3566662876 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55630fab3c50 0x55630fab6070 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.066+0000 7f7eeafd6740 1 --2- 172.21.15.12:0/3566662876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55630fa2d980 0x55630fa2de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.066+0000 7f7eeafd6740 1 --2- 172.21.15.12:0/3566662876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55630fa34a00 0x55630fa45b70 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.066+0000 7f7eeafd6740 1 --2- 172.21.15.12:0/3566662876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55630fa2b4d0 0x55630fa45630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.066+0000 7f7eeafd6740 1 -- 172.21.15.12:0/3566662876 >> 172.21.15.12:0/3566662876 conn(0x55630fa25ba0 msgr2=0x55630fa24370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.066+0000 7f7ec9ffb640 0 client.16146 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:25:17.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.066+0000 7f7eeafd6740 1 -- 172.21.15.12:0/3566662876 shutdown_connections 2024-04-22T13:25:17.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.066+0000 7f7eeafd6740 1 -- 172.21.15.12:0/3566662876 wait complete. 2024-04-22T13:25:17.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.067+0000 7f7eeafd6740 20 client.16146 trim_cache size 0 max 16384 2024-04-22T13:25:17.069 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:25:17.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.067+0000 7f7ee83ef640 1 -- 172.21.15.12:0/1665509120 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7ecc0a3f50 msgr2=0x7f7ecc0a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.067+0000 7f7ee83ef640 1 --2- 172.21.15.12:0/1665509120 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7ecc0a3f50 0x7f7ecc0a6410 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8005fd0 tx=0x7f7ed8009500 comp rx=0 tx=0).stop 2024-04-22T13:25:17.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.067+0000 7f7ee83ef640 1 -- 172.21.15.12:0/1665509120 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ee0106160 msgr2=0x7f7ee019e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.067+0000 7f7ee83ef640 1 --2- 172.21.15.12:0/1665509120 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ee0106160 0x7f7ee019e960 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0036040 tx=0x7f7ed00045e0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.067+0000 7f7ee83ef640 1 -- 172.21.15.12:0/1665509120 shutdown_connections 2024-04-22T13:25:17.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.067+0000 7f7ee83ef640 1 --2- 172.21.15.12:0/1665509120 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7ecc0a3f50 0x7f7ecc0a6410 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.067+0000 7f7ee83ef640 1 --2- 172.21.15.12:0/1665509120 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7ee01985d0 0x7f7ee0198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.067+0000 7f7ee83ef640 1 --2- 172.21.15.12:0/1665509120 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7ee0106160 0x7f7ee019e960 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.067+0000 7f7ee83ef640 1 --2- 172.21.15.12:0/1665509120 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ee0101090 0x7f7ee019e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.067+0000 7f7ee83ef640 1 -- 172.21.15.12:0/1665509120 >> 172.21.15.12:0/1665509120 conn(0x7f7ee0077f50 msgr2=0x7f7ee0069580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.067+0000 7f7ee83ef640 1 -- 172.21.15.12:0/1665509120 shutdown_connections 2024-04-22T13:25:17.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.067+0000 7f7ee83ef640 1 -- 172.21.15.12:0/1665509120 wait complete. 2024-04-22T13:25:17.076 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:25:17.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.189+0000 7f4cdb73d640 1 -- 172.21.15.12:0/1934434335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cd406a820 msgr2=0x7f4cd406ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.189+0000 7f4cdb73d640 1 --2- 172.21.15.12:0/1934434335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cd406a820 0x7f4cd406ac00 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009900 tx=0x7f4cd002bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:17.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.189+0000 7f4cdb73d640 1 -- 172.21.15.12:0/1934434335 shutdown_connections 2024-04-22T13:25:17.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.189+0000 7f4cdb73d640 1 --2- 172.21.15.12:0/1934434335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4cd4107270 0x7f4cd410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.189+0000 7f4cdb73d640 1 --2- 172.21.15.12:0/1934434335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cd406b1d0 0x7f4cd4106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.189+0000 7f4cdb73d640 1 --2- 172.21.15.12:0/1934434335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cd406a820 0x7f4cd406ac00 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.189+0000 7f4cdb73d640 1 -- 172.21.15.12:0/1934434335 >> 172.21.15.12:0/1934434335 conn(0x7f4cd4077eb0 msgr2=0x7f4cd40782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.189+0000 7f4cdb73d640 1 -- 172.21.15.12:0/1934434335 shutdown_connections 2024-04-22T13:25:17.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.189+0000 7f4cdb73d640 1 -- 172.21.15.12:0/1934434335 wait complete. 2024-04-22T13:25:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.190+0000 7f4cdb73d640 1 Processor -- start 2024-04-22T13:25:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.190+0000 7f4cdb73d640 1 -- start start 2024-04-22T13:25:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.190+0000 7f4cdb73d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4cd406a820 0x7f4cd41987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.190+0000 7f4cdb73d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cd4107270 0x7f4cd4198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.190+0000 7f4cdb73d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cd419bff0 0x7f4cd419c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.190+0000 7f4cdb73d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd4112220 con 0x7f4cd4107270 2024-04-22T13:25:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.190+0000 7f4cdb73d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd41120a0 con 0x7f4cd406a820 2024-04-22T13:25:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.190+0000 7f4cdb73d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd41123a0 con 0x7f4cd419bff0 2024-04-22T13:25:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.190+0000 7f4cd8cd8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cd4107270 0x7f4cd4198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.190+0000 7f4cd8cd8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cd4107270 0x7f4cd4198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55618/0 (socket says 172.21.15.12:55618) 2024-04-22T13:25:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.190+0000 7f4cd8cd8640 1 -- 172.21.15.12:0/1398336777 learned_addr learned my addr 172.21.15.12:0/1398336777 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.190+0000 7f4cd9cda640 1 --2- 172.21.15.12:0/1398336777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cd419bff0 0x7f4cd419c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.190+0000 7f4cd94d9640 1 --2- 172.21.15.12:0/1398336777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4cd406a820 0x7f4cd41987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.190+0000 7f4cd8cd8640 1 -- 172.21.15.12:0/1398336777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cd419bff0 msgr2=0x7f4cd419c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.190+0000 7f4cd8cd8640 1 --2- 172.21.15.12:0/1398336777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cd419bff0 0x7f4cd419c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.190+0000 7f4cd8cd8640 1 -- 172.21.15.12:0/1398336777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4cd406a820 msgr2=0x7f4cd41987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.190+0000 7f4cd8cd8640 1 --2- 172.21.15.12:0/1398336777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4cd406a820 0x7f4cd41987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.190+0000 7f4cd8cd8640 1 -- 172.21.15.12:0/1398336777 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd419cae0 con 0x7f4cd4107270 2024-04-22T13:25:17.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.190+0000 7f4cd9cda640 1 --2- 172.21.15.12:0/1398336777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cd419bff0 0x7f4cd419c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:17.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.191+0000 7f4cd8cd8640 1 --2- 172.21.15.12:0/1398336777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cd4107270 0x7f4cd4198d20 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4cc400e9f0 tx=0x7f4cc400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.191+0000 7f4cca7fc640 1 -- 172.21.15.12:0/1398336777 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc400ccc0 con 0x7f4cd4107270 2024-04-22T13:25:17.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.191+0000 7f4cdb73d640 1 -- 172.21.15.12:0/1398336777 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cd419cdd0 con 0x7f4cd4107270 2024-04-22T13:25:17.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.191+0000 7f4cca7fc640 1 -- 172.21.15.12:0/1398336777 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cc40054d0 con 0x7f4cd4107270 2024-04-22T13:25:17.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.191+0000 7f4cdb73d640 1 -- 172.21.15.12:0/1398336777 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cd41010a0 con 0x7f4cd4107270 2024-04-22T13:25:17.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.191+0000 7f4cca7fc640 1 -- 172.21.15.12:0/1398336777 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4021440 con 0x7f4cd4107270 2024-04-22T13:25:17.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.194+0000 7f4cde406740 1 Processor -- start 2024-04-22T13:25:17.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.194+0000 7f4cde406740 1 -- start start 2024-04-22T13:25:17.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.194+0000 7f4cde406740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570b3f114d0 0x5570b3f118b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.194+0000 7f4cde406740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570b3f11e80 0x5570b3f1a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.194+0000 7f4cde406740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b3f1aa00 0x5570b3f1ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.194+0000 7f4cde406740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5570b3e761f0 con 0x5570b3f114d0 2024-04-22T13:25:17.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.194+0000 7f4cde406740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5570b3e134b0 con 0x5570b3f1aa00 2024-04-22T13:25:17.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.194+0000 7f4cde406740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5570b3e721b0 con 0x5570b3f11e80 2024-04-22T13:25:17.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.194+0000 7f4cd9cda640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570b3f114d0 0x5570b3f118b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.194+0000 7f4cd9cda640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570b3f114d0 0x5570b3f118b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55620/0 (socket says 172.21.15.12:55620) 2024-04-22T13:25:17.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.194+0000 7f4cd9cda640 1 -- 172.21.15.12:0/1952235316 learned_addr learned my addr 172.21.15.12:0/1952235316 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:17.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.195+0000 7f4cd94d9640 1 --2- 172.21.15.12:0/1952235316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570b3f11e80 0x5570b3f1a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.195+0000 7f4cca7fc640 1 -- 172.21.15.12:0/1398336777 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f4cc40046c0 con 0x7f4cd4107270 2024-04-22T13:25:17.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.195+0000 7f4cd9cda640 1 -- 172.21.15.12:0/1952235316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570b3f11e80 msgr2=0x5570b3f1a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.195+0000 7f4cd9cda640 1 --2- 172.21.15.12:0/1952235316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570b3f11e80 0x5570b3f1a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.195+0000 7f4cd9cda640 1 -- 172.21.15.12:0/1952235316 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b3f1aa00 msgr2=0x5570b3f1ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.195+0000 7f4cd9cda640 1 --2- 172.21.15.12:0/1952235316 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b3f1aa00 0x5570b3f1ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.195+0000 7f4cd9cda640 1 -- 172.21.15.12:0/1952235316 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570b3e051b0 con 0x5570b3f114d0 2024-04-22T13:25:17.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.195+0000 7f4cd94d9640 1 --2- 172.21.15.12:0/1952235316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570b3f11e80 0x5570b3f1a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:17.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.195+0000 7f4cd9cda640 1 --2- 172.21.15.12:0/1952235316 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570b3f114d0 0x5570b3f118b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc011790 tx=0x7f4ccc011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6cef630bfd28fd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.195+0000 7f4cbf7fe640 1 -- 172.21.15.12:0/1952235316 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc003c30 con 0x5570b3f114d0 2024-04-22T13:25:17.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.195+0000 7f4cbf7fe640 1 -- 172.21.15.12:0/1952235316 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ccc00fe50 con 0x5570b3f114d0 2024-04-22T13:25:17.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.195+0000 7f4cbf7fe640 1 -- 172.21.15.12:0/1952235316 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc0193e0 con 0x5570b3f114d0 2024-04-22T13:25:17.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.196+0000 7f4cca7fc640 1 --2- 172.21.15.12:0/1398336777 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4cc00a3ed0 0x7f4cc00a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.196+0000 7f4cd94d9640 1 --2- 172.21.15.12:0/1398336777 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4cc00a3ed0 0x7f4cc00a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.196+0000 7f4cca7fc640 1 -- 172.21.15.12:0/1398336777 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f4cc40d0df0 con 0x7f4cd4107270 2024-04-22T13:25:17.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.196+0000 7f4cd94d9640 1 --2- 172.21.15.12:0/1398336777 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4cc00a3ed0 0x7f4cc00a6390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0005ab0 tx=0x7f4cd0005a20 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.196+0000 7f4cde406740 1 -- 172.21.15.12:0/1952235316 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570b3f114d0 msgr2=0x5570b3f118b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.196+0000 7f4cde406740 1 --2- 172.21.15.12:0/1952235316 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570b3f114d0 0x5570b3f118b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc011790 tx=0x7f4ccc011c50 comp rx=0 tx=0).stop 2024-04-22T13:25:17.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.196+0000 7f4cde406740 1 -- 172.21.15.12:0/1952235316 shutdown_connections 2024-04-22T13:25:17.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.196+0000 7f4cde406740 1 --2- 172.21.15.12:0/1952235316 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b3f1aa00 0x5570b3f1ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.196+0000 7f4cde406740 1 --2- 172.21.15.12:0/1952235316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570b3f11e80 0x5570b3f1a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.196+0000 7f4cde406740 1 --2- 172.21.15.12:0/1952235316 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570b3f114d0 0x5570b3f118b0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.196+0000 7f4cde406740 1 -- 172.21.15.12:0/1952235316 >> 172.21.15.12:0/1952235316 conn(0x5570b3f0bba0 msgr2=0x5570b3f0bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.197+0000 7f4cde406740 1 -- 172.21.15.12:0/1952235316 shutdown_connections 2024-04-22T13:25:17.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.197+0000 7f4cde406740 1 -- 172.21.15.12:0/1952235316 wait complete. 2024-04-22T13:25:17.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.197+0000 7f4cde406740 1 -- start start 2024-04-22T13:25:17.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.197+0000 7f4cde406740 1 Processor -- start 2024-04-22T13:25:17.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.197+0000 7f4cde406740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570b3f114d0 0x5570b3f1a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.198+0000 7f4cde406740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570b3f11e80 0x5570b3f17700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.198+0000 7f4cde406740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b3f1aa00 0x5570b3f17d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.198+0000 7f4cde406740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5570b3f1d970 con 0x5570b3f114d0 2024-04-22T13:25:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.198+0000 7f4cd94d9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570b3f114d0 0x5570b3f1a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.198+0000 7f4cd94d9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570b3f114d0 0x5570b3f1a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55634/0 (socket says 172.21.15.12:55634) 2024-04-22T13:25:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.198+0000 7f4cde406740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5570b3e2f500 con 0x5570b3f1aa00 2024-04-22T13:25:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.198+0000 7f4cd94d9640 1 -- 172.21.15.12:0/1042585038 learned_addr learned my addr 172.21.15.12:0/1042585038 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.198+0000 7f4cde406740 1 -- 172.21.15.12:0/1042585038 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5570b3f1daf0 con 0x5570b3f11e80 2024-04-22T13:25:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.198+0000 7f4cde406740 10 client.0 _ll_register_callbacks cb 0x5570b3dfdb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.198+0000 7f4cd9cda640 1 --2- 172.21.15.12:0/1042585038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b3f1aa00 0x5570b3f17d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.198+0000 7f4cd8cd8640 1 --2- 172.21.15.12:0/1042585038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570b3f11e80 0x5570b3f17700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.198+0000 7f4cd94d9640 1 -- 172.21.15.12:0/1042585038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570b3f11e80 msgr2=0x5570b3f17700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.198+0000 7f4cd94d9640 1 --2- 172.21.15.12:0/1042585038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570b3f11e80 0x5570b3f17700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.198+0000 7f4cd94d9640 1 -- 172.21.15.12:0/1042585038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b3f1aa00 msgr2=0x5570b3f17d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.198+0000 7f4cd94d9640 1 --2- 172.21.15.12:0/1042585038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b3f1aa00 0x5570b3f17d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.198+0000 7f4cd94d9640 1 -- 172.21.15.12:0/1042585038 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570b3b883f0 con 0x5570b3f114d0 2024-04-22T13:25:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.198+0000 7f4cd94d9640 1 --2- 172.21.15.12:0/1042585038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570b3f114d0 0x5570b3f1a370 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0000c00 tx=0x7f4cd002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e4c660b1a812d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.198+0000 7f4cbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.199+0000 7f4cbcff9640 1 -- 172.21.15.12:0/1042585038 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd004f030 con 0x5570b3f114d0 2024-04-22T13:25:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.199+0000 7f4cbcff9640 1 -- 172.21.15.12:0/1042585038 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f4cd003a040 con 0x5570b3f114d0 2024-04-22T13:25:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.199+0000 7f4cde406740 1 -- 172.21.15.12:0/1042585038 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ccc00fe50 con 0x5570b3f114d0 2024-04-22T13:25:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.199+0000 7f4cbcff9640 1 -- 172.21.15.12:0/1042585038 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4cd0058020 con 0x5570b3f114d0 2024-04-22T13:25:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.199+0000 7f4cde406740 10 client.16158 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.199+0000 7f4cde406740 1 -- 172.21.15.12:0/1042585038 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5570b3c527d0 con 0x5570b3f114d0 2024-04-22T13:25:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.199+0000 7f4cbcff9640 1 -- 172.21.15.12:0/1042585038 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ccc00fe50 con 0x5570b3f114d0 2024-04-22T13:25:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.199+0000 7f4cbcff9640 1 -- 172.21.15.12:0/1042585038 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4cd00037b0 con 0x5570b3f114d0 2024-04-22T13:25:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.199+0000 7f4cde406740 10 client.16158 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.199+0000 7f4cde406740 10 client.16158 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:25:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.200+0000 7f4cde406740 20 client.16158 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.200+0000 7f4cde406740 1 --2- 172.21.15.12:0/1042585038 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5570b3f2aba0 0x5570b3f2cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.200+0000 7f4cde406740 4 client.16158 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:25:17.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.200+0000 7f4cde406740 1 -- 172.21.15.12:0/1042585038 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570b3e15930 con 0x5570b3f2aba0 2024-04-22T13:25:17.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.200+0000 7f4cd8cd8640 1 --2- 172.21.15.12:0/1042585038 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5570b3f2aba0 0x5570b3f2cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.200+0000 7f4cd8cd8640 1 --2- 172.21.15.12:0/1042585038 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5570b3f2aba0 0x5570b3f2cfc0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7068212747128777 server_cookie=efbd1d7d12c5452e in_seq=0 out_seq=0 2024-04-22T13:25:17.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.200+0000 7f4cbcff9640 10 client.16158 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:25:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.201+0000 7f4cbcff9640 1 -- 172.21.15.12:0/1042585038 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5570b3e15930 con 0x5570b3f2aba0 2024-04-22T13:25:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.201+0000 7f4cbcff9640 10 client.16158 handle_command_reply: tid=0 2024-04-22T13:25:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.201+0000 7f4cde406740 1 client.16158 shutdown 2024-04-22T13:25:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.202+0000 7f4cde406740 1 -- 172.21.15.12:0/1042585038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570b3f114d0 msgr2=0x5570b3f1a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.202+0000 7f4cde406740 1 --2- 172.21.15.12:0/1042585038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570b3f114d0 0x5570b3f1a370 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0000c00 tx=0x7f4cd002bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.202+0000 7f4cde406740 1 -- 172.21.15.12:0/1042585038 shutdown_connections 2024-04-22T13:25:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.202+0000 7f4cde406740 1 --2- 172.21.15.12:0/1042585038 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5570b3f2aba0 0x5570b3f2cfc0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.202+0000 7f4cde406740 1 --2- 172.21.15.12:0/1042585038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b3f1aa00 0x5570b3f17d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.202+0000 7f4cde406740 1 --2- 172.21.15.12:0/1042585038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570b3f11e80 0x5570b3f17700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.202+0000 7f4cde406740 1 --2- 172.21.15.12:0/1042585038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570b3f114d0 0x5570b3f1a370 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.202+0000 7f4cde406740 1 -- 172.21.15.12:0/1042585038 >> 172.21.15.12:0/1042585038 conn(0x5570b3f0bba0 msgr2=0x5570b3f0d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.202+0000 7f4cbcff9640 0 client.16158 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:25:17.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.202+0000 7f4cde406740 1 -- 172.21.15.12:0/1042585038 shutdown_connections 2024-04-22T13:25:17.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.202+0000 7f4cde406740 1 -- 172.21.15.12:0/1042585038 wait complete. 2024-04-22T13:25:17.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.202+0000 7f4cde406740 20 client.16158 trim_cache size 0 max 16384 2024-04-22T13:25:17.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.219+0000 7f4cde406740 1 Processor -- start 2024-04-22T13:25:17.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.219+0000 7f4cde406740 1 -- start start 2024-04-22T13:25:17.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.219+0000 7f4cde406740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570b3f114d0 0x5570b3f26830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.219+0000 7f4cde406740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b3f11e80 0x5570b3f16860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.219+0000 7f4cde406740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570b3f1aa00 0x5570b3f12ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.219+0000 7f4cde406740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd41afd50 con 0x5570b3f1aa00 2024-04-22T13:25:17.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.219+0000 7f4cde406740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd41b1890 con 0x5570b3f11e80 2024-04-22T13:25:17.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.219+0000 7f4cde406740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd41b0420 con 0x5570b3f114d0 2024-04-22T13:25:17.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.220+0000 7f4cd9cda640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570b3f1aa00 0x5570b3f12ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.220+0000 7f4cd9cda640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570b3f1aa00 0x5570b3f12ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55644/0 (socket says 172.21.15.12:55644) 2024-04-22T13:25:17.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.220+0000 7f4cd9cda640 1 -- 172.21.15.12:0/2190603037 learned_addr learned my addr 172.21.15.12:0/2190603037 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:17.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.220+0000 7f4cd8cd8640 1 --2- 172.21.15.12:0/2190603037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b3f11e80 0x5570b3f16860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.220+0000 7f4cd94d9640 1 --2- 172.21.15.12:0/2190603037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570b3f114d0 0x5570b3f26830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.220+0000 7f4cd9cda640 1 -- 172.21.15.12:0/2190603037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570b3f114d0 msgr2=0x5570b3f26830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.220+0000 7f4cd9cda640 1 --2- 172.21.15.12:0/2190603037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570b3f114d0 0x5570b3f26830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.220+0000 7f4cd9cda640 1 -- 172.21.15.12:0/2190603037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b3f11e80 msgr2=0x5570b3f16860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.220+0000 7f4cd9cda640 1 --2- 172.21.15.12:0/2190603037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b3f11e80 0x5570b3f16860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.220+0000 7f4cd9cda640 1 -- 172.21.15.12:0/2190603037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570b3f11c20 con 0x5570b3f1aa00 2024-04-22T13:25:17.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.220+0000 7f4cd8cd8640 1 --2- 172.21.15.12:0/2190603037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b3f11e80 0x5570b3f16860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:17.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.220+0000 7f4cd9cda640 1 --2- 172.21.15.12:0/2190603037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570b3f1aa00 0x5570b3f12ae0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc011760 tx=0x7f4ccc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dab51665e2dad46 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.220+0000 7f4cbeffd640 1 -- 172.21.15.12:0/2190603037 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc02a070 con 0x5570b3f1aa00 2024-04-22T13:25:17.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.220+0000 7f4cbeffd640 1 -- 172.21.15.12:0/2190603037 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ccc005050 con 0x5570b3f1aa00 2024-04-22T13:25:17.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.220+0000 7f4cbeffd640 1 -- 172.21.15.12:0/2190603037 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc01f040 con 0x5570b3f1aa00 2024-04-22T13:25:17.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.221+0000 7f4cde406740 1 -- 172.21.15.12:0/2190603037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570b3f1aa00 msgr2=0x5570b3f12ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.221+0000 7f4cde406740 1 --2- 172.21.15.12:0/2190603037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570b3f1aa00 0x5570b3f12ae0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc011760 tx=0x7f4ccc009f90 comp rx=0 tx=0).stop 2024-04-22T13:25:17.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.221+0000 7f4cde406740 1 -- 172.21.15.12:0/2190603037 shutdown_connections 2024-04-22T13:25:17.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.221+0000 7f4cde406740 1 --2- 172.21.15.12:0/2190603037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570b3f1aa00 0x5570b3f12ae0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.221+0000 7f4cde406740 1 --2- 172.21.15.12:0/2190603037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b3f11e80 0x5570b3f16860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.221+0000 7f4cde406740 1 --2- 172.21.15.12:0/2190603037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570b3f114d0 0x5570b3f26830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.221+0000 7f4cde406740 1 -- 172.21.15.12:0/2190603037 >> 172.21.15.12:0/2190603037 conn(0x5570b3f0bba0 msgr2=0x5570b3f123d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.221+0000 7f4cde406740 1 -- 172.21.15.12:0/2190603037 shutdown_connections 2024-04-22T13:25:17.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.221+0000 7f4cde406740 1 -- 172.21.15.12:0/2190603037 wait complete. 2024-04-22T13:25:17.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.221+0000 7f4cde406740 1 -- start start 2024-04-22T13:25:17.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.222+0000 7f4cde406740 1 Processor -- start 2024-04-22T13:25:17.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.222+0000 7f4cde406740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570b3f114d0 0x5570b3f23270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.222+0000 7f4cde406740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b3f11e80 0x5570b3f1e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.222+0000 7f4cde406740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570b3f1aa00 0x5570b3f1ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.222+0000 7f4cde406740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd41b11c0 con 0x5570b3f114d0 2024-04-22T13:25:17.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.222+0000 7f4cde406740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd41b0af0 con 0x5570b3f11e80 2024-04-22T13:25:17.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.222+0000 7f4cde406740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd41add50 con 0x5570b3f1aa00 2024-04-22T13:25:17.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.222+0000 7f4cde406740 10 client.0 _ll_register_callbacks cb 0x7f4cd002c6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:17.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.222+0000 7f4cd94d9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570b3f114d0 0x5570b3f23270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.222+0000 7f4cd94d9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570b3f114d0 0x5570b3f23270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55652/0 (socket says 172.21.15.12:55652) 2024-04-22T13:25:17.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.222+0000 7f4cd94d9640 1 -- 172.21.15.12:0/611100924 learned_addr learned my addr 172.21.15.12:0/611100924 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:17.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.222+0000 7f4cd8cd8640 1 --2- 172.21.15.12:0/611100924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b3f11e80 0x5570b3f1e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.222+0000 7f4cd9cda640 1 --2- 172.21.15.12:0/611100924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570b3f1aa00 0x5570b3f1ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.222+0000 7f4cd94d9640 1 -- 172.21.15.12:0/611100924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570b3f1aa00 msgr2=0x5570b3f1ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.222+0000 7f4cd94d9640 1 --2- 172.21.15.12:0/611100924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570b3f1aa00 0x5570b3f1ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.222+0000 7f4cd94d9640 1 -- 172.21.15.12:0/611100924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b3f11e80 msgr2=0x5570b3f1e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.222+0000 7f4cd94d9640 1 --2- 172.21.15.12:0/611100924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b3f11e80 0x5570b3f1e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.222+0000 7f4cd94d9640 1 -- 172.21.15.12:0/611100924 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570b3f90150 con 0x5570b3f114d0 2024-04-22T13:25:17.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.222+0000 7f4cd8cd8640 1 --2- 172.21.15.12:0/611100924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b3f11e80 0x5570b3f1e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:17.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.223+0000 7f4cd94d9640 1 --2- 172.21.15.12:0/611100924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570b3f114d0 0x5570b3f23270 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f4cd002c320 tx=0x7f4cd0004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f2f73b56c0fe05 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.223+0000 7f4cbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:17.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.223+0000 7f4cbcff9640 1 -- 172.21.15.12:0/611100924 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0061020 con 0x5570b3f114d0 2024-04-22T13:25:17.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.223+0000 7f4cbcff9640 1 -- 172.21.15.12:0/611100924 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f4cd00046c0 con 0x5570b3f114d0 2024-04-22T13:25:17.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.223+0000 7f4cde406740 1 -- 172.21.15.12:0/611100924 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ccc005050 con 0x5570b3f114d0 2024-04-22T13:25:17.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.223+0000 7f4cbcff9640 1 -- 172.21.15.12:0/611100924 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4cd004f040 con 0x5570b3f114d0 2024-04-22T13:25:17.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.223+0000 7f4cde406740 10 client.16164 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:17.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.223+0000 7f4cde406740 1 -- 172.21.15.12:0/611100924 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5570b3f912d0 con 0x5570b3f114d0 2024-04-22T13:25:17.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.223+0000 7f4cbcff9640 1 -- 172.21.15.12:0/611100924 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ccc005050 con 0x5570b3f114d0 2024-04-22T13:25:17.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.223+0000 7f4cbcff9640 1 -- 172.21.15.12:0/611100924 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4cd00667d0 con 0x5570b3f114d0 2024-04-22T13:25:17.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.223+0000 7f4cde406740 10 client.16164 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:17.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.223+0000 7f4cde406740 10 client.16164 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:25:17.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.223+0000 7f4cde406740 20 client.16164 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:17.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.223+0000 7f4cde406740 1 --2- 172.21.15.12:0/611100924 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5570b3f91a10 0x5570b3f93e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.223+0000 7f4cde406740 4 client.16164 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:25:17.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.223+0000 7f4cde406740 1 -- 172.21.15.12:0/611100924 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5570b3e83c10 con 0x5570b3f91a10 2024-04-22T13:25:17.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.224+0000 7f4cd8cd8640 1 --2- 172.21.15.12:0/611100924 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5570b3f91a10 0x5570b3f93e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.224+0000 7f4cbcff9640 10 client.16164 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:25:17.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.224+0000 7f4cd8cd8640 1 --2- 172.21.15.12:0/611100924 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5570b3f91a10 0x5570b3f93e30 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=353079122df830b5 server_cookie=7dd55b888d8559ec in_seq=0 out_seq=0 2024-04-22T13:25:17.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.224+0000 7f4cbcff9640 1 -- 172.21.15.12:0/611100924 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5570b3e83c10 con 0x5570b3f91a10 2024-04-22T13:25:17.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.224+0000 7f4cbcff9640 10 client.16164 handle_command_reply: tid=0 2024-04-22T13:25:17.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.224+0000 7f4cde406740 1 client.16164 shutdown 2024-04-22T13:25:17.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.224+0000 7f4cde406740 1 -- 172.21.15.12:0/611100924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570b3f114d0 msgr2=0x5570b3f23270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.224+0000 7f4cde406740 1 --2- 172.21.15.12:0/611100924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570b3f114d0 0x5570b3f23270 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f4cd002c320 tx=0x7f4cd0004230 comp rx=0 tx=0).stop 2024-04-22T13:25:17.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.224+0000 7f4cde406740 1 -- 172.21.15.12:0/611100924 shutdown_connections 2024-04-22T13:25:17.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.224+0000 7f4cde406740 1 --2- 172.21.15.12:0/611100924 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5570b3f91a10 0x5570b3f93e30 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.224+0000 7f4cde406740 1 --2- 172.21.15.12:0/611100924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570b3f1aa00 0x5570b3f1ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.224+0000 7f4cbcff9640 0 client.16164 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:25:17.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.224+0000 7f4cde406740 1 --2- 172.21.15.12:0/611100924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570b3f11e80 0x5570b3f1e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.224+0000 7f4cde406740 1 --2- 172.21.15.12:0/611100924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570b3f114d0 0x5570b3f23270 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.224+0000 7f4cde406740 1 -- 172.21.15.12:0/611100924 >> 172.21.15.12:0/611100924 conn(0x5570b3f0bba0 msgr2=0x5570b3f166e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.225+0000 7f4cde406740 1 -- 172.21.15.12:0/611100924 shutdown_connections 2024-04-22T13:25:17.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.225+0000 7f4cde406740 1 -- 172.21.15.12:0/611100924 wait complete. 2024-04-22T13:25:17.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.225+0000 7f4cde406740 20 client.16164 trim_cache size 0 max 16384 2024-04-22T13:25:17.227 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:25:17.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.225+0000 7f4cdb73d640 1 -- 172.21.15.12:0/1398336777 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4cc00a3ed0 msgr2=0x7f4cc00a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.225+0000 7f4cdb73d640 1 --2- 172.21.15.12:0/1398336777 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4cc00a3ed0 0x7f4cc00a6390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0005ab0 tx=0x7f4cd0005a20 comp rx=0 tx=0).stop 2024-04-22T13:25:17.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.225+0000 7f4cdb73d640 1 -- 172.21.15.12:0/1398336777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cd4107270 msgr2=0x7f4cd4198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.225+0000 7f4cdb73d640 1 --2- 172.21.15.12:0/1398336777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cd4107270 0x7f4cd4198d20 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4cc400e9f0 tx=0x7f4cc400eeb0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.225+0000 7f4cdb73d640 1 -- 172.21.15.12:0/1398336777 shutdown_connections 2024-04-22T13:25:17.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.225+0000 7f4cdb73d640 1 --2- 172.21.15.12:0/1398336777 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4cc00a3ed0 0x7f4cc00a6390 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.225+0000 7f4cdb73d640 1 --2- 172.21.15.12:0/1398336777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4cd419bff0 0x7f4cd419c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.225+0000 7f4cdb73d640 1 --2- 172.21.15.12:0/1398336777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4cd4107270 0x7f4cd4198d20 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.225+0000 7f4cdb73d640 1 --2- 172.21.15.12:0/1398336777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4cd406a820 0x7f4cd41987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.225+0000 7f4cdb73d640 1 -- 172.21.15.12:0/1398336777 >> 172.21.15.12:0/1398336777 conn(0x7f4cd4077eb0 msgr2=0x7f4cd4108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.225+0000 7f4cdb73d640 1 -- 172.21.15.12:0/1398336777 shutdown_connections 2024-04-22T13:25:17.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.225+0000 7f4cdb73d640 1 -- 172.21.15.12:0/1398336777 wait complete. 2024-04-22T13:25:17.232 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:25:17.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.350+0000 7f47ba8fd640 1 -- 172.21.15.12:0/2357458494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b410c510 msgr2=0x7f47b410e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.350+0000 7f47ba8fd640 1 --2- 172.21.15.12:0/2357458494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b410c510 0x7f47b410e9b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009ca0 tx=0x7f47ac02c230 comp rx=0 tx=0).stop 2024-04-22T13:25:17.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.350+0000 7f47ba8fd640 1 -- 172.21.15.12:0/2357458494 shutdown_connections 2024-04-22T13:25:17.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.350+0000 7f47ba8fd640 1 --2- 172.21.15.12:0/2357458494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b410c510 0x7f47b410e9b0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.350+0000 7f47ba8fd640 1 --2- 172.21.15.12:0/2357458494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47b4103af0 0x7f47b410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.350+0000 7f47ba8fd640 1 --2- 172.21.15.12:0/2357458494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47b41031d0 0x7f47b41035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.350+0000 7f47ba8fd640 1 -- 172.21.15.12:0/2357458494 >> 172.21.15.12:0/2357458494 conn(0x7f47b4077f50 msgr2=0x7f47b4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.351+0000 7f47ba8fd640 1 -- 172.21.15.12:0/2357458494 shutdown_connections 2024-04-22T13:25:17.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.351+0000 7f47ba8fd640 1 -- 172.21.15.12:0/2357458494 wait complete. 2024-04-22T13:25:17.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.351+0000 7f47ba8fd640 1 Processor -- start 2024-04-22T13:25:17.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.351+0000 7f47ba8fd640 1 -- start start 2024-04-22T13:25:17.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.352+0000 7f47ba8fd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47b41031d0 0x7f47b4100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.352+0000 7f47ba8fd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b4103af0 0x7f47b4101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.352+0000 7f47ba8fd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47b410c510 0x7f47b41018a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.352+0000 7f47ba8fd640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41121a0 con 0x7f47b4103af0 2024-04-22T13:25:17.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.352+0000 7f47ba8fd640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f47b4112020 con 0x7f47b410c510 2024-04-22T13:25:17.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.352+0000 7f47ba8fd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f47b4112320 con 0x7f47b41031d0 2024-04-22T13:25:17.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.352+0000 7f47b37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b4103af0 0x7f47b4101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.352+0000 7f47b37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b4103af0 0x7f47b4101360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55666/0 (socket says 172.21.15.12:55666) 2024-04-22T13:25:17.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.352+0000 7f47b37fe640 1 -- 172.21.15.12:0/2929671467 learned_addr learned my addr 172.21.15.12:0/2929671467 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:17.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.352+0000 7f47b3fff640 1 --2- 172.21.15.12:0/2929671467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47b41031d0 0x7f47b4100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.352+0000 7f47b8e9a640 1 --2- 172.21.15.12:0/2929671467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47b410c510 0x7f47b41018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.352+0000 7f47b37fe640 1 -- 172.21.15.12:0/2929671467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47b41031d0 msgr2=0x7f47b4100e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.352+0000 7f47b37fe640 1 --2- 172.21.15.12:0/2929671467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47b41031d0 0x7f47b4100e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.352+0000 7f47b37fe640 1 -- 172.21.15.12:0/2929671467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47b410c510 msgr2=0x7f47b41018a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.352+0000 7f47b37fe640 1 --2- 172.21.15.12:0/2929671467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47b410c510 0x7f47b41018a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.352+0000 7f47b37fe640 1 -- 172.21.15.12:0/2929671467 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b41a6b60 con 0x7f47b4103af0 2024-04-22T13:25:17.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.352+0000 7f47b3fff640 1 --2- 172.21.15.12:0/2929671467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47b41031d0 0x7f47b4100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:17.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.352+0000 7f47b8e9a640 1 --2- 172.21.15.12:0/2929671467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47b410c510 0x7f47b41018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:17.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.353+0000 7f47b37fe640 1 --2- 172.21.15.12:0/2929671467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b4103af0 0x7f47b4101360 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f47a800d770 tx=0x7f47a800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.353+0000 7f47b17fa640 1 -- 172.21.15.12:0/2929671467 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a8017070 con 0x7f47b4103af0 2024-04-22T13:25:17.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.353+0000 7f47b17fa640 1 -- 172.21.15.12:0/2929671467 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47a80054d0 con 0x7f47b4103af0 2024-04-22T13:25:17.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.353+0000 7f47ba8fd640 1 -- 172.21.15.12:0/2929671467 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47b41a6e50 con 0x7f47b4103af0 2024-04-22T13:25:17.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.353+0000 7f47b17fa640 1 -- 172.21.15.12:0/2929671467 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a8004c30 con 0x7f47b4103af0 2024-04-22T13:25:17.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.353+0000 7f47ba8fd640 1 -- 172.21.15.12:0/2929671467 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47b41a7310 con 0x7f47b4103af0 2024-04-22T13:25:17.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.356+0000 7f47bd5c8740 1 Processor -- start 2024-04-22T13:25:17.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.356+0000 7f47b17fa640 1 -- 172.21.15.12:0/2929671467 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f47a8004dd0 con 0x7f47b4103af0 2024-04-22T13:25:17.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.356+0000 7f47bd5c8740 1 -- start start 2024-04-22T13:25:17.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.356+0000 7f47bd5c8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558df2b734d0 0x558df2b738b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.356+0000 7f47bd5c8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558df2b73e80 0x558df2b7c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.356+0000 7f47bd5c8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558df2b7ca00 0x558df2b7ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.356+0000 7f47bd5c8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558df2ad81f0 con 0x558df2b734d0 2024-04-22T13:25:17.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.356+0000 7f47bd5c8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x558df2a754b0 con 0x558df2b7ca00 2024-04-22T13:25:17.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.357+0000 7f47bd5c8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558df2ad41b0 con 0x558df2b73e80 2024-04-22T13:25:17.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.357+0000 7f47b8e9a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558df2b734d0 0x558df2b738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.357+0000 7f47b8e9a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558df2b734d0 0x558df2b738b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55672/0 (socket says 172.21.15.12:55672) 2024-04-22T13:25:17.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.357+0000 7f47b8e9a640 1 -- 172.21.15.12:0/635924770 learned_addr learned my addr 172.21.15.12:0/635924770 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:17.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.357+0000 7f47b3fff640 1 --2- 172.21.15.12:0/635924770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558df2b73e80 0x558df2b7c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.357+0000 7f47b37fe640 1 --2- 172.21.15.12:0/635924770 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558df2b7ca00 0x558df2b7ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.357+0000 7f47b8e9a640 1 -- 172.21.15.12:0/635924770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558df2b73e80 msgr2=0x558df2b7c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.357+0000 7f47b8e9a640 1 --2- 172.21.15.12:0/635924770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558df2b73e80 0x558df2b7c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.357+0000 7f47b8e9a640 1 -- 172.21.15.12:0/635924770 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558df2b7ca00 msgr2=0x558df2b7ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.357+0000 7f47b8e9a640 1 --2- 172.21.15.12:0/635924770 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558df2b7ca00 0x558df2b7ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.357+0000 7f47b8e9a640 1 -- 172.21.15.12:0/635924770 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558df2a671b0 con 0x558df2b734d0 2024-04-22T13:25:17.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.357+0000 7f47b3fff640 1 --2- 172.21.15.12:0/635924770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558df2b73e80 0x558df2b7c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:17.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.357+0000 7f47b37fe640 1 --2- 172.21.15.12:0/635924770 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558df2b7ca00 0x558df2b7ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:17.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.357+0000 7f47b17fa640 1 --2- 172.21.15.12:0/2929671467 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f479c0a3f80 0x7f479c0a6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.357+0000 7f47b3fff640 1 --2- 172.21.15.12:0/2929671467 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f479c0a3f80 0x7f479c0a6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.357+0000 7f47b17fa640 1 -- 172.21.15.12:0/2929671467 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f47a80d04a0 con 0x7f47b4103af0 2024-04-22T13:25:17.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.357+0000 7f47b8e9a640 1 --2- 172.21.15.12:0/635924770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558df2b734d0 0x558df2b738b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f47ac036000 tx=0x7f47ac032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a70fbd895ec42 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.358+0000 7f479a7fc640 1 -- 172.21.15.12:0/635924770 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac043020 con 0x558df2b734d0 2024-04-22T13:25:17.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.358+0000 7f479a7fc640 1 -- 172.21.15.12:0/635924770 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47ac003b70 con 0x558df2b734d0 2024-04-22T13:25:17.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.358+0000 7f479a7fc640 1 -- 172.21.15.12:0/635924770 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac039040 con 0x558df2b734d0 2024-04-22T13:25:17.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.359+0000 7f47bd5c8740 1 -- 172.21.15.12:0/635924770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558df2b734d0 msgr2=0x558df2b738b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.359+0000 7f47b3fff640 1 --2- 172.21.15.12:0/2929671467 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f479c0a3f80 0x7f479c0a6440 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f47a4005fd0 tx=0x7f47a4005f40 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.359+0000 7f47bd5c8740 1 --2- 172.21.15.12:0/635924770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558df2b734d0 0x558df2b738b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f47ac036000 tx=0x7f47ac032340 comp rx=0 tx=0).stop 2024-04-22T13:25:17.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.359+0000 7f47bd5c8740 1 -- 172.21.15.12:0/635924770 shutdown_connections 2024-04-22T13:25:17.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.359+0000 7f47bd5c8740 1 --2- 172.21.15.12:0/635924770 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558df2b7ca00 0x558df2b7ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.359+0000 7f47bd5c8740 1 --2- 172.21.15.12:0/635924770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558df2b73e80 0x558df2b7c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.359+0000 7f47bd5c8740 1 --2- 172.21.15.12:0/635924770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558df2b734d0 0x558df2b738b0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.359+0000 7f47bd5c8740 1 -- 172.21.15.12:0/635924770 >> 172.21.15.12:0/635924770 conn(0x558df2b6dba0 msgr2=0x558df2b6df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.359+0000 7f47bd5c8740 1 -- 172.21.15.12:0/635924770 shutdown_connections 2024-04-22T13:25:17.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.359+0000 7f47bd5c8740 1 -- 172.21.15.12:0/635924770 wait complete. 2024-04-22T13:25:17.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.359+0000 7f47bd5c8740 1 -- start start 2024-04-22T13:25:17.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.359+0000 7f47bd5c8740 1 Processor -- start 2024-04-22T13:25:17.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.360+0000 7f47bd5c8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558df2b734d0 0x558df2b7c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.360+0000 7f47bd5c8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558df2b73e80 0x558df2b79720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.360+0000 7f47bd5c8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558df2b7ca00 0x558df2b79d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.360+0000 7f47bd5c8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558df2b7f910 con 0x558df2b7ca00 2024-04-22T13:25:17.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.360+0000 7f47bd5c8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x558df2a91500 con 0x558df2b73e80 2024-04-22T13:25:17.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.360+0000 7f47bd5c8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558df2b7fa90 con 0x558df2b734d0 2024-04-22T13:25:17.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.360+0000 7f47bd5c8740 10 client.0 _ll_register_callbacks cb 0x558df2a5fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:17.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.360+0000 7f47b8e9a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558df2b7ca00 0x558df2b79d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.360+0000 7f47b3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558df2b734d0 0x558df2b7c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.360+0000 7f47b8e9a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558df2b7ca00 0x558df2b79d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55682/0 (socket says 172.21.15.12:55682) 2024-04-22T13:25:17.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.360+0000 7f47b8e9a640 1 -- 172.21.15.12:0/2039305855 learned_addr learned my addr 172.21.15.12:0/2039305855 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:17.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.360+0000 7f47b37fe640 1 --2- 172.21.15.12:0/2039305855 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558df2b73e80 0x558df2b79720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.360+0000 7f47b8e9a640 1 -- 172.21.15.12:0/2039305855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558df2b734d0 msgr2=0x558df2b7c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.360+0000 7f47b8e9a640 1 --2- 172.21.15.12:0/2039305855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558df2b734d0 0x558df2b7c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.360+0000 7f47b8e9a640 1 -- 172.21.15.12:0/2039305855 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558df2b73e80 msgr2=0x558df2b79720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.360+0000 7f47b8e9a640 1 --2- 172.21.15.12:0/2039305855 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558df2b73e80 0x558df2b79720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.360+0000 7f47b8e9a640 1 -- 172.21.15.12:0/2039305855 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558df27ea3f0 con 0x558df2b7ca00 2024-04-22T13:25:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.361+0000 7f47b8e9a640 1 --2- 172.21.15.12:0/2039305855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558df2b7ca00 0x558df2b79d30 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f47ac037040 tx=0x7f47ac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c09413cb70f1efb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.361+0000 7f47937fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.361+0000 7f47937fe640 1 -- 172.21.15.12:0/2039305855 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac04c020 con 0x558df2b7ca00 2024-04-22T13:25:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.361+0000 7f47937fe640 1 -- 172.21.15.12:0/2039305855 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f47ac0328c0 con 0x558df2b7ca00 2024-04-22T13:25:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.361+0000 7f47bd5c8740 1 -- 172.21.15.12:0/2039305855 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47ac003b70 con 0x558df2b7ca00 2024-04-22T13:25:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.361+0000 7f47937fe640 1 -- 172.21.15.12:0/2039305855 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f47ac057050 con 0x558df2b7ca00 2024-04-22T13:25:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.361+0000 7f47bd5c8740 10 client.16176 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.361+0000 7f47bd5c8740 1 -- 172.21.15.12:0/2039305855 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558df28b47d0 con 0x558df2b7ca00 2024-04-22T13:25:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.361+0000 7f47937fe640 1 -- 172.21.15.12:0/2039305855 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47ac003b70 con 0x558df2b7ca00 2024-04-22T13:25:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.362+0000 7f47937fe640 1 -- 172.21.15.12:0/2039305855 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f47ac03f860 con 0x558df2b7ca00 2024-04-22T13:25:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.362+0000 7f47bd5c8740 10 client.16176 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.362+0000 7f47bd5c8740 10 client.16176 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:25:17.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.362+0000 7f47bd5c8740 20 client.16176 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:17.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.362+0000 7f47bd5c8740 1 --2- 172.21.15.12:0/2039305855 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x558df2b8caf0 0x558df2b8eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.362+0000 7f47bd5c8740 4 client.16176 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:25:17.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.362+0000 7f47bd5c8740 1 -- 172.21.15.12:0/2039305855 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558df2a77930 con 0x558df2b8caf0 2024-04-22T13:25:17.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.362+0000 7f47b3fff640 1 --2- 172.21.15.12:0/2039305855 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x558df2b8caf0 0x558df2b8eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.363+0000 7f47b3fff640 1 --2- 172.21.15.12:0/2039305855 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x558df2b8caf0 0x558df2b8eee0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=397e310431903713 server_cookie=c2cb7b7def064313 in_seq=0 out_seq=0 2024-04-22T13:25:17.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.363+0000 7f47937fe640 10 client.16176 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:25:17.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.363+0000 7f47937fe640 1 -- 172.21.15.12:0/2039305855 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558df2a77930 con 0x558df2b8caf0 2024-04-22T13:25:17.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.363+0000 7f47937fe640 10 client.16176 handle_command_reply: tid=0 2024-04-22T13:25:17.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.363+0000 7f47bd5c8740 1 client.16176 shutdown 2024-04-22T13:25:17.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.364+0000 7f47bd5c8740 1 -- 172.21.15.12:0/2039305855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558df2b7ca00 msgr2=0x558df2b79d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.364+0000 7f47bd5c8740 1 --2- 172.21.15.12:0/2039305855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558df2b7ca00 0x558df2b79d30 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f47ac037040 tx=0x7f47ac0042f0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.364+0000 7f47bd5c8740 1 -- 172.21.15.12:0/2039305855 shutdown_connections 2024-04-22T13:25:17.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.364+0000 7f47bd5c8740 1 --2- 172.21.15.12:0/2039305855 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x558df2b8caf0 0x558df2b8eee0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.364+0000 7f47bd5c8740 1 --2- 172.21.15.12:0/2039305855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558df2b7ca00 0x558df2b79d30 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.364+0000 7f47bd5c8740 1 --2- 172.21.15.12:0/2039305855 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558df2b73e80 0x558df2b79720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.364+0000 7f47937fe640 0 client.16176 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:25:17.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.364+0000 7f47bd5c8740 1 --2- 172.21.15.12:0/2039305855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558df2b734d0 0x558df2b7c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.364+0000 7f47bd5c8740 1 -- 172.21.15.12:0/2039305855 >> 172.21.15.12:0/2039305855 conn(0x558df2b6dba0 msgr2=0x558df2b6f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.364+0000 7f47bd5c8740 1 -- 172.21.15.12:0/2039305855 shutdown_connections 2024-04-22T13:25:17.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.364+0000 7f47bd5c8740 1 -- 172.21.15.12:0/2039305855 wait complete. 2024-04-22T13:25:17.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.364+0000 7f47bd5c8740 20 client.16176 trim_cache size 0 max 16384 2024-04-22T13:25:17.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.382+0000 7f47bd5c8740 1 Processor -- start 2024-04-22T13:25:17.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.382+0000 7f47bd5c8740 1 -- start start 2024-04-22T13:25:17.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.383+0000 7f47bd5c8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558df2b734d0 0x558df2b6ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.383+0000 7f47bd5c8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558df2b73e80 0x558df2b7be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.383+0000 7f47bd5c8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558df2b7ca00 0x558df2b7c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.383+0000 7f47bd5c8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41afca0 con 0x558df2b734d0 2024-04-22T13:25:17.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.383+0000 7f47bd5c8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41b17e0 con 0x558df2b7ca00 2024-04-22T13:25:17.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.383+0000 7f47bd5c8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41b0370 con 0x558df2b73e80 2024-04-22T13:25:17.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.383+0000 7f47b3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558df2b734d0 0x558df2b6ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.383+0000 7f47b3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558df2b734d0 0x558df2b6ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55688/0 (socket says 172.21.15.12:55688) 2024-04-22T13:25:17.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.383+0000 7f47b3fff640 1 -- 172.21.15.12:0/2384312599 learned_addr learned my addr 172.21.15.12:0/2384312599 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:17.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.383+0000 7f47b8e9a640 1 --2- 172.21.15.12:0/2384312599 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558df2b7ca00 0x558df2b7c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.383+0000 7f47b37fe640 1 --2- 172.21.15.12:0/2384312599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558df2b73e80 0x558df2b7be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.383+0000 7f47b3fff640 1 -- 172.21.15.12:0/2384312599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558df2b73e80 msgr2=0x558df2b7be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.383+0000 7f47b3fff640 1 --2- 172.21.15.12:0/2384312599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558df2b73e80 0x558df2b7be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.383+0000 7f47b3fff640 1 -- 172.21.15.12:0/2384312599 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558df2b7ca00 msgr2=0x558df2b7c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.383+0000 7f47b3fff640 1 --2- 172.21.15.12:0/2384312599 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558df2b7ca00 0x558df2b7c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.383+0000 7f47b3fff640 1 -- 172.21.15.12:0/2384312599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558df2b6e4e0 con 0x558df2b734d0 2024-04-22T13:25:17.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.383+0000 7f47b8e9a640 1 --2- 172.21.15.12:0/2384312599 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558df2b7ca00 0x558df2b7c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:17.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.384+0000 7f47b3fff640 1 --2- 172.21.15.12:0/2384312599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558df2b734d0 0x558df2b6ce40 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x558df2b6ec10 tx=0x7f47a4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0caa2f00c1fd48 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.384+0000 7f4799ffb640 1 -- 172.21.15.12:0/2384312599 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a4011c70 con 0x558df2b734d0 2024-04-22T13:25:17.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.384+0000 7f4799ffb640 1 -- 172.21.15.12:0/2384312599 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47a402c070 con 0x558df2b734d0 2024-04-22T13:25:17.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.384+0000 7f4799ffb640 1 -- 172.21.15.12:0/2384312599 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a4027520 con 0x558df2b734d0 2024-04-22T13:25:17.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.384+0000 7f47bd5c8740 1 -- 172.21.15.12:0/2384312599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558df2b734d0 msgr2=0x558df2b6ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.384+0000 7f47bd5c8740 1 --2- 172.21.15.12:0/2384312599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558df2b734d0 0x558df2b6ce40 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x558df2b6ec10 tx=0x7f47a4016000 comp rx=0 tx=0).stop 2024-04-22T13:25:17.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.385+0000 7f47bd5c8740 1 -- 172.21.15.12:0/2384312599 shutdown_connections 2024-04-22T13:25:17.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.385+0000 7f47bd5c8740 1 --2- 172.21.15.12:0/2384312599 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558df2b7ca00 0x558df2b7c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.385+0000 7f47bd5c8740 1 --2- 172.21.15.12:0/2384312599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558df2b73e80 0x558df2b7be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.385+0000 7f47bd5c8740 1 --2- 172.21.15.12:0/2384312599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558df2b734d0 0x558df2b6ce40 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.385+0000 7f47bd5c8740 1 -- 172.21.15.12:0/2384312599 >> 172.21.15.12:0/2384312599 conn(0x558df2b6dba0 msgr2=0x558df2b74c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.385+0000 7f47bd5c8740 1 -- 172.21.15.12:0/2384312599 shutdown_connections 2024-04-22T13:25:17.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.385+0000 7f47bd5c8740 1 -- 172.21.15.12:0/2384312599 wait complete. 2024-04-22T13:25:17.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.385+0000 7f47bd5c8740 1 -- start start 2024-04-22T13:25:17.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.385+0000 7f47bd5c8740 1 Processor -- start 2024-04-22T13:25:17.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.385+0000 7f47bd5c8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558df2b734d0 0x558df2b8fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.385+0000 7f47bd5c8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558df2b73e80 0x558df2b90240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.386+0000 7f47bd5c8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558df2b7ca00 0x558df2bf1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.386+0000 7f47bd5c8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41b1110 con 0x558df2b734d0 2024-04-22T13:25:17.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.386+0000 7f47bd5c8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41b0a40 con 0x558df2b73e80 2024-04-22T13:25:17.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.386+0000 7f47bd5c8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41adc80 con 0x558df2b7ca00 2024-04-22T13:25:17.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.386+0000 7f47bd5c8740 10 client.0 _ll_register_callbacks cb 0x7f47ac004190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:17.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.386+0000 7f47b3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558df2b734d0 0x558df2b8fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.386+0000 7f47b3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558df2b734d0 0x558df2b8fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55696/0 (socket says 172.21.15.12:55696) 2024-04-22T13:25:17.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.386+0000 7f47b3fff640 1 -- 172.21.15.12:0/3231427467 learned_addr learned my addr 172.21.15.12:0/3231427467 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:17.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.386+0000 7f47b37fe640 1 --2- 172.21.15.12:0/3231427467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558df2b73e80 0x558df2b90240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.386+0000 7f47b8e9a640 1 --2- 172.21.15.12:0/3231427467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558df2b7ca00 0x558df2bf1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.386+0000 7f47b37fe640 1 -- 172.21.15.12:0/3231427467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558df2b7ca00 msgr2=0x558df2bf1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.386+0000 7f47b37fe640 1 --2- 172.21.15.12:0/3231427467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558df2b7ca00 0x558df2bf1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.386+0000 7f47b37fe640 1 -- 172.21.15.12:0/3231427467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558df2b734d0 msgr2=0x558df2b8fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.386+0000 7f47b37fe640 1 --2- 172.21.15.12:0/3231427467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558df2b734d0 0x558df2b8fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.386+0000 7f47b37fe640 1 -- 172.21.15.12:0/3231427467 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558df2bf2210 con 0x558df2b73e80 2024-04-22T13:25:17.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.386+0000 7f47b3fff640 1 --2- 172.21.15.12:0/3231427467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558df2b734d0 0x558df2b8fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:17.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.386+0000 7f47b37fe640 1 --2- 172.21.15.12:0/3231427467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558df2b73e80 0x558df2b90240 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f47a80cda00 tx=0x7f47a809bc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be988d3c286199af server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.386+0000 7f47937fe640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-22T13:25:17.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.387+0000 7f47937fe640 1 -- 172.21.15.12:0/3231427467 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a80b5060 con 0x558df2b73e80 2024-04-22T13:25:17.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.387+0000 7f47bd5c8740 1 -- 172.21.15.12:0/3231427467 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47a402c070 con 0x558df2b73e80 2024-04-22T13:25:17.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.387+0000 7f47937fe640 1 -- 172.21.15.12:0/3231427467 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f47a80d3c60 con 0x558df2b73e80 2024-04-22T13:25:17.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.387+0000 7f47937fe640 1 -- 172.21.15.12:0/3231427467 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f47a8020040 con 0x558df2b73e80 2024-04-22T13:25:17.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.387+0000 7f47bd5c8740 10 client.25619 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:17.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.387+0000 7f47bd5c8740 1 -- 172.21.15.12:0/3231427467 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558df2bf3420 con 0x558df2b73e80 2024-04-22T13:25:17.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.387+0000 7f47937fe640 1 -- 172.21.15.12:0/3231427467 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47a402c070 con 0x558df2b73e80 2024-04-22T13:25:17.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.387+0000 7f47937fe640 1 -- 172.21.15.12:0/3231427467 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f47a80a0170 con 0x558df2b73e80 2024-04-22T13:25:17.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.387+0000 7f47bd5c8740 10 client.25619 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:17.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.387+0000 7f47bd5c8740 10 client.25619 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:25:17.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.387+0000 7f47bd5c8740 20 client.25619 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:17.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.387+0000 7f47bd5c8740 1 --2- 172.21.15.12:0/3231427467 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x558df2bf3b50 0x558df2bf5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.387+0000 7f47bd5c8740 4 client.25619 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:25:17.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.387+0000 7f47bd5c8740 1 -- 172.21.15.12:0/3231427467 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558df2ae5c10 con 0x558df2bf3b50 2024-04-22T13:25:17.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.388+0000 7f47b3fff640 1 --2- 172.21.15.12:0/3231427467 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x558df2bf3b50 0x558df2bf5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.388+0000 7f47937fe640 10 client.25619 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:25:17.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.388+0000 7f47b3fff640 1 --2- 172.21.15.12:0/3231427467 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x558df2bf3b50 0x558df2bf5f70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f1ea6f6be4cc4380 server_cookie=3be9183998c76f85 in_seq=0 out_seq=0 2024-04-22T13:25:17.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.388+0000 7f47937fe640 1 -- 172.21.15.12:0/3231427467 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558df2ae5c10 con 0x558df2bf3b50 2024-04-22T13:25:17.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.388+0000 7f47937fe640 10 client.25619 handle_command_reply: tid=0 2024-04-22T13:25:17.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.389+0000 7f47bd5c8740 1 client.25619 shutdown 2024-04-22T13:25:17.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.389+0000 7f47bd5c8740 1 -- 172.21.15.12:0/3231427467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558df2b73e80 msgr2=0x558df2b90240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.389+0000 7f47bd5c8740 1 --2- 172.21.15.12:0/3231427467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558df2b73e80 0x558df2b90240 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f47a80cda00 tx=0x7f47a809bc70 comp rx=0 tx=0).stop 2024-04-22T13:25:17.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.389+0000 7f47bd5c8740 1 -- 172.21.15.12:0/3231427467 shutdown_connections 2024-04-22T13:25:17.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.389+0000 7f47bd5c8740 1 --2- 172.21.15.12:0/3231427467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558df2b7ca00 0x558df2bf1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.389+0000 7f47bd5c8740 1 --2- 172.21.15.12:0/3231427467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558df2b73e80 0x558df2b90240 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.389+0000 7f47bd5c8740 1 --2- 172.21.15.12:0/3231427467 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x558df2bf3b50 0x558df2bf5f70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.389+0000 7f47bd5c8740 1 --2- 172.21.15.12:0/3231427467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558df2b734d0 0x558df2b8fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.389+0000 7f47bd5c8740 1 -- 172.21.15.12:0/3231427467 >> 172.21.15.12:0/3231427467 conn(0x558df2b6dba0 msgr2=0x558df2b742b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.389+0000 7f47937fe640 0 client.25619 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:25:17.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.389+0000 7f47bd5c8740 1 -- 172.21.15.12:0/3231427467 shutdown_connections 2024-04-22T13:25:17.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.389+0000 7f47bd5c8740 1 -- 172.21.15.12:0/3231427467 wait complete. 2024-04-22T13:25:17.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.389+0000 7f47bd5c8740 20 client.25619 trim_cache size 0 max 16384 2024-04-22T13:25:17.390 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:25:17.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.390+0000 7f47ba8fd640 1 -- 172.21.15.12:0/2929671467 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f479c0a3f80 msgr2=0x7f479c0a6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.390+0000 7f47ba8fd640 1 --2- 172.21.15.12:0/2929671467 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f479c0a3f80 0x7f479c0a6440 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f47a4005fd0 tx=0x7f47a4005f40 comp rx=0 tx=0).stop 2024-04-22T13:25:17.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.390+0000 7f47ba8fd640 1 -- 172.21.15.12:0/2929671467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b4103af0 msgr2=0x7f47b4101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.390+0000 7f47ba8fd640 1 --2- 172.21.15.12:0/2929671467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b4103af0 0x7f47b4101360 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f47a800d770 tx=0x7f47a800dc30 comp rx=0 tx=0).stop 2024-04-22T13:25:17.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.390+0000 7f47ba8fd640 1 -- 172.21.15.12:0/2929671467 shutdown_connections 2024-04-22T13:25:17.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.390+0000 7f47ba8fd640 1 --2- 172.21.15.12:0/2929671467 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f479c0a3f80 0x7f479c0a6440 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.390+0000 7f47ba8fd640 1 --2- 172.21.15.12:0/2929671467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47b410c510 0x7f47b41018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.390+0000 7f47ba8fd640 1 --2- 172.21.15.12:0/2929671467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47b4103af0 0x7f47b4101360 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.390+0000 7f47ba8fd640 1 --2- 172.21.15.12:0/2929671467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47b41031d0 0x7f47b4100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.390+0000 7f47ba8fd640 1 -- 172.21.15.12:0/2929671467 >> 172.21.15.12:0/2929671467 conn(0x7f47b4077f50 msgr2=0x7f47b40fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.390+0000 7f47ba8fd640 1 -- 172.21.15.12:0/2929671467 shutdown_connections 2024-04-22T13:25:17.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.390+0000 7f47ba8fd640 1 -- 172.21.15.12:0/2929671467 wait complete. 2024-04-22T13:25:17.401 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:25:17.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.537+0000 7faeaa4fd640 1 -- 172.21.15.12:0/1876797702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faea4102bc0 msgr2=0x7faea4106c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.537+0000 7faeaa4fd640 1 --2- 172.21.15.12:0/1876797702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faea4102bc0 0x7faea4106c00 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fae98009900 tx=0x7fae9802bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:17.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.537+0000 7faeaa4fd640 1 -- 172.21.15.12:0/1876797702 shutdown_connections 2024-04-22T13:25:17.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.537+0000 7faeaa4fd640 1 --2- 172.21.15.12:0/1876797702 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faea41072e0 0x7faea410eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.537+0000 7faeaa4fd640 1 --2- 172.21.15.12:0/1876797702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faea4102bc0 0x7faea4106c00 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.537+0000 7faeaa4fd640 1 --2- 172.21.15.12:0/1876797702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faea4102210 0x7faea41025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.537+0000 7faeaa4fd640 1 -- 172.21.15.12:0/1876797702 >> 172.21.15.12:0/1876797702 conn(0x7faea40fdab0 msgr2=0x7faea40ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.537+0000 7faeaa4fd640 1 -- 172.21.15.12:0/1876797702 shutdown_connections 2024-04-22T13:25:17.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.537+0000 7faeaa4fd640 1 -- 172.21.15.12:0/1876797702 wait complete. 2024-04-22T13:25:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.538+0000 7faeaa4fd640 1 Processor -- start 2024-04-22T13:25:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.538+0000 7faeaa4fd640 1 -- start start 2024-04-22T13:25:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.539+0000 7faeaa4fd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faea4102210 0x7faea419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.539+0000 7faeaa4fd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faea41072e0 0x7faea419ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.539+0000 7faeaa4fd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faea41988d0 0x7faea4198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.539+0000 7faeaa4fd640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7faea4112220 con 0x7faea4102210 2024-04-22T13:25:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.539+0000 7faeaa4fd640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faea41120a0 con 0x7faea41988d0 2024-04-22T13:25:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.539+0000 7faeaa4fd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faea41123a0 con 0x7faea41072e0 2024-04-22T13:25:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.539+0000 7faea3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faea4102210 0x7faea419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.539+0000 7faea8a9a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faea41988d0 0x7faea4198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.539+0000 7faea37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faea41072e0 0x7faea419ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.539+0000 7faea8a9a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faea41988d0 0x7faea4198d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:44796/0 (socket says 172.21.15.12:44796) 2024-04-22T13:25:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.539+0000 7faea8a9a640 1 -- 172.21.15.12:0/2153817464 learned_addr learned my addr 172.21.15.12:0/2153817464 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:17.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.539+0000 7faea37fe640 1 -- 172.21.15.12:0/2153817464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faea41988d0 msgr2=0x7faea4198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.539+0000 7faea37fe640 1 --2- 172.21.15.12:0/2153817464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faea41988d0 0x7faea4198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.539+0000 7faea37fe640 1 -- 172.21.15.12:0/2153817464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faea4102210 msgr2=0x7faea419e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.539+0000 7faea37fe640 1 --2- 172.21.15.12:0/2153817464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faea4102210 0x7faea419e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.539+0000 7faea37fe640 1 -- 172.21.15.12:0/2153817464 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea41993f0 con 0x7faea41072e0 2024-04-22T13:25:17.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.539+0000 7faea8a9a640 1 --2- 172.21.15.12:0/2153817464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faea41988d0 0x7faea4198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:17.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.539+0000 7faea3fff640 1 --2- 172.21.15.12:0/2153817464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faea4102210 0x7faea419e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:17.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.540+0000 7faea37fe640 1 --2- 172.21.15.12:0/2153817464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faea41072e0 0x7faea419ec10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faea4103110 tx=0x7fae980045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.540+0000 7faea17fa640 1 -- 172.21.15.12:0/2153817464 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98043020 con 0x7faea41072e0 2024-04-22T13:25:17.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.540+0000 7faeaa4fd640 1 -- 172.21.15.12:0/2153817464 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faea4199680 con 0x7faea41072e0 2024-04-22T13:25:17.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.540+0000 7faea17fa640 1 -- 172.21.15.12:0/2153817464 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae9802c9f0 con 0x7faea41072e0 2024-04-22T13:25:17.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.540+0000 7faeaa4fd640 1 -- 172.21.15.12:0/2153817464 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faea41a5880 con 0x7faea41072e0 2024-04-22T13:25:17.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.540+0000 7faea17fa640 1 -- 172.21.15.12:0/2153817464 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae980038e0 con 0x7faea41072e0 2024-04-22T13:25:17.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.543+0000 7faea17fa640 1 -- 172.21.15.12:0/2153817464 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fae98046020 con 0x7faea41072e0 2024-04-22T13:25:17.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.543+0000 7faea17fa640 1 --2- 172.21.15.12:0/2153817464 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fae8c0a3f50 0x7fae8c0a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.543+0000 7faead1c8740 1 Processor -- start 2024-04-22T13:25:17.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.544+0000 7faead1c8740 1 -- start start 2024-04-22T13:25:17.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.544+0000 7faea17fa640 1 -- 172.21.15.12:0/2153817464 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fae980f2680 con 0x7faea41072e0 2024-04-22T13:25:17.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.544+0000 7faea3fff640 1 --2- 172.21.15.12:0/2153817464 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fae8c0a3f50 0x7fae8c0a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.544+0000 7faead1c8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b44bdc2330 0x55b44bdc2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.544+0000 7faead1c8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b44bdc2ce0 0x55b44bdcb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.544+0000 7faead1c8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b44bdcbe40 0x55b44bdcc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.544+0000 7faead1c8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b44bd271f0 con 0x55b44bdc2ce0 2024-04-22T13:25:17.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.544+0000 7faead1c8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b44bcc44b0 con 0x55b44bdcbe40 2024-04-22T13:25:17.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.544+0000 7faead1c8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b44bd231b0 con 0x55b44bdc2330 2024-04-22T13:25:17.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.544+0000 7faea8a9a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b44bdcbe40 0x55b44bdcc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.544+0000 7faea37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b44bdc2ce0 0x55b44bdcb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.544+0000 7faea8a9a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b44bdcbe40 0x55b44bdcc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:44802/0 (socket says 172.21.15.12:44802) 2024-04-22T13:25:17.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.544+0000 7faea8a9a640 1 -- 172.21.15.12:0/1279500073 learned_addr learned my addr 172.21.15.12:0/1279500073 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:17.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.544+0000 7faea3fff640 1 --2- 172.21.15.12:0/1279500073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b44bdc2330 0x55b44bdc2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.544+0000 7faea3fff640 1 --2- 172.21.15.12:0/2153817464 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fae8c0a3f50 0x7fae8c0a6410 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faea40ffda0 tx=0x7fae94011810 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.544+0000 7faea8a9a640 1 -- 172.21.15.12:0/1279500073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b44bdc2330 msgr2=0x55b44bdc2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.545+0000 7faea8a9a640 1 --2- 172.21.15.12:0/1279500073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b44bdc2330 0x55b44bdc2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.545+0000 7faea8a9a640 1 -- 172.21.15.12:0/1279500073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b44bdc2ce0 msgr2=0x55b44bdcb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.545+0000 7faea8a9a640 1 --2- 172.21.15.12:0/1279500073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b44bdc2ce0 0x55b44bdcb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.545+0000 7faea8a9a640 1 -- 172.21.15.12:0/1279500073 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b44bcb61b0 con 0x55b44bdcbe40 2024-04-22T13:25:17.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.545+0000 7faea8a9a640 1 --2- 172.21.15.12:0/1279500073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b44bdcbe40 0x55b44bdcc2c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fae9c011850 tx=0x7fae9c011d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5961973c469b0f45 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.545+0000 7fae8a7fc640 1 -- 172.21.15.12:0/1279500073 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c011330 con 0x55b44bdcbe40 2024-04-22T13:25:17.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.545+0000 7fae8a7fc640 1 -- 172.21.15.12:0/1279500073 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae9c00fe50 con 0x55b44bdcbe40 2024-04-22T13:25:17.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.545+0000 7fae8a7fc640 1 -- 172.21.15.12:0/1279500073 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c0193e0 con 0x55b44bdcbe40 2024-04-22T13:25:17.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.546+0000 7faead1c8740 1 -- 172.21.15.12:0/1279500073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b44bdcbe40 msgr2=0x55b44bdcc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.546+0000 7faead1c8740 1 --2- 172.21.15.12:0/1279500073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b44bdcbe40 0x55b44bdcc2c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fae9c011850 tx=0x7fae9c011d10 comp rx=0 tx=0).stop 2024-04-22T13:25:17.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.546+0000 7faead1c8740 1 -- 172.21.15.12:0/1279500073 shutdown_connections 2024-04-22T13:25:17.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.546+0000 7faead1c8740 1 --2- 172.21.15.12:0/1279500073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b44bdcbe40 0x55b44bdcc2c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.546+0000 7faead1c8740 1 --2- 172.21.15.12:0/1279500073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b44bdc2ce0 0x55b44bdcb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.546+0000 7faead1c8740 1 --2- 172.21.15.12:0/1279500073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b44bdc2330 0x55b44bdc2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.546+0000 7faead1c8740 1 -- 172.21.15.12:0/1279500073 >> 172.21.15.12:0/1279500073 conn(0x55b44bdbcba0 msgr2=0x55b44bdbcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.546+0000 7faead1c8740 1 -- 172.21.15.12:0/1279500073 shutdown_connections 2024-04-22T13:25:17.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.546+0000 7faead1c8740 1 -- 172.21.15.12:0/1279500073 wait complete. 2024-04-22T13:25:17.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.546+0000 7faead1c8740 1 -- start start 2024-04-22T13:25:17.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.547+0000 7faead1c8740 1 Processor -- start 2024-04-22T13:25:17.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.547+0000 7faead1c8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b44bdc2330 0x55b44bdd20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.547+0000 7faead1c8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b44bdc2ce0 0x55b44bdcf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.547+0000 7faead1c8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b44bdcbe40 0x55b44bdcf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.547+0000 7faead1c8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b44bdd2dc0 con 0x55b44bdc2330 2024-04-22T13:25:17.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.547+0000 7faead1c8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b44bce0500 con 0x55b44bdc2ce0 2024-04-22T13:25:17.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.547+0000 7faead1c8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b44bdd2f40 con 0x55b44bdcbe40 2024-04-22T13:25:17.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.547+0000 7faead1c8740 10 client.0 _ll_register_callbacks cb 0x55b44bcaeb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:17.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.547+0000 7faea3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b44bdc2330 0x55b44bdd20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.547+0000 7faea3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b44bdc2330 0x55b44bdd20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55724/0 (socket says 172.21.15.12:55724) 2024-04-22T13:25:17.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.547+0000 7faea3fff640 1 -- 172.21.15.12:0/1376359997 learned_addr learned my addr 172.21.15.12:0/1376359997 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:17.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.547+0000 7faea37fe640 1 --2- 172.21.15.12:0/1376359997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b44bdc2ce0 0x55b44bdcf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.547+0000 7faea8a9a640 1 --2- 172.21.15.12:0/1376359997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b44bdcbe40 0x55b44bdcf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.547+0000 7faea3fff640 1 -- 172.21.15.12:0/1376359997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b44bdcbe40 msgr2=0x55b44bdcf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.548+0000 7faea3fff640 1 --2- 172.21.15.12:0/1376359997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b44bdcbe40 0x55b44bdcf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.548+0000 7faea3fff640 1 -- 172.21.15.12:0/1376359997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b44bdc2ce0 msgr2=0x55b44bdcf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.548+0000 7faea3fff640 1 --2- 172.21.15.12:0/1376359997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b44bdc2ce0 0x55b44bdcf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.548+0000 7faea3fff640 1 -- 172.21.15.12:0/1376359997 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b44ba393f0 con 0x55b44bdc2330 2024-04-22T13:25:17.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.548+0000 7faea8a9a640 1 --2- 172.21.15.12:0/1376359997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b44bdcbe40 0x55b44bdcf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:17.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.548+0000 7faea37fe640 1 --2- 172.21.15.12:0/1376359997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b44bdc2ce0 0x55b44bdcf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:17.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.548+0000 7faea3fff640 1 --2- 172.21.15.12:0/1376359997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b44bdc2330 0x55b44bdd20a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fae940037e0 tx=0x7fae94016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b987f9843746b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.548+0000 7fae837fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:17.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.548+0000 7fae837fe640 1 -- 172.21.15.12:0/1376359997 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae94017040 con 0x55b44bdc2330 2024-04-22T13:25:17.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.548+0000 7fae837fe640 1 -- 172.21.15.12:0/1376359997 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fae9400f690 con 0x55b44bdc2330 2024-04-22T13:25:17.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.548+0000 7faead1c8740 1 -- 172.21.15.12:0/1376359997 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae9c00fe50 con 0x55b44bdc2330 2024-04-22T13:25:17.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.548+0000 7fae837fe640 1 -- 172.21.15.12:0/1376359997 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fae9400f890 con 0x55b44bdc2330 2024-04-22T13:25:17.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.548+0000 7faead1c8740 10 client.16194 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:17.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.548+0000 7faead1c8740 1 -- 172.21.15.12:0/1376359997 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b44bb037d0 con 0x55b44bdc2330 2024-04-22T13:25:17.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.549+0000 7fae837fe640 1 -- 172.21.15.12:0/1376359997 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae9c00fe50 con 0x55b44bdc2330 2024-04-22T13:25:17.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.549+0000 7fae837fe640 1 -- 172.21.15.12:0/1376359997 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fae94030b80 con 0x55b44bdc2330 2024-04-22T13:25:17.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.549+0000 7faead1c8740 10 client.16194 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:17.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.549+0000 7faead1c8740 10 client.16194 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:25:17.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.549+0000 7faead1c8740 20 client.16194 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:17.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.549+0000 7faead1c8740 1 --2- 172.21.15.12:0/1376359997 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55b44bddff10 0x55b44bde23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.549+0000 7faead1c8740 4 client.16194 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:25:17.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.549+0000 7faead1c8740 1 -- 172.21.15.12:0/1376359997 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b44bcc6930 con 0x55b44bddff10 2024-04-22T13:25:17.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.549+0000 7faea37fe640 1 --2- 172.21.15.12:0/1376359997 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55b44bddff10 0x55b44bde23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.550+0000 7faea37fe640 1 --2- 172.21.15.12:0/1376359997 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55b44bddff10 0x55b44bde23b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=75a531ed8a4eb812 server_cookie=9066aedac92afbf8 in_seq=0 out_seq=0 2024-04-22T13:25:17.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.550+0000 7fae837fe640 10 client.16194 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:25:17.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.550+0000 7fae837fe640 1 -- 172.21.15.12:0/1376359997 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b44bcc6930 con 0x55b44bddff10 2024-04-22T13:25:17.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.550+0000 7fae837fe640 10 client.16194 handle_command_reply: tid=0 2024-04-22T13:25:17.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.550+0000 7faead1c8740 1 client.16194 shutdown 2024-04-22T13:25:17.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.551+0000 7faead1c8740 1 -- 172.21.15.12:0/1376359997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b44bdc2330 msgr2=0x55b44bdd20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.551+0000 7faead1c8740 1 --2- 172.21.15.12:0/1376359997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b44bdc2330 0x55b44bdd20a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fae940037e0 tx=0x7fae94016000 comp rx=0 tx=0).stop 2024-04-22T13:25:17.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.551+0000 7faead1c8740 1 -- 172.21.15.12:0/1376359997 shutdown_connections 2024-04-22T13:25:17.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.551+0000 7faead1c8740 1 --2- 172.21.15.12:0/1376359997 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55b44bddff10 0x55b44bde23b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.551+0000 7faead1c8740 1 --2- 172.21.15.12:0/1376359997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b44bdcbe40 0x55b44bdcf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.551+0000 7faead1c8740 1 --2- 172.21.15.12:0/1376359997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b44bdc2ce0 0x55b44bdcf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.551+0000 7faead1c8740 1 --2- 172.21.15.12:0/1376359997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b44bdc2330 0x55b44bdd20a0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.551+0000 7faead1c8740 1 -- 172.21.15.12:0/1376359997 >> 172.21.15.12:0/1376359997 conn(0x55b44bdbcba0 msgr2=0x55b44bdbe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.551+0000 7fae837fe640 0 client.16194 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:25:17.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.551+0000 7faead1c8740 1 -- 172.21.15.12:0/1376359997 shutdown_connections 2024-04-22T13:25:17.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.551+0000 7faead1c8740 1 -- 172.21.15.12:0/1376359997 wait complete. 2024-04-22T13:25:17.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.551+0000 7faead1c8740 20 client.16194 trim_cache size 0 max 16384 2024-04-22T13:25:17.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.572+0000 7faead1c8740 1 Processor -- start 2024-04-22T13:25:17.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.573+0000 7faead1c8740 1 -- start start 2024-04-22T13:25:17.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.573+0000 7faead1c8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b44bdc2330 0x55b44bddbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.573+0000 7faead1c8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b44bdc2ce0 0x55b44bdce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.573+0000 7faead1c8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b44bdcbe40 0x55b44bdbba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.573+0000 7faead1c8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fae9000e650 con 0x55b44bdc2330 2024-04-22T13:25:17.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.573+0000 7faead1c8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fae90010190 con 0x55b44bdcbe40 2024-04-22T13:25:17.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.573+0000 7faead1c8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fae9000ed20 con 0x55b44bdc2ce0 2024-04-22T13:25:17.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.573+0000 7faea3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b44bdc2330 0x55b44bddbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.573+0000 7faea3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b44bdc2330 0x55b44bddbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55728/0 (socket says 172.21.15.12:55728) 2024-04-22T13:25:17.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.573+0000 7faea3fff640 1 -- 172.21.15.12:0/3765707679 learned_addr learned my addr 172.21.15.12:0/3765707679 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:17.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.573+0000 7faea37fe640 1 --2- 172.21.15.12:0/3765707679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b44bdc2ce0 0x55b44bdce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.573+0000 7faea8a9a640 1 --2- 172.21.15.12:0/3765707679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b44bdcbe40 0x55b44bdbba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.573+0000 7faea3fff640 1 -- 172.21.15.12:0/3765707679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b44bdc2ce0 msgr2=0x55b44bdce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.573+0000 7faea3fff640 1 --2- 172.21.15.12:0/3765707679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b44bdc2ce0 0x55b44bdce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.573+0000 7faea3fff640 1 -- 172.21.15.12:0/3765707679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b44bdcbe40 msgr2=0x55b44bdbba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.573+0000 7faea3fff640 1 --2- 172.21.15.12:0/3765707679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b44bdcbe40 0x55b44bdbba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.573+0000 7faea3fff640 1 -- 172.21.15.12:0/3765707679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b44bdc1560 con 0x55b44bdc2330 2024-04-22T13:25:17.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.573+0000 7faea8a9a640 1 --2- 172.21.15.12:0/3765707679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b44bdcbe40 0x55b44bdbba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:17.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.574+0000 7faea3fff640 1 --2- 172.21.15.12:0/3765707679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b44bdc2330 0x55b44bddbcf0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fae94030f50 tx=0x7fae94000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d019987542c569 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.574+0000 7fae89ffb640 1 -- 172.21.15.12:0/3765707679 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae94017070 con 0x55b44bdc2330 2024-04-22T13:25:17.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.574+0000 7fae89ffb640 1 -- 172.21.15.12:0/3765707679 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae94019050 con 0x55b44bdc2330 2024-04-22T13:25:17.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.574+0000 7fae89ffb640 1 -- 172.21.15.12:0/3765707679 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae94015030 con 0x55b44bdc2330 2024-04-22T13:25:17.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.575+0000 7faead1c8740 1 -- 172.21.15.12:0/3765707679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b44bdc2330 msgr2=0x55b44bddbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.575+0000 7faead1c8740 1 --2- 172.21.15.12:0/3765707679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b44bdc2330 0x55b44bddbcf0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fae94030f50 tx=0x7fae94000bc0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.575+0000 7faead1c8740 1 -- 172.21.15.12:0/3765707679 shutdown_connections 2024-04-22T13:25:17.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.575+0000 7faead1c8740 1 --2- 172.21.15.12:0/3765707679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b44bdcbe40 0x55b44bdbba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.575+0000 7faead1c8740 1 --2- 172.21.15.12:0/3765707679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b44bdc2ce0 0x55b44bdce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.575+0000 7faead1c8740 1 --2- 172.21.15.12:0/3765707679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b44bdc2330 0x55b44bddbcf0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.575+0000 7faead1c8740 1 -- 172.21.15.12:0/3765707679 >> 172.21.15.12:0/3765707679 conn(0x55b44bdbcba0 msgr2=0x55b44bdbb370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.575+0000 7faead1c8740 1 -- 172.21.15.12:0/3765707679 shutdown_connections 2024-04-22T13:25:17.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.575+0000 7faead1c8740 1 -- 172.21.15.12:0/3765707679 wait complete. 2024-04-22T13:25:17.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.575+0000 7faead1c8740 1 -- start start 2024-04-22T13:25:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.575+0000 7faead1c8740 1 Processor -- start 2024-04-22T13:25:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.576+0000 7faead1c8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b44bdc2330 0x55b44bdd8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.576+0000 7faead1c8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b44bdc2ce0 0x55b44bdd0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.576+0000 7faead1c8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b44bdcbe40 0x55b44bdd1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.576+0000 7faead1c8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fae9000fac0 con 0x55b44bdc2ce0 2024-04-22T13:25:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.576+0000 7faead1c8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fae9000f3f0 con 0x55b44bdc2330 2024-04-22T13:25:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.576+0000 7faead1c8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fae9000c630 con 0x55b44bdcbe40 2024-04-22T13:25:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.576+0000 7faead1c8740 10 client.0 _ll_register_callbacks cb 0x7fae940035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.576+0000 7faea37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b44bdc2ce0 0x55b44bdd0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.576+0000 7faea37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b44bdc2ce0 0x55b44bdd0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55734/0 (socket says 172.21.15.12:55734) 2024-04-22T13:25:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.576+0000 7faea37fe640 1 -- 172.21.15.12:0/2133354508 learned_addr learned my addr 172.21.15.12:0/2133354508 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.576+0000 7faea3fff640 1 --2- 172.21.15.12:0/2133354508 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b44bdc2330 0x55b44bdd8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.576+0000 7faea8a9a640 1 --2- 172.21.15.12:0/2133354508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b44bdcbe40 0x55b44bdd1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.576+0000 7faea37fe640 1 -- 172.21.15.12:0/2133354508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b44bdcbe40 msgr2=0x55b44bdd1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.576+0000 7faea37fe640 1 --2- 172.21.15.12:0/2133354508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b44bdcbe40 0x55b44bdd1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.576+0000 7faea37fe640 1 -- 172.21.15.12:0/2133354508 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b44bdc2330 msgr2=0x55b44bdd8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.576+0000 7faea37fe640 1 --2- 172.21.15.12:0/2133354508 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b44bdc2330 0x55b44bdd8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.576+0000 7faea37fe640 1 -- 172.21.15.12:0/2133354508 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b44be45540 con 0x55b44bdc2ce0 2024-04-22T13:25:17.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.576+0000 7faea37fe640 1 --2- 172.21.15.12:0/2133354508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b44bdc2ce0 0x55b44bdd0ec0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fae980baec0 tx=0x7fae980be430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24367c6759695337 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.577+0000 7fae837fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:17.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.577+0000 7fae837fe640 1 -- 172.21.15.12:0/2133354508 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae980c71b0 con 0x55b44bdc2ce0 2024-04-22T13:25:17.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.577+0000 7fae837fe640 1 -- 172.21.15.12:0/2133354508 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fae980c6d20 con 0x55b44bdc2ce0 2024-04-22T13:25:17.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.577+0000 7faead1c8740 1 -- 172.21.15.12:0/2133354508 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae94019050 con 0x55b44bdc2ce0 2024-04-22T13:25:17.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.577+0000 7fae837fe640 1 -- 172.21.15.12:0/2133354508 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fae980c6f20 con 0x55b44bdc2ce0 2024-04-22T13:25:17.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.577+0000 7faead1c8740 10 client.16200 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:17.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.577+0000 7faead1c8740 1 -- 172.21.15.12:0/2133354508 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b44be46710 con 0x55b44bdc2ce0 2024-04-22T13:25:17.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.577+0000 7fae837fe640 1 -- 172.21.15.12:0/2133354508 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae94019050 con 0x55b44bdc2ce0 2024-04-22T13:25:17.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.577+0000 7fae837fe640 1 -- 172.21.15.12:0/2133354508 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fae980ba490 con 0x55b44bdc2ce0 2024-04-22T13:25:17.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.577+0000 7faead1c8740 10 client.16200 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:17.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.577+0000 7faead1c8740 10 client.16200 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:25:17.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.577+0000 7faead1c8740 20 client.16200 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:17.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.577+0000 7faead1c8740 1 --2- 172.21.15.12:0/2133354508 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55b44be46eb0 0x55b44be492d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.577+0000 7faead1c8740 4 client.16200 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:25:17.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.577+0000 7faead1c8740 1 -- 172.21.15.12:0/2133354508 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b44bd34c10 con 0x55b44be46eb0 2024-04-22T13:25:17.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.578+0000 7faea3fff640 1 --2- 172.21.15.12:0/2133354508 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55b44be46eb0 0x55b44be492d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.578+0000 7fae837fe640 10 client.16200 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:25:17.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.578+0000 7faea3fff640 1 --2- 172.21.15.12:0/2133354508 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55b44be46eb0 0x55b44be492d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f8f97a5fa1d2031 server_cookie=82b1718a94670323 in_seq=0 out_seq=0 2024-04-22T13:25:17.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.578+0000 7fae837fe640 1 -- 172.21.15.12:0/2133354508 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b44bd34c10 con 0x55b44be46eb0 2024-04-22T13:25:17.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.579+0000 7fae837fe640 10 client.16200 handle_command_reply: tid=0 2024-04-22T13:25:17.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.579+0000 7faead1c8740 1 client.16200 shutdown 2024-04-22T13:25:17.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.579+0000 7faead1c8740 1 -- 172.21.15.12:0/2133354508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b44bdc2ce0 msgr2=0x55b44bdd0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.579+0000 7faead1c8740 1 --2- 172.21.15.12:0/2133354508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b44bdc2ce0 0x55b44bdd0ec0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fae980baec0 tx=0x7fae980be430 comp rx=0 tx=0).stop 2024-04-22T13:25:17.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.579+0000 7faead1c8740 1 -- 172.21.15.12:0/2133354508 shutdown_connections 2024-04-22T13:25:17.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.579+0000 7faead1c8740 1 --2- 172.21.15.12:0/2133354508 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55b44be46eb0 0x55b44be492d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.579+0000 7faead1c8740 1 --2- 172.21.15.12:0/2133354508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b44bdcbe40 0x55b44bdd1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.579+0000 7fae837fe640 0 client.16200 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:25:17.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.579+0000 7faead1c8740 1 --2- 172.21.15.12:0/2133354508 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b44bdc2ce0 0x55b44bdd0ec0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.579+0000 7faead1c8740 1 --2- 172.21.15.12:0/2133354508 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b44bdc2330 0x55b44bdd8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.579+0000 7faead1c8740 1 -- 172.21.15.12:0/2133354508 >> 172.21.15.12:0/2133354508 conn(0x55b44bdbcba0 msgr2=0x55b44bdbb370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.579+0000 7faead1c8740 1 -- 172.21.15.12:0/2133354508 shutdown_connections 2024-04-22T13:25:17.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.579+0000 7faead1c8740 1 -- 172.21.15.12:0/2133354508 wait complete. 2024-04-22T13:25:17.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.579+0000 7faead1c8740 20 client.16200 trim_cache size 0 max 16384 2024-04-22T13:25:17.579 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:25:17.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.580+0000 7faeaa4fd640 1 -- 172.21.15.12:0/2153817464 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fae8c0a3f50 msgr2=0x7fae8c0a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.580+0000 7faeaa4fd640 1 --2- 172.21.15.12:0/2153817464 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fae8c0a3f50 0x7fae8c0a6410 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faea40ffda0 tx=0x7fae94011810 comp rx=0 tx=0).stop 2024-04-22T13:25:17.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.580+0000 7faeaa4fd640 1 -- 172.21.15.12:0/2153817464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faea41072e0 msgr2=0x7faea419ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.580+0000 7faeaa4fd640 1 --2- 172.21.15.12:0/2153817464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faea41072e0 0x7faea419ec10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faea4103110 tx=0x7fae980045e0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.580+0000 7faeaa4fd640 1 -- 172.21.15.12:0/2153817464 shutdown_connections 2024-04-22T13:25:17.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.580+0000 7faeaa4fd640 1 --2- 172.21.15.12:0/2153817464 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fae8c0a3f50 0x7fae8c0a6410 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.580+0000 7faeaa4fd640 1 --2- 172.21.15.12:0/2153817464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faea41988d0 0x7faea4198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.580+0000 7faeaa4fd640 1 --2- 172.21.15.12:0/2153817464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faea41072e0 0x7faea419ec10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.580+0000 7faeaa4fd640 1 --2- 172.21.15.12:0/2153817464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faea4102210 0x7faea419e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.580+0000 7faeaa4fd640 1 -- 172.21.15.12:0/2153817464 >> 172.21.15.12:0/2153817464 conn(0x7faea40fdab0 msgr2=0x7faea40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.580+0000 7faeaa4fd640 1 -- 172.21.15.12:0/2153817464 shutdown_connections 2024-04-22T13:25:17.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.580+0000 7faeaa4fd640 1 -- 172.21.15.12:0/2153817464 wait complete. 2024-04-22T13:25:17.588 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:25:17.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.700+0000 7fe7a0811640 1 -- 172.21.15.12:0/3264405480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe79c10e900 msgr2=0x7fe79c110cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.700+0000 7fe7a0811640 1 --2- 172.21.15.12:0/3264405480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe79c10e900 0x7fe79c110cf0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe794009ca0 tx=0x7fe79402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.700+0000 7fe7a0811640 1 -- 172.21.15.12:0/3264405480 shutdown_connections 2024-04-22T13:25:17.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.700+0000 7fe7a0811640 1 --2- 172.21.15.12:0/3264405480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe79c10e900 0x7fe79c110cf0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.700+0000 7fe7a0811640 1 --2- 172.21.15.12:0/3264405480 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe79c101670 0x7fe79c10e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.700+0000 7fe7a0811640 1 --2- 172.21.15.12:0/3264405480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe79c100d50 0x7fe79c101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.700+0000 7fe7a0811640 1 -- 172.21.15.12:0/3264405480 >> 172.21.15.12:0/3264405480 conn(0x7fe79c077f50 msgr2=0x7fe79c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.701+0000 7fe7a0811640 1 -- 172.21.15.12:0/3264405480 shutdown_connections 2024-04-22T13:25:17.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.701+0000 7fe7a0811640 1 -- 172.21.15.12:0/3264405480 wait complete. 2024-04-22T13:25:17.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.701+0000 7fe7a0811640 1 Processor -- start 2024-04-22T13:25:17.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.702+0000 7fe7a0811640 1 -- start start 2024-04-22T13:25:17.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.702+0000 7fe7a0811640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe79c100d50 0x7fe79c1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.702+0000 7fe7a0811640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe79c101670 0x7fe79c1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.702+0000 7fe7a0811640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe79c10e900 0x7fe79c19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.702+0000 7fe799d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe79c101670 0x7fe79c1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.702+0000 7fe7a0811640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c1143c0 con 0x7fe79c101670 2024-04-22T13:25:17.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.702+0000 7fe7a0811640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c114240 con 0x7fe79c100d50 2024-04-22T13:25:17.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.702+0000 7fe7a0811640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c114540 con 0x7fe79c10e900 2024-04-22T13:25:17.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.702+0000 7fe799d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe79c101670 0x7fe79c1a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55750/0 (socket says 172.21.15.12:55750) 2024-04-22T13:25:17.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.702+0000 7fe799d9b640 1 -- 172.21.15.12:0/861027013 learned_addr learned my addr 172.21.15.12:0/861027013 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:17.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.702+0000 7fe79a59c640 1 --2- 172.21.15.12:0/861027013 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe79c100d50 0x7fe79c1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.702+0000 7fe79ad9d640 1 --2- 172.21.15.12:0/861027013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe79c10e900 0x7fe79c19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.702+0000 7fe79a59c640 1 -- 172.21.15.12:0/861027013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe79c10e900 msgr2=0x7fe79c19a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.702+0000 7fe79a59c640 1 --2- 172.21.15.12:0/861027013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe79c10e900 0x7fe79c19a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.702+0000 7fe79a59c640 1 -- 172.21.15.12:0/861027013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe79c101670 msgr2=0x7fe79c1a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.702+0000 7fe79a59c640 1 --2- 172.21.15.12:0/861027013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe79c101670 0x7fe79c1a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.702+0000 7fe79a59c640 1 -- 172.21.15.12:0/861027013 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79c19b040 con 0x7fe79c100d50 2024-04-22T13:25:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.703+0000 7fe79a59c640 1 --2- 172.21.15.12:0/861027013 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe79c100d50 0x7fe79c1a0670 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00d720 tx=0x7fe78c00dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.703+0000 7fe78b7fe640 1 -- 172.21.15.12:0/861027013 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c017070 con 0x7fe79c100d50 2024-04-22T13:25:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.703+0000 7fe7a0811640 1 -- 172.21.15.12:0/861027013 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe79c19b330 con 0x7fe79c100d50 2024-04-22T13:25:17.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.703+0000 7fe78b7fe640 1 -- 172.21.15.12:0/861027013 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe78c00de30 con 0x7fe79c100d50 2024-04-22T13:25:17.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.703+0000 7fe78b7fe640 1 -- 172.21.15.12:0/861027013 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c0047a0 con 0x7fe79c100d50 2024-04-22T13:25:17.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.703+0000 7fe7a0811640 1 -- 172.21.15.12:0/861027013 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe79c1a75f0 con 0x7fe79c100d50 2024-04-22T13:25:17.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.707+0000 7fe78b7fe640 1 -- 172.21.15.12:0/861027013 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fe78c0099d0 con 0x7fe79c100d50 2024-04-22T13:25:17.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.707+0000 7fe7a3400740 1 Processor -- start 2024-04-22T13:25:17.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.707+0000 7fe7a3400740 1 -- start start 2024-04-22T13:25:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.708+0000 7fe78b7fe640 1 --2- 172.21.15.12:0/861027013 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe7840a3f50 0x7fe7840a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.708+0000 7fe7a3400740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1371494d0 0x55e1371498b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.708+0000 7fe79ad9d640 1 --2- 172.21.15.12:0/861027013 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe7840a3f50 0x7fe7840a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.708+0000 7fe78b7fe640 1 -- 172.21.15.12:0/861027013 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fe78c0d0710 con 0x7fe79c100d50 2024-04-22T13:25:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.708+0000 7fe7a3400740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e137149e80 0x55e137152430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.708+0000 7fe7a3400740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e137152a00 0x55e137154e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.708+0000 7fe7a3400740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e1370ae1f0 con 0x55e137152a00 2024-04-22T13:25:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.708+0000 7fe79a59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1371494d0 0x55e1371498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.708+0000 7fe79a59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1371494d0 0x55e1371498b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.12:54444/0 (socket says 172.21.15.12:54444) 2024-04-22T13:25:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.708+0000 7fe79a59c640 1 -- 172.21.15.12:0/1803380997 learned_addr learned my addr 172.21.15.12:0/1803380997 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.708+0000 7fe7a3400740 1 -- 172.21.15.12:0/1803380997 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e13704b4b0 con 0x55e137149e80 2024-04-22T13:25:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.708+0000 7fe79ad9d640 1 --2- 172.21.15.12:0/1803380997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e137152a00 0x55e137154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.708+0000 7fe7a3400740 1 -- 172.21.15.12:0/1803380997 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e1370aa1b0 con 0x55e1371494d0 2024-04-22T13:25:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.709+0000 7fe79ad9d640 1 --2- 172.21.15.12:0/861027013 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe7840a3f50 0x7fe7840a6410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe794006010 tx=0x7fe794005c50 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.709+0000 7fe799d9b640 1 --2- 172.21.15.12:0/1803380997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e137149e80 0x55e137152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.709+0000 7fe79ad9d640 1 -- 172.21.15.12:0/1803380997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1371494d0 msgr2=0x55e1371498b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.709+0000 7fe79ad9d640 1 --2- 172.21.15.12:0/1803380997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1371494d0 0x55e1371498b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.709+0000 7fe79ad9d640 1 -- 172.21.15.12:0/1803380997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e137149e80 msgr2=0x55e137152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.709+0000 7fe79ad9d640 1 --2- 172.21.15.12:0/1803380997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e137149e80 0x55e137152430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.709+0000 7fe79ad9d640 1 -- 172.21.15.12:0/1803380997 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e13703d1b0 con 0x55e137152a00 2024-04-22T13:25:17.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.709+0000 7fe799d9b640 1 --2- 172.21.15.12:0/1803380997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e137149e80 0x55e137152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:17.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.709+0000 7fe79ad9d640 1 --2- 172.21.15.12:0/1803380997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e137152a00 0x55e137154e50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fe79402c630 tx=0x7fe79403c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d459ae69462fd14e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.709+0000 7fe788ff9640 1 -- 172.21.15.12:0/1803380997 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79404f070 con 0x55e137152a00 2024-04-22T13:25:17.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.709+0000 7fe788ff9640 1 -- 172.21.15.12:0/1803380997 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe79403a040 con 0x55e137152a00 2024-04-22T13:25:17.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.709+0000 7fe788ff9640 1 -- 172.21.15.12:0/1803380997 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794046040 con 0x55e137152a00 2024-04-22T13:25:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.710+0000 7fe7a3400740 1 -- 172.21.15.12:0/1803380997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e137152a00 msgr2=0x55e137154e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.710+0000 7fe7a3400740 1 --2- 172.21.15.12:0/1803380997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e137152a00 0x55e137154e50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fe79402c630 tx=0x7fe79403c000 comp rx=0 tx=0).stop 2024-04-22T13:25:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.710+0000 7fe7a3400740 1 -- 172.21.15.12:0/1803380997 shutdown_connections 2024-04-22T13:25:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.710+0000 7fe7a3400740 1 --2- 172.21.15.12:0/1803380997 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e137152a00 0x55e137154e50 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.710+0000 7fe7a3400740 1 --2- 172.21.15.12:0/1803380997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e137149e80 0x55e137152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.710+0000 7fe7a3400740 1 --2- 172.21.15.12:0/1803380997 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e1371494d0 0x55e1371498b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.710+0000 7fe7a3400740 1 -- 172.21.15.12:0/1803380997 >> 172.21.15.12:0/1803380997 conn(0x55e137143ba0 msgr2=0x55e137143f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.710+0000 7fe7a3400740 1 -- 172.21.15.12:0/1803380997 shutdown_connections 2024-04-22T13:25:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.710+0000 7fe7a3400740 1 -- 172.21.15.12:0/1803380997 wait complete. 2024-04-22T13:25:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.710+0000 7fe7a3400740 1 -- start start 2024-04-22T13:25:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.710+0000 7fe7a3400740 1 Processor -- start 2024-04-22T13:25:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.711+0000 7fe7a3400740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1371494d0 0x55e1371522e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.711+0000 7fe7a3400740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e137149e80 0x55e13714f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.711+0000 7fe7a3400740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e137152a00 0x55e13714fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.711+0000 7fe7a3400740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e137155910 con 0x55e1371494d0 2024-04-22T13:25:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.711+0000 7fe7a3400740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e137067500 con 0x55e137152a00 2024-04-22T13:25:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.711+0000 7fe7a3400740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e137155a90 con 0x55e137149e80 2024-04-22T13:25:17.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.711+0000 7fe7a3400740 10 client.0 _ll_register_callbacks cb 0x55e137035b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.711+0000 7fe79a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1371494d0 0x55e1371522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.711+0000 7fe79a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1371494d0 0x55e1371522e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55780/0 (socket says 172.21.15.12:55780) 2024-04-22T13:25:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.711+0000 7fe79a59c640 1 -- 172.21.15.12:0/3667384430 learned_addr learned my addr 172.21.15.12:0/3667384430 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.711+0000 7fe799d9b640 1 --2- 172.21.15.12:0/3667384430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e137149e80 0x55e13714f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.711+0000 7fe79ad9d640 1 --2- 172.21.15.12:0/3667384430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e137152a00 0x55e13714fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.711+0000 7fe79a59c640 1 -- 172.21.15.12:0/3667384430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e137149e80 msgr2=0x55e13714f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.711+0000 7fe79a59c640 1 --2- 172.21.15.12:0/3667384430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e137149e80 0x55e13714f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.711+0000 7fe79a59c640 1 -- 172.21.15.12:0/3667384430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e137152a00 msgr2=0x55e13714fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.711+0000 7fe79a59c640 1 --2- 172.21.15.12:0/3667384430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e137152a00 0x55e13714fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.711+0000 7fe79a59c640 1 -- 172.21.15.12:0/3667384430 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e136dc03f0 con 0x55e1371494d0 2024-04-22T13:25:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.711+0000 7fe79ad9d640 1 --2- 172.21.15.12:0/3667384430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e137152a00 0x55e13714fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.711+0000 7fe79a59c640 1 --2- 172.21.15.12:0/3667384430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1371494d0 0x55e1371522e0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe78c09c1c0 tx=0x7fe78c09bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eef3d66dd93d628 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.711+0000 7fe781ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.712+0000 7fe781ffb640 1 -- 172.21.15.12:0/3667384430 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c09c4c0 con 0x55e1371494d0 2024-04-22T13:25:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.712+0000 7fe781ffb640 1 -- 172.21.15.12:0/3667384430 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fe78c098b30 con 0x55e1371494d0 2024-04-22T13:25:17.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.712+0000 7fe7a3400740 1 -- 172.21.15.12:0/3667384430 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe79403a040 con 0x55e1371494d0 2024-04-22T13:25:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.712+0000 7fe781ffb640 1 -- 172.21.15.12:0/3667384430 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe78c0a0870 con 0x55e1371494d0 2024-04-22T13:25:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.712+0000 7fe7a3400740 10 client.16212 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.712+0000 7fe7a3400740 1 -- 172.21.15.12:0/3667384430 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e136e8a7d0 con 0x55e1371494d0 2024-04-22T13:25:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.712+0000 7fe781ffb640 1 -- 172.21.15.12:0/3667384430 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe79403a040 con 0x55e1371494d0 2024-04-22T13:25:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.712+0000 7fe781ffb640 1 -- 172.21.15.12:0/3667384430 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe78c0982a0 con 0x55e1371494d0 2024-04-22T13:25:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.712+0000 7fe7a3400740 10 client.16212 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.712+0000 7fe7a3400740 10 client.16212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:25:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.712+0000 7fe7a3400740 20 client.16212 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.712+0000 7fe7a3400740 1 --2- 172.21.15.12:0/3667384430 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e137162ba0 0x55e137164f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.712+0000 7fe7a3400740 4 client.16212 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:25:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.712+0000 7fe7a3400740 1 -- 172.21.15.12:0/3667384430 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e13704d930 con 0x55e137162ba0 2024-04-22T13:25:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.713+0000 7fe799d9b640 1 --2- 172.21.15.12:0/3667384430 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e137162ba0 0x55e137164f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.713+0000 7fe781ffb640 10 client.16212 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.713+0000 7fe799d9b640 1 --2- 172.21.15.12:0/3667384430 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e137162ba0 0x55e137164f90 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e379e7faa2f5f961 server_cookie=41c793c72450d698 in_seq=0 out_seq=0 2024-04-22T13:25:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.714+0000 7fe781ffb640 1 -- 172.21.15.12:0/3667384430 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e13704d930 con 0x55e137162ba0 2024-04-22T13:25:17.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.714+0000 7fe781ffb640 10 client.16212 handle_command_reply: tid=0 2024-04-22T13:25:17.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.714+0000 7fe7a3400740 1 client.16212 shutdown 2024-04-22T13:25:17.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.714+0000 7fe7a3400740 1 -- 172.21.15.12:0/3667384430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1371494d0 msgr2=0x55e1371522e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.714+0000 7fe7a3400740 1 --2- 172.21.15.12:0/3667384430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1371494d0 0x55e1371522e0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe78c09c1c0 tx=0x7fe78c09bf50 comp rx=0 tx=0).stop 2024-04-22T13:25:17.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.714+0000 7fe7a3400740 1 -- 172.21.15.12:0/3667384430 shutdown_connections 2024-04-22T13:25:17.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.714+0000 7fe7a3400740 1 --2- 172.21.15.12:0/3667384430 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e137162ba0 0x55e137164f90 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.714+0000 7fe7a3400740 1 --2- 172.21.15.12:0/3667384430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e137152a00 0x55e13714fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.714+0000 7fe7a3400740 1 --2- 172.21.15.12:0/3667384430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e137149e80 0x55e13714f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.714+0000 7fe781ffb640 0 client.16212 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:17.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.714+0000 7fe7a3400740 1 --2- 172.21.15.12:0/3667384430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1371494d0 0x55e1371522e0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.714+0000 7fe7a3400740 1 -- 172.21.15.12:0/3667384430 >> 172.21.15.12:0/3667384430 conn(0x55e137143ba0 msgr2=0x55e1371458f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.714+0000 7fe7a3400740 1 -- 172.21.15.12:0/3667384430 shutdown_connections 2024-04-22T13:25:17.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.714+0000 7fe7a3400740 1 -- 172.21.15.12:0/3667384430 wait complete. 2024-04-22T13:25:17.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.714+0000 7fe7a3400740 20 client.16212 trim_cache size 0 max 16384 2024-04-22T13:25:17.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.728+0000 7fe7a3400740 1 Processor -- start 2024-04-22T13:25:17.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.728+0000 7fe7a3400740 1 -- start start 2024-04-22T13:25:17.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.729+0000 7fe7a3400740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1371494d0 0x55e13715e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.729+0000 7fe7a3400740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e137149e80 0x55e13714ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.729+0000 7fe7a3400740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e137152a00 0x55e137151da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.729+0000 7fe7a3400740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c1afc20 con 0x55e1371494d0 2024-04-22T13:25:17.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.729+0000 7fe7a3400740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c1b1760 con 0x55e137149e80 2024-04-22T13:25:17.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.729+0000 7fe7a3400740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c1b02f0 con 0x55e137152a00 2024-04-22T13:25:17.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.729+0000 7fe79a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1371494d0 0x55e13715e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.729+0000 7fe79a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1371494d0 0x55e13715e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55790/0 (socket says 172.21.15.12:55790) 2024-04-22T13:25:17.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.729+0000 7fe79a59c640 1 -- 172.21.15.12:0/3385255245 learned_addr learned my addr 172.21.15.12:0/3385255245 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:17.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.729+0000 7fe79ad9d640 1 --2- 172.21.15.12:0/3385255245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e137152a00 0x55e137151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.729+0000 7fe799d9b640 1 --2- 172.21.15.12:0/3385255245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e137149e80 0x55e13714ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.729+0000 7fe79a59c640 1 -- 172.21.15.12:0/3385255245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e137152a00 msgr2=0x55e137151da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.729+0000 7fe79a59c640 1 --2- 172.21.15.12:0/3385255245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e137152a00 0x55e137151da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.729+0000 7fe79a59c640 1 -- 172.21.15.12:0/3385255245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e137149e80 msgr2=0x55e13714ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.729+0000 7fe79a59c640 1 --2- 172.21.15.12:0/3385255245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e137149e80 0x55e13714ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.729+0000 7fe79a59c640 1 -- 172.21.15.12:0/3385255245 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e137165300 con 0x55e1371494d0 2024-04-22T13:25:17.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.729+0000 7fe79ad9d640 1 --2- 172.21.15.12:0/3385255245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e137152a00 0x55e137151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:17.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.729+0000 7fe79a59c640 1 --2- 172.21.15.12:0/3385255245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1371494d0 0x55e13715e800 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fe78c009610 tx=0x7fe78c09cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e651960253628a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.730+0000 7fe783fff640 1 -- 172.21.15.12:0/3385255245 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c0a5130 con 0x55e1371494d0 2024-04-22T13:25:17.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.730+0000 7fe783fff640 1 -- 172.21.15.12:0/3385255245 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe78c0b5070 con 0x55e1371494d0 2024-04-22T13:25:17.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.730+0000 7fe783fff640 1 -- 172.21.15.12:0/3385255245 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c098210 con 0x55e1371494d0 2024-04-22T13:25:17.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.730+0000 7fe7a3400740 1 -- 172.21.15.12:0/3385255245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1371494d0 msgr2=0x55e13715e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.730+0000 7fe7a3400740 1 --2- 172.21.15.12:0/3385255245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1371494d0 0x55e13715e800 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fe78c009610 tx=0x7fe78c09cd00 comp rx=0 tx=0).stop 2024-04-22T13:25:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.730+0000 7fe7a3400740 1 -- 172.21.15.12:0/3385255245 shutdown_connections 2024-04-22T13:25:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.730+0000 7fe7a3400740 1 --2- 172.21.15.12:0/3385255245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e137152a00 0x55e137151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.730+0000 7fe7a3400740 1 --2- 172.21.15.12:0/3385255245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e137149e80 0x55e13714ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.730+0000 7fe7a3400740 1 --2- 172.21.15.12:0/3385255245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1371494d0 0x55e13715e800 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.730+0000 7fe7a3400740 1 -- 172.21.15.12:0/3385255245 >> 172.21.15.12:0/3385255245 conn(0x55e137143ba0 msgr2=0x55e137142c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.730+0000 7fe7a3400740 1 -- 172.21.15.12:0/3385255245 shutdown_connections 2024-04-22T13:25:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.730+0000 7fe7a3400740 1 -- 172.21.15.12:0/3385255245 wait complete. 2024-04-22T13:25:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.731+0000 7fe7a3400740 1 -- start start 2024-04-22T13:25:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.731+0000 7fe7a3400740 1 Processor -- start 2024-04-22T13:25:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.731+0000 7fe7a3400740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1371494d0 0x55e137156790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.731+0000 7fe7a3400740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e137149e80 0x55e137156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.731+0000 7fe7a3400740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e137152a00 0x55e137157210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.731+0000 7fe7a3400740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c1b1090 con 0x55e137152a00 2024-04-22T13:25:17.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.731+0000 7fe7a3400740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c1b09c0 con 0x55e1371494d0 2024-04-22T13:25:17.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.731+0000 7fe7a3400740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c1adc20 con 0x55e137149e80 2024-04-22T13:25:17.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.731+0000 7fe7a3400740 10 client.0 _ll_register_callbacks cb 0x7fe78c09c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:17.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.731+0000 7fe79ad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e137152a00 0x55e137157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.731+0000 7fe79ad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e137152a00 0x55e137157210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55800/0 (socket says 172.21.15.12:55800) 2024-04-22T13:25:17.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.731+0000 7fe79ad9d640 1 -- 172.21.15.12:0/2336955827 learned_addr learned my addr 172.21.15.12:0/2336955827 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:17.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.731+0000 7fe799d9b640 1 --2- 172.21.15.12:0/2336955827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e137149e80 0x55e137156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.731+0000 7fe79a59c640 1 --2- 172.21.15.12:0/2336955827 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1371494d0 0x55e137156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.732+0000 7fe79ad9d640 1 -- 172.21.15.12:0/2336955827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e137149e80 msgr2=0x55e137156cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.732+0000 7fe79ad9d640 1 --2- 172.21.15.12:0/2336955827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e137149e80 0x55e137156cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.732+0000 7fe79ad9d640 1 -- 172.21.15.12:0/2336955827 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1371494d0 msgr2=0x55e137156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:17.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.732+0000 7fe79ad9d640 1 --2- 172.21.15.12:0/2336955827 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1371494d0 0x55e137156790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.732+0000 7fe79ad9d640 1 -- 172.21.15.12:0/2336955827 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e13714ff40 con 0x55e137152a00 2024-04-22T13:25:17.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.732+0000 7fe79ad9d640 1 --2- 172.21.15.12:0/2336955827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e137152a00 0x55e137157210 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe79402c630 tx=0x7fe794003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb05ca06863573d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:17.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.732+0000 7fe781ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:17.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.732+0000 7fe781ffb640 1 -- 172.21.15.12:0/2336955827 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79404f040 con 0x55e137152a00 2024-04-22T13:25:17.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.732+0000 7fe781ffb640 1 -- 172.21.15.12:0/2336955827 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fe794034640 con 0x55e137152a00 2024-04-22T13:25:17.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.732+0000 7fe7a3400740 1 -- 172.21.15.12:0/2336955827 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe78c0b5070 con 0x55e137152a00 2024-04-22T13:25:17.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.732+0000 7fe781ffb640 1 -- 172.21.15.12:0/2336955827 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe794034840 con 0x55e137152a00 2024-04-22T13:25:17.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.732+0000 7fe7a3400740 10 client.16218 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:17.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.732+0000 7fe7a3400740 1 -- 172.21.15.12:0/2336955827 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e1371c9250 con 0x55e137152a00 2024-04-22T13:25:17.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.733+0000 7fe781ffb640 1 -- 172.21.15.12:0/2336955827 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe79403a040 con 0x55e137152a00 2024-04-22T13:25:17.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.733+0000 7fe781ffb640 1 -- 172.21.15.12:0/2336955827 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe794049030 con 0x55e137152a00 2024-04-22T13:25:17.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.733+0000 7fe7a3400740 10 client.16218 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:17.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.733+0000 7fe7a3400740 10 client.16218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:25:17.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.733+0000 7fe7a3400740 20 client.16218 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:17.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.733+0000 7fe7a3400740 1 --2- 172.21.15.12:0/2336955827 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e137162ba0 0x55e1371cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:17.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.733+0000 7fe7a3400740 4 client.16218 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:25:17.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.733+0000 7fe7a3400740 1 -- 172.21.15.12:0/2336955827 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fe78c0b5070 con 0x55e137162ba0 2024-04-22T13:25:17.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.733+0000 7fe79a59c640 1 --2- 172.21.15.12:0/2336955827 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e137162ba0 0x55e1371cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:17.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.734+0000 7fe781ffb640 10 client.16218 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:17.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.734+0000 7fe79a59c640 1 --2- 172.21.15.12:0/2336955827 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e137162ba0 0x55e1371cb620 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1507075e2ccdd40a server_cookie=fda6460567eb74f in_seq=0 out_seq=0 2024-04-22T13:25:17.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.734+0000 7fe781ffb640 1 -- 172.21.15.12:0/2336955827 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fe78c0b5070 con 0x55e137162ba0 2024-04-22T13:25:17.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.734+0000 7fe781ffb640 10 client.16218 handle_command_reply: tid=0 2024-04-22T13:25:17.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.734+0000 7fe7a3400740 1 client.16218 shutdown 2024-04-22T13:25:17.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.734+0000 7fe7a3400740 1 -- 172.21.15.12:0/2336955827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e137152a00 msgr2=0x55e137157210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.734+0000 7fe7a3400740 1 --2- 172.21.15.12:0/2336955827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e137152a00 0x55e137157210 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe79402c630 tx=0x7fe794003b00 comp rx=0 tx=0).stop 2024-04-22T13:25:17.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.734+0000 7fe7a3400740 1 -- 172.21.15.12:0/2336955827 shutdown_connections 2024-04-22T13:25:17.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.734+0000 7fe7a3400740 1 --2- 172.21.15.12:0/2336955827 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e137162ba0 0x55e1371cb620 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.734+0000 7fe781ffb640 0 client.16218 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:17.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.734+0000 7fe7a3400740 1 --2- 172.21.15.12:0/2336955827 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e137152a00 0x55e137157210 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.734+0000 7fe7a3400740 1 --2- 172.21.15.12:0/2336955827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e137149e80 0x55e137156cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.734+0000 7fe7a3400740 1 --2- 172.21.15.12:0/2336955827 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1371494d0 0x55e137156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.734+0000 7fe7a3400740 1 -- 172.21.15.12:0/2336955827 >> 172.21.15.12:0/2336955827 conn(0x55e137143ba0 msgr2=0x55e13714a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.734+0000 7fe7a3400740 1 -- 172.21.15.12:0/2336955827 shutdown_connections 2024-04-22T13:25:17.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.734+0000 7fe7a3400740 1 -- 172.21.15.12:0/2336955827 wait complete. 2024-04-22T13:25:17.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.734+0000 7fe7a3400740 20 client.16218 trim_cache size 0 max 16384 2024-04-22T13:25:17.736 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:25:17.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.735+0000 7fe7a0811640 1 -- 172.21.15.12:0/861027013 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe7840a3f50 msgr2=0x7fe7840a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.735+0000 7fe7a0811640 1 --2- 172.21.15.12:0/861027013 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe7840a3f50 0x7fe7840a6410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe794006010 tx=0x7fe794005c50 comp rx=0 tx=0).stop 2024-04-22T13:25:17.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.735+0000 7fe7a0811640 1 -- 172.21.15.12:0/861027013 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe79c100d50 msgr2=0x7fe79c1a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:17.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.735+0000 7fe7a0811640 1 --2- 172.21.15.12:0/861027013 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe79c100d50 0x7fe79c1a0670 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00d720 tx=0x7fe78c00dbe0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.735+0000 7fe7a0811640 1 -- 172.21.15.12:0/861027013 shutdown_connections 2024-04-22T13:25:17.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.735+0000 7fe7a0811640 1 --2- 172.21.15.12:0/861027013 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe7840a3f50 0x7fe7840a6410 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.735+0000 7fe7a0811640 1 --2- 172.21.15.12:0/861027013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe79c10e900 0x7fe79c19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.735+0000 7fe7a0811640 1 --2- 172.21.15.12:0/861027013 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe79c101670 0x7fe79c1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.735+0000 7fe7a0811640 1 --2- 172.21.15.12:0/861027013 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe79c100d50 0x7fe79c1a0670 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:17.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.735+0000 7fe7a0811640 1 -- 172.21.15.12:0/861027013 >> 172.21.15.12:0/861027013 conn(0x7fe79c077f50 msgr2=0x7fe79c10ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:17.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.735+0000 7fe7a0811640 1 -- 172.21.15.12:0/861027013 shutdown_connections 2024-04-22T13:25:17.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:17.735+0000 7fe7a0811640 1 -- 172.21.15.12:0/861027013 wait complete. 2024-04-22T13:25:17.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:17 smithi134 ceph-mon[31338]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-22T13:25:17.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:17 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/4275890086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:25:17.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:17 smithi111 ceph-mon[31361]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-22T13:25:17.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:17 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/4275890086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:25:18.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:17 smithi012 ceph-mon[28265]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-22T13:25:18.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:17 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/4275890086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:25:18.743 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:25:18.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:25:18.743 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:25:18.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.874+0000 7fef0907a640 1 -- 172.21.15.12:0/1565030628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef04103970 msgr2=0x7fef0410c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:18.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.874+0000 7fef0907a640 1 --2- 172.21.15.12:0/1565030628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef04103970 0x7fef0410c030 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7feef8009900 tx=0x7feef802bcd0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.875+0000 7fef0907a640 1 -- 172.21.15.12:0/1565030628 shutdown_connections 2024-04-22T13:25:18.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.875+0000 7fef0907a640 1 --2- 172.21.15.12:0/1565030628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef0410c570 0x7fef0410e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.875+0000 7fef0907a640 1 --2- 172.21.15.12:0/1565030628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef04103970 0x7fef0410c030 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.875+0000 7fef0907a640 1 --2- 172.21.15.12:0/1565030628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef04103050 0x7fef04103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.875+0000 7fef0907a640 1 -- 172.21.15.12:0/1565030628 >> 172.21.15.12:0/1565030628 conn(0x7fef04077fb0 msgr2=0x7fef040783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:18.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.875+0000 7fef0907a640 1 -- 172.21.15.12:0/1565030628 shutdown_connections 2024-04-22T13:25:18.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.875+0000 7fef0907a640 1 -- 172.21.15.12:0/1565030628 wait complete. 2024-04-22T13:25:18.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.875+0000 7fef0907a640 1 Processor -- start 2024-04-22T13:25:18.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.876+0000 7fef0907a640 1 -- start start 2024-04-22T13:25:18.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.876+0000 7fef0907a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef04103050 0x7fef04100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:18.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.876+0000 7fef0907a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef04103970 0x7fef041013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:18.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.876+0000 7fef0907a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef0410c570 0x7fef04101900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:18.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.876+0000 7fef0907a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fef041121b0 con 0x7fef04103970 2024-04-22T13:25:18.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.876+0000 7fef0907a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fef04112030 con 0x7fef0410c570 2024-04-22T13:25:18.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.876+0000 7fef0907a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fef04112330 con 0x7fef04103050 2024-04-22T13:25:18.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.876+0000 7fef0259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef04103970 0x7fef041013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:18.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.876+0000 7fef02d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef04103050 0x7fef04100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:18.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.876+0000 7fef02d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef04103050 0x7fef04100e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.12:54480/0 (socket says 172.21.15.12:54480) 2024-04-22T13:25:18.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.876+0000 7fef0259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef04103970 0x7fef041013c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55808/0 (socket says 172.21.15.12:55808) 2024-04-22T13:25:18.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.876+0000 7fef02d9d640 1 -- 172.21.15.12:0/2318118020 learned_addr learned my addr 172.21.15.12:0/2318118020 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:18.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.876+0000 7fef0359e640 1 --2- 172.21.15.12:0/2318118020 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef0410c570 0x7fef04101900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:18.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.876+0000 7fef0259c640 1 -- 172.21.15.12:0/2318118020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef04103050 msgr2=0x7fef04100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:18.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.876+0000 7fef0259c640 1 --2- 172.21.15.12:0/2318118020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef04103050 0x7fef04100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.876+0000 7fef0259c640 1 -- 172.21.15.12:0/2318118020 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef0410c570 msgr2=0x7fef04101900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:18.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.876+0000 7fef0259c640 1 --2- 172.21.15.12:0/2318118020 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef0410c570 0x7fef04101900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.876+0000 7fef0259c640 1 -- 172.21.15.12:0/2318118020 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef041a6c10 con 0x7fef04103970 2024-04-22T13:25:18.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.877+0000 7fef02d9d640 1 --2- 172.21.15.12:0/2318118020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef04103050 0x7fef04100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:18.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.877+0000 7fef0259c640 1 --2- 172.21.15.12:0/2318118020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef04103970 0x7fef041013c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7feef8036040 tx=0x7feef802d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:18.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.877+0000 7feeebfff640 1 -- 172.21.15.12:0/2318118020 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef8043020 con 0x7fef04103970 2024-04-22T13:25:18.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.877+0000 7feeebfff640 1 -- 172.21.15.12:0/2318118020 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feef802dc90 con 0x7fef04103970 2024-04-22T13:25:18.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.877+0000 7feeebfff640 1 -- 172.21.15.12:0/2318118020 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef8003ac0 con 0x7fef04103970 2024-04-22T13:25:18.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.877+0000 7fef0907a640 1 -- 172.21.15.12:0/2318118020 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef041a6ea0 con 0x7fef04103970 2024-04-22T13:25:18.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.877+0000 7fef0907a640 1 -- 172.21.15.12:0/2318118020 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef041a7360 con 0x7fef04103970 2024-04-22T13:25:18.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.879+0000 7fef0bc6a740 1 Processor -- start 2024-04-22T13:25:18.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.880+0000 7fef0bc6a740 1 -- start start 2024-04-22T13:25:18.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.880+0000 7feeebfff640 1 -- 172.21.15.12:0/2318118020 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7feef8046020 con 0x7fef04103970 2024-04-22T13:25:18.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.880+0000 7fef0bc6a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a2f1088e0 0x563a2f108cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:18.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.880+0000 7fef0bc6a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563a2f109290 0x563a2f111840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:18.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.880+0000 7fef0bc6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a2f111e10 0x563a2f114260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:18.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.880+0000 7fef0bc6a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563a2f06cf00 con 0x563a2f1088e0 2024-04-22T13:25:18.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.880+0000 7fef0bc6a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x563a2f00a590 con 0x563a2f109290 2024-04-22T13:25:18.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.880+0000 7fef0bc6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563a2f068ec0 con 0x563a2f111e10 2024-04-22T13:25:18.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.880+0000 7fef0359e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a2f1088e0 0x563a2f108cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:18.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.880+0000 7fef0359e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a2f1088e0 0x563a2f108cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55812/0 (socket says 172.21.15.12:55812) 2024-04-22T13:25:18.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.880+0000 7fef0359e640 1 -- 172.21.15.12:0/3404388601 learned_addr learned my addr 172.21.15.12:0/3404388601 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:18.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.880+0000 7fef0259c640 1 --2- 172.21.15.12:0/3404388601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a2f111e10 0x563a2f114260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:18.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.880+0000 7fef02d9d640 1 --2- 172.21.15.12:0/3404388601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563a2f109290 0x563a2f111840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:18.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.880+0000 7fef0359e640 1 -- 172.21.15.12:0/3404388601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a2f111e10 msgr2=0x563a2f114260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:18.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.880+0000 7fef0359e640 1 --2- 172.21.15.12:0/3404388601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a2f111e10 0x563a2f114260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.880+0000 7fef0359e640 1 -- 172.21.15.12:0/3404388601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563a2f109290 msgr2=0x563a2f111840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:18.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.880+0000 7fef0359e640 1 --2- 172.21.15.12:0/3404388601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563a2f109290 0x563a2f111840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.880+0000 7fef0359e640 1 -- 172.21.15.12:0/3404388601 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a2ee477e0 con 0x563a2f1088e0 2024-04-22T13:25:18.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.880+0000 7fef02d9d640 1 --2- 172.21.15.12:0/3404388601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563a2f109290 0x563a2f111840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:18.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.880+0000 7fef0259c640 1 --2- 172.21.15.12:0/3404388601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a2f111e10 0x563a2f114260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:18.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.881+0000 7feeebfff640 1 --2- 172.21.15.12:0/2318118020 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7feeec0a3ed0 0x7feeec0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:18.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.881+0000 7fef0359e640 1 --2- 172.21.15.12:0/3404388601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a2f1088e0 0x563a2f108cc0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7feefc009f60 tx=0x7feefc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac69c7e3f7e031f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:18.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.881+0000 7feeebfff640 1 -- 172.21.15.12:0/2318118020 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7feef80f2970 con 0x7fef04103970 2024-04-22T13:25:18.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.881+0000 7fef02d9d640 1 --2- 172.21.15.12:0/2318118020 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7feeec0a3ed0 0x7feeec0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:18.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.881+0000 7feee97fa640 1 -- 172.21.15.12:0/3404388601 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc016070 con 0x563a2f1088e0 2024-04-22T13:25:18.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.881+0000 7feee97fa640 1 -- 172.21.15.12:0/3404388601 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feefc004d20 con 0x563a2f1088e0 2024-04-22T13:25:18.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.881+0000 7feee97fa640 1 -- 172.21.15.12:0/3404388601 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc012790 con 0x563a2f1088e0 2024-04-22T13:25:18.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.882+0000 7fef0bc6a740 1 -- 172.21.15.12:0/3404388601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a2f1088e0 msgr2=0x563a2f108cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:18.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.882+0000 7fef02d9d640 1 --2- 172.21.15.12:0/2318118020 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7feeec0a3ed0 0x7feeec0a6390 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7feef400c010 tx=0x7feef400b480 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:18.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.882+0000 7fef0bc6a740 1 --2- 172.21.15.12:0/3404388601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a2f1088e0 0x563a2f108cc0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7feefc009f60 tx=0x7feefc009f90 comp rx=0 tx=0).stop 2024-04-22T13:25:18.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.882+0000 7fef0bc6a740 1 -- 172.21.15.12:0/3404388601 shutdown_connections 2024-04-22T13:25:18.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.882+0000 7fef0bc6a740 1 --2- 172.21.15.12:0/3404388601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a2f111e10 0x563a2f114260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.882+0000 7fef0bc6a740 1 --2- 172.21.15.12:0/3404388601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563a2f109290 0x563a2f111840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.882+0000 7fef0bc6a740 1 --2- 172.21.15.12:0/3404388601 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a2f1088e0 0x563a2f108cc0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.882+0000 7fef0bc6a740 1 -- 172.21.15.12:0/3404388601 >> 172.21.15.12:0/3404388601 conn(0x563a2f102fb0 msgr2=0x563a2f103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:18.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.882+0000 7fef0bc6a740 1 -- 172.21.15.12:0/3404388601 shutdown_connections 2024-04-22T13:25:18.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.882+0000 7fef0bc6a740 1 -- 172.21.15.12:0/3404388601 wait complete. 2024-04-22T13:25:18.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.882+0000 7fef0bc6a740 1 -- start start 2024-04-22T13:25:18.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.882+0000 7fef0bc6a740 1 Processor -- start 2024-04-22T13:25:18.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.883+0000 7fef0bc6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a2f1088e0 0x563a2f10ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:18.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.883+0000 7fef0bc6a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563a2f109290 0x563a2f10f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:18.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.883+0000 7fef0bc6a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a2f111e10 0x563a2f10f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:18.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.883+0000 7fef0bc6a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563a2f114d80 con 0x563a2f111e10 2024-04-22T13:25:18.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.883+0000 7fef0bc6a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x563a2f026580 con 0x563a2f109290 2024-04-22T13:25:18.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.883+0000 7fef0bc6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563a2f114f00 con 0x563a2f1088e0 2024-04-22T13:25:18.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.883+0000 7fef0bc6a740 10 client.0 _ll_register_callbacks cb 0x563a2f02c650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:18.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.883+0000 7fef0359e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a2f111e10 0x563a2f10f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:18.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.883+0000 7fef0359e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a2f111e10 0x563a2f10f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55818/0 (socket says 172.21.15.12:55818) 2024-04-22T13:25:18.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.883+0000 7fef0359e640 1 -- 172.21.15.12:0/1531761721 learned_addr learned my addr 172.21.15.12:0/1531761721 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:18.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.883+0000 7fef02d9d640 1 --2- 172.21.15.12:0/1531761721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a2f1088e0 0x563a2f10ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.883+0000 7fef0259c640 1 --2- 172.21.15.12:0/1531761721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563a2f109290 0x563a2f10f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.883+0000 7fef0359e640 1 -- 172.21.15.12:0/1531761721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a2f1088e0 msgr2=0x563a2f10ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.883+0000 7fef0359e640 1 --2- 172.21.15.12:0/1531761721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a2f1088e0 0x563a2f10ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.883+0000 7fef0359e640 1 -- 172.21.15.12:0/1531761721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563a2f109290 msgr2=0x563a2f10f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.883+0000 7fef0359e640 1 --2- 172.21.15.12:0/1531761721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563a2f109290 0x563a2f10f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.883+0000 7fef0359e640 1 -- 172.21.15.12:0/1531761721 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a2eea7f70 con 0x563a2f111e10 2024-04-22T13:25:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.883+0000 7fef0359e640 1 --2- 172.21.15.12:0/1531761721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a2f111e10 0x563a2f10f8f0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7feefc004c50 tx=0x7feefc004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7a2d9d14aa921 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.883+0000 7feee6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.884+0000 7feee6ff5640 1 -- 172.21.15.12:0/1531761721 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc016030 con 0x563a2f111e10 2024-04-22T13:25:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.884+0000 7feee6ff5640 1 -- 172.21.15.12:0/1531761721 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7feefc003b80 con 0x563a2f111e10 2024-04-22T13:25:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.884+0000 7fef0bc6a740 1 -- 172.21.15.12:0/1531761721 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feefc004d20 con 0x563a2f111e10 2024-04-22T13:25:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.884+0000 7feee6ff5640 1 -- 172.21.15.12:0/1531761721 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feefc003d80 con 0x563a2f111e10 2024-04-22T13:25:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.884+0000 7fef0bc6a740 10 client.16230 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.884+0000 7fef0bc6a740 1 -- 172.21.15.12:0/1531761721 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563a2ed3f170 con 0x563a2f111e10 2024-04-22T13:25:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.884+0000 7feee6ff5640 1 -- 172.21.15.12:0/1531761721 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feefc004d20 con 0x563a2f111e10 2024-04-22T13:25:18.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.884+0000 7feee6ff5640 1 -- 172.21.15.12:0/1531761721 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7feefc00f040 con 0x563a2f111e10 2024-04-22T13:25:18.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.884+0000 7fef0bc6a740 10 client.16230 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:18.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.884+0000 7fef0bc6a740 10 client.16230 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:25:18.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.884+0000 7fef0bc6a740 20 client.16230 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:18.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.884+0000 7fef0bc6a740 1 --2- 172.21.15.12:0/1531761721 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x563a2f1222c0 0x563a2f1246b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:18.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.884+0000 7fef0bc6a740 4 client.16230 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:25:18.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.884+0000 7fef0bc6a740 1 -- 172.21.15.12:0/1531761721 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a2f07a920 con 0x563a2f1222c0 2024-04-22T13:25:18.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.885+0000 7fef02d9d640 1 --2- 172.21.15.12:0/1531761721 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x563a2f1222c0 0x563a2f1246b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:18.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.885+0000 7fef02d9d640 1 --2- 172.21.15.12:0/1531761721 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x563a2f1222c0 0x563a2f1246b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85cf660fcf2fc3c7 server_cookie=98e1cd20bd3f96e7 in_seq=0 out_seq=0 2024-04-22T13:25:18.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.885+0000 7feee6ff5640 10 client.16230 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:18.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.886+0000 7feee6ff5640 1 -- 172.21.15.12:0/1531761721 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563a2f07a920 con 0x563a2f1222c0 2024-04-22T13:25:18.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.886+0000 7feee6ff5640 10 client.16230 handle_command_reply: tid=0 2024-04-22T13:25:18.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.886+0000 7fef0bc6a740 1 client.16230 shutdown 2024-04-22T13:25:18.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.886+0000 7fef0bc6a740 1 -- 172.21.15.12:0/1531761721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a2f111e10 msgr2=0x563a2f10f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:18.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.886+0000 7fef0bc6a740 1 --2- 172.21.15.12:0/1531761721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a2f111e10 0x563a2f10f8f0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7feefc004c50 tx=0x7feefc004c80 comp rx=0 tx=0).stop 2024-04-22T13:25:18.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.886+0000 7fef0bc6a740 1 -- 172.21.15.12:0/1531761721 shutdown_connections 2024-04-22T13:25:18.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.886+0000 7fef0bc6a740 1 --2- 172.21.15.12:0/1531761721 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x563a2f1222c0 0x563a2f1246b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.886+0000 7fef0bc6a740 1 --2- 172.21.15.12:0/1531761721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a2f111e10 0x563a2f10f8f0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.886+0000 7feee6ff5640 0 client.16230 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:18.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.886+0000 7fef0bc6a740 1 --2- 172.21.15.12:0/1531761721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563a2f109290 0x563a2f10f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.886+0000 7fef0bc6a740 1 --2- 172.21.15.12:0/1531761721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a2f1088e0 0x563a2f10ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.886+0000 7fef0bc6a740 1 -- 172.21.15.12:0/1531761721 >> 172.21.15.12:0/1531761721 conn(0x563a2f102fb0 msgr2=0x563a2f104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:18.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.886+0000 7fef0bc6a740 1 -- 172.21.15.12:0/1531761721 shutdown_connections 2024-04-22T13:25:18.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.886+0000 7fef0bc6a740 1 -- 172.21.15.12:0/1531761721 wait complete. 2024-04-22T13:25:18.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.886+0000 7fef0bc6a740 20 client.16230 trim_cache size 0 max 16384 2024-04-22T13:25:18.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.904+0000 7fef0bc6a740 1 Processor -- start 2024-04-22T13:25:18.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.904+0000 7fef0bc6a740 1 -- start start 2024-04-22T13:25:18.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.904+0000 7fef0bc6a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a2f1088e0 0x563a2f1115a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:18.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.904+0000 7fef0bc6a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563a2f109290 0x563a2f11dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:18.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.904+0000 7fef0bc6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a2f111e10 0x563a2f129e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:18.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.904+0000 7fef0bc6a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fef041afc40 con 0x563a2f1088e0 2024-04-22T13:25:18.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.904+0000 7fef0bc6a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fef041b1780 con 0x563a2f109290 2024-04-22T13:25:18.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.904+0000 7fef0bc6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fef041b0310 con 0x563a2f111e10 2024-04-22T13:25:18.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.904+0000 7fef02d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a2f1088e0 0x563a2f1115a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:18.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.904+0000 7fef02d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a2f1088e0 0x563a2f1115a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55824/0 (socket says 172.21.15.12:55824) 2024-04-22T13:25:18.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.904+0000 7fef02d9d640 1 -- 172.21.15.12:0/2456182454 learned_addr learned my addr 172.21.15.12:0/2456182454 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:18.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.905+0000 7fef0259c640 1 --2- 172.21.15.12:0/2456182454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563a2f109290 0x563a2f11dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:18.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.905+0000 7fef0359e640 1 --2- 172.21.15.12:0/2456182454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a2f111e10 0x563a2f129e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:18.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.905+0000 7fef02d9d640 1 -- 172.21.15.12:0/2456182454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a2f111e10 msgr2=0x563a2f129e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:18.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.905+0000 7fef02d9d640 1 --2- 172.21.15.12:0/2456182454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a2f111e10 0x563a2f129e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.905+0000 7fef02d9d640 1 -- 172.21.15.12:0/2456182454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563a2f109290 msgr2=0x563a2f11dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:18.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.905+0000 7fef02d9d640 1 --2- 172.21.15.12:0/2456182454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563a2f109290 0x563a2f11dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.905+0000 7fef02d9d640 1 -- 172.21.15.12:0/2456182454 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a2f10dfe0 con 0x563a2f1088e0 2024-04-22T13:25:18.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.905+0000 7fef0259c640 1 --2- 172.21.15.12:0/2456182454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563a2f109290 0x563a2f11dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:18.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.905+0000 7fef0359e640 1 --2- 172.21.15.12:0/2456182454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a2f111e10 0x563a2f129e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:18.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.905+0000 7fef02d9d640 1 --2- 172.21.15.12:0/2456182454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a2f1088e0 0x563a2f1115a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7feef4005fd0 tx=0x7feef4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb27597ddc48e17 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:18.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.905+0000 7feee8ff9640 1 -- 172.21.15.12:0/2456182454 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4005020 con 0x563a2f1088e0 2024-04-22T13:25:18.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.905+0000 7feee8ff9640 1 -- 172.21.15.12:0/2456182454 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feef40047a0 con 0x563a2f1088e0 2024-04-22T13:25:18.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.905+0000 7feee8ff9640 1 -- 172.21.15.12:0/2456182454 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef402a290 con 0x563a2f1088e0 2024-04-22T13:25:18.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.906+0000 7fef0bc6a740 1 -- 172.21.15.12:0/2456182454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a2f1088e0 msgr2=0x563a2f1115a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:18.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.906+0000 7fef0bc6a740 1 --2- 172.21.15.12:0/2456182454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a2f1088e0 0x563a2f1115a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7feef4005fd0 tx=0x7feef4015000 comp rx=0 tx=0).stop 2024-04-22T13:25:18.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.906+0000 7fef0bc6a740 1 -- 172.21.15.12:0/2456182454 shutdown_connections 2024-04-22T13:25:18.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.906+0000 7fef0bc6a740 1 --2- 172.21.15.12:0/2456182454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a2f111e10 0x563a2f129e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.906+0000 7fef0bc6a740 1 --2- 172.21.15.12:0/2456182454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563a2f109290 0x563a2f11dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.906+0000 7fef0bc6a740 1 --2- 172.21.15.12:0/2456182454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a2f1088e0 0x563a2f1115a0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.906+0000 7fef0bc6a740 1 -- 172.21.15.12:0/2456182454 >> 172.21.15.12:0/2456182454 conn(0x563a2f102fb0 msgr2=0x563a2f10f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:18.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.906+0000 7fef0bc6a740 1 -- 172.21.15.12:0/2456182454 shutdown_connections 2024-04-22T13:25:18.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.906+0000 7fef0bc6a740 1 -- 172.21.15.12:0/2456182454 wait complete. 2024-04-22T13:25:18.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.906+0000 7fef0bc6a740 1 -- start start 2024-04-22T13:25:18.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.907+0000 7fef0bc6a740 1 Processor -- start 2024-04-22T13:25:18.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.907+0000 7fef0bc6a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a2f1088e0 0x563a2f10bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:18.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.907+0000 7fef0bc6a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563a2f109290 0x563a2f10c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:18.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.907+0000 7fef0bc6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a2f111e10 0x563a2f10c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:18.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.907+0000 7fef0bc6a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fef041b10b0 con 0x563a2f1088e0 2024-04-22T13:25:18.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.907+0000 7fef0bc6a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fef041b09e0 con 0x563a2f109290 2024-04-22T13:25:18.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.907+0000 7fef0bc6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fef041adc40 con 0x563a2f111e10 2024-04-22T13:25:18.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.907+0000 7fef02d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a2f1088e0 0x563a2f10bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:18.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.907+0000 7fef0bc6a740 10 client.0 _ll_register_callbacks cb 0x7feefc009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:18.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.907+0000 7fef02d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a2f1088e0 0x563a2f10bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55832/0 (socket says 172.21.15.12:55832) 2024-04-22T13:25:18.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.907+0000 7fef02d9d640 1 -- 172.21.15.12:0/2175930806 learned_addr learned my addr 172.21.15.12:0/2175930806 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:18.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.907+0000 7fef0259c640 1 --2- 172.21.15.12:0/2175930806 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563a2f109290 0x563a2f10c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:18.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.907+0000 7fef0359e640 1 --2- 172.21.15.12:0/2175930806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a2f111e10 0x563a2f10c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:18.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.907+0000 7fef02d9d640 1 -- 172.21.15.12:0/2175930806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a2f111e10 msgr2=0x563a2f10c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:18.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.907+0000 7fef02d9d640 1 --2- 172.21.15.12:0/2175930806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a2f111e10 0x563a2f10c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.907+0000 7fef02d9d640 1 -- 172.21.15.12:0/2175930806 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563a2f109290 msgr2=0x563a2f10c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:18.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.907+0000 7fef02d9d640 1 --2- 172.21.15.12:0/2175930806 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563a2f109290 0x563a2f10c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.907+0000 7fef02d9d640 1 -- 172.21.15.12:0/2175930806 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a2f187980 con 0x563a2f1088e0 2024-04-22T13:25:18.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.908+0000 7fef02d9d640 1 --2- 172.21.15.12:0/2175930806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a2f1088e0 0x563a2f10bdb0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7feef4010e30 tx=0x7feef4010e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0ea5cc4f165258 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:18.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.908+0000 7feee6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:18.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.908+0000 7feee6ff5640 1 -- 172.21.15.12:0/2175930806 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4038020 con 0x563a2f1088e0 2024-04-22T13:25:18.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.908+0000 7feee6ff5640 1 -- 172.21.15.12:0/2175930806 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7feef4004b00 con 0x563a2f1088e0 2024-04-22T13:25:18.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.908+0000 7fef0bc6a740 1 -- 172.21.15.12:0/2175930806 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feef40047a0 con 0x563a2f1088e0 2024-04-22T13:25:18.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.908+0000 7feee6ff5640 1 -- 172.21.15.12:0/2175930806 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feef4043050 con 0x563a2f1088e0 2024-04-22T13:25:18.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.908+0000 7fef0bc6a740 10 client.16236 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:18.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.908+0000 7fef0bc6a740 1 -- 172.21.15.12:0/2175930806 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563a2f188ca0 con 0x563a2f1088e0 2024-04-22T13:25:18.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.908+0000 7feee6ff5640 1 -- 172.21.15.12:0/2175930806 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feef4004d00 con 0x563a2f1088e0 2024-04-22T13:25:18.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.908+0000 7feee6ff5640 1 -- 172.21.15.12:0/2175930806 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7feef402a400 con 0x563a2f1088e0 2024-04-22T13:25:18.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.908+0000 7fef0bc6a740 10 client.16236 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:18.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.908+0000 7fef0bc6a740 10 client.16236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:25:18.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.908+0000 7fef0bc6a740 20 client.16236 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:18.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.909+0000 7fef0bc6a740 1 --2- 172.21.15.12:0/2175930806 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x563a2f1222c0 0x563a2f18b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:18.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.909+0000 7fef0bc6a740 4 client.16236 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:25:18.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.909+0000 7fef0bc6a740 1 -- 172.21.15.12:0/2175930806 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7feef40047a0 con 0x563a2f1222c0 2024-04-22T13:25:18.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.909+0000 7fef0259c640 1 --2- 172.21.15.12:0/2175930806 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x563a2f1222c0 0x563a2f18b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:18.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.909+0000 7fef0259c640 1 --2- 172.21.15.12:0/2175930806 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x563a2f1222c0 0x563a2f18b3c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0f3b7c9dd824117 server_cookie=d5d7a50ba9ce454f in_seq=0 out_seq=0 2024-04-22T13:25:18.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.909+0000 7feee6ff5640 10 client.16236 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:18.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.910+0000 7feee6ff5640 1 -- 172.21.15.12:0/2175930806 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7feef40047a0 con 0x563a2f1222c0 2024-04-22T13:25:18.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.910+0000 7feee6ff5640 10 client.16236 handle_command_reply: tid=0 2024-04-22T13:25:18.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.910+0000 7fef0bc6a740 1 client.16236 shutdown 2024-04-22T13:25:18.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.910+0000 7fef0bc6a740 1 -- 172.21.15.12:0/2175930806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a2f1088e0 msgr2=0x563a2f10bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:18.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.910+0000 7fef0bc6a740 1 --2- 172.21.15.12:0/2175930806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a2f1088e0 0x563a2f10bdb0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7feef4010e30 tx=0x7feef4010e60 comp rx=0 tx=0).stop 2024-04-22T13:25:18.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.911+0000 7fef0bc6a740 1 -- 172.21.15.12:0/2175930806 shutdown_connections 2024-04-22T13:25:18.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.911+0000 7fef0bc6a740 1 --2- 172.21.15.12:0/2175930806 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x563a2f1222c0 0x563a2f18b3c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.911+0000 7fef0bc6a740 1 --2- 172.21.15.12:0/2175930806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a2f111e10 0x563a2f10c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.911+0000 7fef0bc6a740 1 --2- 172.21.15.12:0/2175930806 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563a2f109290 0x563a2f10c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.911+0000 7feee6ff5640 0 client.16236 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:18.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.911+0000 7fef0bc6a740 1 --2- 172.21.15.12:0/2175930806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a2f1088e0 0x563a2f10bdb0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.911+0000 7fef0bc6a740 1 -- 172.21.15.12:0/2175930806 >> 172.21.15.12:0/2175930806 conn(0x563a2f102fb0 msgr2=0x563a2f1042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:18.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.911+0000 7fef0bc6a740 1 -- 172.21.15.12:0/2175930806 shutdown_connections 2024-04-22T13:25:18.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.911+0000 7fef0bc6a740 1 -- 172.21.15.12:0/2175930806 wait complete. 2024-04-22T13:25:18.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.911+0000 7fef0bc6a740 20 client.16236 trim_cache size 0 max 16384 2024-04-22T13:25:18.911 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:25:18.911 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-04-22T13:25:18.911 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "67a103cb-b619-407e-b1c9-e7afdd5ab456", 2024-04-22T13:25:18.911 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-04-22T13:25:18.911 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:25:18.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.911+0000 7fef0907a640 1 -- 172.21.15.12:0/2318118020 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7feeec0a3ed0 msgr2=0x7feeec0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:18.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.911+0000 7fef0907a640 1 --2- 172.21.15.12:0/2318118020 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7feeec0a3ed0 0x7feeec0a6390 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7feef400c010 tx=0x7feef400b480 comp rx=0 tx=0).stop 2024-04-22T13:25:18.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.911+0000 7fef0907a640 1 -- 172.21.15.12:0/2318118020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef04103970 msgr2=0x7fef041013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:18.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.912+0000 7fef0907a640 1 --2- 172.21.15.12:0/2318118020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef04103970 0x7fef041013c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7feef8036040 tx=0x7feef802d890 comp rx=0 tx=0).stop 2024-04-22T13:25:18.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.912+0000 7fef0907a640 1 -- 172.21.15.12:0/2318118020 shutdown_connections 2024-04-22T13:25:18.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.912+0000 7fef0907a640 1 --2- 172.21.15.12:0/2318118020 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7feeec0a3ed0 0x7feeec0a6390 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.912+0000 7fef0907a640 1 --2- 172.21.15.12:0/2318118020 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fef0410c570 0x7fef04101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.912+0000 7fef0907a640 1 --2- 172.21.15.12:0/2318118020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fef04103970 0x7fef041013c0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.912+0000 7fef0907a640 1 --2- 172.21.15.12:0/2318118020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fef04103050 0x7fef04100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:18.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.912+0000 7fef0907a640 1 -- 172.21.15.12:0/2318118020 >> 172.21.15.12:0/2318118020 conn(0x7fef04077fb0 msgr2=0x7fef040fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:18.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.912+0000 7fef0907a640 1 -- 172.21.15.12:0/2318118020 shutdown_connections 2024-04-22T13:25:18.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:18.912+0000 7fef0907a640 1 -- 172.21.15.12:0/2318118020 wait complete. 2024-04-22T13:25:18.922 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:25:18.922 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:25:19.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.051+0000 7f5e7d17e640 1 -- 172.21.15.12:0/1433898885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e78103ae0 msgr2=0x7f5e7810bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:19.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.051+0000 7f5e7d17e640 1 --2- 172.21.15.12:0/1433898885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e78103ae0 0x7f5e7810bfc0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c009900 tx=0x7f5e6c02bd60 comp rx=0 tx=0).stop 2024-04-22T13:25:19.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.051+0000 7f5e7d17e640 1 -- 172.21.15.12:0/1433898885 shutdown_connections 2024-04-22T13:25:19.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.051+0000 7f5e7d17e640 1 --2- 172.21.15.12:0/1433898885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e7810c500 0x7f5e7810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.051+0000 7f5e7d17e640 1 --2- 172.21.15.12:0/1433898885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e78103ae0 0x7f5e7810bfc0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.051+0000 7f5e7d17e640 1 --2- 172.21.15.12:0/1433898885 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e781031c0 0x7f5e781035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.051+0000 7f5e7d17e640 1 -- 172.21.15.12:0/1433898885 >> 172.21.15.12:0/1433898885 conn(0x7f5e78077fd0 msgr2=0x7f5e780783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:19.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.051+0000 7f5e7d17e640 1 -- 172.21.15.12:0/1433898885 shutdown_connections 2024-04-22T13:25:19.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.051+0000 7f5e7d17e640 1 -- 172.21.15.12:0/1433898885 wait complete. 2024-04-22T13:25:19.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.052+0000 7f5e7d17e640 1 Processor -- start 2024-04-22T13:25:19.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.052+0000 7f5e7d17e640 1 -- start start 2024-04-22T13:25:19.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.052+0000 7f5e7d17e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e781031c0 0x7f5e78100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:19.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.052+0000 7f5e7d17e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e7810c500 0x7f5e78101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:19.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.052+0000 7f5e7d17e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e78101890 0x7f5e781a6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:19.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.053+0000 7f5e7d17e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5e78112170 con 0x7f5e78101890 2024-04-22T13:25:19.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.053+0000 7f5e7d17e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5e78111ff0 con 0x7f5e7810c500 2024-04-22T13:25:19.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.053+0000 7f5e7d17e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5e781122f0 con 0x7f5e781031c0 2024-04-22T13:25:19.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.053+0000 7f5e7659c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e7810c500 0x7f5e78101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:19.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.053+0000 7f5e7659c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e7810c500 0x7f5e78101350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:44952/0 (socket says 172.21.15.12:44952) 2024-04-22T13:25:19.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.053+0000 7f5e7659c640 1 -- 172.21.15.12:0/2518256812 learned_addr learned my addr 172.21.15.12:0/2518256812 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:19.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.053+0000 7f5e76d9d640 1 --2- 172.21.15.12:0/2518256812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e781031c0 0x7f5e78100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:19.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.053+0000 7f5e7759e640 1 --2- 172.21.15.12:0/2518256812 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e78101890 0x7f5e781a6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:19.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.053+0000 7f5e7659c640 1 -- 172.21.15.12:0/2518256812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e781031c0 msgr2=0x7f5e78100e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:19.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.053+0000 7f5e7659c640 1 --2- 172.21.15.12:0/2518256812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e781031c0 0x7f5e78100e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.053+0000 7f5e7659c640 1 -- 172.21.15.12:0/2518256812 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e78101890 msgr2=0x7f5e781a6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:19.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.053+0000 7f5e7659c640 1 --2- 172.21.15.12:0/2518256812 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e78101890 0x7f5e781a6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.053+0000 7f5e7659c640 1 -- 172.21.15.12:0/2518256812 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e781a7210 con 0x7f5e7810c500 2024-04-22T13:25:19.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.053+0000 7f5e76d9d640 1 --2- 172.21.15.12:0/2518256812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e781031c0 0x7f5e78100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:19.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.053+0000 7f5e7759e640 1 --2- 172.21.15.12:0/2518256812 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e78101890 0x7f5e781a6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:19.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.053+0000 7f5e7659c640 1 --2- 172.21.15.12:0/2518256812 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e7810c500 0x7f5e78101350 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c035000 tx=0x7f5e6c0045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:19.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.054+0000 7f5e5ffff640 1 -- 172.21.15.12:0/2518256812 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c043020 con 0x7f5e7810c500 2024-04-22T13:25:19.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.054+0000 7f5e7d17e640 1 -- 172.21.15.12:0/2518256812 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e781a74a0 con 0x7f5e7810c500 2024-04-22T13:25:19.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.054+0000 7f5e5ffff640 1 -- 172.21.15.12:0/2518256812 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e6c02dc30 con 0x7f5e7810c500 2024-04-22T13:25:19.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.054+0000 7f5e5ffff640 1 -- 172.21.15.12:0/2518256812 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c003a10 con 0x7f5e7810c500 2024-04-22T13:25:19.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.054+0000 7f5e7d17e640 1 -- 172.21.15.12:0/2518256812 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e781a79f0 con 0x7f5e7810c500 2024-04-22T13:25:19.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.056+0000 7f5e5ffff640 1 -- 172.21.15.12:0/2518256812 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f5e6c046020 con 0x7f5e7810c500 2024-04-22T13:25:19.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.056+0000 7f5e7fd6b740 1 Processor -- start 2024-04-22T13:25:19.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.057+0000 7f5e7fd6b740 1 -- start start 2024-04-22T13:25:19.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.057+0000 7f5e5ffff640 1 --2- 172.21.15.12:0/2518256812 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5e600a3f80 0x7f5e600a6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:19.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.057+0000 7f5e7fd6b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e542ea5580 0x55e542ea5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:19.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.057+0000 7f5e7fd6b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e542ea5ea0 0x55e542eae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:19.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.057+0000 7f5e7fd6b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e542eaea80 0x55e542eb0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:19.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.057+0000 7f5e7fd6b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e542e0a1e0 con 0x55e542eaea80 2024-04-22T13:25:19.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.057+0000 7f5e7fd6b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e542da7390 con 0x55e542ea5580 2024-04-22T13:25:19.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.057+0000 7f5e7fd6b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e542e061a0 con 0x55e542ea5ea0 2024-04-22T13:25:19.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.057+0000 7f5e5ffff640 1 -- 172.21.15.12:0/2518256812 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f5e6c0f2990 con 0x7f5e7810c500 2024-04-22T13:25:19.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.057+0000 7f5e7759e640 1 --2- 172.21.15.12:0/2518256812 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5e600a3f80 0x7f5e600a6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:19.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.057+0000 7f5e7759e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e542eaea80 0x55e542eb0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:19.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.057+0000 7f5e7759e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e542eaea80 0x55e542eb0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55870/0 (socket says 172.21.15.12:55870) 2024-04-22T13:25:19.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.057+0000 7f5e7759e640 1 -- 172.21.15.12:0/1944528127 learned_addr learned my addr 172.21.15.12:0/1944528127 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:19.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.057+0000 7f5e76d9d640 1 --2- 172.21.15.12:0/1944528127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e542ea5580 0x55e542ea5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:19.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.057+0000 7f5e7659c640 1 --2- 172.21.15.12:0/1944528127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e542ea5ea0 0x55e542eae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:19.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.057+0000 7f5e7659c640 1 -- 172.21.15.12:0/1944528127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e542ea5580 msgr2=0x55e542ea5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:19.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.057+0000 7f5e7659c640 1 --2- 172.21.15.12:0/1944528127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e542ea5580 0x55e542ea5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.057+0000 7f5e7659c640 1 -- 172.21.15.12:0/1944528127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e542eaea80 msgr2=0x55e542eb0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:19.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.057+0000 7f5e7659c640 1 --2- 172.21.15.12:0/1944528127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e542eaea80 0x55e542eb0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.057+0000 7f5e7659c640 1 -- 172.21.15.12:0/1944528127 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e542be67d0 con 0x55e542ea5ea0 2024-04-22T13:25:19.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.057+0000 7f5e76d9d640 1 --2- 172.21.15.12:0/1944528127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e542ea5580 0x55e542ea5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:19.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.057+0000 7f5e7759e640 1 --2- 172.21.15.12:0/1944528127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e542eaea80 0x55e542eb0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:19.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.058+0000 7f5e7759e640 1 --2- 172.21.15.12:0/2518256812 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5e600a3f80 0x7f5e600a6440 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5e70009fd0 tx=0x7f5e70009f20 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:19.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.058+0000 7f5e7659c640 1 --2- 172.21.15.12:0/1944528127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e542ea5ea0 0x55e542eae540 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c0f0000 tx=0x7f5e6c0ba4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8033c9758f4dad4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:19.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.058+0000 7f5e5d7fa640 1 -- 172.21.15.12:0/1944528127 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c0ed6e0 con 0x55e542ea5ea0 2024-04-22T13:25:19.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.058+0000 7f5e5d7fa640 1 -- 172.21.15.12:0/1944528127 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e6c0be850 con 0x55e542ea5ea0 2024-04-22T13:25:19.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.058+0000 7f5e5d7fa640 1 -- 172.21.15.12:0/1944528127 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c0e0200 con 0x55e542ea5ea0 2024-04-22T13:25:19.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.059+0000 7f5e7fd6b740 1 -- 172.21.15.12:0/1944528127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e542ea5ea0 msgr2=0x55e542eae540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:19.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.059+0000 7f5e7fd6b740 1 --2- 172.21.15.12:0/1944528127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e542ea5ea0 0x55e542eae540 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c0f0000 tx=0x7f5e6c0ba4f0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.059+0000 7f5e7fd6b740 1 -- 172.21.15.12:0/1944528127 shutdown_connections 2024-04-22T13:25:19.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.059+0000 7f5e7fd6b740 1 --2- 172.21.15.12:0/1944528127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e542eaea80 0x55e542eb0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.059+0000 7f5e7fd6b740 1 --2- 172.21.15.12:0/1944528127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e542ea5ea0 0x55e542eae540 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.059+0000 7f5e7fd6b740 1 --2- 172.21.15.12:0/1944528127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e542ea5580 0x55e542ea5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.059+0000 7f5e7fd6b740 1 -- 172.21.15.12:0/1944528127 >> 172.21.15.12:0/1944528127 conn(0x55e542e9fcb0 msgr2=0x55e542ea0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:19.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.059+0000 7f5e7fd6b740 1 -- 172.21.15.12:0/1944528127 shutdown_connections 2024-04-22T13:25:19.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.059+0000 7f5e7fd6b740 1 -- 172.21.15.12:0/1944528127 wait complete. 2024-04-22T13:25:19.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.059+0000 7f5e7fd6b740 1 -- start start 2024-04-22T13:25:19.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.059+0000 7f5e7fd6b740 1 Processor -- start 2024-04-22T13:25:19.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.060+0000 7f5e7fd6b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e542ea5580 0x55e542eabac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:19.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.060+0000 7f5e7fd6b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e542ea5ea0 0x55e542eac000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:19.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.060+0000 7f5e7fd6b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e542eaea80 0x55e542eac540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:19.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.060+0000 7f5e7fd6b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e542eb1630 con 0x55e542ea5ea0 2024-04-22T13:25:19.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.060+0000 7f5e7fd6b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e542dc32f0 con 0x55e542ea5580 2024-04-22T13:25:19.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.060+0000 7f5e7fd6b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e542eb17b0 con 0x55e542eaea80 2024-04-22T13:25:19.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.060+0000 7f5e7fd6b740 10 client.0 _ll_register_callbacks cb 0x55e542dd0580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:19.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.060+0000 7f5e7659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e542ea5ea0 0x55e542eac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:19.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.060+0000 7f5e76d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e542ea5580 0x55e542eabac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:19.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.060+0000 7f5e7659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e542ea5ea0 0x55e542eac000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55878/0 (socket says 172.21.15.12:55878) 2024-04-22T13:25:19.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.060+0000 7f5e7659c640 1 -- 172.21.15.12:0/3452039054 learned_addr learned my addr 172.21.15.12:0/3452039054 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:19.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.060+0000 7f5e7759e640 1 --2- 172.21.15.12:0/3452039054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e542eaea80 0x55e542eac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:19.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.060+0000 7f5e76d9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e542ea5580 0x55e542eabac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:44974/0 (socket says 172.21.15.12:44974) 2024-04-22T13:25:19.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.060+0000 7f5e7659c640 1 -- 172.21.15.12:0/3452039054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e542eaea80 msgr2=0x55e542eac540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:19.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.060+0000 7f5e7659c640 1 --2- 172.21.15.12:0/3452039054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e542eaea80 0x55e542eac540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.060+0000 7f5e7659c640 1 -- 172.21.15.12:0/3452039054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e542ea5580 msgr2=0x55e542eabac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:19.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.060+0000 7f5e7659c640 1 --2- 172.21.15.12:0/3452039054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e542ea5580 0x55e542eabac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.060+0000 7f5e7659c640 1 -- 172.21.15.12:0/3452039054 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e542b1c3f0 con 0x55e542ea5ea0 2024-04-22T13:25:19.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.061+0000 7f5e7659c640 1 --2- 172.21.15.12:0/3452039054 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e542ea5ea0 0x55e542eac000 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c0c6f90 tx=0x7f5e6c005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=522109f825a82487 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:19.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.061+0000 7f5e5aff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:19.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.061+0000 7f5e5aff5640 1 -- 172.21.15.12:0/3452039054 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c0c2c00 con 0x55e542ea5ea0 2024-04-22T13:25:19.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.061+0000 7f5e5aff5640 1 -- 172.21.15.12:0/3452039054 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f5e6c0c7030 con 0x55e542ea5ea0 2024-04-22T13:25:19.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.061+0000 7f5e7fd6b740 1 -- 172.21.15.12:0/3452039054 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e6c0be850 con 0x55e542ea5ea0 2024-04-22T13:25:19.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.061+0000 7f5e5aff5640 1 -- 172.21.15.12:0/3452039054 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5e6c0c74c0 con 0x55e542ea5ea0 2024-04-22T13:25:19.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.061+0000 7f5e7fd6b740 10 client.16248 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:19.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.061+0000 7f5e7fd6b740 1 -- 172.21.15.12:0/3452039054 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e542adc0f0 con 0x55e542ea5ea0 2024-04-22T13:25:19.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.061+0000 7f5e5aff5640 1 -- 172.21.15.12:0/3452039054 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e6c0be850 con 0x55e542ea5ea0 2024-04-22T13:25:19.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.061+0000 7f5e5aff5640 1 -- 172.21.15.12:0/3452039054 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5e6c0d4070 con 0x55e542ea5ea0 2024-04-22T13:25:19.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.062+0000 7f5e7fd6b740 10 client.16248 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:19.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.062+0000 7f5e7fd6b740 10 client.16248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:25:19.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.062+0000 7f5e7fd6b740 20 client.16248 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:19.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.062+0000 7f5e7fd6b740 1 --2- 172.21.15.12:0/3452039054 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e542ebe5d0 0x55e542ec09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:19.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.062+0000 7f5e7fd6b740 4 client.16248 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:25:19.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.062+0000 7f5e7fd6b740 1 -- 172.21.15.12:0/3452039054 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e542e17c00 con 0x55e542ebe5d0 2024-04-22T13:25:19.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.062+0000 7f5e76d9d640 1 --2- 172.21.15.12:0/3452039054 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e542ebe5d0 0x55e542ec09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:19.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.062+0000 7f5e5aff5640 10 client.16248 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:19.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.062+0000 7f5e76d9d640 1 --2- 172.21.15.12:0/3452039054 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e542ebe5d0 0x55e542ec09c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8d755f91f40516a server_cookie=676dee4626b66492 in_seq=0 out_seq=0 2024-04-22T13:25:19.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.063+0000 7f5e5aff5640 1 -- 172.21.15.12:0/3452039054 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e542e17c00 con 0x55e542ebe5d0 2024-04-22T13:25:19.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.063+0000 7f5e5aff5640 10 client.16248 handle_command_reply: tid=0 2024-04-22T13:25:19.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.063+0000 7f5e7fd6b740 1 client.16248 shutdown 2024-04-22T13:25:19.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.063+0000 7f5e7fd6b740 1 -- 172.21.15.12:0/3452039054 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e542ea5ea0 msgr2=0x55e542eac000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:19.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.063+0000 7f5e7fd6b740 1 --2- 172.21.15.12:0/3452039054 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e542ea5ea0 0x55e542eac000 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c0c6f90 tx=0x7f5e6c005760 comp rx=0 tx=0).stop 2024-04-22T13:25:19.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.063+0000 7f5e7fd6b740 1 -- 172.21.15.12:0/3452039054 shutdown_connections 2024-04-22T13:25:19.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.063+0000 7f5e7fd6b740 1 --2- 172.21.15.12:0/3452039054 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e542ebe5d0 0x55e542ec09c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.063+0000 7f5e5aff5640 0 client.16248 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:19.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.063+0000 7f5e7fd6b740 1 --2- 172.21.15.12:0/3452039054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e542eaea80 0x55e542eac540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.063+0000 7f5e7fd6b740 1 --2- 172.21.15.12:0/3452039054 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e542ea5ea0 0x55e542eac000 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.063+0000 7f5e7fd6b740 1 --2- 172.21.15.12:0/3452039054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e542ea5580 0x55e542eabac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.063+0000 7f5e7fd6b740 1 -- 172.21.15.12:0/3452039054 >> 172.21.15.12:0/3452039054 conn(0x55e542e9fcb0 msgr2=0x55e542ea1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:19.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.063+0000 7f5e7fd6b740 1 -- 172.21.15.12:0/3452039054 shutdown_connections 2024-04-22T13:25:19.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.063+0000 7f5e7fd6b740 1 -- 172.21.15.12:0/3452039054 wait complete. 2024-04-22T13:25:19.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.064+0000 7f5e7fd6b740 20 client.16248 trim_cache size 0 max 16384 2024-04-22T13:25:19.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.085+0000 7f5e7fd6b740 1 Processor -- start 2024-04-22T13:25:19.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.085+0000 7f5e7fd6b740 1 -- start start 2024-04-22T13:25:19.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.085+0000 7f5e7fd6b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e542ea5580 0x55e542eae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:19.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.085+0000 7f5e7fd6b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e542ea5ea0 0x55e542eba440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:19.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.085+0000 7f5e7fd6b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e542eaea80 0x55e542ec62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:19.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.085+0000 7f5e7fd6b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5e781b01b0 con 0x55e542eaea80 2024-04-22T13:25:19.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.085+0000 7f5e7fd6b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5e781b1cf0 con 0x55e542ea5ea0 2024-04-22T13:25:19.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.085+0000 7f5e7fd6b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5e781b0880 con 0x55e542ea5580 2024-04-22T13:25:19.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.085+0000 7f5e7659c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e542ea5ea0 0x55e542eba440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:19.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.085+0000 7f5e7759e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e542eaea80 0x55e542ec62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:19.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.085+0000 7f5e7659c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e542ea5ea0 0x55e542eba440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:44988/0 (socket says 172.21.15.12:44988) 2024-04-22T13:25:19.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.085+0000 7f5e7659c640 1 -- 172.21.15.12:0/4103466886 learned_addr learned my addr 172.21.15.12:0/4103466886 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:19.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.086+0000 7f5e76d9d640 1 --2- 172.21.15.12:0/4103466886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e542ea5580 0x55e542eae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:19.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.086+0000 7f5e7659c640 1 -- 172.21.15.12:0/4103466886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e542ea5580 msgr2=0x55e542eae1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:19.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.086+0000 7f5e7659c640 1 --2- 172.21.15.12:0/4103466886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e542ea5580 0x55e542eae1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.086+0000 7f5e7659c640 1 -- 172.21.15.12:0/4103466886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e542eaea80 msgr2=0x55e542ec62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:19.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.086+0000 7f5e7659c640 1 --2- 172.21.15.12:0/4103466886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e542eaea80 0x55e542ec62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.086+0000 7f5e7659c640 1 -- 172.21.15.12:0/4103466886 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e542e9f4f0 con 0x55e542ea5ea0 2024-04-22T13:25:19.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.086+0000 7f5e7759e640 1 --2- 172.21.15.12:0/4103466886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e542eaea80 0x55e542ec62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:19.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.086+0000 7f5e7659c640 1 --2- 172.21.15.12:0/4103466886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e542ea5ea0 0x55e542eba440 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c006010 tx=0x7f5e6c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72264f6899b72b0a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:19.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.086+0000 7f5e5cff9640 1 -- 172.21.15.12:0/4103466886 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c0be1b0 con 0x55e542ea5ea0 2024-04-22T13:25:19.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.086+0000 7f5e5cff9640 1 -- 172.21.15.12:0/4103466886 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e6c0dc020 con 0x55e542ea5ea0 2024-04-22T13:25:19.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.086+0000 7f5e5cff9640 1 -- 172.21.15.12:0/4103466886 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c0c2c00 con 0x55e542ea5ea0 2024-04-22T13:25:19.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.087+0000 7f5e7fd6b740 1 -- 172.21.15.12:0/4103466886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e542ea5ea0 msgr2=0x55e542eba440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:19.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.087+0000 7f5e7fd6b740 1 --2- 172.21.15.12:0/4103466886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e542ea5ea0 0x55e542eba440 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c006010 tx=0x7f5e6c009260 comp rx=0 tx=0).stop 2024-04-22T13:25:19.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.087+0000 7f5e7fd6b740 1 -- 172.21.15.12:0/4103466886 shutdown_connections 2024-04-22T13:25:19.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.087+0000 7f5e7fd6b740 1 --2- 172.21.15.12:0/4103466886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e542eaea80 0x55e542ec62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.087+0000 7f5e7fd6b740 1 --2- 172.21.15.12:0/4103466886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e542ea5ea0 0x55e542eba440 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.087+0000 7f5e7fd6b740 1 --2- 172.21.15.12:0/4103466886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e542ea5580 0x55e542eae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.087+0000 7f5e7fd6b740 1 -- 172.21.15.12:0/4103466886 >> 172.21.15.12:0/4103466886 conn(0x55e542e9fcb0 msgr2=0x55e542ea6da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:19.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.087+0000 7f5e7fd6b740 1 -- 172.21.15.12:0/4103466886 shutdown_connections 2024-04-22T13:25:19.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.087+0000 7f5e7fd6b740 1 -- 172.21.15.12:0/4103466886 wait complete. 2024-04-22T13:25:19.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.087+0000 7f5e7fd6b740 1 -- start start 2024-04-22T13:25:19.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.088+0000 7f5e7fd6b740 1 Processor -- start 2024-04-22T13:25:19.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.088+0000 7f5e7fd6b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e542ea5580 0x55e542eb6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:19.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.088+0000 7f5e7fd6b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e542ea5ea0 0x55e542eb1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:19.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.088+0000 7f5e7fd6b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e542eaea80 0x55e542eb2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:19.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.088+0000 7f5e7fd6b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5e781b1620 con 0x55e542eaea80 2024-04-22T13:25:19.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.088+0000 7f5e7fd6b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5e781b0f50 con 0x55e542ea5ea0 2024-04-22T13:25:19.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.088+0000 7f5e7fd6b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5e781ae190 con 0x55e542ea5580 2024-04-22T13:25:19.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.088+0000 7f5e7fd6b740 10 client.0 _ll_register_callbacks cb 0x7f5e78068f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:19.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.088+0000 7f5e76d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e542ea5580 0x55e542eb6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:19.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.088+0000 7f5e7659c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e542ea5ea0 0x55e542eb1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:19.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.088+0000 7f5e7759e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e542eaea80 0x55e542eb2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:19.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.088+0000 7f5e7659c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e542ea5ea0 0x55e542eb1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:45004/0 (socket says 172.21.15.12:45004) 2024-04-22T13:25:19.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.088+0000 7f5e76d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e542ea5580 0x55e542eb6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.12:54582/0 (socket says 172.21.15.12:54582) 2024-04-22T13:25:19.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.088+0000 7f5e7659c640 1 -- 172.21.15.12:0/2627322618 learned_addr learned my addr 172.21.15.12:0/2627322618 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:19.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.088+0000 7f5e76d9d640 1 -- 172.21.15.12:0/2627322618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e542ea5ea0 msgr2=0x55e542eb1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:19.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.088+0000 7f5e76d9d640 1 --2- 172.21.15.12:0/2627322618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e542ea5ea0 0x55e542eb1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.088+0000 7f5e76d9d640 1 -- 172.21.15.12:0/2627322618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e542eaea80 msgr2=0x55e542eb2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:19.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.088+0000 7f5e76d9d640 1 --2- 172.21.15.12:0/2627322618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e542eaea80 0x55e542eb2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.088+0000 7f5e76d9d640 1 -- 172.21.15.12:0/2627322618 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e542eb2b00 con 0x55e542ea5580 2024-04-22T13:25:19.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.089+0000 7f5e7659c640 1 --2- 172.21.15.12:0/2627322618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e542ea5ea0 0x55e542eb1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:19.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.089+0000 7f5e76d9d640 1 --2- 172.21.15.12:0/2627322618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e542ea5580 0x55e542eb6bb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55e542ea0820 tx=0x7f5e68011ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49ec2877b6fa2b77 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:19.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.089+0000 7f5e5aff5640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-22T13:25:19.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.089+0000 7f5e5aff5640 1 -- 172.21.15.12:0/2627322618 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6801ddf0 con 0x55e542ea5580 2024-04-22T13:25:19.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.089+0000 7f5e7fd6b740 1 -- 172.21.15.12:0/2627322618 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e6c0dc020 con 0x55e542ea5580 2024-04-22T13:25:19.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.089+0000 7f5e5aff5640 1 -- 172.21.15.12:0/2627322618 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f5e68004020 con 0x55e542ea5580 2024-04-22T13:25:19.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.089+0000 7f5e5aff5640 1 -- 172.21.15.12:0/2627322618 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5e6802d020 con 0x55e542ea5580 2024-04-22T13:25:19.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.089+0000 7f5e7fd6b740 10 client.25624 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:19.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.089+0000 7f5e7fd6b740 1 -- 172.21.15.12:0/2627322618 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e542f24a40 con 0x55e542ea5580 2024-04-22T13:25:19.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.090+0000 7f5e5aff5640 1 -- 172.21.15.12:0/2627322618 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e6c0dc020 con 0x55e542ea5580 2024-04-22T13:25:19.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.090+0000 7f5e5aff5640 1 -- 172.21.15.12:0/2627322618 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5e68003610 con 0x55e542ea5580 2024-04-22T13:25:19.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.090+0000 7f5e7fd6b740 10 client.25624 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:19.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.090+0000 7f5e7fd6b740 10 client.25624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:25:19.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.090+0000 7f5e7fd6b740 20 client.25624 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:19.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.090+0000 7f5e7fd6b740 1 --2- 172.21.15.12:0/2627322618 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e542ebe5d0 0x55e542f27150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:19.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.090+0000 7f5e7fd6b740 4 client.25624 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:25:19.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.090+0000 7f5e7fd6b740 1 -- 172.21.15.12:0/2627322618 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e542e11a70 con 0x55e542ebe5d0 2024-04-22T13:25:19.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.090+0000 7f5e7659c640 1 --2- 172.21.15.12:0/2627322618 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e542ebe5d0 0x55e542f27150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:19.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.091+0000 7f5e5aff5640 10 client.25624 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:19.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.091+0000 7f5e7659c640 1 --2- 172.21.15.12:0/2627322618 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e542ebe5d0 0x55e542f27150 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5028df48163dca0 server_cookie=1c420f0078c5f648 in_seq=0 out_seq=0 2024-04-22T13:25:19.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.091+0000 7f5e5aff5640 1 -- 172.21.15.12:0/2627322618 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e542e11a70 con 0x55e542ebe5d0 2024-04-22T13:25:19.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.091+0000 7f5e5aff5640 10 client.25624 handle_command_reply: tid=0 2024-04-22T13:25:19.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.091+0000 7f5e7fd6b740 1 client.25624 shutdown 2024-04-22T13:25:19.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.091+0000 7f5e7fd6b740 1 -- 172.21.15.12:0/2627322618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e542ea5580 msgr2=0x55e542eb6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:19.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.091+0000 7f5e7fd6b740 1 --2- 172.21.15.12:0/2627322618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e542ea5580 0x55e542eb6bb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55e542ea0820 tx=0x7f5e68011ea0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.091+0000 7f5e7fd6b740 1 -- 172.21.15.12:0/2627322618 shutdown_connections 2024-04-22T13:25:19.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.091+0000 7f5e7fd6b740 1 --2- 172.21.15.12:0/2627322618 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e542ebe5d0 0x55e542f27150 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.091+0000 7f5e7fd6b740 1 --2- 172.21.15.12:0/2627322618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e542eaea80 0x55e542eb2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.091+0000 7f5e7fd6b740 1 --2- 172.21.15.12:0/2627322618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e542ea5ea0 0x55e542eb1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.091+0000 7f5e7fd6b740 1 --2- 172.21.15.12:0/2627322618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e542ea5580 0x55e542eb6bb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.091+0000 7f5e7fd6b740 1 -- 172.21.15.12:0/2627322618 >> 172.21.15.12:0/2627322618 conn(0x55e542e9fcb0 msgr2=0x55e542eaba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:19.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.091+0000 7f5e5aff5640 0 client.25624 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:19.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.091+0000 7f5e7fd6b740 1 -- 172.21.15.12:0/2627322618 shutdown_connections 2024-04-22T13:25:19.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.091+0000 7f5e7fd6b740 1 -- 172.21.15.12:0/2627322618 wait complete. 2024-04-22T13:25:19.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.091+0000 7f5e7fd6b740 20 client.25624 trim_cache size 0 max 16384 2024-04-22T13:25:19.094 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:25:19.095 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:25:19.095 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-04-22T13:25:19.095 INFO:teuthology.orchestra.run.smithi012.stdout: "67a103cb-b619-407e-b1c9-e7afdd5ab456": { 2024-04-22T13:25:19.095 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-04-22T13:25:19.095 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "67a103cb-b619-407e-b1c9-e7afdd5ab456", 2024-04-22T13:25:19.095 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-04-22T13:25:19.095 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:25:19.095 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:25:19.095 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:25:19.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.092+0000 7f5e7d17e640 1 -- 172.21.15.12:0/2518256812 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5e600a3f80 msgr2=0x7f5e600a6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:19.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.092+0000 7f5e7d17e640 1 --2- 172.21.15.12:0/2518256812 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5e600a3f80 0x7f5e600a6440 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5e70009fd0 tx=0x7f5e70009f20 comp rx=0 tx=0).stop 2024-04-22T13:25:19.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.092+0000 7f5e7d17e640 1 -- 172.21.15.12:0/2518256812 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e7810c500 msgr2=0x7f5e78101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:19.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.092+0000 7f5e7d17e640 1 --2- 172.21.15.12:0/2518256812 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e7810c500 0x7f5e78101350 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c035000 tx=0x7f5e6c0045e0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.092+0000 7f5e7d17e640 1 -- 172.21.15.12:0/2518256812 shutdown_connections 2024-04-22T13:25:19.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.092+0000 7f5e7d17e640 1 --2- 172.21.15.12:0/2518256812 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5e600a3f80 0x7f5e600a6440 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.092+0000 7f5e7d17e640 1 --2- 172.21.15.12:0/2518256812 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e78101890 0x7f5e781a6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.092+0000 7f5e7d17e640 1 --2- 172.21.15.12:0/2518256812 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e7810c500 0x7f5e78101350 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.093+0000 7f5e7d17e640 1 --2- 172.21.15.12:0/2518256812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e781031c0 0x7f5e78100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:19.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.093+0000 7f5e7d17e640 1 -- 172.21.15.12:0/2518256812 >> 172.21.15.12:0/2518256812 conn(0x7f5e78077fd0 msgr2=0x7f5e780fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:19.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.093+0000 7f5e7d17e640 1 -- 172.21.15.12:0/2518256812 shutdown_connections 2024-04-22T13:25:19.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:19.093+0000 7f5e7d17e640 1 -- 172.21.15.12:0/2518256812 wait complete. 2024-04-22T13:25:19.103 INFO:tasks.cephfs.filesystem:scrub status for tag:67a103cb-b619-407e-b1c9-e7afdd5ab456 - {'path': '/', 'tag': '67a103cb-b619-407e-b1c9-e7afdd5ab456', 'options': 'recursive,force'} 2024-04-22T13:25:19.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:19 smithi134 ceph-mon[31338]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-22T13:25:19.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:19 smithi111 ceph-mon[31361]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-22T13:25:20.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:19 smithi012 ceph-mon[28265]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-22T13:25:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.639+0000 7f78ef6d2640 1 -- 172.21.15.12:0/3103787326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78e8101aa0 msgr2=0x7f78e8105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.639+0000 7f78ef6d2640 1 --2- 172.21.15.12:0/3103787326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78e8101aa0 0x7f78e8105a90 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f78d8009900 tx=0x7f78d802bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.639+0000 7f78ef6d2640 1 -- 172.21.15.12:0/3103787326 shutdown_connections 2024-04-22T13:25:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.639+0000 7f78ef6d2640 1 --2- 172.21.15.12:0/3103787326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78e8106170 0x7f78e810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.639+0000 7f78ef6d2640 1 --2- 172.21.15.12:0/3103787326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78e8101aa0 0x7f78e8105a90 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.639+0000 7f78ef6d2640 1 --2- 172.21.15.12:0/3103787326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78e81010f0 0x7f78e81014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.639+0000 7f78ef6d2640 1 -- 172.21.15.12:0/3103787326 >> 172.21.15.12:0/3103787326 conn(0x7f78e8077fd0 msgr2=0x7f78e80783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.639+0000 7f78ef6d2640 1 -- 172.21.15.12:0/3103787326 shutdown_connections 2024-04-22T13:25:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.639+0000 7f78ef6d2640 1 -- 172.21.15.12:0/3103787326 wait complete. 2024-04-22T13:25:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.640+0000 7f78ef6d2640 1 Processor -- start 2024-04-22T13:25:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.640+0000 7f78ef6d2640 1 -- start start 2024-04-22T13:25:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.640+0000 7f78ef6d2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78e81010f0 0x7f78e81975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:20.638 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.640+0000 7f78ef6d2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78e8106170 0x7f78e8197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.640+0000 7f78ef6d2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78e819be70 0x7f78e819c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.640+0000 7f78ef6d2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f78e81110b0 con 0x7f78e819be70 2024-04-22T13:25:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.640+0000 7f78ef6d2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f78e8110f30 con 0x7f78e8106170 2024-04-22T13:25:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.640+0000 7f78ef6d2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f78e8111230 con 0x7f78e81010f0 2024-04-22T13:25:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.640+0000 7f78edc6f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78e819be70 0x7f78e819c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.640+0000 7f78edc6f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78e819be70 0x7f78e819c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55904/0 (socket says 172.21.15.12:55904) 2024-04-22T13:25:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.640+0000 7f78ed46e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78e81010f0 0x7f78e81975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.640+0000 7f78edc6f640 1 -- 172.21.15.12:0/2850750742 learned_addr learned my addr 172.21.15.12:0/2850750742 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.640+0000 7f78ecc6d640 1 --2- 172.21.15.12:0/2850750742 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78e8106170 0x7f78e8197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.641+0000 7f78ed46e640 1 -- 172.21.15.12:0/2850750742 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78e8106170 msgr2=0x7f78e8197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.641+0000 7f78ed46e640 1 --2- 172.21.15.12:0/2850750742 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78e8106170 0x7f78e8197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.641+0000 7f78ed46e640 1 -- 172.21.15.12:0/2850750742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78e819be70 msgr2=0x7f78e819c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.641+0000 7f78ed46e640 1 --2- 172.21.15.12:0/2850750742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78e819be70 0x7f78e819c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.639 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.641+0000 7f78ed46e640 1 -- 172.21.15.12:0/2850750742 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e819c960 con 0x7f78e81010f0 2024-04-22T13:25:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.641+0000 7f78edc6f640 1 --2- 172.21.15.12:0/2850750742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78e819be70 0x7f78e819c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.641+0000 7f78ed46e640 1 --2- 172.21.15.12:0/2850750742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78e81010f0 0x7f78e81975b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f78e400d720 tx=0x7f78e400dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.641+0000 7f78de7fc640 1 -- 172.21.15.12:0/2850750742 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e4017070 con 0x7f78e81010f0 2024-04-22T13:25:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.641+0000 7f78ef6d2640 1 -- 172.21.15.12:0/2850750742 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78e819cc50 con 0x7f78e81010f0 2024-04-22T13:25:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.641+0000 7f78de7fc640 1 -- 172.21.15.12:0/2850750742 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78e400de30 con 0x7f78e81010f0 2024-04-22T13:25:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.641+0000 7f78ef6d2640 1 -- 172.21.15.12:0/2850750742 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78e80698f0 con 0x7f78e81010f0 2024-04-22T13:25:20.640 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.641+0000 7f78de7fc640 1 -- 172.21.15.12:0/2850750742 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e40047a0 con 0x7f78e81010f0 2024-04-22T13:25:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.644+0000 7f78f23a6740 1 Processor -- start 2024-04-22T13:25:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.644+0000 7f78f23a6740 1 -- start start 2024-04-22T13:25:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.644+0000 7f78f23a6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423062d8d0 0x56423062c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.644+0000 7f78f23a6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56423062cb60 0x56423062cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.644+0000 7f78f23a6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56423063a4f0 0x56423063c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.644+0000 7f78f23a6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5642305918d0 con 0x56423063a4f0 2024-04-22T13:25:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.644+0000 7f78f23a6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56423052ee80 con 0x56423062cb60 2024-04-22T13:25:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.644+0000 7f78f23a6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56423058d890 con 0x56423062d8d0 2024-04-22T13:25:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.644+0000 7f78ecc6d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56423063a4f0 0x56423063c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.644+0000 7f78edc6f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423062d8d0 0x56423062c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.644+0000 7f78ecc6d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56423063a4f0 0x56423063c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55914/0 (socket says 172.21.15.12:55914) 2024-04-22T13:25:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.644+0000 7f78ecc6d640 1 -- 172.21.15.12:0/410776892 learned_addr learned my addr 172.21.15.12:0/410776892 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.644+0000 7f78ecc6d640 1 -- 172.21.15.12:0/410776892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423062d8d0 msgr2=0x56423062c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.644+0000 7f78ecc6d640 1 --2- 172.21.15.12:0/410776892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423062d8d0 0x56423062c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.644+0000 7f78ecc6d640 1 -- 172.21.15.12:0/410776892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56423062cb60 msgr2=0x56423062cfe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T13:25:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.644+0000 7f78ecc6d640 1 --2- 172.21.15.12:0/410776892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56423062cb60 0x56423062cfe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.644+0000 7f78ecc6d640 1 -- 172.21.15.12:0/410776892 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564230550320 con 0x56423063a4f0 2024-04-22T13:25:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.645+0000 7f78edc6f640 1 --2- 172.21.15.12:0/410776892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423062d8d0 0x56423062c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.645+0000 7f78ecc6d640 1 --2- 172.21.15.12:0/410776892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56423063a4f0 0x56423063c8e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f78d8009900 tx=0x7f78d802c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3742f4c33cd4b6a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.645+0000 7f78d37fe640 1 -- 172.21.15.12:0/410776892 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d8045020 con 0x56423063a4f0 2024-04-22T13:25:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.645+0000 7f78de7fc640 1 -- 172.21.15.12:0/2850750742 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f78e40099d0 con 0x7f78e81010f0 2024-04-22T13:25:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.645+0000 7f78d37fe640 1 -- 172.21.15.12:0/410776892 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78d802c7e0 con 0x56423063a4f0 2024-04-22T13:25:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.645+0000 7f78d37fe640 1 -- 172.21.15.12:0/410776892 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d80046c0 con 0x56423063a4f0 2024-04-22T13:25:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.646+0000 7f78f23a6740 1 -- 172.21.15.12:0/410776892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56423063a4f0 msgr2=0x56423063c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.646+0000 7f78f23a6740 1 --2- 172.21.15.12:0/410776892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56423063a4f0 0x56423063c8e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f78d8009900 tx=0x7f78d802c610 comp rx=0 tx=0).stop 2024-04-22T13:25:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.646+0000 7f78de7fc640 1 --2- 172.21.15.12:0/2850750742 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f78d40a3f70 0x7f78d40a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.646+0000 7f78f23a6740 1 -- 172.21.15.12:0/410776892 shutdown_connections 2024-04-22T13:25:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.646+0000 7f78f23a6740 1 --2- 172.21.15.12:0/410776892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56423063a4f0 0x56423063c8e0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.646+0000 7f78f23a6740 1 --2- 172.21.15.12:0/410776892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56423062cb60 0x56423062cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.646+0000 7f78f23a6740 1 --2- 172.21.15.12:0/410776892 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423062d8d0 0x56423062c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.646+0000 7f78f23a6740 1 -- 172.21.15.12:0/410776892 >> 172.21.15.12:0/410776892 conn(0x5642305f3c90 msgr2=0x564230627850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.646+0000 7f78f23a6740 1 -- 172.21.15.12:0/410776892 shutdown_connections 2024-04-22T13:25:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.646+0000 7f78de7fc640 1 -- 172.21.15.12:0/2850750742 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f78e40d0660 con 0x7f78e81010f0 2024-04-22T13:25:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.646+0000 7f78edc6f640 1 --2- 172.21.15.12:0/2850750742 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f78d40a3f70 0x7f78d40a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.646+0000 7f78f23a6740 1 -- 172.21.15.12:0/410776892 wait complete. 2024-04-22T13:25:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.647+0000 7f78f23a6740 1 -- start start 2024-04-22T13:25:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.647+0000 7f78edc6f640 1 --2- 172.21.15.12:0/2850750742 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f78d40a3f70 0x7f78d40a6430 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f78e0005c40 tx=0x7f78e0009760 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.647+0000 7f78f23a6740 1 Processor -- start 2024-04-22T13:25:20.649 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.648+0000 7f78f23a6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56423062cb60 0x564230639eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.648+0000 7f78f23a6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56423062d8d0 0x564230637240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.648+0000 7f78f23a6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423063a4f0 0x564230637850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.648+0000 7f78f23a6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56423063d1c0 con 0x56423062cb60 2024-04-22T13:25:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.648+0000 7f78f23a6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564230519eb0 con 0x56423062d8d0 2024-04-22T13:25:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.648+0000 7f78f23a6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56423063d340 con 0x56423063a4f0 2024-04-22T13:25:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.648+0000 7f78f23a6740 10 client.0 _ll_register_callbacks cb 0x564230546b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.648+0000 7f78ed46e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56423062cb60 0x564230639eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.648+0000 7f78ed46e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56423062cb60 0x564230639eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55918/0 (socket says 172.21.15.12:55918) 2024-04-22T13:25:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.648+0000 7f78ed46e640 1 -- 172.21.15.12:0/2548794226 learned_addr learned my addr 172.21.15.12:0/2548794226 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.648+0000 7f78edc6f640 1 --2- 172.21.15.12:0/2548794226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423063a4f0 0x564230637850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.648+0000 7f78ecc6d640 1 --2- 172.21.15.12:0/2548794226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56423062d8d0 0x564230637240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.648+0000 7f78ed46e640 1 -- 172.21.15.12:0/2548794226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423063a4f0 msgr2=0x564230637850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:20.650 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.648+0000 7f78ed46e640 1 --2- 172.21.15.12:0/2548794226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423063a4f0 0x564230637850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.648+0000 7f78ed46e640 1 -- 172.21.15.12:0/2548794226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56423062d8d0 msgr2=0x564230637240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.648+0000 7f78ed46e640 1 --2- 172.21.15.12:0/2548794226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56423062d8d0 0x564230637240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.648+0000 7f78ed46e640 1 -- 172.21.15.12:0/2548794226 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642302a4210 con 0x56423062cb60 2024-04-22T13:25:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.649+0000 7f78ed46e640 1 --2- 172.21.15.12:0/2548794226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56423062cb60 0x564230639eb0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f78e409c110 tx=0x7f78e409c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480add35e257e41b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.649+0000 7f78d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.649+0000 7f78d0ff9640 1 -- 172.21.15.12:0/2548794226 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e409c410 con 0x56423062cb60 2024-04-22T13:25:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.649+0000 7f78f23a6740 1 -- 172.21.15.12:0/2548794226 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78d802c7e0 con 0x56423062cb60 2024-04-22T13:25:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.649+0000 7f78d0ff9640 1 -- 172.21.15.12:0/2548794226 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f78e4098a80 con 0x56423062cb60 2024-04-22T13:25:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.649+0000 7f78d0ff9640 1 -- 172.21.15.12:0/2548794226 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f78e40a07c0 con 0x56423062cb60 2024-04-22T13:25:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.649+0000 7f78f23a6740 10 client.16266 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.649+0000 7f78f23a6740 1 -- 172.21.15.12:0/2548794226 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564230370620 con 0x56423062cb60 2024-04-22T13:25:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.649+0000 7f78d0ff9640 1 -- 172.21.15.12:0/2548794226 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78d802c7e0 con 0x56423062cb60 2024-04-22T13:25:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.649+0000 7f78d0ff9640 1 -- 172.21.15.12:0/2548794226 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f78e40cded0 con 0x56423062cb60 2024-04-22T13:25:20.651 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.650+0000 7f78f23a6740 10 client.16266 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.650+0000 7f78f23a6740 10 client.16266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:25:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.650+0000 7f78f23a6740 20 client.16266 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.650+0000 7f78f23a6740 1 --2- 172.21.15.12:0/2548794226 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56423064a520 0x56423064c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.650+0000 7f78f23a6740 4 client.16266 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:25:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.650+0000 7f78f23a6740 1 -- 172.21.15.12:0/2548794226 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642305a5880 con 0x56423064a520 2024-04-22T13:25:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.650+0000 7f78ecc6d640 1 --2- 172.21.15.12:0/2548794226 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56423064a520 0x56423064c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.650+0000 7f78d0ff9640 10 client.16266 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.650+0000 7f78ecc6d640 1 --2- 172.21.15.12:0/2548794226 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56423064a520 0x56423064c910 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c9adb94c3228021 server_cookie=c5247bada7d8e35c in_seq=0 out_seq=0 2024-04-22T13:25:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.651+0000 7f78d0ff9640 1 -- 172.21.15.12:0/2548794226 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5642305a5880 con 0x56423064a520 2024-04-22T13:25:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.651+0000 7f78d0ff9640 10 client.16266 handle_command_reply: tid=0 2024-04-22T13:25:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.651+0000 7f78f23a6740 1 client.16266 shutdown 2024-04-22T13:25:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.651+0000 7f78f23a6740 1 -- 172.21.15.12:0/2548794226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56423062cb60 msgr2=0x564230639eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:20.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.651+0000 7f78f23a6740 1 --2- 172.21.15.12:0/2548794226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56423062cb60 0x564230639eb0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f78e409c110 tx=0x7f78e409c040 comp rx=0 tx=0).stop 2024-04-22T13:25:20.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.651+0000 7f78f23a6740 1 -- 172.21.15.12:0/2548794226 shutdown_connections 2024-04-22T13:25:20.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.651+0000 7f78f23a6740 1 --2- 172.21.15.12:0/2548794226 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56423064a520 0x56423064c910 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.651+0000 7f78f23a6740 1 --2- 172.21.15.12:0/2548794226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423063a4f0 0x564230637850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.651+0000 7f78d0ff9640 0 client.16266 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:20.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.651+0000 7f78f23a6740 1 --2- 172.21.15.12:0/2548794226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56423062d8d0 0x564230637240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.651+0000 7f78f23a6740 1 --2- 172.21.15.12:0/2548794226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56423062cb60 0x564230639eb0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.651+0000 7f78f23a6740 1 -- 172.21.15.12:0/2548794226 >> 172.21.15.12:0/2548794226 conn(0x5642305f3c90 msgr2=0x5642306295a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:20.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.652+0000 7f78f23a6740 1 -- 172.21.15.12:0/2548794226 shutdown_connections 2024-04-22T13:25:20.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.652+0000 7f78f23a6740 1 -- 172.21.15.12:0/2548794226 wait complete. 2024-04-22T13:25:20.653 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.652+0000 7f78f23a6740 20 client.16266 trim_cache size 0 max 16384 2024-04-22T13:25:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.678+0000 7f78f23a6740 1 Processor -- start 2024-04-22T13:25:20.675 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.678+0000 7f78f23a6740 1 -- start start 2024-04-22T13:25:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.679+0000 7f78f23a6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56423062cb60 0x5642306460c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.679+0000 7f78f23a6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56423062d8d0 0x564230636220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.679+0000 7f78f23a6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423063a4f0 0x564230639970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.679+0000 7f78f23a6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f78e81ab6c0 con 0x56423062d8d0 2024-04-22T13:25:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.679+0000 7f78f23a6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f78e81ad200 con 0x56423062cb60 2024-04-22T13:25:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.679+0000 7f78f23a6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f78e81abd90 con 0x56423063a4f0 2024-04-22T13:25:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.679+0000 7f78ecc6d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56423062d8d0 0x564230636220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.679+0000 7f78ecc6d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56423062d8d0 0x564230636220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55928/0 (socket says 172.21.15.12:55928) 2024-04-22T13:25:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.679+0000 7f78ecc6d640 1 -- 172.21.15.12:0/3939257446 learned_addr learned my addr 172.21.15.12:0/3939257446 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.679+0000 7f78edc6f640 1 --2- 172.21.15.12:0/3939257446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423063a4f0 0x564230639970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.679+0000 7f78ed46e640 1 --2- 172.21.15.12:0/3939257446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56423062cb60 0x5642306460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.679+0000 7f78ecc6d640 1 -- 172.21.15.12:0/3939257446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423063a4f0 msgr2=0x564230639970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.679+0000 7f78ecc6d640 1 --2- 172.21.15.12:0/3939257446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423063a4f0 0x564230639970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.679+0000 7f78ecc6d640 1 -- 172.21.15.12:0/3939257446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56423062cb60 msgr2=0x5642306460c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.679+0000 7f78ecc6d640 1 --2- 172.21.15.12:0/3939257446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56423062cb60 0x5642306460c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.679+0000 7f78ecc6d640 1 -- 172.21.15.12:0/3939257446 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56423062be30 con 0x56423062d8d0 2024-04-22T13:25:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.679+0000 7f78ed46e640 1 --2- 172.21.15.12:0/3939257446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56423062cb60 0x5642306460c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.680+0000 7f78ecc6d640 1 --2- 172.21.15.12:0/3939257446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56423062d8d0 0x564230636220 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f78d80096c0 tx=0x7f78d8005340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7ab61cfcac48e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:20.677 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.680+0000 7f78d2ffd640 1 -- 172.21.15.12:0/3939257446 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d8050070 con 0x56423062d8d0 2024-04-22T13:25:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.684+0000 7f78d2ffd640 1 -- 172.21.15.12:0/3939257446 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78d802c7e0 con 0x56423062d8d0 2024-04-22T13:25:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.684+0000 7f78d2ffd640 1 -- 172.21.15.12:0/3939257446 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d8045040 con 0x56423062d8d0 2024-04-22T13:25:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.684+0000 7f78f23a6740 1 -- 172.21.15.12:0/3939257446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56423062d8d0 msgr2=0x564230636220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.684+0000 7f78f23a6740 1 --2- 172.21.15.12:0/3939257446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56423062d8d0 0x564230636220 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f78d80096c0 tx=0x7f78d8005340 comp rx=0 tx=0).stop 2024-04-22T13:25:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.685+0000 7f78f23a6740 1 -- 172.21.15.12:0/3939257446 shutdown_connections 2024-04-22T13:25:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.685+0000 7f78f23a6740 1 --2- 172.21.15.12:0/3939257446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423063a4f0 0x564230639970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.685+0000 7f78f23a6740 1 --2- 172.21.15.12:0/3939257446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56423062d8d0 0x564230636220 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.685+0000 7f78f23a6740 1 --2- 172.21.15.12:0/3939257446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56423062cb60 0x5642306460c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.685+0000 7f78f23a6740 1 -- 172.21.15.12:0/3939257446 >> 172.21.15.12:0/3939257446 conn(0x5642305f3c90 msgr2=0x564230635b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.685+0000 7f78f23a6740 1 -- 172.21.15.12:0/3939257446 shutdown_connections 2024-04-22T13:25:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.685+0000 7f78f23a6740 1 -- 172.21.15.12:0/3939257446 wait complete. 2024-04-22T13:25:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.685+0000 7f78f23a6740 1 -- start start 2024-04-22T13:25:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.685+0000 7f78f23a6740 1 Processor -- start 2024-04-22T13:25:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.686+0000 7f78f23a6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56423062cb60 0x5642306340c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.686+0000 7f78f23a6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56423062d8d0 0x564230634600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.686+0000 7f78f23a6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423063a4f0 0x564230634b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.686+0000 7f78f23a6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f78e81acb30 con 0x56423062cb60 2024-04-22T13:25:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.686+0000 7f78ed46e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56423062cb60 0x5642306340c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.686+0000 7f78f23a6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f78e81ac460 con 0x56423062d8d0 2024-04-22T13:25:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.686+0000 7f78ed46e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56423062cb60 0x5642306340c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55932/0 (socket says 172.21.15.12:55932) 2024-04-22T13:25:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.686+0000 7f78ed46e640 1 -- 172.21.15.12:0/3016734455 learned_addr learned my addr 172.21.15.12:0/3016734455 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.686+0000 7f78f23a6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f78e81a96a0 con 0x56423063a4f0 2024-04-22T13:25:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.686+0000 7f78ecc6d640 1 --2- 172.21.15.12:0/3016734455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56423062d8d0 0x564230634600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.686+0000 7f78edc6f640 1 --2- 172.21.15.12:0/3016734455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423063a4f0 0x564230634b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.686+0000 7f78f23a6740 10 client.0 _ll_register_callbacks cb 0x7f78e4005750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.686+0000 7f78ed46e640 1 -- 172.21.15.12:0/3016734455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423063a4f0 msgr2=0x564230634b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.686+0000 7f78ed46e640 1 --2- 172.21.15.12:0/3016734455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423063a4f0 0x564230634b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.686+0000 7f78ed46e640 1 -- 172.21.15.12:0/3016734455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56423062d8d0 msgr2=0x564230634600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.686+0000 7f78ed46e640 1 --2- 172.21.15.12:0/3016734455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56423062d8d0 0x564230634600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.686+0000 7f78ed46e640 1 -- 172.21.15.12:0/3016734455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642306afbb0 con 0x56423062cb60 2024-04-22T13:25:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.686+0000 7f78edc6f640 1 --2- 172.21.15.12:0/3016734455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423063a4f0 0x564230634b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.686+0000 7f78ecc6d640 1 --2- 172.21.15.12:0/3016734455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56423062d8d0 0x564230634600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.687+0000 7f78ed46e640 1 --2- 172.21.15.12:0/3016734455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56423062cb60 0x5642306340c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f78e409a6c0 tx=0x7f78e409cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59efd3d6b8bba400 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.687+0000 7f78d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.688+0000 7f78d0ff9640 1 -- 172.21.15.12:0/3016734455 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e409c1e0 con 0x56423062cb60 2024-04-22T13:25:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.688+0000 7f78d0ff9640 1 -- 172.21.15.12:0/3016734455 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f78e4098a80 con 0x56423062cb60 2024-04-22T13:25:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.688+0000 7f78f23a6740 1 -- 172.21.15.12:0/3016734455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78d802c7e0 con 0x56423062cb60 2024-04-22T13:25:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.688+0000 7f78d0ff9640 1 -- 172.21.15.12:0/3016734455 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f78e409c410 con 0x56423062cb60 2024-04-22T13:25:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.688+0000 7f78f23a6740 10 client.16272 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.688+0000 7f78f23a6740 1 -- 172.21.15.12:0/3016734455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5642306b0e80 con 0x56423062cb60 2024-04-22T13:25:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.688+0000 7f78d0ff9640 1 -- 172.21.15.12:0/3016734455 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78d802c7e0 con 0x56423062cb60 2024-04-22T13:25:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.688+0000 7f78d0ff9640 1 -- 172.21.15.12:0/3016734455 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f78e409b2d0 con 0x56423062cb60 2024-04-22T13:25:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.688+0000 7f78f23a6740 10 client.16272 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.688+0000 7f78f23a6740 10 client.16272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:25:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.688+0000 7f78f23a6740 20 client.16272 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.688+0000 7f78f23a6740 1 --2- 172.21.15.12:0/3016734455 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56423064a520 0x5642306b3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.688+0000 7f78f23a6740 4 client.16272 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-22T13:25:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.688+0000 7f78f23a6740 1 -- 172.21.15.12:0/3016734455 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5642305313d0 con 0x56423064a520 2024-04-22T13:25:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.689+0000 7f78ecc6d640 1 --2- 172.21.15.12:0/3016734455 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56423064a520 0x5642306b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.689+0000 7f78d0ff9640 10 client.16272 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:20.688 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.689+0000 7f78ecc6d640 1 --2- 172.21.15.12:0/3016734455 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56423064a520 0x5642306b3480 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94f2c844caab9746 server_cookie=10988974da6eed7f in_seq=0 out_seq=0 2024-04-22T13:25:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.693+0000 7f78d0ff9640 1 -- 172.21.15.12:0/3016734455 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5642305313d0 con 0x56423064a520 2024-04-22T13:25:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.693+0000 7f78d0ff9640 10 client.16272 handle_command_reply: tid=0 2024-04-22T13:25:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.693+0000 7f78f23a6740 1 client.16272 shutdown 2024-04-22T13:25:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.693+0000 7f78f23a6740 1 -- 172.21.15.12:0/3016734455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56423062cb60 msgr2=0x5642306340c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.693+0000 7f78f23a6740 1 --2- 172.21.15.12:0/3016734455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56423062cb60 0x5642306340c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f78e409a6c0 tx=0x7f78e409cc40 comp rx=0 tx=0).stop 2024-04-22T13:25:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.693+0000 7f78f23a6740 1 -- 172.21.15.12:0/3016734455 shutdown_connections 2024-04-22T13:25:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.693+0000 7f78f23a6740 1 --2- 172.21.15.12:0/3016734455 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56423064a520 0x5642306b3480 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.693+0000 7f78f23a6740 1 --2- 172.21.15.12:0/3016734455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56423063a4f0 0x564230634b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.693+0000 7f78f23a6740 1 --2- 172.21.15.12:0/3016734455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56423062d8d0 0x564230634600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.690 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.693+0000 7f78f23a6740 1 --2- 172.21.15.12:0/3016734455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56423062cb60 0x5642306340c0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.693+0000 7f78f23a6740 1 -- 172.21.15.12:0/3016734455 >> 172.21.15.12:0/3016734455 conn(0x5642305f3c90 msgr2=0x5642306363a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.693+0000 7f78d0ff9640 0 client.16272 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.693+0000 7f78f23a6740 1 -- 172.21.15.12:0/3016734455 shutdown_connections 2024-04-22T13:25:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.693+0000 7f78f23a6740 1 -- 172.21.15.12:0/3016734455 wait complete. 2024-04-22T13:25:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.693+0000 7f78f23a6740 20 client.16272 trim_cache size 0 max 16384 2024-04-22T13:25:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-04-22T13:25:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-04-22T13:25:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-04-22T13:25:20.691 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-04-22T13:25:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.694+0000 7f78ef6d2640 1 -- 172.21.15.12:0/2850750742 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f78d40a3f70 msgr2=0x7f78d40a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.694+0000 7f78ef6d2640 1 --2- 172.21.15.12:0/2850750742 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f78d40a3f70 0x7f78d40a6430 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f78e0005c40 tx=0x7f78e0009760 comp rx=0 tx=0).stop 2024-04-22T13:25:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.694+0000 7f78ef6d2640 1 -- 172.21.15.12:0/2850750742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78e81010f0 msgr2=0x7f78e81975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.694+0000 7f78ef6d2640 1 --2- 172.21.15.12:0/2850750742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78e81010f0 0x7f78e81975b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f78e400d720 tx=0x7f78e400dbe0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.694+0000 7f78ef6d2640 1 -- 172.21.15.12:0/2850750742 shutdown_connections 2024-04-22T13:25:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.694+0000 7f78ef6d2640 1 --2- 172.21.15.12:0/2850750742 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f78d40a3f70 0x7f78d40a6430 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.694+0000 7f78ef6d2640 1 --2- 172.21.15.12:0/2850750742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f78e819be70 0x7f78e819c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.694+0000 7f78ef6d2640 1 --2- 172.21.15.12:0/2850750742 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78e8106170 0x7f78e8197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.694+0000 7f78ef6d2640 1 --2- 172.21.15.12:0/2850750742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78e81010f0 0x7f78e81975b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.694+0000 7f78ef6d2640 1 -- 172.21.15.12:0/2850750742 >> 172.21.15.12:0/2850750742 conn(0x7f78e8077fd0 msgr2=0x7f78e80ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.694+0000 7f78ef6d2640 1 -- 172.21.15.12:0/2850750742 shutdown_connections 2024-04-22T13:25:20.693 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:20.694+0000 7f78ef6d2640 1 -- 172.21.15.12:0/2850750742 wait complete. 2024-04-22T13:25:20.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:20 smithi111 ceph-mon[31361]: scrub queued for path: / 2024-04-22T13:25:20.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:20 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:25:20.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:20 smithi111 ceph-mon[31361]: scrub summary: active paths [/] 2024-04-22T13:25:20.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:20 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:25:20.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:20 smithi111 ceph-mon[31361]: scrub summary: idle 2024-04-22T13:25:21.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:20 smithi012 ceph-mon[28265]: scrub queued for path: / 2024-04-22T13:25:21.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:20 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:25:21.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:20 smithi012 ceph-mon[28265]: scrub summary: active paths [/] 2024-04-22T13:25:21.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:20 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:25:21.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:20 smithi012 ceph-mon[28265]: scrub summary: idle 2024-04-22T13:25:21.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:20 smithi134 ceph-mon[31338]: scrub queued for path: / 2024-04-22T13:25:21.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:20 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:25:21.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:20 smithi134 ceph-mon[31338]: scrub summary: active paths [/] 2024-04-22T13:25:21.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:20 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:25:21.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:20 smithi134 ceph-mon[31338]: scrub summary: idle 2024-04-22T13:25:21.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:21 smithi111 ceph-mon[31361]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-22T13:25:21.988 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:21 smithi012 ceph-mon[28265]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-22T13:25:22.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:21 smithi134 ceph-mon[31338]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-22T13:25:23.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:23 smithi111 ceph-mon[31361]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 767 B/s wr, 14 op/s 2024-04-22T13:25:24.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:23 smithi012 ceph-mon[28265]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 767 B/s wr, 14 op/s 2024-04-22T13:25:24.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:23 smithi134 ceph-mon[31338]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 767 B/s wr, 14 op/s 2024-04-22T13:25:25.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:25 smithi111 ceph-mon[31361]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-22T13:25:26.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:25 smithi012 ceph-mon[28265]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-22T13:25:26.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:25 smithi134 ceph-mon[31338]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-22T13:25:27.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:27 smithi111 ceph-mon[31361]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-22T13:25:28.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:27 smithi012 ceph-mon[28265]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-22T13:25:28.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:27 smithi134 ceph-mon[31338]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-22T13:25:29.381 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:25:29.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.521+0000 7f9097ca5640 1 -- 172.21.15.12:0/988875859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9090069aa0 msgr2=0x7f909010ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:29.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.521+0000 7f9097ca5640 1 --2- 172.21.15.12:0/988875859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9090069aa0 0x7f909010ede0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f9088009ca0 tx=0x7f908802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:25:29.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.521+0000 7f9097ca5640 1 -- 172.21.15.12:0/988875859 shutdown_connections 2024-04-22T13:25:29.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.521+0000 7f9097ca5640 1 --2- 172.21.15.12:0/988875859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9090069aa0 0x7f909010ede0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:29.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.521+0000 7f9097ca5640 1 --2- 172.21.15.12:0/988875859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90900690e0 0x7f9090069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:29.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.521+0000 7f9097ca5640 1 --2- 172.21.15.12:0/988875859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9090106f50 0x7f9090107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:29.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.521+0000 7f9097ca5640 1 -- 172.21.15.12:0/988875859 >> 172.21.15.12:0/988875859 conn(0x7f9090077e30 msgr2=0x7f9090078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:29.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.522+0000 7f9097ca5640 1 -- 172.21.15.12:0/988875859 shutdown_connections 2024-04-22T13:25:29.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.522+0000 7f9097ca5640 1 -- 172.21.15.12:0/988875859 wait complete. 2024-04-22T13:25:29.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.522+0000 7f9097ca5640 1 Processor -- start 2024-04-22T13:25:29.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.522+0000 7f9097ca5640 1 -- start start 2024-04-22T13:25:29.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.522+0000 7f9097ca5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90900690e0 0x7f9090198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:29.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.522+0000 7f9097ca5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9090069aa0 0x7f9090198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:29.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.522+0000 7f9097ca5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9090106f50 0x7f909019cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:29.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.522+0000 7f9097ca5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9090112150 con 0x7f9090106f50 2024-04-22T13:25:29.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.522+0000 7f9097ca5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9090111fd0 con 0x7f9090069aa0 2024-04-22T13:25:29.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.522+0000 7f9097ca5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f90901122d0 con 0x7f90900690e0 2024-04-22T13:25:29.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.523+0000 7f9096242640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9090106f50 0x7f909019cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:29.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.523+0000 7f9096242640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9090106f50 0x7f909019cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60892/0 (socket says 172.21.15.12:60892) 2024-04-22T13:25:29.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.523+0000 7f9096242640 1 -- 172.21.15.12:0/1538683669 learned_addr learned my addr 172.21.15.12:0/1538683669 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:29.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.523+0000 7f9095240640 1 --2- 172.21.15.12:0/1538683669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9090069aa0 0x7f9090198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:29.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.523+0000 7f9095a41640 1 --2- 172.21.15.12:0/1538683669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90900690e0 0x7f9090198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:29.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.523+0000 7f9096242640 1 -- 172.21.15.12:0/1538683669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90900690e0 msgr2=0x7f9090198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:29.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.523+0000 7f9096242640 1 --2- 172.21.15.12:0/1538683669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90900690e0 0x7f9090198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:29.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.523+0000 7f9096242640 1 -- 172.21.15.12:0/1538683669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9090069aa0 msgr2=0x7f9090198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:29.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.523+0000 7f9096242640 1 --2- 172.21.15.12:0/1538683669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9090069aa0 0x7f9090198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:29.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.523+0000 7f9096242640 1 -- 172.21.15.12:0/1538683669 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f909019d5d0 con 0x7f9090106f50 2024-04-22T13:25:29.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.523+0000 7f9095240640 1 --2- 172.21.15.12:0/1538683669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9090069aa0 0x7f9090198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:29.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.523+0000 7f9096242640 1 --2- 172.21.15.12:0/1538683669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9090106f50 0x7f909019cea0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f908802c700 tx=0x7f90880346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:29.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.523+0000 7f9086ffd640 1 -- 172.21.15.12:0/1538683669 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088041020 con 0x7f9090106f50 2024-04-22T13:25:29.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.524+0000 7f9086ffd640 1 -- 172.21.15.12:0/1538683669 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9088032960 con 0x7f9090106f50 2024-04-22T13:25:29.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.524+0000 7f9097ca5640 1 -- 172.21.15.12:0/1538683669 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f909019d860 con 0x7f9090106f50 2024-04-22T13:25:29.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.524+0000 7f9086ffd640 1 -- 172.21.15.12:0/1538683669 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088039040 con 0x7f9090106f50 2024-04-22T13:25:29.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.524+0000 7f9097ca5640 1 -- 172.21.15.12:0/1538683669 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90901a5150 con 0x7f9090106f50 2024-04-22T13:25:29.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.525+0000 7f9097ca5640 1 -- 172.21.15.12:0/1538683669 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9090068de0 con 0x7f9090106f50 2024-04-22T13:25:29.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.527+0000 7f9086ffd640 1 -- 172.21.15.12:0/1538683669 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f9088044020 con 0x7f9090106f50 2024-04-22T13:25:29.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.528+0000 7f9086ffd640 1 --2- 172.21.15.12:0/1538683669 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f907c0a3fd0 0x7f907c0a6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:29.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.528+0000 7f9086ffd640 1 -- 172.21.15.12:0/1538683669 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f90880f2ab0 con 0x7f9090106f50 2024-04-22T13:25:29.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.528+0000 7f9095a41640 1 --2- 172.21.15.12:0/1538683669 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f907c0a3fd0 0x7f907c0a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:29.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.529+0000 7f9095a41640 1 --2- 172.21.15.12:0/1538683669 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f907c0a3fd0 0x7f907c0a6490 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f908c009ae0 tx=0x7f908c005f90 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:29.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.535+0000 7f9086ffd640 1 -- 172.21.15.12:0/1538683669 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f908804b020 con 0x7f9090106f50 2024-04-22T13:25:29.692 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:29 smithi012 ceph-mon[28265]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-22T13:25:29.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.924+0000 7f9097ca5640 1 -- 172.21.15.12:0/1538683669 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f909010a500 con 0x7f9090106f50 2024-04-22T13:25:29.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.924+0000 7f9086ffd640 1 -- 172.21.15.12:0/1538683669 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7f90880bf070 con 0x7f9090106f50 2024-04-22T13:25:29.921 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:25:29.921 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:25:29.921 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:25:29.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.926+0000 7f9097ca5640 1 -- 172.21.15.12:0/1538683669 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f907c0a3fd0 msgr2=0x7f907c0a6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:29.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.926+0000 7f9097ca5640 1 --2- 172.21.15.12:0/1538683669 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f907c0a3fd0 0x7f907c0a6490 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f908c009ae0 tx=0x7f908c005f90 comp rx=0 tx=0).stop 2024-04-22T13:25:29.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.926+0000 7f9097ca5640 1 -- 172.21.15.12:0/1538683669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9090106f50 msgr2=0x7f909019cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:29.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.926+0000 7f9097ca5640 1 --2- 172.21.15.12:0/1538683669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9090106f50 0x7f909019cea0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f908802c700 tx=0x7f90880346d0 comp rx=0 tx=0).stop 2024-04-22T13:25:29.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.926+0000 7f9097ca5640 1 -- 172.21.15.12:0/1538683669 shutdown_connections 2024-04-22T13:25:29.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.927+0000 7f9097ca5640 1 --2- 172.21.15.12:0/1538683669 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f907c0a3fd0 0x7f907c0a6490 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:29.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.927+0000 7f9097ca5640 1 --2- 172.21.15.12:0/1538683669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9090106f50 0x7f909019cea0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:29.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.927+0000 7f9097ca5640 1 --2- 172.21.15.12:0/1538683669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9090069aa0 0x7f9090198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:29.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.927+0000 7f9097ca5640 1 --2- 172.21.15.12:0/1538683669 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90900690e0 0x7f9090198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:29.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.927+0000 7f9097ca5640 1 -- 172.21.15.12:0/1538683669 >> 172.21.15.12:0/1538683669 conn(0x7f9090077e30 msgr2=0x7f90900fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:29.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.927+0000 7f9097ca5640 1 -- 172.21.15.12:0/1538683669 shutdown_connections 2024-04-22T13:25:29.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:29.927+0000 7f9097ca5640 1 -- 172.21.15.12:0/1538683669 wait complete. 2024-04-22T13:25:29.934 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db --set-id 3ff6b7a --release --await' 2024-04-22T13:25:29.934 DEBUG:teuthology.orchestra.run.smithi012:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24509 quiesce db --set-id 3ff6b7a --release --await 2024-04-22T13:25:29.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:29 smithi111 ceph-mon[31361]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-22T13:25:30.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.048+0000 7f95e2b3d640 1 -- 172.21.15.12:0/2917863352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95dc103af0 msgr2=0x7f95dc10bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:30.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.048+0000 7f95e2b3d640 1 --2- 172.21.15.12:0/2917863352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95dc103af0 0x7f95dc10bfd0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f95c8009900 tx=0x7f95c802bd60 comp rx=0 tx=0).stop 2024-04-22T13:25:30.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.048+0000 7f95e2b3d640 1 -- 172.21.15.12:0/2917863352 shutdown_connections 2024-04-22T13:25:30.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.048+0000 7f95e2b3d640 1 --2- 172.21.15.12:0/2917863352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f95dc10c510 0x7f95dc10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.048+0000 7f95e2b3d640 1 --2- 172.21.15.12:0/2917863352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95dc103af0 0x7f95dc10bfd0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.048+0000 7f95e2b3d640 1 --2- 172.21.15.12:0/2917863352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95dc1031d0 0x7f95dc1035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.048+0000 7f95e2b3d640 1 -- 172.21.15.12:0/2917863352 >> 172.21.15.12:0/2917863352 conn(0x7f95dc077f50 msgr2=0x7f95dc078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:30.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.048+0000 7f95e2b3d640 1 -- 172.21.15.12:0/2917863352 shutdown_connections 2024-04-22T13:25:30.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.048+0000 7f95e2b3d640 1 -- 172.21.15.12:0/2917863352 wait complete. 2024-04-22T13:25:30.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.049+0000 7f95e2b3d640 1 Processor -- start 2024-04-22T13:25:30.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.049+0000 7f95e2b3d640 1 -- start start 2024-04-22T13:25:30.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.049+0000 7f95e2b3d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f95dc1031d0 0x7f95dc19e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:30.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.049+0000 7f95e2b3d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95dc103af0 0x7f95dc19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:30.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.049+0000 7f95e2b3d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95dc10c510 0x7f95dc1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:30.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.049+0000 7f95e2b3d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc1121d0 con 0x7f95dc103af0 2024-04-22T13:25:30.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.049+0000 7f95e2b3d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc112050 con 0x7f95dc1031d0 2024-04-22T13:25:30.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.049+0000 7f95e2b3d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc112350 con 0x7f95dc10c510 2024-04-22T13:25:30.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.049+0000 7f95dbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95dc103af0 0x7f95dc19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:30.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.049+0000 7f95dbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95dc103af0 0x7f95dc19e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60912/0 (socket says 172.21.15.12:60912) 2024-04-22T13:25:30.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.049+0000 7f95dbfff640 1 -- 172.21.15.12:0/1878961901 learned_addr learned my addr 172.21.15.12:0/1878961901 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:30.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.050+0000 7f95e08d9640 1 --2- 172.21.15.12:0/1878961901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f95dc1031d0 0x7f95dc19e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:30.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.050+0000 7f95e10da640 1 --2- 172.21.15.12:0/1878961901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95dc10c510 0x7f95dc1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:30.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.050+0000 7f95dbfff640 1 -- 172.21.15.12:0/1878961901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95dc10c510 msgr2=0x7f95dc1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:30.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.050+0000 7f95dbfff640 1 --2- 172.21.15.12:0/1878961901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95dc10c510 0x7f95dc1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.050+0000 7f95dbfff640 1 -- 172.21.15.12:0/1878961901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f95dc1031d0 msgr2=0x7f95dc19e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:30.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.050+0000 7f95dbfff640 1 --2- 172.21.15.12:0/1878961901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f95dc1031d0 0x7f95dc19e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.050+0000 7f95dbfff640 1 -- 172.21.15.12:0/1878961901 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95dc198d30 con 0x7f95dc103af0 2024-04-22T13:25:30.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.050+0000 7f95dbfff640 1 --2- 172.21.15.12:0/1878961901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95dc103af0 0x7f95dc19e940 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f95c8000c00 tx=0x7f95c802d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:30.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.050+0000 7f95d9ffb640 1 -- 172.21.15.12:0/1878961901 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c8043020 con 0x7f95dc103af0 2024-04-22T13:25:30.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.050+0000 7f95e2b3d640 1 -- 172.21.15.12:0/1878961901 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95dc198fc0 con 0x7f95dc103af0 2024-04-22T13:25:30.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.050+0000 7f95d9ffb640 1 -- 172.21.15.12:0/1878961901 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95c802dde0 con 0x7f95dc103af0 2024-04-22T13:25:30.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.050+0000 7f95d9ffb640 1 -- 172.21.15.12:0/1878961901 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c8003b80 con 0x7f95dc103af0 2024-04-22T13:25:30.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.051+0000 7f95e2b3d640 1 -- 172.21.15.12:0/1878961901 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95dc1a52c0 con 0x7f95dc103af0 2024-04-22T13:25:30.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.054+0000 7f95e5807740 1 Processor -- start 2024-04-22T13:25:30.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.054+0000 7f95e5807740 1 -- start start 2024-04-22T13:25:30.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.054+0000 7f95d9ffb640 1 -- 172.21.15.12:0/1878961901 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f95c8046020 con 0x7f95dc103af0 2024-04-22T13:25:30.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.054+0000 7f95e5807740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f0bede040 0x557f0bede420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:30.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.054+0000 7f95e5807740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f0bede960 0x557f0beeb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:30.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.054+0000 7f95e5807740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f0beebc20 0x557f0beee010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:30.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.054+0000 7f95e5807740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557f0be43500 con 0x557f0beebc20 2024-04-22T13:25:30.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.054+0000 7f95e5807740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557f0bde96a0 con 0x557f0bede960 2024-04-22T13:25:30.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.054+0000 7f95e5807740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557f0be3f4c0 con 0x557f0bede040 2024-04-22T13:25:30.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.054+0000 7f95dbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f0beebc20 0x557f0beee010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:30.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.054+0000 7f95e10da640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f0bede040 0x557f0bede420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:30.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.054+0000 7f95e10da640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f0bede040 0x557f0bede420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.12:60682/0 (socket says 172.21.15.12:60682) 2024-04-22T13:25:30.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.054+0000 7f95e10da640 1 -- 172.21.15.12:0/1814126731 learned_addr learned my addr 172.21.15.12:0/1814126731 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:30.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.054+0000 7f95e08d9640 1 --2- 172.21.15.12:0/1814126731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f0bede960 0x557f0beeb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:30.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.055+0000 7f95e10da640 1 -- 172.21.15.12:0/1814126731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f0bede960 msgr2=0x557f0beeb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:30.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.055+0000 7f95e10da640 1 --2- 172.21.15.12:0/1814126731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f0bede960 0x557f0beeb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.055+0000 7f95e10da640 1 -- 172.21.15.12:0/1814126731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f0beebc20 msgr2=0x557f0beee010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:30.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.055+0000 7f95e10da640 1 --2- 172.21.15.12:0/1814126731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f0beebc20 0x557f0beee010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.055+0000 7f95d9ffb640 1 --2- 172.21.15.12:0/1878961901 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f95cc0a3ed0 0x7f95cc0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:30.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.055+0000 7f95e10da640 1 -- 172.21.15.12:0/1814126731 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f0be085e0 con 0x557f0bede040 2024-04-22T13:25:30.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.055+0000 7f95dbfff640 1 --2- 172.21.15.12:0/1814126731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f0beebc20 0x557f0beee010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:30.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.055+0000 7f95e08d9640 1 --2- 172.21.15.12:0/1878961901 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f95cc0a3ed0 0x7f95cc0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:30.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.055+0000 7f95d9ffb640 1 -- 172.21.15.12:0/1878961901 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f95c80f2860 con 0x7f95dc103af0 2024-04-22T13:25:30.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.055+0000 7f95e10da640 1 --2- 172.21.15.12:0/1814126731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f0bede040 0x557f0bede420 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009e60 tx=0x7f95d4010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e77d02bfe28ecb16 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:30.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.056+0000 7f95c6ffd640 1 -- 172.21.15.12:0/1814126731 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d4016070 con 0x557f0bede040 2024-04-22T13:25:30.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.056+0000 7f95c6ffd640 1 -- 172.21.15.12:0/1814126731 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95d4004d20 con 0x557f0bede040 2024-04-22T13:25:30.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.056+0000 7f95e5807740 1 -- 172.21.15.12:0/1814126731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f0bede040 msgr2=0x557f0bede420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:30.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.056+0000 7f95e5807740 1 --2- 172.21.15.12:0/1814126731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f0bede040 0x557f0bede420 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009e60 tx=0x7f95d4010740 comp rx=0 tx=0).stop 2024-04-22T13:25:30.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.056+0000 7f95e08d9640 1 --2- 172.21.15.12:0/1878961901 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f95cc0a3ed0 0x7f95cc0a6390 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x557f0bedf220 tx=0x7f95d000b4d0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:30.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.056+0000 7f95e5807740 1 -- 172.21.15.12:0/1814126731 shutdown_connections 2024-04-22T13:25:30.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.056+0000 7f95e5807740 1 --2- 172.21.15.12:0/1814126731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f0beebc20 0x557f0beee010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.056+0000 7f95e5807740 1 --2- 172.21.15.12:0/1814126731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f0bede960 0x557f0beeb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.056+0000 7f95e5807740 1 --2- 172.21.15.12:0/1814126731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f0bede040 0x557f0bede420 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.056+0000 7f95e5807740 1 -- 172.21.15.12:0/1814126731 >> 172.21.15.12:0/1814126731 conn(0x557f0bed8a40 msgr2=0x557f0bed8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:30.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.056+0000 7f95e5807740 1 -- 172.21.15.12:0/1814126731 shutdown_connections 2024-04-22T13:25:30.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.056+0000 7f95e5807740 1 -- 172.21.15.12:0/1814126731 wait complete. 2024-04-22T13:25:30.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.057+0000 7f95e5807740 1 -- start start 2024-04-22T13:25:30.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.057+0000 7f95e5807740 1 Processor -- start 2024-04-22T13:25:30.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.057+0000 7f95e5807740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f0bede040 0x557f0beeb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:30.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.057+0000 7f95e5807740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f0bede960 0x557f0bee8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:30.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.058+0000 7f95e08d9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f0bede040 0x557f0beeb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:30.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.058+0000 7f95e08d9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f0bede040 0x557f0beeb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60934/0 (socket says 172.21.15.12:60934) 2024-04-22T13:25:30.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.058+0000 7f95e08d9640 1 -- 172.21.15.12:0/1746036084 learned_addr learned my addr 172.21.15.12:0/1746036084 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:30.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.058+0000 7f95dbfff640 1 --2- 172.21.15.12:0/1746036084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f0bede960 0x557f0bee8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:30.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.058+0000 7f95e5807740 1 --2- 172.21.15.12:0/1746036084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f0beebc20 0x557f0bee8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:30.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.058+0000 7f95e5807740 1 -- 172.21.15.12:0/1746036084 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557f0beeea30 con 0x557f0bede040 2024-04-22T13:25:30.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.058+0000 7f95e5807740 1 -- 172.21.15.12:0/1746036084 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557f0bdfca60 con 0x557f0bede960 2024-04-22T13:25:30.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.058+0000 7f95e5807740 1 -- 172.21.15.12:0/1746036084 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557f0beeebb0 con 0x557f0beebc20 2024-04-22T13:25:30.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.058+0000 7f95e10da640 1 --2- 172.21.15.12:0/1746036084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f0beebc20 0x557f0bee8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:30.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.058+0000 7f95e08d9640 1 -- 172.21.15.12:0/1746036084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f0beebc20 msgr2=0x557f0bee8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:30.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.058+0000 7f95e08d9640 1 --2- 172.21.15.12:0/1746036084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f0beebc20 0x557f0bee8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.058+0000 7f95e08d9640 1 -- 172.21.15.12:0/1746036084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f0bede960 msgr2=0x557f0bee8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:30.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.058+0000 7f95e08d9640 1 --2- 172.21.15.12:0/1746036084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f0bede960 0x557f0bee8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.058+0000 7f95e08d9640 1 -- 172.21.15.12:0/1746036084 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f0bb55920 con 0x557f0bede040 2024-04-22T13:25:30.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.059+0000 7f95e5807740 10 client.0 _ll_register_callbacks cb 0x557f0bb55110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:30.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.059+0000 7f95e5807740 1 -- 172.21.15.12:0/1746036084 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95d4004d20 con 0x557f0bede040 2024-04-22T13:25:30.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.059+0000 7f95e08d9640 1 --2- 172.21.15.12:0/1746036084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f0bede040 0x557f0beeb5d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f95d0005cc0 tx=0x7f95d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5fb69c421ce961a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:30.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.059+0000 7f95bbfff640 10 client.16290 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:30.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.059+0000 7f95bbfff640 1 -- 172.21.15.12:0/1746036084 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d0029680 con 0x557f0bede040 2024-04-22T13:25:30.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.059+0000 7f95bbfff640 1 -- 172.21.15.12:0/1746036084 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f95d0029820 con 0x557f0bede040 2024-04-22T13:25:30.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.059+0000 7f95bbfff640 1 -- 172.21.15.12:0/1746036084 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f95d0029a20 con 0x557f0bede040 2024-04-22T13:25:30.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.059+0000 7f95e5807740 10 client.16290 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:30.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.059+0000 7f95e5807740 1 -- 172.21.15.12:0/1746036084 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({fsmap=28}) -- 0x557f0bc21d30 con 0x557f0bede040 2024-04-22T13:25:30.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.060+0000 7f95bbfff640 1 -- 172.21.15.12:0/1746036084 <== mon.0 v2:172.21.15.12:3300/0 4 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f95d0033420 con 0x557f0bede040 2024-04-22T13:25:30.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.060+0000 7f95e5807740 10 client.16290 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:30.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.060+0000 7f95e5807740 10 client.16290 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-22T13:25:30.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.060+0000 7f95e5807740 20 client.16290 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:30.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.060+0000 7f95e5807740 1 --2- 172.21.15.12:0/1746036084 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557f0befbb60 0x557f0befdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:30.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.060+0000 7f95e5807740 4 client.16290 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:25:30.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.060+0000 7f95e5807740 1 -- 172.21.15.12:0/1746036084 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f0be16a10 con 0x557f0befbb60 2024-04-22T13:25:30.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.060+0000 7f95dbfff640 1 --2- 172.21.15.12:0/1746036084 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557f0befbb60 0x557f0befdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:30.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.060+0000 7f95dbfff640 1 --2- 172.21.15.12:0/1746036084 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557f0befbb60 0x557f0befdfb0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7271825aaa85bb4b server_cookie=ed302e28378f94fe in_seq=0 out_seq=0 2024-04-22T13:25:30.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.061+0000 7f95bbfff640 10 client.16290 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:30.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.061+0000 7f95bbfff640 1 -- 172.21.15.12:0/1746036084 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557f0be16a10 con 0x557f0befbb60 2024-04-22T13:25:30.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.061+0000 7f95bbfff640 10 client.16290 handle_command_reply: tid=0 2024-04-22T13:25:30.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.061+0000 7f95e5807740 1 client.16290 shutdown 2024-04-22T13:25:30.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.062+0000 7f95e5807740 1 -- 172.21.15.12:0/1746036084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f0bede040 msgr2=0x557f0beeb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:30.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.062+0000 7f95e5807740 1 --2- 172.21.15.12:0/1746036084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f0bede040 0x557f0beeb5d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f95d0005cc0 tx=0x7f95d00046c0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.062+0000 7f95e5807740 1 -- 172.21.15.12:0/1746036084 shutdown_connections 2024-04-22T13:25:30.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.062+0000 7f95e5807740 1 --2- 172.21.15.12:0/1746036084 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557f0befbb60 0x557f0befdfb0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.062+0000 7f95e5807740 1 --2- 172.21.15.12:0/1746036084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f0beebc20 0x557f0bee8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.062+0000 7f95e5807740 1 --2- 172.21.15.12:0/1746036084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f0bede960 0x557f0bee8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.062+0000 7f95bbfff640 0 client.16290 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:30.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.062+0000 7f95e5807740 1 --2- 172.21.15.12:0/1746036084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f0bede040 0x557f0beeb5d0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.062+0000 7f95e5807740 1 -- 172.21.15.12:0/1746036084 >> 172.21.15.12:0/1746036084 conn(0x557f0bed8a40 msgr2=0x557f0beda790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:30.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.062+0000 7f95e5807740 1 -- 172.21.15.12:0/1746036084 shutdown_connections 2024-04-22T13:25:30.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.062+0000 7f95e5807740 1 -- 172.21.15.12:0/1746036084 wait complete. 2024-04-22T13:25:30.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.062+0000 7f95e5807740 20 client.16290 trim_cache size 0 max 16384 2024-04-22T13:25:30.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.096+0000 7f95e5807740 1 Processor -- start 2024-04-22T13:25:30.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.096+0000 7f95e5807740 1 -- start start 2024-04-22T13:25:30.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.096+0000 7f95e5807740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f0bede960 0x557f0bef79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:30.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.096+0000 7f95e5807740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f0beebc20 0x557f0beea790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:30.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.096+0000 7f95e5807740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f0beeacd0 0x557f0bf038c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:30.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.097+0000 7f95e5807740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc1ab920 con 0x557f0bede960 2024-04-22T13:25:30.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.097+0000 7f95e5807740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc1ad460 con 0x557f0beeacd0 2024-04-22T13:25:30.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.097+0000 7f95e5807740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc1abff0 con 0x557f0beebc20 2024-04-22T13:25:30.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.097+0000 7f95e08d9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f0bede960 0x557f0bef79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:30.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.097+0000 7f95e08d9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f0bede960 0x557f0bef79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60942/0 (socket says 172.21.15.12:60942) 2024-04-22T13:25:30.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.097+0000 7f95e08d9640 1 -- 172.21.15.12:0/2703512006 learned_addr learned my addr 172.21.15.12:0/2703512006 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:30.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.097+0000 7f95dbfff640 1 --2- 172.21.15.12:0/2703512006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f0beebc20 0x557f0beea790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:30.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.097+0000 7f95e10da640 1 --2- 172.21.15.12:0/2703512006 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f0beeacd0 0x557f0bf038c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:30.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.097+0000 7f95e08d9640 1 -- 172.21.15.12:0/2703512006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f0beebc20 msgr2=0x557f0beea790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:30.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.097+0000 7f95e08d9640 1 --2- 172.21.15.12:0/2703512006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f0beebc20 0x557f0beea790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.097+0000 7f95e08d9640 1 -- 172.21.15.12:0/2703512006 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f0beeacd0 msgr2=0x557f0bf038c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:30.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.097+0000 7f95e08d9640 1 --2- 172.21.15.12:0/2703512006 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f0beeacd0 0x557f0bf038c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.097+0000 7f95e08d9640 1 -- 172.21.15.12:0/2703512006 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f0bee38b0 con 0x557f0bede960 2024-04-22T13:25:30.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.097+0000 7f95dbfff640 1 --2- 172.21.15.12:0/2703512006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f0beebc20 0x557f0beea790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:30.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.097+0000 7f95e08d9640 1 --2- 172.21.15.12:0/2703512006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f0bede960 0x557f0bef79b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f95d0013fd0 tx=0x7f95d0020070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0e8432a9db4f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:30.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.098+0000 7f95c67fc640 1 -- 172.21.15.12:0/2703512006 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d000be90 con 0x557f0bede960 2024-04-22T13:25:30.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.098+0000 7f95c67fc640 1 -- 172.21.15.12:0/2703512006 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95d4004d20 con 0x557f0bede960 2024-04-22T13:25:30.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.098+0000 7f95c67fc640 1 -- 172.21.15.12:0/2703512006 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d0029e20 con 0x557f0bede960 2024-04-22T13:25:30.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.099+0000 7f95e5807740 1 -- 172.21.15.12:0/2703512006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f0bede960 msgr2=0x557f0bef79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:30.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.099+0000 7f95e5807740 1 --2- 172.21.15.12:0/2703512006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f0bede960 0x557f0bef79b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f95d0013fd0 tx=0x7f95d0020070 comp rx=0 tx=0).stop 2024-04-22T13:25:30.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.099+0000 7f95e5807740 1 -- 172.21.15.12:0/2703512006 shutdown_connections 2024-04-22T13:25:30.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.099+0000 7f95e5807740 1 --2- 172.21.15.12:0/2703512006 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f0beeacd0 0x557f0bf038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.099+0000 7f95e5807740 1 --2- 172.21.15.12:0/2703512006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f0beebc20 0x557f0beea790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.099+0000 7f95e5807740 1 --2- 172.21.15.12:0/2703512006 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f0bede960 0x557f0bef79b0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.099+0000 7f95e5807740 1 -- 172.21.15.12:0/2703512006 >> 172.21.15.12:0/2703512006 conn(0x557f0bed8a40 msgr2=0x557f0beea080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:30.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.099+0000 7f95e5807740 1 -- 172.21.15.12:0/2703512006 shutdown_connections 2024-04-22T13:25:30.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.099+0000 7f95e5807740 1 -- 172.21.15.12:0/2703512006 wait complete. 2024-04-22T13:25:30.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.099+0000 7f95e5807740 1 -- start start 2024-04-22T13:25:30.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.099+0000 7f95e5807740 1 Processor -- start 2024-04-22T13:25:30.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.100+0000 7f95e5807740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f0bede960 0x557f0bef40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:30.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.100+0000 7f95e5807740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f0beeacd0 0x557f0bef0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:30.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.100+0000 7f95e5807740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f0beebc20 0x557f0bef13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:30.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.100+0000 7f95e5807740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc1acd90 con 0x557f0beebc20 2024-04-22T13:25:30.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.100+0000 7f95e08d9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f0bede960 0x557f0bef40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:30.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.100+0000 7f95dbfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f0beeacd0 0x557f0bef0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:30.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.100+0000 7f95e08d9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f0bede960 0x557f0bef40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.12:60696/0 (socket says 172.21.15.12:60696) 2024-04-22T13:25:30.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.100+0000 7f95e08d9640 1 -- 172.21.15.12:0/2281772995 learned_addr learned my addr 172.21.15.12:0/2281772995 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:30.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.100+0000 7f95dbfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f0beeacd0 0x557f0bef0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:47158/0 (socket says 172.21.15.12:47158) 2024-04-22T13:25:30.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.100+0000 7f95e5807740 1 -- 172.21.15.12:0/2281772995 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc1ac6c0 con 0x557f0beeacd0 2024-04-22T13:25:30.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.100+0000 7f95e5807740 1 -- 172.21.15.12:0/2281772995 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f95dc1a9920 con 0x557f0bede960 2024-04-22T13:25:30.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.100+0000 7f95e10da640 1 --2- 172.21.15.12:0/2281772995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f0beebc20 0x557f0bef13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:30.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.100+0000 7f95e5807740 10 client.0 _ll_register_callbacks cb 0x7f95d001d3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:30.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.101+0000 7f95e10da640 1 -- 172.21.15.12:0/2281772995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f0bede960 msgr2=0x557f0bef40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:30.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.101+0000 7f95e10da640 1 --2- 172.21.15.12:0/2281772995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f0bede960 0x557f0bef40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.101+0000 7f95e10da640 1 -- 172.21.15.12:0/2281772995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f0beeacd0 msgr2=0x557f0bef0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:30.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.101+0000 7f95e10da640 1 --2- 172.21.15.12:0/2281772995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f0beeacd0 0x557f0bef0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.101+0000 7f95e10da640 1 -- 172.21.15.12:0/2281772995 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f0bef1910 con 0x557f0beebc20 2024-04-22T13:25:30.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.101+0000 7f95e10da640 1 --2- 172.21.15.12:0/2281772995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f0beebc20 0x557f0bef13d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f95d4000f80 tx=0x7f95d4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab571769182a3c4b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:30.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.101+0000 7f95bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:30.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.101+0000 7f95bbfff640 1 -- 172.21.15.12:0/2281772995 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d4016040 con 0x557f0beebc20 2024-04-22T13:25:30.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.101+0000 7f95bbfff640 1 -- 172.21.15.12:0/2281772995 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f95d40128e0 con 0x557f0beebc20 2024-04-22T13:25:30.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.101+0000 7f95e5807740 1 -- 172.21.15.12:0/2281772995 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95d4004d20 con 0x557f0beebc20 2024-04-22T13:25:30.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.102+0000 7f95bbfff640 1 -- 172.21.15.12:0/2281772995 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f95d4012ae0 con 0x557f0beebc20 2024-04-22T13:25:30.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.102+0000 7f95e5807740 10 client.16296 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:30.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.102+0000 7f95e5807740 1 -- 172.21.15.12:0/2281772995 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557f0bf62e20 con 0x557f0beebc20 2024-04-22T13:25:30.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.102+0000 7f95bbfff640 1 -- 172.21.15.12:0/2281772995 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95d4004d20 con 0x557f0beebc20 2024-04-22T13:25:30.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.102+0000 7f95bbfff640 1 -- 172.21.15.12:0/2281772995 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f95d400f040 con 0x557f0beebc20 2024-04-22T13:25:30.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.102+0000 7f95e5807740 10 client.16296 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:30.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.102+0000 7f95e5807740 10 client.16296 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-22T13:25:30.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.102+0000 7f95e5807740 20 client.16296 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:30.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.102+0000 7f95e5807740 1 --2- 172.21.15.12:0/2281772995 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557f0bf631b0 0x557f0bf655a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:30.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.102+0000 7f95e5807740 4 client.16296 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3ff6b7a", "release": true, "await": true}] 2024-04-22T13:25:30.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.102+0000 7f95e5807740 1 -- 172.21.15.12:0/2281772995 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3ff6b7a", "release": true, "await": true}) -- 0x557f0be50f20 con 0x557f0bf631b0 2024-04-22T13:25:30.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.103+0000 7f95e08d9640 1 --2- 172.21.15.12:0/2281772995 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557f0bf631b0 0x557f0bf655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:30.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.103+0000 7f95bbfff640 10 client.16296 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:30.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.103+0000 7f95e08d9640 1 --2- 172.21.15.12:0/2281772995 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557f0bf631b0 0x557f0bf655a0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2ed1a4d8d030e38 server_cookie=329cb460519a3c9c in_seq=0 out_seq=0 2024-04-22T13:25:30.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.105+0000 7f95bbfff640 1 -- 172.21.15.12:0/2281772995 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+587 (crc 0 0 0) 0x557f0be50f20 con 0x557f0bf631b0 2024-04-22T13:25:30.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.105+0000 7f95bbfff640 10 client.16296 handle_command_reply: tid=0 2024-04-22T13:25:30.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.105+0000 7f95e5807740 1 client.16296 shutdown 2024-04-22T13:25:30.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.106+0000 7f95e5807740 1 -- 172.21.15.12:0/2281772995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f0beebc20 msgr2=0x557f0bef13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:30.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.106+0000 7f95e5807740 1 --2- 172.21.15.12:0/2281772995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f0beebc20 0x557f0bef13d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f95d4000f80 tx=0x7f95d4009bd0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.106+0000 7f95e5807740 1 -- 172.21.15.12:0/2281772995 shutdown_connections 2024-04-22T13:25:30.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.106+0000 7f95e5807740 1 --2- 172.21.15.12:0/2281772995 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557f0bf631b0 0x557f0bf655a0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.106+0000 7f95e5807740 1 --2- 172.21.15.12:0/2281772995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557f0beebc20 0x557f0bef13d0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.106+0000 7f95e5807740 1 --2- 172.21.15.12:0/2281772995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557f0beeacd0 0x557f0bef0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.106+0000 7f95e5807740 1 --2- 172.21.15.12:0/2281772995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557f0bede960 0x557f0bef40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.106+0000 7f95e5807740 1 -- 172.21.15.12:0/2281772995 >> 172.21.15.12:0/2281772995 conn(0x557f0bed8a40 msgr2=0x557f0bed7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:30.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.106+0000 7f95bbfff640 0 client.16296 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:30.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.106+0000 7f95e5807740 1 -- 172.21.15.12:0/2281772995 shutdown_connections 2024-04-22T13:25:30.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.106+0000 7f95e5807740 1 -- 172.21.15.12:0/2281772995 wait complete. 2024-04-22T13:25:30.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.106+0000 7f95e5807740 20 client.16296 trim_cache size 0 max 16384 2024-04-22T13:25:30.104 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:25:30.104 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 28, 2024-04-22T13:25:30.104 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24509, 2024-04-22T13:25:30.104 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 8, 2024-04-22T13:25:30.104 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": { 2024-04-22T13:25:30.104 INFO:teuthology.orchestra.run.smithi012.stdout: "3ff6b7a": { 2024-04-22T13:25:30.104 INFO:teuthology.orchestra.run.smithi012.stdout: "version": 8, 2024-04-22T13:25:30.105 INFO:teuthology.orchestra.run.smithi012.stdout: "age_ref": 0.0, 2024-04-22T13:25:30.105 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-04-22T13:25:30.105 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-04-22T13:25:30.105 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-04-22T13:25:30.105 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:25:30.105 INFO:teuthology.orchestra.run.smithi012.stdout: "timeout": 60.0, 2024-04-22T13:25:30.105 INFO:teuthology.orchestra.run.smithi012.stdout: "expiration": 85.3, 2024-04-22T13:25:30.105 INFO:teuthology.orchestra.run.smithi012.stdout: "members": { 2024-04-22T13:25:30.105 INFO:teuthology.orchestra.run.smithi012.stdout: "file:/": { 2024-04-22T13:25:30.105 INFO:teuthology.orchestra.run.smithi012.stdout: "excluded": false, 2024-04-22T13:25:30.105 INFO:teuthology.orchestra.run.smithi012.stdout: "state": { 2024-04-22T13:25:30.105 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RELEASED", 2024-04-22T13:25:30.105 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0 2024-04-22T13:25:30.105 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:25:30.105 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:25:30.105 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:25:30.106 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:25:30.106 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:25:30.106 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:25:30.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.107+0000 7f95e2b3d640 1 -- 172.21.15.12:0/1878961901 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f95cc0a3ed0 msgr2=0x7f95cc0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:30.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.107+0000 7f95e2b3d640 1 --2- 172.21.15.12:0/1878961901 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f95cc0a3ed0 0x7f95cc0a6390 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x557f0bedf220 tx=0x7f95d000b4d0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.107+0000 7f95e2b3d640 1 -- 172.21.15.12:0/1878961901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95dc103af0 msgr2=0x7f95dc19e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:30.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.107+0000 7f95e2b3d640 1 --2- 172.21.15.12:0/1878961901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95dc103af0 0x7f95dc19e940 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f95c8000c00 tx=0x7f95c802d920 comp rx=0 tx=0).stop 2024-04-22T13:25:30.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.107+0000 7f95e2b3d640 1 -- 172.21.15.12:0/1878961901 shutdown_connections 2024-04-22T13:25:30.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.107+0000 7f95e2b3d640 1 --2- 172.21.15.12:0/1878961901 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f95cc0a3ed0 0x7f95cc0a6390 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.107+0000 7f95e2b3d640 1 --2- 172.21.15.12:0/1878961901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95dc10c510 0x7f95dc1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.107+0000 7f95e2b3d640 1 --2- 172.21.15.12:0/1878961901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f95dc103af0 0x7f95dc19e940 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.107+0000 7f95e2b3d640 1 --2- 172.21.15.12:0/1878961901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f95dc1031d0 0x7f95dc19e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:30.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.107+0000 7f95e2b3d640 1 -- 172.21.15.12:0/1878961901 >> 172.21.15.12:0/1878961901 conn(0x7f95dc077f50 msgr2=0x7f95dc0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:30.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.107+0000 7f95e2b3d640 1 -- 172.21.15.12:0/1878961901 shutdown_connections 2024-04-22T13:25:30.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:30.107+0000 7f95e2b3d640 1 -- 172.21.15.12:0/1878961901 wait complete. 2024-04-22T13:25:30.117 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set_id: 3ff6b7a, seconds elapsed: 26.6 2024-04-22T13:25:30.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:29 smithi134 ceph-mon[31338]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-22T13:25:30.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:30 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1538683669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:25:31.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:30 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1538683669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:25:31.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:30 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1538683669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:25:31.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:31 smithi111 ceph-mon[31361]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-22T13:25:32.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:31 smithi012 ceph-mon[28265]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-22T13:25:32.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:31 smithi134 ceph-mon[31338]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-22T13:25:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.831+0000 7f27e3e70640 1 -- 172.21.15.12:0/2535973361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27dc102b60 msgr2=0x7f27dc106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.831+0000 7f27e3e70640 1 --2- 172.21.15.12:0/2535973361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27dc102b60 0x7f27dc106ba0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f27cc009900 tx=0x7f27cc02bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.832+0000 7f27e3e70640 1 -- 172.21.15.12:0/2535973361 shutdown_connections 2024-04-22T13:25:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.832+0000 7f27e3e70640 1 --2- 172.21.15.12:0/2535973361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27dc107280 0x7f27dc10ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.832+0000 7f27e3e70640 1 --2- 172.21.15.12:0/2535973361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27dc102b60 0x7f27dc106ba0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.832+0000 7f27e3e70640 1 --2- 172.21.15.12:0/2535973361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27dc1021b0 0x7f27dc102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.832+0000 7f27e3e70640 1 -- 172.21.15.12:0/2535973361 >> 172.21.15.12:0/2535973361 conn(0x7f27dc0fda10 msgr2=0x7f27dc0ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.832+0000 7f27e3e70640 1 -- 172.21.15.12:0/2535973361 shutdown_connections 2024-04-22T13:25:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.832+0000 7f27e3e70640 1 -- 172.21.15.12:0/2535973361 wait complete. 2024-04-22T13:25:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.833+0000 7f27e3e70640 1 Processor -- start 2024-04-22T13:25:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.833+0000 7f27e3e70640 1 -- start start 2024-04-22T13:25:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.833+0000 7f27e3e70640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27dc1021b0 0x7f27dc1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.833+0000 7f27e3e70640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27dc107280 0x7f27dc1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.834+0000 7f27e3e70640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27dc19cd60 0x7f27dc19d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.834+0000 7f27e3e70640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc1121c0 con 0x7f27dc1021b0 2024-04-22T13:25:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.834+0000 7f27e1c0c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27dc1021b0 0x7f27dc1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.834+0000 7f27e3e70640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc112040 con 0x7f27dc19cd60 2024-04-22T13:25:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.834+0000 7f27e3e70640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc112340 con 0x7f27dc107280 2024-04-22T13:25:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.834+0000 7f27e1c0c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27dc1021b0 0x7f27dc1984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42742/0 (socket says 172.21.15.12:42742) 2024-04-22T13:25:33.832 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.834+0000 7f27e1c0c640 1 -- 172.21.15.12:0/2346173161 learned_addr learned my addr 172.21.15.12:0/2346173161 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.834+0000 7f27e140b640 1 --2- 172.21.15.12:0/2346173161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27dc107280 0x7f27dc1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.834+0000 7f27e240d640 1 --2- 172.21.15.12:0/2346173161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27dc19cd60 0x7f27dc19d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.834+0000 7f27e1c0c640 1 -- 172.21.15.12:0/2346173161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27dc107280 msgr2=0x7f27dc1989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.834+0000 7f27e1c0c640 1 --2- 172.21.15.12:0/2346173161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27dc107280 0x7f27dc1989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.834+0000 7f27e1c0c640 1 -- 172.21.15.12:0/2346173161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27dc19cd60 msgr2=0x7f27dc19d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.834+0000 7f27e1c0c640 1 --2- 172.21.15.12:0/2346173161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27dc19cd60 0x7f27dc19d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.834+0000 7f27e1c0c640 1 -- 172.21.15.12:0/2346173161 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27dc19d850 con 0x7f27dc1021b0 2024-04-22T13:25:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.835+0000 7f27e1c0c640 1 --2- 172.21.15.12:0/2346173161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27dc1021b0 0x7f27dc1984a0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f27d800d950 tx=0x7f27d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.835+0000 7f27d2ffd640 1 -- 172.21.15.12:0/2346173161 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d8017070 con 0x7f27dc1021b0 2024-04-22T13:25:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.835+0000 7f27d2ffd640 1 -- 172.21.15.12:0/2346173161 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27d8009c30 con 0x7f27dc1021b0 2024-04-22T13:25:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.835+0000 7f27d2ffd640 1 -- 172.21.15.12:0/2346173161 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d800f620 con 0x7f27dc1021b0 2024-04-22T13:25:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.835+0000 7f27e3e70640 1 -- 172.21.15.12:0/2346173161 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27dc19db40 con 0x7f27dc1021b0 2024-04-22T13:25:33.833 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.835+0000 7f27e3e70640 1 -- 172.21.15.12:0/2346173161 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27dc1a5240 con 0x7f27dc1021b0 2024-04-22T13:25:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.838+0000 7f27d2ffd640 1 -- 172.21.15.12:0/2346173161 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f27d8010460 con 0x7f27dc1021b0 2024-04-22T13:25:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.838+0000 7f27e6a59740 1 Processor -- start 2024-04-22T13:25:33.834 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.839+0000 7f27e6a59740 1 -- start start 2024-04-22T13:25:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.839+0000 7f27e6a59740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558b1dd368d0 0x558b1dd35620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.839+0000 7f27e6a59740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558b1dd35b60 0x558b1dd35fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.839+0000 7f27e6a59740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b1dd3f0e0 0x558b1dd414d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.839+0000 7f27e6a59740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558b1dc9a8d0 con 0x558b1dd368d0 2024-04-22T13:25:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.839+0000 7f27e6a59740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x558b1dc37e80 con 0x558b1dd35b60 2024-04-22T13:25:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.839+0000 7f27e6a59740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558b1dc96890 con 0x558b1dd3f0e0 2024-04-22T13:25:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.839+0000 7f27e240d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558b1dd368d0 0x558b1dd35620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.839+0000 7f27e240d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558b1dd368d0 0x558b1dd35620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42752/0 (socket says 172.21.15.12:42752) 2024-04-22T13:25:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.839+0000 7f27e240d640 1 -- 172.21.15.12:0/3040013436 learned_addr learned my addr 172.21.15.12:0/3040013436 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.839+0000 7f27e140b640 1 --2- 172.21.15.12:0/3040013436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b1dd3f0e0 0x558b1dd414d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.839+0000 7f27d2ffd640 1 --2- 172.21.15.12:0/2346173161 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f27c80a4070 0x7f27c80a6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.840+0000 7f27e240d640 1 -- 172.21.15.12:0/3040013436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b1dd3f0e0 msgr2=0x558b1dd414d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.840+0000 7f27e240d640 1 --2- 172.21.15.12:0/3040013436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b1dd3f0e0 0x558b1dd414d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.840+0000 7f27e240d640 1 -- 172.21.15.12:0/3040013436 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558b1dd35b60 msgr2=0x558b1dd35fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.840+0000 7f27e1c0c640 1 --2- 172.21.15.12:0/3040013436 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558b1dd35b60 0x558b1dd35fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.840+0000 7f27e240d640 1 --2- 172.21.15.12:0/3040013436 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558b1dd35b60 0x558b1dd35fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.840+0000 7f27e240d640 1 -- 172.21.15.12:0/3040013436 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b1dc59320 con 0x558b1dd368d0 2024-04-22T13:25:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.840+0000 7f27e140b640 1 --2- 172.21.15.12:0/3040013436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b1dd3f0e0 0x558b1dd414d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:25:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.840+0000 7f27e240d640 1 --2- 172.21.15.12:0/2346173161 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f27c80a4070 0x7f27c80a6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.840+0000 7f27d2ffd640 1 -- 172.21.15.12:0/2346173161 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f27d80d05a0 con 0x7f27dc1021b0 2024-04-22T13:25:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.840+0000 7f27e240d640 1 --2- 172.21.15.12:0/3040013436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558b1dd368d0 0x558b1dd35620 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f27d4010440 tx=0x7f27d4010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d877c6cbdd5d4980 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.841+0000 7f27c7fff640 1 -- 172.21.15.12:0/3040013436 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d4022070 con 0x558b1dd368d0 2024-04-22T13:25:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.841+0000 7f27c7fff640 1 -- 172.21.15.12:0/3040013436 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27d401d430 con 0x558b1dd368d0 2024-04-22T13:25:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.841+0000 7f27c7fff640 1 -- 172.21.15.12:0/3040013436 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d40255a0 con 0x558b1dd368d0 2024-04-22T13:25:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.841+0000 7f27e6a59740 1 -- 172.21.15.12:0/3040013436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558b1dd368d0 msgr2=0x558b1dd35620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.841+0000 7f27e240d640 1 --2- 172.21.15.12:0/2346173161 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f27c80a4070 0x7f27c80a6530 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f27d401f000 tx=0x7f27d401c8c0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.841+0000 7f27e6a59740 1 --2- 172.21.15.12:0/3040013436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558b1dd368d0 0x558b1dd35620 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f27d4010440 tx=0x7f27d4010900 comp rx=0 tx=0).stop 2024-04-22T13:25:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.842+0000 7f27e6a59740 1 -- 172.21.15.12:0/3040013436 shutdown_connections 2024-04-22T13:25:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.842+0000 7f27e6a59740 1 --2- 172.21.15.12:0/3040013436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b1dd3f0e0 0x558b1dd414d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.842+0000 7f27e6a59740 1 --2- 172.21.15.12:0/3040013436 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558b1dd35b60 0x558b1dd35fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.842+0000 7f27e6a59740 1 --2- 172.21.15.12:0/3040013436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558b1dd368d0 0x558b1dd35620 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.842+0000 7f27e6a59740 1 -- 172.21.15.12:0/3040013436 >> 172.21.15.12:0/3040013436 conn(0x558b1dcfcc90 msgr2=0x558b1dd30850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.842+0000 7f27e6a59740 1 -- 172.21.15.12:0/3040013436 shutdown_connections 2024-04-22T13:25:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.842+0000 7f27e6a59740 1 -- 172.21.15.12:0/3040013436 wait complete. 2024-04-22T13:25:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.842+0000 7f27e6a59740 1 -- start start 2024-04-22T13:25:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.842+0000 7f27e6a59740 1 Processor -- start 2024-04-22T13:25:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.843+0000 7f27e6a59740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558b1dd35b60 0x558b1dd3eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.843+0000 7f27e6a59740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558b1dd368d0 0x558b1dd3be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.843+0000 7f27e6a59740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b1dd3f0e0 0x558b1dd3c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.843+0000 7f27e6a59740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558b1dd41e60 con 0x558b1dd35b60 2024-04-22T13:25:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.843+0000 7f27e6a59740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x558b1dc22eb0 con 0x558b1dd368d0 2024-04-22T13:25:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.843+0000 7f27e1c0c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558b1dd35b60 0x558b1dd3eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.843+0000 7f27e6a59740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558b1dd41fe0 con 0x558b1dd3f0e0 2024-04-22T13:25:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.843+0000 7f27e1c0c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558b1dd35b60 0x558b1dd3eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42766/0 (socket says 172.21.15.12:42766) 2024-04-22T13:25:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.843+0000 7f27e1c0c640 1 -- 172.21.15.12:0/3214295048 learned_addr learned my addr 172.21.15.12:0/3214295048 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.844+0000 7f27e6a59740 10 client.0 _ll_register_callbacks cb 0x558b1dc4fb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.844+0000 7f27e240d640 1 --2- 172.21.15.12:0/3214295048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b1dd3f0e0 0x558b1dd3c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.844+0000 7f27e140b640 1 --2- 172.21.15.12:0/3214295048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558b1dd368d0 0x558b1dd3be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.844+0000 7f27e1c0c640 1 -- 172.21.15.12:0/3214295048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b1dd3f0e0 msgr2=0x558b1dd3c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.844+0000 7f27e1c0c640 1 --2- 172.21.15.12:0/3214295048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b1dd3f0e0 0x558b1dd3c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.844+0000 7f27e1c0c640 1 -- 172.21.15.12:0/3214295048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558b1dd368d0 msgr2=0x558b1dd3be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.844+0000 7f27e1c0c640 1 --2- 172.21.15.12:0/3214295048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558b1dd368d0 0x558b1dd3be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.844+0000 7f27e1c0c640 1 -- 172.21.15.12:0/3214295048 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b1d9ad210 con 0x558b1dd35b60 2024-04-22T13:25:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.844+0000 7f27e140b640 1 --2- 172.21.15.12:0/3214295048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558b1dd368d0 0x558b1dd3be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.844+0000 7f27e1c0c640 1 --2- 172.21.15.12:0/3214295048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558b1dd35b60 0x558b1dd3eac0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f27d8098860 tx=0x7f27d8009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c653729e07dc51a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.844+0000 7f27c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.844+0000 7f27c57fa640 1 -- 172.21.15.12:0/3214295048 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d8020070 con 0x558b1dd35b60 2024-04-22T13:25:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.844+0000 7f27c57fa640 1 -- 172.21.15.12:0/3214295048 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f27d80d0310 con 0x558b1dd35b60 2024-04-22T13:25:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.844+0000 7f27e6a59740 1 -- 172.21.15.12:0/3214295048 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27d401d430 con 0x558b1dd35b60 2024-04-22T13:25:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.845+0000 7f27c57fa640 1 -- 172.21.15.12:0/3214295048 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f27d80a4fc0 con 0x558b1dd35b60 2024-04-22T13:25:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.845+0000 7f27e6a59740 10 client.16308 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.845+0000 7f27e6a59740 1 -- 172.21.15.12:0/3214295048 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558b1da79620 con 0x558b1dd35b60 2024-04-22T13:25:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.845+0000 7f27c57fa640 1 -- 172.21.15.12:0/3214295048 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27d80cde10 con 0x558b1dd35b60 2024-04-22T13:25:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.845+0000 7f27c57fa640 1 -- 172.21.15.12:0/3214295048 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f27d80b5ad0 con 0x558b1dd35b60 2024-04-22T13:25:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.845+0000 7f27e6a59740 10 client.16308 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.845+0000 7f27e6a59740 10 client.16308 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:25:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.845+0000 7f27e6a59740 20 client.16308 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.845+0000 7f27e6a59740 1 --2- 172.21.15.12:0/3214295048 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x558b1dd4f0f0 0x558b1dd514e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.845+0000 7f27e6a59740 4 client.16308 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:25:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.845+0000 7f27e6a59740 1 -- 172.21.15.12:0/3214295048 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f27d401d430 con 0x558b1dd4f0f0 2024-04-22T13:25:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.845+0000 7f27e140b640 1 --2- 172.21.15.12:0/3214295048 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x558b1dd4f0f0 0x558b1dd514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.846+0000 7f27c57fa640 10 client.16308 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.846+0000 7f27e140b640 1 --2- 172.21.15.12:0/3214295048 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x558b1dd4f0f0 0x558b1dd514e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8f29708d96cb2a0 server_cookie=57f7104e5ac44305 in_seq=0 out_seq=0 2024-04-22T13:25:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.847+0000 7f27c57fa640 1 -- 172.21.15.12:0/3214295048 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f27d401d430 con 0x558b1dd4f0f0 2024-04-22T13:25:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.847+0000 7f27c57fa640 10 client.16308 handle_command_reply: tid=0 2024-04-22T13:25:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.847+0000 7f27e6a59740 1 client.16308 shutdown 2024-04-22T13:25:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.847+0000 7f27e6a59740 1 -- 172.21.15.12:0/3214295048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558b1dd35b60 msgr2=0x558b1dd3eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.847+0000 7f27e6a59740 1 --2- 172.21.15.12:0/3214295048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558b1dd35b60 0x558b1dd3eac0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f27d8098860 tx=0x7f27d8009de0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.847+0000 7f27e6a59740 1 -- 172.21.15.12:0/3214295048 shutdown_connections 2024-04-22T13:25:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.847+0000 7f27e6a59740 1 --2- 172.21.15.12:0/3214295048 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x558b1dd4f0f0 0x558b1dd514e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.847+0000 7f27e6a59740 1 --2- 172.21.15.12:0/3214295048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b1dd3f0e0 0x558b1dd3c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.847+0000 7f27c57fa640 0 client.16308 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.847+0000 7f27e6a59740 1 --2- 172.21.15.12:0/3214295048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558b1dd368d0 0x558b1dd3be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.847+0000 7f27e6a59740 1 --2- 172.21.15.12:0/3214295048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558b1dd35b60 0x558b1dd3eac0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.847+0000 7f27e6a59740 1 -- 172.21.15.12:0/3214295048 >> 172.21.15.12:0/3214295048 conn(0x558b1dcfcc90 msgr2=0x558b1dd325a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.847+0000 7f27e6a59740 1 -- 172.21.15.12:0/3214295048 shutdown_connections 2024-04-22T13:25:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.847+0000 7f27e6a59740 1 -- 172.21.15.12:0/3214295048 wait complete. 2024-04-22T13:25:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.847+0000 7f27e6a59740 20 client.16308 trim_cache size 0 max 16384 2024-04-22T13:25:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.877+0000 7f27e6a59740 1 Processor -- start 2024-04-22T13:25:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.877+0000 7f27e6a59740 1 -- start start 2024-04-22T13:25:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.877+0000 7f27e6a59740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558b1dd35b60 0x558b1dd376e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.878+0000 7f27e6a59740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b1dd368d0 0x558b1dd4ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.878+0000 7f27e6a59740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558b1dd3f0e0 0x558b1dd56d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.878+0000 7f27e6a59740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc1ab660 con 0x558b1dd35b60 2024-04-22T13:25:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.878+0000 7f27e6a59740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc1ad1a0 con 0x558b1dd3f0e0 2024-04-22T13:25:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.878+0000 7f27e6a59740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc1abd30 con 0x558b1dd368d0 2024-04-22T13:25:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.878+0000 7f27e1c0c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558b1dd35b60 0x558b1dd376e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.878+0000 7f27e1c0c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558b1dd35b60 0x558b1dd376e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42776/0 (socket says 172.21.15.12:42776) 2024-04-22T13:25:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.878+0000 7f27e1c0c640 1 -- 172.21.15.12:0/1804547401 learned_addr learned my addr 172.21.15.12:0/1804547401 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.878+0000 7f27e140b640 1 --2- 172.21.15.12:0/1804547401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b1dd368d0 0x558b1dd4ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.878+0000 7f27e1c0c640 1 -- 172.21.15.12:0/1804547401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b1dd368d0 msgr2=0x558b1dd4ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.878+0000 7f27e1c0c640 1 --2- 172.21.15.12:0/1804547401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b1dd368d0 0x558b1dd4ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.878+0000 7f27e1c0c640 1 -- 172.21.15.12:0/1804547401 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558b1dd3f0e0 msgr2=0x558b1dd56d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.878+0000 7f27e240d640 1 --2- 172.21.15.12:0/1804547401 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558b1dd3f0e0 0x558b1dd56d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.878+0000 7f27e1c0c640 1 --2- 172.21.15.12:0/1804547401 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558b1dd3f0e0 0x558b1dd56d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.878+0000 7f27e1c0c640 1 -- 172.21.15.12:0/1804547401 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b1dd41840 con 0x558b1dd35b60 2024-04-22T13:25:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.878+0000 7f27e1c0c640 1 --2- 172.21.15.12:0/1804547401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558b1dd35b60 0x558b1dd376e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f27d8000c00 tx=0x7f27d80a07c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd0cdc1c3a6d2cc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.879+0000 7f27c77fe640 1 -- 172.21.15.12:0/1804547401 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d8020030 con 0x558b1dd35b60 2024-04-22T13:25:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.879+0000 7f27c77fe640 1 -- 172.21.15.12:0/1804547401 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27d80d0110 con 0x558b1dd35b60 2024-04-22T13:25:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.879+0000 7f27c77fe640 1 -- 172.21.15.12:0/1804547401 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d8098150 con 0x558b1dd35b60 2024-04-22T13:25:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.880+0000 7f27e6a59740 1 -- 172.21.15.12:0/1804547401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558b1dd35b60 msgr2=0x558b1dd376e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.880+0000 7f27e6a59740 1 --2- 172.21.15.12:0/1804547401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558b1dd35b60 0x558b1dd376e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f27d8000c00 tx=0x7f27d80a07c0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.880+0000 7f27e6a59740 1 -- 172.21.15.12:0/1804547401 shutdown_connections 2024-04-22T13:25:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.880+0000 7f27e6a59740 1 --2- 172.21.15.12:0/1804547401 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558b1dd3f0e0 0x558b1dd56d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.880+0000 7f27e6a59740 1 --2- 172.21.15.12:0/1804547401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b1dd368d0 0x558b1dd4ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.880+0000 7f27e6a59740 1 --2- 172.21.15.12:0/1804547401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558b1dd35b60 0x558b1dd376e0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.880+0000 7f27e6a59740 1 -- 172.21.15.12:0/1804547401 >> 172.21.15.12:0/1804547401 conn(0x558b1dcfcc90 msgr2=0x558b1dd36fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.880+0000 7f27e6a59740 1 -- 172.21.15.12:0/1804547401 shutdown_connections 2024-04-22T13:25:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.880+0000 7f27e6a59740 1 -- 172.21.15.12:0/1804547401 wait complete. 2024-04-22T13:25:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.880+0000 7f27e6a59740 1 -- start start 2024-04-22T13:25:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.880+0000 7f27e6a59740 1 Processor -- start 2024-04-22T13:25:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.881+0000 7f27e6a59740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558b1dd368d0 0x558b1dd47740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.881+0000 7f27e6a59740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558b1dd3f0e0 0x558b1dd388b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.881+0000 7f27e6a59740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b1dd38e40 0x558b1dd392a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.881+0000 7f27e6a59740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc1acad0 con 0x558b1dd368d0 2024-04-22T13:25:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.881+0000 7f27e6a59740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc1ac400 con 0x558b1dd3f0e0 2024-04-22T13:25:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.881+0000 7f27e6a59740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc1a9640 con 0x558b1dd38e40 2024-04-22T13:25:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.881+0000 7f27e1c0c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558b1dd368d0 0x558b1dd47740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.881+0000 7f27e6a59740 10 client.0 _ll_register_callbacks cb 0x7f27d809bf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.882+0000 7f27e1c0c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558b1dd368d0 0x558b1dd47740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42786/0 (socket says 172.21.15.12:42786) 2024-04-22T13:25:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.882+0000 7f27e1c0c640 1 -- 172.21.15.12:0/31643165 learned_addr learned my addr 172.21.15.12:0/31643165 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.882+0000 7f27e140b640 1 --2- 172.21.15.12:0/31643165 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558b1dd3f0e0 0x558b1dd388b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.882+0000 7f27e240d640 1 --2- 172.21.15.12:0/31643165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b1dd38e40 0x558b1dd392a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.882+0000 7f27e140b640 1 -- 172.21.15.12:0/31643165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b1dd38e40 msgr2=0x558b1dd392a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.882+0000 7f27e140b640 1 --2- 172.21.15.12:0/31643165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b1dd38e40 0x558b1dd392a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.882+0000 7f27e140b640 1 -- 172.21.15.12:0/31643165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558b1dd368d0 msgr2=0x558b1dd47740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.882+0000 7f27e140b640 1 --2- 172.21.15.12:0/31643165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558b1dd368d0 0x558b1dd47740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.882+0000 7f27e140b640 1 -- 172.21.15.12:0/31643165 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b1ddb47c0 con 0x558b1dd3f0e0 2024-04-22T13:25:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.882+0000 7f27e1c0c640 1 --2- 172.21.15.12:0/31643165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558b1dd368d0 0x558b1dd47740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.882+0000 7f27e140b640 1 --2- 172.21.15.12:0/31643165 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558b1dd3f0e0 0x558b1dd388b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f27cc02c170 tx=0x7f27cc005870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eece8e4d3a8a5059 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.882+0000 7f27c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-22T13:25:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.882+0000 7f27c57fa640 1 -- 172.21.15.12:0/31643165 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc02cbb0 con 0x558b1dd3f0e0 2024-04-22T13:25:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.882+0000 7f27c57fa640 1 -- 172.21.15.12:0/31643165 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f27cc005030 con 0x558b1dd3f0e0 2024-04-22T13:25:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.882+0000 7f27e6a59740 1 -- 172.21.15.12:0/31643165 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27d80d0110 con 0x558b1dd3f0e0 2024-04-22T13:25:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.883+0000 7f27c57fa640 1 -- 172.21.15.12:0/31643165 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f27cc056050 con 0x558b1dd3f0e0 2024-04-22T13:25:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.883+0000 7f27e6a59740 10 client.25742 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.883+0000 7f27e6a59740 1 -- 172.21.15.12:0/31643165 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558b1ddb5ad0 con 0x558b1dd3f0e0 2024-04-22T13:25:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.883+0000 7f27c57fa640 1 -- 172.21.15.12:0/31643165 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27d80d0110 con 0x558b1dd3f0e0 2024-04-22T13:25:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.884+0000 7f27c57fa640 1 -- 172.21.15.12:0/31643165 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f27cc0035f0 con 0x558b1dd3f0e0 2024-04-22T13:25:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.884+0000 7f27e6a59740 10 client.25742 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.884+0000 7f27e6a59740 10 client.25742 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:25:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.884+0000 7f27e6a59740 20 client.25742 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.884+0000 7f27e6a59740 1 --2- 172.21.15.12:0/31643165 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x558b1ddb6250 0x558b1ddb8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.884+0000 7f27e6a59740 4 client.25742 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-22T13:25:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.884+0000 7f27e6a59740 1 -- 172.21.15.12:0/31643165 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558b1dcae880 con 0x558b1ddb6250 2024-04-22T13:25:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.884+0000 7f27e1c0c640 1 --2- 172.21.15.12:0/31643165 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x558b1ddb6250 0x558b1ddb8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.885+0000 7f27e1c0c640 1 --2- 172.21.15.12:0/31643165 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x558b1ddb6250 0x558b1ddb8670 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0c17b2db9f939dc server_cookie=2b65f07ab586e2b9 in_seq=0 out_seq=0 2024-04-22T13:25:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.885+0000 7f27c57fa640 10 client.25742 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.889+0000 7f27c57fa640 1 -- 172.21.15.12:0/31643165 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558b1dcae880 con 0x558b1ddb6250 2024-04-22T13:25:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.889+0000 7f27c57fa640 10 client.25742 handle_command_reply: tid=0 2024-04-22T13:25:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.889+0000 7f27e6a59740 1 client.25742 shutdown 2024-04-22T13:25:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.889+0000 7f27e6a59740 1 -- 172.21.15.12:0/31643165 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558b1dd3f0e0 msgr2=0x558b1dd388b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.889+0000 7f27e6a59740 1 --2- 172.21.15.12:0/31643165 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558b1dd3f0e0 0x558b1dd388b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f27cc02c170 tx=0x7f27cc005870 comp rx=0 tx=0).stop 2024-04-22T13:25:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.889+0000 7f27e6a59740 1 -- 172.21.15.12:0/31643165 shutdown_connections 2024-04-22T13:25:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.889+0000 7f27e6a59740 1 --2- 172.21.15.12:0/31643165 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x558b1ddb6250 0x558b1ddb8670 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.889+0000 7f27e6a59740 1 --2- 172.21.15.12:0/31643165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558b1dd38e40 0x558b1dd392a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.889+0000 7f27e6a59740 1 --2- 172.21.15.12:0/31643165 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558b1dd3f0e0 0x558b1dd388b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.889+0000 7f27e6a59740 1 --2- 172.21.15.12:0/31643165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558b1dd368d0 0x558b1dd47740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.889+0000 7f27e6a59740 1 -- 172.21.15.12:0/31643165 >> 172.21.15.12:0/31643165 conn(0x558b1dcfcc90 msgr2=0x558b1dd3a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.889+0000 7f27c57fa640 0 client.25742 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.890+0000 7f27e6a59740 1 -- 172.21.15.12:0/31643165 shutdown_connections 2024-04-22T13:25:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.890+0000 7f27e6a59740 1 -- 172.21.15.12:0/31643165 wait complete. 2024-04-22T13:25:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.890+0000 7f27e6a59740 20 client.25742 trim_cache size 0 max 16384 2024-04-22T13:25:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-04-22T13:25:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-04-22T13:25:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-04-22T13:25:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-04-22T13:25:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.891+0000 7f27e3e70640 1 -- 172.21.15.12:0/2346173161 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f27c80a4070 msgr2=0x7f27c80a6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.891+0000 7f27e3e70640 1 --2- 172.21.15.12:0/2346173161 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f27c80a4070 0x7f27c80a6530 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f27d401f000 tx=0x7f27d401c8c0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.891+0000 7f27e3e70640 1 -- 172.21.15.12:0/2346173161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27dc1021b0 msgr2=0x7f27dc1984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.891+0000 7f27e3e70640 1 --2- 172.21.15.12:0/2346173161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27dc1021b0 0x7f27dc1984a0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f27d800d950 tx=0x7f27d800de10 comp rx=0 tx=0).stop 2024-04-22T13:25:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.891+0000 7f27e3e70640 1 -- 172.21.15.12:0/2346173161 shutdown_connections 2024-04-22T13:25:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.891+0000 7f27e3e70640 1 --2- 172.21.15.12:0/2346173161 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f27c80a4070 0x7f27c80a6530 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.891+0000 7f27e3e70640 1 --2- 172.21.15.12:0/2346173161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27dc19cd60 0x7f27dc19d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.891+0000 7f27e3e70640 1 --2- 172.21.15.12:0/2346173161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27dc107280 0x7f27dc1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.891+0000 7f27e3e70640 1 --2- 172.21.15.12:0/2346173161 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f27dc1021b0 0x7f27dc1984a0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.891+0000 7f27e3e70640 1 -- 172.21.15.12:0/2346173161 >> 172.21.15.12:0/2346173161 conn(0x7f27dc0fda10 msgr2=0x7f27dc0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.891+0000 7f27e3e70640 1 -- 172.21.15.12:0/2346173161 shutdown_connections 2024-04-22T13:25:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:33.891+0000 7f27e3e70640 1 -- 172.21.15.12:0/2346173161 wait complete. 2024-04-22T13:25:33.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:33 smithi111 ceph-mon[31361]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-22T13:25:34.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:33 smithi012 ceph-mon[28265]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-22T13:25:34.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:33 smithi134 ceph-mon[31338]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-22T13:25:35.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:35 smithi111 ceph-mon[31361]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:25:36.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:35 smithi012 ceph-mon[28265]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:25:36.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:35 smithi134 ceph-mon[31338]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:25:37.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:37 smithi111 ceph-mon[31361]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:25:38.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:37 smithi012 ceph-mon[28265]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:25:38.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:37 smithi134 ceph-mon[31338]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:25:39.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:39 smithi111 ceph-mon[31361]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-04-22T13:25:39.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:39 smithi111 ceph-mon[31361]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-22T13:25:40.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:39 smithi012 ceph-mon[28265]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-04-22T13:25:40.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:39 smithi012 ceph-mon[28265]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-22T13:25:40.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:39 smithi134 ceph-mon[31338]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-04-22T13:25:40.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:39 smithi134 ceph-mon[31338]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-22T13:25:41.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:41 smithi111 ceph-mon[31361]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-04-22T13:25:42.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:41 smithi012 ceph-mon[28265]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-04-22T13:25:42.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:41 smithi134 ceph-mon[31338]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-04-22T13:25:43.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:43 smithi111 ceph-mon[31361]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-04-22T13:25:44.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:43 smithi012 ceph-mon[28265]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-04-22T13:25:44.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:43 smithi134 ceph-mon[31338]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-04-22T13:25:45.938 INFO:tasks.workunit.client.0.smithi012.stderr:Updating files: 93% (11495/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-22T13:25:46.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:45 smithi012 ceph-mon[28265]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-04-22T13:25:46.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:45 smithi134 ceph-mon[31338]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-04-22T13:25:46.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:45 smithi111 ceph-mon[31361]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-04-22T13:25:46.830 INFO:tasks.workunit.client.0.smithi012.stderr:Note: switching to 'd69f13d827952bb07bb9b7973bca14f97d764d6f'. 2024-04-22T13:25:46.830 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-04-22T13:25:46.830 INFO:tasks.workunit.client.0.smithi012.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-22T13:25:46.830 INFO:tasks.workunit.client.0.smithi012.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-22T13:25:46.830 INFO:tasks.workunit.client.0.smithi012.stderr:state without impacting any branches by switching back to a branch. 2024-04-22T13:25:46.830 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-04-22T13:25:46.830 INFO:tasks.workunit.client.0.smithi012.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-22T13:25:46.830 INFO:tasks.workunit.client.0.smithi012.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-22T13:25:46.830 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-04-22T13:25:46.830 INFO:tasks.workunit.client.0.smithi012.stderr: git switch -c 2024-04-22T13:25:46.831 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-04-22T13:25:46.831 INFO:tasks.workunit.client.0.smithi012.stderr:Or undo this operation with: 2024-04-22T13:25:46.831 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-04-22T13:25:46.831 INFO:tasks.workunit.client.0.smithi012.stderr: git switch - 2024-04-22T13:25:46.831 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-04-22T13:25:46.831 INFO:tasks.workunit.client.0.smithi012.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-22T13:25:46.831 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-04-22T13:25:46.831 INFO:tasks.workunit.client.0.smithi012.stderr:HEAD is now at d69f13d8279 mds/cache: request_drop_foreign_locks shouldn't strip local xlocks 2024-04-22T13:25:46.836 DEBUG:teuthology.orchestra.run.smithi012:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-22T13:25:46.876 INFO:tasks.workunit.client.0.smithi012.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-22T13:25:46.878 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-22T13:25:46.878 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-22T13:25:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.984+0000 7f3f27055640 1 -- 172.21.15.12:0/1063469968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f20101b40 msgr2=0x7f3f2010dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.984+0000 7f3f27055640 1 --2- 172.21.15.12:0/1063469968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f20101b40 0x7f3f2010dc20 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f3f18009cd0 tx=0x7f3f1802c0e0 comp rx=0 tx=0).stop 2024-04-22T13:25:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.984+0000 7f3f27055640 1 -- 172.21.15.12:0/1063469968 shutdown_connections 2024-04-22T13:25:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.984+0000 7f3f27055640 1 --2- 172.21.15.12:0/1063469968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f20101b40 0x7f3f2010dc20 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.984+0000 7f3f27055640 1 --2- 172.21.15.12:0/1063469968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f201011a0 0x7f3f20101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.984+0000 7f3f27055640 1 --2- 172.21.15.12:0/1063469968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f20106fe0 0x7f3f201073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.984+0000 7f3f27055640 1 -- 172.21.15.12:0/1063469968 >> 172.21.15.12:0/1063469968 conn(0x7f3f20076b80 msgr2=0x7f3f20076f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.984+0000 7f3f27055640 1 -- 172.21.15.12:0/1063469968 shutdown_connections 2024-04-22T13:25:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.984+0000 7f3f27055640 1 -- 172.21.15.12:0/1063469968 wait complete. 2024-04-22T13:25:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.985+0000 7f3f27055640 1 Processor -- start 2024-04-22T13:25:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.985+0000 7f3f27055640 1 -- start start 2024-04-22T13:25:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.985+0000 7f3f27055640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f201011a0 0x7f3f20198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.985+0000 7f3f27055640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f20101b40 0x7f3f20198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.985+0000 7f3f27055640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f20106fe0 0x7f3f2019cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.985+0000 7f3f27055640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3f201110f0 con 0x7f3f201011a0 2024-04-22T13:25:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.985+0000 7f3f27055640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3f20110f70 con 0x7f3f20101b40 2024-04-22T13:25:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.985+0000 7f3f27055640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3f20111270 con 0x7f3f20106fe0 2024-04-22T13:25:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.985+0000 7f3f24df1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f201011a0 0x7f3f20198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.985+0000 7f3f24df1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f201011a0 0x7f3f20198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52862/0 (socket says 172.21.15.12:52862) 2024-04-22T13:25:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.985+0000 7f3f24df1640 1 -- 172.21.15.12:0/653202317 learned_addr learned my addr 172.21.15.12:0/653202317 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.985+0000 7f3f255f2640 1 --2- 172.21.15.12:0/653202317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f20106fe0 0x7f3f2019cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.985+0000 7f3f1ffff640 1 --2- 172.21.15.12:0/653202317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f20101b40 0x7f3f20198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.986+0000 7f3f24df1640 1 -- 172.21.15.12:0/653202317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f20106fe0 msgr2=0x7f3f2019cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.986+0000 7f3f24df1640 1 --2- 172.21.15.12:0/653202317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f20106fe0 0x7f3f2019cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.986+0000 7f3f24df1640 1 -- 172.21.15.12:0/653202317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f20101b40 msgr2=0x7f3f20198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.986+0000 7f3f24df1640 1 --2- 172.21.15.12:0/653202317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f20101b40 0x7f3f20198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.986+0000 7f3f24df1640 1 -- 172.21.15.12:0/653202317 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f2019d630 con 0x7f3f201011a0 2024-04-22T13:25:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.986+0000 7f3f1ffff640 1 --2- 172.21.15.12:0/653202317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f20101b40 0x7f3f20198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.986+0000 7f3f24df1640 1 --2- 172.21.15.12:0/653202317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f201011a0 0x7f3f20198640 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f3f1400d950 tx=0x7f3f1400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.986+0000 7f3f1dffb640 1 -- 172.21.15.12:0/653202317 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f14017070 con 0x7f3f201011a0 2024-04-22T13:25:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.986+0000 7f3f1dffb640 1 -- 172.21.15.12:0/653202317 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f14009c30 con 0x7f3f201011a0 2024-04-22T13:25:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.986+0000 7f3f27055640 1 -- 172.21.15.12:0/653202317 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f2019d920 con 0x7f3f201011a0 2024-04-22T13:25:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.986+0000 7f3f1dffb640 1 -- 172.21.15.12:0/653202317 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1400f620 con 0x7f3f201011a0 2024-04-22T13:25:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.986+0000 7f3f27055640 1 -- 172.21.15.12:0/653202317 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f201a51b0 con 0x7f3f201011a0 2024-04-22T13:25:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.988+0000 7f3f1dffb640 1 -- 172.21.15.12:0/653202317 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3f14010460 con 0x7f3f201011a0 2024-04-22T13:25:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.989+0000 7f3f29c47740 1 Processor -- start 2024-04-22T13:25:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.989+0000 7f3f1dffb640 1 --2- 172.21.15.12:0/653202317 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3f100a3dc0 0x7f3f100a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.989+0000 7f3f29c47740 1 -- start start 2024-04-22T13:25:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.989+0000 7f3f255f2640 1 --2- 172.21.15.12:0/653202317 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3f100a3dc0 0x7f3f100a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.989+0000 7f3f1dffb640 1 -- 172.21.15.12:0/653202317 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f3f140d0410 con 0x7f3f201011a0 2024-04-22T13:25:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.989+0000 7f3f29c47740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0fc8ce8d0 0x55d0fc8cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.989+0000 7f3f29c47740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0fc8cdb60 0x55d0fc8cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.989+0000 7f3f29c47740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0fc8db4f0 0x55d0fc8dd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.989+0000 7f3f29c47740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d0fc8328d0 con 0x55d0fc8cdb60 2024-04-22T13:25:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.989+0000 7f3f29c47740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d0fc7cfe80 con 0x55d0fc8db4f0 2024-04-22T13:25:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.989+0000 7f3f29c47740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d0fc82e890 con 0x55d0fc8ce8d0 2024-04-22T13:25:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.989+0000 7f3f1ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0fc8cdb60 0x55d0fc8cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.989+0000 7f3f1ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0fc8cdb60 0x55d0fc8cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52872/0 (socket says 172.21.15.12:52872) 2024-04-22T13:25:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.989+0000 7f3f1ffff640 1 -- 172.21.15.12:0/1641836049 learned_addr learned my addr 172.21.15.12:0/1641836049 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.989+0000 7f3f255f2640 1 --2- 172.21.15.12:0/1641836049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0fc8db4f0 0x55d0fc8dd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.989+0000 7f3f24df1640 1 --2- 172.21.15.12:0/1641836049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0fc8ce8d0 0x55d0fc8cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.989+0000 7f3f1ffff640 1 -- 172.21.15.12:0/1641836049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0fc8ce8d0 msgr2=0x55d0fc8cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.989+0000 7f3f1ffff640 1 --2- 172.21.15.12:0/1641836049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0fc8ce8d0 0x55d0fc8cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.989+0000 7f3f1ffff640 1 -- 172.21.15.12:0/1641836049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0fc8db4f0 msgr2=0x55d0fc8dd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.989+0000 7f3f255f2640 1 --2- 172.21.15.12:0/653202317 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3f100a3dc0 0x7f3f100a6280 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3f18005cc0 tx=0x7f3f18009900 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.989+0000 7f3f1ffff640 1 --2- 172.21.15.12:0/1641836049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0fc8db4f0 0x55d0fc8dd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.989+0000 7f3f1ffff640 1 -- 172.21.15.12:0/1641836049 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0fc7f1320 con 0x55d0fc8cdb60 2024-04-22T13:25:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.989+0000 7f3f24df1640 1 --2- 172.21.15.12:0/1641836049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0fc8ce8d0 0x55d0fc8cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:25:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.990+0000 7f3f1ffff640 1 --2- 172.21.15.12:0/1641836049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0fc8cdb60 0x55d0fc8cdfe0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009ce0 tx=0x7f3f0c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9569f8a46b076c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.990+0000 7f3f0affd640 1 -- 172.21.15.12:0/1641836049 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c012450 con 0x55d0fc8cdb60 2024-04-22T13:25:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.990+0000 7f3f0affd640 1 -- 172.21.15.12:0/1641836049 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f0c012aa0 con 0x55d0fc8cdb60 2024-04-22T13:25:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.990+0000 7f3f0affd640 1 -- 172.21.15.12:0/1641836049 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c00fb70 con 0x55d0fc8cdb60 2024-04-22T13:25:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.990+0000 7f3f29c47740 1 -- 172.21.15.12:0/1641836049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0fc8cdb60 msgr2=0x55d0fc8cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.990+0000 7f3f29c47740 1 --2- 172.21.15.12:0/1641836049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0fc8cdb60 0x55d0fc8cdfe0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009ce0 tx=0x7f3f0c0117b0 comp rx=0 tx=0).stop 2024-04-22T13:25:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.990+0000 7f3f29c47740 1 -- 172.21.15.12:0/1641836049 shutdown_connections 2024-04-22T13:25:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.991+0000 7f3f29c47740 1 --2- 172.21.15.12:0/1641836049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0fc8db4f0 0x55d0fc8dd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.991+0000 7f3f29c47740 1 --2- 172.21.15.12:0/1641836049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0fc8cdb60 0x55d0fc8cdfe0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.991+0000 7f3f29c47740 1 --2- 172.21.15.12:0/1641836049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0fc8ce8d0 0x55d0fc8cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.991+0000 7f3f29c47740 1 -- 172.21.15.12:0/1641836049 >> 172.21.15.12:0/1641836049 conn(0x55d0fc894c90 msgr2=0x55d0fc8c8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.991+0000 7f3f29c47740 1 -- 172.21.15.12:0/1641836049 shutdown_connections 2024-04-22T13:25:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.991+0000 7f3f29c47740 1 -- 172.21.15.12:0/1641836049 wait complete. 2024-04-22T13:25:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.991+0000 7f3f29c47740 1 -- start start 2024-04-22T13:25:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.991+0000 7f3f29c47740 1 Processor -- start 2024-04-22T13:25:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.991+0000 7f3f29c47740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0fc8cdb60 0x55d0fc8dae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.991+0000 7f3f29c47740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0fc8ce8d0 0x55d0fc8d81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.991+0000 7f3f29c47740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0fc8db4f0 0x55d0fc8d8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.991+0000 7f3f29c47740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d0fc8de220 con 0x55d0fc8cdb60 2024-04-22T13:25:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.991+0000 7f3f29c47740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d0fc7baeb0 con 0x55d0fc8db4f0 2024-04-22T13:25:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.991+0000 7f3f29c47740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d0fc8de3a0 con 0x55d0fc8ce8d0 2024-04-22T13:25:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.991+0000 7f3f29c47740 10 client.0 _ll_register_callbacks cb 0x55d0fc7e7b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.991+0000 7f3f24df1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0fc8cdb60 0x55d0fc8dae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.992+0000 7f3f24df1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0fc8cdb60 0x55d0fc8dae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52884/0 (socket says 172.21.15.12:52884) 2024-04-22T13:25:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.992+0000 7f3f24df1640 1 -- 172.21.15.12:0/611720857 learned_addr learned my addr 172.21.15.12:0/611720857 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.992+0000 7f3f255f2640 1 --2- 172.21.15.12:0/611720857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0fc8db4f0 0x55d0fc8d8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.992+0000 7f3f1ffff640 1 --2- 172.21.15.12:0/611720857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0fc8ce8d0 0x55d0fc8d81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.992+0000 7f3f24df1640 1 -- 172.21.15.12:0/611720857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0fc8ce8d0 msgr2=0x55d0fc8d81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.992+0000 7f3f24df1640 1 --2- 172.21.15.12:0/611720857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0fc8ce8d0 0x55d0fc8d81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.992+0000 7f3f24df1640 1 -- 172.21.15.12:0/611720857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0fc8db4f0 msgr2=0x55d0fc8d8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.992+0000 7f3f24df1640 1 --2- 172.21.15.12:0/611720857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0fc8db4f0 0x55d0fc8d8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.992+0000 7f3f24df1640 1 -- 172.21.15.12:0/611720857 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0fc545210 con 0x55d0fc8cdb60 2024-04-22T13:25:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.992+0000 7f3f24df1640 1 --2- 172.21.15.12:0/611720857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0fc8cdb60 0x55d0fc8dae50 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3f1409c390 tx=0x7f3f1409cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73e723bd481c0c69 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.992+0000 7f3efffff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.992+0000 7f3efffff640 1 -- 172.21.15.12:0/611720857 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f14020040 con 0x55d0fc8cdb60 2024-04-22T13:25:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.992+0000 7f3efffff640 1 -- 172.21.15.12:0/611720857 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f3f1409b850 con 0x55d0fc8cdb60 2024-04-22T13:25:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.992+0000 7f3f29c47740 1 -- 172.21.15.12:0/611720857 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f0c012aa0 con 0x55d0fc8cdb60 2024-04-22T13:25:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.992+0000 7f3efffff640 1 -- 172.21.15.12:0/611720857 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3f140a5070 con 0x55d0fc8cdb60 2024-04-22T13:25:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.992+0000 7f3f29c47740 10 client.16326 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.992+0000 7f3f29c47740 1 -- 172.21.15.12:0/611720857 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d0fc611620 con 0x55d0fc8cdb60 2024-04-22T13:25:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.992+0000 7f3efffff640 1 -- 172.21.15.12:0/611720857 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f0c012aa0 con 0x55d0fc8cdb60 2024-04-22T13:25:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.993+0000 7f3efffff640 1 -- 172.21.15.12:0/611720857 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3f14010750 con 0x55d0fc8cdb60 2024-04-22T13:25:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.993+0000 7f3f29c47740 10 client.16326 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.993+0000 7f3f29c47740 10 client.16326 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:25:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.993+0000 7f3f29c47740 20 client.16326 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.993+0000 7f3f29c47740 1 --2- 172.21.15.12:0/611720857 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d0fc8eb440 0x55d0fc8ed870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.993+0000 7f3f29c47740 4 client.16326 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:25:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.993+0000 7f3f29c47740 1 -- 172.21.15.12:0/611720857 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0fc846880 con 0x55d0fc8eb440 2024-04-22T13:25:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.993+0000 7f3f1ffff640 1 --2- 172.21.15.12:0/611720857 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d0fc8eb440 0x55d0fc8ed870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.993+0000 7f3efffff640 10 client.16326 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.993+0000 7f3f1ffff640 1 --2- 172.21.15.12:0/611720857 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d0fc8eb440 0x55d0fc8ed870 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c169d889128be32 server_cookie=7c9a96e51fe8d6e6 in_seq=0 out_seq=0 2024-04-22T13:25:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.994+0000 7f3efffff640 1 -- 172.21.15.12:0/611720857 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d0fc846880 con 0x55d0fc8eb440 2024-04-22T13:25:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.994+0000 7f3efffff640 10 client.16326 handle_command_reply: tid=0 2024-04-22T13:25:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.994+0000 7f3f29c47740 1 client.16326 shutdown 2024-04-22T13:25:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.994+0000 7f3f29c47740 1 -- 172.21.15.12:0/611720857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0fc8cdb60 msgr2=0x55d0fc8dae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.994+0000 7f3f29c47740 1 --2- 172.21.15.12:0/611720857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0fc8cdb60 0x55d0fc8dae50 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3f1409c390 tx=0x7f3f1409cc40 comp rx=0 tx=0).stop 2024-04-22T13:25:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.994+0000 7f3f29c47740 1 -- 172.21.15.12:0/611720857 shutdown_connections 2024-04-22T13:25:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.994+0000 7f3f29c47740 1 --2- 172.21.15.12:0/611720857 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d0fc8eb440 0x55d0fc8ed870 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.994+0000 7f3f29c47740 1 --2- 172.21.15.12:0/611720857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0fc8db4f0 0x55d0fc8d8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.994+0000 7f3f29c47740 1 --2- 172.21.15.12:0/611720857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0fc8ce8d0 0x55d0fc8d81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.994+0000 7f3efffff640 0 client.16326 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.994+0000 7f3f29c47740 1 --2- 172.21.15.12:0/611720857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0fc8cdb60 0x55d0fc8dae50 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.994+0000 7f3f29c47740 1 -- 172.21.15.12:0/611720857 >> 172.21.15.12:0/611720857 conn(0x55d0fc894c90 msgr2=0x55d0fc8ca5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.994+0000 7f3f29c47740 1 -- 172.21.15.12:0/611720857 shutdown_connections 2024-04-22T13:25:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.994+0000 7f3f29c47740 1 -- 172.21.15.12:0/611720857 wait complete. 2024-04-22T13:25:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:46.995+0000 7f3f29c47740 20 client.16326 trim_cache size 0 max 16384 2024-04-22T13:25:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.009+0000 7f3f29c47740 1 Processor -- start 2024-04-22T13:25:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.009+0000 7f3f29c47740 1 -- start start 2024-04-22T13:25:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.010+0000 7f3f29c47740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0fc8cdb60 0x55d0fc8e7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.010+0000 7f3f29c47740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0fc8ce8d0 0x55d0fc8c3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.010+0000 7f3f29c47740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0fc8db4f0 0x55d0fc8da910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.010+0000 7f3f29c47740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3f201ab710 con 0x55d0fc8db4f0 2024-04-22T13:25:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.010+0000 7f3f29c47740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3f201ad250 con 0x55d0fc8ce8d0 2024-04-22T13:25:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.010+0000 7f3f29c47740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3f201abde0 con 0x55d0fc8cdb60 2024-04-22T13:25:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.010+0000 7f3f255f2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0fc8db4f0 0x55d0fc8da910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.010+0000 7f3f255f2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0fc8db4f0 0x55d0fc8da910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52888/0 (socket says 172.21.15.12:52888) 2024-04-22T13:25:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.010+0000 7f3f255f2640 1 -- 172.21.15.12:0/2557341659 learned_addr learned my addr 172.21.15.12:0/2557341659 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.010+0000 7f3f1ffff640 1 --2- 172.21.15.12:0/2557341659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0fc8ce8d0 0x55d0fc8c3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.010+0000 7f3f24df1640 1 --2- 172.21.15.12:0/2557341659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0fc8cdb60 0x55d0fc8e7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.010+0000 7f3f255f2640 1 -- 172.21.15.12:0/2557341659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0fc8cdb60 msgr2=0x55d0fc8e7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.010+0000 7f3f255f2640 1 --2- 172.21.15.12:0/2557341659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0fc8cdb60 0x55d0fc8e7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.010+0000 7f3f255f2640 1 -- 172.21.15.12:0/2557341659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0fc8ce8d0 msgr2=0x55d0fc8c3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.010+0000 7f3f255f2640 1 --2- 172.21.15.12:0/2557341659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0fc8ce8d0 0x55d0fc8c3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.010+0000 7f3f255f2640 1 -- 172.21.15.12:0/2557341659 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0fc8c3f30 con 0x55d0fc8db4f0 2024-04-22T13:25:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.010+0000 7f3f1ffff640 1 --2- 172.21.15.12:0/2557341659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0fc8ce8d0 0x55d0fc8c3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:25:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.010+0000 7f3f24df1640 1 --2- 172.21.15.12:0/2557341659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0fc8cdb60 0x55d0fc8e7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:25:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.010+0000 7f3f255f2640 1 --2- 172.21.15.12:0/2557341659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0fc8db4f0 0x55d0fc8da910 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f3f1802c6f0 tx=0x7f3f18040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554ae52ffaef2df6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.010+0000 7f3f0a7fc640 1 -- 172.21.15.12:0/2557341659 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f18039040 con 0x55d0fc8db4f0 2024-04-22T13:25:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.010+0000 7f3f0a7fc640 1 -- 172.21.15.12:0/2557341659 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f18047070 con 0x55d0fc8db4f0 2024-04-22T13:25:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.010+0000 7f3f0a7fc640 1 -- 172.21.15.12:0/2557341659 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f180346a0 con 0x55d0fc8db4f0 2024-04-22T13:25:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.011+0000 7f3f29c47740 1 -- 172.21.15.12:0/2557341659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0fc8db4f0 msgr2=0x55d0fc8da910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.011+0000 7f3f29c47740 1 --2- 172.21.15.12:0/2557341659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0fc8db4f0 0x55d0fc8da910 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f3f1802c6f0 tx=0x7f3f18040000 comp rx=0 tx=0).stop 2024-04-22T13:25:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.011+0000 7f3f29c47740 1 -- 172.21.15.12:0/2557341659 shutdown_connections 2024-04-22T13:25:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.011+0000 7f3f29c47740 1 --2- 172.21.15.12:0/2557341659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0fc8db4f0 0x55d0fc8da910 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.011+0000 7f3f29c47740 1 --2- 172.21.15.12:0/2557341659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0fc8ce8d0 0x55d0fc8c3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.011+0000 7f3f29c47740 1 --2- 172.21.15.12:0/2557341659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0fc8cdb60 0x55d0fc8e7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.011+0000 7f3f29c47740 1 -- 172.21.15.12:0/2557341659 >> 172.21.15.12:0/2557341659 conn(0x55d0fc894c90 msgr2=0x55d0fc8d6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.011+0000 7f3f29c47740 1 -- 172.21.15.12:0/2557341659 shutdown_connections 2024-04-22T13:25:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.011+0000 7f3f29c47740 1 -- 172.21.15.12:0/2557341659 wait complete. 2024-04-22T13:25:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.011+0000 7f3f29c47740 1 -- start start 2024-04-22T13:25:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.011+0000 7f3f29c47740 1 Processor -- start 2024-04-22T13:25:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.012+0000 7f3f29c47740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0fc8cdb60 0x55d0fc8d5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.012+0000 7f3f29c47740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0fc8ce8d0 0x55d0fc8e3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.012+0000 7f3f29c47740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0fc8db4f0 0x55d0fc8dc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.012+0000 7f3f29c47740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3f201acb80 con 0x55d0fc8ce8d0 2024-04-22T13:25:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.012+0000 7f3f29c47740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3f201ac4b0 con 0x55d0fc8db4f0 2024-04-22T13:25:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.012+0000 7f3f29c47740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3f201a96f0 con 0x55d0fc8cdb60 2024-04-22T13:25:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.012+0000 7f3f29c47740 10 client.0 _ll_register_callbacks cb 0x7f3f1409bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.012+0000 7f3f1ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0fc8ce8d0 0x55d0fc8e3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.012+0000 7f3f1ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0fc8ce8d0 0x55d0fc8e3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52896/0 (socket says 172.21.15.12:52896) 2024-04-22T13:25:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.012+0000 7f3f1ffff640 1 -- 172.21.15.12:0/4127975843 learned_addr learned my addr 172.21.15.12:0/4127975843 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.012+0000 7f3f255f2640 1 --2- 172.21.15.12:0/4127975843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0fc8db4f0 0x55d0fc8dc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.012+0000 7f3f24df1640 1 --2- 172.21.15.12:0/4127975843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0fc8cdb60 0x55d0fc8d5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.012+0000 7f3f1ffff640 1 -- 172.21.15.12:0/4127975843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0fc8cdb60 msgr2=0x55d0fc8d5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.012+0000 7f3f1ffff640 1 --2- 172.21.15.12:0/4127975843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0fc8cdb60 0x55d0fc8d5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.012+0000 7f3f1ffff640 1 -- 172.21.15.12:0/4127975843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0fc8db4f0 msgr2=0x55d0fc8dc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.012+0000 7f3f1ffff640 1 --2- 172.21.15.12:0/4127975843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0fc8db4f0 0x55d0fc8dc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.012+0000 7f3f1ffff640 1 -- 172.21.15.12:0/4127975843 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0fc8dc9a0 con 0x55d0fc8ce8d0 2024-04-22T13:25:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.012+0000 7f3f24df1640 1 --2- 172.21.15.12:0/4127975843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0fc8cdb60 0x55d0fc8d5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.012+0000 7f3f1ffff640 1 --2- 172.21.15.12:0/4127975843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0fc8ce8d0 0x55d0fc8e3630 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c00f700 tx=0x7f3f0c009400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c8c050f2747424 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.012+0000 7f3efffff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.013+0000 7f3efffff640 1 -- 172.21.15.12:0/4127975843 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c02f070 con 0x55d0fc8ce8d0 2024-04-22T13:25:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.013+0000 7f3efffff640 1 -- 172.21.15.12:0/4127975843 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f3f0c005240 con 0x55d0fc8ce8d0 2024-04-22T13:25:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.013+0000 7f3f29c47740 1 -- 172.21.15.12:0/4127975843 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f18047070 con 0x55d0fc8ce8d0 2024-04-22T13:25:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.013+0000 7f3efffff640 1 -- 172.21.15.12:0/4127975843 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3f0c03b050 con 0x55d0fc8ce8d0 2024-04-22T13:25:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.013+0000 7f3f29c47740 10 client.16332 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.013+0000 7f3f29c47740 1 -- 172.21.15.12:0/4127975843 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d0fc952080 con 0x55d0fc8ce8d0 2024-04-22T13:25:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.013+0000 7f3efffff640 1 -- 172.21.15.12:0/4127975843 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f18047070 con 0x55d0fc8ce8d0 2024-04-22T13:25:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.013+0000 7f3efffff640 1 -- 172.21.15.12:0/4127975843 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3f0c02b1d0 con 0x55d0fc8ce8d0 2024-04-22T13:25:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.013+0000 7f3f29c47740 10 client.16332 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.013+0000 7f3f29c47740 10 client.16332 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:25:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.013+0000 7f3f29c47740 20 client.16332 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.013+0000 7f3f29c47740 1 --2- 172.21.15.12:0/4127975843 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d0fc952670 0x55d0fc954a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.013+0000 7f3f29c47740 4 client.16332 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-22T13:25:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.013+0000 7f3f29c47740 1 -- 172.21.15.12:0/4127975843 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d0fc7d23d0 con 0x55d0fc952670 2024-04-22T13:25:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.013+0000 7f3f24df1640 1 --2- 172.21.15.12:0/4127975843 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d0fc952670 0x55d0fc954a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.014+0000 7f3f24df1640 1 --2- 172.21.15.12:0/4127975843 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d0fc952670 0x55d0fc954a90 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0c5f98167eb2d0b server_cookie=3a6c891bed89b552 in_seq=0 out_seq=0 2024-04-22T13:25:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.014+0000 7f3efffff640 10 client.16332 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.019+0000 7f3efffff640 1 -- 172.21.15.12:0/4127975843 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d0fc7d23d0 con 0x55d0fc952670 2024-04-22T13:25:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.019+0000 7f3efffff640 10 client.16332 handle_command_reply: tid=0 2024-04-22T13:25:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.019+0000 7f3f29c47740 1 client.16332 shutdown 2024-04-22T13:25:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.020+0000 7f3f29c47740 1 -- 172.21.15.12:0/4127975843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0fc8ce8d0 msgr2=0x55d0fc8e3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.020+0000 7f3f29c47740 1 --2- 172.21.15.12:0/4127975843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0fc8ce8d0 0x55d0fc8e3630 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c00f700 tx=0x7f3f0c009400 comp rx=0 tx=0).stop 2024-04-22T13:25:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.020+0000 7f3f29c47740 1 -- 172.21.15.12:0/4127975843 shutdown_connections 2024-04-22T13:25:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.020+0000 7f3f29c47740 1 --2- 172.21.15.12:0/4127975843 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d0fc952670 0x55d0fc954a90 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.020+0000 7f3f29c47740 1 --2- 172.21.15.12:0/4127975843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d0fc8db4f0 0x55d0fc8dc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.020+0000 7f3f29c47740 1 --2- 172.21.15.12:0/4127975843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d0fc8ce8d0 0x55d0fc8e3630 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.020+0000 7f3efffff640 0 client.16332 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.020+0000 7f3f29c47740 1 --2- 172.21.15.12:0/4127975843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d0fc8cdb60 0x55d0fc8d5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.020+0000 7f3f29c47740 1 -- 172.21.15.12:0/4127975843 >> 172.21.15.12:0/4127975843 conn(0x55d0fc894c90 msgr2=0x55d0fc8c9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.020+0000 7f3f29c47740 1 -- 172.21.15.12:0/4127975843 shutdown_connections 2024-04-22T13:25:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.020+0000 7f3f29c47740 1 -- 172.21.15.12:0/4127975843 wait complete. 2024-04-22T13:25:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.020+0000 7f3f29c47740 20 client.16332 trim_cache size 0 max 16384 2024-04-22T13:25:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-04-22T13:25:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-04-22T13:25:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-04-22T13:25:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-04-22T13:25:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.020+0000 7f3f27055640 1 -- 172.21.15.12:0/653202317 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3f100a3dc0 msgr2=0x7f3f100a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.020+0000 7f3f27055640 1 --2- 172.21.15.12:0/653202317 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3f100a3dc0 0x7f3f100a6280 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3f18005cc0 tx=0x7f3f18009900 comp rx=0 tx=0).stop 2024-04-22T13:25:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.020+0000 7f3f27055640 1 -- 172.21.15.12:0/653202317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f201011a0 msgr2=0x7f3f20198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.020+0000 7f3f27055640 1 --2- 172.21.15.12:0/653202317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f201011a0 0x7f3f20198640 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f3f1400d950 tx=0x7f3f1400de10 comp rx=0 tx=0).stop 2024-04-22T13:25:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.021+0000 7f3f27055640 1 -- 172.21.15.12:0/653202317 shutdown_connections 2024-04-22T13:25:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.021+0000 7f3f27055640 1 --2- 172.21.15.12:0/653202317 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3f100a3dc0 0x7f3f100a6280 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.021+0000 7f3f27055640 1 --2- 172.21.15.12:0/653202317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f20106fe0 0x7f3f2019cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.021+0000 7f3f27055640 1 --2- 172.21.15.12:0/653202317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f20101b40 0x7f3f20198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.021+0000 7f3f27055640 1 --2- 172.21.15.12:0/653202317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3f201011a0 0x7f3f20198640 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.021+0000 7f3f27055640 1 -- 172.21.15.12:0/653202317 >> 172.21.15.12:0/653202317 conn(0x7f3f20076b80 msgr2=0x7f3f20105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.021+0000 7f3f27055640 1 -- 172.21.15.12:0/653202317 shutdown_connections 2024-04-22T13:25:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:25:47.021+0000 7f3f27055640 1 -- 172.21.15.12:0/653202317 wait complete. 2024-04-22T13:25:47.479 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-22T13:25:47.534 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-22T13:25:47.575 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-22T13:25:47.577 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-22T13:25:47.577 INFO:tasks.workunit.client.0.smithi012.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-22T13:25:47.616 INFO:tasks.workunit.client.0.smithi012.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-22T13:25:47.620 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-22T13:25:47.620 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-22T13:25:47.676 INFO:tasks.workunit:Running workunits matching suites/fsync-tester.sh on client.0... 2024-04-22T13:25:47.677 INFO:tasks.workunit:Running workunit suites/fsync-tester.sh... 2024-04-22T13:25:47.678 DEBUG:teuthology.orchestra.run.smithi012:workunit test suites/fsync-tester.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d69f13d827952bb07bb9b7973bca14f97d764d6f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-04-22T13:25:47.738 INFO:tasks.workunit.client.0.smithi012.stderr:+ wget http://download.ceph.com/qa/fsync-tester.c 2024-04-22T13:25:47.759 INFO:tasks.workunit.client.0.smithi012.stderr:--2024-04-22 13:25:47-- http://download.ceph.com/qa/fsync-tester.c 2024-04-22T13:25:47.760 INFO:tasks.workunit.client.0.smithi012.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-04-22T13:25:47.781 INFO:tasks.workunit.client.0.smithi012.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-04-22T13:25:47.802 INFO:tasks.workunit.client.0.smithi012.stderr:HTTP request sent, awaiting response... 200 OK 2024-04-22T13:25:47.802 INFO:tasks.workunit.client.0.smithi012.stderr:Length: 3024 (3.0K) [application/octet-stream] 2024-04-22T13:25:47.804 INFO:tasks.workunit.client.0.smithi012.stderr:Saving to: ‘fsync-tester.c’ 2024-04-22T13:25:47.807 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-04-22T13:25:47.807 INFO:tasks.workunit.client.0.smithi012.stderr: 0K .. 100% 212M=0s 2024-04-22T13:25:47.807 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-04-22T13:25:47.807 INFO:tasks.workunit.client.0.smithi012.stderr:2024-04-22 13:25:47 (212 MB/s) - ‘fsync-tester.c’ saved [3024/3024] 2024-04-22T13:25:47.807 INFO:tasks.workunit.client.0.smithi012.stderr: 2024-04-22T13:25:47.807 INFO:tasks.workunit.client.0.smithi012.stderr:+ gcc -D_GNU_SOURCE fsync-tester.c -o fsync-tester 2024-04-22T13:25:47.878 INFO:tasks.workunit.client.0.smithi012.stderr:+ ./fsync-tester 2024-04-22T13:25:48.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:47 smithi012 ceph-mon[28265]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-04-22T13:25:48.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:47 smithi134 ceph-mon[31338]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-04-22T13:25:48.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:47 smithi111 ceph-mon[31361]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-04-22T13:25:49.104 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:25:49.104 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:25:49.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.217+0000 7f5a3510d640 1 -- 172.21.15.12:0/394288732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a30103ae0 msgr2=0x7f5a3010bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.217+0000 7f5a3510d640 1 --2- 172.21.15.12:0/394288732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a30103ae0 0x7f5a3010bfc0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5a18009900 tx=0x7f5a1802bd60 comp rx=0 tx=0).stop 2024-04-22T13:25:49.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.218+0000 7f5a3510d640 1 -- 172.21.15.12:0/394288732 shutdown_connections 2024-04-22T13:25:49.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.218+0000 7f5a3510d640 1 --2- 172.21.15.12:0/394288732 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a3010c500 0x7f5a3010e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.218+0000 7f5a3510d640 1 --2- 172.21.15.12:0/394288732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a30103ae0 0x7f5a3010bfc0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.218+0000 7f5a3510d640 1 --2- 172.21.15.12:0/394288732 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a301031c0 0x7f5a301035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.218+0000 7f5a3510d640 1 -- 172.21.15.12:0/394288732 >> 172.21.15.12:0/394288732 conn(0x7f5a30078000 msgr2=0x7f5a30078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:49.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.218+0000 7f5a3510d640 1 -- 172.21.15.12:0/394288732 shutdown_connections 2024-04-22T13:25:49.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.218+0000 7f5a3510d640 1 -- 172.21.15.12:0/394288732 wait complete. 2024-04-22T13:25:49.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.218+0000 7f5a3510d640 1 Processor -- start 2024-04-22T13:25:49.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.219+0000 7f5a3510d640 1 -- start start 2024-04-22T13:25:49.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.219+0000 7f5a3510d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a301031c0 0x7f5a301a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.219+0000 7f5a3510d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a30103ae0 0x7f5a301a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.219+0000 7f5a3510d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a3010c500 0x7f5a3019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.219+0000 7f5a3510d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5a30112190 con 0x7f5a301031c0 2024-04-22T13:25:49.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.219+0000 7f5a3510d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5a30112010 con 0x7f5a30103ae0 2024-04-22T13:25:49.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.219+0000 7f5a3510d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5a30112310 con 0x7f5a3010c500 2024-04-22T13:25:49.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.219+0000 7f5a2ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a301031c0 0x7f5a301a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.219+0000 7f5a2ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a301031c0 0x7f5a301a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52908/0 (socket says 172.21.15.12:52908) 2024-04-22T13:25:49.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.219+0000 7f5a2ed9d640 1 -- 172.21.15.12:0/630895781 learned_addr learned my addr 172.21.15.12:0/630895781 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:49.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.219+0000 7f5a2e59c640 1 --2- 172.21.15.12:0/630895781 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a30103ae0 0x7f5a301a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.219+0000 7f5a2f59e640 1 --2- 172.21.15.12:0/630895781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a3010c500 0x7f5a3019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.219+0000 7f5a2ed9d640 1 -- 172.21.15.12:0/630895781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a3010c500 msgr2=0x7f5a3019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.219+0000 7f5a2ed9d640 1 --2- 172.21.15.12:0/630895781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a3010c500 0x7f5a3019a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.219+0000 7f5a2ed9d640 1 -- 172.21.15.12:0/630895781 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a30103ae0 msgr2=0x7f5a301a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.219+0000 7f5a2ed9d640 1 --2- 172.21.15.12:0/630895781 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a30103ae0 0x7f5a301a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.219+0000 7f5a2ed9d640 1 -- 172.21.15.12:0/630895781 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a3019aff0 con 0x7f5a301031c0 2024-04-22T13:25:49.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.220+0000 7f5a2ed9d640 1 --2- 172.21.15.12:0/630895781 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a301031c0 0x7f5a301a0670 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000c970 tx=0x7f5a2000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:49.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.220+0000 7f5a27fff640 1 -- 172.21.15.12:0/630895781 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20007bb0 con 0x7f5a301031c0 2024-04-22T13:25:49.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.220+0000 7f5a3510d640 1 -- 172.21.15.12:0/630895781 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a3019b2e0 con 0x7f5a301031c0 2024-04-22T13:25:49.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.220+0000 7f5a27fff640 1 -- 172.21.15.12:0/630895781 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a20007d50 con 0x7f5a301031c0 2024-04-22T13:25:49.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.220+0000 7f5a27fff640 1 -- 172.21.15.12:0/630895781 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2000f450 con 0x7f5a301031c0 2024-04-22T13:25:49.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.220+0000 7f5a3510d640 1 -- 172.21.15.12:0/630895781 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a301a75f0 con 0x7f5a301031c0 2024-04-22T13:25:49.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.222+0000 7f5a27fff640 1 -- 172.21.15.12:0/630895781 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5a2000f5f0 con 0x7f5a301031c0 2024-04-22T13:25:49.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.223+0000 7f5a37dd0740 1 Processor -- start 2024-04-22T13:25:49.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.223+0000 7f5a37dd0740 1 -- start start 2024-04-22T13:25:49.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.223+0000 7f5a27fff640 1 --2- 172.21.15.12:0/630895781 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5a1c0a3d20 0x7f5a1c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.223+0000 7f5a37dd0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556885d2580 0x5556885d2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.223+0000 7f5a37dd0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556885d2ea0 0x5556885db540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.223+0000 7f5a37dd0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556885dba80 0x5556885dde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.223+0000 7f5a37dd0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5556885371e0 con 0x5556885dba80 2024-04-22T13:25:49.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.223+0000 7f5a37dd0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5556884d4390 con 0x5556885d2ea0 2024-04-22T13:25:49.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.223+0000 7f5a37dd0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5556885331a0 con 0x5556885d2580 2024-04-22T13:25:49.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.223+0000 7f5a2f59e640 1 --2- 172.21.15.12:0/630895781 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5a1c0a3d20 0x7f5a1c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.223+0000 7f5a27fff640 1 -- 172.21.15.12:0/630895781 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f5a200d0480 con 0x7f5a301031c0 2024-04-22T13:25:49.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.223+0000 7f5a2e59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556885d2ea0 0x5556885db540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.223+0000 7f5a2f59e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556885dba80 0x5556885dde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.223+0000 7f5a2e59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556885d2ea0 0x5556885db540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:44128/0 (socket says 172.21.15.12:44128) 2024-04-22T13:25:49.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.223+0000 7f5a2e59c640 1 -- 172.21.15.12:0/2734662323 learned_addr learned my addr 172.21.15.12:0/2734662323 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:49.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.224+0000 7f5a2ed9d640 1 --2- 172.21.15.12:0/2734662323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556885d2580 0x5556885d2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.224+0000 7f5a2e59c640 1 -- 172.21.15.12:0/2734662323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556885d2580 msgr2=0x5556885d2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:49.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.224+0000 7f5a2e59c640 1 --2- 172.21.15.12:0/2734662323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556885d2580 0x5556885d2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.224+0000 7f5a2e59c640 1 -- 172.21.15.12:0/2734662323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556885dba80 msgr2=0x5556885dde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:49.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.224+0000 7f5a2e59c640 1 --2- 172.21.15.12:0/2734662323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556885dba80 0x5556885dde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.224+0000 7f5a2e59c640 1 -- 172.21.15.12:0/2734662323 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556883137d0 con 0x5556885d2ea0 2024-04-22T13:25:49.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.224+0000 7f5a2f59e640 1 --2- 172.21.15.12:0/630895781 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5a1c0a3d20 0x7f5a1c0a61e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5a28009fd0 tx=0x7f5a28009da0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:49.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.224+0000 7f5a2e59c640 1 --2- 172.21.15.12:0/2734662323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556885d2ea0 0x5556885db540 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5a18009900 tx=0x7f5a18004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2914dda78db1b9c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:49.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.224+0000 7f5a257fa640 1 -- 172.21.15.12:0/2734662323 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18004230 con 0x5556885d2ea0 2024-04-22T13:25:49.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.224+0000 7f5a257fa640 1 -- 172.21.15.12:0/2734662323 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a1802cdc0 con 0x5556885d2ea0 2024-04-22T13:25:49.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.224+0000 7f5a257fa640 1 -- 172.21.15.12:0/2734662323 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18004920 con 0x5556885d2ea0 2024-04-22T13:25:49.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.225+0000 7f5a37dd0740 1 -- 172.21.15.12:0/2734662323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556885d2ea0 msgr2=0x5556885db540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.225+0000 7f5a37dd0740 1 --2- 172.21.15.12:0/2734662323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556885d2ea0 0x5556885db540 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5a18009900 tx=0x7f5a18004040 comp rx=0 tx=0).stop 2024-04-22T13:25:49.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.225+0000 7f5a37dd0740 1 -- 172.21.15.12:0/2734662323 shutdown_connections 2024-04-22T13:25:49.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.225+0000 7f5a37dd0740 1 --2- 172.21.15.12:0/2734662323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556885dba80 0x5556885dde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.225+0000 7f5a37dd0740 1 --2- 172.21.15.12:0/2734662323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556885d2ea0 0x5556885db540 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.225+0000 7f5a37dd0740 1 --2- 172.21.15.12:0/2734662323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556885d2580 0x5556885d2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.225+0000 7f5a37dd0740 1 -- 172.21.15.12:0/2734662323 >> 172.21.15.12:0/2734662323 conn(0x5556885cccb0 msgr2=0x5556885cd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:49.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.225+0000 7f5a37dd0740 1 -- 172.21.15.12:0/2734662323 shutdown_connections 2024-04-22T13:25:49.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.225+0000 7f5a37dd0740 1 -- 172.21.15.12:0/2734662323 wait complete. 2024-04-22T13:25:49.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.226+0000 7f5a37dd0740 1 -- start start 2024-04-22T13:25:49.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.226+0000 7f5a37dd0740 1 Processor -- start 2024-04-22T13:25:49.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.226+0000 7f5a37dd0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556885d2580 0x5556885d8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.226+0000 7f5a37dd0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556885d2ea0 0x5556885d8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.226+0000 7f5a37dd0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556885dba80 0x5556885d94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.226+0000 7f5a37dd0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5556885de810 con 0x5556885d2ea0 2024-04-22T13:25:49.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.226+0000 7f5a37dd0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5556884f02f0 con 0x5556885dba80 2024-04-22T13:25:49.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.226+0000 7f5a37dd0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5556885de990 con 0x5556885d2580 2024-04-22T13:25:49.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.226+0000 7f5a37dd0740 10 client.0 _ll_register_callbacks cb 0x5556884fd580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:49.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.226+0000 7f5a2e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556885d2ea0 0x5556885d8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.226+0000 7f5a2ed9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556885d2580 0x5556885d8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.226+0000 7f5a2e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556885d2ea0 0x5556885d8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52928/0 (socket says 172.21.15.12:52928) 2024-04-22T13:25:49.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.226+0000 7f5a2e59c640 1 -- 172.21.15.12:0/185882994 learned_addr learned my addr 172.21.15.12:0/185882994 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:49.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.226+0000 7f5a2f59e640 1 --2- 172.21.15.12:0/185882994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556885dba80 0x5556885d94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.226+0000 7f5a2ed9d640 1 -- 172.21.15.12:0/185882994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556885dba80 msgr2=0x5556885d94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:49.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.226+0000 7f5a2ed9d640 1 --2- 172.21.15.12:0/185882994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556885dba80 0x5556885d94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.226+0000 7f5a2ed9d640 1 -- 172.21.15.12:0/185882994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556885d2ea0 msgr2=0x5556885d8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:49.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.226+0000 7f5a2ed9d640 1 --2- 172.21.15.12:0/185882994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556885d2ea0 0x5556885d8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.226+0000 7f5a2ed9d640 1 -- 172.21.15.12:0/185882994 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556882493f0 con 0x5556885d2580 2024-04-22T13:25:49.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.227+0000 7f5a2ed9d640 1 --2- 172.21.15.12:0/185882994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556885d2580 0x5556885d8a40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5a200cdc20 tx=0x7f5a200055f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebf8b8befc72e865 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:49.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.227+0000 7f5a167fc640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-22T13:25:49.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.227+0000 7f5a167fc640 1 -- 172.21.15.12:0/185882994 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2001d040 con 0x5556885d2580 2024-04-22T13:25:49.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.227+0000 7f5a167fc640 1 -- 172.21.15.12:0/185882994 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f5a200046c0 con 0x5556885d2580 2024-04-22T13:25:49.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.227+0000 7f5a37dd0740 1 -- 172.21.15.12:0/185882994 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a1802cdc0 con 0x5556885d2580 2024-04-22T13:25:49.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.227+0000 7f5a167fc640 1 -- 172.21.15.12:0/185882994 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5a200a2b90 con 0x5556885d2580 2024-04-22T13:25:49.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.227+0000 7f5a37dd0740 10 client.25699 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:49.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.227+0000 7f5a37dd0740 1 -- 172.21.15.12:0/185882994 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5556882090f0 con 0x5556885d2580 2024-04-22T13:25:49.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.227+0000 7f5a167fc640 1 -- 172.21.15.12:0/185882994 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a1802cdc0 con 0x5556885d2580 2024-04-22T13:25:49.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.228+0000 7f5a167fc640 1 -- 172.21.15.12:0/185882994 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5a200a2560 con 0x5556885d2580 2024-04-22T13:25:49.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.228+0000 7f5a37dd0740 10 client.25699 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:49.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.228+0000 7f5a37dd0740 10 client.25699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:25:49.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.228+0000 7f5a37dd0740 20 client.25699 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:49.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.228+0000 7f5a37dd0740 1 --2- 172.21.15.12:0/185882994 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5556885eba60 0x5556885edf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.228+0000 7f5a37dd0740 4 client.25699 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:25:49.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.228+0000 7f5a37dd0740 1 -- 172.21.15.12:0/185882994 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555688544c00 con 0x5556885eba60 2024-04-22T13:25:49.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.228+0000 7f5a2e59c640 1 --2- 172.21.15.12:0/185882994 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5556885eba60 0x5556885edf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.228+0000 7f5a167fc640 10 client.25699 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:49.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.228+0000 7f5a2e59c640 1 --2- 172.21.15.12:0/185882994 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5556885eba60 0x5556885edf20 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55f3cf761ba4b8df server_cookie=41fff2798d118deb in_seq=0 out_seq=0 2024-04-22T13:25:49.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.229+0000 7f5a167fc640 1 -- 172.21.15.12:0/185882994 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555688544c00 con 0x5556885eba60 2024-04-22T13:25:49.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.229+0000 7f5a167fc640 10 client.25699 handle_command_reply: tid=0 2024-04-22T13:25:49.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.229+0000 7f5a37dd0740 1 client.25699 shutdown 2024-04-22T13:25:49.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.229+0000 7f5a37dd0740 1 -- 172.21.15.12:0/185882994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556885d2580 msgr2=0x5556885d8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.229+0000 7f5a37dd0740 1 --2- 172.21.15.12:0/185882994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556885d2580 0x5556885d8a40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5a200cdc20 tx=0x7f5a200055f0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.229+0000 7f5a37dd0740 1 -- 172.21.15.12:0/185882994 shutdown_connections 2024-04-22T13:25:49.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.229+0000 7f5a37dd0740 1 --2- 172.21.15.12:0/185882994 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5556885eba60 0x5556885edf20 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.229+0000 7f5a37dd0740 1 --2- 172.21.15.12:0/185882994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556885dba80 0x5556885d94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.229+0000 7f5a167fc640 0 client.25699 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:49.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.229+0000 7f5a37dd0740 1 --2- 172.21.15.12:0/185882994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556885d2ea0 0x5556885d8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.229+0000 7f5a37dd0740 1 --2- 172.21.15.12:0/185882994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556885d2580 0x5556885d8a40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.229+0000 7f5a37dd0740 1 -- 172.21.15.12:0/185882994 >> 172.21.15.12:0/185882994 conn(0x5556885cccb0 msgr2=0x5556885cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:49.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.229+0000 7f5a37dd0740 1 -- 172.21.15.12:0/185882994 shutdown_connections 2024-04-22T13:25:49.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.229+0000 7f5a37dd0740 1 -- 172.21.15.12:0/185882994 wait complete. 2024-04-22T13:25:49.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.230+0000 7f5a37dd0740 20 client.25699 trim_cache size 0 max 16384 2024-04-22T13:25:49.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.248+0000 7f5a37dd0740 1 Processor -- start 2024-04-22T13:25:49.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.248+0000 7f5a37dd0740 1 -- start start 2024-04-22T13:25:49.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.249+0000 7f5a37dd0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556885d2580 0x5556885db170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.249+0000 7f5a37dd0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556885d2ea0 0x5556885d7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.249+0000 7f5a37dd0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556885dba80 0x5556885e7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.249+0000 7f5a37dd0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5a301afb10 con 0x5556885d2580 2024-04-22T13:25:49.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.249+0000 7f5a37dd0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5a301b1650 con 0x5556885dba80 2024-04-22T13:25:49.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.249+0000 7f5a37dd0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5a301b01e0 con 0x5556885d2ea0 2024-04-22T13:25:49.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.249+0000 7f5a2ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556885d2580 0x5556885db170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.249+0000 7f5a2ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556885d2580 0x5556885db170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52940/0 (socket says 172.21.15.12:52940) 2024-04-22T13:25:49.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.249+0000 7f5a2ed9d640 1 -- 172.21.15.12:0/278632469 learned_addr learned my addr 172.21.15.12:0/278632469 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:49.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.249+0000 7f5a2e59c640 1 --2- 172.21.15.12:0/278632469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556885d2ea0 0x5556885d7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.249+0000 7f5a2f59e640 1 --2- 172.21.15.12:0/278632469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556885dba80 0x5556885e7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.249+0000 7f5a2ed9d640 1 -- 172.21.15.12:0/278632469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556885d2ea0 msgr2=0x5556885d7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:49.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.249+0000 7f5a2ed9d640 1 --2- 172.21.15.12:0/278632469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556885d2ea0 0x5556885d7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.249+0000 7f5a2ed9d640 1 -- 172.21.15.12:0/278632469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556885dba80 msgr2=0x5556885e7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:49.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.249+0000 7f5a2ed9d640 1 --2- 172.21.15.12:0/278632469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556885dba80 0x5556885e7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.249+0000 7f5a2ed9d640 1 -- 172.21.15.12:0/278632469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556885d18a0 con 0x5556885d2580 2024-04-22T13:25:49.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.250+0000 7f5a2ed9d640 1 --2- 172.21.15.12:0/278632469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556885d2580 0x5556885db170 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5a20098ef0 tx=0x7f5a2009aa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ededb9e4ecdbb163 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:49.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.250+0000 7f5a24ff9640 1 -- 172.21.15.12:0/278632469 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2001d030 con 0x5556885d2580 2024-04-22T13:25:49.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.250+0000 7f5a24ff9640 1 -- 172.21.15.12:0/278632469 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a200a28d0 con 0x5556885d2580 2024-04-22T13:25:49.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.250+0000 7f5a24ff9640 1 -- 172.21.15.12:0/278632469 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a200bb1e0 con 0x5556885d2580 2024-04-22T13:25:49.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.251+0000 7f5a37dd0740 1 -- 172.21.15.12:0/278632469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556885d2580 msgr2=0x5556885db170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.251+0000 7f5a37dd0740 1 --2- 172.21.15.12:0/278632469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556885d2580 0x5556885db170 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5a20098ef0 tx=0x7f5a2009aa30 comp rx=0 tx=0).stop 2024-04-22T13:25:49.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.251+0000 7f5a37dd0740 1 -- 172.21.15.12:0/278632469 shutdown_connections 2024-04-22T13:25:49.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.251+0000 7f5a37dd0740 1 --2- 172.21.15.12:0/278632469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556885dba80 0x5556885e7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.251+0000 7f5a37dd0740 1 --2- 172.21.15.12:0/278632469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556885d2ea0 0x5556885d7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.251+0000 7f5a37dd0740 1 --2- 172.21.15.12:0/278632469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556885d2580 0x5556885db170 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.251+0000 7f5a37dd0740 1 -- 172.21.15.12:0/278632469 >> 172.21.15.12:0/278632469 conn(0x5556885cccb0 msgr2=0x5556885cbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:49.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.251+0000 7f5a37dd0740 1 -- 172.21.15.12:0/278632469 shutdown_connections 2024-04-22T13:25:49.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.251+0000 7f5a37dd0740 1 -- 172.21.15.12:0/278632469 wait complete. 2024-04-22T13:25:49.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.251+0000 7f5a37dd0740 1 -- start start 2024-04-22T13:25:49.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.251+0000 7f5a37dd0740 1 Processor -- start 2024-04-22T13:25:49.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.252+0000 7f5a37dd0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556885d2580 0x5556885df7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.252+0000 7f5a37dd0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556885d2ea0 0x5556885dfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.252+0000 7f5a37dd0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556885dba80 0x5556885e0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.252+0000 7f5a37dd0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5a301b0f80 con 0x5556885d2ea0 2024-04-22T13:25:49.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.252+0000 7f5a37dd0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5a301b08b0 con 0x5556885d2580 2024-04-22T13:25:49.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.252+0000 7f5a37dd0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5a301adaf0 con 0x5556885dba80 2024-04-22T13:25:49.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.252+0000 7f5a37dd0740 10 client.0 _ll_register_callbacks cb 0x7f5a200058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:49.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.252+0000 7f5a2f59e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556885dba80 0x5556885e0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.252+0000 7f5a2f59e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556885dba80 0x5556885e0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.12:51290/0 (socket says 172.21.15.12:51290) 2024-04-22T13:25:49.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.252+0000 7f5a2f59e640 1 -- 172.21.15.12:0/3725274250 learned_addr learned my addr 172.21.15.12:0/3725274250 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:49.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.252+0000 7f5a2ed9d640 1 --2- 172.21.15.12:0/3725274250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556885d2580 0x5556885df7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.252+0000 7f5a2e59c640 1 --2- 172.21.15.12:0/3725274250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556885d2ea0 0x5556885dfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.252+0000 7f5a2f59e640 1 -- 172.21.15.12:0/3725274250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556885d2580 msgr2=0x5556885df7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:49.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.252+0000 7f5a2f59e640 1 --2- 172.21.15.12:0/3725274250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556885d2580 0x5556885df7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.252+0000 7f5a2f59e640 1 -- 172.21.15.12:0/3725274250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556885d2ea0 msgr2=0x5556885dfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:49.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.252+0000 7f5a2f59e640 1 --2- 172.21.15.12:0/3725274250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556885d2ea0 0x5556885dfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.253+0000 7f5a2f59e640 1 -- 172.21.15.12:0/3725274250 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555688651160 con 0x5556885dba80 2024-04-22T13:25:49.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.253+0000 7f5a2ed9d640 1 --2- 172.21.15.12:0/3725274250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556885d2580 0x5556885df7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:49.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.253+0000 7f5a2e59c640 1 --2- 172.21.15.12:0/3725274250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556885d2ea0 0x5556885dfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:49.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.253+0000 7f5a2f59e640 1 --2- 172.21.15.12:0/3725274250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556885dba80 0x5556885e0230 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5a28005fd0 tx=0x7f5a28009600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=335b95d5b62689d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:49.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.253+0000 7f5a167fc640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-22T13:25:49.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.253+0000 7f5a167fc640 1 -- 172.21.15.12:0/3725274250 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a28004530 con 0x5556885dba80 2024-04-22T13:25:49.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.253+0000 7f5a167fc640 1 -- 172.21.15.12:0/3725274250 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f5a280046d0 con 0x5556885dba80 2024-04-22T13:25:49.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.253+0000 7f5a37dd0740 1 -- 172.21.15.12:0/3725274250 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a200a28d0 con 0x5556885dba80 2024-04-22T13:25:49.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.253+0000 7f5a167fc640 1 -- 172.21.15.12:0/3725274250 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5a28004b20 con 0x5556885dba80 2024-04-22T13:25:49.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.253+0000 7f5a37dd0740 10 client.25705 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:49.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.253+0000 7f5a37dd0740 1 -- 172.21.15.12:0/3725274250 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555688652470 con 0x5556885dba80 2024-04-22T13:25:49.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.254+0000 7f5a167fc640 1 -- 172.21.15.12:0/3725274250 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a200a28d0 con 0x5556885dba80 2024-04-22T13:25:49.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.254+0000 7f5a167fc640 1 -- 172.21.15.12:0/3725274250 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5a2800f040 con 0x5556885dba80 2024-04-22T13:25:49.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.254+0000 7f5a37dd0740 10 client.25705 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:49.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.254+0000 7f5a37dd0740 10 client.25705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:25:49.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.254+0000 7f5a37dd0740 20 client.25705 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:49.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.254+0000 7f5a37dd0740 1 --2- 172.21.15.12:0/3725274250 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555688652c70 0x555688655090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.254+0000 7f5a37dd0740 4 client.25705 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:25:49.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.254+0000 7f5a37dd0740 1 -- 172.21.15.12:0/3725274250 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55568853ea70 con 0x555688652c70 2024-04-22T13:25:49.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.254+0000 7f5a2ed9d640 1 --2- 172.21.15.12:0/3725274250 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555688652c70 0x555688655090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.254+0000 7f5a2ed9d640 1 --2- 172.21.15.12:0/3725274250 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555688652c70 0x555688655090 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87a2aeac746b1a09 server_cookie=4106af96b7ca0e4d in_seq=0 out_seq=0 2024-04-22T13:25:49.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.254+0000 7f5a167fc640 10 client.25705 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:49.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.255+0000 7f5a167fc640 1 -- 172.21.15.12:0/3725274250 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55568853ea70 con 0x555688652c70 2024-04-22T13:25:49.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.255+0000 7f5a167fc640 10 client.25705 handle_command_reply: tid=0 2024-04-22T13:25:49.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.255+0000 7f5a37dd0740 1 client.25705 shutdown 2024-04-22T13:25:49.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.255+0000 7f5a37dd0740 1 -- 172.21.15.12:0/3725274250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556885dba80 msgr2=0x5556885e0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.255+0000 7f5a37dd0740 1 --2- 172.21.15.12:0/3725274250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556885dba80 0x5556885e0230 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5a28005fd0 tx=0x7f5a28009600 comp rx=0 tx=0).stop 2024-04-22T13:25:49.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.255+0000 7f5a37dd0740 1 -- 172.21.15.12:0/3725274250 shutdown_connections 2024-04-22T13:25:49.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.255+0000 7f5a37dd0740 1 --2- 172.21.15.12:0/3725274250 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555688652c70 0x555688655090 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.255+0000 7f5a37dd0740 1 --2- 172.21.15.12:0/3725274250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556885dba80 0x5556885e0230 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.255+0000 7f5a37dd0740 1 --2- 172.21.15.12:0/3725274250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556885d2ea0 0x5556885dfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.255+0000 7f5a167fc640 0 client.25705 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:49.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.255+0000 7f5a37dd0740 1 --2- 172.21.15.12:0/3725274250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556885d2580 0x5556885df7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.255+0000 7f5a37dd0740 1 -- 172.21.15.12:0/3725274250 >> 172.21.15.12:0/3725274250 conn(0x5556885cccb0 msgr2=0x5556885cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:49.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.255+0000 7f5a37dd0740 1 -- 172.21.15.12:0/3725274250 shutdown_connections 2024-04-22T13:25:49.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.255+0000 7f5a37dd0740 1 -- 172.21.15.12:0/3725274250 wait complete. 2024-04-22T13:25:49.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.255+0000 7f5a37dd0740 20 client.25705 trim_cache size 0 max 16384 2024-04-22T13:25:49.256 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:25:49.256 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-04-22T13:25:49.256 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-04-22T13:25:49.256 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:25:49.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.256+0000 7f5a3510d640 1 -- 172.21.15.12:0/630895781 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5a1c0a3d20 msgr2=0x7f5a1c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.256+0000 7f5a3510d640 1 --2- 172.21.15.12:0/630895781 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5a1c0a3d20 0x7f5a1c0a61e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5a28009fd0 tx=0x7f5a28009da0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.256+0000 7f5a3510d640 1 -- 172.21.15.12:0/630895781 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a301031c0 msgr2=0x7f5a301a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.256+0000 7f5a3510d640 1 --2- 172.21.15.12:0/630895781 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a301031c0 0x7f5a301a0670 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000c970 tx=0x7f5a2000ce30 comp rx=0 tx=0).stop 2024-04-22T13:25:49.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.256+0000 7f5a3510d640 1 -- 172.21.15.12:0/630895781 shutdown_connections 2024-04-22T13:25:49.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.256+0000 7f5a3510d640 1 --2- 172.21.15.12:0/630895781 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5a1c0a3d20 0x7f5a1c0a61e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.256+0000 7f5a3510d640 1 --2- 172.21.15.12:0/630895781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a3010c500 0x7f5a3019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.256+0000 7f5a3510d640 1 --2- 172.21.15.12:0/630895781 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a30103ae0 0x7f5a301a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.256+0000 7f5a3510d640 1 --2- 172.21.15.12:0/630895781 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5a301031c0 0x7f5a301a0670 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.256+0000 7f5a3510d640 1 -- 172.21.15.12:0/630895781 >> 172.21.15.12:0/630895781 conn(0x7f5a30078000 msgr2=0x7f5a30108410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:49.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.256+0000 7f5a3510d640 1 -- 172.21.15.12:0/630895781 shutdown_connections 2024-04-22T13:25:49.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.256+0000 7f5a3510d640 1 -- 172.21.15.12:0/630895781 wait complete. 2024-04-22T13:25:49.264 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:25:49.264 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:25:49.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.376+0000 7f6d647c7640 1 -- 172.21.15.12:0/2115640262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d5c107260 msgr2=0x7f6d5c10ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.376+0000 7f6d647c7640 1 --2- 172.21.15.12:0/2115640262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d5c107260 0x7f6d5c10ee30 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6d54009ca0 tx=0x7f6d5402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.376+0000 7f6d647c7640 1 -- 172.21.15.12:0/2115640262 shutdown_connections 2024-04-22T13:25:49.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.376+0000 7f6d647c7640 1 --2- 172.21.15.12:0/2115640262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d5c107260 0x7f6d5c10ee30 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.376+0000 7f6d647c7640 1 --2- 172.21.15.12:0/2115640262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d5c06b200 0x7f6d5c106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.376+0000 7f6d647c7640 1 --2- 172.21.15.12:0/2115640262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d5c06a850 0x7f6d5c06ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.376+0000 7f6d647c7640 1 -- 172.21.15.12:0/2115640262 >> 172.21.15.12:0/2115640262 conn(0x7f6d5c077c20 msgr2=0x7f6d5c078030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:49.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.376+0000 7f6d647c7640 1 -- 172.21.15.12:0/2115640262 shutdown_connections 2024-04-22T13:25:49.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.376+0000 7f6d647c7640 1 -- 172.21.15.12:0/2115640262 wait complete. 2024-04-22T13:25:49.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.376+0000 7f6d647c7640 1 Processor -- start 2024-04-22T13:25:49.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.376+0000 7f6d647c7640 1 -- start start 2024-04-22T13:25:49.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.377+0000 7f6d647c7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d5c06a850 0x7f6d5c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.377+0000 7f6d647c7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d5c107260 0x7f6d5c198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.377+0000 7f6d647c7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d5c19cfc0 0x7f6d5c19d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.377+0000 7f6d647c7640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c1121d0 con 0x7f6d5c107260 2024-04-22T13:25:49.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.377+0000 7f6d647c7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c112050 con 0x7f6d5c06a850 2024-04-22T13:25:49.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.377+0000 7f6d647c7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c112350 con 0x7f6d5c19cfc0 2024-04-22T13:25:49.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.377+0000 7f6d61d62640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d5c107260 0x7f6d5c198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.377+0000 7f6d61d62640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d5c107260 0x7f6d5c198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52972/0 (socket says 172.21.15.12:52972) 2024-04-22T13:25:49.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.377+0000 7f6d61d62640 1 -- 172.21.15.12:0/3603828510 learned_addr learned my addr 172.21.15.12:0/3603828510 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:49.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.377+0000 7f6d62563640 1 --2- 172.21.15.12:0/3603828510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d5c06a850 0x7f6d5c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.377+0000 7f6d62d64640 1 --2- 172.21.15.12:0/3603828510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d5c19cfc0 0x7f6d5c19d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.377+0000 7f6d61d62640 1 -- 172.21.15.12:0/3603828510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d5c19cfc0 msgr2=0x7f6d5c19d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.377+0000 7f6d61d62640 1 --2- 172.21.15.12:0/3603828510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d5c19cfc0 0x7f6d5c19d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.377+0000 7f6d61d62640 1 -- 172.21.15.12:0/3603828510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d5c06a850 msgr2=0x7f6d5c198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.377+0000 7f6d61d62640 1 --2- 172.21.15.12:0/3603828510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d5c06a850 0x7f6d5c198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.377+0000 7f6d61d62640 1 -- 172.21.15.12:0/3603828510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5c19dab0 con 0x7f6d5c107260 2024-04-22T13:25:49.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.377+0000 7f6d61d62640 1 --2- 172.21.15.12:0/3603828510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d5c107260 0x7f6d5c198c90 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6d5800d770 tx=0x7f6d5800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:49.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.378+0000 7f6d537fe640 1 -- 172.21.15.12:0/3603828510 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d58017070 con 0x7f6d5c107260 2024-04-22T13:25:49.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.378+0000 7f6d647c7640 1 -- 172.21.15.12:0/3603828510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d5c19dda0 con 0x7f6d5c107260 2024-04-22T13:25:49.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.378+0000 7f6d537fe640 1 -- 172.21.15.12:0/3603828510 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d58009c30 con 0x7f6d5c107260 2024-04-22T13:25:49.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.378+0000 7f6d537fe640 1 -- 172.21.15.12:0/3603828510 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5800f620 con 0x7f6d5c107260 2024-04-22T13:25:49.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.378+0000 7f6d647c7640 1 -- 172.21.15.12:0/3603828510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d5c1a55e0 con 0x7f6d5c107260 2024-04-22T13:25:49.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.379+0000 7f6d647c7640 1 -- 172.21.15.12:0/3603828510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d5c103660 con 0x7f6d5c107260 2024-04-22T13:25:49.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.380+0000 7f6d537fe640 1 -- 172.21.15.12:0/3603828510 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6d5801a070 con 0x7f6d5c107260 2024-04-22T13:25:49.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.381+0000 7f6d537fe640 1 --2- 172.21.15.12:0/3603828510 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6d480a3d20 0x7f6d480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.381+0000 7f6d537fe640 1 -- 172.21.15.12:0/3603828510 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f6d580d0220 con 0x7f6d5c107260 2024-04-22T13:25:49.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.381+0000 7f6d62563640 1 --2- 172.21.15.12:0/3603828510 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6d480a3d20 0x7f6d480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.381+0000 7f6d62563640 1 --2- 172.21.15.12:0/3603828510 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6d480a3d20 0x7f6d480a61e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c009bd0 tx=0x7f6d4c005f90 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:49.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.390+0000 7f6d537fe640 1 -- 172.21.15.12:0/3603828510 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6d58009de0 con 0x7f6d5c107260 2024-04-22T13:25:49.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.617+0000 7f6d647c7640 1 -- 172.21.15.12:0/3603828510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d5c199a50 con 0x7f6d5c107260 2024-04-22T13:25:49.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.618+0000 7f6d537fe640 1 -- 172.21.15.12:0/3603828510 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7f6d5809c260 con 0x7f6d5c107260 2024-04-22T13:25:49.615 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:25:49.615 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:25:49.615 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:25:49.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.620+0000 7f6d647c7640 1 -- 172.21.15.12:0/3603828510 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6d480a3d20 msgr2=0x7f6d480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.620+0000 7f6d647c7640 1 --2- 172.21.15.12:0/3603828510 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6d480a3d20 0x7f6d480a61e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c009bd0 tx=0x7f6d4c005f90 comp rx=0 tx=0).stop 2024-04-22T13:25:49.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.620+0000 7f6d647c7640 1 -- 172.21.15.12:0/3603828510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d5c107260 msgr2=0x7f6d5c198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.620+0000 7f6d647c7640 1 --2- 172.21.15.12:0/3603828510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d5c107260 0x7f6d5c198c90 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6d5800d770 tx=0x7f6d5800dc30 comp rx=0 tx=0).stop 2024-04-22T13:25:49.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.621+0000 7f6d647c7640 1 -- 172.21.15.12:0/3603828510 shutdown_connections 2024-04-22T13:25:49.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.621+0000 7f6d647c7640 1 --2- 172.21.15.12:0/3603828510 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6d480a3d20 0x7f6d480a61e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.621+0000 7f6d647c7640 1 --2- 172.21.15.12:0/3603828510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d5c19cfc0 0x7f6d5c19d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.621+0000 7f6d647c7640 1 --2- 172.21.15.12:0/3603828510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d5c107260 0x7f6d5c198c90 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.621+0000 7f6d647c7640 1 --2- 172.21.15.12:0/3603828510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d5c06a850 0x7f6d5c198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.621+0000 7f6d647c7640 1 -- 172.21.15.12:0/3603828510 >> 172.21.15.12:0/3603828510 conn(0x7f6d5c077c20 msgr2=0x7f6d5c0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:49.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.621+0000 7f6d647c7640 1 -- 172.21.15.12:0/3603828510 shutdown_connections 2024-04-22T13:25:49.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.621+0000 7f6d647c7640 1 -- 172.21.15.12:0/3603828510 wait complete. 2024-04-22T13:25:49.629 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:25:49.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.744+0000 7ff8baaf2640 1 -- 172.21.15.12:0/206026366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8b406a360 msgr2=0x7ff8b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.744+0000 7ff8baaf2640 1 --2- 172.21.15.12:0/206026366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8b406a360 0x7ff8b406a7a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0009930 tx=0x7ff8a002bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:49.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.744+0000 7ff8baaf2640 1 -- 172.21.15.12:0/206026366 shutdown_connections 2024-04-22T13:25:49.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.744+0000 7ff8baaf2640 1 --2- 172.21.15.12:0/206026366 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8b406ace0 0x7ff8b410ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.744+0000 7ff8baaf2640 1 --2- 172.21.15.12:0/206026366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8b406a360 0x7ff8b406a7a0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.744+0000 7ff8baaf2640 1 --2- 172.21.15.12:0/206026366 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8b4107fd0 0x7ff8b41083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.744+0000 7ff8baaf2640 1 -- 172.21.15.12:0/206026366 >> 172.21.15.12:0/206026366 conn(0x7ff8b4077ef0 msgr2=0x7ff8b4078300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:49.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.744+0000 7ff8baaf2640 1 -- 172.21.15.12:0/206026366 shutdown_connections 2024-04-22T13:25:49.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.744+0000 7ff8baaf2640 1 -- 172.21.15.12:0/206026366 wait complete. 2024-04-22T13:25:49.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.745+0000 7ff8baaf2640 1 Processor -- start 2024-04-22T13:25:49.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.745+0000 7ff8baaf2640 1 -- start start 2024-04-22T13:25:49.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.745+0000 7ff8baaf2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8b406a360 0x7ff8b419e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.745+0000 7ff8baaf2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8b406ace0 0x7ff8b419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.745+0000 7ff8baaf2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8b4107fd0 0x7ff8b41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.745+0000 7ff8baaf2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b41121d0 con 0x7ff8b406a360 2024-04-22T13:25:49.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.745+0000 7ff8baaf2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b4112050 con 0x7ff8b4107fd0 2024-04-22T13:25:49.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.745+0000 7ff8baaf2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b4112350 con 0x7ff8b406ace0 2024-04-22T13:25:49.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.745+0000 7ff8b888e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8b406a360 0x7ff8b419e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.746+0000 7ff8b888e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8b406a360 0x7ff8b419e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52994/0 (socket says 172.21.15.12:52994) 2024-04-22T13:25:49.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.746+0000 7ff8b888e640 1 -- 172.21.15.12:0/227737685 learned_addr learned my addr 172.21.15.12:0/227737685 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:49.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.746+0000 7ff8b908f640 1 --2- 172.21.15.12:0/227737685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8b4107fd0 0x7ff8b41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.746+0000 7ff8b3fff640 1 --2- 172.21.15.12:0/227737685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8b406ace0 0x7ff8b419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.746+0000 7ff8b888e640 1 -- 172.21.15.12:0/227737685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8b406ace0 msgr2=0x7ff8b419e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.746+0000 7ff8b888e640 1 --2- 172.21.15.12:0/227737685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8b406ace0 0x7ff8b419e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.746+0000 7ff8b888e640 1 -- 172.21.15.12:0/227737685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8b4107fd0 msgr2=0x7ff8b41985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.746+0000 7ff8b888e640 1 --2- 172.21.15.12:0/227737685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8b4107fd0 0x7ff8b41985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.746+0000 7ff8b888e640 1 -- 172.21.15.12:0/227737685 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b4198d30 con 0x7ff8b406a360 2024-04-22T13:25:49.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.746+0000 7ff8b908f640 1 --2- 172.21.15.12:0/227737685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8b4107fd0 0x7ff8b41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:49.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.746+0000 7ff8b888e640 1 --2- 172.21.15.12:0/227737685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8b406a360 0x7ff8b419e400 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ff8a800d950 tx=0x7ff8a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:49.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.746+0000 7ff8b1ffb640 1 -- 172.21.15.12:0/227737685 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8017030 con 0x7ff8b406a360 2024-04-22T13:25:49.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.746+0000 7ff8baaf2640 1 -- 172.21.15.12:0/227737685 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b4199020 con 0x7ff8b406a360 2024-04-22T13:25:49.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.746+0000 7ff8b1ffb640 1 -- 172.21.15.12:0/227737685 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8a8004020 con 0x7ff8b406a360 2024-04-22T13:25:49.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.746+0000 7ff8b1ffb640 1 -- 172.21.15.12:0/227737685 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a80047a0 con 0x7ff8b406a360 2024-04-22T13:25:49.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.746+0000 7ff8baaf2640 1 -- 172.21.15.12:0/227737685 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b41a52c0 con 0x7ff8b406a360 2024-04-22T13:25:49.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.749+0000 7ff8b1ffb640 1 -- 172.21.15.12:0/227737685 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff8a8009a30 con 0x7ff8b406a360 2024-04-22T13:25:49.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.749+0000 7ff8b1ffb640 1 --2- 172.21.15.12:0/227737685 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff8a40a3d20 0x7ff8a40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.749+0000 7ff8b1ffb640 1 -- 172.21.15.12:0/227737685 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7ff8a80d0160 con 0x7ff8b406a360 2024-04-22T13:25:49.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.749+0000 7ff8b3fff640 1 --2- 172.21.15.12:0/227737685 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff8a40a3d20 0x7ff8a40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.750+0000 7ff8b3fff640 1 --2- 172.21.15.12:0/227737685 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff8a40a3d20 0x7ff8a40a61e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0009900 tx=0x7ff8a0005fd0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:49.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.750+0000 7ff8bd7c1740 1 Processor -- start 2024-04-22T13:25:49.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.750+0000 7ff8bd7c1740 1 -- start start 2024-04-22T13:25:49.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.751+0000 7ff8bd7c1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5ff4244d0 0x55d5ff4248b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.751+0000 7ff8bd7c1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5ff424e80 0x55d5ff42d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.751+0000 7ff8bd7c1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d5ff42da00 0x55d5ff42fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.751+0000 7ff8bd7c1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d5ff3891f0 con 0x55d5ff424e80 2024-04-22T13:25:49.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.751+0000 7ff8bd7c1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d5ff3264b0 con 0x55d5ff42da00 2024-04-22T13:25:49.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.751+0000 7ff8bd7c1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d5ff3851b0 con 0x55d5ff4244d0 2024-04-22T13:25:49.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.751+0000 7ff8b3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5ff424e80 0x55d5ff42d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.751+0000 7ff8b3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5ff424e80 0x55d5ff42d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53006/0 (socket says 172.21.15.12:53006) 2024-04-22T13:25:49.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.751+0000 7ff8b3fff640 1 -- 172.21.15.12:0/1351027139 learned_addr learned my addr 172.21.15.12:0/1351027139 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:49.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.751+0000 7ff8b888e640 1 --2- 172.21.15.12:0/1351027139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5ff4244d0 0x55d5ff4248b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.751+0000 7ff8b908f640 1 --2- 172.21.15.12:0/1351027139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d5ff42da00 0x55d5ff42fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.751+0000 7ff8b3fff640 1 -- 172.21.15.12:0/1351027139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5ff4244d0 msgr2=0x55d5ff4248b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:49.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.751+0000 7ff8b3fff640 1 --2- 172.21.15.12:0/1351027139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5ff4244d0 0x55d5ff4248b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.751+0000 7ff8b3fff640 1 -- 172.21.15.12:0/1351027139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d5ff42da00 msgr2=0x55d5ff42fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:49.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.751+0000 7ff8b3fff640 1 --2- 172.21.15.12:0/1351027139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d5ff42da00 0x55d5ff42fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.751+0000 7ff8b3fff640 1 -- 172.21.15.12:0/1351027139 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5ff3181b0 con 0x55d5ff424e80 2024-04-22T13:25:49.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.751+0000 7ff8b3fff640 1 --2- 172.21.15.12:0/1351027139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5ff424e80 0x55d5ff42d430 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0000c00 tx=0x7ff8a003d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b41376cc069f09 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:49.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.752+0000 7ff89effd640 1 -- 172.21.15.12:0/1351027139 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a00044e0 con 0x55d5ff424e80 2024-04-22T13:25:49.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.752+0000 7ff89effd640 1 -- 172.21.15.12:0/1351027139 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8a004f070 con 0x55d5ff424e80 2024-04-22T13:25:49.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.752+0000 7ff89effd640 1 -- 172.21.15.12:0/1351027139 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a00035f0 con 0x55d5ff424e80 2024-04-22T13:25:49.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.752+0000 7ff8bd7c1740 1 -- 172.21.15.12:0/1351027139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5ff424e80 msgr2=0x55d5ff42d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.752+0000 7ff8bd7c1740 1 --2- 172.21.15.12:0/1351027139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5ff424e80 0x55d5ff42d430 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0000c00 tx=0x7ff8a003d000 comp rx=0 tx=0).stop 2024-04-22T13:25:49.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.752+0000 7ff8bd7c1740 1 -- 172.21.15.12:0/1351027139 shutdown_connections 2024-04-22T13:25:49.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.752+0000 7ff8bd7c1740 1 --2- 172.21.15.12:0/1351027139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d5ff42da00 0x55d5ff42fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.752+0000 7ff8bd7c1740 1 --2- 172.21.15.12:0/1351027139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5ff424e80 0x55d5ff42d430 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.752+0000 7ff8bd7c1740 1 --2- 172.21.15.12:0/1351027139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5ff4244d0 0x55d5ff4248b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.752+0000 7ff8bd7c1740 1 -- 172.21.15.12:0/1351027139 >> 172.21.15.12:0/1351027139 conn(0x55d5ff41eba0 msgr2=0x55d5ff41ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:49.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.753+0000 7ff8bd7c1740 1 -- 172.21.15.12:0/1351027139 shutdown_connections 2024-04-22T13:25:49.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.753+0000 7ff8bd7c1740 1 -- 172.21.15.12:0/1351027139 wait complete. 2024-04-22T13:25:49.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.753+0000 7ff8bd7c1740 1 -- start start 2024-04-22T13:25:49.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.753+0000 7ff8bd7c1740 1 Processor -- start 2024-04-22T13:25:49.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.753+0000 7ff8bd7c1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5ff4244d0 0x55d5ff42d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.753+0000 7ff8bd7c1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d5ff424e80 0x55d5ff42a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.753+0000 7ff8bd7c1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5ff42da00 0x55d5ff42abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.753+0000 7ff8bd7c1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d5ff430910 con 0x55d5ff4244d0 2024-04-22T13:25:49.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.753+0000 7ff8bd7c1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d5ff342500 con 0x55d5ff424e80 2024-04-22T13:25:49.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.753+0000 7ff8bd7c1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d5ff430a90 con 0x55d5ff42da00 2024-04-22T13:25:49.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.753+0000 7ff8bd7c1740 10 client.0 _ll_register_callbacks cb 0x55d5ff310b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:49.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.754+0000 7ff8b888e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5ff4244d0 0x55d5ff42d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.754+0000 7ff8b888e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5ff4244d0 0x55d5ff42d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53022/0 (socket says 172.21.15.12:53022) 2024-04-22T13:25:49.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.754+0000 7ff8b888e640 1 -- 172.21.15.12:0/1794136162 learned_addr learned my addr 172.21.15.12:0/1794136162 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:49.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.754+0000 7ff8b3fff640 1 --2- 172.21.15.12:0/1794136162 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d5ff424e80 0x55d5ff42a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.754+0000 7ff8b908f640 1 --2- 172.21.15.12:0/1794136162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5ff42da00 0x55d5ff42abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.754+0000 7ff8b888e640 1 -- 172.21.15.12:0/1794136162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5ff42da00 msgr2=0x55d5ff42abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:49.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.754+0000 7ff8b888e640 1 --2- 172.21.15.12:0/1794136162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5ff42da00 0x55d5ff42abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.754+0000 7ff8b888e640 1 -- 172.21.15.12:0/1794136162 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d5ff424e80 msgr2=0x55d5ff42a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:49.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.754+0000 7ff8b888e640 1 --2- 172.21.15.12:0/1794136162 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d5ff424e80 0x55d5ff42a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.754+0000 7ff8b888e640 1 -- 172.21.15.12:0/1794136162 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5ff09b3f0 con 0x55d5ff4244d0 2024-04-22T13:25:49.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.754+0000 7ff8b888e640 1 --2- 172.21.15.12:0/1794136162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5ff4244d0 0x55d5ff42d2d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff8a809c0e0 tx=0x7ff8a80057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79035ea034674045 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:49.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.754+0000 7ff893fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:49.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.754+0000 7ff893fff640 1 -- 172.21.15.12:0/1794136162 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a80a4dc0 con 0x55d5ff4244d0 2024-04-22T13:25:49.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.754+0000 7ff893fff640 1 -- 172.21.15.12:0/1794136162 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7ff8a80041d0 con 0x55d5ff4244d0 2024-04-22T13:25:49.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.754+0000 7ff8bd7c1740 1 -- 172.21.15.12:0/1794136162 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8a004f070 con 0x55d5ff4244d0 2024-04-22T13:25:49.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.754+0000 7ff893fff640 1 -- 172.21.15.12:0/1794136162 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff8a80a01b0 con 0x55d5ff4244d0 2024-04-22T13:25:49.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.754+0000 7ff8bd7c1740 10 client.16368 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:49.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.754+0000 7ff8bd7c1740 1 -- 172.21.15.12:0/1794136162 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d5ff1657d0 con 0x55d5ff4244d0 2024-04-22T13:25:49.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.755+0000 7ff893fff640 1 -- 172.21.15.12:0/1794136162 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8a004f070 con 0x55d5ff4244d0 2024-04-22T13:25:49.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.755+0000 7ff893fff640 1 -- 172.21.15.12:0/1794136162 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff8a80987c0 con 0x55d5ff4244d0 2024-04-22T13:25:49.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.755+0000 7ff8bd7c1740 10 client.16368 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:49.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.755+0000 7ff8bd7c1740 10 client.16368 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:25:49.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.755+0000 7ff8bd7c1740 20 client.16368 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:49.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.755+0000 7ff8bd7c1740 1 --2- 172.21.15.12:0/1794136162 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55d5ff43dbd0 0x55d5ff440000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.755+0000 7ff8bd7c1740 4 client.16368 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:25:49.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.755+0000 7ff8bd7c1740 1 -- 172.21.15.12:0/1794136162 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5ff328930 con 0x55d5ff43dbd0 2024-04-22T13:25:49.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.755+0000 7ff8b3fff640 1 --2- 172.21.15.12:0/1794136162 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55d5ff43dbd0 0x55d5ff440000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.755+0000 7ff8b3fff640 1 --2- 172.21.15.12:0/1794136162 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55d5ff43dbd0 0x55d5ff440000 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6f755e1847157728 server_cookie=d5981ce494c4b322 in_seq=0 out_seq=0 2024-04-22T13:25:49.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.755+0000 7ff893fff640 10 client.16368 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:25:49.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.756+0000 7ff893fff640 1 -- 172.21.15.12:0/1794136162 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d5ff328930 con 0x55d5ff43dbd0 2024-04-22T13:25:49.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.756+0000 7ff893fff640 10 client.16368 handle_command_reply: tid=0 2024-04-22T13:25:49.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.756+0000 7ff8bd7c1740 1 client.16368 shutdown 2024-04-22T13:25:49.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.756+0000 7ff8bd7c1740 1 -- 172.21.15.12:0/1794136162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5ff4244d0 msgr2=0x55d5ff42d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.756+0000 7ff8bd7c1740 1 --2- 172.21.15.12:0/1794136162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5ff4244d0 0x55d5ff42d2d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff8a809c0e0 tx=0x7ff8a80057e0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.756+0000 7ff8bd7c1740 1 -- 172.21.15.12:0/1794136162 shutdown_connections 2024-04-22T13:25:49.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.756+0000 7ff8bd7c1740 1 --2- 172.21.15.12:0/1794136162 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55d5ff43dbd0 0x55d5ff440000 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.756+0000 7ff8bd7c1740 1 --2- 172.21.15.12:0/1794136162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5ff42da00 0x55d5ff42abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.756+0000 7ff8bd7c1740 1 --2- 172.21.15.12:0/1794136162 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d5ff424e80 0x55d5ff42a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.756+0000 7ff893fff640 0 client.16368 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:25:49.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.756+0000 7ff8bd7c1740 1 --2- 172.21.15.12:0/1794136162 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5ff4244d0 0x55d5ff42d2d0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.756+0000 7ff8bd7c1740 1 -- 172.21.15.12:0/1794136162 >> 172.21.15.12:0/1794136162 conn(0x55d5ff41eba0 msgr2=0x55d5ff4208f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:49.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.757+0000 7ff8bd7c1740 1 -- 172.21.15.12:0/1794136162 shutdown_connections 2024-04-22T13:25:49.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.757+0000 7ff8bd7c1740 1 -- 172.21.15.12:0/1794136162 wait complete. 2024-04-22T13:25:49.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.757+0000 7ff8bd7c1740 20 client.16368 trim_cache size 0 max 16384 2024-04-22T13:25:49.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.785+0000 7ff8bd7c1740 1 Processor -- start 2024-04-22T13:25:49.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.785+0000 7ff8bd7c1740 1 -- start start 2024-04-22T13:25:49.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.786+0000 7ff8bd7c1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d5ff4244d0 0x55d5ff4397d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.786+0000 7ff8bd7c1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5ff424e80 0x55d5ff41da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.786+0000 7ff8bd7c1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5ff42da00 0x55d5ff42cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.786+0000 7ff8bd7c1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b41ada50 con 0x55d5ff424e80 2024-04-22T13:25:49.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.786+0000 7ff8bd7c1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b41af590 con 0x55d5ff4244d0 2024-04-22T13:25:49.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.786+0000 7ff8bd7c1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b41ae120 con 0x55d5ff42da00 2024-04-22T13:25:49.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.786+0000 7ff8b3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5ff424e80 0x55d5ff41da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.786+0000 7ff8b3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5ff424e80 0x55d5ff41da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53036/0 (socket says 172.21.15.12:53036) 2024-04-22T13:25:49.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.786+0000 7ff8b3fff640 1 -- 172.21.15.12:0/464153675 learned_addr learned my addr 172.21.15.12:0/464153675 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:49.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.786+0000 7ff8b888e640 1 --2- 172.21.15.12:0/464153675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d5ff4244d0 0x55d5ff4397d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.786+0000 7ff8b908f640 1 --2- 172.21.15.12:0/464153675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5ff42da00 0x55d5ff42cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.786+0000 7ff8b3fff640 1 -- 172.21.15.12:0/464153675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5ff42da00 msgr2=0x55d5ff42cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:49.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.786+0000 7ff8b3fff640 1 --2- 172.21.15.12:0/464153675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5ff42da00 0x55d5ff42cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.786+0000 7ff8b3fff640 1 -- 172.21.15.12:0/464153675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d5ff4244d0 msgr2=0x55d5ff4397d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:49.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.786+0000 7ff8b3fff640 1 --2- 172.21.15.12:0/464153675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d5ff4244d0 0x55d5ff4397d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.786+0000 7ff8b3fff640 1 -- 172.21.15.12:0/464153675 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5ff42af30 con 0x55d5ff424e80 2024-04-22T13:25:49.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.786+0000 7ff8b3fff640 1 --2- 172.21.15.12:0/464153675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5ff424e80 0x55d5ff41da80 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0000c00 tx=0x7ff8a0005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6de27aa2a86d19e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:49.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.787+0000 7ff89e7fc640 1 -- 172.21.15.12:0/464153675 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a005dbc0 con 0x55d5ff424e80 2024-04-22T13:25:49.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.787+0000 7ff89e7fc640 1 -- 172.21.15.12:0/464153675 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8a004f070 con 0x55d5ff424e80 2024-04-22T13:25:49.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.787+0000 7ff89e7fc640 1 -- 172.21.15.12:0/464153675 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0003de0 con 0x55d5ff424e80 2024-04-22T13:25:49.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.787+0000 7ff8bd7c1740 1 -- 172.21.15.12:0/464153675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5ff424e80 msgr2=0x55d5ff41da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.787+0000 7ff8bd7c1740 1 --2- 172.21.15.12:0/464153675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5ff424e80 0x55d5ff41da80 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0000c00 tx=0x7ff8a0005b50 comp rx=0 tx=0).stop 2024-04-22T13:25:49.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.787+0000 7ff8bd7c1740 1 -- 172.21.15.12:0/464153675 shutdown_connections 2024-04-22T13:25:49.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.787+0000 7ff8bd7c1740 1 --2- 172.21.15.12:0/464153675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5ff42da00 0x55d5ff42cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.787+0000 7ff8bd7c1740 1 --2- 172.21.15.12:0/464153675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5ff424e80 0x55d5ff41da80 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.787+0000 7ff8bd7c1740 1 --2- 172.21.15.12:0/464153675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d5ff4244d0 0x55d5ff4397d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.787+0000 7ff8bd7c1740 1 -- 172.21.15.12:0/464153675 >> 172.21.15.12:0/464153675 conn(0x55d5ff41eba0 msgr2=0x55d5ff41d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:49.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.787+0000 7ff8bd7c1740 1 -- 172.21.15.12:0/464153675 shutdown_connections 2024-04-22T13:25:49.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.787+0000 7ff8bd7c1740 1 -- 172.21.15.12:0/464153675 wait complete. 2024-04-22T13:25:49.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.788+0000 7ff8bd7c1740 1 -- start start 2024-04-22T13:25:49.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.788+0000 7ff8bd7c1740 1 Processor -- start 2024-04-22T13:25:49.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.788+0000 7ff8bd7c1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5ff4244d0 0x55d5ff436070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.788+0000 7ff8bd7c1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5ff424e80 0x55d5ff4313e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.788+0000 7ff8bd7c1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d5ff42da00 0x55d5ff431920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.788+0000 7ff8bd7c1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b41aeec0 con 0x55d5ff4244d0 2024-04-22T13:25:49.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.788+0000 7ff8bd7c1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b41ae7f0 con 0x55d5ff42da00 2024-04-22T13:25:49.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.788+0000 7ff8bd7c1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b41aba30 con 0x55d5ff424e80 2024-04-22T13:25:49.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.788+0000 7ff8bd7c1740 10 client.0 _ll_register_callbacks cb 0x7ff8a809bab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:49.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.788+0000 7ff8b888e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5ff4244d0 0x55d5ff436070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.788+0000 7ff8b888e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5ff4244d0 0x55d5ff436070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53044/0 (socket says 172.21.15.12:53044) 2024-04-22T13:25:49.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.788+0000 7ff8b888e640 1 -- 172.21.15.12:0/4036341071 learned_addr learned my addr 172.21.15.12:0/4036341071 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:49.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.788+0000 7ff8b3fff640 1 --2- 172.21.15.12:0/4036341071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5ff424e80 0x55d5ff4313e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.788+0000 7ff8b908f640 1 --2- 172.21.15.12:0/4036341071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d5ff42da00 0x55d5ff431920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.789+0000 7ff8b888e640 1 -- 172.21.15.12:0/4036341071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5ff424e80 msgr2=0x55d5ff4313e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:49.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.789+0000 7ff8b888e640 1 --2- 172.21.15.12:0/4036341071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5ff424e80 0x55d5ff4313e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.789+0000 7ff8b888e640 1 -- 172.21.15.12:0/4036341071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d5ff42da00 msgr2=0x55d5ff431920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:49.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.789+0000 7ff8b888e640 1 --2- 172.21.15.12:0/4036341071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d5ff42da00 0x55d5ff431920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.789+0000 7ff8b888e640 1 -- 172.21.15.12:0/4036341071 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5ff432130 con 0x55d5ff4244d0 2024-04-22T13:25:49.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.789+0000 7ff8b888e640 1 --2- 172.21.15.12:0/4036341071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5ff4244d0 0x55d5ff436070 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8097fa0 tx=0x7ff8a80043e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f32ede4bc750f18d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:49.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.789+0000 7ff893fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:49.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.789+0000 7ff893fff640 1 -- 172.21.15.12:0/4036341071 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a809b7a0 con 0x55d5ff4244d0 2024-04-22T13:25:49.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.789+0000 7ff893fff640 1 -- 172.21.15.12:0/4036341071 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7ff8a809b010 con 0x55d5ff4244d0 2024-04-22T13:25:49.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.789+0000 7ff8bd7c1740 1 -- 172.21.15.12:0/4036341071 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8a004f070 con 0x55d5ff4244d0 2024-04-22T13:25:49.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.789+0000 7ff893fff640 1 -- 172.21.15.12:0/4036341071 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff8a80a4dc0 con 0x55d5ff4244d0 2024-04-22T13:25:49.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.789+0000 7ff8bd7c1740 10 client.16374 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.789+0000 7ff8bd7c1740 1 -- 172.21.15.12:0/4036341071 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d5ff4a4050 con 0x55d5ff4244d0 2024-04-22T13:25:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.789+0000 7ff893fff640 1 -- 172.21.15.12:0/4036341071 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8a004f070 con 0x55d5ff4244d0 2024-04-22T13:25:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.790+0000 7ff893fff640 1 -- 172.21.15.12:0/4036341071 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff8a8066040 con 0x55d5ff4244d0 2024-04-22T13:25:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.790+0000 7ff8bd7c1740 10 client.16374 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.790+0000 7ff8bd7c1740 10 client.16374 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:25:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.790+0000 7ff8bd7c1740 20 client.16374 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.790+0000 7ff8bd7c1740 1 --2- 172.21.15.12:0/4036341071 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55d5ff43dbd0 0x55d5ff4a6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.790+0000 7ff8bd7c1740 4 client.16374 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:25:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.790+0000 7ff8bd7c1740 1 -- 172.21.15.12:0/4036341071 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d5ff396c10 con 0x55d5ff43dbd0 2024-04-22T13:25:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.790+0000 7ff8b3fff640 1 --2- 172.21.15.12:0/4036341071 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55d5ff43dbd0 0x55d5ff4a6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.791+0000 7ff8b3fff640 1 --2- 172.21.15.12:0/4036341071 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55d5ff43dbd0 0x55d5ff4a6780 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8ff45b93a781d5b server_cookie=ddac797ed60a8f95 in_seq=0 out_seq=0 2024-04-22T13:25:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.791+0000 7ff893fff640 10 client.16374 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:25:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.791+0000 7ff893fff640 1 -- 172.21.15.12:0/4036341071 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d5ff396c10 con 0x55d5ff43dbd0 2024-04-22T13:25:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.791+0000 7ff893fff640 10 client.16374 handle_command_reply: tid=0 2024-04-22T13:25:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.791+0000 7ff8bd7c1740 1 client.16374 shutdown 2024-04-22T13:25:49.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.791+0000 7ff8bd7c1740 1 -- 172.21.15.12:0/4036341071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5ff4244d0 msgr2=0x55d5ff436070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.791+0000 7ff8bd7c1740 1 --2- 172.21.15.12:0/4036341071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5ff4244d0 0x55d5ff436070 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8097fa0 tx=0x7ff8a80043e0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.791+0000 7ff8bd7c1740 1 -- 172.21.15.12:0/4036341071 shutdown_connections 2024-04-22T13:25:49.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.791+0000 7ff8bd7c1740 1 --2- 172.21.15.12:0/4036341071 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55d5ff43dbd0 0x55d5ff4a6780 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.791+0000 7ff8bd7c1740 1 --2- 172.21.15.12:0/4036341071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d5ff42da00 0x55d5ff431920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.791+0000 7ff893fff640 0 client.16374 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:25:49.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.791+0000 7ff8bd7c1740 1 --2- 172.21.15.12:0/4036341071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5ff424e80 0x55d5ff4313e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.791+0000 7ff8bd7c1740 1 --2- 172.21.15.12:0/4036341071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5ff4244d0 0x55d5ff436070 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.791+0000 7ff8bd7c1740 1 -- 172.21.15.12:0/4036341071 >> 172.21.15.12:0/4036341071 conn(0x55d5ff41eba0 msgr2=0x55d5ff41dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:49.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.791+0000 7ff8bd7c1740 1 -- 172.21.15.12:0/4036341071 shutdown_connections 2024-04-22T13:25:49.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.791+0000 7ff8bd7c1740 1 -- 172.21.15.12:0/4036341071 wait complete. 2024-04-22T13:25:49.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.791+0000 7ff8bd7c1740 20 client.16374 trim_cache size 0 max 16384 2024-04-22T13:25:49.794 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:25:49.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.792+0000 7ff8baaf2640 1 -- 172.21.15.12:0/227737685 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff8a40a3d20 msgr2=0x7ff8a40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.792+0000 7ff8baaf2640 1 --2- 172.21.15.12:0/227737685 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff8a40a3d20 0x7ff8a40a61e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0009900 tx=0x7ff8a0005fd0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.792+0000 7ff8baaf2640 1 -- 172.21.15.12:0/227737685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8b406a360 msgr2=0x7ff8b419e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.792+0000 7ff8baaf2640 1 --2- 172.21.15.12:0/227737685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8b406a360 0x7ff8b419e400 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ff8a800d950 tx=0x7ff8a800de10 comp rx=0 tx=0).stop 2024-04-22T13:25:49.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.792+0000 7ff8baaf2640 1 -- 172.21.15.12:0/227737685 shutdown_connections 2024-04-22T13:25:49.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.792+0000 7ff8baaf2640 1 --2- 172.21.15.12:0/227737685 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff8a40a3d20 0x7ff8a40a61e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.792+0000 7ff8baaf2640 1 --2- 172.21.15.12:0/227737685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8b4107fd0 0x7ff8b41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.792+0000 7ff8baaf2640 1 --2- 172.21.15.12:0/227737685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8b406ace0 0x7ff8b419e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.792+0000 7ff8baaf2640 1 --2- 172.21.15.12:0/227737685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8b406a360 0x7ff8b419e400 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.792+0000 7ff8baaf2640 1 -- 172.21.15.12:0/227737685 >> 172.21.15.12:0/227737685 conn(0x7ff8b4077ef0 msgr2=0x7ff8b40ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:49.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.792+0000 7ff8baaf2640 1 -- 172.21.15.12:0/227737685 shutdown_connections 2024-04-22T13:25:49.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.792+0000 7ff8baaf2640 1 -- 172.21.15.12:0/227737685 wait complete. 2024-04-22T13:25:49.799 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:25:49.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.913+0000 7fe021e4e640 1 -- 172.21.15.12:0/923496730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe01c06ace0 msgr2=0x7fe01c10ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.913+0000 7fe021e4e640 1 --2- 172.21.15.12:0/923496730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe01c06ace0 0x7fe01c10ee00 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe014009cd0 tx=0x7fe01402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.913+0000 7fe021e4e640 1 -- 172.21.15.12:0/923496730 shutdown_connections 2024-04-22T13:25:49.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.913+0000 7fe021e4e640 1 --2- 172.21.15.12:0/923496730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe01c06ace0 0x7fe01c10ee00 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.909 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.913+0000 7fe021e4e640 1 --2- 172.21.15.12:0/923496730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe01c06a360 0x7fe01c06a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.913+0000 7fe021e4e640 1 --2- 172.21.15.12:0/923496730 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe01c107fd0 0x7fe01c1083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.913+0000 7fe021e4e640 1 -- 172.21.15.12:0/923496730 >> 172.21.15.12:0/923496730 conn(0x7fe01c077ef0 msgr2=0x7fe01c078300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:49.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.913+0000 7fe021e4e640 1 -- 172.21.15.12:0/923496730 shutdown_connections 2024-04-22T13:25:49.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.913+0000 7fe021e4e640 1 -- 172.21.15.12:0/923496730 wait complete. 2024-04-22T13:25:49.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.913+0000 7fe021e4e640 1 Processor -- start 2024-04-22T13:25:49.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.914+0000 7fe021e4e640 1 -- start start 2024-04-22T13:25:49.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.914+0000 7fe021e4e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe01c06a360 0x7fe01c100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.914+0000 7fe021e4e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe01c06ace0 0x7fe01c1013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.914+0000 7fe021e4e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe01c107fd0 0x7fe01c101900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.914+0000 7fe021e4e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe01c1121b0 con 0x7fe01c06a360 2024-04-22T13:25:49.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.914+0000 7fe021e4e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe01c112030 con 0x7fe01c107fd0 2024-04-22T13:25:49.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.914+0000 7fe021e4e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe01c112330 con 0x7fe01c06ace0 2024-04-22T13:25:49.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.914+0000 7fe01b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe01c06a360 0x7fe01c100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.914+0000 7fe01b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe01c06a360 0x7fe01c100e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53058/0 (socket says 172.21.15.12:53058) 2024-04-22T13:25:49.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.914+0000 7fe01b7fe640 1 -- 172.21.15.12:0/1372746924 learned_addr learned my addr 172.21.15.12:0/1372746924 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:49.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.914+0000 7fe01affd640 1 --2- 172.21.15.12:0/1372746924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe01c06ace0 0x7fe01c1013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.914+0000 7fe01bfff640 1 --2- 172.21.15.12:0/1372746924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe01c107fd0 0x7fe01c101900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.914+0000 7fe01b7fe640 1 -- 172.21.15.12:0/1372746924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe01c06ace0 msgr2=0x7fe01c1013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.914+0000 7fe01b7fe640 1 --2- 172.21.15.12:0/1372746924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe01c06ace0 0x7fe01c1013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.914+0000 7fe01b7fe640 1 -- 172.21.15.12:0/1372746924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe01c107fd0 msgr2=0x7fe01c101900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.914+0000 7fe01b7fe640 1 --2- 172.21.15.12:0/1372746924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe01c107fd0 0x7fe01c101900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.914+0000 7fe01b7fe640 1 -- 172.21.15.12:0/1372746924 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01c1a6c30 con 0x7fe01c06a360 2024-04-22T13:25:49.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.915+0000 7fe01b7fe640 1 --2- 172.21.15.12:0/1372746924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe01c06a360 0x7fe01c100e80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe00c00e9f0 tx=0x7fe00c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:49.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.915+0000 7fe018ff9640 1 -- 172.21.15.12:0/1372746924 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00c00cd80 con 0x7fe01c06a360 2024-04-22T13:25:49.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.915+0000 7fe021e4e640 1 -- 172.21.15.12:0/1372746924 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe01c1a6f20 con 0x7fe01c06a360 2024-04-22T13:25:49.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.915+0000 7fe021e4e640 1 -- 172.21.15.12:0/1372746924 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe01c1a7430 con 0x7fe01c06a360 2024-04-22T13:25:49.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.915+0000 7fe018ff9640 1 -- 172.21.15.12:0/1372746924 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe00c0108e0 con 0x7fe01c06a360 2024-04-22T13:25:49.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.915+0000 7fe018ff9640 1 -- 172.21.15.12:0/1372746924 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00c0188b0 con 0x7fe01c06a360 2024-04-22T13:25:49.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.917+0000 7fe024a41740 1 Processor -- start 2024-04-22T13:25:49.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.917+0000 7fe024a41740 1 -- start start 2024-04-22T13:25:49.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.917+0000 7fe018ff9640 1 -- 172.21.15.12:0/1372746924 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe00c018a50 con 0x7fe01c06a360 2024-04-22T13:25:49.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.917+0000 7fe024a41740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614bb5224d0 0x5614bb5228b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.918+0000 7fe024a41740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614bb522e80 0x5614bb52b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.918+0000 7fe024a41740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614bb52ba00 0x5614bb52de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.918+0000 7fe024a41740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5614bb4871f0 con 0x5614bb52ba00 2024-04-22T13:25:49.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.918+0000 7fe024a41740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5614bb4244b0 con 0x5614bb522e80 2024-04-22T13:25:49.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.918+0000 7fe024a41740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5614bb4831b0 con 0x5614bb5224d0 2024-04-22T13:25:49.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.918+0000 7fe01affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614bb52ba00 0x5614bb52de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.918+0000 7fe01affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614bb52ba00 0x5614bb52de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53062/0 (socket says 172.21.15.12:53062) 2024-04-22T13:25:49.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.918+0000 7fe01affd640 1 -- 172.21.15.12:0/3670083442 learned_addr learned my addr 172.21.15.12:0/3670083442 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:49.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.918+0000 7fe01b7fe640 1 --2- 172.21.15.12:0/3670083442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614bb522e80 0x5614bb52b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.918+0000 7fe01bfff640 1 --2- 172.21.15.12:0/3670083442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614bb5224d0 0x5614bb5228b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.918+0000 7fe01affd640 1 -- 172.21.15.12:0/3670083442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614bb5224d0 msgr2=0x5614bb5228b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:49.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.918+0000 7fe01affd640 1 --2- 172.21.15.12:0/3670083442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614bb5224d0 0x5614bb5228b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.918+0000 7fe01affd640 1 -- 172.21.15.12:0/3670083442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614bb522e80 msgr2=0x5614bb52b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:49.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.918+0000 7fe01affd640 1 --2- 172.21.15.12:0/3670083442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614bb522e80 0x5614bb52b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.918+0000 7fe018ff9640 1 --2- 172.21.15.12:0/1372746924 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe0080a3d20 0x7fe0080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.918+0000 7fe01affd640 1 -- 172.21.15.12:0/3670083442 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614bb4161b0 con 0x5614bb52ba00 2024-04-22T13:25:49.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.918+0000 7fe01bfff640 1 --2- 172.21.15.12:0/1372746924 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe0080a3d20 0x7fe0080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.918+0000 7fe01affd640 1 --2- 172.21.15.12:0/3670083442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614bb52ba00 0x5614bb52de50 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe004009ca0 tx=0x7fe004011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dda9ac15e641d7c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:49.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.918+0000 7fe018ff9640 1 -- 172.21.15.12:0/1372746924 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fe00c0d08b0 con 0x7fe01c06a360 2024-04-22T13:25:49.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.918+0000 7fe011ffb640 1 -- 172.21.15.12:0/3670083442 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00401e070 con 0x5614bb52ba00 2024-04-22T13:25:49.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.919+0000 7fe011ffb640 1 -- 172.21.15.12:0/3670083442 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe004019ae0 con 0x5614bb52ba00 2024-04-22T13:25:49.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.919+0000 7fe011ffb640 1 -- 172.21.15.12:0/3670083442 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00400e9b0 con 0x5614bb52ba00 2024-04-22T13:25:49.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.919+0000 7fe024a41740 1 -- 172.21.15.12:0/3670083442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614bb52ba00 msgr2=0x5614bb52de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.919+0000 7fe024a41740 1 --2- 172.21.15.12:0/3670083442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614bb52ba00 0x5614bb52de50 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe004009ca0 tx=0x7fe004011810 comp rx=0 tx=0).stop 2024-04-22T13:25:49.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.919+0000 7fe01bfff640 1 --2- 172.21.15.12:0/1372746924 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe0080a3d20 0x7fe0080a61e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe01402c630 tx=0x7fe01402e040 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:49.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.919+0000 7fe024a41740 1 -- 172.21.15.12:0/3670083442 shutdown_connections 2024-04-22T13:25:49.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.919+0000 7fe024a41740 1 --2- 172.21.15.12:0/3670083442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614bb52ba00 0x5614bb52de50 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.919+0000 7fe024a41740 1 --2- 172.21.15.12:0/3670083442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614bb522e80 0x5614bb52b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.919+0000 7fe024a41740 1 --2- 172.21.15.12:0/3670083442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614bb5224d0 0x5614bb5228b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.919+0000 7fe024a41740 1 -- 172.21.15.12:0/3670083442 >> 172.21.15.12:0/3670083442 conn(0x5614bb51cba0 msgr2=0x5614bb51cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:49.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.919+0000 7fe024a41740 1 -- 172.21.15.12:0/3670083442 shutdown_connections 2024-04-22T13:25:49.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.919+0000 7fe024a41740 1 -- 172.21.15.12:0/3670083442 wait complete. 2024-04-22T13:25:49.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.920+0000 7fe024a41740 1 -- start start 2024-04-22T13:25:49.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.920+0000 7fe024a41740 1 Processor -- start 2024-04-22T13:25:49.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.921+0000 7fe024a41740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614bb5224d0 0x5614bb52b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.921+0000 7fe024a41740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614bb522e80 0x5614bb528620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.921+0000 7fe024a41740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614bb52ba00 0x5614bb528b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.921+0000 7fe024a41740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5614bb52e970 con 0x5614bb52ba00 2024-04-22T13:25:49.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.921+0000 7fe024a41740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5614bb440500 con 0x5614bb522e80 2024-04-22T13:25:49.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.921+0000 7fe024a41740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5614bb52eaf0 con 0x5614bb5224d0 2024-04-22T13:25:49.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.921+0000 7fe024a41740 10 client.0 _ll_register_callbacks cb 0x5614bb40eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:49.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.921+0000 7fe01bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614bb52ba00 0x5614bb528b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.921+0000 7fe01bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614bb52ba00 0x5614bb528b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53070/0 (socket says 172.21.15.12:53070) 2024-04-22T13:25:49.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.921+0000 7fe01bfff640 1 -- 172.21.15.12:0/2927896391 learned_addr learned my addr 172.21.15.12:0/2927896391 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:49.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.921+0000 7fe01b7fe640 1 --2- 172.21.15.12:0/2927896391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614bb5224d0 0x5614bb52b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.921+0000 7fe01affd640 1 --2- 172.21.15.12:0/2927896391 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614bb522e80 0x5614bb528620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.921+0000 7fe01bfff640 1 -- 172.21.15.12:0/2927896391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614bb5224d0 msgr2=0x5614bb52b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:49.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.921+0000 7fe01bfff640 1 --2- 172.21.15.12:0/2927896391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614bb5224d0 0x5614bb52b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.921+0000 7fe01bfff640 1 -- 172.21.15.12:0/2927896391 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614bb522e80 msgr2=0x5614bb528620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:49.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.921+0000 7fe01bfff640 1 --2- 172.21.15.12:0/2927896391 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614bb522e80 0x5614bb528620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.921+0000 7fe01bfff640 1 -- 172.21.15.12:0/2927896391 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614bb1993f0 con 0x5614bb52ba00 2024-04-22T13:25:49.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.921+0000 7fe01affd640 1 --2- 172.21.15.12:0/2927896391 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614bb522e80 0x5614bb528620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:49.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.921+0000 7fe01bfff640 1 --2- 172.21.15.12:0/2927896391 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614bb52ba00 0x5614bb528b60 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe014006010 tx=0x7fe01402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e53130806c2a3c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:49.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.921+0000 7fdffaffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:49.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.921+0000 7fdffaffd640 1 -- 172.21.15.12:0/2927896391 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe014051060 con 0x5614bb52ba00 2024-04-22T13:25:49.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.921+0000 7fdffaffd640 1 -- 172.21.15.12:0/2927896391 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fe014003970 con 0x5614bb52ba00 2024-04-22T13:25:49.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.921+0000 7fe024a41740 1 -- 172.21.15.12:0/2927896391 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe004019ae0 con 0x5614bb52ba00 2024-04-22T13:25:49.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.922+0000 7fdffaffd640 1 -- 172.21.15.12:0/2927896391 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe014039040 con 0x5614bb52ba00 2024-04-22T13:25:49.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.922+0000 7fe024a41740 10 client.16386 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:49.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.922+0000 7fe024a41740 1 -- 172.21.15.12:0/2927896391 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5614bb2637d0 con 0x5614bb52ba00 2024-04-22T13:25:49.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.922+0000 7fdffaffd640 1 -- 172.21.15.12:0/2927896391 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe004019ae0 con 0x5614bb52ba00 2024-04-22T13:25:49.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.922+0000 7fdffaffd640 1 -- 172.21.15.12:0/2927896391 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe01404c460 con 0x5614bb52ba00 2024-04-22T13:25:49.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.922+0000 7fe024a41740 10 client.16386 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:49.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.922+0000 7fe024a41740 10 client.16386 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:25:49.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.922+0000 7fe024a41740 20 client.16386 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:49.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.922+0000 7fe024a41740 1 --2- 172.21.15.12:0/2927896391 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5614bb53bc20 0x5614bb53e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.922+0000 7fe024a41740 4 client.16386 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:25:49.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.922+0000 7fe024a41740 1 -- 172.21.15.12:0/2927896391 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614bb426930 con 0x5614bb53bc20 2024-04-22T13:25:49.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.923+0000 7fe01b7fe640 1 --2- 172.21.15.12:0/2927896391 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5614bb53bc20 0x5614bb53e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.923+0000 7fe01b7fe640 1 --2- 172.21.15.12:0/2927896391 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5614bb53bc20 0x5614bb53e030 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=91a658a439742dff server_cookie=a32968d8daa1cca2 in_seq=0 out_seq=0 2024-04-22T13:25:49.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.923+0000 7fdffaffd640 10 client.16386 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:25:49.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.924+0000 7fdffaffd640 1 -- 172.21.15.12:0/2927896391 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5614bb426930 con 0x5614bb53bc20 2024-04-22T13:25:49.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.924+0000 7fdffaffd640 10 client.16386 handle_command_reply: tid=0 2024-04-22T13:25:49.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.924+0000 7fe024a41740 1 client.16386 shutdown 2024-04-22T13:25:49.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.924+0000 7fe024a41740 1 -- 172.21.15.12:0/2927896391 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614bb52ba00 msgr2=0x5614bb528b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.924+0000 7fe024a41740 1 --2- 172.21.15.12:0/2927896391 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614bb52ba00 0x5614bb528b60 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe014006010 tx=0x7fe01402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.924+0000 7fe024a41740 1 -- 172.21.15.12:0/2927896391 shutdown_connections 2024-04-22T13:25:49.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.924+0000 7fe024a41740 1 --2- 172.21.15.12:0/2927896391 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5614bb53bc20 0x5614bb53e030 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.924+0000 7fe024a41740 1 --2- 172.21.15.12:0/2927896391 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614bb52ba00 0x5614bb528b60 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.924+0000 7fe024a41740 1 --2- 172.21.15.12:0/2927896391 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614bb522e80 0x5614bb528620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.924+0000 7fdffaffd640 0 client.16386 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:25:49.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.924+0000 7fe024a41740 1 --2- 172.21.15.12:0/2927896391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614bb5224d0 0x5614bb52b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.924+0000 7fe024a41740 1 -- 172.21.15.12:0/2927896391 >> 172.21.15.12:0/2927896391 conn(0x5614bb51cba0 msgr2=0x5614bb51e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:49.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.924+0000 7fe024a41740 1 -- 172.21.15.12:0/2927896391 shutdown_connections 2024-04-22T13:25:49.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.924+0000 7fe024a41740 1 -- 172.21.15.12:0/2927896391 wait complete. 2024-04-22T13:25:49.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.924+0000 7fe024a41740 20 client.16386 trim_cache size 0 max 16384 2024-04-22T13:25:49.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.949+0000 7fe024a41740 1 Processor -- start 2024-04-22T13:25:49.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.949+0000 7fe024a41740 1 -- start start 2024-04-22T13:25:49.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.949+0000 7fe024a41740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614bb5224d0 0x5614bb528b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.949+0000 7fe024a41740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614bb522e80 0x5614bb537830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.949+0000 7fe024a41740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614bb52ba00 0x5614bb52ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.949+0000 7fe024a41740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc00e5b0 con 0x5614bb5224d0 2024-04-22T13:25:49.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.949+0000 7fe024a41740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc0100f0 con 0x5614bb52ba00 2024-04-22T13:25:49.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.949+0000 7fe024a41740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc00ec80 con 0x5614bb522e80 2024-04-22T13:25:49.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.949+0000 7fe01b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614bb5224d0 0x5614bb528b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.949+0000 7fe01b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614bb5224d0 0x5614bb528b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53084/0 (socket says 172.21.15.12:53084) 2024-04-22T13:25:49.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.949+0000 7fe01b7fe640 1 -- 172.21.15.12:0/3147864460 learned_addr learned my addr 172.21.15.12:0/3147864460 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:49.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.949+0000 7fe01affd640 1 --2- 172.21.15.12:0/3147864460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614bb522e80 0x5614bb537830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.949+0000 7fe01bfff640 1 --2- 172.21.15.12:0/3147864460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614bb52ba00 0x5614bb52ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.949+0000 7fe01b7fe640 1 -- 172.21.15.12:0/3147864460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614bb522e80 msgr2=0x5614bb537830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:49.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.949+0000 7fe01b7fe640 1 --2- 172.21.15.12:0/3147864460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614bb522e80 0x5614bb537830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.949+0000 7fe01b7fe640 1 -- 172.21.15.12:0/3147864460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614bb52ba00 msgr2=0x5614bb52ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:49.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.949+0000 7fe01b7fe640 1 --2- 172.21.15.12:0/3147864460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614bb52ba00 0x5614bb52ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.949+0000 7fe01b7fe640 1 -- 172.21.15.12:0/3147864460 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614bb522c20 con 0x5614bb5224d0 2024-04-22T13:25:49.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.950+0000 7fe01b7fe640 1 --2- 172.21.15.12:0/3147864460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614bb5224d0 0x5614bb528b40 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe00c09c8e0 tx=0x7fe00c087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd5241355128095a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:49.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.950+0000 7fe0117fa640 1 -- 172.21.15.12:0/3147864460 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00c09ac50 con 0x5614bb5224d0 2024-04-22T13:25:49.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.950+0000 7fe0117fa640 1 -- 172.21.15.12:0/3147864460 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe00c0ce2d0 con 0x5614bb5224d0 2024-04-22T13:25:49.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.950+0000 7fe0117fa640 1 -- 172.21.15.12:0/3147864460 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00c09bda0 con 0x5614bb5224d0 2024-04-22T13:25:49.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.950+0000 7fe024a41740 1 -- 172.21.15.12:0/3147864460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614bb5224d0 msgr2=0x5614bb528b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.950+0000 7fe024a41740 1 --2- 172.21.15.12:0/3147864460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614bb5224d0 0x5614bb528b40 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe00c09c8e0 tx=0x7fe00c087040 comp rx=0 tx=0).stop 2024-04-22T13:25:49.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.951+0000 7fe024a41740 1 -- 172.21.15.12:0/3147864460 shutdown_connections 2024-04-22T13:25:49.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.951+0000 7fe024a41740 1 --2- 172.21.15.12:0/3147864460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614bb52ba00 0x5614bb52ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.951+0000 7fe024a41740 1 --2- 172.21.15.12:0/3147864460 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614bb522e80 0x5614bb537830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.951+0000 7fe024a41740 1 --2- 172.21.15.12:0/3147864460 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614bb5224d0 0x5614bb528b40 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.951+0000 7fe024a41740 1 -- 172.21.15.12:0/3147864460 >> 172.21.15.12:0/3147864460 conn(0x5614bb51cba0 msgr2=0x5614bb51bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:49.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.951+0000 7fe024a41740 1 -- 172.21.15.12:0/3147864460 shutdown_connections 2024-04-22T13:25:49.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.951+0000 7fe024a41740 1 -- 172.21.15.12:0/3147864460 wait complete. 2024-04-22T13:25:49.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.951+0000 7fe024a41740 1 -- start start 2024-04-22T13:25:49.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.951+0000 7fe024a41740 1 Processor -- start 2024-04-22T13:25:49.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.951+0000 7fe024a41740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614bb5224d0 0x5614bb52f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.951+0000 7fe024a41740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614bb522e80 0x5614bb52fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.951+0000 7fe024a41740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614bb52ba00 0x5614bb530310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.951+0000 7fe024a41740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc00fa20 con 0x5614bb5224d0 2024-04-22T13:25:49.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.951+0000 7fe024a41740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc00f350 con 0x5614bb522e80 2024-04-22T13:25:49.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.951+0000 7fe024a41740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc00c590 con 0x5614bb52ba00 2024-04-22T13:25:49.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.951+0000 7fe024a41740 10 client.0 _ll_register_callbacks cb 0x7fe014009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:49.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.952+0000 7fe01b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614bb5224d0 0x5614bb52f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.952+0000 7fe01b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614bb5224d0 0x5614bb52f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53100/0 (socket says 172.21.15.12:53100) 2024-04-22T13:25:49.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.952+0000 7fe01b7fe640 1 -- 172.21.15.12:0/3225891945 learned_addr learned my addr 172.21.15.12:0/3225891945 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:49.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.952+0000 7fe01bfff640 1 --2- 172.21.15.12:0/3225891945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614bb52ba00 0x5614bb530310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.952+0000 7fe01affd640 1 --2- 172.21.15.12:0/3225891945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614bb522e80 0x5614bb52fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.952+0000 7fe01b7fe640 1 -- 172.21.15.12:0/3225891945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614bb52ba00 msgr2=0x5614bb530310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:49.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.952+0000 7fe01b7fe640 1 --2- 172.21.15.12:0/3225891945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614bb52ba00 0x5614bb530310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.952+0000 7fe01b7fe640 1 -- 172.21.15.12:0/3225891945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614bb522e80 msgr2=0x5614bb52fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:49.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.952+0000 7fe01b7fe640 1 --2- 172.21.15.12:0/3225891945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614bb522e80 0x5614bb52fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.952+0000 7fe01b7fe640 1 -- 172.21.15.12:0/3225891945 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614bb5a12e0 con 0x5614bb5224d0 2024-04-22T13:25:49.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.952+0000 7fe01b7fe640 1 --2- 172.21.15.12:0/3225891945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614bb5224d0 0x5614bb52f890 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fe00c099320 tx=0x7fe00c005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9b0da050cccb98 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:49.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.952+0000 7fdffaffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:49.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.952+0000 7fdffaffd640 1 -- 172.21.15.12:0/3225891945 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00c087220 con 0x5614bb5224d0 2024-04-22T13:25:49.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.952+0000 7fdffaffd640 1 -- 172.21.15.12:0/3225891945 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fe00c09dbd0 con 0x5614bb5224d0 2024-04-22T13:25:49.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.952+0000 7fe024a41740 1 -- 172.21.15.12:0/3225891945 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe00c0ce2d0 con 0x5614bb5224d0 2024-04-22T13:25:49.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.952+0000 7fdffaffd640 1 -- 172.21.15.12:0/3225891945 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe00c09ddd0 con 0x5614bb5224d0 2024-04-22T13:25:49.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.952+0000 7fe024a41740 10 client.16392 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:49.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.952+0000 7fe024a41740 1 -- 172.21.15.12:0/3225891945 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5614bb5a25c0 con 0x5614bb5224d0 2024-04-22T13:25:49.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.953+0000 7fdffaffd640 1 -- 172.21.15.12:0/3225891945 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe00c0ce2d0 con 0x5614bb5224d0 2024-04-22T13:25:49.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.953+0000 7fdffaffd640 1 -- 172.21.15.12:0/3225891945 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe00c0986a0 con 0x5614bb5224d0 2024-04-22T13:25:49.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.953+0000 7fe024a41740 10 client.16392 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:49.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.953+0000 7fe024a41740 10 client.16392 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:25:49.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.953+0000 7fe024a41740 20 client.16392 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:49.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.953+0000 7fe024a41740 1 --2- 172.21.15.12:0/3225891945 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5614bb5a2c70 0x5614bb5a5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:49.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.953+0000 7fe024a41740 4 client.16392 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:25:49.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.953+0000 7fe024a41740 1 -- 172.21.15.12:0/3225891945 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5614bb494c10 con 0x5614bb5a2c70 2024-04-22T13:25:49.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.953+0000 7fe01affd640 1 --2- 172.21.15.12:0/3225891945 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5614bb5a2c70 0x5614bb5a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:49.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.954+0000 7fe01affd640 1 --2- 172.21.15.12:0/3225891945 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5614bb5a2c70 0x5614bb5a5090 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f9fc0de6bdfd0c55 server_cookie=176b916ad084728 in_seq=0 out_seq=0 2024-04-22T13:25:49.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.954+0000 7fdffaffd640 10 client.16392 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:25:49.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.954+0000 7fdffaffd640 1 -- 172.21.15.12:0/3225891945 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5614bb494c10 con 0x5614bb5a2c70 2024-04-22T13:25:49.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.954+0000 7fdffaffd640 10 client.16392 handle_command_reply: tid=0 2024-04-22T13:25:49.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.954+0000 7fe024a41740 1 client.16392 shutdown 2024-04-22T13:25:49.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.954+0000 7fe024a41740 1 -- 172.21.15.12:0/3225891945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614bb5224d0 msgr2=0x5614bb52f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.954+0000 7fe024a41740 1 --2- 172.21.15.12:0/3225891945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614bb5224d0 0x5614bb52f890 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fe00c099320 tx=0x7fe00c005c50 comp rx=0 tx=0).stop 2024-04-22T13:25:49.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.954+0000 7fe024a41740 1 -- 172.21.15.12:0/3225891945 shutdown_connections 2024-04-22T13:25:49.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.954+0000 7fe024a41740 1 --2- 172.21.15.12:0/3225891945 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5614bb5a2c70 0x5614bb5a5090 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.954+0000 7fe024a41740 1 --2- 172.21.15.12:0/3225891945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614bb52ba00 0x5614bb530310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.954+0000 7fe024a41740 1 --2- 172.21.15.12:0/3225891945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614bb522e80 0x5614bb52fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.954+0000 7fdffaffd640 0 client.16392 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:25:49.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.954+0000 7fe024a41740 1 --2- 172.21.15.12:0/3225891945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614bb5224d0 0x5614bb52f890 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.954+0000 7fe024a41740 1 -- 172.21.15.12:0/3225891945 >> 172.21.15.12:0/3225891945 conn(0x5614bb51cba0 msgr2=0x5614bb523260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:49.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.954+0000 7fe024a41740 1 -- 172.21.15.12:0/3225891945 shutdown_connections 2024-04-22T13:25:49.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.954+0000 7fe024a41740 1 -- 172.21.15.12:0/3225891945 wait complete. 2024-04-22T13:25:49.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.954+0000 7fe024a41740 20 client.16392 trim_cache size 0 max 16384 2024-04-22T13:25:49.956 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:25:49.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.955+0000 7fe021e4e640 1 -- 172.21.15.12:0/1372746924 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe0080a3d20 msgr2=0x7fe0080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.955+0000 7fe021e4e640 1 --2- 172.21.15.12:0/1372746924 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe0080a3d20 0x7fe0080a61e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe01402c630 tx=0x7fe01402e040 comp rx=0 tx=0).stop 2024-04-22T13:25:49.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.955+0000 7fe021e4e640 1 -- 172.21.15.12:0/1372746924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe01c06a360 msgr2=0x7fe01c100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:49.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.955+0000 7fe021e4e640 1 --2- 172.21.15.12:0/1372746924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe01c06a360 0x7fe01c100e80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe00c00e9f0 tx=0x7fe00c00eeb0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.955+0000 7fe021e4e640 1 -- 172.21.15.12:0/1372746924 shutdown_connections 2024-04-22T13:25:49.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.955+0000 7fe021e4e640 1 --2- 172.21.15.12:0/1372746924 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe0080a3d20 0x7fe0080a61e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.955+0000 7fe021e4e640 1 --2- 172.21.15.12:0/1372746924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe01c107fd0 0x7fe01c101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.955+0000 7fe021e4e640 1 --2- 172.21.15.12:0/1372746924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe01c06ace0 0x7fe01c1013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.955+0000 7fe021e4e640 1 --2- 172.21.15.12:0/1372746924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe01c06a360 0x7fe01c100e80 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:49.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.955+0000 7fe021e4e640 1 -- 172.21.15.12:0/1372746924 >> 172.21.15.12:0/1372746924 conn(0x7fe01c077ef0 msgr2=0x7fe01c0ffb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:49.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.955+0000 7fe021e4e640 1 -- 172.21.15.12:0/1372746924 shutdown_connections 2024-04-22T13:25:49.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:49.955+0000 7fe021e4e640 1 -- 172.21.15.12:0/1372746924 wait complete. 2024-04-22T13:25:49.961 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:49 smithi012 ceph-mon[28265]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-04-22T13:25:49.961 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:49 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3603828510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:25:49.963 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:25:50.029 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:49 smithi111 ceph-mon[31361]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-04-22T13:25:50.029 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:49 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/3603828510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:25:50.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.076+0000 7f5c44a5c640 1 -- 172.21.15.12:0/2821798342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c40101090 msgr2=0x7f5c40101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.076+0000 7f5c44a5c640 1 --2- 172.21.15.12:0/2821798342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c40101090 0x7f5c40101470 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f5c30009900 tx=0x7f5c3002bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:50.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.076+0000 7f5c44a5c640 1 -- 172.21.15.12:0/2821798342 shutdown_connections 2024-04-22T13:25:50.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.076+0000 7f5c44a5c640 1 --2- 172.21.15.12:0/2821798342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c40106160 0x7f5c4010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.076+0000 7f5c44a5c640 1 --2- 172.21.15.12:0/2821798342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c40101a40 0x7f5c40105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.076+0000 7f5c44a5c640 1 --2- 172.21.15.12:0/2821798342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c40101090 0x7f5c40101470 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.076+0000 7f5c44a5c640 1 -- 172.21.15.12:0/2821798342 >> 172.21.15.12:0/2821798342 conn(0x7f5c40077f50 msgr2=0x7f5c40078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:50.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.076+0000 7f5c44a5c640 1 -- 172.21.15.12:0/2821798342 shutdown_connections 2024-04-22T13:25:50.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.076+0000 7f5c44a5c640 1 -- 172.21.15.12:0/2821798342 wait complete. 2024-04-22T13:25:50.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.077+0000 7f5c44a5c640 1 Processor -- start 2024-04-22T13:25:50.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.077+0000 7f5c44a5c640 1 -- start start 2024-04-22T13:25:50.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.077+0000 7f5c44a5c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c40101090 0x7f5c4010e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.077+0000 7f5c44a5c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c40106160 0x7f5c4010ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.077+0000 7f5c44a5c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c4010f140 0x7f5c401a6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.077+0000 7f5c44a5c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5c401121e0 con 0x7f5c4010f140 2024-04-22T13:25:50.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.077+0000 7f5c44a5c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5c40112060 con 0x7f5c40101090 2024-04-22T13:25:50.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.077+0000 7f5c44a5c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5c40112360 con 0x7f5c40106160 2024-04-22T13:25:50.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.077+0000 7f5c3ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c4010f140 0x7f5c401a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.077+0000 7f5c3ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c4010f140 0x7f5c401a6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53116/0 (socket says 172.21.15.12:53116) 2024-04-22T13:25:50.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.077+0000 7f5c3e59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c40101090 0x7f5c4010e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.077+0000 7f5c3ed9d640 1 -- 172.21.15.12:0/2238194930 learned_addr learned my addr 172.21.15.12:0/2238194930 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:50.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.077+0000 7f5c3dd9b640 1 --2- 172.21.15.12:0/2238194930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c40106160 0x7f5c4010ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.078+0000 7f5c3ed9d640 1 -- 172.21.15.12:0/2238194930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c40106160 msgr2=0x7f5c4010ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.078+0000 7f5c3ed9d640 1 --2- 172.21.15.12:0/2238194930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c40106160 0x7f5c4010ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.078+0000 7f5c3ed9d640 1 -- 172.21.15.12:0/2238194930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c40101090 msgr2=0x7f5c4010e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.078+0000 7f5c3ed9d640 1 --2- 172.21.15.12:0/2238194930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c40101090 0x7f5c4010e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.078+0000 7f5c3ed9d640 1 -- 172.21.15.12:0/2238194930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c401a7050 con 0x7f5c4010f140 2024-04-22T13:25:50.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.078+0000 7f5c3dd9b640 1 --2- 172.21.15.12:0/2238194930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c40106160 0x7f5c4010ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:50.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.078+0000 7f5c3ed9d640 1 --2- 172.21.15.12:0/2238194930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c4010f140 0x7f5c401a6950 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f5c38009fd0 tx=0x7f5c3800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:50.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.078+0000 7f5c377fe640 1 -- 172.21.15.12:0/2238194930 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c38017070 con 0x7f5c4010f140 2024-04-22T13:25:50.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.078+0000 7f5c44a5c640 1 -- 172.21.15.12:0/2238194930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c401a7340 con 0x7f5c4010f140 2024-04-22T13:25:50.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.078+0000 7f5c377fe640 1 -- 172.21.15.12:0/2238194930 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c38003e20 con 0x7f5c4010f140 2024-04-22T13:25:50.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.078+0000 7f5c377fe640 1 -- 172.21.15.12:0/2238194930 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c38010460 con 0x7f5c4010f140 2024-04-22T13:25:50.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.078+0000 7f5c44a5c640 1 -- 172.21.15.12:0/2238194930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c401a7890 con 0x7f5c4010f140 2024-04-22T13:25:50.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.081+0000 7f5c47651740 1 Processor -- start 2024-04-22T13:25:50.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.081+0000 7f5c377fe640 1 -- 172.21.15.12:0/2238194930 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5c38003970 con 0x7f5c4010f140 2024-04-22T13:25:50.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.082+0000 7f5c47651740 1 -- start start 2024-04-22T13:25:50.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.082+0000 7f5c47651740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1aa4784d0 0x55c1aa4788b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.082+0000 7f5c47651740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1aa478e80 0x55c1aa481430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.082+0000 7f5c47651740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1aa481a00 0x55c1aa483e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.082+0000 7f5c47651740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c1aa3dd1f0 con 0x55c1aa478e80 2024-04-22T13:25:50.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.082+0000 7f5c47651740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c1aa37a4b0 con 0x55c1aa481a00 2024-04-22T13:25:50.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.082+0000 7f5c47651740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c1aa3d91b0 con 0x55c1aa4784d0 2024-04-22T13:25:50.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.082+0000 7f5c3e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1aa478e80 0x55c1aa481430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.082+0000 7f5c3e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1aa478e80 0x55c1aa481430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53132/0 (socket says 172.21.15.12:53132) 2024-04-22T13:25:50.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.082+0000 7f5c3e59c640 1 -- 172.21.15.12:0/3419993800 learned_addr learned my addr 172.21.15.12:0/3419993800 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:50.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.082+0000 7f5c3dd9b640 1 --2- 172.21.15.12:0/3419993800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1aa4784d0 0x55c1aa4788b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.082+0000 7f5c377fe640 1 --2- 172.21.15.12:0/2238194930 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5c2c0a3d50 0x7f5c2c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.082+0000 7f5c3dd9b640 1 --2- 172.21.15.12:0/3419993800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1aa481a00 0x55c1aa483e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.083+0000 7f5c3ed9d640 1 --2- 172.21.15.12:0/2238194930 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5c2c0a3d50 0x7f5c2c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.083+0000 7f5c3e59c640 1 -- 172.21.15.12:0/3419993800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1aa4784d0 msgr2=0x55c1aa4788b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:50.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.083+0000 7f5c3e59c640 1 --2- 172.21.15.12:0/3419993800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1aa4784d0 0x55c1aa4788b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.083+0000 7f5c3e59c640 1 -- 172.21.15.12:0/3419993800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1aa481a00 msgr2=0x55c1aa483e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:50.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.083+0000 7f5c3e59c640 1 --2- 172.21.15.12:0/3419993800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1aa481a00 0x55c1aa483e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.083+0000 7f5c3e59c640 1 -- 172.21.15.12:0/3419993800 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1aa36c1b0 con 0x55c1aa478e80 2024-04-22T13:25:50.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.083+0000 7f5c377fe640 1 -- 172.21.15.12:0/2238194930 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f5c380d0460 con 0x7f5c4010f140 2024-04-22T13:25:50.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.083+0000 7f5c3e59c640 1 --2- 172.21.15.12:0/3419993800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1aa478e80 0x55c1aa481430 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f5c30009900 tx=0x7f5c3002c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb3f152f68188a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:50.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.083+0000 7f5c3ed9d640 1 --2- 172.21.15.12:0/2238194930 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5c2c0a3d50 0x7f5c2c0a6210 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5c380cdcb0 tx=0x7f5c3801e780 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:50.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.083+0000 7f5c34ff9640 1 -- 172.21.15.12:0/3419993800 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c30045020 con 0x55c1aa478e80 2024-04-22T13:25:50.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.083+0000 7f5c34ff9640 1 -- 172.21.15.12:0/3419993800 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c3002c7e0 con 0x55c1aa478e80 2024-04-22T13:25:50.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.083+0000 7f5c34ff9640 1 -- 172.21.15.12:0/3419993800 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c300046c0 con 0x55c1aa478e80 2024-04-22T13:25:50.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.084+0000 7f5c47651740 1 -- 172.21.15.12:0/3419993800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1aa478e80 msgr2=0x55c1aa481430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.084+0000 7f5c47651740 1 --2- 172.21.15.12:0/3419993800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1aa478e80 0x55c1aa481430 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f5c30009900 tx=0x7f5c3002c6c0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.084+0000 7f5c47651740 1 -- 172.21.15.12:0/3419993800 shutdown_connections 2024-04-22T13:25:50.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.084+0000 7f5c47651740 1 --2- 172.21.15.12:0/3419993800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1aa481a00 0x55c1aa483e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.084+0000 7f5c47651740 1 --2- 172.21.15.12:0/3419993800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1aa478e80 0x55c1aa481430 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.084+0000 7f5c47651740 1 --2- 172.21.15.12:0/3419993800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1aa4784d0 0x55c1aa4788b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.084+0000 7f5c47651740 1 -- 172.21.15.12:0/3419993800 >> 172.21.15.12:0/3419993800 conn(0x55c1aa472ba0 msgr2=0x55c1aa472f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:50.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.084+0000 7f5c47651740 1 -- 172.21.15.12:0/3419993800 shutdown_connections 2024-04-22T13:25:50.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.084+0000 7f5c47651740 1 -- 172.21.15.12:0/3419993800 wait complete. 2024-04-22T13:25:50.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.084+0000 7f5c47651740 1 -- start start 2024-04-22T13:25:50.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.084+0000 7f5c47651740 1 Processor -- start 2024-04-22T13:25:50.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.085+0000 7f5c47651740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1aa4784d0 0x55c1aa4812d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.085+0000 7f5c47651740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1aa478e80 0x55c1aa47e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.085+0000 7f5c47651740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1aa481a00 0x55c1aa47ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.085+0000 7f5c47651740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c1aa484910 con 0x55c1aa478e80 2024-04-22T13:25:50.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.085+0000 7f5c47651740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c1aa396500 con 0x55c1aa481a00 2024-04-22T13:25:50.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.085+0000 7f5c47651740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c1aa484a90 con 0x55c1aa4784d0 2024-04-22T13:25:50.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.085+0000 7f5c47651740 10 client.0 _ll_register_callbacks cb 0x55c1aa364b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:50.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.085+0000 7f5c3e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1aa478e80 0x55c1aa47e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.085+0000 7f5c3e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1aa478e80 0x55c1aa47e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53142/0 (socket says 172.21.15.12:53142) 2024-04-22T13:25:50.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.085+0000 7f5c3e59c640 1 -- 172.21.15.12:0/1760799581 learned_addr learned my addr 172.21.15.12:0/1760799581 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:50.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.085+0000 7f5c3dd9b640 1 --2- 172.21.15.12:0/1760799581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1aa4784d0 0x55c1aa4812d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.085+0000 7f5c3dd9b640 1 --2- 172.21.15.12:0/1760799581 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1aa481a00 0x55c1aa47ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.085+0000 7f5c3e59c640 1 -- 172.21.15.12:0/1760799581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1aa4784d0 msgr2=0x55c1aa4812d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:50.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.085+0000 7f5c3e59c640 1 --2- 172.21.15.12:0/1760799581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1aa4784d0 0x55c1aa4812d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.085+0000 7f5c3e59c640 1 -- 172.21.15.12:0/1760799581 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1aa481a00 msgr2=0x55c1aa47ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:50.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.085+0000 7f5c3e59c640 1 --2- 172.21.15.12:0/1760799581 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1aa481a00 0x55c1aa47ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.085+0000 7f5c3e59c640 1 -- 172.21.15.12:0/1760799581 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1aa0ef3f0 con 0x55c1aa478e80 2024-04-22T13:25:50.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.085+0000 7f5c3e59c640 1 --2- 172.21.15.12:0/1760799581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1aa478e80 0x55c1aa47e660 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5c3002c170 tx=0x7f5c30005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a8b74f6e956192 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:50.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.085+0000 7f5c25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:50.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.086+0000 7f5c25ffb640 1 -- 172.21.15.12:0/1760799581 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c30045030 con 0x55c1aa478e80 2024-04-22T13:25:50.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.086+0000 7f5c25ffb640 1 -- 172.21.15.12:0/1760799581 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f5c300035f0 con 0x55c1aa478e80 2024-04-22T13:25:50.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.086+0000 7f5c47651740 1 -- 172.21.15.12:0/1760799581 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c3002c7e0 con 0x55c1aa478e80 2024-04-22T13:25:50.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.086+0000 7f5c25ffb640 1 -- 172.21.15.12:0/1760799581 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5c30058020 con 0x55c1aa478e80 2024-04-22T13:25:50.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.086+0000 7f5c47651740 10 client.16404 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:50.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.086+0000 7f5c47651740 1 -- 172.21.15.12:0/1760799581 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c1aa1b97d0 con 0x55c1aa478e80 2024-04-22T13:25:50.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.086+0000 7f5c25ffb640 1 -- 172.21.15.12:0/1760799581 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c3002c7e0 con 0x55c1aa478e80 2024-04-22T13:25:50.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.086+0000 7f5c25ffb640 1 -- 172.21.15.12:0/1760799581 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5c300049c0 con 0x55c1aa478e80 2024-04-22T13:25:50.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.086+0000 7f5c47651740 10 client.16404 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:50.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.086+0000 7f5c47651740 10 client.16404 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:25:50.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.086+0000 7f5c47651740 20 client.16404 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:50.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.086+0000 7f5c47651740 1 --2- 172.21.15.12:0/1760799581 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55c1aa491a00 0x55c1aa493df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.086+0000 7f5c47651740 4 client.16404 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:25:50.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.086+0000 7f5c47651740 1 -- 172.21.15.12:0/1760799581 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1aa37c930 con 0x55c1aa491a00 2024-04-22T13:25:50.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.087+0000 7f5c3dd9b640 1 --2- 172.21.15.12:0/1760799581 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55c1aa491a00 0x55c1aa493df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.087+0000 7f5c3dd9b640 1 --2- 172.21.15.12:0/1760799581 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55c1aa491a00 0x55c1aa493df0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=541629492589a6f3 server_cookie=c44ef850a7e8529f in_seq=0 out_seq=0 2024-04-22T13:25:50.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.087+0000 7f5c25ffb640 10 client.16404 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:25:50.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.088+0000 7f5c25ffb640 1 -- 172.21.15.12:0/1760799581 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c1aa37c930 con 0x55c1aa491a00 2024-04-22T13:25:50.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.088+0000 7f5c25ffb640 10 client.16404 handle_command_reply: tid=0 2024-04-22T13:25:50.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.088+0000 7f5c47651740 1 client.16404 shutdown 2024-04-22T13:25:50.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.088+0000 7f5c47651740 1 -- 172.21.15.12:0/1760799581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1aa478e80 msgr2=0x55c1aa47e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.088+0000 7f5c47651740 1 --2- 172.21.15.12:0/1760799581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1aa478e80 0x55c1aa47e660 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5c3002c170 tx=0x7f5c30005fd0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.088+0000 7f5c47651740 1 -- 172.21.15.12:0/1760799581 shutdown_connections 2024-04-22T13:25:50.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.088+0000 7f5c47651740 1 --2- 172.21.15.12:0/1760799581 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1aa481a00 0x55c1aa47ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.088+0000 7f5c47651740 1 --2- 172.21.15.12:0/1760799581 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55c1aa491a00 0x55c1aa493df0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.088+0000 7f5c47651740 1 --2- 172.21.15.12:0/1760799581 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1aa478e80 0x55c1aa47e660 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.088+0000 7f5c25ffb640 0 client.16404 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:25:50.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.088+0000 7f5c47651740 1 --2- 172.21.15.12:0/1760799581 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1aa4784d0 0x55c1aa4812d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.088+0000 7f5c47651740 1 -- 172.21.15.12:0/1760799581 >> 172.21.15.12:0/1760799581 conn(0x55c1aa472ba0 msgr2=0x55c1aa4748f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:50.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.088+0000 7f5c47651740 1 -- 172.21.15.12:0/1760799581 shutdown_connections 2024-04-22T13:25:50.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.088+0000 7f5c47651740 1 -- 172.21.15.12:0/1760799581 wait complete. 2024-04-22T13:25:50.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.088+0000 7f5c47651740 20 client.16404 trim_cache size 0 max 16384 2024-04-22T13:25:50.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.102+0000 7f5c47651740 1 Processor -- start 2024-04-22T13:25:50.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.102+0000 7f5c47651740 1 -- start start 2024-04-22T13:25:50.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.102+0000 7f5c47651740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1aa4784d0 0x55c1aa48d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.102+0000 7f5c47651740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1aa478e80 0x55c1aa471a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.102+0000 7f5c47651740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1aa481a00 0x55c1aa480d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.102+0000 7f5c47651740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5c401afc00 con 0x55c1aa478e80 2024-04-22T13:25:50.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.102+0000 7f5c47651740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5c401b1740 con 0x55c1aa481a00 2024-04-22T13:25:50.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.102+0000 7f5c47651740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5c401b02d0 con 0x55c1aa4784d0 2024-04-22T13:25:50.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.102+0000 7f5c3e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1aa478e80 0x55c1aa471a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.102+0000 7f5c3e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1aa478e80 0x55c1aa471a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53146/0 (socket says 172.21.15.12:53146) 2024-04-22T13:25:50.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.102+0000 7f5c3e59c640 1 -- 172.21.15.12:0/3318797423 learned_addr learned my addr 172.21.15.12:0/3318797423 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:50.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.102+0000 7f5c3dd9b640 1 --2- 172.21.15.12:0/3318797423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1aa4784d0 0x55c1aa48d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.102+0000 7f5c3dd9b640 1 --2- 172.21.15.12:0/3318797423 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1aa481a00 0x55c1aa480d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.103+0000 7f5c3e59c640 1 -- 172.21.15.12:0/3318797423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1aa4784d0 msgr2=0x55c1aa48d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:50.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.103+0000 7f5c3e59c640 1 --2- 172.21.15.12:0/3318797423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1aa4784d0 0x55c1aa48d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.103+0000 7f5c3e59c640 1 -- 172.21.15.12:0/3318797423 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1aa481a00 msgr2=0x55c1aa480d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:50.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.103+0000 7f5c3e59c640 1 --2- 172.21.15.12:0/3318797423 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1aa481a00 0x55c1aa480d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.103+0000 7f5c3e59c640 1 -- 172.21.15.12:0/3318797423 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1aa4777c0 con 0x55c1aa478e80 2024-04-22T13:25:50.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.103+0000 7f5c3dd9b640 1 --2- 172.21.15.12:0/3318797423 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1aa481a00 0x55c1aa480d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:50.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.103+0000 7f5c3e59c640 1 --2- 172.21.15.12:0/3318797423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1aa478e80 0x55c1aa471a80 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f5c30004f50 tx=0x7f5c30005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ef0daca31ae610 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:50.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.103+0000 7f5c27fff640 1 -- 172.21.15.12:0/3318797423 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c30062070 con 0x55c1aa478e80 2024-04-22T13:25:50.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.103+0000 7f5c27fff640 1 -- 172.21.15.12:0/3318797423 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c3002c7e0 con 0x55c1aa478e80 2024-04-22T13:25:50.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.103+0000 7f5c27fff640 1 -- 172.21.15.12:0/3318797423 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c30045070 con 0x55c1aa478e80 2024-04-22T13:25:50.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.104+0000 7f5c47651740 1 -- 172.21.15.12:0/3318797423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1aa478e80 msgr2=0x55c1aa471a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.104+0000 7f5c47651740 1 --2- 172.21.15.12:0/3318797423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1aa478e80 0x55c1aa471a80 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f5c30004f50 tx=0x7f5c30005050 comp rx=0 tx=0).stop 2024-04-22T13:25:50.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.104+0000 7f5c47651740 1 -- 172.21.15.12:0/3318797423 shutdown_connections 2024-04-22T13:25:50.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.104+0000 7f5c47651740 1 --2- 172.21.15.12:0/3318797423 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1aa481a00 0x55c1aa480d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.104+0000 7f5c47651740 1 --2- 172.21.15.12:0/3318797423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1aa478e80 0x55c1aa471a80 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.104+0000 7f5c47651740 1 --2- 172.21.15.12:0/3318797423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1aa4784d0 0x55c1aa48d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.104+0000 7f5c47651740 1 -- 172.21.15.12:0/3318797423 >> 172.21.15.12:0/3318797423 conn(0x55c1aa472ba0 msgr2=0x55c1aa471370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:50.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.104+0000 7f5c47651740 1 -- 172.21.15.12:0/3318797423 shutdown_connections 2024-04-22T13:25:50.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.104+0000 7f5c47651740 1 -- 172.21.15.12:0/3318797423 wait complete. 2024-04-22T13:25:50.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.104+0000 7f5c47651740 1 -- start start 2024-04-22T13:25:50.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.104+0000 7f5c47651740 1 Processor -- start 2024-04-22T13:25:50.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.104+0000 7f5c47651740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1aa4784d0 0x55c1aa48a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.104+0000 7f5c47651740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1aa478e80 0x55c1aa4853c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.104+0000 7f5c47651740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1aa481a00 0x55c1aa485900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.104+0000 7f5c47651740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5c401b1070 con 0x55c1aa478e80 2024-04-22T13:25:50.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.104+0000 7f5c47651740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5c401b09a0 con 0x55c1aa4784d0 2024-04-22T13:25:50.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.104+0000 7f5c47651740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5c401adc00 con 0x55c1aa481a00 2024-04-22T13:25:50.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.105+0000 7f5c47651740 10 client.0 _ll_register_callbacks cb 0x7f5c3002c610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:50.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.105+0000 7f5c3e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1aa478e80 0x55c1aa4853c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.105+0000 7f5c3e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1aa478e80 0x55c1aa4853c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53158/0 (socket says 172.21.15.12:53158) 2024-04-22T13:25:50.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.105+0000 7f5c3e59c640 1 -- 172.21.15.12:0/3328401568 learned_addr learned my addr 172.21.15.12:0/3328401568 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:50.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.105+0000 7f5c3dd9b640 1 --2- 172.21.15.12:0/3328401568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1aa481a00 0x55c1aa485900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.105+0000 7f5c3dd9b640 1 --2- 172.21.15.12:0/3328401568 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1aa4784d0 0x55c1aa48a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.105+0000 7f5c3e59c640 1 -- 172.21.15.12:0/3328401568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1aa481a00 msgr2=0x55c1aa485900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:50.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.105+0000 7f5c3e59c640 1 --2- 172.21.15.12:0/3328401568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1aa481a00 0x55c1aa485900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.105+0000 7f5c3e59c640 1 -- 172.21.15.12:0/3328401568 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1aa4784d0 msgr2=0x55c1aa48a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:50.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.105+0000 7f5c3e59c640 1 --2- 172.21.15.12:0/3328401568 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1aa4784d0 0x55c1aa48a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.105+0000 7f5c3e59c640 1 -- 172.21.15.12:0/3328401568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1aa486100 con 0x55c1aa478e80 2024-04-22T13:25:50.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.105+0000 7f5c3dd9b640 1 --2- 172.21.15.12:0/3328401568 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1aa4784d0 0x55c1aa48a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:25:50.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.105+0000 7f5c3e59c640 1 --2- 172.21.15.12:0/3328401568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1aa478e80 0x55c1aa4853c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f5c30004fa0 tx=0x7f5c30000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff540483ccd24ac server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:50.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.105+0000 7f5c25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:50.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.105+0000 7f5c25ffb640 1 -- 172.21.15.12:0/3328401568 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c30062020 con 0x55c1aa478e80 2024-04-22T13:25:50.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.105+0000 7f5c25ffb640 1 -- 172.21.15.12:0/3328401568 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f5c300035f0 con 0x55c1aa478e80 2024-04-22T13:25:50.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.105+0000 7f5c47651740 1 -- 172.21.15.12:0/3328401568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c3002c7e0 con 0x55c1aa478e80 2024-04-22T13:25:50.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.105+0000 7f5c25ffb640 1 -- 172.21.15.12:0/3328401568 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5c30045040 con 0x55c1aa478e80 2024-04-22T13:25:50.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.105+0000 7f5c47651740 10 client.16410 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:50.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.105+0000 7f5c47651740 1 -- 172.21.15.12:0/3328401568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c1aa4f7bd0 con 0x55c1aa478e80 2024-04-22T13:25:50.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.106+0000 7f5c25ffb640 1 -- 172.21.15.12:0/3328401568 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c3002c7e0 con 0x55c1aa478e80 2024-04-22T13:25:50.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.106+0000 7f5c25ffb640 1 -- 172.21.15.12:0/3328401568 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5c30041630 con 0x55c1aa478e80 2024-04-22T13:25:50.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.106+0000 7f5c47651740 10 client.16410 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:50.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.106+0000 7f5c47651740 10 client.16410 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:25:50.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.106+0000 7f5c47651740 20 client.16410 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:50.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.106+0000 7f5c47651740 1 --2- 172.21.15.12:0/3328401568 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55c1aa4f8150 0x55c1aa4fa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.106+0000 7f5c47651740 4 client.16410 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:25:50.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.106+0000 7f5c47651740 1 -- 172.21.15.12:0/3328401568 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c1aa3eac10 con 0x55c1aa4f8150 2024-04-22T13:25:50.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.106+0000 7f5c3dd9b640 1 --2- 172.21.15.12:0/3328401568 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55c1aa4f8150 0x55c1aa4fa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.107+0000 7f5c3dd9b640 1 --2- 172.21.15.12:0/3328401568 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55c1aa4f8150 0x55c1aa4fa570 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=70b1387ac3230065 server_cookie=d7c5361b49b52444 in_seq=0 out_seq=0 2024-04-22T13:25:50.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.107+0000 7f5c25ffb640 10 client.16410 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:25:50.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.107+0000 7f5c25ffb640 1 -- 172.21.15.12:0/3328401568 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c1aa3eac10 con 0x55c1aa4f8150 2024-04-22T13:25:50.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.107+0000 7f5c25ffb640 10 client.16410 handle_command_reply: tid=0 2024-04-22T13:25:50.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.107+0000 7f5c47651740 1 client.16410 shutdown 2024-04-22T13:25:50.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.107+0000 7f5c47651740 1 -- 172.21.15.12:0/3328401568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1aa478e80 msgr2=0x55c1aa4853c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.107+0000 7f5c47651740 1 --2- 172.21.15.12:0/3328401568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1aa478e80 0x55c1aa4853c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f5c30004fa0 tx=0x7f5c30000bc0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.107+0000 7f5c47651740 1 -- 172.21.15.12:0/3328401568 shutdown_connections 2024-04-22T13:25:50.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.107+0000 7f5c47651740 1 --2- 172.21.15.12:0/3328401568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1aa481a00 0x55c1aa485900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.107+0000 7f5c47651740 1 --2- 172.21.15.12:0/3328401568 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55c1aa4f8150 0x55c1aa4fa570 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.107+0000 7f5c47651740 1 --2- 172.21.15.12:0/3328401568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1aa478e80 0x55c1aa4853c0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.107+0000 7f5c47651740 1 --2- 172.21.15.12:0/3328401568 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1aa4784d0 0x55c1aa48a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.107+0000 7f5c25ffb640 0 client.16410 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:25:50.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.107+0000 7f5c47651740 1 -- 172.21.15.12:0/3328401568 >> 172.21.15.12:0/3328401568 conn(0x55c1aa472ba0 msgr2=0x55c1aa471c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:50.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.107+0000 7f5c47651740 1 -- 172.21.15.12:0/3328401568 shutdown_connections 2024-04-22T13:25:50.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.107+0000 7f5c47651740 1 -- 172.21.15.12:0/3328401568 wait complete. 2024-04-22T13:25:50.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.107+0000 7f5c47651740 20 client.16410 trim_cache size 0 max 16384 2024-04-22T13:25:50.110 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:25:50.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.108+0000 7f5c44a5c640 1 -- 172.21.15.12:0/2238194930 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5c2c0a3d50 msgr2=0x7f5c2c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.108+0000 7f5c44a5c640 1 --2- 172.21.15.12:0/2238194930 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5c2c0a3d50 0x7f5c2c0a6210 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5c380cdcb0 tx=0x7f5c3801e780 comp rx=0 tx=0).stop 2024-04-22T13:25:50.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.108+0000 7f5c44a5c640 1 -- 172.21.15.12:0/2238194930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c4010f140 msgr2=0x7f5c401a6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.108+0000 7f5c44a5c640 1 --2- 172.21.15.12:0/2238194930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c4010f140 0x7f5c401a6950 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f5c38009fd0 tx=0x7f5c3800b260 comp rx=0 tx=0).stop 2024-04-22T13:25:50.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.108+0000 7f5c44a5c640 1 -- 172.21.15.12:0/2238194930 shutdown_connections 2024-04-22T13:25:50.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.108+0000 7f5c44a5c640 1 --2- 172.21.15.12:0/2238194930 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5c2c0a3d50 0x7f5c2c0a6210 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.108+0000 7f5c44a5c640 1 --2- 172.21.15.12:0/2238194930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c4010f140 0x7f5c401a6950 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.108+0000 7f5c44a5c640 1 --2- 172.21.15.12:0/2238194930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c40106160 0x7f5c4010ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.108+0000 7f5c44a5c640 1 --2- 172.21.15.12:0/2238194930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c40101090 0x7f5c4010e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.108+0000 7f5c44a5c640 1 -- 172.21.15.12:0/2238194930 >> 172.21.15.12:0/2238194930 conn(0x7f5c40077f50 msgr2=0x7f5c400ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:50.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.108+0000 7f5c44a5c640 1 -- 172.21.15.12:0/2238194930 shutdown_connections 2024-04-22T13:25:50.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.108+0000 7f5c44a5c640 1 -- 172.21.15.12:0/2238194930 wait complete. 2024-04-22T13:25:50.115 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:25:50.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:49 smithi134 ceph-mon[31338]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-04-22T13:25:50.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:49 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/3603828510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:25:50.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.229+0000 7f5da7074640 1 -- 172.21.15.12:0/2275077732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5da0106160 msgr2=0x7f5da010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.229+0000 7f5da7074640 1 --2- 172.21.15.12:0/2275077732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5da0106160 0x7f5da010dd30 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009ca0 tx=0x7f5d9802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.229+0000 7f5da7074640 1 -- 172.21.15.12:0/2275077732 shutdown_connections 2024-04-22T13:25:50.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.229+0000 7f5da7074640 1 --2- 172.21.15.12:0/2275077732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5da0106160 0x7f5da010dd30 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.229+0000 7f5da7074640 1 --2- 172.21.15.12:0/2275077732 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5da0101a40 0x7f5da0105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.229+0000 7f5da7074640 1 --2- 172.21.15.12:0/2275077732 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5da0101090 0x7f5da0101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.229+0000 7f5da7074640 1 -- 172.21.15.12:0/2275077732 >> 172.21.15.12:0/2275077732 conn(0x7f5da0077f50 msgr2=0x7f5da0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:50.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.230+0000 7f5da7074640 1 -- 172.21.15.12:0/2275077732 shutdown_connections 2024-04-22T13:25:50.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.230+0000 7f5da7074640 1 -- 172.21.15.12:0/2275077732 wait complete. 2024-04-22T13:25:50.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.230+0000 7f5da7074640 1 Processor -- start 2024-04-22T13:25:50.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.230+0000 7f5da7074640 1 -- start start 2024-04-22T13:25:50.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.230+0000 7f5da7074640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5da0101090 0x7f5da01975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.230+0000 7f5da7074640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5da0106160 0x7f5da0197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.230+0000 7f5da7074640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5da019beb0 0x7f5da019c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.230+0000 7f5da7074640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5da01110b0 con 0x7f5da0106160 2024-04-22T13:25:50.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.230+0000 7f5da7074640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5da0110f30 con 0x7f5da0101090 2024-04-22T13:25:50.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.230+0000 7f5da7074640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5da0111230 con 0x7f5da019beb0 2024-04-22T13:25:50.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.230+0000 7f5d9ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5da0106160 0x7f5da0197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.230+0000 7f5d9ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5da0106160 0x7f5da0197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53170/0 (socket says 172.21.15.12:53170) 2024-04-22T13:25:50.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.231+0000 7f5d9ffff640 1 -- 172.21.15.12:0/1539406860 learned_addr learned my addr 172.21.15.12:0/1539406860 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:50.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.231+0000 7f5da4e10640 1 --2- 172.21.15.12:0/1539406860 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5da0101090 0x7f5da01975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.231+0000 7f5da5611640 1 --2- 172.21.15.12:0/1539406860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5da019beb0 0x7f5da019c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.231+0000 7f5d9ffff640 1 -- 172.21.15.12:0/1539406860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5da019beb0 msgr2=0x7f5da019c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.231+0000 7f5d9ffff640 1 --2- 172.21.15.12:0/1539406860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5da019beb0 0x7f5da019c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.231+0000 7f5d9ffff640 1 -- 172.21.15.12:0/1539406860 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5da0101090 msgr2=0x7f5da01975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.231+0000 7f5d9ffff640 1 --2- 172.21.15.12:0/1539406860 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5da0101090 0x7f5da01975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.231+0000 7f5d9ffff640 1 -- 172.21.15.12:0/1539406860 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da019c9a0 con 0x7f5da0106160 2024-04-22T13:25:50.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.231+0000 7f5da5611640 1 --2- 172.21.15.12:0/1539406860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5da019beb0 0x7f5da019c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:25:50.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.231+0000 7f5d9ffff640 1 --2- 172.21.15.12:0/1539406860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5da0106160 0x7f5da0197b30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00e9c0 tx=0x7f5d8c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:50.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.231+0000 7f5d9dffb640 1 -- 172.21.15.12:0/1539406860 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c00cd80 con 0x7f5da0106160 2024-04-22T13:25:50.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.231+0000 7f5d9dffb640 1 -- 172.21.15.12:0/1539406860 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d8c0054d0 con 0x7f5da0106160 2024-04-22T13:25:50.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.231+0000 7f5da7074640 1 -- 172.21.15.12:0/1539406860 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da019cc90 con 0x7f5da0106160 2024-04-22T13:25:50.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.231+0000 7f5d9dffb640 1 -- 172.21.15.12:0/1539406860 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c010430 con 0x7f5da0106160 2024-04-22T13:25:50.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.231+0000 7f5da7074640 1 -- 172.21.15.12:0/1539406860 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da01a4480 con 0x7f5da0106160 2024-04-22T13:25:50.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.233+0000 7f5d9dffb640 1 -- 172.21.15.12:0/1539406860 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5d8c004fd0 con 0x7f5da0106160 2024-04-22T13:25:50.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.234+0000 7f5d9dffb640 1 --2- 172.21.15.12:0/1539406860 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5d900a3d20 0x7f5d900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.234+0000 7f5da9c6e740 1 Processor -- start 2024-04-22T13:25:50.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.234+0000 7f5da9c6e740 1 -- start start 2024-04-22T13:25:50.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.234+0000 7f5d9dffb640 1 -- 172.21.15.12:0/1539406860 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f5d8c0d0c60 con 0x7f5da0106160 2024-04-22T13:25:50.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.234+0000 7f5da4e10640 1 --2- 172.21.15.12:0/1539406860 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5d900a3d20 0x7f5d900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.234+0000 7f5da9c6e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b54cca20f0 0x55b54cca24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.234+0000 7f5da9c6e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b54cca2aa0 0x55b54ccada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.234+0000 7f5da9c6e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b54ccadf70 0x55b54ccb0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.234+0000 7f5da9c6e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b54cc051f0 con 0x55b54cca20f0 2024-04-22T13:25:50.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.234+0000 7f5da9c6e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b54cba24b0 con 0x55b54cca2aa0 2024-04-22T13:25:50.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.234+0000 7f5da9c6e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b54cc011b0 con 0x55b54ccadf70 2024-04-22T13:25:50.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.234+0000 7f5da4e10640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b54cca20f0 0x55b54cca24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.235+0000 7f5da4e10640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b54cca20f0 0x55b54cca24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53176/0 (socket says 172.21.15.12:53176) 2024-04-22T13:25:50.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.235+0000 7f5da4e10640 1 -- 172.21.15.12:0/1866010040 learned_addr learned my addr 172.21.15.12:0/1866010040 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:50.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.235+0000 7f5d9ffff640 1 --2- 172.21.15.12:0/1866010040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b54cca2aa0 0x55b54ccada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.235+0000 7f5da5611640 1 --2- 172.21.15.12:0/1866010040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b54ccadf70 0x55b54ccb0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.235+0000 7f5da4e10640 1 --2- 172.21.15.12:0/1539406860 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5d900a3d20 0x7f5d900a61e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5d94013510 tx=0x7f5d94013910 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:50.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.235+0000 7f5da4e10640 1 -- 172.21.15.12:0/1866010040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b54ccadf70 msgr2=0x55b54ccb0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:50.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.235+0000 7f5da4e10640 1 --2- 172.21.15.12:0/1866010040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b54ccadf70 0x55b54ccb0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.235+0000 7f5da4e10640 1 -- 172.21.15.12:0/1866010040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b54cca2aa0 msgr2=0x55b54ccada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:50.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.235+0000 7f5da4e10640 1 --2- 172.21.15.12:0/1866010040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b54cca2aa0 0x55b54ccada30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.235+0000 7f5da4e10640 1 -- 172.21.15.12:0/1866010040 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b54cb941b0 con 0x55b54cca20f0 2024-04-22T13:25:50.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.235+0000 7f5d9ffff640 1 --2- 172.21.15.12:0/1866010040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b54cca2aa0 0x55b54ccada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:50.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.235+0000 7f5da4e10640 1 --2- 172.21.15.12:0/1866010040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b54cca20f0 0x55b54cca24d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400f7a0 tx=0x7f5d9401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d806ad505ca1b8d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:50.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.235+0000 7f5d8affd640 1 -- 172.21.15.12:0/1866010040 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94013030 con 0x55b54cca20f0 2024-04-22T13:25:50.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.235+0000 7f5d8affd640 1 -- 172.21.15.12:0/1866010040 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d940131d0 con 0x55b54cca20f0 2024-04-22T13:25:50.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.235+0000 7f5d8affd640 1 -- 172.21.15.12:0/1866010040 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94011970 con 0x55b54cca20f0 2024-04-22T13:25:50.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.236+0000 7f5da9c6e740 1 -- 172.21.15.12:0/1866010040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b54cca20f0 msgr2=0x55b54cca24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.236+0000 7f5da9c6e740 1 --2- 172.21.15.12:0/1866010040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b54cca20f0 0x55b54cca24d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400f7a0 tx=0x7f5d9401d000 comp rx=0 tx=0).stop 2024-04-22T13:25:50.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.236+0000 7f5da9c6e740 1 -- 172.21.15.12:0/1866010040 shutdown_connections 2024-04-22T13:25:50.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.236+0000 7f5da9c6e740 1 --2- 172.21.15.12:0/1866010040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b54ccadf70 0x55b54ccb0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.236+0000 7f5da9c6e740 1 --2- 172.21.15.12:0/1866010040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b54cca2aa0 0x55b54ccada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.236+0000 7f5da9c6e740 1 --2- 172.21.15.12:0/1866010040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b54cca20f0 0x55b54cca24d0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.236+0000 7f5da9c6e740 1 -- 172.21.15.12:0/1866010040 >> 172.21.15.12:0/1866010040 conn(0x55b54cc9cdb0 msgr2=0x55b54cc9d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:50.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.236+0000 7f5da9c6e740 1 -- 172.21.15.12:0/1866010040 shutdown_connections 2024-04-22T13:25:50.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.236+0000 7f5da9c6e740 1 -- 172.21.15.12:0/1866010040 wait complete. 2024-04-22T13:25:50.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.236+0000 7f5da9c6e740 1 -- start start 2024-04-22T13:25:50.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.236+0000 7f5da9c6e740 1 Processor -- start 2024-04-22T13:25:50.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.237+0000 7f5da9c6e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b54cca20f0 0x55b54ccad970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.237+0000 7f5da9c6e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b54cca2aa0 0x55b54ccaad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.237+0000 7f5da9c6e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b54ccadf70 0x55b54ccab310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.237+0000 7f5da9c6e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b54ccb0d00 con 0x55b54cca2aa0 2024-04-22T13:25:50.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.237+0000 7f5da9c6e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b54cbbe500 con 0x55b54ccadf70 2024-04-22T13:25:50.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.237+0000 7f5da9c6e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b54ccb0e80 con 0x55b54cca20f0 2024-04-22T13:25:50.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.237+0000 7f5da9c6e740 10 client.0 _ll_register_callbacks cb 0x55b54cb8cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:50.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.237+0000 7f5d9ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b54cca2aa0 0x55b54ccaad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.237+0000 7f5d9ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b54cca2aa0 0x55b54ccaad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53188/0 (socket says 172.21.15.12:53188) 2024-04-22T13:25:50.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.237+0000 7f5d9ffff640 1 -- 172.21.15.12:0/2061015032 learned_addr learned my addr 172.21.15.12:0/2061015032 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:50.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.237+0000 7f5da5611640 1 --2- 172.21.15.12:0/2061015032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b54ccadf70 0x55b54ccab310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.237+0000 7f5da4e10640 1 --2- 172.21.15.12:0/2061015032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b54cca20f0 0x55b54ccad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.237+0000 7f5d9ffff640 1 -- 172.21.15.12:0/2061015032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b54cca20f0 msgr2=0x55b54ccad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:50.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.237+0000 7f5d9ffff640 1 --2- 172.21.15.12:0/2061015032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b54cca20f0 0x55b54ccad970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.237+0000 7f5d9ffff640 1 -- 172.21.15.12:0/2061015032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b54ccadf70 msgr2=0x55b54ccab310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:50.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.237+0000 7f5d9ffff640 1 --2- 172.21.15.12:0/2061015032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b54ccadf70 0x55b54ccab310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.237+0000 7f5d9ffff640 1 -- 172.21.15.12:0/2061015032 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b54c9173f0 con 0x55b54cca2aa0 2024-04-22T13:25:50.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.238+0000 7f5d9ffff640 1 --2- 172.21.15.12:0/2061015032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b54cca2aa0 0x55b54ccaad00 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c09c950 tx=0x7f5d8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f17be11e36357 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:50.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.238+0000 7f5d7ffff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:50.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.238+0000 7f5d7ffff640 1 -- 172.21.15.12:0/2061015032 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c0b5070 con 0x55b54cca2aa0 2024-04-22T13:25:50.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.238+0000 7f5d7ffff640 1 -- 172.21.15.12:0/2061015032 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f5d8c09ceb0 con 0x55b54cca2aa0 2024-04-22T13:25:50.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.238+0000 7f5da9c6e740 1 -- 172.21.15.12:0/2061015032 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d940131d0 con 0x55b54cca2aa0 2024-04-22T13:25:50.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.238+0000 7f5d7ffff640 1 -- 172.21.15.12:0/2061015032 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5d8c0a1000 con 0x55b54cca2aa0 2024-04-22T13:25:50.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.238+0000 7f5da9c6e740 10 client.16422 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:50.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.238+0000 7f5da9c6e740 1 -- 172.21.15.12:0/2061015032 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b54c9e17d0 con 0x55b54cca2aa0 2024-04-22T13:25:50.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.238+0000 7f5d7ffff640 1 -- 172.21.15.12:0/2061015032 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d940131d0 con 0x55b54cca2aa0 2024-04-22T13:25:50.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.238+0000 7f5d7ffff640 1 -- 172.21.15.12:0/2061015032 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5d8c063ce0 con 0x55b54cca2aa0 2024-04-22T13:25:50.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.238+0000 7f5da9c6e740 10 client.16422 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:50.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.238+0000 7f5da9c6e740 10 client.16422 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:25:50.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.238+0000 7f5da9c6e740 20 client.16422 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:50.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.238+0000 7f5da9c6e740 1 --2- 172.21.15.12:0/2061015032 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55b54ccbdf40 0x55b54ccc0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.238+0000 7f5da9c6e740 4 client.16422 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:25:50.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.238+0000 7f5da9c6e740 1 -- 172.21.15.12:0/2061015032 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b54cba4930 con 0x55b54ccbdf40 2024-04-22T13:25:50.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.239+0000 7f5da4e10640 1 --2- 172.21.15.12:0/2061015032 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55b54ccbdf40 0x55b54ccc0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.239+0000 7f5d7ffff640 10 client.16422 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:25:50.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.239+0000 7f5da4e10640 1 --2- 172.21.15.12:0/2061015032 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55b54ccbdf40 0x55b54ccc0370 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ebda85937f88db33 server_cookie=8ff748b127aff561 in_seq=0 out_seq=0 2024-04-22T13:25:50.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.240+0000 7f5d7ffff640 1 -- 172.21.15.12:0/2061015032 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b54cba4930 con 0x55b54ccbdf40 2024-04-22T13:25:50.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.240+0000 7f5d7ffff640 10 client.16422 handle_command_reply: tid=0 2024-04-22T13:25:50.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.240+0000 7f5da9c6e740 1 client.16422 shutdown 2024-04-22T13:25:50.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.240+0000 7f5da9c6e740 1 -- 172.21.15.12:0/2061015032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b54cca2aa0 msgr2=0x55b54ccaad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.240+0000 7f5da9c6e740 1 --2- 172.21.15.12:0/2061015032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b54cca2aa0 0x55b54ccaad00 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c09c950 tx=0x7f5d8c0046c0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.240+0000 7f5da9c6e740 1 -- 172.21.15.12:0/2061015032 shutdown_connections 2024-04-22T13:25:50.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.240+0000 7f5da9c6e740 1 --2- 172.21.15.12:0/2061015032 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55b54ccbdf40 0x55b54ccc0370 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.240+0000 7f5da9c6e740 1 --2- 172.21.15.12:0/2061015032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b54ccadf70 0x55b54ccab310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.240+0000 7f5d7ffff640 0 client.16422 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:25:50.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.240+0000 7f5da9c6e740 1 --2- 172.21.15.12:0/2061015032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b54cca2aa0 0x55b54ccaad00 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.240+0000 7f5da9c6e740 1 --2- 172.21.15.12:0/2061015032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b54cca20f0 0x55b54ccad970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.240+0000 7f5da9c6e740 1 -- 172.21.15.12:0/2061015032 >> 172.21.15.12:0/2061015032 conn(0x55b54cc9cdb0 msgr2=0x55b54cc9eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:50.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.240+0000 7f5da9c6e740 1 -- 172.21.15.12:0/2061015032 shutdown_connections 2024-04-22T13:25:50.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.240+0000 7f5da9c6e740 1 -- 172.21.15.12:0/2061015032 wait complete. 2024-04-22T13:25:50.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.240+0000 7f5da9c6e740 20 client.16422 trim_cache size 0 max 16384 2024-04-22T13:25:50.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.254+0000 7f5da9c6e740 1 Processor -- start 2024-04-22T13:25:50.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.254+0000 7f5da9c6e740 1 -- start start 2024-04-22T13:25:50.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.254+0000 7f5da9c6e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b54cca20f0 0x55b54ccb9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.254+0000 7f5da9c6e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b54cca2aa0 0x55b54cc99a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.254+0000 7f5da9c6e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b54ccadf70 0x55b54ccad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.254+0000 7f5da9c6e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5da01ab890 con 0x55b54cca20f0 2024-04-22T13:25:50.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.254+0000 7f5da9c6e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5da01ad3d0 con 0x55b54ccadf70 2024-04-22T13:25:50.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.254+0000 7f5da9c6e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5da01abf60 con 0x55b54cca2aa0 2024-04-22T13:25:50.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.254+0000 7f5da4e10640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b54cca20f0 0x55b54ccb9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.254+0000 7f5da4e10640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b54cca20f0 0x55b54ccb9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53198/0 (socket says 172.21.15.12:53198) 2024-04-22T13:25:50.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.254+0000 7f5da4e10640 1 -- 172.21.15.12:0/3703748323 learned_addr learned my addr 172.21.15.12:0/3703748323 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:50.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.255+0000 7f5d9ffff640 1 --2- 172.21.15.12:0/3703748323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b54cca2aa0 0x55b54cc99a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.255+0000 7f5da5611640 1 --2- 172.21.15.12:0/3703748323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b54ccadf70 0x55b54ccad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.255+0000 7f5da4e10640 1 -- 172.21.15.12:0/3703748323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b54cca2aa0 msgr2=0x55b54cc99a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:50.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.255+0000 7f5da4e10640 1 --2- 172.21.15.12:0/3703748323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b54cca2aa0 0x55b54cc99a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.255+0000 7f5da4e10640 1 -- 172.21.15.12:0/3703748323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b54ccadf70 msgr2=0x55b54ccad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:50.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.255+0000 7f5da4e10640 1 --2- 172.21.15.12:0/3703748323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b54ccadf70 0x55b54ccad430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.255+0000 7f5da4e10640 1 -- 172.21.15.12:0/3703748323 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b54cc95070 con 0x55b54cca20f0 2024-04-22T13:25:50.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.255+0000 7f5d9ffff640 1 --2- 172.21.15.12:0/3703748323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b54cca2aa0 0x55b54cc99a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:50.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.255+0000 7f5da4e10640 1 --2- 172.21.15.12:0/3703748323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b54cca20f0 0x55b54ccb9c00 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f5d940058c0 tx=0x7f5d94005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b71819c045c5bd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:50.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.255+0000 7f5d8a7fc640 1 -- 172.21.15.12:0/3703748323 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94034070 con 0x55b54cca20f0 2024-04-22T13:25:50.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.255+0000 7f5d8a7fc640 1 -- 172.21.15.12:0/3703748323 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d940130c0 con 0x55b54cca20f0 2024-04-22T13:25:50.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.255+0000 7f5d8a7fc640 1 -- 172.21.15.12:0/3703748323 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9402e7f0 con 0x55b54cca20f0 2024-04-22T13:25:50.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.256+0000 7f5da9c6e740 1 -- 172.21.15.12:0/3703748323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b54cca20f0 msgr2=0x55b54ccb9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.256+0000 7f5da9c6e740 1 --2- 172.21.15.12:0/3703748323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b54cca20f0 0x55b54ccb9c00 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f5d940058c0 tx=0x7f5d94005850 comp rx=0 tx=0).stop 2024-04-22T13:25:50.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.256+0000 7f5da9c6e740 1 -- 172.21.15.12:0/3703748323 shutdown_connections 2024-04-22T13:25:50.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.256+0000 7f5da9c6e740 1 --2- 172.21.15.12:0/3703748323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b54ccadf70 0x55b54ccad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.256+0000 7f5da9c6e740 1 --2- 172.21.15.12:0/3703748323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b54cca2aa0 0x55b54cc99a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.256+0000 7f5da9c6e740 1 --2- 172.21.15.12:0/3703748323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b54cca20f0 0x55b54ccb9c00 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.256+0000 7f5da9c6e740 1 -- 172.21.15.12:0/3703748323 >> 172.21.15.12:0/3703748323 conn(0x55b54cc9cdb0 msgr2=0x55b54cc99370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:50.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.256+0000 7f5da9c6e740 1 -- 172.21.15.12:0/3703748323 shutdown_connections 2024-04-22T13:25:50.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.256+0000 7f5da9c6e740 1 -- 172.21.15.12:0/3703748323 wait complete. 2024-04-22T13:25:50.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.256+0000 7f5da9c6e740 1 -- start start 2024-04-22T13:25:50.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.257+0000 7f5da9c6e740 1 Processor -- start 2024-04-22T13:25:50.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.257+0000 7f5da9c6e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b54cca20f0 0x55b54ccb2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.257+0000 7f5da9c6e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b54cca2aa0 0x55b54ccb27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.257+0000 7f5da9c6e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b54ccadf70 0x55b54cd23480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.257+0000 7f5da9c6e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5da01acd00 con 0x55b54ccadf70 2024-04-22T13:25:50.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.257+0000 7f5da9c6e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5da01ac630 con 0x55b54cca2aa0 2024-04-22T13:25:50.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.257+0000 7f5da9c6e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5da01a9870 con 0x55b54cca20f0 2024-04-22T13:25:50.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.257+0000 7f5da9c6e740 10 client.0 _ll_register_callbacks cb 0x7f5d8c09c790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:50.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.257+0000 7f5da5611640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b54ccadf70 0x55b54cd23480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.257+0000 7f5da5611640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b54ccadf70 0x55b54cd23480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53214/0 (socket says 172.21.15.12:53214) 2024-04-22T13:25:50.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.257+0000 7f5da5611640 1 -- 172.21.15.12:0/2901796912 learned_addr learned my addr 172.21.15.12:0/2901796912 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:50.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.257+0000 7f5da4e10640 1 --2- 172.21.15.12:0/2901796912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b54cca20f0 0x55b54ccb2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.257+0000 7f5d9ffff640 1 --2- 172.21.15.12:0/2901796912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b54cca2aa0 0x55b54ccb27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.258+0000 7f5da5611640 1 -- 172.21.15.12:0/2901796912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b54cca20f0 msgr2=0x55b54ccb2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:50.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.258+0000 7f5da5611640 1 --2- 172.21.15.12:0/2901796912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b54cca20f0 0x55b54ccb2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.258+0000 7f5da5611640 1 -- 172.21.15.12:0/2901796912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b54cca2aa0 msgr2=0x55b54ccb27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:50.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.258+0000 7f5da5611640 1 --2- 172.21.15.12:0/2901796912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b54cca2aa0 0x55b54ccb27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.258+0000 7f5da5611640 1 -- 172.21.15.12:0/2901796912 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b54cd239c0 con 0x55b54ccadf70 2024-04-22T13:25:50.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.258+0000 7f5da4e10640 1 --2- 172.21.15.12:0/2901796912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b54cca20f0 0x55b54ccb2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:50.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.258+0000 7f5da5611640 1 --2- 172.21.15.12:0/2901796912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b54ccadf70 0x55b54cd23480 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009ca0 tx=0x7f5d98009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec530ebfd092480c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:50.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.258+0000 7f5d7ffff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:50.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.258+0000 7f5d7ffff640 1 -- 172.21.15.12:0/2901796912 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9804d060 con 0x55b54ccadf70 2024-04-22T13:25:50.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.258+0000 7f5d7ffff640 1 -- 172.21.15.12:0/2901796912 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f5d98032790 con 0x55b54ccadf70 2024-04-22T13:25:50.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.258+0000 7f5da9c6e740 1 -- 172.21.15.12:0/2901796912 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d940130c0 con 0x55b54ccadf70 2024-04-22T13:25:50.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.258+0000 7f5d7ffff640 1 -- 172.21.15.12:0/2901796912 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5d98039040 con 0x55b54ccadf70 2024-04-22T13:25:50.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.258+0000 7f5da9c6e740 10 client.16428 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:50.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.258+0000 7f5da9c6e740 1 -- 172.21.15.12:0/2901796912 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b54cd24cc0 con 0x55b54ccadf70 2024-04-22T13:25:50.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.258+0000 7f5d7ffff640 1 -- 172.21.15.12:0/2901796912 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d940130c0 con 0x55b54ccadf70 2024-04-22T13:25:50.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.258+0000 7f5d7ffff640 1 -- 172.21.15.12:0/2901796912 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5d980045d0 con 0x55b54ccadf70 2024-04-22T13:25:50.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.259+0000 7f5da9c6e740 10 client.16428 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:50.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.259+0000 7f5da9c6e740 10 client.16428 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:25:50.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.259+0000 7f5da9c6e740 20 client.16428 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:50.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.259+0000 7f5da9c6e740 1 --2- 172.21.15.12:0/2901796912 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55b54cd25460 0x55b54cd27880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.259+0000 7f5da9c6e740 4 client.16428 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:25:50.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.259+0000 7f5da9c6e740 1 -- 172.21.15.12:0/2901796912 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b54cc12c10 con 0x55b54cd25460 2024-04-22T13:25:50.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.259+0000 7f5da4e10640 1 --2- 172.21.15.12:0/2901796912 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55b54cd25460 0x55b54cd27880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.259+0000 7f5d7ffff640 10 client.16428 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:25:50.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.259+0000 7f5da4e10640 1 --2- 172.21.15.12:0/2901796912 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55b54cd25460 0x55b54cd27880 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=69db8640ef29c69e server_cookie=c34573747c2b55f7 in_seq=0 out_seq=0 2024-04-22T13:25:50.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.259+0000 7f5d7ffff640 1 -- 172.21.15.12:0/2901796912 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b54cc12c10 con 0x55b54cd25460 2024-04-22T13:25:50.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.259+0000 7f5d7ffff640 10 client.16428 handle_command_reply: tid=0 2024-04-22T13:25:50.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.259+0000 7f5da9c6e740 1 client.16428 shutdown 2024-04-22T13:25:50.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.260+0000 7f5da9c6e740 1 -- 172.21.15.12:0/2901796912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b54ccadf70 msgr2=0x55b54cd23480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.260+0000 7f5da9c6e740 1 --2- 172.21.15.12:0/2901796912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b54ccadf70 0x55b54cd23480 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009ca0 tx=0x7f5d98009960 comp rx=0 tx=0).stop 2024-04-22T13:25:50.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.260+0000 7f5da9c6e740 1 -- 172.21.15.12:0/2901796912 shutdown_connections 2024-04-22T13:25:50.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.260+0000 7f5da9c6e740 1 --2- 172.21.15.12:0/2901796912 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55b54cd25460 0x55b54cd27880 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.260+0000 7f5da9c6e740 1 --2- 172.21.15.12:0/2901796912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b54ccadf70 0x55b54cd23480 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.260+0000 7f5d7ffff640 0 client.16428 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:25:50.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.260+0000 7f5da9c6e740 1 --2- 172.21.15.12:0/2901796912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b54cca2aa0 0x55b54ccb27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.260+0000 7f5da9c6e740 1 --2- 172.21.15.12:0/2901796912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b54cca20f0 0x55b54ccb2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.260+0000 7f5da9c6e740 1 -- 172.21.15.12:0/2901796912 >> 172.21.15.12:0/2901796912 conn(0x55b54cc9cdb0 msgr2=0x55b54cc9e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:50.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.260+0000 7f5da9c6e740 1 -- 172.21.15.12:0/2901796912 shutdown_connections 2024-04-22T13:25:50.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.260+0000 7f5da9c6e740 1 -- 172.21.15.12:0/2901796912 wait complete. 2024-04-22T13:25:50.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.260+0000 7f5da9c6e740 20 client.16428 trim_cache size 0 max 16384 2024-04-22T13:25:50.262 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:25:50.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.260+0000 7f5da7074640 1 -- 172.21.15.12:0/1539406860 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5d900a3d20 msgr2=0x7f5d900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.260+0000 7f5da7074640 1 --2- 172.21.15.12:0/1539406860 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5d900a3d20 0x7f5d900a61e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5d94013510 tx=0x7f5d94013910 comp rx=0 tx=0).stop 2024-04-22T13:25:50.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.261+0000 7f5da7074640 1 -- 172.21.15.12:0/1539406860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5da0106160 msgr2=0x7f5da0197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.261+0000 7f5da7074640 1 --2- 172.21.15.12:0/1539406860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5da0106160 0x7f5da0197b30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00e9c0 tx=0x7f5d8c00ee80 comp rx=0 tx=0).stop 2024-04-22T13:25:50.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.261+0000 7f5da7074640 1 -- 172.21.15.12:0/1539406860 shutdown_connections 2024-04-22T13:25:50.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.261+0000 7f5da7074640 1 --2- 172.21.15.12:0/1539406860 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5d900a3d20 0x7f5d900a61e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.261+0000 7f5da7074640 1 --2- 172.21.15.12:0/1539406860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5da019beb0 0x7f5da019c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.261+0000 7f5da7074640 1 --2- 172.21.15.12:0/1539406860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5da0106160 0x7f5da0197b30 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.261+0000 7f5da7074640 1 --2- 172.21.15.12:0/1539406860 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5da0101090 0x7f5da01975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.261+0000 7f5da7074640 1 -- 172.21.15.12:0/1539406860 >> 172.21.15.12:0/1539406860 conn(0x7f5da0077f50 msgr2=0x7f5da00ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:50.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.261+0000 7f5da7074640 1 -- 172.21.15.12:0/1539406860 shutdown_connections 2024-04-22T13:25:50.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.261+0000 7f5da7074640 1 -- 172.21.15.12:0/1539406860 wait complete. 2024-04-22T13:25:50.268 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:25:50.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.382+0000 7f081590e640 1 -- 172.21.15.12:0/1185747708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0810102b50 msgr2=0x7f0810106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.382+0000 7f081590e640 1 --2- 172.21.15.12:0/1185747708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0810102b50 0x7f0810106b90 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f07f8009900 tx=0x7f07f802bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:50.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.382+0000 7f081590e640 1 -- 172.21.15.12:0/1185747708 shutdown_connections 2024-04-22T13:25:50.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.382+0000 7f081590e640 1 --2- 172.21.15.12:0/1185747708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0810107270 0x7f081010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.382+0000 7f081590e640 1 --2- 172.21.15.12:0/1185747708 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0810102b50 0x7f0810106b90 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.382+0000 7f081590e640 1 --2- 172.21.15.12:0/1185747708 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08101021a0 0x7f0810102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.382+0000 7f081590e640 1 -- 172.21.15.12:0/1185747708 >> 172.21.15.12:0/1185747708 conn(0x7f0810079350 msgr2=0x7f0810079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:50.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.382+0000 7f081590e640 1 -- 172.21.15.12:0/1185747708 shutdown_connections 2024-04-22T13:25:50.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.382+0000 7f081590e640 1 -- 172.21.15.12:0/1185747708 wait complete. 2024-04-22T13:25:50.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.382+0000 7f081590e640 1 Processor -- start 2024-04-22T13:25:50.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.382+0000 7f081590e640 1 -- start start 2024-04-22T13:25:50.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.383+0000 7f081590e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08101021a0 0x7f081019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.383+0000 7f081590e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0810107270 0x7f081019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.383+0000 7f081590e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0810198590 0x7f0810198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.383+0000 7f081590e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0810112190 con 0x7f08101021a0 2024-04-22T13:25:50.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.383+0000 7f081590e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0810112010 con 0x7f0810107270 2024-04-22T13:25:50.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.383+0000 7f081590e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0810112310 con 0x7f0810198590 2024-04-22T13:25:50.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.383+0000 7f080effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08101021a0 0x7f081019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.383+0000 7f080effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08101021a0 0x7f081019e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53220/0 (socket says 172.21.15.12:53220) 2024-04-22T13:25:50.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.383+0000 7f080effd640 1 -- 172.21.15.12:0/884183970 learned_addr learned my addr 172.21.15.12:0/884183970 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:50.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.383+0000 7f080f7fe640 1 --2- 172.21.15.12:0/884183970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0810198590 0x7f0810198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.383+0000 7f080e7fc640 1 --2- 172.21.15.12:0/884183970 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0810107270 0x7f081019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.383+0000 7f080effd640 1 -- 172.21.15.12:0/884183970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0810198590 msgr2=0x7f0810198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.383+0000 7f080effd640 1 --2- 172.21.15.12:0/884183970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0810198590 0x7f0810198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.383+0000 7f080effd640 1 -- 172.21.15.12:0/884183970 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0810107270 msgr2=0x7f081019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.383+0000 7f080effd640 1 --2- 172.21.15.12:0/884183970 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0810107270 0x7f081019e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.383+0000 7f080effd640 1 -- 172.21.15.12:0/884183970 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08101990b0 con 0x7f08101021a0 2024-04-22T13:25:50.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.383+0000 7f080effd640 1 --2- 172.21.15.12:0/884183970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08101021a0 0x7f081019e3e0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f080000db80 tx=0x7f080000b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:50.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.384+0000 7f081490c640 1 -- 172.21.15.12:0/884183970 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0800017070 con 0x7f08101021a0 2024-04-22T13:25:50.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.384+0000 7f081590e640 1 -- 172.21.15.12:0/884183970 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08101993a0 con 0x7f08101021a0 2024-04-22T13:25:50.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.384+0000 7f081490c640 1 -- 172.21.15.12:0/884183970 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0800003b70 con 0x7f08101021a0 2024-04-22T13:25:50.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.384+0000 7f081490c640 1 -- 172.21.15.12:0/884183970 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08000106c0 con 0x7f08101021a0 2024-04-22T13:25:50.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.384+0000 7f081590e640 1 -- 172.21.15.12:0/884183970 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08101a5640 con 0x7f08101021a0 2024-04-22T13:25:50.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.386+0000 7f081490c640 1 -- 172.21.15.12:0/884183970 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0800004020 con 0x7f08101021a0 2024-04-22T13:25:50.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.386+0000 7f081490c640 1 --2- 172.21.15.12:0/884183970 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f07fc0a3d20 0x7f07fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.387+0000 7f081490c640 1 -- 172.21.15.12:0/884183970 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f08000d0840 con 0x7f08101021a0 2024-04-22T13:25:50.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.387+0000 7f080e7fc640 1 --2- 172.21.15.12:0/884183970 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f07fc0a3d20 0x7f07fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.387+0000 7f080e7fc640 1 --2- 172.21.15.12:0/884183970 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f07fc0a3d20 0x7f07fc0a61e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f07f8006010 tx=0x7f07f802c690 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:50.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.388+0000 7f08185d9740 1 Processor -- start 2024-04-22T13:25:50.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.388+0000 7f08185d9740 1 -- start start 2024-04-22T13:25:50.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.388+0000 7f08185d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de68d7c4d0 0x55de68d7c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.388+0000 7f08185d9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de68d7ce80 0x55de68d85430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.388+0000 7f08185d9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de68d85a00 0x55de68d87e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.388+0000 7f08185d9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55de68ce11f0 con 0x55de68d7ce80 2024-04-22T13:25:50.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.388+0000 7f08185d9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55de68c7e4b0 con 0x55de68d85a00 2024-04-22T13:25:50.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.388+0000 7f08185d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55de68cdd1b0 con 0x55de68d7c4d0 2024-04-22T13:25:50.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.388+0000 7f080e7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de68d7ce80 0x55de68d85430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.388+0000 7f080e7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de68d7ce80 0x55de68d85430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53222/0 (socket says 172.21.15.12:53222) 2024-04-22T13:25:50.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.388+0000 7f080e7fc640 1 -- 172.21.15.12:0/1308823335 learned_addr learned my addr 172.21.15.12:0/1308823335 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:50.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.388+0000 7f080effd640 1 --2- 172.21.15.12:0/1308823335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de68d7c4d0 0x55de68d7c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.388+0000 7f080f7fe640 1 --2- 172.21.15.12:0/1308823335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de68d85a00 0x55de68d87e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.389+0000 7f080e7fc640 1 -- 172.21.15.12:0/1308823335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de68d7c4d0 msgr2=0x55de68d7c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:50.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.389+0000 7f080e7fc640 1 --2- 172.21.15.12:0/1308823335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de68d7c4d0 0x55de68d7c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.389+0000 7f080e7fc640 1 -- 172.21.15.12:0/1308823335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de68d85a00 msgr2=0x55de68d87e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:50.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.389+0000 7f080e7fc640 1 --2- 172.21.15.12:0/1308823335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de68d85a00 0x55de68d87e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.389+0000 7f080e7fc640 1 -- 172.21.15.12:0/1308823335 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de68c701b0 con 0x55de68d7ce80 2024-04-22T13:25:50.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.389+0000 7f080effd640 1 --2- 172.21.15.12:0/1308823335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de68d7c4d0 0x55de68d7c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:50.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.389+0000 7f080e7fc640 1 --2- 172.21.15.12:0/1308823335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de68d7ce80 0x55de68d85430 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f07f8009900 tx=0x7f07f803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86fc53d023d60021 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:50.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.389+0000 7f0805ffb640 1 -- 172.21.15.12:0/1308823335 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f802cc20 con 0x55de68d7ce80 2024-04-22T13:25:50.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.389+0000 7f0805ffb640 1 -- 172.21.15.12:0/1308823335 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07f804f070 con 0x55de68d7ce80 2024-04-22T13:25:50.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.389+0000 7f0805ffb640 1 -- 172.21.15.12:0/1308823335 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f80035f0 con 0x55de68d7ce80 2024-04-22T13:25:50.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.390+0000 7f08185d9740 1 -- 172.21.15.12:0/1308823335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de68d7ce80 msgr2=0x55de68d85430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.390+0000 7f08185d9740 1 --2- 172.21.15.12:0/1308823335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de68d7ce80 0x55de68d85430 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f07f8009900 tx=0x7f07f803d000 comp rx=0 tx=0).stop 2024-04-22T13:25:50.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.390+0000 7f08185d9740 1 -- 172.21.15.12:0/1308823335 shutdown_connections 2024-04-22T13:25:50.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.390+0000 7f08185d9740 1 --2- 172.21.15.12:0/1308823335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de68d85a00 0x55de68d87e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.390+0000 7f08185d9740 1 --2- 172.21.15.12:0/1308823335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de68d7ce80 0x55de68d85430 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.390+0000 7f08185d9740 1 --2- 172.21.15.12:0/1308823335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de68d7c4d0 0x55de68d7c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.390+0000 7f08185d9740 1 -- 172.21.15.12:0/1308823335 >> 172.21.15.12:0/1308823335 conn(0x55de68d76ba0 msgr2=0x55de68d76f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:50.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.390+0000 7f08185d9740 1 -- 172.21.15.12:0/1308823335 shutdown_connections 2024-04-22T13:25:50.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.390+0000 7f08185d9740 1 -- 172.21.15.12:0/1308823335 wait complete. 2024-04-22T13:25:50.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.390+0000 7f08185d9740 1 -- start start 2024-04-22T13:25:50.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.390+0000 7f08185d9740 1 Processor -- start 2024-04-22T13:25:50.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.391+0000 7f08185d9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de68d7c4d0 0x55de68d85330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.391+0000 7f08185d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de68d7ce80 0x55de68d826c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.391+0000 7f08185d9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de68d85a00 0x55de68d82cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.391+0000 7f08185d9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55de68d88910 con 0x55de68d7c4d0 2024-04-22T13:25:50.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.391+0000 7f08185d9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55de68c9a500 con 0x55de68d85a00 2024-04-22T13:25:50.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.391+0000 7f08185d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55de68d88a90 con 0x55de68d7ce80 2024-04-22T13:25:50.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.391+0000 7f08185d9740 10 client.0 _ll_register_callbacks cb 0x55de68c68b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:50.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.391+0000 7f080effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de68d7c4d0 0x55de68d85330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.391+0000 7f080effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de68d7c4d0 0x55de68d85330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53234/0 (socket says 172.21.15.12:53234) 2024-04-22T13:25:50.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.391+0000 7f080effd640 1 -- 172.21.15.12:0/489287538 learned_addr learned my addr 172.21.15.12:0/489287538 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:50.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.391+0000 7f080f7fe640 1 --2- 172.21.15.12:0/489287538 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de68d85a00 0x55de68d82cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.391+0000 7f080e7fc640 1 --2- 172.21.15.12:0/489287538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de68d7ce80 0x55de68d826c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.391+0000 7f080effd640 1 -- 172.21.15.12:0/489287538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de68d7ce80 msgr2=0x55de68d826c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:50.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.391+0000 7f080effd640 1 --2- 172.21.15.12:0/489287538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de68d7ce80 0x55de68d826c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.391+0000 7f080effd640 1 -- 172.21.15.12:0/489287538 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de68d85a00 msgr2=0x55de68d82cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:50.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.391+0000 7f080effd640 1 --2- 172.21.15.12:0/489287538 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de68d85a00 0x55de68d82cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.391+0000 7f080effd640 1 -- 172.21.15.12:0/489287538 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de689f33f0 con 0x55de68d7c4d0 2024-04-22T13:25:50.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.391+0000 7f080f7fe640 1 --2- 172.21.15.12:0/489287538 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de68d85a00 0x55de68d82cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:50.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.392+0000 7f080effd640 1 --2- 172.21.15.12:0/489287538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de68d7c4d0 0x55de68d85330 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f080009c4d0 tx=0x7f080009cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e32a2d4642f800 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:50.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.392+0000 7f07eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:50.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.392+0000 7f07eeffd640 1 -- 172.21.15.12:0/489287538 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08000a5440 con 0x55de68d7c4d0 2024-04-22T13:25:50.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.392+0000 7f07eeffd640 1 -- 172.21.15.12:0/489287538 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f0800087070 con 0x55de68d7c4d0 2024-04-22T13:25:50.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.392+0000 7f08185d9740 1 -- 172.21.15.12:0/489287538 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07f804f070 con 0x55de68d7c4d0 2024-04-22T13:25:50.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.392+0000 7f07eeffd640 1 -- 172.21.15.12:0/489287538 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f08000ce200 con 0x55de68d7c4d0 2024-04-22T13:25:50.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.392+0000 7f08185d9740 10 client.16440 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:50.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.392+0000 7f08185d9740 1 -- 172.21.15.12:0/489287538 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55de68abd7d0 con 0x55de68d7c4d0 2024-04-22T13:25:50.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.392+0000 7f07eeffd640 1 -- 172.21.15.12:0/489287538 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07f804f070 con 0x55de68d7c4d0 2024-04-22T13:25:50.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.392+0000 7f07eeffd640 1 -- 172.21.15.12:0/489287538 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0800023060 con 0x55de68d7c4d0 2024-04-22T13:25:50.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.392+0000 7f08185d9740 10 client.16440 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:50.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.392+0000 7f08185d9740 10 client.16440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:25:50.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.392+0000 7f08185d9740 20 client.16440 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:50.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.392+0000 7f08185d9740 1 --2- 172.21.15.12:0/489287538 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55de68d95b80 0x55de68d97fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.392+0000 7f08185d9740 4 client.16440 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:25:50.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.392+0000 7f08185d9740 1 -- 172.21.15.12:0/489287538 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de68c80930 con 0x55de68d95b80 2024-04-22T13:25:50.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.393+0000 7f080e7fc640 1 --2- 172.21.15.12:0/489287538 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55de68d95b80 0x55de68d97fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.393+0000 7f07eeffd640 10 client.16440 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:50.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.393+0000 7f080e7fc640 1 --2- 172.21.15.12:0/489287538 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55de68d95b80 0x55de68d97fb0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb070518c7a66353 server_cookie=43cb21530ac66721 in_seq=0 out_seq=0 2024-04-22T13:25:50.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.394+0000 7f07eeffd640 1 -- 172.21.15.12:0/489287538 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55de68c80930 con 0x55de68d95b80 2024-04-22T13:25:50.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.394+0000 7f07eeffd640 10 client.16440 handle_command_reply: tid=0 2024-04-22T13:25:50.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.394+0000 7f08185d9740 1 client.16440 shutdown 2024-04-22T13:25:50.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.394+0000 7f08185d9740 1 -- 172.21.15.12:0/489287538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de68d7c4d0 msgr2=0x55de68d85330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.394+0000 7f08185d9740 1 --2- 172.21.15.12:0/489287538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de68d7c4d0 0x55de68d85330 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f080009c4d0 tx=0x7f080009cf90 comp rx=0 tx=0).stop 2024-04-22T13:25:50.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.394+0000 7f08185d9740 1 -- 172.21.15.12:0/489287538 shutdown_connections 2024-04-22T13:25:50.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.394+0000 7f08185d9740 1 --2- 172.21.15.12:0/489287538 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55de68d95b80 0x55de68d97fb0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.394+0000 7f08185d9740 1 --2- 172.21.15.12:0/489287538 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de68d85a00 0x55de68d82cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.394+0000 7f08185d9740 1 --2- 172.21.15.12:0/489287538 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de68d7ce80 0x55de68d826c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.394+0000 7f08185d9740 1 --2- 172.21.15.12:0/489287538 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de68d7c4d0 0x55de68d85330 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.394+0000 7f08185d9740 1 -- 172.21.15.12:0/489287538 >> 172.21.15.12:0/489287538 conn(0x55de68d76ba0 msgr2=0x55de68d788f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:50.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.394+0000 7f07eeffd640 0 client.16440 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:50.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.394+0000 7f08185d9740 1 -- 172.21.15.12:0/489287538 shutdown_connections 2024-04-22T13:25:50.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.394+0000 7f08185d9740 1 -- 172.21.15.12:0/489287538 wait complete. 2024-04-22T13:25:50.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.394+0000 7f08185d9740 20 client.16440 trim_cache size 0 max 16384 2024-04-22T13:25:50.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.408+0000 7f08185d9740 1 Processor -- start 2024-04-22T13:25:50.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.408+0000 7f08185d9740 1 -- start start 2024-04-22T13:25:50.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.408+0000 7f08185d9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de68d7c4d0 0x55de68d917d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.408+0000 7f08185d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de68d7ce80 0x55de68d84df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.408+0000 7f08185d9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de68d85a00 0x55de68d85330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.408+0000 7f08185d9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f08101adae0 con 0x55de68d85a00 2024-04-22T13:25:50.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.408+0000 7f08185d9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f08101af620 con 0x55de68d7c4d0 2024-04-22T13:25:50.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.408+0000 7f08185d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f08101ae1b0 con 0x55de68d7ce80 2024-04-22T13:25:50.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.408+0000 7f080f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de68d85a00 0x55de68d85330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.408+0000 7f080f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de68d85a00 0x55de68d85330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53240/0 (socket says 172.21.15.12:53240) 2024-04-22T13:25:50.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.408+0000 7f080e7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de68d7ce80 0x55de68d84df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.408+0000 7f080f7fe640 1 -- 172.21.15.12:0/2235314102 learned_addr learned my addr 172.21.15.12:0/2235314102 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:50.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.408+0000 7f080effd640 1 --2- 172.21.15.12:0/2235314102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de68d7c4d0 0x55de68d917d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.408+0000 7f080e7fc640 1 -- 172.21.15.12:0/2235314102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de68d7c4d0 msgr2=0x55de68d917d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:50.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.408+0000 7f080e7fc640 1 --2- 172.21.15.12:0/2235314102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de68d7c4d0 0x55de68d917d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.408+0000 7f080e7fc640 1 -- 172.21.15.12:0/2235314102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de68d85a00 msgr2=0x55de68d85330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:50.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.408+0000 7f080e7fc640 1 --2- 172.21.15.12:0/2235314102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de68d85a00 0x55de68d85330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.409+0000 7f080e7fc640 1 -- 172.21.15.12:0/2235314102 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de68d70fe0 con 0x55de68d7ce80 2024-04-22T13:25:50.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.409+0000 7f080effd640 1 --2- 172.21.15.12:0/2235314102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de68d7c4d0 0x55de68d917d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:50.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.409+0000 7f080f7fe640 1 --2- 172.21.15.12:0/2235314102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de68d85a00 0x55de68d85330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:50.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.409+0000 7f080e7fc640 1 --2- 172.21.15.12:0/2235314102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de68d7ce80 0x55de68d84df0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55de68d77c10 tx=0x7f07f8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cfc867f19d48bee server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:50.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.409+0000 7f08057fa640 1 -- 172.21.15.12:0/2235314102 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f805c070 con 0x55de68d7ce80 2024-04-22T13:25:50.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.409+0000 7f08057fa640 1 -- 172.21.15.12:0/2235314102 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07f804f030 con 0x55de68d7ce80 2024-04-22T13:25:50.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.409+0000 7f08057fa640 1 -- 172.21.15.12:0/2235314102 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f8003850 con 0x55de68d7ce80 2024-04-22T13:25:50.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.410+0000 7f08185d9740 1 -- 172.21.15.12:0/2235314102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de68d7ce80 msgr2=0x55de68d84df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.410+0000 7f08185d9740 1 --2- 172.21.15.12:0/2235314102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de68d7ce80 0x55de68d84df0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55de68d77c10 tx=0x7f07f8004ad0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.410+0000 7f08185d9740 1 -- 172.21.15.12:0/2235314102 shutdown_connections 2024-04-22T13:25:50.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.410+0000 7f08185d9740 1 --2- 172.21.15.12:0/2235314102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de68d85a00 0x55de68d85330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.410+0000 7f08185d9740 1 --2- 172.21.15.12:0/2235314102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de68d7ce80 0x55de68d84df0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.410+0000 7f08185d9740 1 --2- 172.21.15.12:0/2235314102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de68d7c4d0 0x55de68d917d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.410+0000 7f08185d9740 1 -- 172.21.15.12:0/2235314102 >> 172.21.15.12:0/2235314102 conn(0x55de68d76ba0 msgr2=0x55de68d75370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:50.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.410+0000 7f08185d9740 1 -- 172.21.15.12:0/2235314102 shutdown_connections 2024-04-22T13:25:50.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.410+0000 7f08185d9740 1 -- 172.21.15.12:0/2235314102 wait complete. 2024-04-22T13:25:50.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.410+0000 7f08185d9740 1 -- start start 2024-04-22T13:25:50.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.410+0000 7f08185d9740 1 Processor -- start 2024-04-22T13:25:50.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.411+0000 7f08185d9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de68d7c4d0 0x55de68d8ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.411+0000 7f08185d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de68d7ce80 0x55de68d7ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.411+0000 7f08185d9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de68d85a00 0x55de68d7f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.411+0000 7f08185d9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f08101aef50 con 0x55de68d85a00 2024-04-22T13:25:50.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.411+0000 7f08185d9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f08101ae880 con 0x55de68d7c4d0 2024-04-22T13:25:50.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.411+0000 7f08185d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f08101abac0 con 0x55de68d7ce80 2024-04-22T13:25:50.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.411+0000 7f08185d9740 10 client.0 _ll_register_callbacks cb 0x7f0800098a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:50.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.411+0000 7f080f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de68d85a00 0x55de68d7f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.411+0000 7f080e7fc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de68d7ce80 0x55de68d7ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.411+0000 7f080f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de68d85a00 0x55de68d7f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53246/0 (socket says 172.21.15.12:53246) 2024-04-22T13:25:50.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.411+0000 7f080f7fe640 1 -- 172.21.15.12:0/4256302727 learned_addr learned my addr 172.21.15.12:0/4256302727 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:50.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.411+0000 7f080effd640 1 --2- 172.21.15.12:0/4256302727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de68d7c4d0 0x55de68d8ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.411+0000 7f080f7fe640 1 -- 172.21.15.12:0/4256302727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de68d7ce80 msgr2=0x55de68d7ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:50.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.411+0000 7f080f7fe640 1 --2- 172.21.15.12:0/4256302727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de68d7ce80 0x55de68d7ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.411+0000 7f080f7fe640 1 -- 172.21.15.12:0/4256302727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de68d7c4d0 msgr2=0x55de68d8ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:50.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.411+0000 7f080f7fe640 1 --2- 172.21.15.12:0/4256302727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de68d7c4d0 0x55de68d8ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.411+0000 7f080f7fe640 1 -- 172.21.15.12:0/4256302727 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de68d7f7e0 con 0x55de68d85a00 2024-04-22T13:25:50.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.411+0000 7f080e7fc640 1 --2- 172.21.15.12:0/4256302727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de68d7ce80 0x55de68d7ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:50.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.411+0000 7f080f7fe640 1 --2- 172.21.15.12:0/4256302727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de68d85a00 0x55de68d7f2a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f080800bea0 tx=0x7f080800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651f4e5bc02728eb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:50.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.411+0000 7f07eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:50.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.411+0000 7f07eeffd640 1 -- 172.21.15.12:0/4256302727 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0808013520 con 0x55de68d85a00 2024-04-22T13:25:50.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.412+0000 7f07eeffd640 1 -- 172.21.15.12:0/4256302727 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f08080136c0 con 0x55de68d85a00 2024-04-22T13:25:50.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.412+0000 7f08185d9740 1 -- 172.21.15.12:0/4256302727 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07f804f030 con 0x55de68d85a00 2024-04-22T13:25:50.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.412+0000 7f07eeffd640 1 -- 172.21.15.12:0/4256302727 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f08080138c0 con 0x55de68d85a00 2024-04-22T13:25:50.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.412+0000 7f08185d9740 10 client.16446 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:50.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.412+0000 7f08185d9740 1 -- 172.21.15.12:0/4256302727 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55de68dfbe50 con 0x55de68d85a00 2024-04-22T13:25:50.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.412+0000 7f07eeffd640 1 -- 172.21.15.12:0/4256302727 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07f804f030 con 0x55de68d85a00 2024-04-22T13:25:50.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.412+0000 7f07eeffd640 1 -- 172.21.15.12:0/4256302727 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0808011b10 con 0x55de68d85a00 2024-04-22T13:25:50.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.412+0000 7f08185d9740 10 client.16446 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:50.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.412+0000 7f08185d9740 10 client.16446 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:25:50.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.412+0000 7f08185d9740 20 client.16446 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:50.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.412+0000 7f08185d9740 1 --2- 172.21.15.12:0/4256302727 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55de68dfc650 0x55de68dfea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:50.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.412+0000 7f08185d9740 4 client.16446 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:25:50.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.412+0000 7f08185d9740 1 -- 172.21.15.12:0/4256302727 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55de68ceec10 con 0x55de68dfc650 2024-04-22T13:25:50.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.412+0000 7f080effd640 1 --2- 172.21.15.12:0/4256302727 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55de68dfc650 0x55de68dfea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:50.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.413+0000 7f080effd640 1 --2- 172.21.15.12:0/4256302727 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55de68dfc650 0x55de68dfea70 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7939525adfb823e5 server_cookie=5890fdffcbe4cd72 in_seq=0 out_seq=0 2024-04-22T13:25:50.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.413+0000 7f07eeffd640 10 client.16446 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:50.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.413+0000 7f07eeffd640 1 -- 172.21.15.12:0/4256302727 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55de68ceec10 con 0x55de68dfc650 2024-04-22T13:25:50.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.413+0000 7f07eeffd640 10 client.16446 handle_command_reply: tid=0 2024-04-22T13:25:50.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.413+0000 7f08185d9740 1 client.16446 shutdown 2024-04-22T13:25:50.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.413+0000 7f08185d9740 1 -- 172.21.15.12:0/4256302727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de68d85a00 msgr2=0x55de68d7f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.413+0000 7f08185d9740 1 --2- 172.21.15.12:0/4256302727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de68d85a00 0x55de68d7f2a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f080800bea0 tx=0x7f080800bf90 comp rx=0 tx=0).stop 2024-04-22T13:25:50.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.413+0000 7f08185d9740 1 -- 172.21.15.12:0/4256302727 shutdown_connections 2024-04-22T13:25:50.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.413+0000 7f08185d9740 1 --2- 172.21.15.12:0/4256302727 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55de68dfc650 0x55de68dfea70 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.413+0000 7f08185d9740 1 --2- 172.21.15.12:0/4256302727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de68d85a00 0x55de68d7f2a0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.413+0000 7f08185d9740 1 --2- 172.21.15.12:0/4256302727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de68d7ce80 0x55de68d7ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.413+0000 7f08185d9740 1 --2- 172.21.15.12:0/4256302727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de68d7c4d0 0x55de68d8ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.413+0000 7f07eeffd640 0 client.16446 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:50.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.413+0000 7f08185d9740 1 -- 172.21.15.12:0/4256302727 >> 172.21.15.12:0/4256302727 conn(0x55de68d76ba0 msgr2=0x55de68d75c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:50.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.413+0000 7f08185d9740 1 -- 172.21.15.12:0/4256302727 shutdown_connections 2024-04-22T13:25:50.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.413+0000 7f08185d9740 1 -- 172.21.15.12:0/4256302727 wait complete. 2024-04-22T13:25:50.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.413+0000 7f08185d9740 20 client.16446 trim_cache size 0 max 16384 2024-04-22T13:25:50.416 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:25:50.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.414+0000 7f081590e640 1 -- 172.21.15.12:0/884183970 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f07fc0a3d20 msgr2=0x7f07fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.414+0000 7f081590e640 1 --2- 172.21.15.12:0/884183970 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f07fc0a3d20 0x7f07fc0a61e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f07f8006010 tx=0x7f07f802c690 comp rx=0 tx=0).stop 2024-04-22T13:25:50.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.414+0000 7f081590e640 1 -- 172.21.15.12:0/884183970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08101021a0 msgr2=0x7f081019e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:50.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.414+0000 7f081590e640 1 --2- 172.21.15.12:0/884183970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08101021a0 0x7f081019e3e0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f080000db80 tx=0x7f080000b630 comp rx=0 tx=0).stop 2024-04-22T13:25:50.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.414+0000 7f081590e640 1 -- 172.21.15.12:0/884183970 shutdown_connections 2024-04-22T13:25:50.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.414+0000 7f081590e640 1 --2- 172.21.15.12:0/884183970 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f07fc0a3d20 0x7f07fc0a61e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.414+0000 7f081590e640 1 --2- 172.21.15.12:0/884183970 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0810198590 0x7f0810198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.414+0000 7f081590e640 1 --2- 172.21.15.12:0/884183970 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0810107270 0x7f081019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.414+0000 7f081590e640 1 --2- 172.21.15.12:0/884183970 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f08101021a0 0x7f081019e3e0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:50.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.414+0000 7f081590e640 1 -- 172.21.15.12:0/884183970 >> 172.21.15.12:0/884183970 conn(0x7f0810079350 msgr2=0x7f0810100310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:50.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.414+0000 7f081590e640 1 -- 172.21.15.12:0/884183970 shutdown_connections 2024-04-22T13:25:50.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:50.414+0000 7f081590e640 1 -- 172.21.15.12:0/884183970 wait complete. 2024-04-22T13:25:51.422 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:25:51.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:25:51.423 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:25:51.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.538+0000 7f2eb08d2640 1 -- 172.21.15.12:0/3313962816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2eac06ac80 msgr2=0x7f2eac10e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:51.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.538+0000 7f2eb08d2640 1 --2- 172.21.15.12:0/3313962816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2eac06ac80 0x7f2eac10e420 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0009900 tx=0x7f2ea002bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:51.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.539+0000 7f2eb08d2640 1 -- 172.21.15.12:0/3313962816 shutdown_connections 2024-04-22T13:25:51.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.539+0000 7f2eb08d2640 1 --2- 172.21.15.12:0/3313962816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2eac10e960 0x7f2eac110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.539+0000 7f2eb08d2640 1 --2- 172.21.15.12:0/3313962816 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2eac06ac80 0x7f2eac10e420 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.539+0000 7f2eb08d2640 1 --2- 172.21.15.12:0/3313962816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2eac06a360 0x7f2eac06a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.539+0000 7f2eb08d2640 1 -- 172.21.15.12:0/3313962816 >> 172.21.15.12:0/3313962816 conn(0x7f2eac077ef0 msgr2=0x7f2eac078300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:51.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.539+0000 7f2eb08d2640 1 -- 172.21.15.12:0/3313962816 shutdown_connections 2024-04-22T13:25:51.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.539+0000 7f2eb08d2640 1 -- 172.21.15.12:0/3313962816 wait complete. 2024-04-22T13:25:51.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.539+0000 7f2eb08d2640 1 Processor -- start 2024-04-22T13:25:51.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.540+0000 7f2eb08d2640 1 -- start start 2024-04-22T13:25:51.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.540+0000 7f2eb08d2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2eac06a360 0x7f2eac1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.540+0000 7f2eb08d2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2eac06ac80 0x7f2eac1a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.540+0000 7f2eb08d2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2eac10e960 0x7f2eac19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.540+0000 7f2eb08d2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac1143c0 con 0x7f2eac06a360 2024-04-22T13:25:51.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.540+0000 7f2eb08d2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac114240 con 0x7f2eac06ac80 2024-04-22T13:25:51.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.540+0000 7f2eb08d2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac114540 con 0x7f2eac10e960 2024-04-22T13:25:51.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.540+0000 7f2eaa59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2eac06a360 0x7f2eac1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.540+0000 7f2eaa59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2eac06a360 0x7f2eac1a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53270/0 (socket says 172.21.15.12:53270) 2024-04-22T13:25:51.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.540+0000 7f2eaa59c640 1 -- 172.21.15.12:0/2274665469 learned_addr learned my addr 172.21.15.12:0/2274665469 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:51.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.540+0000 7f2eaad9d640 1 --2- 172.21.15.12:0/2274665469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2eac10e960 0x7f2eac19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.540+0000 7f2ea9d9b640 1 --2- 172.21.15.12:0/2274665469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2eac06ac80 0x7f2eac1a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.540+0000 7f2eaa59c640 1 -- 172.21.15.12:0/2274665469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2eac10e960 msgr2=0x7f2eac19a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:51.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.540+0000 7f2eaa59c640 1 --2- 172.21.15.12:0/2274665469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2eac10e960 0x7f2eac19a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.540+0000 7f2eaa59c640 1 -- 172.21.15.12:0/2274665469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2eac06ac80 msgr2=0x7f2eac1a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:51.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.540+0000 7f2eaa59c640 1 --2- 172.21.15.12:0/2274665469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2eac06ac80 0x7f2eac1a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.540+0000 7f2eaa59c640 1 -- 172.21.15.12:0/2274665469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eac19afe0 con 0x7f2eac06a360 2024-04-22T13:25:51.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.541+0000 7f2eaa59c640 1 --2- 172.21.15.12:0/2274665469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2eac06a360 0x7f2eac1a0660 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00c920 tx=0x7f2e9c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:51.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.541+0000 7f2e9b7fe640 1 -- 172.21.15.12:0/2274665469 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c007c70 con 0x7f2eac06a360 2024-04-22T13:25:51.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.541+0000 7f2e9b7fe640 1 -- 172.21.15.12:0/2274665469 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e9c007e10 con 0x7f2eac06a360 2024-04-22T13:25:51.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.541+0000 7f2eb08d2640 1 -- 172.21.15.12:0/2274665469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eac19b2d0 con 0x7f2eac06a360 2024-04-22T13:25:51.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.541+0000 7f2e9b7fe640 1 -- 172.21.15.12:0/2274665469 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c018820 con 0x7f2eac06a360 2024-04-22T13:25:51.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.541+0000 7f2eb08d2640 1 -- 172.21.15.12:0/2274665469 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eac1a75e0 con 0x7f2eac06a360 2024-04-22T13:25:51.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.544+0000 7f2eb359e740 1 Processor -- start 2024-04-22T13:25:51.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.544+0000 7f2eb359e740 1 -- start start 2024-04-22T13:25:51.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.544+0000 7f2eb359e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f37dfa98e0 0x55f37dfa9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.544+0000 7f2eb359e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37dfaa290 0x55f37dfb2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.544+0000 7f2eb359e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f37dfb2e10 0x55f37dfb5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.544+0000 7f2eb359e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f37df0df00 con 0x55f37dfaa290 2024-04-22T13:25:51.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.544+0000 7f2eb359e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f37deab590 con 0x55f37dfa98e0 2024-04-22T13:25:51.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.544+0000 7f2eb359e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f37df09ec0 con 0x55f37dfb2e10 2024-04-22T13:25:51.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.545+0000 7f2ea9d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f37dfb2e10 0x55f37dfb5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.545+0000 7f2ea9d9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f37dfb2e10 0x55f37dfb5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.12:51608/0 (socket says 172.21.15.12:51608) 2024-04-22T13:25:51.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.545+0000 7f2eaad9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f37dfa98e0 0x55f37dfa9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.545+0000 7f2ea9d9b640 1 -- 172.21.15.12:0/3817986516 learned_addr learned my addr 172.21.15.12:0/3817986516 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:51.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.545+0000 7f2e9b7fe640 1 -- 172.21.15.12:0/2274665469 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2e9c00f450 con 0x7f2eac06a360 2024-04-22T13:25:51.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.545+0000 7f2ea9d9b640 1 -- 172.21.15.12:0/3817986516 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f37dfa98e0 msgr2=0x55f37dfa9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:51.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.545+0000 7f2ea9d9b640 1 --2- 172.21.15.12:0/3817986516 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f37dfa98e0 0x55f37dfa9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.545+0000 7f2ea9d9b640 1 -- 172.21.15.12:0/3817986516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37dfaa290 msgr2=0x55f37dfb2840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-22T13:25:51.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.545+0000 7f2ea9d9b640 1 --2- 172.21.15.12:0/3817986516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37dfaa290 0x55f37dfb2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.545+0000 7f2ea9d9b640 1 -- 172.21.15.12:0/3817986516 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f37dce87e0 con 0x55f37dfb2e10 2024-04-22T13:25:51.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.545+0000 7f2ea9d9b640 1 --2- 172.21.15.12:0/3817986516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f37dfb2e10 0x55f37dfb5260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0009900 tx=0x7f2ea0004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9c03377abe8667a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:51.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.546+0000 7f2e98ff9640 1 -- 172.21.15.12:0/3817986516 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea0045020 con 0x55f37dfb2e10 2024-04-22T13:25:51.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.546+0000 7f2e9b7fe640 1 --2- 172.21.15.12:0/2274665469 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2e940a3d20 0x7f2e940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.546+0000 7f2e98ff9640 1 -- 172.21.15.12:0/3817986516 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ea002cc90 con 0x55f37dfb2e10 2024-04-22T13:25:51.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.546+0000 7f2e98ff9640 1 -- 172.21.15.12:0/3817986516 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea0004920 con 0x55f37dfb2e10 2024-04-22T13:25:51.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.546+0000 7f2eaad9d640 1 --2- 172.21.15.12:0/2274665469 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2e940a3d20 0x7f2e940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.546+0000 7f2e9b7fe640 1 -- 172.21.15.12:0/2274665469 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f2e9c0d0830 con 0x7f2eac06a360 2024-04-22T13:25:51.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.547+0000 7f2eaad9d640 1 --2- 172.21.15.12:0/2274665469 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2e940a3d20 0x7f2e940a61e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2eac068dd0 tx=0x7f2ea4009600 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:51.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.547+0000 7f2eb359e740 1 -- 172.21.15.12:0/3817986516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f37dfb2e10 msgr2=0x55f37dfb5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:51.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.547+0000 7f2eb359e740 1 --2- 172.21.15.12:0/3817986516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f37dfb2e10 0x55f37dfb5260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0009900 tx=0x7f2ea0004040 comp rx=0 tx=0).stop 2024-04-22T13:25:51.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.547+0000 7f2eb359e740 1 -- 172.21.15.12:0/3817986516 shutdown_connections 2024-04-22T13:25:51.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.547+0000 7f2eb359e740 1 --2- 172.21.15.12:0/3817986516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f37dfb2e10 0x55f37dfb5260 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.547+0000 7f2eb359e740 1 --2- 172.21.15.12:0/3817986516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37dfaa290 0x55f37dfb2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.547+0000 7f2eb359e740 1 --2- 172.21.15.12:0/3817986516 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f37dfa98e0 0x55f37dfa9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.547+0000 7f2eb359e740 1 -- 172.21.15.12:0/3817986516 >> 172.21.15.12:0/3817986516 conn(0x55f37dfa3fb0 msgr2=0x55f37dfa4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:51.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.547+0000 7f2eb359e740 1 -- 172.21.15.12:0/3817986516 shutdown_connections 2024-04-22T13:25:51.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.547+0000 7f2eb359e740 1 -- 172.21.15.12:0/3817986516 wait complete. 2024-04-22T13:25:51.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.547+0000 7f2eb359e740 1 -- start start 2024-04-22T13:25:51.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.547+0000 7f2eb359e740 1 Processor -- start 2024-04-22T13:25:51.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.548+0000 7f2eb359e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f37dfa98e0 0x55f37dfafe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.548+0000 7f2eb359e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37dfaa290 0x55f37dfb0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.548+0000 7f2eb359e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f37dfb2e10 0x55f37dfb08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.548+0000 7f2eb359e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f37dfb5d20 con 0x55f37dfaa290 2024-04-22T13:25:51.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.548+0000 7f2eb359e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f37dec7580 con 0x55f37dfb2e10 2024-04-22T13:25:51.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.548+0000 7f2eb359e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f37dfb5ea0 con 0x55f37dfa98e0 2024-04-22T13:25:51.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.548+0000 7f2eb359e740 10 client.0 _ll_register_callbacks cb 0x55f37decd650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:51.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.548+0000 7f2ea9d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37dfaa290 0x55f37dfb0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.548+0000 7f2eaa59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f37dfa98e0 0x55f37dfafe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.548+0000 7f2ea9d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37dfaa290 0x55f37dfb0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53294/0 (socket says 172.21.15.12:53294) 2024-04-22T13:25:51.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.548+0000 7f2ea9d9b640 1 -- 172.21.15.12:0/578178045 learned_addr learned my addr 172.21.15.12:0/578178045 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:51.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.548+0000 7f2eaad9d640 1 --2- 172.21.15.12:0/578178045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f37dfb2e10 0x55f37dfb08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.548+0000 7f2ea9d9b640 1 -- 172.21.15.12:0/578178045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f37dfa98e0 msgr2=0x55f37dfafe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:51.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.548+0000 7f2ea9d9b640 1 --2- 172.21.15.12:0/578178045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f37dfa98e0 0x55f37dfafe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.548+0000 7f2ea9d9b640 1 -- 172.21.15.12:0/578178045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f37dfb2e10 msgr2=0x55f37dfb08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:51.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.548+0000 7f2ea9d9b640 1 --2- 172.21.15.12:0/578178045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f37dfb2e10 0x55f37dfb08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.548+0000 7f2ea9d9b640 1 -- 172.21.15.12:0/578178045 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f37dd48f70 con 0x55f37dfaa290 2024-04-22T13:25:51.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.548+0000 7f2eaa59c640 1 --2- 172.21.15.12:0/578178045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f37dfa98e0 0x55f37dfafe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:51.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.548+0000 7f2eaad9d640 1 --2- 172.21.15.12:0/578178045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f37dfb2e10 0x55f37dfb08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:51.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.549+0000 7f2ea9d9b640 1 --2- 172.21.15.12:0/578178045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37dfaa290 0x55f37dfb0390 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f2ea002ca10 tx=0x7f2ea002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9181759f4ed2cc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:51.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.549+0000 7f2e91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:51.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.549+0000 7f2e91ffb640 1 -- 172.21.15.12:0/578178045 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea004e020 con 0x55f37dfaa290 2024-04-22T13:25:51.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.549+0000 7f2e91ffb640 1 -- 172.21.15.12:0/578178045 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f2ea0003800 con 0x55f37dfaa290 2024-04-22T13:25:51.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.549+0000 7f2eb359e740 1 -- 172.21.15.12:0/578178045 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ea002cc90 con 0x55f37dfaa290 2024-04-22T13:25:51.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.549+0000 7f2e91ffb640 1 -- 172.21.15.12:0/578178045 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2ea0045070 con 0x55f37dfaa290 2024-04-22T13:25:51.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.549+0000 7f2eb359e740 10 client.16455 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:51.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.549+0000 7f2eb359e740 1 -- 172.21.15.12:0/578178045 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f37dbe0170 con 0x55f37dfaa290 2024-04-22T13:25:51.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.549+0000 7f2e91ffb640 1 -- 172.21.15.12:0/578178045 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ea002cc90 con 0x55f37dfaa290 2024-04-22T13:25:51.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.549+0000 7f2e91ffb640 1 -- 172.21.15.12:0/578178045 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2ea0004c40 con 0x55f37dfaa290 2024-04-22T13:25:51.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.549+0000 7f2eb359e740 10 client.16455 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:51.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.549+0000 7f2eb359e740 10 client.16455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:25:51.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.549+0000 7f2eb359e740 20 client.16455 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:51.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.550+0000 7f2eb359e740 1 --2- 172.21.15.12:0/578178045 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f37dfc31d0 0x55f37dfc55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.550+0000 7f2eb359e740 4 client.16455 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:25:51.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.550+0000 7f2eb359e740 1 -- 172.21.15.12:0/578178045 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f37df1b920 con 0x55f37dfc31d0 2024-04-22T13:25:51.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.550+0000 7f2eaa59c640 1 --2- 172.21.15.12:0/578178045 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f37dfc31d0 0x55f37dfc55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.550+0000 7f2eaa59c640 1 --2- 172.21.15.12:0/578178045 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f37dfc31d0 0x55f37dfc55c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8fa65f09a8f5a8b server_cookie=34c5d86e7ef1d4df in_seq=0 out_seq=0 2024-04-22T13:25:51.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.550+0000 7f2e91ffb640 10 client.16455 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:51.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.551+0000 7f2e91ffb640 1 -- 172.21.15.12:0/578178045 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f37df1b920 con 0x55f37dfc31d0 2024-04-22T13:25:51.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.551+0000 7f2e91ffb640 10 client.16455 handle_command_reply: tid=0 2024-04-22T13:25:51.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.551+0000 7f2eb359e740 1 client.16455 shutdown 2024-04-22T13:25:51.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.551+0000 7f2eb359e740 1 -- 172.21.15.12:0/578178045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37dfaa290 msgr2=0x55f37dfb0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:51.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.551+0000 7f2eb359e740 1 --2- 172.21.15.12:0/578178045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37dfaa290 0x55f37dfb0390 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f2ea002ca10 tx=0x7f2ea002bc30 comp rx=0 tx=0).stop 2024-04-22T13:25:51.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.551+0000 7f2eb359e740 1 -- 172.21.15.12:0/578178045 shutdown_connections 2024-04-22T13:25:51.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.551+0000 7f2eb359e740 1 --2- 172.21.15.12:0/578178045 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f37dfc31d0 0x55f37dfc55c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.551+0000 7f2eb359e740 1 --2- 172.21.15.12:0/578178045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f37dfb2e10 0x55f37dfb08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.551+0000 7f2e91ffb640 0 client.16455 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:51.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.551+0000 7f2eb359e740 1 --2- 172.21.15.12:0/578178045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37dfaa290 0x55f37dfb0390 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.551+0000 7f2eb359e740 1 --2- 172.21.15.12:0/578178045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f37dfa98e0 0x55f37dfafe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.551+0000 7f2eb359e740 1 -- 172.21.15.12:0/578178045 >> 172.21.15.12:0/578178045 conn(0x55f37dfa3fb0 msgr2=0x55f37dfa5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:51.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.551+0000 7f2eb359e740 1 -- 172.21.15.12:0/578178045 shutdown_connections 2024-04-22T13:25:51.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.551+0000 7f2eb359e740 1 -- 172.21.15.12:0/578178045 wait complete. 2024-04-22T13:25:51.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.552+0000 7f2eb359e740 20 client.16455 trim_cache size 0 max 16384 2024-04-22T13:25:51.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.590+0000 7f2eb359e740 1 Processor -- start 2024-04-22T13:25:51.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.590+0000 7f2eb359e740 1 -- start start 2024-04-22T13:25:51.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.590+0000 7f2eb359e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37dfa98e0 0x55f37dfb2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.590+0000 7f2eb359e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f37dfaa290 0x55f37dfbe800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.590+0000 7f2eb359e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f37dfb2e10 0x55f37dfc85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.590+0000 7f2eb359e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac1afbb0 con 0x55f37dfa98e0 2024-04-22T13:25:51.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.590+0000 7f2eb359e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac1b16f0 con 0x55f37dfb2e10 2024-04-22T13:25:51.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.590+0000 7f2eb359e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac1b0280 con 0x55f37dfaa290 2024-04-22T13:25:51.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.590+0000 7f2eaa59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37dfa98e0 0x55f37dfb2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.590+0000 7f2eaa59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37dfa98e0 0x55f37dfb2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53306/0 (socket says 172.21.15.12:53306) 2024-04-22T13:25:51.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.590+0000 7f2eaa59c640 1 -- 172.21.15.12:0/906809769 learned_addr learned my addr 172.21.15.12:0/906809769 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:51.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.590+0000 7f2ea9d9b640 1 --2- 172.21.15.12:0/906809769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f37dfaa290 0x55f37dfbe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.591+0000 7f2eaad9d640 1 --2- 172.21.15.12:0/906809769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f37dfb2e10 0x55f37dfc85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.591+0000 7f2eaa59c640 1 -- 172.21.15.12:0/906809769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f37dfaa290 msgr2=0x55f37dfbe800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:51.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.591+0000 7f2eaa59c640 1 --2- 172.21.15.12:0/906809769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f37dfaa290 0x55f37dfbe800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.591+0000 7f2eaa59c640 1 -- 172.21.15.12:0/906809769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f37dfb2e10 msgr2=0x55f37dfc85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:51.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.591+0000 7f2eaa59c640 1 --2- 172.21.15.12:0/906809769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f37dfb2e10 0x55f37dfc85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.591+0000 7f2eaa59c640 1 -- 172.21.15.12:0/906809769 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f37dfa45c0 con 0x55f37dfa98e0 2024-04-22T13:25:51.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.591+0000 7f2ea9d9b640 1 --2- 172.21.15.12:0/906809769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f37dfaa290 0x55f37dfbe800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:25:51.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.591+0000 7f2eaad9d640 1 --2- 172.21.15.12:0/906809769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f37dfb2e10 0x55f37dfc85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:51.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.591+0000 7f2eaa59c640 1 --2- 172.21.15.12:0/906809769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37dfa98e0 0x55f37dfb2580 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c09c7b0 tx=0x7f2e9c087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40bb3fc807c510cf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:51.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.591+0000 7f2e93fff640 1 -- 172.21.15.12:0/906809769 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c0a0bd0 con 0x55f37dfa98e0 2024-04-22T13:25:51.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.591+0000 7f2e93fff640 1 -- 172.21.15.12:0/906809769 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e9c0a5490 con 0x55f37dfa98e0 2024-04-22T13:25:51.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.591+0000 7f2e93fff640 1 -- 172.21.15.12:0/906809769 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c098e90 con 0x55f37dfa98e0 2024-04-22T13:25:51.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.592+0000 7f2eb359e740 1 -- 172.21.15.12:0/906809769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37dfa98e0 msgr2=0x55f37dfb2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:51.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.592+0000 7f2eb359e740 1 --2- 172.21.15.12:0/906809769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37dfa98e0 0x55f37dfb2580 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c09c7b0 tx=0x7f2e9c087040 comp rx=0 tx=0).stop 2024-04-22T13:25:51.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.592+0000 7f2eb359e740 1 -- 172.21.15.12:0/906809769 shutdown_connections 2024-04-22T13:25:51.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.592+0000 7f2eb359e740 1 --2- 172.21.15.12:0/906809769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f37dfb2e10 0x55f37dfc85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.592+0000 7f2eb359e740 1 --2- 172.21.15.12:0/906809769 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f37dfaa290 0x55f37dfbe800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.592+0000 7f2eb359e740 1 --2- 172.21.15.12:0/906809769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37dfa98e0 0x55f37dfb2580 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.592+0000 7f2eb359e740 1 -- 172.21.15.12:0/906809769 >> 172.21.15.12:0/906809769 conn(0x55f37dfa3fb0 msgr2=0x55f37dfbe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:51.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.592+0000 7f2eb359e740 1 -- 172.21.15.12:0/906809769 shutdown_connections 2024-04-22T13:25:51.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.592+0000 7f2eb359e740 1 -- 172.21.15.12:0/906809769 wait complete. 2024-04-22T13:25:51.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.592+0000 7f2eb359e740 1 -- start start 2024-04-22T13:25:51.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.592+0000 7f2eb359e740 1 Processor -- start 2024-04-22T13:25:51.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.593+0000 7f2eb359e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f37dfa98e0 0x55f37dfc6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.593+0000 7f2eb359e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f37dfaa290 0x55f37dfc6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.593+0000 7f2eb359e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37dfb2e10 0x55f37dfc6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.593+0000 7f2eb359e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac1b1020 con 0x55f37dfb2e10 2024-04-22T13:25:51.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.593+0000 7f2eb359e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac1b0950 con 0x55f37dfa98e0 2024-04-22T13:25:51.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.593+0000 7f2eb359e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2eac1adb90 con 0x55f37dfaa290 2024-04-22T13:25:51.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.593+0000 7f2eb359e740 10 client.0 _ll_register_callbacks cb 0x7f2ea4009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:51.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.593+0000 7f2eaad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37dfb2e10 0x55f37dfc6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.593+0000 7f2eaad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37dfb2e10 0x55f37dfc6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53308/0 (socket says 172.21.15.12:53308) 2024-04-22T13:25:51.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.593+0000 7f2eaad9d640 1 -- 172.21.15.12:0/206961898 learned_addr learned my addr 172.21.15.12:0/206961898 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:51.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.593+0000 7f2ea9d9b640 1 --2- 172.21.15.12:0/206961898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f37dfaa290 0x55f37dfc6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.593+0000 7f2eaa59c640 1 --2- 172.21.15.12:0/206961898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f37dfa98e0 0x55f37dfc6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.593+0000 7f2eaad9d640 1 -- 172.21.15.12:0/206961898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f37dfaa290 msgr2=0x55f37dfc6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:51.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.593+0000 7f2eaad9d640 1 --2- 172.21.15.12:0/206961898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f37dfaa290 0x55f37dfc6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.593+0000 7f2eaad9d640 1 -- 172.21.15.12:0/206961898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f37dfa98e0 msgr2=0x55f37dfc6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:51.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.593+0000 7f2eaad9d640 1 --2- 172.21.15.12:0/206961898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f37dfa98e0 0x55f37dfc6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.593+0000 7f2eaad9d640 1 -- 172.21.15.12:0/206961898 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f37e0284b0 con 0x55f37dfb2e10 2024-04-22T13:25:51.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.593+0000 7f2eaa59c640 1 --2- 172.21.15.12:0/206961898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f37dfa98e0 0x55f37dfc6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:25:51.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.593+0000 7f2eaad9d640 1 --2- 172.21.15.12:0/206961898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37dfb2e10 0x55f37dfc6bb0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4020470 tx=0x7f2ea4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b36ce9d899301a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:51.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.593+0000 7f2e91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:51.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.594+0000 7f2e91ffb640 1 -- 172.21.15.12:0/206961898 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea402f060 con 0x55f37dfb2e10 2024-04-22T13:25:51.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.594+0000 7f2eb359e740 1 -- 172.21.15.12:0/206961898 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e9c0a5490 con 0x55f37dfb2e10 2024-04-22T13:25:51.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.594+0000 7f2e91ffb640 1 -- 172.21.15.12:0/206961898 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f2ea401d600 con 0x55f37dfb2e10 2024-04-22T13:25:51.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.594+0000 7f2e91ffb640 1 -- 172.21.15.12:0/206961898 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2ea401d800 con 0x55f37dfb2e10 2024-04-22T13:25:51.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.594+0000 7f2eb359e740 10 client.16461 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:51.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.594+0000 7f2eb359e740 1 -- 172.21.15.12:0/206961898 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f37e029850 con 0x55f37dfb2e10 2024-04-22T13:25:51.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.594+0000 7f2e91ffb640 1 -- 172.21.15.12:0/206961898 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e9c0a5490 con 0x55f37dfb2e10 2024-04-22T13:25:51.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.594+0000 7f2e91ffb640 1 -- 172.21.15.12:0/206961898 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2ea4029070 con 0x55f37dfb2e10 2024-04-22T13:25:51.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.594+0000 7f2eb359e740 10 client.16461 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:51.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.594+0000 7f2eb359e740 10 client.16461 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:25:51.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.594+0000 7f2eb359e740 20 client.16461 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:51.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.594+0000 7f2eb359e740 1 --2- 172.21.15.12:0/206961898 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f37dfc31d0 0x55f37e02c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.594+0000 7f2eb359e740 4 client.16461 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:25:51.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.594+0000 7f2eb359e740 1 -- 172.21.15.12:0/206961898 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f37df15790 con 0x55f37dfc31d0 2024-04-22T13:25:51.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.595+0000 7f2eaa59c640 1 --2- 172.21.15.12:0/206961898 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f37dfc31d0 0x55f37e02c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.595+0000 7f2eaa59c640 1 --2- 172.21.15.12:0/206961898 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f37dfc31d0 0x55f37e02c040 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77e41d2ce2e0393b server_cookie=d8b3fcef3e791a81 in_seq=0 out_seq=0 2024-04-22T13:25:51.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.595+0000 7f2e91ffb640 10 client.16461 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:51.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.596+0000 7f2e91ffb640 1 -- 172.21.15.12:0/206961898 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f37df15790 con 0x55f37dfc31d0 2024-04-22T13:25:51.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.596+0000 7f2e91ffb640 10 client.16461 handle_command_reply: tid=0 2024-04-22T13:25:51.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.596+0000 7f2eb359e740 1 client.16461 shutdown 2024-04-22T13:25:51.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.596+0000 7f2eb359e740 1 -- 172.21.15.12:0/206961898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37dfb2e10 msgr2=0x55f37dfc6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:51.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.596+0000 7f2eb359e740 1 --2- 172.21.15.12:0/206961898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37dfb2e10 0x55f37dfc6bb0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4020470 tx=0x7f2ea4004a40 comp rx=0 tx=0).stop 2024-04-22T13:25:51.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.596+0000 7f2eb359e740 1 -- 172.21.15.12:0/206961898 shutdown_connections 2024-04-22T13:25:51.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.597+0000 7f2eb359e740 1 --2- 172.21.15.12:0/206961898 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f37dfc31d0 0x55f37e02c040 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.597+0000 7f2eb359e740 1 --2- 172.21.15.12:0/206961898 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37dfb2e10 0x55f37dfc6bb0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.597+0000 7f2eb359e740 1 --2- 172.21.15.12:0/206961898 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f37dfaa290 0x55f37dfc6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.597+0000 7f2e91ffb640 0 client.16461 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:51.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.597+0000 7f2eb359e740 1 --2- 172.21.15.12:0/206961898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f37dfa98e0 0x55f37dfc6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.597+0000 7f2eb359e740 1 -- 172.21.15.12:0/206961898 >> 172.21.15.12:0/206961898 conn(0x55f37dfa3fb0 msgr2=0x55f37dfbe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:51.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.597+0000 7f2eb359e740 1 -- 172.21.15.12:0/206961898 shutdown_connections 2024-04-22T13:25:51.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.597+0000 7f2eb359e740 1 -- 172.21.15.12:0/206961898 wait complete. 2024-04-22T13:25:51.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.597+0000 7f2eb359e740 20 client.16461 trim_cache size 0 max 16384 2024-04-22T13:25:51.599 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:25:51.599 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-04-22T13:25:51.599 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "6628281c-e6d8-4da8-af0a-dd5be23738bf", 2024-04-22T13:25:51.599 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-04-22T13:25:51.599 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:25:51.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.597+0000 7f2eb08d2640 1 -- 172.21.15.12:0/2274665469 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2e940a3d20 msgr2=0x7f2e940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:51.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.597+0000 7f2eb08d2640 1 --2- 172.21.15.12:0/2274665469 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2e940a3d20 0x7f2e940a61e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2eac068dd0 tx=0x7f2ea4009600 comp rx=0 tx=0).stop 2024-04-22T13:25:51.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.597+0000 7f2eb08d2640 1 -- 172.21.15.12:0/2274665469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2eac06a360 msgr2=0x7f2eac1a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:51.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.597+0000 7f2eb08d2640 1 --2- 172.21.15.12:0/2274665469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2eac06a360 0x7f2eac1a0660 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00c920 tx=0x7f2e9c00cde0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.597+0000 7f2eb08d2640 1 -- 172.21.15.12:0/2274665469 shutdown_connections 2024-04-22T13:25:51.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.597+0000 7f2eb08d2640 1 --2- 172.21.15.12:0/2274665469 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2e940a3d20 0x7f2e940a61e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.597+0000 7f2eb08d2640 1 --2- 172.21.15.12:0/2274665469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2eac10e960 0x7f2eac19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.597+0000 7f2eb08d2640 1 --2- 172.21.15.12:0/2274665469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2eac06ac80 0x7f2eac1a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.597+0000 7f2eb08d2640 1 --2- 172.21.15.12:0/2274665469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2eac06a360 0x7f2eac1a0660 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.597+0000 7f2eb08d2640 1 -- 172.21.15.12:0/2274665469 >> 172.21.15.12:0/2274665469 conn(0x7f2eac077ef0 msgr2=0x7f2eac10fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:51.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.597+0000 7f2eb08d2640 1 -- 172.21.15.12:0/2274665469 shutdown_connections 2024-04-22T13:25:51.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.597+0000 7f2eb08d2640 1 -- 172.21.15.12:0/2274665469 wait complete. 2024-04-22T13:25:51.605 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:25:51.605 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:25:51.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.720+0000 7facd0ed9640 1 -- 172.21.15.12:0/2740415607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faccc103150 msgr2=0x7faccc103530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:51.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.720+0000 7facd0ed9640 1 --2- 172.21.15.12:0/2740415607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faccc103150 0x7faccc103530 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7facbc009900 tx=0x7facbc02bcd0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.720+0000 7facd0ed9640 1 -- 172.21.15.12:0/2740415607 shutdown_connections 2024-04-22T13:25:51.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.720+0000 7facd0ed9640 1 --2- 172.21.15.12:0/2740415607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faccc10c520 0x7faccc10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.720+0000 7facd0ed9640 1 --2- 172.21.15.12:0/2740415607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faccc103a70 0x7faccc10bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.720+0000 7facd0ed9640 1 --2- 172.21.15.12:0/2740415607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faccc103150 0x7faccc103530 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.720+0000 7facd0ed9640 1 -- 172.21.15.12:0/2740415607 >> 172.21.15.12:0/2740415607 conn(0x7faccc077c00 msgr2=0x7faccc0ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:51.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.720+0000 7facd0ed9640 1 -- 172.21.15.12:0/2740415607 shutdown_connections 2024-04-22T13:25:51.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.720+0000 7facd0ed9640 1 -- 172.21.15.12:0/2740415607 wait complete. 2024-04-22T13:25:51.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.721+0000 7facd0ed9640 1 Processor -- start 2024-04-22T13:25:51.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.721+0000 7facd0ed9640 1 -- start start 2024-04-22T13:25:51.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.721+0000 7facd0ed9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faccc103150 0x7faccc198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.721+0000 7facd0ed9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faccc103a70 0x7faccc198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.721+0000 7facd0ed9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faccc10c520 0x7faccc19d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.721+0000 7facd0ed9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7faccc1121a0 con 0x7faccc10c520 2024-04-22T13:25:51.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.721+0000 7facd0ed9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faccc112020 con 0x7faccc103a70 2024-04-22T13:25:51.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.721+0000 7facd0ed9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faccc112320 con 0x7faccc103150 2024-04-22T13:25:51.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.721+0000 7faccad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faccc10c520 0x7faccc19d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.722+0000 7faccad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faccc10c520 0x7faccc19d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58190/0 (socket says 172.21.15.12:58190) 2024-04-22T13:25:51.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.722+0000 7faccad9d640 1 -- 172.21.15.12:0/2500362275 learned_addr learned my addr 172.21.15.12:0/2500362275 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:51.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.722+0000 7facca59c640 1 --2- 172.21.15.12:0/2500362275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faccc103150 0x7faccc198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.722+0000 7facc3fff640 1 --2- 172.21.15.12:0/2500362275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faccc103a70 0x7faccc198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.722+0000 7faccad9d640 1 -- 172.21.15.12:0/2500362275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faccc103150 msgr2=0x7faccc198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:51.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.722+0000 7faccad9d640 1 --2- 172.21.15.12:0/2500362275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faccc103150 0x7faccc198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.722+0000 7faccad9d640 1 -- 172.21.15.12:0/2500362275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faccc103a70 msgr2=0x7faccc198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:51.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.722+0000 7faccad9d640 1 --2- 172.21.15.12:0/2500362275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faccc103a70 0x7faccc198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.722+0000 7faccad9d640 1 -- 172.21.15.12:0/2500362275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faccc19d730 con 0x7faccc10c520 2024-04-22T13:25:51.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.722+0000 7faccad9d640 1 --2- 172.21.15.12:0/2500362275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faccc10c520 0x7faccc19d000 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7facc4009fd0 tx=0x7facc400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:51.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.722+0000 7facc37fe640 1 -- 172.21.15.12:0/2500362275 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc4017070 con 0x7faccc10c520 2024-04-22T13:25:51.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.722+0000 7facc37fe640 1 -- 172.21.15.12:0/2500362275 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facc4003e20 con 0x7faccc10c520 2024-04-22T13:25:51.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.722+0000 7facd0ed9640 1 -- 172.21.15.12:0/2500362275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faccc19da20 con 0x7faccc10c520 2024-04-22T13:25:51.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.722+0000 7facc37fe640 1 -- 172.21.15.12:0/2500362275 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc4010460 con 0x7faccc10c520 2024-04-22T13:25:51.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.722+0000 7facd0ed9640 1 -- 172.21.15.12:0/2500362275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faccc1a52b0 con 0x7faccc10c520 2024-04-22T13:25:51.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.724+0000 7facc37fe640 1 -- 172.21.15.12:0/2500362275 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7facc4003970 con 0x7faccc10c520 2024-04-22T13:25:51.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.725+0000 7facd3ba9740 1 Processor -- start 2024-04-22T13:25:51.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.725+0000 7facd3ba9740 1 -- start start 2024-04-22T13:25:51.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.725+0000 7facd3ba9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560d38846580 0x560d38846960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.725+0000 7facd3ba9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d38846ea0 0x560d3884f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.726+0000 7facd3ba9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d3884fa80 0x560d38851e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.726+0000 7facd3ba9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560d387ab1e0 con 0x560d3884fa80 2024-04-22T13:25:51.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.726+0000 7facd3ba9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560d38748390 con 0x560d38846580 2024-04-22T13:25:51.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.726+0000 7facd3ba9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560d387a71a0 con 0x560d38846ea0 2024-04-22T13:25:51.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.726+0000 7facc37fe640 1 --2- 172.21.15.12:0/2500362275 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7facb80a3d20 0x7facb80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.726+0000 7facc3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d3884fa80 0x560d38851e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.726+0000 7facc3fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560d38846580 0x560d38846960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.726+0000 7facc3fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560d38846580 0x560d38846960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:45626/0 (socket says 172.21.15.12:45626) 2024-04-22T13:25:51.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.726+0000 7facc3fff640 1 -- 172.21.15.12:0/1307952709 learned_addr learned my addr 172.21.15.12:0/1307952709 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:51.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.726+0000 7facca59c640 1 --2- 172.21.15.12:0/1307952709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d38846ea0 0x560d3884f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.726+0000 7faccad9d640 1 --2- 172.21.15.12:0/2500362275 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7facb80a3d20 0x7facb80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.726+0000 7facc3fff640 1 -- 172.21.15.12:0/1307952709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d38846ea0 msgr2=0x560d3884f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:51.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.726+0000 7facc3fff640 1 --2- 172.21.15.12:0/1307952709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d38846ea0 0x560d3884f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.726+0000 7facc37fe640 1 -- 172.21.15.12:0/2500362275 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7facc40d04d0 con 0x7faccc10c520 2024-04-22T13:25:51.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.726+0000 7facc3fff640 1 -- 172.21.15.12:0/1307952709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d3884fa80 msgr2=0x560d38851e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:51.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.726+0000 7facc3fff640 1 --2- 172.21.15.12:0/1307952709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d3884fa80 0x560d38851e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.726+0000 7facc3fff640 1 -- 172.21.15.12:0/1307952709 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d385877d0 con 0x560d38846580 2024-04-22T13:25:51.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.726+0000 7facc3fff640 1 --2- 172.21.15.12:0/1307952709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560d38846580 0x560d38846960 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7facb4014a40 tx=0x7facb40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d4a2a274b04c264 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:51.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.727+0000 7facc0ff9640 1 -- 172.21.15.12:0/1307952709 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb40142d0 con 0x560d38846580 2024-04-22T13:25:51.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.727+0000 7faccad9d640 1 --2- 172.21.15.12:0/2500362275 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7facb80a3d20 0x7facb80a61e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7facc40cdd20 tx=0x7facc4067010 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:51.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.727+0000 7facc0ff9640 1 -- 172.21.15.12:0/1307952709 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facb4011d50 con 0x560d38846580 2024-04-22T13:25:51.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.727+0000 7facc0ff9640 1 -- 172.21.15.12:0/1307952709 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb4004a00 con 0x560d38846580 2024-04-22T13:25:51.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.727+0000 7facd3ba9740 1 -- 172.21.15.12:0/1307952709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560d38846580 msgr2=0x560d38846960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:51.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.727+0000 7facd3ba9740 1 --2- 172.21.15.12:0/1307952709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560d38846580 0x560d38846960 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7facb4014a40 tx=0x7facb40046c0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.727+0000 7facd3ba9740 1 -- 172.21.15.12:0/1307952709 shutdown_connections 2024-04-22T13:25:51.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.727+0000 7facd3ba9740 1 --2- 172.21.15.12:0/1307952709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d3884fa80 0x560d38851e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.727+0000 7facd3ba9740 1 --2- 172.21.15.12:0/1307952709 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d38846ea0 0x560d3884f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.727+0000 7facd3ba9740 1 --2- 172.21.15.12:0/1307952709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560d38846580 0x560d38846960 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.727+0000 7facd3ba9740 1 -- 172.21.15.12:0/1307952709 >> 172.21.15.12:0/1307952709 conn(0x560d38840cb0 msgr2=0x560d38841090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:51.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.728+0000 7facd3ba9740 1 -- 172.21.15.12:0/1307952709 shutdown_connections 2024-04-22T13:25:51.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.728+0000 7facd3ba9740 1 -- 172.21.15.12:0/1307952709 wait complete. 2024-04-22T13:25:51.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.728+0000 7facd3ba9740 1 -- start start 2024-04-22T13:25:51.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.728+0000 7facd3ba9740 1 Processor -- start 2024-04-22T13:25:51.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.728+0000 7facd3ba9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d38846580 0x560d3884cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.728+0000 7facd3ba9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d38846ea0 0x560d3884d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.728+0000 7facd3ba9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560d3884fa80 0x560d3884d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.728+0000 7facd3ba9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560d38852870 con 0x560d38846580 2024-04-22T13:25:51.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.728+0000 7facd3ba9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560d387642f0 con 0x560d3884fa80 2024-04-22T13:25:51.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.728+0000 7facd3ba9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560d388529f0 con 0x560d38846ea0 2024-04-22T13:25:51.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.728+0000 7facd3ba9740 10 client.0 _ll_register_callbacks cb 0x560d38771580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:51.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.728+0000 7facc3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d38846580 0x560d3884cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.728+0000 7facc3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d38846580 0x560d3884cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58196/0 (socket says 172.21.15.12:58196) 2024-04-22T13:25:51.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.728+0000 7facc3fff640 1 -- 172.21.15.12:0/522030637 learned_addr learned my addr 172.21.15.12:0/522030637 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:51.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.728+0000 7facca59c640 1 --2- 172.21.15.12:0/522030637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d38846ea0 0x560d3884d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.729+0000 7facc3fff640 1 --2- 172.21.15.12:0/522030637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560d3884fa80 0x560d3884d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.729+0000 7facc3fff640 1 -- 172.21.15.12:0/522030637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d38846ea0 msgr2=0x560d3884d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:51.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.729+0000 7facc3fff640 1 --2- 172.21.15.12:0/522030637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d38846ea0 0x560d3884d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.729+0000 7facc3fff640 1 -- 172.21.15.12:0/522030637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560d3884fa80 msgr2=0x560d3884d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:51.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.729+0000 7facc3fff640 1 --2- 172.21.15.12:0/522030637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560d3884fa80 0x560d3884d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.729+0000 7facc3fff640 1 -- 172.21.15.12:0/522030637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d384bd3f0 con 0x560d38846580 2024-04-22T13:25:51.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.729+0000 7facc3fff640 1 --2- 172.21.15.12:0/522030637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d38846580 0x560d3884cae0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7facb4005d00 tx=0x7facb4005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59cb59c803baae84 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:51.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.729+0000 7facb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:51.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.729+0000 7facb1ffb640 1 -- 172.21.15.12:0/522030637 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb4011aa0 con 0x560d38846580 2024-04-22T13:25:51.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.729+0000 7facb1ffb640 1 -- 172.21.15.12:0/522030637 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7facb4012450 con 0x560d38846580 2024-04-22T13:25:51.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.729+0000 7facd3ba9740 1 -- 172.21.15.12:0/522030637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facb4011d50 con 0x560d38846580 2024-04-22T13:25:51.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.729+0000 7facb1ffb640 1 -- 172.21.15.12:0/522030637 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7facb4012650 con 0x560d38846580 2024-04-22T13:25:51.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.729+0000 7facd3ba9740 10 client.16473 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:51.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.729+0000 7facd3ba9740 1 -- 172.21.15.12:0/522030637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560d3847d0f0 con 0x560d38846580 2024-04-22T13:25:51.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.730+0000 7facb1ffb640 1 -- 172.21.15.12:0/522030637 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facb4011d50 con 0x560d38846580 2024-04-22T13:25:51.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.730+0000 7facb1ffb640 1 -- 172.21.15.12:0/522030637 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7facb401f400 con 0x560d38846580 2024-04-22T13:25:51.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.730+0000 7facd3ba9740 10 client.16473 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:51.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.730+0000 7facd3ba9740 10 client.16473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:25:51.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.730+0000 7facd3ba9740 20 client.16473 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:51.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.730+0000 7facd3ba9740 1 --2- 172.21.15.12:0/522030637 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560d3885f930 0x560d38861d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.730+0000 7facd3ba9740 4 client.16473 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:25:51.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.730+0000 7facd3ba9740 1 -- 172.21.15.12:0/522030637 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d387b8c00 con 0x560d3885f930 2024-04-22T13:25:51.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.730+0000 7facca59c640 1 --2- 172.21.15.12:0/522030637 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560d3885f930 0x560d38861d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.730+0000 7facca59c640 1 --2- 172.21.15.12:0/522030637 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560d3885f930 0x560d38861d20 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7bbdd76fea5572d server_cookie=850b24e9a91e7c02 in_seq=0 out_seq=0 2024-04-22T13:25:51.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.730+0000 7facb1ffb640 10 client.16473 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:51.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.731+0000 7facb1ffb640 1 -- 172.21.15.12:0/522030637 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560d387b8c00 con 0x560d3885f930 2024-04-22T13:25:51.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.731+0000 7facb1ffb640 10 client.16473 handle_command_reply: tid=0 2024-04-22T13:25:51.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.731+0000 7facd3ba9740 1 client.16473 shutdown 2024-04-22T13:25:51.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.731+0000 7facd3ba9740 1 -- 172.21.15.12:0/522030637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d38846580 msgr2=0x560d3884cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:51.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.731+0000 7facd3ba9740 1 --2- 172.21.15.12:0/522030637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d38846580 0x560d3884cae0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7facb4005d00 tx=0x7facb4005c50 comp rx=0 tx=0).stop 2024-04-22T13:25:51.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.731+0000 7facd3ba9740 1 -- 172.21.15.12:0/522030637 shutdown_connections 2024-04-22T13:25:51.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.731+0000 7facd3ba9740 1 --2- 172.21.15.12:0/522030637 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560d3885f930 0x560d38861d20 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.731+0000 7facd3ba9740 1 --2- 172.21.15.12:0/522030637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560d3884fa80 0x560d3884d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.731+0000 7facd3ba9740 1 --2- 172.21.15.12:0/522030637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d38846ea0 0x560d3884d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.731+0000 7facb1ffb640 0 client.16473 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:51.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.731+0000 7facd3ba9740 1 --2- 172.21.15.12:0/522030637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d38846580 0x560d3884cae0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.731+0000 7facd3ba9740 1 -- 172.21.15.12:0/522030637 >> 172.21.15.12:0/522030637 conn(0x560d38840cb0 msgr2=0x560d38842a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:51.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.731+0000 7facd3ba9740 1 -- 172.21.15.12:0/522030637 shutdown_connections 2024-04-22T13:25:51.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.731+0000 7facd3ba9740 1 -- 172.21.15.12:0/522030637 wait complete. 2024-04-22T13:25:51.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.732+0000 7facd3ba9740 20 client.16473 trim_cache size 0 max 16384 2024-04-22T13:25:51.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.750+0000 7facd3ba9740 1 Processor -- start 2024-04-22T13:25:51.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.750+0000 7facd3ba9740 1 -- start start 2024-04-22T13:25:51.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.750+0000 7facd3ba9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d38846580 0x560d3884f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.750+0000 7facd3ba9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560d38846ea0 0x560d3885b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.750+0000 7facd3ba9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d3884fa80 0x560d3884bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.750+0000 7facd3ba9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7faccc1adaf0 con 0x560d3884fa80 2024-04-22T13:25:51.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.750+0000 7facd3ba9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faccc1af630 con 0x560d38846ea0 2024-04-22T13:25:51.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.750+0000 7facd3ba9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faccc1ae1c0 con 0x560d38846580 2024-04-22T13:25:51.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.750+0000 7facc3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d3884fa80 0x560d3884bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.750+0000 7facc3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d3884fa80 0x560d3884bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58212/0 (socket says 172.21.15.12:58212) 2024-04-22T13:25:51.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.750+0000 7facc3fff640 1 -- 172.21.15.12:0/1779694593 learned_addr learned my addr 172.21.15.12:0/1779694593 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:51.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.750+0000 7facc3fff640 1 --2- 172.21.15.12:0/1779694593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d38846580 0x560d3884f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.750+0000 7facca59c640 1 --2- 172.21.15.12:0/1779694593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560d38846ea0 0x560d3885b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.750+0000 7facc3fff640 1 -- 172.21.15.12:0/1779694593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d38846580 msgr2=0x560d3884f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:51.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.750+0000 7facc3fff640 1 --2- 172.21.15.12:0/1779694593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d38846580 0x560d3884f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.750+0000 7facc3fff640 1 -- 172.21.15.12:0/1779694593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560d38846ea0 msgr2=0x560d3885b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:51.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.750+0000 7facc3fff640 1 --2- 172.21.15.12:0/1779694593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560d38846ea0 0x560d3885b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.750+0000 7facc3fff640 1 -- 172.21.15.12:0/1779694593 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d38862380 con 0x560d3884fa80 2024-04-22T13:25:51.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.751+0000 7facc3fff640 1 --2- 172.21.15.12:0/1779694593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d3884fa80 0x560d3884bc40 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7facb40059f0 tx=0x7facb4011fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68927d8e3099a88e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:51.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.751+0000 7facb3fff640 1 -- 172.21.15.12:0/1779694593 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb401f400 con 0x560d3884fa80 2024-04-22T13:25:51.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.751+0000 7facb3fff640 1 -- 172.21.15.12:0/1779694593 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facb401f5a0 con 0x560d3884fa80 2024-04-22T13:25:51.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.751+0000 7facb3fff640 1 -- 172.21.15.12:0/1779694593 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb4005070 con 0x560d3884fa80 2024-04-22T13:25:51.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.752+0000 7facd3ba9740 1 -- 172.21.15.12:0/1779694593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d3884fa80 msgr2=0x560d3884bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:51.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.752+0000 7facd3ba9740 1 --2- 172.21.15.12:0/1779694593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d3884fa80 0x560d3884bc40 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7facb40059f0 tx=0x7facb4011fa0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.752+0000 7facd3ba9740 1 -- 172.21.15.12:0/1779694593 shutdown_connections 2024-04-22T13:25:51.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.752+0000 7facd3ba9740 1 --2- 172.21.15.12:0/1779694593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d3884fa80 0x560d3884bc40 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.752+0000 7facd3ba9740 1 --2- 172.21.15.12:0/1779694593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560d38846ea0 0x560d3885b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.752+0000 7facd3ba9740 1 --2- 172.21.15.12:0/1779694593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d38846580 0x560d3884f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.752+0000 7facd3ba9740 1 -- 172.21.15.12:0/1779694593 >> 172.21.15.12:0/1779694593 conn(0x560d38840cb0 msgr2=0x560d3884caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.752+0000 7facd3ba9740 1 -- 172.21.15.12:0/1779694593 shutdown_connections 2024-04-22T13:25:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.752+0000 7facd3ba9740 1 -- 172.21.15.12:0/1779694593 wait complete. 2024-04-22T13:25:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.752+0000 7facd3ba9740 1 -- start start 2024-04-22T13:25:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.752+0000 7facd3ba9740 1 Processor -- start 2024-04-22T13:25:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.752+0000 7facd3ba9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560d38846580 0x560d38852dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.752+0000 7facd3ba9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d38846ea0 0x560d38853300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.752+0000 7facd3ba9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d3884fa80 0x560d388c3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.752+0000 7facd3ba9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7faccc1aef60 con 0x560d38846ea0 2024-04-22T13:25:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.752+0000 7facd3ba9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faccc1ae890 con 0x560d38846580 2024-04-22T13:25:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.752+0000 7facd3ba9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faccc1abad0 con 0x560d3884fa80 2024-04-22T13:25:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.752+0000 7facd3ba9740 10 client.0 _ll_register_callbacks cb 0x7facb4014470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:25:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.752+0000 7facca59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d38846ea0 0x560d38853300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.753+0000 7facca59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d38846ea0 0x560d38853300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58226/0 (socket says 172.21.15.12:58226) 2024-04-22T13:25:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.753+0000 7facca59c640 1 -- 172.21.15.12:0/135906245 learned_addr learned my addr 172.21.15.12:0/135906245 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:25:51.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.753+0000 7facc3fff640 1 --2- 172.21.15.12:0/135906245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d3884fa80 0x560d388c3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.753+0000 7facc3fff640 1 --2- 172.21.15.12:0/135906245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560d38846580 0x560d38852dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.753+0000 7facca59c640 1 -- 172.21.15.12:0/135906245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d3884fa80 msgr2=0x560d388c3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:51.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.753+0000 7facca59c640 1 --2- 172.21.15.12:0/135906245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d3884fa80 0x560d388c3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.753+0000 7facca59c640 1 -- 172.21.15.12:0/135906245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560d38846580 msgr2=0x560d38852dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:25:51.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.753+0000 7facca59c640 1 --2- 172.21.15.12:0/135906245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560d38846580 0x560d38852dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.753+0000 7facca59c640 1 -- 172.21.15.12:0/135906245 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d388c4350 con 0x560d38846ea0 2024-04-22T13:25:51.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.753+0000 7facc3fff640 1 --2- 172.21.15.12:0/135906245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560d38846580 0x560d38852dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:25:51.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.753+0000 7facca59c640 1 --2- 172.21.15.12:0/135906245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d38846ea0 0x560d38853300 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7facbc009900 tx=0x7facbc004420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b14d2919bc6020c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:25:51.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.753+0000 7facb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:25:51.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.753+0000 7facb1ffb640 1 -- 172.21.15.12:0/135906245 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc051060 con 0x560d38846ea0 2024-04-22T13:25:51.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.753+0000 7facb1ffb640 1 -- 172.21.15.12:0/135906245 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7facbc02ca90 con 0x560d38846ea0 2024-04-22T13:25:51.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.753+0000 7facd3ba9740 1 -- 172.21.15.12:0/135906245 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facb401f5a0 con 0x560d38846ea0 2024-04-22T13:25:51.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.753+0000 7facb1ffb640 1 -- 172.21.15.12:0/135906245 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7facbc02cc90 con 0x560d38846ea0 2024-04-22T13:25:51.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.753+0000 7facd3ba9740 10 client.16479 fetch_fsmap learned FSMap version 28 2024-04-22T13:25:51.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.753+0000 7facd3ba9740 1 -- 172.21.15.12:0/135906245 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560d388c56d0 con 0x560d38846ea0 2024-04-22T13:25:51.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.754+0000 7facb1ffb640 1 -- 172.21.15.12:0/135906245 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facb401f5a0 con 0x560d38846ea0 2024-04-22T13:25:51.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.754+0000 7facb1ffb640 1 -- 172.21.15.12:0/135906245 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7facbc0049d0 con 0x560d38846ea0 2024-04-22T13:25:51.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.754+0000 7facd3ba9740 10 client.16479 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:25:51.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.754+0000 7facd3ba9740 10 client.16479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:25:51.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.754+0000 7facd3ba9740 20 client.16479 populate_metadata read hostname 'smithi012' 2024-04-22T13:25:51.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.754+0000 7facd3ba9740 1 --2- 172.21.15.12:0/135906245 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560d388c5eb0 0x560d388c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:25:51.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.754+0000 7facd3ba9740 4 client.16479 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:25:51.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.754+0000 7facd3ba9740 1 -- 172.21.15.12:0/135906245 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560d387b2a70 con 0x560d388c5eb0 2024-04-22T13:25:51.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.754+0000 7facc3fff640 1 --2- 172.21.15.12:0/135906245 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560d388c5eb0 0x560d388c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:25:51.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.754+0000 7facb1ffb640 10 client.16479 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:51.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.754+0000 7facc3fff640 1 --2- 172.21.15.12:0/135906245 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560d388c5eb0 0x560d388c82d0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1d8f248acb48a32 server_cookie=2d33ce2d937b9351 in_seq=0 out_seq=0 2024-04-22T13:25:51.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.755+0000 7facb1ffb640 1 -- 172.21.15.12:0/135906245 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560d387b2a70 con 0x560d388c5eb0 2024-04-22T13:25:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.755+0000 7facb1ffb640 10 client.16479 handle_command_reply: tid=0 2024-04-22T13:25:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.755+0000 7facd3ba9740 1 client.16479 shutdown 2024-04-22T13:25:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.755+0000 7facd3ba9740 1 -- 172.21.15.12:0/135906245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d38846ea0 msgr2=0x560d38853300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.755+0000 7facd3ba9740 1 --2- 172.21.15.12:0/135906245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d38846ea0 0x560d38853300 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7facbc009900 tx=0x7facbc004420 comp rx=0 tx=0).stop 2024-04-22T13:25:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.755+0000 7facd3ba9740 1 -- 172.21.15.12:0/135906245 shutdown_connections 2024-04-22T13:25:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.755+0000 7facd3ba9740 1 --2- 172.21.15.12:0/135906245 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560d388c5eb0 0x560d388c82d0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.755+0000 7facd3ba9740 1 --2- 172.21.15.12:0/135906245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560d3884fa80 0x560d388c3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.755+0000 7facd3ba9740 1 --2- 172.21.15.12:0/135906245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560d38846ea0 0x560d38853300 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.755+0000 7facb1ffb640 0 client.16479 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:25:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.755+0000 7facd3ba9740 1 --2- 172.21.15.12:0/135906245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560d38846580 0x560d38852dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.755+0000 7facd3ba9740 1 -- 172.21.15.12:0/135906245 >> 172.21.15.12:0/135906245 conn(0x560d38840cb0 msgr2=0x560d3884caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.755+0000 7facd3ba9740 1 -- 172.21.15.12:0/135906245 shutdown_connections 2024-04-22T13:25:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.755+0000 7facd3ba9740 1 -- 172.21.15.12:0/135906245 wait complete. 2024-04-22T13:25:51.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.755+0000 7facd3ba9740 20 client.16479 trim_cache size 0 max 16384 2024-04-22T13:25:51.758 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:25:51.758 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:25:51.758 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-04-22T13:25:51.758 INFO:teuthology.orchestra.run.smithi012.stdout: "6628281c-e6d8-4da8-af0a-dd5be23738bf": { 2024-04-22T13:25:51.758 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-04-22T13:25:51.758 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "6628281c-e6d8-4da8-af0a-dd5be23738bf", 2024-04-22T13:25:51.758 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-04-22T13:25:51.758 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:25:51.758 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:25:51.758 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:25:51.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.755+0000 7facd0ed9640 1 -- 172.21.15.12:0/2500362275 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7facb80a3d20 msgr2=0x7facb80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:51.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.755+0000 7facd0ed9640 1 --2- 172.21.15.12:0/2500362275 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7facb80a3d20 0x7facb80a61e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7facc40cdd20 tx=0x7facc4067010 comp rx=0 tx=0).stop 2024-04-22T13:25:51.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.755+0000 7facd0ed9640 1 -- 172.21.15.12:0/2500362275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faccc10c520 msgr2=0x7faccc19d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:25:51.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.755+0000 7facd0ed9640 1 --2- 172.21.15.12:0/2500362275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faccc10c520 0x7faccc19d000 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7facc4009fd0 tx=0x7facc400b260 comp rx=0 tx=0).stop 2024-04-22T13:25:51.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.756+0000 7facd0ed9640 1 -- 172.21.15.12:0/2500362275 shutdown_connections 2024-04-22T13:25:51.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.756+0000 7facd0ed9640 1 --2- 172.21.15.12:0/2500362275 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7facb80a3d20 0x7facb80a61e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.756+0000 7facd0ed9640 1 --2- 172.21.15.12:0/2500362275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faccc10c520 0x7faccc19d000 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.756+0000 7facd0ed9640 1 --2- 172.21.15.12:0/2500362275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faccc103a70 0x7faccc198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.756+0000 7facd0ed9640 1 --2- 172.21.15.12:0/2500362275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faccc103150 0x7faccc198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:25:51.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.756+0000 7facd0ed9640 1 -- 172.21.15.12:0/2500362275 >> 172.21.15.12:0/2500362275 conn(0x7faccc077c00 msgr2=0x7faccc0fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:25:51.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.756+0000 7facd0ed9640 1 -- 172.21.15.12:0/2500362275 shutdown_connections 2024-04-22T13:25:51.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:25:51.756+0000 7facd0ed9640 1 -- 172.21.15.12:0/2500362275 wait complete. 2024-04-22T13:25:51.763 INFO:tasks.cephfs.filesystem:scrub status for tag:6628281c-e6d8-4da8-af0a-dd5be23738bf - {'path': '/', 'tag': '6628281c-e6d8-4da8-af0a-dd5be23738bf', 'options': 'recursive,force'} 2024-04-22T13:25:52.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:51 smithi012 ceph-mon[28265]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-22T13:25:52.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:51 smithi134 ceph-mon[31338]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-22T13:25:52.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:51 smithi111 ceph-mon[31361]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-22T13:25:53.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:52 smithi012 ceph-mon[28265]: scrub queued for path: / 2024-04-22T13:25:53.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:52 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:25:53.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:52 smithi012 ceph-mon[28265]: scrub summary: active paths [/] 2024-04-22T13:25:53.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:52 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:25:53.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:52 smithi134 ceph-mon[31338]: scrub queued for path: / 2024-04-22T13:25:53.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:52 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:25:53.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:52 smithi134 ceph-mon[31338]: scrub summary: active paths [/] 2024-04-22T13:25:53.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:52 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:25:53.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:52 smithi111 ceph-mon[31361]: scrub queued for path: / 2024-04-22T13:25:53.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:52 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:25:53.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:52 smithi111 ceph-mon[31361]: scrub summary: active paths [/] 2024-04-22T13:25:53.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:52 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:25:54.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:53 smithi012 ceph-mon[28265]: pgmap v56: 97 pgs: 97 active+clean; 2.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 435 KiB/s wr, 16 op/s 2024-04-22T13:25:54.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:53 smithi134 ceph-mon[31338]: pgmap v56: 97 pgs: 97 active+clean; 2.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 435 KiB/s wr, 16 op/s 2024-04-22T13:25:54.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:53 smithi111 ceph-mon[31361]: pgmap v56: 97 pgs: 97 active+clean; 2.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 435 KiB/s wr, 16 op/s 2024-04-22T13:25:56.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:55 smithi012 ceph-mon[28265]: pgmap v57: 97 pgs: 97 active+clean; 2.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 435 KiB/s wr, 12 op/s 2024-04-22T13:25:56.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:55 smithi012 ceph-mon[28265]: scrub summary: idle 2024-04-22T13:25:56.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:55 smithi134 ceph-mon[31338]: pgmap v57: 97 pgs: 97 active+clean; 2.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 435 KiB/s wr, 12 op/s 2024-04-22T13:25:56.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:55 smithi134 ceph-mon[31338]: scrub summary: idle 2024-04-22T13:25:56.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:55 smithi111 ceph-mon[31361]: pgmap v57: 97 pgs: 97 active+clean; 2.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 435 KiB/s wr, 12 op/s 2024-04-22T13:25:56.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:55 smithi111 ceph-mon[31361]: scrub summary: idle 2024-04-22T13:25:58.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:57 smithi012 ceph-mon[28265]: pgmap v58: 97 pgs: 97 active+clean; 2.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 435 KiB/s wr, 12 op/s 2024-04-22T13:25:58.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:57 smithi134 ceph-mon[31338]: pgmap v58: 97 pgs: 97 active+clean; 2.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 435 KiB/s wr, 12 op/s 2024-04-22T13:25:58.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:57 smithi111 ceph-mon[31361]: pgmap v58: 97 pgs: 97 active+clean; 2.4 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 435 KiB/s wr, 12 op/s 2024-04-22T13:26:00.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:25:59 smithi012 ceph-mon[28265]: pgmap v59: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 866 KiB/s wr, 17 op/s 2024-04-22T13:26:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.114+0000 7fab1e2e0640 1 -- 172.21.15.12:0/1932259685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab181010f0 msgr2=0x7fab181014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.114+0000 7fab1e2e0640 1 --2- 172.21.15.12:0/1932259685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab181010f0 0x7fab181014d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fab08009900 tx=0x7fab0802bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.114+0000 7fab1e2e0640 1 -- 172.21.15.12:0/1932259685 shutdown_connections 2024-04-22T13:26:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.114+0000 7fab1e2e0640 1 --2- 172.21.15.12:0/1932259685 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab18106170 0x7fab1810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.114+0000 7fab1e2e0640 1 --2- 172.21.15.12:0/1932259685 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab18101aa0 0x7fab18105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.114+0000 7fab1e2e0640 1 --2- 172.21.15.12:0/1932259685 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab181010f0 0x7fab181014d0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.114+0000 7fab1e2e0640 1 -- 172.21.15.12:0/1932259685 >> 172.21.15.12:0/1932259685 conn(0x7fab18077fd0 msgr2=0x7fab180783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.114+0000 7fab1e2e0640 1 -- 172.21.15.12:0/1932259685 shutdown_connections 2024-04-22T13:26:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.114+0000 7fab1e2e0640 1 -- 172.21.15.12:0/1932259685 wait complete. 2024-04-22T13:26:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.114+0000 7fab1e2e0640 1 Processor -- start 2024-04-22T13:26:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.115+0000 7fab1e2e0640 1 -- start start 2024-04-22T13:26:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.115+0000 7fab1e2e0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab181010f0 0x7fab18102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.115+0000 7fab1e2e0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab18106170 0x7fab18103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.115+0000 7fab1e2e0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab181035b0 0x7fab181a6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.115+0000 7fab1e2e0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fab181110b0 con 0x7fab181035b0 2024-04-22T13:26:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.115+0000 7fab1e2e0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fab18110f30 con 0x7fab181010f0 2024-04-22T13:26:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.115+0000 7fab1e2e0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fab18111230 con 0x7fab18106170 2024-04-22T13:26:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.115+0000 7fab1c87d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab181035b0 0x7fab181a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.115+0000 7fab17fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab181010f0 0x7fab18102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.115+0000 7fab1c87d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab181035b0 0x7fab181a6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58246/0 (socket says 172.21.15.12:58246) 2024-04-22T13:26:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.115+0000 7fab1c87d640 1 -- 172.21.15.12:0/2824513265 learned_addr learned my addr 172.21.15.12:0/2824513265 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.115+0000 7fab177fe640 1 --2- 172.21.15.12:0/2824513265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab18106170 0x7fab18103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.116+0000 7fab1c87d640 1 -- 172.21.15.12:0/2824513265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab18106170 msgr2=0x7fab18103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.116+0000 7fab1c87d640 1 --2- 172.21.15.12:0/2824513265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab18106170 0x7fab18103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.116+0000 7fab1c87d640 1 -- 172.21.15.12:0/2824513265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab181010f0 msgr2=0x7fab18102b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.116+0000 7fab1c87d640 1 --2- 172.21.15.12:0/2824513265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab181010f0 0x7fab18102b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.116+0000 7fab1c87d640 1 -- 172.21.15.12:0/2824513265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab181a6f60 con 0x7fab181035b0 2024-04-22T13:26:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.116+0000 7fab17fff640 1 --2- 172.21.15.12:0/2824513265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab181010f0 0x7fab18102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.116+0000 7fab177fe640 1 --2- 172.21.15.12:0/2824513265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab18106170 0x7fab18103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.116+0000 7fab1c87d640 1 --2- 172.21.15.12:0/2824513265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab181035b0 0x7fab181a6860 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fab1000eaa0 tx=0x7fab1000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.116+0000 7fab157fa640 1 -- 172.21.15.12:0/2824513265 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1000cdb0 con 0x7fab181035b0 2024-04-22T13:26:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.116+0000 7fab157fa640 1 -- 172.21.15.12:0/2824513265 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab10005020 con 0x7fab181035b0 2024-04-22T13:26:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.116+0000 7fab1e2e0640 1 -- 172.21.15.12:0/2824513265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab181a7250 con 0x7fab181035b0 2024-04-22T13:26:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.116+0000 7fab157fa640 1 -- 172.21.15.12:0/2824513265 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab100116d0 con 0x7fab181035b0 2024-04-22T13:26:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.116+0000 7fab1e2e0640 1 -- 172.21.15.12:0/2824513265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab181a7750 con 0x7fab181035b0 2024-04-22T13:26:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.118+0000 7fab157fa640 1 -- 172.21.15.12:0/2824513265 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fab100051d0 con 0x7fab181035b0 2024-04-22T13:26:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.119+0000 7fab20fab740 1 Processor -- start 2024-04-22T13:26:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.119+0000 7fab20fab740 1 -- start start 2024-04-22T13:26:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.119+0000 7fab20fab740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602472038d0 0x560247202620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.119+0000 7fab20fab740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560247202b60 0x560247202fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.119+0000 7fab20fab740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56024720c0e0 0x56024720e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.119+0000 7fab20fab740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5602471678d0 con 0x5602472038d0 2024-04-22T13:26:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.119+0000 7fab20fab740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560247104e80 con 0x56024720c0e0 2024-04-22T13:26:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.119+0000 7fab20fab740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560247163890 con 0x560247202b60 2024-04-22T13:26:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.119+0000 7fab177fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602472038d0 0x560247202620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.119+0000 7fab177fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602472038d0 0x560247202620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58260/0 (socket says 172.21.15.12:58260) 2024-04-22T13:26:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.119+0000 7fab17fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560247202b60 0x560247202fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.119+0000 7fab177fe640 1 -- 172.21.15.12:0/1073998602 learned_addr learned my addr 172.21.15.12:0/1073998602 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.119+0000 7fab157fa640 1 --2- 172.21.15.12:0/2824513265 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fab040a3df0 0x7fab040a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.119+0000 7fab177fe640 1 --2- 172.21.15.12:0/1073998602 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56024720c0e0 0x56024720e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.119+0000 7fab1c87d640 1 --2- 172.21.15.12:0/2824513265 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fab040a3df0 0x7fab040a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.120+0000 7fab17fff640 1 -- 172.21.15.12:0/1073998602 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56024720c0e0 msgr2=0x56024720e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.120+0000 7fab17fff640 1 --2- 172.21.15.12:0/1073998602 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56024720c0e0 0x56024720e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.120+0000 7fab17fff640 1 -- 172.21.15.12:0/1073998602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602472038d0 msgr2=0x560247202620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.120+0000 7fab17fff640 1 --2- 172.21.15.12:0/1073998602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602472038d0 0x560247202620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.120+0000 7fab17fff640 1 -- 172.21.15.12:0/1073998602 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560247126320 con 0x560247202b60 2024-04-22T13:26:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.120+0000 7fab157fa640 1 -- 172.21.15.12:0/2824513265 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fab100d08b0 con 0x7fab181035b0 2024-04-22T13:26:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.120+0000 7fab17fff640 1 --2- 172.21.15.12:0/1073998602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560247202b60 0x560247202fe0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fab08000c00 tx=0x7fab0802c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23a1e6dd25f07abb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.120+0000 7fab1c87d640 1 --2- 172.21.15.12:0/2824513265 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fab040a3df0 0x7fab040a62b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fab10098d50 tx=0x7fab1006f3e0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.120+0000 7fab0e7fc640 1 -- 172.21.15.12:0/1073998602 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab08045070 con 0x560247202b60 2024-04-22T13:26:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.120+0000 7fab0e7fc640 1 -- 172.21.15.12:0/1073998602 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab0802c7e0 con 0x560247202b60 2024-04-22T13:26:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.120+0000 7fab0e7fc640 1 -- 172.21.15.12:0/1073998602 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab08003b60 con 0x560247202b60 2024-04-22T13:26:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.121+0000 7fab20fab740 1 -- 172.21.15.12:0/1073998602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560247202b60 msgr2=0x560247202fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.121+0000 7fab20fab740 1 --2- 172.21.15.12:0/1073998602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560247202b60 0x560247202fe0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fab08000c00 tx=0x7fab0802c6a0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.121+0000 7fab20fab740 1 -- 172.21.15.12:0/1073998602 shutdown_connections 2024-04-22T13:26:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.121+0000 7fab20fab740 1 --2- 172.21.15.12:0/1073998602 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56024720c0e0 0x56024720e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.121+0000 7fab20fab740 1 --2- 172.21.15.12:0/1073998602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560247202b60 0x560247202fe0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.121+0000 7fab20fab740 1 --2- 172.21.15.12:0/1073998602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5602472038d0 0x560247202620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.121+0000 7fab20fab740 1 -- 172.21.15.12:0/1073998602 >> 172.21.15.12:0/1073998602 conn(0x5602471c9c90 msgr2=0x5602471fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.121+0000 7fab20fab740 1 -- 172.21.15.12:0/1073998602 shutdown_connections 2024-04-22T13:26:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.121+0000 7fab20fab740 1 -- 172.21.15.12:0/1073998602 wait complete. 2024-04-22T13:26:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.121+0000 7fab20fab740 1 -- start start 2024-04-22T13:26:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.122+0000 7fab20fab740 1 Processor -- start 2024-04-22T13:26:00.124 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.122+0000 7fab20fab740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560247202b60 0x56024720ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.122+0000 7fab20fab740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602472038d0 0x560247208e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.122+0000 7fab20fab740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56024720c0e0 0x5602472093b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.122+0000 7fab20fab740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56024720edb0 con 0x560247202b60 2024-04-22T13:26:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.122+0000 7fab20fab740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5602470efeb0 con 0x56024720c0e0 2024-04-22T13:26:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.122+0000 7fab20fab740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56024720ef30 con 0x5602472038d0 2024-04-22T13:26:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.122+0000 7fab20fab740 10 client.0 _ll_register_callbacks cb 0x56024711cb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.122+0000 7fab177fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560247202b60 0x56024720ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.122+0000 7fab17fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602472038d0 0x560247208e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.122+0000 7fab17fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602472038d0 0x560247208e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.12:52736/0 (socket says 172.21.15.12:52736) 2024-04-22T13:26:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.122+0000 7fab177fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560247202b60 0x56024720ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58274/0 (socket says 172.21.15.12:58274) 2024-04-22T13:26:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.122+0000 7fab17fff640 1 -- 172.21.15.12:0/263605223 learned_addr learned my addr 172.21.15.12:0/263605223 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.122+0000 7fab177fe640 1 --2- 172.21.15.12:0/263605223 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56024720c0e0 0x5602472093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.122+0000 7fab177fe640 1 -- 172.21.15.12:0/263605223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602472038d0 msgr2=0x560247208e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.122+0000 7fab177fe640 1 --2- 172.21.15.12:0/263605223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602472038d0 0x560247208e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.125 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.122+0000 7fab177fe640 1 -- 172.21.15.12:0/263605223 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56024720c0e0 msgr2=0x5602472093b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.122+0000 7fab177fe640 1 --2- 172.21.15.12:0/263605223 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56024720c0e0 0x5602472093b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.122+0000 7fab177fe640 1 -- 172.21.15.12:0/263605223 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560246e7a210 con 0x560247202b60 2024-04-22T13:26:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.122+0000 7fab17fff640 1 --2- 172.21.15.12:0/263605223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602472038d0 0x560247208e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.123+0000 7fab177fe640 1 --2- 172.21.15.12:0/263605223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560247202b60 0x56024720ba80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x5602471fe9e0 tx=0x7fab00014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630c5939f33be4f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.123+0000 7faaf77fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:26:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.123+0000 7faaf77fe640 1 -- 172.21.15.12:0/263605223 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab00005020 con 0x560247202b60 2024-04-22T13:26:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.123+0000 7faaf77fe640 1 -- 172.21.15.12:0/263605223 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fab000051c0 con 0x560247202b60 2024-04-22T13:26:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.123+0000 7fab20fab740 1 -- 172.21.15.12:0/263605223 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab0802c7e0 con 0x560247202b60 2024-04-22T13:26:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.123+0000 7faaf77fe640 1 -- 172.21.15.12:0/263605223 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fab000053c0 con 0x560247202b60 2024-04-22T13:26:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.123+0000 7fab20fab740 10 client.16491 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.123+0000 7fab20fab740 1 -- 172.21.15.12:0/263605223 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560246f46620 con 0x560247202b60 2024-04-22T13:26:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.123+0000 7faaf77fe640 1 -- 172.21.15.12:0/263605223 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab0802c7e0 con 0x560247202b60 2024-04-22T13:26:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.123+0000 7faaf77fe640 1 -- 172.21.15.12:0/263605223 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fab000216a0 con 0x560247202b60 2024-04-22T13:26:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.123+0000 7fab20fab740 10 client.16491 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:00.126 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.123+0000 7fab20fab740 10 client.16491 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:26:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.123+0000 7fab20fab740 20 client.16491 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.124+0000 7fab20fab740 1 --2- 172.21.15.12:0/263605223 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56024721c090 0x56024721e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.124+0000 7fab20fab740 4 client.16491 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:26:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.124+0000 7fab20fab740 1 -- 172.21.15.12:0/263605223 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56024717b880 con 0x56024721c090 2024-04-22T13:26:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.124+0000 7fab17fff640 1 --2- 172.21.15.12:0/263605223 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56024721c090 0x56024721e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.124+0000 7faaf77fe640 10 client.16491 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.124+0000 7fab17fff640 1 --2- 172.21.15.12:0/263605223 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56024721c090 0x56024721e4c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e16bc6b74ff32be server_cookie=54d63b262632e89a in_seq=0 out_seq=0 2024-04-22T13:26:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.125+0000 7faaf77fe640 1 -- 172.21.15.12:0/263605223 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56024717b880 con 0x56024721c090 2024-04-22T13:26:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.125+0000 7faaf77fe640 10 client.16491 handle_command_reply: tid=0 2024-04-22T13:26:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.125+0000 7fab20fab740 1 client.16491 shutdown 2024-04-22T13:26:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.125+0000 7fab20fab740 1 -- 172.21.15.12:0/263605223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560247202b60 msgr2=0x56024720ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.125+0000 7fab20fab740 1 --2- 172.21.15.12:0/263605223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560247202b60 0x56024720ba80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x5602471fe9e0 tx=0x7fab00014040 comp rx=0 tx=0).stop 2024-04-22T13:26:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.125+0000 7fab20fab740 1 -- 172.21.15.12:0/263605223 shutdown_connections 2024-04-22T13:26:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.125+0000 7fab20fab740 1 --2- 172.21.15.12:0/263605223 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56024721c090 0x56024721e4c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.125+0000 7fab20fab740 1 --2- 172.21.15.12:0/263605223 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56024720c0e0 0x5602472093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.125+0000 7fab20fab740 1 --2- 172.21.15.12:0/263605223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602472038d0 0x560247208e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.125+0000 7faaf77fe640 0 client.16491 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.125+0000 7fab20fab740 1 --2- 172.21.15.12:0/263605223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560247202b60 0x56024720ba80 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.125+0000 7fab20fab740 1 -- 172.21.15.12:0/263605223 >> 172.21.15.12:0/263605223 conn(0x5602471c9c90 msgr2=0x5602471ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.125+0000 7fab20fab740 1 -- 172.21.15.12:0/263605223 shutdown_connections 2024-04-22T13:26:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.125+0000 7fab20fab740 1 -- 172.21.15.12:0/263605223 wait complete. 2024-04-22T13:26:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.125+0000 7fab20fab740 20 client.16491 trim_cache size 0 max 16384 2024-04-22T13:26:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.140+0000 7fab20fab740 1 Processor -- start 2024-04-22T13:26:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.141+0000 7fab20fab740 1 -- start start 2024-04-22T13:26:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.141+0000 7fab20fab740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560247202b60 0x560247217cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.141+0000 7fab20fab740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602472038d0 0x5602472045d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.141+0000 7fab20fab740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56024720c0e0 0x56024720b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.141+0000 7fab20fab740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fab181afb10 con 0x560247202b60 2024-04-22T13:26:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.141+0000 7fab20fab740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fab181b1650 con 0x56024720c0e0 2024-04-22T13:26:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.141+0000 7fab20fab740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fab181b01e0 con 0x5602472038d0 2024-04-22T13:26:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.141+0000 7fab177fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560247202b60 0x560247217cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.141+0000 7fab177fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560247202b60 0x560247217cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58278/0 (socket says 172.21.15.12:58278) 2024-04-22T13:26:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.141+0000 7fab177fe640 1 -- 172.21.15.12:0/772222969 learned_addr learned my addr 172.21.15.12:0/772222969 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.141+0000 7fab177fe640 1 -- 172.21.15.12:0/772222969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602472038d0 msgr2=0x5602472045d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.141+0000 7fab177fe640 1 --2- 172.21.15.12:0/772222969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602472038d0 0x5602472045d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.141+0000 7fab177fe640 1 -- 172.21.15.12:0/772222969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56024720c0e0 msgr2=0x56024720b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.141+0000 7fab177fe640 1 --2- 172.21.15.12:0/772222969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56024720c0e0 0x56024720b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.141+0000 7fab177fe640 1 -- 172.21.15.12:0/772222969 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560247218820 con 0x560247202b60 2024-04-22T13:26:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.141+0000 7fab177fe640 1 --2- 172.21.15.12:0/772222969 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560247202b60 0x560247217cb0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fab000125f0 tx=0x7fab0000b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62bad0422293a698 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.142+0000 7fab0dffb640 1 -- 172.21.15.12:0/772222969 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab00038070 con 0x560247202b60 2024-04-22T13:26:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.142+0000 7fab0dffb640 1 -- 172.21.15.12:0/772222969 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab00005560 con 0x560247202b60 2024-04-22T13:26:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.142+0000 7fab0dffb640 1 -- 172.21.15.12:0/772222969 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab000048c0 con 0x560247202b60 2024-04-22T13:26:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.142+0000 7fab20fab740 1 -- 172.21.15.12:0/772222969 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560247202b60 msgr2=0x560247217cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.142+0000 7fab20fab740 1 --2- 172.21.15.12:0/772222969 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560247202b60 0x560247217cb0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fab000125f0 tx=0x7fab0000b400 comp rx=0 tx=0).stop 2024-04-22T13:26:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.142+0000 7fab20fab740 1 -- 172.21.15.12:0/772222969 shutdown_connections 2024-04-22T13:26:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.142+0000 7fab20fab740 1 --2- 172.21.15.12:0/772222969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56024720c0e0 0x56024720b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.142+0000 7fab20fab740 1 --2- 172.21.15.12:0/772222969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602472038d0 0x5602472045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.142+0000 7fab20fab740 1 --2- 172.21.15.12:0/772222969 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560247202b60 0x560247217cb0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.142+0000 7fab20fab740 1 -- 172.21.15.12:0/772222969 >> 172.21.15.12:0/772222969 conn(0x5602471c9c90 msgr2=0x560247203ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.142+0000 7fab20fab740 1 -- 172.21.15.12:0/772222969 shutdown_connections 2024-04-22T13:26:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.142+0000 7fab20fab740 1 -- 172.21.15.12:0/772222969 wait complete. 2024-04-22T13:26:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.143+0000 7fab20fab740 1 -- start start 2024-04-22T13:26:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.143+0000 7fab20fab740 1 Processor -- start 2024-04-22T13:26:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.143+0000 7fab20fab740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560247202b60 0x560247214470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.143+0000 7fab20fab740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602472038d0 0x56024720cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.143+0000 7fab20fab740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56024720c0e0 0x56024720d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.143+0000 7fab20fab740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fab181b0f80 con 0x560247202b60 2024-04-22T13:26:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.143+0000 7fab20fab740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fab181b08b0 con 0x56024720c0e0 2024-04-22T13:26:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.143+0000 7fab20fab740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fab181adaf0 con 0x5602472038d0 2024-04-22T13:26:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.143+0000 7fab20fab740 10 client.0 _ll_register_callbacks cb 0x7fab000124e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.143+0000 7fab177fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560247202b60 0x560247214470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.143+0000 7fab17fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602472038d0 0x56024720cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.143+0000 7fab177fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560247202b60 0x560247214470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:58290/0 (socket says 172.21.15.12:58290) 2024-04-22T13:26:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.143+0000 7fab177fe640 1 -- 172.21.15.12:0/1544417952 learned_addr learned my addr 172.21.15.12:0/1544417952 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.144+0000 7fab177fe640 1 --2- 172.21.15.12:0/1544417952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56024720c0e0 0x56024720d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.144+0000 7fab17fff640 1 -- 172.21.15.12:0/1544417952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56024720c0e0 msgr2=0x56024720d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.144+0000 7fab17fff640 1 --2- 172.21.15.12:0/1544417952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56024720c0e0 0x56024720d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.144+0000 7fab17fff640 1 -- 172.21.15.12:0/1544417952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560247202b60 msgr2=0x560247214470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.144+0000 7fab17fff640 1 --2- 172.21.15.12:0/1544417952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560247202b60 0x560247214470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.144+0000 7fab17fff640 1 -- 172.21.15.12:0/1544417952 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56024720d710 con 0x5602472038d0 2024-04-22T13:26:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.144+0000 7fab177fe640 1 --2- 172.21.15.12:0/1544417952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560247202b60 0x560247214470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.144+0000 7fab17fff640 1 --2- 172.21.15.12:0/1544417952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602472038d0 0x56024720cae0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x560247208670 tx=0x7fab080095b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77269915b18044d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.144+0000 7faaf77fe640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-22T13:26:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.144+0000 7faaf77fe640 1 -- 172.21.15.12:0/1544417952 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab08045040 con 0x5602472038d0 2024-04-22T13:26:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.144+0000 7fab20fab740 1 -- 172.21.15.12:0/1544417952 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab00005560 con 0x5602472038d0 2024-04-22T13:26:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.144+0000 7faaf77fe640 1 -- 172.21.15.12:0/1544417952 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fab08003a70 con 0x5602472038d0 2024-04-22T13:26:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.144+0000 7faaf77fe640 1 -- 172.21.15.12:0/1544417952 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fab0802cca0 con 0x5602472038d0 2024-04-22T13:26:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.144+0000 7fab20fab740 10 client.25849 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.144+0000 7fab20fab740 1 -- 172.21.15.12:0/1544417952 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560247282640 con 0x5602472038d0 2024-04-22T13:26:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.145+0000 7faaf77fe640 1 -- 172.21.15.12:0/1544417952 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab00005560 con 0x5602472038d0 2024-04-22T13:26:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.145+0000 7faaf77fe640 1 -- 172.21.15.12:0/1544417952 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fab0804a910 con 0x5602472038d0 2024-04-22T13:26:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.145+0000 7fab20fab740 10 client.25849 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.145+0000 7fab20fab740 10 client.25849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:26:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.145+0000 7fab20fab740 20 client.25849 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.145+0000 7fab20fab740 1 --2- 172.21.15.12:0/1544417952 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560247282870 0x560247284c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.145+0000 7fab20fab740 4 client.25849 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-22T13:26:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.145+0000 7fab20fab740 1 -- 172.21.15.12:0/1544417952 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5602471073d0 con 0x560247282870 2024-04-22T13:26:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.145+0000 7fab177fe640 1 --2- 172.21.15.12:0/1544417952 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560247282870 0x560247284c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.145+0000 7faaf77fe640 10 client.25849 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.145+0000 7fab177fe640 1 --2- 172.21.15.12:0/1544417952 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560247282870 0x560247284c60 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99f9ba4e4c886fad server_cookie=a3096e197413944c in_seq=0 out_seq=0 2024-04-22T13:26:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.156+0000 7faaf77fe640 1 -- 172.21.15.12:0/1544417952 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5602471073d0 con 0x560247282870 2024-04-22T13:26:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.156+0000 7faaf77fe640 10 client.25849 handle_command_reply: tid=0 2024-04-22T13:26:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.156+0000 7fab20fab740 1 client.25849 shutdown 2024-04-22T13:26:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.156+0000 7fab20fab740 1 -- 172.21.15.12:0/1544417952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602472038d0 msgr2=0x56024720cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.156+0000 7fab20fab740 1 --2- 172.21.15.12:0/1544417952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602472038d0 0x56024720cae0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x560247208670 tx=0x7fab080095b0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.156+0000 7fab20fab740 1 -- 172.21.15.12:0/1544417952 shutdown_connections 2024-04-22T13:26:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.156+0000 7fab20fab740 1 --2- 172.21.15.12:0/1544417952 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560247282870 0x560247284c60 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.156+0000 7fab20fab740 1 --2- 172.21.15.12:0/1544417952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56024720c0e0 0x56024720d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.156+0000 7faaf77fe640 0 client.25849 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.156+0000 7fab20fab740 1 --2- 172.21.15.12:0/1544417952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5602472038d0 0x56024720cae0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.156+0000 7fab20fab740 1 --2- 172.21.15.12:0/1544417952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560247202b60 0x560247214470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.156+0000 7fab20fab740 1 -- 172.21.15.12:0/1544417952 >> 172.21.15.12:0/1544417952 conn(0x5602471c9c90 msgr2=0x5602471fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.157+0000 7fab20fab740 1 -- 172.21.15.12:0/1544417952 shutdown_connections 2024-04-22T13:26:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.157+0000 7fab20fab740 1 -- 172.21.15.12:0/1544417952 wait complete. 2024-04-22T13:26:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.157+0000 7fab20fab740 20 client.25849 trim_cache size 0 max 16384 2024-04-22T13:26:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-04-22T13:26:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-04-22T13:26:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-04-22T13:26:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-04-22T13:26:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.157+0000 7fab1e2e0640 1 -- 172.21.15.12:0/2824513265 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fab040a3df0 msgr2=0x7fab040a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.157+0000 7fab1e2e0640 1 --2- 172.21.15.12:0/2824513265 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fab040a3df0 0x7fab040a62b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fab10098d50 tx=0x7fab1006f3e0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.157+0000 7fab1e2e0640 1 -- 172.21.15.12:0/2824513265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab181035b0 msgr2=0x7fab181a6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.157+0000 7fab1e2e0640 1 --2- 172.21.15.12:0/2824513265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab181035b0 0x7fab181a6860 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fab1000eaa0 tx=0x7fab1000ef60 comp rx=0 tx=0).stop 2024-04-22T13:26:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.157+0000 7fab1e2e0640 1 -- 172.21.15.12:0/2824513265 shutdown_connections 2024-04-22T13:26:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.157+0000 7fab1e2e0640 1 --2- 172.21.15.12:0/2824513265 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fab040a3df0 0x7fab040a62b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.157+0000 7fab1e2e0640 1 --2- 172.21.15.12:0/2824513265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab181035b0 0x7fab181a6860 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.157+0000 7fab1e2e0640 1 --2- 172.21.15.12:0/2824513265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab18106170 0x7fab18103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.157+0000 7fab1e2e0640 1 --2- 172.21.15.12:0/2824513265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab181010f0 0x7fab18102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:00.156 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.157+0000 7fab1e2e0640 1 -- 172.21.15.12:0/2824513265 >> 172.21.15.12:0/2824513265 conn(0x7fab18077fd0 msgr2=0x7fab18104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.157+0000 7fab1e2e0640 1 -- 172.21.15.12:0/2824513265 shutdown_connections 2024-04-22T13:26:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:00.157+0000 7fab1e2e0640 1 -- 172.21.15.12:0/2824513265 wait complete. 2024-04-22T13:26:00.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:25:59 smithi134 ceph-mon[31338]: pgmap v59: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 866 KiB/s wr, 17 op/s 2024-04-22T13:26:00.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:25:59 smithi111 ceph-mon[31361]: pgmap v59: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 866 KiB/s wr, 17 op/s 2024-04-22T13:26:02.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:01 smithi012 ceph-mon[28265]: pgmap v60: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 865 KiB/s wr, 13 op/s 2024-04-22T13:26:02.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:01 smithi134 ceph-mon[31338]: pgmap v60: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 865 KiB/s wr, 13 op/s 2024-04-22T13:26:02.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:01 smithi111 ceph-mon[31361]: pgmap v60: 97 pgs: 97 active+clean; 2.5 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 865 KiB/s wr, 13 op/s 2024-04-22T13:26:04.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:03 smithi012 ceph-mon[28265]: pgmap v61: 97 pgs: 97 active+clean; 251 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.8 MiB/s wr, 172 op/s 2024-04-22T13:26:04.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:03 smithi134 ceph-mon[31338]: pgmap v61: 97 pgs: 97 active+clean; 251 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.8 MiB/s wr, 172 op/s 2024-04-22T13:26:04.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:03 smithi111 ceph-mon[31361]: pgmap v61: 97 pgs: 97 active+clean; 251 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.8 MiB/s wr, 172 op/s 2024-04-22T13:26:06.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:05 smithi012 ceph-mon[28265]: pgmap v62: 97 pgs: 97 active+clean; 251 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-04-22T13:26:06.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:05 smithi134 ceph-mon[31338]: pgmap v62: 97 pgs: 97 active+clean; 251 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-04-22T13:26:06.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:05 smithi111 ceph-mon[31361]: pgmap v62: 97 pgs: 97 active+clean; 251 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-04-22T13:26:08.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:07 smithi012 ceph-mon[28265]: pgmap v63: 97 pgs: 97 active+clean; 251 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-04-22T13:26:08.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:07 smithi134 ceph-mon[31338]: pgmap v63: 97 pgs: 97 active+clean; 251 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-04-22T13:26:08.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:07 smithi111 ceph-mon[31361]: pgmap v63: 97 pgs: 97 active+clean; 251 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-04-22T13:26:10.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:09 smithi012 ceph-mon[28265]: pgmap v64: 97 pgs: 97 active+clean; 257 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 3.2 MiB/s wr, 518 op/s 2024-04-22T13:26:10.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:09 smithi134 ceph-mon[31338]: pgmap v64: 97 pgs: 97 active+clean; 257 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 3.2 MiB/s wr, 518 op/s 2024-04-22T13:26:10.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:09 smithi111 ceph-mon[31361]: pgmap v64: 97 pgs: 97 active+clean; 257 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 3.2 MiB/s wr, 518 op/s 2024-04-22T13:26:12.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:11 smithi012 ceph-mon[28265]: pgmap v65: 97 pgs: 97 active+clean; 257 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.8 MiB/s wr, 513 op/s 2024-04-22T13:26:12.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:11 smithi134 ceph-mon[31338]: pgmap v65: 97 pgs: 97 active+clean; 257 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.8 MiB/s wr, 513 op/s 2024-04-22T13:26:12.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:11 smithi111 ceph-mon[31361]: pgmap v65: 97 pgs: 97 active+clean; 257 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.8 MiB/s wr, 513 op/s 2024-04-22T13:26:13.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:12 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:26:13.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:12 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:26:13.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:12 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:26:13.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.270+0000 7eff82081640 1 -- 172.21.15.12:0/1499945922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff7c106170 msgr2=0x7eff7c10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:13.267 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.270+0000 7eff82081640 1 --2- 172.21.15.12:0/1499945922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff7c106170 0x7eff7c10dd40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7eff74009be0 tx=0x7eff7402bef0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.270+0000 7eff82081640 1 -- 172.21.15.12:0/1499945922 shutdown_connections 2024-04-22T13:26:13.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.270+0000 7eff82081640 1 --2- 172.21.15.12:0/1499945922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff7c106170 0x7eff7c10dd40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.270+0000 7eff82081640 1 --2- 172.21.15.12:0/1499945922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff7c101aa0 0x7eff7c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.270+0000 7eff82081640 1 --2- 172.21.15.12:0/1499945922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff7c1010f0 0x7eff7c1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.270+0000 7eff82081640 1 -- 172.21.15.12:0/1499945922 >> 172.21.15.12:0/1499945922 conn(0x7eff7c077fd0 msgr2=0x7eff7c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:13.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.271+0000 7eff82081640 1 -- 172.21.15.12:0/1499945922 shutdown_connections 2024-04-22T13:26:13.268 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.271+0000 7eff82081640 1 -- 172.21.15.12:0/1499945922 wait complete. 2024-04-22T13:26:13.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.271+0000 7eff82081640 1 Processor -- start 2024-04-22T13:26:13.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.272+0000 7eff82081640 1 -- start start 2024-04-22T13:26:13.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.272+0000 7eff82081640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff7c1010f0 0x7eff7c1994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:13.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.272+0000 7eff82081640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff7c106170 0x7eff7c1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:13.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.272+0000 7eff82081640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff7c199f70 0x7eff7c1a4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:13.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.272+0000 7eff82081640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7eff7c1110e0 con 0x7eff7c1010f0 2024-04-22T13:26:13.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.272+0000 7eff82081640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7eff7c110f60 con 0x7eff7c199f70 2024-04-22T13:26:13.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.272+0000 7eff82081640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7eff7c111260 con 0x7eff7c106170 2024-04-22T13:26:13.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.272+0000 7eff7b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff7c1010f0 0x7eff7c1994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:13.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.272+0000 7eff7b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff7c1010f0 0x7eff7c1994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48936/0 (socket says 172.21.15.12:48936) 2024-04-22T13:26:13.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.272+0000 7eff7b7fe640 1 -- 172.21.15.12:0/192117481 learned_addr learned my addr 172.21.15.12:0/192117481 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:13.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.272+0000 7eff7bfff640 1 --2- 172.21.15.12:0/192117481 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff7c199f70 0x7eff7c1a4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:13.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.272+0000 7eff7affd640 1 --2- 172.21.15.12:0/192117481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff7c106170 0x7eff7c1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:13.270 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.272+0000 7eff7b7fe640 1 -- 172.21.15.12:0/192117481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff7c106170 msgr2=0x7eff7c1999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:13.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.272+0000 7eff7b7fe640 1 --2- 172.21.15.12:0/192117481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff7c106170 0x7eff7c1999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.272+0000 7eff7b7fe640 1 -- 172.21.15.12:0/192117481 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff7c199f70 msgr2=0x7eff7c1a4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:13.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.272+0000 7eff7b7fe640 1 --2- 172.21.15.12:0/192117481 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff7c199f70 0x7eff7c1a4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.272+0000 7eff7b7fe640 1 -- 172.21.15.12:0/192117481 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff7c1a4cf0 con 0x7eff7c1010f0 2024-04-22T13:26:13.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.272+0000 7eff7affd640 1 --2- 172.21.15.12:0/192117481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff7c106170 0x7eff7c1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:13.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.273+0000 7eff7b7fe640 1 --2- 172.21.15.12:0/192117481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff7c1010f0 0x7eff7c1994a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7eff6c00e9f0 tx=0x7eff6c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:13.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.273+0000 7eff78ff9640 1 -- 172.21.15.12:0/192117481 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6c00ccc0 con 0x7eff7c1010f0 2024-04-22T13:26:13.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.273+0000 7eff82081640 1 -- 172.21.15.12:0/192117481 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff7c1a4fe0 con 0x7eff7c1010f0 2024-04-22T13:26:13.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.273+0000 7eff78ff9640 1 -- 172.21.15.12:0/192117481 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff6c0054d0 con 0x7eff7c1010f0 2024-04-22T13:26:13.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.273+0000 7eff78ff9640 1 -- 172.21.15.12:0/192117481 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6c010430 con 0x7eff7c1010f0 2024-04-22T13:26:13.271 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.273+0000 7eff82081640 1 -- 172.21.15.12:0/192117481 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff7c1a54a0 con 0x7eff7c1010f0 2024-04-22T13:26:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.276+0000 7eff84c71740 1 Processor -- start 2024-04-22T13:26:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.276+0000 7eff84c71740 1 -- start start 2024-04-22T13:26:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.276+0000 7eff78ff9640 1 -- 172.21.15.12:0/192117481 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7eff6c0046c0 con 0x7eff7c1010f0 2024-04-22T13:26:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.276+0000 7eff84c71740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b2e38738d0 0x55b2e3872620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.276+0000 7eff84c71740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e3872b60 0x55b2e3872fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.276+0000 7eff84c71740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2e387c0e0 0x55b2e387e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.276+0000 7eff84c71740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b2e37d78d0 con 0x55b2e387c0e0 2024-04-22T13:26:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.276+0000 7eff84c71740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b2e3774e80 con 0x55b2e38738d0 2024-04-22T13:26:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.276+0000 7eff84c71740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b2e37d3890 con 0x55b2e3872b60 2024-04-22T13:26:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.276+0000 7eff7affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2e387c0e0 0x55b2e387e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.276+0000 7eff7bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b2e38738d0 0x55b2e3872620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.276+0000 7eff7affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2e387c0e0 0x55b2e387e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48942/0 (socket says 172.21.15.12:48942) 2024-04-22T13:26:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.276+0000 7eff7affd640 1 -- 172.21.15.12:0/1115811891 learned_addr learned my addr 172.21.15.12:0/1115811891 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.277+0000 7eff7b7fe640 1 --2- 172.21.15.12:0/1115811891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e3872b60 0x55b2e3872fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:13.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.277+0000 7eff7bfff640 1 -- 172.21.15.12:0/1115811891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e3872b60 msgr2=0x55b2e3872fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:13.281 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.277+0000 7eff7bfff640 1 --2- 172.21.15.12:0/1115811891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e3872b60 0x55b2e3872fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.277+0000 7eff7bfff640 1 -- 172.21.15.12:0/1115811891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2e387c0e0 msgr2=0x55b2e387e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.277+0000 7eff7bfff640 1 --2- 172.21.15.12:0/1115811891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2e387c0e0 0x55b2e387e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.277+0000 7eff7bfff640 1 -- 172.21.15.12:0/1115811891 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2e3796320 con 0x55b2e38738d0 2024-04-22T13:26:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.277+0000 7eff7affd640 1 --2- 172.21.15.12:0/1115811891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2e387c0e0 0x55b2e387e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.278+0000 7eff7bfff640 1 --2- 172.21.15.12:0/1115811891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b2e38738d0 0x55b2e3872620 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7eff74034fd0 tx=0x7eff74005c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=998d01ca0aaf4b70 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.278+0000 7eff78ff9640 1 --2- 172.21.15.12:0/192117481 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7eff640a3e40 0x7eff640a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.278+0000 7eff61ffb640 1 -- 172.21.15.12:0/1115811891 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff74043030 con 0x55b2e38738d0 2024-04-22T13:26:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.278+0000 7eff7b7fe640 1 --2- 172.21.15.12:0/192117481 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7eff640a3e40 0x7eff640a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.278+0000 7eff61ffb640 1 -- 172.21.15.12:0/1115811891 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff740334a0 con 0x55b2e38738d0 2024-04-22T13:26:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.278+0000 7eff61ffb640 1 -- 172.21.15.12:0/1115811891 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff74003dc0 con 0x55b2e38738d0 2024-04-22T13:26:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.278+0000 7eff78ff9640 1 -- 172.21.15.12:0/192117481 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7eff6c0d0a90 con 0x7eff7c1010f0 2024-04-22T13:26:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.279+0000 7eff84c71740 1 -- 172.21.15.12:0/1115811891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b2e38738d0 msgr2=0x55b2e3872620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.279+0000 7eff7b7fe640 1 --2- 172.21.15.12:0/192117481 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7eff640a3e40 0x7eff640a6300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x55b2e3874030 tx=0x7eff6c0d4410 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.279+0000 7eff84c71740 1 --2- 172.21.15.12:0/1115811891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b2e38738d0 0x55b2e3872620 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7eff74034fd0 tx=0x7eff74005c60 comp rx=0 tx=0).stop 2024-04-22T13:26:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.279+0000 7eff84c71740 1 -- 172.21.15.12:0/1115811891 shutdown_connections 2024-04-22T13:26:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.279+0000 7eff84c71740 1 --2- 172.21.15.12:0/1115811891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2e387c0e0 0x55b2e387e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.279+0000 7eff84c71740 1 --2- 172.21.15.12:0/1115811891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e3872b60 0x55b2e3872fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.279+0000 7eff84c71740 1 --2- 172.21.15.12:0/1115811891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b2e38738d0 0x55b2e3872620 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.279+0000 7eff84c71740 1 -- 172.21.15.12:0/1115811891 >> 172.21.15.12:0/1115811891 conn(0x55b2e3839c90 msgr2=0x55b2e386d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.279+0000 7eff84c71740 1 -- 172.21.15.12:0/1115811891 shutdown_connections 2024-04-22T13:26:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.279+0000 7eff84c71740 1 -- 172.21.15.12:0/1115811891 wait complete. 2024-04-22T13:26:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.279+0000 7eff84c71740 1 -- start start 2024-04-22T13:26:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.280+0000 7eff84c71740 1 Processor -- start 2024-04-22T13:26:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.280+0000 7eff84c71740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e3872b60 0x55b2e387baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.280+0000 7eff84c71740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b2e38738d0 0x55b2e3878e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.280+0000 7eff84c71740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2e387c0e0 0x55b2e3879490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.280+0000 7eff84c71740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b2e387edb0 con 0x55b2e387c0e0 2024-04-22T13:26:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.280+0000 7eff84c71740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b2e375feb0 con 0x55b2e38738d0 2024-04-22T13:26:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.280+0000 7eff84c71740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b2e387ef30 con 0x55b2e3872b60 2024-04-22T13:26:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.280+0000 7eff84c71740 10 client.0 _ll_register_callbacks cb 0x55b2e378cb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.280+0000 7eff7bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e3872b60 0x55b2e387baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.280+0000 7eff7affd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b2e38738d0 0x55b2e3878e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.280+0000 7eff7bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e3872b60 0x55b2e387baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.12:51844/0 (socket says 172.21.15.12:51844) 2024-04-22T13:26:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.280+0000 7eff7affd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b2e38738d0 0x55b2e3878e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:43984/0 (socket says 172.21.15.12:43984) 2024-04-22T13:26:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.280+0000 7eff7bfff640 1 -- 172.21.15.12:0/1781616407 learned_addr learned my addr 172.21.15.12:0/1781616407 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.280+0000 7eff7bfff640 1 --2- 172.21.15.12:0/1781616407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2e387c0e0 0x55b2e3879490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.280+0000 7eff7bfff640 1 -- 172.21.15.12:0/1781616407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b2e38738d0 msgr2=0x55b2e3878e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.280+0000 7eff7bfff640 1 --2- 172.21.15.12:0/1781616407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b2e38738d0 0x55b2e3878e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.281+0000 7eff7bfff640 1 -- 172.21.15.12:0/1781616407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2e387c0e0 msgr2=0x55b2e3879490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.281+0000 7eff7bfff640 1 --2- 172.21.15.12:0/1781616407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2e387c0e0 0x55b2e3879490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.281+0000 7eff7bfff640 1 -- 172.21.15.12:0/1781616407 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2e34ea210 con 0x55b2e3872b60 2024-04-22T13:26:13.284 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.281+0000 7eff7affd640 1 --2- 172.21.15.12:0/1781616407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b2e38738d0 0x55b2e3878e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.281+0000 7eff7bfff640 1 --2- 172.21.15.12:0/1781616407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e3872b60 0x55b2e387baf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7eff740099a0 tx=0x7eff74009480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba9b2add57cc8921 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.281+0000 7eff5affd640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-22T13:26:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.281+0000 7eff5affd640 1 -- 172.21.15.12:0/1781616407 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff74043040 con 0x55b2e3872b60 2024-04-22T13:26:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.281+0000 7eff5affd640 1 -- 172.21.15.12:0/1781616407 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7eff7404d030 con 0x55b2e3872b60 2024-04-22T13:26:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.281+0000 7eff84c71740 1 -- 172.21.15.12:0/1781616407 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff740334a0 con 0x55b2e3872b60 2024-04-22T13:26:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.281+0000 7eff5affd640 1 -- 172.21.15.12:0/1781616407 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7eff74033c30 con 0x55b2e3872b60 2024-04-22T13:26:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.281+0000 7eff84c71740 10 client.25861 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.281+0000 7eff84c71740 1 -- 172.21.15.12:0/1781616407 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b2e35b6620 con 0x55b2e3872b60 2024-04-22T13:26:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.281+0000 7eff5affd640 1 -- 172.21.15.12:0/1781616407 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff740334a0 con 0x55b2e3872b60 2024-04-22T13:26:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.282+0000 7eff5affd640 1 -- 172.21.15.12:0/1781616407 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7eff74004040 con 0x55b2e3872b60 2024-04-22T13:26:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.282+0000 7eff84c71740 10 client.25861 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.282+0000 7eff84c71740 10 client.25861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:26:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.282+0000 7eff84c71740 20 client.25861 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.282+0000 7eff84c71740 1 --2- 172.21.15.12:0/1781616407 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b2e38903b0 0x55b2e38927a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.282+0000 7eff84c71740 4 client.25861 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:26:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.282+0000 7eff84c71740 1 -- 172.21.15.12:0/1781616407 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2e37eb880 con 0x55b2e38903b0 2024-04-22T13:26:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.282+0000 7eff7affd640 1 --2- 172.21.15.12:0/1781616407 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b2e38903b0 0x55b2e38927a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.282+0000 7eff5affd640 10 client.25861 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.282+0000 7eff7affd640 1 --2- 172.21.15.12:0/1781616407 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b2e38903b0 0x55b2e38927a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc80292c124942 server_cookie=4b41af3a5da8ea34 in_seq=0 out_seq=0 2024-04-22T13:26:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.283+0000 7eff5affd640 1 -- 172.21.15.12:0/1781616407 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b2e37eb880 con 0x55b2e38903b0 2024-04-22T13:26:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.283+0000 7eff5affd640 10 client.25861 handle_command_reply: tid=0 2024-04-22T13:26:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.283+0000 7eff84c71740 1 client.25861 shutdown 2024-04-22T13:26:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.283+0000 7eff84c71740 1 -- 172.21.15.12:0/1781616407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e3872b60 msgr2=0x55b2e387baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.283+0000 7eff84c71740 1 --2- 172.21.15.12:0/1781616407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e3872b60 0x55b2e387baf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7eff740099a0 tx=0x7eff74009480 comp rx=0 tx=0).stop 2024-04-22T13:26:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.283+0000 7eff84c71740 1 -- 172.21.15.12:0/1781616407 shutdown_connections 2024-04-22T13:26:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.283+0000 7eff84c71740 1 --2- 172.21.15.12:0/1781616407 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b2e38903b0 0x55b2e38927a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.283+0000 7eff84c71740 1 --2- 172.21.15.12:0/1781616407 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2e387c0e0 0x55b2e3879490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.283+0000 7eff84c71740 1 --2- 172.21.15.12:0/1781616407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b2e38738d0 0x55b2e3878e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.283+0000 7eff84c71740 1 --2- 172.21.15.12:0/1781616407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e3872b60 0x55b2e387baf0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.283+0000 7eff5affd640 0 client.25861 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.283+0000 7eff84c71740 1 -- 172.21.15.12:0/1781616407 >> 172.21.15.12:0/1781616407 conn(0x55b2e3839c90 msgr2=0x55b2e386f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.283+0000 7eff84c71740 1 -- 172.21.15.12:0/1781616407 shutdown_connections 2024-04-22T13:26:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.283+0000 7eff84c71740 1 -- 172.21.15.12:0/1781616407 wait complete. 2024-04-22T13:26:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.283+0000 7eff84c71740 20 client.25861 trim_cache size 0 max 16384 2024-04-22T13:26:13.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.301+0000 7eff84c71740 1 Processor -- start 2024-04-22T13:26:13.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.301+0000 7eff84c71740 1 -- start start 2024-04-22T13:26:13.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.301+0000 7eff84c71740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b2e3872b60 0x55b2e3887cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:13.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.301+0000 7eff84c71740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e38738d0 0x55b2e3877ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:13.299 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.301+0000 7eff84c71740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2e387c0e0 0x55b2e387b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.301+0000 7eff84c71740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7eff7c1ab720 con 0x55b2e387c0e0 2024-04-22T13:26:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.301+0000 7eff84c71740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7eff7c1ad260 con 0x55b2e3872b60 2024-04-22T13:26:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.301+0000 7eff84c71740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7eff7c1abdf0 con 0x55b2e38738d0 2024-04-22T13:26:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.301+0000 7eff7bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2e387c0e0 0x55b2e387b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.301+0000 7eff7affd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e38738d0 0x55b2e3877ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.301+0000 7eff7bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2e387c0e0 0x55b2e387b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:48948/0 (socket says 172.21.15.12:48948) 2024-04-22T13:26:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.301+0000 7eff7bfff640 1 -- 172.21.15.12:0/4204585154 learned_addr learned my addr 172.21.15.12:0/4204585154 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.301+0000 7eff7affd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e38738d0 0x55b2e3877ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.12:51856/0 (socket says 172.21.15.12:51856) 2024-04-22T13:26:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.302+0000 7eff7bfff640 1 --2- 172.21.15.12:0/4204585154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b2e3872b60 0x55b2e3887cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.302+0000 7eff7bfff640 1 -- 172.21.15.12:0/4204585154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e38738d0 msgr2=0x55b2e3877ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.302+0000 7eff7bfff640 1 --2- 172.21.15.12:0/4204585154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e38738d0 0x55b2e3877ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.302+0000 7eff7bfff640 1 -- 172.21.15.12:0/4204585154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b2e3872b60 msgr2=0x55b2e3887cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.302+0000 7eff7bfff640 1 --2- 172.21.15.12:0/4204585154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b2e3872b60 0x55b2e3887cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.302+0000 7eff7bfff640 1 -- 172.21.15.12:0/4204585154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2e3872990 con 0x55b2e387c0e0 2024-04-22T13:26:13.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.302+0000 7eff7affd640 1 --2- 172.21.15.12:0/4204585154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e38738d0 0x55b2e3877ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:13.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.302+0000 7eff7bfff640 1 --2- 172.21.15.12:0/4204585154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2e387c0e0 0x55b2e387b5b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7eff74002230 tx=0x7eff7402c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e08b16f814fa8c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:13.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.302+0000 7eff617fa640 1 -- 172.21.15.12:0/4204585154 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff74043070 con 0x55b2e387c0e0 2024-04-22T13:26:13.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.302+0000 7eff617fa640 1 -- 172.21.15.12:0/4204585154 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff7404c020 con 0x55b2e387c0e0 2024-04-22T13:26:13.301 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.302+0000 7eff617fa640 1 -- 172.21.15.12:0/4204585154 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff74004790 con 0x55b2e387c0e0 2024-04-22T13:26:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.303+0000 7eff84c71740 1 -- 172.21.15.12:0/4204585154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2e387c0e0 msgr2=0x55b2e387b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.303+0000 7eff84c71740 1 --2- 172.21.15.12:0/4204585154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2e387c0e0 0x55b2e387b5b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7eff74002230 tx=0x7eff7402c890 comp rx=0 tx=0).stop 2024-04-22T13:26:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.303+0000 7eff84c71740 1 -- 172.21.15.12:0/4204585154 shutdown_connections 2024-04-22T13:26:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.303+0000 7eff84c71740 1 --2- 172.21.15.12:0/4204585154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2e387c0e0 0x55b2e387b5b0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.303+0000 7eff84c71740 1 --2- 172.21.15.12:0/4204585154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e38738d0 0x55b2e3877ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.303+0000 7eff84c71740 1 --2- 172.21.15.12:0/4204585154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b2e3872b60 0x55b2e3887cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.303+0000 7eff84c71740 1 -- 172.21.15.12:0/4204585154 >> 172.21.15.12:0/4204585154 conn(0x55b2e3839c90 msgr2=0x55b2e38777b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.303+0000 7eff84c71740 1 -- 172.21.15.12:0/4204585154 shutdown_connections 2024-04-22T13:26:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.303+0000 7eff84c71740 1 -- 172.21.15.12:0/4204585154 wait complete. 2024-04-22T13:26:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.303+0000 7eff84c71740 1 -- start start 2024-04-22T13:26:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.304+0000 7eff84c71740 1 Processor -- start 2024-04-22T13:26:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.304+0000 7eff84c71740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b2e3872b60 0x55b2e3884590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.304+0000 7eff84c71740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e38738d0 0x55b2e3875890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.304+0000 7eff84c71740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2e387c0e0 0x55b2e3875dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.304+0000 7eff84c71740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7eff7c1acb90 con 0x55b2e387c0e0 2024-04-22T13:26:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.304+0000 7eff84c71740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7eff7c1ac4c0 con 0x55b2e3872b60 2024-04-22T13:26:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.304+0000 7eff84c71740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7eff7c1a9720 con 0x55b2e38738d0 2024-04-22T13:26:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.304+0000 7eff84c71740 10 client.0 _ll_register_callbacks cb 0x7eff70009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.304+0000 7eff7bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b2e3872b60 0x55b2e3884590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.304+0000 7eff7bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2e387c0e0 0x55b2e3875dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.304+0000 7eff7affd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e38738d0 0x55b2e3875890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:13.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.304+0000 7eff7bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b2e3872b60 0x55b2e3884590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:44000/0 (socket says 172.21.15.12:44000) 2024-04-22T13:26:13.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.304+0000 7eff7bfff640 1 -- 172.21.15.12:0/2891631083 learned_addr learned my addr 172.21.15.12:0/2891631083 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:13.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.304+0000 7eff7bfff640 1 -- 172.21.15.12:0/2891631083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e38738d0 msgr2=0x55b2e3875890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:13.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.304+0000 7eff7bfff640 1 --2- 172.21.15.12:0/2891631083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e38738d0 0x55b2e3875890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.304+0000 7eff7bfff640 1 -- 172.21.15.12:0/2891631083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2e387c0e0 msgr2=0x55b2e3875dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:13.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.304+0000 7eff7bfff640 1 --2- 172.21.15.12:0/2891631083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2e387c0e0 0x55b2e3875dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.304+0000 7eff7bfff640 1 -- 172.21.15.12:0/2891631083 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2e3876600 con 0x55b2e3872b60 2024-04-22T13:26:13.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.305+0000 7eff7bfff640 1 --2- 172.21.15.12:0/2891631083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b2e3872b60 0x55b2e3884590 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7eff740099a0 tx=0x7eff74033ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bb28131d9b859b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:13.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.305+0000 7eff5affd640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-22T13:26:13.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.305+0000 7eff5affd640 1 -- 172.21.15.12:0/2891631083 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff74043040 con 0x55b2e3872b60 2024-04-22T13:26:13.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.305+0000 7eff5affd640 1 -- 172.21.15.12:0/2891631083 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7eff74051030 con 0x55b2e3872b60 2024-04-22T13:26:13.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.305+0000 7eff84c71740 1 -- 172.21.15.12:0/2891631083 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff7404c020 con 0x55b2e3872b60 2024-04-22T13:26:13.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.305+0000 7eff5affd640 1 -- 172.21.15.12:0/2891631083 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7eff740335f0 con 0x55b2e3872b60 2024-04-22T13:26:13.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.305+0000 7eff84c71740 10 client.25931 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:13.306 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.305+0000 7eff84c71740 1 -- 172.21.15.12:0/2891631083 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b2e38f6600 con 0x55b2e3872b60 2024-04-22T13:26:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.305+0000 7eff5affd640 1 -- 172.21.15.12:0/2891631083 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff7404c020 con 0x55b2e3872b60 2024-04-22T13:26:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.305+0000 7eff5affd640 1 -- 172.21.15.12:0/2891631083 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7eff74034920 con 0x55b2e3872b60 2024-04-22T13:26:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.305+0000 7eff84c71740 10 client.25931 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.306+0000 7eff84c71740 10 client.25931 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:26:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.306+0000 7eff84c71740 20 client.25931 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.306+0000 7eff84c71740 1 --2- 172.21.15.12:0/2891631083 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b2e38f69e0 0x55b2e38f8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.306+0000 7eff84c71740 4 client.25931 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-22T13:26:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.306+0000 7eff84c71740 1 -- 172.21.15.12:0/2891631083 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b2e37773d0 con 0x55b2e38f69e0 2024-04-22T13:26:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.306+0000 7eff7affd640 1 --2- 172.21.15.12:0/2891631083 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b2e38f69e0 0x55b2e38f8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.306+0000 7eff7affd640 1 --2- 172.21.15.12:0/2891631083 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b2e38f69e0 0x55b2e38f8e00 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5c1b59209ebf003 server_cookie=1986f13dee74d1c2 in_seq=0 out_seq=0 2024-04-22T13:26:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.306+0000 7eff5affd640 10 client.25931 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.316+0000 7eff5affd640 1 -- 172.21.15.12:0/2891631083 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b2e37773d0 con 0x55b2e38f69e0 2024-04-22T13:26:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.316+0000 7eff5affd640 10 client.25931 handle_command_reply: tid=0 2024-04-22T13:26:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.316+0000 7eff84c71740 1 client.25931 shutdown 2024-04-22T13:26:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.316+0000 7eff84c71740 1 -- 172.21.15.12:0/2891631083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b2e3872b60 msgr2=0x55b2e3884590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.316+0000 7eff84c71740 1 --2- 172.21.15.12:0/2891631083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b2e3872b60 0x55b2e3884590 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7eff740099a0 tx=0x7eff74033ad0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.316+0000 7eff84c71740 1 -- 172.21.15.12:0/2891631083 shutdown_connections 2024-04-22T13:26:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.316+0000 7eff84c71740 1 --2- 172.21.15.12:0/2891631083 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b2e38f69e0 0x55b2e38f8e00 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.316+0000 7eff84c71740 1 --2- 172.21.15.12:0/2891631083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b2e387c0e0 0x55b2e3875dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.316+0000 7eff5affd640 0 client.25931 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.316+0000 7eff84c71740 1 --2- 172.21.15.12:0/2891631083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2e38738d0 0x55b2e3875890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.316+0000 7eff84c71740 1 --2- 172.21.15.12:0/2891631083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b2e3872b60 0x55b2e3884590 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.316+0000 7eff84c71740 1 -- 172.21.15.12:0/2891631083 >> 172.21.15.12:0/2891631083 conn(0x55b2e3839c90 msgr2=0x55b2e386c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.317+0000 7eff84c71740 1 -- 172.21.15.12:0/2891631083 shutdown_connections 2024-04-22T13:26:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.317+0000 7eff84c71740 1 -- 172.21.15.12:0/2891631083 wait complete. 2024-04-22T13:26:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.317+0000 7eff84c71740 20 client.25931 trim_cache size 0 max 16384 2024-04-22T13:26:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-04-22T13:26:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-04-22T13:26:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-04-22T13:26:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-04-22T13:26:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.317+0000 7eff82081640 1 -- 172.21.15.12:0/192117481 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7eff640a3e40 msgr2=0x7eff640a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.317+0000 7eff82081640 1 --2- 172.21.15.12:0/192117481 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7eff640a3e40 0x7eff640a6300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x55b2e3874030 tx=0x7eff6c0d4410 comp rx=0 tx=0).stop 2024-04-22T13:26:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.317+0000 7eff82081640 1 -- 172.21.15.12:0/192117481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff7c1010f0 msgr2=0x7eff7c1994a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.317+0000 7eff82081640 1 --2- 172.21.15.12:0/192117481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff7c1010f0 0x7eff7c1994a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7eff6c00e9f0 tx=0x7eff6c00eeb0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.317+0000 7eff82081640 1 -- 172.21.15.12:0/192117481 shutdown_connections 2024-04-22T13:26:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.317+0000 7eff82081640 1 --2- 172.21.15.12:0/192117481 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7eff640a3e40 0x7eff640a6300 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.317+0000 7eff82081640 1 --2- 172.21.15.12:0/192117481 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff7c199f70 0x7eff7c1a4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.317+0000 7eff82081640 1 --2- 172.21.15.12:0/192117481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff7c106170 0x7eff7c1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.317+0000 7eff82081640 1 --2- 172.21.15.12:0/192117481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff7c1010f0 0x7eff7c1994a0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:13.316 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.317+0000 7eff82081640 1 -- 172.21.15.12:0/192117481 >> 172.21.15.12:0/192117481 conn(0x7eff7c077fd0 msgr2=0x7eff7c0fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.317+0000 7eff82081640 1 -- 172.21.15.12:0/192117481 shutdown_connections 2024-04-22T13:26:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:13.317+0000 7eff82081640 1 -- 172.21.15.12:0/192117481 wait complete. 2024-04-22T13:26:14.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:13 smithi012 ceph-mon[28265]: pgmap v66: 97 pgs: 97 active+clean; 258 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.5 MiB/s wr, 850 op/s 2024-04-22T13:26:14.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:13 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:26:14.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:13 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:26:14.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:13 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:26:14.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:13 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:26:14.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:13 smithi134 ceph-mon[31338]: pgmap v66: 97 pgs: 97 active+clean; 258 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.5 MiB/s wr, 850 op/s 2024-04-22T13:26:14.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:13 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:26:14.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:13 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:26:14.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:13 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:26:14.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:13 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:26:14.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:13 smithi111 ceph-mon[31361]: pgmap v66: 97 pgs: 97 active+clean; 258 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.5 MiB/s wr, 850 op/s 2024-04-22T13:26:14.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:13 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:26:14.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:13 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:26:14.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:13 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:26:14.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:13 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:26:15.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:14 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:26:15.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:14 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:26:15.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:14 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:26:15.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:14 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:26:15.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:14 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:26:15.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:14 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:26:15.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:14 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:26:15.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:14 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:26:15.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:14 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:26:16.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:15 smithi012 ceph-mon[28265]: pgmap v67: 97 pgs: 97 active+clean; 258 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.5 MiB/s wr, 690 op/s 2024-04-22T13:26:16.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:15 smithi134 ceph-mon[31338]: pgmap v67: 97 pgs: 97 active+clean; 258 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.5 MiB/s wr, 690 op/s 2024-04-22T13:26:16.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:15 smithi111 ceph-mon[31361]: pgmap v67: 97 pgs: 97 active+clean; 258 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.5 MiB/s wr, 690 op/s 2024-04-22T13:26:18.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:17 smithi012 ceph-mon[28265]: pgmap v68: 97 pgs: 97 active+clean; 258 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.5 MiB/s wr, 690 op/s 2024-04-22T13:26:18.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:17 smithi134 ceph-mon[31338]: pgmap v68: 97 pgs: 97 active+clean; 258 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.5 MiB/s wr, 690 op/s 2024-04-22T13:26:18.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:17 smithi111 ceph-mon[31361]: pgmap v68: 97 pgs: 97 active+clean; 258 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 3.5 MiB/s wr, 690 op/s 2024-04-22T13:26:20.028 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:19 smithi111 ceph-mon[31361]: pgmap v69: 97 pgs: 97 active+clean; 259 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.2 MiB/s wr, 1.10k op/s 2024-04-22T13:26:20.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:19 smithi012 ceph-mon[28265]: pgmap v69: 97 pgs: 97 active+clean; 259 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.2 MiB/s wr, 1.10k op/s 2024-04-22T13:26:20.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:19 smithi134 ceph-mon[31338]: pgmap v69: 97 pgs: 97 active+clean; 259 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.2 MiB/s wr, 1.10k op/s 2024-04-22T13:26:21.764 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:26:21.764 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:26:21.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.882+0000 7f8e36de0640 1 -- 172.21.15.12:0/216766071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e301021a0 msgr2=0x7f8e30102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:21.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.882+0000 7f8e36de0640 1 --2- 172.21.15.12:0/216766071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e301021a0 0x7f8e30102580 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f8e24009900 tx=0x7f8e2402bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:21.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.883+0000 7f8e36de0640 1 -- 172.21.15.12:0/216766071 shutdown_connections 2024-04-22T13:26:21.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.883+0000 7f8e36de0640 1 --2- 172.21.15.12:0/216766071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e30107270 0x7f8e3010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.883+0000 7f8e36de0640 1 --2- 172.21.15.12:0/216766071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e30102b50 0x7f8e30106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.883+0000 7f8e36de0640 1 --2- 172.21.15.12:0/216766071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e301021a0 0x7f8e30102580 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.880 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.883+0000 7f8e36de0640 1 -- 172.21.15.12:0/216766071 >> 172.21.15.12:0/216766071 conn(0x7f8e300fda40 msgr2=0x7f8e300ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:21.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.883+0000 7f8e36de0640 1 -- 172.21.15.12:0/216766071 shutdown_connections 2024-04-22T13:26:21.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.883+0000 7f8e36de0640 1 -- 172.21.15.12:0/216766071 wait complete. 2024-04-22T13:26:21.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.883+0000 7f8e36de0640 1 Processor -- start 2024-04-22T13:26:21.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.883+0000 7f8e36de0640 1 -- start start 2024-04-22T13:26:21.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.884+0000 7f8e36de0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e301021a0 0x7f8e301987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:21.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.884+0000 7f8e36de0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e30107270 0x7f8e30198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:21.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.884+0000 7f8e36de0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e3019d020 0x7f8e3019d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:21.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.884+0000 7f8e36de0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8e301121b0 con 0x7f8e3019d020 2024-04-22T13:26:21.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.884+0000 7f8e36de0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8e30112030 con 0x7f8e301021a0 2024-04-22T13:26:21.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.884+0000 7f8e36de0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8e30112330 con 0x7f8e30107270 2024-04-22T13:26:21.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.884+0000 7f8e3537d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e3019d020 0x7f8e3019d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:21.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.884+0000 7f8e34b7c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e301021a0 0x7f8e301987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:21.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.884+0000 7f8e3537d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e3019d020 0x7f8e3019d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42950/0 (socket says 172.21.15.12:42950) 2024-04-22T13:26:21.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.884+0000 7f8e3537d640 1 -- 172.21.15.12:0/2450020805 learned_addr learned my addr 172.21.15.12:0/2450020805 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:21.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.884+0000 7f8e2ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e30107270 0x7f8e30198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:21.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.884+0000 7f8e34b7c640 1 -- 172.21.15.12:0/2450020805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e30107270 msgr2=0x7f8e30198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:21.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.884+0000 7f8e34b7c640 1 --2- 172.21.15.12:0/2450020805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e30107270 0x7f8e30198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.884+0000 7f8e34b7c640 1 -- 172.21.15.12:0/2450020805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e3019d020 msgr2=0x7f8e3019d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:21.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.884+0000 7f8e34b7c640 1 --2- 172.21.15.12:0/2450020805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e3019d020 0x7f8e3019d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.884+0000 7f8e34b7c640 1 -- 172.21.15.12:0/2450020805 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e3019db10 con 0x7f8e301021a0 2024-04-22T13:26:21.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.884+0000 7f8e34b7c640 1 --2- 172.21.15.12:0/2450020805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e301021a0 0x7f8e301987b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8e24036040 tx=0x7f8e240045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:21.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.884+0000 7f8e2dffb640 1 -- 172.21.15.12:0/2450020805 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24043020 con 0x7f8e301021a0 2024-04-22T13:26:21.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.885+0000 7f8e36de0640 1 -- 172.21.15.12:0/2450020805 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e3019dda0 con 0x7f8e301021a0 2024-04-22T13:26:21.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.885+0000 7f8e2dffb640 1 -- 172.21.15.12:0/2450020805 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e2402ca70 con 0x7f8e301021a0 2024-04-22T13:26:21.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.885+0000 7f8e2dffb640 1 -- 172.21.15.12:0/2450020805 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24003a50 con 0x7f8e301021a0 2024-04-22T13:26:21.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.885+0000 7f8e36de0640 1 -- 172.21.15.12:0/2450020805 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e301a55f0 con 0x7f8e301021a0 2024-04-22T13:26:21.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.887+0000 7f8e2dffb640 1 -- 172.21.15.12:0/2450020805 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8e24046020 con 0x7f8e301021a0 2024-04-22T13:26:21.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.887+0000 7f8e399cd740 1 Processor -- start 2024-04-22T13:26:21.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.888+0000 7f8e399cd740 1 -- start start 2024-04-22T13:26:21.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.888+0000 7f8e399cd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d227882580 0x55d227882960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:21.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.888+0000 7f8e399cd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d227882ea0 0x55d22788b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:21.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.888+0000 7f8e399cd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d22788ba80 0x55d22788de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:21.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.888+0000 7f8e399cd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d2277e71e0 con 0x55d227882580 2024-04-22T13:26:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.888+0000 7f8e399cd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d227784390 con 0x55d22788ba80 2024-04-22T13:26:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.888+0000 7f8e2dffb640 1 --2- 172.21.15.12:0/2450020805 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f8e200a3d20 0x7f8e200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.888+0000 7f8e399cd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d2277e31a0 con 0x55d227882ea0 2024-04-22T13:26:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.888+0000 7f8e34b7c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d227882ea0 0x55d22788b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.888+0000 7f8e34b7c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d227882ea0 0x55d22788b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.12:38596/0 (socket says 172.21.15.12:38596) 2024-04-22T13:26:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.888+0000 7f8e34b7c640 1 -- 172.21.15.12:0/2459519870 learned_addr learned my addr 172.21.15.12:0/2459519870 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.888+0000 7f8e3537d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d227882580 0x55d227882960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.888+0000 7f8e2ffff640 1 --2- 172.21.15.12:0/2459519870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d22788ba80 0x55d22788de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.888+0000 7f8e2dffb640 1 -- 172.21.15.12:0/2450020805 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f8e240f2610 con 0x7f8e301021a0 2024-04-22T13:26:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.888+0000 7f8e3537d640 1 --2- 172.21.15.12:0/2450020805 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f8e200a3d20 0x7f8e200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.888+0000 7f8e34b7c640 1 -- 172.21.15.12:0/2459519870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d22788ba80 msgr2=0x55d22788de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.888+0000 7f8e34b7c640 1 --2- 172.21.15.12:0/2459519870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d22788ba80 0x55d22788de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.888+0000 7f8e34b7c640 1 -- 172.21.15.12:0/2459519870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d227882580 msgr2=0x55d227882960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:21.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.889+0000 7f8e34b7c640 1 --2- 172.21.15.12:0/2459519870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d227882580 0x55d227882960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.889+0000 7f8e34b7c640 1 -- 172.21.15.12:0/2459519870 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2275c37d0 con 0x55d227882ea0 2024-04-22T13:26:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.889+0000 7f8e3537d640 1 --2- 172.21.15.12:0/2459519870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d227882580 0x55d227882960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.889+0000 7f8e34b7c640 1 --2- 172.21.15.12:0/2459519870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d227882ea0 0x55d22788b540 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8e240c6e50 tx=0x7f8e24091320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ec42cfd47bb1afd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.889+0000 7f8e1affd640 1 -- 172.21.15.12:0/2459519870 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e240bee10 con 0x55d227882ea0 2024-04-22T13:26:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.889+0000 7f8e1affd640 1 -- 172.21.15.12:0/2459519870 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e2408f070 con 0x55d227882ea0 2024-04-22T13:26:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.889+0000 7f8e3537d640 1 --2- 172.21.15.12:0/2450020805 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f8e200a3d20 0x7f8e200a61e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8e28009c30 tx=0x7f8e28009500 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.889+0000 7f8e1affd640 1 -- 172.21.15.12:0/2459519870 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e240bad80 con 0x55d227882ea0 2024-04-22T13:26:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.890+0000 7f8e399cd740 1 -- 172.21.15.12:0/2459519870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d227882ea0 msgr2=0x55d22788b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.890+0000 7f8e399cd740 1 --2- 172.21.15.12:0/2459519870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d227882ea0 0x55d22788b540 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8e240c6e50 tx=0x7f8e24091320 comp rx=0 tx=0).stop 2024-04-22T13:26:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.890+0000 7f8e399cd740 1 -- 172.21.15.12:0/2459519870 shutdown_connections 2024-04-22T13:26:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.890+0000 7f8e399cd740 1 --2- 172.21.15.12:0/2459519870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d22788ba80 0x55d22788de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.890+0000 7f8e399cd740 1 --2- 172.21.15.12:0/2459519870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d227882ea0 0x55d22788b540 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.890+0000 7f8e399cd740 1 --2- 172.21.15.12:0/2459519870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d227882580 0x55d227882960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.890+0000 7f8e399cd740 1 -- 172.21.15.12:0/2459519870 >> 172.21.15.12:0/2459519870 conn(0x55d22787ccb0 msgr2=0x55d22787d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:21.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.890+0000 7f8e399cd740 1 -- 172.21.15.12:0/2459519870 shutdown_connections 2024-04-22T13:26:21.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.890+0000 7f8e399cd740 1 -- 172.21.15.12:0/2459519870 wait complete. 2024-04-22T13:26:21.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.890+0000 7f8e399cd740 1 -- start start 2024-04-22T13:26:21.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.890+0000 7f8e399cd740 1 Processor -- start 2024-04-22T13:26:21.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.891+0000 7f8e399cd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d227882580 0x55d227888b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:21.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.891+0000 7f8e399cd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d227882ea0 0x55d2278890a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:21.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.891+0000 7f8e399cd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d22788ba80 0x55d2278895e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:21.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.891+0000 7f8e399cd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d22788e630 con 0x55d227882ea0 2024-04-22T13:26:21.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.891+0000 7f8e399cd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d2277a02f0 con 0x55d22788ba80 2024-04-22T13:26:21.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.891+0000 7f8e399cd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d22788e7b0 con 0x55d227882580 2024-04-22T13:26:21.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.891+0000 7f8e399cd740 10 client.0 _ll_register_callbacks cb 0x55d2277ad580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:21.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.891+0000 7f8e34b7c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d227882580 0x55d227888b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:21.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.891+0000 7f8e2ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d227882ea0 0x55d2278890a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:21.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.891+0000 7f8e3537d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d22788ba80 0x55d2278895e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:21.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.891+0000 7f8e34b7c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d227882580 0x55d227888b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.12:38610/0 (socket says 172.21.15.12:38610) 2024-04-22T13:26:21.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.891+0000 7f8e34b7c640 1 -- 172.21.15.12:0/3804468418 learned_addr learned my addr 172.21.15.12:0/3804468418 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:21.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.891+0000 7f8e34b7c640 1 -- 172.21.15.12:0/3804468418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d22788ba80 msgr2=0x55d2278895e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:21.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.891+0000 7f8e34b7c640 1 --2- 172.21.15.12:0/3804468418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d22788ba80 0x55d2278895e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.891+0000 7f8e34b7c640 1 -- 172.21.15.12:0/3804468418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d227882ea0 msgr2=0x55d2278890a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:21.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.891+0000 7f8e34b7c640 1 --2- 172.21.15.12:0/3804468418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d227882ea0 0x55d2278890a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.891+0000 7f8e34b7c640 1 -- 172.21.15.12:0/3804468418 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2274f93f0 con 0x55d227882580 2024-04-22T13:26:21.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.891+0000 7f8e2ffff640 1 --2- 172.21.15.12:0/3804468418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d227882ea0 0x55d2278890a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.892+0000 7f8e34b7c640 1 --2- 172.21.15.12:0/3804468418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d227882580 0x55d227888b60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8e240bab70 tx=0x7f8e240c7380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3444020e68b6bc9a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.892+0000 7f8e0ffff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-22T13:26:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.892+0000 7f8e0ffff640 1 -- 172.21.15.12:0/3804468418 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e240c2ac0 con 0x55d227882580 2024-04-22T13:26:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.892+0000 7f8e0ffff640 1 -- 172.21.15.12:0/3804468418 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f8e240be970 con 0x55d227882580 2024-04-22T13:26:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.892+0000 7f8e399cd740 1 -- 172.21.15.12:0/3804468418 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e2408f070 con 0x55d227882580 2024-04-22T13:26:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.892+0000 7f8e0ffff640 1 -- 172.21.15.12:0/3804468418 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8e240db030 con 0x55d227882580 2024-04-22T13:26:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.892+0000 7f8e399cd740 10 client.25879 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.892+0000 7f8e399cd740 1 -- 172.21.15.12:0/3804468418 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d2274b90f0 con 0x55d227882580 2024-04-22T13:26:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.892+0000 7f8e0ffff640 1 -- 172.21.15.12:0/3804468418 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e2408f070 con 0x55d227882580 2024-04-22T13:26:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.892+0000 7f8e0ffff640 1 -- 172.21.15.12:0/3804468418 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8e240c6ef0 con 0x55d227882580 2024-04-22T13:26:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.892+0000 7f8e399cd740 10 client.25879 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.892+0000 7f8e399cd740 10 client.25879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:26:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.892+0000 7f8e399cd740 20 client.25879 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.892+0000 7f8e399cd740 1 --2- 172.21.15.12:0/3804468418 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d22789b6f0 0x55d22789dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:21.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.892+0000 7f8e399cd740 4 client.25879 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:26:21.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.892+0000 7f8e399cd740 1 -- 172.21.15.12:0/3804468418 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2277f4c00 con 0x55d22789b6f0 2024-04-22T13:26:21.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.893+0000 7f8e2ffff640 1 --2- 172.21.15.12:0/3804468418 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d22789b6f0 0x55d22789dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:21.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.893+0000 7f8e2ffff640 1 --2- 172.21.15.12:0/3804468418 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d22789b6f0 0x55d22789dae0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cd6fd1339a83c80 server_cookie=c46fd24880f13c71 in_seq=0 out_seq=0 2024-04-22T13:26:21.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.893+0000 7f8e0ffff640 10 client.25879 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:21.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.894+0000 7f8e0ffff640 1 -- 172.21.15.12:0/3804468418 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d2277f4c00 con 0x55d22789b6f0 2024-04-22T13:26:21.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.894+0000 7f8e0ffff640 10 client.25879 handle_command_reply: tid=0 2024-04-22T13:26:21.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.894+0000 7f8e399cd740 1 client.25879 shutdown 2024-04-22T13:26:21.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.894+0000 7f8e399cd740 1 -- 172.21.15.12:0/3804468418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d227882580 msgr2=0x55d227888b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:21.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.894+0000 7f8e399cd740 1 --2- 172.21.15.12:0/3804468418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d227882580 0x55d227888b60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8e240bab70 tx=0x7f8e240c7380 comp rx=0 tx=0).stop 2024-04-22T13:26:21.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.894+0000 7f8e399cd740 1 -- 172.21.15.12:0/3804468418 shutdown_connections 2024-04-22T13:26:21.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.894+0000 7f8e399cd740 1 --2- 172.21.15.12:0/3804468418 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d22789b6f0 0x55d22789dae0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.894+0000 7f8e399cd740 1 --2- 172.21.15.12:0/3804468418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d22788ba80 0x55d2278895e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.894+0000 7f8e399cd740 1 --2- 172.21.15.12:0/3804468418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d227882ea0 0x55d2278890a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.894+0000 7f8e0ffff640 0 client.25879 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:21.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.894+0000 7f8e399cd740 1 --2- 172.21.15.12:0/3804468418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d227882580 0x55d227888b60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.894+0000 7f8e399cd740 1 -- 172.21.15.12:0/3804468418 >> 172.21.15.12:0/3804468418 conn(0x55d22787ccb0 msgr2=0x55d22787ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:21.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.894+0000 7f8e399cd740 1 -- 172.21.15.12:0/3804468418 shutdown_connections 2024-04-22T13:26:21.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.894+0000 7f8e399cd740 1 -- 172.21.15.12:0/3804468418 wait complete. 2024-04-22T13:26:21.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.894+0000 7f8e399cd740 20 client.25879 trim_cache size 0 max 16384 2024-04-22T13:26:21.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.913+0000 7f8e399cd740 1 Processor -- start 2024-04-22T13:26:21.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.913+0000 7f8e399cd740 1 -- start start 2024-04-22T13:26:21.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.914+0000 7f8e399cd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d227882580 0x55d22788b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:21.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.914+0000 7f8e399cd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d227882ea0 0x55d227897440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:21.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.914+0000 7f8e399cd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d22788ba80 0x55d22787bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:21.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.914+0000 7f8e399cd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8e301adb00 con 0x55d227882ea0 2024-04-22T13:26:21.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.914+0000 7f8e399cd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8e301af640 con 0x55d22788ba80 2024-04-22T13:26:21.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.914+0000 7f8e399cd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8e301ae1d0 con 0x55d227882580 2024-04-22T13:26:21.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.914+0000 7f8e2ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d227882ea0 0x55d227897440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:21.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.914+0000 7f8e2ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d227882ea0 0x55d227897440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42986/0 (socket says 172.21.15.12:42986) 2024-04-22T13:26:21.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.914+0000 7f8e2ffff640 1 -- 172.21.15.12:0/1591923317 learned_addr learned my addr 172.21.15.12:0/1591923317 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:21.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.914+0000 7f8e3537d640 1 --2- 172.21.15.12:0/1591923317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d22788ba80 0x55d22787bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:21.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.914+0000 7f8e34b7c640 1 --2- 172.21.15.12:0/1591923317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d227882580 0x55d22788b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:21.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.914+0000 7f8e2ffff640 1 -- 172.21.15.12:0/1591923317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d227882580 msgr2=0x55d22788b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:21.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.914+0000 7f8e2ffff640 1 --2- 172.21.15.12:0/1591923317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d227882580 0x55d22788b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.914+0000 7f8e2ffff640 1 -- 172.21.15.12:0/1591923317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d22788ba80 msgr2=0x55d22787bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:21.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.914+0000 7f8e2ffff640 1 --2- 172.21.15.12:0/1591923317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d22788ba80 0x55d22787bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.914+0000 7f8e2ffff640 1 -- 172.21.15.12:0/1591923317 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2278870b0 con 0x55d227882ea0 2024-04-22T13:26:21.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.914+0000 7f8e34b7c640 1 --2- 172.21.15.12:0/1591923317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d227882580 0x55d22788b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:26:21.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.914+0000 7f8e2ffff640 1 --2- 172.21.15.12:0/1591923317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d227882ea0 0x55d227897440 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x55d22787de40 tx=0x7f8e1c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded874b82e4f9be8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:21.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.914+0000 7f8e1a7fc640 1 -- 172.21.15.12:0/1591923317 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c004230 con 0x55d227882ea0 2024-04-22T13:26:21.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.915+0000 7f8e1a7fc640 1 -- 172.21.15.12:0/1591923317 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e1c0043d0 con 0x55d227882ea0 2024-04-22T13:26:21.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.915+0000 7f8e1a7fc640 1 -- 172.21.15.12:0/1591923317 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c0039e0 con 0x55d227882ea0 2024-04-22T13:26:21.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.915+0000 7f8e399cd740 1 -- 172.21.15.12:0/1591923317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d227882ea0 msgr2=0x55d227897440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:21.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.915+0000 7f8e399cd740 1 --2- 172.21.15.12:0/1591923317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d227882ea0 0x55d227897440 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x55d22787de40 tx=0x7f8e1c005f90 comp rx=0 tx=0).stop 2024-04-22T13:26:21.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.915+0000 7f8e399cd740 1 -- 172.21.15.12:0/1591923317 shutdown_connections 2024-04-22T13:26:21.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.915+0000 7f8e399cd740 1 --2- 172.21.15.12:0/1591923317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d22788ba80 0x55d22787bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.915+0000 7f8e399cd740 1 --2- 172.21.15.12:0/1591923317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d227882ea0 0x55d227897440 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.915+0000 7f8e399cd740 1 --2- 172.21.15.12:0/1591923317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d227882580 0x55d22788b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.915+0000 7f8e399cd740 1 -- 172.21.15.12:0/1591923317 >> 172.21.15.12:0/1591923317 conn(0x55d22787ccb0 msgr2=0x55d22787b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:21.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.915+0000 7f8e399cd740 1 -- 172.21.15.12:0/1591923317 shutdown_connections 2024-04-22T13:26:21.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.915+0000 7f8e399cd740 1 -- 172.21.15.12:0/1591923317 wait complete. 2024-04-22T13:26:21.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.916+0000 7f8e399cd740 1 -- start start 2024-04-22T13:26:21.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.916+0000 7f8e399cd740 1 Processor -- start 2024-04-22T13:26:21.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.916+0000 7f8e399cd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d227882580 0x55d227893a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:21.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.916+0000 7f8e399cd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d227882ea0 0x55d22788ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:21.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.916+0000 7f8e399cd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d22788ba80 0x55d22788f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:21.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.916+0000 7f8e399cd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8e301aef70 con 0x55d22788ba80 2024-04-22T13:26:21.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.916+0000 7f8e399cd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8e301ae8a0 con 0x55d227882580 2024-04-22T13:26:21.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.916+0000 7f8e399cd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8e301abb00 con 0x55d227882ea0 2024-04-22T13:26:21.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.916+0000 7f8e399cd740 10 client.0 _ll_register_callbacks cb 0x7f8e28013480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:21.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.916+0000 7f8e34b7c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d227882580 0x55d227893a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:21.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.917+0000 7f8e2ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d227882ea0 0x55d22788ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:21.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.917+0000 7f8e34b7c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d227882580 0x55d227893a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:43938/0 (socket says 172.21.15.12:43938) 2024-04-22T13:26:21.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.917+0000 7f8e34b7c640 1 -- 172.21.15.12:0/2878401993 learned_addr learned my addr 172.21.15.12:0/2878401993 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:21.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.917+0000 7f8e3537d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d22788ba80 0x55d22788f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:21.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.917+0000 7f8e2ffff640 1 -- 172.21.15.12:0/2878401993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d227882580 msgr2=0x55d227893a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:21.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.917+0000 7f8e2ffff640 1 --2- 172.21.15.12:0/2878401993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d227882580 0x55d227893a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.917+0000 7f8e2ffff640 1 -- 172.21.15.12:0/2878401993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d22788ba80 msgr2=0x55d22788f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:21.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.917+0000 7f8e2ffff640 1 --2- 172.21.15.12:0/2878401993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d22788ba80 0x55d22788f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.917+0000 7f8e2ffff640 1 -- 172.21.15.12:0/2878401993 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d22788f990 con 0x55d227882ea0 2024-04-22T13:26:21.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.917+0000 7f8e3537d640 1 --2- 172.21.15.12:0/2878401993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d22788ba80 0x55d22788f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:21.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.917+0000 7f8e34b7c640 1 --2- 172.21.15.12:0/2878401993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d227882580 0x55d227893a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:21.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.917+0000 7f8e2ffff640 1 --2- 172.21.15.12:0/2878401993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d227882ea0 0x55d22788ef10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55d22788e740 tx=0x7f8e1c004030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13c0db9f66227085 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:21.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.917+0000 7f8e0ffff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-22T13:26:21.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.917+0000 7f8e0ffff640 1 -- 172.21.15.12:0/2878401993 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c005240 con 0x55d227882ea0 2024-04-22T13:26:21.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.917+0000 7f8e0ffff640 1 -- 172.21.15.12:0/2878401993 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f8e1c0053e0 con 0x55d227882ea0 2024-04-22T13:26:21.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.917+0000 7f8e399cd740 1 -- 172.21.15.12:0/2878401993 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e1c0043d0 con 0x55d227882ea0 2024-04-22T13:26:21.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.918+0000 7f8e0ffff640 1 -- 172.21.15.12:0/2878401993 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8e1c004220 con 0x55d227882ea0 2024-04-22T13:26:21.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.918+0000 7f8e399cd740 10 client.25885 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:21.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.918+0000 7f8e399cd740 1 -- 172.21.15.12:0/2878401993 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d227901890 con 0x55d227882ea0 2024-04-22T13:26:21.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.918+0000 7f8e0ffff640 1 -- 172.21.15.12:0/2878401993 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e1c0043d0 con 0x55d227882ea0 2024-04-22T13:26:21.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.918+0000 7f8e0ffff640 1 -- 172.21.15.12:0/2878401993 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8e1c0203e0 con 0x55d227882ea0 2024-04-22T13:26:21.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.918+0000 7f8e399cd740 10 client.25885 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:21.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.918+0000 7f8e399cd740 10 client.25885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:26:21.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.918+0000 7f8e399cd740 20 client.25885 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:21.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.918+0000 7f8e399cd740 1 --2- 172.21.15.12:0/2878401993 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d227901f50 0x55d227904370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:21.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.918+0000 7f8e399cd740 4 client.25885 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:26:21.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.918+0000 7f8e399cd740 1 -- 172.21.15.12:0/2878401993 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d2277eea70 con 0x55d227901f50 2024-04-22T13:26:21.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.918+0000 7f8e34b7c640 1 --2- 172.21.15.12:0/2878401993 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d227901f50 0x55d227904370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:21.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.919+0000 7f8e34b7c640 1 --2- 172.21.15.12:0/2878401993 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d227901f50 0x55d227904370 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b17c14d5cca5fe server_cookie=1de9948dac89fb62 in_seq=0 out_seq=0 2024-04-22T13:26:21.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.919+0000 7f8e0ffff640 10 client.25885 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:21.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.919+0000 7f8e0ffff640 1 -- 172.21.15.12:0/2878401993 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d2277eea70 con 0x55d227901f50 2024-04-22T13:26:21.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.919+0000 7f8e0ffff640 10 client.25885 handle_command_reply: tid=0 2024-04-22T13:26:21.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.919+0000 7f8e399cd740 1 client.25885 shutdown 2024-04-22T13:26:21.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.919+0000 7f8e399cd740 1 -- 172.21.15.12:0/2878401993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d227882ea0 msgr2=0x55d22788ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:21.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.919+0000 7f8e399cd740 1 --2- 172.21.15.12:0/2878401993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d227882ea0 0x55d22788ef10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55d22788e740 tx=0x7f8e1c004030 comp rx=0 tx=0).stop 2024-04-22T13:26:21.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.919+0000 7f8e399cd740 1 -- 172.21.15.12:0/2878401993 shutdown_connections 2024-04-22T13:26:21.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.919+0000 7f8e399cd740 1 --2- 172.21.15.12:0/2878401993 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d227901f50 0x55d227904370 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.919+0000 7f8e399cd740 1 --2- 172.21.15.12:0/2878401993 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d22788ba80 0x55d22788f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.919+0000 7f8e0ffff640 0 client.25885 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:21.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.919+0000 7f8e399cd740 1 --2- 172.21.15.12:0/2878401993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d227882ea0 0x55d22788ef10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.919+0000 7f8e399cd740 1 --2- 172.21.15.12:0/2878401993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d227882580 0x55d227893a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.919+0000 7f8e399cd740 1 -- 172.21.15.12:0/2878401993 >> 172.21.15.12:0/2878401993 conn(0x55d22787ccb0 msgr2=0x55d22787bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:21.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.919+0000 7f8e399cd740 1 -- 172.21.15.12:0/2878401993 shutdown_connections 2024-04-22T13:26:21.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.919+0000 7f8e399cd740 1 -- 172.21.15.12:0/2878401993 wait complete. 2024-04-22T13:26:21.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.919+0000 7f8e399cd740 20 client.25885 trim_cache size 0 max 16384 2024-04-22T13:26:21.922 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:26:21.922 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-04-22T13:26:21.922 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-04-22T13:26:21.922 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:26:21.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.920+0000 7f8e36de0640 1 -- 172.21.15.12:0/2450020805 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f8e200a3d20 msgr2=0x7f8e200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:21.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.920+0000 7f8e36de0640 1 --2- 172.21.15.12:0/2450020805 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f8e200a3d20 0x7f8e200a61e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8e28009c30 tx=0x7f8e28009500 comp rx=0 tx=0).stop 2024-04-22T13:26:21.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.920+0000 7f8e36de0640 1 -- 172.21.15.12:0/2450020805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e301021a0 msgr2=0x7f8e301987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:21.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.920+0000 7f8e36de0640 1 --2- 172.21.15.12:0/2450020805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e301021a0 0x7f8e301987b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8e24036040 tx=0x7f8e240045e0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.920+0000 7f8e36de0640 1 -- 172.21.15.12:0/2450020805 shutdown_connections 2024-04-22T13:26:21.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.920+0000 7f8e36de0640 1 --2- 172.21.15.12:0/2450020805 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f8e200a3d20 0x7f8e200a61e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.920+0000 7f8e36de0640 1 --2- 172.21.15.12:0/2450020805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e3019d020 0x7f8e3019d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.920+0000 7f8e36de0640 1 --2- 172.21.15.12:0/2450020805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e30107270 0x7f8e30198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.920+0000 7f8e36de0640 1 --2- 172.21.15.12:0/2450020805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e301021a0 0x7f8e301987b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:21.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.920+0000 7f8e36de0640 1 -- 172.21.15.12:0/2450020805 >> 172.21.15.12:0/2450020805 conn(0x7f8e300fda40 msgr2=0x7f8e300feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:21.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.920+0000 7f8e36de0640 1 -- 172.21.15.12:0/2450020805 shutdown_connections 2024-04-22T13:26:21.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:21.920+0000 7f8e36de0640 1 -- 172.21.15.12:0/2450020805 wait complete. 2024-04-22T13:26:21.929 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:26:21.929 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:26:22.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.044+0000 7f5d338e4640 1 -- 172.21.15.12:0/670574894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d2c0690e0 msgr2=0x7f5d2c069540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.044+0000 7f5d338e4640 1 --2- 172.21.15.12:0/670574894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d2c0690e0 0x7f5d2c069540 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c009930 tx=0x7f5d1c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:22.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.045+0000 7f5d338e4640 1 -- 172.21.15.12:0/670574894 shutdown_connections 2024-04-22T13:26:22.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.045+0000 7f5d338e4640 1 --2- 172.21.15.12:0/670574894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d2c069a80 0x7f5d2c10ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.045+0000 7f5d338e4640 1 --2- 172.21.15.12:0/670574894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d2c0690e0 0x7f5d2c069540 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.045+0000 7f5d338e4640 1 --2- 172.21.15.12:0/670574894 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d2c107f70 0x7f5d2c108350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.045+0000 7f5d338e4640 1 -- 172.21.15.12:0/670574894 >> 172.21.15.12:0/670574894 conn(0x7f5d2c077e30 msgr2=0x7f5d2c078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:22.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.045+0000 7f5d338e4640 1 -- 172.21.15.12:0/670574894 shutdown_connections 2024-04-22T13:26:22.042 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.045+0000 7f5d338e4640 1 -- 172.21.15.12:0/670574894 wait complete. 2024-04-22T13:26:22.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.045+0000 7f5d338e4640 1 Processor -- start 2024-04-22T13:26:22.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.046+0000 7f5d338e4640 1 -- start start 2024-04-22T13:26:22.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.046+0000 7f5d338e4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d2c0690e0 0x7f5d2c101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.046+0000 7f5d338e4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d2c069a80 0x7f5d2c1017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.046+0000 7f5d338e4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d2c107f70 0x7f5d2c101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.046+0000 7f5d338e4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c112130 con 0x7f5d2c0690e0 2024-04-22T13:26:22.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.046+0000 7f5d338e4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c111fb0 con 0x7f5d2c107f70 2024-04-22T13:26:22.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.046+0000 7f5d338e4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c1122b0 con 0x7f5d2c069a80 2024-04-22T13:26:22.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.046+0000 7f5d31680640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d2c0690e0 0x7f5d2c101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.046+0000 7f5d31680640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d2c0690e0 0x7f5d2c101270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43020/0 (socket says 172.21.15.12:43020) 2024-04-22T13:26:22.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.046+0000 7f5d31680640 1 -- 172.21.15.12:0/650984596 learned_addr learned my addr 172.21.15.12:0/650984596 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:22.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.046+0000 7f5d31e81640 1 --2- 172.21.15.12:0/650984596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d2c107f70 0x7f5d2c101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.046+0000 7f5d30e7f640 1 --2- 172.21.15.12:0/650984596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d2c069a80 0x7f5d2c1017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.046+0000 7f5d31680640 1 -- 172.21.15.12:0/650984596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d2c069a80 msgr2=0x7f5d2c1017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.046+0000 7f5d31680640 1 --2- 172.21.15.12:0/650984596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d2c069a80 0x7f5d2c1017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.046+0000 7f5d31680640 1 -- 172.21.15.12:0/650984596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d2c107f70 msgr2=0x7f5d2c101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.046+0000 7f5d31680640 1 --2- 172.21.15.12:0/650984596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d2c107f70 0x7f5d2c101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.046+0000 7f5d31680640 1 -- 172.21.15.12:0/650984596 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d2c1a6bb0 con 0x7f5d2c0690e0 2024-04-22T13:26:22.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.046+0000 7f5d31680640 1 --2- 172.21.15.12:0/650984596 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d2c0690e0 0x7f5d2c101270 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800e9f0 tx=0x7f5d2800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:22.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.047+0000 7f5d227fc640 1 -- 172.21.15.12:0/650984596 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2800ccc0 con 0x7f5d2c0690e0 2024-04-22T13:26:22.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.047+0000 7f5d227fc640 1 -- 172.21.15.12:0/650984596 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d280108e0 con 0x7f5d2c0690e0 2024-04-22T13:26:22.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.047+0000 7f5d338e4640 1 -- 172.21.15.12:0/650984596 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d2c1a6ea0 con 0x7f5d2c0690e0 2024-04-22T13:26:22.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.047+0000 7f5d227fc640 1 -- 172.21.15.12:0/650984596 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d28018880 con 0x7f5d2c0690e0 2024-04-22T13:26:22.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.047+0000 7f5d338e4640 1 -- 172.21.15.12:0/650984596 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d2c1a7310 con 0x7f5d2c0690e0 2024-04-22T13:26:22.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.048+0000 7f5d338e4640 1 -- 172.21.15.12:0/650984596 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d2c068de0 con 0x7f5d2c0690e0 2024-04-22T13:26:22.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.049+0000 7f5d227fc640 1 -- 172.21.15.12:0/650984596 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5d28010430 con 0x7f5d2c0690e0 2024-04-22T13:26:22.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.050+0000 7f5d227fc640 1 --2- 172.21.15.12:0/650984596 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5d180a3d20 0x7f5d180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.050+0000 7f5d30e7f640 1 --2- 172.21.15.12:0/650984596 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5d180a3d20 0x7f5d180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.050+0000 7f5d227fc640 1 -- 172.21.15.12:0/650984596 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f5d280d0f40 con 0x7f5d2c0690e0 2024-04-22T13:26:22.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.050+0000 7f5d30e7f640 1 --2- 172.21.15.12:0/650984596 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5d180a3d20 0x7f5d180a61e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c005ab0 tx=0x7f5d1c005fd0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:22.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.054+0000 7f5d227fc640 1 -- 172.21.15.12:0/650984596 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5d2809d630 con 0x7f5d2c0690e0 2024-04-22T13:26:22.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:21 smithi012 ceph-mon[28265]: pgmap v70: 97 pgs: 97 active+clean; 259 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.4 MiB/s wr, 749 op/s 2024-04-22T13:26:22.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:21 smithi134 ceph-mon[31338]: pgmap v70: 97 pgs: 97 active+clean; 259 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.4 MiB/s wr, 749 op/s 2024-04-22T13:26:22.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:21 smithi111 ceph-mon[31361]: pgmap v70: 97 pgs: 97 active+clean; 259 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.4 MiB/s wr, 749 op/s 2024-04-22T13:26:22.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.319+0000 7f5d338e4640 1 -- 172.21.15.12:0/650984596 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d2c1027a0 con 0x7f5d2c0690e0 2024-04-22T13:26:22.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.320+0000 7f5d227fc640 1 -- 172.21.15.12:0/650984596 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7f5d2c1027a0 con 0x7f5d2c0690e0 2024-04-22T13:26:22.317 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:26:22.317 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:26:22.317 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:26:22.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.321+0000 7f5d338e4640 1 -- 172.21.15.12:0/650984596 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5d180a3d20 msgr2=0x7f5d180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.321+0000 7f5d338e4640 1 --2- 172.21.15.12:0/650984596 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5d180a3d20 0x7f5d180a61e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c005ab0 tx=0x7f5d1c005fd0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.322+0000 7f5d338e4640 1 -- 172.21.15.12:0/650984596 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d2c0690e0 msgr2=0x7f5d2c101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.322+0000 7f5d338e4640 1 --2- 172.21.15.12:0/650984596 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d2c0690e0 0x7f5d2c101270 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800e9f0 tx=0x7f5d2800edf0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.322+0000 7f5d338e4640 1 -- 172.21.15.12:0/650984596 shutdown_connections 2024-04-22T13:26:22.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.322+0000 7f5d338e4640 1 --2- 172.21.15.12:0/650984596 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5d180a3d20 0x7f5d180a61e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.322+0000 7f5d338e4640 1 --2- 172.21.15.12:0/650984596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d2c107f70 0x7f5d2c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.322+0000 7f5d338e4640 1 --2- 172.21.15.12:0/650984596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d2c069a80 0x7f5d2c1017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.322+0000 7f5d338e4640 1 --2- 172.21.15.12:0/650984596 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d2c0690e0 0x7f5d2c101270 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.322+0000 7f5d338e4640 1 -- 172.21.15.12:0/650984596 >> 172.21.15.12:0/650984596 conn(0x7f5d2c077e30 msgr2=0x7f5d2c1000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:22.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.322+0000 7f5d338e4640 1 -- 172.21.15.12:0/650984596 shutdown_connections 2024-04-22T13:26:22.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.322+0000 7f5d338e4640 1 -- 172.21.15.12:0/650984596 wait complete. 2024-04-22T13:26:22.331 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:26:22.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.448+0000 7f78682a8640 1 -- 172.21.15.12:0/3775177480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f786010c510 msgr2=0x7f786010e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.448+0000 7f78682a8640 1 --2- 172.21.15.12:0/3775177480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f786010c510 0x7f786010e9b0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f7858009ca0 tx=0x7f785802c230 comp rx=0 tx=0).stop 2024-04-22T13:26:22.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.448+0000 7f78682a8640 1 -- 172.21.15.12:0/3775177480 shutdown_connections 2024-04-22T13:26:22.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.448+0000 7f78682a8640 1 --2- 172.21.15.12:0/3775177480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f786010c510 0x7f786010e9b0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.448+0000 7f78682a8640 1 --2- 172.21.15.12:0/3775177480 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7860103af0 0x7f786010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.448+0000 7f78682a8640 1 --2- 172.21.15.12:0/3775177480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78601031d0 0x7f78601035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.448+0000 7f78682a8640 1 -- 172.21.15.12:0/3775177480 >> 172.21.15.12:0/3775177480 conn(0x7f7860077f50 msgr2=0x7f7860078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:22.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.449+0000 7f78682a8640 1 -- 172.21.15.12:0/3775177480 shutdown_connections 2024-04-22T13:26:22.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.449+0000 7f78682a8640 1 -- 172.21.15.12:0/3775177480 wait complete. 2024-04-22T13:26:22.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.449+0000 7f78682a8640 1 Processor -- start 2024-04-22T13:26:22.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.449+0000 7f78682a8640 1 -- start start 2024-04-22T13:26:22.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.449+0000 7f78682a8640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78601031d0 0x7f7860198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.449+0000 7f78682a8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7860103af0 0x7f7860198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.449+0000 7f78682a8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f786010c510 0x7f786019cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.449+0000 7f78682a8640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f78601121a0 con 0x7f786010c510 2024-04-22T13:26:22.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.449+0000 7f78682a8640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7860112020 con 0x7f78601031d0 2024-04-22T13:26:22.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.449+0000 7f78682a8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7860112320 con 0x7f7860103af0 2024-04-22T13:26:22.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.450+0000 7f7866044640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78601031d0 0x7f7860198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.450+0000 7f7866044640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78601031d0 0x7f7860198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:43974/0 (socket says 172.21.15.12:43974) 2024-04-22T13:26:22.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.450+0000 7f7866845640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f786010c510 0x7f786019cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.450+0000 7f7865843640 1 --2- 172.21.15.12:0/1256711036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7860103af0 0x7f7860198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.450+0000 7f7866044640 1 -- 172.21.15.12:0/1256711036 learned_addr learned my addr 172.21.15.12:0/1256711036 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:22.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.450+0000 7f7866044640 1 -- 172.21.15.12:0/1256711036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7860103af0 msgr2=0x7f7860198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.450+0000 7f7866044640 1 --2- 172.21.15.12:0/1256711036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7860103af0 0x7f7860198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.450+0000 7f7866044640 1 -- 172.21.15.12:0/1256711036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f786010c510 msgr2=0x7f786019cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.450+0000 7f7866044640 1 --2- 172.21.15.12:0/1256711036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f786010c510 0x7f786019cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.450+0000 7f7866044640 1 -- 172.21.15.12:0/1256711036 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f786019d6f0 con 0x7f78601031d0 2024-04-22T13:26:22.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.450+0000 7f7866845640 1 --2- 172.21.15.12:0/1256711036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f786010c510 0x7f786019cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:22.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.450+0000 7f7866044640 1 --2- 172.21.15.12:0/1256711036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78601031d0 0x7f7860198700 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f785c00e9f0 tx=0x7f785c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:22.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.450+0000 7f78577fe640 1 -- 172.21.15.12:0/1256711036 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c00cd80 con 0x7f78601031d0 2024-04-22T13:26:22.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.450+0000 7f78682a8640 1 -- 172.21.15.12:0/1256711036 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f786019d9e0 con 0x7f78601031d0 2024-04-22T13:26:22.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.450+0000 7f78577fe640 1 -- 172.21.15.12:0/1256711036 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f785c0054d0 con 0x7f78601031d0 2024-04-22T13:26:22.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.450+0000 7f78577fe640 1 -- 172.21.15.12:0/1256711036 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c010690 con 0x7f78601031d0 2024-04-22T13:26:22.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.450+0000 7f78682a8640 1 -- 172.21.15.12:0/1256711036 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78601a5270 con 0x7f78601031d0 2024-04-22T13:26:22.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.453+0000 7f78577fe640 1 -- 172.21.15.12:0/1256711036 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f785c010830 con 0x7f78601031d0 2024-04-22T13:26:22.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.453+0000 7f786af7b740 1 Processor -- start 2024-04-22T13:26:22.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.453+0000 7f786af7b740 1 -- start start 2024-04-22T13:26:22.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.454+0000 7f786af7b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649c19b84d0 0x5649c19b88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.454+0000 7f786af7b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649c19b8e80 0x5649c19c1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.454+0000 7f786af7b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649c19c1a00 0x5649c19c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.454+0000 7f786af7b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5649c191d1f0 con 0x5649c19c1a00 2024-04-22T13:26:22.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.454+0000 7f786af7b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5649c18ba4b0 con 0x5649c19b8e80 2024-04-22T13:26:22.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.454+0000 7f786af7b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5649c19191b0 con 0x5649c19b84d0 2024-04-22T13:26:22.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.454+0000 7f78577fe640 1 --2- 172.21.15.12:0/1256711036 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f784c0a3d20 0x7f784c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.454+0000 7f7866044640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649c19b8e80 0x5649c19c1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.454+0000 7f7866845640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649c19b84d0 0x5649c19b88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.454+0000 7f7866845640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649c19b84d0 0x5649c19b88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.12:38666/0 (socket says 172.21.15.12:38666) 2024-04-22T13:26:22.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.454+0000 7f7866845640 1 -- 172.21.15.12:0/4223819572 learned_addr learned my addr 172.21.15.12:0/4223819572 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:22.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.454+0000 7f7865843640 1 --2- 172.21.15.12:0/4223819572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649c19c1a00 0x5649c19c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.454+0000 7f7866845640 1 --2- 172.21.15.12:0/1256711036 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f784c0a3d20 0x7f784c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.454+0000 7f7866044640 1 -- 172.21.15.12:0/4223819572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649c19b84d0 msgr2=0x5649c19b88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.454+0000 7f7866044640 1 --2- 172.21.15.12:0/4223819572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649c19b84d0 0x5649c19b88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.454+0000 7f7866044640 1 -- 172.21.15.12:0/4223819572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649c19c1a00 msgr2=0x5649c19c3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.454+0000 7f7866044640 1 --2- 172.21.15.12:0/4223819572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649c19c1a00 0x5649c19c3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.454+0000 7f78577fe640 1 -- 172.21.15.12:0/1256711036 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f785c0d0be0 con 0x7f78601031d0 2024-04-22T13:26:22.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.454+0000 7f7866044640 1 -- 172.21.15.12:0/4223819572 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649c18ac1b0 con 0x5649c19b8e80 2024-04-22T13:26:22.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.454+0000 7f7865843640 1 --2- 172.21.15.12:0/4223819572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649c19c1a00 0x5649c19c3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:22.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.454+0000 7f7866845640 1 --2- 172.21.15.12:0/1256711036 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f784c0a3d20 0x7f784c0a61e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7858037040 tx=0x7f7858005c50 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:22.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.454+0000 7f7866044640 1 --2- 172.21.15.12:0/4223819572 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649c19b8e80 0x5649c19c1430 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f785c0ce380 tx=0x7f785c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ec5f7f7d31f4f8f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:22.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.455+0000 7f7854bfe640 1 -- 172.21.15.12:0/4223819572 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c0a0f80 con 0x5649c19b8e80 2024-04-22T13:26:22.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.455+0000 7f7854bfe640 1 -- 172.21.15.12:0/4223819572 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f785c0a5840 con 0x5649c19b8e80 2024-04-22T13:26:22.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.455+0000 7f7854bfe640 1 -- 172.21.15.12:0/4223819572 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c0bd230 con 0x5649c19b8e80 2024-04-22T13:26:22.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.455+0000 7f786af7b740 1 -- 172.21.15.12:0/4223819572 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649c19b8e80 msgr2=0x5649c19c1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.455+0000 7f786af7b740 1 --2- 172.21.15.12:0/4223819572 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649c19b8e80 0x5649c19c1430 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f785c0ce380 tx=0x7f785c014000 comp rx=0 tx=0).stop 2024-04-22T13:26:22.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.456+0000 7f786af7b740 1 -- 172.21.15.12:0/4223819572 shutdown_connections 2024-04-22T13:26:22.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.456+0000 7f786af7b740 1 --2- 172.21.15.12:0/4223819572 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649c19c1a00 0x5649c19c3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.456+0000 7f786af7b740 1 --2- 172.21.15.12:0/4223819572 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649c19b8e80 0x5649c19c1430 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.456+0000 7f786af7b740 1 --2- 172.21.15.12:0/4223819572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649c19b84d0 0x5649c19b88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.456+0000 7f786af7b740 1 -- 172.21.15.12:0/4223819572 >> 172.21.15.12:0/4223819572 conn(0x5649c19b2ba0 msgr2=0x5649c19b2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:22.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.456+0000 7f786af7b740 1 -- 172.21.15.12:0/4223819572 shutdown_connections 2024-04-22T13:26:22.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.456+0000 7f786af7b740 1 -- 172.21.15.12:0/4223819572 wait complete. 2024-04-22T13:26:22.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.456+0000 7f786af7b740 1 -- start start 2024-04-22T13:26:22.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.456+0000 7f786af7b740 1 Processor -- start 2024-04-22T13:26:22.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.456+0000 7f786af7b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649c19b84d0 0x5649c19c1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.456+0000 7f786af7b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649c19b8e80 0x5649c19be5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.456+0000 7f786af7b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649c19c1a00 0x5649c19beb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.457+0000 7f786af7b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5649c19c48b0 con 0x5649c19c1a00 2024-04-22T13:26:22.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.457+0000 7f786af7b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5649c18d6500 con 0x5649c19b8e80 2024-04-22T13:26:22.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.457+0000 7f786af7b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5649c19c4a30 con 0x5649c19b84d0 2024-04-22T13:26:22.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.457+0000 7f7866044640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649c19b84d0 0x5649c19c1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.457+0000 7f7866845640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649c19c1a00 0x5649c19beb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.457+0000 7f7865843640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649c19b8e80 0x5649c19be5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.457+0000 7f7866044640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649c19b84d0 0x5649c19c1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.12:38680/0 (socket says 172.21.15.12:38680) 2024-04-22T13:26:22.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.457+0000 7f786af7b740 10 client.0 _ll_register_callbacks cb 0x5649c18a4b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:22.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.457+0000 7f7866044640 1 -- 172.21.15.12:0/443421596 learned_addr learned my addr 172.21.15.12:0/443421596 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:22.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.457+0000 7f7865843640 1 -- 172.21.15.12:0/443421596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649c19b84d0 msgr2=0x5649c19c1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.457+0000 7f7865843640 1 --2- 172.21.15.12:0/443421596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649c19b84d0 0x5649c19c1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.457+0000 7f7865843640 1 -- 172.21.15.12:0/443421596 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649c19c1a00 msgr2=0x5649c19beb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.457+0000 7f7865843640 1 --2- 172.21.15.12:0/443421596 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649c19c1a00 0x5649c19beb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.457+0000 7f7865843640 1 -- 172.21.15.12:0/443421596 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649c162f3f0 con 0x5649c19b8e80 2024-04-22T13:26:22.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.457+0000 7f7866845640 1 --2- 172.21.15.12:0/443421596 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649c19c1a00 0x5649c19beb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:22.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.457+0000 7f7865843640 1 --2- 172.21.15.12:0/443421596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649c19b8e80 0x5649c19be5f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7850013a50 tx=0x7f7850013f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d2e3ec3f97dc66c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:22.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.457+0000 7f7849ffb640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-22T13:26:22.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.457+0000 7f7849ffb640 1 -- 172.21.15.12:0/443421596 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785000ee20 con 0x5649c19b8e80 2024-04-22T13:26:22.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.457+0000 7f7849ffb640 1 -- 172.21.15.12:0/443421596 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f78500047a0 con 0x5649c19b8e80 2024-04-22T13:26:22.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.457+0000 7f786af7b740 1 -- 172.21.15.12:0/443421596 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f785c0a5840 con 0x5649c19b8e80 2024-04-22T13:26:22.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.458+0000 7f7849ffb640 1 -- 172.21.15.12:0/443421596 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7850004020 con 0x5649c19b8e80 2024-04-22T13:26:22.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.458+0000 7f786af7b740 10 client.25967 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:22.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.458+0000 7f786af7b740 1 -- 172.21.15.12:0/443421596 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5649c16f97d0 con 0x5649c19b8e80 2024-04-22T13:26:22.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.458+0000 7f7849ffb640 1 -- 172.21.15.12:0/443421596 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f785c0a5840 con 0x5649c19b8e80 2024-04-22T13:26:22.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.458+0000 7f7849ffb640 1 -- 172.21.15.12:0/443421596 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f78500052c0 con 0x5649c19b8e80 2024-04-22T13:26:22.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.458+0000 7f786af7b740 10 client.25967 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:22.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.458+0000 7f786af7b740 10 client.25967 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:26:22.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.458+0000 7f786af7b740 20 client.25967 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:22.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.458+0000 7f786af7b740 1 --2- 172.21.15.12:0/443421596 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5649c19d1b60 0x5649c19d3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.458+0000 7f786af7b740 4 client.25967 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:26:22.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.458+0000 7f786af7b740 1 -- 172.21.15.12:0/443421596 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649c18bc930 con 0x5649c19d1b60 2024-04-22T13:26:22.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.458+0000 7f7866044640 1 --2- 172.21.15.12:0/443421596 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5649c19d1b60 0x5649c19d3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.459+0000 7f7866044640 1 --2- 172.21.15.12:0/443421596 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5649c19d1b60 0x5649c19d3f50 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2df5498f4d8c646c server_cookie=e4d0cb63fc72b3c0 in_seq=0 out_seq=0 2024-04-22T13:26:22.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.459+0000 7f7849ffb640 10 client.25967 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:26:22.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.459+0000 7f7849ffb640 1 -- 172.21.15.12:0/443421596 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5649c18bc930 con 0x5649c19d1b60 2024-04-22T13:26:22.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.459+0000 7f7849ffb640 10 client.25967 handle_command_reply: tid=0 2024-04-22T13:26:22.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.459+0000 7f786af7b740 1 client.25967 shutdown 2024-04-22T13:26:22.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.459+0000 7f786af7b740 1 -- 172.21.15.12:0/443421596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649c19b8e80 msgr2=0x5649c19be5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.459+0000 7f786af7b740 1 --2- 172.21.15.12:0/443421596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649c19b8e80 0x5649c19be5f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7850013a50 tx=0x7f7850013f10 comp rx=0 tx=0).stop 2024-04-22T13:26:22.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.459+0000 7f786af7b740 1 -- 172.21.15.12:0/443421596 shutdown_connections 2024-04-22T13:26:22.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.459+0000 7f786af7b740 1 --2- 172.21.15.12:0/443421596 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5649c19d1b60 0x5649c19d3f50 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.459+0000 7f786af7b740 1 --2- 172.21.15.12:0/443421596 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649c19c1a00 0x5649c19beb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.459+0000 7f786af7b740 1 --2- 172.21.15.12:0/443421596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649c19b8e80 0x5649c19be5f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.459+0000 7f786af7b740 1 --2- 172.21.15.12:0/443421596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649c19b84d0 0x5649c19c1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.459+0000 7f786af7b740 1 -- 172.21.15.12:0/443421596 >> 172.21.15.12:0/443421596 conn(0x5649c19b2ba0 msgr2=0x5649c19b48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:22.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.459+0000 7f7849ffb640 0 client.25967 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:26:22.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.460+0000 7f786af7b740 1 -- 172.21.15.12:0/443421596 shutdown_connections 2024-04-22T13:26:22.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.460+0000 7f786af7b740 1 -- 172.21.15.12:0/443421596 wait complete. 2024-04-22T13:26:22.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.460+0000 7f786af7b740 20 client.25967 trim_cache size 0 max 16384 2024-04-22T13:26:22.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.477+0000 7f786af7b740 1 Processor -- start 2024-04-22T13:26:22.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.477+0000 7f786af7b740 1 -- start start 2024-04-22T13:26:22.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.477+0000 7f786af7b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649c19b2ba0 0x5649c19cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.477+0000 7f786af7b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649c19b84d0 0x5649c19c0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.477+0000 7f786af7b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649c19b8e80 0x5649c19c1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.477+0000 7f786af7b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f78601ab960 con 0x5649c19b8e80 2024-04-22T13:26:22.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.477+0000 7f786af7b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f78601ad4a0 con 0x5649c19b84d0 2024-04-22T13:26:22.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.477+0000 7f786af7b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f78601ac030 con 0x5649c19b2ba0 2024-04-22T13:26:22.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.478+0000 7f7866044640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649c19b2ba0 0x5649c19cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.478+0000 7f7865843640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649c19b84d0 0x5649c19c0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.478+0000 7f7866845640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649c19b8e80 0x5649c19c1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.478+0000 7f7866044640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649c19b2ba0 0x5649c19cd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.12:38692/0 (socket says 172.21.15.12:38692) 2024-04-22T13:26:22.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.478+0000 7f7866845640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649c19b8e80 0x5649c19c1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43066/0 (socket says 172.21.15.12:43066) 2024-04-22T13:26:22.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.478+0000 7f7865843640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649c19b84d0 0x5649c19c0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:43998/0 (socket says 172.21.15.12:43998) 2024-04-22T13:26:22.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.478+0000 7f7866044640 1 -- 172.21.15.12:0/3531065519 learned_addr learned my addr 172.21.15.12:0/3531065519 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:22.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.478+0000 7f7865843640 1 -- 172.21.15.12:0/3531065519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649c19b2ba0 msgr2=0x5649c19cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.478+0000 7f7865843640 1 --2- 172.21.15.12:0/3531065519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649c19b2ba0 0x5649c19cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.478+0000 7f7865843640 1 -- 172.21.15.12:0/3531065519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649c19b8e80 msgr2=0x5649c19c1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.478+0000 7f7865843640 1 --2- 172.21.15.12:0/3531065519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649c19b8e80 0x5649c19c1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.478+0000 7f7865843640 1 -- 172.21.15.12:0/3531065519 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649c19b8c20 con 0x5649c19b84d0 2024-04-22T13:26:22.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.478+0000 7f7865843640 1 --2- 172.21.15.12:0/3531065519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649c19b84d0 0x5649c19c0d20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7850005700 tx=0x7f7850003870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f1fe680d22d05de server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:22.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.478+0000 7f784bfff640 1 -- 172.21.15.12:0/3531065519 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78500041c0 con 0x5649c19b84d0 2024-04-22T13:26:22.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.478+0000 7f784bfff640 1 -- 172.21.15.12:0/3531065519 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7850003d90 con 0x5649c19b84d0 2024-04-22T13:26:22.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.478+0000 7f784bfff640 1 -- 172.21.15.12:0/3531065519 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7850027810 con 0x5649c19b84d0 2024-04-22T13:26:22.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.479+0000 7f786af7b740 1 -- 172.21.15.12:0/3531065519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649c19b84d0 msgr2=0x5649c19c0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.479+0000 7f786af7b740 1 --2- 172.21.15.12:0/3531065519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649c19b84d0 0x5649c19c0d20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7850005700 tx=0x7f7850003870 comp rx=0 tx=0).stop 2024-04-22T13:26:22.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.479+0000 7f786af7b740 1 -- 172.21.15.12:0/3531065519 shutdown_connections 2024-04-22T13:26:22.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.479+0000 7f786af7b740 1 --2- 172.21.15.12:0/3531065519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649c19b8e80 0x5649c19c1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.479+0000 7f786af7b740 1 --2- 172.21.15.12:0/3531065519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649c19b84d0 0x5649c19c0d20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.479+0000 7f786af7b740 1 --2- 172.21.15.12:0/3531065519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649c19b2ba0 0x5649c19cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.479+0000 7f786af7b740 1 -- 172.21.15.12:0/3531065519 >> 172.21.15.12:0/3531065519 conn(0x5649c19bec90 msgr2=0x5649c19b9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:22.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.479+0000 7f786af7b740 1 -- 172.21.15.12:0/3531065519 shutdown_connections 2024-04-22T13:26:22.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.479+0000 7f786af7b740 1 -- 172.21.15.12:0/3531065519 wait complete. 2024-04-22T13:26:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.480+0000 7f786af7b740 1 -- start start 2024-04-22T13:26:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.480+0000 7f786af7b740 1 Processor -- start 2024-04-22T13:26:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.480+0000 7f786af7b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649c19b2ba0 0x5649c19bb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.480+0000 7f786af7b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649c19b84d0 0x5649c19bba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.480+0000 7f786af7b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649c19b8e80 0x5649c19bbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.480+0000 7f786af7b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f78601acdd0 con 0x5649c19b8e80 2024-04-22T13:26:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.480+0000 7f786af7b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f78601ac700 con 0x5649c19b2ba0 2024-04-22T13:26:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.480+0000 7f786af7b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f78601a9940 con 0x5649c19b84d0 2024-04-22T13:26:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.480+0000 7f7866044640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649c19b2ba0 0x5649c19bb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.480+0000 7f7865843640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649c19b84d0 0x5649c19bba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.480+0000 7f786af7b740 10 client.0 _ll_register_callbacks cb 0x7f7850009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.480+0000 7f7866044640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649c19b2ba0 0x5649c19bb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:44014/0 (socket says 172.21.15.12:44014) 2024-04-22T13:26:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.480+0000 7f7866044640 1 -- 172.21.15.12:0/4003142912 learned_addr learned my addr 172.21.15.12:0/4003142912 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.480+0000 7f7866845640 1 --2- 172.21.15.12:0/4003142912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649c19b8e80 0x5649c19bbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.481+0000 7f7866044640 1 -- 172.21.15.12:0/4003142912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649c19b84d0 msgr2=0x5649c19bba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.481+0000 7f7866044640 1 --2- 172.21.15.12:0/4003142912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649c19b84d0 0x5649c19bba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.481+0000 7f7866044640 1 -- 172.21.15.12:0/4003142912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649c19b8e80 msgr2=0x5649c19bbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.481+0000 7f7866044640 1 --2- 172.21.15.12:0/4003142912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649c19b8e80 0x5649c19bbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.481+0000 7f7866044640 1 -- 172.21.15.12:0/4003142912 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649c1a37290 con 0x5649c19b2ba0 2024-04-22T13:26:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.481+0000 7f7866044640 1 --2- 172.21.15.12:0/4003142912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649c19b2ba0 0x5649c19bb4e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f785c018000 tx=0x7f785c0c4040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9dc7ce28da6cccf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.481+0000 7f7849ffb640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-22T13:26:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.481+0000 7f7849ffb640 1 -- 172.21.15.12:0/4003142912 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c0b5020 con 0x5649c19b2ba0 2024-04-22T13:26:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.481+0000 7f7849ffb640 1 -- 172.21.15.12:0/4003142912 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f785c0046c0 con 0x5649c19b2ba0 2024-04-22T13:26:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.481+0000 7f786af7b740 1 -- 172.21.15.12:0/4003142912 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7850003d90 con 0x5649c19b2ba0 2024-04-22T13:26:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.481+0000 7f7849ffb640 1 -- 172.21.15.12:0/4003142912 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f785c0a0f80 con 0x5649c19b2ba0 2024-04-22T13:26:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.481+0000 7f786af7b740 10 client.25973 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.481+0000 7f786af7b740 1 -- 172.21.15.12:0/4003142912 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5649c1a385b0 con 0x5649c19b2ba0 2024-04-22T13:26:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.481+0000 7f7849ffb640 1 -- 172.21.15.12:0/4003142912 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7850003d90 con 0x5649c19b2ba0 2024-04-22T13:26:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.482+0000 7f7849ffb640 1 -- 172.21.15.12:0/4003142912 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f785c0bd0d0 con 0x5649c19b2ba0 2024-04-22T13:26:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.482+0000 7f786af7b740 10 client.25973 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.482+0000 7f786af7b740 10 client.25973 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:26:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.482+0000 7f786af7b740 20 client.25973 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.482+0000 7f786af7b740 1 --2- 172.21.15.12:0/4003142912 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5649c19c1a00 0x5649c1a3a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.482+0000 7f786af7b740 4 client.25973 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:26:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.482+0000 7f786af7b740 1 -- 172.21.15.12:0/4003142912 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5649c192ac10 con 0x5649c19c1a00 2024-04-22T13:26:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.482+0000 7f7865843640 1 --2- 172.21.15.12:0/4003142912 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5649c19c1a00 0x5649c1a3a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.482+0000 7f7865843640 1 --2- 172.21.15.12:0/4003142912 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5649c19c1a00 0x5649c1a3a970 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d3100143b34950ff server_cookie=5dd35f0bc8be044f in_seq=0 out_seq=0 2024-04-22T13:26:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.482+0000 7f7849ffb640 10 client.25973 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:26:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.483+0000 7f7849ffb640 1 -- 172.21.15.12:0/4003142912 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5649c192ac10 con 0x5649c19c1a00 2024-04-22T13:26:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.483+0000 7f7849ffb640 10 client.25973 handle_command_reply: tid=0 2024-04-22T13:26:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.483+0000 7f786af7b740 1 client.25973 shutdown 2024-04-22T13:26:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.483+0000 7f786af7b740 1 -- 172.21.15.12:0/4003142912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649c19b2ba0 msgr2=0x5649c19bb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.483+0000 7f786af7b740 1 --2- 172.21.15.12:0/4003142912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649c19b2ba0 0x5649c19bb4e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f785c018000 tx=0x7f785c0c4040 comp rx=0 tx=0).stop 2024-04-22T13:26:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.483+0000 7f786af7b740 1 -- 172.21.15.12:0/4003142912 shutdown_connections 2024-04-22T13:26:22.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.483+0000 7f786af7b740 1 --2- 172.21.15.12:0/4003142912 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5649c19c1a00 0x5649c1a3a970 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.483+0000 7f7849ffb640 0 client.25973 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:26:22.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.483+0000 7f786af7b740 1 --2- 172.21.15.12:0/4003142912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5649c19b8e80 0x5649c19bbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.483+0000 7f786af7b740 1 --2- 172.21.15.12:0/4003142912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5649c19b84d0 0x5649c19bba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.483+0000 7f786af7b740 1 --2- 172.21.15.12:0/4003142912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5649c19b2ba0 0x5649c19bb4e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.483+0000 7f786af7b740 1 -- 172.21.15.12:0/4003142912 >> 172.21.15.12:0/4003142912 conn(0x5649c19bec90 msgr2=0x5649c19b9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:22.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.483+0000 7f786af7b740 1 -- 172.21.15.12:0/4003142912 shutdown_connections 2024-04-22T13:26:22.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.483+0000 7f786af7b740 1 -- 172.21.15.12:0/4003142912 wait complete. 2024-04-22T13:26:22.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.483+0000 7f786af7b740 20 client.25973 trim_cache size 0 max 16384 2024-04-22T13:26:22.486 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:26:22.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.484+0000 7f78682a8640 1 -- 172.21.15.12:0/1256711036 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f784c0a3d20 msgr2=0x7f784c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.484+0000 7f78682a8640 1 --2- 172.21.15.12:0/1256711036 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f784c0a3d20 0x7f784c0a61e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7858037040 tx=0x7f7858005c50 comp rx=0 tx=0).stop 2024-04-22T13:26:22.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.484+0000 7f78682a8640 1 -- 172.21.15.12:0/1256711036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78601031d0 msgr2=0x7f7860198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.484+0000 7f78682a8640 1 --2- 172.21.15.12:0/1256711036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78601031d0 0x7f7860198700 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f785c00e9f0 tx=0x7f785c00edf0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.484+0000 7f78682a8640 1 -- 172.21.15.12:0/1256711036 shutdown_connections 2024-04-22T13:26:22.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.484+0000 7f78682a8640 1 --2- 172.21.15.12:0/1256711036 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f784c0a3d20 0x7f784c0a61e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.484+0000 7f78682a8640 1 --2- 172.21.15.12:0/1256711036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f786010c510 0x7f786019cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.484+0000 7f78682a8640 1 --2- 172.21.15.12:0/1256711036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7860103af0 0x7f7860198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.484+0000 7f78682a8640 1 --2- 172.21.15.12:0/1256711036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f78601031d0 0x7f7860198700 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.484+0000 7f78682a8640 1 -- 172.21.15.12:0/1256711036 >> 172.21.15.12:0/1256711036 conn(0x7f7860077f50 msgr2=0x7f78600fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:22.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.484+0000 7f78682a8640 1 -- 172.21.15.12:0/1256711036 shutdown_connections 2024-04-22T13:26:22.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.484+0000 7f78682a8640 1 -- 172.21.15.12:0/1256711036 wait complete. 2024-04-22T13:26:22.492 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:26:22.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.609+0000 7f672f632640 1 -- 172.21.15.12:0/2029679726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6728107250 msgr2=0x7f672810ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.609+0000 7f672f632640 1 --2- 172.21.15.12:0/2029679726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6728107250 0x7f672810ee20 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f6720009ca0 tx=0x7f672002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.609+0000 7f672f632640 1 -- 172.21.15.12:0/2029679726 shutdown_connections 2024-04-22T13:26:22.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.609+0000 7f672f632640 1 --2- 172.21.15.12:0/2029679726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6728107250 0x7f672810ee20 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.609+0000 7f672f632640 1 --2- 172.21.15.12:0/2029679726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6728102b30 0x7f6728106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.609+0000 7f672f632640 1 --2- 172.21.15.12:0/2029679726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6728102180 0x7f6728102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.609+0000 7f672f632640 1 -- 172.21.15.12:0/2029679726 >> 172.21.15.12:0/2029679726 conn(0x7f67280fd9c0 msgr2=0x7f67280ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:22.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.609+0000 7f672f632640 1 -- 172.21.15.12:0/2029679726 shutdown_connections 2024-04-22T13:26:22.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.609+0000 7f672f632640 1 -- 172.21.15.12:0/2029679726 wait complete. 2024-04-22T13:26:22.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.610+0000 7f672f632640 1 Processor -- start 2024-04-22T13:26:22.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.610+0000 7f672f632640 1 -- start start 2024-04-22T13:26:22.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.610+0000 7f672f632640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6728102180 0x7f67281a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.610+0000 7f672f632640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6728107250 0x7f67281a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.610+0000 7f672f632640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f672819a800 0x7f672819acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.610+0000 7f672f632640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f67281121c0 con 0x7f672819a800 2024-04-22T13:26:22.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.610+0000 7f672f632640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6728112040 con 0x7f6728102180 2024-04-22T13:26:22.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.610+0000 7f672f632640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6728112340 con 0x7f6728107250 2024-04-22T13:26:22.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.610+0000 7f672d3ce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6728102180 0x7f67281a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.610+0000 7f672cbcd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6728107250 0x7f67281a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.610+0000 7f672d3ce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6728102180 0x7f67281a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:44036/0 (socket says 172.21.15.12:44036) 2024-04-22T13:26:22.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.610+0000 7f672d3ce640 1 -- 172.21.15.12:0/3829660904 learned_addr learned my addr 172.21.15.12:0/3829660904 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:22.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.610+0000 7f672dbcf640 1 --2- 172.21.15.12:0/3829660904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f672819a800 0x7f672819acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.610+0000 7f672cbcd640 1 -- 172.21.15.12:0/3829660904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6728102180 msgr2=0x7f67281a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.610+0000 7f672cbcd640 1 --2- 172.21.15.12:0/3829660904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6728102180 0x7f67281a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.610+0000 7f672cbcd640 1 -- 172.21.15.12:0/3829660904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f672819a800 msgr2=0x7f672819acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.610+0000 7f672cbcd640 1 --2- 172.21.15.12:0/3829660904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f672819a800 0x7f672819acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.610+0000 7f672cbcd640 1 -- 172.21.15.12:0/3829660904 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f672819b390 con 0x7f6728107250 2024-04-22T13:26:22.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.611+0000 7f672cbcd640 1 --2- 172.21.15.12:0/3829660904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6728107250 0x7f67281a0b30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f671800e9c0 tx=0x7f671800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:22.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.611+0000 7f671e7fc640 1 -- 172.21.15.12:0/3829660904 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671800ccc0 con 0x7f6728107250 2024-04-22T13:26:22.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.611+0000 7f672f632640 1 -- 172.21.15.12:0/3829660904 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f672819b680 con 0x7f6728107250 2024-04-22T13:26:22.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.611+0000 7f671e7fc640 1 -- 172.21.15.12:0/3829660904 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6718005020 con 0x7f6728107250 2024-04-22T13:26:22.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.611+0000 7f671e7fc640 1 -- 172.21.15.12:0/3829660904 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6718004a00 con 0x7f6728107250 2024-04-22T13:26:22.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.611+0000 7f672f632640 1 -- 172.21.15.12:0/3829660904 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67281a7770 con 0x7f6728107250 2024-04-22T13:26:22.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.614+0000 7f671e7fc640 1 -- 172.21.15.12:0/3829660904 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f67180051d0 con 0x7f6728107250 2024-04-22T13:26:22.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.614+0000 7f6732219740 1 Processor -- start 2024-04-22T13:26:22.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.614+0000 7f6732219740 1 -- start start 2024-04-22T13:26:22.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.614+0000 7f6732219740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4cc3524d0 0x55e4cc3528b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.614+0000 7f671e7fc640 1 --2- 172.21.15.12:0/3829660904 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f67140a3d20 0x7f67140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.614+0000 7f6732219740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4cc352e80 0x55e4cc35b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.614+0000 7f6732219740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4cc35ba00 0x55e4cc35de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.614+0000 7f6732219740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e4cc2b71f0 con 0x55e4cc352e80 2024-04-22T13:26:22.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.614+0000 7f6732219740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e4cc2544b0 con 0x55e4cc35ba00 2024-04-22T13:26:22.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.614+0000 7f6732219740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e4cc2b31b0 con 0x55e4cc3524d0 2024-04-22T13:26:22.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.615+0000 7f672dbcf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4cc3524d0 0x55e4cc3528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.615+0000 7f672d3ce640 1 --2- 172.21.15.12:0/3829660904 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f67140a3d20 0x7f67140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.615+0000 7f672dbcf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4cc3524d0 0x55e4cc3528b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.12:38726/0 (socket says 172.21.15.12:38726) 2024-04-22T13:26:22.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.615+0000 7f672dbcf640 1 -- 172.21.15.12:0/679605717 learned_addr learned my addr 172.21.15.12:0/679605717 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:22.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.615+0000 7f672cbcd640 1 --2- 172.21.15.12:0/679605717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4cc352e80 0x55e4cc35b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.615+0000 7f672dbcf640 1 --2- 172.21.15.12:0/679605717 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4cc35ba00 0x55e4cc35de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.615+0000 7f671e7fc640 1 -- 172.21.15.12:0/3829660904 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f67180d06d0 con 0x7f6728107250 2024-04-22T13:26:22.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.615+0000 7f672dbcf640 1 -- 172.21.15.12:0/679605717 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4cc35ba00 msgr2=0x55e4cc35de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.615+0000 7f672dbcf640 1 --2- 172.21.15.12:0/679605717 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4cc35ba00 0x55e4cc35de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.615+0000 7f672dbcf640 1 -- 172.21.15.12:0/679605717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4cc352e80 msgr2=0x55e4cc35b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.615+0000 7f672dbcf640 1 --2- 172.21.15.12:0/679605717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4cc352e80 0x55e4cc35b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.615+0000 7f672dbcf640 1 -- 172.21.15.12:0/679605717 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4cc2461b0 con 0x55e4cc3524d0 2024-04-22T13:26:22.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.615+0000 7f672cbcd640 1 --2- 172.21.15.12:0/679605717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4cc352e80 0x55e4cc35b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:26:22.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.615+0000 7f672d3ce640 1 --2- 172.21.15.12:0/3829660904 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f67140a3d20 0x7f67140a61e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6724005fd0 tx=0x7f672400b4a0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:22.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.615+0000 7f672dbcf640 1 --2- 172.21.15.12:0/679605717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4cc3524d0 0x55e4cc3528b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6720037040 tx=0x7f672002e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65c2698b663653cd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:22.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.615+0000 7f67137fe640 1 -- 172.21.15.12:0/679605717 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6720039040 con 0x55e4cc3524d0 2024-04-22T13:26:22.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.615+0000 7f67137fe640 1 -- 172.21.15.12:0/679605717 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6720032970 con 0x55e4cc3524d0 2024-04-22T13:26:22.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.615+0000 7f67137fe640 1 -- 172.21.15.12:0/679605717 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6720004270 con 0x55e4cc3524d0 2024-04-22T13:26:22.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.616+0000 7f6732219740 1 -- 172.21.15.12:0/679605717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4cc3524d0 msgr2=0x55e4cc3528b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.616+0000 7f6732219740 1 --2- 172.21.15.12:0/679605717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4cc3524d0 0x55e4cc3528b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6720037040 tx=0x7f672002e040 comp rx=0 tx=0).stop 2024-04-22T13:26:22.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.616+0000 7f6732219740 1 -- 172.21.15.12:0/679605717 shutdown_connections 2024-04-22T13:26:22.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.616+0000 7f6732219740 1 --2- 172.21.15.12:0/679605717 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4cc35ba00 0x55e4cc35de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.616+0000 7f6732219740 1 --2- 172.21.15.12:0/679605717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4cc352e80 0x55e4cc35b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.616+0000 7f6732219740 1 --2- 172.21.15.12:0/679605717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4cc3524d0 0x55e4cc3528b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.616+0000 7f6732219740 1 -- 172.21.15.12:0/679605717 >> 172.21.15.12:0/679605717 conn(0x55e4cc34cba0 msgr2=0x55e4cc34cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:22.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.616+0000 7f6732219740 1 -- 172.21.15.12:0/679605717 shutdown_connections 2024-04-22T13:26:22.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.616+0000 7f6732219740 1 -- 172.21.15.12:0/679605717 wait complete. 2024-04-22T13:26:22.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.616+0000 7f6732219740 1 -- start start 2024-04-22T13:26:22.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.617+0000 7f6732219740 1 Processor -- start 2024-04-22T13:26:22.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.617+0000 7f6732219740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4cc3524d0 0x55e4cc35b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.617+0000 7f6732219740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4cc352e80 0x55e4cc358680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.617+0000 7f6732219740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4cc35ba00 0x55e4cc358c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.617+0000 7f6732219740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e4cc35e8b0 con 0x55e4cc35ba00 2024-04-22T13:26:22.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.617+0000 7f6732219740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e4cc270500 con 0x55e4cc352e80 2024-04-22T13:26:22.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.617+0000 7f6732219740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e4cc35ea30 con 0x55e4cc3524d0 2024-04-22T13:26:22.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.617+0000 7f6732219740 10 client.0 _ll_register_callbacks cb 0x55e4cc23eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:22.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.617+0000 7f672dbcf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4cc35ba00 0x55e4cc358c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.617+0000 7f672cbcd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4cc352e80 0x55e4cc358680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.617+0000 7f672d3ce640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4cc3524d0 0x55e4cc35b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.617+0000 7f672cbcd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4cc352e80 0x55e4cc358680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:44050/0 (socket says 172.21.15.12:44050) 2024-04-22T13:26:22.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.617+0000 7f672dbcf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4cc35ba00 0x55e4cc358c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43104/0 (socket says 172.21.15.12:43104) 2024-04-22T13:26:22.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.617+0000 7f672cbcd640 1 -- 172.21.15.12:0/388473486 learned_addr learned my addr 172.21.15.12:0/388473486 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:22.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.617+0000 7f672cbcd640 1 -- 172.21.15.12:0/388473486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4cc3524d0 msgr2=0x55e4cc35b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.618+0000 7f672cbcd640 1 --2- 172.21.15.12:0/388473486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4cc3524d0 0x55e4cc35b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.618+0000 7f672cbcd640 1 -- 172.21.15.12:0/388473486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4cc35ba00 msgr2=0x55e4cc358c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.618+0000 7f672cbcd640 1 --2- 172.21.15.12:0/388473486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4cc35ba00 0x55e4cc358c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.618+0000 7f672cbcd640 1 -- 172.21.15.12:0/388473486 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4cbfc93f0 con 0x55e4cc352e80 2024-04-22T13:26:22.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.618+0000 7f672cbcd640 1 --2- 172.21.15.12:0/388473486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4cc352e80 0x55e4cc358680 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f67180cded0 tx=0x7f671809c2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71be697fd108bbeb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:22.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.618+0000 7f6710ff9640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-22T13:26:22.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.618+0000 7f6710ff9640 1 -- 172.21.15.12:0/388473486 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671809cdc0 con 0x55e4cc352e80 2024-04-22T13:26:22.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.618+0000 7f6732219740 1 -- 172.21.15.12:0/388473486 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6720032970 con 0x55e4cc352e80 2024-04-22T13:26:22.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.618+0000 7f6710ff9640 1 -- 172.21.15.12:0/388473486 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f67180afa60 con 0x55e4cc352e80 2024-04-22T13:26:22.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.618+0000 7f6710ff9640 1 -- 172.21.15.12:0/388473486 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f67180af550 con 0x55e4cc352e80 2024-04-22T13:26:22.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.618+0000 7f6732219740 10 client.25985 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:22.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.618+0000 7f6732219740 1 -- 172.21.15.12:0/388473486 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e4cc0937d0 con 0x55e4cc352e80 2024-04-22T13:26:22.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.618+0000 7f6710ff9640 1 -- 172.21.15.12:0/388473486 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6720032970 con 0x55e4cc352e80 2024-04-22T13:26:22.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.618+0000 7f6710ff9640 1 -- 172.21.15.12:0/388473486 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6718098d30 con 0x55e4cc352e80 2024-04-22T13:26:22.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.619+0000 7f6732219740 10 client.25985 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:22.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.619+0000 7f6732219740 10 client.25985 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:26:22.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.619+0000 7f6732219740 20 client.25985 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:22.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.619+0000 7f6732219740 1 --2- 172.21.15.12:0/388473486 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55e4cc36be00 0x55e4cc36e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.619+0000 7f6732219740 4 client.25985 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:26:22.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.619+0000 7f6732219740 1 -- 172.21.15.12:0/388473486 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4cc256930 con 0x55e4cc36be00 2024-04-22T13:26:22.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.619+0000 7f672d3ce640 1 --2- 172.21.15.12:0/388473486 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55e4cc36be00 0x55e4cc36e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.619+0000 7f672d3ce640 1 --2- 172.21.15.12:0/388473486 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55e4cc36be00 0x55e4cc36e1f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c3c6b7d947888908 server_cookie=f28a7cff7a02ab47 in_seq=0 out_seq=0 2024-04-22T13:26:22.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.619+0000 7f6710ff9640 10 client.25985 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:26:22.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.620+0000 7f6710ff9640 1 -- 172.21.15.12:0/388473486 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e4cc256930 con 0x55e4cc36be00 2024-04-22T13:26:22.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.620+0000 7f6710ff9640 10 client.25985 handle_command_reply: tid=0 2024-04-22T13:26:22.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.620+0000 7f6732219740 1 client.25985 shutdown 2024-04-22T13:26:22.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.620+0000 7f6732219740 1 -- 172.21.15.12:0/388473486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4cc352e80 msgr2=0x55e4cc358680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.620+0000 7f6732219740 1 --2- 172.21.15.12:0/388473486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4cc352e80 0x55e4cc358680 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f67180cded0 tx=0x7f671809c2c0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.620+0000 7f6732219740 1 -- 172.21.15.12:0/388473486 shutdown_connections 2024-04-22T13:26:22.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.620+0000 7f6732219740 1 --2- 172.21.15.12:0/388473486 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55e4cc36be00 0x55e4cc36e1f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.620+0000 7f6732219740 1 --2- 172.21.15.12:0/388473486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4cc35ba00 0x55e4cc358c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.620+0000 7f6732219740 1 --2- 172.21.15.12:0/388473486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4cc352e80 0x55e4cc358680 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.620+0000 7f6732219740 1 --2- 172.21.15.12:0/388473486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4cc3524d0 0x55e4cc35b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.620+0000 7f6710ff9640 0 client.25985 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:26:22.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.620+0000 7f6732219740 1 -- 172.21.15.12:0/388473486 >> 172.21.15.12:0/388473486 conn(0x55e4cc34cba0 msgr2=0x55e4cc34e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:22.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.620+0000 7f6732219740 1 -- 172.21.15.12:0/388473486 shutdown_connections 2024-04-22T13:26:22.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.620+0000 7f6732219740 1 -- 172.21.15.12:0/388473486 wait complete. 2024-04-22T13:26:22.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.620+0000 7f6732219740 20 client.25985 trim_cache size 0 max 16384 2024-04-22T13:26:22.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.636+0000 7f6732219740 1 Processor -- start 2024-04-22T13:26:22.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.636+0000 7f6732219740 1 -- start start 2024-04-22T13:26:22.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.636+0000 7f6732219740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4cc3524d0 0x55e4cc34bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.636+0000 7f6732219740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4cc352e80 0x55e4cc35adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.636+0000 7f6732219740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4cc35ba00 0x55e4cc35b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.636+0000 7f6732219740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f67281afbe0 con 0x55e4cc3524d0 2024-04-22T13:26:22.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.636+0000 7f6732219740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f67281b1720 con 0x55e4cc35ba00 2024-04-22T13:26:22.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.636+0000 7f6732219740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f67281b02b0 con 0x55e4cc352e80 2024-04-22T13:26:22.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.636+0000 7f672d3ce640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4cc3524d0 0x55e4cc34bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.636+0000 7f672dbcf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4cc35ba00 0x55e4cc35b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.636+0000 7f672cbcd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4cc352e80 0x55e4cc35adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.636+0000 7f672dbcf640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4cc35ba00 0x55e4cc35b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:44066/0 (socket says 172.21.15.12:44066) 2024-04-22T13:26:22.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.636+0000 7f672dbcf640 1 -- 172.21.15.12:0/1616646800 learned_addr learned my addr 172.21.15.12:0/1616646800 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:22.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.636+0000 7f672d3ce640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4cc3524d0 0x55e4cc34bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43116/0 (socket says 172.21.15.12:43116) 2024-04-22T13:26:22.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.636+0000 7f672dbcf640 1 -- 172.21.15.12:0/1616646800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4cc352e80 msgr2=0x55e4cc35adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.636+0000 7f672dbcf640 1 --2- 172.21.15.12:0/1616646800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4cc352e80 0x55e4cc35adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.636+0000 7f672dbcf640 1 -- 172.21.15.12:0/1616646800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4cc3524d0 msgr2=0x55e4cc34bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.637+0000 7f672dbcf640 1 --2- 172.21.15.12:0/1616646800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4cc3524d0 0x55e4cc34bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.637+0000 7f672dbcf640 1 -- 172.21.15.12:0/1616646800 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4cc358f70 con 0x55e4cc35ba00 2024-04-22T13:26:22.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.637+0000 7f672d3ce640 1 --2- 172.21.15.12:0/1616646800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4cc3524d0 0x55e4cc34bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:22.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.637+0000 7f672cbcd640 1 --2- 172.21.15.12:0/1616646800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4cc352e80 0x55e4cc35adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:22.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.637+0000 7f672dbcf640 1 --2- 172.21.15.12:0/1616646800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4cc35ba00 0x55e4cc35b2f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6720003dc0 tx=0x7f6720003df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8ca131f323624e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:22.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.637+0000 7f6712ffd640 1 -- 172.21.15.12:0/1616646800 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6720053070 con 0x55e4cc35ba00 2024-04-22T13:26:22.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.637+0000 7f6712ffd640 1 -- 172.21.15.12:0/1616646800 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6720032970 con 0x55e4cc35ba00 2024-04-22T13:26:22.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.637+0000 7f6712ffd640 1 -- 172.21.15.12:0/1616646800 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6720039040 con 0x55e4cc35ba00 2024-04-22T13:26:22.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.638+0000 7f6732219740 1 -- 172.21.15.12:0/1616646800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4cc35ba00 msgr2=0x55e4cc35b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.638+0000 7f6732219740 1 --2- 172.21.15.12:0/1616646800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4cc35ba00 0x55e4cc35b2f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6720003dc0 tx=0x7f6720003df0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.638+0000 7f6732219740 1 -- 172.21.15.12:0/1616646800 shutdown_connections 2024-04-22T13:26:22.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.638+0000 7f6732219740 1 --2- 172.21.15.12:0/1616646800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4cc35ba00 0x55e4cc35b2f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.638+0000 7f6732219740 1 --2- 172.21.15.12:0/1616646800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4cc352e80 0x55e4cc35adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.638+0000 7f6732219740 1 --2- 172.21.15.12:0/1616646800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4cc3524d0 0x55e4cc34bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.638+0000 7f6732219740 1 -- 172.21.15.12:0/1616646800 >> 172.21.15.12:0/1616646800 conn(0x55e4cc34cba0 msgr2=0x55e4cc34cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:22.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.638+0000 7f6732219740 1 -- 172.21.15.12:0/1616646800 shutdown_connections 2024-04-22T13:26:22.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.638+0000 7f6732219740 1 -- 172.21.15.12:0/1616646800 wait complete. 2024-04-22T13:26:22.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.638+0000 7f6732219740 1 -- start start 2024-04-22T13:26:22.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.638+0000 7f6732219740 1 Processor -- start 2024-04-22T13:26:22.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.639+0000 7f6732219740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4cc3524d0 0x55e4cc36ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.639+0000 7f6732219740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4cc352e80 0x55e4cc36f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.639+0000 7f6732219740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4cc35ba00 0x55e4cc36f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.639+0000 7f6732219740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f67281b1050 con 0x55e4cc3524d0 2024-04-22T13:26:22.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.639+0000 7f6732219740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f67281b0980 con 0x55e4cc35ba00 2024-04-22T13:26:22.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.639+0000 7f6732219740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f67281adbc0 con 0x55e4cc352e80 2024-04-22T13:26:22.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.639+0000 7f6732219740 10 client.0 _ll_register_callbacks cb 0x7f671809c490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:22.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.639+0000 7f672d3ce640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4cc3524d0 0x55e4cc36ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.639+0000 7f672d3ce640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4cc3524d0 0x55e4cc36ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43122/0 (socket says 172.21.15.12:43122) 2024-04-22T13:26:22.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.639+0000 7f672cbcd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4cc352e80 0x55e4cc36f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.639+0000 7f672d3ce640 1 -- 172.21.15.12:0/3079414502 learned_addr learned my addr 172.21.15.12:0/3079414502 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:22.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.639+0000 7f672dbcf640 1 --2- 172.21.15.12:0/3079414502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4cc35ba00 0x55e4cc36f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.639+0000 7f672d3ce640 1 -- 172.21.15.12:0/3079414502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4cc352e80 msgr2=0x55e4cc36f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.639+0000 7f672d3ce640 1 --2- 172.21.15.12:0/3079414502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4cc352e80 0x55e4cc36f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.639+0000 7f672d3ce640 1 -- 172.21.15.12:0/3079414502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4cc35ba00 msgr2=0x55e4cc36f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.639+0000 7f672d3ce640 1 --2- 172.21.15.12:0/3079414502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4cc35ba00 0x55e4cc36f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.639+0000 7f672d3ce640 1 -- 172.21.15.12:0/3079414502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4cc3d11e0 con 0x55e4cc3524d0 2024-04-22T13:26:22.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.639+0000 7f672cbcd640 1 --2- 172.21.15.12:0/3079414502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4cc352e80 0x55e4cc36f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:22.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.639+0000 7f672d3ce640 1 --2- 172.21.15.12:0/3079414502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4cc3524d0 0x55e4cc36ee50 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f6724004390 tx=0x7f6724004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597b582bfbc7631a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:22.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.639+0000 7f6710ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:26:22.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.639+0000 7f6710ff9640 1 -- 172.21.15.12:0/3079414502 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67240278f0 con 0x55e4cc3524d0 2024-04-22T13:26:22.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.639+0000 7f6710ff9640 1 -- 172.21.15.12:0/3079414502 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f6724011910 con 0x55e4cc3524d0 2024-04-22T13:26:22.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.639+0000 7f6732219740 1 -- 172.21.15.12:0/3079414502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6720032970 con 0x55e4cc3524d0 2024-04-22T13:26:22.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.640+0000 7f6710ff9640 1 -- 172.21.15.12:0/3079414502 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6724011b10 con 0x55e4cc3524d0 2024-04-22T13:26:22.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.640+0000 7f6732219740 10 client.16575 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:22.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.640+0000 7f6732219740 1 -- 172.21.15.12:0/3079414502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e4cc3d2520 con 0x55e4cc3524d0 2024-04-22T13:26:22.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.640+0000 7f6710ff9640 1 -- 172.21.15.12:0/3079414502 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6720032970 con 0x55e4cc3524d0 2024-04-22T13:26:22.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.640+0000 7f6710ff9640 1 -- 172.21.15.12:0/3079414502 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f672401e420 con 0x55e4cc3524d0 2024-04-22T13:26:22.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.640+0000 7f6732219740 10 client.16575 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:22.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.640+0000 7f6732219740 10 client.16575 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:26:22.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.640+0000 7f6732219740 20 client.16575 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:22.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.640+0000 7f6732219740 1 --2- 172.21.15.12:0/3079414502 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55e4cc3d2bd0 0x55e4cc3d4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.640+0000 7f6732219740 4 client.16575 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:26:22.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.640+0000 7f6732219740 1 -- 172.21.15.12:0/3079414502 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e4cc2c4c10 con 0x55e4cc3d2bd0 2024-04-22T13:26:22.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.640+0000 7f672cbcd640 1 --2- 172.21.15.12:0/3079414502 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55e4cc3d2bd0 0x55e4cc3d4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.641+0000 7f6710ff9640 10 client.16575 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:26:22.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.641+0000 7f672cbcd640 1 --2- 172.21.15.12:0/3079414502 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55e4cc3d2bd0 0x55e4cc3d4ff0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9379c48df0c9dddd server_cookie=d93a3c7f97ce2aaa in_seq=0 out_seq=0 2024-04-22T13:26:22.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.641+0000 7f6710ff9640 1 -- 172.21.15.12:0/3079414502 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e4cc2c4c10 con 0x55e4cc3d2bd0 2024-04-22T13:26:22.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.641+0000 7f6710ff9640 10 client.16575 handle_command_reply: tid=0 2024-04-22T13:26:22.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.641+0000 7f6732219740 1 client.16575 shutdown 2024-04-22T13:26:22.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.641+0000 7f6732219740 1 -- 172.21.15.12:0/3079414502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4cc3524d0 msgr2=0x55e4cc36ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.641+0000 7f6732219740 1 --2- 172.21.15.12:0/3079414502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4cc3524d0 0x55e4cc36ee50 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f6724004390 tx=0x7f6724004730 comp rx=0 tx=0).stop 2024-04-22T13:26:22.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.641+0000 7f6732219740 1 -- 172.21.15.12:0/3079414502 shutdown_connections 2024-04-22T13:26:22.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.641+0000 7f6732219740 1 --2- 172.21.15.12:0/3079414502 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55e4cc3d2bd0 0x55e4cc3d4ff0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.641+0000 7f6732219740 1 --2- 172.21.15.12:0/3079414502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4cc35ba00 0x55e4cc36f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.641+0000 7f6710ff9640 0 client.16575 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:26:22.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.641+0000 7f6732219740 1 --2- 172.21.15.12:0/3079414502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4cc352e80 0x55e4cc36f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.641+0000 7f6732219740 1 --2- 172.21.15.12:0/3079414502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e4cc3524d0 0x55e4cc36ee50 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.641+0000 7f6732219740 1 -- 172.21.15.12:0/3079414502 >> 172.21.15.12:0/3079414502 conn(0x55e4cc34cba0 msgr2=0x55e4cc367510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:22.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.641+0000 7f6732219740 1 -- 172.21.15.12:0/3079414502 shutdown_connections 2024-04-22T13:26:22.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.641+0000 7f6732219740 1 -- 172.21.15.12:0/3079414502 wait complete. 2024-04-22T13:26:22.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.641+0000 7f6732219740 20 client.16575 trim_cache size 0 max 16384 2024-04-22T13:26:22.644 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:26:22.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.642+0000 7f672f632640 1 -- 172.21.15.12:0/3829660904 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f67140a3d20 msgr2=0x7f67140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.642+0000 7f672f632640 1 --2- 172.21.15.12:0/3829660904 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f67140a3d20 0x7f67140a61e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6724005fd0 tx=0x7f672400b4a0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.642+0000 7f672f632640 1 -- 172.21.15.12:0/3829660904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6728107250 msgr2=0x7f67281a0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.642+0000 7f672f632640 1 --2- 172.21.15.12:0/3829660904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6728107250 0x7f67281a0b30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f671800e9c0 tx=0x7f671800ee80 comp rx=0 tx=0).stop 2024-04-22T13:26:22.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.642+0000 7f672f632640 1 -- 172.21.15.12:0/3829660904 shutdown_connections 2024-04-22T13:26:22.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.642+0000 7f672f632640 1 --2- 172.21.15.12:0/3829660904 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f67140a3d20 0x7f67140a61e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.642+0000 7f672f632640 1 --2- 172.21.15.12:0/3829660904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f672819a800 0x7f672819acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.642+0000 7f672f632640 1 --2- 172.21.15.12:0/3829660904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6728107250 0x7f67281a0b30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.642+0000 7f672f632640 1 --2- 172.21.15.12:0/3829660904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6728102180 0x7f67281a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.642+0000 7f672f632640 1 -- 172.21.15.12:0/3829660904 >> 172.21.15.12:0/3829660904 conn(0x7f67280fd9c0 msgr2=0x7f67280ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:22.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.642+0000 7f672f632640 1 -- 172.21.15.12:0/3829660904 shutdown_connections 2024-04-22T13:26:22.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.642+0000 7f672f632640 1 -- 172.21.15.12:0/3829660904 wait complete. 2024-04-22T13:26:22.651 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:26:22.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.769+0000 7f43971d0640 1 -- 172.21.15.12:0/476048339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4390102b40 msgr2=0x7f4390106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.769+0000 7f43971d0640 1 --2- 172.21.15.12:0/476048339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4390102b40 0x7f4390106b80 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f4380009900 tx=0x7f438002bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:22.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.769+0000 7f43971d0640 1 -- 172.21.15.12:0/476048339 shutdown_connections 2024-04-22T13:26:22.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.769+0000 7f43971d0640 1 --2- 172.21.15.12:0/476048339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4390107260 0x7f439010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.769+0000 7f43971d0640 1 --2- 172.21.15.12:0/476048339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4390102b40 0x7f4390106b80 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.769+0000 7f43971d0640 1 --2- 172.21.15.12:0/476048339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4390102190 0x7f4390102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.769+0000 7f43971d0640 1 -- 172.21.15.12:0/476048339 >> 172.21.15.12:0/476048339 conn(0x7f43900fd9f0 msgr2=0x7f43900ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:22.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.769+0000 7f43971d0640 1 -- 172.21.15.12:0/476048339 shutdown_connections 2024-04-22T13:26:22.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.769+0000 7f43971d0640 1 -- 172.21.15.12:0/476048339 wait complete. 2024-04-22T13:26:22.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.770+0000 7f43971d0640 1 Processor -- start 2024-04-22T13:26:22.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.770+0000 7f43971d0640 1 -- start start 2024-04-22T13:26:22.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.770+0000 7f43971d0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4390102190 0x7f4390198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.770+0000 7f43971d0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4390107260 0x7f4390198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.769 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.770+0000 7f43971d0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f439019d000 0x7f439019d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.770+0000 7f43971d0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f43901121d0 con 0x7f4390107260 2024-04-22T13:26:22.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.770+0000 7f43971d0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4390112050 con 0x7f439019d000 2024-04-22T13:26:22.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.770+0000 7f43971d0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4390112350 con 0x7f4390102190 2024-04-22T13:26:22.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.770+0000 7f4387fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4390107260 0x7f4390198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.771+0000 7f439576d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f439019d000 0x7f439019d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.771+0000 7f4387fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4390107260 0x7f4390198cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43144/0 (socket says 172.21.15.12:43144) 2024-04-22T13:26:22.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.771+0000 7f4387fff640 1 -- 172.21.15.12:0/1749984605 learned_addr learned my addr 172.21.15.12:0/1749984605 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:22.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.771+0000 7f4394f6c640 1 --2- 172.21.15.12:0/1749984605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4390102190 0x7f4390198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.771+0000 7f4387fff640 1 -- 172.21.15.12:0/1749984605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4390102190 msgr2=0x7f4390198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.771+0000 7f4387fff640 1 --2- 172.21.15.12:0/1749984605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4390102190 0x7f4390198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.771+0000 7f4387fff640 1 -- 172.21.15.12:0/1749984605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f439019d000 msgr2=0x7f439019d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.771+0000 7f4387fff640 1 --2- 172.21.15.12:0/1749984605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f439019d000 0x7f439019d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.771+0000 7f4387fff640 1 -- 172.21.15.12:0/1749984605 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439019daf0 con 0x7f4390107260 2024-04-22T13:26:22.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.771+0000 7f439576d640 1 --2- 172.21.15.12:0/1749984605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f439019d000 0x7f439019d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:22.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.771+0000 7f4387fff640 1 --2- 172.21.15.12:0/1749984605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4390107260 0x7f4390198cd0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f4380000c00 tx=0x7f438002c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:22.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.771+0000 7f4385ffb640 1 -- 172.21.15.12:0/1749984605 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4380043020 con 0x7f4390107260 2024-04-22T13:26:22.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.771+0000 7f43971d0640 1 -- 172.21.15.12:0/1749984605 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f439019dd80 con 0x7f4390107260 2024-04-22T13:26:22.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.771+0000 7f4385ffb640 1 -- 172.21.15.12:0/1749984605 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f438002caf0 con 0x7f4390107260 2024-04-22T13:26:22.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.771+0000 7f4385ffb640 1 -- 172.21.15.12:0/1749984605 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4380003b20 con 0x7f4390107260 2024-04-22T13:26:22.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.772+0000 7f43971d0640 1 -- 172.21.15.12:0/1749984605 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43901a55d0 con 0x7f4390107260 2024-04-22T13:26:22.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.774+0000 7f4385ffb640 1 -- 172.21.15.12:0/1749984605 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4380046020 con 0x7f4390107260 2024-04-22T13:26:22.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.775+0000 7f4399dbd740 1 Processor -- start 2024-04-22T13:26:22.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.775+0000 7f4385ffb640 1 --2- 172.21.15.12:0/1749984605 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f43780a3da0 0x7f43780a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.775+0000 7f4399dbd740 1 -- start start 2024-04-22T13:26:22.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.775+0000 7f439576d640 1 --2- 172.21.15.12:0/1749984605 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f43780a3da0 0x7f43780a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.775+0000 7f4385ffb640 1 -- 172.21.15.12:0/1749984605 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f43800f2630 con 0x7f4390107260 2024-04-22T13:26:22.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.775+0000 7f4399dbd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562df83fd4d0 0x562df83fd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.775+0000 7f4399dbd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562df83fde80 0x562df8406430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.775+0000 7f4399dbd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562df8406a00 0x562df8408e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.775+0000 7f4399dbd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x562df83621f0 con 0x562df83fde80 2024-04-22T13:26:22.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.775+0000 7f4399dbd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x562df82ff4b0 con 0x562df8406a00 2024-04-22T13:26:22.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.775+0000 7f4399dbd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562df835e1b0 con 0x562df83fd4d0 2024-04-22T13:26:22.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.775+0000 7f4394f6c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562df83fd4d0 0x562df83fd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.775+0000 7f4387fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562df83fde80 0x562df8406430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.775+0000 7f4394f6c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562df83fd4d0 0x562df83fd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.12:38798/0 (socket says 172.21.15.12:38798) 2024-04-22T13:26:22.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.775+0000 7f4394f6c640 1 -- 172.21.15.12:0/2745640496 learned_addr learned my addr 172.21.15.12:0/2745640496 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:22.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.775+0000 7f439576d640 1 --2- 172.21.15.12:0/2745640496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562df8406a00 0x562df8408e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.776+0000 7f4394f6c640 1 -- 172.21.15.12:0/2745640496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562df8406a00 msgr2=0x562df8408e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.776+0000 7f439576d640 1 --2- 172.21.15.12:0/1749984605 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f43780a3da0 0x7f43780a6260 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4388005fd0 tx=0x7f4388011b50 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:22.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.776+0000 7f4394f6c640 1 --2- 172.21.15.12:0/2745640496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562df8406a00 0x562df8408e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.776+0000 7f4394f6c640 1 -- 172.21.15.12:0/2745640496 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562df83fde80 msgr2=0x562df8406430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.776+0000 7f4394f6c640 1 --2- 172.21.15.12:0/2745640496 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562df83fde80 0x562df8406430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.776+0000 7f4394f6c640 1 -- 172.21.15.12:0/2745640496 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562df82f11b0 con 0x562df83fd4d0 2024-04-22T13:26:22.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.776+0000 7f4394f6c640 1 --2- 172.21.15.12:0/2745640496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562df83fd4d0 0x562df83fd8b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f438c009cf0 tx=0x7f438c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7433b0800a444470 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:22.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.776+0000 7f4376ffd640 1 -- 172.21.15.12:0/2745640496 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c004230 con 0x562df83fd4d0 2024-04-22T13:26:22.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.776+0000 7f4376ffd640 1 -- 172.21.15.12:0/2745640496 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f438c0043d0 con 0x562df83fd4d0 2024-04-22T13:26:22.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.776+0000 7f4376ffd640 1 -- 172.21.15.12:0/2745640496 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c019780 con 0x562df83fd4d0 2024-04-22T13:26:22.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.777+0000 7f4399dbd740 1 -- 172.21.15.12:0/2745640496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562df83fd4d0 msgr2=0x562df83fd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.777+0000 7f4399dbd740 1 --2- 172.21.15.12:0/2745640496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562df83fd4d0 0x562df83fd8b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f438c009cf0 tx=0x7f438c011810 comp rx=0 tx=0).stop 2024-04-22T13:26:22.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.777+0000 7f4399dbd740 1 -- 172.21.15.12:0/2745640496 shutdown_connections 2024-04-22T13:26:22.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.777+0000 7f4399dbd740 1 --2- 172.21.15.12:0/2745640496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562df8406a00 0x562df8408e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.777+0000 7f4399dbd740 1 --2- 172.21.15.12:0/2745640496 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562df83fde80 0x562df8406430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.777+0000 7f4399dbd740 1 --2- 172.21.15.12:0/2745640496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562df83fd4d0 0x562df83fd8b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.777+0000 7f4399dbd740 1 -- 172.21.15.12:0/2745640496 >> 172.21.15.12:0/2745640496 conn(0x562df83f7ba0 msgr2=0x562df83f7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:22.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.777+0000 7f4399dbd740 1 -- 172.21.15.12:0/2745640496 shutdown_connections 2024-04-22T13:26:22.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.777+0000 7f4399dbd740 1 -- 172.21.15.12:0/2745640496 wait complete. 2024-04-22T13:26:22.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.777+0000 7f4399dbd740 1 -- start start 2024-04-22T13:26:22.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.777+0000 7f4399dbd740 1 Processor -- start 2024-04-22T13:26:22.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.778+0000 7f4399dbd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562df83fd4d0 0x562df84062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.778+0000 7f4399dbd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562df83fde80 0x562df8403670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.778+0000 7f4399dbd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562df8406a00 0x562df8403bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.778+0000 7f4399dbd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x562df8409970 con 0x562df83fd4d0 2024-04-22T13:26:22.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.778+0000 7f4399dbd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x562df831b500 con 0x562df83fde80 2024-04-22T13:26:22.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.778+0000 7f4399dbd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562df8409af0 con 0x562df8406a00 2024-04-22T13:26:22.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.778+0000 7f4399dbd740 10 client.0 _ll_register_callbacks cb 0x562df82e9b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:22.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.778+0000 7f4394f6c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562df83fd4d0 0x562df84062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.778+0000 7f439576d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562df8406a00 0x562df8403bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.778+0000 7f4387fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562df83fde80 0x562df8403670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.778+0000 7f439576d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562df8406a00 0x562df8403bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.12:38802/0 (socket says 172.21.15.12:38802) 2024-04-22T13:26:22.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.778+0000 7f439576d640 1 -- 172.21.15.12:0/119738575 learned_addr learned my addr 172.21.15.12:0/119738575 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:22.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.778+0000 7f4394f6c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562df83fd4d0 0x562df84062e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43168/0 (socket says 172.21.15.12:43168) 2024-04-22T13:26:22.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.778+0000 7f439576d640 1 -- 172.21.15.12:0/119738575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562df83fde80 msgr2=0x562df8403670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.778+0000 7f439576d640 1 --2- 172.21.15.12:0/119738575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562df83fde80 0x562df8403670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.778+0000 7f439576d640 1 -- 172.21.15.12:0/119738575 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562df83fd4d0 msgr2=0x562df84062e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.778+0000 7f439576d640 1 --2- 172.21.15.12:0/119738575 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562df83fd4d0 0x562df84062e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.778+0000 7f439576d640 1 -- 172.21.15.12:0/119738575 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562df80743f0 con 0x562df8406a00 2024-04-22T13:26:22.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.779+0000 7f4387fff640 1 --2- 172.21.15.12:0/119738575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562df83fde80 0x562df8403670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:22.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.779+0000 7f439576d640 1 --2- 172.21.15.12:0/119738575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562df8406a00 0x562df8403bd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f438800fa80 tx=0x7f4388004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4802016fd78761c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:22.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.779+0000 7f436ffff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-22T13:26:22.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.779+0000 7f436ffff640 1 -- 172.21.15.12:0/119738575 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4388017040 con 0x562df8406a00 2024-04-22T13:26:22.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.779+0000 7f436ffff640 1 -- 172.21.15.12:0/119738575 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f4388028440 con 0x562df8406a00 2024-04-22T13:26:22.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.779+0000 7f4399dbd740 1 -- 172.21.15.12:0/119738575 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f438c0043d0 con 0x562df8406a00 2024-04-22T13:26:22.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.779+0000 7f436ffff640 1 -- 172.21.15.12:0/119738575 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4388028680 con 0x562df8406a00 2024-04-22T13:26:22.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.779+0000 7f4399dbd740 10 client.25939 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:22.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.779+0000 7f4399dbd740 1 -- 172.21.15.12:0/119738575 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562df813e7d0 con 0x562df8406a00 2024-04-22T13:26:22.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.779+0000 7f436ffff640 1 -- 172.21.15.12:0/119738575 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f438c0043d0 con 0x562df8406a00 2024-04-22T13:26:22.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.780+0000 7f436ffff640 1 -- 172.21.15.12:0/119738575 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4388026020 con 0x562df8406a00 2024-04-22T13:26:22.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.780+0000 7f4399dbd740 10 client.25939 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:22.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.780+0000 7f4399dbd740 10 client.25939 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:26:22.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.780+0000 7f4399dbd740 20 client.25939 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:22.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.780+0000 7f4399dbd740 1 --2- 172.21.15.12:0/119738575 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x562df8416b80 0x562df8418fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.780+0000 7f4399dbd740 4 client.25939 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:26:22.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.780+0000 7f4399dbd740 1 -- 172.21.15.12:0/119738575 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562df8301930 con 0x562df8416b80 2024-04-22T13:26:22.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.780+0000 7f4394f6c640 1 --2- 172.21.15.12:0/119738575 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x562df8416b80 0x562df8418fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.780+0000 7f4394f6c640 1 --2- 172.21.15.12:0/119738575 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x562df8416b80 0x562df8418fb0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8f318e96404c57c8 server_cookie=eacdbf712873eca6 in_seq=0 out_seq=0 2024-04-22T13:26:22.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.780+0000 7f436ffff640 10 client.25939 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:26:22.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.781+0000 7f436ffff640 1 -- 172.21.15.12:0/119738575 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562df8301930 con 0x562df8416b80 2024-04-22T13:26:22.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.781+0000 7f436ffff640 10 client.25939 handle_command_reply: tid=0 2024-04-22T13:26:22.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.781+0000 7f4399dbd740 1 client.25939 shutdown 2024-04-22T13:26:22.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.781+0000 7f4399dbd740 1 -- 172.21.15.12:0/119738575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562df8406a00 msgr2=0x562df8403bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.781+0000 7f4399dbd740 1 --2- 172.21.15.12:0/119738575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562df8406a00 0x562df8403bd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f438800fa80 tx=0x7f4388004ab0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.781+0000 7f4399dbd740 1 -- 172.21.15.12:0/119738575 shutdown_connections 2024-04-22T13:26:22.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.781+0000 7f4399dbd740 1 --2- 172.21.15.12:0/119738575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562df8406a00 0x562df8403bd0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.781+0000 7f4399dbd740 1 --2- 172.21.15.12:0/119738575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562df83fde80 0x562df8403670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.781+0000 7f4399dbd740 1 --2- 172.21.15.12:0/119738575 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x562df8416b80 0x562df8418fb0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.781+0000 7f4399dbd740 1 --2- 172.21.15.12:0/119738575 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562df83fd4d0 0x562df84062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.781+0000 7f4399dbd740 1 -- 172.21.15.12:0/119738575 >> 172.21.15.12:0/119738575 conn(0x562df83f7ba0 msgr2=0x562df83f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:22.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.781+0000 7f436ffff640 0 client.25939 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:26:22.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.781+0000 7f4399dbd740 1 -- 172.21.15.12:0/119738575 shutdown_connections 2024-04-22T13:26:22.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.781+0000 7f4399dbd740 1 -- 172.21.15.12:0/119738575 wait complete. 2024-04-22T13:26:22.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.781+0000 7f4399dbd740 20 client.25939 trim_cache size 0 max 16384 2024-04-22T13:26:22.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.797+0000 7f4399dbd740 1 Processor -- start 2024-04-22T13:26:22.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.797+0000 7f4399dbd740 1 -- start start 2024-04-22T13:26:22.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.797+0000 7f4399dbd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562df83fd4d0 0x562df8402f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.797+0000 7f4399dbd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562df83fde80 0x562df83f9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.797+0000 7f4399dbd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562df8406a00 0x562df83f6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.797+0000 7f4399dbd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f43901adaf0 con 0x562df8406a00 2024-04-22T13:26:22.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.797+0000 7f4399dbd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f43901af630 con 0x562df83fd4d0 2024-04-22T13:26:22.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.797+0000 7f4399dbd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f43901ae1c0 con 0x562df83fde80 2024-04-22T13:26:22.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.798+0000 7f439576d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562df8406a00 0x562df83f6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.798+0000 7f4387fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562df83fde80 0x562df83f9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.798+0000 7f4394f6c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562df83fd4d0 0x562df8402f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.798+0000 7f4387fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562df83fde80 0x562df83f9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.12:38808/0 (socket says 172.21.15.12:38808) 2024-04-22T13:26:22.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.798+0000 7f4394f6c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562df83fd4d0 0x562df8402f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:44120/0 (socket says 172.21.15.12:44120) 2024-04-22T13:26:22.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.798+0000 7f4387fff640 1 -- 172.21.15.12:0/3788209564 learned_addr learned my addr 172.21.15.12:0/3788209564 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:22.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.798+0000 7f439576d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562df8406a00 0x562df83f6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43180/0 (socket says 172.21.15.12:43180) 2024-04-22T13:26:22.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.798+0000 7f439576d640 1 -- 172.21.15.12:0/3788209564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562df83fde80 msgr2=0x562df83f9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.798+0000 7f439576d640 1 --2- 172.21.15.12:0/3788209564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562df83fde80 0x562df83f9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.798+0000 7f439576d640 1 -- 172.21.15.12:0/3788209564 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562df83fd4d0 msgr2=0x562df8402f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.798+0000 7f439576d640 1 --2- 172.21.15.12:0/3788209564 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562df83fd4d0 0x562df8402f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.798+0000 7f439576d640 1 -- 172.21.15.12:0/3788209564 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562df83ff270 con 0x562df8406a00 2024-04-22T13:26:22.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.798+0000 7f4394f6c640 1 --2- 172.21.15.12:0/3788209564 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562df83fd4d0 0x562df8402f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:22.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.798+0000 7f439576d640 1 --2- 172.21.15.12:0/3788209564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562df8406a00 0x562df83f6a80 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f4388032ac0 tx=0x7f4388005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea0561d666a5e18 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:22.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.798+0000 7f43767fc640 1 -- 172.21.15.12:0/3788209564 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4388037030 con 0x562df8406a00 2024-04-22T13:26:22.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.798+0000 7f43767fc640 1 -- 172.21.15.12:0/3788209564 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4388019070 con 0x562df8406a00 2024-04-22T13:26:22.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.798+0000 7f43767fc640 1 -- 172.21.15.12:0/3788209564 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4388017040 con 0x562df8406a00 2024-04-22T13:26:22.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.799+0000 7f4399dbd740 1 -- 172.21.15.12:0/3788209564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562df8406a00 msgr2=0x562df83f6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.799+0000 7f4399dbd740 1 --2- 172.21.15.12:0/3788209564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562df8406a00 0x562df83f6a80 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f4388032ac0 tx=0x7f4388005b50 comp rx=0 tx=0).stop 2024-04-22T13:26:22.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.799+0000 7f4399dbd740 1 -- 172.21.15.12:0/3788209564 shutdown_connections 2024-04-22T13:26:22.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.799+0000 7f4399dbd740 1 --2- 172.21.15.12:0/3788209564 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562df8406a00 0x562df83f6a80 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.799+0000 7f4399dbd740 1 --2- 172.21.15.12:0/3788209564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562df83fde80 0x562df83f9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.799+0000 7f4399dbd740 1 --2- 172.21.15.12:0/3788209564 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562df83fd4d0 0x562df8402f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.799+0000 7f4399dbd740 1 -- 172.21.15.12:0/3788209564 >> 172.21.15.12:0/3788209564 conn(0x562df83f7ba0 msgr2=0x562df83f6370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:22.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.799+0000 7f4399dbd740 1 -- 172.21.15.12:0/3788209564 shutdown_connections 2024-04-22T13:26:22.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.799+0000 7f4399dbd740 1 -- 172.21.15.12:0/3788209564 wait complete. 2024-04-22T13:26:22.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.800+0000 7f4399dbd740 1 -- start start 2024-04-22T13:26:22.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.800+0000 7f4399dbd740 1 Processor -- start 2024-04-22T13:26:22.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.800+0000 7f4399dbd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562df83fd4d0 0x562df840ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.800+0000 7f4399dbd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562df83fde80 0x562df840f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.800+0000 7f4399dbd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562df8406a00 0x562df83ff960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.800+0000 7f4399dbd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f43901aef60 con 0x562df83fde80 2024-04-22T13:26:22.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.800+0000 7f4399dbd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f43901ae890 con 0x562df83fd4d0 2024-04-22T13:26:22.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.800+0000 7f4399dbd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f43901abaf0 con 0x562df8406a00 2024-04-22T13:26:22.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.800+0000 7f4399dbd740 10 client.0 _ll_register_callbacks cb 0x7f438800f960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:22.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.800+0000 7f4387fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562df83fde80 0x562df840f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.800+0000 7f4387fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562df83fde80 0x562df840f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43192/0 (socket says 172.21.15.12:43192) 2024-04-22T13:26:22.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.800+0000 7f4387fff640 1 -- 172.21.15.12:0/3992792917 learned_addr learned my addr 172.21.15.12:0/3992792917 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:22.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.800+0000 7f439576d640 1 --2- 172.21.15.12:0/3992792917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562df8406a00 0x562df83ff960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.800+0000 7f4394f6c640 1 --2- 172.21.15.12:0/3992792917 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562df83fd4d0 0x562df840ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.801+0000 7f4387fff640 1 -- 172.21.15.12:0/3992792917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562df8406a00 msgr2=0x562df83ff960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.801+0000 7f4387fff640 1 --2- 172.21.15.12:0/3992792917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562df8406a00 0x562df83ff960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.801+0000 7f4387fff640 1 -- 172.21.15.12:0/3992792917 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562df83fd4d0 msgr2=0x562df840ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.801+0000 7f4387fff640 1 --2- 172.21.15.12:0/3992792917 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562df83fd4d0 0x562df840ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.801+0000 7f4387fff640 1 -- 172.21.15.12:0/3992792917 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562df83fffa0 con 0x562df83fde80 2024-04-22T13:26:22.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.801+0000 7f439576d640 1 --2- 172.21.15.12:0/3992792917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562df8406a00 0x562df83ff960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:22.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.801+0000 7f4387fff640 1 --2- 172.21.15.12:0/3992792917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562df83fde80 0x562df840f280 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f43800bab90 tx=0x7f43800035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218f81c7a2e53fe4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:22.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.801+0000 7f436ffff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:26:22.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.801+0000 7f436ffff640 1 -- 172.21.15.12:0/3992792917 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43800be730 con 0x562df83fde80 2024-04-22T13:26:22.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.801+0000 7f436ffff640 1 -- 172.21.15.12:0/3992792917 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f4380003cc0 con 0x562df83fde80 2024-04-22T13:26:22.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.801+0000 7f4399dbd740 1 -- 172.21.15.12:0/3992792917 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4388019070 con 0x562df83fde80 2024-04-22T13:26:22.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.801+0000 7f436ffff640 1 -- 172.21.15.12:0/3992792917 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f43800c2ae0 con 0x562df83fde80 2024-04-22T13:26:22.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.801+0000 7f4399dbd740 10 client.16593 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:22.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.801+0000 7f4399dbd740 1 -- 172.21.15.12:0/3992792917 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562df847caa0 con 0x562df83fde80 2024-04-22T13:26:22.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.802+0000 7f436ffff640 1 -- 172.21.15.12:0/3992792917 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4388019070 con 0x562df83fde80 2024-04-22T13:26:22.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.802+0000 7f436ffff640 1 -- 172.21.15.12:0/3992792917 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f43800ba480 con 0x562df83fde80 2024-04-22T13:26:22.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.802+0000 7f4399dbd740 10 client.16593 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:22.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.802+0000 7f4399dbd740 10 client.16593 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:26:22.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.802+0000 7f4399dbd740 20 client.16593 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:22.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.802+0000 7f4399dbd740 1 --2- 172.21.15.12:0/3992792917 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x562df847d2a0 0x562df847f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.802+0000 7f4399dbd740 4 client.16593 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:26:22.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.802+0000 7f4399dbd740 1 -- 172.21.15.12:0/3992792917 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562df836fc10 con 0x562df847d2a0 2024-04-22T13:26:22.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.802+0000 7f4394f6c640 1 --2- 172.21.15.12:0/3992792917 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x562df847d2a0 0x562df847f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.802+0000 7f4394f6c640 1 --2- 172.21.15.12:0/3992792917 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x562df847d2a0 0x562df847f6c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6120596d211e4c61 server_cookie=6d1ff0c5637261c5 in_seq=0 out_seq=0 2024-04-22T13:26:22.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.802+0000 7f436ffff640 10 client.16593 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:26:22.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.803+0000 7f436ffff640 1 -- 172.21.15.12:0/3992792917 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562df836fc10 con 0x562df847d2a0 2024-04-22T13:26:22.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.803+0000 7f436ffff640 10 client.16593 handle_command_reply: tid=0 2024-04-22T13:26:22.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.803+0000 7f4399dbd740 1 client.16593 shutdown 2024-04-22T13:26:22.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.803+0000 7f4399dbd740 1 -- 172.21.15.12:0/3992792917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562df83fde80 msgr2=0x562df840f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.803+0000 7f4399dbd740 1 --2- 172.21.15.12:0/3992792917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562df83fde80 0x562df840f280 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f43800bab90 tx=0x7f43800035f0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.803+0000 7f4399dbd740 1 -- 172.21.15.12:0/3992792917 shutdown_connections 2024-04-22T13:26:22.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.803+0000 7f4399dbd740 1 --2- 172.21.15.12:0/3992792917 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562df8406a00 0x562df83ff960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.803+0000 7f4399dbd740 1 --2- 172.21.15.12:0/3992792917 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x562df847d2a0 0x562df847f6c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.803+0000 7f4399dbd740 1 --2- 172.21.15.12:0/3992792917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562df83fde80 0x562df840f280 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.803+0000 7f4399dbd740 1 --2- 172.21.15.12:0/3992792917 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562df83fd4d0 0x562df840ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.803+0000 7f436ffff640 0 client.16593 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:26:22.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.803+0000 7f4399dbd740 1 -- 172.21.15.12:0/3992792917 >> 172.21.15.12:0/3992792917 conn(0x562df83f7ba0 msgr2=0x562df83feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:22.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.803+0000 7f4399dbd740 1 -- 172.21.15.12:0/3992792917 shutdown_connections 2024-04-22T13:26:22.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.803+0000 7f4399dbd740 1 -- 172.21.15.12:0/3992792917 wait complete. 2024-04-22T13:26:22.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.803+0000 7f4399dbd740 20 client.16593 trim_cache size 0 max 16384 2024-04-22T13:26:22.806 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:26:22.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.804+0000 7f43971d0640 1 -- 172.21.15.12:0/1749984605 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f43780a3da0 msgr2=0x7f43780a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.804+0000 7f43971d0640 1 --2- 172.21.15.12:0/1749984605 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f43780a3da0 0x7f43780a6260 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4388005fd0 tx=0x7f4388011b50 comp rx=0 tx=0).stop 2024-04-22T13:26:22.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.804+0000 7f43971d0640 1 -- 172.21.15.12:0/1749984605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4390107260 msgr2=0x7f4390198cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.804+0000 7f43971d0640 1 --2- 172.21.15.12:0/1749984605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4390107260 0x7f4390198cd0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f4380000c00 tx=0x7f438002c660 comp rx=0 tx=0).stop 2024-04-22T13:26:22.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.804+0000 7f43971d0640 1 -- 172.21.15.12:0/1749984605 shutdown_connections 2024-04-22T13:26:22.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.804+0000 7f43971d0640 1 --2- 172.21.15.12:0/1749984605 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f43780a3da0 0x7f43780a6260 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.804+0000 7f43971d0640 1 --2- 172.21.15.12:0/1749984605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f439019d000 0x7f439019d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.804+0000 7f43971d0640 1 --2- 172.21.15.12:0/1749984605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4390107260 0x7f4390198cd0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.804+0000 7f43971d0640 1 --2- 172.21.15.12:0/1749984605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4390102190 0x7f4390198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.804+0000 7f43971d0640 1 -- 172.21.15.12:0/1749984605 >> 172.21.15.12:0/1749984605 conn(0x7f43900fd9f0 msgr2=0x7f43900fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:22.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.804+0000 7f43971d0640 1 -- 172.21.15.12:0/1749984605 shutdown_connections 2024-04-22T13:26:22.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.804+0000 7f43971d0640 1 -- 172.21.15.12:0/1749984605 wait complete. 2024-04-22T13:26:22.812 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:26:22.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.930+0000 7f498bcba640 1 -- 172.21.15.12:0/2067962265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f498410c510 msgr2=0x7f498410e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.930+0000 7f498bcba640 1 --2- 172.21.15.12:0/2067962265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f498410c510 0x7f498410e9b0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f497c009ca0 tx=0x7f497c02c230 comp rx=0 tx=0).stop 2024-04-22T13:26:22.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.930+0000 7f498bcba640 1 -- 172.21.15.12:0/2067962265 shutdown_connections 2024-04-22T13:26:22.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.930+0000 7f498bcba640 1 --2- 172.21.15.12:0/2067962265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f498410c510 0x7f498410e9b0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.930+0000 7f498bcba640 1 --2- 172.21.15.12:0/2067962265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4984103af0 0x7f498410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.930+0000 7f498bcba640 1 --2- 172.21.15.12:0/2067962265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49841031d0 0x7f49841035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.930+0000 7f498bcba640 1 -- 172.21.15.12:0/2067962265 >> 172.21.15.12:0/2067962265 conn(0x7f4984077f50 msgr2=0x7f4984078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:22.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.930+0000 7f498bcba640 1 -- 172.21.15.12:0/2067962265 shutdown_connections 2024-04-22T13:26:22.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.930+0000 7f498bcba640 1 -- 172.21.15.12:0/2067962265 wait complete. 2024-04-22T13:26:22.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.930+0000 7f498bcba640 1 Processor -- start 2024-04-22T13:26:22.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.930+0000 7f498bcba640 1 -- start start 2024-04-22T13:26:22.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.931+0000 7f498bcba640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49841031d0 0x7f49841986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.931+0000 7f498bcba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4984103af0 0x7f4984198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.931+0000 7f498bcba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f498410c510 0x7f498419cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.931+0000 7f498bcba640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4984112180 con 0x7f4984103af0 2024-04-22T13:26:22.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.931+0000 7f498bcba640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4984112000 con 0x7f49841031d0 2024-04-22T13:26:22.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.931+0000 7f498bcba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4984112300 con 0x7f498410c510 2024-04-22T13:26:22.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.931+0000 7f4989255640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4984103af0 0x7f4984198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.931+0000 7f4989255640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4984103af0 0x7f4984198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43208/0 (socket says 172.21.15.12:43208) 2024-04-22T13:26:22.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.931+0000 7f4989255640 1 -- 172.21.15.12:0/2193192929 learned_addr learned my addr 172.21.15.12:0/2193192929 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:22.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.931+0000 7f4989a56640 1 --2- 172.21.15.12:0/2193192929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49841031d0 0x7f49841986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.931+0000 7f498a257640 1 --2- 172.21.15.12:0/2193192929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f498410c510 0x7f498419cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.931+0000 7f4989255640 1 -- 172.21.15.12:0/2193192929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f498410c510 msgr2=0x7f498419cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.931+0000 7f4989255640 1 --2- 172.21.15.12:0/2193192929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f498410c510 0x7f498419cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.931+0000 7f4989255640 1 -- 172.21.15.12:0/2193192929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49841031d0 msgr2=0x7f49841986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.931+0000 7f4989255640 1 --2- 172.21.15.12:0/2193192929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49841031d0 0x7f49841986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.931+0000 7f4989255640 1 -- 172.21.15.12:0/2193192929 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498419d6d0 con 0x7f4984103af0 2024-04-22T13:26:22.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.931+0000 7f4989a56640 1 --2- 172.21.15.12:0/2193192929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49841031d0 0x7f49841986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:22.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.931+0000 7f498a257640 1 --2- 172.21.15.12:0/2193192929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f498410c510 0x7f498419cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:22.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.932+0000 7f4989255640 1 --2- 172.21.15.12:0/2193192929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4984103af0 0x7f4984198c20 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f497400e9f0 tx=0x7f497400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:22.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.932+0000 7f497affd640 1 -- 172.21.15.12:0/2193192929 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497400cd80 con 0x7f4984103af0 2024-04-22T13:26:22.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.932+0000 7f497affd640 1 -- 172.21.15.12:0/2193192929 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49740108e0 con 0x7f4984103af0 2024-04-22T13:26:22.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.932+0000 7f498bcba640 1 -- 172.21.15.12:0/2193192929 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f498419d9c0 con 0x7f4984103af0 2024-04-22T13:26:22.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.932+0000 7f497affd640 1 -- 172.21.15.12:0/2193192929 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974018980 con 0x7f4984103af0 2024-04-22T13:26:22.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.932+0000 7f498bcba640 1 -- 172.21.15.12:0/2193192929 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49841a5250 con 0x7f4984103af0 2024-04-22T13:26:22.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.934+0000 7f497affd640 1 -- 172.21.15.12:0/2193192929 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4974010430 con 0x7f4984103af0 2024-04-22T13:26:22.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.934+0000 7f498e988740 1 Processor -- start 2024-04-22T13:26:22.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.935+0000 7f498e988740 1 -- start start 2024-04-22T13:26:22.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.935+0000 7f498e988740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56050ebe54d0 0x56050ebe58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.935+0000 7f498e988740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050ebe5e80 0x56050ebee430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.935+0000 7f498e988740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56050ebeea00 0x56050ebf0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.935+0000 7f498e988740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56050eb4a1f0 con 0x56050ebeea00 2024-04-22T13:26:22.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.935+0000 7f498e988740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56050eae74b0 con 0x56050ebe54d0 2024-04-22T13:26:22.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.935+0000 7f498e988740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56050eb461b0 con 0x56050ebe5e80 2024-04-22T13:26:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.935+0000 7f4989a56640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050ebe5e80 0x56050ebee430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.935+0000 7f4989255640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56050ebeea00 0x56050ebf0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.935+0000 7f4989a56640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050ebe5e80 0x56050ebee430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.12:38836/0 (socket says 172.21.15.12:38836) 2024-04-22T13:26:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.935+0000 7f4989a56640 1 -- 172.21.15.12:0/723086913 learned_addr learned my addr 172.21.15.12:0/723086913 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.935+0000 7f498a257640 1 --2- 172.21.15.12:0/723086913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56050ebe54d0 0x56050ebe58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.935+0000 7f497affd640 1 --2- 172.21.15.12:0/2193192929 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f49700a3d20 0x7f49700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.935+0000 7f4989a56640 1 -- 172.21.15.12:0/723086913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56050ebe54d0 msgr2=0x56050ebe58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.935+0000 7f4989a56640 1 --2- 172.21.15.12:0/723086913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56050ebe54d0 0x56050ebe58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.935+0000 7f4989a56640 1 -- 172.21.15.12:0/723086913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56050ebeea00 msgr2=0x56050ebf0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.935+0000 7f4989a56640 1 --2- 172.21.15.12:0/723086913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56050ebeea00 0x56050ebf0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.935+0000 7f4989a56640 1 -- 172.21.15.12:0/723086913 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56050ead91b0 con 0x56050ebe5e80 2024-04-22T13:26:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.936+0000 7f498a257640 1 --2- 172.21.15.12:0/2193192929 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f49700a3d20 0x7f49700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.936+0000 7f497affd640 1 -- 172.21.15.12:0/2193192929 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f49740d0a70 con 0x7f4984103af0 2024-04-22T13:26:22.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.936+0000 7f4989a56640 1 --2- 172.21.15.12:0/723086913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050ebe5e80 0x56050ebee430 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f49800115f0 tx=0x7f4980011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b2642f98d56bec6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:22.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.936+0000 7f496ffff640 1 -- 172.21.15.12:0/723086913 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498001e070 con 0x56050ebe5e80 2024-04-22T13:26:22.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.936+0000 7f496ffff640 1 -- 172.21.15.12:0/723086913 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f498000faf0 con 0x56050ebe5e80 2024-04-22T13:26:22.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.936+0000 7f496ffff640 1 -- 172.21.15.12:0/723086913 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4980019b20 con 0x56050ebe5e80 2024-04-22T13:26:22.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.937+0000 7f498e988740 1 -- 172.21.15.12:0/723086913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050ebe5e80 msgr2=0x56050ebee430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.937+0000 7f498e988740 1 --2- 172.21.15.12:0/723086913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050ebe5e80 0x56050ebee430 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f49800115f0 tx=0x7f4980011ab0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.937+0000 7f498a257640 1 --2- 172.21.15.12:0/2193192929 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f49700a3d20 0x7f49700a61e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f497c004af0 tx=0x7f497c02c230 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:22.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.937+0000 7f498e988740 1 -- 172.21.15.12:0/723086913 shutdown_connections 2024-04-22T13:26:22.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.937+0000 7f498e988740 1 --2- 172.21.15.12:0/723086913 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56050ebeea00 0x56050ebf0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.937+0000 7f498e988740 1 --2- 172.21.15.12:0/723086913 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050ebe5e80 0x56050ebee430 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.937+0000 7f498e988740 1 --2- 172.21.15.12:0/723086913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56050ebe54d0 0x56050ebe58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.937+0000 7f498e988740 1 -- 172.21.15.12:0/723086913 >> 172.21.15.12:0/723086913 conn(0x56050ebdfba0 msgr2=0x56050ebdff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:22.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.937+0000 7f498e988740 1 -- 172.21.15.12:0/723086913 shutdown_connections 2024-04-22T13:26:22.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.937+0000 7f498e988740 1 -- 172.21.15.12:0/723086913 wait complete. 2024-04-22T13:26:22.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.937+0000 7f498e988740 1 -- start start 2024-04-22T13:26:22.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.937+0000 7f498e988740 1 Processor -- start 2024-04-22T13:26:22.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.938+0000 7f498e988740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050ebe54d0 0x56050ebee290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.938+0000 7f498e988740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56050ebe5e80 0x56050ebeb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.938+0000 7f498e988740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56050ebeea00 0x56050ebebb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.938+0000 7f498e988740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56050ebf1970 con 0x56050ebeea00 2024-04-22T13:26:22.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.938+0000 7f498e988740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56050eb03500 con 0x56050ebe5e80 2024-04-22T13:26:22.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.938+0000 7f498e988740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56050ebf1af0 con 0x56050ebe54d0 2024-04-22T13:26:22.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.938+0000 7f498e988740 10 client.0 _ll_register_callbacks cb 0x56050ead1b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:22.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.938+0000 7f498a257640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56050ebeea00 0x56050ebebb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.938+0000 7f4989a56640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050ebe54d0 0x56050ebee290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.938+0000 7f498a257640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56050ebeea00 0x56050ebebb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43234/0 (socket says 172.21.15.12:43234) 2024-04-22T13:26:22.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.938+0000 7f498a257640 1 -- 172.21.15.12:0/631150710 learned_addr learned my addr 172.21.15.12:0/631150710 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:22.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.938+0000 7f4989255640 1 --2- 172.21.15.12:0/631150710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56050ebe5e80 0x56050ebeb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.938+0000 7f4989a56640 1 -- 172.21.15.12:0/631150710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56050ebe5e80 msgr2=0x56050ebeb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.938+0000 7f4989a56640 1 --2- 172.21.15.12:0/631150710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56050ebe5e80 0x56050ebeb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.938+0000 7f4989a56640 1 -- 172.21.15.12:0/631150710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56050ebeea00 msgr2=0x56050ebebb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.938+0000 7f4989a56640 1 --2- 172.21.15.12:0/631150710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56050ebeea00 0x56050ebebb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.938+0000 7f4989a56640 1 -- 172.21.15.12:0/631150710 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56050e85c3f0 con 0x56050ebe54d0 2024-04-22T13:26:22.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.938+0000 7f4989a56640 1 --2- 172.21.15.12:0/631150710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050ebe54d0 0x56050ebee290 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x56050ebe0c10 tx=0x7f4980019780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=847ac1d69caf1ef0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:22.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.938+0000 7f496d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-22T13:26:22.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.939+0000 7f496d7fa640 1 -- 172.21.15.12:0/631150710 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498001e040 con 0x56050ebe54d0 2024-04-22T13:26:22.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.939+0000 7f496d7fa640 1 -- 172.21.15.12:0/631150710 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f498000f640 con 0x56050ebe54d0 2024-04-22T13:26:22.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.939+0000 7f498e988740 1 -- 172.21.15.12:0/631150710 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f498000faf0 con 0x56050ebe54d0 2024-04-22T13:26:22.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.939+0000 7f496d7fa640 1 -- 172.21.15.12:0/631150710 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f498000f840 con 0x56050ebe54d0 2024-04-22T13:26:22.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.939+0000 7f498e988740 10 client.25957 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:22.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.939+0000 7f498e988740 1 -- 172.21.15.12:0/631150710 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56050e9267d0 con 0x56050ebe54d0 2024-04-22T13:26:22.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.939+0000 7f496d7fa640 1 -- 172.21.15.12:0/631150710 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f498000faf0 con 0x56050ebe54d0 2024-04-22T13:26:22.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.939+0000 7f496d7fa640 1 -- 172.21.15.12:0/631150710 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4980021070 con 0x56050ebe54d0 2024-04-22T13:26:22.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.939+0000 7f498e988740 10 client.25957 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:22.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.939+0000 7f498e988740 10 client.25957 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:26:22.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.939+0000 7f498e988740 20 client.25957 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:22.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.939+0000 7f498e988740 1 --2- 172.21.15.12:0/631150710 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56050ebfead0 0x56050ec00ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.939+0000 7f498e988740 4 client.25957 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:26:22.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.939+0000 7f498e988740 1 -- 172.21.15.12:0/631150710 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56050eae9930 con 0x56050ebfead0 2024-04-22T13:26:22.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.940+0000 7f4989255640 1 --2- 172.21.15.12:0/631150710 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56050ebfead0 0x56050ec00ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.940+0000 7f4989255640 1 --2- 172.21.15.12:0/631150710 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56050ebfead0 0x56050ec00ec0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=26cd7b27dd6b7b04 server_cookie=1edb4f94251e5fbb in_seq=0 out_seq=0 2024-04-22T13:26:22.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.940+0000 7f496d7fa640 10 client.25957 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:26:22.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.941+0000 7f496d7fa640 1 -- 172.21.15.12:0/631150710 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56050eae9930 con 0x56050ebfead0 2024-04-22T13:26:22.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.941+0000 7f496d7fa640 10 client.25957 handle_command_reply: tid=0 2024-04-22T13:26:22.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.941+0000 7f498e988740 1 client.25957 shutdown 2024-04-22T13:26:22.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.941+0000 7f498e988740 1 -- 172.21.15.12:0/631150710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050ebe54d0 msgr2=0x56050ebee290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.941+0000 7f498e988740 1 --2- 172.21.15.12:0/631150710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050ebe54d0 0x56050ebee290 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x56050ebe0c10 tx=0x7f4980019780 comp rx=0 tx=0).stop 2024-04-22T13:26:22.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.941+0000 7f498e988740 1 -- 172.21.15.12:0/631150710 shutdown_connections 2024-04-22T13:26:22.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.941+0000 7f498e988740 1 --2- 172.21.15.12:0/631150710 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56050ebfead0 0x56050ec00ec0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.941+0000 7f496d7fa640 0 client.25957 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:26:22.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.941+0000 7f498e988740 1 --2- 172.21.15.12:0/631150710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56050ebeea00 0x56050ebebb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.941+0000 7f498e988740 1 --2- 172.21.15.12:0/631150710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56050ebe5e80 0x56050ebeb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.941+0000 7f498e988740 1 --2- 172.21.15.12:0/631150710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050ebe54d0 0x56050ebee290 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.941+0000 7f498e988740 1 -- 172.21.15.12:0/631150710 >> 172.21.15.12:0/631150710 conn(0x56050ebdfba0 msgr2=0x56050ebe18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:22.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.941+0000 7f498e988740 1 -- 172.21.15.12:0/631150710 shutdown_connections 2024-04-22T13:26:22.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.941+0000 7f498e988740 1 -- 172.21.15.12:0/631150710 wait complete. 2024-04-22T13:26:22.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.941+0000 7f498e988740 20 client.25957 trim_cache size 0 max 16384 2024-04-22T13:26:22.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.957+0000 7f498e988740 1 Processor -- start 2024-04-22T13:26:22.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.957+0000 7f498e988740 1 -- start start 2024-04-22T13:26:22.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.957+0000 7f498e988740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56050ebe54d0 0x56050ebfa890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.957+0000 7f498e988740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050ebe5e80 0x56050ebdee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.957+0000 7f498e988740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56050ebeea00 0x56050ebedd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.957+0000 7f498e988740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f49841adad0 con 0x56050ebe54d0 2024-04-22T13:26:22.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.957+0000 7f498e988740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f49841af610 con 0x56050ebeea00 2024-04-22T13:26:22.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.957+0000 7f498e988740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f49841ae1a0 con 0x56050ebe5e80 2024-04-22T13:26:22.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.957+0000 7f4989a56640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56050ebe54d0 0x56050ebfa890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.957+0000 7f4989a56640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56050ebe54d0 0x56050ebfa890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43238/0 (socket says 172.21.15.12:43238) 2024-04-22T13:26:22.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.957+0000 7f4989a56640 1 -- 172.21.15.12:0/2253874388 learned_addr learned my addr 172.21.15.12:0/2253874388 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:22.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.957+0000 7f4989255640 1 --2- 172.21.15.12:0/2253874388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050ebe5e80 0x56050ebdee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.957+0000 7f498a257640 1 --2- 172.21.15.12:0/2253874388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56050ebeea00 0x56050ebedd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.957+0000 7f4989a56640 1 -- 172.21.15.12:0/2253874388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050ebe5e80 msgr2=0x56050ebdee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.957+0000 7f4989a56640 1 --2- 172.21.15.12:0/2253874388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050ebe5e80 0x56050ebdee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.957+0000 7f4989a56640 1 -- 172.21.15.12:0/2253874388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56050ebeea00 msgr2=0x56050ebedd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.957+0000 7f4989a56640 1 --2- 172.21.15.12:0/2253874388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56050ebeea00 0x56050ebedd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.958+0000 7f4989a56640 1 -- 172.21.15.12:0/2253874388 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56050ebe47f0 con 0x56050ebe54d0 2024-04-22T13:26:22.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.958+0000 7f4989255640 1 --2- 172.21.15.12:0/2253874388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050ebe5e80 0x56050ebdee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:22.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.958+0000 7f498a257640 1 --2- 172.21.15.12:0/2253874388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56050ebeea00 0x56050ebedd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:22.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.958+0000 7f4989a56640 1 --2- 172.21.15.12:0/2253874388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56050ebe54d0 0x56050ebfa890 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f4980005610 tx=0x7f4980013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491908e670cbd7b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:22.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.958+0000 7f496f7fe640 1 -- 172.21.15.12:0/2253874388 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498001e040 con 0x56050ebe54d0 2024-04-22T13:26:22.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.958+0000 7f496f7fe640 1 -- 172.21.15.12:0/2253874388 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4980009b50 con 0x56050ebe54d0 2024-04-22T13:26:22.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.958+0000 7f496f7fe640 1 -- 172.21.15.12:0/2253874388 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4980003850 con 0x56050ebe54d0 2024-04-22T13:26:22.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.959+0000 7f498e988740 1 -- 172.21.15.12:0/2253874388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56050ebe54d0 msgr2=0x56050ebfa890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.959+0000 7f498e988740 1 --2- 172.21.15.12:0/2253874388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56050ebe54d0 0x56050ebfa890 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f4980005610 tx=0x7f4980013000 comp rx=0 tx=0).stop 2024-04-22T13:26:22.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.959+0000 7f498e988740 1 -- 172.21.15.12:0/2253874388 shutdown_connections 2024-04-22T13:26:22.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.959+0000 7f498e988740 1 --2- 172.21.15.12:0/2253874388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56050ebeea00 0x56050ebedd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.959+0000 7f498e988740 1 --2- 172.21.15.12:0/2253874388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050ebe5e80 0x56050ebdee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.959+0000 7f498e988740 1 --2- 172.21.15.12:0/2253874388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56050ebe54d0 0x56050ebfa890 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.959+0000 7f498e988740 1 -- 172.21.15.12:0/2253874388 >> 172.21.15.12:0/2253874388 conn(0x56050ebdfba0 msgr2=0x56050ebeb740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:22.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.959+0000 7f498e988740 1 -- 172.21.15.12:0/2253874388 shutdown_connections 2024-04-22T13:26:22.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.959+0000 7f498e988740 1 -- 172.21.15.12:0/2253874388 wait complete. 2024-04-22T13:26:22.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.959+0000 7f498e988740 1 -- start start 2024-04-22T13:26:22.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.959+0000 7f498e988740 1 Processor -- start 2024-04-22T13:26:22.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.960+0000 7f498e988740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050ebe54d0 0x56050ebf6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.960+0000 7f498e988740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56050ebe5e80 0x56050ebf7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.960+0000 7f498e988740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56050ebeea00 0x56050ebf21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.960+0000 7f498e988740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f49841aef40 con 0x56050ebeea00 2024-04-22T13:26:22.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.960+0000 7f498e988740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f49841ae870 con 0x56050ebe5e80 2024-04-22T13:26:22.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.960+0000 7f498e988740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f49841abab0 con 0x56050ebe54d0 2024-04-22T13:26:22.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.960+0000 7f498e988740 10 client.0 _ll_register_callbacks cb 0x7f4980005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:22.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.960+0000 7f498a257640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56050ebeea00 0x56050ebf21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.960+0000 7f4989255640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56050ebe5e80 0x56050ebf7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.960+0000 7f4989a56640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050ebe54d0 0x56050ebf6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.960+0000 7f498a257640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56050ebeea00 0x56050ebf21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43242/0 (socket says 172.21.15.12:43242) 2024-04-22T13:26:22.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.960+0000 7f498a257640 1 -- 172.21.15.12:0/3432197274 learned_addr learned my addr 172.21.15.12:0/3432197274 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:22.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.960+0000 7f4989255640 1 -- 172.21.15.12:0/3432197274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050ebe54d0 msgr2=0x56050ebf6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.960+0000 7f4989255640 1 --2- 172.21.15.12:0/3432197274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050ebe54d0 0x56050ebf6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.960+0000 7f4989255640 1 -- 172.21.15.12:0/3432197274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56050ebeea00 msgr2=0x56050ebf21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:22.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.960+0000 7f4989255640 1 --2- 172.21.15.12:0/3432197274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56050ebeea00 0x56050ebf21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.960+0000 7f4989255640 1 -- 172.21.15.12:0/3432197274 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56050ebf29e0 con 0x56050ebe5e80 2024-04-22T13:26:22.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.960+0000 7f498a257640 1 --2- 172.21.15.12:0/3432197274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56050ebeea00 0x56050ebf21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:22.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.961+0000 7f4989255640 1 --2- 172.21.15.12:0/3432197274 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56050ebe5e80 0x56050ebf7200 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f49740ce1b0 tx=0x7f49740bd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be30f23f061f137d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:22.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.961+0000 7f496d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-22T13:26:22.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.961+0000 7f496d7fa640 1 -- 172.21.15.12:0/3432197274 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49740d0e60 con 0x56050ebe5e80 2024-04-22T13:26:22.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.961+0000 7f496d7fa640 1 -- 172.21.15.12:0/3432197274 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f4974099120 con 0x56050ebe5e80 2024-04-22T13:26:22.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.961+0000 7f498e988740 1 -- 172.21.15.12:0/3432197274 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4980009b50 con 0x56050ebe5e80 2024-04-22T13:26:22.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.961+0000 7f496d7fa640 1 -- 172.21.15.12:0/3432197274 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f497409c410 con 0x56050ebe5e80 2024-04-22T13:26:22.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.961+0000 7f498e988740 10 client.26027 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:22.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.961+0000 7f498e988740 1 -- 172.21.15.12:0/3432197274 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56050ec64a50 con 0x56050ebe5e80 2024-04-22T13:26:22.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.961+0000 7f496d7fa640 1 -- 172.21.15.12:0/3432197274 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49740b7080 con 0x56050ebe5e80 2024-04-22T13:26:22.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.961+0000 7f496d7fa640 1 -- 172.21.15.12:0/3432197274 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f49740ce430 con 0x56050ebe5e80 2024-04-22T13:26:22.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.961+0000 7f498e988740 10 client.26027 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:22.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.961+0000 7f498e988740 10 client.26027 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:26:22.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.961+0000 7f498e988740 20 client.26027 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:22.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.961+0000 7f498e988740 1 --2- 172.21.15.12:0/3432197274 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56050ebfead0 0x56050ec67260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:22.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.962+0000 7f498e988740 4 client.26027 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:26:22.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.962+0000 7f498e988740 1 -- 172.21.15.12:0/3432197274 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56050eb57c10 con 0x56050ebfead0 2024-04-22T13:26:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.962+0000 7f4989a56640 1 --2- 172.21.15.12:0/3432197274 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56050ebfead0 0x56050ec67260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.962+0000 7f4989a56640 1 --2- 172.21.15.12:0/3432197274 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56050ebfead0 0x56050ec67260 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a09956af63b14d82 server_cookie=5c325430e7a2ce55 in_seq=0 out_seq=0 2024-04-22T13:26:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.962+0000 7f496d7fa640 10 client.26027 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:26:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.962+0000 7f496d7fa640 1 -- 172.21.15.12:0/3432197274 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56050eb57c10 con 0x56050ebfead0 2024-04-22T13:26:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.962+0000 7f496d7fa640 10 client.26027 handle_command_reply: tid=0 2024-04-22T13:26:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.962+0000 7f498e988740 1 client.26027 shutdown 2024-04-22T13:26:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.962+0000 7f498e988740 1 -- 172.21.15.12:0/3432197274 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56050ebe5e80 msgr2=0x56050ebf7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.962+0000 7f498e988740 1 --2- 172.21.15.12:0/3432197274 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56050ebe5e80 0x56050ebf7200 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f49740ce1b0 tx=0x7f49740bd040 comp rx=0 tx=0).stop 2024-04-22T13:26:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.963+0000 7f498e988740 1 -- 172.21.15.12:0/3432197274 shutdown_connections 2024-04-22T13:26:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.963+0000 7f498e988740 1 --2- 172.21.15.12:0/3432197274 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56050ebfead0 0x56050ec67260 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.963+0000 7f498e988740 1 --2- 172.21.15.12:0/3432197274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56050ebeea00 0x56050ebf21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.963+0000 7f498e988740 1 --2- 172.21.15.12:0/3432197274 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56050ebe5e80 0x56050ebf7200 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.963+0000 7f498e988740 1 --2- 172.21.15.12:0/3432197274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56050ebe54d0 0x56050ebf6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.963+0000 7f496d7fa640 0 client.26027 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:26:22.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.963+0000 7f498e988740 1 -- 172.21.15.12:0/3432197274 >> 172.21.15.12:0/3432197274 conn(0x56050ebdfba0 msgr2=0x56050ebf9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:22.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.963+0000 7f498e988740 1 -- 172.21.15.12:0/3432197274 shutdown_connections 2024-04-22T13:26:22.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.963+0000 7f498e988740 1 -- 172.21.15.12:0/3432197274 wait complete. 2024-04-22T13:26:22.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.963+0000 7f498e988740 20 client.26027 trim_cache size 0 max 16384 2024-04-22T13:26:22.966 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:26:22.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.963+0000 7f498bcba640 1 -- 172.21.15.12:0/2193192929 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f49700a3d20 msgr2=0x7f49700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.963+0000 7f498bcba640 1 --2- 172.21.15.12:0/2193192929 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f49700a3d20 0x7f49700a61e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f497c004af0 tx=0x7f497c02c230 comp rx=0 tx=0).stop 2024-04-22T13:26:22.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.963+0000 7f498bcba640 1 -- 172.21.15.12:0/2193192929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4984103af0 msgr2=0x7f4984198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:22.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.963+0000 7f498bcba640 1 --2- 172.21.15.12:0/2193192929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4984103af0 0x7f4984198c20 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f497400e9f0 tx=0x7f497400eeb0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.963+0000 7f498bcba640 1 -- 172.21.15.12:0/2193192929 shutdown_connections 2024-04-22T13:26:22.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.963+0000 7f498bcba640 1 --2- 172.21.15.12:0/2193192929 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f49700a3d20 0x7f49700a61e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.963+0000 7f498bcba640 1 --2- 172.21.15.12:0/2193192929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f498410c510 0x7f498419cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.963+0000 7f498bcba640 1 --2- 172.21.15.12:0/2193192929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4984103af0 0x7f4984198c20 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.963+0000 7f498bcba640 1 --2- 172.21.15.12:0/2193192929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49841031d0 0x7f49841986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:22.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.963+0000 7f498bcba640 1 -- 172.21.15.12:0/2193192929 >> 172.21.15.12:0/2193192929 conn(0x7f4984077f50 msgr2=0x7f49840fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:22.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.963+0000 7f498bcba640 1 -- 172.21.15.12:0/2193192929 shutdown_connections 2024-04-22T13:26:22.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:22.964+0000 7f498bcba640 1 -- 172.21.15.12:0/2193192929 wait complete. 2024-04-22T13:26:22.971 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:22 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/650984596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:26:22.972 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:26:23.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.090+0000 7f6444ef2640 1 -- 172.21.15.12:0/1523972048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f644010e900 msgr2=0x7f6440110cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:23.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.090+0000 7f6444ef2640 1 --2- 172.21.15.12:0/1523972048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f644010e900 0x7f6440110cf0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f6438009ca0 tx=0x7f643802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.090+0000 7f6444ef2640 1 -- 172.21.15.12:0/1523972048 shutdown_connections 2024-04-22T13:26:23.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.090+0000 7f6444ef2640 1 --2- 172.21.15.12:0/1523972048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f644010e900 0x7f6440110cf0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.090+0000 7f6444ef2640 1 --2- 172.21.15.12:0/1523972048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6440101670 0x7f644010e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.090+0000 7f6444ef2640 1 --2- 172.21.15.12:0/1523972048 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6440100d50 0x7f6440101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.090+0000 7f6444ef2640 1 -- 172.21.15.12:0/1523972048 >> 172.21.15.12:0/1523972048 conn(0x7f6440077f50 msgr2=0x7f6440078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:23.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.090+0000 7f6444ef2640 1 -- 172.21.15.12:0/1523972048 shutdown_connections 2024-04-22T13:26:23.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.090+0000 7f6444ef2640 1 -- 172.21.15.12:0/1523972048 wait complete. 2024-04-22T13:26:23.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.091+0000 7f6444ef2640 1 Processor -- start 2024-04-22T13:26:23.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.091+0000 7f6444ef2640 1 -- start start 2024-04-22T13:26:23.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.091+0000 7f6444ef2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6440100d50 0x7f64401a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:23.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.091+0000 7f6444ef2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6440101670 0x7f64401a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:23.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.091+0000 7f6444ef2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f644010e900 0x7f644019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:23.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.091+0000 7f6444ef2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f64401143a0 con 0x7f6440101670 2024-04-22T13:26:23.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.091+0000 7f6444ef2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6440114220 con 0x7f644010e900 2024-04-22T13:26:23.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.091+0000 7f6444ef2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6440114520 con 0x7f6440100d50 2024-04-22T13:26:23.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.091+0000 7f6437fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6440101670 0x7f64401a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:23.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.091+0000 7f643e59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6440100d50 0x7f64401a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:23.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.091+0000 7f6437fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6440101670 0x7f64401a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43262/0 (socket says 172.21.15.12:43262) 2024-04-22T13:26:23.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.091+0000 7f643ed9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f644010e900 0x7f644019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:23.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.091+0000 7f643e59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6440100d50 0x7f64401a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.12:38898/0 (socket says 172.21.15.12:38898) 2024-04-22T13:26:23.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.091+0000 7f6437fff640 1 -- 172.21.15.12:0/1646669284 learned_addr learned my addr 172.21.15.12:0/1646669284 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:23.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.091+0000 7f643ed9d640 1 -- 172.21.15.12:0/1646669284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6440100d50 msgr2=0x7f64401a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:23.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.091+0000 7f643ed9d640 1 --2- 172.21.15.12:0/1646669284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6440100d50 0x7f64401a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.091+0000 7f643ed9d640 1 -- 172.21.15.12:0/1646669284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6440101670 msgr2=0x7f64401a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:23.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.091+0000 7f643ed9d640 1 --2- 172.21.15.12:0/1646669284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6440101670 0x7f64401a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.091+0000 7f643ed9d640 1 -- 172.21.15.12:0/1646669284 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644019b020 con 0x7f644010e900 2024-04-22T13:26:23.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.091+0000 7f6437fff640 1 --2- 172.21.15.12:0/1646669284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6440101670 0x7f64401a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:23.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.091+0000 7f643e59c640 1 --2- 172.21.15.12:0/1646669284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6440100d50 0x7f64401a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:23.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.092+0000 7f643ed9d640 1 --2- 172.21.15.12:0/1646669284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f644010e900 0x7f644019a760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f643802c630 tx=0x7f64380346d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:23.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.092+0000 7f64377fe640 1 -- 172.21.15.12:0/1646669284 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438041020 con 0x7f644010e900 2024-04-22T13:26:23.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.092+0000 7f6444ef2640 1 -- 172.21.15.12:0/1646669284 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f644019b2b0 con 0x7f644010e900 2024-04-22T13:26:23.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.092+0000 7f64377fe640 1 -- 172.21.15.12:0/1646669284 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6438034ca0 con 0x7f644010e900 2024-04-22T13:26:23.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.092+0000 7f6444ef2640 1 -- 172.21.15.12:0/1646669284 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64401a73f0 con 0x7f644010e900 2024-04-22T13:26:23.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.092+0000 7f64377fe640 1 -- 172.21.15.12:0/1646669284 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438039040 con 0x7f644010e900 2024-04-22T13:26:23.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.094+0000 7f64377fe640 1 -- 172.21.15.12:0/1646669284 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6438044020 con 0x7f644010e900 2024-04-22T13:26:23.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.094+0000 7f6447bbf740 1 Processor -- start 2024-04-22T13:26:23.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.095+0000 7f6447bbf740 1 -- start start 2024-04-22T13:26:23.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.095+0000 7f6447bbf740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f28f5664d0 0x55f28f5668b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:23.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.095+0000 7f6447bbf740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f28f566e80 0x55f28f56f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:23.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.095+0000 7f6447bbf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f28f56fa00 0x55f28f571e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:23.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.095+0000 7f6447bbf740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f28f4cb1f0 con 0x55f28f566e80 2024-04-22T13:26:23.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.095+0000 7f6447bbf740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f28f4684b0 con 0x55f28f5664d0 2024-04-22T13:26:23.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.095+0000 7f6447bbf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f28f4c71b0 con 0x55f28f56fa00 2024-04-22T13:26:23.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.095+0000 7f643e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f28f566e80 0x55f28f56f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:23.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.095+0000 7f643e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f28f566e80 0x55f28f56f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43272/0 (socket says 172.21.15.12:43272) 2024-04-22T13:26:23.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.095+0000 7f6437fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f28f5664d0 0x55f28f5668b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:23.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.095+0000 7f643e59c640 1 -- 172.21.15.12:0/3705850450 learned_addr learned my addr 172.21.15.12:0/3705850450 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:23.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.095+0000 7f6437fff640 1 --2- 172.21.15.12:0/3705850450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f28f56fa00 0x55f28f571e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:23.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.095+0000 7f64377fe640 1 --2- 172.21.15.12:0/1646669284 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f642c0a3d20 0x7f642c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:23.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.095+0000 7f643ed9d640 1 --2- 172.21.15.12:0/1646669284 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f642c0a3d20 0x7f642c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:23.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.095+0000 7f643e59c640 1 -- 172.21.15.12:0/3705850450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f28f56fa00 msgr2=0x55f28f571e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:23.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.095+0000 7f643e59c640 1 --2- 172.21.15.12:0/3705850450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f28f56fa00 0x55f28f571e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.095+0000 7f643e59c640 1 -- 172.21.15.12:0/3705850450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f28f5664d0 msgr2=0x55f28f5668b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:23.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.095+0000 7f643e59c640 1 --2- 172.21.15.12:0/3705850450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f28f5664d0 0x55f28f5668b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.095+0000 7f643e59c640 1 -- 172.21.15.12:0/3705850450 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f28f45a1b0 con 0x55f28f566e80 2024-04-22T13:26:23.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.095+0000 7f6437fff640 1 --2- 172.21.15.12:0/3705850450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f28f5664d0 0x55f28f5668b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:23.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.095+0000 7f64377fe640 1 -- 172.21.15.12:0/1646669284 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f64380f2250 con 0x7f644010e900 2024-04-22T13:26:23.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.095+0000 7f643e59c640 1 --2- 172.21.15.12:0/3705850450 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f28f566e80 0x55f28f56f430 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f64300115f0 tx=0x7f6430011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444e43264afd27a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:23.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.096+0000 7f6434ff9640 1 -- 172.21.15.12:0/3705850450 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643001e070 con 0x55f28f566e80 2024-04-22T13:26:23.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.096+0000 7f643ed9d640 1 --2- 172.21.15.12:0/1646669284 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f642c0a3d20 0x7f642c0a61e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f64380efff0 tx=0x7f643802c0f0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:23.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.096+0000 7f6434ff9640 1 -- 172.21.15.12:0/3705850450 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f643000faf0 con 0x55f28f566e80 2024-04-22T13:26:23.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.096+0000 7f6434ff9640 1 -- 172.21.15.12:0/3705850450 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6430019a50 con 0x55f28f566e80 2024-04-22T13:26:23.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.096+0000 7f6447bbf740 1 -- 172.21.15.12:0/3705850450 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f28f566e80 msgr2=0x55f28f56f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:23.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.096+0000 7f6447bbf740 1 --2- 172.21.15.12:0/3705850450 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f28f566e80 0x55f28f56f430 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f64300115f0 tx=0x7f6430011ab0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.097+0000 7f6447bbf740 1 -- 172.21.15.12:0/3705850450 shutdown_connections 2024-04-22T13:26:23.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.097+0000 7f6447bbf740 1 --2- 172.21.15.12:0/3705850450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f28f56fa00 0x55f28f571e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.097+0000 7f6447bbf740 1 --2- 172.21.15.12:0/3705850450 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f28f566e80 0x55f28f56f430 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.097+0000 7f6447bbf740 1 --2- 172.21.15.12:0/3705850450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f28f5664d0 0x55f28f5668b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.097+0000 7f6447bbf740 1 -- 172.21.15.12:0/3705850450 >> 172.21.15.12:0/3705850450 conn(0x55f28f560ba0 msgr2=0x55f28f560f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:23.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.097+0000 7f6447bbf740 1 -- 172.21.15.12:0/3705850450 shutdown_connections 2024-04-22T13:26:23.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.097+0000 7f6447bbf740 1 -- 172.21.15.12:0/3705850450 wait complete. 2024-04-22T13:26:23.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.097+0000 7f6447bbf740 1 -- start start 2024-04-22T13:26:23.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.097+0000 7f6447bbf740 1 Processor -- start 2024-04-22T13:26:23.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.097+0000 7f6447bbf740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f28f5664d0 0x55f28f56f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:23.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.098+0000 7f6447bbf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f28f566e80 0x55f28f56c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:23.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.098+0000 7f6447bbf740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f28f56fa00 0x55f28f56cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:23.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.098+0000 7f6447bbf740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f28f572970 con 0x55f28f56fa00 2024-04-22T13:26:23.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.098+0000 7f6447bbf740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f28f484500 con 0x55f28f5664d0 2024-04-22T13:26:23.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.098+0000 7f6447bbf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f28f572af0 con 0x55f28f566e80 2024-04-22T13:26:23.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.098+0000 7f6447bbf740 10 client.0 _ll_register_callbacks cb 0x55f28f452b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:23.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.098+0000 7f6437fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f28f5664d0 0x55f28f56f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:23.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.098+0000 7f6437fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f28f56fa00 0x55f28f56cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:23.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.098+0000 7f6437fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f28f56fa00 0x55f28f56cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43284/0 (socket says 172.21.15.12:43284) 2024-04-22T13:26:23.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.098+0000 7f6437fff640 1 -- 172.21.15.12:0/2601880475 learned_addr learned my addr 172.21.15.12:0/2601880475 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:23.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.098+0000 7f643e59c640 1 --2- 172.21.15.12:0/2601880475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f28f566e80 0x55f28f56c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:23.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.098+0000 7f6437fff640 1 -- 172.21.15.12:0/2601880475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f28f566e80 msgr2=0x55f28f56c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:23.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.098+0000 7f6437fff640 1 --2- 172.21.15.12:0/2601880475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f28f566e80 0x55f28f56c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.098+0000 7f6437fff640 1 -- 172.21.15.12:0/2601880475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f28f5664d0 msgr2=0x55f28f56f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:23.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.098+0000 7f6437fff640 1 --2- 172.21.15.12:0/2601880475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f28f5664d0 0x55f28f56f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.098+0000 7f6437fff640 1 -- 172.21.15.12:0/2601880475 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f28f1dd3f0 con 0x55f28f56fa00 2024-04-22T13:26:23.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.098+0000 7f6437fff640 1 --2- 172.21.15.12:0/2601880475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f28f56fa00 0x55f28f56cc10 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f642800f520 tx=0x7f64280095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e8a961a3ef9bdf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:23.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.099+0000 7f6425ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:26:23.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.099+0000 7f6425ffb640 1 -- 172.21.15.12:0/2601880475 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642801a040 con 0x55f28f56fa00 2024-04-22T13:26:23.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.099+0000 7f6425ffb640 1 -- 172.21.15.12:0/2601880475 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f6428005220 con 0x55f28f56fa00 2024-04-22T13:26:23.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.099+0000 7f6447bbf740 1 -- 172.21.15.12:0/2601880475 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f643000faf0 con 0x55f28f56fa00 2024-04-22T13:26:23.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.099+0000 7f6425ffb640 1 -- 172.21.15.12:0/2601880475 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6428005420 con 0x55f28f56fa00 2024-04-22T13:26:23.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.099+0000 7f6447bbf740 10 client.16623 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:23.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.099+0000 7f6447bbf740 1 -- 172.21.15.12:0/2601880475 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f28f2a77d0 con 0x55f28f56fa00 2024-04-22T13:26:23.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.099+0000 7f6425ffb640 1 -- 172.21.15.12:0/2601880475 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f643000faf0 con 0x55f28f56fa00 2024-04-22T13:26:23.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.099+0000 7f6425ffb640 1 -- 172.21.15.12:0/2601880475 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f642801e580 con 0x55f28f56fa00 2024-04-22T13:26:23.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.099+0000 7f6447bbf740 10 client.16623 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:23.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.099+0000 7f6447bbf740 10 client.16623 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:26:23.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.099+0000 7f6447bbf740 20 client.16623 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:23.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.099+0000 7f6447bbf740 1 --2- 172.21.15.12:0/2601880475 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f28f57fc50 0x55f28f5820b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:23.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.100+0000 7f6447bbf740 4 client.16623 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:26:23.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.100+0000 7f6447bbf740 1 -- 172.21.15.12:0/2601880475 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f28f46a930 con 0x55f28f57fc50 2024-04-22T13:26:23.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.100+0000 7f643e59c640 1 --2- 172.21.15.12:0/2601880475 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f28f57fc50 0x55f28f5820b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:23.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.100+0000 7f643e59c640 1 --2- 172.21.15.12:0/2601880475 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f28f57fc50 0x55f28f5820b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbcde005436538 server_cookie=c3464686686a4d86 in_seq=0 out_seq=0 2024-04-22T13:26:23.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.100+0000 7f6425ffb640 10 client.16623 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:23.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.101+0000 7f6425ffb640 1 -- 172.21.15.12:0/2601880475 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f28f46a930 con 0x55f28f57fc50 2024-04-22T13:26:23.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.101+0000 7f6425ffb640 10 client.16623 handle_command_reply: tid=0 2024-04-22T13:26:23.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.101+0000 7f6447bbf740 1 client.16623 shutdown 2024-04-22T13:26:23.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.101+0000 7f6447bbf740 1 -- 172.21.15.12:0/2601880475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f28f56fa00 msgr2=0x55f28f56cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:23.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.101+0000 7f6447bbf740 1 --2- 172.21.15.12:0/2601880475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f28f56fa00 0x55f28f56cc10 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f642800f520 tx=0x7f64280095e0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.101+0000 7f6447bbf740 1 -- 172.21.15.12:0/2601880475 shutdown_connections 2024-04-22T13:26:23.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.101+0000 7f6447bbf740 1 --2- 172.21.15.12:0/2601880475 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f28f57fc50 0x55f28f5820b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.101+0000 7f6447bbf740 1 --2- 172.21.15.12:0/2601880475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f28f56fa00 0x55f28f56cc10 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.101+0000 7f6447bbf740 1 --2- 172.21.15.12:0/2601880475 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f28f566e80 0x55f28f56c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.101+0000 7f6447bbf740 1 --2- 172.21.15.12:0/2601880475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f28f5664d0 0x55f28f56f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.101+0000 7f6425ffb640 0 client.16623 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:23.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.101+0000 7f6447bbf740 1 -- 172.21.15.12:0/2601880475 >> 172.21.15.12:0/2601880475 conn(0x55f28f560ba0 msgr2=0x55f28f5628f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:23.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.101+0000 7f6447bbf740 1 -- 172.21.15.12:0/2601880475 shutdown_connections 2024-04-22T13:26:23.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.101+0000 7f6447bbf740 1 -- 172.21.15.12:0/2601880475 wait complete. 2024-04-22T13:26:23.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.101+0000 7f6447bbf740 20 client.16623 trim_cache size 0 max 16384 2024-04-22T13:26:23.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.116+0000 7f6447bbf740 1 Processor -- start 2024-04-22T13:26:23.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.116+0000 7f6447bbf740 1 -- start start 2024-04-22T13:26:23.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.116+0000 7f6447bbf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f28f5664d0 0x55f28f57b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:23.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.116+0000 7f6447bbf740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f28f566e80 0x55f28f562700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:23.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.116+0000 7f6447bbf740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f28f56fa00 0x55f28f56edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:23.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.116+0000 7f6447bbf740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f64401afc50 con 0x55f28f566e80 2024-04-22T13:26:23.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.116+0000 7f6447bbf740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f64401b1790 con 0x55f28f56fa00 2024-04-22T13:26:23.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.116+0000 7f6447bbf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f64401b0320 con 0x55f28f5664d0 2024-04-22T13:26:23.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.117+0000 7f643e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f28f566e80 0x55f28f562700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:23.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.117+0000 7f6437fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f28f5664d0 0x55f28f57b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:23.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.117+0000 7f643e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f28f566e80 0x55f28f562700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43298/0 (socket says 172.21.15.12:43298) 2024-04-22T13:26:23.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.117+0000 7f643e59c640 1 -- 172.21.15.12:0/1198461733 learned_addr learned my addr 172.21.15.12:0/1198461733 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:23.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.117+0000 7f6437fff640 1 --2- 172.21.15.12:0/1198461733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f28f56fa00 0x55f28f56edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:23.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.117+0000 7f643e59c640 1 -- 172.21.15.12:0/1198461733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f28f5664d0 msgr2=0x55f28f57b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:23.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.117+0000 7f643e59c640 1 --2- 172.21.15.12:0/1198461733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f28f5664d0 0x55f28f57b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.117+0000 7f643e59c640 1 -- 172.21.15.12:0/1198461733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f28f56fa00 msgr2=0x55f28f56edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:23.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.117+0000 7f643e59c640 1 --2- 172.21.15.12:0/1198461733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f28f56fa00 0x55f28f56edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.117+0000 7f643e59c640 1 -- 172.21.15.12:0/1198461733 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f28f565790 con 0x55f28f566e80 2024-04-22T13:26:23.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.117+0000 7f6437fff640 1 --2- 172.21.15.12:0/1198461733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f28f5664d0 0x55f28f57b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:23.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.117+0000 7f643e59c640 1 --2- 172.21.15.12:0/1198461733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f28f566e80 0x55f28f562700 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6430009230 tx=0x7f6430005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df9a45249421486 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:23.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.117+0000 7f6427fff640 1 -- 172.21.15.12:0/1198461733 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643001e040 con 0x55f28f566e80 2024-04-22T13:26:23.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.117+0000 7f6427fff640 1 -- 172.21.15.12:0/1198461733 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6430005040 con 0x55f28f566e80 2024-04-22T13:26:23.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.117+0000 7f6427fff640 1 -- 172.21.15.12:0/1198461733 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6430019950 con 0x55f28f566e80 2024-04-22T13:26:23.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.118+0000 7f6447bbf740 1 -- 172.21.15.12:0/1198461733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f28f566e80 msgr2=0x55f28f562700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:23.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.118+0000 7f6447bbf740 1 --2- 172.21.15.12:0/1198461733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f28f566e80 0x55f28f562700 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6430009230 tx=0x7f6430005850 comp rx=0 tx=0).stop 2024-04-22T13:26:23.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.118+0000 7f6447bbf740 1 -- 172.21.15.12:0/1198461733 shutdown_connections 2024-04-22T13:26:23.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.118+0000 7f6447bbf740 1 --2- 172.21.15.12:0/1198461733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f28f56fa00 0x55f28f56edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.118+0000 7f6447bbf740 1 --2- 172.21.15.12:0/1198461733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f28f566e80 0x55f28f562700 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.118+0000 7f6447bbf740 1 --2- 172.21.15.12:0/1198461733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f28f5664d0 0x55f28f57b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.118+0000 7f6447bbf740 1 -- 172.21.15.12:0/1198461733 >> 172.21.15.12:0/1198461733 conn(0x55f28f560ba0 msgr2=0x55f28f561ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:23.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.118+0000 7f6447bbf740 1 -- 172.21.15.12:0/1198461733 shutdown_connections 2024-04-22T13:26:23.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.118+0000 7f6447bbf740 1 -- 172.21.15.12:0/1198461733 wait complete. 2024-04-22T13:26:23.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.118+0000 7f6447bbf740 1 -- start start 2024-04-22T13:26:23.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.119+0000 7f6447bbf740 1 Processor -- start 2024-04-22T13:26:23.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.119+0000 7f6447bbf740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f28f5664d0 0x55f28f5782e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:23.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.119+0000 7f6447bbf740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f28f566e80 0x55f28f569160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:23.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.119+0000 7f6447bbf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f28f56fa00 0x55f28f5698a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:23.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.119+0000 7f6447bbf740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f64401b10c0 con 0x55f28f566e80 2024-04-22T13:26:23.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.119+0000 7f6447bbf740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f64401b09f0 con 0x55f28f5664d0 2024-04-22T13:26:23.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.119+0000 7f6447bbf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f64401adc30 con 0x55f28f56fa00 2024-04-22T13:26:23.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.119+0000 7f6447bbf740 10 client.0 _ll_register_callbacks cb 0x7f642800f3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:23.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.119+0000 7f643e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f28f566e80 0x55f28f569160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:23.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.119+0000 7f643e59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f28f566e80 0x55f28f569160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43314/0 (socket says 172.21.15.12:43314) 2024-04-22T13:26:23.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.119+0000 7f6437fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f28f5664d0 0x55f28f5782e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:23.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.119+0000 7f643e59c640 1 -- 172.21.15.12:0/217829975 learned_addr learned my addr 172.21.15.12:0/217829975 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:23.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.119+0000 7f6437fff640 1 --2- 172.21.15.12:0/217829975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f28f56fa00 0x55f28f5698a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:23.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.119+0000 7f643e59c640 1 -- 172.21.15.12:0/217829975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f28f56fa00 msgr2=0x55f28f5698a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:23.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.119+0000 7f643e59c640 1 --2- 172.21.15.12:0/217829975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f28f56fa00 0x55f28f5698a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.119+0000 7f643e59c640 1 -- 172.21.15.12:0/217829975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f28f5664d0 msgr2=0x55f28f5782e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:23.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.119+0000 7f643e59c640 1 --2- 172.21.15.12:0/217829975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f28f5664d0 0x55f28f5782e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.119+0000 7f643e59c640 1 -- 172.21.15.12:0/217829975 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f28f5e51f0 con 0x55f28f566e80 2024-04-22T13:26:23.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.119+0000 7f6437fff640 1 --2- 172.21.15.12:0/217829975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f28f5664d0 0x55f28f5782e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:23.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.119+0000 7f643e59c640 1 --2- 172.21.15.12:0/217829975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f28f566e80 0x55f28f569160 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f64300115c0 tx=0x7f6430005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a213d57c7ffd1eee server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:23.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.119+0000 7f6425ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:26:23.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.120+0000 7f6425ffb640 1 -- 172.21.15.12:0/217829975 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643001e040 con 0x55f28f566e80 2024-04-22T13:26:23.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.120+0000 7f6425ffb640 1 -- 172.21.15.12:0/217829975 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f643000f850 con 0x55f28f566e80 2024-04-22T13:26:23.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.120+0000 7f6447bbf740 1 -- 172.21.15.12:0/217829975 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6430005040 con 0x55f28f566e80 2024-04-22T13:26:23.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.120+0000 7f6425ffb640 1 -- 172.21.15.12:0/217829975 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f643000fa50 con 0x55f28f566e80 2024-04-22T13:26:23.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.120+0000 7f6447bbf740 10 client.16629 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:23.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.120+0000 7f6447bbf740 1 -- 172.21.15.12:0/217829975 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f28f5e6350 con 0x55f28f566e80 2024-04-22T13:26:23.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.120+0000 7f6425ffb640 1 -- 172.21.15.12:0/217829975 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6430005040 con 0x55f28f566e80 2024-04-22T13:26:23.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.120+0000 7f6425ffb640 1 -- 172.21.15.12:0/217829975 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6430019860 con 0x55f28f566e80 2024-04-22T13:26:23.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.120+0000 7f6447bbf740 10 client.16629 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:23.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.120+0000 7f6447bbf740 10 client.16629 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:26:23.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.120+0000 7f6447bbf740 20 client.16629 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:23.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.120+0000 7f6447bbf740 1 --2- 172.21.15.12:0/217829975 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f28f5e67f0 0x55f28f5e8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:23.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.120+0000 7f6447bbf740 4 client.16629 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:26:23.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.120+0000 7f6447bbf740 1 -- 172.21.15.12:0/217829975 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f28f4d8c10 con 0x55f28f5e67f0 2024-04-22T13:26:23.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.120+0000 7f6437fff640 1 --2- 172.21.15.12:0/217829975 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f28f5e67f0 0x55f28f5e8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:23.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.121+0000 7f6437fff640 1 --2- 172.21.15.12:0/217829975 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f28f5e67f0 0x55f28f5e8c10 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff3ef99001ecab94 server_cookie=1ee11ae59dd5d9d0 in_seq=0 out_seq=0 2024-04-22T13:26:23.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.121+0000 7f6425ffb640 10 client.16629 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:23.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.121+0000 7f6425ffb640 1 -- 172.21.15.12:0/217829975 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f28f4d8c10 con 0x55f28f5e67f0 2024-04-22T13:26:23.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.121+0000 7f6425ffb640 10 client.16629 handle_command_reply: tid=0 2024-04-22T13:26:23.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.121+0000 7f6447bbf740 1 client.16629 shutdown 2024-04-22T13:26:23.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.121+0000 7f6447bbf740 1 -- 172.21.15.12:0/217829975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f28f566e80 msgr2=0x55f28f569160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:23.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.121+0000 7f6447bbf740 1 --2- 172.21.15.12:0/217829975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f28f566e80 0x55f28f569160 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f64300115c0 tx=0x7f6430005da0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.121+0000 7f6447bbf740 1 -- 172.21.15.12:0/217829975 shutdown_connections 2024-04-22T13:26:23.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.121+0000 7f6447bbf740 1 --2- 172.21.15.12:0/217829975 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f28f5e67f0 0x55f28f5e8c10 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.121+0000 7f6447bbf740 1 --2- 172.21.15.12:0/217829975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f28f56fa00 0x55f28f5698a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.121+0000 7f6447bbf740 1 --2- 172.21.15.12:0/217829975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f28f566e80 0x55f28f569160 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.121+0000 7f6447bbf740 1 --2- 172.21.15.12:0/217829975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f28f5664d0 0x55f28f5782e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.121+0000 7f6447bbf740 1 -- 172.21.15.12:0/217829975 >> 172.21.15.12:0/217829975 conn(0x55f28f560ba0 msgr2=0x55f28f55fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:23.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.121+0000 7f6425ffb640 0 client.16629 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:23.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.121+0000 7f6447bbf740 1 -- 172.21.15.12:0/217829975 shutdown_connections 2024-04-22T13:26:23.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.121+0000 7f6447bbf740 1 -- 172.21.15.12:0/217829975 wait complete. 2024-04-22T13:26:23.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.121+0000 7f6447bbf740 20 client.16629 trim_cache size 0 max 16384 2024-04-22T13:26:23.125 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:26:23.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.122+0000 7f6444ef2640 1 -- 172.21.15.12:0/1646669284 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f642c0a3d20 msgr2=0x7f642c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:23.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.122+0000 7f6444ef2640 1 --2- 172.21.15.12:0/1646669284 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f642c0a3d20 0x7f642c0a61e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f64380efff0 tx=0x7f643802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.122+0000 7f6444ef2640 1 -- 172.21.15.12:0/1646669284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f644010e900 msgr2=0x7f644019a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:23.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.122+0000 7f6444ef2640 1 --2- 172.21.15.12:0/1646669284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f644010e900 0x7f644019a760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f643802c630 tx=0x7f64380346d0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.122+0000 7f6444ef2640 1 -- 172.21.15.12:0/1646669284 shutdown_connections 2024-04-22T13:26:23.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.122+0000 7f6444ef2640 1 --2- 172.21.15.12:0/1646669284 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f642c0a3d20 0x7f642c0a61e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.122+0000 7f6444ef2640 1 --2- 172.21.15.12:0/1646669284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f644010e900 0x7f644019a760 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.122+0000 7f6444ef2640 1 --2- 172.21.15.12:0/1646669284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6440101670 0x7f64401a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.122+0000 7f6444ef2640 1 --2- 172.21.15.12:0/1646669284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6440100d50 0x7f64401a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:23.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.122+0000 7f6444ef2640 1 -- 172.21.15.12:0/1646669284 >> 172.21.15.12:0/1646669284 conn(0x7f6440077f50 msgr2=0x7f644010ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:23.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.122+0000 7f6444ef2640 1 -- 172.21.15.12:0/1646669284 shutdown_connections 2024-04-22T13:26:23.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:23.122+0000 7f6444ef2640 1 -- 172.21.15.12:0/1646669284 wait complete. 2024-04-22T13:26:23.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:22 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/650984596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:26:23.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:22 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/650984596' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:26:24.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:23 smithi012 ceph-mon[28265]: pgmap v71: 97 pgs: 97 active+clean; 259 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.4 MiB/s wr, 1.09k op/s 2024-04-22T13:26:24.131 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:26:24.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:26:24.131 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:26:24.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:23 smithi134 ceph-mon[31338]: pgmap v71: 97 pgs: 97 active+clean; 259 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.4 MiB/s wr, 1.09k op/s 2024-04-22T13:26:24.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:23 smithi111 ceph-mon[31361]: pgmap v71: 97 pgs: 97 active+clean; 259 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.4 MiB/s wr, 1.09k op/s 2024-04-22T13:26:24.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.251+0000 7f974fc6d640 1 -- 172.21.15.12:0/2562712210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9748100fc0 msgr2=0x7f97481013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:24.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.251+0000 7f974fc6d640 1 --2- 172.21.15.12:0/2562712210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9748100fc0 0x7f97481013a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f9744009900 tx=0x7f974402bd60 comp rx=0 tx=0).stop 2024-04-22T13:26:24.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.252+0000 7f974fc6d640 1 -- 172.21.15.12:0/2562712210 shutdown_connections 2024-04-22T13:26:24.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.252+0000 7f974fc6d640 1 --2- 172.21.15.12:0/2562712210 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97481062e0 0x7f9748106770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.252+0000 7f974fc6d640 1 --2- 172.21.15.12:0/2562712210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97481018e0 0x7f9748105da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.252+0000 7f974fc6d640 1 --2- 172.21.15.12:0/2562712210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9748100fc0 0x7f97481013a0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.252+0000 7f974fc6d640 1 -- 172.21.15.12:0/2562712210 >> 172.21.15.12:0/2562712210 conn(0x7f9748077f50 msgr2=0x7f9748078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:24.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.252+0000 7f974fc6d640 1 -- 172.21.15.12:0/2562712210 shutdown_connections 2024-04-22T13:26:24.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.252+0000 7f974fc6d640 1 -- 172.21.15.12:0/2562712210 wait complete. 2024-04-22T13:26:24.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.252+0000 7f974fc6d640 1 Processor -- start 2024-04-22T13:26:24.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.252+0000 7f974fc6d640 1 -- start start 2024-04-22T13:26:24.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.252+0000 7f974fc6d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9748100fc0 0x7f974819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.252+0000 7f974fc6d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97481018e0 0x7f974819f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.252+0000 7f974fc6d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97481062e0 0x7f97481a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.252+0000 7f974fc6d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9748118790 con 0x7f97481062e0 2024-04-22T13:26:24.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.252+0000 7f974fc6d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9748118610 con 0x7f9748100fc0 2024-04-22T13:26:24.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.252+0000 7f974fc6d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9748118910 con 0x7f97481018e0 2024-04-22T13:26:24.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.253+0000 7f974da09640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9748100fc0 0x7f974819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.253+0000 7f974e20a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97481062e0 0x7f97481a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.253+0000 7f974d208640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97481018e0 0x7f974819f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.253+0000 7f974da09640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9748100fc0 0x7f974819ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:44226/0 (socket says 172.21.15.12:44226) 2024-04-22T13:26:24.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.253+0000 7f974e20a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97481062e0 0x7f97481a3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43330/0 (socket says 172.21.15.12:43330) 2024-04-22T13:26:24.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.253+0000 7f974da09640 1 -- 172.21.15.12:0/1730513519 learned_addr learned my addr 172.21.15.12:0/1730513519 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:24.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.253+0000 7f974e20a640 1 -- 172.21.15.12:0/1730513519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97481018e0 msgr2=0x7f974819f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:24.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.253+0000 7f974e20a640 1 --2- 172.21.15.12:0/1730513519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97481018e0 0x7f974819f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.253+0000 7f974e20a640 1 -- 172.21.15.12:0/1730513519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9748100fc0 msgr2=0x7f974819ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:24.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.253+0000 7f974e20a640 1 --2- 172.21.15.12:0/1730513519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9748100fc0 0x7f974819ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.253+0000 7f974e20a640 1 -- 172.21.15.12:0/1730513519 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97481a3d80 con 0x7f97481062e0 2024-04-22T13:26:24.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.253+0000 7f974e20a640 1 --2- 172.21.15.12:0/1730513519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97481062e0 0x7f97481a3650 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f9740009fd0 tx=0x7f974000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:24.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.253+0000 7f973effd640 1 -- 172.21.15.12:0/1730513519 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974000ce20 con 0x7f97481062e0 2024-04-22T13:26:24.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.253+0000 7f973effd640 1 -- 172.21.15.12:0/1730513519 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9740005020 con 0x7f97481062e0 2024-04-22T13:26:24.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.253+0000 7f974fc6d640 1 -- 172.21.15.12:0/1730513519 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97481a4070 con 0x7f97481062e0 2024-04-22T13:26:24.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.253+0000 7f973effd640 1 -- 172.21.15.12:0/1730513519 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9740010690 con 0x7f97481062e0 2024-04-22T13:26:24.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.253+0000 7f974fc6d640 1 -- 172.21.15.12:0/1730513519 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97481ab900 con 0x7f97481062e0 2024-04-22T13:26:24.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.256+0000 7f973effd640 1 -- 172.21.15.12:0/1730513519 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f97400054d0 con 0x7f97481062e0 2024-04-22T13:26:24.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.257+0000 7f9752866740 1 Processor -- start 2024-04-22T13:26:24.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.257+0000 7f9752866740 1 -- start start 2024-04-22T13:26:24.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.257+0000 7f973effd640 1 --2- 172.21.15.12:0/1730513519 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f97340a3da0 0x7f97340a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.258+0000 7f9752866740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1b5d148e0 0x55b1b5d14cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.258+0000 7f9752866740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1b5d15290 0x55b1b5d1d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.258+0000 7f973effd640 1 -- 172.21.15.12:0/1730513519 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f97400d0d70 con 0x7f97481062e0 2024-04-22T13:26:24.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.258+0000 7f974d208640 1 --2- 172.21.15.12:0/1730513519 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f97340a3da0 0x7f97340a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.258+0000 7f9752866740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1b5d1de10 0x55b1b5d20260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.258+0000 7f9752866740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b1b5c78f00 con 0x55b1b5d148e0 2024-04-22T13:26:24.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.258+0000 7f9752866740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b1b5c16590 con 0x55b1b5d1de10 2024-04-22T13:26:24.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.258+0000 7f9752866740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b1b5c74ec0 con 0x55b1b5d15290 2024-04-22T13:26:24.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.258+0000 7f974da09640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1b5d148e0 0x55b1b5d14cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.258+0000 7f974da09640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1b5d148e0 0x55b1b5d14cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43336/0 (socket says 172.21.15.12:43336) 2024-04-22T13:26:24.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.258+0000 7f974da09640 1 -- 172.21.15.12:0/2632122768 learned_addr learned my addr 172.21.15.12:0/2632122768 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:24.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.258+0000 7f974e20a640 1 --2- 172.21.15.12:0/2632122768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1b5d1de10 0x55b1b5d20260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.258+0000 7f974d208640 1 --2- 172.21.15.12:0/2632122768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1b5d15290 0x55b1b5d1d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.258+0000 7f974da09640 1 -- 172.21.15.12:0/2632122768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1b5d15290 msgr2=0x55b1b5d1d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:24.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.258+0000 7f974da09640 1 --2- 172.21.15.12:0/2632122768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1b5d15290 0x55b1b5d1d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.258+0000 7f974da09640 1 -- 172.21.15.12:0/2632122768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1b5d1de10 msgr2=0x55b1b5d20260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:24.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.258+0000 7f974da09640 1 --2- 172.21.15.12:0/2632122768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1b5d1de10 0x55b1b5d20260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.258+0000 7f974da09640 1 -- 172.21.15.12:0/2632122768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1b5a537e0 con 0x55b1b5d148e0 2024-04-22T13:26:24.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.258+0000 7f974d208640 1 --2- 172.21.15.12:0/2632122768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1b5d15290 0x55b1b5d1d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:26:24.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.258+0000 7f974d208640 1 --2- 172.21.15.12:0/1730513519 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f97340a3da0 0x7f97340a6260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9748078860 tx=0x7f9738011440 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:24.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.258+0000 7f974da09640 1 --2- 172.21.15.12:0/2632122768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1b5d148e0 0x55b1b5d14cc0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f9744036040 tx=0x7f97440095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b289e9c2ae7ede0c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:24.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.258+0000 7f9733fff640 1 -- 172.21.15.12:0/2632122768 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9744004040 con 0x55b1b5d148e0 2024-04-22T13:26:24.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.259+0000 7f9733fff640 1 -- 172.21.15.12:0/2632122768 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f974402cdc0 con 0x55b1b5d148e0 2024-04-22T13:26:24.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.259+0000 7f9733fff640 1 -- 172.21.15.12:0/2632122768 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97440055f0 con 0x55b1b5d148e0 2024-04-22T13:26:24.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.259+0000 7f9752866740 1 -- 172.21.15.12:0/2632122768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1b5d148e0 msgr2=0x55b1b5d14cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:24.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.259+0000 7f9752866740 1 --2- 172.21.15.12:0/2632122768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1b5d148e0 0x55b1b5d14cc0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f9744036040 tx=0x7f97440095e0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.259+0000 7f9752866740 1 -- 172.21.15.12:0/2632122768 shutdown_connections 2024-04-22T13:26:24.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.259+0000 7f9752866740 1 --2- 172.21.15.12:0/2632122768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1b5d1de10 0x55b1b5d20260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.259+0000 7f9752866740 1 --2- 172.21.15.12:0/2632122768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1b5d15290 0x55b1b5d1d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.259+0000 7f9752866740 1 --2- 172.21.15.12:0/2632122768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1b5d148e0 0x55b1b5d14cc0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.259+0000 7f9752866740 1 -- 172.21.15.12:0/2632122768 >> 172.21.15.12:0/2632122768 conn(0x55b1b5d0efb0 msgr2=0x55b1b5d0f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:24.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.259+0000 7f9752866740 1 -- 172.21.15.12:0/2632122768 shutdown_connections 2024-04-22T13:26:24.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.259+0000 7f9752866740 1 -- 172.21.15.12:0/2632122768 wait complete. 2024-04-22T13:26:24.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.260+0000 7f9752866740 1 -- start start 2024-04-22T13:26:24.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.260+0000 7f9752866740 1 Processor -- start 2024-04-22T13:26:24.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.260+0000 7f9752866740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1b5d148e0 0x55b1b5d1ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.260+0000 7f9752866740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1b5d15290 0x55b1b5d1b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.260+0000 7f9752866740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1b5d1de10 0x55b1b5d1b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.260+0000 7f9752866740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b1b5d20d20 con 0x55b1b5d1de10 2024-04-22T13:26:24.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.260+0000 7f9752866740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b1b5c32580 con 0x55b1b5d15290 2024-04-22T13:26:24.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.260+0000 7f9752866740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b1b5d20ea0 con 0x55b1b5d148e0 2024-04-22T13:26:24.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.260+0000 7f9752866740 10 client.0 _ll_register_callbacks cb 0x55b1b5c38650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:24.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.260+0000 7f974e20a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1b5d1de10 0x55b1b5d1b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.260+0000 7f974da09640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1b5d148e0 0x55b1b5d1ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.260+0000 7f974e20a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1b5d1de10 0x55b1b5d1b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43344/0 (socket says 172.21.15.12:43344) 2024-04-22T13:26:24.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.260+0000 7f974e20a640 1 -- 172.21.15.12:0/83172686 learned_addr learned my addr 172.21.15.12:0/83172686 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:24.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.261+0000 7f974d208640 1 --2- 172.21.15.12:0/83172686 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1b5d15290 0x55b1b5d1b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.261+0000 7f974e20a640 1 -- 172.21.15.12:0/83172686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1b5d148e0 msgr2=0x55b1b5d1ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:24.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.261+0000 7f974e20a640 1 --2- 172.21.15.12:0/83172686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1b5d148e0 0x55b1b5d1ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.261+0000 7f974e20a640 1 -- 172.21.15.12:0/83172686 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1b5d15290 msgr2=0x55b1b5d1b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:24.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.261+0000 7f974e20a640 1 --2- 172.21.15.12:0/83172686 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1b5d15290 0x55b1b5d1b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.261+0000 7f974e20a640 1 -- 172.21.15.12:0/83172686 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1b5ab3f70 con 0x55b1b5d1de10 2024-04-22T13:26:24.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.261+0000 7f974da09640 1 --2- 172.21.15.12:0/83172686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1b5d148e0 0x55b1b5d1ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:24.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.261+0000 7f974e20a640 1 --2- 172.21.15.12:0/83172686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1b5d1de10 0x55b1b5d1b820 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f974009ca60 tx=0x7f974009ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10d7bab94324b78 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:24.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.261+0000 7f97317fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:26:24.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.261+0000 7f97317fa640 1 -- 172.21.15.12:0/83172686 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97400b5070 con 0x55b1b5d1de10 2024-04-22T13:26:24.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.261+0000 7f97317fa640 1 -- 172.21.15.12:0/83172686 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f97400993d0 con 0x55b1b5d1de10 2024-04-22T13:26:24.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.261+0000 7f9752866740 1 -- 172.21.15.12:0/83172686 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f974402cdc0 con 0x55b1b5d1de10 2024-04-22T13:26:24.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.261+0000 7f97317fa640 1 -- 172.21.15.12:0/83172686 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f97400a59d0 con 0x55b1b5d1de10 2024-04-22T13:26:24.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.261+0000 7f9752866740 10 client.16641 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:24.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.261+0000 7f9752866740 1 -- 172.21.15.12:0/83172686 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b1b594b170 con 0x55b1b5d1de10 2024-04-22T13:26:24.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.262+0000 7f97317fa640 1 -- 172.21.15.12:0/83172686 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97400a5540 con 0x55b1b5d1de10 2024-04-22T13:26:24.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.262+0000 7f97317fa640 1 -- 172.21.15.12:0/83172686 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9740098ab0 con 0x55b1b5d1de10 2024-04-22T13:26:24.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.262+0000 7f9752866740 10 client.16641 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:24.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.262+0000 7f9752866740 10 client.16641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:26:24.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.262+0000 7f9752866740 20 client.16641 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:24.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.262+0000 7f9752866740 1 --2- 172.21.15.12:0/83172686 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b1b5d2e310 0x55b1b5d30750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.262+0000 7f9752866740 4 client.16641 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:26:24.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.262+0000 7f9752866740 1 -- 172.21.15.12:0/83172686 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f974402cdc0 con 0x55b1b5d2e310 2024-04-22T13:26:24.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.262+0000 7f974da09640 1 --2- 172.21.15.12:0/83172686 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b1b5d2e310 0x55b1b5d30750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.262+0000 7f974da09640 1 --2- 172.21.15.12:0/83172686 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b1b5d2e310 0x55b1b5d30750 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84d28f2d2423bdcb server_cookie=c91c8342b96c5780 in_seq=0 out_seq=0 2024-04-22T13:26:24.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.262+0000 7f97317fa640 10 client.16641 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:24.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.263+0000 7f97317fa640 1 -- 172.21.15.12:0/83172686 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f974402cdc0 con 0x55b1b5d2e310 2024-04-22T13:26:24.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.263+0000 7f97317fa640 10 client.16641 handle_command_reply: tid=0 2024-04-22T13:26:24.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.263+0000 7f9752866740 1 client.16641 shutdown 2024-04-22T13:26:24.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.263+0000 7f9752866740 1 -- 172.21.15.12:0/83172686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1b5d1de10 msgr2=0x55b1b5d1b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:24.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.263+0000 7f9752866740 1 --2- 172.21.15.12:0/83172686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1b5d1de10 0x55b1b5d1b820 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f974009ca60 tx=0x7f974009ce20 comp rx=0 tx=0).stop 2024-04-22T13:26:24.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.263+0000 7f9752866740 1 -- 172.21.15.12:0/83172686 shutdown_connections 2024-04-22T13:26:24.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.263+0000 7f9752866740 1 --2- 172.21.15.12:0/83172686 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b1b5d2e310 0x55b1b5d30750 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.263+0000 7f9752866740 1 --2- 172.21.15.12:0/83172686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1b5d1de10 0x55b1b5d1b820 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.263+0000 7f9752866740 1 --2- 172.21.15.12:0/83172686 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1b5d15290 0x55b1b5d1b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.263+0000 7f9752866740 1 --2- 172.21.15.12:0/83172686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1b5d148e0 0x55b1b5d1ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.263+0000 7f97317fa640 0 client.16641 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:24.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.263+0000 7f9752866740 1 -- 172.21.15.12:0/83172686 >> 172.21.15.12:0/83172686 conn(0x55b1b5d0efb0 msgr2=0x55b1b5d10d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:24.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.263+0000 7f9752866740 1 -- 172.21.15.12:0/83172686 shutdown_connections 2024-04-22T13:26:24.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.263+0000 7f9752866740 1 -- 172.21.15.12:0/83172686 wait complete. 2024-04-22T13:26:24.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.264+0000 7f9752866740 20 client.16641 trim_cache size 0 max 16384 2024-04-22T13:26:24.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.283+0000 7f9752866740 1 Processor -- start 2024-04-22T13:26:24.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.283+0000 7f9752866740 1 -- start start 2024-04-22T13:26:24.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.284+0000 7f9752866740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1b5d148e0 0x55b1b5d1d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.284+0000 7f9752866740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1b5d15290 0x55b1b5d29c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.284+0000 7f9752866740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1b5d1de10 0x55b1b5d36020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.284+0000 7f9752866740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f97481b40e0 con 0x55b1b5d15290 2024-04-22T13:26:24.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.284+0000 7f9752866740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f97481b5c20 con 0x55b1b5d1de10 2024-04-22T13:26:24.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.284+0000 7f9752866740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f97481b47b0 con 0x55b1b5d148e0 2024-04-22T13:26:24.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.284+0000 7f974d208640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1b5d15290 0x55b1b5d29c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.284+0000 7f974d208640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1b5d15290 0x55b1b5d29c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43346/0 (socket says 172.21.15.12:43346) 2024-04-22T13:26:24.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.284+0000 7f974d208640 1 -- 172.21.15.12:0/3488333408 learned_addr learned my addr 172.21.15.12:0/3488333408 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:24.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.284+0000 7f974e20a640 1 --2- 172.21.15.12:0/3488333408 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1b5d1de10 0x55b1b5d36020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.284+0000 7f974da09640 1 --2- 172.21.15.12:0/3488333408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1b5d148e0 0x55b1b5d1d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.284+0000 7f974d208640 1 -- 172.21.15.12:0/3488333408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1b5d148e0 msgr2=0x55b1b5d1d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:24.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.284+0000 7f974d208640 1 --2- 172.21.15.12:0/3488333408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1b5d148e0 0x55b1b5d1d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.284+0000 7f974d208640 1 -- 172.21.15.12:0/3488333408 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1b5d1de10 msgr2=0x55b1b5d36020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:24.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.284+0000 7f974d208640 1 --2- 172.21.15.12:0/3488333408 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1b5d1de10 0x55b1b5d36020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.284+0000 7f974d208640 1 -- 172.21.15.12:0/3488333408 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1b5d20200 con 0x55b1b5d15290 2024-04-22T13:26:24.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.284+0000 7f974e20a640 1 --2- 172.21.15.12:0/3488333408 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1b5d1de10 0x55b1b5d36020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:24.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.284+0000 7f974d208640 1 --2- 172.21.15.12:0/3488333408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1b5d15290 0x55b1b5d29c40 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9738011940 tx=0x7f9738005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b116c0915520e166 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:24.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.284+0000 7f97337fe640 1 -- 172.21.15.12:0/3488333408 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9738017040 con 0x55b1b5d15290 2024-04-22T13:26:24.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.285+0000 7f97337fe640 1 -- 172.21.15.12:0/3488333408 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9738019050 con 0x55b1b5d15290 2024-04-22T13:26:24.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.285+0000 7f97337fe640 1 -- 172.21.15.12:0/3488333408 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9738028a10 con 0x55b1b5d15290 2024-04-22T13:26:24.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.285+0000 7f9752866740 1 -- 172.21.15.12:0/3488333408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1b5d15290 msgr2=0x55b1b5d29c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:24.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.285+0000 7f9752866740 1 --2- 172.21.15.12:0/3488333408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1b5d15290 0x55b1b5d29c40 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9738011940 tx=0x7f9738005f90 comp rx=0 tx=0).stop 2024-04-22T13:26:24.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.285+0000 7f9752866740 1 -- 172.21.15.12:0/3488333408 shutdown_connections 2024-04-22T13:26:24.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.285+0000 7f9752866740 1 --2- 172.21.15.12:0/3488333408 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1b5d1de10 0x55b1b5d36020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.285+0000 7f9752866740 1 --2- 172.21.15.12:0/3488333408 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1b5d15290 0x55b1b5d29c40 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.285+0000 7f9752866740 1 --2- 172.21.15.12:0/3488333408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1b5d148e0 0x55b1b5d1d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.285+0000 7f9752866740 1 -- 172.21.15.12:0/3488333408 >> 172.21.15.12:0/3488333408 conn(0x55b1b5d0efb0 msgr2=0x55b1b5d0e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:24.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.285+0000 7f9752866740 1 -- 172.21.15.12:0/3488333408 shutdown_connections 2024-04-22T13:26:24.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.285+0000 7f9752866740 1 -- 172.21.15.12:0/3488333408 wait complete. 2024-04-22T13:26:24.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.286+0000 7f9752866740 1 -- start start 2024-04-22T13:26:24.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.286+0000 7f9752866740 1 Processor -- start 2024-04-22T13:26:24.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.286+0000 7f9752866740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1b5d148e0 0x55b1b5d21e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.286+0000 7f9752866740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1b5d15290 0x55b1b5d22390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.286+0000 7f9752866740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1b5d1de10 0x55b1b5d228d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.286+0000 7f9752866740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f97481b5550 con 0x55b1b5d148e0 2024-04-22T13:26:24.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.286+0000 7f9752866740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f97481b4e80 con 0x55b1b5d1de10 2024-04-22T13:26:24.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.286+0000 7f9752866740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f97481b20c0 con 0x55b1b5d15290 2024-04-22T13:26:24.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.286+0000 7f9752866740 10 client.0 _ll_register_callbacks cb 0x7f974009c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:24.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.286+0000 7f974da09640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1b5d148e0 0x55b1b5d21e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.286+0000 7f974d208640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1b5d15290 0x55b1b5d22390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.286+0000 7f974da09640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1b5d148e0 0x55b1b5d21e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43348/0 (socket says 172.21.15.12:43348) 2024-04-22T13:26:24.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.286+0000 7f974da09640 1 -- 172.21.15.12:0/3405593698 learned_addr learned my addr 172.21.15.12:0/3405593698 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:24.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.286+0000 7f974e20a640 1 --2- 172.21.15.12:0/3405593698 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1b5d1de10 0x55b1b5d228d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.287+0000 7f974da09640 1 -- 172.21.15.12:0/3405593698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1b5d15290 msgr2=0x55b1b5d22390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:24.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.287+0000 7f974da09640 1 --2- 172.21.15.12:0/3405593698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1b5d15290 0x55b1b5d22390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.287+0000 7f974da09640 1 -- 172.21.15.12:0/3405593698 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1b5d1de10 msgr2=0x55b1b5d228d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:24.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.287+0000 7f974da09640 1 --2- 172.21.15.12:0/3405593698 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1b5d1de10 0x55b1b5d228d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.287+0000 7f974da09640 1 -- 172.21.15.12:0/3405593698 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1b5d93b30 con 0x55b1b5d148e0 2024-04-22T13:26:24.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.287+0000 7f974d208640 1 --2- 172.21.15.12:0/3405593698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1b5d15290 0x55b1b5d22390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:24.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.287+0000 7f974e20a640 1 --2- 172.21.15.12:0/3405593698 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1b5d1de10 0x55b1b5d228d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:24.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.287+0000 7f97317fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:26:24.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.287+0000 7f974da09640 1 --2- 172.21.15.12:0/3405593698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1b5d148e0 0x55b1b5d21e50 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f9744009430 tx=0x7f9744005150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba8b22591f0c543 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:24.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.287+0000 7f97317fa640 1 -- 172.21.15.12:0/3405593698 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9744058060 con 0x55b1b5d148e0 2024-04-22T13:26:24.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.287+0000 7f97317fa640 1 -- 172.21.15.12:0/3405593698 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f9744053630 con 0x55b1b5d148e0 2024-04-22T13:26:24.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.287+0000 7f9752866740 1 -- 172.21.15.12:0/3405593698 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9738019050 con 0x55b1b5d148e0 2024-04-22T13:26:24.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.287+0000 7f97317fa640 1 -- 172.21.15.12:0/3405593698 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9744053830 con 0x55b1b5d148e0 2024-04-22T13:26:24.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.288+0000 7f9752866740 10 client.16647 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:24.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.288+0000 7f9752866740 1 -- 172.21.15.12:0/3405593698 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b1b5d94ed0 con 0x55b1b5d148e0 2024-04-22T13:26:24.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.288+0000 7f97317fa640 1 -- 172.21.15.12:0/3405593698 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9738019050 con 0x55b1b5d148e0 2024-04-22T13:26:24.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.288+0000 7f97317fa640 1 -- 172.21.15.12:0/3405593698 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f974405d4f0 con 0x55b1b5d148e0 2024-04-22T13:26:24.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.288+0000 7f9752866740 10 client.16647 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:24.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.288+0000 7f9752866740 10 client.16647 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:26:24.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.288+0000 7f9752866740 20 client.16647 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:24.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.288+0000 7f9752866740 1 --2- 172.21.15.12:0/3405593698 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b1b5d951c0 0x55b1b5d975b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.288+0000 7f9752866740 4 client.16647 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:26:24.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.288+0000 7f9752866740 1 -- 172.21.15.12:0/3405593698 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b1b5c86920 con 0x55b1b5d951c0 2024-04-22T13:26:24.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.288+0000 7f974d208640 1 --2- 172.21.15.12:0/3405593698 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b1b5d951c0 0x55b1b5d975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.289+0000 7f97317fa640 10 client.16647 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:24.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.289+0000 7f974d208640 1 --2- 172.21.15.12:0/3405593698 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b1b5d951c0 0x55b1b5d975b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be0d39a7b45672b3 server_cookie=a171ca6465785fb2 in_seq=0 out_seq=0 2024-04-22T13:26:24.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.290+0000 7f97317fa640 1 -- 172.21.15.12:0/3405593698 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b1b5c86920 con 0x55b1b5d951c0 2024-04-22T13:26:24.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.290+0000 7f97317fa640 10 client.16647 handle_command_reply: tid=0 2024-04-22T13:26:24.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.290+0000 7f9752866740 1 client.16647 shutdown 2024-04-22T13:26:24.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.290+0000 7f9752866740 1 -- 172.21.15.12:0/3405593698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1b5d148e0 msgr2=0x55b1b5d21e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:24.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.290+0000 7f9752866740 1 --2- 172.21.15.12:0/3405593698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1b5d148e0 0x55b1b5d21e50 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f9744009430 tx=0x7f9744005150 comp rx=0 tx=0).stop 2024-04-22T13:26:24.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.290+0000 7f9752866740 1 -- 172.21.15.12:0/3405593698 shutdown_connections 2024-04-22T13:26:24.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.290+0000 7f9752866740 1 --2- 172.21.15.12:0/3405593698 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b1b5d951c0 0x55b1b5d975b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.290+0000 7f9752866740 1 --2- 172.21.15.12:0/3405593698 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1b5d1de10 0x55b1b5d228d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.290+0000 7f9752866740 1 --2- 172.21.15.12:0/3405593698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1b5d15290 0x55b1b5d22390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.290+0000 7f97317fa640 0 client.16647 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:24.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.290+0000 7f9752866740 1 --2- 172.21.15.12:0/3405593698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1b5d148e0 0x55b1b5d21e50 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.290+0000 7f9752866740 1 -- 172.21.15.12:0/3405593698 >> 172.21.15.12:0/3405593698 conn(0x55b1b5d0efb0 msgr2=0x55b1b5d1ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:24.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.290+0000 7f9752866740 1 -- 172.21.15.12:0/3405593698 shutdown_connections 2024-04-22T13:26:24.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.290+0000 7f9752866740 1 -- 172.21.15.12:0/3405593698 wait complete. 2024-04-22T13:26:24.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.290+0000 7f9752866740 20 client.16647 trim_cache size 0 max 16384 2024-04-22T13:26:24.293 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:26:24.293 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-04-22T13:26:24.293 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "fe4911cd-e9a2-4304-901e-631f3a2f961f", 2024-04-22T13:26:24.293 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-04-22T13:26:24.293 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:26:24.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.291+0000 7f974fc6d640 1 -- 172.21.15.12:0/1730513519 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f97340a3da0 msgr2=0x7f97340a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:24.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.291+0000 7f974fc6d640 1 --2- 172.21.15.12:0/1730513519 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f97340a3da0 0x7f97340a6260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9748078860 tx=0x7f9738011440 comp rx=0 tx=0).stop 2024-04-22T13:26:24.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.291+0000 7f974fc6d640 1 -- 172.21.15.12:0/1730513519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97481062e0 msgr2=0x7f97481a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:24.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.291+0000 7f974fc6d640 1 --2- 172.21.15.12:0/1730513519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97481062e0 0x7f97481a3650 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f9740009fd0 tx=0x7f974000ef10 comp rx=0 tx=0).stop 2024-04-22T13:26:24.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.291+0000 7f974fc6d640 1 -- 172.21.15.12:0/1730513519 shutdown_connections 2024-04-22T13:26:24.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.291+0000 7f974fc6d640 1 --2- 172.21.15.12:0/1730513519 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f97340a3da0 0x7f97340a6260 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.291+0000 7f974fc6d640 1 --2- 172.21.15.12:0/1730513519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f97481062e0 0x7f97481a3650 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.291+0000 7f974fc6d640 1 --2- 172.21.15.12:0/1730513519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97481018e0 0x7f974819f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.291+0000 7f974fc6d640 1 --2- 172.21.15.12:0/1730513519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9748100fc0 0x7f974819ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.291+0000 7f974fc6d640 1 -- 172.21.15.12:0/1730513519 >> 172.21.15.12:0/1730513519 conn(0x7f9748077f50 msgr2=0x7f9748103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:24.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.291+0000 7f974fc6d640 1 -- 172.21.15.12:0/1730513519 shutdown_connections 2024-04-22T13:26:24.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.291+0000 7f974fc6d640 1 -- 172.21.15.12:0/1730513519 wait complete. 2024-04-22T13:26:24.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:26:24.300 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:26:24.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.415+0000 7f02f9be2640 1 -- 172.21.15.12:0/3291196964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02f4103ae0 msgr2=0x7f02f410bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:24.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.415+0000 7f02f9be2640 1 --2- 172.21.15.12:0/3291196964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02f4103ae0 0x7f02f410bfc0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f02e8009900 tx=0x7f02e802bd60 comp rx=0 tx=0).stop 2024-04-22T13:26:24.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.415+0000 7f02f9be2640 1 -- 172.21.15.12:0/3291196964 shutdown_connections 2024-04-22T13:26:24.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.415+0000 7f02f9be2640 1 --2- 172.21.15.12:0/3291196964 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02f410c500 0x7f02f410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.415+0000 7f02f9be2640 1 --2- 172.21.15.12:0/3291196964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02f4103ae0 0x7f02f410bfc0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.415+0000 7f02f9be2640 1 --2- 172.21.15.12:0/3291196964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02f41031c0 0x7f02f41035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.415+0000 7f02f9be2640 1 -- 172.21.15.12:0/3291196964 >> 172.21.15.12:0/3291196964 conn(0x7f02f4078000 msgr2=0x7f02f4078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:24.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.415+0000 7f02f9be2640 1 -- 172.21.15.12:0/3291196964 shutdown_connections 2024-04-22T13:26:24.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.415+0000 7f02f9be2640 1 -- 172.21.15.12:0/3291196964 wait complete. 2024-04-22T13:26:24.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.416+0000 7f02f9be2640 1 Processor -- start 2024-04-22T13:26:24.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.416+0000 7f02f9be2640 1 -- start start 2024-04-22T13:26:24.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.416+0000 7f02f9be2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02f41031c0 0x7f02f41986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.416+0000 7f02f9be2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02f4103ae0 0x7f02f4198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.416+0000 7f02f9be2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02f410c500 0x7f02f419cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.416+0000 7f02f9be2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f02f4112190 con 0x7f02f41031c0 2024-04-22T13:26:24.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.416+0000 7f02f9be2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f02f4112010 con 0x7f02f4103ae0 2024-04-22T13:26:24.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.416+0000 7f02f9be2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f02f4112310 con 0x7f02f410c500 2024-04-22T13:26:24.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.416+0000 7f02f37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02f41031c0 0x7f02f41986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.416+0000 7f02f37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02f41031c0 0x7f02f41986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43368/0 (socket says 172.21.15.12:43368) 2024-04-22T13:26:24.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.416+0000 7f02f2ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02f4103ae0 0x7f02f4198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.416+0000 7f02f37fe640 1 -- 172.21.15.12:0/2316869933 learned_addr learned my addr 172.21.15.12:0/2316869933 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:24.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.416+0000 7f02f3fff640 1 --2- 172.21.15.12:0/2316869933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02f410c500 0x7f02f419cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.416+0000 7f02f37fe640 1 -- 172.21.15.12:0/2316869933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02f410c500 msgr2=0x7f02f419cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:24.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.416+0000 7f02f37fe640 1 --2- 172.21.15.12:0/2316869933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02f410c500 0x7f02f419cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.416+0000 7f02f37fe640 1 -- 172.21.15.12:0/2316869933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02f4103ae0 msgr2=0x7f02f4198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:24.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.416+0000 7f02f37fe640 1 --2- 172.21.15.12:0/2316869933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02f4103ae0 0x7f02f4198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.416+0000 7f02f37fe640 1 -- 172.21.15.12:0/2316869933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f419d6e0 con 0x7f02f41031c0 2024-04-22T13:26:24.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.417+0000 7f02f2ffd640 1 --2- 172.21.15.12:0/2316869933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02f4103ae0 0x7f02f4198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:24.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.417+0000 7f02f37fe640 1 --2- 172.21.15.12:0/2316869933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02f41031c0 0x7f02f41986f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f02e400e9f0 tx=0x7f02e400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:24.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.417+0000 7f02f0ff9640 1 -- 172.21.15.12:0/2316869933 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e400cd80 con 0x7f02f41031c0 2024-04-22T13:26:24.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.417+0000 7f02f0ff9640 1 -- 172.21.15.12:0/2316869933 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02e40054d0 con 0x7f02f41031c0 2024-04-22T13:26:24.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.417+0000 7f02f9be2640 1 -- 172.21.15.12:0/2316869933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02f419d9d0 con 0x7f02f41031c0 2024-04-22T13:26:24.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.417+0000 7f02f0ff9640 1 -- 172.21.15.12:0/2316869933 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e4010690 con 0x7f02f41031c0 2024-04-22T13:26:24.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.417+0000 7f02f9be2640 1 -- 172.21.15.12:0/2316869933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02f41a5260 con 0x7f02f41031c0 2024-04-22T13:26:24.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.420+0000 7f02f0ff9640 1 -- 172.21.15.12:0/2316869933 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f02e4004e90 con 0x7f02f41031c0 2024-04-22T13:26:24.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.420+0000 7f02fc7cc740 1 Processor -- start 2024-04-22T13:26:24.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.420+0000 7f02fc7cc740 1 -- start start 2024-04-22T13:26:24.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.420+0000 7f02fc7cc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56082b867c20 0x56082b868000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.420+0000 7f02fc7cc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56082b8685d0 0x56082b854760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.420+0000 7f02fc7cc740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56082b854d80 0x56082b8551e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.420+0000 7f02fc7cc740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56082b7bf1e0 con 0x56082b854d80 2024-04-22T13:26:24.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.420+0000 7f02fc7cc740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56082b75c390 con 0x56082b867c20 2024-04-22T13:26:24.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.420+0000 7f02fc7cc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56082b7bb1a0 con 0x56082b8685d0 2024-04-22T13:26:24.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.421+0000 7f02f3fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56082b867c20 0x56082b868000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.421+0000 7f02f3fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56082b867c20 0x56082b868000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:44280/0 (socket says 172.21.15.12:44280) 2024-04-22T13:26:24.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.421+0000 7f02f3fff640 1 -- 172.21.15.12:0/3354763146 learned_addr learned my addr 172.21.15.12:0/3354763146 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:24.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.421+0000 7f02f0ff9640 1 --2- 172.21.15.12:0/2316869933 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f02dc0a3d20 0x7f02dc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.421+0000 7f02f2ffd640 1 --2- 172.21.15.12:0/3354763146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56082b854d80 0x56082b8551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.421+0000 7f02f37fe640 1 --2- 172.21.15.12:0/3354763146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56082b8685d0 0x56082b854760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.421+0000 7f02f3fff640 1 --2- 172.21.15.12:0/2316869933 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f02dc0a3d20 0x7f02dc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.421+0000 7f02f0ff9640 1 -- 172.21.15.12:0/2316869933 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f02e40d0ba0 con 0x7f02f41031c0 2024-04-22T13:26:24.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.421+0000 7f02f3fff640 1 -- 172.21.15.12:0/3354763146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56082b8685d0 msgr2=0x56082b854760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:24.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.421+0000 7f02f3fff640 1 --2- 172.21.15.12:0/3354763146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56082b8685d0 0x56082b854760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.421+0000 7f02f3fff640 1 -- 172.21.15.12:0/3354763146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56082b854d80 msgr2=0x56082b8551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:24.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.421+0000 7f02f3fff640 1 --2- 172.21.15.12:0/3354763146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56082b854d80 0x56082b8551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.421+0000 7f02f3fff640 1 -- 172.21.15.12:0/3354763146 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56082b59b7d0 con 0x56082b867c20 2024-04-22T13:26:24.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.421+0000 7f02f37fe640 1 --2- 172.21.15.12:0/3354763146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56082b8685d0 0x56082b854760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:26:24.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.421+0000 7f02f3fff640 1 --2- 172.21.15.12:0/3354763146 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56082b867c20 0x56082b868000 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f02ec005c40 tx=0x7f02ec013f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc52a8b9bc3ec221 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:24.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.421+0000 7f02f3fff640 1 --2- 172.21.15.12:0/2316869933 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f02dc0a3d20 0x7f02dc0a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f02ec023000 tx=0x7f02ec020030 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:24.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.421+0000 7f02d9ffb640 1 -- 172.21.15.12:0/3354763146 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ec004b20 con 0x56082b867c20 2024-04-22T13:26:24.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.421+0000 7f02d9ffb640 1 -- 172.21.15.12:0/3354763146 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02ec0134e0 con 0x56082b867c20 2024-04-22T13:26:24.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.422+0000 7f02d9ffb640 1 -- 172.21.15.12:0/3354763146 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ec00e5f0 con 0x56082b867c20 2024-04-22T13:26:24.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.422+0000 7f02fc7cc740 1 -- 172.21.15.12:0/3354763146 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56082b867c20 msgr2=0x56082b868000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:24.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.422+0000 7f02fc7cc740 1 --2- 172.21.15.12:0/3354763146 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56082b867c20 0x56082b868000 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f02ec005c40 tx=0x7f02ec013f50 comp rx=0 tx=0).stop 2024-04-22T13:26:24.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.422+0000 7f02fc7cc740 1 -- 172.21.15.12:0/3354763146 shutdown_connections 2024-04-22T13:26:24.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.422+0000 7f02fc7cc740 1 --2- 172.21.15.12:0/3354763146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56082b854d80 0x56082b8551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.422+0000 7f02fc7cc740 1 --2- 172.21.15.12:0/3354763146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56082b8685d0 0x56082b854760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.422+0000 7f02fc7cc740 1 --2- 172.21.15.12:0/3354763146 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56082b867c20 0x56082b868000 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.422+0000 7f02fc7cc740 1 -- 172.21.15.12:0/3354763146 >> 172.21.15.12:0/3354763146 conn(0x56082b857220 msgr2=0x56082b857600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:24.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.422+0000 7f02fc7cc740 1 -- 172.21.15.12:0/3354763146 shutdown_connections 2024-04-22T13:26:24.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.422+0000 7f02fc7cc740 1 -- 172.21.15.12:0/3354763146 wait complete. 2024-04-22T13:26:24.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.423+0000 7f02fc7cc740 1 -- start start 2024-04-22T13:26:24.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.423+0000 7f02fc7cc740 1 Processor -- start 2024-04-22T13:26:24.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.423+0000 7f02fc7cc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56082b854d80 0x56082b86bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.423+0000 7f02fc7cc740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56082b867c20 0x56082b86c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.423+0000 7f02fc7cc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56082b8685d0 0x56082b86c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.423+0000 7f02fc7cc740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56082b86f060 con 0x56082b867c20 2024-04-22T13:26:24.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.423+0000 7f02fc7cc740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56082b7782f0 con 0x56082b8685d0 2024-04-22T13:26:24.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.423+0000 7f02fc7cc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56082b86f1e0 con 0x56082b854d80 2024-04-22T13:26:24.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.423+0000 7f02fc7cc740 10 client.0 _ll_register_callbacks cb 0x56082b785580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:24.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.423+0000 7f02f2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56082b867c20 0x56082b86c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.423+0000 7f02f2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56082b867c20 0x56082b86c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43380/0 (socket says 172.21.15.12:43380) 2024-04-22T13:26:24.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.423+0000 7f02f2ffd640 1 -- 172.21.15.12:0/3815027724 learned_addr learned my addr 172.21.15.12:0/3815027724 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:24.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.423+0000 7f02f3fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56082b8685d0 0x56082b86c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.423+0000 7f02f37fe640 1 --2- 172.21.15.12:0/3815027724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56082b854d80 0x56082b86bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.424+0000 7f02f2ffd640 1 -- 172.21.15.12:0/3815027724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56082b854d80 msgr2=0x56082b86bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:24.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.424+0000 7f02f2ffd640 1 --2- 172.21.15.12:0/3815027724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56082b854d80 0x56082b86bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.424+0000 7f02f2ffd640 1 -- 172.21.15.12:0/3815027724 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56082b8685d0 msgr2=0x56082b86c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:24.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.424+0000 7f02f2ffd640 1 --2- 172.21.15.12:0/3815027724 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56082b8685d0 0x56082b86c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.424+0000 7f02f2ffd640 1 -- 172.21.15.12:0/3815027724 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56082b4d13f0 con 0x56082b867c20 2024-04-22T13:26:24.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.424+0000 7f02f2ffd640 1 --2- 172.21.15.12:0/3815027724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56082b867c20 0x56082b86c0e0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f02e8035000 tx=0x7f02e802cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d8252897824377 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:24.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.424+0000 7f02d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:26:24.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.424+0000 7f02d2ffd640 1 -- 172.21.15.12:0/3815027724 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e8044070 con 0x56082b867c20 2024-04-22T13:26:24.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.424+0000 7f02d2ffd640 1 -- 172.21.15.12:0/3815027724 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f02e80035f0 con 0x56082b867c20 2024-04-22T13:26:24.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.424+0000 7f02fc7cc740 1 -- 172.21.15.12:0/3815027724 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56082b7ccc00 con 0x56082b867c20 2024-04-22T13:26:24.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.424+0000 7f02d2ffd640 1 -- 172.21.15.12:0/3815027724 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f02e802c720 con 0x56082b867c20 2024-04-22T13:26:24.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.424+0000 7f02fc7cc740 10 client.16659 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:24.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.424+0000 7f02fc7cc740 1 -- 172.21.15.12:0/3815027724 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56082b4910f0 con 0x56082b867c20 2024-04-22T13:26:24.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.424+0000 7f02d2ffd640 1 -- 172.21.15.12:0/3815027724 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56082b7ccc00 con 0x56082b867c20 2024-04-22T13:26:24.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.425+0000 7f02d2ffd640 1 -- 172.21.15.12:0/3815027724 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f02e80349a0 con 0x56082b867c20 2024-04-22T13:26:24.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.425+0000 7f02fc7cc740 10 client.16659 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:24.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.425+0000 7f02fc7cc740 10 client.16659 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:26:24.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.425+0000 7f02fc7cc740 20 client.16659 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:24.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.425+0000 7f02fc7cc740 1 --2- 172.21.15.12:0/3815027724 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56082b87c220 0x56082b87e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.425+0000 7f02fc7cc740 4 client.16659 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:26:24.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.425+0000 7f02fc7cc740 1 -- 172.21.15.12:0/3815027724 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56082b7c6a70 con 0x56082b87c220 2024-04-22T13:26:24.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.425+0000 7f02f37fe640 1 --2- 172.21.15.12:0/3815027724 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56082b87c220 0x56082b87e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.425+0000 7f02d2ffd640 10 client.16659 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:24.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.425+0000 7f02f37fe640 1 --2- 172.21.15.12:0/3815027724 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56082b87c220 0x56082b87e6a0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f96c9feaaf475 server_cookie=37c774a7305a2b6f in_seq=0 out_seq=0 2024-04-22T13:26:24.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.426+0000 7f02d2ffd640 1 -- 172.21.15.12:0/3815027724 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56082b7c6a70 con 0x56082b87c220 2024-04-22T13:26:24.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.426+0000 7f02d2ffd640 10 client.16659 handle_command_reply: tid=0 2024-04-22T13:26:24.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.426+0000 7f02fc7cc740 1 client.16659 shutdown 2024-04-22T13:26:24.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.426+0000 7f02fc7cc740 1 -- 172.21.15.12:0/3815027724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56082b867c20 msgr2=0x56082b86c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:24.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.426+0000 7f02fc7cc740 1 --2- 172.21.15.12:0/3815027724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56082b867c20 0x56082b86c0e0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f02e8035000 tx=0x7f02e802cb40 comp rx=0 tx=0).stop 2024-04-22T13:26:24.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.426+0000 7f02fc7cc740 1 -- 172.21.15.12:0/3815027724 shutdown_connections 2024-04-22T13:26:24.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.426+0000 7f02fc7cc740 1 --2- 172.21.15.12:0/3815027724 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56082b87c220 0x56082b87e6a0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.426+0000 7f02fc7cc740 1 --2- 172.21.15.12:0/3815027724 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56082b8685d0 0x56082b86c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.426+0000 7f02fc7cc740 1 --2- 172.21.15.12:0/3815027724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56082b867c20 0x56082b86c0e0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.426+0000 7f02d2ffd640 0 client.16659 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:24.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.426+0000 7f02fc7cc740 1 --2- 172.21.15.12:0/3815027724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56082b854d80 0x56082b86bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.426+0000 7f02fc7cc740 1 -- 172.21.15.12:0/3815027724 >> 172.21.15.12:0/3815027724 conn(0x56082b857220 msgr2=0x56082b856980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:24.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.426+0000 7f02fc7cc740 1 -- 172.21.15.12:0/3815027724 shutdown_connections 2024-04-22T13:26:24.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.426+0000 7f02fc7cc740 1 -- 172.21.15.12:0/3815027724 wait complete. 2024-04-22T13:26:24.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.427+0000 7f02fc7cc740 20 client.16659 trim_cache size 0 max 16384 2024-04-22T13:26:24.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.446+0000 7f02fc7cc740 1 Processor -- start 2024-04-22T13:26:24.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.446+0000 7f02fc7cc740 1 -- start start 2024-04-22T13:26:24.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.446+0000 7f02fc7cc740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56082b854d80 0x56082b86e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.446+0000 7f02fc7cc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56082b867c20 0x56082b877f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.446+0000 7f02fc7cc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56082b8685d0 0x56082b853bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.446+0000 7f02fc7cc740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f02f41ab850 con 0x56082b854d80 2024-04-22T13:26:24.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.446+0000 7f02fc7cc740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f02f41ad390 con 0x56082b867c20 2024-04-22T13:26:24.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.446+0000 7f02fc7cc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f02f41abf20 con 0x56082b8685d0 2024-04-22T13:26:24.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.447+0000 7f02f37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56082b854d80 0x56082b86e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.447+0000 7f02f2ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56082b867c20 0x56082b877f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.447+0000 7f02f37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56082b854d80 0x56082b86e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43394/0 (socket says 172.21.15.12:43394) 2024-04-22T13:26:24.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.447+0000 7f02f37fe640 1 -- 172.21.15.12:0/2190328264 learned_addr learned my addr 172.21.15.12:0/2190328264 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:24.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.447+0000 7f02f3fff640 1 --2- 172.21.15.12:0/2190328264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56082b8685d0 0x56082b853bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.447+0000 7f02f37fe640 1 -- 172.21.15.12:0/2190328264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56082b8685d0 msgr2=0x56082b853bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:24.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.447+0000 7f02f37fe640 1 --2- 172.21.15.12:0/2190328264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56082b8685d0 0x56082b853bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.447+0000 7f02f37fe640 1 -- 172.21.15.12:0/2190328264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56082b867c20 msgr2=0x56082b877f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:24.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.447+0000 7f02f37fe640 1 --2- 172.21.15.12:0/2190328264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56082b867c20 0x56082b877f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.447+0000 7f02f37fe640 1 -- 172.21.15.12:0/2190328264 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56082b86ab00 con 0x56082b854d80 2024-04-22T13:26:24.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.447+0000 7f02f2ffd640 1 --2- 172.21.15.12:0/2190328264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56082b867c20 0x56082b877f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:24.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.447+0000 7f02f37fe640 1 --2- 172.21.15.12:0/2190328264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56082b854d80 0x56082b86e2d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f02e409cb20 tx=0x7f02e4007840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3553b9a742715e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:24.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.447+0000 7f02d97fa640 1 -- 172.21.15.12:0/2190328264 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e40b6070 con 0x56082b854d80 2024-04-22T13:26:24.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.447+0000 7f02d97fa640 1 -- 172.21.15.12:0/2190328264 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02e4005680 con 0x56082b854d80 2024-04-22T13:26:24.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.447+0000 7f02d97fa640 1 -- 172.21.15.12:0/2190328264 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e40a5800 con 0x56082b854d80 2024-04-22T13:26:24.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.448+0000 7f02fc7cc740 1 -- 172.21.15.12:0/2190328264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56082b854d80 msgr2=0x56082b86e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:24.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.448+0000 7f02fc7cc740 1 --2- 172.21.15.12:0/2190328264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56082b854d80 0x56082b86e2d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f02e409cb20 tx=0x7f02e4007840 comp rx=0 tx=0).stop 2024-04-22T13:26:24.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.448+0000 7f02fc7cc740 1 -- 172.21.15.12:0/2190328264 shutdown_connections 2024-04-22T13:26:24.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.448+0000 7f02fc7cc740 1 --2- 172.21.15.12:0/2190328264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56082b8685d0 0x56082b853bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.448+0000 7f02fc7cc740 1 --2- 172.21.15.12:0/2190328264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56082b867c20 0x56082b877f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.448+0000 7f02fc7cc740 1 --2- 172.21.15.12:0/2190328264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56082b854d80 0x56082b86e2d0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.448+0000 7f02fc7cc740 1 -- 172.21.15.12:0/2190328264 >> 172.21.15.12:0/2190328264 conn(0x56082b857220 msgr2=0x56082b8534e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:24.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.448+0000 7f02fc7cc740 1 -- 172.21.15.12:0/2190328264 shutdown_connections 2024-04-22T13:26:24.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.448+0000 7f02fc7cc740 1 -- 172.21.15.12:0/2190328264 wait complete. 2024-04-22T13:26:24.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.449+0000 7f02fc7cc740 1 -- start start 2024-04-22T13:26:24.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.449+0000 7f02fc7cc740 1 Processor -- start 2024-04-22T13:26:24.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.449+0000 7f02fc7cc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56082b854d80 0x56082b874690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.449+0000 7f02fc7cc740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56082b867c20 0x56082b871470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.449+0000 7f02fc7cc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56082b8685d0 0x56082b8719b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.449+0000 7f02fc7cc740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f02f41accc0 con 0x56082b867c20 2024-04-22T13:26:24.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.449+0000 7f02fc7cc740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f02f41ac5f0 con 0x56082b854d80 2024-04-22T13:26:24.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.449+0000 7f02fc7cc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f02f41a9830 con 0x56082b8685d0 2024-04-22T13:26:24.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.449+0000 7f02fc7cc740 10 client.0 _ll_register_callbacks cb 0x7f02e8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:24.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.449+0000 7f02f37fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56082b854d80 0x56082b874690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.449+0000 7f02f37fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56082b854d80 0x56082b874690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:44298/0 (socket says 172.21.15.12:44298) 2024-04-22T13:26:24.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.449+0000 7f02f2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56082b867c20 0x56082b871470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.449+0000 7f02f37fe640 1 -- 172.21.15.12:0/164399713 learned_addr learned my addr 172.21.15.12:0/164399713 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:24.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.449+0000 7f02f3fff640 1 --2- 172.21.15.12:0/164399713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56082b8685d0 0x56082b8719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.450+0000 7f02f2ffd640 1 -- 172.21.15.12:0/164399713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56082b8685d0 msgr2=0x56082b8719b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:24.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.450+0000 7f02f2ffd640 1 --2- 172.21.15.12:0/164399713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56082b8685d0 0x56082b8719b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.450+0000 7f02f2ffd640 1 -- 172.21.15.12:0/164399713 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56082b854d80 msgr2=0x56082b874690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:24.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.450+0000 7f02f2ffd640 1 --2- 172.21.15.12:0/164399713 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56082b854d80 0x56082b874690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.450+0000 7f02f2ffd640 1 -- 172.21.15.12:0/164399713 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56082b871ef0 con 0x56082b867c20 2024-04-22T13:26:24.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.450+0000 7f02f37fe640 1 --2- 172.21.15.12:0/164399713 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56082b854d80 0x56082b874690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:24.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.450+0000 7f02f2ffd640 1 --2- 172.21.15.12:0/164399713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56082b867c20 0x56082b871470 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f02e8036040 tx=0x7f02e8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b3ac608d27eb87 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:24.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.450+0000 7f02d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:26:24.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.450+0000 7f02d2ffd640 1 -- 172.21.15.12:0/164399713 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e8044040 con 0x56082b867c20 2024-04-22T13:26:24.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.450+0000 7f02d2ffd640 1 -- 172.21.15.12:0/164399713 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f02e8034440 con 0x56082b867c20 2024-04-22T13:26:24.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.450+0000 7f02fc7cc740 1 -- 172.21.15.12:0/164399713 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02e4005680 con 0x56082b867c20 2024-04-22T13:26:24.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.450+0000 7f02d2ffd640 1 -- 172.21.15.12:0/164399713 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f02e8034640 con 0x56082b867c20 2024-04-22T13:26:24.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.450+0000 7f02fc7cc740 10 client.16665 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:24.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.450+0000 7f02fc7cc740 1 -- 172.21.15.12:0/164399713 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56082b8e35c0 con 0x56082b867c20 2024-04-22T13:26:24.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.450+0000 7f02d2ffd640 1 -- 172.21.15.12:0/164399713 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02e4005680 con 0x56082b867c20 2024-04-22T13:26:24.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.451+0000 7f02d2ffd640 1 -- 172.21.15.12:0/164399713 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f02e8051640 con 0x56082b867c20 2024-04-22T13:26:24.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.451+0000 7f02fc7cc740 10 client.16665 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:24.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.451+0000 7f02fc7cc740 10 client.16665 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:26:24.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.451+0000 7f02fc7cc740 20 client.16665 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:24.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.451+0000 7f02fc7cc740 1 --2- 172.21.15.12:0/164399713 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56082b8e3d20 0x56082b8e6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:24.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.451+0000 7f02fc7cc740 4 client.16665 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:26:24.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.451+0000 7f02fc7cc740 1 -- 172.21.15.12:0/164399713 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56082b742590 con 0x56082b8e3d20 2024-04-22T13:26:24.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.451+0000 7f02f37fe640 1 --2- 172.21.15.12:0/164399713 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56082b8e3d20 0x56082b8e6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:24.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.451+0000 7f02d2ffd640 10 client.16665 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:24.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.451+0000 7f02f37fe640 1 --2- 172.21.15.12:0/164399713 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56082b8e3d20 0x56082b8e6140 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca6481ecef8d359a server_cookie=90c3acddfa0b47b in_seq=0 out_seq=0 2024-04-22T13:26:24.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.451+0000 7f02d2ffd640 1 -- 172.21.15.12:0/164399713 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56082b742590 con 0x56082b8e3d20 2024-04-22T13:26:24.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.451+0000 7f02d2ffd640 10 client.16665 handle_command_reply: tid=0 2024-04-22T13:26:24.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.451+0000 7f02fc7cc740 1 client.16665 shutdown 2024-04-22T13:26:24.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.452+0000 7f02fc7cc740 1 -- 172.21.15.12:0/164399713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56082b867c20 msgr2=0x56082b871470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:24.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.452+0000 7f02fc7cc740 1 --2- 172.21.15.12:0/164399713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56082b867c20 0x56082b871470 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f02e8036040 tx=0x7f02e8005fd0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.452+0000 7f02fc7cc740 1 -- 172.21.15.12:0/164399713 shutdown_connections 2024-04-22T13:26:24.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.452+0000 7f02fc7cc740 1 --2- 172.21.15.12:0/164399713 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56082b8e3d20 0x56082b8e6140 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.452+0000 7f02fc7cc740 1 --2- 172.21.15.12:0/164399713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56082b8685d0 0x56082b8719b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.452+0000 7f02fc7cc740 1 --2- 172.21.15.12:0/164399713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56082b867c20 0x56082b871470 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.452+0000 7f02fc7cc740 1 --2- 172.21.15.12:0/164399713 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56082b854d80 0x56082b874690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.452+0000 7f02fc7cc740 1 -- 172.21.15.12:0/164399713 >> 172.21.15.12:0/164399713 conn(0x56082b857220 msgr2=0x56082b853d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:24.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.452+0000 7f02d2ffd640 0 client.16665 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:24.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.452+0000 7f02fc7cc740 1 -- 172.21.15.12:0/164399713 shutdown_connections 2024-04-22T13:26:24.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.452+0000 7f02fc7cc740 1 -- 172.21.15.12:0/164399713 wait complete. 2024-04-22T13:26:24.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.452+0000 7f02fc7cc740 20 client.16665 trim_cache size 0 max 16384 2024-04-22T13:26:24.455 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:26:24.455 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:26:24.455 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-04-22T13:26:24.455 INFO:teuthology.orchestra.run.smithi012.stdout: "fe4911cd-e9a2-4304-901e-631f3a2f961f": { 2024-04-22T13:26:24.455 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-04-22T13:26:24.455 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "fe4911cd-e9a2-4304-901e-631f3a2f961f", 2024-04-22T13:26:24.455 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-04-22T13:26:24.456 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:26:24.456 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:26:24.456 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:26:24.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.452+0000 7f02f9be2640 1 -- 172.21.15.12:0/2316869933 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f02dc0a3d20 msgr2=0x7f02dc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:24.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.452+0000 7f02f9be2640 1 --2- 172.21.15.12:0/2316869933 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f02dc0a3d20 0x7f02dc0a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f02ec023000 tx=0x7f02ec020030 comp rx=0 tx=0).stop 2024-04-22T13:26:24.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.452+0000 7f02f9be2640 1 -- 172.21.15.12:0/2316869933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02f41031c0 msgr2=0x7f02f41986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:24.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.452+0000 7f02f9be2640 1 --2- 172.21.15.12:0/2316869933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02f41031c0 0x7f02f41986f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f02e400e9f0 tx=0x7f02e400edf0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.453+0000 7f02f9be2640 1 -- 172.21.15.12:0/2316869933 shutdown_connections 2024-04-22T13:26:24.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.453+0000 7f02f9be2640 1 --2- 172.21.15.12:0/2316869933 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f02dc0a3d20 0x7f02dc0a61e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.453+0000 7f02f9be2640 1 --2- 172.21.15.12:0/2316869933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02f410c500 0x7f02f419cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.453+0000 7f02f9be2640 1 --2- 172.21.15.12:0/2316869933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02f4103ae0 0x7f02f4198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.453+0000 7f02f9be2640 1 --2- 172.21.15.12:0/2316869933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f02f41031c0 0x7f02f41986f0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:24.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.453+0000 7f02f9be2640 1 -- 172.21.15.12:0/2316869933 >> 172.21.15.12:0/2316869933 conn(0x7f02f4078000 msgr2=0x7f02f40fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:24.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.453+0000 7f02f9be2640 1 -- 172.21.15.12:0/2316869933 shutdown_connections 2024-04-22T13:26:24.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:24.453+0000 7f02f9be2640 1 -- 172.21.15.12:0/2316869933 wait complete. 2024-04-22T13:26:24.462 INFO:tasks.cephfs.filesystem:scrub status for tag:fe4911cd-e9a2-4304-901e-631f3a2f961f - {'path': '/', 'tag': 'fe4911cd-e9a2-4304-901e-631f3a2f961f', 'options': 'recursive,force'} 2024-04-22T13:26:25.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:24 smithi012 ceph-mon[28265]: scrub queued for path: / 2024-04-22T13:26:25.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:24 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:26:25.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:24 smithi012 ceph-mon[28265]: scrub summary: active paths [/] 2024-04-22T13:26:25.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:24 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:26:25.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:24 smithi012 ceph-mon[28265]: scrub summary: idle 2024-04-22T13:26:25.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:24 smithi134 ceph-mon[31338]: scrub queued for path: / 2024-04-22T13:26:25.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:24 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:26:25.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:24 smithi134 ceph-mon[31338]: scrub summary: active paths [/] 2024-04-22T13:26:25.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:24 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:26:25.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:24 smithi134 ceph-mon[31338]: scrub summary: idle 2024-04-22T13:26:25.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:24 smithi111 ceph-mon[31361]: scrub queued for path: / 2024-04-22T13:26:25.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:24 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:26:25.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:24 smithi111 ceph-mon[31361]: scrub summary: active paths [/] 2024-04-22T13:26:25.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:24 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:26:25.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:24 smithi111 ceph-mon[31361]: scrub summary: idle 2024-04-22T13:26:26.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:25 smithi012 ceph-mon[28265]: pgmap v72: 97 pgs: 97 active+clean; 259 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.7 MiB/s wr, 752 op/s 2024-04-22T13:26:26.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:25 smithi134 ceph-mon[31338]: pgmap v72: 97 pgs: 97 active+clean; 259 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.7 MiB/s wr, 752 op/s 2024-04-22T13:26:26.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:25 smithi111 ceph-mon[31361]: pgmap v72: 97 pgs: 97 active+clean; 259 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.7 MiB/s wr, 752 op/s 2024-04-22T13:26:26.411 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.412+0000 7fcf1c3b6640 1 -- 172.21.15.12:0/675205541 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf1406aa30 msgr2=0x7fcf1406ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.412+0000 7fcf1c3b6640 1 --2- 172.21.15.12:0/675205541 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf1406aa30 0x7fcf1406ae10 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fcf10009900 tx=0x7fcf1002bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.412+0000 7fcf1c3b6640 1 -- 172.21.15.12:0/675205541 shutdown_connections 2024-04-22T13:26:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.412+0000 7fcf1c3b6640 1 --2- 172.21.15.12:0/675205541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf141072e0 0x7fcf1410ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.412+0000 7fcf1c3b6640 1 --2- 172.21.15.12:0/675205541 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf1406b3e0 0x7fcf14106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.412+0000 7fcf1c3b6640 1 --2- 172.21.15.12:0/675205541 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf1406aa30 0x7fcf1406ae10 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.412+0000 7fcf1c3b6640 1 -- 172.21.15.12:0/675205541 >> 172.21.15.12:0/675205541 conn(0x7fcf14077ba0 msgr2=0x7fcf14077fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.412+0000 7fcf1c3b6640 1 -- 172.21.15.12:0/675205541 shutdown_connections 2024-04-22T13:26:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.412+0000 7fcf1c3b6640 1 -- 172.21.15.12:0/675205541 wait complete. 2024-04-22T13:26:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.413+0000 7fcf1c3b6640 1 Processor -- start 2024-04-22T13:26:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.413+0000 7fcf1c3b6640 1 -- start start 2024-04-22T13:26:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.413+0000 7fcf1c3b6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf1406aa30 0x7fcf141986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.413+0000 7fcf1c3b6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf141072e0 0x7fcf14198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.413+0000 7fcf1c3b6640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf1419cfa0 0x7fcf1419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.413+0000 7fcf1c3b6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcf14112220 con 0x7fcf141072e0 2024-04-22T13:26:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.413+0000 7fcf1c3b6640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcf141120a0 con 0x7fcf1419cfa0 2024-04-22T13:26:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.413+0000 7fcf1c3b6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcf141123a0 con 0x7fcf1406aa30 2024-04-22T13:26:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.414+0000 7fcf19951640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf141072e0 0x7fcf14198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.414+0000 7fcf19951640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf141072e0 0x7fcf14198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43420/0 (socket says 172.21.15.12:43420) 2024-04-22T13:26:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.414+0000 7fcf19951640 1 -- 172.21.15.12:0/655450311 learned_addr learned my addr 172.21.15.12:0/655450311 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.414+0000 7fcf1a152640 1 --2- 172.21.15.12:0/655450311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf1406aa30 0x7fcf141986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.414+0000 7fcf1a953640 1 --2- 172.21.15.12:0/655450311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf1419cfa0 0x7fcf1419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.414+0000 7fcf19951640 1 -- 172.21.15.12:0/655450311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf1406aa30 msgr2=0x7fcf141986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.414+0000 7fcf19951640 1 --2- 172.21.15.12:0/655450311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf1406aa30 0x7fcf141986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.414+0000 7fcf19951640 1 -- 172.21.15.12:0/655450311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf1419cfa0 msgr2=0x7fcf1419d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.414+0000 7fcf19951640 1 --2- 172.21.15.12:0/655450311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf1419cfa0 0x7fcf1419d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.414+0000 7fcf19951640 1 -- 172.21.15.12:0/655450311 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1419da90 con 0x7fcf141072e0 2024-04-22T13:26:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.414+0000 7fcf19951640 1 --2- 172.21.15.12:0/655450311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf141072e0 0x7fcf14198c20 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400d950 tx=0x7fcf0400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.414+0000 7fcf0b7fe640 1 -- 172.21.15.12:0/655450311 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf04017070 con 0x7fcf141072e0 2024-04-22T13:26:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.414+0000 7fcf0b7fe640 1 -- 172.21.15.12:0/655450311 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf04009c30 con 0x7fcf141072e0 2024-04-22T13:26:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.414+0000 7fcf0b7fe640 1 -- 172.21.15.12:0/655450311 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf040047a0 con 0x7fcf141072e0 2024-04-22T13:26:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.414+0000 7fcf1c3b6640 1 -- 172.21.15.12:0/655450311 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf1419dd80 con 0x7fcf141072e0 2024-04-22T13:26:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.415+0000 7fcf1c3b6640 1 -- 172.21.15.12:0/655450311 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf141a5520 con 0x7fcf141072e0 2024-04-22T13:26:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.417+0000 7fcf0b7fe640 1 -- 172.21.15.12:0/655450311 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcf04004020 con 0x7fcf141072e0 2024-04-22T13:26:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.418+0000 7fcf1efab740 1 Processor -- start 2024-04-22T13:26:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.418+0000 7fcf0b7fe640 1 --2- 172.21.15.12:0/655450311 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fcf000a3dc0 0x7fcf000a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.418+0000 7fcf0b7fe640 1 -- 172.21.15.12:0/655450311 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fcf040d0420 con 0x7fcf141072e0 2024-04-22T13:26:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.418+0000 7fcf1efab740 1 -- start start 2024-04-22T13:26:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.418+0000 7fcf1a953640 1 --2- 172.21.15.12:0/655450311 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fcf000a3dc0 0x7fcf000a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.418+0000 7fcf1efab740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570385878d0 0x557038586620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.418+0000 7fcf1efab740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557038586b60 0x557038586fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.418+0000 7fcf1efab740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570385944f0 0x5570385968e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.418+0000 7fcf1efab740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5570384eb8d0 con 0x5570385944f0 2024-04-22T13:26:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.418+0000 7fcf1efab740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557038488e80 con 0x5570385878d0 2024-04-22T13:26:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.418+0000 7fcf1efab740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5570384e7890 con 0x557038586b60 2024-04-22T13:26:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.418+0000 7fcf1a152640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570385878d0 0x557038586620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.418+0000 7fcf1a152640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570385878d0 0x557038586620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:44342/0 (socket says 172.21.15.12:44342) 2024-04-22T13:26:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.418+0000 7fcf1a152640 1 -- 172.21.15.12:0/48498232 learned_addr learned my addr 172.21.15.12:0/48498232 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.419+0000 7fcf1a953640 1 --2- 172.21.15.12:0/655450311 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fcf000a3dc0 0x7fcf000a6280 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c005fd0 tx=0x7fcf0c009500 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.419+0000 7fcf1a953640 1 --2- 172.21.15.12:0/48498232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570385944f0 0x5570385968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.419+0000 7fcf19951640 1 --2- 172.21.15.12:0/48498232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557038586b60 0x557038586fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.419+0000 7fcf1a152640 1 -- 172.21.15.12:0/48498232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557038586b60 msgr2=0x557038586fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.419+0000 7fcf1a152640 1 --2- 172.21.15.12:0/48498232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557038586b60 0x557038586fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.419+0000 7fcf1a152640 1 -- 172.21.15.12:0/48498232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570385944f0 msgr2=0x5570385968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.419+0000 7fcf1a152640 1 --2- 172.21.15.12:0/48498232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570385944f0 0x5570385968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.419+0000 7fcf1a152640 1 -- 172.21.15.12:0/48498232 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570384aa320 con 0x5570385878d0 2024-04-22T13:26:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.419+0000 7fcf1a152640 1 --2- 172.21.15.12:0/48498232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570385878d0 0x557038586620 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcf10009900 tx=0x7fcf1002c610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12ece1c43000e3b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.419+0000 7fcf08bfe640 1 -- 172.21.15.12:0/48498232 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf10045020 con 0x5570385878d0 2024-04-22T13:26:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.419+0000 7fcf08bfe640 1 -- 172.21.15.12:0/48498232 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf1002c7e0 con 0x5570385878d0 2024-04-22T13:26:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.419+0000 7fcf08bfe640 1 -- 172.21.15.12:0/48498232 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf100046c0 con 0x5570385878d0 2024-04-22T13:26:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.420+0000 7fcf1efab740 1 -- 172.21.15.12:0/48498232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570385878d0 msgr2=0x557038586620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.420+0000 7fcf1efab740 1 --2- 172.21.15.12:0/48498232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570385878d0 0x557038586620 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcf10009900 tx=0x7fcf1002c610 comp rx=0 tx=0).stop 2024-04-22T13:26:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.420+0000 7fcf1efab740 1 -- 172.21.15.12:0/48498232 shutdown_connections 2024-04-22T13:26:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.420+0000 7fcf1efab740 1 --2- 172.21.15.12:0/48498232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570385944f0 0x5570385968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.420+0000 7fcf1efab740 1 --2- 172.21.15.12:0/48498232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557038586b60 0x557038586fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.420+0000 7fcf1efab740 1 --2- 172.21.15.12:0/48498232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570385878d0 0x557038586620 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.420+0000 7fcf1efab740 1 -- 172.21.15.12:0/48498232 >> 172.21.15.12:0/48498232 conn(0x55703854dc90 msgr2=0x557038581850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.420+0000 7fcf1efab740 1 -- 172.21.15.12:0/48498232 shutdown_connections 2024-04-22T13:26:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.420+0000 7fcf1efab740 1 -- 172.21.15.12:0/48498232 wait complete. 2024-04-22T13:26:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.420+0000 7fcf1efab740 1 -- start start 2024-04-22T13:26:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.421+0000 7fcf1efab740 1 Processor -- start 2024-04-22T13:26:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.421+0000 7fcf1efab740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557038586b60 0x557038593e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.421+0000 7fcf1efab740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570385878d0 0x5570385911f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.421+0000 7fcf1efab740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570385944f0 0x557038591750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.421+0000 7fcf1efab740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5570385971c0 con 0x557038586b60 2024-04-22T13:26:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.421+0000 7fcf1efab740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557038473eb0 con 0x5570385878d0 2024-04-22T13:26:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.421+0000 7fcf1efab740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557038597340 con 0x5570385944f0 2024-04-22T13:26:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.421+0000 7fcf1efab740 10 client.0 _ll_register_callbacks cb 0x5570384a0b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.421+0000 7fcf1a152640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557038586b60 0x557038593e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.421+0000 7fcf19951640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570385878d0 0x5570385911f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.421+0000 7fcf1a152640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557038586b60 0x557038593e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43442/0 (socket says 172.21.15.12:43442) 2024-04-22T13:26:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.421+0000 7fcf1a152640 1 -- 172.21.15.12:0/3613393662 learned_addr learned my addr 172.21.15.12:0/3613393662 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.421+0000 7fcf1a953640 1 --2- 172.21.15.12:0/3613393662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570385944f0 0x557038591750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.421+0000 7fcf19951640 1 -- 172.21.15.12:0/3613393662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570385944f0 msgr2=0x557038591750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.421+0000 7fcf19951640 1 --2- 172.21.15.12:0/3613393662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570385944f0 0x557038591750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.421+0000 7fcf19951640 1 -- 172.21.15.12:0/3613393662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557038586b60 msgr2=0x557038593e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.421+0000 7fcf19951640 1 --2- 172.21.15.12:0/3613393662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557038586b60 0x557038593e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.421+0000 7fcf19951640 1 -- 172.21.15.12:0/3613393662 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570381fe210 con 0x5570385878d0 2024-04-22T13:26:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.421+0000 7fcf1a152640 1 --2- 172.21.15.12:0/3613393662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557038586b60 0x557038593e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.422+0000 7fcf19951640 1 --2- 172.21.15.12:0/3613393662 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570385878d0 0x5570385911f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcf040cdbc0 tx=0x7fcf040987c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a0a338a7f53845e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.422+0000 7fcefdffb640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-22T13:26:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.422+0000 7fcefdffb640 1 -- 172.21.15.12:0/3613393662 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0409c410 con 0x5570385878d0 2024-04-22T13:26:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.422+0000 7fcefdffb640 1 -- 172.21.15.12:0/3613393662 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fcf04098a80 con 0x5570385878d0 2024-04-22T13:26:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.422+0000 7fcf1efab740 1 -- 172.21.15.12:0/3613393662 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf1002c7e0 con 0x5570385878d0 2024-04-22T13:26:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.422+0000 7fcefdffb640 1 -- 172.21.15.12:0/3613393662 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcf040b9020 con 0x5570385878d0 2024-04-22T13:26:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.422+0000 7fcf1efab740 10 client.26093 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.422+0000 7fcf1efab740 1 -- 172.21.15.12:0/3613393662 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5570382ca620 con 0x5570385878d0 2024-04-22T13:26:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.422+0000 7fcefdffb640 1 -- 172.21.15.12:0/3613393662 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf1002c7e0 con 0x5570385878d0 2024-04-22T13:26:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.422+0000 7fcefdffb640 1 -- 172.21.15.12:0/3613393662 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fcf040b5c90 con 0x5570385878d0 2024-04-22T13:26:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.422+0000 7fcf1efab740 10 client.26093 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.422+0000 7fcf1efab740 10 client.26093 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:26:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.422+0000 7fcf1efab740 20 client.26093 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.422+0000 7fcf1efab740 1 --2- 172.21.15.12:0/3613393662 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5570385a4560 0x5570385a6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.422+0000 7fcf1efab740 4 client.26093 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:26:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.422+0000 7fcf1efab740 1 -- 172.21.15.12:0/3613393662 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570384ff880 con 0x5570385a4560 2024-04-22T13:26:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.423+0000 7fcf1a152640 1 --2- 172.21.15.12:0/3613393662 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5570385a4560 0x5570385a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.423+0000 7fcefdffb640 10 client.26093 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.423+0000 7fcf1a152640 1 --2- 172.21.15.12:0/3613393662 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5570385a4560 0x5570385a6970 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e118b560b61a621 server_cookie=a3543ed452835a01 in_seq=0 out_seq=0 2024-04-22T13:26:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.424+0000 7fcefdffb640 1 -- 172.21.15.12:0/3613393662 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5570384ff880 con 0x5570385a4560 2024-04-22T13:26:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.424+0000 7fcefdffb640 10 client.26093 handle_command_reply: tid=0 2024-04-22T13:26:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.424+0000 7fcf1efab740 1 client.26093 shutdown 2024-04-22T13:26:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.424+0000 7fcf1efab740 1 -- 172.21.15.12:0/3613393662 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570385878d0 msgr2=0x5570385911f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.424+0000 7fcf1efab740 1 --2- 172.21.15.12:0/3613393662 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570385878d0 0x5570385911f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcf040cdbc0 tx=0x7fcf040987c0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.424+0000 7fcf1efab740 1 -- 172.21.15.12:0/3613393662 shutdown_connections 2024-04-22T13:26:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.424+0000 7fcf1efab740 1 --2- 172.21.15.12:0/3613393662 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5570385a4560 0x5570385a6970 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.424+0000 7fcf1efab740 1 --2- 172.21.15.12:0/3613393662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570385944f0 0x557038591750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.424+0000 7fcf1efab740 1 --2- 172.21.15.12:0/3613393662 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5570385878d0 0x5570385911f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.424+0000 7fcf1efab740 1 --2- 172.21.15.12:0/3613393662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557038586b60 0x557038593e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.424+0000 7fcefdffb640 0 client.26093 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.424+0000 7fcf1efab740 1 -- 172.21.15.12:0/3613393662 >> 172.21.15.12:0/3613393662 conn(0x55703854dc90 msgr2=0x5570385835a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.424+0000 7fcf1efab740 1 -- 172.21.15.12:0/3613393662 shutdown_connections 2024-04-22T13:26:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.424+0000 7fcf1efab740 1 -- 172.21.15.12:0/3613393662 wait complete. 2024-04-22T13:26:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.424+0000 7fcf1efab740 20 client.26093 trim_cache size 0 max 16384 2024-04-22T13:26:26.440 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.439+0000 7fcf1efab740 1 Processor -- start 2024-04-22T13:26:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.439+0000 7fcf1efab740 1 -- start start 2024-04-22T13:26:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.439+0000 7fcf1efab740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557038586b60 0x5570385a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.439+0000 7fcf1efab740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570385878d0 0x557038593920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.439+0000 7fcf1efab740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570385944f0 0x557038593e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.439+0000 7fcf1efab740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcf141ab780 con 0x5570385944f0 2024-04-22T13:26:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.439+0000 7fcf1efab740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcf141ad2c0 con 0x557038586b60 2024-04-22T13:26:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.439+0000 7fcf1efab740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcf141abe50 con 0x5570385878d0 2024-04-22T13:26:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.440+0000 7fcf1a953640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570385944f0 0x557038593e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.440+0000 7fcf1a953640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570385944f0 0x557038593e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43458/0 (socket says 172.21.15.12:43458) 2024-04-22T13:26:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.440+0000 7fcf1a953640 1 -- 172.21.15.12:0/992364535 learned_addr learned my addr 172.21.15.12:0/992364535 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.440+0000 7fcf1a152640 1 --2- 172.21.15.12:0/992364535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557038586b60 0x5570385a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.440+0000 7fcf19951640 1 --2- 172.21.15.12:0/992364535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570385878d0 0x557038593920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.440+0000 7fcf1a953640 1 -- 172.21.15.12:0/992364535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570385878d0 msgr2=0x557038593920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.440+0000 7fcf1a953640 1 --2- 172.21.15.12:0/992364535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570385878d0 0x557038593920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.441 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.440+0000 7fcf1a953640 1 -- 172.21.15.12:0/992364535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557038586b60 msgr2=0x5570385a00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.440+0000 7fcf1a953640 1 --2- 172.21.15.12:0/992364535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557038586b60 0x5570385a00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.440+0000 7fcf1a953640 1 -- 172.21.15.12:0/992364535 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557038585e30 con 0x5570385944f0 2024-04-22T13:26:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.440+0000 7fcf1a953640 1 --2- 172.21.15.12:0/992364535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570385944f0 0x557038593e60 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c011920 tx=0x7fcf0c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec1df9499744146 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.440+0000 7fcefffff640 1 -- 172.21.15.12:0/992364535 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c003970 con 0x5570385944f0 2024-04-22T13:26:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.440+0000 7fcefffff640 1 -- 172.21.15.12:0/992364535 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf0c01f930 con 0x5570385944f0 2024-04-22T13:26:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.440+0000 7fcefffff640 1 -- 172.21.15.12:0/992364535 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c004670 con 0x5570385944f0 2024-04-22T13:26:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.441+0000 7fcf1efab740 1 -- 172.21.15.12:0/992364535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570385944f0 msgr2=0x557038593e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.441+0000 7fcf1efab740 1 --2- 172.21.15.12:0/992364535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570385944f0 0x557038593e60 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c011920 tx=0x7fcf0c013000 comp rx=0 tx=0).stop 2024-04-22T13:26:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.441+0000 7fcf1efab740 1 -- 172.21.15.12:0/992364535 shutdown_connections 2024-04-22T13:26:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.441+0000 7fcf1efab740 1 --2- 172.21.15.12:0/992364535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570385944f0 0x557038593e60 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.441+0000 7fcf1efab740 1 --2- 172.21.15.12:0/992364535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570385878d0 0x557038593920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.441+0000 7fcf1efab740 1 --2- 172.21.15.12:0/992364535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557038586b60 0x5570385a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.441+0000 7fcf1efab740 1 -- 172.21.15.12:0/992364535 >> 172.21.15.12:0/992364535 conn(0x55703854dc90 msgr2=0x557038581380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.441+0000 7fcf1efab740 1 -- 172.21.15.12:0/992364535 shutdown_connections 2024-04-22T13:26:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.441+0000 7fcf1efab740 1 -- 172.21.15.12:0/992364535 wait complete. 2024-04-22T13:26:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.441+0000 7fcf1efab740 1 -- start start 2024-04-22T13:26:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.442+0000 7fcf1efab740 1 Processor -- start 2024-04-22T13:26:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.442+0000 7fcf1efab740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557038586b60 0x55703859c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.442+0000 7fcf1efab740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570385878d0 0x55703859cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.442+0000 7fcf1efab740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570385944f0 0x55703858d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.442+0000 7fcf1efab740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcf141acbf0 con 0x5570385878d0 2024-04-22T13:26:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.442+0000 7fcf1efab740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcf141ac520 con 0x557038586b60 2024-04-22T13:26:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.442+0000 7fcf1efab740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcf141a97a0 con 0x5570385944f0 2024-04-22T13:26:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.442+0000 7fcf1efab740 10 client.0 _ll_register_callbacks cb 0x7fcf040057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.442+0000 7fcf19951640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570385878d0 0x55703859cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.442+0000 7fcf19951640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570385878d0 0x55703859cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43460/0 (socket says 172.21.15.12:43460) 2024-04-22T13:26:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.442+0000 7fcf19951640 1 -- 172.21.15.12:0/913910464 learned_addr learned my addr 172.21.15.12:0/913910464 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.442+0000 7fcf1a152640 1 --2- 172.21.15.12:0/913910464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557038586b60 0x55703859c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.442+0000 7fcf1a953640 1 --2- 172.21.15.12:0/913910464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570385944f0 0x55703858d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.442+0000 7fcf19951640 1 -- 172.21.15.12:0/913910464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570385944f0 msgr2=0x55703858d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.442+0000 7fcf19951640 1 --2- 172.21.15.12:0/913910464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570385944f0 0x55703858d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.442+0000 7fcf19951640 1 -- 172.21.15.12:0/913910464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557038586b60 msgr2=0x55703859c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.442+0000 7fcf19951640 1 --2- 172.21.15.12:0/913910464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557038586b60 0x55703859c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.442+0000 7fcf19951640 1 -- 172.21.15.12:0/913910464 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55703858e100 con 0x5570385878d0 2024-04-22T13:26:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.442+0000 7fcf1a152640 1 --2- 172.21.15.12:0/913910464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557038586b60 0x55703859c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.442+0000 7fcf19951640 1 --2- 172.21.15.12:0/913910464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570385878d0 0x55703859cb10 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fcf0409c3a0 tx=0x7fcf040bb4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd766eea5f120d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.442+0000 7fcefdffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:26:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.443+0000 7fcefdffb640 1 -- 172.21.15.12:0/913910464 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0409c410 con 0x5570385878d0 2024-04-22T13:26:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.443+0000 7fcefdffb640 1 -- 172.21.15.12:0/913910464 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fcf040b5b90 con 0x5570385878d0 2024-04-22T13:26:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.443+0000 7fcf1efab740 1 -- 172.21.15.12:0/913910464 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf0c01f930 con 0x5570385878d0 2024-04-22T13:26:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.443+0000 7fcefdffb640 1 -- 172.21.15.12:0/913910464 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcf0409bf50 con 0x5570385878d0 2024-04-22T13:26:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.443+0000 7fcf1efab740 10 client.16683 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.443+0000 7fcf1efab740 1 -- 172.21.15.12:0/913910464 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55703860a680 con 0x5570385878d0 2024-04-22T13:26:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.443+0000 7fcefdffb640 1 -- 172.21.15.12:0/913910464 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf0c01f930 con 0x5570385878d0 2024-04-22T13:26:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.443+0000 7fcefdffb640 1 -- 172.21.15.12:0/913910464 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fcf0409b2d0 con 0x5570385878d0 2024-04-22T13:26:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.443+0000 7fcf1efab740 10 client.16683 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.443+0000 7fcf1efab740 10 client.16683 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:26:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.443+0000 7fcf1efab740 20 client.16683 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.443+0000 7fcf1efab740 1 --2- 172.21.15.12:0/913910464 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55703860ada0 0x55703860d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.443+0000 7fcf1efab740 4 client.16683 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-22T13:26:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.443+0000 7fcf1efab740 1 -- 172.21.15.12:0/913910464 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55703848b3d0 con 0x55703860ada0 2024-04-22T13:26:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.444+0000 7fcf1a152640 1 --2- 172.21.15.12:0/913910464 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55703860ada0 0x55703860d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.444+0000 7fcefdffb640 10 client.16683 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.444+0000 7fcf1a152640 1 --2- 172.21.15.12:0/913910464 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55703860ada0 0x55703860d1c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db93870776c15063 server_cookie=e8108374880e5d5a in_seq=0 out_seq=0 2024-04-22T13:26:26.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.454+0000 7fcefdffb640 1 -- 172.21.15.12:0/913910464 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55703848b3d0 con 0x55703860ada0 2024-04-22T13:26:26.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.454+0000 7fcefdffb640 10 client.16683 handle_command_reply: tid=0 2024-04-22T13:26:26.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.454+0000 7fcf1efab740 1 client.16683 shutdown 2024-04-22T13:26:26.452 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.455+0000 7fcf1efab740 1 -- 172.21.15.12:0/913910464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570385878d0 msgr2=0x55703859cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:26.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.455+0000 7fcf1efab740 1 --2- 172.21.15.12:0/913910464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570385878d0 0x55703859cb10 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fcf0409c3a0 tx=0x7fcf040bb4d0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.455+0000 7fcf1efab740 1 -- 172.21.15.12:0/913910464 shutdown_connections 2024-04-22T13:26:26.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.455+0000 7fcf1efab740 1 --2- 172.21.15.12:0/913910464 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55703860ada0 0x55703860d1c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.455+0000 7fcf1efab740 1 --2- 172.21.15.12:0/913910464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5570385944f0 0x55703858d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.455+0000 7fcf1efab740 1 --2- 172.21.15.12:0/913910464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5570385878d0 0x55703859cb10 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.455+0000 7fcefdffb640 0 client.16683 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:26.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.455+0000 7fcf1efab740 1 --2- 172.21.15.12:0/913910464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557038586b60 0x55703859c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.455+0000 7fcf1efab740 1 -- 172.21.15.12:0/913910464 >> 172.21.15.12:0/913910464 conn(0x55703854dc90 msgr2=0x55703858f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:26.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.455+0000 7fcf1efab740 1 -- 172.21.15.12:0/913910464 shutdown_connections 2024-04-22T13:26:26.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.455+0000 7fcf1efab740 1 -- 172.21.15.12:0/913910464 wait complete. 2024-04-22T13:26:26.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.455+0000 7fcf1efab740 20 client.16683 trim_cache size 0 max 16384 2024-04-22T13:26:26.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-04-22T13:26:26.453 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-04-22T13:26:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-04-22T13:26:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-04-22T13:26:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.455+0000 7fcf1c3b6640 1 -- 172.21.15.12:0/655450311 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fcf000a3dc0 msgr2=0x7fcf000a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.455+0000 7fcf1c3b6640 1 --2- 172.21.15.12:0/655450311 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fcf000a3dc0 0x7fcf000a6280 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c005fd0 tx=0x7fcf0c009500 comp rx=0 tx=0).stop 2024-04-22T13:26:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.455+0000 7fcf1c3b6640 1 -- 172.21.15.12:0/655450311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf141072e0 msgr2=0x7fcf14198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.455+0000 7fcf1c3b6640 1 --2- 172.21.15.12:0/655450311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf141072e0 0x7fcf14198c20 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400d950 tx=0x7fcf0400de10 comp rx=0 tx=0).stop 2024-04-22T13:26:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.455+0000 7fcf1c3b6640 1 -- 172.21.15.12:0/655450311 shutdown_connections 2024-04-22T13:26:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.455+0000 7fcf1c3b6640 1 --2- 172.21.15.12:0/655450311 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fcf000a3dc0 0x7fcf000a6280 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.455+0000 7fcf1c3b6640 1 --2- 172.21.15.12:0/655450311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf1419cfa0 0x7fcf1419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.455+0000 7fcf1c3b6640 1 --2- 172.21.15.12:0/655450311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcf141072e0 0x7fcf14198c20 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.455+0000 7fcf1c3b6640 1 --2- 172.21.15.12:0/655450311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf1406aa30 0x7fcf141986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.455+0000 7fcf1c3b6640 1 -- 172.21.15.12:0/655450311 >> 172.21.15.12:0/655450311 conn(0x7fcf14077ba0 msgr2=0x7fcf140ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.456+0000 7fcf1c3b6640 1 -- 172.21.15.12:0/655450311 shutdown_connections 2024-04-22T13:26:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:26.456+0000 7fcf1c3b6640 1 -- 172.21.15.12:0/655450311 wait complete. 2024-04-22T13:26:28.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:27 smithi012 ceph-mon[28265]: pgmap v73: 97 pgs: 97 active+clean; 259 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.7 MiB/s wr, 752 op/s 2024-04-22T13:26:28.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:27 smithi134 ceph-mon[31338]: pgmap v73: 97 pgs: 97 active+clean; 259 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.7 MiB/s wr, 752 op/s 2024-04-22T13:26:28.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:27 smithi111 ceph-mon[31361]: pgmap v73: 97 pgs: 97 active+clean; 259 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.7 MiB/s wr, 752 op/s 2024-04-22T13:26:30.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:29 smithi012 ceph-mon[28265]: pgmap v74: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 MiB/s wr, 3.09k op/s 2024-04-22T13:26:30.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:29 smithi134 ceph-mon[31338]: pgmap v74: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 MiB/s wr, 3.09k op/s 2024-04-22T13:26:30.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:29 smithi111 ceph-mon[31361]: pgmap v74: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 MiB/s wr, 3.09k op/s 2024-04-22T13:26:32.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:31 smithi012 ceph-mon[28265]: pgmap v75: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 MiB/s wr, 2.68k op/s 2024-04-22T13:26:32.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:31 smithi134 ceph-mon[31338]: pgmap v75: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 MiB/s wr, 2.68k op/s 2024-04-22T13:26:32.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:31 smithi111 ceph-mon[31361]: pgmap v75: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 22 MiB/s wr, 2.68k op/s 2024-04-22T13:26:34.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:33 smithi012 ceph-mon[28265]: pgmap v76: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 MiB/s wr, 2.68k op/s 2024-04-22T13:26:34.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:33 smithi134 ceph-mon[31338]: pgmap v76: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 MiB/s wr, 2.68k op/s 2024-04-22T13:26:34.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:33 smithi111 ceph-mon[31361]: pgmap v76: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 MiB/s wr, 2.68k op/s 2024-04-22T13:26:36.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:35 smithi012 ceph-mon[28265]: pgmap v77: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 MiB/s wr, 2.35k op/s 2024-04-22T13:26:36.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:35 smithi134 ceph-mon[31338]: pgmap v77: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 MiB/s wr, 2.35k op/s 2024-04-22T13:26:36.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:35 smithi111 ceph-mon[31361]: pgmap v77: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 MiB/s wr, 2.35k op/s 2024-04-22T13:26:38.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:37 smithi012 ceph-mon[28265]: pgmap v78: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 MiB/s wr, 2.35k op/s 2024-04-22T13:26:38.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:37 smithi134 ceph-mon[31338]: pgmap v78: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 MiB/s wr, 2.35k op/s 2024-04-22T13:26:38.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:37 smithi111 ceph-mon[31361]: pgmap v78: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 MiB/s wr, 2.35k op/s 2024-04-22T13:26:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.550+0000 7f2ea9bef640 1 -- 172.21.15.12:0/2025665455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ea41031e0 msgr2=0x7f2ea41035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.550+0000 7f2ea9bef640 1 --2- 172.21.15.12:0/2025665455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ea41031e0 0x7f2ea41035c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f2e94009900 tx=0x7f2e9402bd60 comp rx=0 tx=0).stop 2024-04-22T13:26:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.550+0000 7f2ea9bef640 1 -- 172.21.15.12:0/2025665455 shutdown_connections 2024-04-22T13:26:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.550+0000 7f2ea9bef640 1 --2- 172.21.15.12:0/2025665455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ea410c520 0x7f2ea410e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.550+0000 7f2ea9bef640 1 --2- 172.21.15.12:0/2025665455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ea4103b00 0x7f2ea410bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.550+0000 7f2ea9bef640 1 --2- 172.21.15.12:0/2025665455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ea41031e0 0x7f2ea41035c0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.550+0000 7f2ea9bef640 1 -- 172.21.15.12:0/2025665455 >> 172.21.15.12:0/2025665455 conn(0x7f2ea4077fd0 msgr2=0x7f2ea40783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.550+0000 7f2ea9bef640 1 -- 172.21.15.12:0/2025665455 shutdown_connections 2024-04-22T13:26:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.550+0000 7f2ea9bef640 1 -- 172.21.15.12:0/2025665455 wait complete. 2024-04-22T13:26:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.550+0000 7f2ea9bef640 1 Processor -- start 2024-04-22T13:26:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.550+0000 7f2ea9bef640 1 -- start start 2024-04-22T13:26:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.551+0000 7f2ea9bef640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ea41031e0 0x7f2ea4100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.551+0000 7f2ea9bef640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ea4103b00 0x7f2ea4101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.551+0000 7f2ea9bef640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ea410c520 0x7f2ea41018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.551+0000 7f2ea9bef640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea4112190 con 0x7f2ea410c520 2024-04-22T13:26:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.551+0000 7f2ea9bef640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea4112010 con 0x7f2ea4103b00 2024-04-22T13:26:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.551+0000 7f2ea9bef640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea4112310 con 0x7f2ea41031e0 2024-04-22T13:26:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.551+0000 7f2ea3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ea410c520 0x7f2ea41018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.551+0000 7f2ea3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ea410c520 0x7f2ea41018b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39308/0 (socket says 172.21.15.12:39308) 2024-04-22T13:26:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.551+0000 7f2ea3fff640 1 -- 172.21.15.12:0/2590185419 learned_addr learned my addr 172.21.15.12:0/2590185419 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.551+0000 7f2ea37fe640 1 --2- 172.21.15.12:0/2590185419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ea41031e0 0x7f2ea4100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.551+0000 7f2ea2ffd640 1 --2- 172.21.15.12:0/2590185419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ea4103b00 0x7f2ea4101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.551+0000 7f2ea3fff640 1 -- 172.21.15.12:0/2590185419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ea41031e0 msgr2=0x7f2ea4100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.551+0000 7f2ea3fff640 1 --2- 172.21.15.12:0/2590185419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ea41031e0 0x7f2ea4100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.551+0000 7f2ea3fff640 1 -- 172.21.15.12:0/2590185419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ea4103b00 msgr2=0x7f2ea4101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.551+0000 7f2ea3fff640 1 --2- 172.21.15.12:0/2590185419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ea4103b00 0x7f2ea4101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.551+0000 7f2ea3fff640 1 -- 172.21.15.12:0/2590185419 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea41a6bb0 con 0x7f2ea410c520 2024-04-22T13:26:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.551+0000 7f2ea2ffd640 1 --2- 172.21.15.12:0/2590185419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ea4103b00 0x7f2ea4101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:26:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.551+0000 7f2ea3fff640 1 --2- 172.21.15.12:0/2590185419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ea410c520 0x7f2ea41018b0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00eaa0 tx=0x7f2e9c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.552+0000 7f2ea0ff9640 1 -- 172.21.15.12:0/2590185419 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c00cdb0 con 0x7f2ea410c520 2024-04-22T13:26:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.552+0000 7f2ea0ff9640 1 -- 172.21.15.12:0/2590185419 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e9c005020 con 0x7f2ea410c520 2024-04-22T13:26:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.552+0000 7f2ea9bef640 1 -- 172.21.15.12:0/2590185419 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ea41a6ea0 con 0x7f2ea410c520 2024-04-22T13:26:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.552+0000 7f2ea0ff9640 1 -- 172.21.15.12:0/2590185419 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c0116d0 con 0x7f2ea410c520 2024-04-22T13:26:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.552+0000 7f2ea9bef640 1 -- 172.21.15.12:0/2590185419 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ea41a73c0 con 0x7f2ea410c520 2024-04-22T13:26:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.554+0000 7f2ea0ff9640 1 -- 172.21.15.12:0/2590185419 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2e9c011870 con 0x7f2ea410c520 2024-04-22T13:26:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.554+0000 7f2eac7d7740 1 Processor -- start 2024-04-22T13:26:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.554+0000 7f2eac7d7740 1 -- start start 2024-04-22T13:26:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.555+0000 7f2eac7d7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e2db10a8d0 0x55e2db109620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.555+0000 7f2eac7d7740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2db109b60 0x55e2db109fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.555+0000 7f2eac7d7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2db1130e0 0x55e2db1154d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.555+0000 7f2eac7d7740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e2db06e8d0 con 0x55e2db109b60 2024-04-22T13:26:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.555+0000 7f2eac7d7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e2db00be80 con 0x55e2db10a8d0 2024-04-22T13:26:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.555+0000 7f2eac7d7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e2db06a890 con 0x55e2db1130e0 2024-04-22T13:26:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.555+0000 7f2ea37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2db109b60 0x55e2db109fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.555+0000 7f2ea0ff9640 1 --2- 172.21.15.12:0/2590185419 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2e8c0a3e40 0x7f2e8c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.555+0000 7f2ea37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2db109b60 0x55e2db109fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39312/0 (socket says 172.21.15.12:39312) 2024-04-22T13:26:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.555+0000 7f2ea2ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2db1130e0 0x55e2db1154d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.555+0000 7f2ea37fe640 1 -- 172.21.15.12:0/3372146682 learned_addr learned my addr 172.21.15.12:0/3372146682 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.555+0000 7f2ea2ffd640 1 --2- 172.21.15.12:0/3372146682 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e2db10a8d0 0x55e2db109620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.555+0000 7f2ea3fff640 1 --2- 172.21.15.12:0/2590185419 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2e8c0a3e40 0x7f2e8c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.555+0000 7f2ea0ff9640 1 -- 172.21.15.12:0/2590185419 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f2e9c0d0870 con 0x7f2ea410c520 2024-04-22T13:26:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.555+0000 7f2ea2ffd640 1 -- 172.21.15.12:0/3372146682 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e2db10a8d0 msgr2=0x55e2db109620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.555+0000 7f2ea2ffd640 1 --2- 172.21.15.12:0/3372146682 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e2db10a8d0 0x55e2db109620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.555+0000 7f2ea2ffd640 1 -- 172.21.15.12:0/3372146682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2db109b60 msgr2=0x55e2db109fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.555+0000 7f2ea2ffd640 1 --2- 172.21.15.12:0/3372146682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2db109b60 0x55e2db109fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.555+0000 7f2ea2ffd640 1 -- 172.21.15.12:0/3372146682 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2db02d320 con 0x55e2db1130e0 2024-04-22T13:26:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.555+0000 7f2ea37fe640 1 --2- 172.21.15.12:0/3372146682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2db109b60 0x55e2db109fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.556+0000 7f2ea2ffd640 1 --2- 172.21.15.12:0/3372146682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2db1130e0 0x55e2db1154d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2ea41028c0 tx=0x7f2e9800c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d63dede39ea59a82 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.556+0000 7f2ea3fff640 1 --2- 172.21.15.12:0/2590185419 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2e8c0a3e40 0x7f2e8c0a6300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c0ce0c0 tx=0x7f2e9c098500 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.556+0000 7f2e89ffb640 1 -- 172.21.15.12:0/3372146682 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e980142d0 con 0x55e2db1130e0 2024-04-22T13:26:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.556+0000 7f2e89ffb640 1 -- 172.21.15.12:0/3372146682 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e9800f760 con 0x55e2db1130e0 2024-04-22T13:26:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.556+0000 7f2e89ffb640 1 -- 172.21.15.12:0/3372146682 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e98003830 con 0x55e2db1130e0 2024-04-22T13:26:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.557+0000 7f2eac7d7740 1 -- 172.21.15.12:0/3372146682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2db1130e0 msgr2=0x55e2db1154d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.557+0000 7f2eac7d7740 1 --2- 172.21.15.12:0/3372146682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2db1130e0 0x55e2db1154d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2ea41028c0 tx=0x7f2e9800c510 comp rx=0 tx=0).stop 2024-04-22T13:26:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.557+0000 7f2eac7d7740 1 -- 172.21.15.12:0/3372146682 shutdown_connections 2024-04-22T13:26:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.557+0000 7f2eac7d7740 1 --2- 172.21.15.12:0/3372146682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2db1130e0 0x55e2db1154d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.557+0000 7f2eac7d7740 1 --2- 172.21.15.12:0/3372146682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2db109b60 0x55e2db109fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.557+0000 7f2eac7d7740 1 --2- 172.21.15.12:0/3372146682 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e2db10a8d0 0x55e2db109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.557+0000 7f2eac7d7740 1 -- 172.21.15.12:0/3372146682 >> 172.21.15.12:0/3372146682 conn(0x55e2db0d0c90 msgr2=0x55e2db104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.557+0000 7f2eac7d7740 1 -- 172.21.15.12:0/3372146682 shutdown_connections 2024-04-22T13:26:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.557+0000 7f2eac7d7740 1 -- 172.21.15.12:0/3372146682 wait complete. 2024-04-22T13:26:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.558+0000 7f2eac7d7740 1 -- start start 2024-04-22T13:26:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.558+0000 7f2eac7d7740 1 Processor -- start 2024-04-22T13:26:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.558+0000 7f2eac7d7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e2db109b60 0x55e2db112ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.558+0000 7f2eac7d7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2db10a8d0 0x55e2db10fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.558+0000 7f2eac7d7740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2db1130e0 0x55e2db110450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.558+0000 7f2eac7d7740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e2db115db0 con 0x55e2db1130e0 2024-04-22T13:26:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.558+0000 7f2eac7d7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e2daff6eb0 con 0x55e2db109b60 2024-04-22T13:26:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.558+0000 7f2eac7d7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e2db115f30 con 0x55e2db10a8d0 2024-04-22T13:26:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.558+0000 7f2eac7d7740 10 client.0 _ll_register_callbacks cb 0x55e2db023b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.559+0000 7f2ea2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2db1130e0 0x55e2db110450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.559+0000 7f2ea2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2db1130e0 0x55e2db110450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39328/0 (socket says 172.21.15.12:39328) 2024-04-22T13:26:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.559+0000 7f2ea2ffd640 1 -- 172.21.15.12:0/2355101675 learned_addr learned my addr 172.21.15.12:0/2355101675 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.559+0000 7f2ea37fe640 1 --2- 172.21.15.12:0/2355101675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2db10a8d0 0x55e2db10fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.559+0000 7f2ea2ffd640 1 --2- 172.21.15.12:0/2355101675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e2db109b60 0x55e2db112ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.559+0000 7f2ea2ffd640 1 -- 172.21.15.12:0/2355101675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2db10a8d0 msgr2=0x55e2db10fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.559+0000 7f2ea2ffd640 1 --2- 172.21.15.12:0/2355101675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2db10a8d0 0x55e2db10fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.559+0000 7f2ea2ffd640 1 -- 172.21.15.12:0/2355101675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e2db109b60 msgr2=0x55e2db112ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.559+0000 7f2ea2ffd640 1 --2- 172.21.15.12:0/2355101675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e2db109b60 0x55e2db112ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.559+0000 7f2ea2ffd640 1 -- 172.21.15.12:0/2355101675 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2dad81210 con 0x55e2db1130e0 2024-04-22T13:26:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.559+0000 7f2ea2ffd640 1 --2- 172.21.15.12:0/2355101675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2db1130e0 0x55e2db110450 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f2e9800cde0 tx=0x7f2e98001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3ab08bf14f6d8c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.559+0000 7f2e82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:26:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.559+0000 7f2e82ffd640 1 -- 172.21.15.12:0/2355101675 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9800fbd0 con 0x55e2db1130e0 2024-04-22T13:26:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.559+0000 7f2e82ffd640 1 -- 172.21.15.12:0/2355101675 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f2e98004030 con 0x55e2db1130e0 2024-04-22T13:26:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.559+0000 7f2eac7d7740 1 -- 172.21.15.12:0/2355101675 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e9800f760 con 0x55e2db1130e0 2024-04-22T13:26:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.559+0000 7f2e82ffd640 1 -- 172.21.15.12:0/2355101675 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2e98004230 con 0x55e2db1130e0 2024-04-22T13:26:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.559+0000 7f2eac7d7740 10 client.16695 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.559+0000 7f2eac7d7740 1 -- 172.21.15.12:0/2355101675 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e2dae4d620 con 0x55e2db1130e0 2024-04-22T13:26:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.560+0000 7f2e82ffd640 1 -- 172.21.15.12:0/2355101675 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e9800f760 con 0x55e2db1130e0 2024-04-22T13:26:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.560+0000 7f2e82ffd640 1 -- 172.21.15.12:0/2355101675 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2e980048e0 con 0x55e2db1130e0 2024-04-22T13:26:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.560+0000 7f2eac7d7740 10 client.16695 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.560+0000 7f2eac7d7740 10 client.16695 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:26:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.560+0000 7f2eac7d7740 20 client.16695 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.560+0000 7f2eac7d7740 1 --2- 172.21.15.12:0/2355101675 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e2db122fc0 0x55e2db1253b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.560+0000 7f2eac7d7740 4 client.16695 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:26:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.560+0000 7f2eac7d7740 1 -- 172.21.15.12:0/2355101675 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2db082880 con 0x55e2db122fc0 2024-04-22T13:26:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.560+0000 7f2ea37fe640 1 --2- 172.21.15.12:0/2355101675 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e2db122fc0 0x55e2db1253b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.561+0000 7f2e82ffd640 10 client.16695 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.561+0000 7f2ea37fe640 1 --2- 172.21.15.12:0/2355101675 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e2db122fc0 0x55e2db1253b0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=608ddb880059b899 server_cookie=b9e4cdfcd18ece01 in_seq=0 out_seq=0 2024-04-22T13:26:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.561+0000 7f2e82ffd640 1 -- 172.21.15.12:0/2355101675 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e2db082880 con 0x55e2db122fc0 2024-04-22T13:26:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.561+0000 7f2e82ffd640 10 client.16695 handle_command_reply: tid=0 2024-04-22T13:26:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.561+0000 7f2eac7d7740 1 client.16695 shutdown 2024-04-22T13:26:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.562+0000 7f2eac7d7740 1 -- 172.21.15.12:0/2355101675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2db1130e0 msgr2=0x55e2db110450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.562+0000 7f2eac7d7740 1 --2- 172.21.15.12:0/2355101675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2db1130e0 0x55e2db110450 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f2e9800cde0 tx=0x7f2e98001e50 comp rx=0 tx=0).stop 2024-04-22T13:26:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.562+0000 7f2eac7d7740 1 -- 172.21.15.12:0/2355101675 shutdown_connections 2024-04-22T13:26:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.562+0000 7f2eac7d7740 1 --2- 172.21.15.12:0/2355101675 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e2db122fc0 0x55e2db1253b0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.562+0000 7f2eac7d7740 1 --2- 172.21.15.12:0/2355101675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2db1130e0 0x55e2db110450 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.562+0000 7f2eac7d7740 1 --2- 172.21.15.12:0/2355101675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2db10a8d0 0x55e2db10fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.562+0000 7f2e82ffd640 0 client.16695 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.562+0000 7f2eac7d7740 1 --2- 172.21.15.12:0/2355101675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e2db109b60 0x55e2db112ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.562+0000 7f2eac7d7740 1 -- 172.21.15.12:0/2355101675 >> 172.21.15.12:0/2355101675 conn(0x55e2db0d0c90 msgr2=0x55e2db1065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.562+0000 7f2eac7d7740 1 -- 172.21.15.12:0/2355101675 shutdown_connections 2024-04-22T13:26:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.562+0000 7f2eac7d7740 1 -- 172.21.15.12:0/2355101675 wait complete. 2024-04-22T13:26:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.562+0000 7f2eac7d7740 20 client.16695 trim_cache size 0 max 16384 2024-04-22T13:26:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.589+0000 7f2eac7d7740 1 Processor -- start 2024-04-22T13:26:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.589+0000 7f2eac7d7740 1 -- start start 2024-04-22T13:26:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.589+0000 7f2eac7d7740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2db109b60 0x55e2db11ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.589+0000 7f2eac7d7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2db10a8d0 0x55e2db112570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.589+0000 7f2eac7d7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e2db1130e0 0x55e2db112ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.589+0000 7f2eac7d7740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea41afc30 con 0x55e2db109b60 2024-04-22T13:26:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.589+0000 7f2eac7d7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea41b1770 con 0x55e2db1130e0 2024-04-22T13:26:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.589+0000 7f2eac7d7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea41b0300 con 0x55e2db10a8d0 2024-04-22T13:26:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.590+0000 7f2ea2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2db109b60 0x55e2db11ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.590+0000 7f2ea2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2db109b60 0x55e2db11ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39338/0 (socket says 172.21.15.12:39338) 2024-04-22T13:26:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.590+0000 7f2ea2ffd640 1 -- 172.21.15.12:0/2326698562 learned_addr learned my addr 172.21.15.12:0/2326698562 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.590+0000 7f2ea37fe640 1 --2- 172.21.15.12:0/2326698562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2db10a8d0 0x55e2db112570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.590+0000 7f2ea2ffd640 1 --2- 172.21.15.12:0/2326698562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e2db1130e0 0x55e2db112ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.590+0000 7f2ea37fe640 1 -- 172.21.15.12:0/2326698562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e2db1130e0 msgr2=0x55e2db112ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:39.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.590+0000 7f2ea37fe640 1 --2- 172.21.15.12:0/2326698562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e2db1130e0 0x55e2db112ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.590+0000 7f2ea37fe640 1 -- 172.21.15.12:0/2326698562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2db109b60 msgr2=0x55e2db11ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:39.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.590+0000 7f2ea37fe640 1 --2- 172.21.15.12:0/2326698562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2db109b60 0x55e2db11ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.590+0000 7f2ea37fe640 1 -- 172.21.15.12:0/2326698562 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2db110a50 con 0x55e2db10a8d0 2024-04-22T13:26:39.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.590+0000 7f2ea2ffd640 1 --2- 172.21.15.12:0/2326698562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2db109b60 0x55e2db11ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:39.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.590+0000 7f2ea37fe640 1 --2- 172.21.15.12:0/2326698562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2db10a8d0 0x55e2db112570 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2e94005c40 tx=0x7f2e94009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc2aa8e278c42b4c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:39.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.591+0000 7f2e897fa640 1 -- 172.21.15.12:0/2326698562 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e94004410 con 0x55e2db10a8d0 2024-04-22T13:26:39.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.591+0000 7f2e897fa640 1 -- 172.21.15.12:0/2326698562 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e9402cdc0 con 0x55e2db10a8d0 2024-04-22T13:26:39.588 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.591+0000 7f2e897fa640 1 -- 172.21.15.12:0/2326698562 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e940046c0 con 0x55e2db10a8d0 2024-04-22T13:26:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.591+0000 7f2eac7d7740 1 -- 172.21.15.12:0/2326698562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2db10a8d0 msgr2=0x55e2db112570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.591+0000 7f2eac7d7740 1 --2- 172.21.15.12:0/2326698562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2db10a8d0 0x55e2db112570 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2e94005c40 tx=0x7f2e94009260 comp rx=0 tx=0).stop 2024-04-22T13:26:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.591+0000 7f2eac7d7740 1 -- 172.21.15.12:0/2326698562 shutdown_connections 2024-04-22T13:26:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.591+0000 7f2eac7d7740 1 --2- 172.21.15.12:0/2326698562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e2db1130e0 0x55e2db112ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.591+0000 7f2eac7d7740 1 --2- 172.21.15.12:0/2326698562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2db10a8d0 0x55e2db112570 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.591+0000 7f2eac7d7740 1 --2- 172.21.15.12:0/2326698562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2db109b60 0x55e2db11ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.591+0000 7f2eac7d7740 1 -- 172.21.15.12:0/2326698562 >> 172.21.15.12:0/2326698562 conn(0x55e2db0d0c90 msgr2=0x55e2db1059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.592+0000 7f2eac7d7740 1 -- 172.21.15.12:0/2326698562 shutdown_connections 2024-04-22T13:26:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.592+0000 7f2eac7d7740 1 -- 172.21.15.12:0/2326698562 wait complete. 2024-04-22T13:26:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.592+0000 7f2eac7d7740 1 -- start start 2024-04-22T13:26:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.592+0000 7f2eac7d7740 1 Processor -- start 2024-04-22T13:26:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.592+0000 7f2eac7d7740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2db109b60 0x55e2db11b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.592+0000 7f2eac7d7740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2db10a8d0 0x55e2db10d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.592+0000 7f2eac7d7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e2db1130e0 0x55e2db10d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.592+0000 7f2eac7d7740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea41b10a0 con 0x55e2db10a8d0 2024-04-22T13:26:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.592+0000 7f2eac7d7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea41b09d0 con 0x55e2db1130e0 2024-04-22T13:26:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.592+0000 7f2eac7d7740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea41adc10 con 0x55e2db109b60 2024-04-22T13:26:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.592+0000 7f2eac7d7740 10 client.0 _ll_register_callbacks cb 0x7f2e94005e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.593+0000 7f2ea37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2db10a8d0 0x55e2db10d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.593+0000 7f2ea37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2db10a8d0 0x55e2db10d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39350/0 (socket says 172.21.15.12:39350) 2024-04-22T13:26:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.593+0000 7f2ea37fe640 1 -- 172.21.15.12:0/2318251237 learned_addr learned my addr 172.21.15.12:0/2318251237 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.593+0000 7f2ea2ffd640 1 --2- 172.21.15.12:0/2318251237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e2db1130e0 0x55e2db10d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.593+0000 7f2ea2ffd640 1 --2- 172.21.15.12:0/2318251237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2db109b60 0x55e2db11b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.593+0000 7f2ea37fe640 1 -- 172.21.15.12:0/2318251237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2db109b60 msgr2=0x55e2db11b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.593+0000 7f2ea37fe640 1 --2- 172.21.15.12:0/2318251237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2db109b60 0x55e2db11b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.593+0000 7f2ea37fe640 1 -- 172.21.15.12:0/2318251237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e2db1130e0 msgr2=0x55e2db10d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.593+0000 7f2ea37fe640 1 --2- 172.21.15.12:0/2318251237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e2db1130e0 0x55e2db10d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.593+0000 7f2ea37fe640 1 -- 172.21.15.12:0/2318251237 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2db108e30 con 0x55e2db10a8d0 2024-04-22T13:26:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.593+0000 7f2ea37fe640 1 --2- 172.21.15.12:0/2318251237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2db10a8d0 0x55e2db10d2a0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f2e94000c00 tx=0x7f2e940056c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3cfd828b5b4f105 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.593+0000 7f2e82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:26:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.593+0000 7f2e82ffd640 1 -- 172.21.15.12:0/2318251237 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e94058060 con 0x55e2db10a8d0 2024-04-22T13:26:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.593+0000 7f2eac7d7740 1 -- 172.21.15.12:0/2318251237 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e9402cdc0 con 0x55e2db10a8d0 2024-04-22T13:26:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.593+0000 7f2e82ffd640 1 -- 172.21.15.12:0/2318251237 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f2e9402cb20 con 0x55e2db10a8d0 2024-04-22T13:26:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.594+0000 7f2e82ffd640 1 -- 172.21.15.12:0/2318251237 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2e940035f0 con 0x55e2db10a8d0 2024-04-22T13:26:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.594+0000 7f2eac7d7740 10 client.16701 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.594+0000 7f2eac7d7740 1 -- 172.21.15.12:0/2318251237 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e2db189400 con 0x55e2db10a8d0 2024-04-22T13:26:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.594+0000 7f2e82ffd640 1 -- 172.21.15.12:0/2318251237 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e9402cdc0 con 0x55e2db10a8d0 2024-04-22T13:26:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.594+0000 7f2e82ffd640 1 -- 172.21.15.12:0/2318251237 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2e94053730 con 0x55e2db10a8d0 2024-04-22T13:26:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.594+0000 7f2eac7d7740 10 client.16701 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.594+0000 7f2eac7d7740 10 client.16701 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-22T13:26:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.594+0000 7f2eac7d7740 20 client.16701 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.594+0000 7f2eac7d7740 1 --2- 172.21.15.12:0/2318251237 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e2db189b20 0x55e2db18bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.594+0000 7f2eac7d7740 4 client.16701 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-22T13:26:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.594+0000 7f2eac7d7740 1 -- 172.21.15.12:0/2318251237 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e2db00e3d0 con 0x55e2db189b20 2024-04-22T13:26:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.594+0000 7f2ea2ffd640 1 --2- 172.21.15.12:0/2318251237 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e2db189b20 0x55e2db18bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.595+0000 7f2ea2ffd640 1 --2- 172.21.15.12:0/2318251237 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e2db189b20 0x55e2db18bf40 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be72631295c62d09 server_cookie=caae1505ab17447a in_seq=0 out_seq=0 2024-04-22T13:26:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.595+0000 7f2e82ffd640 10 client.16701 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:39.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.607+0000 7f2e82ffd640 1 -- 172.21.15.12:0/2318251237 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e2db00e3d0 con 0x55e2db189b20 2024-04-22T13:26:39.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.607+0000 7f2e82ffd640 10 client.16701 handle_command_reply: tid=0 2024-04-22T13:26:39.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.607+0000 7f2eac7d7740 1 client.16701 shutdown 2024-04-22T13:26:39.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.607+0000 7f2eac7d7740 1 -- 172.21.15.12:0/2318251237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2db10a8d0 msgr2=0x55e2db10d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:39.603 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.607+0000 7f2eac7d7740 1 --2- 172.21.15.12:0/2318251237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2db10a8d0 0x55e2db10d2a0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f2e94000c00 tx=0x7f2e940056c0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.607+0000 7f2eac7d7740 1 -- 172.21.15.12:0/2318251237 shutdown_connections 2024-04-22T13:26:39.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.607+0000 7f2eac7d7740 1 --2- 172.21.15.12:0/2318251237 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e2db189b20 0x55e2db18bf40 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.607+0000 7f2eac7d7740 1 --2- 172.21.15.12:0/2318251237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e2db1130e0 0x55e2db10d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.607+0000 7f2eac7d7740 1 --2- 172.21.15.12:0/2318251237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2db10a8d0 0x55e2db10d2a0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.607+0000 7f2e82ffd640 0 client.16701 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:39.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.607+0000 7f2eac7d7740 1 --2- 172.21.15.12:0/2318251237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e2db109b60 0x55e2db11b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.607+0000 7f2eac7d7740 1 -- 172.21.15.12:0/2318251237 >> 172.21.15.12:0/2318251237 conn(0x55e2db0d0c90 msgr2=0x55e2db106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:39.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.607+0000 7f2eac7d7740 1 -- 172.21.15.12:0/2318251237 shutdown_connections 2024-04-22T13:26:39.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.607+0000 7f2eac7d7740 1 -- 172.21.15.12:0/2318251237 wait complete. 2024-04-22T13:26:39.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.607+0000 7f2eac7d7740 20 client.16701 trim_cache size 0 max 16384 2024-04-22T13:26:39.604 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:{ 2024-04-22T13:26:39.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "message": "", 2024-04-22T13:26:39.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout: "return_code": 0 2024-04-22T13:26:39.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stdout:} 2024-04-22T13:26:39.605 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.608+0000 7f2ea9bef640 1 -- 172.21.15.12:0/2590185419 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2e8c0a3e40 msgr2=0x7f2e8c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:39.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.608+0000 7f2ea9bef640 1 --2- 172.21.15.12:0/2590185419 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2e8c0a3e40 0x7f2e8c0a6300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c0ce0c0 tx=0x7f2e9c098500 comp rx=0 tx=0).stop 2024-04-22T13:26:39.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.608+0000 7f2ea9bef640 1 -- 172.21.15.12:0/2590185419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ea410c520 msgr2=0x7f2ea41018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:39.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.608+0000 7f2ea9bef640 1 --2- 172.21.15.12:0/2590185419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ea410c520 0x7f2ea41018b0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00eaa0 tx=0x7f2e9c00ef60 comp rx=0 tx=0).stop 2024-04-22T13:26:39.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.608+0000 7f2ea9bef640 1 -- 172.21.15.12:0/2590185419 shutdown_connections 2024-04-22T13:26:39.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.608+0000 7f2ea9bef640 1 --2- 172.21.15.12:0/2590185419 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2e8c0a3e40 0x7f2e8c0a6300 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.608+0000 7f2ea9bef640 1 --2- 172.21.15.12:0/2590185419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ea410c520 0x7f2ea41018b0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.608+0000 7f2ea9bef640 1 --2- 172.21.15.12:0/2590185419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ea4103b00 0x7f2ea4101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.608+0000 7f2ea9bef640 1 --2- 172.21.15.12:0/2590185419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ea41031e0 0x7f2ea4100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:39.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.608+0000 7f2ea9bef640 1 -- 172.21.15.12:0/2590185419 >> 172.21.15.12:0/2590185419 conn(0x7f2ea4077fd0 msgr2=0x7f2ea40fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:39.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.608+0000 7f2ea9bef640 1 -- 172.21.15.12:0/2590185419 shutdown_connections 2024-04-22T13:26:39.606 INFO:teuthology.task.background_exec.ubuntu@smithi012.front.sepia.ceph.com.smithi012.stderr:2024-04-22T13:26:39.608+0000 7f2ea9bef640 1 -- 172.21.15.12:0/2590185419 wait complete. 2024-04-22T13:26:40.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:39 smithi012 ceph-mon[28265]: pgmap v79: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 MiB/s wr, 2.35k op/s 2024-04-22T13:26:40.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:39 smithi134 ceph-mon[31338]: pgmap v79: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 MiB/s wr, 2.35k op/s 2024-04-22T13:26:40.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:39 smithi111 ceph-mon[31361]: pgmap v79: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 MiB/s wr, 2.35k op/s 2024-04-22T13:26:41.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:40 smithi134 ceph-mon[31338]: pgmap v80: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 866 KiB/s wr, 11 op/s 2024-04-22T13:26:41.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:40 smithi111 ceph-mon[31361]: pgmap v80: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 866 KiB/s wr, 11 op/s 2024-04-22T13:26:41.336 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:40 smithi012 ceph-mon[28265]: pgmap v80: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 866 KiB/s wr, 11 op/s 2024-04-22T13:26:43.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:43 smithi012 ceph-mon[28265]: pgmap v81: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.3 MiB/s wr, 19 op/s 2024-04-22T13:26:43.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:43 smithi134 ceph-mon[31338]: pgmap v81: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.3 MiB/s wr, 19 op/s 2024-04-22T13:26:43.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:43 smithi111 ceph-mon[31361]: pgmap v81: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.3 MiB/s wr, 19 op/s 2024-04-22T13:26:45.586 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:45 smithi012 ceph-mon[28265]: pgmap v82: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 868 KiB/s wr, 13 op/s 2024-04-22T13:26:45.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:45 smithi134 ceph-mon[31338]: pgmap v82: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 868 KiB/s wr, 13 op/s 2024-04-22T13:26:45.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:45 smithi111 ceph-mon[31361]: pgmap v82: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 868 KiB/s wr, 13 op/s 2024-04-22T13:26:47.586 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:47 smithi012 ceph-mon[28265]: pgmap v83: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 868 KiB/s wr, 13 op/s 2024-04-22T13:26:47.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:47 smithi134 ceph-mon[31338]: pgmap v83: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 868 KiB/s wr, 13 op/s 2024-04-22T13:26:47.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:47 smithi111 ceph-mon[31361]: pgmap v83: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 868 KiB/s wr, 13 op/s 2024-04-22T13:26:48.693 INFO:tasks.workunit.client.0.smithi012.stdout:setting up random write file 2024-04-22T13:26:48.694 INFO:tasks.workunit.client.0.smithi012.stdout:done setting up random write file 2024-04-22T13:26:48.694 INFO:tasks.workunit.client.0.smithi012.stdout:starting fsync run 2024-04-22T13:26:48.694 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0003s fsync time: 0.0096s 2024-04-22T13:26:48.694 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0003s fsync time: 0.0055s 2024-04-22T13:26:48.694 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0003s fsync time: 0.0053s 2024-04-22T13:26:48.694 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0050s 2024-04-22T13:26:48.694 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0003s fsync time: 0.0051s 2024-04-22T13:26:48.694 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0003s fsync time: 0.0051s 2024-04-22T13:26:48.694 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0003s fsync time: 0.0058s 2024-04-22T13:26:48.694 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0053s 2024-04-22T13:26:48.694 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0052s 2024-04-22T13:26:48.694 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0049s 2024-04-22T13:26:48.694 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0049s 2024-04-22T13:26:48.695 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0050s 2024-04-22T13:26:48.695 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0003s fsync time: 0.0050s 2024-04-22T13:26:48.695 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0057s 2024-04-22T13:26:48.695 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0044s 2024-04-22T13:26:48.695 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0048s 2024-04-22T13:26:48.695 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0003s fsync time: 0.0043s 2024-04-22T13:26:48.695 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0047s 2024-04-22T13:26:48.695 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0041s 2024-04-22T13:26:48.695 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0123s 2024-04-22T13:26:48.695 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0043s 2024-04-22T13:26:48.695 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0044s 2024-04-22T13:26:48.695 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0045s 2024-04-22T13:26:48.695 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0044s 2024-04-22T13:26:48.695 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0052s 2024-04-22T13:26:48.695 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0003s fsync time: 0.0051s 2024-04-22T13:26:48.696 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0004s fsync time: 0.0046s 2024-04-22T13:26:48.696 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0048s 2024-04-22T13:26:48.696 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0046s 2024-04-22T13:26:48.696 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0046s 2024-04-22T13:26:48.696 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0045s 2024-04-22T13:26:48.696 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0042s 2024-04-22T13:26:48.696 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0048s 2024-04-22T13:26:48.696 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0003s fsync time: 0.0050s 2024-04-22T13:26:48.696 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0003s fsync time: 0.0059s 2024-04-22T13:26:48.696 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0044s 2024-04-22T13:26:48.696 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0003s fsync time: 0.0060s 2024-04-22T13:26:48.696 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0003s fsync time: 0.3395s 2024-04-22T13:26:48.696 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0052s 2024-04-22T13:26:48.696 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0003s fsync time: 0.0054s 2024-04-22T13:26:48.696 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0003s fsync time: 0.0057s 2024-04-22T13:26:48.696 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0052s 2024-04-22T13:26:48.696 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0051s 2024-04-22T13:26:48.697 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0003s fsync time: 0.0053s 2024-04-22T13:26:48.697 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0054s 2024-04-22T13:26:48.697 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0056s 2024-04-22T13:26:48.697 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0054s 2024-04-22T13:26:48.697 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0003s fsync time: 0.0054s 2024-04-22T13:26:48.697 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0051s 2024-04-22T13:26:48.697 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0003s fsync time: 0.0050s 2024-04-22T13:26:48.697 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0049s 2024-04-22T13:26:48.697 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0003s fsync time: 0.0053s 2024-04-22T13:26:48.697 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0055s 2024-04-22T13:26:48.697 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0054s 2024-04-22T13:26:48.697 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0003s fsync time: 0.0051s 2024-04-22T13:26:48.697 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0050s 2024-04-22T13:26:48.697 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0054s 2024-04-22T13:26:48.697 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0003s fsync time: 0.0051s 2024-04-22T13:26:48.697 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0002s fsync time: 0.0052s 2024-04-22T13:26:48.697 INFO:tasks.workunit.client.0.smithi012.stdout:write time: 0.0003s fsync time: 0.0050s 2024-04-22T13:26:48.698 INFO:tasks.workunit.client.0.smithi012.stdout:run done 61 fsyncs total, killing random writer 2024-04-22T13:26:48.698 INFO:tasks.workunit.client.0.smithi012.stderr:+ echo /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-04-22T13:26:48.698 INFO:tasks.workunit.client.0.smithi012.stdout:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-04-22T13:26:48.698 INFO:tasks.workunit.client.0.smithi012.stderr:+ whereis lsof 2024-04-22T13:26:48.708 INFO:tasks.workunit.client.0.smithi012.stdout:lsof: /usr/bin/lsof /usr/share/man/man1/lsof.1.gz 2024-04-22T13:26:48.708 INFO:tasks.workunit.client.0.smithi012.stderr:+ lsof 2024-04-22T13:26:48.728 INFO:tasks.workunit.client.0.smithi012.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/1c495296acf4acf184749bebd4e9f930d6796de889e1ff6f9b979f6228faf1d4/merged 2024-04-22T13:26:48.728 INFO:tasks.workunit.client.0.smithi012.stderr: Output information may be incomplete. 2024-04-22T13:26:48.728 INFO:tasks.workunit.client.0.smithi012.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/ef1cf92469688d547aca6eebcb3ca3f7362f3fba06f8da3934128301342b5166/merged 2024-04-22T13:26:48.728 INFO:tasks.workunit.client.0.smithi012.stderr: Output information may be incomplete. 2024-04-22T13:26:48.728 INFO:tasks.workunit.client.0.smithi012.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/0e8cc77728201e747918eb64815f34ba52b7a80d282cae8952338eba2780b49a/merged 2024-04-22T13:26:48.728 INFO:tasks.workunit.client.0.smithi012.stderr: Output information may be incomplete. 2024-04-22T13:26:48.728 INFO:tasks.workunit.client.0.smithi012.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/68fcadce5d3846175e2755aafc8cb83c4c7f17cab8aa4d836e8bd163fc3900f7/merged 2024-04-22T13:26:48.729 INFO:tasks.workunit.client.0.smithi012.stderr: Output information may be incomplete. 2024-04-22T13:26:48.729 INFO:tasks.workunit.client.0.smithi012.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/0223d1c101dacbedb94c66853128199c37e415261c66af6de033a0c6d726e2d5/merged 2024-04-22T13:26:48.729 INFO:tasks.workunit.client.0.smithi012.stderr: Output information may be incomplete. 2024-04-22T13:26:48.729 INFO:tasks.workunit.client.0.smithi012.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/87e990431e87ab74b03d7a919fc2998069a5dba0ea1898e37e9175d82a7a69d4/merged 2024-04-22T13:26:48.729 INFO:tasks.workunit.client.0.smithi012.stderr: Output information may be incomplete. 2024-04-22T13:26:48.729 INFO:tasks.workunit.client.0.smithi012.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/5892adaf22ea61f84b0a2ffc8e8c9d62efa464aa6d977a180ff3a61a92089105/merged 2024-04-22T13:26:48.729 INFO:tasks.workunit.client.0.smithi012.stderr: Output information may be incomplete. 2024-04-22T13:26:48.729 INFO:tasks.workunit.client.0.smithi012.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/51da75e1a4d9c6dacc8b9b2b6e269815d7c56d5027ea7b1ace7a48ed5d0c7ec2/merged 2024-04-22T13:26:48.729 INFO:tasks.workunit.client.0.smithi012.stderr: Output information may be incomplete. 2024-04-22T13:26:48.729 INFO:tasks.workunit.client.0.smithi012.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/7cbf3e9ef7060645288479948a258cbebd21d24191b3d7d1c0b916582422087d/merged 2024-04-22T13:26:48.729 INFO:tasks.workunit.client.0.smithi012.stderr: Output information may be incomplete. 2024-04-22T13:26:48.729 INFO:tasks.workunit.client.0.smithi012.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/609502d657c464da21381791a269ea5cefafeb591a4c2813b13a3c41e1e03411/merged 2024-04-22T13:26:48.729 INFO:tasks.workunit.client.0.smithi012.stderr: Output information may be incomplete. 2024-04-22T13:26:48.787 INFO:tasks.workunit.client.0.smithi012.stdout:COMMAND PID TID TASKCMD USER FD TYPE DEVICE SIZE/OFF NODE NAME 2024-04-22T13:26:48.787 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 1 root cwd unknown /proc/1/cwd (readlink: Permission denied) 2024-04-22T13:26:48.788 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 1 root rtd unknown /proc/1/root (readlink: Permission denied) 2024-04-22T13:26:48.788 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 1 root txt unknown /proc/1/exe (readlink: Permission denied) 2024-04-22T13:26:48.788 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 1 root NOFD /proc/1/fd (opendir: Permission denied) 2024-04-22T13:26:48.788 INFO:tasks.workunit.client.0.smithi012.stdout:kthreadd 2 root cwd unknown /proc/2/cwd (readlink: Permission denied) 2024-04-22T13:26:48.788 INFO:tasks.workunit.client.0.smithi012.stdout:kthreadd 2 root rtd unknown /proc/2/root (readlink: Permission denied) 2024-04-22T13:26:48.788 INFO:tasks.workunit.client.0.smithi012.stdout:kthreadd 2 root txt unknown /proc/2/exe (readlink: Permission denied) 2024-04-22T13:26:48.788 INFO:tasks.workunit.client.0.smithi012.stdout:kthreadd 2 root NOFD /proc/2/fd (opendir: Permission denied) 2024-04-22T13:26:48.788 INFO:tasks.workunit.client.0.smithi012.stdout:rcu_gp 3 root cwd unknown /proc/3/cwd (readlink: Permission denied) 2024-04-22T13:26:48.788 INFO:tasks.workunit.client.0.smithi012.stdout:rcu_gp 3 root rtd unknown /proc/3/root (readlink: Permission denied) 2024-04-22T13:26:48.788 INFO:tasks.workunit.client.0.smithi012.stdout:rcu_gp 3 root txt unknown /proc/3/exe (readlink: Permission denied) 2024-04-22T13:26:48.788 INFO:tasks.workunit.client.0.smithi012.stdout:rcu_gp 3 root NOFD /proc/3/fd (opendir: Permission denied) 2024-04-22T13:26:48.788 INFO:tasks.workunit.client.0.smithi012.stdout:rcu_par_g 4 root cwd unknown /proc/4/cwd (readlink: Permission denied) 2024-04-22T13:26:48.788 INFO:tasks.workunit.client.0.smithi012.stdout:rcu_par_g 4 root rtd unknown /proc/4/root (readlink: Permission denied) 2024-04-22T13:26:48.788 INFO:tasks.workunit.client.0.smithi012.stdout:rcu_par_g 4 root txt unknown /proc/4/exe (readlink: Permission denied) 2024-04-22T13:26:48.788 INFO:tasks.workunit.client.0.smithi012.stdout:rcu_par_g 4 root NOFD /proc/4/fd (opendir: Permission denied) 2024-04-22T13:26:48.788 INFO:tasks.workunit.client.0.smithi012.stdout:slub_flus 5 root cwd unknown /proc/5/cwd (readlink: Permission denied) 2024-04-22T13:26:48.789 INFO:tasks.workunit.client.0.smithi012.stdout:slub_flus 5 root rtd unknown /proc/5/root (readlink: Permission denied) 2024-04-22T13:26:48.789 INFO:tasks.workunit.client.0.smithi012.stdout:slub_flus 5 root txt unknown /proc/5/exe (readlink: Permission denied) 2024-04-22T13:26:48.789 INFO:tasks.workunit.client.0.smithi012.stdout:slub_flus 5 root NOFD /proc/5/fd (opendir: Permission denied) 2024-04-22T13:26:48.789 INFO:tasks.workunit.client.0.smithi012.stdout:netns 6 root cwd unknown /proc/6/cwd (readlink: Permission denied) 2024-04-22T13:26:48.789 INFO:tasks.workunit.client.0.smithi012.stdout:netns 6 root rtd unknown /proc/6/root (readlink: Permission denied) 2024-04-22T13:26:48.789 INFO:tasks.workunit.client.0.smithi012.stdout:netns 6 root txt unknown /proc/6/exe (readlink: Permission denied) 2024-04-22T13:26:48.789 INFO:tasks.workunit.client.0.smithi012.stdout:netns 6 root NOFD /proc/6/fd (opendir: Permission denied) 2024-04-22T13:26:48.789 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/0 8 root cwd unknown /proc/8/cwd (readlink: Permission denied) 2024-04-22T13:26:48.789 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/0 8 root rtd unknown /proc/8/root (readlink: Permission denied) 2024-04-22T13:26:48.789 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/0 8 root txt unknown /proc/8/exe (readlink: Permission denied) 2024-04-22T13:26:48.789 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/0 8 root NOFD /proc/8/fd (opendir: Permission denied) 2024-04-22T13:26:48.789 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 9 root cwd unknown /proc/9/cwd (readlink: Permission denied) 2024-04-22T13:26:48.789 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 9 root rtd unknown /proc/9/root (readlink: Permission denied) 2024-04-22T13:26:48.789 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 9 root txt unknown /proc/9/exe (readlink: Permission denied) 2024-04-22T13:26:48.789 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 9 root NOFD /proc/9/fd (opendir: Permission denied) 2024-04-22T13:26:48.789 INFO:tasks.workunit.client.0.smithi012.stdout:mm_percpu 10 root cwd unknown /proc/10/cwd (readlink: Permission denied) 2024-04-22T13:26:48.790 INFO:tasks.workunit.client.0.smithi012.stdout:mm_percpu 10 root rtd unknown /proc/10/root (readlink: Permission denied) 2024-04-22T13:26:48.790 INFO:tasks.workunit.client.0.smithi012.stdout:mm_percpu 10 root txt unknown /proc/10/exe (readlink: Permission denied) 2024-04-22T13:26:48.790 INFO:tasks.workunit.client.0.smithi012.stdout:mm_percpu 10 root NOFD /proc/10/fd (opendir: Permission denied) 2024-04-22T13:26:48.790 INFO:tasks.workunit.client.0.smithi012.stdout:rcu_tasks 12 root cwd unknown /proc/12/cwd (readlink: Permission denied) 2024-04-22T13:26:48.790 INFO:tasks.workunit.client.0.smithi012.stdout:rcu_tasks 12 root rtd unknown /proc/12/root (readlink: Permission denied) 2024-04-22T13:26:48.790 INFO:tasks.workunit.client.0.smithi012.stdout:rcu_tasks 12 root txt unknown /proc/12/exe (readlink: Permission denied) 2024-04-22T13:26:48.790 INFO:tasks.workunit.client.0.smithi012.stdout:rcu_tasks 12 root NOFD /proc/12/fd (opendir: Permission denied) 2024-04-22T13:26:48.790 INFO:tasks.workunit.client.0.smithi012.stdout:rcu_tasks 13 root cwd unknown /proc/13/cwd (readlink: Permission denied) 2024-04-22T13:26:48.790 INFO:tasks.workunit.client.0.smithi012.stdout:rcu_tasks 13 root rtd unknown /proc/13/root (readlink: Permission denied) 2024-04-22T13:26:48.790 INFO:tasks.workunit.client.0.smithi012.stdout:rcu_tasks 13 root txt unknown /proc/13/exe (readlink: Permission denied) 2024-04-22T13:26:48.790 INFO:tasks.workunit.client.0.smithi012.stdout:rcu_tasks 13 root NOFD /proc/13/fd (opendir: Permission denied) 2024-04-22T13:26:48.790 INFO:tasks.workunit.client.0.smithi012.stdout:rcu_tasks 14 root cwd unknown /proc/14/cwd (readlink: Permission denied) 2024-04-22T13:26:48.790 INFO:tasks.workunit.client.0.smithi012.stdout:rcu_tasks 14 root rtd unknown /proc/14/root (readlink: Permission denied) 2024-04-22T13:26:48.790 INFO:tasks.workunit.client.0.smithi012.stdout:rcu_tasks 14 root txt unknown /proc/14/exe (readlink: Permission denied) 2024-04-22T13:26:48.790 INFO:tasks.workunit.client.0.smithi012.stdout:rcu_tasks 14 root NOFD /proc/14/fd (opendir: Permission denied) 2024-04-22T13:26:48.790 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 15 root cwd unknown /proc/15/cwd (readlink: Permission denied) 2024-04-22T13:26:48.791 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 15 root rtd unknown /proc/15/root (readlink: Permission denied) 2024-04-22T13:26:48.791 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 15 root txt unknown /proc/15/exe (readlink: Permission denied) 2024-04-22T13:26:48.791 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 15 root NOFD /proc/15/fd (opendir: Permission denied) 2024-04-22T13:26:48.791 INFO:tasks.workunit.client.0.smithi012.stdout:pr/ttyS1 16 root cwd unknown /proc/16/cwd (readlink: Permission denied) 2024-04-22T13:26:48.791 INFO:tasks.workunit.client.0.smithi012.stdout:pr/ttyS1 16 root rtd unknown /proc/16/root (readlink: Permission denied) 2024-04-22T13:26:48.791 INFO:tasks.workunit.client.0.smithi012.stdout:pr/ttyS1 16 root txt unknown /proc/16/exe (readlink: Permission denied) 2024-04-22T13:26:48.791 INFO:tasks.workunit.client.0.smithi012.stdout:pr/ttyS1 16 root NOFD /proc/16/fd (opendir: Permission denied) 2024-04-22T13:26:48.791 INFO:tasks.workunit.client.0.smithi012.stdout:rcu_preem 17 root cwd unknown /proc/17/cwd (readlink: Permission denied) 2024-04-22T13:26:48.791 INFO:tasks.workunit.client.0.smithi012.stdout:rcu_preem 17 root rtd unknown /proc/17/root (readlink: Permission denied) 2024-04-22T13:26:48.791 INFO:tasks.workunit.client.0.smithi012.stdout:rcu_preem 17 root txt unknown /proc/17/exe (readlink: Permission denied) 2024-04-22T13:26:48.791 INFO:tasks.workunit.client.0.smithi012.stdout:rcu_preem 17 root NOFD /proc/17/fd (opendir: Permission denied) 2024-04-22T13:26:48.791 INFO:tasks.workunit.client.0.smithi012.stdout:migration 18 root cwd unknown /proc/18/cwd (readlink: Permission denied) 2024-04-22T13:26:48.791 INFO:tasks.workunit.client.0.smithi012.stdout:migration 18 root rtd unknown /proc/18/root (readlink: Permission denied) 2024-04-22T13:26:48.791 INFO:tasks.workunit.client.0.smithi012.stdout:migration 18 root txt unknown /proc/18/exe (readlink: Permission denied) 2024-04-22T13:26:48.791 INFO:tasks.workunit.client.0.smithi012.stdout:migration 18 root NOFD /proc/18/fd (opendir: Permission denied) 2024-04-22T13:26:48.791 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 19 root cwd unknown /proc/19/cwd (readlink: Permission denied) 2024-04-22T13:26:48.791 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 19 root rtd unknown /proc/19/root (readlink: Permission denied) 2024-04-22T13:26:48.792 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 19 root txt unknown /proc/19/exe (readlink: Permission denied) 2024-04-22T13:26:48.792 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 19 root NOFD /proc/19/fd (opendir: Permission denied) 2024-04-22T13:26:48.792 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/0 21 root cwd unknown /proc/21/cwd (readlink: Permission denied) 2024-04-22T13:26:48.792 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/0 21 root rtd unknown /proc/21/root (readlink: Permission denied) 2024-04-22T13:26:48.792 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/0 21 root txt unknown /proc/21/exe (readlink: Permission denied) 2024-04-22T13:26:48.792 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/0 21 root NOFD /proc/21/fd (opendir: Permission denied) 2024-04-22T13:26:48.792 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/1 22 root cwd unknown /proc/22/cwd (readlink: Permission denied) 2024-04-22T13:26:48.792 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/1 22 root rtd unknown /proc/22/root (readlink: Permission denied) 2024-04-22T13:26:48.792 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/1 22 root txt unknown /proc/22/exe (readlink: Permission denied) 2024-04-22T13:26:48.792 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/1 22 root NOFD /proc/22/fd (opendir: Permission denied) 2024-04-22T13:26:48.792 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 23 root cwd unknown /proc/23/cwd (readlink: Permission denied) 2024-04-22T13:26:48.792 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 23 root rtd unknown /proc/23/root (readlink: Permission denied) 2024-04-22T13:26:48.792 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 23 root txt unknown /proc/23/exe (readlink: Permission denied) 2024-04-22T13:26:48.792 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 23 root NOFD /proc/23/fd (opendir: Permission denied) 2024-04-22T13:26:48.792 INFO:tasks.workunit.client.0.smithi012.stdout:migration 24 root cwd unknown /proc/24/cwd (readlink: Permission denied) 2024-04-22T13:26:48.792 INFO:tasks.workunit.client.0.smithi012.stdout:migration 24 root rtd unknown /proc/24/root (readlink: Permission denied) 2024-04-22T13:26:48.793 INFO:tasks.workunit.client.0.smithi012.stdout:migration 24 root txt unknown /proc/24/exe (readlink: Permission denied) 2024-04-22T13:26:48.793 INFO:tasks.workunit.client.0.smithi012.stdout:migration 24 root NOFD /proc/24/fd (opendir: Permission denied) 2024-04-22T13:26:48.793 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 25 root cwd unknown /proc/25/cwd (readlink: Permission denied) 2024-04-22T13:26:48.793 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 25 root rtd unknown /proc/25/root (readlink: Permission denied) 2024-04-22T13:26:48.793 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 25 root txt unknown /proc/25/exe (readlink: Permission denied) 2024-04-22T13:26:48.793 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 25 root NOFD /proc/25/fd (opendir: Permission denied) 2024-04-22T13:26:48.793 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/1 27 root cwd unknown /proc/27/cwd (readlink: Permission denied) 2024-04-22T13:26:48.793 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/1 27 root rtd unknown /proc/27/root (readlink: Permission denied) 2024-04-22T13:26:48.793 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/1 27 root txt unknown /proc/27/exe (readlink: Permission denied) 2024-04-22T13:26:48.793 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/1 27 root NOFD /proc/27/fd (opendir: Permission denied) 2024-04-22T13:26:48.793 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/2 28 root cwd unknown /proc/28/cwd (readlink: Permission denied) 2024-04-22T13:26:48.793 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/2 28 root rtd unknown /proc/28/root (readlink: Permission denied) 2024-04-22T13:26:48.793 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/2 28 root txt unknown /proc/28/exe (readlink: Permission denied) 2024-04-22T13:26:48.793 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/2 28 root NOFD /proc/28/fd (opendir: Permission denied) 2024-04-22T13:26:48.793 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 29 root cwd unknown /proc/29/cwd (readlink: Permission denied) 2024-04-22T13:26:48.794 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 29 root rtd unknown /proc/29/root (readlink: Permission denied) 2024-04-22T13:26:48.794 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 29 root txt unknown /proc/29/exe (readlink: Permission denied) 2024-04-22T13:26:48.794 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 29 root NOFD /proc/29/fd (opendir: Permission denied) 2024-04-22T13:26:48.794 INFO:tasks.workunit.client.0.smithi012.stdout:migration 30 root cwd unknown /proc/30/cwd (readlink: Permission denied) 2024-04-22T13:26:48.794 INFO:tasks.workunit.client.0.smithi012.stdout:migration 30 root rtd unknown /proc/30/root (readlink: Permission denied) 2024-04-22T13:26:48.794 INFO:tasks.workunit.client.0.smithi012.stdout:migration 30 root txt unknown /proc/30/exe (readlink: Permission denied) 2024-04-22T13:26:48.794 INFO:tasks.workunit.client.0.smithi012.stdout:migration 30 root NOFD /proc/30/fd (opendir: Permission denied) 2024-04-22T13:26:48.794 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 31 root cwd unknown /proc/31/cwd (readlink: Permission denied) 2024-04-22T13:26:48.794 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 31 root rtd unknown /proc/31/root (readlink: Permission denied) 2024-04-22T13:26:48.794 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 31 root txt unknown /proc/31/exe (readlink: Permission denied) 2024-04-22T13:26:48.794 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 31 root NOFD /proc/31/fd (opendir: Permission denied) 2024-04-22T13:26:48.794 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/2 33 root cwd unknown /proc/33/cwd (readlink: Permission denied) 2024-04-22T13:26:48.794 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/2 33 root rtd unknown /proc/33/root (readlink: Permission denied) 2024-04-22T13:26:48.794 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/2 33 root txt unknown /proc/33/exe (readlink: Permission denied) 2024-04-22T13:26:48.795 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/2 33 root NOFD /proc/33/fd (opendir: Permission denied) 2024-04-22T13:26:48.795 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/3 34 root cwd unknown /proc/34/cwd (readlink: Permission denied) 2024-04-22T13:26:48.795 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/3 34 root rtd unknown /proc/34/root (readlink: Permission denied) 2024-04-22T13:26:48.795 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/3 34 root txt unknown /proc/34/exe (readlink: Permission denied) 2024-04-22T13:26:48.795 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/3 34 root NOFD /proc/34/fd (opendir: Permission denied) 2024-04-22T13:26:48.795 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 35 root cwd unknown /proc/35/cwd (readlink: Permission denied) 2024-04-22T13:26:48.795 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 35 root rtd unknown /proc/35/root (readlink: Permission denied) 2024-04-22T13:26:48.795 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 35 root txt unknown /proc/35/exe (readlink: Permission denied) 2024-04-22T13:26:48.795 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 35 root NOFD /proc/35/fd (opendir: Permission denied) 2024-04-22T13:26:48.795 INFO:tasks.workunit.client.0.smithi012.stdout:migration 36 root cwd unknown /proc/36/cwd (readlink: Permission denied) 2024-04-22T13:26:48.795 INFO:tasks.workunit.client.0.smithi012.stdout:migration 36 root rtd unknown /proc/36/root (readlink: Permission denied) 2024-04-22T13:26:48.795 INFO:tasks.workunit.client.0.smithi012.stdout:migration 36 root txt unknown /proc/36/exe (readlink: Permission denied) 2024-04-22T13:26:48.795 INFO:tasks.workunit.client.0.smithi012.stdout:migration 36 root NOFD /proc/36/fd (opendir: Permission denied) 2024-04-22T13:26:48.795 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 37 root cwd unknown /proc/37/cwd (readlink: Permission denied) 2024-04-22T13:26:48.796 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 37 root rtd unknown /proc/37/root (readlink: Permission denied) 2024-04-22T13:26:48.796 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 37 root txt unknown /proc/37/exe (readlink: Permission denied) 2024-04-22T13:26:48.796 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 37 root NOFD /proc/37/fd (opendir: Permission denied) 2024-04-22T13:26:48.796 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/3 39 root cwd unknown /proc/39/cwd (readlink: Permission denied) 2024-04-22T13:26:48.796 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/3 39 root rtd unknown /proc/39/root (readlink: Permission denied) 2024-04-22T13:26:48.796 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/3 39 root txt unknown /proc/39/exe (readlink: Permission denied) 2024-04-22T13:26:48.796 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/3 39 root NOFD /proc/39/fd (opendir: Permission denied) 2024-04-22T13:26:48.796 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/4 40 root cwd unknown /proc/40/cwd (readlink: Permission denied) 2024-04-22T13:26:48.796 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/4 40 root rtd unknown /proc/40/root (readlink: Permission denied) 2024-04-22T13:26:48.796 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/4 40 root txt unknown /proc/40/exe (readlink: Permission denied) 2024-04-22T13:26:48.796 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/4 40 root NOFD /proc/40/fd (opendir: Permission denied) 2024-04-22T13:26:48.796 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 41 root cwd unknown /proc/41/cwd (readlink: Permission denied) 2024-04-22T13:26:48.796 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 41 root rtd unknown /proc/41/root (readlink: Permission denied) 2024-04-22T13:26:48.796 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 41 root txt unknown /proc/41/exe (readlink: Permission denied) 2024-04-22T13:26:48.796 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 41 root NOFD /proc/41/fd (opendir: Permission denied) 2024-04-22T13:26:48.796 INFO:tasks.workunit.client.0.smithi012.stdout:migration 42 root cwd unknown /proc/42/cwd (readlink: Permission denied) 2024-04-22T13:26:48.797 INFO:tasks.workunit.client.0.smithi012.stdout:migration 42 root rtd unknown /proc/42/root (readlink: Permission denied) 2024-04-22T13:26:48.797 INFO:tasks.workunit.client.0.smithi012.stdout:migration 42 root txt unknown /proc/42/exe (readlink: Permission denied) 2024-04-22T13:26:48.797 INFO:tasks.workunit.client.0.smithi012.stdout:migration 42 root NOFD /proc/42/fd (opendir: Permission denied) 2024-04-22T13:26:48.797 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 43 root cwd unknown /proc/43/cwd (readlink: Permission denied) 2024-04-22T13:26:48.797 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 43 root rtd unknown /proc/43/root (readlink: Permission denied) 2024-04-22T13:26:48.797 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 43 root txt unknown /proc/43/exe (readlink: Permission denied) 2024-04-22T13:26:48.797 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 43 root NOFD /proc/43/fd (opendir: Permission denied) 2024-04-22T13:26:48.797 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/4 45 root cwd unknown /proc/45/cwd (readlink: Permission denied) 2024-04-22T13:26:48.797 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/4 45 root rtd unknown /proc/45/root (readlink: Permission denied) 2024-04-22T13:26:48.797 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/4 45 root txt unknown /proc/45/exe (readlink: Permission denied) 2024-04-22T13:26:48.797 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/4 45 root NOFD /proc/45/fd (opendir: Permission denied) 2024-04-22T13:26:48.797 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/5 46 root cwd unknown /proc/46/cwd (readlink: Permission denied) 2024-04-22T13:26:48.797 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/5 46 root rtd unknown /proc/46/root (readlink: Permission denied) 2024-04-22T13:26:48.797 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/5 46 root txt unknown /proc/46/exe (readlink: Permission denied) 2024-04-22T13:26:48.797 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/5 46 root NOFD /proc/46/fd (opendir: Permission denied) 2024-04-22T13:26:48.797 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 47 root cwd unknown /proc/47/cwd (readlink: Permission denied) 2024-04-22T13:26:48.797 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 47 root rtd unknown /proc/47/root (readlink: Permission denied) 2024-04-22T13:26:48.798 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 47 root txt unknown /proc/47/exe (readlink: Permission denied) 2024-04-22T13:26:48.798 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 47 root NOFD /proc/47/fd (opendir: Permission denied) 2024-04-22T13:26:48.798 INFO:tasks.workunit.client.0.smithi012.stdout:migration 48 root cwd unknown /proc/48/cwd (readlink: Permission denied) 2024-04-22T13:26:48.798 INFO:tasks.workunit.client.0.smithi012.stdout:migration 48 root rtd unknown /proc/48/root (readlink: Permission denied) 2024-04-22T13:26:48.798 INFO:tasks.workunit.client.0.smithi012.stdout:migration 48 root txt unknown /proc/48/exe (readlink: Permission denied) 2024-04-22T13:26:48.798 INFO:tasks.workunit.client.0.smithi012.stdout:migration 48 root NOFD /proc/48/fd (opendir: Permission denied) 2024-04-22T13:26:48.798 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 49 root cwd unknown /proc/49/cwd (readlink: Permission denied) 2024-04-22T13:26:48.798 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 49 root rtd unknown /proc/49/root (readlink: Permission denied) 2024-04-22T13:26:48.798 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 49 root txt unknown /proc/49/exe (readlink: Permission denied) 2024-04-22T13:26:48.798 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 49 root NOFD /proc/49/fd (opendir: Permission denied) 2024-04-22T13:26:48.798 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/5 51 root cwd unknown /proc/51/cwd (readlink: Permission denied) 2024-04-22T13:26:48.798 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/5 51 root rtd unknown /proc/51/root (readlink: Permission denied) 2024-04-22T13:26:48.798 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/5 51 root txt unknown /proc/51/exe (readlink: Permission denied) 2024-04-22T13:26:48.798 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/5 51 root NOFD /proc/51/fd (opendir: Permission denied) 2024-04-22T13:26:48.799 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/6 52 root cwd unknown /proc/52/cwd (readlink: Permission denied) 2024-04-22T13:26:48.799 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/6 52 root rtd unknown /proc/52/root (readlink: Permission denied) 2024-04-22T13:26:48.799 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/6 52 root txt unknown /proc/52/exe (readlink: Permission denied) 2024-04-22T13:26:48.799 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/6 52 root NOFD /proc/52/fd (opendir: Permission denied) 2024-04-22T13:26:48.799 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 53 root cwd unknown /proc/53/cwd (readlink: Permission denied) 2024-04-22T13:26:48.799 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 53 root rtd unknown /proc/53/root (readlink: Permission denied) 2024-04-22T13:26:48.799 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 53 root txt unknown /proc/53/exe (readlink: Permission denied) 2024-04-22T13:26:48.799 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 53 root NOFD /proc/53/fd (opendir: Permission denied) 2024-04-22T13:26:48.799 INFO:tasks.workunit.client.0.smithi012.stdout:migration 54 root cwd unknown /proc/54/cwd (readlink: Permission denied) 2024-04-22T13:26:48.799 INFO:tasks.workunit.client.0.smithi012.stdout:migration 54 root rtd unknown /proc/54/root (readlink: Permission denied) 2024-04-22T13:26:48.799 INFO:tasks.workunit.client.0.smithi012.stdout:migration 54 root txt unknown /proc/54/exe (readlink: Permission denied) 2024-04-22T13:26:48.799 INFO:tasks.workunit.client.0.smithi012.stdout:migration 54 root NOFD /proc/54/fd (opendir: Permission denied) 2024-04-22T13:26:48.799 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 55 root cwd unknown /proc/55/cwd (readlink: Permission denied) 2024-04-22T13:26:48.799 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 55 root rtd unknown /proc/55/root (readlink: Permission denied) 2024-04-22T13:26:48.799 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 55 root txt unknown /proc/55/exe (readlink: Permission denied) 2024-04-22T13:26:48.799 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 55 root NOFD /proc/55/fd (opendir: Permission denied) 2024-04-22T13:26:48.800 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/6 57 root cwd unknown /proc/57/cwd (readlink: Permission denied) 2024-04-22T13:26:48.800 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/6 57 root rtd unknown /proc/57/root (readlink: Permission denied) 2024-04-22T13:26:48.800 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/6 57 root txt unknown /proc/57/exe (readlink: Permission denied) 2024-04-22T13:26:48.800 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/6 57 root NOFD /proc/57/fd (opendir: Permission denied) 2024-04-22T13:26:48.800 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/7 58 root cwd unknown /proc/58/cwd (readlink: Permission denied) 2024-04-22T13:26:48.800 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/7 58 root rtd unknown /proc/58/root (readlink: Permission denied) 2024-04-22T13:26:48.800 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/7 58 root txt unknown /proc/58/exe (readlink: Permission denied) 2024-04-22T13:26:48.800 INFO:tasks.workunit.client.0.smithi012.stdout:cpuhp/7 58 root NOFD /proc/58/fd (opendir: Permission denied) 2024-04-22T13:26:48.800 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 59 root cwd unknown /proc/59/cwd (readlink: Permission denied) 2024-04-22T13:26:48.800 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 59 root rtd unknown /proc/59/root (readlink: Permission denied) 2024-04-22T13:26:48.800 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 59 root txt unknown /proc/59/exe (readlink: Permission denied) 2024-04-22T13:26:48.800 INFO:tasks.workunit.client.0.smithi012.stdout:idle_inje 59 root NOFD /proc/59/fd (opendir: Permission denied) 2024-04-22T13:26:48.800 INFO:tasks.workunit.client.0.smithi012.stdout:migration 60 root cwd unknown /proc/60/cwd (readlink: Permission denied) 2024-04-22T13:26:48.800 INFO:tasks.workunit.client.0.smithi012.stdout:migration 60 root rtd unknown /proc/60/root (readlink: Permission denied) 2024-04-22T13:26:48.800 INFO:tasks.workunit.client.0.smithi012.stdout:migration 60 root txt unknown /proc/60/exe (readlink: Permission denied) 2024-04-22T13:26:48.800 INFO:tasks.workunit.client.0.smithi012.stdout:migration 60 root NOFD /proc/60/fd (opendir: Permission denied) 2024-04-22T13:26:48.801 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 61 root cwd unknown /proc/61/cwd (readlink: Permission denied) 2024-04-22T13:26:48.801 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 61 root rtd unknown /proc/61/root (readlink: Permission denied) 2024-04-22T13:26:48.801 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 61 root txt unknown /proc/61/exe (readlink: Permission denied) 2024-04-22T13:26:48.801 INFO:tasks.workunit.client.0.smithi012.stdout:ksoftirqd 61 root NOFD /proc/61/fd (opendir: Permission denied) 2024-04-22T13:26:48.801 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/7 63 root cwd unknown /proc/63/cwd (readlink: Permission denied) 2024-04-22T13:26:48.801 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/7 63 root rtd unknown /proc/63/root (readlink: Permission denied) 2024-04-22T13:26:48.801 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/7 63 root txt unknown /proc/63/exe (readlink: Permission denied) 2024-04-22T13:26:48.801 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/7 63 root NOFD /proc/63/fd (opendir: Permission denied) 2024-04-22T13:26:48.801 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 65 root cwd unknown /proc/65/cwd (readlink: Permission denied) 2024-04-22T13:26:48.801 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 65 root rtd unknown /proc/65/root (readlink: Permission denied) 2024-04-22T13:26:48.801 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 65 root txt unknown /proc/65/exe (readlink: Permission denied) 2024-04-22T13:26:48.801 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 65 root NOFD /proc/65/fd (opendir: Permission denied) 2024-04-22T13:26:48.801 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 66 root cwd unknown /proc/66/cwd (readlink: Permission denied) 2024-04-22T13:26:48.801 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 66 root rtd unknown /proc/66/root (readlink: Permission denied) 2024-04-22T13:26:48.801 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 66 root txt unknown /proc/66/exe (readlink: Permission denied) 2024-04-22T13:26:48.801 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 66 root NOFD /proc/66/fd (opendir: Permission denied) 2024-04-22T13:26:48.802 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 68 root cwd unknown /proc/68/cwd (readlink: Permission denied) 2024-04-22T13:26:48.802 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 68 root rtd unknown /proc/68/root (readlink: Permission denied) 2024-04-22T13:26:48.802 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 68 root txt unknown /proc/68/exe (readlink: Permission denied) 2024-04-22T13:26:48.802 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 68 root NOFD /proc/68/fd (opendir: Permission denied) 2024-04-22T13:26:48.802 INFO:tasks.workunit.client.0.smithi012.stdout:kdevtmpfs 71 root cwd unknown /proc/71/cwd (readlink: Permission denied) 2024-04-22T13:26:48.802 INFO:tasks.workunit.client.0.smithi012.stdout:kdevtmpfs 71 root rtd unknown /proc/71/root (readlink: Permission denied) 2024-04-22T13:26:48.802 INFO:tasks.workunit.client.0.smithi012.stdout:kdevtmpfs 71 root txt unknown /proc/71/exe (readlink: Permission denied) 2024-04-22T13:26:48.802 INFO:tasks.workunit.client.0.smithi012.stdout:kdevtmpfs 71 root NOFD /proc/71/fd (opendir: Permission denied) 2024-04-22T13:26:48.802 INFO:tasks.workunit.client.0.smithi012.stdout:inet_frag 72 root cwd unknown /proc/72/cwd (readlink: Permission denied) 2024-04-22T13:26:48.802 INFO:tasks.workunit.client.0.smithi012.stdout:inet_frag 72 root rtd unknown /proc/72/root (readlink: Permission denied) 2024-04-22T13:26:48.802 INFO:tasks.workunit.client.0.smithi012.stdout:inet_frag 72 root txt unknown /proc/72/exe (readlink: Permission denied) 2024-04-22T13:26:48.802 INFO:tasks.workunit.client.0.smithi012.stdout:inet_frag 72 root NOFD /proc/72/fd (opendir: Permission denied) 2024-04-22T13:26:48.802 INFO:tasks.workunit.client.0.smithi012.stdout:kauditd 73 root cwd unknown /proc/73/cwd (readlink: Permission denied) 2024-04-22T13:26:48.802 INFO:tasks.workunit.client.0.smithi012.stdout:kauditd 73 root rtd unknown /proc/73/root (readlink: Permission denied) 2024-04-22T13:26:48.802 INFO:tasks.workunit.client.0.smithi012.stdout:kauditd 73 root txt unknown /proc/73/exe (readlink: Permission denied) 2024-04-22T13:26:48.803 INFO:tasks.workunit.client.0.smithi012.stdout:kauditd 73 root NOFD /proc/73/fd (opendir: Permission denied) 2024-04-22T13:26:48.803 INFO:tasks.workunit.client.0.smithi012.stdout:khungtask 75 root cwd unknown /proc/75/cwd (readlink: Permission denied) 2024-04-22T13:26:48.803 INFO:tasks.workunit.client.0.smithi012.stdout:khungtask 75 root rtd unknown /proc/75/root (readlink: Permission denied) 2024-04-22T13:26:48.803 INFO:tasks.workunit.client.0.smithi012.stdout:khungtask 75 root txt unknown /proc/75/exe (readlink: Permission denied) 2024-04-22T13:26:48.803 INFO:tasks.workunit.client.0.smithi012.stdout:khungtask 75 root NOFD /proc/75/fd (opendir: Permission denied) 2024-04-22T13:26:48.803 INFO:tasks.workunit.client.0.smithi012.stdout:oom_reape 76 root cwd unknown /proc/76/cwd (readlink: Permission denied) 2024-04-22T13:26:48.803 INFO:tasks.workunit.client.0.smithi012.stdout:oom_reape 76 root rtd unknown /proc/76/root (readlink: Permission denied) 2024-04-22T13:26:48.803 INFO:tasks.workunit.client.0.smithi012.stdout:oom_reape 76 root txt unknown /proc/76/exe (readlink: Permission denied) 2024-04-22T13:26:48.803 INFO:tasks.workunit.client.0.smithi012.stdout:oom_reape 76 root NOFD /proc/76/fd (opendir: Permission denied) 2024-04-22T13:26:48.803 INFO:tasks.workunit.client.0.smithi012.stdout:writeback 77 root cwd unknown /proc/77/cwd (readlink: Permission denied) 2024-04-22T13:26:48.803 INFO:tasks.workunit.client.0.smithi012.stdout:writeback 77 root rtd unknown /proc/77/root (readlink: Permission denied) 2024-04-22T13:26:48.803 INFO:tasks.workunit.client.0.smithi012.stdout:writeback 77 root txt unknown /proc/77/exe (readlink: Permission denied) 2024-04-22T13:26:48.803 INFO:tasks.workunit.client.0.smithi012.stdout:writeback 77 root NOFD /proc/77/fd (opendir: Permission denied) 2024-04-22T13:26:48.803 INFO:tasks.workunit.client.0.smithi012.stdout:kcompactd 78 root cwd unknown /proc/78/cwd (readlink: Permission denied) 2024-04-22T13:26:48.803 INFO:tasks.workunit.client.0.smithi012.stdout:kcompactd 78 root rtd unknown /proc/78/root (readlink: Permission denied) 2024-04-22T13:26:48.803 INFO:tasks.workunit.client.0.smithi012.stdout:kcompactd 78 root txt unknown /proc/78/exe (readlink: Permission denied) 2024-04-22T13:26:48.804 INFO:tasks.workunit.client.0.smithi012.stdout:kcompactd 78 root NOFD /proc/78/fd (opendir: Permission denied) 2024-04-22T13:26:48.804 INFO:tasks.workunit.client.0.smithi012.stdout:ksmd 79 root cwd unknown /proc/79/cwd (readlink: Permission denied) 2024-04-22T13:26:48.804 INFO:tasks.workunit.client.0.smithi012.stdout:ksmd 79 root rtd unknown /proc/79/root (readlink: Permission denied) 2024-04-22T13:26:48.804 INFO:tasks.workunit.client.0.smithi012.stdout:ksmd 79 root txt unknown /proc/79/exe (readlink: Permission denied) 2024-04-22T13:26:48.804 INFO:tasks.workunit.client.0.smithi012.stdout:ksmd 79 root NOFD /proc/79/fd (opendir: Permission denied) 2024-04-22T13:26:48.804 INFO:tasks.workunit.client.0.smithi012.stdout:khugepage 81 root cwd unknown /proc/81/cwd (readlink: Permission denied) 2024-04-22T13:26:48.804 INFO:tasks.workunit.client.0.smithi012.stdout:khugepage 81 root rtd unknown /proc/81/root (readlink: Permission denied) 2024-04-22T13:26:48.804 INFO:tasks.workunit.client.0.smithi012.stdout:khugepage 81 root txt unknown /proc/81/exe (readlink: Permission denied) 2024-04-22T13:26:48.804 INFO:tasks.workunit.client.0.smithi012.stdout:khugepage 81 root NOFD /proc/81/fd (opendir: Permission denied) 2024-04-22T13:26:48.804 INFO:tasks.workunit.client.0.smithi012.stdout:cryptd 82 root cwd unknown /proc/82/cwd (readlink: Permission denied) 2024-04-22T13:26:48.804 INFO:tasks.workunit.client.0.smithi012.stdout:cryptd 82 root rtd unknown /proc/82/root (readlink: Permission denied) 2024-04-22T13:26:48.804 INFO:tasks.workunit.client.0.smithi012.stdout:cryptd 82 root txt unknown /proc/82/exe (readlink: Permission denied) 2024-04-22T13:26:48.804 INFO:tasks.workunit.client.0.smithi012.stdout:cryptd 82 root NOFD /proc/82/fd (opendir: Permission denied) 2024-04-22T13:26:48.804 INFO:tasks.workunit.client.0.smithi012.stdout:kintegrit 83 root cwd unknown /proc/83/cwd (readlink: Permission denied) 2024-04-22T13:26:48.804 INFO:tasks.workunit.client.0.smithi012.stdout:kintegrit 83 root rtd unknown /proc/83/root (readlink: Permission denied) 2024-04-22T13:26:48.804 INFO:tasks.workunit.client.0.smithi012.stdout:kintegrit 83 root txt unknown /proc/83/exe (readlink: Permission denied) 2024-04-22T13:26:48.805 INFO:tasks.workunit.client.0.smithi012.stdout:kintegrit 83 root NOFD /proc/83/fd (opendir: Permission denied) 2024-04-22T13:26:48.805 INFO:tasks.workunit.client.0.smithi012.stdout:kblockd 84 root cwd unknown /proc/84/cwd (readlink: Permission denied) 2024-04-22T13:26:48.805 INFO:tasks.workunit.client.0.smithi012.stdout:kblockd 84 root rtd unknown /proc/84/root (readlink: Permission denied) 2024-04-22T13:26:48.805 INFO:tasks.workunit.client.0.smithi012.stdout:kblockd 84 root txt unknown /proc/84/exe (readlink: Permission denied) 2024-04-22T13:26:48.805 INFO:tasks.workunit.client.0.smithi012.stdout:kblockd 84 root NOFD /proc/84/fd (opendir: Permission denied) 2024-04-22T13:26:48.805 INFO:tasks.workunit.client.0.smithi012.stdout:blkcg_pun 85 root cwd unknown /proc/85/cwd (readlink: Permission denied) 2024-04-22T13:26:48.805 INFO:tasks.workunit.client.0.smithi012.stdout:blkcg_pun 85 root rtd unknown /proc/85/root (readlink: Permission denied) 2024-04-22T13:26:48.805 INFO:tasks.workunit.client.0.smithi012.stdout:blkcg_pun 85 root txt unknown /proc/85/exe (readlink: Permission denied) 2024-04-22T13:26:48.805 INFO:tasks.workunit.client.0.smithi012.stdout:blkcg_pun 85 root NOFD /proc/85/fd (opendir: Permission denied) 2024-04-22T13:26:48.805 INFO:tasks.workunit.client.0.smithi012.stdout:tpm_dev_w 86 root cwd unknown /proc/86/cwd (readlink: Permission denied) 2024-04-22T13:26:48.805 INFO:tasks.workunit.client.0.smithi012.stdout:tpm_dev_w 86 root rtd unknown /proc/86/root (readlink: Permission denied) 2024-04-22T13:26:48.805 INFO:tasks.workunit.client.0.smithi012.stdout:tpm_dev_w 86 root txt unknown /proc/86/exe (readlink: Permission denied) 2024-04-22T13:26:48.805 INFO:tasks.workunit.client.0.smithi012.stdout:tpm_dev_w 86 root NOFD /proc/86/fd (opendir: Permission denied) 2024-04-22T13:26:48.805 INFO:tasks.workunit.client.0.smithi012.stdout:md 87 root cwd unknown /proc/87/cwd (readlink: Permission denied) 2024-04-22T13:26:48.806 INFO:tasks.workunit.client.0.smithi012.stdout:md 87 root rtd unknown /proc/87/root (readlink: Permission denied) 2024-04-22T13:26:48.806 INFO:tasks.workunit.client.0.smithi012.stdout:md 87 root txt unknown /proc/87/exe (readlink: Permission denied) 2024-04-22T13:26:48.806 INFO:tasks.workunit.client.0.smithi012.stdout:md 87 root NOFD /proc/87/fd (opendir: Permission denied) 2024-04-22T13:26:48.806 INFO:tasks.workunit.client.0.smithi012.stdout:md_bitmap 88 root cwd unknown /proc/88/cwd (readlink: Permission denied) 2024-04-22T13:26:48.806 INFO:tasks.workunit.client.0.smithi012.stdout:md_bitmap 88 root rtd unknown /proc/88/root (readlink: Permission denied) 2024-04-22T13:26:48.806 INFO:tasks.workunit.client.0.smithi012.stdout:md_bitmap 88 root txt unknown /proc/88/exe (readlink: Permission denied) 2024-04-22T13:26:48.806 INFO:tasks.workunit.client.0.smithi012.stdout:md_bitmap 88 root NOFD /proc/88/fd (opendir: Permission denied) 2024-04-22T13:26:48.806 INFO:tasks.workunit.client.0.smithi012.stdout:edac-poll 89 root cwd unknown /proc/89/cwd (readlink: Permission denied) 2024-04-22T13:26:48.806 INFO:tasks.workunit.client.0.smithi012.stdout:edac-poll 89 root rtd unknown /proc/89/root (readlink: Permission denied) 2024-04-22T13:26:48.806 INFO:tasks.workunit.client.0.smithi012.stdout:edac-poll 89 root txt unknown /proc/89/exe (readlink: Permission denied) 2024-04-22T13:26:48.806 INFO:tasks.workunit.client.0.smithi012.stdout:edac-poll 89 root NOFD /proc/89/fd (opendir: Permission denied) 2024-04-22T13:26:48.806 INFO:tasks.workunit.client.0.smithi012.stdout:watchdogd 90 root cwd unknown /proc/90/cwd (readlink: Permission denied) 2024-04-22T13:26:48.806 INFO:tasks.workunit.client.0.smithi012.stdout:watchdogd 90 root rtd unknown /proc/90/root (readlink: Permission denied) 2024-04-22T13:26:48.806 INFO:tasks.workunit.client.0.smithi012.stdout:watchdogd 90 root txt unknown /proc/90/exe (readlink: Permission denied) 2024-04-22T13:26:48.806 INFO:tasks.workunit.client.0.smithi012.stdout:watchdogd 90 root NOFD /proc/90/fd (opendir: Permission denied) 2024-04-22T13:26:48.807 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/6 91 root cwd unknown /proc/91/cwd (readlink: Permission denied) 2024-04-22T13:26:48.807 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/6 91 root rtd unknown /proc/91/root (readlink: Permission denied) 2024-04-22T13:26:48.807 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/6 91 root txt unknown /proc/91/exe (readlink: Permission denied) 2024-04-22T13:26:48.807 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/6 91 root NOFD /proc/91/fd (opendir: Permission denied) 2024-04-22T13:26:48.807 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/2 92 root cwd unknown /proc/92/cwd (readlink: Permission denied) 2024-04-22T13:26:48.807 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/2 92 root rtd unknown /proc/92/root (readlink: Permission denied) 2024-04-22T13:26:48.807 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/2 92 root txt unknown /proc/92/exe (readlink: Permission denied) 2024-04-22T13:26:48.807 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/2 92 root NOFD /proc/92/fd (opendir: Permission denied) 2024-04-22T13:26:48.807 INFO:tasks.workunit.client.0.smithi012.stdout:kswapd0 94 root cwd unknown /proc/94/cwd (readlink: Permission denied) 2024-04-22T13:26:48.807 INFO:tasks.workunit.client.0.smithi012.stdout:kswapd0 94 root rtd unknown /proc/94/root (readlink: Permission denied) 2024-04-22T13:26:48.807 INFO:tasks.workunit.client.0.smithi012.stdout:kswapd0 94 root txt unknown /proc/94/exe (readlink: Permission denied) 2024-04-22T13:26:48.807 INFO:tasks.workunit.client.0.smithi012.stdout:kswapd0 94 root NOFD /proc/94/fd (opendir: Permission denied) 2024-04-22T13:26:48.807 INFO:tasks.workunit.client.0.smithi012.stdout:kthrotld 100 root cwd unknown /proc/100/cwd (readlink: Permission denied) 2024-04-22T13:26:48.807 INFO:tasks.workunit.client.0.smithi012.stdout:kthrotld 100 root rtd unknown /proc/100/root (readlink: Permission denied) 2024-04-22T13:26:48.807 INFO:tasks.workunit.client.0.smithi012.stdout:kthrotld 100 root txt unknown /proc/100/exe (readlink: Permission denied) 2024-04-22T13:26:48.807 INFO:tasks.workunit.client.0.smithi012.stdout:kthrotld 100 root NOFD /proc/100/fd (opendir: Permission denied) 2024-04-22T13:26:48.807 INFO:tasks.workunit.client.0.smithi012.stdout:acpi_ther 108 root cwd unknown /proc/108/cwd (readlink: Permission denied) 2024-04-22T13:26:48.808 INFO:tasks.workunit.client.0.smithi012.stdout:acpi_ther 108 root rtd unknown /proc/108/root (readlink: Permission denied) 2024-04-22T13:26:48.808 INFO:tasks.workunit.client.0.smithi012.stdout:acpi_ther 108 root txt unknown /proc/108/exe (readlink: Permission denied) 2024-04-22T13:26:48.808 INFO:tasks.workunit.client.0.smithi012.stdout:acpi_ther 108 root NOFD /proc/108/fd (opendir: Permission denied) 2024-04-22T13:26:48.808 INFO:tasks.workunit.client.0.smithi012.stdout:kmpath_rd 109 root cwd unknown /proc/109/cwd (readlink: Permission denied) 2024-04-22T13:26:48.808 INFO:tasks.workunit.client.0.smithi012.stdout:kmpath_rd 109 root rtd unknown /proc/109/root (readlink: Permission denied) 2024-04-22T13:26:48.808 INFO:tasks.workunit.client.0.smithi012.stdout:kmpath_rd 109 root txt unknown /proc/109/exe (readlink: Permission denied) 2024-04-22T13:26:48.808 INFO:tasks.workunit.client.0.smithi012.stdout:kmpath_rd 109 root NOFD /proc/109/fd (opendir: Permission denied) 2024-04-22T13:26:48.808 INFO:tasks.workunit.client.0.smithi012.stdout:kaluad 110 root cwd unknown /proc/110/cwd (readlink: Permission denied) 2024-04-22T13:26:48.808 INFO:tasks.workunit.client.0.smithi012.stdout:kaluad 110 root rtd unknown /proc/110/root (readlink: Permission denied) 2024-04-22T13:26:48.808 INFO:tasks.workunit.client.0.smithi012.stdout:kaluad 110 root txt unknown /proc/110/exe (readlink: Permission denied) 2024-04-22T13:26:48.808 INFO:tasks.workunit.client.0.smithi012.stdout:kaluad 110 root NOFD /proc/110/fd (opendir: Permission denied) 2024-04-22T13:26:48.808 INFO:tasks.workunit.client.0.smithi012.stdout:mld 111 root cwd unknown /proc/111/cwd (readlink: Permission denied) 2024-04-22T13:26:48.808 INFO:tasks.workunit.client.0.smithi012.stdout:mld 111 root rtd unknown /proc/111/root (readlink: Permission denied) 2024-04-22T13:26:48.808 INFO:tasks.workunit.client.0.smithi012.stdout:mld 111 root txt unknown /proc/111/exe (readlink: Permission denied) 2024-04-22T13:26:48.808 INFO:tasks.workunit.client.0.smithi012.stdout:mld 111 root NOFD /proc/111/fd (opendir: Permission denied) 2024-04-22T13:26:48.808 INFO:tasks.workunit.client.0.smithi012.stdout:ipv6_addr 112 root cwd unknown /proc/112/cwd (readlink: Permission denied) 2024-04-22T13:26:48.808 INFO:tasks.workunit.client.0.smithi012.stdout:ipv6_addr 112 root rtd unknown /proc/112/root (readlink: Permission denied) 2024-04-22T13:26:48.809 INFO:tasks.workunit.client.0.smithi012.stdout:ipv6_addr 112 root txt unknown /proc/112/exe (readlink: Permission denied) 2024-04-22T13:26:48.809 INFO:tasks.workunit.client.0.smithi012.stdout:ipv6_addr 112 root NOFD /proc/112/fd (opendir: Permission denied) 2024-04-22T13:26:48.809 INFO:tasks.workunit.client.0.smithi012.stdout:kstrp 121 root cwd unknown /proc/121/cwd (readlink: Permission denied) 2024-04-22T13:26:48.809 INFO:tasks.workunit.client.0.smithi012.stdout:kstrp 121 root rtd unknown /proc/121/root (readlink: Permission denied) 2024-04-22T13:26:48.809 INFO:tasks.workunit.client.0.smithi012.stdout:kstrp 121 root txt unknown /proc/121/exe (readlink: Permission denied) 2024-04-22T13:26:48.809 INFO:tasks.workunit.client.0.smithi012.stdout:kstrp 121 root NOFD /proc/121/fd (opendir: Permission denied) 2024-04-22T13:26:48.809 INFO:tasks.workunit.client.0.smithi012.stdout:zswap-shr 129 root cwd unknown /proc/129/cwd (readlink: Permission denied) 2024-04-22T13:26:48.809 INFO:tasks.workunit.client.0.smithi012.stdout:zswap-shr 129 root rtd unknown /proc/129/root (readlink: Permission denied) 2024-04-22T13:26:48.809 INFO:tasks.workunit.client.0.smithi012.stdout:zswap-shr 129 root txt unknown /proc/129/exe (readlink: Permission denied) 2024-04-22T13:26:48.809 INFO:tasks.workunit.client.0.smithi012.stdout:zswap-shr 129 root NOFD /proc/129/fd (opendir: Permission denied) 2024-04-22T13:26:48.809 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 130 root cwd unknown /proc/130/cwd (readlink: Permission denied) 2024-04-22T13:26:48.809 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 130 root rtd unknown /proc/130/root (readlink: Permission denied) 2024-04-22T13:26:48.809 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 130 root txt unknown /proc/130/exe (readlink: Permission denied) 2024-04-22T13:26:48.809 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 130 root NOFD /proc/130/fd (opendir: Permission denied) 2024-04-22T13:26:48.809 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/1 237 root cwd unknown /proc/237/cwd (readlink: Permission denied) 2024-04-22T13:26:48.809 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/1 237 root rtd unknown /proc/237/root (readlink: Permission denied) 2024-04-22T13:26:48.810 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/1 237 root txt unknown /proc/237/exe (readlink: Permission denied) 2024-04-22T13:26:48.810 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/1 237 root NOFD /proc/237/fd (opendir: Permission denied) 2024-04-22T13:26:48.810 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/2 277 root cwd unknown /proc/277/cwd (readlink: Permission denied) 2024-04-22T13:26:48.810 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/2 277 root rtd unknown /proc/277/root (readlink: Permission denied) 2024-04-22T13:26:48.810 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/2 277 root txt unknown /proc/277/exe (readlink: Permission denied) 2024-04-22T13:26:48.810 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/2 277 root NOFD /proc/277/fd (opendir: Permission denied) 2024-04-22T13:26:48.810 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/3 279 root cwd unknown /proc/279/cwd (readlink: Permission denied) 2024-04-22T13:26:48.810 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/3 279 root rtd unknown /proc/279/root (readlink: Permission denied) 2024-04-22T13:26:48.810 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/3 279 root txt unknown /proc/279/exe (readlink: Permission denied) 2024-04-22T13:26:48.810 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/3 279 root NOFD /proc/279/fd (opendir: Permission denied) 2024-04-22T13:26:48.810 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/7 283 root cwd unknown /proc/283/cwd (readlink: Permission denied) 2024-04-22T13:26:48.810 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/7 283 root rtd unknown /proc/283/root (readlink: Permission denied) 2024-04-22T13:26:48.810 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/7 283 root txt unknown /proc/283/exe (readlink: Permission denied) 2024-04-22T13:26:48.810 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/7 283 root NOFD /proc/283/fd (opendir: Permission denied) 2024-04-22T13:26:48.810 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/4 286 root cwd unknown /proc/286/cwd (readlink: Permission denied) 2024-04-22T13:26:48.811 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/4 286 root rtd unknown /proc/286/root (readlink: Permission denied) 2024-04-22T13:26:48.811 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/4 286 root txt unknown /proc/286/exe (readlink: Permission denied) 2024-04-22T13:26:48.811 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/4 286 root NOFD /proc/286/fd (opendir: Permission denied) 2024-04-22T13:26:48.811 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/5 295 root cwd unknown /proc/295/cwd (readlink: Permission denied) 2024-04-22T13:26:48.811 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/5 295 root rtd unknown /proc/295/root (readlink: Permission denied) 2024-04-22T13:26:48.811 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/5 295 root txt unknown /proc/295/exe (readlink: Permission denied) 2024-04-22T13:26:48.811 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/5 295 root NOFD /proc/295/fd (opendir: Permission denied) 2024-04-22T13:26:48.811 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/0 379 root cwd unknown /proc/379/cwd (readlink: Permission denied) 2024-04-22T13:26:48.811 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/0 379 root rtd unknown /proc/379/root (readlink: Permission denied) 2024-04-22T13:26:48.811 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/0 379 root txt unknown /proc/379/exe (readlink: Permission denied) 2024-04-22T13:26:48.811 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/0 379 root NOFD /proc/379/fd (opendir: Permission denied) 2024-04-22T13:26:48.811 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/5 406 root cwd unknown /proc/406/cwd (readlink: Permission denied) 2024-04-22T13:26:48.811 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/5 406 root rtd unknown /proc/406/root (readlink: Permission denied) 2024-04-22T13:26:48.811 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/5 406 root txt unknown /proc/406/exe (readlink: Permission denied) 2024-04-22T13:26:48.811 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/5 406 root NOFD /proc/406/fd (opendir: Permission denied) 2024-04-22T13:26:48.811 INFO:tasks.workunit.client.0.smithi012.stdout:ixgbe 436 root cwd unknown /proc/436/cwd (readlink: Permission denied) 2024-04-22T13:26:48.811 INFO:tasks.workunit.client.0.smithi012.stdout:ixgbe 436 root rtd unknown /proc/436/root (readlink: Permission denied) 2024-04-22T13:26:48.812 INFO:tasks.workunit.client.0.smithi012.stdout:ixgbe 436 root txt unknown /proc/436/exe (readlink: Permission denied) 2024-04-22T13:26:48.812 INFO:tasks.workunit.client.0.smithi012.stdout:ixgbe 436 root NOFD /proc/436/fd (opendir: Permission denied) 2024-04-22T13:26:48.812 INFO:tasks.workunit.client.0.smithi012.stdout:ata_sff 439 root cwd unknown /proc/439/cwd (readlink: Permission denied) 2024-04-22T13:26:48.812 INFO:tasks.workunit.client.0.smithi012.stdout:ata_sff 439 root rtd unknown /proc/439/root (readlink: Permission denied) 2024-04-22T13:26:48.812 INFO:tasks.workunit.client.0.smithi012.stdout:ata_sff 439 root txt unknown /proc/439/exe (readlink: Permission denied) 2024-04-22T13:26:48.812 INFO:tasks.workunit.client.0.smithi012.stdout:ata_sff 439 root NOFD /proc/439/fd (opendir: Permission denied) 2024-04-22T13:26:48.812 INFO:tasks.workunit.client.0.smithi012.stdout:nvme-wq 443 root cwd unknown /proc/443/cwd (readlink: Permission denied) 2024-04-22T13:26:48.812 INFO:tasks.workunit.client.0.smithi012.stdout:nvme-wq 443 root rtd unknown /proc/443/root (readlink: Permission denied) 2024-04-22T13:26:48.812 INFO:tasks.workunit.client.0.smithi012.stdout:nvme-wq 443 root txt unknown /proc/443/exe (readlink: Permission denied) 2024-04-22T13:26:48.812 INFO:tasks.workunit.client.0.smithi012.stdout:nvme-wq 443 root NOFD /proc/443/fd (opendir: Permission denied) 2024-04-22T13:26:48.812 INFO:tasks.workunit.client.0.smithi012.stdout:nvme-rese 444 root cwd unknown /proc/444/cwd (readlink: Permission denied) 2024-04-22T13:26:48.812 INFO:tasks.workunit.client.0.smithi012.stdout:nvme-rese 444 root rtd unknown /proc/444/root (readlink: Permission denied) 2024-04-22T13:26:48.812 INFO:tasks.workunit.client.0.smithi012.stdout:nvme-rese 444 root txt unknown /proc/444/exe (readlink: Permission denied) 2024-04-22T13:26:48.812 INFO:tasks.workunit.client.0.smithi012.stdout:nvme-rese 444 root NOFD /proc/444/fd (opendir: Permission denied) 2024-04-22T13:26:48.812 INFO:tasks.workunit.client.0.smithi012.stdout:nvme-dele 446 root cwd unknown /proc/446/cwd (readlink: Permission denied) 2024-04-22T13:26:48.812 INFO:tasks.workunit.client.0.smithi012.stdout:nvme-dele 446 root rtd unknown /proc/446/root (readlink: Permission denied) 2024-04-22T13:26:48.813 INFO:tasks.workunit.client.0.smithi012.stdout:nvme-dele 446 root txt unknown /proc/446/exe (readlink: Permission denied) 2024-04-22T13:26:48.813 INFO:tasks.workunit.client.0.smithi012.stdout:nvme-dele 446 root NOFD /proc/446/fd (opendir: Permission denied) 2024-04-22T13:26:48.813 INFO:tasks.workunit.client.0.smithi012.stdout:nvme-auth 447 root cwd unknown /proc/447/cwd (readlink: Permission denied) 2024-04-22T13:26:48.813 INFO:tasks.workunit.client.0.smithi012.stdout:nvme-auth 447 root rtd unknown /proc/447/root (readlink: Permission denied) 2024-04-22T13:26:48.813 INFO:tasks.workunit.client.0.smithi012.stdout:nvme-auth 447 root txt unknown /proc/447/exe (readlink: Permission denied) 2024-04-22T13:26:48.813 INFO:tasks.workunit.client.0.smithi012.stdout:nvme-auth 447 root NOFD /proc/447/fd (opendir: Permission denied) 2024-04-22T13:26:48.813 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_0 449 root cwd unknown /proc/449/cwd (readlink: Permission denied) 2024-04-22T13:26:48.813 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_0 449 root rtd unknown /proc/449/root (readlink: Permission denied) 2024-04-22T13:26:48.813 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_0 449 root txt unknown /proc/449/exe (readlink: Permission denied) 2024-04-22T13:26:48.813 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_0 449 root NOFD /proc/449/fd (opendir: Permission denied) 2024-04-22T13:26:48.813 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 450 root cwd unknown /proc/450/cwd (readlink: Permission denied) 2024-04-22T13:26:48.813 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 450 root rtd unknown /proc/450/root (readlink: Permission denied) 2024-04-22T13:26:48.813 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 450 root txt unknown /proc/450/exe (readlink: Permission denied) 2024-04-22T13:26:48.813 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 450 root NOFD /proc/450/fd (opendir: Permission denied) 2024-04-22T13:26:48.813 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_1 451 root cwd unknown /proc/451/cwd (readlink: Permission denied) 2024-04-22T13:26:48.813 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_1 451 root rtd unknown /proc/451/root (readlink: Permission denied) 2024-04-22T13:26:48.813 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_1 451 root txt unknown /proc/451/exe (readlink: Permission denied) 2024-04-22T13:26:48.814 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_1 451 root NOFD /proc/451/fd (opendir: Permission denied) 2024-04-22T13:26:48.814 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 452 root cwd unknown /proc/452/cwd (readlink: Permission denied) 2024-04-22T13:26:48.814 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 452 root rtd unknown /proc/452/root (readlink: Permission denied) 2024-04-22T13:26:48.814 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 452 root txt unknown /proc/452/exe (readlink: Permission denied) 2024-04-22T13:26:48.814 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 452 root NOFD /proc/452/fd (opendir: Permission denied) 2024-04-22T13:26:48.814 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_2 453 root cwd unknown /proc/453/cwd (readlink: Permission denied) 2024-04-22T13:26:48.814 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_2 453 root rtd unknown /proc/453/root (readlink: Permission denied) 2024-04-22T13:26:48.814 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_2 453 root txt unknown /proc/453/exe (readlink: Permission denied) 2024-04-22T13:26:48.814 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_2 453 root NOFD /proc/453/fd (opendir: Permission denied) 2024-04-22T13:26:48.814 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 454 root cwd unknown /proc/454/cwd (readlink: Permission denied) 2024-04-22T13:26:48.814 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 454 root rtd unknown /proc/454/root (readlink: Permission denied) 2024-04-22T13:26:48.814 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 454 root txt unknown /proc/454/exe (readlink: Permission denied) 2024-04-22T13:26:48.814 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 454 root NOFD /proc/454/fd (opendir: Permission denied) 2024-04-22T13:26:48.814 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_3 455 root cwd unknown /proc/455/cwd (readlink: Permission denied) 2024-04-22T13:26:48.814 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_3 455 root rtd unknown /proc/455/root (readlink: Permission denied) 2024-04-22T13:26:48.815 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_3 455 root txt unknown /proc/455/exe (readlink: Permission denied) 2024-04-22T13:26:48.815 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_3 455 root NOFD /proc/455/fd (opendir: Permission denied) 2024-04-22T13:26:48.815 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 456 root cwd unknown /proc/456/cwd (readlink: Permission denied) 2024-04-22T13:26:48.815 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 456 root rtd unknown /proc/456/root (readlink: Permission denied) 2024-04-22T13:26:48.815 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 456 root txt unknown /proc/456/exe (readlink: Permission denied) 2024-04-22T13:26:48.815 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 456 root NOFD /proc/456/fd (opendir: Permission denied) 2024-04-22T13:26:48.815 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_4 458 root cwd unknown /proc/458/cwd (readlink: Permission denied) 2024-04-22T13:26:48.815 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_4 458 root rtd unknown /proc/458/root (readlink: Permission denied) 2024-04-22T13:26:48.815 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_4 458 root txt unknown /proc/458/exe (readlink: Permission denied) 2024-04-22T13:26:48.815 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_4 458 root NOFD /proc/458/fd (opendir: Permission denied) 2024-04-22T13:26:48.815 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 459 root cwd unknown /proc/459/cwd (readlink: Permission denied) 2024-04-22T13:26:48.815 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 459 root rtd unknown /proc/459/root (readlink: Permission denied) 2024-04-22T13:26:48.815 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 459 root txt unknown /proc/459/exe (readlink: Permission denied) 2024-04-22T13:26:48.815 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 459 root NOFD /proc/459/fd (opendir: Permission denied) 2024-04-22T13:26:48.815 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_5 460 root cwd unknown /proc/460/cwd (readlink: Permission denied) 2024-04-22T13:26:48.815 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_5 460 root rtd unknown /proc/460/root (readlink: Permission denied) 2024-04-22T13:26:48.815 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_5 460 root txt unknown /proc/460/exe (readlink: Permission denied) 2024-04-22T13:26:48.816 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_5 460 root NOFD /proc/460/fd (opendir: Permission denied) 2024-04-22T13:26:48.816 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 461 root cwd unknown /proc/461/cwd (readlink: Permission denied) 2024-04-22T13:26:48.816 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 461 root rtd unknown /proc/461/root (readlink: Permission denied) 2024-04-22T13:26:48.816 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 461 root txt unknown /proc/461/exe (readlink: Permission denied) 2024-04-22T13:26:48.816 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 461 root NOFD /proc/461/fd (opendir: Permission denied) 2024-04-22T13:26:48.816 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_6 462 root cwd unknown /proc/462/cwd (readlink: Permission denied) 2024-04-22T13:26:48.816 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_6 462 root rtd unknown /proc/462/root (readlink: Permission denied) 2024-04-22T13:26:48.816 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_6 462 root txt unknown /proc/462/exe (readlink: Permission denied) 2024-04-22T13:26:48.816 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_6 462 root NOFD /proc/462/fd (opendir: Permission denied) 2024-04-22T13:26:48.816 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 463 root cwd unknown /proc/463/cwd (readlink: Permission denied) 2024-04-22T13:26:48.816 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 463 root rtd unknown /proc/463/root (readlink: Permission denied) 2024-04-22T13:26:48.816 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 463 root txt unknown /proc/463/exe (readlink: Permission denied) 2024-04-22T13:26:48.816 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 463 root NOFD /proc/463/fd (opendir: Permission denied) 2024-04-22T13:26:48.816 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_7 464 root cwd unknown /proc/464/cwd (readlink: Permission denied) 2024-04-22T13:26:48.816 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_7 464 root rtd unknown /proc/464/root (readlink: Permission denied) 2024-04-22T13:26:48.817 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_7 464 root txt unknown /proc/464/exe (readlink: Permission denied) 2024-04-22T13:26:48.817 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_7 464 root NOFD /proc/464/fd (opendir: Permission denied) 2024-04-22T13:26:48.817 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 465 root cwd unknown /proc/465/cwd (readlink: Permission denied) 2024-04-22T13:26:48.817 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 465 root rtd unknown /proc/465/root (readlink: Permission denied) 2024-04-22T13:26:48.817 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 465 root txt unknown /proc/465/exe (readlink: Permission denied) 2024-04-22T13:26:48.817 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 465 root NOFD /proc/465/fd (opendir: Permission denied) 2024-04-22T13:26:48.817 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_8 466 root cwd unknown /proc/466/cwd (readlink: Permission denied) 2024-04-22T13:26:48.817 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_8 466 root rtd unknown /proc/466/root (readlink: Permission denied) 2024-04-22T13:26:48.817 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_8 466 root txt unknown /proc/466/exe (readlink: Permission denied) 2024-04-22T13:26:48.817 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_8 466 root NOFD /proc/466/fd (opendir: Permission denied) 2024-04-22T13:26:48.817 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 467 root cwd unknown /proc/467/cwd (readlink: Permission denied) 2024-04-22T13:26:48.817 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 467 root rtd unknown /proc/467/root (readlink: Permission denied) 2024-04-22T13:26:48.817 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 467 root txt unknown /proc/467/exe (readlink: Permission denied) 2024-04-22T13:26:48.817 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 467 root NOFD /proc/467/fd (opendir: Permission denied) 2024-04-22T13:26:48.817 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_9 468 root cwd unknown /proc/468/cwd (readlink: Permission denied) 2024-04-22T13:26:48.817 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_9 468 root rtd unknown /proc/468/root (readlink: Permission denied) 2024-04-22T13:26:48.817 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_9 468 root txt unknown /proc/468/exe (readlink: Permission denied) 2024-04-22T13:26:48.818 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_eh_9 468 root NOFD /proc/468/fd (opendir: Permission denied) 2024-04-22T13:26:48.818 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 469 root cwd unknown /proc/469/cwd (readlink: Permission denied) 2024-04-22T13:26:48.818 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 469 root rtd unknown /proc/469/root (readlink: Permission denied) 2024-04-22T13:26:48.818 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 469 root txt unknown /proc/469/exe (readlink: Permission denied) 2024-04-22T13:26:48.818 INFO:tasks.workunit.client.0.smithi012.stdout:scsi_tmf_ 469 root NOFD /proc/469/fd (opendir: Permission denied) 2024-04-22T13:26:48.818 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 471 root cwd unknown /proc/471/cwd (readlink: Permission denied) 2024-04-22T13:26:48.818 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 471 root rtd unknown /proc/471/root (readlink: Permission denied) 2024-04-22T13:26:48.818 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 471 root txt unknown /proc/471/exe (readlink: Permission denied) 2024-04-22T13:26:48.818 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 471 root NOFD /proc/471/fd (opendir: Permission denied) 2024-04-22T13:26:48.818 INFO:tasks.workunit.client.0.smithi012.stdout:jbd2/sda1 497 root cwd unknown /proc/497/cwd (readlink: Permission denied) 2024-04-22T13:26:48.818 INFO:tasks.workunit.client.0.smithi012.stdout:jbd2/sda1 497 root rtd unknown /proc/497/root (readlink: Permission denied) 2024-04-22T13:26:48.818 INFO:tasks.workunit.client.0.smithi012.stdout:jbd2/sda1 497 root txt unknown /proc/497/exe (readlink: Permission denied) 2024-04-22T13:26:48.818 INFO:tasks.workunit.client.0.smithi012.stdout:jbd2/sda1 497 root NOFD /proc/497/fd (opendir: Permission denied) 2024-04-22T13:26:48.818 INFO:tasks.workunit.client.0.smithi012.stdout:ext4-rsv- 498 root cwd unknown /proc/498/cwd (readlink: Permission denied) 2024-04-22T13:26:48.818 INFO:tasks.workunit.client.0.smithi012.stdout:ext4-rsv- 498 root rtd unknown /proc/498/root (readlink: Permission denied) 2024-04-22T13:26:48.819 INFO:tasks.workunit.client.0.smithi012.stdout:ext4-rsv- 498 root txt unknown /proc/498/exe (readlink: Permission denied) 2024-04-22T13:26:48.819 INFO:tasks.workunit.client.0.smithi012.stdout:ext4-rsv- 498 root NOFD /proc/498/fd (opendir: Permission denied) 2024-04-22T13:26:48.819 INFO:tasks.workunit.client.0.smithi012.stdout:systemd-j 574 root cwd unknown /proc/574/cwd (readlink: Permission denied) 2024-04-22T13:26:48.819 INFO:tasks.workunit.client.0.smithi012.stdout:systemd-j 574 root rtd unknown /proc/574/root (readlink: Permission denied) 2024-04-22T13:26:48.819 INFO:tasks.workunit.client.0.smithi012.stdout:systemd-j 574 root txt unknown /proc/574/exe (readlink: Permission denied) 2024-04-22T13:26:48.819 INFO:tasks.workunit.client.0.smithi012.stdout:systemd-j 574 root NOFD /proc/574/fd (opendir: Permission denied) 2024-04-22T13:26:48.819 INFO:tasks.workunit.client.0.smithi012.stdout:ext4lazyi 580 root cwd unknown /proc/580/cwd (readlink: Permission denied) 2024-04-22T13:26:48.819 INFO:tasks.workunit.client.0.smithi012.stdout:ext4lazyi 580 root rtd unknown /proc/580/root (readlink: Permission denied) 2024-04-22T13:26:48.819 INFO:tasks.workunit.client.0.smithi012.stdout:ext4lazyi 580 root txt unknown /proc/580/exe (readlink: Permission denied) 2024-04-22T13:26:48.819 INFO:tasks.workunit.client.0.smithi012.stdout:ext4lazyi 580 root NOFD /proc/580/fd (opendir: Permission denied) 2024-04-22T13:26:48.819 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/2 587 root cwd unknown /proc/587/cwd (readlink: Permission denied) 2024-04-22T13:26:48.819 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/2 587 root rtd unknown /proc/587/root (readlink: Permission denied) 2024-04-22T13:26:48.819 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/2 587 root txt unknown /proc/587/exe (readlink: Permission denied) 2024-04-22T13:26:48.819 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/2 587 root NOFD /proc/587/fd (opendir: Permission denied) 2024-04-22T13:26:48.819 INFO:tasks.workunit.client.0.smithi012.stdout:systemd-u 593 root cwd unknown /proc/593/cwd (readlink: Permission denied) 2024-04-22T13:26:48.819 INFO:tasks.workunit.client.0.smithi012.stdout:systemd-u 593 root rtd unknown /proc/593/root (readlink: Permission denied) 2024-04-22T13:26:48.819 INFO:tasks.workunit.client.0.smithi012.stdout:systemd-u 593 root txt unknown /proc/593/exe (readlink: Permission denied) 2024-04-22T13:26:48.820 INFO:tasks.workunit.client.0.smithi012.stdout:systemd-u 593 root NOFD /proc/593/fd (opendir: Permission denied) 2024-04-22T13:26:48.820 INFO:tasks.workunit.client.0.smithi012.stdout:ipmi-msgh 646 root cwd unknown /proc/646/cwd (readlink: Permission denied) 2024-04-22T13:26:48.820 INFO:tasks.workunit.client.0.smithi012.stdout:ipmi-msgh 646 root rtd unknown /proc/646/root (readlink: Permission denied) 2024-04-22T13:26:48.820 INFO:tasks.workunit.client.0.smithi012.stdout:ipmi-msgh 646 root txt unknown /proc/646/exe (readlink: Permission denied) 2024-04-22T13:26:48.820 INFO:tasks.workunit.client.0.smithi012.stdout:ipmi-msgh 646 root NOFD /proc/646/fd (opendir: Permission denied) 2024-04-22T13:26:48.820 INFO:tasks.workunit.client.0.smithi012.stdout:kipmi0 660 root cwd unknown /proc/660/cwd (readlink: Permission denied) 2024-04-22T13:26:48.820 INFO:tasks.workunit.client.0.smithi012.stdout:kipmi0 660 root rtd unknown /proc/660/root (readlink: Permission denied) 2024-04-22T13:26:48.820 INFO:tasks.workunit.client.0.smithi012.stdout:kipmi0 660 root txt unknown /proc/660/exe (readlink: Permission denied) 2024-04-22T13:26:48.820 INFO:tasks.workunit.client.0.smithi012.stdout:kipmi0 660 root NOFD /proc/660/fd (opendir: Permission denied) 2024-04-22T13:26:48.820 INFO:tasks.workunit.client.0.smithi012.stdout:rpcbind 673 rpc cwd unknown /proc/673/cwd (readlink: Permission denied) 2024-04-22T13:26:48.820 INFO:tasks.workunit.client.0.smithi012.stdout:rpcbind 673 rpc rtd unknown /proc/673/root (readlink: Permission denied) 2024-04-22T13:26:48.820 INFO:tasks.workunit.client.0.smithi012.stdout:rpcbind 673 rpc txt unknown /proc/673/exe (readlink: Permission denied) 2024-04-22T13:26:48.820 INFO:tasks.workunit.client.0.smithi012.stdout:rpcbind 673 rpc NOFD /proc/673/fd (opendir: Permission denied) 2024-04-22T13:26:48.820 INFO:tasks.workunit.client.0.smithi012.stdout:auditd 675 root cwd unknown /proc/675/cwd (readlink: Permission denied) 2024-04-22T13:26:48.820 INFO:tasks.workunit.client.0.smithi012.stdout:auditd 675 root rtd unknown /proc/675/root (readlink: Permission denied) 2024-04-22T13:26:48.820 INFO:tasks.workunit.client.0.smithi012.stdout:auditd 675 root txt unknown /proc/675/exe (readlink: Permission denied) 2024-04-22T13:26:48.820 INFO:tasks.workunit.client.0.smithi012.stdout:auditd 675 root NOFD /proc/675/fd (opendir: Permission denied) 2024-04-22T13:26:48.821 INFO:tasks.workunit.client.0.smithi012.stdout:auditd 675 678 auditd root cwd unknown /proc/675/task/678/cwd (readlink: Permission denied) 2024-04-22T13:26:48.821 INFO:tasks.workunit.client.0.smithi012.stdout:auditd 675 678 auditd root rtd unknown /proc/675/task/678/root (readlink: Permission denied) 2024-04-22T13:26:48.821 INFO:tasks.workunit.client.0.smithi012.stdout:auditd 675 678 auditd root txt unknown /proc/675/task/678/exe (readlink: Permission denied) 2024-04-22T13:26:48.821 INFO:tasks.workunit.client.0.smithi012.stdout:auditd 675 678 auditd root NOFD /proc/675/task/678/fd (opendir: Permission denied) 2024-04-22T13:26:48.821 INFO:tasks.workunit.client.0.smithi012.stdout:auditd 675 680 auditd root cwd unknown /proc/675/task/680/cwd (readlink: Permission denied) 2024-04-22T13:26:48.821 INFO:tasks.workunit.client.0.smithi012.stdout:auditd 675 680 auditd root rtd unknown /proc/675/task/680/root (readlink: Permission denied) 2024-04-22T13:26:48.821 INFO:tasks.workunit.client.0.smithi012.stdout:auditd 675 680 auditd root txt unknown /proc/675/task/680/exe (readlink: Permission denied) 2024-04-22T13:26:48.821 INFO:tasks.workunit.client.0.smithi012.stdout:auditd 675 680 auditd root NOFD /proc/675/task/680/fd (opendir: Permission denied) 2024-04-22T13:26:48.821 INFO:tasks.workunit.client.0.smithi012.stdout:rpciod 676 root cwd unknown /proc/676/cwd (readlink: Permission denied) 2024-04-22T13:26:48.821 INFO:tasks.workunit.client.0.smithi012.stdout:rpciod 676 root rtd unknown /proc/676/root (readlink: Permission denied) 2024-04-22T13:26:48.821 INFO:tasks.workunit.client.0.smithi012.stdout:rpciod 676 root txt unknown /proc/676/exe (readlink: Permission denied) 2024-04-22T13:26:48.821 INFO:tasks.workunit.client.0.smithi012.stdout:rpciod 676 root NOFD /proc/676/fd (opendir: Permission denied) 2024-04-22T13:26:48.821 INFO:tasks.workunit.client.0.smithi012.stdout:xprtiod 677 root cwd unknown /proc/677/cwd (readlink: Permission denied) 2024-04-22T13:26:48.821 INFO:tasks.workunit.client.0.smithi012.stdout:xprtiod 677 root rtd unknown /proc/677/root (readlink: Permission denied) 2024-04-22T13:26:48.821 INFO:tasks.workunit.client.0.smithi012.stdout:xprtiod 677 root txt unknown /proc/677/exe (readlink: Permission denied) 2024-04-22T13:26:48.821 INFO:tasks.workunit.client.0.smithi012.stdout:xprtiod 677 root NOFD /proc/677/fd (opendir: Permission denied) 2024-04-22T13:26:48.822 INFO:tasks.workunit.client.0.smithi012.stdout:sedispatc 679 root cwd unknown /proc/679/cwd (readlink: Permission denied) 2024-04-22T13:26:48.822 INFO:tasks.workunit.client.0.smithi012.stdout:sedispatc 679 root rtd unknown /proc/679/root (readlink: Permission denied) 2024-04-22T13:26:48.822 INFO:tasks.workunit.client.0.smithi012.stdout:sedispatc 679 root txt unknown /proc/679/exe (readlink: Permission denied) 2024-04-22T13:26:48.822 INFO:tasks.workunit.client.0.smithi012.stdout:sedispatc 679 root NOFD /proc/679/fd (opendir: Permission denied) 2024-04-22T13:26:48.822 INFO:tasks.workunit.client.0.smithi012.stdout:dbus-brok 717 dbus cwd unknown /proc/717/cwd (readlink: Permission denied) 2024-04-22T13:26:48.822 INFO:tasks.workunit.client.0.smithi012.stdout:dbus-brok 717 dbus rtd unknown /proc/717/root (readlink: Permission denied) 2024-04-22T13:26:48.822 INFO:tasks.workunit.client.0.smithi012.stdout:dbus-brok 717 dbus txt unknown /proc/717/exe (readlink: Permission denied) 2024-04-22T13:26:48.822 INFO:tasks.workunit.client.0.smithi012.stdout:dbus-brok 717 dbus NOFD /proc/717/fd (opendir: Permission denied) 2024-04-22T13:26:48.822 INFO:tasks.workunit.client.0.smithi012.stdout:dbus-brok 718 dbus cwd unknown /proc/718/cwd (readlink: Permission denied) 2024-04-22T13:26:48.822 INFO:tasks.workunit.client.0.smithi012.stdout:dbus-brok 718 dbus rtd unknown /proc/718/root (readlink: Permission denied) 2024-04-22T13:26:48.822 INFO:tasks.workunit.client.0.smithi012.stdout:dbus-brok 718 dbus txt unknown /proc/718/exe (readlink: Permission denied) 2024-04-22T13:26:48.822 INFO:tasks.workunit.client.0.smithi012.stdout:dbus-brok 718 dbus NOFD /proc/718/fd (opendir: Permission denied) 2024-04-22T13:26:48.822 INFO:tasks.workunit.client.0.smithi012.stdout:NetworkMa 719 root cwd unknown /proc/719/cwd (readlink: Permission denied) 2024-04-22T13:26:48.822 INFO:tasks.workunit.client.0.smithi012.stdout:NetworkMa 719 root rtd unknown /proc/719/root (readlink: Permission denied) 2024-04-22T13:26:48.822 INFO:tasks.workunit.client.0.smithi012.stdout:NetworkMa 719 root txt unknown /proc/719/exe (readlink: Permission denied) 2024-04-22T13:26:48.822 INFO:tasks.workunit.client.0.smithi012.stdout:NetworkMa 719 root NOFD /proc/719/fd (opendir: Permission denied) 2024-04-22T13:26:48.823 INFO:tasks.workunit.client.0.smithi012.stdout:NetworkMa 719 760 gmain root cwd unknown /proc/719/task/760/cwd (readlink: Permission denied) 2024-04-22T13:26:48.823 INFO:tasks.workunit.client.0.smithi012.stdout:NetworkMa 719 760 gmain root rtd unknown /proc/719/task/760/root (readlink: Permission denied) 2024-04-22T13:26:48.823 INFO:tasks.workunit.client.0.smithi012.stdout:NetworkMa 719 760 gmain root txt unknown /proc/719/task/760/exe (readlink: Permission denied) 2024-04-22T13:26:48.823 INFO:tasks.workunit.client.0.smithi012.stdout:NetworkMa 719 760 gmain root NOFD /proc/719/task/760/fd (opendir: Permission denied) 2024-04-22T13:26:48.823 INFO:tasks.workunit.client.0.smithi012.stdout:NetworkMa 719 761 gdbus root cwd unknown /proc/719/task/761/cwd (readlink: Permission denied) 2024-04-22T13:26:48.823 INFO:tasks.workunit.client.0.smithi012.stdout:NetworkMa 719 761 gdbus root rtd unknown /proc/719/task/761/root (readlink: Permission denied) 2024-04-22T13:26:48.823 INFO:tasks.workunit.client.0.smithi012.stdout:NetworkMa 719 761 gdbus root txt unknown /proc/719/task/761/exe (readlink: Permission denied) 2024-04-22T13:26:48.823 INFO:tasks.workunit.client.0.smithi012.stdout:NetworkMa 719 761 gdbus root NOFD /proc/719/task/761/fd (opendir: Permission denied) 2024-04-22T13:26:48.823 INFO:tasks.workunit.client.0.smithi012.stdout:irqbalanc 722 root cwd unknown /proc/722/cwd (readlink: Permission denied) 2024-04-22T13:26:48.823 INFO:tasks.workunit.client.0.smithi012.stdout:irqbalanc 722 root rtd unknown /proc/722/root (readlink: Permission denied) 2024-04-22T13:26:48.823 INFO:tasks.workunit.client.0.smithi012.stdout:irqbalanc 722 root txt unknown /proc/722/exe (readlink: Permission denied) 2024-04-22T13:26:48.823 INFO:tasks.workunit.client.0.smithi012.stdout:irqbalanc 722 root NOFD /proc/722/fd (opendir: Permission denied) 2024-04-22T13:26:48.823 INFO:tasks.workunit.client.0.smithi012.stdout:irqbalanc 722 740 gmain root cwd unknown /proc/722/task/740/cwd (readlink: Permission denied) 2024-04-22T13:26:48.823 INFO:tasks.workunit.client.0.smithi012.stdout:irqbalanc 722 740 gmain root rtd unknown /proc/722/task/740/root (readlink: Permission denied) 2024-04-22T13:26:48.823 INFO:tasks.workunit.client.0.smithi012.stdout:irqbalanc 722 740 gmain root txt unknown /proc/722/task/740/exe (readlink: Permission denied) 2024-04-22T13:26:48.824 INFO:tasks.workunit.client.0.smithi012.stdout:irqbalanc 722 740 gmain root NOFD /proc/722/task/740/fd (opendir: Permission denied) 2024-04-22T13:26:48.824 INFO:tasks.workunit.client.0.smithi012.stdout:lsmd 723 libstoragemgmt cwd unknown /proc/723/cwd (readlink: Permission denied) 2024-04-22T13:26:48.824 INFO:tasks.workunit.client.0.smithi012.stdout:lsmd 723 libstoragemgmt rtd unknown /proc/723/root (readlink: Permission denied) 2024-04-22T13:26:48.824 INFO:tasks.workunit.client.0.smithi012.stdout:lsmd 723 libstoragemgmt txt unknown /proc/723/exe (readlink: Permission denied) 2024-04-22T13:26:48.824 INFO:tasks.workunit.client.0.smithi012.stdout:lsmd 723 libstoragemgmt NOFD /proc/723/fd (opendir: Permission denied) 2024-04-22T13:26:48.824 INFO:tasks.workunit.client.0.smithi012.stdout:mcelog 724 root cwd unknown /proc/724/cwd (readlink: Permission denied) 2024-04-22T13:26:48.824 INFO:tasks.workunit.client.0.smithi012.stdout:mcelog 724 root rtd unknown /proc/724/root (readlink: Permission denied) 2024-04-22T13:26:48.824 INFO:tasks.workunit.client.0.smithi012.stdout:mcelog 724 root txt unknown /proc/724/exe (readlink: Permission denied) 2024-04-22T13:26:48.824 INFO:tasks.workunit.client.0.smithi012.stdout:mcelog 724 root NOFD /proc/724/fd (opendir: Permission denied) 2024-04-22T13:26:48.824 INFO:tasks.workunit.client.0.smithi012.stdout:smartd 725 root cwd unknown /proc/725/cwd (readlink: Permission denied) 2024-04-22T13:26:48.824 INFO:tasks.workunit.client.0.smithi012.stdout:smartd 725 root rtd unknown /proc/725/root (readlink: Permission denied) 2024-04-22T13:26:48.824 INFO:tasks.workunit.client.0.smithi012.stdout:smartd 725 root txt unknown /proc/725/exe (readlink: Permission denied) 2024-04-22T13:26:48.824 INFO:tasks.workunit.client.0.smithi012.stdout:smartd 725 root NOFD /proc/725/fd (opendir: Permission denied) 2024-04-22T13:26:48.824 INFO:tasks.workunit.client.0.smithi012.stdout:systemd-l 730 root cwd unknown /proc/730/cwd (readlink: Permission denied) 2024-04-22T13:26:48.824 INFO:tasks.workunit.client.0.smithi012.stdout:systemd-l 730 root rtd unknown /proc/730/root (readlink: Permission denied) 2024-04-22T13:26:48.824 INFO:tasks.workunit.client.0.smithi012.stdout:systemd-l 730 root txt unknown /proc/730/exe (readlink: Permission denied) 2024-04-22T13:26:48.825 INFO:tasks.workunit.client.0.smithi012.stdout:systemd-l 730 root NOFD /proc/730/fd (opendir: Permission denied) 2024-04-22T13:26:48.825 INFO:tasks.workunit.client.0.smithi012.stdout:sshd 796 root cwd unknown /proc/796/cwd (readlink: Permission denied) 2024-04-22T13:26:48.825 INFO:tasks.workunit.client.0.smithi012.stdout:sshd 796 root rtd unknown /proc/796/root (readlink: Permission denied) 2024-04-22T13:26:48.825 INFO:tasks.workunit.client.0.smithi012.stdout:sshd 796 root txt unknown /proc/796/exe (readlink: Permission denied) 2024-04-22T13:26:48.825 INFO:tasks.workunit.client.0.smithi012.stdout:sshd 796 root NOFD /proc/796/fd (opendir: Permission denied) 2024-04-22T13:26:48.825 INFO:tasks.workunit.client.0.smithi012.stdout:gssproxy 800 root cwd unknown /proc/800/cwd (readlink: Permission denied) 2024-04-22T13:26:48.825 INFO:tasks.workunit.client.0.smithi012.stdout:gssproxy 800 root rtd unknown /proc/800/root (readlink: Permission denied) 2024-04-22T13:26:48.825 INFO:tasks.workunit.client.0.smithi012.stdout:gssproxy 800 root txt unknown /proc/800/exe (readlink: Permission denied) 2024-04-22T13:26:48.825 INFO:tasks.workunit.client.0.smithi012.stdout:gssproxy 800 root NOFD /proc/800/fd (opendir: Permission denied) 2024-04-22T13:26:48.825 INFO:tasks.workunit.client.0.smithi012.stdout:gssproxy 800 801 gssproxy root cwd unknown /proc/800/task/801/cwd (readlink: Permission denied) 2024-04-22T13:26:48.825 INFO:tasks.workunit.client.0.smithi012.stdout:gssproxy 800 801 gssproxy root rtd unknown /proc/800/task/801/root (readlink: Permission denied) 2024-04-22T13:26:48.825 INFO:tasks.workunit.client.0.smithi012.stdout:gssproxy 800 801 gssproxy root txt unknown /proc/800/task/801/exe (readlink: Permission denied) 2024-04-22T13:26:48.825 INFO:tasks.workunit.client.0.smithi012.stdout:gssproxy 800 801 gssproxy root NOFD /proc/800/task/801/fd (opendir: Permission denied) 2024-04-22T13:26:48.825 INFO:tasks.workunit.client.0.smithi012.stdout:gssproxy 800 802 gssproxy root cwd unknown /proc/800/task/802/cwd (readlink: Permission denied) 2024-04-22T13:26:48.825 INFO:tasks.workunit.client.0.smithi012.stdout:gssproxy 800 802 gssproxy root rtd unknown /proc/800/task/802/root (readlink: Permission denied) 2024-04-22T13:26:48.825 INFO:tasks.workunit.client.0.smithi012.stdout:gssproxy 800 802 gssproxy root txt unknown /proc/800/task/802/exe (readlink: Permission denied) 2024-04-22T13:26:48.825 INFO:tasks.workunit.client.0.smithi012.stdout:gssproxy 800 802 gssproxy root NOFD /proc/800/task/802/fd (opendir: Permission denied) 2024-04-22T13:26:48.826 INFO:tasks.workunit.client.0.smithi012.stdout:gssproxy 800 803 gssproxy root cwd unknown /proc/800/task/803/cwd (readlink: Permission denied) 2024-04-22T13:26:48.826 INFO:tasks.workunit.client.0.smithi012.stdout:gssproxy 800 803 gssproxy root rtd unknown /proc/800/task/803/root (readlink: Permission denied) 2024-04-22T13:26:48.826 INFO:tasks.workunit.client.0.smithi012.stdout:gssproxy 800 803 gssproxy root txt unknown /proc/800/task/803/exe (readlink: Permission denied) 2024-04-22T13:26:48.826 INFO:tasks.workunit.client.0.smithi012.stdout:gssproxy 800 803 gssproxy root NOFD /proc/800/task/803/fd (opendir: Permission denied) 2024-04-22T13:26:48.826 INFO:tasks.workunit.client.0.smithi012.stdout:gssproxy 800 804 gssproxy root cwd unknown /proc/800/task/804/cwd (readlink: Permission denied) 2024-04-22T13:26:48.826 INFO:tasks.workunit.client.0.smithi012.stdout:gssproxy 800 804 gssproxy root rtd unknown /proc/800/task/804/root (readlink: Permission denied) 2024-04-22T13:26:48.826 INFO:tasks.workunit.client.0.smithi012.stdout:gssproxy 800 804 gssproxy root txt unknown /proc/800/task/804/exe (readlink: Permission denied) 2024-04-22T13:26:48.826 INFO:tasks.workunit.client.0.smithi012.stdout:gssproxy 800 804 gssproxy root NOFD /proc/800/task/804/fd (opendir: Permission denied) 2024-04-22T13:26:48.826 INFO:tasks.workunit.client.0.smithi012.stdout:gssproxy 800 805 gssproxy root cwd unknown /proc/800/task/805/cwd (readlink: Permission denied) 2024-04-22T13:26:48.826 INFO:tasks.workunit.client.0.smithi012.stdout:gssproxy 800 805 gssproxy root rtd unknown /proc/800/task/805/root (readlink: Permission denied) 2024-04-22T13:26:48.826 INFO:tasks.workunit.client.0.smithi012.stdout:gssproxy 800 805 gssproxy root txt unknown /proc/800/task/805/exe (readlink: Permission denied) 2024-04-22T13:26:48.826 INFO:tasks.workunit.client.0.smithi012.stdout:gssproxy 800 805 gssproxy root NOFD /proc/800/task/805/fd (opendir: Permission denied) 2024-04-22T13:26:48.826 INFO:tasks.workunit.client.0.smithi012.stdout:atd 807 root cwd unknown /proc/807/cwd (readlink: Permission denied) 2024-04-22T13:26:48.826 INFO:tasks.workunit.client.0.smithi012.stdout:atd 807 root rtd unknown /proc/807/root (readlink: Permission denied) 2024-04-22T13:26:48.826 INFO:tasks.workunit.client.0.smithi012.stdout:atd 807 root txt unknown /proc/807/exe (readlink: Permission denied) 2024-04-22T13:26:48.827 INFO:tasks.workunit.client.0.smithi012.stdout:atd 807 root NOFD /proc/807/fd (opendir: Permission denied) 2024-04-22T13:26:48.827 INFO:tasks.workunit.client.0.smithi012.stdout:crond 808 root cwd unknown /proc/808/cwd (readlink: Permission denied) 2024-04-22T13:26:48.827 INFO:tasks.workunit.client.0.smithi012.stdout:crond 808 root rtd unknown /proc/808/root (readlink: Permission denied) 2024-04-22T13:26:48.827 INFO:tasks.workunit.client.0.smithi012.stdout:crond 808 root txt unknown /proc/808/exe (readlink: Permission denied) 2024-04-22T13:26:48.827 INFO:tasks.workunit.client.0.smithi012.stdout:crond 808 root NOFD /proc/808/fd (opendir: Permission denied) 2024-04-22T13:26:48.827 INFO:tasks.workunit.client.0.smithi012.stdout:nrpe 832 nrpe cwd unknown /proc/832/cwd (readlink: Permission denied) 2024-04-22T13:26:48.827 INFO:tasks.workunit.client.0.smithi012.stdout:nrpe 832 nrpe rtd unknown /proc/832/root (readlink: Permission denied) 2024-04-22T13:26:48.827 INFO:tasks.workunit.client.0.smithi012.stdout:nrpe 832 nrpe txt unknown /proc/832/exe (readlink: Permission denied) 2024-04-22T13:26:48.827 INFO:tasks.workunit.client.0.smithi012.stdout:nrpe 832 nrpe NOFD /proc/832/fd (opendir: Permission denied) 2024-04-22T13:26:48.827 INFO:tasks.workunit.client.0.smithi012.stdout:agetty 5173 root cwd unknown /proc/5173/cwd (readlink: Permission denied) 2024-04-22T13:26:48.827 INFO:tasks.workunit.client.0.smithi012.stdout:agetty 5173 root rtd unknown /proc/5173/root (readlink: Permission denied) 2024-04-22T13:26:48.827 INFO:tasks.workunit.client.0.smithi012.stdout:agetty 5173 root txt unknown /proc/5173/exe (readlink: Permission denied) 2024-04-22T13:26:48.827 INFO:tasks.workunit.client.0.smithi012.stdout:agetty 5173 root NOFD /proc/5173/fd (opendir: Permission denied) 2024-04-22T13:26:48.827 INFO:tasks.workunit.client.0.smithi012.stdout:agetty 5174 root cwd unknown /proc/5174/cwd (readlink: Permission denied) 2024-04-22T13:26:48.827 INFO:tasks.workunit.client.0.smithi012.stdout:agetty 5174 root rtd unknown /proc/5174/root (readlink: Permission denied) 2024-04-22T13:26:48.828 INFO:tasks.workunit.client.0.smithi012.stdout:agetty 5174 root txt unknown /proc/5174/exe (readlink: Permission denied) 2024-04-22T13:26:48.828 INFO:tasks.workunit.client.0.smithi012.stdout:agetty 5174 root NOFD /proc/5174/fd (opendir: Permission denied) 2024-04-22T13:26:48.828 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu cwd DIR 8,1 4096 2 / 2024-04-22T13:26:48.828 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu rtd DIR 8,1 4096 2 / 2024-04-22T13:26:48.828 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu txt REG 8,1 98216 18583 /usr/lib/systemd/systemd 2024-04-22T13:26:48.828 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu DEL REG 8,1 596 /etc/selinux/targeted/contexts/files/file_contexts.bin 2024-04-22T13:26:48.828 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-22T13:26:48.828 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-04-22T13:26:48.828 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-04-22T13:26:48.828 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-22T13:26:48.828 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu mem REG 8,1 1714208 7271 /usr/lib64/libp11-kit.so.0.3.1 2024-04-22T13:26:48.828 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-22T13:26:48.828 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu DEL REG 8,1 62167 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2024-04-22T13:26:48.828 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-04-22T13:26:48.828 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-04-22T13:26:48.828 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu mem REG 8,1 28552 7287 /usr/lib64/libattr.so.1.1.2501 2024-04-22T13:26:48.829 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-22T13:26:48.829 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-04-22T13:26:48.829 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-04-22T13:26:48.829 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-04-22T13:26:48.829 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu mem REG 8,1 3785856 46432 /usr/lib64/systemd/libsystemd-shared-252.so 2024-04-22T13:26:48.829 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu mem REG 8,1 40424 8439 /usr/lib64/libeconf.so.0.4.1 2024-04-22T13:26:48.829 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-04-22T13:26:48.829 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-04-22T13:26:48.829 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-04-22T13:26:48.829 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu mem REG 8,1 2157904 46429 /usr/lib64/systemd/libsystemd-core-252.so 2024-04-22T13:26:48.829 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-04-22T13:26:48.829 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu mem REG 8,1 40496 7288 /usr/lib64/libacl.so.1.1.2301 2024-04-22T13:26:48.829 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu mem REG 8,1 291512 7639 /usr/lib64/libmount.so.1.1.0 2024-04-22T13:26:48.829 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-22T13:26:48.829 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu mem REG 8,1 107424 7663 /usr/lib64/libkmod.so.2.3.6 2024-04-22T13:26:48.829 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-04-22T13:26:48.829 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu mem REG 8,1 69592 8448 /usr/lib64/libpam.so.0.85.1 2024-04-22T13:26:48.830 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu mem REG 8,1 127680 7455 /usr/lib64/libseccomp.so.2.5.2 2024-04-22T13:26:48.830 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-04-22T13:26:48.830 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-22T13:26:48.830 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu mem REG 0,19 0 19 /sys/fs/selinux/status 2024-04-22T13:26:48.830 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu 0r CHR 1,3 0t0 4 /dev/null 2024-04-22T13:26:48.830 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu 1u unix 0x0000000000000000 0t0 24213 type=STREAM (CONNECTED) 2024-04-22T13:26:48.830 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu 2u unix 0x0000000000000000 0t0 24213 type=STREAM (CONNECTED) 2024-04-22T13:26:48.830 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu 3u unix 0x0000000000000000 0t0 24241 type=DGRAM (CONNECTED) 2024-04-22T13:26:48.830 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu 4u a_inode 0,14 0 2081 [eventpoll:5,6,8,9,11,12,14,15,16,17,19,20,21] 2024-04-22T13:26:48.830 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu 5u a_inode 0,14 0 2081 [signalfd] 2024-04-22T13:26:48.830 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu 6r a_inode 0,14 0 2081 inotify 2024-04-22T13:26:48.830 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu 7r DIR 0,25 0 3933 /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service 2024-04-22T13:26:48.830 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu 8u a_inode 0,14 0 2081 [timerfd] 2024-04-22T13:26:48.830 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu 9u a_inode 0,14 0 2081 [eventpoll:10,13] 2024-04-22T13:26:48.830 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu 10r REG 0,20 0 24230 /proc/5367/mountinfo 2024-04-22T13:26:48.831 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu 11r a_inode 0,14 0 2081 inotify 2024-04-22T13:26:48.831 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu 12u unix 0x0000000000000000 0t0 24263 /run/user/1000/bus type=STREAM (LISTEN) 2024-04-22T13:26:48.831 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu 13r a_inode 0,14 0 2081 inotify 2024-04-22T13:26:48.831 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu 14r REG 0,20 0 4026532076 /proc/swaps 2024-04-22T13:26:48.831 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu 15u netlink 0t0 24245 KOBJECT_UEVENT 2024-04-22T13:26:48.831 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu 16u unix 0x0000000000000000 0t0 24250 /run/user/1000/systemd/notify type=DGRAM (UNCONNECTED) 2024-04-22T13:26:48.831 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu 17u unix 0x0000000000000000 0t0 24251 type=DGRAM (CONNECTED) 2024-04-22T13:26:48.831 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu 18u unix 0x0000000000000000 0t0 24252 type=DGRAM (CONNECTED) 2024-04-22T13:26:48.831 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu 19u unix 0x0000000000000000 0t0 24253 /run/user/1000/systemd/private type=STREAM (LISTEN) 2024-04-22T13:26:48.831 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu 20u unix 0x0000000000000000 0t0 24255 type=STREAM (CONNECTED) 2024-04-22T13:26:48.831 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 5367 ubuntu 21u a_inode 0,14 0 2081 [timerfd] 2024-04-22T13:26:48.831 INFO:tasks.workunit.client.0.smithi012.stdout:(sd-pam) 5369 ubuntu cwd unknown /proc/5369/cwd (readlink: Permission denied) 2024-04-22T13:26:48.831 INFO:tasks.workunit.client.0.smithi012.stdout:(sd-pam) 5369 ubuntu rtd unknown /proc/5369/root (readlink: Permission denied) 2024-04-22T13:26:48.831 INFO:tasks.workunit.client.0.smithi012.stdout:(sd-pam) 5369 ubuntu txt unknown /proc/5369/exe (readlink: Permission denied) 2024-04-22T13:26:48.831 INFO:tasks.workunit.client.0.smithi012.stdout:(sd-pam) 5369 ubuntu NOFD /proc/5369/fd (opendir: Permission denied) 2024-04-22T13:26:48.831 INFO:tasks.workunit.client.0.smithi012.stdout:tls-strp 5552 root cwd unknown /proc/5552/cwd (readlink: Permission denied) 2024-04-22T13:26:48.832 INFO:tasks.workunit.client.0.smithi012.stdout:tls-strp 5552 root rtd unknown /proc/5552/root (readlink: Permission denied) 2024-04-22T13:26:48.832 INFO:tasks.workunit.client.0.smithi012.stdout:tls-strp 5552 root txt unknown /proc/5552/exe (readlink: Permission denied) 2024-04-22T13:26:48.832 INFO:tasks.workunit.client.0.smithi012.stdout:tls-strp 5552 root NOFD /proc/5552/fd (opendir: Permission denied) 2024-04-22T13:26:48.832 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 polkitd cwd unknown /proc/5635/cwd (readlink: Permission denied) 2024-04-22T13:26:48.832 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 polkitd rtd unknown /proc/5635/root (readlink: Permission denied) 2024-04-22T13:26:48.832 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 polkitd txt unknown /proc/5635/exe (readlink: Permission denied) 2024-04-22T13:26:48.832 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 polkitd NOFD /proc/5635/fd (opendir: Permission denied) 2024-04-22T13:26:48.832 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5637 gmain polkitd cwd unknown /proc/5635/task/5637/cwd (readlink: Permission denied) 2024-04-22T13:26:48.832 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5637 gmain polkitd rtd unknown /proc/5635/task/5637/root (readlink: Permission denied) 2024-04-22T13:26:48.832 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5637 gmain polkitd txt unknown /proc/5635/task/5637/exe (readlink: Permission denied) 2024-04-22T13:26:48.832 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5637 gmain polkitd NOFD /proc/5635/task/5637/fd (opendir: Permission denied) 2024-04-22T13:26:48.832 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5638 gdbus polkitd cwd unknown /proc/5635/task/5638/cwd (readlink: Permission denied) 2024-04-22T13:26:48.832 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5638 gdbus polkitd rtd unknown /proc/5635/task/5638/root (readlink: Permission denied) 2024-04-22T13:26:48.832 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5638 gdbus polkitd txt unknown /proc/5635/task/5638/exe (readlink: Permission denied) 2024-04-22T13:26:48.832 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5638 gdbus polkitd NOFD /proc/5635/task/5638/fd (opendir: Permission denied) 2024-04-22T13:26:48.833 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5639 JS\x20Hel polkitd cwd unknown /proc/5635/task/5639/cwd (readlink: Permission denied) 2024-04-22T13:26:48.833 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5639 JS\x20Hel polkitd rtd unknown /proc/5635/task/5639/root (readlink: Permission denied) 2024-04-22T13:26:48.833 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5639 JS\x20Hel polkitd txt unknown /proc/5635/task/5639/exe (readlink: Permission denied) 2024-04-22T13:26:48.833 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5639 JS\x20Hel polkitd NOFD /proc/5635/task/5639/fd (opendir: Permission denied) 2024-04-22T13:26:48.833 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5640 JS\x20Hel polkitd cwd unknown /proc/5635/task/5640/cwd (readlink: Permission denied) 2024-04-22T13:26:48.833 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5640 JS\x20Hel polkitd rtd unknown /proc/5635/task/5640/root (readlink: Permission denied) 2024-04-22T13:26:48.833 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5640 JS\x20Hel polkitd txt unknown /proc/5635/task/5640/exe (readlink: Permission denied) 2024-04-22T13:26:48.833 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5640 JS\x20Hel polkitd NOFD /proc/5635/task/5640/fd (opendir: Permission denied) 2024-04-22T13:26:48.833 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5641 JS\x20Hel polkitd cwd unknown /proc/5635/task/5641/cwd (readlink: Permission denied) 2024-04-22T13:26:48.833 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5641 JS\x20Hel polkitd rtd unknown /proc/5635/task/5641/root (readlink: Permission denied) 2024-04-22T13:26:48.833 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5641 JS\x20Hel polkitd txt unknown /proc/5635/task/5641/exe (readlink: Permission denied) 2024-04-22T13:26:48.833 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5641 JS\x20Hel polkitd NOFD /proc/5635/task/5641/fd (opendir: Permission denied) 2024-04-22T13:26:48.833 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5642 JS\x20Hel polkitd cwd unknown /proc/5635/task/5642/cwd (readlink: Permission denied) 2024-04-22T13:26:48.833 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5642 JS\x20Hel polkitd rtd unknown /proc/5635/task/5642/root (readlink: Permission denied) 2024-04-22T13:26:48.833 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5642 JS\x20Hel polkitd txt unknown /proc/5635/task/5642/exe (readlink: Permission denied) 2024-04-22T13:26:48.834 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5642 JS\x20Hel polkitd NOFD /proc/5635/task/5642/fd (opendir: Permission denied) 2024-04-22T13:26:48.834 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5643 JS\x20Hel polkitd cwd unknown /proc/5635/task/5643/cwd (readlink: Permission denied) 2024-04-22T13:26:48.834 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5643 JS\x20Hel polkitd rtd unknown /proc/5635/task/5643/root (readlink: Permission denied) 2024-04-22T13:26:48.834 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5643 JS\x20Hel polkitd txt unknown /proc/5635/task/5643/exe (readlink: Permission denied) 2024-04-22T13:26:48.834 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5643 JS\x20Hel polkitd NOFD /proc/5635/task/5643/fd (opendir: Permission denied) 2024-04-22T13:26:48.834 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5644 JS\x20Hel polkitd cwd unknown /proc/5635/task/5644/cwd (readlink: Permission denied) 2024-04-22T13:26:48.834 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5644 JS\x20Hel polkitd rtd unknown /proc/5635/task/5644/root (readlink: Permission denied) 2024-04-22T13:26:48.834 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5644 JS\x20Hel polkitd txt unknown /proc/5635/task/5644/exe (readlink: Permission denied) 2024-04-22T13:26:48.834 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5644 JS\x20Hel polkitd NOFD /proc/5635/task/5644/fd (opendir: Permission denied) 2024-04-22T13:26:48.834 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5645 JS\x20Hel polkitd cwd unknown /proc/5635/task/5645/cwd (readlink: Permission denied) 2024-04-22T13:26:48.834 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5645 JS\x20Hel polkitd rtd unknown /proc/5635/task/5645/root (readlink: Permission denied) 2024-04-22T13:26:48.834 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5645 JS\x20Hel polkitd txt unknown /proc/5635/task/5645/exe (readlink: Permission denied) 2024-04-22T13:26:48.834 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5645 JS\x20Hel polkitd NOFD /proc/5635/task/5645/fd (opendir: Permission denied) 2024-04-22T13:26:48.834 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5646 JS\x20Hel polkitd cwd unknown /proc/5635/task/5646/cwd (readlink: Permission denied) 2024-04-22T13:26:48.835 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5646 JS\x20Hel polkitd rtd unknown /proc/5635/task/5646/root (readlink: Permission denied) 2024-04-22T13:26:48.835 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5646 JS\x20Hel polkitd txt unknown /proc/5635/task/5646/exe (readlink: Permission denied) 2024-04-22T13:26:48.835 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5646 JS\x20Hel polkitd NOFD /proc/5635/task/5646/fd (opendir: Permission denied) 2024-04-22T13:26:48.835 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5647 polkitd polkitd cwd unknown /proc/5635/task/5647/cwd (readlink: Permission denied) 2024-04-22T13:26:48.835 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5647 polkitd polkitd rtd unknown /proc/5635/task/5647/root (readlink: Permission denied) 2024-04-22T13:26:48.835 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5647 polkitd polkitd txt unknown /proc/5635/task/5647/exe (readlink: Permission denied) 2024-04-22T13:26:48.835 INFO:tasks.workunit.client.0.smithi012.stdout:polkitd 5635 5647 polkitd polkitd NOFD /proc/5635/task/5647/fd (opendir: Permission denied) 2024-04-22T13:26:48.835 INFO:tasks.workunit.client.0.smithi012.stdout:rsyslogd 5865 root cwd unknown /proc/5865/cwd (readlink: Permission denied) 2024-04-22T13:26:48.835 INFO:tasks.workunit.client.0.smithi012.stdout:rsyslogd 5865 root rtd unknown /proc/5865/root (readlink: Permission denied) 2024-04-22T13:26:48.835 INFO:tasks.workunit.client.0.smithi012.stdout:rsyslogd 5865 root txt unknown /proc/5865/exe (readlink: Permission denied) 2024-04-22T13:26:48.835 INFO:tasks.workunit.client.0.smithi012.stdout:rsyslogd 5865 root NOFD /proc/5865/fd (opendir: Permission denied) 2024-04-22T13:26:48.835 INFO:tasks.workunit.client.0.smithi012.stdout:rsyslogd 5865 5867 in:imjour root cwd unknown /proc/5865/task/5867/cwd (readlink: Permission denied) 2024-04-22T13:26:48.835 INFO:tasks.workunit.client.0.smithi012.stdout:rsyslogd 5865 5867 in:imjour root rtd unknown /proc/5865/task/5867/root (readlink: Permission denied) 2024-04-22T13:26:48.835 INFO:tasks.workunit.client.0.smithi012.stdout:rsyslogd 5865 5867 in:imjour root txt unknown /proc/5865/task/5867/exe (readlink: Permission denied) 2024-04-22T13:26:48.835 INFO:tasks.workunit.client.0.smithi012.stdout:rsyslogd 5865 5867 in:imjour root NOFD /proc/5865/task/5867/fd (opendir: Permission denied) 2024-04-22T13:26:48.835 INFO:tasks.workunit.client.0.smithi012.stdout:rsyslogd 5865 5868 rs:main root cwd unknown /proc/5865/task/5868/cwd (readlink: Permission denied) 2024-04-22T13:26:48.836 INFO:tasks.workunit.client.0.smithi012.stdout:rsyslogd 5865 5868 rs:main root rtd unknown /proc/5865/task/5868/root (readlink: Permission denied) 2024-04-22T13:26:48.836 INFO:tasks.workunit.client.0.smithi012.stdout:rsyslogd 5865 5868 rs:main root txt unknown /proc/5865/task/5868/exe (readlink: Permission denied) 2024-04-22T13:26:48.836 INFO:tasks.workunit.client.0.smithi012.stdout:rsyslogd 5865 5868 rs:main root NOFD /proc/5865/task/5868/fd (opendir: Permission denied) 2024-04-22T13:26:48.836 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 19126 root cwd unknown /proc/19126/cwd (readlink: Permission denied) 2024-04-22T13:26:48.836 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 19126 root rtd unknown /proc/19126/root (readlink: Permission denied) 2024-04-22T13:26:48.836 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 19126 root txt unknown /proc/19126/exe (readlink: Permission denied) 2024-04-22T13:26:48.836 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 19126 root NOFD /proc/19126/fd (opendir: Permission denied) 2024-04-22T13:26:48.836 INFO:tasks.workunit.client.0.smithi012.stdout:kdmflush/ 19265 root cwd unknown /proc/19265/cwd (readlink: Permission denied) 2024-04-22T13:26:48.836 INFO:tasks.workunit.client.0.smithi012.stdout:kdmflush/ 19265 root rtd unknown /proc/19265/root (readlink: Permission denied) 2024-04-22T13:26:48.836 INFO:tasks.workunit.client.0.smithi012.stdout:kdmflush/ 19265 root txt unknown /proc/19265/exe (readlink: Permission denied) 2024-04-22T13:26:48.836 INFO:tasks.workunit.client.0.smithi012.stdout:kdmflush/ 19265 root NOFD /proc/19265/fd (opendir: Permission denied) 2024-04-22T13:26:48.836 INFO:tasks.workunit.client.0.smithi012.stdout:kdmflush/ 19294 root cwd unknown /proc/19294/cwd (readlink: Permission denied) 2024-04-22T13:26:48.836 INFO:tasks.workunit.client.0.smithi012.stdout:kdmflush/ 19294 root rtd unknown /proc/19294/root (readlink: Permission denied) 2024-04-22T13:26:48.836 INFO:tasks.workunit.client.0.smithi012.stdout:kdmflush/ 19294 root txt unknown /proc/19294/exe (readlink: Permission denied) 2024-04-22T13:26:48.836 INFO:tasks.workunit.client.0.smithi012.stdout:kdmflush/ 19294 root NOFD /proc/19294/fd (opendir: Permission denied) 2024-04-22T13:26:48.836 INFO:tasks.workunit.client.0.smithi012.stdout:kdmflush/ 19323 root cwd unknown /proc/19323/cwd (readlink: Permission denied) 2024-04-22T13:26:48.836 INFO:tasks.workunit.client.0.smithi012.stdout:kdmflush/ 19323 root rtd unknown /proc/19323/root (readlink: Permission denied) 2024-04-22T13:26:48.837 INFO:tasks.workunit.client.0.smithi012.stdout:kdmflush/ 19323 root txt unknown /proc/19323/exe (readlink: Permission denied) 2024-04-22T13:26:48.837 INFO:tasks.workunit.client.0.smithi012.stdout:kdmflush/ 19323 root NOFD /proc/19323/fd (opendir: Permission denied) 2024-04-22T13:26:48.837 INFO:tasks.workunit.client.0.smithi012.stdout:kdmflush/ 19352 root cwd unknown /proc/19352/cwd (readlink: Permission denied) 2024-04-22T13:26:48.837 INFO:tasks.workunit.client.0.smithi012.stdout:kdmflush/ 19352 root rtd unknown /proc/19352/root (readlink: Permission denied) 2024-04-22T13:26:48.837 INFO:tasks.workunit.client.0.smithi012.stdout:kdmflush/ 19352 root txt unknown /proc/19352/exe (readlink: Permission denied) 2024-04-22T13:26:48.837 INFO:tasks.workunit.client.0.smithi012.stdout:kdmflush/ 19352 root NOFD /proc/19352/fd (opendir: Permission denied) 2024-04-22T13:26:48.837 INFO:tasks.workunit.client.0.smithi012.stdout:kdmflush/ 19381 root cwd unknown /proc/19381/cwd (readlink: Permission denied) 2024-04-22T13:26:48.837 INFO:tasks.workunit.client.0.smithi012.stdout:kdmflush/ 19381 root rtd unknown /proc/19381/root (readlink: Permission denied) 2024-04-22T13:26:48.837 INFO:tasks.workunit.client.0.smithi012.stdout:kdmflush/ 19381 root txt unknown /proc/19381/exe (readlink: Permission denied) 2024-04-22T13:26:48.837 INFO:tasks.workunit.client.0.smithi012.stdout:kdmflush/ 19381 root NOFD /proc/19381/fd (opendir: Permission denied) 2024-04-22T13:26:48.837 INFO:tasks.workunit.client.0.smithi012.stdout:xfsalloc 19545 root cwd unknown /proc/19545/cwd (readlink: Permission denied) 2024-04-22T13:26:48.837 INFO:tasks.workunit.client.0.smithi012.stdout:xfsalloc 19545 root rtd unknown /proc/19545/root (readlink: Permission denied) 2024-04-22T13:26:48.837 INFO:tasks.workunit.client.0.smithi012.stdout:xfsalloc 19545 root txt unknown /proc/19545/exe (readlink: Permission denied) 2024-04-22T13:26:48.837 INFO:tasks.workunit.client.0.smithi012.stdout:xfsalloc 19545 root NOFD /proc/19545/fd (opendir: Permission denied) 2024-04-22T13:26:48.837 INFO:tasks.workunit.client.0.smithi012.stdout:xfs_mru_c 19546 root cwd unknown /proc/19546/cwd (readlink: Permission denied) 2024-04-22T13:26:48.837 INFO:tasks.workunit.client.0.smithi012.stdout:xfs_mru_c 19546 root rtd unknown /proc/19546/root (readlink: Permission denied) 2024-04-22T13:26:48.838 INFO:tasks.workunit.client.0.smithi012.stdout:xfs_mru_c 19546 root txt unknown /proc/19546/exe (readlink: Permission denied) 2024-04-22T13:26:48.838 INFO:tasks.workunit.client.0.smithi012.stdout:xfs_mru_c 19546 root NOFD /proc/19546/fd (opendir: Permission denied) 2024-04-22T13:26:48.838 INFO:tasks.workunit.client.0.smithi012.stdout:xfs-buf/d 19547 root cwd unknown /proc/19547/cwd (readlink: Permission denied) 2024-04-22T13:26:48.838 INFO:tasks.workunit.client.0.smithi012.stdout:xfs-buf/d 19547 root rtd unknown /proc/19547/root (readlink: Permission denied) 2024-04-22T13:26:48.838 INFO:tasks.workunit.client.0.smithi012.stdout:xfs-buf/d 19547 root txt unknown /proc/19547/exe (readlink: Permission denied) 2024-04-22T13:26:48.838 INFO:tasks.workunit.client.0.smithi012.stdout:xfs-buf/d 19547 root NOFD /proc/19547/fd (opendir: Permission denied) 2024-04-22T13:26:48.838 INFO:tasks.workunit.client.0.smithi012.stdout:xfs-conv/ 19548 root cwd unknown /proc/19548/cwd (readlink: Permission denied) 2024-04-22T13:26:48.838 INFO:tasks.workunit.client.0.smithi012.stdout:xfs-conv/ 19548 root rtd unknown /proc/19548/root (readlink: Permission denied) 2024-04-22T13:26:48.838 INFO:tasks.workunit.client.0.smithi012.stdout:xfs-conv/ 19548 root txt unknown /proc/19548/exe (readlink: Permission denied) 2024-04-22T13:26:48.838 INFO:tasks.workunit.client.0.smithi012.stdout:xfs-conv/ 19548 root NOFD /proc/19548/fd (opendir: Permission denied) 2024-04-22T13:26:48.838 INFO:tasks.workunit.client.0.smithi012.stdout:xfs-recla 19549 root cwd unknown /proc/19549/cwd (readlink: Permission denied) 2024-04-22T13:26:48.838 INFO:tasks.workunit.client.0.smithi012.stdout:xfs-recla 19549 root rtd unknown /proc/19549/root (readlink: Permission denied) 2024-04-22T13:26:48.838 INFO:tasks.workunit.client.0.smithi012.stdout:xfs-recla 19549 root txt unknown /proc/19549/exe (readlink: Permission denied) 2024-04-22T13:26:48.838 INFO:tasks.workunit.client.0.smithi012.stdout:xfs-recla 19549 root NOFD /proc/19549/fd (opendir: Permission denied) 2024-04-22T13:26:48.838 INFO:tasks.workunit.client.0.smithi012.stdout:xfs-block 19550 root cwd unknown /proc/19550/cwd (readlink: Permission denied) 2024-04-22T13:26:48.839 INFO:tasks.workunit.client.0.smithi012.stdout:xfs-block 19550 root rtd unknown /proc/19550/root (readlink: Permission denied) 2024-04-22T13:26:48.839 INFO:tasks.workunit.client.0.smithi012.stdout:xfs-block 19550 root txt unknown /proc/19550/exe (readlink: Permission denied) 2024-04-22T13:26:48.839 INFO:tasks.workunit.client.0.smithi012.stdout:xfs-block 19550 root NOFD /proc/19550/fd (opendir: Permission denied) 2024-04-22T13:26:48.839 INFO:tasks.workunit.client.0.smithi012.stdout:xfs-inode 19551 root cwd unknown /proc/19551/cwd (readlink: Permission denied) 2024-04-22T13:26:48.839 INFO:tasks.workunit.client.0.smithi012.stdout:xfs-inode 19551 root rtd unknown /proc/19551/root (readlink: Permission denied) 2024-04-22T13:26:48.839 INFO:tasks.workunit.client.0.smithi012.stdout:xfs-inode 19551 root txt unknown /proc/19551/exe (readlink: Permission denied) 2024-04-22T13:26:48.839 INFO:tasks.workunit.client.0.smithi012.stdout:xfs-inode 19551 root NOFD /proc/19551/fd (opendir: Permission denied) 2024-04-22T13:26:48.839 INFO:tasks.workunit.client.0.smithi012.stdout:xfs-log/d 19552 root cwd unknown /proc/19552/cwd (readlink: Permission denied) 2024-04-22T13:26:48.839 INFO:tasks.workunit.client.0.smithi012.stdout:xfs-log/d 19552 root rtd unknown /proc/19552/root (readlink: Permission denied) 2024-04-22T13:26:48.839 INFO:tasks.workunit.client.0.smithi012.stdout:xfs-log/d 19552 root txt unknown /proc/19552/exe (readlink: Permission denied) 2024-04-22T13:26:48.839 INFO:tasks.workunit.client.0.smithi012.stdout:xfs-log/d 19552 root NOFD /proc/19552/fd (opendir: Permission denied) 2024-04-22T13:26:48.839 INFO:tasks.workunit.client.0.smithi012.stdout:xfs-cil/d 19553 root cwd unknown /proc/19553/cwd (readlink: Permission denied) 2024-04-22T13:26:48.839 INFO:tasks.workunit.client.0.smithi012.stdout:xfs-cil/d 19553 root rtd unknown /proc/19553/root (readlink: Permission denied) 2024-04-22T13:26:48.839 INFO:tasks.workunit.client.0.smithi012.stdout:xfs-cil/d 19553 root txt unknown /proc/19553/exe (readlink: Permission denied) 2024-04-22T13:26:48.839 INFO:tasks.workunit.client.0.smithi012.stdout:xfs-cil/d 19553 root NOFD /proc/19553/fd (opendir: Permission denied) 2024-04-22T13:26:48.839 INFO:tasks.workunit.client.0.smithi012.stdout:xfsaild/d 19554 root cwd unknown /proc/19554/cwd (readlink: Permission denied) 2024-04-22T13:26:48.839 INFO:tasks.workunit.client.0.smithi012.stdout:xfsaild/d 19554 root rtd unknown /proc/19554/root (readlink: Permission denied) 2024-04-22T13:26:48.840 INFO:tasks.workunit.client.0.smithi012.stdout:xfsaild/d 19554 root txt unknown /proc/19554/exe (readlink: Permission denied) 2024-04-22T13:26:48.840 INFO:tasks.workunit.client.0.smithi012.stdout:xfsaild/d 19554 root NOFD /proc/19554/fd (opendir: Permission denied) 2024-04-22T13:26:48.840 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/4 19672 root cwd unknown /proc/19672/cwd (readlink: Permission denied) 2024-04-22T13:26:48.840 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/4 19672 root rtd unknown /proc/19672/root (readlink: Permission denied) 2024-04-22T13:26:48.840 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/4 19672 root txt unknown /proc/19672/exe (readlink: Permission denied) 2024-04-22T13:26:48.840 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/4 19672 root NOFD /proc/19672/fd (opendir: Permission denied) 2024-04-22T13:26:48.840 INFO:tasks.workunit.client.0.smithi012.stdout:sshd 20484 root cwd unknown /proc/20484/cwd (readlink: Permission denied) 2024-04-22T13:26:48.840 INFO:tasks.workunit.client.0.smithi012.stdout:sshd 20484 root rtd unknown /proc/20484/root (readlink: Permission denied) 2024-04-22T13:26:48.840 INFO:tasks.workunit.client.0.smithi012.stdout:sshd 20484 root txt unknown /proc/20484/exe (readlink: Permission denied) 2024-04-22T13:26:48.840 INFO:tasks.workunit.client.0.smithi012.stdout:sshd 20484 root NOFD /proc/20484/fd (opendir: Permission denied) 2024-04-22T13:26:48.840 INFO:tasks.workunit.client.0.smithi012.stdout:sshd 20487 ubuntu cwd unknown /proc/20487/cwd (readlink: Permission denied) 2024-04-22T13:26:48.840 INFO:tasks.workunit.client.0.smithi012.stdout:sshd 20487 ubuntu rtd unknown /proc/20487/root (readlink: Permission denied) 2024-04-22T13:26:48.840 INFO:tasks.workunit.client.0.smithi012.stdout:sshd 20487 ubuntu txt unknown /proc/20487/exe (readlink: Permission denied) 2024-04-22T13:26:48.840 INFO:tasks.workunit.client.0.smithi012.stdout:sshd 20487 ubuntu NOFD /proc/20487/fd (opendir: Permission denied) 2024-04-22T13:26:48.840 INFO:tasks.workunit.client.0.smithi012.stdout:chronyd 20547 chrony cwd unknown /proc/20547/cwd (readlink: Permission denied) 2024-04-22T13:26:48.840 INFO:tasks.workunit.client.0.smithi012.stdout:chronyd 20547 chrony rtd unknown /proc/20547/root (readlink: Permission denied) 2024-04-22T13:26:48.841 INFO:tasks.workunit.client.0.smithi012.stdout:chronyd 20547 chrony txt unknown /proc/20547/exe (readlink: Permission denied) 2024-04-22T13:26:48.841 INFO:tasks.workunit.client.0.smithi012.stdout:chronyd 20547 chrony NOFD /proc/20547/fd (opendir: Permission denied) 2024-04-22T13:26:48.841 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/4 20729 root cwd unknown /proc/20729/cwd (readlink: Permission denied) 2024-04-22T13:26:48.841 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/4 20729 root rtd unknown /proc/20729/root (readlink: Permission denied) 2024-04-22T13:26:48.841 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/4 20729 root txt unknown /proc/20729/exe (readlink: Permission denied) 2024-04-22T13:26:48.841 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/4 20729 root NOFD /proc/20729/fd (opendir: Permission denied) 2024-04-22T13:26:48.841 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-cras 20846 ceph cwd unknown /proc/20846/cwd (readlink: Permission denied) 2024-04-22T13:26:48.841 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-cras 20846 ceph rtd unknown /proc/20846/root (readlink: Permission denied) 2024-04-22T13:26:48.841 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-cras 20846 ceph txt unknown /proc/20846/exe (readlink: Permission denied) 2024-04-22T13:26:48.841 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-cras 20846 ceph NOFD /proc/20846/fd (opendir: Permission denied) 2024-04-22T13:26:48.841 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 21480 root cwd unknown /proc/21480/cwd (readlink: Permission denied) 2024-04-22T13:26:48.841 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 21480 root rtd unknown /proc/21480/root (readlink: Permission denied) 2024-04-22T13:26:48.841 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 21480 root txt unknown /proc/21480/exe (readlink: Permission denied) 2024-04-22T13:26:48.841 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/u 21480 root NOFD /proc/21480/fd (opendir: Permission denied) 2024-04-22T13:26:48.841 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/1 21957 root cwd unknown /proc/21957/cwd (readlink: Permission denied) 2024-04-22T13:26:48.841 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/1 21957 root rtd unknown /proc/21957/root (readlink: Permission denied) 2024-04-22T13:26:48.842 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/1 21957 root txt unknown /proc/21957/exe (readlink: Permission denied) 2024-04-22T13:26:48.842 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/1 21957 root NOFD /proc/21957/fd (opendir: Permission denied) 2024-04-22T13:26:48.842 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/0 21972 root cwd unknown /proc/21972/cwd (readlink: Permission denied) 2024-04-22T13:26:48.842 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/0 21972 root rtd unknown /proc/21972/root (readlink: Permission denied) 2024-04-22T13:26:48.842 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/0 21972 root txt unknown /proc/21972/exe (readlink: Permission denied) 2024-04-22T13:26:48.842 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/0 21972 root NOFD /proc/21972/fd (opendir: Permission denied) 2024-04-22T13:26:48.842 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 27241 root cwd unknown /proc/27241/cwd (readlink: Permission denied) 2024-04-22T13:26:48.842 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 27241 root rtd unknown /proc/27241/root (readlink: Permission denied) 2024-04-22T13:26:48.842 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 27241 root txt unknown /proc/27241/exe (readlink: Permission denied) 2024-04-22T13:26:48.842 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 27241 root NOFD /proc/27241/fd (opendir: Permission denied) 2024-04-22T13:26:48.842 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 27258 root cwd unknown /proc/27258/cwd (readlink: Permission denied) 2024-04-22T13:26:48.842 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 27258 root rtd unknown /proc/27258/root (readlink: Permission denied) 2024-04-22T13:26:48.842 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 27258 root txt unknown /proc/27258/exe (readlink: Permission denied) 2024-04-22T13:26:48.842 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 27258 root NOFD /proc/27258/fd (opendir: Permission denied) 2024-04-22T13:26:48.842 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 27259 root cwd unknown /proc/27259/cwd (readlink: Permission denied) 2024-04-22T13:26:48.842 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 27259 root rtd unknown /proc/27259/root (readlink: Permission denied) 2024-04-22T13:26:48.842 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 27259 root txt unknown /proc/27259/exe (readlink: Permission denied) 2024-04-22T13:26:48.843 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 27259 root NOFD /proc/27259/fd (opendir: Permission denied) 2024-04-22T13:26:48.843 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 27276 root cwd unknown /proc/27276/cwd (readlink: Permission denied) 2024-04-22T13:26:48.843 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 27276 root rtd unknown /proc/27276/root (readlink: Permission denied) 2024-04-22T13:26:48.843 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 27276 root txt unknown /proc/27276/exe (readlink: Permission denied) 2024-04-22T13:26:48.843 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 27276 root NOFD /proc/27276/fd (opendir: Permission denied) 2024-04-22T13:26:48.843 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/6 28206 root cwd unknown /proc/28206/cwd (readlink: Permission denied) 2024-04-22T13:26:48.843 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/6 28206 root rtd unknown /proc/28206/root (readlink: Permission denied) 2024-04-22T13:26:48.843 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/6 28206 root txt unknown /proc/28206/exe (readlink: Permission denied) 2024-04-22T13:26:48.843 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/6 28206 root NOFD /proc/28206/fd (opendir: Permission denied) 2024-04-22T13:26:48.843 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 28261 root cwd unknown /proc/28261/cwd (readlink: Permission denied) 2024-04-22T13:26:48.843 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 28261 root rtd unknown /proc/28261/root (readlink: Permission denied) 2024-04-22T13:26:48.843 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 28261 root txt unknown /proc/28261/exe (readlink: Permission denied) 2024-04-22T13:26:48.843 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 28261 root NOFD /proc/28261/fd (opendir: Permission denied) 2024-04-22T13:26:48.843 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 28263 root cwd unknown /proc/28263/cwd (readlink: Permission denied) 2024-04-22T13:26:48.843 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 28263 root rtd unknown /proc/28263/root (readlink: Permission denied) 2024-04-22T13:26:48.843 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 28263 root txt unknown /proc/28263/exe (readlink: Permission denied) 2024-04-22T13:26:48.844 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 28263 root NOFD /proc/28263/fd (opendir: Permission denied) 2024-04-22T13:26:48.844 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 ceph cwd unknown /proc/28265/cwd (readlink: Permission denied) 2024-04-22T13:26:48.844 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 ceph rtd unknown /proc/28265/root (readlink: Permission denied) 2024-04-22T13:26:48.844 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 ceph txt unknown /proc/28265/exe (readlink: Permission denied) 2024-04-22T13:26:48.844 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 ceph NOFD /proc/28265/fd (opendir: Permission denied) 2024-04-22T13:26:48.844 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28266 log ceph cwd unknown /proc/28265/task/28266/cwd (readlink: Permission denied) 2024-04-22T13:26:48.844 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28266 log ceph rtd unknown /proc/28265/task/28266/root (readlink: Permission denied) 2024-04-22T13:26:48.844 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28266 log ceph txt unknown /proc/28265/task/28266/exe (readlink: Permission denied) 2024-04-22T13:26:48.844 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28266 log ceph NOFD /proc/28265/task/28266/fd (opendir: Permission denied) 2024-04-22T13:26:48.844 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28267 service ceph cwd unknown /proc/28265/task/28267/cwd (readlink: Permission denied) 2024-04-22T13:26:48.844 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28267 service ceph rtd unknown /proc/28265/task/28267/root (readlink: Permission denied) 2024-04-22T13:26:48.844 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28267 service ceph txt unknown /proc/28265/task/28267/exe (readlink: Permission denied) 2024-04-22T13:26:48.844 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28267 service ceph NOFD /proc/28265/task/28267/fd (opendir: Permission denied) 2024-04-22T13:26:48.844 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28268 admin_soc ceph cwd unknown /proc/28265/task/28268/cwd (readlink: Permission denied) 2024-04-22T13:26:48.844 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28268 admin_soc ceph rtd unknown /proc/28265/task/28268/root (readlink: Permission denied) 2024-04-22T13:26:48.844 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28268 admin_soc ceph txt unknown /proc/28265/task/28268/exe (readlink: Permission denied) 2024-04-22T13:26:48.845 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28268 admin_soc ceph NOFD /proc/28265/task/28268/fd (opendir: Permission denied) 2024-04-22T13:26:48.845 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28269 signal_ha ceph cwd unknown /proc/28265/task/28269/cwd (readlink: Permission denied) 2024-04-22T13:26:48.845 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28269 signal_ha ceph rtd unknown /proc/28265/task/28269/root (readlink: Permission denied) 2024-04-22T13:26:48.845 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28269 signal_ha ceph txt unknown /proc/28265/task/28269/exe (readlink: Permission denied) 2024-04-22T13:26:48.845 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28269 signal_ha ceph NOFD /proc/28265/task/28269/fd (opendir: Permission denied) 2024-04-22T13:26:48.845 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28270 rocksdb:l ceph cwd unknown /proc/28265/task/28270/cwd (readlink: Permission denied) 2024-04-22T13:26:48.845 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28270 rocksdb:l ceph rtd unknown /proc/28265/task/28270/root (readlink: Permission denied) 2024-04-22T13:26:48.845 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28270 rocksdb:l ceph txt unknown /proc/28265/task/28270/exe (readlink: Permission denied) 2024-04-22T13:26:48.845 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28270 rocksdb:l ceph NOFD /proc/28265/task/28270/fd (opendir: Permission denied) 2024-04-22T13:26:48.845 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28271 rocksdb:h ceph cwd unknown /proc/28265/task/28271/cwd (readlink: Permission denied) 2024-04-22T13:26:48.845 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28271 rocksdb:h ceph rtd unknown /proc/28265/task/28271/root (readlink: Permission denied) 2024-04-22T13:26:48.845 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28271 rocksdb:h ceph txt unknown /proc/28265/task/28271/exe (readlink: Permission denied) 2024-04-22T13:26:48.845 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28271 rocksdb:h ceph NOFD /proc/28265/task/28271/fd (opendir: Permission denied) 2024-04-22T13:26:48.845 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28287 ceph-mon ceph cwd unknown /proc/28265/task/28287/cwd (readlink: Permission denied) 2024-04-22T13:26:48.845 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28287 ceph-mon ceph rtd unknown /proc/28265/task/28287/root (readlink: Permission denied) 2024-04-22T13:26:48.845 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28287 ceph-mon ceph txt unknown /proc/28265/task/28287/exe (readlink: Permission denied) 2024-04-22T13:26:48.845 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28287 ceph-mon ceph NOFD /proc/28265/task/28287/fd (opendir: Permission denied) 2024-04-22T13:26:48.846 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28288 fn_monsto ceph cwd unknown /proc/28265/task/28288/cwd (readlink: Permission denied) 2024-04-22T13:26:48.846 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28288 fn_monsto ceph rtd unknown /proc/28265/task/28288/root (readlink: Permission denied) 2024-04-22T13:26:48.846 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28288 fn_monsto ceph txt unknown /proc/28265/task/28288/exe (readlink: Permission denied) 2024-04-22T13:26:48.846 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28288 fn_monsto ceph NOFD /proc/28265/task/28288/fd (opendir: Permission denied) 2024-04-22T13:26:48.846 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28289 msgr-work ceph cwd unknown /proc/28265/task/28289/cwd (readlink: Permission denied) 2024-04-22T13:26:48.846 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28289 msgr-work ceph rtd unknown /proc/28265/task/28289/root (readlink: Permission denied) 2024-04-22T13:26:48.846 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28289 msgr-work ceph txt unknown /proc/28265/task/28289/exe (readlink: Permission denied) 2024-04-22T13:26:48.846 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28289 msgr-work ceph NOFD /proc/28265/task/28289/fd (opendir: Permission denied) 2024-04-22T13:26:48.846 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28290 msgr-work ceph cwd unknown /proc/28265/task/28290/cwd (readlink: Permission denied) 2024-04-22T13:26:48.846 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28290 msgr-work ceph rtd unknown /proc/28265/task/28290/root (readlink: Permission denied) 2024-04-22T13:26:48.846 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28290 msgr-work ceph txt unknown /proc/28265/task/28290/exe (readlink: Permission denied) 2024-04-22T13:26:48.846 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28290 msgr-work ceph NOFD /proc/28265/task/28290/fd (opendir: Permission denied) 2024-04-22T13:26:48.846 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28291 msgr-work ceph cwd unknown /proc/28265/task/28291/cwd (readlink: Permission denied) 2024-04-22T13:26:48.846 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28291 msgr-work ceph rtd unknown /proc/28265/task/28291/root (readlink: Permission denied) 2024-04-22T13:26:48.846 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28291 msgr-work ceph txt unknown /proc/28265/task/28291/exe (readlink: Permission denied) 2024-04-22T13:26:48.846 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28291 msgr-work ceph NOFD /proc/28265/task/28291/fd (opendir: Permission denied) 2024-04-22T13:26:48.846 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28292 OpHistory ceph cwd unknown /proc/28265/task/28292/cwd (readlink: Permission denied) 2024-04-22T13:26:48.847 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28292 OpHistory ceph rtd unknown /proc/28265/task/28292/root (readlink: Permission denied) 2024-04-22T13:26:48.847 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28292 OpHistory ceph txt unknown /proc/28265/task/28292/exe (readlink: Permission denied) 2024-04-22T13:26:48.847 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28292 OpHistory ceph NOFD /proc/28265/task/28292/fd (opendir: Permission denied) 2024-04-22T13:26:48.847 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28293 fin ceph cwd unknown /proc/28265/task/28293/cwd (readlink: Permission denied) 2024-04-22T13:26:48.847 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28293 fin ceph rtd unknown /proc/28265/task/28293/root (readlink: Permission denied) 2024-04-22T13:26:48.847 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28293 fin ceph txt unknown /proc/28265/task/28293/exe (readlink: Permission denied) 2024-04-22T13:26:48.847 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28293 fin ceph NOFD /proc/28265/task/28293/fd (opendir: Permission denied) 2024-04-22T13:26:48.847 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28294 safe_time ceph cwd unknown /proc/28265/task/28294/cwd (readlink: Permission denied) 2024-04-22T13:26:48.847 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28294 safe_time ceph rtd unknown /proc/28265/task/28294/root (readlink: Permission denied) 2024-04-22T13:26:48.847 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28294 safe_time ceph txt unknown /proc/28265/task/28294/exe (readlink: Permission denied) 2024-04-22T13:26:48.847 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28294 safe_time ceph NOFD /proc/28265/task/28294/fd (opendir: Permission denied) 2024-04-22T13:26:48.847 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28295 cpu_tp ceph cwd unknown /proc/28265/task/28295/cwd (readlink: Permission denied) 2024-04-22T13:26:48.847 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28295 cpu_tp ceph rtd unknown /proc/28265/task/28295/root (readlink: Permission denied) 2024-04-22T13:26:48.847 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28295 cpu_tp ceph txt unknown /proc/28265/task/28295/exe (readlink: Permission denied) 2024-04-22T13:26:48.847 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28295 cpu_tp ceph NOFD /proc/28265/task/28295/fd (opendir: Permission denied) 2024-04-22T13:26:48.847 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28296 cpu_tp ceph cwd unknown /proc/28265/task/28296/cwd (readlink: Permission denied) 2024-04-22T13:26:48.848 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28296 cpu_tp ceph rtd unknown /proc/28265/task/28296/root (readlink: Permission denied) 2024-04-22T13:26:48.848 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28296 cpu_tp ceph txt unknown /proc/28265/task/28296/exe (readlink: Permission denied) 2024-04-22T13:26:48.848 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28296 cpu_tp ceph NOFD /proc/28265/task/28296/fd (opendir: Permission denied) 2024-04-22T13:26:48.848 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28297 cpu_tp ceph cwd unknown /proc/28265/task/28297/cwd (readlink: Permission denied) 2024-04-22T13:26:48.848 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28297 cpu_tp ceph rtd unknown /proc/28265/task/28297/root (readlink: Permission denied) 2024-04-22T13:26:48.848 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28297 cpu_tp ceph txt unknown /proc/28265/task/28297/exe (readlink: Permission denied) 2024-04-22T13:26:48.848 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28297 cpu_tp ceph NOFD /proc/28265/task/28297/fd (opendir: Permission denied) 2024-04-22T13:26:48.848 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28298 cpu_tp ceph cwd unknown /proc/28265/task/28298/cwd (readlink: Permission denied) 2024-04-22T13:26:48.848 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28298 cpu_tp ceph rtd unknown /proc/28265/task/28298/root (readlink: Permission denied) 2024-04-22T13:26:48.848 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28298 cpu_tp ceph txt unknown /proc/28265/task/28298/exe (readlink: Permission denied) 2024-04-22T13:26:48.848 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28298 cpu_tp ceph NOFD /proc/28265/task/28298/fd (opendir: Permission denied) 2024-04-22T13:26:48.848 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28299 ms_dispat ceph cwd unknown /proc/28265/task/28299/cwd (readlink: Permission denied) 2024-04-22T13:26:48.848 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28299 ms_dispat ceph rtd unknown /proc/28265/task/28299/root (readlink: Permission denied) 2024-04-22T13:26:48.848 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28299 ms_dispat ceph txt unknown /proc/28265/task/28299/exe (readlink: Permission denied) 2024-04-22T13:26:48.848 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28299 ms_dispat ceph NOFD /proc/28265/task/28299/fd (opendir: Permission denied) 2024-04-22T13:26:48.848 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28300 ms_local ceph cwd unknown /proc/28265/task/28300/cwd (readlink: Permission denied) 2024-04-22T13:26:48.848 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28300 ms_local ceph rtd unknown /proc/28265/task/28300/root (readlink: Permission denied) 2024-04-22T13:26:48.849 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28300 ms_local ceph txt unknown /proc/28265/task/28300/exe (readlink: Permission denied) 2024-04-22T13:26:48.849 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28300 ms_local ceph NOFD /proc/28265/task/28300/fd (opendir: Permission denied) 2024-04-22T13:26:48.849 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28301 safe_time ceph cwd unknown /proc/28265/task/28301/cwd (readlink: Permission denied) 2024-04-22T13:26:48.849 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28301 safe_time ceph rtd unknown /proc/28265/task/28301/root (readlink: Permission denied) 2024-04-22T13:26:48.849 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28301 safe_time ceph txt unknown /proc/28265/task/28301/exe (readlink: Permission denied) 2024-04-22T13:26:48.849 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28301 safe_time ceph NOFD /proc/28265/task/28301/fd (opendir: Permission denied) 2024-04-22T13:26:48.849 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28302 ms_dispat ceph cwd unknown /proc/28265/task/28302/cwd (readlink: Permission denied) 2024-04-22T13:26:48.849 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28302 ms_dispat ceph rtd unknown /proc/28265/task/28302/root (readlink: Permission denied) 2024-04-22T13:26:48.849 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28302 ms_dispat ceph txt unknown /proc/28265/task/28302/exe (readlink: Permission denied) 2024-04-22T13:26:48.849 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28302 ms_dispat ceph NOFD /proc/28265/task/28302/fd (opendir: Permission denied) 2024-04-22T13:26:48.849 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28303 ms_local ceph cwd unknown /proc/28265/task/28303/cwd (readlink: Permission denied) 2024-04-22T13:26:48.849 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28303 ms_local ceph rtd unknown /proc/28265/task/28303/root (readlink: Permission denied) 2024-04-22T13:26:48.849 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28303 ms_local ceph txt unknown /proc/28265/task/28303/exe (readlink: Permission denied) 2024-04-22T13:26:48.849 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 28303 ms_local ceph NOFD /proc/28265/task/28303/fd (opendir: Permission denied) 2024-04-22T13:26:48.849 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 67661 rstore_co ceph cwd unknown /proc/28265/task/67661/cwd (readlink: Permission denied) 2024-04-22T13:26:48.849 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 67661 rstore_co ceph rtd unknown /proc/28265/task/67661/root (readlink: Permission denied) 2024-04-22T13:26:48.849 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 67661 rstore_co ceph txt unknown /proc/28265/task/67661/exe (readlink: Permission denied) 2024-04-22T13:26:48.850 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mon 28265 67661 rstore_co ceph NOFD /proc/28265/task/67661/fd (opendir: Permission denied) 2024-04-22T13:26:48.850 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 28500 root cwd unknown /proc/28500/cwd (readlink: Permission denied) 2024-04-22T13:26:48.850 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 28500 root rtd unknown /proc/28500/root (readlink: Permission denied) 2024-04-22T13:26:48.850 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 28500 root txt unknown /proc/28500/exe (readlink: Permission denied) 2024-04-22T13:26:48.850 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 28500 root NOFD /proc/28500/fd (opendir: Permission denied) 2024-04-22T13:26:48.850 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 28502 root cwd unknown /proc/28502/cwd (readlink: Permission denied) 2024-04-22T13:26:48.850 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 28502 root rtd unknown /proc/28502/root (readlink: Permission denied) 2024-04-22T13:26:48.850 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 28502 root txt unknown /proc/28502/exe (readlink: Permission denied) 2024-04-22T13:26:48.850 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 28502 root NOFD /proc/28502/fd (opendir: Permission denied) 2024-04-22T13:26:48.850 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 ceph cwd unknown /proc/28504/cwd (readlink: Permission denied) 2024-04-22T13:26:48.850 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 ceph rtd unknown /proc/28504/root (readlink: Permission denied) 2024-04-22T13:26:48.850 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 ceph txt unknown /proc/28504/exe (readlink: Permission denied) 2024-04-22T13:26:48.850 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 ceph NOFD /proc/28504/fd (opendir: Permission denied) 2024-04-22T13:26:48.850 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80248 log ceph cwd unknown /proc/28504/task/80248/cwd (readlink: Permission denied) 2024-04-22T13:26:48.850 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80248 log ceph rtd unknown /proc/28504/task/80248/root (readlink: Permission denied) 2024-04-22T13:26:48.851 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80248 log ceph txt unknown /proc/28504/task/80248/exe (readlink: Permission denied) 2024-04-22T13:26:48.851 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80248 log ceph NOFD /proc/28504/task/80248/fd (opendir: Permission denied) 2024-04-22T13:26:48.851 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80250 msgr-work ceph cwd unknown /proc/28504/task/80250/cwd (readlink: Permission denied) 2024-04-22T13:26:48.851 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80250 msgr-work ceph rtd unknown /proc/28504/task/80250/root (readlink: Permission denied) 2024-04-22T13:26:48.851 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80250 msgr-work ceph txt unknown /proc/28504/task/80250/exe (readlink: Permission denied) 2024-04-22T13:26:48.851 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80250 msgr-work ceph NOFD /proc/28504/task/80250/fd (opendir: Permission denied) 2024-04-22T13:26:48.851 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80251 msgr-work ceph cwd unknown /proc/28504/task/80251/cwd (readlink: Permission denied) 2024-04-22T13:26:48.851 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80251 msgr-work ceph rtd unknown /proc/28504/task/80251/root (readlink: Permission denied) 2024-04-22T13:26:48.851 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80251 msgr-work ceph txt unknown /proc/28504/task/80251/exe (readlink: Permission denied) 2024-04-22T13:26:48.851 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80251 msgr-work ceph NOFD /proc/28504/task/80251/fd (opendir: Permission denied) 2024-04-22T13:26:48.851 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80252 msgr-work ceph cwd unknown /proc/28504/task/80252/cwd (readlink: Permission denied) 2024-04-22T13:26:48.851 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80252 msgr-work ceph rtd unknown /proc/28504/task/80252/root (readlink: Permission denied) 2024-04-22T13:26:48.851 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80252 msgr-work ceph txt unknown /proc/28504/task/80252/exe (readlink: Permission denied) 2024-04-22T13:26:48.851 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80252 msgr-work ceph NOFD /proc/28504/task/80252/fd (opendir: Permission denied) 2024-04-22T13:26:48.851 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80256 service ceph cwd unknown /proc/28504/task/80256/cwd (readlink: Permission denied) 2024-04-22T13:26:48.851 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80256 service ceph rtd unknown /proc/28504/task/80256/root (readlink: Permission denied) 2024-04-22T13:26:48.852 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80256 service ceph txt unknown /proc/28504/task/80256/exe (readlink: Permission denied) 2024-04-22T13:26:48.852 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80256 service ceph NOFD /proc/28504/task/80256/fd (opendir: Permission denied) 2024-04-22T13:26:48.852 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80257 admin_soc ceph cwd unknown /proc/28504/task/80257/cwd (readlink: Permission denied) 2024-04-22T13:26:48.852 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80257 admin_soc ceph rtd unknown /proc/28504/task/80257/root (readlink: Permission denied) 2024-04-22T13:26:48.852 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80257 admin_soc ceph txt unknown /proc/28504/task/80257/exe (readlink: Permission denied) 2024-04-22T13:26:48.852 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80257 admin_soc ceph NOFD /proc/28504/task/80257/fd (opendir: Permission denied) 2024-04-22T13:26:48.852 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80258 ceph_time ceph cwd unknown /proc/28504/task/80258/cwd (readlink: Permission denied) 2024-04-22T13:26:48.852 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80258 ceph_time ceph rtd unknown /proc/28504/task/80258/root (readlink: Permission denied) 2024-04-22T13:26:48.852 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80258 ceph_time ceph txt unknown /proc/28504/task/80258/exe (readlink: Permission denied) 2024-04-22T13:26:48.852 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80258 ceph_time ceph NOFD /proc/28504/task/80258/fd (opendir: Permission denied) 2024-04-22T13:26:48.852 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80259 fn_anonym ceph cwd unknown /proc/28504/task/80259/cwd (readlink: Permission denied) 2024-04-22T13:26:48.852 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80259 fn_anonym ceph rtd unknown /proc/28504/task/80259/root (readlink: Permission denied) 2024-04-22T13:26:48.852 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80259 fn_anonym ceph txt unknown /proc/28504/task/80259/exe (readlink: Permission denied) 2024-04-22T13:26:48.852 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80259 fn_anonym ceph NOFD /proc/28504/task/80259/fd (opendir: Permission denied) 2024-04-22T13:26:48.852 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80260 signal_ha ceph cwd unknown /proc/28504/task/80260/cwd (readlink: Permission denied) 2024-04-22T13:26:48.853 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80260 signal_ha ceph rtd unknown /proc/28504/task/80260/root (readlink: Permission denied) 2024-04-22T13:26:48.853 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80260 signal_ha ceph txt unknown /proc/28504/task/80260/exe (readlink: Permission denied) 2024-04-22T13:26:48.853 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80260 signal_ha ceph NOFD /proc/28504/task/80260/fd (opendir: Permission denied) 2024-04-22T13:26:48.853 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80261 mgrsb-fin ceph cwd unknown /proc/28504/task/80261/cwd (readlink: Permission denied) 2024-04-22T13:26:48.853 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80261 mgrsb-fin ceph rtd unknown /proc/28504/task/80261/root (readlink: Permission denied) 2024-04-22T13:26:48.853 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80261 mgrsb-fin ceph txt unknown /proc/28504/task/80261/exe (readlink: Permission denied) 2024-04-22T13:26:48.853 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80261 mgrsb-fin ceph NOFD /proc/28504/task/80261/fd (opendir: Permission denied) 2024-04-22T13:26:48.853 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80262 ms_dispat ceph cwd unknown /proc/28504/task/80262/cwd (readlink: Permission denied) 2024-04-22T13:26:48.853 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80262 ms_dispat ceph rtd unknown /proc/28504/task/80262/root (readlink: Permission denied) 2024-04-22T13:26:48.853 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80262 ms_dispat ceph txt unknown /proc/28504/task/80262/exe (readlink: Permission denied) 2024-04-22T13:26:48.853 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80262 ms_dispat ceph NOFD /proc/28504/task/80262/fd (opendir: Permission denied) 2024-04-22T13:26:48.854 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80263 ms_local ceph cwd unknown /proc/28504/task/80263/cwd (readlink: Permission denied) 2024-04-22T13:26:48.854 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80263 ms_local ceph rtd unknown /proc/28504/task/80263/root (readlink: Permission denied) 2024-04-22T13:26:48.854 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80263 ms_local ceph txt unknown /proc/28504/task/80263/exe (readlink: Permission denied) 2024-04-22T13:26:48.854 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80263 ms_local ceph NOFD /proc/28504/task/80263/fd (opendir: Permission denied) 2024-04-22T13:26:48.854 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80264 io_contex ceph cwd unknown /proc/28504/task/80264/cwd (readlink: Permission denied) 2024-04-22T13:26:48.854 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80264 io_contex ceph rtd unknown /proc/28504/task/80264/root (readlink: Permission denied) 2024-04-22T13:26:48.854 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80264 io_contex ceph txt unknown /proc/28504/task/80264/exe (readlink: Permission denied) 2024-04-22T13:26:48.854 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80264 io_contex ceph NOFD /proc/28504/task/80264/fd (opendir: Permission denied) 2024-04-22T13:26:48.854 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80265 io_contex ceph cwd unknown /proc/28504/task/80265/cwd (readlink: Permission denied) 2024-04-22T13:26:48.854 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80265 io_contex ceph rtd unknown /proc/28504/task/80265/root (readlink: Permission denied) 2024-04-22T13:26:48.854 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80265 io_contex ceph txt unknown /proc/28504/task/80265/exe (readlink: Permission denied) 2024-04-22T13:26:48.854 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80265 io_contex ceph NOFD /proc/28504/task/80265/fd (opendir: Permission denied) 2024-04-22T13:26:48.855 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80266 safe_time ceph cwd unknown /proc/28504/task/80266/cwd (readlink: Permission denied) 2024-04-22T13:26:48.855 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80266 safe_time ceph rtd unknown /proc/28504/task/80266/root (readlink: Permission denied) 2024-04-22T13:26:48.855 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80266 safe_time ceph txt unknown /proc/28504/task/80266/exe (readlink: Permission denied) 2024-04-22T13:26:48.855 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80266 safe_time ceph NOFD /proc/28504/task/80266/fd (opendir: Permission denied) 2024-04-22T13:26:48.855 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80267 safe_time ceph cwd unknown /proc/28504/task/80267/cwd (readlink: Permission denied) 2024-04-22T13:26:48.855 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80267 safe_time ceph rtd unknown /proc/28504/task/80267/root (readlink: Permission denied) 2024-04-22T13:26:48.855 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80267 safe_time ceph txt unknown /proc/28504/task/80267/exe (readlink: Permission denied) 2024-04-22T13:26:48.855 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80267 safe_time ceph NOFD /proc/28504/task/80267/fd (opendir: Permission denied) 2024-04-22T13:26:48.855 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80268 safe_time ceph cwd unknown /proc/28504/task/80268/cwd (readlink: Permission denied) 2024-04-22T13:26:48.855 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80268 safe_time ceph rtd unknown /proc/28504/task/80268/root (readlink: Permission denied) 2024-04-22T13:26:48.855 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80268 safe_time ceph txt unknown /proc/28504/task/80268/exe (readlink: Permission denied) 2024-04-22T13:26:48.855 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80268 safe_time ceph NOFD /proc/28504/task/80268/fd (opendir: Permission denied) 2024-04-22T13:26:48.855 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80269 fn_anonym ceph cwd unknown /proc/28504/task/80269/cwd (readlink: Permission denied) 2024-04-22T13:26:48.855 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80269 fn_anonym ceph rtd unknown /proc/28504/task/80269/root (readlink: Permission denied) 2024-04-22T13:26:48.855 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80269 fn_anonym ceph txt unknown /proc/28504/task/80269/exe (readlink: Permission denied) 2024-04-22T13:26:48.856 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80269 fn_anonym ceph NOFD /proc/28504/task/80269/fd (opendir: Permission denied) 2024-04-22T13:26:48.856 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80270 flusher ceph cwd unknown /proc/28504/task/80270/cwd (readlink: Permission denied) 2024-04-22T13:26:48.856 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80270 flusher ceph rtd unknown /proc/28504/task/80270/root (readlink: Permission denied) 2024-04-22T13:26:48.856 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80270 flusher ceph txt unknown /proc/28504/task/80270/exe (readlink: Permission denied) 2024-04-22T13:26:48.856 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80270 flusher ceph NOFD /proc/28504/task/80270/fd (opendir: Permission denied) 2024-04-22T13:26:48.856 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80271 safe_time ceph cwd unknown /proc/28504/task/80271/cwd (readlink: Permission denied) 2024-04-22T13:26:48.856 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80271 safe_time ceph rtd unknown /proc/28504/task/80271/root (readlink: Permission denied) 2024-04-22T13:26:48.856 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80271 safe_time ceph txt unknown /proc/28504/task/80271/exe (readlink: Permission denied) 2024-04-22T13:26:48.856 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 80271 safe_time ceph NOFD /proc/28504/task/80271/fd (opendir: Permission denied) 2024-04-22T13:26:48.856 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81101 mgr-fin ceph cwd unknown /proc/28504/task/81101/cwd (readlink: Permission denied) 2024-04-22T13:26:48.856 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81101 mgr-fin ceph rtd unknown /proc/28504/task/81101/root (readlink: Permission denied) 2024-04-22T13:26:48.856 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81101 mgr-fin ceph txt unknown /proc/28504/task/81101/exe (readlink: Permission denied) 2024-04-22T13:26:48.856 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81101 mgr-fin ceph NOFD /proc/28504/task/81101/fd (opendir: Permission denied) 2024-04-22T13:26:48.856 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81102 ms_dispat ceph cwd unknown /proc/28504/task/81102/cwd (readlink: Permission denied) 2024-04-22T13:26:48.856 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81102 ms_dispat ceph rtd unknown /proc/28504/task/81102/root (readlink: Permission denied) 2024-04-22T13:26:48.857 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81102 ms_dispat ceph txt unknown /proc/28504/task/81102/exe (readlink: Permission denied) 2024-04-22T13:26:48.857 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81102 ms_dispat ceph NOFD /proc/28504/task/81102/fd (opendir: Permission denied) 2024-04-22T13:26:48.857 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81103 ms_local ceph cwd unknown /proc/28504/task/81103/cwd (readlink: Permission denied) 2024-04-22T13:26:48.857 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81103 ms_local ceph rtd unknown /proc/28504/task/81103/root (readlink: Permission denied) 2024-04-22T13:26:48.857 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81103 ms_local ceph txt unknown /proc/28504/task/81103/exe (readlink: Permission denied) 2024-04-22T13:26:48.857 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81103 ms_local ceph NOFD /proc/28504/task/81103/fd (opendir: Permission denied) 2024-04-22T13:26:48.857 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81104 safe_time ceph cwd unknown /proc/28504/task/81104/cwd (readlink: Permission denied) 2024-04-22T13:26:48.857 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81104 safe_time ceph rtd unknown /proc/28504/task/81104/root (readlink: Permission denied) 2024-04-22T13:26:48.857 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81104 safe_time ceph txt unknown /proc/28504/task/81104/exe (readlink: Permission denied) 2024-04-22T13:26:48.857 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81104 safe_time ceph NOFD /proc/28504/task/81104/fd (opendir: Permission denied) 2024-04-22T13:26:48.857 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81105 cmdfin ceph cwd unknown /proc/28504/task/81105/cwd (readlink: Permission denied) 2024-04-22T13:26:48.857 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81105 cmdfin ceph rtd unknown /proc/28504/task/81105/root (readlink: Permission denied) 2024-04-22T13:26:48.858 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81105 cmdfin ceph txt unknown /proc/28504/task/81105/exe (readlink: Permission denied) 2024-04-22T13:26:48.858 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81105 cmdfin ceph NOFD /proc/28504/task/81105/fd (opendir: Permission denied) 2024-04-22T13:26:48.858 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81109 io_contex ceph cwd unknown /proc/28504/task/81109/cwd (readlink: Permission denied) 2024-04-22T13:26:48.858 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81109 io_contex ceph rtd unknown /proc/28504/task/81109/root (readlink: Permission denied) 2024-04-22T13:26:48.858 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81109 io_contex ceph txt unknown /proc/28504/task/81109/exe (readlink: Permission denied) 2024-04-22T13:26:48.858 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81109 io_contex ceph NOFD /proc/28504/task/81109/fd (opendir: Permission denied) 2024-04-22T13:26:48.858 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81110 io_contex ceph cwd unknown /proc/28504/task/81110/cwd (readlink: Permission denied) 2024-04-22T13:26:48.858 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81110 io_contex ceph rtd unknown /proc/28504/task/81110/root (readlink: Permission denied) 2024-04-22T13:26:48.858 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81110 io_contex ceph txt unknown /proc/28504/task/81110/exe (readlink: Permission denied) 2024-04-22T13:26:48.858 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81110 io_contex ceph NOFD /proc/28504/task/81110/fd (opendir: Permission denied) 2024-04-22T13:26:48.858 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81111 ceph_time ceph cwd unknown /proc/28504/task/81111/cwd (readlink: Permission denied) 2024-04-22T13:26:48.858 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81111 ceph_time ceph rtd unknown /proc/28504/task/81111/root (readlink: Permission denied) 2024-04-22T13:26:48.858 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81111 ceph_time ceph txt unknown /proc/28504/task/81111/exe (readlink: Permission denied) 2024-04-22T13:26:48.858 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81111 ceph_time ceph NOFD /proc/28504/task/81111/fd (opendir: Permission denied) 2024-04-22T13:26:48.859 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81112 ms_dispat ceph cwd unknown /proc/28504/task/81112/cwd (readlink: Permission denied) 2024-04-22T13:26:48.859 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81112 ms_dispat ceph rtd unknown /proc/28504/task/81112/root (readlink: Permission denied) 2024-04-22T13:26:48.859 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81112 ms_dispat ceph txt unknown /proc/28504/task/81112/exe (readlink: Permission denied) 2024-04-22T13:26:48.859 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81112 ms_dispat ceph NOFD /proc/28504/task/81112/fd (opendir: Permission denied) 2024-04-22T13:26:48.859 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81113 ms_local ceph cwd unknown /proc/28504/task/81113/cwd (readlink: Permission denied) 2024-04-22T13:26:48.859 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81113 ms_local ceph rtd unknown /proc/28504/task/81113/root (readlink: Permission denied) 2024-04-22T13:26:48.859 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81113 ms_local ceph txt unknown /proc/28504/task/81113/exe (readlink: Permission denied) 2024-04-22T13:26:48.859 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81113 ms_local ceph NOFD /proc/28504/task/81113/fd (opendir: Permission denied) 2024-04-22T13:26:48.859 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81114 safe_time ceph cwd unknown /proc/28504/task/81114/cwd (readlink: Permission denied) 2024-04-22T13:26:48.859 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81114 safe_time ceph rtd unknown /proc/28504/task/81114/root (readlink: Permission denied) 2024-04-22T13:26:48.859 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81114 safe_time ceph txt unknown /proc/28504/task/81114/exe (readlink: Permission denied) 2024-04-22T13:26:48.859 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81114 safe_time ceph NOFD /proc/28504/task/81114/fd (opendir: Permission denied) 2024-04-22T13:26:48.859 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81115 safe_time ceph cwd unknown /proc/28504/task/81115/cwd (readlink: Permission denied) 2024-04-22T13:26:48.859 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81115 safe_time ceph rtd unknown /proc/28504/task/81115/root (readlink: Permission denied) 2024-04-22T13:26:48.859 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81115 safe_time ceph txt unknown /proc/28504/task/81115/exe (readlink: Permission denied) 2024-04-22T13:26:48.860 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81115 safe_time ceph NOFD /proc/28504/task/81115/fd (opendir: Permission denied) 2024-04-22T13:26:48.860 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81116 balancer ceph cwd unknown /proc/28504/task/81116/cwd (readlink: Permission denied) 2024-04-22T13:26:48.860 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81116 balancer ceph rtd unknown /proc/28504/task/81116/root (readlink: Permission denied) 2024-04-22T13:26:48.860 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81116 balancer ceph txt unknown /proc/28504/task/81116/exe (readlink: Permission denied) 2024-04-22T13:26:48.860 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81116 balancer ceph NOFD /proc/28504/task/81116/fd (opendir: Permission denied) 2024-04-22T13:26:48.860 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81117 m-fin-bal ceph cwd unknown /proc/28504/task/81117/cwd (readlink: Permission denied) 2024-04-22T13:26:48.860 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81117 m-fin-bal ceph rtd unknown /proc/28504/task/81117/root (readlink: Permission denied) 2024-04-22T13:26:48.860 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81117 m-fin-bal ceph txt unknown /proc/28504/task/81117/exe (readlink: Permission denied) 2024-04-22T13:26:48.860 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81117 m-fin-bal ceph NOFD /proc/28504/task/81117/fd (opendir: Permission denied) 2024-04-22T13:26:48.860 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81118 mgr-fin ceph cwd unknown /proc/28504/task/81118/cwd (readlink: Permission denied) 2024-04-22T13:26:48.860 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81118 mgr-fin ceph rtd unknown /proc/28504/task/81118/root (readlink: Permission denied) 2024-04-22T13:26:48.860 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81118 mgr-fin ceph txt unknown /proc/28504/task/81118/exe (readlink: Permission denied) 2024-04-22T13:26:48.860 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81118 mgr-fin ceph NOFD /proc/28504/task/81118/fd (opendir: Permission denied) 2024-04-22T13:26:48.860 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81119 mgr-fin ceph cwd unknown /proc/28504/task/81119/cwd (readlink: Permission denied) 2024-04-22T13:26:48.860 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81119 mgr-fin ceph rtd unknown /proc/28504/task/81119/root (readlink: Permission denied) 2024-04-22T13:26:48.861 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81119 mgr-fin ceph txt unknown /proc/28504/task/81119/exe (readlink: Permission denied) 2024-04-22T13:26:48.861 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81119 mgr-fin ceph NOFD /proc/28504/task/81119/fd (opendir: Permission denied) 2024-04-22T13:26:48.861 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81120 mgr-fin ceph cwd unknown /proc/28504/task/81120/cwd (readlink: Permission denied) 2024-04-22T13:26:48.861 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81120 mgr-fin ceph rtd unknown /proc/28504/task/81120/root (readlink: Permission denied) 2024-04-22T13:26:48.861 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81120 mgr-fin ceph txt unknown /proc/28504/task/81120/exe (readlink: Permission denied) 2024-04-22T13:26:48.861 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81120 mgr-fin ceph NOFD /proc/28504/task/81120/fd (opendir: Permission denied) 2024-04-22T13:26:48.861 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81121 mgr-fin ceph cwd unknown /proc/28504/task/81121/cwd (readlink: Permission denied) 2024-04-22T13:26:48.861 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81121 mgr-fin ceph rtd unknown /proc/28504/task/81121/root (readlink: Permission denied) 2024-04-22T13:26:48.861 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81121 mgr-fin ceph txt unknown /proc/28504/task/81121/exe (readlink: Permission denied) 2024-04-22T13:26:48.861 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81121 mgr-fin ceph NOFD /proc/28504/task/81121/fd (opendir: Permission denied) 2024-04-22T13:26:48.861 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81122 mgr-fin ceph cwd unknown /proc/28504/task/81122/cwd (readlink: Permission denied) 2024-04-22T13:26:48.861 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81122 mgr-fin ceph rtd unknown /proc/28504/task/81122/root (readlink: Permission denied) 2024-04-22T13:26:48.861 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81122 mgr-fin ceph txt unknown /proc/28504/task/81122/exe (readlink: Permission denied) 2024-04-22T13:26:48.861 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81122 mgr-fin ceph NOFD /proc/28504/task/81122/fd (opendir: Permission denied) 2024-04-22T13:26:48.861 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81123 mgr-fin ceph cwd unknown /proc/28504/task/81123/cwd (readlink: Permission denied) 2024-04-22T13:26:48.862 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81123 mgr-fin ceph rtd unknown /proc/28504/task/81123/root (readlink: Permission denied) 2024-04-22T13:26:48.862 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81123 mgr-fin ceph txt unknown /proc/28504/task/81123/exe (readlink: Permission denied) 2024-04-22T13:26:48.862 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81123 mgr-fin ceph NOFD /proc/28504/task/81123/fd (opendir: Permission denied) 2024-04-22T13:26:48.862 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81124 mgr-fin ceph cwd unknown /proc/28504/task/81124/cwd (readlink: Permission denied) 2024-04-22T13:26:48.862 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81124 mgr-fin ceph rtd unknown /proc/28504/task/81124/root (readlink: Permission denied) 2024-04-22T13:26:48.862 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81124 mgr-fin ceph txt unknown /proc/28504/task/81124/exe (readlink: Permission denied) 2024-04-22T13:26:48.862 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81124 mgr-fin ceph NOFD /proc/28504/task/81124/fd (opendir: Permission denied) 2024-04-22T13:26:48.862 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81125 mgr-fin ceph cwd unknown /proc/28504/task/81125/cwd (readlink: Permission denied) 2024-04-22T13:26:48.862 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81125 mgr-fin ceph rtd unknown /proc/28504/task/81125/root (readlink: Permission denied) 2024-04-22T13:26:48.862 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81125 mgr-fin ceph txt unknown /proc/28504/task/81125/exe (readlink: Permission denied) 2024-04-22T13:26:48.862 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81125 mgr-fin ceph NOFD /proc/28504/task/81125/fd (opendir: Permission denied) 2024-04-22T13:26:48.862 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81126 mgr-fin ceph cwd unknown /proc/28504/task/81126/cwd (readlink: Permission denied) 2024-04-22T13:26:48.862 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81126 mgr-fin ceph rtd unknown /proc/28504/task/81126/root (readlink: Permission denied) 2024-04-22T13:26:48.862 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81126 mgr-fin ceph txt unknown /proc/28504/task/81126/exe (readlink: Permission denied) 2024-04-22T13:26:48.863 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81126 mgr-fin ceph NOFD /proc/28504/task/81126/fd (opendir: Permission denied) 2024-04-22T13:26:48.863 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81127 mgr-fin ceph cwd unknown /proc/28504/task/81127/cwd (readlink: Permission denied) 2024-04-22T13:26:48.863 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81127 mgr-fin ceph rtd unknown /proc/28504/task/81127/root (readlink: Permission denied) 2024-04-22T13:26:48.863 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81127 mgr-fin ceph txt unknown /proc/28504/task/81127/exe (readlink: Permission denied) 2024-04-22T13:26:48.863 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81127 mgr-fin ceph NOFD /proc/28504/task/81127/fd (opendir: Permission denied) 2024-04-22T13:26:48.863 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81128 mgr-fin ceph cwd unknown /proc/28504/task/81128/cwd (readlink: Permission denied) 2024-04-22T13:26:48.863 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81128 mgr-fin ceph rtd unknown /proc/28504/task/81128/root (readlink: Permission denied) 2024-04-22T13:26:48.863 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81128 mgr-fin ceph txt unknown /proc/28504/task/81128/exe (readlink: Permission denied) 2024-04-22T13:26:48.863 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81128 mgr-fin ceph NOFD /proc/28504/task/81128/fd (opendir: Permission denied) 2024-04-22T13:26:48.863 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81129 mgr-fin ceph cwd unknown /proc/28504/task/81129/cwd (readlink: Permission denied) 2024-04-22T13:26:48.863 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81129 mgr-fin ceph rtd unknown /proc/28504/task/81129/root (readlink: Permission denied) 2024-04-22T13:26:48.863 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81129 mgr-fin ceph txt unknown /proc/28504/task/81129/exe (readlink: Permission denied) 2024-04-22T13:26:48.863 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81129 mgr-fin ceph NOFD /proc/28504/task/81129/fd (opendir: Permission denied) 2024-04-22T13:26:48.863 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81130 mgr-fin ceph cwd unknown /proc/28504/task/81130/cwd (readlink: Permission denied) 2024-04-22T13:26:48.863 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81130 mgr-fin ceph rtd unknown /proc/28504/task/81130/root (readlink: Permission denied) 2024-04-22T13:26:48.863 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81130 mgr-fin ceph txt unknown /proc/28504/task/81130/exe (readlink: Permission denied) 2024-04-22T13:26:48.864 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81130 mgr-fin ceph NOFD /proc/28504/task/81130/fd (opendir: Permission denied) 2024-04-22T13:26:48.864 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81131 mgr-fin ceph cwd unknown /proc/28504/task/81131/cwd (readlink: Permission denied) 2024-04-22T13:26:48.864 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81131 mgr-fin ceph rtd unknown /proc/28504/task/81131/root (readlink: Permission denied) 2024-04-22T13:26:48.864 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81131 mgr-fin ceph txt unknown /proc/28504/task/81131/exe (readlink: Permission denied) 2024-04-22T13:26:48.864 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81131 mgr-fin ceph NOFD /proc/28504/task/81131/fd (opendir: Permission denied) 2024-04-22T13:26:48.864 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81132 mgr-fin ceph cwd unknown /proc/28504/task/81132/cwd (readlink: Permission denied) 2024-04-22T13:26:48.864 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81132 mgr-fin ceph rtd unknown /proc/28504/task/81132/root (readlink: Permission denied) 2024-04-22T13:26:48.864 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81132 mgr-fin ceph txt unknown /proc/28504/task/81132/exe (readlink: Permission denied) 2024-04-22T13:26:48.864 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81132 mgr-fin ceph NOFD /proc/28504/task/81132/fd (opendir: Permission denied) 2024-04-22T13:26:48.864 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81133 mgr-fin ceph cwd unknown /proc/28504/task/81133/cwd (readlink: Permission denied) 2024-04-22T13:26:48.864 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81133 mgr-fin ceph rtd unknown /proc/28504/task/81133/root (readlink: Permission denied) 2024-04-22T13:26:48.864 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81133 mgr-fin ceph txt unknown /proc/28504/task/81133/exe (readlink: Permission denied) 2024-04-22T13:26:48.864 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81133 mgr-fin ceph NOFD /proc/28504/task/81133/fd (opendir: Permission denied) 2024-04-22T13:26:48.864 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81134 cephadm ceph cwd unknown /proc/28504/task/81134/cwd (readlink: Permission denied) 2024-04-22T13:26:48.865 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81134 cephadm ceph rtd unknown /proc/28504/task/81134/root (readlink: Permission denied) 2024-04-22T13:26:48.865 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81134 cephadm ceph txt unknown /proc/28504/task/81134/exe (readlink: Permission denied) 2024-04-22T13:26:48.865 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81134 cephadm ceph NOFD /proc/28504/task/81134/fd (opendir: Permission denied) 2024-04-22T13:26:48.865 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81135 m-fin-cep ceph cwd unknown /proc/28504/task/81135/cwd (readlink: Permission denied) 2024-04-22T13:26:48.865 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81135 m-fin-cep ceph rtd unknown /proc/28504/task/81135/root (readlink: Permission denied) 2024-04-22T13:26:48.865 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81135 m-fin-cep ceph txt unknown /proc/28504/task/81135/exe (readlink: Permission denied) 2024-04-22T13:26:48.865 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81135 m-fin-cep ceph NOFD /proc/28504/task/81135/fd (opendir: Permission denied) 2024-04-22T13:26:48.865 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81136 mgr-fin ceph cwd unknown /proc/28504/task/81136/cwd (readlink: Permission denied) 2024-04-22T13:26:48.865 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81136 mgr-fin ceph rtd unknown /proc/28504/task/81136/root (readlink: Permission denied) 2024-04-22T13:26:48.865 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81136 mgr-fin ceph txt unknown /proc/28504/task/81136/exe (readlink: Permission denied) 2024-04-22T13:26:48.865 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81136 mgr-fin ceph NOFD /proc/28504/task/81136/fd (opendir: Permission denied) 2024-04-22T13:26:48.865 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81137 cephadm ceph cwd unknown /proc/28504/task/81137/cwd (readlink: Permission denied) 2024-04-22T13:26:48.865 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81137 cephadm ceph rtd unknown /proc/28504/task/81137/root (readlink: Permission denied) 2024-04-22T13:26:48.866 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81137 cephadm ceph txt unknown /proc/28504/task/81137/exe (readlink: Permission denied) 2024-04-22T13:26:48.866 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81137 cephadm ceph NOFD /proc/28504/task/81137/fd (opendir: Permission denied) 2024-04-22T13:26:48.866 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81138 mgr-fin ceph cwd unknown /proc/28504/task/81138/cwd (readlink: Permission denied) 2024-04-22T13:26:48.866 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81138 mgr-fin ceph rtd unknown /proc/28504/task/81138/root (readlink: Permission denied) 2024-04-22T13:26:48.866 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81138 mgr-fin ceph txt unknown /proc/28504/task/81138/exe (readlink: Permission denied) 2024-04-22T13:26:48.866 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81138 mgr-fin ceph NOFD /proc/28504/task/81138/fd (opendir: Permission denied) 2024-04-22T13:26:48.866 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81139 crash ceph cwd unknown /proc/28504/task/81139/cwd (readlink: Permission denied) 2024-04-22T13:26:48.866 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81139 crash ceph rtd unknown /proc/28504/task/81139/root (readlink: Permission denied) 2024-04-22T13:26:48.866 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81139 crash ceph txt unknown /proc/28504/task/81139/exe (readlink: Permission denied) 2024-04-22T13:26:48.866 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81139 crash ceph NOFD /proc/28504/task/81139/fd (opendir: Permission denied) 2024-04-22T13:26:48.866 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81140 m-fin-cra ceph cwd unknown /proc/28504/task/81140/cwd (readlink: Permission denied) 2024-04-22T13:26:48.866 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81140 m-fin-cra ceph rtd unknown /proc/28504/task/81140/root (readlink: Permission denied) 2024-04-22T13:26:48.866 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81140 m-fin-cra ceph txt unknown /proc/28504/task/81140/exe (readlink: Permission denied) 2024-04-22T13:26:48.866 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81140 m-fin-cra ceph NOFD /proc/28504/task/81140/fd (opendir: Permission denied) 2024-04-22T13:26:48.867 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81141 dashboard ceph cwd unknown /proc/28504/task/81141/cwd (readlink: Permission denied) 2024-04-22T13:26:48.867 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81141 dashboard ceph rtd unknown /proc/28504/task/81141/root (readlink: Permission denied) 2024-04-22T13:26:48.867 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81141 dashboard ceph txt unknown /proc/28504/task/81141/exe (readlink: Permission denied) 2024-04-22T13:26:48.867 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81141 dashboard ceph NOFD /proc/28504/task/81141/fd (opendir: Permission denied) 2024-04-22T13:26:48.867 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81142 m-fin-das ceph cwd unknown /proc/28504/task/81142/cwd (readlink: Permission denied) 2024-04-22T13:26:48.867 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81142 m-fin-das ceph rtd unknown /proc/28504/task/81142/root (readlink: Permission denied) 2024-04-22T13:26:48.867 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81142 m-fin-das ceph txt unknown /proc/28504/task/81142/exe (readlink: Permission denied) 2024-04-22T13:26:48.867 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81142 m-fin-das ceph NOFD /proc/28504/task/81142/fd (opendir: Permission denied) 2024-04-22T13:26:48.867 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81143 mgr-fin ceph cwd unknown /proc/28504/task/81143/cwd (readlink: Permission denied) 2024-04-22T13:26:48.867 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81143 mgr-fin ceph rtd unknown /proc/28504/task/81143/root (readlink: Permission denied) 2024-04-22T13:26:48.867 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81143 mgr-fin ceph txt unknown /proc/28504/task/81143/exe (readlink: Permission denied) 2024-04-22T13:26:48.867 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81143 mgr-fin ceph NOFD /proc/28504/task/81143/fd (opendir: Permission denied) 2024-04-22T13:26:48.867 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81144 devicehea ceph cwd unknown /proc/28504/task/81144/cwd (readlink: Permission denied) 2024-04-22T13:26:48.867 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81144 devicehea ceph rtd unknown /proc/28504/task/81144/root (readlink: Permission denied) 2024-04-22T13:26:48.867 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81144 devicehea ceph txt unknown /proc/28504/task/81144/exe (readlink: Permission denied) 2024-04-22T13:26:48.868 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81144 devicehea ceph NOFD /proc/28504/task/81144/fd (opendir: Permission denied) 2024-04-22T13:26:48.868 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81145 m-fin-dev ceph cwd unknown /proc/28504/task/81145/cwd (readlink: Permission denied) 2024-04-22T13:26:48.868 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81145 m-fin-dev ceph rtd unknown /proc/28504/task/81145/root (readlink: Permission denied) 2024-04-22T13:26:48.868 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81145 m-fin-dev ceph txt unknown /proc/28504/task/81145/exe (readlink: Permission denied) 2024-04-22T13:26:48.868 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81145 m-fin-dev ceph NOFD /proc/28504/task/81145/fd (opendir: Permission denied) 2024-04-22T13:26:48.868 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81146 mgr-fin ceph cwd unknown /proc/28504/task/81146/cwd (readlink: Permission denied) 2024-04-22T13:26:48.868 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81146 mgr-fin ceph rtd unknown /proc/28504/task/81146/root (readlink: Permission denied) 2024-04-22T13:26:48.868 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81146 mgr-fin ceph txt unknown /proc/28504/task/81146/exe (readlink: Permission denied) 2024-04-22T13:26:48.868 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81146 mgr-fin ceph NOFD /proc/28504/task/81146/fd (opendir: Permission denied) 2024-04-22T13:26:48.868 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81147 mgr-fin ceph cwd unknown /proc/28504/task/81147/cwd (readlink: Permission denied) 2024-04-22T13:26:48.868 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81147 mgr-fin ceph rtd unknown /proc/28504/task/81147/root (readlink: Permission denied) 2024-04-22T13:26:48.868 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81147 mgr-fin ceph txt unknown /proc/28504/task/81147/exe (readlink: Permission denied) 2024-04-22T13:26:48.868 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81147 mgr-fin ceph NOFD /proc/28504/task/81147/fd (opendir: Permission denied) 2024-04-22T13:26:48.868 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81148 mgr-fin ceph cwd unknown /proc/28504/task/81148/cwd (readlink: Permission denied) 2024-04-22T13:26:48.868 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81148 mgr-fin ceph rtd unknown /proc/28504/task/81148/root (readlink: Permission denied) 2024-04-22T13:26:48.868 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81148 mgr-fin ceph txt unknown /proc/28504/task/81148/exe (readlink: Permission denied) 2024-04-22T13:26:48.869 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81148 mgr-fin ceph NOFD /proc/28504/task/81148/fd (opendir: Permission denied) 2024-04-22T13:26:48.869 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81150 m-fin-ios ceph cwd unknown /proc/28504/task/81150/cwd (readlink: Permission denied) 2024-04-22T13:26:48.869 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81150 m-fin-ios ceph rtd unknown /proc/28504/task/81150/root (readlink: Permission denied) 2024-04-22T13:26:48.869 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81150 m-fin-ios ceph txt unknown /proc/28504/task/81150/exe (readlink: Permission denied) 2024-04-22T13:26:48.869 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81150 m-fin-ios ceph NOFD /proc/28504/task/81150/fd (opendir: Permission denied) 2024-04-22T13:26:48.869 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81151 mgr-fin ceph cwd unknown /proc/28504/task/81151/cwd (readlink: Permission denied) 2024-04-22T13:26:48.869 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81151 mgr-fin ceph rtd unknown /proc/28504/task/81151/root (readlink: Permission denied) 2024-04-22T13:26:48.869 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81151 mgr-fin ceph txt unknown /proc/28504/task/81151/exe (readlink: Permission denied) 2024-04-22T13:26:48.869 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81151 mgr-fin ceph NOFD /proc/28504/task/81151/fd (opendir: Permission denied) 2024-04-22T13:26:48.869 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81152 mgr-fin ceph cwd unknown /proc/28504/task/81152/cwd (readlink: Permission denied) 2024-04-22T13:26:48.869 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81152 mgr-fin ceph rtd unknown /proc/28504/task/81152/root (readlink: Permission denied) 2024-04-22T13:26:48.869 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81152 mgr-fin ceph txt unknown /proc/28504/task/81152/exe (readlink: Permission denied) 2024-04-22T13:26:48.869 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81152 mgr-fin ceph NOFD /proc/28504/task/81152/fd (opendir: Permission denied) 2024-04-22T13:26:48.869 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81154 m-fin-nfs ceph cwd unknown /proc/28504/task/81154/cwd (readlink: Permission denied) 2024-04-22T13:26:48.869 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81154 m-fin-nfs ceph rtd unknown /proc/28504/task/81154/root (readlink: Permission denied) 2024-04-22T13:26:48.870 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81154 m-fin-nfs ceph txt unknown /proc/28504/task/81154/exe (readlink: Permission denied) 2024-04-22T13:26:48.870 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81154 m-fin-nfs ceph NOFD /proc/28504/task/81154/fd (opendir: Permission denied) 2024-04-22T13:26:48.870 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81155 mgr-fin ceph cwd unknown /proc/28504/task/81155/cwd (readlink: Permission denied) 2024-04-22T13:26:48.870 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81155 mgr-fin ceph rtd unknown /proc/28504/task/81155/root (readlink: Permission denied) 2024-04-22T13:26:48.870 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81155 mgr-fin ceph txt unknown /proc/28504/task/81155/exe (readlink: Permission denied) 2024-04-22T13:26:48.870 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81155 mgr-fin ceph NOFD /proc/28504/task/81155/fd (opendir: Permission denied) 2024-04-22T13:26:48.870 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81157 mgr-fin ceph cwd unknown /proc/28504/task/81157/cwd (readlink: Permission denied) 2024-04-22T13:26:48.870 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81157 mgr-fin ceph rtd unknown /proc/28504/task/81157/root (readlink: Permission denied) 2024-04-22T13:26:48.870 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81157 mgr-fin ceph txt unknown /proc/28504/task/81157/exe (readlink: Permission denied) 2024-04-22T13:26:48.870 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81157 mgr-fin ceph NOFD /proc/28504/task/81157/fd (opendir: Permission denied) 2024-04-22T13:26:48.870 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81158 m-fin-orc ceph cwd unknown /proc/28504/task/81158/cwd (readlink: Permission denied) 2024-04-22T13:26:48.870 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81158 m-fin-orc ceph rtd unknown /proc/28504/task/81158/root (readlink: Permission denied) 2024-04-22T13:26:48.870 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81158 m-fin-orc ceph txt unknown /proc/28504/task/81158/exe (readlink: Permission denied) 2024-04-22T13:26:48.870 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81158 m-fin-orc ceph NOFD /proc/28504/task/81158/fd (opendir: Permission denied) 2024-04-22T13:26:48.871 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81159 mgr-fin ceph cwd unknown /proc/28504/task/81159/cwd (readlink: Permission denied) 2024-04-22T13:26:48.871 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81159 mgr-fin ceph rtd unknown /proc/28504/task/81159/root (readlink: Permission denied) 2024-04-22T13:26:48.871 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81159 mgr-fin ceph txt unknown /proc/28504/task/81159/exe (readlink: Permission denied) 2024-04-22T13:26:48.871 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81159 mgr-fin ceph NOFD /proc/28504/task/81159/fd (opendir: Permission denied) 2024-04-22T13:26:48.871 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81160 pg_autosc ceph cwd unknown /proc/28504/task/81160/cwd (readlink: Permission denied) 2024-04-22T13:26:48.871 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81160 pg_autosc ceph rtd unknown /proc/28504/task/81160/root (readlink: Permission denied) 2024-04-22T13:26:48.871 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81160 pg_autosc ceph txt unknown /proc/28504/task/81160/exe (readlink: Permission denied) 2024-04-22T13:26:48.871 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81160 pg_autosc ceph NOFD /proc/28504/task/81160/fd (opendir: Permission denied) 2024-04-22T13:26:48.871 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81161 m-fin-pg_ ceph cwd unknown /proc/28504/task/81161/cwd (readlink: Permission denied) 2024-04-22T13:26:48.871 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81161 m-fin-pg_ ceph rtd unknown /proc/28504/task/81161/root (readlink: Permission denied) 2024-04-22T13:26:48.871 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81161 m-fin-pg_ ceph txt unknown /proc/28504/task/81161/exe (readlink: Permission denied) 2024-04-22T13:26:48.871 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81161 m-fin-pg_ ceph NOFD /proc/28504/task/81161/fd (opendir: Permission denied) 2024-04-22T13:26:48.871 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81162 progress ceph cwd unknown /proc/28504/task/81162/cwd (readlink: Permission denied) 2024-04-22T13:26:48.871 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81162 progress ceph rtd unknown /proc/28504/task/81162/root (readlink: Permission denied) 2024-04-22T13:26:48.872 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81162 progress ceph txt unknown /proc/28504/task/81162/exe (readlink: Permission denied) 2024-04-22T13:26:48.872 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81162 progress ceph NOFD /proc/28504/task/81162/fd (opendir: Permission denied) 2024-04-22T13:26:48.872 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81163 m-fin-pro ceph cwd unknown /proc/28504/task/81163/cwd (readlink: Permission denied) 2024-04-22T13:26:48.872 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81163 m-fin-pro ceph rtd unknown /proc/28504/task/81163/root (readlink: Permission denied) 2024-04-22T13:26:48.872 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81163 m-fin-pro ceph txt unknown /proc/28504/task/81163/exe (readlink: Permission denied) 2024-04-22T13:26:48.872 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81163 m-fin-pro ceph NOFD /proc/28504/task/81163/fd (opendir: Permission denied) 2024-04-22T13:26:48.872 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81164 mgr-fin ceph cwd unknown /proc/28504/task/81164/cwd (readlink: Permission denied) 2024-04-22T13:26:48.872 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81164 mgr-fin ceph rtd unknown /proc/28504/task/81164/root (readlink: Permission denied) 2024-04-22T13:26:48.872 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81164 mgr-fin ceph txt unknown /proc/28504/task/81164/exe (readlink: Permission denied) 2024-04-22T13:26:48.872 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81164 mgr-fin ceph NOFD /proc/28504/task/81164/fd (opendir: Permission denied) 2024-04-22T13:26:48.872 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81168 io_contex ceph cwd unknown /proc/28504/task/81168/cwd (readlink: Permission denied) 2024-04-22T13:26:48.872 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81168 io_contex ceph rtd unknown /proc/28504/task/81168/root (readlink: Permission denied) 2024-04-22T13:26:48.872 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81168 io_contex ceph txt unknown /proc/28504/task/81168/exe (readlink: Permission denied) 2024-04-22T13:26:48.872 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81168 io_contex ceph NOFD /proc/28504/task/81168/fd (opendir: Permission denied) 2024-04-22T13:26:48.873 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81169 io_contex ceph cwd unknown /proc/28504/task/81169/cwd (readlink: Permission denied) 2024-04-22T13:26:48.873 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81169 io_contex ceph rtd unknown /proc/28504/task/81169/root (readlink: Permission denied) 2024-04-22T13:26:48.873 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81169 io_contex ceph txt unknown /proc/28504/task/81169/exe (readlink: Permission denied) 2024-04-22T13:26:48.873 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81169 io_contex ceph NOFD /proc/28504/task/81169/fd (opendir: Permission denied) 2024-04-22T13:26:48.873 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81170 ceph_time ceph cwd unknown /proc/28504/task/81170/cwd (readlink: Permission denied) 2024-04-22T13:26:48.873 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81170 ceph_time ceph rtd unknown /proc/28504/task/81170/root (readlink: Permission denied) 2024-04-22T13:26:48.873 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81170 ceph_time ceph txt unknown /proc/28504/task/81170/exe (readlink: Permission denied) 2024-04-22T13:26:48.873 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81170 ceph_time ceph NOFD /proc/28504/task/81170/fd (opendir: Permission denied) 2024-04-22T13:26:48.873 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81171 ms_dispat ceph cwd unknown /proc/28504/task/81171/cwd (readlink: Permission denied) 2024-04-22T13:26:48.873 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81171 ms_dispat ceph rtd unknown /proc/28504/task/81171/root (readlink: Permission denied) 2024-04-22T13:26:48.873 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81171 ms_dispat ceph txt unknown /proc/28504/task/81171/exe (readlink: Permission denied) 2024-04-22T13:26:48.873 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81171 ms_dispat ceph NOFD /proc/28504/task/81171/fd (opendir: Permission denied) 2024-04-22T13:26:48.873 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81172 ms_local ceph cwd unknown /proc/28504/task/81172/cwd (readlink: Permission denied) 2024-04-22T13:26:48.873 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81172 ms_local ceph rtd unknown /proc/28504/task/81172/root (readlink: Permission denied) 2024-04-22T13:26:48.874 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81172 ms_local ceph txt unknown /proc/28504/task/81172/exe (readlink: Permission denied) 2024-04-22T13:26:48.874 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81172 ms_local ceph NOFD /proc/28504/task/81172/fd (opendir: Permission denied) 2024-04-22T13:26:48.874 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81173 safe_time ceph cwd unknown /proc/28504/task/81173/cwd (readlink: Permission denied) 2024-04-22T13:26:48.874 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81173 safe_time ceph rtd unknown /proc/28504/task/81173/root (readlink: Permission denied) 2024-04-22T13:26:48.874 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81173 safe_time ceph txt unknown /proc/28504/task/81173/exe (readlink: Permission denied) 2024-04-22T13:26:48.874 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81173 safe_time ceph NOFD /proc/28504/task/81173/fd (opendir: Permission denied) 2024-04-22T13:26:48.874 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81174 safe_time ceph cwd unknown /proc/28504/task/81174/cwd (readlink: Permission denied) 2024-04-22T13:26:48.874 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81174 safe_time ceph rtd unknown /proc/28504/task/81174/root (readlink: Permission denied) 2024-04-22T13:26:48.874 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81174 safe_time ceph txt unknown /proc/28504/task/81174/exe (readlink: Permission denied) 2024-04-22T13:26:48.874 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81174 safe_time ceph NOFD /proc/28504/task/81174/fd (opendir: Permission denied) 2024-04-22T13:26:48.874 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81175 mgr-fin ceph cwd unknown /proc/28504/task/81175/cwd (readlink: Permission denied) 2024-04-22T13:26:48.874 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81175 mgr-fin ceph rtd unknown /proc/28504/task/81175/root (readlink: Permission denied) 2024-04-22T13:26:48.874 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81175 mgr-fin ceph txt unknown /proc/28504/task/81175/exe (readlink: Permission denied) 2024-04-22T13:26:48.874 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81175 mgr-fin ceph NOFD /proc/28504/task/81175/fd (opendir: Permission denied) 2024-04-22T13:26:48.874 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81177 m-fin-rbd ceph cwd unknown /proc/28504/task/81177/cwd (readlink: Permission denied) 2024-04-22T13:26:48.875 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81177 m-fin-rbd ceph rtd unknown /proc/28504/task/81177/root (readlink: Permission denied) 2024-04-22T13:26:48.875 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81177 m-fin-rbd ceph txt unknown /proc/28504/task/81177/exe (readlink: Permission denied) 2024-04-22T13:26:48.875 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81177 m-fin-rbd ceph NOFD /proc/28504/task/81177/fd (opendir: Permission denied) 2024-04-22T13:26:48.875 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81178 restful ceph cwd unknown /proc/28504/task/81178/cwd (readlink: Permission denied) 2024-04-22T13:26:48.875 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81178 restful ceph rtd unknown /proc/28504/task/81178/root (readlink: Permission denied) 2024-04-22T13:26:48.875 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81178 restful ceph txt unknown /proc/28504/task/81178/exe (readlink: Permission denied) 2024-04-22T13:26:48.875 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81178 restful ceph NOFD /proc/28504/task/81178/fd (opendir: Permission denied) 2024-04-22T13:26:48.875 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81179 m-fin-res ceph cwd unknown /proc/28504/task/81179/cwd (readlink: Permission denied) 2024-04-22T13:26:48.875 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81179 m-fin-res ceph rtd unknown /proc/28504/task/81179/root (readlink: Permission denied) 2024-04-22T13:26:48.875 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81179 m-fin-res ceph txt unknown /proc/28504/task/81179/exe (readlink: Permission denied) 2024-04-22T13:26:48.875 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81179 m-fin-res ceph NOFD /proc/28504/task/81179/fd (opendir: Permission denied) 2024-04-22T13:26:48.875 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81180 mgr-fin ceph cwd unknown /proc/28504/task/81180/cwd (readlink: Permission denied) 2024-04-22T13:26:48.875 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81180 mgr-fin ceph rtd unknown /proc/28504/task/81180/root (readlink: Permission denied) 2024-04-22T13:26:48.875 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81180 mgr-fin ceph txt unknown /proc/28504/task/81180/exe (readlink: Permission denied) 2024-04-22T13:26:48.876 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81180 mgr-fin ceph NOFD /proc/28504/task/81180/fd (opendir: Permission denied) 2024-04-22T13:26:48.876 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81181 mgr-fin ceph cwd unknown /proc/28504/task/81181/cwd (readlink: Permission denied) 2024-04-22T13:26:48.876 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81181 mgr-fin ceph rtd unknown /proc/28504/task/81181/root (readlink: Permission denied) 2024-04-22T13:26:48.876 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81181 mgr-fin ceph txt unknown /proc/28504/task/81181/exe (readlink: Permission denied) 2024-04-22T13:26:48.876 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81181 mgr-fin ceph NOFD /proc/28504/task/81181/fd (opendir: Permission denied) 2024-04-22T13:26:48.876 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81182 mgr-fin ceph cwd unknown /proc/28504/task/81182/cwd (readlink: Permission denied) 2024-04-22T13:26:48.876 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81182 mgr-fin ceph rtd unknown /proc/28504/task/81182/root (readlink: Permission denied) 2024-04-22T13:26:48.876 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81182 mgr-fin ceph txt unknown /proc/28504/task/81182/exe (readlink: Permission denied) 2024-04-22T13:26:48.876 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81182 mgr-fin ceph NOFD /proc/28504/task/81182/fd (opendir: Permission denied) 2024-04-22T13:26:48.876 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81183 mgr-fin ceph cwd unknown /proc/28504/task/81183/cwd (readlink: Permission denied) 2024-04-22T13:26:48.876 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81183 mgr-fin ceph rtd unknown /proc/28504/task/81183/root (readlink: Permission denied) 2024-04-22T13:26:48.876 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81183 mgr-fin ceph txt unknown /proc/28504/task/81183/exe (readlink: Permission denied) 2024-04-22T13:26:48.876 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81183 mgr-fin ceph NOFD /proc/28504/task/81183/fd (opendir: Permission denied) 2024-04-22T13:26:48.876 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81184 mgr-fin ceph cwd unknown /proc/28504/task/81184/cwd (readlink: Permission denied) 2024-04-22T13:26:48.876 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81184 mgr-fin ceph rtd unknown /proc/28504/task/81184/root (readlink: Permission denied) 2024-04-22T13:26:48.876 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81184 mgr-fin ceph txt unknown /proc/28504/task/81184/exe (readlink: Permission denied) 2024-04-22T13:26:48.877 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81184 mgr-fin ceph NOFD /proc/28504/task/81184/fd (opendir: Permission denied) 2024-04-22T13:26:48.877 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81185 mgr-fin ceph cwd unknown /proc/28504/task/81185/cwd (readlink: Permission denied) 2024-04-22T13:26:48.877 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81185 mgr-fin ceph rtd unknown /proc/28504/task/81185/root (readlink: Permission denied) 2024-04-22T13:26:48.877 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81185 mgr-fin ceph txt unknown /proc/28504/task/81185/exe (readlink: Permission denied) 2024-04-22T13:26:48.877 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81185 mgr-fin ceph NOFD /proc/28504/task/81185/fd (opendir: Permission denied) 2024-04-22T13:26:48.877 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81189 io_contex ceph cwd unknown /proc/28504/task/81189/cwd (readlink: Permission denied) 2024-04-22T13:26:48.877 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81189 io_contex ceph rtd unknown /proc/28504/task/81189/root (readlink: Permission denied) 2024-04-22T13:26:48.877 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81189 io_contex ceph txt unknown /proc/28504/task/81189/exe (readlink: Permission denied) 2024-04-22T13:26:48.877 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81189 io_contex ceph NOFD /proc/28504/task/81189/fd (opendir: Permission denied) 2024-04-22T13:26:48.877 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81190 io_contex ceph cwd unknown /proc/28504/task/81190/cwd (readlink: Permission denied) 2024-04-22T13:26:48.877 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81190 io_contex ceph rtd unknown /proc/28504/task/81190/root (readlink: Permission denied) 2024-04-22T13:26:48.877 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81190 io_contex ceph txt unknown /proc/28504/task/81190/exe (readlink: Permission denied) 2024-04-22T13:26:48.877 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81190 io_contex ceph NOFD /proc/28504/task/81190/fd (opendir: Permission denied) 2024-04-22T13:26:48.877 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81191 ceph_time ceph cwd unknown /proc/28504/task/81191/cwd (readlink: Permission denied) 2024-04-22T13:26:48.877 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81191 ceph_time ceph rtd unknown /proc/28504/task/81191/root (readlink: Permission denied) 2024-04-22T13:26:48.878 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81191 ceph_time ceph txt unknown /proc/28504/task/81191/exe (readlink: Permission denied) 2024-04-22T13:26:48.878 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81191 ceph_time ceph NOFD /proc/28504/task/81191/fd (opendir: Permission denied) 2024-04-22T13:26:48.878 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81192 ms_dispat ceph cwd unknown /proc/28504/task/81192/cwd (readlink: Permission denied) 2024-04-22T13:26:48.878 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81192 ms_dispat ceph rtd unknown /proc/28504/task/81192/root (readlink: Permission denied) 2024-04-22T13:26:48.878 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81192 ms_dispat ceph txt unknown /proc/28504/task/81192/exe (readlink: Permission denied) 2024-04-22T13:26:48.878 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81192 ms_dispat ceph NOFD /proc/28504/task/81192/fd (opendir: Permission denied) 2024-04-22T13:26:48.878 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81193 ms_local ceph cwd unknown /proc/28504/task/81193/cwd (readlink: Permission denied) 2024-04-22T13:26:48.878 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81193 ms_local ceph rtd unknown /proc/28504/task/81193/root (readlink: Permission denied) 2024-04-22T13:26:48.878 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81193 ms_local ceph txt unknown /proc/28504/task/81193/exe (readlink: Permission denied) 2024-04-22T13:26:48.878 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81193 ms_local ceph NOFD /proc/28504/task/81193/fd (opendir: Permission denied) 2024-04-22T13:26:48.878 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81194 safe_time ceph cwd unknown /proc/28504/task/81194/cwd (readlink: Permission denied) 2024-04-22T13:26:48.878 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81194 safe_time ceph rtd unknown /proc/28504/task/81194/root (readlink: Permission denied) 2024-04-22T13:26:48.878 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81194 safe_time ceph txt unknown /proc/28504/task/81194/exe (readlink: Permission denied) 2024-04-22T13:26:48.878 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81194 safe_time ceph NOFD /proc/28504/task/81194/fd (opendir: Permission denied) 2024-04-22T13:26:48.879 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81195 safe_time ceph cwd unknown /proc/28504/task/81195/cwd (readlink: Permission denied) 2024-04-22T13:26:48.879 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81195 safe_time ceph rtd unknown /proc/28504/task/81195/root (readlink: Permission denied) 2024-04-22T13:26:48.879 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81195 safe_time ceph txt unknown /proc/28504/task/81195/exe (readlink: Permission denied) 2024-04-22T13:26:48.879 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81195 safe_time ceph NOFD /proc/28504/task/81195/fd (opendir: Permission denied) 2024-04-22T13:26:48.879 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81199 m-fin-sna ceph cwd unknown /proc/28504/task/81199/cwd (readlink: Permission denied) 2024-04-22T13:26:48.879 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81199 m-fin-sna ceph rtd unknown /proc/28504/task/81199/root (readlink: Permission denied) 2024-04-22T13:26:48.879 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81199 m-fin-sna ceph txt unknown /proc/28504/task/81199/exe (readlink: Permission denied) 2024-04-22T13:26:48.879 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81199 m-fin-sna ceph NOFD /proc/28504/task/81199/fd (opendir: Permission denied) 2024-04-22T13:26:48.879 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81201 m-fin-sta ceph cwd unknown /proc/28504/task/81201/cwd (readlink: Permission denied) 2024-04-22T13:26:48.879 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81201 m-fin-sta ceph rtd unknown /proc/28504/task/81201/root (readlink: Permission denied) 2024-04-22T13:26:48.879 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81201 m-fin-sta ceph txt unknown /proc/28504/task/81201/exe (readlink: Permission denied) 2024-04-22T13:26:48.879 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81201 m-fin-sta ceph NOFD /proc/28504/task/81201/fd (opendir: Permission denied) 2024-04-22T13:26:48.879 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81202 telemetry ceph cwd unknown /proc/28504/task/81202/cwd (readlink: Permission denied) 2024-04-22T13:26:48.879 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81202 telemetry ceph rtd unknown /proc/28504/task/81202/root (readlink: Permission denied) 2024-04-22T13:26:48.879 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81202 telemetry ceph txt unknown /proc/28504/task/81202/exe (readlink: Permission denied) 2024-04-22T13:26:48.880 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81202 telemetry ceph NOFD /proc/28504/task/81202/fd (opendir: Permission denied) 2024-04-22T13:26:48.880 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81203 m-fin-tel ceph cwd unknown /proc/28504/task/81203/cwd (readlink: Permission denied) 2024-04-22T13:26:48.880 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81203 m-fin-tel ceph rtd unknown /proc/28504/task/81203/root (readlink: Permission denied) 2024-04-22T13:26:48.880 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81203 m-fin-tel ceph txt unknown /proc/28504/task/81203/exe (readlink: Permission denied) 2024-04-22T13:26:48.880 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81203 m-fin-tel ceph NOFD /proc/28504/task/81203/fd (opendir: Permission denied) 2024-04-22T13:26:48.880 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81204 mgr-fin ceph cwd unknown /proc/28504/task/81204/cwd (readlink: Permission denied) 2024-04-22T13:26:48.880 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81204 mgr-fin ceph rtd unknown /proc/28504/task/81204/root (readlink: Permission denied) 2024-04-22T13:26:48.880 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81204 mgr-fin ceph txt unknown /proc/28504/task/81204/exe (readlink: Permission denied) 2024-04-22T13:26:48.880 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81204 mgr-fin ceph NOFD /proc/28504/task/81204/fd (opendir: Permission denied) 2024-04-22T13:26:48.880 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81208 io_contex ceph cwd unknown /proc/28504/task/81208/cwd (readlink: Permission denied) 2024-04-22T13:26:48.880 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81208 io_contex ceph rtd unknown /proc/28504/task/81208/root (readlink: Permission denied) 2024-04-22T13:26:48.880 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81208 io_contex ceph txt unknown /proc/28504/task/81208/exe (readlink: Permission denied) 2024-04-22T13:26:48.880 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81208 io_contex ceph NOFD /proc/28504/task/81208/fd (opendir: Permission denied) 2024-04-22T13:26:48.880 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81209 io_contex ceph cwd unknown /proc/28504/task/81209/cwd (readlink: Permission denied) 2024-04-22T13:26:48.881 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81209 io_contex ceph rtd unknown /proc/28504/task/81209/root (readlink: Permission denied) 2024-04-22T13:26:48.881 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81209 io_contex ceph txt unknown /proc/28504/task/81209/exe (readlink: Permission denied) 2024-04-22T13:26:48.881 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81209 io_contex ceph NOFD /proc/28504/task/81209/fd (opendir: Permission denied) 2024-04-22T13:26:48.881 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81210 ceph_time ceph cwd unknown /proc/28504/task/81210/cwd (readlink: Permission denied) 2024-04-22T13:26:48.881 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81210 ceph_time ceph rtd unknown /proc/28504/task/81210/root (readlink: Permission denied) 2024-04-22T13:26:48.881 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81210 ceph_time ceph txt unknown /proc/28504/task/81210/exe (readlink: Permission denied) 2024-04-22T13:26:48.881 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81210 ceph_time ceph NOFD /proc/28504/task/81210/fd (opendir: Permission denied) 2024-04-22T13:26:48.881 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81211 ms_dispat ceph cwd unknown /proc/28504/task/81211/cwd (readlink: Permission denied) 2024-04-22T13:26:48.881 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81211 ms_dispat ceph rtd unknown /proc/28504/task/81211/root (readlink: Permission denied) 2024-04-22T13:26:48.881 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81211 ms_dispat ceph txt unknown /proc/28504/task/81211/exe (readlink: Permission denied) 2024-04-22T13:26:48.881 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81211 ms_dispat ceph NOFD /proc/28504/task/81211/fd (opendir: Permission denied) 2024-04-22T13:26:48.881 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81212 ms_local ceph cwd unknown /proc/28504/task/81212/cwd (readlink: Permission denied) 2024-04-22T13:26:48.881 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81212 ms_local ceph rtd unknown /proc/28504/task/81212/root (readlink: Permission denied) 2024-04-22T13:26:48.881 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81212 ms_local ceph txt unknown /proc/28504/task/81212/exe (readlink: Permission denied) 2024-04-22T13:26:48.881 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81212 ms_local ceph NOFD /proc/28504/task/81212/fd (opendir: Permission denied) 2024-04-22T13:26:48.882 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81213 safe_time ceph cwd unknown /proc/28504/task/81213/cwd (readlink: Permission denied) 2024-04-22T13:26:48.882 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81213 safe_time ceph rtd unknown /proc/28504/task/81213/root (readlink: Permission denied) 2024-04-22T13:26:48.882 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81213 safe_time ceph txt unknown /proc/28504/task/81213/exe (readlink: Permission denied) 2024-04-22T13:26:48.882 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81213 safe_time ceph NOFD /proc/28504/task/81213/fd (opendir: Permission denied) 2024-04-22T13:26:48.882 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81214 safe_time ceph cwd unknown /proc/28504/task/81214/cwd (readlink: Permission denied) 2024-04-22T13:26:48.882 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81214 safe_time ceph rtd unknown /proc/28504/task/81214/root (readlink: Permission denied) 2024-04-22T13:26:48.882 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81214 safe_time ceph txt unknown /proc/28504/task/81214/exe (readlink: Permission denied) 2024-04-22T13:26:48.882 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81214 safe_time ceph NOFD /proc/28504/task/81214/fd (opendir: Permission denied) 2024-04-22T13:26:48.882 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81215 mgr-fin ceph cwd unknown /proc/28504/task/81215/cwd (readlink: Permission denied) 2024-04-22T13:26:48.882 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81215 mgr-fin ceph rtd unknown /proc/28504/task/81215/root (readlink: Permission denied) 2024-04-22T13:26:48.882 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81215 mgr-fin ceph txt unknown /proc/28504/task/81215/exe (readlink: Permission denied) 2024-04-22T13:26:48.882 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81215 mgr-fin ceph NOFD /proc/28504/task/81215/fd (opendir: Permission denied) 2024-04-22T13:26:48.882 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81216 mgr-fin ceph cwd unknown /proc/28504/task/81216/cwd (readlink: Permission denied) 2024-04-22T13:26:48.882 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81216 mgr-fin ceph rtd unknown /proc/28504/task/81216/root (readlink: Permission denied) 2024-04-22T13:26:48.883 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81216 mgr-fin ceph txt unknown /proc/28504/task/81216/exe (readlink: Permission denied) 2024-04-22T13:26:48.883 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81216 mgr-fin ceph NOFD /proc/28504/task/81216/fd (opendir: Permission denied) 2024-04-22T13:26:48.883 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81217 mgr-fin ceph cwd unknown /proc/28504/task/81217/cwd (readlink: Permission denied) 2024-04-22T13:26:48.883 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81217 mgr-fin ceph rtd unknown /proc/28504/task/81217/root (readlink: Permission denied) 2024-04-22T13:26:48.883 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81217 mgr-fin ceph txt unknown /proc/28504/task/81217/exe (readlink: Permission denied) 2024-04-22T13:26:48.883 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81217 mgr-fin ceph NOFD /proc/28504/task/81217/fd (opendir: Permission denied) 2024-04-22T13:26:48.883 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81218 mgr-fin ceph cwd unknown /proc/28504/task/81218/cwd (readlink: Permission denied) 2024-04-22T13:26:48.883 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81218 mgr-fin ceph rtd unknown /proc/28504/task/81218/root (readlink: Permission denied) 2024-04-22T13:26:48.883 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81218 mgr-fin ceph txt unknown /proc/28504/task/81218/exe (readlink: Permission denied) 2024-04-22T13:26:48.883 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81218 mgr-fin ceph NOFD /proc/28504/task/81218/fd (opendir: Permission denied) 2024-04-22T13:26:48.883 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81219 mgr-fin ceph cwd unknown /proc/28504/task/81219/cwd (readlink: Permission denied) 2024-04-22T13:26:48.883 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81219 mgr-fin ceph rtd unknown /proc/28504/task/81219/root (readlink: Permission denied) 2024-04-22T13:26:48.883 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81219 mgr-fin ceph txt unknown /proc/28504/task/81219/exe (readlink: Permission denied) 2024-04-22T13:26:48.883 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81219 mgr-fin ceph NOFD /proc/28504/task/81219/fd (opendir: Permission denied) 2024-04-22T13:26:48.883 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81220 mgr-fin ceph cwd unknown /proc/28504/task/81220/cwd (readlink: Permission denied) 2024-04-22T13:26:48.883 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81220 mgr-fin ceph rtd unknown /proc/28504/task/81220/root (readlink: Permission denied) 2024-04-22T13:26:48.884 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81220 mgr-fin ceph txt unknown /proc/28504/task/81220/exe (readlink: Permission denied) 2024-04-22T13:26:48.884 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81220 mgr-fin ceph NOFD /proc/28504/task/81220/fd (opendir: Permission denied) 2024-04-22T13:26:48.884 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81221 mgr-fin ceph cwd unknown /proc/28504/task/81221/cwd (readlink: Permission denied) 2024-04-22T13:26:48.884 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81221 mgr-fin ceph rtd unknown /proc/28504/task/81221/root (readlink: Permission denied) 2024-04-22T13:26:48.884 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81221 mgr-fin ceph txt unknown /proc/28504/task/81221/exe (readlink: Permission denied) 2024-04-22T13:26:48.884 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81221 mgr-fin ceph NOFD /proc/28504/task/81221/fd (opendir: Permission denied) 2024-04-22T13:26:48.884 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81222 mgr-fin ceph cwd unknown /proc/28504/task/81222/cwd (readlink: Permission denied) 2024-04-22T13:26:48.884 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81222 mgr-fin ceph rtd unknown /proc/28504/task/81222/root (readlink: Permission denied) 2024-04-22T13:26:48.884 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81222 mgr-fin ceph txt unknown /proc/28504/task/81222/exe (readlink: Permission denied) 2024-04-22T13:26:48.884 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81222 mgr-fin ceph NOFD /proc/28504/task/81222/fd (opendir: Permission denied) 2024-04-22T13:26:48.884 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81223 mgr-fin ceph cwd unknown /proc/28504/task/81223/cwd (readlink: Permission denied) 2024-04-22T13:26:48.884 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81223 mgr-fin ceph rtd unknown /proc/28504/task/81223/root (readlink: Permission denied) 2024-04-22T13:26:48.884 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81223 mgr-fin ceph txt unknown /proc/28504/task/81223/exe (readlink: Permission denied) 2024-04-22T13:26:48.884 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81223 mgr-fin ceph NOFD /proc/28504/task/81223/fd (opendir: Permission denied) 2024-04-22T13:26:48.884 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81224 mgr-fin ceph cwd unknown /proc/28504/task/81224/cwd (readlink: Permission denied) 2024-04-22T13:26:48.884 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81224 mgr-fin ceph rtd unknown /proc/28504/task/81224/root (readlink: Permission denied) 2024-04-22T13:26:48.885 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81224 mgr-fin ceph txt unknown /proc/28504/task/81224/exe (readlink: Permission denied) 2024-04-22T13:26:48.885 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81224 mgr-fin ceph NOFD /proc/28504/task/81224/fd (opendir: Permission denied) 2024-04-22T13:26:48.885 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81225 mgr-fin ceph cwd unknown /proc/28504/task/81225/cwd (readlink: Permission denied) 2024-04-22T13:26:48.885 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81225 mgr-fin ceph rtd unknown /proc/28504/task/81225/root (readlink: Permission denied) 2024-04-22T13:26:48.885 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81225 mgr-fin ceph txt unknown /proc/28504/task/81225/exe (readlink: Permission denied) 2024-04-22T13:26:48.885 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81225 mgr-fin ceph NOFD /proc/28504/task/81225/fd (opendir: Permission denied) 2024-04-22T13:26:48.885 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81226 mgr-fin ceph cwd unknown /proc/28504/task/81226/cwd (readlink: Permission denied) 2024-04-22T13:26:48.885 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81226 mgr-fin ceph rtd unknown /proc/28504/task/81226/root (readlink: Permission denied) 2024-04-22T13:26:48.885 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81226 mgr-fin ceph txt unknown /proc/28504/task/81226/exe (readlink: Permission denied) 2024-04-22T13:26:48.885 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81226 mgr-fin ceph NOFD /proc/28504/task/81226/fd (opendir: Permission denied) 2024-04-22T13:26:48.885 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81228 m-fin-vol ceph cwd unknown /proc/28504/task/81228/cwd (readlink: Permission denied) 2024-04-22T13:26:48.885 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81228 m-fin-vol ceph rtd unknown /proc/28504/task/81228/root (readlink: Permission denied) 2024-04-22T13:26:48.885 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81228 m-fin-vol ceph txt unknown /proc/28504/task/81228/exe (readlink: Permission denied) 2024-04-22T13:26:48.885 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81228 m-fin-vol ceph NOFD /proc/28504/task/81228/fd (opendir: Permission denied) 2024-04-22T13:26:48.886 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81229 io_contex ceph cwd unknown /proc/28504/task/81229/cwd (readlink: Permission denied) 2024-04-22T13:26:48.886 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81229 io_contex ceph rtd unknown /proc/28504/task/81229/root (readlink: Permission denied) 2024-04-22T13:26:48.886 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81229 io_contex ceph txt unknown /proc/28504/task/81229/exe (readlink: Permission denied) 2024-04-22T13:26:48.886 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81229 io_contex ceph NOFD /proc/28504/task/81229/fd (opendir: Permission denied) 2024-04-22T13:26:48.886 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81230 io_contex ceph cwd unknown /proc/28504/task/81230/cwd (readlink: Permission denied) 2024-04-22T13:26:48.886 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81230 io_contex ceph rtd unknown /proc/28504/task/81230/root (readlink: Permission denied) 2024-04-22T13:26:48.886 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81230 io_contex ceph txt unknown /proc/28504/task/81230/exe (readlink: Permission denied) 2024-04-22T13:26:48.886 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81230 io_contex ceph NOFD /proc/28504/task/81230/fd (opendir: Permission denied) 2024-04-22T13:26:48.886 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81276 dashboard ceph cwd unknown /proc/28504/task/81276/cwd (readlink: Permission denied) 2024-04-22T13:26:48.886 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81276 dashboard ceph rtd unknown /proc/28504/task/81276/root (readlink: Permission denied) 2024-04-22T13:26:48.886 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81276 dashboard ceph txt unknown /proc/28504/task/81276/exe (readlink: Permission denied) 2024-04-22T13:26:48.886 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81276 dashboard ceph NOFD /proc/28504/task/81276/fd (opendir: Permission denied) 2024-04-22T13:26:48.886 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81277 dashboard ceph cwd unknown /proc/28504/task/81277/cwd (readlink: Permission denied) 2024-04-22T13:26:48.886 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81277 dashboard ceph rtd unknown /proc/28504/task/81277/root (readlink: Permission denied) 2024-04-22T13:26:48.887 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81277 dashboard ceph txt unknown /proc/28504/task/81277/exe (readlink: Permission denied) 2024-04-22T13:26:48.887 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81277 dashboard ceph NOFD /proc/28504/task/81277/fd (opendir: Permission denied) 2024-04-22T13:26:48.887 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81278 dashboard ceph cwd unknown /proc/28504/task/81278/cwd (readlink: Permission denied) 2024-04-22T13:26:48.887 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81278 dashboard ceph rtd unknown /proc/28504/task/81278/root (readlink: Permission denied) 2024-04-22T13:26:48.887 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81278 dashboard ceph txt unknown /proc/28504/task/81278/exe (readlink: Permission denied) 2024-04-22T13:26:48.887 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81278 dashboard ceph NOFD /proc/28504/task/81278/fd (opendir: Permission denied) 2024-04-22T13:26:48.887 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81279 dashboard ceph cwd unknown /proc/28504/task/81279/cwd (readlink: Permission denied) 2024-04-22T13:26:48.887 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81279 dashboard ceph rtd unknown /proc/28504/task/81279/root (readlink: Permission denied) 2024-04-22T13:26:48.887 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81279 dashboard ceph txt unknown /proc/28504/task/81279/exe (readlink: Permission denied) 2024-04-22T13:26:48.887 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81279 dashboard ceph NOFD /proc/28504/task/81279/fd (opendir: Permission denied) 2024-04-22T13:26:48.887 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81280 dashboard ceph cwd unknown /proc/28504/task/81280/cwd (readlink: Permission denied) 2024-04-22T13:26:48.887 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81280 dashboard ceph rtd unknown /proc/28504/task/81280/root (readlink: Permission denied) 2024-04-22T13:26:48.887 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81280 dashboard ceph txt unknown /proc/28504/task/81280/exe (readlink: Permission denied) 2024-04-22T13:26:48.887 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81280 dashboard ceph NOFD /proc/28504/task/81280/fd (opendir: Permission denied) 2024-04-22T13:26:48.887 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81281 dashboard ceph cwd unknown /proc/28504/task/81281/cwd (readlink: Permission denied) 2024-04-22T13:26:48.888 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81281 dashboard ceph rtd unknown /proc/28504/task/81281/root (readlink: Permission denied) 2024-04-22T13:26:48.888 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81281 dashboard ceph txt unknown /proc/28504/task/81281/exe (readlink: Permission denied) 2024-04-22T13:26:48.888 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81281 dashboard ceph NOFD /proc/28504/task/81281/fd (opendir: Permission denied) 2024-04-22T13:26:48.888 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81282 dashboard ceph cwd unknown /proc/28504/task/81282/cwd (readlink: Permission denied) 2024-04-22T13:26:48.888 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81282 dashboard ceph rtd unknown /proc/28504/task/81282/root (readlink: Permission denied) 2024-04-22T13:26:48.888 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81282 dashboard ceph txt unknown /proc/28504/task/81282/exe (readlink: Permission denied) 2024-04-22T13:26:48.888 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81282 dashboard ceph NOFD /proc/28504/task/81282/fd (opendir: Permission denied) 2024-04-22T13:26:48.888 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81283 dashboard ceph cwd unknown /proc/28504/task/81283/cwd (readlink: Permission denied) 2024-04-22T13:26:48.888 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81283 dashboard ceph rtd unknown /proc/28504/task/81283/root (readlink: Permission denied) 2024-04-22T13:26:48.888 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81283 dashboard ceph txt unknown /proc/28504/task/81283/exe (readlink: Permission denied) 2024-04-22T13:26:48.888 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81283 dashboard ceph NOFD /proc/28504/task/81283/fd (opendir: Permission denied) 2024-04-22T13:26:48.888 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81284 dashboard ceph cwd unknown /proc/28504/task/81284/cwd (readlink: Permission denied) 2024-04-22T13:26:48.888 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81284 dashboard ceph rtd unknown /proc/28504/task/81284/root (readlink: Permission denied) 2024-04-22T13:26:48.888 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81284 dashboard ceph txt unknown /proc/28504/task/81284/exe (readlink: Permission denied) 2024-04-22T13:26:48.889 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81284 dashboard ceph NOFD /proc/28504/task/81284/fd (opendir: Permission denied) 2024-04-22T13:26:48.889 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81285 dashboard ceph cwd unknown /proc/28504/task/81285/cwd (readlink: Permission denied) 2024-04-22T13:26:48.889 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81285 dashboard ceph rtd unknown /proc/28504/task/81285/root (readlink: Permission denied) 2024-04-22T13:26:48.889 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81285 dashboard ceph txt unknown /proc/28504/task/81285/exe (readlink: Permission denied) 2024-04-22T13:26:48.889 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81285 dashboard ceph NOFD /proc/28504/task/81285/fd (opendir: Permission denied) 2024-04-22T13:26:48.889 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81286 dashboard ceph cwd unknown /proc/28504/task/81286/cwd (readlink: Permission denied) 2024-04-22T13:26:48.889 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81286 dashboard ceph rtd unknown /proc/28504/task/81286/root (readlink: Permission denied) 2024-04-22T13:26:48.889 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81286 dashboard ceph txt unknown /proc/28504/task/81286/exe (readlink: Permission denied) 2024-04-22T13:26:48.889 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81286 dashboard ceph NOFD /proc/28504/task/81286/fd (opendir: Permission denied) 2024-04-22T13:26:48.889 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81314 mgr-fin ceph cwd unknown /proc/28504/task/81314/cwd (readlink: Permission denied) 2024-04-22T13:26:48.889 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81314 mgr-fin ceph rtd unknown /proc/28504/task/81314/root (readlink: Permission denied) 2024-04-22T13:26:48.889 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81314 mgr-fin ceph txt unknown /proc/28504/task/81314/exe (readlink: Permission denied) 2024-04-22T13:26:48.889 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81314 mgr-fin ceph NOFD /proc/28504/task/81314/fd (opendir: Permission denied) 2024-04-22T13:26:48.889 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81317 mgr-fin ceph cwd unknown /proc/28504/task/81317/cwd (readlink: Permission denied) 2024-04-22T13:26:48.889 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81317 mgr-fin ceph rtd unknown /proc/28504/task/81317/root (readlink: Permission denied) 2024-04-22T13:26:48.890 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81317 mgr-fin ceph txt unknown /proc/28504/task/81317/exe (readlink: Permission denied) 2024-04-22T13:26:48.890 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81317 mgr-fin ceph NOFD /proc/28504/task/81317/fd (opendir: Permission denied) 2024-04-22T13:26:48.890 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81318 mgr-fin ceph cwd unknown /proc/28504/task/81318/cwd (readlink: Permission denied) 2024-04-22T13:26:48.890 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81318 mgr-fin ceph rtd unknown /proc/28504/task/81318/root (readlink: Permission denied) 2024-04-22T13:26:48.890 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81318 mgr-fin ceph txt unknown /proc/28504/task/81318/exe (readlink: Permission denied) 2024-04-22T13:26:48.890 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81318 mgr-fin ceph NOFD /proc/28504/task/81318/fd (opendir: Permission denied) 2024-04-22T13:26:48.890 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81320 mgr-fin ceph cwd unknown /proc/28504/task/81320/cwd (readlink: Permission denied) 2024-04-22T13:26:48.890 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81320 mgr-fin ceph rtd unknown /proc/28504/task/81320/root (readlink: Permission denied) 2024-04-22T13:26:48.890 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81320 mgr-fin ceph txt unknown /proc/28504/task/81320/exe (readlink: Permission denied) 2024-04-22T13:26:48.890 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81320 mgr-fin ceph NOFD /proc/28504/task/81320/fd (opendir: Permission denied) 2024-04-22T13:26:48.890 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81322 mgr-fin ceph cwd unknown /proc/28504/task/81322/cwd (readlink: Permission denied) 2024-04-22T13:26:48.890 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81322 mgr-fin ceph rtd unknown /proc/28504/task/81322/root (readlink: Permission denied) 2024-04-22T13:26:48.890 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81322 mgr-fin ceph txt unknown /proc/28504/task/81322/exe (readlink: Permission denied) 2024-04-22T13:26:48.890 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81322 mgr-fin ceph NOFD /proc/28504/task/81322/fd (opendir: Permission denied) 2024-04-22T13:26:48.890 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81323 mgr-fin ceph cwd unknown /proc/28504/task/81323/cwd (readlink: Permission denied) 2024-04-22T13:26:48.891 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81323 mgr-fin ceph rtd unknown /proc/28504/task/81323/root (readlink: Permission denied) 2024-04-22T13:26:48.891 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81323 mgr-fin ceph txt unknown /proc/28504/task/81323/exe (readlink: Permission denied) 2024-04-22T13:26:48.891 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81323 mgr-fin ceph NOFD /proc/28504/task/81323/fd (opendir: Permission denied) 2024-04-22T13:26:48.891 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81324 mgr-fin ceph cwd unknown /proc/28504/task/81324/cwd (readlink: Permission denied) 2024-04-22T13:26:48.891 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81324 mgr-fin ceph rtd unknown /proc/28504/task/81324/root (readlink: Permission denied) 2024-04-22T13:26:48.891 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81324 mgr-fin ceph txt unknown /proc/28504/task/81324/exe (readlink: Permission denied) 2024-04-22T13:26:48.891 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81324 mgr-fin ceph NOFD /proc/28504/task/81324/fd (opendir: Permission denied) 2024-04-22T13:26:48.891 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81326 mgr-fin ceph cwd unknown /proc/28504/task/81326/cwd (readlink: Permission denied) 2024-04-22T13:26:48.891 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81326 mgr-fin ceph rtd unknown /proc/28504/task/81326/root (readlink: Permission denied) 2024-04-22T13:26:48.891 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81326 mgr-fin ceph txt unknown /proc/28504/task/81326/exe (readlink: Permission denied) 2024-04-22T13:26:48.891 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81326 mgr-fin ceph NOFD /proc/28504/task/81326/fd (opendir: Permission denied) 2024-04-22T13:26:48.891 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81327 mgr-fin ceph cwd unknown /proc/28504/task/81327/cwd (readlink: Permission denied) 2024-04-22T13:26:48.891 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81327 mgr-fin ceph rtd unknown /proc/28504/task/81327/root (readlink: Permission denied) 2024-04-22T13:26:48.891 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81327 mgr-fin ceph txt unknown /proc/28504/task/81327/exe (readlink: Permission denied) 2024-04-22T13:26:48.891 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81327 mgr-fin ceph NOFD /proc/28504/task/81327/fd (opendir: Permission denied) 2024-04-22T13:26:48.892 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81328 mgr-fin ceph cwd unknown /proc/28504/task/81328/cwd (readlink: Permission denied) 2024-04-22T13:26:48.892 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81328 mgr-fin ceph rtd unknown /proc/28504/task/81328/root (readlink: Permission denied) 2024-04-22T13:26:48.892 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81328 mgr-fin ceph txt unknown /proc/28504/task/81328/exe (readlink: Permission denied) 2024-04-22T13:26:48.892 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81328 mgr-fin ceph NOFD /proc/28504/task/81328/fd (opendir: Permission denied) 2024-04-22T13:26:48.892 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81329 mgr-fin ceph cwd unknown /proc/28504/task/81329/cwd (readlink: Permission denied) 2024-04-22T13:26:48.892 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81329 mgr-fin ceph rtd unknown /proc/28504/task/81329/root (readlink: Permission denied) 2024-04-22T13:26:48.892 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81329 mgr-fin ceph txt unknown /proc/28504/task/81329/exe (readlink: Permission denied) 2024-04-22T13:26:48.892 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81329 mgr-fin ceph NOFD /proc/28504/task/81329/fd (opendir: Permission denied) 2024-04-22T13:26:48.892 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81348 dashboard ceph cwd unknown /proc/28504/task/81348/cwd (readlink: Permission denied) 2024-04-22T13:26:48.892 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81348 dashboard ceph rtd unknown /proc/28504/task/81348/root (readlink: Permission denied) 2024-04-22T13:26:48.892 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81348 dashboard ceph txt unknown /proc/28504/task/81348/exe (readlink: Permission denied) 2024-04-22T13:26:48.892 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81348 dashboard ceph NOFD /proc/28504/task/81348/fd (opendir: Permission denied) 2024-04-22T13:26:48.892 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81350 mgr-fin ceph cwd unknown /proc/28504/task/81350/cwd (readlink: Permission denied) 2024-04-22T13:26:48.892 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81350 mgr-fin ceph rtd unknown /proc/28504/task/81350/root (readlink: Permission denied) 2024-04-22T13:26:48.892 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81350 mgr-fin ceph txt unknown /proc/28504/task/81350/exe (readlink: Permission denied) 2024-04-22T13:26:48.893 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81350 mgr-fin ceph NOFD /proc/28504/task/81350/fd (opendir: Permission denied) 2024-04-22T13:26:48.893 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81351 mgr-fin ceph cwd unknown /proc/28504/task/81351/cwd (readlink: Permission denied) 2024-04-22T13:26:48.893 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81351 mgr-fin ceph rtd unknown /proc/28504/task/81351/root (readlink: Permission denied) 2024-04-22T13:26:48.893 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81351 mgr-fin ceph txt unknown /proc/28504/task/81351/exe (readlink: Permission denied) 2024-04-22T13:26:48.893 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81351 mgr-fin ceph NOFD /proc/28504/task/81351/fd (opendir: Permission denied) 2024-04-22T13:26:48.893 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81352 mgr-fin ceph cwd unknown /proc/28504/task/81352/cwd (readlink: Permission denied) 2024-04-22T13:26:48.893 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81352 mgr-fin ceph rtd unknown /proc/28504/task/81352/root (readlink: Permission denied) 2024-04-22T13:26:48.893 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81352 mgr-fin ceph txt unknown /proc/28504/task/81352/exe (readlink: Permission denied) 2024-04-22T13:26:48.893 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81352 mgr-fin ceph NOFD /proc/28504/task/81352/fd (opendir: Permission denied) 2024-04-22T13:26:48.893 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81353 mgr-fin ceph cwd unknown /proc/28504/task/81353/cwd (readlink: Permission denied) 2024-04-22T13:26:48.893 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81353 mgr-fin ceph rtd unknown /proc/28504/task/81353/root (readlink: Permission denied) 2024-04-22T13:26:48.893 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81353 mgr-fin ceph txt unknown /proc/28504/task/81353/exe (readlink: Permission denied) 2024-04-22T13:26:48.893 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81353 mgr-fin ceph NOFD /proc/28504/task/81353/fd (opendir: Permission denied) 2024-04-22T13:26:48.893 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81354 mgr-fin ceph cwd unknown /proc/28504/task/81354/cwd (readlink: Permission denied) 2024-04-22T13:26:48.893 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81354 mgr-fin ceph rtd unknown /proc/28504/task/81354/root (readlink: Permission denied) 2024-04-22T13:26:48.893 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81354 mgr-fin ceph txt unknown /proc/28504/task/81354/exe (readlink: Permission denied) 2024-04-22T13:26:48.894 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81354 mgr-fin ceph NOFD /proc/28504/task/81354/fd (opendir: Permission denied) 2024-04-22T13:26:48.894 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81355 mgr-fin ceph cwd unknown /proc/28504/task/81355/cwd (readlink: Permission denied) 2024-04-22T13:26:48.894 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81355 mgr-fin ceph rtd unknown /proc/28504/task/81355/root (readlink: Permission denied) 2024-04-22T13:26:48.894 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81355 mgr-fin ceph txt unknown /proc/28504/task/81355/exe (readlink: Permission denied) 2024-04-22T13:26:48.894 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81355 mgr-fin ceph NOFD /proc/28504/task/81355/fd (opendir: Permission denied) 2024-04-22T13:26:48.894 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81356 mgr-fin ceph cwd unknown /proc/28504/task/81356/cwd (readlink: Permission denied) 2024-04-22T13:26:48.894 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81356 mgr-fin ceph rtd unknown /proc/28504/task/81356/root (readlink: Permission denied) 2024-04-22T13:26:48.894 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81356 mgr-fin ceph txt unknown /proc/28504/task/81356/exe (readlink: Permission denied) 2024-04-22T13:26:48.894 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81356 mgr-fin ceph NOFD /proc/28504/task/81356/fd (opendir: Permission denied) 2024-04-22T13:26:48.894 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81357 mgr-fin ceph cwd unknown /proc/28504/task/81357/cwd (readlink: Permission denied) 2024-04-22T13:26:48.894 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81357 mgr-fin ceph rtd unknown /proc/28504/task/81357/root (readlink: Permission denied) 2024-04-22T13:26:48.894 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81357 mgr-fin ceph txt unknown /proc/28504/task/81357/exe (readlink: Permission denied) 2024-04-22T13:26:48.894 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81357 mgr-fin ceph NOFD /proc/28504/task/81357/fd (opendir: Permission denied) 2024-04-22T13:26:48.894 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81358 mgr-fin ceph cwd unknown /proc/28504/task/81358/cwd (readlink: Permission denied) 2024-04-22T13:26:48.894 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81358 mgr-fin ceph rtd unknown /proc/28504/task/81358/root (readlink: Permission denied) 2024-04-22T13:26:48.894 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81358 mgr-fin ceph txt unknown /proc/28504/task/81358/exe (readlink: Permission denied) 2024-04-22T13:26:48.895 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81358 mgr-fin ceph NOFD /proc/28504/task/81358/fd (opendir: Permission denied) 2024-04-22T13:26:48.895 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81359 mgr-fin ceph cwd unknown /proc/28504/task/81359/cwd (readlink: Permission denied) 2024-04-22T13:26:48.895 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81359 mgr-fin ceph rtd unknown /proc/28504/task/81359/root (readlink: Permission denied) 2024-04-22T13:26:48.895 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81359 mgr-fin ceph txt unknown /proc/28504/task/81359/exe (readlink: Permission denied) 2024-04-22T13:26:48.895 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81359 mgr-fin ceph NOFD /proc/28504/task/81359/fd (opendir: Permission denied) 2024-04-22T13:26:48.895 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81360 mgr-fin ceph cwd unknown /proc/28504/task/81360/cwd (readlink: Permission denied) 2024-04-22T13:26:48.895 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81360 mgr-fin ceph rtd unknown /proc/28504/task/81360/root (readlink: Permission denied) 2024-04-22T13:26:48.895 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81360 mgr-fin ceph txt unknown /proc/28504/task/81360/exe (readlink: Permission denied) 2024-04-22T13:26:48.895 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81360 mgr-fin ceph NOFD /proc/28504/task/81360/fd (opendir: Permission denied) 2024-04-22T13:26:48.895 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81756 devicehea ceph cwd unknown /proc/28504/task/81756/cwd (readlink: Permission denied) 2024-04-22T13:26:48.895 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81756 devicehea ceph rtd unknown /proc/28504/task/81756/root (readlink: Permission denied) 2024-04-22T13:26:48.895 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81756 devicehea ceph txt unknown /proc/28504/task/81756/exe (readlink: Permission denied) 2024-04-22T13:26:48.895 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81756 devicehea ceph NOFD /proc/28504/task/81756/fd (opendir: Permission denied) 2024-04-22T13:26:48.895 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81773 io_contex ceph cwd unknown /proc/28504/task/81773/cwd (readlink: Permission denied) 2024-04-22T13:26:48.895 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81773 io_contex ceph rtd unknown /proc/28504/task/81773/root (readlink: Permission denied) 2024-04-22T13:26:48.896 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81773 io_contex ceph txt unknown /proc/28504/task/81773/exe (readlink: Permission denied) 2024-04-22T13:26:48.896 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81773 io_contex ceph NOFD /proc/28504/task/81773/fd (opendir: Permission denied) 2024-04-22T13:26:48.896 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81774 io_contex ceph cwd unknown /proc/28504/task/81774/cwd (readlink: Permission denied) 2024-04-22T13:26:48.896 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81774 io_contex ceph rtd unknown /proc/28504/task/81774/root (readlink: Permission denied) 2024-04-22T13:26:48.896 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81774 io_contex ceph txt unknown /proc/28504/task/81774/exe (readlink: Permission denied) 2024-04-22T13:26:48.896 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81774 io_contex ceph NOFD /proc/28504/task/81774/fd (opendir: Permission denied) 2024-04-22T13:26:48.896 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81775 ceph_time ceph cwd unknown /proc/28504/task/81775/cwd (readlink: Permission denied) 2024-04-22T13:26:48.896 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81775 ceph_time ceph rtd unknown /proc/28504/task/81775/root (readlink: Permission denied) 2024-04-22T13:26:48.896 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81775 ceph_time ceph txt unknown /proc/28504/task/81775/exe (readlink: Permission denied) 2024-04-22T13:26:48.896 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81775 ceph_time ceph NOFD /proc/28504/task/81775/fd (opendir: Permission denied) 2024-04-22T13:26:48.896 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81776 ms_dispat ceph cwd unknown /proc/28504/task/81776/cwd (readlink: Permission denied) 2024-04-22T13:26:48.896 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81776 ms_dispat ceph rtd unknown /proc/28504/task/81776/root (readlink: Permission denied) 2024-04-22T13:26:48.896 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81776 ms_dispat ceph txt unknown /proc/28504/task/81776/exe (readlink: Permission denied) 2024-04-22T13:26:48.896 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81776 ms_dispat ceph NOFD /proc/28504/task/81776/fd (opendir: Permission denied) 2024-04-22T13:26:48.896 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81777 ms_local ceph cwd unknown /proc/28504/task/81777/cwd (readlink: Permission denied) 2024-04-22T13:26:48.896 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81777 ms_local ceph rtd unknown /proc/28504/task/81777/root (readlink: Permission denied) 2024-04-22T13:26:48.897 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81777 ms_local ceph txt unknown /proc/28504/task/81777/exe (readlink: Permission denied) 2024-04-22T13:26:48.897 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81777 ms_local ceph NOFD /proc/28504/task/81777/fd (opendir: Permission denied) 2024-04-22T13:26:48.897 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81778 safe_time ceph cwd unknown /proc/28504/task/81778/cwd (readlink: Permission denied) 2024-04-22T13:26:48.897 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81778 safe_time ceph rtd unknown /proc/28504/task/81778/root (readlink: Permission denied) 2024-04-22T13:26:48.897 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81778 safe_time ceph txt unknown /proc/28504/task/81778/exe (readlink: Permission denied) 2024-04-22T13:26:48.897 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81778 safe_time ceph NOFD /proc/28504/task/81778/fd (opendir: Permission denied) 2024-04-22T13:26:48.897 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81779 safe_time ceph cwd unknown /proc/28504/task/81779/cwd (readlink: Permission denied) 2024-04-22T13:26:48.897 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81779 safe_time ceph rtd unknown /proc/28504/task/81779/root (readlink: Permission denied) 2024-04-22T13:26:48.897 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81779 safe_time ceph txt unknown /proc/28504/task/81779/exe (readlink: Permission denied) 2024-04-22T13:26:48.897 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 81779 safe_time ceph NOFD /proc/28504/task/81779/fd (opendir: Permission denied) 2024-04-22T13:26:48.897 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85789 ceph_time ceph cwd unknown /proc/28504/task/85789/cwd (readlink: Permission denied) 2024-04-22T13:26:48.897 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85789 ceph_time ceph rtd unknown /proc/28504/task/85789/root (readlink: Permission denied) 2024-04-22T13:26:48.897 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85789 ceph_time ceph txt unknown /proc/28504/task/85789/exe (readlink: Permission denied) 2024-04-22T13:26:48.897 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85789 ceph_time ceph NOFD /proc/28504/task/85789/fd (opendir: Permission denied) 2024-04-22T13:26:48.897 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85790 fn_anonym ceph cwd unknown /proc/28504/task/85790/cwd (readlink: Permission denied) 2024-04-22T13:26:48.897 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85790 fn_anonym ceph rtd unknown /proc/28504/task/85790/root (readlink: Permission denied) 2024-04-22T13:26:48.897 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85790 fn_anonym ceph txt unknown /proc/28504/task/85790/exe (readlink: Permission denied) 2024-04-22T13:26:48.898 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85790 fn_anonym ceph NOFD /proc/28504/task/85790/fd (opendir: Permission denied) 2024-04-22T13:26:48.898 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85791 safe_time ceph cwd unknown /proc/28504/task/85791/cwd (readlink: Permission denied) 2024-04-22T13:26:48.898 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85791 safe_time ceph rtd unknown /proc/28504/task/85791/root (readlink: Permission denied) 2024-04-22T13:26:48.898 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85791 safe_time ceph txt unknown /proc/28504/task/85791/exe (readlink: Permission denied) 2024-04-22T13:26:48.898 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85791 safe_time ceph NOFD /proc/28504/task/85791/fd (opendir: Permission denied) 2024-04-22T13:26:48.898 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85792 fn_anonym ceph cwd unknown /proc/28504/task/85792/cwd (readlink: Permission denied) 2024-04-22T13:26:48.898 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85792 fn_anonym ceph rtd unknown /proc/28504/task/85792/root (readlink: Permission denied) 2024-04-22T13:26:48.898 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85792 fn_anonym ceph txt unknown /proc/28504/task/85792/exe (readlink: Permission denied) 2024-04-22T13:26:48.898 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85792 fn_anonym ceph NOFD /proc/28504/task/85792/fd (opendir: Permission denied) 2024-04-22T13:26:48.898 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85793 flusher ceph cwd unknown /proc/28504/task/85793/cwd (readlink: Permission denied) 2024-04-22T13:26:48.898 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85793 flusher ceph rtd unknown /proc/28504/task/85793/root (readlink: Permission denied) 2024-04-22T13:26:48.898 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85793 flusher ceph txt unknown /proc/28504/task/85793/exe (readlink: Permission denied) 2024-04-22T13:26:48.898 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85793 flusher ceph NOFD /proc/28504/task/85793/fd (opendir: Permission denied) 2024-04-22T13:26:48.898 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85794 ms_dispat ceph cwd unknown /proc/28504/task/85794/cwd (readlink: Permission denied) 2024-04-22T13:26:48.898 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85794 ms_dispat ceph rtd unknown /proc/28504/task/85794/root (readlink: Permission denied) 2024-04-22T13:26:48.898 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85794 ms_dispat ceph txt unknown /proc/28504/task/85794/exe (readlink: Permission denied) 2024-04-22T13:26:48.899 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85794 ms_dispat ceph NOFD /proc/28504/task/85794/fd (opendir: Permission denied) 2024-04-22T13:26:48.899 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85795 ms_local ceph cwd unknown /proc/28504/task/85795/cwd (readlink: Permission denied) 2024-04-22T13:26:48.899 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85795 ms_local ceph rtd unknown /proc/28504/task/85795/root (readlink: Permission denied) 2024-04-22T13:26:48.899 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85795 ms_local ceph txt unknown /proc/28504/task/85795/exe (readlink: Permission denied) 2024-04-22T13:26:48.899 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85795 ms_local ceph NOFD /proc/28504/task/85795/fd (opendir: Permission denied) 2024-04-22T13:26:48.899 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85796 safe_time ceph cwd unknown /proc/28504/task/85796/cwd (readlink: Permission denied) 2024-04-22T13:26:48.899 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85796 safe_time ceph rtd unknown /proc/28504/task/85796/root (readlink: Permission denied) 2024-04-22T13:26:48.899 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85796 safe_time ceph txt unknown /proc/28504/task/85796/exe (readlink: Permission denied) 2024-04-22T13:26:48.899 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85796 safe_time ceph NOFD /proc/28504/task/85796/fd (opendir: Permission denied) 2024-04-22T13:26:48.899 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85797 m-fin-sna ceph cwd unknown /proc/28504/task/85797/cwd (readlink: Permission denied) 2024-04-22T13:26:48.899 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85797 m-fin-sna ceph rtd unknown /proc/28504/task/85797/root (readlink: Permission denied) 2024-04-22T13:26:48.899 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85797 m-fin-sna ceph txt unknown /proc/28504/task/85797/exe (readlink: Permission denied) 2024-04-22T13:26:48.899 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 85797 m-fin-sna ceph NOFD /proc/28504/task/85797/fd (opendir: Permission denied) 2024-04-22T13:26:48.899 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 87827 m-fin-sna ceph cwd unknown /proc/28504/task/87827/cwd (readlink: Permission denied) 2024-04-22T13:26:48.900 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 87827 m-fin-sna ceph rtd unknown /proc/28504/task/87827/root (readlink: Permission denied) 2024-04-22T13:26:48.900 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 87827 m-fin-sna ceph txt unknown /proc/28504/task/87827/exe (readlink: Permission denied) 2024-04-22T13:26:48.900 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mgr 28504 87827 m-fin-sna ceph NOFD /proc/28504/task/87827/fd (opendir: Permission denied) 2024-04-22T13:26:48.900 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 29639 root cwd unknown /proc/29639/cwd (readlink: Permission denied) 2024-04-22T13:26:48.900 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 29639 root rtd unknown /proc/29639/root (readlink: Permission denied) 2024-04-22T13:26:48.900 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 29639 root txt unknown /proc/29639/exe (readlink: Permission denied) 2024-04-22T13:26:48.900 INFO:tasks.workunit.client.0.smithi012.stdout:systemd 29639 root NOFD /proc/29639/fd (opendir: Permission denied) 2024-04-22T13:26:48.900 INFO:tasks.workunit.client.0.smithi012.stdout:(sd-pam) 29641 root cwd unknown /proc/29641/cwd (readlink: Permission denied) 2024-04-22T13:26:48.900 INFO:tasks.workunit.client.0.smithi012.stdout:(sd-pam) 29641 root rtd unknown /proc/29641/root (readlink: Permission denied) 2024-04-22T13:26:48.900 INFO:tasks.workunit.client.0.smithi012.stdout:(sd-pam) 29641 root txt unknown /proc/29641/exe (readlink: Permission denied) 2024-04-22T13:26:48.900 INFO:tasks.workunit.client.0.smithi012.stdout:(sd-pam) 29641 root NOFD /proc/29641/fd (opendir: Permission denied) 2024-04-22T13:26:48.900 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/2 31158 root cwd unknown /proc/31158/cwd (readlink: Permission denied) 2024-04-22T13:26:48.900 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/2 31158 root rtd unknown /proc/31158/root (readlink: Permission denied) 2024-04-22T13:26:48.900 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/2 31158 root txt unknown /proc/31158/exe (readlink: Permission denied) 2024-04-22T13:26:48.900 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/2 31158 root NOFD /proc/31158/fd (opendir: Permission denied) 2024-04-22T13:26:48.900 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/5 38095 root cwd unknown /proc/38095/cwd (readlink: Permission denied) 2024-04-22T13:26:48.901 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/5 38095 root rtd unknown /proc/38095/root (readlink: Permission denied) 2024-04-22T13:26:48.901 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/5 38095 root txt unknown /proc/38095/exe (readlink: Permission denied) 2024-04-22T13:26:48.901 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/5 38095 root NOFD /proc/38095/fd (opendir: Permission denied) 2024-04-22T13:26:48.901 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 42725 root cwd unknown /proc/42725/cwd (readlink: Permission denied) 2024-04-22T13:26:48.901 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 42725 root rtd unknown /proc/42725/root (readlink: Permission denied) 2024-04-22T13:26:48.901 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 42725 root txt unknown /proc/42725/exe (readlink: Permission denied) 2024-04-22T13:26:48.901 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 42725 root NOFD /proc/42725/fd (opendir: Permission denied) 2024-04-22T13:26:48.901 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 42727 root cwd unknown /proc/42727/cwd (readlink: Permission denied) 2024-04-22T13:26:48.901 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 42727 root rtd unknown /proc/42727/root (readlink: Permission denied) 2024-04-22T13:26:48.901 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 42727 root txt unknown /proc/42727/exe (readlink: Permission denied) 2024-04-22T13:26:48.901 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 42727 root NOFD /proc/42727/fd (opendir: Permission denied) 2024-04-22T13:26:48.901 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 ceph cwd unknown /proc/42729/cwd (readlink: Permission denied) 2024-04-22T13:26:48.901 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 ceph rtd unknown /proc/42729/root (readlink: Permission denied) 2024-04-22T13:26:48.901 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 ceph txt unknown /proc/42729/exe (readlink: Permission denied) 2024-04-22T13:26:48.901 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 ceph NOFD /proc/42729/fd (opendir: Permission denied) 2024-04-22T13:26:48.901 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42730 log ceph cwd unknown /proc/42729/task/42730/cwd (readlink: Permission denied) 2024-04-22T13:26:48.901 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42730 log ceph rtd unknown /proc/42729/task/42730/root (readlink: Permission denied) 2024-04-22T13:26:48.902 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42730 log ceph txt unknown /proc/42729/task/42730/exe (readlink: Permission denied) 2024-04-22T13:26:48.902 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42730 log ceph NOFD /proc/42729/task/42730/fd (opendir: Permission denied) 2024-04-22T13:26:48.902 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42732 msgr-work ceph cwd unknown /proc/42729/task/42732/cwd (readlink: Permission denied) 2024-04-22T13:26:48.902 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42732 msgr-work ceph rtd unknown /proc/42729/task/42732/root (readlink: Permission denied) 2024-04-22T13:26:48.902 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42732 msgr-work ceph txt unknown /proc/42729/task/42732/exe (readlink: Permission denied) 2024-04-22T13:26:48.902 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42732 msgr-work ceph NOFD /proc/42729/task/42732/fd (opendir: Permission denied) 2024-04-22T13:26:48.902 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42733 msgr-work ceph cwd unknown /proc/42729/task/42733/cwd (readlink: Permission denied) 2024-04-22T13:26:48.902 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42733 msgr-work ceph rtd unknown /proc/42729/task/42733/root (readlink: Permission denied) 2024-04-22T13:26:48.902 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42733 msgr-work ceph txt unknown /proc/42729/task/42733/exe (readlink: Permission denied) 2024-04-22T13:26:48.902 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42733 msgr-work ceph NOFD /proc/42729/task/42733/fd (opendir: Permission denied) 2024-04-22T13:26:48.902 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42734 msgr-work ceph cwd unknown /proc/42729/task/42734/cwd (readlink: Permission denied) 2024-04-22T13:26:48.902 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42734 msgr-work ceph rtd unknown /proc/42729/task/42734/root (readlink: Permission denied) 2024-04-22T13:26:48.902 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42734 msgr-work ceph txt unknown /proc/42729/task/42734/exe (readlink: Permission denied) 2024-04-22T13:26:48.902 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42734 msgr-work ceph NOFD /proc/42729/task/42734/fd (opendir: Permission denied) 2024-04-22T13:26:48.903 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42738 service ceph cwd unknown /proc/42729/task/42738/cwd (readlink: Permission denied) 2024-04-22T13:26:48.903 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42738 service ceph rtd unknown /proc/42729/task/42738/root (readlink: Permission denied) 2024-04-22T13:26:48.903 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42738 service ceph txt unknown /proc/42729/task/42738/exe (readlink: Permission denied) 2024-04-22T13:26:48.903 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42738 service ceph NOFD /proc/42729/task/42738/fd (opendir: Permission denied) 2024-04-22T13:26:48.903 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42739 admin_soc ceph cwd unknown /proc/42729/task/42739/cwd (readlink: Permission denied) 2024-04-22T13:26:48.903 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42739 admin_soc ceph rtd unknown /proc/42729/task/42739/root (readlink: Permission denied) 2024-04-22T13:26:48.903 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42739 admin_soc ceph txt unknown /proc/42729/task/42739/exe (readlink: Permission denied) 2024-04-22T13:26:48.903 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42739 admin_soc ceph NOFD /proc/42729/task/42739/fd (opendir: Permission denied) 2024-04-22T13:26:48.903 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42881 signal_ha ceph cwd unknown /proc/42729/task/42881/cwd (readlink: Permission denied) 2024-04-22T13:26:48.903 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42881 signal_ha ceph rtd unknown /proc/42729/task/42881/root (readlink: Permission denied) 2024-04-22T13:26:48.903 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42881 signal_ha ceph txt unknown /proc/42729/task/42881/exe (readlink: Permission denied) 2024-04-22T13:26:48.903 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42881 signal_ha ceph NOFD /proc/42729/task/42881/fd (opendir: Permission denied) 2024-04-22T13:26:48.903 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42882 io_contex ceph cwd unknown /proc/42729/task/42882/cwd (readlink: Permission denied) 2024-04-22T13:26:48.903 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42882 io_contex ceph rtd unknown /proc/42729/task/42882/root (readlink: Permission denied) 2024-04-22T13:26:48.903 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42882 io_contex ceph txt unknown /proc/42729/task/42882/exe (readlink: Permission denied) 2024-04-22T13:26:48.903 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42882 io_contex ceph NOFD /proc/42729/task/42882/fd (opendir: Permission denied) 2024-04-22T13:26:48.904 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42883 io_contex ceph cwd unknown /proc/42729/task/42883/cwd (readlink: Permission denied) 2024-04-22T13:26:48.904 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42883 io_contex ceph rtd unknown /proc/42729/task/42883/root (readlink: Permission denied) 2024-04-22T13:26:48.904 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42883 io_contex ceph txt unknown /proc/42729/task/42883/exe (readlink: Permission denied) 2024-04-22T13:26:48.904 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42883 io_contex ceph NOFD /proc/42729/task/42883/fd (opendir: Permission denied) 2024-04-22T13:26:48.904 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42995 OpHistory ceph cwd unknown /proc/42729/task/42995/cwd (readlink: Permission denied) 2024-04-22T13:26:48.904 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42995 OpHistory ceph rtd unknown /proc/42729/task/42995/root (readlink: Permission denied) 2024-04-22T13:26:48.904 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42995 OpHistory ceph txt unknown /proc/42729/task/42995/exe (readlink: Permission denied) 2024-04-22T13:26:48.904 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42995 OpHistory ceph NOFD /proc/42729/task/42995/fd (opendir: Permission denied) 2024-04-22T13:26:48.904 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42996 ceph_time ceph cwd unknown /proc/42729/task/42996/cwd (readlink: Permission denied) 2024-04-22T13:26:48.904 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42996 ceph_time ceph rtd unknown /proc/42729/task/42996/root (readlink: Permission denied) 2024-04-22T13:26:48.904 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42996 ceph_time ceph txt unknown /proc/42729/task/42996/exe (readlink: Permission denied) 2024-04-22T13:26:48.904 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 42996 ceph_time ceph NOFD /proc/42729/task/42996/fd (opendir: Permission denied) 2024-04-22T13:26:48.904 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43187 ceph-osd ceph cwd unknown /proc/42729/task/43187/cwd (readlink: Permission denied) 2024-04-22T13:26:48.904 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43187 ceph-osd ceph rtd unknown /proc/42729/task/43187/root (readlink: Permission denied) 2024-04-22T13:26:48.904 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43187 ceph-osd ceph txt unknown /proc/42729/task/43187/exe (readlink: Permission denied) 2024-04-22T13:26:48.904 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43187 ceph-osd ceph NOFD /proc/42729/task/43187/fd (opendir: Permission denied) 2024-04-22T13:26:48.904 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43188 safe_time ceph cwd unknown /proc/42729/task/43188/cwd (readlink: Permission denied) 2024-04-22T13:26:48.905 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43188 safe_time ceph rtd unknown /proc/42729/task/43188/root (readlink: Permission denied) 2024-04-22T13:26:48.905 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43188 safe_time ceph txt unknown /proc/42729/task/43188/exe (readlink: Permission denied) 2024-04-22T13:26:48.905 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43188 safe_time ceph NOFD /proc/42729/task/43188/fd (opendir: Permission denied) 2024-04-22T13:26:48.905 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43189 safe_time ceph cwd unknown /proc/42729/task/43189/cwd (readlink: Permission denied) 2024-04-22T13:26:48.905 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43189 safe_time ceph rtd unknown /proc/42729/task/43189/root (readlink: Permission denied) 2024-04-22T13:26:48.905 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43189 safe_time ceph txt unknown /proc/42729/task/43189/exe (readlink: Permission denied) 2024-04-22T13:26:48.905 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43189 safe_time ceph NOFD /proc/42729/task/43189/fd (opendir: Permission denied) 2024-04-22T13:26:48.905 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43190 safe_time ceph cwd unknown /proc/42729/task/43190/cwd (readlink: Permission denied) 2024-04-22T13:26:48.905 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43190 safe_time ceph rtd unknown /proc/42729/task/43190/root (readlink: Permission denied) 2024-04-22T13:26:48.905 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43190 safe_time ceph txt unknown /proc/42729/task/43190/exe (readlink: Permission denied) 2024-04-22T13:26:48.905 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43190 safe_time ceph NOFD /proc/42729/task/43190/fd (opendir: Permission denied) 2024-04-22T13:26:48.905 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43191 safe_time ceph cwd unknown /proc/42729/task/43191/cwd (readlink: Permission denied) 2024-04-22T13:26:48.905 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43191 safe_time ceph rtd unknown /proc/42729/task/43191/root (readlink: Permission denied) 2024-04-22T13:26:48.905 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43191 safe_time ceph txt unknown /proc/42729/task/43191/exe (readlink: Permission denied) 2024-04-22T13:26:48.905 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43191 safe_time ceph NOFD /proc/42729/task/43191/fd (opendir: Permission denied) 2024-04-22T13:26:48.905 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43192 fn_anonym ceph cwd unknown /proc/42729/task/43192/cwd (readlink: Permission denied) 2024-04-22T13:26:48.905 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43192 fn_anonym ceph rtd unknown /proc/42729/task/43192/root (readlink: Permission denied) 2024-04-22T13:26:48.906 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43192 fn_anonym ceph txt unknown /proc/42729/task/43192/exe (readlink: Permission denied) 2024-04-22T13:26:48.906 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43192 fn_anonym ceph NOFD /proc/42729/task/43192/fd (opendir: Permission denied) 2024-04-22T13:26:48.906 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43197 rocksdb:l ceph cwd unknown /proc/42729/task/43197/cwd (readlink: Permission denied) 2024-04-22T13:26:48.906 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43197 rocksdb:l ceph rtd unknown /proc/42729/task/43197/root (readlink: Permission denied) 2024-04-22T13:26:48.906 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43197 rocksdb:l ceph txt unknown /proc/42729/task/43197/exe (readlink: Permission denied) 2024-04-22T13:26:48.906 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43197 rocksdb:l ceph NOFD /proc/42729/task/43197/fd (opendir: Permission denied) 2024-04-22T13:26:48.906 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43198 rocksdb:l ceph cwd unknown /proc/42729/task/43198/cwd (readlink: Permission denied) 2024-04-22T13:26:48.906 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43198 rocksdb:l ceph rtd unknown /proc/42729/task/43198/root (readlink: Permission denied) 2024-04-22T13:26:48.906 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43198 rocksdb:l ceph txt unknown /proc/42729/task/43198/exe (readlink: Permission denied) 2024-04-22T13:26:48.906 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43198 rocksdb:l ceph NOFD /proc/42729/task/43198/fd (opendir: Permission denied) 2024-04-22T13:26:48.906 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43199 rocksdb:l ceph cwd unknown /proc/42729/task/43199/cwd (readlink: Permission denied) 2024-04-22T13:26:48.906 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43199 rocksdb:l ceph rtd unknown /proc/42729/task/43199/root (readlink: Permission denied) 2024-04-22T13:26:48.906 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43199 rocksdb:l ceph txt unknown /proc/42729/task/43199/exe (readlink: Permission denied) 2024-04-22T13:26:48.906 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43199 rocksdb:l ceph NOFD /proc/42729/task/43199/fd (opendir: Permission denied) 2024-04-22T13:26:48.907 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43200 rocksdb:h ceph cwd unknown /proc/42729/task/43200/cwd (readlink: Permission denied) 2024-04-22T13:26:48.907 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43200 rocksdb:h ceph rtd unknown /proc/42729/task/43200/root (readlink: Permission denied) 2024-04-22T13:26:48.907 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43200 rocksdb:h ceph txt unknown /proc/42729/task/43200/exe (readlink: Permission denied) 2024-04-22T13:26:48.907 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43200 rocksdb:h ceph NOFD /proc/42729/task/43200/fd (opendir: Permission denied) 2024-04-22T13:26:48.907 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43615 bstore_ai ceph cwd unknown /proc/42729/task/43615/cwd (readlink: Permission denied) 2024-04-22T13:26:48.907 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43615 bstore_ai ceph rtd unknown /proc/42729/task/43615/root (readlink: Permission denied) 2024-04-22T13:26:48.907 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43615 bstore_ai ceph txt unknown /proc/42729/task/43615/exe (readlink: Permission denied) 2024-04-22T13:26:48.907 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43615 bstore_ai ceph NOFD /proc/42729/task/43615/fd (opendir: Permission denied) 2024-04-22T13:26:48.907 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43877 bstore_ai ceph cwd unknown /proc/42729/task/43877/cwd (readlink: Permission denied) 2024-04-22T13:26:48.907 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43877 bstore_ai ceph rtd unknown /proc/42729/task/43877/root (readlink: Permission denied) 2024-04-22T13:26:48.907 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43877 bstore_ai ceph txt unknown /proc/42729/task/43877/exe (readlink: Permission denied) 2024-04-22T13:26:48.907 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 43877 bstore_ai ceph NOFD /proc/42729/task/43877/fd (opendir: Permission denied) 2024-04-22T13:26:48.907 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44060 ceph-osd ceph cwd unknown /proc/42729/task/44060/cwd (readlink: Permission denied) 2024-04-22T13:26:48.907 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44060 ceph-osd ceph rtd unknown /proc/42729/task/44060/root (readlink: Permission denied) 2024-04-22T13:26:48.907 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44060 ceph-osd ceph txt unknown /proc/42729/task/44060/exe (readlink: Permission denied) 2024-04-22T13:26:48.907 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44060 ceph-osd ceph NOFD /proc/42729/task/44060/fd (opendir: Permission denied) 2024-04-22T13:26:48.907 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44061 cfin ceph cwd unknown /proc/42729/task/44061/cwd (readlink: Permission denied) 2024-04-22T13:26:48.908 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44061 cfin ceph rtd unknown /proc/42729/task/44061/root (readlink: Permission denied) 2024-04-22T13:26:48.908 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44061 cfin ceph txt unknown /proc/42729/task/44061/exe (readlink: Permission denied) 2024-04-22T13:26:48.908 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44061 cfin ceph NOFD /proc/42729/task/44061/fd (opendir: Permission denied) 2024-04-22T13:26:48.908 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44062 bstore_kv ceph cwd unknown /proc/42729/task/44062/cwd (readlink: Permission denied) 2024-04-22T13:26:48.908 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44062 bstore_kv ceph rtd unknown /proc/42729/task/44062/root (readlink: Permission denied) 2024-04-22T13:26:48.908 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44062 bstore_kv ceph txt unknown /proc/42729/task/44062/exe (readlink: Permission denied) 2024-04-22T13:26:48.908 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44062 bstore_kv ceph NOFD /proc/42729/task/44062/fd (opendir: Permission denied) 2024-04-22T13:26:48.908 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44063 bstore_kv ceph cwd unknown /proc/42729/task/44063/cwd (readlink: Permission denied) 2024-04-22T13:26:48.908 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44063 bstore_kv ceph rtd unknown /proc/42729/task/44063/root (readlink: Permission denied) 2024-04-22T13:26:48.908 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44063 bstore_kv ceph txt unknown /proc/42729/task/44063/exe (readlink: Permission denied) 2024-04-22T13:26:48.908 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44063 bstore_kv ceph NOFD /proc/42729/task/44063/fd (opendir: Permission denied) 2024-04-22T13:26:48.908 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44064 bstore_me ceph cwd unknown /proc/42729/task/44064/cwd (readlink: Permission denied) 2024-04-22T13:26:48.908 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44064 bstore_me ceph rtd unknown /proc/42729/task/44064/root (readlink: Permission denied) 2024-04-22T13:26:48.909 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44064 bstore_me ceph txt unknown /proc/42729/task/44064/exe (readlink: Permission denied) 2024-04-22T13:26:48.909 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44064 bstore_me ceph NOFD /proc/42729/task/44064/fd (opendir: Permission denied) 2024-04-22T13:26:48.909 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44065 ms_dispat ceph cwd unknown /proc/42729/task/44065/cwd (readlink: Permission denied) 2024-04-22T13:26:48.909 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44065 ms_dispat ceph rtd unknown /proc/42729/task/44065/root (readlink: Permission denied) 2024-04-22T13:26:48.909 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44065 ms_dispat ceph txt unknown /proc/42729/task/44065/exe (readlink: Permission denied) 2024-04-22T13:26:48.909 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44065 ms_dispat ceph NOFD /proc/42729/task/44065/fd (opendir: Permission denied) 2024-04-22T13:26:48.909 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44066 ms_local ceph cwd unknown /proc/42729/task/44066/cwd (readlink: Permission denied) 2024-04-22T13:26:48.909 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44066 ms_local ceph rtd unknown /proc/42729/task/44066/root (readlink: Permission denied) 2024-04-22T13:26:48.909 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44066 ms_local ceph txt unknown /proc/42729/task/44066/exe (readlink: Permission denied) 2024-04-22T13:26:48.909 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44066 ms_local ceph NOFD /proc/42729/task/44066/fd (opendir: Permission denied) 2024-04-22T13:26:48.909 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44067 safe_time ceph cwd unknown /proc/42729/task/44067/cwd (readlink: Permission denied) 2024-04-22T13:26:48.909 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44067 safe_time ceph rtd unknown /proc/42729/task/44067/root (readlink: Permission denied) 2024-04-22T13:26:48.909 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44067 safe_time ceph txt unknown /proc/42729/task/44067/exe (readlink: Permission denied) 2024-04-22T13:26:48.909 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44067 safe_time ceph NOFD /proc/42729/task/44067/fd (opendir: Permission denied) 2024-04-22T13:26:48.909 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44068 safe_time ceph cwd unknown /proc/42729/task/44068/cwd (readlink: Permission denied) 2024-04-22T13:26:48.909 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44068 safe_time ceph rtd unknown /proc/42729/task/44068/root (readlink: Permission denied) 2024-04-22T13:26:48.909 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44068 safe_time ceph txt unknown /proc/42729/task/44068/exe (readlink: Permission denied) 2024-04-22T13:26:48.910 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44068 safe_time ceph NOFD /proc/42729/task/44068/fd (opendir: Permission denied) 2024-04-22T13:26:48.910 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44069 ms_dispat ceph cwd unknown /proc/42729/task/44069/cwd (readlink: Permission denied) 2024-04-22T13:26:48.910 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44069 ms_dispat ceph rtd unknown /proc/42729/task/44069/root (readlink: Permission denied) 2024-04-22T13:26:48.910 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44069 ms_dispat ceph txt unknown /proc/42729/task/44069/exe (readlink: Permission denied) 2024-04-22T13:26:48.910 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44069 ms_dispat ceph NOFD /proc/42729/task/44069/fd (opendir: Permission denied) 2024-04-22T13:26:48.910 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44070 ms_local ceph cwd unknown /proc/42729/task/44070/cwd (readlink: Permission denied) 2024-04-22T13:26:48.910 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44070 ms_local ceph rtd unknown /proc/42729/task/44070/root (readlink: Permission denied) 2024-04-22T13:26:48.910 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44070 ms_local ceph txt unknown /proc/42729/task/44070/exe (readlink: Permission denied) 2024-04-22T13:26:48.910 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44070 ms_local ceph NOFD /proc/42729/task/44070/fd (opendir: Permission denied) 2024-04-22T13:26:48.910 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44071 ms_dispat ceph cwd unknown /proc/42729/task/44071/cwd (readlink: Permission denied) 2024-04-22T13:26:48.910 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44071 ms_dispat ceph rtd unknown /proc/42729/task/44071/root (readlink: Permission denied) 2024-04-22T13:26:48.910 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44071 ms_dispat ceph txt unknown /proc/42729/task/44071/exe (readlink: Permission denied) 2024-04-22T13:26:48.910 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44071 ms_dispat ceph NOFD /proc/42729/task/44071/fd (opendir: Permission denied) 2024-04-22T13:26:48.910 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44072 ms_local ceph cwd unknown /proc/42729/task/44072/cwd (readlink: Permission denied) 2024-04-22T13:26:48.910 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44072 ms_local ceph rtd unknown /proc/42729/task/44072/root (readlink: Permission denied) 2024-04-22T13:26:48.911 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44072 ms_local ceph txt unknown /proc/42729/task/44072/exe (readlink: Permission denied) 2024-04-22T13:26:48.911 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44072 ms_local ceph NOFD /proc/42729/task/44072/fd (opendir: Permission denied) 2024-04-22T13:26:48.911 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44073 ms_dispat ceph cwd unknown /proc/42729/task/44073/cwd (readlink: Permission denied) 2024-04-22T13:26:48.911 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44073 ms_dispat ceph rtd unknown /proc/42729/task/44073/root (readlink: Permission denied) 2024-04-22T13:26:48.911 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44073 ms_dispat ceph txt unknown /proc/42729/task/44073/exe (readlink: Permission denied) 2024-04-22T13:26:48.911 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44073 ms_dispat ceph NOFD /proc/42729/task/44073/fd (opendir: Permission denied) 2024-04-22T13:26:48.911 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44074 ms_local ceph cwd unknown /proc/42729/task/44074/cwd (readlink: Permission denied) 2024-04-22T13:26:48.911 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44074 ms_local ceph rtd unknown /proc/42729/task/44074/root (readlink: Permission denied) 2024-04-22T13:26:48.911 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44074 ms_local ceph txt unknown /proc/42729/task/44074/exe (readlink: Permission denied) 2024-04-22T13:26:48.911 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44074 ms_local ceph NOFD /proc/42729/task/44074/fd (opendir: Permission denied) 2024-04-22T13:26:48.911 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44075 ms_dispat ceph cwd unknown /proc/42729/task/44075/cwd (readlink: Permission denied) 2024-04-22T13:26:48.911 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44075 ms_dispat ceph rtd unknown /proc/42729/task/44075/root (readlink: Permission denied) 2024-04-22T13:26:48.911 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44075 ms_dispat ceph txt unknown /proc/42729/task/44075/exe (readlink: Permission denied) 2024-04-22T13:26:48.911 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44075 ms_dispat ceph NOFD /proc/42729/task/44075/fd (opendir: Permission denied) 2024-04-22T13:26:48.911 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44076 ms_local ceph cwd unknown /proc/42729/task/44076/cwd (readlink: Permission denied) 2024-04-22T13:26:48.912 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44076 ms_local ceph rtd unknown /proc/42729/task/44076/root (readlink: Permission denied) 2024-04-22T13:26:48.912 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44076 ms_local ceph txt unknown /proc/42729/task/44076/exe (readlink: Permission denied) 2024-04-22T13:26:48.912 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44076 ms_local ceph NOFD /proc/42729/task/44076/fd (opendir: Permission denied) 2024-04-22T13:26:48.912 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44077 ms_dispat ceph cwd unknown /proc/42729/task/44077/cwd (readlink: Permission denied) 2024-04-22T13:26:48.912 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44077 ms_dispat ceph rtd unknown /proc/42729/task/44077/root (readlink: Permission denied) 2024-04-22T13:26:48.912 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44077 ms_dispat ceph txt unknown /proc/42729/task/44077/exe (readlink: Permission denied) 2024-04-22T13:26:48.912 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44077 ms_dispat ceph NOFD /proc/42729/task/44077/fd (opendir: Permission denied) 2024-04-22T13:26:48.912 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44078 ms_local ceph cwd unknown /proc/42729/task/44078/cwd (readlink: Permission denied) 2024-04-22T13:26:48.912 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44078 ms_local ceph rtd unknown /proc/42729/task/44078/root (readlink: Permission denied) 2024-04-22T13:26:48.912 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44078 ms_local ceph txt unknown /proc/42729/task/44078/exe (readlink: Permission denied) 2024-04-22T13:26:48.912 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44078 ms_local ceph NOFD /proc/42729/task/44078/fd (opendir: Permission denied) 2024-04-22T13:26:48.912 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44079 ms_dispat ceph cwd unknown /proc/42729/task/44079/cwd (readlink: Permission denied) 2024-04-22T13:26:48.912 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44079 ms_dispat ceph rtd unknown /proc/42729/task/44079/root (readlink: Permission denied) 2024-04-22T13:26:48.912 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44079 ms_dispat ceph txt unknown /proc/42729/task/44079/exe (readlink: Permission denied) 2024-04-22T13:26:48.912 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44079 ms_dispat ceph NOFD /proc/42729/task/44079/fd (opendir: Permission denied) 2024-04-22T13:26:48.912 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44080 ms_local ceph cwd unknown /proc/42729/task/44080/cwd (readlink: Permission denied) 2024-04-22T13:26:48.912 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44080 ms_local ceph rtd unknown /proc/42729/task/44080/root (readlink: Permission denied) 2024-04-22T13:26:48.913 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44080 ms_local ceph txt unknown /proc/42729/task/44080/exe (readlink: Permission denied) 2024-04-22T13:26:48.913 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44080 ms_local ceph NOFD /proc/42729/task/44080/fd (opendir: Permission denied) 2024-04-22T13:26:48.913 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44081 fn_anonym ceph cwd unknown /proc/42729/task/44081/cwd (readlink: Permission denied) 2024-04-22T13:26:48.913 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44081 fn_anonym ceph rtd unknown /proc/42729/task/44081/root (readlink: Permission denied) 2024-04-22T13:26:48.913 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44081 fn_anonym ceph txt unknown /proc/42729/task/44081/exe (readlink: Permission denied) 2024-04-22T13:26:48.913 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44081 fn_anonym ceph NOFD /proc/42729/task/44081/fd (opendir: Permission denied) 2024-04-22T13:26:48.913 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44082 finisher ceph cwd unknown /proc/42729/task/44082/cwd (readlink: Permission denied) 2024-04-22T13:26:48.913 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44082 finisher ceph rtd unknown /proc/42729/task/44082/root (readlink: Permission denied) 2024-04-22T13:26:48.913 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44082 finisher ceph txt unknown /proc/42729/task/44082/exe (readlink: Permission denied) 2024-04-22T13:26:48.913 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44082 finisher ceph NOFD /proc/42729/task/44082/fd (opendir: Permission denied) 2024-04-22T13:26:48.913 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44083 safe_time ceph cwd unknown /proc/42729/task/44083/cwd (readlink: Permission denied) 2024-04-22T13:26:48.913 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44083 safe_time ceph rtd unknown /proc/42729/task/44083/root (readlink: Permission denied) 2024-04-22T13:26:48.913 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44083 safe_time ceph txt unknown /proc/42729/task/44083/exe (readlink: Permission denied) 2024-04-22T13:26:48.913 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44083 safe_time ceph NOFD /proc/42729/task/44083/fd (opendir: Permission denied) 2024-04-22T13:26:48.913 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44084 safe_time ceph cwd unknown /proc/42729/task/44084/cwd (readlink: Permission denied) 2024-04-22T13:26:48.913 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44084 safe_time ceph rtd unknown /proc/42729/task/44084/root (readlink: Permission denied) 2024-04-22T13:26:48.914 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44084 safe_time ceph txt unknown /proc/42729/task/44084/exe (readlink: Permission denied) 2024-04-22T13:26:48.914 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44084 safe_time ceph NOFD /proc/42729/task/44084/fd (opendir: Permission denied) 2024-04-22T13:26:48.914 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44085 ceph-osd ceph cwd unknown /proc/42729/task/44085/cwd (readlink: Permission denied) 2024-04-22T13:26:48.914 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44085 ceph-osd ceph rtd unknown /proc/42729/task/44085/root (readlink: Permission denied) 2024-04-22T13:26:48.914 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44085 ceph-osd ceph txt unknown /proc/42729/task/44085/exe (readlink: Permission denied) 2024-04-22T13:26:48.914 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44085 ceph-osd ceph NOFD /proc/42729/task/44085/fd (opendir: Permission denied) 2024-04-22T13:26:48.914 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44086 osd_srv_a ceph cwd unknown /proc/42729/task/44086/cwd (readlink: Permission denied) 2024-04-22T13:26:48.914 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44086 osd_srv_a ceph rtd unknown /proc/42729/task/44086/root (readlink: Permission denied) 2024-04-22T13:26:48.914 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44086 osd_srv_a ceph txt unknown /proc/42729/task/44086/exe (readlink: Permission denied) 2024-04-22T13:26:48.914 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44086 osd_srv_a ceph NOFD /proc/42729/task/44086/fd (opendir: Permission denied) 2024-04-22T13:26:48.914 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44087 tp_osd_tp ceph cwd unknown /proc/42729/task/44087/cwd (readlink: Permission denied) 2024-04-22T13:26:48.914 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44087 tp_osd_tp ceph rtd unknown /proc/42729/task/44087/root (readlink: Permission denied) 2024-04-22T13:26:48.914 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44087 tp_osd_tp ceph txt unknown /proc/42729/task/44087/exe (readlink: Permission denied) 2024-04-22T13:26:48.914 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44087 tp_osd_tp ceph NOFD /proc/42729/task/44087/fd (opendir: Permission denied) 2024-04-22T13:26:48.914 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44088 tp_osd_tp ceph cwd unknown /proc/42729/task/44088/cwd (readlink: Permission denied) 2024-04-22T13:26:48.914 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44088 tp_osd_tp ceph rtd unknown /proc/42729/task/44088/root (readlink: Permission denied) 2024-04-22T13:26:48.914 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44088 tp_osd_tp ceph txt unknown /proc/42729/task/44088/exe (readlink: Permission denied) 2024-04-22T13:26:48.915 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44088 tp_osd_tp ceph NOFD /proc/42729/task/44088/fd (opendir: Permission denied) 2024-04-22T13:26:48.915 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44089 tp_osd_tp ceph cwd unknown /proc/42729/task/44089/cwd (readlink: Permission denied) 2024-04-22T13:26:48.915 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44089 tp_osd_tp ceph rtd unknown /proc/42729/task/44089/root (readlink: Permission denied) 2024-04-22T13:26:48.915 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44089 tp_osd_tp ceph txt unknown /proc/42729/task/44089/exe (readlink: Permission denied) 2024-04-22T13:26:48.915 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44089 tp_osd_tp ceph NOFD /proc/42729/task/44089/fd (opendir: Permission denied) 2024-04-22T13:26:48.915 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44090 tp_osd_tp ceph cwd unknown /proc/42729/task/44090/cwd (readlink: Permission denied) 2024-04-22T13:26:48.915 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44090 tp_osd_tp ceph rtd unknown /proc/42729/task/44090/root (readlink: Permission denied) 2024-04-22T13:26:48.915 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44090 tp_osd_tp ceph txt unknown /proc/42729/task/44090/exe (readlink: Permission denied) 2024-04-22T13:26:48.915 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44090 tp_osd_tp ceph NOFD /proc/42729/task/44090/fd (opendir: Permission denied) 2024-04-22T13:26:48.915 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44091 tp_osd_tp ceph cwd unknown /proc/42729/task/44091/cwd (readlink: Permission denied) 2024-04-22T13:26:48.915 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44091 tp_osd_tp ceph rtd unknown /proc/42729/task/44091/root (readlink: Permission denied) 2024-04-22T13:26:48.915 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44091 tp_osd_tp ceph txt unknown /proc/42729/task/44091/exe (readlink: Permission denied) 2024-04-22T13:26:48.915 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44091 tp_osd_tp ceph NOFD /proc/42729/task/44091/fd (opendir: Permission denied) 2024-04-22T13:26:48.915 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44092 tp_osd_tp ceph cwd unknown /proc/42729/task/44092/cwd (readlink: Permission denied) 2024-04-22T13:26:48.915 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44092 tp_osd_tp ceph rtd unknown /proc/42729/task/44092/root (readlink: Permission denied) 2024-04-22T13:26:48.916 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44092 tp_osd_tp ceph txt unknown /proc/42729/task/44092/exe (readlink: Permission denied) 2024-04-22T13:26:48.916 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44092 tp_osd_tp ceph NOFD /proc/42729/task/44092/fd (opendir: Permission denied) 2024-04-22T13:26:48.916 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44093 tp_osd_tp ceph cwd unknown /proc/42729/task/44093/cwd (readlink: Permission denied) 2024-04-22T13:26:48.916 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44093 tp_osd_tp ceph rtd unknown /proc/42729/task/44093/root (readlink: Permission denied) 2024-04-22T13:26:48.916 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44093 tp_osd_tp ceph txt unknown /proc/42729/task/44093/exe (readlink: Permission denied) 2024-04-22T13:26:48.916 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44093 tp_osd_tp ceph NOFD /proc/42729/task/44093/fd (opendir: Permission denied) 2024-04-22T13:26:48.916 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44094 tp_osd_tp ceph cwd unknown /proc/42729/task/44094/cwd (readlink: Permission denied) 2024-04-22T13:26:48.916 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44094 tp_osd_tp ceph rtd unknown /proc/42729/task/44094/root (readlink: Permission denied) 2024-04-22T13:26:48.916 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44094 tp_osd_tp ceph txt unknown /proc/42729/task/44094/exe (readlink: Permission denied) 2024-04-22T13:26:48.916 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44094 tp_osd_tp ceph NOFD /proc/42729/task/44094/fd (opendir: Permission denied) 2024-04-22T13:26:48.916 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44095 tp_osd_tp ceph cwd unknown /proc/42729/task/44095/cwd (readlink: Permission denied) 2024-04-22T13:26:48.916 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44095 tp_osd_tp ceph rtd unknown /proc/42729/task/44095/root (readlink: Permission denied) 2024-04-22T13:26:48.916 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44095 tp_osd_tp ceph txt unknown /proc/42729/task/44095/exe (readlink: Permission denied) 2024-04-22T13:26:48.916 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44095 tp_osd_tp ceph NOFD /proc/42729/task/44095/fd (opendir: Permission denied) 2024-04-22T13:26:48.916 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44096 tp_osd_tp ceph cwd unknown /proc/42729/task/44096/cwd (readlink: Permission denied) 2024-04-22T13:26:48.916 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44096 tp_osd_tp ceph rtd unknown /proc/42729/task/44096/root (readlink: Permission denied) 2024-04-22T13:26:48.916 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44096 tp_osd_tp ceph txt unknown /proc/42729/task/44096/exe (readlink: Permission denied) 2024-04-22T13:26:48.917 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44096 tp_osd_tp ceph NOFD /proc/42729/task/44096/fd (opendir: Permission denied) 2024-04-22T13:26:48.917 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44097 tp_osd_tp ceph cwd unknown /proc/42729/task/44097/cwd (readlink: Permission denied) 2024-04-22T13:26:48.917 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44097 tp_osd_tp ceph rtd unknown /proc/42729/task/44097/root (readlink: Permission denied) 2024-04-22T13:26:48.917 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44097 tp_osd_tp ceph txt unknown /proc/42729/task/44097/exe (readlink: Permission denied) 2024-04-22T13:26:48.917 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44097 tp_osd_tp ceph NOFD /proc/42729/task/44097/fd (opendir: Permission denied) 2024-04-22T13:26:48.917 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44098 tp_osd_tp ceph cwd unknown /proc/42729/task/44098/cwd (readlink: Permission denied) 2024-04-22T13:26:48.917 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44098 tp_osd_tp ceph rtd unknown /proc/42729/task/44098/root (readlink: Permission denied) 2024-04-22T13:26:48.917 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44098 tp_osd_tp ceph txt unknown /proc/42729/task/44098/exe (readlink: Permission denied) 2024-04-22T13:26:48.917 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44098 tp_osd_tp ceph NOFD /proc/42729/task/44098/fd (opendir: Permission denied) 2024-04-22T13:26:48.917 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44099 tp_osd_tp ceph cwd unknown /proc/42729/task/44099/cwd (readlink: Permission denied) 2024-04-22T13:26:48.917 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44099 tp_osd_tp ceph rtd unknown /proc/42729/task/44099/root (readlink: Permission denied) 2024-04-22T13:26:48.917 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44099 tp_osd_tp ceph txt unknown /proc/42729/task/44099/exe (readlink: Permission denied) 2024-04-22T13:26:48.917 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44099 tp_osd_tp ceph NOFD /proc/42729/task/44099/fd (opendir: Permission denied) 2024-04-22T13:26:48.917 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44100 tp_osd_tp ceph cwd unknown /proc/42729/task/44100/cwd (readlink: Permission denied) 2024-04-22T13:26:48.917 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44100 tp_osd_tp ceph rtd unknown /proc/42729/task/44100/root (readlink: Permission denied) 2024-04-22T13:26:48.917 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44100 tp_osd_tp ceph txt unknown /proc/42729/task/44100/exe (readlink: Permission denied) 2024-04-22T13:26:48.918 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44100 tp_osd_tp ceph NOFD /proc/42729/task/44100/fd (opendir: Permission denied) 2024-04-22T13:26:48.918 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44101 tp_osd_tp ceph cwd unknown /proc/42729/task/44101/cwd (readlink: Permission denied) 2024-04-22T13:26:48.918 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44101 tp_osd_tp ceph rtd unknown /proc/42729/task/44101/root (readlink: Permission denied) 2024-04-22T13:26:48.918 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44101 tp_osd_tp ceph txt unknown /proc/42729/task/44101/exe (readlink: Permission denied) 2024-04-22T13:26:48.918 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44101 tp_osd_tp ceph NOFD /proc/42729/task/44101/fd (opendir: Permission denied) 2024-04-22T13:26:48.918 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44102 tp_osd_tp ceph cwd unknown /proc/42729/task/44102/cwd (readlink: Permission denied) 2024-04-22T13:26:48.918 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44102 tp_osd_tp ceph rtd unknown /proc/42729/task/44102/root (readlink: Permission denied) 2024-04-22T13:26:48.918 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44102 tp_osd_tp ceph txt unknown /proc/42729/task/44102/exe (readlink: Permission denied) 2024-04-22T13:26:48.918 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44102 tp_osd_tp ceph NOFD /proc/42729/task/44102/fd (opendir: Permission denied) 2024-04-22T13:26:48.918 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44103 osd_srv_h ceph cwd unknown /proc/42729/task/44103/cwd (readlink: Permission denied) 2024-04-22T13:26:48.918 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44103 osd_srv_h ceph rtd unknown /proc/42729/task/44103/root (readlink: Permission denied) 2024-04-22T13:26:48.918 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44103 osd_srv_h ceph txt unknown /proc/42729/task/44103/exe (readlink: Permission denied) 2024-04-22T13:26:48.918 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 42729 44103 osd_srv_h ceph NOFD /proc/42729/task/44103/fd (opendir: Permission denied) 2024-04-22T13:26:48.918 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 44132 root cwd unknown /proc/44132/cwd (readlink: Permission denied) 2024-04-22T13:26:48.918 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 44132 root rtd unknown /proc/44132/root (readlink: Permission denied) 2024-04-22T13:26:48.919 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 44132 root txt unknown /proc/44132/exe (readlink: Permission denied) 2024-04-22T13:26:48.919 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 44132 root NOFD /proc/44132/fd (opendir: Permission denied) 2024-04-22T13:26:48.919 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 44161 root cwd unknown /proc/44161/cwd (readlink: Permission denied) 2024-04-22T13:26:48.919 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 44161 root rtd unknown /proc/44161/root (readlink: Permission denied) 2024-04-22T13:26:48.919 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 44161 root txt unknown /proc/44161/exe (readlink: Permission denied) 2024-04-22T13:26:48.919 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 44161 root NOFD /proc/44161/fd (opendir: Permission denied) 2024-04-22T13:26:48.919 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/0 48924 root cwd unknown /proc/48924/cwd (readlink: Permission denied) 2024-04-22T13:26:48.919 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/0 48924 root rtd unknown /proc/48924/root (readlink: Permission denied) 2024-04-22T13:26:48.919 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/0 48924 root txt unknown /proc/48924/exe (readlink: Permission denied) 2024-04-22T13:26:48.919 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/0 48924 root NOFD /proc/48924/fd (opendir: Permission denied) 2024-04-22T13:26:48.919 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 49663 root cwd unknown /proc/49663/cwd (readlink: Permission denied) 2024-04-22T13:26:48.919 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 49663 root rtd unknown /proc/49663/root (readlink: Permission denied) 2024-04-22T13:26:48.919 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 49663 root txt unknown /proc/49663/exe (readlink: Permission denied) 2024-04-22T13:26:48.919 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 49663 root NOFD /proc/49663/fd (opendir: Permission denied) 2024-04-22T13:26:48.919 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 49665 root cwd unknown /proc/49665/cwd (readlink: Permission denied) 2024-04-22T13:26:48.919 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 49665 root rtd unknown /proc/49665/root (readlink: Permission denied) 2024-04-22T13:26:48.920 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 49665 root txt unknown /proc/49665/exe (readlink: Permission denied) 2024-04-22T13:26:48.920 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 49665 root NOFD /proc/49665/fd (opendir: Permission denied) 2024-04-22T13:26:48.920 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 ceph cwd unknown /proc/49667/cwd (readlink: Permission denied) 2024-04-22T13:26:48.920 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 ceph rtd unknown /proc/49667/root (readlink: Permission denied) 2024-04-22T13:26:48.920 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 ceph txt unknown /proc/49667/exe (readlink: Permission denied) 2024-04-22T13:26:48.920 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 ceph NOFD /proc/49667/fd (opendir: Permission denied) 2024-04-22T13:26:48.920 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49668 log ceph cwd unknown /proc/49667/task/49668/cwd (readlink: Permission denied) 2024-04-22T13:26:48.920 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49668 log ceph rtd unknown /proc/49667/task/49668/root (readlink: Permission denied) 2024-04-22T13:26:48.920 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49668 log ceph txt unknown /proc/49667/task/49668/exe (readlink: Permission denied) 2024-04-22T13:26:48.920 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49668 log ceph NOFD /proc/49667/task/49668/fd (opendir: Permission denied) 2024-04-22T13:26:48.920 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49670 msgr-work ceph cwd unknown /proc/49667/task/49670/cwd (readlink: Permission denied) 2024-04-22T13:26:48.920 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49670 msgr-work ceph rtd unknown /proc/49667/task/49670/root (readlink: Permission denied) 2024-04-22T13:26:48.920 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49670 msgr-work ceph txt unknown /proc/49667/task/49670/exe (readlink: Permission denied) 2024-04-22T13:26:48.920 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49670 msgr-work ceph NOFD /proc/49667/task/49670/fd (opendir: Permission denied) 2024-04-22T13:26:48.921 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49671 msgr-work ceph cwd unknown /proc/49667/task/49671/cwd (readlink: Permission denied) 2024-04-22T13:26:48.921 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49671 msgr-work ceph rtd unknown /proc/49667/task/49671/root (readlink: Permission denied) 2024-04-22T13:26:48.921 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49671 msgr-work ceph txt unknown /proc/49667/task/49671/exe (readlink: Permission denied) 2024-04-22T13:26:48.921 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49671 msgr-work ceph NOFD /proc/49667/task/49671/fd (opendir: Permission denied) 2024-04-22T13:26:48.921 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49672 msgr-work ceph cwd unknown /proc/49667/task/49672/cwd (readlink: Permission denied) 2024-04-22T13:26:48.921 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49672 msgr-work ceph rtd unknown /proc/49667/task/49672/root (readlink: Permission denied) 2024-04-22T13:26:48.921 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49672 msgr-work ceph txt unknown /proc/49667/task/49672/exe (readlink: Permission denied) 2024-04-22T13:26:48.921 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49672 msgr-work ceph NOFD /proc/49667/task/49672/fd (opendir: Permission denied) 2024-04-22T13:26:48.921 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49676 service ceph cwd unknown /proc/49667/task/49676/cwd (readlink: Permission denied) 2024-04-22T13:26:48.921 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49676 service ceph rtd unknown /proc/49667/task/49676/root (readlink: Permission denied) 2024-04-22T13:26:48.921 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49676 service ceph txt unknown /proc/49667/task/49676/exe (readlink: Permission denied) 2024-04-22T13:26:48.921 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49676 service ceph NOFD /proc/49667/task/49676/fd (opendir: Permission denied) 2024-04-22T13:26:48.921 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49677 admin_soc ceph cwd unknown /proc/49667/task/49677/cwd (readlink: Permission denied) 2024-04-22T13:26:48.921 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49677 admin_soc ceph rtd unknown /proc/49667/task/49677/root (readlink: Permission denied) 2024-04-22T13:26:48.921 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49677 admin_soc ceph txt unknown /proc/49667/task/49677/exe (readlink: Permission denied) 2024-04-22T13:26:48.922 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49677 admin_soc ceph NOFD /proc/49667/task/49677/fd (opendir: Permission denied) 2024-04-22T13:26:48.922 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49819 signal_ha ceph cwd unknown /proc/49667/task/49819/cwd (readlink: Permission denied) 2024-04-22T13:26:48.922 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49819 signal_ha ceph rtd unknown /proc/49667/task/49819/root (readlink: Permission denied) 2024-04-22T13:26:48.922 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49819 signal_ha ceph txt unknown /proc/49667/task/49819/exe (readlink: Permission denied) 2024-04-22T13:26:48.922 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49819 signal_ha ceph NOFD /proc/49667/task/49819/fd (opendir: Permission denied) 2024-04-22T13:26:48.922 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49820 io_contex ceph cwd unknown /proc/49667/task/49820/cwd (readlink: Permission denied) 2024-04-22T13:26:48.922 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49820 io_contex ceph rtd unknown /proc/49667/task/49820/root (readlink: Permission denied) 2024-04-22T13:26:48.922 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49820 io_contex ceph txt unknown /proc/49667/task/49820/exe (readlink: Permission denied) 2024-04-22T13:26:48.922 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49820 io_contex ceph NOFD /proc/49667/task/49820/fd (opendir: Permission denied) 2024-04-22T13:26:48.922 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49821 io_contex ceph cwd unknown /proc/49667/task/49821/cwd (readlink: Permission denied) 2024-04-22T13:26:48.922 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49821 io_contex ceph rtd unknown /proc/49667/task/49821/root (readlink: Permission denied) 2024-04-22T13:26:48.922 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49821 io_contex ceph txt unknown /proc/49667/task/49821/exe (readlink: Permission denied) 2024-04-22T13:26:48.922 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49821 io_contex ceph NOFD /proc/49667/task/49821/fd (opendir: Permission denied) 2024-04-22T13:26:48.922 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49931 OpHistory ceph cwd unknown /proc/49667/task/49931/cwd (readlink: Permission denied) 2024-04-22T13:26:48.922 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49931 OpHistory ceph rtd unknown /proc/49667/task/49931/root (readlink: Permission denied) 2024-04-22T13:26:48.922 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49931 OpHistory ceph txt unknown /proc/49667/task/49931/exe (readlink: Permission denied) 2024-04-22T13:26:48.923 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49931 OpHistory ceph NOFD /proc/49667/task/49931/fd (opendir: Permission denied) 2024-04-22T13:26:48.923 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49932 ceph_time ceph cwd unknown /proc/49667/task/49932/cwd (readlink: Permission denied) 2024-04-22T13:26:48.923 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49932 ceph_time ceph rtd unknown /proc/49667/task/49932/root (readlink: Permission denied) 2024-04-22T13:26:48.923 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49932 ceph_time ceph txt unknown /proc/49667/task/49932/exe (readlink: Permission denied) 2024-04-22T13:26:48.923 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49932 ceph_time ceph NOFD /proc/49667/task/49932/fd (opendir: Permission denied) 2024-04-22T13:26:48.923 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49971 ceph-osd ceph cwd unknown /proc/49667/task/49971/cwd (readlink: Permission denied) 2024-04-22T13:26:48.923 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49971 ceph-osd ceph rtd unknown /proc/49667/task/49971/root (readlink: Permission denied) 2024-04-22T13:26:48.923 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49971 ceph-osd ceph txt unknown /proc/49667/task/49971/exe (readlink: Permission denied) 2024-04-22T13:26:48.923 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49971 ceph-osd ceph NOFD /proc/49667/task/49971/fd (opendir: Permission denied) 2024-04-22T13:26:48.923 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49979 ceph-osd ceph cwd unknown /proc/49667/task/49979/cwd (readlink: Permission denied) 2024-04-22T13:26:48.923 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49979 ceph-osd ceph rtd unknown /proc/49667/task/49979/root (readlink: Permission denied) 2024-04-22T13:26:48.923 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49979 ceph-osd ceph txt unknown /proc/49667/task/49979/exe (readlink: Permission denied) 2024-04-22T13:26:48.923 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 49979 ceph-osd ceph NOFD /proc/49667/task/49979/fd (opendir: Permission denied) 2024-04-22T13:26:48.923 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50010 ceph-osd ceph cwd unknown /proc/49667/task/50010/cwd (readlink: Permission denied) 2024-04-22T13:26:48.923 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50010 ceph-osd ceph rtd unknown /proc/49667/task/50010/root (readlink: Permission denied) 2024-04-22T13:26:48.924 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50010 ceph-osd ceph txt unknown /proc/49667/task/50010/exe (readlink: Permission denied) 2024-04-22T13:26:48.924 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50010 ceph-osd ceph NOFD /proc/49667/task/50010/fd (opendir: Permission denied) 2024-04-22T13:26:48.924 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50029 ceph-osd ceph cwd unknown /proc/49667/task/50029/cwd (readlink: Permission denied) 2024-04-22T13:26:48.924 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50029 ceph-osd ceph rtd unknown /proc/49667/task/50029/root (readlink: Permission denied) 2024-04-22T13:26:48.924 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50029 ceph-osd ceph txt unknown /proc/49667/task/50029/exe (readlink: Permission denied) 2024-04-22T13:26:48.924 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50029 ceph-osd ceph NOFD /proc/49667/task/50029/fd (opendir: Permission denied) 2024-04-22T13:26:48.924 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50051 ceph-osd ceph cwd unknown /proc/49667/task/50051/cwd (readlink: Permission denied) 2024-04-22T13:26:48.924 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50051 ceph-osd ceph rtd unknown /proc/49667/task/50051/root (readlink: Permission denied) 2024-04-22T13:26:48.924 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50051 ceph-osd ceph txt unknown /proc/49667/task/50051/exe (readlink: Permission denied) 2024-04-22T13:26:48.924 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50051 ceph-osd ceph NOFD /proc/49667/task/50051/fd (opendir: Permission denied) 2024-04-22T13:26:48.924 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50074 ceph-osd ceph cwd unknown /proc/49667/task/50074/cwd (readlink: Permission denied) 2024-04-22T13:26:48.924 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50074 ceph-osd ceph rtd unknown /proc/49667/task/50074/root (readlink: Permission denied) 2024-04-22T13:26:48.924 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50074 ceph-osd ceph txt unknown /proc/49667/task/50074/exe (readlink: Permission denied) 2024-04-22T13:26:48.924 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50074 ceph-osd ceph NOFD /proc/49667/task/50074/fd (opendir: Permission denied) 2024-04-22T13:26:48.924 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50105 ceph-osd ceph cwd unknown /proc/49667/task/50105/cwd (readlink: Permission denied) 2024-04-22T13:26:48.925 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50105 ceph-osd ceph rtd unknown /proc/49667/task/50105/root (readlink: Permission denied) 2024-04-22T13:26:48.925 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50105 ceph-osd ceph txt unknown /proc/49667/task/50105/exe (readlink: Permission denied) 2024-04-22T13:26:48.925 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50105 ceph-osd ceph NOFD /proc/49667/task/50105/fd (opendir: Permission denied) 2024-04-22T13:26:48.925 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50165 ceph-osd ceph cwd unknown /proc/49667/task/50165/cwd (readlink: Permission denied) 2024-04-22T13:26:48.925 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50165 ceph-osd ceph rtd unknown /proc/49667/task/50165/root (readlink: Permission denied) 2024-04-22T13:26:48.925 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50165 ceph-osd ceph txt unknown /proc/49667/task/50165/exe (readlink: Permission denied) 2024-04-22T13:26:48.925 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50165 ceph-osd ceph NOFD /proc/49667/task/50165/fd (opendir: Permission denied) 2024-04-22T13:26:48.925 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50167 ceph-osd ceph cwd unknown /proc/49667/task/50167/cwd (readlink: Permission denied) 2024-04-22T13:26:48.925 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50167 ceph-osd ceph rtd unknown /proc/49667/task/50167/root (readlink: Permission denied) 2024-04-22T13:26:48.925 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50167 ceph-osd ceph txt unknown /proc/49667/task/50167/exe (readlink: Permission denied) 2024-04-22T13:26:48.925 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50167 ceph-osd ceph NOFD /proc/49667/task/50167/fd (opendir: Permission denied) 2024-04-22T13:26:48.925 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50168 safe_time ceph cwd unknown /proc/49667/task/50168/cwd (readlink: Permission denied) 2024-04-22T13:26:48.925 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50168 safe_time ceph rtd unknown /proc/49667/task/50168/root (readlink: Permission denied) 2024-04-22T13:26:48.925 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50168 safe_time ceph txt unknown /proc/49667/task/50168/exe (readlink: Permission denied) 2024-04-22T13:26:48.925 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50168 safe_time ceph NOFD /proc/49667/task/50168/fd (opendir: Permission denied) 2024-04-22T13:26:48.926 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50169 safe_time ceph cwd unknown /proc/49667/task/50169/cwd (readlink: Permission denied) 2024-04-22T13:26:48.926 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50169 safe_time ceph rtd unknown /proc/49667/task/50169/root (readlink: Permission denied) 2024-04-22T13:26:48.926 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50169 safe_time ceph txt unknown /proc/49667/task/50169/exe (readlink: Permission denied) 2024-04-22T13:26:48.926 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50169 safe_time ceph NOFD /proc/49667/task/50169/fd (opendir: Permission denied) 2024-04-22T13:26:48.926 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50170 safe_time ceph cwd unknown /proc/49667/task/50170/cwd (readlink: Permission denied) 2024-04-22T13:26:48.926 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50170 safe_time ceph rtd unknown /proc/49667/task/50170/root (readlink: Permission denied) 2024-04-22T13:26:48.926 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50170 safe_time ceph txt unknown /proc/49667/task/50170/exe (readlink: Permission denied) 2024-04-22T13:26:48.926 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50170 safe_time ceph NOFD /proc/49667/task/50170/fd (opendir: Permission denied) 2024-04-22T13:26:48.926 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50171 safe_time ceph cwd unknown /proc/49667/task/50171/cwd (readlink: Permission denied) 2024-04-22T13:26:48.926 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50171 safe_time ceph rtd unknown /proc/49667/task/50171/root (readlink: Permission denied) 2024-04-22T13:26:48.926 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50171 safe_time ceph txt unknown /proc/49667/task/50171/exe (readlink: Permission denied) 2024-04-22T13:26:48.926 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50171 safe_time ceph NOFD /proc/49667/task/50171/fd (opendir: Permission denied) 2024-04-22T13:26:48.926 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50172 fn_anonym ceph cwd unknown /proc/49667/task/50172/cwd (readlink: Permission denied) 2024-04-22T13:26:48.926 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50172 fn_anonym ceph rtd unknown /proc/49667/task/50172/root (readlink: Permission denied) 2024-04-22T13:26:48.927 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50172 fn_anonym ceph txt unknown /proc/49667/task/50172/exe (readlink: Permission denied) 2024-04-22T13:26:48.927 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50172 fn_anonym ceph NOFD /proc/49667/task/50172/fd (opendir: Permission denied) 2024-04-22T13:26:48.927 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50175 rocksdb:l ceph cwd unknown /proc/49667/task/50175/cwd (readlink: Permission denied) 2024-04-22T13:26:48.927 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50175 rocksdb:l ceph rtd unknown /proc/49667/task/50175/root (readlink: Permission denied) 2024-04-22T13:26:48.927 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50175 rocksdb:l ceph txt unknown /proc/49667/task/50175/exe (readlink: Permission denied) 2024-04-22T13:26:48.927 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50175 rocksdb:l ceph NOFD /proc/49667/task/50175/fd (opendir: Permission denied) 2024-04-22T13:26:48.927 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50176 rocksdb:l ceph cwd unknown /proc/49667/task/50176/cwd (readlink: Permission denied) 2024-04-22T13:26:48.927 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50176 rocksdb:l ceph rtd unknown /proc/49667/task/50176/root (readlink: Permission denied) 2024-04-22T13:26:48.927 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50176 rocksdb:l ceph txt unknown /proc/49667/task/50176/exe (readlink: Permission denied) 2024-04-22T13:26:48.927 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50176 rocksdb:l ceph NOFD /proc/49667/task/50176/fd (opendir: Permission denied) 2024-04-22T13:26:48.927 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50177 rocksdb:l ceph cwd unknown /proc/49667/task/50177/cwd (readlink: Permission denied) 2024-04-22T13:26:48.927 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50177 rocksdb:l ceph rtd unknown /proc/49667/task/50177/root (readlink: Permission denied) 2024-04-22T13:26:48.927 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50177 rocksdb:l ceph txt unknown /proc/49667/task/50177/exe (readlink: Permission denied) 2024-04-22T13:26:48.927 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50177 rocksdb:l ceph NOFD /proc/49667/task/50177/fd (opendir: Permission denied) 2024-04-22T13:26:48.927 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50178 rocksdb:h ceph cwd unknown /proc/49667/task/50178/cwd (readlink: Permission denied) 2024-04-22T13:26:48.927 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50178 rocksdb:h ceph rtd unknown /proc/49667/task/50178/root (readlink: Permission denied) 2024-04-22T13:26:48.928 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50178 rocksdb:h ceph txt unknown /proc/49667/task/50178/exe (readlink: Permission denied) 2024-04-22T13:26:48.928 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50178 rocksdb:h ceph NOFD /proc/49667/task/50178/fd (opendir: Permission denied) 2024-04-22T13:26:48.928 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50579 bstore_ai ceph cwd unknown /proc/49667/task/50579/cwd (readlink: Permission denied) 2024-04-22T13:26:48.928 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50579 bstore_ai ceph rtd unknown /proc/49667/task/50579/root (readlink: Permission denied) 2024-04-22T13:26:48.928 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50579 bstore_ai ceph txt unknown /proc/49667/task/50579/exe (readlink: Permission denied) 2024-04-22T13:26:48.928 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50579 bstore_ai ceph NOFD /proc/49667/task/50579/fd (opendir: Permission denied) 2024-04-22T13:26:48.928 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50843 bstore_ai ceph cwd unknown /proc/49667/task/50843/cwd (readlink: Permission denied) 2024-04-22T13:26:48.928 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50843 bstore_ai ceph rtd unknown /proc/49667/task/50843/root (readlink: Permission denied) 2024-04-22T13:26:48.928 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50843 bstore_ai ceph txt unknown /proc/49667/task/50843/exe (readlink: Permission denied) 2024-04-22T13:26:48.928 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 50843 bstore_ai ceph NOFD /proc/49667/task/50843/fd (opendir: Permission denied) 2024-04-22T13:26:48.928 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51025 ceph-osd ceph cwd unknown /proc/49667/task/51025/cwd (readlink: Permission denied) 2024-04-22T13:26:48.928 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51025 ceph-osd ceph rtd unknown /proc/49667/task/51025/root (readlink: Permission denied) 2024-04-22T13:26:48.928 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51025 ceph-osd ceph txt unknown /proc/49667/task/51025/exe (readlink: Permission denied) 2024-04-22T13:26:48.928 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51025 ceph-osd ceph NOFD /proc/49667/task/51025/fd (opendir: Permission denied) 2024-04-22T13:26:48.929 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51026 cfin ceph cwd unknown /proc/49667/task/51026/cwd (readlink: Permission denied) 2024-04-22T13:26:48.929 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51026 cfin ceph rtd unknown /proc/49667/task/51026/root (readlink: Permission denied) 2024-04-22T13:26:48.929 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51026 cfin ceph txt unknown /proc/49667/task/51026/exe (readlink: Permission denied) 2024-04-22T13:26:48.929 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51026 cfin ceph NOFD /proc/49667/task/51026/fd (opendir: Permission denied) 2024-04-22T13:26:48.929 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51027 bstore_kv ceph cwd unknown /proc/49667/task/51027/cwd (readlink: Permission denied) 2024-04-22T13:26:48.929 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51027 bstore_kv ceph rtd unknown /proc/49667/task/51027/root (readlink: Permission denied) 2024-04-22T13:26:48.929 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51027 bstore_kv ceph txt unknown /proc/49667/task/51027/exe (readlink: Permission denied) 2024-04-22T13:26:48.929 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51027 bstore_kv ceph NOFD /proc/49667/task/51027/fd (opendir: Permission denied) 2024-04-22T13:26:48.929 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51028 bstore_kv ceph cwd unknown /proc/49667/task/51028/cwd (readlink: Permission denied) 2024-04-22T13:26:48.929 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51028 bstore_kv ceph rtd unknown /proc/49667/task/51028/root (readlink: Permission denied) 2024-04-22T13:26:48.929 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51028 bstore_kv ceph txt unknown /proc/49667/task/51028/exe (readlink: Permission denied) 2024-04-22T13:26:48.929 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51028 bstore_kv ceph NOFD /proc/49667/task/51028/fd (opendir: Permission denied) 2024-04-22T13:26:48.929 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51029 bstore_me ceph cwd unknown /proc/49667/task/51029/cwd (readlink: Permission denied) 2024-04-22T13:26:48.930 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51029 bstore_me ceph rtd unknown /proc/49667/task/51029/root (readlink: Permission denied) 2024-04-22T13:26:48.930 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51029 bstore_me ceph txt unknown /proc/49667/task/51029/exe (readlink: Permission denied) 2024-04-22T13:26:48.930 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51029 bstore_me ceph NOFD /proc/49667/task/51029/fd (opendir: Permission denied) 2024-04-22T13:26:48.930 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51030 ms_dispat ceph cwd unknown /proc/49667/task/51030/cwd (readlink: Permission denied) 2024-04-22T13:26:48.930 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51030 ms_dispat ceph rtd unknown /proc/49667/task/51030/root (readlink: Permission denied) 2024-04-22T13:26:48.930 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51030 ms_dispat ceph txt unknown /proc/49667/task/51030/exe (readlink: Permission denied) 2024-04-22T13:26:48.930 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51030 ms_dispat ceph NOFD /proc/49667/task/51030/fd (opendir: Permission denied) 2024-04-22T13:26:48.930 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51031 ms_local ceph cwd unknown /proc/49667/task/51031/cwd (readlink: Permission denied) 2024-04-22T13:26:48.930 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51031 ms_local ceph rtd unknown /proc/49667/task/51031/root (readlink: Permission denied) 2024-04-22T13:26:48.930 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51031 ms_local ceph txt unknown /proc/49667/task/51031/exe (readlink: Permission denied) 2024-04-22T13:26:48.930 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51031 ms_local ceph NOFD /proc/49667/task/51031/fd (opendir: Permission denied) 2024-04-22T13:26:48.930 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51032 safe_time ceph cwd unknown /proc/49667/task/51032/cwd (readlink: Permission denied) 2024-04-22T13:26:48.930 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51032 safe_time ceph rtd unknown /proc/49667/task/51032/root (readlink: Permission denied) 2024-04-22T13:26:48.930 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51032 safe_time ceph txt unknown /proc/49667/task/51032/exe (readlink: Permission denied) 2024-04-22T13:26:48.930 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51032 safe_time ceph NOFD /proc/49667/task/51032/fd (opendir: Permission denied) 2024-04-22T13:26:48.931 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51033 safe_time ceph cwd unknown /proc/49667/task/51033/cwd (readlink: Permission denied) 2024-04-22T13:26:48.931 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51033 safe_time ceph rtd unknown /proc/49667/task/51033/root (readlink: Permission denied) 2024-04-22T13:26:48.931 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51033 safe_time ceph txt unknown /proc/49667/task/51033/exe (readlink: Permission denied) 2024-04-22T13:26:48.931 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51033 safe_time ceph NOFD /proc/49667/task/51033/fd (opendir: Permission denied) 2024-04-22T13:26:48.931 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51034 ms_dispat ceph cwd unknown /proc/49667/task/51034/cwd (readlink: Permission denied) 2024-04-22T13:26:48.931 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51034 ms_dispat ceph rtd unknown /proc/49667/task/51034/root (readlink: Permission denied) 2024-04-22T13:26:48.931 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51034 ms_dispat ceph txt unknown /proc/49667/task/51034/exe (readlink: Permission denied) 2024-04-22T13:26:48.931 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51034 ms_dispat ceph NOFD /proc/49667/task/51034/fd (opendir: Permission denied) 2024-04-22T13:26:48.931 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51035 ms_local ceph cwd unknown /proc/49667/task/51035/cwd (readlink: Permission denied) 2024-04-22T13:26:48.931 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51035 ms_local ceph rtd unknown /proc/49667/task/51035/root (readlink: Permission denied) 2024-04-22T13:26:48.931 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51035 ms_local ceph txt unknown /proc/49667/task/51035/exe (readlink: Permission denied) 2024-04-22T13:26:48.931 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51035 ms_local ceph NOFD /proc/49667/task/51035/fd (opendir: Permission denied) 2024-04-22T13:26:48.931 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51036 ms_dispat ceph cwd unknown /proc/49667/task/51036/cwd (readlink: Permission denied) 2024-04-22T13:26:48.931 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51036 ms_dispat ceph rtd unknown /proc/49667/task/51036/root (readlink: Permission denied) 2024-04-22T13:26:48.931 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51036 ms_dispat ceph txt unknown /proc/49667/task/51036/exe (readlink: Permission denied) 2024-04-22T13:26:48.932 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51036 ms_dispat ceph NOFD /proc/49667/task/51036/fd (opendir: Permission denied) 2024-04-22T13:26:48.932 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51037 ms_local ceph cwd unknown /proc/49667/task/51037/cwd (readlink: Permission denied) 2024-04-22T13:26:48.932 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51037 ms_local ceph rtd unknown /proc/49667/task/51037/root (readlink: Permission denied) 2024-04-22T13:26:48.932 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51037 ms_local ceph txt unknown /proc/49667/task/51037/exe (readlink: Permission denied) 2024-04-22T13:26:48.932 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51037 ms_local ceph NOFD /proc/49667/task/51037/fd (opendir: Permission denied) 2024-04-22T13:26:48.932 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51038 ms_dispat ceph cwd unknown /proc/49667/task/51038/cwd (readlink: Permission denied) 2024-04-22T13:26:48.932 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51038 ms_dispat ceph rtd unknown /proc/49667/task/51038/root (readlink: Permission denied) 2024-04-22T13:26:48.932 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51038 ms_dispat ceph txt unknown /proc/49667/task/51038/exe (readlink: Permission denied) 2024-04-22T13:26:48.932 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51038 ms_dispat ceph NOFD /proc/49667/task/51038/fd (opendir: Permission denied) 2024-04-22T13:26:48.932 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51039 ms_local ceph cwd unknown /proc/49667/task/51039/cwd (readlink: Permission denied) 2024-04-22T13:26:48.932 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51039 ms_local ceph rtd unknown /proc/49667/task/51039/root (readlink: Permission denied) 2024-04-22T13:26:48.932 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51039 ms_local ceph txt unknown /proc/49667/task/51039/exe (readlink: Permission denied) 2024-04-22T13:26:48.932 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51039 ms_local ceph NOFD /proc/49667/task/51039/fd (opendir: Permission denied) 2024-04-22T13:26:48.932 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51040 ms_dispat ceph cwd unknown /proc/49667/task/51040/cwd (readlink: Permission denied) 2024-04-22T13:26:48.932 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51040 ms_dispat ceph rtd unknown /proc/49667/task/51040/root (readlink: Permission denied) 2024-04-22T13:26:48.933 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51040 ms_dispat ceph txt unknown /proc/49667/task/51040/exe (readlink: Permission denied) 2024-04-22T13:26:48.933 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51040 ms_dispat ceph NOFD /proc/49667/task/51040/fd (opendir: Permission denied) 2024-04-22T13:26:48.933 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51041 ms_local ceph cwd unknown /proc/49667/task/51041/cwd (readlink: Permission denied) 2024-04-22T13:26:48.933 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51041 ms_local ceph rtd unknown /proc/49667/task/51041/root (readlink: Permission denied) 2024-04-22T13:26:48.933 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51041 ms_local ceph txt unknown /proc/49667/task/51041/exe (readlink: Permission denied) 2024-04-22T13:26:48.933 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51041 ms_local ceph NOFD /proc/49667/task/51041/fd (opendir: Permission denied) 2024-04-22T13:26:48.933 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51042 ms_dispat ceph cwd unknown /proc/49667/task/51042/cwd (readlink: Permission denied) 2024-04-22T13:26:48.933 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51042 ms_dispat ceph rtd unknown /proc/49667/task/51042/root (readlink: Permission denied) 2024-04-22T13:26:48.933 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51042 ms_dispat ceph txt unknown /proc/49667/task/51042/exe (readlink: Permission denied) 2024-04-22T13:26:48.933 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51042 ms_dispat ceph NOFD /proc/49667/task/51042/fd (opendir: Permission denied) 2024-04-22T13:26:48.933 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51043 ms_local ceph cwd unknown /proc/49667/task/51043/cwd (readlink: Permission denied) 2024-04-22T13:26:48.933 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51043 ms_local ceph rtd unknown /proc/49667/task/51043/root (readlink: Permission denied) 2024-04-22T13:26:48.933 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51043 ms_local ceph txt unknown /proc/49667/task/51043/exe (readlink: Permission denied) 2024-04-22T13:26:48.933 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51043 ms_local ceph NOFD /proc/49667/task/51043/fd (opendir: Permission denied) 2024-04-22T13:26:48.933 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51044 ms_dispat ceph cwd unknown /proc/49667/task/51044/cwd (readlink: Permission denied) 2024-04-22T13:26:48.934 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51044 ms_dispat ceph rtd unknown /proc/49667/task/51044/root (readlink: Permission denied) 2024-04-22T13:26:48.934 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51044 ms_dispat ceph txt unknown /proc/49667/task/51044/exe (readlink: Permission denied) 2024-04-22T13:26:48.934 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51044 ms_dispat ceph NOFD /proc/49667/task/51044/fd (opendir: Permission denied) 2024-04-22T13:26:48.934 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51045 ms_local ceph cwd unknown /proc/49667/task/51045/cwd (readlink: Permission denied) 2024-04-22T13:26:48.934 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51045 ms_local ceph rtd unknown /proc/49667/task/51045/root (readlink: Permission denied) 2024-04-22T13:26:48.934 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51045 ms_local ceph txt unknown /proc/49667/task/51045/exe (readlink: Permission denied) 2024-04-22T13:26:48.934 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51045 ms_local ceph NOFD /proc/49667/task/51045/fd (opendir: Permission denied) 2024-04-22T13:26:48.934 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51046 fn_anonym ceph cwd unknown /proc/49667/task/51046/cwd (readlink: Permission denied) 2024-04-22T13:26:48.934 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51046 fn_anonym ceph rtd unknown /proc/49667/task/51046/root (readlink: Permission denied) 2024-04-22T13:26:48.934 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51046 fn_anonym ceph txt unknown /proc/49667/task/51046/exe (readlink: Permission denied) 2024-04-22T13:26:48.934 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51046 fn_anonym ceph NOFD /proc/49667/task/51046/fd (opendir: Permission denied) 2024-04-22T13:26:48.934 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51047 finisher ceph cwd unknown /proc/49667/task/51047/cwd (readlink: Permission denied) 2024-04-22T13:26:48.934 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51047 finisher ceph rtd unknown /proc/49667/task/51047/root (readlink: Permission denied) 2024-04-22T13:26:48.934 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51047 finisher ceph txt unknown /proc/49667/task/51047/exe (readlink: Permission denied) 2024-04-22T13:26:48.935 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51047 finisher ceph NOFD /proc/49667/task/51047/fd (opendir: Permission denied) 2024-04-22T13:26:48.935 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51048 safe_time ceph cwd unknown /proc/49667/task/51048/cwd (readlink: Permission denied) 2024-04-22T13:26:48.935 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51048 safe_time ceph rtd unknown /proc/49667/task/51048/root (readlink: Permission denied) 2024-04-22T13:26:48.935 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51048 safe_time ceph txt unknown /proc/49667/task/51048/exe (readlink: Permission denied) 2024-04-22T13:26:48.935 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51048 safe_time ceph NOFD /proc/49667/task/51048/fd (opendir: Permission denied) 2024-04-22T13:26:48.935 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51049 safe_time ceph cwd unknown /proc/49667/task/51049/cwd (readlink: Permission denied) 2024-04-22T13:26:48.935 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51049 safe_time ceph rtd unknown /proc/49667/task/51049/root (readlink: Permission denied) 2024-04-22T13:26:48.935 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51049 safe_time ceph txt unknown /proc/49667/task/51049/exe (readlink: Permission denied) 2024-04-22T13:26:48.935 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51049 safe_time ceph NOFD /proc/49667/task/51049/fd (opendir: Permission denied) 2024-04-22T13:26:48.935 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51050 ceph-osd ceph cwd unknown /proc/49667/task/51050/cwd (readlink: Permission denied) 2024-04-22T13:26:48.935 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51050 ceph-osd ceph rtd unknown /proc/49667/task/51050/root (readlink: Permission denied) 2024-04-22T13:26:48.935 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51050 ceph-osd ceph txt unknown /proc/49667/task/51050/exe (readlink: Permission denied) 2024-04-22T13:26:48.935 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51050 ceph-osd ceph NOFD /proc/49667/task/51050/fd (opendir: Permission denied) 2024-04-22T13:26:48.936 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51051 osd_srv_a ceph cwd unknown /proc/49667/task/51051/cwd (readlink: Permission denied) 2024-04-22T13:26:48.936 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51051 osd_srv_a ceph rtd unknown /proc/49667/task/51051/root (readlink: Permission denied) 2024-04-22T13:26:48.936 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51051 osd_srv_a ceph txt unknown /proc/49667/task/51051/exe (readlink: Permission denied) 2024-04-22T13:26:48.936 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51051 osd_srv_a ceph NOFD /proc/49667/task/51051/fd (opendir: Permission denied) 2024-04-22T13:26:48.936 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51052 tp_osd_tp ceph cwd unknown /proc/49667/task/51052/cwd (readlink: Permission denied) 2024-04-22T13:26:48.936 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51052 tp_osd_tp ceph rtd unknown /proc/49667/task/51052/root (readlink: Permission denied) 2024-04-22T13:26:48.936 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51052 tp_osd_tp ceph txt unknown /proc/49667/task/51052/exe (readlink: Permission denied) 2024-04-22T13:26:48.936 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51052 tp_osd_tp ceph NOFD /proc/49667/task/51052/fd (opendir: Permission denied) 2024-04-22T13:26:48.936 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51053 tp_osd_tp ceph cwd unknown /proc/49667/task/51053/cwd (readlink: Permission denied) 2024-04-22T13:26:48.936 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51053 tp_osd_tp ceph rtd unknown /proc/49667/task/51053/root (readlink: Permission denied) 2024-04-22T13:26:48.936 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51053 tp_osd_tp ceph txt unknown /proc/49667/task/51053/exe (readlink: Permission denied) 2024-04-22T13:26:48.936 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51053 tp_osd_tp ceph NOFD /proc/49667/task/51053/fd (opendir: Permission denied) 2024-04-22T13:26:48.936 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51054 tp_osd_tp ceph cwd unknown /proc/49667/task/51054/cwd (readlink: Permission denied) 2024-04-22T13:26:48.936 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51054 tp_osd_tp ceph rtd unknown /proc/49667/task/51054/root (readlink: Permission denied) 2024-04-22T13:26:48.936 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51054 tp_osd_tp ceph txt unknown /proc/49667/task/51054/exe (readlink: Permission denied) 2024-04-22T13:26:48.937 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51054 tp_osd_tp ceph NOFD /proc/49667/task/51054/fd (opendir: Permission denied) 2024-04-22T13:26:48.937 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51055 tp_osd_tp ceph cwd unknown /proc/49667/task/51055/cwd (readlink: Permission denied) 2024-04-22T13:26:48.937 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51055 tp_osd_tp ceph rtd unknown /proc/49667/task/51055/root (readlink: Permission denied) 2024-04-22T13:26:48.937 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51055 tp_osd_tp ceph txt unknown /proc/49667/task/51055/exe (readlink: Permission denied) 2024-04-22T13:26:48.937 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51055 tp_osd_tp ceph NOFD /proc/49667/task/51055/fd (opendir: Permission denied) 2024-04-22T13:26:48.937 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51056 tp_osd_tp ceph cwd unknown /proc/49667/task/51056/cwd (readlink: Permission denied) 2024-04-22T13:26:48.937 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51056 tp_osd_tp ceph rtd unknown /proc/49667/task/51056/root (readlink: Permission denied) 2024-04-22T13:26:48.937 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51056 tp_osd_tp ceph txt unknown /proc/49667/task/51056/exe (readlink: Permission denied) 2024-04-22T13:26:48.937 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51056 tp_osd_tp ceph NOFD /proc/49667/task/51056/fd (opendir: Permission denied) 2024-04-22T13:26:48.937 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51057 tp_osd_tp ceph cwd unknown /proc/49667/task/51057/cwd (readlink: Permission denied) 2024-04-22T13:26:48.937 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51057 tp_osd_tp ceph rtd unknown /proc/49667/task/51057/root (readlink: Permission denied) 2024-04-22T13:26:48.937 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51057 tp_osd_tp ceph txt unknown /proc/49667/task/51057/exe (readlink: Permission denied) 2024-04-22T13:26:48.937 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51057 tp_osd_tp ceph NOFD /proc/49667/task/51057/fd (opendir: Permission denied) 2024-04-22T13:26:48.937 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51058 tp_osd_tp ceph cwd unknown /proc/49667/task/51058/cwd (readlink: Permission denied) 2024-04-22T13:26:48.937 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51058 tp_osd_tp ceph rtd unknown /proc/49667/task/51058/root (readlink: Permission denied) 2024-04-22T13:26:48.938 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51058 tp_osd_tp ceph txt unknown /proc/49667/task/51058/exe (readlink: Permission denied) 2024-04-22T13:26:48.938 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51058 tp_osd_tp ceph NOFD /proc/49667/task/51058/fd (opendir: Permission denied) 2024-04-22T13:26:48.938 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51059 tp_osd_tp ceph cwd unknown /proc/49667/task/51059/cwd (readlink: Permission denied) 2024-04-22T13:26:48.938 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51059 tp_osd_tp ceph rtd unknown /proc/49667/task/51059/root (readlink: Permission denied) 2024-04-22T13:26:48.938 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51059 tp_osd_tp ceph txt unknown /proc/49667/task/51059/exe (readlink: Permission denied) 2024-04-22T13:26:48.938 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51059 tp_osd_tp ceph NOFD /proc/49667/task/51059/fd (opendir: Permission denied) 2024-04-22T13:26:48.938 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51060 tp_osd_tp ceph cwd unknown /proc/49667/task/51060/cwd (readlink: Permission denied) 2024-04-22T13:26:48.938 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51060 tp_osd_tp ceph rtd unknown /proc/49667/task/51060/root (readlink: Permission denied) 2024-04-22T13:26:48.938 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51060 tp_osd_tp ceph txt unknown /proc/49667/task/51060/exe (readlink: Permission denied) 2024-04-22T13:26:48.938 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51060 tp_osd_tp ceph NOFD /proc/49667/task/51060/fd (opendir: Permission denied) 2024-04-22T13:26:48.938 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51061 tp_osd_tp ceph cwd unknown /proc/49667/task/51061/cwd (readlink: Permission denied) 2024-04-22T13:26:48.938 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51061 tp_osd_tp ceph rtd unknown /proc/49667/task/51061/root (readlink: Permission denied) 2024-04-22T13:26:48.938 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51061 tp_osd_tp ceph txt unknown /proc/49667/task/51061/exe (readlink: Permission denied) 2024-04-22T13:26:48.938 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51061 tp_osd_tp ceph NOFD /proc/49667/task/51061/fd (opendir: Permission denied) 2024-04-22T13:26:48.938 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51062 tp_osd_tp ceph cwd unknown /proc/49667/task/51062/cwd (readlink: Permission denied) 2024-04-22T13:26:48.939 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51062 tp_osd_tp ceph rtd unknown /proc/49667/task/51062/root (readlink: Permission denied) 2024-04-22T13:26:48.939 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51062 tp_osd_tp ceph txt unknown /proc/49667/task/51062/exe (readlink: Permission denied) 2024-04-22T13:26:48.939 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51062 tp_osd_tp ceph NOFD /proc/49667/task/51062/fd (opendir: Permission denied) 2024-04-22T13:26:48.939 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51063 tp_osd_tp ceph cwd unknown /proc/49667/task/51063/cwd (readlink: Permission denied) 2024-04-22T13:26:48.939 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51063 tp_osd_tp ceph rtd unknown /proc/49667/task/51063/root (readlink: Permission denied) 2024-04-22T13:26:48.939 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51063 tp_osd_tp ceph txt unknown /proc/49667/task/51063/exe (readlink: Permission denied) 2024-04-22T13:26:48.939 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51063 tp_osd_tp ceph NOFD /proc/49667/task/51063/fd (opendir: Permission denied) 2024-04-22T13:26:48.939 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51064 tp_osd_tp ceph cwd unknown /proc/49667/task/51064/cwd (readlink: Permission denied) 2024-04-22T13:26:48.939 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51064 tp_osd_tp ceph rtd unknown /proc/49667/task/51064/root (readlink: Permission denied) 2024-04-22T13:26:48.939 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51064 tp_osd_tp ceph txt unknown /proc/49667/task/51064/exe (readlink: Permission denied) 2024-04-22T13:26:48.939 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51064 tp_osd_tp ceph NOFD /proc/49667/task/51064/fd (opendir: Permission denied) 2024-04-22T13:26:48.939 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51065 tp_osd_tp ceph cwd unknown /proc/49667/task/51065/cwd (readlink: Permission denied) 2024-04-22T13:26:48.939 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51065 tp_osd_tp ceph rtd unknown /proc/49667/task/51065/root (readlink: Permission denied) 2024-04-22T13:26:48.939 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51065 tp_osd_tp ceph txt unknown /proc/49667/task/51065/exe (readlink: Permission denied) 2024-04-22T13:26:48.940 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51065 tp_osd_tp ceph NOFD /proc/49667/task/51065/fd (opendir: Permission denied) 2024-04-22T13:26:48.940 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51066 tp_osd_tp ceph cwd unknown /proc/49667/task/51066/cwd (readlink: Permission denied) 2024-04-22T13:26:48.940 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51066 tp_osd_tp ceph rtd unknown /proc/49667/task/51066/root (readlink: Permission denied) 2024-04-22T13:26:48.940 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51066 tp_osd_tp ceph txt unknown /proc/49667/task/51066/exe (readlink: Permission denied) 2024-04-22T13:26:48.940 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51066 tp_osd_tp ceph NOFD /proc/49667/task/51066/fd (opendir: Permission denied) 2024-04-22T13:26:48.940 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51067 tp_osd_tp ceph cwd unknown /proc/49667/task/51067/cwd (readlink: Permission denied) 2024-04-22T13:26:48.940 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51067 tp_osd_tp ceph rtd unknown /proc/49667/task/51067/root (readlink: Permission denied) 2024-04-22T13:26:48.940 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51067 tp_osd_tp ceph txt unknown /proc/49667/task/51067/exe (readlink: Permission denied) 2024-04-22T13:26:48.940 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51067 tp_osd_tp ceph NOFD /proc/49667/task/51067/fd (opendir: Permission denied) 2024-04-22T13:26:48.940 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51068 osd_srv_h ceph cwd unknown /proc/49667/task/51068/cwd (readlink: Permission denied) 2024-04-22T13:26:48.940 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51068 osd_srv_h ceph rtd unknown /proc/49667/task/51068/root (readlink: Permission denied) 2024-04-22T13:26:48.940 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51068 osd_srv_h ceph txt unknown /proc/49667/task/51068/exe (readlink: Permission denied) 2024-04-22T13:26:48.940 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 49667 51068 osd_srv_h ceph NOFD /proc/49667/task/51068/fd (opendir: Permission denied) 2024-04-22T13:26:48.941 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 51080 root cwd unknown /proc/51080/cwd (readlink: Permission denied) 2024-04-22T13:26:48.941 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 51080 root rtd unknown /proc/51080/root (readlink: Permission denied) 2024-04-22T13:26:48.941 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 51080 root txt unknown /proc/51080/exe (readlink: Permission denied) 2024-04-22T13:26:48.941 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 51080 root NOFD /proc/51080/fd (opendir: Permission denied) 2024-04-22T13:26:48.941 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 51108 root cwd unknown /proc/51108/cwd (readlink: Permission denied) 2024-04-22T13:26:48.941 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 51108 root rtd unknown /proc/51108/root (readlink: Permission denied) 2024-04-22T13:26:48.941 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 51108 root txt unknown /proc/51108/exe (readlink: Permission denied) 2024-04-22T13:26:48.941 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 51108 root NOFD /proc/51108/fd (opendir: Permission denied) 2024-04-22T13:26:48.941 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/7 53608 root cwd unknown /proc/53608/cwd (readlink: Permission denied) 2024-04-22T13:26:48.941 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/7 53608 root rtd unknown /proc/53608/root (readlink: Permission denied) 2024-04-22T13:26:48.941 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/7 53608 root txt unknown /proc/53608/exe (readlink: Permission denied) 2024-04-22T13:26:48.941 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/7 53608 root NOFD /proc/53608/fd (opendir: Permission denied) 2024-04-22T13:26:48.941 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/7 53685 root cwd unknown /proc/53685/cwd (readlink: Permission denied) 2024-04-22T13:26:48.941 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/7 53685 root rtd unknown /proc/53685/root (readlink: Permission denied) 2024-04-22T13:26:48.941 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/7 53685 root txt unknown /proc/53685/exe (readlink: Permission denied) 2024-04-22T13:26:48.942 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/7 53685 root NOFD /proc/53685/fd (opendir: Permission denied) 2024-04-22T13:26:48.942 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/1 54047 root cwd unknown /proc/54047/cwd (readlink: Permission denied) 2024-04-22T13:26:48.942 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/1 54047 root rtd unknown /proc/54047/root (readlink: Permission denied) 2024-04-22T13:26:48.942 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/1 54047 root txt unknown /proc/54047/exe (readlink: Permission denied) 2024-04-22T13:26:48.942 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/1 54047 root NOFD /proc/54047/fd (opendir: Permission denied) 2024-04-22T13:26:48.942 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 56517 root cwd unknown /proc/56517/cwd (readlink: Permission denied) 2024-04-22T13:26:48.942 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 56517 root rtd unknown /proc/56517/root (readlink: Permission denied) 2024-04-22T13:26:48.942 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 56517 root txt unknown /proc/56517/exe (readlink: Permission denied) 2024-04-22T13:26:48.942 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 56517 root NOFD /proc/56517/fd (opendir: Permission denied) 2024-04-22T13:26:48.942 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 56519 root cwd unknown /proc/56519/cwd (readlink: Permission denied) 2024-04-22T13:26:48.942 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 56519 root rtd unknown /proc/56519/root (readlink: Permission denied) 2024-04-22T13:26:48.942 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 56519 root txt unknown /proc/56519/exe (readlink: Permission denied) 2024-04-22T13:26:48.942 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 56519 root NOFD /proc/56519/fd (opendir: Permission denied) 2024-04-22T13:26:48.942 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 ceph cwd unknown /proc/56521/cwd (readlink: Permission denied) 2024-04-22T13:26:48.943 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 ceph rtd unknown /proc/56521/root (readlink: Permission denied) 2024-04-22T13:26:48.943 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 ceph txt unknown /proc/56521/exe (readlink: Permission denied) 2024-04-22T13:26:48.943 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 ceph NOFD /proc/56521/fd (opendir: Permission denied) 2024-04-22T13:26:48.943 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56522 log ceph cwd unknown /proc/56521/task/56522/cwd (readlink: Permission denied) 2024-04-22T13:26:48.943 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56522 log ceph rtd unknown /proc/56521/task/56522/root (readlink: Permission denied) 2024-04-22T13:26:48.943 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56522 log ceph txt unknown /proc/56521/task/56522/exe (readlink: Permission denied) 2024-04-22T13:26:48.943 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56522 log ceph NOFD /proc/56521/task/56522/fd (opendir: Permission denied) 2024-04-22T13:26:48.943 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56524 msgr-work ceph cwd unknown /proc/56521/task/56524/cwd (readlink: Permission denied) 2024-04-22T13:26:48.943 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56524 msgr-work ceph rtd unknown /proc/56521/task/56524/root (readlink: Permission denied) 2024-04-22T13:26:48.943 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56524 msgr-work ceph txt unknown /proc/56521/task/56524/exe (readlink: Permission denied) 2024-04-22T13:26:48.943 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56524 msgr-work ceph NOFD /proc/56521/task/56524/fd (opendir: Permission denied) 2024-04-22T13:26:48.943 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56525 msgr-work ceph cwd unknown /proc/56521/task/56525/cwd (readlink: Permission denied) 2024-04-22T13:26:48.943 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56525 msgr-work ceph rtd unknown /proc/56521/task/56525/root (readlink: Permission denied) 2024-04-22T13:26:48.943 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56525 msgr-work ceph txt unknown /proc/56521/task/56525/exe (readlink: Permission denied) 2024-04-22T13:26:48.943 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56525 msgr-work ceph NOFD /proc/56521/task/56525/fd (opendir: Permission denied) 2024-04-22T13:26:48.944 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56526 msgr-work ceph cwd unknown /proc/56521/task/56526/cwd (readlink: Permission denied) 2024-04-22T13:26:48.944 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56526 msgr-work ceph rtd unknown /proc/56521/task/56526/root (readlink: Permission denied) 2024-04-22T13:26:48.944 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56526 msgr-work ceph txt unknown /proc/56521/task/56526/exe (readlink: Permission denied) 2024-04-22T13:26:48.944 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56526 msgr-work ceph NOFD /proc/56521/task/56526/fd (opendir: Permission denied) 2024-04-22T13:26:48.944 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56530 service ceph cwd unknown /proc/56521/task/56530/cwd (readlink: Permission denied) 2024-04-22T13:26:48.944 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56530 service ceph rtd unknown /proc/56521/task/56530/root (readlink: Permission denied) 2024-04-22T13:26:48.944 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56530 service ceph txt unknown /proc/56521/task/56530/exe (readlink: Permission denied) 2024-04-22T13:26:48.944 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56530 service ceph NOFD /proc/56521/task/56530/fd (opendir: Permission denied) 2024-04-22T13:26:48.944 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56531 admin_soc ceph cwd unknown /proc/56521/task/56531/cwd (readlink: Permission denied) 2024-04-22T13:26:48.944 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56531 admin_soc ceph rtd unknown /proc/56521/task/56531/root (readlink: Permission denied) 2024-04-22T13:26:48.944 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56531 admin_soc ceph txt unknown /proc/56521/task/56531/exe (readlink: Permission denied) 2024-04-22T13:26:48.944 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56531 admin_soc ceph NOFD /proc/56521/task/56531/fd (opendir: Permission denied) 2024-04-22T13:26:48.944 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56673 signal_ha ceph cwd unknown /proc/56521/task/56673/cwd (readlink: Permission denied) 2024-04-22T13:26:48.944 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56673 signal_ha ceph rtd unknown /proc/56521/task/56673/root (readlink: Permission denied) 2024-04-22T13:26:48.945 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56673 signal_ha ceph txt unknown /proc/56521/task/56673/exe (readlink: Permission denied) 2024-04-22T13:26:48.945 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56673 signal_ha ceph NOFD /proc/56521/task/56673/fd (opendir: Permission denied) 2024-04-22T13:26:48.945 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56674 io_contex ceph cwd unknown /proc/56521/task/56674/cwd (readlink: Permission denied) 2024-04-22T13:26:48.945 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56674 io_contex ceph rtd unknown /proc/56521/task/56674/root (readlink: Permission denied) 2024-04-22T13:26:48.945 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56674 io_contex ceph txt unknown /proc/56521/task/56674/exe (readlink: Permission denied) 2024-04-22T13:26:48.945 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56674 io_contex ceph NOFD /proc/56521/task/56674/fd (opendir: Permission denied) 2024-04-22T13:26:48.945 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56675 io_contex ceph cwd unknown /proc/56521/task/56675/cwd (readlink: Permission denied) 2024-04-22T13:26:48.945 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56675 io_contex ceph rtd unknown /proc/56521/task/56675/root (readlink: Permission denied) 2024-04-22T13:26:48.945 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56675 io_contex ceph txt unknown /proc/56521/task/56675/exe (readlink: Permission denied) 2024-04-22T13:26:48.945 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56675 io_contex ceph NOFD /proc/56521/task/56675/fd (opendir: Permission denied) 2024-04-22T13:26:48.945 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56788 OpHistory ceph cwd unknown /proc/56521/task/56788/cwd (readlink: Permission denied) 2024-04-22T13:26:48.945 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56788 OpHistory ceph rtd unknown /proc/56521/task/56788/root (readlink: Permission denied) 2024-04-22T13:26:48.945 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56788 OpHistory ceph txt unknown /proc/56521/task/56788/exe (readlink: Permission denied) 2024-04-22T13:26:48.945 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56788 OpHistory ceph NOFD /proc/56521/task/56788/fd (opendir: Permission denied) 2024-04-22T13:26:48.945 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56789 ceph_time ceph cwd unknown /proc/56521/task/56789/cwd (readlink: Permission denied) 2024-04-22T13:26:48.946 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56789 ceph_time ceph rtd unknown /proc/56521/task/56789/root (readlink: Permission denied) 2024-04-22T13:26:48.946 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56789 ceph_time ceph txt unknown /proc/56521/task/56789/exe (readlink: Permission denied) 2024-04-22T13:26:48.946 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56789 ceph_time ceph NOFD /proc/56521/task/56789/fd (opendir: Permission denied) 2024-04-22T13:26:48.946 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56798 ceph-osd ceph cwd unknown /proc/56521/task/56798/cwd (readlink: Permission denied) 2024-04-22T13:26:48.946 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56798 ceph-osd ceph rtd unknown /proc/56521/task/56798/root (readlink: Permission denied) 2024-04-22T13:26:48.946 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56798 ceph-osd ceph txt unknown /proc/56521/task/56798/exe (readlink: Permission denied) 2024-04-22T13:26:48.946 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56798 ceph-osd ceph NOFD /proc/56521/task/56798/fd (opendir: Permission denied) 2024-04-22T13:26:48.946 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56814 ceph-osd ceph cwd unknown /proc/56521/task/56814/cwd (readlink: Permission denied) 2024-04-22T13:26:48.946 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56814 ceph-osd ceph rtd unknown /proc/56521/task/56814/root (readlink: Permission denied) 2024-04-22T13:26:48.946 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56814 ceph-osd ceph txt unknown /proc/56521/task/56814/exe (readlink: Permission denied) 2024-04-22T13:26:48.946 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56814 ceph-osd ceph NOFD /proc/56521/task/56814/fd (opendir: Permission denied) 2024-04-22T13:26:48.946 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56850 ceph-osd ceph cwd unknown /proc/56521/task/56850/cwd (readlink: Permission denied) 2024-04-22T13:26:48.946 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56850 ceph-osd ceph rtd unknown /proc/56521/task/56850/root (readlink: Permission denied) 2024-04-22T13:26:48.947 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56850 ceph-osd ceph txt unknown /proc/56521/task/56850/exe (readlink: Permission denied) 2024-04-22T13:26:48.947 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56850 ceph-osd ceph NOFD /proc/56521/task/56850/fd (opendir: Permission denied) 2024-04-22T13:26:48.947 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56853 ceph-osd ceph cwd unknown /proc/56521/task/56853/cwd (readlink: Permission denied) 2024-04-22T13:26:48.947 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56853 ceph-osd ceph rtd unknown /proc/56521/task/56853/root (readlink: Permission denied) 2024-04-22T13:26:48.947 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56853 ceph-osd ceph txt unknown /proc/56521/task/56853/exe (readlink: Permission denied) 2024-04-22T13:26:48.947 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56853 ceph-osd ceph NOFD /proc/56521/task/56853/fd (opendir: Permission denied) 2024-04-22T13:26:48.947 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56875 ceph-osd ceph cwd unknown /proc/56521/task/56875/cwd (readlink: Permission denied) 2024-04-22T13:26:48.947 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56875 ceph-osd ceph rtd unknown /proc/56521/task/56875/root (readlink: Permission denied) 2024-04-22T13:26:48.947 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56875 ceph-osd ceph txt unknown /proc/56521/task/56875/exe (readlink: Permission denied) 2024-04-22T13:26:48.947 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56875 ceph-osd ceph NOFD /proc/56521/task/56875/fd (opendir: Permission denied) 2024-04-22T13:26:48.947 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56899 ceph-osd ceph cwd unknown /proc/56521/task/56899/cwd (readlink: Permission denied) 2024-04-22T13:26:48.947 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56899 ceph-osd ceph rtd unknown /proc/56521/task/56899/root (readlink: Permission denied) 2024-04-22T13:26:48.947 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56899 ceph-osd ceph txt unknown /proc/56521/task/56899/exe (readlink: Permission denied) 2024-04-22T13:26:48.948 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56899 ceph-osd ceph NOFD /proc/56521/task/56899/fd (opendir: Permission denied) 2024-04-22T13:26:48.948 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56907 ceph-osd ceph cwd unknown /proc/56521/task/56907/cwd (readlink: Permission denied) 2024-04-22T13:26:48.948 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56907 ceph-osd ceph rtd unknown /proc/56521/task/56907/root (readlink: Permission denied) 2024-04-22T13:26:48.948 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56907 ceph-osd ceph txt unknown /proc/56521/task/56907/exe (readlink: Permission denied) 2024-04-22T13:26:48.948 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56907 ceph-osd ceph NOFD /proc/56521/task/56907/fd (opendir: Permission denied) 2024-04-22T13:26:48.948 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56927 ceph-osd ceph cwd unknown /proc/56521/task/56927/cwd (readlink: Permission denied) 2024-04-22T13:26:48.948 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56927 ceph-osd ceph rtd unknown /proc/56521/task/56927/root (readlink: Permission denied) 2024-04-22T13:26:48.948 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56927 ceph-osd ceph txt unknown /proc/56521/task/56927/exe (readlink: Permission denied) 2024-04-22T13:26:48.948 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56927 ceph-osd ceph NOFD /proc/56521/task/56927/fd (opendir: Permission denied) 2024-04-22T13:26:48.948 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56929 ceph-osd ceph cwd unknown /proc/56521/task/56929/cwd (readlink: Permission denied) 2024-04-22T13:26:48.948 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56929 ceph-osd ceph rtd unknown /proc/56521/task/56929/root (readlink: Permission denied) 2024-04-22T13:26:48.948 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56929 ceph-osd ceph txt unknown /proc/56521/task/56929/exe (readlink: Permission denied) 2024-04-22T13:26:48.948 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56929 ceph-osd ceph NOFD /proc/56521/task/56929/fd (opendir: Permission denied) 2024-04-22T13:26:48.948 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56930 safe_time ceph cwd unknown /proc/56521/task/56930/cwd (readlink: Permission denied) 2024-04-22T13:26:48.949 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56930 safe_time ceph rtd unknown /proc/56521/task/56930/root (readlink: Permission denied) 2024-04-22T13:26:48.949 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56930 safe_time ceph txt unknown /proc/56521/task/56930/exe (readlink: Permission denied) 2024-04-22T13:26:48.949 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56930 safe_time ceph NOFD /proc/56521/task/56930/fd (opendir: Permission denied) 2024-04-22T13:26:48.949 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56931 safe_time ceph cwd unknown /proc/56521/task/56931/cwd (readlink: Permission denied) 2024-04-22T13:26:48.949 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56931 safe_time ceph rtd unknown /proc/56521/task/56931/root (readlink: Permission denied) 2024-04-22T13:26:48.949 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56931 safe_time ceph txt unknown /proc/56521/task/56931/exe (readlink: Permission denied) 2024-04-22T13:26:48.949 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56931 safe_time ceph NOFD /proc/56521/task/56931/fd (opendir: Permission denied) 2024-04-22T13:26:48.949 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56932 safe_time ceph cwd unknown /proc/56521/task/56932/cwd (readlink: Permission denied) 2024-04-22T13:26:48.949 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56932 safe_time ceph rtd unknown /proc/56521/task/56932/root (readlink: Permission denied) 2024-04-22T13:26:48.949 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56932 safe_time ceph txt unknown /proc/56521/task/56932/exe (readlink: Permission denied) 2024-04-22T13:26:48.949 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56932 safe_time ceph NOFD /proc/56521/task/56932/fd (opendir: Permission denied) 2024-04-22T13:26:48.949 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56933 safe_time ceph cwd unknown /proc/56521/task/56933/cwd (readlink: Permission denied) 2024-04-22T13:26:48.949 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56933 safe_time ceph rtd unknown /proc/56521/task/56933/root (readlink: Permission denied) 2024-04-22T13:26:48.950 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56933 safe_time ceph txt unknown /proc/56521/task/56933/exe (readlink: Permission denied) 2024-04-22T13:26:48.950 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56933 safe_time ceph NOFD /proc/56521/task/56933/fd (opendir: Permission denied) 2024-04-22T13:26:48.950 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56934 fn_anonym ceph cwd unknown /proc/56521/task/56934/cwd (readlink: Permission denied) 2024-04-22T13:26:48.950 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56934 fn_anonym ceph rtd unknown /proc/56521/task/56934/root (readlink: Permission denied) 2024-04-22T13:26:48.950 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56934 fn_anonym ceph txt unknown /proc/56521/task/56934/exe (readlink: Permission denied) 2024-04-22T13:26:48.950 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56934 fn_anonym ceph NOFD /proc/56521/task/56934/fd (opendir: Permission denied) 2024-04-22T13:26:48.950 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56939 rocksdb:l ceph cwd unknown /proc/56521/task/56939/cwd (readlink: Permission denied) 2024-04-22T13:26:48.950 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56939 rocksdb:l ceph rtd unknown /proc/56521/task/56939/root (readlink: Permission denied) 2024-04-22T13:26:48.950 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56939 rocksdb:l ceph txt unknown /proc/56521/task/56939/exe (readlink: Permission denied) 2024-04-22T13:26:48.950 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56939 rocksdb:l ceph NOFD /proc/56521/task/56939/fd (opendir: Permission denied) 2024-04-22T13:26:48.950 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56940 rocksdb:l ceph cwd unknown /proc/56521/task/56940/cwd (readlink: Permission denied) 2024-04-22T13:26:48.950 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56940 rocksdb:l ceph rtd unknown /proc/56521/task/56940/root (readlink: Permission denied) 2024-04-22T13:26:48.950 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56940 rocksdb:l ceph txt unknown /proc/56521/task/56940/exe (readlink: Permission denied) 2024-04-22T13:26:48.950 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56940 rocksdb:l ceph NOFD /proc/56521/task/56940/fd (opendir: Permission denied) 2024-04-22T13:26:48.951 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56941 rocksdb:l ceph cwd unknown /proc/56521/task/56941/cwd (readlink: Permission denied) 2024-04-22T13:26:48.951 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56941 rocksdb:l ceph rtd unknown /proc/56521/task/56941/root (readlink: Permission denied) 2024-04-22T13:26:48.951 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56941 rocksdb:l ceph txt unknown /proc/56521/task/56941/exe (readlink: Permission denied) 2024-04-22T13:26:48.951 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56941 rocksdb:l ceph NOFD /proc/56521/task/56941/fd (opendir: Permission denied) 2024-04-22T13:26:48.951 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56942 rocksdb:h ceph cwd unknown /proc/56521/task/56942/cwd (readlink: Permission denied) 2024-04-22T13:26:48.951 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56942 rocksdb:h ceph rtd unknown /proc/56521/task/56942/root (readlink: Permission denied) 2024-04-22T13:26:48.951 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56942 rocksdb:h ceph txt unknown /proc/56521/task/56942/exe (readlink: Permission denied) 2024-04-22T13:26:48.951 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 56942 rocksdb:h ceph NOFD /proc/56521/task/56942/fd (opendir: Permission denied) 2024-04-22T13:26:48.951 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57441 bstore_ai ceph cwd unknown /proc/56521/task/57441/cwd (readlink: Permission denied) 2024-04-22T13:26:48.951 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57441 bstore_ai ceph rtd unknown /proc/56521/task/57441/root (readlink: Permission denied) 2024-04-22T13:26:48.951 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57441 bstore_ai ceph txt unknown /proc/56521/task/57441/exe (readlink: Permission denied) 2024-04-22T13:26:48.951 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57441 bstore_ai ceph NOFD /proc/56521/task/57441/fd (opendir: Permission denied) 2024-04-22T13:26:48.951 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57623 bstore_ai ceph cwd unknown /proc/56521/task/57623/cwd (readlink: Permission denied) 2024-04-22T13:26:48.951 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57623 bstore_ai ceph rtd unknown /proc/56521/task/57623/root (readlink: Permission denied) 2024-04-22T13:26:48.951 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57623 bstore_ai ceph txt unknown /proc/56521/task/57623/exe (readlink: Permission denied) 2024-04-22T13:26:48.952 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57623 bstore_ai ceph NOFD /proc/56521/task/57623/fd (opendir: Permission denied) 2024-04-22T13:26:48.952 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57805 ceph-osd ceph cwd unknown /proc/56521/task/57805/cwd (readlink: Permission denied) 2024-04-22T13:26:48.952 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57805 ceph-osd ceph rtd unknown /proc/56521/task/57805/root (readlink: Permission denied) 2024-04-22T13:26:48.952 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57805 ceph-osd ceph txt unknown /proc/56521/task/57805/exe (readlink: Permission denied) 2024-04-22T13:26:48.952 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57805 ceph-osd ceph NOFD /proc/56521/task/57805/fd (opendir: Permission denied) 2024-04-22T13:26:48.952 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57806 cfin ceph cwd unknown /proc/56521/task/57806/cwd (readlink: Permission denied) 2024-04-22T13:26:48.952 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57806 cfin ceph rtd unknown /proc/56521/task/57806/root (readlink: Permission denied) 2024-04-22T13:26:48.952 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57806 cfin ceph txt unknown /proc/56521/task/57806/exe (readlink: Permission denied) 2024-04-22T13:26:48.952 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57806 cfin ceph NOFD /proc/56521/task/57806/fd (opendir: Permission denied) 2024-04-22T13:26:48.952 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57807 bstore_kv ceph cwd unknown /proc/56521/task/57807/cwd (readlink: Permission denied) 2024-04-22T13:26:48.952 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57807 bstore_kv ceph rtd unknown /proc/56521/task/57807/root (readlink: Permission denied) 2024-04-22T13:26:48.952 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57807 bstore_kv ceph txt unknown /proc/56521/task/57807/exe (readlink: Permission denied) 2024-04-22T13:26:48.952 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57807 bstore_kv ceph NOFD /proc/56521/task/57807/fd (opendir: Permission denied) 2024-04-22T13:26:48.952 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57808 bstore_kv ceph cwd unknown /proc/56521/task/57808/cwd (readlink: Permission denied) 2024-04-22T13:26:48.952 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57808 bstore_kv ceph rtd unknown /proc/56521/task/57808/root (readlink: Permission denied) 2024-04-22T13:26:48.952 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57808 bstore_kv ceph txt unknown /proc/56521/task/57808/exe (readlink: Permission denied) 2024-04-22T13:26:48.952 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57808 bstore_kv ceph NOFD /proc/56521/task/57808/fd (opendir: Permission denied) 2024-04-22T13:26:48.953 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57809 bstore_me ceph cwd unknown /proc/56521/task/57809/cwd (readlink: Permission denied) 2024-04-22T13:26:48.953 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57809 bstore_me ceph rtd unknown /proc/56521/task/57809/root (readlink: Permission denied) 2024-04-22T13:26:48.953 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57809 bstore_me ceph txt unknown /proc/56521/task/57809/exe (readlink: Permission denied) 2024-04-22T13:26:48.953 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57809 bstore_me ceph NOFD /proc/56521/task/57809/fd (opendir: Permission denied) 2024-04-22T13:26:48.953 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57810 ms_dispat ceph cwd unknown /proc/56521/task/57810/cwd (readlink: Permission denied) 2024-04-22T13:26:48.953 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57810 ms_dispat ceph rtd unknown /proc/56521/task/57810/root (readlink: Permission denied) 2024-04-22T13:26:48.953 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57810 ms_dispat ceph txt unknown /proc/56521/task/57810/exe (readlink: Permission denied) 2024-04-22T13:26:48.953 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57810 ms_dispat ceph NOFD /proc/56521/task/57810/fd (opendir: Permission denied) 2024-04-22T13:26:48.953 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57811 ms_local ceph cwd unknown /proc/56521/task/57811/cwd (readlink: Permission denied) 2024-04-22T13:26:48.953 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57811 ms_local ceph rtd unknown /proc/56521/task/57811/root (readlink: Permission denied) 2024-04-22T13:26:48.953 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57811 ms_local ceph txt unknown /proc/56521/task/57811/exe (readlink: Permission denied) 2024-04-22T13:26:48.953 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57811 ms_local ceph NOFD /proc/56521/task/57811/fd (opendir: Permission denied) 2024-04-22T13:26:48.953 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57812 safe_time ceph cwd unknown /proc/56521/task/57812/cwd (readlink: Permission denied) 2024-04-22T13:26:48.953 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57812 safe_time ceph rtd unknown /proc/56521/task/57812/root (readlink: Permission denied) 2024-04-22T13:26:48.954 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57812 safe_time ceph txt unknown /proc/56521/task/57812/exe (readlink: Permission denied) 2024-04-22T13:26:48.954 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57812 safe_time ceph NOFD /proc/56521/task/57812/fd (opendir: Permission denied) 2024-04-22T13:26:48.954 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57813 safe_time ceph cwd unknown /proc/56521/task/57813/cwd (readlink: Permission denied) 2024-04-22T13:26:48.954 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57813 safe_time ceph rtd unknown /proc/56521/task/57813/root (readlink: Permission denied) 2024-04-22T13:26:48.954 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57813 safe_time ceph txt unknown /proc/56521/task/57813/exe (readlink: Permission denied) 2024-04-22T13:26:48.954 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57813 safe_time ceph NOFD /proc/56521/task/57813/fd (opendir: Permission denied) 2024-04-22T13:26:48.954 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57814 ms_dispat ceph cwd unknown /proc/56521/task/57814/cwd (readlink: Permission denied) 2024-04-22T13:26:48.954 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57814 ms_dispat ceph rtd unknown /proc/56521/task/57814/root (readlink: Permission denied) 2024-04-22T13:26:48.954 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57814 ms_dispat ceph txt unknown /proc/56521/task/57814/exe (readlink: Permission denied) 2024-04-22T13:26:48.954 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57814 ms_dispat ceph NOFD /proc/56521/task/57814/fd (opendir: Permission denied) 2024-04-22T13:26:48.954 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57815 ms_local ceph cwd unknown /proc/56521/task/57815/cwd (readlink: Permission denied) 2024-04-22T13:26:48.954 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57815 ms_local ceph rtd unknown /proc/56521/task/57815/root (readlink: Permission denied) 2024-04-22T13:26:48.954 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57815 ms_local ceph txt unknown /proc/56521/task/57815/exe (readlink: Permission denied) 2024-04-22T13:26:48.954 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57815 ms_local ceph NOFD /proc/56521/task/57815/fd (opendir: Permission denied) 2024-04-22T13:26:48.954 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57816 ms_dispat ceph cwd unknown /proc/56521/task/57816/cwd (readlink: Permission denied) 2024-04-22T13:26:48.955 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57816 ms_dispat ceph rtd unknown /proc/56521/task/57816/root (readlink: Permission denied) 2024-04-22T13:26:48.955 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57816 ms_dispat ceph txt unknown /proc/56521/task/57816/exe (readlink: Permission denied) 2024-04-22T13:26:48.955 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57816 ms_dispat ceph NOFD /proc/56521/task/57816/fd (opendir: Permission denied) 2024-04-22T13:26:48.955 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57817 ms_local ceph cwd unknown /proc/56521/task/57817/cwd (readlink: Permission denied) 2024-04-22T13:26:48.955 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57817 ms_local ceph rtd unknown /proc/56521/task/57817/root (readlink: Permission denied) 2024-04-22T13:26:48.955 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57817 ms_local ceph txt unknown /proc/56521/task/57817/exe (readlink: Permission denied) 2024-04-22T13:26:48.955 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57817 ms_local ceph NOFD /proc/56521/task/57817/fd (opendir: Permission denied) 2024-04-22T13:26:48.955 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57818 ms_dispat ceph cwd unknown /proc/56521/task/57818/cwd (readlink: Permission denied) 2024-04-22T13:26:48.955 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57818 ms_dispat ceph rtd unknown /proc/56521/task/57818/root (readlink: Permission denied) 2024-04-22T13:26:48.955 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57818 ms_dispat ceph txt unknown /proc/56521/task/57818/exe (readlink: Permission denied) 2024-04-22T13:26:48.955 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57818 ms_dispat ceph NOFD /proc/56521/task/57818/fd (opendir: Permission denied) 2024-04-22T13:26:48.955 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57819 ms_local ceph cwd unknown /proc/56521/task/57819/cwd (readlink: Permission denied) 2024-04-22T13:26:48.955 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57819 ms_local ceph rtd unknown /proc/56521/task/57819/root (readlink: Permission denied) 2024-04-22T13:26:48.955 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57819 ms_local ceph txt unknown /proc/56521/task/57819/exe (readlink: Permission denied) 2024-04-22T13:26:48.955 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57819 ms_local ceph NOFD /proc/56521/task/57819/fd (opendir: Permission denied) 2024-04-22T13:26:48.955 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57820 ms_dispat ceph cwd unknown /proc/56521/task/57820/cwd (readlink: Permission denied) 2024-04-22T13:26:48.956 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57820 ms_dispat ceph rtd unknown /proc/56521/task/57820/root (readlink: Permission denied) 2024-04-22T13:26:48.956 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57820 ms_dispat ceph txt unknown /proc/56521/task/57820/exe (readlink: Permission denied) 2024-04-22T13:26:48.956 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57820 ms_dispat ceph NOFD /proc/56521/task/57820/fd (opendir: Permission denied) 2024-04-22T13:26:48.956 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57821 ms_local ceph cwd unknown /proc/56521/task/57821/cwd (readlink: Permission denied) 2024-04-22T13:26:48.956 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57821 ms_local ceph rtd unknown /proc/56521/task/57821/root (readlink: Permission denied) 2024-04-22T13:26:48.956 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57821 ms_local ceph txt unknown /proc/56521/task/57821/exe (readlink: Permission denied) 2024-04-22T13:26:48.956 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57821 ms_local ceph NOFD /proc/56521/task/57821/fd (opendir: Permission denied) 2024-04-22T13:26:48.956 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57822 ms_dispat ceph cwd unknown /proc/56521/task/57822/cwd (readlink: Permission denied) 2024-04-22T13:26:48.956 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57822 ms_dispat ceph rtd unknown /proc/56521/task/57822/root (readlink: Permission denied) 2024-04-22T13:26:48.956 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57822 ms_dispat ceph txt unknown /proc/56521/task/57822/exe (readlink: Permission denied) 2024-04-22T13:26:48.956 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57822 ms_dispat ceph NOFD /proc/56521/task/57822/fd (opendir: Permission denied) 2024-04-22T13:26:48.956 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57823 ms_local ceph cwd unknown /proc/56521/task/57823/cwd (readlink: Permission denied) 2024-04-22T13:26:48.956 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57823 ms_local ceph rtd unknown /proc/56521/task/57823/root (readlink: Permission denied) 2024-04-22T13:26:48.956 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57823 ms_local ceph txt unknown /proc/56521/task/57823/exe (readlink: Permission denied) 2024-04-22T13:26:48.957 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57823 ms_local ceph NOFD /proc/56521/task/57823/fd (opendir: Permission denied) 2024-04-22T13:26:48.957 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57824 ms_dispat ceph cwd unknown /proc/56521/task/57824/cwd (readlink: Permission denied) 2024-04-22T13:26:48.957 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57824 ms_dispat ceph rtd unknown /proc/56521/task/57824/root (readlink: Permission denied) 2024-04-22T13:26:48.957 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57824 ms_dispat ceph txt unknown /proc/56521/task/57824/exe (readlink: Permission denied) 2024-04-22T13:26:48.957 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57824 ms_dispat ceph NOFD /proc/56521/task/57824/fd (opendir: Permission denied) 2024-04-22T13:26:48.957 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57825 ms_local ceph cwd unknown /proc/56521/task/57825/cwd (readlink: Permission denied) 2024-04-22T13:26:48.957 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57825 ms_local ceph rtd unknown /proc/56521/task/57825/root (readlink: Permission denied) 2024-04-22T13:26:48.957 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57825 ms_local ceph txt unknown /proc/56521/task/57825/exe (readlink: Permission denied) 2024-04-22T13:26:48.957 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57825 ms_local ceph NOFD /proc/56521/task/57825/fd (opendir: Permission denied) 2024-04-22T13:26:48.957 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57826 fn_anonym ceph cwd unknown /proc/56521/task/57826/cwd (readlink: Permission denied) 2024-04-22T13:26:48.957 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57826 fn_anonym ceph rtd unknown /proc/56521/task/57826/root (readlink: Permission denied) 2024-04-22T13:26:48.957 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57826 fn_anonym ceph txt unknown /proc/56521/task/57826/exe (readlink: Permission denied) 2024-04-22T13:26:48.957 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57826 fn_anonym ceph NOFD /proc/56521/task/57826/fd (opendir: Permission denied) 2024-04-22T13:26:48.957 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57827 finisher ceph cwd unknown /proc/56521/task/57827/cwd (readlink: Permission denied) 2024-04-22T13:26:48.957 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57827 finisher ceph rtd unknown /proc/56521/task/57827/root (readlink: Permission denied) 2024-04-22T13:26:48.957 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57827 finisher ceph txt unknown /proc/56521/task/57827/exe (readlink: Permission denied) 2024-04-22T13:26:48.958 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57827 finisher ceph NOFD /proc/56521/task/57827/fd (opendir: Permission denied) 2024-04-22T13:26:48.958 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57828 safe_time ceph cwd unknown /proc/56521/task/57828/cwd (readlink: Permission denied) 2024-04-22T13:26:48.958 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57828 safe_time ceph rtd unknown /proc/56521/task/57828/root (readlink: Permission denied) 2024-04-22T13:26:48.958 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57828 safe_time ceph txt unknown /proc/56521/task/57828/exe (readlink: Permission denied) 2024-04-22T13:26:48.958 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57828 safe_time ceph NOFD /proc/56521/task/57828/fd (opendir: Permission denied) 2024-04-22T13:26:48.958 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57829 safe_time ceph cwd unknown /proc/56521/task/57829/cwd (readlink: Permission denied) 2024-04-22T13:26:48.958 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57829 safe_time ceph rtd unknown /proc/56521/task/57829/root (readlink: Permission denied) 2024-04-22T13:26:48.958 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57829 safe_time ceph txt unknown /proc/56521/task/57829/exe (readlink: Permission denied) 2024-04-22T13:26:48.958 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57829 safe_time ceph NOFD /proc/56521/task/57829/fd (opendir: Permission denied) 2024-04-22T13:26:48.958 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57830 ceph-osd ceph cwd unknown /proc/56521/task/57830/cwd (readlink: Permission denied) 2024-04-22T13:26:48.958 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57830 ceph-osd ceph rtd unknown /proc/56521/task/57830/root (readlink: Permission denied) 2024-04-22T13:26:48.958 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57830 ceph-osd ceph txt unknown /proc/56521/task/57830/exe (readlink: Permission denied) 2024-04-22T13:26:48.958 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57830 ceph-osd ceph NOFD /proc/56521/task/57830/fd (opendir: Permission denied) 2024-04-22T13:26:48.958 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57831 osd_srv_a ceph cwd unknown /proc/56521/task/57831/cwd (readlink: Permission denied) 2024-04-22T13:26:48.958 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57831 osd_srv_a ceph rtd unknown /proc/56521/task/57831/root (readlink: Permission denied) 2024-04-22T13:26:48.959 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57831 osd_srv_a ceph txt unknown /proc/56521/task/57831/exe (readlink: Permission denied) 2024-04-22T13:26:48.959 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57831 osd_srv_a ceph NOFD /proc/56521/task/57831/fd (opendir: Permission denied) 2024-04-22T13:26:48.959 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57832 tp_osd_tp ceph cwd unknown /proc/56521/task/57832/cwd (readlink: Permission denied) 2024-04-22T13:26:48.959 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57832 tp_osd_tp ceph rtd unknown /proc/56521/task/57832/root (readlink: Permission denied) 2024-04-22T13:26:48.959 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57832 tp_osd_tp ceph txt unknown /proc/56521/task/57832/exe (readlink: Permission denied) 2024-04-22T13:26:48.959 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57832 tp_osd_tp ceph NOFD /proc/56521/task/57832/fd (opendir: Permission denied) 2024-04-22T13:26:48.959 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57833 tp_osd_tp ceph cwd unknown /proc/56521/task/57833/cwd (readlink: Permission denied) 2024-04-22T13:26:48.959 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57833 tp_osd_tp ceph rtd unknown /proc/56521/task/57833/root (readlink: Permission denied) 2024-04-22T13:26:48.959 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57833 tp_osd_tp ceph txt unknown /proc/56521/task/57833/exe (readlink: Permission denied) 2024-04-22T13:26:48.959 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57833 tp_osd_tp ceph NOFD /proc/56521/task/57833/fd (opendir: Permission denied) 2024-04-22T13:26:48.959 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57834 tp_osd_tp ceph cwd unknown /proc/56521/task/57834/cwd (readlink: Permission denied) 2024-04-22T13:26:48.959 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57834 tp_osd_tp ceph rtd unknown /proc/56521/task/57834/root (readlink: Permission denied) 2024-04-22T13:26:48.959 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57834 tp_osd_tp ceph txt unknown /proc/56521/task/57834/exe (readlink: Permission denied) 2024-04-22T13:26:48.959 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57834 tp_osd_tp ceph NOFD /proc/56521/task/57834/fd (opendir: Permission denied) 2024-04-22T13:26:48.959 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57835 tp_osd_tp ceph cwd unknown /proc/56521/task/57835/cwd (readlink: Permission denied) 2024-04-22T13:26:48.959 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57835 tp_osd_tp ceph rtd unknown /proc/56521/task/57835/root (readlink: Permission denied) 2024-04-22T13:26:48.960 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57835 tp_osd_tp ceph txt unknown /proc/56521/task/57835/exe (readlink: Permission denied) 2024-04-22T13:26:48.960 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57835 tp_osd_tp ceph NOFD /proc/56521/task/57835/fd (opendir: Permission denied) 2024-04-22T13:26:48.960 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57836 tp_osd_tp ceph cwd unknown /proc/56521/task/57836/cwd (readlink: Permission denied) 2024-04-22T13:26:48.960 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57836 tp_osd_tp ceph rtd unknown /proc/56521/task/57836/root (readlink: Permission denied) 2024-04-22T13:26:48.960 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57836 tp_osd_tp ceph txt unknown /proc/56521/task/57836/exe (readlink: Permission denied) 2024-04-22T13:26:48.960 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57836 tp_osd_tp ceph NOFD /proc/56521/task/57836/fd (opendir: Permission denied) 2024-04-22T13:26:48.960 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57837 tp_osd_tp ceph cwd unknown /proc/56521/task/57837/cwd (readlink: Permission denied) 2024-04-22T13:26:48.960 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57837 tp_osd_tp ceph rtd unknown /proc/56521/task/57837/root (readlink: Permission denied) 2024-04-22T13:26:48.960 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57837 tp_osd_tp ceph txt unknown /proc/56521/task/57837/exe (readlink: Permission denied) 2024-04-22T13:26:48.960 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57837 tp_osd_tp ceph NOFD /proc/56521/task/57837/fd (opendir: Permission denied) 2024-04-22T13:26:48.960 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57838 tp_osd_tp ceph cwd unknown /proc/56521/task/57838/cwd (readlink: Permission denied) 2024-04-22T13:26:48.960 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57838 tp_osd_tp ceph rtd unknown /proc/56521/task/57838/root (readlink: Permission denied) 2024-04-22T13:26:48.960 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57838 tp_osd_tp ceph txt unknown /proc/56521/task/57838/exe (readlink: Permission denied) 2024-04-22T13:26:48.960 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57838 tp_osd_tp ceph NOFD /proc/56521/task/57838/fd (opendir: Permission denied) 2024-04-22T13:26:48.961 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57839 tp_osd_tp ceph cwd unknown /proc/56521/task/57839/cwd (readlink: Permission denied) 2024-04-22T13:26:48.961 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57839 tp_osd_tp ceph rtd unknown /proc/56521/task/57839/root (readlink: Permission denied) 2024-04-22T13:26:48.961 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57839 tp_osd_tp ceph txt unknown /proc/56521/task/57839/exe (readlink: Permission denied) 2024-04-22T13:26:48.961 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57839 tp_osd_tp ceph NOFD /proc/56521/task/57839/fd (opendir: Permission denied) 2024-04-22T13:26:48.961 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57840 tp_osd_tp ceph cwd unknown /proc/56521/task/57840/cwd (readlink: Permission denied) 2024-04-22T13:26:48.961 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57840 tp_osd_tp ceph rtd unknown /proc/56521/task/57840/root (readlink: Permission denied) 2024-04-22T13:26:48.961 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57840 tp_osd_tp ceph txt unknown /proc/56521/task/57840/exe (readlink: Permission denied) 2024-04-22T13:26:48.961 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57840 tp_osd_tp ceph NOFD /proc/56521/task/57840/fd (opendir: Permission denied) 2024-04-22T13:26:48.961 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57841 tp_osd_tp ceph cwd unknown /proc/56521/task/57841/cwd (readlink: Permission denied) 2024-04-22T13:26:48.961 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57841 tp_osd_tp ceph rtd unknown /proc/56521/task/57841/root (readlink: Permission denied) 2024-04-22T13:26:48.961 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57841 tp_osd_tp ceph txt unknown /proc/56521/task/57841/exe (readlink: Permission denied) 2024-04-22T13:26:48.961 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57841 tp_osd_tp ceph NOFD /proc/56521/task/57841/fd (opendir: Permission denied) 2024-04-22T13:26:48.961 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57842 tp_osd_tp ceph cwd unknown /proc/56521/task/57842/cwd (readlink: Permission denied) 2024-04-22T13:26:48.961 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57842 tp_osd_tp ceph rtd unknown /proc/56521/task/57842/root (readlink: Permission denied) 2024-04-22T13:26:48.961 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57842 tp_osd_tp ceph txt unknown /proc/56521/task/57842/exe (readlink: Permission denied) 2024-04-22T13:26:48.961 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57842 tp_osd_tp ceph NOFD /proc/56521/task/57842/fd (opendir: Permission denied) 2024-04-22T13:26:48.961 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57843 tp_osd_tp ceph cwd unknown /proc/56521/task/57843/cwd (readlink: Permission denied) 2024-04-22T13:26:48.962 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57843 tp_osd_tp ceph rtd unknown /proc/56521/task/57843/root (readlink: Permission denied) 2024-04-22T13:26:48.962 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57843 tp_osd_tp ceph txt unknown /proc/56521/task/57843/exe (readlink: Permission denied) 2024-04-22T13:26:48.962 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57843 tp_osd_tp ceph NOFD /proc/56521/task/57843/fd (opendir: Permission denied) 2024-04-22T13:26:48.962 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57844 tp_osd_tp ceph cwd unknown /proc/56521/task/57844/cwd (readlink: Permission denied) 2024-04-22T13:26:48.962 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57844 tp_osd_tp ceph rtd unknown /proc/56521/task/57844/root (readlink: Permission denied) 2024-04-22T13:26:48.962 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57844 tp_osd_tp ceph txt unknown /proc/56521/task/57844/exe (readlink: Permission denied) 2024-04-22T13:26:48.962 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57844 tp_osd_tp ceph NOFD /proc/56521/task/57844/fd (opendir: Permission denied) 2024-04-22T13:26:48.962 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57845 tp_osd_tp ceph cwd unknown /proc/56521/task/57845/cwd (readlink: Permission denied) 2024-04-22T13:26:48.962 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57845 tp_osd_tp ceph rtd unknown /proc/56521/task/57845/root (readlink: Permission denied) 2024-04-22T13:26:48.962 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57845 tp_osd_tp ceph txt unknown /proc/56521/task/57845/exe (readlink: Permission denied) 2024-04-22T13:26:48.962 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57845 tp_osd_tp ceph NOFD /proc/56521/task/57845/fd (opendir: Permission denied) 2024-04-22T13:26:48.962 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57846 tp_osd_tp ceph cwd unknown /proc/56521/task/57846/cwd (readlink: Permission denied) 2024-04-22T13:26:48.962 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57846 tp_osd_tp ceph rtd unknown /proc/56521/task/57846/root (readlink: Permission denied) 2024-04-22T13:26:48.962 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57846 tp_osd_tp ceph txt unknown /proc/56521/task/57846/exe (readlink: Permission denied) 2024-04-22T13:26:48.962 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57846 tp_osd_tp ceph NOFD /proc/56521/task/57846/fd (opendir: Permission denied) 2024-04-22T13:26:48.963 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57847 tp_osd_tp ceph cwd unknown /proc/56521/task/57847/cwd (readlink: Permission denied) 2024-04-22T13:26:48.963 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57847 tp_osd_tp ceph rtd unknown /proc/56521/task/57847/root (readlink: Permission denied) 2024-04-22T13:26:48.963 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57847 tp_osd_tp ceph txt unknown /proc/56521/task/57847/exe (readlink: Permission denied) 2024-04-22T13:26:48.963 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57847 tp_osd_tp ceph NOFD /proc/56521/task/57847/fd (opendir: Permission denied) 2024-04-22T13:26:48.963 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57848 osd_srv_h ceph cwd unknown /proc/56521/task/57848/cwd (readlink: Permission denied) 2024-04-22T13:26:48.963 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57848 osd_srv_h ceph rtd unknown /proc/56521/task/57848/root (readlink: Permission denied) 2024-04-22T13:26:48.963 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57848 osd_srv_h ceph txt unknown /proc/56521/task/57848/exe (readlink: Permission denied) 2024-04-22T13:26:48.963 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 56521 57848 osd_srv_h ceph NOFD /proc/56521/task/57848/fd (opendir: Permission denied) 2024-04-22T13:26:48.963 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 57942 root cwd unknown /proc/57942/cwd (readlink: Permission denied) 2024-04-22T13:26:48.963 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 57942 root rtd unknown /proc/57942/root (readlink: Permission denied) 2024-04-22T13:26:48.963 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 57942 root txt unknown /proc/57942/exe (readlink: Permission denied) 2024-04-22T13:26:48.963 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 57942 root NOFD /proc/57942/fd (opendir: Permission denied) 2024-04-22T13:26:48.963 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/3 57945 root cwd unknown /proc/57945/cwd (readlink: Permission denied) 2024-04-22T13:26:48.963 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/3 57945 root rtd unknown /proc/57945/root (readlink: Permission denied) 2024-04-22T13:26:48.963 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/3 57945 root txt unknown /proc/57945/exe (readlink: Permission denied) 2024-04-22T13:26:48.964 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/3 57945 root NOFD /proc/57945/fd (opendir: Permission denied) 2024-04-22T13:26:48.964 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 57960 root cwd unknown /proc/57960/cwd (readlink: Permission denied) 2024-04-22T13:26:48.964 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 57960 root rtd unknown /proc/57960/root (readlink: Permission denied) 2024-04-22T13:26:48.964 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 57960 root txt unknown /proc/57960/exe (readlink: Permission denied) 2024-04-22T13:26:48.964 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 57960 root NOFD /proc/57960/fd (opendir: Permission denied) 2024-04-22T13:26:48.964 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/3 61083 root cwd unknown /proc/61083/cwd (readlink: Permission denied) 2024-04-22T13:26:48.964 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/3 61083 root rtd unknown /proc/61083/root (readlink: Permission denied) 2024-04-22T13:26:48.964 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/3 61083 root txt unknown /proc/61083/exe (readlink: Permission denied) 2024-04-22T13:26:48.964 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/3 61083 root NOFD /proc/61083/fd (opendir: Permission denied) 2024-04-22T13:26:48.964 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/4 62654 root cwd unknown /proc/62654/cwd (readlink: Permission denied) 2024-04-22T13:26:48.964 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/4 62654 root rtd unknown /proc/62654/root (readlink: Permission denied) 2024-04-22T13:26:48.964 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/4 62654 root txt unknown /proc/62654/exe (readlink: Permission denied) 2024-04-22T13:26:48.964 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/4 62654 root NOFD /proc/62654/fd (opendir: Permission denied) 2024-04-22T13:26:48.964 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 63392 root cwd unknown /proc/63392/cwd (readlink: Permission denied) 2024-04-22T13:26:48.965 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 63392 root rtd unknown /proc/63392/root (readlink: Permission denied) 2024-04-22T13:26:48.965 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 63392 root txt unknown /proc/63392/exe (readlink: Permission denied) 2024-04-22T13:26:48.965 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 63392 root NOFD /proc/63392/fd (opendir: Permission denied) 2024-04-22T13:26:48.965 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 63394 root cwd unknown /proc/63394/cwd (readlink: Permission denied) 2024-04-22T13:26:48.965 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 63394 root rtd unknown /proc/63394/root (readlink: Permission denied) 2024-04-22T13:26:48.965 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 63394 root txt unknown /proc/63394/exe (readlink: Permission denied) 2024-04-22T13:26:48.965 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 63394 root NOFD /proc/63394/fd (opendir: Permission denied) 2024-04-22T13:26:48.965 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 ceph cwd unknown /proc/63396/cwd (readlink: Permission denied) 2024-04-22T13:26:48.965 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 ceph rtd unknown /proc/63396/root (readlink: Permission denied) 2024-04-22T13:26:48.965 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 ceph txt unknown /proc/63396/exe (readlink: Permission denied) 2024-04-22T13:26:48.965 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 ceph NOFD /proc/63396/fd (opendir: Permission denied) 2024-04-22T13:26:48.965 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63397 log ceph cwd unknown /proc/63396/task/63397/cwd (readlink: Permission denied) 2024-04-22T13:26:48.965 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63397 log ceph rtd unknown /proc/63396/task/63397/root (readlink: Permission denied) 2024-04-22T13:26:48.965 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63397 log ceph txt unknown /proc/63396/task/63397/exe (readlink: Permission denied) 2024-04-22T13:26:48.966 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63397 log ceph NOFD /proc/63396/task/63397/fd (opendir: Permission denied) 2024-04-22T13:26:48.966 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63399 msgr-work ceph cwd unknown /proc/63396/task/63399/cwd (readlink: Permission denied) 2024-04-22T13:26:48.966 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63399 msgr-work ceph rtd unknown /proc/63396/task/63399/root (readlink: Permission denied) 2024-04-22T13:26:48.966 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63399 msgr-work ceph txt unknown /proc/63396/task/63399/exe (readlink: Permission denied) 2024-04-22T13:26:48.966 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63399 msgr-work ceph NOFD /proc/63396/task/63399/fd (opendir: Permission denied) 2024-04-22T13:26:48.966 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63400 msgr-work ceph cwd unknown /proc/63396/task/63400/cwd (readlink: Permission denied) 2024-04-22T13:26:48.966 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63400 msgr-work ceph rtd unknown /proc/63396/task/63400/root (readlink: Permission denied) 2024-04-22T13:26:48.966 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63400 msgr-work ceph txt unknown /proc/63396/task/63400/exe (readlink: Permission denied) 2024-04-22T13:26:48.966 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63400 msgr-work ceph NOFD /proc/63396/task/63400/fd (opendir: Permission denied) 2024-04-22T13:26:48.966 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63401 msgr-work ceph cwd unknown /proc/63396/task/63401/cwd (readlink: Permission denied) 2024-04-22T13:26:48.966 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63401 msgr-work ceph rtd unknown /proc/63396/task/63401/root (readlink: Permission denied) 2024-04-22T13:26:48.966 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63401 msgr-work ceph txt unknown /proc/63396/task/63401/exe (readlink: Permission denied) 2024-04-22T13:26:48.966 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63401 msgr-work ceph NOFD /proc/63396/task/63401/fd (opendir: Permission denied) 2024-04-22T13:26:48.966 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63405 service ceph cwd unknown /proc/63396/task/63405/cwd (readlink: Permission denied) 2024-04-22T13:26:48.966 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63405 service ceph rtd unknown /proc/63396/task/63405/root (readlink: Permission denied) 2024-04-22T13:26:48.967 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63405 service ceph txt unknown /proc/63396/task/63405/exe (readlink: Permission denied) 2024-04-22T13:26:48.967 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63405 service ceph NOFD /proc/63396/task/63405/fd (opendir: Permission denied) 2024-04-22T13:26:48.967 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63406 admin_soc ceph cwd unknown /proc/63396/task/63406/cwd (readlink: Permission denied) 2024-04-22T13:26:48.967 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63406 admin_soc ceph rtd unknown /proc/63396/task/63406/root (readlink: Permission denied) 2024-04-22T13:26:48.967 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63406 admin_soc ceph txt unknown /proc/63396/task/63406/exe (readlink: Permission denied) 2024-04-22T13:26:48.967 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63406 admin_soc ceph NOFD /proc/63396/task/63406/fd (opendir: Permission denied) 2024-04-22T13:26:48.967 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63548 signal_ha ceph cwd unknown /proc/63396/task/63548/cwd (readlink: Permission denied) 2024-04-22T13:26:48.967 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63548 signal_ha ceph rtd unknown /proc/63396/task/63548/root (readlink: Permission denied) 2024-04-22T13:26:48.967 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63548 signal_ha ceph txt unknown /proc/63396/task/63548/exe (readlink: Permission denied) 2024-04-22T13:26:48.967 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63548 signal_ha ceph NOFD /proc/63396/task/63548/fd (opendir: Permission denied) 2024-04-22T13:26:48.967 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63549 io_contex ceph cwd unknown /proc/63396/task/63549/cwd (readlink: Permission denied) 2024-04-22T13:26:48.967 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63549 io_contex ceph rtd unknown /proc/63396/task/63549/root (readlink: Permission denied) 2024-04-22T13:26:48.967 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63549 io_contex ceph txt unknown /proc/63396/task/63549/exe (readlink: Permission denied) 2024-04-22T13:26:48.967 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63549 io_contex ceph NOFD /proc/63396/task/63549/fd (opendir: Permission denied) 2024-04-22T13:26:48.967 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63550 io_contex ceph cwd unknown /proc/63396/task/63550/cwd (readlink: Permission denied) 2024-04-22T13:26:48.968 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63550 io_contex ceph rtd unknown /proc/63396/task/63550/root (readlink: Permission denied) 2024-04-22T13:26:48.968 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63550 io_contex ceph txt unknown /proc/63396/task/63550/exe (readlink: Permission denied) 2024-04-22T13:26:48.968 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63550 io_contex ceph NOFD /proc/63396/task/63550/fd (opendir: Permission denied) 2024-04-22T13:26:48.968 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63662 OpHistory ceph cwd unknown /proc/63396/task/63662/cwd (readlink: Permission denied) 2024-04-22T13:26:48.968 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63662 OpHistory ceph rtd unknown /proc/63396/task/63662/root (readlink: Permission denied) 2024-04-22T13:26:48.968 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63662 OpHistory ceph txt unknown /proc/63396/task/63662/exe (readlink: Permission denied) 2024-04-22T13:26:48.968 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63662 OpHistory ceph NOFD /proc/63396/task/63662/fd (opendir: Permission denied) 2024-04-22T13:26:48.968 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63663 ceph_time ceph cwd unknown /proc/63396/task/63663/cwd (readlink: Permission denied) 2024-04-22T13:26:48.968 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63663 ceph_time ceph rtd unknown /proc/63396/task/63663/root (readlink: Permission denied) 2024-04-22T13:26:48.968 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63663 ceph_time ceph txt unknown /proc/63396/task/63663/exe (readlink: Permission denied) 2024-04-22T13:26:48.968 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63663 ceph_time ceph NOFD /proc/63396/task/63663/fd (opendir: Permission denied) 2024-04-22T13:26:48.968 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63813 ceph-osd ceph cwd unknown /proc/63396/task/63813/cwd (readlink: Permission denied) 2024-04-22T13:26:48.968 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63813 ceph-osd ceph rtd unknown /proc/63396/task/63813/root (readlink: Permission denied) 2024-04-22T13:26:48.968 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63813 ceph-osd ceph txt unknown /proc/63396/task/63813/exe (readlink: Permission denied) 2024-04-22T13:26:48.968 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63813 ceph-osd ceph NOFD /proc/63396/task/63813/fd (opendir: Permission denied) 2024-04-22T13:26:48.968 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63814 safe_time ceph cwd unknown /proc/63396/task/63814/cwd (readlink: Permission denied) 2024-04-22T13:26:48.969 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63814 safe_time ceph rtd unknown /proc/63396/task/63814/root (readlink: Permission denied) 2024-04-22T13:26:48.969 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63814 safe_time ceph txt unknown /proc/63396/task/63814/exe (readlink: Permission denied) 2024-04-22T13:26:48.969 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63814 safe_time ceph NOFD /proc/63396/task/63814/fd (opendir: Permission denied) 2024-04-22T13:26:48.969 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63815 safe_time ceph cwd unknown /proc/63396/task/63815/cwd (readlink: Permission denied) 2024-04-22T13:26:48.969 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63815 safe_time ceph rtd unknown /proc/63396/task/63815/root (readlink: Permission denied) 2024-04-22T13:26:48.969 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63815 safe_time ceph txt unknown /proc/63396/task/63815/exe (readlink: Permission denied) 2024-04-22T13:26:48.969 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63815 safe_time ceph NOFD /proc/63396/task/63815/fd (opendir: Permission denied) 2024-04-22T13:26:48.969 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63816 safe_time ceph cwd unknown /proc/63396/task/63816/cwd (readlink: Permission denied) 2024-04-22T13:26:48.969 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63816 safe_time ceph rtd unknown /proc/63396/task/63816/root (readlink: Permission denied) 2024-04-22T13:26:48.969 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63816 safe_time ceph txt unknown /proc/63396/task/63816/exe (readlink: Permission denied) 2024-04-22T13:26:48.969 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63816 safe_time ceph NOFD /proc/63396/task/63816/fd (opendir: Permission denied) 2024-04-22T13:26:48.969 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63817 safe_time ceph cwd unknown /proc/63396/task/63817/cwd (readlink: Permission denied) 2024-04-22T13:26:48.969 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63817 safe_time ceph rtd unknown /proc/63396/task/63817/root (readlink: Permission denied) 2024-04-22T13:26:48.969 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63817 safe_time ceph txt unknown /proc/63396/task/63817/exe (readlink: Permission denied) 2024-04-22T13:26:48.969 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63817 safe_time ceph NOFD /proc/63396/task/63817/fd (opendir: Permission denied) 2024-04-22T13:26:48.970 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63818 fn_anonym ceph cwd unknown /proc/63396/task/63818/cwd (readlink: Permission denied) 2024-04-22T13:26:48.970 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63818 fn_anonym ceph rtd unknown /proc/63396/task/63818/root (readlink: Permission denied) 2024-04-22T13:26:48.970 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63818 fn_anonym ceph txt unknown /proc/63396/task/63818/exe (readlink: Permission denied) 2024-04-22T13:26:48.970 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63818 fn_anonym ceph NOFD /proc/63396/task/63818/fd (opendir: Permission denied) 2024-04-22T13:26:48.970 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63826 rocksdb:l ceph cwd unknown /proc/63396/task/63826/cwd (readlink: Permission denied) 2024-04-22T13:26:48.970 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63826 rocksdb:l ceph rtd unknown /proc/63396/task/63826/root (readlink: Permission denied) 2024-04-22T13:26:48.970 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63826 rocksdb:l ceph txt unknown /proc/63396/task/63826/exe (readlink: Permission denied) 2024-04-22T13:26:48.970 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63826 rocksdb:l ceph NOFD /proc/63396/task/63826/fd (opendir: Permission denied) 2024-04-22T13:26:48.970 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63827 rocksdb:l ceph cwd unknown /proc/63396/task/63827/cwd (readlink: Permission denied) 2024-04-22T13:26:48.970 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63827 rocksdb:l ceph rtd unknown /proc/63396/task/63827/root (readlink: Permission denied) 2024-04-22T13:26:48.970 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63827 rocksdb:l ceph txt unknown /proc/63396/task/63827/exe (readlink: Permission denied) 2024-04-22T13:26:48.970 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63827 rocksdb:l ceph NOFD /proc/63396/task/63827/fd (opendir: Permission denied) 2024-04-22T13:26:48.970 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63828 rocksdb:l ceph cwd unknown /proc/63396/task/63828/cwd (readlink: Permission denied) 2024-04-22T13:26:48.971 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63828 rocksdb:l ceph rtd unknown /proc/63396/task/63828/root (readlink: Permission denied) 2024-04-22T13:26:48.971 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63828 rocksdb:l ceph txt unknown /proc/63396/task/63828/exe (readlink: Permission denied) 2024-04-22T13:26:48.971 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63828 rocksdb:l ceph NOFD /proc/63396/task/63828/fd (opendir: Permission denied) 2024-04-22T13:26:48.971 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63829 rocksdb:h ceph cwd unknown /proc/63396/task/63829/cwd (readlink: Permission denied) 2024-04-22T13:26:48.971 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63829 rocksdb:h ceph rtd unknown /proc/63396/task/63829/root (readlink: Permission denied) 2024-04-22T13:26:48.971 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63829 rocksdb:h ceph txt unknown /proc/63396/task/63829/exe (readlink: Permission denied) 2024-04-22T13:26:48.971 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 63829 rocksdb:h ceph NOFD /proc/63396/task/63829/fd (opendir: Permission denied) 2024-04-22T13:26:48.971 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64248 bstore_ai ceph cwd unknown /proc/63396/task/64248/cwd (readlink: Permission denied) 2024-04-22T13:26:48.971 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64248 bstore_ai ceph rtd unknown /proc/63396/task/64248/root (readlink: Permission denied) 2024-04-22T13:26:48.971 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64248 bstore_ai ceph txt unknown /proc/63396/task/64248/exe (readlink: Permission denied) 2024-04-22T13:26:48.971 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64248 bstore_ai ceph NOFD /proc/63396/task/64248/fd (opendir: Permission denied) 2024-04-22T13:26:48.971 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64442 bstore_ai ceph cwd unknown /proc/63396/task/64442/cwd (readlink: Permission denied) 2024-04-22T13:26:48.971 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64442 bstore_ai ceph rtd unknown /proc/63396/task/64442/root (readlink: Permission denied) 2024-04-22T13:26:48.971 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64442 bstore_ai ceph txt unknown /proc/63396/task/64442/exe (readlink: Permission denied) 2024-04-22T13:26:48.971 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64442 bstore_ai ceph NOFD /proc/63396/task/64442/fd (opendir: Permission denied) 2024-04-22T13:26:48.972 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64625 ceph-osd ceph cwd unknown /proc/63396/task/64625/cwd (readlink: Permission denied) 2024-04-22T13:26:48.972 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64625 ceph-osd ceph rtd unknown /proc/63396/task/64625/root (readlink: Permission denied) 2024-04-22T13:26:48.972 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64625 ceph-osd ceph txt unknown /proc/63396/task/64625/exe (readlink: Permission denied) 2024-04-22T13:26:48.972 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64625 ceph-osd ceph NOFD /proc/63396/task/64625/fd (opendir: Permission denied) 2024-04-22T13:26:48.972 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64626 cfin ceph cwd unknown /proc/63396/task/64626/cwd (readlink: Permission denied) 2024-04-22T13:26:48.972 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64626 cfin ceph rtd unknown /proc/63396/task/64626/root (readlink: Permission denied) 2024-04-22T13:26:48.972 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64626 cfin ceph txt unknown /proc/63396/task/64626/exe (readlink: Permission denied) 2024-04-22T13:26:48.972 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64626 cfin ceph NOFD /proc/63396/task/64626/fd (opendir: Permission denied) 2024-04-22T13:26:48.972 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64627 bstore_kv ceph cwd unknown /proc/63396/task/64627/cwd (readlink: Permission denied) 2024-04-22T13:26:48.972 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64627 bstore_kv ceph rtd unknown /proc/63396/task/64627/root (readlink: Permission denied) 2024-04-22T13:26:48.972 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64627 bstore_kv ceph txt unknown /proc/63396/task/64627/exe (readlink: Permission denied) 2024-04-22T13:26:48.972 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64627 bstore_kv ceph NOFD /proc/63396/task/64627/fd (opendir: Permission denied) 2024-04-22T13:26:48.972 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64628 bstore_kv ceph cwd unknown /proc/63396/task/64628/cwd (readlink: Permission denied) 2024-04-22T13:26:48.972 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64628 bstore_kv ceph rtd unknown /proc/63396/task/64628/root (readlink: Permission denied) 2024-04-22T13:26:48.972 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64628 bstore_kv ceph txt unknown /proc/63396/task/64628/exe (readlink: Permission denied) 2024-04-22T13:26:48.972 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64628 bstore_kv ceph NOFD /proc/63396/task/64628/fd (opendir: Permission denied) 2024-04-22T13:26:48.973 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64629 bstore_me ceph cwd unknown /proc/63396/task/64629/cwd (readlink: Permission denied) 2024-04-22T13:26:48.973 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64629 bstore_me ceph rtd unknown /proc/63396/task/64629/root (readlink: Permission denied) 2024-04-22T13:26:48.973 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64629 bstore_me ceph txt unknown /proc/63396/task/64629/exe (readlink: Permission denied) 2024-04-22T13:26:48.973 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64629 bstore_me ceph NOFD /proc/63396/task/64629/fd (opendir: Permission denied) 2024-04-22T13:26:48.973 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64630 ms_dispat ceph cwd unknown /proc/63396/task/64630/cwd (readlink: Permission denied) 2024-04-22T13:26:48.973 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64630 ms_dispat ceph rtd unknown /proc/63396/task/64630/root (readlink: Permission denied) 2024-04-22T13:26:48.973 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64630 ms_dispat ceph txt unknown /proc/63396/task/64630/exe (readlink: Permission denied) 2024-04-22T13:26:48.973 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64630 ms_dispat ceph NOFD /proc/63396/task/64630/fd (opendir: Permission denied) 2024-04-22T13:26:48.973 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64631 ms_local ceph cwd unknown /proc/63396/task/64631/cwd (readlink: Permission denied) 2024-04-22T13:26:48.973 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64631 ms_local ceph rtd unknown /proc/63396/task/64631/root (readlink: Permission denied) 2024-04-22T13:26:48.973 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64631 ms_local ceph txt unknown /proc/63396/task/64631/exe (readlink: Permission denied) 2024-04-22T13:26:48.973 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64631 ms_local ceph NOFD /proc/63396/task/64631/fd (opendir: Permission denied) 2024-04-22T13:26:48.973 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64632 safe_time ceph cwd unknown /proc/63396/task/64632/cwd (readlink: Permission denied) 2024-04-22T13:26:48.973 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64632 safe_time ceph rtd unknown /proc/63396/task/64632/root (readlink: Permission denied) 2024-04-22T13:26:48.974 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64632 safe_time ceph txt unknown /proc/63396/task/64632/exe (readlink: Permission denied) 2024-04-22T13:26:48.974 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64632 safe_time ceph NOFD /proc/63396/task/64632/fd (opendir: Permission denied) 2024-04-22T13:26:48.974 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64633 safe_time ceph cwd unknown /proc/63396/task/64633/cwd (readlink: Permission denied) 2024-04-22T13:26:48.974 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64633 safe_time ceph rtd unknown /proc/63396/task/64633/root (readlink: Permission denied) 2024-04-22T13:26:48.974 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64633 safe_time ceph txt unknown /proc/63396/task/64633/exe (readlink: Permission denied) 2024-04-22T13:26:48.974 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64633 safe_time ceph NOFD /proc/63396/task/64633/fd (opendir: Permission denied) 2024-04-22T13:26:48.974 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64634 ms_dispat ceph cwd unknown /proc/63396/task/64634/cwd (readlink: Permission denied) 2024-04-22T13:26:48.974 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64634 ms_dispat ceph rtd unknown /proc/63396/task/64634/root (readlink: Permission denied) 2024-04-22T13:26:48.974 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64634 ms_dispat ceph txt unknown /proc/63396/task/64634/exe (readlink: Permission denied) 2024-04-22T13:26:48.974 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64634 ms_dispat ceph NOFD /proc/63396/task/64634/fd (opendir: Permission denied) 2024-04-22T13:26:48.974 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64635 ms_local ceph cwd unknown /proc/63396/task/64635/cwd (readlink: Permission denied) 2024-04-22T13:26:48.974 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64635 ms_local ceph rtd unknown /proc/63396/task/64635/root (readlink: Permission denied) 2024-04-22T13:26:48.974 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64635 ms_local ceph txt unknown /proc/63396/task/64635/exe (readlink: Permission denied) 2024-04-22T13:26:48.974 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64635 ms_local ceph NOFD /proc/63396/task/64635/fd (opendir: Permission denied) 2024-04-22T13:26:48.974 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64636 ms_dispat ceph cwd unknown /proc/63396/task/64636/cwd (readlink: Permission denied) 2024-04-22T13:26:48.975 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64636 ms_dispat ceph rtd unknown /proc/63396/task/64636/root (readlink: Permission denied) 2024-04-22T13:26:48.975 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64636 ms_dispat ceph txt unknown /proc/63396/task/64636/exe (readlink: Permission denied) 2024-04-22T13:26:48.975 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64636 ms_dispat ceph NOFD /proc/63396/task/64636/fd (opendir: Permission denied) 2024-04-22T13:26:48.975 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64637 ms_local ceph cwd unknown /proc/63396/task/64637/cwd (readlink: Permission denied) 2024-04-22T13:26:48.975 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64637 ms_local ceph rtd unknown /proc/63396/task/64637/root (readlink: Permission denied) 2024-04-22T13:26:48.975 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64637 ms_local ceph txt unknown /proc/63396/task/64637/exe (readlink: Permission denied) 2024-04-22T13:26:48.975 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64637 ms_local ceph NOFD /proc/63396/task/64637/fd (opendir: Permission denied) 2024-04-22T13:26:48.975 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64638 ms_dispat ceph cwd unknown /proc/63396/task/64638/cwd (readlink: Permission denied) 2024-04-22T13:26:48.975 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64638 ms_dispat ceph rtd unknown /proc/63396/task/64638/root (readlink: Permission denied) 2024-04-22T13:26:48.975 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64638 ms_dispat ceph txt unknown /proc/63396/task/64638/exe (readlink: Permission denied) 2024-04-22T13:26:48.975 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64638 ms_dispat ceph NOFD /proc/63396/task/64638/fd (opendir: Permission denied) 2024-04-22T13:26:48.975 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64639 ms_local ceph cwd unknown /proc/63396/task/64639/cwd (readlink: Permission denied) 2024-04-22T13:26:48.975 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64639 ms_local ceph rtd unknown /proc/63396/task/64639/root (readlink: Permission denied) 2024-04-22T13:26:48.975 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64639 ms_local ceph txt unknown /proc/63396/task/64639/exe (readlink: Permission denied) 2024-04-22T13:26:48.975 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64639 ms_local ceph NOFD /proc/63396/task/64639/fd (opendir: Permission denied) 2024-04-22T13:26:48.976 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64640 ms_dispat ceph cwd unknown /proc/63396/task/64640/cwd (readlink: Permission denied) 2024-04-22T13:26:48.976 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64640 ms_dispat ceph rtd unknown /proc/63396/task/64640/root (readlink: Permission denied) 2024-04-22T13:26:48.976 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64640 ms_dispat ceph txt unknown /proc/63396/task/64640/exe (readlink: Permission denied) 2024-04-22T13:26:48.976 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64640 ms_dispat ceph NOFD /proc/63396/task/64640/fd (opendir: Permission denied) 2024-04-22T13:26:48.976 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64641 ms_local ceph cwd unknown /proc/63396/task/64641/cwd (readlink: Permission denied) 2024-04-22T13:26:48.976 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64641 ms_local ceph rtd unknown /proc/63396/task/64641/root (readlink: Permission denied) 2024-04-22T13:26:48.976 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64641 ms_local ceph txt unknown /proc/63396/task/64641/exe (readlink: Permission denied) 2024-04-22T13:26:48.976 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64641 ms_local ceph NOFD /proc/63396/task/64641/fd (opendir: Permission denied) 2024-04-22T13:26:48.976 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64642 ms_dispat ceph cwd unknown /proc/63396/task/64642/cwd (readlink: Permission denied) 2024-04-22T13:26:48.976 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64642 ms_dispat ceph rtd unknown /proc/63396/task/64642/root (readlink: Permission denied) 2024-04-22T13:26:48.976 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64642 ms_dispat ceph txt unknown /proc/63396/task/64642/exe (readlink: Permission denied) 2024-04-22T13:26:48.976 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64642 ms_dispat ceph NOFD /proc/63396/task/64642/fd (opendir: Permission denied) 2024-04-22T13:26:48.976 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64643 ms_local ceph cwd unknown /proc/63396/task/64643/cwd (readlink: Permission denied) 2024-04-22T13:26:48.976 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64643 ms_local ceph rtd unknown /proc/63396/task/64643/root (readlink: Permission denied) 2024-04-22T13:26:48.976 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64643 ms_local ceph txt unknown /proc/63396/task/64643/exe (readlink: Permission denied) 2024-04-22T13:26:48.976 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64643 ms_local ceph NOFD /proc/63396/task/64643/fd (opendir: Permission denied) 2024-04-22T13:26:48.976 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64644 ms_dispat ceph cwd unknown /proc/63396/task/64644/cwd (readlink: Permission denied) 2024-04-22T13:26:48.977 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64644 ms_dispat ceph rtd unknown /proc/63396/task/64644/root (readlink: Permission denied) 2024-04-22T13:26:48.977 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64644 ms_dispat ceph txt unknown /proc/63396/task/64644/exe (readlink: Permission denied) 2024-04-22T13:26:48.977 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64644 ms_dispat ceph NOFD /proc/63396/task/64644/fd (opendir: Permission denied) 2024-04-22T13:26:48.977 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64645 ms_local ceph cwd unknown /proc/63396/task/64645/cwd (readlink: Permission denied) 2024-04-22T13:26:48.977 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64645 ms_local ceph rtd unknown /proc/63396/task/64645/root (readlink: Permission denied) 2024-04-22T13:26:48.977 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64645 ms_local ceph txt unknown /proc/63396/task/64645/exe (readlink: Permission denied) 2024-04-22T13:26:48.977 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64645 ms_local ceph NOFD /proc/63396/task/64645/fd (opendir: Permission denied) 2024-04-22T13:26:48.977 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64646 fn_anonym ceph cwd unknown /proc/63396/task/64646/cwd (readlink: Permission denied) 2024-04-22T13:26:48.977 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64646 fn_anonym ceph rtd unknown /proc/63396/task/64646/root (readlink: Permission denied) 2024-04-22T13:26:48.977 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64646 fn_anonym ceph txt unknown /proc/63396/task/64646/exe (readlink: Permission denied) 2024-04-22T13:26:48.977 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64646 fn_anonym ceph NOFD /proc/63396/task/64646/fd (opendir: Permission denied) 2024-04-22T13:26:48.977 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64647 finisher ceph cwd unknown /proc/63396/task/64647/cwd (readlink: Permission denied) 2024-04-22T13:26:48.977 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64647 finisher ceph rtd unknown /proc/63396/task/64647/root (readlink: Permission denied) 2024-04-22T13:26:48.977 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64647 finisher ceph txt unknown /proc/63396/task/64647/exe (readlink: Permission denied) 2024-04-22T13:26:48.978 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64647 finisher ceph NOFD /proc/63396/task/64647/fd (opendir: Permission denied) 2024-04-22T13:26:48.978 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64648 safe_time ceph cwd unknown /proc/63396/task/64648/cwd (readlink: Permission denied) 2024-04-22T13:26:48.978 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64648 safe_time ceph rtd unknown /proc/63396/task/64648/root (readlink: Permission denied) 2024-04-22T13:26:48.978 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64648 safe_time ceph txt unknown /proc/63396/task/64648/exe (readlink: Permission denied) 2024-04-22T13:26:48.978 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64648 safe_time ceph NOFD /proc/63396/task/64648/fd (opendir: Permission denied) 2024-04-22T13:26:48.978 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64649 safe_time ceph cwd unknown /proc/63396/task/64649/cwd (readlink: Permission denied) 2024-04-22T13:26:48.978 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64649 safe_time ceph rtd unknown /proc/63396/task/64649/root (readlink: Permission denied) 2024-04-22T13:26:48.978 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64649 safe_time ceph txt unknown /proc/63396/task/64649/exe (readlink: Permission denied) 2024-04-22T13:26:48.978 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64649 safe_time ceph NOFD /proc/63396/task/64649/fd (opendir: Permission denied) 2024-04-22T13:26:48.978 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64650 ceph-osd ceph cwd unknown /proc/63396/task/64650/cwd (readlink: Permission denied) 2024-04-22T13:26:48.978 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64650 ceph-osd ceph rtd unknown /proc/63396/task/64650/root (readlink: Permission denied) 2024-04-22T13:26:48.978 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64650 ceph-osd ceph txt unknown /proc/63396/task/64650/exe (readlink: Permission denied) 2024-04-22T13:26:48.978 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64650 ceph-osd ceph NOFD /proc/63396/task/64650/fd (opendir: Permission denied) 2024-04-22T13:26:48.978 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64651 osd_srv_a ceph cwd unknown /proc/63396/task/64651/cwd (readlink: Permission denied) 2024-04-22T13:26:48.978 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64651 osd_srv_a ceph rtd unknown /proc/63396/task/64651/root (readlink: Permission denied) 2024-04-22T13:26:48.978 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64651 osd_srv_a ceph txt unknown /proc/63396/task/64651/exe (readlink: Permission denied) 2024-04-22T13:26:48.978 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64651 osd_srv_a ceph NOFD /proc/63396/task/64651/fd (opendir: Permission denied) 2024-04-22T13:26:48.979 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64652 tp_osd_tp ceph cwd unknown /proc/63396/task/64652/cwd (readlink: Permission denied) 2024-04-22T13:26:48.979 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64652 tp_osd_tp ceph rtd unknown /proc/63396/task/64652/root (readlink: Permission denied) 2024-04-22T13:26:48.979 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64652 tp_osd_tp ceph txt unknown /proc/63396/task/64652/exe (readlink: Permission denied) 2024-04-22T13:26:48.979 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64652 tp_osd_tp ceph NOFD /proc/63396/task/64652/fd (opendir: Permission denied) 2024-04-22T13:26:48.979 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64653 tp_osd_tp ceph cwd unknown /proc/63396/task/64653/cwd (readlink: Permission denied) 2024-04-22T13:26:48.979 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64653 tp_osd_tp ceph rtd unknown /proc/63396/task/64653/root (readlink: Permission denied) 2024-04-22T13:26:48.979 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64653 tp_osd_tp ceph txt unknown /proc/63396/task/64653/exe (readlink: Permission denied) 2024-04-22T13:26:48.979 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64653 tp_osd_tp ceph NOFD /proc/63396/task/64653/fd (opendir: Permission denied) 2024-04-22T13:26:48.979 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64654 tp_osd_tp ceph cwd unknown /proc/63396/task/64654/cwd (readlink: Permission denied) 2024-04-22T13:26:48.979 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64654 tp_osd_tp ceph rtd unknown /proc/63396/task/64654/root (readlink: Permission denied) 2024-04-22T13:26:48.979 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64654 tp_osd_tp ceph txt unknown /proc/63396/task/64654/exe (readlink: Permission denied) 2024-04-22T13:26:48.979 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64654 tp_osd_tp ceph NOFD /proc/63396/task/64654/fd (opendir: Permission denied) 2024-04-22T13:26:48.979 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64655 tp_osd_tp ceph cwd unknown /proc/63396/task/64655/cwd (readlink: Permission denied) 2024-04-22T13:26:48.979 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64655 tp_osd_tp ceph rtd unknown /proc/63396/task/64655/root (readlink: Permission denied) 2024-04-22T13:26:48.979 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64655 tp_osd_tp ceph txt unknown /proc/63396/task/64655/exe (readlink: Permission denied) 2024-04-22T13:26:48.979 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64655 tp_osd_tp ceph NOFD /proc/63396/task/64655/fd (opendir: Permission denied) 2024-04-22T13:26:48.979 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64656 tp_osd_tp ceph cwd unknown /proc/63396/task/64656/cwd (readlink: Permission denied) 2024-04-22T13:26:48.980 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64656 tp_osd_tp ceph rtd unknown /proc/63396/task/64656/root (readlink: Permission denied) 2024-04-22T13:26:48.980 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64656 tp_osd_tp ceph txt unknown /proc/63396/task/64656/exe (readlink: Permission denied) 2024-04-22T13:26:48.980 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64656 tp_osd_tp ceph NOFD /proc/63396/task/64656/fd (opendir: Permission denied) 2024-04-22T13:26:48.980 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64657 tp_osd_tp ceph cwd unknown /proc/63396/task/64657/cwd (readlink: Permission denied) 2024-04-22T13:26:48.980 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64657 tp_osd_tp ceph rtd unknown /proc/63396/task/64657/root (readlink: Permission denied) 2024-04-22T13:26:48.980 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64657 tp_osd_tp ceph txt unknown /proc/63396/task/64657/exe (readlink: Permission denied) 2024-04-22T13:26:48.980 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64657 tp_osd_tp ceph NOFD /proc/63396/task/64657/fd (opendir: Permission denied) 2024-04-22T13:26:48.980 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64658 tp_osd_tp ceph cwd unknown /proc/63396/task/64658/cwd (readlink: Permission denied) 2024-04-22T13:26:48.980 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64658 tp_osd_tp ceph rtd unknown /proc/63396/task/64658/root (readlink: Permission denied) 2024-04-22T13:26:48.980 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64658 tp_osd_tp ceph txt unknown /proc/63396/task/64658/exe (readlink: Permission denied) 2024-04-22T13:26:48.980 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64658 tp_osd_tp ceph NOFD /proc/63396/task/64658/fd (opendir: Permission denied) 2024-04-22T13:26:48.980 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64659 tp_osd_tp ceph cwd unknown /proc/63396/task/64659/cwd (readlink: Permission denied) 2024-04-22T13:26:48.980 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64659 tp_osd_tp ceph rtd unknown /proc/63396/task/64659/root (readlink: Permission denied) 2024-04-22T13:26:48.980 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64659 tp_osd_tp ceph txt unknown /proc/63396/task/64659/exe (readlink: Permission denied) 2024-04-22T13:26:48.980 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64659 tp_osd_tp ceph NOFD /proc/63396/task/64659/fd (opendir: Permission denied) 2024-04-22T13:26:48.980 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64660 tp_osd_tp ceph cwd unknown /proc/63396/task/64660/cwd (readlink: Permission denied) 2024-04-22T13:26:48.981 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64660 tp_osd_tp ceph rtd unknown /proc/63396/task/64660/root (readlink: Permission denied) 2024-04-22T13:26:48.981 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64660 tp_osd_tp ceph txt unknown /proc/63396/task/64660/exe (readlink: Permission denied) 2024-04-22T13:26:48.981 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64660 tp_osd_tp ceph NOFD /proc/63396/task/64660/fd (opendir: Permission denied) 2024-04-22T13:26:48.981 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64661 tp_osd_tp ceph cwd unknown /proc/63396/task/64661/cwd (readlink: Permission denied) 2024-04-22T13:26:48.981 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64661 tp_osd_tp ceph rtd unknown /proc/63396/task/64661/root (readlink: Permission denied) 2024-04-22T13:26:48.981 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64661 tp_osd_tp ceph txt unknown /proc/63396/task/64661/exe (readlink: Permission denied) 2024-04-22T13:26:48.981 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64661 tp_osd_tp ceph NOFD /proc/63396/task/64661/fd (opendir: Permission denied) 2024-04-22T13:26:48.981 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64662 tp_osd_tp ceph cwd unknown /proc/63396/task/64662/cwd (readlink: Permission denied) 2024-04-22T13:26:48.981 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64662 tp_osd_tp ceph rtd unknown /proc/63396/task/64662/root (readlink: Permission denied) 2024-04-22T13:26:48.981 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64662 tp_osd_tp ceph txt unknown /proc/63396/task/64662/exe (readlink: Permission denied) 2024-04-22T13:26:48.981 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64662 tp_osd_tp ceph NOFD /proc/63396/task/64662/fd (opendir: Permission denied) 2024-04-22T13:26:48.981 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64663 tp_osd_tp ceph cwd unknown /proc/63396/task/64663/cwd (readlink: Permission denied) 2024-04-22T13:26:48.981 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64663 tp_osd_tp ceph rtd unknown /proc/63396/task/64663/root (readlink: Permission denied) 2024-04-22T13:26:48.981 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64663 tp_osd_tp ceph txt unknown /proc/63396/task/64663/exe (readlink: Permission denied) 2024-04-22T13:26:48.981 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64663 tp_osd_tp ceph NOFD /proc/63396/task/64663/fd (opendir: Permission denied) 2024-04-22T13:26:48.982 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64664 tp_osd_tp ceph cwd unknown /proc/63396/task/64664/cwd (readlink: Permission denied) 2024-04-22T13:26:48.982 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64664 tp_osd_tp ceph rtd unknown /proc/63396/task/64664/root (readlink: Permission denied) 2024-04-22T13:26:48.982 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64664 tp_osd_tp ceph txt unknown /proc/63396/task/64664/exe (readlink: Permission denied) 2024-04-22T13:26:48.982 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64664 tp_osd_tp ceph NOFD /proc/63396/task/64664/fd (opendir: Permission denied) 2024-04-22T13:26:48.982 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64665 tp_osd_tp ceph cwd unknown /proc/63396/task/64665/cwd (readlink: Permission denied) 2024-04-22T13:26:48.982 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64665 tp_osd_tp ceph rtd unknown /proc/63396/task/64665/root (readlink: Permission denied) 2024-04-22T13:26:48.982 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64665 tp_osd_tp ceph txt unknown /proc/63396/task/64665/exe (readlink: Permission denied) 2024-04-22T13:26:48.982 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64665 tp_osd_tp ceph NOFD /proc/63396/task/64665/fd (opendir: Permission denied) 2024-04-22T13:26:48.982 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64666 tp_osd_tp ceph cwd unknown /proc/63396/task/64666/cwd (readlink: Permission denied) 2024-04-22T13:26:48.982 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64666 tp_osd_tp ceph rtd unknown /proc/63396/task/64666/root (readlink: Permission denied) 2024-04-22T13:26:48.982 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64666 tp_osd_tp ceph txt unknown /proc/63396/task/64666/exe (readlink: Permission denied) 2024-04-22T13:26:48.982 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64666 tp_osd_tp ceph NOFD /proc/63396/task/64666/fd (opendir: Permission denied) 2024-04-22T13:26:48.982 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64667 tp_osd_tp ceph cwd unknown /proc/63396/task/64667/cwd (readlink: Permission denied) 2024-04-22T13:26:48.982 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64667 tp_osd_tp ceph rtd unknown /proc/63396/task/64667/root (readlink: Permission denied) 2024-04-22T13:26:48.982 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64667 tp_osd_tp ceph txt unknown /proc/63396/task/64667/exe (readlink: Permission denied) 2024-04-22T13:26:48.983 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64667 tp_osd_tp ceph NOFD /proc/63396/task/64667/fd (opendir: Permission denied) 2024-04-22T13:26:48.983 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64668 osd_srv_h ceph cwd unknown /proc/63396/task/64668/cwd (readlink: Permission denied) 2024-04-22T13:26:48.983 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64668 osd_srv_h ceph rtd unknown /proc/63396/task/64668/root (readlink: Permission denied) 2024-04-22T13:26:48.983 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64668 osd_srv_h ceph txt unknown /proc/63396/task/64668/exe (readlink: Permission denied) 2024-04-22T13:26:48.983 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-osd 63396 64668 osd_srv_h ceph NOFD /proc/63396/task/64668/fd (opendir: Permission denied) 2024-04-22T13:26:48.983 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 64694 root cwd unknown /proc/64694/cwd (readlink: Permission denied) 2024-04-22T13:26:48.983 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 64694 root rtd unknown /proc/64694/root (readlink: Permission denied) 2024-04-22T13:26:48.983 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 64694 root txt unknown /proc/64694/exe (readlink: Permission denied) 2024-04-22T13:26:48.983 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 64694 root NOFD /proc/64694/fd (opendir: Permission denied) 2024-04-22T13:26:48.983 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 64711 root cwd unknown /proc/64711/cwd (readlink: Permission denied) 2024-04-22T13:26:48.983 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 64711 root rtd unknown /proc/64711/root (readlink: Permission denied) 2024-04-22T13:26:48.983 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 64711 root txt unknown /proc/64711/exe (readlink: Permission denied) 2024-04-22T13:26:48.983 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 64711 root NOFD /proc/64711/fd (opendir: Permission denied) 2024-04-22T13:26:48.983 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/6 67596 root cwd unknown /proc/67596/cwd (readlink: Permission denied) 2024-04-22T13:26:48.983 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/6 67596 root rtd unknown /proc/67596/root (readlink: Permission denied) 2024-04-22T13:26:48.983 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/6 67596 root txt unknown /proc/67596/exe (readlink: Permission denied) 2024-04-22T13:26:48.984 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/6 67596 root NOFD /proc/67596/fd (opendir: Permission denied) 2024-04-22T13:26:48.984 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 68484 root cwd unknown /proc/68484/cwd (readlink: Permission denied) 2024-04-22T13:26:48.984 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 68484 root rtd unknown /proc/68484/root (readlink: Permission denied) 2024-04-22T13:26:48.984 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 68484 root txt unknown /proc/68484/exe (readlink: Permission denied) 2024-04-22T13:26:48.984 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 68484 root NOFD /proc/68484/fd (opendir: Permission denied) 2024-04-22T13:26:48.984 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 68501 root cwd unknown /proc/68501/cwd (readlink: Permission denied) 2024-04-22T13:26:48.984 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 68501 root rtd unknown /proc/68501/root (readlink: Permission denied) 2024-04-22T13:26:48.984 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 68501 root txt unknown /proc/68501/exe (readlink: Permission denied) 2024-04-22T13:26:48.984 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 68501 root NOFD /proc/68501/fd (opendir: Permission denied) 2024-04-22T13:26:48.984 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 68502 root cwd unknown /proc/68502/cwd (readlink: Permission denied) 2024-04-22T13:26:48.984 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 68502 root rtd unknown /proc/68502/root (readlink: Permission denied) 2024-04-22T13:26:48.984 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 68502 root txt unknown /proc/68502/exe (readlink: Permission denied) 2024-04-22T13:26:48.984 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 68502 root NOFD /proc/68502/fd (opendir: Permission denied) 2024-04-22T13:26:48.984 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 68519 root cwd unknown /proc/68519/cwd (readlink: Permission denied) 2024-04-22T13:26:48.984 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 68519 root rtd unknown /proc/68519/root (readlink: Permission denied) 2024-04-22T13:26:48.984 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 68519 root txt unknown /proc/68519/exe (readlink: Permission denied) 2024-04-22T13:26:48.985 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 68519 root NOFD /proc/68519/fd (opendir: Permission denied) 2024-04-22T13:26:48.985 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 68520 root cwd unknown /proc/68520/cwd (readlink: Permission denied) 2024-04-22T13:26:48.985 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 68520 root rtd unknown /proc/68520/root (readlink: Permission denied) 2024-04-22T13:26:48.985 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 68520 root txt unknown /proc/68520/exe (readlink: Permission denied) 2024-04-22T13:26:48.985 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 68520 root NOFD /proc/68520/fd (opendir: Permission denied) 2024-04-22T13:26:48.985 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 68537 root cwd unknown /proc/68537/cwd (readlink: Permission denied) 2024-04-22T13:26:48.985 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 68537 root rtd unknown /proc/68537/root (readlink: Permission denied) 2024-04-22T13:26:48.985 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 68537 root txt unknown /proc/68537/exe (readlink: Permission denied) 2024-04-22T13:26:48.985 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 68537 root NOFD /proc/68537/fd (opendir: Permission denied) 2024-04-22T13:26:48.985 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 68538 root cwd unknown /proc/68538/cwd (readlink: Permission denied) 2024-04-22T13:26:48.985 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 68538 root rtd unknown /proc/68538/root (readlink: Permission denied) 2024-04-22T13:26:48.985 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 68538 root txt unknown /proc/68538/exe (readlink: Permission denied) 2024-04-22T13:26:48.985 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 68538 root NOFD /proc/68538/fd (opendir: Permission denied) 2024-04-22T13:26:48.985 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 68555 root cwd unknown /proc/68555/cwd (readlink: Permission denied) 2024-04-22T13:26:48.985 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 68555 root rtd unknown /proc/68555/root (readlink: Permission denied) 2024-04-22T13:26:48.985 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 68555 root txt unknown /proc/68555/exe (readlink: Permission denied) 2024-04-22T13:26:48.985 INFO:tasks.workunit.client.0.smithi012.stdout:journalct 68555 root NOFD /proc/68555/fd (opendir: Permission denied) 2024-04-22T13:26:48.986 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 69039 root cwd unknown /proc/69039/cwd (readlink: Permission denied) 2024-04-22T13:26:48.986 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 69039 root rtd unknown /proc/69039/root (readlink: Permission denied) 2024-04-22T13:26:48.986 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 69039 root txt unknown /proc/69039/exe (readlink: Permission denied) 2024-04-22T13:26:48.986 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 69039 root NOFD /proc/69039/fd (opendir: Permission denied) 2024-04-22T13:26:48.986 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 69041 root cwd unknown /proc/69041/cwd (readlink: Permission denied) 2024-04-22T13:26:48.986 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 69041 root rtd unknown /proc/69041/root (readlink: Permission denied) 2024-04-22T13:26:48.986 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 69041 root txt unknown /proc/69041/exe (readlink: Permission denied) 2024-04-22T13:26:48.986 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 69041 root NOFD /proc/69041/fd (opendir: Permission denied) 2024-04-22T13:26:48.986 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 ceph cwd unknown /proc/69043/cwd (readlink: Permission denied) 2024-04-22T13:26:48.986 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 ceph rtd unknown /proc/69043/root (readlink: Permission denied) 2024-04-22T13:26:48.986 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 ceph txt unknown /proc/69043/exe (readlink: Permission denied) 2024-04-22T13:26:48.986 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 ceph NOFD /proc/69043/fd (opendir: Permission denied) 2024-04-22T13:26:48.986 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69044 log ceph cwd unknown /proc/69043/task/69044/cwd (readlink: Permission denied) 2024-04-22T13:26:48.986 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69044 log ceph rtd unknown /proc/69043/task/69044/root (readlink: Permission denied) 2024-04-22T13:26:48.986 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69044 log ceph txt unknown /proc/69043/task/69044/exe (readlink: Permission denied) 2024-04-22T13:26:48.986 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69044 log ceph NOFD /proc/69043/task/69044/fd (opendir: Permission denied) 2024-04-22T13:26:48.987 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69046 msgr-work ceph cwd unknown /proc/69043/task/69046/cwd (readlink: Permission denied) 2024-04-22T13:26:48.987 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69046 msgr-work ceph rtd unknown /proc/69043/task/69046/root (readlink: Permission denied) 2024-04-22T13:26:48.987 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69046 msgr-work ceph txt unknown /proc/69043/task/69046/exe (readlink: Permission denied) 2024-04-22T13:26:48.987 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69046 msgr-work ceph NOFD /proc/69043/task/69046/fd (opendir: Permission denied) 2024-04-22T13:26:48.987 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69047 msgr-work ceph cwd unknown /proc/69043/task/69047/cwd (readlink: Permission denied) 2024-04-22T13:26:48.987 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69047 msgr-work ceph rtd unknown /proc/69043/task/69047/root (readlink: Permission denied) 2024-04-22T13:26:48.987 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69047 msgr-work ceph txt unknown /proc/69043/task/69047/exe (readlink: Permission denied) 2024-04-22T13:26:48.987 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69047 msgr-work ceph NOFD /proc/69043/task/69047/fd (opendir: Permission denied) 2024-04-22T13:26:48.987 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69048 msgr-work ceph cwd unknown /proc/69043/task/69048/cwd (readlink: Permission denied) 2024-04-22T13:26:48.987 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69048 msgr-work ceph rtd unknown /proc/69043/task/69048/root (readlink: Permission denied) 2024-04-22T13:26:48.987 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69048 msgr-work ceph txt unknown /proc/69043/task/69048/exe (readlink: Permission denied) 2024-04-22T13:26:48.987 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69048 msgr-work ceph NOFD /proc/69043/task/69048/fd (opendir: Permission denied) 2024-04-22T13:26:48.987 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69052 service ceph cwd unknown /proc/69043/task/69052/cwd (readlink: Permission denied) 2024-04-22T13:26:48.987 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69052 service ceph rtd unknown /proc/69043/task/69052/root (readlink: Permission denied) 2024-04-22T13:26:48.987 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69052 service ceph txt unknown /proc/69043/task/69052/exe (readlink: Permission denied) 2024-04-22T13:26:48.987 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69052 service ceph NOFD /proc/69043/task/69052/fd (opendir: Permission denied) 2024-04-22T13:26:48.988 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69053 admin_soc ceph cwd unknown /proc/69043/task/69053/cwd (readlink: Permission denied) 2024-04-22T13:26:48.988 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69053 admin_soc ceph rtd unknown /proc/69043/task/69053/root (readlink: Permission denied) 2024-04-22T13:26:48.988 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69053 admin_soc ceph txt unknown /proc/69043/task/69053/exe (readlink: Permission denied) 2024-04-22T13:26:48.988 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69053 admin_soc ceph NOFD /proc/69043/task/69053/fd (opendir: Permission denied) 2024-04-22T13:26:48.988 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69054 signal_ha ceph cwd unknown /proc/69043/task/69054/cwd (readlink: Permission denied) 2024-04-22T13:26:48.988 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69054 signal_ha ceph rtd unknown /proc/69043/task/69054/root (readlink: Permission denied) 2024-04-22T13:26:48.988 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69054 signal_ha ceph txt unknown /proc/69043/task/69054/exe (readlink: Permission denied) 2024-04-22T13:26:48.988 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69054 signal_ha ceph NOFD /proc/69043/task/69054/fd (opendir: Permission denied) 2024-04-22T13:26:48.988 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69055 io_contex ceph cwd unknown /proc/69043/task/69055/cwd (readlink: Permission denied) 2024-04-22T13:26:48.988 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69055 io_contex ceph rtd unknown /proc/69043/task/69055/root (readlink: Permission denied) 2024-04-22T13:26:48.988 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69055 io_contex ceph txt unknown /proc/69043/task/69055/exe (readlink: Permission denied) 2024-04-22T13:26:48.988 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69055 io_contex ceph NOFD /proc/69043/task/69055/fd (opendir: Permission denied) 2024-04-22T13:26:48.988 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69056 io_contex ceph cwd unknown /proc/69043/task/69056/cwd (readlink: Permission denied) 2024-04-22T13:26:48.988 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69056 io_contex ceph rtd unknown /proc/69043/task/69056/root (readlink: Permission denied) 2024-04-22T13:26:48.988 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69056 io_contex ceph txt unknown /proc/69043/task/69056/exe (readlink: Permission denied) 2024-04-22T13:26:48.988 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69056 io_contex ceph NOFD /proc/69043/task/69056/fd (opendir: Permission denied) 2024-04-22T13:26:48.989 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69057 ms_dispat ceph cwd unknown /proc/69043/task/69057/cwd (readlink: Permission denied) 2024-04-22T13:26:48.989 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69057 ms_dispat ceph rtd unknown /proc/69043/task/69057/root (readlink: Permission denied) 2024-04-22T13:26:48.989 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69057 ms_dispat ceph txt unknown /proc/69043/task/69057/exe (readlink: Permission denied) 2024-04-22T13:26:48.989 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69057 ms_dispat ceph NOFD /proc/69043/task/69057/fd (opendir: Permission denied) 2024-04-22T13:26:48.989 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69058 ms_local ceph cwd unknown /proc/69043/task/69058/cwd (readlink: Permission denied) 2024-04-22T13:26:48.989 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69058 ms_local ceph rtd unknown /proc/69043/task/69058/root (readlink: Permission denied) 2024-04-22T13:26:48.989 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69058 ms_local ceph txt unknown /proc/69043/task/69058/exe (readlink: Permission denied) 2024-04-22T13:26:48.989 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69058 ms_local ceph NOFD /proc/69043/task/69058/fd (opendir: Permission denied) 2024-04-22T13:26:48.989 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69059 safe_time ceph cwd unknown /proc/69043/task/69059/cwd (readlink: Permission denied) 2024-04-22T13:26:48.989 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69059 safe_time ceph rtd unknown /proc/69043/task/69059/root (readlink: Permission denied) 2024-04-22T13:26:48.989 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69059 safe_time ceph txt unknown /proc/69043/task/69059/exe (readlink: Permission denied) 2024-04-22T13:26:48.989 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69059 safe_time ceph NOFD /proc/69043/task/69059/fd (opendir: Permission denied) 2024-04-22T13:26:48.989 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69060 safe_time ceph cwd unknown /proc/69043/task/69060/cwd (readlink: Permission denied) 2024-04-22T13:26:48.989 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69060 safe_time ceph rtd unknown /proc/69043/task/69060/root (readlink: Permission denied) 2024-04-22T13:26:48.989 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69060 safe_time ceph txt unknown /proc/69043/task/69060/exe (readlink: Permission denied) 2024-04-22T13:26:48.989 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69060 safe_time ceph NOFD /proc/69043/task/69060/fd (opendir: Permission denied) 2024-04-22T13:26:48.990 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69061 ceph-mds ceph cwd unknown /proc/69043/task/69061/cwd (readlink: Permission denied) 2024-04-22T13:26:48.990 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69061 ceph-mds ceph rtd unknown /proc/69043/task/69061/root (readlink: Permission denied) 2024-04-22T13:26:48.990 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69061 ceph-mds ceph txt unknown /proc/69043/task/69061/exe (readlink: Permission denied) 2024-04-22T13:26:48.990 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69061 ceph-mds ceph NOFD /proc/69043/task/69061/fd (opendir: Permission denied) 2024-04-22T13:26:48.990 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69171 safe_time ceph cwd unknown /proc/69043/task/69171/cwd (readlink: Permission denied) 2024-04-22T13:26:48.990 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69171 safe_time ceph rtd unknown /proc/69043/task/69171/root (readlink: Permission denied) 2024-04-22T13:26:48.990 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69171 safe_time ceph txt unknown /proc/69043/task/69171/exe (readlink: Permission denied) 2024-04-22T13:26:48.990 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69043 69171 safe_time ceph NOFD /proc/69043/task/69171/fd (opendir: Permission denied) 2024-04-22T13:26:48.990 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 69402 root cwd unknown /proc/69402/cwd (readlink: Permission denied) 2024-04-22T13:26:48.990 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 69402 root rtd unknown /proc/69402/root (readlink: Permission denied) 2024-04-22T13:26:48.990 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 69402 root txt unknown /proc/69402/exe (readlink: Permission denied) 2024-04-22T13:26:48.990 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 69402 root NOFD /proc/69402/fd (opendir: Permission denied) 2024-04-22T13:26:48.990 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 69404 root cwd unknown /proc/69404/cwd (readlink: Permission denied) 2024-04-22T13:26:48.990 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 69404 root rtd unknown /proc/69404/root (readlink: Permission denied) 2024-04-22T13:26:48.991 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 69404 root txt unknown /proc/69404/exe (readlink: Permission denied) 2024-04-22T13:26:48.991 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 69404 root NOFD /proc/69404/fd (opendir: Permission denied) 2024-04-22T13:26:48.991 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 ceph cwd unknown /proc/69425/cwd (readlink: Permission denied) 2024-04-22T13:26:48.991 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 ceph rtd unknown /proc/69425/root (readlink: Permission denied) 2024-04-22T13:26:48.991 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 ceph txt unknown /proc/69425/exe (readlink: Permission denied) 2024-04-22T13:26:48.991 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 ceph NOFD /proc/69425/fd (opendir: Permission denied) 2024-04-22T13:26:48.991 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69428 log ceph cwd unknown /proc/69425/task/69428/cwd (readlink: Permission denied) 2024-04-22T13:26:48.991 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69428 log ceph rtd unknown /proc/69425/task/69428/root (readlink: Permission denied) 2024-04-22T13:26:48.991 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69428 log ceph txt unknown /proc/69425/task/69428/exe (readlink: Permission denied) 2024-04-22T13:26:48.991 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69428 log ceph NOFD /proc/69425/task/69428/fd (opendir: Permission denied) 2024-04-22T13:26:48.991 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69430 msgr-work ceph cwd unknown /proc/69425/task/69430/cwd (readlink: Permission denied) 2024-04-22T13:26:48.991 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69430 msgr-work ceph rtd unknown /proc/69425/task/69430/root (readlink: Permission denied) 2024-04-22T13:26:48.991 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69430 msgr-work ceph txt unknown /proc/69425/task/69430/exe (readlink: Permission denied) 2024-04-22T13:26:48.991 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69430 msgr-work ceph NOFD /proc/69425/task/69430/fd (opendir: Permission denied) 2024-04-22T13:26:48.991 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69431 msgr-work ceph cwd unknown /proc/69425/task/69431/cwd (readlink: Permission denied) 2024-04-22T13:26:48.991 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69431 msgr-work ceph rtd unknown /proc/69425/task/69431/root (readlink: Permission denied) 2024-04-22T13:26:48.992 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69431 msgr-work ceph txt unknown /proc/69425/task/69431/exe (readlink: Permission denied) 2024-04-22T13:26:48.992 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69431 msgr-work ceph NOFD /proc/69425/task/69431/fd (opendir: Permission denied) 2024-04-22T13:26:48.992 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69432 msgr-work ceph cwd unknown /proc/69425/task/69432/cwd (readlink: Permission denied) 2024-04-22T13:26:48.992 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69432 msgr-work ceph rtd unknown /proc/69425/task/69432/root (readlink: Permission denied) 2024-04-22T13:26:48.992 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69432 msgr-work ceph txt unknown /proc/69425/task/69432/exe (readlink: Permission denied) 2024-04-22T13:26:48.992 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69432 msgr-work ceph NOFD /proc/69425/task/69432/fd (opendir: Permission denied) 2024-04-22T13:26:48.992 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69436 service ceph cwd unknown /proc/69425/task/69436/cwd (readlink: Permission denied) 2024-04-22T13:26:48.992 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69436 service ceph rtd unknown /proc/69425/task/69436/root (readlink: Permission denied) 2024-04-22T13:26:48.992 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69436 service ceph txt unknown /proc/69425/task/69436/exe (readlink: Permission denied) 2024-04-22T13:26:48.992 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69436 service ceph NOFD /proc/69425/task/69436/fd (opendir: Permission denied) 2024-04-22T13:26:48.992 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69437 admin_soc ceph cwd unknown /proc/69425/task/69437/cwd (readlink: Permission denied) 2024-04-22T13:26:48.992 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69437 admin_soc ceph rtd unknown /proc/69425/task/69437/root (readlink: Permission denied) 2024-04-22T13:26:48.992 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69437 admin_soc ceph txt unknown /proc/69425/task/69437/exe (readlink: Permission denied) 2024-04-22T13:26:48.992 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69437 admin_soc ceph NOFD /proc/69425/task/69437/fd (opendir: Permission denied) 2024-04-22T13:26:48.992 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69438 signal_ha ceph cwd unknown /proc/69425/task/69438/cwd (readlink: Permission denied) 2024-04-22T13:26:48.993 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69438 signal_ha ceph rtd unknown /proc/69425/task/69438/root (readlink: Permission denied) 2024-04-22T13:26:48.993 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69438 signal_ha ceph txt unknown /proc/69425/task/69438/exe (readlink: Permission denied) 2024-04-22T13:26:48.993 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69438 signal_ha ceph NOFD /proc/69425/task/69438/fd (opendir: Permission denied) 2024-04-22T13:26:48.993 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69439 io_contex ceph cwd unknown /proc/69425/task/69439/cwd (readlink: Permission denied) 2024-04-22T13:26:48.993 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69439 io_contex ceph rtd unknown /proc/69425/task/69439/root (readlink: Permission denied) 2024-04-22T13:26:48.993 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69439 io_contex ceph txt unknown /proc/69425/task/69439/exe (readlink: Permission denied) 2024-04-22T13:26:48.993 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69439 io_contex ceph NOFD /proc/69425/task/69439/fd (opendir: Permission denied) 2024-04-22T13:26:48.993 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69440 io_contex ceph cwd unknown /proc/69425/task/69440/cwd (readlink: Permission denied) 2024-04-22T13:26:48.993 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69440 io_contex ceph rtd unknown /proc/69425/task/69440/root (readlink: Permission denied) 2024-04-22T13:26:48.993 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69440 io_contex ceph txt unknown /proc/69425/task/69440/exe (readlink: Permission denied) 2024-04-22T13:26:48.993 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69440 io_contex ceph NOFD /proc/69425/task/69440/fd (opendir: Permission denied) 2024-04-22T13:26:48.993 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69441 ms_dispat ceph cwd unknown /proc/69425/task/69441/cwd (readlink: Permission denied) 2024-04-22T13:26:48.993 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69441 ms_dispat ceph rtd unknown /proc/69425/task/69441/root (readlink: Permission denied) 2024-04-22T13:26:48.993 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69441 ms_dispat ceph txt unknown /proc/69425/task/69441/exe (readlink: Permission denied) 2024-04-22T13:26:48.993 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69441 ms_dispat ceph NOFD /proc/69425/task/69441/fd (opendir: Permission denied) 2024-04-22T13:26:48.994 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69442 ms_local ceph cwd unknown /proc/69425/task/69442/cwd (readlink: Permission denied) 2024-04-22T13:26:48.994 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69442 ms_local ceph rtd unknown /proc/69425/task/69442/root (readlink: Permission denied) 2024-04-22T13:26:48.994 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69442 ms_local ceph txt unknown /proc/69425/task/69442/exe (readlink: Permission denied) 2024-04-22T13:26:48.994 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69442 ms_local ceph NOFD /proc/69425/task/69442/fd (opendir: Permission denied) 2024-04-22T13:26:48.994 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69443 safe_time ceph cwd unknown /proc/69425/task/69443/cwd (readlink: Permission denied) 2024-04-22T13:26:48.994 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69443 safe_time ceph rtd unknown /proc/69425/task/69443/root (readlink: Permission denied) 2024-04-22T13:26:48.994 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69443 safe_time ceph txt unknown /proc/69425/task/69443/exe (readlink: Permission denied) 2024-04-22T13:26:48.994 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69443 safe_time ceph NOFD /proc/69425/task/69443/fd (opendir: Permission denied) 2024-04-22T13:26:48.994 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69444 safe_time ceph cwd unknown /proc/69425/task/69444/cwd (readlink: Permission denied) 2024-04-22T13:26:48.994 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69444 safe_time ceph rtd unknown /proc/69425/task/69444/root (readlink: Permission denied) 2024-04-22T13:26:48.994 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69444 safe_time ceph txt unknown /proc/69425/task/69444/exe (readlink: Permission denied) 2024-04-22T13:26:48.994 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69444 safe_time ceph NOFD /proc/69425/task/69444/fd (opendir: Permission denied) 2024-04-22T13:26:48.994 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69445 ceph-mds ceph cwd unknown /proc/69425/task/69445/cwd (readlink: Permission denied) 2024-04-22T13:26:48.994 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69445 ceph-mds ceph rtd unknown /proc/69425/task/69445/root (readlink: Permission denied) 2024-04-22T13:26:48.995 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69445 ceph-mds ceph txt unknown /proc/69425/task/69445/exe (readlink: Permission denied) 2024-04-22T13:26:48.995 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69445 ceph-mds ceph NOFD /proc/69425/task/69445/fd (opendir: Permission denied) 2024-04-22T13:26:48.995 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69450 safe_time ceph cwd unknown /proc/69425/task/69450/cwd (readlink: Permission denied) 2024-04-22T13:26:48.995 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69450 safe_time ceph rtd unknown /proc/69425/task/69450/root (readlink: Permission denied) 2024-04-22T13:26:48.995 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69450 safe_time ceph txt unknown /proc/69425/task/69450/exe (readlink: Permission denied) 2024-04-22T13:26:48.995 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 69425 69450 safe_time ceph NOFD /proc/69425/task/69450/fd (opendir: Permission denied) 2024-04-22T13:26:48.995 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 69989 root cwd unknown /proc/69989/cwd (readlink: Permission denied) 2024-04-22T13:26:48.995 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 69989 root rtd unknown /proc/69989/root (readlink: Permission denied) 2024-04-22T13:26:48.995 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 69989 root txt unknown /proc/69989/exe (readlink: Permission denied) 2024-04-22T13:26:48.995 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 69989 root NOFD /proc/69989/fd (opendir: Permission denied) 2024-04-22T13:26:48.995 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 69991 root cwd unknown /proc/69991/cwd (readlink: Permission denied) 2024-04-22T13:26:48.995 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 69991 root rtd unknown /proc/69991/root (readlink: Permission denied) 2024-04-22T13:26:48.995 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 69991 root txt unknown /proc/69991/exe (readlink: Permission denied) 2024-04-22T13:26:48.995 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 69991 root NOFD /proc/69991/fd (opendir: Permission denied) 2024-04-22T13:26:48.995 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 ceph cwd unknown /proc/70015/cwd (readlink: Permission denied) 2024-04-22T13:26:48.996 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 ceph rtd unknown /proc/70015/root (readlink: Permission denied) 2024-04-22T13:26:48.996 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 ceph txt unknown /proc/70015/exe (readlink: Permission denied) 2024-04-22T13:26:48.996 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 ceph NOFD /proc/70015/fd (opendir: Permission denied) 2024-04-22T13:26:48.996 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70033 log ceph cwd unknown /proc/70015/task/70033/cwd (readlink: Permission denied) 2024-04-22T13:26:48.996 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70033 log ceph rtd unknown /proc/70015/task/70033/root (readlink: Permission denied) 2024-04-22T13:26:48.996 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70033 log ceph txt unknown /proc/70015/task/70033/exe (readlink: Permission denied) 2024-04-22T13:26:48.996 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70033 log ceph NOFD /proc/70015/task/70033/fd (opendir: Permission denied) 2024-04-22T13:26:48.996 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70035 msgr-work ceph cwd unknown /proc/70015/task/70035/cwd (readlink: Permission denied) 2024-04-22T13:26:48.996 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70035 msgr-work ceph rtd unknown /proc/70015/task/70035/root (readlink: Permission denied) 2024-04-22T13:26:48.996 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70035 msgr-work ceph txt unknown /proc/70015/task/70035/exe (readlink: Permission denied) 2024-04-22T13:26:48.996 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70035 msgr-work ceph NOFD /proc/70015/task/70035/fd (opendir: Permission denied) 2024-04-22T13:26:48.996 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70036 msgr-work ceph cwd unknown /proc/70015/task/70036/cwd (readlink: Permission denied) 2024-04-22T13:26:48.996 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70036 msgr-work ceph rtd unknown /proc/70015/task/70036/root (readlink: Permission denied) 2024-04-22T13:26:48.996 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70036 msgr-work ceph txt unknown /proc/70015/task/70036/exe (readlink: Permission denied) 2024-04-22T13:26:48.996 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70036 msgr-work ceph NOFD /proc/70015/task/70036/fd (opendir: Permission denied) 2024-04-22T13:26:48.997 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70037 msgr-work ceph cwd unknown /proc/70015/task/70037/cwd (readlink: Permission denied) 2024-04-22T13:26:48.997 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70037 msgr-work ceph rtd unknown /proc/70015/task/70037/root (readlink: Permission denied) 2024-04-22T13:26:48.997 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70037 msgr-work ceph txt unknown /proc/70015/task/70037/exe (readlink: Permission denied) 2024-04-22T13:26:48.997 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70037 msgr-work ceph NOFD /proc/70015/task/70037/fd (opendir: Permission denied) 2024-04-22T13:26:48.997 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70041 service ceph cwd unknown /proc/70015/task/70041/cwd (readlink: Permission denied) 2024-04-22T13:26:48.997 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70041 service ceph rtd unknown /proc/70015/task/70041/root (readlink: Permission denied) 2024-04-22T13:26:48.997 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70041 service ceph txt unknown /proc/70015/task/70041/exe (readlink: Permission denied) 2024-04-22T13:26:48.997 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70041 service ceph NOFD /proc/70015/task/70041/fd (opendir: Permission denied) 2024-04-22T13:26:48.997 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70042 admin_soc ceph cwd unknown /proc/70015/task/70042/cwd (readlink: Permission denied) 2024-04-22T13:26:48.997 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70042 admin_soc ceph rtd unknown /proc/70015/task/70042/root (readlink: Permission denied) 2024-04-22T13:26:48.997 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70042 admin_soc ceph txt unknown /proc/70015/task/70042/exe (readlink: Permission denied) 2024-04-22T13:26:48.997 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70042 admin_soc ceph NOFD /proc/70015/task/70042/fd (opendir: Permission denied) 2024-04-22T13:26:48.997 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70043 signal_ha ceph cwd unknown /proc/70015/task/70043/cwd (readlink: Permission denied) 2024-04-22T13:26:48.997 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70043 signal_ha ceph rtd unknown /proc/70015/task/70043/root (readlink: Permission denied) 2024-04-22T13:26:48.998 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70043 signal_ha ceph txt unknown /proc/70015/task/70043/exe (readlink: Permission denied) 2024-04-22T13:26:48.998 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70043 signal_ha ceph NOFD /proc/70015/task/70043/fd (opendir: Permission denied) 2024-04-22T13:26:48.998 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70044 io_contex ceph cwd unknown /proc/70015/task/70044/cwd (readlink: Permission denied) 2024-04-22T13:26:48.998 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70044 io_contex ceph rtd unknown /proc/70015/task/70044/root (readlink: Permission denied) 2024-04-22T13:26:48.998 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70044 io_contex ceph txt unknown /proc/70015/task/70044/exe (readlink: Permission denied) 2024-04-22T13:26:48.998 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70044 io_contex ceph NOFD /proc/70015/task/70044/fd (opendir: Permission denied) 2024-04-22T13:26:48.998 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70045 io_contex ceph cwd unknown /proc/70015/task/70045/cwd (readlink: Permission denied) 2024-04-22T13:26:48.998 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70045 io_contex ceph rtd unknown /proc/70015/task/70045/root (readlink: Permission denied) 2024-04-22T13:26:48.998 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70045 io_contex ceph txt unknown /proc/70015/task/70045/exe (readlink: Permission denied) 2024-04-22T13:26:48.998 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70045 io_contex ceph NOFD /proc/70015/task/70045/fd (opendir: Permission denied) 2024-04-22T13:26:48.998 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70046 ms_dispat ceph cwd unknown /proc/70015/task/70046/cwd (readlink: Permission denied) 2024-04-22T13:26:48.998 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70046 ms_dispat ceph rtd unknown /proc/70015/task/70046/root (readlink: Permission denied) 2024-04-22T13:26:48.998 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70046 ms_dispat ceph txt unknown /proc/70015/task/70046/exe (readlink: Permission denied) 2024-04-22T13:26:48.999 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70046 ms_dispat ceph NOFD /proc/70015/task/70046/fd (opendir: Permission denied) 2024-04-22T13:26:48.999 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70047 ms_local ceph cwd unknown /proc/70015/task/70047/cwd (readlink: Permission denied) 2024-04-22T13:26:48.999 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70047 ms_local ceph rtd unknown /proc/70015/task/70047/root (readlink: Permission denied) 2024-04-22T13:26:48.999 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70047 ms_local ceph txt unknown /proc/70015/task/70047/exe (readlink: Permission denied) 2024-04-22T13:26:48.999 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70047 ms_local ceph NOFD /proc/70015/task/70047/fd (opendir: Permission denied) 2024-04-22T13:26:48.999 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70048 safe_time ceph cwd unknown /proc/70015/task/70048/cwd (readlink: Permission denied) 2024-04-22T13:26:48.999 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70048 safe_time ceph rtd unknown /proc/70015/task/70048/root (readlink: Permission denied) 2024-04-22T13:26:48.999 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70048 safe_time ceph txt unknown /proc/70015/task/70048/exe (readlink: Permission denied) 2024-04-22T13:26:48.999 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70048 safe_time ceph NOFD /proc/70015/task/70048/fd (opendir: Permission denied) 2024-04-22T13:26:48.999 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70049 safe_time ceph cwd unknown /proc/70015/task/70049/cwd (readlink: Permission denied) 2024-04-22T13:26:48.999 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70049 safe_time ceph rtd unknown /proc/70015/task/70049/root (readlink: Permission denied) 2024-04-22T13:26:48.999 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70049 safe_time ceph txt unknown /proc/70015/task/70049/exe (readlink: Permission denied) 2024-04-22T13:26:48.999 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70049 safe_time ceph NOFD /proc/70015/task/70049/fd (opendir: Permission denied) 2024-04-22T13:26:48.999 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70050 ceph-mds ceph cwd unknown /proc/70015/task/70050/cwd (readlink: Permission denied) 2024-04-22T13:26:48.999 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70050 ceph-mds ceph rtd unknown /proc/70015/task/70050/root (readlink: Permission denied) 2024-04-22T13:26:48.999 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70050 ceph-mds ceph txt unknown /proc/70015/task/70050/exe (readlink: Permission denied) 2024-04-22T13:26:49.000 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70050 ceph-mds ceph NOFD /proc/70015/task/70050/fd (opendir: Permission denied) 2024-04-22T13:26:49.000 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70055 safe_time ceph cwd unknown /proc/70015/task/70055/cwd (readlink: Permission denied) 2024-04-22T13:26:49.000 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70055 safe_time ceph rtd unknown /proc/70015/task/70055/root (readlink: Permission denied) 2024-04-22T13:26:49.000 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70055 safe_time ceph txt unknown /proc/70015/task/70055/exe (readlink: Permission denied) 2024-04-22T13:26:49.000 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 70055 safe_time ceph NOFD /proc/70015/task/70055/fd (opendir: Permission denied) 2024-04-22T13:26:49.000 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71928 ceph_time ceph cwd unknown /proc/70015/task/71928/cwd (readlink: Permission denied) 2024-04-22T13:26:49.000 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71928 ceph_time ceph rtd unknown /proc/70015/task/71928/root (readlink: Permission denied) 2024-04-22T13:26:49.000 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71928 ceph_time ceph txt unknown /proc/70015/task/71928/exe (readlink: Permission denied) 2024-04-22T13:26:49.000 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71928 ceph_time ceph NOFD /proc/70015/task/71928/fd (opendir: Permission denied) 2024-04-22T13:26:49.000 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71930 OpHistory ceph cwd unknown /proc/70015/task/71930/cwd (readlink: Permission denied) 2024-04-22T13:26:49.000 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71930 OpHistory ceph rtd unknown /proc/70015/task/71930/root (readlink: Permission denied) 2024-04-22T13:26:49.000 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71930 OpHistory ceph txt unknown /proc/70015/task/71930/exe (readlink: Permission denied) 2024-04-22T13:26:49.000 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71930 OpHistory ceph NOFD /proc/70015/task/71930/fd (opendir: Permission denied) 2024-04-22T13:26:49.000 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71933 ms_dispat ceph cwd unknown /proc/70015/task/71933/cwd (readlink: Permission denied) 2024-04-22T13:26:49.000 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71933 ms_dispat ceph rtd unknown /proc/70015/task/71933/root (readlink: Permission denied) 2024-04-22T13:26:49.001 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71933 ms_dispat ceph txt unknown /proc/70015/task/71933/exe (readlink: Permission denied) 2024-04-22T13:26:49.001 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71933 ms_dispat ceph NOFD /proc/70015/task/71933/fd (opendir: Permission denied) 2024-04-22T13:26:49.001 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71935 ms_dispat ceph cwd unknown /proc/70015/task/71935/cwd (readlink: Permission denied) 2024-04-22T13:26:49.001 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71935 ms_dispat ceph rtd unknown /proc/70015/task/71935/root (readlink: Permission denied) 2024-04-22T13:26:49.001 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71935 ms_dispat ceph txt unknown /proc/70015/task/71935/exe (readlink: Permission denied) 2024-04-22T13:26:49.001 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71935 ms_dispat ceph NOFD /proc/70015/task/71935/fd (opendir: Permission denied) 2024-04-22T13:26:49.001 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71937 mds_rank_ ceph cwd unknown /proc/70015/task/71937/cwd (readlink: Permission denied) 2024-04-22T13:26:49.001 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71937 mds_rank_ ceph rtd unknown /proc/70015/task/71937/root (readlink: Permission denied) 2024-04-22T13:26:49.001 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71937 mds_rank_ ceph txt unknown /proc/70015/task/71937/exe (readlink: Permission denied) 2024-04-22T13:26:49.001 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71937 mds_rank_ ceph NOFD /proc/70015/task/71937/fd (opendir: Permission denied) 2024-04-22T13:26:49.001 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71939 PQ_Finish ceph cwd unknown /proc/70015/task/71939/cwd (readlink: Permission denied) 2024-04-22T13:26:49.001 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71939 PQ_Finish ceph rtd unknown /proc/70015/task/71939/root (readlink: Permission denied) 2024-04-22T13:26:49.001 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71939 PQ_Finish ceph txt unknown /proc/70015/task/71939/exe (readlink: Permission denied) 2024-04-22T13:26:49.001 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71939 PQ_Finish ceph NOFD /proc/70015/task/71939/fd (opendir: Permission denied) 2024-04-22T13:26:49.001 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71940 safe_time ceph cwd unknown /proc/70015/task/71940/cwd (readlink: Permission denied) 2024-04-22T13:26:49.002 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71940 safe_time ceph rtd unknown /proc/70015/task/71940/root (readlink: Permission denied) 2024-04-22T13:26:49.002 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71940 safe_time ceph txt unknown /proc/70015/task/71940/exe (readlink: Permission denied) 2024-04-22T13:26:49.002 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71940 safe_time ceph NOFD /proc/70015/task/71940/fd (opendir: Permission denied) 2024-04-22T13:26:49.002 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71942 MR_Finish ceph cwd unknown /proc/70015/task/71942/cwd (readlink: Permission denied) 2024-04-22T13:26:49.002 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71942 MR_Finish ceph rtd unknown /proc/70015/task/71942/root (readlink: Permission denied) 2024-04-22T13:26:49.002 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71942 MR_Finish ceph txt unknown /proc/70015/task/71942/exe (readlink: Permission denied) 2024-04-22T13:26:49.002 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71942 MR_Finish ceph NOFD /proc/70015/task/71942/fd (opendir: Permission denied) 2024-04-22T13:26:49.002 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71946 md_submit ceph cwd unknown /proc/70015/task/71946/cwd (readlink: Permission denied) 2024-04-22T13:26:49.002 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71946 md_submit ceph rtd unknown /proc/70015/task/71946/root (readlink: Permission denied) 2024-04-22T13:26:49.002 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71946 md_submit ceph txt unknown /proc/70015/task/71946/exe (readlink: Permission denied) 2024-04-22T13:26:49.002 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70015 71946 md_submit ceph NOFD /proc/70015/task/71946/fd (opendir: Permission denied) 2024-04-22T13:26:49.002 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 70376 root cwd unknown /proc/70376/cwd (readlink: Permission denied) 2024-04-22T13:26:49.002 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 70376 root rtd unknown /proc/70376/root (readlink: Permission denied) 2024-04-22T13:26:49.002 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 70376 root txt unknown /proc/70376/exe (readlink: Permission denied) 2024-04-22T13:26:49.003 INFO:tasks.workunit.client.0.smithi012.stdout:conmon 70376 root NOFD /proc/70376/fd (opendir: Permission denied) 2024-04-22T13:26:49.003 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 70378 root cwd unknown /proc/70378/cwd (readlink: Permission denied) 2024-04-22T13:26:49.003 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 70378 root rtd unknown /proc/70378/root (readlink: Permission denied) 2024-04-22T13:26:49.005 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 70378 root txt unknown /proc/70378/exe (readlink: Permission denied) 2024-04-22T13:26:49.005 INFO:tasks.workunit.client.0.smithi012.stdout:podman-in 70378 root NOFD /proc/70378/fd (opendir: Permission denied) 2024-04-22T13:26:49.006 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 ceph cwd unknown /proc/70380/cwd (readlink: Permission denied) 2024-04-22T13:26:49.006 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 ceph rtd unknown /proc/70380/root (readlink: Permission denied) 2024-04-22T13:26:49.006 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 ceph txt unknown /proc/70380/exe (readlink: Permission denied) 2024-04-22T13:26:49.006 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 ceph NOFD /proc/70380/fd (opendir: Permission denied) 2024-04-22T13:26:49.006 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70381 log ceph cwd unknown /proc/70380/task/70381/cwd (readlink: Permission denied) 2024-04-22T13:26:49.006 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70381 log ceph rtd unknown /proc/70380/task/70381/root (readlink: Permission denied) 2024-04-22T13:26:49.006 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70381 log ceph txt unknown /proc/70380/task/70381/exe (readlink: Permission denied) 2024-04-22T13:26:49.006 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70381 log ceph NOFD /proc/70380/task/70381/fd (opendir: Permission denied) 2024-04-22T13:26:49.006 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70383 msgr-work ceph cwd unknown /proc/70380/task/70383/cwd (readlink: Permission denied) 2024-04-22T13:26:49.006 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70383 msgr-work ceph rtd unknown /proc/70380/task/70383/root (readlink: Permission denied) 2024-04-22T13:26:49.006 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70383 msgr-work ceph txt unknown /proc/70380/task/70383/exe (readlink: Permission denied) 2024-04-22T13:26:49.006 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70383 msgr-work ceph NOFD /proc/70380/task/70383/fd (opendir: Permission denied) 2024-04-22T13:26:49.006 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70384 msgr-work ceph cwd unknown /proc/70380/task/70384/cwd (readlink: Permission denied) 2024-04-22T13:26:49.007 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70384 msgr-work ceph rtd unknown /proc/70380/task/70384/root (readlink: Permission denied) 2024-04-22T13:26:49.007 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70384 msgr-work ceph txt unknown /proc/70380/task/70384/exe (readlink: Permission denied) 2024-04-22T13:26:49.007 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70384 msgr-work ceph NOFD /proc/70380/task/70384/fd (opendir: Permission denied) 2024-04-22T13:26:49.007 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70385 msgr-work ceph cwd unknown /proc/70380/task/70385/cwd (readlink: Permission denied) 2024-04-22T13:26:49.007 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70385 msgr-work ceph rtd unknown /proc/70380/task/70385/root (readlink: Permission denied) 2024-04-22T13:26:49.007 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70385 msgr-work ceph txt unknown /proc/70380/task/70385/exe (readlink: Permission denied) 2024-04-22T13:26:49.007 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70385 msgr-work ceph NOFD /proc/70380/task/70385/fd (opendir: Permission denied) 2024-04-22T13:26:49.007 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70389 service ceph cwd unknown /proc/70380/task/70389/cwd (readlink: Permission denied) 2024-04-22T13:26:49.007 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70389 service ceph rtd unknown /proc/70380/task/70389/root (readlink: Permission denied) 2024-04-22T13:26:49.007 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70389 service ceph txt unknown /proc/70380/task/70389/exe (readlink: Permission denied) 2024-04-22T13:26:49.007 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70389 service ceph NOFD /proc/70380/task/70389/fd (opendir: Permission denied) 2024-04-22T13:26:49.007 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70390 admin_soc ceph cwd unknown /proc/70380/task/70390/cwd (readlink: Permission denied) 2024-04-22T13:26:49.007 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70390 admin_soc ceph rtd unknown /proc/70380/task/70390/root (readlink: Permission denied) 2024-04-22T13:26:49.007 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70390 admin_soc ceph txt unknown /proc/70380/task/70390/exe (readlink: Permission denied) 2024-04-22T13:26:49.007 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70390 admin_soc ceph NOFD /proc/70380/task/70390/fd (opendir: Permission denied) 2024-04-22T13:26:49.008 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70391 signal_ha ceph cwd unknown /proc/70380/task/70391/cwd (readlink: Permission denied) 2024-04-22T13:26:49.008 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70391 signal_ha ceph rtd unknown /proc/70380/task/70391/root (readlink: Permission denied) 2024-04-22T13:26:49.008 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70391 signal_ha ceph txt unknown /proc/70380/task/70391/exe (readlink: Permission denied) 2024-04-22T13:26:49.008 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70391 signal_ha ceph NOFD /proc/70380/task/70391/fd (opendir: Permission denied) 2024-04-22T13:26:49.008 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70392 io_contex ceph cwd unknown /proc/70380/task/70392/cwd (readlink: Permission denied) 2024-04-22T13:26:49.008 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70392 io_contex ceph rtd unknown /proc/70380/task/70392/root (readlink: Permission denied) 2024-04-22T13:26:49.008 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70392 io_contex ceph txt unknown /proc/70380/task/70392/exe (readlink: Permission denied) 2024-04-22T13:26:49.008 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70392 io_contex ceph NOFD /proc/70380/task/70392/fd (opendir: Permission denied) 2024-04-22T13:26:49.008 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70393 io_contex ceph cwd unknown /proc/70380/task/70393/cwd (readlink: Permission denied) 2024-04-22T13:26:49.008 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70393 io_contex ceph rtd unknown /proc/70380/task/70393/root (readlink: Permission denied) 2024-04-22T13:26:49.008 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70393 io_contex ceph txt unknown /proc/70380/task/70393/exe (readlink: Permission denied) 2024-04-22T13:26:49.008 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70393 io_contex ceph NOFD /proc/70380/task/70393/fd (opendir: Permission denied) 2024-04-22T13:26:49.008 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70394 ms_dispat ceph cwd unknown /proc/70380/task/70394/cwd (readlink: Permission denied) 2024-04-22T13:26:49.008 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70394 ms_dispat ceph rtd unknown /proc/70380/task/70394/root (readlink: Permission denied) 2024-04-22T13:26:49.009 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70394 ms_dispat ceph txt unknown /proc/70380/task/70394/exe (readlink: Permission denied) 2024-04-22T13:26:49.009 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70394 ms_dispat ceph NOFD /proc/70380/task/70394/fd (opendir: Permission denied) 2024-04-22T13:26:49.009 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70395 ms_local ceph cwd unknown /proc/70380/task/70395/cwd (readlink: Permission denied) 2024-04-22T13:26:49.009 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70395 ms_local ceph rtd unknown /proc/70380/task/70395/root (readlink: Permission denied) 2024-04-22T13:26:49.009 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70395 ms_local ceph txt unknown /proc/70380/task/70395/exe (readlink: Permission denied) 2024-04-22T13:26:49.009 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70395 ms_local ceph NOFD /proc/70380/task/70395/fd (opendir: Permission denied) 2024-04-22T13:26:49.009 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70396 safe_time ceph cwd unknown /proc/70380/task/70396/cwd (readlink: Permission denied) 2024-04-22T13:26:49.009 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70396 safe_time ceph rtd unknown /proc/70380/task/70396/root (readlink: Permission denied) 2024-04-22T13:26:49.009 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70396 safe_time ceph txt unknown /proc/70380/task/70396/exe (readlink: Permission denied) 2024-04-22T13:26:49.009 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70396 safe_time ceph NOFD /proc/70380/task/70396/fd (opendir: Permission denied) 2024-04-22T13:26:49.009 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70397 safe_time ceph cwd unknown /proc/70380/task/70397/cwd (readlink: Permission denied) 2024-04-22T13:26:49.009 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70397 safe_time ceph rtd unknown /proc/70380/task/70397/root (readlink: Permission denied) 2024-04-22T13:26:49.009 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70397 safe_time ceph txt unknown /proc/70380/task/70397/exe (readlink: Permission denied) 2024-04-22T13:26:49.009 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70397 safe_time ceph NOFD /proc/70380/task/70397/fd (opendir: Permission denied) 2024-04-22T13:26:49.009 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70398 ceph-mds ceph cwd unknown /proc/70380/task/70398/cwd (readlink: Permission denied) 2024-04-22T13:26:49.010 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70398 ceph-mds ceph rtd unknown /proc/70380/task/70398/root (readlink: Permission denied) 2024-04-22T13:26:49.010 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70398 ceph-mds ceph txt unknown /proc/70380/task/70398/exe (readlink: Permission denied) 2024-04-22T13:26:49.010 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70398 ceph-mds ceph NOFD /proc/70380/task/70398/fd (opendir: Permission denied) 2024-04-22T13:26:49.010 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70403 safe_time ceph cwd unknown /proc/70380/task/70403/cwd (readlink: Permission denied) 2024-04-22T13:26:49.010 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70403 safe_time ceph rtd unknown /proc/70380/task/70403/root (readlink: Permission denied) 2024-04-22T13:26:49.010 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70403 safe_time ceph txt unknown /proc/70380/task/70403/exe (readlink: Permission denied) 2024-04-22T13:26:49.010 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 70403 safe_time ceph NOFD /proc/70380/task/70403/fd (opendir: Permission denied) 2024-04-22T13:26:49.010 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71929 ceph_time ceph cwd unknown /proc/70380/task/71929/cwd (readlink: Permission denied) 2024-04-22T13:26:49.010 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71929 ceph_time ceph rtd unknown /proc/70380/task/71929/root (readlink: Permission denied) 2024-04-22T13:26:49.010 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71929 ceph_time ceph txt unknown /proc/70380/task/71929/exe (readlink: Permission denied) 2024-04-22T13:26:49.010 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71929 ceph_time ceph NOFD /proc/70380/task/71929/fd (opendir: Permission denied) 2024-04-22T13:26:49.010 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71931 OpHistory ceph cwd unknown /proc/70380/task/71931/cwd (readlink: Permission denied) 2024-04-22T13:26:49.010 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71931 OpHistory ceph rtd unknown /proc/70380/task/71931/root (readlink: Permission denied) 2024-04-22T13:26:49.010 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71931 OpHistory ceph txt unknown /proc/70380/task/71931/exe (readlink: Permission denied) 2024-04-22T13:26:49.011 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71931 OpHistory ceph NOFD /proc/70380/task/71931/fd (opendir: Permission denied) 2024-04-22T13:26:49.011 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71932 ms_dispat ceph cwd unknown /proc/70380/task/71932/cwd (readlink: Permission denied) 2024-04-22T13:26:49.011 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71932 ms_dispat ceph rtd unknown /proc/70380/task/71932/root (readlink: Permission denied) 2024-04-22T13:26:49.011 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71932 ms_dispat ceph txt unknown /proc/70380/task/71932/exe (readlink: Permission denied) 2024-04-22T13:26:49.011 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71932 ms_dispat ceph NOFD /proc/70380/task/71932/fd (opendir: Permission denied) 2024-04-22T13:26:49.011 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71934 ms_dispat ceph cwd unknown /proc/70380/task/71934/cwd (readlink: Permission denied) 2024-04-22T13:26:49.011 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71934 ms_dispat ceph rtd unknown /proc/70380/task/71934/root (readlink: Permission denied) 2024-04-22T13:26:49.011 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71934 ms_dispat ceph txt unknown /proc/70380/task/71934/exe (readlink: Permission denied) 2024-04-22T13:26:49.011 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71934 ms_dispat ceph NOFD /proc/70380/task/71934/fd (opendir: Permission denied) 2024-04-22T13:26:49.011 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71936 mds_rank_ ceph cwd unknown /proc/70380/task/71936/cwd (readlink: Permission denied) 2024-04-22T13:26:49.011 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71936 mds_rank_ ceph rtd unknown /proc/70380/task/71936/root (readlink: Permission denied) 2024-04-22T13:26:49.011 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71936 mds_rank_ ceph txt unknown /proc/70380/task/71936/exe (readlink: Permission denied) 2024-04-22T13:26:49.011 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71936 mds_rank_ ceph NOFD /proc/70380/task/71936/fd (opendir: Permission denied) 2024-04-22T13:26:49.011 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71938 PQ_Finish ceph cwd unknown /proc/70380/task/71938/cwd (readlink: Permission denied) 2024-04-22T13:26:49.011 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71938 PQ_Finish ceph rtd unknown /proc/70380/task/71938/root (readlink: Permission denied) 2024-04-22T13:26:49.012 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71938 PQ_Finish ceph txt unknown /proc/70380/task/71938/exe (readlink: Permission denied) 2024-04-22T13:26:49.012 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71938 PQ_Finish ceph NOFD /proc/70380/task/71938/fd (opendir: Permission denied) 2024-04-22T13:26:49.012 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71941 safe_time ceph cwd unknown /proc/70380/task/71941/cwd (readlink: Permission denied) 2024-04-22T13:26:49.012 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71941 safe_time ceph rtd unknown /proc/70380/task/71941/root (readlink: Permission denied) 2024-04-22T13:26:49.012 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71941 safe_time ceph txt unknown /proc/70380/task/71941/exe (readlink: Permission denied) 2024-04-22T13:26:49.012 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71941 safe_time ceph NOFD /proc/70380/task/71941/fd (opendir: Permission denied) 2024-04-22T13:26:49.012 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71943 MR_Finish ceph cwd unknown /proc/70380/task/71943/cwd (readlink: Permission denied) 2024-04-22T13:26:49.012 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71943 MR_Finish ceph rtd unknown /proc/70380/task/71943/root (readlink: Permission denied) 2024-04-22T13:26:49.012 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71943 MR_Finish ceph txt unknown /proc/70380/task/71943/exe (readlink: Permission denied) 2024-04-22T13:26:49.012 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71943 MR_Finish ceph NOFD /proc/70380/task/71943/fd (opendir: Permission denied) 2024-04-22T13:26:49.012 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71947 md_submit ceph cwd unknown /proc/70380/task/71947/cwd (readlink: Permission denied) 2024-04-22T13:26:49.012 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71947 md_submit ceph rtd unknown /proc/70380/task/71947/root (readlink: Permission denied) 2024-04-22T13:26:49.012 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71947 md_submit ceph txt unknown /proc/70380/task/71947/exe (readlink: Permission denied) 2024-04-22T13:26:49.012 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-mds 70380 71947 md_submit ceph NOFD /proc/70380/task/71947/fd (opendir: Permission denied) 2024-04-22T13:26:49.012 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/5 73837 root cwd unknown /proc/73837/cwd (readlink: Permission denied) 2024-04-22T13:26:49.013 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/5 73837 root rtd unknown /proc/73837/root (readlink: Permission denied) 2024-04-22T13:26:49.013 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/5 73837 root txt unknown /proc/73837/exe (readlink: Permission denied) 2024-04-22T13:26:49.013 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/5 73837 root NOFD /proc/73837/fd (opendir: Permission denied) 2024-04-22T13:26:49.013 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/2 73926 root cwd unknown /proc/73926/cwd (readlink: Permission denied) 2024-04-22T13:26:49.013 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/2 73926 root rtd unknown /proc/73926/root (readlink: Permission denied) 2024-04-22T13:26:49.013 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/2 73926 root txt unknown /proc/73926/exe (readlink: Permission denied) 2024-04-22T13:26:49.013 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/2 73926 root NOFD /proc/73926/fd (opendir: Permission denied) 2024-04-22T13:26:49.013 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/0 74824 root cwd unknown /proc/74824/cwd (readlink: Permission denied) 2024-04-22T13:26:49.013 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/0 74824 root rtd unknown /proc/74824/root (readlink: Permission denied) 2024-04-22T13:26:49.013 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/0 74824 root txt unknown /proc/74824/exe (readlink: Permission denied) 2024-04-22T13:26:49.013 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/0 74824 root NOFD /proc/74824/fd (opendir: Permission denied) 2024-04-22T13:26:49.013 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/4 79085 root cwd unknown /proc/79085/cwd (readlink: Permission denied) 2024-04-22T13:26:49.013 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/4 79085 root rtd unknown /proc/79085/root (readlink: Permission denied) 2024-04-22T13:26:49.013 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/4 79085 root txt unknown /proc/79085/exe (readlink: Permission denied) 2024-04-22T13:26:49.013 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/4 79085 root NOFD /proc/79085/fd (opendir: Permission denied) 2024-04-22T13:26:49.013 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/7 79409 root cwd unknown /proc/79409/cwd (readlink: Permission denied) 2024-04-22T13:26:49.014 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/7 79409 root rtd unknown /proc/79409/root (readlink: Permission denied) 2024-04-22T13:26:49.014 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/7 79409 root txt unknown /proc/79409/exe (readlink: Permission denied) 2024-04-22T13:26:49.014 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/7 79409 root NOFD /proc/79409/fd (opendir: Permission denied) 2024-04-22T13:26:49.014 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/1 80602 root cwd unknown /proc/80602/cwd (readlink: Permission denied) 2024-04-22T13:26:49.014 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/1 80602 root rtd unknown /proc/80602/root (readlink: Permission denied) 2024-04-22T13:26:49.014 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/1 80602 root txt unknown /proc/80602/exe (readlink: Permission denied) 2024-04-22T13:26:49.014 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/1 80602 root NOFD /proc/80602/fd (opendir: Permission denied) 2024-04-22T13:26:49.014 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/5 80782 root cwd unknown /proc/80782/cwd (readlink: Permission denied) 2024-04-22T13:26:49.014 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/5 80782 root rtd unknown /proc/80782/root (readlink: Permission denied) 2024-04-22T13:26:49.014 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/5 80782 root txt unknown /proc/80782/exe (readlink: Permission denied) 2024-04-22T13:26:49.014 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/5 80782 root NOFD /proc/80782/fd (opendir: Permission denied) 2024-04-22T13:26:49.014 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-msgr 80894 root cwd unknown /proc/80894/cwd (readlink: Permission denied) 2024-04-22T13:26:49.014 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-msgr 80894 root rtd unknown /proc/80894/root (readlink: Permission denied) 2024-04-22T13:26:49.014 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-msgr 80894 root txt unknown /proc/80894/exe (readlink: Permission denied) 2024-04-22T13:26:49.015 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-msgr 80894 root NOFD /proc/80894/fd (opendir: Permission denied) 2024-04-22T13:26:49.015 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-watc 80895 root cwd unknown /proc/80895/cwd (readlink: Permission denied) 2024-04-22T13:26:49.015 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-watc 80895 root rtd unknown /proc/80895/root (readlink: Permission denied) 2024-04-22T13:26:49.015 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-watc 80895 root txt unknown /proc/80895/exe (readlink: Permission denied) 2024-04-22T13:26:49.015 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-watc 80895 root NOFD /proc/80895/fd (opendir: Permission denied) 2024-04-22T13:26:49.015 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-comp 80896 root cwd unknown /proc/80896/cwd (readlink: Permission denied) 2024-04-22T13:26:49.015 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-comp 80896 root rtd unknown /proc/80896/root (readlink: Permission denied) 2024-04-22T13:26:49.015 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-comp 80896 root txt unknown /proc/80896/exe (readlink: Permission denied) 2024-04-22T13:26:49.015 INFO:tasks.workunit.client.0.smithi012.stdout:ceph-comp 80896 root NOFD /proc/80896/fd (opendir: Permission denied) 2024-04-22T13:26:49.015 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/1 80916 root cwd unknown /proc/80916/cwd (readlink: Permission denied) 2024-04-22T13:26:49.015 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/1 80916 root rtd unknown /proc/80916/root (readlink: Permission denied) 2024-04-22T13:26:49.015 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/1 80916 root txt unknown /proc/80916/exe (readlink: Permission denied) 2024-04-22T13:26:49.015 INFO:tasks.workunit.client.0.smithi012.stdout:kworker/1 80916 root NOFD /proc/80916/fd (opendir: Permission denied) 2024-04-22T13:26:49.015 INFO:tasks.workunit.client.0.smithi012.stdout:sshd 81196 root cwd unknown /proc/81196/cwd (readlink: Permission denied) 2024-04-22T13:26:49.015 INFO:tasks.workunit.client.0.smithi012.stdout:sshd 81196 root rtd unknown /proc/81196/root (readlink: Permission denied) 2024-04-22T13:26:49.016 INFO:tasks.workunit.client.0.smithi012.stdout:sshd 81196 root txt unknown /proc/81196/exe (readlink: Permission denied) 2024-04-22T13:26:49.016 INFO:tasks.workunit.client.0.smithi012.stdout:sshd 81196 root NOFD /proc/81196/fd (opendir: Permission denied) 2024-04-22T13:26:49.016 INFO:tasks.workunit.client.0.smithi012.stdout:sshd 81256 root cwd unknown /proc/81256/cwd (readlink: Permission denied) 2024-04-22T13:26:49.016 INFO:tasks.workunit.client.0.smithi012.stdout:sshd 81256 root rtd unknown /proc/81256/root (readlink: Permission denied) 2024-04-22T13:26:49.016 INFO:tasks.workunit.client.0.smithi012.stdout:sshd 81256 root txt unknown /proc/81256/exe (readlink: Permission denied) 2024-04-22T13:26:49.016 INFO:tasks.workunit.client.0.smithi012.stdout:sshd 81256 root NOFD /proc/81256/fd (opendir: Permission denied) 2024-04-22T13:26:49.016 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 83784 root cwd unknown /proc/83784/cwd (readlink: Permission denied) 2024-04-22T13:26:49.016 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 83784 root rtd unknown /proc/83784/root (readlink: Permission denied) 2024-04-22T13:26:49.016 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 83784 root txt unknown /proc/83784/exe (readlink: Permission denied) 2024-04-22T13:26:49.016 INFO:tasks.workunit.client.0.smithi012.stdout:sudo 83784 root NOFD /proc/83784/fd (opendir: Permission denied) 2024-04-22T13:26:49.016 INFO:tasks.workunit.client.0.smithi012.stdout:daemon-he 83812 root cwd unknown /proc/83812/cwd (readlink: Permission denied) 2024-04-22T13:26:49.016 INFO:tasks.workunit.client.0.smithi012.stdout:daemon-he 83812 root rtd unknown /proc/83812/root (readlink: Permission denied) 2024-04-22T13:26:49.016 INFO:tasks.workunit.client.0.smithi012.stdout:daemon-he 83812 root txt unknown /proc/83812/exe (readlink: Permission denied) 2024-04-22T13:26:49.016 INFO:tasks.workunit.client.0.smithi012.stdout:daemon-he 83812 root NOFD /proc/83812/fd (opendir: Permission denied) 2024-04-22T13:26:49.016 INFO:tasks.workunit.client.0.smithi012.stdout:bash 83822 root cwd unknown /proc/83822/cwd (readlink: Permission denied) 2024-04-22T13:26:49.017 INFO:tasks.workunit.client.0.smithi012.stdout:bash 83822 root rtd unknown /proc/83822/root (readlink: Permission denied) 2024-04-22T13:26:49.017 INFO:tasks.workunit.client.0.smithi012.stdout:bash 83822 root txt unknown /proc/83822/exe (readlink: Permission denied) 2024-04-22T13:26:49.017 INFO:tasks.workunit.client.0.smithi012.stdout:bash 83822 root NOFD /proc/83822/fd (opendir: Permission denied) 2024-04-22T13:26:49.017 INFO:tasks.workunit.client.0.smithi012.stdout:timeout 87192 ubuntu cwd DIR 0,115 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T13:26:49.017 INFO:tasks.workunit.client.0.smithi012.stdout:timeout 87192 ubuntu rtd DIR 8,1 4096 2 / 2024-04-22T13:26:49.017 INFO:tasks.workunit.client.0.smithi012.stdout:timeout 87192 ubuntu txt REG 8,1 41144 7580 /usr/bin/timeout 2024-04-22T13:26:49.017 INFO:tasks.workunit.client.0.smithi012.stdout:timeout 87192 ubuntu mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-04-22T13:26:49.017 INFO:tasks.workunit.client.0.smithi012.stdout:timeout 87192 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-22T13:26:49.017 INFO:tasks.workunit.client.0.smithi012.stdout:timeout 87192 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-22T13:26:49.017 INFO:tasks.workunit.client.0.smithi012.stdout:timeout 87192 ubuntu mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-04-22T13:26:49.017 INFO:tasks.workunit.client.0.smithi012.stdout:timeout 87192 ubuntu mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-04-22T13:26:49.017 INFO:tasks.workunit.client.0.smithi012.stdout:timeout 87192 ubuntu mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-04-22T13:26:49.017 INFO:tasks.workunit.client.0.smithi012.stdout:timeout 87192 ubuntu mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-04-22T13:26:49.017 INFO:tasks.workunit.client.0.smithi012.stdout:timeout 87192 ubuntu mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-04-22T13:26:49.017 INFO:tasks.workunit.client.0.smithi012.stdout:timeout 87192 ubuntu mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-04-22T13:26:49.018 INFO:tasks.workunit.client.0.smithi012.stdout:timeout 87192 ubuntu mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-04-22T13:26:49.018 INFO:tasks.workunit.client.0.smithi012.stdout:timeout 87192 ubuntu mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-04-22T13:26:49.018 INFO:tasks.workunit.client.0.smithi012.stdout:timeout 87192 ubuntu mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-04-22T13:26:49.018 INFO:tasks.workunit.client.0.smithi012.stdout:timeout 87192 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-22T13:26:49.018 INFO:tasks.workunit.client.0.smithi012.stdout:timeout 87192 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-22T13:26:49.018 INFO:tasks.workunit.client.0.smithi012.stdout:timeout 87192 ubuntu mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-04-22T13:26:49.018 INFO:tasks.workunit.client.0.smithi012.stdout:timeout 87192 ubuntu 0r FIFO 0,13 0t0 236596 pipe 2024-04-22T13:26:49.018 INFO:tasks.workunit.client.0.smithi012.stdout:timeout 87192 ubuntu 1w FIFO 0,13 0t0 236597 pipe 2024-04-22T13:26:49.018 INFO:tasks.workunit.client.0.smithi012.stdout:timeout 87192 ubuntu 2w FIFO 0,13 0t0 236598 pipe 2024-04-22T13:26:49.018 INFO:tasks.workunit.client.0.smithi012.stdout:fsync-tes 87210 ubuntu cwd DIR 0,115 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T13:26:49.018 INFO:tasks.workunit.client.0.smithi012.stdout:fsync-tes 87210 ubuntu rtd DIR 8,1 4096 2 / 2024-04-22T13:26:49.018 INFO:tasks.workunit.client.0.smithi012.stdout:fsync-tes 87210 ubuntu txt REG 8,1 1389072 7149 /usr/bin/bash 2024-04-22T13:26:49.018 INFO:tasks.workunit.client.0.smithi012.stdout:fsync-tes 87210 ubuntu mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-04-22T13:26:49.018 INFO:tasks.workunit.client.0.smithi012.stdout:fsync-tes 87210 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-22T13:26:49.018 INFO:tasks.workunit.client.0.smithi012.stdout:fsync-tes 87210 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-22T13:26:49.019 INFO:tasks.workunit.client.0.smithi012.stdout:fsync-tes 87210 ubuntu mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-04-22T13:26:49.019 INFO:tasks.workunit.client.0.smithi012.stdout:fsync-tes 87210 ubuntu mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-04-22T13:26:49.019 INFO:tasks.workunit.client.0.smithi012.stdout:fsync-tes 87210 ubuntu mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-04-22T13:26:49.019 INFO:tasks.workunit.client.0.smithi012.stdout:fsync-tes 87210 ubuntu mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-04-22T13:26:49.019 INFO:tasks.workunit.client.0.smithi012.stdout:fsync-tes 87210 ubuntu mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-04-22T13:26:49.019 INFO:tasks.workunit.client.0.smithi012.stdout:fsync-tes 87210 ubuntu mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-04-22T13:26:49.019 INFO:tasks.workunit.client.0.smithi012.stdout:fsync-tes 87210 ubuntu mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-04-22T13:26:49.019 INFO:tasks.workunit.client.0.smithi012.stdout:fsync-tes 87210 ubuntu mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-04-22T13:26:49.019 INFO:tasks.workunit.client.0.smithi012.stdout:fsync-tes 87210 ubuntu mem REG 8,1 195080 7147 /usr/lib64/libtinfo.so.6.2 2024-04-22T13:26:49.019 INFO:tasks.workunit.client.0.smithi012.stdout:fsync-tes 87210 ubuntu mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-04-22T13:26:49.019 INFO:tasks.workunit.client.0.smithi012.stdout:fsync-tes 87210 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-22T13:26:49.019 INFO:tasks.workunit.client.0.smithi012.stdout:fsync-tes 87210 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-22T13:26:49.019 INFO:tasks.workunit.client.0.smithi012.stdout:fsync-tes 87210 ubuntu mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-04-22T13:26:49.019 INFO:tasks.workunit.client.0.smithi012.stdout:fsync-tes 87210 ubuntu 0r FIFO 0,13 0t0 236596 pipe 2024-04-22T13:26:49.019 INFO:tasks.workunit.client.0.smithi012.stdout:fsync-tes 87210 ubuntu 1w FIFO 0,13 0t0 236597 pipe 2024-04-22T13:26:49.020 INFO:tasks.workunit.client.0.smithi012.stdout:fsync-tes 87210 ubuntu 2w FIFO 0,13 0t0 236598 pipe 2024-04-22T13:26:49.020 INFO:tasks.workunit.client.0.smithi012.stdout:fsync-tes 87210 ubuntu 255r REG 8,1 207 527993 /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-04-22T13:26:49.020 INFO:tasks.workunit.client.0.smithi012.stdout:sleep 88659 root cwd unknown /proc/88659/cwd (readlink: Permission denied) 2024-04-22T13:26:49.020 INFO:tasks.workunit.client.0.smithi012.stdout:sleep 88659 root rtd unknown /proc/88659/root (readlink: Permission denied) 2024-04-22T13:26:49.020 INFO:tasks.workunit.client.0.smithi012.stdout:sleep 88659 root txt unknown /proc/88659/exe (readlink: Permission denied) 2024-04-22T13:26:49.020 INFO:tasks.workunit.client.0.smithi012.stdout:sleep 88659 root NOFD /proc/88659/fd (opendir: Permission denied) 2024-04-22T13:26:49.020 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88661 ubuntu cwd DIR 0,115 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T13:26:49.020 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88661 ubuntu rtd DIR 8,1 4096 2 / 2024-04-22T13:26:49.020 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88661 ubuntu txt REG 8,1 179232 9797 /usr/bin/lsof 2024-04-22T13:26:49.020 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88661 ubuntu mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-22T13:26:49.020 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88661 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-22T13:26:49.020 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88661 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-22T13:26:49.020 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88661 ubuntu mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-22T13:26:49.020 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88661 ubuntu mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-22T13:26:49.021 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88661 ubuntu mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-22T13:26:49.021 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88661 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-22T13:26:49.021 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88661 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-22T13:26:49.021 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88661 ubuntu mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-22T13:26:49.021 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88661 ubuntu mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-22T13:26:49.021 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88661 ubuntu mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-22T13:26:49.021 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88661 ubuntu mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-22T13:26:49.021 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88661 ubuntu mem REG 8,1 196224 8914 /usr/lib64/libtirpc.so.3.0.0 2024-04-22T13:26:49.021 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88661 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-22T13:26:49.021 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88661 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-22T13:26:49.021 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88661 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-22T13:26:49.021 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88661 ubuntu 0r FIFO 0,13 0t0 236596 pipe 2024-04-22T13:26:49.021 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88661 ubuntu 1w FIFO 0,13 0t0 236597 pipe 2024-04-22T13:26:49.021 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88661 ubuntu 2w FIFO 0,13 0t0 236598 pipe 2024-04-22T13:26:49.021 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88661 ubuntu 3r DIR 0,20 0 1 /proc 2024-04-22T13:26:49.021 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88661 ubuntu 4r DIR 0,20 0 235930 /proc/88661/fd 2024-04-22T13:26:49.022 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88661 ubuntu 5w FIFO 0,13 0t0 235935 pipe 2024-04-22T13:26:49.022 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88661 ubuntu 6r FIFO 0,13 0t0 235936 pipe 2024-04-22T13:26:49.022 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88662 ubuntu cwd DIR 0,115 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T13:26:49.022 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88662 ubuntu rtd DIR 8,1 4096 2 / 2024-04-22T13:26:49.022 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88662 ubuntu txt REG 8,1 179232 9797 /usr/bin/lsof 2024-04-22T13:26:49.022 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88662 ubuntu mem REG 8,1 4487176 7490 /usr/lib64/libcrypto.so.3.0.7 2024-04-22T13:26:49.022 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88662 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-04-22T13:26:49.022 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88662 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-04-22T13:26:49.022 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88662 ubuntu mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-04-22T13:26:49.022 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88662 ubuntu mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-04-22T13:26:49.022 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88662 ubuntu mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-04-22T13:26:49.022 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88662 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-04-22T13:26:49.022 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88662 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-04-22T13:26:49.022 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88662 ubuntu mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-04-22T13:26:49.023 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88662 ubuntu mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-04-22T13:26:49.023 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88662 ubuntu mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-04-22T13:26:49.023 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88662 ubuntu mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-04-22T13:26:49.023 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88662 ubuntu mem REG 8,1 196224 8914 /usr/lib64/libtirpc.so.3.0.0 2024-04-22T13:26:49.023 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88662 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-04-22T13:26:49.023 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88662 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-04-22T13:26:49.023 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88662 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-04-22T13:26:49.023 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88662 ubuntu 4r FIFO 0,13 0t0 235935 pipe 2024-04-22T13:26:49.023 INFO:tasks.workunit.client.0.smithi012.stdout:lsof 88662 ubuntu 7w FIFO 0,13 0t0 235936 pipe 2024-04-22T13:26:49.023 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-22T13:26:49.024 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T13:26:49.067 INFO:tasks.workunit:Stopping ['suites/fsync-tester.sh'] on client.0... 2024-04-22T13:26:49.067 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-22T13:26:49.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:49 smithi012 ceph-mon[28265]: pgmap v84: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.3 MiB/s wr, 19 op/s 2024-04-22T13:26:49.483 DEBUG:teuthology.parallel:result is None 2024-04-22T13:26:49.484 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-22T13:26:49.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:49 smithi134 ceph-mon[31338]: pgmap v84: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.3 MiB/s wr, 19 op/s 2024-04-22T13:26:49.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:49 smithi111 ceph-mon[31361]: pgmap v84: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.3 MiB/s wr, 19 op/s 2024-04-22T13:26:50.098 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-22T13:26:50.098 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-22T13:26:50.109 INFO:tasks.quiescer:joining Quiescers 2024-04-22T13:26:50.109 INFO:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-04-22T13:26:50.109 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:26:50.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.217+0000 7fe4b28e4640 1 -- 172.21.15.12:0/160444336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4ac105e30 msgr2=0x7fe4ac106210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:50.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.217+0000 7fe4b28e4640 1 --2- 172.21.15.12:0/160444336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4ac105e30 0x7fe4ac106210 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fe49c009930 tx=0x7fe49c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:50.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.217+0000 7fe4b28e4640 1 -- 172.21.15.12:0/160444336 shutdown_connections 2024-04-22T13:26:50.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.217+0000 7fe4b28e4640 1 --2- 172.21.15.12:0/160444336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4ac101b60 0x7fe4ac10dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.217+0000 7fe4b28e4640 1 --2- 172.21.15.12:0/160444336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4ac1011a0 0x7fe4ac101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.217+0000 7fe4b28e4640 1 --2- 172.21.15.12:0/160444336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4ac105e30 0x7fe4ac106210 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.217+0000 7fe4b28e4640 1 -- 172.21.15.12:0/160444336 >> 172.21.15.12:0/160444336 conn(0x7fe4ac077e30 msgr2=0x7fe4ac078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:50.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.217+0000 7fe4b28e4640 1 -- 172.21.15.12:0/160444336 shutdown_connections 2024-04-22T13:26:50.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.217+0000 7fe4b28e4640 1 -- 172.21.15.12:0/160444336 wait complete. 2024-04-22T13:26:50.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.217+0000 7fe4b28e4640 1 Processor -- start 2024-04-22T13:26:50.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.218+0000 7fe4b28e4640 1 -- start start 2024-04-22T13:26:50.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.218+0000 7fe4b28e4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4ac1011a0 0x7fe4ac1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:50.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.218+0000 7fe4b28e4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4ac101b60 0x7fe4ac198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:50.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.218+0000 7fe4b28e4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4ac105e30 0x7fe4ac19cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:50.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.218+0000 7fe4b28e4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac1110e0 con 0x7fe4ac1011a0 2024-04-22T13:26:50.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.218+0000 7fe4b28e4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac110f60 con 0x7fe4ac101b60 2024-04-22T13:26:50.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.218+0000 7fe4b28e4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac111260 con 0x7fe4ac105e30 2024-04-22T13:26:50.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.218+0000 7fe4abfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4ac1011a0 0x7fe4ac1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:50.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.218+0000 7fe4abfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4ac1011a0 0x7fe4ac1986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41206/0 (socket says 172.21.15.12:41206) 2024-04-22T13:26:50.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.218+0000 7fe4ab7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4ac101b60 0x7fe4ac198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:50.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.218+0000 7fe4abfff640 1 -- 172.21.15.12:0/3981960212 learned_addr learned my addr 172.21.15.12:0/3981960212 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:50.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.218+0000 7fe4b0e81640 1 --2- 172.21.15.12:0/3981960212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4ac105e30 0x7fe4ac19cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:50.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.218+0000 7fe4ab7fe640 1 -- 172.21.15.12:0/3981960212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4ac105e30 msgr2=0x7fe4ac19cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:50.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.218+0000 7fe4ab7fe640 1 --2- 172.21.15.12:0/3981960212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4ac105e30 0x7fe4ac19cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.218+0000 7fe4ab7fe640 1 -- 172.21.15.12:0/3981960212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4ac1011a0 msgr2=0x7fe4ac1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:50.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.218+0000 7fe4ab7fe640 1 --2- 172.21.15.12:0/3981960212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4ac1011a0 0x7fe4ac1986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.218+0000 7fe4ab7fe640 1 -- 172.21.15.12:0/3981960212 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4ac19d6b0 con 0x7fe4ac101b60 2024-04-22T13:26:50.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.218+0000 7fe4abfff640 1 --2- 172.21.15.12:0/3981960212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4ac1011a0 0x7fe4ac1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:50.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.218+0000 7fe4b0e81640 1 --2- 172.21.15.12:0/3981960212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4ac105e30 0x7fe4ac19cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:26:50.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.219+0000 7fe4ab7fe640 1 --2- 172.21.15.12:0/3981960212 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4ac101b60 0x7fe4ac198c00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe4a000e9f0 tx=0x7fe4a000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:50.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.219+0000 7fe4a97fa640 1 -- 172.21.15.12:0/3981960212 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a000ccc0 con 0x7fe4ac101b60 2024-04-22T13:26:50.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.219+0000 7fe4b28e4640 1 -- 172.21.15.12:0/3981960212 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4ac19d9a0 con 0x7fe4ac101b60 2024-04-22T13:26:50.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.219+0000 7fe4a97fa640 1 -- 172.21.15.12:0/3981960212 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4a00108e0 con 0x7fe4ac101b60 2024-04-22T13:26:50.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.219+0000 7fe4a97fa640 1 -- 172.21.15.12:0/3981960212 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0018880 con 0x7fe4ac101b60 2024-04-22T13:26:50.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.219+0000 7fe4b28e4640 1 -- 172.21.15.12:0/3981960212 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4ac1a5230 con 0x7fe4ac101b60 2024-04-22T13:26:50.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.220+0000 7fe4b28e4640 1 -- 172.21.15.12:0/3981960212 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4ac102570 con 0x7fe4ac101b60 2024-04-22T13:26:50.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.221+0000 7fe4a97fa640 1 -- 172.21.15.12:0/3981960212 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe4a0010430 con 0x7fe4ac101b60 2024-04-22T13:26:50.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.222+0000 7fe4a97fa640 1 --2- 172.21.15.12:0/3981960212 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe4940a3d20 0x7fe4940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:50.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.222+0000 7fe4abfff640 1 --2- 172.21.15.12:0/3981960212 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe4940a3d20 0x7fe4940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:50.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.222+0000 7fe4a97fa640 1 -- 172.21.15.12:0/3981960212 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fe4a00d0b40 con 0x7fe4ac101b60 2024-04-22T13:26:50.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.222+0000 7fe4abfff640 1 --2- 172.21.15.12:0/3981960212 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe4940a3d20 0x7fe4940a61e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe49c000c00 tx=0x7fe49c0058d0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:50.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.226+0000 7fe4a97fa640 1 -- 172.21.15.12:0/3981960212 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe4a009cbe0 con 0x7fe4ac101b60 2024-04-22T13:26:50.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.453+0000 7fe4b28e4640 1 -- 172.21.15.12:0/3981960212 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4ac068de0 con 0x7fe4ac101b60 2024-04-22T13:26:50.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.454+0000 7fe4a97fa640 1 -- 172.21.15.12:0/3981960212 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7fe4ac068de0 con 0x7fe4ac101b60 2024-04-22T13:26:50.450 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:26:50.450 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:26:50.450 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:26:50.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.456+0000 7fe4b28e4640 1 -- 172.21.15.12:0/3981960212 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe4940a3d20 msgr2=0x7fe4940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:50.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.456+0000 7fe4b28e4640 1 --2- 172.21.15.12:0/3981960212 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe4940a3d20 0x7fe4940a61e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe49c000c00 tx=0x7fe49c0058d0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.456+0000 7fe4b28e4640 1 -- 172.21.15.12:0/3981960212 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4ac101b60 msgr2=0x7fe4ac198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:50.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.456+0000 7fe4b28e4640 1 --2- 172.21.15.12:0/3981960212 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4ac101b60 0x7fe4ac198c00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe4a000e9f0 tx=0x7fe4a000eeb0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.456+0000 7fe4b28e4640 1 -- 172.21.15.12:0/3981960212 shutdown_connections 2024-04-22T13:26:50.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.456+0000 7fe4b28e4640 1 --2- 172.21.15.12:0/3981960212 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe4940a3d20 0x7fe4940a61e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.456+0000 7fe4b28e4640 1 --2- 172.21.15.12:0/3981960212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4ac105e30 0x7fe4ac19cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.456+0000 7fe4b28e4640 1 --2- 172.21.15.12:0/3981960212 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4ac101b60 0x7fe4ac198c00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.456+0000 7fe4b28e4640 1 --2- 172.21.15.12:0/3981960212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe4ac1011a0 0x7fe4ac1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.456+0000 7fe4b28e4640 1 -- 172.21.15.12:0/3981960212 >> 172.21.15.12:0/3981960212 conn(0x7fe4ac077e30 msgr2=0x7fe4ac069420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:50.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.456+0000 7fe4b28e4640 1 -- 172.21.15.12:0/3981960212 shutdown_connections 2024-04-22T13:26:50.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.456+0000 7fe4b28e4640 1 -- 172.21.15.12:0/3981960212 wait complete. 2024-04-22T13:26:50.463 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24509 quiesce db --cancel --all' 2024-04-22T13:26:50.464 DEBUG:teuthology.orchestra.run.smithi012:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24509 quiesce db --cancel --all 2024-04-22T13:26:50.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.575+0000 7f3abeba4640 1 -- 172.21.15.12:0/181724668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ab806a840 msgr2=0x7f3ab806ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:50.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.575+0000 7f3abeba4640 1 --2- 172.21.15.12:0/181724668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ab806a840 0x7f3ab806ac20 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8009900 tx=0x7f3aa802bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:50.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.575+0000 7f3abeba4640 1 -- 172.21.15.12:0/181724668 shutdown_connections 2024-04-22T13:26:50.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.575+0000 7f3abeba4640 1 --2- 172.21.15.12:0/181724668 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ab8107270 0x7f3ab810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.575+0000 7f3abeba4640 1 --2- 172.21.15.12:0/181724668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ab806b1f0 0x7f3ab8106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.575+0000 7f3abeba4640 1 --2- 172.21.15.12:0/181724668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ab806a840 0x7f3ab806ac20 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.575+0000 7f3abeba4640 1 -- 172.21.15.12:0/181724668 >> 172.21.15.12:0/181724668 conn(0x7f3ab8077eb0 msgr2=0x7f3ab80782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:50.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.575+0000 7f3abeba4640 1 -- 172.21.15.12:0/181724668 shutdown_connections 2024-04-22T13:26:50.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.575+0000 7f3abeba4640 1 -- 172.21.15.12:0/181724668 wait complete. 2024-04-22T13:26:50.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.576+0000 7f3abeba4640 1 Processor -- start 2024-04-22T13:26:50.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.576+0000 7f3abeba4640 1 -- start start 2024-04-22T13:26:50.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.576+0000 7f3abeba4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ab806a840 0x7f3ab8100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:50.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.576+0000 7f3abeba4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ab8107270 0x7f3ab81013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:50.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.576+0000 7f3abeba4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ab8101900 0x7f3ab81a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:50.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.576+0000 7f3abeba4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81121d0 con 0x7f3ab806a840 2024-04-22T13:26:50.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.576+0000 7f3abeba4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab8112050 con 0x7f3ab8107270 2024-04-22T13:26:50.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.576+0000 7f3abeba4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab8112350 con 0x7f3ab8101900 2024-04-22T13:26:50.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.576+0000 7f3abc940640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ab806a840 0x7f3ab8100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:50.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.576+0000 7f3abc940640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ab806a840 0x7f3ab8100e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41220/0 (socket says 172.21.15.12:41220) 2024-04-22T13:26:50.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.576+0000 7f3abc940640 1 -- 172.21.15.12:0/1581516351 learned_addr learned my addr 172.21.15.12:0/1581516351 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:50.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.576+0000 7f3abd141640 1 --2- 172.21.15.12:0/1581516351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ab8101900 0x7f3ab81a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:50.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.576+0000 7f3ab7fff640 1 --2- 172.21.15.12:0/1581516351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ab8107270 0x7f3ab81013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:50.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.576+0000 7f3abc940640 1 -- 172.21.15.12:0/1581516351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ab8101900 msgr2=0x7f3ab81a68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:50.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.576+0000 7f3abc940640 1 --2- 172.21.15.12:0/1581516351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ab8101900 0x7f3ab81a68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.576+0000 7f3abc940640 1 -- 172.21.15.12:0/1581516351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ab8107270 msgr2=0x7f3ab81013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:50.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.576+0000 7f3abc940640 1 --2- 172.21.15.12:0/1581516351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ab8107270 0x7f3ab81013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.577+0000 7f3abc940640 1 -- 172.21.15.12:0/1581516351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab81a6fe0 con 0x7f3ab806a840 2024-04-22T13:26:50.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.577+0000 7f3ab7fff640 1 --2- 172.21.15.12:0/1581516351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ab8107270 0x7f3ab81013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:26:50.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.577+0000 7f3abc940640 1 --2- 172.21.15.12:0/1581516351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ab806a840 0x7f3ab8100e80 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8005940 tx=0x7f3aa802c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:50.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.577+0000 7f3ab5ffb640 1 -- 172.21.15.12:0/1581516351 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa8043020 con 0x7f3ab806a840 2024-04-22T13:26:50.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.577+0000 7f3ab5ffb640 1 -- 172.21.15.12:0/1581516351 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aa802ca70 con 0x7f3ab806a840 2024-04-22T13:26:50.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.577+0000 7f3abeba4640 1 -- 172.21.15.12:0/1581516351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab81a7270 con 0x7f3ab806a840 2024-04-22T13:26:50.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.577+0000 7f3ab5ffb640 1 -- 172.21.15.12:0/1581516351 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa80039f0 con 0x7f3ab806a840 2024-04-22T13:26:50.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.577+0000 7f3abeba4640 1 -- 172.21.15.12:0/1581516351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab81a77c0 con 0x7f3ab806a840 2024-04-22T13:26:50.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.579+0000 7f3ab5ffb640 1 -- 172.21.15.12:0/1581516351 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3aa8046020 con 0x7f3ab806a840 2024-04-22T13:26:50.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.580+0000 7f3ac1794740 1 Processor -- start 2024-04-22T13:26:50.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.580+0000 7f3ac1794740 1 -- start start 2024-04-22T13:26:50.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.580+0000 7f3ab5ffb640 1 --2- 172.21.15.12:0/1581516351 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3aa00a3d50 0x7f3aa00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:50.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.580+0000 7f3ac1794740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559125efed10 0x559125eff0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:50.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.580+0000 7f3ac1794740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559125eff6c0 0x559125f07c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:50.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.580+0000 7f3ac1794740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559125f08240 0x559125f0a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:50.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.580+0000 7f3ac1794740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559125e64000 con 0x559125eff6c0 2024-04-22T13:26:50.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.580+0000 7f3ac1794740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559125e00b20 con 0x559125f08240 2024-04-22T13:26:50.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.580+0000 7f3ac1794740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559125e5ffc0 con 0x559125efed10 2024-04-22T13:26:50.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.580+0000 7f3abd141640 1 --2- 172.21.15.12:0/1581516351 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3aa00a3d50 0x7f3aa00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:50.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.580+0000 7f3ab7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559125eff6c0 0x559125f07c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:50.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.580+0000 7f3ab5ffb640 1 -- 172.21.15.12:0/1581516351 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f3aa80f2750 con 0x7f3ab806a840 2024-04-22T13:26:50.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.580+0000 7f3ab7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559125eff6c0 0x559125f07c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41234/0 (socket says 172.21.15.12:41234) 2024-04-22T13:26:50.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.580+0000 7f3ab7fff640 1 -- 172.21.15.12:0/543300961 learned_addr learned my addr 172.21.15.12:0/543300961 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:50.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.580+0000 7f3abc940640 1 --2- 172.21.15.12:0/543300961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559125efed10 0x559125eff0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:50.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.580+0000 7f3abd141640 1 --2- 172.21.15.12:0/543300961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559125f08240 0x559125f0a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:50.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.580+0000 7f3ab7fff640 1 -- 172.21.15.12:0/543300961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559125efed10 msgr2=0x559125eff0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:50.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.580+0000 7f3ab7fff640 1 --2- 172.21.15.12:0/543300961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559125efed10 0x559125eff0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.580+0000 7f3ab7fff640 1 -- 172.21.15.12:0/543300961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559125f08240 msgr2=0x559125f0a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:50.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.580+0000 7f3ab7fff640 1 --2- 172.21.15.12:0/543300961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559125f08240 0x559125f0a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.580+0000 7f3ab7fff640 1 -- 172.21.15.12:0/543300961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559125e1aea0 con 0x559125eff6c0 2024-04-22T13:26:50.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.581+0000 7f3abd141640 1 --2- 172.21.15.12:0/1581516351 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3aa00a3d50 0x7f3aa00a6210 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0009fd0 tx=0x7f3ab0013b00 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:50.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.581+0000 7f3ab7fff640 1 --2- 172.21.15.12:0/543300961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559125eff6c0 0x559125f07c70 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009cb0 tx=0x7f3aac0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b88a912040545f61 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:50.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.581+0000 7f3a9effd640 1 -- 172.21.15.12:0/543300961 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac012450 con 0x559125eff6c0 2024-04-22T13:26:50.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.581+0000 7f3a9effd640 1 -- 172.21.15.12:0/543300961 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aac012aa0 con 0x559125eff6c0 2024-04-22T13:26:50.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.581+0000 7f3a9effd640 1 -- 172.21.15.12:0/543300961 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac00fb90 con 0x559125eff6c0 2024-04-22T13:26:50.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.581+0000 7f3ac1794740 1 -- 172.21.15.12:0/543300961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559125eff6c0 msgr2=0x559125f07c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:50.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.581+0000 7f3ac1794740 1 --2- 172.21.15.12:0/543300961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559125eff6c0 0x559125f07c70 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009cb0 tx=0x7f3aac0117b0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.581+0000 7f3ac1794740 1 -- 172.21.15.12:0/543300961 shutdown_connections 2024-04-22T13:26:50.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.581+0000 7f3ac1794740 1 --2- 172.21.15.12:0/543300961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559125f08240 0x559125f0a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.581+0000 7f3ac1794740 1 --2- 172.21.15.12:0/543300961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559125eff6c0 0x559125f07c70 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.581+0000 7f3ac1794740 1 --2- 172.21.15.12:0/543300961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559125efed10 0x559125eff0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.581+0000 7f3ac1794740 1 -- 172.21.15.12:0/543300961 >> 172.21.15.12:0/543300961 conn(0x559125ef93e0 msgr2=0x559125ef97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:50.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.582+0000 7f3ac1794740 1 -- 172.21.15.12:0/543300961 shutdown_connections 2024-04-22T13:26:50.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.582+0000 7f3ac1794740 1 -- 172.21.15.12:0/543300961 wait complete. 2024-04-22T13:26:50.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.582+0000 7f3ac1794740 1 -- start start 2024-04-22T13:26:50.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.582+0000 7f3ac1794740 1 Processor -- start 2024-04-22T13:26:50.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.582+0000 7f3ac1794740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559125efed10 0x559125f05170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:50.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.582+0000 7f3ac1794740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559125eff6c0 0x559125f056b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:50.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.582+0000 7f3ac1794740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559125f08240 0x559125f05bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:50.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.582+0000 7f3ac1794740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559125f0b0c0 con 0x559125f08240 2024-04-22T13:26:50.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.582+0000 7f3ac1794740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559125dfe510 con 0x559125eff6c0 2024-04-22T13:26:50.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.582+0000 7f3ac1794740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559125f0b240 con 0x559125efed10 2024-04-22T13:26:50.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.582+0000 7f3ac1794740 10 client.0 _ll_register_callbacks cb 0x559125dfdf00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:50.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.582+0000 7f3abd141640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559125f08240 0x559125f05bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:50.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.582+0000 7f3abd141640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559125f08240 0x559125f05bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41246/0 (socket says 172.21.15.12:41246) 2024-04-22T13:26:50.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.582+0000 7f3abd141640 1 -- 172.21.15.12:0/373802909 learned_addr learned my addr 172.21.15.12:0/373802909 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:50.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.582+0000 7f3abc940640 1 --2- 172.21.15.12:0/373802909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559125efed10 0x559125f05170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:50.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.583+0000 7f3ab7fff640 1 --2- 172.21.15.12:0/373802909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559125eff6c0 0x559125f056b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:50.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.583+0000 7f3abd141640 1 -- 172.21.15.12:0/373802909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559125efed10 msgr2=0x559125f05170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:50.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.583+0000 7f3abd141640 1 --2- 172.21.15.12:0/373802909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559125efed10 0x559125f05170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.583+0000 7f3abd141640 1 -- 172.21.15.12:0/373802909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559125eff6c0 msgr2=0x559125f056b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:50.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.583+0000 7f3abd141640 1 --2- 172.21.15.12:0/373802909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559125eff6c0 0x559125f056b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.583+0000 7f3abd141640 1 -- 172.21.15.12:0/373802909 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559125c42a20 con 0x559125f08240 2024-04-22T13:26:50.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.583+0000 7f3ab7fff640 1 --2- 172.21.15.12:0/373802909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559125eff6c0 0x559125f056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:50.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.583+0000 7f3abd141640 1 --2- 172.21.15.12:0/373802909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559125f08240 0x559125f05bf0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0003df0 tx=0x7f3ab0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e948ba61a72df5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:50.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.583+0000 7f3a97fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:26:50.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.583+0000 7f3a97fff640 1 -- 172.21.15.12:0/373802909 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0024020 con 0x559125f08240 2024-04-22T13:26:50.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.583+0000 7f3a97fff640 1 -- 172.21.15.12:0/373802909 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f3ab0004300 con 0x559125f08240 2024-04-22T13:26:50.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.583+0000 7f3ac1794740 1 -- 172.21.15.12:0/373802909 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aac012aa0 con 0x559125f08240 2024-04-22T13:26:50.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.583+0000 7f3a97fff640 1 -- 172.21.15.12:0/373802909 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3ab002f050 con 0x559125f08240 2024-04-22T13:26:50.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.583+0000 7f3ac1794740 10 client.16719 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:50.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.583+0000 7f3ac1794740 1 -- 172.21.15.12:0/373802909 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559125ca31b0 con 0x559125f08240 2024-04-22T13:26:50.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.584+0000 7f3a97fff640 1 -- 172.21.15.12:0/373802909 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aac012aa0 con 0x559125f08240 2024-04-22T13:26:50.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.584+0000 7f3a97fff640 1 -- 172.21.15.12:0/373802909 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3ab001f070 con 0x559125f08240 2024-04-22T13:26:50.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.584+0000 7f3ac1794740 10 client.16719 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:50.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.584+0000 7f3ac1794740 10 client.16719 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-22T13:26:50.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.584+0000 7f3ac1794740 20 client.16719 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:50.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.584+0000 7f3ac1794740 1 --2- 172.21.15.12:0/373802909 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559125f187b0 0x559125f1ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:50.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.584+0000 7f3ac1794740 4 client.16719 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:26:50.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.584+0000 7f3ac1794740 1 -- 172.21.15.12:0/373802909 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559125e71a20 con 0x559125f187b0 2024-04-22T13:26:50.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.584+0000 7f3abc940640 1 --2- 172.21.15.12:0/373802909 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559125f187b0 0x559125f1ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:50.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.584+0000 7f3abc940640 1 --2- 172.21.15.12:0/373802909 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559125f187b0 0x559125f1ac00 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d054052036ec416e server_cookie=5759ca403274d858 in_seq=0 out_seq=0 2024-04-22T13:26:50.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.584+0000 7f3a97fff640 10 client.16719 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:50.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.585+0000 7f3a97fff640 1 -- 172.21.15.12:0/373802909 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559125e71a20 con 0x559125f187b0 2024-04-22T13:26:50.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.585+0000 7f3a97fff640 10 client.16719 handle_command_reply: tid=0 2024-04-22T13:26:50.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.585+0000 7f3ac1794740 1 client.16719 shutdown 2024-04-22T13:26:50.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.585+0000 7f3ac1794740 1 -- 172.21.15.12:0/373802909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559125f08240 msgr2=0x559125f05bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:50.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.585+0000 7f3ac1794740 1 --2- 172.21.15.12:0/373802909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559125f08240 0x559125f05bf0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0003df0 tx=0x7f3ab0004ab0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.585+0000 7f3ac1794740 1 -- 172.21.15.12:0/373802909 shutdown_connections 2024-04-22T13:26:50.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.585+0000 7f3ac1794740 1 --2- 172.21.15.12:0/373802909 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559125f187b0 0x559125f1ac00 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.585+0000 7f3ac1794740 1 --2- 172.21.15.12:0/373802909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559125f08240 0x559125f05bf0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.585+0000 7f3ac1794740 1 --2- 172.21.15.12:0/373802909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559125eff6c0 0x559125f056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.585+0000 7f3a97fff640 0 client.16719 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:50.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.585+0000 7f3ac1794740 1 --2- 172.21.15.12:0/373802909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559125efed10 0x559125f05170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.585+0000 7f3ac1794740 1 -- 172.21.15.12:0/373802909 >> 172.21.15.12:0/373802909 conn(0x559125ef93e0 msgr2=0x559125ef8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:50.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.585+0000 7f3ac1794740 1 -- 172.21.15.12:0/373802909 shutdown_connections 2024-04-22T13:26:50.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.585+0000 7f3ac1794740 1 -- 172.21.15.12:0/373802909 wait complete. 2024-04-22T13:26:50.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.585+0000 7f3ac1794740 20 client.16719 trim_cache size 0 max 16384 2024-04-22T13:26:50.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.603+0000 7f3ac1794740 1 Processor -- start 2024-04-22T13:26:50.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.603+0000 7f3ac1794740 1 -- start start 2024-04-22T13:26:50.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.603+0000 7f3ac1794740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559125efed10 0x559125f140e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:50.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.603+0000 7f3ac1794740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559125eff6c0 0x559125f05840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:50.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.603+0000 7f3ac1794740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559125f08240 0x559125f204d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:50.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.603+0000 7f3ac1794740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81afb60 con 0x559125f08240 2024-04-22T13:26:50.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.603+0000 7f3ac1794740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81b16a0 con 0x559125efed10 2024-04-22T13:26:50.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.603+0000 7f3ac1794740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81b0230 con 0x559125eff6c0 2024-04-22T13:26:50.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.603+0000 7f3abd141640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559125f08240 0x559125f204d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:50.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.603+0000 7f3abd141640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559125f08240 0x559125f204d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41248/0 (socket says 172.21.15.12:41248) 2024-04-22T13:26:50.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.603+0000 7f3abd141640 1 -- 172.21.15.12:0/1515739412 learned_addr learned my addr 172.21.15.12:0/1515739412 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:50.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.603+0000 7f3ab7fff640 1 --2- 172.21.15.12:0/1515739412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559125eff6c0 0x559125f05840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:50.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.603+0000 7f3abc940640 1 --2- 172.21.15.12:0/1515739412 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559125efed10 0x559125f140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:50.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.603+0000 7f3abd141640 1 -- 172.21.15.12:0/1515739412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559125eff6c0 msgr2=0x559125f05840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:50.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.603+0000 7f3abd141640 1 --2- 172.21.15.12:0/1515739412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559125eff6c0 0x559125f05840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.603+0000 7f3abd141640 1 -- 172.21.15.12:0/1515739412 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559125efed10 msgr2=0x559125f140e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:50.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.603+0000 7f3abd141640 1 --2- 172.21.15.12:0/1515739412 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559125efed10 0x559125f140e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.603+0000 7f3abd141640 1 -- 172.21.15.12:0/1515739412 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559125b36310 con 0x559125f08240 2024-04-22T13:26:50.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.603+0000 7f3abd141640 1 --2- 172.21.15.12:0/1515739412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559125f08240 0x559125f204d0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f3ab000f430 tx=0x7f3ab0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24fbe1cf955d1db7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:50.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.603+0000 7f3a9e7fc640 1 -- 172.21.15.12:0/1515739412 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0038070 con 0x559125f08240 2024-04-22T13:26:50.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.603+0000 7f3a9e7fc640 1 -- 172.21.15.12:0/1515739412 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ab0004d20 con 0x559125f08240 2024-04-22T13:26:50.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.603+0000 7f3a9e7fc640 1 -- 172.21.15.12:0/1515739412 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0024070 con 0x559125f08240 2024-04-22T13:26:50.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.604+0000 7f3ac1794740 1 -- 172.21.15.12:0/1515739412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559125f08240 msgr2=0x559125f204d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:50.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.604+0000 7f3ac1794740 1 --2- 172.21.15.12:0/1515739412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559125f08240 0x559125f204d0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f3ab000f430 tx=0x7f3ab0003970 comp rx=0 tx=0).stop 2024-04-22T13:26:50.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.604+0000 7f3ac1794740 1 -- 172.21.15.12:0/1515739412 shutdown_connections 2024-04-22T13:26:50.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.604+0000 7f3ac1794740 1 --2- 172.21.15.12:0/1515739412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559125f08240 0x559125f204d0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.604+0000 7f3ac1794740 1 --2- 172.21.15.12:0/1515739412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559125eff6c0 0x559125f05840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.604+0000 7f3ac1794740 1 --2- 172.21.15.12:0/1515739412 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559125efed10 0x559125f140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.604+0000 7f3ac1794740 1 -- 172.21.15.12:0/1515739412 >> 172.21.15.12:0/1515739412 conn(0x559125ef93e0 msgr2=0x559125f05130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:50.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.604+0000 7f3ac1794740 1 -- 172.21.15.12:0/1515739412 shutdown_connections 2024-04-22T13:26:50.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.604+0000 7f3ac1794740 1 -- 172.21.15.12:0/1515739412 wait complete. 2024-04-22T13:26:50.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.604+0000 7f3ac1794740 1 -- start start 2024-04-22T13:26:50.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.604+0000 7f3ac1794740 1 Processor -- start 2024-04-22T13:26:50.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.605+0000 7f3ac1794740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559125efed10 0x559125f0c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:50.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.605+0000 7f3ac1794740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559125eff6c0 0x559125f0ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:50.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.605+0000 7f3ac1794740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559125f08240 0x559125f7dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:50.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.605+0000 7f3ac1794740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81b0fd0 con 0x559125eff6c0 2024-04-22T13:26:50.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.605+0000 7f3ac1794740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81b0900 con 0x559125efed10 2024-04-22T13:26:50.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.605+0000 7f3ac1794740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab81adb60 con 0x559125f08240 2024-04-22T13:26:50.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.605+0000 7f3ac1794740 10 client.0 _ll_register_callbacks cb 0x7f3ab0003a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:50.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.605+0000 7f3ab7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559125eff6c0 0x559125f0ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:50.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.605+0000 7f3ab7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559125eff6c0 0x559125f0ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:41262/0 (socket says 172.21.15.12:41262) 2024-04-22T13:26:50.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.605+0000 7f3ab7fff640 1 -- 172.21.15.12:0/1028071204 learned_addr learned my addr 172.21.15.12:0/1028071204 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:50.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.605+0000 7f3abc940640 1 --2- 172.21.15.12:0/1028071204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559125efed10 0x559125f0c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:50.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.605+0000 7f3abd141640 1 --2- 172.21.15.12:0/1028071204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559125f08240 0x559125f7dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:50.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.605+0000 7f3ab7fff640 1 -- 172.21.15.12:0/1028071204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559125f08240 msgr2=0x559125f7dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:50.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.605+0000 7f3ab7fff640 1 --2- 172.21.15.12:0/1028071204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559125f08240 0x559125f7dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.605+0000 7f3ab7fff640 1 -- 172.21.15.12:0/1028071204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559125efed10 msgr2=0x559125f0c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:50.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.605+0000 7f3ab7fff640 1 --2- 172.21.15.12:0/1028071204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559125efed10 0x559125f0c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.605+0000 7f3ab7fff640 1 -- 172.21.15.12:0/1028071204 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559125f7e2b0 con 0x559125eff6c0 2024-04-22T13:26:50.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.605+0000 7f3abc940640 1 --2- 172.21.15.12:0/1028071204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559125efed10 0x559125f0c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:50.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.606+0000 7f3ab7fff640 1 --2- 172.21.15.12:0/1028071204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559125eff6c0 0x559125f0ccb0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009cb0 tx=0x7f3aac012750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42eae4c38af77640 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:50.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.606+0000 7f3a97fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:26:50.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.606+0000 7f3a97fff640 1 -- 172.21.15.12:0/1028071204 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac005040 con 0x559125eff6c0 2024-04-22T13:26:50.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.606+0000 7f3a97fff640 1 -- 172.21.15.12:0/1028071204 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f3aac0051e0 con 0x559125eff6c0 2024-04-22T13:26:50.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.606+0000 7f3ac1794740 1 -- 172.21.15.12:0/1028071204 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ab0004d20 con 0x559125eff6c0 2024-04-22T13:26:50.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.606+0000 7f3a97fff640 1 -- 172.21.15.12:0/1028071204 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3aac033020 con 0x559125eff6c0 2024-04-22T13:26:50.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.606+0000 7f3ac1794740 10 client.16725 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:50.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.606+0000 7f3ac1794740 1 -- 172.21.15.12:0/1028071204 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559125f7f600 con 0x559125eff6c0 2024-04-22T13:26:50.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.606+0000 7f3a97fff640 1 -- 172.21.15.12:0/1028071204 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ab0004d20 con 0x559125eff6c0 2024-04-22T13:26:50.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.606+0000 7f3a97fff640 1 -- 172.21.15.12:0/1028071204 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3aac0314c0 con 0x559125eff6c0 2024-04-22T13:26:50.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.606+0000 7f3ac1794740 10 client.16725 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:50.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.606+0000 7f3ac1794740 10 client.16725 resolve_mds: validated gid 24509 aka daemon mds.b 2024-04-22T13:26:50.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.606+0000 7f3ac1794740 20 client.16725 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:50.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.606+0000 7f3ac1794740 1 --2- 172.21.15.12:0/1028071204 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559125f187b0 0x559125f81aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:50.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.606+0000 7f3ac1794740 4 client.16725 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-04-22T13:26:50.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.606+0000 7f3ac1794740 1 -- 172.21.15.12:0/1028071204 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x559125e6b890 con 0x559125f187b0 2024-04-22T13:26:50.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.607+0000 7f3abc940640 1 --2- 172.21.15.12:0/1028071204 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559125f187b0 0x559125f81aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:50.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.607+0000 7f3abc940640 1 --2- 172.21.15.12:0/1028071204 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559125f187b0 0x559125f81aa0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6681100a2c857247 server_cookie=a68b505f6e68d221 in_seq=0 out_seq=0 2024-04-22T13:26:50.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.607+0000 7f3a97fff640 10 client.16725 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:50.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.607+0000 7f3a97fff640 1 -- 172.21.15.12:0/1028071204 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+79 (crc 0 0 0) 0x559125e6b890 con 0x559125f187b0 2024-04-22T13:26:50.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.607+0000 7f3a97fff640 10 client.16725 handle_command_reply: tid=0 2024-04-22T13:26:50.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.607+0000 7f3ac1794740 1 client.16725 shutdown 2024-04-22T13:26:50.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.607+0000 7f3ac1794740 1 -- 172.21.15.12:0/1028071204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559125eff6c0 msgr2=0x559125f0ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:50.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.607+0000 7f3ac1794740 1 --2- 172.21.15.12:0/1028071204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559125eff6c0 0x559125f0ccb0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009cb0 tx=0x7f3aac012750 comp rx=0 tx=0).stop 2024-04-22T13:26:50.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.607+0000 7f3ac1794740 1 -- 172.21.15.12:0/1028071204 shutdown_connections 2024-04-22T13:26:50.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.607+0000 7f3ac1794740 1 --2- 172.21.15.12:0/1028071204 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559125f187b0 0x559125f81aa0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.607+0000 7f3ac1794740 1 --2- 172.21.15.12:0/1028071204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559125f08240 0x559125f7dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.607+0000 7f3ac1794740 1 --2- 172.21.15.12:0/1028071204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559125eff6c0 0x559125f0ccb0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.607+0000 7f3a97fff640 0 client.16725 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:50.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.607+0000 7f3ac1794740 1 --2- 172.21.15.12:0/1028071204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559125efed10 0x559125f0c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.607+0000 7f3ac1794740 1 -- 172.21.15.12:0/1028071204 >> 172.21.15.12:0/1028071204 conn(0x559125ef93e0 msgr2=0x559125f059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:50.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.607+0000 7f3ac1794740 1 -- 172.21.15.12:0/1028071204 shutdown_connections 2024-04-22T13:26:50.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.607+0000 7f3ac1794740 1 -- 172.21.15.12:0/1028071204 wait complete. 2024-04-22T13:26:50.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.607+0000 7f3ac1794740 20 client.16725 trim_cache size 0 max 16384 2024-04-22T13:26:50.609 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:26:50.609 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 28, 2024-04-22T13:26:50.609 INFO:teuthology.orchestra.run.smithi012.stdout: "leader": 24509, 2024-04-22T13:26:50.609 INFO:teuthology.orchestra.run.smithi012.stdout: "set_version": 8, 2024-04-22T13:26:50.609 INFO:teuthology.orchestra.run.smithi012.stdout: "sets": {} 2024-04-22T13:26:50.609 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:26:50.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.608+0000 7f3abeba4640 1 -- 172.21.15.12:0/1581516351 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3aa00a3d50 msgr2=0x7f3aa00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:50.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.608+0000 7f3abeba4640 1 --2- 172.21.15.12:0/1581516351 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3aa00a3d50 0x7f3aa00a6210 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0009fd0 tx=0x7f3ab0013b00 comp rx=0 tx=0).stop 2024-04-22T13:26:50.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.608+0000 7f3abeba4640 1 -- 172.21.15.12:0/1581516351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ab806a840 msgr2=0x7f3ab8100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:50.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.608+0000 7f3abeba4640 1 --2- 172.21.15.12:0/1581516351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ab806a840 0x7f3ab8100e80 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8005940 tx=0x7f3aa802c5f0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.608+0000 7f3abeba4640 1 -- 172.21.15.12:0/1581516351 shutdown_connections 2024-04-22T13:26:50.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.608+0000 7f3abeba4640 1 --2- 172.21.15.12:0/1581516351 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3aa00a3d50 0x7f3aa00a6210 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.608+0000 7f3abeba4640 1 --2- 172.21.15.12:0/1581516351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ab8101900 0x7f3ab81a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.608+0000 7f3abeba4640 1 --2- 172.21.15.12:0/1581516351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ab8107270 0x7f3ab81013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.608+0000 7f3abeba4640 1 --2- 172.21.15.12:0/1581516351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3ab806a840 0x7f3ab8100e80 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:50.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.608+0000 7f3abeba4640 1 -- 172.21.15.12:0/1581516351 >> 172.21.15.12:0/1581516351 conn(0x7f3ab8077eb0 msgr2=0x7f3ab8108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:50.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.608+0000 7f3abeba4640 1 -- 172.21.15.12:0/1581516351 shutdown_connections 2024-04-22T13:26:50.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:50.608+0000 7f3abeba4640 1 -- 172.21.15.12:0/1581516351 wait complete. 2024-04-22T13:26:50.614 INFO:tasks.quiescer:done joining Quiescers 2024-04-22T13:26:50.615 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-22T13:26:50.624 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi012.front.sepia.ceph.com 2024-04-22T13:26:50.632 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-22T13:26:50.642 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-22T13:26:50.642 INFO:teuthology.task.exec:Executing custom commands... 2024-04-22T13:26:50.643 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi012.front.sepia.ceph.com 2024-04-22T13:26:50.643 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-22T13:26:50.698 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-22T13:26:51.368 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:51 smithi012 ceph-mon[28265]: pgmap v85: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 867 KiB/s wr, 13 op/s 2024-04-22T13:26:51.368 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:51 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3981960212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:26:51.528 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:51 smithi111 ceph-mon[31361]: pgmap v85: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 867 KiB/s wr, 13 op/s 2024-04-22T13:26:51.529 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:51 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/3981960212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:26:51.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:51 smithi134 ceph-mon[31338]: pgmap v85: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 867 KiB/s wr, 13 op/s 2024-04-22T13:26:51.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:51 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/3981960212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:26:53.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:53 smithi012 ceph-mon[28265]: pgmap v86: 97 pgs: 97 active+clean; 464 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 22 op/s 2024-04-22T13:26:53.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:53 smithi134 ceph-mon[31338]: pgmap v86: 97 pgs: 97 active+clean; 464 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 22 op/s 2024-04-22T13:26:53.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:53 smithi111 ceph-mon[31361]: pgmap v86: 97 pgs: 97 active+clean; 464 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 16 MiB/s wr, 22 op/s 2024-04-22T13:26:54.462 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:26:54.463 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:26:54.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.581+0000 7f588e63a640 1 -- 172.21.15.12:0/249473585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5888102180 msgr2=0x7f5888102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:54.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.581+0000 7f588e63a640 1 --2- 172.21.15.12:0/249473585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5888102180 0x7f5888102560 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f5878009900 tx=0x7f587802bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:54.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.581+0000 7f588e63a640 1 -- 172.21.15.12:0/249473585 shutdown_connections 2024-04-22T13:26:54.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.581+0000 7f588e63a640 1 --2- 172.21.15.12:0/249473585 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5888107250 0x7f588810ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.581+0000 7f588e63a640 1 --2- 172.21.15.12:0/249473585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5888102b30 0x7f5888106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.581+0000 7f588e63a640 1 --2- 172.21.15.12:0/249473585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5888102180 0x7f5888102560 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.581+0000 7f588e63a640 1 -- 172.21.15.12:0/249473585 >> 172.21.15.12:0/249473585 conn(0x7f58880fda20 msgr2=0x7f58880ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:54.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.581+0000 7f588e63a640 1 -- 172.21.15.12:0/249473585 shutdown_connections 2024-04-22T13:26:54.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.582+0000 7f588e63a640 1 -- 172.21.15.12:0/249473585 wait complete. 2024-04-22T13:26:54.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.582+0000 7f588e63a640 1 Processor -- start 2024-04-22T13:26:54.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.582+0000 7f588e63a640 1 -- start start 2024-04-22T13:26:54.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.582+0000 7f588e63a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5888102180 0x7f5888074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:54.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.582+0000 7f588e63a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5888107250 0x7f5888074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:54.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.582+0000 7f588e63a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5888074cd0 0x7f58881a6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:54.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.582+0000 7f588e63a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f58881121b0 con 0x7f5888074cd0 2024-04-22T13:26:54.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.582+0000 7f588e63a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5888112030 con 0x7f5888107250 2024-04-22T13:26:54.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.582+0000 7f588e63a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5888112330 con 0x7f5888102180 2024-04-22T13:26:54.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.583+0000 7f588cbd7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5888074cd0 0x7f58881a6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:54.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.583+0000 7f588cbd7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5888074cd0 0x7f58881a6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35968/0 (socket says 172.21.15.12:35968) 2024-04-22T13:26:54.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.583+0000 7f588cbd7640 1 -- 172.21.15.12:0/2179550671 learned_addr learned my addr 172.21.15.12:0/2179550671 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:54.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.583+0000 7f58877fe640 1 --2- 172.21.15.12:0/2179550671 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5888107250 0x7f5888074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:54.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.583+0000 7f5887fff640 1 --2- 172.21.15.12:0/2179550671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5888102180 0x7f5888074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:54.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.583+0000 7f588cbd7640 1 -- 172.21.15.12:0/2179550671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5888102180 msgr2=0x7f5888074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:54.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.583+0000 7f588cbd7640 1 --2- 172.21.15.12:0/2179550671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5888102180 0x7f5888074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.583+0000 7f588cbd7640 1 -- 172.21.15.12:0/2179550671 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5888107250 msgr2=0x7f5888074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:54.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.583+0000 7f588cbd7640 1 --2- 172.21.15.12:0/2179550671 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5888107250 0x7f5888074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.583+0000 7f588cbd7640 1 -- 172.21.15.12:0/2179550671 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58881a70c0 con 0x7f5888074cd0 2024-04-22T13:26:54.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.583+0000 7f588cbd7640 1 --2- 172.21.15.12:0/2179550671 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5888074cd0 0x7f58881a6a20 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f588000eb10 tx=0x7f588000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:54.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.583+0000 7f58857fa640 1 -- 172.21.15.12:0/2179550671 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5880004b20 con 0x7f5888074cd0 2024-04-22T13:26:54.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.583+0000 7f588e63a640 1 -- 172.21.15.12:0/2179550671 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58881a73b0 con 0x7f5888074cd0 2024-04-22T13:26:54.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.583+0000 7f58857fa640 1 -- 172.21.15.12:0/2179550671 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5880004cc0 con 0x7f5888074cd0 2024-04-22T13:26:54.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.583+0000 7f58857fa640 1 -- 172.21.15.12:0/2179550671 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5880010670 con 0x7f5888074cd0 2024-04-22T13:26:54.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.583+0000 7f588e63a640 1 -- 172.21.15.12:0/2179550671 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58881a78b0 con 0x7f5888074cd0 2024-04-22T13:26:54.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.585+0000 7f58857fa640 1 -- 172.21.15.12:0/2179550671 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5880005020 con 0x7f5888074cd0 2024-04-22T13:26:54.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.586+0000 7f589121f740 1 Processor -- start 2024-04-22T13:26:54.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.586+0000 7f589121f740 1 -- start start 2024-04-22T13:26:54.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.586+0000 7f58857fa640 1 --2- 172.21.15.12:0/2179550671 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f58740a3d20 0x7f58740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:54.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.586+0000 7f589121f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e6f275580 0x558e6f275960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:54.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.586+0000 7f589121f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e6f275ea0 0x558e6f27e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:54.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.586+0000 7f589121f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6f27ea80 0x558e6f280e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:54.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.586+0000 7f589121f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558e6f1da1e0 con 0x558e6f275580 2024-04-22T13:26:54.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.586+0000 7f589121f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x558e6f177390 con 0x558e6f275ea0 2024-04-22T13:26:54.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.586+0000 7f589121f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558e6f1d61a0 con 0x558e6f27ea80 2024-04-22T13:26:54.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.586+0000 7f58857fa640 1 -- 172.21.15.12:0/2179550671 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f58800d0d10 con 0x7f5888074cd0 2024-04-22T13:26:54.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.586+0000 7f58877fe640 1 --2- 172.21.15.12:0/2179550671 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f58740a3d20 0x7f58740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:54.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.586+0000 7f5887fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e6f275580 0x558e6f275960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:54.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.586+0000 7f5887fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e6f275580 0x558e6f275960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35980/0 (socket says 172.21.15.12:35980) 2024-04-22T13:26:54.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.586+0000 7f5887fff640 1 -- 172.21.15.12:0/3448690956 learned_addr learned my addr 172.21.15.12:0/3448690956 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:54.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.586+0000 7f588cbd7640 1 --2- 172.21.15.12:0/3448690956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6f27ea80 0x558e6f280e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:54.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.586+0000 7f58877fe640 1 --2- 172.21.15.12:0/3448690956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e6f275ea0 0x558e6f27e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:54.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.587+0000 7f5887fff640 1 -- 172.21.15.12:0/3448690956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6f27ea80 msgr2=0x558e6f280e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:54.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.587+0000 7f5887fff640 1 --2- 172.21.15.12:0/3448690956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6f27ea80 0x558e6f280e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.587+0000 7f5887fff640 1 -- 172.21.15.12:0/3448690956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e6f275ea0 msgr2=0x558e6f27e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:54.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.587+0000 7f5887fff640 1 --2- 172.21.15.12:0/3448690956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e6f275ea0 0x558e6f27e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.587+0000 7f5887fff640 1 -- 172.21.15.12:0/3448690956 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e6efb67d0 con 0x558e6f275580 2024-04-22T13:26:54.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.587+0000 7f58877fe640 1 --2- 172.21.15.12:0/2179550671 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f58740a3d20 0x7f58740a61e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5870009710 tx=0x7f5870013880 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:54.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.587+0000 7f5887fff640 1 --2- 172.21.15.12:0/3448690956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e6f275580 0x558e6f275960 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f5878009900 tx=0x7f58780095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4853a64836463999 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:54.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.587+0000 7f587e7fc640 1 -- 172.21.15.12:0/3448690956 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5878045020 con 0x558e6f275580 2024-04-22T13:26:54.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.587+0000 7f587e7fc640 1 -- 172.21.15.12:0/3448690956 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f587802c7e0 con 0x558e6f275580 2024-04-22T13:26:54.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.587+0000 7f587e7fc640 1 -- 172.21.15.12:0/3448690956 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5878005440 con 0x558e6f275580 2024-04-22T13:26:54.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.588+0000 7f589121f740 1 -- 172.21.15.12:0/3448690956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e6f275580 msgr2=0x558e6f275960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:54.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.588+0000 7f589121f740 1 --2- 172.21.15.12:0/3448690956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e6f275580 0x558e6f275960 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f5878009900 tx=0x7f58780095f0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.588+0000 7f589121f740 1 -- 172.21.15.12:0/3448690956 shutdown_connections 2024-04-22T13:26:54.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.588+0000 7f589121f740 1 --2- 172.21.15.12:0/3448690956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6f27ea80 0x558e6f280e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.588+0000 7f589121f740 1 --2- 172.21.15.12:0/3448690956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e6f275ea0 0x558e6f27e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.588+0000 7f589121f740 1 --2- 172.21.15.12:0/3448690956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e6f275580 0x558e6f275960 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.588+0000 7f589121f740 1 -- 172.21.15.12:0/3448690956 >> 172.21.15.12:0/3448690956 conn(0x558e6f26fcb0 msgr2=0x558e6f270090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:54.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.588+0000 7f589121f740 1 -- 172.21.15.12:0/3448690956 shutdown_connections 2024-04-22T13:26:54.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.588+0000 7f589121f740 1 -- 172.21.15.12:0/3448690956 wait complete. 2024-04-22T13:26:54.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.588+0000 7f589121f740 1 -- start start 2024-04-22T13:26:54.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.588+0000 7f589121f740 1 Processor -- start 2024-04-22T13:26:54.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.589+0000 7f589121f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e6f275580 0x558e6f27bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:54.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.589+0000 7f589121f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6f275ea0 0x558e6f27bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:54.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.589+0000 7f589121f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e6f27ea80 0x558e6f27c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:54.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.589+0000 7f589121f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558e6f281810 con 0x558e6f275580 2024-04-22T13:26:54.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.589+0000 7f589121f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x558e6f1932f0 con 0x558e6f27ea80 2024-04-22T13:26:54.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.589+0000 7f589121f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558e6f281990 con 0x558e6f275ea0 2024-04-22T13:26:54.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.589+0000 7f589121f740 10 client.0 _ll_register_callbacks cb 0x558e6f1a0580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:54.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.589+0000 7f5887fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e6f275580 0x558e6f27bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:54.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.589+0000 7f5887fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e6f275580 0x558e6f27bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:35996/0 (socket says 172.21.15.12:35996) 2024-04-22T13:26:54.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.589+0000 7f5887fff640 1 -- 172.21.15.12:0/3634962240 learned_addr learned my addr 172.21.15.12:0/3634962240 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:54.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.589+0000 7f58877fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6f275ea0 0x558e6f27bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:54.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.589+0000 7f588cbd7640 1 --2- 172.21.15.12:0/3634962240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e6f27ea80 0x558e6f27c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:54.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.589+0000 7f5887fff640 1 -- 172.21.15.12:0/3634962240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6f275ea0 msgr2=0x558e6f27bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:54.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.589+0000 7f5887fff640 1 --2- 172.21.15.12:0/3634962240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6f275ea0 0x558e6f27bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.589+0000 7f5887fff640 1 -- 172.21.15.12:0/3634962240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e6f27ea80 msgr2=0x558e6f27c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:54.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.589+0000 7f5887fff640 1 --2- 172.21.15.12:0/3634962240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e6f27ea80 0x558e6f27c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.589+0000 7f5887fff640 1 -- 172.21.15.12:0/3634962240 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e6eeec3f0 con 0x558e6f275580 2024-04-22T13:26:54.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.589+0000 7f58877fe640 1 --2- 172.21.15.12:0/3634962240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6f275ea0 0x558e6f27bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:54.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.589+0000 7f5887fff640 1 --2- 172.21.15.12:0/3634962240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e6f275580 0x558e6f27bab0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f58780054a0 tx=0x7f587802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c123366cd90a9776 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:54.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.589+0000 7f58677fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:26:54.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.589+0000 7f58677fe640 1 -- 172.21.15.12:0/3634962240 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587804e070 con 0x558e6f275580 2024-04-22T13:26:54.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.589+0000 7f58677fe640 1 -- 172.21.15.12:0/3634962240 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f5878004ad0 con 0x558e6f275580 2024-04-22T13:26:54.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.590+0000 7f589121f740 1 -- 172.21.15.12:0/3634962240 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f587802c7e0 con 0x558e6f275580 2024-04-22T13:26:54.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.590+0000 7f58677fe640 1 -- 172.21.15.12:0/3634962240 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5878004cd0 con 0x558e6f275580 2024-04-22T13:26:54.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.590+0000 7f589121f740 10 client.16737 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:54.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.590+0000 7f589121f740 1 -- 172.21.15.12:0/3634962240 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558e6eeac0f0 con 0x558e6f275580 2024-04-22T13:26:54.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.590+0000 7f58677fe640 1 -- 172.21.15.12:0/3634962240 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f587802c7e0 con 0x558e6f275580 2024-04-22T13:26:54.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.590+0000 7f58677fe640 1 -- 172.21.15.12:0/3634962240 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5878041480 con 0x558e6f275580 2024-04-22T13:26:54.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.590+0000 7f589121f740 10 client.16737 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:54.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.590+0000 7f589121f740 10 client.16737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:26:54.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.590+0000 7f589121f740 20 client.16737 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:54.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.590+0000 7f589121f740 1 --2- 172.21.15.12:0/3634962240 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x558e6f28e8a0 0x558e6f290c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:54.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.590+0000 7f589121f740 4 client.16737 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:26:54.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.590+0000 7f589121f740 1 -- 172.21.15.12:0/3634962240 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e6f1e7c00 con 0x558e6f28e8a0 2024-04-22T13:26:54.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.590+0000 7f58877fe640 1 --2- 172.21.15.12:0/3634962240 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x558e6f28e8a0 0x558e6f290c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:54.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.591+0000 7f58877fe640 1 --2- 172.21.15.12:0/3634962240 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x558e6f28e8a0 0x558e6f290c90 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bd5424cac94fca6 server_cookie=8b11f6b0e4703c5c in_seq=0 out_seq=0 2024-04-22T13:26:54.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.591+0000 7f58677fe640 10 client.16737 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:54.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.591+0000 7f58677fe640 1 -- 172.21.15.12:0/3634962240 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558e6f1e7c00 con 0x558e6f28e8a0 2024-04-22T13:26:54.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.591+0000 7f58677fe640 10 client.16737 handle_command_reply: tid=0 2024-04-22T13:26:54.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.591+0000 7f589121f740 1 client.16737 shutdown 2024-04-22T13:26:54.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.592+0000 7f589121f740 1 -- 172.21.15.12:0/3634962240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e6f275580 msgr2=0x558e6f27bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:54.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.592+0000 7f589121f740 1 --2- 172.21.15.12:0/3634962240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e6f275580 0x558e6f27bab0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f58780054a0 tx=0x7f587802bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:54.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.592+0000 7f589121f740 1 -- 172.21.15.12:0/3634962240 shutdown_connections 2024-04-22T13:26:54.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.592+0000 7f589121f740 1 --2- 172.21.15.12:0/3634962240 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x558e6f28e8a0 0x558e6f290c90 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.592+0000 7f589121f740 1 --2- 172.21.15.12:0/3634962240 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e6f27ea80 0x558e6f27c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.592+0000 7f58677fe640 0 client.16737 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:54.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.592+0000 7f589121f740 1 --2- 172.21.15.12:0/3634962240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6f275ea0 0x558e6f27bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.592+0000 7f589121f740 1 --2- 172.21.15.12:0/3634962240 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e6f275580 0x558e6f27bab0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.592+0000 7f589121f740 1 -- 172.21.15.12:0/3634962240 >> 172.21.15.12:0/3634962240 conn(0x558e6f26fcb0 msgr2=0x558e6f271a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:54.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.592+0000 7f589121f740 1 -- 172.21.15.12:0/3634962240 shutdown_connections 2024-04-22T13:26:54.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.592+0000 7f589121f740 1 -- 172.21.15.12:0/3634962240 wait complete. 2024-04-22T13:26:54.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.592+0000 7f589121f740 20 client.16737 trim_cache size 0 max 16384 2024-04-22T13:26:54.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.609+0000 7f589121f740 1 Processor -- start 2024-04-22T13:26:54.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.609+0000 7f589121f740 1 -- start start 2024-04-22T13:26:54.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.610+0000 7f589121f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e6f275580 0x558e6f27e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:54.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.610+0000 7f589121f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6f275ea0 0x558e6f28a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:54.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.610+0000 7f589121f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e6f27ea80 0x558e6f2962c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:54.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.610+0000 7f589121f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f58881afc60 con 0x558e6f27ea80 2024-04-22T13:26:54.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.610+0000 7f589121f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f58881b17a0 con 0x558e6f275580 2024-04-22T13:26:54.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.610+0000 7f589121f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f58881b0330 con 0x558e6f275ea0 2024-04-22T13:26:54.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.610+0000 7f588cbd7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e6f27ea80 0x558e6f2962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:54.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.610+0000 7f588cbd7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e6f27ea80 0x558e6f2962c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36004/0 (socket says 172.21.15.12:36004) 2024-04-22T13:26:54.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.610+0000 7f58877fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6f275ea0 0x558e6f28a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.610+0000 7f588cbd7640 1 -- 172.21.15.12:0/2074311310 learned_addr learned my addr 172.21.15.12:0/2074311310 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.610+0000 7f5887fff640 1 --2- 172.21.15.12:0/2074311310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e6f275580 0x558e6f27e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.610+0000 7f588cbd7640 1 -- 172.21.15.12:0/2074311310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6f275ea0 msgr2=0x558e6f28a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.610+0000 7f588cbd7640 1 --2- 172.21.15.12:0/2074311310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6f275ea0 0x558e6f28a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.610+0000 7f588cbd7640 1 -- 172.21.15.12:0/2074311310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e6f275580 msgr2=0x558e6f27e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.610+0000 7f588cbd7640 1 --2- 172.21.15.12:0/2074311310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e6f275580 0x558e6f27e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.610+0000 7f588cbd7640 1 -- 172.21.15.12:0/2074311310 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e6f275cd0 con 0x558e6f27ea80 2024-04-22T13:26:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.610+0000 7f588cbd7640 1 --2- 172.21.15.12:0/2074311310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e6f27ea80 0x558e6f2962c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5880099670 tx=0x7f588000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6d685f280c2cb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.610+0000 7f587dffb640 1 -- 172.21.15.12:0/2074311310 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58800bbe60 con 0x558e6f27ea80 2024-04-22T13:26:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.610+0000 7f587dffb640 1 -- 172.21.15.12:0/2074311310 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58800a1050 con 0x558e6f27ea80 2024-04-22T13:26:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.610+0000 7f587dffb640 1 -- 172.21.15.12:0/2074311310 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58800bd0c0 con 0x558e6f27ea80 2024-04-22T13:26:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.611+0000 7f589121f740 1 -- 172.21.15.12:0/2074311310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e6f27ea80 msgr2=0x558e6f2962c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.611+0000 7f589121f740 1 --2- 172.21.15.12:0/2074311310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e6f27ea80 0x558e6f2962c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5880099670 tx=0x7f588000cee0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.611+0000 7f589121f740 1 -- 172.21.15.12:0/2074311310 shutdown_connections 2024-04-22T13:26:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.611+0000 7f589121f740 1 --2- 172.21.15.12:0/2074311310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e6f27ea80 0x558e6f2962c0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.611+0000 7f589121f740 1 --2- 172.21.15.12:0/2074311310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6f275ea0 0x558e6f28a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.611+0000 7f589121f740 1 --2- 172.21.15.12:0/2074311310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e6f275580 0x558e6f27e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.611+0000 7f589121f740 1 -- 172.21.15.12:0/2074311310 >> 172.21.15.12:0/2074311310 conn(0x558e6f26fcb0 msgr2=0x558e6f27a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:54.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.611+0000 7f589121f740 1 -- 172.21.15.12:0/2074311310 shutdown_connections 2024-04-22T13:26:54.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.611+0000 7f589121f740 1 -- 172.21.15.12:0/2074311310 wait complete. 2024-04-22T13:26:54.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.611+0000 7f589121f740 1 -- start start 2024-04-22T13:26:54.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.612+0000 7f589121f740 1 Processor -- start 2024-04-22T13:26:54.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.612+0000 7f589121f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e6f275580 0x558e6f286eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:54.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.612+0000 7f589121f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6f275ea0 0x558e6f282220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:54.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.612+0000 7f589121f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e6f27ea80 0x558e6f282760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:54.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.612+0000 7f589121f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f58881b10d0 con 0x558e6f27ea80 2024-04-22T13:26:54.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.612+0000 7f589121f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f58881b0a00 con 0x558e6f275580 2024-04-22T13:26:54.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.612+0000 7f589121f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f58881adc60 con 0x558e6f275ea0 2024-04-22T13:26:54.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.612+0000 7f589121f740 10 client.0 _ll_register_callbacks cb 0x7f5878009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:54.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.612+0000 7f588cbd7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e6f27ea80 0x558e6f282760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:54.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.612+0000 7f58877fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6f275ea0 0x558e6f282220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:54.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.612+0000 7f5887fff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e6f275580 0x558e6f286eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:54.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.612+0000 7f588cbd7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e6f27ea80 0x558e6f282760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36018/0 (socket says 172.21.15.12:36018) 2024-04-22T13:26:54.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.612+0000 7f588cbd7640 1 -- 172.21.15.12:0/459863890 learned_addr learned my addr 172.21.15.12:0/459863890 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:54.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.612+0000 7f588cbd7640 1 -- 172.21.15.12:0/459863890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6f275ea0 msgr2=0x558e6f282220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:54.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.612+0000 7f588cbd7640 1 --2- 172.21.15.12:0/459863890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6f275ea0 0x558e6f282220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.612+0000 7f588cbd7640 1 -- 172.21.15.12:0/459863890 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e6f275580 msgr2=0x558e6f286eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:54.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.612+0000 7f588cbd7640 1 --2- 172.21.15.12:0/459863890 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e6f275580 0x558e6f286eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.612+0000 7f588cbd7640 1 -- 172.21.15.12:0/459863890 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e6f282e50 con 0x558e6f27ea80 2024-04-22T13:26:54.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.612+0000 7f58877fe640 1 --2- 172.21.15.12:0/459863890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6f275ea0 0x558e6f282220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:54.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.612+0000 7f5887fff640 1 --2- 172.21.15.12:0/459863890 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e6f275580 0x558e6f286eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:54.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.612+0000 7f588cbd7640 1 --2- 172.21.15.12:0/459863890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e6f27ea80 0x558e6f282760 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f588000cfb0 tx=0x7f5880099380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ecff934f4a4dba3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:54.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.612+0000 7f58677fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:26:54.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.613+0000 7f58677fe640 1 -- 172.21.15.12:0/459863890 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58800bbe60 con 0x558e6f27ea80 2024-04-22T13:26:54.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.613+0000 7f58677fe640 1 -- 172.21.15.12:0/459863890 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f588009cf00 con 0x558e6f27ea80 2024-04-22T13:26:54.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.613+0000 7f589121f740 1 -- 172.21.15.12:0/459863890 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58800a1050 con 0x558e6f27ea80 2024-04-22T13:26:54.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.613+0000 7f58677fe640 1 -- 172.21.15.12:0/459863890 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f588009d100 con 0x558e6f27ea80 2024-04-22T13:26:54.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.613+0000 7f589121f740 10 client.16743 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:54.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.613+0000 7f589121f740 1 -- 172.21.15.12:0/459863890 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558e6f2f4a40 con 0x558e6f27ea80 2024-04-22T13:26:54.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.613+0000 7f58677fe640 1 -- 172.21.15.12:0/459863890 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58800a1050 con 0x558e6f27ea80 2024-04-22T13:26:54.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.613+0000 7f58677fe640 1 -- 172.21.15.12:0/459863890 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f58800ceba0 con 0x558e6f27ea80 2024-04-22T13:26:54.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.613+0000 7f589121f740 10 client.16743 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:54.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.613+0000 7f589121f740 10 client.16743 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:26:54.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.613+0000 7f589121f740 20 client.16743 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:54.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.613+0000 7f589121f740 1 --2- 172.21.15.12:0/459863890 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x558e6f2f5180 0x558e6f2f75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:54.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.613+0000 7f589121f740 4 client.16743 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:26:54.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.613+0000 7f589121f740 1 -- 172.21.15.12:0/459863890 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558e6f1e1a70 con 0x558e6f2f5180 2024-04-22T13:26:54.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.613+0000 7f5887fff640 1 --2- 172.21.15.12:0/459863890 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x558e6f2f5180 0x558e6f2f75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:54.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.614+0000 7f58677fe640 10 client.16743 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:54.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.614+0000 7f5887fff640 1 --2- 172.21.15.12:0/459863890 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x558e6f2f5180 0x558e6f2f75a0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffced9b34b35a0eb server_cookie=6817ed805574da04 in_seq=0 out_seq=0 2024-04-22T13:26:54.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.614+0000 7f58677fe640 1 -- 172.21.15.12:0/459863890 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558e6f1e1a70 con 0x558e6f2f5180 2024-04-22T13:26:54.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.614+0000 7f58677fe640 10 client.16743 handle_command_reply: tid=0 2024-04-22T13:26:54.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.614+0000 7f589121f740 1 client.16743 shutdown 2024-04-22T13:26:54.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.614+0000 7f589121f740 1 -- 172.21.15.12:0/459863890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e6f27ea80 msgr2=0x558e6f282760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:54.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.614+0000 7f589121f740 1 --2- 172.21.15.12:0/459863890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e6f27ea80 0x558e6f282760 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f588000cfb0 tx=0x7f5880099380 comp rx=0 tx=0).stop 2024-04-22T13:26:54.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.614+0000 7f589121f740 1 -- 172.21.15.12:0/459863890 shutdown_connections 2024-04-22T13:26:54.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.614+0000 7f589121f740 1 --2- 172.21.15.12:0/459863890 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x558e6f2f5180 0x558e6f2f75a0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.614+0000 7f589121f740 1 --2- 172.21.15.12:0/459863890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558e6f27ea80 0x558e6f282760 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.614+0000 7f589121f740 1 --2- 172.21.15.12:0/459863890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558e6f275ea0 0x558e6f282220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.614+0000 7f589121f740 1 --2- 172.21.15.12:0/459863890 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e6f275580 0x558e6f286eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.614+0000 7f58677fe640 0 client.16743 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:54.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.614+0000 7f589121f740 1 -- 172.21.15.12:0/459863890 >> 172.21.15.12:0/459863890 conn(0x558e6f26fcb0 msgr2=0x558e6f26e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:54.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.614+0000 7f589121f740 1 -- 172.21.15.12:0/459863890 shutdown_connections 2024-04-22T13:26:54.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.614+0000 7f589121f740 1 -- 172.21.15.12:0/459863890 wait complete. 2024-04-22T13:26:54.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.614+0000 7f589121f740 20 client.16743 trim_cache size 0 max 16384 2024-04-22T13:26:54.616 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:26:54.616 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-04-22T13:26:54.616 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-04-22T13:26:54.617 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:26:54.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.615+0000 7f588e63a640 1 -- 172.21.15.12:0/2179550671 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f58740a3d20 msgr2=0x7f58740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:54.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.615+0000 7f588e63a640 1 --2- 172.21.15.12:0/2179550671 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f58740a3d20 0x7f58740a61e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5870009710 tx=0x7f5870013880 comp rx=0 tx=0).stop 2024-04-22T13:26:54.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.615+0000 7f588e63a640 1 -- 172.21.15.12:0/2179550671 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5888074cd0 msgr2=0x7f58881a6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:54.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.615+0000 7f588e63a640 1 --2- 172.21.15.12:0/2179550671 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5888074cd0 0x7f58881a6a20 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f588000eb10 tx=0x7f588000c2d0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.615+0000 7f588e63a640 1 -- 172.21.15.12:0/2179550671 shutdown_connections 2024-04-22T13:26:54.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.615+0000 7f588e63a640 1 --2- 172.21.15.12:0/2179550671 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f58740a3d20 0x7f58740a61e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.615+0000 7f588e63a640 1 --2- 172.21.15.12:0/2179550671 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5888074cd0 0x7f58881a6a20 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.615+0000 7f588e63a640 1 --2- 172.21.15.12:0/2179550671 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5888107250 0x7f5888074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.615+0000 7f588e63a640 1 --2- 172.21.15.12:0/2179550671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5888102180 0x7f5888074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.615+0000 7f588e63a640 1 -- 172.21.15.12:0/2179550671 >> 172.21.15.12:0/2179550671 conn(0x7f58880fda20 msgr2=0x7f58880fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:54.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.615+0000 7f588e63a640 1 -- 172.21.15.12:0/2179550671 shutdown_connections 2024-04-22T13:26:54.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.615+0000 7f588e63a640 1 -- 172.21.15.12:0/2179550671 wait complete. 2024-04-22T13:26:54.621 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:26:54.621 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:26:54.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.728+0000 7efcadde2640 1 -- 172.21.15.12:0/4105881504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efca810c5a0 msgr2=0x7efca810e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:54.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.728+0000 7efcadde2640 1 --2- 172.21.15.12:0/4105881504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efca810c5a0 0x7efca810e9d0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7efca0009ca0 tx=0x7efca002c180 comp rx=0 tx=0).stop 2024-04-22T13:26:54.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.728+0000 7efcadde2640 1 -- 172.21.15.12:0/4105881504 shutdown_connections 2024-04-22T13:26:54.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.728+0000 7efcadde2640 1 --2- 172.21.15.12:0/4105881504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efca810c5a0 0x7efca810e9d0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.728+0000 7efcadde2640 1 --2- 172.21.15.12:0/4105881504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efca8103af0 0x7efca810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.728+0000 7efcadde2640 1 --2- 172.21.15.12:0/4105881504 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efca81031d0 0x7efca81035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.728+0000 7efcadde2640 1 -- 172.21.15.12:0/4105881504 >> 172.21.15.12:0/4105881504 conn(0x7efca80771d0 msgr2=0x7efca80775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:54.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.728+0000 7efcadde2640 1 -- 172.21.15.12:0/4105881504 shutdown_connections 2024-04-22T13:26:54.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.728+0000 7efcadde2640 1 -- 172.21.15.12:0/4105881504 wait complete. 2024-04-22T13:26:54.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.728+0000 7efcadde2640 1 Processor -- start 2024-04-22T13:26:54.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.729+0000 7efcadde2640 1 -- start start 2024-04-22T13:26:54.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.729+0000 7efcadde2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efca81031d0 0x7efca81986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:54.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.729+0000 7efcadde2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efca8103af0 0x7efca8198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:54.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.729+0000 7efcadde2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efca810c5a0 0x7efca819cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:54.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.729+0000 7efcadde2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efca81121a0 con 0x7efca81031d0 2024-04-22T13:26:54.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.729+0000 7efcadde2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efca8112020 con 0x7efca810c5a0 2024-04-22T13:26:54.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.729+0000 7efcadde2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efca8112320 con 0x7efca8103af0 2024-04-22T13:26:54.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.729+0000 7efca77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efca81031d0 0x7efca81986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:54.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.729+0000 7efca77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efca81031d0 0x7efca81986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36030/0 (socket says 172.21.15.12:36030) 2024-04-22T13:26:54.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.729+0000 7efca77fe640 1 -- 172.21.15.12:0/3130454877 learned_addr learned my addr 172.21.15.12:0/3130454877 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:54.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.729+0000 7efca7fff640 1 --2- 172.21.15.12:0/3130454877 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efca810c5a0 0x7efca819cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:54.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.729+0000 7efca6ffd640 1 --2- 172.21.15.12:0/3130454877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efca8103af0 0x7efca8198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:54.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.729+0000 7efca77fe640 1 -- 172.21.15.12:0/3130454877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efca8103af0 msgr2=0x7efca8198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:54.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.729+0000 7efca77fe640 1 --2- 172.21.15.12:0/3130454877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efca8103af0 0x7efca8198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.729+0000 7efca77fe640 1 -- 172.21.15.12:0/3130454877 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efca810c5a0 msgr2=0x7efca819cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:54.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.729+0000 7efca77fe640 1 --2- 172.21.15.12:0/3130454877 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efca810c5a0 0x7efca819cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.729+0000 7efca77fe640 1 -- 172.21.15.12:0/3130454877 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca819d6b0 con 0x7efca81031d0 2024-04-22T13:26:54.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.729+0000 7efca77fe640 1 --2- 172.21.15.12:0/3130454877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efca81031d0 0x7efca81986c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7efc9800d950 tx=0x7efc9800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:54.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.730+0000 7efca4ff9640 1 -- 172.21.15.12:0/3130454877 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc98017070 con 0x7efca81031d0 2024-04-22T13:26:54.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.730+0000 7efca4ff9640 1 -- 172.21.15.12:0/3130454877 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc98009c30 con 0x7efca81031d0 2024-04-22T13:26:54.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.730+0000 7efcadde2640 1 -- 172.21.15.12:0/3130454877 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca819d9a0 con 0x7efca81031d0 2024-04-22T13:26:54.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.730+0000 7efca4ff9640 1 -- 172.21.15.12:0/3130454877 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9800f620 con 0x7efca81031d0 2024-04-22T13:26:54.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.730+0000 7efcadde2640 1 -- 172.21.15.12:0/3130454877 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca81a5230 con 0x7efca81031d0 2024-04-22T13:26:54.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.731+0000 7efcadde2640 1 -- 172.21.15.12:0/3130454877 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efca8069230 con 0x7efca81031d0 2024-04-22T13:26:54.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.732+0000 7efca4ff9640 1 -- 172.21.15.12:0/3130454877 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efc98005030 con 0x7efca81031d0 2024-04-22T13:26:54.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.732+0000 7efca4ff9640 1 --2- 172.21.15.12:0/3130454877 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7efc900a3d20 0x7efc900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:54.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.732+0000 7efca6ffd640 1 --2- 172.21.15.12:0/3130454877 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7efc900a3d20 0x7efc900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:54.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.732+0000 7efca4ff9640 1 -- 172.21.15.12:0/3130454877 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7efc980d0320 con 0x7efca81031d0 2024-04-22T13:26:54.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.733+0000 7efca6ffd640 1 --2- 172.21.15.12:0/3130454877 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7efc900a3d20 0x7efc900a61e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efca81006c0 tx=0x7efc9c00b390 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:54.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.736+0000 7efca4ff9640 1 -- 172.21.15.12:0/3130454877 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efc98009de0 con 0x7efca81031d0 2024-04-22T13:26:54.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.964+0000 7efcadde2640 1 -- 172.21.15.12:0/3130454877 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efca8199a10 con 0x7efca81031d0 2024-04-22T13:26:54.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.965+0000 7efca4ff9640 1 -- 172.21.15.12:0/3130454877 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7efc9809c3c0 con 0x7efca81031d0 2024-04-22T13:26:54.961 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:26:54.961 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:26:54.961 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:26:54.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.967+0000 7efcadde2640 1 -- 172.21.15.12:0/3130454877 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7efc900a3d20 msgr2=0x7efc900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:54.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.967+0000 7efcadde2640 1 --2- 172.21.15.12:0/3130454877 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7efc900a3d20 0x7efc900a61e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efca81006c0 tx=0x7efc9c00b390 comp rx=0 tx=0).stop 2024-04-22T13:26:54.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.967+0000 7efcadde2640 1 -- 172.21.15.12:0/3130454877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efca81031d0 msgr2=0x7efca81986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:54.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.967+0000 7efcadde2640 1 --2- 172.21.15.12:0/3130454877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efca81031d0 0x7efca81986c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7efc9800d950 tx=0x7efc9800de10 comp rx=0 tx=0).stop 2024-04-22T13:26:54.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.967+0000 7efcadde2640 1 -- 172.21.15.12:0/3130454877 shutdown_connections 2024-04-22T13:26:54.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.967+0000 7efcadde2640 1 --2- 172.21.15.12:0/3130454877 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7efc900a3d20 0x7efc900a61e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.967+0000 7efcadde2640 1 --2- 172.21.15.12:0/3130454877 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efca810c5a0 0x7efca819cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.967+0000 7efcadde2640 1 --2- 172.21.15.12:0/3130454877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efca8103af0 0x7efca8198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.967+0000 7efcadde2640 1 --2- 172.21.15.12:0/3130454877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efca81031d0 0x7efca81986c0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:54.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.967+0000 7efcadde2640 1 -- 172.21.15.12:0/3130454877 >> 172.21.15.12:0/3130454877 conn(0x7efca80771d0 msgr2=0x7efca80fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:54.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.968+0000 7efcadde2640 1 -- 172.21.15.12:0/3130454877 shutdown_connections 2024-04-22T13:26:54.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:54.968+0000 7efcadde2640 1 -- 172.21.15.12:0/3130454877 wait complete. 2024-04-22T13:26:54.975 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:26:55.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.082+0000 7f2a92759640 1 -- 172.21.15.12:0/452963076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a8c103970 msgr2=0x7f2a8c10c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.082+0000 7f2a92759640 1 --2- 172.21.15.12:0/452963076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a8c103970 0x7f2a8c10c030 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009900 tx=0x7f2a8002bcd0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.083+0000 7f2a92759640 1 -- 172.21.15.12:0/452963076 shutdown_connections 2024-04-22T13:26:55.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.083+0000 7f2a92759640 1 --2- 172.21.15.12:0/452963076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a8c10c570 0x7f2a8c10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.083+0000 7f2a92759640 1 --2- 172.21.15.12:0/452963076 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a8c103970 0x7f2a8c10c030 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.083+0000 7f2a92759640 1 --2- 172.21.15.12:0/452963076 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a8c103050 0x7f2a8c103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.083+0000 7f2a92759640 1 -- 172.21.15.12:0/452963076 >> 172.21.15.12:0/452963076 conn(0x7f2a8c077fb0 msgr2=0x7f2a8c0783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:55.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.083+0000 7f2a92759640 1 -- 172.21.15.12:0/452963076 shutdown_connections 2024-04-22T13:26:55.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.083+0000 7f2a92759640 1 -- 172.21.15.12:0/452963076 wait complete. 2024-04-22T13:26:55.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.083+0000 7f2a92759640 1 Processor -- start 2024-04-22T13:26:55.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.083+0000 7f2a92759640 1 -- start start 2024-04-22T13:26:55.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.083+0000 7f2a92759640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a8c103050 0x7f2a8c19e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.083+0000 7f2a92759640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a8c103970 0x7f2a8c19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.083+0000 7f2a92759640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a8c10c570 0x7f2a8c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.084+0000 7f2a92759640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c112190 con 0x7f2a8c103970 2024-04-22T13:26:55.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.084+0000 7f2a92759640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c112010 con 0x7f2a8c10c570 2024-04-22T13:26:55.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.084+0000 7f2a92759640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c112310 con 0x7f2a8c103050 2024-04-22T13:26:55.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.084+0000 7f2a8b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a8c103970 0x7f2a8c19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.084+0000 7f2a8b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a8c103970 0x7f2a8c19e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36044/0 (socket says 172.21.15.12:36044) 2024-04-22T13:26:55.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.084+0000 7f2a8b7fe640 1 -- 172.21.15.12:0/3017126089 learned_addr learned my addr 172.21.15.12:0/3017126089 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:55.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.084+0000 7f2a8bfff640 1 --2- 172.21.15.12:0/3017126089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a8c103050 0x7f2a8c19e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.084+0000 7f2a90cf6640 1 --2- 172.21.15.12:0/3017126089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a8c10c570 0x7f2a8c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.084+0000 7f2a8b7fe640 1 -- 172.21.15.12:0/3017126089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a8c103050 msgr2=0x7f2a8c19e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.084+0000 7f2a8b7fe640 1 --2- 172.21.15.12:0/3017126089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a8c103050 0x7f2a8c19e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.084+0000 7f2a8b7fe640 1 -- 172.21.15.12:0/3017126089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a8c10c570 msgr2=0x7f2a8c1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.084+0000 7f2a8b7fe640 1 --2- 172.21.15.12:0/3017126089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a8c10c570 0x7f2a8c1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.084+0000 7f2a8b7fe640 1 -- 172.21.15.12:0/3017126089 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8c198d30 con 0x7f2a8c103970 2024-04-22T13:26:55.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.084+0000 7f2a8b7fe640 1 --2- 172.21.15.12:0/3017126089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a8c103970 0x7f2a8c19e940 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2a80005940 tx=0x7f2a8002d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.084+0000 7f2a897fa640 1 -- 172.21.15.12:0/3017126089 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80043020 con 0x7f2a8c103970 2024-04-22T13:26:55.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.084+0000 7f2a897fa640 1 -- 172.21.15.12:0/3017126089 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a8002dc90 con 0x7f2a8c103970 2024-04-22T13:26:55.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.084+0000 7f2a92759640 1 -- 172.21.15.12:0/3017126089 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a8c198fc0 con 0x7f2a8c103970 2024-04-22T13:26:55.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.084+0000 7f2a897fa640 1 -- 172.21.15.12:0/3017126089 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80003ab0 con 0x7f2a8c103970 2024-04-22T13:26:55.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.084+0000 7f2a92759640 1 -- 172.21.15.12:0/3017126089 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a8c1a52c0 con 0x7f2a8c103970 2024-04-22T13:26:55.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.087+0000 7f2a897fa640 1 -- 172.21.15.12:0/3017126089 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2a80046020 con 0x7f2a8c103970 2024-04-22T13:26:55.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.087+0000 7f2a9542e740 1 Processor -- start 2024-04-22T13:26:55.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.087+0000 7f2a9542e740 1 -- start start 2024-04-22T13:26:55.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.087+0000 7f2a897fa640 1 --2- 172.21.15.12:0/3017126089 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2a740a3d20 0x7f2a740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.087+0000 7f2a9542e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fa850c4d0 0x559fa850c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.087+0000 7f2a9542e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa850ce80 0x559fa8515430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.087+0000 7f2a9542e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559fa8515a00 0x559fa8517e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.087+0000 7f2a9542e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559fa84711f0 con 0x559fa850c4d0 2024-04-22T13:26:55.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.087+0000 7f2a9542e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559fa840e4b0 con 0x559fa8515a00 2024-04-22T13:26:55.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.087+0000 7f2a9542e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559fa846d1b0 con 0x559fa850ce80 2024-04-22T13:26:55.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.087+0000 7f2a8bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fa850c4d0 0x559fa850c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.087+0000 7f2a90cf6640 1 --2- 172.21.15.12:0/3017126089 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2a740a3d20 0x7f2a740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.087+0000 7f2a8bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fa850c4d0 0x559fa850c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36046/0 (socket says 172.21.15.12:36046) 2024-04-22T13:26:55.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.087+0000 7f2a897fa640 1 -- 172.21.15.12:0/3017126089 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f2a800f27e0 con 0x7f2a8c103970 2024-04-22T13:26:55.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.087+0000 7f2a8bfff640 1 -- 172.21.15.12:0/3853187895 learned_addr learned my addr 172.21.15.12:0/3853187895 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:55.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.087+0000 7f2a8b7fe640 1 --2- 172.21.15.12:0/3853187895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa850ce80 0x559fa8515430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.087+0000 7f2a90cf6640 1 --2- 172.21.15.12:0/3853187895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559fa8515a00 0x559fa8517e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.088+0000 7f2a8bfff640 1 -- 172.21.15.12:0/3853187895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa850ce80 msgr2=0x559fa8515430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.088+0000 7f2a8bfff640 1 --2- 172.21.15.12:0/3853187895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa850ce80 0x559fa8515430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.088+0000 7f2a8bfff640 1 -- 172.21.15.12:0/3853187895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559fa8515a00 msgr2=0x559fa8517e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.088+0000 7f2a8bfff640 1 --2- 172.21.15.12:0/3853187895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559fa8515a00 0x559fa8517e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.088+0000 7f2a8bfff640 1 -- 172.21.15.12:0/3853187895 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fa84001b0 con 0x559fa850c4d0 2024-04-22T13:26:55.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.088+0000 7f2a90cf6640 1 --2- 172.21.15.12:0/3017126089 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2a740a3d20 0x7f2a740a61e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400afd0 tx=0x7f2a8400acc0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.088+0000 7f2a8bfff640 1 --2- 172.21.15.12:0/3853187895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fa850c4d0 0x559fa850c8b0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00bcd0 tx=0x7f2a7c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=960b58bb2db1fe27 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.088+0000 7f2a727fc640 1 -- 172.21.15.12:0/3853187895 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c005220 con 0x559fa850c4d0 2024-04-22T13:26:55.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.088+0000 7f2a727fc640 1 -- 172.21.15.12:0/3853187895 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a7c004c50 con 0x559fa850c4d0 2024-04-22T13:26:55.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.088+0000 7f2a727fc640 1 -- 172.21.15.12:0/3853187895 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c011640 con 0x559fa850c4d0 2024-04-22T13:26:55.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.089+0000 7f2a9542e740 1 -- 172.21.15.12:0/3853187895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fa850c4d0 msgr2=0x559fa850c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.089+0000 7f2a9542e740 1 --2- 172.21.15.12:0/3853187895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fa850c4d0 0x559fa850c8b0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00bcd0 tx=0x7f2a7c0107a0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.089+0000 7f2a9542e740 1 -- 172.21.15.12:0/3853187895 shutdown_connections 2024-04-22T13:26:55.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.089+0000 7f2a9542e740 1 --2- 172.21.15.12:0/3853187895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559fa8515a00 0x559fa8517e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.089+0000 7f2a9542e740 1 --2- 172.21.15.12:0/3853187895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa850ce80 0x559fa8515430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.089+0000 7f2a9542e740 1 --2- 172.21.15.12:0/3853187895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fa850c4d0 0x559fa850c8b0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.089+0000 7f2a9542e740 1 -- 172.21.15.12:0/3853187895 >> 172.21.15.12:0/3853187895 conn(0x559fa8506ba0 msgr2=0x559fa8506f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:55.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.089+0000 7f2a9542e740 1 -- 172.21.15.12:0/3853187895 shutdown_connections 2024-04-22T13:26:55.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.089+0000 7f2a9542e740 1 -- 172.21.15.12:0/3853187895 wait complete. 2024-04-22T13:26:55.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.089+0000 7f2a9542e740 1 -- start start 2024-04-22T13:26:55.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.089+0000 7f2a9542e740 1 Processor -- start 2024-04-22T13:26:55.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.089+0000 7f2a9542e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fa850c4d0 0x559fa8515310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.089+0000 7f2a9542e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa850ce80 0x559fa85126a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.089+0000 7f2a9542e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559fa8515a00 0x559fa8512c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.090+0000 7f2a9542e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559fa8518970 con 0x559fa850c4d0 2024-04-22T13:26:55.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.090+0000 7f2a9542e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559fa842a500 con 0x559fa8515a00 2024-04-22T13:26:55.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.090+0000 7f2a9542e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559fa8518af0 con 0x559fa850ce80 2024-04-22T13:26:55.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.090+0000 7f2a9542e740 10 client.0 _ll_register_callbacks cb 0x559fa83f8b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:55.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.090+0000 7f2a8bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fa850c4d0 0x559fa8515310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.090+0000 7f2a8bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fa850c4d0 0x559fa8515310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36054/0 (socket says 172.21.15.12:36054) 2024-04-22T13:26:55.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.090+0000 7f2a8bfff640 1 -- 172.21.15.12:0/1171231204 learned_addr learned my addr 172.21.15.12:0/1171231204 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:55.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.090+0000 7f2a8b7fe640 1 --2- 172.21.15.12:0/1171231204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa850ce80 0x559fa85126a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.090+0000 7f2a90cf6640 1 --2- 172.21.15.12:0/1171231204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559fa8515a00 0x559fa8512c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.090+0000 7f2a8bfff640 1 -- 172.21.15.12:0/1171231204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa850ce80 msgr2=0x559fa85126a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.090+0000 7f2a8bfff640 1 --2- 172.21.15.12:0/1171231204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa850ce80 0x559fa85126a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.090+0000 7f2a8bfff640 1 -- 172.21.15.12:0/1171231204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559fa8515a00 msgr2=0x559fa8512c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.090+0000 7f2a8bfff640 1 --2- 172.21.15.12:0/1171231204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559fa8515a00 0x559fa8512c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.090+0000 7f2a8bfff640 1 -- 172.21.15.12:0/1171231204 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fa81833f0 con 0x559fa850c4d0 2024-04-22T13:26:55.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.090+0000 7f2a8bfff640 1 --2- 172.21.15.12:0/1171231204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fa850c4d0 0x559fa8515310 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c006010 tx=0x7f2a7c00ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bc47f9c541bbff0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.090+0000 7f2a6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:26:55.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.090+0000 7f2a6b7fe640 1 -- 172.21.15.12:0/1171231204 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c02a060 con 0x559fa850c4d0 2024-04-22T13:26:55.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.090+0000 7f2a6b7fe640 1 -- 172.21.15.12:0/1171231204 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f2a7c004e00 con 0x559fa850c4d0 2024-04-22T13:26:55.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.090+0000 7f2a9542e740 1 -- 172.21.15.12:0/1171231204 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a7c004c50 con 0x559fa850c4d0 2024-04-22T13:26:55.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.090+0000 7f2a6b7fe640 1 -- 172.21.15.12:0/1171231204 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2a7c00be80 con 0x559fa850c4d0 2024-04-22T13:26:55.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.090+0000 7f2a9542e740 10 client.16761 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:55.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.090+0000 7f2a9542e740 1 -- 172.21.15.12:0/1171231204 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559fa824d7d0 con 0x559fa850c4d0 2024-04-22T13:26:55.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.091+0000 7f2a6b7fe640 1 -- 172.21.15.12:0/1171231204 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a7c004c50 con 0x559fa850c4d0 2024-04-22T13:26:55.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.091+0000 7f2a6b7fe640 1 -- 172.21.15.12:0/1171231204 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2a7c011440 con 0x559fa850c4d0 2024-04-22T13:26:55.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.091+0000 7f2a9542e740 10 client.16761 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:55.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.091+0000 7f2a9542e740 10 client.16761 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:26:55.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.091+0000 7f2a9542e740 20 client.16761 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:55.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.091+0000 7f2a9542e740 1 --2- 172.21.15.12:0/1171231204 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x559fa8525bb0 0x559fa8527fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.091+0000 7f2a9542e740 4 client.16761 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:26:55.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.091+0000 7f2a9542e740 1 -- 172.21.15.12:0/1171231204 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fa8410930 con 0x559fa8525bb0 2024-04-22T13:26:55.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.091+0000 7f2a8b7fe640 1 --2- 172.21.15.12:0/1171231204 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x559fa8525bb0 0x559fa8527fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.091+0000 7f2a6b7fe640 10 client.16761 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:26:55.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.091+0000 7f2a8b7fe640 1 --2- 172.21.15.12:0/1171231204 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x559fa8525bb0 0x559fa8527fa0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4ce11c47ea8c5a45 server_cookie=45f0250b9a60e629 in_seq=0 out_seq=0 2024-04-22T13:26:55.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.092+0000 7f2a6b7fe640 1 -- 172.21.15.12:0/1171231204 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559fa8410930 con 0x559fa8525bb0 2024-04-22T13:26:55.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.092+0000 7f2a6b7fe640 10 client.16761 handle_command_reply: tid=0 2024-04-22T13:26:55.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.092+0000 7f2a9542e740 1 client.16761 shutdown 2024-04-22T13:26:55.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.092+0000 7f2a9542e740 1 -- 172.21.15.12:0/1171231204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fa850c4d0 msgr2=0x559fa8515310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.092+0000 7f2a9542e740 1 --2- 172.21.15.12:0/1171231204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fa850c4d0 0x559fa8515310 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c006010 tx=0x7f2a7c00ba20 comp rx=0 tx=0).stop 2024-04-22T13:26:55.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.092+0000 7f2a9542e740 1 -- 172.21.15.12:0/1171231204 shutdown_connections 2024-04-22T13:26:55.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.092+0000 7f2a9542e740 1 --2- 172.21.15.12:0/1171231204 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x559fa8525bb0 0x559fa8527fa0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.092+0000 7f2a9542e740 1 --2- 172.21.15.12:0/1171231204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559fa8515a00 0x559fa8512c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.092+0000 7f2a9542e740 1 --2- 172.21.15.12:0/1171231204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa850ce80 0x559fa85126a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.092+0000 7f2a6b7fe640 0 client.16761 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:26:55.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.092+0000 7f2a9542e740 1 --2- 172.21.15.12:0/1171231204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fa850c4d0 0x559fa8515310 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.092+0000 7f2a9542e740 1 -- 172.21.15.12:0/1171231204 >> 172.21.15.12:0/1171231204 conn(0x559fa8506ba0 msgr2=0x559fa85088f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:55.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.092+0000 7f2a9542e740 1 -- 172.21.15.12:0/1171231204 shutdown_connections 2024-04-22T13:26:55.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.092+0000 7f2a9542e740 1 -- 172.21.15.12:0/1171231204 wait complete. 2024-04-22T13:26:55.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.092+0000 7f2a9542e740 20 client.16761 trim_cache size 0 max 16384 2024-04-22T13:26:55.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.106+0000 7f2a9542e740 1 Processor -- start 2024-04-22T13:26:55.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.106+0000 7f2a9542e740 1 -- start start 2024-04-22T13:26:55.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.106+0000 7f2a9542e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fa850c4d0 0x559fa8521830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.106+0000 7f2a9542e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559fa850ce80 0x559fa8505b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.106+0000 7f2a9542e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa8515a00 0x559fa8514dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.106+0000 7f2a9542e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1adae0 con 0x559fa850c4d0 2024-04-22T13:26:55.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.106+0000 7f2a9542e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1af620 con 0x559fa850ce80 2024-04-22T13:26:55.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.106+0000 7f2a9542e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1ae1b0 con 0x559fa8515a00 2024-04-22T13:26:55.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.106+0000 7f2a8bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fa850c4d0 0x559fa8521830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.106+0000 7f2a8bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fa850c4d0 0x559fa8521830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36066/0 (socket says 172.21.15.12:36066) 2024-04-22T13:26:55.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.106+0000 7f2a8bfff640 1 -- 172.21.15.12:0/2091776721 learned_addr learned my addr 172.21.15.12:0/2091776721 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:55.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.106+0000 7f2a8b7fe640 1 --2- 172.21.15.12:0/2091776721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559fa850ce80 0x559fa8505b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.106+0000 7f2a8bfff640 1 -- 172.21.15.12:0/2091776721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa8515a00 msgr2=0x559fa8514dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.106+0000 7f2a8bfff640 1 --2- 172.21.15.12:0/2091776721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa8515a00 0x559fa8514dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.106+0000 7f2a8bfff640 1 -- 172.21.15.12:0/2091776721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559fa850ce80 msgr2=0x559fa8505b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.106+0000 7f2a8bfff640 1 --2- 172.21.15.12:0/2091776721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559fa850ce80 0x559fa8505b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.106+0000 7f2a8bfff640 1 -- 172.21.15.12:0/2091776721 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fa8512a10 con 0x559fa850c4d0 2024-04-22T13:26:55.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.106+0000 7f2a8b7fe640 1 --2- 172.21.15.12:0/2091776721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559fa850ce80 0x559fa8505b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:55.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.107+0000 7f2a8bfff640 1 --2- 172.21.15.12:0/2091776721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fa850c4d0 0x559fa8521830 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c000c00 tx=0x7f2a7c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa42560d295eb9e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.107+0000 7f2a71ffb640 1 -- 172.21.15.12:0/2091776721 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c02a050 con 0x559fa850c4d0 2024-04-22T13:26:55.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.107+0000 7f2a71ffb640 1 -- 172.21.15.12:0/2091776721 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a7c011b60 con 0x559fa850c4d0 2024-04-22T13:26:55.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.107+0000 7f2a71ffb640 1 -- 172.21.15.12:0/2091776721 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c003850 con 0x559fa850c4d0 2024-04-22T13:26:55.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.107+0000 7f2a9542e740 1 -- 172.21.15.12:0/2091776721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fa850c4d0 msgr2=0x559fa8521830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.107+0000 7f2a9542e740 1 --2- 172.21.15.12:0/2091776721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fa850c4d0 0x559fa8521830 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c000c00 tx=0x7f2a7c0107a0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.107+0000 7f2a9542e740 1 -- 172.21.15.12:0/2091776721 shutdown_connections 2024-04-22T13:26:55.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.107+0000 7f2a9542e740 1 --2- 172.21.15.12:0/2091776721 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa8515a00 0x559fa8514dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.107+0000 7f2a9542e740 1 --2- 172.21.15.12:0/2091776721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559fa850ce80 0x559fa8505b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.107+0000 7f2a9542e740 1 --2- 172.21.15.12:0/2091776721 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fa850c4d0 0x559fa8521830 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.107+0000 7f2a9542e740 1 -- 172.21.15.12:0/2091776721 >> 172.21.15.12:0/2091776721 conn(0x559fa8506ba0 msgr2=0x559fa85116d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:55.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.107+0000 7f2a9542e740 1 -- 172.21.15.12:0/2091776721 shutdown_connections 2024-04-22T13:26:55.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.107+0000 7f2a9542e740 1 -- 172.21.15.12:0/2091776721 wait complete. 2024-04-22T13:26:55.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.108+0000 7f2a9542e740 1 -- start start 2024-04-22T13:26:55.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.108+0000 7f2a9542e740 1 Processor -- start 2024-04-22T13:26:55.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.108+0000 7f2a9542e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa850c4d0 0x559fa851e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.108+0000 7f2a9542e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559fa850ce80 0x559fa8519580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.108+0000 7f2a9542e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fa8515a00 0x559fa8519b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.108+0000 7f2a9542e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1aef50 con 0x559fa8515a00 2024-04-22T13:26:55.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.108+0000 7f2a9542e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1ae880 con 0x559fa850ce80 2024-04-22T13:26:55.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.108+0000 7f2a9542e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c1abac0 con 0x559fa850c4d0 2024-04-22T13:26:55.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.108+0000 7f2a9542e740 10 client.0 _ll_register_callbacks cb 0x7f2a7c00bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:55.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.108+0000 7f2a90cf6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fa8515a00 0x559fa8519b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.108+0000 7f2a90cf6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fa8515a00 0x559fa8519b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36078/0 (socket says 172.21.15.12:36078) 2024-04-22T13:26:55.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.108+0000 7f2a90cf6640 1 -- 172.21.15.12:0/4074195878 learned_addr learned my addr 172.21.15.12:0/4074195878 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:55.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.108+0000 7f2a8b7fe640 1 --2- 172.21.15.12:0/4074195878 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559fa850ce80 0x559fa8519580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.108+0000 7f2a8bfff640 1 --2- 172.21.15.12:0/4074195878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa850c4d0 0x559fa851e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.109+0000 7f2a90cf6640 1 -- 172.21.15.12:0/4074195878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa850c4d0 msgr2=0x559fa851e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.109+0000 7f2a90cf6640 1 --2- 172.21.15.12:0/4074195878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa850c4d0 0x559fa851e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.109+0000 7f2a90cf6640 1 -- 172.21.15.12:0/4074195878 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559fa850ce80 msgr2=0x559fa8519580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.109+0000 7f2a90cf6640 1 --2- 172.21.15.12:0/4074195878 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559fa850ce80 0x559fa8519580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.109+0000 7f2a90cf6640 1 -- 172.21.15.12:0/4074195878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fa851a390 con 0x559fa8515a00 2024-04-22T13:26:55.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.109+0000 7f2a8b7fe640 1 --2- 172.21.15.12:0/4074195878 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559fa850ce80 0x559fa8519580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:26:55.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.109+0000 7f2a90cf6640 1 --2- 172.21.15.12:0/4074195878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fa8515a00 0x559fa8519b90 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400a810 tx=0x7f2a84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5800b979d2f719 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.109+0000 7f2a6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:26:55.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.109+0000 7f2a6b7fe640 1 -- 172.21.15.12:0/4074195878 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84020d60 con 0x559fa8515a00 2024-04-22T13:26:55.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.109+0000 7f2a6b7fe640 1 -- 172.21.15.12:0/4074195878 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f2a84004b20 con 0x559fa8515a00 2024-04-22T13:26:55.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.109+0000 7f2a9542e740 1 -- 172.21.15.12:0/4074195878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a7c011b60 con 0x559fa8515a00 2024-04-22T13:26:55.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.109+0000 7f2a6b7fe640 1 -- 172.21.15.12:0/4074195878 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2a84004d20 con 0x559fa8515a00 2024-04-22T13:26:55.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.109+0000 7f2a9542e740 10 client.16767 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:55.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.109+0000 7f2a9542e740 1 -- 172.21.15.12:0/4074195878 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559fa858bee0 con 0x559fa8515a00 2024-04-22T13:26:55.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.109+0000 7f2a6b7fe640 1 -- 172.21.15.12:0/4074195878 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a7c011b60 con 0x559fa8515a00 2024-04-22T13:26:55.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.110+0000 7f2a6b7fe640 1 -- 172.21.15.12:0/4074195878 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2a8401c040 con 0x559fa8515a00 2024-04-22T13:26:55.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.110+0000 7f2a9542e740 10 client.16767 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:55.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.110+0000 7f2a9542e740 10 client.16767 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:26:55.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.110+0000 7f2a9542e740 20 client.16767 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:55.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.110+0000 7f2a9542e740 1 --2- 172.21.15.12:0/4074195878 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x559fa858c660 0x559fa858ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.110+0000 7f2a9542e740 4 client.16767 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:26:55.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.110+0000 7f2a9542e740 1 -- 172.21.15.12:0/4074195878 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559fa847ec10 con 0x559fa858c660 2024-04-22T13:26:55.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.110+0000 7f2a8bfff640 1 --2- 172.21.15.12:0/4074195878 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x559fa858c660 0x559fa858ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.110+0000 7f2a8bfff640 1 --2- 172.21.15.12:0/4074195878 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x559fa858c660 0x559fa858ea80 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a53ae65f66ebeb22 server_cookie=a2f371fd2db61e78 in_seq=0 out_seq=0 2024-04-22T13:26:55.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.110+0000 7f2a6b7fe640 10 client.16767 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:26:55.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.110+0000 7f2a6b7fe640 1 -- 172.21.15.12:0/4074195878 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559fa847ec10 con 0x559fa858c660 2024-04-22T13:26:55.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.110+0000 7f2a6b7fe640 10 client.16767 handle_command_reply: tid=0 2024-04-22T13:26:55.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.110+0000 7f2a9542e740 1 client.16767 shutdown 2024-04-22T13:26:55.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.111+0000 7f2a9542e740 1 -- 172.21.15.12:0/4074195878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fa8515a00 msgr2=0x559fa8519b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.111+0000 7f2a9542e740 1 --2- 172.21.15.12:0/4074195878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fa8515a00 0x559fa8519b90 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400a810 tx=0x7f2a84004a40 comp rx=0 tx=0).stop 2024-04-22T13:26:55.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.111+0000 7f2a9542e740 1 -- 172.21.15.12:0/4074195878 shutdown_connections 2024-04-22T13:26:55.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.111+0000 7f2a9542e740 1 --2- 172.21.15.12:0/4074195878 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x559fa858c660 0x559fa858ea80 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.111+0000 7f2a9542e740 1 --2- 172.21.15.12:0/4074195878 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559fa8515a00 0x559fa8519b90 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.111+0000 7f2a6b7fe640 0 client.16767 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:26:55.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.111+0000 7f2a9542e740 1 --2- 172.21.15.12:0/4074195878 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559fa850ce80 0x559fa8519580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.111+0000 7f2a9542e740 1 --2- 172.21.15.12:0/4074195878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559fa850c4d0 0x559fa851e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.111+0000 7f2a9542e740 1 -- 172.21.15.12:0/4074195878 >> 172.21.15.12:0/4074195878 conn(0x559fa8506ba0 msgr2=0x559fa85116d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:55.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.111+0000 7f2a9542e740 1 -- 172.21.15.12:0/4074195878 shutdown_connections 2024-04-22T13:26:55.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.111+0000 7f2a9542e740 1 -- 172.21.15.12:0/4074195878 wait complete. 2024-04-22T13:26:55.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.111+0000 7f2a9542e740 20 client.16767 trim_cache size 0 max 16384 2024-04-22T13:26:55.113 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:26:55.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.111+0000 7f2a92759640 1 -- 172.21.15.12:0/3017126089 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2a740a3d20 msgr2=0x7f2a740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.111+0000 7f2a92759640 1 --2- 172.21.15.12:0/3017126089 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2a740a3d20 0x7f2a740a61e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400afd0 tx=0x7f2a8400acc0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.111+0000 7f2a92759640 1 -- 172.21.15.12:0/3017126089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a8c103970 msgr2=0x7f2a8c19e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.111+0000 7f2a92759640 1 --2- 172.21.15.12:0/3017126089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a8c103970 0x7f2a8c19e940 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2a80005940 tx=0x7f2a8002d890 comp rx=0 tx=0).stop 2024-04-22T13:26:55.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.111+0000 7f2a92759640 1 -- 172.21.15.12:0/3017126089 shutdown_connections 2024-04-22T13:26:55.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.111+0000 7f2a92759640 1 --2- 172.21.15.12:0/3017126089 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2a740a3d20 0x7f2a740a61e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.111+0000 7f2a92759640 1 --2- 172.21.15.12:0/3017126089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a8c10c570 0x7f2a8c1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.111+0000 7f2a92759640 1 --2- 172.21.15.12:0/3017126089 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a8c103970 0x7f2a8c19e940 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.111+0000 7f2a92759640 1 --2- 172.21.15.12:0/3017126089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a8c103050 0x7f2a8c19e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.111+0000 7f2a92759640 1 -- 172.21.15.12:0/3017126089 >> 172.21.15.12:0/3017126089 conn(0x7f2a8c077fb0 msgr2=0x7f2a8c0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:55.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.111+0000 7f2a92759640 1 -- 172.21.15.12:0/3017126089 shutdown_connections 2024-04-22T13:26:55.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.111+0000 7f2a92759640 1 -- 172.21.15.12:0/3017126089 wait complete. 2024-04-22T13:26:55.118 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:26:55.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.226+0000 7f0969985640 1 -- 172.21.15.12:0/1635822826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0964102b40 msgr2=0x7f0964106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.226+0000 7f0969985640 1 --2- 172.21.15.12:0/1635822826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0964102b40 0x7f0964106b80 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f0958009900 tx=0x7f095802bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:55.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.226+0000 7f0969985640 1 -- 172.21.15.12:0/1635822826 shutdown_connections 2024-04-22T13:26:55.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.226+0000 7f0969985640 1 --2- 172.21.15.12:0/1635822826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0964107260 0x7f096410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.226+0000 7f0969985640 1 --2- 172.21.15.12:0/1635822826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0964102b40 0x7f0964106b80 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.226+0000 7f0969985640 1 --2- 172.21.15.12:0/1635822826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0964102190 0x7f0964102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.226+0000 7f0969985640 1 -- 172.21.15.12:0/1635822826 >> 172.21.15.12:0/1635822826 conn(0x7f09640fd990 msgr2=0x7f09640ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:55.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.226+0000 7f0969985640 1 -- 172.21.15.12:0/1635822826 shutdown_connections 2024-04-22T13:26:55.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.226+0000 7f0969985640 1 -- 172.21.15.12:0/1635822826 wait complete. 2024-04-22T13:26:55.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.227+0000 7f0969985640 1 Processor -- start 2024-04-22T13:26:55.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.227+0000 7f0969985640 1 -- start start 2024-04-22T13:26:55.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.227+0000 7f0969985640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0964102190 0x7f096419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.227+0000 7f0969985640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0964107260 0x7f096419e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.227+0000 7f0969985640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0964198590 0x7f0964198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.227+0000 7f0969985640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f09641121d0 con 0x7f0964107260 2024-04-22T13:26:55.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.227+0000 7f0969985640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0964112050 con 0x7f0964102190 2024-04-22T13:26:55.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.227+0000 7f0969985640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0964112350 con 0x7f0964198590 2024-04-22T13:26:55.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.227+0000 7f09627fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0964107260 0x7f096419e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.227+0000 7f09627fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0964107260 0x7f096419e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36090/0 (socket says 172.21.15.12:36090) 2024-04-22T13:26:55.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.227+0000 7f09627fc640 1 -- 172.21.15.12:0/3666150070 learned_addr learned my addr 172.21.15.12:0/3666150070 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:55.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.227+0000 7f0962ffd640 1 --2- 172.21.15.12:0/3666150070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0964102190 0x7f096419e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.228+0000 7f09637fe640 1 --2- 172.21.15.12:0/3666150070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0964198590 0x7f0964198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.228+0000 7f09627fc640 1 -- 172.21.15.12:0/3666150070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0964198590 msgr2=0x7f0964198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.228+0000 7f09627fc640 1 --2- 172.21.15.12:0/3666150070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0964198590 0x7f0964198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.228+0000 7f09627fc640 1 -- 172.21.15.12:0/3666150070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0964102190 msgr2=0x7f096419e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.228+0000 7f09627fc640 1 --2- 172.21.15.12:0/3666150070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0964102190 0x7f096419e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.228+0000 7f09627fc640 1 -- 172.21.15.12:0/3666150070 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09641990b0 con 0x7f0964107260 2024-04-22T13:26:55.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.228+0000 7f09627fc640 1 --2- 172.21.15.12:0/3666150070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0964107260 0x7f096419e920 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f0958000c00 tx=0x7f095802c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.228+0000 7f0968983640 1 -- 172.21.15.12:0/3666150070 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0958043020 con 0x7f0964107260 2024-04-22T13:26:55.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.228+0000 7f0968983640 1 -- 172.21.15.12:0/3666150070 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f095802caf0 con 0x7f0964107260 2024-04-22T13:26:55.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.228+0000 7f0968983640 1 -- 172.21.15.12:0/3666150070 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0958003b20 con 0x7f0964107260 2024-04-22T13:26:55.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.228+0000 7f0969985640 1 -- 172.21.15.12:0/3666150070 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0964199340 con 0x7f0964107260 2024-04-22T13:26:55.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.228+0000 7f0969985640 1 -- 172.21.15.12:0/3666150070 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09641a5590 con 0x7f0964107260 2024-04-22T13:26:55.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.230+0000 7f0968983640 1 -- 172.21.15.12:0/3666150070 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0958046020 con 0x7f0964107260 2024-04-22T13:26:55.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.231+0000 7f096c575740 1 Processor -- start 2024-04-22T13:26:55.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.231+0000 7f096c575740 1 -- start start 2024-04-22T13:26:55.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.231+0000 7f096c575740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603dfaa24d0 0x5603dfaa28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.231+0000 7f096c575740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603dfaa2e80 0x5603dfaab430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.231+0000 7f096c575740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603dfaaba00 0x5603dfaade50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.231+0000 7f096c575740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5603dfa071f0 con 0x5603dfaa2e80 2024-04-22T13:26:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.231+0000 7f096c575740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5603df9a44b0 con 0x5603dfaaba00 2024-04-22T13:26:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.231+0000 7f096c575740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5603dfa031b0 con 0x5603dfaa24d0 2024-04-22T13:26:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.231+0000 7f0962ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603dfaa2e80 0x5603dfaab430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.231+0000 7f0968983640 1 --2- 172.21.15.12:0/3666150070 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f094c0a3d20 0x7f094c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.231+0000 7f0962ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603dfaa2e80 0x5603dfaab430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36094/0 (socket says 172.21.15.12:36094) 2024-04-22T13:26:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.231+0000 7f0962ffd640 1 -- 172.21.15.12:0/959779114 learned_addr learned my addr 172.21.15.12:0/959779114 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.231+0000 7f09637fe640 1 --2- 172.21.15.12:0/959779114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603dfaa24d0 0x5603dfaa28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.231+0000 7f09637fe640 1 --2- 172.21.15.12:0/3666150070 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f094c0a3d20 0x7f094c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.231+0000 7f0968983640 1 -- 172.21.15.12:0/3666150070 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f09580f2660 con 0x7f0964107260 2024-04-22T13:26:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.231+0000 7f09627fc640 1 --2- 172.21.15.12:0/959779114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603dfaaba00 0x5603dfaade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.231+0000 7f0962ffd640 1 -- 172.21.15.12:0/959779114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603dfaa24d0 msgr2=0x5603dfaa28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.231+0000 7f0962ffd640 1 --2- 172.21.15.12:0/959779114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603dfaa24d0 0x5603dfaa28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.231+0000 7f0962ffd640 1 -- 172.21.15.12:0/959779114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603dfaaba00 msgr2=0x5603dfaade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.231+0000 7f0962ffd640 1 --2- 172.21.15.12:0/959779114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603dfaaba00 0x5603dfaade50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.231+0000 7f0962ffd640 1 -- 172.21.15.12:0/959779114 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603df9961b0 con 0x5603dfaa2e80 2024-04-22T13:26:55.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.231+0000 7f09637fe640 1 --2- 172.21.15.12:0/959779114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603dfaa24d0 0x5603dfaa28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:26:55.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.231+0000 7f09627fc640 1 --2- 172.21.15.12:0/959779114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603dfaaba00 0x5603dfaade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:26:55.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.232+0000 7f0962ffd640 1 --2- 172.21.15.12:0/959779114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603dfaa2e80 0x5603dfaab430 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f0954009cf0 tx=0x7f0954011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba1f59c7d83d893 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.232+0000 7f09637fe640 1 --2- 172.21.15.12:0/3666150070 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f094c0a3d20 0x7f094c0a61e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f095c005fd0 tx=0x7f095c009500 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.232+0000 7f0949ffb640 1 -- 172.21.15.12:0/959779114 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954004230 con 0x5603dfaa2e80 2024-04-22T13:26:55.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.232+0000 7f0949ffb640 1 -- 172.21.15.12:0/959779114 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09540043d0 con 0x5603dfaa2e80 2024-04-22T13:26:55.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.232+0000 7f0949ffb640 1 -- 172.21.15.12:0/959779114 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954019880 con 0x5603dfaa2e80 2024-04-22T13:26:55.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.232+0000 7f096c575740 1 -- 172.21.15.12:0/959779114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603dfaa2e80 msgr2=0x5603dfaab430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.232+0000 7f096c575740 1 --2- 172.21.15.12:0/959779114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603dfaa2e80 0x5603dfaab430 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f0954009cf0 tx=0x7f0954011810 comp rx=0 tx=0).stop 2024-04-22T13:26:55.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.232+0000 7f096c575740 1 -- 172.21.15.12:0/959779114 shutdown_connections 2024-04-22T13:26:55.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.232+0000 7f096c575740 1 --2- 172.21.15.12:0/959779114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603dfaaba00 0x5603dfaade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.232+0000 7f096c575740 1 --2- 172.21.15.12:0/959779114 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603dfaa2e80 0x5603dfaab430 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.232+0000 7f096c575740 1 --2- 172.21.15.12:0/959779114 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603dfaa24d0 0x5603dfaa28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.233+0000 7f096c575740 1 -- 172.21.15.12:0/959779114 >> 172.21.15.12:0/959779114 conn(0x5603dfa9cba0 msgr2=0x5603dfa9cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.233+0000 7f096c575740 1 -- 172.21.15.12:0/959779114 shutdown_connections 2024-04-22T13:26:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.233+0000 7f096c575740 1 -- 172.21.15.12:0/959779114 wait complete. 2024-04-22T13:26:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.233+0000 7f096c575740 1 -- start start 2024-04-22T13:26:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.233+0000 7f096c575740 1 Processor -- start 2024-04-22T13:26:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.233+0000 7f096c575740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603dfaa24d0 0x5603dfaab2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.233+0000 7f096c575740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603dfaa2e80 0x5603dfaa8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.233+0000 7f096c575740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603dfaaba00 0x5603dfaa8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.233+0000 7f096c575740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5603dfaae970 con 0x5603dfaaba00 2024-04-22T13:26:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.233+0000 7f096c575740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5603df9c0500 con 0x5603dfaa2e80 2024-04-22T13:26:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.233+0000 7f096c575740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5603dfaaeaf0 con 0x5603dfaa24d0 2024-04-22T13:26:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.233+0000 7f096c575740 10 client.0 _ll_register_callbacks cb 0x5603df98eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.234+0000 7f09637fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603dfaaba00 0x5603dfaa8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.234+0000 7f09637fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603dfaaba00 0x5603dfaa8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36102/0 (socket says 172.21.15.12:36102) 2024-04-22T13:26:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.234+0000 7f09637fe640 1 -- 172.21.15.12:0/778052155 learned_addr learned my addr 172.21.15.12:0/778052155 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.234+0000 7f09627fc640 1 --2- 172.21.15.12:0/778052155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603dfaa2e80 0x5603dfaa8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.234+0000 7f0962ffd640 1 --2- 172.21.15.12:0/778052155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603dfaa24d0 0x5603dfaab2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.234+0000 7f09637fe640 1 -- 172.21.15.12:0/778052155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603dfaa24d0 msgr2=0x5603dfaab2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.234+0000 7f09637fe640 1 --2- 172.21.15.12:0/778052155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603dfaa24d0 0x5603dfaab2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.234+0000 7f09637fe640 1 -- 172.21.15.12:0/778052155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603dfaa2e80 msgr2=0x5603dfaa8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.234+0000 7f09637fe640 1 --2- 172.21.15.12:0/778052155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603dfaa2e80 0x5603dfaa8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.234+0000 7f09637fe640 1 -- 172.21.15.12:0/778052155 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603df7193f0 con 0x5603dfaaba00 2024-04-22T13:26:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.234+0000 7f09627fc640 1 --2- 172.21.15.12:0/778052155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603dfaa2e80 0x5603dfaa8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:26:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.234+0000 7f09637fe640 1 --2- 172.21.15.12:0/778052155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603dfaaba00 0x5603dfaa8bc0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f095c010e40 tx=0x7f095c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d2e8baf010a9cc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.234+0000 7f0942ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:26:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.234+0000 7f0942ffd640 1 -- 172.21.15.12:0/778052155 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c00edf0 con 0x5603dfaaba00 2024-04-22T13:26:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.234+0000 7f0942ffd640 1 -- 172.21.15.12:0/778052155 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f095c01e410 con 0x5603dfaaba00 2024-04-22T13:26:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.234+0000 7f096c575740 1 -- 172.21.15.12:0/778052155 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09540043d0 con 0x5603dfaaba00 2024-04-22T13:26:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.234+0000 7f0942ffd640 1 -- 172.21.15.12:0/778052155 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f095c01e610 con 0x5603dfaaba00 2024-04-22T13:26:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.234+0000 7f096c575740 10 client.16779 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.234+0000 7f096c575740 1 -- 172.21.15.12:0/778052155 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5603df7e37d0 con 0x5603dfaaba00 2024-04-22T13:26:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.235+0000 7f0942ffd640 1 -- 172.21.15.12:0/778052155 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09540043d0 con 0x5603dfaaba00 2024-04-22T13:26:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.235+0000 7f0942ffd640 1 -- 172.21.15.12:0/778052155 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f095c0288f0 con 0x5603dfaaba00 2024-04-22T13:26:55.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.235+0000 7f096c575740 10 client.16779 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:55.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.235+0000 7f096c575740 10 client.16779 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:26:55.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.235+0000 7f096c575740 20 client.16779 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:55.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.235+0000 7f096c575740 1 --2- 172.21.15.12:0/778052155 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5603dfabbb80 0x5603dfabdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.235+0000 7f096c575740 4 client.16779 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:26:55.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.235+0000 7f096c575740 1 -- 172.21.15.12:0/778052155 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603df9a6930 con 0x5603dfabbb80 2024-04-22T13:26:55.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.235+0000 7f0962ffd640 1 --2- 172.21.15.12:0/778052155 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5603dfabbb80 0x5603dfabdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.235+0000 7f0962ffd640 1 --2- 172.21.15.12:0/778052155 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5603dfabbb80 0x5603dfabdfd0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=562ba6483eb68972 server_cookie=d9e1364739e3b726 in_seq=0 out_seq=0 2024-04-22T13:26:55.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.235+0000 7f0942ffd640 10 client.16779 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:26:55.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.236+0000 7f0942ffd640 1 -- 172.21.15.12:0/778052155 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5603df9a6930 con 0x5603dfabbb80 2024-04-22T13:26:55.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.236+0000 7f0942ffd640 10 client.16779 handle_command_reply: tid=0 2024-04-22T13:26:55.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.236+0000 7f096c575740 1 client.16779 shutdown 2024-04-22T13:26:55.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.236+0000 7f096c575740 1 -- 172.21.15.12:0/778052155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603dfaaba00 msgr2=0x5603dfaa8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.236+0000 7f096c575740 1 --2- 172.21.15.12:0/778052155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603dfaaba00 0x5603dfaa8bc0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f095c010e40 tx=0x7f095c004a40 comp rx=0 tx=0).stop 2024-04-22T13:26:55.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.236+0000 7f096c575740 1 -- 172.21.15.12:0/778052155 shutdown_connections 2024-04-22T13:26:55.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.236+0000 7f096c575740 1 --2- 172.21.15.12:0/778052155 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5603dfabbb80 0x5603dfabdfd0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.236+0000 7f096c575740 1 --2- 172.21.15.12:0/778052155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603dfaaba00 0x5603dfaa8bc0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.236+0000 7f0942ffd640 0 client.16779 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:26:55.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.236+0000 7f096c575740 1 --2- 172.21.15.12:0/778052155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603dfaa2e80 0x5603dfaa8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.236+0000 7f096c575740 1 --2- 172.21.15.12:0/778052155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603dfaa24d0 0x5603dfaab2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.236+0000 7f096c575740 1 -- 172.21.15.12:0/778052155 >> 172.21.15.12:0/778052155 conn(0x5603dfa9cba0 msgr2=0x5603dfa9e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:55.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.236+0000 7f096c575740 1 -- 172.21.15.12:0/778052155 shutdown_connections 2024-04-22T13:26:55.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.236+0000 7f096c575740 1 -- 172.21.15.12:0/778052155 wait complete. 2024-04-22T13:26:55.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.236+0000 7f096c575740 20 client.16779 trim_cache size 0 max 16384 2024-04-22T13:26:55.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.250+0000 7f096c575740 1 Processor -- start 2024-04-22T13:26:55.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.250+0000 7f096c575740 1 -- start start 2024-04-22T13:26:55.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.250+0000 7f096c575740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603dfaa24d0 0x5603dfab7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.250+0000 7f096c575740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603dfaa2e80 0x5603dfaaad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.250+0000 7f096c575740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603dfaaba00 0x5603dfaab2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.250+0000 7f096c575740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f09641adbb0 con 0x5603dfaa2e80 2024-04-22T13:26:55.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.250+0000 7f096c575740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f09641af6f0 con 0x5603dfaa24d0 2024-04-22T13:26:55.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.250+0000 7f096c575740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f09641ae280 con 0x5603dfaaba00 2024-04-22T13:26:55.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.250+0000 7f09627fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603dfaa2e80 0x5603dfaaad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.250+0000 7f09627fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603dfaa2e80 0x5603dfaaad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36114/0 (socket says 172.21.15.12:36114) 2024-04-22T13:26:55.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.250+0000 7f09627fc640 1 -- 172.21.15.12:0/3714465958 learned_addr learned my addr 172.21.15.12:0/3714465958 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:55.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.250+0000 7f0962ffd640 1 --2- 172.21.15.12:0/3714465958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603dfaa24d0 0x5603dfab7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.250+0000 7f09637fe640 1 --2- 172.21.15.12:0/3714465958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603dfaaba00 0x5603dfaab2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.250+0000 7f09627fc640 1 -- 172.21.15.12:0/3714465958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603dfaaba00 msgr2=0x5603dfaab2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.250+0000 7f09627fc640 1 --2- 172.21.15.12:0/3714465958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603dfaaba00 0x5603dfaab2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.250+0000 7f09627fc640 1 -- 172.21.15.12:0/3714465958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603dfaa24d0 msgr2=0x5603dfab7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.250+0000 7f09627fc640 1 --2- 172.21.15.12:0/3714465958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603dfaa24d0 0x5603dfab7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.250+0000 7f09627fc640 1 -- 172.21.15.12:0/3714465958 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603dfaa6bd0 con 0x5603dfaa2e80 2024-04-22T13:26:55.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.250+0000 7f09627fc640 1 --2- 172.21.15.12:0/3714465958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603dfaa2e80 0x5603dfaaad90 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f09580bb130 tx=0x7f09580a9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26621baf4cb874d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.251+0000 7f09497fa640 1 -- 172.21.15.12:0/3714465958 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09580be0c0 con 0x5603dfaa2e80 2024-04-22T13:26:55.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.251+0000 7f09497fa640 1 -- 172.21.15.12:0/3714465958 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09580bee60 con 0x5603dfaa2e80 2024-04-22T13:26:55.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.251+0000 7f09497fa640 1 -- 172.21.15.12:0/3714465958 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09580badd0 con 0x5603dfaa2e80 2024-04-22T13:26:55.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.251+0000 7f096c575740 1 -- 172.21.15.12:0/3714465958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603dfaa2e80 msgr2=0x5603dfaaad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.251+0000 7f096c575740 1 --2- 172.21.15.12:0/3714465958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603dfaa2e80 0x5603dfaaad90 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f09580bb130 tx=0x7f09580a9040 comp rx=0 tx=0).stop 2024-04-22T13:26:55.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.251+0000 7f096c575740 1 -- 172.21.15.12:0/3714465958 shutdown_connections 2024-04-22T13:26:55.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.251+0000 7f096c575740 1 --2- 172.21.15.12:0/3714465958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603dfaaba00 0x5603dfaab2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.251+0000 7f096c575740 1 --2- 172.21.15.12:0/3714465958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603dfaa2e80 0x5603dfaaad90 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.251+0000 7f096c575740 1 --2- 172.21.15.12:0/3714465958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603dfaa24d0 0x5603dfab7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.251+0000 7f096c575740 1 -- 172.21.15.12:0/3714465958 >> 172.21.15.12:0/3714465958 conn(0x5603dfa9cba0 msgr2=0x5603dfa9bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:55.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.251+0000 7f096c575740 1 -- 172.21.15.12:0/3714465958 shutdown_connections 2024-04-22T13:26:55.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.251+0000 7f096c575740 1 -- 172.21.15.12:0/3714465958 wait complete. 2024-04-22T13:26:55.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.251+0000 7f096c575740 1 -- start start 2024-04-22T13:26:55.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.252+0000 7f096c575740 1 Processor -- start 2024-04-22T13:26:55.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.252+0000 7f096c575740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603dfaa24d0 0x5603dfaa53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.252+0000 7f096c575740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603dfaa2e80 0x5603dfaa5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.252+0000 7f096c575740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603dfaaba00 0x5603dfaa5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.252+0000 7f096c575740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f09641af020 con 0x5603dfaaba00 2024-04-22T13:26:55.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.252+0000 7f096c575740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f09641ae950 con 0x5603dfaa24d0 2024-04-22T13:26:55.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.252+0000 7f096c575740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f09641abbb0 con 0x5603dfaa2e80 2024-04-22T13:26:55.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.252+0000 7f096c575740 10 client.0 _ll_register_callbacks cb 0x7f095c010ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:55.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.252+0000 7f09637fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603dfaaba00 0x5603dfaa5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.252+0000 7f0962ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603dfaa24d0 0x5603dfaa53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.252+0000 7f09637fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603dfaaba00 0x5603dfaa5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36116/0 (socket says 172.21.15.12:36116) 2024-04-22T13:26:55.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.252+0000 7f09637fe640 1 -- 172.21.15.12:0/3147914935 learned_addr learned my addr 172.21.15.12:0/3147914935 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:55.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.252+0000 7f09627fc640 1 --2- 172.21.15.12:0/3147914935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603dfaa2e80 0x5603dfaa5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.252+0000 7f0962ffd640 1 -- 172.21.15.12:0/3147914935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603dfaa2e80 msgr2=0x5603dfaa5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.252+0000 7f0962ffd640 1 --2- 172.21.15.12:0/3147914935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603dfaa2e80 0x5603dfaa5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.252+0000 7f0962ffd640 1 -- 172.21.15.12:0/3147914935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603dfaaba00 msgr2=0x5603dfaa5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.252+0000 7f0962ffd640 1 --2- 172.21.15.12:0/3147914935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603dfaaba00 0x5603dfaa5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.252+0000 7f0962ffd640 1 -- 172.21.15.12:0/3147914935 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603dfb211c0 con 0x5603dfaa24d0 2024-04-22T13:26:55.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.253+0000 7f0962ffd640 1 --2- 172.21.15.12:0/3147914935 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603dfaa24d0 0x5603dfaa53f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x5603dfa9dc10 tx=0x7f0954009f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca4ba0f0ad3af703 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.253+0000 7f0942ffd640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-22T13:26:55.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.253+0000 7f0942ffd640 1 -- 172.21.15.12:0/3147914935 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954030070 con 0x5603dfaa24d0 2024-04-22T13:26:55.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.253+0000 7f096c575740 1 -- 172.21.15.12:0/3147914935 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09580bee60 con 0x5603dfaa24d0 2024-04-22T13:26:55.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.253+0000 7f0942ffd640 1 -- 172.21.15.12:0/3147914935 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f0954005040 con 0x5603dfaa24d0 2024-04-22T13:26:55.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.253+0000 7f0942ffd640 1 -- 172.21.15.12:0/3147914935 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f095403c050 con 0x5603dfaa24d0 2024-04-22T13:26:55.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.253+0000 7f096c575740 10 client.26198 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:55.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.253+0000 7f096c575740 1 -- 172.21.15.12:0/3147914935 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5603dfb22500 con 0x5603dfaa24d0 2024-04-22T13:26:55.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.253+0000 7f0942ffd640 1 -- 172.21.15.12:0/3147914935 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09580bee60 con 0x5603dfaa24d0 2024-04-22T13:26:55.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.253+0000 7f0942ffd640 1 -- 172.21.15.12:0/3147914935 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f095402c7c0 con 0x5603dfaa24d0 2024-04-22T13:26:55.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.253+0000 7f096c575740 10 client.26198 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:55.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.253+0000 7f096c575740 10 client.26198 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:26:55.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.253+0000 7f096c575740 20 client.26198 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:55.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.253+0000 7f096c575740 1 --2- 172.21.15.12:0/3147914935 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5603dfb22a90 0x5603dfb24e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.253+0000 7f096c575740 4 client.26198 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:26:55.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.253+0000 7f096c575740 1 -- 172.21.15.12:0/3147914935 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5603dfa14c10 con 0x5603dfb22a90 2024-04-22T13:26:55.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.254+0000 7f09627fc640 1 --2- 172.21.15.12:0/3147914935 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5603dfb22a90 0x5603dfb24e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.254+0000 7f0942ffd640 10 client.26198 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:26:55.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.254+0000 7f09627fc640 1 --2- 172.21.15.12:0/3147914935 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5603dfb22a90 0x5603dfb24e80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ea075323e5ad3699 server_cookie=e5b37bbfbe38819b in_seq=0 out_seq=0 2024-04-22T13:26:55.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.254+0000 7f0942ffd640 1 -- 172.21.15.12:0/3147914935 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5603dfa14c10 con 0x5603dfb22a90 2024-04-22T13:26:55.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.254+0000 7f0942ffd640 10 client.26198 handle_command_reply: tid=0 2024-04-22T13:26:55.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.254+0000 7f096c575740 1 client.26198 shutdown 2024-04-22T13:26:55.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.254+0000 7f096c575740 1 -- 172.21.15.12:0/3147914935 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603dfaa24d0 msgr2=0x5603dfaa53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.254+0000 7f096c575740 1 --2- 172.21.15.12:0/3147914935 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603dfaa24d0 0x5603dfaa53f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x5603dfa9dc10 tx=0x7f0954009f20 comp rx=0 tx=0).stop 2024-04-22T13:26:55.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.254+0000 7f096c575740 1 -- 172.21.15.12:0/3147914935 shutdown_connections 2024-04-22T13:26:55.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.254+0000 7f096c575740 1 --2- 172.21.15.12:0/3147914935 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5603dfb22a90 0x5603dfb24e80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.254+0000 7f096c575740 1 --2- 172.21.15.12:0/3147914935 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5603dfaaba00 0x5603dfaa5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.254+0000 7f096c575740 1 --2- 172.21.15.12:0/3147914935 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5603dfaa2e80 0x5603dfaa5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.254+0000 7f0942ffd640 0 client.26198 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:26:55.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.254+0000 7f096c575740 1 --2- 172.21.15.12:0/3147914935 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603dfaa24d0 0x5603dfaa53f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.254+0000 7f096c575740 1 -- 172.21.15.12:0/3147914935 >> 172.21.15.12:0/3147914935 conn(0x5603dfa9cba0 msgr2=0x5603dfa9b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:55.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.254+0000 7f096c575740 1 -- 172.21.15.12:0/3147914935 shutdown_connections 2024-04-22T13:26:55.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.254+0000 7f096c575740 1 -- 172.21.15.12:0/3147914935 wait complete. 2024-04-22T13:26:55.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.254+0000 7f096c575740 20 client.26198 trim_cache size 0 max 16384 2024-04-22T13:26:55.257 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:26:55.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.255+0000 7f0969985640 1 -- 172.21.15.12:0/3666150070 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f094c0a3d20 msgr2=0x7f094c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.255+0000 7f0969985640 1 --2- 172.21.15.12:0/3666150070 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f094c0a3d20 0x7f094c0a61e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f095c005fd0 tx=0x7f095c009500 comp rx=0 tx=0).stop 2024-04-22T13:26:55.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.255+0000 7f0969985640 1 -- 172.21.15.12:0/3666150070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0964107260 msgr2=0x7f096419e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.255+0000 7f0969985640 1 --2- 172.21.15.12:0/3666150070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0964107260 0x7f096419e920 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f0958000c00 tx=0x7f095802c660 comp rx=0 tx=0).stop 2024-04-22T13:26:55.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.255+0000 7f0969985640 1 -- 172.21.15.12:0/3666150070 shutdown_connections 2024-04-22T13:26:55.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.255+0000 7f0969985640 1 --2- 172.21.15.12:0/3666150070 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f094c0a3d20 0x7f094c0a61e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.255+0000 7f0969985640 1 --2- 172.21.15.12:0/3666150070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0964198590 0x7f0964198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.255+0000 7f0969985640 1 --2- 172.21.15.12:0/3666150070 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0964107260 0x7f096419e920 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.255+0000 7f0969985640 1 --2- 172.21.15.12:0/3666150070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0964102190 0x7f096419e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.255+0000 7f0969985640 1 -- 172.21.15.12:0/3666150070 >> 172.21.15.12:0/3666150070 conn(0x7f09640fd990 msgr2=0x7f09640fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:55.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.255+0000 7f0969985640 1 -- 172.21.15.12:0/3666150070 shutdown_connections 2024-04-22T13:26:55.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.255+0000 7f0969985640 1 -- 172.21.15.12:0/3666150070 wait complete. 2024-04-22T13:26:55.262 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:26:55.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.370+0000 7f2bc45b6640 1 -- 172.21.15.12:0/1429346980 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bbc101090 msgr2=0x7f2bbc101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.370+0000 7f2bc45b6640 1 --2- 172.21.15.12:0/1429346980 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bbc101090 0x7f2bbc101470 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8009900 tx=0x7f2bb802bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:55.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.370+0000 7f2bc45b6640 1 -- 172.21.15.12:0/1429346980 shutdown_connections 2024-04-22T13:26:55.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.370+0000 7f2bc45b6640 1 --2- 172.21.15.12:0/1429346980 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bbc106160 0x7f2bbc10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.370+0000 7f2bc45b6640 1 --2- 172.21.15.12:0/1429346980 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2bbc101a40 0x7f2bbc105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.370+0000 7f2bc45b6640 1 --2- 172.21.15.12:0/1429346980 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bbc101090 0x7f2bbc101470 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.370+0000 7f2bc45b6640 1 -- 172.21.15.12:0/1429346980 >> 172.21.15.12:0/1429346980 conn(0x7f2bbc077f50 msgr2=0x7f2bbc078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:55.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.370+0000 7f2bc45b6640 1 -- 172.21.15.12:0/1429346980 shutdown_connections 2024-04-22T13:26:55.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.370+0000 7f2bc45b6640 1 -- 172.21.15.12:0/1429346980 wait complete. 2024-04-22T13:26:55.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.370+0000 7f2bc45b6640 1 Processor -- start 2024-04-22T13:26:55.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.370+0000 7f2bc45b6640 1 -- start start 2024-04-22T13:26:55.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.371+0000 7f2bc45b6640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2bbc101090 0x7f2bbc19e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.371+0000 7f2bc45b6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bbc106160 0x7f2bbc19e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.371+0000 7f2bc45b6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bbc1985c0 0x7f2bbc198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.371+0000 7f2bc45b6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc1110d0 con 0x7f2bbc1985c0 2024-04-22T13:26:55.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.371+0000 7f2bc45b6640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc110f50 con 0x7f2bbc101090 2024-04-22T13:26:55.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.371+0000 7f2bc45b6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc111250 con 0x7f2bbc106160 2024-04-22T13:26:55.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.371+0000 7f2bc2b53640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bbc1985c0 0x7f2bbc198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.371+0000 7f2bc2b53640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bbc1985c0 0x7f2bbc198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36134/0 (socket says 172.21.15.12:36134) 2024-04-22T13:26:55.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.371+0000 7f2bc2b53640 1 -- 172.21.15.12:0/3951771655 learned_addr learned my addr 172.21.15.12:0/3951771655 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:55.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.371+0000 7f2bc2352640 1 --2- 172.21.15.12:0/3951771655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2bbc101090 0x7f2bbc19e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.371+0000 7f2bc1b51640 1 --2- 172.21.15.12:0/3951771655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bbc106160 0x7f2bbc19e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.371+0000 7f2bc2b53640 1 -- 172.21.15.12:0/3951771655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bbc106160 msgr2=0x7f2bbc19e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.371+0000 7f2bc2b53640 1 --2- 172.21.15.12:0/3951771655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bbc106160 0x7f2bbc19e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.371+0000 7f2bc2b53640 1 -- 172.21.15.12:0/3951771655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2bbc101090 msgr2=0x7f2bbc19e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.371+0000 7f2bc2b53640 1 --2- 172.21.15.12:0/3951771655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2bbc101090 0x7f2bbc19e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.371+0000 7f2bc2b53640 1 -- 172.21.15.12:0/3951771655 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bbc1990e0 con 0x7f2bbc1985c0 2024-04-22T13:26:55.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.371+0000 7f2bc1b51640 1 --2- 172.21.15.12:0/3951771655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bbc106160 0x7f2bbc19e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:26:55.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.371+0000 7f2bc2b53640 1 --2- 172.21.15.12:0/3951771655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bbc1985c0 0x7f2bbc198a70 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f2bb400eb10 tx=0x7f2bb400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.371+0000 7f2bb37fe640 1 -- 172.21.15.12:0/3951771655 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb4004b20 con 0x7f2bbc1985c0 2024-04-22T13:26:55.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.371+0000 7f2bc45b6640 1 -- 172.21.15.12:0/3951771655 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bbc1993d0 con 0x7f2bbc1985c0 2024-04-22T13:26:55.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.371+0000 7f2bb37fe640 1 -- 172.21.15.12:0/3951771655 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bb4004cc0 con 0x7f2bbc1985c0 2024-04-22T13:26:55.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.371+0000 7f2bb37fe640 1 -- 172.21.15.12:0/3951771655 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb4010670 con 0x7f2bbc1985c0 2024-04-22T13:26:55.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.371+0000 7f2bc45b6640 1 -- 172.21.15.12:0/3951771655 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bbc1a5620 con 0x7f2bbc1985c0 2024-04-22T13:26:55.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.374+0000 7f2bb37fe640 1 -- 172.21.15.12:0/3951771655 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2bb4005020 con 0x7f2bbc1985c0 2024-04-22T13:26:55.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.374+0000 7f2bc71aa740 1 Processor -- start 2024-04-22T13:26:55.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.374+0000 7f2bc71aa740 1 -- start start 2024-04-22T13:26:55.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.374+0000 7f2bc71aa740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56136ae1a4d0 0x56136ae1a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.374+0000 7f2bc71aa740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56136ae1ae80 0x56136ae23430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.374+0000 7f2bc71aa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56136ae23a00 0x56136ae25e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.374+0000 7f2bc71aa740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56136ad7f1f0 con 0x56136ae1a4d0 2024-04-22T13:26:55.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.374+0000 7f2bc71aa740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56136ad1c4b0 con 0x56136ae1ae80 2024-04-22T13:26:55.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.374+0000 7f2bc71aa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56136ad7b1b0 con 0x56136ae23a00 2024-04-22T13:26:55.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.374+0000 7f2bc1b51640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56136ae1a4d0 0x56136ae1a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.375+0000 7f2bc1b51640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56136ae1a4d0 0x56136ae1a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36146/0 (socket says 172.21.15.12:36146) 2024-04-22T13:26:55.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.375+0000 7f2bc1b51640 1 -- 172.21.15.12:0/1092661306 learned_addr learned my addr 172.21.15.12:0/1092661306 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:55.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.375+0000 7f2bc2352640 1 --2- 172.21.15.12:0/1092661306 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56136ae1ae80 0x56136ae23430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.375+0000 7f2bb37fe640 1 --2- 172.21.15.12:0/3951771655 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2ba80a3d20 0x7f2ba80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.375+0000 7f2bc1b51640 1 --2- 172.21.15.12:0/1092661306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56136ae23a00 0x56136ae25e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.375+0000 7f2bc2b53640 1 --2- 172.21.15.12:0/3951771655 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2ba80a3d20 0x7f2ba80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.375+0000 7f2bc1b51640 1 -- 172.21.15.12:0/1092661306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56136ae23a00 msgr2=0x56136ae25e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.375+0000 7f2bc1b51640 1 --2- 172.21.15.12:0/1092661306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56136ae23a00 0x56136ae25e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.375+0000 7f2bc1b51640 1 -- 172.21.15.12:0/1092661306 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56136ae1ae80 msgr2=0x56136ae23430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.375+0000 7f2bc1b51640 1 --2- 172.21.15.12:0/1092661306 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56136ae1ae80 0x56136ae23430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.375+0000 7f2bc1b51640 1 -- 172.21.15.12:0/1092661306 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56136ad0e1b0 con 0x56136ae1a4d0 2024-04-22T13:26:55.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.375+0000 7f2bc2352640 1 --2- 172.21.15.12:0/1092661306 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56136ae1ae80 0x56136ae23430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:55.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.375+0000 7f2bb37fe640 1 -- 172.21.15.12:0/3951771655 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f2bb40d0da0 con 0x7f2bbc1985c0 2024-04-22T13:26:55.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.375+0000 7f2bc1b51640 1 --2- 172.21.15.12:0/1092661306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56136ae1a4d0 0x56136ae1a8b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f2bac009fc0 tx=0x7f2bac009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b29a1d0c812e77 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.375+0000 7f2bb0bfe640 1 -- 172.21.15.12:0/1092661306 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac004030 con 0x56136ae1a4d0 2024-04-22T13:26:55.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.375+0000 7f2bb0bfe640 1 -- 172.21.15.12:0/1092661306 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bac0035f0 con 0x56136ae1a4d0 2024-04-22T13:26:55.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.375+0000 7f2bc2b53640 1 --2- 172.21.15.12:0/3951771655 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2ba80a3d20 0x7f2ba80a61e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4099190 tx=0x7f2bb400efd0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.375+0000 7f2bb0bfe640 1 -- 172.21.15.12:0/1092661306 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac0116d0 con 0x56136ae1a4d0 2024-04-22T13:26:55.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.376+0000 7f2bc71aa740 1 -- 172.21.15.12:0/1092661306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56136ae1a4d0 msgr2=0x56136ae1a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.376+0000 7f2bc71aa740 1 --2- 172.21.15.12:0/1092661306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56136ae1a4d0 0x56136ae1a8b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f2bac009fc0 tx=0x7f2bac009660 comp rx=0 tx=0).stop 2024-04-22T13:26:55.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.376+0000 7f2bc71aa740 1 -- 172.21.15.12:0/1092661306 shutdown_connections 2024-04-22T13:26:55.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.376+0000 7f2bc71aa740 1 --2- 172.21.15.12:0/1092661306 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56136ae23a00 0x56136ae25e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.376+0000 7f2bc71aa740 1 --2- 172.21.15.12:0/1092661306 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56136ae1ae80 0x56136ae23430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.376+0000 7f2bc71aa740 1 --2- 172.21.15.12:0/1092661306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56136ae1a4d0 0x56136ae1a8b0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.376+0000 7f2bc71aa740 1 -- 172.21.15.12:0/1092661306 >> 172.21.15.12:0/1092661306 conn(0x56136ae14ba0 msgr2=0x56136ae14f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:55.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.376+0000 7f2bc71aa740 1 -- 172.21.15.12:0/1092661306 shutdown_connections 2024-04-22T13:26:55.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.376+0000 7f2bc71aa740 1 -- 172.21.15.12:0/1092661306 wait complete. 2024-04-22T13:26:55.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.376+0000 7f2bc71aa740 1 -- start start 2024-04-22T13:26:55.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.377+0000 7f2bc71aa740 1 Processor -- start 2024-04-22T13:26:55.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.377+0000 7f2bc71aa740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56136ae1a4d0 0x56136ae23300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.377+0000 7f2bc71aa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56136ae1ae80 0x56136ae20690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.377+0000 7f2bc71aa740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56136ae23a00 0x56136ae20c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.377+0000 7f2bc71aa740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56136ae26970 con 0x56136ae23a00 2024-04-22T13:26:55.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.377+0000 7f2bc71aa740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56136ad38500 con 0x56136ae1a4d0 2024-04-22T13:26:55.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.377+0000 7f2bc71aa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56136ae26af0 con 0x56136ae1ae80 2024-04-22T13:26:55.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.377+0000 7f2bc71aa740 10 client.0 _ll_register_callbacks cb 0x56136ad06b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:55.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.377+0000 7f2bc1b51640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56136ae23a00 0x56136ae20c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.377+0000 7f2bc1b51640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56136ae23a00 0x56136ae20c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36162/0 (socket says 172.21.15.12:36162) 2024-04-22T13:26:55.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.377+0000 7f2bc1b51640 1 -- 172.21.15.12:0/1939950456 learned_addr learned my addr 172.21.15.12:0/1939950456 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:55.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.377+0000 7f2bc1b51640 1 --2- 172.21.15.12:0/1939950456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56136ae1a4d0 0x56136ae23300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.377+0000 7f2bc2352640 1 --2- 172.21.15.12:0/1939950456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56136ae1ae80 0x56136ae20690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.377+0000 7f2bc1b51640 1 -- 172.21.15.12:0/1939950456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56136ae1ae80 msgr2=0x56136ae20690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.377+0000 7f2bc1b51640 1 --2- 172.21.15.12:0/1939950456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56136ae1ae80 0x56136ae20690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.377+0000 7f2bc1b51640 1 -- 172.21.15.12:0/1939950456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56136ae1a4d0 msgr2=0x56136ae23300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.377+0000 7f2bc1b51640 1 --2- 172.21.15.12:0/1939950456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56136ae1a4d0 0x56136ae23300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.377+0000 7f2bc1b51640 1 -- 172.21.15.12:0/1939950456 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56136aa913f0 con 0x56136ae23a00 2024-04-22T13:26:55.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.377+0000 7f2bc1b51640 1 --2- 172.21.15.12:0/1939950456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56136ae23a00 0x56136ae20c10 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f2bac009550 tx=0x7f2bac009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a744be828aca50 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.377+0000 7f2ba5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:26:55.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.378+0000 7f2ba5ffb640 1 -- 172.21.15.12:0/1939950456 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac00fd90 con 0x56136ae23a00 2024-04-22T13:26:55.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.378+0000 7f2ba5ffb640 1 -- 172.21.15.12:0/1939950456 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f2bac005030 con 0x56136ae23a00 2024-04-22T13:26:55.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.378+0000 7f2bc71aa740 1 -- 172.21.15.12:0/1939950456 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bac0035f0 con 0x56136ae23a00 2024-04-22T13:26:55.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.378+0000 7f2ba5ffb640 1 -- 172.21.15.12:0/1939950456 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2bac005230 con 0x56136ae23a00 2024-04-22T13:26:55.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.378+0000 7f2bc71aa740 10 client.16797 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:55.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.378+0000 7f2bc71aa740 1 -- 172.21.15.12:0/1939950456 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56136ab5b7d0 con 0x56136ae23a00 2024-04-22T13:26:55.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.378+0000 7f2ba5ffb640 1 -- 172.21.15.12:0/1939950456 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bac0035f0 con 0x56136ae23a00 2024-04-22T13:26:55.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.378+0000 7f2ba5ffb640 1 -- 172.21.15.12:0/1939950456 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2bac011470 con 0x56136ae23a00 2024-04-22T13:26:55.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.378+0000 7f2bc71aa740 10 client.16797 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:55.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.378+0000 7f2bc71aa740 10 client.16797 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:26:55.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.378+0000 7f2bc71aa740 20 client.16797 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:55.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.378+0000 7f2bc71aa740 1 --2- 172.21.15.12:0/1939950456 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x56136ae33a80 0x56136ae35e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.378+0000 7f2bc71aa740 4 client.16797 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:26:55.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.378+0000 7f2bc71aa740 1 -- 172.21.15.12:0/1939950456 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56136ad1e930 con 0x56136ae33a80 2024-04-22T13:26:55.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.378+0000 7f2bc2352640 1 --2- 172.21.15.12:0/1939950456 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x56136ae33a80 0x56136ae35e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.379+0000 7f2bc2352640 1 --2- 172.21.15.12:0/1939950456 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x56136ae33a80 0x56136ae35e70 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5c0e5660f25a0010 server_cookie=67460521aedf525c in_seq=0 out_seq=0 2024-04-22T13:26:55.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.379+0000 7f2ba5ffb640 10 client.16797 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:26:55.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.380+0000 7f2ba5ffb640 1 -- 172.21.15.12:0/1939950456 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56136ad1e930 con 0x56136ae33a80 2024-04-22T13:26:55.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.380+0000 7f2ba5ffb640 10 client.16797 handle_command_reply: tid=0 2024-04-22T13:26:55.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.380+0000 7f2bc71aa740 1 client.16797 shutdown 2024-04-22T13:26:55.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.380+0000 7f2bc71aa740 1 -- 172.21.15.12:0/1939950456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56136ae23a00 msgr2=0x56136ae20c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.380+0000 7f2bc71aa740 1 --2- 172.21.15.12:0/1939950456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56136ae23a00 0x56136ae20c10 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f2bac009550 tx=0x7f2bac009ad0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.380+0000 7f2bc71aa740 1 -- 172.21.15.12:0/1939950456 shutdown_connections 2024-04-22T13:26:55.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.380+0000 7f2bc71aa740 1 --2- 172.21.15.12:0/1939950456 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x56136ae33a80 0x56136ae35e70 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.380+0000 7f2bc71aa740 1 --2- 172.21.15.12:0/1939950456 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56136ae23a00 0x56136ae20c10 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.380+0000 7f2ba5ffb640 0 client.16797 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:26:55.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.380+0000 7f2bc71aa740 1 --2- 172.21.15.12:0/1939950456 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56136ae1ae80 0x56136ae20690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.380+0000 7f2bc71aa740 1 --2- 172.21.15.12:0/1939950456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56136ae1a4d0 0x56136ae23300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.380+0000 7f2bc71aa740 1 -- 172.21.15.12:0/1939950456 >> 172.21.15.12:0/1939950456 conn(0x56136ae14ba0 msgr2=0x56136ae168f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:55.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.380+0000 7f2bc71aa740 1 -- 172.21.15.12:0/1939950456 shutdown_connections 2024-04-22T13:26:55.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.380+0000 7f2bc71aa740 1 -- 172.21.15.12:0/1939950456 wait complete. 2024-04-22T13:26:55.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.380+0000 7f2bc71aa740 20 client.16797 trim_cache size 0 max 16384 2024-04-22T13:26:55.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.393+0000 7f2bc71aa740 1 Processor -- start 2024-04-22T13:26:55.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.393+0000 7f2bc71aa740 1 -- start start 2024-04-22T13:26:55.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.393+0000 7f2bc71aa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56136ae1a4d0 0x56136ae2f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.393+0000 7f2bc71aa740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56136ae1ae80 0x56136ae13b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.393+0000 7f2bc71aa740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56136ae23a00 0x56136ae22dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.393+0000 7f2bc71aa740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc1adb00 con 0x56136ae1ae80 2024-04-22T13:26:55.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.393+0000 7f2bc71aa740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc1af640 con 0x56136ae23a00 2024-04-22T13:26:55.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.393+0000 7f2bc71aa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc1ae1d0 con 0x56136ae1a4d0 2024-04-22T13:26:55.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.394+0000 7f2bc2352640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56136ae1ae80 0x56136ae13b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.394+0000 7f2bc2352640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56136ae1ae80 0x56136ae13b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36170/0 (socket says 172.21.15.12:36170) 2024-04-22T13:26:55.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.394+0000 7f2bc2352640 1 -- 172.21.15.12:0/1797596259 learned_addr learned my addr 172.21.15.12:0/1797596259 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:55.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.394+0000 7f2bc1b51640 1 --2- 172.21.15.12:0/1797596259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56136ae1a4d0 0x56136ae2f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.394+0000 7f2bc1b51640 1 --2- 172.21.15.12:0/1797596259 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56136ae23a00 0x56136ae22dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.394+0000 7f2bc2352640 1 -- 172.21.15.12:0/1797596259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56136ae1a4d0 msgr2=0x56136ae2f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.394+0000 7f2bc2352640 1 --2- 172.21.15.12:0/1797596259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56136ae1a4d0 0x56136ae2f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.394+0000 7f2bc2352640 1 -- 172.21.15.12:0/1797596259 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56136ae23a00 msgr2=0x56136ae22dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.394+0000 7f2bc2352640 1 --2- 172.21.15.12:0/1797596259 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56136ae23a00 0x56136ae22dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.394+0000 7f2bc2352640 1 -- 172.21.15.12:0/1797596259 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56136ae19560 con 0x56136ae1ae80 2024-04-22T13:26:55.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.394+0000 7f2bc1b51640 1 --2- 172.21.15.12:0/1797596259 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56136ae23a00 0x56136ae22dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:26:55.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.394+0000 7f2bc2352640 1 --2- 172.21.15.12:0/1797596259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56136ae1ae80 0x56136ae13b40 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8009900 tx=0x7f2bb8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ddc418f9780b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.394+0000 7f2ba7fff640 1 -- 172.21.15.12:0/1797596259 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb80042d0 con 0x56136ae1ae80 2024-04-22T13:26:55.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.394+0000 7f2ba7fff640 1 -- 172.21.15.12:0/1797596259 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bb802cc90 con 0x56136ae1ae80 2024-04-22T13:26:55.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.394+0000 7f2ba7fff640 1 -- 172.21.15.12:0/1797596259 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8004920 con 0x56136ae1ae80 2024-04-22T13:26:55.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.395+0000 7f2bc71aa740 1 -- 172.21.15.12:0/1797596259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56136ae1ae80 msgr2=0x56136ae13b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.395+0000 7f2bc71aa740 1 --2- 172.21.15.12:0/1797596259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56136ae1ae80 0x56136ae13b40 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8009900 tx=0x7f2bb8005fd0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.395+0000 7f2bc71aa740 1 -- 172.21.15.12:0/1797596259 shutdown_connections 2024-04-22T13:26:55.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.395+0000 7f2bc71aa740 1 --2- 172.21.15.12:0/1797596259 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56136ae23a00 0x56136ae22dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.395+0000 7f2bc71aa740 1 --2- 172.21.15.12:0/1797596259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56136ae1ae80 0x56136ae13b40 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.395+0000 7f2bc71aa740 1 --2- 172.21.15.12:0/1797596259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56136ae1a4d0 0x56136ae2f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.395+0000 7f2bc71aa740 1 -- 172.21.15.12:0/1797596259 >> 172.21.15.12:0/1797596259 conn(0x56136ae14ba0 msgr2=0x56136ae16860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:55.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.395+0000 7f2bc71aa740 1 -- 172.21.15.12:0/1797596259 shutdown_connections 2024-04-22T13:26:55.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.395+0000 7f2bc71aa740 1 -- 172.21.15.12:0/1797596259 wait complete. 2024-04-22T13:26:55.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.395+0000 7f2bc71aa740 1 -- start start 2024-04-22T13:26:55.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.395+0000 7f2bc71aa740 1 Processor -- start 2024-04-22T13:26:55.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.395+0000 7f2bc71aa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56136ae1a4d0 0x56136ae2bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.395+0000 7f2bc71aa740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56136ae1ae80 0x56136ae1cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.395+0000 7f2bc71aa740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56136ae23a00 0x56136ae1d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.395+0000 7f2bc71aa740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc1aef70 con 0x56136ae23a00 2024-04-22T13:26:55.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.395+0000 7f2bc71aa740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc1ae8a0 con 0x56136ae1ae80 2024-04-22T13:26:55.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.395+0000 7f2bc71aa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc1abb00 con 0x56136ae1a4d0 2024-04-22T13:26:55.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.395+0000 7f2bc71aa740 10 client.0 _ll_register_callbacks cb 0x7f2bac00f9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:55.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.395+0000 7f2bc1b51640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56136ae23a00 0x56136ae1d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.396+0000 7f2bc1b51640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56136ae23a00 0x56136ae1d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36182/0 (socket says 172.21.15.12:36182) 2024-04-22T13:26:55.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.396+0000 7f2bc1b51640 1 -- 172.21.15.12:0/4133153266 learned_addr learned my addr 172.21.15.12:0/4133153266 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:55.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.396+0000 7f2bc2352640 1 --2- 172.21.15.12:0/4133153266 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56136ae1ae80 0x56136ae1cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.396+0000 7f2bc1b51640 1 --2- 172.21.15.12:0/4133153266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56136ae1a4d0 0x56136ae2bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.396+0000 7f2bc1b51640 1 -- 172.21.15.12:0/4133153266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56136ae1a4d0 msgr2=0x56136ae2bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.396+0000 7f2bc1b51640 1 --2- 172.21.15.12:0/4133153266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56136ae1a4d0 0x56136ae2bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.396+0000 7f2bc1b51640 1 -- 172.21.15.12:0/4133153266 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56136ae1ae80 msgr2=0x56136ae1cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.396+0000 7f2bc1b51640 1 --2- 172.21.15.12:0/4133153266 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56136ae1ae80 0x56136ae1cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.396+0000 7f2bc1b51640 1 -- 172.21.15.12:0/4133153266 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56136ae1d990 con 0x56136ae23a00 2024-04-22T13:26:55.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.396+0000 7f2bc1b51640 1 --2- 172.21.15.12:0/4133153266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56136ae23a00 0x56136ae1d420 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f2bac001eb0 tx=0x7f2bac009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a979729e22c4b6b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.396+0000 7f2ba5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:26:55.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.396+0000 7f2ba5ffb640 1 -- 172.21.15.12:0/4133153266 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac034040 con 0x56136ae23a00 2024-04-22T13:26:55.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.396+0000 7f2ba5ffb640 1 -- 172.21.15.12:0/4133153266 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f2bac00fd90 con 0x56136ae23a00 2024-04-22T13:26:55.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.396+0000 7f2bc71aa740 1 -- 172.21.15.12:0/4133153266 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bb802cc90 con 0x56136ae23a00 2024-04-22T13:26:55.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.396+0000 7f2ba5ffb640 1 -- 172.21.15.12:0/4133153266 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2bac03e050 con 0x56136ae23a00 2024-04-22T13:26:55.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.396+0000 7f2bc71aa740 10 client.16803 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:55.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.396+0000 7f2bc71aa740 1 -- 172.21.15.12:0/4133153266 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56136ae99b80 con 0x56136ae23a00 2024-04-22T13:26:55.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.396+0000 7f2ba5ffb640 1 -- 172.21.15.12:0/4133153266 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bb802cc90 con 0x56136ae23a00 2024-04-22T13:26:55.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.397+0000 7f2ba5ffb640 1 -- 172.21.15.12:0/4133153266 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2bac0035f0 con 0x56136ae23a00 2024-04-22T13:26:55.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.397+0000 7f2bc71aa740 10 client.16803 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:55.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.397+0000 7f2bc71aa740 10 client.16803 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:26:55.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.397+0000 7f2bc71aa740 20 client.16803 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:55.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.397+0000 7f2bc71aa740 1 --2- 172.21.15.12:0/4133153266 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x56136ae9a260 0x56136ae9c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.397+0000 7f2bc71aa740 4 client.16803 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:26:55.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.397+0000 7f2bc71aa740 1 -- 172.21.15.12:0/4133153266 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56136ad8cc10 con 0x56136ae9a260 2024-04-22T13:26:55.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.397+0000 7f2bc2352640 1 --2- 172.21.15.12:0/4133153266 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x56136ae9a260 0x56136ae9c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.397+0000 7f2ba5ffb640 10 client.16803 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:26:55.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.397+0000 7f2bc2352640 1 --2- 172.21.15.12:0/4133153266 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x56136ae9a260 0x56136ae9c680 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fac6a0aa21a58428 server_cookie=d09340e9a4cba66e in_seq=0 out_seq=0 2024-04-22T13:26:55.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.397+0000 7f2ba5ffb640 1 -- 172.21.15.12:0/4133153266 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56136ad8cc10 con 0x56136ae9a260 2024-04-22T13:26:55.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.397+0000 7f2ba5ffb640 10 client.16803 handle_command_reply: tid=0 2024-04-22T13:26:55.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.397+0000 7f2bc71aa740 1 client.16803 shutdown 2024-04-22T13:26:55.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.397+0000 7f2bc71aa740 1 -- 172.21.15.12:0/4133153266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56136ae23a00 msgr2=0x56136ae1d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.397+0000 7f2bc71aa740 1 --2- 172.21.15.12:0/4133153266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56136ae23a00 0x56136ae1d420 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f2bac001eb0 tx=0x7f2bac009ad0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.397+0000 7f2bc71aa740 1 -- 172.21.15.12:0/4133153266 shutdown_connections 2024-04-22T13:26:55.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.398+0000 7f2bc71aa740 1 --2- 172.21.15.12:0/4133153266 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x56136ae9a260 0x56136ae9c680 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.398+0000 7f2bc71aa740 1 --2- 172.21.15.12:0/4133153266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56136ae23a00 0x56136ae1d420 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.398+0000 7f2bc71aa740 1 --2- 172.21.15.12:0/4133153266 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56136ae1ae80 0x56136ae1cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.398+0000 7f2ba5ffb640 0 client.16803 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:26:55.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.398+0000 7f2bc71aa740 1 --2- 172.21.15.12:0/4133153266 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56136ae1a4d0 0x56136ae2bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.398+0000 7f2bc71aa740 1 -- 172.21.15.12:0/4133153266 >> 172.21.15.12:0/4133153266 conn(0x56136ae14ba0 msgr2=0x56136ae1f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:55.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.398+0000 7f2bc71aa740 1 -- 172.21.15.12:0/4133153266 shutdown_connections 2024-04-22T13:26:55.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.398+0000 7f2bc71aa740 1 -- 172.21.15.12:0/4133153266 wait complete. 2024-04-22T13:26:55.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.398+0000 7f2bc71aa740 20 client.16803 trim_cache size 0 max 16384 2024-04-22T13:26:55.399 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:26:55.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.398+0000 7f2bc45b6640 1 -- 172.21.15.12:0/3951771655 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2ba80a3d20 msgr2=0x7f2ba80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.398+0000 7f2bc45b6640 1 --2- 172.21.15.12:0/3951771655 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2ba80a3d20 0x7f2ba80a61e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4099190 tx=0x7f2bb400efd0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.398+0000 7f2bc45b6640 1 -- 172.21.15.12:0/3951771655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bbc1985c0 msgr2=0x7f2bbc198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.398+0000 7f2bc45b6640 1 --2- 172.21.15.12:0/3951771655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bbc1985c0 0x7f2bbc198a70 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f2bb400eb10 tx=0x7f2bb400c2d0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.398+0000 7f2bc45b6640 1 -- 172.21.15.12:0/3951771655 shutdown_connections 2024-04-22T13:26:55.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.398+0000 7f2bc45b6640 1 --2- 172.21.15.12:0/3951771655 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2ba80a3d20 0x7f2ba80a61e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.398+0000 7f2bc45b6640 1 --2- 172.21.15.12:0/3951771655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2bbc1985c0 0x7f2bbc198a70 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.398+0000 7f2bc45b6640 1 --2- 172.21.15.12:0/3951771655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2bbc106160 0x7f2bbc19e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.398+0000 7f2bc45b6640 1 --2- 172.21.15.12:0/3951771655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2bbc101090 0x7f2bbc19e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.398+0000 7f2bc45b6640 1 -- 172.21.15.12:0/3951771655 >> 172.21.15.12:0/3951771655 conn(0x7f2bbc077f50 msgr2=0x7f2bbc0694c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:55.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.398+0000 7f2bc45b6640 1 -- 172.21.15.12:0/3951771655 shutdown_connections 2024-04-22T13:26:55.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.398+0000 7f2bc45b6640 1 -- 172.21.15.12:0/3951771655 wait complete. 2024-04-22T13:26:55.405 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:26:55.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.513+0000 7f66fee3a640 1 -- 172.21.15.12:0/4007744405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66f8107030 msgr2=0x7f66f8107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.513+0000 7f66fee3a640 1 --2- 172.21.15.12:0/4007744405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66f8107030 0x7f66f8107410 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f66e8009930 tx=0x7f66e802bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:55.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.514+0000 7f66fee3a640 1 -- 172.21.15.12:0/4007744405 shutdown_connections 2024-04-22T13:26:55.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.514+0000 7f66fee3a640 1 --2- 172.21.15.12:0/4007744405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66f8101b40 0x7f66f810dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.514+0000 7f66fee3a640 1 --2- 172.21.15.12:0/4007744405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66f81011a0 0x7f66f8101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.514+0000 7f66fee3a640 1 --2- 172.21.15.12:0/4007744405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66f8107030 0x7f66f8107410 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.514+0000 7f66fee3a640 1 -- 172.21.15.12:0/4007744405 >> 172.21.15.12:0/4007744405 conn(0x7f66f8076c50 msgr2=0x7f66f8077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:55.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.514+0000 7f66fee3a640 1 -- 172.21.15.12:0/4007744405 shutdown_connections 2024-04-22T13:26:55.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.514+0000 7f66fee3a640 1 -- 172.21.15.12:0/4007744405 wait complete. 2024-04-22T13:26:55.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.514+0000 7f66fee3a640 1 Processor -- start 2024-04-22T13:26:55.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.514+0000 7f66fee3a640 1 -- start start 2024-04-22T13:26:55.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.514+0000 7f66fee3a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66f81011a0 0x7f66f81995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.514+0000 7f66fee3a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66f8101b40 0x7f66f8199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.514+0000 7f66fee3a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66f8107030 0x7f66f81a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.514+0000 7f66fee3a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f66f81110c0 con 0x7f66f8107030 2024-04-22T13:26:55.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.514+0000 7f66fee3a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f66f8110f40 con 0x7f66f8101b40 2024-04-22T13:26:55.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.514+0000 7f66fee3a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66f8111240 con 0x7f66f81011a0 2024-04-22T13:26:55.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.515+0000 7f66fd3d7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66f8107030 0x7f66f81a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.515+0000 7f66fd3d7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66f8107030 0x7f66f81a4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36206/0 (socket says 172.21.15.12:36206) 2024-04-22T13:26:55.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.515+0000 7f66fd3d7640 1 -- 172.21.15.12:0/747545228 learned_addr learned my addr 172.21.15.12:0/747545228 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:55.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.515+0000 7f66fcbd6640 1 --2- 172.21.15.12:0/747545228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66f81011a0 0x7f66f81995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.515+0000 7f66f7fff640 1 --2- 172.21.15.12:0/747545228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66f8101b40 0x7f66f8199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.515+0000 7f66fd3d7640 1 -- 172.21.15.12:0/747545228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66f81011a0 msgr2=0x7f66f81995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.515+0000 7f66fd3d7640 1 --2- 172.21.15.12:0/747545228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66f81011a0 0x7f66f81995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.515+0000 7f66fd3d7640 1 -- 172.21.15.12:0/747545228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66f8101b40 msgr2=0x7f66f8199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.515+0000 7f66fd3d7640 1 --2- 172.21.15.12:0/747545228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66f8101b40 0x7f66f8199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.515+0000 7f66fd3d7640 1 -- 172.21.15.12:0/747545228 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f81a4b60 con 0x7f66f8107030 2024-04-22T13:26:55.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.515+0000 7f66fcbd6640 1 --2- 172.21.15.12:0/747545228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66f81011a0 0x7f66f81995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:26:55.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.515+0000 7f66f7fff640 1 --2- 172.21.15.12:0/747545228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66f8101b40 0x7f66f8199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:26:55.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.515+0000 7f66fd3d7640 1 --2- 172.21.15.12:0/747545228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66f8107030 0x7f66f81a4620 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f66f000eaa0 tx=0x7f66f000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.515+0000 7f66f5ffb640 1 -- 172.21.15.12:0/747545228 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f000cdb0 con 0x7f66f8107030 2024-04-22T13:26:55.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.515+0000 7f66f5ffb640 1 -- 172.21.15.12:0/747545228 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66f0005020 con 0x7f66f8107030 2024-04-22T13:26:55.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.515+0000 7f66fee3a640 1 -- 172.21.15.12:0/747545228 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66f81a4d30 con 0x7f66f8107030 2024-04-22T13:26:55.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.515+0000 7f66f5ffb640 1 -- 172.21.15.12:0/747545228 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f00116d0 con 0x7f66f8107030 2024-04-22T13:26:55.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.516+0000 7f66fee3a640 1 -- 172.21.15.12:0/747545228 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66f81a5200 con 0x7f66f8107030 2024-04-22T13:26:55.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.518+0000 7f66f5ffb640 1 -- 172.21.15.12:0/747545228 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f66f00051d0 con 0x7f66f8107030 2024-04-22T13:26:55.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.518+0000 7f6701a21740 1 Processor -- start 2024-04-22T13:26:55.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.518+0000 7f6701a21740 1 -- start start 2024-04-22T13:26:55.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.518+0000 7f6701a21740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fd70ff44d0 0x55fd70ff48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.518+0000 7f6701a21740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd70ff4e80 0x55fd70ffd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.518+0000 7f6701a21740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd70ffda00 0x55fd70fffe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.518+0000 7f6701a21740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fd70f591f0 con 0x55fd70ff44d0 2024-04-22T13:26:55.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.518+0000 7f6701a21740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fd70ef64b0 con 0x55fd70ffda00 2024-04-22T13:26:55.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.518+0000 7f6701a21740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fd70f551b0 con 0x55fd70ff4e80 2024-04-22T13:26:55.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.518+0000 7f66f5ffb640 1 --2- 172.21.15.12:0/747545228 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f66e00a3d20 0x7f66e00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.518+0000 7f66f7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fd70ff44d0 0x55fd70ff48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.518+0000 7f66f7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fd70ff44d0 0x55fd70ff48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36216/0 (socket says 172.21.15.12:36216) 2024-04-22T13:26:55.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.518+0000 7f66f7fff640 1 -- 172.21.15.12:0/1965125573 learned_addr learned my addr 172.21.15.12:0/1965125573 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:55.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.518+0000 7f66fd3d7640 1 --2- 172.21.15.12:0/747545228 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f66e00a3d20 0x7f66e00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.519+0000 7f66f5ffb640 1 -- 172.21.15.12:0/747545228 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f66f00d0a00 con 0x7f66f8107030 2024-04-22T13:26:55.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.519+0000 7f66f7fff640 1 --2- 172.21.15.12:0/1965125573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd70ffda00 0x55fd70fffe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.519+0000 7f66fcbd6640 1 --2- 172.21.15.12:0/1965125573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd70ff4e80 0x55fd70ffd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.519+0000 7f66f7fff640 1 -- 172.21.15.12:0/1965125573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd70ff4e80 msgr2=0x55fd70ffd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.519+0000 7f66f7fff640 1 --2- 172.21.15.12:0/1965125573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd70ff4e80 0x55fd70ffd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.519+0000 7f66f7fff640 1 -- 172.21.15.12:0/1965125573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd70ffda00 msgr2=0x55fd70fffe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.519+0000 7f66f7fff640 1 --2- 172.21.15.12:0/1965125573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd70ffda00 0x55fd70fffe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.519+0000 7f66f7fff640 1 -- 172.21.15.12:0/1965125573 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd70ee81b0 con 0x55fd70ff44d0 2024-04-22T13:26:55.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.519+0000 7f66f7fff640 1 --2- 172.21.15.12:0/1965125573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fd70ff44d0 0x55fd70ff48b0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00be80 tx=0x7f66ec00b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d07756d3576bdb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.519+0000 7f66fd3d7640 1 --2- 172.21.15.12:0/747545228 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f66e00a3d20 0x7f66e00a61e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f66f00993c0 tx=0x7f66f00bd040 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.519+0000 7f66deffd640 1 -- 172.21.15.12:0/1965125573 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec019030 con 0x55fd70ff44d0 2024-04-22T13:26:55.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.519+0000 7f66deffd640 1 -- 172.21.15.12:0/1965125573 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66ec013d10 con 0x55fd70ff44d0 2024-04-22T13:26:55.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.519+0000 7f66deffd640 1 -- 172.21.15.12:0/1965125573 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec01d990 con 0x55fd70ff44d0 2024-04-22T13:26:55.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.520+0000 7f6701a21740 1 -- 172.21.15.12:0/1965125573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fd70ff44d0 msgr2=0x55fd70ff48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.520+0000 7f6701a21740 1 --2- 172.21.15.12:0/1965125573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fd70ff44d0 0x55fd70ff48b0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00be80 tx=0x7f66ec00b730 comp rx=0 tx=0).stop 2024-04-22T13:26:55.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.520+0000 7f6701a21740 1 -- 172.21.15.12:0/1965125573 shutdown_connections 2024-04-22T13:26:55.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.520+0000 7f6701a21740 1 --2- 172.21.15.12:0/1965125573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd70ffda00 0x55fd70fffe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.520+0000 7f6701a21740 1 --2- 172.21.15.12:0/1965125573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd70ff4e80 0x55fd70ffd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.520+0000 7f6701a21740 1 --2- 172.21.15.12:0/1965125573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fd70ff44d0 0x55fd70ff48b0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.520+0000 7f6701a21740 1 -- 172.21.15.12:0/1965125573 >> 172.21.15.12:0/1965125573 conn(0x55fd70feeba0 msgr2=0x55fd70feef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:55.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.520+0000 7f6701a21740 1 -- 172.21.15.12:0/1965125573 shutdown_connections 2024-04-22T13:26:55.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.520+0000 7f6701a21740 1 -- 172.21.15.12:0/1965125573 wait complete. 2024-04-22T13:26:55.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.520+0000 7f6701a21740 1 -- start start 2024-04-22T13:26:55.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.520+0000 7f6701a21740 1 Processor -- start 2024-04-22T13:26:55.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.521+0000 7f6701a21740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fd70ff44d0 0x55fd70ffd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.521+0000 7f6701a21740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd70ff4e80 0x55fd70ffa650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.521+0000 7f6701a21740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd70ffda00 0x55fd70ffabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.521+0000 7f6701a21740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fd71000970 con 0x55fd70ff44d0 2024-04-22T13:26:55.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.521+0000 7f6701a21740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fd70f12500 con 0x55fd70ff4e80 2024-04-22T13:26:55.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.521+0000 7f6701a21740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fd71000af0 con 0x55fd70ffda00 2024-04-22T13:26:55.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.521+0000 7f6701a21740 10 client.0 _ll_register_callbacks cb 0x55fd70ee0b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:55.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.521+0000 7f66f7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fd70ff44d0 0x55fd70ffd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.521+0000 7f66f7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fd70ff44d0 0x55fd70ffd2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36224/0 (socket says 172.21.15.12:36224) 2024-04-22T13:26:55.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.521+0000 7f66f7fff640 1 -- 172.21.15.12:0/443954362 learned_addr learned my addr 172.21.15.12:0/443954362 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:55.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.521+0000 7f66fcbd6640 1 --2- 172.21.15.12:0/443954362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd70ff4e80 0x55fd70ffa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.521+0000 7f66f7fff640 1 --2- 172.21.15.12:0/443954362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd70ffda00 0x55fd70ffabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.521+0000 7f66f7fff640 1 -- 172.21.15.12:0/443954362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd70ffda00 msgr2=0x55fd70ffabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.521+0000 7f66f7fff640 1 --2- 172.21.15.12:0/443954362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd70ffda00 0x55fd70ffabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.521+0000 7f66f7fff640 1 -- 172.21.15.12:0/443954362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd70ff4e80 msgr2=0x55fd70ffa650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.521+0000 7f66f7fff640 1 --2- 172.21.15.12:0/443954362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd70ff4e80 0x55fd70ffa650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.521+0000 7f66f7fff640 1 -- 172.21.15.12:0/443954362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd70c6b3f0 con 0x55fd70ff44d0 2024-04-22T13:26:55.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.521+0000 7f66f7fff640 1 --2- 172.21.15.12:0/443954362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fd70ff44d0 0x55fd70ffd2c0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f66ec0058c0 tx=0x7f66ec00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a9a6e5f771f8584 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.521+0000 7f66d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:26:55.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.522+0000 7f66d7fff640 1 -- 172.21.15.12:0/443954362 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec027030 con 0x55fd70ff44d0 2024-04-22T13:26:55.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.522+0000 7f66d7fff640 1 -- 172.21.15.12:0/443954362 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f66ec004030 con 0x55fd70ff44d0 2024-04-22T13:26:55.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.522+0000 7f6701a21740 1 -- 172.21.15.12:0/443954362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66ec013d10 con 0x55fd70ff44d0 2024-04-22T13:26:55.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.522+0000 7f66d7fff640 1 -- 172.21.15.12:0/443954362 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f66ec031050 con 0x55fd70ff44d0 2024-04-22T13:26:55.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.522+0000 7f6701a21740 10 client.16815 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:55.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.522+0000 7f6701a21740 1 -- 172.21.15.12:0/443954362 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fd70d357d0 con 0x55fd70ff44d0 2024-04-22T13:26:55.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.522+0000 7f66d7fff640 1 -- 172.21.15.12:0/443954362 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66ec013d10 con 0x55fd70ff44d0 2024-04-22T13:26:55.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.522+0000 7f66d7fff640 1 -- 172.21.15.12:0/443954362 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f66ec011910 con 0x55fd70ff44d0 2024-04-22T13:26:55.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.522+0000 7f6701a21740 10 client.16815 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:55.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.522+0000 7f6701a21740 10 client.16815 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:26:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.522+0000 7f6701a21740 20 client.16815 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.522+0000 7f6701a21740 1 --2- 172.21.15.12:0/443954362 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55fd7100db70 0x55fd7100ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.522+0000 7f6701a21740 4 client.16815 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:26:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.522+0000 7f6701a21740 1 -- 172.21.15.12:0/443954362 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd70ef8930 con 0x55fd7100db70 2024-04-22T13:26:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.522+0000 7f66fcbd6640 1 --2- 172.21.15.12:0/443954362 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55fd7100db70 0x55fd7100ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.523+0000 7f66d7fff640 10 client.16815 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:26:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.523+0000 7f66fcbd6640 1 --2- 172.21.15.12:0/443954362 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55fd7100db70 0x55fd7100ff60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7356dfcbcba3d386 server_cookie=4d5aaed3c3ae1994 in_seq=0 out_seq=0 2024-04-22T13:26:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.523+0000 7f66d7fff640 1 -- 172.21.15.12:0/443954362 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fd70ef8930 con 0x55fd7100db70 2024-04-22T13:26:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.523+0000 7f66d7fff640 10 client.16815 handle_command_reply: tid=0 2024-04-22T13:26:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.523+0000 7f6701a21740 1 client.16815 shutdown 2024-04-22T13:26:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.524+0000 7f6701a21740 1 -- 172.21.15.12:0/443954362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fd70ff44d0 msgr2=0x55fd70ffd2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.524+0000 7f6701a21740 1 --2- 172.21.15.12:0/443954362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fd70ff44d0 0x55fd70ffd2c0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f66ec0058c0 tx=0x7f66ec00bfd0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.524+0000 7f6701a21740 1 -- 172.21.15.12:0/443954362 shutdown_connections 2024-04-22T13:26:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.524+0000 7f6701a21740 1 --2- 172.21.15.12:0/443954362 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55fd7100db70 0x55fd7100ff60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.524+0000 7f6701a21740 1 --2- 172.21.15.12:0/443954362 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd70ffda00 0x55fd70ffabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.524+0000 7f66d7fff640 0 client.16815 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:26:55.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.524+0000 7f6701a21740 1 --2- 172.21.15.12:0/443954362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd70ff4e80 0x55fd70ffa650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.524+0000 7f6701a21740 1 --2- 172.21.15.12:0/443954362 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fd70ff44d0 0x55fd70ffd2c0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.524+0000 7f6701a21740 1 -- 172.21.15.12:0/443954362 >> 172.21.15.12:0/443954362 conn(0x55fd70feeba0 msgr2=0x55fd70ff08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:55.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.524+0000 7f6701a21740 1 -- 172.21.15.12:0/443954362 shutdown_connections 2024-04-22T13:26:55.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.524+0000 7f6701a21740 1 -- 172.21.15.12:0/443954362 wait complete. 2024-04-22T13:26:55.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.524+0000 7f6701a21740 20 client.16815 trim_cache size 0 max 16384 2024-04-22T13:26:55.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.537+0000 7f6701a21740 1 Processor -- start 2024-04-22T13:26:55.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.537+0000 7f6701a21740 1 -- start start 2024-04-22T13:26:55.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.537+0000 7f6701a21740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd70ff44d0 0x55fd71009830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.537+0000 7f6701a21740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd70ff4e80 0x55fd70ff9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.537+0000 7f6701a21740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fd70ffda00 0x55fd70ffcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.537+0000 7f6701a21740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f66f81adb40 con 0x55fd70ffda00 2024-04-22T13:26:55.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.537+0000 7f6701a21740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f66f81af680 con 0x55fd70ff44d0 2024-04-22T13:26:55.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.537+0000 7f6701a21740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66f81ae210 con 0x55fd70ff4e80 2024-04-22T13:26:55.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.537+0000 7f66f7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fd70ffda00 0x55fd70ffcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.537+0000 7f66f7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fd70ffda00 0x55fd70ffcd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36226/0 (socket says 172.21.15.12:36226) 2024-04-22T13:26:55.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.537+0000 7f66f7fff640 1 -- 172.21.15.12:0/2540145012 learned_addr learned my addr 172.21.15.12:0/2540145012 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:55.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.538+0000 7f66fcbd6640 1 --2- 172.21.15.12:0/2540145012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd70ff4e80 0x55fd70ff9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.538+0000 7f66f7fff640 1 --2- 172.21.15.12:0/2540145012 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd70ff44d0 0x55fd71009830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.538+0000 7f66f7fff640 1 -- 172.21.15.12:0/2540145012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd70ff4e80 msgr2=0x55fd70ff9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.538+0000 7f66f7fff640 1 --2- 172.21.15.12:0/2540145012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd70ff4e80 0x55fd70ff9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.538+0000 7f66f7fff640 1 -- 172.21.15.12:0/2540145012 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd70ff44d0 msgr2=0x55fd71009830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.538+0000 7f66f7fff640 1 --2- 172.21.15.12:0/2540145012 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd70ff44d0 0x55fd71009830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.538+0000 7f66f7fff640 1 -- 172.21.15.12:0/2540145012 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd710102d0 con 0x55fd70ffda00 2024-04-22T13:26:55.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.538+0000 7f66f7fff640 1 --2- 172.21.15.12:0/2540145012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fd70ffda00 0x55fd70ffcd80 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00be80 tx=0x7f66ec000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2236074389564a67 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.538+0000 7f66de7fc640 1 -- 172.21.15.12:0/2540145012 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec027040 con 0x55fd70ffda00 2024-04-22T13:26:55.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.538+0000 7f66de7fc640 1 -- 172.21.15.12:0/2540145012 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66ec013390 con 0x55fd70ffda00 2024-04-22T13:26:55.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.538+0000 7f66de7fc640 1 -- 172.21.15.12:0/2540145012 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec019040 con 0x55fd70ffda00 2024-04-22T13:26:55.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.539+0000 7f6701a21740 1 -- 172.21.15.12:0/2540145012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fd70ffda00 msgr2=0x55fd70ffcd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.539+0000 7f6701a21740 1 --2- 172.21.15.12:0/2540145012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fd70ffda00 0x55fd70ffcd80 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00be80 tx=0x7f66ec000bc0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.539+0000 7f6701a21740 1 -- 172.21.15.12:0/2540145012 shutdown_connections 2024-04-22T13:26:55.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.539+0000 7f6701a21740 1 --2- 172.21.15.12:0/2540145012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fd70ffda00 0x55fd70ffcd80 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.539+0000 7f6701a21740 1 --2- 172.21.15.12:0/2540145012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd70ff4e80 0x55fd70ff9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.539+0000 7f6701a21740 1 --2- 172.21.15.12:0/2540145012 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd70ff44d0 0x55fd71009830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.539+0000 7f6701a21740 1 -- 172.21.15.12:0/2540145012 >> 172.21.15.12:0/2540145012 conn(0x55fd70feeba0 msgr2=0x55fd70ff8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:55.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.539+0000 7f6701a21740 1 -- 172.21.15.12:0/2540145012 shutdown_connections 2024-04-22T13:26:55.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.539+0000 7f6701a21740 1 -- 172.21.15.12:0/2540145012 wait complete. 2024-04-22T13:26:55.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.539+0000 7f6701a21740 1 -- start start 2024-04-22T13:26:55.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.539+0000 7f6701a21740 1 Processor -- start 2024-04-22T13:26:55.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.539+0000 7f6701a21740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fd70ff44d0 0x55fd71005f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.539+0000 7f6701a21740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd70ff4e80 0x55fd70ff6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.539+0000 7f6701a21740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd70ffda00 0x55fd70ff7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.539+0000 7f6701a21740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f66f81aefb0 con 0x55fd70ff44d0 2024-04-22T13:26:55.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.539+0000 7f6701a21740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f66f81ae8e0 con 0x55fd70ffda00 2024-04-22T13:26:55.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.539+0000 7f6701a21740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66f81abb40 con 0x55fd70ff4e80 2024-04-22T13:26:55.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.540+0000 7f6701a21740 10 client.0 _ll_register_callbacks cb 0x7f66ec013040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:55.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.540+0000 7f66f7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fd70ff44d0 0x55fd71005f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.540+0000 7f66f7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fd70ff44d0 0x55fd71005f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36242/0 (socket says 172.21.15.12:36242) 2024-04-22T13:26:55.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.540+0000 7f66f7fff640 1 -- 172.21.15.12:0/2010932805 learned_addr learned my addr 172.21.15.12:0/2010932805 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:55.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.540+0000 7f66fcbd6640 1 --2- 172.21.15.12:0/2010932805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd70ff4e80 0x55fd70ff6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.540+0000 7f66f7fff640 1 --2- 172.21.15.12:0/2010932805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd70ffda00 0x55fd70ff7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.540+0000 7f66f7fff640 1 -- 172.21.15.12:0/2010932805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd70ff4e80 msgr2=0x55fd70ff6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.540+0000 7f66f7fff640 1 --2- 172.21.15.12:0/2010932805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd70ff4e80 0x55fd70ff6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.540+0000 7f66f7fff640 1 -- 172.21.15.12:0/2010932805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd70ffda00 msgr2=0x55fd70ff7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.540+0000 7f66f7fff640 1 --2- 172.21.15.12:0/2010932805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd70ffda00 0x55fd70ff7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.540+0000 7f66f7fff640 1 -- 172.21.15.12:0/2010932805 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd70ff7940 con 0x55fd70ff44d0 2024-04-22T13:26:55.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.540+0000 7f66fcbd6640 1 --2- 172.21.15.12:0/2010932805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd70ff4e80 0x55fd70ff6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:26:55.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.540+0000 7f66f7fff640 1 --2- 172.21.15.12:0/2010932805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fd70ff44d0 0x55fd71005f00 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00c010 tx=0x7f66ec0130f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7ea9157726a676 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.540+0000 7f66d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:26:55.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.540+0000 7f66d7fff640 1 -- 172.21.15.12:0/2010932805 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec019070 con 0x55fd70ff44d0 2024-04-22T13:26:55.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.540+0000 7f66d7fff640 1 -- 172.21.15.12:0/2010932805 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f66ec004450 con 0x55fd70ff44d0 2024-04-22T13:26:55.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.540+0000 7f6701a21740 1 -- 172.21.15.12:0/2010932805 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66ec013390 con 0x55fd70ff44d0 2024-04-22T13:26:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.540+0000 7f66d7fff640 1 -- 172.21.15.12:0/2010932805 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f66ec04a050 con 0x55fd70ff44d0 2024-04-22T13:26:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.540+0000 7f6701a21740 10 client.16821 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.540+0000 7f6701a21740 1 -- 172.21.15.12:0/2010932805 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fd71073bf0 con 0x55fd70ff44d0 2024-04-22T13:26:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.541+0000 7f66d7fff640 1 -- 172.21.15.12:0/2010932805 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66ec013390 con 0x55fd70ff44d0 2024-04-22T13:26:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.541+0000 7f66d7fff640 1 -- 172.21.15.12:0/2010932805 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f66ec034040 con 0x55fd70ff44d0 2024-04-22T13:26:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.541+0000 7f6701a21740 10 client.16821 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.541+0000 7f6701a21740 10 client.16821 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:26:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.541+0000 7f6701a21740 20 client.16821 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.541+0000 7f6701a21740 1 --2- 172.21.15.12:0/2010932805 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55fd710742d0 0x55fd710766f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.541+0000 7f6701a21740 4 client.16821 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:26:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.541+0000 7f6701a21740 1 -- 172.21.15.12:0/2010932805 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fd70f66c10 con 0x55fd710742d0 2024-04-22T13:26:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.541+0000 7f66fcbd6640 1 --2- 172.21.15.12:0/2010932805 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55fd710742d0 0x55fd710766f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.542+0000 7f66d7fff640 10 client.16821 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:26:55.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.542+0000 7f66fcbd6640 1 --2- 172.21.15.12:0/2010932805 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55fd710742d0 0x55fd710766f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=848ce07be79a69b4 server_cookie=c467282e8f64c2e4 in_seq=0 out_seq=0 2024-04-22T13:26:55.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.542+0000 7f66d7fff640 1 -- 172.21.15.12:0/2010932805 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fd70f66c10 con 0x55fd710742d0 2024-04-22T13:26:55.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.542+0000 7f66d7fff640 10 client.16821 handle_command_reply: tid=0 2024-04-22T13:26:55.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.542+0000 7f6701a21740 1 client.16821 shutdown 2024-04-22T13:26:55.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.542+0000 7f6701a21740 1 -- 172.21.15.12:0/2010932805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fd70ff44d0 msgr2=0x55fd71005f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.542+0000 7f6701a21740 1 --2- 172.21.15.12:0/2010932805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fd70ff44d0 0x55fd71005f00 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00c010 tx=0x7f66ec0130f0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.542+0000 7f6701a21740 1 -- 172.21.15.12:0/2010932805 shutdown_connections 2024-04-22T13:26:55.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.542+0000 7f6701a21740 1 --2- 172.21.15.12:0/2010932805 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55fd710742d0 0x55fd710766f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.542+0000 7f6701a21740 1 --2- 172.21.15.12:0/2010932805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fd70ffda00 0x55fd70ff7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.542+0000 7f66d7fff640 0 client.16821 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:26:55.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.542+0000 7f6701a21740 1 --2- 172.21.15.12:0/2010932805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fd70ff4e80 0x55fd70ff6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.542+0000 7f6701a21740 1 --2- 172.21.15.12:0/2010932805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fd70ff44d0 0x55fd71005f00 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.542+0000 7f6701a21740 1 -- 172.21.15.12:0/2010932805 >> 172.21.15.12:0/2010932805 conn(0x55fd70feeba0 msgr2=0x55fd70ff9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:55.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.542+0000 7f6701a21740 1 -- 172.21.15.12:0/2010932805 shutdown_connections 2024-04-22T13:26:55.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.542+0000 7f6701a21740 1 -- 172.21.15.12:0/2010932805 wait complete. 2024-04-22T13:26:55.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.542+0000 7f6701a21740 20 client.16821 trim_cache size 0 max 16384 2024-04-22T13:26:55.544 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:26:55.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.543+0000 7f66fee3a640 1 -- 172.21.15.12:0/747545228 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f66e00a3d20 msgr2=0x7f66e00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.543+0000 7f66fee3a640 1 --2- 172.21.15.12:0/747545228 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f66e00a3d20 0x7f66e00a61e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f66f00993c0 tx=0x7f66f00bd040 comp rx=0 tx=0).stop 2024-04-22T13:26:55.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.543+0000 7f66fee3a640 1 -- 172.21.15.12:0/747545228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66f8107030 msgr2=0x7f66f81a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.543+0000 7f66fee3a640 1 --2- 172.21.15.12:0/747545228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66f8107030 0x7f66f81a4620 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f66f000eaa0 tx=0x7f66f000ef60 comp rx=0 tx=0).stop 2024-04-22T13:26:55.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.543+0000 7f66fee3a640 1 -- 172.21.15.12:0/747545228 shutdown_connections 2024-04-22T13:26:55.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.543+0000 7f66fee3a640 1 --2- 172.21.15.12:0/747545228 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f66e00a3d20 0x7f66e00a61e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.543+0000 7f66fee3a640 1 --2- 172.21.15.12:0/747545228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f66f8107030 0x7f66f81a4620 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.543+0000 7f66fee3a640 1 --2- 172.21.15.12:0/747545228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66f8101b40 0x7f66f8199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.543+0000 7f66fee3a640 1 --2- 172.21.15.12:0/747545228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f66f81011a0 0x7f66f81995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.543+0000 7f66fee3a640 1 -- 172.21.15.12:0/747545228 >> 172.21.15.12:0/747545228 conn(0x7f66f8076c50 msgr2=0x7f66f80ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:55.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.543+0000 7f66fee3a640 1 -- 172.21.15.12:0/747545228 shutdown_connections 2024-04-22T13:26:55.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.543+0000 7f66fee3a640 1 -- 172.21.15.12:0/747545228 wait complete. 2024-04-22T13:26:55.548 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:55 smithi012 ceph-mon[28265]: pgmap v87: 97 pgs: 97 active+clean; 464 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-04-22T13:26:55.548 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:55 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3130454877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:26:55.550 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:26:55.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.658+0000 7fcabee8c640 1 -- 172.21.15.12:0/92964672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcab8101a50 msgr2=0x7fcab8105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.658+0000 7fcabee8c640 1 --2- 172.21.15.12:0/92964672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcab8101a50 0x7fcab8105a90 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009900 tx=0x7fcaa402bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:55.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.658+0000 7fcabee8c640 1 -- 172.21.15.12:0/92964672 shutdown_connections 2024-04-22T13:26:55.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.658+0000 7fcabee8c640 1 --2- 172.21.15.12:0/92964672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcab8106170 0x7fcab810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.658+0000 7fcabee8c640 1 --2- 172.21.15.12:0/92964672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcab8101a50 0x7fcab8105a90 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.658+0000 7fcabee8c640 1 --2- 172.21.15.12:0/92964672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcab81010a0 0x7fcab8101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.658+0000 7fcabee8c640 1 -- 172.21.15.12:0/92964672 >> 172.21.15.12:0/92964672 conn(0x7fcab8077eb0 msgr2=0x7fcab80782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:55.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.658+0000 7fcabee8c640 1 -- 172.21.15.12:0/92964672 shutdown_connections 2024-04-22T13:26:55.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.658+0000 7fcabee8c640 1 -- 172.21.15.12:0/92964672 wait complete. 2024-04-22T13:26:55.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.659+0000 7fcabee8c640 1 Processor -- start 2024-04-22T13:26:55.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.659+0000 7fcabee8c640 1 -- start start 2024-04-22T13:26:55.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.659+0000 7fcabee8c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcab81010a0 0x7fcab81976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.659+0000 7fcabee8c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcab8106170 0x7fcab8197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.659+0000 7fcabee8c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcab819bfa0 0x7fcab819c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.659+0000 7fcabee8c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcab81110b0 con 0x7fcab81010a0 2024-04-22T13:26:55.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.659+0000 7fcabee8c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcab8110f30 con 0x7fcab8106170 2024-04-22T13:26:55.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.659+0000 7fcabee8c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcab8111230 con 0x7fcab819bfa0 2024-04-22T13:26:55.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.659+0000 7fcabcc28640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcab81010a0 0x7fcab81976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.659+0000 7fcabcc28640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcab81010a0 0x7fcab81976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36260/0 (socket says 172.21.15.12:36260) 2024-04-22T13:26:55.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.659+0000 7fcabcc28640 1 -- 172.21.15.12:0/2493849082 learned_addr learned my addr 172.21.15.12:0/2493849082 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:55.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.659+0000 7fcab7fff640 1 --2- 172.21.15.12:0/2493849082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcab8106170 0x7fcab8197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.659+0000 7fcabd429640 1 --2- 172.21.15.12:0/2493849082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcab819bfa0 0x7fcab819c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.659+0000 7fcabcc28640 1 -- 172.21.15.12:0/2493849082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcab819bfa0 msgr2=0x7fcab819c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.659+0000 7fcabcc28640 1 --2- 172.21.15.12:0/2493849082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcab819bfa0 0x7fcab819c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.659+0000 7fcabcc28640 1 -- 172.21.15.12:0/2493849082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcab8106170 msgr2=0x7fcab8197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.659+0000 7fcabcc28640 1 --2- 172.21.15.12:0/2493849082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcab8106170 0x7fcab8197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.659+0000 7fcabcc28640 1 -- 172.21.15.12:0/2493849082 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab819ca90 con 0x7fcab81010a0 2024-04-22T13:26:55.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.659+0000 7fcabcc28640 1 --2- 172.21.15.12:0/2493849082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcab81010a0 0x7fcab81976e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00d770 tx=0x7fcaac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.660+0000 7fcab5ffb640 1 -- 172.21.15.12:0/2493849082 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac017070 con 0x7fcab81010a0 2024-04-22T13:26:55.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.660+0000 7fcab5ffb640 1 -- 172.21.15.12:0/2493849082 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaac0054d0 con 0x7fcab81010a0 2024-04-22T13:26:55.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.660+0000 7fcabee8c640 1 -- 172.21.15.12:0/2493849082 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcab819cd80 con 0x7fcab81010a0 2024-04-22T13:26:55.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.660+0000 7fcab5ffb640 1 -- 172.21.15.12:0/2493849082 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac004c30 con 0x7fcab81010a0 2024-04-22T13:26:55.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.660+0000 7fcabee8c640 1 -- 172.21.15.12:0/2493849082 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcab81a44f0 con 0x7fcab81010a0 2024-04-22T13:26:55.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.662+0000 7fcab5ffb640 1 -- 172.21.15.12:0/2493849082 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcaac004e10 con 0x7fcab81010a0 2024-04-22T13:26:55.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.662+0000 7fcac1a84740 1 Processor -- start 2024-04-22T13:26:55.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.662+0000 7fcac1a84740 1 -- start start 2024-04-22T13:26:55.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.662+0000 7fcab5ffb640 1 --2- 172.21.15.12:0/2493849082 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fcaa80a3d20 0x7fcaa80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.663+0000 7fcac1a84740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644a65cd4d0 0x5644a65cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.663+0000 7fcac1a84740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644a65cde80 0x5644a65d6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.663+0000 7fcac1a84740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644a65d6a00 0x5644a65d8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.663+0000 7fcac1a84740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5644a65321f0 con 0x5644a65cde80 2024-04-22T13:26:55.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.663+0000 7fcac1a84740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5644a64cf4b0 con 0x5644a65cd4d0 2024-04-22T13:26:55.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.663+0000 7fcac1a84740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5644a652e1b0 con 0x5644a65d6a00 2024-04-22T13:26:55.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.663+0000 7fcabd429640 1 --2- 172.21.15.12:0/2493849082 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fcaa80a3d20 0x7fcaa80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.663+0000 7fcab5ffb640 1 -- 172.21.15.12:0/2493849082 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fcaac0d02e0 con 0x7fcab81010a0 2024-04-22T13:26:55.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.663+0000 7fcab7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644a65cde80 0x5644a65d6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.663+0000 7fcab7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644a65cde80 0x5644a65d6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36264/0 (socket says 172.21.15.12:36264) 2024-04-22T13:26:55.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.663+0000 7fcab7fff640 1 -- 172.21.15.12:0/2074548552 learned_addr learned my addr 172.21.15.12:0/2074548552 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:55.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.663+0000 7fcabcc28640 1 --2- 172.21.15.12:0/2074548552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644a65cd4d0 0x5644a65cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.663+0000 7fcabd429640 1 --2- 172.21.15.12:0/2074548552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644a65d6a00 0x5644a65d8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.663+0000 7fcab7fff640 1 -- 172.21.15.12:0/2074548552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644a65d6a00 msgr2=0x5644a65d8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.663+0000 7fcab7fff640 1 --2- 172.21.15.12:0/2074548552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644a65d6a00 0x5644a65d8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.663+0000 7fcab7fff640 1 -- 172.21.15.12:0/2074548552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644a65cd4d0 msgr2=0x5644a65cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.663+0000 7fcab7fff640 1 --2- 172.21.15.12:0/2074548552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644a65cd4d0 0x5644a65cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.663+0000 7fcab7fff640 1 -- 172.21.15.12:0/2074548552 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644a64c11b0 con 0x5644a65cde80 2024-04-22T13:26:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.663+0000 7fcabd429640 1 --2- 172.21.15.12:0/2493849082 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fcaa80a3d20 0x7fcaa80a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009370 tx=0x7fcab0013ae0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.663+0000 7fcab7fff640 1 --2- 172.21.15.12:0/2074548552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644a65cde80 0x5644a65d6430 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009900 tx=0x7fcaa402c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a892a8ab46b6fc88 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.663+0000 7fcaa2ffd640 1 -- 172.21.15.12:0/2074548552 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4045020 con 0x5644a65cde80 2024-04-22T13:26:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.664+0000 7fcaa2ffd640 1 -- 172.21.15.12:0/2074548552 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaa402c7e0 con 0x5644a65cde80 2024-04-22T13:26:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.664+0000 7fcaa2ffd640 1 -- 172.21.15.12:0/2074548552 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa40046c0 con 0x5644a65cde80 2024-04-22T13:26:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.664+0000 7fcac1a84740 1 -- 172.21.15.12:0/2074548552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644a65cde80 msgr2=0x5644a65d6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.664+0000 7fcac1a84740 1 --2- 172.21.15.12:0/2074548552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644a65cde80 0x5644a65d6430 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009900 tx=0x7fcaa402c610 comp rx=0 tx=0).stop 2024-04-22T13:26:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.664+0000 7fcac1a84740 1 -- 172.21.15.12:0/2074548552 shutdown_connections 2024-04-22T13:26:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.664+0000 7fcac1a84740 1 --2- 172.21.15.12:0/2074548552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644a65d6a00 0x5644a65d8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.664+0000 7fcac1a84740 1 --2- 172.21.15.12:0/2074548552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644a65cde80 0x5644a65d6430 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.664+0000 7fcac1a84740 1 --2- 172.21.15.12:0/2074548552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644a65cd4d0 0x5644a65cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.664+0000 7fcac1a84740 1 -- 172.21.15.12:0/2074548552 >> 172.21.15.12:0/2074548552 conn(0x5644a65c7ba0 msgr2=0x5644a65c7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.664+0000 7fcac1a84740 1 -- 172.21.15.12:0/2074548552 shutdown_connections 2024-04-22T13:26:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.664+0000 7fcac1a84740 1 -- 172.21.15.12:0/2074548552 wait complete. 2024-04-22T13:26:55.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.665+0000 7fcac1a84740 1 -- start start 2024-04-22T13:26:55.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.665+0000 7fcac1a84740 1 Processor -- start 2024-04-22T13:26:55.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.665+0000 7fcac1a84740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644a65cd4d0 0x5644a65de850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.665+0000 7fcac1a84740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644a65cde80 0x5644a65ded90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.665+0000 7fcac1a84740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644a65d6a00 0x5644a65d58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.665+0000 7fcac1a84740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5644a65d9910 con 0x5644a65cde80 2024-04-22T13:26:55.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.665+0000 7fcac1a84740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5644a64eb500 con 0x5644a65cd4d0 2024-04-22T13:26:55.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.665+0000 7fcac1a84740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5644a65d9a90 con 0x5644a65d6a00 2024-04-22T13:26:55.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.665+0000 7fcac1a84740 10 client.0 _ll_register_callbacks cb 0x5644a64b9b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:55.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.665+0000 7fcab7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644a65cde80 0x5644a65ded90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.665+0000 7fcab7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644a65cde80 0x5644a65ded90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36280/0 (socket says 172.21.15.12:36280) 2024-04-22T13:26:55.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.665+0000 7fcab7fff640 1 -- 172.21.15.12:0/1619121904 learned_addr learned my addr 172.21.15.12:0/1619121904 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:55.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.665+0000 7fcabd429640 1 --2- 172.21.15.12:0/1619121904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644a65d6a00 0x5644a65d58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.665+0000 7fcabcc28640 1 --2- 172.21.15.12:0/1619121904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644a65cd4d0 0x5644a65de850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.665+0000 7fcab7fff640 1 -- 172.21.15.12:0/1619121904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644a65d6a00 msgr2=0x5644a65d58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.665+0000 7fcab7fff640 1 --2- 172.21.15.12:0/1619121904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644a65d6a00 0x5644a65d58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.665+0000 7fcab7fff640 1 -- 172.21.15.12:0/1619121904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644a65cd4d0 msgr2=0x5644a65de850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.665+0000 7fcab7fff640 1 --2- 172.21.15.12:0/1619121904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644a65cd4d0 0x5644a65de850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.665+0000 7fcab7fff640 1 -- 172.21.15.12:0/1619121904 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644a62443f0 con 0x5644a65cde80 2024-04-22T13:26:55.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.666+0000 7fcabcc28640 1 --2- 172.21.15.12:0/1619121904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644a65cd4d0 0x5644a65de850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:26:55.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.666+0000 7fcabd429640 1 --2- 172.21.15.12:0/1619121904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644a65d6a00 0x5644a65d58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:26:55.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.666+0000 7fcab7fff640 1 --2- 172.21.15.12:0/1619121904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644a65cde80 0x5644a65ded90 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4046000 tx=0x7fcaa402ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856bd304be28a133 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.666+0000 7fca97fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:26:55.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.666+0000 7fca97fff640 1 -- 172.21.15.12:0/1619121904 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4045070 con 0x5644a65cde80 2024-04-22T13:26:55.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.666+0000 7fca97fff640 1 -- 172.21.15.12:0/1619121904 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fcaa40037f0 con 0x5644a65cde80 2024-04-22T13:26:55.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.666+0000 7fcac1a84740 1 -- 172.21.15.12:0/1619121904 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaa402c7e0 con 0x5644a65cde80 2024-04-22T13:26:55.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.666+0000 7fca97fff640 1 -- 172.21.15.12:0/1619121904 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcaa405b050 con 0x5644a65cde80 2024-04-22T13:26:55.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.666+0000 7fcac1a84740 10 client.16833 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:55.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.666+0000 7fcac1a84740 1 -- 172.21.15.12:0/1619121904 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5644a630e7d0 con 0x5644a65cde80 2024-04-22T13:26:55.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.666+0000 7fca97fff640 1 -- 172.21.15.12:0/1619121904 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaa402c7e0 con 0x5644a65cde80 2024-04-22T13:26:55.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.666+0000 7fca97fff640 1 -- 172.21.15.12:0/1619121904 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fcaa4004cc0 con 0x5644a65cde80 2024-04-22T13:26:55.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.666+0000 7fcac1a84740 10 client.16833 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:55.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.666+0000 7fcac1a84740 10 client.16833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:26:55.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.666+0000 7fcac1a84740 20 client.16833 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:55.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.666+0000 7fcac1a84740 1 --2- 172.21.15.12:0/1619121904 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5644a65e8c50 0x5644a65eb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.666+0000 7fcac1a84740 4 client.16833 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:26:55.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.666+0000 7fcac1a84740 1 -- 172.21.15.12:0/1619121904 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644a64d1930 con 0x5644a65e8c50 2024-04-22T13:26:55.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.667+0000 7fcabcc28640 1 --2- 172.21.15.12:0/1619121904 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5644a65e8c50 0x5644a65eb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.667+0000 7fca97fff640 10 client.16833 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:55.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.667+0000 7fcabcc28640 1 --2- 172.21.15.12:0/1619121904 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5644a65e8c50 0x5644a65eb040 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8481f2dacd15e1c server_cookie=d6ccd28066f00777 in_seq=0 out_seq=0 2024-04-22T13:26:55.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.668+0000 7fca97fff640 1 -- 172.21.15.12:0/1619121904 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5644a64d1930 con 0x5644a65e8c50 2024-04-22T13:26:55.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.668+0000 7fca97fff640 10 client.16833 handle_command_reply: tid=0 2024-04-22T13:26:55.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.668+0000 7fcac1a84740 1 client.16833 shutdown 2024-04-22T13:26:55.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.668+0000 7fcac1a84740 1 -- 172.21.15.12:0/1619121904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644a65cde80 msgr2=0x5644a65ded90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.668+0000 7fcac1a84740 1 --2- 172.21.15.12:0/1619121904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644a65cde80 0x5644a65ded90 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4046000 tx=0x7fcaa402ccb0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.668+0000 7fcac1a84740 1 -- 172.21.15.12:0/1619121904 shutdown_connections 2024-04-22T13:26:55.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.668+0000 7fcac1a84740 1 --2- 172.21.15.12:0/1619121904 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5644a65e8c50 0x5644a65eb040 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.668+0000 7fcac1a84740 1 --2- 172.21.15.12:0/1619121904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644a65d6a00 0x5644a65d58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.668+0000 7fca97fff640 0 client.16833 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:55.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.668+0000 7fcac1a84740 1 --2- 172.21.15.12:0/1619121904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644a65cde80 0x5644a65ded90 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.668+0000 7fcac1a84740 1 --2- 172.21.15.12:0/1619121904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644a65cd4d0 0x5644a65de850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.668+0000 7fcac1a84740 1 -- 172.21.15.12:0/1619121904 >> 172.21.15.12:0/1619121904 conn(0x5644a65c7ba0 msgr2=0x5644a65c98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:55.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.668+0000 7fcac1a84740 1 -- 172.21.15.12:0/1619121904 shutdown_connections 2024-04-22T13:26:55.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.668+0000 7fcac1a84740 1 -- 172.21.15.12:0/1619121904 wait complete. 2024-04-22T13:26:55.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.669+0000 7fcac1a84740 20 client.16833 trim_cache size 0 max 16384 2024-04-22T13:26:55.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:55 smithi134 ceph-mon[31338]: pgmap v87: 97 pgs: 97 active+clean; 464 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-04-22T13:26:55.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:55 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/3130454877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:26:55.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.682+0000 7fcac1a84740 1 Processor -- start 2024-04-22T13:26:55.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.682+0000 7fcac1a84740 1 -- start start 2024-04-22T13:26:55.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.682+0000 7fcac1a84740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644a65cd4d0 0x5644a65dec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.682+0000 7fcac1a84740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644a65cde80 0x5644a65c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.682+0000 7fcac1a84740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644a65d6a00 0x5644a65f0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.682+0000 7fcac1a84740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcab81ab6a0 con 0x5644a65d6a00 2024-04-22T13:26:55.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.682+0000 7fcac1a84740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcab81ad1e0 con 0x5644a65cd4d0 2024-04-22T13:26:55.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.682+0000 7fcac1a84740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcab81abd70 con 0x5644a65cde80 2024-04-22T13:26:55.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.682+0000 7fcabd429640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644a65d6a00 0x5644a65f0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.682+0000 7fcabd429640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644a65d6a00 0x5644a65f0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36284/0 (socket says 172.21.15.12:36284) 2024-04-22T13:26:55.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.682+0000 7fcabd429640 1 -- 172.21.15.12:0/3030794627 learned_addr learned my addr 172.21.15.12:0/3030794627 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:55.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.682+0000 7fcab7fff640 1 --2- 172.21.15.12:0/3030794627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644a65cde80 0x5644a65c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.682+0000 7fcabcc28640 1 --2- 172.21.15.12:0/3030794627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644a65cd4d0 0x5644a65dec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.682+0000 7fcabd429640 1 -- 172.21.15.12:0/3030794627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644a65cde80 msgr2=0x5644a65c6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.682+0000 7fcabd429640 1 --2- 172.21.15.12:0/3030794627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644a65cde80 0x5644a65c6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.682+0000 7fcabd429640 1 -- 172.21.15.12:0/3030794627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644a65cd4d0 msgr2=0x5644a65dec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.682+0000 7fcabd429640 1 --2- 172.21.15.12:0/3030794627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644a65cd4d0 0x5644a65dec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.682+0000 7fcabd429640 1 -- 172.21.15.12:0/3030794627 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644a65d3e00 con 0x5644a65d6a00 2024-04-22T13:26:55.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.683+0000 7fcabd429640 1 --2- 172.21.15.12:0/3030794627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644a65d6a00 0x5644a65f0610 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fcab0013320 tx=0x7fcab001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386591ec470a395 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.683+0000 7fcaa27fc640 1 -- 172.21.15.12:0/3030794627 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab00205f0 con 0x5644a65d6a00 2024-04-22T13:26:55.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.683+0000 7fcaa27fc640 1 -- 172.21.15.12:0/3030794627 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcab0024070 con 0x5644a65d6a00 2024-04-22T13:26:55.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.683+0000 7fcaa27fc640 1 -- 172.21.15.12:0/3030794627 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab0012ba0 con 0x5644a65d6a00 2024-04-22T13:26:55.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.683+0000 7fcac1a84740 1 -- 172.21.15.12:0/3030794627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644a65d6a00 msgr2=0x5644a65f0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.683+0000 7fcac1a84740 1 --2- 172.21.15.12:0/3030794627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644a65d6a00 0x5644a65f0610 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fcab0013320 tx=0x7fcab001c000 comp rx=0 tx=0).stop 2024-04-22T13:26:55.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.683+0000 7fcac1a84740 1 -- 172.21.15.12:0/3030794627 shutdown_connections 2024-04-22T13:26:55.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.683+0000 7fcac1a84740 1 --2- 172.21.15.12:0/3030794627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644a65d6a00 0x5644a65f0610 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.683+0000 7fcac1a84740 1 --2- 172.21.15.12:0/3030794627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644a65cde80 0x5644a65c6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.684 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.683+0000 7fcac1a84740 1 --2- 172.21.15.12:0/3030794627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644a65cd4d0 0x5644a65dec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.683+0000 7fcac1a84740 1 -- 172.21.15.12:0/3030794627 >> 172.21.15.12:0/3030794627 conn(0x5644a65c7ba0 msgr2=0x5644a65c6370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:55.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.683+0000 7fcac1a84740 1 -- 172.21.15.12:0/3030794627 shutdown_connections 2024-04-22T13:26:55.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.683+0000 7fcac1a84740 1 -- 172.21.15.12:0/3030794627 wait complete. 2024-04-22T13:26:55.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.684+0000 7fcac1a84740 1 -- start start 2024-04-22T13:26:55.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.684+0000 7fcac1a84740 1 Processor -- start 2024-04-22T13:26:55.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.684+0000 7fcac1a84740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644a65cd4d0 0x5644a65e0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.684+0000 7fcac1a84740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644a65cde80 0x5644a65e0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.684+0000 7fcac1a84740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644a65d6a00 0x5644a65e12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.684+0000 7fcac1a84740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcab81acb10 con 0x5644a65cde80 2024-04-22T13:26:55.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.684+0000 7fcac1a84740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcab81ac440 con 0x5644a65d6a00 2024-04-22T13:26:55.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.684+0000 7fcac1a84740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcab81a96a0 con 0x5644a65cd4d0 2024-04-22T13:26:55.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.684+0000 7fcac1a84740 10 client.0 _ll_register_callbacks cb 0x7fcaa402c350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:55.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.684+0000 7fcab7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644a65cde80 0x5644a65e0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.684+0000 7fcab7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644a65cde80 0x5644a65e0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36300/0 (socket says 172.21.15.12:36300) 2024-04-22T13:26:55.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.684+0000 7fcab7fff640 1 -- 172.21.15.12:0/2903555299 learned_addr learned my addr 172.21.15.12:0/2903555299 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:55.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.684+0000 7fcabd429640 1 --2- 172.21.15.12:0/2903555299 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644a65d6a00 0x5644a65e12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.684+0000 7fcabcc28640 1 --2- 172.21.15.12:0/2903555299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644a65cd4d0 0x5644a65e0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.685+0000 7fcab7fff640 1 -- 172.21.15.12:0/2903555299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644a65cd4d0 msgr2=0x5644a65e0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.685+0000 7fcab7fff640 1 --2- 172.21.15.12:0/2903555299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644a65cd4d0 0x5644a65e0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.685+0000 7fcab7fff640 1 -- 172.21.15.12:0/2903555299 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644a65d6a00 msgr2=0x5644a65e12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:55.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.685+0000 7fcab7fff640 1 --2- 172.21.15.12:0/2903555299 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644a65d6a00 0x5644a65e12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.685+0000 7fcab7fff640 1 -- 172.21.15.12:0/2903555299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644a664df00 con 0x5644a65cde80 2024-04-22T13:26:55.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.685+0000 7fcabcc28640 1 --2- 172.21.15.12:0/2903555299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644a65cd4d0 0x5644a65e0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:26:55.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.685+0000 7fcabd429640 1 --2- 172.21.15.12:0/2903555299 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644a65d6a00 0x5644a65e12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:26:55.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.685+0000 7fcab7fff640 1 --2- 172.21.15.12:0/2903555299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644a65cde80 0x5644a65e0db0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x5644a65d5110 tx=0x7fcaa4041480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49446ca9318dc74 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:55.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.685+0000 7fca97fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:26:55.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.685+0000 7fca97fff640 1 -- 172.21.15.12:0/2903555299 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4045040 con 0x5644a65cde80 2024-04-22T13:26:55.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.685+0000 7fca97fff640 1 -- 172.21.15.12:0/2903555299 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fcaa4041b00 con 0x5644a65cde80 2024-04-22T13:26:55.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.685+0000 7fcac1a84740 1 -- 172.21.15.12:0/2903555299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcab0024070 con 0x5644a65cde80 2024-04-22T13:26:55.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.685+0000 7fca97fff640 1 -- 172.21.15.12:0/2903555299 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcaa4067050 con 0x5644a65cde80 2024-04-22T13:26:55.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.685+0000 7fcac1a84740 10 client.16839 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:55.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.685+0000 7fcac1a84740 1 -- 172.21.15.12:0/2903555299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5644a664f1e0 con 0x5644a65cde80 2024-04-22T13:26:55.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.685+0000 7fca97fff640 1 -- 172.21.15.12:0/2903555299 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcab0024070 con 0x5644a65cde80 2024-04-22T13:26:55.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.685+0000 7fca97fff640 1 -- 172.21.15.12:0/2903555299 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fcaa4034440 con 0x5644a65cde80 2024-04-22T13:26:55.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.685+0000 7fcac1a84740 10 client.16839 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:55.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.685+0000 7fcac1a84740 10 client.16839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:26:55.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.685+0000 7fcac1a84740 20 client.16839 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:55.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.685+0000 7fcac1a84740 1 --2- 172.21.15.12:0/2903555299 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5644a664f8d0 0x5644a6651cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:55.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.685+0000 7fcac1a84740 4 client.16839 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:26:55.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.685+0000 7fcac1a84740 1 -- 172.21.15.12:0/2903555299 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5644a653fc10 con 0x5644a664f8d0 2024-04-22T13:26:55.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.686+0000 7fcabcc28640 1 --2- 172.21.15.12:0/2903555299 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5644a664f8d0 0x5644a6651cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:55.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.686+0000 7fcabcc28640 1 --2- 172.21.15.12:0/2903555299 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5644a664f8d0 0x5644a6651cf0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=383ea71851a1e91 server_cookie=60cc7df01f8fe2e9 in_seq=0 out_seq=0 2024-04-22T13:26:55.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.686+0000 7fca97fff640 10 client.16839 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:55.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.686+0000 7fca97fff640 1 -- 172.21.15.12:0/2903555299 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5644a653fc10 con 0x5644a664f8d0 2024-04-22T13:26:55.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.686+0000 7fca97fff640 10 client.16839 handle_command_reply: tid=0 2024-04-22T13:26:55.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.687+0000 7fcac1a84740 1 client.16839 shutdown 2024-04-22T13:26:55.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.687+0000 7fcac1a84740 1 -- 172.21.15.12:0/2903555299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644a65cde80 msgr2=0x5644a65e0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.687+0000 7fcac1a84740 1 --2- 172.21.15.12:0/2903555299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644a65cde80 0x5644a65e0db0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x5644a65d5110 tx=0x7fcaa4041480 comp rx=0 tx=0).stop 2024-04-22T13:26:55.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.687+0000 7fcac1a84740 1 -- 172.21.15.12:0/2903555299 shutdown_connections 2024-04-22T13:26:55.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.687+0000 7fcac1a84740 1 --2- 172.21.15.12:0/2903555299 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5644a664f8d0 0x5644a6651cf0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.687+0000 7fcac1a84740 1 --2- 172.21.15.12:0/2903555299 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644a65d6a00 0x5644a65e12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.687+0000 7fcac1a84740 1 --2- 172.21.15.12:0/2903555299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644a65cde80 0x5644a65e0db0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.687+0000 7fca97fff640 0 client.16839 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:55.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.687+0000 7fcac1a84740 1 --2- 172.21.15.12:0/2903555299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644a65cd4d0 0x5644a65e0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.687+0000 7fcac1a84740 1 -- 172.21.15.12:0/2903555299 >> 172.21.15.12:0/2903555299 conn(0x5644a65c7ba0 msgr2=0x5644a65deab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:55.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.687+0000 7fcac1a84740 1 -- 172.21.15.12:0/2903555299 shutdown_connections 2024-04-22T13:26:55.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.687+0000 7fcac1a84740 1 -- 172.21.15.12:0/2903555299 wait complete. 2024-04-22T13:26:55.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.687+0000 7fcac1a84740 20 client.16839 trim_cache size 0 max 16384 2024-04-22T13:26:55.688 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:26:55.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.687+0000 7fcabee8c640 1 -- 172.21.15.12:0/2493849082 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fcaa80a3d20 msgr2=0x7fcaa80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.687+0000 7fcabee8c640 1 --2- 172.21.15.12:0/2493849082 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fcaa80a3d20 0x7fcaa80a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009370 tx=0x7fcab0013ae0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.687+0000 7fcabee8c640 1 -- 172.21.15.12:0/2493849082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcab81010a0 msgr2=0x7fcab81976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:55.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.687+0000 7fcabee8c640 1 --2- 172.21.15.12:0/2493849082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcab81010a0 0x7fcab81976e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00d770 tx=0x7fcaac00dc30 comp rx=0 tx=0).stop 2024-04-22T13:26:55.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.687+0000 7fcabee8c640 1 -- 172.21.15.12:0/2493849082 shutdown_connections 2024-04-22T13:26:55.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.687+0000 7fcabee8c640 1 --2- 172.21.15.12:0/2493849082 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fcaa80a3d20 0x7fcaa80a61e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.687+0000 7fcabee8c640 1 --2- 172.21.15.12:0/2493849082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcab819bfa0 0x7fcab819c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.687+0000 7fcabee8c640 1 --2- 172.21.15.12:0/2493849082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcab8106170 0x7fcab8197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.687+0000 7fcabee8c640 1 --2- 172.21.15.12:0/2493849082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcab81010a0 0x7fcab81976e0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:55.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.687+0000 7fcabee8c640 1 -- 172.21.15.12:0/2493849082 >> 172.21.15.12:0/2493849082 conn(0x7fcab8077eb0 msgr2=0x7fcab80ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:55.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.688+0000 7fcabee8c640 1 -- 172.21.15.12:0/2493849082 shutdown_connections 2024-04-22T13:26:55.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:55.688+0000 7fcabee8c640 1 -- 172.21.15.12:0/2493849082 wait complete. 2024-04-22T13:26:55.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:55 smithi111 ceph-mon[31361]: pgmap v87: 97 pgs: 97 active+clean; 464 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-04-22T13:26:55.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:55 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/3130454877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:26:56.695 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:26:56.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:26:56.695 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:26:56.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.818+0000 7fea4a350640 1 -- 172.21.15.12:0/4107026794 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea44102190 msgr2=0x7fea44102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:56.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.818+0000 7fea4a350640 1 --2- 172.21.15.12:0/4107026794 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea44102190 0x7fea44102570 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fea34009900 tx=0x7fea3402bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:56.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.818+0000 7fea4a350640 1 -- 172.21.15.12:0/4107026794 shutdown_connections 2024-04-22T13:26:56.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.818+0000 7fea4a350640 1 --2- 172.21.15.12:0/4107026794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea44107260 0x7fea4410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.818+0000 7fea4a350640 1 --2- 172.21.15.12:0/4107026794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea44102b40 0x7fea44106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.818+0000 7fea4a350640 1 --2- 172.21.15.12:0/4107026794 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea44102190 0x7fea44102570 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.818+0000 7fea4a350640 1 -- 172.21.15.12:0/4107026794 >> 172.21.15.12:0/4107026794 conn(0x7fea440fd990 msgr2=0x7fea440ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:56.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.818+0000 7fea4a350640 1 -- 172.21.15.12:0/4107026794 shutdown_connections 2024-04-22T13:26:56.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.818+0000 7fea4a350640 1 -- 172.21.15.12:0/4107026794 wait complete. 2024-04-22T13:26:56.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.819+0000 7fea4a350640 1 Processor -- start 2024-04-22T13:26:56.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.819+0000 7fea4a350640 1 -- start start 2024-04-22T13:26:56.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.819+0000 7fea4a350640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea44102190 0x7fea44198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.819+0000 7fea4a350640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea44107260 0x7fea44198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.819+0000 7fea4a350640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea4419d020 0x7fea4419d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.819+0000 7fea4a350640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fea441121a0 con 0x7fea44102190 2024-04-22T13:26:56.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.819+0000 7fea4a350640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fea44112020 con 0x7fea44107260 2024-04-22T13:26:56.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.819+0000 7fea4a350640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fea44112320 con 0x7fea4419d020 2024-04-22T13:26:56.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.819+0000 7fea43fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea44102190 0x7fea44198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.819+0000 7fea43fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea44102190 0x7fea44198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36312/0 (socket says 172.21.15.12:36312) 2024-04-22T13:26:56.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.819+0000 7fea43fff640 1 -- 172.21.15.12:0/1309802957 learned_addr learned my addr 172.21.15.12:0/1309802957 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:56.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.820+0000 7fea437fe640 1 --2- 172.21.15.12:0/1309802957 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea44107260 0x7fea44198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.820+0000 7fea488ed640 1 --2- 172.21.15.12:0/1309802957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea4419d020 0x7fea4419d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.820+0000 7fea43fff640 1 -- 172.21.15.12:0/1309802957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea4419d020 msgr2=0x7fea4419d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:56.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.820+0000 7fea43fff640 1 --2- 172.21.15.12:0/1309802957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea4419d020 0x7fea4419d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.820+0000 7fea43fff640 1 -- 172.21.15.12:0/1309802957 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea44107260 msgr2=0x7fea44198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:56.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.820+0000 7fea43fff640 1 --2- 172.21.15.12:0/1309802957 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea44107260 0x7fea44198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.820+0000 7fea43fff640 1 -- 172.21.15.12:0/1309802957 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4419db10 con 0x7fea44102190 2024-04-22T13:26:56.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.820+0000 7fea43fff640 1 --2- 172.21.15.12:0/1309802957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea44102190 0x7fea44198760 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fea34005940 tx=0x7fea340045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:56.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.820+0000 7fea417fa640 1 -- 172.21.15.12:0/1309802957 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea34043020 con 0x7fea44102190 2024-04-22T13:26:56.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.820+0000 7fea417fa640 1 -- 172.21.15.12:0/1309802957 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea3402caf0 con 0x7fea44102190 2024-04-22T13:26:56.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.820+0000 7fea4a350640 1 -- 172.21.15.12:0/1309802957 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea4419dda0 con 0x7fea44102190 2024-04-22T13:26:56.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.820+0000 7fea417fa640 1 -- 172.21.15.12:0/1309802957 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea34003af0 con 0x7fea44102190 2024-04-22T13:26:56.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.820+0000 7fea4a350640 1 -- 172.21.15.12:0/1309802957 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea441a55c0 con 0x7fea44102190 2024-04-22T13:26:56.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.823+0000 7fea417fa640 1 -- 172.21.15.12:0/1309802957 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fea34046020 con 0x7fea44102190 2024-04-22T13:26:56.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.823+0000 7fea4d01a740 1 Processor -- start 2024-04-22T13:26:56.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.823+0000 7fea4d01a740 1 -- start start 2024-04-22T13:26:56.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.823+0000 7fea4d01a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de20af18e0 0x55de20af1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.823+0000 7fea4d01a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de20af2290 0x55de20afa840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.823+0000 7fea4d01a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de20afae10 0x55de20afd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.823+0000 7fea4d01a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55de20a55f00 con 0x55de20af2290 2024-04-22T13:26:56.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.823+0000 7fea4d01a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55de209f3590 con 0x55de20afae10 2024-04-22T13:26:56.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.823+0000 7fea4d01a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55de20a51ec0 con 0x55de20af18e0 2024-04-22T13:26:56.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.823+0000 7fea43fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de20af2290 0x55de20afa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.823+0000 7fea43fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de20af2290 0x55de20afa840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36320/0 (socket says 172.21.15.12:36320) 2024-04-22T13:26:56.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.823+0000 7fea43fff640 1 -- 172.21.15.12:0/616759427 learned_addr learned my addr 172.21.15.12:0/616759427 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:56.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.823+0000 7fea488ed640 1 --2- 172.21.15.12:0/616759427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de20af18e0 0x55de20af1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.823+0000 7fea437fe640 1 --2- 172.21.15.12:0/616759427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de20afae10 0x55de20afd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.823+0000 7fea417fa640 1 --2- 172.21.15.12:0/1309802957 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fea2c0a3d20 0x7fea2c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.823+0000 7fea488ed640 1 --2- 172.21.15.12:0/1309802957 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fea2c0a3d20 0x7fea2c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.824+0000 7fea417fa640 1 -- 172.21.15.12:0/1309802957 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fea340f28e0 con 0x7fea44102190 2024-04-22T13:26:56.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.824+0000 7fea43fff640 1 -- 172.21.15.12:0/616759427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de20af18e0 msgr2=0x55de20af1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:56.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.824+0000 7fea43fff640 1 --2- 172.21.15.12:0/616759427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de20af18e0 0x55de20af1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.824+0000 7fea43fff640 1 -- 172.21.15.12:0/616759427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de20afae10 msgr2=0x55de20afd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:56.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.824+0000 7fea43fff640 1 --2- 172.21.15.12:0/616759427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de20afae10 0x55de20afd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.824+0000 7fea43fff640 1 -- 172.21.15.12:0/616759427 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de208307e0 con 0x55de20af2290 2024-04-22T13:26:56.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.824+0000 7fea437fe640 1 --2- 172.21.15.12:0/616759427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de20afae10 0x55de20afd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:26:56.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.824+0000 7fea43fff640 1 --2- 172.21.15.12:0/616759427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de20af2290 0x55de20afa840 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fea340bad90 tx=0x7fea3402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3efc44aecc122b7e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:56.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.824+0000 7fea488ed640 1 --2- 172.21.15.12:0/1309802957 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fea2c0a3d20 0x7fea2c0a61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fea3c0094d0 tx=0x7fea3c004ab0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:56.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.824+0000 7fea2a7fc640 1 -- 172.21.15.12:0/616759427 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea340be4c0 con 0x55de20af2290 2024-04-22T13:26:56.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.824+0000 7fea2a7fc640 1 -- 172.21.15.12:0/616759427 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea340c75f0 con 0x55de20af2290 2024-04-22T13:26:56.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.824+0000 7fea2a7fc640 1 -- 172.21.15.12:0/616759427 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea340be2e0 con 0x55de20af2290 2024-04-22T13:26:56.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.825+0000 7fea4d01a740 1 -- 172.21.15.12:0/616759427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de20af2290 msgr2=0x55de20afa840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:56.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.825+0000 7fea4d01a740 1 --2- 172.21.15.12:0/616759427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de20af2290 0x55de20afa840 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fea340bad90 tx=0x7fea3402bc30 comp rx=0 tx=0).stop 2024-04-22T13:26:56.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.825+0000 7fea4d01a740 1 -- 172.21.15.12:0/616759427 shutdown_connections 2024-04-22T13:26:56.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.825+0000 7fea4d01a740 1 --2- 172.21.15.12:0/616759427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de20afae10 0x55de20afd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.825+0000 7fea4d01a740 1 --2- 172.21.15.12:0/616759427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de20af2290 0x55de20afa840 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.825+0000 7fea4d01a740 1 --2- 172.21.15.12:0/616759427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de20af18e0 0x55de20af1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.825+0000 7fea4d01a740 1 -- 172.21.15.12:0/616759427 >> 172.21.15.12:0/616759427 conn(0x55de20aebfb0 msgr2=0x55de20aec390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:56.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.825+0000 7fea4d01a740 1 -- 172.21.15.12:0/616759427 shutdown_connections 2024-04-22T13:26:56.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.825+0000 7fea4d01a740 1 -- 172.21.15.12:0/616759427 wait complete. 2024-04-22T13:26:56.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.825+0000 7fea4d01a740 1 -- start start 2024-04-22T13:26:56.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.825+0000 7fea4d01a740 1 Processor -- start 2024-04-22T13:26:56.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.826+0000 7fea4d01a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de20af18e0 0x55de20af7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.826+0000 7fea4d01a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de20af2290 0x55de20af8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.826+0000 7fea4d01a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de20afae10 0x55de20af8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.826+0000 7fea4d01a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55de20afdb40 con 0x55de20af2290 2024-04-22T13:26:56.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.826+0000 7fea4d01a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55de20a0f580 con 0x55de20af18e0 2024-04-22T13:26:56.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.826+0000 7fea4d01a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55de20afdcc0 con 0x55de20afae10 2024-04-22T13:26:56.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.826+0000 7fea4d01a740 10 client.0 _ll_register_callbacks cb 0x55de20a15650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:56.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.826+0000 7fea437fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de20af2290 0x55de20af8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.826+0000 7fea437fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de20af2290 0x55de20af8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36336/0 (socket says 172.21.15.12:36336) 2024-04-22T13:26:56.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.826+0000 7fea437fe640 1 -- 172.21.15.12:0/2097414133 learned_addr learned my addr 172.21.15.12:0/2097414133 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:56.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.826+0000 7fea43fff640 1 --2- 172.21.15.12:0/2097414133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de20af18e0 0x55de20af7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.826+0000 7fea488ed640 1 --2- 172.21.15.12:0/2097414133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de20afae10 0x55de20af8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.826+0000 7fea437fe640 1 -- 172.21.15.12:0/2097414133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de20afae10 msgr2=0x55de20af8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:56.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.826+0000 7fea437fe640 1 --2- 172.21.15.12:0/2097414133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de20afae10 0x55de20af8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.826+0000 7fea437fe640 1 -- 172.21.15.12:0/2097414133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de20af18e0 msgr2=0x55de20af7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:56.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.826+0000 7fea437fe640 1 --2- 172.21.15.12:0/2097414133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de20af18e0 0x55de20af7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.826+0000 7fea437fe640 1 -- 172.21.15.12:0/2097414133 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de20890f70 con 0x55de20af2290 2024-04-22T13:26:56.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.826+0000 7fea437fe640 1 --2- 172.21.15.12:0/2097414133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de20af2290 0x55de20af8320 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fea3800bf80 tx=0x7fea380118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5e9fa69f0f0737 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:56.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.826+0000 7fea237fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:26:56.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.826+0000 7fea237fe640 1 -- 172.21.15.12:0/2097414133 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38004020 con 0x55de20af2290 2024-04-22T13:26:56.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.827+0000 7fea237fe640 1 -- 172.21.15.12:0/2097414133 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fea380041c0 con 0x55de20af2290 2024-04-22T13:26:56.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.827+0000 7fea4d01a740 1 -- 172.21.15.12:0/2097414133 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55de20a63920 con 0x55de20af2290 2024-04-22T13:26:56.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.827+0000 7fea237fe640 1 -- 172.21.15.12:0/2097414133 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fea380043c0 con 0x55de20af2290 2024-04-22T13:26:56.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.827+0000 7fea4d01a740 10 client.16851 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:56.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.827+0000 7fea4d01a740 1 -- 172.21.15.12:0/2097414133 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55de20728170 con 0x55de20af2290 2024-04-22T13:26:56.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.827+0000 7fea237fe640 1 -- 172.21.15.12:0/2097414133 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55de20a63920 con 0x55de20af2290 2024-04-22T13:26:56.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.827+0000 7fea237fe640 1 -- 172.21.15.12:0/2097414133 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fea38004c60 con 0x55de20af2290 2024-04-22T13:26:56.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.827+0000 7fea4d01a740 10 client.16851 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:56.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.827+0000 7fea4d01a740 10 client.16851 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:26:56.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.827+0000 7fea4d01a740 20 client.16851 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:56.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.827+0000 7fea4d01a740 1 --2- 172.21.15.12:0/2097414133 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55de20b0b0c0 0x55de20b0d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.827+0000 7fea4d01a740 4 client.16851 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:26:56.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.827+0000 7fea4d01a740 1 -- 172.21.15.12:0/2097414133 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de20a5d790 con 0x55de20b0b0c0 2024-04-22T13:26:56.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.827+0000 7fea43fff640 1 --2- 172.21.15.12:0/2097414133 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55de20b0b0c0 0x55de20b0d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.828+0000 7fea43fff640 1 --2- 172.21.15.12:0/2097414133 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55de20b0b0c0 0x55de20b0d540 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=717e17fe54bbe7ba server_cookie=f373761074ce5e20 in_seq=0 out_seq=0 2024-04-22T13:26:56.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.828+0000 7fea237fe640 10 client.16851 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:56.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.828+0000 7fea237fe640 1 -- 172.21.15.12:0/2097414133 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55de20a5d790 con 0x55de20b0b0c0 2024-04-22T13:26:56.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.828+0000 7fea237fe640 10 client.16851 handle_command_reply: tid=0 2024-04-22T13:26:56.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.828+0000 7fea4d01a740 1 client.16851 shutdown 2024-04-22T13:26:56.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.829+0000 7fea4d01a740 1 -- 172.21.15.12:0/2097414133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de20af2290 msgr2=0x55de20af8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:56.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.829+0000 7fea4d01a740 1 --2- 172.21.15.12:0/2097414133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de20af2290 0x55de20af8320 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fea3800bf80 tx=0x7fea380118d0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.829+0000 7fea4d01a740 1 -- 172.21.15.12:0/2097414133 shutdown_connections 2024-04-22T13:26:56.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.829+0000 7fea4d01a740 1 --2- 172.21.15.12:0/2097414133 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55de20b0b0c0 0x55de20b0d540 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.829+0000 7fea4d01a740 1 --2- 172.21.15.12:0/2097414133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de20afae10 0x55de20af8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.829+0000 7fea4d01a740 1 --2- 172.21.15.12:0/2097414133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de20af2290 0x55de20af8320 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.829+0000 7fea4d01a740 1 --2- 172.21.15.12:0/2097414133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de20af18e0 0x55de20af7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.829+0000 7fea237fe640 0 client.16851 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:56.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.829+0000 7fea4d01a740 1 -- 172.21.15.12:0/2097414133 >> 172.21.15.12:0/2097414133 conn(0x55de20aebfb0 msgr2=0x55de20aedd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:56.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.829+0000 7fea4d01a740 1 -- 172.21.15.12:0/2097414133 shutdown_connections 2024-04-22T13:26:56.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.829+0000 7fea4d01a740 1 -- 172.21.15.12:0/2097414133 wait complete. 2024-04-22T13:26:56.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.829+0000 7fea4d01a740 20 client.16851 trim_cache size 0 max 16384 2024-04-22T13:26:56.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.847+0000 7fea4d01a740 1 Processor -- start 2024-04-22T13:26:56.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.847+0000 7fea4d01a740 1 -- start start 2024-04-22T13:26:56.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.847+0000 7fea4d01a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de20af18e0 0x55de20afa510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.847+0000 7fea4d01a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de20af2290 0x55de20b06950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.847+0000 7fea4d01a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de20afae10 0x55de20b12e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.847+0000 7fea4d01a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fea441adaa0 con 0x55de20af18e0 2024-04-22T13:26:56.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.847+0000 7fea4d01a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fea441af5e0 con 0x55de20afae10 2024-04-22T13:26:56.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.847+0000 7fea4d01a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fea441ae170 con 0x55de20af2290 2024-04-22T13:26:56.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.847+0000 7fea43fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de20af18e0 0x55de20afa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.847+0000 7fea43fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de20af18e0 0x55de20afa510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36344/0 (socket says 172.21.15.12:36344) 2024-04-22T13:26:56.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.847+0000 7fea43fff640 1 -- 172.21.15.12:0/2642845061 learned_addr learned my addr 172.21.15.12:0/2642845061 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:56.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.847+0000 7fea437fe640 1 --2- 172.21.15.12:0/2642845061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de20af2290 0x55de20b06950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.847+0000 7fea43fff640 1 -- 172.21.15.12:0/2642845061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de20af2290 msgr2=0x55de20b06950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:56.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.847+0000 7fea43fff640 1 --2- 172.21.15.12:0/2642845061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de20af2290 0x55de20b06950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.847+0000 7fea488ed640 1 --2- 172.21.15.12:0/2642845061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de20afae10 0x55de20b12e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.847+0000 7fea43fff640 1 -- 172.21.15.12:0/2642845061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de20afae10 msgr2=0x55de20b12e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:56.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.847+0000 7fea43fff640 1 --2- 172.21.15.12:0/2642845061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de20afae10 0x55de20b12e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.847+0000 7fea43fff640 1 -- 172.21.15.12:0/2642845061 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de20af13b0 con 0x55de20af18e0 2024-04-22T13:26:56.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.847+0000 7fea488ed640 1 --2- 172.21.15.12:0/2642845061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de20afae10 0x55de20b12e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:26:56.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.847+0000 7fea43fff640 1 --2- 172.21.15.12:0/2642845061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de20af18e0 0x55de20afa510 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fea340be910 tx=0x7fea340baca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd343e92ed43b6c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:56.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.848+0000 7fea29ffb640 1 -- 172.21.15.12:0/2642845061 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea340be4c0 con 0x55de20af18e0 2024-04-22T13:26:56.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.848+0000 7fea29ffb640 1 -- 172.21.15.12:0/2642845061 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea340bf080 con 0x55de20af18e0 2024-04-22T13:26:56.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.848+0000 7fea29ffb640 1 -- 172.21.15.12:0/2642845061 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea340053b0 con 0x55de20af18e0 2024-04-22T13:26:56.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.848+0000 7fea4d01a740 1 -- 172.21.15.12:0/2642845061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de20af18e0 msgr2=0x55de20afa510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:56.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.848+0000 7fea4d01a740 1 --2- 172.21.15.12:0/2642845061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de20af18e0 0x55de20afa510 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fea340be910 tx=0x7fea340baca0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.848+0000 7fea4d01a740 1 -- 172.21.15.12:0/2642845061 shutdown_connections 2024-04-22T13:26:56.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.848+0000 7fea4d01a740 1 --2- 172.21.15.12:0/2642845061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de20afae10 0x55de20b12e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.848+0000 7fea4d01a740 1 --2- 172.21.15.12:0/2642845061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de20af2290 0x55de20b06950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.848+0000 7fea4d01a740 1 --2- 172.21.15.12:0/2642845061 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de20af18e0 0x55de20afa510 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.848+0000 7fea4d01a740 1 -- 172.21.15.12:0/2642845061 >> 172.21.15.12:0/2642845061 conn(0x55de20aebfb0 msgr2=0x55de20aedc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:56.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.848+0000 7fea4d01a740 1 -- 172.21.15.12:0/2642845061 shutdown_connections 2024-04-22T13:26:56.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.848+0000 7fea4d01a740 1 -- 172.21.15.12:0/2642845061 wait complete. 2024-04-22T13:26:56.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.849+0000 7fea4d01a740 1 -- start start 2024-04-22T13:26:56.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.849+0000 7fea4d01a740 1 Processor -- start 2024-04-22T13:26:56.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.849+0000 7fea4d01a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de20af18e0 0x55de20aff000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.849+0000 7fea4d01a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de20af2290 0x55de20aff540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.849+0000 7fea4d01a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de20afae10 0x55de20b74a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.849+0000 7fea4d01a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fea441aef10 con 0x55de20af18e0 2024-04-22T13:26:56.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.849+0000 7fea4d01a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fea441ae840 con 0x55de20afae10 2024-04-22T13:26:56.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.849+0000 7fea4d01a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fea441abaa0 con 0x55de20af2290 2024-04-22T13:26:56.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.849+0000 7fea4d01a740 10 client.0 _ll_register_callbacks cb 0x7fea3800bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:56.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.849+0000 7fea43fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de20af18e0 0x55de20aff000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.849+0000 7fea43fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de20af18e0 0x55de20aff000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36348/0 (socket says 172.21.15.12:36348) 2024-04-22T13:26:56.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.849+0000 7fea43fff640 1 -- 172.21.15.12:0/3257300410 learned_addr learned my addr 172.21.15.12:0/3257300410 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:56.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.849+0000 7fea437fe640 1 --2- 172.21.15.12:0/3257300410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de20af2290 0x55de20aff540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.849+0000 7fea488ed640 1 --2- 172.21.15.12:0/3257300410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de20afae10 0x55de20b74a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.849+0000 7fea43fff640 1 -- 172.21.15.12:0/3257300410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de20af2290 msgr2=0x55de20aff540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:56.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.849+0000 7fea43fff640 1 --2- 172.21.15.12:0/3257300410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de20af2290 0x55de20aff540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.849+0000 7fea43fff640 1 -- 172.21.15.12:0/3257300410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de20afae10 msgr2=0x55de20b74a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:56.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.849+0000 7fea43fff640 1 --2- 172.21.15.12:0/3257300410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de20afae10 0x55de20b74a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.850+0000 7fea43fff640 1 -- 172.21.15.12:0/3257300410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de20b75020 con 0x55de20af18e0 2024-04-22T13:26:56.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.850+0000 7fea488ed640 1 --2- 172.21.15.12:0/3257300410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de20afae10 0x55de20b74a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:26:56.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.850+0000 7fea43fff640 1 --2- 172.21.15.12:0/3257300410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de20af18e0 0x55de20aff000 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fea340be680 tx=0x7fea34009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ed6054f6ecc799 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:56.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.850+0000 7fea237fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:26:56.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.850+0000 7fea237fe640 1 -- 172.21.15.12:0/3257300410 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea34088060 con 0x55de20af18e0 2024-04-22T13:26:56.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.850+0000 7fea237fe640 1 -- 172.21.15.12:0/3257300410 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fea34003c90 con 0x55de20af18e0 2024-04-22T13:26:56.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.850+0000 7fea4d01a740 1 -- 172.21.15.12:0/3257300410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea340bf080 con 0x55de20af18e0 2024-04-22T13:26:56.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.850+0000 7fea237fe640 1 -- 172.21.15.12:0/3257300410 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fea340be980 con 0x55de20af18e0 2024-04-22T13:26:56.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.850+0000 7fea4d01a740 10 client.16857 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:56.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.850+0000 7fea4d01a740 1 -- 172.21.15.12:0/3257300410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55de20b75020 con 0x55de20af18e0 2024-04-22T13:26:56.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.850+0000 7fea237fe640 1 -- 172.21.15.12:0/3257300410 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea340bf080 con 0x55de20af18e0 2024-04-22T13:26:56.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.850+0000 7fea237fe640 1 -- 172.21.15.12:0/3257300410 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fea34089250 con 0x55de20af18e0 2024-04-22T13:26:56.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.850+0000 7fea4d01a740 10 client.16857 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:56.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.850+0000 7fea4d01a740 10 client.16857 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:26:56.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.850+0000 7fea4d01a740 20 client.16857 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:56.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.850+0000 7fea4d01a740 1 --2- 172.21.15.12:0/3257300410 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55de20b764f0 0x55de20b78910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.850+0000 7fea4d01a740 4 client.16857 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:26:56.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.850+0000 7fea4d01a740 1 -- 172.21.15.12:0/3257300410 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55de209d9c50 con 0x55de20b764f0 2024-04-22T13:26:56.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.851+0000 7fea437fe640 1 --2- 172.21.15.12:0/3257300410 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55de20b764f0 0x55de20b78910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.851+0000 7fea437fe640 1 --2- 172.21.15.12:0/3257300410 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55de20b764f0 0x55de20b78910 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e1add0472df9e85 server_cookie=51e3d107bfd2dee9 in_seq=0 out_seq=0 2024-04-22T13:26:56.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.851+0000 7fea237fe640 10 client.16857 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:56.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.852+0000 7fea237fe640 1 -- 172.21.15.12:0/3257300410 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55de209d9c50 con 0x55de20b764f0 2024-04-22T13:26:56.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.852+0000 7fea237fe640 10 client.16857 handle_command_reply: tid=0 2024-04-22T13:26:56.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.852+0000 7fea4d01a740 1 client.16857 shutdown 2024-04-22T13:26:56.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.853+0000 7fea4d01a740 1 -- 172.21.15.12:0/3257300410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de20af18e0 msgr2=0x55de20aff000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:56.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.853+0000 7fea4d01a740 1 --2- 172.21.15.12:0/3257300410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de20af18e0 0x55de20aff000 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fea340be680 tx=0x7fea34009560 comp rx=0 tx=0).stop 2024-04-22T13:26:56.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.853+0000 7fea4d01a740 1 -- 172.21.15.12:0/3257300410 shutdown_connections 2024-04-22T13:26:56.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.853+0000 7fea4d01a740 1 --2- 172.21.15.12:0/3257300410 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55de20b764f0 0x55de20b78910 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.853+0000 7fea4d01a740 1 --2- 172.21.15.12:0/3257300410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de20afae10 0x55de20b74a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.853+0000 7fea4d01a740 1 --2- 172.21.15.12:0/3257300410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55de20af2290 0x55de20aff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.853+0000 7fea4d01a740 1 --2- 172.21.15.12:0/3257300410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55de20af18e0 0x55de20aff000 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.853+0000 7fea237fe640 0 client.16857 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:56.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.853+0000 7fea4d01a740 1 -- 172.21.15.12:0/3257300410 >> 172.21.15.12:0/3257300410 conn(0x55de20aebfb0 msgr2=0x55de20aed400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:56.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.853+0000 7fea4d01a740 1 -- 172.21.15.12:0/3257300410 shutdown_connections 2024-04-22T13:26:56.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.853+0000 7fea4d01a740 1 -- 172.21.15.12:0/3257300410 wait complete. 2024-04-22T13:26:56.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.853+0000 7fea4d01a740 20 client.16857 trim_cache size 0 max 16384 2024-04-22T13:26:56.856 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:26:56.856 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-04-22T13:26:56.856 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "ec693a53-51d8-43fb-9c98-85807274cb82", 2024-04-22T13:26:56.856 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-04-22T13:26:56.856 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:26:56.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.853+0000 7fea4a350640 1 -- 172.21.15.12:0/1309802957 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fea2c0a3d20 msgr2=0x7fea2c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:56.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.853+0000 7fea4a350640 1 --2- 172.21.15.12:0/1309802957 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fea2c0a3d20 0x7fea2c0a61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fea3c0094d0 tx=0x7fea3c004ab0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.853+0000 7fea4a350640 1 -- 172.21.15.12:0/1309802957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea44102190 msgr2=0x7fea44198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:56.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.853+0000 7fea4a350640 1 --2- 172.21.15.12:0/1309802957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea44102190 0x7fea44198760 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fea34005940 tx=0x7fea340045e0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.853+0000 7fea4a350640 1 -- 172.21.15.12:0/1309802957 shutdown_connections 2024-04-22T13:26:56.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.853+0000 7fea4a350640 1 --2- 172.21.15.12:0/1309802957 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fea2c0a3d20 0x7fea2c0a61e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.853+0000 7fea4a350640 1 --2- 172.21.15.12:0/1309802957 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea4419d020 0x7fea4419d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.853+0000 7fea4a350640 1 --2- 172.21.15.12:0/1309802957 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea44107260 0x7fea44198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.853+0000 7fea4a350640 1 --2- 172.21.15.12:0/1309802957 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fea44102190 0x7fea44198760 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.853+0000 7fea4a350640 1 -- 172.21.15.12:0/1309802957 >> 172.21.15.12:0/1309802957 conn(0x7fea440fd990 msgr2=0x7fea440fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:56.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.853+0000 7fea4a350640 1 -- 172.21.15.12:0/1309802957 shutdown_connections 2024-04-22T13:26:56.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.853+0000 7fea4a350640 1 -- 172.21.15.12:0/1309802957 wait complete. 2024-04-22T13:26:56.860 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:26:56.861 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:26:56.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.969+0000 7ff075df4640 1 -- 172.21.15.12:0/1003595542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff070107270 msgr2=0x7ff07010ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:56.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.969+0000 7ff075df4640 1 --2- 172.21.15.12:0/1003595542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff070107270 0x7ff07010ee40 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ff068009ca0 tx=0x7ff06802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.969+0000 7ff075df4640 1 -- 172.21.15.12:0/1003595542 shutdown_connections 2024-04-22T13:26:56.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.969+0000 7ff075df4640 1 --2- 172.21.15.12:0/1003595542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff070107270 0x7ff07010ee40 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.969+0000 7ff075df4640 1 --2- 172.21.15.12:0/1003595542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff070102b50 0x7ff070106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.969+0000 7ff075df4640 1 --2- 172.21.15.12:0/1003595542 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff0701021a0 0x7ff070102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.969+0000 7ff075df4640 1 -- 172.21.15.12:0/1003595542 >> 172.21.15.12:0/1003595542 conn(0x7ff0700793e0 msgr2=0x7ff0700797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:56.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.969+0000 7ff075df4640 1 -- 172.21.15.12:0/1003595542 shutdown_connections 2024-04-22T13:26:56.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.969+0000 7ff075df4640 1 -- 172.21.15.12:0/1003595542 wait complete. 2024-04-22T13:26:56.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.970+0000 7ff075df4640 1 Processor -- start 2024-04-22T13:26:56.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.970+0000 7ff075df4640 1 -- start start 2024-04-22T13:26:56.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.970+0000 7ff075df4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0701021a0 0x7ff070198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.970+0000 7ff075df4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff070107270 0x7ff070198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.970+0000 7ff075df4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff07019cfb0 0x7ff07019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.970+0000 7ff075df4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff0701121e0 con 0x7ff07019cfb0 2024-04-22T13:26:56.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.970+0000 7ff075df4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff070112060 con 0x7ff070107270 2024-04-22T13:26:56.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.970+0000 7ff075df4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff070112360 con 0x7ff0701021a0 2024-04-22T13:26:56.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.970+0000 7ff06ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff07019cfb0 0x7ff07019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.970+0000 7ff06ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff07019cfb0 0x7ff07019d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36364/0 (socket says 172.21.15.12:36364) 2024-04-22T13:26:56.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.970+0000 7ff06ffff640 1 -- 172.21.15.12:0/2765818272 learned_addr learned my addr 172.21.15.12:0/2765818272 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:56.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.970+0000 7ff06f7fe640 1 --2- 172.21.15.12:0/2765818272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0701021a0 0x7ff070198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.970+0000 7ff06effd640 1 --2- 172.21.15.12:0/2765818272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff070107270 0x7ff070198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.970+0000 7ff06ffff640 1 -- 172.21.15.12:0/2765818272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0701021a0 msgr2=0x7ff070198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:56.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.970+0000 7ff06ffff640 1 --2- 172.21.15.12:0/2765818272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0701021a0 0x7ff070198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.970+0000 7ff06ffff640 1 -- 172.21.15.12:0/2765818272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff070107270 msgr2=0x7ff070198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:56.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.970+0000 7ff06ffff640 1 --2- 172.21.15.12:0/2765818272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff070107270 0x7ff070198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.970+0000 7ff06ffff640 1 -- 172.21.15.12:0/2765818272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff07019daa0 con 0x7ff07019cfb0 2024-04-22T13:26:56.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.970+0000 7ff06effd640 1 --2- 172.21.15.12:0/2765818272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff070107270 0x7ff070198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:26:56.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.970+0000 7ff06f7fe640 1 --2- 172.21.15.12:0/2765818272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0701021a0 0x7ff070198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:26:56.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.971+0000 7ff06ffff640 1 --2- 172.21.15.12:0/2765818272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff07019cfb0 0x7ff07019d460 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ff068005e30 tx=0x7ff068032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:56.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.971+0000 7ff06cff9640 1 -- 172.21.15.12:0/2765818272 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff068041020 con 0x7ff07019cfb0 2024-04-22T13:26:56.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.971+0000 7ff06cff9640 1 -- 172.21.15.12:0/2765818272 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff068032970 con 0x7ff07019cfb0 2024-04-22T13:26:56.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.971+0000 7ff075df4640 1 -- 172.21.15.12:0/2765818272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff07019dd30 con 0x7ff07019cfb0 2024-04-22T13:26:56.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.971+0000 7ff06cff9640 1 -- 172.21.15.12:0/2765818272 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff068039040 con 0x7ff07019cfb0 2024-04-22T13:26:56.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.971+0000 7ff075df4640 1 -- 172.21.15.12:0/2765818272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0701a54f0 con 0x7ff07019cfb0 2024-04-22T13:26:56.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.973+0000 7ff06cff9640 1 -- 172.21.15.12:0/2765818272 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff068044020 con 0x7ff07019cfb0 2024-04-22T13:26:56.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.973+0000 7ff0789e6740 1 Processor -- start 2024-04-22T13:26:56.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.974+0000 7ff0789e6740 1 -- start start 2024-04-22T13:26:56.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.974+0000 7ff0789e6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3371c22b0 0x55e3371c2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.974+0000 7ff0789e6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3371c2bd0 0x55e3371cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.974+0000 7ff0789e6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3371cfe90 0x55e3371d2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.974+0000 7ff0789e6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e3371271e0 con 0x55e3371cfe90 2024-04-22T13:26:56.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.974+0000 7ff0789e6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e3370c4390 con 0x55e3371c2bd0 2024-04-22T13:26:56.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.974+0000 7ff0789e6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e3371231a0 con 0x55e3371c22b0 2024-04-22T13:26:56.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.974+0000 7ff06cff9640 1 --2- 172.21.15.12:0/2765818272 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff0580a3d20 0x7ff0580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.974+0000 7ff06effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3371cfe90 0x55e3371d2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.974+0000 7ff06ffff640 1 --2- 172.21.15.12:0/2765818272 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff0580a3d20 0x7ff0580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.974+0000 7ff06effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3371cfe90 0x55e3371d2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36366/0 (socket says 172.21.15.12:36366) 2024-04-22T13:26:56.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.974+0000 7ff06effd640 1 -- 172.21.15.12:0/2250619820 learned_addr learned my addr 172.21.15.12:0/2250619820 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:56.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.974+0000 7ff06f7fe640 1 --2- 172.21.15.12:0/2250619820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3371c2bd0 0x55e3371cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.974+0000 7ff06cff9640 1 -- 172.21.15.12:0/2765818272 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7ff0680f2500 con 0x7ff07019cfb0 2024-04-22T13:26:56.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.974+0000 7ff06effd640 1 --2- 172.21.15.12:0/2250619820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3371c22b0 0x55e3371c2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.974+0000 7ff06effd640 1 -- 172.21.15.12:0/2250619820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3371c22b0 msgr2=0x55e3371c2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:56.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.974+0000 7ff06effd640 1 --2- 172.21.15.12:0/2250619820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3371c22b0 0x55e3371c2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.974+0000 7ff06effd640 1 -- 172.21.15.12:0/2250619820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3371c2bd0 msgr2=0x55e3371cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:56.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.974+0000 7ff06effd640 1 --2- 172.21.15.12:0/2250619820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3371c2bd0 0x55e3371cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.974+0000 7ff06effd640 1 -- 172.21.15.12:0/2250619820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e336f037d0 con 0x55e3371cfe90 2024-04-22T13:26:56.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.974+0000 7ff06ffff640 1 --2- 172.21.15.12:0/2765818272 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff0580a3d20 0x7ff0580a61e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff0680efe60 tx=0x7ff0680b3a90 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:56.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.975+0000 7ff06effd640 1 --2- 172.21.15.12:0/2250619820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3371cfe90 0x55e3371d2280 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7ff0640149a0 tx=0x7ff06400c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb246ca6fc5dbfd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:56.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.975+0000 7ff055ffb640 1 -- 172.21.15.12:0/2250619820 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0640142d0 con 0x55e3371cfe90 2024-04-22T13:26:56.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.975+0000 7ff055ffb640 1 -- 172.21.15.12:0/2250619820 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0640054d0 con 0x55e3371cfe90 2024-04-22T13:26:56.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.975+0000 7ff055ffb640 1 -- 172.21.15.12:0/2250619820 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06400f760 con 0x55e3371cfe90 2024-04-22T13:26:56.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.975+0000 7ff0789e6740 1 -- 172.21.15.12:0/2250619820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3371cfe90 msgr2=0x55e3371d2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:56.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.975+0000 7ff0789e6740 1 --2- 172.21.15.12:0/2250619820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3371cfe90 0x55e3371d2280 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7ff0640149a0 tx=0x7ff06400c4d0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.975+0000 7ff0789e6740 1 -- 172.21.15.12:0/2250619820 shutdown_connections 2024-04-22T13:26:56.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.975+0000 7ff0789e6740 1 --2- 172.21.15.12:0/2250619820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3371cfe90 0x55e3371d2280 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.975+0000 7ff0789e6740 1 --2- 172.21.15.12:0/2250619820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3371c2bd0 0x55e3371cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.975+0000 7ff0789e6740 1 --2- 172.21.15.12:0/2250619820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3371c22b0 0x55e3371c2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.975+0000 7ff0789e6740 1 -- 172.21.15.12:0/2250619820 >> 172.21.15.12:0/2250619820 conn(0x55e3371bccb0 msgr2=0x55e3371bd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:56.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.975+0000 7ff0789e6740 1 -- 172.21.15.12:0/2250619820 shutdown_connections 2024-04-22T13:26:56.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.976+0000 7ff0789e6740 1 -- 172.21.15.12:0/2250619820 wait complete. 2024-04-22T13:26:56.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.976+0000 7ff0789e6740 1 -- start start 2024-04-22T13:26:56.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.976+0000 7ff0789e6740 1 Processor -- start 2024-04-22T13:26:56.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.976+0000 7ff0789e6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3371c22b0 0x55e3371ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.976+0000 7ff0789e6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3371c2bd0 0x55e3371cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.976+0000 7ff0789e6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3371cfe90 0x55e3371cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.976+0000 7ff0789e6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e3371d2bf0 con 0x55e3371cfe90 2024-04-22T13:26:56.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.976+0000 7ff0789e6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e3370e02f0 con 0x55e3371c2bd0 2024-04-22T13:26:56.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.976+0000 7ff0789e6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e3371d2d70 con 0x55e3371c22b0 2024-04-22T13:26:56.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.976+0000 7ff0789e6740 10 client.0 _ll_register_callbacks cb 0x55e3370ed580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:56.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.976+0000 7ff06effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3371cfe90 0x55e3371cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.976+0000 7ff06effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3371cfe90 0x55e3371cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36380/0 (socket says 172.21.15.12:36380) 2024-04-22T13:26:56.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.976+0000 7ff06effd640 1 -- 172.21.15.12:0/3237830319 learned_addr learned my addr 172.21.15.12:0/3237830319 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:56.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.977+0000 7ff06effd640 1 --2- 172.21.15.12:0/3237830319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3371c22b0 0x55e3371ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.977+0000 7ff06f7fe640 1 --2- 172.21.15.12:0/3237830319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3371c2bd0 0x55e3371cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.977+0000 7ff06effd640 1 -- 172.21.15.12:0/3237830319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3371c22b0 msgr2=0x55e3371ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:56.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.977+0000 7ff06effd640 1 --2- 172.21.15.12:0/3237830319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3371c22b0 0x55e3371ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.977+0000 7ff06effd640 1 -- 172.21.15.12:0/3237830319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3371c2bd0 msgr2=0x55e3371cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:56.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.977+0000 7ff06effd640 1 --2- 172.21.15.12:0/3237830319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3371c2bd0 0x55e3371cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.977+0000 7ff06effd640 1 -- 172.21.15.12:0/3237830319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e336e393f0 con 0x55e3371cfe90 2024-04-22T13:26:56.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.977+0000 7ff06effd640 1 --2- 172.21.15.12:0/3237830319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3371cfe90 0x55e3371cd960 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7ff064004c40 tx=0x7ff064004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58dd258eda50fbc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:56.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.977+0000 7ff04effd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:26:56.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.977+0000 7ff04effd640 1 -- 172.21.15.12:0/3237830319 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff064005000 con 0x55e3371cfe90 2024-04-22T13:26:56.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.977+0000 7ff04effd640 1 -- 172.21.15.12:0/3237830319 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7ff0640051a0 con 0x55e3371cfe90 2024-04-22T13:26:56.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.977+0000 7ff0789e6740 1 -- 172.21.15.12:0/3237830319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0640054d0 con 0x55e3371cfe90 2024-04-22T13:26:56.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.977+0000 7ff04effd640 1 -- 172.21.15.12:0/3237830319 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff06400f760 con 0x55e3371cfe90 2024-04-22T13:26:56.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.977+0000 7ff0789e6740 10 client.16869 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:56.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.977+0000 7ff0789e6740 1 -- 172.21.15.12:0/3237830319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e336df90f0 con 0x55e3371cfe90 2024-04-22T13:26:56.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.978+0000 7ff04effd640 1 -- 172.21.15.12:0/3237830319 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0640054d0 con 0x55e3371cfe90 2024-04-22T13:26:56.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.978+0000 7ff04effd640 1 -- 172.21.15.12:0/3237830319 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff0640126c0 con 0x55e3371cfe90 2024-04-22T13:26:56.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.978+0000 7ff0789e6740 10 client.16869 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:56.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.978+0000 7ff0789e6740 10 client.16869 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:26:56.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.978+0000 7ff0789e6740 20 client.16869 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:56.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.978+0000 7ff0789e6740 1 --2- 172.21.15.12:0/3237830319 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e3371dfd70 0x55e3371e2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.978+0000 7ff0789e6740 4 client.16869 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:26:56.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.978+0000 7ff0789e6740 1 -- 172.21.15.12:0/3237830319 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e337134c00 con 0x55e3371dfd70 2024-04-22T13:26:56.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.978+0000 7ff06f7fe640 1 --2- 172.21.15.12:0/3237830319 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e3371dfd70 0x55e3371e2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.978+0000 7ff06f7fe640 1 --2- 172.21.15.12:0/3237830319 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e3371dfd70 0x55e3371e2160 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2424e4f553d43631 server_cookie=a69548868b468271 in_seq=0 out_seq=0 2024-04-22T13:26:56.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.978+0000 7ff04effd640 10 client.16869 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:56.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.979+0000 7ff04effd640 1 -- 172.21.15.12:0/3237830319 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e337134c00 con 0x55e3371dfd70 2024-04-22T13:26:56.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.979+0000 7ff04effd640 10 client.16869 handle_command_reply: tid=0 2024-04-22T13:26:56.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.979+0000 7ff0789e6740 1 client.16869 shutdown 2024-04-22T13:26:56.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.979+0000 7ff0789e6740 1 -- 172.21.15.12:0/3237830319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3371cfe90 msgr2=0x55e3371cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:56.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.979+0000 7ff0789e6740 1 --2- 172.21.15.12:0/3237830319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3371cfe90 0x55e3371cd960 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7ff064004c40 tx=0x7ff064004b40 comp rx=0 tx=0).stop 2024-04-22T13:26:56.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.979+0000 7ff0789e6740 1 -- 172.21.15.12:0/3237830319 shutdown_connections 2024-04-22T13:26:56.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.979+0000 7ff0789e6740 1 --2- 172.21.15.12:0/3237830319 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e3371dfd70 0x55e3371e2160 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.979+0000 7ff0789e6740 1 --2- 172.21.15.12:0/3237830319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3371cfe90 0x55e3371cd960 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.979+0000 7ff0789e6740 1 --2- 172.21.15.12:0/3237830319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3371c2bd0 0x55e3371cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.979+0000 7ff04effd640 0 client.16869 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:56.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.979+0000 7ff0789e6740 1 --2- 172.21.15.12:0/3237830319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3371c22b0 0x55e3371ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.979+0000 7ff0789e6740 1 -- 172.21.15.12:0/3237830319 >> 172.21.15.12:0/3237830319 conn(0x55e3371bccb0 msgr2=0x55e3371bea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:56.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.979+0000 7ff0789e6740 1 -- 172.21.15.12:0/3237830319 shutdown_connections 2024-04-22T13:26:56.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.979+0000 7ff0789e6740 1 -- 172.21.15.12:0/3237830319 wait complete. 2024-04-22T13:26:56.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.979+0000 7ff0789e6740 20 client.16869 trim_cache size 0 max 16384 2024-04-22T13:26:56.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.997+0000 7ff0789e6740 1 Processor -- start 2024-04-22T13:26:56.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.997+0000 7ff0789e6740 1 -- start start 2024-04-22T13:26:56.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.997+0000 7ff0789e6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3371c22b0 0x55e3371cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.997+0000 7ff0789e6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3371c2bd0 0x55e3371dbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.997+0000 7ff0789e6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3371cfe90 0x55e3371cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.997+0000 7ff0789e6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff0701ab7f0 con 0x55e3371cfe90 2024-04-22T13:26:56.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.997+0000 7ff0789e6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff0701ad330 con 0x55e3371c2bd0 2024-04-22T13:26:56.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.997+0000 7ff0789e6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff0701abec0 con 0x55e3371c22b0 2024-04-22T13:26:56.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.997+0000 7ff06effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3371cfe90 0x55e3371cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.997+0000 7ff06effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3371cfe90 0x55e3371cd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36396/0 (socket says 172.21.15.12:36396) 2024-04-22T13:26:56.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.997+0000 7ff06effd640 1 -- 172.21.15.12:0/2960136403 learned_addr learned my addr 172.21.15.12:0/2960136403 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:56.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.997+0000 7ff06effd640 1 --2- 172.21.15.12:0/2960136403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3371c22b0 0x55e3371cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.997+0000 7ff06f7fe640 1 --2- 172.21.15.12:0/2960136403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3371c2bd0 0x55e3371dbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.997+0000 7ff06effd640 1 -- 172.21.15.12:0/2960136403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3371c22b0 msgr2=0x55e3371cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:56.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.997+0000 7ff06effd640 1 --2- 172.21.15.12:0/2960136403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3371c22b0 0x55e3371cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.997+0000 7ff06effd640 1 -- 172.21.15.12:0/2960136403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3371c2bd0 msgr2=0x55e3371dbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:56.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.997+0000 7ff06effd640 1 --2- 172.21.15.12:0/2960136403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3371c2bd0 0x55e3371dbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.997+0000 7ff06effd640 1 -- 172.21.15.12:0/2960136403 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3371c2a00 con 0x55e3371cfe90 2024-04-22T13:26:56.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.997+0000 7ff06f7fe640 1 --2- 172.21.15.12:0/2960136403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3371c2bd0 0x55e3371dbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:26:56.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.998+0000 7ff06effd640 1 --2- 172.21.15.12:0/2960136403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3371cfe90 0x55e3371cd5b0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7ff06400c820 tx=0x7ff064023040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab0833d8a31d916 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:56.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.998+0000 7ff0557fa640 1 -- 172.21.15.12:0/2960136403 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0640048d0 con 0x55e3371cfe90 2024-04-22T13:26:56.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.998+0000 7ff0557fa640 1 -- 172.21.15.12:0/2960136403 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff06400c8b0 con 0x55e3371cfe90 2024-04-22T13:26:56.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.998+0000 7ff0557fa640 1 -- 172.21.15.12:0/2960136403 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff064005220 con 0x55e3371cfe90 2024-04-22T13:26:56.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.998+0000 7ff0789e6740 1 -- 172.21.15.12:0/2960136403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3371cfe90 msgr2=0x55e3371cd5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:56.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.998+0000 7ff0789e6740 1 --2- 172.21.15.12:0/2960136403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3371cfe90 0x55e3371cd5b0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7ff06400c820 tx=0x7ff064023040 comp rx=0 tx=0).stop 2024-04-22T13:26:56.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.998+0000 7ff0789e6740 1 -- 172.21.15.12:0/2960136403 shutdown_connections 2024-04-22T13:26:56.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.998+0000 7ff0789e6740 1 --2- 172.21.15.12:0/2960136403 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3371cfe90 0x55e3371cd5b0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.998+0000 7ff0789e6740 1 --2- 172.21.15.12:0/2960136403 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3371c2bd0 0x55e3371dbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.998+0000 7ff0789e6740 1 --2- 172.21.15.12:0/2960136403 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3371c22b0 0x55e3371cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:56.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.998+0000 7ff0789e6740 1 -- 172.21.15.12:0/2960136403 >> 172.21.15.12:0/2960136403 conn(0x55e3371bccb0 msgr2=0x55e3371ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:56.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.998+0000 7ff0789e6740 1 -- 172.21.15.12:0/2960136403 shutdown_connections 2024-04-22T13:26:56.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.998+0000 7ff0789e6740 1 -- 172.21.15.12:0/2960136403 wait complete. 2024-04-22T13:26:56.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.999+0000 7ff0789e6740 1 -- start start 2024-04-22T13:26:56.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.999+0000 7ff0789e6740 1 Processor -- start 2024-04-22T13:26:56.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.999+0000 7ff0789e6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3371c22b0 0x55e3371d81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.999+0000 7ff0789e6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3371c2bd0 0x55e3371c9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.999+0000 7ff0789e6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3371cfe90 0x55e3371c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:56.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.999+0000 7ff0789e6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff0701acc60 con 0x55e3371cfe90 2024-04-22T13:26:56.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.999+0000 7ff0789e6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff0701ac590 con 0x55e3371c2bd0 2024-04-22T13:26:56.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.999+0000 7ff0789e6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff0701a9810 con 0x55e3371c22b0 2024-04-22T13:26:56.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.999+0000 7ff0789e6740 10 client.0 _ll_register_callbacks cb 0x7ff0600095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:26:56.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.999+0000 7ff06effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3371cfe90 0x55e3371c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.999+0000 7ff06effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3371cfe90 0x55e3371c9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36410/0 (socket says 172.21.15.12:36410) 2024-04-22T13:26:56.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.999+0000 7ff06effd640 1 -- 172.21.15.12:0/2664650265 learned_addr learned my addr 172.21.15.12:0/2664650265 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:26:56.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.999+0000 7ff06effd640 1 --2- 172.21.15.12:0/2664650265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3371c22b0 0x55e3371d81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:56.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:56.999+0000 7ff06f7fe640 1 --2- 172.21.15.12:0/2664650265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3371c2bd0 0x55e3371c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:57.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.000+0000 7ff06effd640 1 -- 172.21.15.12:0/2664650265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3371c22b0 msgr2=0x55e3371d81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:57.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.000+0000 7ff06effd640 1 --2- 172.21.15.12:0/2664650265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3371c22b0 0x55e3371d81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.000+0000 7ff06effd640 1 -- 172.21.15.12:0/2664650265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3371c2bd0 msgr2=0x55e3371c9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:26:57.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.000+0000 7ff06effd640 1 --2- 172.21.15.12:0/2664650265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3371c2bd0 0x55e3371c9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.000+0000 7ff06effd640 1 -- 172.21.15.12:0/2664650265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3371ca1e0 con 0x55e3371cfe90 2024-04-22T13:26:57.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.000+0000 7ff06effd640 1 --2- 172.21.15.12:0/2664650265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3371cfe90 0x55e3371c9c70 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ff06400c510 tx=0x7ff0640054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd298c4590e7e95 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:26:57.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.000+0000 7ff04effd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:26:57.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.000+0000 7ff04effd640 1 -- 172.21.15.12:0/2664650265 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff064005210 con 0x55e3371cfe90 2024-04-22T13:26:57.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.000+0000 7ff0789e6740 1 -- 172.21.15.12:0/2664650265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff06400c8b0 con 0x55e3371cfe90 2024-04-22T13:26:57.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.000+0000 7ff04effd640 1 -- 172.21.15.12:0/2664650265 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7ff06402a070 con 0x55e3371cfe90 2024-04-22T13:26:57.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.001+0000 7ff04effd640 1 -- 172.21.15.12:0/2664650265 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff064012450 con 0x55e3371cfe90 2024-04-22T13:26:57.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.001+0000 7ff0789e6740 10 client.16875 fetch_fsmap learned FSMap version 28 2024-04-22T13:26:57.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.001+0000 7ff0789e6740 1 -- 172.21.15.12:0/2664650265 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e337245e40 con 0x55e3371cfe90 2024-04-22T13:26:57.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.001+0000 7ff04effd640 1 -- 172.21.15.12:0/2664650265 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff06400c8b0 con 0x55e3371cfe90 2024-04-22T13:26:57.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.001+0000 7ff04effd640 1 -- 172.21.15.12:0/2664650265 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff064011490 con 0x55e3371cfe90 2024-04-22T13:26:57.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.001+0000 7ff0789e6740 10 client.16875 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:26:57.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.001+0000 7ff0789e6740 10 client.16875 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:26:57.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.001+0000 7ff0789e6740 20 client.16875 populate_metadata read hostname 'smithi012' 2024-04-22T13:26:57.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.001+0000 7ff0789e6740 1 --2- 172.21.15.12:0/2664650265 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e337246410 0x55e337248830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:26:57.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.001+0000 7ff0789e6740 4 client.16875 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:26:57.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.001+0000 7ff0789e6740 1 -- 172.21.15.12:0/2664650265 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e33712ea70 con 0x55e337246410 2024-04-22T13:26:57.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.001+0000 7ff06f7fe640 1 --2- 172.21.15.12:0/2664650265 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e337246410 0x55e337248830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:26:57.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.002+0000 7ff04effd640 10 client.16875 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:57.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.002+0000 7ff06f7fe640 1 --2- 172.21.15.12:0/2664650265 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e337246410 0x55e337248830 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c0d53546d7fb8f2 server_cookie=a280c2f6dd0204d in_seq=0 out_seq=0 2024-04-22T13:26:57.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.002+0000 7ff04effd640 1 -- 172.21.15.12:0/2664650265 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e33712ea70 con 0x55e337246410 2024-04-22T13:26:57.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.002+0000 7ff04effd640 10 client.16875 handle_command_reply: tid=0 2024-04-22T13:26:57.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.002+0000 7ff0789e6740 1 client.16875 shutdown 2024-04-22T13:26:57.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.002+0000 7ff0789e6740 1 -- 172.21.15.12:0/2664650265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3371cfe90 msgr2=0x55e3371c9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:57.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.002+0000 7ff0789e6740 1 --2- 172.21.15.12:0/2664650265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3371cfe90 0x55e3371c9c70 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ff06400c510 tx=0x7ff0640054d0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.002+0000 7ff0789e6740 1 -- 172.21.15.12:0/2664650265 shutdown_connections 2024-04-22T13:26:57.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.002+0000 7ff0789e6740 1 --2- 172.21.15.12:0/2664650265 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e337246410 0x55e337248830 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.002+0000 7ff0789e6740 1 --2- 172.21.15.12:0/2664650265 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3371cfe90 0x55e3371c9c70 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.002+0000 7ff0789e6740 1 --2- 172.21.15.12:0/2664650265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e3371c2bd0 0x55e3371c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.002+0000 7ff04effd640 0 client.16875 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:26:57.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.002+0000 7ff0789e6740 1 --2- 172.21.15.12:0/2664650265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e3371c22b0 0x55e3371d81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.002+0000 7ff0789e6740 1 -- 172.21.15.12:0/2664650265 >> 172.21.15.12:0/2664650265 conn(0x55e3371bccb0 msgr2=0x55e3371bbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:57.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.002+0000 7ff0789e6740 1 -- 172.21.15.12:0/2664650265 shutdown_connections 2024-04-22T13:26:57.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.002+0000 7ff0789e6740 1 -- 172.21.15.12:0/2664650265 wait complete. 2024-04-22T13:26:57.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.002+0000 7ff0789e6740 20 client.16875 trim_cache size 0 max 16384 2024-04-22T13:26:57.003 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:26:57.003 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:26:57.004 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-04-22T13:26:57.004 INFO:teuthology.orchestra.run.smithi012.stdout: "ec693a53-51d8-43fb-9c98-85807274cb82": { 2024-04-22T13:26:57.004 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-04-22T13:26:57.004 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "ec693a53-51d8-43fb-9c98-85807274cb82", 2024-04-22T13:26:57.004 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-04-22T13:26:57.004 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:26:57.004 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:26:57.004 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:26:57.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.003+0000 7ff075df4640 1 -- 172.21.15.12:0/2765818272 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff0580a3d20 msgr2=0x7ff0580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:57.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.003+0000 7ff075df4640 1 --2- 172.21.15.12:0/2765818272 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff0580a3d20 0x7ff0580a61e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff0680efe60 tx=0x7ff0680b3a90 comp rx=0 tx=0).stop 2024-04-22T13:26:57.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.003+0000 7ff075df4640 1 -- 172.21.15.12:0/2765818272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff07019cfb0 msgr2=0x7ff07019d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:26:57.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.003+0000 7ff075df4640 1 --2- 172.21.15.12:0/2765818272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff07019cfb0 0x7ff07019d460 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ff068005e30 tx=0x7ff068032420 comp rx=0 tx=0).stop 2024-04-22T13:26:57.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.003+0000 7ff075df4640 1 -- 172.21.15.12:0/2765818272 shutdown_connections 2024-04-22T13:26:57.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.003+0000 7ff075df4640 1 --2- 172.21.15.12:0/2765818272 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff0580a3d20 0x7ff0580a61e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.003+0000 7ff075df4640 1 --2- 172.21.15.12:0/2765818272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff07019cfb0 0x7ff07019d460 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.003+0000 7ff075df4640 1 --2- 172.21.15.12:0/2765818272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff070107270 0x7ff070198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.003+0000 7ff075df4640 1 --2- 172.21.15.12:0/2765818272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0701021a0 0x7ff070198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:26:57.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.003+0000 7ff075df4640 1 -- 172.21.15.12:0/2765818272 >> 172.21.15.12:0/2765818272 conn(0x7ff0700793e0 msgr2=0x7ff0700ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:26:57.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.003+0000 7ff075df4640 1 -- 172.21.15.12:0/2765818272 shutdown_connections 2024-04-22T13:26:57.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:26:57.003+0000 7ff075df4640 1 -- 172.21.15.12:0/2765818272 wait complete. 2024-04-22T13:26:57.010 INFO:tasks.cephfs.filesystem:scrub status for tag:ec693a53-51d8-43fb-9c98-85807274cb82 - {'path': '/', 'tag': 'ec693a53-51d8-43fb-9c98-85807274cb82', 'options': 'recursive,force'} 2024-04-22T13:26:57.586 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:57 smithi012 ceph-mon[28265]: pgmap v88: 97 pgs: 97 active+clean; 464 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-04-22T13:26:57.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:57 smithi134 ceph-mon[31338]: pgmap v88: 97 pgs: 97 active+clean; 464 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-04-22T13:26:57.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:57 smithi111 ceph-mon[31361]: pgmap v88: 97 pgs: 97 active+clean; 464 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 16 MiB/s wr, 14 op/s 2024-04-22T13:26:58.586 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:58 smithi012 ceph-mon[28265]: scrub queued for path: / 2024-04-22T13:26:58.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:58 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:26:58.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:58 smithi012 ceph-mon[28265]: scrub summary: active paths [/] 2024-04-22T13:26:58.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:58 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:26:58.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:58 smithi134 ceph-mon[31338]: scrub queued for path: / 2024-04-22T13:26:58.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:58 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:26:58.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:58 smithi134 ceph-mon[31338]: scrub summary: active paths [/] 2024-04-22T13:26:58.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:58 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:26:58.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:58 smithi111 ceph-mon[31361]: scrub queued for path: / 2024-04-22T13:26:58.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:58 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:26:58.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:58 smithi111 ceph-mon[31361]: scrub summary: active paths [/] 2024-04-22T13:26:58.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:58 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:26:59.586 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:26:59 smithi012 ceph-mon[28265]: pgmap v89: 97 pgs: 97 active+clean; 515 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 22 op/s 2024-04-22T13:26:59.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:26:59 smithi134 ceph-mon[31338]: pgmap v89: 97 pgs: 97 active+clean; 515 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 22 op/s 2024-04-22T13:26:59.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:26:59 smithi111 ceph-mon[31361]: pgmap v89: 97 pgs: 97 active+clean; 515 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 MiB/s wr, 22 op/s 2024-04-22T13:27:00.586 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:00 smithi012 ceph-mon[28265]: scrub summary: idle 2024-04-22T13:27:00.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:00 smithi134 ceph-mon[31338]: scrub summary: idle 2024-04-22T13:27:00.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:00 smithi111 ceph-mon[31361]: scrub summary: idle 2024-04-22T13:27:01.586 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:01 smithi012 ceph-mon[28265]: pgmap v90: 97 pgs: 97 active+clean; 515 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 21 MiB/s wr, 17 op/s 2024-04-22T13:27:01.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:01 smithi134 ceph-mon[31338]: pgmap v90: 97 pgs: 97 active+clean; 515 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 21 MiB/s wr, 17 op/s 2024-04-22T13:27:01.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:01 smithi111 ceph-mon[31361]: pgmap v90: 97 pgs: 97 active+clean; 515 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 21 MiB/s wr, 17 op/s 2024-04-22T13:27:03.586 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:03 smithi012 ceph-mon[28265]: pgmap v91: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 MiB/s wr, 42 op/s 2024-04-22T13:27:03.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:03 smithi134 ceph-mon[31338]: pgmap v91: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 MiB/s wr, 42 op/s 2024-04-22T13:27:03.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:03 smithi111 ceph-mon[31361]: pgmap v91: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 MiB/s wr, 42 op/s 2024-04-22T13:27:05.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:05 smithi012 ceph-mon[28265]: pgmap v92: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.4 MiB/s wr, 33 op/s 2024-04-22T13:27:05.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:05 smithi134 ceph-mon[31338]: pgmap v92: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.4 MiB/s wr, 33 op/s 2024-04-22T13:27:05.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:05 smithi111 ceph-mon[31361]: pgmap v92: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.4 MiB/s wr, 33 op/s 2024-04-22T13:27:07.586 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:07 smithi012 ceph-mon[28265]: pgmap v93: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.4 MiB/s wr, 33 op/s 2024-04-22T13:27:07.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:07 smithi134 ceph-mon[31338]: pgmap v93: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.4 MiB/s wr, 33 op/s 2024-04-22T13:27:07.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:07 smithi111 ceph-mon[31361]: pgmap v93: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.4 MiB/s wr, 33 op/s 2024-04-22T13:27:09.586 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:09 smithi012 ceph-mon[28265]: pgmap v94: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.4 MiB/s wr, 38 op/s 2024-04-22T13:27:09.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:09 smithi134 ceph-mon[31338]: pgmap v94: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.4 MiB/s wr, 38 op/s 2024-04-22T13:27:09.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:09 smithi111 ceph-mon[31361]: pgmap v94: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.4 MiB/s wr, 38 op/s 2024-04-22T13:27:11.586 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:11 smithi012 ceph-mon[28265]: pgmap v95: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-04-22T13:27:11.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:11 smithi134 ceph-mon[31338]: pgmap v95: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-04-22T13:27:11.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:11 smithi111 ceph-mon[31361]: pgmap v95: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-04-22T13:27:13.586 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:13 smithi012 ceph-mon[28265]: pgmap v96: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 33 op/s 2024-04-22T13:27:13.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:13 smithi134 ceph-mon[31338]: pgmap v96: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 33 op/s 2024-04-22T13:27:13.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:13 smithi111 ceph-mon[31361]: pgmap v96: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 33 op/s 2024-04-22T13:27:14.586 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:14 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:27:14.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:14 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:27:14.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:14 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:27:15.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:15 smithi012 ceph-mon[28265]: pgmap v97: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-04-22T13:27:15.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:15 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:27:15.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:15 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:27:15.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:15 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:27:15.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:15 smithi134 ceph-mon[31338]: pgmap v97: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-04-22T13:27:15.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:15 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:27:15.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:15 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:27:15.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:15 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:27:15.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:15 smithi111 ceph-mon[31361]: pgmap v97: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-04-22T13:27:15.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:15 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:27:15.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:15 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:27:15.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:15 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:27:17.586 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:17 smithi012 ceph-mon[28265]: pgmap v98: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-04-22T13:27:17.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:17 smithi134 ceph-mon[31338]: pgmap v98: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-04-22T13:27:17.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:17 smithi111 ceph-mon[31361]: pgmap v98: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-04-22T13:27:19.587 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:19 smithi012 ceph-mon[28265]: pgmap v99: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-04-22T13:27:19.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:19 smithi134 ceph-mon[31338]: pgmap v99: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-04-22T13:27:19.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:19 smithi111 ceph-mon[31361]: pgmap v99: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-04-22T13:27:21.586 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:21 smithi012 ceph-mon[28265]: pgmap v100: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-22T13:27:21.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:21 smithi134 ceph-mon[31338]: pgmap v100: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-22T13:27:21.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:21 smithi111 ceph-mon[31361]: pgmap v100: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-22T13:27:23.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:23 smithi134 ceph-mon[31338]: pgmap v101: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-22T13:27:23.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:23 smithi111 ceph-mon[31361]: pgmap v101: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-22T13:27:23.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:23 smithi012 ceph-mon[28265]: pgmap v101: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-22T13:27:25.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:25 smithi134 ceph-mon[31338]: pgmap v102: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:27:25.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:25 smithi111 ceph-mon[31361]: pgmap v102: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:27:25.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:25 smithi012 ceph-mon[28265]: pgmap v102: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:27:27.011 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:27:27.011 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:27:27.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.134+0000 7ff5d73c7640 1 -- 172.21.15.12:0/2551077300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d0100fb0 msgr2=0x7ff5d0110be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.134+0000 7ff5d73c7640 1 --2- 172.21.15.12:0/2551077300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d0100fb0 0x7ff5d0110be0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8009ca0 tx=0x7ff5c802c180 comp rx=0 tx=0).stop 2024-04-22T13:27:27.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.134+0000 7ff5d73c7640 1 -- 172.21.15.12:0/2551077300 shutdown_connections 2024-04-22T13:27:27.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.134+0000 7ff5d73c7640 1 --2- 172.21.15.12:0/2551077300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d0100fb0 0x7ff5d0110be0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.134+0000 7ff5d73c7640 1 --2- 172.21.15.12:0/2551077300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5d0103250 0x7ff5d0100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.134+0000 7ff5d73c7640 1 --2- 172.21.15.12:0/2551077300 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5d0102e70 0x7ff5d0100450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.134+0000 7ff5d73c7640 1 -- 172.21.15.12:0/2551077300 >> 172.21.15.12:0/2551077300 conn(0x7ff5d00773a0 msgr2=0x7ff5d00fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:27.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.134+0000 7ff5d73c7640 1 -- 172.21.15.12:0/2551077300 shutdown_connections 2024-04-22T13:27:27.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.134+0000 7ff5d73c7640 1 -- 172.21.15.12:0/2551077300 wait complete. 2024-04-22T13:27:27.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.134+0000 7ff5d73c7640 1 Processor -- start 2024-04-22T13:27:27.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.134+0000 7ff5d73c7640 1 -- start start 2024-04-22T13:27:27.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.135+0000 7ff5d73c7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5d0100fb0 0x7ff5d01a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.135+0000 7ff5d73c7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d0102e70 0x7ff5d01a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.135+0000 7ff5d73c7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5d0103250 0x7ff5d019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.135+0000 7ff5d73c7640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d01143b0 con 0x7ff5d0102e70 2024-04-22T13:27:27.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.135+0000 7ff5d73c7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d0114230 con 0x7ff5d0103250 2024-04-22T13:27:27.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.135+0000 7ff5d73c7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d0114530 con 0x7ff5d0100fb0 2024-04-22T13:27:27.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.135+0000 7ff5d4962640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d0102e70 0x7ff5d01a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.135+0000 7ff5d4962640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d0102e70 0x7ff5d01a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53824/0 (socket says 172.21.15.12:53824) 2024-04-22T13:27:27.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.135+0000 7ff5d4962640 1 -- 172.21.15.12:0/356681705 learned_addr learned my addr 172.21.15.12:0/356681705 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:27.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.135+0000 7ff5d5964640 1 --2- 172.21.15.12:0/356681705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5d0103250 0x7ff5d019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.135+0000 7ff5d5163640 1 --2- 172.21.15.12:0/356681705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5d0100fb0 0x7ff5d01a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.135+0000 7ff5d4962640 1 -- 172.21.15.12:0/356681705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5d0100fb0 msgr2=0x7ff5d01a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.135+0000 7ff5d4962640 1 --2- 172.21.15.12:0/356681705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5d0100fb0 0x7ff5d01a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.135+0000 7ff5d4962640 1 -- 172.21.15.12:0/356681705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5d0103250 msgr2=0x7ff5d019a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.135+0000 7ff5d4962640 1 --2- 172.21.15.12:0/356681705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5d0103250 0x7ff5d019a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.135+0000 7ff5d4962640 1 -- 172.21.15.12:0/356681705 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d019af90 con 0x7ff5d0102e70 2024-04-22T13:27:27.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.135+0000 7ff5d5163640 1 --2- 172.21.15.12:0/356681705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5d0100fb0 0x7ff5d01a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:27.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.135+0000 7ff5d4962640 1 --2- 172.21.15.12:0/356681705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d0102e70 0x7ff5d01a0b50 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000e9f0 tx=0x7ff5c000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:27.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.135+0000 7ff5c67fc640 1 -- 172.21.15.12:0/356681705 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c000cd80 con 0x7ff5d0102e70 2024-04-22T13:27:27.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.135+0000 7ff5d73c7640 1 -- 172.21.15.12:0/356681705 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5d019b280 con 0x7ff5d0102e70 2024-04-22T13:27:27.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.135+0000 7ff5c67fc640 1 -- 172.21.15.12:0/356681705 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5c00054d0 con 0x7ff5d0102e70 2024-04-22T13:27:27.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.135+0000 7ff5c67fc640 1 -- 172.21.15.12:0/356681705 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0010690 con 0x7ff5d0102e70 2024-04-22T13:27:27.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.136+0000 7ff5d73c7640 1 -- 172.21.15.12:0/356681705 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5d01a7590 con 0x7ff5d0102e70 2024-04-22T13:27:27.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.138+0000 7ff5d9fbc740 1 Processor -- start 2024-04-22T13:27:27.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.138+0000 7ff5c67fc640 1 -- 172.21.15.12:0/356681705 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff5c0004e90 con 0x7ff5d0102e70 2024-04-22T13:27:27.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.138+0000 7ff5d9fbc740 1 -- start start 2024-04-22T13:27:27.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.138+0000 7ff5d9fbc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b86ef3580 0x557b86ef3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.138+0000 7ff5d9fbc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b86ef3ea0 0x557b86efc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.138+0000 7ff5d9fbc740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b86efca80 0x557b86efee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.138+0000 7ff5d9fbc740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557b86e581e0 con 0x557b86efca80 2024-04-22T13:27:27.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.138+0000 7ff5d9fbc740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557b86df5390 con 0x557b86ef3580 2024-04-22T13:27:27.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.138+0000 7ff5d9fbc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557b86e541a0 con 0x557b86ef3ea0 2024-04-22T13:27:27.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.139+0000 7ff5d4962640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b86efca80 0x557b86efee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.139+0000 7ff5d4962640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b86efca80 0x557b86efee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53838/0 (socket says 172.21.15.12:53838) 2024-04-22T13:27:27.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.139+0000 7ff5d4962640 1 -- 172.21.15.12:0/1075932879 learned_addr learned my addr 172.21.15.12:0/1075932879 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:27.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.139+0000 7ff5d5163640 1 --2- 172.21.15.12:0/1075932879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b86ef3ea0 0x557b86efc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.139+0000 7ff5d5964640 1 --2- 172.21.15.12:0/1075932879 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b86ef3580 0x557b86ef3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.139+0000 7ff5c67fc640 1 --2- 172.21.15.12:0/356681705 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff5bc0a3d20 0x7ff5bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.139+0000 7ff5d4962640 1 -- 172.21.15.12:0/1075932879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b86ef3ea0 msgr2=0x557b86efc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.139+0000 7ff5d4962640 1 --2- 172.21.15.12:0/1075932879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b86ef3ea0 0x557b86efc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.139+0000 7ff5d4962640 1 -- 172.21.15.12:0/1075932879 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b86ef3580 msgr2=0x557b86ef3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.139+0000 7ff5d4962640 1 --2- 172.21.15.12:0/1075932879 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b86ef3580 0x557b86ef3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.139+0000 7ff5d4962640 1 -- 172.21.15.12:0/1075932879 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b86c347d0 con 0x557b86efca80 2024-04-22T13:27:27.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.139+0000 7ff5d5964640 1 --2- 172.21.15.12:0/356681705 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff5bc0a3d20 0x7ff5bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.139+0000 7ff5c67fc640 1 -- 172.21.15.12:0/356681705 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7ff5c00d0be0 con 0x7ff5d0102e70 2024-04-22T13:27:27.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.139+0000 7ff5d4962640 1 --2- 172.21.15.12:0/1075932879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b86efca80 0x557b86efee70 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7ff5c00ce2f0 tx=0x7ff5c00c7ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e3aa2bffe6bc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:27.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.139+0000 7ff5bb7fe640 1 -- 172.21.15.12:0/1075932879 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c009c710 con 0x557b86efca80 2024-04-22T13:27:27.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.140+0000 7ff5d5964640 1 --2- 172.21.15.12:0/356681705 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff5bc0a3d20 0x7ff5bc0a61e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8004af0 tx=0x7ff5c802e040 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:27.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.140+0000 7ff5bb7fe640 1 -- 172.21.15.12:0/1075932879 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5c0005680 con 0x557b86efca80 2024-04-22T13:27:27.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.140+0000 7ff5bb7fe640 1 -- 172.21.15.12:0/1075932879 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0005180 con 0x557b86efca80 2024-04-22T13:27:27.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.140+0000 7ff5d9fbc740 1 -- 172.21.15.12:0/1075932879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b86efca80 msgr2=0x557b86efee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.140+0000 7ff5d9fbc740 1 --2- 172.21.15.12:0/1075932879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b86efca80 0x557b86efee70 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7ff5c00ce2f0 tx=0x7ff5c00c7ce0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.140+0000 7ff5d9fbc740 1 -- 172.21.15.12:0/1075932879 shutdown_connections 2024-04-22T13:27:27.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.140+0000 7ff5d9fbc740 1 --2- 172.21.15.12:0/1075932879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b86efca80 0x557b86efee70 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.140+0000 7ff5d9fbc740 1 --2- 172.21.15.12:0/1075932879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b86ef3ea0 0x557b86efc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.140+0000 7ff5d9fbc740 1 --2- 172.21.15.12:0/1075932879 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b86ef3580 0x557b86ef3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.140+0000 7ff5d9fbc740 1 -- 172.21.15.12:0/1075932879 >> 172.21.15.12:0/1075932879 conn(0x557b86eedcb0 msgr2=0x557b86eee090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:27.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.140+0000 7ff5d9fbc740 1 -- 172.21.15.12:0/1075932879 shutdown_connections 2024-04-22T13:27:27.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.140+0000 7ff5d9fbc740 1 -- 172.21.15.12:0/1075932879 wait complete. 2024-04-22T13:27:27.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.141+0000 7ff5d9fbc740 1 -- start start 2024-04-22T13:27:27.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.141+0000 7ff5d9fbc740 1 Processor -- start 2024-04-22T13:27:27.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.141+0000 7ff5d9fbc740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b86ef3580 0x557b86ef9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.141+0000 7ff5d9fbc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b86ef3ea0 0x557b86ef9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.141+0000 7ff5d9fbc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b86efca80 0x557b86efa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.141+0000 7ff5d9fbc740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557b86eff7b0 con 0x557b86ef3580 2024-04-22T13:27:27.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.141+0000 7ff5d9fbc740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557b86e112f0 con 0x557b86efca80 2024-04-22T13:27:27.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.141+0000 7ff5d9fbc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557b86eff930 con 0x557b86ef3ea0 2024-04-22T13:27:27.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.141+0000 7ff5d9fbc740 10 client.0 _ll_register_callbacks cb 0x557b86e1e580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:27.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.141+0000 7ff5d5163640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b86ef3580 0x557b86ef9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.141+0000 7ff5d5163640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b86ef3580 0x557b86ef9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53842/0 (socket says 172.21.15.12:53842) 2024-04-22T13:27:27.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.141+0000 7ff5d5163640 1 -- 172.21.15.12:0/302733948 learned_addr learned my addr 172.21.15.12:0/302733948 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:27.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.141+0000 7ff5d5964640 1 --2- 172.21.15.12:0/302733948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b86efca80 0x557b86efa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.141+0000 7ff5d4962640 1 --2- 172.21.15.12:0/302733948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b86ef3ea0 0x557b86ef9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.142+0000 7ff5d5163640 1 -- 172.21.15.12:0/302733948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b86ef3ea0 msgr2=0x557b86ef9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.142+0000 7ff5d5163640 1 --2- 172.21.15.12:0/302733948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b86ef3ea0 0x557b86ef9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.142+0000 7ff5d5163640 1 -- 172.21.15.12:0/302733948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b86efca80 msgr2=0x557b86efa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.142+0000 7ff5d5163640 1 --2- 172.21.15.12:0/302733948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b86efca80 0x557b86efa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.142+0000 7ff5d5163640 1 -- 172.21.15.12:0/302733948 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b86b6a3f0 con 0x557b86ef3580 2024-04-22T13:27:27.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.142+0000 7ff5d5163640 1 --2- 172.21.15.12:0/302733948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b86ef3580 0x557b86ef9aa0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc00bb40 tx=0x7ff5cc0044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df78cb1368d923da server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:27.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.142+0000 7ff5b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:27:27.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.142+0000 7ff5b8ff9640 1 -- 172.21.15.12:0/302733948 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc005220 con 0x557b86ef3580 2024-04-22T13:27:27.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.142+0000 7ff5b8ff9640 1 -- 172.21.15.12:0/302733948 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7ff5cc0053c0 con 0x557b86ef3580 2024-04-22T13:27:27.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.142+0000 7ff5d9fbc740 1 -- 172.21.15.12:0/302733948 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5c0005680 con 0x557b86ef3580 2024-04-22T13:27:27.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.142+0000 7ff5b8ff9640 1 -- 172.21.15.12:0/302733948 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff5cc0055c0 con 0x557b86ef3580 2024-04-22T13:27:27.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.142+0000 7ff5d9fbc740 10 client.16887 fetch_fsmap learned FSMap version 28 2024-04-22T13:27:27.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.142+0000 7ff5d9fbc740 1 -- 172.21.15.12:0/302733948 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557b86b2a0f0 con 0x557b86ef3580 2024-04-22T13:27:27.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.142+0000 7ff5b8ff9640 1 -- 172.21.15.12:0/302733948 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5c0005680 con 0x557b86ef3580 2024-04-22T13:27:27.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.143+0000 7ff5b8ff9640 1 -- 172.21.15.12:0/302733948 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff5cc019030 con 0x557b86ef3580 2024-04-22T13:27:27.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.143+0000 7ff5d9fbc740 10 client.16887 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:27:27.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.143+0000 7ff5d9fbc740 10 client.16887 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:27:27.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.143+0000 7ff5d9fbc740 20 client.16887 populate_metadata read hostname 'smithi012' 2024-04-22T13:27:27.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.143+0000 7ff5d9fbc740 1 --2- 172.21.15.12:0/302733948 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557b86f0c970 0x557b86f0edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.143+0000 7ff5d9fbc740 4 client.16887 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:27.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.143+0000 7ff5d9fbc740 1 -- 172.21.15.12:0/302733948 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b86e65c00 con 0x557b86f0c970 2024-04-22T13:27:27.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.143+0000 7ff5d4962640 1 --2- 172.21.15.12:0/302733948 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557b86f0c970 0x557b86f0edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.143+0000 7ff5d4962640 1 --2- 172.21.15.12:0/302733948 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557b86f0c970 0x557b86f0edd0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1db50b233b08c2e3 server_cookie=421f30ca12fcc2d0 in_seq=0 out_seq=0 2024-04-22T13:27:27.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.143+0000 7ff5b8ff9640 10 client.16887 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:27:27.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.144+0000 7ff5b8ff9640 1 -- 172.21.15.12:0/302733948 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557b86e65c00 con 0x557b86f0c970 2024-04-22T13:27:27.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.144+0000 7ff5b8ff9640 10 client.16887 handle_command_reply: tid=0 2024-04-22T13:27:27.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.144+0000 7ff5d9fbc740 1 client.16887 shutdown 2024-04-22T13:27:27.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.144+0000 7ff5d9fbc740 1 -- 172.21.15.12:0/302733948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b86ef3580 msgr2=0x557b86ef9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.144+0000 7ff5d9fbc740 1 --2- 172.21.15.12:0/302733948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b86ef3580 0x557b86ef9aa0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc00bb40 tx=0x7ff5cc0044b0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.144+0000 7ff5d9fbc740 1 -- 172.21.15.12:0/302733948 shutdown_connections 2024-04-22T13:27:27.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.144+0000 7ff5d9fbc740 1 --2- 172.21.15.12:0/302733948 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557b86f0c970 0x557b86f0edd0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.144+0000 7ff5d9fbc740 1 --2- 172.21.15.12:0/302733948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b86efca80 0x557b86efa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.144+0000 7ff5d9fbc740 1 --2- 172.21.15.12:0/302733948 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b86ef3ea0 0x557b86ef9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.144+0000 7ff5b8ff9640 0 client.16887 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:27:27.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.144+0000 7ff5d9fbc740 1 --2- 172.21.15.12:0/302733948 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b86ef3580 0x557b86ef9aa0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.144+0000 7ff5d9fbc740 1 -- 172.21.15.12:0/302733948 >> 172.21.15.12:0/302733948 conn(0x557b86eedcb0 msgr2=0x557b86eefa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:27.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.145+0000 7ff5d9fbc740 1 -- 172.21.15.12:0/302733948 shutdown_connections 2024-04-22T13:27:27.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.145+0000 7ff5d9fbc740 1 -- 172.21.15.12:0/302733948 wait complete. 2024-04-22T13:27:27.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.145+0000 7ff5d9fbc740 20 client.16887 trim_cache size 0 max 16384 2024-04-22T13:27:27.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.163+0000 7ff5d9fbc740 1 Processor -- start 2024-04-22T13:27:27.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.163+0000 7ff5d9fbc740 1 -- start start 2024-04-22T13:27:27.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.163+0000 7ff5d9fbc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b86ef3580 0x557b86efc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.163+0000 7ff5d9fbc740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b86ef3ea0 0x557b86f086a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.163+0000 7ff5d9fbc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b86efca80 0x557b86eef900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.163+0000 7ff5d9fbc740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d01afc70 con 0x557b86ef3ea0 2024-04-22T13:27:27.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.163+0000 7ff5d9fbc740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d01b17b0 con 0x557b86ef3580 2024-04-22T13:27:27.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.163+0000 7ff5d9fbc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d01b0340 con 0x557b86efca80 2024-04-22T13:27:27.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.163+0000 7ff5d4962640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b86ef3ea0 0x557b86f086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.163+0000 7ff5d4962640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b86ef3ea0 0x557b86f086a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53848/0 (socket says 172.21.15.12:53848) 2024-04-22T13:27:27.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.163+0000 7ff5d4962640 1 -- 172.21.15.12:0/3306883931 learned_addr learned my addr 172.21.15.12:0/3306883931 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:27.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.163+0000 7ff5d5163640 1 --2- 172.21.15.12:0/3306883931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b86ef3580 0x557b86efc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.163+0000 7ff5d5964640 1 --2- 172.21.15.12:0/3306883931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b86efca80 0x557b86eef900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.163+0000 7ff5d4962640 1 -- 172.21.15.12:0/3306883931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b86efca80 msgr2=0x557b86eef900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.163 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.163+0000 7ff5d4962640 1 --2- 172.21.15.12:0/3306883931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b86efca80 0x557b86eef900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.163+0000 7ff5d4962640 1 -- 172.21.15.12:0/3306883931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b86ef3580 msgr2=0x557b86efc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.163+0000 7ff5d4962640 1 --2- 172.21.15.12:0/3306883931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b86ef3580 0x557b86efc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.163+0000 7ff5d4962640 1 -- 172.21.15.12:0/3306883931 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b86ef3cd0 con 0x557b86ef3ea0 2024-04-22T13:27:27.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.163+0000 7ff5d5964640 1 --2- 172.21.15.12:0/3306883931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b86efca80 0x557b86eef900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:27.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.163+0000 7ff5d5163640 1 --2- 172.21.15.12:0/3306883931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b86ef3580 0x557b86efc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:27.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.163+0000 7ff5d4962640 1 --2- 172.21.15.12:0/3306883931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b86ef3ea0 0x557b86f086a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7ff5c00ce980 tx=0x7ff5c0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a468dd232f812f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:27.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.164+0000 7ff5baffd640 1 -- 172.21.15.12:0/3306883931 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c00b1410 con 0x557b86ef3ea0 2024-04-22T13:27:27.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.164+0000 7ff5baffd640 1 -- 172.21.15.12:0/3306883931 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5c0005680 con 0x557b86ef3ea0 2024-04-22T13:27:27.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.164+0000 7ff5baffd640 1 -- 172.21.15.12:0/3306883931 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c009c530 con 0x557b86ef3ea0 2024-04-22T13:27:27.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.164+0000 7ff5d9fbc740 1 -- 172.21.15.12:0/3306883931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b86ef3ea0 msgr2=0x557b86f086a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.164+0000 7ff5d9fbc740 1 --2- 172.21.15.12:0/3306883931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b86ef3ea0 0x557b86f086a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7ff5c00ce980 tx=0x7ff5c0009310 comp rx=0 tx=0).stop 2024-04-22T13:27:27.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.164+0000 7ff5d9fbc740 1 -- 172.21.15.12:0/3306883931 shutdown_connections 2024-04-22T13:27:27.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.164+0000 7ff5d9fbc740 1 --2- 172.21.15.12:0/3306883931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b86efca80 0x557b86eef900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.164+0000 7ff5d9fbc740 1 --2- 172.21.15.12:0/3306883931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b86ef3ea0 0x557b86f086a0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.164+0000 7ff5d9fbc740 1 --2- 172.21.15.12:0/3306883931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b86ef3580 0x557b86efc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.164 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.164+0000 7ff5d9fbc740 1 -- 172.21.15.12:0/3306883931 >> 172.21.15.12:0/3306883931 conn(0x557b86eedcb0 msgr2=0x557b86ef89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:27.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.164+0000 7ff5d9fbc740 1 -- 172.21.15.12:0/3306883931 shutdown_connections 2024-04-22T13:27:27.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.164+0000 7ff5d9fbc740 1 -- 172.21.15.12:0/3306883931 wait complete. 2024-04-22T13:27:27.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.164+0000 7ff5d9fbc740 1 -- start start 2024-04-22T13:27:27.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.165+0000 7ff5d9fbc740 1 Processor -- start 2024-04-22T13:27:27.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.165+0000 7ff5d9fbc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b86ef3580 0x557b86f04bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.165+0000 7ff5d9fbc740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b86ef3ea0 0x557b86f050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.165+0000 7ff5d9fbc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b86efca80 0x557b86f01b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.165+0000 7ff5d9fbc740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d01b10e0 con 0x557b86ef3ea0 2024-04-22T13:27:27.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.165+0000 7ff5d9fbc740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d01b0a10 con 0x557b86ef3580 2024-04-22T13:27:27.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.165+0000 7ff5d9fbc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d01adc70 con 0x557b86efca80 2024-04-22T13:27:27.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.165+0000 7ff5d9fbc740 10 client.0 _ll_register_callbacks cb 0x7ff5cc004370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:27.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.165+0000 7ff5d4962640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b86ef3ea0 0x557b86f050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.165+0000 7ff5d4962640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b86ef3ea0 0x557b86f050f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53850/0 (socket says 172.21.15.12:53850) 2024-04-22T13:27:27.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.165+0000 7ff5d4962640 1 -- 172.21.15.12:0/3936712235 learned_addr learned my addr 172.21.15.12:0/3936712235 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:27.165 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.165+0000 7ff5d5964640 1 --2- 172.21.15.12:0/3936712235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b86efca80 0x557b86f01b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.165+0000 7ff5d5163640 1 --2- 172.21.15.12:0/3936712235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b86ef3580 0x557b86f04bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.165+0000 7ff5d4962640 1 -- 172.21.15.12:0/3936712235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b86efca80 msgr2=0x557b86f01b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.165+0000 7ff5d4962640 1 --2- 172.21.15.12:0/3936712235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b86efca80 0x557b86f01b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.165+0000 7ff5d4962640 1 -- 172.21.15.12:0/3936712235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b86ef3580 msgr2=0x557b86f04bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.165+0000 7ff5d4962640 1 --2- 172.21.15.12:0/3936712235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b86ef3580 0x557b86f04bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.166+0000 7ff5d4962640 1 -- 172.21.15.12:0/3936712235 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b86f02270 con 0x557b86ef3ea0 2024-04-22T13:27:27.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.166+0000 7ff5d4962640 1 --2- 172.21.15.12:0/3936712235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b86ef3ea0 0x557b86f050f0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7ff5c00ce430 tx=0x7ff5c00b61b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2b37928df167b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:27.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.166+0000 7ff5b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:27:27.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.166+0000 7ff5b8ff9640 1 -- 172.21.15.12:0/3936712235 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c00b1410 con 0x557b86ef3ea0 2024-04-22T13:27:27.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.166+0000 7ff5b8ff9640 1 -- 172.21.15.12:0/3936712235 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7ff5c009aed0 con 0x557b86ef3ea0 2024-04-22T13:27:27.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.166+0000 7ff5d9fbc740 1 -- 172.21.15.12:0/3936712235 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5c0005680 con 0x557b86ef3ea0 2024-04-22T13:27:27.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.166+0000 7ff5b8ff9640 1 -- 172.21.15.12:0/3936712235 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff5c009d2d0 con 0x557b86ef3ea0 2024-04-22T13:27:27.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.166+0000 7ff5d9fbc740 10 client.16893 fetch_fsmap learned FSMap version 28 2024-04-22T13:27:27.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.166+0000 7ff5d9fbc740 1 -- 172.21.15.12:0/3936712235 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557b86f73880 con 0x557b86ef3ea0 2024-04-22T13:27:27.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.166+0000 7ff5b8ff9640 1 -- 172.21.15.12:0/3936712235 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5c0005680 con 0x557b86ef3ea0 2024-04-22T13:27:27.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.166+0000 7ff5b8ff9640 1 -- 172.21.15.12:0/3936712235 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff5c00ce600 con 0x557b86ef3ea0 2024-04-22T13:27:27.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.166+0000 7ff5d9fbc740 10 client.16893 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:27:27.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.166+0000 7ff5d9fbc740 10 client.16893 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:27:27.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.166+0000 7ff5d9fbc740 20 client.16893 populate_metadata read hostname 'smithi012' 2024-04-22T13:27:27.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.167+0000 7ff5d9fbc740 1 --2- 172.21.15.12:0/3936712235 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557b86f73aa0 0x557b86f75e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.167+0000 7ff5d9fbc740 4 client.16893 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:27:27.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.167+0000 7ff5d9fbc740 1 -- 172.21.15.12:0/3936712235 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557b86e5fa70 con 0x557b86f73aa0 2024-04-22T13:27:27.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.167+0000 7ff5d5163640 1 --2- 172.21.15.12:0/3936712235 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557b86f73aa0 0x557b86f75e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.167+0000 7ff5d5163640 1 --2- 172.21.15.12:0/3936712235 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557b86f73aa0 0x557b86f75e90 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd376288c9720a01 server_cookie=9503b36277dfa65e in_seq=0 out_seq=0 2024-04-22T13:27:27.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.167+0000 7ff5b8ff9640 10 client.16893 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:27:27.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.167+0000 7ff5b8ff9640 1 -- 172.21.15.12:0/3936712235 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557b86e5fa70 con 0x557b86f73aa0 2024-04-22T13:27:27.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.167+0000 7ff5b8ff9640 10 client.16893 handle_command_reply: tid=0 2024-04-22T13:27:27.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.167+0000 7ff5d9fbc740 1 client.16893 shutdown 2024-04-22T13:27:27.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.167+0000 7ff5d9fbc740 1 -- 172.21.15.12:0/3936712235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b86ef3ea0 msgr2=0x557b86f050f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.167+0000 7ff5d9fbc740 1 --2- 172.21.15.12:0/3936712235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b86ef3ea0 0x557b86f050f0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7ff5c00ce430 tx=0x7ff5c00b61b0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.168+0000 7ff5d9fbc740 1 -- 172.21.15.12:0/3936712235 shutdown_connections 2024-04-22T13:27:27.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.168+0000 7ff5d9fbc740 1 --2- 172.21.15.12:0/3936712235 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557b86f73aa0 0x557b86f75e90 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.168+0000 7ff5d9fbc740 1 --2- 172.21.15.12:0/3936712235 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557b86efca80 0x557b86f01b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.168+0000 7ff5d9fbc740 1 --2- 172.21.15.12:0/3936712235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557b86ef3ea0 0x557b86f050f0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.168+0000 7ff5d9fbc740 1 --2- 172.21.15.12:0/3936712235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b86ef3580 0x557b86f04bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.168+0000 7ff5b8ff9640 0 client.16893 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:27:27.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.168+0000 7ff5d9fbc740 1 -- 172.21.15.12:0/3936712235 >> 172.21.15.12:0/3936712235 conn(0x557b86eedcb0 msgr2=0x557b86efa2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:27.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.168+0000 7ff5d9fbc740 1 -- 172.21.15.12:0/3936712235 shutdown_connections 2024-04-22T13:27:27.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.168+0000 7ff5d9fbc740 1 -- 172.21.15.12:0/3936712235 wait complete. 2024-04-22T13:27:27.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.168+0000 7ff5d9fbc740 20 client.16893 trim_cache size 0 max 16384 2024-04-22T13:27:27.169 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:27:27.169 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-04-22T13:27:27.170 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-04-22T13:27:27.170 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:27:27.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.168+0000 7ff5d73c7640 1 -- 172.21.15.12:0/356681705 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff5bc0a3d20 msgr2=0x7ff5bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.168+0000 7ff5d73c7640 1 --2- 172.21.15.12:0/356681705 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff5bc0a3d20 0x7ff5bc0a61e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8004af0 tx=0x7ff5c802e040 comp rx=0 tx=0).stop 2024-04-22T13:27:27.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.168+0000 7ff5d73c7640 1 -- 172.21.15.12:0/356681705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d0102e70 msgr2=0x7ff5d01a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.168+0000 7ff5d73c7640 1 --2- 172.21.15.12:0/356681705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d0102e70 0x7ff5d01a0b50 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000e9f0 tx=0x7ff5c000edf0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.168+0000 7ff5d73c7640 1 -- 172.21.15.12:0/356681705 shutdown_connections 2024-04-22T13:27:27.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.168+0000 7ff5d73c7640 1 --2- 172.21.15.12:0/356681705 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff5bc0a3d20 0x7ff5bc0a61e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.168+0000 7ff5d73c7640 1 --2- 172.21.15.12:0/356681705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5d0103250 0x7ff5d019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.168+0000 7ff5d73c7640 1 --2- 172.21.15.12:0/356681705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5d0102e70 0x7ff5d01a0b50 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.168+0000 7ff5d73c7640 1 --2- 172.21.15.12:0/356681705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5d0100fb0 0x7ff5d01a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.168+0000 7ff5d73c7640 1 -- 172.21.15.12:0/356681705 >> 172.21.15.12:0/356681705 conn(0x7ff5d00773a0 msgr2=0x7ff5d00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:27.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.168+0000 7ff5d73c7640 1 -- 172.21.15.12:0/356681705 shutdown_connections 2024-04-22T13:27:27.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.168+0000 7ff5d73c7640 1 -- 172.21.15.12:0/356681705 wait complete. 2024-04-22T13:27:27.175 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:27:27.175 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:27:27.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.283+0000 7fd83c2e0640 1 -- 172.21.15.12:0/2665430811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd83410c5a0 msgr2=0x7fd83410e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.283+0000 7fd83c2e0640 1 --2- 172.21.15.12:0/2665430811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd83410c5a0 0x7fd83410e9d0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009ca0 tx=0x7fd82c02c190 comp rx=0 tx=0).stop 2024-04-22T13:27:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.283+0000 7fd83c2e0640 1 -- 172.21.15.12:0/2665430811 shutdown_connections 2024-04-22T13:27:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.283+0000 7fd83c2e0640 1 --2- 172.21.15.12:0/2665430811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd83410c5a0 0x7fd83410e9d0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.283+0000 7fd83c2e0640 1 --2- 172.21.15.12:0/2665430811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd834103af0 0x7fd83410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.283+0000 7fd83c2e0640 1 --2- 172.21.15.12:0/2665430811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8341031d0 0x7fd8341035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.283+0000 7fd83c2e0640 1 -- 172.21.15.12:0/2665430811 >> 172.21.15.12:0/2665430811 conn(0x7fd8340771d0 msgr2=0x7fd8340775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.283+0000 7fd83c2e0640 1 -- 172.21.15.12:0/2665430811 shutdown_connections 2024-04-22T13:27:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.283+0000 7fd83c2e0640 1 -- 172.21.15.12:0/2665430811 wait complete. 2024-04-22T13:27:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.283+0000 7fd83c2e0640 1 Processor -- start 2024-04-22T13:27:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.283+0000 7fd83c2e0640 1 -- start start 2024-04-22T13:27:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.284+0000 7fd83c2e0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8341031d0 0x7fd834198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.284+0000 7fd83c2e0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd834103af0 0x7fd834198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.284+0000 7fd83c2e0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd83410c5a0 0x7fd83419cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.284+0000 7fd83c2e0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd8341121c0 con 0x7fd8341031d0 2024-04-22T13:27:27.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.284+0000 7fd83c2e0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd834112040 con 0x7fd834103af0 2024-04-22T13:27:27.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.284+0000 7fd83c2e0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd834112340 con 0x7fd83410c5a0 2024-04-22T13:27:27.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.284+0000 7fd83a07c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8341031d0 0x7fd834198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.284+0000 7fd83a07c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8341031d0 0x7fd834198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53858/0 (socket says 172.21.15.12:53858) 2024-04-22T13:27:27.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.284+0000 7fd83a07c640 1 -- 172.21.15.12:0/859302457 learned_addr learned my addr 172.21.15.12:0/859302457 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:27.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.284+0000 7fd83a87d640 1 --2- 172.21.15.12:0/859302457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd83410c5a0 0x7fd83419cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.284+0000 7fd83987b640 1 --2- 172.21.15.12:0/859302457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd834103af0 0x7fd834198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.284+0000 7fd83a07c640 1 -- 172.21.15.12:0/859302457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd83410c5a0 msgr2=0x7fd83419cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.284+0000 7fd83a07c640 1 --2- 172.21.15.12:0/859302457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd83410c5a0 0x7fd83419cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.284+0000 7fd83a07c640 1 -- 172.21.15.12:0/859302457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd834103af0 msgr2=0x7fd834198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.284+0000 7fd83a07c640 1 --2- 172.21.15.12:0/859302457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd834103af0 0x7fd834198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.284+0000 7fd83a07c640 1 -- 172.21.15.12:0/859302457 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd83419d620 con 0x7fd8341031d0 2024-04-22T13:27:27.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.284+0000 7fd83987b640 1 --2- 172.21.15.12:0/859302457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd834103af0 0x7fd834198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:27.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.284+0000 7fd83a07c640 1 --2- 172.21.15.12:0/859302457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8341031d0 0x7fd834198680 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd83000d770 tx=0x7fd83000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:27.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.284+0000 7fd82b7fe640 1 -- 172.21.15.12:0/859302457 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd830017070 con 0x7fd8341031d0 2024-04-22T13:27:27.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.284+0000 7fd83c2e0640 1 -- 172.21.15.12:0/859302457 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd83419d910 con 0x7fd8341031d0 2024-04-22T13:27:27.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.284+0000 7fd82b7fe640 1 -- 172.21.15.12:0/859302457 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8300054d0 con 0x7fd8341031d0 2024-04-22T13:27:27.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.284+0000 7fd82b7fe640 1 -- 172.21.15.12:0/859302457 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd830004c30 con 0x7fd8341031d0 2024-04-22T13:27:27.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.285+0000 7fd83c2e0640 1 -- 172.21.15.12:0/859302457 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8341a51a0 con 0x7fd8341031d0 2024-04-22T13:27:27.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.285+0000 7fd83c2e0640 1 -- 172.21.15.12:0/859302457 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd834069230 con 0x7fd8341031d0 2024-04-22T13:27:27.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.287+0000 7fd82b7fe640 1 -- 172.21.15.12:0/859302457 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd830004dd0 con 0x7fd8341031d0 2024-04-22T13:27:27.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.287+0000 7fd82b7fe640 1 --2- 172.21.15.12:0/859302457 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd8200a3d20 0x7fd8200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.287+0000 7fd83987b640 1 --2- 172.21.15.12:0/859302457 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd8200a3d20 0x7fd8200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.287+0000 7fd82b7fe640 1 -- 172.21.15.12:0/859302457 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fd8300d0670 con 0x7fd8341031d0 2024-04-22T13:27:27.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.288+0000 7fd83987b640 1 --2- 172.21.15.12:0/859302457 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd8200a3d20 0x7fd8200a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd834100620 tx=0x7fd824005d20 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:27.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.291+0000 7fd82b7fe640 1 -- 172.21.15.12:0/859302457 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd83009cd60 con 0x7fd8341031d0 2024-04-22T13:27:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.522+0000 7fd83c2e0640 1 -- 172.21.15.12:0/859302457 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8341a53d0 con 0x7fd8341031d0 2024-04-22T13:27:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.523+0000 7fd82b7fe640 1 -- 172.21.15.12:0/859302457 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7fd83009c660 con 0x7fd8341031d0 2024-04-22T13:27:27.519 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:27:27.519 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:27:27.519 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:27:27.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.525+0000 7fd83c2e0640 1 -- 172.21.15.12:0/859302457 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd8200a3d20 msgr2=0x7fd8200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.525+0000 7fd83c2e0640 1 --2- 172.21.15.12:0/859302457 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd8200a3d20 0x7fd8200a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd834100620 tx=0x7fd824005d20 comp rx=0 tx=0).stop 2024-04-22T13:27:27.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.525+0000 7fd83c2e0640 1 -- 172.21.15.12:0/859302457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8341031d0 msgr2=0x7fd834198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.525+0000 7fd83c2e0640 1 --2- 172.21.15.12:0/859302457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8341031d0 0x7fd834198680 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd83000d770 tx=0x7fd83000dc30 comp rx=0 tx=0).stop 2024-04-22T13:27:27.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.525+0000 7fd83c2e0640 1 -- 172.21.15.12:0/859302457 shutdown_connections 2024-04-22T13:27:27.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.525+0000 7fd83c2e0640 1 --2- 172.21.15.12:0/859302457 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd8200a3d20 0x7fd8200a61e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.525+0000 7fd83c2e0640 1 --2- 172.21.15.12:0/859302457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd83410c5a0 0x7fd83419cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.525+0000 7fd83c2e0640 1 --2- 172.21.15.12:0/859302457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd834103af0 0x7fd834198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.525+0000 7fd83c2e0640 1 --2- 172.21.15.12:0/859302457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd8341031d0 0x7fd834198680 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.525+0000 7fd83c2e0640 1 -- 172.21.15.12:0/859302457 >> 172.21.15.12:0/859302457 conn(0x7fd8340771d0 msgr2=0x7fd8340fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:27.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.525+0000 7fd83c2e0640 1 -- 172.21.15.12:0/859302457 shutdown_connections 2024-04-22T13:27:27.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.525+0000 7fd83c2e0640 1 -- 172.21.15.12:0/859302457 wait complete. 2024-04-22T13:27:27.532 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:27:27.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.641+0000 7f302456d640 1 -- 172.21.15.12:0/2707169022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f301c106160 msgr2=0x7f301c10dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.641+0000 7f302456d640 1 --2- 172.21.15.12:0/2707169022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f301c106160 0x7f301c10dd30 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f3014009ca0 tx=0x7f301402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.641+0000 7f302456d640 1 -- 172.21.15.12:0/2707169022 shutdown_connections 2024-04-22T13:27:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.641+0000 7f302456d640 1 --2- 172.21.15.12:0/2707169022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f301c106160 0x7f301c10dd30 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.641+0000 7f302456d640 1 --2- 172.21.15.12:0/2707169022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f301c101a40 0x7f301c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.641+0000 7f302456d640 1 --2- 172.21.15.12:0/2707169022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f301c101090 0x7f301c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.641+0000 7f302456d640 1 -- 172.21.15.12:0/2707169022 >> 172.21.15.12:0/2707169022 conn(0x7f301c077f50 msgr2=0x7f301c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.641+0000 7f302456d640 1 -- 172.21.15.12:0/2707169022 shutdown_connections 2024-04-22T13:27:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.641+0000 7f302456d640 1 -- 172.21.15.12:0/2707169022 wait complete. 2024-04-22T13:27:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.641+0000 7f302456d640 1 Processor -- start 2024-04-22T13:27:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.642+0000 7f302456d640 1 -- start start 2024-04-22T13:27:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.642+0000 7f302456d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f301c101090 0x7f301c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.642+0000 7f302456d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f301c106160 0x7f301c198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.642+0000 7f302456d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f301c19cfc0 0x7f301c19d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.642+0000 7f302456d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f301c1121e0 con 0x7f301c106160 2024-04-22T13:27:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.642+0000 7f302456d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f301c112060 con 0x7f301c101090 2024-04-22T13:27:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.642+0000 7f302456d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f301c112360 con 0x7f301c19cfc0 2024-04-22T13:27:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.642+0000 7f3021b08640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f301c106160 0x7f301c198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.642+0000 7f3021b08640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f301c106160 0x7f301c198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53880/0 (socket says 172.21.15.12:53880) 2024-04-22T13:27:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.642+0000 7f3021b08640 1 -- 172.21.15.12:0/3850576593 learned_addr learned my addr 172.21.15.12:0/3850576593 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.642+0000 7f3022b0a640 1 --2- 172.21.15.12:0/3850576593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f301c19cfc0 0x7f301c19d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.642+0000 7f3022309640 1 --2- 172.21.15.12:0/3850576593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f301c101090 0x7f301c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.642+0000 7f3021b08640 1 -- 172.21.15.12:0/3850576593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f301c19cfc0 msgr2=0x7f301c19d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.642+0000 7f3021b08640 1 --2- 172.21.15.12:0/3850576593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f301c19cfc0 0x7f301c19d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.642+0000 7f3021b08640 1 -- 172.21.15.12:0/3850576593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f301c101090 msgr2=0x7f301c198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.642+0000 7f3021b08640 1 --2- 172.21.15.12:0/3850576593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f301c101090 0x7f301c198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.642+0000 7f3021b08640 1 -- 172.21.15.12:0/3850576593 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f301c19dab0 con 0x7f301c106160 2024-04-22T13:27:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.642+0000 7f3022309640 1 --2- 172.21.15.12:0/3850576593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f301c101090 0x7f301c198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:27.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.642+0000 7f3021b08640 1 --2- 172.21.15.12:0/3850576593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f301c106160 0x7f301c198c40 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f300c00d740 tx=0x7f300c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.642+0000 7f30137fe640 1 -- 172.21.15.12:0/3850576593 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c017070 con 0x7f301c106160 2024-04-22T13:27:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.642+0000 7f30137fe640 1 -- 172.21.15.12:0/3850576593 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f300c00de50 con 0x7f301c106160 2024-04-22T13:27:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.642+0000 7f302456d640 1 -- 172.21.15.12:0/3850576593 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f301c19dda0 con 0x7f301c106160 2024-04-22T13:27:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.642+0000 7f30137fe640 1 -- 172.21.15.12:0/3850576593 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c004ce0 con 0x7f301c106160 2024-04-22T13:27:27.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.643+0000 7f302456d640 1 -- 172.21.15.12:0/3850576593 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f301c1a5540 con 0x7f301c106160 2024-04-22T13:27:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.645+0000 7f30137fe640 1 -- 172.21.15.12:0/3850576593 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f300c004e80 con 0x7f301c106160 2024-04-22T13:27:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.645+0000 7f3027234740 1 Processor -- start 2024-04-22T13:27:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.645+0000 7f3027234740 1 -- start start 2024-04-22T13:27:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.645+0000 7f3027234740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595f15f24d0 0x5595f15f28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.645+0000 7f3027234740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595f15f2e80 0x5595f15fb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.645+0000 7f3027234740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595f15fba00 0x5595f15fde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.645+0000 7f3027234740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5595f15571f0 con 0x5595f15f24d0 2024-04-22T13:27:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.645+0000 7f3027234740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5595f14f44b0 con 0x5595f15f2e80 2024-04-22T13:27:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.645+0000 7f3027234740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5595f15531b0 con 0x5595f15fba00 2024-04-22T13:27:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.646+0000 7f30137fe640 1 --2- 172.21.15.12:0/3850576593 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f30080a3d20 0x7f30080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.646+0000 7f3022b0a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595f15f24d0 0x5595f15f28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.646+0000 7f3022b0a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595f15f24d0 0x5595f15f28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53892/0 (socket says 172.21.15.12:53892) 2024-04-22T13:27:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.646+0000 7f3022b0a640 1 -- 172.21.15.12:0/416949502 learned_addr learned my addr 172.21.15.12:0/416949502 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.646+0000 7f3022309640 1 --2- 172.21.15.12:0/416949502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595f15f2e80 0x5595f15fb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.646+0000 7f3022b0a640 1 --2- 172.21.15.12:0/3850576593 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f30080a3d20 0x7f30080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.646+0000 7f30137fe640 1 -- 172.21.15.12:0/3850576593 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f300c0d0180 con 0x7f301c106160 2024-04-22T13:27:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.646+0000 7f3021b08640 1 --2- 172.21.15.12:0/416949502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595f15fba00 0x5595f15fde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.646+0000 7f3022b0a640 1 -- 172.21.15.12:0/416949502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595f15fba00 msgr2=0x5595f15fde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.646+0000 7f3022b0a640 1 --2- 172.21.15.12:0/416949502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595f15fba00 0x5595f15fde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.646+0000 7f3022b0a640 1 -- 172.21.15.12:0/416949502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595f15f2e80 msgr2=0x5595f15fb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.646+0000 7f3022b0a640 1 --2- 172.21.15.12:0/416949502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595f15f2e80 0x5595f15fb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.646+0000 7f3022b0a640 1 -- 172.21.15.12:0/416949502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595f14e61b0 con 0x5595f15f24d0 2024-04-22T13:27:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.646+0000 7f3021b08640 1 --2- 172.21.15.12:0/416949502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595f15fba00 0x5595f15fde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.646+0000 7f3022b0a640 1 --2- 172.21.15.12:0/416949502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595f15f24d0 0x5595f15f28b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f3014009430 tx=0x7f3014032f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926cab15eef81819 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.646+0000 7f3022b0a640 1 --2- 172.21.15.12:0/3850576593 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f30080a3d20 0x7f30080a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3014009a60 tx=0x7f3014041000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.646+0000 7f3010bfe640 1 -- 172.21.15.12:0/416949502 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3014051070 con 0x5595f15f24d0 2024-04-22T13:27:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.647+0000 7f3010bfe640 1 -- 172.21.15.12:0/416949502 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3014048070 con 0x5595f15f24d0 2024-04-22T13:27:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.647+0000 7f3010bfe640 1 -- 172.21.15.12:0/416949502 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3014039040 con 0x5595f15f24d0 2024-04-22T13:27:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.647+0000 7f3027234740 1 -- 172.21.15.12:0/416949502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595f15f24d0 msgr2=0x5595f15f28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.647+0000 7f3027234740 1 --2- 172.21.15.12:0/416949502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595f15f24d0 0x5595f15f28b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f3014009430 tx=0x7f3014032f10 comp rx=0 tx=0).stop 2024-04-22T13:27:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.647+0000 7f3027234740 1 -- 172.21.15.12:0/416949502 shutdown_connections 2024-04-22T13:27:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.647+0000 7f3027234740 1 --2- 172.21.15.12:0/416949502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595f15fba00 0x5595f15fde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.647+0000 7f3027234740 1 --2- 172.21.15.12:0/416949502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595f15f2e80 0x5595f15fb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.647+0000 7f3027234740 1 --2- 172.21.15.12:0/416949502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595f15f24d0 0x5595f15f28b0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.647+0000 7f3027234740 1 -- 172.21.15.12:0/416949502 >> 172.21.15.12:0/416949502 conn(0x5595f15ecba0 msgr2=0x5595f15ecf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:27.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.647+0000 7f3027234740 1 -- 172.21.15.12:0/416949502 shutdown_connections 2024-04-22T13:27:27.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.647+0000 7f3027234740 1 -- 172.21.15.12:0/416949502 wait complete. 2024-04-22T13:27:27.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.648+0000 7f3027234740 1 -- start start 2024-04-22T13:27:27.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.648+0000 7f3027234740 1 Processor -- start 2024-04-22T13:27:27.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.648+0000 7f3027234740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595f15f24d0 0x5595f15fb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.648+0000 7f3027234740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595f15f2e80 0x5595f15f8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.648+0000 7f3027234740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595f15fba00 0x5595f15f8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.648+0000 7f3027234740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5595f15fe910 con 0x5595f15f2e80 2024-04-22T13:27:27.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.648+0000 7f3027234740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5595f1510500 con 0x5595f15fba00 2024-04-22T13:27:27.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.648+0000 7f3027234740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5595f15fea90 con 0x5595f15f24d0 2024-04-22T13:27:27.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.648+0000 7f3027234740 10 client.0 _ll_register_callbacks cb 0x5595f14deb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:27.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.648+0000 7f3021b08640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595f15f2e80 0x5595f15f8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.648+0000 7f3021b08640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595f15f2e80 0x5595f15f8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53900/0 (socket says 172.21.15.12:53900) 2024-04-22T13:27:27.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.648+0000 7f3021b08640 1 -- 172.21.15.12:0/2189879358 learned_addr learned my addr 172.21.15.12:0/2189879358 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:27.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.648+0000 7f3022b0a640 1 --2- 172.21.15.12:0/2189879358 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595f15fba00 0x5595f15f8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.648+0000 7f3022309640 1 --2- 172.21.15.12:0/2189879358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595f15f24d0 0x5595f15fb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.648+0000 7f3021b08640 1 -- 172.21.15.12:0/2189879358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595f15f24d0 msgr2=0x5595f15fb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.648+0000 7f3021b08640 1 --2- 172.21.15.12:0/2189879358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595f15f24d0 0x5595f15fb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.648+0000 7f3021b08640 1 -- 172.21.15.12:0/2189879358 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595f15fba00 msgr2=0x5595f15f8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.648+0000 7f3021b08640 1 --2- 172.21.15.12:0/2189879358 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595f15fba00 0x5595f15f8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.648+0000 7f3021b08640 1 -- 172.21.15.12:0/2189879358 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595f12693f0 con 0x5595f15f2e80 2024-04-22T13:27:27.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.649+0000 7f3021b08640 1 --2- 172.21.15.12:0/2189879358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595f15f2e80 0x5595f15f8700 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f300c09c100 tx=0x7f300c09c170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d03b3c3084e552 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:27.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.649+0000 7f3005ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:27:27.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.649+0000 7f3005ffb640 1 -- 172.21.15.12:0/2189879358 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c020040 con 0x5595f15f2e80 2024-04-22T13:27:27.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.649+0000 7f3005ffb640 1 -- 172.21.15.12:0/2189879358 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f300c0987e0 con 0x5595f15f2e80 2024-04-22T13:27:27.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.649+0000 7f3027234740 1 -- 172.21.15.12:0/2189879358 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3014048070 con 0x5595f15f2e80 2024-04-22T13:27:27.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.649+0000 7f3005ffb640 1 -- 172.21.15.12:0/2189879358 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f300c0b9020 con 0x5595f15f2e80 2024-04-22T13:27:27.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.649+0000 7f3027234740 10 client.16911 fetch_fsmap learned FSMap version 28 2024-04-22T13:27:27.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.649+0000 7f3027234740 1 -- 172.21.15.12:0/2189879358 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5595f13337d0 con 0x5595f15f2e80 2024-04-22T13:27:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.649+0000 7f3005ffb640 1 -- 172.21.15.12:0/2189879358 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3014048070 con 0x5595f15f2e80 2024-04-22T13:27:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.649+0000 7f3005ffb640 1 -- 172.21.15.12:0/2189879358 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f300c096ce0 con 0x5595f15f2e80 2024-04-22T13:27:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.649+0000 7f3027234740 10 client.16911 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:27:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.649+0000 7f3027234740 10 client.16911 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:27:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.649+0000 7f3027234740 20 client.16911 populate_metadata read hostname 'smithi012' 2024-04-22T13:27:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.649+0000 7f3027234740 1 --2- 172.21.15.12:0/2189879358 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5595f160bca0 0x5595f160e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.649+0000 7f3027234740 4 client.16911 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.649+0000 7f3027234740 1 -- 172.21.15.12:0/2189879358 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595f14f6930 con 0x5595f160bca0 2024-04-22T13:27:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.650+0000 7f3022309640 1 --2- 172.21.15.12:0/2189879358 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5595f160bca0 0x5595f160e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.650+0000 7f3005ffb640 10 client.16911 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:27:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.650+0000 7f3022309640 1 --2- 172.21.15.12:0/2189879358 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5595f160bca0 0x5595f160e0b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f7cc2cbe9bee3ae1 server_cookie=acbb8725e463553 in_seq=0 out_seq=0 2024-04-22T13:27:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.651+0000 7f3005ffb640 1 -- 172.21.15.12:0/2189879358 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5595f14f6930 con 0x5595f160bca0 2024-04-22T13:27:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.651+0000 7f3005ffb640 10 client.16911 handle_command_reply: tid=0 2024-04-22T13:27:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.651+0000 7f3027234740 1 client.16911 shutdown 2024-04-22T13:27:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.651+0000 7f3027234740 1 -- 172.21.15.12:0/2189879358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595f15f2e80 msgr2=0x5595f15f8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.651+0000 7f3027234740 1 --2- 172.21.15.12:0/2189879358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595f15f2e80 0x5595f15f8700 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f300c09c100 tx=0x7f300c09c170 comp rx=0 tx=0).stop 2024-04-22T13:27:27.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.651+0000 7f3027234740 1 -- 172.21.15.12:0/2189879358 shutdown_connections 2024-04-22T13:27:27.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.651+0000 7f3027234740 1 --2- 172.21.15.12:0/2189879358 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5595f160bca0 0x5595f160e0b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.651+0000 7f3027234740 1 --2- 172.21.15.12:0/2189879358 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595f15fba00 0x5595f15f8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.651+0000 7f3027234740 1 --2- 172.21.15.12:0/2189879358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595f15f2e80 0x5595f15f8700 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.651+0000 7f3005ffb640 0 client.16911 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:27:27.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.651+0000 7f3027234740 1 --2- 172.21.15.12:0/2189879358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595f15f24d0 0x5595f15fb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.651+0000 7f3027234740 1 -- 172.21.15.12:0/2189879358 >> 172.21.15.12:0/2189879358 conn(0x5595f15ecba0 msgr2=0x5595f15ee8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:27.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.651+0000 7f3027234740 1 -- 172.21.15.12:0/2189879358 shutdown_connections 2024-04-22T13:27:27.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.651+0000 7f3027234740 1 -- 172.21.15.12:0/2189879358 wait complete. 2024-04-22T13:27:27.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.651+0000 7f3027234740 20 client.16911 trim_cache size 0 max 16384 2024-04-22T13:27:27.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.665+0000 7f3027234740 1 Processor -- start 2024-04-22T13:27:27.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.665+0000 7f3027234740 1 -- start start 2024-04-22T13:27:27.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.665+0000 7f3027234740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595f15f24d0 0x5595f15ee7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.665+0000 7f3027234740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595f15f2e80 0x5595f15fae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.665+0000 7f3027234740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595f15fba00 0x5595f15fb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.665+0000 7f3027234740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f301c1ada70 con 0x5595f15f24d0 2024-04-22T13:27:27.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.665+0000 7f3027234740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f301c1af5b0 con 0x5595f15fba00 2024-04-22T13:27:27.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.665+0000 7f3027234740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f301c1ae140 con 0x5595f15f2e80 2024-04-22T13:27:27.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.665+0000 7f3022309640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595f15f24d0 0x5595f15ee7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.665+0000 7f3022309640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595f15f24d0 0x5595f15ee7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53902/0 (socket says 172.21.15.12:53902) 2024-04-22T13:27:27.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.665+0000 7f3022309640 1 -- 172.21.15.12:0/1294989660 learned_addr learned my addr 172.21.15.12:0/1294989660 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:27.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.665+0000 7f3021b08640 1 --2- 172.21.15.12:0/1294989660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595f15f2e80 0x5595f15fae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.665+0000 7f3022b0a640 1 --2- 172.21.15.12:0/1294989660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595f15fba00 0x5595f15fb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.665+0000 7f3022309640 1 -- 172.21.15.12:0/1294989660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595f15f2e80 msgr2=0x5595f15fae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.665+0000 7f3022309640 1 --2- 172.21.15.12:0/1294989660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595f15f2e80 0x5595f15fae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.665+0000 7f3022309640 1 -- 172.21.15.12:0/1294989660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595f15fba00 msgr2=0x5595f15fb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.665+0000 7f3022309640 1 --2- 172.21.15.12:0/1294989660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595f15fba00 0x5595f15fb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.665+0000 7f3022309640 1 -- 172.21.15.12:0/1294989660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595f160a840 con 0x5595f15f24d0 2024-04-22T13:27:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.665+0000 7f3021b08640 1 --2- 172.21.15.12:0/1294989660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595f15f2e80 0x5595f15fae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.665+0000 7f3022309640 1 --2- 172.21.15.12:0/1294989660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595f15f24d0 0x5595f15ee7b0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x5595f15f3280 tx=0x7f30180094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87bd0010f3e63e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.666+0000 7f3007fff640 1 -- 172.21.15.12:0/1294989660 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3018014110 con 0x5595f15f24d0 2024-04-22T13:27:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.666+0000 7f3007fff640 1 -- 172.21.15.12:0/1294989660 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30180142b0 con 0x5595f15f24d0 2024-04-22T13:27:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.666+0000 7f3007fff640 1 -- 172.21.15.12:0/1294989660 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3018004730 con 0x5595f15f24d0 2024-04-22T13:27:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.666+0000 7f3027234740 1 -- 172.21.15.12:0/1294989660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595f15f24d0 msgr2=0x5595f15ee7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.666+0000 7f3027234740 1 --2- 172.21.15.12:0/1294989660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595f15f24d0 0x5595f15ee7b0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x5595f15f3280 tx=0x7f30180094a0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.666+0000 7f3027234740 1 -- 172.21.15.12:0/1294989660 shutdown_connections 2024-04-22T13:27:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.666+0000 7f3027234740 1 --2- 172.21.15.12:0/1294989660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595f15fba00 0x5595f15fb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.666+0000 7f3027234740 1 --2- 172.21.15.12:0/1294989660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595f15f2e80 0x5595f15fae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.666+0000 7f3027234740 1 --2- 172.21.15.12:0/1294989660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595f15f24d0 0x5595f15ee7b0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.666+0000 7f3027234740 1 -- 172.21.15.12:0/1294989660 >> 172.21.15.12:0/1294989660 conn(0x5595f15ecba0 msgr2=0x5595f15f32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.666+0000 7f3027234740 1 -- 172.21.15.12:0/1294989660 shutdown_connections 2024-04-22T13:27:27.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.666+0000 7f3027234740 1 -- 172.21.15.12:0/1294989660 wait complete. 2024-04-22T13:27:27.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.666+0000 7f3027234740 1 -- start start 2024-04-22T13:27:27.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.667+0000 7f3027234740 1 Processor -- start 2024-04-22T13:27:27.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.667+0000 7f3027234740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595f15f24d0 0x5595f1603cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.667+0000 7f3027234740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595f15f2e80 0x5595f16041f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.667+0000 7f3027234740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595f15fba00 0x5595f1600230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.667+0000 7f3027234740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f301c1aeee0 con 0x5595f15f2e80 2024-04-22T13:27:27.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.667+0000 7f3027234740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f301c1ae810 con 0x5595f15f24d0 2024-04-22T13:27:27.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.667+0000 7f3027234740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f301c1aba70 con 0x5595f15fba00 2024-04-22T13:27:27.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.667+0000 7f3027234740 10 client.0 _ll_register_callbacks cb 0x7f300c09bda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:27.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.667+0000 7f3021b08640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595f15f2e80 0x5595f16041f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.667+0000 7f3021b08640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595f15f2e80 0x5595f16041f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53912/0 (socket says 172.21.15.12:53912) 2024-04-22T13:27:27.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.667+0000 7f3021b08640 1 -- 172.21.15.12:0/1562258515 learned_addr learned my addr 172.21.15.12:0/1562258515 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:27.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.667+0000 7f3022309640 1 --2- 172.21.15.12:0/1562258515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595f15f24d0 0x5595f1603cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.667+0000 7f3022b0a640 1 --2- 172.21.15.12:0/1562258515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595f15fba00 0x5595f1600230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.667+0000 7f3021b08640 1 -- 172.21.15.12:0/1562258515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595f15fba00 msgr2=0x5595f1600230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.667+0000 7f3021b08640 1 --2- 172.21.15.12:0/1562258515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595f15fba00 0x5595f1600230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.667+0000 7f3021b08640 1 -- 172.21.15.12:0/1562258515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595f15f24d0 msgr2=0x5595f1603cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.667+0000 7f3021b08640 1 --2- 172.21.15.12:0/1562258515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595f15f24d0 0x5595f1603cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.667+0000 7f3021b08640 1 -- 172.21.15.12:0/1562258515 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595f15f20b0 con 0x5595f15f2e80 2024-04-22T13:27:27.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.667+0000 7f3021b08640 1 --2- 172.21.15.12:0/1562258515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595f15f2e80 0x5595f16041f0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f300c0cdf20 tx=0x7f300c09c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef351d7b538fae7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:27.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.667+0000 7f3005ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:27:27.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.668+0000 7f3005ffb640 1 -- 172.21.15.12:0/1562258515 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c07f060 con 0x5595f15f2e80 2024-04-22T13:27:27.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.668+0000 7f3005ffb640 1 -- 172.21.15.12:0/1562258515 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f300c09b5c0 con 0x5595f15f2e80 2024-04-22T13:27:27.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.668+0000 7f3027234740 1 -- 172.21.15.12:0/1562258515 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5595f1564c10 con 0x5595f15f2e80 2024-04-22T13:27:27.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.668+0000 7f3005ffb640 1 -- 172.21.15.12:0/1562258515 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f300c020030 con 0x5595f15f2e80 2024-04-22T13:27:27.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.668+0000 7f3027234740 10 client.16917 fetch_fsmap learned FSMap version 28 2024-04-22T13:27:27.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.668+0000 7f3027234740 1 -- 172.21.15.12:0/1562258515 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5595f1672c10 con 0x5595f15f2e80 2024-04-22T13:27:27.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.668+0000 7f3005ffb640 1 -- 172.21.15.12:0/1562258515 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5595f1564c10 con 0x5595f15f2e80 2024-04-22T13:27:27.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.668+0000 7f3005ffb640 1 -- 172.21.15.12:0/1562258515 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f300c096ce0 con 0x5595f15f2e80 2024-04-22T13:27:27.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.668+0000 7f3027234740 10 client.16917 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:27:27.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.668+0000 7f3027234740 10 client.16917 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:27:27.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.668+0000 7f3027234740 20 client.16917 populate_metadata read hostname 'smithi012' 2024-04-22T13:27:27.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.668+0000 7f3027234740 1 --2- 172.21.15.12:0/1562258515 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5595f1673170 0x5595f1675590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.668+0000 7f3027234740 4 client.16917 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:27:27.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.668+0000 7f3027234740 1 -- 172.21.15.12:0/1562258515 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5595f155ea80 con 0x5595f1673170 2024-04-22T13:27:27.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.668+0000 7f3022309640 1 --2- 172.21.15.12:0/1562258515 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5595f1673170 0x5595f1675590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.669+0000 7f3022309640 1 --2- 172.21.15.12:0/1562258515 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5595f1673170 0x5595f1675590 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=58ee96becc1cb208 server_cookie=c8f9e5a078c0e089 in_seq=0 out_seq=0 2024-04-22T13:27:27.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.669+0000 7f3005ffb640 10 client.16917 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:27:27.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.669+0000 7f3005ffb640 1 -- 172.21.15.12:0/1562258515 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5595f155ea80 con 0x5595f1673170 2024-04-22T13:27:27.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.669+0000 7f3005ffb640 10 client.16917 handle_command_reply: tid=0 2024-04-22T13:27:27.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.669+0000 7f3027234740 1 client.16917 shutdown 2024-04-22T13:27:27.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.669+0000 7f3027234740 1 -- 172.21.15.12:0/1562258515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595f15f2e80 msgr2=0x5595f16041f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.669+0000 7f3027234740 1 --2- 172.21.15.12:0/1562258515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595f15f2e80 0x5595f16041f0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f300c0cdf20 tx=0x7f300c09c870 comp rx=0 tx=0).stop 2024-04-22T13:27:27.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.669+0000 7f3027234740 1 -- 172.21.15.12:0/1562258515 shutdown_connections 2024-04-22T13:27:27.670 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.669+0000 7f3027234740 1 --2- 172.21.15.12:0/1562258515 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5595f1673170 0x5595f1675590 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.669+0000 7f3027234740 1 --2- 172.21.15.12:0/1562258515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5595f15fba00 0x5595f1600230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.669+0000 7f3027234740 1 --2- 172.21.15.12:0/1562258515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5595f15f2e80 0x5595f16041f0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.669+0000 7f3005ffb640 0 client.16917 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:27:27.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.669+0000 7f3027234740 1 --2- 172.21.15.12:0/1562258515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5595f15f24d0 0x5595f1603cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.669+0000 7f3027234740 1 -- 172.21.15.12:0/1562258515 >> 172.21.15.12:0/1562258515 conn(0x5595f15ecba0 msgr2=0x5595f15ee630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:27.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.669+0000 7f3027234740 1 -- 172.21.15.12:0/1562258515 shutdown_connections 2024-04-22T13:27:27.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.669+0000 7f3027234740 1 -- 172.21.15.12:0/1562258515 wait complete. 2024-04-22T13:27:27.671 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.669+0000 7f3027234740 20 client.16917 trim_cache size 0 max 16384 2024-04-22T13:27:27.671 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:27:27.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.670+0000 7f302456d640 1 -- 172.21.15.12:0/3850576593 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f30080a3d20 msgr2=0x7f30080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.670+0000 7f302456d640 1 --2- 172.21.15.12:0/3850576593 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f30080a3d20 0x7f30080a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3014009a60 tx=0x7f3014041000 comp rx=0 tx=0).stop 2024-04-22T13:27:27.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.670+0000 7f302456d640 1 -- 172.21.15.12:0/3850576593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f301c106160 msgr2=0x7f301c198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.670+0000 7f302456d640 1 --2- 172.21.15.12:0/3850576593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f301c106160 0x7f301c198c40 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f300c00d740 tx=0x7f300c00dc00 comp rx=0 tx=0).stop 2024-04-22T13:27:27.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.670+0000 7f302456d640 1 -- 172.21.15.12:0/3850576593 shutdown_connections 2024-04-22T13:27:27.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.670+0000 7f302456d640 1 --2- 172.21.15.12:0/3850576593 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f30080a3d20 0x7f30080a61e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.670+0000 7f302456d640 1 --2- 172.21.15.12:0/3850576593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f301c19cfc0 0x7f301c19d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.670+0000 7f302456d640 1 --2- 172.21.15.12:0/3850576593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f301c106160 0x7f301c198c40 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.670+0000 7f302456d640 1 --2- 172.21.15.12:0/3850576593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f301c101090 0x7f301c198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.670+0000 7f302456d640 1 -- 172.21.15.12:0/3850576593 >> 172.21.15.12:0/3850576593 conn(0x7f301c077f50 msgr2=0x7f301c0ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:27.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.670+0000 7f302456d640 1 -- 172.21.15.12:0/3850576593 shutdown_connections 2024-04-22T13:27:27.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.670+0000 7f302456d640 1 -- 172.21.15.12:0/3850576593 wait complete. 2024-04-22T13:27:27.675 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:27 smithi012 ceph-mon[28265]: pgmap v103: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:27:27.677 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:27:27.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:27 smithi134 ceph-mon[31338]: pgmap v103: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:27:27.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:27 smithi111 ceph-mon[31361]: pgmap v103: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:27:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.784+0000 7f4b2a73d640 1 -- 172.21.15.12:0/967758554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b241014a0 msgr2=0x7f4b24106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.784+0000 7f4b2a73d640 1 --2- 172.21.15.12:0/967758554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b241014a0 0x7f4b24106bd0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f4b14009900 tx=0x7f4b1402bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.784+0000 7f4b2a73d640 1 -- 172.21.15.12:0/967758554 shutdown_connections 2024-04-22T13:27:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.784+0000 7f4b2a73d640 1 --2- 172.21.15.12:0/967758554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b241072b0 0x7f4b2410eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.784+0000 7f4b2a73d640 1 --2- 172.21.15.12:0/967758554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b241014a0 0x7f4b24106bd0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.784+0000 7f4b2a73d640 1 --2- 172.21.15.12:0/967758554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b24100af0 0x7f4b24100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.784+0000 7f4b2a73d640 1 -- 172.21.15.12:0/967758554 >> 172.21.15.12:0/967758554 conn(0x7f4b240760e0 msgr2=0x7f4b240764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.784+0000 7f4b2a73d640 1 -- 172.21.15.12:0/967758554 shutdown_connections 2024-04-22T13:27:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.784+0000 7f4b2a73d640 1 -- 172.21.15.12:0/967758554 wait complete. 2024-04-22T13:27:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.785+0000 7f4b2a73d640 1 Processor -- start 2024-04-22T13:27:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.785+0000 7f4b2a73d640 1 -- start start 2024-04-22T13:27:27.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.785+0000 7f4b2a73d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b24100af0 0x7f4b24101a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.785+0000 7f4b2a73d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b241072b0 0x7f4b24101fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.785+0000 7f4b2a73d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b241024e0 0x7f4b241a6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.785+0000 7f4b2a73d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4b24112260 con 0x7f4b24100af0 2024-04-22T13:27:27.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.785+0000 7f4b2a73d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4b241120e0 con 0x7f4b241024e0 2024-04-22T13:27:27.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.785+0000 7f4b2a73d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4b241123e0 con 0x7f4b241072b0 2024-04-22T13:27:27.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.786+0000 7f4b23fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b24100af0 0x7f4b24101a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.786+0000 7f4b23fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b24100af0 0x7f4b24101a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53922/0 (socket says 172.21.15.12:53922) 2024-04-22T13:27:27.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.786+0000 7f4b23fff640 1 -- 172.21.15.12:0/1434711568 learned_addr learned my addr 172.21.15.12:0/1434711568 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:27.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.786+0000 7f4b28cda640 1 --2- 172.21.15.12:0/1434711568 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b241024e0 0x7f4b241a6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.786+0000 7f4b23fff640 1 -- 172.21.15.12:0/1434711568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b241072b0 msgr2=0x7f4b24101fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.786+0000 7f4b23fff640 1 --2- 172.21.15.12:0/1434711568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b241072b0 0x7f4b24101fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.786+0000 7f4b23fff640 1 -- 172.21.15.12:0/1434711568 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b241024e0 msgr2=0x7f4b241a6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.786+0000 7f4b23fff640 1 --2- 172.21.15.12:0/1434711568 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b241024e0 0x7f4b241a6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.786+0000 7f4b23fff640 1 -- 172.21.15.12:0/1434711568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b241a70a0 con 0x7f4b24100af0 2024-04-22T13:27:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.786+0000 7f4b23fff640 1 --2- 172.21.15.12:0/1434711568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b24100af0 0x7f4b24101a60 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c00d950 tx=0x7f4b0c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.786+0000 7f4b217fa640 1 -- 172.21.15.12:0/1434711568 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c017030 con 0x7f4b24100af0 2024-04-22T13:27:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.786+0000 7f4b2a73d640 1 -- 172.21.15.12:0/1434711568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b241a7390 con 0x7f4b24100af0 2024-04-22T13:27:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.786+0000 7f4b217fa640 1 -- 172.21.15.12:0/1434711568 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b0c004020 con 0x7f4b24100af0 2024-04-22T13:27:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.786+0000 7f4b217fa640 1 -- 172.21.15.12:0/1434711568 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c0047a0 con 0x7f4b24100af0 2024-04-22T13:27:27.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.786+0000 7f4b2a73d640 1 -- 172.21.15.12:0/1434711568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b241a7930 con 0x7f4b24100af0 2024-04-22T13:27:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.789+0000 7f4b2d406740 1 Processor -- start 2024-04-22T13:27:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.789+0000 7f4b217fa640 1 -- 172.21.15.12:0/1434711568 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4b0c009a30 con 0x7f4b24100af0 2024-04-22T13:27:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.789+0000 7f4b2d406740 1 -- start start 2024-04-22T13:27:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.789+0000 7f4b2d406740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331405d4d0 0x56331405d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.789+0000 7f4b2d406740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56331405de80 0x563314066430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.789+0000 7f4b2d406740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563314066a00 0x563314068e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.789+0000 7f4b2d406740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563313fc21f0 con 0x563314066a00 2024-04-22T13:27:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.789+0000 7f4b2d406740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x563313f5f4b0 con 0x56331405de80 2024-04-22T13:27:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.789+0000 7f4b2d406740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563313fbe1b0 con 0x56331405d4d0 2024-04-22T13:27:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.789+0000 7f4b237fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563314066a00 0x563314068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.789+0000 7f4b237fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563314066a00 0x563314068e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53928/0 (socket says 172.21.15.12:53928) 2024-04-22T13:27:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.789+0000 7f4b237fe640 1 -- 172.21.15.12:0/119312032 learned_addr learned my addr 172.21.15.12:0/119312032 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.789+0000 7f4b28cda640 1 --2- 172.21.15.12:0/119312032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331405d4d0 0x56331405d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.789+0000 7f4b23fff640 1 --2- 172.21.15.12:0/119312032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56331405de80 0x563314066430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.789+0000 7f4b237fe640 1 -- 172.21.15.12:0/119312032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331405d4d0 msgr2=0x56331405d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.790+0000 7f4b237fe640 1 --2- 172.21.15.12:0/119312032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331405d4d0 0x56331405d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.790+0000 7f4b237fe640 1 -- 172.21.15.12:0/119312032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56331405de80 msgr2=0x563314066430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.790+0000 7f4b237fe640 1 --2- 172.21.15.12:0/119312032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56331405de80 0x563314066430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.790+0000 7f4b217fa640 1 --2- 172.21.15.12:0/1434711568 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4b100a3d20 0x7f4b100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.790+0000 7f4b237fe640 1 -- 172.21.15.12:0/119312032 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563313f511b0 con 0x563314066a00 2024-04-22T13:27:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.790+0000 7f4b23fff640 1 --2- 172.21.15.12:0/119312032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56331405de80 0x563314066430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.790+0000 7f4b28cda640 1 --2- 172.21.15.12:0/1434711568 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4b100a3d20 0x7f4b100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.790+0000 7f4b217fa640 1 -- 172.21.15.12:0/1434711568 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f4b0c0d02a0 con 0x7f4b24100af0 2024-04-22T13:27:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.790+0000 7f4b237fe640 1 --2- 172.21.15.12:0/119312032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563314066a00 0x563314068e50 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f4b14009230 tx=0x7f4b14005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f6add3e8437e94 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.790+0000 7f4b1a7fc640 1 -- 172.21.15.12:0/119312032 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b14041020 con 0x563314066a00 2024-04-22T13:27:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.790+0000 7f4b1a7fc640 1 -- 172.21.15.12:0/119312032 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b1402cc90 con 0x563314066a00 2024-04-22T13:27:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.790+0000 7f4b1a7fc640 1 -- 172.21.15.12:0/119312032 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b140046c0 con 0x563314066a00 2024-04-22T13:27:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.791+0000 7f4b2d406740 1 -- 172.21.15.12:0/119312032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563314066a00 msgr2=0x563314068e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.791+0000 7f4b28cda640 1 --2- 172.21.15.12:0/1434711568 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4b100a3d20 0x7f4b100a61e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c005fd0 tx=0x7f4b1c009760 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.791+0000 7f4b2d406740 1 --2- 172.21.15.12:0/119312032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563314066a00 0x563314068e50 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f4b14009230 tx=0x7f4b14005d50 comp rx=0 tx=0).stop 2024-04-22T13:27:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.791+0000 7f4b2d406740 1 -- 172.21.15.12:0/119312032 shutdown_connections 2024-04-22T13:27:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.791+0000 7f4b2d406740 1 --2- 172.21.15.12:0/119312032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563314066a00 0x563314068e50 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.791+0000 7f4b2d406740 1 --2- 172.21.15.12:0/119312032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56331405de80 0x563314066430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.791+0000 7f4b2d406740 1 --2- 172.21.15.12:0/119312032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331405d4d0 0x56331405d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.791+0000 7f4b2d406740 1 -- 172.21.15.12:0/119312032 >> 172.21.15.12:0/119312032 conn(0x563314057ba0 msgr2=0x563314057f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.791+0000 7f4b2d406740 1 -- 172.21.15.12:0/119312032 shutdown_connections 2024-04-22T13:27:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.791+0000 7f4b2d406740 1 -- 172.21.15.12:0/119312032 wait complete. 2024-04-22T13:27:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.791+0000 7f4b2d406740 1 -- start start 2024-04-22T13:27:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.791+0000 7f4b2d406740 1 Processor -- start 2024-04-22T13:27:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.792+0000 7f4b2d406740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56331405d4d0 0x563314066320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.792+0000 7f4b2d406740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331405de80 0x5633140636b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.792+0000 7f4b2d406740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563314066a00 0x563314063c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.792+0000 7f4b2d406740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563314069910 con 0x563314066a00 2024-04-22T13:27:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.792+0000 7f4b2d406740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x563313f7b500 con 0x56331405d4d0 2024-04-22T13:27:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.792+0000 7f4b2d406740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563314069a90 con 0x56331405de80 2024-04-22T13:27:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.792+0000 7f4b2d406740 10 client.0 _ll_register_callbacks cb 0x563313f49b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.792+0000 7f4b28cda640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563314066a00 0x563314063c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.792+0000 7f4b28cda640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563314066a00 0x563314063c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53938/0 (socket says 172.21.15.12:53938) 2024-04-22T13:27:27.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.792+0000 7f4b28cda640 1 -- 172.21.15.12:0/1723633963 learned_addr learned my addr 172.21.15.12:0/1723633963 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:27.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.792+0000 7f4b23fff640 1 --2- 172.21.15.12:0/1723633963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56331405d4d0 0x563314066320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.792+0000 7f4b237fe640 1 --2- 172.21.15.12:0/1723633963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331405de80 0x5633140636b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.792+0000 7f4b28cda640 1 -- 172.21.15.12:0/1723633963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331405de80 msgr2=0x5633140636b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.792+0000 7f4b28cda640 1 --2- 172.21.15.12:0/1723633963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331405de80 0x5633140636b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.792+0000 7f4b28cda640 1 -- 172.21.15.12:0/1723633963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56331405d4d0 msgr2=0x563314066320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.792+0000 7f4b28cda640 1 --2- 172.21.15.12:0/1723633963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56331405d4d0 0x563314066320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.792+0000 7f4b28cda640 1 -- 172.21.15.12:0/1723633963 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563313cd43f0 con 0x563314066a00 2024-04-22T13:27:27.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.792+0000 7f4b237fe640 1 --2- 172.21.15.12:0/1723633963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331405de80 0x5633140636b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:27.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.792+0000 7f4b23fff640 1 --2- 172.21.15.12:0/1723633963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56331405d4d0 0x563314066320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:27.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.792+0000 7f4b28cda640 1 --2- 172.21.15.12:0/1723633963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563314066a00 0x563314063c50 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009730 tx=0x7f4b1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a58d872548bce4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:27.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.792+0000 7f4b037fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:27:27.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.792+0000 7f4b037fe640 1 -- 172.21.15.12:0/1723633963 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c02a060 con 0x563314066a00 2024-04-22T13:27:27.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.792+0000 7f4b037fe640 1 -- 172.21.15.12:0/1723633963 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f4b1c010e40 con 0x563314066a00 2024-04-22T13:27:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.792+0000 7f4b2d406740 1 -- 172.21.15.12:0/1723633963 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b1402cc90 con 0x563314066a00 2024-04-22T13:27:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.793+0000 7f4b037fe640 1 -- 172.21.15.12:0/1723633963 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4b1c01db90 con 0x563314066a00 2024-04-22T13:27:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.793+0000 7f4b2d406740 10 client.16929 fetch_fsmap learned FSMap version 28 2024-04-22T13:27:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.793+0000 7f4b2d406740 1 -- 172.21.15.12:0/1723633963 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563313d9e7d0 con 0x563314066a00 2024-04-22T13:27:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.793+0000 7f4b037fe640 1 -- 172.21.15.12:0/1723633963 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b1402cc90 con 0x563314066a00 2024-04-22T13:27:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.793+0000 7f4b037fe640 1 -- 172.21.15.12:0/1723633963 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4b1c019040 con 0x563314066a00 2024-04-22T13:27:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.793+0000 7f4b2d406740 10 client.16929 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:27:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.793+0000 7f4b2d406740 10 client.16929 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:27:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.793+0000 7f4b2d406740 20 client.16929 populate_metadata read hostname 'smithi012' 2024-04-22T13:27:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.793+0000 7f4b2d406740 1 --2- 172.21.15.12:0/1723633963 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x563314076c60 0x5633140790e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.793+0000 7f4b2d406740 4 client.16929 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.793+0000 7f4b2d406740 1 -- 172.21.15.12:0/1723633963 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563313f61930 con 0x563314076c60 2024-04-22T13:27:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.793+0000 7f4b23fff640 1 --2- 172.21.15.12:0/1723633963 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x563314076c60 0x5633140790e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.794+0000 7f4b23fff640 1 --2- 172.21.15.12:0/1723633963 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x563314076c60 0x5633140790e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=341bfc8c62539edd server_cookie=53af7f98ff0789aa in_seq=0 out_seq=0 2024-04-22T13:27:27.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.794+0000 7f4b037fe640 10 client.16929 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:27:27.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.794+0000 7f4b037fe640 1 -- 172.21.15.12:0/1723633963 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563313f61930 con 0x563314076c60 2024-04-22T13:27:27.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.794+0000 7f4b037fe640 10 client.16929 handle_command_reply: tid=0 2024-04-22T13:27:27.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.794+0000 7f4b2d406740 1 client.16929 shutdown 2024-04-22T13:27:27.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.795+0000 7f4b2d406740 1 -- 172.21.15.12:0/1723633963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563314066a00 msgr2=0x563314063c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.795+0000 7f4b2d406740 1 --2- 172.21.15.12:0/1723633963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563314066a00 0x563314063c50 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009730 tx=0x7f4b1c004a40 comp rx=0 tx=0).stop 2024-04-22T13:27:27.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.795+0000 7f4b2d406740 1 -- 172.21.15.12:0/1723633963 shutdown_connections 2024-04-22T13:27:27.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.795+0000 7f4b2d406740 1 --2- 172.21.15.12:0/1723633963 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x563314076c60 0x5633140790e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.795+0000 7f4b2d406740 1 --2- 172.21.15.12:0/1723633963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563314066a00 0x563314063c50 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.795+0000 7f4b2d406740 1 --2- 172.21.15.12:0/1723633963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331405de80 0x5633140636b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.795+0000 7f4b037fe640 0 client.16929 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:27:27.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.795+0000 7f4b2d406740 1 --2- 172.21.15.12:0/1723633963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56331405d4d0 0x563314066320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.795+0000 7f4b2d406740 1 -- 172.21.15.12:0/1723633963 >> 172.21.15.12:0/1723633963 conn(0x563314057ba0 msgr2=0x5633140598f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:27.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.795+0000 7f4b2d406740 1 -- 172.21.15.12:0/1723633963 shutdown_connections 2024-04-22T13:27:27.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.795+0000 7f4b2d406740 1 -- 172.21.15.12:0/1723633963 wait complete. 2024-04-22T13:27:27.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.795+0000 7f4b2d406740 20 client.16929 trim_cache size 0 max 16384 2024-04-22T13:27:27.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.808+0000 7f4b2d406740 1 Processor -- start 2024-04-22T13:27:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.808+0000 7f4b2d406740 1 -- start start 2024-04-22T13:27:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.808+0000 7f4b2d406740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56331405d4d0 0x5633140727d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.808+0000 7f4b2d406740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331405de80 0x563314065de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.808+0000 7f4b2d406740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563314066a00 0x563314066320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.808+0000 7f4b2d406740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4b241afd60 con 0x563314066a00 2024-04-22T13:27:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.808+0000 7f4b2d406740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4b241b18a0 con 0x56331405d4d0 2024-04-22T13:27:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.808+0000 7f4b2d406740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4b241b0430 con 0x56331405de80 2024-04-22T13:27:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.808+0000 7f4b28cda640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563314066a00 0x563314066320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.808+0000 7f4b28cda640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563314066a00 0x563314066320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53944/0 (socket says 172.21.15.12:53944) 2024-04-22T13:27:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.808+0000 7f4b28cda640 1 -- 172.21.15.12:0/816993991 learned_addr learned my addr 172.21.15.12:0/816993991 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.809+0000 7f4b28cda640 1 -- 172.21.15.12:0/816993991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331405de80 msgr2=0x563314065de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.809+0000 7f4b28cda640 1 --2- 172.21.15.12:0/816993991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331405de80 0x563314065de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.809+0000 7f4b23fff640 1 --2- 172.21.15.12:0/816993991 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56331405d4d0 0x5633140727d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.809+0000 7f4b28cda640 1 -- 172.21.15.12:0/816993991 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56331405d4d0 msgr2=0x5633140727d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.809+0000 7f4b28cda640 1 --2- 172.21.15.12:0/816993991 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56331405d4d0 0x5633140727d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.809+0000 7f4b28cda640 1 -- 172.21.15.12:0/816993991 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56331405dc20 con 0x563314066a00 2024-04-22T13:27:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.809+0000 7f4b28cda640 1 --2- 172.21.15.12:0/816993991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563314066a00 0x563314066320 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c0255d0 tx=0x7f4b1c010510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c86912c932ffa729 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.809+0000 7f4b19ffb640 1 -- 172.21.15.12:0/816993991 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c004cb0 con 0x563314066a00 2024-04-22T13:27:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.809+0000 7f4b19ffb640 1 -- 172.21.15.12:0/816993991 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b1c004e50 con 0x563314066a00 2024-04-22T13:27:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.809+0000 7f4b19ffb640 1 -- 172.21.15.12:0/816993991 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c025dd0 con 0x563314066a00 2024-04-22T13:27:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.810+0000 7f4b2d406740 1 -- 172.21.15.12:0/816993991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563314066a00 msgr2=0x563314066320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.810+0000 7f4b2d406740 1 --2- 172.21.15.12:0/816993991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563314066a00 0x563314066320 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c0255d0 tx=0x7f4b1c010510 comp rx=0 tx=0).stop 2024-04-22T13:27:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.810+0000 7f4b2d406740 1 -- 172.21.15.12:0/816993991 shutdown_connections 2024-04-22T13:27:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.810+0000 7f4b2d406740 1 --2- 172.21.15.12:0/816993991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563314066a00 0x563314066320 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.810+0000 7f4b2d406740 1 --2- 172.21.15.12:0/816993991 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331405de80 0x563314065de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.810+0000 7f4b2d406740 1 --2- 172.21.15.12:0/816993991 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56331405d4d0 0x5633140727d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.810+0000 7f4b2d406740 1 -- 172.21.15.12:0/816993991 >> 172.21.15.12:0/816993991 conn(0x563314057ba0 msgr2=0x563314056370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.810+0000 7f4b2d406740 1 -- 172.21.15.12:0/816993991 shutdown_connections 2024-04-22T13:27:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.810+0000 7f4b2d406740 1 -- 172.21.15.12:0/816993991 wait complete. 2024-04-22T13:27:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.810+0000 7f4b2d406740 1 -- start start 2024-04-22T13:27:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.810+0000 7f4b2d406740 1 Processor -- start 2024-04-22T13:27:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.810+0000 7f4b2d406740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56331405d4d0 0x56331406f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.810+0000 7f4b2d406740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331405de80 0x56331406a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.810+0000 7f4b2d406740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563314066a00 0x56331406acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.810+0000 7f4b2d406740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4b241b11d0 con 0x56331405d4d0 2024-04-22T13:27:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.810+0000 7f4b2d406740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4b241b0b00 con 0x563314066a00 2024-04-22T13:27:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.810+0000 7f4b2d406740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4b241add40 con 0x56331405de80 2024-04-22T13:27:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.810+0000 7f4b2d406740 10 client.0 _ll_register_callbacks cb 0x7f4b1c0105a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.810+0000 7f4b23fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56331405d4d0 0x56331406f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.811+0000 7f4b23fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56331405d4d0 0x56331406f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53946/0 (socket says 172.21.15.12:53946) 2024-04-22T13:27:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.811+0000 7f4b23fff640 1 -- 172.21.15.12:0/2148585696 learned_addr learned my addr 172.21.15.12:0/2148585696 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.811+0000 7f4b23fff640 1 -- 172.21.15.12:0/2148585696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331405de80 msgr2=0x56331406a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.811+0000 7f4b237fe640 1 --2- 172.21.15.12:0/2148585696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331405de80 0x56331406a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.811+0000 7f4b23fff640 1 --2- 172.21.15.12:0/2148585696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331405de80 0x56331406a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.811+0000 7f4b23fff640 1 -- 172.21.15.12:0/2148585696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563314066a00 msgr2=0x56331406acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.811+0000 7f4b28cda640 1 --2- 172.21.15.12:0/2148585696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563314066a00 0x56331406acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.811+0000 7f4b23fff640 1 --2- 172.21.15.12:0/2148585696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563314066a00 0x56331406acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.812 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.811+0000 7f4b23fff640 1 -- 172.21.15.12:0/2148585696 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633140dc220 con 0x56331405d4d0 2024-04-22T13:27:27.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.811+0000 7f4b28cda640 1 --2- 172.21.15.12:0/2148585696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563314066a00 0x56331406acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:27.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.811+0000 7f4b23fff640 1 --2- 172.21.15.12:0/2148585696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56331405d4d0 0x56331406f250 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c0cdaa0 tx=0x7f4b0c09be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba7b72a91367246 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:27.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.811+0000 7f4b037fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:27:27.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.811+0000 7f4b037fe640 1 -- 172.21.15.12:0/2148585696 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c0a0640 con 0x56331405d4d0 2024-04-22T13:27:27.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.811+0000 7f4b037fe640 1 -- 172.21.15.12:0/2148585696 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f4b0c09a590 con 0x56331405d4d0 2024-04-22T13:27:27.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.811+0000 7f4b2d406740 1 -- 172.21.15.12:0/2148585696 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b1c004e50 con 0x56331405d4d0 2024-04-22T13:27:27.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.811+0000 7f4b037fe640 1 -- 172.21.15.12:0/2148585696 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4b0c085020 con 0x56331405d4d0 2024-04-22T13:27:27.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.811+0000 7f4b2d406740 10 client.16935 fetch_fsmap learned FSMap version 28 2024-04-22T13:27:27.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.811+0000 7f4b2d406740 1 -- 172.21.15.12:0/2148585696 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5633140dd3c0 con 0x56331405d4d0 2024-04-22T13:27:27.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.811+0000 7f4b037fe640 1 -- 172.21.15.12:0/2148585696 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b1c004e50 con 0x56331405d4d0 2024-04-22T13:27:27.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.812+0000 7f4b037fe640 1 -- 172.21.15.12:0/2148585696 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4b0c0d3cc0 con 0x56331405d4d0 2024-04-22T13:27:27.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.812+0000 7f4b2d406740 10 client.16935 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:27:27.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.812+0000 7f4b2d406740 10 client.16935 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:27:27.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.812+0000 7f4b2d406740 20 client.16935 populate_metadata read hostname 'smithi012' 2024-04-22T13:27:27.813 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.812+0000 7f4b2d406740 1 --2- 172.21.15.12:0/2148585696 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5633140dda90 0x5633140dfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.812+0000 7f4b2d406740 4 client.16935 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:27:27.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.812+0000 7f4b2d406740 1 -- 172.21.15.12:0/2148585696 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563313fcfc10 con 0x5633140dda90 2024-04-22T13:27:27.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.812+0000 7f4b237fe640 1 --2- 172.21.15.12:0/2148585696 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5633140dda90 0x5633140dfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.812+0000 7f4b237fe640 1 --2- 172.21.15.12:0/2148585696 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5633140dda90 0x5633140dfeb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bbe01fd9ff1890eb server_cookie=f08803faa15e1ec5 in_seq=0 out_seq=0 2024-04-22T13:27:27.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.812+0000 7f4b037fe640 10 client.16935 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:27:27.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.812+0000 7f4b037fe640 1 -- 172.21.15.12:0/2148585696 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563313fcfc10 con 0x5633140dda90 2024-04-22T13:27:27.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.812+0000 7f4b037fe640 10 client.16935 handle_command_reply: tid=0 2024-04-22T13:27:27.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.812+0000 7f4b2d406740 1 client.16935 shutdown 2024-04-22T13:27:27.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.813+0000 7f4b2d406740 1 -- 172.21.15.12:0/2148585696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56331405d4d0 msgr2=0x56331406f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.813+0000 7f4b2d406740 1 --2- 172.21.15.12:0/2148585696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56331405d4d0 0x56331406f250 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c0cdaa0 tx=0x7f4b0c09be90 comp rx=0 tx=0).stop 2024-04-22T13:27:27.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.813+0000 7f4b2d406740 1 -- 172.21.15.12:0/2148585696 shutdown_connections 2024-04-22T13:27:27.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.813+0000 7f4b2d406740 1 --2- 172.21.15.12:0/2148585696 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5633140dda90 0x5633140dfeb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.813+0000 7f4b2d406740 1 --2- 172.21.15.12:0/2148585696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563314066a00 0x56331406acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.813+0000 7f4b037fe640 0 client.16935 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:27:27.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.813+0000 7f4b2d406740 1 --2- 172.21.15.12:0/2148585696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56331405de80 0x56331406a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.813+0000 7f4b2d406740 1 --2- 172.21.15.12:0/2148585696 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56331405d4d0 0x56331406f250 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.813+0000 7f4b2d406740 1 -- 172.21.15.12:0/2148585696 >> 172.21.15.12:0/2148585696 conn(0x563314057ba0 msgr2=0x56331405eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:27.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.813+0000 7f4b2d406740 1 -- 172.21.15.12:0/2148585696 shutdown_connections 2024-04-22T13:27:27.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.813+0000 7f4b2d406740 1 -- 172.21.15.12:0/2148585696 wait complete. 2024-04-22T13:27:27.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.813+0000 7f4b2d406740 20 client.16935 trim_cache size 0 max 16384 2024-04-22T13:27:27.815 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:27:27.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.813+0000 7f4b2a73d640 1 -- 172.21.15.12:0/1434711568 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4b100a3d20 msgr2=0x7f4b100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.813+0000 7f4b2a73d640 1 --2- 172.21.15.12:0/1434711568 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4b100a3d20 0x7f4b100a61e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c005fd0 tx=0x7f4b1c009760 comp rx=0 tx=0).stop 2024-04-22T13:27:27.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.813+0000 7f4b2a73d640 1 -- 172.21.15.12:0/1434711568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b24100af0 msgr2=0x7f4b24101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.813+0000 7f4b2a73d640 1 --2- 172.21.15.12:0/1434711568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b24100af0 0x7f4b24101a60 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c00d950 tx=0x7f4b0c00de10 comp rx=0 tx=0).stop 2024-04-22T13:27:27.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.813+0000 7f4b2a73d640 1 -- 172.21.15.12:0/1434711568 shutdown_connections 2024-04-22T13:27:27.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.813+0000 7f4b2a73d640 1 --2- 172.21.15.12:0/1434711568 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4b100a3d20 0x7f4b100a61e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.813+0000 7f4b2a73d640 1 --2- 172.21.15.12:0/1434711568 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b241024e0 0x7f4b241a6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.813+0000 7f4b2a73d640 1 --2- 172.21.15.12:0/1434711568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4b241072b0 0x7f4b24101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.813+0000 7f4b2a73d640 1 --2- 172.21.15.12:0/1434711568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4b24100af0 0x7f4b24101a60 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.813+0000 7f4b2a73d640 1 -- 172.21.15.12:0/1434711568 >> 172.21.15.12:0/1434711568 conn(0x7f4b240760e0 msgr2=0x7f4b240fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:27.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.814+0000 7f4b2a73d640 1 -- 172.21.15.12:0/1434711568 shutdown_connections 2024-04-22T13:27:27.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.814+0000 7f4b2a73d640 1 -- 172.21.15.12:0/1434711568 wait complete. 2024-04-22T13:27:27.820 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:27:27.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.928+0000 7f6f15f9f640 1 -- 172.21.15.12:0/3094290591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f1006b1f0 msgr2=0x7f6f10106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.928+0000 7f6f15f9f640 1 --2- 172.21.15.12:0/3094290591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f1006b1f0 0x7f6f10106b90 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009900 tx=0x7f6f0402bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:27.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.928+0000 7f6f15f9f640 1 -- 172.21.15.12:0/3094290591 shutdown_connections 2024-04-22T13:27:27.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.928+0000 7f6f15f9f640 1 --2- 172.21.15.12:0/3094290591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f10107270 0x7f6f1010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.928+0000 7f6f15f9f640 1 --2- 172.21.15.12:0/3094290591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f1006b1f0 0x7f6f10106b90 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.928+0000 7f6f15f9f640 1 --2- 172.21.15.12:0/3094290591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f1006a840 0x7f6f1006ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.928+0000 7f6f15f9f640 1 -- 172.21.15.12:0/3094290591 >> 172.21.15.12:0/3094290591 conn(0x7f6f10077eb0 msgr2=0x7f6f100782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:27.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.928+0000 7f6f15f9f640 1 -- 172.21.15.12:0/3094290591 shutdown_connections 2024-04-22T13:27:27.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.928+0000 7f6f15f9f640 1 -- 172.21.15.12:0/3094290591 wait complete. 2024-04-22T13:27:27.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.929+0000 7f6f15f9f640 1 Processor -- start 2024-04-22T13:27:27.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.929+0000 7f6f15f9f640 1 -- start start 2024-04-22T13:27:27.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.929+0000 7f6f15f9f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f1006a840 0x7f6f10198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.929+0000 7f6f15f9f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f10107270 0x7f6f10198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.929+0000 7f6f15f9f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f1019cfe0 0x7f6f1019d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.929+0000 7f6f15f9f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6f10112200 con 0x7f6f10107270 2024-04-22T13:27:27.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.929+0000 7f6f15f9f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6f10112080 con 0x7f6f1006a840 2024-04-22T13:27:27.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.929+0000 7f6f15f9f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6f10112380 con 0x7f6f1019cfe0 2024-04-22T13:27:27.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.929+0000 7f6f0effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f10107270 0x7f6f10198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.929+0000 7f6f0effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f10107270 0x7f6f10198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53956/0 (socket says 172.21.15.12:53956) 2024-04-22T13:27:27.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.929+0000 7f6f0effd640 1 -- 172.21.15.12:0/2551570819 learned_addr learned my addr 172.21.15.12:0/2551570819 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:27.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.930+0000 7f6f0f7fe640 1 --2- 172.21.15.12:0/2551570819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f1006a840 0x7f6f10198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.930+0000 7f6f0ffff640 1 --2- 172.21.15.12:0/2551570819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f1019cfe0 0x7f6f1019d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.930+0000 7f6f0effd640 1 -- 172.21.15.12:0/2551570819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f1019cfe0 msgr2=0x7f6f1019d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.930+0000 7f6f0effd640 1 --2- 172.21.15.12:0/2551570819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f1019cfe0 0x7f6f1019d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.930+0000 7f6f0effd640 1 -- 172.21.15.12:0/2551570819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f1006a840 msgr2=0x7f6f10198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.930+0000 7f6f0effd640 1 --2- 172.21.15.12:0/2551570819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f1006a840 0x7f6f10198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.930+0000 7f6f0effd640 1 -- 172.21.15.12:0/2551570819 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1019dad0 con 0x7f6f10107270 2024-04-22T13:27:27.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.930+0000 7f6f0effd640 1 --2- 172.21.15.12:0/2551570819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f10107270 0x7f6f10198c60 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f6f04000c00 tx=0x7f6f040045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:27.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.930+0000 7f6f0cff9640 1 -- 172.21.15.12:0/2551570819 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f04043020 con 0x7f6f10107270 2024-04-22T13:27:27.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.930+0000 7f6f0cff9640 1 -- 172.21.15.12:0/2551570819 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f0402cb70 con 0x7f6f10107270 2024-04-22T13:27:27.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.930+0000 7f6f15f9f640 1 -- 172.21.15.12:0/2551570819 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f1019dd60 con 0x7f6f10107270 2024-04-22T13:27:27.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.930+0000 7f6f0cff9640 1 -- 172.21.15.12:0/2551570819 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f04003b50 con 0x7f6f10107270 2024-04-22T13:27:27.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.930+0000 7f6f15f9f640 1 -- 172.21.15.12:0/2551570819 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f101a55b0 con 0x7f6f10107270 2024-04-22T13:27:27.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.933+0000 7f6f0cff9640 1 -- 172.21.15.12:0/2551570819 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6f04046020 con 0x7f6f10107270 2024-04-22T13:27:27.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.933+0000 7f6f18b8a740 1 Processor -- start 2024-04-22T13:27:27.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.933+0000 7f6f18b8a740 1 -- start start 2024-04-22T13:27:27.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.934+0000 7f6f18b8a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560017f93d20 0x560017f94100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.934+0000 7f6f18b8a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560017f946d0 0x560017f80650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.934+0000 7f6f18b8a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017f80c70 0x560017f810d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.934+0000 7f6f18b8a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560017eeb1f0 con 0x560017f93d20 2024-04-22T13:27:27.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.934+0000 7f6f18b8a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560017e884b0 con 0x560017f946d0 2024-04-22T13:27:27.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.934+0000 7f6f18b8a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560017ee71b0 con 0x560017f80c70 2024-04-22T13:27:27.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.934+0000 7f6f0ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560017f93d20 0x560017f94100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.934+0000 7f6f0ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560017f93d20 0x560017f94100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53964/0 (socket says 172.21.15.12:53964) 2024-04-22T13:27:27.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.934+0000 7f6f0ffff640 1 -- 172.21.15.12:0/2199543437 learned_addr learned my addr 172.21.15.12:0/2199543437 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:27.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.934+0000 7f6f0effd640 1 --2- 172.21.15.12:0/2199543437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017f80c70 0x560017f810d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.934+0000 7f6f0cff9640 1 --2- 172.21.15.12:0/2551570819 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6ef80a3da0 0x7f6ef80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.934+0000 7f6f0f7fe640 1 --2- 172.21.15.12:0/2199543437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560017f946d0 0x560017f80650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.934+0000 7f6f0ffff640 1 -- 172.21.15.12:0/2199543437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017f80c70 msgr2=0x560017f810d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.934+0000 7f6f0ffff640 1 --2- 172.21.15.12:0/2199543437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017f80c70 0x560017f810d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.934+0000 7f6f0ffff640 1 -- 172.21.15.12:0/2199543437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560017f946d0 msgr2=0x560017f80650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.934+0000 7f6f0ffff640 1 --2- 172.21.15.12:0/2199543437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560017f946d0 0x560017f80650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.934+0000 7f6f0ffff640 1 -- 172.21.15.12:0/2199543437 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560017e7a1b0 con 0x560017f93d20 2024-04-22T13:27:27.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.934+0000 7f6f0f7fe640 1 --2- 172.21.15.12:0/2199543437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560017f946d0 0x560017f80650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:27.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.934+0000 7f6f0cff9640 1 -- 172.21.15.12:0/2551570819 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f6f040f2710 con 0x7f6f10107270 2024-04-22T13:27:27.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.934+0000 7f6f0ffff640 1 --2- 172.21.15.12:0/2551570819 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6ef80a3da0 0x7f6ef80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.934+0000 7f6f0ffff640 1 --2- 172.21.15.12:0/2199543437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560017f93d20 0x560017f94100 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f6f08010440 tx=0x7f6f08010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c858d007dc2356a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:27.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.935+0000 7f6ef5ffb640 1 -- 172.21.15.12:0/2199543437 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0801d400 con 0x560017f93d20 2024-04-22T13:27:27.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.935+0000 7f6ef5ffb640 1 -- 172.21.15.12:0/2199543437 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f080134c0 con 0x560017f93d20 2024-04-22T13:27:27.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.935+0000 7f6ef5ffb640 1 -- 172.21.15.12:0/2199543437 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f080117a0 con 0x560017f93d20 2024-04-22T13:27:27.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.935+0000 7f6f18b8a740 1 -- 172.21.15.12:0/2199543437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560017f93d20 msgr2=0x560017f94100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.935+0000 7f6f0ffff640 1 --2- 172.21.15.12:0/2551570819 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6ef80a3da0 0x7f6ef80a6260 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6f08021000 tx=0x7f6f0801e000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:27.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.935+0000 7f6f18b8a740 1 --2- 172.21.15.12:0/2199543437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560017f93d20 0x560017f94100 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f6f08010440 tx=0x7f6f08010900 comp rx=0 tx=0).stop 2024-04-22T13:27:27.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.935+0000 7f6f18b8a740 1 -- 172.21.15.12:0/2199543437 shutdown_connections 2024-04-22T13:27:27.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.935+0000 7f6f18b8a740 1 --2- 172.21.15.12:0/2199543437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017f80c70 0x560017f810d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.935+0000 7f6f18b8a740 1 --2- 172.21.15.12:0/2199543437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560017f946d0 0x560017f80650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.935+0000 7f6f18b8a740 1 --2- 172.21.15.12:0/2199543437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560017f93d20 0x560017f94100 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.935+0000 7f6f18b8a740 1 -- 172.21.15.12:0/2199543437 >> 172.21.15.12:0/2199543437 conn(0x560017f83110 msgr2=0x560017f834f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:27.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.935+0000 7f6f18b8a740 1 -- 172.21.15.12:0/2199543437 shutdown_connections 2024-04-22T13:27:27.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.935+0000 7f6f18b8a740 1 -- 172.21.15.12:0/2199543437 wait complete. 2024-04-22T13:27:27.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.936+0000 7f6f18b8a740 1 -- start start 2024-04-22T13:27:27.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.936+0000 7f6f18b8a740 1 Processor -- start 2024-04-22T13:27:27.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.936+0000 7f6f18b8a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017f80c70 0x560017f9a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.936+0000 7f6f18b8a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560017f93d20 0x560017f97990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.936+0000 7f6f18b8a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560017f946d0 0x560017f97f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.936+0000 7f6f18b8a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560017f9b160 con 0x560017f946d0 2024-04-22T13:27:27.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.936+0000 7f6f18b8a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560017ea4500 con 0x560017f93d20 2024-04-22T13:27:27.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.936+0000 7f6f18b8a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560017f9b2e0 con 0x560017f80c70 2024-04-22T13:27:27.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.936+0000 7f6f18b8a740 10 client.0 _ll_register_callbacks cb 0x560017e72b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:27.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.936+0000 7f6f0ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560017f946d0 0x560017f97f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.936+0000 7f6f0ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560017f946d0 0x560017f97f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53970/0 (socket says 172.21.15.12:53970) 2024-04-22T13:27:27.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.936+0000 7f6f0ffff640 1 -- 172.21.15.12:0/2514457039 learned_addr learned my addr 172.21.15.12:0/2514457039 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:27.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.936+0000 7f6f0f7fe640 1 --2- 172.21.15.12:0/2514457039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017f80c70 0x560017f9a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.937+0000 7f6f0effd640 1 --2- 172.21.15.12:0/2514457039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560017f93d20 0x560017f97990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.937+0000 7f6f0ffff640 1 -- 172.21.15.12:0/2514457039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017f80c70 msgr2=0x560017f9a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.937+0000 7f6f0ffff640 1 --2- 172.21.15.12:0/2514457039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017f80c70 0x560017f9a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.937+0000 7f6f0ffff640 1 -- 172.21.15.12:0/2514457039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560017f93d20 msgr2=0x560017f97990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.937+0000 7f6f0ffff640 1 --2- 172.21.15.12:0/2514457039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560017f93d20 0x560017f97990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.937+0000 7f6f0ffff640 1 -- 172.21.15.12:0/2514457039 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560017bfd3f0 con 0x560017f946d0 2024-04-22T13:27:27.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.937+0000 7f6f0ffff640 1 --2- 172.21.15.12:0/2514457039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560017f946d0 0x560017f97f10 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f6f08000f80 tx=0x7f6f08011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b656e0a32a5130 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:27.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.937+0000 7f6eeeffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:27:27.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.937+0000 7f6eeeffd640 1 -- 172.21.15.12:0/2514457039 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08003b80 con 0x560017f946d0 2024-04-22T13:27:27.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.937+0000 7f6eeeffd640 1 -- 172.21.15.12:0/2514457039 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f6f08003d20 con 0x560017f946d0 2024-04-22T13:27:27.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.937+0000 7f6f18b8a740 1 -- 172.21.15.12:0/2514457039 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f080134c0 con 0x560017f946d0 2024-04-22T13:27:27.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.937+0000 7f6eeeffd640 1 -- 172.21.15.12:0/2514457039 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6f08027480 con 0x560017f946d0 2024-04-22T13:27:27.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.937+0000 7f6f18b8a740 10 client.16947 fetch_fsmap learned FSMap version 28 2024-04-22T13:27:27.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.937+0000 7f6f18b8a740 1 -- 172.21.15.12:0/2514457039 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560017cc77d0 con 0x560017f946d0 2024-04-22T13:27:27.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.937+0000 7f6eeeffd640 1 -- 172.21.15.12:0/2514457039 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f080134c0 con 0x560017f946d0 2024-04-22T13:27:27.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.937+0000 7f6eeeffd640 1 -- 172.21.15.12:0/2514457039 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6f0800f040 con 0x560017f946d0 2024-04-22T13:27:27.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.938+0000 7f6f18b8a740 10 client.16947 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:27:27.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.938+0000 7f6f18b8a740 10 client.16947 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:27:27.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.938+0000 7f6f18b8a740 20 client.16947 populate_metadata read hostname 'smithi012' 2024-04-22T13:27:27.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.938+0000 7f6f18b8a740 1 --2- 172.21.15.12:0/2514457039 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x560017fa8450 0x560017faa840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.938+0000 7f6f18b8a740 4 client.16947 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:27.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.938+0000 7f6f18b8a740 1 -- 172.21.15.12:0/2514457039 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560017e8a930 con 0x560017fa8450 2024-04-22T13:27:27.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.938+0000 7f6f0f7fe640 1 --2- 172.21.15.12:0/2514457039 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x560017fa8450 0x560017faa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.938+0000 7f6f0f7fe640 1 --2- 172.21.15.12:0/2514457039 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x560017fa8450 0x560017faa840 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b692383b190d3a84 server_cookie=3489850b8ed61103 in_seq=0 out_seq=0 2024-04-22T13:27:27.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.938+0000 7f6eeeffd640 10 client.16947 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:27:27.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.939+0000 7f6eeeffd640 1 -- 172.21.15.12:0/2514457039 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560017e8a930 con 0x560017fa8450 2024-04-22T13:27:27.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.939+0000 7f6eeeffd640 10 client.16947 handle_command_reply: tid=0 2024-04-22T13:27:27.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.939+0000 7f6f18b8a740 1 client.16947 shutdown 2024-04-22T13:27:27.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.939+0000 7f6f18b8a740 1 -- 172.21.15.12:0/2514457039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560017f946d0 msgr2=0x560017f97f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.939+0000 7f6f18b8a740 1 --2- 172.21.15.12:0/2514457039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560017f946d0 0x560017f97f10 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f6f08000f80 tx=0x7f6f08011d20 comp rx=0 tx=0).stop 2024-04-22T13:27:27.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.939+0000 7f6f18b8a740 1 -- 172.21.15.12:0/2514457039 shutdown_connections 2024-04-22T13:27:27.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.939+0000 7f6f18b8a740 1 --2- 172.21.15.12:0/2514457039 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x560017fa8450 0x560017faa840 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.939+0000 7f6f18b8a740 1 --2- 172.21.15.12:0/2514457039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560017f946d0 0x560017f97f10 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.939+0000 7f6f18b8a740 1 --2- 172.21.15.12:0/2514457039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560017f93d20 0x560017f97990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.939+0000 7f6f18b8a740 1 --2- 172.21.15.12:0/2514457039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017f80c70 0x560017f9a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.939+0000 7f6eeeffd640 0 client.16947 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:27:27.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.939+0000 7f6f18b8a740 1 -- 172.21.15.12:0/2514457039 >> 172.21.15.12:0/2514457039 conn(0x560017f83110 msgr2=0x560017f82870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:27.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.939+0000 7f6f18b8a740 1 -- 172.21.15.12:0/2514457039 shutdown_connections 2024-04-22T13:27:27.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.939+0000 7f6f18b8a740 1 -- 172.21.15.12:0/2514457039 wait complete. 2024-04-22T13:27:27.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.940+0000 7f6f18b8a740 20 client.16947 trim_cache size 0 max 16384 2024-04-22T13:27:27.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.953+0000 7f6f18b8a740 1 Processor -- start 2024-04-22T13:27:27.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.953+0000 7f6f18b8a740 1 -- start start 2024-04-22T13:27:27.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.953+0000 7f6f18b8a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017f80c70 0x560017fa4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.953+0000 7f6f18b8a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560017f93d20 0x560017f96af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.953+0000 7f6f18b8a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560017f946d0 0x560017f82680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.953+0000 7f6f18b8a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6f101ab800 con 0x560017f93d20 2024-04-22T13:27:27.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.953+0000 7f6f18b8a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6f101ad340 con 0x560017f946d0 2024-04-22T13:27:27.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.953+0000 7f6f18b8a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6f101abed0 con 0x560017f80c70 2024-04-22T13:27:27.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.953+0000 7f6f0effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560017f93d20 0x560017f96af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.953+0000 7f6f0effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560017f93d20 0x560017f96af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53982/0 (socket says 172.21.15.12:53982) 2024-04-22T13:27:27.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.953+0000 7f6f0effd640 1 -- 172.21.15.12:0/3886788589 learned_addr learned my addr 172.21.15.12:0/3886788589 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:27.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.953+0000 7f6f0f7fe640 1 --2- 172.21.15.12:0/3886788589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017f80c70 0x560017fa4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.953+0000 7f6f0effd640 1 -- 172.21.15.12:0/3886788589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017f80c70 msgr2=0x560017fa4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.953+0000 7f6f0effd640 1 --2- 172.21.15.12:0/3886788589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017f80c70 0x560017fa4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.953+0000 7f6f0effd640 1 -- 172.21.15.12:0/3886788589 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560017f946d0 msgr2=0x560017f82680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.953+0000 7f6f0ffff640 1 --2- 172.21.15.12:0/3886788589 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560017f946d0 0x560017f82680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.953+0000 7f6f0effd640 1 --2- 172.21.15.12:0/3886788589 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560017f946d0 0x560017f82680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.953+0000 7f6f0effd640 1 -- 172.21.15.12:0/3886788589 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560017faab70 con 0x560017f93d20 2024-04-22T13:27:27.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.953+0000 7f6f0f7fe640 1 --2- 172.21.15.12:0/3886788589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017f80c70 0x560017fa4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:27.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.953+0000 7f6f0effd640 1 --2- 172.21.15.12:0/3886788589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560017f93d20 0x560017f96af0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f6f04005ab0 tx=0x7f6f040a9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7264728ff7aa8b35 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:27.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.954+0000 7f6ef57fa640 1 -- 172.21.15.12:0/3886788589 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f040f8020 con 0x560017f93d20 2024-04-22T13:27:27.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.954+0000 7f6ef57fa640 1 -- 172.21.15.12:0/3886788589 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f040beeb0 con 0x560017f93d20 2024-04-22T13:27:27.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.954+0000 7f6ef57fa640 1 -- 172.21.15.12:0/3886788589 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f040c7420 con 0x560017f93d20 2024-04-22T13:27:27.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.954+0000 7f6f18b8a740 1 -- 172.21.15.12:0/3886788589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560017f93d20 msgr2=0x560017f96af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.954+0000 7f6f18b8a740 1 --2- 172.21.15.12:0/3886788589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560017f93d20 0x560017f96af0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f6f04005ab0 tx=0x7f6f040a9040 comp rx=0 tx=0).stop 2024-04-22T13:27:27.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.954+0000 7f6f18b8a740 1 -- 172.21.15.12:0/3886788589 shutdown_connections 2024-04-22T13:27:27.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.954+0000 7f6f18b8a740 1 --2- 172.21.15.12:0/3886788589 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560017f946d0 0x560017f82680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.954+0000 7f6f18b8a740 1 --2- 172.21.15.12:0/3886788589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560017f93d20 0x560017f96af0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.954+0000 7f6f18b8a740 1 --2- 172.21.15.12:0/3886788589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017f80c70 0x560017fa4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.954+0000 7f6f18b8a740 1 -- 172.21.15.12:0/3886788589 >> 172.21.15.12:0/3886788589 conn(0x560017f83110 msgr2=0x560017f81f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:27.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.954+0000 7f6f18b8a740 1 -- 172.21.15.12:0/3886788589 shutdown_connections 2024-04-22T13:27:27.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.954+0000 7f6f18b8a740 1 -- 172.21.15.12:0/3886788589 wait complete. 2024-04-22T13:27:27.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.954+0000 7f6f18b8a740 1 -- start start 2024-04-22T13:27:27.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.955+0000 7f6f18b8a740 1 Processor -- start 2024-04-22T13:27:27.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.955+0000 7f6f18b8a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017f80c70 0x560017fa06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.955+0000 7f6f18b8a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560017f93d20 0x560017f9bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.955+0000 7f6f18b8a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560017f946d0 0x560017f9c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.955+0000 7f6f18b8a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6f101acc70 con 0x560017f946d0 2024-04-22T13:27:27.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.955+0000 7f6f18b8a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6f101ac5a0 con 0x560017f93d20 2024-04-22T13:27:27.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.955+0000 7f6f18b8a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6f101a9820 con 0x560017f80c70 2024-04-22T13:27:27.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.955+0000 7f6f18b8a740 10 client.0 _ll_register_callbacks cb 0x7f6f08009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:27.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.955+0000 7f6f0ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560017f946d0 0x560017f9c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.955+0000 7f6f0ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560017f946d0 0x560017f9c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:53990/0 (socket says 172.21.15.12:53990) 2024-04-22T13:27:27.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.955+0000 7f6f0ffff640 1 -- 172.21.15.12:0/1682907851 learned_addr learned my addr 172.21.15.12:0/1682907851 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:27.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.955+0000 7f6f0effd640 1 --2- 172.21.15.12:0/1682907851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560017f93d20 0x560017f9bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.955+0000 7f6f0f7fe640 1 --2- 172.21.15.12:0/1682907851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017f80c70 0x560017fa06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.955+0000 7f6f0ffff640 1 -- 172.21.15.12:0/1682907851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017f80c70 msgr2=0x560017fa06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.955+0000 7f6f0ffff640 1 --2- 172.21.15.12:0/1682907851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017f80c70 0x560017fa06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.955+0000 7f6f0ffff640 1 -- 172.21.15.12:0/1682907851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560017f93d20 msgr2=0x560017f9bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:27.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.955+0000 7f6f0ffff640 1 --2- 172.21.15.12:0/1682907851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560017f93d20 0x560017f9bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.955+0000 7f6f0ffff640 1 -- 172.21.15.12:0/1682907851 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560017f9c640 con 0x560017f946d0 2024-04-22T13:27:27.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.956+0000 7f6f0ffff640 1 --2- 172.21.15.12:0/1682907851 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560017f946d0 0x560017f9c080 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f6f08010940 tx=0x7f6f08005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd43ac18f6a3ed44 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:27.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.956+0000 7f6eeeffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:27:27.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.956+0000 7f6eeeffd640 1 -- 172.21.15.12:0/1682907851 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08009a00 con 0x560017f946d0 2024-04-22T13:27:27.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.956+0000 7f6f18b8a740 1 -- 172.21.15.12:0/1682907851 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f040beeb0 con 0x560017f946d0 2024-04-22T13:27:27.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.956+0000 7f6eeeffd640 1 -- 172.21.15.12:0/1682907851 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f6f080111e0 con 0x560017f946d0 2024-04-22T13:27:27.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.956+0000 7f6eeeffd640 1 -- 172.21.15.12:0/1682907851 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6f08048050 con 0x560017f946d0 2024-04-22T13:27:27.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.956+0000 7f6f18b8a740 10 client.16953 fetch_fsmap learned FSMap version 28 2024-04-22T13:27:27.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.956+0000 7f6f18b8a740 1 -- 172.21.15.12:0/1682907851 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56001800e640 con 0x560017f946d0 2024-04-22T13:27:27.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.956+0000 7f6eeeffd640 1 -- 172.21.15.12:0/1682907851 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f040beeb0 con 0x560017f946d0 2024-04-22T13:27:27.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.956+0000 7f6eeeffd640 1 -- 172.21.15.12:0/1682907851 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6f0800f030 con 0x560017f946d0 2024-04-22T13:27:27.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.956+0000 7f6f18b8a740 10 client.16953 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:27:27.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.956+0000 7f6f18b8a740 10 client.16953 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:27:27.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.956+0000 7f6f18b8a740 20 client.16953 populate_metadata read hostname 'smithi012' 2024-04-22T13:27:27.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.956+0000 7f6f18b8a740 1 --2- 172.21.15.12:0/1682907851 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x56001800ecd0 0x5600180110f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:27.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.956+0000 7f6f18b8a740 4 client.16953 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:27:27.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.956+0000 7f6f18b8a740 1 -- 172.21.15.12:0/1682907851 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560017ef8c10 con 0x56001800ecd0 2024-04-22T13:27:27.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.957+0000 7f6f0f7fe640 1 --2- 172.21.15.12:0/1682907851 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x56001800ecd0 0x5600180110f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:27.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.957+0000 7f6f0f7fe640 1 --2- 172.21.15.12:0/1682907851 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x56001800ecd0 0x5600180110f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5eb8a72adeecbb30 server_cookie=a19834eb19163fe0 in_seq=0 out_seq=0 2024-04-22T13:27:27.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.957+0000 7f6eeeffd640 10 client.16953 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:27:27.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.957+0000 7f6eeeffd640 1 -- 172.21.15.12:0/1682907851 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560017ef8c10 con 0x56001800ecd0 2024-04-22T13:27:27.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.957+0000 7f6eeeffd640 10 client.16953 handle_command_reply: tid=0 2024-04-22T13:27:27.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.957+0000 7f6f18b8a740 1 client.16953 shutdown 2024-04-22T13:27:27.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.957+0000 7f6f18b8a740 1 -- 172.21.15.12:0/1682907851 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560017f946d0 msgr2=0x560017f9c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.957+0000 7f6f18b8a740 1 --2- 172.21.15.12:0/1682907851 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560017f946d0 0x560017f9c080 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f6f08010940 tx=0x7f6f08005ab0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.957+0000 7f6f18b8a740 1 -- 172.21.15.12:0/1682907851 shutdown_connections 2024-04-22T13:27:27.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.957+0000 7f6f18b8a740 1 --2- 172.21.15.12:0/1682907851 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x56001800ecd0 0x5600180110f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.957+0000 7f6f18b8a740 1 --2- 172.21.15.12:0/1682907851 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560017f946d0 0x560017f9c080 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.957+0000 7f6f18b8a740 1 --2- 172.21.15.12:0/1682907851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560017f93d20 0x560017f9bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.957+0000 7f6eeeffd640 0 client.16953 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:27:27.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.957+0000 7f6f18b8a740 1 --2- 172.21.15.12:0/1682907851 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560017f80c70 0x560017fa06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.957+0000 7f6f18b8a740 1 -- 172.21.15.12:0/1682907851 >> 172.21.15.12:0/1682907851 conn(0x560017f83110 msgr2=0x560017f7f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:27.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.958+0000 7f6f18b8a740 1 -- 172.21.15.12:0/1682907851 shutdown_connections 2024-04-22T13:27:27.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.958+0000 7f6f18b8a740 1 -- 172.21.15.12:0/1682907851 wait complete. 2024-04-22T13:27:27.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.958+0000 7f6f18b8a740 20 client.16953 trim_cache size 0 max 16384 2024-04-22T13:27:27.960 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:27:27.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.958+0000 7f6f15f9f640 1 -- 172.21.15.12:0/2551570819 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6ef80a3da0 msgr2=0x7f6ef80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.958+0000 7f6f15f9f640 1 --2- 172.21.15.12:0/2551570819 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6ef80a3da0 0x7f6ef80a6260 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6f08021000 tx=0x7f6f0801e000 comp rx=0 tx=0).stop 2024-04-22T13:27:27.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.958+0000 7f6f15f9f640 1 -- 172.21.15.12:0/2551570819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f10107270 msgr2=0x7f6f10198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:27.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.958+0000 7f6f15f9f640 1 --2- 172.21.15.12:0/2551570819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f10107270 0x7f6f10198c60 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f6f04000c00 tx=0x7f6f040045e0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.958+0000 7f6f15f9f640 1 -- 172.21.15.12:0/2551570819 shutdown_connections 2024-04-22T13:27:27.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.958+0000 7f6f15f9f640 1 --2- 172.21.15.12:0/2551570819 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6ef80a3da0 0x7f6ef80a6260 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.958+0000 7f6f15f9f640 1 --2- 172.21.15.12:0/2551570819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f1019cfe0 0x7f6f1019d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.958+0000 7f6f15f9f640 1 --2- 172.21.15.12:0/2551570819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6f10107270 0x7f6f10198c60 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.958+0000 7f6f15f9f640 1 --2- 172.21.15.12:0/2551570819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f1006a840 0x7f6f10198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:27.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.958+0000 7f6f15f9f640 1 -- 172.21.15.12:0/2551570819 >> 172.21.15.12:0/2551570819 conn(0x7f6f10077eb0 msgr2=0x7f6f10108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:27.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.958+0000 7f6f15f9f640 1 -- 172.21.15.12:0/2551570819 shutdown_connections 2024-04-22T13:27:27.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:27.958+0000 7f6f15f9f640 1 -- 172.21.15.12:0/2551570819 wait complete. 2024-04-22T13:27:27.965 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:27:28.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.074+0000 7fbafe559640 1 -- 172.21.15.12:0/3555416190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbaf8107260 msgr2=0x7fbaf810ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:28.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.074+0000 7fbafe559640 1 --2- 172.21.15.12:0/3555416190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbaf8107260 0x7fbaf810ee30 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0009ca0 tx=0x7fbaf002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.074+0000 7fbafe559640 1 -- 172.21.15.12:0/3555416190 shutdown_connections 2024-04-22T13:27:28.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.074+0000 7fbafe559640 1 --2- 172.21.15.12:0/3555416190 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbaf8107260 0x7fbaf810ee30 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.074+0000 7fbafe559640 1 --2- 172.21.15.12:0/3555416190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbaf8102b40 0x7fbaf8106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.074+0000 7fbafe559640 1 --2- 172.21.15.12:0/3555416190 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbaf8102190 0x7fbaf8102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.074+0000 7fbafe559640 1 -- 172.21.15.12:0/3555416190 >> 172.21.15.12:0/3555416190 conn(0x7fbaf80fd990 msgr2=0x7fbaf80ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:28.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.074+0000 7fbafe559640 1 -- 172.21.15.12:0/3555416190 shutdown_connections 2024-04-22T13:27:28.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.074+0000 7fbafe559640 1 -- 172.21.15.12:0/3555416190 wait complete. 2024-04-22T13:27:28.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.074+0000 7fbafe559640 1 Processor -- start 2024-04-22T13:27:28.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.074+0000 7fbafe559640 1 -- start start 2024-04-22T13:27:28.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.075+0000 7fbafe559640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbaf8102190 0x7fbaf81a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.075+0000 7fbafe559640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbaf8107260 0x7fbaf81a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.075+0000 7fbafe559640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbaf819a680 0x7fbaf819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.075+0000 7fbafe559640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf81121b0 con 0x7fbaf8107260 2024-04-22T13:27:28.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.075+0000 7fbafe559640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf8112030 con 0x7fbaf819a680 2024-04-22T13:27:28.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.075+0000 7fbafe559640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf8112330 con 0x7fbaf8102190 2024-04-22T13:27:28.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.075+0000 7fbaf77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbaf8107260 0x7fbaf81a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.075+0000 7fbaf77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbaf8107260 0x7fbaf81a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54004/0 (socket says 172.21.15.12:54004) 2024-04-22T13:27:28.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.075+0000 7fbaf77fe640 1 -- 172.21.15.12:0/1059032032 learned_addr learned my addr 172.21.15.12:0/1059032032 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:28.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.075+0000 7fbaf7fff640 1 --2- 172.21.15.12:0/1059032032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbaf8102190 0x7fbaf81a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.075+0000 7fbafcaf6640 1 --2- 172.21.15.12:0/1059032032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbaf819a680 0x7fbaf819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.075+0000 7fbaf77fe640 1 -- 172.21.15.12:0/1059032032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbaf8102190 msgr2=0x7fbaf81a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:28.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.075+0000 7fbaf77fe640 1 --2- 172.21.15.12:0/1059032032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbaf8102190 0x7fbaf81a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.075+0000 7fbaf77fe640 1 -- 172.21.15.12:0/1059032032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbaf819a680 msgr2=0x7fbaf819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:28.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.075+0000 7fbaf77fe640 1 --2- 172.21.15.12:0/1059032032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbaf819a680 0x7fbaf819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.075+0000 7fbaf77fe640 1 -- 172.21.15.12:0/1059032032 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf819b280 con 0x7fbaf8107260 2024-04-22T13:27:28.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.075+0000 7fbaf77fe640 1 --2- 172.21.15.12:0/1059032032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbaf8107260 0x7fbaf81a0b00 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fbaec005fd0 tx=0x7fbaec00dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:28.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.076+0000 7fbaf57fa640 1 -- 172.21.15.12:0/1059032032 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec017070 con 0x7fbaf8107260 2024-04-22T13:27:28.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.076+0000 7fbaf57fa640 1 -- 172.21.15.12:0/1059032032 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaec00de50 con 0x7fbaf8107260 2024-04-22T13:27:28.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.076+0000 7fbafe559640 1 -- 172.21.15.12:0/1059032032 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf81a7360 con 0x7fbaf8107260 2024-04-22T13:27:28.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.076+0000 7fbaf57fa640 1 -- 172.21.15.12:0/1059032032 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec003c20 con 0x7fbaf8107260 2024-04-22T13:27:28.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.076+0000 7fbafe559640 1 -- 172.21.15.12:0/1059032032 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf81a7760 con 0x7fbaf8107260 2024-04-22T13:27:28.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.078+0000 7fbaf57fa640 1 -- 172.21.15.12:0/1059032032 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbaec003dc0 con 0x7fbaf8107260 2024-04-22T13:27:28.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.078+0000 7fbb01220740 1 Processor -- start 2024-04-22T13:27:28.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.078+0000 7fbb01220740 1 -- start start 2024-04-22T13:27:28.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.078+0000 7fbb01220740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4bd1a84d0 0x55f4bd1a88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.078+0000 7fbb01220740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bd1a8e80 0x55f4bd1b1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.078+0000 7fbb01220740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4bd1b1a00 0x55f4bd1b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.078+0000 7fbb01220740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f4bd10d1f0 con 0x55f4bd1a84d0 2024-04-22T13:27:28.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.078+0000 7fbb01220740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f4bd0aa4b0 con 0x55f4bd1b1a00 2024-04-22T13:27:28.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.078+0000 7fbb01220740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f4bd1091b0 con 0x55f4bd1a8e80 2024-04-22T13:27:28.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.078+0000 7fbafcaf6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4bd1a84d0 0x55f4bd1a88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.078+0000 7fbaf57fa640 1 --2- 172.21.15.12:0/1059032032 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbae00a3d20 0x7fbae00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.078+0000 7fbafcaf6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4bd1a84d0 0x55f4bd1a88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54018/0 (socket says 172.21.15.12:54018) 2024-04-22T13:27:28.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.078+0000 7fbafcaf6640 1 -- 172.21.15.12:0/87425700 learned_addr learned my addr 172.21.15.12:0/87425700 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:28.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.079+0000 7fbafcaf6640 1 --2- 172.21.15.12:0/1059032032 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbae00a3d20 0x7fbae00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.079+0000 7fbaf57fa640 1 -- 172.21.15.12:0/1059032032 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fbaec0d0390 con 0x7fbaf8107260 2024-04-22T13:27:28.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.079+0000 7fbaf77fe640 1 --2- 172.21.15.12:0/87425700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4bd1b1a00 0x55f4bd1b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.079+0000 7fbaf7fff640 1 --2- 172.21.15.12:0/87425700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bd1a8e80 0x55f4bd1b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.079+0000 7fbafcaf6640 1 -- 172.21.15.12:0/87425700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bd1a8e80 msgr2=0x55f4bd1b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:28.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.079+0000 7fbafcaf6640 1 --2- 172.21.15.12:0/87425700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bd1a8e80 0x55f4bd1b1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.079+0000 7fbafcaf6640 1 -- 172.21.15.12:0/87425700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4bd1b1a00 msgr2=0x55f4bd1b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:28.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.079+0000 7fbafcaf6640 1 --2- 172.21.15.12:0/87425700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4bd1b1a00 0x55f4bd1b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.079+0000 7fbafcaf6640 1 -- 172.21.15.12:0/87425700 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4bd09c1b0 con 0x55f4bd1a84d0 2024-04-22T13:27:28.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.079+0000 7fbaf77fe640 1 --2- 172.21.15.12:0/87425700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4bd1b1a00 0x55f4bd1b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:28.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.079+0000 7fbafcaf6640 1 --2- 172.21.15.12:0/87425700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4bd1a84d0 0x55f4bd1a88b0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fbaf002c700 tx=0x7fbaf0032cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73d718d68e20b25 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:28.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.079+0000 7fbade7fc640 1 -- 172.21.15.12:0/87425700 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0050070 con 0x55f4bd1a84d0 2024-04-22T13:27:28.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.079+0000 7fbade7fc640 1 -- 172.21.15.12:0/87425700 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaf0003b70 con 0x55f4bd1a84d0 2024-04-22T13:27:28.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.079+0000 7fbade7fc640 1 -- 172.21.15.12:0/87425700 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0039040 con 0x55f4bd1a84d0 2024-04-22T13:27:28.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.079+0000 7fbafcaf6640 1 --2- 172.21.15.12:0/1059032032 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbae00a3d20 0x7fbae00a61e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0032460 tx=0x7fbaf0048000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:28.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.080+0000 7fbb01220740 1 -- 172.21.15.12:0/87425700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4bd1a84d0 msgr2=0x55f4bd1a88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:28.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.080+0000 7fbb01220740 1 --2- 172.21.15.12:0/87425700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4bd1a84d0 0x55f4bd1a88b0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fbaf002c700 tx=0x7fbaf0032cc0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.080+0000 7fbb01220740 1 -- 172.21.15.12:0/87425700 shutdown_connections 2024-04-22T13:27:28.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.080+0000 7fbb01220740 1 --2- 172.21.15.12:0/87425700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4bd1b1a00 0x55f4bd1b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.080+0000 7fbb01220740 1 --2- 172.21.15.12:0/87425700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bd1a8e80 0x55f4bd1b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.080+0000 7fbb01220740 1 --2- 172.21.15.12:0/87425700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4bd1a84d0 0x55f4bd1a88b0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.080+0000 7fbb01220740 1 -- 172.21.15.12:0/87425700 >> 172.21.15.12:0/87425700 conn(0x55f4bd1a2ba0 msgr2=0x55f4bd1a2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:28.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.080+0000 7fbb01220740 1 -- 172.21.15.12:0/87425700 shutdown_connections 2024-04-22T13:27:28.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.080+0000 7fbb01220740 1 -- 172.21.15.12:0/87425700 wait complete. 2024-04-22T13:27:28.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.080+0000 7fbb01220740 1 -- start start 2024-04-22T13:27:28.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.081+0000 7fbb01220740 1 Processor -- start 2024-04-22T13:27:28.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.081+0000 7fbb01220740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4bd1a84d0 0x55f4bd1b1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.081+0000 7fbb01220740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4bd1a8e80 0x55f4bd1ae5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.081+0000 7fbb01220740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bd1b1a00 0x55f4bd1aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.081+0000 7fbb01220740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f4bd1b4970 con 0x55f4bd1a8e80 2024-04-22T13:27:28.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.081+0000 7fbb01220740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f4bd0c6500 con 0x55f4bd1a84d0 2024-04-22T13:27:28.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.081+0000 7fbb01220740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f4bd1b4af0 con 0x55f4bd1b1a00 2024-04-22T13:27:28.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.081+0000 7fbb01220740 10 client.0 _ll_register_callbacks cb 0x55f4bd094b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:28.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.081+0000 7fbaf77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4bd1a8e80 0x55f4bd1ae5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.081+0000 7fbaf77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4bd1a8e80 0x55f4bd1ae5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54034/0 (socket says 172.21.15.12:54034) 2024-04-22T13:27:28.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.081+0000 7fbaf77fe640 1 -- 172.21.15.12:0/1011940811 learned_addr learned my addr 172.21.15.12:0/1011940811 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:28.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.081+0000 7fbaf7fff640 1 --2- 172.21.15.12:0/1011940811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4bd1a84d0 0x55f4bd1b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.081+0000 7fbafcaf6640 1 --2- 172.21.15.12:0/1011940811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bd1b1a00 0x55f4bd1aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.081+0000 7fbaf77fe640 1 -- 172.21.15.12:0/1011940811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bd1b1a00 msgr2=0x55f4bd1aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:28.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.081+0000 7fbaf77fe640 1 --2- 172.21.15.12:0/1011940811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bd1b1a00 0x55f4bd1aeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.081+0000 7fbaf77fe640 1 -- 172.21.15.12:0/1011940811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4bd1a84d0 msgr2=0x55f4bd1b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:28.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.081+0000 7fbaf77fe640 1 --2- 172.21.15.12:0/1011940811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4bd1a84d0 0x55f4bd1b1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.081+0000 7fbaf77fe640 1 -- 172.21.15.12:0/1011940811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4bce1f3f0 con 0x55f4bd1a8e80 2024-04-22T13:27:28.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.081+0000 7fbafcaf6640 1 --2- 172.21.15.12:0/1011940811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bd1b1a00 0x55f4bd1aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:28.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.081+0000 7fbaf7fff640 1 --2- 172.21.15.12:0/1011940811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4bd1a84d0 0x55f4bd1b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:28.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.081+0000 7fbaf77fe640 1 --2- 172.21.15.12:0/1011940811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4bd1a8e80 0x55f4bd1ae5b0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fbaec09c080 tx=0x7fbaec09c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333288dea8a623a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:28.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.081+0000 7fbad77fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:27:28.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.081+0000 7fbad77fe640 1 -- 172.21.15.12:0/1011940811 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec0b4020 con 0x55f4bd1a8e80 2024-04-22T13:27:28.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.082+0000 7fbad77fe640 1 -- 172.21.15.12:0/1011940811 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fbaec0989f0 con 0x55f4bd1a8e80 2024-04-22T13:27:28.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.082+0000 7fbb01220740 1 -- 172.21.15.12:0/1011940811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaf0003b70 con 0x55f4bd1a8e80 2024-04-22T13:27:28.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.082+0000 7fbad77fe640 1 -- 172.21.15.12:0/1011940811 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbaec0bb170 con 0x55f4bd1a8e80 2024-04-22T13:27:28.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.082+0000 7fbb01220740 10 client.16965 fetch_fsmap learned FSMap version 28 2024-04-22T13:27:28.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.082+0000 7fbb01220740 1 -- 172.21.15.12:0/1011940811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f4bcee97d0 con 0x55f4bd1a8e80 2024-04-22T13:27:28.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.082+0000 7fbad77fe640 1 -- 172.21.15.12:0/1011940811 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaf0003b70 con 0x55f4bd1a8e80 2024-04-22T13:27:28.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.082+0000 7fbad77fe640 1 -- 172.21.15.12:0/1011940811 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbaec0d0070 con 0x55f4bd1a8e80 2024-04-22T13:27:28.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.082+0000 7fbb01220740 10 client.16965 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:27:28.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.082+0000 7fbb01220740 10 client.16965 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:27:28.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.082+0000 7fbb01220740 20 client.16965 populate_metadata read hostname 'smithi012' 2024-04-22T13:27:28.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.082+0000 7fbb01220740 1 --2- 172.21.15.12:0/1011940811 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55f4bd1c1a00 0x55f4bd1c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.082+0000 7fbb01220740 4 client.16965 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:28.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.082+0000 7fbb01220740 1 -- 172.21.15.12:0/1011940811 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4bd0ac930 con 0x55f4bd1c1a00 2024-04-22T13:27:28.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.082+0000 7fbaf7fff640 1 --2- 172.21.15.12:0/1011940811 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55f4bd1c1a00 0x55f4bd1c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.083+0000 7fbaf7fff640 1 --2- 172.21.15.12:0/1011940811 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55f4bd1c1a00 0x55f4bd1c3e50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f12fcfa0af14ef6d server_cookie=ebab607ad1ec986e in_seq=0 out_seq=0 2024-04-22T13:27:28.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.083+0000 7fbad77fe640 10 client.16965 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:27:28.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.083+0000 7fbad77fe640 1 -- 172.21.15.12:0/1011940811 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f4bd0ac930 con 0x55f4bd1c1a00 2024-04-22T13:27:28.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.083+0000 7fbad77fe640 10 client.16965 handle_command_reply: tid=0 2024-04-22T13:27:28.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.083+0000 7fbb01220740 1 client.16965 shutdown 2024-04-22T13:27:28.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.084+0000 7fbb01220740 1 -- 172.21.15.12:0/1011940811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4bd1a8e80 msgr2=0x55f4bd1ae5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:28.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.084+0000 7fbb01220740 1 --2- 172.21.15.12:0/1011940811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4bd1a8e80 0x55f4bd1ae5b0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fbaec09c080 tx=0x7fbaec09c380 comp rx=0 tx=0).stop 2024-04-22T13:27:28.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.084+0000 7fbb01220740 1 -- 172.21.15.12:0/1011940811 shutdown_connections 2024-04-22T13:27:28.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.084+0000 7fbb01220740 1 --2- 172.21.15.12:0/1011940811 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55f4bd1c1a00 0x55f4bd1c3e50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.084+0000 7fbb01220740 1 --2- 172.21.15.12:0/1011940811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bd1b1a00 0x55f4bd1aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.084+0000 7fbad77fe640 0 client.16965 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:27:28.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.084+0000 7fbb01220740 1 --2- 172.21.15.12:0/1011940811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4bd1a8e80 0x55f4bd1ae5b0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.084+0000 7fbb01220740 1 --2- 172.21.15.12:0/1011940811 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4bd1a84d0 0x55f4bd1b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.084+0000 7fbb01220740 1 -- 172.21.15.12:0/1011940811 >> 172.21.15.12:0/1011940811 conn(0x55f4bd1a2ba0 msgr2=0x55f4bd1a48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:28.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.084+0000 7fbb01220740 1 -- 172.21.15.12:0/1011940811 shutdown_connections 2024-04-22T13:27:28.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.084+0000 7fbb01220740 1 -- 172.21.15.12:0/1011940811 wait complete. 2024-04-22T13:27:28.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.084+0000 7fbb01220740 20 client.16965 trim_cache size 0 max 16384 2024-04-22T13:27:28.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.097+0000 7fbb01220740 1 Processor -- start 2024-04-22T13:27:28.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.097+0000 7fbb01220740 1 -- start start 2024-04-22T13:27:28.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.097+0000 7fbb01220740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bd1a84d0 0x55f4bd1bd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.097+0000 7fbb01220740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4bd1a8e80 0x55f4bd1b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.097+0000 7fbb01220740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4bd1b1a00 0x55f4bd1b1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.097+0000 7fbb01220740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf81afc30 con 0x55f4bd1a8e80 2024-04-22T13:27:28.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.097+0000 7fbb01220740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf81b1770 con 0x55f4bd1b1a00 2024-04-22T13:27:28.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.097+0000 7fbb01220740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf81b0300 con 0x55f4bd1a84d0 2024-04-22T13:27:28.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.098+0000 7fbaf77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4bd1a8e80 0x55f4bd1b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.098+0000 7fbaf77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4bd1a8e80 0x55f4bd1b0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54042/0 (socket says 172.21.15.12:54042) 2024-04-22T13:27:28.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.098+0000 7fbaf77fe640 1 -- 172.21.15.12:0/2371888471 learned_addr learned my addr 172.21.15.12:0/2371888471 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:28.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.098+0000 7fbaf7fff640 1 --2- 172.21.15.12:0/2371888471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bd1a84d0 0x55f4bd1bd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.098+0000 7fbafcaf6640 1 --2- 172.21.15.12:0/2371888471 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4bd1b1a00 0x55f4bd1b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.098+0000 7fbaf77fe640 1 -- 172.21.15.12:0/2371888471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bd1a84d0 msgr2=0x55f4bd1bd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:28.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.098+0000 7fbaf77fe640 1 --2- 172.21.15.12:0/2371888471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bd1a84d0 0x55f4bd1bd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.098+0000 7fbaf77fe640 1 -- 172.21.15.12:0/2371888471 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4bd1b1a00 msgr2=0x55f4bd1b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:28.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.098+0000 7fbaf77fe640 1 --2- 172.21.15.12:0/2371888471 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4bd1b1a00 0x55f4bd1b1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.098+0000 7fbaf77fe640 1 -- 172.21.15.12:0/2371888471 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4bd1a29d0 con 0x55f4bd1a8e80 2024-04-22T13:27:28.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.098+0000 7fbafcaf6640 1 --2- 172.21.15.12:0/2371888471 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4bd1b1a00 0x55f4bd1b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:28.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.098+0000 7fbaf77fe640 1 --2- 172.21.15.12:0/2371888471 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4bd1a8e80 0x55f4bd1b0ce0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fbaec0ce130 tx=0x7fbaec09a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7c0d6be2bae482 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:28.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.098+0000 7fbaddffb640 1 -- 172.21.15.12:0/2371888471 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec0b4070 con 0x55f4bd1a8e80 2024-04-22T13:27:28.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.098+0000 7fbaddffb640 1 -- 172.21.15.12:0/2371888471 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaec09b240 con 0x55f4bd1a8e80 2024-04-22T13:27:28.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.098+0000 7fbaddffb640 1 -- 172.21.15.12:0/2371888471 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec0989f0 con 0x55f4bd1a8e80 2024-04-22T13:27:28.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.099+0000 7fbb01220740 1 -- 172.21.15.12:0/2371888471 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4bd1a8e80 msgr2=0x55f4bd1b0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:28.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.099+0000 7fbb01220740 1 --2- 172.21.15.12:0/2371888471 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4bd1a8e80 0x55f4bd1b0ce0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fbaec0ce130 tx=0x7fbaec09a890 comp rx=0 tx=0).stop 2024-04-22T13:27:28.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.099+0000 7fbb01220740 1 -- 172.21.15.12:0/2371888471 shutdown_connections 2024-04-22T13:27:28.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.099+0000 7fbb01220740 1 --2- 172.21.15.12:0/2371888471 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4bd1b1a00 0x55f4bd1b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.099+0000 7fbb01220740 1 --2- 172.21.15.12:0/2371888471 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4bd1a8e80 0x55f4bd1b0ce0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.099+0000 7fbb01220740 1 --2- 172.21.15.12:0/2371888471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bd1a84d0 0x55f4bd1bd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.099+0000 7fbb01220740 1 -- 172.21.15.12:0/2371888471 >> 172.21.15.12:0/2371888471 conn(0x55f4bd1a2ba0 msgr2=0x55f4bd1a9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:28.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.099+0000 7fbb01220740 1 -- 172.21.15.12:0/2371888471 shutdown_connections 2024-04-22T13:27:28.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.099+0000 7fbb01220740 1 -- 172.21.15.12:0/2371888471 wait complete. 2024-04-22T13:27:28.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.099+0000 7fbb01220740 1 -- start start 2024-04-22T13:27:28.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.099+0000 7fbb01220740 1 Processor -- start 2024-04-22T13:27:28.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.100+0000 7fbb01220740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4bd1a84d0 0x55f4bd1ab570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.100+0000 7fbb01220740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4bd1a8e80 0x55f4bd1abab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.100+0000 7fbb01220740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bd1b1a00 0x55f4bd1ac010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.100+0000 7fbb01220740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf81b10a0 con 0x55f4bd1a8e80 2024-04-22T13:27:28.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.100+0000 7fbb01220740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf81b09d0 con 0x55f4bd1a84d0 2024-04-22T13:27:28.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.100+0000 7fbb01220740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf81adc10 con 0x55f4bd1b1a00 2024-04-22T13:27:28.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.100+0000 7fbb01220740 10 client.0 _ll_register_callbacks cb 0x7fbaec09bd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:28.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.100+0000 7fbaf77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4bd1a8e80 0x55f4bd1abab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.100+0000 7fbaf77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4bd1a8e80 0x55f4bd1abab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54056/0 (socket says 172.21.15.12:54056) 2024-04-22T13:27:28.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.100+0000 7fbaf77fe640 1 -- 172.21.15.12:0/2306018972 learned_addr learned my addr 172.21.15.12:0/2306018972 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:28.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.100+0000 7fbafcaf6640 1 --2- 172.21.15.12:0/2306018972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bd1b1a00 0x55f4bd1ac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.100+0000 7fbaf77fe640 1 -- 172.21.15.12:0/2306018972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bd1b1a00 msgr2=0x55f4bd1ac010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:28.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.100+0000 7fbaf77fe640 1 --2- 172.21.15.12:0/2306018972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bd1b1a00 0x55f4bd1ac010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.100+0000 7fbaf77fe640 1 -- 172.21.15.12:0/2306018972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4bd1a84d0 msgr2=0x55f4bd1ab570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:28.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.100+0000 7fbaf77fe640 1 --2- 172.21.15.12:0/2306018972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4bd1a84d0 0x55f4bd1ab570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.100+0000 7fbaf77fe640 1 -- 172.21.15.12:0/2306018972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4bd227260 con 0x55f4bd1a8e80 2024-04-22T13:27:28.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.100+0000 7fbaf77fe640 1 --2- 172.21.15.12:0/2306018972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4bd1a8e80 0x55f4bd1abab0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fbaec0d0360 tx=0x7fbaec0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4304032aecc8f51f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:28.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.100+0000 7fbad77fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:27:28.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.100+0000 7fbad77fe640 1 -- 172.21.15.12:0/2306018972 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec085060 con 0x55f4bd1a8e80 2024-04-22T13:27:28.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.100+0000 7fbad77fe640 1 -- 172.21.15.12:0/2306018972 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fbaec09b7d0 con 0x55f4bd1a8e80 2024-04-22T13:27:28.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.100+0000 7fbb01220740 1 -- 172.21.15.12:0/2306018972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaec09b240 con 0x55f4bd1a8e80 2024-04-22T13:27:28.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.100+0000 7fbad77fe640 1 -- 172.21.15.12:0/2306018972 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbaec0b4070 con 0x55f4bd1a8e80 2024-04-22T13:27:28.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.100+0000 7fbb01220740 10 client.16971 fetch_fsmap learned FSMap version 28 2024-04-22T13:27:28.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.100+0000 7fbb01220740 1 -- 172.21.15.12:0/2306018972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f4bd227260 con 0x55f4bd1a8e80 2024-04-22T13:27:28.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.101+0000 7fbad77fe640 1 -- 172.21.15.12:0/2306018972 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaec09b240 con 0x55f4bd1a8e80 2024-04-22T13:27:28.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.101+0000 7fbad77fe640 1 -- 172.21.15.12:0/2306018972 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbaec0d0040 con 0x55f4bd1a8e80 2024-04-22T13:27:28.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.101+0000 7fbb01220740 10 client.16971 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:27:28.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.101+0000 7fbb01220740 10 client.16971 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:27:28.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.101+0000 7fbb01220740 20 client.16971 populate_metadata read hostname 'smithi012' 2024-04-22T13:27:28.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.101+0000 7fbb01220740 1 --2- 172.21.15.12:0/2306018972 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55f4bd228730 0x55f4bd22ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.101+0000 7fbb01220740 4 client.16971 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:27:28.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.101+0000 7fbb01220740 1 -- 172.21.15.12:0/2306018972 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f4bd11ac10 con 0x55f4bd228730 2024-04-22T13:27:28.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.101+0000 7fbaf7fff640 1 --2- 172.21.15.12:0/2306018972 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55f4bd228730 0x55f4bd22ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.101+0000 7fbad77fe640 10 client.16971 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:27:28.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.101+0000 7fbaf7fff640 1 --2- 172.21.15.12:0/2306018972 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55f4bd228730 0x55f4bd22ab20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=22f68e95463c1d3b server_cookie=709588fd10550d9 in_seq=0 out_seq=0 2024-04-22T13:27:28.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.102+0000 7fbad77fe640 1 -- 172.21.15.12:0/2306018972 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f4bd11ac10 con 0x55f4bd228730 2024-04-22T13:27:28.103 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.102+0000 7fbad77fe640 10 client.16971 handle_command_reply: tid=0 2024-04-22T13:27:28.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.102+0000 7fbb01220740 1 client.16971 shutdown 2024-04-22T13:27:28.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.102+0000 7fbb01220740 1 -- 172.21.15.12:0/2306018972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4bd1a8e80 msgr2=0x55f4bd1abab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:28.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.102+0000 7fbb01220740 1 --2- 172.21.15.12:0/2306018972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4bd1a8e80 0x55f4bd1abab0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fbaec0d0360 tx=0x7fbaec0036f0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.102+0000 7fbb01220740 1 -- 172.21.15.12:0/2306018972 shutdown_connections 2024-04-22T13:27:28.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.102+0000 7fbb01220740 1 --2- 172.21.15.12:0/2306018972 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55f4bd228730 0x55f4bd22ab20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.102+0000 7fbb01220740 1 --2- 172.21.15.12:0/2306018972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f4bd1b1a00 0x55f4bd1ac010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.102+0000 7fbad77fe640 0 client.16971 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:27:28.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.102+0000 7fbb01220740 1 --2- 172.21.15.12:0/2306018972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f4bd1a8e80 0x55f4bd1abab0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.102+0000 7fbb01220740 1 --2- 172.21.15.12:0/2306018972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4bd1a84d0 0x55f4bd1ab570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.102+0000 7fbb01220740 1 -- 172.21.15.12:0/2306018972 >> 172.21.15.12:0/2306018972 conn(0x55f4bd1a2ba0 msgr2=0x55f4bd1a1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:28.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.102+0000 7fbb01220740 1 -- 172.21.15.12:0/2306018972 shutdown_connections 2024-04-22T13:27:28.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.102+0000 7fbb01220740 1 -- 172.21.15.12:0/2306018972 wait complete. 2024-04-22T13:27:28.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.102+0000 7fbb01220740 20 client.16971 trim_cache size 0 max 16384 2024-04-22T13:27:28.105 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:27:28.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.102+0000 7fbafe559640 1 -- 172.21.15.12:0/1059032032 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbae00a3d20 msgr2=0x7fbae00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:28.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.102+0000 7fbafe559640 1 --2- 172.21.15.12:0/1059032032 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbae00a3d20 0x7fbae00a61e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0032460 tx=0x7fbaf0048000 comp rx=0 tx=0).stop 2024-04-22T13:27:28.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.102+0000 7fbafe559640 1 -- 172.21.15.12:0/1059032032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbaf8107260 msgr2=0x7fbaf81a0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:28.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.102+0000 7fbafe559640 1 --2- 172.21.15.12:0/1059032032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbaf8107260 0x7fbaf81a0b00 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fbaec005fd0 tx=0x7fbaec00dd20 comp rx=0 tx=0).stop 2024-04-22T13:27:28.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.103+0000 7fbafe559640 1 -- 172.21.15.12:0/1059032032 shutdown_connections 2024-04-22T13:27:28.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.103+0000 7fbafe559640 1 --2- 172.21.15.12:0/1059032032 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbae00a3d20 0x7fbae00a61e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.103+0000 7fbafe559640 1 --2- 172.21.15.12:0/1059032032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbaf819a680 0x7fbaf819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.103+0000 7fbafe559640 1 --2- 172.21.15.12:0/1059032032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbaf8107260 0x7fbaf81a0b00 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.103+0000 7fbafe559640 1 --2- 172.21.15.12:0/1059032032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbaf8102190 0x7fbaf81a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.103+0000 7fbafe559640 1 -- 172.21.15.12:0/1059032032 >> 172.21.15.12:0/1059032032 conn(0x7fbaf80fd990 msgr2=0x7fbaf80fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:28.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.103+0000 7fbafe559640 1 -- 172.21.15.12:0/1059032032 shutdown_connections 2024-04-22T13:27:28.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.103+0000 7fbafe559640 1 -- 172.21.15.12:0/1059032032 wait complete. 2024-04-22T13:27:28.109 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:27:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.218+0000 7fed41aac640 1 -- 172.21.15.12:0/2041991119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed3c102b40 msgr2=0x7fed3c106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.218+0000 7fed41aac640 1 --2- 172.21.15.12:0/2041991119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed3c102b40 0x7fed3c106b80 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fed30009900 tx=0x7fed3002bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.218+0000 7fed41aac640 1 -- 172.21.15.12:0/2041991119 shutdown_connections 2024-04-22T13:27:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.218+0000 7fed41aac640 1 --2- 172.21.15.12:0/2041991119 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed3c107260 0x7fed3c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.218+0000 7fed41aac640 1 --2- 172.21.15.12:0/2041991119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed3c102b40 0x7fed3c106b80 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.218+0000 7fed41aac640 1 --2- 172.21.15.12:0/2041991119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed3c102190 0x7fed3c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.218+0000 7fed41aac640 1 -- 172.21.15.12:0/2041991119 >> 172.21.15.12:0/2041991119 conn(0x7fed3c0fd990 msgr2=0x7fed3c0ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:28.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.218+0000 7fed41aac640 1 -- 172.21.15.12:0/2041991119 shutdown_connections 2024-04-22T13:27:28.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.218+0000 7fed41aac640 1 -- 172.21.15.12:0/2041991119 wait complete. 2024-04-22T13:27:28.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.218+0000 7fed41aac640 1 Processor -- start 2024-04-22T13:27:28.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.219+0000 7fed41aac640 1 -- start start 2024-04-22T13:27:28.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.219+0000 7fed41aac640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed3c102190 0x7fed3c074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.219+0000 7fed41aac640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed3c107260 0x7fed3c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.219+0000 7fed41aac640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed3c074cd0 0x7fed3c1a69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.219+0000 7fed41aac640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c112200 con 0x7fed3c074cd0 2024-04-22T13:27:28.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.219+0000 7fed41aac640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c112080 con 0x7fed3c107260 2024-04-22T13:27:28.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.219+0000 7fed41aac640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c112380 con 0x7fed3c102190 2024-04-22T13:27:28.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.219+0000 7fed3bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed3c074cd0 0x7fed3c1a69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.219+0000 7fed3bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed3c074cd0 0x7fed3c1a69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54078/0 (socket says 172.21.15.12:54078) 2024-04-22T13:27:28.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.219+0000 7fed3bfff640 1 -- 172.21.15.12:0/219374434 learned_addr learned my addr 172.21.15.12:0/219374434 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:28.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.219+0000 7fed3b7fe640 1 --2- 172.21.15.12:0/219374434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed3c102190 0x7fed3c074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.219+0000 7fed3affd640 1 --2- 172.21.15.12:0/219374434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed3c107260 0x7fed3c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.219+0000 7fed3bfff640 1 -- 172.21.15.12:0/219374434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed3c102190 msgr2=0x7fed3c074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:28.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.219+0000 7fed3bfff640 1 --2- 172.21.15.12:0/219374434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed3c102190 0x7fed3c074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.219+0000 7fed3bfff640 1 -- 172.21.15.12:0/219374434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed3c107260 msgr2=0x7fed3c074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:28.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.219+0000 7fed3bfff640 1 --2- 172.21.15.12:0/219374434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed3c107260 0x7fed3c074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.219+0000 7fed3bfff640 1 -- 172.21.15.12:0/219374434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed3c1a70b0 con 0x7fed3c074cd0 2024-04-22T13:27:28.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.219+0000 7fed3affd640 1 --2- 172.21.15.12:0/219374434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed3c107260 0x7fed3c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:28.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.219+0000 7fed3bfff640 1 --2- 172.21.15.12:0/219374434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed3c074cd0 0x7fed3c1a69b0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fed3400b970 tx=0x7fed3400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:28.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.220+0000 7fed38ff9640 1 -- 172.21.15.12:0/219374434 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34019070 con 0x7fed3c074cd0 2024-04-22T13:27:28.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.220+0000 7fed38ff9640 1 -- 172.21.15.12:0/219374434 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed34014440 con 0x7fed3c074cd0 2024-04-22T13:27:28.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.220+0000 7fed41aac640 1 -- 172.21.15.12:0/219374434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed3c1a73a0 con 0x7fed3c074cd0 2024-04-22T13:27:28.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.220+0000 7fed38ff9640 1 -- 172.21.15.12:0/219374434 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed340075d0 con 0x7fed3c074cd0 2024-04-22T13:27:28.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.220+0000 7fed41aac640 1 -- 172.21.15.12:0/219374434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed3c1a78f0 con 0x7fed3c074cd0 2024-04-22T13:27:28.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.222+0000 7fed38ff9640 1 -- 172.21.15.12:0/219374434 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fed3401c070 con 0x7fed3c074cd0 2024-04-22T13:27:28.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.222+0000 7fed44778740 1 Processor -- start 2024-04-22T13:27:28.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.222+0000 7fed44778740 1 -- start start 2024-04-22T13:27:28.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.222+0000 7fed44778740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597d91ac4d0 0x5597d91ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.222+0000 7fed44778740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597d91ace80 0x5597d91b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.223+0000 7fed44778740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597d91b5a00 0x5597d91b7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.223+0000 7fed44778740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5597d91111f0 con 0x5597d91ace80 2024-04-22T13:27:28.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.223+0000 7fed44778740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5597d90ae4b0 con 0x5597d91ac4d0 2024-04-22T13:27:28.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.223+0000 7fed44778740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5597d910d1b0 con 0x5597d91b5a00 2024-04-22T13:27:28.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.223+0000 7fed3b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597d91ace80 0x5597d91b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.223+0000 7fed3b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597d91ace80 0x5597d91b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54084/0 (socket says 172.21.15.12:54084) 2024-04-22T13:27:28.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.223+0000 7fed3b7fe640 1 -- 172.21.15.12:0/3446971349 learned_addr learned my addr 172.21.15.12:0/3446971349 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:28.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.223+0000 7fed38ff9640 1 --2- 172.21.15.12:0/219374434 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fed240a3dd0 0x7fed240a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.223+0000 7fed3affd640 1 --2- 172.21.15.12:0/3446971349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597d91b5a00 0x5597d91b7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.223+0000 7fed3b7fe640 1 -- 172.21.15.12:0/3446971349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597d91b5a00 msgr2=0x5597d91b7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:28.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.223+0000 7fed3b7fe640 1 --2- 172.21.15.12:0/3446971349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597d91b5a00 0x5597d91b7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.223+0000 7fed3b7fe640 1 -- 172.21.15.12:0/3446971349 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597d91ac4d0 msgr2=0x5597d91ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:28.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.223+0000 7fed3b7fe640 1 --2- 172.21.15.12:0/3446971349 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597d91ac4d0 0x5597d91ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.223+0000 7fed3b7fe640 1 -- 172.21.15.12:0/3446971349 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597d90a01b0 con 0x5597d91ace80 2024-04-22T13:27:28.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.223+0000 7fed3bfff640 1 --2- 172.21.15.12:0/219374434 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fed240a3dd0 0x7fed240a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.223+0000 7fed3affd640 1 --2- 172.21.15.12:0/3446971349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597d91b5a00 0x5597d91b7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:28.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.223+0000 7fed38ff9640 1 -- 172.21.15.12:0/219374434 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fed340cfb60 con 0x7fed3c074cd0 2024-04-22T13:27:28.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.223+0000 7fed3b7fe640 1 --2- 172.21.15.12:0/3446971349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597d91ace80 0x5597d91b5430 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fed2c00bd10 tx=0x7fed2c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e74d6d397c7ae9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:28.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.223+0000 7fed21ffb640 1 -- 172.21.15.12:0/3446971349 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2c004030 con 0x5597d91ace80 2024-04-22T13:27:28.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.223+0000 7fed21ffb640 1 -- 172.21.15.12:0/3446971349 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed2c0035f0 con 0x5597d91ace80 2024-04-22T13:27:28.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.223+0000 7fed21ffb640 1 -- 172.21.15.12:0/3446971349 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2c011750 con 0x5597d91ace80 2024-04-22T13:27:28.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.223+0000 7fed3bfff640 1 --2- 172.21.15.12:0/219374434 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fed240a3dd0 0x7fed240a6290 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fed3409bca0 tx=0x7fed340bc040 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:28.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.224+0000 7fed44778740 1 -- 172.21.15.12:0/3446971349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597d91ace80 msgr2=0x5597d91b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:28.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.224+0000 7fed44778740 1 --2- 172.21.15.12:0/3446971349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597d91ace80 0x5597d91b5430 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fed2c00bd10 tx=0x7fed2c0107a0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.224+0000 7fed44778740 1 -- 172.21.15.12:0/3446971349 shutdown_connections 2024-04-22T13:27:28.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.224+0000 7fed44778740 1 --2- 172.21.15.12:0/3446971349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597d91b5a00 0x5597d91b7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.224+0000 7fed44778740 1 --2- 172.21.15.12:0/3446971349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597d91ace80 0x5597d91b5430 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.224+0000 7fed44778740 1 --2- 172.21.15.12:0/3446971349 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597d91ac4d0 0x5597d91ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.224+0000 7fed44778740 1 -- 172.21.15.12:0/3446971349 >> 172.21.15.12:0/3446971349 conn(0x5597d91a6ba0 msgr2=0x5597d91a6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:28.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.224+0000 7fed44778740 1 -- 172.21.15.12:0/3446971349 shutdown_connections 2024-04-22T13:27:28.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.224+0000 7fed44778740 1 -- 172.21.15.12:0/3446971349 wait complete. 2024-04-22T13:27:28.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.224+0000 7fed44778740 1 -- start start 2024-04-22T13:27:28.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.225+0000 7fed44778740 1 Processor -- start 2024-04-22T13:27:28.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.225+0000 7fed44778740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597d91ac4d0 0x5597d91b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.225+0000 7fed44778740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597d91ace80 0x5597d91b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.225+0000 7fed44778740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597d91b5a00 0x5597d91b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.225+0000 7fed44778740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5597d91b8970 con 0x5597d91ac4d0 2024-04-22T13:27:28.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.225+0000 7fed44778740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5597d90ca500 con 0x5597d91ace80 2024-04-22T13:27:28.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.225+0000 7fed44778740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5597d91b8af0 con 0x5597d91b5a00 2024-04-22T13:27:28.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.225+0000 7fed44778740 10 client.0 _ll_register_callbacks cb 0x5597d9098b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:28.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.225+0000 7fed3affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597d91ac4d0 0x5597d91b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.225+0000 7fed3affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597d91ac4d0 0x5597d91b52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54100/0 (socket says 172.21.15.12:54100) 2024-04-22T13:27:28.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.225+0000 7fed3affd640 1 -- 172.21.15.12:0/1541798418 learned_addr learned my addr 172.21.15.12:0/1541798418 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:28.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.225+0000 7fed3b7fe640 1 --2- 172.21.15.12:0/1541798418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597d91ace80 0x5597d91b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.225+0000 7fed3affd640 1 --2- 172.21.15.12:0/1541798418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597d91b5a00 0x5597d91b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.225+0000 7fed3affd640 1 -- 172.21.15.12:0/1541798418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597d91b5a00 msgr2=0x5597d91b2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:28.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.225+0000 7fed3affd640 1 --2- 172.21.15.12:0/1541798418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597d91b5a00 0x5597d91b2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.225+0000 7fed3affd640 1 -- 172.21.15.12:0/1541798418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597d91ace80 msgr2=0x5597d91b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:28.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.225+0000 7fed3affd640 1 --2- 172.21.15.12:0/1541798418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597d91ace80 0x5597d91b2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.225+0000 7fed3affd640 1 -- 172.21.15.12:0/1541798418 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597d8e233f0 con 0x5597d91ac4d0 2024-04-22T13:27:28.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.225+0000 7fed3b7fe640 1 --2- 172.21.15.12:0/1541798418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597d91ace80 0x5597d91b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:27:28.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.226+0000 7fed3affd640 1 --2- 172.21.15.12:0/1541798418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597d91ac4d0 0x5597d91b52c0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fed30009320 tx=0x7fed30005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38a405924e7f0ac server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:28.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.226+0000 7fed1affd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:27:28.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.226+0000 7fed1affd640 1 -- 172.21.15.12:0/1541798418 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3002cbf0 con 0x5597d91ac4d0 2024-04-22T13:27:28.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.226+0000 7fed1affd640 1 -- 172.21.15.12:0/1541798418 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fed30005040 con 0x5597d91ac4d0 2024-04-22T13:27:28.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.226+0000 7fed44778740 1 -- 172.21.15.12:0/1541798418 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed2c0035f0 con 0x5597d91ac4d0 2024-04-22T13:27:28.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.226+0000 7fed1affd640 1 -- 172.21.15.12:0/1541798418 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fed30005240 con 0x5597d91ac4d0 2024-04-22T13:27:28.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.226+0000 7fed44778740 10 client.16983 fetch_fsmap learned FSMap version 28 2024-04-22T13:27:28.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.226+0000 7fed44778740 1 -- 172.21.15.12:0/1541798418 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5597d8eed7d0 con 0x5597d91ac4d0 2024-04-22T13:27:28.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.226+0000 7fed1affd640 1 -- 172.21.15.12:0/1541798418 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed2c0035f0 con 0x5597d91ac4d0 2024-04-22T13:27:28.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.226+0000 7fed1affd640 1 -- 172.21.15.12:0/1541798418 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fed300035f0 con 0x5597d91ac4d0 2024-04-22T13:27:28.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.226+0000 7fed44778740 10 client.16983 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:27:28.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.226+0000 7fed44778740 10 client.16983 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:27:28.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.226+0000 7fed44778740 20 client.16983 populate_metadata read hostname 'smithi012' 2024-04-22T13:27:28.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.226+0000 7fed44778740 1 --2- 172.21.15.12:0/1541798418 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5597d91c5bd0 0x5597d91c8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.226+0000 7fed44778740 4 client.16983 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:28.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.226+0000 7fed44778740 1 -- 172.21.15.12:0/1541798418 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597d90b0930 con 0x5597d91c5bd0 2024-04-22T13:27:28.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.227+0000 7fed3b7fe640 1 --2- 172.21.15.12:0/1541798418 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5597d91c5bd0 0x5597d91c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.227+0000 7fed1affd640 10 client.16983 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:27:28.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.227+0000 7fed3b7fe640 1 --2- 172.21.15.12:0/1541798418 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5597d91c5bd0 0x5597d91c8020 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=796d92cdabcd9e59 server_cookie=3b2e3485c03c80bd in_seq=0 out_seq=0 2024-04-22T13:27:28.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.228+0000 7fed1affd640 1 -- 172.21.15.12:0/1541798418 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5597d90b0930 con 0x5597d91c5bd0 2024-04-22T13:27:28.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.228+0000 7fed1affd640 10 client.16983 handle_command_reply: tid=0 2024-04-22T13:27:28.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.228+0000 7fed44778740 1 client.16983 shutdown 2024-04-22T13:27:28.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.228+0000 7fed44778740 1 -- 172.21.15.12:0/1541798418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597d91ac4d0 msgr2=0x5597d91b52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:28.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.228+0000 7fed44778740 1 --2- 172.21.15.12:0/1541798418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597d91ac4d0 0x5597d91b52c0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fed30009320 tx=0x7fed30005b50 comp rx=0 tx=0).stop 2024-04-22T13:27:28.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.228+0000 7fed44778740 1 -- 172.21.15.12:0/1541798418 shutdown_connections 2024-04-22T13:27:28.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.228+0000 7fed44778740 1 --2- 172.21.15.12:0/1541798418 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5597d91c5bd0 0x5597d91c8020 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.228+0000 7fed44778740 1 --2- 172.21.15.12:0/1541798418 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597d91b5a00 0x5597d91b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.228+0000 7fed44778740 1 --2- 172.21.15.12:0/1541798418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597d91ace80 0x5597d91b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.228+0000 7fed1affd640 0 client.16983 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:27:28.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.228+0000 7fed44778740 1 --2- 172.21.15.12:0/1541798418 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597d91ac4d0 0x5597d91b52c0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.228+0000 7fed44778740 1 -- 172.21.15.12:0/1541798418 >> 172.21.15.12:0/1541798418 conn(0x5597d91a6ba0 msgr2=0x5597d91a88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:28.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.228+0000 7fed44778740 1 -- 172.21.15.12:0/1541798418 shutdown_connections 2024-04-22T13:27:28.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.228+0000 7fed44778740 1 -- 172.21.15.12:0/1541798418 wait complete. 2024-04-22T13:27:28.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.228+0000 7fed44778740 20 client.16983 trim_cache size 0 max 16384 2024-04-22T13:27:28.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.241+0000 7fed44778740 1 Processor -- start 2024-04-22T13:27:28.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.241+0000 7fed44778740 1 -- start start 2024-04-22T13:27:28.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.242+0000 7fed44778740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597d91ac4d0 0x5597d91c1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.242+0000 7fed44778740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597d91ace80 0x5597d91b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.242+0000 7fed44778740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597d91b5a00 0x5597d91b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.242+0000 7fed44778740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1afd80 con 0x5597d91b5a00 2024-04-22T13:27:28.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.242+0000 7fed44778740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1b18c0 con 0x5597d91ac4d0 2024-04-22T13:27:28.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.242+0000 7fed44778740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1b0450 con 0x5597d91ace80 2024-04-22T13:27:28.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.242+0000 7fed3affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597d91b5a00 0x5597d91b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.242+0000 7fed3affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597d91b5a00 0x5597d91b52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54104/0 (socket says 172.21.15.12:54104) 2024-04-22T13:27:28.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.242+0000 7fed3affd640 1 -- 172.21.15.12:0/2369630658 learned_addr learned my addr 172.21.15.12:0/2369630658 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:28.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.242+0000 7fed3affd640 1 -- 172.21.15.12:0/2369630658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597d91ace80 msgr2=0x5597d91b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:28.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.242+0000 7fed3b7fe640 1 --2- 172.21.15.12:0/2369630658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597d91ace80 0x5597d91b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.242+0000 7fed3affd640 1 --2- 172.21.15.12:0/2369630658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597d91ace80 0x5597d91b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.242+0000 7fed3affd640 1 -- 172.21.15.12:0/2369630658 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597d91ac4d0 msgr2=0x5597d91c1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:28.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.242+0000 7fed3affd640 1 --2- 172.21.15.12:0/2369630658 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597d91ac4d0 0x5597d91c1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.242+0000 7fed3affd640 1 -- 172.21.15.12:0/2369630658 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597d91b7df0 con 0x5597d91b5a00 2024-04-22T13:27:28.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.242+0000 7fed3b7fe640 1 --2- 172.21.15.12:0/2369630658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597d91ace80 0x5597d91b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:28.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.242+0000 7fed3affd640 1 --2- 172.21.15.12:0/2369630658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597d91b5a00 0x5597d91b52c0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fed30005930 tx=0x7fed30005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5d0ba1afdc634c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:28.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.242+0000 7fed217fa640 1 -- 172.21.15.12:0/2369630658 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30004390 con 0x5597d91b5a00 2024-04-22T13:27:28.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.242+0000 7fed217fa640 1 -- 172.21.15.12:0/2369630658 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed30045070 con 0x5597d91b5a00 2024-04-22T13:27:28.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.242+0000 7fed217fa640 1 -- 172.21.15.12:0/2369630658 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3002c9f0 con 0x5597d91b5a00 2024-04-22T13:27:28.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.243+0000 7fed44778740 1 -- 172.21.15.12:0/2369630658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597d91b5a00 msgr2=0x5597d91b52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:28.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.243+0000 7fed44778740 1 --2- 172.21.15.12:0/2369630658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597d91b5a00 0x5597d91b52c0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fed30005930 tx=0x7fed30005fd0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.243+0000 7fed44778740 1 -- 172.21.15.12:0/2369630658 shutdown_connections 2024-04-22T13:27:28.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.243+0000 7fed44778740 1 --2- 172.21.15.12:0/2369630658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597d91b5a00 0x5597d91b52c0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.243+0000 7fed44778740 1 --2- 172.21.15.12:0/2369630658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597d91ace80 0x5597d91b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.243+0000 7fed44778740 1 --2- 172.21.15.12:0/2369630658 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597d91ac4d0 0x5597d91c1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.243+0000 7fed44778740 1 -- 172.21.15.12:0/2369630658 >> 172.21.15.12:0/2369630658 conn(0x5597d91a6ba0 msgr2=0x5597d91adb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:28.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.243+0000 7fed44778740 1 -- 172.21.15.12:0/2369630658 shutdown_connections 2024-04-22T13:27:28.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.243+0000 7fed44778740 1 -- 172.21.15.12:0/2369630658 wait complete. 2024-04-22T13:27:28.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.243+0000 7fed44778740 1 -- start start 2024-04-22T13:27:28.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.244+0000 7fed44778740 1 Processor -- start 2024-04-22T13:27:28.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.244+0000 7fed44778740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597d91ac4d0 0x5597d92244c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.244+0000 7fed44778740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597d91ace80 0x5597d9224a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.244+0000 7fed44778740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597d91b5a00 0x5597d91b9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.244+0000 7fed44778740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1b11f0 con 0x5597d91ace80 2024-04-22T13:27:28.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.244+0000 7fed44778740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1b0b20 con 0x5597d91b5a00 2024-04-22T13:27:28.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.244+0000 7fed44778740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1add80 con 0x5597d91ac4d0 2024-04-22T13:27:28.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.244+0000 7fed44778740 10 client.0 _ll_register_callbacks cb 0x7fed30004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:28.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.244+0000 7fed3b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597d91ace80 0x5597d9224a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.244+0000 7fed3b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597d91ace80 0x5597d9224a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54120/0 (socket says 172.21.15.12:54120) 2024-04-22T13:27:28.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.244+0000 7fed3b7fe640 1 -- 172.21.15.12:0/1190785326 learned_addr learned my addr 172.21.15.12:0/1190785326 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:28.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.244+0000 7fed3affd640 1 --2- 172.21.15.12:0/1190785326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597d91ac4d0 0x5597d92244c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.244+0000 7fed3b7fe640 1 -- 172.21.15.12:0/1190785326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597d91ac4d0 msgr2=0x5597d92244c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:28.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.244+0000 7fed3b7fe640 1 --2- 172.21.15.12:0/1190785326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597d91ac4d0 0x5597d92244c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.244+0000 7fed3b7fe640 1 -- 172.21.15.12:0/1190785326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597d91b5a00 msgr2=0x5597d91b9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:28.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.244+0000 7fed3affd640 1 --2- 172.21.15.12:0/1190785326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597d91b5a00 0x5597d91b9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.244+0000 7fed3b7fe640 1 --2- 172.21.15.12:0/1190785326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597d91b5a00 0x5597d91b9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.244+0000 7fed3b7fe640 1 -- 172.21.15.12:0/1190785326 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597d91b29c0 con 0x5597d91ace80 2024-04-22T13:27:28.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.244+0000 7fed3affd640 1 --2- 172.21.15.12:0/1190785326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597d91b5a00 0x5597d91b9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:28.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.244+0000 7fed3b7fe640 1 --2- 172.21.15.12:0/1190785326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597d91ace80 0x5597d9224a00 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fed2c010ca0 tx=0x7fed2c003e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d26586a2c57507 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:28.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.244+0000 7fed1affd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:27:28.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.245+0000 7fed1affd640 1 -- 172.21.15.12:0/1190785326 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2c0037f0 con 0x5597d91ace80 2024-04-22T13:27:28.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.245+0000 7fed1affd640 1 -- 172.21.15.12:0/1190785326 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fed2c003990 con 0x5597d91ace80 2024-04-22T13:27:28.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.245+0000 7fed44778740 1 -- 172.21.15.12:0/1190785326 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed30045070 con 0x5597d91ace80 2024-04-22T13:27:28.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.245+0000 7fed1affd640 1 -- 172.21.15.12:0/1190785326 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fed2c038050 con 0x5597d91ace80 2024-04-22T13:27:28.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.245+0000 7fed44778740 10 client.16989 fetch_fsmap learned FSMap version 28 2024-04-22T13:27:28.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.245+0000 7fed44778740 1 -- 172.21.15.12:0/1190785326 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5597d922baa0 con 0x5597d91ace80 2024-04-22T13:27:28.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.245+0000 7fed1affd640 1 -- 172.21.15.12:0/1190785326 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed30045070 con 0x5597d91ace80 2024-04-22T13:27:28.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.245+0000 7fed1affd640 1 -- 172.21.15.12:0/1190785326 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fed2c0291c0 con 0x5597d91ace80 2024-04-22T13:27:28.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.245+0000 7fed44778740 10 client.16989 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:27:28.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.245+0000 7fed44778740 10 client.16989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:27:28.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.245+0000 7fed44778740 20 client.16989 populate_metadata read hostname 'smithi012' 2024-04-22T13:27:28.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.245+0000 7fed44778740 1 --2- 172.21.15.12:0/1190785326 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5597d922bff0 0x5597d922e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:28.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.245+0000 7fed44778740 4 client.16989 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:27:28.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.245+0000 7fed44778740 1 -- 172.21.15.12:0/1190785326 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5597d911ec10 con 0x5597d922bff0 2024-04-22T13:27:28.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.245+0000 7fed3affd640 1 --2- 172.21.15.12:0/1190785326 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5597d922bff0 0x5597d922e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:28.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.246+0000 7fed1affd640 10 client.16989 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:27:28.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.246+0000 7fed3affd640 1 --2- 172.21.15.12:0/1190785326 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5597d922bff0 0x5597d922e410 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75d5ebb707768903 server_cookie=4eadcdcbae8df6a0 in_seq=0 out_seq=0 2024-04-22T13:27:28.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.246+0000 7fed1affd640 1 -- 172.21.15.12:0/1190785326 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5597d911ec10 con 0x5597d922bff0 2024-04-22T13:27:28.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.246+0000 7fed1affd640 10 client.16989 handle_command_reply: tid=0 2024-04-22T13:27:28.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.246+0000 7fed44778740 1 client.16989 shutdown 2024-04-22T13:27:28.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.246+0000 7fed44778740 1 -- 172.21.15.12:0/1190785326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597d91ace80 msgr2=0x5597d9224a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:28.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.246+0000 7fed44778740 1 --2- 172.21.15.12:0/1190785326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597d91ace80 0x5597d9224a00 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fed2c010ca0 tx=0x7fed2c003e90 comp rx=0 tx=0).stop 2024-04-22T13:27:28.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.246+0000 7fed44778740 1 -- 172.21.15.12:0/1190785326 shutdown_connections 2024-04-22T13:27:28.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.246+0000 7fed44778740 1 --2- 172.21.15.12:0/1190785326 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5597d922bff0 0x5597d922e410 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.246+0000 7fed44778740 1 --2- 172.21.15.12:0/1190785326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597d91b5a00 0x5597d91b9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.246+0000 7fed44778740 1 --2- 172.21.15.12:0/1190785326 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597d91ace80 0x5597d9224a00 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.246+0000 7fed1affd640 0 client.16989 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:27:28.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.246+0000 7fed44778740 1 --2- 172.21.15.12:0/1190785326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5597d91ac4d0 0x5597d92244c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.246+0000 7fed44778740 1 -- 172.21.15.12:0/1190785326 >> 172.21.15.12:0/1190785326 conn(0x5597d91a6ba0 msgr2=0x5597d91a5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:28.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.246+0000 7fed44778740 1 -- 172.21.15.12:0/1190785326 shutdown_connections 2024-04-22T13:27:28.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.246+0000 7fed44778740 1 -- 172.21.15.12:0/1190785326 wait complete. 2024-04-22T13:27:28.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.246+0000 7fed44778740 20 client.16989 trim_cache size 0 max 16384 2024-04-22T13:27:28.251 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:27:28.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.247+0000 7fed41aac640 1 -- 172.21.15.12:0/219374434 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fed240a3dd0 msgr2=0x7fed240a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:28.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.247+0000 7fed41aac640 1 --2- 172.21.15.12:0/219374434 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fed240a3dd0 0x7fed240a6290 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fed3409bca0 tx=0x7fed340bc040 comp rx=0 tx=0).stop 2024-04-22T13:27:28.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.247+0000 7fed41aac640 1 -- 172.21.15.12:0/219374434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed3c074cd0 msgr2=0x7fed3c1a69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:28.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.247+0000 7fed41aac640 1 --2- 172.21.15.12:0/219374434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed3c074cd0 0x7fed3c1a69b0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fed3400b970 tx=0x7fed3400be30 comp rx=0 tx=0).stop 2024-04-22T13:27:28.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.247+0000 7fed41aac640 1 -- 172.21.15.12:0/219374434 shutdown_connections 2024-04-22T13:27:28.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.247+0000 7fed41aac640 1 --2- 172.21.15.12:0/219374434 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fed240a3dd0 0x7fed240a6290 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.247+0000 7fed41aac640 1 --2- 172.21.15.12:0/219374434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fed3c074cd0 0x7fed3c1a69b0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.247+0000 7fed41aac640 1 --2- 172.21.15.12:0/219374434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed3c107260 0x7fed3c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.247+0000 7fed41aac640 1 --2- 172.21.15.12:0/219374434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed3c102190 0x7fed3c074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:28.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.247+0000 7fed41aac640 1 -- 172.21.15.12:0/219374434 >> 172.21.15.12:0/219374434 conn(0x7fed3c0fd990 msgr2=0x7fed3c0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:28.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.247+0000 7fed41aac640 1 -- 172.21.15.12:0/219374434 shutdown_connections 2024-04-22T13:27:28.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:28.247+0000 7fed41aac640 1 -- 172.21.15.12:0/219374434 wait complete. 2024-04-22T13:27:28.586 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:28 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/859302457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:27:28.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:28 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/859302457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:27:28.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:28 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/859302457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:27:29.255 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:27:29.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:27:29.255 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:27:29.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.379+0000 7ff59323f640 1 -- 172.21.15.12:0/446518088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff58c102bc0 msgr2=0x7ff58c106c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:29.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.379+0000 7ff59323f640 1 --2- 172.21.15.12:0/446518088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff58c102bc0 0x7ff58c106c00 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff574009900 tx=0x7ff57402bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:29.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.379+0000 7ff59323f640 1 -- 172.21.15.12:0/446518088 shutdown_connections 2024-04-22T13:27:29.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.379+0000 7ff59323f640 1 --2- 172.21.15.12:0/446518088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff58c1072e0 0x7ff58c10eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.379+0000 7ff59323f640 1 --2- 172.21.15.12:0/446518088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff58c102bc0 0x7ff58c106c00 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.379+0000 7ff59323f640 1 --2- 172.21.15.12:0/446518088 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff58c102210 0x7ff58c1025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.379+0000 7ff59323f640 1 -- 172.21.15.12:0/446518088 >> 172.21.15.12:0/446518088 conn(0x7ff58c0fdab0 msgr2=0x7ff58c0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:29.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.379+0000 7ff59323f640 1 -- 172.21.15.12:0/446518088 shutdown_connections 2024-04-22T13:27:29.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.379+0000 7ff59323f640 1 -- 172.21.15.12:0/446518088 wait complete. 2024-04-22T13:27:29.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.380+0000 7ff59323f640 1 Processor -- start 2024-04-22T13:27:29.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.380+0000 7ff59323f640 1 -- start start 2024-04-22T13:27:29.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.380+0000 7ff59323f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff58c102210 0x7ff58c1a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.380+0000 7ff59323f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff58c1072e0 0x7ff58c1a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.380+0000 7ff59323f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff58c19a840 0x7ff58c19acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.380+0000 7ff59323f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c112220 con 0x7ff58c102210 2024-04-22T13:27:29.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.380+0000 7ff59323f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c1120a0 con 0x7ff58c1072e0 2024-04-22T13:27:29.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.380+0000 7ff59323f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c1123a0 con 0x7ff58c19a840 2024-04-22T13:27:29.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.380+0000 7ff590fdb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff58c102210 0x7ff58c1a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.380+0000 7ff590fdb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff58c102210 0x7ff58c1a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54128/0 (socket says 172.21.15.12:54128) 2024-04-22T13:27:29.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.380+0000 7ff590fdb640 1 -- 172.21.15.12:0/4232955684 learned_addr learned my addr 172.21.15.12:0/4232955684 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.380+0000 7ff58bfff640 1 --2- 172.21.15.12:0/4232955684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff58c1072e0 0x7ff58c1a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.380+0000 7ff5917dc640 1 --2- 172.21.15.12:0/4232955684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff58c19a840 0x7ff58c19acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.380+0000 7ff590fdb640 1 -- 172.21.15.12:0/4232955684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff58c19a840 msgr2=0x7ff58c19acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.380+0000 7ff590fdb640 1 --2- 172.21.15.12:0/4232955684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff58c19a840 0x7ff58c19acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.380+0000 7ff590fdb640 1 -- 172.21.15.12:0/4232955684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff58c1072e0 msgr2=0x7ff58c1a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.380+0000 7ff590fdb640 1 --2- 172.21.15.12:0/4232955684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff58c1072e0 0x7ff58c1a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.380+0000 7ff590fdb640 1 -- 172.21.15.12:0/4232955684 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58c19b3d0 con 0x7ff58c102210 2024-04-22T13:27:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.380+0000 7ff5917dc640 1 --2- 172.21.15.12:0/4232955684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff58c19a840 0x7ff58c19acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.380+0000 7ff590fdb640 1 --2- 172.21.15.12:0/4232955684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff58c102210 0x7ff58c1a06d0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff58400eab0 tx=0x7ff58400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.381+0000 7ff589ffb640 1 -- 172.21.15.12:0/4232955684 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58400ce40 con 0x7ff58c102210 2024-04-22T13:27:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.381+0000 7ff589ffb640 1 -- 172.21.15.12:0/4232955684 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff584011920 con 0x7ff58c102210 2024-04-22T13:27:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.381+0000 7ff59323f640 1 -- 172.21.15.12:0/4232955684 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff58c19b6c0 con 0x7ff58c102210 2024-04-22T13:27:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.381+0000 7ff589ffb640 1 -- 172.21.15.12:0/4232955684 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584010a30 con 0x7ff58c102210 2024-04-22T13:27:29.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.381+0000 7ff59323f640 1 -- 172.21.15.12:0/4232955684 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff58c1a7870 con 0x7ff58c102210 2024-04-22T13:27:29.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.383+0000 7ff589ffb640 1 -- 172.21.15.12:0/4232955684 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff584010bd0 con 0x7ff58c102210 2024-04-22T13:27:29.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.383+0000 7ff595e2c740 1 Processor -- start 2024-04-22T13:27:29.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.383+0000 7ff595e2c740 1 -- start start 2024-04-22T13:27:29.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.384+0000 7ff595e2c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563f3577f8e0 0x563f3577fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.384+0000 7ff595e2c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f35780290 0x563f35788840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.384+0000 7ff595e2c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f35788e10 0x563f3578b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.384+0000 7ff595e2c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563f356e3f00 con 0x563f35780290 2024-04-22T13:27:29.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.384+0000 7ff595e2c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x563f35681590 con 0x563f3577f8e0 2024-04-22T13:27:29.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.384+0000 7ff595e2c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563f356dfec0 con 0x563f35788e10 2024-04-22T13:27:29.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.384+0000 7ff589ffb640 1 --2- 172.21.15.12:0/4232955684 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff5780a3d20 0x7ff5780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.384+0000 7ff590fdb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f35780290 0x563f35788840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.384+0000 7ff590fdb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f35780290 0x563f35788840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54136/0 (socket says 172.21.15.12:54136) 2024-04-22T13:27:29.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.384+0000 7ff590fdb640 1 -- 172.21.15.12:0/2190789859 learned_addr learned my addr 172.21.15.12:0/2190789859 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:29.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.384+0000 7ff5917dc640 1 --2- 172.21.15.12:0/4232955684 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff5780a3d20 0x7ff5780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.384+0000 7ff58bfff640 1 --2- 172.21.15.12:0/2190789859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f35788e10 0x563f3578b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.384+0000 7ff5917dc640 1 --2- 172.21.15.12:0/2190789859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563f3577f8e0 0x563f3577fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.384+0000 7ff589ffb640 1 -- 172.21.15.12:0/4232955684 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7ff5840d0e40 con 0x7ff58c102210 2024-04-22T13:27:29.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.384+0000 7ff590fdb640 1 -- 172.21.15.12:0/2190789859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f35788e10 msgr2=0x563f3578b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:29.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.384+0000 7ff590fdb640 1 --2- 172.21.15.12:0/2190789859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f35788e10 0x563f3578b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.384+0000 7ff590fdb640 1 -- 172.21.15.12:0/2190789859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563f3577f8e0 msgr2=0x563f3577fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:29.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.384+0000 7ff590fdb640 1 --2- 172.21.15.12:0/2190789859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563f3577f8e0 0x563f3577fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.384+0000 7ff590fdb640 1 -- 172.21.15.12:0/2190789859 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f354be7e0 con 0x563f35780290 2024-04-22T13:27:29.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.384+0000 7ff58bfff640 1 --2- 172.21.15.12:0/2190789859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f35788e10 0x563f3578b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:27:29.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.385+0000 7ff5917dc640 1 --2- 172.21.15.12:0/4232955684 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff5780a3d20 0x7ff5780a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff57c005d50 tx=0x7ff57c00ba00 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:29.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.385+0000 7ff590fdb640 1 --2- 172.21.15.12:0/2190789859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f35780290 0x563f35788840 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff5840ce4f0 tx=0x7ff5840104d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67bb5bc0dc484b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:29.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.385+0000 7ff582ffd640 1 -- 172.21.15.12:0/2190789859 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58409cdd0 con 0x563f35780290 2024-04-22T13:27:29.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.385+0000 7ff582ffd640 1 -- 172.21.15.12:0/2190789859 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5840a1180 con 0x563f35780290 2024-04-22T13:27:29.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.385+0000 7ff582ffd640 1 -- 172.21.15.12:0/2190789859 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5840995d0 con 0x563f35780290 2024-04-22T13:27:29.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.385+0000 7ff595e2c740 1 -- 172.21.15.12:0/2190789859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f35780290 msgr2=0x563f35788840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:29.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.385+0000 7ff595e2c740 1 --2- 172.21.15.12:0/2190789859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f35780290 0x563f35788840 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff5840ce4f0 tx=0x7ff5840104d0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.385+0000 7ff595e2c740 1 -- 172.21.15.12:0/2190789859 shutdown_connections 2024-04-22T13:27:29.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.385+0000 7ff595e2c740 1 --2- 172.21.15.12:0/2190789859 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f35788e10 0x563f3578b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.385+0000 7ff595e2c740 1 --2- 172.21.15.12:0/2190789859 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f35780290 0x563f35788840 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.385+0000 7ff595e2c740 1 --2- 172.21.15.12:0/2190789859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563f3577f8e0 0x563f3577fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.386+0000 7ff595e2c740 1 -- 172.21.15.12:0/2190789859 >> 172.21.15.12:0/2190789859 conn(0x563f35779fb0 msgr2=0x563f3577a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:29.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.386+0000 7ff595e2c740 1 -- 172.21.15.12:0/2190789859 shutdown_connections 2024-04-22T13:27:29.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.386+0000 7ff595e2c740 1 -- 172.21.15.12:0/2190789859 wait complete. 2024-04-22T13:27:29.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.386+0000 7ff595e2c740 1 -- start start 2024-04-22T13:27:29.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.386+0000 7ff595e2c740 1 Processor -- start 2024-04-22T13:27:29.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.386+0000 7ff595e2c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3577f8e0 0x563f35785e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.386+0000 7ff595e2c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f35780290 0x563f357863c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.386+0000 7ff595e2c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563f35788e10 0x563f35786900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.386+0000 7ff595e2c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563f3578bd80 con 0x563f35780290 2024-04-22T13:27:29.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.386+0000 7ff595e2c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x563f3569d580 con 0x563f35788e10 2024-04-22T13:27:29.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.386+0000 7ff595e2c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563f3578bf00 con 0x563f3577f8e0 2024-04-22T13:27:29.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.387+0000 7ff595e2c740 10 client.0 _ll_register_callbacks cb 0x563f356a3650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:29.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.387+0000 7ff58bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f35780290 0x563f357863c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.387+0000 7ff58bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f35780290 0x563f357863c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54148/0 (socket says 172.21.15.12:54148) 2024-04-22T13:27:29.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.387+0000 7ff58bfff640 1 -- 172.21.15.12:0/1012075573 learned_addr learned my addr 172.21.15.12:0/1012075573 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:29.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.387+0000 7ff5917dc640 1 --2- 172.21.15.12:0/1012075573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563f35788e10 0x563f35786900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.387+0000 7ff590fdb640 1 --2- 172.21.15.12:0/1012075573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3577f8e0 0x563f35785e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.387+0000 7ff58bfff640 1 -- 172.21.15.12:0/1012075573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3577f8e0 msgr2=0x563f35785e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:29.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.387+0000 7ff58bfff640 1 --2- 172.21.15.12:0/1012075573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3577f8e0 0x563f35785e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.387+0000 7ff58bfff640 1 -- 172.21.15.12:0/1012075573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563f35788e10 msgr2=0x563f35786900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:29.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.387+0000 7ff58bfff640 1 --2- 172.21.15.12:0/1012075573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563f35788e10 0x563f35786900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.387+0000 7ff58bfff640 1 -- 172.21.15.12:0/1012075573 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f3551ef70 con 0x563f35780290 2024-04-22T13:27:29.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.387+0000 7ff590fdb640 1 --2- 172.21.15.12:0/1012075573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3577f8e0 0x563f35785e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:29.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.387+0000 7ff58bfff640 1 --2- 172.21.15.12:0/1012075573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f35780290 0x563f357863c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7ff574009900 tx=0x7ff57402c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c729fdef8f185283 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:29.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.387+0000 7ff56bfff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:27:29.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.387+0000 7ff56bfff640 1 -- 172.21.15.12:0/1012075573 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57402c610 con 0x563f35780290 2024-04-22T13:27:29.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.387+0000 7ff56bfff640 1 -- 172.21.15.12:0/1012075573 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7ff574005030 con 0x563f35780290 2024-04-22T13:27:29.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.387+0000 7ff595e2c740 1 -- 172.21.15.12:0/1012075573 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5840a1180 con 0x563f35780290 2024-04-22T13:27:29.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.387+0000 7ff56bfff640 1 -- 172.21.15.12:0/1012075573 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff574051050 con 0x563f35780290 2024-04-22T13:27:29.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.387+0000 7ff595e2c740 10 client.17001 fetch_fsmap learned FSMap version 28 2024-04-22T13:27:29.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.387+0000 7ff595e2c740 1 -- 172.21.15.12:0/1012075573 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563f353b6170 con 0x563f35780290 2024-04-22T13:27:29.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.388+0000 7ff56bfff640 1 -- 172.21.15.12:0/1012075573 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5840a1180 con 0x563f35780290 2024-04-22T13:27:29.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.388+0000 7ff56bfff640 1 -- 172.21.15.12:0/1012075573 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff5740035f0 con 0x563f35780290 2024-04-22T13:27:29.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.388+0000 7ff595e2c740 10 client.17001 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:27:29.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.388+0000 7ff595e2c740 10 client.17001 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:27:29.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.388+0000 7ff595e2c740 20 client.17001 populate_metadata read hostname 'smithi012' 2024-04-22T13:27:29.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.388+0000 7ff595e2c740 1 --2- 172.21.15.12:0/1012075573 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x563f35799440 0x563f3579b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.388+0000 7ff595e2c740 4 client.17001 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:29.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.388+0000 7ff595e2c740 1 -- 172.21.15.12:0/1012075573 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f356f1920 con 0x563f35799440 2024-04-22T13:27:29.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.388+0000 7ff590fdb640 1 --2- 172.21.15.12:0/1012075573 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x563f35799440 0x563f3579b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.388+0000 7ff590fdb640 1 --2- 172.21.15.12:0/1012075573 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x563f35799440 0x563f3579b8c0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=431cb35816132ae0 server_cookie=a0f346420348c75d in_seq=0 out_seq=0 2024-04-22T13:27:29.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.388+0000 7ff56bfff640 10 client.17001 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:27:29.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.389+0000 7ff56bfff640 1 -- 172.21.15.12:0/1012075573 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563f356f1920 con 0x563f35799440 2024-04-22T13:27:29.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.389+0000 7ff56bfff640 10 client.17001 handle_command_reply: tid=0 2024-04-22T13:27:29.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.389+0000 7ff595e2c740 1 client.17001 shutdown 2024-04-22T13:27:29.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.389+0000 7ff595e2c740 1 -- 172.21.15.12:0/1012075573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f35780290 msgr2=0x563f357863c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:29.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.389+0000 7ff595e2c740 1 --2- 172.21.15.12:0/1012075573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f35780290 0x563f357863c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7ff574009900 tx=0x7ff57402c9f0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.389+0000 7ff595e2c740 1 -- 172.21.15.12:0/1012075573 shutdown_connections 2024-04-22T13:27:29.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.389+0000 7ff595e2c740 1 --2- 172.21.15.12:0/1012075573 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x563f35799440 0x563f3579b8c0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.389+0000 7ff595e2c740 1 --2- 172.21.15.12:0/1012075573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563f35788e10 0x563f35786900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.389+0000 7ff595e2c740 1 --2- 172.21.15.12:0/1012075573 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f35780290 0x563f357863c0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.389+0000 7ff56bfff640 0 client.17001 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:27:29.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.389+0000 7ff595e2c740 1 --2- 172.21.15.12:0/1012075573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f3577f8e0 0x563f35785e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.389+0000 7ff595e2c740 1 -- 172.21.15.12:0/1012075573 >> 172.21.15.12:0/1012075573 conn(0x563f35779fb0 msgr2=0x563f3577bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:29.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.390+0000 7ff595e2c740 1 -- 172.21.15.12:0/1012075573 shutdown_connections 2024-04-22T13:27:29.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.390+0000 7ff595e2c740 1 -- 172.21.15.12:0/1012075573 wait complete. 2024-04-22T13:27:29.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.390+0000 7ff595e2c740 20 client.17001 trim_cache size 0 max 16384 2024-04-22T13:27:29.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.407+0000 7ff595e2c740 1 Processor -- start 2024-04-22T13:27:29.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.407+0000 7ff595e2c740 1 -- start start 2024-04-22T13:27:29.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.407+0000 7ff595e2c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f35779fb0 0x563f35794c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.407+0000 7ff595e2c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563f3577f8e0 0x563f35778e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.408+0000 7ff595e2c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f35780290 0x563f357a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.408+0000 7ff595e2c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c1afcb0 con 0x563f35779fb0 2024-04-22T13:27:29.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.408+0000 7ff595e2c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c1b17f0 con 0x563f3577f8e0 2024-04-22T13:27:29.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.408+0000 7ff595e2c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c1b0380 con 0x563f35780290 2024-04-22T13:27:29.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.408+0000 7ff590fdb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f35779fb0 0x563f35794c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.408+0000 7ff590fdb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f35779fb0 0x563f35794c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54152/0 (socket says 172.21.15.12:54152) 2024-04-22T13:27:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.408+0000 7ff590fdb640 1 -- 172.21.15.12:0/1812290151 learned_addr learned my addr 172.21.15.12:0/1812290151 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.408+0000 7ff590fdb640 1 -- 172.21.15.12:0/1812290151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f35780290 msgr2=0x563f357a11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.408+0000 7ff590fdb640 1 --2- 172.21.15.12:0/1812290151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f35780290 0x563f357a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.408+0000 7ff58bfff640 1 --2- 172.21.15.12:0/1812290151 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563f3577f8e0 0x563f35778e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.408+0000 7ff590fdb640 1 -- 172.21.15.12:0/1812290151 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563f3577f8e0 msgr2=0x563f35778e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.408+0000 7ff590fdb640 1 --2- 172.21.15.12:0/1812290151 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563f3577f8e0 0x563f35778e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.408+0000 7ff590fdb640 1 -- 172.21.15.12:0/1812290151 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f357843f0 con 0x563f35779fb0 2024-04-22T13:27:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.408+0000 7ff590fdb640 1 --2- 172.21.15.12:0/1812290151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f35779fb0 0x563f35794c40 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff584099230 tx=0x7ff5840bbf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d97c5bb3848f29 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.408+0000 7ff5827fc640 1 -- 172.21.15.12:0/1812290151 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5840a1180 con 0x563f35779fb0 2024-04-22T13:27:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.408+0000 7ff5827fc640 1 -- 172.21.15.12:0/1812290151 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff58409cdd0 con 0x563f35779fb0 2024-04-22T13:27:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.408+0000 7ff5827fc640 1 -- 172.21.15.12:0/1812290151 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5840ce800 con 0x563f35779fb0 2024-04-22T13:27:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.409+0000 7ff595e2c740 1 -- 172.21.15.12:0/1812290151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f35779fb0 msgr2=0x563f35794c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:29.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.409+0000 7ff595e2c740 1 --2- 172.21.15.12:0/1812290151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f35779fb0 0x563f35794c40 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff584099230 tx=0x7ff5840bbf90 comp rx=0 tx=0).stop 2024-04-22T13:27:29.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.409+0000 7ff595e2c740 1 -- 172.21.15.12:0/1812290151 shutdown_connections 2024-04-22T13:27:29.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.409+0000 7ff595e2c740 1 --2- 172.21.15.12:0/1812290151 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f35780290 0x563f357a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.409+0000 7ff595e2c740 1 --2- 172.21.15.12:0/1812290151 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563f3577f8e0 0x563f35778e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.409+0000 7ff595e2c740 1 --2- 172.21.15.12:0/1812290151 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f35779fb0 0x563f35794c40 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.409+0000 7ff595e2c740 1 -- 172.21.15.12:0/1812290151 >> 172.21.15.12:0/1812290151 conn(0x563f35784ae0 msgr2=0x563f35778780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:29.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.409+0000 7ff595e2c740 1 -- 172.21.15.12:0/1812290151 shutdown_connections 2024-04-22T13:27:29.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.409+0000 7ff595e2c740 1 -- 172.21.15.12:0/1812290151 wait complete. 2024-04-22T13:27:29.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.409+0000 7ff595e2c740 1 -- start start 2024-04-22T13:27:29.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.409+0000 7ff595e2c740 1 Processor -- start 2024-04-22T13:27:29.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.410+0000 7ff595e2c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f35779fb0 0x563f3578e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.410+0000 7ff595e2c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f3577f8e0 0x563f3578e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.410+0000 7ff595e2c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563f35780290 0x563f3578ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.410+0000 7ff595e2c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c1b1120 con 0x563f3577f8e0 2024-04-22T13:27:29.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.410+0000 7ff595e2c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c1b0a50 con 0x563f35780290 2024-04-22T13:27:29.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.410+0000 7ff595e2c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c1adc90 con 0x563f35779fb0 2024-04-22T13:27:29.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.410+0000 7ff595e2c740 10 client.0 _ll_register_callbacks cb 0x7ff574009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:29.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.410+0000 7ff58bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f3577f8e0 0x563f3578e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.410+0000 7ff58bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f3577f8e0 0x563f3578e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54166/0 (socket says 172.21.15.12:54166) 2024-04-22T13:27:29.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.410+0000 7ff58bfff640 1 -- 172.21.15.12:0/1935284774 learned_addr learned my addr 172.21.15.12:0/1935284774 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:29.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.410+0000 7ff590fdb640 1 --2- 172.21.15.12:0/1935284774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f35779fb0 0x563f3578e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.410+0000 7ff5917dc640 1 --2- 172.21.15.12:0/1935284774 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563f35780290 0x563f3578ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.410+0000 7ff58bfff640 1 -- 172.21.15.12:0/1935284774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f35779fb0 msgr2=0x563f3578e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:29.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.410+0000 7ff58bfff640 1 --2- 172.21.15.12:0/1935284774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f35779fb0 0x563f3578e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.410+0000 7ff58bfff640 1 -- 172.21.15.12:0/1935284774 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563f35780290 msgr2=0x563f3578ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:29.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.410+0000 7ff58bfff640 1 --2- 172.21.15.12:0/1935284774 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563f35780290 0x563f3578ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.410+0000 7ff58bfff640 1 -- 172.21.15.12:0/1935284774 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f357ffcc0 con 0x563f3577f8e0 2024-04-22T13:27:29.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.410+0000 7ff590fdb640 1 --2- 172.21.15.12:0/1935284774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f35779fb0 0x563f3578e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:29.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.410+0000 7ff5917dc640 1 --2- 172.21.15.12:0/1935284774 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563f35780290 0x563f3578ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:29.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.410+0000 7ff58bfff640 1 --2- 172.21.15.12:0/1935284774 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f3577f8e0 0x563f3578e660 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7ff574009900 tx=0x7ff574004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86775eebf442e6e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:29.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.410+0000 7ff56bfff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:27:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.411+0000 7ff56bfff640 1 -- 172.21.15.12:0/1935284774 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff574000e60 con 0x563f3577f8e0 2024-04-22T13:27:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.411+0000 7ff595e2c740 1 -- 172.21.15.12:0/1935284774 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff58409cdd0 con 0x563f3577f8e0 2024-04-22T13:27:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.411+0000 7ff56bfff640 1 -- 172.21.15.12:0/1935284774 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7ff574005040 con 0x563f3577f8e0 2024-04-22T13:27:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.411+0000 7ff56bfff640 1 -- 172.21.15.12:0/1935284774 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff5740037f0 con 0x563f3577f8e0 2024-04-22T13:27:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.411+0000 7ff595e2c740 10 client.17007 fetch_fsmap learned FSMap version 28 2024-04-22T13:27:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.411+0000 7ff595e2c740 1 -- 172.21.15.12:0/1935284774 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563f35801000 con 0x563f3577f8e0 2024-04-22T13:27:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.411+0000 7ff56bfff640 1 -- 172.21.15.12:0/1935284774 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff58409cdd0 con 0x563f3577f8e0 2024-04-22T13:27:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.411+0000 7ff56bfff640 1 -- 172.21.15.12:0/1935284774 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff574004c30 con 0x563f3577f8e0 2024-04-22T13:27:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.411+0000 7ff595e2c740 10 client.17007 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:27:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.411+0000 7ff595e2c740 10 client.17007 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:27:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.411+0000 7ff595e2c740 20 client.17007 populate_metadata read hostname 'smithi012' 2024-04-22T13:27:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.411+0000 7ff595e2c740 1 --2- 172.21.15.12:0/1935284774 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x563f35788e10 0x563f358036d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.411+0000 7ff595e2c740 4 client.17007 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:27:29.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.411+0000 7ff595e2c740 1 -- 172.21.15.12:0/1935284774 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563f356eb790 con 0x563f35788e10 2024-04-22T13:27:29.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.411+0000 7ff590fdb640 1 --2- 172.21.15.12:0/1935284774 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x563f35788e10 0x563f358036d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.412+0000 7ff590fdb640 1 --2- 172.21.15.12:0/1935284774 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x563f35788e10 0x563f358036d0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a100c3046e061d7e server_cookie=5715032eecb55882 in_seq=0 out_seq=0 2024-04-22T13:27:29.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.412+0000 7ff56bfff640 10 client.17007 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:27:29.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.413+0000 7ff56bfff640 1 -- 172.21.15.12:0/1935284774 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563f356eb790 con 0x563f35788e10 2024-04-22T13:27:29.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.413+0000 7ff56bfff640 10 client.17007 handle_command_reply: tid=0 2024-04-22T13:27:29.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.413+0000 7ff595e2c740 1 client.17007 shutdown 2024-04-22T13:27:29.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.413+0000 7ff595e2c740 1 -- 172.21.15.12:0/1935284774 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f3577f8e0 msgr2=0x563f3578e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:29.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.413+0000 7ff595e2c740 1 --2- 172.21.15.12:0/1935284774 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f3577f8e0 0x563f3578e660 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7ff574009900 tx=0x7ff574004610 comp rx=0 tx=0).stop 2024-04-22T13:27:29.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.413+0000 7ff595e2c740 1 -- 172.21.15.12:0/1935284774 shutdown_connections 2024-04-22T13:27:29.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.413+0000 7ff595e2c740 1 --2- 172.21.15.12:0/1935284774 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x563f35788e10 0x563f358036d0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.413+0000 7ff595e2c740 1 --2- 172.21.15.12:0/1935284774 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563f35780290 0x563f3578ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.413+0000 7ff595e2c740 1 --2- 172.21.15.12:0/1935284774 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563f3577f8e0 0x563f3578e660 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.413+0000 7ff595e2c740 1 --2- 172.21.15.12:0/1935284774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f35779fb0 0x563f3578e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.413+0000 7ff56bfff640 0 client.17007 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:27:29.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.413+0000 7ff595e2c740 1 -- 172.21.15.12:0/1935284774 >> 172.21.15.12:0/1935284774 conn(0x563f35784ae0 msgr2=0x563f35779010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:29.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.413+0000 7ff595e2c740 1 -- 172.21.15.12:0/1935284774 shutdown_connections 2024-04-22T13:27:29.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.413+0000 7ff595e2c740 1 -- 172.21.15.12:0/1935284774 wait complete. 2024-04-22T13:27:29.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.414+0000 7ff595e2c740 20 client.17007 trim_cache size 0 max 16384 2024-04-22T13:27:29.415 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:27:29.415 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-04-22T13:27:29.415 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "289e1af8-1403-4134-af3c-c347aeb1e919", 2024-04-22T13:27:29.415 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-04-22T13:27:29.415 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:27:29.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.414+0000 7ff59323f640 1 -- 172.21.15.12:0/4232955684 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff5780a3d20 msgr2=0x7ff5780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:29.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.414+0000 7ff59323f640 1 --2- 172.21.15.12:0/4232955684 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff5780a3d20 0x7ff5780a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff57c005d50 tx=0x7ff57c00ba00 comp rx=0 tx=0).stop 2024-04-22T13:27:29.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.414+0000 7ff59323f640 1 -- 172.21.15.12:0/4232955684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff58c102210 msgr2=0x7ff58c1a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:29.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.414+0000 7ff59323f640 1 --2- 172.21.15.12:0/4232955684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff58c102210 0x7ff58c1a06d0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff58400eab0 tx=0x7ff58400ef70 comp rx=0 tx=0).stop 2024-04-22T13:27:29.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.414+0000 7ff59323f640 1 -- 172.21.15.12:0/4232955684 shutdown_connections 2024-04-22T13:27:29.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.414+0000 7ff59323f640 1 --2- 172.21.15.12:0/4232955684 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff5780a3d20 0x7ff5780a61e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.414+0000 7ff59323f640 1 --2- 172.21.15.12:0/4232955684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff58c19a840 0x7ff58c19acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.414+0000 7ff59323f640 1 --2- 172.21.15.12:0/4232955684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff58c1072e0 0x7ff58c1a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.414+0000 7ff59323f640 1 --2- 172.21.15.12:0/4232955684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff58c102210 0x7ff58c1a06d0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.414+0000 7ff59323f640 1 -- 172.21.15.12:0/4232955684 >> 172.21.15.12:0/4232955684 conn(0x7ff58c0fdab0 msgr2=0x7ff58c0fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:29.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.414+0000 7ff59323f640 1 -- 172.21.15.12:0/4232955684 shutdown_connections 2024-04-22T13:27:29.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.414+0000 7ff59323f640 1 -- 172.21.15.12:0/4232955684 wait complete. 2024-04-22T13:27:29.421 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:27:29.421 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:27:29.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.530+0000 7f377ebb2640 1 -- 172.21.15.12:0/1495097224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37781021a0 msgr2=0x7f3778102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:29.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.530+0000 7f377ebb2640 1 --2- 172.21.15.12:0/1495097224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37781021a0 0x7f3778102580 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f376c009900 tx=0x7f376c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:29.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.530+0000 7f377ebb2640 1 -- 172.21.15.12:0/1495097224 shutdown_connections 2024-04-22T13:27:29.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.530+0000 7f377ebb2640 1 --2- 172.21.15.12:0/1495097224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3778107270 0x7f377810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.530+0000 7f377ebb2640 1 --2- 172.21.15.12:0/1495097224 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3778102b50 0x7f3778106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.530+0000 7f377ebb2640 1 --2- 172.21.15.12:0/1495097224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f37781021a0 0x7f3778102580 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.530+0000 7f377ebb2640 1 -- 172.21.15.12:0/1495097224 >> 172.21.15.12:0/1495097224 conn(0x7f37780fda40 msgr2=0x7f37780ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:29.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.530+0000 7f377ebb2640 1 -- 172.21.15.12:0/1495097224 shutdown_connections 2024-04-22T13:27:29.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.530+0000 7f377ebb2640 1 -- 172.21.15.12:0/1495097224 wait complete. 2024-04-22T13:27:29.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.531+0000 7f377ebb2640 1 Processor -- start 2024-04-22T13:27:29.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.531+0000 7f377ebb2640 1 -- start start 2024-04-22T13:27:29.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.531+0000 7f377ebb2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37781021a0 0x7f3778198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.531+0000 7f377ebb2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3778107270 0x7f3778198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.531+0000 7f377ebb2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f377819d020 0x7f377819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.531+0000 7f377ebb2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f37781121e0 con 0x7f377819d020 2024-04-22T13:27:29.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.531+0000 7f377ebb2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3778112060 con 0x7f37781021a0 2024-04-22T13:27:29.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.531+0000 7f377ebb2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3778112360 con 0x7f3778107270 2024-04-22T13:27:29.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.531+0000 7f377d14f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f377819d020 0x7f377819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.531+0000 7f377d14f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f377819d020 0x7f377819d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54184/0 (socket says 172.21.15.12:54184) 2024-04-22T13:27:29.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.531+0000 7f377d14f640 1 -- 172.21.15.12:0/305391774 learned_addr learned my addr 172.21.15.12:0/305391774 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:29.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.531+0000 7f3777fff640 1 --2- 172.21.15.12:0/305391774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3778107270 0x7f3778198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.531+0000 7f377c94e640 1 --2- 172.21.15.12:0/305391774 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37781021a0 0x7f3778198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.532+0000 7f377d14f640 1 -- 172.21.15.12:0/305391774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3778107270 msgr2=0x7f3778198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:29.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.532+0000 7f377d14f640 1 --2- 172.21.15.12:0/305391774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3778107270 0x7f3778198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.532+0000 7f377d14f640 1 -- 172.21.15.12:0/305391774 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37781021a0 msgr2=0x7f3778198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:29.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.532+0000 7f377d14f640 1 --2- 172.21.15.12:0/305391774 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37781021a0 0x7f3778198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.532+0000 7f377d14f640 1 -- 172.21.15.12:0/305391774 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377819db10 con 0x7f377819d020 2024-04-22T13:27:29.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.532+0000 7f377d14f640 1 --2- 172.21.15.12:0/305391774 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f377819d020 0x7f377819d4d0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f377000eaa0 tx=0x7f377000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:29.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.532+0000 7f3775ffb640 1 -- 172.21.15.12:0/305391774 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377000cdb0 con 0x7f377819d020 2024-04-22T13:27:29.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.532+0000 7f3775ffb640 1 -- 172.21.15.12:0/305391774 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3770005020 con 0x7f377819d020 2024-04-22T13:27:29.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.532+0000 7f377ebb2640 1 -- 172.21.15.12:0/305391774 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f377819de00 con 0x7f377819d020 2024-04-22T13:27:29.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.532+0000 7f3775ffb640 1 -- 172.21.15.12:0/305391774 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37700116d0 con 0x7f377819d020 2024-04-22T13:27:29.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.532+0000 7f377ebb2640 1 -- 172.21.15.12:0/305391774 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37781a55f0 con 0x7f377819d020 2024-04-22T13:27:29.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.534+0000 7f3775ffb640 1 -- 172.21.15.12:0/305391774 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f37700051d0 con 0x7f377819d020 2024-04-22T13:27:29.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.535+0000 7f37817a3740 1 Processor -- start 2024-04-22T13:27:29.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.535+0000 7f37817a3740 1 -- start start 2024-04-22T13:27:29.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.535+0000 7f3775ffb640 1 --2- 172.21.15.12:0/305391774 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f37680a3d20 0x7f37680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.535+0000 7f37817a3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a18b3ee580 0x55a18b3ee960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.535+0000 7f37817a3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a18b3eeea0 0x55a18b3f7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.535+0000 7f37817a3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a18b3f7a80 0x55a18b3f9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.535+0000 7f37817a3740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a18b3531e0 con 0x55a18b3ee580 2024-04-22T13:27:29.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.535+0000 7f37817a3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a18b2f0390 con 0x55a18b3f7a80 2024-04-22T13:27:29.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.535+0000 7f37817a3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a18b34f1a0 con 0x55a18b3eeea0 2024-04-22T13:27:29.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.535+0000 7f3777fff640 1 --2- 172.21.15.12:0/305391774 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f37680a3d20 0x7f37680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.535+0000 7f377c94e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a18b3ee580 0x55a18b3ee960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.535+0000 7f3775ffb640 1 -- 172.21.15.12:0/305391774 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f37700d0840 con 0x7f377819d020 2024-04-22T13:27:29.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.535+0000 7f377c94e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a18b3ee580 0x55a18b3ee960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54194/0 (socket says 172.21.15.12:54194) 2024-04-22T13:27:29.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.535+0000 7f377c94e640 1 -- 172.21.15.12:0/2197973674 learned_addr learned my addr 172.21.15.12:0/2197973674 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:29.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.535+0000 7f3777fff640 1 --2- 172.21.15.12:0/2197973674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a18b3eeea0 0x55a18b3f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.536+0000 7f377c94e640 1 -- 172.21.15.12:0/2197973674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a18b3eeea0 msgr2=0x55a18b3f7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:29.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.536+0000 7f377c94e640 1 --2- 172.21.15.12:0/2197973674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a18b3eeea0 0x55a18b3f7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.536+0000 7f377c94e640 1 -- 172.21.15.12:0/2197973674 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a18b3f7a80 msgr2=0x55a18b3f9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:29.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.536+0000 7f377d14f640 1 --2- 172.21.15.12:0/2197973674 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a18b3f7a80 0x55a18b3f9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.536+0000 7f377c94e640 1 --2- 172.21.15.12:0/2197973674 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a18b3f7a80 0x55a18b3f9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.536+0000 7f377c94e640 1 -- 172.21.15.12:0/2197973674 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a18b12f7d0 con 0x55a18b3ee580 2024-04-22T13:27:29.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.536+0000 7f377d14f640 1 --2- 172.21.15.12:0/2197973674 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a18b3f7a80 0x55a18b3f9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:29.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.536+0000 7f3777fff640 1 --2- 172.21.15.12:0/305391774 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f37680a3d20 0x7f37680a61e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f376400bfd0 tx=0x7f376400bb00 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:29.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.536+0000 7f377c94e640 1 --2- 172.21.15.12:0/2197973674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a18b3ee580 0x55a18b3ee960 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f376c009900 tx=0x7f376c02c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e29c39b775eb76 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:29.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.536+0000 7f3762ffd640 1 -- 172.21.15.12:0/2197973674 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c045020 con 0x55a18b3ee580 2024-04-22T13:27:29.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.536+0000 7f3762ffd640 1 -- 172.21.15.12:0/2197973674 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f376c02c7e0 con 0x55a18b3ee580 2024-04-22T13:27:29.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.536+0000 7f3762ffd640 1 -- 172.21.15.12:0/2197973674 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c0046c0 con 0x55a18b3ee580 2024-04-22T13:27:29.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.537+0000 7f37817a3740 1 -- 172.21.15.12:0/2197973674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a18b3ee580 msgr2=0x55a18b3ee960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:29.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.537+0000 7f37817a3740 1 --2- 172.21.15.12:0/2197973674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a18b3ee580 0x55a18b3ee960 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f376c009900 tx=0x7f376c02c6c0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.537+0000 7f37817a3740 1 -- 172.21.15.12:0/2197973674 shutdown_connections 2024-04-22T13:27:29.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.537+0000 7f37817a3740 1 --2- 172.21.15.12:0/2197973674 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a18b3f7a80 0x55a18b3f9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.537+0000 7f37817a3740 1 --2- 172.21.15.12:0/2197973674 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a18b3eeea0 0x55a18b3f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.537+0000 7f37817a3740 1 --2- 172.21.15.12:0/2197973674 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a18b3ee580 0x55a18b3ee960 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.537+0000 7f37817a3740 1 -- 172.21.15.12:0/2197973674 >> 172.21.15.12:0/2197973674 conn(0x55a18b3e8cb0 msgr2=0x55a18b3e9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:29.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.537+0000 7f37817a3740 1 -- 172.21.15.12:0/2197973674 shutdown_connections 2024-04-22T13:27:29.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.537+0000 7f37817a3740 1 -- 172.21.15.12:0/2197973674 wait complete. 2024-04-22T13:27:29.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.537+0000 7f37817a3740 1 -- start start 2024-04-22T13:27:29.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.537+0000 7f37817a3740 1 Processor -- start 2024-04-22T13:27:29.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.538+0000 7f37817a3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a18b3ee580 0x55a18b3f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.538+0000 7f37817a3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a18b3eeea0 0x55a18b3f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.538+0000 7f37817a3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a18b3f7a80 0x55a18b3f5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.538+0000 7f37817a3740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a18b3fa810 con 0x55a18b3eeea0 2024-04-22T13:27:29.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.538+0000 7f37817a3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a18b30c2f0 con 0x55a18b3ee580 2024-04-22T13:27:29.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.538+0000 7f37817a3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a18b3fa990 con 0x55a18b3f7a80 2024-04-22T13:27:29.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.538+0000 7f37817a3740 10 client.0 _ll_register_callbacks cb 0x55a18b319580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:29.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.538+0000 7f3777fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a18b3eeea0 0x55a18b3f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.538+0000 7f3777fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a18b3eeea0 0x55a18b3f4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54210/0 (socket says 172.21.15.12:54210) 2024-04-22T13:27:29.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.538+0000 7f3777fff640 1 -- 172.21.15.12:0/1257005928 learned_addr learned my addr 172.21.15.12:0/1257005928 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:29.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.538+0000 7f377c94e640 1 --2- 172.21.15.12:0/1257005928 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a18b3ee580 0x55a18b3f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.538+0000 7f377d14f640 1 --2- 172.21.15.12:0/1257005928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a18b3f7a80 0x55a18b3f5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.538+0000 7f3777fff640 1 -- 172.21.15.12:0/1257005928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a18b3f7a80 msgr2=0x55a18b3f5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:29.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.538+0000 7f3777fff640 1 --2- 172.21.15.12:0/1257005928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a18b3f7a80 0x55a18b3f5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.538+0000 7f3777fff640 1 -- 172.21.15.12:0/1257005928 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a18b3ee580 msgr2=0x55a18b3f4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:29.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.538+0000 7f3777fff640 1 --2- 172.21.15.12:0/1257005928 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a18b3ee580 0x55a18b3f4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.538+0000 7f3777fff640 1 -- 172.21.15.12:0/1257005928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a18b0653f0 con 0x55a18b3eeea0 2024-04-22T13:27:29.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.538+0000 7f3777fff640 1 --2- 172.21.15.12:0/1257005928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a18b3eeea0 0x55a18b3f4f50 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f376400eb80 tx=0x7f3764004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdffbc1e4933d2fe server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:29.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.538+0000 7f3757fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:27:29.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.538+0000 7f3757fff640 1 -- 172.21.15.12:0/1257005928 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376401e9a0 con 0x55a18b3eeea0 2024-04-22T13:27:29.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.538+0000 7f3757fff640 1 -- 172.21.15.12:0/1257005928 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f376401eb40 con 0x55a18b3eeea0 2024-04-22T13:27:29.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.539+0000 7f37817a3740 1 -- 172.21.15.12:0/1257005928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f376c02c7e0 con 0x55a18b3eeea0 2024-04-22T13:27:29.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.539+0000 7f3757fff640 1 -- 172.21.15.12:0/1257005928 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f376401ed40 con 0x55a18b3eeea0 2024-04-22T13:27:29.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.539+0000 7f37817a3740 10 client.17019 fetch_fsmap learned FSMap version 28 2024-04-22T13:27:29.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.539+0000 7f37817a3740 1 -- 172.21.15.12:0/1257005928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a18b0250f0 con 0x55a18b3eeea0 2024-04-22T13:27:29.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.539+0000 7f3757fff640 1 -- 172.21.15.12:0/1257005928 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f376c02c7e0 con 0x55a18b3eeea0 2024-04-22T13:27:29.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.539+0000 7f3757fff640 1 -- 172.21.15.12:0/1257005928 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f376401f070 con 0x55a18b3eeea0 2024-04-22T13:27:29.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.539+0000 7f37817a3740 10 client.17019 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:27:29.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.539+0000 7f37817a3740 10 client.17019 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:27:29.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.539+0000 7f37817a3740 20 client.17019 populate_metadata read hostname 'smithi012' 2024-04-22T13:27:29.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.539+0000 7f37817a3740 1 --2- 172.21.15.12:0/1257005928 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55a18b407900 0x55a18b409d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.539+0000 7f37817a3740 4 client.17019 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:29.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.539+0000 7f37817a3740 1 -- 172.21.15.12:0/1257005928 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a18b360c00 con 0x55a18b407900 2024-04-22T13:27:29.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.539+0000 7f377c94e640 1 --2- 172.21.15.12:0/1257005928 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55a18b407900 0x55a18b409d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.540+0000 7f3757fff640 10 client.17019 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:27:29.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.540+0000 7f377c94e640 1 --2- 172.21.15.12:0/1257005928 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55a18b407900 0x55a18b409d50 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53ca304e2f2b3d17 server_cookie=18c8588faf3ba4b3 in_seq=0 out_seq=0 2024-04-22T13:27:29.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.540+0000 7f3757fff640 1 -- 172.21.15.12:0/1257005928 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a18b360c00 con 0x55a18b407900 2024-04-22T13:27:29.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.540+0000 7f3757fff640 10 client.17019 handle_command_reply: tid=0 2024-04-22T13:27:29.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.540+0000 7f37817a3740 1 client.17019 shutdown 2024-04-22T13:27:29.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.541+0000 7f37817a3740 1 -- 172.21.15.12:0/1257005928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a18b3eeea0 msgr2=0x55a18b3f4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:29.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.541+0000 7f37817a3740 1 --2- 172.21.15.12:0/1257005928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a18b3eeea0 0x55a18b3f4f50 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f376400eb80 tx=0x7f3764004730 comp rx=0 tx=0).stop 2024-04-22T13:27:29.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.541+0000 7f37817a3740 1 -- 172.21.15.12:0/1257005928 shutdown_connections 2024-04-22T13:27:29.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.541+0000 7f37817a3740 1 --2- 172.21.15.12:0/1257005928 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55a18b407900 0x55a18b409d50 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.541+0000 7f37817a3740 1 --2- 172.21.15.12:0/1257005928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a18b3f7a80 0x55a18b3f5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.541+0000 7f37817a3740 1 --2- 172.21.15.12:0/1257005928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a18b3eeea0 0x55a18b3f4f50 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.541+0000 7f37817a3740 1 --2- 172.21.15.12:0/1257005928 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a18b3ee580 0x55a18b3f4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.541+0000 7f37817a3740 1 -- 172.21.15.12:0/1257005928 >> 172.21.15.12:0/1257005928 conn(0x55a18b3e8cb0 msgr2=0x55a18b3eaa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:29.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.541+0000 7f3757fff640 0 client.17019 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:27:29.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.541+0000 7f37817a3740 1 -- 172.21.15.12:0/1257005928 shutdown_connections 2024-04-22T13:27:29.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.541+0000 7f37817a3740 1 -- 172.21.15.12:0/1257005928 wait complete. 2024-04-22T13:27:29.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.541+0000 7f37817a3740 20 client.17019 trim_cache size 0 max 16384 2024-04-22T13:27:29.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.558+0000 7f37817a3740 1 Processor -- start 2024-04-22T13:27:29.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.558+0000 7f37817a3740 1 -- start start 2024-04-22T13:27:29.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.558+0000 7f37817a3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a18b3ee580 0x55a18b3f7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.558+0000 7f37817a3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a18b3eeea0 0x55a18b403700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.558+0000 7f37817a3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a18b3f7a80 0x55a18b3e7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.558+0000 7f37817a3740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f37781ad9f0 con 0x55a18b3eeea0 2024-04-22T13:27:29.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.558+0000 7f37817a3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f37781af530 con 0x55a18b3f7a80 2024-04-22T13:27:29.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.558+0000 7f37817a3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f37781ae0c0 con 0x55a18b3ee580 2024-04-22T13:27:29.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.558+0000 7f3777fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a18b3eeea0 0x55a18b403700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.559+0000 7f3777fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a18b3eeea0 0x55a18b403700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54222/0 (socket says 172.21.15.12:54222) 2024-04-22T13:27:29.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.559+0000 7f3777fff640 1 -- 172.21.15.12:0/1741266677 learned_addr learned my addr 172.21.15.12:0/1741266677 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:29.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.559+0000 7f377c94e640 1 --2- 172.21.15.12:0/1741266677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a18b3ee580 0x55a18b3f7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.559+0000 7f3777fff640 1 -- 172.21.15.12:0/1741266677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a18b3ee580 msgr2=0x55a18b3f7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:29.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.559+0000 7f3777fff640 1 --2- 172.21.15.12:0/1741266677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a18b3ee580 0x55a18b3f7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.559+0000 7f3777fff640 1 -- 172.21.15.12:0/1741266677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a18b3f7a80 msgr2=0x55a18b3e7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:29.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.559+0000 7f3777fff640 1 --2- 172.21.15.12:0/1741266677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a18b3f7a80 0x55a18b3e7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.559+0000 7f3777fff640 1 -- 172.21.15.12:0/1741266677 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a18b3eecd0 con 0x55a18b3eeea0 2024-04-22T13:27:29.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.559+0000 7f3777fff640 1 --2- 172.21.15.12:0/1741266677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a18b3eeea0 0x55a18b403700 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f376400eb50 tx=0x7f37640125b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15555ff05ac6248e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:29.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.559+0000 7f37627fc640 1 -- 172.21.15.12:0/1741266677 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3764012620 con 0x55a18b3eeea0 2024-04-22T13:27:29.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.559+0000 7f37627fc640 1 -- 172.21.15.12:0/1741266677 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f376400bd10 con 0x55a18b3eeea0 2024-04-22T13:27:29.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.559+0000 7f37627fc640 1 -- 172.21.15.12:0/1741266677 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3764032480 con 0x55a18b3eeea0 2024-04-22T13:27:29.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.560+0000 7f37817a3740 1 -- 172.21.15.12:0/1741266677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a18b3eeea0 msgr2=0x55a18b403700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:29.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.560+0000 7f37817a3740 1 --2- 172.21.15.12:0/1741266677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a18b3eeea0 0x55a18b403700 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f376400eb50 tx=0x7f37640125b0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.560+0000 7f37817a3740 1 -- 172.21.15.12:0/1741266677 shutdown_connections 2024-04-22T13:27:29.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.560+0000 7f37817a3740 1 --2- 172.21.15.12:0/1741266677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a18b3f7a80 0x55a18b3e7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.560+0000 7f37817a3740 1 --2- 172.21.15.12:0/1741266677 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a18b3eeea0 0x55a18b403700 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.560+0000 7f37817a3740 1 --2- 172.21.15.12:0/1741266677 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a18b3ee580 0x55a18b3f7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.560+0000 7f37817a3740 1 -- 172.21.15.12:0/1741266677 >> 172.21.15.12:0/1741266677 conn(0x55a18b3e8cb0 msgr2=0x55a18b3e74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:29.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.560+0000 7f37817a3740 1 -- 172.21.15.12:0/1741266677 shutdown_connections 2024-04-22T13:27:29.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.560+0000 7f37817a3740 1 -- 172.21.15.12:0/1741266677 wait complete. 2024-04-22T13:27:29.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.560+0000 7f37817a3740 1 -- start start 2024-04-22T13:27:29.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.560+0000 7f37817a3740 1 Processor -- start 2024-04-22T13:27:29.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.560+0000 7f37817a3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a18b3ee580 0x55a18b3fb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.560+0000 7f37817a3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a18b3eeea0 0x55a18b3fbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.560+0000 7f37817a3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a18b3f7a80 0x55a18b3fc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.560+0000 7f37817a3740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f37781aee60 con 0x55a18b3eeea0 2024-04-22T13:27:29.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.561+0000 7f37817a3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f37781ae790 con 0x55a18b3ee580 2024-04-22T13:27:29.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.561+0000 7f37817a3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f37781ab9f0 con 0x55a18b3f7a80 2024-04-22T13:27:29.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.561+0000 7f37817a3740 10 client.0 _ll_register_callbacks cb 0x7f376400b9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:29.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.561+0000 7f3777fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a18b3eeea0 0x55a18b3fbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.561+0000 7f3777fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a18b3eeea0 0x55a18b3fbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:54238/0 (socket says 172.21.15.12:54238) 2024-04-22T13:27:29.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.561+0000 7f3777fff640 1 -- 172.21.15.12:0/2031533782 learned_addr learned my addr 172.21.15.12:0/2031533782 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:29.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.561+0000 7f377d14f640 1 --2- 172.21.15.12:0/2031533782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a18b3f7a80 0x55a18b3fc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.561+0000 7f3777fff640 1 -- 172.21.15.12:0/2031533782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a18b3f7a80 msgr2=0x55a18b3fc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:29.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.561+0000 7f3777fff640 1 --2- 172.21.15.12:0/2031533782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a18b3f7a80 0x55a18b3fc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.561+0000 7f3777fff640 1 -- 172.21.15.12:0/2031533782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a18b3ee580 msgr2=0x55a18b3fb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:29.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.561+0000 7f3777fff640 1 --2- 172.21.15.12:0/2031533782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a18b3ee580 0x55a18b3fb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.561+0000 7f3777fff640 1 -- 172.21.15.12:0/2031533782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a18b46d0d0 con 0x55a18b3eeea0 2024-04-22T13:27:29.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.561+0000 7f3777fff640 1 --2- 172.21.15.12:0/2031533782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a18b3eeea0 0x55a18b3fbd40 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f3764012440 tx=0x7f37640108f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da6113eccd4fbbf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:29.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.561+0000 7f3757fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:27:29.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.561+0000 7f3757fff640 1 -- 172.21.15.12:0/2031533782 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3764010cc0 con 0x55a18b3eeea0 2024-04-22T13:27:29.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.561+0000 7f3757fff640 1 -- 172.21.15.12:0/2031533782 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f3764005040 con 0x55a18b3eeea0 2024-04-22T13:27:29.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.561+0000 7f37817a3740 1 -- 172.21.15.12:0/2031533782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f376400bd10 con 0x55a18b3eeea0 2024-04-22T13:27:29.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.561+0000 7f3757fff640 1 -- 172.21.15.12:0/2031533782 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3764049050 con 0x55a18b3eeea0 2024-04-22T13:27:29.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.561+0000 7f37817a3740 10 client.17025 fetch_fsmap learned FSMap version 28 2024-04-22T13:27:29.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.561+0000 7f37817a3740 1 -- 172.21.15.12:0/2031533782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a18b46e3a0 con 0x55a18b3eeea0 2024-04-22T13:27:29.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.562+0000 7f3757fff640 1 -- 172.21.15.12:0/2031533782 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f376400bd10 con 0x55a18b3eeea0 2024-04-22T13:27:29.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.562+0000 7f3757fff640 1 -- 172.21.15.12:0/2031533782 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3764004850 con 0x55a18b3eeea0 2024-04-22T13:27:29.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.562+0000 7f37817a3740 10 client.17025 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:27:29.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.562+0000 7f37817a3740 10 client.17025 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:27:29.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.562+0000 7f37817a3740 20 client.17025 populate_metadata read hostname 'smithi012' 2024-04-22T13:27:29.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.562+0000 7f37817a3740 1 --2- 172.21.15.12:0/2031533782 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55a18b46e9b0 0x55a18b470dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:29.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.562+0000 7f37817a3740 4 client.17025 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:27:29.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.562+0000 7f37817a3740 1 -- 172.21.15.12:0/2031533782 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a18b35aa70 con 0x55a18b46e9b0 2024-04-22T13:27:29.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.562+0000 7f377c94e640 1 --2- 172.21.15.12:0/2031533782 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55a18b46e9b0 0x55a18b470dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:29.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.562+0000 7f377c94e640 1 --2- 172.21.15.12:0/2031533782 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55a18b46e9b0 0x55a18b470dd0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a758943e33794cc4 server_cookie=2f597acd79b55bc5 in_seq=0 out_seq=0 2024-04-22T13:27:29.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.562+0000 7f3757fff640 10 client.17025 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:27:29.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.562+0000 7f3757fff640 1 -- 172.21.15.12:0/2031533782 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a18b35aa70 con 0x55a18b46e9b0 2024-04-22T13:27:29.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.562+0000 7f3757fff640 10 client.17025 handle_command_reply: tid=0 2024-04-22T13:27:29.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.563+0000 7f37817a3740 1 client.17025 shutdown 2024-04-22T13:27:29.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.563+0000 7f37817a3740 1 -- 172.21.15.12:0/2031533782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a18b3eeea0 msgr2=0x55a18b3fbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:29.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.563+0000 7f37817a3740 1 --2- 172.21.15.12:0/2031533782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a18b3eeea0 0x55a18b3fbd40 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f3764012440 tx=0x7f37640108f0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.563+0000 7f37817a3740 1 -- 172.21.15.12:0/2031533782 shutdown_connections 2024-04-22T13:27:29.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.563+0000 7f37817a3740 1 --2- 172.21.15.12:0/2031533782 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55a18b46e9b0 0x55a18b470dd0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.563+0000 7f37817a3740 1 --2- 172.21.15.12:0/2031533782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a18b3f7a80 0x55a18b3fc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.563+0000 7f37817a3740 1 --2- 172.21.15.12:0/2031533782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a18b3eeea0 0x55a18b3fbd40 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.563+0000 7f3757fff640 0 client.17025 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:27:29.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.563+0000 7f37817a3740 1 --2- 172.21.15.12:0/2031533782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a18b3ee580 0x55a18b3fb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.563+0000 7f37817a3740 1 -- 172.21.15.12:0/2031533782 >> 172.21.15.12:0/2031533782 conn(0x55a18b3e8cb0 msgr2=0x55a18b3f5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:29.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.563+0000 7f37817a3740 1 -- 172.21.15.12:0/2031533782 shutdown_connections 2024-04-22T13:27:29.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.563+0000 7f37817a3740 1 -- 172.21.15.12:0/2031533782 wait complete. 2024-04-22T13:27:29.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.563+0000 7f37817a3740 20 client.17025 trim_cache size 0 max 16384 2024-04-22T13:27:29.564 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:27:29.564 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:27:29.564 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-04-22T13:27:29.564 INFO:teuthology.orchestra.run.smithi012.stdout: "289e1af8-1403-4134-af3c-c347aeb1e919": { 2024-04-22T13:27:29.564 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-04-22T13:27:29.565 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "289e1af8-1403-4134-af3c-c347aeb1e919", 2024-04-22T13:27:29.565 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-04-22T13:27:29.565 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:27:29.565 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:27:29.565 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:27:29.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.563+0000 7f377ebb2640 1 -- 172.21.15.12:0/305391774 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f37680a3d20 msgr2=0x7f37680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:29.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.563+0000 7f377ebb2640 1 --2- 172.21.15.12:0/305391774 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f37680a3d20 0x7f37680a61e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f376400bfd0 tx=0x7f376400bb00 comp rx=0 tx=0).stop 2024-04-22T13:27:29.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.563+0000 7f377ebb2640 1 -- 172.21.15.12:0/305391774 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f377819d020 msgr2=0x7f377819d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:29.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.563+0000 7f377ebb2640 1 --2- 172.21.15.12:0/305391774 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f377819d020 0x7f377819d4d0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f377000eaa0 tx=0x7f377000ef60 comp rx=0 tx=0).stop 2024-04-22T13:27:29.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.563+0000 7f377ebb2640 1 -- 172.21.15.12:0/305391774 shutdown_connections 2024-04-22T13:27:29.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.563+0000 7f377ebb2640 1 --2- 172.21.15.12:0/305391774 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f37680a3d20 0x7f37680a61e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.563+0000 7f377ebb2640 1 --2- 172.21.15.12:0/305391774 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f377819d020 0x7f377819d4d0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.563+0000 7f377ebb2640 1 --2- 172.21.15.12:0/305391774 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3778107270 0x7f3778198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.563+0000 7f377ebb2640 1 --2- 172.21.15.12:0/305391774 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f37781021a0 0x7f3778198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:29.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.563+0000 7f377ebb2640 1 -- 172.21.15.12:0/305391774 >> 172.21.15.12:0/305391774 conn(0x7f37780fda40 msgr2=0x7f37780fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:29.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.564+0000 7f377ebb2640 1 -- 172.21.15.12:0/305391774 shutdown_connections 2024-04-22T13:27:29.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:29.564+0000 7f377ebb2640 1 -- 172.21.15.12:0/305391774 wait complete. 2024-04-22T13:27:29.569 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:29 smithi012 ceph-mon[28265]: pgmap v104: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:27:29.570 INFO:tasks.cephfs.filesystem:scrub status for tag:289e1af8-1403-4134-af3c-c347aeb1e919 - {'path': '/', 'tag': '289e1af8-1403-4134-af3c-c347aeb1e919', 'options': 'recursive,force'} 2024-04-22T13:27:29.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:29 smithi134 ceph-mon[31338]: pgmap v104: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:27:29.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:29 smithi111 ceph-mon[31361]: pgmap v104: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:27:30.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:30 smithi134 ceph-mon[31338]: scrub queued for path: / 2024-04-22T13:27:30.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:30 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:27:30.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:30 smithi134 ceph-mon[31338]: scrub summary: active paths [/] 2024-04-22T13:27:30.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:30 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:27:30.678 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:30 smithi134 ceph-mon[31338]: scrub summary: idle 2024-04-22T13:27:30.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:30 smithi111 ceph-mon[31361]: scrub queued for path: / 2024-04-22T13:27:30.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:30 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:27:30.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:30 smithi111 ceph-mon[31361]: scrub summary: active paths [/] 2024-04-22T13:27:30.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:30 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:27:30.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:30 smithi111 ceph-mon[31361]: scrub summary: idle 2024-04-22T13:27:30.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:30 smithi012 ceph-mon[28265]: scrub queued for path: / 2024-04-22T13:27:30.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:30 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:27:30.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:30 smithi012 ceph-mon[28265]: scrub summary: active paths [/] 2024-04-22T13:27:30.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:30 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:27:30.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:30 smithi012 ceph-mon[28265]: scrub summary: idle 2024-04-22T13:27:31.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:31 smithi134 ceph-mon[31338]: pgmap v105: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:27:31.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:31 smithi111 ceph-mon[31361]: pgmap v105: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:27:31.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:31 smithi012 ceph-mon[28265]: pgmap v105: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:27:33.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:33 smithi134 ceph-mon[31338]: pgmap v106: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:27:33.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:33 smithi111 ceph-mon[31361]: pgmap v106: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:27:33.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:33 smithi012 ceph-mon[28265]: pgmap v106: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:27:35.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:35 smithi134 ceph-mon[31338]: pgmap v107: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:27:35.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:35 smithi111 ceph-mon[31361]: pgmap v107: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:27:35.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:35 smithi012 ceph-mon[28265]: pgmap v107: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:27:37.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:37 smithi134 ceph-mon[31338]: pgmap v108: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:27:37.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:37 smithi111 ceph-mon[31361]: pgmap v108: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:27:37.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:37 smithi012 ceph-mon[28265]: pgmap v108: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:27:39.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:39 smithi134 ceph-mon[31338]: pgmap v109: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:27:39.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:39 smithi111 ceph-mon[31361]: pgmap v109: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:27:39.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:39 smithi012 ceph-mon[28265]: pgmap v109: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:27:41.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:41 smithi134 ceph-mon[31338]: pgmap v110: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:27:41.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:41 smithi111 ceph-mon[31361]: pgmap v110: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:27:41.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:41 smithi012 ceph-mon[28265]: pgmap v110: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:27:43.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:43 smithi134 ceph-mon[31338]: pgmap v111: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 13 op/s 2024-04-22T13:27:43.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:43 smithi111 ceph-mon[31361]: pgmap v111: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 13 op/s 2024-04-22T13:27:43.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:43 smithi012 ceph-mon[28265]: pgmap v111: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 13 op/s 2024-04-22T13:27:45.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:45 smithi134 ceph-mon[31338]: pgmap v112: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:27:45.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:45 smithi111 ceph-mon[31361]: pgmap v112: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:27:45.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:45 smithi012 ceph-mon[28265]: pgmap v112: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:27:47.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:47 smithi134 ceph-mon[31338]: pgmap v113: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:27:47.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:47 smithi111 ceph-mon[31361]: pgmap v113: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:27:47.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:47 smithi012 ceph-mon[28265]: pgmap v113: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:27:49.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:49 smithi134 ceph-mon[31338]: pgmap v114: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:27:49.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:49 smithi111 ceph-mon[31361]: pgmap v114: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:27:49.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:49 smithi012 ceph-mon[28265]: pgmap v114: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:27:51.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:51 smithi134 ceph-mon[31338]: pgmap v115: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:27:51.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:51 smithi111 ceph-mon[31361]: pgmap v115: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:27:51.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:51 smithi012 ceph-mon[28265]: pgmap v115: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:27:53.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:53 smithi134 ceph-mon[31338]: pgmap v116: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:27:53.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:53 smithi111 ceph-mon[31361]: pgmap v116: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:27:53.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:53 smithi012 ceph-mon[28265]: pgmap v116: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:27:55.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:55 smithi134 ceph-mon[31338]: pgmap v117: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:27:55.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:55 smithi111 ceph-mon[31361]: pgmap v117: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:27:55.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:55 smithi012 ceph-mon[28265]: pgmap v117: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:27:57.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:57 smithi134 ceph-mon[31338]: pgmap v118: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:27:57.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:57 smithi111 ceph-mon[31361]: pgmap v118: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:27:57.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:57 smithi012 ceph-mon[28265]: pgmap v118: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:27:59.571 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:27:59.572 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:27:59.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:27:59 smithi134 ceph-mon[31338]: pgmap v119: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:27:59.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.694+0000 7fd928b24640 1 -- 172.21.15.12:0/1605479856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd924103ae0 msgr2=0x7fd92410bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:59.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.694+0000 7fd928b24640 1 --2- 172.21.15.12:0/1605479856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd924103ae0 0x7fd92410bfc0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fd918009900 tx=0x7fd91802bcd0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.694+0000 7fd928b24640 1 -- 172.21.15.12:0/1605479856 shutdown_connections 2024-04-22T13:27:59.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.694+0000 7fd928b24640 1 --2- 172.21.15.12:0/1605479856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd92410c590 0x7fd92410e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.694+0000 7fd928b24640 1 --2- 172.21.15.12:0/1605479856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd924103ae0 0x7fd92410bfc0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.694+0000 7fd928b24640 1 --2- 172.21.15.12:0/1605479856 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9241031c0 0x7fd9241035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.694+0000 7fd928b24640 1 -- 172.21.15.12:0/1605479856 >> 172.21.15.12:0/1605479856 conn(0x7fd924077f70 msgr2=0x7fd924078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:59.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.694+0000 7fd928b24640 1 -- 172.21.15.12:0/1605479856 shutdown_connections 2024-04-22T13:27:59.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.694+0000 7fd928b24640 1 -- 172.21.15.12:0/1605479856 wait complete. 2024-04-22T13:27:59.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.695+0000 7fd928b24640 1 Processor -- start 2024-04-22T13:27:59.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.695+0000 7fd928b24640 1 -- start start 2024-04-22T13:27:59.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.695+0000 7fd928b24640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9241031c0 0x7fd92419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:59.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.695+0000 7fd928b24640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd924103ae0 0x7fd92419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:59.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.695+0000 7fd928b24640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd92410c590 0x7fd924198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:59.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.695+0000 7fd928b24640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd9241121b0 con 0x7fd9241031c0 2024-04-22T13:27:59.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.695+0000 7fd928b24640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd924112030 con 0x7fd924103ae0 2024-04-22T13:27:59.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.695+0000 7fd928b24640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd924112330 con 0x7fd92410c590 2024-04-22T13:27:59.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.695+0000 7fd92259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9241031c0 0x7fd92419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:59.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.695+0000 7fd92259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9241031c0 0x7fd92419e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52038/0 (socket says 172.21.15.12:52038) 2024-04-22T13:27:59.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.695+0000 7fd92259c640 1 -- 172.21.15.12:0/675302712 learned_addr learned my addr 172.21.15.12:0/675302712 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:59.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.696+0000 7fd922d9d640 1 --2- 172.21.15.12:0/675302712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd92410c590 0x7fd924198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:59.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.696+0000 7fd921d9b640 1 --2- 172.21.15.12:0/675302712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd924103ae0 0x7fd92419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:59.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.696+0000 7fd92259c640 1 -- 172.21.15.12:0/675302712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd92410c590 msgr2=0x7fd924198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:59.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.696+0000 7fd92259c640 1 --2- 172.21.15.12:0/675302712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd92410c590 0x7fd924198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.696+0000 7fd92259c640 1 -- 172.21.15.12:0/675302712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd924103ae0 msgr2=0x7fd92419e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:59.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.696+0000 7fd92259c640 1 --2- 172.21.15.12:0/675302712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd924103ae0 0x7fd92419e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.696+0000 7fd92259c640 1 -- 172.21.15.12:0/675302712 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd924198d80 con 0x7fd9241031c0 2024-04-22T13:27:59.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.696+0000 7fd921d9b640 1 --2- 172.21.15.12:0/675302712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd924103ae0 0x7fd92419e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:59.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.696+0000 7fd92259c640 1 --2- 172.21.15.12:0/675302712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9241031c0 0x7fd92419e450 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd91400ebc0 tx=0x7fd91400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:59.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.696+0000 7fd9137fe640 1 -- 172.21.15.12:0/675302712 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd914010630 con 0x7fd9241031c0 2024-04-22T13:27:59.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.696+0000 7fd9137fe640 1 -- 172.21.15.12:0/675302712 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd914010c80 con 0x7fd9241031c0 2024-04-22T13:27:59.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.696+0000 7fd928b24640 1 -- 172.21.15.12:0/675302712 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd924199070 con 0x7fd9241031c0 2024-04-22T13:27:59.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.696+0000 7fd9137fe640 1 -- 172.21.15.12:0/675302712 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd914018b60 con 0x7fd9241031c0 2024-04-22T13:27:59.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.696+0000 7fd928b24640 1 -- 172.21.15.12:0/675302712 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9241a5310 con 0x7fd9241031c0 2024-04-22T13:27:59.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.698+0000 7fd9137fe640 1 -- 172.21.15.12:0/675302712 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd914018d00 con 0x7fd9241031c0 2024-04-22T13:27:59.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.699+0000 7fd92b7f2740 1 Processor -- start 2024-04-22T13:27:59.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.699+0000 7fd92b7f2740 1 -- start start 2024-04-22T13:27:59.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.699+0000 7fd92b7f2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a76b4a7270 0x55a76b4a7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:59.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.699+0000 7fd92b7f2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a76b4a7b90 0x55a76b4b4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:59.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.699+0000 7fd92b7f2740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76b4b4e90 0x55a76b4b7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:59.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.699+0000 7fd9137fe640 1 --2- 172.21.15.12:0/675302712 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd90c0a3d20 0x7fd90c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:59.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.699+0000 7fd92b7f2740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a76b40c1e0 con 0x55a76b4b4e90 2024-04-22T13:27:59.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.699+0000 7fd92b7f2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a76b3a9390 con 0x55a76b4a7b90 2024-04-22T13:27:59.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.699+0000 7fd92b7f2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a76b4081a0 con 0x55a76b4a7270 2024-04-22T13:27:59.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.700+0000 7fd922d9d640 1 --2- 172.21.15.12:0/675302712 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd90c0a3d20 0x7fd90c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:59.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.700+0000 7fd9137fe640 1 -- 172.21.15.12:0/675302712 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fd9140d0f50 con 0x7fd9241031c0 2024-04-22T13:27:59.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.700+0000 7fd921d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76b4b4e90 0x55a76b4b7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:59.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.700+0000 7fd921d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76b4b4e90 0x55a76b4b7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52046/0 (socket says 172.21.15.12:52046) 2024-04-22T13:27:59.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.700+0000 7fd921d9b640 1 -- 172.21.15.12:0/2520636926 learned_addr learned my addr 172.21.15.12:0/2520636926 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:59.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.700+0000 7fd92259c640 1 --2- 172.21.15.12:0/2520636926 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a76b4a7b90 0x55a76b4b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:59.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.700+0000 7fd922d9d640 1 --2- 172.21.15.12:0/2520636926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a76b4a7270 0x55a76b4a7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:59.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.700+0000 7fd921d9b640 1 -- 172.21.15.12:0/2520636926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a76b4a7270 msgr2=0x55a76b4a7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:59.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.700+0000 7fd921d9b640 1 --2- 172.21.15.12:0/2520636926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a76b4a7270 0x55a76b4a7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.700+0000 7fd921d9b640 1 -- 172.21.15.12:0/2520636926 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a76b4a7b90 msgr2=0x55a76b4b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:59.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.700+0000 7fd921d9b640 1 --2- 172.21.15.12:0/2520636926 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a76b4a7b90 0x55a76b4b4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.700+0000 7fd921d9b640 1 -- 172.21.15.12:0/2520636926 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a76b1e87d0 con 0x55a76b4b4e90 2024-04-22T13:27:59.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.700+0000 7fd922d9d640 1 --2- 172.21.15.12:0/675302712 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd90c0a3d20 0x7fd90c0a61e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd91c005e00 tx=0x7fd91c013830 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:59.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.700+0000 7fd921d9b640 1 --2- 172.21.15.12:0/2520636926 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76b4b4e90 0x55a76b4b7280 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fd918006010 tx=0x7fd918005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dcf3f5c4b555887 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:59.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.700+0000 7fd910ff9640 1 -- 172.21.15.12:0/2520636926 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd918041020 con 0x55a76b4b4e90 2024-04-22T13:27:59.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.700+0000 7fd910ff9640 1 -- 172.21.15.12:0/2520636926 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd918005040 con 0x55a76b4b4e90 2024-04-22T13:27:59.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.700+0000 7fd910ff9640 1 -- 172.21.15.12:0/2520636926 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91802cbe0 con 0x55a76b4b4e90 2024-04-22T13:27:59.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.701+0000 7fd92b7f2740 1 -- 172.21.15.12:0/2520636926 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76b4b4e90 msgr2=0x55a76b4b7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:59.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.701+0000 7fd92b7f2740 1 --2- 172.21.15.12:0/2520636926 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76b4b4e90 0x55a76b4b7280 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fd918006010 tx=0x7fd918005e60 comp rx=0 tx=0).stop 2024-04-22T13:27:59.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.701+0000 7fd92b7f2740 1 -- 172.21.15.12:0/2520636926 shutdown_connections 2024-04-22T13:27:59.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.701+0000 7fd92b7f2740 1 --2- 172.21.15.12:0/2520636926 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76b4b4e90 0x55a76b4b7280 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.701+0000 7fd92b7f2740 1 --2- 172.21.15.12:0/2520636926 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a76b4a7b90 0x55a76b4b4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.701+0000 7fd92b7f2740 1 --2- 172.21.15.12:0/2520636926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a76b4a7270 0x55a76b4a7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.701+0000 7fd92b7f2740 1 -- 172.21.15.12:0/2520636926 >> 172.21.15.12:0/2520636926 conn(0x55a76b4a1cb0 msgr2=0x55a76b4a2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:59.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.701+0000 7fd92b7f2740 1 -- 172.21.15.12:0/2520636926 shutdown_connections 2024-04-22T13:27:59.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.701+0000 7fd92b7f2740 1 -- 172.21.15.12:0/2520636926 wait complete. 2024-04-22T13:27:59.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.702+0000 7fd92b7f2740 1 -- start start 2024-04-22T13:27:59.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.702+0000 7fd92b7f2740 1 Processor -- start 2024-04-22T13:27:59.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.702+0000 7fd92b7f2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a76b4a7270 0x55a76b4b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:59.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.702+0000 7fd92b7f2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a76b4a7b90 0x55a76b4b2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:59.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.702+0000 7fd92b7f2740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76b4b4e90 0x55a76b4b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:59.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.702+0000 7fd92b7f2740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a76b4b7c20 con 0x55a76b4b4e90 2024-04-22T13:27:59.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.702+0000 7fd92b7f2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a76b3c52f0 con 0x55a76b4a7270 2024-04-22T13:27:59.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.702+0000 7fd92b7f2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a76b4b7da0 con 0x55a76b4a7b90 2024-04-22T13:27:59.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.702+0000 7fd92b7f2740 10 client.0 _ll_register_callbacks cb 0x55a76b3d2580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:59.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.702+0000 7fd922d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76b4b4e90 0x55a76b4b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:59.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.702+0000 7fd922d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76b4b4e90 0x55a76b4b2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52054/0 (socket says 172.21.15.12:52054) 2024-04-22T13:27:59.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.702+0000 7fd922d9d640 1 -- 172.21.15.12:0/682235956 learned_addr learned my addr 172.21.15.12:0/682235956 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:59.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.702+0000 7fd92259c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a76b4a7270 0x55a76b4b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:59.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.702+0000 7fd921d9b640 1 --2- 172.21.15.12:0/682235956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a76b4a7b90 0x55a76b4b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:59.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.702+0000 7fd922d9d640 1 -- 172.21.15.12:0/682235956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a76b4a7b90 msgr2=0x55a76b4b2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:59.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.702+0000 7fd922d9d640 1 --2- 172.21.15.12:0/682235956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a76b4a7b90 0x55a76b4b2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.702+0000 7fd922d9d640 1 -- 172.21.15.12:0/682235956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a76b4a7270 msgr2=0x55a76b4b1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:59.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.702+0000 7fd922d9d640 1 --2- 172.21.15.12:0/682235956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a76b4a7270 0x55a76b4b1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.703+0000 7fd922d9d640 1 -- 172.21.15.12:0/682235956 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a76b11e3f0 con 0x55a76b4b4e90 2024-04-22T13:27:59.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.703+0000 7fd921d9b640 1 --2- 172.21.15.12:0/682235956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a76b4a7b90 0x55a76b4b2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:59.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.703+0000 7fd92259c640 1 --2- 172.21.15.12:0/682235956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a76b4a7270 0x55a76b4b1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:27:59.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.703+0000 7fd922d9d640 1 --2- 172.21.15.12:0/682235956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76b4b4e90 0x55a76b4b2870 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fd91c011bb0 tx=0x7fd91c005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137932c2a6028959 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:59.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.703+0000 7fd909ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:27:59.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.703+0000 7fd909ffb640 1 -- 172.21.15.12:0/682235956 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c023030 con 0x55a76b4b4e90 2024-04-22T13:27:59.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.703+0000 7fd909ffb640 1 -- 172.21.15.12:0/682235956 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fd91c00fad0 con 0x55a76b4b4e90 2024-04-22T13:27:59.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.703+0000 7fd92b7f2740 1 -- 172.21.15.12:0/682235956 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd918005040 con 0x55a76b4b4e90 2024-04-22T13:27:59.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.703+0000 7fd909ffb640 1 -- 172.21.15.12:0/682235956 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd91c02c020 con 0x55a76b4b4e90 2024-04-22T13:27:59.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.703+0000 7fd92b7f2740 10 client.17037 fetch_fsmap learned FSMap version 28 2024-04-22T13:27:59.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.703+0000 7fd92b7f2740 1 -- 172.21.15.12:0/682235956 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a76b0de0f0 con 0x55a76b4b4e90 2024-04-22T13:27:59.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.703+0000 7fd909ffb640 1 -- 172.21.15.12:0/682235956 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd918005040 con 0x55a76b4b4e90 2024-04-22T13:27:59.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.703+0000 7fd909ffb640 1 -- 172.21.15.12:0/682235956 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd91c02a4d0 con 0x55a76b4b4e90 2024-04-22T13:27:59.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.703+0000 7fd92b7f2740 10 client.17037 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:27:59.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.704+0000 7fd92b7f2740 10 client.17037 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:27:59.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.704+0000 7fd92b7f2740 20 client.17037 populate_metadata read hostname 'smithi012' 2024-04-22T13:27:59.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.704+0000 7fd92b7f2740 1 --2- 172.21.15.12:0/682235956 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55a76b4c4de0 0x55a76b4c7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:59.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.704+0000 7fd92b7f2740 4 client.17037 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:27:59.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.704+0000 7fd92b7f2740 1 -- 172.21.15.12:0/682235956 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a76b419c00 con 0x55a76b4c4de0 2024-04-22T13:27:59.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.704+0000 7fd92259c640 1 --2- 172.21.15.12:0/682235956 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55a76b4c4de0 0x55a76b4c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:59.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.704+0000 7fd92259c640 1 --2- 172.21.15.12:0/682235956 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55a76b4c4de0 0x55a76b4c7260 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67e8e98b38d75602 server_cookie=dec064fcd74e1d1c in_seq=0 out_seq=0 2024-04-22T13:27:59.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.704+0000 7fd909ffb640 10 client.17037 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:27:59.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.705+0000 7fd909ffb640 1 -- 172.21.15.12:0/682235956 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a76b419c00 con 0x55a76b4c4de0 2024-04-22T13:27:59.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.705+0000 7fd909ffb640 10 client.17037 handle_command_reply: tid=0 2024-04-22T13:27:59.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.705+0000 7fd92b7f2740 1 client.17037 shutdown 2024-04-22T13:27:59.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.705+0000 7fd92b7f2740 1 -- 172.21.15.12:0/682235956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76b4b4e90 msgr2=0x55a76b4b2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:59.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.705+0000 7fd92b7f2740 1 --2- 172.21.15.12:0/682235956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76b4b4e90 0x55a76b4b2870 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fd91c011bb0 tx=0x7fd91c005ce0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.705+0000 7fd92b7f2740 1 -- 172.21.15.12:0/682235956 shutdown_connections 2024-04-22T13:27:59.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.705+0000 7fd92b7f2740 1 --2- 172.21.15.12:0/682235956 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55a76b4c4de0 0x55a76b4c7260 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.705+0000 7fd92b7f2740 1 --2- 172.21.15.12:0/682235956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76b4b4e90 0x55a76b4b2870 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.705+0000 7fd92b7f2740 1 --2- 172.21.15.12:0/682235956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a76b4a7b90 0x55a76b4b2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.705+0000 7fd92b7f2740 1 --2- 172.21.15.12:0/682235956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a76b4a7270 0x55a76b4b1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.705+0000 7fd909ffb640 0 client.17037 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:27:59.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.705+0000 7fd92b7f2740 1 -- 172.21.15.12:0/682235956 >> 172.21.15.12:0/682235956 conn(0x55a76b4a1cb0 msgr2=0x55a76b4a3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:59.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.705+0000 7fd92b7f2740 1 -- 172.21.15.12:0/682235956 shutdown_connections 2024-04-22T13:27:59.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.705+0000 7fd92b7f2740 1 -- 172.21.15.12:0/682235956 wait complete. 2024-04-22T13:27:59.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.705+0000 7fd92b7f2740 20 client.17037 trim_cache size 0 max 16384 2024-04-22T13:27:59.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.723+0000 7fd92b7f2740 1 Processor -- start 2024-04-22T13:27:59.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.723+0000 7fd92b7f2740 1 -- start start 2024-04-22T13:27:59.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.723+0000 7fd92b7f2740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76b4a7270 0x55a76b4c0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:59.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.723+0000 7fd92b7f2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a76b4a7b90 0x55a76b4a2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:59.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.723+0000 7fd92b7f2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a76b4b4e90 0x55a76b4b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:59.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.723+0000 7fd92b7f2740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd9241ab8c0 con 0x55a76b4a7270 2024-04-22T13:27:59.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.723+0000 7fd92b7f2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd9241ad400 con 0x55a76b4b4e90 2024-04-22T13:27:59.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.723+0000 7fd92b7f2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd9241abf90 con 0x55a76b4a7b90 2024-04-22T13:27:59.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.723+0000 7fd92259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76b4a7270 0x55a76b4c0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:59.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.723+0000 7fd92259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76b4a7270 0x55a76b4c0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52058/0 (socket says 172.21.15.12:52058) 2024-04-22T13:27:59.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.723+0000 7fd92259c640 1 -- 172.21.15.12:0/3892925254 learned_addr learned my addr 172.21.15.12:0/3892925254 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:59.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.724+0000 7fd921d9b640 1 --2- 172.21.15.12:0/3892925254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a76b4a7b90 0x55a76b4a2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:59.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.724+0000 7fd92259c640 1 -- 172.21.15.12:0/3892925254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a76b4a7b90 msgr2=0x55a76b4a2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:59.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.724+0000 7fd92259c640 1 --2- 172.21.15.12:0/3892925254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a76b4a7b90 0x55a76b4a2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.724+0000 7fd922d9d640 1 --2- 172.21.15.12:0/3892925254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a76b4b4e90 0x55a76b4b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:59.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.724+0000 7fd92259c640 1 -- 172.21.15.12:0/3892925254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a76b4b4e90 msgr2=0x55a76b4b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:59.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.724+0000 7fd92259c640 1 --2- 172.21.15.12:0/3892925254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a76b4b4e90 0x55a76b4b16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.724+0000 7fd92259c640 1 -- 172.21.15.12:0/3892925254 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a76b4b4cb0 con 0x55a76b4a7270 2024-04-22T13:27:59.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.724+0000 7fd921d9b640 1 --2- 172.21.15.12:0/3892925254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a76b4a7b90 0x55a76b4a2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:27:59.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.724+0000 7fd92259c640 1 --2- 172.21.15.12:0/3892925254 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76b4a7270 0x55a76b4c0b20 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fd914099860 tx=0x7fd9140d0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b9f9ed9ca8580 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:59.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.724+0000 7fd90bfff640 1 -- 172.21.15.12:0/3892925254 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9140a1290 con 0x55a76b4a7270 2024-04-22T13:27:59.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.724+0000 7fd90bfff640 1 -- 172.21.15.12:0/3892925254 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9140a5b50 con 0x55a76b4a7270 2024-04-22T13:27:59.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.724+0000 7fd90bfff640 1 -- 172.21.15.12:0/3892925254 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9140ad2d0 con 0x55a76b4a7270 2024-04-22T13:27:59.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.725+0000 7fd92b7f2740 1 -- 172.21.15.12:0/3892925254 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76b4a7270 msgr2=0x55a76b4c0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:59.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.725+0000 7fd92b7f2740 1 --2- 172.21.15.12:0/3892925254 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76b4a7270 0x55a76b4c0b20 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fd914099860 tx=0x7fd9140d0040 comp rx=0 tx=0).stop 2024-04-22T13:27:59.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.725+0000 7fd92b7f2740 1 -- 172.21.15.12:0/3892925254 shutdown_connections 2024-04-22T13:27:59.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.725+0000 7fd92b7f2740 1 --2- 172.21.15.12:0/3892925254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a76b4b4e90 0x55a76b4b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.725+0000 7fd92b7f2740 1 --2- 172.21.15.12:0/3892925254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a76b4a7b90 0x55a76b4a2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.725+0000 7fd92b7f2740 1 --2- 172.21.15.12:0/3892925254 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76b4a7270 0x55a76b4c0b20 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.725+0000 7fd92b7f2740 1 -- 172.21.15.12:0/3892925254 >> 172.21.15.12:0/3892925254 conn(0x55a76b4a1cb0 msgr2=0x55a76b4a3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:59.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.725+0000 7fd92b7f2740 1 -- 172.21.15.12:0/3892925254 shutdown_connections 2024-04-22T13:27:59.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.725+0000 7fd92b7f2740 1 -- 172.21.15.12:0/3892925254 wait complete. 2024-04-22T13:27:59.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.725+0000 7fd92b7f2740 1 -- start start 2024-04-22T13:27:59.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.725+0000 7fd92b7f2740 1 Processor -- start 2024-04-22T13:27:59.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.725+0000 7fd92b7f2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a76b4a7270 0x55a76b4bd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:59.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.725+0000 7fd92b7f2740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76b4a7b90 0x55a76b4b85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:59.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.725+0000 7fd92b7f2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a76b4b4e90 0x55a76b4b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:59.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.725+0000 7fd92b7f2740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd9241acd30 con 0x55a76b4a7b90 2024-04-22T13:27:59.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.725+0000 7fd92b7f2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd9241ac660 con 0x55a76b4b4e90 2024-04-22T13:27:59.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.725+0000 7fd92b7f2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd9241a98a0 con 0x55a76b4a7270 2024-04-22T13:27:59.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.725+0000 7fd92b7f2740 10 client.0 _ll_register_callbacks cb 0x7fd91c00c980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:27:59.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.725+0000 7fd921d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76b4a7b90 0x55a76b4b85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:59.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.725+0000 7fd921d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76b4a7b90 0x55a76b4b85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52072/0 (socket says 172.21.15.12:52072) 2024-04-22T13:27:59.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.725+0000 7fd921d9b640 1 -- 172.21.15.12:0/3212145346 learned_addr learned my addr 172.21.15.12:0/3212145346 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:59.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.726+0000 7fd922d9d640 1 --2- 172.21.15.12:0/3212145346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a76b4b4e90 0x55a76b4b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:59.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.726+0000 7fd92259c640 1 --2- 172.21.15.12:0/3212145346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a76b4a7270 0x55a76b4bd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:59.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.726+0000 7fd921d9b640 1 -- 172.21.15.12:0/3212145346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a76b4a7270 msgr2=0x55a76b4bd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:59.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.726+0000 7fd921d9b640 1 --2- 172.21.15.12:0/3212145346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a76b4a7270 0x55a76b4bd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.726+0000 7fd921d9b640 1 -- 172.21.15.12:0/3212145346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a76b4b4e90 msgr2=0x55a76b4b8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:27:59.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.726+0000 7fd921d9b640 1 --2- 172.21.15.12:0/3212145346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a76b4b4e90 0x55a76b4b8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.726+0000 7fd921d9b640 1 -- 172.21.15.12:0/3212145346 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a76b4b9130 con 0x55a76b4a7b90 2024-04-22T13:27:59.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.726+0000 7fd921d9b640 1 --2- 172.21.15.12:0/3212145346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76b4a7b90 0x55a76b4b85b0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fd918009900 tx=0x7fd9180095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3397df31a11d9107 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:59.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.726+0000 7fd909ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:27:59.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.726+0000 7fd909ffb640 1 -- 172.21.15.12:0/3212145346 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd918041040 con 0x55a76b4a7b90 2024-04-22T13:27:59.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.726+0000 7fd909ffb640 1 -- 172.21.15.12:0/3212145346 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fd91802cc90 con 0x55a76b4a7b90 2024-04-22T13:27:59.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.726+0000 7fd92b7f2740 1 -- 172.21.15.12:0/3212145346 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9140a5b50 con 0x55a76b4a7b90 2024-04-22T13:27:59.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.726+0000 7fd909ffb640 1 -- 172.21.15.12:0/3212145346 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd918054050 con 0x55a76b4a7b90 2024-04-22T13:27:59.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.726+0000 7fd92b7f2740 10 client.17043 fetch_fsmap learned FSMap version 28 2024-04-22T13:27:59.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.726+0000 7fd92b7f2740 1 -- 172.21.15.12:0/3212145346 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a76b52b1a0 con 0x55a76b4a7b90 2024-04-22T13:27:59.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.726+0000 7fd909ffb640 1 -- 172.21.15.12:0/3212145346 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9140a5b50 con 0x55a76b4a7b90 2024-04-22T13:27:59.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.727+0000 7fd909ffb640 1 -- 172.21.15.12:0/3212145346 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd918046680 con 0x55a76b4a7b90 2024-04-22T13:27:59.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.727+0000 7fd92b7f2740 10 client.17043 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:27:59.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.727+0000 7fd92b7f2740 10 client.17043 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:27:59.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.727+0000 7fd92b7f2740 20 client.17043 populate_metadata read hostname 'smithi012' 2024-04-22T13:27:59.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.727+0000 7fd92b7f2740 1 --2- 172.21.15.12:0/3212145346 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55a76b52b590 0x55a76b52d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:59.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.727+0000 7fd92b7f2740 4 client.17043 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:27:59.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.727+0000 7fd92b7f2740 1 -- 172.21.15.12:0/3212145346 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a76b413a70 con 0x55a76b52b590 2024-04-22T13:27:59.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.727+0000 7fd92259c640 1 --2- 172.21.15.12:0/3212145346 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55a76b52b590 0x55a76b52d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:59.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.727+0000 7fd909ffb640 10 client.17043 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:27:59.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.727+0000 7fd92259c640 1 --2- 172.21.15.12:0/3212145346 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55a76b52b590 0x55a76b52d980 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc79314e1f468cc6 server_cookie=79f673aa5e6f7422 in_seq=0 out_seq=0 2024-04-22T13:27:59.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.727+0000 7fd909ffb640 1 -- 172.21.15.12:0/3212145346 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a76b413a70 con 0x55a76b52b590 2024-04-22T13:27:59.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.727+0000 7fd909ffb640 10 client.17043 handle_command_reply: tid=0 2024-04-22T13:27:59.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.727+0000 7fd92b7f2740 1 client.17043 shutdown 2024-04-22T13:27:59.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.728+0000 7fd92b7f2740 1 -- 172.21.15.12:0/3212145346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76b4a7b90 msgr2=0x55a76b4b85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:59.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.728+0000 7fd92b7f2740 1 --2- 172.21.15.12:0/3212145346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76b4a7b90 0x55a76b4b85b0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fd918009900 tx=0x7fd9180095f0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.728+0000 7fd92b7f2740 1 -- 172.21.15.12:0/3212145346 shutdown_connections 2024-04-22T13:27:59.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.728+0000 7fd92b7f2740 1 --2- 172.21.15.12:0/3212145346 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55a76b52b590 0x55a76b52d980 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.728+0000 7fd92b7f2740 1 --2- 172.21.15.12:0/3212145346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a76b4b4e90 0x55a76b4b8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.728+0000 7fd92b7f2740 1 --2- 172.21.15.12:0/3212145346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a76b4a7b90 0x55a76b4b85b0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.728+0000 7fd909ffb640 0 client.17043 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:27:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.728+0000 7fd92b7f2740 1 --2- 172.21.15.12:0/3212145346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a76b4a7270 0x55a76b4bd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.728+0000 7fd92b7f2740 1 -- 172.21.15.12:0/3212145346 >> 172.21.15.12:0/3212145346 conn(0x55a76b4a1cb0 msgr2=0x55a76b4a3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.728+0000 7fd92b7f2740 1 -- 172.21.15.12:0/3212145346 shutdown_connections 2024-04-22T13:27:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.728+0000 7fd92b7f2740 1 -- 172.21.15.12:0/3212145346 wait complete. 2024-04-22T13:27:59.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.728+0000 7fd92b7f2740 20 client.17043 trim_cache size 0 max 16384 2024-04-22T13:27:59.734 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:27:59.734 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-04-22T13:27:59.734 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-04-22T13:27:59.734 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:27:59.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.728+0000 7fd928b24640 1 -- 172.21.15.12:0/675302712 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd90c0a3d20 msgr2=0x7fd90c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:59.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.728+0000 7fd928b24640 1 --2- 172.21.15.12:0/675302712 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd90c0a3d20 0x7fd90c0a61e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd91c005e00 tx=0x7fd91c013830 comp rx=0 tx=0).stop 2024-04-22T13:27:59.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.728+0000 7fd928b24640 1 -- 172.21.15.12:0/675302712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9241031c0 msgr2=0x7fd92419e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:59.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.728+0000 7fd928b24640 1 --2- 172.21.15.12:0/675302712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9241031c0 0x7fd92419e450 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd91400ebc0 tx=0x7fd91400c6a0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.728+0000 7fd928b24640 1 -- 172.21.15.12:0/675302712 shutdown_connections 2024-04-22T13:27:59.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.728+0000 7fd928b24640 1 --2- 172.21.15.12:0/675302712 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd90c0a3d20 0x7fd90c0a61e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.728+0000 7fd928b24640 1 --2- 172.21.15.12:0/675302712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd92410c590 0x7fd924198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.728+0000 7fd928b24640 1 --2- 172.21.15.12:0/675302712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd924103ae0 0x7fd92419e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.728+0000 7fd928b24640 1 --2- 172.21.15.12:0/675302712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9241031c0 0x7fd92419e450 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.728+0000 7fd928b24640 1 -- 172.21.15.12:0/675302712 >> 172.21.15.12:0/675302712 conn(0x7fd924077f70 msgr2=0x7fd9240ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:59.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.728+0000 7fd928b24640 1 -- 172.21.15.12:0/675302712 shutdown_connections 2024-04-22T13:27:59.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.728+0000 7fd928b24640 1 -- 172.21.15.12:0/675302712 wait complete. 2024-04-22T13:27:59.736 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:27:59 smithi012 ceph-mon[28265]: pgmap v119: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:27:59.737 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:27:59.738 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:27:59.738 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:27:59 smithi111 ceph-mon[31361]: pgmap v119: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:27:59.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.844+0000 7f49c3396640 1 -- 172.21.15.12:0/3059505614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49bc0690e0 msgr2=0x7f49bc069540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:59.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.844+0000 7f49c3396640 1 --2- 172.21.15.12:0/3059505614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49bc0690e0 0x7f49bc069540 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f49ac009900 tx=0x7f49ac02bc30 comp rx=0 tx=0).stop 2024-04-22T13:27:59.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.844+0000 7f49c3396640 1 -- 172.21.15.12:0/3059505614 shutdown_connections 2024-04-22T13:27:59.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.844+0000 7f49c3396640 1 --2- 172.21.15.12:0/3059505614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49bc069a80 0x7f49bc10eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.844+0000 7f49c3396640 1 --2- 172.21.15.12:0/3059505614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49bc0690e0 0x7f49bc069540 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.844+0000 7f49c3396640 1 --2- 172.21.15.12:0/3059505614 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49bc107fe0 0x7f49bc1083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.844+0000 7f49c3396640 1 -- 172.21.15.12:0/3059505614 >> 172.21.15.12:0/3059505614 conn(0x7f49bc077e30 msgr2=0x7f49bc078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:27:59.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.844+0000 7f49c3396640 1 -- 172.21.15.12:0/3059505614 shutdown_connections 2024-04-22T13:27:59.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.844+0000 7f49c3396640 1 -- 172.21.15.12:0/3059505614 wait complete. 2024-04-22T13:27:59.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.844+0000 7f49c3396640 1 Processor -- start 2024-04-22T13:27:59.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.844+0000 7f49c3396640 1 -- start start 2024-04-22T13:27:59.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.845+0000 7f49c3396640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49bc0690e0 0x7f49bc1a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:59.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.845+0000 7f49c3396640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49bc069a80 0x7f49bc1a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:59.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.845+0000 7f49c3396640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49bc107fe0 0x7f49bc19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:59.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.845+0000 7f49c3396640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc112140 con 0x7f49bc069a80 2024-04-22T13:27:59.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.845+0000 7f49c3396640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc111fc0 con 0x7f49bc0690e0 2024-04-22T13:27:59.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.845+0000 7f49c3396640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc1122c0 con 0x7f49bc107fe0 2024-04-22T13:27:59.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.845+0000 7f49c0931640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49bc069a80 0x7f49bc1a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:59.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.845+0000 7f49c0931640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49bc069a80 0x7f49bc1a0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52094/0 (socket says 172.21.15.12:52094) 2024-04-22T13:27:59.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.845+0000 7f49c0931640 1 -- 172.21.15.12:0/1482517559 learned_addr learned my addr 172.21.15.12:0/1482517559 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:27:59.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.845+0000 7f49c1933640 1 --2- 172.21.15.12:0/1482517559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49bc107fe0 0x7f49bc19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:59.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.845+0000 7f49c1132640 1 --2- 172.21.15.12:0/1482517559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49bc0690e0 0x7f49bc1a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:59.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.845+0000 7f49c0931640 1 -- 172.21.15.12:0/1482517559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49bc107fe0 msgr2=0x7f49bc19a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:59.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.845+0000 7f49c0931640 1 --2- 172.21.15.12:0/1482517559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49bc107fe0 0x7f49bc19a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.845+0000 7f49c0931640 1 -- 172.21.15.12:0/1482517559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49bc0690e0 msgr2=0x7f49bc1a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:27:59.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.845+0000 7f49c0931640 1 --2- 172.21.15.12:0/1482517559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49bc0690e0 0x7f49bc1a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:27:59.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.845+0000 7f49c0931640 1 -- 172.21.15.12:0/1482517559 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49bc19af60 con 0x7f49bc069a80 2024-04-22T13:27:59.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.845+0000 7f49c1132640 1 --2- 172.21.15.12:0/1482517559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49bc0690e0 0x7f49bc1a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:27:59.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.845+0000 7f49c0931640 1 --2- 172.21.15.12:0/1482517559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49bc069a80 0x7f49bc1a0b20 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f49ac006010 tx=0x7f49ac005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:59.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.845+0000 7f49b27fc640 1 -- 172.21.15.12:0/1482517559 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac043020 con 0x7f49bc069a80 2024-04-22T13:27:59.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.845+0000 7f49b27fc640 1 -- 172.21.15.12:0/1482517559 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49ac0043d0 con 0x7f49bc069a80 2024-04-22T13:27:59.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.845+0000 7f49c3396640 1 -- 172.21.15.12:0/1482517559 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49bc19b1f0 con 0x7f49bc069a80 2024-04-22T13:27:59.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.845+0000 7f49b27fc640 1 -- 172.21.15.12:0/1482517559 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac003b20 con 0x7f49bc069a80 2024-04-22T13:27:59.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.846+0000 7f49c3396640 1 -- 172.21.15.12:0/1482517559 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49bc1a7560 con 0x7f49bc069a80 2024-04-22T13:27:59.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.846+0000 7f49c3396640 1 -- 172.21.15.12:0/1482517559 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49bc19b560 con 0x7f49bc069a80 2024-04-22T13:27:59.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.848+0000 7f49b27fc640 1 -- 172.21.15.12:0/1482517559 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f49ac046020 con 0x7f49bc069a80 2024-04-22T13:27:59.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.848+0000 7f49b27fc640 1 --2- 172.21.15.12:0/1482517559 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f49a80a3d20 0x7f49a80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:27:59.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.848+0000 7f49c1132640 1 --2- 172.21.15.12:0/1482517559 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f49a80a3d20 0x7f49a80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:27:59.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.848+0000 7f49b27fc640 1 -- 172.21.15.12:0/1482517559 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f49ac0f2210 con 0x7f49bc069a80 2024-04-22T13:27:59.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.849+0000 7f49c1132640 1 --2- 172.21.15.12:0/1482517559 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f49a80a3d20 0x7f49a80a61e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f49b8009ba0 tx=0x7f49b8005f90 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:27:59.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:27:59.853+0000 7f49b27fc640 1 -- 172.21.15.12:0/1482517559 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f49ac04d070 con 0x7f49bc069a80 2024-04-22T13:28:00.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.081+0000 7f49c3396640 1 -- 172.21.15.12:0/1482517559 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49bc1a7730 con 0x7f49bc069a80 2024-04-22T13:28:00.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.082+0000 7f49b27fc640 1 -- 172.21.15.12:0/1482517559 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7f49ac0bea10 con 0x7f49bc069a80 2024-04-22T13:28:00.079 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:28:00.079 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:28:00.079 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:28:00.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.085+0000 7f49c3396640 1 -- 172.21.15.12:0/1482517559 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f49a80a3d20 msgr2=0x7f49a80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.085+0000 7f49c3396640 1 --2- 172.21.15.12:0/1482517559 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f49a80a3d20 0x7f49a80a61e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f49b8009ba0 tx=0x7f49b8005f90 comp rx=0 tx=0).stop 2024-04-22T13:28:00.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.085+0000 7f49c3396640 1 -- 172.21.15.12:0/1482517559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49bc069a80 msgr2=0x7f49bc1a0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.085+0000 7f49c3396640 1 --2- 172.21.15.12:0/1482517559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49bc069a80 0x7f49bc1a0b20 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f49ac006010 tx=0x7f49ac005e20 comp rx=0 tx=0).stop 2024-04-22T13:28:00.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.085+0000 7f49c3396640 1 -- 172.21.15.12:0/1482517559 shutdown_connections 2024-04-22T13:28:00.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.085+0000 7f49c3396640 1 --2- 172.21.15.12:0/1482517559 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f49a80a3d20 0x7f49a80a61e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.085+0000 7f49c3396640 1 --2- 172.21.15.12:0/1482517559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49bc107fe0 0x7f49bc19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.085+0000 7f49c3396640 1 --2- 172.21.15.12:0/1482517559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49bc069a80 0x7f49bc1a0b20 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.085+0000 7f49c3396640 1 --2- 172.21.15.12:0/1482517559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49bc0690e0 0x7f49bc1a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.085+0000 7f49c3396640 1 -- 172.21.15.12:0/1482517559 >> 172.21.15.12:0/1482517559 conn(0x7f49bc077e30 msgr2=0x7f49bc10d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:00.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.085+0000 7f49c3396640 1 -- 172.21.15.12:0/1482517559 shutdown_connections 2024-04-22T13:28:00.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.085+0000 7f49c3396640 1 -- 172.21.15.12:0/1482517559 wait complete. 2024-04-22T13:28:00.093 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:28:00.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.201+0000 7f8331b04640 1 -- 172.21.15.12:0/2132836233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f832c101090 msgr2=0x7f832c101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.201+0000 7f8331b04640 1 --2- 172.21.15.12:0/2132836233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f832c101090 0x7f832c101470 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f831c009900 tx=0x7f831c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:00.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.201+0000 7f8331b04640 1 -- 172.21.15.12:0/2132836233 shutdown_connections 2024-04-22T13:28:00.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.201+0000 7f8331b04640 1 --2- 172.21.15.12:0/2132836233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f832c106160 0x7f832c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.201+0000 7f8331b04640 1 --2- 172.21.15.12:0/2132836233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f832c101a40 0x7f832c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.201+0000 7f8331b04640 1 --2- 172.21.15.12:0/2132836233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f832c101090 0x7f832c101470 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.201+0000 7f8331b04640 1 -- 172.21.15.12:0/2132836233 >> 172.21.15.12:0/2132836233 conn(0x7f832c077f50 msgr2=0x7f832c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:00.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.201+0000 7f8331b04640 1 -- 172.21.15.12:0/2132836233 shutdown_connections 2024-04-22T13:28:00.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.201+0000 7f8331b04640 1 -- 172.21.15.12:0/2132836233 wait complete. 2024-04-22T13:28:00.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.201+0000 7f8331b04640 1 Processor -- start 2024-04-22T13:28:00.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.201+0000 7f8331b04640 1 -- start start 2024-04-22T13:28:00.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.202+0000 7f8331b04640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f832c101090 0x7f832c19e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.202+0000 7f8331b04640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f832c106160 0x7f832c19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.202+0000 7f8331b04640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f832c1985b0 0x7f832c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.202+0000 7f8331b04640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f832c1110d0 con 0x7f832c1985b0 2024-04-22T13:28:00.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.202+0000 7f8331b04640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f832c110f50 con 0x7f832c106160 2024-04-22T13:28:00.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.202+0000 7f8331b04640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f832c111250 con 0x7f832c101090 2024-04-22T13:28:00.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.202+0000 7f832bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f832c1985b0 0x7f832c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.202+0000 7f832bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f832c1985b0 0x7f832c198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52102/0 (socket says 172.21.15.12:52102) 2024-04-22T13:28:00.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.202+0000 7f832bfff640 1 -- 172.21.15.12:0/1538627181 learned_addr learned my addr 172.21.15.12:0/1538627181 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:00.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.202+0000 7f832b7fe640 1 --2- 172.21.15.12:0/1538627181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f832c101090 0x7f832c19e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.202+0000 7f832affd640 1 --2- 172.21.15.12:0/1538627181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f832c106160 0x7f832c19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.202+0000 7f832bfff640 1 -- 172.21.15.12:0/1538627181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f832c101090 msgr2=0x7f832c19e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.202+0000 7f832bfff640 1 --2- 172.21.15.12:0/1538627181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f832c101090 0x7f832c19e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.202+0000 7f832bfff640 1 -- 172.21.15.12:0/1538627181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f832c106160 msgr2=0x7f832c19e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.202+0000 7f832bfff640 1 --2- 172.21.15.12:0/1538627181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f832c106160 0x7f832c19e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.202+0000 7f832bfff640 1 -- 172.21.15.12:0/1538627181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832c1990d0 con 0x7f832c1985b0 2024-04-22T13:28:00.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.202+0000 7f832bfff640 1 --2- 172.21.15.12:0/1538627181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f832c1985b0 0x7f832c198a60 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f832400eaa0 tx=0x7f832400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:00.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.202+0000 7f8328ff9640 1 -- 172.21.15.12:0/1538627181 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832400cdb0 con 0x7f832c1985b0 2024-04-22T13:28:00.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.202+0000 7f8331b04640 1 -- 172.21.15.12:0/1538627181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f832c1993c0 con 0x7f832c1985b0 2024-04-22T13:28:00.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.202+0000 7f8328ff9640 1 -- 172.21.15.12:0/1538627181 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8324005020 con 0x7f832c1985b0 2024-04-22T13:28:00.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.202+0000 7f8328ff9640 1 -- 172.21.15.12:0/1538627181 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83240116d0 con 0x7f832c1985b0 2024-04-22T13:28:00.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.203+0000 7f8331b04640 1 -- 172.21.15.12:0/1538627181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f832c1a5610 con 0x7f832c1985b0 2024-04-22T13:28:00.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.205+0000 7f8328ff9640 1 -- 172.21.15.12:0/1538627181 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f83240054a0 con 0x7f832c1985b0 2024-04-22T13:28:00.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.205+0000 7f83347d4740 1 Processor -- start 2024-04-22T13:28:00.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.205+0000 7f83347d4740 1 -- start start 2024-04-22T13:28:00.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.205+0000 7f83347d4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc8b74d0 0x55c5cc8b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.205+0000 7f83347d4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5cc8b7e80 0x55c5cc8c0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.205+0000 7f83347d4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5cc8c0a00 0x55c5cc8c2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.205+0000 7f83347d4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c5cc81c1f0 con 0x55c5cc8c0a00 2024-04-22T13:28:00.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.205+0000 7f83347d4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c5cc7b94b0 con 0x55c5cc8b7e80 2024-04-22T13:28:00.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.205+0000 7f83347d4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c5cc8181b0 con 0x55c5cc8b74d0 2024-04-22T13:28:00.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.205+0000 7f832affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5cc8c0a00 0x55c5cc8c2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.205+0000 7f832affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5cc8c0a00 0x55c5cc8c2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52108/0 (socket says 172.21.15.12:52108) 2024-04-22T13:28:00.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.205+0000 7f832affd640 1 -- 172.21.15.12:0/1063338103 learned_addr learned my addr 172.21.15.12:0/1063338103 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:00.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.205+0000 7f832affd640 1 --2- 172.21.15.12:0/1063338103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc8b74d0 0x55c5cc8b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.205+0000 7f8328ff9640 1 --2- 172.21.15.12:0/1538627181 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f83140a3d20 0x7f83140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.205+0000 7f832b7fe640 1 --2- 172.21.15.12:0/1063338103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5cc8b7e80 0x55c5cc8c0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.206+0000 7f832affd640 1 -- 172.21.15.12:0/1063338103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc8b74d0 msgr2=0x55c5cc8b78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.206+0000 7f832affd640 1 --2- 172.21.15.12:0/1063338103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc8b74d0 0x55c5cc8b78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.206+0000 7f832affd640 1 -- 172.21.15.12:0/1063338103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5cc8b7e80 msgr2=0x55c5cc8c0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.206+0000 7f832affd640 1 --2- 172.21.15.12:0/1063338103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5cc8b7e80 0x55c5cc8c0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.206+0000 7f832affd640 1 -- 172.21.15.12:0/1063338103 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5cc7ab1b0 con 0x55c5cc8c0a00 2024-04-22T13:28:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.206+0000 7f832b7fe640 1 --2- 172.21.15.12:0/1063338103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5cc8b7e80 0x55c5cc8c0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.206+0000 7f832bfff640 1 --2- 172.21.15.12:0/1538627181 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f83140a3d20 0x7f83140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.206+0000 7f8328ff9640 1 -- 172.21.15.12:0/1538627181 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f83240d0830 con 0x7f832c1985b0 2024-04-22T13:28:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.206+0000 7f832affd640 1 --2- 172.21.15.12:0/1063338103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5cc8c0a00 0x55c5cc8c2e50 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f8320005d00 tx=0x7f8320005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8744c247c94d1576 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.206+0000 7f8311ffb640 1 -- 172.21.15.12:0/1063338103 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320012ca0 con 0x55c5cc8c0a00 2024-04-22T13:28:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.206+0000 7f8311ffb640 1 -- 172.21.15.12:0/1063338103 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8320012e40 con 0x55c5cc8c0a00 2024-04-22T13:28:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.206+0000 7f8311ffb640 1 -- 172.21.15.12:0/1063338103 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83200049e0 con 0x55c5cc8c0a00 2024-04-22T13:28:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.207+0000 7f832bfff640 1 --2- 172.21.15.12:0/1538627181 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f83140a3d20 0x7f83140a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f83240ce080 tx=0x7f8324087040 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.207+0000 7f83347d4740 1 -- 172.21.15.12:0/1063338103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5cc8c0a00 msgr2=0x55c5cc8c2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.207+0000 7f83347d4740 1 --2- 172.21.15.12:0/1063338103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5cc8c0a00 0x55c5cc8c2e50 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f8320005d00 tx=0x7f8320005c90 comp rx=0 tx=0).stop 2024-04-22T13:28:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.207+0000 7f83347d4740 1 -- 172.21.15.12:0/1063338103 shutdown_connections 2024-04-22T13:28:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.207+0000 7f83347d4740 1 --2- 172.21.15.12:0/1063338103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5cc8c0a00 0x55c5cc8c2e50 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.207+0000 7f83347d4740 1 --2- 172.21.15.12:0/1063338103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5cc8b7e80 0x55c5cc8c0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.207+0000 7f83347d4740 1 --2- 172.21.15.12:0/1063338103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc8b74d0 0x55c5cc8b78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.207+0000 7f83347d4740 1 -- 172.21.15.12:0/1063338103 >> 172.21.15.12:0/1063338103 conn(0x55c5cc8b1ba0 msgr2=0x55c5cc8b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.207+0000 7f83347d4740 1 -- 172.21.15.12:0/1063338103 shutdown_connections 2024-04-22T13:28:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.207+0000 7f83347d4740 1 -- 172.21.15.12:0/1063338103 wait complete. 2024-04-22T13:28:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.207+0000 7f83347d4740 1 -- start start 2024-04-22T13:28:00.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.207+0000 7f83347d4740 1 Processor -- start 2024-04-22T13:28:00.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.208+0000 7f83347d4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc8b74d0 0x55c5cc8c0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.208+0000 7f83347d4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5cc8b7e80 0x55c5cc8bd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.208+0000 7f83347d4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5cc8c0a00 0x55c5cc8bdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.208+0000 7f83347d4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c5cc8c3970 con 0x55c5cc8b7e80 2024-04-22T13:28:00.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.208+0000 7f83347d4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c5cc7d5500 con 0x55c5cc8c0a00 2024-04-22T13:28:00.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.208+0000 7f83347d4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c5cc8c3af0 con 0x55c5cc8b74d0 2024-04-22T13:28:00.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.208+0000 7f83347d4740 10 client.0 _ll_register_callbacks cb 0x55c5cc7a3b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:00.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.208+0000 7f832b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5cc8b7e80 0x55c5cc8bd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.208+0000 7f832b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5cc8b7e80 0x55c5cc8bd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52110/0 (socket says 172.21.15.12:52110) 2024-04-22T13:28:00.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.208+0000 7f832b7fe640 1 -- 172.21.15.12:0/2879981554 learned_addr learned my addr 172.21.15.12:0/2879981554 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:00.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.208+0000 7f832affd640 1 --2- 172.21.15.12:0/2879981554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5cc8c0a00 0x55c5cc8bdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.208+0000 7f832affd640 1 --2- 172.21.15.12:0/2879981554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc8b74d0 0x55c5cc8c0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.208+0000 7f832b7fe640 1 -- 172.21.15.12:0/2879981554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc8b74d0 msgr2=0x55c5cc8c0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.208+0000 7f832b7fe640 1 --2- 172.21.15.12:0/2879981554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc8b74d0 0x55c5cc8c0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.208+0000 7f832b7fe640 1 -- 172.21.15.12:0/2879981554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5cc8c0a00 msgr2=0x55c5cc8bdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.208+0000 7f832b7fe640 1 --2- 172.21.15.12:0/2879981554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5cc8c0a00 0x55c5cc8bdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.208+0000 7f832b7fe640 1 -- 172.21.15.12:0/2879981554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5cc52e3f0 con 0x55c5cc8b7e80 2024-04-22T13:28:00.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.208+0000 7f832b7fe640 1 --2- 172.21.15.12:0/2879981554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5cc8b7e80 0x55c5cc8bd6c0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f831c009900 tx=0x7f831c02c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0021bca34d7cc3e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:00.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.208+0000 7f830affd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:00.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.208+0000 7f830affd640 1 -- 172.21.15.12:0/2879981554 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c02c610 con 0x55c5cc8b7e80 2024-04-22T13:28:00.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.208+0000 7f830affd640 1 -- 172.21.15.12:0/2879981554 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f831c005030 con 0x55c5cc8b7e80 2024-04-22T13:28:00.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.208+0000 7f83347d4740 1 -- 172.21.15.12:0/2879981554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c5cc7bb930 con 0x55c5cc8b7e80 2024-04-22T13:28:00.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.209+0000 7f830affd640 1 -- 172.21.15.12:0/2879981554 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f831c005230 con 0x55c5cc8b7e80 2024-04-22T13:28:00.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.209+0000 7f83347d4740 10 client.17061 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:00.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.209+0000 7f83347d4740 1 -- 172.21.15.12:0/2879981554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c5cc5f87d0 con 0x55c5cc8b7e80 2024-04-22T13:28:00.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.209+0000 7f830affd640 1 -- 172.21.15.12:0/2879981554 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c5cc7bb930 con 0x55c5cc8b7e80 2024-04-22T13:28:00.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.209+0000 7f830affd640 1 -- 172.21.15.12:0/2879981554 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f831c003710 con 0x55c5cc8b7e80 2024-04-22T13:28:00.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.209+0000 7f83347d4740 10 client.17061 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:00.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.209+0000 7f83347d4740 10 client.17061 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:28:00.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.209+0000 7f83347d4740 20 client.17061 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:00.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.209+0000 7f83347d4740 1 --2- 172.21.15.12:0/2879981554 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55c5cc8d0ba0 0x55c5cc8d2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.209+0000 7f83347d4740 4 client.17061 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:00.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.209+0000 7f83347d4740 1 -- 172.21.15.12:0/2879981554 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5cc829c10 con 0x55c5cc8d0ba0 2024-04-22T13:28:00.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.209+0000 7f832affd640 1 --2- 172.21.15.12:0/2879981554 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55c5cc8d0ba0 0x55c5cc8d2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.210+0000 7f832affd640 1 --2- 172.21.15.12:0/2879981554 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55c5cc8d0ba0 0x55c5cc8d2f90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=45a5c240f452b7d4 server_cookie=c7d03e50d2189237 in_seq=0 out_seq=0 2024-04-22T13:28:00.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.210+0000 7f830affd640 10 client.17061 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:28:00.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.210+0000 7f830affd640 1 -- 172.21.15.12:0/2879981554 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c5cc829c10 con 0x55c5cc8d0ba0 2024-04-22T13:28:00.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.210+0000 7f830affd640 10 client.17061 handle_command_reply: tid=0 2024-04-22T13:28:00.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.210+0000 7f83347d4740 1 client.17061 shutdown 2024-04-22T13:28:00.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.211+0000 7f83347d4740 1 -- 172.21.15.12:0/2879981554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5cc8b7e80 msgr2=0x55c5cc8bd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.211+0000 7f83347d4740 1 --2- 172.21.15.12:0/2879981554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5cc8b7e80 0x55c5cc8bd6c0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f831c009900 tx=0x7f831c02c9f0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.211+0000 7f83347d4740 1 -- 172.21.15.12:0/2879981554 shutdown_connections 2024-04-22T13:28:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.211+0000 7f83347d4740 1 --2- 172.21.15.12:0/2879981554 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55c5cc8d0ba0 0x55c5cc8d2f90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.211+0000 7f83347d4740 1 --2- 172.21.15.12:0/2879981554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5cc8c0a00 0x55c5cc8bdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.211+0000 7f83347d4740 1 --2- 172.21.15.12:0/2879981554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5cc8b7e80 0x55c5cc8bd6c0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.211+0000 7f830affd640 0 client.17061 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:28:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.211+0000 7f83347d4740 1 --2- 172.21.15.12:0/2879981554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc8b74d0 0x55c5cc8c0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.211+0000 7f83347d4740 1 -- 172.21.15.12:0/2879981554 >> 172.21.15.12:0/2879981554 conn(0x55c5cc8b1ba0 msgr2=0x55c5cc8b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.211+0000 7f83347d4740 1 -- 172.21.15.12:0/2879981554 shutdown_connections 2024-04-22T13:28:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.211+0000 7f83347d4740 1 -- 172.21.15.12:0/2879981554 wait complete. 2024-04-22T13:28:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.211+0000 7f83347d4740 20 client.17061 trim_cache size 0 max 16384 2024-04-22T13:28:00.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.225+0000 7f83347d4740 1 Processor -- start 2024-04-22T13:28:00.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.225+0000 7f83347d4740 1 -- start start 2024-04-22T13:28:00.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.225+0000 7f83347d4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5cc8b74d0 0x55c5cc8cc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.225+0000 7f83347d4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5cc8b7e80 0x55c5cc8b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.225+0000 7f83347d4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc8c0a00 0x55c5cc8bfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.225+0000 7f83347d4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f832c1adb70 con 0x55c5cc8b74d0 2024-04-22T13:28:00.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.225+0000 7f83347d4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f832c1af6b0 con 0x55c5cc8b7e80 2024-04-22T13:28:00.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.225+0000 7f83347d4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f832c1ae240 con 0x55c5cc8c0a00 2024-04-22T13:28:00.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.225+0000 7f832affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5cc8b74d0 0x55c5cc8cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.225+0000 7f832affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5cc8b74d0 0x55c5cc8cc830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52112/0 (socket says 172.21.15.12:52112) 2024-04-22T13:28:00.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.225+0000 7f832affd640 1 -- 172.21.15.12:0/3702872430 learned_addr learned my addr 172.21.15.12:0/3702872430 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:00.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.225+0000 7f832b7fe640 1 --2- 172.21.15.12:0/3702872430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5cc8b7e80 0x55c5cc8b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.225+0000 7f832affd640 1 --2- 172.21.15.12:0/3702872430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc8c0a00 0x55c5cc8bfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.225+0000 7f832affd640 1 -- 172.21.15.12:0/3702872430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc8c0a00 msgr2=0x55c5cc8bfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.225+0000 7f832affd640 1 --2- 172.21.15.12:0/3702872430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc8c0a00 0x55c5cc8bfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.225+0000 7f832affd640 1 -- 172.21.15.12:0/3702872430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5cc8b7e80 msgr2=0x55c5cc8b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.225+0000 7f832affd640 1 --2- 172.21.15.12:0/3702872430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5cc8b7e80 0x55c5cc8b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.225+0000 7f832affd640 1 -- 172.21.15.12:0/3702872430 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5cc8be040 con 0x55c5cc8b74d0 2024-04-22T13:28:00.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.225+0000 7f832b7fe640 1 --2- 172.21.15.12:0/3702872430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5cc8b7e80 0x55c5cc8b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:00.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.226+0000 7f832affd640 1 --2- 172.21.15.12:0/3702872430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5cc8b74d0 0x55c5cc8cc830 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f8320012720 tx=0x7f8320014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=658217f9e9e14afc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:00.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.226+0000 7f83117fa640 1 -- 172.21.15.12:0/3702872430 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320030070 con 0x55c5cc8b74d0 2024-04-22T13:28:00.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.226+0000 7f83117fa640 1 -- 172.21.15.12:0/3702872430 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8320016070 con 0x55c5cc8b74d0 2024-04-22T13:28:00.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.226+0000 7f83117fa640 1 -- 172.21.15.12:0/3702872430 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320010bd0 con 0x55c5cc8b74d0 2024-04-22T13:28:00.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.226+0000 7f83347d4740 1 -- 172.21.15.12:0/3702872430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5cc8b74d0 msgr2=0x55c5cc8cc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.226+0000 7f83347d4740 1 --2- 172.21.15.12:0/3702872430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5cc8b74d0 0x55c5cc8cc830 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f8320012720 tx=0x7f8320014040 comp rx=0 tx=0).stop 2024-04-22T13:28:00.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.226+0000 7f83347d4740 1 -- 172.21.15.12:0/3702872430 shutdown_connections 2024-04-22T13:28:00.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.226+0000 7f83347d4740 1 --2- 172.21.15.12:0/3702872430 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc8c0a00 0x55c5cc8bfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.226+0000 7f83347d4740 1 --2- 172.21.15.12:0/3702872430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5cc8b7e80 0x55c5cc8b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.226+0000 7f83347d4740 1 --2- 172.21.15.12:0/3702872430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5cc8b74d0 0x55c5cc8cc830 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.226+0000 7f83347d4740 1 -- 172.21.15.12:0/3702872430 >> 172.21.15.12:0/3702872430 conn(0x55c5cc8b1ba0 msgr2=0x55c5cc8bc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:00.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.226+0000 7f83347d4740 1 -- 172.21.15.12:0/3702872430 shutdown_connections 2024-04-22T13:28:00.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.226+0000 7f83347d4740 1 -- 172.21.15.12:0/3702872430 wait complete. 2024-04-22T13:28:00.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.227+0000 7f83347d4740 1 -- start start 2024-04-22T13:28:00.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.227+0000 7f83347d4740 1 Processor -- start 2024-04-22T13:28:00.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.227+0000 7f83347d4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5cc8b74d0 0x55c5cc92f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.227+0000 7f83347d4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc8b7e80 0x55c5cc92f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.227+0000 7f83347d4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5cc8c0a00 0x55c5cc8c4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.227+0000 7f83347d4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f832c1aefe0 con 0x55c5cc8c0a00 2024-04-22T13:28:00.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.227+0000 7f83347d4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f832c1ae910 con 0x55c5cc8b74d0 2024-04-22T13:28:00.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.227+0000 7f83347d4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f832c1abb70 con 0x55c5cc8b7e80 2024-04-22T13:28:00.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.227+0000 7f83347d4740 10 client.0 _ll_register_callbacks cb 0x7f831c004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:00.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.227+0000 7f832affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5cc8c0a00 0x55c5cc8c4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.227+0000 7f832affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5cc8c0a00 0x55c5cc8c4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52122/0 (socket says 172.21.15.12:52122) 2024-04-22T13:28:00.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.227+0000 7f832affd640 1 -- 172.21.15.12:0/2589000016 learned_addr learned my addr 172.21.15.12:0/2589000016 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:00.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.228+0000 7f832affd640 1 --2- 172.21.15.12:0/2589000016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5cc8b74d0 0x55c5cc92f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.228+0000 7f832b7fe640 1 --2- 172.21.15.12:0/2589000016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc8b7e80 0x55c5cc92f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.228+0000 7f832affd640 1 -- 172.21.15.12:0/2589000016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc8b7e80 msgr2=0x55c5cc92f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.228+0000 7f832affd640 1 --2- 172.21.15.12:0/2589000016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc8b7e80 0x55c5cc92f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.228+0000 7f832affd640 1 -- 172.21.15.12:0/2589000016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5cc8b74d0 msgr2=0x55c5cc92f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.228+0000 7f832affd640 1 --2- 172.21.15.12:0/2589000016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5cc8b74d0 0x55c5cc92f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.228+0000 7f832affd640 1 -- 172.21.15.12:0/2589000016 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5cc8c4800 con 0x55c5cc8c0a00 2024-04-22T13:28:00.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.228+0000 7f832affd640 1 --2- 172.21.15.12:0/2589000016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5cc8c0a00 0x55c5cc8c4110 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f832000bc70 tx=0x7f8320012520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3139b836c42558 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:00.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.228+0000 7f830affd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:00.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.228+0000 7f830affd640 1 -- 172.21.15.12:0/2589000016 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320030070 con 0x55c5cc8c0a00 2024-04-22T13:28:00.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.228+0000 7f830affd640 1 -- 172.21.15.12:0/2589000016 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f8320010d60 con 0x55c5cc8c0a00 2024-04-22T13:28:00.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.228+0000 7f83347d4740 1 -- 172.21.15.12:0/2589000016 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8320016070 con 0x55c5cc8c0a00 2024-04-22T13:28:00.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.228+0000 7f830affd640 1 -- 172.21.15.12:0/2589000016 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f83200054e0 con 0x55c5cc8c0a00 2024-04-22T13:28:00.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.229+0000 7f83347d4740 10 client.17067 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:00.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.229+0000 7f83347d4740 1 -- 172.21.15.12:0/2589000016 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c5cc936b20 con 0x55c5cc8c0a00 2024-04-22T13:28:00.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.229+0000 7f830affd640 1 -- 172.21.15.12:0/2589000016 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8320016070 con 0x55c5cc8c0a00 2024-04-22T13:28:00.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.229+0000 7f830affd640 1 -- 172.21.15.12:0/2589000016 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8320003a70 con 0x55c5cc8c0a00 2024-04-22T13:28:00.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.229+0000 7f83347d4740 10 client.17067 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:00.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.229+0000 7f83347d4740 10 client.17067 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:28:00.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.229+0000 7f83347d4740 20 client.17067 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:00.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.229+0000 7f83347d4740 1 --2- 172.21.15.12:0/2589000016 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55c5cc8d0ba0 0x55c5cc938eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.229+0000 7f83347d4740 4 client.17067 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:28:00.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.229+0000 7f83347d4740 1 -- 172.21.15.12:0/2589000016 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c5cc823a80 con 0x55c5cc8d0ba0 2024-04-22T13:28:00.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.229+0000 7f832b7fe640 1 --2- 172.21.15.12:0/2589000016 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55c5cc8d0ba0 0x55c5cc938eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.230+0000 7f830affd640 10 client.17067 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:28:00.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.230+0000 7f832b7fe640 1 --2- 172.21.15.12:0/2589000016 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55c5cc8d0ba0 0x55c5cc938eb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5a9db737e65a01bc server_cookie=7f7fb0c2b25eac65 in_seq=0 out_seq=0 2024-04-22T13:28:00.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.230+0000 7f830affd640 1 -- 172.21.15.12:0/2589000016 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c5cc823a80 con 0x55c5cc8d0ba0 2024-04-22T13:28:00.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.230+0000 7f830affd640 10 client.17067 handle_command_reply: tid=0 2024-04-22T13:28:00.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.230+0000 7f83347d4740 1 client.17067 shutdown 2024-04-22T13:28:00.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.230+0000 7f83347d4740 1 -- 172.21.15.12:0/2589000016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5cc8c0a00 msgr2=0x55c5cc8c4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.230+0000 7f83347d4740 1 --2- 172.21.15.12:0/2589000016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5cc8c0a00 0x55c5cc8c4110 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f832000bc70 tx=0x7f8320012520 comp rx=0 tx=0).stop 2024-04-22T13:28:00.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.230+0000 7f83347d4740 1 -- 172.21.15.12:0/2589000016 shutdown_connections 2024-04-22T13:28:00.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.230+0000 7f83347d4740 1 --2- 172.21.15.12:0/2589000016 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55c5cc8d0ba0 0x55c5cc938eb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.230+0000 7f83347d4740 1 --2- 172.21.15.12:0/2589000016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5cc8c0a00 0x55c5cc8c4110 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.230+0000 7f83347d4740 1 --2- 172.21.15.12:0/2589000016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5cc8b7e80 0x55c5cc92f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.230+0000 7f830affd640 0 client.17067 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:28:00.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.230+0000 7f83347d4740 1 --2- 172.21.15.12:0/2589000016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5cc8b74d0 0x55c5cc92f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.230+0000 7f83347d4740 1 -- 172.21.15.12:0/2589000016 >> 172.21.15.12:0/2589000016 conn(0x55c5cc8b1ba0 msgr2=0x55c5cc8b0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:00.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.230+0000 7f83347d4740 1 -- 172.21.15.12:0/2589000016 shutdown_connections 2024-04-22T13:28:00.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.230+0000 7f83347d4740 1 -- 172.21.15.12:0/2589000016 wait complete. 2024-04-22T13:28:00.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.230+0000 7f83347d4740 20 client.17067 trim_cache size 0 max 16384 2024-04-22T13:28:00.234 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:28:00.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.231+0000 7f8331b04640 1 -- 172.21.15.12:0/1538627181 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f83140a3d20 msgr2=0x7f83140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.231+0000 7f8331b04640 1 --2- 172.21.15.12:0/1538627181 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f83140a3d20 0x7f83140a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f83240ce080 tx=0x7f8324087040 comp rx=0 tx=0).stop 2024-04-22T13:28:00.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.231+0000 7f8331b04640 1 -- 172.21.15.12:0/1538627181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f832c1985b0 msgr2=0x7f832c198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.231+0000 7f8331b04640 1 --2- 172.21.15.12:0/1538627181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f832c1985b0 0x7f832c198a60 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f832400eaa0 tx=0x7f832400ef60 comp rx=0 tx=0).stop 2024-04-22T13:28:00.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.231+0000 7f8331b04640 1 -- 172.21.15.12:0/1538627181 shutdown_connections 2024-04-22T13:28:00.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.231+0000 7f8331b04640 1 --2- 172.21.15.12:0/1538627181 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f83140a3d20 0x7f83140a61e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.231+0000 7f8331b04640 1 --2- 172.21.15.12:0/1538627181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f832c1985b0 0x7f832c198a60 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.231+0000 7f8331b04640 1 --2- 172.21.15.12:0/1538627181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f832c106160 0x7f832c19e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.231+0000 7f8331b04640 1 --2- 172.21.15.12:0/1538627181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f832c101090 0x7f832c19e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.231+0000 7f8331b04640 1 -- 172.21.15.12:0/1538627181 >> 172.21.15.12:0/1538627181 conn(0x7f832c077f50 msgr2=0x7f832c0694e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:00.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.231+0000 7f8331b04640 1 -- 172.21.15.12:0/1538627181 shutdown_connections 2024-04-22T13:28:00.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.231+0000 7f8331b04640 1 -- 172.21.15.12:0/1538627181 wait complete. 2024-04-22T13:28:00.239 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:28:00.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.346+0000 7f2192fef640 1 -- 172.21.15.12:0/1789574230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f218c1010a0 msgr2=0x7f218c101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.346+0000 7f2192fef640 1 --2- 172.21.15.12:0/1789574230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f218c1010a0 0x7f218c101480 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f217c009900 tx=0x7f217c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:00.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.346+0000 7f2192fef640 1 -- 172.21.15.12:0/1789574230 shutdown_connections 2024-04-22T13:28:00.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.346+0000 7f2192fef640 1 --2- 172.21.15.12:0/1789574230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f218c106170 0x7f218c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.346+0000 7f2192fef640 1 --2- 172.21.15.12:0/1789574230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f218c101a50 0x7f218c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.346+0000 7f2192fef640 1 --2- 172.21.15.12:0/1789574230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f218c1010a0 0x7f218c101480 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.346+0000 7f2192fef640 1 -- 172.21.15.12:0/1789574230 >> 172.21.15.12:0/1789574230 conn(0x7f218c077eb0 msgr2=0x7f218c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:00.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.346+0000 7f2192fef640 1 -- 172.21.15.12:0/1789574230 shutdown_connections 2024-04-22T13:28:00.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.346+0000 7f2192fef640 1 -- 172.21.15.12:0/1789574230 wait complete. 2024-04-22T13:28:00.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.347+0000 7f2192fef640 1 Processor -- start 2024-04-22T13:28:00.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.347+0000 7f2192fef640 1 -- start start 2024-04-22T13:28:00.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.347+0000 7f2192fef640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f218c1010a0 0x7f218c102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.347+0000 7f2192fef640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f218c106170 0x7f218c103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.347+0000 7f2192fef640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f218c1035b0 0x7f218c1a6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.347+0000 7f2192fef640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f218c1110d0 con 0x7f218c1035b0 2024-04-22T13:28:00.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.347+0000 7f2192fef640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f218c110f50 con 0x7f218c1010a0 2024-04-22T13:28:00.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.347+0000 7f2192fef640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f218c111250 con 0x7f218c106170 2024-04-22T13:28:00.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.347+0000 7f219158c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f218c1035b0 0x7f218c1a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.347+0000 7f219158c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f218c1035b0 0x7f218c1a6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52142/0 (socket says 172.21.15.12:52142) 2024-04-22T13:28:00.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.347+0000 7f219158c640 1 -- 172.21.15.12:0/2303399523 learned_addr learned my addr 172.21.15.12:0/2303399523 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:00.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.347+0000 7f2190d8b640 1 --2- 172.21.15.12:0/2303399523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f218c1010a0 0x7f218c102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.347+0000 7f218bfff640 1 --2- 172.21.15.12:0/2303399523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f218c106170 0x7f218c103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.347+0000 7f219158c640 1 -- 172.21.15.12:0/2303399523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f218c106170 msgr2=0x7f218c103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.347+0000 7f219158c640 1 --2- 172.21.15.12:0/2303399523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f218c106170 0x7f218c103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.347+0000 7f219158c640 1 -- 172.21.15.12:0/2303399523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f218c1010a0 msgr2=0x7f218c102b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.347+0000 7f219158c640 1 --2- 172.21.15.12:0/2303399523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f218c1010a0 0x7f218c102b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.347+0000 7f219158c640 1 -- 172.21.15.12:0/2303399523 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f218c1a7030 con 0x7f218c1035b0 2024-04-22T13:28:00.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.347+0000 7f2190d8b640 1 --2- 172.21.15.12:0/2303399523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f218c1010a0 0x7f218c102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:00.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.347+0000 7f219158c640 1 --2- 172.21.15.12:0/2303399523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f218c1035b0 0x7f218c1a6930 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f218400eb10 tx=0x7f218400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:00.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.348+0000 7f2189ffb640 1 -- 172.21.15.12:0/2303399523 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2184004b20 con 0x7f218c1035b0 2024-04-22T13:28:00.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.348+0000 7f2189ffb640 1 -- 172.21.15.12:0/2303399523 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2184004cc0 con 0x7f218c1035b0 2024-04-22T13:28:00.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.348+0000 7f2189ffb640 1 -- 172.21.15.12:0/2303399523 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2184010670 con 0x7f218c1035b0 2024-04-22T13:28:00.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.348+0000 7f2192fef640 1 -- 172.21.15.12:0/2303399523 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f218c1a7320 con 0x7f218c1035b0 2024-04-22T13:28:00.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.348+0000 7f2192fef640 1 -- 172.21.15.12:0/2303399523 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f218c1a7820 con 0x7f218c1035b0 2024-04-22T13:28:00.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.350+0000 7f2189ffb640 1 -- 172.21.15.12:0/2303399523 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2184005020 con 0x7f218c1035b0 2024-04-22T13:28:00.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.350+0000 7f2195bdc740 1 Processor -- start 2024-04-22T13:28:00.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.351+0000 7f2195bdc740 1 -- start start 2024-04-22T13:28:00.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.351+0000 7f2195bdc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5a84584d0 0x55c5a84588b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.351+0000 7f2195bdc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5a8458e80 0x55c5a8461430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.351+0000 7f2195bdc740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5a8461a00 0x55c5a8463e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.351+0000 7f2195bdc740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c5a83bd1f0 con 0x55c5a8461a00 2024-04-22T13:28:00.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.351+0000 7f2195bdc740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c5a835a4b0 con 0x55c5a84584d0 2024-04-22T13:28:00.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.351+0000 7f2195bdc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c5a83b91b0 con 0x55c5a8458e80 2024-04-22T13:28:00.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.351+0000 7f218bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5a8461a00 0x55c5a8463e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.351+0000 7f218bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5a8461a00 0x55c5a8463e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52150/0 (socket says 172.21.15.12:52150) 2024-04-22T13:28:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.351+0000 7f218bfff640 1 -- 172.21.15.12:0/943815042 learned_addr learned my addr 172.21.15.12:0/943815042 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.351+0000 7f2189ffb640 1 --2- 172.21.15.12:0/2303399523 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f21780a3d50 0x7f21780a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.351+0000 7f2190d8b640 1 --2- 172.21.15.12:0/943815042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5a8458e80 0x55c5a8461430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.351+0000 7f218bfff640 1 --2- 172.21.15.12:0/943815042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5a84584d0 0x55c5a84588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.351+0000 7f219158c640 1 --2- 172.21.15.12:0/2303399523 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f21780a3d50 0x7f21780a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.351+0000 7f218bfff640 1 -- 172.21.15.12:0/943815042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5a8458e80 msgr2=0x55c5a8461430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.351+0000 7f218bfff640 1 --2- 172.21.15.12:0/943815042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5a8458e80 0x55c5a8461430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.351+0000 7f218bfff640 1 -- 172.21.15.12:0/943815042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5a84584d0 msgr2=0x55c5a84588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.351+0000 7f218bfff640 1 --2- 172.21.15.12:0/943815042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5a84584d0 0x55c5a84588b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.351+0000 7f218bfff640 1 -- 172.21.15.12:0/943815042 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5a834c1b0 con 0x55c5a8461a00 2024-04-22T13:28:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.351+0000 7f2189ffb640 1 -- 172.21.15.12:0/2303399523 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f21840d0df0 con 0x7f218c1035b0 2024-04-22T13:28:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.352+0000 7f218bfff640 1 --2- 172.21.15.12:0/943815042 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5a8461a00 0x55c5a8463e50 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f217400cfd0 tx=0x7f217400c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=572f2abdd682f4f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.352+0000 7f219158c640 1 --2- 172.21.15.12:0/2303399523 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f21780a3d50 0x7f21780a6210 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f21840ce640 tx=0x7f218406e820 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.352+0000 7f2182ffd640 1 -- 172.21.15.12:0/943815042 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2174019030 con 0x55c5a8461a00 2024-04-22T13:28:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.352+0000 7f2182ffd640 1 -- 172.21.15.12:0/943815042 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21740054d0 con 0x55c5a8461a00 2024-04-22T13:28:00.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.352+0000 7f2182ffd640 1 -- 172.21.15.12:0/943815042 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2174011520 con 0x55c5a8461a00 2024-04-22T13:28:00.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.352+0000 7f2195bdc740 1 -- 172.21.15.12:0/943815042 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5a8461a00 msgr2=0x55c5a8463e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.352+0000 7f2195bdc740 1 --2- 172.21.15.12:0/943815042 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5a8461a00 0x55c5a8463e50 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f217400cfd0 tx=0x7f217400c670 comp rx=0 tx=0).stop 2024-04-22T13:28:00.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.352+0000 7f2195bdc740 1 -- 172.21.15.12:0/943815042 shutdown_connections 2024-04-22T13:28:00.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.352+0000 7f2195bdc740 1 --2- 172.21.15.12:0/943815042 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5a8461a00 0x55c5a8463e50 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.352+0000 7f2195bdc740 1 --2- 172.21.15.12:0/943815042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5a8458e80 0x55c5a8461430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.352+0000 7f2195bdc740 1 --2- 172.21.15.12:0/943815042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5a84584d0 0x55c5a84588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.352+0000 7f2195bdc740 1 -- 172.21.15.12:0/943815042 >> 172.21.15.12:0/943815042 conn(0x55c5a8452ba0 msgr2=0x55c5a8452f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.352+0000 7f2195bdc740 1 -- 172.21.15.12:0/943815042 shutdown_connections 2024-04-22T13:28:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.352+0000 7f2195bdc740 1 -- 172.21.15.12:0/943815042 wait complete. 2024-04-22T13:28:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.353+0000 7f2195bdc740 1 -- start start 2024-04-22T13:28:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.353+0000 7f2195bdc740 1 Processor -- start 2024-04-22T13:28:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.353+0000 7f2195bdc740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5a84584d0 0x55c5a8461300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.353+0000 7f2195bdc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5a8458e80 0x55c5a845e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.353+0000 7f2195bdc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5a8461a00 0x55c5a845ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.353+0000 7f2195bdc740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c5a8464970 con 0x55c5a84584d0 2024-04-22T13:28:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.353+0000 7f2195bdc740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c5a8376500 con 0x55c5a8458e80 2024-04-22T13:28:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.353+0000 7f2195bdc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c5a8464af0 con 0x55c5a8461a00 2024-04-22T13:28:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.353+0000 7f2195bdc740 10 client.0 _ll_register_callbacks cb 0x55c5a8344b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.353+0000 7f218bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5a84584d0 0x55c5a8461300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.353+0000 7f218bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5a84584d0 0x55c5a8461300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52162/0 (socket says 172.21.15.12:52162) 2024-04-22T13:28:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.353+0000 7f218bfff640 1 -- 172.21.15.12:0/2367559756 learned_addr learned my addr 172.21.15.12:0/2367559756 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.354+0000 7f2190d8b640 1 --2- 172.21.15.12:0/2367559756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5a8458e80 0x55c5a845e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.354+0000 7f218bfff640 1 -- 172.21.15.12:0/2367559756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5a8461a00 msgr2=0x55c5a845ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.354+0000 7f218bfff640 1 --2- 172.21.15.12:0/2367559756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5a8461a00 0x55c5a845ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.354+0000 7f218bfff640 1 -- 172.21.15.12:0/2367559756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5a8458e80 msgr2=0x55c5a845e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.354+0000 7f218bfff640 1 --2- 172.21.15.12:0/2367559756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5a8458e80 0x55c5a845e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.354+0000 7f218bfff640 1 -- 172.21.15.12:0/2367559756 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5a80cf3f0 con 0x55c5a84584d0 2024-04-22T13:28:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.354+0000 7f218bfff640 1 --2- 172.21.15.12:0/2367559756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5a84584d0 0x55c5a8461300 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f21740056b0 tx=0x7f21740056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad7e28fbb599800 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.354+0000 7f216bfff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.354+0000 7f216bfff640 1 -- 172.21.15.12:0/2367559756 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2174019040 con 0x55c5a84584d0 2024-04-22T13:28:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.354+0000 7f216bfff640 1 -- 172.21.15.12:0/2367559756 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f21740048c0 con 0x55c5a84584d0 2024-04-22T13:28:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.354+0000 7f2195bdc740 1 -- 172.21.15.12:0/2367559756 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21740054d0 con 0x55c5a84584d0 2024-04-22T13:28:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.354+0000 7f216bfff640 1 -- 172.21.15.12:0/2367559756 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2174011520 con 0x55c5a84584d0 2024-04-22T13:28:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.354+0000 7f2195bdc740 10 client.17079 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.354+0000 7f2195bdc740 1 -- 172.21.15.12:0/2367559756 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c5a81997d0 con 0x55c5a84584d0 2024-04-22T13:28:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.354+0000 7f216bfff640 1 -- 172.21.15.12:0/2367559756 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21740054d0 con 0x55c5a84584d0 2024-04-22T13:28:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.355+0000 7f216bfff640 1 -- 172.21.15.12:0/2367559756 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f217401e540 con 0x55c5a84584d0 2024-04-22T13:28:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.355+0000 7f2195bdc740 10 client.17079 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.355+0000 7f2195bdc740 10 client.17079 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:28:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.355+0000 7f2195bdc740 20 client.17079 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.355+0000 7f2195bdc740 1 --2- 172.21.15.12:0/2367559756 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55c5a8471a80 0x55c5a8473e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.355+0000 7f2195bdc740 4 client.17079 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.355+0000 7f2195bdc740 1 -- 172.21.15.12:0/2367559756 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5a835c930 con 0x55c5a8471a80 2024-04-22T13:28:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.355+0000 7f2190d8b640 1 --2- 172.21.15.12:0/2367559756 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55c5a8471a80 0x55c5a8473e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.355+0000 7f216bfff640 10 client.17079 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:28:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.355+0000 7f2190d8b640 1 --2- 172.21.15.12:0/2367559756 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55c5a8471a80 0x55c5a8473e70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fd003a88c008dc41 server_cookie=6413b706045ca8a2 in_seq=0 out_seq=0 2024-04-22T13:28:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.356+0000 7f216bfff640 1 -- 172.21.15.12:0/2367559756 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c5a835c930 con 0x55c5a8471a80 2024-04-22T13:28:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.356+0000 7f216bfff640 10 client.17079 handle_command_reply: tid=0 2024-04-22T13:28:00.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.356+0000 7f2195bdc740 1 client.17079 shutdown 2024-04-22T13:28:00.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.356+0000 7f2195bdc740 1 -- 172.21.15.12:0/2367559756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5a84584d0 msgr2=0x55c5a8461300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.356+0000 7f2195bdc740 1 --2- 172.21.15.12:0/2367559756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5a84584d0 0x55c5a8461300 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f21740056b0 tx=0x7f21740056e0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.356+0000 7f2195bdc740 1 -- 172.21.15.12:0/2367559756 shutdown_connections 2024-04-22T13:28:00.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.356+0000 7f2195bdc740 1 --2- 172.21.15.12:0/2367559756 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55c5a8471a80 0x55c5a8473e70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.356+0000 7f2195bdc740 1 --2- 172.21.15.12:0/2367559756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5a8461a00 0x55c5a845ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.356+0000 7f2195bdc740 1 --2- 172.21.15.12:0/2367559756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5a8458e80 0x55c5a845e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.356+0000 7f216bfff640 0 client.17079 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:28:00.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.356+0000 7f2195bdc740 1 --2- 172.21.15.12:0/2367559756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5a84584d0 0x55c5a8461300 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.356+0000 7f2195bdc740 1 -- 172.21.15.12:0/2367559756 >> 172.21.15.12:0/2367559756 conn(0x55c5a8452ba0 msgr2=0x55c5a84548f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:00.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.356+0000 7f2195bdc740 1 -- 172.21.15.12:0/2367559756 shutdown_connections 2024-04-22T13:28:00.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.356+0000 7f2195bdc740 1 -- 172.21.15.12:0/2367559756 wait complete. 2024-04-22T13:28:00.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.356+0000 7f2195bdc740 20 client.17079 trim_cache size 0 max 16384 2024-04-22T13:28:00.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.369+0000 7f2195bdc740 1 Processor -- start 2024-04-22T13:28:00.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.369+0000 7f2195bdc740 1 -- start start 2024-04-22T13:28:00.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.370+0000 7f2195bdc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5a84584d0 0x55c5a846d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.370+0000 7f2195bdc740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5a8458e80 0x55c5a8451c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.370+0000 7f2195bdc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5a8461a00 0x55c5a8479470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.370+0000 7f2195bdc740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f218c1afd20 con 0x55c5a8458e80 2024-04-22T13:28:00.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.370+0000 7f2195bdc740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f218c1b1860 con 0x55c5a8461a00 2024-04-22T13:28:00.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.370+0000 7f2195bdc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f218c1b03f0 con 0x55c5a84584d0 2024-04-22T13:28:00.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.370+0000 7f2190d8b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5a8458e80 0x55c5a8451c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.370+0000 7f2190d8b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5a8458e80 0x55c5a8451c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52164/0 (socket says 172.21.15.12:52164) 2024-04-22T13:28:00.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.370+0000 7f2190d8b640 1 -- 172.21.15.12:0/3586082401 learned_addr learned my addr 172.21.15.12:0/3586082401 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:00.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.370+0000 7f218bfff640 1 --2- 172.21.15.12:0/3586082401 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5a8461a00 0x55c5a8479470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.370+0000 7f2190d8b640 1 -- 172.21.15.12:0/3586082401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5a84584d0 msgr2=0x55c5a846d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.370+0000 7f2190d8b640 1 --2- 172.21.15.12:0/3586082401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5a84584d0 0x55c5a846d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.370+0000 7f2190d8b640 1 -- 172.21.15.12:0/3586082401 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5a8461a00 msgr2=0x55c5a8479470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.370+0000 7f2190d8b640 1 --2- 172.21.15.12:0/3586082401 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5a8461a00 0x55c5a8479470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.370+0000 7f2190d8b640 1 -- 172.21.15.12:0/3586082401 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5a8457e90 con 0x55c5a8458e80 2024-04-22T13:28:00.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.370+0000 7f218bfff640 1 --2- 172.21.15.12:0/3586082401 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5a8461a00 0x55c5a8479470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:00.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.370+0000 7f2190d8b640 1 --2- 172.21.15.12:0/3586082401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5a8458e80 0x55c5a8451c00 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f217c000c00 tx=0x7f217c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b482795f71d5ba74 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:00.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.370+0000 7f21827fc640 1 -- 172.21.15.12:0/3586082401 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217c047050 con 0x55c5a8458e80 2024-04-22T13:28:00.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.370+0000 7f21827fc640 1 -- 172.21.15.12:0/3586082401 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f217c02cc90 con 0x55c5a8458e80 2024-04-22T13:28:00.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.370+0000 7f21827fc640 1 -- 172.21.15.12:0/3586082401 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217c0052a0 con 0x55c5a8458e80 2024-04-22T13:28:00.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.371+0000 7f2195bdc740 1 -- 172.21.15.12:0/3586082401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5a8458e80 msgr2=0x55c5a8451c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.371+0000 7f2195bdc740 1 --2- 172.21.15.12:0/3586082401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5a8458e80 0x55c5a8451c00 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f217c000c00 tx=0x7f217c005b50 comp rx=0 tx=0).stop 2024-04-22T13:28:00.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.371+0000 7f2195bdc740 1 -- 172.21.15.12:0/3586082401 shutdown_connections 2024-04-22T13:28:00.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.371+0000 7f2195bdc740 1 --2- 172.21.15.12:0/3586082401 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5a8461a00 0x55c5a8479470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.371+0000 7f2195bdc740 1 --2- 172.21.15.12:0/3586082401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5a8458e80 0x55c5a8451c00 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.371+0000 7f2195bdc740 1 --2- 172.21.15.12:0/3586082401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5a84584d0 0x55c5a846d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.371+0000 7f2195bdc740 1 -- 172.21.15.12:0/3586082401 >> 172.21.15.12:0/3586082401 conn(0x55c5a8452ba0 msgr2=0x55c5a8453dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:00.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.371+0000 7f2195bdc740 1 -- 172.21.15.12:0/3586082401 shutdown_connections 2024-04-22T13:28:00.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.371+0000 7f2195bdc740 1 -- 172.21.15.12:0/3586082401 wait complete. 2024-04-22T13:28:00.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.371+0000 7f2195bdc740 1 -- start start 2024-04-22T13:28:00.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.371+0000 7f2195bdc740 1 Processor -- start 2024-04-22T13:28:00.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.372+0000 7f2195bdc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5a84584d0 0x55c5a8469a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.372+0000 7f2195bdc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5a8458e80 0x55c5a8469f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.372+0000 7f2195bdc740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5a8461a00 0x55c5a8464ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.372+0000 7f2195bdc740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f218c1b1190 con 0x55c5a8461a00 2024-04-22T13:28:00.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.372+0000 7f2195bdc740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f218c1b0ac0 con 0x55c5a8458e80 2024-04-22T13:28:00.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.372+0000 7f2195bdc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f218c1add00 con 0x55c5a84584d0 2024-04-22T13:28:00.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.372+0000 7f2195bdc740 10 client.0 _ll_register_callbacks cb 0x7f21740053c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:00.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.372+0000 7f218bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5a8461a00 0x55c5a8464ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.372+0000 7f218bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5a8461a00 0x55c5a8464ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52172/0 (socket says 172.21.15.12:52172) 2024-04-22T13:28:00.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.372+0000 7f218bfff640 1 -- 172.21.15.12:0/1484906596 learned_addr learned my addr 172.21.15.12:0/1484906596 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:00.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.372+0000 7f2190d8b640 1 --2- 172.21.15.12:0/1484906596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5a8458e80 0x55c5a8469f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.372+0000 7f218bfff640 1 --2- 172.21.15.12:0/1484906596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5a84584d0 0x55c5a8469a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.372+0000 7f218bfff640 1 -- 172.21.15.12:0/1484906596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5a84584d0 msgr2=0x55c5a8469a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.372+0000 7f218bfff640 1 --2- 172.21.15.12:0/1484906596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5a84584d0 0x55c5a8469a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.372+0000 7f218bfff640 1 -- 172.21.15.12:0/1484906596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5a8458e80 msgr2=0x55c5a8469f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.372+0000 7f218bfff640 1 --2- 172.21.15.12:0/1484906596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5a8458e80 0x55c5a8469f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.372+0000 7f218bfff640 1 -- 172.21.15.12:0/1484906596 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5a8465650 con 0x55c5a8461a00 2024-04-22T13:28:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.372+0000 7f218bfff640 1 --2- 172.21.15.12:0/1484906596 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5a8461a00 0x55c5a8464ec0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f217400cfd0 tx=0x7f2174005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f7229192222051 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.372+0000 7f216bfff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.372+0000 7f216bfff640 1 -- 172.21.15.12:0/1484906596 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217401e660 con 0x55c5a8461a00 2024-04-22T13:28:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.373+0000 7f2195bdc740 1 -- 172.21.15.12:0/1484906596 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f217c02cc90 con 0x55c5a8461a00 2024-04-22T13:28:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.373+0000 7f216bfff640 1 -- 172.21.15.12:0/1484906596 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f217401eb80 con 0x55c5a8461a00 2024-04-22T13:28:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.373+0000 7f216bfff640 1 -- 172.21.15.12:0/1484906596 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f217401edc0 con 0x55c5a8461a00 2024-04-22T13:28:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.373+0000 7f2195bdc740 10 client.17085 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.373+0000 7f2195bdc740 1 -- 172.21.15.12:0/1484906596 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c5a84d7690 con 0x55c5a8461a00 2024-04-22T13:28:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.373+0000 7f216bfff640 1 -- 172.21.15.12:0/1484906596 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f217c02cc90 con 0x55c5a8461a00 2024-04-22T13:28:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.373+0000 7f216bfff640 1 -- 172.21.15.12:0/1484906596 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f21740116c0 con 0x55c5a8461a00 2024-04-22T13:28:00.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.373+0000 7f2195bdc740 10 client.17085 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:00.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.373+0000 7f2195bdc740 10 client.17085 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:28:00.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.373+0000 7f2195bdc740 20 client.17085 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:00.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.373+0000 7f2195bdc740 1 --2- 172.21.15.12:0/1484906596 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55c5a84d7d50 0x55c5a84da170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.373+0000 7f2195bdc740 4 client.17085 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:28:00.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.373+0000 7f2195bdc740 1 -- 172.21.15.12:0/1484906596 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c5a83cac10 con 0x55c5a84d7d50 2024-04-22T13:28:00.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.373+0000 7f2190d8b640 1 --2- 172.21.15.12:0/1484906596 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55c5a84d7d50 0x55c5a84da170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.374+0000 7f216bfff640 10 client.17085 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:28:00.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.374+0000 7f2190d8b640 1 --2- 172.21.15.12:0/1484906596 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55c5a84d7d50 0x55c5a84da170 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ed451945f1ebe5c0 server_cookie=2661c0ce1debd037 in_seq=0 out_seq=0 2024-04-22T13:28:00.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.374+0000 7f216bfff640 1 -- 172.21.15.12:0/1484906596 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c5a83cac10 con 0x55c5a84d7d50 2024-04-22T13:28:00.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.374+0000 7f216bfff640 10 client.17085 handle_command_reply: tid=0 2024-04-22T13:28:00.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.374+0000 7f2195bdc740 1 client.17085 shutdown 2024-04-22T13:28:00.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.374+0000 7f2195bdc740 1 -- 172.21.15.12:0/1484906596 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5a8461a00 msgr2=0x55c5a8464ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.374+0000 7f2195bdc740 1 --2- 172.21.15.12:0/1484906596 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5a8461a00 0x55c5a8464ec0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f217400cfd0 tx=0x7f2174005c50 comp rx=0 tx=0).stop 2024-04-22T13:28:00.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.374+0000 7f2195bdc740 1 -- 172.21.15.12:0/1484906596 shutdown_connections 2024-04-22T13:28:00.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.374+0000 7f2195bdc740 1 --2- 172.21.15.12:0/1484906596 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55c5a84d7d50 0x55c5a84da170 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.374+0000 7f2195bdc740 1 --2- 172.21.15.12:0/1484906596 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c5a8461a00 0x55c5a8464ec0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.374+0000 7f2195bdc740 1 --2- 172.21.15.12:0/1484906596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c5a8458e80 0x55c5a8469f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.374+0000 7f216bfff640 0 client.17085 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:28:00.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.374+0000 7f2195bdc740 1 --2- 172.21.15.12:0/1484906596 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c5a84584d0 0x55c5a8469a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.374+0000 7f2195bdc740 1 -- 172.21.15.12:0/1484906596 >> 172.21.15.12:0/1484906596 conn(0x55c5a8452ba0 msgr2=0x55c5a8451a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:00.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.374+0000 7f2195bdc740 1 -- 172.21.15.12:0/1484906596 shutdown_connections 2024-04-22T13:28:00.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.374+0000 7f2195bdc740 1 -- 172.21.15.12:0/1484906596 wait complete. 2024-04-22T13:28:00.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.374+0000 7f2195bdc740 20 client.17085 trim_cache size 0 max 16384 2024-04-22T13:28:00.377 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:28:00.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.375+0000 7f2192fef640 1 -- 172.21.15.12:0/2303399523 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f21780a3d50 msgr2=0x7f21780a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.375+0000 7f2192fef640 1 --2- 172.21.15.12:0/2303399523 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f21780a3d50 0x7f21780a6210 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f21840ce640 tx=0x7f218406e820 comp rx=0 tx=0).stop 2024-04-22T13:28:00.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.375+0000 7f2192fef640 1 -- 172.21.15.12:0/2303399523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f218c1035b0 msgr2=0x7f218c1a6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.375+0000 7f2192fef640 1 --2- 172.21.15.12:0/2303399523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f218c1035b0 0x7f218c1a6930 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f218400eb10 tx=0x7f218400c2d0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.375+0000 7f2192fef640 1 -- 172.21.15.12:0/2303399523 shutdown_connections 2024-04-22T13:28:00.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.375+0000 7f2192fef640 1 --2- 172.21.15.12:0/2303399523 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f21780a3d50 0x7f21780a6210 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.375+0000 7f2192fef640 1 --2- 172.21.15.12:0/2303399523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f218c1035b0 0x7f218c1a6930 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.375+0000 7f2192fef640 1 --2- 172.21.15.12:0/2303399523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f218c106170 0x7f218c103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.375+0000 7f2192fef640 1 --2- 172.21.15.12:0/2303399523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f218c1010a0 0x7f218c102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.375+0000 7f2192fef640 1 -- 172.21.15.12:0/2303399523 >> 172.21.15.12:0/2303399523 conn(0x7f218c077eb0 msgr2=0x7f218c06a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:00.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.375+0000 7f2192fef640 1 -- 172.21.15.12:0/2303399523 shutdown_connections 2024-04-22T13:28:00.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.375+0000 7f2192fef640 1 -- 172.21.15.12:0/2303399523 wait complete. 2024-04-22T13:28:00.382 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:28:00.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.502+0000 7f418736d640 1 -- 172.21.15.12:0/3563001860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4180106160 msgr2=0x7f418010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.502+0000 7f418736d640 1 --2- 172.21.15.12:0/3563001860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4180106160 0x7f418010dd30 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f4178009ca0 tx=0x7f417802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.502+0000 7f418736d640 1 -- 172.21.15.12:0/3563001860 shutdown_connections 2024-04-22T13:28:00.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.502+0000 7f418736d640 1 --2- 172.21.15.12:0/3563001860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4180106160 0x7f418010dd30 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.502+0000 7f418736d640 1 --2- 172.21.15.12:0/3563001860 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4180101a40 0x7f4180105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.502+0000 7f418736d640 1 --2- 172.21.15.12:0/3563001860 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4180101090 0x7f4180101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.502+0000 7f418736d640 1 -- 172.21.15.12:0/3563001860 >> 172.21.15.12:0/3563001860 conn(0x7f4180077f50 msgr2=0x7f4180078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:00.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.502+0000 7f418736d640 1 -- 172.21.15.12:0/3563001860 shutdown_connections 2024-04-22T13:28:00.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.502+0000 7f418736d640 1 -- 172.21.15.12:0/3563001860 wait complete. 2024-04-22T13:28:00.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.503+0000 7f418736d640 1 Processor -- start 2024-04-22T13:28:00.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.503+0000 7f418736d640 1 -- start start 2024-04-22T13:28:00.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.503+0000 7f418736d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4180101090 0x7f4180102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.503+0000 7f418736d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4180106160 0x7f4180103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.503+0000 7f418736d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41801035a0 0x7f41801a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.503+0000 7f418736d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f41801110b0 con 0x7f41801035a0 2024-04-22T13:28:00.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.503+0000 7f418736d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4180110f30 con 0x7f4180101090 2024-04-22T13:28:00.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.503+0000 7f418736d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4180111230 con 0x7f4180106160 2024-04-22T13:28:00.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.503+0000 7f418590a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41801035a0 0x7f41801a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.503+0000 7f418590a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41801035a0 0x7f41801a68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52190/0 (socket says 172.21.15.12:52190) 2024-04-22T13:28:00.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.503+0000 7f418590a640 1 -- 172.21.15.12:0/3824853163 learned_addr learned my addr 172.21.15.12:0/3824853163 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:00.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.503+0000 7f4184908640 1 --2- 172.21.15.12:0/3824853163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4180106160 0x7f4180103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.503+0000 7f4185109640 1 --2- 172.21.15.12:0/3824853163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4180101090 0x7f4180102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.503+0000 7f418590a640 1 -- 172.21.15.12:0/3824853163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4180106160 msgr2=0x7f4180103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.503+0000 7f418590a640 1 --2- 172.21.15.12:0/3824853163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4180106160 0x7f4180103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.503+0000 7f418590a640 1 -- 172.21.15.12:0/3824853163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4180101090 msgr2=0x7f4180102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.503+0000 7f418590a640 1 --2- 172.21.15.12:0/3824853163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4180101090 0x7f4180102b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.503+0000 7f418590a640 1 -- 172.21.15.12:0/3824853163 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41801a6fc0 con 0x7f41801035a0 2024-04-22T13:28:00.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.503+0000 7f4185109640 1 --2- 172.21.15.12:0/3824853163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4180101090 0x7f4180102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:00.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.503+0000 7f4184908640 1 --2- 172.21.15.12:0/3824853163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4180106160 0x7f4180103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:00.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.504+0000 7f418590a640 1 --2- 172.21.15.12:0/3824853163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41801035a0 0x7f41801a68c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f4178005e30 tx=0x7f4178032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:00.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.504+0000 7f41767fc640 1 -- 172.21.15.12:0/3824853163 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4178041020 con 0x7f41801035a0 2024-04-22T13:28:00.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.504+0000 7f41767fc640 1 -- 172.21.15.12:0/3824853163 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4178032970 con 0x7f41801035a0 2024-04-22T13:28:00.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.504+0000 7f418736d640 1 -- 172.21.15.12:0/3824853163 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41801a7250 con 0x7f41801035a0 2024-04-22T13:28:00.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.504+0000 7f41767fc640 1 -- 172.21.15.12:0/3824853163 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4178039040 con 0x7f41801035a0 2024-04-22T13:28:00.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.504+0000 7f418736d640 1 -- 172.21.15.12:0/3824853163 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41801a7740 con 0x7f41801035a0 2024-04-22T13:28:00.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.506+0000 7f41767fc640 1 -- 172.21.15.12:0/3824853163 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4178044020 con 0x7f41801035a0 2024-04-22T13:28:00.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.506+0000 7f418a034740 1 Processor -- start 2024-04-22T13:28:00.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.506+0000 7f418a034740 1 -- start start 2024-04-22T13:28:00.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.507+0000 7f418a034740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5558e6ea04d0 0x5558e6ea08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.507+0000 7f418a034740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558e6ea0e80 0x5558e6ea9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.507+0000 7f418a034740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558e6ea9a00 0x5558e6eabe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.507+0000 7f418a034740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5558e6e051f0 con 0x5558e6ea9a00 2024-04-22T13:28:00.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.507+0000 7f418a034740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5558e6da24b0 con 0x5558e6ea04d0 2024-04-22T13:28:00.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.507+0000 7f418a034740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5558e6e011b0 con 0x5558e6ea0e80 2024-04-22T13:28:00.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.507+0000 7f41767fc640 1 --2- 172.21.15.12:0/3824853163 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f416c0a3d50 0x7f416c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.507+0000 7f4184908640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558e6ea9a00 0x5558e6eabe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.507+0000 7f4184908640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558e6ea9a00 0x5558e6eabe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52202/0 (socket says 172.21.15.12:52202) 2024-04-22T13:28:00.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.507+0000 7f4184908640 1 -- 172.21.15.12:0/1000182159 learned_addr learned my addr 172.21.15.12:0/1000182159 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:00.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.507+0000 7f418590a640 1 --2- 172.21.15.12:0/3824853163 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f416c0a3d50 0x7f416c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.507+0000 7f4185109640 1 --2- 172.21.15.12:0/1000182159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558e6ea0e80 0x5558e6ea9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.507+0000 7f4184908640 1 --2- 172.21.15.12:0/1000182159 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5558e6ea04d0 0x5558e6ea08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.507+0000 7f41767fc640 1 -- 172.21.15.12:0/3824853163 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f41780f2480 con 0x7f41801035a0 2024-04-22T13:28:00.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.507+0000 7f4184908640 1 -- 172.21.15.12:0/1000182159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558e6ea0e80 msgr2=0x5558e6ea9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.507+0000 7f4184908640 1 --2- 172.21.15.12:0/1000182159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558e6ea0e80 0x5558e6ea9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.507+0000 7f4184908640 1 -- 172.21.15.12:0/1000182159 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5558e6ea04d0 msgr2=0x5558e6ea08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.507+0000 7f4184908640 1 --2- 172.21.15.12:0/1000182159 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5558e6ea04d0 0x5558e6ea08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.507+0000 7f4184908640 1 -- 172.21.15.12:0/1000182159 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558e6d941b0 con 0x5558e6ea9a00 2024-04-22T13:28:00.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.507+0000 7f4185109640 1 --2- 172.21.15.12:0/1000182159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558e6ea0e80 0x5558e6ea9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:00.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.507+0000 7f418590a640 1 --2- 172.21.15.12:0/3824853163 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f416c0a3d50 0x7f416c0a6210 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f41780efde0 tx=0x7f417808ffc0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:00.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.508+0000 7f4184908640 1 --2- 172.21.15.12:0/1000182159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558e6ea9a00 0x5558e6eabe50 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f41700058c0 tx=0x7f417000b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aabb7b9d4a8093f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:00.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.508+0000 7f416b7fe640 1 -- 172.21.15.12:0/1000182159 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4170004030 con 0x5558e6ea9a00 2024-04-22T13:28:00.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.508+0000 7f416b7fe640 1 -- 172.21.15.12:0/1000182159 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f417000fbb0 con 0x5558e6ea9a00 2024-04-22T13:28:00.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.508+0000 7f416b7fe640 1 -- 172.21.15.12:0/1000182159 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41700037f0 con 0x5558e6ea9a00 2024-04-22T13:28:00.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.508+0000 7f418a034740 1 -- 172.21.15.12:0/1000182159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558e6ea9a00 msgr2=0x5558e6eabe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.508+0000 7f418a034740 1 --2- 172.21.15.12:0/1000182159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558e6ea9a00 0x5558e6eabe50 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f41700058c0 tx=0x7f417000b4c0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.508+0000 7f418a034740 1 -- 172.21.15.12:0/1000182159 shutdown_connections 2024-04-22T13:28:00.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.508+0000 7f418a034740 1 --2- 172.21.15.12:0/1000182159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558e6ea9a00 0x5558e6eabe50 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.508+0000 7f418a034740 1 --2- 172.21.15.12:0/1000182159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558e6ea0e80 0x5558e6ea9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.508+0000 7f418a034740 1 --2- 172.21.15.12:0/1000182159 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5558e6ea04d0 0x5558e6ea08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.508+0000 7f418a034740 1 -- 172.21.15.12:0/1000182159 >> 172.21.15.12:0/1000182159 conn(0x5558e6e9aba0 msgr2=0x5558e6e9af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:00.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.509+0000 7f418a034740 1 -- 172.21.15.12:0/1000182159 shutdown_connections 2024-04-22T13:28:00.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.509+0000 7f418a034740 1 -- 172.21.15.12:0/1000182159 wait complete. 2024-04-22T13:28:00.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.509+0000 7f418a034740 1 -- start start 2024-04-22T13:28:00.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.509+0000 7f418a034740 1 Processor -- start 2024-04-22T13:28:00.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.509+0000 7f418a034740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558e6ea04d0 0x5558e6ea9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.509+0000 7f418a034740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5558e6ea0e80 0x5558e6ea66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.509+0000 7f418a034740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558e6ea9a00 0x5558e6ea6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.509+0000 7f418a034740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5558e6eac970 con 0x5558e6ea04d0 2024-04-22T13:28:00.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.509+0000 7f418a034740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5558e6dbe500 con 0x5558e6ea0e80 2024-04-22T13:28:00.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.509+0000 7f418a034740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5558e6eacaf0 con 0x5558e6ea9a00 2024-04-22T13:28:00.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.509+0000 7f418a034740 10 client.0 _ll_register_callbacks cb 0x5558e6d8cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:00.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.509+0000 7f4184908640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558e6ea04d0 0x5558e6ea9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.510+0000 7f4184908640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558e6ea04d0 0x5558e6ea9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52212/0 (socket says 172.21.15.12:52212) 2024-04-22T13:28:00.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.510+0000 7f4184908640 1 -- 172.21.15.12:0/2133972344 learned_addr learned my addr 172.21.15.12:0/2133972344 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:00.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.510+0000 7f4185109640 1 --2- 172.21.15.12:0/2133972344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5558e6ea0e80 0x5558e6ea66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.510+0000 7f4184908640 1 -- 172.21.15.12:0/2133972344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558e6ea9a00 msgr2=0x5558e6ea6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.510+0000 7f4184908640 1 --2- 172.21.15.12:0/2133972344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558e6ea9a00 0x5558e6ea6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.510+0000 7f4184908640 1 -- 172.21.15.12:0/2133972344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5558e6ea0e80 msgr2=0x5558e6ea66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.510+0000 7f4184908640 1 --2- 172.21.15.12:0/2133972344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5558e6ea0e80 0x5558e6ea66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.510+0000 7f4184908640 1 -- 172.21.15.12:0/2133972344 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558e6b173f0 con 0x5558e6ea04d0 2024-04-22T13:28:00.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.510+0000 7f4184908640 1 --2- 172.21.15.12:0/2133972344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558e6ea04d0 0x5558e6ea9320 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f4170005750 tx=0x7f417000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd77c9cf90ebeb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:00.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.510+0000 7f4168ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:00.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.510+0000 7f4168ff9640 1 -- 172.21.15.12:0/2133972344 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417000f700 con 0x5558e6ea04d0 2024-04-22T13:28:00.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.510+0000 7f4168ff9640 1 -- 172.21.15.12:0/2133972344 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f417000f8a0 con 0x5558e6ea04d0 2024-04-22T13:28:00.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.510+0000 7f418a034740 1 -- 172.21.15.12:0/2133972344 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f417000fbb0 con 0x5558e6ea04d0 2024-04-22T13:28:00.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.510+0000 7f4168ff9640 1 -- 172.21.15.12:0/2133972344 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4170005020 con 0x5558e6ea04d0 2024-04-22T13:28:00.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.510+0000 7f418a034740 10 client.17097 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:00.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.510+0000 7f418a034740 1 -- 172.21.15.12:0/2133972344 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5558e6be17d0 con 0x5558e6ea04d0 2024-04-22T13:28:00.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.510+0000 7f4168ff9640 1 -- 172.21.15.12:0/2133972344 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f417000fbb0 con 0x5558e6ea04d0 2024-04-22T13:28:00.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.511+0000 7f4168ff9640 1 -- 172.21.15.12:0/2133972344 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f41700123f0 con 0x5558e6ea04d0 2024-04-22T13:28:00.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.511+0000 7f418a034740 10 client.17097 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:00.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.511+0000 7f418a034740 10 client.17097 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:28:00.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.511+0000 7f418a034740 20 client.17097 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:00.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.511+0000 7f418a034740 1 --2- 172.21.15.12:0/2133972344 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x5558e6eb9af0 0x5558e6ebbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.511+0000 7f418a034740 4 client.17097 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:00.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.511+0000 7f418a034740 1 -- 172.21.15.12:0/2133972344 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558e6da4930 con 0x5558e6eb9af0 2024-04-22T13:28:00.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.511+0000 7f4185109640 1 --2- 172.21.15.12:0/2133972344 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x5558e6eb9af0 0x5558e6ebbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.511+0000 7f4168ff9640 10 client.17097 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:28:00.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.511+0000 7f4185109640 1 --2- 172.21.15.12:0/2133972344 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x5558e6eb9af0 0x5558e6ebbee0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1b21a9a31189cf99 server_cookie=ae887c17caf6e728 in_seq=0 out_seq=0 2024-04-22T13:28:00.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.512+0000 7f4168ff9640 1 -- 172.21.15.12:0/2133972344 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5558e6da4930 con 0x5558e6eb9af0 2024-04-22T13:28:00.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.512+0000 7f4168ff9640 10 client.17097 handle_command_reply: tid=0 2024-04-22T13:28:00.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.512+0000 7f418a034740 1 client.17097 shutdown 2024-04-22T13:28:00.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.512+0000 7f418a034740 1 -- 172.21.15.12:0/2133972344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558e6ea04d0 msgr2=0x5558e6ea9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.512+0000 7f418a034740 1 --2- 172.21.15.12:0/2133972344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558e6ea04d0 0x5558e6ea9320 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f4170005750 tx=0x7f417000b410 comp rx=0 tx=0).stop 2024-04-22T13:28:00.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.512+0000 7f418a034740 1 -- 172.21.15.12:0/2133972344 shutdown_connections 2024-04-22T13:28:00.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.512+0000 7f418a034740 1 --2- 172.21.15.12:0/2133972344 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558e6ea9a00 0x5558e6ea6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.512+0000 7f418a034740 1 --2- 172.21.15.12:0/2133972344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5558e6ea0e80 0x5558e6ea66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.512+0000 7f418a034740 1 --2- 172.21.15.12:0/2133972344 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x5558e6eb9af0 0x5558e6ebbee0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.512+0000 7f418a034740 1 --2- 172.21.15.12:0/2133972344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558e6ea04d0 0x5558e6ea9320 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.512+0000 7f4168ff9640 0 client.17097 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:28:00.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.512+0000 7f418a034740 1 -- 172.21.15.12:0/2133972344 >> 172.21.15.12:0/2133972344 conn(0x5558e6e9aba0 msgr2=0x5558e6e9c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:00.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.512+0000 7f418a034740 1 -- 172.21.15.12:0/2133972344 shutdown_connections 2024-04-22T13:28:00.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.512+0000 7f418a034740 1 -- 172.21.15.12:0/2133972344 wait complete. 2024-04-22T13:28:00.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.512+0000 7f418a034740 20 client.17097 trim_cache size 0 max 16384 2024-04-22T13:28:00.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.526+0000 7f418a034740 1 Processor -- start 2024-04-22T13:28:00.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.526+0000 7f418a034740 1 -- start start 2024-04-22T13:28:00.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.526+0000 7f418a034740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5558e6ea04d0 0x5558e6ea5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.526+0000 7f418a034740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558e6ea0e80 0x5558e6ea8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.526+0000 7f418a034740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558e6ea9a00 0x5558e6ea9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.526+0000 7f418a034740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f41801afcb0 con 0x5558e6ea0e80 2024-04-22T13:28:00.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.526+0000 7f418a034740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f41801b17f0 con 0x5558e6ea04d0 2024-04-22T13:28:00.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.526+0000 7f418a034740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f41801b0380 con 0x5558e6ea9a00 2024-04-22T13:28:00.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.526+0000 7f4185109640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558e6ea0e80 0x5558e6ea8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.526+0000 7f4185109640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558e6ea0e80 0x5558e6ea8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52218/0 (socket says 172.21.15.12:52218) 2024-04-22T13:28:00.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.526+0000 7f4185109640 1 -- 172.21.15.12:0/593981772 learned_addr learned my addr 172.21.15.12:0/593981772 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:00.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.526+0000 7f4184908640 1 --2- 172.21.15.12:0/593981772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558e6ea9a00 0x5558e6ea9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.526+0000 7f4184908640 1 --2- 172.21.15.12:0/593981772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5558e6ea04d0 0x5558e6ea5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.526+0000 7f4185109640 1 -- 172.21.15.12:0/593981772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558e6ea9a00 msgr2=0x5558e6ea9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.526+0000 7f4185109640 1 --2- 172.21.15.12:0/593981772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558e6ea9a00 0x5558e6ea9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.526+0000 7f4185109640 1 -- 172.21.15.12:0/593981772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5558e6ea04d0 msgr2=0x5558e6ea5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.526+0000 7f4185109640 1 --2- 172.21.15.12:0/593981772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5558e6ea04d0 0x5558e6ea5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.526+0000 7f4185109640 1 -- 172.21.15.12:0/593981772 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558e6ea4c00 con 0x5558e6ea0e80 2024-04-22T13:28:00.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.526+0000 7f4184908640 1 --2- 172.21.15.12:0/593981772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5558e6ea04d0 0x5558e6ea5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:00.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.526+0000 7f4185109640 1 --2- 172.21.15.12:0/593981772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558e6ea0e80 0x5558e6ea8de0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x5558e6ea1280 tx=0x7f417c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7660a3eaabd2649a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:00.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.527+0000 7f416affd640 1 -- 172.21.15.12:0/593981772 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c026070 con 0x5558e6ea0e80 2024-04-22T13:28:00.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.527+0000 7f416affd640 1 -- 172.21.15.12:0/593981772 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f417c004310 con 0x5558e6ea0e80 2024-04-22T13:28:00.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.527+0000 7f416affd640 1 -- 172.21.15.12:0/593981772 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c0039e0 con 0x5558e6ea0e80 2024-04-22T13:28:00.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.527+0000 7f418a034740 1 -- 172.21.15.12:0/593981772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558e6ea0e80 msgr2=0x5558e6ea8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.527+0000 7f418a034740 1 --2- 172.21.15.12:0/593981772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558e6ea0e80 0x5558e6ea8de0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x5558e6ea1280 tx=0x7f417c005f90 comp rx=0 tx=0).stop 2024-04-22T13:28:00.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.527+0000 7f418a034740 1 -- 172.21.15.12:0/593981772 shutdown_connections 2024-04-22T13:28:00.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.527+0000 7f418a034740 1 --2- 172.21.15.12:0/593981772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558e6ea9a00 0x5558e6ea9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.527+0000 7f418a034740 1 --2- 172.21.15.12:0/593981772 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558e6ea0e80 0x5558e6ea8de0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.527+0000 7f418a034740 1 --2- 172.21.15.12:0/593981772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5558e6ea04d0 0x5558e6ea5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.527+0000 7f418a034740 1 -- 172.21.15.12:0/593981772 >> 172.21.15.12:0/593981772 conn(0x5558e6e9aba0 msgr2=0x5558e6ea4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:00.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.527+0000 7f418a034740 1 -- 172.21.15.12:0/593981772 shutdown_connections 2024-04-22T13:28:00.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.527+0000 7f418a034740 1 -- 172.21.15.12:0/593981772 wait complete. 2024-04-22T13:28:00.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.528+0000 7f418a034740 1 -- start start 2024-04-22T13:28:00.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.528+0000 7f418a034740 1 Processor -- start 2024-04-22T13:28:00.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.528+0000 7f418a034740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5558e6ea04d0 0x5558e6ebef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.528+0000 7f418a034740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558e6ea0e80 0x5558e6ebc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.528+0000 7f418a034740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558e6ea9a00 0x5558e6ebc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.528+0000 7f418a034740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f41801b1120 con 0x5558e6ea9a00 2024-04-22T13:28:00.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.528+0000 7f418a034740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f41801b0a50 con 0x5558e6ea04d0 2024-04-22T13:28:00.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.528+0000 7f418a034740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f41801adc90 con 0x5558e6ea0e80 2024-04-22T13:28:00.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.528+0000 7f418a034740 10 client.0 _ll_register_callbacks cb 0x7f4170014880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:00.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.528+0000 7f4184908640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558e6ea9a00 0x5558e6ebc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.528+0000 7f4184908640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558e6ea9a00 0x5558e6ebc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52228/0 (socket says 172.21.15.12:52228) 2024-04-22T13:28:00.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.528+0000 7f4184908640 1 -- 172.21.15.12:0/927734748 learned_addr learned my addr 172.21.15.12:0/927734748 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:00.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.528+0000 7f4185109640 1 --2- 172.21.15.12:0/927734748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558e6ea0e80 0x5558e6ebc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.528+0000 7f4184908640 1 --2- 172.21.15.12:0/927734748 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5558e6ea04d0 0x5558e6ebef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.528+0000 7f4184908640 1 -- 172.21.15.12:0/927734748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558e6ea0e80 msgr2=0x5558e6ebc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.528+0000 7f4184908640 1 --2- 172.21.15.12:0/927734748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558e6ea0e80 0x5558e6ebc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.528+0000 7f4184908640 1 -- 172.21.15.12:0/927734748 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5558e6ea04d0 msgr2=0x5558e6ebef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.528+0000 7f4184908640 1 --2- 172.21.15.12:0/927734748 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5558e6ea04d0 0x5558e6ebef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.529+0000 7f4184908640 1 -- 172.21.15.12:0/927734748 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558e6ebcee0 con 0x5558e6ea9a00 2024-04-22T13:28:00.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.529+0000 7f4184908640 1 --2- 172.21.15.12:0/927734748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558e6ea9a00 0x5558e6ebc9a0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f417000b830 tx=0x7f4170024040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=812e9e33327023cb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:00.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.529+0000 7f4168ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:00.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.529+0000 7f4168ff9640 1 -- 172.21.15.12:0/927734748 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4170012cd0 con 0x5558e6ea9a00 2024-04-22T13:28:00.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.529+0000 7f4168ff9640 1 -- 172.21.15.12:0/927734748 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f41700051c0 con 0x5558e6ea9a00 2024-04-22T13:28:00.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.529+0000 7f418a034740 1 -- 172.21.15.12:0/927734748 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f417c004310 con 0x5558e6ea9a00 2024-04-22T13:28:00.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.529+0000 7f4168ff9640 1 -- 172.21.15.12:0/927734748 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f41700053c0 con 0x5558e6ea9a00 2024-04-22T13:28:00.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.529+0000 7f418a034740 10 client.17103 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:00.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.529+0000 7f418a034740 1 -- 172.21.15.12:0/927734748 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5558e6f1f900 con 0x5558e6ea9a00 2024-04-22T13:28:00.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.529+0000 7f4168ff9640 1 -- 172.21.15.12:0/927734748 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f417c004310 con 0x5558e6ea9a00 2024-04-22T13:28:00.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.529+0000 7f4168ff9640 1 -- 172.21.15.12:0/927734748 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f41700048b0 con 0x5558e6ea9a00 2024-04-22T13:28:00.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.529+0000 7f418a034740 10 client.17103 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:00.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.529+0000 7f418a034740 10 client.17103 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:28:00.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.529+0000 7f418a034740 20 client.17103 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:00.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.529+0000 7f418a034740 1 --2- 172.21.15.12:0/927734748 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x5558e6f1fff0 0x5558e6f22410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.529+0000 7f418a034740 4 client.17103 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:28:00.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.529+0000 7f418a034740 1 -- 172.21.15.12:0/927734748 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5558e6e12c10 con 0x5558e6f1fff0 2024-04-22T13:28:00.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.530+0000 7f4185109640 1 --2- 172.21.15.12:0/927734748 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x5558e6f1fff0 0x5558e6f22410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.530+0000 7f4185109640 1 --2- 172.21.15.12:0/927734748 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x5558e6f1fff0 0x5558e6f22410 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f294cb3dc30744e3 server_cookie=60f8d39fdc397651 in_seq=0 out_seq=0 2024-04-22T13:28:00.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.530+0000 7f4168ff9640 10 client.17103 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:28:00.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.530+0000 7f4168ff9640 1 -- 172.21.15.12:0/927734748 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5558e6e12c10 con 0x5558e6f1fff0 2024-04-22T13:28:00.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.530+0000 7f4168ff9640 10 client.17103 handle_command_reply: tid=0 2024-04-22T13:28:00.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.530+0000 7f418a034740 1 client.17103 shutdown 2024-04-22T13:28:00.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.530+0000 7f418a034740 1 -- 172.21.15.12:0/927734748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558e6ea9a00 msgr2=0x5558e6ebc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.530+0000 7f418a034740 1 --2- 172.21.15.12:0/927734748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558e6ea9a00 0x5558e6ebc9a0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f417000b830 tx=0x7f4170024040 comp rx=0 tx=0).stop 2024-04-22T13:28:00.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.530+0000 7f418a034740 1 -- 172.21.15.12:0/927734748 shutdown_connections 2024-04-22T13:28:00.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.530+0000 7f418a034740 1 --2- 172.21.15.12:0/927734748 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x5558e6f1fff0 0x5558e6f22410 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.530+0000 7f418a034740 1 --2- 172.21.15.12:0/927734748 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5558e6ea9a00 0x5558e6ebc9a0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.530+0000 7f418a034740 1 --2- 172.21.15.12:0/927734748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5558e6ea0e80 0x5558e6ebc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.530+0000 7f418a034740 1 --2- 172.21.15.12:0/927734748 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5558e6ea04d0 0x5558e6ebef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.530+0000 7f4168ff9640 0 client.17103 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:28:00.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.530+0000 7f418a034740 1 -- 172.21.15.12:0/927734748 >> 172.21.15.12:0/927734748 conn(0x5558e6e9aba0 msgr2=0x5558e6ea56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:00.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.531+0000 7f418a034740 1 -- 172.21.15.12:0/927734748 shutdown_connections 2024-04-22T13:28:00.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.531+0000 7f418a034740 1 -- 172.21.15.12:0/927734748 wait complete. 2024-04-22T13:28:00.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.531+0000 7f418a034740 20 client.17103 trim_cache size 0 max 16384 2024-04-22T13:28:00.536 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:28:00.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.531+0000 7f418736d640 1 -- 172.21.15.12:0/3824853163 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f416c0a3d50 msgr2=0x7f416c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.531+0000 7f418736d640 1 --2- 172.21.15.12:0/3824853163 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f416c0a3d50 0x7f416c0a6210 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f41780efde0 tx=0x7f417808ffc0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.531+0000 7f418736d640 1 -- 172.21.15.12:0/3824853163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41801035a0 msgr2=0x7f41801a68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.531+0000 7f418736d640 1 --2- 172.21.15.12:0/3824853163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41801035a0 0x7f41801a68c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f4178005e30 tx=0x7f4178032420 comp rx=0 tx=0).stop 2024-04-22T13:28:00.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.531+0000 7f418736d640 1 -- 172.21.15.12:0/3824853163 shutdown_connections 2024-04-22T13:28:00.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.531+0000 7f418736d640 1 --2- 172.21.15.12:0/3824853163 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f416c0a3d50 0x7f416c0a6210 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.531+0000 7f418736d640 1 --2- 172.21.15.12:0/3824853163 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f41801035a0 0x7f41801a68c0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.531+0000 7f418736d640 1 --2- 172.21.15.12:0/3824853163 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4180106160 0x7f4180103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.531+0000 7f418736d640 1 --2- 172.21.15.12:0/3824853163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4180101090 0x7f4180102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.531+0000 7f418736d640 1 -- 172.21.15.12:0/3824853163 >> 172.21.15.12:0/3824853163 conn(0x7f4180077f50 msgr2=0x7f41800694d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:00.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.531+0000 7f418736d640 1 -- 172.21.15.12:0/3824853163 shutdown_connections 2024-04-22T13:28:00.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.531+0000 7f418736d640 1 -- 172.21.15.12:0/3824853163 wait complete. 2024-04-22T13:28:00.539 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:28:00.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.647+0000 7f0095c74640 1 -- 172.21.15.12:0/3934019502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0090101a40 msgr2=0x7f0090105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.647+0000 7f0095c74640 1 --2- 172.21.15.12:0/3934019502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0090101a40 0x7f0090105a80 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f0084009900 tx=0x7f008402bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:00.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.647+0000 7f0095c74640 1 -- 172.21.15.12:0/3934019502 shutdown_connections 2024-04-22T13:28:00.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.647+0000 7f0095c74640 1 --2- 172.21.15.12:0/3934019502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0090106160 0x7f009010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.647+0000 7f0095c74640 1 --2- 172.21.15.12:0/3934019502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0090101a40 0x7f0090105a80 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.647+0000 7f0095c74640 1 --2- 172.21.15.12:0/3934019502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0090101090 0x7f0090101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.647+0000 7f0095c74640 1 -- 172.21.15.12:0/3934019502 >> 172.21.15.12:0/3934019502 conn(0x7f0090077f50 msgr2=0x7f0090078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:00.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.647+0000 7f0095c74640 1 -- 172.21.15.12:0/3934019502 shutdown_connections 2024-04-22T13:28:00.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.647+0000 7f0095c74640 1 -- 172.21.15.12:0/3934019502 wait complete. 2024-04-22T13:28:00.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.647+0000 7f0095c74640 1 Processor -- start 2024-04-22T13:28:00.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.647+0000 7f0095c74640 1 -- start start 2024-04-22T13:28:00.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.648+0000 7f0095c74640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0090101090 0x7f0090198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.648+0000 7f0095c74640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0090106160 0x7f0090198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.648+0000 7f0095c74640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f009019d000 0x7f009019d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.648+0000 7f0095c74640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f00901121a0 con 0x7f0090101090 2024-04-22T13:28:00.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.648+0000 7f0095c74640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0090112020 con 0x7f009019d000 2024-04-22T13:28:00.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.648+0000 7f0095c74640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0090112320 con 0x7f0090106160 2024-04-22T13:28:00.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.648+0000 7f008f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0090101090 0x7f0090198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.648+0000 7f008f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0090101090 0x7f0090198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52250/0 (socket says 172.21.15.12:52250) 2024-04-22T13:28:00.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.648+0000 7f008f7fe640 1 -- 172.21.15.12:0/2760400374 learned_addr learned my addr 172.21.15.12:0/2760400374 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:00.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.648+0000 7f008effd640 1 --2- 172.21.15.12:0/2760400374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0090106160 0x7f0090198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.648+0000 7f008ffff640 1 --2- 172.21.15.12:0/2760400374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f009019d000 0x7f009019d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.648+0000 7f008f7fe640 1 -- 172.21.15.12:0/2760400374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0090106160 msgr2=0x7f0090198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.648+0000 7f008f7fe640 1 --2- 172.21.15.12:0/2760400374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0090106160 0x7f0090198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.648+0000 7f008f7fe640 1 -- 172.21.15.12:0/2760400374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f009019d000 msgr2=0x7f009019d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.648+0000 7f008f7fe640 1 --2- 172.21.15.12:0/2760400374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f009019d000 0x7f009019d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.648+0000 7f008f7fe640 1 -- 172.21.15.12:0/2760400374 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f009019daf0 con 0x7f0090101090 2024-04-22T13:28:00.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.648+0000 7f008ffff640 1 --2- 172.21.15.12:0/2760400374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f009019d000 0x7f009019d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:00.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.648+0000 7f008f7fe640 1 --2- 172.21.15.12:0/2760400374 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0090101090 0x7f0090198740 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f008000d950 tx=0x7f008000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:00.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.648+0000 7f008cff9640 1 -- 172.21.15.12:0/2760400374 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0080017070 con 0x7f0090101090 2024-04-22T13:28:00.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.649+0000 7f0095c74640 1 -- 172.21.15.12:0/2760400374 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f009019dde0 con 0x7f0090101090 2024-04-22T13:28:00.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.649+0000 7f008cff9640 1 -- 172.21.15.12:0/2760400374 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0080009c30 con 0x7f0090101090 2024-04-22T13:28:00.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.649+0000 7f008cff9640 1 -- 172.21.15.12:0/2760400374 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008000f620 con 0x7f0090101090 2024-04-22T13:28:00.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.649+0000 7f0095c74640 1 -- 172.21.15.12:0/2760400374 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00901a55f0 con 0x7f0090101090 2024-04-22T13:28:00.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.651+0000 7f008cff9640 1 -- 172.21.15.12:0/2760400374 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0080010460 con 0x7f0090101090 2024-04-22T13:28:00.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.651+0000 7f009886c740 1 Processor -- start 2024-04-22T13:28:00.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.651+0000 7f009886c740 1 -- start start 2024-04-22T13:28:00.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.651+0000 7f009886c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648327ba250 0x5648327ba630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.651+0000 7f009886c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648327bac00 0x5648327c7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.651+0000 7f009886c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5648327c7e10 0x5648327ca220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.651+0000 7f009886c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56483271f1f0 con 0x5648327bac00 2024-04-22T13:28:00.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.651+0000 7f009886c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5648326bc4b0 con 0x5648327c7e10 2024-04-22T13:28:00.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.651+0000 7f009886c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56483271b1b0 con 0x5648327ba250 2024-04-22T13:28:00.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.651+0000 7f008f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648327bac00 0x5648327c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.651+0000 7f008f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648327bac00 0x5648327c7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52260/0 (socket says 172.21.15.12:52260) 2024-04-22T13:28:00.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.651+0000 7f008f7fe640 1 -- 172.21.15.12:0/3131367562 learned_addr learned my addr 172.21.15.12:0/3131367562 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:00.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.651+0000 7f008cff9640 1 --2- 172.21.15.12:0/2760400374 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f00780a3d20 0x7f00780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.652+0000 7f008effd640 1 --2- 172.21.15.12:0/3131367562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5648327c7e10 0x5648327ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.652+0000 7f008ffff640 1 --2- 172.21.15.12:0/3131367562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648327ba250 0x5648327ba630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.652+0000 7f008ffff640 1 --2- 172.21.15.12:0/2760400374 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f00780a3d20 0x7f00780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.652+0000 7f008f7fe640 1 -- 172.21.15.12:0/3131367562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648327ba250 msgr2=0x5648327ba630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.652+0000 7f008f7fe640 1 --2- 172.21.15.12:0/3131367562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648327ba250 0x5648327ba630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.652+0000 7f008f7fe640 1 -- 172.21.15.12:0/3131367562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5648327c7e10 msgr2=0x5648327ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.652+0000 7f008f7fe640 1 --2- 172.21.15.12:0/3131367562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5648327c7e10 0x5648327ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.652+0000 7f008cff9640 1 -- 172.21.15.12:0/2760400374 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f00800d03f0 con 0x7f0090101090 2024-04-22T13:28:00.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.652+0000 7f008f7fe640 1 -- 172.21.15.12:0/3131367562 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648326ae1b0 con 0x5648327bac00 2024-04-22T13:28:00.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.652+0000 7f008effd640 1 --2- 172.21.15.12:0/3131367562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5648327c7e10 0x5648327ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:00.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.652+0000 7f008f7fe640 1 --2- 172.21.15.12:0/3131367562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648327bac00 0x5648327c7840 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f00800d3fd0 tx=0x7f0080010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61abf2dab8fbaf17 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:00.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.652+0000 7f008ffff640 1 --2- 172.21.15.12:0/2760400374 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f00780a3d20 0x7f00780a61e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0088005fd0 tx=0x7f0088005c80 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:00.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.652+0000 7f0075ffb640 1 -- 172.21.15.12:0/3131367562 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0080020030 con 0x5648327bac00 2024-04-22T13:28:00.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.652+0000 7f0075ffb640 1 -- 172.21.15.12:0/3131367562 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f008009cae0 con 0x5648327bac00 2024-04-22T13:28:00.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.652+0000 7f0075ffb640 1 -- 172.21.15.12:0/3131367562 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00800a5050 con 0x5648327bac00 2024-04-22T13:28:00.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.653+0000 7f009886c740 1 -- 172.21.15.12:0/3131367562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648327bac00 msgr2=0x5648327c7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.653+0000 7f009886c740 1 --2- 172.21.15.12:0/3131367562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648327bac00 0x5648327c7840 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f00800d3fd0 tx=0x7f0080010cb0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.653+0000 7f009886c740 1 -- 172.21.15.12:0/3131367562 shutdown_connections 2024-04-22T13:28:00.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.653+0000 7f009886c740 1 --2- 172.21.15.12:0/3131367562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5648327c7e10 0x5648327ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.653+0000 7f009886c740 1 --2- 172.21.15.12:0/3131367562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648327bac00 0x5648327c7840 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.653+0000 7f009886c740 1 --2- 172.21.15.12:0/3131367562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648327ba250 0x5648327ba630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.653+0000 7f009886c740 1 -- 172.21.15.12:0/3131367562 >> 172.21.15.12:0/3131367562 conn(0x5648327b4ba0 msgr2=0x5648327b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:00.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.653+0000 7f009886c740 1 -- 172.21.15.12:0/3131367562 shutdown_connections 2024-04-22T13:28:00.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.653+0000 7f009886c740 1 -- 172.21.15.12:0/3131367562 wait complete. 2024-04-22T13:28:00.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.653+0000 7f009886c740 1 -- start start 2024-04-22T13:28:00.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.653+0000 7f009886c740 1 Processor -- start 2024-04-22T13:28:00.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.654+0000 7f009886c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648327ba250 0x5648327c7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.654+0000 7f009886c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648327bac00 0x5648327c4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.654+0000 7f009886c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5648327c7e10 0x5648327c5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.654+0000 7f009886c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5648327cac80 con 0x5648327ba250 2024-04-22T13:28:00.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.654+0000 7f009886c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5648326d8500 con 0x5648327c7e10 2024-04-22T13:28:00.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.654+0000 7f009886c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5648327cae00 con 0x5648327bac00 2024-04-22T13:28:00.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.654+0000 7f009886c740 10 client.0 _ll_register_callbacks cb 0x5648326a6b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:00.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.654+0000 7f008f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648327ba250 0x5648327c7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.654+0000 7f008f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648327ba250 0x5648327c7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52274/0 (socket says 172.21.15.12:52274) 2024-04-22T13:28:00.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.654+0000 7f008f7fe640 1 -- 172.21.15.12:0/565400337 learned_addr learned my addr 172.21.15.12:0/565400337 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:00.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.654+0000 7f008ffff640 1 --2- 172.21.15.12:0/565400337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5648327c7e10 0x5648327c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.654+0000 7f008effd640 1 --2- 172.21.15.12:0/565400337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648327bac00 0x5648327c4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.654+0000 7f008f7fe640 1 -- 172.21.15.12:0/565400337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648327bac00 msgr2=0x5648327c4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.654+0000 7f008f7fe640 1 --2- 172.21.15.12:0/565400337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648327bac00 0x5648327c4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.654+0000 7f008f7fe640 1 -- 172.21.15.12:0/565400337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5648327c7e10 msgr2=0x5648327c5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.654+0000 7f008f7fe640 1 --2- 172.21.15.12:0/565400337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5648327c7e10 0x5648327c5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.654+0000 7f008f7fe640 1 -- 172.21.15.12:0/565400337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648324313f0 con 0x5648327ba250 2024-04-22T13:28:00.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.654+0000 7f008ffff640 1 --2- 172.21.15.12:0/565400337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5648327c7e10 0x5648327c5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:00.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.654+0000 7f008f7fe640 1 --2- 172.21.15.12:0/565400337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648327ba250 0x5648327c7710 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f00800cdc40 tx=0x7f00800a0800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb6c6c23e8e800c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:00.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.654+0000 7f006effd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:00.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.654+0000 7f006effd640 1 -- 172.21.15.12:0/565400337 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0080020050 con 0x5648327ba250 2024-04-22T13:28:00.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.654+0000 7f006effd640 1 -- 172.21.15.12:0/565400337 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f0080056030 con 0x5648327ba250 2024-04-22T13:28:00.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.654+0000 7f009886c740 1 -- 172.21.15.12:0/565400337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f008009cae0 con 0x5648327ba250 2024-04-22T13:28:00.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.655+0000 7f006effd640 1 -- 172.21.15.12:0/565400337 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0080009de0 con 0x5648327ba250 2024-04-22T13:28:00.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.655+0000 7f009886c740 10 client.17115 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:00.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.655+0000 7f009886c740 1 -- 172.21.15.12:0/565400337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5648324fb7d0 con 0x5648327ba250 2024-04-22T13:28:00.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.655+0000 7f006effd640 1 -- 172.21.15.12:0/565400337 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f008009cae0 con 0x5648327ba250 2024-04-22T13:28:00.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.655+0000 7f006effd640 1 -- 172.21.15.12:0/565400337 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0080098130 con 0x5648327ba250 2024-04-22T13:28:00.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.655+0000 7f009886c740 10 client.17115 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:00.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.655+0000 7f009886c740 10 client.17115 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:28:00.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.655+0000 7f009886c740 20 client.17115 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:00.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.655+0000 7f009886c740 1 --2- 172.21.15.12:0/565400337 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5648327d7dd0 0x5648327da1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.655+0000 7f009886c740 4 client.17115 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:00.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.655+0000 7f009886c740 1 -- 172.21.15.12:0/565400337 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648326be930 con 0x5648327d7dd0 2024-04-22T13:28:00.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.655+0000 7f008effd640 1 --2- 172.21.15.12:0/565400337 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5648327d7dd0 0x5648327da1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.656+0000 7f006effd640 10 client.17115 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:28:00.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.656+0000 7f008effd640 1 --2- 172.21.15.12:0/565400337 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5648327d7dd0 0x5648327da1c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4b00e6c1a4590bf7 server_cookie=12aded5866bb39e5 in_seq=0 out_seq=0 2024-04-22T13:28:00.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.656+0000 7f006effd640 1 -- 172.21.15.12:0/565400337 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5648326be930 con 0x5648327d7dd0 2024-04-22T13:28:00.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.656+0000 7f006effd640 10 client.17115 handle_command_reply: tid=0 2024-04-22T13:28:00.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.656+0000 7f009886c740 1 client.17115 shutdown 2024-04-22T13:28:00.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.657+0000 7f009886c740 1 -- 172.21.15.12:0/565400337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648327ba250 msgr2=0x5648327c7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.657+0000 7f009886c740 1 --2- 172.21.15.12:0/565400337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648327ba250 0x5648327c7710 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f00800cdc40 tx=0x7f00800a0800 comp rx=0 tx=0).stop 2024-04-22T13:28:00.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.657+0000 7f009886c740 1 -- 172.21.15.12:0/565400337 shutdown_connections 2024-04-22T13:28:00.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.657+0000 7f009886c740 1 --2- 172.21.15.12:0/565400337 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5648327d7dd0 0x5648327da1c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.657+0000 7f009886c740 1 --2- 172.21.15.12:0/565400337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5648327c7e10 0x5648327c5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.657+0000 7f009886c740 1 --2- 172.21.15.12:0/565400337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648327bac00 0x5648327c4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.657+0000 7f006effd640 0 client.17115 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:28:00.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.657+0000 7f009886c740 1 --2- 172.21.15.12:0/565400337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648327ba250 0x5648327c7710 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.657+0000 7f009886c740 1 -- 172.21.15.12:0/565400337 >> 172.21.15.12:0/565400337 conn(0x5648327b4ba0 msgr2=0x5648327b68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:00.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.657+0000 7f009886c740 1 -- 172.21.15.12:0/565400337 shutdown_connections 2024-04-22T13:28:00.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.657+0000 7f009886c740 1 -- 172.21.15.12:0/565400337 wait complete. 2024-04-22T13:28:00.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.657+0000 7f009886c740 20 client.17115 trim_cache size 0 max 16384 2024-04-22T13:28:00.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.670+0000 7f009886c740 1 Processor -- start 2024-04-22T13:28:00.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.670+0000 7f009886c740 1 -- start start 2024-04-22T13:28:00.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.670+0000 7f009886c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648327ba250 0x5648327d3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.670+0000 7f009886c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648327bac00 0x5648327b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.670+0000 7f009886c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5648327c7e10 0x5648327c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.670+0000 7f009886c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f00901ab8f0 con 0x5648327ba250 2024-04-22T13:28:00.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.670+0000 7f009886c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f00901ad430 con 0x5648327c7e10 2024-04-22T13:28:00.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.670+0000 7f009886c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f00901abfc0 con 0x5648327bac00 2024-04-22T13:28:00.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.670+0000 7f008f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648327ba250 0x5648327d3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.671+0000 7f008f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648327ba250 0x5648327d3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52284/0 (socket says 172.21.15.12:52284) 2024-04-22T13:28:00.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.671+0000 7f008f7fe640 1 -- 172.21.15.12:0/2211991390 learned_addr learned my addr 172.21.15.12:0/2211991390 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:00.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.671+0000 7f008effd640 1 --2- 172.21.15.12:0/2211991390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648327bac00 0x5648327b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.671+0000 7f008f7fe640 1 -- 172.21.15.12:0/2211991390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648327bac00 msgr2=0x5648327b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.671+0000 7f008f7fe640 1 --2- 172.21.15.12:0/2211991390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648327bac00 0x5648327b3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.671+0000 7f008f7fe640 1 -- 172.21.15.12:0/2211991390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5648327c7e10 msgr2=0x5648327c71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.671+0000 7f008f7fe640 1 --2- 172.21.15.12:0/2211991390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5648327c7e10 0x5648327c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.671+0000 7f008f7fe640 1 -- 172.21.15.12:0/2211991390 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648327ca680 con 0x5648327ba250 2024-04-22T13:28:00.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.671+0000 7f008effd640 1 --2- 172.21.15.12:0/2211991390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648327bac00 0x5648327b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:00.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.671+0000 7f008f7fe640 1 --2- 172.21.15.12:0/2211991390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648327ba250 0x5648327d3b80 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f0080098db0 tx=0x7f0080010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d936ef75e6916cf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:00.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.671+0000 7f00757fa640 1 -- 172.21.15.12:0/2211991390 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0080020030 con 0x5648327ba250 2024-04-22T13:28:00.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.671+0000 7f00757fa640 1 -- 172.21.15.12:0/2211991390 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f008009cae0 con 0x5648327ba250 2024-04-22T13:28:00.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.671+0000 7f00757fa640 1 -- 172.21.15.12:0/2211991390 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00800109b0 con 0x5648327ba250 2024-04-22T13:28:00.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.672+0000 7f009886c740 1 -- 172.21.15.12:0/2211991390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648327ba250 msgr2=0x5648327d3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.672+0000 7f009886c740 1 --2- 172.21.15.12:0/2211991390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648327ba250 0x5648327d3b80 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f0080098db0 tx=0x7f0080010cb0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.672+0000 7f009886c740 1 -- 172.21.15.12:0/2211991390 shutdown_connections 2024-04-22T13:28:00.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.672+0000 7f009886c740 1 --2- 172.21.15.12:0/2211991390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5648327c7e10 0x5648327c71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.672+0000 7f009886c740 1 --2- 172.21.15.12:0/2211991390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648327bac00 0x5648327b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.672+0000 7f009886c740 1 --2- 172.21.15.12:0/2211991390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648327ba250 0x5648327d3b80 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.672+0000 7f009886c740 1 -- 172.21.15.12:0/2211991390 >> 172.21.15.12:0/2211991390 conn(0x5648327b4ba0 msgr2=0x5648327c0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:00.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.672+0000 7f009886c740 1 -- 172.21.15.12:0/2211991390 shutdown_connections 2024-04-22T13:28:00.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.672+0000 7f009886c740 1 -- 172.21.15.12:0/2211991390 wait complete. 2024-04-22T13:28:00.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.672+0000 7f009886c740 1 -- start start 2024-04-22T13:28:00.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.672+0000 7f009886c740 1 Processor -- start 2024-04-22T13:28:00.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.672+0000 7f009886c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648327ba250 0x5648327cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.673+0000 7f009886c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5648327bac00 0x5648327cc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.673+0000 7f009886c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648327c7e10 0x5648327cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.673+0000 7f009886c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f00901acd60 con 0x5648327c7e10 2024-04-22T13:28:00.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.673+0000 7f009886c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f00901ac690 con 0x5648327bac00 2024-04-22T13:28:00.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.673+0000 7f009886c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f00901a98f0 con 0x5648327ba250 2024-04-22T13:28:00.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.673+0000 7f009886c740 10 client.0 _ll_register_callbacks cb 0x7f008009bfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:00.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.673+0000 7f008ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648327c7e10 0x5648327cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.673+0000 7f008ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648327c7e10 0x5648327cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52298/0 (socket says 172.21.15.12:52298) 2024-04-22T13:28:00.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.673+0000 7f008ffff640 1 -- 172.21.15.12:0/1383735693 learned_addr learned my addr 172.21.15.12:0/1383735693 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:00.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.673+0000 7f008f7fe640 1 --2- 172.21.15.12:0/1383735693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648327ba250 0x5648327cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.673+0000 7f008effd640 1 --2- 172.21.15.12:0/1383735693 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5648327bac00 0x5648327cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.673+0000 7f008ffff640 1 -- 172.21.15.12:0/1383735693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648327ba250 msgr2=0x5648327cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.673+0000 7f008ffff640 1 --2- 172.21.15.12:0/1383735693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648327ba250 0x5648327cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.673+0000 7f008ffff640 1 -- 172.21.15.12:0/1383735693 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5648327bac00 msgr2=0x5648327cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.673+0000 7f008ffff640 1 --2- 172.21.15.12:0/1383735693 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5648327bac00 0x5648327cc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.673+0000 7f008ffff640 1 -- 172.21.15.12:0/1383735693 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56483283d290 con 0x5648327c7e10 2024-04-22T13:28:00.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.673+0000 7f008f7fe640 1 --2- 172.21.15.12:0/1383735693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648327ba250 0x5648327cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:00.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.673+0000 7f008effd640 1 --2- 172.21.15.12:0/1383735693 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5648327bac00 0x5648327cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:00.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.673+0000 7f008ffff640 1 --2- 172.21.15.12:0/1383735693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648327c7e10 0x5648327cc6d0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f008800fae0 tx=0x7f008800a830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d17568f7a0c2c20 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:00.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.673+0000 7f006effd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:00.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.673+0000 7f006effd640 1 -- 172.21.15.12:0/1383735693 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008801c9f0 con 0x5648327c7e10 2024-04-22T13:28:00.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.673+0000 7f006effd640 1 -- 172.21.15.12:0/1383735693 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f008801cb90 con 0x5648327c7e10 2024-04-22T13:28:00.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.673+0000 7f009886c740 1 -- 172.21.15.12:0/1383735693 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f008009cae0 con 0x5648327c7e10 2024-04-22T13:28:00.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.674+0000 7f006effd640 1 -- 172.21.15.12:0/1383735693 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f008801cd90 con 0x5648327c7e10 2024-04-22T13:28:00.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.674+0000 7f009886c740 10 client.17121 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:00.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.674+0000 7f009886c740 1 -- 172.21.15.12:0/1383735693 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56483283e520 con 0x5648327c7e10 2024-04-22T13:28:00.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.674+0000 7f006effd640 1 -- 172.21.15.12:0/1383735693 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f008009cae0 con 0x5648327c7e10 2024-04-22T13:28:00.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.674+0000 7f006effd640 1 -- 172.21.15.12:0/1383735693 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f008801d4a0 con 0x5648327c7e10 2024-04-22T13:28:00.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.674+0000 7f009886c740 10 client.17121 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:00.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.674+0000 7f009886c740 10 client.17121 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:28:00.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.674+0000 7f009886c740 20 client.17121 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:00.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.674+0000 7f009886c740 1 --2- 172.21.15.12:0/1383735693 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56483283ecd0 0x5648328410f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.674+0000 7f009886c740 4 client.17121 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:28:00.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.674+0000 7f009886c740 1 -- 172.21.15.12:0/1383735693 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56483272cc10 con 0x56483283ecd0 2024-04-22T13:28:00.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.674+0000 7f008f7fe640 1 --2- 172.21.15.12:0/1383735693 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56483283ecd0 0x5648328410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.675+0000 7f006effd640 10 client.17121 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:28:00.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.675+0000 7f008f7fe640 1 --2- 172.21.15.12:0/1383735693 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56483283ecd0 0x5648328410f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f84ecba0190f3ee server_cookie=360597da7a536f5f in_seq=0 out_seq=0 2024-04-22T13:28:00.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.675+0000 7f006effd640 1 -- 172.21.15.12:0/1383735693 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56483272cc10 con 0x56483283ecd0 2024-04-22T13:28:00.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.675+0000 7f006effd640 10 client.17121 handle_command_reply: tid=0 2024-04-22T13:28:00.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.675+0000 7f009886c740 1 client.17121 shutdown 2024-04-22T13:28:00.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.675+0000 7f009886c740 1 -- 172.21.15.12:0/1383735693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648327c7e10 msgr2=0x5648327cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.675+0000 7f009886c740 1 --2- 172.21.15.12:0/1383735693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648327c7e10 0x5648327cc6d0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f008800fae0 tx=0x7f008800a830 comp rx=0 tx=0).stop 2024-04-22T13:28:00.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.675+0000 7f009886c740 1 -- 172.21.15.12:0/1383735693 shutdown_connections 2024-04-22T13:28:00.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.675+0000 7f009886c740 1 --2- 172.21.15.12:0/1383735693 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56483283ecd0 0x5648328410f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.675+0000 7f009886c740 1 --2- 172.21.15.12:0/1383735693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5648327c7e10 0x5648327cc6d0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.675+0000 7f009886c740 1 --2- 172.21.15.12:0/1383735693 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5648327bac00 0x5648327cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.675+0000 7f006effd640 0 client.17121 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:28:00.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.675+0000 7f009886c740 1 --2- 172.21.15.12:0/1383735693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5648327ba250 0x5648327cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.675+0000 7f009886c740 1 -- 172.21.15.12:0/1383735693 >> 172.21.15.12:0/1383735693 conn(0x5648327b4ba0 msgr2=0x5648327c0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:00.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.675+0000 7f009886c740 1 -- 172.21.15.12:0/1383735693 shutdown_connections 2024-04-22T13:28:00.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.675+0000 7f009886c740 1 -- 172.21.15.12:0/1383735693 wait complete. 2024-04-22T13:28:00.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.675+0000 7f009886c740 20 client.17121 trim_cache size 0 max 16384 2024-04-22T13:28:00.679 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:28:00.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.676+0000 7f0095c74640 1 -- 172.21.15.12:0/2760400374 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f00780a3d20 msgr2=0x7f00780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.676+0000 7f0095c74640 1 --2- 172.21.15.12:0/2760400374 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f00780a3d20 0x7f00780a61e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0088005fd0 tx=0x7f0088005c80 comp rx=0 tx=0).stop 2024-04-22T13:28:00.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.676+0000 7f0095c74640 1 -- 172.21.15.12:0/2760400374 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0090101090 msgr2=0x7f0090198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.676+0000 7f0095c74640 1 --2- 172.21.15.12:0/2760400374 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0090101090 0x7f0090198740 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f008000d950 tx=0x7f008000de10 comp rx=0 tx=0).stop 2024-04-22T13:28:00.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.676+0000 7f0095c74640 1 -- 172.21.15.12:0/2760400374 shutdown_connections 2024-04-22T13:28:00.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.676+0000 7f0095c74640 1 --2- 172.21.15.12:0/2760400374 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f00780a3d20 0x7f00780a61e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.676+0000 7f0095c74640 1 --2- 172.21.15.12:0/2760400374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f009019d000 0x7f009019d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.676+0000 7f0095c74640 1 --2- 172.21.15.12:0/2760400374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0090106160 0x7f0090198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.676+0000 7f0095c74640 1 --2- 172.21.15.12:0/2760400374 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0090101090 0x7f0090198740 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.676+0000 7f0095c74640 1 -- 172.21.15.12:0/2760400374 >> 172.21.15.12:0/2760400374 conn(0x7f0090077f50 msgr2=0x7f00900ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:00.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.676+0000 7f0095c74640 1 -- 172.21.15.12:0/2760400374 shutdown_connections 2024-04-22T13:28:00.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.676+0000 7f0095c74640 1 -- 172.21.15.12:0/2760400374 wait complete. 2024-04-22T13:28:00.681 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:00 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1482517559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:28:00.682 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:00 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1482517559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:28:00.683 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:28:00.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:00 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1482517559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:28:00.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.790+0000 7ff559a0f640 1 -- 172.21.15.12:0/2995519515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff554101a50 msgr2=0x7ff554105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.790+0000 7ff559a0f640 1 --2- 172.21.15.12:0/2995519515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff554101a50 0x7ff554105a90 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009900 tx=0x7ff53c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:00.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.790+0000 7ff559a0f640 1 -- 172.21.15.12:0/2995519515 shutdown_connections 2024-04-22T13:28:00.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.790+0000 7ff559a0f640 1 --2- 172.21.15.12:0/2995519515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff554106170 0x7ff55410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.790+0000 7ff559a0f640 1 --2- 172.21.15.12:0/2995519515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff554101a50 0x7ff554105a90 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.790+0000 7ff559a0f640 1 --2- 172.21.15.12:0/2995519515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5541010a0 0x7ff554101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.790+0000 7ff559a0f640 1 -- 172.21.15.12:0/2995519515 >> 172.21.15.12:0/2995519515 conn(0x7ff554077eb0 msgr2=0x7ff5540782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:00.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.790+0000 7ff559a0f640 1 -- 172.21.15.12:0/2995519515 shutdown_connections 2024-04-22T13:28:00.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.790+0000 7ff559a0f640 1 -- 172.21.15.12:0/2995519515 wait complete. 2024-04-22T13:28:00.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.791+0000 7ff559a0f640 1 Processor -- start 2024-04-22T13:28:00.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.791+0000 7ff559a0f640 1 -- start start 2024-04-22T13:28:00.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.791+0000 7ff559a0f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5541010a0 0x7ff554199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.791+0000 7ff559a0f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff554106170 0x7ff554199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.791+0000 7ff559a0f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff55419a0f0 0x7ff5541a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.791+0000 7ff559a0f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff5541110b0 con 0x7ff554106170 2024-04-22T13:28:00.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.791+0000 7ff559a0f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff554110f30 con 0x7ff55419a0f0 2024-04-22T13:28:00.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.792+0000 7ff559a0f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff554111230 con 0x7ff5541010a0 2024-04-22T13:28:00.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.792+0000 7ff5527fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff554106170 0x7ff554199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.792+0000 7ff5527fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff554106170 0x7ff554199b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52306/0 (socket says 172.21.15.12:52306) 2024-04-22T13:28:00.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.792+0000 7ff5527fc640 1 -- 172.21.15.12:0/2776951115 learned_addr learned my addr 172.21.15.12:0/2776951115 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:00.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.792+0000 7ff552ffd640 1 --2- 172.21.15.12:0/2776951115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5541010a0 0x7ff554199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.792+0000 7ff5537fe640 1 --2- 172.21.15.12:0/2776951115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff55419a0f0 0x7ff5541a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.792+0000 7ff5527fc640 1 -- 172.21.15.12:0/2776951115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5541010a0 msgr2=0x7ff554199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.792+0000 7ff5527fc640 1 --2- 172.21.15.12:0/2776951115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5541010a0 0x7ff554199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.792+0000 7ff5527fc640 1 -- 172.21.15.12:0/2776951115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff55419a0f0 msgr2=0x7ff5541a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.792+0000 7ff5527fc640 1 --2- 172.21.15.12:0/2776951115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff55419a0f0 0x7ff5541a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.792+0000 7ff5527fc640 1 -- 172.21.15.12:0/2776951115 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5541a4ea0 con 0x7ff554106170 2024-04-22T13:28:00.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.792+0000 7ff5537fe640 1 --2- 172.21.15.12:0/2776951115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff55419a0f0 0x7ff5541a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:00.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.792+0000 7ff5527fc640 1 --2- 172.21.15.12:0/2776951115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff554106170 0x7ff554199b60 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7ff53c036040 tx=0x7ff53c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:00.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.792+0000 7ff558a0d640 1 -- 172.21.15.12:0/2776951115 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c043020 con 0x7ff554106170 2024-04-22T13:28:00.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.792+0000 7ff558a0d640 1 -- 172.21.15.12:0/2776951115 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff53c02cb70 con 0x7ff554106170 2024-04-22T13:28:00.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.792+0000 7ff559a0f640 1 -- 172.21.15.12:0/2776951115 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5541a5130 con 0x7ff554106170 2024-04-22T13:28:00.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.792+0000 7ff558a0d640 1 -- 172.21.15.12:0/2776951115 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c003bf0 con 0x7ff554106170 2024-04-22T13:28:00.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.792+0000 7ff559a0f640 1 -- 172.21.15.12:0/2776951115 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5541a5680 con 0x7ff554106170 2024-04-22T13:28:00.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.795+0000 7ff558a0d640 1 -- 172.21.15.12:0/2776951115 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff53c046020 con 0x7ff554106170 2024-04-22T13:28:00.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.795+0000 7ff55c5fd740 1 Processor -- start 2024-04-22T13:28:00.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.795+0000 7ff55c5fd740 1 -- start start 2024-04-22T13:28:00.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.795+0000 7ff55c5fd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584ba9f14d0 0x5584ba9f18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.795+0000 7ff55c5fd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584ba9f1e80 0x5584ba9fa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.795+0000 7ff55c5fd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5584ba9faa00 0x5584ba9fce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.795+0000 7ff55c5fd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5584ba9561f0 con 0x5584ba9f14d0 2024-04-22T13:28:00.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.795+0000 7ff55c5fd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5584ba8f34b0 con 0x5584ba9faa00 2024-04-22T13:28:00.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.795+0000 7ff55c5fd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5584ba9521b0 con 0x5584ba9f1e80 2024-04-22T13:28:00.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.795+0000 7ff5537fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584ba9f14d0 0x5584ba9f18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.795+0000 7ff5537fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584ba9f14d0 0x5584ba9f18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52310/0 (socket says 172.21.15.12:52310) 2024-04-22T13:28:00.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.795+0000 7ff5537fe640 1 -- 172.21.15.12:0/604317825 learned_addr learned my addr 172.21.15.12:0/604317825 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:00.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.795+0000 7ff5527fc640 1 --2- 172.21.15.12:0/604317825 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5584ba9faa00 0x5584ba9fce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.795+0000 7ff552ffd640 1 --2- 172.21.15.12:0/604317825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584ba9f1e80 0x5584ba9fa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.795+0000 7ff558a0d640 1 --2- 172.21.15.12:0/2776951115 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff5400a3d20 0x7ff5400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.795+0000 7ff5537fe640 1 -- 172.21.15.12:0/604317825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584ba9f1e80 msgr2=0x5584ba9fa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.795+0000 7ff5537fe640 1 --2- 172.21.15.12:0/604317825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584ba9f1e80 0x5584ba9fa430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.795+0000 7ff5537fe640 1 -- 172.21.15.12:0/604317825 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5584ba9faa00 msgr2=0x5584ba9fce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.795+0000 7ff5537fe640 1 --2- 172.21.15.12:0/604317825 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5584ba9faa00 0x5584ba9fce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.795+0000 7ff5537fe640 1 -- 172.21.15.12:0/604317825 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584ba8e51b0 con 0x5584ba9f14d0 2024-04-22T13:28:00.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.795+0000 7ff5527fc640 1 --2- 172.21.15.12:0/604317825 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5584ba9faa00 0x5584ba9fce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:00.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.796+0000 7ff558a0d640 1 -- 172.21.15.12:0/2776951115 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7ff53c0f2850 con 0x7ff554106170 2024-04-22T13:28:00.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.796+0000 7ff5537fe640 1 --2- 172.21.15.12:0/2776951115 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff5400a3d20 0x7ff5400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.796+0000 7ff5537fe640 1 --2- 172.21.15.12:0/604317825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584ba9f14d0 0x5584ba9f18b0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ff54400f4b0 tx=0x7ff54400f970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac44b12972863064 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:00.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.796+0000 7ff549ffb640 1 -- 172.21.15.12:0/604317825 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54401c400 con 0x5584ba9f14d0 2024-04-22T13:28:00.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.796+0000 7ff549ffb640 1 -- 172.21.15.12:0/604317825 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff54401ca50 con 0x5584ba9f14d0 2024-04-22T13:28:00.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.796+0000 7ff549ffb640 1 -- 172.21.15.12:0/604317825 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54401b780 con 0x5584ba9f14d0 2024-04-22T13:28:00.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.797+0000 7ff55c5fd740 1 -- 172.21.15.12:0/604317825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584ba9f14d0 msgr2=0x5584ba9f18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.797+0000 7ff55c5fd740 1 --2- 172.21.15.12:0/604317825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584ba9f14d0 0x5584ba9f18b0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ff54400f4b0 tx=0x7ff54400f970 comp rx=0 tx=0).stop 2024-04-22T13:28:00.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.797+0000 7ff5537fe640 1 --2- 172.21.15.12:0/2776951115 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff5400a3d20 0x7ff5400a61e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff544020000 tx=0x7ff54401d000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:00.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.797+0000 7ff55c5fd740 1 -- 172.21.15.12:0/604317825 shutdown_connections 2024-04-22T13:28:00.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.797+0000 7ff55c5fd740 1 --2- 172.21.15.12:0/604317825 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5584ba9faa00 0x5584ba9fce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.797+0000 7ff55c5fd740 1 --2- 172.21.15.12:0/604317825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584ba9f1e80 0x5584ba9fa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.797+0000 7ff55c5fd740 1 --2- 172.21.15.12:0/604317825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584ba9f14d0 0x5584ba9f18b0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.797+0000 7ff55c5fd740 1 -- 172.21.15.12:0/604317825 >> 172.21.15.12:0/604317825 conn(0x5584ba9ebba0 msgr2=0x5584ba9ebf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:00.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.797+0000 7ff55c5fd740 1 -- 172.21.15.12:0/604317825 shutdown_connections 2024-04-22T13:28:00.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.797+0000 7ff55c5fd740 1 -- 172.21.15.12:0/604317825 wait complete. 2024-04-22T13:28:00.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.797+0000 7ff55c5fd740 1 -- start start 2024-04-22T13:28:00.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.797+0000 7ff55c5fd740 1 Processor -- start 2024-04-22T13:28:00.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.797+0000 7ff55c5fd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584ba9f14d0 0x5584ba9fa390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.797+0000 7ff55c5fd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584ba9f1e80 0x5584ba9f7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.798+0000 7ff55c5fd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5584ba9faa00 0x5584ba9f7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.798+0000 7ff55c5fd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5584ba9fd970 con 0x5584ba9f14d0 2024-04-22T13:28:00.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.798+0000 7ff55c5fd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5584ba90f500 con 0x5584ba9faa00 2024-04-22T13:28:00.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.798+0000 7ff55c5fd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5584ba9fdaf0 con 0x5584ba9f1e80 2024-04-22T13:28:00.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.798+0000 7ff55c5fd740 10 client.0 _ll_register_callbacks cb 0x5584ba8ddb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:00.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.798+0000 7ff552ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584ba9f14d0 0x5584ba9fa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.798+0000 7ff552ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584ba9f14d0 0x5584ba9fa390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52326/0 (socket says 172.21.15.12:52326) 2024-04-22T13:28:00.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.798+0000 7ff552ffd640 1 -- 172.21.15.12:0/1289586995 learned_addr learned my addr 172.21.15.12:0/1289586995 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:00.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.798+0000 7ff5537fe640 1 --2- 172.21.15.12:0/1289586995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5584ba9faa00 0x5584ba9f7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.798+0000 7ff5527fc640 1 --2- 172.21.15.12:0/1289586995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584ba9f1e80 0x5584ba9f7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.798+0000 7ff552ffd640 1 -- 172.21.15.12:0/1289586995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584ba9f1e80 msgr2=0x5584ba9f7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.798+0000 7ff552ffd640 1 --2- 172.21.15.12:0/1289586995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584ba9f1e80 0x5584ba9f7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.798+0000 7ff552ffd640 1 -- 172.21.15.12:0/1289586995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5584ba9faa00 msgr2=0x5584ba9f7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.798+0000 7ff552ffd640 1 --2- 172.21.15.12:0/1289586995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5584ba9faa00 0x5584ba9f7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.798+0000 7ff552ffd640 1 -- 172.21.15.12:0/1289586995 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584ba6683f0 con 0x5584ba9f14d0 2024-04-22T13:28:00.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.798+0000 7ff5527fc640 1 --2- 172.21.15.12:0/1289586995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584ba9f1e80 0x5584ba9f7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:00.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.798+0000 7ff552ffd640 1 --2- 172.21.15.12:0/1289586995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584ba9f14d0 0x5584ba9fa390 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7ff54c012700 tx=0x7ff54c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8fa3001dee5a119 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:00.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.798+0000 7ff532ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:00.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.798+0000 7ff532ffd640 1 -- 172.21.15.12:0/1289586995 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c004050 con 0x5584ba9f14d0 2024-04-22T13:28:00.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.798+0000 7ff532ffd640 1 -- 172.21.15.12:0/1289586995 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7ff54c0041f0 con 0x5584ba9f14d0 2024-04-22T13:28:00.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.798+0000 7ff55c5fd740 1 -- 172.21.15.12:0/1289586995 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff54401ca50 con 0x5584ba9f14d0 2024-04-22T13:28:00.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.798+0000 7ff532ffd640 1 -- 172.21.15.12:0/1289586995 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff54c0043f0 con 0x5584ba9f14d0 2024-04-22T13:28:00.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.798+0000 7ff55c5fd740 10 client.17133 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:00.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.798+0000 7ff55c5fd740 1 -- 172.21.15.12:0/1289586995 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5584ba7327d0 con 0x5584ba9f14d0 2024-04-22T13:28:00.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.799+0000 7ff532ffd640 1 -- 172.21.15.12:0/1289586995 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff54401ca50 con 0x5584ba9f14d0 2024-04-22T13:28:00.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.799+0000 7ff532ffd640 1 -- 172.21.15.12:0/1289586995 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff54c003c10 con 0x5584ba9f14d0 2024-04-22T13:28:00.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.799+0000 7ff55c5fd740 10 client.17133 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:00.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.799+0000 7ff55c5fd740 10 client.17133 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:00.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.799+0000 7ff55c5fd740 20 client.17133 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:00.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.799+0000 7ff55c5fd740 1 --2- 172.21.15.12:0/1289586995 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5584baa0ac70 0x5584baa0d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.799+0000 7ff55c5fd740 4 client.17133 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:00.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.799+0000 7ff55c5fd740 1 -- 172.21.15.12:0/1289586995 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584ba8f5930 con 0x5584baa0ac70 2024-04-22T13:28:00.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.799+0000 7ff5527fc640 1 --2- 172.21.15.12:0/1289586995 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5584baa0ac70 0x5584baa0d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.799+0000 7ff532ffd640 10 client.17133 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:00.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.799+0000 7ff5527fc640 1 --2- 172.21.15.12:0/1289586995 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5584baa0ac70 0x5584baa0d0c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea7be5a12e558632 server_cookie=34eeb57f73e16ef3 in_seq=0 out_seq=0 2024-04-22T13:28:00.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.800+0000 7ff532ffd640 1 -- 172.21.15.12:0/1289586995 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5584ba8f5930 con 0x5584baa0ac70 2024-04-22T13:28:00.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.800+0000 7ff532ffd640 10 client.17133 handle_command_reply: tid=0 2024-04-22T13:28:00.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.800+0000 7ff55c5fd740 1 client.17133 shutdown 2024-04-22T13:28:00.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.800+0000 7ff55c5fd740 1 -- 172.21.15.12:0/1289586995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584ba9f14d0 msgr2=0x5584ba9fa390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.800+0000 7ff55c5fd740 1 --2- 172.21.15.12:0/1289586995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584ba9f14d0 0x5584ba9fa390 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7ff54c012700 tx=0x7ff54c012bc0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.800+0000 7ff55c5fd740 1 -- 172.21.15.12:0/1289586995 shutdown_connections 2024-04-22T13:28:00.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.800+0000 7ff55c5fd740 1 --2- 172.21.15.12:0/1289586995 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5584baa0ac70 0x5584baa0d0c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.800+0000 7ff55c5fd740 1 --2- 172.21.15.12:0/1289586995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5584ba9faa00 0x5584ba9f7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.800+0000 7ff55c5fd740 1 --2- 172.21.15.12:0/1289586995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584ba9f1e80 0x5584ba9f7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.800+0000 7ff55c5fd740 1 --2- 172.21.15.12:0/1289586995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584ba9f14d0 0x5584ba9fa390 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.800+0000 7ff532ffd640 0 client.17133 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:00.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.800+0000 7ff55c5fd740 1 -- 172.21.15.12:0/1289586995 >> 172.21.15.12:0/1289586995 conn(0x5584ba9ebba0 msgr2=0x5584ba9ed8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:00.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.801+0000 7ff55c5fd740 1 -- 172.21.15.12:0/1289586995 shutdown_connections 2024-04-22T13:28:00.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.801+0000 7ff55c5fd740 1 -- 172.21.15.12:0/1289586995 wait complete. 2024-04-22T13:28:00.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.801+0000 7ff55c5fd740 20 client.17133 trim_cache size 0 max 16384 2024-04-22T13:28:00.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.814+0000 7ff55c5fd740 1 Processor -- start 2024-04-22T13:28:00.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.814+0000 7ff55c5fd740 1 -- start start 2024-04-22T13:28:00.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.814+0000 7ff55c5fd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5584ba9f14d0 0x5584baa06860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.814+0000 7ff55c5fd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584ba9f1e80 0x5584ba9f6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.814+0000 7ff55c5fd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584ba9faa00 0x5584ba9eaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.814+0000 7ff55c5fd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff5541ada70 con 0x5584ba9faa00 2024-04-22T13:28:00.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.814+0000 7ff55c5fd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff5541af5b0 con 0x5584ba9f14d0 2024-04-22T13:28:00.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.814+0000 7ff55c5fd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff5541ae140 con 0x5584ba9f1e80 2024-04-22T13:28:00.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.814+0000 7ff5537fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584ba9faa00 0x5584ba9eaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.814+0000 7ff5537fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584ba9faa00 0x5584ba9eaa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52330/0 (socket says 172.21.15.12:52330) 2024-04-22T13:28:00.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.814+0000 7ff5537fe640 1 -- 172.21.15.12:0/3411672352 learned_addr learned my addr 172.21.15.12:0/3411672352 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:00.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.814+0000 7ff5527fc640 1 --2- 172.21.15.12:0/3411672352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584ba9f1e80 0x5584ba9f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.814+0000 7ff552ffd640 1 --2- 172.21.15.12:0/3411672352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5584ba9f14d0 0x5584baa06860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.814+0000 7ff5537fe640 1 -- 172.21.15.12:0/3411672352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584ba9f1e80 msgr2=0x5584ba9f6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.814+0000 7ff5537fe640 1 --2- 172.21.15.12:0/3411672352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584ba9f1e80 0x5584ba9f6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.814+0000 7ff5537fe640 1 -- 172.21.15.12:0/3411672352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5584ba9f14d0 msgr2=0x5584baa06860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.814+0000 7ff5537fe640 1 --2- 172.21.15.12:0/3411672352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5584ba9f14d0 0x5584baa06860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.814+0000 7ff5537fe640 1 -- 172.21.15.12:0/3411672352 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584ba9ede90 con 0x5584ba9faa00 2024-04-22T13:28:00.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.814+0000 7ff552ffd640 1 --2- 172.21.15.12:0/3411672352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5584ba9f14d0 0x5584baa06860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:00.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.815+0000 7ff5537fe640 1 --2- 172.21.15.12:0/3411672352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584ba9faa00 0x5584ba9eaa80 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7ff54400fe70 tx=0x7ff544027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb7aa74dd528c93 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:00.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.815+0000 7ff5497fa640 1 -- 172.21.15.12:0/3411672352 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff544034020 con 0x5584ba9faa00 2024-04-22T13:28:00.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.815+0000 7ff5497fa640 1 -- 172.21.15.12:0/3411672352 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff54402a040 con 0x5584ba9faa00 2024-04-22T13:28:00.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.815+0000 7ff5497fa640 1 -- 172.21.15.12:0/3411672352 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff544004e60 con 0x5584ba9faa00 2024-04-22T13:28:00.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.815+0000 7ff55c5fd740 1 -- 172.21.15.12:0/3411672352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584ba9faa00 msgr2=0x5584ba9eaa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.815+0000 7ff55c5fd740 1 --2- 172.21.15.12:0/3411672352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584ba9faa00 0x5584ba9eaa80 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7ff54400fe70 tx=0x7ff544027000 comp rx=0 tx=0).stop 2024-04-22T13:28:00.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.815+0000 7ff55c5fd740 1 -- 172.21.15.12:0/3411672352 shutdown_connections 2024-04-22T13:28:00.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.815+0000 7ff55c5fd740 1 --2- 172.21.15.12:0/3411672352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584ba9faa00 0x5584ba9eaa80 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.815+0000 7ff55c5fd740 1 --2- 172.21.15.12:0/3411672352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584ba9f1e80 0x5584ba9f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.815+0000 7ff55c5fd740 1 --2- 172.21.15.12:0/3411672352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5584ba9f14d0 0x5584baa06860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.815+0000 7ff55c5fd740 1 -- 172.21.15.12:0/3411672352 >> 172.21.15.12:0/3411672352 conn(0x5584ba9ebba0 msgr2=0x5584ba9ea370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:00.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.815+0000 7ff55c5fd740 1 -- 172.21.15.12:0/3411672352 shutdown_connections 2024-04-22T13:28:00.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.815+0000 7ff55c5fd740 1 -- 172.21.15.12:0/3411672352 wait complete. 2024-04-22T13:28:00.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.816+0000 7ff55c5fd740 1 -- start start 2024-04-22T13:28:00.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.816+0000 7ff55c5fd740 1 Processor -- start 2024-04-22T13:28:00.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.816+0000 7ff55c5fd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584ba9f14d0 0x5584baa02d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.816+0000 7ff55c5fd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5584ba9f1e80 0x5584baa032b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.816+0000 7ff55c5fd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584ba9faa00 0x5584ba9f3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.816+0000 7ff55c5fd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff5541aeee0 con 0x5584ba9f14d0 2024-04-22T13:28:00.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.816+0000 7ff55c5fd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff5541ae810 con 0x5584ba9f1e80 2024-04-22T13:28:00.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.816+0000 7ff55c5fd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff5541aba70 con 0x5584ba9faa00 2024-04-22T13:28:00.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.816+0000 7ff55c5fd740 10 client.0 _ll_register_callbacks cb 0x7ff54c009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:00.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.816+0000 7ff552ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584ba9f14d0 0x5584baa02d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.816+0000 7ff552ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584ba9f14d0 0x5584baa02d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52332/0 (socket says 172.21.15.12:52332) 2024-04-22T13:28:00.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.816+0000 7ff552ffd640 1 -- 172.21.15.12:0/3151667532 learned_addr learned my addr 172.21.15.12:0/3151667532 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:00.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.816+0000 7ff5537fe640 1 --2- 172.21.15.12:0/3151667532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584ba9faa00 0x5584ba9f3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.817+0000 7ff5527fc640 1 --2- 172.21.15.12:0/3151667532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5584ba9f1e80 0x5584baa032b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.817+0000 7ff552ffd640 1 -- 172.21.15.12:0/3151667532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584ba9faa00 msgr2=0x5584ba9f3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.817+0000 7ff552ffd640 1 --2- 172.21.15.12:0/3151667532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584ba9faa00 0x5584ba9f3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.817+0000 7ff552ffd640 1 -- 172.21.15.12:0/3151667532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5584ba9f1e80 msgr2=0x5584baa032b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:00.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.817+0000 7ff552ffd640 1 --2- 172.21.15.12:0/3151667532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5584ba9f1e80 0x5584baa032b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.817+0000 7ff552ffd640 1 -- 172.21.15.12:0/3151667532 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584ba9f47a0 con 0x5584ba9f14d0 2024-04-22T13:28:00.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.817+0000 7ff5537fe640 1 --2- 172.21.15.12:0/3151667532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584ba9faa00 0x5584ba9f3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:00.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.817+0000 7ff5527fc640 1 --2- 172.21.15.12:0/3151667532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5584ba9f1e80 0x5584baa032b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:00.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.817+0000 7ff552ffd640 1 --2- 172.21.15.12:0/3151667532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584ba9f14d0 0x5584baa02d70 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009e80 tx=0x7ff54c025850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d7e67bc75a0345d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:00.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.817+0000 7ff532ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:00.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.817+0000 7ff532ffd640 1 -- 172.21.15.12:0/3151667532 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c025c50 con 0x5584ba9f14d0 2024-04-22T13:28:00.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.817+0000 7ff532ffd640 1 -- 172.21.15.12:0/3151667532 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7ff54c018040 con 0x5584ba9f14d0 2024-04-22T13:28:00.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.817+0000 7ff55c5fd740 1 -- 172.21.15.12:0/3151667532 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff54402a040 con 0x5584ba9f14d0 2024-04-22T13:28:00.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.817+0000 7ff532ffd640 1 -- 172.21.15.12:0/3151667532 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff54c025df0 con 0x5584ba9f14d0 2024-04-22T13:28:00.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.817+0000 7ff55c5fd740 10 client.17139 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:00.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.817+0000 7ff55c5fd740 1 -- 172.21.15.12:0/3151667532 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5584baa70e30 con 0x5584ba9f14d0 2024-04-22T13:28:00.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.817+0000 7ff532ffd640 1 -- 172.21.15.12:0/3151667532 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff54402a040 con 0x5584ba9f14d0 2024-04-22T13:28:00.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.817+0000 7ff532ffd640 1 -- 172.21.15.12:0/3151667532 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff54c0388b0 con 0x5584ba9f14d0 2024-04-22T13:28:00.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.817+0000 7ff55c5fd740 10 client.17139 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:00.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.817+0000 7ff55c5fd740 10 client.17139 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:00.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.817+0000 7ff55c5fd740 20 client.17139 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:00.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.817+0000 7ff55c5fd740 1 --2- 172.21.15.12:0/3151667532 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5584baa71590 0x5584baa739b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:00.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.817+0000 7ff55c5fd740 4 client.17139 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:28:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.817+0000 7ff55c5fd740 1 -- 172.21.15.12:0/3151667532 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5584ba963c10 con 0x5584baa71590 2024-04-22T13:28:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.818+0000 7ff5527fc640 1 --2- 172.21.15.12:0/3151667532 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5584baa71590 0x5584baa739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.818+0000 7ff5527fc640 1 --2- 172.21.15.12:0/3151667532 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5584baa71590 0x5584baa739b0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d99dbe50a01c9ec server_cookie=d724acc87507f8a1 in_seq=0 out_seq=0 2024-04-22T13:28:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.818+0000 7ff532ffd640 10 client.17139 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:00.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.818+0000 7ff532ffd640 1 -- 172.21.15.12:0/3151667532 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5584ba963c10 con 0x5584baa71590 2024-04-22T13:28:00.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.818+0000 7ff532ffd640 10 client.17139 handle_command_reply: tid=0 2024-04-22T13:28:00.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.818+0000 7ff55c5fd740 1 client.17139 shutdown 2024-04-22T13:28:00.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.819+0000 7ff55c5fd740 1 -- 172.21.15.12:0/3151667532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584ba9f14d0 msgr2=0x5584baa02d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.819+0000 7ff55c5fd740 1 --2- 172.21.15.12:0/3151667532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584ba9f14d0 0x5584baa02d70 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009e80 tx=0x7ff54c025850 comp rx=0 tx=0).stop 2024-04-22T13:28:00.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.819+0000 7ff55c5fd740 1 -- 172.21.15.12:0/3151667532 shutdown_connections 2024-04-22T13:28:00.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.819+0000 7ff55c5fd740 1 --2- 172.21.15.12:0/3151667532 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5584baa71590 0x5584baa739b0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.819+0000 7ff55c5fd740 1 --2- 172.21.15.12:0/3151667532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5584ba9faa00 0x5584ba9f3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.819+0000 7ff532ffd640 0 client.17139 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:00.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.819+0000 7ff55c5fd740 1 --2- 172.21.15.12:0/3151667532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5584ba9f1e80 0x5584baa032b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.819+0000 7ff55c5fd740 1 --2- 172.21.15.12:0/3151667532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5584ba9f14d0 0x5584baa02d70 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.819+0000 7ff55c5fd740 1 -- 172.21.15.12:0/3151667532 >> 172.21.15.12:0/3151667532 conn(0x5584ba9ebba0 msgr2=0x5584ba9ea370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:00.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.819+0000 7ff55c5fd740 1 -- 172.21.15.12:0/3151667532 shutdown_connections 2024-04-22T13:28:00.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.819+0000 7ff55c5fd740 1 -- 172.21.15.12:0/3151667532 wait complete. 2024-04-22T13:28:00.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.819+0000 7ff55c5fd740 20 client.17139 trim_cache size 0 max 16384 2024-04-22T13:28:00.826 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:28:00.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.819+0000 7ff559a0f640 1 -- 172.21.15.12:0/2776951115 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff5400a3d20 msgr2=0x7ff5400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.819+0000 7ff559a0f640 1 --2- 172.21.15.12:0/2776951115 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff5400a3d20 0x7ff5400a61e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff544020000 tx=0x7ff54401d000 comp rx=0 tx=0).stop 2024-04-22T13:28:00.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.819+0000 7ff559a0f640 1 -- 172.21.15.12:0/2776951115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff554106170 msgr2=0x7ff554199b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:00.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.819+0000 7ff559a0f640 1 --2- 172.21.15.12:0/2776951115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff554106170 0x7ff554199b60 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7ff53c036040 tx=0x7ff53c0045e0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.819+0000 7ff559a0f640 1 -- 172.21.15.12:0/2776951115 shutdown_connections 2024-04-22T13:28:00.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.819+0000 7ff559a0f640 1 --2- 172.21.15.12:0/2776951115 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff5400a3d20 0x7ff5400a61e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.819+0000 7ff559a0f640 1 --2- 172.21.15.12:0/2776951115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff55419a0f0 0x7ff5541a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.819+0000 7ff559a0f640 1 --2- 172.21.15.12:0/2776951115 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff554106170 0x7ff554199b60 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.819+0000 7ff559a0f640 1 --2- 172.21.15.12:0/2776951115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5541010a0 0x7ff554199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:00.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.819+0000 7ff559a0f640 1 -- 172.21.15.12:0/2776951115 >> 172.21.15.12:0/2776951115 conn(0x7ff554077eb0 msgr2=0x7ff5540ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:00.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.819+0000 7ff559a0f640 1 -- 172.21.15.12:0/2776951115 shutdown_connections 2024-04-22T13:28:00.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:00.819+0000 7ff559a0f640 1 -- 172.21.15.12:0/2776951115 wait complete. 2024-04-22T13:28:01.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:01 smithi134 ceph-mon[31338]: pgmap v120: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:01.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:01 smithi111 ceph-mon[31361]: pgmap v120: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:01.831 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:28:01.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:28:01.832 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:28:01.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:01 smithi012 ceph-mon[28265]: pgmap v120: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:01.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.955+0000 7f5fa7d04640 1 -- 172.21.15.12:0/4100114301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fa0107260 msgr2=0x7f5fa010ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:01.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.955+0000 7f5fa7d04640 1 --2- 172.21.15.12:0/4100114301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fa0107260 0x7f5fa010ee30 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f5f98009ca0 tx=0x7f5f9802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.955+0000 7f5fa7d04640 1 -- 172.21.15.12:0/4100114301 shutdown_connections 2024-04-22T13:28:01.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.955+0000 7f5fa7d04640 1 --2- 172.21.15.12:0/4100114301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fa0107260 0x7f5fa010ee30 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.955+0000 7f5fa7d04640 1 --2- 172.21.15.12:0/4100114301 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fa0102b40 0x7f5fa0106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.955+0000 7f5fa7d04640 1 --2- 172.21.15.12:0/4100114301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fa0102190 0x7f5fa0102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.955+0000 7f5fa7d04640 1 -- 172.21.15.12:0/4100114301 >> 172.21.15.12:0/4100114301 conn(0x7f5fa00fd990 msgr2=0x7f5fa00ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:01.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.955+0000 7f5fa7d04640 1 -- 172.21.15.12:0/4100114301 shutdown_connections 2024-04-22T13:28:01.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.955+0000 7f5fa7d04640 1 -- 172.21.15.12:0/4100114301 wait complete. 2024-04-22T13:28:01.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.956+0000 7f5fa7d04640 1 Processor -- start 2024-04-22T13:28:01.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.956+0000 7f5fa7d04640 1 -- start start 2024-04-22T13:28:01.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.956+0000 7f5fa7d04640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fa0102190 0x7f5fa0198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:01.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.956+0000 7f5fa7d04640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fa0107260 0x7f5fa0198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:01.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.956+0000 7f5fa7d04640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fa019d010 0x7f5fa019d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:01.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.956+0000 7f5fa7d04640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa01121d0 con 0x7f5fa0107260 2024-04-22T13:28:01.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.956+0000 7f5fa7d04640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa0112050 con 0x7f5fa019d010 2024-04-22T13:28:01.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.956+0000 7f5fa7d04640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa0112350 con 0x7f5fa0102190 2024-04-22T13:28:01.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.956+0000 7f5fa529f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fa0107260 0x7f5fa0198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:01.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.956+0000 7f5fa529f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fa0107260 0x7f5fa0198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36458/0 (socket says 172.21.15.12:36458) 2024-04-22T13:28:01.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.956+0000 7f5fa529f640 1 -- 172.21.15.12:0/3777758597 learned_addr learned my addr 172.21.15.12:0/3777758597 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:01.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.956+0000 7f5fa5aa0640 1 --2- 172.21.15.12:0/3777758597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fa0102190 0x7f5fa0198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:01.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.956+0000 7f5fa62a1640 1 --2- 172.21.15.12:0/3777758597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fa019d010 0x7f5fa019d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:01.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.957+0000 7f5fa529f640 1 -- 172.21.15.12:0/3777758597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fa0102190 msgr2=0x7f5fa0198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:01.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.957+0000 7f5fa529f640 1 --2- 172.21.15.12:0/3777758597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fa0102190 0x7f5fa0198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.957+0000 7f5fa529f640 1 -- 172.21.15.12:0/3777758597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fa019d010 msgr2=0x7f5fa019d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:01.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.957+0000 7f5fa529f640 1 --2- 172.21.15.12:0/3777758597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fa019d010 0x7f5fa019d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.957+0000 7f5fa529f640 1 -- 172.21.15.12:0/3777758597 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa019db00 con 0x7f5fa0107260 2024-04-22T13:28:01.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.957+0000 7f5fa5aa0640 1 --2- 172.21.15.12:0/3777758597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fa0102190 0x7f5fa0198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:01.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.957+0000 7f5fa529f640 1 --2- 172.21.15.12:0/3777758597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fa0107260 0x7f5fa0198c90 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000c970 tx=0x7f5f9000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:01.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.957+0000 7f5f96ffd640 1 -- 172.21.15.12:0/3777758597 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f90007bb0 con 0x7f5fa0107260 2024-04-22T13:28:01.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.957+0000 7f5fa7d04640 1 -- 172.21.15.12:0/3777758597 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fa019ddf0 con 0x7f5fa0107260 2024-04-22T13:28:01.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.957+0000 7f5f96ffd640 1 -- 172.21.15.12:0/3777758597 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f90007d50 con 0x7f5fa0107260 2024-04-22T13:28:01.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.957+0000 7f5f96ffd640 1 -- 172.21.15.12:0/3777758597 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9000f450 con 0x7f5fa0107260 2024-04-22T13:28:01.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.957+0000 7f5fa7d04640 1 -- 172.21.15.12:0/3777758597 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fa01a55e0 con 0x7f5fa0107260 2024-04-22T13:28:01.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.959+0000 7f5f96ffd640 1 -- 172.21.15.12:0/3777758597 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5f90004ec0 con 0x7f5fa0107260 2024-04-22T13:28:01.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.960+0000 7f5faa9d4740 1 Processor -- start 2024-04-22T13:28:01.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.960+0000 7f5faa9d4740 1 -- start start 2024-04-22T13:28:01.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.960+0000 7f5f96ffd640 1 --2- 172.21.15.12:0/3777758597 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5f8c0a3d20 0x7f5f8c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:01.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.960+0000 7f5faa9d4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf2761e8e0 0x55cf2761ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:01.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.960+0000 7f5faa9d4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf2761f290 0x55cf27627840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:01.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.960+0000 7f5faa9d4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf27627e10 0x55cf2762a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:01.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.960+0000 7f5faa9d4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55cf27582f00 con 0x55cf2761f290 2024-04-22T13:28:01.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.960+0000 7f5faa9d4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55cf27520590 con 0x55cf2761e8e0 2024-04-22T13:28:01.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.960+0000 7f5faa9d4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cf2757eec0 con 0x55cf27627e10 2024-04-22T13:28:01.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.960+0000 7f5fa62a1640 1 --2- 172.21.15.12:0/3777758597 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5f8c0a3d20 0x7f5f8c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:01.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.960+0000 7f5f96ffd640 1 -- 172.21.15.12:0/3777758597 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f5f900d0400 con 0x7f5fa0107260 2024-04-22T13:28:01.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.960+0000 7f5fa529f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf2761f290 0x55cf27627840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:01.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.961+0000 7f5fa529f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf2761f290 0x55cf27627840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36462/0 (socket says 172.21.15.12:36462) 2024-04-22T13:28:01.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.961+0000 7f5fa529f640 1 -- 172.21.15.12:0/2479798406 learned_addr learned my addr 172.21.15.12:0/2479798406 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:01.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.961+0000 7f5fa62a1640 1 --2- 172.21.15.12:0/2479798406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf27627e10 0x55cf2762a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:01.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.961+0000 7f5fa5aa0640 1 --2- 172.21.15.12:0/2479798406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf2761e8e0 0x55cf2761ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:01.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.961+0000 7f5fa529f640 1 -- 172.21.15.12:0/2479798406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf27627e10 msgr2=0x55cf2762a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:01.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.961+0000 7f5fa529f640 1 --2- 172.21.15.12:0/2479798406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf27627e10 0x55cf2762a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.961+0000 7f5fa529f640 1 -- 172.21.15.12:0/2479798406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf2761e8e0 msgr2=0x55cf2761ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:01.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.961+0000 7f5fa529f640 1 --2- 172.21.15.12:0/2479798406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf2761e8e0 0x55cf2761ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.961+0000 7f5fa529f640 1 -- 172.21.15.12:0/2479798406 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf2735d7e0 con 0x55cf2761f290 2024-04-22T13:28:01.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.961+0000 7f5fa62a1640 1 --2- 172.21.15.12:0/3777758597 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5f8c0a3d20 0x7f5f8c0a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5f98004af0 tx=0x7f5f98032ca0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:01.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.961+0000 7f5fa529f640 1 --2- 172.21.15.12:0/2479798406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf2761f290 0x55cf27627840 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f5f900a4ad0 tx=0x7f5f90098800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc4023b45cd311ad server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:01.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.961+0000 7f5f8bfff640 1 -- 172.21.15.12:0/2479798406 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9001d040 con 0x55cf2761f290 2024-04-22T13:28:01.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.961+0000 7f5f8bfff640 1 -- 172.21.15.12:0/2479798406 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f900046c0 con 0x55cf2761f290 2024-04-22T13:28:01.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.961+0000 7f5f8bfff640 1 -- 172.21.15.12:0/2479798406 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f900a0740 con 0x55cf2761f290 2024-04-22T13:28:01.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.962+0000 7f5faa9d4740 1 -- 172.21.15.12:0/2479798406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf2761f290 msgr2=0x55cf27627840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:01.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.962+0000 7f5faa9d4740 1 --2- 172.21.15.12:0/2479798406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf2761f290 0x55cf27627840 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f5f900a4ad0 tx=0x7f5f90098800 comp rx=0 tx=0).stop 2024-04-22T13:28:01.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.962+0000 7f5faa9d4740 1 -- 172.21.15.12:0/2479798406 shutdown_connections 2024-04-22T13:28:01.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.962+0000 7f5faa9d4740 1 --2- 172.21.15.12:0/2479798406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf27627e10 0x55cf2762a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.962+0000 7f5faa9d4740 1 --2- 172.21.15.12:0/2479798406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf2761f290 0x55cf27627840 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.962+0000 7f5faa9d4740 1 --2- 172.21.15.12:0/2479798406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf2761e8e0 0x55cf2761ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.962+0000 7f5faa9d4740 1 -- 172.21.15.12:0/2479798406 >> 172.21.15.12:0/2479798406 conn(0x55cf27618fb0 msgr2=0x55cf27619390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:01.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.962+0000 7f5faa9d4740 1 -- 172.21.15.12:0/2479798406 shutdown_connections 2024-04-22T13:28:01.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.962+0000 7f5faa9d4740 1 -- 172.21.15.12:0/2479798406 wait complete. 2024-04-22T13:28:01.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.963+0000 7f5faa9d4740 1 -- start start 2024-04-22T13:28:01.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.963+0000 7f5faa9d4740 1 Processor -- start 2024-04-22T13:28:01.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.963+0000 7f5faa9d4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf2761e8e0 0x55cf27624e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:01.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.963+0000 7f5faa9d4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf2761f290 0x55cf27625390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:01.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.963+0000 7f5faa9d4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf27627e10 0x55cf276258d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:01.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.963+0000 7f5faa9d4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55cf2762ad80 con 0x55cf2761f290 2024-04-22T13:28:01.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.963+0000 7f5faa9d4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55cf2753c580 con 0x55cf2761e8e0 2024-04-22T13:28:01.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.963+0000 7f5faa9d4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cf2762af00 con 0x55cf27627e10 2024-04-22T13:28:01.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.963+0000 7f5faa9d4740 10 client.0 _ll_register_callbacks cb 0x55cf27542650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:01.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.963+0000 7f5fa529f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf2761f290 0x55cf27625390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:01.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.963+0000 7f5fa529f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf2761f290 0x55cf27625390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36468/0 (socket says 172.21.15.12:36468) 2024-04-22T13:28:01.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.963+0000 7f5fa529f640 1 -- 172.21.15.12:0/2074222905 learned_addr learned my addr 172.21.15.12:0/2074222905 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:01.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.963+0000 7f5fa5aa0640 1 --2- 172.21.15.12:0/2074222905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf2761e8e0 0x55cf27624e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:01.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.963+0000 7f5fa62a1640 1 --2- 172.21.15.12:0/2074222905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf27627e10 0x55cf276258d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:01.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.964+0000 7f5fa529f640 1 -- 172.21.15.12:0/2074222905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf27627e10 msgr2=0x55cf276258d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:01.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.964+0000 7f5fa529f640 1 --2- 172.21.15.12:0/2074222905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf27627e10 0x55cf276258d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.964+0000 7f5fa529f640 1 -- 172.21.15.12:0/2074222905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf2761e8e0 msgr2=0x55cf27624e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:01.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.964+0000 7f5fa529f640 1 --2- 172.21.15.12:0/2074222905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf2761e8e0 0x55cf27624e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.964+0000 7f5fa529f640 1 -- 172.21.15.12:0/2074222905 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf273bdf70 con 0x55cf2761f290 2024-04-22T13:28:01.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.964+0000 7f5fa529f640 1 --2- 172.21.15.12:0/2074222905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf2761f290 0x55cf27625390 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000afd0 tx=0x7f5f90012e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd99d7d587ed73e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:01.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.964+0000 7f5f897fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:01.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.964+0000 7f5f897fa640 1 -- 172.21.15.12:0/2074222905 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9001d070 con 0x55cf2761f290 2024-04-22T13:28:01.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.964+0000 7f5f897fa640 1 -- 172.21.15.12:0/2074222905 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f5f900cddc0 con 0x55cf2761f290 2024-04-22T13:28:01.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.964+0000 7f5faa9d4740 1 -- 172.21.15.12:0/2074222905 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cf27590920 con 0x55cf2761f290 2024-04-22T13:28:01.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.964+0000 7f5f897fa640 1 -- 172.21.15.12:0/2074222905 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5f900cdfc0 con 0x55cf2761f290 2024-04-22T13:28:01.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.964+0000 7f5faa9d4740 10 client.17151 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:01.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.964+0000 7f5faa9d4740 1 -- 172.21.15.12:0/2074222905 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cf27255170 con 0x55cf2761f290 2024-04-22T13:28:01.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.964+0000 7f5f897fa640 1 -- 172.21.15.12:0/2074222905 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55cf27590920 con 0x55cf2761f290 2024-04-22T13:28:01.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.964+0000 7f5f897fa640 1 -- 172.21.15.12:0/2074222905 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5f900b5b10 con 0x55cf2761f290 2024-04-22T13:28:01.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.965+0000 7f5faa9d4740 10 client.17151 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:01.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.965+0000 7f5faa9d4740 10 client.17151 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:01.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.965+0000 7f5faa9d4740 20 client.17151 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:01.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.965+0000 7f5faa9d4740 1 --2- 172.21.15.12:0/2074222905 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55cf27638230 0x55cf2763a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:01.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.965+0000 7f5faa9d4740 4 client.17151 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:01.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.965+0000 7f5faa9d4740 1 -- 172.21.15.12:0/2074222905 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf2758a790 con 0x55cf27638230 2024-04-22T13:28:01.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.965+0000 7f5fa5aa0640 1 --2- 172.21.15.12:0/2074222905 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55cf27638230 0x55cf2763a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:01.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.965+0000 7f5f897fa640 10 client.17151 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:01.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.965+0000 7f5fa5aa0640 1 --2- 172.21.15.12:0/2074222905 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55cf27638230 0x55cf2763a620 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fd9e148134555e9 server_cookie=3ba746c0578fb5a2 in_seq=0 out_seq=0 2024-04-22T13:28:01.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.966+0000 7f5f897fa640 1 -- 172.21.15.12:0/2074222905 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cf2758a790 con 0x55cf27638230 2024-04-22T13:28:01.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.966+0000 7f5f897fa640 10 client.17151 handle_command_reply: tid=0 2024-04-22T13:28:01.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.966+0000 7f5faa9d4740 1 client.17151 shutdown 2024-04-22T13:28:01.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.966+0000 7f5faa9d4740 1 -- 172.21.15.12:0/2074222905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf2761f290 msgr2=0x55cf27625390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:01.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.966+0000 7f5faa9d4740 1 --2- 172.21.15.12:0/2074222905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf2761f290 0x55cf27625390 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000afd0 tx=0x7f5f90012e70 comp rx=0 tx=0).stop 2024-04-22T13:28:01.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.966+0000 7f5faa9d4740 1 -- 172.21.15.12:0/2074222905 shutdown_connections 2024-04-22T13:28:01.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.966+0000 7f5faa9d4740 1 --2- 172.21.15.12:0/2074222905 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55cf27638230 0x55cf2763a620 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.966+0000 7f5faa9d4740 1 --2- 172.21.15.12:0/2074222905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf27627e10 0x55cf276258d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.966+0000 7f5faa9d4740 1 --2- 172.21.15.12:0/2074222905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf2761f290 0x55cf27625390 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.966+0000 7f5f897fa640 0 client.17151 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:01.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.966+0000 7f5faa9d4740 1 --2- 172.21.15.12:0/2074222905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf2761e8e0 0x55cf27624e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.966+0000 7f5faa9d4740 1 -- 172.21.15.12:0/2074222905 >> 172.21.15.12:0/2074222905 conn(0x55cf27618fb0 msgr2=0x55cf2761ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:01.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.966+0000 7f5faa9d4740 1 -- 172.21.15.12:0/2074222905 shutdown_connections 2024-04-22T13:28:01.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.967+0000 7f5faa9d4740 1 -- 172.21.15.12:0/2074222905 wait complete. 2024-04-22T13:28:01.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.967+0000 7f5faa9d4740 20 client.17151 trim_cache size 0 max 16384 2024-04-22T13:28:01.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.984+0000 7f5faa9d4740 1 Processor -- start 2024-04-22T13:28:01.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.985+0000 7f5faa9d4740 1 -- start start 2024-04-22T13:28:01.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.985+0000 7f5faa9d4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf2761e8e0 0x55cf27627580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:01.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.985+0000 7f5faa9d4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf2761f290 0x55cf27623fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:01.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.985+0000 7f5faa9d4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf27627e10 0x55cf27633860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:01.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.985+0000 7f5faa9d4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa01adb40 con 0x55cf27627e10 2024-04-22T13:28:01.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.985+0000 7f5faa9d4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa01af680 con 0x55cf2761f290 2024-04-22T13:28:01.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.985+0000 7f5faa9d4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa01ae210 con 0x55cf2761e8e0 2024-04-22T13:28:01.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.985+0000 7f5fa62a1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf27627e10 0x55cf27633860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:01.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.985+0000 7f5fa62a1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf27627e10 0x55cf27633860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36480/0 (socket says 172.21.15.12:36480) 2024-04-22T13:28:01.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.985+0000 7f5fa62a1640 1 -- 172.21.15.12:0/2512613499 learned_addr learned my addr 172.21.15.12:0/2512613499 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:01.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.985+0000 7f5fa5aa0640 1 --2- 172.21.15.12:0/2512613499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf2761e8e0 0x55cf27627580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:01.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.985+0000 7f5fa529f640 1 --2- 172.21.15.12:0/2512613499 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf2761f290 0x55cf27623fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:01.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.985+0000 7f5fa62a1640 1 -- 172.21.15.12:0/2512613499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf2761e8e0 msgr2=0x55cf27627580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:01.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.985+0000 7f5fa62a1640 1 --2- 172.21.15.12:0/2512613499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf2761e8e0 0x55cf27627580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.985+0000 7f5fa62a1640 1 -- 172.21.15.12:0/2512613499 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf2761f290 msgr2=0x55cf27623fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:01.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.985+0000 7f5fa62a1640 1 --2- 172.21.15.12:0/2512613499 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf2761f290 0x55cf27623fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.985+0000 7f5fa62a1640 1 -- 172.21.15.12:0/2512613499 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf2761eff0 con 0x55cf27627e10 2024-04-22T13:28:01.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.985+0000 7f5fa529f640 1 --2- 172.21.15.12:0/2512613499 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf2761f290 0x55cf27623fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:01.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.985+0000 7f5fa5aa0640 1 --2- 172.21.15.12:0/2512613499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf2761e8e0 0x55cf27627580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:01.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.985+0000 7f5fa62a1640 1 --2- 172.21.15.12:0/2512613499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf27627e10 0x55cf27633860 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f5f98009ca0 tx=0x7f5f9802e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1323c310d5286b33 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:01.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.986+0000 7f5f8b7fe640 1 -- 172.21.15.12:0/2512613499 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f98052070 con 0x55cf27627e10 2024-04-22T13:28:01.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.986+0000 7f5f8b7fe640 1 -- 172.21.15.12:0/2512613499 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f98003b70 con 0x55cf27627e10 2024-04-22T13:28:01.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.986+0000 7f5f8b7fe640 1 -- 172.21.15.12:0/2512613499 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f98039040 con 0x55cf27627e10 2024-04-22T13:28:01.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.986+0000 7f5faa9d4740 1 -- 172.21.15.12:0/2512613499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf27627e10 msgr2=0x55cf27633860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:01.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.986+0000 7f5faa9d4740 1 --2- 172.21.15.12:0/2512613499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf27627e10 0x55cf27633860 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f5f98009ca0 tx=0x7f5f9802e050 comp rx=0 tx=0).stop 2024-04-22T13:28:01.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.986+0000 7f5faa9d4740 1 -- 172.21.15.12:0/2512613499 shutdown_connections 2024-04-22T13:28:01.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.986+0000 7f5faa9d4740 1 --2- 172.21.15.12:0/2512613499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf27627e10 0x55cf27633860 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.986+0000 7f5faa9d4740 1 --2- 172.21.15.12:0/2512613499 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf2761f290 0x55cf27623fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.986+0000 7f5faa9d4740 1 --2- 172.21.15.12:0/2512613499 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf2761e8e0 0x55cf27627580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.986+0000 7f5faa9d4740 1 -- 172.21.15.12:0/2512613499 >> 172.21.15.12:0/2512613499 conn(0x55cf27618fb0 msgr2=0x55cf27633150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:01.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.986+0000 7f5faa9d4740 1 -- 172.21.15.12:0/2512613499 shutdown_connections 2024-04-22T13:28:01.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.986+0000 7f5faa9d4740 1 -- 172.21.15.12:0/2512613499 wait complete. 2024-04-22T13:28:01.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.987+0000 7f5faa9d4740 1 -- start start 2024-04-22T13:28:01.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.987+0000 7f5faa9d4740 1 Processor -- start 2024-04-22T13:28:01.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.987+0000 7f5faa9d4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf2761e8e0 0x55cf2763af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:01.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.987+0000 7f5faa9d4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf2761f290 0x55cf2763b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:01.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.987+0000 7f5faa9d4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf27627e10 0x55cf2763ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:01.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.987+0000 7f5faa9d4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa01aefb0 con 0x55cf2761f290 2024-04-22T13:28:01.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.987+0000 7f5faa9d4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa01ae8e0 con 0x55cf27627e10 2024-04-22T13:28:01.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.987+0000 7f5faa9d4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa01abb20 con 0x55cf2761e8e0 2024-04-22T13:28:01.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.987+0000 7f5faa9d4740 10 client.0 _ll_register_callbacks cb 0x7f5f9009c210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:01.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.987+0000 7f5fa529f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf2761f290 0x55cf2763b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:01.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.987+0000 7f5fa529f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf2761f290 0x55cf2763b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36486/0 (socket says 172.21.15.12:36486) 2024-04-22T13:28:01.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.987+0000 7f5fa529f640 1 -- 172.21.15.12:0/4048028768 learned_addr learned my addr 172.21.15.12:0/4048028768 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:01.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.987+0000 7f5fa62a1640 1 --2- 172.21.15.12:0/4048028768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf27627e10 0x55cf2763ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:01.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.988+0000 7f5fa5aa0640 1 --2- 172.21.15.12:0/4048028768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf2761e8e0 0x55cf2763af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:01.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.988+0000 7f5fa529f640 1 -- 172.21.15.12:0/4048028768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf2761e8e0 msgr2=0x55cf2763af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:01.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.988+0000 7f5fa529f640 1 --2- 172.21.15.12:0/4048028768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf2761e8e0 0x55cf2763af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.988+0000 7f5fa529f640 1 -- 172.21.15.12:0/4048028768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf27627e10 msgr2=0x55cf2763ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:01.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.988+0000 7f5fa529f640 1 --2- 172.21.15.12:0/4048028768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf27627e10 0x55cf2763ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.988+0000 7f5fa529f640 1 -- 172.21.15.12:0/4048028768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf2769d410 con 0x55cf2761f290 2024-04-22T13:28:01.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.988+0000 7f5fa62a1640 1 --2- 172.21.15.12:0/4048028768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf27627e10 0x55cf2763ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:01.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.988+0000 7f5fa5aa0640 1 --2- 172.21.15.12:0/4048028768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf2761e8e0 0x55cf2763af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:01.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.988+0000 7f5fa529f640 1 --2- 172.21.15.12:0/4048028768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf2761f290 0x55cf2763b4c0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f5f90098dc0 tx=0x7f5f9009be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee80f055ec4541a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:01.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.988+0000 7f5f897fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:01.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.988+0000 7f5f897fa640 1 -- 172.21.15.12:0/4048028768 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f900a5000 con 0x55cf2761f290 2024-04-22T13:28:01.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.988+0000 7f5f897fa640 1 -- 172.21.15.12:0/4048028768 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f5f900cddc0 con 0x55cf2761f290 2024-04-22T13:28:01.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.988+0000 7f5faa9d4740 1 -- 172.21.15.12:0/4048028768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f98003b70 con 0x55cf2761f290 2024-04-22T13:28:01.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.988+0000 7f5f897fa640 1 -- 172.21.15.12:0/4048028768 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5f900cdfc0 con 0x55cf2761f290 2024-04-22T13:28:01.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.988+0000 7f5faa9d4740 10 client.17157 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:01.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.988+0000 7f5faa9d4740 1 -- 172.21.15.12:0/4048028768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cf2769d410 con 0x55cf2761f290 2024-04-22T13:28:01.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.988+0000 7f5f897fa640 1 -- 172.21.15.12:0/4048028768 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f98003b70 con 0x55cf2761f290 2024-04-22T13:28:01.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.988+0000 7f5f897fa640 1 -- 172.21.15.12:0/4048028768 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5f900a7060 con 0x55cf2761f290 2024-04-22T13:28:01.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.989+0000 7f5faa9d4740 10 client.17157 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:01.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.989+0000 7f5faa9d4740 10 client.17157 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:01.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.989+0000 7f5faa9d4740 20 client.17157 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:01.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.989+0000 7f5faa9d4740 1 --2- 172.21.15.12:0/4048028768 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55cf2769e9c0 0x55cf276a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:01.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.989+0000 7f5faa9d4740 4 client.17157 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:28:01.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.989+0000 7f5faa9d4740 1 -- 172.21.15.12:0/4048028768 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cf27506c50 con 0x55cf2769e9c0 2024-04-22T13:28:01.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.989+0000 7f5fa5aa0640 1 --2- 172.21.15.12:0/4048028768 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55cf2769e9c0 0x55cf276a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:01.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.989+0000 7f5fa5aa0640 1 --2- 172.21.15.12:0/4048028768 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55cf2769e9c0 0x55cf276a0de0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcb907b2fc7045a3 server_cookie=45ba8ac99cf288de in_seq=0 out_seq=0 2024-04-22T13:28:01.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.989+0000 7f5f897fa640 10 client.17157 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:01.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.990+0000 7f5f897fa640 1 -- 172.21.15.12:0/4048028768 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cf27506c50 con 0x55cf2769e9c0 2024-04-22T13:28:01.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.990+0000 7f5f897fa640 10 client.17157 handle_command_reply: tid=0 2024-04-22T13:28:01.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.990+0000 7f5faa9d4740 1 client.17157 shutdown 2024-04-22T13:28:01.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.991+0000 7f5faa9d4740 1 -- 172.21.15.12:0/4048028768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf2761f290 msgr2=0x55cf2763b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:01.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.991+0000 7f5faa9d4740 1 --2- 172.21.15.12:0/4048028768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf2761f290 0x55cf2763b4c0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f5f90098dc0 tx=0x7f5f9009be60 comp rx=0 tx=0).stop 2024-04-22T13:28:01.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.991+0000 7f5faa9d4740 1 -- 172.21.15.12:0/4048028768 shutdown_connections 2024-04-22T13:28:01.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.991+0000 7f5faa9d4740 1 --2- 172.21.15.12:0/4048028768 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55cf2769e9c0 0x55cf276a0de0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.991+0000 7f5faa9d4740 1 --2- 172.21.15.12:0/4048028768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cf27627e10 0x55cf2763ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.991+0000 7f5faa9d4740 1 --2- 172.21.15.12:0/4048028768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cf2761f290 0x55cf2763b4c0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.991+0000 7f5f897fa640 0 client.17157 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:01.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.991+0000 7f5faa9d4740 1 --2- 172.21.15.12:0/4048028768 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cf2761e8e0 0x55cf2763af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.991+0000 7f5faa9d4740 1 -- 172.21.15.12:0/4048028768 >> 172.21.15.12:0/4048028768 conn(0x55cf27618fb0 msgr2=0x55cf276339e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:01.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.991+0000 7f5faa9d4740 1 -- 172.21.15.12:0/4048028768 shutdown_connections 2024-04-22T13:28:01.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.991+0000 7f5faa9d4740 1 -- 172.21.15.12:0/4048028768 wait complete. 2024-04-22T13:28:01.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.991+0000 7f5faa9d4740 20 client.17157 trim_cache size 0 max 16384 2024-04-22T13:28:01.994 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:28:01.994 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-04-22T13:28:01.994 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "b97826b2-d694-4a06-a7ce-0a6ff9231baf", 2024-04-22T13:28:01.994 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-04-22T13:28:01.994 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:28:01.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.991+0000 7f5fa7d04640 1 -- 172.21.15.12:0/3777758597 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5f8c0a3d20 msgr2=0x7f5f8c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:01.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.991+0000 7f5fa7d04640 1 --2- 172.21.15.12:0/3777758597 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5f8c0a3d20 0x7f5f8c0a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5f98004af0 tx=0x7f5f98032ca0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.991+0000 7f5fa7d04640 1 -- 172.21.15.12:0/3777758597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fa0107260 msgr2=0x7f5fa0198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:01.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.991+0000 7f5fa7d04640 1 --2- 172.21.15.12:0/3777758597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fa0107260 0x7f5fa0198c90 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000c970 tx=0x7f5f9000ce30 comp rx=0 tx=0).stop 2024-04-22T13:28:01.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.992+0000 7f5fa7d04640 1 -- 172.21.15.12:0/3777758597 shutdown_connections 2024-04-22T13:28:01.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.992+0000 7f5fa7d04640 1 --2- 172.21.15.12:0/3777758597 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5f8c0a3d20 0x7f5f8c0a61e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.992+0000 7f5fa7d04640 1 --2- 172.21.15.12:0/3777758597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fa019d010 0x7f5fa019d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.992+0000 7f5fa7d04640 1 --2- 172.21.15.12:0/3777758597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fa0107260 0x7f5fa0198c90 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.992+0000 7f5fa7d04640 1 --2- 172.21.15.12:0/3777758597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fa0102190 0x7f5fa0198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:01.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.992+0000 7f5fa7d04640 1 -- 172.21.15.12:0/3777758597 >> 172.21.15.12:0/3777758597 conn(0x7f5fa00fd990 msgr2=0x7f5fa00ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:01.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.992+0000 7f5fa7d04640 1 -- 172.21.15.12:0/3777758597 shutdown_connections 2024-04-22T13:28:01.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:01.992+0000 7f5fa7d04640 1 -- 172.21.15.12:0/3777758597 wait complete. 2024-04-22T13:28:01.999 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:28:01.999 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:28:02.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.147+0000 7f1590c4e640 1 -- 172.21.15.12:0/938818952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f158c10ec50 msgr2=0x7f158c111040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:02.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.147+0000 7f1590c4e640 1 --2- 172.21.15.12:0/938818952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f158c10ec50 0x7f158c111040 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f1584009ca0 tx=0x7f158402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.148+0000 7f1590c4e640 1 -- 172.21.15.12:0/938818952 shutdown_connections 2024-04-22T13:28:02.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.148+0000 7f1590c4e640 1 --2- 172.21.15.12:0/938818952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f158c10ec50 0x7f158c111040 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.148+0000 7f1590c4e640 1 --2- 172.21.15.12:0/938818952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f158c06aed0 0x7f158c10e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.148+0000 7f1590c4e640 1 --2- 172.21.15.12:0/938818952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f158c06a5b0 0x7f158c06a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.148+0000 7f1590c4e640 1 -- 172.21.15.12:0/938818952 >> 172.21.15.12:0/938818952 conn(0x7f158c077c40 msgr2=0x7f158c078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:02.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.148+0000 7f1590c4e640 1 -- 172.21.15.12:0/938818952 shutdown_connections 2024-04-22T13:28:02.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.148+0000 7f1590c4e640 1 -- 172.21.15.12:0/938818952 wait complete. 2024-04-22T13:28:02.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.148+0000 7f1590c4e640 1 Processor -- start 2024-04-22T13:28:02.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.148+0000 7f1590c4e640 1 -- start start 2024-04-22T13:28:02.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.149+0000 7f1590c4e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f158c06a5b0 0x7f158c1a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:02.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.149+0000 7f1590c4e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f158c06aed0 0x7f158c1a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:02.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.149+0000 7f1590c4e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f158c10ec50 0x7f158c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:02.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.149+0000 7f1590c4e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f158c114430 con 0x7f158c10ec50 2024-04-22T13:28:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.149+0000 7f1590c4e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f158c1142b0 con 0x7f158c06a5b0 2024-04-22T13:28:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.149+0000 7f1590c4e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f158c1145b0 con 0x7f158c06aed0 2024-04-22T13:28:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.149+0000 7f158ad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f158c10ec50 0x7f158c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.149+0000 7f158ad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f158c10ec50 0x7f158c19a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36512/0 (socket says 172.21.15.12:36512) 2024-04-22T13:28:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.149+0000 7f158ad9d640 1 -- 172.21.15.12:0/3053063397 learned_addr learned my addr 172.21.15.12:0/3053063397 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.149+0000 7f1589d9b640 1 --2- 172.21.15.12:0/3053063397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f158c06aed0 0x7f158c1a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.149+0000 7f158a59c640 1 --2- 172.21.15.12:0/3053063397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f158c06a5b0 0x7f158c1a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.149+0000 7f158ad9d640 1 -- 172.21.15.12:0/3053063397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f158c06aed0 msgr2=0x7f158c1a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.149+0000 7f158ad9d640 1 --2- 172.21.15.12:0/3053063397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f158c06aed0 0x7f158c1a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.149+0000 7f158ad9d640 1 -- 172.21.15.12:0/3053063397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f158c06a5b0 msgr2=0x7f158c1a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.149+0000 7f158ad9d640 1 --2- 172.21.15.12:0/3053063397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f158c06a5b0 0x7f158c1a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.149+0000 7f158ad9d640 1 -- 172.21.15.12:0/3053063397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158c19afd0 con 0x7f158c10ec50 2024-04-22T13:28:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.149+0000 7f158ad9d640 1 --2- 172.21.15.12:0/3053063397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f158c10ec50 0x7f158c19a710 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f15840098d0 tx=0x7f1584032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:02.148 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.149+0000 7f157b7fe640 1 -- 172.21.15.12:0/3053063397 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1584041020 con 0x7f158c10ec50 2024-04-22T13:28:02.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.149+0000 7f1590c4e640 1 -- 172.21.15.12:0/3053063397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f158c19b260 con 0x7f158c10ec50 2024-04-22T13:28:02.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.149+0000 7f157b7fe640 1 -- 172.21.15.12:0/3053063397 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1584032970 con 0x7f158c10ec50 2024-04-22T13:28:02.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.149+0000 7f157b7fe640 1 -- 172.21.15.12:0/3053063397 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1584039040 con 0x7f158c10ec50 2024-04-22T13:28:02.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.149+0000 7f1590c4e640 1 -- 172.21.15.12:0/3053063397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f158c1a73f0 con 0x7f158c10ec50 2024-04-22T13:28:02.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.152+0000 7f157b7fe640 1 -- 172.21.15.12:0/3053063397 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1584044020 con 0x7f158c10ec50 2024-04-22T13:28:02.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.152+0000 7f1593840740 1 Processor -- start 2024-04-22T13:28:02.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.152+0000 7f1593840740 1 -- start start 2024-04-22T13:28:02.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.152+0000 7f1593840740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c75309580 0x560c75309960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:02.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.152+0000 7f1593840740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c75309ea0 0x560c75312540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:02.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.152+0000 7f1593840740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c75312a80 0x560c75314e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:02.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.152+0000 7f1593840740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560c7526e1e0 con 0x560c75312a80 2024-04-22T13:28:02.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.152+0000 7f1593840740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560c7520b390 con 0x560c75309580 2024-04-22T13:28:02.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.152+0000 7f1593840740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560c7526a1a0 con 0x560c75309ea0 2024-04-22T13:28:02.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.152+0000 7f157b7fe640 1 --2- 172.21.15.12:0/3053063397 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f15740a3d20 0x7f15740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:02.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.152+0000 7f1589d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c75312a80 0x560c75314e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:02.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.152+0000 7f1589d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c75312a80 0x560c75314e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36528/0 (socket says 172.21.15.12:36528) 2024-04-22T13:28:02.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.153+0000 7f1589d9b640 1 -- 172.21.15.12:0/2167205154 learned_addr learned my addr 172.21.15.12:0/2167205154 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:02.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.153+0000 7f158ad9d640 1 --2- 172.21.15.12:0/3053063397 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f15740a3d20 0x7f15740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:02.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.153+0000 7f158a59c640 1 --2- 172.21.15.12:0/2167205154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c75309ea0 0x560c75312540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:02.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.153+0000 7f1589d9b640 1 --2- 172.21.15.12:0/2167205154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c75309580 0x560c75309960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:02.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.153+0000 7f157b7fe640 1 -- 172.21.15.12:0/3053063397 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f15840f2380 con 0x7f158c10ec50 2024-04-22T13:28:02.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.153+0000 7f1589d9b640 1 -- 172.21.15.12:0/2167205154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c75309ea0 msgr2=0x560c75312540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:02.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.153+0000 7f1589d9b640 1 --2- 172.21.15.12:0/2167205154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c75309ea0 0x560c75312540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.153+0000 7f1589d9b640 1 -- 172.21.15.12:0/2167205154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c75309580 msgr2=0x560c75309960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:02.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.153+0000 7f1589d9b640 1 --2- 172.21.15.12:0/2167205154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c75309580 0x560c75309960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.153+0000 7f1589d9b640 1 -- 172.21.15.12:0/2167205154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c7504a7d0 con 0x560c75312a80 2024-04-22T13:28:02.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.153+0000 7f1589d9b640 1 --2- 172.21.15.12:0/2167205154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c75312a80 0x560c75314e70 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f1580013560 tx=0x7f158000a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8c5155fa31f586 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:02.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.153+0000 7f158ad9d640 1 --2- 172.21.15.12:0/3053063397 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f15740a3d20 0x7f15740a61e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f15840efce0 tx=0x7f158408dd60 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:02.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.153+0000 7f1578ff9640 1 -- 172.21.15.12:0/2167205154 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1580019030 con 0x560c75312a80 2024-04-22T13:28:02.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.153+0000 7f1578ff9640 1 -- 172.21.15.12:0/2167205154 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15800110d0 con 0x560c75312a80 2024-04-22T13:28:02.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.153+0000 7f1578ff9640 1 -- 172.21.15.12:0/2167205154 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1580004d30 con 0x560c75312a80 2024-04-22T13:28:02.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.154+0000 7f1593840740 1 -- 172.21.15.12:0/2167205154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c75312a80 msgr2=0x560c75314e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:02.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.154+0000 7f1593840740 1 --2- 172.21.15.12:0/2167205154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c75312a80 0x560c75314e70 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f1580013560 tx=0x7f158000a750 comp rx=0 tx=0).stop 2024-04-22T13:28:02.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.154+0000 7f1593840740 1 -- 172.21.15.12:0/2167205154 shutdown_connections 2024-04-22T13:28:02.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.154+0000 7f1593840740 1 --2- 172.21.15.12:0/2167205154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c75312a80 0x560c75314e70 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.154+0000 7f1593840740 1 --2- 172.21.15.12:0/2167205154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c75309ea0 0x560c75312540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.154+0000 7f1593840740 1 --2- 172.21.15.12:0/2167205154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c75309580 0x560c75309960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.154+0000 7f1593840740 1 -- 172.21.15.12:0/2167205154 >> 172.21.15.12:0/2167205154 conn(0x560c75303cb0 msgr2=0x560c75304090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:02.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.154+0000 7f1593840740 1 -- 172.21.15.12:0/2167205154 shutdown_connections 2024-04-22T13:28:02.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.154+0000 7f1593840740 1 -- 172.21.15.12:0/2167205154 wait complete. 2024-04-22T13:28:02.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.154+0000 7f1593840740 1 -- start start 2024-04-22T13:28:02.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.154+0000 7f1593840740 1 Processor -- start 2024-04-22T13:28:02.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.155+0000 7f1593840740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c75309580 0x560c7530faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:02.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.155+0000 7f1593840740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c75309ea0 0x560c7530ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:02.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.155+0000 7f1593840740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c75312a80 0x560c75310520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:02.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.155+0000 7f1593840740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560c75315870 con 0x560c75309580 2024-04-22T13:28:02.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.155+0000 7f1593840740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560c752272f0 con 0x560c75309ea0 2024-04-22T13:28:02.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.155+0000 7f1593840740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560c753159f0 con 0x560c75312a80 2024-04-22T13:28:02.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.155+0000 7f1593840740 10 client.0 _ll_register_callbacks cb 0x560c75234580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:02.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.155+0000 7f1589d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c75309580 0x560c7530faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:02.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.155+0000 7f1589d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c75309580 0x560c7530faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36534/0 (socket says 172.21.15.12:36534) 2024-04-22T13:28:02.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.155+0000 7f1589d9b640 1 -- 172.21.15.12:0/251845450 learned_addr learned my addr 172.21.15.12:0/251845450 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:02.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.155+0000 7f1589d9b640 1 --2- 172.21.15.12:0/251845450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c75312a80 0x560c75310520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:02.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.155+0000 7f158a59c640 1 --2- 172.21.15.12:0/251845450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c75309ea0 0x560c7530ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:02.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.155+0000 7f1589d9b640 1 -- 172.21.15.12:0/251845450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c75312a80 msgr2=0x560c75310520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:02.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.155+0000 7f1589d9b640 1 --2- 172.21.15.12:0/251845450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c75312a80 0x560c75310520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.155+0000 7f1589d9b640 1 -- 172.21.15.12:0/251845450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c75309ea0 msgr2=0x560c7530ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:02.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.155+0000 7f1589d9b640 1 --2- 172.21.15.12:0/251845450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c75309ea0 0x560c7530ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.155+0000 7f1589d9b640 1 -- 172.21.15.12:0/251845450 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c74f803f0 con 0x560c75309580 2024-04-22T13:28:02.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.155+0000 7f158a59c640 1 --2- 172.21.15.12:0/251845450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c75309ea0 0x560c7530ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:02.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.155+0000 7f1589d9b640 1 --2- 172.21.15.12:0/251845450 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c75309580 0x560c7530faa0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f15800057c0 tx=0x7f1580005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650c63200c4fb725 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:02.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.155+0000 7f1571ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:02.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.155+0000 7f1571ffb640 1 -- 172.21.15.12:0/251845450 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1580019040 con 0x560c75309580 2024-04-22T13:28:02.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.155+0000 7f1571ffb640 1 -- 172.21.15.12:0/251845450 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f1580005200 con 0x560c75309580 2024-04-22T13:28:02.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.155+0000 7f1593840740 1 -- 172.21.15.12:0/251845450 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15800110d0 con 0x560c75309580 2024-04-22T13:28:02.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.156+0000 7f1571ffb640 1 -- 172.21.15.12:0/251845450 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1580031050 con 0x560c75309580 2024-04-22T13:28:02.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.156+0000 7f1593840740 10 client.17169 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:02.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.156+0000 7f1593840740 1 -- 172.21.15.12:0/251845450 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560c74f400f0 con 0x560c75309580 2024-04-22T13:28:02.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.156+0000 7f1571ffb640 1 -- 172.21.15.12:0/251845450 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15800110d0 con 0x560c75309580 2024-04-22T13:28:02.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.156+0000 7f1571ffb640 1 -- 172.21.15.12:0/251845450 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f158001e460 con 0x560c75309580 2024-04-22T13:28:02.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.156+0000 7f1593840740 10 client.17169 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:02.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.156+0000 7f1593840740 10 client.17169 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:02.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.156+0000 7f1593840740 20 client.17169 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:02.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.156+0000 7f1593840740 1 --2- 172.21.15.12:0/251845450 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560c75322990 0x560c75324d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:02.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.156+0000 7f1593840740 4 client.17169 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:02.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.156+0000 7f1593840740 1 -- 172.21.15.12:0/251845450 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c7527bc00 con 0x560c75322990 2024-04-22T13:28:02.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.156+0000 7f158a59c640 1 --2- 172.21.15.12:0/251845450 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560c75322990 0x560c75324d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:02.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.157+0000 7f158a59c640 1 --2- 172.21.15.12:0/251845450 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560c75322990 0x560c75324d80 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57da83e0b6a7a2c1 server_cookie=ec88d76c65ce8fa0 in_seq=0 out_seq=0 2024-04-22T13:28:02.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.157+0000 7f1571ffb640 10 client.17169 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:02.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.157+0000 7f1571ffb640 1 -- 172.21.15.12:0/251845450 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560c7527bc00 con 0x560c75322990 2024-04-22T13:28:02.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.157+0000 7f1571ffb640 10 client.17169 handle_command_reply: tid=0 2024-04-22T13:28:02.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.158+0000 7f1593840740 1 client.17169 shutdown 2024-04-22T13:28:02.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.158+0000 7f1593840740 1 -- 172.21.15.12:0/251845450 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c75309580 msgr2=0x560c7530faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:02.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.158+0000 7f1593840740 1 --2- 172.21.15.12:0/251845450 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c75309580 0x560c7530faa0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f15800057c0 tx=0x7f1580005750 comp rx=0 tx=0).stop 2024-04-22T13:28:02.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.158+0000 7f1593840740 1 -- 172.21.15.12:0/251845450 shutdown_connections 2024-04-22T13:28:02.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.158+0000 7f1593840740 1 --2- 172.21.15.12:0/251845450 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560c75322990 0x560c75324d80 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.158+0000 7f1593840740 1 --2- 172.21.15.12:0/251845450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c75312a80 0x560c75310520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.158+0000 7f1571ffb640 0 client.17169 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:02.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.158+0000 7f1593840740 1 --2- 172.21.15.12:0/251845450 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c75309ea0 0x560c7530ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.158+0000 7f1593840740 1 --2- 172.21.15.12:0/251845450 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c75309580 0x560c7530faa0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.158+0000 7f1593840740 1 -- 172.21.15.12:0/251845450 >> 172.21.15.12:0/251845450 conn(0x560c75303cb0 msgr2=0x560c75305a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:02.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.158+0000 7f1593840740 1 -- 172.21.15.12:0/251845450 shutdown_connections 2024-04-22T13:28:02.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.158+0000 7f1593840740 1 -- 172.21.15.12:0/251845450 wait complete. 2024-04-22T13:28:02.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.158+0000 7f1593840740 20 client.17169 trim_cache size 0 max 16384 2024-04-22T13:28:02.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.175+0000 7f1593840740 1 Processor -- start 2024-04-22T13:28:02.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.175+0000 7f1593840740 1 -- start start 2024-04-22T13:28:02.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.175+0000 7f1593840740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c75309580 0x560c753121d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:02.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.175+0000 7f1593840740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c75309ea0 0x560c7531e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:02.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.175+0000 7f1593840740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c75312a80 0x560c75302bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:02.177 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.175+0000 7f1593840740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f158c1afc60 con 0x560c75312a80 2024-04-22T13:28:02.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.175+0000 7f1593840740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f158c1b17a0 con 0x560c75309580 2024-04-22T13:28:02.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.175+0000 7f1593840740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f158c1b0330 con 0x560c75309ea0 2024-04-22T13:28:02.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.176+0000 7f1589d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c75312a80 0x560c75302bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:02.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.176+0000 7f1589d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c75312a80 0x560c75302bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36546/0 (socket says 172.21.15.12:36546) 2024-04-22T13:28:02.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.176+0000 7f1589d9b640 1 -- 172.21.15.12:0/1744032944 learned_addr learned my addr 172.21.15.12:0/1744032944 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:02.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.176+0000 7f158a59c640 1 --2- 172.21.15.12:0/1744032944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c75309ea0 0x560c7531e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:02.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.176+0000 7f1589d9b640 1 -- 172.21.15.12:0/1744032944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c75309ea0 msgr2=0x560c7531e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:02.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.176+0000 7f1589d9b640 1 --2- 172.21.15.12:0/1744032944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c75309ea0 0x560c7531e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.176+0000 7f1589d9b640 1 -- 172.21.15.12:0/1744032944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c75309580 msgr2=0x560c753121d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:02.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.176+0000 7f1589d9b640 1 --2- 172.21.15.12:0/1744032944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c75309580 0x560c753121d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.176+0000 7f1589d9b640 1 -- 172.21.15.12:0/1744032944 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c7530dff0 con 0x560c75312a80 2024-04-22T13:28:02.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.176+0000 7f1589d9b640 1 --2- 172.21.15.12:0/1744032944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c75312a80 0x560c75302bf0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f1580005cb0 tx=0x7f158000aae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d07f2ee159cb449 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:02.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.176+0000 7f1573fff640 1 -- 172.21.15.12:0/1744032944 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1580019040 con 0x560c75312a80 2024-04-22T13:28:02.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.176+0000 7f1573fff640 1 -- 172.21.15.12:0/1744032944 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15800110e0 con 0x560c75312a80 2024-04-22T13:28:02.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.176+0000 7f1573fff640 1 -- 172.21.15.12:0/1744032944 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1580004f60 con 0x560c75312a80 2024-04-22T13:28:02.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.177+0000 7f1593840740 1 -- 172.21.15.12:0/1744032944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c75312a80 msgr2=0x560c75302bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:02.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.177+0000 7f1593840740 1 --2- 172.21.15.12:0/1744032944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c75312a80 0x560c75302bf0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f1580005cb0 tx=0x7f158000aae0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.177+0000 7f1593840740 1 -- 172.21.15.12:0/1744032944 shutdown_connections 2024-04-22T13:28:02.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.177+0000 7f1593840740 1 --2- 172.21.15.12:0/1744032944 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c75312a80 0x560c75302bf0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.177+0000 7f1593840740 1 --2- 172.21.15.12:0/1744032944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c75309ea0 0x560c7531e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.177+0000 7f1593840740 1 --2- 172.21.15.12:0/1744032944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c75309580 0x560c753121d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.177+0000 7f1593840740 1 -- 172.21.15.12:0/1744032944 >> 172.21.15.12:0/1744032944 conn(0x560c75303cb0 msgr2=0x560c753024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:02.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.177+0000 7f1593840740 1 -- 172.21.15.12:0/1744032944 shutdown_connections 2024-04-22T13:28:02.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.177+0000 7f1593840740 1 -- 172.21.15.12:0/1744032944 wait complete. 2024-04-22T13:28:02.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.177+0000 7f1593840740 1 -- start start 2024-04-22T13:28:02.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.177+0000 7f1593840740 1 Processor -- start 2024-04-22T13:28:02.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.177+0000 7f1593840740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c75309580 0x560c7531acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:02.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.177+0000 7f1593840740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c75309ea0 0x560c7531b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:02.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.177+0000 7f1593840740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c75312a80 0x560c75316460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:02.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.177+0000 7f1593840740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f158c1b10d0 con 0x560c75312a80 2024-04-22T13:28:02.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.177+0000 7f1593840740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f158c1b0a00 con 0x560c75309ea0 2024-04-22T13:28:02.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.177+0000 7f1593840740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f158c1adc40 con 0x560c75309580 2024-04-22T13:28:02.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.177+0000 7f1593840740 10 client.0 _ll_register_callbacks cb 0x7f158000af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:02.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.177+0000 7f1589d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c75312a80 0x560c75316460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:02.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.177+0000 7f1589d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c75312a80 0x560c75316460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36562/0 (socket says 172.21.15.12:36562) 2024-04-22T13:28:02.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.177+0000 7f1589d9b640 1 -- 172.21.15.12:0/3751660705 learned_addr learned my addr 172.21.15.12:0/3751660705 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:02.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.178+0000 7f158a59c640 1 --2- 172.21.15.12:0/3751660705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c75309ea0 0x560c7531b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:02.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.178+0000 7f1589d9b640 1 --2- 172.21.15.12:0/3751660705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c75309580 0x560c7531acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:02.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.178+0000 7f1589d9b640 1 -- 172.21.15.12:0/3751660705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c75309580 msgr2=0x560c7531acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:02.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.178+0000 7f1589d9b640 1 --2- 172.21.15.12:0/3751660705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c75309580 0x560c7531acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.178+0000 7f1589d9b640 1 -- 172.21.15.12:0/3751660705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c75309ea0 msgr2=0x560c7531b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:02.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.178+0000 7f1589d9b640 1 --2- 172.21.15.12:0/3751660705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c75309ea0 0x560c7531b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.178+0000 7f1589d9b640 1 -- 172.21.15.12:0/3751660705 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c75316c60 con 0x560c75312a80 2024-04-22T13:28:02.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.178+0000 7f158a59c640 1 --2- 172.21.15.12:0/3751660705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c75309ea0 0x560c7531b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:02.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.178+0000 7f1589d9b640 1 --2- 172.21.15.12:0/3751660705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c75312a80 0x560c75316460 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f15800113b0 tx=0x7f1580011050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c523f14c62674c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:02.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.178+0000 7f1571ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:02.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.178+0000 7f1571ffb640 1 -- 172.21.15.12:0/3751660705 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158003f040 con 0x560c75312a80 2024-04-22T13:28:02.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.178+0000 7f1571ffb640 1 -- 172.21.15.12:0/3751660705 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f158001e660 con 0x560c75312a80 2024-04-22T13:28:02.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.178+0000 7f1593840740 1 -- 172.21.15.12:0/3751660705 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560c75275a70 con 0x560c75312a80 2024-04-22T13:28:02.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.178+0000 7f1571ffb640 1 -- 172.21.15.12:0/3751660705 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1580013020 con 0x560c75312a80 2024-04-22T13:28:02.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.178+0000 7f1593840740 10 client.17175 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:02.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.178+0000 7f1593840740 1 -- 172.21.15.12:0/3751660705 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560c753889f0 con 0x560c75312a80 2024-04-22T13:28:02.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.178+0000 7f1571ffb640 1 -- 172.21.15.12:0/3751660705 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560c75275a70 con 0x560c75312a80 2024-04-22T13:28:02.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.179+0000 7f1571ffb640 1 -- 172.21.15.12:0/3751660705 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f15800116b0 con 0x560c75312a80 2024-04-22T13:28:02.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.179+0000 7f1593840740 10 client.17175 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:02.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.179+0000 7f1593840740 10 client.17175 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:02.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.179+0000 7f1593840740 20 client.17175 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:02.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.179+0000 7f1593840740 1 --2- 172.21.15.12:0/3751660705 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560c75388f30 0x560c7538b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:02.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.179+0000 7f1593840740 4 client.17175 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:28:02.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.179+0000 7f1593840740 1 -- 172.21.15.12:0/3751660705 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560c751f1590 con 0x560c75388f30 2024-04-22T13:28:02.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.179+0000 7f158a59c640 1 --2- 172.21.15.12:0/3751660705 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560c75388f30 0x560c7538b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:02.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.179+0000 7f1571ffb640 10 client.17175 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:02.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.179+0000 7f158a59c640 1 --2- 172.21.15.12:0/3751660705 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560c75388f30 0x560c7538b350 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3594d89827dd1e32 server_cookie=25f744448169738b in_seq=0 out_seq=0 2024-04-22T13:28:02.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.180+0000 7f1571ffb640 1 -- 172.21.15.12:0/3751660705 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560c751f1590 con 0x560c75388f30 2024-04-22T13:28:02.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.180+0000 7f1571ffb640 10 client.17175 handle_command_reply: tid=0 2024-04-22T13:28:02.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.180+0000 7f1593840740 1 client.17175 shutdown 2024-04-22T13:28:02.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.180+0000 7f1593840740 1 -- 172.21.15.12:0/3751660705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c75312a80 msgr2=0x560c75316460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:02.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.180+0000 7f1593840740 1 --2- 172.21.15.12:0/3751660705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c75312a80 0x560c75316460 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f15800113b0 tx=0x7f1580011050 comp rx=0 tx=0).stop 2024-04-22T13:28:02.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.180+0000 7f1593840740 1 -- 172.21.15.12:0/3751660705 shutdown_connections 2024-04-22T13:28:02.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.180+0000 7f1593840740 1 --2- 172.21.15.12:0/3751660705 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560c75388f30 0x560c7538b350 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.180+0000 7f1593840740 1 --2- 172.21.15.12:0/3751660705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c75312a80 0x560c75316460 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.180+0000 7f1571ffb640 0 client.17175 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:02.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.180+0000 7f1593840740 1 --2- 172.21.15.12:0/3751660705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c75309ea0 0x560c7531b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.180+0000 7f1593840740 1 --2- 172.21.15.12:0/3751660705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560c75309580 0x560c7531acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.180+0000 7f1593840740 1 -- 172.21.15.12:0/3751660705 >> 172.21.15.12:0/3751660705 conn(0x560c75303cb0 msgr2=0x560c753102f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:02.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.180+0000 7f1593840740 1 -- 172.21.15.12:0/3751660705 shutdown_connections 2024-04-22T13:28:02.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.180+0000 7f1593840740 1 -- 172.21.15.12:0/3751660705 wait complete. 2024-04-22T13:28:02.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.180+0000 7f1593840740 20 client.17175 trim_cache size 0 max 16384 2024-04-22T13:28:02.183 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:28:02.183 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:28:02.183 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-04-22T13:28:02.183 INFO:teuthology.orchestra.run.smithi012.stdout: "b97826b2-d694-4a06-a7ce-0a6ff9231baf": { 2024-04-22T13:28:02.184 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-04-22T13:28:02.184 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "b97826b2-d694-4a06-a7ce-0a6ff9231baf", 2024-04-22T13:28:02.184 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-04-22T13:28:02.184 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:28:02.184 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:28:02.184 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:28:02.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.180+0000 7f1590c4e640 1 -- 172.21.15.12:0/3053063397 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f15740a3d20 msgr2=0x7f15740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:02.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.180+0000 7f1590c4e640 1 --2- 172.21.15.12:0/3053063397 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f15740a3d20 0x7f15740a61e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f15840efce0 tx=0x7f158408dd60 comp rx=0 tx=0).stop 2024-04-22T13:28:02.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.180+0000 7f1590c4e640 1 -- 172.21.15.12:0/3053063397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f158c10ec50 msgr2=0x7f158c19a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:02.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.180+0000 7f1590c4e640 1 --2- 172.21.15.12:0/3053063397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f158c10ec50 0x7f158c19a710 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f15840098d0 tx=0x7f1584032420 comp rx=0 tx=0).stop 2024-04-22T13:28:02.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.180+0000 7f1590c4e640 1 -- 172.21.15.12:0/3053063397 shutdown_connections 2024-04-22T13:28:02.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.180+0000 7f1590c4e640 1 --2- 172.21.15.12:0/3053063397 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f15740a3d20 0x7f15740a61e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.180+0000 7f1590c4e640 1 --2- 172.21.15.12:0/3053063397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f158c10ec50 0x7f158c19a710 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.180+0000 7f1590c4e640 1 --2- 172.21.15.12:0/3053063397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f158c06aed0 0x7f158c1a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.180+0000 7f1590c4e640 1 --2- 172.21.15.12:0/3053063397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f158c06a5b0 0x7f158c1a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:02.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.180+0000 7f1590c4e640 1 -- 172.21.15.12:0/3053063397 >> 172.21.15.12:0/3053063397 conn(0x7f158c077c40 msgr2=0x7f158c1101d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:02.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.180+0000 7f1590c4e640 1 -- 172.21.15.12:0/3053063397 shutdown_connections 2024-04-22T13:28:02.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:02.180+0000 7f1590c4e640 1 -- 172.21.15.12:0/3053063397 wait complete. 2024-04-22T13:28:02.190 INFO:tasks.cephfs.filesystem:scrub status for tag:b97826b2-d694-4a06-a7ce-0a6ff9231baf - {'path': '/', 'tag': 'b97826b2-d694-4a06-a7ce-0a6ff9231baf', 'options': 'recursive,force'} 2024-04-22T13:28:03.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:03 smithi134 ceph-mon[31338]: scrub queued for path: / 2024-04-22T13:28:03.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:03 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:28:03.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:03 smithi134 ceph-mon[31338]: scrub summary: active paths [/] 2024-04-22T13:28:03.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:03 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:28:03.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:03 smithi134 ceph-mon[31338]: pgmap v121: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-22T13:28:03.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:03 smithi111 ceph-mon[31361]: scrub queued for path: / 2024-04-22T13:28:03.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:03 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:28:03.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:03 smithi111 ceph-mon[31361]: scrub summary: active paths [/] 2024-04-22T13:28:03.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:03 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:28:03.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:03 smithi111 ceph-mon[31361]: pgmap v121: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-22T13:28:03.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:03 smithi012 ceph-mon[28265]: scrub queued for path: / 2024-04-22T13:28:03.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:03 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:28:03.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:03 smithi012 ceph-mon[28265]: scrub summary: active paths [/] 2024-04-22T13:28:03.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:03 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:28:03.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:03 smithi012 ceph-mon[28265]: pgmap v121: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-22T13:28:05.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:05 smithi134 ceph-mon[31338]: pgmap v122: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-22T13:28:05.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:05 smithi134 ceph-mon[31338]: scrub summary: idle 2024-04-22T13:28:05.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:05 smithi111 ceph-mon[31361]: pgmap v122: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-22T13:28:05.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:05 smithi111 ceph-mon[31361]: scrub summary: idle 2024-04-22T13:28:05.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:05 smithi012 ceph-mon[28265]: pgmap v122: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-22T13:28:05.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:05 smithi012 ceph-mon[28265]: scrub summary: idle 2024-04-22T13:28:07.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:07 smithi134 ceph-mon[31338]: pgmap v123: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-22T13:28:07.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:07 smithi111 ceph-mon[31361]: pgmap v123: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-22T13:28:07.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:07 smithi012 ceph-mon[28265]: pgmap v123: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-22T13:28:09.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:09 smithi134 ceph-mon[31338]: pgmap v124: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-22T13:28:09.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:09 smithi111 ceph-mon[31361]: pgmap v124: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-22T13:28:09.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:09 smithi012 ceph-mon[28265]: pgmap v124: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-22T13:28:11.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:11 smithi111 ceph-mon[31361]: pgmap v125: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-22T13:28:11.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:11 smithi012 ceph-mon[28265]: pgmap v125: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-22T13:28:11.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:11 smithi134 ceph-mon[31338]: pgmap v125: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-04-22T13:28:13.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:13 smithi111 ceph-mon[31361]: pgmap v126: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-22T13:28:13.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:13 smithi012 ceph-mon[28265]: pgmap v126: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-22T13:28:13.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:13 smithi134 ceph-mon[31338]: pgmap v126: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-22T13:28:15.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:15 smithi111 ceph-mon[31361]: pgmap v127: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-22T13:28:15.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:15 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:28:15.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:15 smithi012 ceph-mon[28265]: pgmap v127: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-22T13:28:15.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:15 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:28:15.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:15 smithi134 ceph-mon[31338]: pgmap v127: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-22T13:28:15.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:15 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:28:16.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:16 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:28:16.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:16 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:28:16.727 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:16 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:28:16.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:16 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:28:16.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:16 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:28:16.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:16 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:28:16.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:16 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:28:16.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:16 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:28:16.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:16 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:28:17.828 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:17 smithi134 ceph-mon[31338]: pgmap v128: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-22T13:28:17.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:17 smithi012 ceph-mon[28265]: pgmap v128: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-22T13:28:17.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:17 smithi111 ceph-mon[31361]: pgmap v128: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-22T13:28:19.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:19 smithi012 ceph-mon[28265]: pgmap v129: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-22T13:28:19.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:19 smithi134 ceph-mon[31338]: pgmap v129: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-22T13:28:19.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:19 smithi111 ceph-mon[31361]: pgmap v129: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-22T13:28:21.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:21 smithi012 ceph-mon[28265]: pgmap v130: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-22T13:28:21.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:21 smithi134 ceph-mon[31338]: pgmap v130: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-22T13:28:21.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:21 smithi111 ceph-mon[31361]: pgmap v130: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-22T13:28:23.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:23 smithi012 ceph-mon[28265]: pgmap v131: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-22T13:28:23.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:23 smithi134 ceph-mon[31338]: pgmap v131: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-22T13:28:23.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:23 smithi111 ceph-mon[31361]: pgmap v131: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-22T13:28:25.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:25 smithi012 ceph-mon[28265]: pgmap v132: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:25.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:25 smithi134 ceph-mon[31338]: pgmap v132: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:25.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:25 smithi111 ceph-mon[31361]: pgmap v132: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:27.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:27 smithi012 ceph-mon[28265]: pgmap v133: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:27.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:27 smithi134 ceph-mon[31338]: pgmap v133: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:27.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:27 smithi111 ceph-mon[31361]: pgmap v133: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:29.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:29 smithi012 ceph-mon[28265]: pgmap v134: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:28:29.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:29 smithi134 ceph-mon[31338]: pgmap v134: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:28:29.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:29 smithi111 ceph-mon[31361]: pgmap v134: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:28:31.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:31 smithi134 ceph-mon[31338]: pgmap v135: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:31.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:31 smithi111 ceph-mon[31361]: pgmap v135: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:32.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:31 smithi012 ceph-mon[28265]: pgmap v135: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:32.191 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:28:32.191 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:28:32.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.312+0000 7f90de681640 1 -- 172.21.15.12:0/3091426938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d806af70 msgr2=0x7f90d810ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.312+0000 7f90de681640 1 --2- 172.21.15.12:0/3091426938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d806af70 0x7f90d810ede0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f90d0009cd0 tx=0x7f90d002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.312+0000 7f90de681640 1 -- 172.21.15.12:0/3091426938 shutdown_connections 2024-04-22T13:28:32.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.312+0000 7f90de681640 1 --2- 172.21.15.12:0/3091426938 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d806af70 0x7f90d810ede0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.312+0000 7f90de681640 1 --2- 172.21.15.12:0/3091426938 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90d806a5b0 0x7f90d806aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.312+0000 7f90de681640 1 --2- 172.21.15.12:0/3091426938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90d8108060 0x7f90d8108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.312+0000 7f90de681640 1 -- 172.21.15.12:0/3091426938 >> 172.21.15.12:0/3091426938 conn(0x7f90d8077c40 msgr2=0x7f90d8078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:32.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.312+0000 7f90de681640 1 -- 172.21.15.12:0/3091426938 shutdown_connections 2024-04-22T13:28:32.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.312+0000 7f90de681640 1 -- 172.21.15.12:0/3091426938 wait complete. 2024-04-22T13:28:32.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.312+0000 7f90de681640 1 Processor -- start 2024-04-22T13:28:32.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.312+0000 7f90de681640 1 -- start start 2024-04-22T13:28:32.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.313+0000 7f90de681640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90d806a5b0 0x7f90d8198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.313+0000 7f90de681640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d806af70 0x7f90d8198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.313+0000 7f90de681640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90d8108060 0x7f90d819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.313+0000 7f90de681640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f90d8112180 con 0x7f90d806af70 2024-04-22T13:28:32.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.313+0000 7f90de681640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f90d8112000 con 0x7f90d806a5b0 2024-04-22T13:28:32.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.313+0000 7f90de681640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f90d8112300 con 0x7f90d8108060 2024-04-22T13:28:32.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.313+0000 7f90d77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d806af70 0x7f90d8198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.313+0000 7f90d77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d806af70 0x7f90d8198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60866/0 (socket says 172.21.15.12:60866) 2024-04-22T13:28:32.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.313+0000 7f90d77fe640 1 -- 172.21.15.12:0/40124380 learned_addr learned my addr 172.21.15.12:0/40124380 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:32.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.313+0000 7f90dcc1e640 1 --2- 172.21.15.12:0/40124380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90d8108060 0x7f90d819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.313+0000 7f90d7fff640 1 --2- 172.21.15.12:0/40124380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90d806a5b0 0x7f90d8198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.313+0000 7f90d77fe640 1 -- 172.21.15.12:0/40124380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90d8108060 msgr2=0x7f90d819cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.313+0000 7f90d77fe640 1 --2- 172.21.15.12:0/40124380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90d8108060 0x7f90d819cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.313+0000 7f90d77fe640 1 -- 172.21.15.12:0/40124380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90d806a5b0 msgr2=0x7f90d8198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.313+0000 7f90d77fe640 1 --2- 172.21.15.12:0/40124380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90d806a5b0 0x7f90d8198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.313+0000 7f90d77fe640 1 -- 172.21.15.12:0/40124380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d819d4c0 con 0x7f90d806af70 2024-04-22T13:28:32.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.314+0000 7f90d77fe640 1 --2- 172.21.15.12:0/40124380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d806af70 0x7f90d8198a60 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f90c800c920 tx=0x7f90c800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:32.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.314+0000 7f90d57fa640 1 -- 172.21.15.12:0/40124380 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c8007c70 con 0x7f90d806af70 2024-04-22T13:28:32.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.314+0000 7f90d57fa640 1 -- 172.21.15.12:0/40124380 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90c8007e10 con 0x7f90d806af70 2024-04-22T13:28:32.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.314+0000 7f90de681640 1 -- 172.21.15.12:0/40124380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90d819d7b0 con 0x7f90d806af70 2024-04-22T13:28:32.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.314+0000 7f90d57fa640 1 -- 172.21.15.12:0/40124380 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c8018820 con 0x7f90d806af70 2024-04-22T13:28:32.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.314+0000 7f90de681640 1 -- 172.21.15.12:0/40124380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90d81a5040 con 0x7f90d806af70 2024-04-22T13:28:32.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.316+0000 7f90d57fa640 1 -- 172.21.15.12:0/40124380 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f90c800f450 con 0x7f90d806af70 2024-04-22T13:28:32.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.316+0000 7f90e1278740 1 Processor -- start 2024-04-22T13:28:32.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.317+0000 7f90e1278740 1 -- start start 2024-04-22T13:28:32.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.317+0000 7f90e1278740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583afa49580 0x5583afa49960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.317+0000 7f90e1278740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583afa49ea0 0x5583afa52540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.317+0000 7f90e1278740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583afa52a80 0x5583afa54e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.317+0000 7f90e1278740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5583af9ae1e0 con 0x5583afa49ea0 2024-04-22T13:28:32.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.317+0000 7f90e1278740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5583af94b390 con 0x5583afa49580 2024-04-22T13:28:32.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.317+0000 7f90d57fa640 1 --2- 172.21.15.12:0/40124380 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f90c40a3d20 0x7f90c40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.317+0000 7f90e1278740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5583af9aa1a0 con 0x5583afa52a80 2024-04-22T13:28:32.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.317+0000 7f90d7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583afa49ea0 0x5583afa52540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.317+0000 7f90dcc1e640 1 --2- 172.21.15.12:0/40124380 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f90c40a3d20 0x7f90c40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.317+0000 7f90d7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583afa49ea0 0x5583afa52540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60872/0 (socket says 172.21.15.12:60872) 2024-04-22T13:28:32.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.317+0000 7f90d7fff640 1 -- 172.21.15.12:0/644269911 learned_addr learned my addr 172.21.15.12:0/644269911 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:32.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.317+0000 7f90dcc1e640 1 --2- 172.21.15.12:0/644269911 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583afa49580 0x5583afa49960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.317+0000 7f90d57fa640 1 -- 172.21.15.12:0/40124380 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f90c80d0940 con 0x7f90d806af70 2024-04-22T13:28:32.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.317+0000 7f90d77fe640 1 --2- 172.21.15.12:0/644269911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583afa52a80 0x5583afa54e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.317+0000 7f90d7fff640 1 -- 172.21.15.12:0/644269911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583afa52a80 msgr2=0x5583afa54e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:32.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.317+0000 7f90d7fff640 1 --2- 172.21.15.12:0/644269911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583afa52a80 0x5583afa54e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.317+0000 7f90d7fff640 1 -- 172.21.15.12:0/644269911 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583afa49580 msgr2=0x5583afa49960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:32.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.317+0000 7f90d7fff640 1 --2- 172.21.15.12:0/644269911 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583afa49580 0x5583afa49960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.317+0000 7f90d7fff640 1 -- 172.21.15.12:0/644269911 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583af78a7d0 con 0x5583afa49ea0 2024-04-22T13:28:32.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.317+0000 7f90dcc1e640 1 --2- 172.21.15.12:0/644269911 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583afa49580 0x5583afa49960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:32.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.318+0000 7f90d7fff640 1 --2- 172.21.15.12:0/644269911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583afa49ea0 0x5583afa52540 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f90c000cc60 tx=0x7f90c000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=556f983f4d2bdf17 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:32.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.318+0000 7f90dcc1e640 1 --2- 172.21.15.12:0/40124380 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f90c40a3d20 0x7f90c40a61e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f90d0005cc0 tx=0x7f90d002e040 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:32.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.318+0000 7f90ce7fc640 1 -- 172.21.15.12:0/644269911 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c0014070 con 0x5583afa49ea0 2024-04-22T13:28:32.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.318+0000 7f90ce7fc640 1 -- 172.21.15.12:0/644269911 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90c00047a0 con 0x5583afa49ea0 2024-04-22T13:28:32.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.318+0000 7f90ce7fc640 1 -- 172.21.15.12:0/644269911 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c0010690 con 0x5583afa49ea0 2024-04-22T13:28:32.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.318+0000 7f90e1278740 1 -- 172.21.15.12:0/644269911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583afa49ea0 msgr2=0x5583afa52540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.318+0000 7f90e1278740 1 --2- 172.21.15.12:0/644269911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583afa49ea0 0x5583afa52540 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f90c000cc60 tx=0x7f90c000e720 comp rx=0 tx=0).stop 2024-04-22T13:28:32.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.318+0000 7f90e1278740 1 -- 172.21.15.12:0/644269911 shutdown_connections 2024-04-22T13:28:32.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.318+0000 7f90e1278740 1 --2- 172.21.15.12:0/644269911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583afa52a80 0x5583afa54e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.318+0000 7f90e1278740 1 --2- 172.21.15.12:0/644269911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583afa49ea0 0x5583afa52540 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.318+0000 7f90e1278740 1 --2- 172.21.15.12:0/644269911 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583afa49580 0x5583afa49960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.318+0000 7f90e1278740 1 -- 172.21.15.12:0/644269911 >> 172.21.15.12:0/644269911 conn(0x5583afa43cb0 msgr2=0x5583afa44090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:32.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.318+0000 7f90e1278740 1 -- 172.21.15.12:0/644269911 shutdown_connections 2024-04-22T13:28:32.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.319+0000 7f90e1278740 1 -- 172.21.15.12:0/644269911 wait complete. 2024-04-22T13:28:32.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.319+0000 7f90e1278740 1 -- start start 2024-04-22T13:28:32.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.319+0000 7f90e1278740 1 Processor -- start 2024-04-22T13:28:32.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.319+0000 7f90e1278740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583afa49580 0x5583afa4fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.319+0000 7f90e1278740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583afa49ea0 0x5583afa4ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.319+0000 7f90e1278740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583afa52a80 0x5583afa50490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.319+0000 7f90e1278740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5583afa55870 con 0x5583afa49580 2024-04-22T13:28:32.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.319+0000 7f90e1278740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5583af9672f0 con 0x5583afa52a80 2024-04-22T13:28:32.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.319+0000 7f90e1278740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5583afa559f0 con 0x5583afa49ea0 2024-04-22T13:28:32.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.319+0000 7f90e1278740 10 client.0 _ll_register_callbacks cb 0x5583af974580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:32.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.319+0000 7f90d7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583afa49580 0x5583afa4fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.320+0000 7f90d7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583afa49580 0x5583afa4fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60876/0 (socket says 172.21.15.12:60876) 2024-04-22T13:28:32.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.320+0000 7f90d7fff640 1 -- 172.21.15.12:0/4136300597 learned_addr learned my addr 172.21.15.12:0/4136300597 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:32.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.320+0000 7f90d77fe640 1 --2- 172.21.15.12:0/4136300597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583afa49ea0 0x5583afa4ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.320+0000 7f90dcc1e640 1 --2- 172.21.15.12:0/4136300597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583afa52a80 0x5583afa50490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.320+0000 7f90d7fff640 1 -- 172.21.15.12:0/4136300597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583afa49ea0 msgr2=0x5583afa4ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:32.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.320+0000 7f90d7fff640 1 --2- 172.21.15.12:0/4136300597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583afa49ea0 0x5583afa4ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.320+0000 7f90d7fff640 1 -- 172.21.15.12:0/4136300597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583afa52a80 msgr2=0x5583afa50490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:32.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.320+0000 7f90d7fff640 1 --2- 172.21.15.12:0/4136300597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583afa52a80 0x5583afa50490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.320+0000 7f90d7fff640 1 -- 172.21.15.12:0/4136300597 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583af6c03f0 con 0x5583afa49580 2024-04-22T13:28:32.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.320+0000 7f90d77fe640 1 --2- 172.21.15.12:0/4136300597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583afa49ea0 0x5583afa4ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:32.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.320+0000 7f90d7fff640 1 --2- 172.21.15.12:0/4136300597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583afa49580 0x5583afa4fa10 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f90c0010be0 tx=0x7f90c0010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9514b6b8317f9f4c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:32.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.320+0000 7f90b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:32.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.320+0000 7f90b77fe640 1 -- 172.21.15.12:0/4136300597 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c0014070 con 0x5583afa49580 2024-04-22T13:28:32.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.320+0000 7f90b77fe640 1 -- 172.21.15.12:0/4136300597 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f90c0004020 con 0x5583afa49580 2024-04-22T13:28:32.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.320+0000 7f90e1278740 1 -- 172.21.15.12:0/4136300597 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90c00047a0 con 0x5583afa49580 2024-04-22T13:28:32.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.320+0000 7f90b77fe640 1 -- 172.21.15.12:0/4136300597 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f90c0004220 con 0x5583afa49580 2024-04-22T13:28:32.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.320+0000 7f90e1278740 10 client.17187 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:32.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.320+0000 7f90e1278740 1 -- 172.21.15.12:0/4136300597 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5583af6800f0 con 0x5583afa49580 2024-04-22T13:28:32.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.321+0000 7f90b77fe640 1 -- 172.21.15.12:0/4136300597 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90c00047a0 con 0x5583afa49580 2024-04-22T13:28:32.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.321+0000 7f90b77fe640 1 -- 172.21.15.12:0/4136300597 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f90c0022410 con 0x5583afa49580 2024-04-22T13:28:32.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.321+0000 7f90e1278740 10 client.17187 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:32.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.321+0000 7f90e1278740 10 client.17187 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:32.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.321+0000 7f90e1278740 20 client.17187 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:32.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.321+0000 7f90e1278740 1 --2- 172.21.15.12:0/4136300597 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5583afa62970 0x5583afa64d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.321+0000 7f90e1278740 4 client.17187 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:32.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.321+0000 7f90e1278740 1 -- 172.21.15.12:0/4136300597 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583af9bbc00 con 0x5583afa62970 2024-04-22T13:28:32.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.321+0000 7f90d77fe640 1 --2- 172.21.15.12:0/4136300597 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5583afa62970 0x5583afa64d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.321+0000 7f90d77fe640 1 --2- 172.21.15.12:0/4136300597 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5583afa62970 0x5583afa64d60 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2b3996ccf708920 server_cookie=cad5fedfa34b6ba7 in_seq=0 out_seq=0 2024-04-22T13:28:32.325 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.321+0000 7f90b77fe640 10 client.17187 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:32.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.322+0000 7f90b77fe640 1 -- 172.21.15.12:0/4136300597 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5583af9bbc00 con 0x5583afa62970 2024-04-22T13:28:32.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.322+0000 7f90b77fe640 10 client.17187 handle_command_reply: tid=0 2024-04-22T13:28:32.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.322+0000 7f90e1278740 1 client.17187 shutdown 2024-04-22T13:28:32.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.322+0000 7f90e1278740 1 -- 172.21.15.12:0/4136300597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583afa49580 msgr2=0x5583afa4fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.322+0000 7f90e1278740 1 --2- 172.21.15.12:0/4136300597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583afa49580 0x5583afa4fa10 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f90c0010be0 tx=0x7f90c0010c10 comp rx=0 tx=0).stop 2024-04-22T13:28:32.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.322+0000 7f90e1278740 1 -- 172.21.15.12:0/4136300597 shutdown_connections 2024-04-22T13:28:32.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.322+0000 7f90e1278740 1 --2- 172.21.15.12:0/4136300597 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5583afa62970 0x5583afa64d60 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.322+0000 7f90e1278740 1 --2- 172.21.15.12:0/4136300597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583afa52a80 0x5583afa50490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.322+0000 7f90e1278740 1 --2- 172.21.15.12:0/4136300597 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583afa49ea0 0x5583afa4ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.322+0000 7f90e1278740 1 --2- 172.21.15.12:0/4136300597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583afa49580 0x5583afa4fa10 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.322+0000 7f90b77fe640 0 client.17187 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:32.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.322+0000 7f90e1278740 1 -- 172.21.15.12:0/4136300597 >> 172.21.15.12:0/4136300597 conn(0x5583afa43cb0 msgr2=0x5583afa45a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:32.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.323+0000 7f90e1278740 1 -- 172.21.15.12:0/4136300597 shutdown_connections 2024-04-22T13:28:32.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.323+0000 7f90e1278740 1 -- 172.21.15.12:0/4136300597 wait complete. 2024-04-22T13:28:32.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.323+0000 7f90e1278740 20 client.17187 trim_cache size 0 max 16384 2024-04-22T13:28:32.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.340+0000 7f90e1278740 1 Processor -- start 2024-04-22T13:28:32.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.340+0000 7f90e1278740 1 -- start start 2024-04-22T13:28:32.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.341+0000 7f90e1278740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583afa49580 0x5583afa52140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.341+0000 7f90e1278740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583afa49ea0 0x5583afa5e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.341+0000 7f90e1278740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583afa52a80 0x5583afa6a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.341+0000 7f90e1278740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f90d81ad920 con 0x5583afa52a80 2024-04-22T13:28:32.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.341+0000 7f90e1278740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f90d81af460 con 0x5583afa49580 2024-04-22T13:28:32.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.341+0000 7f90e1278740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f90d81adff0 con 0x5583afa49ea0 2024-04-22T13:28:32.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.341+0000 7f90dcc1e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583afa52a80 0x5583afa6a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.341+0000 7f90dcc1e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583afa52a80 0x5583afa6a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60892/0 (socket says 172.21.15.12:60892) 2024-04-22T13:28:32.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.341+0000 7f90dcc1e640 1 -- 172.21.15.12:0/2725740877 learned_addr learned my addr 172.21.15.12:0/2725740877 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:32.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.341+0000 7f90dcc1e640 1 -- 172.21.15.12:0/2725740877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583afa49ea0 msgr2=0x5583afa5e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:32.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.341+0000 7f90d77fe640 1 --2- 172.21.15.12:0/2725740877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583afa49ea0 0x5583afa5e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.341+0000 7f90d7fff640 1 --2- 172.21.15.12:0/2725740877 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583afa49580 0x5583afa52140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.341+0000 7f90dcc1e640 1 --2- 172.21.15.12:0/2725740877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583afa49ea0 0x5583afa5e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.341+0000 7f90dcc1e640 1 -- 172.21.15.12:0/2725740877 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583afa49580 msgr2=0x5583afa52140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:32.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.341+0000 7f90dcc1e640 1 --2- 172.21.15.12:0/2725740877 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583afa49580 0x5583afa52140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.341+0000 7f90dcc1e640 1 -- 172.21.15.12:0/2725740877 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583afa5f210 con 0x5583afa52a80 2024-04-22T13:28:32.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.341+0000 7f90d77fe640 1 --2- 172.21.15.12:0/2725740877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583afa49ea0 0x5583afa5e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:32.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.341+0000 7f90d7fff640 1 --2- 172.21.15.12:0/2725740877 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583afa49580 0x5583afa52140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:32.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.341+0000 7f90dcc1e640 1 --2- 172.21.15.12:0/2725740877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583afa52a80 0x5583afa6a3e0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f90d0009cd0 tx=0x7f90d0005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0f4e3da576c8d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:32.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.341+0000 7f90cdffb640 1 -- 172.21.15.12:0/2725740877 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0039040 con 0x5583afa52a80 2024-04-22T13:28:32.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.341+0000 7f90cdffb640 1 -- 172.21.15.12:0/2725740877 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90d0003b70 con 0x5583afa52a80 2024-04-22T13:28:32.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.341+0000 7f90cdffb640 1 -- 172.21.15.12:0/2725740877 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d00346a0 con 0x5583afa52a80 2024-04-22T13:28:32.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.342+0000 7f90e1278740 1 -- 172.21.15.12:0/2725740877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583afa52a80 msgr2=0x5583afa6a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.342+0000 7f90e1278740 1 --2- 172.21.15.12:0/2725740877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583afa52a80 0x5583afa6a3e0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f90d0009cd0 tx=0x7f90d0005c50 comp rx=0 tx=0).stop 2024-04-22T13:28:32.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.342+0000 7f90e1278740 1 -- 172.21.15.12:0/2725740877 shutdown_connections 2024-04-22T13:28:32.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.342+0000 7f90e1278740 1 --2- 172.21.15.12:0/2725740877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583afa52a80 0x5583afa6a3e0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.342+0000 7f90e1278740 1 --2- 172.21.15.12:0/2725740877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583afa49ea0 0x5583afa5e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.342+0000 7f90e1278740 1 --2- 172.21.15.12:0/2725740877 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583afa49580 0x5583afa52140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.342+0000 7f90e1278740 1 -- 172.21.15.12:0/2725740877 >> 172.21.15.12:0/2725740877 conn(0x5583afa43cb0 msgr2=0x5583afa4e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:32.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.342+0000 7f90e1278740 1 -- 172.21.15.12:0/2725740877 shutdown_connections 2024-04-22T13:28:32.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.342+0000 7f90e1278740 1 -- 172.21.15.12:0/2725740877 wait complete. 2024-04-22T13:28:32.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.342+0000 7f90e1278740 1 -- start start 2024-04-22T13:28:32.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.343+0000 7f90e1278740 1 Processor -- start 2024-04-22T13:28:32.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.343+0000 7f90e1278740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583afa49580 0x5583afac0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.343+0000 7f90e1278740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583afa49ea0 0x5583afac1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.343+0000 7f90e1278740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583afa52a80 0x5583afa55e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.343+0000 7f90e1278740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f90d81aed90 con 0x5583afa49580 2024-04-22T13:28:32.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.343+0000 7f90e1278740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f90d81ae6c0 con 0x5583afa52a80 2024-04-22T13:28:32.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.343+0000 7f90e1278740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f90d81ab900 con 0x5583afa49ea0 2024-04-22T13:28:32.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.343+0000 7f90e1278740 10 client.0 _ll_register_callbacks cb 0x7f90c000cab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:32.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.343+0000 7f90d7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583afa49580 0x5583afac0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.343+0000 7f90d7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583afa49580 0x5583afac0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60894/0 (socket says 172.21.15.12:60894) 2024-04-22T13:28:32.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.343+0000 7f90d7fff640 1 -- 172.21.15.12:0/4185885406 learned_addr learned my addr 172.21.15.12:0/4185885406 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:32.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.343+0000 7f90d77fe640 1 --2- 172.21.15.12:0/4185885406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583afa49ea0 0x5583afac1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.343+0000 7f90dcc1e640 1 --2- 172.21.15.12:0/4185885406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583afa52a80 0x5583afa55e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.343+0000 7f90d7fff640 1 -- 172.21.15.12:0/4185885406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583afa49ea0 msgr2=0x5583afac1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:32.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.343+0000 7f90d7fff640 1 --2- 172.21.15.12:0/4185885406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583afa49ea0 0x5583afac1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.343+0000 7f90d7fff640 1 -- 172.21.15.12:0/4185885406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583afa52a80 msgr2=0x5583afa55e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:32.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.343+0000 7f90d7fff640 1 --2- 172.21.15.12:0/4185885406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583afa52a80 0x5583afa55e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.343+0000 7f90d7fff640 1 -- 172.21.15.12:0/4185885406 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583afa56510 con 0x5583afa49580 2024-04-22T13:28:32.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.343+0000 7f90dcc1e640 1 --2- 172.21.15.12:0/4185885406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583afa52a80 0x5583afa55e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:32.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.343+0000 7f90d7fff640 1 --2- 172.21.15.12:0/4185885406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583afa49580 0x5583afac0f50 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f90c000d010 tx=0x7f90c00107b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285a980005f89cb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:32.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.343+0000 7f90b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:32.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.344+0000 7f90b77fe640 1 -- 172.21.15.12:0/4185885406 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c002f030 con 0x5583afa49580 2024-04-22T13:28:32.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.344+0000 7f90b77fe640 1 -- 172.21.15.12:0/4185885406 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f90c001f070 con 0x5583afa49580 2024-04-22T13:28:32.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.344+0000 7f90e1278740 1 -- 172.21.15.12:0/4185885406 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90d0003b70 con 0x5583afa49580 2024-04-22T13:28:32.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.344+0000 7f90b77fe640 1 -- 172.21.15.12:0/4185885406 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f90c0014040 con 0x5583afa49580 2024-04-22T13:28:32.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.344+0000 7f90e1278740 10 client.17193 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:32.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.344+0000 7f90e1278740 1 -- 172.21.15.12:0/4185885406 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5583afac86d0 con 0x5583afa49580 2024-04-22T13:28:32.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.344+0000 7f90b77fe640 1 -- 172.21.15.12:0/4185885406 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90d0003b70 con 0x5583afa49580 2024-04-22T13:28:32.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.344+0000 7f90b77fe640 1 -- 172.21.15.12:0/4185885406 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f90c0005040 con 0x5583afa49580 2024-04-22T13:28:32.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.344+0000 7f90e1278740 10 client.17193 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:32.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.344+0000 7f90e1278740 10 client.17193 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:32.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.344+0000 7f90e1278740 20 client.17193 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:32.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.344+0000 7f90e1278740 1 --2- 172.21.15.12:0/4185885406 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5583afac8e50 0x5583afacb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.344+0000 7f90e1278740 4 client.17193 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:28:32.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.344+0000 7f90e1278740 1 -- 172.21.15.12:0/4185885406 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5583af9b5a70 con 0x5583afac8e50 2024-04-22T13:28:32.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.345+0000 7f90d77fe640 1 --2- 172.21.15.12:0/4185885406 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5583afac8e50 0x5583afacb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.345+0000 7f90d77fe640 1 --2- 172.21.15.12:0/4185885406 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5583afac8e50 0x5583afacb270 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5508b60209a90066 server_cookie=b541a4b17c4336ec in_seq=0 out_seq=0 2024-04-22T13:28:32.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.345+0000 7f90b77fe640 10 client.17193 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:32.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.345+0000 7f90b77fe640 1 -- 172.21.15.12:0/4185885406 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5583af9b5a70 con 0x5583afac8e50 2024-04-22T13:28:32.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.345+0000 7f90b77fe640 10 client.17193 handle_command_reply: tid=0 2024-04-22T13:28:32.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.345+0000 7f90e1278740 1 client.17193 shutdown 2024-04-22T13:28:32.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.345+0000 7f90e1278740 1 -- 172.21.15.12:0/4185885406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583afa49580 msgr2=0x5583afac0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.345+0000 7f90e1278740 1 --2- 172.21.15.12:0/4185885406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583afa49580 0x5583afac0f50 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f90c000d010 tx=0x7f90c00107b0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.345+0000 7f90e1278740 1 -- 172.21.15.12:0/4185885406 shutdown_connections 2024-04-22T13:28:32.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.345+0000 7f90e1278740 1 --2- 172.21.15.12:0/4185885406 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5583afac8e50 0x5583afacb270 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.345+0000 7f90e1278740 1 --2- 172.21.15.12:0/4185885406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583afa52a80 0x5583afa55e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.345+0000 7f90e1278740 1 --2- 172.21.15.12:0/4185885406 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583afa49ea0 0x5583afac1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.345+0000 7f90b77fe640 0 client.17193 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:32.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.345+0000 7f90e1278740 1 --2- 172.21.15.12:0/4185885406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583afa49580 0x5583afac0f50 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.345+0000 7f90e1278740 1 -- 172.21.15.12:0/4185885406 >> 172.21.15.12:0/4185885406 conn(0x5583afa43cb0 msgr2=0x5583afa4f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:32.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.345+0000 7f90e1278740 1 -- 172.21.15.12:0/4185885406 shutdown_connections 2024-04-22T13:28:32.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.346+0000 7f90e1278740 1 -- 172.21.15.12:0/4185885406 wait complete. 2024-04-22T13:28:32.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.346+0000 7f90e1278740 20 client.17193 trim_cache size 0 max 16384 2024-04-22T13:28:32.350 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:28:32.350 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-04-22T13:28:32.350 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-04-22T13:28:32.350 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:28:32.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.346+0000 7f90de681640 1 -- 172.21.15.12:0/40124380 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f90c40a3d20 msgr2=0x7f90c40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.346+0000 7f90de681640 1 --2- 172.21.15.12:0/40124380 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f90c40a3d20 0x7f90c40a61e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f90d0005cc0 tx=0x7f90d002e040 comp rx=0 tx=0).stop 2024-04-22T13:28:32.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.346+0000 7f90de681640 1 -- 172.21.15.12:0/40124380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d806af70 msgr2=0x7f90d8198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.346+0000 7f90de681640 1 --2- 172.21.15.12:0/40124380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d806af70 0x7f90d8198a60 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f90c800c920 tx=0x7f90c800cde0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.346+0000 7f90de681640 1 -- 172.21.15.12:0/40124380 shutdown_connections 2024-04-22T13:28:32.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.346+0000 7f90de681640 1 --2- 172.21.15.12:0/40124380 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f90c40a3d20 0x7f90c40a61e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.346+0000 7f90de681640 1 --2- 172.21.15.12:0/40124380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90d8108060 0x7f90d819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.346+0000 7f90de681640 1 --2- 172.21.15.12:0/40124380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f90d806af70 0x7f90d8198a60 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.346+0000 7f90de681640 1 --2- 172.21.15.12:0/40124380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90d806a5b0 0x7f90d8198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.346+0000 7f90de681640 1 -- 172.21.15.12:0/40124380 >> 172.21.15.12:0/40124380 conn(0x7f90d8077c40 msgr2=0x7f90d80ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:32.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.346+0000 7f90de681640 1 -- 172.21.15.12:0/40124380 shutdown_connections 2024-04-22T13:28:32.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.346+0000 7f90de681640 1 -- 172.21.15.12:0/40124380 wait complete. 2024-04-22T13:28:32.354 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:28:32.354 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:28:32.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.460+0000 7ff804ee9640 1 -- 172.21.15.12:0/1364626221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff800101110 msgr2=0x7ff800101570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.460+0000 7ff804ee9640 1 --2- 172.21.15.12:0/1364626221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff800101110 0x7ff800101570 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009930 tx=0x7ff7e802bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:32.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.460+0000 7ff804ee9640 1 -- 172.21.15.12:0/1364626221 shutdown_connections 2024-04-22T13:28:32.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.460+0000 7ff804ee9640 1 --2- 172.21.15.12:0/1364626221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff800101ab0 0x7ff80010dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.460+0000 7ff804ee9640 1 --2- 172.21.15.12:0/1364626221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff800101110 0x7ff800101570 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.460+0000 7ff804ee9640 1 --2- 172.21.15.12:0/1364626221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff800107010 0x7ff8001073f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.460+0000 7ff804ee9640 1 -- 172.21.15.12:0/1364626221 >> 172.21.15.12:0/1364626221 conn(0x7ff800077e30 msgr2=0x7ff800078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:32.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.460+0000 7ff804ee9640 1 -- 172.21.15.12:0/1364626221 shutdown_connections 2024-04-22T13:28:32.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.460+0000 7ff804ee9640 1 -- 172.21.15.12:0/1364626221 wait complete. 2024-04-22T13:28:32.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.461+0000 7ff804ee9640 1 Processor -- start 2024-04-22T13:28:32.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.461+0000 7ff804ee9640 1 -- start start 2024-04-22T13:28:32.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.461+0000 7ff804ee9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff800101110 0x7ff8001a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.461+0000 7ff804ee9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff800101ab0 0x7ff8001a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.461+0000 7ff804ee9640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff800107010 0x7ff80019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.461+0000 7ff804ee9640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff800112250 con 0x7ff800101ab0 2024-04-22T13:28:32.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.461+0000 7ff804ee9640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff8001120d0 con 0x7ff800107010 2024-04-22T13:28:32.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.461+0000 7ff804ee9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff8001123d0 con 0x7ff800101110 2024-04-22T13:28:32.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.461+0000 7ff7f7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff800101ab0 0x7ff8001a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.461+0000 7ff7f7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff800101ab0 0x7ff8001a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60906/0 (socket says 172.21.15.12:60906) 2024-04-22T13:28:32.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.461+0000 7ff7f7fff640 1 -- 172.21.15.12:0/3996174335 learned_addr learned my addr 172.21.15.12:0/3996174335 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:32.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.461+0000 7ff7fe59c640 1 --2- 172.21.15.12:0/3996174335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff800101110 0x7ff8001a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.461+0000 7ff7fed9d640 1 --2- 172.21.15.12:0/3996174335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff800107010 0x7ff80019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.461+0000 7ff7f7fff640 1 -- 172.21.15.12:0/3996174335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff800101110 msgr2=0x7ff8001a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.461+0000 7ff7f7fff640 1 --2- 172.21.15.12:0/3996174335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff800101110 0x7ff8001a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.461+0000 7ff7f7fff640 1 -- 172.21.15.12:0/3996174335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff800107010 msgr2=0x7ff80019a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.461+0000 7ff7f7fff640 1 --2- 172.21.15.12:0/3996174335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff800107010 0x7ff80019a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.461+0000 7ff7f7fff640 1 -- 172.21.15.12:0/3996174335 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff80019b010 con 0x7ff800101ab0 2024-04-22T13:28:32.461 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.462+0000 7ff7f7fff640 1 --2- 172.21.15.12:0/3996174335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff800101ab0 0x7ff8001a0bd0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009230 tx=0x7ff7e802c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:32.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.462+0000 7ff7f77fe640 1 -- 172.21.15.12:0/3996174335 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8043020 con 0x7ff800101ab0 2024-04-22T13:28:32.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.462+0000 7ff804ee9640 1 -- 172.21.15.12:0/3996174335 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff80019b2a0 con 0x7ff800101ab0 2024-04-22T13:28:32.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.462+0000 7ff7f77fe640 1 -- 172.21.15.12:0/3996174335 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7e802caf0 con 0x7ff800101ab0 2024-04-22T13:28:32.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.462+0000 7ff7f77fe640 1 -- 172.21.15.12:0/3996174335 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8003af0 con 0x7ff800101ab0 2024-04-22T13:28:32.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.462+0000 7ff804ee9640 1 -- 172.21.15.12:0/3996174335 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8001a7610 con 0x7ff800101ab0 2024-04-22T13:28:32.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.463+0000 7ff804ee9640 1 -- 172.21.15.12:0/3996174335 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff800102580 con 0x7ff800101ab0 2024-04-22T13:28:32.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.464+0000 7ff7f77fe640 1 -- 172.21.15.12:0/3996174335 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff7e8046020 con 0x7ff800101ab0 2024-04-22T13:28:32.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.464+0000 7ff7f77fe640 1 --2- 172.21.15.12:0/3996174335 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff7ec0a3d20 0x7ff7ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.465+0000 7ff7fe59c640 1 --2- 172.21.15.12:0/3996174335 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff7ec0a3d20 0x7ff7ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.465+0000 7ff7f77fe640 1 -- 172.21.15.12:0/3996174335 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7ff7e80f26e0 con 0x7ff800101ab0 2024-04-22T13:28:32.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.465+0000 7ff7fe59c640 1 --2- 172.21.15.12:0/3996174335 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff7ec0a3d20 0x7ff7ec0a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0005d00 tx=0x7ff7f000b410 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:32.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.469+0000 7ff7f77fe640 1 -- 172.21.15.12:0/3996174335 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff7e804d020 con 0x7ff800101ab0 2024-04-22T13:28:32.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.701+0000 7ff804ee9640 1 -- 172.21.15.12:0/3996174335 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff80019b5f0 con 0x7ff800101ab0 2024-04-22T13:28:32.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.702+0000 7ff7f77fe640 1 -- 172.21.15.12:0/3996174335 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7ff7e80beee0 con 0x7ff800101ab0 2024-04-22T13:28:32.699 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:28:32.699 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:28:32.699 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:28:32.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.704+0000 7ff804ee9640 1 -- 172.21.15.12:0/3996174335 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff7ec0a3d20 msgr2=0x7ff7ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.704+0000 7ff804ee9640 1 --2- 172.21.15.12:0/3996174335 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff7ec0a3d20 0x7ff7ec0a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0005d00 tx=0x7ff7f000b410 comp rx=0 tx=0).stop 2024-04-22T13:28:32.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.704+0000 7ff804ee9640 1 -- 172.21.15.12:0/3996174335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff800101ab0 msgr2=0x7ff8001a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.704+0000 7ff804ee9640 1 --2- 172.21.15.12:0/3996174335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff800101ab0 0x7ff8001a0bd0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009230 tx=0x7ff7e802c710 comp rx=0 tx=0).stop 2024-04-22T13:28:32.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.704+0000 7ff804ee9640 1 -- 172.21.15.12:0/3996174335 shutdown_connections 2024-04-22T13:28:32.701 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.704+0000 7ff804ee9640 1 --2- 172.21.15.12:0/3996174335 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff7ec0a3d20 0x7ff7ec0a61e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.704+0000 7ff804ee9640 1 --2- 172.21.15.12:0/3996174335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff800107010 0x7ff80019a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.704+0000 7ff804ee9640 1 --2- 172.21.15.12:0/3996174335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff800101ab0 0x7ff8001a0bd0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.704+0000 7ff804ee9640 1 --2- 172.21.15.12:0/3996174335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff800101110 0x7ff8001a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.704+0000 7ff804ee9640 1 -- 172.21.15.12:0/3996174335 >> 172.21.15.12:0/3996174335 conn(0x7ff800077e30 msgr2=0x7ff80010ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:32.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.704+0000 7ff804ee9640 1 -- 172.21.15.12:0/3996174335 shutdown_connections 2024-04-22T13:28:32.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.704+0000 7ff804ee9640 1 -- 172.21.15.12:0/3996174335 wait complete. 2024-04-22T13:28:32.713 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:28:32.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.821+0000 7f47d5af1640 1 -- 172.21.15.12:0/45213515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47d006b1f0 msgr2=0x7f47d0106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.821+0000 7f47d5af1640 1 --2- 172.21.15.12:0/45213515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47d006b1f0 0x7f47d0106b90 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f47c4009900 tx=0x7f47c402bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:32.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.821+0000 7f47d5af1640 1 -- 172.21.15.12:0/45213515 shutdown_connections 2024-04-22T13:28:32.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.821+0000 7f47d5af1640 1 --2- 172.21.15.12:0/45213515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47d0107270 0x7f47d010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.821+0000 7f47d5af1640 1 --2- 172.21.15.12:0/45213515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47d006b1f0 0x7f47d0106b90 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.821+0000 7f47d5af1640 1 --2- 172.21.15.12:0/45213515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47d006a840 0x7f47d006ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.821+0000 7f47d5af1640 1 -- 172.21.15.12:0/45213515 >> 172.21.15.12:0/45213515 conn(0x7f47d0077eb0 msgr2=0x7f47d00782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:32.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.821+0000 7f47d5af1640 1 -- 172.21.15.12:0/45213515 shutdown_connections 2024-04-22T13:28:32.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.821+0000 7f47d5af1640 1 -- 172.21.15.12:0/45213515 wait complete. 2024-04-22T13:28:32.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.822+0000 7f47d5af1640 1 Processor -- start 2024-04-22T13:28:32.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.822+0000 7f47d5af1640 1 -- start start 2024-04-22T13:28:32.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.822+0000 7f47d5af1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47d006a840 0x7f47d01986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.822+0000 7f47d5af1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47d0107270 0x7f47d0198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.822+0000 7f47d5af1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47d019cfb0 0x7f47d019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.822+0000 7f47d5af1640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f47d01121b0 con 0x7f47d019cfb0 2024-04-22T13:28:32.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.822+0000 7f47d5af1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f47d0112030 con 0x7f47d0107270 2024-04-22T13:28:32.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.822+0000 7f47d5af1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f47d0112330 con 0x7f47d006a840 2024-04-22T13:28:32.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.822+0000 7f47cffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47d019cfb0 0x7f47d019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.823+0000 7f47cffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47d019cfb0 0x7f47d019d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60928/0 (socket says 172.21.15.12:60928) 2024-04-22T13:28:32.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.823+0000 7f47cffff640 1 -- 172.21.15.12:0/2223747310 learned_addr learned my addr 172.21.15.12:0/2223747310 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:32.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.823+0000 7f47ceffd640 1 --2- 172.21.15.12:0/2223747310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47d0107270 0x7f47d0198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.823+0000 7f47cf7fe640 1 --2- 172.21.15.12:0/2223747310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47d006a840 0x7f47d01986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.823+0000 7f47cffff640 1 -- 172.21.15.12:0/2223747310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47d006a840 msgr2=0x7f47d01986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.823+0000 7f47cffff640 1 --2- 172.21.15.12:0/2223747310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47d006a840 0x7f47d01986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.823+0000 7f47cffff640 1 -- 172.21.15.12:0/2223747310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47d0107270 msgr2=0x7f47d0198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.823+0000 7f47cffff640 1 --2- 172.21.15.12:0/2223747310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47d0107270 0x7f47d0198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.823+0000 7f47cffff640 1 -- 172.21.15.12:0/2223747310 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d019daa0 con 0x7f47d019cfb0 2024-04-22T13:28:32.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.823+0000 7f47cffff640 1 --2- 172.21.15.12:0/2223747310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47d019cfb0 0x7f47d019d460 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f47c800eaa0 tx=0x7f47c800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:32.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.823+0000 7f47ccff9640 1 -- 172.21.15.12:0/2223747310 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c800cdb0 con 0x7f47d019cfb0 2024-04-22T13:28:32.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.823+0000 7f47d5af1640 1 -- 172.21.15.12:0/2223747310 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47d019dd90 con 0x7f47d019cfb0 2024-04-22T13:28:32.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.823+0000 7f47ccff9640 1 -- 172.21.15.12:0/2223747310 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47c8005020 con 0x7f47d019cfb0 2024-04-22T13:28:32.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.823+0000 7f47ccff9640 1 -- 172.21.15.12:0/2223747310 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c80116d0 con 0x7f47d019cfb0 2024-04-22T13:28:32.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.823+0000 7f47d5af1640 1 -- 172.21.15.12:0/2223747310 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47d01a5550 con 0x7f47d019cfb0 2024-04-22T13:28:32.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.826+0000 7f47ccff9640 1 -- 172.21.15.12:0/2223747310 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f47c80051d0 con 0x7f47d019cfb0 2024-04-22T13:28:32.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.826+0000 7f47d87b6740 1 Processor -- start 2024-04-22T13:28:32.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.826+0000 7f47d87b6740 1 -- start start 2024-04-22T13:28:32.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.826+0000 7f47d87b6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc39104d0 0x55fdc39108b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.826+0000 7f47d87b6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdc3910e80 0x55fdc3919430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.826+0000 7f47d87b6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdc3919a00 0x55fdc391be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.826+0000 7f47d87b6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fdc38751f0 con 0x55fdc3919a00 2024-04-22T13:28:32.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.826+0000 7f47d87b6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fdc38124b0 con 0x55fdc3910e80 2024-04-22T13:28:32.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.826+0000 7f47d87b6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fdc38711b0 con 0x55fdc39104d0 2024-04-22T13:28:32.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.826+0000 7f47ceffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdc3919a00 0x55fdc391be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.829 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.826+0000 7f47ceffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdc3919a00 0x55fdc391be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60932/0 (socket says 172.21.15.12:60932) 2024-04-22T13:28:32.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.826+0000 7f47ceffd640 1 -- 172.21.15.12:0/1070765655 learned_addr learned my addr 172.21.15.12:0/1070765655 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:32.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.826+0000 7f47ceffd640 1 --2- 172.21.15.12:0/1070765655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc39104d0 0x55fdc39108b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.826+0000 7f47cf7fe640 1 --2- 172.21.15.12:0/1070765655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdc3910e80 0x55fdc3919430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.826+0000 7f47ccff9640 1 --2- 172.21.15.12:0/2223747310 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f47b80a3d20 0x7f47b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.827+0000 7f47ceffd640 1 -- 172.21.15.12:0/1070765655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc39104d0 msgr2=0x55fdc39108b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:32.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.827+0000 7f47ceffd640 1 --2- 172.21.15.12:0/1070765655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc39104d0 0x55fdc39108b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.827+0000 7f47ceffd640 1 -- 172.21.15.12:0/1070765655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdc3910e80 msgr2=0x55fdc3919430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:32.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.827+0000 7f47ceffd640 1 --2- 172.21.15.12:0/1070765655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdc3910e80 0x55fdc3919430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.827+0000 7f47cffff640 1 --2- 172.21.15.12:0/2223747310 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f47b80a3d20 0x7f47b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.827+0000 7f47ceffd640 1 -- 172.21.15.12:0/1070765655 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdc38041b0 con 0x55fdc3919a00 2024-04-22T13:28:32.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.827+0000 7f47ccff9640 1 -- 172.21.15.12:0/2223747310 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f47c80d08f0 con 0x7f47d019cfb0 2024-04-22T13:28:32.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.827+0000 7f47ceffd640 1 --2- 172.21.15.12:0/1070765655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdc3919a00 0x55fdc391be50 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f47d006a420 tx=0x7f47c4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53da3a52e926990 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:32.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.827+0000 7f47b5ffb640 1 -- 172.21.15.12:0/1070765655 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c404a070 con 0x55fdc3919a00 2024-04-22T13:28:32.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.827+0000 7f47b5ffb640 1 -- 172.21.15.12:0/1070765655 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47c402cc90 con 0x55fdc3919a00 2024-04-22T13:28:32.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.827+0000 7f47b5ffb640 1 -- 172.21.15.12:0/1070765655 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c40046c0 con 0x55fdc3919a00 2024-04-22T13:28:32.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.828+0000 7f47d87b6740 1 -- 172.21.15.12:0/1070765655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdc3919a00 msgr2=0x55fdc391be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.828+0000 7f47cffff640 1 --2- 172.21.15.12:0/2223747310 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f47b80a3d20 0x7f47b80a61e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f47c809c5e0 tx=0x7f47c8098c00 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:32.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.828+0000 7f47d87b6740 1 --2- 172.21.15.12:0/1070765655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdc3919a00 0x55fdc391be50 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f47d006a420 tx=0x7f47c4005e80 comp rx=0 tx=0).stop 2024-04-22T13:28:32.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.828+0000 7f47d87b6740 1 -- 172.21.15.12:0/1070765655 shutdown_connections 2024-04-22T13:28:32.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.828+0000 7f47d87b6740 1 --2- 172.21.15.12:0/1070765655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdc3919a00 0x55fdc391be50 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.828+0000 7f47d87b6740 1 --2- 172.21.15.12:0/1070765655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdc3910e80 0x55fdc3919430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.828+0000 7f47d87b6740 1 --2- 172.21.15.12:0/1070765655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc39104d0 0x55fdc39108b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.828+0000 7f47d87b6740 1 -- 172.21.15.12:0/1070765655 >> 172.21.15.12:0/1070765655 conn(0x55fdc390aba0 msgr2=0x55fdc390af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:32.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.828+0000 7f47d87b6740 1 -- 172.21.15.12:0/1070765655 shutdown_connections 2024-04-22T13:28:32.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.828+0000 7f47d87b6740 1 -- 172.21.15.12:0/1070765655 wait complete. 2024-04-22T13:28:32.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.828+0000 7f47d87b6740 1 -- start start 2024-04-22T13:28:32.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.828+0000 7f47d87b6740 1 Processor -- start 2024-04-22T13:28:32.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.829+0000 7f47d87b6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdc39104d0 0x55fdc3919360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.829+0000 7f47d87b6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdc3910e80 0x55fdc39166f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.829+0000 7f47d87b6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc3919a00 0x55fdc3916d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.829+0000 7f47d87b6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fdc391c910 con 0x55fdc3910e80 2024-04-22T13:28:32.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.829+0000 7f47d87b6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fdc382e500 con 0x55fdc39104d0 2024-04-22T13:28:32.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.829+0000 7f47d87b6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fdc391ca90 con 0x55fdc3919a00 2024-04-22T13:28:32.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.829+0000 7f47d87b6740 10 client.0 _ll_register_callbacks cb 0x55fdc37fcb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:32.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.829+0000 7f47cf7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdc3910e80 0x55fdc39166f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.829+0000 7f47cf7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdc3910e80 0x55fdc39166f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60940/0 (socket says 172.21.15.12:60940) 2024-04-22T13:28:32.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.829+0000 7f47cf7fe640 1 -- 172.21.15.12:0/2642350098 learned_addr learned my addr 172.21.15.12:0/2642350098 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:32.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.829+0000 7f47ceffd640 1 --2- 172.21.15.12:0/2642350098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc3919a00 0x55fdc3916d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.829+0000 7f47ceffd640 1 --2- 172.21.15.12:0/2642350098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdc39104d0 0x55fdc3919360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.829+0000 7f47cf7fe640 1 -- 172.21.15.12:0/2642350098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc3919a00 msgr2=0x55fdc3916d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:32.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.829+0000 7f47cf7fe640 1 --2- 172.21.15.12:0/2642350098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc3919a00 0x55fdc3916d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.829+0000 7f47cf7fe640 1 -- 172.21.15.12:0/2642350098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdc39104d0 msgr2=0x55fdc3919360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:32.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.829+0000 7f47cf7fe640 1 --2- 172.21.15.12:0/2642350098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdc39104d0 0x55fdc3919360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.829+0000 7f47cf7fe640 1 -- 172.21.15.12:0/2642350098 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdc35873f0 con 0x55fdc3910e80 2024-04-22T13:28:32.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.829+0000 7f47ceffd640 1 --2- 172.21.15.12:0/2642350098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdc39104d0 0x55fdc3919360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:32.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.829+0000 7f47cf7fe640 1 --2- 172.21.15.12:0/2642350098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdc3910e80 0x55fdc39166f0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f47c000da10 tx=0x7f47c000ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba92f1431ff87dd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:32.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.829+0000 7f47aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:32.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.829+0000 7f47aeffd640 1 -- 172.21.15.12:0/2642350098 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c001b640 con 0x55fdc3910e80 2024-04-22T13:28:32.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.829+0000 7f47aeffd640 1 -- 172.21.15.12:0/2642350098 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f47c001b7e0 con 0x55fdc3910e80 2024-04-22T13:28:32.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.829+0000 7f47d87b6740 1 -- 172.21.15.12:0/2642350098 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47c402cc90 con 0x55fdc3910e80 2024-04-22T13:28:32.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.830+0000 7f47aeffd640 1 -- 172.21.15.12:0/2642350098 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f47c001b9e0 con 0x55fdc3910e80 2024-04-22T13:28:32.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.830+0000 7f47d87b6740 10 client.17211 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:32.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.830+0000 7f47d87b6740 1 -- 172.21.15.12:0/2642350098 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fdc36517d0 con 0x55fdc3910e80 2024-04-22T13:28:32.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.830+0000 7f47aeffd640 1 -- 172.21.15.12:0/2642350098 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47c402cc90 con 0x55fdc3910e80 2024-04-22T13:28:32.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.830+0000 7f47aeffd640 1 -- 172.21.15.12:0/2642350098 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f47c0019020 con 0x55fdc3910e80 2024-04-22T13:28:32.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.830+0000 7f47d87b6740 10 client.17211 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:32.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.830+0000 7f47d87b6740 10 client.17211 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:28:32.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.830+0000 7f47d87b6740 20 client.17211 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:32.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.830+0000 7f47d87b6740 1 --2- 172.21.15.12:0/2642350098 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55fdc3929ba0 0x55fdc392bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.830+0000 7f47d87b6740 4 client.17211 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:32.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.830+0000 7f47d87b6740 1 -- 172.21.15.12:0/2642350098 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdc3814930 con 0x55fdc3929ba0 2024-04-22T13:28:32.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.830+0000 7f47ceffd640 1 --2- 172.21.15.12:0/2642350098 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55fdc3929ba0 0x55fdc392bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.831+0000 7f47aeffd640 10 client.17211 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:28:32.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.831+0000 7f47ceffd640 1 --2- 172.21.15.12:0/2642350098 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55fdc3929ba0 0x55fdc392bfd0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3b38025a4765bdb4 server_cookie=ab86c2b38a98df85 in_seq=0 out_seq=0 2024-04-22T13:28:32.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.831+0000 7f47aeffd640 1 -- 172.21.15.12:0/2642350098 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fdc3814930 con 0x55fdc3929ba0 2024-04-22T13:28:32.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.831+0000 7f47aeffd640 10 client.17211 handle_command_reply: tid=0 2024-04-22T13:28:32.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.831+0000 7f47d87b6740 1 client.17211 shutdown 2024-04-22T13:28:32.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.832+0000 7f47d87b6740 1 -- 172.21.15.12:0/2642350098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdc3910e80 msgr2=0x55fdc39166f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.832+0000 7f47d87b6740 1 --2- 172.21.15.12:0/2642350098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdc3910e80 0x55fdc39166f0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f47c000da10 tx=0x7f47c000ded0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.832+0000 7f47d87b6740 1 -- 172.21.15.12:0/2642350098 shutdown_connections 2024-04-22T13:28:32.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.832+0000 7f47d87b6740 1 --2- 172.21.15.12:0/2642350098 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55fdc3929ba0 0x55fdc392bfd0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.832+0000 7f47d87b6740 1 --2- 172.21.15.12:0/2642350098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc3919a00 0x55fdc3916d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.832+0000 7f47d87b6740 1 --2- 172.21.15.12:0/2642350098 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdc3910e80 0x55fdc39166f0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.832+0000 7f47aeffd640 0 client.17211 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:28:32.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.832+0000 7f47d87b6740 1 --2- 172.21.15.12:0/2642350098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdc39104d0 0x55fdc3919360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.832+0000 7f47d87b6740 1 -- 172.21.15.12:0/2642350098 >> 172.21.15.12:0/2642350098 conn(0x55fdc390aba0 msgr2=0x55fdc390c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:32.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.832+0000 7f47d87b6740 1 -- 172.21.15.12:0/2642350098 shutdown_connections 2024-04-22T13:28:32.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.832+0000 7f47d87b6740 1 -- 172.21.15.12:0/2642350098 wait complete. 2024-04-22T13:28:32.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.832+0000 7f47d87b6740 20 client.17211 trim_cache size 0 max 16384 2024-04-22T13:28:32.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.845+0000 7f47d87b6740 1 Processor -- start 2024-04-22T13:28:32.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.845+0000 7f47d87b6740 1 -- start start 2024-04-22T13:28:32.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.845+0000 7f47d87b6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdc39104d0 0x55fdc39257d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.845+0000 7f47d87b6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc3910e80 0x55fdc39220c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.845+0000 7f47d87b6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdc3919a00 0x55fdc3909a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.845+0000 7f47d87b6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f47d01adb10 con 0x55fdc39104d0 2024-04-22T13:28:32.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.845+0000 7f47d87b6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f47d01af650 con 0x55fdc3919a00 2024-04-22T13:28:32.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.845+0000 7f47d87b6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f47d01ae1e0 con 0x55fdc3910e80 2024-04-22T13:28:32.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.845+0000 7f47ceffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdc39104d0 0x55fdc39257d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.845+0000 7f47ceffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdc39104d0 0x55fdc39257d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60952/0 (socket says 172.21.15.12:60952) 2024-04-22T13:28:32.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.845+0000 7f47ceffd640 1 -- 172.21.15.12:0/2424393802 learned_addr learned my addr 172.21.15.12:0/2424393802 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:32.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.846+0000 7f47cf7fe640 1 --2- 172.21.15.12:0/2424393802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc3910e80 0x55fdc39220c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.846+0000 7f47ceffd640 1 -- 172.21.15.12:0/2424393802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc3910e80 msgr2=0x55fdc39220c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:32.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.846+0000 7f47ceffd640 1 --2- 172.21.15.12:0/2424393802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc3910e80 0x55fdc39220c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.846+0000 7f47ceffd640 1 -- 172.21.15.12:0/2424393802 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdc3919a00 msgr2=0x55fdc3909a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:32.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.846+0000 7f47ceffd640 1 --2- 172.21.15.12:0/2424393802 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdc3919a00 0x55fdc3909a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.846+0000 7f47ceffd640 1 -- 172.21.15.12:0/2424393802 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdc3917070 con 0x55fdc39104d0 2024-04-22T13:28:32.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.846+0000 7f47cf7fe640 1 --2- 172.21.15.12:0/2424393802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc3910e80 0x55fdc39220c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:32.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.846+0000 7f47ceffd640 1 --2- 172.21.15.12:0/2424393802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdc39104d0 0x55fdc39257d0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f47d006a420 tx=0x7f47c4005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f25a1770280ce11a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:32.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.846+0000 7f47b57fa640 1 -- 172.21.15.12:0/2424393802 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c4058070 con 0x55fdc39104d0 2024-04-22T13:28:32.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.846+0000 7f47b57fa640 1 -- 172.21.15.12:0/2424393802 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47c402cc90 con 0x55fdc39104d0 2024-04-22T13:28:32.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.846+0000 7f47b57fa640 1 -- 172.21.15.12:0/2424393802 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c404a040 con 0x55fdc39104d0 2024-04-22T13:28:32.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.847+0000 7f47d87b6740 1 -- 172.21.15.12:0/2424393802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdc39104d0 msgr2=0x55fdc39257d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.847+0000 7f47d87b6740 1 --2- 172.21.15.12:0/2424393802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdc39104d0 0x55fdc39257d0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f47d006a420 tx=0x7f47c4005de0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.847+0000 7f47d87b6740 1 -- 172.21.15.12:0/2424393802 shutdown_connections 2024-04-22T13:28:32.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.847+0000 7f47d87b6740 1 --2- 172.21.15.12:0/2424393802 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdc3919a00 0x55fdc3909a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.847+0000 7f47d87b6740 1 --2- 172.21.15.12:0/2424393802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc3910e80 0x55fdc39220c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.847+0000 7f47d87b6740 1 --2- 172.21.15.12:0/2424393802 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdc39104d0 0x55fdc39257d0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.847+0000 7f47d87b6740 1 -- 172.21.15.12:0/2424393802 >> 172.21.15.12:0/2424393802 conn(0x55fdc390aba0 msgr2=0x55fdc3909370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:32.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.847+0000 7f47d87b6740 1 -- 172.21.15.12:0/2424393802 shutdown_connections 2024-04-22T13:28:32.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.847+0000 7f47d87b6740 1 -- 172.21.15.12:0/2424393802 wait complete. 2024-04-22T13:28:32.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.847+0000 7f47d87b6740 1 -- start start 2024-04-22T13:28:32.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.847+0000 7f47d87b6740 1 Processor -- start 2024-04-22T13:28:32.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.847+0000 7f47d87b6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdc39104d0 0x55fdc39880d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.847+0000 7f47d87b6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc3910e80 0x55fdc3988610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.847+0000 7f47d87b6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdc3919a00 0x55fdc3988b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.847+0000 7f47d87b6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f47d01aef80 con 0x55fdc39104d0 2024-04-22T13:28:32.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.847+0000 7f47d87b6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f47d01ae8b0 con 0x55fdc3919a00 2024-04-22T13:28:32.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.847+0000 7f47d87b6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f47d01abaf0 con 0x55fdc3910e80 2024-04-22T13:28:32.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.847+0000 7f47d87b6740 10 client.0 _ll_register_callbacks cb 0x7f47c0004440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:32.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.847+0000 7f47ceffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdc39104d0 0x55fdc39880d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.847+0000 7f47ceffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdc39104d0 0x55fdc39880d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60954/0 (socket says 172.21.15.12:60954) 2024-04-22T13:28:32.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.848+0000 7f47ceffd640 1 -- 172.21.15.12:0/1237365152 learned_addr learned my addr 172.21.15.12:0/1237365152 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:32.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.848+0000 7f47cf7fe640 1 --2- 172.21.15.12:0/1237365152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc3910e80 0x55fdc3988610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.848+0000 7f47ceffd640 1 -- 172.21.15.12:0/1237365152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc3910e80 msgr2=0x55fdc3988610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:32.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.848+0000 7f47ceffd640 1 --2- 172.21.15.12:0/1237365152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc3910e80 0x55fdc3988610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.848+0000 7f47ceffd640 1 -- 172.21.15.12:0/1237365152 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdc3919a00 msgr2=0x55fdc3988b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:32.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.848+0000 7f47ceffd640 1 --2- 172.21.15.12:0/1237365152 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdc3919a00 0x55fdc3988b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.848+0000 7f47ceffd640 1 -- 172.21.15.12:0/1237365152 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdc3993180 con 0x55fdc39104d0 2024-04-22T13:28:32.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.848+0000 7f47cf7fe640 1 --2- 172.21.15.12:0/1237365152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc3910e80 0x55fdc3988610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:32.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.848+0000 7f47ceffd640 1 --2- 172.21.15.12:0/1237365152 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdc39104d0 0x55fdc39880d0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f47c402c170 tx=0x7f47c4005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea2e125c44ba24f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:32.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.848+0000 7f47aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:32.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.848+0000 7f47aeffd640 1 -- 172.21.15.12:0/1237365152 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c404a070 con 0x55fdc39104d0 2024-04-22T13:28:32.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.848+0000 7f47aeffd640 1 -- 172.21.15.12:0/1237365152 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f47c4052610 con 0x55fdc39104d0 2024-04-22T13:28:32.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.848+0000 7f47d87b6740 1 -- 172.21.15.12:0/1237365152 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47c402cc90 con 0x55fdc39104d0 2024-04-22T13:28:32.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.848+0000 7f47aeffd640 1 -- 172.21.15.12:0/1237365152 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f47c4061020 con 0x55fdc39104d0 2024-04-22T13:28:32.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.848+0000 7f47d87b6740 10 client.17217 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:32.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.848+0000 7f47d87b6740 1 -- 172.21.15.12:0/1237365152 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fdc39943f0 con 0x55fdc39104d0 2024-04-22T13:28:32.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.848+0000 7f47aeffd640 1 -- 172.21.15.12:0/1237365152 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47c402cc90 con 0x55fdc39104d0 2024-04-22T13:28:32.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.849+0000 7f47aeffd640 1 -- 172.21.15.12:0/1237365152 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f47c4050070 con 0x55fdc39104d0 2024-04-22T13:28:32.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.849+0000 7f47d87b6740 10 client.17217 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:32.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.849+0000 7f47d87b6740 10 client.17217 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:28:32.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.849+0000 7f47d87b6740 20 client.17217 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:32.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.849+0000 7f47d87b6740 1 --2- 172.21.15.12:0/1237365152 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55fdc3994620 0x55fdc3996a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.849+0000 7f47d87b6740 4 client.17217 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:28:32.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.849+0000 7f47d87b6740 1 -- 172.21.15.12:0/1237365152 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fdc3882c10 con 0x55fdc3994620 2024-04-22T13:28:32.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.849+0000 7f47cf7fe640 1 --2- 172.21.15.12:0/1237365152 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55fdc3994620 0x55fdc3996a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.849+0000 7f47cf7fe640 1 --2- 172.21.15.12:0/1237365152 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55fdc3994620 0x55fdc3996a10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=970b8eef137e0490 server_cookie=72d60bdcb9f342ec in_seq=0 out_seq=0 2024-04-22T13:28:32.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.849+0000 7f47aeffd640 10 client.17217 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:28:32.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.849+0000 7f47aeffd640 1 -- 172.21.15.12:0/1237365152 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fdc3882c10 con 0x55fdc3994620 2024-04-22T13:28:32.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.849+0000 7f47aeffd640 10 client.17217 handle_command_reply: tid=0 2024-04-22T13:28:32.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.850+0000 7f47d87b6740 1 client.17217 shutdown 2024-04-22T13:28:32.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.850+0000 7f47d87b6740 1 -- 172.21.15.12:0/1237365152 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdc39104d0 msgr2=0x55fdc39880d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.850+0000 7f47d87b6740 1 --2- 172.21.15.12:0/1237365152 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdc39104d0 0x55fdc39880d0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f47c402c170 tx=0x7f47c4005870 comp rx=0 tx=0).stop 2024-04-22T13:28:32.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.850+0000 7f47d87b6740 1 -- 172.21.15.12:0/1237365152 shutdown_connections 2024-04-22T13:28:32.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.850+0000 7f47d87b6740 1 --2- 172.21.15.12:0/1237365152 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55fdc3994620 0x55fdc3996a10 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.850+0000 7f47d87b6740 1 --2- 172.21.15.12:0/1237365152 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fdc3919a00 0x55fdc3988b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.850+0000 7f47d87b6740 1 --2- 172.21.15.12:0/1237365152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fdc3910e80 0x55fdc3988610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.850+0000 7f47aeffd640 0 client.17217 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:28:32.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.850+0000 7f47d87b6740 1 --2- 172.21.15.12:0/1237365152 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fdc39104d0 0x55fdc39880d0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.850+0000 7f47d87b6740 1 -- 172.21.15.12:0/1237365152 >> 172.21.15.12:0/1237365152 conn(0x55fdc390aba0 msgr2=0x55fdc3909c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:32.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.850+0000 7f47d87b6740 1 -- 172.21.15.12:0/1237365152 shutdown_connections 2024-04-22T13:28:32.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.850+0000 7f47d87b6740 1 -- 172.21.15.12:0/1237365152 wait complete. 2024-04-22T13:28:32.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.850+0000 7f47d87b6740 20 client.17217 trim_cache size 0 max 16384 2024-04-22T13:28:32.854 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:28:32.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.850+0000 7f47d5af1640 1 -- 172.21.15.12:0/2223747310 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f47b80a3d20 msgr2=0x7f47b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.850+0000 7f47d5af1640 1 --2- 172.21.15.12:0/2223747310 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f47b80a3d20 0x7f47b80a61e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f47c809c5e0 tx=0x7f47c8098c00 comp rx=0 tx=0).stop 2024-04-22T13:28:32.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.850+0000 7f47d5af1640 1 -- 172.21.15.12:0/2223747310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47d019cfb0 msgr2=0x7f47d019d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.850+0000 7f47d5af1640 1 --2- 172.21.15.12:0/2223747310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47d019cfb0 0x7f47d019d460 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f47c800eaa0 tx=0x7f47c800ef60 comp rx=0 tx=0).stop 2024-04-22T13:28:32.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.850+0000 7f47d5af1640 1 -- 172.21.15.12:0/2223747310 shutdown_connections 2024-04-22T13:28:32.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.850+0000 7f47d5af1640 1 --2- 172.21.15.12:0/2223747310 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f47b80a3d20 0x7f47b80a61e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.850+0000 7f47d5af1640 1 --2- 172.21.15.12:0/2223747310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f47d019cfb0 0x7f47d019d460 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.850+0000 7f47d5af1640 1 --2- 172.21.15.12:0/2223747310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47d0107270 0x7f47d0198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.850+0000 7f47d5af1640 1 --2- 172.21.15.12:0/2223747310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47d006a840 0x7f47d01986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.850+0000 7f47d5af1640 1 -- 172.21.15.12:0/2223747310 >> 172.21.15.12:0/2223747310 conn(0x7f47d0077eb0 msgr2=0x7f47d0108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:32.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.851+0000 7f47d5af1640 1 -- 172.21.15.12:0/2223747310 shutdown_connections 2024-04-22T13:28:32.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.851+0000 7f47d5af1640 1 -- 172.21.15.12:0/2223747310 wait complete. 2024-04-22T13:28:32.858 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:28:32.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.965+0000 7fbe8705c640 1 -- 172.21.15.12:0/2481184455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe80107260 msgr2=0x7fbe8010ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.965+0000 7fbe8705c640 1 --2- 172.21.15.12:0/2481184455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe80107260 0x7fbe8010ee30 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fbe78009ca0 tx=0x7fbe7802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.965+0000 7fbe8705c640 1 -- 172.21.15.12:0/2481184455 shutdown_connections 2024-04-22T13:28:32.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.965+0000 7fbe8705c640 1 --2- 172.21.15.12:0/2481184455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe80107260 0x7fbe8010ee30 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.965+0000 7fbe8705c640 1 --2- 172.21.15.12:0/2481184455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe80102b40 0x7fbe80106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.965+0000 7fbe8705c640 1 --2- 172.21.15.12:0/2481184455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe80102190 0x7fbe80102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.966+0000 7fbe8705c640 1 -- 172.21.15.12:0/2481184455 >> 172.21.15.12:0/2481184455 conn(0x7fbe800793a0 msgr2=0x7fbe800797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:32.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.966+0000 7fbe8705c640 1 -- 172.21.15.12:0/2481184455 shutdown_connections 2024-04-22T13:28:32.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.966+0000 7fbe8705c640 1 -- 172.21.15.12:0/2481184455 wait complete. 2024-04-22T13:28:32.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.966+0000 7fbe8705c640 1 Processor -- start 2024-04-22T13:28:32.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.966+0000 7fbe8705c640 1 -- start start 2024-04-22T13:28:32.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.966+0000 7fbe8705c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe80102190 0x7fbe80074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.966+0000 7fbe8705c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe80107260 0x7fbe80074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.966+0000 7fbe8705c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe80074cd0 0x7fbe801a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.966+0000 7fbe8705c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbe801121d0 con 0x7fbe80074cd0 2024-04-22T13:28:32.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.966+0000 7fbe8705c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbe80112050 con 0x7fbe80107260 2024-04-22T13:28:32.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.966+0000 7fbe8705c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbe80112350 con 0x7fbe80102190 2024-04-22T13:28:32.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.966+0000 7fbe855f9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe80074cd0 0x7fbe801a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.966+0000 7fbe855f9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe80074cd0 0x7fbe801a68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60970/0 (socket says 172.21.15.12:60970) 2024-04-22T13:28:32.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.966+0000 7fbe855f9640 1 -- 172.21.15.12:0/3174666010 learned_addr learned my addr 172.21.15.12:0/3174666010 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:32.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.967+0000 7fbe84df8640 1 --2- 172.21.15.12:0/3174666010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe80102190 0x7fbe80074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.967+0000 7fbe855f9640 1 -- 172.21.15.12:0/3174666010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe80102190 msgr2=0x7fbe80074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.967+0000 7fbe7ffff640 1 --2- 172.21.15.12:0/3174666010 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe80107260 0x7fbe80074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.967+0000 7fbe855f9640 1 --2- 172.21.15.12:0/3174666010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe80102190 0x7fbe80074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.967+0000 7fbe855f9640 1 -- 172.21.15.12:0/3174666010 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe80107260 msgr2=0x7fbe80074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.967+0000 7fbe855f9640 1 --2- 172.21.15.12:0/3174666010 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe80107260 0x7fbe80074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.967+0000 7fbe855f9640 1 -- 172.21.15.12:0/3174666010 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe801a6fe0 con 0x7fbe80074cd0 2024-04-22T13:28:32.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.967+0000 7fbe7ffff640 1 --2- 172.21.15.12:0/3174666010 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe80107260 0x7fbe80074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:32.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.967+0000 7fbe855f9640 1 --2- 172.21.15.12:0/3174666010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe80074cd0 0x7fbe801a68e0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fbe78005e30 tx=0x7fbe78032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:32.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.967+0000 7fbe7dffb640 1 -- 172.21.15.12:0/3174666010 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe78041020 con 0x7fbe80074cd0 2024-04-22T13:28:32.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.967+0000 7fbe7dffb640 1 -- 172.21.15.12:0/3174666010 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe78032970 con 0x7fbe80074cd0 2024-04-22T13:28:32.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.967+0000 7fbe8705c640 1 -- 172.21.15.12:0/3174666010 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe801a7270 con 0x7fbe80074cd0 2024-04-22T13:28:32.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.967+0000 7fbe7dffb640 1 -- 172.21.15.12:0/3174666010 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe78039040 con 0x7fbe80074cd0 2024-04-22T13:28:32.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.967+0000 7fbe8705c640 1 -- 172.21.15.12:0/3174666010 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe801a77b0 con 0x7fbe80074cd0 2024-04-22T13:28:32.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.969+0000 7fbe7dffb640 1 -- 172.21.15.12:0/3174666010 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbe78044020 con 0x7fbe80074cd0 2024-04-22T13:28:32.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.970+0000 7fbe89c4d740 1 Processor -- start 2024-04-22T13:28:32.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.970+0000 7fbe89c4d740 1 -- start start 2024-04-22T13:28:32.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.970+0000 7fbe89c4d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571ad8754d0 0x5571ad8758b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.970+0000 7fbe89c4d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571ad875e80 0x5571ad87e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.970+0000 7fbe89c4d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571ad87ea00 0x5571ad880e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.970+0000 7fbe89c4d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5571ad7da1f0 con 0x5571ad87ea00 2024-04-22T13:28:32.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.970+0000 7fbe89c4d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5571ad7774b0 con 0x5571ad875e80 2024-04-22T13:28:32.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.970+0000 7fbe89c4d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5571ad7d61b0 con 0x5571ad8754d0 2024-04-22T13:28:32.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.970+0000 7fbe7dffb640 1 --2- 172.21.15.12:0/3174666010 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbe680a3dd0 0x7fbe680a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.970+0000 7fbe7ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571ad87ea00 0x5571ad880e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.970+0000 7fbe7ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571ad87ea00 0x5571ad880e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60986/0 (socket says 172.21.15.12:60986) 2024-04-22T13:28:32.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.970+0000 7fbe855f9640 1 --2- 172.21.15.12:0/3174666010 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbe680a3dd0 0x7fbe680a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.970+0000 7fbe7ffff640 1 -- 172.21.15.12:0/402099544 learned_addr learned my addr 172.21.15.12:0/402099544 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:32.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.970+0000 7fbe84df8640 1 --2- 172.21.15.12:0/402099544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571ad875e80 0x5571ad87e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.970+0000 7fbe7dffb640 1 -- 172.21.15.12:0/3174666010 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fbe780f24a0 con 0x7fbe80074cd0 2024-04-22T13:28:32.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.970+0000 7fbe7ffff640 1 --2- 172.21.15.12:0/402099544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571ad8754d0 0x5571ad8758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.971+0000 7fbe7ffff640 1 -- 172.21.15.12:0/402099544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571ad8754d0 msgr2=0x5571ad8758b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:32.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.971+0000 7fbe7ffff640 1 --2- 172.21.15.12:0/402099544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571ad8754d0 0x5571ad8758b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.971+0000 7fbe7ffff640 1 -- 172.21.15.12:0/402099544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571ad875e80 msgr2=0x5571ad87e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:32.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.971+0000 7fbe7ffff640 1 --2- 172.21.15.12:0/402099544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571ad875e80 0x5571ad87e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.971+0000 7fbe7ffff640 1 -- 172.21.15.12:0/402099544 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571ad7691b0 con 0x5571ad87ea00 2024-04-22T13:28:32.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.971+0000 7fbe855f9640 1 --2- 172.21.15.12:0/3174666010 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbe680a3dd0 0x7fbe680a6290 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbe780efe00 tx=0x7fbe7802e040 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:32.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.971+0000 7fbe7ffff640 1 --2- 172.21.15.12:0/402099544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571ad87ea00 0x5571ad880e50 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fbe74009f70 tx=0x7fbe74009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7192821acd27d5e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:32.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.971+0000 7fbe66ffd640 1 -- 172.21.15.12:0/402099544 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe740150f0 con 0x5571ad87ea00 2024-04-22T13:28:32.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.971+0000 7fbe66ffd640 1 -- 172.21.15.12:0/402099544 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe740035f0 con 0x5571ad87ea00 2024-04-22T13:28:32.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.971+0000 7fbe66ffd640 1 -- 172.21.15.12:0/402099544 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe74013720 con 0x5571ad87ea00 2024-04-22T13:28:32.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.972+0000 7fbe89c4d740 1 -- 172.21.15.12:0/402099544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571ad87ea00 msgr2=0x5571ad880e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.972+0000 7fbe89c4d740 1 --2- 172.21.15.12:0/402099544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571ad87ea00 0x5571ad880e50 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fbe74009f70 tx=0x7fbe74009590 comp rx=0 tx=0).stop 2024-04-22T13:28:32.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.972+0000 7fbe89c4d740 1 -- 172.21.15.12:0/402099544 shutdown_connections 2024-04-22T13:28:32.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.972+0000 7fbe89c4d740 1 --2- 172.21.15.12:0/402099544 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571ad87ea00 0x5571ad880e50 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.972+0000 7fbe89c4d740 1 --2- 172.21.15.12:0/402099544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571ad875e80 0x5571ad87e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.972+0000 7fbe89c4d740 1 --2- 172.21.15.12:0/402099544 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571ad8754d0 0x5571ad8758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.972+0000 7fbe89c4d740 1 -- 172.21.15.12:0/402099544 >> 172.21.15.12:0/402099544 conn(0x5571ad86fba0 msgr2=0x5571ad86ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:32.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.972+0000 7fbe89c4d740 1 -- 172.21.15.12:0/402099544 shutdown_connections 2024-04-22T13:28:32.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.972+0000 7fbe89c4d740 1 -- 172.21.15.12:0/402099544 wait complete. 2024-04-22T13:28:32.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.972+0000 7fbe89c4d740 1 -- start start 2024-04-22T13:28:32.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.972+0000 7fbe89c4d740 1 Processor -- start 2024-04-22T13:28:32.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.972+0000 7fbe89c4d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571ad8754d0 0x5571ad87e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.972+0000 7fbe89c4d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571ad875e80 0x5571ad87b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.973+0000 7fbe89c4d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571ad87ea00 0x5571ad87bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.973+0000 7fbe89c4d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5571ad881970 con 0x5571ad875e80 2024-04-22T13:28:32.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.973+0000 7fbe89c4d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5571ad793500 con 0x5571ad8754d0 2024-04-22T13:28:32.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.973+0000 7fbe89c4d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5571ad881af0 con 0x5571ad87ea00 2024-04-22T13:28:32.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.973+0000 7fbe89c4d740 10 client.0 _ll_register_callbacks cb 0x5571ad761b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:32.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.973+0000 7fbe84df8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571ad875e80 0x5571ad87b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.973+0000 7fbe84df8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571ad875e80 0x5571ad87b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60990/0 (socket says 172.21.15.12:60990) 2024-04-22T13:28:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.973+0000 7fbe84df8640 1 -- 172.21.15.12:0/2024095083 learned_addr learned my addr 172.21.15.12:0/2024095083 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.973+0000 7fbe7ffff640 1 --2- 172.21.15.12:0/2024095083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571ad87ea00 0x5571ad87bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.973+0000 7fbe7ffff640 1 --2- 172.21.15.12:0/2024095083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571ad8754d0 0x5571ad87e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.973+0000 7fbe84df8640 1 -- 172.21.15.12:0/2024095083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571ad87ea00 msgr2=0x5571ad87bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.973+0000 7fbe84df8640 1 --2- 172.21.15.12:0/2024095083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571ad87ea00 0x5571ad87bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.973+0000 7fbe84df8640 1 -- 172.21.15.12:0/2024095083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571ad8754d0 msgr2=0x5571ad87e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.973+0000 7fbe84df8640 1 --2- 172.21.15.12:0/2024095083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571ad8754d0 0x5571ad87e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.973+0000 7fbe84df8640 1 -- 172.21.15.12:0/2024095083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571ad4ec3f0 con 0x5571ad875e80 2024-04-22T13:28:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.973+0000 7fbe7ffff640 1 --2- 172.21.15.12:0/2024095083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571ad8754d0 0x5571ad87e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.973+0000 7fbe84df8640 1 --2- 172.21.15.12:0/2024095083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571ad875e80 0x5571ad87b6d0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fbe70012700 tx=0x7fbe70012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af9b1a17a132529c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.973+0000 7fbe5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.973+0000 7fbe5ffff640 1 -- 172.21.15.12:0/2024095083 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe70004260 con 0x5571ad875e80 2024-04-22T13:28:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.973+0000 7fbe5ffff640 1 -- 172.21.15.12:0/2024095083 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fbe70004400 con 0x5571ad875e80 2024-04-22T13:28:32.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.973+0000 7fbe89c4d740 1 -- 172.21.15.12:0/2024095083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe740035f0 con 0x5571ad875e80 2024-04-22T13:28:32.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.974+0000 7fbe5ffff640 1 -- 172.21.15.12:0/2024095083 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbe700119a0 con 0x5571ad875e80 2024-04-22T13:28:32.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.974+0000 7fbe89c4d740 10 client.17229 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:32.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.974+0000 7fbe89c4d740 1 -- 172.21.15.12:0/2024095083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5571ad5b67d0 con 0x5571ad875e80 2024-04-22T13:28:32.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.974+0000 7fbe5ffff640 1 -- 172.21.15.12:0/2024095083 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe740035f0 con 0x5571ad875e80 2024-04-22T13:28:32.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.974+0000 7fbe5ffff640 1 -- 172.21.15.12:0/2024095083 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbe7000f440 con 0x5571ad875e80 2024-04-22T13:28:32.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.974+0000 7fbe89c4d740 10 client.17229 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:32.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.974+0000 7fbe89c4d740 10 client.17229 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:28:32.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.974+0000 7fbe89c4d740 20 client.17229 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:32.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.974+0000 7fbe89c4d740 1 --2- 172.21.15.12:0/2024095083 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5571ad88ebe0 0x5571ad891010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.974+0000 7fbe89c4d740 4 client.17229 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:32.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.974+0000 7fbe89c4d740 1 -- 172.21.15.12:0/2024095083 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571ad779930 con 0x5571ad88ebe0 2024-04-22T13:28:32.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.974+0000 7fbe7ffff640 1 --2- 172.21.15.12:0/2024095083 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5571ad88ebe0 0x5571ad891010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.975+0000 7fbe5ffff640 10 client.17229 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:28:32.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.975+0000 7fbe7ffff640 1 --2- 172.21.15.12:0/2024095083 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5571ad88ebe0 0x5571ad891010 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8dccb9323ff47b72 server_cookie=c760e7a2388e37d4 in_seq=0 out_seq=0 2024-04-22T13:28:32.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.975+0000 7fbe5ffff640 1 -- 172.21.15.12:0/2024095083 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5571ad779930 con 0x5571ad88ebe0 2024-04-22T13:28:32.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.975+0000 7fbe5ffff640 10 client.17229 handle_command_reply: tid=0 2024-04-22T13:28:32.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.975+0000 7fbe89c4d740 1 client.17229 shutdown 2024-04-22T13:28:32.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.975+0000 7fbe89c4d740 1 -- 172.21.15.12:0/2024095083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571ad875e80 msgr2=0x5571ad87b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.975+0000 7fbe89c4d740 1 --2- 172.21.15.12:0/2024095083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571ad875e80 0x5571ad87b6d0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fbe70012700 tx=0x7fbe70012bc0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.975+0000 7fbe89c4d740 1 -- 172.21.15.12:0/2024095083 shutdown_connections 2024-04-22T13:28:32.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.975+0000 7fbe89c4d740 1 --2- 172.21.15.12:0/2024095083 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5571ad88ebe0 0x5571ad891010 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.976+0000 7fbe89c4d740 1 --2- 172.21.15.12:0/2024095083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571ad87ea00 0x5571ad87bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.976+0000 7fbe89c4d740 1 --2- 172.21.15.12:0/2024095083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571ad875e80 0x5571ad87b6d0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.976+0000 7fbe89c4d740 1 --2- 172.21.15.12:0/2024095083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571ad8754d0 0x5571ad87e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.976+0000 7fbe5ffff640 0 client.17229 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:28:32.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.976+0000 7fbe89c4d740 1 -- 172.21.15.12:0/2024095083 >> 172.21.15.12:0/2024095083 conn(0x5571ad86fba0 msgr2=0x5571ad8718f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:32.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.976+0000 7fbe89c4d740 1 -- 172.21.15.12:0/2024095083 shutdown_connections 2024-04-22T13:28:32.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.976+0000 7fbe89c4d740 1 -- 172.21.15.12:0/2024095083 wait complete. 2024-04-22T13:28:32.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.976+0000 7fbe89c4d740 20 client.17229 trim_cache size 0 max 16384 2024-04-22T13:28:32.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.989+0000 7fbe89c4d740 1 Processor -- start 2024-04-22T13:28:32.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.989+0000 7fbe89c4d740 1 -- start start 2024-04-22T13:28:32.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.989+0000 7fbe89c4d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571ad8754d0 0x5571ad88a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.989+0000 7fbe89c4d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571ad875e80 0x5571ad871600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.989+0000 7fbe89c4d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571ad87ea00 0x5571ad87de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.989+0000 7fbe89c4d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbe801afc50 con 0x5571ad875e80 2024-04-22T13:28:32.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.989+0000 7fbe89c4d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbe801b1790 con 0x5571ad87ea00 2024-04-22T13:28:32.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.989+0000 7fbe89c4d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbe801b0320 con 0x5571ad8754d0 2024-04-22T13:28:32.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.989+0000 7fbe84df8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571ad875e80 0x5571ad871600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.989+0000 7fbe84df8640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571ad875e80 0x5571ad871600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60996/0 (socket says 172.21.15.12:60996) 2024-04-22T13:28:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.989+0000 7fbe84df8640 1 -- 172.21.15.12:0/3838722168 learned_addr learned my addr 172.21.15.12:0/3838722168 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.989+0000 7fbe7ffff640 1 --2- 172.21.15.12:0/3838722168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571ad8754d0 0x5571ad88a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.990+0000 7fbe7ffff640 1 --2- 172.21.15.12:0/3838722168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571ad87ea00 0x5571ad87de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.990+0000 7fbe84df8640 1 -- 172.21.15.12:0/3838722168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571ad8754d0 msgr2=0x5571ad88a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.990+0000 7fbe84df8640 1 --2- 172.21.15.12:0/3838722168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571ad8754d0 0x5571ad88a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.990+0000 7fbe84df8640 1 -- 172.21.15.12:0/3838722168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571ad87ea00 msgr2=0x5571ad87de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.990+0000 7fbe84df8640 1 --2- 172.21.15.12:0/3838722168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571ad87ea00 0x5571ad87de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.990+0000 7fbe84df8640 1 -- 172.21.15.12:0/3838722168 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571ad87c050 con 0x5571ad875e80 2024-04-22T13:28:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.990+0000 7fbe7ffff640 1 --2- 172.21.15.12:0/3838722168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571ad87ea00 0x5571ad87de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.990+0000 7fbe84df8640 1 --2- 172.21.15.12:0/3838722168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571ad875e80 0x5571ad871600 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fbe700058c0 tx=0x7fbe700096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be9ecd80dd133b02 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.990+0000 7fbe667fc640 1 -- 172.21.15.12:0/3838722168 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7000f5c0 con 0x5571ad875e80 2024-04-22T13:28:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.990+0000 7fbe667fc640 1 -- 172.21.15.12:0/3838722168 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe7000fa00 con 0x5571ad875e80 2024-04-22T13:28:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.990+0000 7fbe667fc640 1 -- 172.21.15.12:0/3838722168 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe700037f0 con 0x5571ad875e80 2024-04-22T13:28:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.990+0000 7fbe89c4d740 1 -- 172.21.15.12:0/3838722168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571ad875e80 msgr2=0x5571ad871600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.990+0000 7fbe89c4d740 1 --2- 172.21.15.12:0/3838722168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571ad875e80 0x5571ad871600 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fbe700058c0 tx=0x7fbe700096c0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.991+0000 7fbe89c4d740 1 -- 172.21.15.12:0/3838722168 shutdown_connections 2024-04-22T13:28:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.991+0000 7fbe89c4d740 1 --2- 172.21.15.12:0/3838722168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571ad87ea00 0x5571ad87de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.991+0000 7fbe89c4d740 1 --2- 172.21.15.12:0/3838722168 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571ad875e80 0x5571ad871600 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.991+0000 7fbe89c4d740 1 --2- 172.21.15.12:0/3838722168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571ad8754d0 0x5571ad88a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.991+0000 7fbe89c4d740 1 -- 172.21.15.12:0/3838722168 >> 172.21.15.12:0/3838722168 conn(0x5571ad86fba0 msgr2=0x5571ad870ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.991+0000 7fbe89c4d740 1 -- 172.21.15.12:0/3838722168 shutdown_connections 2024-04-22T13:28:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.991+0000 7fbe89c4d740 1 -- 172.21.15.12:0/3838722168 wait complete. 2024-04-22T13:28:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.991+0000 7fbe89c4d740 1 -- start start 2024-04-22T13:28:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.991+0000 7fbe89c4d740 1 Processor -- start 2024-04-22T13:28:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.991+0000 7fbe89c4d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571ad8754d0 0x5571ad887030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.991+0000 7fbe89c4d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571ad875e80 0x5571ad8823a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.991+0000 7fbe89c4d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571ad87ea00 0x5571ad8828e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.991+0000 7fbe89c4d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbe801b10c0 con 0x5571ad87ea00 2024-04-22T13:28:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.991+0000 7fbe89c4d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbe801b09f0 con 0x5571ad875e80 2024-04-22T13:28:32.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.991+0000 7fbe89c4d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbe801adc30 con 0x5571ad8754d0 2024-04-22T13:28:32.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.991+0000 7fbe89c4d740 10 client.0 _ll_register_callbacks cb 0x7fbe70009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:32.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.992+0000 7fbe7ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571ad87ea00 0x5571ad8828e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.992+0000 7fbe7ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571ad87ea00 0x5571ad8828e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32768/0 (socket says 172.21.15.12:32768) 2024-04-22T13:28:32.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.992+0000 7fbe7ffff640 1 -- 172.21.15.12:0/3536270737 learned_addr learned my addr 172.21.15.12:0/3536270737 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:32.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.992+0000 7fbe7ffff640 1 --2- 172.21.15.12:0/3536270737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571ad8754d0 0x5571ad887030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.992+0000 7fbe84df8640 1 --2- 172.21.15.12:0/3536270737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571ad875e80 0x5571ad8823a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.992+0000 7fbe7ffff640 1 -- 172.21.15.12:0/3536270737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571ad8754d0 msgr2=0x5571ad887030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:32.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.992+0000 7fbe7ffff640 1 --2- 172.21.15.12:0/3536270737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571ad8754d0 0x5571ad887030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.992+0000 7fbe7ffff640 1 -- 172.21.15.12:0/3536270737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571ad875e80 msgr2=0x5571ad8823a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:32.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.992+0000 7fbe7ffff640 1 --2- 172.21.15.12:0/3536270737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571ad875e80 0x5571ad8823a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.992+0000 7fbe7ffff640 1 -- 172.21.15.12:0/3536270737 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571ad883020 con 0x5571ad87ea00 2024-04-22T13:28:32.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.992+0000 7fbe7ffff640 1 --2- 172.21.15.12:0/3536270737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571ad87ea00 0x5571ad8828e0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fbe74009e20 tx=0x7fbe7402aae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b0beed5ece28f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:32.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.992+0000 7fbe5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:32.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.992+0000 7fbe5ffff640 1 -- 172.21.15.12:0/3536270737 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7402ab50 con 0x5571ad87ea00 2024-04-22T13:28:32.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.992+0000 7fbe5ffff640 1 -- 172.21.15.12:0/3536270737 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fbe74004000 con 0x5571ad87ea00 2024-04-22T13:28:32.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.992+0000 7fbe89c4d740 1 -- 172.21.15.12:0/3536270737 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe7000fa00 con 0x5571ad87ea00 2024-04-22T13:28:32.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.992+0000 7fbe5ffff640 1 -- 172.21.15.12:0/3536270737 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbe74004220 con 0x5571ad87ea00 2024-04-22T13:28:32.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.992+0000 7fbe89c4d740 10 client.17235 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:32.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.992+0000 7fbe89c4d740 1 -- 172.21.15.12:0/3536270737 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5571ad8f4ff0 con 0x5571ad87ea00 2024-04-22T13:28:32.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.993+0000 7fbe5ffff640 1 -- 172.21.15.12:0/3536270737 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe7000fa00 con 0x5571ad87ea00 2024-04-22T13:28:32.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.993+0000 7fbe5ffff640 1 -- 172.21.15.12:0/3536270737 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbe7400c070 con 0x5571ad87ea00 2024-04-22T13:28:32.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.993+0000 7fbe89c4d740 10 client.17235 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:32.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.993+0000 7fbe89c4d740 10 client.17235 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:28:32.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.993+0000 7fbe89c4d740 20 client.17235 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:32.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.993+0000 7fbe89c4d740 1 --2- 172.21.15.12:0/3536270737 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5571ad8f5530 0x5571ad8f7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:32.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.993+0000 7fbe89c4d740 4 client.17235 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:28:32.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.993+0000 7fbe89c4d740 1 -- 172.21.15.12:0/3536270737 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5571ad7e7c10 con 0x5571ad8f5530 2024-04-22T13:28:32.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.993+0000 7fbe84df8640 1 --2- 172.21.15.12:0/3536270737 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5571ad8f5530 0x5571ad8f7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:32.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.993+0000 7fbe84df8640 1 --2- 172.21.15.12:0/3536270737 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5571ad8f5530 0x5571ad8f7920 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a0cd344efd183df8 server_cookie=6bd05811c74c2064 in_seq=0 out_seq=0 2024-04-22T13:28:32.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.993+0000 7fbe5ffff640 10 client.17235 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:28:32.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.993+0000 7fbe5ffff640 1 -- 172.21.15.12:0/3536270737 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5571ad7e7c10 con 0x5571ad8f5530 2024-04-22T13:28:32.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.993+0000 7fbe5ffff640 10 client.17235 handle_command_reply: tid=0 2024-04-22T13:28:32.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.994+0000 7fbe89c4d740 1 client.17235 shutdown 2024-04-22T13:28:32.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.994+0000 7fbe89c4d740 1 -- 172.21.15.12:0/3536270737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571ad87ea00 msgr2=0x5571ad8828e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.994+0000 7fbe89c4d740 1 --2- 172.21.15.12:0/3536270737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571ad87ea00 0x5571ad8828e0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fbe74009e20 tx=0x7fbe7402aae0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.994+0000 7fbe89c4d740 1 -- 172.21.15.12:0/3536270737 shutdown_connections 2024-04-22T13:28:32.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.994+0000 7fbe89c4d740 1 --2- 172.21.15.12:0/3536270737 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5571ad8f5530 0x5571ad8f7920 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.994+0000 7fbe89c4d740 1 --2- 172.21.15.12:0/3536270737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5571ad87ea00 0x5571ad8828e0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.994+0000 7fbe5ffff640 0 client.17235 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:28:32.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.994+0000 7fbe89c4d740 1 --2- 172.21.15.12:0/3536270737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571ad875e80 0x5571ad8823a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.994+0000 7fbe89c4d740 1 --2- 172.21.15.12:0/3536270737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5571ad8754d0 0x5571ad887030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.994+0000 7fbe89c4d740 1 -- 172.21.15.12:0/3536270737 >> 172.21.15.12:0/3536270737 conn(0x5571ad86fba0 msgr2=0x5571ad871780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:32.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.994+0000 7fbe89c4d740 1 -- 172.21.15.12:0/3536270737 shutdown_connections 2024-04-22T13:28:32.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.994+0000 7fbe89c4d740 1 -- 172.21.15.12:0/3536270737 wait complete. 2024-04-22T13:28:32.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.994+0000 7fbe89c4d740 20 client.17235 trim_cache size 0 max 16384 2024-04-22T13:28:32.997 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:28:32.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.994+0000 7fbe8705c640 1 -- 172.21.15.12:0/3174666010 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbe680a3dd0 msgr2=0x7fbe680a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.994+0000 7fbe8705c640 1 --2- 172.21.15.12:0/3174666010 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbe680a3dd0 0x7fbe680a6290 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbe780efe00 tx=0x7fbe7802e040 comp rx=0 tx=0).stop 2024-04-22T13:28:32.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.994+0000 7fbe8705c640 1 -- 172.21.15.12:0/3174666010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe80074cd0 msgr2=0x7fbe801a68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:32.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.994+0000 7fbe8705c640 1 --2- 172.21.15.12:0/3174666010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe80074cd0 0x7fbe801a68e0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fbe78005e30 tx=0x7fbe78032420 comp rx=0 tx=0).stop 2024-04-22T13:28:32.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.994+0000 7fbe8705c640 1 -- 172.21.15.12:0/3174666010 shutdown_connections 2024-04-22T13:28:32.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.994+0000 7fbe8705c640 1 --2- 172.21.15.12:0/3174666010 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbe680a3dd0 0x7fbe680a6290 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.994+0000 7fbe8705c640 1 --2- 172.21.15.12:0/3174666010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbe80074cd0 0x7fbe801a68e0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.994+0000 7fbe8705c640 1 --2- 172.21.15.12:0/3174666010 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe80107260 0x7fbe80074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.994+0000 7fbe8705c640 1 --2- 172.21.15.12:0/3174666010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe80102190 0x7fbe80074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:32.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.994+0000 7fbe8705c640 1 -- 172.21.15.12:0/3174666010 >> 172.21.15.12:0/3174666010 conn(0x7fbe800793a0 msgr2=0x7fbe800ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:32.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.995+0000 7fbe8705c640 1 -- 172.21.15.12:0/3174666010 shutdown_connections 2024-04-22T13:28:32.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:32.995+0000 7fbe8705c640 1 -- 172.21.15.12:0/3174666010 wait complete. 2024-04-22T13:28:33.002 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:28:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.109+0000 7efd6a79f640 1 -- 172.21.15.12:0/245972405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd6410c510 msgr2=0x7efd6410e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.109+0000 7efd6a79f640 1 --2- 172.21.15.12:0/245972405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd6410c510 0x7efd6410e9b0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009ca0 tx=0x7efd5c02c230 comp rx=0 tx=0).stop 2024-04-22T13:28:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.109+0000 7efd6a79f640 1 -- 172.21.15.12:0/245972405 shutdown_connections 2024-04-22T13:28:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.109+0000 7efd6a79f640 1 --2- 172.21.15.12:0/245972405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd6410c510 0x7efd6410e9b0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.109+0000 7efd6a79f640 1 --2- 172.21.15.12:0/245972405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd64103af0 0x7efd6410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.109+0000 7efd6a79f640 1 --2- 172.21.15.12:0/245972405 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd641031d0 0x7efd641035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.109+0000 7efd6a79f640 1 -- 172.21.15.12:0/245972405 >> 172.21.15.12:0/245972405 conn(0x7efd64077f50 msgr2=0x7efd64078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.109+0000 7efd6a79f640 1 -- 172.21.15.12:0/245972405 shutdown_connections 2024-04-22T13:28:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.109+0000 7efd6a79f640 1 -- 172.21.15.12:0/245972405 wait complete. 2024-04-22T13:28:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.109+0000 7efd6a79f640 1 Processor -- start 2024-04-22T13:28:33.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.109+0000 7efd6a79f640 1 -- start start 2024-04-22T13:28:33.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.110+0000 7efd6a79f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd641031d0 0x7efd64198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.110+0000 7efd6a79f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd64103af0 0x7efd64198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.110+0000 7efd6a79f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd6410c510 0x7efd6419cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.110+0000 7efd6a79f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efd641121a0 con 0x7efd64103af0 2024-04-22T13:28:33.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.110+0000 7efd6a79f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efd64112020 con 0x7efd641031d0 2024-04-22T13:28:33.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.110+0000 7efd6a79f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efd64112320 con 0x7efd6410c510 2024-04-22T13:28:33.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.110+0000 7efd637fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd64103af0 0x7efd64198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.110+0000 7efd637fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd64103af0 0x7efd64198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32776/0 (socket says 172.21.15.12:32776) 2024-04-22T13:28:33.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.110+0000 7efd637fe640 1 -- 172.21.15.12:0/3860059494 learned_addr learned my addr 172.21.15.12:0/3860059494 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:33.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.110+0000 7efd68d3c640 1 --2- 172.21.15.12:0/3860059494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd6410c510 0x7efd6419cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.110+0000 7efd63fff640 1 --2- 172.21.15.12:0/3860059494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd641031d0 0x7efd64198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.110+0000 7efd637fe640 1 -- 172.21.15.12:0/3860059494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd6410c510 msgr2=0x7efd6419cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:33.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.110+0000 7efd637fe640 1 --2- 172.21.15.12:0/3860059494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd6410c510 0x7efd6419cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.110+0000 7efd637fe640 1 -- 172.21.15.12:0/3860059494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd641031d0 msgr2=0x7efd64198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:33.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.110+0000 7efd637fe640 1 --2- 172.21.15.12:0/3860059494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd641031d0 0x7efd64198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.110+0000 7efd637fe640 1 -- 172.21.15.12:0/3860059494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd6419d6f0 con 0x7efd64103af0 2024-04-22T13:28:33.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.110+0000 7efd637fe640 1 --2- 172.21.15.12:0/3860059494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd64103af0 0x7efd64198c40 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7efd5800e9f0 tx=0x7efd5800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:33.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.110+0000 7efd617fa640 1 -- 172.21.15.12:0/3860059494 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5800cd80 con 0x7efd64103af0 2024-04-22T13:28:33.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.111+0000 7efd617fa640 1 -- 172.21.15.12:0/3860059494 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd580108e0 con 0x7efd64103af0 2024-04-22T13:28:33.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.111+0000 7efd6a79f640 1 -- 172.21.15.12:0/3860059494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd6419d9e0 con 0x7efd64103af0 2024-04-22T13:28:33.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.111+0000 7efd617fa640 1 -- 172.21.15.12:0/3860059494 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd58018980 con 0x7efd64103af0 2024-04-22T13:28:33.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.111+0000 7efd6a79f640 1 -- 172.21.15.12:0/3860059494 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd641a5270 con 0x7efd64103af0 2024-04-22T13:28:33.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.113+0000 7efd617fa640 1 -- 172.21.15.12:0/3860059494 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efd58010430 con 0x7efd64103af0 2024-04-22T13:28:33.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.113+0000 7efd6d38a740 1 Processor -- start 2024-04-22T13:28:33.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.113+0000 7efd6d38a740 1 -- start start 2024-04-22T13:28:33.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.113+0000 7efd6d38a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565111a774d0 0x565111a778b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.113+0000 7efd6d38a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565111a77e80 0x565111a80430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.113+0000 7efd6d38a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565111a80a00 0x565111a82e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.113+0000 7efd6d38a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5651119dc1f0 con 0x565111a80a00 2024-04-22T13:28:33.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.113+0000 7efd6d38a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5651119794b0 con 0x565111a77e80 2024-04-22T13:28:33.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.113+0000 7efd6d38a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5651119d81b0 con 0x565111a774d0 2024-04-22T13:28:33.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.113+0000 7efd637fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565111a80a00 0x565111a82e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.113+0000 7efd617fa640 1 --2- 172.21.15.12:0/3860059494 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7efd4c0a3d20 0x7efd4c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.113+0000 7efd637fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565111a80a00 0x565111a82e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32784/0 (socket says 172.21.15.12:32784) 2024-04-22T13:28:33.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.113+0000 7efd637fe640 1 -- 172.21.15.12:0/1778480801 learned_addr learned my addr 172.21.15.12:0/1778480801 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:33.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.113+0000 7efd68d3c640 1 --2- 172.21.15.12:0/1778480801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565111a774d0 0x565111a778b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.114+0000 7efd63fff640 1 --2- 172.21.15.12:0/1778480801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565111a77e80 0x565111a80430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.114+0000 7efd68d3c640 1 --2- 172.21.15.12:0/3860059494 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7efd4c0a3d20 0x7efd4c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.114+0000 7efd617fa640 1 -- 172.21.15.12:0/3860059494 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7efd580d0ac0 con 0x7efd64103af0 2024-04-22T13:28:33.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.114+0000 7efd637fe640 1 -- 172.21.15.12:0/1778480801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565111a774d0 msgr2=0x565111a778b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:33.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.114+0000 7efd637fe640 1 --2- 172.21.15.12:0/1778480801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565111a774d0 0x565111a778b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.114+0000 7efd637fe640 1 -- 172.21.15.12:0/1778480801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565111a77e80 msgr2=0x565111a80430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:33.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.114+0000 7efd637fe640 1 --2- 172.21.15.12:0/1778480801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565111a77e80 0x565111a80430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.114+0000 7efd637fe640 1 -- 172.21.15.12:0/1778480801 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56511196b1b0 con 0x565111a80a00 2024-04-22T13:28:33.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.114+0000 7efd637fe640 1 --2- 172.21.15.12:0/1778480801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565111a80a00 0x565111a82e50 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7efd580a09a0 tx=0x7efd580875e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a024f09f31d19c6b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:33.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.114+0000 7efd68d3c640 1 --2- 172.21.15.12:0/3860059494 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7efd4c0a3d20 0x7efd4c0a61e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efd5c004af0 tx=0x7efd5c005c50 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:33.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.114+0000 7efd4a7fc640 1 -- 172.21.15.12:0/1778480801 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd580a5720 con 0x565111a80a00 2024-04-22T13:28:33.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.114+0000 7efd4a7fc640 1 -- 172.21.15.12:0/1778480801 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd5809cab0 con 0x565111a80a00 2024-04-22T13:28:33.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.114+0000 7efd4a7fc640 1 -- 172.21.15.12:0/1778480801 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd58099120 con 0x565111a80a00 2024-04-22T13:28:33.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.115+0000 7efd6d38a740 1 -- 172.21.15.12:0/1778480801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565111a80a00 msgr2=0x565111a82e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:33.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.115+0000 7efd6d38a740 1 --2- 172.21.15.12:0/1778480801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565111a80a00 0x565111a82e50 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7efd580a09a0 tx=0x7efd580875e0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.115+0000 7efd6d38a740 1 -- 172.21.15.12:0/1778480801 shutdown_connections 2024-04-22T13:28:33.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.115+0000 7efd6d38a740 1 --2- 172.21.15.12:0/1778480801 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565111a80a00 0x565111a82e50 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.115+0000 7efd6d38a740 1 --2- 172.21.15.12:0/1778480801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565111a77e80 0x565111a80430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.115+0000 7efd6d38a740 1 --2- 172.21.15.12:0/1778480801 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565111a774d0 0x565111a778b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.115+0000 7efd6d38a740 1 -- 172.21.15.12:0/1778480801 >> 172.21.15.12:0/1778480801 conn(0x565111a71ba0 msgr2=0x565111a71f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:33.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.115+0000 7efd6d38a740 1 -- 172.21.15.12:0/1778480801 shutdown_connections 2024-04-22T13:28:33.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.115+0000 7efd6d38a740 1 -- 172.21.15.12:0/1778480801 wait complete. 2024-04-22T13:28:33.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.115+0000 7efd6d38a740 1 -- start start 2024-04-22T13:28:33.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.116+0000 7efd6d38a740 1 Processor -- start 2024-04-22T13:28:33.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.116+0000 7efd6d38a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565111a774d0 0x565111a80380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.116+0000 7efd6d38a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565111a77e80 0x565111a7d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.116+0000 7efd6d38a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565111a80a00 0x565111a7dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.116+0000 7efd6d38a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x565111a83910 con 0x565111a774d0 2024-04-22T13:28:33.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.116+0000 7efd6d38a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x565111995500 con 0x565111a80a00 2024-04-22T13:28:33.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.116+0000 7efd6d38a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x565111a83a90 con 0x565111a77e80 2024-04-22T13:28:33.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.116+0000 7efd6d38a740 10 client.0 _ll_register_callbacks cb 0x565111963b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:33.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.116+0000 7efd63fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565111a774d0 0x565111a80380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.116+0000 7efd63fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565111a774d0 0x565111a80380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32786/0 (socket says 172.21.15.12:32786) 2024-04-22T13:28:33.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.116+0000 7efd63fff640 1 -- 172.21.15.12:0/217395248 learned_addr learned my addr 172.21.15.12:0/217395248 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:33.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.116+0000 7efd68d3c640 1 --2- 172.21.15.12:0/217395248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565111a80a00 0x565111a7dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.116+0000 7efd637fe640 1 --2- 172.21.15.12:0/217395248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565111a77e80 0x565111a7d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.116+0000 7efd63fff640 1 -- 172.21.15.12:0/217395248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565111a77e80 msgr2=0x565111a7d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:33.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.116+0000 7efd63fff640 1 --2- 172.21.15.12:0/217395248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565111a77e80 0x565111a7d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.116+0000 7efd63fff640 1 -- 172.21.15.12:0/217395248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565111a80a00 msgr2=0x565111a7dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:33.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.116+0000 7efd63fff640 1 --2- 172.21.15.12:0/217395248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565111a80a00 0x565111a7dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.116+0000 7efd63fff640 1 -- 172.21.15.12:0/217395248 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651116ee3f0 con 0x565111a774d0 2024-04-22T13:28:33.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.116+0000 7efd68d3c640 1 --2- 172.21.15.12:0/217395248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565111a80a00 0x565111a7dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:33.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.116+0000 7efd63fff640 1 --2- 172.21.15.12:0/217395248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565111a774d0 0x565111a80380 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7efd54012700 tx=0x7efd54012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4998bd7471f0ae server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:33.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.116+0000 7efd437fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:33.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.117+0000 7efd437fe640 1 -- 172.21.15.12:0/217395248 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54004320 con 0x565111a774d0 2024-04-22T13:28:33.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.117+0000 7efd437fe640 1 -- 172.21.15.12:0/217395248 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7efd540119a0 con 0x565111a774d0 2024-04-22T13:28:33.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.117+0000 7efd6d38a740 1 -- 172.21.15.12:0/217395248 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd5809cab0 con 0x565111a774d0 2024-04-22T13:28:33.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.117+0000 7efd437fe640 1 -- 172.21.15.12:0/217395248 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efd540044c0 con 0x565111a774d0 2024-04-22T13:28:33.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.117+0000 7efd6d38a740 10 client.17247 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:33.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.117+0000 7efd6d38a740 1 -- 172.21.15.12:0/217395248 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5651117b87d0 con 0x565111a774d0 2024-04-22T13:28:33.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.117+0000 7efd437fe640 1 -- 172.21.15.12:0/217395248 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd5809cab0 con 0x565111a774d0 2024-04-22T13:28:33.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.117+0000 7efd437fe640 1 -- 172.21.15.12:0/217395248 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7efd5400f440 con 0x565111a774d0 2024-04-22T13:28:33.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.117+0000 7efd6d38a740 10 client.17247 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:33.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.117+0000 7efd6d38a740 10 client.17247 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:28:33.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.117+0000 7efd6d38a740 20 client.17247 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:33.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.117+0000 7efd6d38a740 1 --2- 172.21.15.12:0/217395248 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x565111a90c60 0x565111a93050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.117+0000 7efd6d38a740 4 client.17247 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:33.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.117+0000 7efd6d38a740 1 -- 172.21.15.12:0/217395248 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56511197b930 con 0x565111a90c60 2024-04-22T13:28:33.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.118+0000 7efd637fe640 1 --2- 172.21.15.12:0/217395248 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x565111a90c60 0x565111a93050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.118+0000 7efd637fe640 1 --2- 172.21.15.12:0/217395248 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x565111a90c60 0x565111a93050 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6e83ffb3fe02bfa3 server_cookie=333001bbdcc46c26 in_seq=0 out_seq=0 2024-04-22T13:28:33.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.118+0000 7efd437fe640 10 client.17247 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:28:33.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.119+0000 7efd437fe640 1 -- 172.21.15.12:0/217395248 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56511197b930 con 0x565111a90c60 2024-04-22T13:28:33.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.119+0000 7efd437fe640 10 client.17247 handle_command_reply: tid=0 2024-04-22T13:28:33.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.119+0000 7efd6d38a740 1 client.17247 shutdown 2024-04-22T13:28:33.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.119+0000 7efd6d38a740 1 -- 172.21.15.12:0/217395248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565111a774d0 msgr2=0x565111a80380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:33.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.119+0000 7efd6d38a740 1 --2- 172.21.15.12:0/217395248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565111a774d0 0x565111a80380 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7efd54012700 tx=0x7efd54012bc0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.119+0000 7efd6d38a740 1 -- 172.21.15.12:0/217395248 shutdown_connections 2024-04-22T13:28:33.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.119+0000 7efd6d38a740 1 --2- 172.21.15.12:0/217395248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565111a80a00 0x565111a7dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.119+0000 7efd6d38a740 1 --2- 172.21.15.12:0/217395248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565111a77e80 0x565111a7d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.119+0000 7efd6d38a740 1 --2- 172.21.15.12:0/217395248 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x565111a90c60 0x565111a93050 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.119+0000 7efd6d38a740 1 --2- 172.21.15.12:0/217395248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565111a774d0 0x565111a80380 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.119+0000 7efd437fe640 0 client.17247 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:28:33.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.119+0000 7efd6d38a740 1 -- 172.21.15.12:0/217395248 >> 172.21.15.12:0/217395248 conn(0x565111a71ba0 msgr2=0x565111a738f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:33.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.119+0000 7efd6d38a740 1 -- 172.21.15.12:0/217395248 shutdown_connections 2024-04-22T13:28:33.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.119+0000 7efd6d38a740 1 -- 172.21.15.12:0/217395248 wait complete. 2024-04-22T13:28:33.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.119+0000 7efd6d38a740 20 client.17247 trim_cache size 0 max 16384 2024-04-22T13:28:33.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.132+0000 7efd6d38a740 1 Processor -- start 2024-04-22T13:28:33.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.132+0000 7efd6d38a740 1 -- start start 2024-04-22T13:28:33.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.133+0000 7efd6d38a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565111a774d0 0x565111a8c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.133+0000 7efd6d38a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565111a77e80 0x565111a7fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.133+0000 7efd6d38a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565111a80a00 0x565111a80380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.133+0000 7efd6d38a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efd641adae0 con 0x565111a80a00 2024-04-22T13:28:33.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.133+0000 7efd6d38a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efd641af620 con 0x565111a774d0 2024-04-22T13:28:33.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.133+0000 7efd6d38a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efd641ae1b0 con 0x565111a77e80 2024-04-22T13:28:33.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.133+0000 7efd68d3c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565111a80a00 0x565111a80380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.133+0000 7efd68d3c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565111a80a00 0x565111a80380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32792/0 (socket says 172.21.15.12:32792) 2024-04-22T13:28:33.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.133+0000 7efd68d3c640 1 -- 172.21.15.12:0/4074847032 learned_addr learned my addr 172.21.15.12:0/4074847032 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:33.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.133+0000 7efd63fff640 1 --2- 172.21.15.12:0/4074847032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565111a774d0 0x565111a8c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.133+0000 7efd637fe640 1 --2- 172.21.15.12:0/4074847032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565111a77e80 0x565111a7fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.133+0000 7efd68d3c640 1 -- 172.21.15.12:0/4074847032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565111a77e80 msgr2=0x565111a7fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:33.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.133+0000 7efd68d3c640 1 --2- 172.21.15.12:0/4074847032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565111a77e80 0x565111a7fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.133+0000 7efd68d3c640 1 -- 172.21.15.12:0/4074847032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565111a774d0 msgr2=0x565111a8c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:33.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.133+0000 7efd68d3c640 1 --2- 172.21.15.12:0/4074847032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565111a774d0 0x565111a8c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.133+0000 7efd68d3c640 1 -- 172.21.15.12:0/4074847032 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565111a7e090 con 0x565111a80a00 2024-04-22T13:28:33.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.133+0000 7efd637fe640 1 --2- 172.21.15.12:0/4074847032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565111a77e80 0x565111a7fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:33.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.133+0000 7efd68d3c640 1 --2- 172.21.15.12:0/4074847032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565111a80a00 0x565111a80380 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7efd5c036000 tx=0x7efd5c042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf4dfaff943198c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:33.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.133+0000 7efd49ffb640 1 -- 172.21.15.12:0/4074847032 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c039040 con 0x565111a80a00 2024-04-22T13:28:33.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.133+0000 7efd49ffb640 1 -- 172.21.15.12:0/4074847032 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd5c055070 con 0x565111a80a00 2024-04-22T13:28:33.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.133+0000 7efd49ffb640 1 -- 172.21.15.12:0/4074847032 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c0040d0 con 0x565111a80a00 2024-04-22T13:28:33.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.134+0000 7efd6d38a740 1 -- 172.21.15.12:0/4074847032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565111a80a00 msgr2=0x565111a80380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:33.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.134+0000 7efd6d38a740 1 --2- 172.21.15.12:0/4074847032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565111a80a00 0x565111a80380 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7efd5c036000 tx=0x7efd5c042000 comp rx=0 tx=0).stop 2024-04-22T13:28:33.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.134+0000 7efd6d38a740 1 -- 172.21.15.12:0/4074847032 shutdown_connections 2024-04-22T13:28:33.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.134+0000 7efd6d38a740 1 --2- 172.21.15.12:0/4074847032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565111a80a00 0x565111a80380 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.134+0000 7efd6d38a740 1 --2- 172.21.15.12:0/4074847032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565111a77e80 0x565111a7fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.134+0000 7efd6d38a740 1 --2- 172.21.15.12:0/4074847032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565111a774d0 0x565111a8c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.134+0000 7efd6d38a740 1 -- 172.21.15.12:0/4074847032 >> 172.21.15.12:0/4074847032 conn(0x565111a71ba0 msgr2=0x565111a70c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:33.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.134+0000 7efd6d38a740 1 -- 172.21.15.12:0/4074847032 shutdown_connections 2024-04-22T13:28:33.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.134+0000 7efd6d38a740 1 -- 172.21.15.12:0/4074847032 wait complete. 2024-04-22T13:28:33.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.134+0000 7efd6d38a740 1 -- start start 2024-04-22T13:28:33.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.135+0000 7efd6d38a740 1 Processor -- start 2024-04-22T13:28:33.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.135+0000 7efd6d38a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565111a774d0 0x565111a7a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.135+0000 7efd6d38a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565111a77e80 0x565111a7aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.135+0000 7efd6d38a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565111a80a00 0x565111a7af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.135+0000 7efd6d38a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efd641aef50 con 0x565111a77e80 2024-04-22T13:28:33.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.135+0000 7efd6d38a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efd641ae880 con 0x565111a80a00 2024-04-22T13:28:33.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.135+0000 7efd6d38a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efd641abac0 con 0x565111a774d0 2024-04-22T13:28:33.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.135+0000 7efd6d38a740 10 client.0 _ll_register_callbacks cb 0x7efd54009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:33.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.135+0000 7efd637fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565111a77e80 0x565111a7aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.135+0000 7efd637fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565111a77e80 0x565111a7aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32794/0 (socket says 172.21.15.12:32794) 2024-04-22T13:28:33.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.135+0000 7efd637fe640 1 -- 172.21.15.12:0/2414434236 learned_addr learned my addr 172.21.15.12:0/2414434236 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:33.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.135+0000 7efd63fff640 1 --2- 172.21.15.12:0/2414434236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565111a774d0 0x565111a7a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.135+0000 7efd68d3c640 1 --2- 172.21.15.12:0/2414434236 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565111a80a00 0x565111a7af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.135+0000 7efd637fe640 1 -- 172.21.15.12:0/2414434236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565111a774d0 msgr2=0x565111a7a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:33.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.135+0000 7efd637fe640 1 --2- 172.21.15.12:0/2414434236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565111a774d0 0x565111a7a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.135+0000 7efd637fe640 1 -- 172.21.15.12:0/2414434236 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565111a80a00 msgr2=0x565111a7af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:33.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.135+0000 7efd637fe640 1 --2- 172.21.15.12:0/2414434236 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565111a80a00 0x565111a7af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.135+0000 7efd637fe640 1 -- 172.21.15.12:0/2414434236 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565111af6290 con 0x565111a77e80 2024-04-22T13:28:33.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.135+0000 7efd637fe640 1 --2- 172.21.15.12:0/2414434236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565111a77e80 0x565111a7aa00 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7efd5809ca40 tx=0x7efd58010720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b5afa94d689088f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:33.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.135+0000 7efd437fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:33.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.135+0000 7efd437fe640 1 -- 172.21.15.12:0/2414434236 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5809c880 con 0x565111a77e80 2024-04-22T13:28:33.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.135+0000 7efd437fe640 1 -- 172.21.15.12:0/2414434236 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7efd580ce4e0 con 0x565111a77e80 2024-04-22T13:28:33.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.136+0000 7efd6d38a740 1 -- 172.21.15.12:0/2414434236 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd5c055070 con 0x565111a77e80 2024-04-22T13:28:33.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.136+0000 7efd437fe640 1 -- 172.21.15.12:0/2414434236 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efd580a5720 con 0x565111a77e80 2024-04-22T13:28:33.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.136+0000 7efd6d38a740 10 client.17253 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:33.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.136+0000 7efd6d38a740 1 -- 172.21.15.12:0/2414434236 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565111af74e0 con 0x565111a77e80 2024-04-22T13:28:33.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.136+0000 7efd437fe640 1 -- 172.21.15.12:0/2414434236 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd5c055070 con 0x565111a77e80 2024-04-22T13:28:33.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.136+0000 7efd437fe640 1 -- 172.21.15.12:0/2414434236 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7efd580a5290 con 0x565111a77e80 2024-04-22T13:28:33.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.136+0000 7efd6d38a740 10 client.17253 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:33.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.136+0000 7efd6d38a740 10 client.17253 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:28:33.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.136+0000 7efd6d38a740 20 client.17253 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:33.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.136+0000 7efd6d38a740 1 --2- 172.21.15.12:0/2414434236 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x565111af7820 0x565111af9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.136+0000 7efd6d38a740 4 client.17253 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:28:33.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.136+0000 7efd6d38a740 1 -- 172.21.15.12:0/2414434236 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5651119e9c10 con 0x565111af7820 2024-04-22T13:28:33.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.136+0000 7efd63fff640 1 --2- 172.21.15.12:0/2414434236 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x565111af7820 0x565111af9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.137+0000 7efd63fff640 1 --2- 172.21.15.12:0/2414434236 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x565111af7820 0x565111af9c10 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ff75fd75391b95 server_cookie=a9a4ea3f6d8cf00f in_seq=0 out_seq=0 2024-04-22T13:28:33.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.137+0000 7efd437fe640 10 client.17253 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:28:33.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.137+0000 7efd437fe640 1 -- 172.21.15.12:0/2414434236 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5651119e9c10 con 0x565111af7820 2024-04-22T13:28:33.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.137+0000 7efd437fe640 10 client.17253 handle_command_reply: tid=0 2024-04-22T13:28:33.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.137+0000 7efd6d38a740 1 client.17253 shutdown 2024-04-22T13:28:33.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.137+0000 7efd6d38a740 1 -- 172.21.15.12:0/2414434236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565111a77e80 msgr2=0x565111a7aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:33.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.137+0000 7efd6d38a740 1 --2- 172.21.15.12:0/2414434236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565111a77e80 0x565111a7aa00 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7efd5809ca40 tx=0x7efd58010720 comp rx=0 tx=0).stop 2024-04-22T13:28:33.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.137+0000 7efd6d38a740 1 -- 172.21.15.12:0/2414434236 shutdown_connections 2024-04-22T13:28:33.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.137+0000 7efd6d38a740 1 --2- 172.21.15.12:0/2414434236 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565111a80a00 0x565111a7af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.137+0000 7efd6d38a740 1 --2- 172.21.15.12:0/2414434236 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x565111af7820 0x565111af9c10 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.137+0000 7efd6d38a740 1 --2- 172.21.15.12:0/2414434236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565111a77e80 0x565111a7aa00 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.137+0000 7efd6d38a740 1 --2- 172.21.15.12:0/2414434236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565111a774d0 0x565111a7a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.137+0000 7efd6d38a740 1 -- 172.21.15.12:0/2414434236 >> 172.21.15.12:0/2414434236 conn(0x565111a71ba0 msgr2=0x565111a78260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:33.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.137+0000 7efd437fe640 0 client.17253 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:28:33.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.137+0000 7efd6d38a740 1 -- 172.21.15.12:0/2414434236 shutdown_connections 2024-04-22T13:28:33.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.137+0000 7efd6d38a740 1 -- 172.21.15.12:0/2414434236 wait complete. 2024-04-22T13:28:33.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.137+0000 7efd6d38a740 20 client.17253 trim_cache size 0 max 16384 2024-04-22T13:28:33.141 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:28:33.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.138+0000 7efd6a79f640 1 -- 172.21.15.12:0/3860059494 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7efd4c0a3d20 msgr2=0x7efd4c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:33.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.138+0000 7efd6a79f640 1 --2- 172.21.15.12:0/3860059494 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7efd4c0a3d20 0x7efd4c0a61e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efd5c004af0 tx=0x7efd5c005c50 comp rx=0 tx=0).stop 2024-04-22T13:28:33.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.138+0000 7efd6a79f640 1 -- 172.21.15.12:0/3860059494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd64103af0 msgr2=0x7efd64198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:33.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.138+0000 7efd6a79f640 1 --2- 172.21.15.12:0/3860059494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd64103af0 0x7efd64198c40 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7efd5800e9f0 tx=0x7efd5800eeb0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.138+0000 7efd6a79f640 1 -- 172.21.15.12:0/3860059494 shutdown_connections 2024-04-22T13:28:33.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.138+0000 7efd6a79f640 1 --2- 172.21.15.12:0/3860059494 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7efd4c0a3d20 0x7efd4c0a61e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.138+0000 7efd6a79f640 1 --2- 172.21.15.12:0/3860059494 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efd6410c510 0x7efd6419cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.138+0000 7efd6a79f640 1 --2- 172.21.15.12:0/3860059494 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efd64103af0 0x7efd64198c40 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.138+0000 7efd6a79f640 1 --2- 172.21.15.12:0/3860059494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd641031d0 0x7efd64198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.138+0000 7efd6a79f640 1 -- 172.21.15.12:0/3860059494 >> 172.21.15.12:0/3860059494 conn(0x7efd64077f50 msgr2=0x7efd640fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:33.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.138+0000 7efd6a79f640 1 -- 172.21.15.12:0/3860059494 shutdown_connections 2024-04-22T13:28:33.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.138+0000 7efd6a79f640 1 -- 172.21.15.12:0/3860059494 wait complete. 2024-04-22T13:28:33.146 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:28:33.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.255+0000 7fae4944e640 1 -- 172.21.15.12:0/1050921317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae4410c510 msgr2=0x7fae4410e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:33.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.255+0000 7fae4944e640 1 --2- 172.21.15.12:0/1050921317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae4410c510 0x7fae4410e9b0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fae34009ca0 tx=0x7fae3402c230 comp rx=0 tx=0).stop 2024-04-22T13:28:33.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.255+0000 7fae4944e640 1 -- 172.21.15.12:0/1050921317 shutdown_connections 2024-04-22T13:28:33.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.255+0000 7fae4944e640 1 --2- 172.21.15.12:0/1050921317 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae4410c510 0x7fae4410e9b0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.255+0000 7fae4944e640 1 --2- 172.21.15.12:0/1050921317 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae44103af0 0x7fae4410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.255+0000 7fae4944e640 1 --2- 172.21.15.12:0/1050921317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae441031d0 0x7fae441035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.255+0000 7fae4944e640 1 -- 172.21.15.12:0/1050921317 >> 172.21.15.12:0/1050921317 conn(0x7fae44077f50 msgr2=0x7fae44078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:33.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.255+0000 7fae4944e640 1 -- 172.21.15.12:0/1050921317 shutdown_connections 2024-04-22T13:28:33.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.255+0000 7fae4944e640 1 -- 172.21.15.12:0/1050921317 wait complete. 2024-04-22T13:28:33.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.255+0000 7fae4944e640 1 Processor -- start 2024-04-22T13:28:33.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.255+0000 7fae4944e640 1 -- start start 2024-04-22T13:28:33.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.256+0000 7fae4944e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae441031d0 0x7fae441986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.256+0000 7fae4944e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae44103af0 0x7fae44198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.256+0000 7fae4944e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae4410c510 0x7fae4419cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.256+0000 7fae4944e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fae441121a0 con 0x7fae44103af0 2024-04-22T13:28:33.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.256+0000 7fae4944e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fae44112020 con 0x7fae441031d0 2024-04-22T13:28:33.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.256+0000 7fae4944e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fae44112320 con 0x7fae4410c510 2024-04-22T13:28:33.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.256+0000 7fae427fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae44103af0 0x7fae44198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.256+0000 7fae427fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae44103af0 0x7fae44198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32808/0 (socket says 172.21.15.12:32808) 2024-04-22T13:28:33.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.256+0000 7fae427fc640 1 -- 172.21.15.12:0/3768703107 learned_addr learned my addr 172.21.15.12:0/3768703107 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:33.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.256+0000 7fae42ffd640 1 --2- 172.21.15.12:0/3768703107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae441031d0 0x7fae441986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.256+0000 7fae437fe640 1 --2- 172.21.15.12:0/3768703107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae4410c510 0x7fae4419cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.256+0000 7fae427fc640 1 -- 172.21.15.12:0/3768703107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae4410c510 msgr2=0x7fae4419cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:33.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.256+0000 7fae427fc640 1 --2- 172.21.15.12:0/3768703107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae4410c510 0x7fae4419cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.256+0000 7fae427fc640 1 -- 172.21.15.12:0/3768703107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae441031d0 msgr2=0x7fae441986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:33.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.256+0000 7fae427fc640 1 --2- 172.21.15.12:0/3768703107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae441031d0 0x7fae441986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.256+0000 7fae427fc640 1 -- 172.21.15.12:0/3768703107 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae4419d6d0 con 0x7fae44103af0 2024-04-22T13:28:33.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.256+0000 7fae427fc640 1 --2- 172.21.15.12:0/3768703107 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae44103af0 0x7fae44198c20 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00d770 tx=0x7fae2c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:33.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.256+0000 7fae3bfff640 1 -- 172.21.15.12:0/3768703107 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c017070 con 0x7fae44103af0 2024-04-22T13:28:33.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.257+0000 7fae4944e640 1 -- 172.21.15.12:0/3768703107 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae4419d9c0 con 0x7fae44103af0 2024-04-22T13:28:33.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.257+0000 7fae3bfff640 1 -- 172.21.15.12:0/3768703107 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae2c0054d0 con 0x7fae44103af0 2024-04-22T13:28:33.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.257+0000 7fae3bfff640 1 -- 172.21.15.12:0/3768703107 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c004c30 con 0x7fae44103af0 2024-04-22T13:28:33.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.257+0000 7fae4944e640 1 -- 172.21.15.12:0/3768703107 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae441a5250 con 0x7fae44103af0 2024-04-22T13:28:33.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.259+0000 7fae3bfff640 1 -- 172.21.15.12:0/3768703107 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fae2c004df0 con 0x7fae44103af0 2024-04-22T13:28:33.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.259+0000 7fae4c042740 1 Processor -- start 2024-04-22T13:28:33.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.259+0000 7fae4c042740 1 -- start start 2024-04-22T13:28:33.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.259+0000 7fae4c042740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586a22d44d0 0x5586a22d48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.259+0000 7fae4c042740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5586a22d4e80 0x5586a22dd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.259+0000 7fae4c042740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586a22dda00 0x5586a22dfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.259+0000 7fae4c042740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5586a22391f0 con 0x5586a22d44d0 2024-04-22T13:28:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.259+0000 7fae4c042740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5586a21d64b0 con 0x5586a22d4e80 2024-04-22T13:28:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.259+0000 7fae4c042740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5586a22351b0 con 0x5586a22dda00 2024-04-22T13:28:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.260+0000 7fae437fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586a22d44d0 0x5586a22d48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.260+0000 7fae437fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586a22d44d0 0x5586a22d48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32812/0 (socket says 172.21.15.12:32812) 2024-04-22T13:28:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.260+0000 7fae437fe640 1 -- 172.21.15.12:0/4261250288 learned_addr learned my addr 172.21.15.12:0/4261250288 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.260+0000 7fae427fc640 1 --2- 172.21.15.12:0/4261250288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586a22dda00 0x5586a22dfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.260+0000 7fae42ffd640 1 --2- 172.21.15.12:0/4261250288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5586a22d4e80 0x5586a22dd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.260+0000 7fae3bfff640 1 --2- 172.21.15.12:0/3768703107 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fae300a3d20 0x7fae300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.260+0000 7fae437fe640 1 -- 172.21.15.12:0/4261250288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586a22dda00 msgr2=0x5586a22dfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.260+0000 7fae437fe640 1 --2- 172.21.15.12:0/4261250288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586a22dda00 0x5586a22dfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.260+0000 7fae437fe640 1 -- 172.21.15.12:0/4261250288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5586a22d4e80 msgr2=0x5586a22dd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:33.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.260+0000 7fae437fe640 1 --2- 172.21.15.12:0/4261250288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5586a22d4e80 0x5586a22dd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.260+0000 7fae437fe640 1 -- 172.21.15.12:0/4261250288 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586a21c81b0 con 0x5586a22d44d0 2024-04-22T13:28:33.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.260+0000 7fae427fc640 1 --2- 172.21.15.12:0/4261250288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586a22dda00 0x5586a22dfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:33.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.260+0000 7fae3bfff640 1 -- 172.21.15.12:0/3768703107 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fae2c0d00a0 con 0x7fae44103af0 2024-04-22T13:28:33.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.260+0000 7fae437fe640 1 --2- 172.21.15.12:0/3768703107 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fae300a3d20 0x7fae300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.260+0000 7fae437fe640 1 --2- 172.21.15.12:0/4261250288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586a22d44d0 0x5586a22d48b0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fae34037040 tx=0x7fae34009980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63aacbc2b5a51c33 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:33.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.260+0000 7fae397fa640 1 -- 172.21.15.12:0/4261250288 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae34039040 con 0x5586a22d44d0 2024-04-22T13:28:33.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.260+0000 7fae397fa640 1 -- 172.21.15.12:0/4261250288 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae34032640 con 0x5586a22d44d0 2024-04-22T13:28:33.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.260+0000 7fae397fa640 1 -- 172.21.15.12:0/4261250288 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3404a640 con 0x5586a22d44d0 2024-04-22T13:28:33.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.261+0000 7fae4c042740 1 -- 172.21.15.12:0/4261250288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586a22d44d0 msgr2=0x5586a22d48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:33.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.261+0000 7fae437fe640 1 --2- 172.21.15.12:0/3768703107 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fae300a3d20 0x7fae300a61e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fae3404c000 tx=0x7fae34045000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:33.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.261+0000 7fae4c042740 1 --2- 172.21.15.12:0/4261250288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586a22d44d0 0x5586a22d48b0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fae34037040 tx=0x7fae34009980 comp rx=0 tx=0).stop 2024-04-22T13:28:33.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.261+0000 7fae4c042740 1 -- 172.21.15.12:0/4261250288 shutdown_connections 2024-04-22T13:28:33.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.261+0000 7fae4c042740 1 --2- 172.21.15.12:0/4261250288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586a22dda00 0x5586a22dfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.261+0000 7fae4c042740 1 --2- 172.21.15.12:0/4261250288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5586a22d4e80 0x5586a22dd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.261+0000 7fae4c042740 1 --2- 172.21.15.12:0/4261250288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586a22d44d0 0x5586a22d48b0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.261+0000 7fae4c042740 1 -- 172.21.15.12:0/4261250288 >> 172.21.15.12:0/4261250288 conn(0x5586a22ceba0 msgr2=0x5586a22cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:33.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.261+0000 7fae4c042740 1 -- 172.21.15.12:0/4261250288 shutdown_connections 2024-04-22T13:28:33.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.261+0000 7fae4c042740 1 -- 172.21.15.12:0/4261250288 wait complete. 2024-04-22T13:28:33.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.262+0000 7fae4c042740 1 -- start start 2024-04-22T13:28:33.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.262+0000 7fae4c042740 1 Processor -- start 2024-04-22T13:28:33.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.262+0000 7fae4c042740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586a22d44d0 0x5586a22dd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.262+0000 7fae4c042740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5586a22d4e80 0x5586a22da6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.262+0000 7fae4c042740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586a22dda00 0x5586a22dad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.262+0000 7fae4c042740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5586a22e0970 con 0x5586a22dda00 2024-04-22T13:28:33.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.262+0000 7fae4c042740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5586a21f2500 con 0x5586a22d4e80 2024-04-22T13:28:33.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.262+0000 7fae4c042740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5586a22e0af0 con 0x5586a22d44d0 2024-04-22T13:28:33.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.262+0000 7fae4c042740 10 client.0 _ll_register_callbacks cb 0x5586a21c0b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:33.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.262+0000 7fae437fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586a22dda00 0x5586a22dad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.262+0000 7fae437fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586a22dda00 0x5586a22dad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32820/0 (socket says 172.21.15.12:32820) 2024-04-22T13:28:33.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.262+0000 7fae437fe640 1 -- 172.21.15.12:0/1612017761 learned_addr learned my addr 172.21.15.12:0/1612017761 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:33.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.262+0000 7fae427fc640 1 --2- 172.21.15.12:0/1612017761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5586a22d4e80 0x5586a22da6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.262+0000 7fae42ffd640 1 --2- 172.21.15.12:0/1612017761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586a22d44d0 0x5586a22dd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.262+0000 7fae437fe640 1 -- 172.21.15.12:0/1612017761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586a22d44d0 msgr2=0x5586a22dd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:33.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.262+0000 7fae437fe640 1 --2- 172.21.15.12:0/1612017761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586a22d44d0 0x5586a22dd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.262+0000 7fae437fe640 1 -- 172.21.15.12:0/1612017761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5586a22d4e80 msgr2=0x5586a22da6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:33.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.262+0000 7fae437fe640 1 --2- 172.21.15.12:0/1612017761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5586a22d4e80 0x5586a22da6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.262+0000 7fae437fe640 1 -- 172.21.15.12:0/1612017761 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586a1f4b3f0 con 0x5586a22dda00 2024-04-22T13:28:33.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.262+0000 7fae42ffd640 1 --2- 172.21.15.12:0/1612017761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586a22d44d0 0x5586a22dd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:33.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.262+0000 7fae427fc640 1 --2- 172.21.15.12:0/1612017761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5586a22d4e80 0x5586a22da6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:33.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.263+0000 7fae437fe640 1 --2- 172.21.15.12:0/1612017761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586a22dda00 0x5586a22dad00 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fae34004af0 tx=0x7fae34032eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a784932f474b3f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:33.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.263+0000 7fae2a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.263+0000 7fae2a7fc640 1 -- 172.21.15.12:0/1612017761 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae34039070 con 0x5586a22dda00 2024-04-22T13:28:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.263+0000 7fae2a7fc640 1 -- 172.21.15.12:0/1612017761 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fae34004690 con 0x5586a22dda00 2024-04-22T13:28:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.263+0000 7fae4c042740 1 -- 172.21.15.12:0/1612017761 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae34032640 con 0x5586a22dda00 2024-04-22T13:28:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.263+0000 7fae2a7fc640 1 -- 172.21.15.12:0/1612017761 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fae3404a650 con 0x5586a22dda00 2024-04-22T13:28:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.263+0000 7fae4c042740 10 client.17265 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.263+0000 7fae4c042740 1 -- 172.21.15.12:0/1612017761 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5586a20157d0 con 0x5586a22dda00 2024-04-22T13:28:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.263+0000 7fae2a7fc640 1 -- 172.21.15.12:0/1612017761 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae34032640 con 0x5586a22dda00 2024-04-22T13:28:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.263+0000 7fae2a7fc640 1 -- 172.21.15.12:0/1612017761 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fae3403f040 con 0x5586a22dda00 2024-04-22T13:28:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.263+0000 7fae4c042740 10 client.17265 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.263+0000 7fae4c042740 10 client.17265 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:28:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.263+0000 7fae4c042740 20 client.17265 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.263+0000 7fae4c042740 1 --2- 172.21.15.12:0/1612017761 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5586a22edb10 0x5586a22eff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.263+0000 7fae4c042740 4 client.17265 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:33.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.263+0000 7fae4c042740 1 -- 172.21.15.12:0/1612017761 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586a21d8930 con 0x5586a22edb10 2024-04-22T13:28:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.264+0000 7fae42ffd640 1 --2- 172.21.15.12:0/1612017761 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5586a22edb10 0x5586a22eff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.264+0000 7fae42ffd640 1 --2- 172.21.15.12:0/1612017761 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5586a22edb10 0x5586a22eff00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4d9a95aa8946d8fc server_cookie=537f69e3d605a427 in_seq=0 out_seq=0 2024-04-22T13:28:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.264+0000 7fae2a7fc640 10 client.17265 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:28:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.265+0000 7fae2a7fc640 1 -- 172.21.15.12:0/1612017761 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5586a21d8930 con 0x5586a22edb10 2024-04-22T13:28:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.265+0000 7fae2a7fc640 10 client.17265 handle_command_reply: tid=0 2024-04-22T13:28:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.265+0000 7fae4c042740 1 client.17265 shutdown 2024-04-22T13:28:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.265+0000 7fae4c042740 1 -- 172.21.15.12:0/1612017761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586a22dda00 msgr2=0x5586a22dad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.265+0000 7fae4c042740 1 --2- 172.21.15.12:0/1612017761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586a22dda00 0x5586a22dad00 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fae34004af0 tx=0x7fae34032eb0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.265+0000 7fae4c042740 1 -- 172.21.15.12:0/1612017761 shutdown_connections 2024-04-22T13:28:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.265+0000 7fae4c042740 1 --2- 172.21.15.12:0/1612017761 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5586a22edb10 0x5586a22eff00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.265+0000 7fae4c042740 1 --2- 172.21.15.12:0/1612017761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586a22dda00 0x5586a22dad00 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.265+0000 7fae4c042740 1 --2- 172.21.15.12:0/1612017761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5586a22d4e80 0x5586a22da6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.265+0000 7fae2a7fc640 0 client.17265 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:28:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.265+0000 7fae4c042740 1 --2- 172.21.15.12:0/1612017761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586a22d44d0 0x5586a22dd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.265+0000 7fae4c042740 1 -- 172.21.15.12:0/1612017761 >> 172.21.15.12:0/1612017761 conn(0x5586a22ceba0 msgr2=0x5586a22d08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:33.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.265+0000 7fae4c042740 1 -- 172.21.15.12:0/1612017761 shutdown_connections 2024-04-22T13:28:33.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.265+0000 7fae4c042740 1 -- 172.21.15.12:0/1612017761 wait complete. 2024-04-22T13:28:33.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.265+0000 7fae4c042740 20 client.17265 trim_cache size 0 max 16384 2024-04-22T13:28:33.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.278+0000 7fae4c042740 1 Processor -- start 2024-04-22T13:28:33.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.278+0000 7fae4c042740 1 -- start start 2024-04-22T13:28:33.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.279+0000 7fae4c042740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586a22d44d0 0x5586a22dce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.279+0000 7fae4c042740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586a22d4e80 0x5586a22dd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.279+0000 7fae4c042740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5586a22dda00 0x5586a22f2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.279+0000 7fae4c042740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fae441ad990 con 0x5586a22d44d0 2024-04-22T13:28:33.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.279+0000 7fae4c042740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fae441af4d0 con 0x5586a22dda00 2024-04-22T13:28:33.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.279+0000 7fae4c042740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fae441ae060 con 0x5586a22d4e80 2024-04-22T13:28:33.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.279+0000 7fae42ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586a22d44d0 0x5586a22dce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.279+0000 7fae42ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586a22d44d0 0x5586a22dce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32828/0 (socket says 172.21.15.12:32828) 2024-04-22T13:28:33.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.279+0000 7fae42ffd640 1 -- 172.21.15.12:0/372181315 learned_addr learned my addr 172.21.15.12:0/372181315 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:33.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.279+0000 7fae42ffd640 1 -- 172.21.15.12:0/372181315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586a22d4e80 msgr2=0x5586a22dd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:33.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.279+0000 7fae427fc640 1 --2- 172.21.15.12:0/372181315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586a22d4e80 0x5586a22dd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.279+0000 7fae42ffd640 1 --2- 172.21.15.12:0/372181315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586a22d4e80 0x5586a22dd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.279+0000 7fae437fe640 1 --2- 172.21.15.12:0/372181315 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5586a22dda00 0x5586a22f2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.279+0000 7fae42ffd640 1 -- 172.21.15.12:0/372181315 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5586a22dda00 msgr2=0x5586a22f2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.279+0000 7fae42ffd640 1 --2- 172.21.15.12:0/372181315 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5586a22dda00 0x5586a22f2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.279+0000 7fae42ffd640 1 -- 172.21.15.12:0/372181315 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586a22cf4e0 con 0x5586a22d44d0 2024-04-22T13:28:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.279+0000 7fae437fe640 1 --2- 172.21.15.12:0/372181315 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5586a22dda00 0x5586a22f2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.279+0000 7fae42ffd640 1 --2- 172.21.15.12:0/372181315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586a22d44d0 0x5586a22dce20 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fae3c000c00 tx=0x7fae3c004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3542cc6413e6fad6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.280+0000 7fae38ff9640 1 -- 172.21.15.12:0/372181315 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c023020 con 0x5586a22d44d0 2024-04-22T13:28:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.280+0000 7fae38ff9640 1 -- 172.21.15.12:0/372181315 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae3c003d20 con 0x5586a22d44d0 2024-04-22T13:28:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.280+0000 7fae38ff9640 1 -- 172.21.15.12:0/372181315 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c0107b0 con 0x5586a22d44d0 2024-04-22T13:28:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.280+0000 7fae4c042740 1 -- 172.21.15.12:0/372181315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586a22d44d0 msgr2=0x5586a22dce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.280+0000 7fae4c042740 1 --2- 172.21.15.12:0/372181315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586a22d44d0 0x5586a22dce20 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fae3c000c00 tx=0x7fae3c004410 comp rx=0 tx=0).stop 2024-04-22T13:28:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.280+0000 7fae4c042740 1 -- 172.21.15.12:0/372181315 shutdown_connections 2024-04-22T13:28:33.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.280+0000 7fae4c042740 1 --2- 172.21.15.12:0/372181315 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5586a22dda00 0x5586a22f2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.280+0000 7fae4c042740 1 --2- 172.21.15.12:0/372181315 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586a22d4e80 0x5586a22dd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.280+0000 7fae4c042740 1 --2- 172.21.15.12:0/372181315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586a22d44d0 0x5586a22dce20 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.280+0000 7fae4c042740 1 -- 172.21.15.12:0/372181315 >> 172.21.15.12:0/372181315 conn(0x5586a22ceba0 msgr2=0x5586a22cdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:33.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.280+0000 7fae4c042740 1 -- 172.21.15.12:0/372181315 shutdown_connections 2024-04-22T13:28:33.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.280+0000 7fae4c042740 1 -- 172.21.15.12:0/372181315 wait complete. 2024-04-22T13:28:33.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.281+0000 7fae4c042740 1 -- start start 2024-04-22T13:28:33.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.281+0000 7fae4c042740 1 Processor -- start 2024-04-22T13:28:33.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.281+0000 7fae4c042740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5586a22d44d0 0x5586a22f1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.281+0000 7fae4c042740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586a22d4e80 0x5586a22f1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.281+0000 7fae4c042740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586a22dda00 0x5586a2352cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.281+0000 7fae4c042740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fae441aee00 con 0x5586a22dda00 2024-04-22T13:28:33.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.281+0000 7fae4c042740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fae441ae730 con 0x5586a22d44d0 2024-04-22T13:28:33.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.281+0000 7fae4c042740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fae441ab990 con 0x5586a22d4e80 2024-04-22T13:28:33.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.281+0000 7fae4c042740 10 client.0 _ll_register_callbacks cb 0x7fae34009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:33.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.281+0000 7fae437fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586a22dda00 0x5586a2352cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.281+0000 7fae437fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586a22dda00 0x5586a2352cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32830/0 (socket says 172.21.15.12:32830) 2024-04-22T13:28:33.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.281+0000 7fae437fe640 1 -- 172.21.15.12:0/1840547351 learned_addr learned my addr 172.21.15.12:0/1840547351 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:33.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.281+0000 7fae42ffd640 1 --2- 172.21.15.12:0/1840547351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5586a22d44d0 0x5586a22f1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.281+0000 7fae427fc640 1 --2- 172.21.15.12:0/1840547351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586a22d4e80 0x5586a22f1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.281+0000 7fae437fe640 1 -- 172.21.15.12:0/1840547351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586a22d4e80 msgr2=0x5586a22f1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:33.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.281+0000 7fae437fe640 1 --2- 172.21.15.12:0/1840547351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586a22d4e80 0x5586a22f1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.281+0000 7fae437fe640 1 -- 172.21.15.12:0/1840547351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5586a22d44d0 msgr2=0x5586a22f1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:33.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.281+0000 7fae437fe640 1 --2- 172.21.15.12:0/1840547351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5586a22d44d0 0x5586a22f1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.281+0000 7fae437fe640 1 -- 172.21.15.12:0/1840547351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586a2353230 con 0x5586a22dda00 2024-04-22T13:28:33.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.282+0000 7fae437fe640 1 --2- 172.21.15.12:0/1840547351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586a22dda00 0x5586a2352cf0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fae34002230 tx=0x7fae34032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1580f23333be1c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:33.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.282+0000 7fae2a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:33.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.282+0000 7fae2a7fc640 1 -- 172.21.15.12:0/1840547351 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae34039070 con 0x5586a22dda00 2024-04-22T13:28:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.282+0000 7fae2a7fc640 1 -- 172.21.15.12:0/1840547351 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fae34004410 con 0x5586a22dda00 2024-04-22T13:28:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.282+0000 7fae4c042740 1 -- 172.21.15.12:0/1840547351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae3c003d20 con 0x5586a22dda00 2024-04-22T13:28:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.282+0000 7fae2a7fc640 1 -- 172.21.15.12:0/1840547351 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fae34074050 con 0x5586a22dda00 2024-04-22T13:28:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.282+0000 7fae4c042740 10 client.17271 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.282+0000 7fae4c042740 1 -- 172.21.15.12:0/1840547351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5586a23543d0 con 0x5586a22dda00 2024-04-22T13:28:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.282+0000 7fae2a7fc640 1 -- 172.21.15.12:0/1840547351 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae3c003d20 con 0x5586a22dda00 2024-04-22T13:28:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.282+0000 7fae2a7fc640 1 -- 172.21.15.12:0/1840547351 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fae34032640 con 0x5586a22dda00 2024-04-22T13:28:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.282+0000 7fae4c042740 10 client.17271 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.282+0000 7fae4c042740 10 client.17271 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:28:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.282+0000 7fae4c042740 20 client.17271 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.282+0000 7fae4c042740 1 --2- 172.21.15.12:0/1840547351 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5586a2354b40 0x5586a2356f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.282+0000 7fae4c042740 4 client.17271 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:28:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.282+0000 7fae4c042740 1 -- 172.21.15.12:0/1840547351 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5586a2246c10 con 0x5586a2354b40 2024-04-22T13:28:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.283+0000 7fae42ffd640 1 --2- 172.21.15.12:0/1840547351 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5586a2354b40 0x5586a2356f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.283+0000 7fae2a7fc640 10 client.17271 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:28:33.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.283+0000 7fae42ffd640 1 --2- 172.21.15.12:0/1840547351 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5586a2354b40 0x5586a2356f30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9bbd2ac8d195a465 server_cookie=d4a8ac4648ed4a81 in_seq=0 out_seq=0 2024-04-22T13:28:33.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.283+0000 7fae2a7fc640 1 -- 172.21.15.12:0/1840547351 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5586a2246c10 con 0x5586a2354b40 2024-04-22T13:28:33.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.283+0000 7fae2a7fc640 10 client.17271 handle_command_reply: tid=0 2024-04-22T13:28:33.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.283+0000 7fae4c042740 1 client.17271 shutdown 2024-04-22T13:28:33.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.284+0000 7fae4c042740 1 -- 172.21.15.12:0/1840547351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586a22dda00 msgr2=0x5586a2352cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:33.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.284+0000 7fae4c042740 1 --2- 172.21.15.12:0/1840547351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586a22dda00 0x5586a2352cf0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fae34002230 tx=0x7fae34032f20 comp rx=0 tx=0).stop 2024-04-22T13:28:33.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.284+0000 7fae4c042740 1 -- 172.21.15.12:0/1840547351 shutdown_connections 2024-04-22T13:28:33.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.284+0000 7fae4c042740 1 --2- 172.21.15.12:0/1840547351 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5586a2354b40 0x5586a2356f30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.284+0000 7fae4c042740 1 --2- 172.21.15.12:0/1840547351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5586a22dda00 0x5586a2352cf0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.284+0000 7fae4c042740 1 --2- 172.21.15.12:0/1840547351 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586a22d4e80 0x5586a22f1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.284+0000 7fae4c042740 1 --2- 172.21.15.12:0/1840547351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5586a22d44d0 0x5586a22f1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.284+0000 7fae2a7fc640 0 client.17271 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:28:33.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.284+0000 7fae4c042740 1 -- 172.21.15.12:0/1840547351 >> 172.21.15.12:0/1840547351 conn(0x5586a22ceba0 msgr2=0x5586a22e8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:33.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.284+0000 7fae4c042740 1 -- 172.21.15.12:0/1840547351 shutdown_connections 2024-04-22T13:28:33.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.284+0000 7fae4c042740 1 -- 172.21.15.12:0/1840547351 wait complete. 2024-04-22T13:28:33.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.284+0000 7fae4c042740 20 client.17271 trim_cache size 0 max 16384 2024-04-22T13:28:33.287 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:28:33.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.284+0000 7fae4944e640 1 -- 172.21.15.12:0/3768703107 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fae300a3d20 msgr2=0x7fae300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:33.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.284+0000 7fae4944e640 1 --2- 172.21.15.12:0/3768703107 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fae300a3d20 0x7fae300a61e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fae3404c000 tx=0x7fae34045000 comp rx=0 tx=0).stop 2024-04-22T13:28:33.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.284+0000 7fae4944e640 1 -- 172.21.15.12:0/3768703107 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae44103af0 msgr2=0x7fae44198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:33.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.284+0000 7fae4944e640 1 --2- 172.21.15.12:0/3768703107 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae44103af0 0x7fae44198c20 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00d770 tx=0x7fae2c00dc30 comp rx=0 tx=0).stop 2024-04-22T13:28:33.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.284+0000 7fae4944e640 1 -- 172.21.15.12:0/3768703107 shutdown_connections 2024-04-22T13:28:33.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.284+0000 7fae4944e640 1 --2- 172.21.15.12:0/3768703107 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fae300a3d20 0x7fae300a61e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.284+0000 7fae4944e640 1 --2- 172.21.15.12:0/3768703107 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae4410c510 0x7fae4419cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.284+0000 7fae4944e640 1 --2- 172.21.15.12:0/3768703107 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fae44103af0 0x7fae44198c20 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.284+0000 7fae4944e640 1 --2- 172.21.15.12:0/3768703107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae441031d0 0x7fae441986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.284+0000 7fae4944e640 1 -- 172.21.15.12:0/3768703107 >> 172.21.15.12:0/3768703107 conn(0x7fae44077f50 msgr2=0x7fae440fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:33.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.284+0000 7fae4944e640 1 -- 172.21.15.12:0/3768703107 shutdown_connections 2024-04-22T13:28:33.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.284+0000 7fae4944e640 1 -- 172.21.15.12:0/3768703107 wait complete. 2024-04-22T13:28:33.292 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:28:33.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.399+0000 7fd95adb6640 1 -- 172.21.15.12:0/2309743703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd95406a840 msgr2=0x7fd95406ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:33.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.399+0000 7fd95adb6640 1 --2- 172.21.15.12:0/2309743703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd95406a840 0x7fd95406ac20 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fd950009900 tx=0x7fd95002bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:33.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.399+0000 7fd95adb6640 1 -- 172.21.15.12:0/2309743703 shutdown_connections 2024-04-22T13:28:33.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.399+0000 7fd95adb6640 1 --2- 172.21.15.12:0/2309743703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd954107270 0x7fd95410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.399+0000 7fd95adb6640 1 --2- 172.21.15.12:0/2309743703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd95406b1f0 0x7fd954106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.399+0000 7fd95adb6640 1 --2- 172.21.15.12:0/2309743703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd95406a840 0x7fd95406ac20 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.399+0000 7fd95adb6640 1 -- 172.21.15.12:0/2309743703 >> 172.21.15.12:0/2309743703 conn(0x7fd954077eb0 msgr2=0x7fd9540782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:33.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.399+0000 7fd95adb6640 1 -- 172.21.15.12:0/2309743703 shutdown_connections 2024-04-22T13:28:33.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.400+0000 7fd95adb6640 1 -- 172.21.15.12:0/2309743703 wait complete. 2024-04-22T13:28:33.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.400+0000 7fd95adb6640 1 Processor -- start 2024-04-22T13:28:33.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.400+0000 7fd95adb6640 1 -- start start 2024-04-22T13:28:33.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.400+0000 7fd95adb6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd95406a840 0x7fd9541986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.400+0000 7fd95adb6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd954107270 0x7fd954198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.400+0000 7fd95adb6640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd95419cf70 0x7fd95419d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.400+0000 7fd95adb6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd9541121d0 con 0x7fd954107270 2024-04-22T13:28:33.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.400+0000 7fd95adb6640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd954112050 con 0x7fd95419cf70 2024-04-22T13:28:33.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.400+0000 7fd95adb6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd954112350 con 0x7fd95406a840 2024-04-22T13:28:33.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.400+0000 7fd94bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd954107270 0x7fd954198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.400+0000 7fd94bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd954107270 0x7fd954198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32858/0 (socket says 172.21.15.12:32858) 2024-04-22T13:28:33.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.400+0000 7fd94bfff640 1 -- 172.21.15.12:0/1969294619 learned_addr learned my addr 172.21.15.12:0/1969294619 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:33.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.401+0000 7fd959353640 1 --2- 172.21.15.12:0/1969294619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd95419cf70 0x7fd95419d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.401+0000 7fd958b52640 1 --2- 172.21.15.12:0/1969294619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd95406a840 0x7fd9541986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.401+0000 7fd94bfff640 1 -- 172.21.15.12:0/1969294619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd95406a840 msgr2=0x7fd9541986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:33.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.401+0000 7fd94bfff640 1 --2- 172.21.15.12:0/1969294619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd95406a840 0x7fd9541986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.401+0000 7fd94bfff640 1 -- 172.21.15.12:0/1969294619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd95419cf70 msgr2=0x7fd95419d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:33.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.401+0000 7fd94bfff640 1 --2- 172.21.15.12:0/1969294619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd95419cf70 0x7fd95419d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.401+0000 7fd94bfff640 1 -- 172.21.15.12:0/1969294619 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd95419da60 con 0x7fd954107270 2024-04-22T13:28:33.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.401+0000 7fd94bfff640 1 --2- 172.21.15.12:0/1969294619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd954107270 0x7fd954198bf0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fd94400e9f0 tx=0x7fd94400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:33.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.401+0000 7fd949ffb640 1 -- 172.21.15.12:0/1969294619 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94400ccc0 con 0x7fd954107270 2024-04-22T13:28:33.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.401+0000 7fd949ffb640 1 -- 172.21.15.12:0/1969294619 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9440054d0 con 0x7fd954107270 2024-04-22T13:28:33.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.401+0000 7fd95adb6640 1 -- 172.21.15.12:0/1969294619 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd95419dd50 con 0x7fd954107270 2024-04-22T13:28:33.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.401+0000 7fd949ffb640 1 -- 172.21.15.12:0/1969294619 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd944010430 con 0x7fd954107270 2024-04-22T13:28:33.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.401+0000 7fd95adb6640 1 -- 172.21.15.12:0/1969294619 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9541a54f0 con 0x7fd954107270 2024-04-22T13:28:33.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.403+0000 7fd949ffb640 1 -- 172.21.15.12:0/1969294619 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd9440046c0 con 0x7fd954107270 2024-04-22T13:28:33.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.404+0000 7fd95d9ad740 1 Processor -- start 2024-04-22T13:28:33.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.404+0000 7fd95d9ad740 1 -- start start 2024-04-22T13:28:33.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.404+0000 7fd949ffb640 1 --2- 172.21.15.12:0/1969294619 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd93c0a3d20 0x7fd93c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.404+0000 7fd95d9ad740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56162adf24d0 0x56162adf28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.404+0000 7fd95d9ad740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56162adf2e80 0x56162adfb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.404+0000 7fd95d9ad740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162adfba00 0x56162adfde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.404+0000 7fd95d9ad740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56162ad571f0 con 0x56162adf2e80 2024-04-22T13:28:33.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.404+0000 7fd95d9ad740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56162acf44b0 con 0x56162adfba00 2024-04-22T13:28:33.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.404+0000 7fd95d9ad740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56162ad531b0 con 0x56162adf24d0 2024-04-22T13:28:33.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.404+0000 7fd94bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56162adf2e80 0x56162adfb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.404+0000 7fd959353640 1 --2- 172.21.15.12:0/1969294619 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd93c0a3d20 0x7fd93c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.404+0000 7fd949ffb640 1 -- 172.21.15.12:0/1969294619 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fd9440d0bb0 con 0x7fd954107270 2024-04-22T13:28:33.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.404+0000 7fd94bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56162adf2e80 0x56162adfb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32862/0 (socket says 172.21.15.12:32862) 2024-04-22T13:28:33.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.404+0000 7fd94bfff640 1 -- 172.21.15.12:0/3785857063 learned_addr learned my addr 172.21.15.12:0/3785857063 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:33.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.404+0000 7fd958b52640 1 --2- 172.21.15.12:0/3785857063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56162adf24d0 0x56162adf28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.405+0000 7fd94bfff640 1 -- 172.21.15.12:0/3785857063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56162adf24d0 msgr2=0x56162adf28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:33.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.405+0000 7fd94bfff640 1 --2- 172.21.15.12:0/3785857063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56162adf24d0 0x56162adf28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.405+0000 7fd94bfff640 1 -- 172.21.15.12:0/3785857063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162adfba00 msgr2=0x56162adfde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:33.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.405+0000 7fd94bfff640 1 --2- 172.21.15.12:0/3785857063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162adfba00 0x56162adfde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.405+0000 7fd94bfff640 1 -- 172.21.15.12:0/3785857063 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56162ace61b0 con 0x56162adf2e80 2024-04-22T13:28:33.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.405+0000 7fd959353640 1 --2- 172.21.15.12:0/1969294619 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd93c0a3d20 0x7fd93c0a61e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd94c007ee0 tx=0x7fd94c007940 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:33.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.405+0000 7fd94bfff640 1 --2- 172.21.15.12:0/3785857063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56162adf2e80 0x56162adfb430 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fd944099570 tx=0x7fd94401c590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea133a9f809252d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:33.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.405+0000 7fd93affd640 1 -- 172.21.15.12:0/3785857063 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd944005680 con 0x56162adf2e80 2024-04-22T13:28:33.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.405+0000 7fd93affd640 1 -- 172.21.15.12:0/3785857063 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd94409d2a0 con 0x56162adf2e80 2024-04-22T13:28:33.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.405+0000 7fd93affd640 1 -- 172.21.15.12:0/3785857063 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd944099210 con 0x56162adf2e80 2024-04-22T13:28:33.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.406+0000 7fd95d9ad740 1 -- 172.21.15.12:0/3785857063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56162adf2e80 msgr2=0x56162adfb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:33.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.406+0000 7fd95d9ad740 1 --2- 172.21.15.12:0/3785857063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56162adf2e80 0x56162adfb430 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fd944099570 tx=0x7fd94401c590 comp rx=0 tx=0).stop 2024-04-22T13:28:33.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.406+0000 7fd95d9ad740 1 -- 172.21.15.12:0/3785857063 shutdown_connections 2024-04-22T13:28:33.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.406+0000 7fd95d9ad740 1 --2- 172.21.15.12:0/3785857063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162adfba00 0x56162adfde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.406+0000 7fd95d9ad740 1 --2- 172.21.15.12:0/3785857063 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56162adf2e80 0x56162adfb430 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.406+0000 7fd95d9ad740 1 --2- 172.21.15.12:0/3785857063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56162adf24d0 0x56162adf28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.406+0000 7fd95d9ad740 1 -- 172.21.15.12:0/3785857063 >> 172.21.15.12:0/3785857063 conn(0x56162adecba0 msgr2=0x56162adecf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:33.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.406+0000 7fd95d9ad740 1 -- 172.21.15.12:0/3785857063 shutdown_connections 2024-04-22T13:28:33.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.406+0000 7fd95d9ad740 1 -- 172.21.15.12:0/3785857063 wait complete. 2024-04-22T13:28:33.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.406+0000 7fd95d9ad740 1 -- start start 2024-04-22T13:28:33.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.406+0000 7fd95d9ad740 1 Processor -- start 2024-04-22T13:28:33.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.406+0000 7fd95d9ad740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56162adf24d0 0x56162adfb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.406+0000 7fd95d9ad740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56162adf2e80 0x56162adf86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.406+0000 7fd95d9ad740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162adfba00 0x56162adf8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.406+0000 7fd95d9ad740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56162adfe970 con 0x56162adf24d0 2024-04-22T13:28:33.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.406+0000 7fd95d9ad740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56162ad10500 con 0x56162adfba00 2024-04-22T13:28:33.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.406+0000 7fd95d9ad740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56162adfeaf0 con 0x56162adf2e80 2024-04-22T13:28:33.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.406+0000 7fd95d9ad740 10 client.0 _ll_register_callbacks cb 0x56162acdeb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:33.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.407+0000 7fd958b52640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56162adf24d0 0x56162adfb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.407+0000 7fd958b52640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56162adf24d0 0x56162adfb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32872/0 (socket says 172.21.15.12:32872) 2024-04-22T13:28:33.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.407+0000 7fd958b52640 1 -- 172.21.15.12:0/1300604665 learned_addr learned my addr 172.21.15.12:0/1300604665 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:33.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.407+0000 7fd959353640 1 --2- 172.21.15.12:0/1300604665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162adfba00 0x56162adf8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.407+0000 7fd94bfff640 1 --2- 172.21.15.12:0/1300604665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56162adf2e80 0x56162adf86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.407+0000 7fd958b52640 1 -- 172.21.15.12:0/1300604665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56162adf2e80 msgr2=0x56162adf86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:33.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.407+0000 7fd958b52640 1 --2- 172.21.15.12:0/1300604665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56162adf2e80 0x56162adf86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.407+0000 7fd958b52640 1 -- 172.21.15.12:0/1300604665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162adfba00 msgr2=0x56162adf8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:33.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.407+0000 7fd958b52640 1 --2- 172.21.15.12:0/1300604665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162adfba00 0x56162adf8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.407+0000 7fd958b52640 1 -- 172.21.15.12:0/1300604665 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56162aa693f0 con 0x56162adf24d0 2024-04-22T13:28:33.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.407+0000 7fd959353640 1 --2- 172.21.15.12:0/1300604665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162adfba00 0x56162adf8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:33.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.407+0000 7fd958b52640 1 --2- 172.21.15.12:0/1300604665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56162adf24d0 0x56162adfb310 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fd95002c170 tx=0x7fd95002ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d791234d950b1cb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:33.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.407+0000 7fd933fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:33.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.407+0000 7fd933fff640 1 -- 172.21.15.12:0/1300604665 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd950005d50 con 0x56162adf24d0 2024-04-22T13:28:33.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.407+0000 7fd933fff640 1 -- 172.21.15.12:0/1300604665 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fd950005450 con 0x56162adf24d0 2024-04-22T13:28:33.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.407+0000 7fd95d9ad740 1 -- 172.21.15.12:0/1300604665 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd94409d2a0 con 0x56162adf24d0 2024-04-22T13:28:33.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.407+0000 7fd933fff640 1 -- 172.21.15.12:0/1300604665 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd95002cc80 con 0x56162adf24d0 2024-04-22T13:28:33.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.407+0000 7fd95d9ad740 10 client.17283 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:33.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.407+0000 7fd95d9ad740 1 -- 172.21.15.12:0/1300604665 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56162ab337d0 con 0x56162adf24d0 2024-04-22T13:28:33.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.408+0000 7fd933fff640 1 -- 172.21.15.12:0/1300604665 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd94409d2a0 con 0x56162adf24d0 2024-04-22T13:28:33.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.408+0000 7fd933fff640 1 -- 172.21.15.12:0/1300604665 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd9500036d0 con 0x56162adf24d0 2024-04-22T13:28:33.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.408+0000 7fd95d9ad740 10 client.17283 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:33.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.408+0000 7fd95d9ad740 10 client.17283 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:33.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.408+0000 7fd95d9ad740 20 client.17283 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:33.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.408+0000 7fd95d9ad740 1 --2- 172.21.15.12:0/1300604665 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56162ae0bb60 0x56162ae0df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.408+0000 7fd95d9ad740 4 client.17283 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:33.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.408+0000 7fd95d9ad740 1 -- 172.21.15.12:0/1300604665 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56162acf6930 con 0x56162ae0bb60 2024-04-22T13:28:33.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.408+0000 7fd94bfff640 1 --2- 172.21.15.12:0/1300604665 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56162ae0bb60 0x56162ae0df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.409+0000 7fd933fff640 10 client.17283 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:33.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.409+0000 7fd94bfff640 1 --2- 172.21.15.12:0/1300604665 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56162ae0bb60 0x56162ae0df90 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d284e167d9c4c87e server_cookie=960263fb15dcbcf8 in_seq=0 out_seq=0 2024-04-22T13:28:33.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.409+0000 7fd933fff640 1 -- 172.21.15.12:0/1300604665 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56162acf6930 con 0x56162ae0bb60 2024-04-22T13:28:33.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.409+0000 7fd933fff640 10 client.17283 handle_command_reply: tid=0 2024-04-22T13:28:33.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.409+0000 7fd95d9ad740 1 client.17283 shutdown 2024-04-22T13:28:33.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.409+0000 7fd95d9ad740 1 -- 172.21.15.12:0/1300604665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56162adf24d0 msgr2=0x56162adfb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:33.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.409+0000 7fd95d9ad740 1 --2- 172.21.15.12:0/1300604665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56162adf24d0 0x56162adfb310 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fd95002c170 tx=0x7fd95002ca90 comp rx=0 tx=0).stop 2024-04-22T13:28:33.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.409+0000 7fd95d9ad740 1 -- 172.21.15.12:0/1300604665 shutdown_connections 2024-04-22T13:28:33.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.409+0000 7fd95d9ad740 1 --2- 172.21.15.12:0/1300604665 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56162ae0bb60 0x56162ae0df90 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.409+0000 7fd95d9ad740 1 --2- 172.21.15.12:0/1300604665 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162adfba00 0x56162adf8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.409+0000 7fd95d9ad740 1 --2- 172.21.15.12:0/1300604665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56162adf2e80 0x56162adf86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.409+0000 7fd933fff640 0 client.17283 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:33.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.409+0000 7fd95d9ad740 1 --2- 172.21.15.12:0/1300604665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56162adf24d0 0x56162adfb310 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.409+0000 7fd95d9ad740 1 -- 172.21.15.12:0/1300604665 >> 172.21.15.12:0/1300604665 conn(0x56162adecba0 msgr2=0x56162adee8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:33.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.410+0000 7fd95d9ad740 1 -- 172.21.15.12:0/1300604665 shutdown_connections 2024-04-22T13:28:33.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.410+0000 7fd95d9ad740 1 -- 172.21.15.12:0/1300604665 wait complete. 2024-04-22T13:28:33.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.410+0000 7fd95d9ad740 20 client.17283 trim_cache size 0 max 16384 2024-04-22T13:28:33.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.423+0000 7fd95d9ad740 1 Processor -- start 2024-04-22T13:28:33.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.423+0000 7fd95d9ad740 1 -- start start 2024-04-22T13:28:33.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.423+0000 7fd95d9ad740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162adf24d0 0x56162ae07830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.423+0000 7fd95d9ad740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56162adf2e80 0x56162adfadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.423+0000 7fd95d9ad740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56162adfba00 0x56162adfb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.423+0000 7fd95d9ad740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd9541ad9b0 con 0x56162adfba00 2024-04-22T13:28:33.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.423+0000 7fd95d9ad740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd9541af4f0 con 0x56162adf24d0 2024-04-22T13:28:33.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.423+0000 7fd95d9ad740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd9541ae080 con 0x56162adf2e80 2024-04-22T13:28:33.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.423+0000 7fd959353640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56162adfba00 0x56162adfb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.423+0000 7fd959353640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56162adfba00 0x56162adfb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32888/0 (socket says 172.21.15.12:32888) 2024-04-22T13:28:33.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.423+0000 7fd959353640 1 -- 172.21.15.12:0/884884227 learned_addr learned my addr 172.21.15.12:0/884884227 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:33.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.424+0000 7fd94bfff640 1 --2- 172.21.15.12:0/884884227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56162adf2e80 0x56162adfadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.424+0000 7fd959353640 1 -- 172.21.15.12:0/884884227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56162adf2e80 msgr2=0x56162adfadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:33.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.424+0000 7fd959353640 1 --2- 172.21.15.12:0/884884227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56162adf2e80 0x56162adfadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.424+0000 7fd958b52640 1 --2- 172.21.15.12:0/884884227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162adf24d0 0x56162ae07830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.424+0000 7fd959353640 1 -- 172.21.15.12:0/884884227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162adf24d0 msgr2=0x56162ae07830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:33.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.424+0000 7fd959353640 1 --2- 172.21.15.12:0/884884227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162adf24d0 0x56162ae07830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.424+0000 7fd959353640 1 -- 172.21.15.12:0/884884227 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56162adf2c20 con 0x56162adfba00 2024-04-22T13:28:33.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.424+0000 7fd959353640 1 --2- 172.21.15.12:0/884884227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56162adfba00 0x56162adfb310 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd94c012690 tx=0x7fd94c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d19a2a9e7a758d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:33.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.424+0000 7fd93a7fc640 1 -- 172.21.15.12:0/884884227 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94c00e980 con 0x56162adfba00 2024-04-22T13:28:33.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.424+0000 7fd93a7fc640 1 -- 172.21.15.12:0/884884227 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd94c016070 con 0x56162adfba00 2024-04-22T13:28:33.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.424+0000 7fd93a7fc640 1 -- 172.21.15.12:0/884884227 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94c01f3f0 con 0x56162adfba00 2024-04-22T13:28:33.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.425+0000 7fd95d9ad740 1 -- 172.21.15.12:0/884884227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56162adfba00 msgr2=0x56162adfb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:33.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.425+0000 7fd95d9ad740 1 --2- 172.21.15.12:0/884884227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56162adfba00 0x56162adfb310 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd94c012690 tx=0x7fd94c014000 comp rx=0 tx=0).stop 2024-04-22T13:28:33.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.425+0000 7fd95d9ad740 1 -- 172.21.15.12:0/884884227 shutdown_connections 2024-04-22T13:28:33.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.425+0000 7fd95d9ad740 1 --2- 172.21.15.12:0/884884227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56162adfba00 0x56162adfb310 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.425+0000 7fd95d9ad740 1 --2- 172.21.15.12:0/884884227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56162adf2e80 0x56162adfadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.425+0000 7fd95d9ad740 1 --2- 172.21.15.12:0/884884227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162adf24d0 0x56162ae07830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.425+0000 7fd95d9ad740 1 -- 172.21.15.12:0/884884227 >> 172.21.15.12:0/884884227 conn(0x56162adecba0 msgr2=0x56162adebc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:33.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.425+0000 7fd95d9ad740 1 -- 172.21.15.12:0/884884227 shutdown_connections 2024-04-22T13:28:33.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.425+0000 7fd95d9ad740 1 -- 172.21.15.12:0/884884227 wait complete. 2024-04-22T13:28:33.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.425+0000 7fd95d9ad740 1 -- start start 2024-04-22T13:28:33.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.425+0000 7fd95d9ad740 1 Processor -- start 2024-04-22T13:28:33.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.425+0000 7fd95d9ad740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162adf24d0 0x56162ae04270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.425+0000 7fd95d9ad740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56162adf2e80 0x56162adff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.425+0000 7fd95d9ad740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56162adfba00 0x56162adffd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.426+0000 7fd95d9ad740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd9541aee20 con 0x56162adf2e80 2024-04-22T13:28:33.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.426+0000 7fd95d9ad740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd9541ae750 con 0x56162adf24d0 2024-04-22T13:28:33.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.426+0000 7fd95d9ad740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd9541ab9b0 con 0x56162adfba00 2024-04-22T13:28:33.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.426+0000 7fd95d9ad740 10 client.0 _ll_register_callbacks cb 0x7fd950004490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:33.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.426+0000 7fd94bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56162adf2e80 0x56162adff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.426+0000 7fd94bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56162adf2e80 0x56162adff5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32892/0 (socket says 172.21.15.12:32892) 2024-04-22T13:28:33.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.426+0000 7fd94bfff640 1 -- 172.21.15.12:0/1866514223 learned_addr learned my addr 172.21.15.12:0/1866514223 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:33.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.426+0000 7fd958b52640 1 --2- 172.21.15.12:0/1866514223 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162adf24d0 0x56162ae04270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.426+0000 7fd959353640 1 --2- 172.21.15.12:0/1866514223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56162adfba00 0x56162adffd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.426+0000 7fd94bfff640 1 -- 172.21.15.12:0/1866514223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56162adfba00 msgr2=0x56162adffd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:33.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.426+0000 7fd94bfff640 1 --2- 172.21.15.12:0/1866514223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56162adfba00 0x56162adffd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.426+0000 7fd94bfff640 1 -- 172.21.15.12:0/1866514223 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162adf24d0 msgr2=0x56162ae04270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.426+0000 7fd94bfff640 1 --2- 172.21.15.12:0/1866514223 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162adf24d0 0x56162ae04270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.426+0000 7fd94bfff640 1 -- 172.21.15.12:0/1866514223 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56162ae710d0 con 0x56162adf2e80 2024-04-22T13:28:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.426+0000 7fd959353640 1 --2- 172.21.15.12:0/1866514223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56162adfba00 0x56162adffd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.426+0000 7fd94bfff640 1 --2- 172.21.15.12:0/1866514223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56162adf2e80 0x56162adff5e0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fd9440ce400 tx=0x7fd9440bbf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26d467f9eacf747 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.426+0000 7fd933fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.426+0000 7fd933fff640 1 -- 172.21.15.12:0/1866514223 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd944005680 con 0x56162adf2e80 2024-04-22T13:28:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.426+0000 7fd933fff640 1 -- 172.21.15.12:0/1866514223 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fd944099210 con 0x56162adf2e80 2024-04-22T13:28:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.426+0000 7fd95d9ad740 1 -- 172.21.15.12:0/1866514223 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd94c016070 con 0x56162adf2e80 2024-04-22T13:28:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.426+0000 7fd933fff640 1 -- 172.21.15.12:0/1866514223 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd9440b6120 con 0x56162adf2e80 2024-04-22T13:28:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.426+0000 7fd95d9ad740 10 client.17289 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.426+0000 7fd95d9ad740 1 -- 172.21.15.12:0/1866514223 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56162ae722b0 con 0x56162adf2e80 2024-04-22T13:28:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.426+0000 7fd933fff640 1 -- 172.21.15.12:0/1866514223 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd94c016070 con 0x56162adf2e80 2024-04-22T13:28:33.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.427+0000 7fd933fff640 1 -- 172.21.15.12:0/1866514223 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd9440ce5d0 con 0x56162adf2e80 2024-04-22T13:28:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.427+0000 7fd95d9ad740 10 client.17289 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.427+0000 7fd95d9ad740 10 client.17289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.427+0000 7fd95d9ad740 20 client.17289 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.427+0000 7fd95d9ad740 1 --2- 172.21.15.12:0/1866514223 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56162ae725f0 0x56162ae749e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.427+0000 7fd95d9ad740 4 client.17289 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:28:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.427+0000 7fd95d9ad740 1 -- 172.21.15.12:0/1866514223 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56162ad64c10 con 0x56162ae725f0 2024-04-22T13:28:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.427+0000 7fd958b52640 1 --2- 172.21.15.12:0/1866514223 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56162ae725f0 0x56162ae749e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.427+0000 7fd958b52640 1 --2- 172.21.15.12:0/1866514223 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56162ae725f0 0x56162ae749e0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa4bfd769cada3a0 server_cookie=5f7c4db3b5b2fa93 in_seq=0 out_seq=0 2024-04-22T13:28:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.427+0000 7fd933fff640 10 client.17289 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.427+0000 7fd933fff640 1 -- 172.21.15.12:0/1866514223 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56162ad64c10 con 0x56162ae725f0 2024-04-22T13:28:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.427+0000 7fd933fff640 10 client.17289 handle_command_reply: tid=0 2024-04-22T13:28:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.427+0000 7fd95d9ad740 1 client.17289 shutdown 2024-04-22T13:28:33.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.428+0000 7fd95d9ad740 1 -- 172.21.15.12:0/1866514223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56162adf2e80 msgr2=0x56162adff5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:33.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.428+0000 7fd95d9ad740 1 --2- 172.21.15.12:0/1866514223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56162adf2e80 0x56162adff5e0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fd9440ce400 tx=0x7fd9440bbf80 comp rx=0 tx=0).stop 2024-04-22T13:28:33.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.428+0000 7fd95d9ad740 1 -- 172.21.15.12:0/1866514223 shutdown_connections 2024-04-22T13:28:33.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.428+0000 7fd95d9ad740 1 --2- 172.21.15.12:0/1866514223 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56162ae725f0 0x56162ae749e0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.428+0000 7fd95d9ad740 1 --2- 172.21.15.12:0/1866514223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56162adfba00 0x56162adffd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.428+0000 7fd95d9ad740 1 --2- 172.21.15.12:0/1866514223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56162adf2e80 0x56162adff5e0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.428+0000 7fd933fff640 0 client.17289 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:33.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.428+0000 7fd95d9ad740 1 --2- 172.21.15.12:0/1866514223 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56162adf24d0 0x56162ae04270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.428+0000 7fd95d9ad740 1 -- 172.21.15.12:0/1866514223 >> 172.21.15.12:0/1866514223 conn(0x56162adecba0 msgr2=0x56162adf3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:33.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.428+0000 7fd95d9ad740 1 -- 172.21.15.12:0/1866514223 shutdown_connections 2024-04-22T13:28:33.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.428+0000 7fd95d9ad740 1 -- 172.21.15.12:0/1866514223 wait complete. 2024-04-22T13:28:33.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.428+0000 7fd95d9ad740 20 client.17289 trim_cache size 0 max 16384 2024-04-22T13:28:33.432 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:28:33.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.428+0000 7fd95adb6640 1 -- 172.21.15.12:0/1969294619 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd93c0a3d20 msgr2=0x7fd93c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:33.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.428+0000 7fd95adb6640 1 --2- 172.21.15.12:0/1969294619 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd93c0a3d20 0x7fd93c0a61e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd94c007ee0 tx=0x7fd94c007940 comp rx=0 tx=0).stop 2024-04-22T13:28:33.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.428+0000 7fd95adb6640 1 -- 172.21.15.12:0/1969294619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd954107270 msgr2=0x7fd954198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:33.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.428+0000 7fd95adb6640 1 --2- 172.21.15.12:0/1969294619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd954107270 0x7fd954198bf0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fd94400e9f0 tx=0x7fd94400eeb0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.428+0000 7fd95adb6640 1 -- 172.21.15.12:0/1969294619 shutdown_connections 2024-04-22T13:28:33.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.428+0000 7fd95adb6640 1 --2- 172.21.15.12:0/1969294619 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd93c0a3d20 0x7fd93c0a61e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.428+0000 7fd95adb6640 1 --2- 172.21.15.12:0/1969294619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd95419cf70 0x7fd95419d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.428+0000 7fd95adb6640 1 --2- 172.21.15.12:0/1969294619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd954107270 0x7fd954198bf0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.428+0000 7fd95adb6640 1 --2- 172.21.15.12:0/1969294619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd95406a840 0x7fd9541986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:33.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.428+0000 7fd95adb6640 1 -- 172.21.15.12:0/1969294619 >> 172.21.15.12:0/1969294619 conn(0x7fd954077eb0 msgr2=0x7fd954100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:33.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.429+0000 7fd95adb6640 1 -- 172.21.15.12:0/1969294619 shutdown_connections 2024-04-22T13:28:33.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:33.429+0000 7fd95adb6640 1 -- 172.21.15.12:0/1969294619 wait complete. 2024-04-22T13:28:33.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:33 smithi134 ceph-mon[31338]: pgmap v136: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:28:33.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:33 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/3996174335' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:28:33.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:33 smithi111 ceph-mon[31361]: pgmap v136: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:28:33.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:33 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/3996174335' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:28:34.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:33 smithi012 ceph-mon[28265]: pgmap v136: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:28:34.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:33 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3996174335' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:28:34.436 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:28:34.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:28:34.437 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:28:34.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.560+0000 7fab2b474640 1 -- 172.21.15.12:0/720114730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab24106e80 msgr2=0x7fab2406a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:34.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.560+0000 7fab2b474640 1 --2- 172.21.15.12:0/720114730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab24106e80 0x7fab2406a5d0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fab20009900 tx=0x7fab2002bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:34.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.560+0000 7fab2b474640 1 -- 172.21.15.12:0/720114730 shutdown_connections 2024-04-22T13:28:34.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.560+0000 7fab2b474640 1 --2- 172.21.15.12:0/720114730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab24104980 0x7fab24104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.560+0000 7fab2b474640 1 --2- 172.21.15.12:0/720114730 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab2406ab10 0x7fab2406af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.560+0000 7fab2b474640 1 --2- 172.21.15.12:0/720114730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab24106e80 0x7fab2406a5d0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.560+0000 7fab2b474640 1 -- 172.21.15.12:0/720114730 >> 172.21.15.12:0/720114730 conn(0x7fab24077eb0 msgr2=0x7fab240782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:34.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.561+0000 7fab2b474640 1 -- 172.21.15.12:0/720114730 shutdown_connections 2024-04-22T13:28:34.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.561+0000 7fab2b474640 1 -- 172.21.15.12:0/720114730 wait complete. 2024-04-22T13:28:34.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.561+0000 7fab2b474640 1 Processor -- start 2024-04-22T13:28:34.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.561+0000 7fab2b474640 1 -- start start 2024-04-22T13:28:34.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.561+0000 7fab2b474640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab2406ab10 0x7fab2419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.561+0000 7fab2b474640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab24104980 0x7fab2419f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.561+0000 7fab2b474640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab24106e80 0x7fab241a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.561+0000 7fab2b474640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fab241187a0 con 0x7fab24104980 2024-04-22T13:28:34.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.561+0000 7fab2b474640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fab24118620 con 0x7fab24106e80 2024-04-22T13:28:34.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.561+0000 7fab2b474640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fab24118920 con 0x7fab2406ab10 2024-04-22T13:28:34.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.562+0000 7fab28a0f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab24104980 0x7fab2419f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.562+0000 7fab28a0f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab24104980 0x7fab2419f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32910/0 (socket says 172.21.15.12:32910) 2024-04-22T13:28:34.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.562+0000 7fab28a0f640 1 -- 172.21.15.12:0/3771346735 learned_addr learned my addr 172.21.15.12:0/3771346735 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:34.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.562+0000 7fab29a11640 1 --2- 172.21.15.12:0/3771346735 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab24106e80 0x7fab241a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.562+0000 7fab29210640 1 --2- 172.21.15.12:0/3771346735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab2406ab10 0x7fab2419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.562+0000 7fab28a0f640 1 -- 172.21.15.12:0/3771346735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab2406ab10 msgr2=0x7fab2419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:34.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.562+0000 7fab28a0f640 1 --2- 172.21.15.12:0/3771346735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab2406ab10 0x7fab2419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.562+0000 7fab28a0f640 1 -- 172.21.15.12:0/3771346735 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab24106e80 msgr2=0x7fab241a3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:34.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.562+0000 7fab28a0f640 1 --2- 172.21.15.12:0/3771346735 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab24106e80 0x7fab241a3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.562+0000 7fab28a0f640 1 -- 172.21.15.12:0/3771346735 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab241a3d30 con 0x7fab24104980 2024-04-22T13:28:34.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.562+0000 7fab29a11640 1 --2- 172.21.15.12:0/3771346735 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab24106e80 0x7fab241a3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:34.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.562+0000 7fab28a0f640 1 --2- 172.21.15.12:0/3771346735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab24104980 0x7fab2419f280 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fab1400d950 tx=0x7fab1400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:34.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.562+0000 7fab1a7fc640 1 -- 172.21.15.12:0/3771346735 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14017070 con 0x7fab24104980 2024-04-22T13:28:34.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.562+0000 7fab2b474640 1 -- 172.21.15.12:0/3771346735 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab241a4020 con 0x7fab24104980 2024-04-22T13:28:34.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.562+0000 7fab1a7fc640 1 -- 172.21.15.12:0/3771346735 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab14009c30 con 0x7fab24104980 2024-04-22T13:28:34.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.562+0000 7fab1a7fc640 1 -- 172.21.15.12:0/3771346735 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1400f620 con 0x7fab24104980 2024-04-22T13:28:34.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.562+0000 7fab2b474640 1 -- 172.21.15.12:0/3771346735 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab241ab8b0 con 0x7fab24104980 2024-04-22T13:28:34.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.564+0000 7fab1a7fc640 1 -- 172.21.15.12:0/3771346735 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fab14010460 con 0x7fab24104980 2024-04-22T13:28:34.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.565+0000 7fab2e06e740 1 Processor -- start 2024-04-22T13:28:34.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.565+0000 7fab2e06e740 1 -- start start 2024-04-22T13:28:34.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.565+0000 7fab2e06e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da4d5da8e0 0x55da4d5dacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.565+0000 7fab2e06e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da4d5db290 0x55da4d5e3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.566+0000 7fab2e06e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da4d5e3e10 0x55da4d5e6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.566+0000 7fab2e06e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55da4d53ef00 con 0x55da4d5e3e10 2024-04-22T13:28:34.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.566+0000 7fab2e06e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55da4d4dc590 con 0x55da4d5db290 2024-04-22T13:28:34.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.566+0000 7fab2e06e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55da4d53aec0 con 0x55da4d5da8e0 2024-04-22T13:28:34.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.566+0000 7fab1a7fc640 1 --2- 172.21.15.12:0/3771346735 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fab100a3d20 0x7fab100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.566+0000 7fab28a0f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da4d5e3e10 0x55da4d5e6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.566+0000 7fab29a11640 1 --2- 172.21.15.12:0/3771346735 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fab100a3d20 0x7fab100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.566+0000 7fab28a0f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da4d5e3e10 0x55da4d5e6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32914/0 (socket says 172.21.15.12:32914) 2024-04-22T13:28:34.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.566+0000 7fab28a0f640 1 -- 172.21.15.12:0/1393891814 learned_addr learned my addr 172.21.15.12:0/1393891814 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:34.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.566+0000 7fab29210640 1 --2- 172.21.15.12:0/1393891814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da4d5db290 0x55da4d5e3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.566+0000 7fab29a11640 1 --2- 172.21.15.12:0/1393891814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da4d5da8e0 0x55da4d5dacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.566+0000 7fab1a7fc640 1 -- 172.21.15.12:0/3771346735 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fab140d0520 con 0x7fab24104980 2024-04-22T13:28:34.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.566+0000 7fab28a0f640 1 -- 172.21.15.12:0/1393891814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da4d5da8e0 msgr2=0x55da4d5dacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:34.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.566+0000 7fab28a0f640 1 --2- 172.21.15.12:0/1393891814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da4d5da8e0 0x55da4d5dacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.566+0000 7fab28a0f640 1 -- 172.21.15.12:0/1393891814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da4d5db290 msgr2=0x55da4d5e3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:34.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.566+0000 7fab28a0f640 1 --2- 172.21.15.12:0/1393891814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da4d5db290 0x55da4d5e3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.566+0000 7fab28a0f640 1 -- 172.21.15.12:0/1393891814 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da4d3197e0 con 0x55da4d5e3e10 2024-04-22T13:28:34.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.566+0000 7fab29a11640 1 --2- 172.21.15.12:0/3771346735 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fab100a3d20 0x7fab100a61e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fab1c005fd0 tx=0x7fab1c009db0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:34.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.566+0000 7fab28a0f640 1 --2- 172.21.15.12:0/1393891814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da4d5e3e10 0x55da4d5e6260 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fab1409c440 tx=0x7fab1409c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=984e9deec18d5bdc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:34.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.567+0000 7fab0f7fe640 1 -- 172.21.15.12:0/1393891814 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14020070 con 0x55da4d5e3e10 2024-04-22T13:28:34.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.567+0000 7fab0f7fe640 1 -- 172.21.15.12:0/1393891814 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab1409cbb0 con 0x55da4d5e3e10 2024-04-22T13:28:34.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.567+0000 7fab0f7fe640 1 -- 172.21.15.12:0/1393891814 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14098b20 con 0x55da4d5e3e10 2024-04-22T13:28:34.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.567+0000 7fab2e06e740 1 -- 172.21.15.12:0/1393891814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da4d5e3e10 msgr2=0x55da4d5e6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:34.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.567+0000 7fab2e06e740 1 --2- 172.21.15.12:0/1393891814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da4d5e3e10 0x55da4d5e6260 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fab1409c440 tx=0x7fab1409c0f0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.567+0000 7fab2e06e740 1 -- 172.21.15.12:0/1393891814 shutdown_connections 2024-04-22T13:28:34.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.567+0000 7fab2e06e740 1 --2- 172.21.15.12:0/1393891814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da4d5e3e10 0x55da4d5e6260 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.567+0000 7fab2e06e740 1 --2- 172.21.15.12:0/1393891814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da4d5db290 0x55da4d5e3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.567+0000 7fab2e06e740 1 --2- 172.21.15.12:0/1393891814 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da4d5da8e0 0x55da4d5dacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.567+0000 7fab2e06e740 1 -- 172.21.15.12:0/1393891814 >> 172.21.15.12:0/1393891814 conn(0x55da4d5d4fb0 msgr2=0x55da4d5d5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:34.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.567+0000 7fab2e06e740 1 -- 172.21.15.12:0/1393891814 shutdown_connections 2024-04-22T13:28:34.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.567+0000 7fab2e06e740 1 -- 172.21.15.12:0/1393891814 wait complete. 2024-04-22T13:28:34.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.568+0000 7fab2e06e740 1 -- start start 2024-04-22T13:28:34.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.568+0000 7fab2e06e740 1 Processor -- start 2024-04-22T13:28:34.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.568+0000 7fab2e06e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da4d5da8e0 0x55da4d5e0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.568+0000 7fab2e06e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da4d5db290 0x55da4d5e1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.568+0000 7fab2e06e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da4d5e3e10 0x55da4d5e1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.568+0000 7fab2e06e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55da4d5e6cc0 con 0x55da4d5da8e0 2024-04-22T13:28:34.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.568+0000 7fab2e06e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55da4d4f8580 con 0x55da4d5e3e10 2024-04-22T13:28:34.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.568+0000 7fab2e06e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55da4d5e6e40 con 0x55da4d5db290 2024-04-22T13:28:34.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.568+0000 7fab29210640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da4d5da8e0 0x55da4d5e0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.568+0000 7fab2e06e740 10 client.0 _ll_register_callbacks cb 0x55da4d4fe650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:34.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.568+0000 7fab29210640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da4d5da8e0 0x55da4d5e0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32916/0 (socket says 172.21.15.12:32916) 2024-04-22T13:28:34.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.568+0000 7fab29210640 1 -- 172.21.15.12:0/4151627217 learned_addr learned my addr 172.21.15.12:0/4151627217 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:34.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.568+0000 7fab28a0f640 1 --2- 172.21.15.12:0/4151627217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da4d5db290 0x55da4d5e1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.569+0000 7fab29a11640 1 --2- 172.21.15.12:0/4151627217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da4d5e3e10 0x55da4d5e1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.569+0000 7fab29210640 1 -- 172.21.15.12:0/4151627217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da4d5db290 msgr2=0x55da4d5e1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:34.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.569+0000 7fab29210640 1 --2- 172.21.15.12:0/4151627217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da4d5db290 0x55da4d5e1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.569+0000 7fab29210640 1 -- 172.21.15.12:0/4151627217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da4d5e3e10 msgr2=0x55da4d5e1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:34.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.569+0000 7fab29210640 1 --2- 172.21.15.12:0/4151627217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da4d5e3e10 0x55da4d5e1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.569+0000 7fab29210640 1 -- 172.21.15.12:0/4151627217 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da4d379f70 con 0x55da4d5da8e0 2024-04-22T13:28:34.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.569+0000 7fab29210640 1 --2- 172.21.15.12:0/4151627217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da4d5da8e0 0x55da4d5e0e10 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fab2002c170 tx=0x7fab2002c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48977219fb2b9a93 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:34.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.569+0000 7fab0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:34.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.569+0000 7fab0cff9640 1 -- 172.21.15.12:0/4151627217 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab20005230 con 0x55da4d5da8e0 2024-04-22T13:28:34.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.569+0000 7fab0cff9640 1 -- 172.21.15.12:0/4151627217 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fab200053d0 con 0x55da4d5da8e0 2024-04-22T13:28:34.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.569+0000 7fab2e06e740 1 -- 172.21.15.12:0/4151627217 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab1409cbb0 con 0x55da4d5da8e0 2024-04-22T13:28:34.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.569+0000 7fab0cff9640 1 -- 172.21.15.12:0/4151627217 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fab2004d020 con 0x55da4d5da8e0 2024-04-22T13:28:34.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.569+0000 7fab2e06e740 10 client.17301 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:34.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.569+0000 7fab2e06e740 1 -- 172.21.15.12:0/4151627217 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55da4d211170 con 0x55da4d5da8e0 2024-04-22T13:28:34.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.570+0000 7fab0cff9640 1 -- 172.21.15.12:0/4151627217 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab1409cbb0 con 0x55da4d5da8e0 2024-04-22T13:28:34.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.570+0000 7fab0cff9640 1 -- 172.21.15.12:0/4151627217 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fab200035f0 con 0x55da4d5da8e0 2024-04-22T13:28:34.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.570+0000 7fab2e06e740 10 client.17301 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:34.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.570+0000 7fab2e06e740 10 client.17301 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:34.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.570+0000 7fab2e06e740 20 client.17301 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:34.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.570+0000 7fab2e06e740 1 --2- 172.21.15.12:0/4151627217 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55da4d5f42a0 0x55da4d5f6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.570+0000 7fab2e06e740 4 client.17301 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:34.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.570+0000 7fab2e06e740 1 -- 172.21.15.12:0/4151627217 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da4d54c920 con 0x55da4d5f42a0 2024-04-22T13:28:34.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.570+0000 7fab28a0f640 1 --2- 172.21.15.12:0/4151627217 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55da4d5f42a0 0x55da4d5f6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.570+0000 7fab28a0f640 1 --2- 172.21.15.12:0/4151627217 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55da4d5f42a0 0x55da4d5f6710 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9e017be86f0b821 server_cookie=c9d4663087b3bb09 in_seq=0 out_seq=0 2024-04-22T13:28:34.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.570+0000 7fab0cff9640 10 client.17301 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:34.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.571+0000 7fab0cff9640 1 -- 172.21.15.12:0/4151627217 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da4d54c920 con 0x55da4d5f42a0 2024-04-22T13:28:34.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.571+0000 7fab0cff9640 10 client.17301 handle_command_reply: tid=0 2024-04-22T13:28:34.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.571+0000 7fab2e06e740 1 client.17301 shutdown 2024-04-22T13:28:34.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.571+0000 7fab2e06e740 1 -- 172.21.15.12:0/4151627217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da4d5da8e0 msgr2=0x55da4d5e0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:34.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.571+0000 7fab2e06e740 1 --2- 172.21.15.12:0/4151627217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da4d5da8e0 0x55da4d5e0e10 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fab2002c170 tx=0x7fab2002c9f0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.571+0000 7fab2e06e740 1 -- 172.21.15.12:0/4151627217 shutdown_connections 2024-04-22T13:28:34.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.571+0000 7fab2e06e740 1 --2- 172.21.15.12:0/4151627217 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55da4d5f42a0 0x55da4d5f6710 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.571+0000 7fab2e06e740 1 --2- 172.21.15.12:0/4151627217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da4d5e3e10 0x55da4d5e1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.571+0000 7fab2e06e740 1 --2- 172.21.15.12:0/4151627217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da4d5db290 0x55da4d5e1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.571+0000 7fab2e06e740 1 --2- 172.21.15.12:0/4151627217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da4d5da8e0 0x55da4d5e0e10 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.571+0000 7fab0cff9640 0 client.17301 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:34.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.571+0000 7fab2e06e740 1 -- 172.21.15.12:0/4151627217 >> 172.21.15.12:0/4151627217 conn(0x55da4d5d4fb0 msgr2=0x55da4d5d6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:34.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.571+0000 7fab2e06e740 1 -- 172.21.15.12:0/4151627217 shutdown_connections 2024-04-22T13:28:34.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.571+0000 7fab2e06e740 1 -- 172.21.15.12:0/4151627217 wait complete. 2024-04-22T13:28:34.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.572+0000 7fab2e06e740 20 client.17301 trim_cache size 0 max 16384 2024-04-22T13:28:34.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.589+0000 7fab2e06e740 1 Processor -- start 2024-04-22T13:28:34.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.589+0000 7fab2e06e740 1 -- start start 2024-04-22T13:28:34.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.589+0000 7fab2e06e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da4d5da8e0 0x55da4d5e3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.589+0000 7fab2e06e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da4d5db290 0x55da4d5efb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.589+0000 7fab2e06e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da4d5e3e10 0x55da4d5fbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.589+0000 7fab2e06e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fab241b40c0 con 0x55da4d5da8e0 2024-04-22T13:28:34.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.589+0000 7fab2e06e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fab241b5c00 con 0x55da4d5e3e10 2024-04-22T13:28:34.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.589+0000 7fab2e06e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fab241b4790 con 0x55da4d5db290 2024-04-22T13:28:34.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.590+0000 7fab29210640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da4d5da8e0 0x55da4d5e3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.590+0000 7fab29210640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da4d5da8e0 0x55da4d5e3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32918/0 (socket says 172.21.15.12:32918) 2024-04-22T13:28:34.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.590+0000 7fab29210640 1 -- 172.21.15.12:0/4089076198 learned_addr learned my addr 172.21.15.12:0/4089076198 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:34.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.590+0000 7fab28a0f640 1 --2- 172.21.15.12:0/4089076198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da4d5db290 0x55da4d5efb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.590+0000 7fab29210640 1 -- 172.21.15.12:0/4089076198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da4d5db290 msgr2=0x55da4d5efb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:34.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.590+0000 7fab29210640 1 --2- 172.21.15.12:0/4089076198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da4d5db290 0x55da4d5efb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.590+0000 7fab29210640 1 -- 172.21.15.12:0/4089076198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da4d5e3e10 msgr2=0x55da4d5fbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:34.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.590+0000 7fab29210640 1 --2- 172.21.15.12:0/4089076198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da4d5e3e10 0x55da4d5fbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.590+0000 7fab29210640 1 -- 172.21.15.12:0/4089076198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da4d5e1d70 con 0x55da4d5da8e0 2024-04-22T13:28:34.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.590+0000 7fab29210640 1 --2- 172.21.15.12:0/4089076198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da4d5da8e0 0x55da4d5e3540 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fab200096c0 tx=0x7fab2002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a0a01ff65ec17c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:34.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.590+0000 7fab0effd640 1 -- 172.21.15.12:0/4089076198 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab20005d50 con 0x55da4d5da8e0 2024-04-22T13:28:34.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.590+0000 7fab0effd640 1 -- 172.21.15.12:0/4089076198 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab20005520 con 0x55da4d5da8e0 2024-04-22T13:28:34.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.590+0000 7fab0effd640 1 -- 172.21.15.12:0/4089076198 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab200046c0 con 0x55da4d5da8e0 2024-04-22T13:28:34.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.591+0000 7fab2e06e740 1 -- 172.21.15.12:0/4089076198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da4d5da8e0 msgr2=0x55da4d5e3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:34.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.591+0000 7fab2e06e740 1 --2- 172.21.15.12:0/4089076198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da4d5da8e0 0x55da4d5e3540 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fab200096c0 tx=0x7fab2002bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:34.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.591+0000 7fab2e06e740 1 -- 172.21.15.12:0/4089076198 shutdown_connections 2024-04-22T13:28:34.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.591+0000 7fab2e06e740 1 --2- 172.21.15.12:0/4089076198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da4d5e3e10 0x55da4d5fbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.591+0000 7fab2e06e740 1 --2- 172.21.15.12:0/4089076198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da4d5db290 0x55da4d5efb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.591+0000 7fab2e06e740 1 --2- 172.21.15.12:0/4089076198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da4d5da8e0 0x55da4d5e3540 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.591+0000 7fab2e06e740 1 -- 172.21.15.12:0/4089076198 >> 172.21.15.12:0/4089076198 conn(0x55da4d5d4fb0 msgr2=0x55da4d5e1660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:34.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.591+0000 7fab2e06e740 1 -- 172.21.15.12:0/4089076198 shutdown_connections 2024-04-22T13:28:34.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.591+0000 7fab2e06e740 1 -- 172.21.15.12:0/4089076198 wait complete. 2024-04-22T13:28:34.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.591+0000 7fab2e06e740 1 -- start start 2024-04-22T13:28:34.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.591+0000 7fab2e06e740 1 Processor -- start 2024-04-22T13:28:34.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.592+0000 7fab2e06e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da4d5da8e0 0x55da4d5dded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.592+0000 7fab2e06e740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da4d5db290 0x55da4d5de410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.592+0000 7fab2e06e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da4d5e3e10 0x55da4d659850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.592+0000 7fab2e06e740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fab241b5530 con 0x55da4d5db290 2024-04-22T13:28:34.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.592+0000 7fab2e06e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fab241b4e60 con 0x55da4d5da8e0 2024-04-22T13:28:34.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.592+0000 7fab2e06e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fab241b20a0 con 0x55da4d5e3e10 2024-04-22T13:28:34.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.592+0000 7fab2e06e740 10 client.0 _ll_register_callbacks cb 0x7fab20009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:34.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.592+0000 7fab28a0f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da4d5db290 0x55da4d5de410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.592+0000 7fab28a0f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da4d5db290 0x55da4d5de410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32922/0 (socket says 172.21.15.12:32922) 2024-04-22T13:28:34.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.592+0000 7fab28a0f640 1 -- 172.21.15.12:0/1676313350 learned_addr learned my addr 172.21.15.12:0/1676313350 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:34.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.592+0000 7fab29210640 1 --2- 172.21.15.12:0/1676313350 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da4d5da8e0 0x55da4d5dded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.592+0000 7fab29a11640 1 --2- 172.21.15.12:0/1676313350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da4d5e3e10 0x55da4d659850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.592+0000 7fab28a0f640 1 -- 172.21.15.12:0/1676313350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da4d5e3e10 msgr2=0x55da4d659850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:34.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.592+0000 7fab28a0f640 1 --2- 172.21.15.12:0/1676313350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da4d5e3e10 0x55da4d659850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.592+0000 7fab28a0f640 1 -- 172.21.15.12:0/1676313350 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da4d5da8e0 msgr2=0x55da4d5dded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:34.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.592+0000 7fab28a0f640 1 --2- 172.21.15.12:0/1676313350 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da4d5da8e0 0x55da4d5dded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.592+0000 7fab28a0f640 1 -- 172.21.15.12:0/1676313350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da4d659d90 con 0x55da4d5db290 2024-04-22T13:28:34.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.592+0000 7fab29210640 1 --2- 172.21.15.12:0/1676313350 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da4d5da8e0 0x55da4d5dded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:34.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.592+0000 7fab29a11640 1 --2- 172.21.15.12:0/1676313350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da4d5e3e10 0x55da4d659850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:34.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.592+0000 7fab28a0f640 1 --2- 172.21.15.12:0/1676313350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da4d5db290 0x55da4d5de410 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fab14098910 tx=0x7fab1409c600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a817e0784ec7044c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:34.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.592+0000 7fab0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:34.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.592+0000 7fab0cff9640 1 -- 172.21.15.12:0/1676313350 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab140b4070 con 0x55da4d5db290 2024-04-22T13:28:34.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.592+0000 7fab0cff9640 1 -- 172.21.15.12:0/1676313350 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fab140cdee0 con 0x55da4d5db290 2024-04-22T13:28:34.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.592+0000 7fab2e06e740 1 -- 172.21.15.12:0/1676313350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab20005520 con 0x55da4d5db290 2024-04-22T13:28:34.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.593+0000 7fab0cff9640 1 -- 172.21.15.12:0/1676313350 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fab14014070 con 0x55da4d5db290 2024-04-22T13:28:34.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.593+0000 7fab2e06e740 10 client.17307 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:34.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.593+0000 7fab2e06e740 1 -- 172.21.15.12:0/1676313350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55da4d65af70 con 0x55da4d5db290 2024-04-22T13:28:34.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.593+0000 7fab0cff9640 1 -- 172.21.15.12:0/1676313350 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab20005520 con 0x55da4d5db290 2024-04-22T13:28:34.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.593+0000 7fab0cff9640 1 -- 172.21.15.12:0/1676313350 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fab140bb2b0 con 0x55da4d5db290 2024-04-22T13:28:34.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.593+0000 7fab2e06e740 10 client.17307 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:34.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.593+0000 7fab2e06e740 10 client.17307 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:34.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.593+0000 7fab2e06e740 20 client.17307 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:34.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.593+0000 7fab2e06e740 1 --2- 172.21.15.12:0/1676313350 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55da4d65b200 0x55da4d65d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.593+0000 7fab2e06e740 4 client.17307 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:28:34.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.593+0000 7fab2e06e740 1 -- 172.21.15.12:0/1676313350 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55da4d546790 con 0x55da4d65b200 2024-04-22T13:28:34.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.593+0000 7fab29210640 1 --2- 172.21.15.12:0/1676313350 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55da4d65b200 0x55da4d65d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.594+0000 7fab29210640 1 --2- 172.21.15.12:0/1676313350 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55da4d65b200 0x55da4d65d5f0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3779ac4dcc5874c0 server_cookie=27600381838fc5d in_seq=0 out_seq=0 2024-04-22T13:28:34.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.594+0000 7fab0cff9640 10 client.17307 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:34.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.595+0000 7fab0cff9640 1 -- 172.21.15.12:0/1676313350 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55da4d546790 con 0x55da4d65b200 2024-04-22T13:28:34.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.595+0000 7fab0cff9640 10 client.17307 handle_command_reply: tid=0 2024-04-22T13:28:34.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.595+0000 7fab2e06e740 1 client.17307 shutdown 2024-04-22T13:28:34.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.595+0000 7fab2e06e740 1 -- 172.21.15.12:0/1676313350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da4d5db290 msgr2=0x55da4d5de410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:34.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.595+0000 7fab2e06e740 1 --2- 172.21.15.12:0/1676313350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da4d5db290 0x55da4d5de410 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fab14098910 tx=0x7fab1409c600 comp rx=0 tx=0).stop 2024-04-22T13:28:34.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.595+0000 7fab2e06e740 1 -- 172.21.15.12:0/1676313350 shutdown_connections 2024-04-22T13:28:34.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.595+0000 7fab2e06e740 1 --2- 172.21.15.12:0/1676313350 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55da4d65b200 0x55da4d65d5f0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.595+0000 7fab2e06e740 1 --2- 172.21.15.12:0/1676313350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55da4d5e3e10 0x55da4d659850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.595+0000 7fab0cff9640 0 client.17307 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:34.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.595+0000 7fab2e06e740 1 --2- 172.21.15.12:0/1676313350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55da4d5db290 0x55da4d5de410 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.595+0000 7fab2e06e740 1 --2- 172.21.15.12:0/1676313350 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da4d5da8e0 0x55da4d5dded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.595+0000 7fab2e06e740 1 -- 172.21.15.12:0/1676313350 >> 172.21.15.12:0/1676313350 conn(0x55da4d5d4fb0 msgr2=0x55da4d5d3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:34.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.595+0000 7fab2e06e740 1 -- 172.21.15.12:0/1676313350 shutdown_connections 2024-04-22T13:28:34.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.595+0000 7fab2e06e740 1 -- 172.21.15.12:0/1676313350 wait complete. 2024-04-22T13:28:34.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.595+0000 7fab2e06e740 20 client.17307 trim_cache size 0 max 16384 2024-04-22T13:28:34.610 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:28:34.610 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-04-22T13:28:34.610 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "23cc2df2-d9ff-44b1-9385-3f2fdaf35c19", 2024-04-22T13:28:34.610 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-04-22T13:28:34.610 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:28:34.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.596+0000 7fab2b474640 1 -- 172.21.15.12:0/3771346735 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fab100a3d20 msgr2=0x7fab100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:34.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.596+0000 7fab2b474640 1 --2- 172.21.15.12:0/3771346735 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fab100a3d20 0x7fab100a61e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fab1c005fd0 tx=0x7fab1c009db0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.596+0000 7fab2b474640 1 -- 172.21.15.12:0/3771346735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab24104980 msgr2=0x7fab2419f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:34.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.596+0000 7fab2b474640 1 --2- 172.21.15.12:0/3771346735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab24104980 0x7fab2419f280 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fab1400d950 tx=0x7fab1400de10 comp rx=0 tx=0).stop 2024-04-22T13:28:34.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.596+0000 7fab2b474640 1 -- 172.21.15.12:0/3771346735 shutdown_connections 2024-04-22T13:28:34.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.596+0000 7fab2b474640 1 --2- 172.21.15.12:0/3771346735 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fab100a3d20 0x7fab100a61e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.596+0000 7fab2b474640 1 --2- 172.21.15.12:0/3771346735 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fab24106e80 0x7fab241a3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.596+0000 7fab2b474640 1 --2- 172.21.15.12:0/3771346735 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fab24104980 0x7fab2419f280 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.596+0000 7fab2b474640 1 --2- 172.21.15.12:0/3771346735 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab2406ab10 0x7fab2419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.596+0000 7fab2b474640 1 -- 172.21.15.12:0/3771346735 >> 172.21.15.12:0/3771346735 conn(0x7fab24077eb0 msgr2=0x7fab24103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:34.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.596+0000 7fab2b474640 1 -- 172.21.15.12:0/3771346735 shutdown_connections 2024-04-22T13:28:34.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.596+0000 7fab2b474640 1 -- 172.21.15.12:0/3771346735 wait complete. 2024-04-22T13:28:34.613 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:28:34.613 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:28:34.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.721+0000 7f54ce031640 1 -- 172.21.15.12:0/368196217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54c81080d0 msgr2=0x7f54c81084b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:34.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.721+0000 7f54ce031640 1 --2- 172.21.15.12:0/368196217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54c81080d0 0x7f54c81084b0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f54b8009930 tx=0x7f54b802bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:34.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.721+0000 7f54ce031640 1 -- 172.21.15.12:0/368196217 shutdown_connections 2024-04-22T13:28:34.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.721+0000 7f54ce031640 1 --2- 172.21.15.12:0/368196217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54c8100e40 0x7f54c810ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.721+0000 7f54ce031640 1 --2- 172.21.15.12:0/368196217 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54c81004c0 0x7f54c8100900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.721+0000 7f54ce031640 1 --2- 172.21.15.12:0/368196217 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54c81080d0 0x7f54c81084b0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.721+0000 7f54ce031640 1 -- 172.21.15.12:0/368196217 >> 172.21.15.12:0/368196217 conn(0x7f54c8077360 msgr2=0x7f54c80fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:34.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.721+0000 7f54ce031640 1 -- 172.21.15.12:0/368196217 shutdown_connections 2024-04-22T13:28:34.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.721+0000 7f54ce031640 1 -- 172.21.15.12:0/368196217 wait complete. 2024-04-22T13:28:34.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.722+0000 7f54ce031640 1 Processor -- start 2024-04-22T13:28:34.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.722+0000 7f54ce031640 1 -- start start 2024-04-22T13:28:34.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.722+0000 7f54ce031640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54c81004c0 0x7f54c819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.722+0000 7f54ce031640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54c8100e40 0x7f54c819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.722+0000 7f54ce031640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54c81080d0 0x7f54c81985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.722+0000 7f54ce031640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f54c8112280 con 0x7f54c81004c0 2024-04-22T13:28:34.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.722+0000 7f54ce031640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f54c8112100 con 0x7f54c8100e40 2024-04-22T13:28:34.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.722+0000 7f54ce031640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54c8112400 con 0x7f54c81080d0 2024-04-22T13:28:34.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.722+0000 7f54c77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54c81004c0 0x7f54c819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.722+0000 7f54c77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54c81004c0 0x7f54c819e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32948/0 (socket says 172.21.15.12:32948) 2024-04-22T13:28:34.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.722+0000 7f54c77fe640 1 -- 172.21.15.12:0/1334068710 learned_addr learned my addr 172.21.15.12:0/1334068710 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:34.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.722+0000 7f54c7fff640 1 --2- 172.21.15.12:0/1334068710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54c81080d0 0x7f54c81985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.722+0000 7f54c6ffd640 1 --2- 172.21.15.12:0/1334068710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54c8100e40 0x7f54c819e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.722+0000 7f54c77fe640 1 -- 172.21.15.12:0/1334068710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54c81080d0 msgr2=0x7f54c81985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:34.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.722+0000 7f54c77fe640 1 --2- 172.21.15.12:0/1334068710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54c81080d0 0x7f54c81985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.723+0000 7f54c77fe640 1 -- 172.21.15.12:0/1334068710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54c8100e40 msgr2=0x7f54c819e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:34.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.723+0000 7f54c77fe640 1 --2- 172.21.15.12:0/1334068710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54c8100e40 0x7f54c819e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.723+0000 7f54c77fe640 1 -- 172.21.15.12:0/1334068710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c8198d40 con 0x7f54c81004c0 2024-04-22T13:28:34.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.723+0000 7f54c7fff640 1 --2- 172.21.15.12:0/1334068710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54c81080d0 0x7f54c81985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:34.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.723+0000 7f54c77fe640 1 --2- 172.21.15.12:0/1334068710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54c81004c0 0x7f54c819e410 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f54b8005940 tx=0x7f54b802ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:34.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.723+0000 7f54c4ff9640 1 -- 172.21.15.12:0/1334068710 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b8043020 con 0x7f54c81004c0 2024-04-22T13:28:34.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.723+0000 7f54c4ff9640 1 -- 172.21.15.12:0/1334068710 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54b802cc80 con 0x7f54c81004c0 2024-04-22T13:28:34.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.723+0000 7f54c4ff9640 1 -- 172.21.15.12:0/1334068710 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b8003c70 con 0x7f54c81004c0 2024-04-22T13:28:34.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.723+0000 7f54ce031640 1 -- 172.21.15.12:0/1334068710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c8198fd0 con 0x7f54c81004c0 2024-04-22T13:28:34.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.723+0000 7f54ce031640 1 -- 172.21.15.12:0/1334068710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c81a52d0 con 0x7f54c81004c0 2024-04-22T13:28:34.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.725+0000 7f54c4ff9640 1 -- 172.21.15.12:0/1334068710 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f54b8046020 con 0x7f54c81004c0 2024-04-22T13:28:34.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.725+0000 7f54d0c24740 1 Processor -- start 2024-04-22T13:28:34.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.725+0000 7f54d0c24740 1 -- start start 2024-04-22T13:28:34.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.726+0000 7f54d0c24740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b16e133580 0x55b16e133960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.726+0000 7f54d0c24740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b16e133ea0 0x55b16e13c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.726+0000 7f54d0c24740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b16e13ca80 0x55b16e13ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.726+0000 7f54d0c24740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b16e0981e0 con 0x55b16e133ea0 2024-04-22T13:28:34.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.726+0000 7f54d0c24740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b16e035390 con 0x55b16e13ca80 2024-04-22T13:28:34.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.726+0000 7f54d0c24740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b16e0941a0 con 0x55b16e133580 2024-04-22T13:28:34.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.726+0000 7f54c77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b16e133ea0 0x55b16e13c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.726+0000 7f54c77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b16e133ea0 0x55b16e13c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32964/0 (socket says 172.21.15.12:32964) 2024-04-22T13:28:34.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.726+0000 7f54c77fe640 1 -- 172.21.15.12:0/4049545361 learned_addr learned my addr 172.21.15.12:0/4049545361 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:34.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.726+0000 7f54c4ff9640 1 --2- 172.21.15.12:0/1334068710 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f54b00a3d20 0x7f54b00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.726+0000 7f54c7fff640 1 --2- 172.21.15.12:0/4049545361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b16e133580 0x55b16e133960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.726+0000 7f54c6ffd640 1 --2- 172.21.15.12:0/4049545361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b16e13ca80 0x55b16e13ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.726+0000 7f54c7fff640 1 --2- 172.21.15.12:0/1334068710 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f54b00a3d20 0x7f54b00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.726+0000 7f54c4ff9640 1 -- 172.21.15.12:0/1334068710 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f54b80f2a90 con 0x7f54c81004c0 2024-04-22T13:28:34.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.726+0000 7f54c77fe640 1 -- 172.21.15.12:0/4049545361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b16e133580 msgr2=0x55b16e133960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:34.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.726+0000 7f54c77fe640 1 --2- 172.21.15.12:0/4049545361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b16e133580 0x55b16e133960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.726+0000 7f54c77fe640 1 -- 172.21.15.12:0/4049545361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b16e13ca80 msgr2=0x55b16e13ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:34.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.726+0000 7f54c77fe640 1 --2- 172.21.15.12:0/4049545361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b16e13ca80 0x55b16e13ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.726+0000 7f54c77fe640 1 -- 172.21.15.12:0/4049545361 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b16de747d0 con 0x55b16e133ea0 2024-04-22T13:28:34.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.726+0000 7f54c77fe640 1 --2- 172.21.15.12:0/4049545361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b16e133ea0 0x55b16e13c540 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f54b80bb560 tx=0x7f54b802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dbb3798c5937881 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:34.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.726+0000 7f54c7fff640 1 --2- 172.21.15.12:0/1334068710 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f54b00a3d20 0x7f54b00a61e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f54c000afd0 tx=0x7f54c0005f90 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:34.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.727+0000 7f54adffb640 1 -- 172.21.15.12:0/4049545361 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b80be960 con 0x55b16e133ea0 2024-04-22T13:28:34.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.727+0000 7f54adffb640 1 -- 172.21.15.12:0/4049545361 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54b80fa040 con 0x55b16e133ea0 2024-04-22T13:28:34.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.727+0000 7f54adffb640 1 -- 172.21.15.12:0/4049545361 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b80be4f0 con 0x55b16e133ea0 2024-04-22T13:28:34.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.727+0000 7f54d0c24740 1 -- 172.21.15.12:0/4049545361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b16e133ea0 msgr2=0x55b16e13c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:34.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.727+0000 7f54d0c24740 1 --2- 172.21.15.12:0/4049545361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b16e133ea0 0x55b16e13c540 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f54b80bb560 tx=0x7f54b802bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:34.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.727+0000 7f54d0c24740 1 -- 172.21.15.12:0/4049545361 shutdown_connections 2024-04-22T13:28:34.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.727+0000 7f54d0c24740 1 --2- 172.21.15.12:0/4049545361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b16e13ca80 0x55b16e13ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.727+0000 7f54d0c24740 1 --2- 172.21.15.12:0/4049545361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b16e133ea0 0x55b16e13c540 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.727+0000 7f54d0c24740 1 --2- 172.21.15.12:0/4049545361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b16e133580 0x55b16e133960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.727+0000 7f54d0c24740 1 -- 172.21.15.12:0/4049545361 >> 172.21.15.12:0/4049545361 conn(0x55b16e12dcb0 msgr2=0x55b16e12e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:34.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.727+0000 7f54d0c24740 1 -- 172.21.15.12:0/4049545361 shutdown_connections 2024-04-22T13:28:34.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.727+0000 7f54d0c24740 1 -- 172.21.15.12:0/4049545361 wait complete. 2024-04-22T13:28:34.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.728+0000 7f54d0c24740 1 -- start start 2024-04-22T13:28:34.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.728+0000 7f54d0c24740 1 Processor -- start 2024-04-22T13:28:34.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.728+0000 7f54d0c24740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b16e133580 0x55b16e139a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.728+0000 7f54d0c24740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b16e133ea0 0x55b16e139fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.728+0000 7f54d0c24740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b16e13ca80 0x55b16e13a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.728+0000 7f54d0c24740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b16e13f630 con 0x55b16e133ea0 2024-04-22T13:28:34.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.728+0000 7f54d0c24740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b16e0512f0 con 0x55b16e133580 2024-04-22T13:28:34.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.728+0000 7f54d0c24740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b16e13f7b0 con 0x55b16e13ca80 2024-04-22T13:28:34.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.728+0000 7f54d0c24740 10 client.0 _ll_register_callbacks cb 0x55b16e05e580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:34.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.728+0000 7f54c6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b16e133ea0 0x55b16e139fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.728+0000 7f54c6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b16e133ea0 0x55b16e139fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32978/0 (socket says 172.21.15.12:32978) 2024-04-22T13:28:34.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.728+0000 7f54c6ffd640 1 -- 172.21.15.12:0/2093777616 learned_addr learned my addr 172.21.15.12:0/2093777616 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:34.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.728+0000 7f54c7fff640 1 --2- 172.21.15.12:0/2093777616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b16e13ca80 0x55b16e13a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.728+0000 7f54c77fe640 1 --2- 172.21.15.12:0/2093777616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b16e133580 0x55b16e139a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.729+0000 7f54c6ffd640 1 -- 172.21.15.12:0/2093777616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b16e13ca80 msgr2=0x55b16e13a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:34.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.729+0000 7f54c6ffd640 1 --2- 172.21.15.12:0/2093777616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b16e13ca80 0x55b16e13a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.729+0000 7f54c6ffd640 1 -- 172.21.15.12:0/2093777616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b16e133580 msgr2=0x55b16e139a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:34.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.729+0000 7f54c6ffd640 1 --2- 172.21.15.12:0/2093777616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b16e133580 0x55b16e139a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.729+0000 7f54c6ffd640 1 -- 172.21.15.12:0/2093777616 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b16ddaa3f0 con 0x55b16e133ea0 2024-04-22T13:28:34.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.729+0000 7f54c6ffd640 1 --2- 172.21.15.12:0/2093777616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b16e133ea0 0x55b16e139fa0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f54bc00f6f0 tx=0x7f54bc00fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57bf492e3feb2d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.729+0000 7f54a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.729+0000 7f54a6ffd640 1 -- 172.21.15.12:0/2093777616 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54bc016070 con 0x55b16e133ea0 2024-04-22T13:28:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.729+0000 7f54a6ffd640 1 -- 172.21.15.12:0/2093777616 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f54bc00bb00 con 0x55b16e133ea0 2024-04-22T13:28:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.729+0000 7f54d0c24740 1 -- 172.21.15.12:0/2093777616 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54b80fa040 con 0x55b16e133ea0 2024-04-22T13:28:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.729+0000 7f54a6ffd640 1 -- 172.21.15.12:0/2093777616 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f54bc00bd20 con 0x55b16e133ea0 2024-04-22T13:28:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.729+0000 7f54d0c24740 10 client.17319 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.729+0000 7f54d0c24740 1 -- 172.21.15.12:0/2093777616 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b16dd6a0f0 con 0x55b16e133ea0 2024-04-22T13:28:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.729+0000 7f54a6ffd640 1 -- 172.21.15.12:0/2093777616 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54b80fa040 con 0x55b16e133ea0 2024-04-22T13:28:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.729+0000 7f54a6ffd640 1 -- 172.21.15.12:0/2093777616 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f54bc01a400 con 0x55b16e133ea0 2024-04-22T13:28:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.730+0000 7f54d0c24740 10 client.17319 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.730+0000 7f54d0c24740 10 client.17319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.730+0000 7f54d0c24740 20 client.17319 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.730+0000 7f54d0c24740 1 --2- 172.21.15.12:0/2093777616 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b16e14c7b0 0x55b16e14ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.730+0000 7f54d0c24740 4 client.17319 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.730+0000 7f54d0c24740 1 -- 172.21.15.12:0/2093777616 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b16e0a5c00 con 0x55b16e14c7b0 2024-04-22T13:28:34.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.730+0000 7f54c77fe640 1 --2- 172.21.15.12:0/2093777616 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b16e14c7b0 0x55b16e14ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.730+0000 7f54a6ffd640 10 client.17319 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:34.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.730+0000 7f54c77fe640 1 --2- 172.21.15.12:0/2093777616 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b16e14c7b0 0x55b16e14ec00 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e44ed50a55d16f36 server_cookie=386bae1469358c99 in_seq=0 out_seq=0 2024-04-22T13:28:34.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.731+0000 7f54a6ffd640 1 -- 172.21.15.12:0/2093777616 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b16e0a5c00 con 0x55b16e14c7b0 2024-04-22T13:28:34.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.731+0000 7f54a6ffd640 10 client.17319 handle_command_reply: tid=0 2024-04-22T13:28:34.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.731+0000 7f54d0c24740 1 client.17319 shutdown 2024-04-22T13:28:34.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.731+0000 7f54d0c24740 1 -- 172.21.15.12:0/2093777616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b16e133ea0 msgr2=0x55b16e139fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:34.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.731+0000 7f54d0c24740 1 --2- 172.21.15.12:0/2093777616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b16e133ea0 0x55b16e139fa0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f54bc00f6f0 tx=0x7f54bc00fbb0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.731+0000 7f54d0c24740 1 -- 172.21.15.12:0/2093777616 shutdown_connections 2024-04-22T13:28:34.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.731+0000 7f54d0c24740 1 --2- 172.21.15.12:0/2093777616 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b16e14c7b0 0x55b16e14ec00 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.731+0000 7f54d0c24740 1 --2- 172.21.15.12:0/2093777616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b16e13ca80 0x55b16e13a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.731+0000 7f54a6ffd640 0 client.17319 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:34.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.731+0000 7f54d0c24740 1 --2- 172.21.15.12:0/2093777616 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b16e133ea0 0x55b16e139fa0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.731+0000 7f54d0c24740 1 --2- 172.21.15.12:0/2093777616 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b16e133580 0x55b16e139a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.731+0000 7f54d0c24740 1 -- 172.21.15.12:0/2093777616 >> 172.21.15.12:0/2093777616 conn(0x55b16e12dcb0 msgr2=0x55b16e12fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:34.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.731+0000 7f54d0c24740 1 -- 172.21.15.12:0/2093777616 shutdown_connections 2024-04-22T13:28:34.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.731+0000 7f54d0c24740 1 -- 172.21.15.12:0/2093777616 wait complete. 2024-04-22T13:28:34.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.731+0000 7f54d0c24740 20 client.17319 trim_cache size 0 max 16384 2024-04-22T13:28:34.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.749+0000 7f54d0c24740 1 Processor -- start 2024-04-22T13:28:34.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.749+0000 7f54d0c24740 1 -- start start 2024-04-22T13:28:34.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.749+0000 7f54d0c24740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b16e133580 0x55b16e13c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.749+0000 7f54d0c24740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b16e133ea0 0x55b16e148410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.749+0000 7f54d0c24740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b16e13ca80 0x55b16e12f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.749+0000 7f54d0c24740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f54c81adab0 con 0x55b16e133ea0 2024-04-22T13:28:34.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.749+0000 7f54d0c24740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f54c81af5f0 con 0x55b16e13ca80 2024-04-22T13:28:34.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.749+0000 7f54d0c24740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54c81ae180 con 0x55b16e133580 2024-04-22T13:28:34.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.749+0000 7f54c6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b16e133ea0 0x55b16e148410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.749+0000 7f54c6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b16e133ea0 0x55b16e148410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32992/0 (socket says 172.21.15.12:32992) 2024-04-22T13:28:34.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.749+0000 7f54c6ffd640 1 -- 172.21.15.12:0/1259231454 learned_addr learned my addr 172.21.15.12:0/1259231454 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:34.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.749+0000 7f54c7fff640 1 --2- 172.21.15.12:0/1259231454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b16e13ca80 0x55b16e12f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.749+0000 7f54c77fe640 1 --2- 172.21.15.12:0/1259231454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b16e133580 0x55b16e13c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.749+0000 7f54c6ffd640 1 -- 172.21.15.12:0/1259231454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b16e133580 msgr2=0x55b16e13c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:34.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.749+0000 7f54c6ffd640 1 --2- 172.21.15.12:0/1259231454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b16e133580 0x55b16e13c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.749+0000 7f54c6ffd640 1 -- 172.21.15.12:0/1259231454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b16e13ca80 msgr2=0x55b16e12f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:34.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.749+0000 7f54c6ffd640 1 --2- 172.21.15.12:0/1259231454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b16e13ca80 0x55b16e12f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.749+0000 7f54c6ffd640 1 -- 172.21.15.12:0/1259231454 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b16e12dae0 con 0x55b16e133ea0 2024-04-22T13:28:34.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.749+0000 7f54c7fff640 1 --2- 172.21.15.12:0/1259231454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b16e13ca80 0x55b16e12f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:34.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.749+0000 7f54c77fe640 1 --2- 172.21.15.12:0/1259231454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b16e133580 0x55b16e13c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:34.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.750+0000 7f54c6ffd640 1 --2- 172.21.15.12:0/1259231454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b16e133ea0 0x55b16e148410 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f54bc01a8a0 tx=0x7f54bc00bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e668177fde39e96 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:34.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.750+0000 7f54ad7fa640 1 -- 172.21.15.12:0/1259231454 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54bc016040 con 0x55b16e133ea0 2024-04-22T13:28:34.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.750+0000 7f54ad7fa640 1 -- 172.21.15.12:0/1259231454 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54bc01ac90 con 0x55b16e133ea0 2024-04-22T13:28:34.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.750+0000 7f54ad7fa640 1 -- 172.21.15.12:0/1259231454 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54bc01ae60 con 0x55b16e133ea0 2024-04-22T13:28:34.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.750+0000 7f54d0c24740 1 -- 172.21.15.12:0/1259231454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b16e133ea0 msgr2=0x55b16e148410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:34.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.750+0000 7f54d0c24740 1 --2- 172.21.15.12:0/1259231454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b16e133ea0 0x55b16e148410 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f54bc01a8a0 tx=0x7f54bc00bfc0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.750+0000 7f54d0c24740 1 -- 172.21.15.12:0/1259231454 shutdown_connections 2024-04-22T13:28:34.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.750+0000 7f54d0c24740 1 --2- 172.21.15.12:0/1259231454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b16e13ca80 0x55b16e12f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.750+0000 7f54d0c24740 1 --2- 172.21.15.12:0/1259231454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b16e133ea0 0x55b16e148410 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.750+0000 7f54d0c24740 1 --2- 172.21.15.12:0/1259231454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b16e133580 0x55b16e13c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.750+0000 7f54d0c24740 1 -- 172.21.15.12:0/1259231454 >> 172.21.15.12:0/1259231454 conn(0x55b16e12dcb0 msgr2=0x55b16e12f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:34.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.750+0000 7f54d0c24740 1 -- 172.21.15.12:0/1259231454 shutdown_connections 2024-04-22T13:28:34.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.750+0000 7f54d0c24740 1 -- 172.21.15.12:0/1259231454 wait complete. 2024-04-22T13:28:34.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.751+0000 7f54d0c24740 1 -- start start 2024-04-22T13:28:34.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.751+0000 7f54d0c24740 1 Processor -- start 2024-04-22T13:28:34.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.751+0000 7f54d0c24740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b16e133580 0x55b16e144920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.751+0000 7f54d0c24740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b16e133ea0 0x55b16e144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.751+0000 7f54d0c24740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b16e13ca80 0x55b16e1418f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.751+0000 7f54d0c24740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f54c81aef20 con 0x55b16e13ca80 2024-04-22T13:28:34.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.751+0000 7f54d0c24740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f54c81ae850 con 0x55b16e133580 2024-04-22T13:28:34.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.751+0000 7f54d0c24740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54c81aba90 con 0x55b16e133ea0 2024-04-22T13:28:34.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.751+0000 7f54d0c24740 10 client.0 _ll_register_callbacks cb 0x7f54bc00d340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:34.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.751+0000 7f54c7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b16e13ca80 0x55b16e1418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.751+0000 7f54c7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b16e13ca80 0x55b16e1418f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32994/0 (socket says 172.21.15.12:32994) 2024-04-22T13:28:34.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.751+0000 7f54c7fff640 1 -- 172.21.15.12:0/2065314359 learned_addr learned my addr 172.21.15.12:0/2065314359 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:34.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.751+0000 7f54c77fe640 1 --2- 172.21.15.12:0/2065314359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b16e133580 0x55b16e144920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.751+0000 7f54c6ffd640 1 --2- 172.21.15.12:0/2065314359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b16e133ea0 0x55b16e144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.752+0000 7f54c7fff640 1 -- 172.21.15.12:0/2065314359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b16e133ea0 msgr2=0x55b16e144e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:34.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.752+0000 7f54c7fff640 1 --2- 172.21.15.12:0/2065314359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b16e133ea0 0x55b16e144e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.752+0000 7f54c7fff640 1 -- 172.21.15.12:0/2065314359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b16e133580 msgr2=0x55b16e144920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:34.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.752+0000 7f54c7fff640 1 --2- 172.21.15.12:0/2065314359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b16e133580 0x55b16e144920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.752+0000 7f54c7fff640 1 -- 172.21.15.12:0/2065314359 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b16e141f30 con 0x55b16e13ca80 2024-04-22T13:28:34.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.752+0000 7f54c7fff640 1 --2- 172.21.15.12:0/2065314359 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b16e13ca80 0x55b16e1418f0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f54c000fb50 tx=0x7f54c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c76f91c7bdf702 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:34.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.752+0000 7f54a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:34.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.752+0000 7f54a6ffd640 1 -- 172.21.15.12:0/2065314359 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c000dbd0 con 0x55b16e13ca80 2024-04-22T13:28:34.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.752+0000 7f54a6ffd640 1 -- 172.21.15.12:0/2065314359 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f54c000dd70 con 0x55b16e13ca80 2024-04-22T13:28:34.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.752+0000 7f54d0c24740 1 -- 172.21.15.12:0/2065314359 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54bc01ac90 con 0x55b16e13ca80 2024-04-22T13:28:34.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.752+0000 7f54a6ffd640 1 -- 172.21.15.12:0/2065314359 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f54c0004b20 con 0x55b16e13ca80 2024-04-22T13:28:34.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.752+0000 7f54d0c24740 10 client.17325 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:34.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.752+0000 7f54d0c24740 1 -- 172.21.15.12:0/2065314359 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b16e1b35f0 con 0x55b16e13ca80 2024-04-22T13:28:34.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.752+0000 7f54a6ffd640 1 -- 172.21.15.12:0/2065314359 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54bc01ac90 con 0x55b16e13ca80 2024-04-22T13:28:34.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.752+0000 7f54a6ffd640 1 -- 172.21.15.12:0/2065314359 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f54c0026070 con 0x55b16e13ca80 2024-04-22T13:28:34.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.752+0000 7f54d0c24740 10 client.17325 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:34.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.752+0000 7f54d0c24740 10 client.17325 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:34.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.752+0000 7f54d0c24740 20 client.17325 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:34.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.752+0000 7f54d0c24740 1 --2- 172.21.15.12:0/2065314359 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b16e1b3e20 0x55b16e1b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.752+0000 7f54d0c24740 4 client.17325 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:28:34.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.752+0000 7f54d0c24740 1 -- 172.21.15.12:0/2065314359 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b16e09fa70 con 0x55b16e1b3e20 2024-04-22T13:28:34.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.753+0000 7f54c77fe640 1 --2- 172.21.15.12:0/2065314359 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b16e1b3e20 0x55b16e1b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.753+0000 7f54c77fe640 1 --2- 172.21.15.12:0/2065314359 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b16e1b3e20 0x55b16e1b6240 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ede2a6b3bedb74d server_cookie=d83d3ae656da686d in_seq=0 out_seq=0 2024-04-22T13:28:34.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.753+0000 7f54a6ffd640 10 client.17325 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:34.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.753+0000 7f54a6ffd640 1 -- 172.21.15.12:0/2065314359 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b16e09fa70 con 0x55b16e1b3e20 2024-04-22T13:28:34.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.753+0000 7f54a6ffd640 10 client.17325 handle_command_reply: tid=0 2024-04-22T13:28:34.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.753+0000 7f54d0c24740 1 client.17325 shutdown 2024-04-22T13:28:34.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.754+0000 7f54d0c24740 1 -- 172.21.15.12:0/2065314359 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b16e13ca80 msgr2=0x55b16e1418f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:34.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.754+0000 7f54d0c24740 1 --2- 172.21.15.12:0/2065314359 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b16e13ca80 0x55b16e1418f0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f54c000fb50 tx=0x7f54c0004a40 comp rx=0 tx=0).stop 2024-04-22T13:28:34.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.754+0000 7f54d0c24740 1 -- 172.21.15.12:0/2065314359 shutdown_connections 2024-04-22T13:28:34.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.754+0000 7f54d0c24740 1 --2- 172.21.15.12:0/2065314359 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b16e1b3e20 0x55b16e1b6240 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.754+0000 7f54d0c24740 1 --2- 172.21.15.12:0/2065314359 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b16e13ca80 0x55b16e1418f0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.754+0000 7f54d0c24740 1 --2- 172.21.15.12:0/2065314359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b16e133ea0 0x55b16e144e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.754+0000 7f54d0c24740 1 --2- 172.21.15.12:0/2065314359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b16e133580 0x55b16e144920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.754+0000 7f54a6ffd640 0 client.17325 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:34.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.754+0000 7f54d0c24740 1 -- 172.21.15.12:0/2065314359 >> 172.21.15.12:0/2065314359 conn(0x55b16e12dcb0 msgr2=0x55b16e12cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:34.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.754+0000 7f54d0c24740 1 -- 172.21.15.12:0/2065314359 shutdown_connections 2024-04-22T13:28:34.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.754+0000 7f54d0c24740 1 -- 172.21.15.12:0/2065314359 wait complete. 2024-04-22T13:28:34.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.754+0000 7f54d0c24740 20 client.17325 trim_cache size 0 max 16384 2024-04-22T13:28:34.760 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:28:34.760 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-04-22T13:28:34.760 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-04-22T13:28:34.760 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:28:34.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.754+0000 7f54ce031640 1 -- 172.21.15.12:0/1334068710 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f54b00a3d20 msgr2=0x7f54b00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:34.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.754+0000 7f54ce031640 1 --2- 172.21.15.12:0/1334068710 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f54b00a3d20 0x7f54b00a61e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f54c000afd0 tx=0x7f54c0005f90 comp rx=0 tx=0).stop 2024-04-22T13:28:34.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.754+0000 7f54ce031640 1 -- 172.21.15.12:0/1334068710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54c81004c0 msgr2=0x7f54c819e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:34.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.754+0000 7f54ce031640 1 --2- 172.21.15.12:0/1334068710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54c81004c0 0x7f54c819e410 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f54b8005940 tx=0x7f54b802ca90 comp rx=0 tx=0).stop 2024-04-22T13:28:34.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.754+0000 7f54ce031640 1 -- 172.21.15.12:0/1334068710 shutdown_connections 2024-04-22T13:28:34.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.754+0000 7f54ce031640 1 --2- 172.21.15.12:0/1334068710 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f54b00a3d20 0x7f54b00a61e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.754+0000 7f54ce031640 1 --2- 172.21.15.12:0/1334068710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54c81080d0 0x7f54c81985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.754+0000 7f54ce031640 1 --2- 172.21.15.12:0/1334068710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54c8100e40 0x7f54c819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.754+0000 7f54ce031640 1 --2- 172.21.15.12:0/1334068710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54c81004c0 0x7f54c819e410 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.754+0000 7f54ce031640 1 -- 172.21.15.12:0/1334068710 >> 172.21.15.12:0/1334068710 conn(0x7f54c8077360 msgr2=0x7f54c8104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:34.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.754+0000 7f54ce031640 1 -- 172.21.15.12:0/1334068710 shutdown_connections 2024-04-22T13:28:34.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.754+0000 7f54ce031640 1 -- 172.21.15.12:0/1334068710 wait complete. 2024-04-22T13:28:34.763 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:28:34.763 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:28:34.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.869+0000 7fac5c9a4640 1 -- 172.21.15.12:0/3973566332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac5810c5a0 msgr2=0x7fac5810e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:34.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.869+0000 7fac5c9a4640 1 --2- 172.21.15.12:0/3973566332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac5810c5a0 0x7fac5810e9d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fac50009ca0 tx=0x7fac5002c180 comp rx=0 tx=0).stop 2024-04-22T13:28:34.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.869+0000 7fac5c9a4640 1 -- 172.21.15.12:0/3973566332 shutdown_connections 2024-04-22T13:28:34.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.869+0000 7fac5c9a4640 1 --2- 172.21.15.12:0/3973566332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac5810c5a0 0x7fac5810e9d0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.869+0000 7fac5c9a4640 1 --2- 172.21.15.12:0/3973566332 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac58103af0 0x7fac5810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.869+0000 7fac5c9a4640 1 --2- 172.21.15.12:0/3973566332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac581031d0 0x7fac581035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.869+0000 7fac5c9a4640 1 -- 172.21.15.12:0/3973566332 >> 172.21.15.12:0/3973566332 conn(0x7fac580771d0 msgr2=0x7fac580775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:34.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.869+0000 7fac5c9a4640 1 -- 172.21.15.12:0/3973566332 shutdown_connections 2024-04-22T13:28:34.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.869+0000 7fac5c9a4640 1 -- 172.21.15.12:0/3973566332 wait complete. 2024-04-22T13:28:34.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.870+0000 7fac5c9a4640 1 Processor -- start 2024-04-22T13:28:34.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.870+0000 7fac5c9a4640 1 -- start start 2024-04-22T13:28:34.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.870+0000 7fac5c9a4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac581031d0 0x7fac581986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.870+0000 7fac5c9a4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac58103af0 0x7fac58198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.870+0000 7fac5c9a4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac5810c5a0 0x7fac5819cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.870+0000 7fac5c9a4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fac581121a0 con 0x7fac581031d0 2024-04-22T13:28:34.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.870+0000 7fac5c9a4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fac58112020 con 0x7fac58103af0 2024-04-22T13:28:34.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.870+0000 7fac5c9a4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fac58112320 con 0x7fac5810c5a0 2024-04-22T13:28:34.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.870+0000 7fac5659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac581031d0 0x7fac581986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.870+0000 7fac5659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac581031d0 0x7fac581986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33012/0 (socket says 172.21.15.12:33012) 2024-04-22T13:28:34.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.870+0000 7fac5659c640 1 -- 172.21.15.12:0/2668414817 learned_addr learned my addr 172.21.15.12:0/2668414817 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:34.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.871+0000 7fac4dd9b640 1 --2- 172.21.15.12:0/2668414817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac58103af0 0x7fac58198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.871+0000 7fac56d9d640 1 --2- 172.21.15.12:0/2668414817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac5810c5a0 0x7fac5819cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.871+0000 7fac5659c640 1 -- 172.21.15.12:0/2668414817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac5810c5a0 msgr2=0x7fac5819cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:34.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.871+0000 7fac5659c640 1 --2- 172.21.15.12:0/2668414817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac5810c5a0 0x7fac5819cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.871+0000 7fac5659c640 1 -- 172.21.15.12:0/2668414817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac58103af0 msgr2=0x7fac58198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:34.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.871+0000 7fac5659c640 1 --2- 172.21.15.12:0/2668414817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac58103af0 0x7fac58198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:34.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.871+0000 7fac5659c640 1 -- 172.21.15.12:0/2668414817 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac5819d6b0 con 0x7fac581031d0 2024-04-22T13:28:34.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.871+0000 7fac56d9d640 1 --2- 172.21.15.12:0/2668414817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac5810c5a0 0x7fac5819cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:34.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.871+0000 7fac4dd9b640 1 --2- 172.21.15.12:0/2668414817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac58103af0 0x7fac58198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:34.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.871+0000 7fac5659c640 1 --2- 172.21.15.12:0/2668414817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac581031d0 0x7fac581986c0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fac4800d950 tx=0x7fac4800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:34.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.871+0000 7fac4ffff640 1 -- 172.21.15.12:0/2668414817 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac48017030 con 0x7fac581031d0 2024-04-22T13:28:34.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.871+0000 7fac4ffff640 1 -- 172.21.15.12:0/2668414817 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac48004020 con 0x7fac581031d0 2024-04-22T13:28:34.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.871+0000 7fac4ffff640 1 -- 172.21.15.12:0/2668414817 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac480047a0 con 0x7fac581031d0 2024-04-22T13:28:34.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.871+0000 7fac5c9a4640 1 -- 172.21.15.12:0/2668414817 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac5819d9a0 con 0x7fac581031d0 2024-04-22T13:28:34.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.871+0000 7fac5c9a4640 1 -- 172.21.15.12:0/2668414817 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac581a5230 con 0x7fac581031d0 2024-04-22T13:28:34.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.872+0000 7fac5c9a4640 1 -- 172.21.15.12:0/2668414817 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac58069230 con 0x7fac581031d0 2024-04-22T13:28:34.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.873+0000 7fac4ffff640 1 -- 172.21.15.12:0/2668414817 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fac48009a30 con 0x7fac581031d0 2024-04-22T13:28:34.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.874+0000 7fac4ffff640 1 --2- 172.21.15.12:0/2668414817 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fac440a3d20 0x7fac440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:34.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.874+0000 7fac4dd9b640 1 --2- 172.21.15.12:0/2668414817 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fac440a3d20 0x7fac440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:34.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.874+0000 7fac4ffff640 1 -- 172.21.15.12:0/2668414817 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fac480d0920 con 0x7fac581031d0 2024-04-22T13:28:34.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.874+0000 7fac4dd9b640 1 --2- 172.21.15.12:0/2668414817 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fac440a3d20 0x7fac440a61e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fac40005e00 tx=0x7fac40005c70 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:34.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:34.878+0000 7fac4ffff640 1 -- 172.21.15.12:0/2668414817 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fac4809d010 con 0x7fac581031d0 2024-04-22T13:28:35.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.105+0000 7fac5c9a4640 1 -- 172.21.15.12:0/2668414817 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac58199a10 con 0x7fac581031d0 2024-04-22T13:28:35.102 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.105+0000 7fac4ffff640 1 -- 172.21.15.12:0/2668414817 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7fac4809c910 con 0x7fac581031d0 2024-04-22T13:28:35.102 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:28:35.102 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:28:35.103 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:28:35.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.108+0000 7fac5c9a4640 1 -- 172.21.15.12:0/2668414817 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fac440a3d20 msgr2=0x7fac440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.108+0000 7fac5c9a4640 1 --2- 172.21.15.12:0/2668414817 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fac440a3d20 0x7fac440a61e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fac40005e00 tx=0x7fac40005c70 comp rx=0 tx=0).stop 2024-04-22T13:28:35.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.108+0000 7fac5c9a4640 1 -- 172.21.15.12:0/2668414817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac581031d0 msgr2=0x7fac581986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.108+0000 7fac5c9a4640 1 --2- 172.21.15.12:0/2668414817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac581031d0 0x7fac581986c0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fac4800d950 tx=0x7fac4800de10 comp rx=0 tx=0).stop 2024-04-22T13:28:35.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.108+0000 7fac5c9a4640 1 -- 172.21.15.12:0/2668414817 shutdown_connections 2024-04-22T13:28:35.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.108+0000 7fac5c9a4640 1 --2- 172.21.15.12:0/2668414817 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fac440a3d20 0x7fac440a61e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.108+0000 7fac5c9a4640 1 --2- 172.21.15.12:0/2668414817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac5810c5a0 0x7fac5819cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.108+0000 7fac5c9a4640 1 --2- 172.21.15.12:0/2668414817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac58103af0 0x7fac58198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.108+0000 7fac5c9a4640 1 --2- 172.21.15.12:0/2668414817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac581031d0 0x7fac581986c0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.108+0000 7fac5c9a4640 1 -- 172.21.15.12:0/2668414817 >> 172.21.15.12:0/2668414817 conn(0x7fac580771d0 msgr2=0x7fac580fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:35.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.108+0000 7fac5c9a4640 1 -- 172.21.15.12:0/2668414817 shutdown_connections 2024-04-22T13:28:35.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.108+0000 7fac5c9a4640 1 -- 172.21.15.12:0/2668414817 wait complete. 2024-04-22T13:28:35.116 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:28:35.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.280+0000 7f96e1fb2640 1 -- 172.21.15.12:0/798916649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96dc106170 msgr2=0x7f96dc10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.280+0000 7f96e1fb2640 1 --2- 172.21.15.12:0/798916649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96dc106170 0x7f96dc10dd40 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f96d4009ca0 tx=0x7f96d402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.280+0000 7f96e1fb2640 1 -- 172.21.15.12:0/798916649 shutdown_connections 2024-04-22T13:28:35.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.280+0000 7f96e1fb2640 1 --2- 172.21.15.12:0/798916649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96dc106170 0x7f96dc10dd40 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.280+0000 7f96e1fb2640 1 --2- 172.21.15.12:0/798916649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96dc101a50 0x7f96dc105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.280+0000 7f96e1fb2640 1 --2- 172.21.15.12:0/798916649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96dc1010a0 0x7f96dc101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.280+0000 7f96e1fb2640 1 -- 172.21.15.12:0/798916649 >> 172.21.15.12:0/798916649 conn(0x7f96dc077eb0 msgr2=0x7f96dc0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:35.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.280+0000 7f96e1fb2640 1 -- 172.21.15.12:0/798916649 shutdown_connections 2024-04-22T13:28:35.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.280+0000 7f96e1fb2640 1 -- 172.21.15.12:0/798916649 wait complete. 2024-04-22T13:28:35.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.280+0000 7f96e1fb2640 1 Processor -- start 2024-04-22T13:28:35.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.281+0000 7f96e1fb2640 1 -- start start 2024-04-22T13:28:35.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.281+0000 7f96e1fb2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96dc1010a0 0x7f96dc10e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.281+0000 7f96e1fb2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96dc106170 0x7f96dc10ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.281+0000 7f96e1fb2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96dc10f150 0x7f96dc1a67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.281+0000 7f96e1fb2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc1121f0 con 0x7f96dc1010a0 2024-04-22T13:28:35.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.281+0000 7f96e1fb2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc112070 con 0x7f96dc106170 2024-04-22T13:28:35.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.281+0000 7f96e1fb2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc112370 con 0x7f96dc10f150 2024-04-22T13:28:35.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.281+0000 7f96db7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96dc1010a0 0x7f96dc10e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.281+0000 7f96db7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96dc1010a0 0x7f96dc10e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33030/0 (socket says 172.21.15.12:33030) 2024-04-22T13:28:35.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.281+0000 7f96db7fe640 1 -- 172.21.15.12:0/3094501047 learned_addr learned my addr 172.21.15.12:0/3094501047 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:35.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.281+0000 7f96daffd640 1 --2- 172.21.15.12:0/3094501047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96dc106170 0x7f96dc10ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.281+0000 7f96dbfff640 1 --2- 172.21.15.12:0/3094501047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96dc10f150 0x7f96dc1a67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.281+0000 7f96db7fe640 1 -- 172.21.15.12:0/3094501047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96dc10f150 msgr2=0x7f96dc1a67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.281+0000 7f96db7fe640 1 --2- 172.21.15.12:0/3094501047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96dc10f150 0x7f96dc1a67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.281+0000 7f96db7fe640 1 -- 172.21.15.12:0/3094501047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96dc106170 msgr2=0x7f96dc10ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.281+0000 7f96db7fe640 1 --2- 172.21.15.12:0/3094501047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96dc106170 0x7f96dc10ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.281+0000 7f96db7fe640 1 -- 172.21.15.12:0/3094501047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96dc1a6ec0 con 0x7f96dc1010a0 2024-04-22T13:28:35.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.281+0000 7f96dbfff640 1 --2- 172.21.15.12:0/3094501047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96dc10f150 0x7f96dc1a67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:35.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.282+0000 7f96db7fe640 1 --2- 172.21.15.12:0/3094501047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96dc1010a0 0x7f96dc10e6d0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f96cc00c920 tx=0x7f96cc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:35.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.282+0000 7f96d8ff9640 1 -- 172.21.15.12:0/3094501047 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc007c70 con 0x7f96dc1010a0 2024-04-22T13:28:35.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.282+0000 7f96d8ff9640 1 -- 172.21.15.12:0/3094501047 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96cc007e10 con 0x7f96dc1010a0 2024-04-22T13:28:35.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.282+0000 7f96e1fb2640 1 -- 172.21.15.12:0/3094501047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96dc1a71b0 con 0x7f96dc1010a0 2024-04-22T13:28:35.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.282+0000 7f96d8ff9640 1 -- 172.21.15.12:0/3094501047 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc018820 con 0x7f96dc1010a0 2024-04-22T13:28:35.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.282+0000 7f96e1fb2640 1 -- 172.21.15.12:0/3094501047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96dc1a7700 con 0x7f96dc1010a0 2024-04-22T13:28:35.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.284+0000 7f96d8ff9640 1 -- 172.21.15.12:0/3094501047 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f96cc00f450 con 0x7f96dc1010a0 2024-04-22T13:28:35.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.284+0000 7f96e4ba0740 1 Processor -- start 2024-04-22T13:28:35.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.284+0000 7f96e4ba0740 1 -- start start 2024-04-22T13:28:35.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.284+0000 7f96d8ff9640 1 --2- 172.21.15.12:0/3094501047 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f96c40a3dd0 0x7f96c40a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.285+0000 7f96e4ba0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dccd23f4d0 0x55dccd23f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.285+0000 7f96e4ba0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd23fe80 0x55dccd248430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.285+0000 7f96dbfff640 1 --2- 172.21.15.12:0/3094501047 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f96c40a3dd0 0x7f96c40a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.285+0000 7f96e4ba0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dccd248a00 0x55dccd24ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.285+0000 7f96d8ff9640 1 -- 172.21.15.12:0/3094501047 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f96cc0d0950 con 0x7f96dc1010a0 2024-04-22T13:28:35.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.285+0000 7f96e4ba0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55dccd1a41f0 con 0x55dccd23fe80 2024-04-22T13:28:35.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.285+0000 7f96e4ba0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55dccd1414b0 con 0x55dccd248a00 2024-04-22T13:28:35.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.285+0000 7f96e4ba0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dccd1a01b0 con 0x55dccd23f4d0 2024-04-22T13:28:35.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.285+0000 7f96daffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd23fe80 0x55dccd248430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.285+0000 7f96daffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd23fe80 0x55dccd248430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33034/0 (socket says 172.21.15.12:33034) 2024-04-22T13:28:35.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.285+0000 7f96daffd640 1 -- 172.21.15.12:0/931940226 learned_addr learned my addr 172.21.15.12:0/931940226 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:35.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.285+0000 7f96dbfff640 1 --2- 172.21.15.12:0/931940226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dccd248a00 0x55dccd24ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.285+0000 7f96db7fe640 1 --2- 172.21.15.12:0/931940226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dccd23f4d0 0x55dccd23f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.285+0000 7f96daffd640 1 -- 172.21.15.12:0/931940226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dccd23f4d0 msgr2=0x55dccd23f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.285+0000 7f96daffd640 1 --2- 172.21.15.12:0/931940226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dccd23f4d0 0x55dccd23f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.285+0000 7f96daffd640 1 -- 172.21.15.12:0/931940226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dccd248a00 msgr2=0x55dccd24ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.285+0000 7f96daffd640 1 --2- 172.21.15.12:0/931940226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dccd248a00 0x55dccd24ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.285+0000 7f96daffd640 1 -- 172.21.15.12:0/931940226 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dccd1331b0 con 0x55dccd23fe80 2024-04-22T13:28:35.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.285+0000 7f96db7fe640 1 --2- 172.21.15.12:0/931940226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dccd23f4d0 0x55dccd23f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:35.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.285+0000 7f96dbfff640 1 --2- 172.21.15.12:0/3094501047 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f96c40a3dd0 0x7f96c40a6290 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f96d4006010 tx=0x7f96d4009940 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:35.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.285+0000 7f96daffd640 1 --2- 172.21.15.12:0/931940226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd23fe80 0x55dccd248430 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f96d000bbf0 tx=0x7f96d00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568b68dfdaf8501f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:35.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.285+0000 7f96c1ffb640 1 -- 172.21.15.12:0/931940226 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d0005020 con 0x55dccd23fe80 2024-04-22T13:28:35.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.286+0000 7f96c1ffb640 1 -- 172.21.15.12:0/931940226 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96d00047a0 con 0x55dccd23fe80 2024-04-22T13:28:35.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.286+0000 7f96c1ffb640 1 -- 172.21.15.12:0/931940226 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d0011680 con 0x55dccd23fe80 2024-04-22T13:28:35.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.286+0000 7f96e4ba0740 1 -- 172.21.15.12:0/931940226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd23fe80 msgr2=0x55dccd248430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.286+0000 7f96e4ba0740 1 --2- 172.21.15.12:0/931940226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd23fe80 0x55dccd248430 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f96d000bbf0 tx=0x7f96d00107a0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.286+0000 7f96e4ba0740 1 -- 172.21.15.12:0/931940226 shutdown_connections 2024-04-22T13:28:35.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.286+0000 7f96e4ba0740 1 --2- 172.21.15.12:0/931940226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dccd248a00 0x55dccd24ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.286+0000 7f96e4ba0740 1 --2- 172.21.15.12:0/931940226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd23fe80 0x55dccd248430 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.286+0000 7f96e4ba0740 1 --2- 172.21.15.12:0/931940226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dccd23f4d0 0x55dccd23f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.286+0000 7f96e4ba0740 1 -- 172.21.15.12:0/931940226 >> 172.21.15.12:0/931940226 conn(0x55dccd239ba0 msgr2=0x55dccd239f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:35.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.286+0000 7f96e4ba0740 1 -- 172.21.15.12:0/931940226 shutdown_connections 2024-04-22T13:28:35.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.286+0000 7f96e4ba0740 1 -- 172.21.15.12:0/931940226 wait complete. 2024-04-22T13:28:35.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.287+0000 7f96e4ba0740 1 -- start start 2024-04-22T13:28:35.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.287+0000 7f96e4ba0740 1 Processor -- start 2024-04-22T13:28:35.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.287+0000 7f96e4ba0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dccd23f4d0 0x55dccd2482e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.287+0000 7f96e4ba0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd23fe80 0x55dccd245670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.287+0000 7f96e4ba0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dccd248a00 0x55dccd245bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.287+0000 7f96e4ba0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55dccd24b970 con 0x55dccd23fe80 2024-04-22T13:28:35.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.287+0000 7f96e4ba0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55dccd15d500 con 0x55dccd248a00 2024-04-22T13:28:35.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.287+0000 7f96e4ba0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dccd24baf0 con 0x55dccd23f4d0 2024-04-22T13:28:35.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.287+0000 7f96e4ba0740 10 client.0 _ll_register_callbacks cb 0x55dccd12bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:35.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.287+0000 7f96daffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd23fe80 0x55dccd245670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.287+0000 7f96daffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd23fe80 0x55dccd245670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33044/0 (socket says 172.21.15.12:33044) 2024-04-22T13:28:35.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.287+0000 7f96daffd640 1 -- 172.21.15.12:0/476808891 learned_addr learned my addr 172.21.15.12:0/476808891 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:35.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.287+0000 7f96db7fe640 1 --2- 172.21.15.12:0/476808891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dccd23f4d0 0x55dccd2482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.287+0000 7f96dbfff640 1 --2- 172.21.15.12:0/476808891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dccd248a00 0x55dccd245bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.287+0000 7f96daffd640 1 -- 172.21.15.12:0/476808891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dccd23f4d0 msgr2=0x55dccd2482e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.287+0000 7f96daffd640 1 --2- 172.21.15.12:0/476808891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dccd23f4d0 0x55dccd2482e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.287+0000 7f96daffd640 1 -- 172.21.15.12:0/476808891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dccd248a00 msgr2=0x55dccd245bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.287+0000 7f96daffd640 1 --2- 172.21.15.12:0/476808891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dccd248a00 0x55dccd245bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.287+0000 7f96daffd640 1 -- 172.21.15.12:0/476808891 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dccceb63f0 con 0x55dccd23fe80 2024-04-22T13:28:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.287+0000 7f96db7fe640 1 --2- 172.21.15.12:0/476808891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dccd23f4d0 0x55dccd2482e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.288+0000 7f96daffd640 1 --2- 172.21.15.12:0/476808891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd23fe80 0x55dccd245670 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f96d001a000 tx=0x7f96d000b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947985f29aae3370 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.288+0000 7f96baffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.288+0000 7f96baffd640 1 -- 172.21.15.12:0/476808891 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d002b060 con 0x55dccd23fe80 2024-04-22T13:28:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.288+0000 7f96baffd640 1 -- 172.21.15.12:0/476808891 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f96d0004e70 con 0x55dccd23fe80 2024-04-22T13:28:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.288+0000 7f96e4ba0740 1 -- 172.21.15.12:0/476808891 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96d00047a0 con 0x55dccd23fe80 2024-04-22T13:28:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.288+0000 7f96baffd640 1 -- 172.21.15.12:0/476808891 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f96d0005070 con 0x55dccd23fe80 2024-04-22T13:28:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.288+0000 7f96e4ba0740 10 client.17343 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.288+0000 7f96e4ba0740 1 -- 172.21.15.12:0/476808891 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dcccf807d0 con 0x55dccd23fe80 2024-04-22T13:28:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.288+0000 7f96baffd640 1 -- 172.21.15.12:0/476808891 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96d00047a0 con 0x55dccd23fe80 2024-04-22T13:28:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.288+0000 7f96baffd640 1 -- 172.21.15.12:0/476808891 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f96d0011b70 con 0x55dccd23fe80 2024-04-22T13:28:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.288+0000 7f96e4ba0740 10 client.17343 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:35.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.288+0000 7f96e4ba0740 10 client.17343 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:28:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.288+0000 7f96e4ba0740 20 client.17343 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.288+0000 7f96e4ba0740 1 --2- 172.21.15.12:0/476808891 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55dccd258a80 0x55dccd25ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.288+0000 7f96e4ba0740 4 client.17343 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.288+0000 7f96e4ba0740 1 -- 172.21.15.12:0/476808891 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dccd143930 con 0x55dccd258a80 2024-04-22T13:28:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.289+0000 7f96db7fe640 1 --2- 172.21.15.12:0/476808891 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55dccd258a80 0x55dccd25ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.289+0000 7f96db7fe640 1 --2- 172.21.15.12:0/476808891 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55dccd258a80 0x55dccd25ae70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c60b23b7f282d6d2 server_cookie=4af40c708cea7d80 in_seq=0 out_seq=0 2024-04-22T13:28:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.289+0000 7f96baffd640 10 client.17343 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:28:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.290+0000 7f96baffd640 1 -- 172.21.15.12:0/476808891 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dccd143930 con 0x55dccd258a80 2024-04-22T13:28:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.290+0000 7f96baffd640 10 client.17343 handle_command_reply: tid=0 2024-04-22T13:28:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.290+0000 7f96e4ba0740 1 client.17343 shutdown 2024-04-22T13:28:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.290+0000 7f96e4ba0740 1 -- 172.21.15.12:0/476808891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd23fe80 msgr2=0x55dccd245670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.290+0000 7f96e4ba0740 1 --2- 172.21.15.12:0/476808891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd23fe80 0x55dccd245670 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f96d001a000 tx=0x7f96d000b920 comp rx=0 tx=0).stop 2024-04-22T13:28:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.290+0000 7f96e4ba0740 1 -- 172.21.15.12:0/476808891 shutdown_connections 2024-04-22T13:28:35.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.290+0000 7f96e4ba0740 1 --2- 172.21.15.12:0/476808891 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55dccd258a80 0x55dccd25ae70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.290+0000 7f96e4ba0740 1 --2- 172.21.15.12:0/476808891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dccd248a00 0x55dccd245bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.290+0000 7f96e4ba0740 1 --2- 172.21.15.12:0/476808891 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd23fe80 0x55dccd245670 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.290+0000 7f96baffd640 0 client.17343 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:28:35.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.290+0000 7f96e4ba0740 1 --2- 172.21.15.12:0/476808891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dccd23f4d0 0x55dccd2482e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.290+0000 7f96e4ba0740 1 -- 172.21.15.12:0/476808891 >> 172.21.15.12:0/476808891 conn(0x55dccd239ba0 msgr2=0x55dccd23b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:35.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.290+0000 7f96e4ba0740 1 -- 172.21.15.12:0/476808891 shutdown_connections 2024-04-22T13:28:35.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.290+0000 7f96e4ba0740 1 -- 172.21.15.12:0/476808891 wait complete. 2024-04-22T13:28:35.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.290+0000 7f96e4ba0740 20 client.17343 trim_cache size 0 max 16384 2024-04-22T13:28:35.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.304+0000 7f96e4ba0740 1 Processor -- start 2024-04-22T13:28:35.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.304+0000 7f96e4ba0740 1 -- start start 2024-04-22T13:28:35.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.304+0000 7f96e4ba0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dccd23f4d0 0x55dccd254830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.304+0000 7f96e4ba0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd23fe80 0x55dccd247da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.304+0000 7f96e4ba0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dccd248a00 0x55dccd2482e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.304+0000 7f96e4ba0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc1afb30 con 0x55dccd23fe80 2024-04-22T13:28:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.304+0000 7f96e4ba0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc1b1670 con 0x55dccd23f4d0 2024-04-22T13:28:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.304+0000 7f96e4ba0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc1b0200 con 0x55dccd248a00 2024-04-22T13:28:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.304+0000 7f96daffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd23fe80 0x55dccd247da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.304+0000 7f96daffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd23fe80 0x55dccd247da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33056/0 (socket says 172.21.15.12:33056) 2024-04-22T13:28:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.304+0000 7f96daffd640 1 -- 172.21.15.12:0/329549102 learned_addr learned my addr 172.21.15.12:0/329549102 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.304+0000 7f96daffd640 1 -- 172.21.15.12:0/329549102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dccd248a00 msgr2=0x55dccd2482e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.304+0000 7f96daffd640 1 --2- 172.21.15.12:0/329549102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dccd248a00 0x55dccd2482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.304+0000 7f96daffd640 1 -- 172.21.15.12:0/329549102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dccd23f4d0 msgr2=0x55dccd254830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.304+0000 7f96daffd640 1 --2- 172.21.15.12:0/329549102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dccd23f4d0 0x55dccd254830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.304+0000 7f96daffd640 1 -- 172.21.15.12:0/329549102 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dccd239380 con 0x55dccd23fe80 2024-04-22T13:28:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.304+0000 7f96daffd640 1 --2- 172.21.15.12:0/329549102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd23fe80 0x55dccd247da0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f96d000b620 tx=0x7f96d00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7421e31369005d81 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:35.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.304+0000 7f96c17fa640 1 -- 172.21.15.12:0/329549102 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d001f070 con 0x55dccd23fe80 2024-04-22T13:28:35.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.304+0000 7f96c17fa640 1 -- 172.21.15.12:0/329549102 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96d0005210 con 0x55dccd23fe80 2024-04-22T13:28:35.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.304+0000 7f96c17fa640 1 -- 172.21.15.12:0/329549102 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d002b040 con 0x55dccd23fe80 2024-04-22T13:28:35.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.305+0000 7f96e4ba0740 1 -- 172.21.15.12:0/329549102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd23fe80 msgr2=0x55dccd247da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.305+0000 7f96e4ba0740 1 --2- 172.21.15.12:0/329549102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd23fe80 0x55dccd247da0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f96d000b620 tx=0x7f96d00107a0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.305+0000 7f96e4ba0740 1 -- 172.21.15.12:0/329549102 shutdown_connections 2024-04-22T13:28:35.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.305+0000 7f96e4ba0740 1 --2- 172.21.15.12:0/329549102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dccd248a00 0x55dccd2482e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.305+0000 7f96e4ba0740 1 --2- 172.21.15.12:0/329549102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd23fe80 0x55dccd247da0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.305+0000 7f96e4ba0740 1 --2- 172.21.15.12:0/329549102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dccd23f4d0 0x55dccd254830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.305+0000 7f96e4ba0740 1 -- 172.21.15.12:0/329549102 >> 172.21.15.12:0/329549102 conn(0x55dccd239ba0 msgr2=0x55dccd240280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:35.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.305+0000 7f96e4ba0740 1 -- 172.21.15.12:0/329549102 shutdown_connections 2024-04-22T13:28:35.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.305+0000 7f96e4ba0740 1 -- 172.21.15.12:0/329549102 wait complete. 2024-04-22T13:28:35.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.305+0000 7f96e4ba0740 1 -- start start 2024-04-22T13:28:35.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.305+0000 7f96e4ba0740 1 Processor -- start 2024-04-22T13:28:35.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.305+0000 7f96e4ba0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dccd23f4d0 0x55dccd2510e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.305+0000 7f96e4ba0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dccd23fe80 0x55dccd241fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.305+0000 7f96e4ba0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd248a00 0x55dccd2424e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.305+0000 7f96e4ba0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc1b0fa0 con 0x55dccd248a00 2024-04-22T13:28:35.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.305+0000 7f96e4ba0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc1b08d0 con 0x55dccd23f4d0 2024-04-22T13:28:35.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.305+0000 7f96e4ba0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc1adb10 con 0x55dccd23fe80 2024-04-22T13:28:35.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.306+0000 7f96e4ba0740 10 client.0 _ll_register_callbacks cb 0x7f96d000b800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:35.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.306+0000 7f96dbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd248a00 0x55dccd2424e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.306+0000 7f96dbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd248a00 0x55dccd2424e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33062/0 (socket says 172.21.15.12:33062) 2024-04-22T13:28:35.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.306+0000 7f96dbfff640 1 -- 172.21.15.12:0/3474128742 learned_addr learned my addr 172.21.15.12:0/3474128742 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:35.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.306+0000 7f96db7fe640 1 --2- 172.21.15.12:0/3474128742 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dccd23f4d0 0x55dccd2510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.306+0000 7f96daffd640 1 --2- 172.21.15.12:0/3474128742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dccd23fe80 0x55dccd241fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.306+0000 7f96dbfff640 1 -- 172.21.15.12:0/3474128742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dccd23fe80 msgr2=0x55dccd241fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.306+0000 7f96dbfff640 1 --2- 172.21.15.12:0/3474128742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dccd23fe80 0x55dccd241fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.306+0000 7f96dbfff640 1 -- 172.21.15.12:0/3474128742 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dccd23f4d0 msgr2=0x55dccd2510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.306+0000 7f96dbfff640 1 --2- 172.21.15.12:0/3474128742 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dccd23f4d0 0x55dccd2510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.308 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.306+0000 7f96dbfff640 1 -- 172.21.15.12:0/3474128742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dccd242cc0 con 0x55dccd248a00 2024-04-22T13:28:35.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.306+0000 7f96dbfff640 1 --2- 172.21.15.12:0/3474128742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd248a00 0x55dccd2424e0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f96d4009ca0 tx=0x7f96d40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b216cf4726d06d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:35.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.306+0000 7f96baffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:35.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.306+0000 7f96baffd640 1 -- 172.21.15.12:0/3474128742 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d4046070 con 0x55dccd248a00 2024-04-22T13:28:35.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.306+0000 7f96baffd640 1 -- 172.21.15.12:0/3474128742 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f96d4004450 con 0x55dccd248a00 2024-04-22T13:28:35.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.306+0000 7f96e4ba0740 1 -- 172.21.15.12:0/3474128742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96d0005210 con 0x55dccd248a00 2024-04-22T13:28:35.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.306+0000 7f96baffd640 1 -- 172.21.15.12:0/3474128742 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f96d405b050 con 0x55dccd248a00 2024-04-22T13:28:35.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.306+0000 7f96e4ba0740 10 client.17349 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:35.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.307+0000 7f96e4ba0740 1 -- 172.21.15.12:0/3474128742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dccd2bec00 con 0x55dccd248a00 2024-04-22T13:28:35.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.307+0000 7f96baffd640 1 -- 172.21.15.12:0/3474128742 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96d0005210 con 0x55dccd248a00 2024-04-22T13:28:35.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.307+0000 7f96baffd640 1 -- 172.21.15.12:0/3474128742 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f96d404dac0 con 0x55dccd248a00 2024-04-22T13:28:35.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.307+0000 7f96e4ba0740 10 client.17349 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:35.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.307+0000 7f96e4ba0740 10 client.17349 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:28:35.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.307+0000 7f96e4ba0740 20 client.17349 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:35.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.307+0000 7f96e4ba0740 1 --2- 172.21.15.12:0/3474128742 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55dccd258a80 0x55dccd2c13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.307+0000 7f96e4ba0740 4 client.17349 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:28:35.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.307+0000 7f96e4ba0740 1 -- 172.21.15.12:0/3474128742 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dccd1b1c10 con 0x55dccd258a80 2024-04-22T13:28:35.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.307+0000 7f96db7fe640 1 --2- 172.21.15.12:0/3474128742 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55dccd258a80 0x55dccd2c13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.307+0000 7f96baffd640 10 client.17349 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:28:35.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.307+0000 7f96db7fe640 1 --2- 172.21.15.12:0/3474128742 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55dccd258a80 0x55dccd2c13e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dfd1f5e6a389bfd9 server_cookie=b8ad6f1e220184f1 in_seq=0 out_seq=0 2024-04-22T13:28:35.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.308+0000 7f96baffd640 1 -- 172.21.15.12:0/3474128742 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dccd1b1c10 con 0x55dccd258a80 2024-04-22T13:28:35.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.308+0000 7f96baffd640 10 client.17349 handle_command_reply: tid=0 2024-04-22T13:28:35.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.308+0000 7f96e4ba0740 1 client.17349 shutdown 2024-04-22T13:28:35.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.308+0000 7f96e4ba0740 1 -- 172.21.15.12:0/3474128742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd248a00 msgr2=0x55dccd2424e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.308+0000 7f96e4ba0740 1 --2- 172.21.15.12:0/3474128742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd248a00 0x55dccd2424e0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f96d4009ca0 tx=0x7f96d40095c0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.308+0000 7f96e4ba0740 1 -- 172.21.15.12:0/3474128742 shutdown_connections 2024-04-22T13:28:35.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.308+0000 7f96e4ba0740 1 --2- 172.21.15.12:0/3474128742 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55dccd258a80 0x55dccd2c13e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.308+0000 7f96e4ba0740 1 --2- 172.21.15.12:0/3474128742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dccd248a00 0x55dccd2424e0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.308+0000 7f96e4ba0740 1 --2- 172.21.15.12:0/3474128742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dccd23fe80 0x55dccd241fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.308+0000 7f96baffd640 0 client.17349 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:28:35.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.308+0000 7f96e4ba0740 1 --2- 172.21.15.12:0/3474128742 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dccd23f4d0 0x55dccd2510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.308+0000 7f96e4ba0740 1 -- 172.21.15.12:0/3474128742 >> 172.21.15.12:0/3474128742 conn(0x55dccd239ba0 msgr2=0x55dccd238c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:35.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.308+0000 7f96e4ba0740 1 -- 172.21.15.12:0/3474128742 shutdown_connections 2024-04-22T13:28:35.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.308+0000 7f96e4ba0740 1 -- 172.21.15.12:0/3474128742 wait complete. 2024-04-22T13:28:35.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.308+0000 7f96e4ba0740 20 client.17349 trim_cache size 0 max 16384 2024-04-22T13:28:35.311 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:28:35.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.308+0000 7f96e1fb2640 1 -- 172.21.15.12:0/3094501047 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f96c40a3dd0 msgr2=0x7f96c40a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.308+0000 7f96e1fb2640 1 --2- 172.21.15.12:0/3094501047 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f96c40a3dd0 0x7f96c40a6290 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f96d4006010 tx=0x7f96d4009940 comp rx=0 tx=0).stop 2024-04-22T13:28:35.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.308+0000 7f96e1fb2640 1 -- 172.21.15.12:0/3094501047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96dc1010a0 msgr2=0x7f96dc10e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.308+0000 7f96e1fb2640 1 --2- 172.21.15.12:0/3094501047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96dc1010a0 0x7f96dc10e6d0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f96cc00c920 tx=0x7f96cc00cde0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.308+0000 7f96e1fb2640 1 -- 172.21.15.12:0/3094501047 shutdown_connections 2024-04-22T13:28:35.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.308+0000 7f96e1fb2640 1 --2- 172.21.15.12:0/3094501047 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f96c40a3dd0 0x7f96c40a6290 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.308+0000 7f96e1fb2640 1 --2- 172.21.15.12:0/3094501047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96dc10f150 0x7f96dc1a67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.308+0000 7f96e1fb2640 1 --2- 172.21.15.12:0/3094501047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96dc106170 0x7f96dc10ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.308+0000 7f96e1fb2640 1 --2- 172.21.15.12:0/3094501047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f96dc1010a0 0x7f96dc10e6d0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.309+0000 7f96e1fb2640 1 -- 172.21.15.12:0/3094501047 >> 172.21.15.12:0/3094501047 conn(0x7f96dc077eb0 msgr2=0x7f96dc0fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:35.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.309+0000 7f96e1fb2640 1 -- 172.21.15.12:0/3094501047 shutdown_connections 2024-04-22T13:28:35.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.309+0000 7f96e1fb2640 1 -- 172.21.15.12:0/3094501047 wait complete. 2024-04-22T13:28:35.316 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:28:35.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.424+0000 7f855366d640 1 -- 172.21.15.12:0/3342787411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f854c06a840 msgr2=0x7f854c06ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.424+0000 7f855366d640 1 --2- 172.21.15.12:0/3342787411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f854c06a840 0x7f854c06ac20 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f8548009900 tx=0x7f854802bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:35.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.424+0000 7f855366d640 1 -- 172.21.15.12:0/3342787411 shutdown_connections 2024-04-22T13:28:35.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.424+0000 7f855366d640 1 --2- 172.21.15.12:0/3342787411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f854c107270 0x7f854c10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.424+0000 7f855366d640 1 --2- 172.21.15.12:0/3342787411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f854c06b1f0 0x7f854c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.424+0000 7f855366d640 1 --2- 172.21.15.12:0/3342787411 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f854c06a840 0x7f854c06ac20 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.424+0000 7f855366d640 1 -- 172.21.15.12:0/3342787411 >> 172.21.15.12:0/3342787411 conn(0x7f854c077eb0 msgr2=0x7f854c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:35.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.424+0000 7f855366d640 1 -- 172.21.15.12:0/3342787411 shutdown_connections 2024-04-22T13:28:35.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.424+0000 7f855366d640 1 -- 172.21.15.12:0/3342787411 wait complete. 2024-04-22T13:28:35.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.424+0000 7f855366d640 1 Processor -- start 2024-04-22T13:28:35.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.425+0000 7f855366d640 1 -- start start 2024-04-22T13:28:35.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.425+0000 7f855366d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f854c06a840 0x7f854c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.425+0000 7f855366d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f854c107270 0x7f854c198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.425+0000 7f855366d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f854c19cfd0 0x7f854c19d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.425+0000 7f855366d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f854c1121b0 con 0x7f854c06a840 2024-04-22T13:28:35.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.425+0000 7f855366d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f854c112030 con 0x7f854c107270 2024-04-22T13:28:35.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.425+0000 7f855366d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f854c112330 con 0x7f854c19cfd0 2024-04-22T13:28:35.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.425+0000 7f8551409640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f854c06a840 0x7f854c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.425+0000 7f8551409640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f854c06a840 0x7f854c198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33084/0 (socket says 172.21.15.12:33084) 2024-04-22T13:28:35.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.425+0000 7f8551409640 1 -- 172.21.15.12:0/2591895335 learned_addr learned my addr 172.21.15.12:0/2591895335 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:35.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.425+0000 7f8551c0a640 1 --2- 172.21.15.12:0/2591895335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f854c19cfd0 0x7f854c19d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.425+0000 7f8550c08640 1 --2- 172.21.15.12:0/2591895335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f854c107270 0x7f854c198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.425+0000 7f8551409640 1 -- 172.21.15.12:0/2591895335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f854c19cfd0 msgr2=0x7f854c19d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.425+0000 7f8551409640 1 --2- 172.21.15.12:0/2591895335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f854c19cfd0 0x7f854c19d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.425+0000 7f8551409640 1 -- 172.21.15.12:0/2591895335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f854c107270 msgr2=0x7f854c198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.425+0000 7f8551409640 1 --2- 172.21.15.12:0/2591895335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f854c107270 0x7f854c198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.425+0000 7f8551409640 1 -- 172.21.15.12:0/2591895335 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854c19dac0 con 0x7f854c06a840 2024-04-22T13:28:35.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.425+0000 7f8550c08640 1 --2- 172.21.15.12:0/2591895335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f854c107270 0x7f854c198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:35.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.425+0000 7f8551409640 1 --2- 172.21.15.12:0/2591895335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f854c06a840 0x7f854c198710 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f8548005940 tx=0x7f854802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:35.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.426+0000 7f85427fc640 1 -- 172.21.15.12:0/2591895335 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8548043020 con 0x7f854c06a840 2024-04-22T13:28:35.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.426+0000 7f85427fc640 1 -- 172.21.15.12:0/2591895335 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f854802cbe0 con 0x7f854c06a840 2024-04-22T13:28:35.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.426+0000 7f85427fc640 1 -- 172.21.15.12:0/2591895335 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8548003bf0 con 0x7f854c06a840 2024-04-22T13:28:35.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.426+0000 7f855366d640 1 -- 172.21.15.12:0/2591895335 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f854c19dd50 con 0x7f854c06a840 2024-04-22T13:28:35.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.426+0000 7f855366d640 1 -- 172.21.15.12:0/2591895335 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f854c1a5520 con 0x7f854c06a840 2024-04-22T13:28:35.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.428+0000 7f85427fc640 1 -- 172.21.15.12:0/2591895335 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8548046020 con 0x7f854c06a840 2024-04-22T13:28:35.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.428+0000 7f8556269740 1 Processor -- start 2024-04-22T13:28:35.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.428+0000 7f8556269740 1 -- start start 2024-04-22T13:28:35.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.429+0000 7f8556269740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563464af24d0 0x563464af28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.429+0000 7f8556269740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563464af2e80 0x563464afb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.429+0000 7f8556269740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563464afba00 0x563464afde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.429+0000 7f8556269740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563464a571f0 con 0x563464af24d0 2024-04-22T13:28:35.432 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.429+0000 7f8556269740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5634649f44b0 con 0x563464af2e80 2024-04-22T13:28:35.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.429+0000 7f8556269740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563464a531b0 con 0x563464afba00 2024-04-22T13:28:35.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.429+0000 7f8551c0a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563464af24d0 0x563464af28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.429+0000 7f8551c0a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563464af24d0 0x563464af28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33086/0 (socket says 172.21.15.12:33086) 2024-04-22T13:28:35.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.429+0000 7f8551c0a640 1 -- 172.21.15.12:0/218721968 learned_addr learned my addr 172.21.15.12:0/218721968 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:35.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.429+0000 7f8551409640 1 --2- 172.21.15.12:0/218721968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563464af2e80 0x563464afb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.429+0000 7f85427fc640 1 --2- 172.21.15.12:0/2591895335 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f85380a3d20 0x7f85380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.429+0000 7f8550c08640 1 --2- 172.21.15.12:0/218721968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563464afba00 0x563464afde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.429+0000 7f8551c0a640 1 -- 172.21.15.12:0/218721968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563464afba00 msgr2=0x563464afde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.429+0000 7f8551c0a640 1 --2- 172.21.15.12:0/218721968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563464afba00 0x563464afde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.429+0000 7f8551c0a640 1 -- 172.21.15.12:0/218721968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563464af2e80 msgr2=0x563464afb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.429+0000 7f8551c0a640 1 --2- 172.21.15.12:0/218721968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563464af2e80 0x563464afb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.429+0000 7f8551c0a640 1 -- 172.21.15.12:0/218721968 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634649e61b0 con 0x563464af24d0 2024-04-22T13:28:35.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.429+0000 7f8550c08640 1 --2- 172.21.15.12:0/218721968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563464afba00 0x563464afde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:35.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.429+0000 7f85427fc640 1 -- 172.21.15.12:0/2591895335 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f85480f2920 con 0x7f854c06a840 2024-04-22T13:28:35.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.429+0000 7f8551c0a640 1 --2- 172.21.15.12:0/2591895335 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f85380a3d20 0x7f85380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.429+0000 7f8551c0a640 1 --2- 172.21.15.12:0/218721968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563464af24d0 0x563464af28b0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f8544010440 tx=0x7f8544010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2404bfaeda70c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:35.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.430+0000 7f85377fe640 1 -- 172.21.15.12:0/218721968 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854401d400 con 0x563464af24d0 2024-04-22T13:28:35.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.430+0000 7f85377fe640 1 -- 172.21.15.12:0/218721968 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f854401da50 con 0x563464af24d0 2024-04-22T13:28:35.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.430+0000 7f85377fe640 1 -- 172.21.15.12:0/218721968 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854401c760 con 0x563464af24d0 2024-04-22T13:28:35.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.430+0000 7f8556269740 1 -- 172.21.15.12:0/218721968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563464af24d0 msgr2=0x563464af28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.430+0000 7f8556269740 1 --2- 172.21.15.12:0/218721968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563464af24d0 0x563464af28b0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f8544010440 tx=0x7f8544010900 comp rx=0 tx=0).stop 2024-04-22T13:28:35.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.430+0000 7f8551c0a640 1 --2- 172.21.15.12:0/2591895335 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f85380a3d20 0x7f85380a61e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8544021000 tx=0x7f854401e000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:35.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.430+0000 7f8556269740 1 -- 172.21.15.12:0/218721968 shutdown_connections 2024-04-22T13:28:35.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.430+0000 7f8556269740 1 --2- 172.21.15.12:0/218721968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563464afba00 0x563464afde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.430+0000 7f8556269740 1 --2- 172.21.15.12:0/218721968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563464af2e80 0x563464afb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.430+0000 7f8556269740 1 --2- 172.21.15.12:0/218721968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563464af24d0 0x563464af28b0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.430+0000 7f8556269740 1 -- 172.21.15.12:0/218721968 >> 172.21.15.12:0/218721968 conn(0x563464aecba0 msgr2=0x563464aecf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:35.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.430+0000 7f8556269740 1 -- 172.21.15.12:0/218721968 shutdown_connections 2024-04-22T13:28:35.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.430+0000 7f8556269740 1 -- 172.21.15.12:0/218721968 wait complete. 2024-04-22T13:28:35.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.431+0000 7f8556269740 1 -- start start 2024-04-22T13:28:35.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.431+0000 7f8556269740 1 Processor -- start 2024-04-22T13:28:35.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.431+0000 7f8556269740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563464af24d0 0x563464afb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.431+0000 7f8556269740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563464af2e80 0x563464af86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.431+0000 7f8556269740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563464afba00 0x563464af8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.431+0000 7f8556269740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563464afe970 con 0x563464af24d0 2024-04-22T13:28:35.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.431+0000 7f8556269740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x563464a10500 con 0x563464afba00 2024-04-22T13:28:35.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.431+0000 7f8556269740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563464afeaf0 con 0x563464af2e80 2024-04-22T13:28:35.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.431+0000 7f8551409640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563464af24d0 0x563464afb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.431+0000 7f8556269740 10 client.0 _ll_register_callbacks cb 0x5634649deb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:35.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.431+0000 7f8551409640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563464af24d0 0x563464afb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33102/0 (socket says 172.21.15.12:33102) 2024-04-22T13:28:35.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.431+0000 7f8551409640 1 -- 172.21.15.12:0/1268220324 learned_addr learned my addr 172.21.15.12:0/1268220324 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:35.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.432+0000 7f8550c08640 1 --2- 172.21.15.12:0/1268220324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563464af2e80 0x563464af86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.432+0000 7f8551409640 1 -- 172.21.15.12:0/1268220324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563464af2e80 msgr2=0x563464af86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.432+0000 7f8551409640 1 --2- 172.21.15.12:0/1268220324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563464af2e80 0x563464af86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.432+0000 7f8551c0a640 1 --2- 172.21.15.12:0/1268220324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563464afba00 0x563464af8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.432+0000 7f8551409640 1 -- 172.21.15.12:0/1268220324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563464afba00 msgr2=0x563464af8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.432+0000 7f8551409640 1 --2- 172.21.15.12:0/1268220324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563464afba00 0x563464af8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.432+0000 7f8551409640 1 -- 172.21.15.12:0/1268220324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634647693f0 con 0x563464af24d0 2024-04-22T13:28:35.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.432+0000 7f8550c08640 1 --2- 172.21.15.12:0/1268220324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563464af2e80 0x563464af86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:35.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.432+0000 7f8551409640 1 --2- 172.21.15.12:0/1268220324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563464af24d0 0x563464afb310 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f85480f0230 tx=0x7f85480035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596ac88f7d80ca4c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:35.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.432+0000 7f8534ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:35.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.432+0000 7f8534ff9640 1 -- 172.21.15.12:0/1268220324 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85480be560 con 0x563464af24d0 2024-04-22T13:28:35.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.432+0000 7f8534ff9640 1 -- 172.21.15.12:0/1268220324 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f85480bb090 con 0x563464af24d0 2024-04-22T13:28:35.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.432+0000 7f8556269740 1 -- 172.21.15.12:0/1268220324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f854401da50 con 0x563464af24d0 2024-04-22T13:28:35.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.432+0000 7f8534ff9640 1 -- 172.21.15.12:0/1268220324 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f85480be380 con 0x563464af24d0 2024-04-22T13:28:35.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.432+0000 7f8556269740 10 client.17361 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:35.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.432+0000 7f8556269740 1 -- 172.21.15.12:0/1268220324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5634648337d0 con 0x563464af24d0 2024-04-22T13:28:35.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.432+0000 7f8534ff9640 1 -- 172.21.15.12:0/1268220324 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f854401da50 con 0x563464af24d0 2024-04-22T13:28:35.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.432+0000 7f8534ff9640 1 -- 172.21.15.12:0/1268220324 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f85480bec80 con 0x563464af24d0 2024-04-22T13:28:35.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.433+0000 7f8556269740 10 client.17361 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:35.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.433+0000 7f8556269740 10 client.17361 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:28:35.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.433+0000 7f8556269740 20 client.17361 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:35.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.433+0000 7f8556269740 1 --2- 172.21.15.12:0/1268220324 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x563464b0bbf0 0x563464b0dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.433+0000 7f8556269740 4 client.17361 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:35.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.433+0000 7f8556269740 1 -- 172.21.15.12:0/1268220324 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634649f6930 con 0x563464b0bbf0 2024-04-22T13:28:35.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.433+0000 7f8550c08640 1 --2- 172.21.15.12:0/1268220324 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x563464b0bbf0 0x563464b0dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.433+0000 7f8534ff9640 10 client.17361 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:28:35.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.433+0000 7f8550c08640 1 --2- 172.21.15.12:0/1268220324 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x563464b0bbf0 0x563464b0dfe0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8ef02bceb7d9b35a server_cookie=a68b10734da05243 in_seq=0 out_seq=0 2024-04-22T13:28:35.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.434+0000 7f8534ff9640 1 -- 172.21.15.12:0/1268220324 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5634649f6930 con 0x563464b0bbf0 2024-04-22T13:28:35.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.434+0000 7f8534ff9640 10 client.17361 handle_command_reply: tid=0 2024-04-22T13:28:35.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.434+0000 7f8556269740 1 client.17361 shutdown 2024-04-22T13:28:35.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.434+0000 7f8556269740 1 -- 172.21.15.12:0/1268220324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563464af24d0 msgr2=0x563464afb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.434+0000 7f8556269740 1 --2- 172.21.15.12:0/1268220324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563464af24d0 0x563464afb310 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f85480f0230 tx=0x7f85480035f0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.434+0000 7f8556269740 1 -- 172.21.15.12:0/1268220324 shutdown_connections 2024-04-22T13:28:35.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.434+0000 7f8556269740 1 --2- 172.21.15.12:0/1268220324 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x563464b0bbf0 0x563464b0dfe0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.434+0000 7f8556269740 1 --2- 172.21.15.12:0/1268220324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563464afba00 0x563464af8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.438 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.434+0000 7f8534ff9640 0 client.17361 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:28:35.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.434+0000 7f8556269740 1 --2- 172.21.15.12:0/1268220324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563464af2e80 0x563464af86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.434+0000 7f8556269740 1 --2- 172.21.15.12:0/1268220324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563464af24d0 0x563464afb310 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.434+0000 7f8556269740 1 -- 172.21.15.12:0/1268220324 >> 172.21.15.12:0/1268220324 conn(0x563464aecba0 msgr2=0x563464aee8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:35.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.434+0000 7f8556269740 1 -- 172.21.15.12:0/1268220324 shutdown_connections 2024-04-22T13:28:35.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.434+0000 7f8556269740 1 -- 172.21.15.12:0/1268220324 wait complete. 2024-04-22T13:28:35.439 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.434+0000 7f8556269740 20 client.17361 trim_cache size 0 max 16384 2024-04-22T13:28:35.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.448+0000 7f8556269740 1 Processor -- start 2024-04-22T13:28:35.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.448+0000 7f8556269740 1 -- start start 2024-04-22T13:28:35.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.448+0000 7f8556269740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563464af24d0 0x563464b07830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.448+0000 7f8556269740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563464af2e80 0x563464b138f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.448+0000 7f8556269740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563464afba00 0x563464b13e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.448+0000 7f8556269740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f854c1ada20 con 0x563464af2e80 2024-04-22T13:28:35.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.448+0000 7f8556269740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f854c1af560 con 0x563464af24d0 2024-04-22T13:28:35.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.448+0000 7f8556269740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f854c1ae0f0 con 0x563464afba00 2024-04-22T13:28:35.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.448+0000 7f8550c08640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563464af2e80 0x563464b138f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.448+0000 7f8550c08640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563464af2e80 0x563464b138f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33112/0 (socket says 172.21.15.12:33112) 2024-04-22T13:28:35.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.448+0000 7f8550c08640 1 -- 172.21.15.12:0/3267336427 learned_addr learned my addr 172.21.15.12:0/3267336427 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:35.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.448+0000 7f8550c08640 1 -- 172.21.15.12:0/3267336427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563464afba00 msgr2=0x563464b13e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.448+0000 7f8550c08640 1 --2- 172.21.15.12:0/3267336427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563464afba00 0x563464b13e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.448+0000 7f8550c08640 1 -- 172.21.15.12:0/3267336427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563464af24d0 msgr2=0x563464b07830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.448+0000 7f8550c08640 1 --2- 172.21.15.12:0/3267336427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563464af24d0 0x563464b07830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.448+0000 7f8550c08640 1 -- 172.21.15.12:0/3267336427 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563464af2c20 con 0x563464af2e80 2024-04-22T13:28:35.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.448+0000 7f8550c08640 1 --2- 172.21.15.12:0/3267336427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563464af2e80 0x563464b138f0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f853c00bd70 tx=0x7f853c00b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b8fe9f7dc7edbc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:35.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.448+0000 7f8536ffd640 1 -- 172.21.15.12:0/3267336427 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c014110 con 0x563464af2e80 2024-04-22T13:28:35.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.448+0000 7f8536ffd640 1 -- 172.21.15.12:0/3267336427 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f853c0142b0 con 0x563464af2e80 2024-04-22T13:28:35.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.448+0000 7f8536ffd640 1 -- 172.21.15.12:0/3267336427 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c01b5b0 con 0x563464af2e80 2024-04-22T13:28:35.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.449+0000 7f8556269740 1 -- 172.21.15.12:0/3267336427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563464af2e80 msgr2=0x563464b138f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.449+0000 7f8556269740 1 --2- 172.21.15.12:0/3267336427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563464af2e80 0x563464b138f0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f853c00bd70 tx=0x7f853c00b7b0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.449+0000 7f8556269740 1 -- 172.21.15.12:0/3267336427 shutdown_connections 2024-04-22T13:28:35.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.449+0000 7f8556269740 1 --2- 172.21.15.12:0/3267336427 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563464afba00 0x563464b13e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.449+0000 7f8556269740 1 --2- 172.21.15.12:0/3267336427 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563464af2e80 0x563464b138f0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.449+0000 7f8556269740 1 --2- 172.21.15.12:0/3267336427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563464af24d0 0x563464b07830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.449+0000 7f8556269740 1 -- 172.21.15.12:0/3267336427 >> 172.21.15.12:0/3267336427 conn(0x563464aecba0 msgr2=0x563464aee860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:35.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.449+0000 7f8556269740 1 -- 172.21.15.12:0/3267336427 shutdown_connections 2024-04-22T13:28:35.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.449+0000 7f8556269740 1 -- 172.21.15.12:0/3267336427 wait complete. 2024-04-22T13:28:35.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.449+0000 7f8556269740 1 -- start start 2024-04-22T13:28:35.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.449+0000 7f8556269740 1 Processor -- start 2024-04-22T13:28:35.451 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.450+0000 7f8556269740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563464af24d0 0x563464aff9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.450+0000 7f8556269740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563464af2e80 0x563464afff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.450+0000 7f8556269740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563464afba00 0x563464b00470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.450+0000 7f8556269740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f854c1aee90 con 0x563464afba00 2024-04-22T13:28:35.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.450+0000 7f8556269740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f854c1ae7c0 con 0x563464af24d0 2024-04-22T13:28:35.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.450+0000 7f8556269740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f854c1aba20 con 0x563464af2e80 2024-04-22T13:28:35.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.450+0000 7f8556269740 10 client.0 _ll_register_callbacks cb 0x7f85480057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:35.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.450+0000 7f8551c0a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563464afba00 0x563464b00470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.450+0000 7f8551c0a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563464afba00 0x563464b00470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33128/0 (socket says 172.21.15.12:33128) 2024-04-22T13:28:35.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.450+0000 7f8551c0a640 1 -- 172.21.15.12:0/385617303 learned_addr learned my addr 172.21.15.12:0/385617303 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:35.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.450+0000 7f8550c08640 1 --2- 172.21.15.12:0/385617303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563464af2e80 0x563464afff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.450+0000 7f8551409640 1 --2- 172.21.15.12:0/385617303 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563464af24d0 0x563464aff9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.450+0000 7f8551c0a640 1 -- 172.21.15.12:0/385617303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563464af2e80 msgr2=0x563464afff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.450+0000 7f8551c0a640 1 --2- 172.21.15.12:0/385617303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563464af2e80 0x563464afff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.450+0000 7f8551c0a640 1 -- 172.21.15.12:0/385617303 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563464af24d0 msgr2=0x563464aff9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.450+0000 7f8551c0a640 1 --2- 172.21.15.12:0/385617303 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563464af24d0 0x563464aff9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.450+0000 7f8551c0a640 1 -- 172.21.15.12:0/385617303 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563464b71400 con 0x563464afba00 2024-04-22T13:28:35.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.450+0000 7f8551c0a640 1 --2- 172.21.15.12:0/385617303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563464afba00 0x563464b00470 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f8544011010 tx=0x7f8544013750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a4be8819227a71 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:35.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.450+0000 7f8534ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:35.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.451+0000 7f8534ff9640 1 -- 172.21.15.12:0/385617303 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8544037060 con 0x563464afba00 2024-04-22T13:28:35.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.451+0000 7f8534ff9640 1 -- 172.21.15.12:0/385617303 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f854401c3a0 con 0x563464afba00 2024-04-22T13:28:35.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.451+0000 7f8556269740 1 -- 172.21.15.12:0/385617303 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563464a64c10 con 0x563464afba00 2024-04-22T13:28:35.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.451+0000 7f8534ff9640 1 -- 172.21.15.12:0/385617303 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f854402e040 con 0x563464afba00 2024-04-22T13:28:35.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.451+0000 7f8556269740 10 client.17367 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:35.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.451+0000 7f8556269740 1 -- 172.21.15.12:0/385617303 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563464b72700 con 0x563464afba00 2024-04-22T13:28:35.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.451+0000 7f8534ff9640 1 -- 172.21.15.12:0/385617303 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563464a64c10 con 0x563464afba00 2024-04-22T13:28:35.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.451+0000 7f8534ff9640 1 -- 172.21.15.12:0/385617303 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f854400f040 con 0x563464afba00 2024-04-22T13:28:35.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.451+0000 7f8556269740 10 client.17367 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:35.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.451+0000 7f8556269740 10 client.17367 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:28:35.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.451+0000 7f8556269740 20 client.17367 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:35.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.451+0000 7f8556269740 1 --2- 172.21.15.12:0/385617303 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x563464b72cb0 0x563464b750d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.451+0000 7f8556269740 4 client.17367 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:28:35.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.451+0000 7f8556269740 1 -- 172.21.15.12:0/385617303 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563464a5ea80 con 0x563464b72cb0 2024-04-22T13:28:35.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.451+0000 7f8551409640 1 --2- 172.21.15.12:0/385617303 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x563464b72cb0 0x563464b750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.452+0000 7f8551409640 1 --2- 172.21.15.12:0/385617303 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x563464b72cb0 0x563464b750d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fdd8ab2540397fdc server_cookie=7dd930ec84d2f332 in_seq=0 out_seq=0 2024-04-22T13:28:35.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.452+0000 7f8534ff9640 10 client.17367 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:28:35.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.452+0000 7f8534ff9640 1 -- 172.21.15.12:0/385617303 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563464a5ea80 con 0x563464b72cb0 2024-04-22T13:28:35.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.452+0000 7f8534ff9640 10 client.17367 handle_command_reply: tid=0 2024-04-22T13:28:35.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.452+0000 7f8556269740 1 client.17367 shutdown 2024-04-22T13:28:35.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.452+0000 7f8556269740 1 -- 172.21.15.12:0/385617303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563464afba00 msgr2=0x563464b00470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.452+0000 7f8556269740 1 --2- 172.21.15.12:0/385617303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563464afba00 0x563464b00470 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f8544011010 tx=0x7f8544013750 comp rx=0 tx=0).stop 2024-04-22T13:28:35.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.452+0000 7f8556269740 1 -- 172.21.15.12:0/385617303 shutdown_connections 2024-04-22T13:28:35.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.452+0000 7f8556269740 1 --2- 172.21.15.12:0/385617303 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x563464b72cb0 0x563464b750d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.452+0000 7f8556269740 1 --2- 172.21.15.12:0/385617303 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563464afba00 0x563464b00470 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.452+0000 7f8556269740 1 --2- 172.21.15.12:0/385617303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563464af2e80 0x563464afff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.452+0000 7f8534ff9640 0 client.17367 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:28:35.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.452+0000 7f8556269740 1 --2- 172.21.15.12:0/385617303 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563464af24d0 0x563464aff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.452+0000 7f8556269740 1 -- 172.21.15.12:0/385617303 >> 172.21.15.12:0/385617303 conn(0x563464aecba0 msgr2=0x563464aedfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:35.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.452+0000 7f8556269740 1 -- 172.21.15.12:0/385617303 shutdown_connections 2024-04-22T13:28:35.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.452+0000 7f8556269740 1 -- 172.21.15.12:0/385617303 wait complete. 2024-04-22T13:28:35.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.452+0000 7f8556269740 20 client.17367 trim_cache size 0 max 16384 2024-04-22T13:28:35.456 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:28:35.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.453+0000 7f855366d640 1 -- 172.21.15.12:0/2591895335 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f85380a3d20 msgr2=0x7f85380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.453+0000 7f855366d640 1 --2- 172.21.15.12:0/2591895335 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f85380a3d20 0x7f85380a61e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8544021000 tx=0x7f854401e000 comp rx=0 tx=0).stop 2024-04-22T13:28:35.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.453+0000 7f855366d640 1 -- 172.21.15.12:0/2591895335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f854c06a840 msgr2=0x7f854c198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.453+0000 7f855366d640 1 --2- 172.21.15.12:0/2591895335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f854c06a840 0x7f854c198710 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f8548005940 tx=0x7f854802c7e0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.453+0000 7f855366d640 1 -- 172.21.15.12:0/2591895335 shutdown_connections 2024-04-22T13:28:35.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.453+0000 7f855366d640 1 --2- 172.21.15.12:0/2591895335 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f85380a3d20 0x7f85380a61e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.453+0000 7f855366d640 1 --2- 172.21.15.12:0/2591895335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f854c19cfd0 0x7f854c19d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.453+0000 7f855366d640 1 --2- 172.21.15.12:0/2591895335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f854c107270 0x7f854c198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.453+0000 7f855366d640 1 --2- 172.21.15.12:0/2591895335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f854c06a840 0x7f854c198710 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.453+0000 7f855366d640 1 -- 172.21.15.12:0/2591895335 >> 172.21.15.12:0/2591895335 conn(0x7f854c077eb0 msgr2=0x7f854c108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:35.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.453+0000 7f855366d640 1 -- 172.21.15.12:0/2591895335 shutdown_connections 2024-04-22T13:28:35.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.453+0000 7f855366d640 1 -- 172.21.15.12:0/2591895335 wait complete. 2024-04-22T13:28:35.460 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:28:35.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.568+0000 7fbd3138f640 1 -- 172.21.15.12:0/749882097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd2c107260 msgr2=0x7fbd2c10ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.568+0000 7fbd3138f640 1 --2- 172.21.15.12:0/749882097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd2c107260 0x7fbd2c10ee30 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fbd24009900 tx=0x7fbd2402bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:35.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.568+0000 7fbd3138f640 1 -- 172.21.15.12:0/749882097 shutdown_connections 2024-04-22T13:28:35.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.568+0000 7fbd3138f640 1 --2- 172.21.15.12:0/749882097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd2c107260 0x7fbd2c10ee30 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.568+0000 7fbd3138f640 1 --2- 172.21.15.12:0/749882097 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd2c102b40 0x7fbd2c106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.568+0000 7fbd3138f640 1 --2- 172.21.15.12:0/749882097 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd2c102190 0x7fbd2c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.568+0000 7fbd3138f640 1 -- 172.21.15.12:0/749882097 >> 172.21.15.12:0/749882097 conn(0x7fbd2c0fd990 msgr2=0x7fbd2c0ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:35.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.568+0000 7fbd3138f640 1 -- 172.21.15.12:0/749882097 shutdown_connections 2024-04-22T13:28:35.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.568+0000 7fbd3138f640 1 -- 172.21.15.12:0/749882097 wait complete. 2024-04-22T13:28:35.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.568+0000 7fbd3138f640 1 Processor -- start 2024-04-22T13:28:35.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.569+0000 7fbd3138f640 1 -- start start 2024-04-22T13:28:35.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.569+0000 7fbd3138f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd2c102190 0x7fbd2c198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.569+0000 7fbd3138f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd2c107260 0x7fbd2c198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.569+0000 7fbd3138f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd2c19cfe0 0x7fbd2c19d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.569+0000 7fbd3138f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c112200 con 0x7fbd2c107260 2024-04-22T13:28:35.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.569+0000 7fbd3138f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c112080 con 0x7fbd2c19cfe0 2024-04-22T13:28:35.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.569+0000 7fbd3138f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c112380 con 0x7fbd2c102190 2024-04-22T13:28:35.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.569+0000 7fbd2a7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd2c107260 0x7fbd2c198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.569+0000 7fbd2a7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd2c107260 0x7fbd2c198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33148/0 (socket says 172.21.15.12:33148) 2024-04-22T13:28:35.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.569+0000 7fbd2a7fc640 1 -- 172.21.15.12:0/1635342 learned_addr learned my addr 172.21.15.12:0/1635342 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:35.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.569+0000 7fbd2affd640 1 --2- 172.21.15.12:0/1635342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd2c102190 0x7fbd2c198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.569+0000 7fbd2b7fe640 1 --2- 172.21.15.12:0/1635342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd2c19cfe0 0x7fbd2c19d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.569+0000 7fbd2a7fc640 1 -- 172.21.15.12:0/1635342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd2c102190 msgr2=0x7fbd2c198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.569+0000 7fbd2a7fc640 1 --2- 172.21.15.12:0/1635342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd2c102190 0x7fbd2c198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.569+0000 7fbd2a7fc640 1 -- 172.21.15.12:0/1635342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd2c19cfe0 msgr2=0x7fbd2c19d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.569+0000 7fbd2a7fc640 1 --2- 172.21.15.12:0/1635342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd2c19cfe0 0x7fbd2c19d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.569+0000 7fbd2a7fc640 1 -- 172.21.15.12:0/1635342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd2c19dad0 con 0x7fbd2c107260 2024-04-22T13:28:35.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.569+0000 7fbd2affd640 1 --2- 172.21.15.12:0/1635342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd2c102190 0x7fbd2c198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:35.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.569+0000 7fbd2a7fc640 1 --2- 172.21.15.12:0/1635342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd2c107260 0x7fbd2c198c60 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000e9c0 tx=0x7fbd2000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:35.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.570+0000 7fbd13fff640 1 -- 172.21.15.12:0/1635342 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2000cd80 con 0x7fbd2c107260 2024-04-22T13:28:35.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.570+0000 7fbd13fff640 1 -- 172.21.15.12:0/1635342 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd200054d0 con 0x7fbd2c107260 2024-04-22T13:28:35.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.570+0000 7fbd3138f640 1 -- 172.21.15.12:0/1635342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd2c19ddc0 con 0x7fbd2c107260 2024-04-22T13:28:35.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.570+0000 7fbd13fff640 1 -- 172.21.15.12:0/1635342 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd20010430 con 0x7fbd2c107260 2024-04-22T13:28:35.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.570+0000 7fbd3138f640 1 -- 172.21.15.12:0/1635342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd2c1a55b0 con 0x7fbd2c107260 2024-04-22T13:28:35.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.572+0000 7fbd13fff640 1 -- 172.21.15.12:0/1635342 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbd20004fd0 con 0x7fbd2c107260 2024-04-22T13:28:35.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.572+0000 7fbd33f7f740 1 Processor -- start 2024-04-22T13:28:35.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.572+0000 7fbd33f7f740 1 -- start start 2024-04-22T13:28:35.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.573+0000 7fbd33f7f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b87f8c3330 0x55b87f8c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.573+0000 7fbd33f7f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b87f8c3ce0 0x55b87f8cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.573+0000 7fbd33f7f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f8cce40 0x55b87f8cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.573+0000 7fbd33f7f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b87f8281f0 con 0x55b87f8c3330 2024-04-22T13:28:35.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.573+0000 7fbd33f7f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b87f7c54b0 con 0x55b87f8c3ce0 2024-04-22T13:28:35.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.573+0000 7fbd33f7f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b87f8241b0 con 0x55b87f8cce40 2024-04-22T13:28:35.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.573+0000 7fbd2b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b87f8c3330 0x55b87f8c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.573+0000 7fbd13fff640 1 --2- 172.21.15.12:0/1635342 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbd140a3d20 0x7fbd140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.573+0000 7fbd2b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b87f8c3330 0x55b87f8c3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33154/0 (socket says 172.21.15.12:33154) 2024-04-22T13:28:35.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.573+0000 7fbd2b7fe640 1 -- 172.21.15.12:0/3349854700 learned_addr learned my addr 172.21.15.12:0/3349854700 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:35.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.573+0000 7fbd2affd640 1 --2- 172.21.15.12:0/3349854700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b87f8c3ce0 0x55b87f8cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.573+0000 7fbd2b7fe640 1 --2- 172.21.15.12:0/1635342 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbd140a3d20 0x7fbd140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.573+0000 7fbd13fff640 1 -- 172.21.15.12:0/1635342 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fbd200d0c30 con 0x7fbd2c107260 2024-04-22T13:28:35.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.573+0000 7fbd2a7fc640 1 --2- 172.21.15.12:0/3349854700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f8cce40 0x55b87f8cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.573+0000 7fbd2b7fe640 1 -- 172.21.15.12:0/3349854700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f8cce40 msgr2=0x55b87f8cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.573+0000 7fbd2b7fe640 1 --2- 172.21.15.12:0/3349854700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f8cce40 0x55b87f8cd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.573+0000 7fbd2b7fe640 1 -- 172.21.15.12:0/3349854700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b87f8c3ce0 msgr2=0x55b87f8cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.573+0000 7fbd2b7fe640 1 --2- 172.21.15.12:0/3349854700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b87f8c3ce0 0x55b87f8cc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.573+0000 7fbd2b7fe640 1 -- 172.21.15.12:0/3349854700 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b87f7b71b0 con 0x55b87f8c3330 2024-04-22T13:28:35.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.573+0000 7fbd2b7fe640 1 --2- 172.21.15.12:0/3349854700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b87f8c3330 0x55b87f8c3710 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fbd2402c780 tx=0x7fbd240043e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c11cc6f98d8829 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:35.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.574+0000 7fbd117fa640 1 -- 172.21.15.12:0/3349854700 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd24005040 con 0x55b87f8c3330 2024-04-22T13:28:35.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.574+0000 7fbd117fa640 1 -- 172.21.15.12:0/3349854700 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd24005690 con 0x55b87f8c3330 2024-04-22T13:28:35.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.574+0000 7fbd117fa640 1 -- 172.21.15.12:0/3349854700 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd240035f0 con 0x55b87f8c3330 2024-04-22T13:28:35.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.574+0000 7fbd33f7f740 1 -- 172.21.15.12:0/3349854700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b87f8c3330 msgr2=0x55b87f8c3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.574+0000 7fbd2b7fe640 1 --2- 172.21.15.12:0/1635342 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbd140a3d20 0x7fbd140a61e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbd2404e000 tx=0x7fbd2404a000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:35.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.574+0000 7fbd33f7f740 1 --2- 172.21.15.12:0/3349854700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b87f8c3330 0x55b87f8c3710 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fbd2402c780 tx=0x7fbd240043e0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.574+0000 7fbd33f7f740 1 -- 172.21.15.12:0/3349854700 shutdown_connections 2024-04-22T13:28:35.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.574+0000 7fbd33f7f740 1 --2- 172.21.15.12:0/3349854700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f8cce40 0x55b87f8cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.574+0000 7fbd33f7f740 1 --2- 172.21.15.12:0/3349854700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b87f8c3ce0 0x55b87f8cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.574+0000 7fbd33f7f740 1 --2- 172.21.15.12:0/3349854700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b87f8c3330 0x55b87f8c3710 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.574+0000 7fbd33f7f740 1 -- 172.21.15.12:0/3349854700 >> 172.21.15.12:0/3349854700 conn(0x55b87f8bdba0 msgr2=0x55b87f8bdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:35.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.574+0000 7fbd33f7f740 1 -- 172.21.15.12:0/3349854700 shutdown_connections 2024-04-22T13:28:35.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.574+0000 7fbd33f7f740 1 -- 172.21.15.12:0/3349854700 wait complete. 2024-04-22T13:28:35.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.575+0000 7fbd33f7f740 1 -- start start 2024-04-22T13:28:35.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.575+0000 7fbd33f7f740 1 Processor -- start 2024-04-22T13:28:35.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.575+0000 7fbd33f7f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b87f8c3330 0x55b87f8d3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.575+0000 7fbd33f7f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b87f8c3ce0 0x55b87f8d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.575+0000 7fbd33f7f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f8cce40 0x55b87f8d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.575+0000 7fbd33f7f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b87f8d3dc0 con 0x55b87f8c3330 2024-04-22T13:28:35.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.575+0000 7fbd33f7f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b87f7e1500 con 0x55b87f8c3ce0 2024-04-22T13:28:35.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.575+0000 7fbd33f7f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b87f8d3f40 con 0x55b87f8cce40 2024-04-22T13:28:35.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.575+0000 7fbd33f7f740 10 client.0 _ll_register_callbacks cb 0x55b87f7afb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:35.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.575+0000 7fbd2affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b87f8c3330 0x55b87f8d3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.575+0000 7fbd2affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b87f8c3330 0x55b87f8d3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33168/0 (socket says 172.21.15.12:33168) 2024-04-22T13:28:35.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.575+0000 7fbd2affd640 1 -- 172.21.15.12:0/1740237465 learned_addr learned my addr 172.21.15.12:0/1740237465 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:35.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.575+0000 7fbd2a7fc640 1 --2- 172.21.15.12:0/1740237465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b87f8c3ce0 0x55b87f8d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.575+0000 7fbd2b7fe640 1 --2- 172.21.15.12:0/1740237465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f8cce40 0x55b87f8d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.575+0000 7fbd2affd640 1 -- 172.21.15.12:0/1740237465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f8cce40 msgr2=0x55b87f8d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.575+0000 7fbd2affd640 1 --2- 172.21.15.12:0/1740237465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f8cce40 0x55b87f8d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.575+0000 7fbd2affd640 1 -- 172.21.15.12:0/1740237465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b87f8c3ce0 msgr2=0x55b87f8d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.575+0000 7fbd2affd640 1 --2- 172.21.15.12:0/1740237465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b87f8c3ce0 0x55b87f8d05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.575+0000 7fbd2affd640 1 -- 172.21.15.12:0/1740237465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b87f53a3f0 con 0x55b87f8c3330 2024-04-22T13:28:35.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.575+0000 7fbd2b7fe640 1 --2- 172.21.15.12:0/1740237465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f8cce40 0x55b87f8d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:35.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.576+0000 7fbd2affd640 1 --2- 172.21.15.12:0/1740237465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b87f8c3330 0x55b87f8d3240 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c003c20 tx=0x7fbd1c004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67c39cfe97e34fbf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:35.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.576+0000 7fbd0eff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:35.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.576+0000 7fbd0eff5640 1 -- 172.21.15.12:0/1740237465 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c004d20 con 0x55b87f8c3330 2024-04-22T13:28:35.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.576+0000 7fbd0eff5640 1 -- 172.21.15.12:0/1740237465 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fbd1c011650 con 0x55b87f8c3330 2024-04-22T13:28:35.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.576+0000 7fbd33f7f740 1 -- 172.21.15.12:0/1740237465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b87f7c7930 con 0x55b87f8c3330 2024-04-22T13:28:35.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.576+0000 7fbd0eff5640 1 -- 172.21.15.12:0/1740237465 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbd1c011890 con 0x55b87f8c3330 2024-04-22T13:28:35.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.576+0000 7fbd33f7f740 10 client.17379 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:35.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.576+0000 7fbd33f7f740 1 -- 172.21.15.12:0/1740237465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b87f6047d0 con 0x55b87f8c3330 2024-04-22T13:28:35.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.576+0000 7fbd0eff5640 1 -- 172.21.15.12:0/1740237465 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b87f7c7930 con 0x55b87f8c3330 2024-04-22T13:28:35.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.576+0000 7fbd0eff5640 1 -- 172.21.15.12:0/1740237465 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbd1c019030 con 0x55b87f8c3330 2024-04-22T13:28:35.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.576+0000 7fbd33f7f740 10 client.17379 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:35.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.576+0000 7fbd33f7f740 10 client.17379 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:28:35.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.576+0000 7fbd33f7f740 20 client.17379 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:35.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.576+0000 7fbd33f7f740 1 --2- 172.21.15.12:0/1740237465 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55b87f8e1010 0x55b87f8e3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.576+0000 7fbd33f7f740 4 client.17379 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:35.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.576+0000 7fbd33f7f740 1 -- 172.21.15.12:0/1740237465 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b87f835c10 con 0x55b87f8e1010 2024-04-22T13:28:35.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.577+0000 7fbd2a7fc640 1 --2- 172.21.15.12:0/1740237465 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55b87f8e1010 0x55b87f8e3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.577+0000 7fbd2a7fc640 1 --2- 172.21.15.12:0/1740237465 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55b87f8e1010 0x55b87f8e3460 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e839352d66c367fe server_cookie=288237925814730d in_seq=0 out_seq=0 2024-04-22T13:28:35.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.577+0000 7fbd0eff5640 10 client.17379 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:28:35.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.578+0000 7fbd0eff5640 1 -- 172.21.15.12:0/1740237465 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b87f835c10 con 0x55b87f8e1010 2024-04-22T13:28:35.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.578+0000 7fbd0eff5640 10 client.17379 handle_command_reply: tid=0 2024-04-22T13:28:35.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.578+0000 7fbd33f7f740 1 client.17379 shutdown 2024-04-22T13:28:35.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.578+0000 7fbd33f7f740 1 -- 172.21.15.12:0/1740237465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b87f8c3330 msgr2=0x55b87f8d3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.578+0000 7fbd33f7f740 1 --2- 172.21.15.12:0/1740237465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b87f8c3330 0x55b87f8d3240 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c003c20 tx=0x7fbd1c004410 comp rx=0 tx=0).stop 2024-04-22T13:28:35.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.578+0000 7fbd33f7f740 1 -- 172.21.15.12:0/1740237465 shutdown_connections 2024-04-22T13:28:35.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.578+0000 7fbd33f7f740 1 --2- 172.21.15.12:0/1740237465 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f8cce40 0x55b87f8d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.578+0000 7fbd33f7f740 1 --2- 172.21.15.12:0/1740237465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b87f8c3ce0 0x55b87f8d05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.578+0000 7fbd33f7f740 1 --2- 172.21.15.12:0/1740237465 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55b87f8e1010 0x55b87f8e3460 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.578+0000 7fbd33f7f740 1 --2- 172.21.15.12:0/1740237465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b87f8c3330 0x55b87f8d3240 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.578+0000 7fbd0eff5640 0 client.17379 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:28:35.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.578+0000 7fbd33f7f740 1 -- 172.21.15.12:0/1740237465 >> 172.21.15.12:0/1740237465 conn(0x55b87f8bdba0 msgr2=0x55b87f8bf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:35.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.578+0000 7fbd33f7f740 1 -- 172.21.15.12:0/1740237465 shutdown_connections 2024-04-22T13:28:35.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.578+0000 7fbd33f7f740 1 -- 172.21.15.12:0/1740237465 wait complete. 2024-04-22T13:28:35.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.578+0000 7fbd33f7f740 20 client.17379 trim_cache size 0 max 16384 2024-04-22T13:28:35.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.591+0000 7fbd33f7f740 1 Processor -- start 2024-04-22T13:28:35.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.591+0000 7fbd33f7f740 1 -- start start 2024-04-22T13:28:35.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.592+0000 7fbd33f7f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b87f8c3330 0x55b87f8dccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.592+0000 7fbd33f7f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f8c3ce0 0x55b87f8cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.592+0000 7fbd33f7f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b87f8cce40 0x55b87f8bca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.592+0000 7fbd33f7f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c1ab8e0 con 0x55b87f8c3330 2024-04-22T13:28:35.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.592+0000 7fbd33f7f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c1ad420 con 0x55b87f8cce40 2024-04-22T13:28:35.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.592+0000 7fbd33f7f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c1abfb0 con 0x55b87f8c3ce0 2024-04-22T13:28:35.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.592+0000 7fbd2affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b87f8c3330 0x55b87f8dccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.592+0000 7fbd2affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b87f8c3330 0x55b87f8dccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33184/0 (socket says 172.21.15.12:33184) 2024-04-22T13:28:35.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.592+0000 7fbd2affd640 1 -- 172.21.15.12:0/3443859517 learned_addr learned my addr 172.21.15.12:0/3443859517 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:35.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.592+0000 7fbd2b7fe640 1 --2- 172.21.15.12:0/3443859517 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b87f8cce40 0x55b87f8bca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.592+0000 7fbd2a7fc640 1 --2- 172.21.15.12:0/3443859517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f8c3ce0 0x55b87f8cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.592+0000 7fbd2affd640 1 -- 172.21.15.12:0/3443859517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f8c3ce0 msgr2=0x55b87f8cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.592+0000 7fbd2affd640 1 --2- 172.21.15.12:0/3443859517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f8c3ce0 0x55b87f8cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.592+0000 7fbd2affd640 1 -- 172.21.15.12:0/3443859517 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b87f8cce40 msgr2=0x55b87f8bca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.592+0000 7fbd2affd640 1 --2- 172.21.15.12:0/3443859517 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b87f8cce40 0x55b87f8bca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.592+0000 7fbd2affd640 1 -- 172.21.15.12:0/3443859517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b87f8c2560 con 0x55b87f8c3330 2024-04-22T13:28:35.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.592+0000 7fbd2a7fc640 1 --2- 172.21.15.12:0/3443859517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f8c3ce0 0x55b87f8cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:35.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.592+0000 7fbd2affd640 1 --2- 172.21.15.12:0/3443859517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b87f8c3330 0x55b87f8dccc0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c003bf0 tx=0x7fbd1c005ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4543dba4bf03f8b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:35.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.592+0000 7fbd10ff9640 1 -- 172.21.15.12:0/3443859517 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c011650 con 0x55b87f8c3330 2024-04-22T13:28:35.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.592+0000 7fbd10ff9640 1 -- 172.21.15.12:0/3443859517 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd1c01a3f0 con 0x55b87f8c3330 2024-04-22T13:28:35.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.592+0000 7fbd10ff9640 1 -- 172.21.15.12:0/3443859517 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c01a710 con 0x55b87f8c3330 2024-04-22T13:28:35.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.593+0000 7fbd33f7f740 1 -- 172.21.15.12:0/3443859517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b87f8c3330 msgr2=0x55b87f8dccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.593+0000 7fbd33f7f740 1 --2- 172.21.15.12:0/3443859517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b87f8c3330 0x55b87f8dccc0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c003bf0 tx=0x7fbd1c005ba0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.593+0000 7fbd33f7f740 1 -- 172.21.15.12:0/3443859517 shutdown_connections 2024-04-22T13:28:35.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.593+0000 7fbd33f7f740 1 --2- 172.21.15.12:0/3443859517 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b87f8cce40 0x55b87f8bca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.593+0000 7fbd33f7f740 1 --2- 172.21.15.12:0/3443859517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f8c3ce0 0x55b87f8cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.593+0000 7fbd33f7f740 1 --2- 172.21.15.12:0/3443859517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b87f8c3330 0x55b87f8dccc0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.593+0000 7fbd33f7f740 1 -- 172.21.15.12:0/3443859517 >> 172.21.15.12:0/3443859517 conn(0x55b87f8bdba0 msgr2=0x55b87f8bc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:35.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.593+0000 7fbd33f7f740 1 -- 172.21.15.12:0/3443859517 shutdown_connections 2024-04-22T13:28:35.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.593+0000 7fbd33f7f740 1 -- 172.21.15.12:0/3443859517 wait complete. 2024-04-22T13:28:35.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.593+0000 7fbd33f7f740 1 -- start start 2024-04-22T13:28:35.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.593+0000 7fbd33f7f740 1 Processor -- start 2024-04-22T13:28:35.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.594+0000 7fbd33f7f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f8c3330 0x55b87f8d2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.594+0000 7fbd33f7f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b87f8c3ce0 0x55b87f8d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.594+0000 7fbd33f7f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b87f8cce40 0x55b87f8d2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.594+0000 7fbd33f7f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c1acd50 con 0x55b87f8c3ce0 2024-04-22T13:28:35.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.594+0000 7fbd33f7f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c1ac680 con 0x55b87f8cce40 2024-04-22T13:28:35.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.594+0000 7fbd33f7f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c1a98e0 con 0x55b87f8c3330 2024-04-22T13:28:35.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.594+0000 7fbd33f7f740 10 client.0 _ll_register_callbacks cb 0x7fbd1c009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:35.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.594+0000 7fbd2a7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b87f8c3ce0 0x55b87f8d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.594+0000 7fbd2a7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b87f8c3ce0 0x55b87f8d27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33194/0 (socket says 172.21.15.12:33194) 2024-04-22T13:28:35.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.594+0000 7fbd2a7fc640 1 -- 172.21.15.12:0/1295226160 learned_addr learned my addr 172.21.15.12:0/1295226160 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:35.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.594+0000 7fbd2affd640 1 --2- 172.21.15.12:0/1295226160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f8c3330 0x55b87f8d2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.594+0000 7fbd2a7fc640 1 -- 172.21.15.12:0/1295226160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f8c3330 msgr2=0x55b87f8d2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.594+0000 7fbd2a7fc640 1 --2- 172.21.15.12:0/1295226160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f8c3330 0x55b87f8d2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.594+0000 7fbd2a7fc640 1 -- 172.21.15.12:0/1295226160 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b87f8cce40 msgr2=0x55b87f8d2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.594+0000 7fbd2b7fe640 1 --2- 172.21.15.12:0/1295226160 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b87f8cce40 0x55b87f8d2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.594+0000 7fbd2a7fc640 1 --2- 172.21.15.12:0/1295226160 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b87f8cce40 0x55b87f8d2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.594+0000 7fbd2a7fc640 1 -- 172.21.15.12:0/1295226160 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b87f9465a0 con 0x55b87f8c3ce0 2024-04-22T13:28:35.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.594+0000 7fbd2b7fe640 1 --2- 172.21.15.12:0/1295226160 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b87f8cce40 0x55b87f8d2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:35.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.594+0000 7fbd2a7fc640 1 --2- 172.21.15.12:0/1295226160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b87f8c3ce0 0x55b87f8d27a0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fbd20004960 tx=0x7fbd20098ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a09f1153d3e1d7d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:35.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.594+0000 7fbd0eff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:35.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.594+0000 7fbd0eff5640 1 -- 172.21.15.12:0/1295226160 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd200a5830 con 0x55b87f8c3ce0 2024-04-22T13:28:35.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.594+0000 7fbd0eff5640 1 -- 172.21.15.12:0/1295226160 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fbd200bd0b0 con 0x55b87f8c3ce0 2024-04-22T13:28:35.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.594+0000 7fbd33f7f740 1 -- 172.21.15.12:0/1295226160 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd1c01a3f0 con 0x55b87f8c3ce0 2024-04-22T13:28:35.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.595+0000 7fbd0eff5640 1 -- 172.21.15.12:0/1295226160 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbd2009cbc0 con 0x55b87f8c3ce0 2024-04-22T13:28:35.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.595+0000 7fbd33f7f740 10 client.17385 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:35.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.595+0000 7fbd33f7f740 1 -- 172.21.15.12:0/1295226160 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b87f947840 con 0x55b87f8c3ce0 2024-04-22T13:28:35.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.595+0000 7fbd0eff5640 1 -- 172.21.15.12:0/1295226160 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd1c01a3f0 con 0x55b87f8c3ce0 2024-04-22T13:28:35.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.595+0000 7fbd0eff5640 1 -- 172.21.15.12:0/1295226160 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbd200a53a0 con 0x55b87f8c3ce0 2024-04-22T13:28:35.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.595+0000 7fbd33f7f740 10 client.17385 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:35.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.595+0000 7fbd33f7f740 10 client.17385 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:28:35.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.595+0000 7fbd33f7f740 20 client.17385 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:35.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.595+0000 7fbd33f7f740 1 --2- 172.21.15.12:0/1295226160 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55b87f947fa0 0x55b87f94a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.595+0000 7fbd33f7f740 4 client.17385 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:28:35.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.595+0000 7fbd33f7f740 1 -- 172.21.15.12:0/1295226160 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b87f82fa80 con 0x55b87f947fa0 2024-04-22T13:28:35.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.595+0000 7fbd2affd640 1 --2- 172.21.15.12:0/1295226160 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55b87f947fa0 0x55b87f94a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.596+0000 7fbd2affd640 1 --2- 172.21.15.12:0/1295226160 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55b87f947fa0 0x55b87f94a390 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b674d7670802c11e server_cookie=4c68aa262553f5f5 in_seq=0 out_seq=0 2024-04-22T13:28:35.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.596+0000 7fbd0eff5640 10 client.17385 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:28:35.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.596+0000 7fbd0eff5640 1 -- 172.21.15.12:0/1295226160 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b87f82fa80 con 0x55b87f947fa0 2024-04-22T13:28:35.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.596+0000 7fbd0eff5640 10 client.17385 handle_command_reply: tid=0 2024-04-22T13:28:35.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.596+0000 7fbd33f7f740 1 client.17385 shutdown 2024-04-22T13:28:35.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.596+0000 7fbd33f7f740 1 -- 172.21.15.12:0/1295226160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b87f8c3ce0 msgr2=0x55b87f8d27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.596+0000 7fbd33f7f740 1 --2- 172.21.15.12:0/1295226160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b87f8c3ce0 0x55b87f8d27a0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fbd20004960 tx=0x7fbd20098ea0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.596+0000 7fbd33f7f740 1 -- 172.21.15.12:0/1295226160 shutdown_connections 2024-04-22T13:28:35.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.596+0000 7fbd33f7f740 1 --2- 172.21.15.12:0/1295226160 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b87f8cce40 0x55b87f8d2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.596+0000 7fbd33f7f740 1 --2- 172.21.15.12:0/1295226160 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55b87f947fa0 0x55b87f94a390 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.596+0000 7fbd33f7f740 1 --2- 172.21.15.12:0/1295226160 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b87f8c3ce0 0x55b87f8d27a0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.596+0000 7fbd33f7f740 1 --2- 172.21.15.12:0/1295226160 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b87f8c3330 0x55b87f8d2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.596+0000 7fbd0eff5640 0 client.17385 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:28:35.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.596+0000 7fbd33f7f740 1 -- 172.21.15.12:0/1295226160 >> 172.21.15.12:0/1295226160 conn(0x55b87f8bdba0 msgr2=0x55b87f8bc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:35.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.596+0000 7fbd33f7f740 1 -- 172.21.15.12:0/1295226160 shutdown_connections 2024-04-22T13:28:35.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.596+0000 7fbd33f7f740 1 -- 172.21.15.12:0/1295226160 wait complete. 2024-04-22T13:28:35.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.596+0000 7fbd33f7f740 20 client.17385 trim_cache size 0 max 16384 2024-04-22T13:28:35.600 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:28:35.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.597+0000 7fbd3138f640 1 -- 172.21.15.12:0/1635342 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbd140a3d20 msgr2=0x7fbd140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.597+0000 7fbd3138f640 1 --2- 172.21.15.12:0/1635342 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbd140a3d20 0x7fbd140a61e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbd2404e000 tx=0x7fbd2404a000 comp rx=0 tx=0).stop 2024-04-22T13:28:35.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.597+0000 7fbd3138f640 1 -- 172.21.15.12:0/1635342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd2c107260 msgr2=0x7fbd2c198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.597+0000 7fbd3138f640 1 --2- 172.21.15.12:0/1635342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd2c107260 0x7fbd2c198c60 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000e9c0 tx=0x7fbd2000ee80 comp rx=0 tx=0).stop 2024-04-22T13:28:35.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.597+0000 7fbd3138f640 1 -- 172.21.15.12:0/1635342 shutdown_connections 2024-04-22T13:28:35.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.597+0000 7fbd3138f640 1 --2- 172.21.15.12:0/1635342 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbd140a3d20 0x7fbd140a61e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.597+0000 7fbd3138f640 1 --2- 172.21.15.12:0/1635342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd2c19cfe0 0x7fbd2c19d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.597+0000 7fbd3138f640 1 --2- 172.21.15.12:0/1635342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd2c107260 0x7fbd2c198c60 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.597+0000 7fbd3138f640 1 --2- 172.21.15.12:0/1635342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd2c102190 0x7fbd2c198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.597+0000 7fbd3138f640 1 -- 172.21.15.12:0/1635342 >> 172.21.15.12:0/1635342 conn(0x7fbd2c0fd990 msgr2=0x7fbd2c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:35.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.597+0000 7fbd3138f640 1 -- 172.21.15.12:0/1635342 shutdown_connections 2024-04-22T13:28:35.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.597+0000 7fbd3138f640 1 -- 172.21.15.12:0/1635342 wait complete. 2024-04-22T13:28:35.605 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:28:35.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.712+0000 7fddad17e640 1 -- 172.21.15.12:0/2972691744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdda8101a50 msgr2=0x7fdda8105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.712+0000 7fddad17e640 1 --2- 172.21.15.12:0/2972691744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdda8101a50 0x7fdda8105a90 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c009900 tx=0x7fdd9c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:35.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.712+0000 7fddad17e640 1 -- 172.21.15.12:0/2972691744 shutdown_connections 2024-04-22T13:28:35.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.712+0000 7fddad17e640 1 --2- 172.21.15.12:0/2972691744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdda8106170 0x7fdda810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.712+0000 7fddad17e640 1 --2- 172.21.15.12:0/2972691744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdda8101a50 0x7fdda8105a90 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.712+0000 7fddad17e640 1 --2- 172.21.15.12:0/2972691744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdda81010a0 0x7fdda8101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.712+0000 7fddad17e640 1 -- 172.21.15.12:0/2972691744 >> 172.21.15.12:0/2972691744 conn(0x7fdda8077eb0 msgr2=0x7fdda80782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:35.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.712+0000 7fddad17e640 1 -- 172.21.15.12:0/2972691744 shutdown_connections 2024-04-22T13:28:35.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.712+0000 7fddad17e640 1 -- 172.21.15.12:0/2972691744 wait complete. 2024-04-22T13:28:35.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.713+0000 7fddad17e640 1 Processor -- start 2024-04-22T13:28:35.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.713+0000 7fddad17e640 1 -- start start 2024-04-22T13:28:35.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.713+0000 7fddad17e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdda81010a0 0x7fdda8197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.713+0000 7fddad17e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdda8106170 0x7fdda8197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.713+0000 7fddad17e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdda819bed0 0x7fdda819c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.713+0000 7fddad17e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdda81110b0 con 0x7fdda819bed0 2024-04-22T13:28:35.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.713+0000 7fddad17e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdda8110f30 con 0x7fdda81010a0 2024-04-22T13:28:35.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.713+0000 7fddad17e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdda8111230 con 0x7fdda8106170 2024-04-22T13:28:35.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.713+0000 7fdda759e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdda819bed0 0x7fdda819c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.714+0000 7fdda759e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdda819bed0 0x7fdda819c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33212/0 (socket says 172.21.15.12:33212) 2024-04-22T13:28:35.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.714+0000 7fdda759e640 1 -- 172.21.15.12:0/4127356365 learned_addr learned my addr 172.21.15.12:0/4127356365 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:35.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.714+0000 7fdda6d9d640 1 --2- 172.21.15.12:0/4127356365 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdda81010a0 0x7fdda8197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.714+0000 7fdda659c640 1 --2- 172.21.15.12:0/4127356365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdda8106170 0x7fdda8197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.714+0000 7fdda6d9d640 1 -- 172.21.15.12:0/4127356365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdda8106170 msgr2=0x7fdda8197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.714+0000 7fdda6d9d640 1 --2- 172.21.15.12:0/4127356365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdda8106170 0x7fdda8197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.714+0000 7fdda6d9d640 1 -- 172.21.15.12:0/4127356365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdda819bed0 msgr2=0x7fdda819c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.714+0000 7fdda6d9d640 1 --2- 172.21.15.12:0/4127356365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdda819bed0 0x7fdda819c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.714+0000 7fdda6d9d640 1 -- 172.21.15.12:0/4127356365 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda819c9c0 con 0x7fdda81010a0 2024-04-22T13:28:35.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.714+0000 7fdda759e640 1 --2- 172.21.15.12:0/4127356365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdda819bed0 0x7fdda819c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:35.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.714+0000 7fdda6d9d640 1 --2- 172.21.15.12:0/4127356365 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdda81010a0 0x7fdda8197610 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdd9800e9f0 tx=0x7fdd9800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:35.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.714+0000 7fdd8ffff640 1 -- 172.21.15.12:0/4127356365 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9800ccc0 con 0x7fdda81010a0 2024-04-22T13:28:35.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.714+0000 7fddad17e640 1 -- 172.21.15.12:0/4127356365 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdda819ccb0 con 0x7fdda81010a0 2024-04-22T13:28:35.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.714+0000 7fdd8ffff640 1 -- 172.21.15.12:0/4127356365 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd980108e0 con 0x7fdda81010a0 2024-04-22T13:28:35.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.714+0000 7fdd8ffff640 1 -- 172.21.15.12:0/4127356365 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98018880 con 0x7fdda81010a0 2024-04-22T13:28:35.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.714+0000 7fddad17e640 1 -- 172.21.15.12:0/4127356365 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda806a840 con 0x7fdda81010a0 2024-04-22T13:28:35.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.718+0000 7fdd8ffff640 1 -- 172.21.15.12:0/4127356365 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdd98010430 con 0x7fdda81010a0 2024-04-22T13:28:35.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.718+0000 7fddafd6f740 1 Processor -- start 2024-04-22T13:28:35.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.718+0000 7fddafd6f740 1 -- start start 2024-04-22T13:28:35.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.718+0000 7fddafd6f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56308cbc34d0 0x56308cbc38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.718+0000 7fddafd6f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56308cbc3e80 0x56308cbcc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.718+0000 7fddafd6f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56308cbcca00 0x56308cbcee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.718+0000 7fddafd6f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56308cb281f0 con 0x56308cbc34d0 2024-04-22T13:28:35.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.718+0000 7fddafd6f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56308cac54b0 con 0x56308cbc3e80 2024-04-22T13:28:35.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.718+0000 7fddafd6f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56308cb241b0 con 0x56308cbcca00 2024-04-22T13:28:35.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.718+0000 7fdda759e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56308cbc34d0 0x56308cbc38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.718+0000 7fdda759e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56308cbc34d0 0x56308cbc38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33222/0 (socket says 172.21.15.12:33222) 2024-04-22T13:28:35.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.718+0000 7fdda759e640 1 -- 172.21.15.12:0/3569399693 learned_addr learned my addr 172.21.15.12:0/3569399693 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:35.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.718+0000 7fdda6d9d640 1 --2- 172.21.15.12:0/3569399693 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56308cbc3e80 0x56308cbcc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.718+0000 7fdda659c640 1 --2- 172.21.15.12:0/3569399693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56308cbcca00 0x56308cbcee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.718+0000 7fdd8ffff640 1 --2- 172.21.15.12:0/4127356365 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fdd900a3d20 0x7fdd900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.718+0000 7fdda759e640 1 -- 172.21.15.12:0/3569399693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56308cbcca00 msgr2=0x56308cbcee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.718+0000 7fdda759e640 1 --2- 172.21.15.12:0/3569399693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56308cbcca00 0x56308cbcee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.718+0000 7fdda759e640 1 -- 172.21.15.12:0/3569399693 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56308cbc3e80 msgr2=0x56308cbcc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.718+0000 7fdda759e640 1 --2- 172.21.15.12:0/3569399693 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56308cbc3e80 0x56308cbcc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.718+0000 7fdd8ffff640 1 -- 172.21.15.12:0/4127356365 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fdd980d0ab0 con 0x7fdda81010a0 2024-04-22T13:28:35.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.718+0000 7fdda759e640 1 -- 172.21.15.12:0/3569399693 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56308cab71b0 con 0x56308cbc34d0 2024-04-22T13:28:35.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.718+0000 7fdda6d9d640 1 --2- 172.21.15.12:0/3569399693 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56308cbc3e80 0x56308cbcc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:35.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.718+0000 7fdda759e640 1 --2- 172.21.15.12:0/4127356365 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fdd900a3d20 0x7fdd900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.718+0000 7fdda759e640 1 --2- 172.21.15.12:0/3569399693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56308cbc34d0 0x56308cbc38b0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fdda000e890 tx=0x7fdda000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e9df1000c26514 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:35.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.718+0000 7fdd8d7fa640 1 -- 172.21.15.12:0/3569399693 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda0007db0 con 0x56308cbc34d0 2024-04-22T13:28:35.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.718+0000 7fdd8d7fa640 1 -- 172.21.15.12:0/3569399693 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdda001f070 con 0x56308cbc34d0 2024-04-22T13:28:35.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.718+0000 7fdd8d7fa640 1 -- 172.21.15.12:0/3569399693 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda0025950 con 0x56308cbc34d0 2024-04-22T13:28:35.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.719+0000 7fddafd6f740 1 -- 172.21.15.12:0/3569399693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56308cbc34d0 msgr2=0x56308cbc38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.719+0000 7fdda759e640 1 --2- 172.21.15.12:0/4127356365 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fdd900a3d20 0x7fdd900a61e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdda0024000 tx=0x7fdda0021000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:35.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.719+0000 7fddafd6f740 1 --2- 172.21.15.12:0/3569399693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56308cbc34d0 0x56308cbc38b0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fdda000e890 tx=0x7fdda000ed50 comp rx=0 tx=0).stop 2024-04-22T13:28:35.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.719+0000 7fddafd6f740 1 -- 172.21.15.12:0/3569399693 shutdown_connections 2024-04-22T13:28:35.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.719+0000 7fddafd6f740 1 --2- 172.21.15.12:0/3569399693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56308cbcca00 0x56308cbcee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.719+0000 7fddafd6f740 1 --2- 172.21.15.12:0/3569399693 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56308cbc3e80 0x56308cbcc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.719+0000 7fddafd6f740 1 --2- 172.21.15.12:0/3569399693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56308cbc34d0 0x56308cbc38b0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.719+0000 7fddafd6f740 1 -- 172.21.15.12:0/3569399693 >> 172.21.15.12:0/3569399693 conn(0x56308cbbdba0 msgr2=0x56308cbbdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:35.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.719+0000 7fddafd6f740 1 -- 172.21.15.12:0/3569399693 shutdown_connections 2024-04-22T13:28:35.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.719+0000 7fddafd6f740 1 -- 172.21.15.12:0/3569399693 wait complete. 2024-04-22T13:28:35.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.719+0000 7fddafd6f740 1 -- start start 2024-04-22T13:28:35.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.719+0000 7fddafd6f740 1 Processor -- start 2024-04-22T13:28:35.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.719+0000 7fddafd6f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56308cbc34d0 0x56308cbcc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.719+0000 7fddafd6f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56308cbc3e80 0x56308cbc96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.719+0000 7fddafd6f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56308cbcca00 0x56308cbc9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.719+0000 7fddafd6f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56308cbcf970 con 0x56308cbc34d0 2024-04-22T13:28:35.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.719+0000 7fddafd6f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56308cae1500 con 0x56308cbc3e80 2024-04-22T13:28:35.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.719+0000 7fddafd6f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56308cbcfaf0 con 0x56308cbcca00 2024-04-22T13:28:35.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.720+0000 7fdda6d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56308cbc34d0 0x56308cbcc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.720+0000 7fddafd6f740 10 client.0 _ll_register_callbacks cb 0x56308caafb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:35.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.720+0000 7fdda6d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56308cbc34d0 0x56308cbcc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33234/0 (socket says 172.21.15.12:33234) 2024-04-22T13:28:35.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.720+0000 7fdda6d9d640 1 -- 172.21.15.12:0/3828706588 learned_addr learned my addr 172.21.15.12:0/3828706588 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:35.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.720+0000 7fdda6d9d640 1 -- 172.21.15.12:0/3828706588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56308cbcca00 msgr2=0x56308cbc9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.720+0000 7fdda759e640 1 --2- 172.21.15.12:0/3828706588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56308cbcca00 0x56308cbc9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.720+0000 7fdda6d9d640 1 --2- 172.21.15.12:0/3828706588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56308cbcca00 0x56308cbc9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.720+0000 7fdda6d9d640 1 -- 172.21.15.12:0/3828706588 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56308cbc3e80 msgr2=0x56308cbc96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.720+0000 7fdda6d9d640 1 --2- 172.21.15.12:0/3828706588 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56308cbc3e80 0x56308cbc96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.720+0000 7fdda6d9d640 1 -- 172.21.15.12:0/3828706588 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56308c83a3f0 con 0x56308cbc34d0 2024-04-22T13:28:35.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.720+0000 7fdda6d9d640 1 --2- 172.21.15.12:0/3828706588 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56308cbc34d0 0x56308cbcc360 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fdd98099520 tx=0x7fdd9809c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325365f13946c33b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:35.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.720+0000 7fdd8aff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:35.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.720+0000 7fdd8aff5640 1 -- 172.21.15.12:0/3828706588 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9809cb50 con 0x56308cbc34d0 2024-04-22T13:28:35.723 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.720+0000 7fdd8aff5640 1 -- 172.21.15.12:0/3828706588 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fdd98010a90 con 0x56308cbc34d0 2024-04-22T13:28:35.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.720+0000 7fddafd6f740 1 -- 172.21.15.12:0/3828706588 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdda001f070 con 0x56308cbc34d0 2024-04-22T13:28:35.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.720+0000 7fdd8aff5640 1 -- 172.21.15.12:0/3828706588 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdd980b5030 con 0x56308cbc34d0 2024-04-22T13:28:35.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.720+0000 7fddafd6f740 10 client.17397 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:35.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.720+0000 7fddafd6f740 1 -- 172.21.15.12:0/3828706588 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56308c9047d0 con 0x56308cbc34d0 2024-04-22T13:28:35.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.720+0000 7fdd8aff5640 1 -- 172.21.15.12:0/3828706588 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdda001f070 con 0x56308cbc34d0 2024-04-22T13:28:35.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.721+0000 7fdd8aff5640 1 -- 172.21.15.12:0/3828706588 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fdd980a5280 con 0x56308cbc34d0 2024-04-22T13:28:35.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.721+0000 7fddafd6f740 10 client.17397 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:35.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.721+0000 7fddafd6f740 10 client.17397 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:28:35.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.721+0000 7fddafd6f740 20 client.17397 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:35.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.721+0000 7fddafd6f740 1 --2- 172.21.15.12:0/3828706588 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56308cbdcc50 0x56308cbdf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.721+0000 7fddafd6f740 4 client.17397 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:35.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.721+0000 7fddafd6f740 1 -- 172.21.15.12:0/3828706588 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56308cac7930 con 0x56308cbdcc50 2024-04-22T13:28:35.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.721+0000 7fdda659c640 1 --2- 172.21.15.12:0/3828706588 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56308cbdcc50 0x56308cbdf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.724 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.721+0000 7fdd8aff5640 10 client.17397 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:28:35.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.721+0000 7fdda659c640 1 --2- 172.21.15.12:0/3828706588 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56308cbdcc50 0x56308cbdf0a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b6584f7e66377e4c server_cookie=3164ef896dc15036 in_seq=0 out_seq=0 2024-04-22T13:28:35.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.722+0000 7fdd8aff5640 1 -- 172.21.15.12:0/3828706588 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56308cac7930 con 0x56308cbdcc50 2024-04-22T13:28:35.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.722+0000 7fdd8aff5640 10 client.17397 handle_command_reply: tid=0 2024-04-22T13:28:35.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.722+0000 7fddafd6f740 1 client.17397 shutdown 2024-04-22T13:28:35.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.722+0000 7fddafd6f740 1 -- 172.21.15.12:0/3828706588 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56308cbc34d0 msgr2=0x56308cbcc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.722+0000 7fddafd6f740 1 --2- 172.21.15.12:0/3828706588 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56308cbc34d0 0x56308cbcc360 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fdd98099520 tx=0x7fdd9809c4b0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.722+0000 7fddafd6f740 1 -- 172.21.15.12:0/3828706588 shutdown_connections 2024-04-22T13:28:35.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.722+0000 7fddafd6f740 1 --2- 172.21.15.12:0/3828706588 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56308cbdcc50 0x56308cbdf0a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.722+0000 7fddafd6f740 1 --2- 172.21.15.12:0/3828706588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56308cbcca00 0x56308cbc9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.722+0000 7fddafd6f740 1 --2- 172.21.15.12:0/3828706588 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56308cbc3e80 0x56308cbc96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.722+0000 7fddafd6f740 1 --2- 172.21.15.12:0/3828706588 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56308cbc34d0 0x56308cbcc360 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.722+0000 7fdd8aff5640 0 client.17397 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:28:35.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.722+0000 7fddafd6f740 1 -- 172.21.15.12:0/3828706588 >> 172.21.15.12:0/3828706588 conn(0x56308cbbdba0 msgr2=0x56308cbbf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:35.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.722+0000 7fddafd6f740 1 -- 172.21.15.12:0/3828706588 shutdown_connections 2024-04-22T13:28:35.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.722+0000 7fddafd6f740 1 -- 172.21.15.12:0/3828706588 wait complete. 2024-04-22T13:28:35.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.722+0000 7fddafd6f740 20 client.17397 trim_cache size 0 max 16384 2024-04-22T13:28:35.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.736+0000 7fddafd6f740 1 Processor -- start 2024-04-22T13:28:35.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.736+0000 7fddafd6f740 1 -- start start 2024-04-22T13:28:35.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.736+0000 7fddafd6f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56308cbc34d0 0x56308cbd8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.736+0000 7fddafd6f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56308cbc3e80 0x56308cbc9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.736+0000 7fddafd6f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56308cbcca00 0x56308cbcbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.736+0000 7fddafd6f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdda81ab920 con 0x56308cbc34d0 2024-04-22T13:28:35.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.736+0000 7fddafd6f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdda81ad460 con 0x56308cbcca00 2024-04-22T13:28:35.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.736+0000 7fddafd6f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdda81abff0 con 0x56308cbc3e80 2024-04-22T13:28:35.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.736+0000 7fdda6d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56308cbc34d0 0x56308cbd8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.736+0000 7fdda6d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56308cbc34d0 0x56308cbd8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33244/0 (socket says 172.21.15.12:33244) 2024-04-22T13:28:35.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.736+0000 7fdda6d9d640 1 -- 172.21.15.12:0/1989216645 learned_addr learned my addr 172.21.15.12:0/1989216645 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:35.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.736+0000 7fdda659c640 1 --2- 172.21.15.12:0/1989216645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56308cbc3e80 0x56308cbc9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.736+0000 7fdda759e640 1 --2- 172.21.15.12:0/1989216645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56308cbcca00 0x56308cbcbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.736+0000 7fdda6d9d640 1 -- 172.21.15.12:0/1989216645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56308cbc3e80 msgr2=0x56308cbc9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.736+0000 7fdda6d9d640 1 --2- 172.21.15.12:0/1989216645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56308cbc3e80 0x56308cbc9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.736+0000 7fdda6d9d640 1 -- 172.21.15.12:0/1989216645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56308cbcca00 msgr2=0x56308cbcbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.736+0000 7fdda6d9d640 1 --2- 172.21.15.12:0/1989216645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56308cbcca00 0x56308cbcbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.736+0000 7fdda6d9d640 1 -- 172.21.15.12:0/1989216645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56308cbca070 con 0x56308cbc34d0 2024-04-22T13:28:35.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.736+0000 7fdda759e640 1 --2- 172.21.15.12:0/1989216645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56308cbcca00 0x56308cbcbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:35.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.736+0000 7fdda6d9d640 1 --2- 172.21.15.12:0/1989216645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56308cbc34d0 0x56308cbd8830 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fdd98000c00 tx=0x7fdd9809c5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e86b42b51227a7dc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:35.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.736+0000 7fdd8cff9640 1 -- 172.21.15.12:0/1989216645 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd980a0e50 con 0x56308cbc34d0 2024-04-22T13:28:35.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.737+0000 7fdd8cff9640 1 -- 172.21.15.12:0/1989216645 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd980a5710 con 0x56308cbc34d0 2024-04-22T13:28:35.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.737+0000 7fdd8cff9640 1 -- 172.21.15.12:0/1989216645 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9809ba10 con 0x56308cbc34d0 2024-04-22T13:28:35.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.737+0000 7fddafd6f740 1 -- 172.21.15.12:0/1989216645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56308cbc34d0 msgr2=0x56308cbd8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.737+0000 7fddafd6f740 1 --2- 172.21.15.12:0/1989216645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56308cbc34d0 0x56308cbd8830 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fdd98000c00 tx=0x7fdd9809c5e0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.737+0000 7fddafd6f740 1 -- 172.21.15.12:0/1989216645 shutdown_connections 2024-04-22T13:28:35.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.737+0000 7fddafd6f740 1 --2- 172.21.15.12:0/1989216645 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56308cbcca00 0x56308cbcbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.737+0000 7fddafd6f740 1 --2- 172.21.15.12:0/1989216645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56308cbc3e80 0x56308cbc9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.737+0000 7fddafd6f740 1 --2- 172.21.15.12:0/1989216645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56308cbc34d0 0x56308cbd8830 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.737+0000 7fddafd6f740 1 -- 172.21.15.12:0/1989216645 >> 172.21.15.12:0/1989216645 conn(0x56308cbbdba0 msgr2=0x56308cbbf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:35.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.737+0000 7fddafd6f740 1 -- 172.21.15.12:0/1989216645 shutdown_connections 2024-04-22T13:28:35.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.737+0000 7fddafd6f740 1 -- 172.21.15.12:0/1989216645 wait complete. 2024-04-22T13:28:35.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.737+0000 7fddafd6f740 1 -- start start 2024-04-22T13:28:35.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.737+0000 7fddafd6f740 1 Processor -- start 2024-04-22T13:28:35.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.738+0000 7fddafd6f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56308cbc34d0 0x56308cbd4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.738+0000 7fddafd6f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56308cbc3e80 0x56308cbd5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.738+0000 7fddafd6f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56308cbcca00 0x56308cbd1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.738+0000 7fddafd6f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fdda81acd90 con 0x56308cbc3e80 2024-04-22T13:28:35.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.738+0000 7fddafd6f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdda81ac6c0 con 0x56308cbc34d0 2024-04-22T13:28:35.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.738+0000 7fddafd6f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdda81a9900 con 0x56308cbcca00 2024-04-22T13:28:35.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.738+0000 7fddafd6f740 10 client.0 _ll_register_callbacks cb 0x7fdd9809c690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:35.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.738+0000 7fdda659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56308cbc3e80 0x56308cbd5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.738+0000 7fdda659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56308cbc3e80 0x56308cbd5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33252/0 (socket says 172.21.15.12:33252) 2024-04-22T13:28:35.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.738+0000 7fdda659c640 1 -- 172.21.15.12:0/3638120909 learned_addr learned my addr 172.21.15.12:0/3638120909 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:35.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.738+0000 7fdda6d9d640 1 --2- 172.21.15.12:0/3638120909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56308cbc34d0 0x56308cbd4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.738+0000 7fdda759e640 1 --2- 172.21.15.12:0/3638120909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56308cbcca00 0x56308cbd1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.738+0000 7fdda659c640 1 -- 172.21.15.12:0/3638120909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56308cbcca00 msgr2=0x56308cbd1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.738+0000 7fdda659c640 1 --2- 172.21.15.12:0/3638120909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56308cbcca00 0x56308cbd1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.738+0000 7fdda659c640 1 -- 172.21.15.12:0/3638120909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56308cbc34d0 msgr2=0x56308cbd4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.738+0000 7fdda659c640 1 --2- 172.21.15.12:0/3638120909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56308cbc34d0 0x56308cbd4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.738+0000 7fdda659c640 1 -- 172.21.15.12:0/3638120909 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56308cbd2380 con 0x56308cbc3e80 2024-04-22T13:28:35.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.738+0000 7fdda659c640 1 --2- 172.21.15.12:0/3638120909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56308cbc3e80 0x56308cbd5280 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fdda806a420 tx=0x7fdd9c047d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e180d3ce1bbe4b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:35.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.738+0000 7fdd8aff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:35.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.739+0000 7fdd8aff5640 1 -- 172.21.15.12:0/3638120909 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c004310 con 0x56308cbc3e80 2024-04-22T13:28:35.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.739+0000 7fdd8aff5640 1 -- 172.21.15.12:0/3638120909 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fdd9c03a040 con 0x56308cbc3e80 2024-04-22T13:28:35.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.739+0000 7fddafd6f740 1 -- 172.21.15.12:0/3638120909 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd980a5710 con 0x56308cbc3e80 2024-04-22T13:28:35.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.739+0000 7fdd8aff5640 1 -- 172.21.15.12:0/3638120909 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdd9c056050 con 0x56308cbc3e80 2024-04-22T13:28:35.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.739+0000 7fddafd6f740 10 client.17403 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:35.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.739+0000 7fddafd6f740 1 -- 172.21.15.12:0/3638120909 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56308cc43b00 con 0x56308cbc3e80 2024-04-22T13:28:35.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.739+0000 7fdd8aff5640 1 -- 172.21.15.12:0/3638120909 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd980a5710 con 0x56308cbc3e80 2024-04-22T13:28:35.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.739+0000 7fdd8aff5640 1 -- 172.21.15.12:0/3638120909 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fdd9c0472d0 con 0x56308cbc3e80 2024-04-22T13:28:35.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.739+0000 7fddafd6f740 10 client.17403 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:35.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.739+0000 7fddafd6f740 10 client.17403 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:28:35.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.739+0000 7fddafd6f740 20 client.17403 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:35.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.739+0000 7fddafd6f740 1 --2- 172.21.15.12:0/3638120909 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56308cc442b0 0x56308cc466d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.739+0000 7fddafd6f740 4 client.17403 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:28:35.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.739+0000 7fddafd6f740 1 -- 172.21.15.12:0/3638120909 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56308cb35c10 con 0x56308cc442b0 2024-04-22T13:28:35.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.739+0000 7fdda6d9d640 1 --2- 172.21.15.12:0/3638120909 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56308cc442b0 0x56308cc466d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.740+0000 7fdda6d9d640 1 --2- 172.21.15.12:0/3638120909 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56308cc442b0 0x56308cc466d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=55613668ecf2d387 server_cookie=d42f10516b0d20 in_seq=0 out_seq=0 2024-04-22T13:28:35.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.740+0000 7fdd8aff5640 10 client.17403 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:28:35.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.740+0000 7fdd8aff5640 1 -- 172.21.15.12:0/3638120909 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56308cb35c10 con 0x56308cc442b0 2024-04-22T13:28:35.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.740+0000 7fdd8aff5640 10 client.17403 handle_command_reply: tid=0 2024-04-22T13:28:35.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.740+0000 7fddafd6f740 1 client.17403 shutdown 2024-04-22T13:28:35.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.740+0000 7fddafd6f740 1 -- 172.21.15.12:0/3638120909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56308cbc3e80 msgr2=0x56308cbd5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.740+0000 7fddafd6f740 1 --2- 172.21.15.12:0/3638120909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56308cbc3e80 0x56308cbd5280 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fdda806a420 tx=0x7fdd9c047d60 comp rx=0 tx=0).stop 2024-04-22T13:28:35.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.740+0000 7fddafd6f740 1 -- 172.21.15.12:0/3638120909 shutdown_connections 2024-04-22T13:28:35.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.740+0000 7fddafd6f740 1 --2- 172.21.15.12:0/3638120909 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56308cc442b0 0x56308cc466d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.740+0000 7fddafd6f740 1 --2- 172.21.15.12:0/3638120909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56308cbcca00 0x56308cbd1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.740+0000 7fddafd6f740 1 --2- 172.21.15.12:0/3638120909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56308cbc3e80 0x56308cbd5280 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.740+0000 7fdd8aff5640 0 client.17403 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:28:35.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.740+0000 7fddafd6f740 1 --2- 172.21.15.12:0/3638120909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56308cbc34d0 0x56308cbd4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.740+0000 7fddafd6f740 1 -- 172.21.15.12:0/3638120909 >> 172.21.15.12:0/3638120909 conn(0x56308cbbdba0 msgr2=0x56308cbbf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:35.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.740+0000 7fddafd6f740 1 -- 172.21.15.12:0/3638120909 shutdown_connections 2024-04-22T13:28:35.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.740+0000 7fddafd6f740 1 -- 172.21.15.12:0/3638120909 wait complete. 2024-04-22T13:28:35.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.740+0000 7fddafd6f740 20 client.17403 trim_cache size 0 max 16384 2024-04-22T13:28:35.744 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:28:35.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.741+0000 7fddad17e640 1 -- 172.21.15.12:0/4127356365 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fdd900a3d20 msgr2=0x7fdd900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.741+0000 7fddad17e640 1 --2- 172.21.15.12:0/4127356365 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fdd900a3d20 0x7fdd900a61e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdda0024000 tx=0x7fdda0021000 comp rx=0 tx=0).stop 2024-04-22T13:28:35.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.741+0000 7fddad17e640 1 -- 172.21.15.12:0/4127356365 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdda81010a0 msgr2=0x7fdda8197610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.741+0000 7fddad17e640 1 --2- 172.21.15.12:0/4127356365 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdda81010a0 0x7fdda8197610 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdd9800e9f0 tx=0x7fdd9800eeb0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.741+0000 7fddad17e640 1 -- 172.21.15.12:0/4127356365 shutdown_connections 2024-04-22T13:28:35.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.741+0000 7fddad17e640 1 --2- 172.21.15.12:0/4127356365 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fdd900a3d20 0x7fdd900a61e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.741+0000 7fddad17e640 1 --2- 172.21.15.12:0/4127356365 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdda819bed0 0x7fdda819c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.741+0000 7fddad17e640 1 --2- 172.21.15.12:0/4127356365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdda8106170 0x7fdda8197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.741+0000 7fddad17e640 1 --2- 172.21.15.12:0/4127356365 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdda81010a0 0x7fdda8197610 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.741+0000 7fddad17e640 1 -- 172.21.15.12:0/4127356365 >> 172.21.15.12:0/4127356365 conn(0x7fdda8077eb0 msgr2=0x7fdda80ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:35.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.741+0000 7fddad17e640 1 -- 172.21.15.12:0/4127356365 shutdown_connections 2024-04-22T13:28:35.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.741+0000 7fddad17e640 1 -- 172.21.15.12:0/4127356365 wait complete. 2024-04-22T13:28:35.747 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:35 smithi012 ceph-mon[28265]: pgmap v137: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:35.747 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:35 smithi012 ceph-mon[28265]: scrub queued for path: / 2024-04-22T13:28:35.747 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:35 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:28:35.747 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:35 smithi012 ceph-mon[28265]: scrub summary: active paths [/] 2024-04-22T13:28:35.748 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:35 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:28:35.748 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:35 smithi012 ceph-mon[28265]: scrub summary: idle 2024-04-22T13:28:35.748 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:35 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2668414817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:28:35.749 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:28:35.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.856+0000 7fd91649a640 1 -- 172.21.15.12:0/3352971270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd91006b1f0 msgr2=0x7fd910106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.856+0000 7fd91649a640 1 --2- 172.21.15.12:0/3352971270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd91006b1f0 0x7fd910106b90 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fd904009900 tx=0x7fd90402bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:35.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.856+0000 7fd91649a640 1 -- 172.21.15.12:0/3352971270 shutdown_connections 2024-04-22T13:28:35.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.856+0000 7fd91649a640 1 --2- 172.21.15.12:0/3352971270 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd910107270 0x7fd91010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.856+0000 7fd91649a640 1 --2- 172.21.15.12:0/3352971270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd91006b1f0 0x7fd910106b90 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.856+0000 7fd91649a640 1 --2- 172.21.15.12:0/3352971270 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd91006a840 0x7fd91006ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.856+0000 7fd91649a640 1 -- 172.21.15.12:0/3352971270 >> 172.21.15.12:0/3352971270 conn(0x7fd910077eb0 msgr2=0x7fd9100782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:35.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.856+0000 7fd91649a640 1 -- 172.21.15.12:0/3352971270 shutdown_connections 2024-04-22T13:28:35.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.856+0000 7fd91649a640 1 -- 172.21.15.12:0/3352971270 wait complete. 2024-04-22T13:28:35.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.856+0000 7fd91649a640 1 Processor -- start 2024-04-22T13:28:35.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.857+0000 7fd91649a640 1 -- start start 2024-04-22T13:28:35.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.857+0000 7fd91649a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd91006a840 0x7fd910100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.857+0000 7fd91649a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd910107270 0x7fd9101013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.857+0000 7fd91649a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd910101900 0x7fd9101a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.857+0000 7fd91649a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd9101121b0 con 0x7fd910107270 2024-04-22T13:28:35.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.857+0000 7fd91649a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd910112030 con 0x7fd910101900 2024-04-22T13:28:35.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.857+0000 7fd91649a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd910112330 con 0x7fd91006a840 2024-04-22T13:28:35.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.857+0000 7fd90f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd910107270 0x7fd9101013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.857+0000 7fd90f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd910107270 0x7fd9101013c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33274/0 (socket says 172.21.15.12:33274) 2024-04-22T13:28:35.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.857+0000 7fd90f7fe640 1 -- 172.21.15.12:0/3199326493 learned_addr learned my addr 172.21.15.12:0/3199326493 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:35.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.857+0000 7fd90ffff640 1 --2- 172.21.15.12:0/3199326493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd91006a840 0x7fd910100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.857+0000 7fd914a37640 1 --2- 172.21.15.12:0/3199326493 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd910101900 0x7fd9101a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.857+0000 7fd90f7fe640 1 -- 172.21.15.12:0/3199326493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd91006a840 msgr2=0x7fd910100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.857+0000 7fd90f7fe640 1 --2- 172.21.15.12:0/3199326493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd91006a840 0x7fd910100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.857+0000 7fd90f7fe640 1 -- 172.21.15.12:0/3199326493 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd910101900 msgr2=0x7fd9101a68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.857+0000 7fd90f7fe640 1 --2- 172.21.15.12:0/3199326493 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd910101900 0x7fd9101a68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.857+0000 7fd90f7fe640 1 -- 172.21.15.12:0/3199326493 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9101a6fc0 con 0x7fd910107270 2024-04-22T13:28:35.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.858+0000 7fd90f7fe640 1 --2- 172.21.15.12:0/3199326493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd910107270 0x7fd9101013c0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fd904036040 tx=0x7fd90402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:35.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.858+0000 7fd90d7fa640 1 -- 172.21.15.12:0/3199326493 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd904043020 con 0x7fd910107270 2024-04-22T13:28:35.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.858+0000 7fd91649a640 1 -- 172.21.15.12:0/3199326493 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9101a7250 con 0x7fd910107270 2024-04-22T13:28:35.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.858+0000 7fd90d7fa640 1 -- 172.21.15.12:0/3199326493 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd90402cbe0 con 0x7fd910107270 2024-04-22T13:28:35.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.858+0000 7fd90d7fa640 1 -- 172.21.15.12:0/3199326493 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd904003bf0 con 0x7fd910107270 2024-04-22T13:28:35.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.858+0000 7fd91649a640 1 -- 172.21.15.12:0/3199326493 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9101a77f0 con 0x7fd910107270 2024-04-22T13:28:35.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.862+0000 7fd90d7fa640 1 -- 172.21.15.12:0/3199326493 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd904046020 con 0x7fd910107270 2024-04-22T13:28:35.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.862+0000 7fd91916b740 1 Processor -- start 2024-04-22T13:28:35.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.862+0000 7fd91916b740 1 -- start start 2024-04-22T13:28:35.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.862+0000 7fd91916b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e20190d4d0 0x55e20190d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.862+0000 7fd91916b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e20190de80 0x55e201916430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.862+0000 7fd91916b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e201916a00 0x55e201918e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.862+0000 7fd91916b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e2018721f0 con 0x55e20190d4d0 2024-04-22T13:28:35.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.862+0000 7fd91916b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e20180f4b0 con 0x55e20190de80 2024-04-22T13:28:35.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.862+0000 7fd91916b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e20186e1b0 con 0x55e201916a00 2024-04-22T13:28:35.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.862+0000 7fd914a37640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e20190d4d0 0x55e20190d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.862+0000 7fd914a37640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e20190d4d0 0x55e20190d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33278/0 (socket says 172.21.15.12:33278) 2024-04-22T13:28:35.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.862+0000 7fd914a37640 1 -- 172.21.15.12:0/3272898310 learned_addr learned my addr 172.21.15.12:0/3272898310 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:35.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.862+0000 7fd90f7fe640 1 --2- 172.21.15.12:0/3272898310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e201916a00 0x55e201918e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.862+0000 7fd90ffff640 1 --2- 172.21.15.12:0/3272898310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e20190de80 0x55e201916430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.862+0000 7fd90d7fa640 1 --2- 172.21.15.12:0/3199326493 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd8f80a3d50 0x7fd8f80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.862+0000 7fd914a37640 1 -- 172.21.15.12:0/3272898310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e201916a00 msgr2=0x55e201918e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.862+0000 7fd914a37640 1 --2- 172.21.15.12:0/3272898310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e201916a00 0x55e201918e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.862+0000 7fd914a37640 1 -- 172.21.15.12:0/3272898310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e20190de80 msgr2=0x55e201916430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.862+0000 7fd914a37640 1 --2- 172.21.15.12:0/3272898310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e20190de80 0x55e201916430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.862+0000 7fd914a37640 1 -- 172.21.15.12:0/3272898310 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2018011b0 con 0x55e20190d4d0 2024-04-22T13:28:35.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.862+0000 7fd90f7fe640 1 --2- 172.21.15.12:0/3272898310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e201916a00 0x55e201918e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:35.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.862+0000 7fd914a37640 1 --2- 172.21.15.12:0/3199326493 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd8f80a3d50 0x7fd8f80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.862+0000 7fd90d7fa640 1 -- 172.21.15.12:0/3199326493 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fd9040f2830 con 0x7fd910107270 2024-04-22T13:28:35.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.862+0000 7fd914a37640 1 --2- 172.21.15.12:0/3272898310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e20190d4d0 0x55e20190d8b0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fd908010440 tx=0x7fd908010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84a1dcc92b00305 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:35.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.862+0000 7fd8f67fc640 1 -- 172.21.15.12:0/3272898310 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90801d400 con 0x55e20190d4d0 2024-04-22T13:28:35.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.862+0000 7fd8f67fc640 1 -- 172.21.15.12:0/3272898310 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd90801da50 con 0x55e20190d4d0 2024-04-22T13:28:35.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.862+0000 7fd8f67fc640 1 -- 172.21.15.12:0/3272898310 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90801c760 con 0x55e20190d4d0 2024-04-22T13:28:35.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.863+0000 7fd91916b740 1 -- 172.21.15.12:0/3272898310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e20190d4d0 msgr2=0x55e20190d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.863+0000 7fd91916b740 1 --2- 172.21.15.12:0/3272898310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e20190d4d0 0x55e20190d8b0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fd908010440 tx=0x7fd908010900 comp rx=0 tx=0).stop 2024-04-22T13:28:35.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.863+0000 7fd914a37640 1 --2- 172.21.15.12:0/3199326493 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd8f80a3d50 0x7fd8f80a6210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd908021000 tx=0x7fd90801e000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:35.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.863+0000 7fd91916b740 1 -- 172.21.15.12:0/3272898310 shutdown_connections 2024-04-22T13:28:35.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.863+0000 7fd91916b740 1 --2- 172.21.15.12:0/3272898310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e201916a00 0x55e201918e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.863+0000 7fd91916b740 1 --2- 172.21.15.12:0/3272898310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e20190de80 0x55e201916430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.863+0000 7fd91916b740 1 --2- 172.21.15.12:0/3272898310 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e20190d4d0 0x55e20190d8b0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.863+0000 7fd91916b740 1 -- 172.21.15.12:0/3272898310 >> 172.21.15.12:0/3272898310 conn(0x55e201907ba0 msgr2=0x55e201907f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:35.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.863+0000 7fd91916b740 1 -- 172.21.15.12:0/3272898310 shutdown_connections 2024-04-22T13:28:35.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.863+0000 7fd91916b740 1 -- 172.21.15.12:0/3272898310 wait complete. 2024-04-22T13:28:35.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.863+0000 7fd91916b740 1 -- start start 2024-04-22T13:28:35.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.863+0000 7fd91916b740 1 Processor -- start 2024-04-22T13:28:35.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.864+0000 7fd91916b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e20190d4d0 0x55e201916390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.864+0000 7fd91916b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e20190de80 0x55e201913720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.864+0000 7fd91916b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e201916a00 0x55e201913d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.864+0000 7fd91916b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e201919970 con 0x55e201916a00 2024-04-22T13:28:35.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.864+0000 7fd91916b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e20182b500 con 0x55e20190de80 2024-04-22T13:28:35.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.864+0000 7fd91916b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e201919af0 con 0x55e20190d4d0 2024-04-22T13:28:35.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.864+0000 7fd91916b740 10 client.0 _ll_register_callbacks cb 0x55e2017f9b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:35.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.864+0000 7fd914a37640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e201916a00 0x55e201913d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.864+0000 7fd914a37640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e201916a00 0x55e201913d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33292/0 (socket says 172.21.15.12:33292) 2024-04-22T13:28:35.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.864+0000 7fd914a37640 1 -- 172.21.15.12:0/2087066084 learned_addr learned my addr 172.21.15.12:0/2087066084 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:35.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.864+0000 7fd90ffff640 1 --2- 172.21.15.12:0/2087066084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e20190d4d0 0x55e201916390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.864+0000 7fd90f7fe640 1 --2- 172.21.15.12:0/2087066084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e20190de80 0x55e201913720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.864+0000 7fd914a37640 1 -- 172.21.15.12:0/2087066084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e20190d4d0 msgr2=0x55e201916390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.864+0000 7fd914a37640 1 --2- 172.21.15.12:0/2087066084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e20190d4d0 0x55e201916390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.864+0000 7fd914a37640 1 -- 172.21.15.12:0/2087066084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e20190de80 msgr2=0x55e201913720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.864+0000 7fd914a37640 1 --2- 172.21.15.12:0/2087066084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e20190de80 0x55e201913720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.864+0000 7fd914a37640 1 -- 172.21.15.12:0/2087066084 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2015843f0 con 0x55e201916a00 2024-04-22T13:28:35.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.864+0000 7fd90ffff640 1 --2- 172.21.15.12:0/2087066084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e20190d4d0 0x55e201916390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:35.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.864+0000 7fd914a37640 1 --2- 172.21.15.12:0/2087066084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e201916a00 0x55e201913d30 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fd908009640 tx=0x7fd90801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f514c71179e34d16 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:35.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.864+0000 7fd8ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:35.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.864+0000 7fd8ef7fe640 1 -- 172.21.15.12:0/2087066084 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90801ca80 con 0x55e201916a00 2024-04-22T13:28:35.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.864+0000 7fd8ef7fe640 1 -- 172.21.15.12:0/2087066084 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fd90801cc20 con 0x55e201916a00 2024-04-22T13:28:35.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.865+0000 7fd91916b740 1 -- 172.21.15.12:0/2087066084 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd90801da50 con 0x55e201916a00 2024-04-22T13:28:35.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.865+0000 7fd8ef7fe640 1 -- 172.21.15.12:0/2087066084 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd90801ce20 con 0x55e201916a00 2024-04-22T13:28:35.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.865+0000 7fd91916b740 10 client.17415 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:35.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.865+0000 7fd91916b740 1 -- 172.21.15.12:0/2087066084 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e20164e7d0 con 0x55e201916a00 2024-04-22T13:28:35.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.865+0000 7fd8ef7fe640 1 -- 172.21.15.12:0/2087066084 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd90801da50 con 0x55e201916a00 2024-04-22T13:28:35.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.865+0000 7fd8ef7fe640 1 -- 172.21.15.12:0/2087066084 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd90800f040 con 0x55e201916a00 2024-04-22T13:28:35.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.865+0000 7fd91916b740 10 client.17415 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:35.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.865+0000 7fd91916b740 10 client.17415 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:35.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.865+0000 7fd91916b740 20 client.17415 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:35.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.865+0000 7fd91916b740 1 --2- 172.21.15.12:0/2087066084 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e20192f360 0x55e201931750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.865+0000 7fd91916b740 4 client.17415 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:35.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.865+0000 7fd91916b740 1 -- 172.21.15.12:0/2087066084 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e201811930 con 0x55e20192f360 2024-04-22T13:28:35.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.865+0000 7fd90ffff640 1 --2- 172.21.15.12:0/2087066084 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e20192f360 0x55e201931750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.866+0000 7fd8ef7fe640 10 client.17415 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:35.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.866+0000 7fd90ffff640 1 --2- 172.21.15.12:0/2087066084 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e20192f360 0x55e201931750 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ef4d1903a02ed45 server_cookie=139e35c7a140f80 in_seq=0 out_seq=0 2024-04-22T13:28:35.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.867+0000 7fd8ef7fe640 1 -- 172.21.15.12:0/2087066084 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e201811930 con 0x55e20192f360 2024-04-22T13:28:35.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.867+0000 7fd8ef7fe640 10 client.17415 handle_command_reply: tid=0 2024-04-22T13:28:35.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.867+0000 7fd91916b740 1 client.17415 shutdown 2024-04-22T13:28:35.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.867+0000 7fd91916b740 1 -- 172.21.15.12:0/2087066084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e201916a00 msgr2=0x55e201913d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.867+0000 7fd91916b740 1 --2- 172.21.15.12:0/2087066084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e201916a00 0x55e201913d30 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fd908009640 tx=0x7fd90801c3a0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.867+0000 7fd91916b740 1 -- 172.21.15.12:0/2087066084 shutdown_connections 2024-04-22T13:28:35.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.867+0000 7fd91916b740 1 --2- 172.21.15.12:0/2087066084 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e20192f360 0x55e201931750 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.867+0000 7fd91916b740 1 --2- 172.21.15.12:0/2087066084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e201916a00 0x55e201913d30 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.867+0000 7fd91916b740 1 --2- 172.21.15.12:0/2087066084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e20190de80 0x55e201913720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.867+0000 7fd8ef7fe640 0 client.17415 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:35.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.867+0000 7fd91916b740 1 --2- 172.21.15.12:0/2087066084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e20190d4d0 0x55e201916390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.867+0000 7fd91916b740 1 -- 172.21.15.12:0/2087066084 >> 172.21.15.12:0/2087066084 conn(0x55e201907ba0 msgr2=0x55e2019098f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:35.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.867+0000 7fd91916b740 1 -- 172.21.15.12:0/2087066084 shutdown_connections 2024-04-22T13:28:35.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.867+0000 7fd91916b740 1 -- 172.21.15.12:0/2087066084 wait complete. 2024-04-22T13:28:35.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.867+0000 7fd91916b740 20 client.17415 trim_cache size 0 max 16384 2024-04-22T13:28:35.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.880+0000 7fd91916b740 1 Processor -- start 2024-04-22T13:28:35.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.880+0000 7fd91916b740 1 -- start start 2024-04-22T13:28:35.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.880+0000 7fd91916b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e20190d4d0 0x55e201915e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.880+0000 7fd91916b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e20190de80 0x55e201916390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.880+0000 7fd91916b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e201916a00 0x55e201936ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.880+0000 7fd91916b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd9101ab850 con 0x55e20190de80 2024-04-22T13:28:35.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.880+0000 7fd91916b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd9101ad390 con 0x55e201916a00 2024-04-22T13:28:35.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.880+0000 7fd91916b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd9101abf20 con 0x55e20190d4d0 2024-04-22T13:28:35.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.880+0000 7fd90f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e20190de80 0x55e201916390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.881+0000 7fd90f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e20190de80 0x55e201916390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33296/0 (socket says 172.21.15.12:33296) 2024-04-22T13:28:35.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.881+0000 7fd90f7fe640 1 -- 172.21.15.12:0/2715861961 learned_addr learned my addr 172.21.15.12:0/2715861961 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:35.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.881+0000 7fd90ffff640 1 --2- 172.21.15.12:0/2715861961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e20190d4d0 0x55e201915e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.881+0000 7fd90f7fe640 1 -- 172.21.15.12:0/2715861961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e20190d4d0 msgr2=0x55e201915e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.881+0000 7fd90f7fe640 1 --2- 172.21.15.12:0/2715861961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e20190d4d0 0x55e201915e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.881+0000 7fd90f7fe640 1 -- 172.21.15.12:0/2715861961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e201916a00 msgr2=0x55e201936ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.881+0000 7fd90f7fe640 1 --2- 172.21.15.12:0/2715861961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e201916a00 0x55e201936ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.881+0000 7fd90f7fe640 1 -- 172.21.15.12:0/2715861961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e201909e90 con 0x55e20190de80 2024-04-22T13:28:35.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.881+0000 7fd90f7fe640 1 --2- 172.21.15.12:0/2715861961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e20190de80 0x55e201916390 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fd9040be630 tx=0x7fd9040df040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26250037f604364 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:35.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.881+0000 7fd8f5ffb640 1 -- 172.21.15.12:0/2715861961 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9040be290 con 0x55e20190de80 2024-04-22T13:28:35.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.881+0000 7fd8f5ffb640 1 -- 172.21.15.12:0/2715861961 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9040bf030 con 0x55e20190de80 2024-04-22T13:28:35.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.881+0000 7fd8f5ffb640 1 -- 172.21.15.12:0/2715861961 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9040babd0 con 0x55e20190de80 2024-04-22T13:28:35.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.882+0000 7fd91916b740 1 -- 172.21.15.12:0/2715861961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e20190de80 msgr2=0x55e201916390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.882+0000 7fd91916b740 1 --2- 172.21.15.12:0/2715861961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e20190de80 0x55e201916390 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fd9040be630 tx=0x7fd9040df040 comp rx=0 tx=0).stop 2024-04-22T13:28:35.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.882+0000 7fd91916b740 1 -- 172.21.15.12:0/2715861961 shutdown_connections 2024-04-22T13:28:35.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.882+0000 7fd91916b740 1 --2- 172.21.15.12:0/2715861961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e201916a00 0x55e201936ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.882+0000 7fd91916b740 1 --2- 172.21.15.12:0/2715861961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e20190de80 0x55e201916390 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.882+0000 7fd91916b740 1 --2- 172.21.15.12:0/2715861961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e20190d4d0 0x55e201915e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.882+0000 7fd91916b740 1 -- 172.21.15.12:0/2715861961 >> 172.21.15.12:0/2715861961 conn(0x55e201907ba0 msgr2=0x55e201906c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:35.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.882+0000 7fd91916b740 1 -- 172.21.15.12:0/2715861961 shutdown_connections 2024-04-22T13:28:35.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.882+0000 7fd91916b740 1 -- 172.21.15.12:0/2715861961 wait complete. 2024-04-22T13:28:35.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.882+0000 7fd91916b740 1 -- start start 2024-04-22T13:28:35.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.882+0000 7fd91916b740 1 Processor -- start 2024-04-22T13:28:35.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.882+0000 7fd91916b740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e20190d4d0 0x55e201910310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.882+0000 7fd91916b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e20190de80 0x55e201910850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.882+0000 7fd91916b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e201916a00 0x55e201910d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.882+0000 7fd91916b740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd9101accc0 con 0x55e20190d4d0 2024-04-22T13:28:35.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.882+0000 7fd91916b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd9101ac5f0 con 0x55e201916a00 2024-04-22T13:28:35.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.883+0000 7fd91916b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd9101a9830 con 0x55e20190de80 2024-04-22T13:28:35.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.883+0000 7fd91916b740 10 client.0 _ll_register_callbacks cb 0x7fd908009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:35.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.883+0000 7fd90ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e20190d4d0 0x55e201910310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.883+0000 7fd90ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e20190d4d0 0x55e201910310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33302/0 (socket says 172.21.15.12:33302) 2024-04-22T13:28:35.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.883+0000 7fd90ffff640 1 -- 172.21.15.12:0/2122018929 learned_addr learned my addr 172.21.15.12:0/2122018929 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:35.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.883+0000 7fd90f7fe640 1 --2- 172.21.15.12:0/2122018929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e20190de80 0x55e201910850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.883+0000 7fd914a37640 1 --2- 172.21.15.12:0/2122018929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e201916a00 0x55e201910d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.883+0000 7fd90ffff640 1 -- 172.21.15.12:0/2122018929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e20190de80 msgr2=0x55e201910850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.883+0000 7fd90ffff640 1 --2- 172.21.15.12:0/2122018929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e20190de80 0x55e201910850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.883+0000 7fd90ffff640 1 -- 172.21.15.12:0/2122018929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e201916a00 msgr2=0x55e201910d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:35.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.883+0000 7fd90ffff640 1 --2- 172.21.15.12:0/2122018929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e201916a00 0x55e201910d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.883+0000 7fd90ffff640 1 -- 172.21.15.12:0/2122018929 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e201994730 con 0x55e20190d4d0 2024-04-22T13:28:35.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.883+0000 7fd914a37640 1 --2- 172.21.15.12:0/2122018929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e201916a00 0x55e201910d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:35.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.883+0000 7fd90ffff640 1 --2- 172.21.15.12:0/2122018929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e20190d4d0 0x55e201910310 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x55e201931e60 tx=0x7fd90000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bebf3248fa02b4f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:35.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.883+0000 7fd8ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:35.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.883+0000 7fd8ef7fe640 1 -- 172.21.15.12:0/2122018929 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd900005020 con 0x55e20190d4d0 2024-04-22T13:28:35.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.883+0000 7fd8ef7fe640 1 -- 172.21.15.12:0/2122018929 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fd9000051c0 con 0x55e20190d4d0 2024-04-22T13:28:35.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.883+0000 7fd91916b740 1 -- 172.21.15.12:0/2122018929 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9040bf030 con 0x55e20190d4d0 2024-04-22T13:28:35.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.883+0000 7fd8ef7fe640 1 -- 172.21.15.12:0/2122018929 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd90002f050 con 0x55e20190d4d0 2024-04-22T13:28:35.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.883+0000 7fd91916b740 10 client.17421 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:35.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.883+0000 7fd91916b740 1 -- 172.21.15.12:0/2122018929 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e201995a40 con 0x55e20190d4d0 2024-04-22T13:28:35.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.884+0000 7fd8ef7fe640 1 -- 172.21.15.12:0/2122018929 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9040bf030 con 0x55e20190d4d0 2024-04-22T13:28:35.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.884+0000 7fd8ef7fe640 1 -- 172.21.15.12:0/2122018929 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd900020200 con 0x55e20190d4d0 2024-04-22T13:28:35.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.884+0000 7fd91916b740 10 client.17421 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:35.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.884+0000 7fd91916b740 10 client.17421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:35.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.884+0000 7fd91916b740 20 client.17421 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:35.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.884+0000 7fd91916b740 1 --2- 172.21.15.12:0/2122018929 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e2019961e0 0x55e201998600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:35.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.884+0000 7fd91916b740 4 client.17421 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:28:35.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.884+0000 7fd91916b740 1 -- 172.21.15.12:0/2122018929 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e20187fc10 con 0x55e2019961e0 2024-04-22T13:28:35.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.884+0000 7fd90f7fe640 1 --2- 172.21.15.12:0/2122018929 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e2019961e0 0x55e201998600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:35.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.884+0000 7fd90f7fe640 1 --2- 172.21.15.12:0/2122018929 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e2019961e0 0x55e201998600 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48ec15c2e36264c4 server_cookie=a6e08c53a4ba0300 in_seq=0 out_seq=0 2024-04-22T13:28:35.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.884+0000 7fd8ef7fe640 10 client.17421 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:35.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.885+0000 7fd8ef7fe640 1 -- 172.21.15.12:0/2122018929 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e20187fc10 con 0x55e2019961e0 2024-04-22T13:28:35.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.885+0000 7fd8ef7fe640 10 client.17421 handle_command_reply: tid=0 2024-04-22T13:28:35.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.885+0000 7fd91916b740 1 client.17421 shutdown 2024-04-22T13:28:35.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.885+0000 7fd91916b740 1 -- 172.21.15.12:0/2122018929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e20190d4d0 msgr2=0x55e201910310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.885+0000 7fd91916b740 1 --2- 172.21.15.12:0/2122018929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e20190d4d0 0x55e201910310 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x55e201931e60 tx=0x7fd90000be70 comp rx=0 tx=0).stop 2024-04-22T13:28:35.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.885+0000 7fd91916b740 1 -- 172.21.15.12:0/2122018929 shutdown_connections 2024-04-22T13:28:35.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.885+0000 7fd91916b740 1 --2- 172.21.15.12:0/2122018929 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e2019961e0 0x55e201998600 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.885+0000 7fd91916b740 1 --2- 172.21.15.12:0/2122018929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e201916a00 0x55e201910d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.885+0000 7fd8ef7fe640 0 client.17421 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:35.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.885+0000 7fd91916b740 1 --2- 172.21.15.12:0/2122018929 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e20190de80 0x55e201910850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.885+0000 7fd91916b740 1 --2- 172.21.15.12:0/2122018929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e20190d4d0 0x55e201910310 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.885+0000 7fd91916b740 1 -- 172.21.15.12:0/2122018929 >> 172.21.15.12:0/2122018929 conn(0x55e201907ba0 msgr2=0x55e201906370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:35.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.885+0000 7fd91916b740 1 -- 172.21.15.12:0/2122018929 shutdown_connections 2024-04-22T13:28:35.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.885+0000 7fd91916b740 1 -- 172.21.15.12:0/2122018929 wait complete. 2024-04-22T13:28:35.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.885+0000 7fd91916b740 20 client.17421 trim_cache size 0 max 16384 2024-04-22T13:28:35.889 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:28:35.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.885+0000 7fd91649a640 1 -- 172.21.15.12:0/3199326493 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd8f80a3d50 msgr2=0x7fd8f80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.885+0000 7fd91649a640 1 --2- 172.21.15.12:0/3199326493 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd8f80a3d50 0x7fd8f80a6210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd908021000 tx=0x7fd90801e000 comp rx=0 tx=0).stop 2024-04-22T13:28:35.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.885+0000 7fd91649a640 1 -- 172.21.15.12:0/3199326493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd910107270 msgr2=0x7fd9101013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:35.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.885+0000 7fd91649a640 1 --2- 172.21.15.12:0/3199326493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd910107270 0x7fd9101013c0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fd904036040 tx=0x7fd90402c7e0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.886+0000 7fd91649a640 1 -- 172.21.15.12:0/3199326493 shutdown_connections 2024-04-22T13:28:35.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.886+0000 7fd91649a640 1 --2- 172.21.15.12:0/3199326493 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd8f80a3d50 0x7fd8f80a6210 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.886+0000 7fd91649a640 1 --2- 172.21.15.12:0/3199326493 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd910101900 0x7fd9101a68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.886+0000 7fd91649a640 1 --2- 172.21.15.12:0/3199326493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd910107270 0x7fd9101013c0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.886+0000 7fd91649a640 1 --2- 172.21.15.12:0/3199326493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd91006a840 0x7fd910100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:35.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.886+0000 7fd91649a640 1 -- 172.21.15.12:0/3199326493 >> 172.21.15.12:0/3199326493 conn(0x7fd910077eb0 msgr2=0x7fd910108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:35.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.886+0000 7fd91649a640 1 -- 172.21.15.12:0/3199326493 shutdown_connections 2024-04-22T13:28:35.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:35.886+0000 7fd91649a640 1 -- 172.21.15.12:0/3199326493 wait complete. 2024-04-22T13:28:35.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:35 smithi134 ceph-mon[31338]: pgmap v137: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:35.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:35 smithi134 ceph-mon[31338]: scrub queued for path: / 2024-04-22T13:28:35.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:35 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:28:35.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:35 smithi134 ceph-mon[31338]: scrub summary: active paths [/] 2024-04-22T13:28:35.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:35 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:28:35.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:35 smithi134 ceph-mon[31338]: scrub summary: idle 2024-04-22T13:28:35.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:35 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2668414817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:28:35.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:35 smithi111 ceph-mon[31361]: pgmap v137: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:35.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:35 smithi111 ceph-mon[31361]: scrub queued for path: / 2024-04-22T13:28:35.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:35 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:28:35.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:35 smithi111 ceph-mon[31361]: scrub summary: active paths [/] 2024-04-22T13:28:35.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:35 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:28:35.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:35 smithi111 ceph-mon[31361]: scrub summary: idle 2024-04-22T13:28:35.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:35 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2668414817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:28:36.893 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:28:36.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:28:36.894 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:28:37.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.017+0000 7f04c10c1640 1 -- 172.21.15.12:0/2133888457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04bc101a50 msgr2=0x7f04bc105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:37.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.017+0000 7f04c10c1640 1 --2- 172.21.15.12:0/2133888457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04bc101a50 0x7f04bc105a90 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009900 tx=0x7f04a402bc30 comp rx=0 tx=0).stop 2024-04-22T13:28:37.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.017+0000 7f04c10c1640 1 -- 172.21.15.12:0/2133888457 shutdown_connections 2024-04-22T13:28:37.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.017+0000 7f04c10c1640 1 --2- 172.21.15.12:0/2133888457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04bc106170 0x7f04bc10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.017+0000 7f04c10c1640 1 --2- 172.21.15.12:0/2133888457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04bc101a50 0x7f04bc105a90 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.017+0000 7f04c10c1640 1 --2- 172.21.15.12:0/2133888457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04bc1010a0 0x7f04bc101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.017+0000 7f04c10c1640 1 -- 172.21.15.12:0/2133888457 >> 172.21.15.12:0/2133888457 conn(0x7f04bc077eb0 msgr2=0x7f04bc0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:37.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.017+0000 7f04c10c1640 1 -- 172.21.15.12:0/2133888457 shutdown_connections 2024-04-22T13:28:37.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.017+0000 7f04c10c1640 1 -- 172.21.15.12:0/2133888457 wait complete. 2024-04-22T13:28:37.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.018+0000 7f04c10c1640 1 Processor -- start 2024-04-22T13:28:37.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.018+0000 7f04c10c1640 1 -- start start 2024-04-22T13:28:37.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.018+0000 7f04c10c1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04bc1010a0 0x7f04bc19e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.018+0000 7f04c10c1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04bc106170 0x7f04bc19e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.018+0000 7f04c10c1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04bc1985e0 0x7f04bc198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.018+0000 7f04c10c1640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc111090 con 0x7f04bc1985e0 2024-04-22T13:28:37.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.018+0000 7f04c10c1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc110f10 con 0x7f04bc106170 2024-04-22T13:28:37.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.018+0000 7f04c10c1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc111210 con 0x7f04bc1010a0 2024-04-22T13:28:37.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.019+0000 7f04bb59e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04bc1985e0 0x7f04bc198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.019+0000 7f04bb59e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04bc1985e0 0x7f04bc198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33308/0 (socket says 172.21.15.12:33308) 2024-04-22T13:28:37.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.019+0000 7f04bb59e640 1 -- 172.21.15.12:0/1124624492 learned_addr learned my addr 172.21.15.12:0/1124624492 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:37.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.019+0000 7f04bad9d640 1 --2- 172.21.15.12:0/1124624492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04bc1010a0 0x7f04bc19e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.019+0000 7f04b259c640 1 --2- 172.21.15.12:0/1124624492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04bc106170 0x7f04bc19e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.019+0000 7f04bb59e640 1 -- 172.21.15.12:0/1124624492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04bc1010a0 msgr2=0x7f04bc19e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:37.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.019+0000 7f04bb59e640 1 --2- 172.21.15.12:0/1124624492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04bc1010a0 0x7f04bc19e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.019+0000 7f04bb59e640 1 -- 172.21.15.12:0/1124624492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04bc106170 msgr2=0x7f04bc19e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:37.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.019+0000 7f04bb59e640 1 --2- 172.21.15.12:0/1124624492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04bc106170 0x7f04bc19e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.019+0000 7f04bb59e640 1 -- 172.21.15.12:0/1124624492 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04bc199100 con 0x7f04bc1985e0 2024-04-22T13:28:37.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.019+0000 7f04bb59e640 1 --2- 172.21.15.12:0/1124624492 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04bc1985e0 0x7f04bc198a90 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f04b400eaa0 tx=0x7f04b400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:37.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.019+0000 7f04b8d99640 1 -- 172.21.15.12:0/1124624492 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b400cdb0 con 0x7f04bc1985e0 2024-04-22T13:28:37.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.019+0000 7f04b8d99640 1 -- 172.21.15.12:0/1124624492 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04b4005020 con 0x7f04bc1985e0 2024-04-22T13:28:37.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.019+0000 7f04c10c1640 1 -- 172.21.15.12:0/1124624492 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04bc1993f0 con 0x7f04bc1985e0 2024-04-22T13:28:37.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.019+0000 7f04b8d99640 1 -- 172.21.15.12:0/1124624492 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b40116d0 con 0x7f04bc1985e0 2024-04-22T13:28:37.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.020+0000 7f04c10c1640 1 -- 172.21.15.12:0/1124624492 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04bc1a55f0 con 0x7f04bc1985e0 2024-04-22T13:28:37.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.022+0000 7f04b8d99640 1 -- 172.21.15.12:0/1124624492 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f04b40051d0 con 0x7f04bc1985e0 2024-04-22T13:28:37.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.022+0000 7f04c3d90740 1 Processor -- start 2024-04-22T13:28:37.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.022+0000 7f04c3d90740 1 -- start start 2024-04-22T13:28:37.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.023+0000 7f04c3d90740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565253acb8e0 0x565253acbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.023+0000 7f04c3d90740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565253acc290 0x565253ad4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.023+0000 7f04c3d90740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565253ad4e10 0x565253ad7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.023+0000 7f04c3d90740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x565253a2ff00 con 0x565253acc290 2024-04-22T13:28:37.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.023+0000 7f04c3d90740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5652539cd590 con 0x565253ad4e10 2024-04-22T13:28:37.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.023+0000 7f04c3d90740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x565253a2bec0 con 0x565253acb8e0 2024-04-22T13:28:37.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.023+0000 7f04b8d99640 1 --2- 172.21.15.12:0/1124624492 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f04a80a3d20 0x7f04a80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.023+0000 7f04bad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565253acc290 0x565253ad4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.023+0000 7f04bad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565253acc290 0x565253ad4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33314/0 (socket says 172.21.15.12:33314) 2024-04-22T13:28:37.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.023+0000 7f04bad9d640 1 -- 172.21.15.12:0/953945332 learned_addr learned my addr 172.21.15.12:0/953945332 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:37.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.023+0000 7f04bb59e640 1 --2- 172.21.15.12:0/1124624492 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f04a80a3d20 0x7f04a80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.023+0000 7f04b259c640 1 --2- 172.21.15.12:0/953945332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565253acb8e0 0x565253acbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.023+0000 7f04b259c640 1 --2- 172.21.15.12:0/953945332 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565253ad4e10 0x565253ad7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.023+0000 7f04b8d99640 1 -- 172.21.15.12:0/1124624492 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f04b40d0950 con 0x7f04bc1985e0 2024-04-22T13:28:37.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.023+0000 7f04bad9d640 1 -- 172.21.15.12:0/953945332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565253acb8e0 msgr2=0x565253acbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:37.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.023+0000 7f04bad9d640 1 --2- 172.21.15.12:0/953945332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565253acb8e0 0x565253acbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.023+0000 7f04bad9d640 1 -- 172.21.15.12:0/953945332 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565253ad4e10 msgr2=0x565253ad7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:37.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.023+0000 7f04bad9d640 1 --2- 172.21.15.12:0/953945332 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565253ad4e10 0x565253ad7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.023+0000 7f04bad9d640 1 -- 172.21.15.12:0/953945332 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56525380a7e0 con 0x565253acc290 2024-04-22T13:28:37.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.023+0000 7f04b259c640 1 --2- 172.21.15.12:0/953945332 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565253ad4e10 0x565253ad7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:37.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.023+0000 7f04bad9d640 1 --2- 172.21.15.12:0/953945332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565253acc290 0x565253ad4840 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f04ac009cf0 tx=0x7f04ac011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b1901d0015ee25 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:37.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.023+0000 7f04bb59e640 1 --2- 172.21.15.12:0/1124624492 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f04a80a3d20 0x7f04a80a61e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f04b40ce6f0 tx=0x7f04b406f430 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:37.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.024+0000 7f04b159a640 1 -- 172.21.15.12:0/953945332 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac004230 con 0x565253acc290 2024-04-22T13:28:37.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.024+0000 7f04b159a640 1 -- 172.21.15.12:0/953945332 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04ac0043d0 con 0x565253acc290 2024-04-22T13:28:37.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.024+0000 7f04b159a640 1 -- 172.21.15.12:0/953945332 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac019880 con 0x565253acc290 2024-04-22T13:28:37.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.024+0000 7f04c3d90740 1 -- 172.21.15.12:0/953945332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565253acc290 msgr2=0x565253ad4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:37.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.024+0000 7f04c3d90740 1 --2- 172.21.15.12:0/953945332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565253acc290 0x565253ad4840 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f04ac009cf0 tx=0x7f04ac011810 comp rx=0 tx=0).stop 2024-04-22T13:28:37.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.024+0000 7f04c3d90740 1 -- 172.21.15.12:0/953945332 shutdown_connections 2024-04-22T13:28:37.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.024+0000 7f04c3d90740 1 --2- 172.21.15.12:0/953945332 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565253ad4e10 0x565253ad7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.024+0000 7f04c3d90740 1 --2- 172.21.15.12:0/953945332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565253acc290 0x565253ad4840 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.024+0000 7f04c3d90740 1 --2- 172.21.15.12:0/953945332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565253acb8e0 0x565253acbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.024+0000 7f04c3d90740 1 -- 172.21.15.12:0/953945332 >> 172.21.15.12:0/953945332 conn(0x565253ac5fb0 msgr2=0x565253ac6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:37.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.024+0000 7f04c3d90740 1 -- 172.21.15.12:0/953945332 shutdown_connections 2024-04-22T13:28:37.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.025+0000 7f04c3d90740 1 -- 172.21.15.12:0/953945332 wait complete. 2024-04-22T13:28:37.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.025+0000 7f04c3d90740 1 -- start start 2024-04-22T13:28:37.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.025+0000 7f04c3d90740 1 Processor -- start 2024-04-22T13:28:37.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.025+0000 7f04c3d90740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565253acb8e0 0x565253ad1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.025+0000 7f04c3d90740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565253acc290 0x565253ad2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.025+0000 7f04c3d90740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565253ad4e10 0x565253ad2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.025+0000 7f04c3d90740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x565253ad7d80 con 0x565253ad4e10 2024-04-22T13:28:37.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.025+0000 7f04c3d90740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5652539e9580 con 0x565253acc290 2024-04-22T13:28:37.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.025+0000 7f04c3d90740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x565253ad7f00 con 0x565253acb8e0 2024-04-22T13:28:37.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.025+0000 7f04c3d90740 10 client.0 _ll_register_callbacks cb 0x5652539ef650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:37.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.025+0000 7f04b259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565253ad4e10 0x565253ad2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.025+0000 7f04b259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565253ad4e10 0x565253ad2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33324/0 (socket says 172.21.15.12:33324) 2024-04-22T13:28:37.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.025+0000 7f04b259c640 1 -- 172.21.15.12:0/3025108705 learned_addr learned my addr 172.21.15.12:0/3025108705 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:37.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.025+0000 7f04b259c640 1 --2- 172.21.15.12:0/3025108705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565253acb8e0 0x565253ad1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.026+0000 7f04bad9d640 1 --2- 172.21.15.12:0/3025108705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565253acc290 0x565253ad2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.026+0000 7f04b259c640 1 -- 172.21.15.12:0/3025108705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565253acb8e0 msgr2=0x565253ad1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:37.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.026+0000 7f04b259c640 1 --2- 172.21.15.12:0/3025108705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565253acb8e0 0x565253ad1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.026+0000 7f04b259c640 1 -- 172.21.15.12:0/3025108705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565253acc290 msgr2=0x565253ad2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:37.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.026+0000 7f04b259c640 1 --2- 172.21.15.12:0/3025108705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565253acc290 0x565253ad2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.026+0000 7f04b259c640 1 -- 172.21.15.12:0/3025108705 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56525386af70 con 0x565253ad4e10 2024-04-22T13:28:37.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.026+0000 7f04bad9d640 1 --2- 172.21.15.12:0/3025108705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565253acc290 0x565253ad2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:37.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.026+0000 7f04b259c640 1 --2- 172.21.15.12:0/3025108705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565253ad4e10 0x565253ad2870 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f04bc06a420 tx=0x7f04a402c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988555d5d179b9da server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:37.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.026+0000 7f04a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:37.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.026+0000 7f04a27fc640 1 -- 172.21.15.12:0/3025108705 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a4005030 con 0x565253ad4e10 2024-04-22T13:28:37.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.026+0000 7f04c3d90740 1 -- 172.21.15.12:0/3025108705 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04ac0043d0 con 0x565253ad4e10 2024-04-22T13:28:37.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.026+0000 7f04a27fc640 1 -- 172.21.15.12:0/3025108705 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f04a40051d0 con 0x565253ad4e10 2024-04-22T13:28:37.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.026+0000 7f04a27fc640 1 -- 172.21.15.12:0/3025108705 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f04a4054050 con 0x565253ad4e10 2024-04-22T13:28:37.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.026+0000 7f04c3d90740 10 client.17433 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:37.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.026+0000 7f04c3d90740 1 -- 172.21.15.12:0/3025108705 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565253702170 con 0x565253ad4e10 2024-04-22T13:28:37.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.026+0000 7f04a27fc640 1 -- 172.21.15.12:0/3025108705 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04ac0043d0 con 0x565253ad4e10 2024-04-22T13:28:37.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.027+0000 7f04a27fc640 1 -- 172.21.15.12:0/3025108705 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f04a4040040 con 0x565253ad4e10 2024-04-22T13:28:37.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.027+0000 7f04c3d90740 10 client.17433 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:37.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.027+0000 7f04c3d90740 10 client.17433 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:37.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.027+0000 7f04c3d90740 20 client.17433 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:37.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.027+0000 7f04c3d90740 1 --2- 172.21.15.12:0/3025108705 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x565253ae5330 0x565253ae7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.027+0000 7f04c3d90740 4 client.17433 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:37.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.027+0000 7f04c3d90740 1 -- 172.21.15.12:0/3025108705 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565253a3d920 con 0x565253ae5330 2024-04-22T13:28:37.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.027+0000 7f04bad9d640 1 --2- 172.21.15.12:0/3025108705 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x565253ae5330 0x565253ae7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.027+0000 7f04a27fc640 10 client.17433 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:37.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.027+0000 7f04bad9d640 1 --2- 172.21.15.12:0/3025108705 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x565253ae5330 0x565253ae7780 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e764235fa10fd797 server_cookie=73461fcbb48c5375 in_seq=0 out_seq=0 2024-04-22T13:28:37.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.028+0000 7f04a27fc640 1 -- 172.21.15.12:0/3025108705 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565253a3d920 con 0x565253ae5330 2024-04-22T13:28:37.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.028+0000 7f04a27fc640 10 client.17433 handle_command_reply: tid=0 2024-04-22T13:28:37.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.028+0000 7f04c3d90740 1 client.17433 shutdown 2024-04-22T13:28:37.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.028+0000 7f04c3d90740 1 -- 172.21.15.12:0/3025108705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565253ad4e10 msgr2=0x565253ad2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:37.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.028+0000 7f04c3d90740 1 --2- 172.21.15.12:0/3025108705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565253ad4e10 0x565253ad2870 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f04bc06a420 tx=0x7f04a402c5d0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.028+0000 7f04c3d90740 1 -- 172.21.15.12:0/3025108705 shutdown_connections 2024-04-22T13:28:37.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.028+0000 7f04c3d90740 1 --2- 172.21.15.12:0/3025108705 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x565253ae5330 0x565253ae7780 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.028+0000 7f04c3d90740 1 --2- 172.21.15.12:0/3025108705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565253ad4e10 0x565253ad2870 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.028+0000 7f04c3d90740 1 --2- 172.21.15.12:0/3025108705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565253acc290 0x565253ad2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.028+0000 7f04a27fc640 0 client.17433 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:37.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.028+0000 7f04c3d90740 1 --2- 172.21.15.12:0/3025108705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565253acb8e0 0x565253ad1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.028+0000 7f04c3d90740 1 -- 172.21.15.12:0/3025108705 >> 172.21.15.12:0/3025108705 conn(0x565253ac5fb0 msgr2=0x565253ac7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:37.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.028+0000 7f04c3d90740 1 -- 172.21.15.12:0/3025108705 shutdown_connections 2024-04-22T13:28:37.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.029+0000 7f04c3d90740 1 -- 172.21.15.12:0/3025108705 wait complete. 2024-04-22T13:28:37.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.029+0000 7f04c3d90740 20 client.17433 trim_cache size 0 max 16384 2024-04-22T13:28:37.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.046+0000 7f04c3d90740 1 Processor -- start 2024-04-22T13:28:37.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.046+0000 7f04c3d90740 1 -- start start 2024-04-22T13:28:37.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.047+0000 7f04c3d90740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565253acb8e0 0x565253ae0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.047+0000 7f04c3d90740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565253acc290 0x565253ad16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.047+0000 7f04c3d90740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565253ad4e10 0x565253aed050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.047+0000 7f04c3d90740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc1adab0 con 0x565253acb8e0 2024-04-22T13:28:37.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.047+0000 7f04c3d90740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc1af5f0 con 0x565253ad4e10 2024-04-22T13:28:37.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.047+0000 7f04c3d90740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc1ae180 con 0x565253acc290 2024-04-22T13:28:37.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.047+0000 7f04b259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565253acb8e0 0x565253ae0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.047+0000 7f04b259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565253acb8e0 0x565253ae0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33326/0 (socket says 172.21.15.12:33326) 2024-04-22T13:28:37.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.047+0000 7f04b259c640 1 -- 172.21.15.12:0/2968777810 learned_addr learned my addr 172.21.15.12:0/2968777810 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:37.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.047+0000 7f04bad9d640 1 --2- 172.21.15.12:0/2968777810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565253acc290 0x565253ad16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.047+0000 7f04b259c640 1 -- 172.21.15.12:0/2968777810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565253acc290 msgr2=0x565253ad16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:37.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.047+0000 7f04b259c640 1 --2- 172.21.15.12:0/2968777810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565253acc290 0x565253ad16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.047+0000 7f04b259c640 1 -- 172.21.15.12:0/2968777810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565253ad4e10 msgr2=0x565253aed050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:37.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.047+0000 7f04b259c640 1 --2- 172.21.15.12:0/2968777810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565253ad4e10 0x565253aed050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.047+0000 7f04b259c640 1 -- 172.21.15.12:0/2968777810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565253acd600 con 0x565253acb8e0 2024-04-22T13:28:37.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.047+0000 7f04bad9d640 1 --2- 172.21.15.12:0/2968777810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565253acc290 0x565253ad16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:37.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.047+0000 7f04b259c640 1 --2- 172.21.15.12:0/2968777810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565253acb8e0 0x565253ae0c70 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f04a40098b0 tx=0x7f04a402cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd79e2df647b0bf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:37.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.047+0000 7f04b0d99640 1 -- 172.21.15.12:0/2968777810 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a4000e60 con 0x565253acb8e0 2024-04-22T13:28:37.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.048+0000 7f04b0d99640 1 -- 172.21.15.12:0/2968777810 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04a4049020 con 0x565253acb8e0 2024-04-22T13:28:37.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.048+0000 7f04b0d99640 1 -- 172.21.15.12:0/2968777810 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a4004920 con 0x565253acb8e0 2024-04-22T13:28:37.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.048+0000 7f04c3d90740 1 -- 172.21.15.12:0/2968777810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565253acb8e0 msgr2=0x565253ae0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:37.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.048+0000 7f04c3d90740 1 --2- 172.21.15.12:0/2968777810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565253acb8e0 0x565253ae0c70 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f04a40098b0 tx=0x7f04a402cd30 comp rx=0 tx=0).stop 2024-04-22T13:28:37.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.048+0000 7f04c3d90740 1 -- 172.21.15.12:0/2968777810 shutdown_connections 2024-04-22T13:28:37.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.048+0000 7f04c3d90740 1 --2- 172.21.15.12:0/2968777810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565253ad4e10 0x565253aed050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.048+0000 7f04c3d90740 1 --2- 172.21.15.12:0/2968777810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565253acc290 0x565253ad16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.048+0000 7f04c3d90740 1 --2- 172.21.15.12:0/2968777810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565253acb8e0 0x565253ae0c70 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.048+0000 7f04c3d90740 1 -- 172.21.15.12:0/2968777810 >> 172.21.15.12:0/2968777810 conn(0x565253ac5fb0 msgr2=0x565253ac4780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:37.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.048+0000 7f04c3d90740 1 -- 172.21.15.12:0/2968777810 shutdown_connections 2024-04-22T13:28:37.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.049+0000 7f04c3d90740 1 -- 172.21.15.12:0/2968777810 wait complete. 2024-04-22T13:28:37.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.049+0000 7f04c3d90740 1 -- start start 2024-04-22T13:28:37.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.049+0000 7f04c3d90740 1 Processor -- start 2024-04-22T13:28:37.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.049+0000 7f04c3d90740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565253acb8e0 0x565253add5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.049+0000 7f04c3d90740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565253acc290 0x565253ad8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.049+0000 7f04c3d90740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565253ad4e10 0x565253ad8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.049+0000 7f04c3d90740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc1aef20 con 0x565253acc290 2024-04-22T13:28:37.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.049+0000 7f04c3d90740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc1ae850 con 0x565253acb8e0 2024-04-22T13:28:37.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.049+0000 7f04c3d90740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc1abab0 con 0x565253ad4e10 2024-04-22T13:28:37.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.049+0000 7f04c3d90740 10 client.0 _ll_register_callbacks cb 0x7f04a4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:37.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.049+0000 7f04bad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565253acc290 0x565253ad8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.049+0000 7f04bad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565253acc290 0x565253ad8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33332/0 (socket says 172.21.15.12:33332) 2024-04-22T13:28:37.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.049+0000 7f04bad9d640 1 -- 172.21.15.12:0/679539647 learned_addr learned my addr 172.21.15.12:0/679539647 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:37.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.049+0000 7f04b259c640 1 --2- 172.21.15.12:0/679539647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565253acb8e0 0x565253add5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.049+0000 7f04b259c640 1 --2- 172.21.15.12:0/679539647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565253ad4e10 0x565253ad8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.050+0000 7f04bad9d640 1 -- 172.21.15.12:0/679539647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565253ad4e10 msgr2=0x565253ad8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:37.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.050+0000 7f04bad9d640 1 --2- 172.21.15.12:0/679539647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565253ad4e10 0x565253ad8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.050+0000 7f04bad9d640 1 -- 172.21.15.12:0/679539647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565253acb8e0 msgr2=0x565253add5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:37.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.050+0000 7f04bad9d640 1 --2- 172.21.15.12:0/679539647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565253acb8e0 0x565253add5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.050+0000 7f04bad9d640 1 -- 172.21.15.12:0/679539647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565253ad96f0 con 0x565253acc290 2024-04-22T13:28:37.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.050+0000 7f04bad9d640 1 --2- 172.21.15.12:0/679539647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565253acc290 0x565253ad8910 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f04ac009250 tx=0x7f04ac009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df67f4d146576eb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:37.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.050+0000 7f04a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:37.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.050+0000 7f04a27fc640 1 -- 172.21.15.12:0/679539647 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac0035f0 con 0x565253acc290 2024-04-22T13:28:37.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.050+0000 7f04a27fc640 1 -- 172.21.15.12:0/679539647 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f04ac003790 con 0x565253acc290 2024-04-22T13:28:37.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.050+0000 7f04c3d90740 1 -- 172.21.15.12:0/679539647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04a4049020 con 0x565253acc290 2024-04-22T13:28:37.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.050+0000 7f04a27fc640 1 -- 172.21.15.12:0/679539647 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f04ac0039b0 con 0x565253acc290 2024-04-22T13:28:37.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.050+0000 7f04c3d90740 10 client.17439 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:37.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.050+0000 7f04c3d90740 1 -- 172.21.15.12:0/679539647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565253b4b910 con 0x565253acc290 2024-04-22T13:28:37.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.050+0000 7f04a27fc640 1 -- 172.21.15.12:0/679539647 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04a4049020 con 0x565253acc290 2024-04-22T13:28:37.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.051+0000 7f04a27fc640 1 -- 172.21.15.12:0/679539647 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f04ac02c020 con 0x565253acc290 2024-04-22T13:28:37.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.051+0000 7f04c3d90740 10 client.17439 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:37.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.051+0000 7f04c3d90740 10 client.17439 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:37.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.051+0000 7f04c3d90740 20 client.17439 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:37.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.051+0000 7f04c3d90740 1 --2- 172.21.15.12:0/679539647 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x565253b4bee0 0x565253b4e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.051+0000 7f04c3d90740 4 client.17439 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:28:37.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.051+0000 7f04c3d90740 1 -- 172.21.15.12:0/679539647 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x565253a37790 con 0x565253b4bee0 2024-04-22T13:28:37.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.051+0000 7f04b259c640 1 --2- 172.21.15.12:0/679539647 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x565253b4bee0 0x565253b4e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.051+0000 7f04a27fc640 10 client.17439 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:37.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.051+0000 7f04b259c640 1 --2- 172.21.15.12:0/679539647 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x565253b4bee0 0x565253b4e300 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3cb92d9c501ddb server_cookie=d8494b5806c13dc8 in_seq=0 out_seq=0 2024-04-22T13:28:37.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.052+0000 7f04a27fc640 1 -- 172.21.15.12:0/679539647 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x565253a37790 con 0x565253b4bee0 2024-04-22T13:28:37.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.052+0000 7f04a27fc640 10 client.17439 handle_command_reply: tid=0 2024-04-22T13:28:37.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.052+0000 7f04c3d90740 1 client.17439 shutdown 2024-04-22T13:28:37.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.053+0000 7f04c3d90740 1 -- 172.21.15.12:0/679539647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565253acc290 msgr2=0x565253ad8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:37.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.053+0000 7f04c3d90740 1 --2- 172.21.15.12:0/679539647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565253acc290 0x565253ad8910 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f04ac009250 tx=0x7f04ac009c20 comp rx=0 tx=0).stop 2024-04-22T13:28:37.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.053+0000 7f04c3d90740 1 -- 172.21.15.12:0/679539647 shutdown_connections 2024-04-22T13:28:37.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.053+0000 7f04c3d90740 1 --2- 172.21.15.12:0/679539647 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x565253b4bee0 0x565253b4e300 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.053+0000 7f04c3d90740 1 --2- 172.21.15.12:0/679539647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565253ad4e10 0x565253ad8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.053+0000 7f04c3d90740 1 --2- 172.21.15.12:0/679539647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565253acc290 0x565253ad8910 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.053+0000 7f04a27fc640 0 client.17439 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:37.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.053+0000 7f04c3d90740 1 --2- 172.21.15.12:0/679539647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565253acb8e0 0x565253add5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.053+0000 7f04c3d90740 1 -- 172.21.15.12:0/679539647 >> 172.21.15.12:0/679539647 conn(0x565253ac5fb0 msgr2=0x565253ac5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:37.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.053+0000 7f04c3d90740 1 -- 172.21.15.12:0/679539647 shutdown_connections 2024-04-22T13:28:37.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.053+0000 7f04c3d90740 1 -- 172.21.15.12:0/679539647 wait complete. 2024-04-22T13:28:37.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.053+0000 7f04c3d90740 20 client.17439 trim_cache size 0 max 16384 2024-04-22T13:28:37.054 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:28:37.054 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-04-22T13:28:37.054 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "abe91ecc-f3ef-47be-9d41-ae8b0d64dc53", 2024-04-22T13:28:37.054 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-04-22T13:28:37.055 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:28:37.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.053+0000 7f04c10c1640 1 -- 172.21.15.12:0/1124624492 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f04a80a3d20 msgr2=0x7f04a80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:37.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.053+0000 7f04c10c1640 1 --2- 172.21.15.12:0/1124624492 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f04a80a3d20 0x7f04a80a61e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f04b40ce6f0 tx=0x7f04b406f430 comp rx=0 tx=0).stop 2024-04-22T13:28:37.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.053+0000 7f04c10c1640 1 -- 172.21.15.12:0/1124624492 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04bc1985e0 msgr2=0x7f04bc198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:37.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.053+0000 7f04c10c1640 1 --2- 172.21.15.12:0/1124624492 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04bc1985e0 0x7f04bc198a90 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f04b400eaa0 tx=0x7f04b400ef60 comp rx=0 tx=0).stop 2024-04-22T13:28:37.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.053+0000 7f04c10c1640 1 -- 172.21.15.12:0/1124624492 shutdown_connections 2024-04-22T13:28:37.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.053+0000 7f04c10c1640 1 --2- 172.21.15.12:0/1124624492 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f04a80a3d20 0x7f04a80a61e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.053+0000 7f04c10c1640 1 --2- 172.21.15.12:0/1124624492 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f04bc1985e0 0x7f04bc198a90 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.053+0000 7f04c10c1640 1 --2- 172.21.15.12:0/1124624492 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04bc106170 0x7f04bc19e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.053+0000 7f04c10c1640 1 --2- 172.21.15.12:0/1124624492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04bc1010a0 0x7f04bc19e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.053+0000 7f04c10c1640 1 -- 172.21.15.12:0/1124624492 >> 172.21.15.12:0/1124624492 conn(0x7f04bc077eb0 msgr2=0x7f04bc06a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:37.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.054+0000 7f04c10c1640 1 -- 172.21.15.12:0/1124624492 shutdown_connections 2024-04-22T13:28:37.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.054+0000 7f04c10c1640 1 -- 172.21.15.12:0/1124624492 wait complete. 2024-04-22T13:28:37.060 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:28:37.061 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:28:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.171+0000 7f0c9b043640 1 -- 172.21.15.12:0/3651725682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c9410c500 msgr2=0x7f0c9410e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.171+0000 7f0c9b043640 1 --2- 172.21.15.12:0/3651725682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c9410c500 0x7f0c9410e9a0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c009ca0 tx=0x7f0c8c02c230 comp rx=0 tx=0).stop 2024-04-22T13:28:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.171+0000 7f0c9b043640 1 -- 172.21.15.12:0/3651725682 shutdown_connections 2024-04-22T13:28:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.171+0000 7f0c9b043640 1 --2- 172.21.15.12:0/3651725682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c9410c500 0x7f0c9410e9a0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.171+0000 7f0c9b043640 1 --2- 172.21.15.12:0/3651725682 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c94103ae0 0x7f0c9410bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.171+0000 7f0c9b043640 1 --2- 172.21.15.12:0/3651725682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c941031c0 0x7f0c941035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.171+0000 7f0c9b043640 1 -- 172.21.15.12:0/3651725682 >> 172.21.15.12:0/3651725682 conn(0x7f0c94078000 msgr2=0x7f0c94078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.171+0000 7f0c9b043640 1 -- 172.21.15.12:0/3651725682 shutdown_connections 2024-04-22T13:28:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.171+0000 7f0c9b043640 1 -- 172.21.15.12:0/3651725682 wait complete. 2024-04-22T13:28:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.171+0000 7f0c9b043640 1 Processor -- start 2024-04-22T13:28:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.172+0000 7f0c9b043640 1 -- start start 2024-04-22T13:28:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.172+0000 7f0c9b043640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c941031c0 0x7f0c941a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.172+0000 7f0c9b043640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c94103ae0 0x7f0c941a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.172+0000 7f0c9b043640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c9410c500 0x7f0c9419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.172+0000 7f0c9b043640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0c94112190 con 0x7f0c94103ae0 2024-04-22T13:28:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.172+0000 7f0c9b043640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0c94112010 con 0x7f0c941031c0 2024-04-22T13:28:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.172+0000 7f0c9b043640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0c94112310 con 0x7f0c9410c500 2024-04-22T13:28:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.172+0000 7f0c93fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c94103ae0 0x7f0c941a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.172+0000 7f0c93fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c94103ae0 0x7f0c941a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33352/0 (socket says 172.21.15.12:33352) 2024-04-22T13:28:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.172+0000 7f0c93fff640 1 -- 172.21.15.12:0/3558034455 learned_addr learned my addr 172.21.15.12:0/3558034455 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.172+0000 7f0c995e0640 1 --2- 172.21.15.12:0/3558034455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c9410c500 0x7f0c9419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.172+0000 7f0c98ddf640 1 --2- 172.21.15.12:0/3558034455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c941031c0 0x7f0c941a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.172+0000 7f0c93fff640 1 -- 172.21.15.12:0/3558034455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c9410c500 msgr2=0x7f0c9419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.172+0000 7f0c93fff640 1 --2- 172.21.15.12:0/3558034455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c9410c500 0x7f0c9419a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.172+0000 7f0c93fff640 1 -- 172.21.15.12:0/3558034455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c941031c0 msgr2=0x7f0c941a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.172+0000 7f0c93fff640 1 --2- 172.21.15.12:0/3558034455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c941031c0 0x7f0c941a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.172+0000 7f0c93fff640 1 -- 172.21.15.12:0/3558034455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c9419aff0 con 0x7f0c94103ae0 2024-04-22T13:28:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.172+0000 7f0c98ddf640 1 --2- 172.21.15.12:0/3558034455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c941031c0 0x7f0c941a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.172+0000 7f0c93fff640 1 --2- 172.21.15.12:0/3558034455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c94103ae0 0x7f0c941a0bb0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000d950 tx=0x7f0c8000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.173+0000 7f0c91ffb640 1 -- 172.21.15.12:0/3558034455 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80017030 con 0x7f0c94103ae0 2024-04-22T13:28:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.173+0000 7f0c91ffb640 1 -- 172.21.15.12:0/3558034455 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c80004020 con 0x7f0c94103ae0 2024-04-22T13:28:37.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.173+0000 7f0c9b043640 1 -- 172.21.15.12:0/3558034455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c9419b2e0 con 0x7f0c94103ae0 2024-04-22T13:28:37.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.173+0000 7f0c91ffb640 1 -- 172.21.15.12:0/3558034455 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c800047a0 con 0x7f0c94103ae0 2024-04-22T13:28:37.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.173+0000 7f0c9b043640 1 -- 172.21.15.12:0/3558034455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c941a75f0 con 0x7f0c94103ae0 2024-04-22T13:28:37.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.175+0000 7f0c91ffb640 1 -- 172.21.15.12:0/3558034455 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0c80009a30 con 0x7f0c94103ae0 2024-04-22T13:28:37.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.175+0000 7f0c9dc3c740 1 Processor -- start 2024-04-22T13:28:37.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.176+0000 7f0c9dc3c740 1 -- start start 2024-04-22T13:28:37.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.176+0000 7f0c9dc3c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564992e30580 0x564992e30960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.176+0000 7f0c9dc3c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564992e30ea0 0x564992e39540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.176+0000 7f0c9dc3c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564992e39a80 0x564992e3be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.176+0000 7f0c9dc3c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564992d951e0 con 0x564992e39a80 2024-04-22T13:28:37.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.176+0000 7f0c9dc3c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564992d32390 con 0x564992e30580 2024-04-22T13:28:37.178 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.176+0000 7f0c9dc3c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564992d911a0 con 0x564992e30ea0 2024-04-22T13:28:37.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.176+0000 7f0c91ffb640 1 --2- 172.21.15.12:0/3558034455 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f0c840a3d20 0x7f0c840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.176+0000 7f0c93fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564992e39a80 0x564992e3be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.176+0000 7f0c93fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564992e39a80 0x564992e3be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33354/0 (socket says 172.21.15.12:33354) 2024-04-22T13:28:37.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.176+0000 7f0c93fff640 1 -- 172.21.15.12:0/4230599055 learned_addr learned my addr 172.21.15.12:0/4230599055 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:37.179 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.176+0000 7f0c995e0640 1 --2- 172.21.15.12:0/3558034455 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f0c840a3d20 0x7f0c840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.176+0000 7f0c98ddf640 1 --2- 172.21.15.12:0/4230599055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564992e30ea0 0x564992e39540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.176+0000 7f0c995e0640 1 --2- 172.21.15.12:0/4230599055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564992e30580 0x564992e30960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.176+0000 7f0c91ffb640 1 -- 172.21.15.12:0/3558034455 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f0c800d01c0 con 0x7f0c94103ae0 2024-04-22T13:28:37.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.176+0000 7f0c93fff640 1 -- 172.21.15.12:0/4230599055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564992e30ea0 msgr2=0x564992e39540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:37.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.176+0000 7f0c93fff640 1 --2- 172.21.15.12:0/4230599055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564992e30ea0 0x564992e39540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.176+0000 7f0c93fff640 1 -- 172.21.15.12:0/4230599055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564992e30580 msgr2=0x564992e30960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:37.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.176+0000 7f0c93fff640 1 --2- 172.21.15.12:0/4230599055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564992e30580 0x564992e30960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.176+0000 7f0c93fff640 1 -- 172.21.15.12:0/4230599055 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564992b717d0 con 0x564992e39a80 2024-04-22T13:28:37.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.176+0000 7f0c93fff640 1 --2- 172.21.15.12:0/4230599055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564992e39a80 0x564992e3be70 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f0c800cd840 tx=0x7f0c8006ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667f81205ff064a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:37.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.177+0000 7f0c995e0640 1 --2- 172.21.15.12:0/3558034455 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f0c840a3d20 0x7f0c840a61e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c004af0 tx=0x7f0c8c02e040 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:37.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.177+0000 7f0c7effd640 1 -- 172.21.15.12:0/4230599055 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80066c60 con 0x564992e39a80 2024-04-22T13:28:37.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.177+0000 7f0c7effd640 1 -- 172.21.15.12:0/4230599055 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c8009c850 con 0x564992e39a80 2024-04-22T13:28:37.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.177+0000 7f0c7effd640 1 -- 172.21.15.12:0/4230599055 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80009df0 con 0x564992e39a80 2024-04-22T13:28:37.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.177+0000 7f0c9dc3c740 1 -- 172.21.15.12:0/4230599055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564992e39a80 msgr2=0x564992e3be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:37.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.177+0000 7f0c9dc3c740 1 --2- 172.21.15.12:0/4230599055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564992e39a80 0x564992e3be70 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f0c800cd840 tx=0x7f0c8006ec40 comp rx=0 tx=0).stop 2024-04-22T13:28:37.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.177+0000 7f0c9dc3c740 1 -- 172.21.15.12:0/4230599055 shutdown_connections 2024-04-22T13:28:37.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.177+0000 7f0c9dc3c740 1 --2- 172.21.15.12:0/4230599055 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564992e39a80 0x564992e3be70 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.177+0000 7f0c9dc3c740 1 --2- 172.21.15.12:0/4230599055 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564992e30ea0 0x564992e39540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.177+0000 7f0c9dc3c740 1 --2- 172.21.15.12:0/4230599055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564992e30580 0x564992e30960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.177+0000 7f0c9dc3c740 1 -- 172.21.15.12:0/4230599055 >> 172.21.15.12:0/4230599055 conn(0x564992e2acb0 msgr2=0x564992e2b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:37.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.177+0000 7f0c9dc3c740 1 -- 172.21.15.12:0/4230599055 shutdown_connections 2024-04-22T13:28:37.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.177+0000 7f0c9dc3c740 1 -- 172.21.15.12:0/4230599055 wait complete. 2024-04-22T13:28:37.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.178+0000 7f0c9dc3c740 1 -- start start 2024-04-22T13:28:37.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.178+0000 7f0c9dc3c740 1 Processor -- start 2024-04-22T13:28:37.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.178+0000 7f0c9dc3c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564992e30580 0x564992e36a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.178+0000 7f0c9dc3c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564992e30ea0 0x564992e36fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.178+0000 7f0c9dc3c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564992e39a80 0x564992e374f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.178+0000 7f0c9dc3c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564992e3c810 con 0x564992e39a80 2024-04-22T13:28:37.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.178+0000 7f0c9dc3c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564992d4e2f0 con 0x564992e30580 2024-04-22T13:28:37.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.178+0000 7f0c9dc3c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564992e3c990 con 0x564992e30ea0 2024-04-22T13:28:37.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.178+0000 7f0c9dc3c740 10 client.0 _ll_register_callbacks cb 0x564992d5b580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:37.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.178+0000 7f0c995e0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564992e39a80 0x564992e374f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.178+0000 7f0c93fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564992e30ea0 0x564992e36fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.178+0000 7f0c995e0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564992e39a80 0x564992e374f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33364/0 (socket says 172.21.15.12:33364) 2024-04-22T13:28:37.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.178+0000 7f0c995e0640 1 -- 172.21.15.12:0/1593520582 learned_addr learned my addr 172.21.15.12:0/1593520582 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:37.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.179+0000 7f0c98ddf640 1 --2- 172.21.15.12:0/1593520582 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564992e30580 0x564992e36a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.179+0000 7f0c995e0640 1 -- 172.21.15.12:0/1593520582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564992e30ea0 msgr2=0x564992e36fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:37.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.179+0000 7f0c995e0640 1 --2- 172.21.15.12:0/1593520582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564992e30ea0 0x564992e36fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.179+0000 7f0c995e0640 1 -- 172.21.15.12:0/1593520582 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564992e30580 msgr2=0x564992e36a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:37.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.179+0000 7f0c995e0640 1 --2- 172.21.15.12:0/1593520582 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564992e30580 0x564992e36a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.179+0000 7f0c995e0640 1 -- 172.21.15.12:0/1593520582 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564992aa73f0 con 0x564992e39a80 2024-04-22T13:28:37.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.179+0000 7f0c93fff640 1 --2- 172.21.15.12:0/1593520582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564992e30ea0 0x564992e36fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:28:37.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.179+0000 7f0c98ddf640 1 --2- 172.21.15.12:0/1593520582 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564992e30580 0x564992e36a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:28:37.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.179+0000 7f0c995e0640 1 --2- 172.21.15.12:0/1593520582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564992e39a80 0x564992e374f0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c036000 tx=0x7f0c8c02c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3482f5d731a90ff5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:37.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.179+0000 7f0c73fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:37.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.179+0000 7f0c73fff640 1 -- 172.21.15.12:0/1593520582 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c051070 con 0x564992e39a80 2024-04-22T13:28:37.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.179+0000 7f0c73fff640 1 -- 172.21.15.12:0/1593520582 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f0c8c034bd0 con 0x564992e39a80 2024-04-22T13:28:37.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.179+0000 7f0c9dc3c740 1 -- 172.21.15.12:0/1593520582 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c8009c850 con 0x564992e39a80 2024-04-22T13:28:37.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.179+0000 7f0c73fff640 1 -- 172.21.15.12:0/1593520582 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0c8c039040 con 0x564992e39a80 2024-04-22T13:28:37.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.179+0000 7f0c9dc3c740 10 client.17451 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:37.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.179+0000 7f0c9dc3c740 1 -- 172.21.15.12:0/1593520582 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564992a670f0 con 0x564992e39a80 2024-04-22T13:28:37.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.179+0000 7f0c73fff640 1 -- 172.21.15.12:0/1593520582 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c8009c850 con 0x564992e39a80 2024-04-22T13:28:37.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.180+0000 7f0c73fff640 1 -- 172.21.15.12:0/1593520582 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0c8c04c070 con 0x564992e39a80 2024-04-22T13:28:37.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.180+0000 7f0c9dc3c740 10 client.17451 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:37.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.180+0000 7f0c9dc3c740 10 client.17451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:37.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.180+0000 7f0c9dc3c740 20 client.17451 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:37.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.180+0000 7f0c9dc3c740 1 --2- 172.21.15.12:0/1593520582 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x564992e49990 0x564992e4bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.180+0000 7f0c9dc3c740 4 client.17451 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:28:37.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.180+0000 7f0c9dc3c740 1 -- 172.21.15.12:0/1593520582 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564992da2c00 con 0x564992e49990 2024-04-22T13:28:37.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.180+0000 7f0c98ddf640 1 --2- 172.21.15.12:0/1593520582 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x564992e49990 0x564992e4bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.180+0000 7f0c73fff640 10 client.17451 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:37.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.180+0000 7f0c98ddf640 1 --2- 172.21.15.12:0/1593520582 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x564992e49990 0x564992e4bdd0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6e11f9c30435395 server_cookie=b7abb3317456ea66 in_seq=0 out_seq=0 2024-04-22T13:28:37.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.181+0000 7f0c73fff640 1 -- 172.21.15.12:0/1593520582 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564992da2c00 con 0x564992e49990 2024-04-22T13:28:37.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.181+0000 7f0c73fff640 10 client.17451 handle_command_reply: tid=0 2024-04-22T13:28:37.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.181+0000 7f0c9dc3c740 1 client.17451 shutdown 2024-04-22T13:28:37.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.181+0000 7f0c9dc3c740 1 -- 172.21.15.12:0/1593520582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564992e39a80 msgr2=0x564992e374f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:37.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.181+0000 7f0c9dc3c740 1 --2- 172.21.15.12:0/1593520582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564992e39a80 0x564992e374f0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c036000 tx=0x7f0c8c02c230 comp rx=0 tx=0).stop 2024-04-22T13:28:37.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.181+0000 7f0c9dc3c740 1 -- 172.21.15.12:0/1593520582 shutdown_connections 2024-04-22T13:28:37.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.181+0000 7f0c9dc3c740 1 --2- 172.21.15.12:0/1593520582 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x564992e49990 0x564992e4bdd0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.181+0000 7f0c9dc3c740 1 --2- 172.21.15.12:0/1593520582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564992e39a80 0x564992e374f0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.181+0000 7f0c9dc3c740 1 --2- 172.21.15.12:0/1593520582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564992e30ea0 0x564992e36fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.181+0000 7f0c9dc3c740 1 --2- 172.21.15.12:0/1593520582 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564992e30580 0x564992e36a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.181+0000 7f0c73fff640 0 client.17451 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:37.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.181+0000 7f0c9dc3c740 1 -- 172.21.15.12:0/1593520582 >> 172.21.15.12:0/1593520582 conn(0x564992e2acb0 msgr2=0x564992e2ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:37.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.181+0000 7f0c9dc3c740 1 -- 172.21.15.12:0/1593520582 shutdown_connections 2024-04-22T13:28:37.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.181+0000 7f0c9dc3c740 1 -- 172.21.15.12:0/1593520582 wait complete. 2024-04-22T13:28:37.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.182+0000 7f0c9dc3c740 20 client.17451 trim_cache size 0 max 16384 2024-04-22T13:28:37.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.200+0000 7f0c9dc3c740 1 Processor -- start 2024-04-22T13:28:37.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.200+0000 7f0c9dc3c740 1 -- start start 2024-04-22T13:28:37.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.200+0000 7f0c9dc3c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564992e30580 0x564992e391a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.200+0000 7f0c9dc3c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564992e30ea0 0x564992e45700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.200+0000 7f0c9dc3c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564992e39a80 0x564992e29bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.200+0000 7f0c9dc3c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0c941afcc0 con 0x564992e30ea0 2024-04-22T13:28:37.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.200+0000 7f0c9dc3c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0c941b1800 con 0x564992e30580 2024-04-22T13:28:37.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.200+0000 7f0c9dc3c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0c941b0390 con 0x564992e39a80 2024-04-22T13:28:37.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.200+0000 7f0c93fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564992e30ea0 0x564992e45700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.200+0000 7f0c93fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564992e30ea0 0x564992e45700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33370/0 (socket says 172.21.15.12:33370) 2024-04-22T13:28:37.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.200+0000 7f0c93fff640 1 -- 172.21.15.12:0/2547564946 learned_addr learned my addr 172.21.15.12:0/2547564946 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:37.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.200+0000 7f0c995e0640 1 --2- 172.21.15.12:0/2547564946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564992e39a80 0x564992e29bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.200+0000 7f0c98ddf640 1 --2- 172.21.15.12:0/2547564946 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564992e30580 0x564992e391a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.200+0000 7f0c93fff640 1 -- 172.21.15.12:0/2547564946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564992e39a80 msgr2=0x564992e29bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:37.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.200+0000 7f0c93fff640 1 --2- 172.21.15.12:0/2547564946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564992e39a80 0x564992e29bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.200+0000 7f0c93fff640 1 -- 172.21.15.12:0/2547564946 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564992e30580 msgr2=0x564992e391a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:37.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.200+0000 7f0c93fff640 1 --2- 172.21.15.12:0/2547564946 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564992e30580 0x564992e391a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.200+0000 7f0c93fff640 1 -- 172.21.15.12:0/2547564946 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564992e3be70 con 0x564992e30ea0 2024-04-22T13:28:37.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.200+0000 7f0c98ddf640 1 --2- 172.21.15.12:0/2547564946 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564992e30580 0x564992e391a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:28:37.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.200+0000 7f0c93fff640 1 --2- 172.21.15.12:0/2547564946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564992e30ea0 0x564992e45700 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f0c800985b0 tx=0x7f0c800983f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c74082fb9bc63b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:37.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.201+0000 7f0c7e7fc640 1 -- 172.21.15.12:0/2547564946 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80063020 con 0x564992e30ea0 2024-04-22T13:28:37.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.201+0000 7f0c7e7fc640 1 -- 172.21.15.12:0/2547564946 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c8009c850 con 0x564992e30ea0 2024-04-22T13:28:37.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.201+0000 7f0c7e7fc640 1 -- 172.21.15.12:0/2547564946 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80009df0 con 0x564992e30ea0 2024-04-22T13:28:37.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.201+0000 7f0c9dc3c740 1 -- 172.21.15.12:0/2547564946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564992e30ea0 msgr2=0x564992e45700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:37.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.201+0000 7f0c9dc3c740 1 --2- 172.21.15.12:0/2547564946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564992e30ea0 0x564992e45700 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f0c800985b0 tx=0x7f0c800983f0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.201+0000 7f0c9dc3c740 1 -- 172.21.15.12:0/2547564946 shutdown_connections 2024-04-22T13:28:37.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.201+0000 7f0c9dc3c740 1 --2- 172.21.15.12:0/2547564946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564992e39a80 0x564992e29bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.201+0000 7f0c9dc3c740 1 --2- 172.21.15.12:0/2547564946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564992e30ea0 0x564992e45700 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.201+0000 7f0c9dc3c740 1 --2- 172.21.15.12:0/2547564946 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564992e30580 0x564992e391a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.201+0000 7f0c9dc3c740 1 -- 172.21.15.12:0/2547564946 >> 172.21.15.12:0/2547564946 conn(0x564992e2acb0 msgr2=0x564992e294e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:37.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.201+0000 7f0c9dc3c740 1 -- 172.21.15.12:0/2547564946 shutdown_connections 2024-04-22T13:28:37.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.201+0000 7f0c9dc3c740 1 -- 172.21.15.12:0/2547564946 wait complete. 2024-04-22T13:28:37.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.202+0000 7f0c9dc3c740 1 -- start start 2024-04-22T13:28:37.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.202+0000 7f0c9dc3c740 1 Processor -- start 2024-04-22T13:28:37.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.202+0000 7f0c9dc3c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564992e30580 0x564992e3d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.202+0000 7f0c9dc3c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564992e30ea0 0x564992e3dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.202+0000 7f0c9dc3c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564992e39a80 0x564992e3e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.202+0000 7f0c9dc3c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0c941b1130 con 0x564992e39a80 2024-04-22T13:28:37.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.202+0000 7f0c9dc3c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0c941b0a60 con 0x564992e30ea0 2024-04-22T13:28:37.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.202+0000 7f0c9dc3c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0c941adca0 con 0x564992e30580 2024-04-22T13:28:37.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.202+0000 7f0c9dc3c740 10 client.0 _ll_register_callbacks cb 0x7f0c8c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:28:37.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.202+0000 7f0c995e0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564992e39a80 0x564992e3e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.202+0000 7f0c995e0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564992e39a80 0x564992e3e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33374/0 (socket says 172.21.15.12:33374) 2024-04-22T13:28:37.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.202+0000 7f0c995e0640 1 -- 172.21.15.12:0/695552312 learned_addr learned my addr 172.21.15.12:0/695552312 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:28:37.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.202+0000 7f0c98ddf640 1 --2- 172.21.15.12:0/695552312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564992e30580 0x564992e3d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.202+0000 7f0c93fff640 1 --2- 172.21.15.12:0/695552312 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564992e30ea0 0x564992e3dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.203+0000 7f0c995e0640 1 -- 172.21.15.12:0/695552312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564992e30580 msgr2=0x564992e3d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:37.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.203+0000 7f0c995e0640 1 --2- 172.21.15.12:0/695552312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564992e30580 0x564992e3d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.203+0000 7f0c995e0640 1 -- 172.21.15.12:0/695552312 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564992e30ea0 msgr2=0x564992e3dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:28:37.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.203+0000 7f0c995e0640 1 --2- 172.21.15.12:0/695552312 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564992e30ea0 0x564992e3dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.203+0000 7f0c995e0640 1 -- 172.21.15.12:0/695552312 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564992eaefc0 con 0x564992e39a80 2024-04-22T13:28:37.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.203+0000 7f0c995e0640 1 --2- 172.21.15.12:0/695552312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564992e39a80 0x564992e3e1a0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c037040 tx=0x7f0c8c043e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de4fcde1198bd33 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:28:37.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.203+0000 7f0c73fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:28:37.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.203+0000 7f0c73fff640 1 -- 172.21.15.12:0/695552312 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c051040 con 0x564992e39a80 2024-04-22T13:28:37.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.203+0000 7f0c73fff640 1 -- 172.21.15.12:0/695552312 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f0c8c032e30 con 0x564992e39a80 2024-04-22T13:28:37.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.203+0000 7f0c9dc3c740 1 -- 172.21.15.12:0/695552312 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c8009c850 con 0x564992e39a80 2024-04-22T13:28:37.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.203+0000 7f0c73fff640 1 -- 172.21.15.12:0/695552312 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0c8c070050 con 0x564992e39a80 2024-04-22T13:28:37.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.203+0000 7f0c9dc3c740 10 client.17457 fetch_fsmap learned FSMap version 28 2024-04-22T13:28:37.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.203+0000 7f0c9dc3c740 1 -- 172.21.15.12:0/695552312 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564992eb0220 con 0x564992e39a80 2024-04-22T13:28:37.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.203+0000 7f0c73fff640 1 -- 172.21.15.12:0/695552312 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c8009c850 con 0x564992e39a80 2024-04-22T13:28:37.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.203+0000 7f0c73fff640 1 -- 172.21.15.12:0/695552312 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0c8c04c040 con 0x564992e39a80 2024-04-22T13:28:37.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.203+0000 7f0c9dc3c740 10 client.17457 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:28:37.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.203+0000 7f0c9dc3c740 10 client.17457 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:28:37.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.204+0000 7f0c9dc3c740 20 client.17457 populate_metadata read hostname 'smithi012' 2024-04-22T13:28:37.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.204+0000 7f0c9dc3c740 1 --2- 172.21.15.12:0/695552312 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x564992eb0990 0x564992eb2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:28:37.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.204+0000 7f0c9dc3c740 4 client.17457 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:28:37.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.204+0000 7f0c9dc3c740 1 -- 172.21.15.12:0/695552312 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564992d9ca70 con 0x564992eb0990 2024-04-22T13:28:37.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.204+0000 7f0c98ddf640 1 --2- 172.21.15.12:0/695552312 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x564992eb0990 0x564992eb2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:28:37.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.204+0000 7f0c73fff640 10 client.17457 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:37.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.204+0000 7f0c98ddf640 1 --2- 172.21.15.12:0/695552312 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x564992eb0990 0x564992eb2db0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da21b5284d758772 server_cookie=42f9b040a15f3f71 in_seq=0 out_seq=0 2024-04-22T13:28:37.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.204+0000 7f0c73fff640 1 -- 172.21.15.12:0/695552312 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564992d9ca70 con 0x564992eb0990 2024-04-22T13:28:37.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.204+0000 7f0c73fff640 10 client.17457 handle_command_reply: tid=0 2024-04-22T13:28:37.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.204+0000 7f0c9dc3c740 1 client.17457 shutdown 2024-04-22T13:28:37.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.204+0000 7f0c9dc3c740 1 -- 172.21.15.12:0/695552312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564992e39a80 msgr2=0x564992e3e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:37.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.204+0000 7f0c9dc3c740 1 --2- 172.21.15.12:0/695552312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564992e39a80 0x564992e3e1a0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c037040 tx=0x7f0c8c043e60 comp rx=0 tx=0).stop 2024-04-22T13:28:37.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.204+0000 7f0c9dc3c740 1 -- 172.21.15.12:0/695552312 shutdown_connections 2024-04-22T13:28:37.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.204+0000 7f0c9dc3c740 1 --2- 172.21.15.12:0/695552312 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x564992eb0990 0x564992eb2db0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.204+0000 7f0c9dc3c740 1 --2- 172.21.15.12:0/695552312 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564992e39a80 0x564992e3e1a0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.204+0000 7f0c9dc3c740 1 --2- 172.21.15.12:0/695552312 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564992e30ea0 0x564992e3dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.204+0000 7f0c73fff640 0 client.17457 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:28:37.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.204+0000 7f0c9dc3c740 1 --2- 172.21.15.12:0/695552312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564992e30580 0x564992e3d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.204+0000 7f0c9dc3c740 1 -- 172.21.15.12:0/695552312 >> 172.21.15.12:0/695552312 conn(0x564992e2acb0 msgr2=0x564992e372c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:37.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.205+0000 7f0c9dc3c740 1 -- 172.21.15.12:0/695552312 shutdown_connections 2024-04-22T13:28:37.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.205+0000 7f0c9dc3c740 1 -- 172.21.15.12:0/695552312 wait complete. 2024-04-22T13:28:37.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.205+0000 7f0c9dc3c740 20 client.17457 trim_cache size 0 max 16384 2024-04-22T13:28:37.207 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:28:37.207 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:28:37.208 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-04-22T13:28:37.208 INFO:teuthology.orchestra.run.smithi012.stdout: "abe91ecc-f3ef-47be-9d41-ae8b0d64dc53": { 2024-04-22T13:28:37.208 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-04-22T13:28:37.208 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "abe91ecc-f3ef-47be-9d41-ae8b0d64dc53", 2024-04-22T13:28:37.208 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-04-22T13:28:37.208 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:28:37.208 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:28:37.208 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:28:37.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.205+0000 7f0c9b043640 1 -- 172.21.15.12:0/3558034455 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f0c840a3d20 msgr2=0x7f0c840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:37.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.205+0000 7f0c9b043640 1 --2- 172.21.15.12:0/3558034455 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f0c840a3d20 0x7f0c840a61e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c004af0 tx=0x7f0c8c02e040 comp rx=0 tx=0).stop 2024-04-22T13:28:37.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.205+0000 7f0c9b043640 1 -- 172.21.15.12:0/3558034455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c94103ae0 msgr2=0x7f0c941a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:28:37.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.205+0000 7f0c9b043640 1 --2- 172.21.15.12:0/3558034455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c94103ae0 0x7f0c941a0bb0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000d950 tx=0x7f0c8000de10 comp rx=0 tx=0).stop 2024-04-22T13:28:37.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.205+0000 7f0c9b043640 1 -- 172.21.15.12:0/3558034455 shutdown_connections 2024-04-22T13:28:37.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.205+0000 7f0c9b043640 1 --2- 172.21.15.12:0/3558034455 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f0c840a3d20 0x7f0c840a61e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.205+0000 7f0c9b043640 1 --2- 172.21.15.12:0/3558034455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c9410c500 0x7f0c9419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.205+0000 7f0c9b043640 1 --2- 172.21.15.12:0/3558034455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0c94103ae0 0x7f0c941a0bb0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.205+0000 7f0c9b043640 1 --2- 172.21.15.12:0/3558034455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c941031c0 0x7f0c941a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:28:37.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.205+0000 7f0c9b043640 1 -- 172.21.15.12:0/3558034455 >> 172.21.15.12:0/3558034455 conn(0x7f0c94078000 msgr2=0x7f0c94108470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:28:37.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.205+0000 7f0c9b043640 1 -- 172.21.15.12:0/3558034455 shutdown_connections 2024-04-22T13:28:37.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:28:37.205+0000 7f0c9b043640 1 -- 172.21.15.12:0/3558034455 wait complete. 2024-04-22T13:28:37.212 INFO:tasks.cephfs.filesystem:scrub status for tag:abe91ecc-f3ef-47be-9d41-ae8b0d64dc53 - {'path': '/', 'tag': 'abe91ecc-f3ef-47be-9d41-ae8b0d64dc53', 'options': 'recursive,force'} 2024-04-22T13:28:37.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:37 smithi134 ceph-mon[31338]: pgmap v138: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:37.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:37 smithi111 ceph-mon[31361]: pgmap v138: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:38.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:37 smithi012 ceph-mon[28265]: pgmap v138: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:38.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:38 smithi134 ceph-mon[31338]: scrub queued for path: / 2024-04-22T13:28:38.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:38 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:28:38.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:38 smithi134 ceph-mon[31338]: scrub summary: active paths [/] 2024-04-22T13:28:38.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:38 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:28:38.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:38 smithi111 ceph-mon[31361]: scrub queued for path: / 2024-04-22T13:28:38.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:38 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:28:38.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:38 smithi111 ceph-mon[31361]: scrub summary: active paths [/] 2024-04-22T13:28:38.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:38 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:28:39.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:38 smithi012 ceph-mon[28265]: scrub queued for path: / 2024-04-22T13:28:39.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:38 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:28:39.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:38 smithi012 ceph-mon[28265]: scrub summary: active paths [/] 2024-04-22T13:28:39.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:38 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:28:39.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:39 smithi134 ceph-mon[31338]: pgmap v139: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 14 op/s 2024-04-22T13:28:39.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:39 smithi111 ceph-mon[31361]: pgmap v139: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 14 op/s 2024-04-22T13:28:40.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:39 smithi012 ceph-mon[28265]: pgmap v139: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 14 op/s 2024-04-22T13:28:40.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:40 smithi134 ceph-mon[31338]: scrub summary: idle 2024-04-22T13:28:40.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:40 smithi111 ceph-mon[31361]: scrub summary: idle 2024-04-22T13:28:41.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:40 smithi012 ceph-mon[28265]: scrub summary: idle 2024-04-22T13:28:41.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:41 smithi134 ceph-mon[31338]: pgmap v140: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 10 op/s 2024-04-22T13:28:41.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:41 smithi111 ceph-mon[31361]: pgmap v140: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 10 op/s 2024-04-22T13:28:42.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:41 smithi012 ceph-mon[28265]: pgmap v140: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 10 op/s 2024-04-22T13:28:43.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:43 smithi134 ceph-mon[31338]: pgmap v141: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 14 op/s 2024-04-22T13:28:43.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:43 smithi111 ceph-mon[31361]: pgmap v141: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 14 op/s 2024-04-22T13:28:44.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:43 smithi012 ceph-mon[28265]: pgmap v141: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 14 op/s 2024-04-22T13:28:45.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:45 smithi134 ceph-mon[31338]: pgmap v142: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 10 op/s 2024-04-22T13:28:45.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:45 smithi111 ceph-mon[31361]: pgmap v142: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 10 op/s 2024-04-22T13:28:46.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:45 smithi012 ceph-mon[28265]: pgmap v142: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 10 op/s 2024-04-22T13:28:47.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:47 smithi134 ceph-mon[31338]: pgmap v143: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 10 op/s 2024-04-22T13:28:47.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:47 smithi111 ceph-mon[31361]: pgmap v143: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 10 op/s 2024-04-22T13:28:48.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:47 smithi012 ceph-mon[28265]: pgmap v143: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 10 op/s 2024-04-22T13:28:49.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:49 smithi134 ceph-mon[31338]: pgmap v144: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 14 op/s 2024-04-22T13:28:49.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:49 smithi111 ceph-mon[31361]: pgmap v144: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 14 op/s 2024-04-22T13:28:50.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:49 smithi012 ceph-mon[28265]: pgmap v144: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 14 op/s 2024-04-22T13:28:51.889 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:51 smithi012 ceph-mon[28265]: pgmap v145: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:51.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:51 smithi134 ceph-mon[31338]: pgmap v145: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:51.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:51 smithi111 ceph-mon[31361]: pgmap v145: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:53.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:53 smithi134 ceph-mon[31338]: pgmap v146: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:28:53.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:53 smithi111 ceph-mon[31361]: pgmap v146: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:28:54.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:53 smithi012 ceph-mon[28265]: pgmap v146: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:28:55.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:55 smithi134 ceph-mon[31338]: pgmap v147: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:55.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:55 smithi111 ceph-mon[31361]: pgmap v147: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:56.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:55 smithi012 ceph-mon[28265]: pgmap v147: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:57.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:57 smithi134 ceph-mon[31338]: pgmap v148: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:57.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:57 smithi111 ceph-mon[31361]: pgmap v148: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:58.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:57 smithi012 ceph-mon[28265]: pgmap v148: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:28:59.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:28:59 smithi134 ceph-mon[31338]: pgmap v149: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:28:59.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:28:59 smithi111 ceph-mon[31361]: pgmap v149: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:29:00.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:28:59 smithi012 ceph-mon[28265]: pgmap v149: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:29:01.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:01 smithi134 ceph-mon[31338]: pgmap v150: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:29:01.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:01 smithi111 ceph-mon[31361]: pgmap v150: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:29:02.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:01 smithi012 ceph-mon[28265]: pgmap v150: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:29:03.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:03 smithi134 ceph-mon[31338]: pgmap v151: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-22T13:29:03.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:03 smithi111 ceph-mon[31361]: pgmap v151: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-22T13:29:04.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:03 smithi012 ceph-mon[28265]: pgmap v151: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-22T13:29:05.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:05 smithi134 ceph-mon[31338]: pgmap v152: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:29:05.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:05 smithi111 ceph-mon[31361]: pgmap v152: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:29:06.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:05 smithi012 ceph-mon[28265]: pgmap v152: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:29:07.213 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:29:07.213 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:29:07.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.335+0000 7f9c00e1f640 1 -- 172.21.15.12:0/847077932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bfc06a5b0 msgr2=0x7f9bfc06aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.335+0000 7f9c00e1f640 1 --2- 172.21.15.12:0/847077932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bfc06a5b0 0x7f9bfc06aa30 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009930 tx=0x7f9bf002bc30 comp rx=0 tx=0).stop 2024-04-22T13:29:07.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.335+0000 7f9c00e1f640 1 -- 172.21.15.12:0/847077932 shutdown_connections 2024-04-22T13:29:07.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.335+0000 7f9c00e1f640 1 --2- 172.21.15.12:0/847077932 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bfc06af70 0x7f9bfc10ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.335+0000 7f9c00e1f640 1 --2- 172.21.15.12:0/847077932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bfc06a5b0 0x7f9bfc06aa30 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.335+0000 7f9c00e1f640 1 --2- 172.21.15.12:0/847077932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bfc108060 0x7f9bfc108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.335+0000 7f9c00e1f640 1 -- 172.21.15.12:0/847077932 >> 172.21.15.12:0/847077932 conn(0x7f9bfc077c40 msgr2=0x7f9bfc078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:07.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.335+0000 7f9c00e1f640 1 -- 172.21.15.12:0/847077932 shutdown_connections 2024-04-22T13:29:07.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.335+0000 7f9c00e1f640 1 -- 172.21.15.12:0/847077932 wait complete. 2024-04-22T13:29:07.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.336+0000 7f9c00e1f640 1 Processor -- start 2024-04-22T13:29:07.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.336+0000 7f9c00e1f640 1 -- start start 2024-04-22T13:29:07.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.336+0000 7f9c00e1f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bfc06a5b0 0x7f9bfc198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.336+0000 7f9c00e1f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bfc06af70 0x7f9bfc198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.336+0000 7f9c00e1f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bfc108060 0x7f9bfc19d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.336+0000 7f9c00e1f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc112180 con 0x7f9bfc108060 2024-04-22T13:29:07.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.336+0000 7f9c00e1f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc112000 con 0x7f9bfc06a5b0 2024-04-22T13:29:07.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.336+0000 7f9c00e1f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc112300 con 0x7f9bfc06af70 2024-04-22T13:29:07.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.337+0000 7f9bfad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bfc108060 0x7f9bfc19d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.337+0000 7f9bfad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bfc108060 0x7f9bfc19d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51796/0 (socket says 172.21.15.12:51796) 2024-04-22T13:29:07.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.337+0000 7f9bfad9d640 1 -- 172.21.15.12:0/2051138594 learned_addr learned my addr 172.21.15.12:0/2051138594 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:07.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.337+0000 7f9bf9d9b640 1 --2- 172.21.15.12:0/2051138594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bfc06af70 0x7f9bfc198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.337+0000 7f9bfa59c640 1 --2- 172.21.15.12:0/2051138594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bfc06a5b0 0x7f9bfc198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.337+0000 7f9bfad9d640 1 -- 172.21.15.12:0/2051138594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bfc06af70 msgr2=0x7f9bfc198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.337+0000 7f9bfad9d640 1 --2- 172.21.15.12:0/2051138594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bfc06af70 0x7f9bfc198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.337+0000 7f9bfad9d640 1 -- 172.21.15.12:0/2051138594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bfc06a5b0 msgr2=0x7f9bfc198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.337+0000 7f9bfad9d640 1 --2- 172.21.15.12:0/2051138594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bfc06a5b0 0x7f9bfc198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.337+0000 7f9bfad9d640 1 -- 172.21.15.12:0/2051138594 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bfc19d730 con 0x7f9bfc108060 2024-04-22T13:29:07.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.337+0000 7f9bfad9d640 1 --2- 172.21.15.12:0/2051138594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bfc108060 0x7f9bfc19d000 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009fd0 tx=0x7f9bf400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:07.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.337+0000 7f9beb7fe640 1 -- 172.21.15.12:0/2051138594 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4017070 con 0x7f9bfc108060 2024-04-22T13:29:07.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.337+0000 7f9beb7fe640 1 -- 172.21.15.12:0/2051138594 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bf4003e20 con 0x7f9bfc108060 2024-04-22T13:29:07.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.337+0000 7f9beb7fe640 1 -- 172.21.15.12:0/2051138594 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4010460 con 0x7f9bfc108060 2024-04-22T13:29:07.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.337+0000 7f9c00e1f640 1 -- 172.21.15.12:0/2051138594 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bfc19da20 con 0x7f9bfc108060 2024-04-22T13:29:07.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.337+0000 7f9c00e1f640 1 -- 172.21.15.12:0/2051138594 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bfc1a52b0 con 0x7f9bfc108060 2024-04-22T13:29:07.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.340+0000 7f9beb7fe640 1 -- 172.21.15.12:0/2051138594 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9bf4003970 con 0x7f9bfc108060 2024-04-22T13:29:07.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.340+0000 7f9c03a0d740 1 Processor -- start 2024-04-22T13:29:07.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.340+0000 7f9c03a0d740 1 -- start start 2024-04-22T13:29:07.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.340+0000 7f9c03a0d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0642bc580 0x55e0642bc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.340+0000 7f9c03a0d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0642bcea0 0x55e0642c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.340+0000 7f9c03a0d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0642c5a80 0x55e0642c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.340+0000 7f9c03a0d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e0642211e0 con 0x55e0642bcea0 2024-04-22T13:29:07.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.340+0000 7f9c03a0d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e0641be390 con 0x55e0642c5a80 2024-04-22T13:29:07.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.340+0000 7f9c03a0d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e06421d1a0 con 0x55e0642bc580 2024-04-22T13:29:07.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.340+0000 7f9beb7fe640 1 --2- 172.21.15.12:0/2051138594 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f9be40a3da0 0x7f9be40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.341+0000 7f9bfa59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0642bcea0 0x55e0642c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.341+0000 7f9bfa59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0642bcea0 0x55e0642c5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51802/0 (socket says 172.21.15.12:51802) 2024-04-22T13:29:07.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.341+0000 7f9bfa59c640 1 -- 172.21.15.12:0/2295836660 learned_addr learned my addr 172.21.15.12:0/2295836660 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:07.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.341+0000 7f9bf9d9b640 1 --2- 172.21.15.12:0/2295836660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0642bc580 0x55e0642bc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.341+0000 7f9bfad9d640 1 --2- 172.21.15.12:0/2051138594 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f9be40a3da0 0x7f9be40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.341+0000 7f9bf9d9b640 1 --2- 172.21.15.12:0/2295836660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0642c5a80 0x55e0642c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.341+0000 7f9beb7fe640 1 -- 172.21.15.12:0/2051138594 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f9bf40d04f0 con 0x7f9bfc108060 2024-04-22T13:29:07.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.341+0000 7f9bfa59c640 1 -- 172.21.15.12:0/2295836660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0642bc580 msgr2=0x55e0642bc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:07.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.341+0000 7f9bfa59c640 1 --2- 172.21.15.12:0/2295836660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0642bc580 0x55e0642bc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.341+0000 7f9bfa59c640 1 -- 172.21.15.12:0/2295836660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0642c5a80 msgr2=0x55e0642c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:07.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.341+0000 7f9bfa59c640 1 --2- 172.21.15.12:0/2295836660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0642c5a80 0x55e0642c7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.341+0000 7f9bfa59c640 1 -- 172.21.15.12:0/2295836660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e063ffd7d0 con 0x55e0642bcea0 2024-04-22T13:29:07.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.341+0000 7f9bfa59c640 1 --2- 172.21.15.12:0/2295836660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0642bcea0 0x55e0642c5540 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00bc60 tx=0x7f9bec00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0f6ea62c1643a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:07.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.341+0000 7f9bfad9d640 1 --2- 172.21.15.12:0/2051138594 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f9be40a3da0 0x7f9be40a6260 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40ce290 tx=0x7f9bf406ae90 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:07.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.341+0000 7f9be8ff9640 1 -- 172.21.15.12:0/2295836660 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec015070 con 0x55e0642bcea0 2024-04-22T13:29:07.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.341+0000 7f9be8ff9640 1 -- 172.21.15.12:0/2295836660 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bec01a910 con 0x55e0642bcea0 2024-04-22T13:29:07.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.341+0000 7f9be8ff9640 1 -- 172.21.15.12:0/2295836660 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec011820 con 0x55e0642bcea0 2024-04-22T13:29:07.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.342+0000 7f9c03a0d740 1 -- 172.21.15.12:0/2295836660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0642bcea0 msgr2=0x55e0642c5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.342+0000 7f9c03a0d740 1 --2- 172.21.15.12:0/2295836660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0642bcea0 0x55e0642c5540 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00bc60 tx=0x7f9bec00f730 comp rx=0 tx=0).stop 2024-04-22T13:29:07.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.342+0000 7f9c03a0d740 1 -- 172.21.15.12:0/2295836660 shutdown_connections 2024-04-22T13:29:07.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.342+0000 7f9c03a0d740 1 --2- 172.21.15.12:0/2295836660 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0642c5a80 0x55e0642c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.342+0000 7f9c03a0d740 1 --2- 172.21.15.12:0/2295836660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0642bcea0 0x55e0642c5540 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.342+0000 7f9c03a0d740 1 --2- 172.21.15.12:0/2295836660 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0642bc580 0x55e0642bc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.342+0000 7f9c03a0d740 1 -- 172.21.15.12:0/2295836660 >> 172.21.15.12:0/2295836660 conn(0x55e0642b6cb0 msgr2=0x55e0642b7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:07.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.342+0000 7f9c03a0d740 1 -- 172.21.15.12:0/2295836660 shutdown_connections 2024-04-22T13:29:07.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.342+0000 7f9c03a0d740 1 -- 172.21.15.12:0/2295836660 wait complete. 2024-04-22T13:29:07.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.343+0000 7f9c03a0d740 1 -- start start 2024-04-22T13:29:07.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.343+0000 7f9c03a0d740 1 Processor -- start 2024-04-22T13:29:07.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.343+0000 7f9c03a0d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0642bc580 0x55e0642c2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.343+0000 7f9c03a0d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0642bcea0 0x55e0642c3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.343+0000 7f9c03a0d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0642c5a80 0x55e0642c3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.343+0000 7f9c03a0d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55e0642c8870 con 0x55e0642c5a80 2024-04-22T13:29:07.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.343+0000 7f9c03a0d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e0641da2f0 con 0x55e0642bcea0 2024-04-22T13:29:07.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.343+0000 7f9c03a0d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e0642c89f0 con 0x55e0642bc580 2024-04-22T13:29:07.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.343+0000 7f9c03a0d740 10 client.0 _ll_register_callbacks cb 0x55e0641e7580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:07.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.343+0000 7f9bf9d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0642c5a80 0x55e0642c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.343+0000 7f9bf9d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0642c5a80 0x55e0642c3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51810/0 (socket says 172.21.15.12:51810) 2024-04-22T13:29:07.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.343+0000 7f9bf9d9b640 1 -- 172.21.15.12:0/1450106181 learned_addr learned my addr 172.21.15.12:0/1450106181 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:07.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.343+0000 7f9bf9d9b640 1 --2- 172.21.15.12:0/1450106181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0642bc580 0x55e0642c2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.343+0000 7f9bfa59c640 1 --2- 172.21.15.12:0/1450106181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0642bcea0 0x55e0642c3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.343+0000 7f9bf9d9b640 1 -- 172.21.15.12:0/1450106181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0642bc580 msgr2=0x55e0642c2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:07.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.343+0000 7f9bf9d9b640 1 --2- 172.21.15.12:0/1450106181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0642bc580 0x55e0642c2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.343+0000 7f9bf9d9b640 1 -- 172.21.15.12:0/1450106181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0642bcea0 msgr2=0x55e0642c3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:07.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.343+0000 7f9bf9d9b640 1 --2- 172.21.15.12:0/1450106181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0642bcea0 0x55e0642c3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.343+0000 7f9bf9d9b640 1 -- 172.21.15.12:0/1450106181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e063f333f0 con 0x55e0642c5a80 2024-04-22T13:29:07.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.343+0000 7f9bfa59c640 1 --2- 172.21.15.12:0/1450106181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0642bcea0 0x55e0642c3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:07.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.344+0000 7f9bf9d9b640 1 --2- 172.21.15.12:0/1450106181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0642c5a80 0x55e0642c3550 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0035000 tx=0x7f9bf0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1efc9ecc3a48ee0d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:07.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.344+0000 7f9be1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:07.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.344+0000 7f9be1ffb640 1 -- 172.21.15.12:0/1450106181 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf00055c0 con 0x55e0642c5a80 2024-04-22T13:29:07.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.344+0000 7f9be1ffb640 1 -- 172.21.15.12:0/1450106181 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f9bf003a040 con 0x55e0642c5a80 2024-04-22T13:29:07.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.344+0000 7f9c03a0d740 1 -- 172.21.15.12:0/1450106181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bec01a910 con 0x55e0642c5a80 2024-04-22T13:29:07.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.344+0000 7f9be1ffb640 1 -- 172.21.15.12:0/1450106181 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9bf0056050 con 0x55e0642c5a80 2024-04-22T13:29:07.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.344+0000 7f9c03a0d740 10 client.17469 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:07.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.344+0000 7f9c03a0d740 1 -- 172.21.15.12:0/1450106181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e063ef30f0 con 0x55e0642c5a80 2024-04-22T13:29:07.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.344+0000 7f9be1ffb640 1 -- 172.21.15.12:0/1450106181 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bec01a910 con 0x55e0642c5a80 2024-04-22T13:29:07.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.344+0000 7f9be1ffb640 1 -- 172.21.15.12:0/1450106181 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9bf00037b0 con 0x55e0642c5a80 2024-04-22T13:29:07.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.344+0000 7f9c03a0d740 10 client.17469 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:07.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.344+0000 7f9c03a0d740 10 client.17469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:07.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.344+0000 7f9c03a0d740 20 client.17469 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:07.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.344+0000 7f9c03a0d740 1 --2- 172.21.15.12:0/1450106181 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e0642d59e0 0x55e0642d7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.344+0000 7f9c03a0d740 4 client.17469 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:07.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.344+0000 7f9c03a0d740 1 -- 172.21.15.12:0/1450106181 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e06422ec00 con 0x55e0642d59e0 2024-04-22T13:29:07.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.345+0000 7f9bfa59c640 1 --2- 172.21.15.12:0/1450106181 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e0642d59e0 0x55e0642d7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.345+0000 7f9bfa59c640 1 --2- 172.21.15.12:0/1450106181 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e0642d59e0 0x55e0642d7e80 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44bfd38388b0bba1 server_cookie=ea37f5f4d70cf88f in_seq=0 out_seq=0 2024-04-22T13:29:07.349 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.345+0000 7f9be1ffb640 10 client.17469 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:07.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.346+0000 7f9be1ffb640 1 -- 172.21.15.12:0/1450106181 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e06422ec00 con 0x55e0642d59e0 2024-04-22T13:29:07.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.346+0000 7f9be1ffb640 10 client.17469 handle_command_reply: tid=0 2024-04-22T13:29:07.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.346+0000 7f9c03a0d740 1 client.17469 shutdown 2024-04-22T13:29:07.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.346+0000 7f9c03a0d740 1 -- 172.21.15.12:0/1450106181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0642c5a80 msgr2=0x55e0642c3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.346+0000 7f9c03a0d740 1 --2- 172.21.15.12:0/1450106181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0642c5a80 0x55e0642c3550 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0035000 tx=0x7f9bf0009260 comp rx=0 tx=0).stop 2024-04-22T13:29:07.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.346+0000 7f9c03a0d740 1 -- 172.21.15.12:0/1450106181 shutdown_connections 2024-04-22T13:29:07.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.346+0000 7f9c03a0d740 1 --2- 172.21.15.12:0/1450106181 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e0642d59e0 0x55e0642d7e80 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.346+0000 7f9c03a0d740 1 --2- 172.21.15.12:0/1450106181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0642c5a80 0x55e0642c3550 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.346+0000 7f9c03a0d740 1 --2- 172.21.15.12:0/1450106181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0642bcea0 0x55e0642c3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.346+0000 7f9be1ffb640 0 client.17469 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:07.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.346+0000 7f9c03a0d740 1 --2- 172.21.15.12:0/1450106181 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0642bc580 0x55e0642c2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.346+0000 7f9c03a0d740 1 -- 172.21.15.12:0/1450106181 >> 172.21.15.12:0/1450106181 conn(0x55e0642b6cb0 msgr2=0x55e0642b8a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:07.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.346+0000 7f9c03a0d740 1 -- 172.21.15.12:0/1450106181 shutdown_connections 2024-04-22T13:29:07.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.346+0000 7f9c03a0d740 1 -- 172.21.15.12:0/1450106181 wait complete. 2024-04-22T13:29:07.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.346+0000 7f9c03a0d740 20 client.17469 trim_cache size 0 max 16384 2024-04-22T13:29:07.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.364+0000 7f9c03a0d740 1 Processor -- start 2024-04-22T13:29:07.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.364+0000 7f9c03a0d740 1 -- start start 2024-04-22T13:29:07.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.364+0000 7f9c03a0d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0642bc580 0x55e0642c5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.364+0000 7f9c03a0d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0642bcea0 0x55e0642d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.364+0000 7f9c03a0d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0642c5a80 0x55e0642c1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.364+0000 7f9c03a0d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc1adac0 con 0x55e0642bcea0 2024-04-22T13:29:07.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.364+0000 7f9c03a0d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc1af600 con 0x55e0642c5a80 2024-04-22T13:29:07.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.364+0000 7f9c03a0d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc1ae190 con 0x55e0642bc580 2024-04-22T13:29:07.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.364+0000 7f9bfa59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0642bcea0 0x55e0642d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.364+0000 7f9bfa59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0642bcea0 0x55e0642d1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51822/0 (socket says 172.21.15.12:51822) 2024-04-22T13:29:07.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.364+0000 7f9bfa59c640 1 -- 172.21.15.12:0/1016042073 learned_addr learned my addr 172.21.15.12:0/1016042073 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:07.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.364+0000 7f9bf9d9b640 1 --2- 172.21.15.12:0/1016042073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0642bc580 0x55e0642c5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.365+0000 7f9bf9d9b640 1 --2- 172.21.15.12:0/1016042073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0642c5a80 0x55e0642c1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.365+0000 7f9bfa59c640 1 -- 172.21.15.12:0/1016042073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0642bc580 msgr2=0x55e0642c5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:07.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.365+0000 7f9bfa59c640 1 --2- 172.21.15.12:0/1016042073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0642bc580 0x55e0642c5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.365+0000 7f9bfa59c640 1 -- 172.21.15.12:0/1016042073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0642c5a80 msgr2=0x55e0642c1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:07.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.365+0000 7f9bfa59c640 1 --2- 172.21.15.12:0/1016042073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0642c5a80 0x55e0642c1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.365+0000 7f9bfa59c640 1 -- 172.21.15.12:0/1016042073 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0642bccd0 con 0x55e0642bcea0 2024-04-22T13:29:07.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.365+0000 7f9bf9d9b640 1 --2- 172.21.15.12:0/1016042073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0642c5a80 0x55e0642c1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:07.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.365+0000 7f9bfa59c640 1 --2- 172.21.15.12:0/1016042073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0642bcea0 0x55e0642d1730 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00fc30 tx=0x7f9bec005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2b940c255daf50 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:07.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.365+0000 7f9be3fff640 1 -- 172.21.15.12:0/1016042073 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec015050 con 0x55e0642bcea0 2024-04-22T13:29:07.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.365+0000 7f9be3fff640 1 -- 172.21.15.12:0/1016042073 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bec0054f0 con 0x55e0642bcea0 2024-04-22T13:29:07.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.365+0000 7f9be3fff640 1 -- 172.21.15.12:0/1016042073 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec021040 con 0x55e0642bcea0 2024-04-22T13:29:07.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.365+0000 7f9c03a0d740 1 -- 172.21.15.12:0/1016042073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0642bcea0 msgr2=0x55e0642d1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.365+0000 7f9c03a0d740 1 --2- 172.21.15.12:0/1016042073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0642bcea0 0x55e0642d1730 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00fc30 tx=0x7f9bec005f30 comp rx=0 tx=0).stop 2024-04-22T13:29:07.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.366+0000 7f9c03a0d740 1 -- 172.21.15.12:0/1016042073 shutdown_connections 2024-04-22T13:29:07.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.366+0000 7f9c03a0d740 1 --2- 172.21.15.12:0/1016042073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0642c5a80 0x55e0642c1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.366+0000 7f9c03a0d740 1 --2- 172.21.15.12:0/1016042073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0642bcea0 0x55e0642d1730 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.366+0000 7f9c03a0d740 1 --2- 172.21.15.12:0/1016042073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0642bc580 0x55e0642c5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.366+0000 7f9c03a0d740 1 -- 172.21.15.12:0/1016042073 >> 172.21.15.12:0/1016042073 conn(0x55e0642b6cb0 msgr2=0x55e0642b54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:07.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.366+0000 7f9c03a0d740 1 -- 172.21.15.12:0/1016042073 shutdown_connections 2024-04-22T13:29:07.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.366+0000 7f9c03a0d740 1 -- 172.21.15.12:0/1016042073 wait complete. 2024-04-22T13:29:07.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.366+0000 7f9c03a0d740 1 -- start start 2024-04-22T13:29:07.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.366+0000 7f9c03a0d740 1 Processor -- start 2024-04-22T13:29:07.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.366+0000 7f9c03a0d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0642bc580 0x55e0642bf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.366+0000 7f9c03a0d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0642bcea0 0x55e0642bfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.366+0000 7f9c03a0d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0642c5a80 0x55e0642c0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.366+0000 7f9c03a0d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc1aef30 con 0x55e0642bcea0 2024-04-22T13:29:07.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.366+0000 7f9c03a0d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc1ae860 con 0x55e0642c5a80 2024-04-22T13:29:07.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.366+0000 7f9c03a0d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc1abaa0 con 0x55e0642bc580 2024-04-22T13:29:07.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.366+0000 7f9c03a0d740 10 client.0 _ll_register_callbacks cb 0x7f9bf00043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:07.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.366+0000 7f9bfa59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0642bcea0 0x55e0642bfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.366+0000 7f9bfa59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0642bcea0 0x55e0642bfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51824/0 (socket says 172.21.15.12:51824) 2024-04-22T13:29:07.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.366+0000 7f9bfa59c640 1 -- 172.21.15.12:0/2431867554 learned_addr learned my addr 172.21.15.12:0/2431867554 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:07.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.367+0000 7f9bf9d9b640 1 --2- 172.21.15.12:0/2431867554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0642bc580 0x55e0642bf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.367+0000 7f9bfa59c640 1 -- 172.21.15.12:0/2431867554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0642bc580 msgr2=0x55e0642bf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:07.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.367+0000 7f9bfa59c640 1 --2- 172.21.15.12:0/2431867554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0642bc580 0x55e0642bf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.367+0000 7f9bfa59c640 1 -- 172.21.15.12:0/2431867554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0642c5a80 msgr2=0x55e0642c0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:07.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.367+0000 7f9bfa59c640 1 --2- 172.21.15.12:0/2431867554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0642c5a80 0x55e0642c0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.367+0000 7f9bfa59c640 1 -- 172.21.15.12:0/2431867554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e06433b0f0 con 0x55e0642bcea0 2024-04-22T13:29:07.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.367+0000 7f9bf9d9b640 1 --2- 172.21.15.12:0/2431867554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0642bc580 0x55e0642bf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:07.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.367+0000 7f9bfa59c640 1 --2- 172.21.15.12:0/2431867554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0642bcea0 0x55e0642bfdc0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00bc30 tx=0x7f9bec005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca4289ea83e2e87 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:07.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.367+0000 7f9be1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:07.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.367+0000 7f9be1ffb640 1 -- 172.21.15.12:0/2431867554 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec021070 con 0x55e0642bcea0 2024-04-22T13:29:07.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.367+0000 7f9be1ffb640 1 -- 172.21.15.12:0/2431867554 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f9bec005040 con 0x55e0642bcea0 2024-04-22T13:29:07.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.367+0000 7f9c03a0d740 1 -- 172.21.15.12:0/2431867554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bec0054f0 con 0x55e0642bcea0 2024-04-22T13:29:07.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.367+0000 7f9be1ffb640 1 -- 172.21.15.12:0/2431867554 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9bec005240 con 0x55e0642bcea0 2024-04-22T13:29:07.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.367+0000 7f9c03a0d740 10 client.17475 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:07.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.367+0000 7f9c03a0d740 1 -- 172.21.15.12:0/2431867554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e06433c350 con 0x55e0642bcea0 2024-04-22T13:29:07.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.367+0000 7f9be1ffb640 1 -- 172.21.15.12:0/2431867554 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bec0054f0 con 0x55e0642bcea0 2024-04-22T13:29:07.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.368+0000 7f9be1ffb640 1 -- 172.21.15.12:0/2431867554 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9bec024030 con 0x55e0642bcea0 2024-04-22T13:29:07.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.368+0000 7f9c03a0d740 10 client.17475 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:07.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.368+0000 7f9c03a0d740 10 client.17475 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:07.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.368+0000 7f9c03a0d740 20 client.17475 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:07.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.368+0000 7f9c03a0d740 1 --2- 172.21.15.12:0/2431867554 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e0642d59e0 0x55e06433e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.368+0000 7f9c03a0d740 4 client.17475 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:29:07.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.368+0000 7f9c03a0d740 1 -- 172.21.15.12:0/2431867554 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e064228a70 con 0x55e0642d59e0 2024-04-22T13:29:07.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.368+0000 7f9bf9d9b640 1 --2- 172.21.15.12:0/2431867554 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e0642d59e0 0x55e06433e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.368+0000 7f9bf9d9b640 1 --2- 172.21.15.12:0/2431867554 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e0642d59e0 0x55e06433e830 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eb1e2db06825780 server_cookie=296389af663b48fc in_seq=0 out_seq=0 2024-04-22T13:29:07.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.368+0000 7f9be1ffb640 10 client.17475 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:07.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.368+0000 7f9be1ffb640 1 -- 172.21.15.12:0/2431867554 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e064228a70 con 0x55e0642d59e0 2024-04-22T13:29:07.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.368+0000 7f9be1ffb640 10 client.17475 handle_command_reply: tid=0 2024-04-22T13:29:07.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.368+0000 7f9c03a0d740 1 client.17475 shutdown 2024-04-22T13:29:07.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.368+0000 7f9c03a0d740 1 -- 172.21.15.12:0/2431867554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0642bcea0 msgr2=0x55e0642bfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.368+0000 7f9c03a0d740 1 --2- 172.21.15.12:0/2431867554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0642bcea0 0x55e0642bfdc0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f9bec00bc30 tx=0x7f9bec005730 comp rx=0 tx=0).stop 2024-04-22T13:29:07.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.369+0000 7f9c03a0d740 1 -- 172.21.15.12:0/2431867554 shutdown_connections 2024-04-22T13:29:07.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.369+0000 7f9c03a0d740 1 --2- 172.21.15.12:0/2431867554 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55e0642d59e0 0x55e06433e830 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.369+0000 7f9c03a0d740 1 --2- 172.21.15.12:0/2431867554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e0642c5a80 0x55e0642c0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.369+0000 7f9c03a0d740 1 --2- 172.21.15.12:0/2431867554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e0642bcea0 0x55e0642bfdc0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.369+0000 7f9be1ffb640 0 client.17475 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:07.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.369+0000 7f9c03a0d740 1 --2- 172.21.15.12:0/2431867554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e0642bc580 0x55e0642bf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.369+0000 7f9c03a0d740 1 -- 172.21.15.12:0/2431867554 >> 172.21.15.12:0/2431867554 conn(0x55e0642b6cb0 msgr2=0x55e0642b54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:07.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.369+0000 7f9c03a0d740 1 -- 172.21.15.12:0/2431867554 shutdown_connections 2024-04-22T13:29:07.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.369+0000 7f9c03a0d740 1 -- 172.21.15.12:0/2431867554 wait complete. 2024-04-22T13:29:07.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.369+0000 7f9c03a0d740 20 client.17475 trim_cache size 0 max 16384 2024-04-22T13:29:07.372 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:29:07.372 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-04-22T13:29:07.372 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-04-22T13:29:07.373 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:29:07.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.369+0000 7f9c00e1f640 1 -- 172.21.15.12:0/2051138594 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f9be40a3da0 msgr2=0x7f9be40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.369+0000 7f9c00e1f640 1 --2- 172.21.15.12:0/2051138594 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f9be40a3da0 0x7f9be40a6260 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40ce290 tx=0x7f9bf406ae90 comp rx=0 tx=0).stop 2024-04-22T13:29:07.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.369+0000 7f9c00e1f640 1 -- 172.21.15.12:0/2051138594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bfc108060 msgr2=0x7f9bfc19d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.369+0000 7f9c00e1f640 1 --2- 172.21.15.12:0/2051138594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bfc108060 0x7f9bfc19d000 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009fd0 tx=0x7f9bf400b260 comp rx=0 tx=0).stop 2024-04-22T13:29:07.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.369+0000 7f9c00e1f640 1 -- 172.21.15.12:0/2051138594 shutdown_connections 2024-04-22T13:29:07.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.369+0000 7f9c00e1f640 1 --2- 172.21.15.12:0/2051138594 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f9be40a3da0 0x7f9be40a6260 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.369+0000 7f9c00e1f640 1 --2- 172.21.15.12:0/2051138594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9bfc108060 0x7f9bfc19d000 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.369+0000 7f9c00e1f640 1 --2- 172.21.15.12:0/2051138594 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bfc06af70 0x7f9bfc198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.369+0000 7f9c00e1f640 1 --2- 172.21.15.12:0/2051138594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bfc06a5b0 0x7f9bfc198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.369+0000 7f9c00e1f640 1 -- 172.21.15.12:0/2051138594 >> 172.21.15.12:0/2051138594 conn(0x7f9bfc077c40 msgr2=0x7f9bfc0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:07.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.369+0000 7f9c00e1f640 1 -- 172.21.15.12:0/2051138594 shutdown_connections 2024-04-22T13:29:07.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.369+0000 7f9c00e1f640 1 -- 172.21.15.12:0/2051138594 wait complete. 2024-04-22T13:29:07.376 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:29:07.376 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:29:07.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.483+0000 7fe1956c1640 1 -- 172.21.15.12:0/2446501473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe190105e30 msgr2=0x7fe190106210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.483+0000 7fe1956c1640 1 --2- 172.21.15.12:0/2446501473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe190105e30 0x7fe190106210 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fe180009930 tx=0x7fe18002bc30 comp rx=0 tx=0).stop 2024-04-22T13:29:07.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.483+0000 7fe1956c1640 1 -- 172.21.15.12:0/2446501473 shutdown_connections 2024-04-22T13:29:07.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.483+0000 7fe1956c1640 1 --2- 172.21.15.12:0/2446501473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe190101b60 0x7fe19010dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.483+0000 7fe1956c1640 1 --2- 172.21.15.12:0/2446501473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1901011a0 0x7fe190101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.483+0000 7fe1956c1640 1 --2- 172.21.15.12:0/2446501473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe190105e30 0x7fe190106210 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.483+0000 7fe1956c1640 1 -- 172.21.15.12:0/2446501473 >> 172.21.15.12:0/2446501473 conn(0x7fe190077e30 msgr2=0x7fe190078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:07.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.483+0000 7fe1956c1640 1 -- 172.21.15.12:0/2446501473 shutdown_connections 2024-04-22T13:29:07.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.483+0000 7fe1956c1640 1 -- 172.21.15.12:0/2446501473 wait complete. 2024-04-22T13:29:07.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.484+0000 7fe1956c1640 1 Processor -- start 2024-04-22T13:29:07.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.484+0000 7fe1956c1640 1 -- start start 2024-04-22T13:29:07.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.484+0000 7fe1956c1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1901011a0 0x7fe1901986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.484+0000 7fe1956c1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe190101b60 0x7fe190198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.484+0000 7fe1956c1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe190105e30 0x7fe19019cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.484+0000 7fe1956c1640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe190112240 con 0x7fe190105e30 2024-04-22T13:29:07.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.484+0000 7fe1956c1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe1901120c0 con 0x7fe190101b60 2024-04-22T13:29:07.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.484+0000 7fe1956c1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe1901123c0 con 0x7fe1901011a0 2024-04-22T13:29:07.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.484+0000 7fe18f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe190105e30 0x7fe19019cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.484+0000 7fe18f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe190105e30 0x7fe19019cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51836/0 (socket says 172.21.15.12:51836) 2024-04-22T13:29:07.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.484+0000 7fe18f7fe640 1 -- 172.21.15.12:0/1052207759 learned_addr learned my addr 172.21.15.12:0/1052207759 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:07.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.485+0000 7fe18effd640 1 --2- 172.21.15.12:0/1052207759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1901011a0 0x7fe1901986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.485+0000 7fe18e7fc640 1 --2- 172.21.15.12:0/1052207759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe190101b60 0x7fe190198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.485+0000 7fe18f7fe640 1 -- 172.21.15.12:0/1052207759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1901011a0 msgr2=0x7fe1901986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.485+0000 7fe18f7fe640 1 --2- 172.21.15.12:0/1052207759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1901011a0 0x7fe1901986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.485+0000 7fe18f7fe640 1 -- 172.21.15.12:0/1052207759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe190101b60 msgr2=0x7fe190198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.485+0000 7fe18f7fe640 1 --2- 172.21.15.12:0/1052207759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe190101b60 0x7fe190198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.485+0000 7fe18f7fe640 1 -- 172.21.15.12:0/1052207759 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19019d6b0 con 0x7fe190105e30 2024-04-22T13:29:07.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.485+0000 7fe18f7fe640 1 --2- 172.21.15.12:0/1052207759 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe190105e30 0x7fe19019cf80 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fe18800eb10 tx=0x7fe18800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:07.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.485+0000 7fe177fff640 1 -- 172.21.15.12:0/1052207759 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe188004b20 con 0x7fe190105e30 2024-04-22T13:29:07.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.485+0000 7fe1956c1640 1 -- 172.21.15.12:0/1052207759 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe19019d9a0 con 0x7fe190105e30 2024-04-22T13:29:07.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.485+0000 7fe177fff640 1 -- 172.21.15.12:0/1052207759 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe188004cc0 con 0x7fe190105e30 2024-04-22T13:29:07.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.485+0000 7fe177fff640 1 -- 172.21.15.12:0/1052207759 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe188010670 con 0x7fe190105e30 2024-04-22T13:29:07.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.485+0000 7fe1956c1640 1 -- 172.21.15.12:0/1052207759 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1901a5230 con 0x7fe190105e30 2024-04-22T13:29:07.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.486+0000 7fe1956c1640 1 -- 172.21.15.12:0/1052207759 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe190102570 con 0x7fe190105e30 2024-04-22T13:29:07.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.487+0000 7fe177fff640 1 -- 172.21.15.12:0/1052207759 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe188005000 con 0x7fe190105e30 2024-04-22T13:29:07.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.488+0000 7fe177fff640 1 --2- 172.21.15.12:0/1052207759 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe1780a3d20 0x7fe1780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.488+0000 7fe18effd640 1 --2- 172.21.15.12:0/1052207759 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe1780a3d20 0x7fe1780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.488+0000 7fe177fff640 1 -- 172.21.15.12:0/1052207759 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fe1880d1040 con 0x7fe190105e30 2024-04-22T13:29:07.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.488+0000 7fe18effd640 1 --2- 172.21.15.12:0/1052207759 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe1780a3d20 0x7fe1780a61e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe180006010 tx=0x7fe18002c7e0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:07.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.492+0000 7fe177fff640 1 -- 172.21.15.12:0/1052207759 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe188053ff0 con 0x7fe190105e30 2024-04-22T13:29:07.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.717+0000 7fe1956c1640 1 -- 172.21.15.12:0/1052207759 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe190068de0 con 0x7fe190105e30 2024-04-22T13:29:07.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.718+0000 7fe177fff640 1 -- 172.21.15.12:0/1052207759 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7fe190068de0 con 0x7fe190105e30 2024-04-22T13:29:07.713 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:29:07.713 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:29:07.714 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:29:07.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.720+0000 7fe1956c1640 1 -- 172.21.15.12:0/1052207759 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe1780a3d20 msgr2=0x7fe1780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.720+0000 7fe1956c1640 1 --2- 172.21.15.12:0/1052207759 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe1780a3d20 0x7fe1780a61e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe180006010 tx=0x7fe18002c7e0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.720+0000 7fe1956c1640 1 -- 172.21.15.12:0/1052207759 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe190105e30 msgr2=0x7fe19019cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.720+0000 7fe1956c1640 1 --2- 172.21.15.12:0/1052207759 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe190105e30 0x7fe19019cf80 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fe18800eb10 tx=0x7fe18800c2d0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.720+0000 7fe1956c1640 1 -- 172.21.15.12:0/1052207759 shutdown_connections 2024-04-22T13:29:07.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.720+0000 7fe1956c1640 1 --2- 172.21.15.12:0/1052207759 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe1780a3d20 0x7fe1780a61e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.720+0000 7fe1956c1640 1 --2- 172.21.15.12:0/1052207759 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe190105e30 0x7fe19019cf80 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.720+0000 7fe1956c1640 1 --2- 172.21.15.12:0/1052207759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe190101b60 0x7fe190198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.720+0000 7fe1956c1640 1 --2- 172.21.15.12:0/1052207759 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1901011a0 0x7fe1901986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.720+0000 7fe1956c1640 1 -- 172.21.15.12:0/1052207759 >> 172.21.15.12:0/1052207759 conn(0x7fe190077e30 msgr2=0x7fe1900fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:07.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.720+0000 7fe1956c1640 1 -- 172.21.15.12:0/1052207759 shutdown_connections 2024-04-22T13:29:07.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.720+0000 7fe1956c1640 1 -- 172.21.15.12:0/1052207759 wait complete. 2024-04-22T13:29:07.727 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:29:07.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.835+0000 7f80a3f32640 1 -- 172.21.15.12:0/4165147069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f809c1021a0 msgr2=0x7f809c102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.835+0000 7f80a3f32640 1 --2- 172.21.15.12:0/4165147069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f809c1021a0 0x7f809c102580 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f8094009900 tx=0x7f809402bc30 comp rx=0 tx=0).stop 2024-04-22T13:29:07.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.835+0000 7f80a3f32640 1 -- 172.21.15.12:0/4165147069 shutdown_connections 2024-04-22T13:29:07.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.835+0000 7f80a3f32640 1 --2- 172.21.15.12:0/4165147069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f809c107270 0x7f809c10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.835+0000 7f80a3f32640 1 --2- 172.21.15.12:0/4165147069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f809c102b50 0x7f809c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.835+0000 7f80a3f32640 1 --2- 172.21.15.12:0/4165147069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f809c1021a0 0x7f809c102580 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.835+0000 7f80a3f32640 1 -- 172.21.15.12:0/4165147069 >> 172.21.15.12:0/4165147069 conn(0x7f809c0fd9c0 msgr2=0x7f809c0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:07.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.836+0000 7f80a3f32640 1 -- 172.21.15.12:0/4165147069 shutdown_connections 2024-04-22T13:29:07.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.836+0000 7f80a3f32640 1 -- 172.21.15.12:0/4165147069 wait complete. 2024-04-22T13:29:07.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.836+0000 7f80a3f32640 1 Processor -- start 2024-04-22T13:29:07.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.836+0000 7f80a3f32640 1 -- start start 2024-04-22T13:29:07.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.836+0000 7f80a3f32640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f809c1021a0 0x7f809c19e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.836+0000 7f80a3f32640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f809c107270 0x7f809c19e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.836+0000 7f80a3f32640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f809c1985e0 0x7f809c198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.836+0000 7f80a3f32640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f809c112200 con 0x7f809c1021a0 2024-04-22T13:29:07.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.836+0000 7f80a3f32640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f809c112080 con 0x7f809c1985e0 2024-04-22T13:29:07.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.836+0000 7f80a3f32640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f809c112380 con 0x7f809c107270 2024-04-22T13:29:07.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.836+0000 7f80a1cce640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f809c1021a0 0x7f809c19e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.836+0000 7f80a1cce640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f809c1021a0 0x7f809c19e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51852/0 (socket says 172.21.15.12:51852) 2024-04-22T13:29:07.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.837+0000 7f80a1cce640 1 -- 172.21.15.12:0/2183133425 learned_addr learned my addr 172.21.15.12:0/2183133425 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:07.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.837+0000 7f80a24cf640 1 --2- 172.21.15.12:0/2183133425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f809c1985e0 0x7f809c198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.837+0000 7f80a14cd640 1 --2- 172.21.15.12:0/2183133425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f809c107270 0x7f809c19e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.837+0000 7f80a1cce640 1 -- 172.21.15.12:0/2183133425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f809c107270 msgr2=0x7f809c19e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.837+0000 7f80a1cce640 1 --2- 172.21.15.12:0/2183133425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f809c107270 0x7f809c19e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.837+0000 7f80a1cce640 1 -- 172.21.15.12:0/2183133425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f809c1985e0 msgr2=0x7f809c198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.837+0000 7f80a1cce640 1 --2- 172.21.15.12:0/2183133425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f809c1985e0 0x7f809c198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.837+0000 7f80a1cce640 1 -- 172.21.15.12:0/2183133425 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f809c199100 con 0x7f809c1021a0 2024-04-22T13:29:07.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.837+0000 7f80a1cce640 1 --2- 172.21.15.12:0/2183133425 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f809c1021a0 0x7f809c19e430 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f8094000c00 tx=0x7f809402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:07.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.837+0000 7f809affd640 1 -- 172.21.15.12:0/2183133425 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8094043020 con 0x7f809c1021a0 2024-04-22T13:29:07.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.837+0000 7f809affd640 1 -- 172.21.15.12:0/2183133425 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80940052b0 con 0x7f809c1021a0 2024-04-22T13:29:07.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.837+0000 7f80a3f32640 1 -- 172.21.15.12:0/2183133425 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f809c199390 con 0x7f809c1021a0 2024-04-22T13:29:07.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.837+0000 7f809affd640 1 -- 172.21.15.12:0/2183133425 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8094003c60 con 0x7f809c1021a0 2024-04-22T13:29:07.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.837+0000 7f80a3f32640 1 -- 172.21.15.12:0/2183133425 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f809c1a55e0 con 0x7f809c1021a0 2024-04-22T13:29:07.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.839+0000 7f809affd640 1 -- 172.21.15.12:0/2183133425 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8094046020 con 0x7f809c1021a0 2024-04-22T13:29:07.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.840+0000 7f80a6bfe740 1 Processor -- start 2024-04-22T13:29:07.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.840+0000 7f80a6bfe740 1 -- start start 2024-04-22T13:29:07.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.840+0000 7f80a6bfe740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596c50b74d0 0x5596c50b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.840+0000 7f80a6bfe740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596c50b7e80 0x5596c50c0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.840+0000 7f80a6bfe740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596c50c0a00 0x5596c50c2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.840+0000 7f80a6bfe740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5596c501c1f0 con 0x5596c50b74d0 2024-04-22T13:29:07.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.840+0000 7f80a6bfe740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5596c4fb94b0 con 0x5596c50c0a00 2024-04-22T13:29:07.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.840+0000 7f80a6bfe740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5596c50181b0 con 0x5596c50b7e80 2024-04-22T13:29:07.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.840+0000 7f809affd640 1 --2- 172.21.15.12:0/2183133425 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f80840a3d20 0x7f80840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.840+0000 7f80a24cf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596c50b74d0 0x5596c50b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.840+0000 7f80a24cf640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596c50b74d0 0x5596c50b78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51856/0 (socket says 172.21.15.12:51856) 2024-04-22T13:29:07.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.840+0000 7f80a24cf640 1 -- 172.21.15.12:0/3242927174 learned_addr learned my addr 172.21.15.12:0/3242927174 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:07.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.840+0000 7f80a24cf640 1 --2- 172.21.15.12:0/2183133425 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f80840a3d20 0x7f80840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.841+0000 7f80a1cce640 1 --2- 172.21.15.12:0/3242927174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596c50b7e80 0x5596c50c0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.841+0000 7f809affd640 1 -- 172.21.15.12:0/2183133425 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f80940f2830 con 0x7f809c1021a0 2024-04-22T13:29:07.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.841+0000 7f80a14cd640 1 --2- 172.21.15.12:0/3242927174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596c50c0a00 0x5596c50c2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.841+0000 7f80a24cf640 1 -- 172.21.15.12:0/3242927174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596c50b7e80 msgr2=0x5596c50c0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:07.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.841+0000 7f80a24cf640 1 --2- 172.21.15.12:0/3242927174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596c50b7e80 0x5596c50c0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.841+0000 7f80a24cf640 1 -- 172.21.15.12:0/3242927174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596c50c0a00 msgr2=0x5596c50c2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:07.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.841+0000 7f80a24cf640 1 --2- 172.21.15.12:0/3242927174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596c50c0a00 0x5596c50c2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.841+0000 7f80a24cf640 1 -- 172.21.15.12:0/3242927174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596c4fab1b0 con 0x5596c50b74d0 2024-04-22T13:29:07.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.841+0000 7f80a1cce640 1 --2- 172.21.15.12:0/3242927174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596c50b7e80 0x5596c50c0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:07.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.841+0000 7f80a24cf640 1 --2- 172.21.15.12:0/3242927174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596c50b74d0 0x5596c50b78b0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f808c012d00 tx=0x7f808c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26978e16b58540e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:07.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.841+0000 7f80a24cf640 1 --2- 172.21.15.12:0/2183133425 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f80840a3d20 0x7f80840a61e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f808c010c40 tx=0x7f808c00c000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:07.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.841+0000 7f8083fff640 1 -- 172.21.15.12:0/3242927174 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c024070 con 0x5596c50b74d0 2024-04-22T13:29:07.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.841+0000 7f8083fff640 1 -- 172.21.15.12:0/3242927174 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f808c009d20 con 0x5596c50b74d0 2024-04-22T13:29:07.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.841+0000 7f8083fff640 1 -- 172.21.15.12:0/3242927174 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c005280 con 0x5596c50b74d0 2024-04-22T13:29:07.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.842+0000 7f80a6bfe740 1 -- 172.21.15.12:0/3242927174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596c50b74d0 msgr2=0x5596c50b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.842+0000 7f80a6bfe740 1 --2- 172.21.15.12:0/3242927174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596c50b74d0 0x5596c50b78b0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f808c012d00 tx=0x7f808c010740 comp rx=0 tx=0).stop 2024-04-22T13:29:07.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.842+0000 7f80a6bfe740 1 -- 172.21.15.12:0/3242927174 shutdown_connections 2024-04-22T13:29:07.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.842+0000 7f80a6bfe740 1 --2- 172.21.15.12:0/3242927174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596c50c0a00 0x5596c50c2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.842+0000 7f80a6bfe740 1 --2- 172.21.15.12:0/3242927174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596c50b7e80 0x5596c50c0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.842+0000 7f80a6bfe740 1 --2- 172.21.15.12:0/3242927174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596c50b74d0 0x5596c50b78b0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.842+0000 7f80a6bfe740 1 -- 172.21.15.12:0/3242927174 >> 172.21.15.12:0/3242927174 conn(0x5596c50b1ba0 msgr2=0x5596c50b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.842+0000 7f80a6bfe740 1 -- 172.21.15.12:0/3242927174 shutdown_connections 2024-04-22T13:29:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.842+0000 7f80a6bfe740 1 -- 172.21.15.12:0/3242927174 wait complete. 2024-04-22T13:29:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.842+0000 7f80a6bfe740 1 -- start start 2024-04-22T13:29:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.843+0000 7f80a6bfe740 1 Processor -- start 2024-04-22T13:29:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.843+0000 7f80a6bfe740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596c50b74d0 0x5596c50c0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.843+0000 7f80a6bfe740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596c50b7e80 0x5596c50bd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.843+0000 7f80a6bfe740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596c50c0a00 0x5596c50bdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.843+0000 7f80a6bfe740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5596c50c3910 con 0x5596c50b7e80 2024-04-22T13:29:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.843+0000 7f80a6bfe740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5596c4fd5500 con 0x5596c50c0a00 2024-04-22T13:29:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.843+0000 7f80a6bfe740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5596c50c3a90 con 0x5596c50b74d0 2024-04-22T13:29:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.843+0000 7f80a6bfe740 10 client.0 _ll_register_callbacks cb 0x5596c4fa3b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.843+0000 7f80a14cd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596c50b7e80 0x5596c50bd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.843+0000 7f80a14cd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596c50b7e80 0x5596c50bd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51870/0 (socket says 172.21.15.12:51870) 2024-04-22T13:29:07.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.843+0000 7f80a14cd640 1 -- 172.21.15.12:0/1881327372 learned_addr learned my addr 172.21.15.12:0/1881327372 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:07.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.843+0000 7f80a1cce640 1 --2- 172.21.15.12:0/1881327372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596c50b74d0 0x5596c50c0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.843+0000 7f80a24cf640 1 --2- 172.21.15.12:0/1881327372 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596c50c0a00 0x5596c50bdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.843+0000 7f80a14cd640 1 -- 172.21.15.12:0/1881327372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596c50b74d0 msgr2=0x5596c50c0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:07.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.843+0000 7f80a14cd640 1 --2- 172.21.15.12:0/1881327372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596c50b74d0 0x5596c50c0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.843+0000 7f80a14cd640 1 -- 172.21.15.12:0/1881327372 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596c50c0a00 msgr2=0x5596c50bdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:07.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.843+0000 7f80a14cd640 1 --2- 172.21.15.12:0/1881327372 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596c50c0a00 0x5596c50bdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.843+0000 7f80a14cd640 1 -- 172.21.15.12:0/1881327372 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596c4d2e3f0 con 0x5596c50b7e80 2024-04-22T13:29:07.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.843+0000 7f80a24cf640 1 --2- 172.21.15.12:0/1881327372 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596c50c0a00 0x5596c50bdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:07.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.844+0000 7f80a14cd640 1 --2- 172.21.15.12:0/1881327372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596c50b7e80 0x5596c50bd700 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f809000ba20 tx=0x7f809000bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c6b970aa566641d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:07.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.844+0000 7f80817fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:07.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.844+0000 7f80817fa640 1 -- 172.21.15.12:0/1881327372 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809001b440 con 0x5596c50b7e80 2024-04-22T13:29:07.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.844+0000 7f80817fa640 1 -- 172.21.15.12:0/1881327372 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f809001b5e0 con 0x5596c50b7e80 2024-04-22T13:29:07.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.844+0000 7f80a6bfe740 1 -- 172.21.15.12:0/1881327372 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5596c4fbb930 con 0x5596c50b7e80 2024-04-22T13:29:07.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.844+0000 7f80817fa640 1 -- 172.21.15.12:0/1881327372 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8090005c10 con 0x5596c50b7e80 2024-04-22T13:29:07.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.844+0000 7f80a6bfe740 10 client.17493 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:07.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.844+0000 7f80a6bfe740 1 -- 172.21.15.12:0/1881327372 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5596c4df87d0 con 0x5596c50b7e80 2024-04-22T13:29:07.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.844+0000 7f80817fa640 1 -- 172.21.15.12:0/1881327372 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5596c4fbb930 con 0x5596c50b7e80 2024-04-22T13:29:07.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.844+0000 7f80817fa640 1 -- 172.21.15.12:0/1881327372 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8090019020 con 0x5596c50b7e80 2024-04-22T13:29:07.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.844+0000 7f80a6bfe740 10 client.17493 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:07.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.844+0000 7f80a6bfe740 10 client.17493 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:29:07.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.844+0000 7f80a6bfe740 20 client.17493 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:07.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.844+0000 7f80a6bfe740 1 --2- 172.21.15.12:0/1881327372 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5596c50d0bb0 0x5596c50d2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.844+0000 7f80a6bfe740 4 client.17493 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:07.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.844+0000 7f80a6bfe740 1 -- 172.21.15.12:0/1881327372 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596c5029c10 con 0x5596c50d0bb0 2024-04-22T13:29:07.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.845+0000 7f80a1cce640 1 --2- 172.21.15.12:0/1881327372 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5596c50d0bb0 0x5596c50d2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.845+0000 7f80a1cce640 1 --2- 172.21.15.12:0/1881327372 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5596c50d0bb0 0x5596c50d2ff0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=eea69a2d3d115ef0 server_cookie=7ee2f6ecd1f871e8 in_seq=0 out_seq=0 2024-04-22T13:29:07.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.845+0000 7f80817fa640 10 client.17493 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:29:07.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.846+0000 7f80817fa640 1 -- 172.21.15.12:0/1881327372 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5596c5029c10 con 0x5596c50d0bb0 2024-04-22T13:29:07.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.846+0000 7f80817fa640 10 client.17493 handle_command_reply: tid=0 2024-04-22T13:29:07.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.846+0000 7f80a6bfe740 1 client.17493 shutdown 2024-04-22T13:29:07.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.846+0000 7f80a6bfe740 1 -- 172.21.15.12:0/1881327372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596c50b7e80 msgr2=0x5596c50bd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.846+0000 7f80a6bfe740 1 --2- 172.21.15.12:0/1881327372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596c50b7e80 0x5596c50bd700 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f809000ba20 tx=0x7f809000bee0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.846+0000 7f80a6bfe740 1 -- 172.21.15.12:0/1881327372 shutdown_connections 2024-04-22T13:29:07.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.846+0000 7f80a6bfe740 1 --2- 172.21.15.12:0/1881327372 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5596c50d0bb0 0x5596c50d2ff0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.846+0000 7f80a6bfe740 1 --2- 172.21.15.12:0/1881327372 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596c50c0a00 0x5596c50bdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.846+0000 7f80a6bfe740 1 --2- 172.21.15.12:0/1881327372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596c50b7e80 0x5596c50bd700 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.846+0000 7f80a6bfe740 1 --2- 172.21.15.12:0/1881327372 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596c50b74d0 0x5596c50c0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.846+0000 7f80817fa640 0 client.17493 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:29:07.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.846+0000 7f80a6bfe740 1 -- 172.21.15.12:0/1881327372 >> 172.21.15.12:0/1881327372 conn(0x5596c50b1ba0 msgr2=0x5596c50b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:07.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.846+0000 7f80a6bfe740 1 -- 172.21.15.12:0/1881327372 shutdown_connections 2024-04-22T13:29:07.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.846+0000 7f80a6bfe740 1 -- 172.21.15.12:0/1881327372 wait complete. 2024-04-22T13:29:07.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.846+0000 7f80a6bfe740 20 client.17493 trim_cache size 0 max 16384 2024-04-22T13:29:07.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.860+0000 7f80a6bfe740 1 Processor -- start 2024-04-22T13:29:07.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.860+0000 7f80a6bfe740 1 -- start start 2024-04-22T13:29:07.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.860+0000 7f80a6bfe740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596c50b74d0 0x5596c50cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.860+0000 7f80a6bfe740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596c50b7e80 0x5596c50bfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.860+0000 7f80a6bfe740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596c50c0a00 0x5596c50c0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.860+0000 7f80a6bfe740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f809c1adab0 con 0x5596c50b74d0 2024-04-22T13:29:07.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.860+0000 7f80a6bfe740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f809c1af5f0 con 0x5596c50c0a00 2024-04-22T13:29:07.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.860+0000 7f80a6bfe740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f809c1ae180 con 0x5596c50b7e80 2024-04-22T13:29:07.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.860+0000 7f80a1cce640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596c50b74d0 0x5596c50cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.860+0000 7f80a1cce640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596c50b74d0 0x5596c50cc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51876/0 (socket says 172.21.15.12:51876) 2024-04-22T13:29:07.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.860+0000 7f80a1cce640 1 -- 172.21.15.12:0/3881746715 learned_addr learned my addr 172.21.15.12:0/3881746715 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:07.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.860+0000 7f80a24cf640 1 --2- 172.21.15.12:0/3881746715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596c50c0a00 0x5596c50c0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.860+0000 7f80a14cd640 1 --2- 172.21.15.12:0/3881746715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596c50b7e80 0x5596c50bfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.860+0000 7f80a1cce640 1 -- 172.21.15.12:0/3881746715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596c50b7e80 msgr2=0x5596c50bfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:07.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.860+0000 7f80a1cce640 1 --2- 172.21.15.12:0/3881746715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596c50b7e80 0x5596c50bfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.860+0000 7f80a1cce640 1 -- 172.21.15.12:0/3881746715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596c50c0a00 msgr2=0x5596c50c0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:07.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.860+0000 7f80a1cce640 1 --2- 172.21.15.12:0/3881746715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596c50c0a00 0x5596c50c0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.860+0000 7f80a1cce640 1 -- 172.21.15.12:0/3881746715 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596c50cf7b0 con 0x5596c50b74d0 2024-04-22T13:29:07.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.861+0000 7f80a1cce640 1 --2- 172.21.15.12:0/3881746715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596c50b74d0 0x5596c50cc7d0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f80940f05d0 tx=0x7f80940df040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce83ce5d24bfdd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:07.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.861+0000 7f80837fe640 1 -- 172.21.15.12:0/3881746715 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809402cca0 con 0x5596c50b74d0 2024-04-22T13:29:07.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.861+0000 7f80837fe640 1 -- 172.21.15.12:0/3881746715 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80940bf030 con 0x5596c50b74d0 2024-04-22T13:29:07.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.861+0000 7f80837fe640 1 -- 172.21.15.12:0/3881746715 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80940bca60 con 0x5596c50b74d0 2024-04-22T13:29:07.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.861+0000 7f80a6bfe740 1 -- 172.21.15.12:0/3881746715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596c50b74d0 msgr2=0x5596c50cc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.861+0000 7f80a6bfe740 1 --2- 172.21.15.12:0/3881746715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596c50b74d0 0x5596c50cc7d0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f80940f05d0 tx=0x7f80940df040 comp rx=0 tx=0).stop 2024-04-22T13:29:07.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.861+0000 7f80a6bfe740 1 -- 172.21.15.12:0/3881746715 shutdown_connections 2024-04-22T13:29:07.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.861+0000 7f80a6bfe740 1 --2- 172.21.15.12:0/3881746715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596c50c0a00 0x5596c50c0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.861+0000 7f80a6bfe740 1 --2- 172.21.15.12:0/3881746715 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596c50b7e80 0x5596c50bfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.861+0000 7f80a6bfe740 1 --2- 172.21.15.12:0/3881746715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596c50b74d0 0x5596c50cc7d0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.861+0000 7f80a6bfe740 1 -- 172.21.15.12:0/3881746715 >> 172.21.15.12:0/3881746715 conn(0x5596c50b1ba0 msgr2=0x5596c50b2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:07.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.861+0000 7f80a6bfe740 1 -- 172.21.15.12:0/3881746715 shutdown_connections 2024-04-22T13:29:07.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.862+0000 7f80a6bfe740 1 -- 172.21.15.12:0/3881746715 wait complete. 2024-04-22T13:29:07.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.862+0000 7f80a6bfe740 1 -- start start 2024-04-22T13:29:07.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.862+0000 7f80a6bfe740 1 Processor -- start 2024-04-22T13:29:07.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.862+0000 7f80a6bfe740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596c50b74d0 0x5596c50baa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.862+0000 7f80a6bfe740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596c50b7e80 0x5596c50baf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.862+0000 7f80a6bfe740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596c50c0a00 0x5596c5136130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.862+0000 7f80a6bfe740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f809c1aef20 con 0x5596c50b7e80 2024-04-22T13:29:07.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.862+0000 7f80a6bfe740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f809c1ae850 con 0x5596c50c0a00 2024-04-22T13:29:07.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.862+0000 7f80a6bfe740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f809c1abab0 con 0x5596c50b74d0 2024-04-22T13:29:07.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.862+0000 7f80a6bfe740 10 client.0 _ll_register_callbacks cb 0x7f8090004380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:07.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.862+0000 7f80a14cd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596c50b7e80 0x5596c50baf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.862+0000 7f80a14cd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596c50b7e80 0x5596c50baf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51890/0 (socket says 172.21.15.12:51890) 2024-04-22T13:29:07.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.862+0000 7f80a14cd640 1 -- 172.21.15.12:0/2003617252 learned_addr learned my addr 172.21.15.12:0/2003617252 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:07.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.862+0000 7f80a1cce640 1 --2- 172.21.15.12:0/2003617252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596c50b74d0 0x5596c50baa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.863+0000 7f80a14cd640 1 -- 172.21.15.12:0/2003617252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596c50b74d0 msgr2=0x5596c50baa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:07.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.863+0000 7f80a14cd640 1 --2- 172.21.15.12:0/2003617252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596c50b74d0 0x5596c50baa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.863+0000 7f80a24cf640 1 --2- 172.21.15.12:0/2003617252 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596c50c0a00 0x5596c5136130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.863+0000 7f80a14cd640 1 -- 172.21.15.12:0/2003617252 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596c50c0a00 msgr2=0x5596c5136130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:07.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.863+0000 7f80a14cd640 1 --2- 172.21.15.12:0/2003617252 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596c50c0a00 0x5596c5136130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.863+0000 7f80a14cd640 1 -- 172.21.15.12:0/2003617252 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596c5136670 con 0x5596c50b7e80 2024-04-22T13:29:07.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.863+0000 7f80a14cd640 1 --2- 172.21.15.12:0/2003617252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596c50b7e80 0x5596c50baf90 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f809000c010 tx=0x7f8090005950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83c2fd04ddd8c37f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:07.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.863+0000 7f80817fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:07.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.863+0000 7f80817fa640 1 -- 172.21.15.12:0/2003617252 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809002b070 con 0x5596c50b7e80 2024-04-22T13:29:07.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.863+0000 7f80a6bfe740 1 -- 172.21.15.12:0/2003617252 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80940bf030 con 0x5596c50b7e80 2024-04-22T13:29:07.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.863+0000 7f80817fa640 1 -- 172.21.15.12:0/2003617252 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f8090005450 con 0x5596c50b7e80 2024-04-22T13:29:07.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.863+0000 7f80817fa640 1 -- 172.21.15.12:0/2003617252 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8090004640 con 0x5596c50b7e80 2024-04-22T13:29:07.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.863+0000 7f80a6bfe740 10 client.17499 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:07.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.863+0000 7f80a6bfe740 1 -- 172.21.15.12:0/2003617252 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5596c5137930 con 0x5596c50b7e80 2024-04-22T13:29:07.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.863+0000 7f80817fa640 1 -- 172.21.15.12:0/2003617252 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80940bf030 con 0x5596c50b7e80 2024-04-22T13:29:07.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.863+0000 7f80817fa640 1 -- 172.21.15.12:0/2003617252 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8090019040 con 0x5596c50b7e80 2024-04-22T13:29:07.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.864+0000 7f80a6bfe740 10 client.17499 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:07.865 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.864+0000 7f80a6bfe740 10 client.17499 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:29:07.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.864+0000 7f80a6bfe740 20 client.17499 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:07.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.864+0000 7f80a6bfe740 1 --2- 172.21.15.12:0/2003617252 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5596c5138100 0x5596c513a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.864+0000 7f80a6bfe740 4 client.17499 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:29:07.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.864+0000 7f80a6bfe740 1 -- 172.21.15.12:0/2003617252 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5596c5023a80 con 0x5596c5138100 2024-04-22T13:29:07.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.864+0000 7f80a1cce640 1 --2- 172.21.15.12:0/2003617252 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5596c5138100 0x5596c513a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.864+0000 7f80a1cce640 1 --2- 172.21.15.12:0/2003617252 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5596c5138100 0x5596c513a520 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d48b7f685288b837 server_cookie=4161080a8d2afc32 in_seq=0 out_seq=0 2024-04-22T13:29:07.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.864+0000 7f80817fa640 10 client.17499 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:29:07.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.864+0000 7f80817fa640 1 -- 172.21.15.12:0/2003617252 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5596c5023a80 con 0x5596c5138100 2024-04-22T13:29:07.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.864+0000 7f80817fa640 10 client.17499 handle_command_reply: tid=0 2024-04-22T13:29:07.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.864+0000 7f80a6bfe740 1 client.17499 shutdown 2024-04-22T13:29:07.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.864+0000 7f80a6bfe740 1 -- 172.21.15.12:0/2003617252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596c50b7e80 msgr2=0x5596c50baf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.864+0000 7f80a6bfe740 1 --2- 172.21.15.12:0/2003617252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596c50b7e80 0x5596c50baf90 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f809000c010 tx=0x7f8090005950 comp rx=0 tx=0).stop 2024-04-22T13:29:07.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.865+0000 7f80a6bfe740 1 -- 172.21.15.12:0/2003617252 shutdown_connections 2024-04-22T13:29:07.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.865+0000 7f80a6bfe740 1 --2- 172.21.15.12:0/2003617252 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5596c5138100 0x5596c513a520 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.865+0000 7f80a6bfe740 1 --2- 172.21.15.12:0/2003617252 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5596c50c0a00 0x5596c5136130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.865+0000 7f80a6bfe740 1 --2- 172.21.15.12:0/2003617252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5596c50b7e80 0x5596c50baf90 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.865+0000 7f80a6bfe740 1 --2- 172.21.15.12:0/2003617252 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5596c50b74d0 0x5596c50baa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.865+0000 7f80817fa640 0 client.17499 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:29:07.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.865+0000 7f80a6bfe740 1 -- 172.21.15.12:0/2003617252 >> 172.21.15.12:0/2003617252 conn(0x5596c50b1ba0 msgr2=0x5596c50b3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:07.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.865+0000 7f80a6bfe740 1 -- 172.21.15.12:0/2003617252 shutdown_connections 2024-04-22T13:29:07.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.865+0000 7f80a6bfe740 1 -- 172.21.15.12:0/2003617252 wait complete. 2024-04-22T13:29:07.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.865+0000 7f80a6bfe740 20 client.17499 trim_cache size 0 max 16384 2024-04-22T13:29:07.867 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:29:07.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.865+0000 7f80a3f32640 1 -- 172.21.15.12:0/2183133425 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f80840a3d20 msgr2=0x7f80840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.865+0000 7f80a3f32640 1 --2- 172.21.15.12:0/2183133425 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f80840a3d20 0x7f80840a61e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f808c010c40 tx=0x7f808c00c000 comp rx=0 tx=0).stop 2024-04-22T13:29:07.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.865+0000 7f80a3f32640 1 -- 172.21.15.12:0/2183133425 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f809c1021a0 msgr2=0x7f809c19e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.865+0000 7f80a3f32640 1 --2- 172.21.15.12:0/2183133425 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f809c1021a0 0x7f809c19e430 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f8094000c00 tx=0x7f809402c7e0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.865+0000 7f80a3f32640 1 -- 172.21.15.12:0/2183133425 shutdown_connections 2024-04-22T13:29:07.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.865+0000 7f80a3f32640 1 --2- 172.21.15.12:0/2183133425 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f80840a3d20 0x7f80840a61e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.865+0000 7f80a3f32640 1 --2- 172.21.15.12:0/2183133425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f809c1985e0 0x7f809c198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.865+0000 7f80a3f32640 1 --2- 172.21.15.12:0/2183133425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f809c107270 0x7f809c19e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.865+0000 7f80a3f32640 1 --2- 172.21.15.12:0/2183133425 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f809c1021a0 0x7f809c19e430 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.865+0000 7f80a3f32640 1 -- 172.21.15.12:0/2183133425 >> 172.21.15.12:0/2183133425 conn(0x7f809c0fd9c0 msgr2=0x7f809c0fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:07.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.865+0000 7f80a3f32640 1 -- 172.21.15.12:0/2183133425 shutdown_connections 2024-04-22T13:29:07.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.865+0000 7f80a3f32640 1 -- 172.21.15.12:0/2183133425 wait complete. 2024-04-22T13:29:07.871 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:29:07.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:07 smithi134 ceph-mon[31338]: pgmap v153: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:29:07.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.980+0000 7f8be9293640 1 -- 172.21.15.12:0/4133966454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8be41010a0 msgr2=0x7f8be4101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.980+0000 7f8be9293640 1 --2- 172.21.15.12:0/4133966454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8be41010a0 0x7f8be4101480 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4009900 tx=0x7f8bd402bc30 comp rx=0 tx=0).stop 2024-04-22T13:29:07.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.980+0000 7f8be9293640 1 -- 172.21.15.12:0/4133966454 shutdown_connections 2024-04-22T13:29:07.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.980+0000 7f8be9293640 1 --2- 172.21.15.12:0/4133966454 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8be4106170 0x7f8be410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.980+0000 7f8be9293640 1 --2- 172.21.15.12:0/4133966454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8be4101a50 0x7f8be4105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.980+0000 7f8be9293640 1 --2- 172.21.15.12:0/4133966454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8be41010a0 0x7f8be4101480 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.980+0000 7f8be9293640 1 -- 172.21.15.12:0/4133966454 >> 172.21.15.12:0/4133966454 conn(0x7f8be4077eb0 msgr2=0x7f8be40782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:07.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.980+0000 7f8be9293640 1 -- 172.21.15.12:0/4133966454 shutdown_connections 2024-04-22T13:29:07.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.980+0000 7f8be9293640 1 -- 172.21.15.12:0/4133966454 wait complete. 2024-04-22T13:29:07.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.981+0000 7f8be9293640 1 Processor -- start 2024-04-22T13:29:07.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.981+0000 7f8be9293640 1 -- start start 2024-04-22T13:29:07.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.981+0000 7f8be9293640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8be41010a0 0x7f8be4199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.981+0000 7f8be9293640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8be4106170 0x7f8be4199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.981+0000 7f8be9293640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8be4199ff0 0x7f8be41a45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.981+0000 7f8be9293640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8be41110e0 con 0x7f8be4106170 2024-04-22T13:29:07.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.981+0000 7f8be9293640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8be4110f60 con 0x7f8be4199ff0 2024-04-22T13:29:07.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.981+0000 7f8be9293640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8be4111260 con 0x7f8be41010a0 2024-04-22T13:29:07.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.981+0000 7f8bdbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8be4106170 0x7f8be4199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.981+0000 7f8bdbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8be4106170 0x7f8be4199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51894/0 (socket says 172.21.15.12:51894) 2024-04-22T13:29:07.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.981+0000 7f8bdbfff640 1 -- 172.21.15.12:0/849775867 learned_addr learned my addr 172.21.15.12:0/849775867 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:07.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.981+0000 7f8be37fe640 1 --2- 172.21.15.12:0/849775867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8be4199ff0 0x7f8be41a45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.982+0000 7f8be2ffd640 1 --2- 172.21.15.12:0/849775867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8be41010a0 0x7f8be4199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.982+0000 7f8bdbfff640 1 -- 172.21.15.12:0/849775867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8be41010a0 msgr2=0x7f8be4199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.982+0000 7f8bdbfff640 1 --2- 172.21.15.12:0/849775867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8be41010a0 0x7f8be4199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.982+0000 7f8bdbfff640 1 -- 172.21.15.12:0/849775867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8be4199ff0 msgr2=0x7f8be41a45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.982+0000 7f8bdbfff640 1 --2- 172.21.15.12:0/849775867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8be4199ff0 0x7f8be41a45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.982+0000 7f8bdbfff640 1 -- 172.21.15.12:0/849775867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be41a4d70 con 0x7f8be4106170 2024-04-22T13:29:07.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.982+0000 7f8bdbfff640 1 --2- 172.21.15.12:0/849775867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8be4106170 0x7f8be4199a60 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00c970 tx=0x7f8bcc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:07.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.982+0000 7f8be0ff9640 1 -- 172.21.15.12:0/849775867 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc014070 con 0x7f8be4106170 2024-04-22T13:29:07.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.982+0000 7f8be0ff9640 1 -- 172.21.15.12:0/849775867 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bcc004440 con 0x7f8be4106170 2024-04-22T13:29:07.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.982+0000 7f8be9293640 1 -- 172.21.15.12:0/849775867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be41a5060 con 0x7f8be4106170 2024-04-22T13:29:07.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.982+0000 7f8be0ff9640 1 -- 172.21.15.12:0/849775867 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc003e50 con 0x7f8be4106170 2024-04-22T13:29:07.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.982+0000 7f8be9293640 1 -- 172.21.15.12:0/849775867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be41a55b0 con 0x7f8be4106170 2024-04-22T13:29:07.982 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:07 smithi111 ceph-mon[31361]: pgmap v153: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:29:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.984+0000 7f8be0ff9640 1 -- 172.21.15.12:0/849775867 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8bcc017070 con 0x7f8be4106170 2024-04-22T13:29:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.984+0000 7f8bebf63740 1 Processor -- start 2024-04-22T13:29:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.985+0000 7f8bebf63740 1 -- start start 2024-04-22T13:29:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.985+0000 7f8bebf63740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5560c05b24d0 0x5560c05b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.985+0000 7f8bebf63740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5560c05b2e80 0x5560c05bb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.985+0000 7f8bebf63740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5560c05bba00 0x5560c05bde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.985+0000 7f8bebf63740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5560c05171f0 con 0x5560c05bba00 2024-04-22T13:29:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.985+0000 7f8bebf63740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5560c04b44b0 con 0x5560c05b2e80 2024-04-22T13:29:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.985+0000 7f8bebf63740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5560c05131b0 con 0x5560c05b24d0 2024-04-22T13:29:07.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.985+0000 7f8bdbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5560c05bba00 0x5560c05bde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.985+0000 7f8be2ffd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5560c05b2e80 0x5560c05bb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.985+0000 7f8bdbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5560c05bba00 0x5560c05bde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51902/0 (socket says 172.21.15.12:51902) 2024-04-22T13:29:07.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.985+0000 7f8bdbfff640 1 -- 172.21.15.12:0/3740922059 learned_addr learned my addr 172.21.15.12:0/3740922059 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:07.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.985+0000 7f8be0ff9640 1 --2- 172.21.15.12:0/849775867 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f8bd00a3ec0 0x7f8bd00a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.985+0000 7f8be37fe640 1 --2- 172.21.15.12:0/3740922059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5560c05b24d0 0x5560c05b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.986+0000 7f8be37fe640 1 --2- 172.21.15.12:0/849775867 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f8bd00a3ec0 0x7f8bd00a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.986+0000 7f8be2ffd640 1 -- 172.21.15.12:0/3740922059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5560c05b24d0 msgr2=0x5560c05b28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:07.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.986+0000 7f8be2ffd640 1 --2- 172.21.15.12:0/3740922059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5560c05b24d0 0x5560c05b28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.986+0000 7f8be2ffd640 1 -- 172.21.15.12:0/3740922059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5560c05bba00 msgr2=0x5560c05bde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:07.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.986+0000 7f8be2ffd640 1 --2- 172.21.15.12:0/3740922059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5560c05bba00 0x5560c05bde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.986+0000 7f8be2ffd640 1 -- 172.21.15.12:0/3740922059 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560c04a61b0 con 0x5560c05b2e80 2024-04-22T13:29:07.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.986+0000 7f8bdbfff640 1 --2- 172.21.15.12:0/3740922059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5560c05bba00 0x5560c05bde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:29:07.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.986+0000 7f8be0ff9640 1 -- 172.21.15.12:0/849775867 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f8bcc0d0340 con 0x7f8be4106170 2024-04-22T13:29:07.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.986+0000 7f8be2ffd640 1 --2- 172.21.15.12:0/3740922059 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5560c05b2e80 0x5560c05bb430 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4009230 tx=0x7f8bd4005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d1d00bd01c3af9e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:07.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.986+0000 7f8be37fe640 1 --2- 172.21.15.12:0/849775867 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f8bd00a3ec0 0x7f8bd00a6380 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc009fd0 tx=0x7f8bdc004ab0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:07.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.986+0000 7f8bd97fa640 1 -- 172.21.15.12:0/3740922059 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4044020 con 0x5560c05b2e80 2024-04-22T13:29:07.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.986+0000 7f8bd97fa640 1 -- 172.21.15.12:0/3740922059 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bd402cc90 con 0x5560c05b2e80 2024-04-22T13:29:07.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.986+0000 7f8bd97fa640 1 -- 172.21.15.12:0/3740922059 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd40046c0 con 0x5560c05b2e80 2024-04-22T13:29:07.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.987+0000 7f8bebf63740 1 -- 172.21.15.12:0/3740922059 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5560c05b2e80 msgr2=0x5560c05bb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.987+0000 7f8bebf63740 1 --2- 172.21.15.12:0/3740922059 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5560c05b2e80 0x5560c05bb430 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4009230 tx=0x7f8bd4005e60 comp rx=0 tx=0).stop 2024-04-22T13:29:07.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.987+0000 7f8bebf63740 1 -- 172.21.15.12:0/3740922059 shutdown_connections 2024-04-22T13:29:07.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.987+0000 7f8bebf63740 1 --2- 172.21.15.12:0/3740922059 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5560c05bba00 0x5560c05bde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.987+0000 7f8bebf63740 1 --2- 172.21.15.12:0/3740922059 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5560c05b2e80 0x5560c05bb430 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.987+0000 7f8bebf63740 1 --2- 172.21.15.12:0/3740922059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5560c05b24d0 0x5560c05b28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.987+0000 7f8bebf63740 1 -- 172.21.15.12:0/3740922059 >> 172.21.15.12:0/3740922059 conn(0x5560c05acba0 msgr2=0x5560c05acf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:07.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.987+0000 7f8bebf63740 1 -- 172.21.15.12:0/3740922059 shutdown_connections 2024-04-22T13:29:07.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.987+0000 7f8bebf63740 1 -- 172.21.15.12:0/3740922059 wait complete. 2024-04-22T13:29:07.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.987+0000 7f8bebf63740 1 -- start start 2024-04-22T13:29:07.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.987+0000 7f8bebf63740 1 Processor -- start 2024-04-22T13:29:07.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.988+0000 7f8bebf63740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5560c05b24d0 0x5560c05bb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.988+0000 7f8bebf63740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5560c05b2e80 0x5560c05b85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.988+0000 7f8bebf63740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5560c05bba00 0x5560c05b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.988+0000 7f8bebf63740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5560c05be910 con 0x5560c05b24d0 2024-04-22T13:29:07.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.988+0000 7f8bebf63740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5560c04d0500 con 0x5560c05b2e80 2024-04-22T13:29:07.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.988+0000 7f8bebf63740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5560c05bea90 con 0x5560c05bba00 2024-04-22T13:29:07.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.988+0000 7f8bebf63740 10 client.0 _ll_register_callbacks cb 0x5560c049eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:07.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.988+0000 7f8be2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5560c05b24d0 0x5560c05bb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.988+0000 7f8be2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5560c05b24d0 0x5560c05bb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51910/0 (socket says 172.21.15.12:51910) 2024-04-22T13:29:07.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.988+0000 7f8be2ffd640 1 -- 172.21.15.12:0/827948491 learned_addr learned my addr 172.21.15.12:0/827948491 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:07.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.988+0000 7f8bdbfff640 1 --2- 172.21.15.12:0/827948491 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5560c05b2e80 0x5560c05b85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.988+0000 7f8be2ffd640 1 -- 172.21.15.12:0/827948491 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5560c05bba00 msgr2=0x5560c05b8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:07.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.988+0000 7f8be2ffd640 1 --2- 172.21.15.12:0/827948491 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5560c05bba00 0x5560c05b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.988+0000 7f8be2ffd640 1 -- 172.21.15.12:0/827948491 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5560c05b2e80 msgr2=0x5560c05b85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:07.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.988+0000 7f8be2ffd640 1 --2- 172.21.15.12:0/827948491 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5560c05b2e80 0x5560c05b85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.988+0000 7f8be2ffd640 1 -- 172.21.15.12:0/827948491 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560c02293f0 con 0x5560c05b24d0 2024-04-22T13:29:07.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.988+0000 7f8bdbfff640 1 --2- 172.21.15.12:0/827948491 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5560c05b2e80 0x5560c05b85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:07.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.988+0000 7f8be2ffd640 1 --2- 172.21.15.12:0/827948491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5560c05b24d0 0x5560c05bb260 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4000c00 tx=0x7f8bd4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ffe6a167015b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:07.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.988+0000 7f8bca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:07.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.988+0000 7f8bca7fc640 1 -- 172.21.15.12:0/827948491 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd404d070 con 0x5560c05b24d0 2024-04-22T13:29:07.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.988+0000 7f8bca7fc640 1 -- 172.21.15.12:0/827948491 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f8bd4004450 con 0x5560c05b24d0 2024-04-22T13:29:07.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.988+0000 7f8bebf63740 1 -- 172.21.15.12:0/827948491 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bd402cc90 con 0x5560c05b24d0 2024-04-22T13:29:07.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.988+0000 7f8bca7fc640 1 -- 172.21.15.12:0/827948491 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8bd4059050 con 0x5560c05b24d0 2024-04-22T13:29:07.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.988+0000 7f8bebf63740 10 client.17511 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:07.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.988+0000 7f8bebf63740 1 -- 172.21.15.12:0/827948491 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5560c02f37d0 con 0x5560c05b24d0 2024-04-22T13:29:07.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.989+0000 7f8bca7fc640 1 -- 172.21.15.12:0/827948491 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bd402cc90 con 0x5560c05b24d0 2024-04-22T13:29:07.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.989+0000 7f8bca7fc640 1 -- 172.21.15.12:0/827948491 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8bd4040830 con 0x5560c05b24d0 2024-04-22T13:29:07.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.989+0000 7f8bebf63740 10 client.17511 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:07.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.989+0000 7f8bebf63740 10 client.17511 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:29:07.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.989+0000 7f8bebf63740 20 client.17511 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:07.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.989+0000 7f8bebf63740 1 --2- 172.21.15.12:0/827948491 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5560c05cb9d0 0x5560c05cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:07.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.989+0000 7f8bebf63740 4 client.17511 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:07.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.989+0000 7f8bebf63740 1 -- 172.21.15.12:0/827948491 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560c04b6930 con 0x5560c05cb9d0 2024-04-22T13:29:07.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.989+0000 7f8bdbfff640 1 --2- 172.21.15.12:0/827948491 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5560c05cb9d0 0x5560c05cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:07.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.989+0000 7f8bca7fc640 10 client.17511 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:29:07.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.989+0000 7f8bdbfff640 1 --2- 172.21.15.12:0/827948491 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5560c05cb9d0 0x5560c05cddc0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=382d43c466c3e139 server_cookie=53eb89bd499325f5 in_seq=0 out_seq=0 2024-04-22T13:29:07.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.990+0000 7f8bca7fc640 1 -- 172.21.15.12:0/827948491 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5560c04b6930 con 0x5560c05cb9d0 2024-04-22T13:29:07.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.990+0000 7f8bca7fc640 10 client.17511 handle_command_reply: tid=0 2024-04-22T13:29:07.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.990+0000 7f8bebf63740 1 client.17511 shutdown 2024-04-22T13:29:07.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.990+0000 7f8bebf63740 1 -- 172.21.15.12:0/827948491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5560c05b24d0 msgr2=0x5560c05bb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:07.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.990+0000 7f8bebf63740 1 --2- 172.21.15.12:0/827948491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5560c05b24d0 0x5560c05bb260 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4000c00 tx=0x7f8bd4009260 comp rx=0 tx=0).stop 2024-04-22T13:29:07.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.990+0000 7f8bebf63740 1 -- 172.21.15.12:0/827948491 shutdown_connections 2024-04-22T13:29:07.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.990+0000 7f8bebf63740 1 --2- 172.21.15.12:0/827948491 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5560c05cb9d0 0x5560c05cddc0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.990+0000 7f8bebf63740 1 --2- 172.21.15.12:0/827948491 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5560c05bba00 0x5560c05b8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.990+0000 7f8bebf63740 1 --2- 172.21.15.12:0/827948491 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5560c05b2e80 0x5560c05b85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.990+0000 7f8bebf63740 1 --2- 172.21.15.12:0/827948491 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5560c05b24d0 0x5560c05bb260 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:07.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.990+0000 7f8bebf63740 1 -- 172.21.15.12:0/827948491 >> 172.21.15.12:0/827948491 conn(0x5560c05acba0 msgr2=0x5560c05ae8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:07.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.990+0000 7f8bca7fc640 0 client.17511 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:29:07.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.990+0000 7f8bebf63740 1 -- 172.21.15.12:0/827948491 shutdown_connections 2024-04-22T13:29:07.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.990+0000 7f8bebf63740 1 -- 172.21.15.12:0/827948491 wait complete. 2024-04-22T13:29:07.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:07.990+0000 7f8bebf63740 20 client.17511 trim_cache size 0 max 16384 2024-04-22T13:29:08.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.004+0000 7f8bebf63740 1 Processor -- start 2024-04-22T13:29:08.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.004+0000 7f8bebf63740 1 -- start start 2024-04-22T13:29:08.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.004+0000 7f8bebf63740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5560c05b24d0 0x5560c05b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.004+0000 7f8bebf63740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5560c05b2e80 0x5560c05bad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.004+0000 7f8bebf63740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5560c05bba00 0x5560c05bb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.004+0000 7f8bebf63740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8be41ab890 con 0x5560c05b2e80 2024-04-22T13:29:08.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.004+0000 7f8bebf63740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8be41ad3d0 con 0x5560c05bba00 2024-04-22T13:29:08.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.004+0000 7f8bebf63740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8be41abf60 con 0x5560c05b24d0 2024-04-22T13:29:08.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.004+0000 7f8bdbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5560c05b2e80 0x5560c05bad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.004+0000 7f8bdbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5560c05b2e80 0x5560c05bad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51914/0 (socket says 172.21.15.12:51914) 2024-04-22T13:29:08.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.004+0000 7f8bdbfff640 1 -- 172.21.15.12:0/1603941258 learned_addr learned my addr 172.21.15.12:0/1603941258 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:08.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.004+0000 7f8be2ffd640 1 --2- 172.21.15.12:0/1603941258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5560c05b24d0 0x5560c05b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.005+0000 7f8bdbfff640 1 -- 172.21.15.12:0/1603941258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5560c05b24d0 msgr2=0x5560c05b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.005+0000 7f8bdbfff640 1 --2- 172.21.15.12:0/1603941258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5560c05b24d0 0x5560c05b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.005+0000 7f8bdbfff640 1 -- 172.21.15.12:0/1603941258 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5560c05bba00 msgr2=0x5560c05bb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.005+0000 7f8be37fe640 1 --2- 172.21.15.12:0/1603941258 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5560c05bba00 0x5560c05bb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.005+0000 7f8bdbfff640 1 --2- 172.21.15.12:0/1603941258 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5560c05bba00 0x5560c05bb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.005+0000 7f8bdbfff640 1 -- 172.21.15.12:0/1603941258 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560c05b2c20 con 0x5560c05b2e80 2024-04-22T13:29:08.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.005+0000 7f8bdbfff640 1 --2- 172.21.15.12:0/1603941258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5560c05b2e80 0x5560c05bad20 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc098d00 tx=0x7f8bcc0984b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec010a3c408d6d8f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:08.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.005+0000 7f8bd8ff9640 1 -- 172.21.15.12:0/1603941258 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc0a06e0 con 0x5560c05b2e80 2024-04-22T13:29:08.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.005+0000 7f8bd8ff9640 1 -- 172.21.15.12:0/1603941258 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bcc0a4fa0 con 0x5560c05b2e80 2024-04-22T13:29:08.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.005+0000 7f8bd8ff9640 1 -- 172.21.15.12:0/1603941258 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc0cdd60 con 0x5560c05b2e80 2024-04-22T13:29:08.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.006+0000 7f8bebf63740 1 -- 172.21.15.12:0/1603941258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5560c05b2e80 msgr2=0x5560c05bad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.006+0000 7f8bebf63740 1 --2- 172.21.15.12:0/1603941258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5560c05b2e80 0x5560c05bad20 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc098d00 tx=0x7f8bcc0984b0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.006+0000 7f8bebf63740 1 -- 172.21.15.12:0/1603941258 shutdown_connections 2024-04-22T13:29:08.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.006+0000 7f8bebf63740 1 --2- 172.21.15.12:0/1603941258 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5560c05bba00 0x5560c05bb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.006+0000 7f8bebf63740 1 --2- 172.21.15.12:0/1603941258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5560c05b2e80 0x5560c05bad20 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.006+0000 7f8bebf63740 1 --2- 172.21.15.12:0/1603941258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5560c05b24d0 0x5560c05b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.006+0000 7f8bebf63740 1 -- 172.21.15.12:0/1603941258 >> 172.21.15.12:0/1603941258 conn(0x5560c05acba0 msgr2=0x5560c05abc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:08.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.006+0000 7f8bebf63740 1 -- 172.21.15.12:0/1603941258 shutdown_connections 2024-04-22T13:29:08.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.006+0000 7f8bebf63740 1 -- 172.21.15.12:0/1603941258 wait complete. 2024-04-22T13:29:08.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.006+0000 7f8bebf63740 1 -- start start 2024-04-22T13:29:08.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.006+0000 7f8bebf63740 1 Processor -- start 2024-04-22T13:29:08.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.006+0000 7f8bebf63740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5560c05b24d0 0x5560c05d0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.006+0000 7f8bebf63740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5560c05b2e80 0x5560c05ce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.006+0000 7f8bebf63740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5560c05bba00 0x5560c05ce680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.006+0000 7f8bebf63740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8be41acd00 con 0x5560c05b24d0 2024-04-22T13:29:08.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.006+0000 7f8bebf63740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8be41ac630 con 0x5560c05bba00 2024-04-22T13:29:08.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.006+0000 7f8bebf63740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8be41a9890 con 0x5560c05b2e80 2024-04-22T13:29:08.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.006+0000 7f8bebf63740 10 client.0 _ll_register_callbacks cb 0x7f8bd4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:08.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.006+0000 7f8be2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5560c05b24d0 0x5560c05d0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.006+0000 7f8be2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5560c05b24d0 0x5560c05d0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51924/0 (socket says 172.21.15.12:51924) 2024-04-22T13:29:08.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.006+0000 7f8be2ffd640 1 -- 172.21.15.12:0/1549062371 learned_addr learned my addr 172.21.15.12:0/1549062371 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:08.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.007+0000 7f8bdbfff640 1 --2- 172.21.15.12:0/1549062371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5560c05b2e80 0x5560c05ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.007+0000 7f8be2ffd640 1 -- 172.21.15.12:0/1549062371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5560c05b2e80 msgr2=0x5560c05ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.007+0000 7f8be2ffd640 1 --2- 172.21.15.12:0/1549062371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5560c05b2e80 0x5560c05ce140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.007+0000 7f8be2ffd640 1 -- 172.21.15.12:0/1549062371 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5560c05bba00 msgr2=0x5560c05ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.007+0000 7f8be2ffd640 1 --2- 172.21.15.12:0/1549062371 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5560c05bba00 0x5560c05ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.007+0000 7f8be2ffd640 1 -- 172.21.15.12:0/1549062371 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560c05ceeb0 con 0x5560c05b24d0 2024-04-22T13:29:08.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.007+0000 7f8bdbfff640 1 --2- 172.21.15.12:0/1549062371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5560c05b2e80 0x5560c05ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:08.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.007+0000 7f8be2ffd640 1 --2- 172.21.15.12:0/1549062371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5560c05b24d0 0x5560c05d0db0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4040da0 tx=0x7f8bd402cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e940c5e79203f6c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:08.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.007+0000 7f8bca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:08.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.007+0000 7f8bca7fc640 1 -- 172.21.15.12:0/1549062371 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4056030 con 0x5560c05b24d0 2024-04-22T13:29:08.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.007+0000 7f8bca7fc640 1 -- 172.21.15.12:0/1549062371 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f8bd4040470 con 0x5560c05b24d0 2024-04-22T13:29:08.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.007+0000 7f8bebf63740 1 -- 172.21.15.12:0/1549062371 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bcc0a4fa0 con 0x5560c05b24d0 2024-04-22T13:29:08.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.007+0000 7f8bca7fc640 1 -- 172.21.15.12:0/1549062371 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8bd404d040 con 0x5560c05b24d0 2024-04-22T13:29:08.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.007+0000 7f8bebf63740 10 client.17517 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:08.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.007+0000 7f8bebf63740 1 -- 172.21.15.12:0/1549062371 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5560c0631ba0 con 0x5560c05b24d0 2024-04-22T13:29:08.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.007+0000 7f8bca7fc640 1 -- 172.21.15.12:0/1549062371 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bcc0a4fa0 con 0x5560c05b24d0 2024-04-22T13:29:08.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.007+0000 7f8bca7fc640 1 -- 172.21.15.12:0/1549062371 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8bd4067480 con 0x5560c05b24d0 2024-04-22T13:29:08.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.007+0000 7f8bebf63740 10 client.17517 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:08.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.007+0000 7f8bebf63740 10 client.17517 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:29:08.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.008+0000 7f8bebf63740 20 client.17517 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:08.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.008+0000 7f8bebf63740 1 --2- 172.21.15.12:0/1549062371 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5560c0632240 0x5560c06346a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.008+0000 7f8bebf63740 4 client.17517 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:29:08.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.008+0000 7f8bebf63740 1 -- 172.21.15.12:0/1549062371 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5560c0524c10 con 0x5560c0632240 2024-04-22T13:29:08.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.008+0000 7f8bdbfff640 1 --2- 172.21.15.12:0/1549062371 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5560c0632240 0x5560c06346a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.008+0000 7f8bca7fc640 10 client.17517 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:29:08.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.008+0000 7f8bdbfff640 1 --2- 172.21.15.12:0/1549062371 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5560c0632240 0x5560c06346a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d6eba35f34ba850d server_cookie=2a8200b44ff0feee in_seq=0 out_seq=0 2024-04-22T13:29:08.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.008+0000 7f8bca7fc640 1 -- 172.21.15.12:0/1549062371 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5560c0524c10 con 0x5560c0632240 2024-04-22T13:29:08.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.008+0000 7f8bca7fc640 10 client.17517 handle_command_reply: tid=0 2024-04-22T13:29:08.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.008+0000 7f8bebf63740 1 client.17517 shutdown 2024-04-22T13:29:08.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.008+0000 7f8bebf63740 1 -- 172.21.15.12:0/1549062371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5560c05b24d0 msgr2=0x5560c05d0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.009+0000 7f8bebf63740 1 --2- 172.21.15.12:0/1549062371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5560c05b24d0 0x5560c05d0db0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4040da0 tx=0x7f8bd402cc60 comp rx=0 tx=0).stop 2024-04-22T13:29:08.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.009+0000 7f8bebf63740 1 -- 172.21.15.12:0/1549062371 shutdown_connections 2024-04-22T13:29:08.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.009+0000 7f8bebf63740 1 --2- 172.21.15.12:0/1549062371 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x5560c0632240 0x5560c06346a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.009+0000 7f8bebf63740 1 --2- 172.21.15.12:0/1549062371 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5560c05bba00 0x5560c05ce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.009+0000 7f8bebf63740 1 --2- 172.21.15.12:0/1549062371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5560c05b2e80 0x5560c05ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.009+0000 7f8bca7fc640 0 client.17517 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:29:08.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.009+0000 7f8bebf63740 1 --2- 172.21.15.12:0/1549062371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5560c05b24d0 0x5560c05d0db0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.009+0000 7f8bebf63740 1 -- 172.21.15.12:0/1549062371 >> 172.21.15.12:0/1549062371 conn(0x5560c05acba0 msgr2=0x5560c05c75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:08.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.009+0000 7f8bebf63740 1 -- 172.21.15.12:0/1549062371 shutdown_connections 2024-04-22T13:29:08.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.009+0000 7f8bebf63740 1 -- 172.21.15.12:0/1549062371 wait complete. 2024-04-22T13:29:08.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.009+0000 7f8bebf63740 20 client.17517 trim_cache size 0 max 16384 2024-04-22T13:29:08.011 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:29:08.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.009+0000 7f8be9293640 1 -- 172.21.15.12:0/849775867 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f8bd00a3ec0 msgr2=0x7f8bd00a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.009+0000 7f8be9293640 1 --2- 172.21.15.12:0/849775867 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f8bd00a3ec0 0x7f8bd00a6380 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc009fd0 tx=0x7f8bdc004ab0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.009+0000 7f8be9293640 1 -- 172.21.15.12:0/849775867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8be4106170 msgr2=0x7f8be4199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.009+0000 7f8be9293640 1 --2- 172.21.15.12:0/849775867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8be4106170 0x7f8be4199a60 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00c970 tx=0x7f8bcc00ce30 comp rx=0 tx=0).stop 2024-04-22T13:29:08.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.009+0000 7f8be9293640 1 -- 172.21.15.12:0/849775867 shutdown_connections 2024-04-22T13:29:08.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.009+0000 7f8be9293640 1 --2- 172.21.15.12:0/849775867 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f8bd00a3ec0 0x7f8bd00a6380 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.009+0000 7f8be9293640 1 --2- 172.21.15.12:0/849775867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8be4199ff0 0x7f8be41a45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.009+0000 7f8be9293640 1 --2- 172.21.15.12:0/849775867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8be4106170 0x7f8be4199a60 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.009+0000 7f8be9293640 1 --2- 172.21.15.12:0/849775867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8be41010a0 0x7f8be4199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.009+0000 7f8be9293640 1 -- 172.21.15.12:0/849775867 >> 172.21.15.12:0/849775867 conn(0x7f8be4077eb0 msgr2=0x7f8be40fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:08.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.009+0000 7f8be9293640 1 -- 172.21.15.12:0/849775867 shutdown_connections 2024-04-22T13:29:08.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.009+0000 7f8be9293640 1 -- 172.21.15.12:0/849775867 wait complete. 2024-04-22T13:29:08.014 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:07 smithi012 ceph-mon[28265]: pgmap v153: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:29:08.016 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:29:08.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.124+0000 7f612a843640 1 -- 172.21.15.12:0/502325965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f612406b1f0 msgr2=0x7f6124106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.124+0000 7f612a843640 1 --2- 172.21.15.12:0/502325965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f612406b1f0 0x7f6124106b90 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f6118009900 tx=0x7f611802bc30 comp rx=0 tx=0).stop 2024-04-22T13:29:08.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.124+0000 7f612a843640 1 -- 172.21.15.12:0/502325965 shutdown_connections 2024-04-22T13:29:08.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.124+0000 7f612a843640 1 --2- 172.21.15.12:0/502325965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6124107270 0x7f612410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.124+0000 7f612a843640 1 --2- 172.21.15.12:0/502325965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f612406b1f0 0x7f6124106b90 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.124+0000 7f612a843640 1 --2- 172.21.15.12:0/502325965 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f612406a840 0x7f612406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.124+0000 7f612a843640 1 -- 172.21.15.12:0/502325965 >> 172.21.15.12:0/502325965 conn(0x7f6124077eb0 msgr2=0x7f61240782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:08.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.124+0000 7f612a843640 1 -- 172.21.15.12:0/502325965 shutdown_connections 2024-04-22T13:29:08.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.124+0000 7f612a843640 1 -- 172.21.15.12:0/502325965 wait complete. 2024-04-22T13:29:08.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.125+0000 7f612a843640 1 Processor -- start 2024-04-22T13:29:08.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.125+0000 7f612a843640 1 -- start start 2024-04-22T13:29:08.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.125+0000 7f612a843640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f612406a840 0x7f6124198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.125+0000 7f612a843640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6124107270 0x7f6124198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.125+0000 7f612a843640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f612419cff0 0x7f612419d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.125+0000 7f612a843640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6124112190 con 0x7f6124107270 2024-04-22T13:29:08.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.125+0000 7f612a843640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6124112010 con 0x7f612419cff0 2024-04-22T13:29:08.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.125+0000 7f612a843640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6124112310 con 0x7f612406a840 2024-04-22T13:29:08.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.125+0000 7f61237fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6124107270 0x7f6124198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.125+0000 7f61237fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6124107270 0x7f6124198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51930/0 (socket says 172.21.15.12:51930) 2024-04-22T13:29:08.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.125+0000 7f61237fe640 1 -- 172.21.15.12:0/3978706046 learned_addr learned my addr 172.21.15.12:0/3978706046 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:08.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.125+0000 7f6123fff640 1 --2- 172.21.15.12:0/3978706046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f612406a840 0x7f6124198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.126+0000 7f6128de0640 1 --2- 172.21.15.12:0/3978706046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f612419cff0 0x7f612419d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.126+0000 7f61237fe640 1 -- 172.21.15.12:0/3978706046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f612406a840 msgr2=0x7f6124198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.126+0000 7f61237fe640 1 --2- 172.21.15.12:0/3978706046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f612406a840 0x7f6124198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.126+0000 7f61237fe640 1 -- 172.21.15.12:0/3978706046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f612419cff0 msgr2=0x7f612419d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.126+0000 7f61237fe640 1 --2- 172.21.15.12:0/3978706046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f612419cff0 0x7f612419d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.126+0000 7f61237fe640 1 -- 172.21.15.12:0/3978706046 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612419dae0 con 0x7f6124107270 2024-04-22T13:29:08.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.126+0000 7f6128de0640 1 --2- 172.21.15.12:0/3978706046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f612419cff0 0x7f612419d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:08.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.126+0000 7f61237fe640 1 --2- 172.21.15.12:0/3978706046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6124107270 0x7f6124198c70 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f6118036040 tx=0x7f611802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:08.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.126+0000 7f61217fa640 1 -- 172.21.15.12:0/3978706046 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6118043020 con 0x7f6124107270 2024-04-22T13:29:08.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.126+0000 7f612a843640 1 -- 172.21.15.12:0/3978706046 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f612419dd70 con 0x7f6124107270 2024-04-22T13:29:08.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.126+0000 7f61217fa640 1 -- 172.21.15.12:0/3978706046 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f611802cbe0 con 0x7f6124107270 2024-04-22T13:29:08.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.126+0000 7f61217fa640 1 -- 172.21.15.12:0/3978706046 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6118003bf0 con 0x7f6124107270 2024-04-22T13:29:08.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.126+0000 7f612a843640 1 -- 172.21.15.12:0/3978706046 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61241a5590 con 0x7f6124107270 2024-04-22T13:29:08.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.128+0000 7f61217fa640 1 -- 172.21.15.12:0/3978706046 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6118046020 con 0x7f6124107270 2024-04-22T13:29:08.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.128+0000 7f612d438740 1 Processor -- start 2024-04-22T13:29:08.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.128+0000 7f612d438740 1 -- start start 2024-04-22T13:29:08.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.129+0000 7f612d438740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe3cd7e4d0 0x55fe3cd7e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.129+0000 7f612d438740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe3cd7ee80 0x55fe3cd87430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.129+0000 7f612d438740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe3cd87a00 0x55fe3cd89e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.129+0000 7f612d438740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fe3cce31f0 con 0x55fe3cd7ee80 2024-04-22T13:29:08.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.129+0000 7f612d438740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fe3cc804b0 con 0x55fe3cd7e4d0 2024-04-22T13:29:08.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.129+0000 7f612d438740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fe3ccdf1b0 con 0x55fe3cd87a00 2024-04-22T13:29:08.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.129+0000 7f6123fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe3cd7ee80 0x55fe3cd87430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.129+0000 7f6123fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe3cd7ee80 0x55fe3cd87430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51942/0 (socket says 172.21.15.12:51942) 2024-04-22T13:29:08.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.129+0000 7f6123fff640 1 -- 172.21.15.12:0/721335592 learned_addr learned my addr 172.21.15.12:0/721335592 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:08.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.129+0000 7f6128de0640 1 --2- 172.21.15.12:0/721335592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe3cd7e4d0 0x55fe3cd7e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.129+0000 7f61237fe640 1 --2- 172.21.15.12:0/721335592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe3cd87a00 0x55fe3cd89e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.129+0000 7f61217fa640 1 --2- 172.21.15.12:0/3978706046 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f610c0a3d20 0x7f610c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.129+0000 7f6123fff640 1 -- 172.21.15.12:0/721335592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe3cd87a00 msgr2=0x55fe3cd89e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.129+0000 7f6123fff640 1 --2- 172.21.15.12:0/721335592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe3cd87a00 0x55fe3cd89e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.129+0000 7f6123fff640 1 -- 172.21.15.12:0/721335592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe3cd7e4d0 msgr2=0x55fe3cd7e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.129+0000 7f6123fff640 1 --2- 172.21.15.12:0/721335592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe3cd7e4d0 0x55fe3cd7e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.129+0000 7f6123fff640 1 -- 172.21.15.12:0/721335592 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe3cc721b0 con 0x55fe3cd7ee80 2024-04-22T13:29:08.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.129+0000 7f6128de0640 1 --2- 172.21.15.12:0/3978706046 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f610c0a3d20 0x7f610c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.129+0000 7f61217fa640 1 -- 172.21.15.12:0/3978706046 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f61180f28e0 con 0x7f6124107270 2024-04-22T13:29:08.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.129+0000 7f6123fff640 1 --2- 172.21.15.12:0/721335592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe3cd7ee80 0x55fe3cd87430 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f611400bd10 tx=0x7f61140107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21984bc024c9cb42 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:08.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.130+0000 7f610a7fc640 1 -- 172.21.15.12:0/721335592 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6114004030 con 0x55fe3cd7ee80 2024-04-22T13:29:08.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.130+0000 7f610a7fc640 1 -- 172.21.15.12:0/721335592 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61140035f0 con 0x55fe3cd7ee80 2024-04-22T13:29:08.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.130+0000 7f610a7fc640 1 -- 172.21.15.12:0/721335592 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6114011750 con 0x55fe3cd7ee80 2024-04-22T13:29:08.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.130+0000 7f612d438740 1 -- 172.21.15.12:0/721335592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe3cd7ee80 msgr2=0x55fe3cd87430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.130+0000 7f6128de0640 1 --2- 172.21.15.12:0/3978706046 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f610c0a3d20 0x7f610c0a61e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f611c009370 tx=0x7f611c004a40 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:08.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.130+0000 7f612d438740 1 --2- 172.21.15.12:0/721335592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe3cd7ee80 0x55fe3cd87430 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f611400bd10 tx=0x7f61140107a0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.130+0000 7f612d438740 1 -- 172.21.15.12:0/721335592 shutdown_connections 2024-04-22T13:29:08.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.130+0000 7f612d438740 1 --2- 172.21.15.12:0/721335592 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe3cd87a00 0x55fe3cd89e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.130+0000 7f612d438740 1 --2- 172.21.15.12:0/721335592 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe3cd7ee80 0x55fe3cd87430 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.130+0000 7f612d438740 1 --2- 172.21.15.12:0/721335592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe3cd7e4d0 0x55fe3cd7e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.130+0000 7f612d438740 1 -- 172.21.15.12:0/721335592 >> 172.21.15.12:0/721335592 conn(0x55fe3cd78ba0 msgr2=0x55fe3cd78f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:08.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.130+0000 7f612d438740 1 -- 172.21.15.12:0/721335592 shutdown_connections 2024-04-22T13:29:08.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.130+0000 7f612d438740 1 -- 172.21.15.12:0/721335592 wait complete. 2024-04-22T13:29:08.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.131+0000 7f612d438740 1 -- start start 2024-04-22T13:29:08.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.131+0000 7f612d438740 1 Processor -- start 2024-04-22T13:29:08.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.131+0000 7f612d438740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe3cd7e4d0 0x55fe3cd87300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.131+0000 7f612d438740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe3cd7ee80 0x55fe3cd84690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.131+0000 7f612d438740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe3cd87a00 0x55fe3cd84c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.131+0000 7f612d438740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fe3cd8a970 con 0x55fe3cd7e4d0 2024-04-22T13:29:08.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.131+0000 7f612d438740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fe3cc9c500 con 0x55fe3cd7ee80 2024-04-22T13:29:08.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.131+0000 7f612d438740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fe3cd8aaf0 con 0x55fe3cd87a00 2024-04-22T13:29:08.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.131+0000 7f6123fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe3cd7e4d0 0x55fe3cd87300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.131+0000 7f612d438740 10 client.0 _ll_register_callbacks cb 0x55fe3cc6ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:08.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.131+0000 7f6123fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe3cd7e4d0 0x55fe3cd87300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51958/0 (socket says 172.21.15.12:51958) 2024-04-22T13:29:08.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.131+0000 7f6123fff640 1 -- 172.21.15.12:0/1820747905 learned_addr learned my addr 172.21.15.12:0/1820747905 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:08.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.131+0000 7f61237fe640 1 --2- 172.21.15.12:0/1820747905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe3cd7ee80 0x55fe3cd84690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.131+0000 7f6128de0640 1 --2- 172.21.15.12:0/1820747905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe3cd87a00 0x55fe3cd84c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.132+0000 7f6123fff640 1 -- 172.21.15.12:0/1820747905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe3cd87a00 msgr2=0x55fe3cd84c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.132+0000 7f6123fff640 1 --2- 172.21.15.12:0/1820747905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe3cd87a00 0x55fe3cd84c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.132+0000 7f6123fff640 1 -- 172.21.15.12:0/1820747905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe3cd7ee80 msgr2=0x55fe3cd84690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.132+0000 7f6123fff640 1 --2- 172.21.15.12:0/1820747905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe3cd7ee80 0x55fe3cd84690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.132+0000 7f6123fff640 1 -- 172.21.15.12:0/1820747905 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe3c9f53f0 con 0x55fe3cd7e4d0 2024-04-22T13:29:08.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.132+0000 7f6128de0640 1 --2- 172.21.15.12:0/1820747905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe3cd87a00 0x55fe3cd84c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:08.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.132+0000 7f6123fff640 1 --2- 172.21.15.12:0/1820747905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe3cd7e4d0 0x55fe3cd87300 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f611401a000 tx=0x7f611400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195ea5e8c4428c07 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:08.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.132+0000 7f61037fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:08.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.132+0000 7f61037fe640 1 -- 172.21.15.12:0/1820747905 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6114029060 con 0x55fe3cd7e4d0 2024-04-22T13:29:08.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.132+0000 7f61037fe640 1 -- 172.21.15.12:0/1820747905 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f6114003a70 con 0x55fe3cd7e4d0 2024-04-22T13:29:08.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.132+0000 7f612d438740 1 -- 172.21.15.12:0/1820747905 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61140035f0 con 0x55fe3cd7e4d0 2024-04-22T13:29:08.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.132+0000 7f61037fe640 1 -- 172.21.15.12:0/1820747905 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6114004000 con 0x55fe3cd7e4d0 2024-04-22T13:29:08.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.132+0000 7f612d438740 10 client.17529 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:08.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.132+0000 7f612d438740 1 -- 172.21.15.12:0/1820747905 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fe3cabf7d0 con 0x55fe3cd7e4d0 2024-04-22T13:29:08.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.132+0000 7f61037fe640 1 -- 172.21.15.12:0/1820747905 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61140035f0 con 0x55fe3cd7e4d0 2024-04-22T13:29:08.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.132+0000 7f61037fe640 1 -- 172.21.15.12:0/1820747905 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6114004770 con 0x55fe3cd7e4d0 2024-04-22T13:29:08.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.133+0000 7f612d438740 10 client.17529 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:08.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.133+0000 7f612d438740 10 client.17529 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:29:08.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.133+0000 7f612d438740 20 client.17529 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:08.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.133+0000 7f612d438740 1 --2- 172.21.15.12:0/1820747905 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55fe3cd97b60 0x55fe3cd99f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.133+0000 7f612d438740 4 client.17529 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:08.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.133+0000 7f612d438740 1 -- 172.21.15.12:0/1820747905 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe3cc82930 con 0x55fe3cd97b60 2024-04-22T13:29:08.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.133+0000 7f61237fe640 1 --2- 172.21.15.12:0/1820747905 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55fe3cd97b60 0x55fe3cd99f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.133+0000 7f61237fe640 1 --2- 172.21.15.12:0/1820747905 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55fe3cd97b60 0x55fe3cd99f50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3a8cfd0d123f8c1d server_cookie=f35f40509775948b in_seq=0 out_seq=0 2024-04-22T13:29:08.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.133+0000 7f61037fe640 10 client.17529 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:29:08.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.134+0000 7f61037fe640 1 -- 172.21.15.12:0/1820747905 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fe3cc82930 con 0x55fe3cd97b60 2024-04-22T13:29:08.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.134+0000 7f61037fe640 10 client.17529 handle_command_reply: tid=0 2024-04-22T13:29:08.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.134+0000 7f612d438740 1 client.17529 shutdown 2024-04-22T13:29:08.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.134+0000 7f612d438740 1 -- 172.21.15.12:0/1820747905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe3cd7e4d0 msgr2=0x55fe3cd87300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.134+0000 7f612d438740 1 --2- 172.21.15.12:0/1820747905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe3cd7e4d0 0x55fe3cd87300 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f611401a000 tx=0x7f611400b8e0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.134+0000 7f612d438740 1 -- 172.21.15.12:0/1820747905 shutdown_connections 2024-04-22T13:29:08.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.134+0000 7f612d438740 1 --2- 172.21.15.12:0/1820747905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe3cd87a00 0x55fe3cd84c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.134+0000 7f612d438740 1 --2- 172.21.15.12:0/1820747905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe3cd7ee80 0x55fe3cd84690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.134+0000 7f612d438740 1 --2- 172.21.15.12:0/1820747905 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55fe3cd97b60 0x55fe3cd99f50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.134+0000 7f612d438740 1 --2- 172.21.15.12:0/1820747905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe3cd7e4d0 0x55fe3cd87300 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.134+0000 7f61037fe640 0 client.17529 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:29:08.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.134+0000 7f612d438740 1 -- 172.21.15.12:0/1820747905 >> 172.21.15.12:0/1820747905 conn(0x55fe3cd78ba0 msgr2=0x55fe3cd7a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:08.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.134+0000 7f612d438740 1 -- 172.21.15.12:0/1820747905 shutdown_connections 2024-04-22T13:29:08.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.134+0000 7f612d438740 1 -- 172.21.15.12:0/1820747905 wait complete. 2024-04-22T13:29:08.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.134+0000 7f612d438740 20 client.17529 trim_cache size 0 max 16384 2024-04-22T13:29:08.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.147+0000 7f612d438740 1 Processor -- start 2024-04-22T13:29:08.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.148+0000 7f612d438740 1 -- start start 2024-04-22T13:29:08.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.148+0000 7f612d438740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe3cd7e4d0 0x55fe3cd86dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.148+0000 7f612d438740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe3cd7ee80 0x55fe3cd87300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.148+0000 7f612d438740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe3cd87a00 0x55fe3cd9cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.148+0000 7f612d438740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f61241adb90 con 0x55fe3cd7e4d0 2024-04-22T13:29:08.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.148+0000 7f612d438740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f61241af6d0 con 0x55fe3cd87a00 2024-04-22T13:29:08.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.148+0000 7f612d438740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f61241ae260 con 0x55fe3cd7ee80 2024-04-22T13:29:08.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.148+0000 7f6123fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe3cd7e4d0 0x55fe3cd86dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.149 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.148+0000 7f6123fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe3cd7e4d0 0x55fe3cd86dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51966/0 (socket says 172.21.15.12:51966) 2024-04-22T13:29:08.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.148+0000 7f6123fff640 1 -- 172.21.15.12:0/2371293314 learned_addr learned my addr 172.21.15.12:0/2371293314 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:08.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.148+0000 7f61237fe640 1 --2- 172.21.15.12:0/2371293314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe3cd7ee80 0x55fe3cd87300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.148+0000 7f6123fff640 1 -- 172.21.15.12:0/2371293314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe3cd7ee80 msgr2=0x55fe3cd87300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.148+0000 7f6123fff640 1 --2- 172.21.15.12:0/2371293314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe3cd7ee80 0x55fe3cd87300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.148+0000 7f6123fff640 1 -- 172.21.15.12:0/2371293314 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe3cd87a00 msgr2=0x55fe3cd9cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.148+0000 7f6123fff640 1 --2- 172.21.15.12:0/2371293314 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe3cd87a00 0x55fe3cd9cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.148+0000 7f6123fff640 1 -- 172.21.15.12:0/2371293314 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe3cd84f80 con 0x55fe3cd7e4d0 2024-04-22T13:29:08.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.148+0000 7f61237fe640 1 --2- 172.21.15.12:0/2371293314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe3cd7ee80 0x55fe3cd87300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:08.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.148+0000 7f6123fff640 1 --2- 172.21.15.12:0/2371293314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe3cd7e4d0 0x55fe3cd86dc0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f61140107e0 tx=0x7f6114010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e41857ec6430196 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:08.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.148+0000 7f6109ffb640 1 -- 172.21.15.12:0/2371293314 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6114029070 con 0x55fe3cd7e4d0 2024-04-22T13:29:08.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.148+0000 7f6109ffb640 1 -- 172.21.15.12:0/2371293314 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61140035f0 con 0x55fe3cd7e4d0 2024-04-22T13:29:08.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.148+0000 7f6109ffb640 1 -- 172.21.15.12:0/2371293314 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61140041a0 con 0x55fe3cd7e4d0 2024-04-22T13:29:08.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.149+0000 7f612d438740 1 -- 172.21.15.12:0/2371293314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe3cd7e4d0 msgr2=0x55fe3cd86dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.149+0000 7f612d438740 1 --2- 172.21.15.12:0/2371293314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe3cd7e4d0 0x55fe3cd86dc0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f61140107e0 tx=0x7f6114010dd0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.149+0000 7f612d438740 1 -- 172.21.15.12:0/2371293314 shutdown_connections 2024-04-22T13:29:08.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.149+0000 7f612d438740 1 --2- 172.21.15.12:0/2371293314 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe3cd87a00 0x55fe3cd9cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.149+0000 7f612d438740 1 --2- 172.21.15.12:0/2371293314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe3cd7ee80 0x55fe3cd87300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.149+0000 7f612d438740 1 --2- 172.21.15.12:0/2371293314 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe3cd7e4d0 0x55fe3cd86dc0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.149+0000 7f612d438740 1 -- 172.21.15.12:0/2371293314 >> 172.21.15.12:0/2371293314 conn(0x55fe3cd78ba0 msgr2=0x55fe3cd77c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:08.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.149+0000 7f612d438740 1 -- 172.21.15.12:0/2371293314 shutdown_connections 2024-04-22T13:29:08.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.149+0000 7f612d438740 1 -- 172.21.15.12:0/2371293314 wait complete. 2024-04-22T13:29:08.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.149+0000 7f612d438740 1 -- start start 2024-04-22T13:29:08.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.150+0000 7f612d438740 1 Processor -- start 2024-04-22T13:29:08.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.150+0000 7f612d438740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe3cd7e4d0 0x55fe3cd9d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.150+0000 7f612d438740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe3cd7ee80 0x55fe3cd9a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.150+0000 7f612d438740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe3cd87a00 0x55fe3cd9ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.150+0000 7f612d438740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f61241af000 con 0x55fe3cd7ee80 2024-04-22T13:29:08.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.150+0000 7f612d438740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f61241ae930 con 0x55fe3cd87a00 2024-04-22T13:29:08.151 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.150+0000 7f612d438740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f61241abb90 con 0x55fe3cd7e4d0 2024-04-22T13:29:08.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.150+0000 7f612d438740 10 client.0 _ll_register_callbacks cb 0x7f6114003d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:08.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.150+0000 7f61237fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe3cd7ee80 0x55fe3cd9a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.150+0000 7f61237fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe3cd7ee80 0x55fe3cd9a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51974/0 (socket says 172.21.15.12:51974) 2024-04-22T13:29:08.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.150+0000 7f61237fe640 1 -- 172.21.15.12:0/2694363643 learned_addr learned my addr 172.21.15.12:0/2694363643 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:08.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.150+0000 7f6128de0640 1 --2- 172.21.15.12:0/2694363643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe3cd87a00 0x55fe3cd9ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.150+0000 7f6123fff640 1 --2- 172.21.15.12:0/2694363643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe3cd7e4d0 0x55fe3cd9d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.150+0000 7f61237fe640 1 -- 172.21.15.12:0/2694363643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe3cd7e4d0 msgr2=0x55fe3cd9d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.150+0000 7f61237fe640 1 --2- 172.21.15.12:0/2694363643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe3cd7e4d0 0x55fe3cd9d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.150+0000 7f61237fe640 1 -- 172.21.15.12:0/2694363643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe3cd87a00 msgr2=0x55fe3cd9ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.150+0000 7f61237fe640 1 --2- 172.21.15.12:0/2694363643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe3cd87a00 0x55fe3cd9ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.150+0000 7f61237fe640 1 -- 172.21.15.12:0/2694363643 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe3cd9b1b0 con 0x55fe3cd7ee80 2024-04-22T13:29:08.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.150+0000 7f6128de0640 1 --2- 172.21.15.12:0/2694363643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe3cd87a00 0x55fe3cd9ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:08.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.150+0000 7f61237fe640 1 --2- 172.21.15.12:0/2694363643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe3cd7ee80 0x55fe3cd9a5d0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f6118005ec0 tx=0x7f6118000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c998fa038e7923 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:08.152 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.150+0000 7f61037fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:08.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.150+0000 7f61037fe640 1 -- 172.21.15.12:0/2694363643 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61180f4e60 con 0x55fe3cd7ee80 2024-04-22T13:29:08.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.151+0000 7f61037fe640 1 -- 172.21.15.12:0/2694363643 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f61180054d0 con 0x55fe3cd7ee80 2024-04-22T13:29:08.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.151+0000 7f612d438740 1 -- 172.21.15.12:0/2694363643 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61140035f0 con 0x55fe3cd7ee80 2024-04-22T13:29:08.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.151+0000 7f61037fe640 1 -- 172.21.15.12:0/2694363643 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f61180bf0e0 con 0x55fe3cd7ee80 2024-04-22T13:29:08.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.151+0000 7f612d438740 10 client.17535 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:08.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.151+0000 7f612d438740 1 -- 172.21.15.12:0/2694363643 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fe3cdfdbc0 con 0x55fe3cd7ee80 2024-04-22T13:29:08.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.151+0000 7f61037fe640 1 -- 172.21.15.12:0/2694363643 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61140035f0 con 0x55fe3cd7ee80 2024-04-22T13:29:08.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.151+0000 7f61037fe640 1 -- 172.21.15.12:0/2694363643 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f61180bb050 con 0x55fe3cd7ee80 2024-04-22T13:29:08.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.151+0000 7f612d438740 10 client.17535 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:08.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.151+0000 7f612d438740 10 client.17535 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:29:08.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.151+0000 7f612d438740 20 client.17535 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:08.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.151+0000 7f612d438740 1 --2- 172.21.15.12:0/2694363643 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55fe3cdfe2a0 0x55fe3ce006c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.151+0000 7f612d438740 4 client.17535 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:29:08.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.151+0000 7f612d438740 1 -- 172.21.15.12:0/2694363643 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fe3ccf0c10 con 0x55fe3cdfe2a0 2024-04-22T13:29:08.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.151+0000 7f6123fff640 1 --2- 172.21.15.12:0/2694363643 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55fe3cdfe2a0 0x55fe3ce006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.152+0000 7f6123fff640 1 --2- 172.21.15.12:0/2694363643 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55fe3cdfe2a0 0x55fe3ce006c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c31b951c0349c6b7 server_cookie=577b4737df159e85 in_seq=0 out_seq=0 2024-04-22T13:29:08.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.152+0000 7f61037fe640 10 client.17535 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:29:08.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.152+0000 7f61037fe640 1 -- 172.21.15.12:0/2694363643 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fe3ccf0c10 con 0x55fe3cdfe2a0 2024-04-22T13:29:08.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.152+0000 7f61037fe640 10 client.17535 handle_command_reply: tid=0 2024-04-22T13:29:08.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.152+0000 7f612d438740 1 client.17535 shutdown 2024-04-22T13:29:08.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.152+0000 7f612d438740 1 -- 172.21.15.12:0/2694363643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe3cd7ee80 msgr2=0x55fe3cd9a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.152+0000 7f612d438740 1 --2- 172.21.15.12:0/2694363643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe3cd7ee80 0x55fe3cd9a5d0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f6118005ec0 tx=0x7f6118000bc0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.152+0000 7f612d438740 1 -- 172.21.15.12:0/2694363643 shutdown_connections 2024-04-22T13:29:08.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.152+0000 7f612d438740 1 --2- 172.21.15.12:0/2694363643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fe3cd87a00 0x55fe3cd9ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.152+0000 7f612d438740 1 --2- 172.21.15.12:0/2694363643 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55fe3cdfe2a0 0x55fe3ce006c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.152+0000 7f612d438740 1 --2- 172.21.15.12:0/2694363643 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fe3cd7ee80 0x55fe3cd9a5d0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.152+0000 7f612d438740 1 --2- 172.21.15.12:0/2694363643 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe3cd7e4d0 0x55fe3cd9d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.152+0000 7f61037fe640 0 client.17535 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:29:08.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.152+0000 7f612d438740 1 -- 172.21.15.12:0/2694363643 >> 172.21.15.12:0/2694363643 conn(0x55fe3cd78ba0 msgr2=0x55fe3cd77370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:08.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.152+0000 7f612d438740 1 -- 172.21.15.12:0/2694363643 shutdown_connections 2024-04-22T13:29:08.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.152+0000 7f612d438740 1 -- 172.21.15.12:0/2694363643 wait complete. 2024-04-22T13:29:08.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.152+0000 7f612d438740 20 client.17535 trim_cache size 0 max 16384 2024-04-22T13:29:08.155 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:29:08.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.153+0000 7f612a843640 1 -- 172.21.15.12:0/3978706046 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f610c0a3d20 msgr2=0x7f610c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.153+0000 7f612a843640 1 --2- 172.21.15.12:0/3978706046 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f610c0a3d20 0x7f610c0a61e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f611c009370 tx=0x7f611c004a40 comp rx=0 tx=0).stop 2024-04-22T13:29:08.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.153+0000 7f612a843640 1 -- 172.21.15.12:0/3978706046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6124107270 msgr2=0x7f6124198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.153+0000 7f612a843640 1 --2- 172.21.15.12:0/3978706046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6124107270 0x7f6124198c70 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f6118036040 tx=0x7f611802c7e0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.153+0000 7f612a843640 1 -- 172.21.15.12:0/3978706046 shutdown_connections 2024-04-22T13:29:08.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.153+0000 7f612a843640 1 --2- 172.21.15.12:0/3978706046 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f610c0a3d20 0x7f610c0a61e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.153+0000 7f612a843640 1 --2- 172.21.15.12:0/3978706046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f612419cff0 0x7f612419d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.153+0000 7f612a843640 1 --2- 172.21.15.12:0/3978706046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6124107270 0x7f6124198c70 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.153+0000 7f612a843640 1 --2- 172.21.15.12:0/3978706046 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f612406a840 0x7f6124198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.153+0000 7f612a843640 1 -- 172.21.15.12:0/3978706046 >> 172.21.15.12:0/3978706046 conn(0x7f6124077eb0 msgr2=0x7f6124108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:08.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.153+0000 7f612a843640 1 -- 172.21.15.12:0/3978706046 shutdown_connections 2024-04-22T13:29:08.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.153+0000 7f612a843640 1 -- 172.21.15.12:0/3978706046 wait complete. 2024-04-22T13:29:08.159 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:29:08.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.269+0000 7fa8ee952640 1 -- 172.21.15.12:0/3349220028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8e8101a50 msgr2=0x7fa8e8105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.269+0000 7fa8ee952640 1 --2- 172.21.15.12:0/3349220028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8e8101a50 0x7fa8e8105a90 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc009900 tx=0x7fa8dc02bc30 comp rx=0 tx=0).stop 2024-04-22T13:29:08.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.269+0000 7fa8ee952640 1 -- 172.21.15.12:0/3349220028 shutdown_connections 2024-04-22T13:29:08.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.269+0000 7fa8ee952640 1 --2- 172.21.15.12:0/3349220028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8e8106170 0x7fa8e810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.269+0000 7fa8ee952640 1 --2- 172.21.15.12:0/3349220028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8e8101a50 0x7fa8e8105a90 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.269+0000 7fa8ee952640 1 --2- 172.21.15.12:0/3349220028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8e81010a0 0x7fa8e8101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.269+0000 7fa8ee952640 1 -- 172.21.15.12:0/3349220028 >> 172.21.15.12:0/3349220028 conn(0x7fa8e8077eb0 msgr2=0x7fa8e80782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:08.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.269+0000 7fa8ee952640 1 -- 172.21.15.12:0/3349220028 shutdown_connections 2024-04-22T13:29:08.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.269+0000 7fa8ee952640 1 -- 172.21.15.12:0/3349220028 wait complete. 2024-04-22T13:29:08.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.269+0000 7fa8ee952640 1 Processor -- start 2024-04-22T13:29:08.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.269+0000 7fa8ee952640 1 -- start start 2024-04-22T13:29:08.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.270+0000 7fa8ee952640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8e81010a0 0x7fa8e8198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.270+0000 7fa8ee952640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8e8106170 0x7fa8e8198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.270+0000 7fa8ee952640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8e819cfd0 0x7fa8e819d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.270+0000 7fa8ee952640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e81121f0 con 0x7fa8e8106170 2024-04-22T13:29:08.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.270+0000 7fa8ee952640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e8112070 con 0x7fa8e819cfd0 2024-04-22T13:29:08.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.270+0000 7fa8ee952640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e8112370 con 0x7fa8e81010a0 2024-04-22T13:29:08.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.270+0000 7fa8e77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8e8106170 0x7fa8e8198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.270+0000 7fa8e77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8e8106170 0x7fa8e8198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51998/0 (socket says 172.21.15.12:51998) 2024-04-22T13:29:08.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.270+0000 7fa8e77fe640 1 -- 172.21.15.12:0/2725829036 learned_addr learned my addr 172.21.15.12:0/2725829036 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:08.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.270+0000 7fa8e7fff640 1 --2- 172.21.15.12:0/2725829036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8e81010a0 0x7fa8e8198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.270+0000 7fa8eceef640 1 --2- 172.21.15.12:0/2725829036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8e819cfd0 0x7fa8e819d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.270+0000 7fa8e77fe640 1 -- 172.21.15.12:0/2725829036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8e81010a0 msgr2=0x7fa8e8198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.270+0000 7fa8e77fe640 1 --2- 172.21.15.12:0/2725829036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8e81010a0 0x7fa8e8198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.270+0000 7fa8e77fe640 1 -- 172.21.15.12:0/2725829036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8e819cfd0 msgr2=0x7fa8e819d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.270+0000 7fa8e77fe640 1 --2- 172.21.15.12:0/2725829036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8e819cfd0 0x7fa8e819d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.270+0000 7fa8e77fe640 1 -- 172.21.15.12:0/2725829036 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e819dac0 con 0x7fa8e8106170 2024-04-22T13:29:08.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.270+0000 7fa8e77fe640 1 --2- 172.21.15.12:0/2725829036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8e8106170 0x7fa8e8198c50 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc036040 tx=0x7fa8dc02c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:08.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.271+0000 7fa8e57fa640 1 -- 172.21.15.12:0/2725829036 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc043020 con 0x7fa8e8106170 2024-04-22T13:29:08.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.271+0000 7fa8ee952640 1 -- 172.21.15.12:0/2725829036 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8e819dd50 con 0x7fa8e8106170 2024-04-22T13:29:08.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.271+0000 7fa8e57fa640 1 -- 172.21.15.12:0/2725829036 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8dc02cbe0 con 0x7fa8e8106170 2024-04-22T13:29:08.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.271+0000 7fa8e57fa640 1 -- 172.21.15.12:0/2725829036 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc003bf0 con 0x7fa8e8106170 2024-04-22T13:29:08.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.271+0000 7fa8ee952640 1 -- 172.21.15.12:0/2725829036 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8e81a5550 con 0x7fa8e8106170 2024-04-22T13:29:08.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.274+0000 7fa8e57fa640 1 -- 172.21.15.12:0/2725829036 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa8dc046020 con 0x7fa8e8106170 2024-04-22T13:29:08.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.274+0000 7fa8f1625740 1 Processor -- start 2024-04-22T13:29:08.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.274+0000 7fa8f1625740 1 -- start start 2024-04-22T13:29:08.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.274+0000 7fa8f1625740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561bfb2324d0 0x561bfb2328b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.275 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.274+0000 7fa8f1625740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bfb232e80 0x561bfb23b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.274+0000 7fa8f1625740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561bfb23ba00 0x561bfb23de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.274+0000 7fa8f1625740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561bfb1971f0 con 0x561bfb23ba00 2024-04-22T13:29:08.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.274+0000 7fa8f1625740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561bfb1344b0 con 0x561bfb2324d0 2024-04-22T13:29:08.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.274+0000 7fa8f1625740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561bfb1931b0 con 0x561bfb232e80 2024-04-22T13:29:08.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.274+0000 7fa8e77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561bfb23ba00 0x561bfb23de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.274+0000 7fa8e77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561bfb23ba00 0x561bfb23de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52014/0 (socket says 172.21.15.12:52014) 2024-04-22T13:29:08.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.274+0000 7fa8e77fe640 1 -- 172.21.15.12:0/3699841502 learned_addr learned my addr 172.21.15.12:0/3699841502 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:08.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.274+0000 7fa8eceef640 1 --2- 172.21.15.12:0/3699841502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561bfb2324d0 0x561bfb2328b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.274+0000 7fa8e7fff640 1 --2- 172.21.15.12:0/3699841502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bfb232e80 0x561bfb23b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.274+0000 7fa8e57fa640 1 --2- 172.21.15.12:0/2725829036 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa8d00a3d20 0x7fa8d00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.274+0000 7fa8e77fe640 1 -- 172.21.15.12:0/3699841502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bfb232e80 msgr2=0x561bfb23b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.274+0000 7fa8e77fe640 1 --2- 172.21.15.12:0/3699841502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bfb232e80 0x561bfb23b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.274+0000 7fa8e77fe640 1 -- 172.21.15.12:0/3699841502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561bfb2324d0 msgr2=0x561bfb2328b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.274+0000 7fa8e77fe640 1 --2- 172.21.15.12:0/3699841502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561bfb2324d0 0x561bfb2328b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.274+0000 7fa8e77fe640 1 -- 172.21.15.12:0/3699841502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bfb1261b0 con 0x561bfb23ba00 2024-04-22T13:29:08.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.274+0000 7fa8e7fff640 1 --2- 172.21.15.12:0/3699841502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bfb232e80 0x561bfb23b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:08.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.275+0000 7fa8eceef640 1 --2- 172.21.15.12:0/2725829036 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa8d00a3d20 0x7fa8d00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.275+0000 7fa8e77fe640 1 --2- 172.21.15.12:0/3699841502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561bfb23ba00 0x561bfb23de50 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc0f0130 tx=0x7fa8dc0baae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d9babdd180407f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:08.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.275+0000 7fa8e57fa640 1 -- 172.21.15.12:0/2725829036 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fa8dc0f2910 con 0x7fa8e8106170 2024-04-22T13:29:08.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.275+0000 7fa8ce7fc640 1 -- 172.21.15.12:0/3699841502 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc0be7e0 con 0x561bfb23ba00 2024-04-22T13:29:08.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.275+0000 7fa8ce7fc640 1 -- 172.21.15.12:0/3699841502 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8dc0df070 con 0x561bfb23ba00 2024-04-22T13:29:08.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.275+0000 7fa8ce7fc640 1 -- 172.21.15.12:0/3699841502 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc0be370 con 0x561bfb23ba00 2024-04-22T13:29:08.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.276+0000 7fa8f1625740 1 -- 172.21.15.12:0/3699841502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561bfb23ba00 msgr2=0x561bfb23de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.276+0000 7fa8eceef640 1 --2- 172.21.15.12:0/2725829036 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa8d00a3d20 0x7fa8d00a61e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0005fd0 tx=0x7fa8e0004a40 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:08.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.276+0000 7fa8f1625740 1 --2- 172.21.15.12:0/3699841502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561bfb23ba00 0x561bfb23de50 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc0f0130 tx=0x7fa8dc0baae0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.276+0000 7fa8f1625740 1 -- 172.21.15.12:0/3699841502 shutdown_connections 2024-04-22T13:29:08.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.276+0000 7fa8f1625740 1 --2- 172.21.15.12:0/3699841502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561bfb23ba00 0x561bfb23de50 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.276+0000 7fa8f1625740 1 --2- 172.21.15.12:0/3699841502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bfb232e80 0x561bfb23b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.276+0000 7fa8f1625740 1 --2- 172.21.15.12:0/3699841502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561bfb2324d0 0x561bfb2328b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.276+0000 7fa8f1625740 1 -- 172.21.15.12:0/3699841502 >> 172.21.15.12:0/3699841502 conn(0x561bfb22cba0 msgr2=0x561bfb22cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:08.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.276+0000 7fa8f1625740 1 -- 172.21.15.12:0/3699841502 shutdown_connections 2024-04-22T13:29:08.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.276+0000 7fa8f1625740 1 -- 172.21.15.12:0/3699841502 wait complete. 2024-04-22T13:29:08.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.276+0000 7fa8f1625740 1 -- start start 2024-04-22T13:29:08.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.276+0000 7fa8f1625740 1 Processor -- start 2024-04-22T13:29:08.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.276+0000 7fa8f1625740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561bfb2324d0 0x561bfb23b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.276+0000 7fa8f1625740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bfb232e80 0x561bfb238670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.276+0000 7fa8f1625740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561bfb23ba00 0x561bfb238bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.276+0000 7fa8f1625740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561bfb23e730 con 0x561bfb23ba00 2024-04-22T13:29:08.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.276+0000 7fa8f1625740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561bfb150500 con 0x561bfb2324d0 2024-04-22T13:29:08.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.276+0000 7fa8f1625740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561bfb23e8b0 con 0x561bfb232e80 2024-04-22T13:29:08.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.276+0000 7fa8f1625740 10 client.0 _ll_register_callbacks cb 0x561bfb11eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:08.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.277+0000 7fa8eceef640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561bfb23ba00 0x561bfb238bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.277+0000 7fa8eceef640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561bfb23ba00 0x561bfb238bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52022/0 (socket says 172.21.15.12:52022) 2024-04-22T13:29:08.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.277+0000 7fa8eceef640 1 -- 172.21.15.12:0/1915388350 learned_addr learned my addr 172.21.15.12:0/1915388350 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:08.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.277+0000 7fa8e7fff640 1 --2- 172.21.15.12:0/1915388350 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561bfb2324d0 0x561bfb23b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.277+0000 7fa8e77fe640 1 --2- 172.21.15.12:0/1915388350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bfb232e80 0x561bfb238670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.277+0000 7fa8eceef640 1 -- 172.21.15.12:0/1915388350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bfb232e80 msgr2=0x561bfb238670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.277+0000 7fa8eceef640 1 --2- 172.21.15.12:0/1915388350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bfb232e80 0x561bfb238670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.277+0000 7fa8eceef640 1 -- 172.21.15.12:0/1915388350 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561bfb2324d0 msgr2=0x561bfb23b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.277+0000 7fa8eceef640 1 --2- 172.21.15.12:0/1915388350 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561bfb2324d0 0x561bfb23b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.277+0000 7fa8eceef640 1 -- 172.21.15.12:0/1915388350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bfaea93f0 con 0x561bfb23ba00 2024-04-22T13:29:08.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.277+0000 7fa8e7fff640 1 --2- 172.21.15.12:0/1915388350 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561bfb2324d0 0x561bfb23b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:08.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.277+0000 7fa8eceef640 1 --2- 172.21.15.12:0/1915388350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561bfb23ba00 0x561bfb238bd0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0012910 tx=0x7fa8e0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0aee2c8709d9e6a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:08.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.277+0000 7fa8c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:08.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.277+0000 7fa8c77fe640 1 -- 172.21.15.12:0/1915388350 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0023070 con 0x561bfb23ba00 2024-04-22T13:29:08.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.277+0000 7fa8c77fe640 1 -- 172.21.15.12:0/1915388350 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fa8e0011cf0 con 0x561bfb23ba00 2024-04-22T13:29:08.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.277+0000 7fa8f1625740 1 -- 172.21.15.12:0/1915388350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561bfb136930 con 0x561bfb23ba00 2024-04-22T13:29:08.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.277+0000 7fa8c77fe640 1 -- 172.21.15.12:0/1915388350 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa8e002f050 con 0x561bfb23ba00 2024-04-22T13:29:08.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.277+0000 7fa8f1625740 10 client.17547 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:08.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.277+0000 7fa8f1625740 1 -- 172.21.15.12:0/1915388350 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561bfaf737d0 con 0x561bfb23ba00 2024-04-22T13:29:08.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.278+0000 7fa8c77fe640 1 -- 172.21.15.12:0/1915388350 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561bfb136930 con 0x561bfb23ba00 2024-04-22T13:29:08.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.278+0000 7fa8c77fe640 1 -- 172.21.15.12:0/1915388350 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa8e000f040 con 0x561bfb23ba00 2024-04-22T13:29:08.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.278+0000 7fa8f1625740 10 client.17547 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:08.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.278+0000 7fa8f1625740 10 client.17547 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:29:08.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.278+0000 7fa8f1625740 20 client.17547 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:08.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.278+0000 7fa8f1625740 1 --2- 172.21.15.12:0/1915388350 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x561bfb24b900 0x561bfb24dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.278+0000 7fa8f1625740 4 client.17547 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:08.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.278+0000 7fa8f1625740 1 -- 172.21.15.12:0/1915388350 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bfb1a4c10 con 0x561bfb24b900 2024-04-22T13:29:08.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.278+0000 7fa8e7fff640 1 --2- 172.21.15.12:0/1915388350 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x561bfb24b900 0x561bfb24dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.278+0000 7fa8c77fe640 10 client.17547 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:29:08.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.278+0000 7fa8e7fff640 1 --2- 172.21.15.12:0/1915388350 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x561bfb24b900 0x561bfb24dda0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fad791da51f6cc19 server_cookie=7dfe3d4884db99f0 in_seq=0 out_seq=0 2024-04-22T13:29:08.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.279+0000 7fa8c77fe640 1 -- 172.21.15.12:0/1915388350 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561bfb1a4c10 con 0x561bfb24b900 2024-04-22T13:29:08.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.279+0000 7fa8c77fe640 10 client.17547 handle_command_reply: tid=0 2024-04-22T13:29:08.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.279+0000 7fa8f1625740 1 client.17547 shutdown 2024-04-22T13:29:08.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.279+0000 7fa8f1625740 1 -- 172.21.15.12:0/1915388350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561bfb23ba00 msgr2=0x561bfb238bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.279+0000 7fa8f1625740 1 --2- 172.21.15.12:0/1915388350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561bfb23ba00 0x561bfb238bd0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0012910 tx=0x7fa8e0004ab0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.279+0000 7fa8f1625740 1 -- 172.21.15.12:0/1915388350 shutdown_connections 2024-04-22T13:29:08.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.279+0000 7fa8f1625740 1 --2- 172.21.15.12:0/1915388350 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x561bfb24b900 0x561bfb24dda0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.279+0000 7fa8f1625740 1 --2- 172.21.15.12:0/1915388350 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561bfb23ba00 0x561bfb238bd0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.279+0000 7fa8c77fe640 0 client.17547 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:29:08.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.279+0000 7fa8f1625740 1 --2- 172.21.15.12:0/1915388350 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bfb232e80 0x561bfb238670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.279+0000 7fa8f1625740 1 --2- 172.21.15.12:0/1915388350 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561bfb2324d0 0x561bfb23b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.279+0000 7fa8f1625740 1 -- 172.21.15.12:0/1915388350 >> 172.21.15.12:0/1915388350 conn(0x561bfb22cba0 msgr2=0x561bfb22e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:08.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.279+0000 7fa8f1625740 1 -- 172.21.15.12:0/1915388350 shutdown_connections 2024-04-22T13:29:08.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.279+0000 7fa8f1625740 1 -- 172.21.15.12:0/1915388350 wait complete. 2024-04-22T13:29:08.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.280+0000 7fa8f1625740 20 client.17547 trim_cache size 0 max 16384 2024-04-22T13:29:08.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.293+0000 7fa8f1625740 1 Processor -- start 2024-04-22T13:29:08.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.293+0000 7fa8f1625740 1 -- start start 2024-04-22T13:29:08.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.293+0000 7fa8f1625740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561bfb2324d0 0x561bfb247560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.293+0000 7fa8f1625740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bfb232e80 0x561bfb2377d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.293+0000 7fa8f1625740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561bfb23ba00 0x561bfb22e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.293+0000 7fa8f1625740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e81ada10 con 0x561bfb23ba00 2024-04-22T13:29:08.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.293+0000 7fa8f1625740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e81af550 con 0x561bfb2324d0 2024-04-22T13:29:08.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.293+0000 7fa8f1625740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e81ae0e0 con 0x561bfb232e80 2024-04-22T13:29:08.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.293+0000 7fa8eceef640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561bfb23ba00 0x561bfb22e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.293+0000 7fa8eceef640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561bfb23ba00 0x561bfb22e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52030/0 (socket says 172.21.15.12:52030) 2024-04-22T13:29:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.293+0000 7fa8eceef640 1 -- 172.21.15.12:0/1137984176 learned_addr learned my addr 172.21.15.12:0/1137984176 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.293+0000 7fa8e7fff640 1 --2- 172.21.15.12:0/1137984176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561bfb2324d0 0x561bfb247560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.293+0000 7fa8e77fe640 1 --2- 172.21.15.12:0/1137984176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bfb232e80 0x561bfb2377d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.293+0000 7fa8eceef640 1 -- 172.21.15.12:0/1137984176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bfb232e80 msgr2=0x561bfb2377d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.293+0000 7fa8eceef640 1 --2- 172.21.15.12:0/1137984176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bfb232e80 0x561bfb2377d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.293+0000 7fa8eceef640 1 -- 172.21.15.12:0/1137984176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561bfb2324d0 msgr2=0x561bfb247560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.293+0000 7fa8eceef640 1 --2- 172.21.15.12:0/1137984176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561bfb2324d0 0x561bfb247560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.293+0000 7fa8eceef640 1 -- 172.21.15.12:0/1137984176 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bfb24a550 con 0x561bfb23ba00 2024-04-22T13:29:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.293+0000 7fa8eceef640 1 --2- 172.21.15.12:0/1137984176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561bfb23ba00 0x561bfb22e700 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fa8e002d040 tx=0x7fa8e0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c7796176c7b7a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.294+0000 7fa8cdffb640 1 -- 172.21.15.12:0/1137984176 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0023040 con 0x561bfb23ba00 2024-04-22T13:29:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.294+0000 7fa8cdffb640 1 -- 172.21.15.12:0/1137984176 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8e0017070 con 0x561bfb23ba00 2024-04-22T13:29:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.294+0000 7fa8cdffb640 1 -- 172.21.15.12:0/1137984176 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e001dea0 con 0x561bfb23ba00 2024-04-22T13:29:08.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.294+0000 7fa8f1625740 1 -- 172.21.15.12:0/1137984176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561bfb23ba00 msgr2=0x561bfb22e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.294+0000 7fa8f1625740 1 --2- 172.21.15.12:0/1137984176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561bfb23ba00 0x561bfb22e700 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fa8e002d040 tx=0x7fa8e0000f40 comp rx=0 tx=0).stop 2024-04-22T13:29:08.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.294+0000 7fa8f1625740 1 -- 172.21.15.12:0/1137984176 shutdown_connections 2024-04-22T13:29:08.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.294+0000 7fa8f1625740 1 --2- 172.21.15.12:0/1137984176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561bfb23ba00 0x561bfb22e700 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.294+0000 7fa8f1625740 1 --2- 172.21.15.12:0/1137984176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bfb232e80 0x561bfb2377d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.294+0000 7fa8f1625740 1 --2- 172.21.15.12:0/1137984176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561bfb2324d0 0x561bfb247560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.294+0000 7fa8f1625740 1 -- 172.21.15.12:0/1137984176 >> 172.21.15.12:0/1137984176 conn(0x561bfb22cba0 msgr2=0x561bfb22dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:08.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.294+0000 7fa8f1625740 1 -- 172.21.15.12:0/1137984176 shutdown_connections 2024-04-22T13:29:08.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.294+0000 7fa8f1625740 1 -- 172.21.15.12:0/1137984176 wait complete. 2024-04-22T13:29:08.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.295+0000 7fa8f1625740 1 -- start start 2024-04-22T13:29:08.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.295+0000 7fa8f1625740 1 Processor -- start 2024-04-22T13:29:08.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.295+0000 7fa8f1625740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bfb2324d0 0x561bfb243740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.295+0000 7fa8f1625740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561bfb232e80 0x561bfb243c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.295+0000 7fa8f1625740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561bfb23ba00 0x561bfb23ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.295+0000 7fa8f1625740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e81aee80 con 0x561bfb232e80 2024-04-22T13:29:08.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.295+0000 7fa8f1625740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e81ae7b0 con 0x561bfb23ba00 2024-04-22T13:29:08.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.295+0000 7fa8f1625740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e81aba10 con 0x561bfb2324d0 2024-04-22T13:29:08.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.295+0000 7fa8f1625740 10 client.0 _ll_register_callbacks cb 0x7fa8e00125c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:08.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.295+0000 7fa8e77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561bfb232e80 0x561bfb243c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.295+0000 7fa8e77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561bfb232e80 0x561bfb243c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52044/0 (socket says 172.21.15.12:52044) 2024-04-22T13:29:08.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.295+0000 7fa8e77fe640 1 -- 172.21.15.12:0/1390862579 learned_addr learned my addr 172.21.15.12:0/1390862579 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:08.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.295+0000 7fa8eceef640 1 --2- 172.21.15.12:0/1390862579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561bfb23ba00 0x561bfb23ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.295+0000 7fa8e7fff640 1 --2- 172.21.15.12:0/1390862579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bfb2324d0 0x561bfb243740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.296+0000 7fa8e77fe640 1 -- 172.21.15.12:0/1390862579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bfb2324d0 msgr2=0x561bfb243740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.296+0000 7fa8e77fe640 1 --2- 172.21.15.12:0/1390862579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bfb2324d0 0x561bfb243740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.296+0000 7fa8e77fe640 1 -- 172.21.15.12:0/1390862579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561bfb23ba00 msgr2=0x561bfb23ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.296+0000 7fa8e77fe640 1 --2- 172.21.15.12:0/1390862579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561bfb23ba00 0x561bfb23ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.296+0000 7fa8e77fe640 1 -- 172.21.15.12:0/1390862579 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bfb23f340 con 0x561bfb232e80 2024-04-22T13:29:08.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.296+0000 7fa8e7fff640 1 --2- 172.21.15.12:0/1390862579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bfb2324d0 0x561bfb243740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.296+0000 7fa8e77fe640 1 --2- 172.21.15.12:0/1390862579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561bfb232e80 0x561bfb243c80 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc0ba920 tx=0x7fa8dc0f0160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b1495433c7d3fe server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.296+0000 7fa8c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.296+0000 7fa8c77fe640 1 -- 172.21.15.12:0/1390862579 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc0be550 con 0x561bfb232e80 2024-04-22T13:29:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.296+0000 7fa8c77fe640 1 -- 172.21.15.12:0/1390862579 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fa8dc0f0330 con 0x561bfb232e80 2024-04-22T13:29:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.296+0000 7fa8f1625740 1 -- 172.21.15.12:0/1390862579 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8e0017070 con 0x561bfb232e80 2024-04-22T13:29:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.296+0000 7fa8c77fe640 1 -- 172.21.15.12:0/1390862579 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa8dc0be370 con 0x561bfb232e80 2024-04-22T13:29:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.296+0000 7fa8f1625740 10 client.17553 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.296+0000 7fa8f1625740 1 -- 172.21.15.12:0/1390862579 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561bfb2b19b0 con 0x561bfb232e80 2024-04-22T13:29:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.296+0000 7fa8c77fe640 1 -- 172.21.15.12:0/1390862579 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8e0017070 con 0x561bfb232e80 2024-04-22T13:29:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.296+0000 7fa8c77fe640 1 -- 172.21.15.12:0/1390862579 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa8dc0bb080 con 0x561bfb232e80 2024-04-22T13:29:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.296+0000 7fa8f1625740 10 client.17553 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.296+0000 7fa8f1625740 10 client.17553 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:29:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.296+0000 7fa8f1625740 20 client.17553 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.296+0000 7fa8f1625740 1 --2- 172.21.15.12:0/1390862579 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x561bfb2b1e60 0x561bfb2b4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.296+0000 7fa8f1625740 4 client.17553 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:29:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.296+0000 7fa8f1625740 1 -- 172.21.15.12:0/1390862579 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561bfb19ea80 con 0x561bfb2b1e60 2024-04-22T13:29:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.297+0000 7fa8e7fff640 1 --2- 172.21.15.12:0/1390862579 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x561bfb2b1e60 0x561bfb2b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.297+0000 7fa8e7fff640 1 --2- 172.21.15.12:0/1390862579 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x561bfb2b1e60 0x561bfb2b4280 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eccf4dbc08b77163 server_cookie=a366bd964eb94428 in_seq=0 out_seq=0 2024-04-22T13:29:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.297+0000 7fa8c77fe640 10 client.17553 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:29:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.297+0000 7fa8c77fe640 1 -- 172.21.15.12:0/1390862579 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561bfb19ea80 con 0x561bfb2b1e60 2024-04-22T13:29:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.297+0000 7fa8c77fe640 10 client.17553 handle_command_reply: tid=0 2024-04-22T13:29:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.297+0000 7fa8f1625740 1 client.17553 shutdown 2024-04-22T13:29:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.298+0000 7fa8f1625740 1 -- 172.21.15.12:0/1390862579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561bfb232e80 msgr2=0x561bfb243c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.298+0000 7fa8f1625740 1 --2- 172.21.15.12:0/1390862579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561bfb232e80 0x561bfb243c80 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc0ba920 tx=0x7fa8dc0f0160 comp rx=0 tx=0).stop 2024-04-22T13:29:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.298+0000 7fa8f1625740 1 -- 172.21.15.12:0/1390862579 shutdown_connections 2024-04-22T13:29:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.298+0000 7fa8f1625740 1 --2- 172.21.15.12:0/1390862579 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x561bfb2b1e60 0x561bfb2b4280 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.298+0000 7fa8f1625740 1 --2- 172.21.15.12:0/1390862579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561bfb23ba00 0x561bfb23ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.298+0000 7fa8c77fe640 0 client.17553 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:29:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.298+0000 7fa8f1625740 1 --2- 172.21.15.12:0/1390862579 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561bfb232e80 0x561bfb243c80 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.298+0000 7fa8f1625740 1 --2- 172.21.15.12:0/1390862579 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561bfb2324d0 0x561bfb243740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.298+0000 7fa8f1625740 1 -- 172.21.15.12:0/1390862579 >> 172.21.15.12:0/1390862579 conn(0x561bfb22cba0 msgr2=0x561bfb22e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.298+0000 7fa8f1625740 1 -- 172.21.15.12:0/1390862579 shutdown_connections 2024-04-22T13:29:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.298+0000 7fa8f1625740 1 -- 172.21.15.12:0/1390862579 wait complete. 2024-04-22T13:29:08.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.298+0000 7fa8f1625740 20 client.17553 trim_cache size 0 max 16384 2024-04-22T13:29:08.305 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:29:08.306 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.298+0000 7fa8ee952640 1 -- 172.21.15.12:0/2725829036 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa8d00a3d20 msgr2=0x7fa8d00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.298+0000 7fa8ee952640 1 --2- 172.21.15.12:0/2725829036 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa8d00a3d20 0x7fa8d00a61e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0005fd0 tx=0x7fa8e0004a40 comp rx=0 tx=0).stop 2024-04-22T13:29:08.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.298+0000 7fa8ee952640 1 -- 172.21.15.12:0/2725829036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8e8106170 msgr2=0x7fa8e8198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.298+0000 7fa8ee952640 1 --2- 172.21.15.12:0/2725829036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8e8106170 0x7fa8e8198c50 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc036040 tx=0x7fa8dc02c7e0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.298+0000 7fa8ee952640 1 -- 172.21.15.12:0/2725829036 shutdown_connections 2024-04-22T13:29:08.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.298+0000 7fa8ee952640 1 --2- 172.21.15.12:0/2725829036 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa8d00a3d20 0x7fa8d00a61e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.298+0000 7fa8ee952640 1 --2- 172.21.15.12:0/2725829036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8e819cfd0 0x7fa8e819d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.298+0000 7fa8ee952640 1 --2- 172.21.15.12:0/2725829036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa8e8106170 0x7fa8e8198c50 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.298+0000 7fa8ee952640 1 --2- 172.21.15.12:0/2725829036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8e81010a0 0x7fa8e8198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.298+0000 7fa8ee952640 1 -- 172.21.15.12:0/2725829036 >> 172.21.15.12:0/2725829036 conn(0x7fa8e8077eb0 msgr2=0x7fa8e8107340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:08.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.299+0000 7fa8ee952640 1 -- 172.21.15.12:0/2725829036 shutdown_connections 2024-04-22T13:29:08.307 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.299+0000 7fa8ee952640 1 -- 172.21.15.12:0/2725829036 wait complete. 2024-04-22T13:29:08.308 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:29:08.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.416+0000 7f8953d7e640 1 -- 172.21.15.12:0/151832276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f894c101a40 msgr2=0x7f894c105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.416+0000 7f8953d7e640 1 --2- 172.21.15.12:0/151832276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f894c101a40 0x7f894c105a80 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f893c009900 tx=0x7f893c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:29:08.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.416+0000 7f8953d7e640 1 -- 172.21.15.12:0/151832276 shutdown_connections 2024-04-22T13:29:08.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.416+0000 7f8953d7e640 1 --2- 172.21.15.12:0/151832276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f894c106160 0x7f894c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.416+0000 7f8953d7e640 1 --2- 172.21.15.12:0/151832276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f894c101a40 0x7f894c105a80 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.416+0000 7f8953d7e640 1 --2- 172.21.15.12:0/151832276 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f894c101090 0x7f894c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.416+0000 7f8953d7e640 1 -- 172.21.15.12:0/151832276 >> 172.21.15.12:0/151832276 conn(0x7f894c077f50 msgr2=0x7f894c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:08.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.416+0000 7f8953d7e640 1 -- 172.21.15.12:0/151832276 shutdown_connections 2024-04-22T13:29:08.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.416+0000 7f8953d7e640 1 -- 172.21.15.12:0/151832276 wait complete. 2024-04-22T13:29:08.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.417+0000 7f8953d7e640 1 Processor -- start 2024-04-22T13:29:08.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.417+0000 7f8953d7e640 1 -- start start 2024-04-22T13:29:08.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.417+0000 7f8953d7e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f894c101090 0x7f894c198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.417+0000 7f8953d7e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f894c106160 0x7f894c198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.417+0000 7f8953d7e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f894c19cff0 0x7f894c19d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.417+0000 7f8953d7e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f894c111080 con 0x7f894c19cff0 2024-04-22T13:29:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.417+0000 7f8953d7e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f894c110f00 con 0x7f894c101090 2024-04-22T13:29:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.417+0000 7f8953d7e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f894c111200 con 0x7f894c106160 2024-04-22T13:29:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.417+0000 7f895231b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f894c19cff0 0x7f894c19d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.417+0000 7f895231b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f894c19cff0 0x7f894c19d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52048/0 (socket says 172.21.15.12:52048) 2024-04-22T13:29:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.417+0000 7f895231b640 1 -- 172.21.15.12:0/2680297333 learned_addr learned my addr 172.21.15.12:0/2680297333 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.417+0000 7f8951b1a640 1 --2- 172.21.15.12:0/2680297333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f894c101090 0x7f894c198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.417+0000 7f8951319640 1 --2- 172.21.15.12:0/2680297333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f894c106160 0x7f894c198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.417+0000 7f895231b640 1 -- 172.21.15.12:0/2680297333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f894c106160 msgr2=0x7f894c198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.417+0000 7f895231b640 1 --2- 172.21.15.12:0/2680297333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f894c106160 0x7f894c198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.417+0000 7f895231b640 1 -- 172.21.15.12:0/2680297333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f894c101090 msgr2=0x7f894c198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.417+0000 7f895231b640 1 --2- 172.21.15.12:0/2680297333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f894c101090 0x7f894c198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.417+0000 7f895231b640 1 -- 172.21.15.12:0/2680297333 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894c19dae0 con 0x7f894c19cff0 2024-04-22T13:29:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.417+0000 7f8951319640 1 --2- 172.21.15.12:0/2680297333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f894c106160 0x7f894c198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.418+0000 7f895231b640 1 --2- 172.21.15.12:0/2680297333 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f894c19cff0 0x7f894c19d4a0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f894400eb10 tx=0x7f894400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.418+0000 7f8942ffd640 1 -- 172.21.15.12:0/2680297333 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8944004b20 con 0x7f894c19cff0 2024-04-22T13:29:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.418+0000 7f8942ffd640 1 -- 172.21.15.12:0/2680297333 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8944004cc0 con 0x7f894c19cff0 2024-04-22T13:29:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.418+0000 7f8953d7e640 1 -- 172.21.15.12:0/2680297333 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f894c19ddd0 con 0x7f894c19cff0 2024-04-22T13:29:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.418+0000 7f8942ffd640 1 -- 172.21.15.12:0/2680297333 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8944010670 con 0x7f894c19cff0 2024-04-22T13:29:08.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.418+0000 7f8953d7e640 1 -- 172.21.15.12:0/2680297333 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f894c1a5540 con 0x7f894c19cff0 2024-04-22T13:29:08.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.420+0000 7f8942ffd640 1 -- 172.21.15.12:0/2680297333 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8944005020 con 0x7f894c19cff0 2024-04-22T13:29:08.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.420+0000 7f895696f740 1 Processor -- start 2024-04-22T13:29:08.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.421+0000 7f895696f740 1 -- start start 2024-04-22T13:29:08.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.421+0000 7f895696f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565516f70250 0x565516f70630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.421+0000 7f895696f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565516f70c00 0x565516f7d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.421+0000 7f895696f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565516f7de10 0x565516f80220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.421+0000 7f895696f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x565516ed51f0 con 0x565516f70250 2024-04-22T13:29:08.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.421+0000 7f895696f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x565516e724b0 con 0x565516f7de10 2024-04-22T13:29:08.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.421+0000 7f895696f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x565516ed11b0 con 0x565516f70c00 2024-04-22T13:29:08.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.421+0000 7f8942ffd640 1 --2- 172.21.15.12:0/2680297333 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f89380a3d20 0x7f89380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.421+0000 7f8951319640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565516f70250 0x565516f70630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.421+0000 7f895231b640 1 --2- 172.21.15.12:0/2680297333 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f89380a3d20 0x7f89380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.421+0000 7f8951319640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565516f70250 0x565516f70630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52064/0 (socket says 172.21.15.12:52064) 2024-04-22T13:29:08.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.421+0000 7f8951319640 1 -- 172.21.15.12:0/3828931704 learned_addr learned my addr 172.21.15.12:0/3828931704 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:08.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.421+0000 7f8951b1a640 1 --2- 172.21.15.12:0/3828931704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565516f70c00 0x565516f7d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.421+0000 7f8942ffd640 1 -- 172.21.15.12:0/2680297333 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f89440d0dc0 con 0x7f894c19cff0 2024-04-22T13:29:08.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.421+0000 7f8951319640 1 --2- 172.21.15.12:0/3828931704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565516f7de10 0x565516f80220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.422+0000 7f8951319640 1 -- 172.21.15.12:0/3828931704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565516f70c00 msgr2=0x565516f7d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.422+0000 7f8951319640 1 --2- 172.21.15.12:0/3828931704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565516f70c00 0x565516f7d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.422+0000 7f8951319640 1 -- 172.21.15.12:0/3828931704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565516f7de10 msgr2=0x565516f80220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.422+0000 7f8951319640 1 --2- 172.21.15.12:0/3828931704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565516f7de10 0x565516f80220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.422+0000 7f8951319640 1 -- 172.21.15.12:0/3828931704 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565516e641b0 con 0x565516f70250 2024-04-22T13:29:08.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.422+0000 7f8951b1a640 1 --2- 172.21.15.12:0/3828931704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565516f70c00 0x565516f7d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:29:08.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.422+0000 7f895231b640 1 --2- 172.21.15.12:0/2680297333 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f89380a3d20 0x7f89380a61e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f894409ca50 tx=0x7f89440921e0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:08.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.422+0000 7f8951319640 1 --2- 172.21.15.12:0/3828931704 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565516f70250 0x565516f70630 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f893c005c40 tx=0x7f893c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a222769d00ab53e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:08.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.422+0000 7f8937fff640 1 -- 172.21.15.12:0/3828931704 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893c02c7e0 con 0x565516f70250 2024-04-22T13:29:08.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.422+0000 7f8937fff640 1 -- 172.21.15.12:0/3828931704 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f893c048070 con 0x565516f70250 2024-04-22T13:29:08.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.422+0000 7f8937fff640 1 -- 172.21.15.12:0/3828931704 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893c004920 con 0x565516f70250 2024-04-22T13:29:08.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.423+0000 7f895696f740 1 -- 172.21.15.12:0/3828931704 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565516f70250 msgr2=0x565516f70630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.423+0000 7f895696f740 1 --2- 172.21.15.12:0/3828931704 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565516f70250 0x565516f70630 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f893c005c40 tx=0x7f893c005bd0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.423+0000 7f895696f740 1 -- 172.21.15.12:0/3828931704 shutdown_connections 2024-04-22T13:29:08.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.423+0000 7f895696f740 1 --2- 172.21.15.12:0/3828931704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565516f7de10 0x565516f80220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.423+0000 7f895696f740 1 --2- 172.21.15.12:0/3828931704 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565516f70c00 0x565516f7d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.423+0000 7f895696f740 1 --2- 172.21.15.12:0/3828931704 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565516f70250 0x565516f70630 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.423+0000 7f895696f740 1 -- 172.21.15.12:0/3828931704 >> 172.21.15.12:0/3828931704 conn(0x565516f6aba0 msgr2=0x565516f6af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:08.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.423+0000 7f895696f740 1 -- 172.21.15.12:0/3828931704 shutdown_connections 2024-04-22T13:29:08.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.423+0000 7f895696f740 1 -- 172.21.15.12:0/3828931704 wait complete. 2024-04-22T13:29:08.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.423+0000 7f895696f740 1 -- start start 2024-04-22T13:29:08.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.423+0000 7f895696f740 1 Processor -- start 2024-04-22T13:29:08.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.424+0000 7f895696f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565516f70250 0x565516f7d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.424+0000 7f895696f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565516f70c00 0x565516f7ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.424+0000 7f895696f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565516f7de10 0x565516f7b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.424+0000 7f895696f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x565516f80ce0 con 0x565516f7de10 2024-04-22T13:29:08.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.424+0000 7f895696f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x565516e8e500 con 0x565516f70250 2024-04-22T13:29:08.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.424+0000 7f895696f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x565516f80e60 con 0x565516f70c00 2024-04-22T13:29:08.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.424+0000 7f895696f740 10 client.0 _ll_register_callbacks cb 0x565516e5cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:08.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.424+0000 7f8951319640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565516f7de10 0x565516f7b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.424+0000 7f8951319640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565516f7de10 0x565516f7b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52066/0 (socket says 172.21.15.12:52066) 2024-04-22T13:29:08.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.424+0000 7f8951319640 1 -- 172.21.15.12:0/1662179071 learned_addr learned my addr 172.21.15.12:0/1662179071 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:08.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.424+0000 7f8951b1a640 1 --2- 172.21.15.12:0/1662179071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565516f70c00 0x565516f7ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.424+0000 7f8951319640 1 -- 172.21.15.12:0/1662179071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565516f70c00 msgr2=0x565516f7ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.424+0000 7f8951319640 1 --2- 172.21.15.12:0/1662179071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565516f70c00 0x565516f7ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.424+0000 7f8951319640 1 -- 172.21.15.12:0/1662179071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565516f70250 msgr2=0x565516f7d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.424+0000 7f8951319640 1 --2- 172.21.15.12:0/1662179071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565516f70250 0x565516f7d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.424+0000 7f8951319640 1 -- 172.21.15.12:0/1662179071 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565516be73f0 con 0x565516f7de10 2024-04-22T13:29:08.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.424+0000 7f8951319640 1 --2- 172.21.15.12:0/1662179071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565516f7de10 0x565516f7b130 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f893c0092a0 tx=0x7f893c0045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b380078666e197e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:08.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.424+0000 7f89357fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:08.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.424+0000 7f89357fa640 1 -- 172.21.15.12:0/1662179071 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893c005240 con 0x565516f7de10 2024-04-22T13:29:08.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.424+0000 7f89357fa640 1 -- 172.21.15.12:0/1662179071 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f893c004a10 con 0x565516f7de10 2024-04-22T13:29:08.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.424+0000 7f895696f740 1 -- 172.21.15.12:0/1662179071 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f893c048070 con 0x565516f7de10 2024-04-22T13:29:08.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.424+0000 7f89357fa640 1 -- 172.21.15.12:0/1662179071 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f893c05a020 con 0x565516f7de10 2024-04-22T13:29:08.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.424+0000 7f895696f740 10 client.17565 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:08.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.424+0000 7f895696f740 1 -- 172.21.15.12:0/1662179071 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565516cb17d0 con 0x565516f7de10 2024-04-22T13:29:08.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.425+0000 7f89357fa640 1 -- 172.21.15.12:0/1662179071 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f893c048070 con 0x565516f7de10 2024-04-22T13:29:08.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.425+0000 7f89357fa640 1 -- 172.21.15.12:0/1662179071 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f893c0037b0 con 0x565516f7de10 2024-04-22T13:29:08.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.425+0000 7f895696f740 10 client.17565 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:08.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.425+0000 7f895696f740 10 client.17565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:08.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.425+0000 7f895696f740 20 client.17565 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:08.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.425+0000 7f895696f740 1 --2- 172.21.15.12:0/1662179071 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x565516f8ded0 0x565516f902c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.425+0000 7f895696f740 4 client.17565 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:08.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.425+0000 7f895696f740 1 -- 172.21.15.12:0/1662179071 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565516e74930 con 0x565516f8ded0 2024-04-22T13:29:08.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.425+0000 7f8951b1a640 1 --2- 172.21.15.12:0/1662179071 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x565516f8ded0 0x565516f902c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.426+0000 7f8951b1a640 1 --2- 172.21.15.12:0/1662179071 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x565516f8ded0 0x565516f902c0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4beef6b73b544a62 server_cookie=f253d80c73cea3d1 in_seq=0 out_seq=0 2024-04-22T13:29:08.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.426+0000 7f89357fa640 10 client.17565 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:08.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.426+0000 7f89357fa640 1 -- 172.21.15.12:0/1662179071 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565516e74930 con 0x565516f8ded0 2024-04-22T13:29:08.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.426+0000 7f89357fa640 10 client.17565 handle_command_reply: tid=0 2024-04-22T13:29:08.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.426+0000 7f895696f740 1 client.17565 shutdown 2024-04-22T13:29:08.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.426+0000 7f895696f740 1 -- 172.21.15.12:0/1662179071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565516f7de10 msgr2=0x565516f7b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.426+0000 7f895696f740 1 --2- 172.21.15.12:0/1662179071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565516f7de10 0x565516f7b130 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f893c0092a0 tx=0x7f893c0045d0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.427+0000 7f895696f740 1 -- 172.21.15.12:0/1662179071 shutdown_connections 2024-04-22T13:29:08.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.427+0000 7f895696f740 1 --2- 172.21.15.12:0/1662179071 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x565516f8ded0 0x565516f902c0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.427+0000 7f895696f740 1 --2- 172.21.15.12:0/1662179071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565516f7de10 0x565516f7b130 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.427+0000 7f895696f740 1 --2- 172.21.15.12:0/1662179071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565516f70c00 0x565516f7ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.427+0000 7f895696f740 1 --2- 172.21.15.12:0/1662179071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565516f70250 0x565516f7d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.427+0000 7f89357fa640 0 client.17565 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:08.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.427+0000 7f895696f740 1 -- 172.21.15.12:0/1662179071 >> 172.21.15.12:0/1662179071 conn(0x565516f6aba0 msgr2=0x565516f6c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:08.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.427+0000 7f895696f740 1 -- 172.21.15.12:0/1662179071 shutdown_connections 2024-04-22T13:29:08.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.427+0000 7f895696f740 1 -- 172.21.15.12:0/1662179071 wait complete. 2024-04-22T13:29:08.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.427+0000 7f895696f740 20 client.17565 trim_cache size 0 max 16384 2024-04-22T13:29:08.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.440+0000 7f895696f740 1 Processor -- start 2024-04-22T13:29:08.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.440+0000 7f895696f740 1 -- start start 2024-04-22T13:29:08.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.440+0000 7f895696f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565516f70250 0x565516f89bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.440+0000 7f895696f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565516f70c00 0x565516f7d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.440+0000 7f895696f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565516f7de10 0x565516f7d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.440+0000 7f895696f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f894c1ab710 con 0x565516f7de10 2024-04-22T13:29:08.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.440+0000 7f895696f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f894c1ad250 con 0x565516f70c00 2024-04-22T13:29:08.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.440+0000 7f895696f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f894c1abde0 con 0x565516f70250 2024-04-22T13:29:08.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.440+0000 7f8951319640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565516f7de10 0x565516f7d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.441 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.440+0000 7f8951319640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565516f7de10 0x565516f7d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52068/0 (socket says 172.21.15.12:52068) 2024-04-22T13:29:08.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.440+0000 7f8951319640 1 -- 172.21.15.12:0/4102384193 learned_addr learned my addr 172.21.15.12:0/4102384193 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:08.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.440+0000 7f8951b1a640 1 --2- 172.21.15.12:0/4102384193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565516f70c00 0x565516f7d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.441+0000 7f8951319640 1 -- 172.21.15.12:0/4102384193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565516f70250 msgr2=0x565516f89bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.441+0000 7f8951319640 1 --2- 172.21.15.12:0/4102384193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565516f70250 0x565516f89bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.441+0000 7f8951319640 1 -- 172.21.15.12:0/4102384193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565516f70c00 msgr2=0x565516f7d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.441+0000 7f8951319640 1 --2- 172.21.15.12:0/4102384193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565516f70c00 0x565516f7d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.441+0000 7f8951319640 1 -- 172.21.15.12:0/4102384193 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565516f6f560 con 0x565516f7de10 2024-04-22T13:29:08.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.441+0000 7f8951b1a640 1 --2- 172.21.15.12:0/4102384193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565516f70c00 0x565516f7d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:08.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.441+0000 7f8951319640 1 --2- 172.21.15.12:0/4102384193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565516f7de10 0x565516f7d790 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f893c005ea0 tx=0x7f893c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7473856cc6d4547 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:08.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.441+0000 7f89377fe640 1 -- 172.21.15.12:0/4102384193 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893c009780 con 0x565516f7de10 2024-04-22T13:29:08.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.441+0000 7f89377fe640 1 -- 172.21.15.12:0/4102384193 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f893c064020 con 0x565516f7de10 2024-04-22T13:29:08.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.441+0000 7f89377fe640 1 -- 172.21.15.12:0/4102384193 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893c0035f0 con 0x565516f7de10 2024-04-22T13:29:08.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.441+0000 7f895696f740 1 -- 172.21.15.12:0/4102384193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565516f7de10 msgr2=0x565516f7d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.442 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.441+0000 7f895696f740 1 --2- 172.21.15.12:0/4102384193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565516f7de10 0x565516f7d790 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f893c005ea0 tx=0x7f893c005b50 comp rx=0 tx=0).stop 2024-04-22T13:29:08.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.441+0000 7f895696f740 1 -- 172.21.15.12:0/4102384193 shutdown_connections 2024-04-22T13:29:08.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.441+0000 7f895696f740 1 --2- 172.21.15.12:0/4102384193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565516f7de10 0x565516f7d790 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.441+0000 7f895696f740 1 --2- 172.21.15.12:0/4102384193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565516f70c00 0x565516f7d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.441+0000 7f895696f740 1 --2- 172.21.15.12:0/4102384193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565516f70250 0x565516f89bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.441+0000 7f895696f740 1 -- 172.21.15.12:0/4102384193 >> 172.21.15.12:0/4102384193 conn(0x565516f6aba0 msgr2=0x565516f79b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:08.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.442+0000 7f895696f740 1 -- 172.21.15.12:0/4102384193 shutdown_connections 2024-04-22T13:29:08.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.442+0000 7f895696f740 1 -- 172.21.15.12:0/4102384193 wait complete. 2024-04-22T13:29:08.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.442+0000 7f895696f740 1 -- start start 2024-04-22T13:29:08.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.442+0000 7f895696f740 1 Processor -- start 2024-04-22T13:29:08.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.442+0000 7f895696f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565516f70250 0x565516f93500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.442+0000 7f895696f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565516f70c00 0x565516f90890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.442+0000 7f895696f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565516f7de10 0x565516f90dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.442+0000 7f895696f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f894c1acb80 con 0x565516f7de10 2024-04-22T13:29:08.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.442+0000 7f895696f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f894c1ac4b0 con 0x565516f70250 2024-04-22T13:29:08.443 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.442+0000 7f895696f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f894c1a9710 con 0x565516f70c00 2024-04-22T13:29:08.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.442+0000 7f895696f740 10 client.0 _ll_register_callbacks cb 0x7f893c0056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:08.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.443+0000 7f8951319640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565516f7de10 0x565516f90dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.443+0000 7f8951319640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565516f7de10 0x565516f90dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52070/0 (socket says 172.21.15.12:52070) 2024-04-22T13:29:08.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.443+0000 7f8951319640 1 -- 172.21.15.12:0/1964420091 learned_addr learned my addr 172.21.15.12:0/1964420091 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:08.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.443+0000 7f8951b1a640 1 --2- 172.21.15.12:0/1964420091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565516f70c00 0x565516f90890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.443+0000 7f8951319640 1 --2- 172.21.15.12:0/1964420091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565516f70250 0x565516f93500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.443+0000 7f8951319640 1 -- 172.21.15.12:0/1964420091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565516f70c00 msgr2=0x565516f90890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.443+0000 7f8951319640 1 --2- 172.21.15.12:0/1964420091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565516f70c00 0x565516f90890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.443+0000 7f8951319640 1 -- 172.21.15.12:0/1964420091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565516f70250 msgr2=0x565516f93500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:08.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.443+0000 7f8951319640 1 --2- 172.21.15.12:0/1964420091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565516f70250 0x565516f93500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.443+0000 7f8951319640 1 -- 172.21.15.12:0/1964420091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565516f91410 con 0x565516f7de10 2024-04-22T13:29:08.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.443+0000 7f8951319640 1 --2- 172.21.15.12:0/1964420091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565516f7de10 0x565516f90dd0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f893c003ca0 tx=0x7f893c0045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb28c49cf667f98d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:08.444 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.443+0000 7f89357fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:08.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.443+0000 7f89357fa640 1 -- 172.21.15.12:0/1964420091 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893c005000 con 0x565516f7de10 2024-04-22T13:29:08.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.443+0000 7f89357fa640 1 -- 172.21.15.12:0/1964420091 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f893c0051a0 con 0x565516f7de10 2024-04-22T13:29:08.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.443+0000 7f895696f740 1 -- 172.21.15.12:0/1964420091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f893c064020 con 0x565516f7de10 2024-04-22T13:29:08.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.443+0000 7f89357fa640 1 -- 172.21.15.12:0/1964420091 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f893c0035f0 con 0x565516f7de10 2024-04-22T13:29:08.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.443+0000 7f895696f740 10 client.17571 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:08.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.443+0000 7f895696f740 1 -- 172.21.15.12:0/1964420091 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565516ff3e70 con 0x565516f7de10 2024-04-22T13:29:08.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.444+0000 7f89357fa640 1 -- 172.21.15.12:0/1964420091 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f893c064020 con 0x565516f7de10 2024-04-22T13:29:08.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.444+0000 7f89357fa640 1 -- 172.21.15.12:0/1964420091 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f893c068020 con 0x565516f7de10 2024-04-22T13:29:08.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.444+0000 7f895696f740 10 client.17571 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:08.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.444+0000 7f895696f740 10 client.17571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:08.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.444+0000 7f895696f740 20 client.17571 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:08.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.444+0000 7f895696f740 1 --2- 172.21.15.12:0/1964420091 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x565516ff43d0 0x565516ff67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:08.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.444+0000 7f895696f740 4 client.17571 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:29:08.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.444+0000 7f895696f740 1 -- 172.21.15.12:0/1964420091 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565516ee2c10 con 0x565516ff43d0 2024-04-22T13:29:08.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.444+0000 7f8951b1a640 1 --2- 172.21.15.12:0/1964420091 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x565516ff43d0 0x565516ff67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:08.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.444+0000 7f89357fa640 10 client.17571 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:08.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.444+0000 7f8951b1a640 1 --2- 172.21.15.12:0/1964420091 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x565516ff43d0 0x565516ff67f0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a794fa56ad2d5997 server_cookie=9a4f9484540325a1 in_seq=0 out_seq=0 2024-04-22T13:29:08.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.445+0000 7f89357fa640 1 -- 172.21.15.12:0/1964420091 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565516ee2c10 con 0x565516ff43d0 2024-04-22T13:29:08.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.445+0000 7f89357fa640 10 client.17571 handle_command_reply: tid=0 2024-04-22T13:29:08.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.445+0000 7f895696f740 1 client.17571 shutdown 2024-04-22T13:29:08.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.445+0000 7f895696f740 1 -- 172.21.15.12:0/1964420091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565516f7de10 msgr2=0x565516f90dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.445+0000 7f895696f740 1 --2- 172.21.15.12:0/1964420091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565516f7de10 0x565516f90dd0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f893c003ca0 tx=0x7f893c0045d0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.445+0000 7f895696f740 1 -- 172.21.15.12:0/1964420091 shutdown_connections 2024-04-22T13:29:08.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.445+0000 7f895696f740 1 --2- 172.21.15.12:0/1964420091 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x565516ff43d0 0x565516ff67f0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.445+0000 7f895696f740 1 --2- 172.21.15.12:0/1964420091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565516f7de10 0x565516f90dd0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.445+0000 7f895696f740 1 --2- 172.21.15.12:0/1964420091 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x565516f70c00 0x565516f90890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.445+0000 7f895696f740 1 --2- 172.21.15.12:0/1964420091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x565516f70250 0x565516f93500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.445+0000 7f89357fa640 0 client.17571 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:08.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.445+0000 7f895696f740 1 -- 172.21.15.12:0/1964420091 >> 172.21.15.12:0/1964420091 conn(0x565516f6aba0 msgr2=0x565516f69370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:08.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.445+0000 7f895696f740 1 -- 172.21.15.12:0/1964420091 shutdown_connections 2024-04-22T13:29:08.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.445+0000 7f895696f740 1 -- 172.21.15.12:0/1964420091 wait complete. 2024-04-22T13:29:08.447 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.445+0000 7f895696f740 20 client.17571 trim_cache size 0 max 16384 2024-04-22T13:29:08.447 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:29:08.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.445+0000 7f8953d7e640 1 -- 172.21.15.12:0/2680297333 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f89380a3d20 msgr2=0x7f89380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.445+0000 7f8953d7e640 1 --2- 172.21.15.12:0/2680297333 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f89380a3d20 0x7f89380a61e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f894409ca50 tx=0x7f89440921e0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.445+0000 7f8953d7e640 1 -- 172.21.15.12:0/2680297333 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f894c19cff0 msgr2=0x7f894c19d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:08.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.445+0000 7f8953d7e640 1 --2- 172.21.15.12:0/2680297333 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f894c19cff0 0x7f894c19d4a0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f894400eb10 tx=0x7f894400c2d0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.445+0000 7f8953d7e640 1 -- 172.21.15.12:0/2680297333 shutdown_connections 2024-04-22T13:29:08.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.446+0000 7f8953d7e640 1 --2- 172.21.15.12:0/2680297333 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f89380a3d20 0x7f89380a61e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.446+0000 7f8953d7e640 1 --2- 172.21.15.12:0/2680297333 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f894c19cff0 0x7f894c19d4a0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.446+0000 7f8953d7e640 1 --2- 172.21.15.12:0/2680297333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f894c106160 0x7f894c198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.446+0000 7f8953d7e640 1 --2- 172.21.15.12:0/2680297333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f894c101090 0x7f894c198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:08.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.446+0000 7f8953d7e640 1 -- 172.21.15.12:0/2680297333 >> 172.21.15.12:0/2680297333 conn(0x7f894c077f50 msgr2=0x7f894c100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:08.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.446+0000 7f8953d7e640 1 -- 172.21.15.12:0/2680297333 shutdown_connections 2024-04-22T13:29:08.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:08.446+0000 7f8953d7e640 1 -- 172.21.15.12:0/2680297333 wait complete. 2024-04-22T13:29:08.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:08 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1052207759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:29:08.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:08 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1052207759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:29:09.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:08 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1052207759' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:29:09.452 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:29:09.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:29:09.453 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:29:09.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.577+0000 7ff8e0a43640 1 -- 172.21.15.12:0/1229876839 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8dc06b1f0 msgr2=0x7ff8dc106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:09.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.577+0000 7ff8e0a43640 1 --2- 172.21.15.12:0/1229876839 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8dc06b1f0 0x7ff8dc106b90 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009900 tx=0x7ff8d002bc30 comp rx=0 tx=0).stop 2024-04-22T13:29:09.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.577+0000 7ff8e0a43640 1 -- 172.21.15.12:0/1229876839 shutdown_connections 2024-04-22T13:29:09.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.577+0000 7ff8e0a43640 1 --2- 172.21.15.12:0/1229876839 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8dc107270 0x7ff8dc10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.577+0000 7ff8e0a43640 1 --2- 172.21.15.12:0/1229876839 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8dc06b1f0 0x7ff8dc106b90 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.577+0000 7ff8e0a43640 1 --2- 172.21.15.12:0/1229876839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8dc06a840 0x7ff8dc06ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.577+0000 7ff8e0a43640 1 -- 172.21.15.12:0/1229876839 >> 172.21.15.12:0/1229876839 conn(0x7ff8dc077eb0 msgr2=0x7ff8dc0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:09.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.577+0000 7ff8e0a43640 1 -- 172.21.15.12:0/1229876839 shutdown_connections 2024-04-22T13:29:09.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.577+0000 7ff8e0a43640 1 -- 172.21.15.12:0/1229876839 wait complete. 2024-04-22T13:29:09.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.578+0000 7ff8e0a43640 1 Processor -- start 2024-04-22T13:29:09.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.578+0000 7ff8e0a43640 1 -- start start 2024-04-22T13:29:09.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.578+0000 7ff8e0a43640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8dc06a840 0x7ff8dc1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.578+0000 7ff8e0a43640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8dc107270 0x7ff8dc1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.578+0000 7ff8e0a43640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8dc19a790 0x7ff8dc19ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.578+0000 7ff8e0a43640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff8dc112190 con 0x7ff8dc06a840 2024-04-22T13:29:09.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.578+0000 7ff8e0a43640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff8dc112010 con 0x7ff8dc107270 2024-04-22T13:29:09.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.578+0000 7ff8e0a43640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff8dc112310 con 0x7ff8dc19a790 2024-04-22T13:29:09.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.578+0000 7ff8da59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8dc06a840 0x7ff8dc1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.578+0000 7ff8da59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8dc06a840 0x7ff8dc1a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52092/0 (socket says 172.21.15.12:52092) 2024-04-22T13:29:09.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.578+0000 7ff8da59c640 1 -- 172.21.15.12:0/3611194262 learned_addr learned my addr 172.21.15.12:0/3611194262 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:09.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.578+0000 7ff8dad9d640 1 --2- 172.21.15.12:0/3611194262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8dc19a790 0x7ff8dc19ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.578+0000 7ff8d9d9b640 1 --2- 172.21.15.12:0/3611194262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8dc107270 0x7ff8dc1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.578+0000 7ff8da59c640 1 -- 172.21.15.12:0/3611194262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8dc19a790 msgr2=0x7ff8dc19ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:09.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.578+0000 7ff8da59c640 1 --2- 172.21.15.12:0/3611194262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8dc19a790 0x7ff8dc19ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.578+0000 7ff8da59c640 1 -- 172.21.15.12:0/3611194262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8dc107270 msgr2=0x7ff8dc1a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:09.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.578+0000 7ff8da59c640 1 --2- 172.21.15.12:0/3611194262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8dc107270 0x7ff8dc1a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.578+0000 7ff8da59c640 1 -- 172.21.15.12:0/3611194262 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8dc19b390 con 0x7ff8dc06a840 2024-04-22T13:29:09.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.578+0000 7ff8da59c640 1 --2- 172.21.15.12:0/3611194262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8dc06a840 0x7ff8dc1a0680 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc00d950 tx=0x7ff8cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:09.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.579+0000 7ff8cb7fe640 1 -- 172.21.15.12:0/3611194262 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc017070 con 0x7ff8dc06a840 2024-04-22T13:29:09.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.579+0000 7ff8cb7fe640 1 -- 172.21.15.12:0/3611194262 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8cc009c30 con 0x7ff8dc06a840 2024-04-22T13:29:09.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.579+0000 7ff8e0a43640 1 -- 172.21.15.12:0/3611194262 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8dc1a7420 con 0x7ff8dc06a840 2024-04-22T13:29:09.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.579+0000 7ff8cb7fe640 1 -- 172.21.15.12:0/3611194262 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc00f620 con 0x7ff8dc06a840 2024-04-22T13:29:09.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.579+0000 7ff8e0a43640 1 -- 172.21.15.12:0/3611194262 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8dc1a77d0 con 0x7ff8dc06a840 2024-04-22T13:29:09.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.581+0000 7ff8e3637740 1 Processor -- start 2024-04-22T13:29:09.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.581+0000 7ff8cb7fe640 1 -- 172.21.15.12:0/3611194262 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff8cc010460 con 0x7ff8dc06a840 2024-04-22T13:29:09.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.581+0000 7ff8e3637740 1 -- start start 2024-04-22T13:29:09.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.582+0000 7ff8e3637740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a16d2a8e0 0x558a16d2acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.582+0000 7ff8e3637740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558a16d2b290 0x558a16d33840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.582+0000 7ff8e3637740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a16d33e10 0x558a16d36260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.582+0000 7ff8e3637740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558a16c8ef00 con 0x558a16d33e10 2024-04-22T13:29:09.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.582+0000 7ff8e3637740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x558a16c2c590 con 0x558a16d2b290 2024-04-22T13:29:09.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.582+0000 7ff8e3637740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558a16c8aec0 con 0x558a16d2a8e0 2024-04-22T13:29:09.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.582+0000 7ff8d9d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a16d33e10 0x558a16d36260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.582+0000 7ff8dad9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a16d2a8e0 0x558a16d2acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.582+0000 7ff8d9d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a16d33e10 0x558a16d36260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52106/0 (socket says 172.21.15.12:52106) 2024-04-22T13:29:09.585 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.582+0000 7ff8da59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558a16d2b290 0x558a16d33840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.582+0000 7ff8d9d9b640 1 -- 172.21.15.12:0/3548402174 learned_addr learned my addr 172.21.15.12:0/3548402174 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:09.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.582+0000 7ff8cb7fe640 1 --2- 172.21.15.12:0/3611194262 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff8c40a3d20 0x7ff8c40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.582+0000 7ff8d9d9b640 1 -- 172.21.15.12:0/3548402174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a16d2a8e0 msgr2=0x558a16d2acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:09.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.582+0000 7ff8d9d9b640 1 --2- 172.21.15.12:0/3548402174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a16d2a8e0 0x558a16d2acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.582+0000 7ff8d9d9b640 1 -- 172.21.15.12:0/3548402174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558a16d2b290 msgr2=0x558a16d33840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:09.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.582+0000 7ff8d9d9b640 1 --2- 172.21.15.12:0/3548402174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558a16d2b290 0x558a16d33840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.582+0000 7ff8d9d9b640 1 -- 172.21.15.12:0/3548402174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a16a697e0 con 0x558a16d33e10 2024-04-22T13:29:09.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.582+0000 7ff8dad9d640 1 --2- 172.21.15.12:0/3548402174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a16d2a8e0 0x558a16d2acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:29:09.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.582+0000 7ff8cb7fe640 1 -- 172.21.15.12:0/3611194262 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7ff8cc0d0360 con 0x7ff8dc06a840 2024-04-22T13:29:09.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.582+0000 7ff8dad9d640 1 --2- 172.21.15.12:0/3611194262 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff8c40a3d20 0x7ff8c40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.583+0000 7ff8d9d9b640 1 --2- 172.21.15.12:0/3548402174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a16d33e10 0x558a16d36260 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009900 tx=0x7ff8d002c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0de7a4bd8e6ae86 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:09.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.583+0000 7ff8c8ff9640 1 -- 172.21.15.12:0/3548402174 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0045020 con 0x558a16d33e10 2024-04-22T13:29:09.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.583+0000 7ff8c8ff9640 1 -- 172.21.15.12:0/3548402174 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8d002c7e0 con 0x558a16d33e10 2024-04-22T13:29:09.586 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.583+0000 7ff8c8ff9640 1 -- 172.21.15.12:0/3548402174 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d00046c0 con 0x558a16d33e10 2024-04-22T13:29:09.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.583+0000 7ff8dad9d640 1 --2- 172.21.15.12:0/3611194262 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff8c40a3d20 0x7ff8c40a61e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4013520 tx=0x7ff8d4004a40 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:09.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.584+0000 7ff8e3637740 1 -- 172.21.15.12:0/3548402174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a16d33e10 msgr2=0x558a16d36260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:09.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.584+0000 7ff8e3637740 1 --2- 172.21.15.12:0/3548402174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a16d33e10 0x558a16d36260 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009900 tx=0x7ff8d002c6c0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.584+0000 7ff8e3637740 1 -- 172.21.15.12:0/3548402174 shutdown_connections 2024-04-22T13:29:09.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.584+0000 7ff8e3637740 1 --2- 172.21.15.12:0/3548402174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a16d33e10 0x558a16d36260 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.584+0000 7ff8e3637740 1 --2- 172.21.15.12:0/3548402174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558a16d2b290 0x558a16d33840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.584+0000 7ff8e3637740 1 --2- 172.21.15.12:0/3548402174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a16d2a8e0 0x558a16d2acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.584+0000 7ff8e3637740 1 -- 172.21.15.12:0/3548402174 >> 172.21.15.12:0/3548402174 conn(0x558a16d24fb0 msgr2=0x558a16d25390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:09.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.584+0000 7ff8e3637740 1 -- 172.21.15.12:0/3548402174 shutdown_connections 2024-04-22T13:29:09.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.584+0000 7ff8e3637740 1 -- 172.21.15.12:0/3548402174 wait complete. 2024-04-22T13:29:09.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.584+0000 7ff8e3637740 1 -- start start 2024-04-22T13:29:09.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.584+0000 7ff8e3637740 1 Processor -- start 2024-04-22T13:29:09.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.584+0000 7ff8e3637740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a16d2a8e0 0x558a16d30cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.587 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.584+0000 7ff8e3637740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558a16d2b290 0x558a16d31230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.584+0000 7ff8e3637740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a16d33e10 0x558a16d31770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.584+0000 7ff8e3637740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x558a16d36d20 con 0x558a16d2a8e0 2024-04-22T13:29:09.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.584+0000 7ff8e3637740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x558a16c48580 con 0x558a16d2b290 2024-04-22T13:29:09.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.584+0000 7ff8e3637740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558a16d36ea0 con 0x558a16d33e10 2024-04-22T13:29:09.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.585+0000 7ff8e3637740 10 client.0 _ll_register_callbacks cb 0x558a16c4e650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:09.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.585+0000 7ff8da59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a16d2a8e0 0x558a16d30cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.585+0000 7ff8da59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a16d2a8e0 0x558a16d30cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52110/0 (socket says 172.21.15.12:52110) 2024-04-22T13:29:09.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.585+0000 7ff8da59c640 1 -- 172.21.15.12:0/3642154583 learned_addr learned my addr 172.21.15.12:0/3642154583 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:09.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.585+0000 7ff8d9d9b640 1 --2- 172.21.15.12:0/3642154583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558a16d2b290 0x558a16d31230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.585+0000 7ff8dad9d640 1 --2- 172.21.15.12:0/3642154583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a16d33e10 0x558a16d31770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.585+0000 7ff8da59c640 1 -- 172.21.15.12:0/3642154583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a16d33e10 msgr2=0x558a16d31770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:09.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.585+0000 7ff8da59c640 1 --2- 172.21.15.12:0/3642154583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a16d33e10 0x558a16d31770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.585+0000 7ff8da59c640 1 -- 172.21.15.12:0/3642154583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558a16d2b290 msgr2=0x558a16d31230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:09.588 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.585+0000 7ff8da59c640 1 --2- 172.21.15.12:0/3642154583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558a16d2b290 0x558a16d31230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.585+0000 7ff8da59c640 1 -- 172.21.15.12:0/3642154583 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a16ac9f70 con 0x558a16d2a8e0 2024-04-22T13:29:09.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.585+0000 7ff8dad9d640 1 --2- 172.21.15.12:0/3642154583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a16d33e10 0x558a16d31770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:09.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.585+0000 7ff8da59c640 1 --2- 172.21.15.12:0/3642154583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a16d2a8e0 0x558a16d30cf0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc098dd0 tx=0x7ff8cc098070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ae7195268f2058 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:09.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.585+0000 7ff8c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:09.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.585+0000 7ff8c1ffb640 1 -- 172.21.15.12:0/3642154583 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc020040 con 0x558a16d2a8e0 2024-04-22T13:29:09.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.585+0000 7ff8c1ffb640 1 -- 172.21.15.12:0/3642154583 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7ff8cc098a70 con 0x558a16d2a8e0 2024-04-22T13:29:09.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.585+0000 7ff8e3637740 1 -- 172.21.15.12:0/3642154583 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8d002c7e0 con 0x558a16d2a8e0 2024-04-22T13:29:09.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.585+0000 7ff8c1ffb640 1 -- 172.21.15.12:0/3642154583 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff8cc09bd60 con 0x558a16d2a8e0 2024-04-22T13:29:09.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.585+0000 7ff8e3637740 10 client.17583 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:09.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.585+0000 7ff8e3637740 1 -- 172.21.15.12:0/3642154583 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558a16961170 con 0x558a16d2a8e0 2024-04-22T13:29:09.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.586+0000 7ff8c1ffb640 1 -- 172.21.15.12:0/3642154583 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8d002c7e0 con 0x558a16d2a8e0 2024-04-22T13:29:09.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.586+0000 7ff8c1ffb640 1 -- 172.21.15.12:0/3642154583 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff8cc098150 con 0x558a16d2a8e0 2024-04-22T13:29:09.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.586+0000 7ff8e3637740 10 client.17583 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:09.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.586+0000 7ff8e3637740 10 client.17583 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:09.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.586+0000 7ff8e3637740 20 client.17583 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:09.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.586+0000 7ff8e3637740 1 --2- 172.21.15.12:0/3642154583 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x558a16d44280 0x558a16d466b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.586+0000 7ff8e3637740 4 client.17583 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:09.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.586+0000 7ff8e3637740 1 -- 172.21.15.12:0/3642154583 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a16c9c920 con 0x558a16d44280 2024-04-22T13:29:09.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.586+0000 7ff8d9d9b640 1 --2- 172.21.15.12:0/3642154583 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x558a16d44280 0x558a16d466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.587+0000 7ff8d9d9b640 1 --2- 172.21.15.12:0/3642154583 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x558a16d44280 0x558a16d466b0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=550d893f0326470d server_cookie=271164d4b1a08e4c in_seq=0 out_seq=0 2024-04-22T13:29:09.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.587+0000 7ff8c1ffb640 10 client.17583 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:09.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.587+0000 7ff8c1ffb640 1 -- 172.21.15.12:0/3642154583 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558a16c9c920 con 0x558a16d44280 2024-04-22T13:29:09.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.587+0000 7ff8c1ffb640 10 client.17583 handle_command_reply: tid=0 2024-04-22T13:29:09.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.587+0000 7ff8e3637740 1 client.17583 shutdown 2024-04-22T13:29:09.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.587+0000 7ff8e3637740 1 -- 172.21.15.12:0/3642154583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a16d2a8e0 msgr2=0x558a16d30cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:09.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.587+0000 7ff8e3637740 1 --2- 172.21.15.12:0/3642154583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a16d2a8e0 0x558a16d30cf0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc098dd0 tx=0x7ff8cc098070 comp rx=0 tx=0).stop 2024-04-22T13:29:09.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.587+0000 7ff8e3637740 1 -- 172.21.15.12:0/3642154583 shutdown_connections 2024-04-22T13:29:09.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.587+0000 7ff8e3637740 1 --2- 172.21.15.12:0/3642154583 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x558a16d44280 0x558a16d466b0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.587+0000 7ff8e3637740 1 --2- 172.21.15.12:0/3642154583 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a16d33e10 0x558a16d31770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.587+0000 7ff8e3637740 1 --2- 172.21.15.12:0/3642154583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558a16d2b290 0x558a16d31230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.587+0000 7ff8c1ffb640 0 client.17583 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:09.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.587+0000 7ff8e3637740 1 --2- 172.21.15.12:0/3642154583 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a16d2a8e0 0x558a16d30cf0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.587+0000 7ff8e3637740 1 -- 172.21.15.12:0/3642154583 >> 172.21.15.12:0/3642154583 conn(0x558a16d24fb0 msgr2=0x558a16d26d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:09.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.588+0000 7ff8e3637740 1 -- 172.21.15.12:0/3642154583 shutdown_connections 2024-04-22T13:29:09.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.588+0000 7ff8e3637740 1 -- 172.21.15.12:0/3642154583 wait complete. 2024-04-22T13:29:09.591 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.588+0000 7ff8e3637740 20 client.17583 trim_cache size 0 max 16384 2024-04-22T13:29:09.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.605+0000 7ff8e3637740 1 Processor -- start 2024-04-22T13:29:09.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.605+0000 7ff8e3637740 1 -- start start 2024-04-22T13:29:09.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.606+0000 7ff8e3637740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558a16d2a8e0 0x558a16d3fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.606+0000 7ff8e3637740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a16d2b290 0x558a16d333b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.606+0000 7ff8e3637740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a16d33e10 0x558a16d26b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.606+0000 7ff8e3637740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff8dc1afc20 con 0x558a16d33e10 2024-04-22T13:29:09.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.606+0000 7ff8e3637740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff8dc1b1760 con 0x558a16d2a8e0 2024-04-22T13:29:09.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.606+0000 7ff8e3637740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff8dc1b02f0 con 0x558a16d2b290 2024-04-22T13:29:09.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.606+0000 7ff8dad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a16d33e10 0x558a16d26b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.606+0000 7ff8dad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a16d33e10 0x558a16d26b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52124/0 (socket says 172.21.15.12:52124) 2024-04-22T13:29:09.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.606+0000 7ff8dad9d640 1 -- 172.21.15.12:0/2471417155 learned_addr learned my addr 172.21.15.12:0/2471417155 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:09.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.606+0000 7ff8da59c640 1 --2- 172.21.15.12:0/2471417155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558a16d2a8e0 0x558a16d3fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.606+0000 7ff8d9d9b640 1 --2- 172.21.15.12:0/2471417155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a16d2b290 0x558a16d333b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.606+0000 7ff8dad9d640 1 -- 172.21.15.12:0/2471417155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a16d2b290 msgr2=0x558a16d333b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:09.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.606+0000 7ff8dad9d640 1 --2- 172.21.15.12:0/2471417155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a16d2b290 0x558a16d333b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.606+0000 7ff8dad9d640 1 -- 172.21.15.12:0/2471417155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558a16d2a8e0 msgr2=0x558a16d3fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:09.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.606+0000 7ff8dad9d640 1 --2- 172.21.15.12:0/2471417155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558a16d2a8e0 0x558a16d3fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.606+0000 7ff8dad9d640 1 -- 172.21.15.12:0/2471417155 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a16d2c8b0 con 0x558a16d33e10 2024-04-22T13:29:09.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.606+0000 7ff8da59c640 1 --2- 172.21.15.12:0/2471417155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558a16d2a8e0 0x558a16d3fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:09.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.606+0000 7ff8dad9d640 1 --2- 172.21.15.12:0/2471417155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a16d33e10 0x558a16d26b10 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4015010 tx=0x7ff8d4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f9b18b8807dfe17 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:09.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.606+0000 7ff8c3fff640 1 -- 172.21.15.12:0/2471417155 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d402b070 con 0x558a16d33e10 2024-04-22T13:29:09.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.606+0000 7ff8c3fff640 1 -- 172.21.15.12:0/2471417155 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8d4010af0 con 0x558a16d33e10 2024-04-22T13:29:09.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.606+0000 7ff8c3fff640 1 -- 172.21.15.12:0/2471417155 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d4026aa0 con 0x558a16d33e10 2024-04-22T13:29:09.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.607+0000 7ff8e3637740 1 -- 172.21.15.12:0/2471417155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a16d33e10 msgr2=0x558a16d26b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:09.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.607+0000 7ff8e3637740 1 --2- 172.21.15.12:0/2471417155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a16d33e10 0x558a16d26b10 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4015010 tx=0x7ff8d4016000 comp rx=0 tx=0).stop 2024-04-22T13:29:09.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.607+0000 7ff8e3637740 1 -- 172.21.15.12:0/2471417155 shutdown_connections 2024-04-22T13:29:09.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.607+0000 7ff8e3637740 1 --2- 172.21.15.12:0/2471417155 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a16d33e10 0x558a16d26b10 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.607+0000 7ff8e3637740 1 --2- 172.21.15.12:0/2471417155 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a16d2b290 0x558a16d333b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.607+0000 7ff8e3637740 1 --2- 172.21.15.12:0/2471417155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558a16d2a8e0 0x558a16d3fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.607+0000 7ff8e3637740 1 -- 172.21.15.12:0/2471417155 >> 172.21.15.12:0/2471417155 conn(0x558a16d24fb0 msgr2=0x558a16d26400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:09.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.607+0000 7ff8e3637740 1 -- 172.21.15.12:0/2471417155 shutdown_connections 2024-04-22T13:29:09.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.607+0000 7ff8e3637740 1 -- 172.21.15.12:0/2471417155 wait complete. 2024-04-22T13:29:09.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.607+0000 7ff8e3637740 1 -- start start 2024-04-22T13:29:09.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.607+0000 7ff8e3637740 1 Processor -- start 2024-04-22T13:29:09.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.608+0000 7ff8e3637740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558a16d2a8e0 0x558a16d3c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.608+0000 7ff8e3637740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a16d2b290 0x558a16d386c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.608+0000 7ff8e3637740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a16d33e10 0x558a16d38c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.608+0000 7ff8e3637740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff8dc1b1090 con 0x558a16d2b290 2024-04-22T13:29:09.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.608+0000 7ff8e3637740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff8dc1b09c0 con 0x558a16d2a8e0 2024-04-22T13:29:09.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.608+0000 7ff8e3637740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff8dc1adc20 con 0x558a16d33e10 2024-04-22T13:29:09.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.608+0000 7ff8e3637740 10 client.0 _ll_register_callbacks cb 0x7ff8cc0d3ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:09.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.608+0000 7ff8d9d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a16d2b290 0x558a16d386c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.608+0000 7ff8d9d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a16d2b290 0x558a16d386c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52136/0 (socket says 172.21.15.12:52136) 2024-04-22T13:29:09.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.608+0000 7ff8d9d9b640 1 -- 172.21.15.12:0/2514454796 learned_addr learned my addr 172.21.15.12:0/2514454796 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:09.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.608+0000 7ff8da59c640 1 --2- 172.21.15.12:0/2514454796 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558a16d2a8e0 0x558a16d3c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.608+0000 7ff8dad9d640 1 --2- 172.21.15.12:0/2514454796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a16d33e10 0x558a16d38c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.608+0000 7ff8d9d9b640 1 -- 172.21.15.12:0/2514454796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a16d33e10 msgr2=0x558a16d38c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:09.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.608+0000 7ff8d9d9b640 1 --2- 172.21.15.12:0/2514454796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a16d33e10 0x558a16d38c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.608+0000 7ff8d9d9b640 1 -- 172.21.15.12:0/2514454796 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558a16d2a8e0 msgr2=0x558a16d3c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:09.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.608+0000 7ff8d9d9b640 1 --2- 172.21.15.12:0/2514454796 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558a16d2a8e0 0x558a16d3c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.608+0000 7ff8d9d9b640 1 -- 172.21.15.12:0/2514454796 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a16d39270 con 0x558a16d2b290 2024-04-22T13:29:09.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.608+0000 7ff8da59c640 1 --2- 172.21.15.12:0/2514454796 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558a16d2a8e0 0x558a16d3c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:09.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.608+0000 7ff8d9d9b640 1 --2- 172.21.15.12:0/2514454796 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a16d2b290 0x558a16d386c0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009900 tx=0x7ff8d002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f5ade57d00d1851 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:09.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.608+0000 7ff8c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:09.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.608+0000 7ff8c1ffb640 1 -- 172.21.15.12:0/2514454796 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0045040 con 0x558a16d2b290 2024-04-22T13:29:09.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.608+0000 7ff8c1ffb640 1 -- 172.21.15.12:0/2514454796 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7ff8d0004040 con 0x558a16d2b290 2024-04-22T13:29:09.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.608+0000 7ff8e3637740 1 -- 172.21.15.12:0/2514454796 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8d4010af0 con 0x558a16d2b290 2024-04-22T13:29:09.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.609+0000 7ff8c1ffb640 1 -- 172.21.15.12:0/2514454796 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff8d005a020 con 0x558a16d2b290 2024-04-22T13:29:09.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.609+0000 7ff8e3637740 10 client.17589 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:09.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.609+0000 7ff8e3637740 1 -- 172.21.15.12:0/2514454796 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558a16dab640 con 0x558a16d2b290 2024-04-22T13:29:09.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.609+0000 7ff8c1ffb640 1 -- 172.21.15.12:0/2514454796 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8d4010af0 con 0x558a16d2b290 2024-04-22T13:29:09.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.609+0000 7ff8c1ffb640 1 -- 172.21.15.12:0/2514454796 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff8d0003c40 con 0x558a16d2b290 2024-04-22T13:29:09.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.609+0000 7ff8e3637740 10 client.17589 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:09.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.609+0000 7ff8e3637740 10 client.17589 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:09.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.609+0000 7ff8e3637740 20 client.17589 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:09.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.609+0000 7ff8e3637740 1 --2- 172.21.15.12:0/2514454796 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x558a16dab9d0 0x558a16daddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.609+0000 7ff8e3637740 4 client.17589 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:29:09.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.609+0000 7ff8e3637740 1 -- 172.21.15.12:0/2514454796 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558a16c96790 con 0x558a16dab9d0 2024-04-22T13:29:09.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.609+0000 7ff8da59c640 1 --2- 172.21.15.12:0/2514454796 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x558a16dab9d0 0x558a16daddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.610+0000 7ff8da59c640 1 --2- 172.21.15.12:0/2514454796 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x558a16dab9d0 0x558a16daddc0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1077c51b0c7c1d59 server_cookie=8fa186352c604931 in_seq=0 out_seq=0 2024-04-22T13:29:09.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.610+0000 7ff8c1ffb640 10 client.17589 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:09.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.611+0000 7ff8c1ffb640 1 -- 172.21.15.12:0/2514454796 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558a16c96790 con 0x558a16dab9d0 2024-04-22T13:29:09.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.611+0000 7ff8c1ffb640 10 client.17589 handle_command_reply: tid=0 2024-04-22T13:29:09.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.611+0000 7ff8e3637740 1 client.17589 shutdown 2024-04-22T13:29:09.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.611+0000 7ff8e3637740 1 -- 172.21.15.12:0/2514454796 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a16d2b290 msgr2=0x558a16d386c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:09.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.611+0000 7ff8e3637740 1 --2- 172.21.15.12:0/2514454796 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a16d2b290 0x558a16d386c0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009900 tx=0x7ff8d002bc30 comp rx=0 tx=0).stop 2024-04-22T13:29:09.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.611+0000 7ff8e3637740 1 -- 172.21.15.12:0/2514454796 shutdown_connections 2024-04-22T13:29:09.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.611+0000 7ff8e3637740 1 --2- 172.21.15.12:0/2514454796 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x558a16dab9d0 0x558a16daddc0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.611+0000 7ff8e3637740 1 --2- 172.21.15.12:0/2514454796 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558a16d33e10 0x558a16d38c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.611+0000 7ff8e3637740 1 --2- 172.21.15.12:0/2514454796 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a16d2b290 0x558a16d386c0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.611+0000 7ff8c1ffb640 0 client.17589 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:09.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.611+0000 7ff8e3637740 1 --2- 172.21.15.12:0/2514454796 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558a16d2a8e0 0x558a16d3c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.611+0000 7ff8e3637740 1 -- 172.21.15.12:0/2514454796 >> 172.21.15.12:0/2514454796 conn(0x558a16d24fb0 msgr2=0x558a16d31540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:09.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.611+0000 7ff8e3637740 1 -- 172.21.15.12:0/2514454796 shutdown_connections 2024-04-22T13:29:09.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.611+0000 7ff8e3637740 1 -- 172.21.15.12:0/2514454796 wait complete. 2024-04-22T13:29:09.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.611+0000 7ff8e3637740 20 client.17589 trim_cache size 0 max 16384 2024-04-22T13:29:09.613 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:29:09.614 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-04-22T13:29:09.614 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "dd3914be-b0de-468d-a798-09daa3f613ef", 2024-04-22T13:29:09.614 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-04-22T13:29:09.614 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:29:09.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.611+0000 7ff8e0a43640 1 -- 172.21.15.12:0/3611194262 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff8c40a3d20 msgr2=0x7ff8c40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:09.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.611+0000 7ff8e0a43640 1 --2- 172.21.15.12:0/3611194262 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff8c40a3d20 0x7ff8c40a61e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4013520 tx=0x7ff8d4004a40 comp rx=0 tx=0).stop 2024-04-22T13:29:09.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.611+0000 7ff8e0a43640 1 -- 172.21.15.12:0/3611194262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8dc06a840 msgr2=0x7ff8dc1a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:09.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.611+0000 7ff8e0a43640 1 --2- 172.21.15.12:0/3611194262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8dc06a840 0x7ff8dc1a0680 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc00d950 tx=0x7ff8cc00de10 comp rx=0 tx=0).stop 2024-04-22T13:29:09.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.612+0000 7ff8e0a43640 1 -- 172.21.15.12:0/3611194262 shutdown_connections 2024-04-22T13:29:09.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.612+0000 7ff8e0a43640 1 --2- 172.21.15.12:0/3611194262 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff8c40a3d20 0x7ff8c40a61e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.612+0000 7ff8e0a43640 1 --2- 172.21.15.12:0/3611194262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff8dc19a790 0x7ff8dc19ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.612+0000 7ff8e0a43640 1 --2- 172.21.15.12:0/3611194262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff8dc107270 0x7ff8dc1a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.612+0000 7ff8e0a43640 1 --2- 172.21.15.12:0/3611194262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff8dc06a840 0x7ff8dc1a0680 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.612+0000 7ff8e0a43640 1 -- 172.21.15.12:0/3611194262 >> 172.21.15.12:0/3611194262 conn(0x7ff8dc077eb0 msgr2=0x7ff8dc10d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:09.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.612+0000 7ff8e0a43640 1 -- 172.21.15.12:0/3611194262 shutdown_connections 2024-04-22T13:29:09.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.612+0000 7ff8e0a43640 1 -- 172.21.15.12:0/3611194262 wait complete. 2024-04-22T13:29:09.618 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:29:09.618 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:29:09.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.727+0000 7fd7dce11640 1 -- 172.21.15.12:0/3727019309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7d8101080 msgr2=0x7fd7d8101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:09.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.728+0000 7fd7dce11640 1 --2- 172.21.15.12:0/3727019309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7d8101080 0x7fd7d8101460 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8009ca0 tx=0x7fd7c802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.728+0000 7fd7dce11640 1 -- 172.21.15.12:0/3727019309 shutdown_connections 2024-04-22T13:29:09.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.728+0000 7fd7dce11640 1 --2- 172.21.15.12:0/3727019309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7d8106150 0x7fd7d810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.728+0000 7fd7dce11640 1 --2- 172.21.15.12:0/3727019309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7d8101a30 0x7fd7d8105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.728+0000 7fd7dce11640 1 --2- 172.21.15.12:0/3727019309 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7d8101080 0x7fd7d8101460 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.728+0000 7fd7dce11640 1 -- 172.21.15.12:0/3727019309 >> 172.21.15.12:0/3727019309 conn(0x7fd7d8078000 msgr2=0x7fd7d8078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:09.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.728+0000 7fd7dce11640 1 -- 172.21.15.12:0/3727019309 shutdown_connections 2024-04-22T13:29:09.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.728+0000 7fd7dce11640 1 -- 172.21.15.12:0/3727019309 wait complete. 2024-04-22T13:29:09.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.728+0000 7fd7dce11640 1 Processor -- start 2024-04-22T13:29:09.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.728+0000 7fd7dce11640 1 -- start start 2024-04-22T13:29:09.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.729+0000 7fd7dce11640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7d8101080 0x7fd7d810e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.729+0000 7fd7dce11640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7d8106150 0x7fd7d810ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.729+0000 7fd7dce11640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7d810f130 0x7fd7d81a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.729+0000 7fd7dce11640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d81121d0 con 0x7fd7d8101080 2024-04-22T13:29:09.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.729+0000 7fd7dce11640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d8112050 con 0x7fd7d8106150 2024-04-22T13:29:09.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.729+0000 7fd7dce11640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d8112350 con 0x7fd7d810f130 2024-04-22T13:29:09.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.729+0000 7fd7d659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7d8101080 0x7fd7d810e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.729+0000 7fd7d659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7d8101080 0x7fd7d810e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52152/0 (socket says 172.21.15.12:52152) 2024-04-22T13:29:09.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.729+0000 7fd7d659c640 1 -- 172.21.15.12:0/3826029485 learned_addr learned my addr 172.21.15.12:0/3826029485 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:09.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.729+0000 7fd7d5d9b640 1 --2- 172.21.15.12:0/3826029485 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7d8106150 0x7fd7d810ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.729+0000 7fd7d6d9d640 1 --2- 172.21.15.12:0/3826029485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7d810f130 0x7fd7d81a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.729+0000 7fd7d659c640 1 -- 172.21.15.12:0/3826029485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7d810f130 msgr2=0x7fd7d81a68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:09.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.729+0000 7fd7d659c640 1 --2- 172.21.15.12:0/3826029485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7d810f130 0x7fd7d81a68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.729+0000 7fd7d659c640 1 -- 172.21.15.12:0/3826029485 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7d8106150 msgr2=0x7fd7d810ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:09.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.729+0000 7fd7d659c640 1 --2- 172.21.15.12:0/3826029485 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7d8106150 0x7fd7d810ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.729+0000 7fd7d659c640 1 -- 172.21.15.12:0/3826029485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d81a6fe0 con 0x7fd7d8101080 2024-04-22T13:29:09.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.729+0000 7fd7d659c640 1 --2- 172.21.15.12:0/3826029485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7d8101080 0x7fd7d810e6b0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8005e30 tx=0x7fd7c80346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:09.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.729+0000 7fd7c77fe640 1 -- 172.21.15.12:0/3826029485 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8039040 con 0x7fd7d8101080 2024-04-22T13:29:09.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.730+0000 7fd7c77fe640 1 -- 172.21.15.12:0/3826029485 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7c8044070 con 0x7fd7d8101080 2024-04-22T13:29:09.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.730+0000 7fd7dce11640 1 -- 172.21.15.12:0/3826029485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d81a7270 con 0x7fd7d8101080 2024-04-22T13:29:09.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.730+0000 7fd7c77fe640 1 -- 172.21.15.12:0/3826029485 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c80042a0 con 0x7fd7d8101080 2024-04-22T13:29:09.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.730+0000 7fd7dce11640 1 -- 172.21.15.12:0/3826029485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d81a7730 con 0x7fd7d8101080 2024-04-22T13:29:09.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.732+0000 7fd7c77fe640 1 -- 172.21.15.12:0/3826029485 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd7c8004440 con 0x7fd7d8101080 2024-04-22T13:29:09.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.732+0000 7fd7dfa00740 1 Processor -- start 2024-04-22T13:29:09.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.732+0000 7fd7dfa00740 1 -- start start 2024-04-22T13:29:09.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.732+0000 7fd7dfa00740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f299b33580 0x55f299b33960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.732+0000 7fd7dfa00740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f299b33ea0 0x55f299b3c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.733+0000 7fd7dfa00740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f299b3ca80 0x55f299b3ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.733+0000 7fd7dfa00740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f299a981e0 con 0x55f299b33580 2024-04-22T13:29:09.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.733+0000 7fd7dfa00740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f299a35390 con 0x55f299b3ca80 2024-04-22T13:29:09.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.733+0000 7fd7dfa00740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f299a941a0 con 0x55f299b33ea0 2024-04-22T13:29:09.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.733+0000 7fd7d6d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f299b33580 0x55f299b33960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.733+0000 7fd7d6d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f299b33580 0x55f299b33960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52166/0 (socket says 172.21.15.12:52166) 2024-04-22T13:29:09.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.733+0000 7fd7d659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f299b33ea0 0x55f299b3c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.733+0000 7fd7d6d9d640 1 -- 172.21.15.12:0/3376944038 learned_addr learned my addr 172.21.15.12:0/3376944038 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:09.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.733+0000 7fd7d5d9b640 1 --2- 172.21.15.12:0/3376944038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f299b3ca80 0x55f299b3ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.733+0000 7fd7c77fe640 1 --2- 172.21.15.12:0/3826029485 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd7c00a3d50 0x7fd7c00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.733+0000 7fd7d6d9d640 1 -- 172.21.15.12:0/3376944038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f299b33ea0 msgr2=0x55f299b3c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:09.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.733+0000 7fd7d6d9d640 1 --2- 172.21.15.12:0/3376944038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f299b33ea0 0x55f299b3c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.733+0000 7fd7d6d9d640 1 -- 172.21.15.12:0/3376944038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f299b3ca80 msgr2=0x55f299b3ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:09.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.733+0000 7fd7d6d9d640 1 --2- 172.21.15.12:0/3376944038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f299b3ca80 0x55f299b3ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.733+0000 7fd7d6d9d640 1 -- 172.21.15.12:0/3376944038 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2998747d0 con 0x55f299b33580 2024-04-22T13:29:09.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.733+0000 7fd7c77fe640 1 -- 172.21.15.12:0/3826029485 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fd7c80f2860 con 0x7fd7d8101080 2024-04-22T13:29:09.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.733+0000 7fd7d6d9d640 1 --2- 172.21.15.12:0/3826029485 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd7c00a3d50 0x7fd7c00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.733+0000 7fd7d6d9d640 1 --2- 172.21.15.12:0/3376944038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f299b33580 0x55f299b33960 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fd7d000b7a0 tx=0x7fd7d000bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd558586fe0f7d6c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:09.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.734+0000 7fd7c4ff9640 1 -- 172.21.15.12:0/3376944038 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0024070 con 0x55f299b33580 2024-04-22T13:29:09.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.734+0000 7fd7c4ff9640 1 -- 172.21.15.12:0/3376944038 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7d001e930 con 0x55f299b33580 2024-04-22T13:29:09.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.734+0000 7fd7c4ff9640 1 -- 172.21.15.12:0/3376944038 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d001c7f0 con 0x55f299b33580 2024-04-22T13:29:09.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.734+0000 7fd7dfa00740 1 -- 172.21.15.12:0/3376944038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f299b33580 msgr2=0x55f299b33960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:09.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.734+0000 7fd7d6d9d640 1 --2- 172.21.15.12:0/3826029485 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd7c00a3d50 0x7fd7c00a6210 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0021000 tx=0x7fd7d001e000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:09.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.734+0000 7fd7dfa00740 1 --2- 172.21.15.12:0/3376944038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f299b33580 0x55f299b33960 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fd7d000b7a0 tx=0x7fd7d000bc60 comp rx=0 tx=0).stop 2024-04-22T13:29:09.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.734+0000 7fd7dfa00740 1 -- 172.21.15.12:0/3376944038 shutdown_connections 2024-04-22T13:29:09.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.734+0000 7fd7dfa00740 1 --2- 172.21.15.12:0/3376944038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f299b3ca80 0x55f299b3ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.734+0000 7fd7dfa00740 1 --2- 172.21.15.12:0/3376944038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f299b33ea0 0x55f299b3c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.734+0000 7fd7dfa00740 1 --2- 172.21.15.12:0/3376944038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f299b33580 0x55f299b33960 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.734+0000 7fd7dfa00740 1 -- 172.21.15.12:0/3376944038 >> 172.21.15.12:0/3376944038 conn(0x55f299b2dcb0 msgr2=0x55f299b2e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:09.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.734+0000 7fd7dfa00740 1 -- 172.21.15.12:0/3376944038 shutdown_connections 2024-04-22T13:29:09.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.734+0000 7fd7dfa00740 1 -- 172.21.15.12:0/3376944038 wait complete. 2024-04-22T13:29:09.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.735+0000 7fd7dfa00740 1 -- start start 2024-04-22T13:29:09.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.735+0000 7fd7dfa00740 1 Processor -- start 2024-04-22T13:29:09.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.735+0000 7fd7dfa00740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f299b33580 0x55f299b39ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.735+0000 7fd7dfa00740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f299b33ea0 0x55f299b3a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.735+0000 7fd7dfa00740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f299b3ca80 0x55f299b3a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.735+0000 7fd7dfa00740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f299b3f870 con 0x55f299b33ea0 2024-04-22T13:29:09.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.735+0000 7fd7dfa00740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f299a512f0 con 0x55f299b33580 2024-04-22T13:29:09.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.735+0000 7fd7dfa00740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f299b3f9f0 con 0x55f299b3ca80 2024-04-22T13:29:09.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.735+0000 7fd7dfa00740 10 client.0 _ll_register_callbacks cb 0x55f299a5e580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:09.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.735+0000 7fd7d5d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f299b33ea0 0x55f299b3a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.735+0000 7fd7d5d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f299b33ea0 0x55f299b3a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52178/0 (socket says 172.21.15.12:52178) 2024-04-22T13:29:09.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.735+0000 7fd7d5d9b640 1 -- 172.21.15.12:0/1766829198 learned_addr learned my addr 172.21.15.12:0/1766829198 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:09.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.735+0000 7fd7d6d9d640 1 --2- 172.21.15.12:0/1766829198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f299b3ca80 0x55f299b3a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.735+0000 7fd7d659c640 1 --2- 172.21.15.12:0/1766829198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f299b33580 0x55f299b39ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.735+0000 7fd7d5d9b640 1 -- 172.21.15.12:0/1766829198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f299b3ca80 msgr2=0x55f299b3a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:09.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.735+0000 7fd7d5d9b640 1 --2- 172.21.15.12:0/1766829198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f299b3ca80 0x55f299b3a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.735+0000 7fd7d5d9b640 1 -- 172.21.15.12:0/1766829198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f299b33580 msgr2=0x55f299b39ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:09.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.735+0000 7fd7d5d9b640 1 --2- 172.21.15.12:0/1766829198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f299b33580 0x55f299b39ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.735+0000 7fd7d5d9b640 1 -- 172.21.15.12:0/1766829198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2997aa3f0 con 0x55f299b33ea0 2024-04-22T13:29:09.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.735+0000 7fd7d6d9d640 1 --2- 172.21.15.12:0/1766829198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f299b3ca80 0x55f299b3a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:09.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.735+0000 7fd7d659c640 1 --2- 172.21.15.12:0/1766829198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f299b33580 0x55f299b39ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:09.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.736+0000 7fd7d5d9b640 1 --2- 172.21.15.12:0/1766829198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f299b33ea0 0x55f299b3a010 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc012730 tx=0x7fd7cc012bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76cfc61daa7260ca server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:09.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.736+0000 7fd7bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:09.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.736+0000 7fd7bdffb640 1 -- 172.21.15.12:0/1766829198 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc004020 con 0x55f299b33ea0 2024-04-22T13:29:09.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.736+0000 7fd7bdffb640 1 -- 172.21.15.12:0/1766829198 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fd7cc0041c0 con 0x55f299b33ea0 2024-04-22T13:29:09.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.736+0000 7fd7dfa00740 1 -- 172.21.15.12:0/1766829198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7d001e930 con 0x55f299b33ea0 2024-04-22T13:29:09.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.736+0000 7fd7bdffb640 1 -- 172.21.15.12:0/1766829198 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd7cc0043c0 con 0x55f299b33ea0 2024-04-22T13:29:09.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.736+0000 7fd7dfa00740 10 client.17601 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:09.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.736+0000 7fd7dfa00740 1 -- 172.21.15.12:0/1766829198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f29976a0f0 con 0x55f299b33ea0 2024-04-22T13:29:09.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.736+0000 7fd7bdffb640 1 -- 172.21.15.12:0/1766829198 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7d001e930 con 0x55f299b33ea0 2024-04-22T13:29:09.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.736+0000 7fd7bdffb640 1 -- 172.21.15.12:0/1766829198 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd7cc003a10 con 0x55f299b33ea0 2024-04-22T13:29:09.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.736+0000 7fd7dfa00740 10 client.17601 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:09.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.736+0000 7fd7dfa00740 10 client.17601 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:09.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.736+0000 7fd7dfa00740 20 client.17601 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:09.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.736+0000 7fd7dfa00740 1 --2- 172.21.15.12:0/1766829198 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f299b4ca40 0x55f299b4ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.736+0000 7fd7dfa00740 4 client.17601 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:09.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.736+0000 7fd7dfa00740 1 -- 172.21.15.12:0/1766829198 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f299aa5c00 con 0x55f299b4ca40 2024-04-22T13:29:09.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.737+0000 7fd7d659c640 1 --2- 172.21.15.12:0/1766829198 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f299b4ca40 0x55f299b4ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.737+0000 7fd7d659c640 1 --2- 172.21.15.12:0/1766829198 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f299b4ca40 0x55f299b4ee90 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f03474dfc6fc3147 server_cookie=95037b6d5b2d1a8c in_seq=0 out_seq=0 2024-04-22T13:29:09.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.737+0000 7fd7bdffb640 10 client.17601 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:09.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.738+0000 7fd7bdffb640 1 -- 172.21.15.12:0/1766829198 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f299aa5c00 con 0x55f299b4ca40 2024-04-22T13:29:09.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.738+0000 7fd7bdffb640 10 client.17601 handle_command_reply: tid=0 2024-04-22T13:29:09.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.738+0000 7fd7dfa00740 1 client.17601 shutdown 2024-04-22T13:29:09.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.738+0000 7fd7dfa00740 1 -- 172.21.15.12:0/1766829198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f299b33ea0 msgr2=0x55f299b3a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:09.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.738+0000 7fd7dfa00740 1 --2- 172.21.15.12:0/1766829198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f299b33ea0 0x55f299b3a010 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc012730 tx=0x7fd7cc012bf0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.738+0000 7fd7dfa00740 1 -- 172.21.15.12:0/1766829198 shutdown_connections 2024-04-22T13:29:09.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.738+0000 7fd7dfa00740 1 --2- 172.21.15.12:0/1766829198 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f299b4ca40 0x55f299b4ee90 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.738+0000 7fd7dfa00740 1 --2- 172.21.15.12:0/1766829198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f299b3ca80 0x55f299b3a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.738+0000 7fd7bdffb640 0 client.17601 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:09.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.738+0000 7fd7dfa00740 1 --2- 172.21.15.12:0/1766829198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f299b33ea0 0x55f299b3a010 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.738+0000 7fd7dfa00740 1 --2- 172.21.15.12:0/1766829198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f299b33580 0x55f299b39ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.738+0000 7fd7dfa00740 1 -- 172.21.15.12:0/1766829198 >> 172.21.15.12:0/1766829198 conn(0x55f299b2dcb0 msgr2=0x55f299b2fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:09.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.738+0000 7fd7dfa00740 1 -- 172.21.15.12:0/1766829198 shutdown_connections 2024-04-22T13:29:09.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.738+0000 7fd7dfa00740 1 -- 172.21.15.12:0/1766829198 wait complete. 2024-04-22T13:29:09.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.738+0000 7fd7dfa00740 20 client.17601 trim_cache size 0 max 16384 2024-04-22T13:29:09.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.756+0000 7fd7dfa00740 1 Processor -- start 2024-04-22T13:29:09.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.756+0000 7fd7dfa00740 1 -- start start 2024-04-22T13:29:09.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.756+0000 7fd7dfa00740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f299b33580 0x55f299b3c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.756+0000 7fd7dfa00740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f299b33ea0 0x55f299b38a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.756+0000 7fd7dfa00740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f299b3ca80 0x55f299b48730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.756+0000 7fd7dfa00740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d81afc90 con 0x55f299b33ea0 2024-04-22T13:29:09.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.756+0000 7fd7dfa00740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d81b17d0 con 0x55f299b33580 2024-04-22T13:29:09.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.756+0000 7fd7dfa00740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d81b0360 con 0x55f299b3ca80 2024-04-22T13:29:09.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.756+0000 7fd7d5d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f299b33ea0 0x55f299b38a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.756+0000 7fd7d5d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f299b33ea0 0x55f299b38a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52194/0 (socket says 172.21.15.12:52194) 2024-04-22T13:29:09.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.756+0000 7fd7d5d9b640 1 -- 172.21.15.12:0/2328521514 learned_addr learned my addr 172.21.15.12:0/2328521514 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:09.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.756+0000 7fd7d659c640 1 --2- 172.21.15.12:0/2328521514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f299b33580 0x55f299b3c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.756+0000 7fd7d5d9b640 1 -- 172.21.15.12:0/2328521514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f299b3ca80 msgr2=0x55f299b48730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:09.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.756+0000 7fd7d5d9b640 1 --2- 172.21.15.12:0/2328521514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f299b3ca80 0x55f299b48730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.756+0000 7fd7d5d9b640 1 -- 172.21.15.12:0/2328521514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f299b33580 msgr2=0x55f299b3c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:09.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.756+0000 7fd7d5d9b640 1 --2- 172.21.15.12:0/2328521514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f299b33580 0x55f299b3c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.756+0000 7fd7d5d9b640 1 -- 172.21.15.12:0/2328521514 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f299b32f50 con 0x55f299b33ea0 2024-04-22T13:29:09.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.756+0000 7fd7d5d9b640 1 --2- 172.21.15.12:0/2328521514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f299b33ea0 0x55f299b38a10 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc000c00 tx=0x7fd7cc00f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd14685959816b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:09.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.756+0000 7fd7bffff640 1 -- 172.21.15.12:0/2328521514 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc003fd0 con 0x55f299b33ea0 2024-04-22T13:29:09.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.757+0000 7fd7bffff640 1 -- 172.21.15.12:0/2328521514 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7cc004170 con 0x55f299b33ea0 2024-04-22T13:29:09.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.757+0000 7fd7bffff640 1 -- 172.21.15.12:0/2328521514 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc01c4c0 con 0x55f299b33ea0 2024-04-22T13:29:09.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.757+0000 7fd7dfa00740 1 -- 172.21.15.12:0/2328521514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f299b33ea0 msgr2=0x55f299b38a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:09.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.757+0000 7fd7dfa00740 1 --2- 172.21.15.12:0/2328521514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f299b33ea0 0x55f299b38a10 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc000c00 tx=0x7fd7cc00f810 comp rx=0 tx=0).stop 2024-04-22T13:29:09.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.757+0000 7fd7dfa00740 1 -- 172.21.15.12:0/2328521514 shutdown_connections 2024-04-22T13:29:09.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.757+0000 7fd7dfa00740 1 --2- 172.21.15.12:0/2328521514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f299b3ca80 0x55f299b48730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.757+0000 7fd7dfa00740 1 --2- 172.21.15.12:0/2328521514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f299b33ea0 0x55f299b38a10 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.757+0000 7fd7dfa00740 1 --2- 172.21.15.12:0/2328521514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f299b33580 0x55f299b3c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.757+0000 7fd7dfa00740 1 -- 172.21.15.12:0/2328521514 >> 172.21.15.12:0/2328521514 conn(0x55f299b2dcb0 msgr2=0x55f299b2f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:09.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.757+0000 7fd7dfa00740 1 -- 172.21.15.12:0/2328521514 shutdown_connections 2024-04-22T13:29:09.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.757+0000 7fd7dfa00740 1 -- 172.21.15.12:0/2328521514 wait complete. 2024-04-22T13:29:09.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.757+0000 7fd7dfa00740 1 -- start start 2024-04-22T13:29:09.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.757+0000 7fd7dfa00740 1 Processor -- start 2024-04-22T13:29:09.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.758+0000 7fd7dfa00740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f299b33580 0x55f299b36990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.758+0000 7fd7dfa00740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f299b33ea0 0x55f299b36ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.758+0000 7fd7dfa00740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f299b3ca80 0x55f299b37410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.758+0000 7fd7dfa00740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d81b1100 con 0x55f299b33ea0 2024-04-22T13:29:09.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.758+0000 7fd7dfa00740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d81b0a30 con 0x55f299b3ca80 2024-04-22T13:29:09.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.758+0000 7fd7dfa00740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d81adc70 con 0x55f299b33580 2024-04-22T13:29:09.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.758+0000 7fd7dfa00740 10 client.0 _ll_register_callbacks cb 0x7fd7cc009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:09.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.758+0000 7fd7d5d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f299b33ea0 0x55f299b36ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.758+0000 7fd7d5d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f299b33ea0 0x55f299b36ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52206/0 (socket says 172.21.15.12:52206) 2024-04-22T13:29:09.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.758+0000 7fd7d5d9b640 1 -- 172.21.15.12:0/154572204 learned_addr learned my addr 172.21.15.12:0/154572204 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:09.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.758+0000 7fd7d6d9d640 1 --2- 172.21.15.12:0/154572204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f299b3ca80 0x55f299b37410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.758+0000 7fd7d659c640 1 --2- 172.21.15.12:0/154572204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f299b33580 0x55f299b36990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.758+0000 7fd7d5d9b640 1 -- 172.21.15.12:0/154572204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f299b33580 msgr2=0x55f299b36990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:09.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.758+0000 7fd7d5d9b640 1 --2- 172.21.15.12:0/154572204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f299b33580 0x55f299b36990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.758+0000 7fd7d5d9b640 1 -- 172.21.15.12:0/154572204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f299b3ca80 msgr2=0x55f299b37410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:09.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.758+0000 7fd7d5d9b640 1 --2- 172.21.15.12:0/154572204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f299b3ca80 0x55f299b37410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.758+0000 7fd7d5d9b640 1 -- 172.21.15.12:0/154572204 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f299bb21d0 con 0x55f299b33ea0 2024-04-22T13:29:09.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.758+0000 7fd7d659c640 1 --2- 172.21.15.12:0/154572204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f299b33580 0x55f299b36990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:09.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.758+0000 7fd7d5d9b640 1 --2- 172.21.15.12:0/154572204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f299b33ea0 0x55f299b36ed0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc003a50 tx=0x7fd7cc0126f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be0da385eabc525 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:09.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.758+0000 7fd7bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:09.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.758+0000 7fd7bdffb640 1 -- 172.21.15.12:0/154572204 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc032070 con 0x55f299b33ea0 2024-04-22T13:29:09.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.758+0000 7fd7bdffb640 1 -- 172.21.15.12:0/154572204 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fd7cc00fb30 con 0x55f299b33ea0 2024-04-22T13:29:09.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.758+0000 7fd7dfa00740 1 -- 172.21.15.12:0/154572204 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7cc004170 con 0x55f299b33ea0 2024-04-22T13:29:09.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.759+0000 7fd7bdffb640 1 -- 172.21.15.12:0/154572204 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd7cc03e050 con 0x55f299b33ea0 2024-04-22T13:29:09.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.759+0000 7fd7dfa00740 10 client.17607 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:09.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.759+0000 7fd7dfa00740 1 -- 172.21.15.12:0/154572204 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f299bb34c0 con 0x55f299b33ea0 2024-04-22T13:29:09.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.759+0000 7fd7bdffb640 1 -- 172.21.15.12:0/154572204 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7cc004170 con 0x55f299b33ea0 2024-04-22T13:29:09.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.759+0000 7fd7bdffb640 1 -- 172.21.15.12:0/154572204 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd7cc01ccd0 con 0x55f299b33ea0 2024-04-22T13:29:09.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.759+0000 7fd7dfa00740 10 client.17607 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:09.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.759+0000 7fd7dfa00740 10 client.17607 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:09.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.759+0000 7fd7dfa00740 20 client.17607 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:09.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.759+0000 7fd7dfa00740 1 --2- 172.21.15.12:0/154572204 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f299bb3ba0 0x55f299bb5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.759+0000 7fd7dfa00740 4 client.17607 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:29:09.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.759+0000 7fd7dfa00740 1 -- 172.21.15.12:0/154572204 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f299a9fa70 con 0x55f299bb3ba0 2024-04-22T13:29:09.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.759+0000 7fd7d659c640 1 --2- 172.21.15.12:0/154572204 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f299bb3ba0 0x55f299bb5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.760+0000 7fd7d659c640 1 --2- 172.21.15.12:0/154572204 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f299bb3ba0 0x55f299bb5fc0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d01a15930097db71 server_cookie=b542ff8cae25cce1 in_seq=0 out_seq=0 2024-04-22T13:29:09.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.760+0000 7fd7bdffb640 10 client.17607 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:09.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.760+0000 7fd7bdffb640 1 -- 172.21.15.12:0/154572204 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f299a9fa70 con 0x55f299bb3ba0 2024-04-22T13:29:09.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.760+0000 7fd7bdffb640 10 client.17607 handle_command_reply: tid=0 2024-04-22T13:29:09.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.760+0000 7fd7dfa00740 1 client.17607 shutdown 2024-04-22T13:29:09.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.760+0000 7fd7dfa00740 1 -- 172.21.15.12:0/154572204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f299b33ea0 msgr2=0x55f299b36ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:09.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.760+0000 7fd7dfa00740 1 --2- 172.21.15.12:0/154572204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f299b33ea0 0x55f299b36ed0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc003a50 tx=0x7fd7cc0126f0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.760+0000 7fd7dfa00740 1 -- 172.21.15.12:0/154572204 shutdown_connections 2024-04-22T13:29:09.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.760+0000 7fd7dfa00740 1 --2- 172.21.15.12:0/154572204 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f299bb3ba0 0x55f299bb5fc0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.760+0000 7fd7dfa00740 1 --2- 172.21.15.12:0/154572204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f299b3ca80 0x55f299b37410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.760+0000 7fd7dfa00740 1 --2- 172.21.15.12:0/154572204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f299b33ea0 0x55f299b36ed0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.760+0000 7fd7bdffb640 0 client.17607 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:09.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.760+0000 7fd7dfa00740 1 --2- 172.21.15.12:0/154572204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f299b33580 0x55f299b36990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.760+0000 7fd7dfa00740 1 -- 172.21.15.12:0/154572204 >> 172.21.15.12:0/154572204 conn(0x55f299b2dcb0 msgr2=0x55f299b2f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:09.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.760+0000 7fd7dfa00740 1 -- 172.21.15.12:0/154572204 shutdown_connections 2024-04-22T13:29:09.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.760+0000 7fd7dfa00740 1 -- 172.21.15.12:0/154572204 wait complete. 2024-04-22T13:29:09.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.760+0000 7fd7dfa00740 20 client.17607 trim_cache size 0 max 16384 2024-04-22T13:29:09.764 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:29:09.764 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-04-22T13:29:09.764 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-04-22T13:29:09.764 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:29:09.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.760+0000 7fd7dce11640 1 -- 172.21.15.12:0/3826029485 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd7c00a3d50 msgr2=0x7fd7c00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:09.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.761+0000 7fd7dce11640 1 --2- 172.21.15.12:0/3826029485 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd7c00a3d50 0x7fd7c00a6210 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0021000 tx=0x7fd7d001e000 comp rx=0 tx=0).stop 2024-04-22T13:29:09.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.761+0000 7fd7dce11640 1 -- 172.21.15.12:0/3826029485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7d8101080 msgr2=0x7fd7d810e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:09.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.761+0000 7fd7dce11640 1 --2- 172.21.15.12:0/3826029485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7d8101080 0x7fd7d810e6b0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8005e30 tx=0x7fd7c80346f0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.761+0000 7fd7dce11640 1 -- 172.21.15.12:0/3826029485 shutdown_connections 2024-04-22T13:29:09.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.761+0000 7fd7dce11640 1 --2- 172.21.15.12:0/3826029485 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd7c00a3d50 0x7fd7c00a6210 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.761+0000 7fd7dce11640 1 --2- 172.21.15.12:0/3826029485 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7d810f130 0x7fd7d81a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.761+0000 7fd7dce11640 1 --2- 172.21.15.12:0/3826029485 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7d8106150 0x7fd7d810ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.761+0000 7fd7dce11640 1 --2- 172.21.15.12:0/3826029485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd7d8101080 0x7fd7d810e6b0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.761+0000 7fd7dce11640 1 -- 172.21.15.12:0/3826029485 >> 172.21.15.12:0/3826029485 conn(0x7fd7d8078000 msgr2=0x7fd7d80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:09.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.761+0000 7fd7dce11640 1 -- 172.21.15.12:0/3826029485 shutdown_connections 2024-04-22T13:29:09.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.761+0000 7fd7dce11640 1 -- 172.21.15.12:0/3826029485 wait complete. 2024-04-22T13:29:09.766 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:09 smithi012 ceph-mon[28265]: pgmap v154: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-22T13:29:09.767 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:29:09.768 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:29:09.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.875+0000 7f84785f4640 1 -- 172.21.15.12:0/2114413777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8470100e50 msgr2=0x7f84701012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:09.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.875+0000 7f84785f4640 1 --2- 172.21.15.12:0/2114413777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8470100e50 0x7f84701012b0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f8460009900 tx=0x7f846002bcd0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.875+0000 7f84785f4640 1 -- 172.21.15.12:0/2114413777 shutdown_connections 2024-04-22T13:29:09.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.875+0000 7f84785f4640 1 --2- 172.21.15.12:0/2114413777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84701017f0 0x7f847010dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.875+0000 7f84785f4640 1 --2- 172.21.15.12:0/2114413777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8470100e50 0x7f84701012b0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.875+0000 7f84785f4640 1 --2- 172.21.15.12:0/2114413777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8470106f10 0x7f84701072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.875+0000 7f84785f4640 1 -- 172.21.15.12:0/2114413777 >> 172.21.15.12:0/2114413777 conn(0x7f8470077f80 msgr2=0x7f8470078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:09.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.875+0000 7f84785f4640 1 -- 172.21.15.12:0/2114413777 shutdown_connections 2024-04-22T13:29:09.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.875+0000 7f84785f4640 1 -- 172.21.15.12:0/2114413777 wait complete. 2024-04-22T13:29:09.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.875+0000 7f84785f4640 1 Processor -- start 2024-04-22T13:29:09.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.876+0000 7f84785f4640 1 -- start start 2024-04-22T13:29:09.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.876+0000 7f84785f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8470100e50 0x7f84701986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.876+0000 7f84785f4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84701017f0 0x7f8470198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.876+0000 7f84785f4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8470106f10 0x7f847019cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.876+0000 7f84785f4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f84701121d0 con 0x7f8470106f10 2024-04-22T13:29:09.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.876+0000 7f84785f4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8470112050 con 0x7f84701017f0 2024-04-22T13:29:09.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.876+0000 7f84785f4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8470112350 con 0x7f8470100e50 2024-04-22T13:29:09.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.876+0000 7f8476b91640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8470106f10 0x7f847019cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.876+0000 7f8476390640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8470100e50 0x7f84701986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.876+0000 7f8476b91640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8470106f10 0x7f847019cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52218/0 (socket says 172.21.15.12:52218) 2024-04-22T13:29:09.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.876+0000 7f8476b91640 1 -- 172.21.15.12:0/2054897409 learned_addr learned my addr 172.21.15.12:0/2054897409 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:09.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.876+0000 7f8475b8f640 1 --2- 172.21.15.12:0/2054897409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84701017f0 0x7f8470198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.876+0000 7f8476b91640 1 -- 172.21.15.12:0/2054897409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8470100e50 msgr2=0x7f84701986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:09.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.876+0000 7f8476b91640 1 --2- 172.21.15.12:0/2054897409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8470100e50 0x7f84701986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.876+0000 7f8476b91640 1 -- 172.21.15.12:0/2054897409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84701017f0 msgr2=0x7f8470198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:09.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.876+0000 7f8476b91640 1 --2- 172.21.15.12:0/2054897409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84701017f0 0x7f8470198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:09.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.876+0000 7f8476b91640 1 -- 172.21.15.12:0/2054897409 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f847019d6c0 con 0x7f8470106f10 2024-04-22T13:29:09.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.876+0000 7f8476390640 1 --2- 172.21.15.12:0/2054897409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8470100e50 0x7f84701986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:29:09.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.876+0000 7f8476b91640 1 --2- 172.21.15.12:0/2054897409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8470106f10 0x7f847019cf90 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f846800cab0 tx=0x7f846800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:09.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.877+0000 7f84677fe640 1 -- 172.21.15.12:0/2054897409 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846800adc0 con 0x7f8470106f10 2024-04-22T13:29:09.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.877+0000 7f84785f4640 1 -- 172.21.15.12:0/2054897409 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f847019d9b0 con 0x7f8470106f10 2024-04-22T13:29:09.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.877+0000 7f84677fe640 1 -- 172.21.15.12:0/2054897409 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f846800f900 con 0x7f8470106f10 2024-04-22T13:29:09.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.877+0000 7f84677fe640 1 -- 172.21.15.12:0/2054897409 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84680189a0 con 0x7f8470106f10 2024-04-22T13:29:09.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.877+0000 7f84785f4640 1 -- 172.21.15.12:0/2054897409 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84701a5240 con 0x7f8470106f10 2024-04-22T13:29:09.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.877+0000 7f84785f4640 1 -- 172.21.15.12:0/2054897409 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8470068550 con 0x7f8470106f10 2024-04-22T13:29:09.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.879+0000 7f84677fe640 1 -- 172.21.15.12:0/2054897409 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f846800f450 con 0x7f8470106f10 2024-04-22T13:29:09.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.880+0000 7f84677fe640 1 --2- 172.21.15.12:0/2054897409 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f845c0a3d20 0x7f845c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:09.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.880+0000 7f8476390640 1 --2- 172.21.15.12:0/2054897409 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f845c0a3d20 0x7f845c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:09.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.880+0000 7f84677fe640 1 -- 172.21.15.12:0/2054897409 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f84680d0b40 con 0x7f8470106f10 2024-04-22T13:29:09.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.880+0000 7f8476390640 1 --2- 172.21.15.12:0/2054897409 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f845c0a3d20 0x7f845c0a61e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f846c005fd0 tx=0x7f846c00b390 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:09.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:09.884+0000 7f84677fe640 1 -- 172.21.15.12:0/2054897409 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8468014030 con 0x7f8470106f10 2024-04-22T13:29:09.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:09 smithi134 ceph-mon[31338]: pgmap v154: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-22T13:29:09.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:09 smithi111 ceph-mon[31361]: pgmap v154: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-22T13:29:10.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.110+0000 7f84785f4640 1 -- 172.21.15.12:0/2054897409 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f847002d5c0 con 0x7f8470106f10 2024-04-22T13:29:10.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.111+0000 7f84677fe640 1 -- 172.21.15.12:0/2054897409 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7f846809d2c0 con 0x7f8470106f10 2024-04-22T13:29:10.107 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:29:10.107 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:29:10.107 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:29:10.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.113+0000 7f84785f4640 1 -- 172.21.15.12:0/2054897409 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f845c0a3d20 msgr2=0x7f845c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.113+0000 7f84785f4640 1 --2- 172.21.15.12:0/2054897409 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f845c0a3d20 0x7f845c0a61e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f846c005fd0 tx=0x7f846c00b390 comp rx=0 tx=0).stop 2024-04-22T13:29:10.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.113+0000 7f84785f4640 1 -- 172.21.15.12:0/2054897409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8470106f10 msgr2=0x7f847019cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.113+0000 7f84785f4640 1 --2- 172.21.15.12:0/2054897409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8470106f10 0x7f847019cf90 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f846800cab0 tx=0x7f846800cf70 comp rx=0 tx=0).stop 2024-04-22T13:29:10.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.113+0000 7f84785f4640 1 -- 172.21.15.12:0/2054897409 shutdown_connections 2024-04-22T13:29:10.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.113+0000 7f84785f4640 1 --2- 172.21.15.12:0/2054897409 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f845c0a3d20 0x7f845c0a61e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.113+0000 7f84785f4640 1 --2- 172.21.15.12:0/2054897409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8470106f10 0x7f847019cf90 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.113+0000 7f84785f4640 1 --2- 172.21.15.12:0/2054897409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84701017f0 0x7f8470198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.113+0000 7f84785f4640 1 --2- 172.21.15.12:0/2054897409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8470100e50 0x7f84701986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.113+0000 7f84785f4640 1 -- 172.21.15.12:0/2054897409 >> 172.21.15.12:0/2054897409 conn(0x7f8470077f80 msgr2=0x7f84700ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:10.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.113+0000 7f84785f4640 1 -- 172.21.15.12:0/2054897409 shutdown_connections 2024-04-22T13:29:10.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.113+0000 7f84785f4640 1 -- 172.21.15.12:0/2054897409 wait complete. 2024-04-22T13:29:10.120 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:29:10.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.229+0000 7f44844d2640 1 -- 172.21.15.12:0/3682652936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f447c102190 msgr2=0x7f447c102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.229+0000 7f44844d2640 1 --2- 172.21.15.12:0/3682652936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f447c102190 0x7f447c102570 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f4478009900 tx=0x7f447802bc30 comp rx=0 tx=0).stop 2024-04-22T13:29:10.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.230+0000 7f44844d2640 1 -- 172.21.15.12:0/3682652936 shutdown_connections 2024-04-22T13:29:10.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.230+0000 7f44844d2640 1 --2- 172.21.15.12:0/3682652936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f447c107260 0x7f447c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.230+0000 7f44844d2640 1 --2- 172.21.15.12:0/3682652936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f447c102b40 0x7f447c106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.230+0000 7f44844d2640 1 --2- 172.21.15.12:0/3682652936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f447c102190 0x7f447c102570 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.230+0000 7f44844d2640 1 -- 172.21.15.12:0/3682652936 >> 172.21.15.12:0/3682652936 conn(0x7f447c0fd990 msgr2=0x7f447c0ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:10.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.230+0000 7f44844d2640 1 -- 172.21.15.12:0/3682652936 shutdown_connections 2024-04-22T13:29:10.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.230+0000 7f44844d2640 1 -- 172.21.15.12:0/3682652936 wait complete. 2024-04-22T13:29:10.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.230+0000 7f44844d2640 1 Processor -- start 2024-04-22T13:29:10.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.230+0000 7f44844d2640 1 -- start start 2024-04-22T13:29:10.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.230+0000 7f44844d2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f447c102190 0x7f447c074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.230+0000 7f44844d2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f447c107260 0x7f447c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.230+0000 7f44844d2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f447c074cd0 0x7f447c1a6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.230+0000 7f44844d2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f447c1121f0 con 0x7f447c107260 2024-04-22T13:29:10.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.230+0000 7f44844d2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f447c112070 con 0x7f447c074cd0 2024-04-22T13:29:10.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.230+0000 7f44844d2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f447c112370 con 0x7f447c102190 2024-04-22T13:29:10.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.231+0000 7f4481a6d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f447c107260 0x7f447c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.231+0000 7f4481a6d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f447c107260 0x7f447c074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52244/0 (socket says 172.21.15.12:52244) 2024-04-22T13:29:10.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.231+0000 7f4481a6d640 1 -- 172.21.15.12:0/464594752 learned_addr learned my addr 172.21.15.12:0/464594752 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:10.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.231+0000 7f4482a6f640 1 --2- 172.21.15.12:0/464594752 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f447c074cd0 0x7f447c1a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.231+0000 7f448226e640 1 --2- 172.21.15.12:0/464594752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f447c102190 0x7f447c074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.231+0000 7f4481a6d640 1 -- 172.21.15.12:0/464594752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f447c102190 msgr2=0x7f447c074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.231+0000 7f4481a6d640 1 --2- 172.21.15.12:0/464594752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f447c102190 0x7f447c074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.231+0000 7f4481a6d640 1 -- 172.21.15.12:0/464594752 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f447c074cd0 msgr2=0x7f447c1a6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.231+0000 7f4481a6d640 1 --2- 172.21.15.12:0/464594752 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f447c074cd0 0x7f447c1a6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.231+0000 7f4481a6d640 1 -- 172.21.15.12:0/464594752 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447c1a7040 con 0x7f447c107260 2024-04-22T13:29:10.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.231+0000 7f4482a6f640 1 --2- 172.21.15.12:0/464594752 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f447c074cd0 0x7f447c1a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:10.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.231+0000 7f4481a6d640 1 --2- 172.21.15.12:0/464594752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f447c107260 0x7f447c074790 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f446c005fd0 tx=0x7f446c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:10.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.231+0000 7f44737fe640 1 -- 172.21.15.12:0/464594752 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c017070 con 0x7f447c107260 2024-04-22T13:29:10.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.231+0000 7f44737fe640 1 -- 172.21.15.12:0/464594752 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f446c009c30 con 0x7f447c107260 2024-04-22T13:29:10.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.231+0000 7f44844d2640 1 -- 172.21.15.12:0/464594752 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f447c1a7330 con 0x7f447c107260 2024-04-22T13:29:10.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.231+0000 7f44737fe640 1 -- 172.21.15.12:0/464594752 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c003d30 con 0x7f447c107260 2024-04-22T13:29:10.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.231+0000 7f44844d2640 1 -- 172.21.15.12:0/464594752 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f447c1a7880 con 0x7f447c107260 2024-04-22T13:29:10.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.234+0000 7f44871a0740 1 Processor -- start 2024-04-22T13:29:10.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.234+0000 7f44871a0740 1 -- start start 2024-04-22T13:29:10.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.235+0000 7f44737fe640 1 -- 172.21.15.12:0/464594752 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f446c003ed0 con 0x7f447c107260 2024-04-22T13:29:10.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.235+0000 7f44871a0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a86e5304d0 0x55a86e5308b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.235+0000 7f44871a0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a86e530e80 0x55a86e539430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.235+0000 7f44871a0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a86e539a00 0x55a86e53be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.235+0000 7f44871a0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a86e4951f0 con 0x55a86e530e80 2024-04-22T13:29:10.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.235+0000 7f44871a0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a86e4324b0 con 0x55a86e539a00 2024-04-22T13:29:10.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.235+0000 7f44871a0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a86e4911b0 con 0x55a86e5304d0 2024-04-22T13:29:10.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.235+0000 7f448226e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a86e530e80 0x55a86e539430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.235+0000 7f448226e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a86e530e80 0x55a86e539430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52260/0 (socket says 172.21.15.12:52260) 2024-04-22T13:29:10.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.235+0000 7f448226e640 1 -- 172.21.15.12:0/12599722 learned_addr learned my addr 172.21.15.12:0/12599722 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:10.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.235+0000 7f4482a6f640 1 --2- 172.21.15.12:0/12599722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a86e5304d0 0x55a86e5308b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.235+0000 7f4481a6d640 1 --2- 172.21.15.12:0/12599722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a86e539a00 0x55a86e53be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.235+0000 7f448226e640 1 -- 172.21.15.12:0/12599722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a86e5304d0 msgr2=0x55a86e5308b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.235+0000 7f448226e640 1 --2- 172.21.15.12:0/12599722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a86e5304d0 0x55a86e5308b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.235+0000 7f448226e640 1 -- 172.21.15.12:0/12599722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a86e539a00 msgr2=0x55a86e53be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.235+0000 7f448226e640 1 --2- 172.21.15.12:0/12599722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a86e539a00 0x55a86e53be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.235+0000 7f448226e640 1 -- 172.21.15.12:0/12599722 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a86e4241b0 con 0x55a86e530e80 2024-04-22T13:29:10.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.235+0000 7f4481a6d640 1 --2- 172.21.15.12:0/12599722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a86e539a00 0x55a86e53be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:10.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.235+0000 7f44737fe640 1 --2- 172.21.15.12:0/464594752 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f44680a3d50 0x7f44680a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.235+0000 7f448226e640 1 --2- 172.21.15.12:0/12599722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a86e530e80 0x55a86e539430 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f4478009900 tx=0x7f4478005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b78344fb388669 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:10.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.236+0000 7f4482a6f640 1 --2- 172.21.15.12:0/464594752 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f44680a3d50 0x7f44680a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.236+0000 7f44737fe640 1 -- 172.21.15.12:0/464594752 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f446c0d04c0 con 0x7f447c107260 2024-04-22T13:29:10.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.236+0000 7f4470bfe640 1 -- 172.21.15.12:0/12599722 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4478044020 con 0x55a86e530e80 2024-04-22T13:29:10.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.236+0000 7f4470bfe640 1 -- 172.21.15.12:0/12599722 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f447802cc90 con 0x55a86e530e80 2024-04-22T13:29:10.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.236+0000 7f4470bfe640 1 -- 172.21.15.12:0/12599722 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44780046c0 con 0x55a86e530e80 2024-04-22T13:29:10.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.236+0000 7f44871a0740 1 -- 172.21.15.12:0/12599722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a86e530e80 msgr2=0x55a86e539430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.236+0000 7f4482a6f640 1 --2- 172.21.15.12:0/464594752 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f44680a3d50 0x7f44680a6210 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4474005fd0 tx=0x7f4474009780 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:10.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.236+0000 7f44871a0740 1 --2- 172.21.15.12:0/12599722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a86e530e80 0x55a86e539430 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f4478009900 tx=0x7f4478005e80 comp rx=0 tx=0).stop 2024-04-22T13:29:10.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.236+0000 7f44871a0740 1 -- 172.21.15.12:0/12599722 shutdown_connections 2024-04-22T13:29:10.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.236+0000 7f44871a0740 1 --2- 172.21.15.12:0/12599722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a86e539a00 0x55a86e53be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.236+0000 7f44871a0740 1 --2- 172.21.15.12:0/12599722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a86e530e80 0x55a86e539430 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.236+0000 7f44871a0740 1 --2- 172.21.15.12:0/12599722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a86e5304d0 0x55a86e5308b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.237+0000 7f44871a0740 1 -- 172.21.15.12:0/12599722 >> 172.21.15.12:0/12599722 conn(0x55a86e52aba0 msgr2=0x55a86e52af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:10.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.237+0000 7f44871a0740 1 -- 172.21.15.12:0/12599722 shutdown_connections 2024-04-22T13:29:10.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.237+0000 7f44871a0740 1 -- 172.21.15.12:0/12599722 wait complete. 2024-04-22T13:29:10.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.237+0000 7f44871a0740 1 -- start start 2024-04-22T13:29:10.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.237+0000 7f44871a0740 1 Processor -- start 2024-04-22T13:29:10.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.237+0000 7f44871a0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a86e5304d0 0x55a86e5392d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.237+0000 7f44871a0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a86e530e80 0x55a86e536660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.237+0000 7f44871a0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a86e539a00 0x55a86e536bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.237+0000 7f44871a0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a86e53c910 con 0x55a86e5304d0 2024-04-22T13:29:10.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.237+0000 7f44871a0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a86e44e500 con 0x55a86e530e80 2024-04-22T13:29:10.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.237+0000 7f44871a0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a86e53ca90 con 0x55a86e539a00 2024-04-22T13:29:10.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.237+0000 7f448226e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a86e5304d0 0x55a86e5392d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.237+0000 7f44871a0740 10 client.0 _ll_register_callbacks cb 0x55a86e41cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:10.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.237+0000 7f448226e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a86e5304d0 0x55a86e5392d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52264/0 (socket says 172.21.15.12:52264) 2024-04-22T13:29:10.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.237+0000 7f448226e640 1 -- 172.21.15.12:0/3767607512 learned_addr learned my addr 172.21.15.12:0/3767607512 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:10.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.238+0000 7f4482a6f640 1 --2- 172.21.15.12:0/3767607512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a86e539a00 0x55a86e536bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.238+0000 7f448226e640 1 -- 172.21.15.12:0/3767607512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a86e539a00 msgr2=0x55a86e536bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.238+0000 7f448226e640 1 --2- 172.21.15.12:0/3767607512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a86e539a00 0x55a86e536bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.238+0000 7f448226e640 1 -- 172.21.15.12:0/3767607512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a86e530e80 msgr2=0x55a86e536660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.238+0000 7f4481a6d640 1 --2- 172.21.15.12:0/3767607512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a86e530e80 0x55a86e536660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.238+0000 7f448226e640 1 --2- 172.21.15.12:0/3767607512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a86e530e80 0x55a86e536660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.238+0000 7f448226e640 1 -- 172.21.15.12:0/3767607512 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a86e1a73f0 con 0x55a86e5304d0 2024-04-22T13:29:10.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.238+0000 7f4482a6f640 1 --2- 172.21.15.12:0/3767607512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a86e539a00 0x55a86e536bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:10.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.238+0000 7f448226e640 1 --2- 172.21.15.12:0/3767607512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a86e5304d0 0x55a86e5392d0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f447802c170 tx=0x7f447802cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db9914f0cffad268 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:10.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.238+0000 7f4465ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:10.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.238+0000 7f4465ffb640 1 -- 172.21.15.12:0/3767607512 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447804d030 con 0x55a86e5304d0 2024-04-22T13:29:10.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.238+0000 7f4465ffb640 1 -- 172.21.15.12:0/3767607512 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f4478004390 con 0x55a86e5304d0 2024-04-22T13:29:10.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.238+0000 7f44871a0740 1 -- 172.21.15.12:0/3767607512 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f447802cc90 con 0x55a86e5304d0 2024-04-22T13:29:10.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.238+0000 7f4465ffb640 1 -- 172.21.15.12:0/3767607512 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4478056020 con 0x55a86e5304d0 2024-04-22T13:29:10.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.238+0000 7f44871a0740 10 client.17625 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:10.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.238+0000 7f44871a0740 1 -- 172.21.15.12:0/3767607512 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a86e2717d0 con 0x55a86e5304d0 2024-04-22T13:29:10.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.238+0000 7f4465ffb640 1 -- 172.21.15.12:0/3767607512 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f447802cc90 con 0x55a86e5304d0 2024-04-22T13:29:10.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.239+0000 7f4465ffb640 1 -- 172.21.15.12:0/3767607512 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f44780407b0 con 0x55a86e5304d0 2024-04-22T13:29:10.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.239+0000 7f44871a0740 10 client.17625 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:10.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.239+0000 7f44871a0740 10 client.17625 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:29:10.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.239+0000 7f44871a0740 20 client.17625 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:10.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.239+0000 7f44871a0740 1 --2- 172.21.15.12:0/3767607512 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55a86e549a00 0x55a86e54bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.239+0000 7f44871a0740 4 client.17625 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:10.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.239+0000 7f44871a0740 1 -- 172.21.15.12:0/3767607512 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a86e434930 con 0x55a86e549a00 2024-04-22T13:29:10.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.239+0000 7f4481a6d640 1 --2- 172.21.15.12:0/3767607512 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55a86e549a00 0x55a86e54bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.239+0000 7f4481a6d640 1 --2- 172.21.15.12:0/3767607512 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55a86e549a00 0x55a86e54bdf0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=552482dccb8648d1 server_cookie=7efe4fa1584473b1 in_seq=0 out_seq=0 2024-04-22T13:29:10.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.239+0000 7f4465ffb640 10 client.17625 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:29:10.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.240+0000 7f4465ffb640 1 -- 172.21.15.12:0/3767607512 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a86e434930 con 0x55a86e549a00 2024-04-22T13:29:10.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.240+0000 7f4465ffb640 10 client.17625 handle_command_reply: tid=0 2024-04-22T13:29:10.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.240+0000 7f44871a0740 1 client.17625 shutdown 2024-04-22T13:29:10.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.240+0000 7f44871a0740 1 -- 172.21.15.12:0/3767607512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a86e5304d0 msgr2=0x55a86e5392d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.240+0000 7f44871a0740 1 --2- 172.21.15.12:0/3767607512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a86e5304d0 0x55a86e5392d0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f447802c170 tx=0x7f447802cb20 comp rx=0 tx=0).stop 2024-04-22T13:29:10.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.240+0000 7f44871a0740 1 -- 172.21.15.12:0/3767607512 shutdown_connections 2024-04-22T13:29:10.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.240+0000 7f44871a0740 1 --2- 172.21.15.12:0/3767607512 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55a86e549a00 0x55a86e54bdf0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.240+0000 7f44871a0740 1 --2- 172.21.15.12:0/3767607512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a86e539a00 0x55a86e536bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.240+0000 7f44871a0740 1 --2- 172.21.15.12:0/3767607512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a86e530e80 0x55a86e536660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.240+0000 7f44871a0740 1 --2- 172.21.15.12:0/3767607512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a86e5304d0 0x55a86e5392d0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.240+0000 7f4465ffb640 0 client.17625 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:29:10.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.240+0000 7f44871a0740 1 -- 172.21.15.12:0/3767607512 >> 172.21.15.12:0/3767607512 conn(0x55a86e52aba0 msgr2=0x55a86e52c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:10.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.240+0000 7f44871a0740 1 -- 172.21.15.12:0/3767607512 shutdown_connections 2024-04-22T13:29:10.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.240+0000 7f44871a0740 1 -- 172.21.15.12:0/3767607512 wait complete. 2024-04-22T13:29:10.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.240+0000 7f44871a0740 20 client.17625 trim_cache size 0 max 16384 2024-04-22T13:29:10.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.254+0000 7f44871a0740 1 Processor -- start 2024-04-22T13:29:10.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.254+0000 7f44871a0740 1 -- start start 2024-04-22T13:29:10.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.254+0000 7f44871a0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a86e5304d0 0x55a86e5457d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.254+0000 7f44871a0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a86e530e80 0x55a86e5357e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.254+0000 7f44871a0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a86e539a00 0x55a86e52c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.254+0000 7f44871a0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f447c1afd00 con 0x55a86e530e80 2024-04-22T13:29:10.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.254+0000 7f44871a0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f447c1b1840 con 0x55a86e5304d0 2024-04-22T13:29:10.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.254+0000 7f44871a0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f447c1b03d0 con 0x55a86e539a00 2024-04-22T13:29:10.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.254+0000 7f4481a6d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a86e530e80 0x55a86e5357e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.254+0000 7f4481a6d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a86e530e80 0x55a86e5357e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52266/0 (socket says 172.21.15.12:52266) 2024-04-22T13:29:10.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.254+0000 7f4481a6d640 1 -- 172.21.15.12:0/2785931395 learned_addr learned my addr 172.21.15.12:0/2785931395 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:10.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.254+0000 7f4482a6f640 1 --2- 172.21.15.12:0/2785931395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a86e539a00 0x55a86e52c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.254+0000 7f448226e640 1 --2- 172.21.15.12:0/2785931395 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a86e5304d0 0x55a86e5457d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.254+0000 7f4481a6d640 1 -- 172.21.15.12:0/2785931395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a86e539a00 msgr2=0x55a86e52c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.254+0000 7f4481a6d640 1 --2- 172.21.15.12:0/2785931395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a86e539a00 0x55a86e52c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.254+0000 7f4481a6d640 1 -- 172.21.15.12:0/2785931395 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a86e5304d0 msgr2=0x55a86e5457d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.254+0000 7f4481a6d640 1 --2- 172.21.15.12:0/2785931395 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a86e5304d0 0x55a86e5457d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.254+0000 7f4481a6d640 1 -- 172.21.15.12:0/2785931395 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a86e52b520 con 0x55a86e530e80 2024-04-22T13:29:10.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.254+0000 7f448226e640 1 --2- 172.21.15.12:0/2785931395 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a86e5304d0 0x55a86e5457d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:10.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.255+0000 7f4481a6d640 1 --2- 172.21.15.12:0/2785931395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a86e530e80 0x55a86e5357e0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f446c098e80 tx=0x7f446c005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d6b015382a4d21 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:10.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.255+0000 7f4467fff640 1 -- 172.21.15.12:0/2785931395 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c09be10 con 0x55a86e530e80 2024-04-22T13:29:10.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.255+0000 7f4467fff640 1 -- 172.21.15.12:0/2785931395 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f446c09cbb0 con 0x55a86e530e80 2024-04-22T13:29:10.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.255+0000 7f4467fff640 1 -- 172.21.15.12:0/2785931395 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c09a7b0 con 0x55a86e530e80 2024-04-22T13:29:10.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.256+0000 7f44871a0740 1 -- 172.21.15.12:0/2785931395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a86e530e80 msgr2=0x55a86e5357e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.256+0000 7f44871a0740 1 --2- 172.21.15.12:0/2785931395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a86e530e80 0x55a86e5357e0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f446c098e80 tx=0x7f446c005db0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.256+0000 7f44871a0740 1 -- 172.21.15.12:0/2785931395 shutdown_connections 2024-04-22T13:29:10.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.256+0000 7f44871a0740 1 --2- 172.21.15.12:0/2785931395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a86e539a00 0x55a86e52c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.256+0000 7f44871a0740 1 --2- 172.21.15.12:0/2785931395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a86e530e80 0x55a86e5357e0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.256+0000 7f44871a0740 1 --2- 172.21.15.12:0/2785931395 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a86e5304d0 0x55a86e5457d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.256+0000 7f44871a0740 1 -- 172.21.15.12:0/2785931395 >> 172.21.15.12:0/2785931395 conn(0x55a86e52aba0 msgr2=0x55a86e52bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:10.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.256+0000 7f44871a0740 1 -- 172.21.15.12:0/2785931395 shutdown_connections 2024-04-22T13:29:10.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.256+0000 7f44871a0740 1 -- 172.21.15.12:0/2785931395 wait complete. 2024-04-22T13:29:10.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.256+0000 7f44871a0740 1 -- start start 2024-04-22T13:29:10.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.256+0000 7f44871a0740 1 Processor -- start 2024-04-22T13:29:10.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.256+0000 7f44871a0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a86e5304d0 0x55a86e541ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.256+0000 7f44871a0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a86e530e80 0x55a86e532ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.256+0000 7f44871a0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a86e539a00 0x55a86e533430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.256+0000 7f44871a0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f447c1b1170 con 0x55a86e530e80 2024-04-22T13:29:10.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.256+0000 7f44871a0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f447c1b0aa0 con 0x55a86e539a00 2024-04-22T13:29:10.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.256+0000 7f44871a0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f447c1add00 con 0x55a86e5304d0 2024-04-22T13:29:10.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.256+0000 7f44871a0740 10 client.0 _ll_register_callbacks cb 0x7f4478005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:10.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.256+0000 7f4481a6d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a86e530e80 0x55a86e532ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.256+0000 7f4481a6d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a86e530e80 0x55a86e532ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52274/0 (socket says 172.21.15.12:52274) 2024-04-22T13:29:10.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.256+0000 7f4481a6d640 1 -- 172.21.15.12:0/2239817138 learned_addr learned my addr 172.21.15.12:0/2239817138 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:10.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.257+0000 7f4482a6f640 1 --2- 172.21.15.12:0/2239817138 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a86e539a00 0x55a86e533430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.257+0000 7f448226e640 1 --2- 172.21.15.12:0/2239817138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a86e5304d0 0x55a86e541ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.257+0000 7f4481a6d640 1 -- 172.21.15.12:0/2239817138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a86e5304d0 msgr2=0x55a86e541ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.257+0000 7f4481a6d640 1 --2- 172.21.15.12:0/2239817138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a86e5304d0 0x55a86e541ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.257+0000 7f4481a6d640 1 -- 172.21.15.12:0/2239817138 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a86e539a00 msgr2=0x55a86e533430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.257+0000 7f4481a6d640 1 --2- 172.21.15.12:0/2239817138 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a86e539a00 0x55a86e533430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.257+0000 7f4481a6d640 1 -- 172.21.15.12:0/2239817138 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a86e5339a0 con 0x55a86e530e80 2024-04-22T13:29:10.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.257+0000 7f4482a6f640 1 --2- 172.21.15.12:0/2239817138 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a86e539a00 0x55a86e533430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:10.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.257+0000 7f4481a6d640 1 --2- 172.21.15.12:0/2239817138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a86e530e80 0x55a86e532ef0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f446c098910 tx=0x7f446c09be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbead2921ba4f5dc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:10.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.257+0000 7f4465ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:10.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.257+0000 7f4465ffb640 1 -- 172.21.15.12:0/2239817138 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f446c0a0860 con 0x55a86e530e80 2024-04-22T13:29:10.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.257+0000 7f4465ffb640 1 -- 172.21.15.12:0/2239817138 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f446c09a7b0 con 0x55a86e530e80 2024-04-22T13:29:10.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.257+0000 7f44871a0740 1 -- 172.21.15.12:0/2239817138 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f446c09cbb0 con 0x55a86e530e80 2024-04-22T13:29:10.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.257+0000 7f4465ffb640 1 -- 172.21.15.12:0/2239817138 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f446c09c4b0 con 0x55a86e530e80 2024-04-22T13:29:10.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.257+0000 7f44871a0740 10 client.17631 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:10.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.257+0000 7f44871a0740 1 -- 172.21.15.12:0/2239817138 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a86e5af9a0 con 0x55a86e530e80 2024-04-22T13:29:10.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.257+0000 7f4465ffb640 1 -- 172.21.15.12:0/2239817138 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f446c09cbb0 con 0x55a86e530e80 2024-04-22T13:29:10.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.257+0000 7f4465ffb640 1 -- 172.21.15.12:0/2239817138 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f44780054d0 con 0x55a86e530e80 2024-04-22T13:29:10.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.258+0000 7f44871a0740 10 client.17631 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:10.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.258+0000 7f44871a0740 10 client.17631 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:29:10.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.258+0000 7f44871a0740 20 client.17631 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:10.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.258+0000 7f44871a0740 1 --2- 172.21.15.12:0/2239817138 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55a86e5b0100 0x55a86e5b2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.258+0000 7f44871a0740 4 client.17631 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:29:10.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.258+0000 7f44871a0740 1 -- 172.21.15.12:0/2239817138 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a86e4a2c10 con 0x55a86e5b0100 2024-04-22T13:29:10.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.258+0000 7f448226e640 1 --2- 172.21.15.12:0/2239817138 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55a86e5b0100 0x55a86e5b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.258+0000 7f4465ffb640 10 client.17631 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:29:10.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.258+0000 7f448226e640 1 --2- 172.21.15.12:0/2239817138 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55a86e5b0100 0x55a86e5b2520 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e09ff99ecafdfbdc server_cookie=5144a8d0203c9615 in_seq=0 out_seq=0 2024-04-22T13:29:10.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.258+0000 7f4465ffb640 1 -- 172.21.15.12:0/2239817138 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a86e4a2c10 con 0x55a86e5b0100 2024-04-22T13:29:10.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.258+0000 7f4465ffb640 10 client.17631 handle_command_reply: tid=0 2024-04-22T13:29:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.258+0000 7f44871a0740 1 client.17631 shutdown 2024-04-22T13:29:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.258+0000 7f44871a0740 1 -- 172.21.15.12:0/2239817138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a86e530e80 msgr2=0x55a86e532ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.258+0000 7f44871a0740 1 --2- 172.21.15.12:0/2239817138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a86e530e80 0x55a86e532ef0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f446c098910 tx=0x7f446c09be10 comp rx=0 tx=0).stop 2024-04-22T13:29:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.258+0000 7f44871a0740 1 -- 172.21.15.12:0/2239817138 shutdown_connections 2024-04-22T13:29:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.258+0000 7f44871a0740 1 --2- 172.21.15.12:0/2239817138 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55a86e5b0100 0x55a86e5b2520 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.258+0000 7f44871a0740 1 --2- 172.21.15.12:0/2239817138 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a86e539a00 0x55a86e533430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.258+0000 7f44871a0740 1 --2- 172.21.15.12:0/2239817138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a86e530e80 0x55a86e532ef0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.258+0000 7f4465ffb640 0 client.17631 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:29:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.258+0000 7f44871a0740 1 --2- 172.21.15.12:0/2239817138 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a86e5304d0 0x55a86e541ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.258+0000 7f44871a0740 1 -- 172.21.15.12:0/2239817138 >> 172.21.15.12:0/2239817138 conn(0x55a86e52aba0 msgr2=0x55a86e529370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.259+0000 7f44871a0740 1 -- 172.21.15.12:0/2239817138 shutdown_connections 2024-04-22T13:29:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.259+0000 7f44871a0740 1 -- 172.21.15.12:0/2239817138 wait complete. 2024-04-22T13:29:10.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.259+0000 7f44871a0740 20 client.17631 trim_cache size 0 max 16384 2024-04-22T13:29:10.262 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:29:10.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.259+0000 7f44844d2640 1 -- 172.21.15.12:0/464594752 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f44680a3d50 msgr2=0x7f44680a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.259+0000 7f44844d2640 1 --2- 172.21.15.12:0/464594752 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f44680a3d50 0x7f44680a6210 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4474005fd0 tx=0x7f4474009780 comp rx=0 tx=0).stop 2024-04-22T13:29:10.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.259+0000 7f44844d2640 1 -- 172.21.15.12:0/464594752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f447c107260 msgr2=0x7f447c074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.259+0000 7f44844d2640 1 --2- 172.21.15.12:0/464594752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f447c107260 0x7f447c074790 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f446c005fd0 tx=0x7f446c00dde0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.259+0000 7f44844d2640 1 -- 172.21.15.12:0/464594752 shutdown_connections 2024-04-22T13:29:10.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.259+0000 7f44844d2640 1 --2- 172.21.15.12:0/464594752 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f44680a3d50 0x7f44680a6210 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.259+0000 7f44844d2640 1 --2- 172.21.15.12:0/464594752 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f447c074cd0 0x7f447c1a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.259+0000 7f44844d2640 1 --2- 172.21.15.12:0/464594752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f447c107260 0x7f447c074790 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.259+0000 7f44844d2640 1 --2- 172.21.15.12:0/464594752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f447c102190 0x7f447c074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.259+0000 7f44844d2640 1 -- 172.21.15.12:0/464594752 >> 172.21.15.12:0/464594752 conn(0x7f447c0fd990 msgr2=0x7f447c0fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:10.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.259+0000 7f44844d2640 1 -- 172.21.15.12:0/464594752 shutdown_connections 2024-04-22T13:29:10.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.259+0000 7f44844d2640 1 -- 172.21.15.12:0/464594752 wait complete. 2024-04-22T13:29:10.265 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:29:10.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.374+0000 7f6ec1f23640 1 -- 172.21.15.12:0/4047631713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ebc106170 msgr2=0x7f6ebc10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.374+0000 7f6ec1f23640 1 --2- 172.21.15.12:0/4047631713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ebc106170 0x7f6ebc10dd40 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009ca0 tx=0x7f6eb402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.374+0000 7f6ec1f23640 1 -- 172.21.15.12:0/4047631713 shutdown_connections 2024-04-22T13:29:10.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.374+0000 7f6ec1f23640 1 --2- 172.21.15.12:0/4047631713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ebc106170 0x7f6ebc10dd40 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.374+0000 7f6ec1f23640 1 --2- 172.21.15.12:0/4047631713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ebc101a50 0x7f6ebc105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.374+0000 7f6ec1f23640 1 --2- 172.21.15.12:0/4047631713 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ebc1010a0 0x7f6ebc101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.374+0000 7f6ec1f23640 1 -- 172.21.15.12:0/4047631713 >> 172.21.15.12:0/4047631713 conn(0x7f6ebc077eb0 msgr2=0x7f6ebc0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:10.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.374+0000 7f6ec1f23640 1 -- 172.21.15.12:0/4047631713 shutdown_connections 2024-04-22T13:29:10.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.374+0000 7f6ec1f23640 1 -- 172.21.15.12:0/4047631713 wait complete. 2024-04-22T13:29:10.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.374+0000 7f6ec1f23640 1 Processor -- start 2024-04-22T13:29:10.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.375+0000 7f6ec1f23640 1 -- start start 2024-04-22T13:29:10.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.375+0000 7f6ec1f23640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ebc1010a0 0x7f6ebc199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.375+0000 7f6ec1f23640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ebc106170 0x7f6ebc1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.375+0000 7f6ec1f23640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ebc199f30 0x7f6ebc1a44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.375+0000 7f6ec1f23640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc1110c0 con 0x7f6ebc199f30 2024-04-22T13:29:10.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.375+0000 7f6ec1f23640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc110f40 con 0x7f6ebc106170 2024-04-22T13:29:10.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.375+0000 7f6ec1f23640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc111240 con 0x7f6ebc1010a0 2024-04-22T13:29:10.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.375+0000 7f6ebbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ebc199f30 0x7f6ebc1a44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.375+0000 7f6ebbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ebc199f30 0x7f6ebc1a44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52302/0 (socket says 172.21.15.12:52302) 2024-04-22T13:29:10.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.375+0000 7f6ebbfff640 1 -- 172.21.15.12:0/2827249664 learned_addr learned my addr 172.21.15.12:0/2827249664 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:10.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.375+0000 7f6ebb7fe640 1 --2- 172.21.15.12:0/2827249664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ebc1010a0 0x7f6ebc199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.375+0000 7f6ebaffd640 1 --2- 172.21.15.12:0/2827249664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ebc106170 0x7f6ebc1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.375+0000 7f6ebbfff640 1 -- 172.21.15.12:0/2827249664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ebc1010a0 msgr2=0x7f6ebc199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.375+0000 7f6ebbfff640 1 --2- 172.21.15.12:0/2827249664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ebc1010a0 0x7f6ebc199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.375+0000 7f6ebbfff640 1 -- 172.21.15.12:0/2827249664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ebc106170 msgr2=0x7f6ebc1999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.375+0000 7f6ebbfff640 1 --2- 172.21.15.12:0/2827249664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ebc106170 0x7f6ebc1999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.375+0000 7f6ebbfff640 1 -- 172.21.15.12:0/2827249664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ebc1a4ce0 con 0x7f6ebc199f30 2024-04-22T13:29:10.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.375+0000 7f6ebbfff640 1 --2- 172.21.15.12:0/2827249664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ebc199f30 0x7f6ebc1a44e0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4005e30 tx=0x7f6eb4034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:10.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.376+0000 7f6eb8ff9640 1 -- 172.21.15.12:0/2827249664 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4039040 con 0x7f6ebc199f30 2024-04-22T13:29:10.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.376+0000 7f6eb8ff9640 1 -- 172.21.15.12:0/2827249664 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eb4044070 con 0x7f6ebc199f30 2024-04-22T13:29:10.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.376+0000 7f6ec1f23640 1 -- 172.21.15.12:0/2827249664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ebc1a4f70 con 0x7f6ebc199f30 2024-04-22T13:29:10.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.376+0000 7f6eb8ff9640 1 -- 172.21.15.12:0/2827249664 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4004040 con 0x7f6ebc199f30 2024-04-22T13:29:10.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.376+0000 7f6ec1f23640 1 -- 172.21.15.12:0/2827249664 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ebc1a5430 con 0x7f6ebc199f30 2024-04-22T13:29:10.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.379+0000 7f6eb8ff9640 1 -- 172.21.15.12:0/2827249664 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6eb40324c0 con 0x7f6ebc199f30 2024-04-22T13:29:10.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.379+0000 7f6ec4bed740 1 Processor -- start 2024-04-22T13:29:10.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.379+0000 7f6ec4bed740 1 -- start start 2024-04-22T13:29:10.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.379+0000 7f6ec4bed740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea103d13b0 0x55ea103d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.379+0000 7f6ec4bed740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea103d1d60 0x55ea103da890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.379+0000 7f6ec4bed740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea103dae60 0x55ea103db2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.379+0000 7f6ec4bed740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ea103361f0 con 0x55ea103d13b0 2024-04-22T13:29:10.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.379+0000 7f6ec4bed740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ea102d34b0 con 0x55ea103dae60 2024-04-22T13:29:10.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.379+0000 7f6ec4bed740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ea103321b0 con 0x55ea103d1d60 2024-04-22T13:29:10.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.379+0000 7f6ebaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea103d13b0 0x55ea103d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.379+0000 7f6ebaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea103d13b0 0x55ea103d1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52312/0 (socket says 172.21.15.12:52312) 2024-04-22T13:29:10.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.379+0000 7f6ebaffd640 1 -- 172.21.15.12:0/1112021176 learned_addr learned my addr 172.21.15.12:0/1112021176 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:10.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.379+0000 7f6ebb7fe640 1 --2- 172.21.15.12:0/1112021176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea103d1d60 0x55ea103da890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.379+0000 7f6ebaffd640 1 --2- 172.21.15.12:0/1112021176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea103dae60 0x55ea103db2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.379+0000 7f6ebb7fe640 1 -- 172.21.15.12:0/1112021176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea103dae60 msgr2=0x55ea103db2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.379+0000 7f6ebb7fe640 1 --2- 172.21.15.12:0/1112021176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea103dae60 0x55ea103db2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.379+0000 7f6eb8ff9640 1 --2- 172.21.15.12:0/2827249664 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6ea40a3d20 0x7f6ea40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.379+0000 7f6ebb7fe640 1 -- 172.21.15.12:0/1112021176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea103d13b0 msgr2=0x55ea103d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.379+0000 7f6ebb7fe640 1 --2- 172.21.15.12:0/1112021176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea103d13b0 0x55ea103d1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.379+0000 7f6ebb7fe640 1 -- 172.21.15.12:0/1112021176 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea102c51b0 con 0x55ea103d1d60 2024-04-22T13:29:10.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.379+0000 7f6ebaffd640 1 --2- 172.21.15.12:0/1112021176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea103d13b0 0x55ea103d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:29:10.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.380+0000 7f6ebbfff640 1 --2- 172.21.15.12:0/2827249664 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6ea40a3d20 0x7f6ea40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.380+0000 7f6ebb7fe640 1 --2- 172.21.15.12:0/1112021176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea103d1d60 0x55ea103da890 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6eac00bbf0 tx=0x7f6eac0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3209ae196b18253 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:10.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.380+0000 7f6eb8ff9640 1 -- 172.21.15.12:0/2827249664 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f6eb40f2870 con 0x7f6ebc199f30 2024-04-22T13:29:10.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.380+0000 7f6ea1ffb640 1 -- 172.21.15.12:0/1112021176 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac004030 con 0x55ea103d1d60 2024-04-22T13:29:10.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.380+0000 7f6ebbfff640 1 --2- 172.21.15.12:0/2827249664 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6ea40a3d20 0x7f6ea40a61e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6eb40f01d0 tx=0x7f6eb40df040 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:10.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.380+0000 7f6ea1ffb640 1 -- 172.21.15.12:0/1112021176 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eac0035f0 con 0x55ea103d1d60 2024-04-22T13:29:10.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.380+0000 7f6ea1ffb640 1 -- 172.21.15.12:0/1112021176 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac0116a0 con 0x55ea103d1d60 2024-04-22T13:29:10.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.381+0000 7f6ec4bed740 1 -- 172.21.15.12:0/1112021176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea103d1d60 msgr2=0x55ea103da890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.381+0000 7f6ec4bed740 1 --2- 172.21.15.12:0/1112021176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea103d1d60 0x55ea103da890 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6eac00bbf0 tx=0x7f6eac0107a0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.381+0000 7f6ec4bed740 1 -- 172.21.15.12:0/1112021176 shutdown_connections 2024-04-22T13:29:10.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.381+0000 7f6ec4bed740 1 --2- 172.21.15.12:0/1112021176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea103dae60 0x55ea103db2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.381+0000 7f6ec4bed740 1 --2- 172.21.15.12:0/1112021176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea103d1d60 0x55ea103da890 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.381+0000 7f6ec4bed740 1 --2- 172.21.15.12:0/1112021176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea103d13b0 0x55ea103d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.381+0000 7f6ec4bed740 1 -- 172.21.15.12:0/1112021176 >> 172.21.15.12:0/1112021176 conn(0x55ea103cbba0 msgr2=0x55ea103cbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:10.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.381+0000 7f6ec4bed740 1 -- 172.21.15.12:0/1112021176 shutdown_connections 2024-04-22T13:29:10.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.381+0000 7f6ec4bed740 1 -- 172.21.15.12:0/1112021176 wait complete. 2024-04-22T13:29:10.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.381+0000 7f6ec4bed740 1 -- start start 2024-04-22T13:29:10.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.381+0000 7f6ec4bed740 1 Processor -- start 2024-04-22T13:29:10.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.381+0000 7f6ec4bed740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea103d13b0 0x55ea103e1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.381+0000 7f6ec4bed740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea103d1d60 0x55ea103de5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.381+0000 7f6ec4bed740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea103dae60 0x55ea103deb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.381+0000 7f6ec4bed740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ea103e1da0 con 0x55ea103d1d60 2024-04-22T13:29:10.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.381+0000 7f6ec4bed740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ea102ef500 con 0x55ea103dae60 2024-04-22T13:29:10.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.381+0000 7f6ec4bed740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ea103e1f20 con 0x55ea103d13b0 2024-04-22T13:29:10.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.381+0000 7f6ec4bed740 10 client.0 _ll_register_callbacks cb 0x55ea102bdb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:10.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.382+0000 7f6ebb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea103d1d60 0x55ea103de5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.382+0000 7f6ebb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea103d1d60 0x55ea103de5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52326/0 (socket says 172.21.15.12:52326) 2024-04-22T13:29:10.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.382+0000 7f6ebb7fe640 1 -- 172.21.15.12:0/3152439656 learned_addr learned my addr 172.21.15.12:0/3152439656 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:10.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.382+0000 7f6ebaffd640 1 --2- 172.21.15.12:0/3152439656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea103d13b0 0x55ea103e1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.382+0000 7f6ebaffd640 1 --2- 172.21.15.12:0/3152439656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea103dae60 0x55ea103deb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.382+0000 7f6ebb7fe640 1 -- 172.21.15.12:0/3152439656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea103d13b0 msgr2=0x55ea103e1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.382+0000 7f6ebb7fe640 1 --2- 172.21.15.12:0/3152439656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea103d13b0 0x55ea103e1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.382+0000 7f6ebb7fe640 1 -- 172.21.15.12:0/3152439656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea103dae60 msgr2=0x55ea103deb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.382+0000 7f6ebb7fe640 1 --2- 172.21.15.12:0/3152439656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea103dae60 0x55ea103deb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.382+0000 7f6ebb7fe640 1 -- 172.21.15.12:0/3152439656 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea100483f0 con 0x55ea103d1d60 2024-04-22T13:29:10.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.382+0000 7f6ebaffd640 1 --2- 172.21.15.12:0/3152439656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea103dae60 0x55ea103deb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:10.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.382+0000 7f6ebb7fe640 1 --2- 172.21.15.12:0/3152439656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea103d1d60 0x55ea103de5e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f6eac000c00 tx=0x7f6eac00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f781e1fb02329392 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:10.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.382+0000 7f6e9affd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:10.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.382+0000 7f6e9affd640 1 -- 172.21.15.12:0/3152439656 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac02a060 con 0x55ea103d1d60 2024-04-22T13:29:10.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.382+0000 7f6e9affd640 1 -- 172.21.15.12:0/3152439656 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f6eac003a30 con 0x55ea103d1d60 2024-04-22T13:29:10.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.382+0000 7f6ec4bed740 1 -- 172.21.15.12:0/3152439656 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6eac0035f0 con 0x55ea103d1d60 2024-04-22T13:29:10.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.382+0000 7f6e9affd640 1 -- 172.21.15.12:0/3152439656 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6eac004000 con 0x55ea103d1d60 2024-04-22T13:29:10.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.382+0000 7f6ec4bed740 10 client.17643 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:10.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.382+0000 7f6ec4bed740 1 -- 172.21.15.12:0/3152439656 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ea101127d0 con 0x55ea103d1d60 2024-04-22T13:29:10.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.383+0000 7f6e9affd640 1 -- 172.21.15.12:0/3152439656 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eac0035f0 con 0x55ea103d1d60 2024-04-22T13:29:10.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.383+0000 7f6e9affd640 1 -- 172.21.15.12:0/3152439656 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6eac004770 con 0x55ea103d1d60 2024-04-22T13:29:10.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.383+0000 7f6ec4bed740 10 client.17643 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:10.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.383+0000 7f6ec4bed740 10 client.17643 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:29:10.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.383+0000 7f6ec4bed740 20 client.17643 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:10.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.383+0000 7f6ec4bed740 1 --2- 172.21.15.12:0/3152439656 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55ea103eeff0 0x55ea103f13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.383+0000 7f6ec4bed740 4 client.17643 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:10.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.383+0000 7f6ec4bed740 1 -- 172.21.15.12:0/3152439656 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea102d5930 con 0x55ea103eeff0 2024-04-22T13:29:10.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.383+0000 7f6ebaffd640 1 --2- 172.21.15.12:0/3152439656 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55ea103eeff0 0x55ea103f13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.383+0000 7f6e9affd640 10 client.17643 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:29:10.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.383+0000 7f6ebaffd640 1 --2- 172.21.15.12:0/3152439656 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55ea103eeff0 0x55ea103f13e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=575b55ef7d61c5ef server_cookie=a22db533c4f26eee in_seq=0 out_seq=0 2024-04-22T13:29:10.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.384+0000 7f6e9affd640 1 -- 172.21.15.12:0/3152439656 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ea102d5930 con 0x55ea103eeff0 2024-04-22T13:29:10.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.384+0000 7f6e9affd640 10 client.17643 handle_command_reply: tid=0 2024-04-22T13:29:10.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.384+0000 7f6ec4bed740 1 client.17643 shutdown 2024-04-22T13:29:10.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.384+0000 7f6ec4bed740 1 -- 172.21.15.12:0/3152439656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea103d1d60 msgr2=0x55ea103de5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.384+0000 7f6ec4bed740 1 --2- 172.21.15.12:0/3152439656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea103d1d60 0x55ea103de5e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f6eac000c00 tx=0x7f6eac00bfd0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.384+0000 7f6ec4bed740 1 -- 172.21.15.12:0/3152439656 shutdown_connections 2024-04-22T13:29:10.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.384+0000 7f6ec4bed740 1 --2- 172.21.15.12:0/3152439656 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55ea103eeff0 0x55ea103f13e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.384+0000 7f6ec4bed740 1 --2- 172.21.15.12:0/3152439656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea103dae60 0x55ea103deb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.384+0000 7f6ec4bed740 1 --2- 172.21.15.12:0/3152439656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea103d1d60 0x55ea103de5e0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.384+0000 7f6e9affd640 0 client.17643 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:29:10.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.384+0000 7f6ec4bed740 1 --2- 172.21.15.12:0/3152439656 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea103d13b0 0x55ea103e1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.384+0000 7f6ec4bed740 1 -- 172.21.15.12:0/3152439656 >> 172.21.15.12:0/3152439656 conn(0x55ea103cbba0 msgr2=0x55ea103cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:10.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.385+0000 7f6ec4bed740 1 -- 172.21.15.12:0/3152439656 shutdown_connections 2024-04-22T13:29:10.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.385+0000 7f6ec4bed740 1 -- 172.21.15.12:0/3152439656 wait complete. 2024-04-22T13:29:10.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.385+0000 7f6ec4bed740 20 client.17643 trim_cache size 0 max 16384 2024-04-22T13:29:10.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.398+0000 7f6ec4bed740 1 Processor -- start 2024-04-22T13:29:10.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.398+0000 7f6ec4bed740 1 -- start start 2024-04-22T13:29:10.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.398+0000 7f6ec4bed740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea103d13b0 0x55ea103ead00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.398+0000 7f6ec4bed740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea103d1d60 0x55ea103e0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.398+0000 7f6ec4bed740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea103dae60 0x55ea103e1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.398+0000 7f6ec4bed740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc1ab740 con 0x55ea103d13b0 2024-04-22T13:29:10.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.398+0000 7f6ec4bed740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc1ad280 con 0x55ea103d1d60 2024-04-22T13:29:10.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.398+0000 7f6ec4bed740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc1abe10 con 0x55ea103dae60 2024-04-22T13:29:10.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.398+0000 7f6ebaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea103d13b0 0x55ea103ead00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.398+0000 7f6ebaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea103d13b0 0x55ea103ead00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52328/0 (socket says 172.21.15.12:52328) 2024-04-22T13:29:10.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.398+0000 7f6ebaffd640 1 -- 172.21.15.12:0/3089472585 learned_addr learned my addr 172.21.15.12:0/3089472585 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:10.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.398+0000 7f6ebaffd640 1 -- 172.21.15.12:0/3089472585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea103dae60 msgr2=0x55ea103e1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.398+0000 7f6ebaffd640 1 --2- 172.21.15.12:0/3089472585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea103dae60 0x55ea103e1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.398+0000 7f6ebaffd640 1 -- 172.21.15.12:0/3089472585 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea103d1d60 msgr2=0x55ea103e0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.398+0000 7f6ebaffd640 1 --2- 172.21.15.12:0/3089472585 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea103d1d60 0x55ea103e0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.398+0000 7f6ebaffd640 1 -- 172.21.15.12:0/3089472585 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea103d0560 con 0x55ea103d13b0 2024-04-22T13:29:10.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.399+0000 7f6ebaffd640 1 --2- 172.21.15.12:0/3089472585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea103d13b0 0x55ea103ead00 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0005e10 tx=0x7f6eb000a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c754740ad5ce477c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:10.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.399+0000 7f6ea17fa640 1 -- 172.21.15.12:0/3089472585 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb001c040 con 0x55ea103d13b0 2024-04-22T13:29:10.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.399+0000 7f6ea17fa640 1 -- 172.21.15.12:0/3089472585 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eb00047a0 con 0x55ea103d13b0 2024-04-22T13:29:10.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.399+0000 7f6ea17fa640 1 -- 172.21.15.12:0/3089472585 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb00223c0 con 0x55ea103d13b0 2024-04-22T13:29:10.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.399+0000 7f6ec4bed740 1 -- 172.21.15.12:0/3089472585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea103d13b0 msgr2=0x55ea103ead00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.399+0000 7f6ec4bed740 1 --2- 172.21.15.12:0/3089472585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea103d13b0 0x55ea103ead00 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0005e10 tx=0x7f6eb000a780 comp rx=0 tx=0).stop 2024-04-22T13:29:10.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.399+0000 7f6ec4bed740 1 -- 172.21.15.12:0/3089472585 shutdown_connections 2024-04-22T13:29:10.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.399+0000 7f6ec4bed740 1 --2- 172.21.15.12:0/3089472585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea103dae60 0x55ea103e1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.399+0000 7f6ec4bed740 1 --2- 172.21.15.12:0/3089472585 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea103d1d60 0x55ea103e0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.399+0000 7f6ec4bed740 1 --2- 172.21.15.12:0/3089472585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea103d13b0 0x55ea103ead00 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.399+0000 7f6ec4bed740 1 -- 172.21.15.12:0/3089472585 >> 172.21.15.12:0/3089472585 conn(0x55ea103cbba0 msgr2=0x55ea103dd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:10.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.399+0000 7f6ec4bed740 1 -- 172.21.15.12:0/3089472585 shutdown_connections 2024-04-22T13:29:10.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.399+0000 7f6ec4bed740 1 -- 172.21.15.12:0/3089472585 wait complete. 2024-04-22T13:29:10.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.400+0000 7f6ec4bed740 1 -- start start 2024-04-22T13:29:10.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.400+0000 7f6ec4bed740 1 Processor -- start 2024-04-22T13:29:10.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.400+0000 7f6ec4bed740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea103d13b0 0x55ea103e7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.400+0000 7f6ec4bed740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea103d1d60 0x55ea103dff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.400+0000 7f6ec4bed740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea103dae60 0x55ea103e04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.400+0000 7f6ec4bed740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc1acbb0 con 0x55ea103d13b0 2024-04-22T13:29:10.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.400+0000 7f6ec4bed740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc1ac4e0 con 0x55ea103dae60 2024-04-22T13:29:10.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.400+0000 7f6ec4bed740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ebc1a9740 con 0x55ea103d1d60 2024-04-22T13:29:10.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.400+0000 7f6ec4bed740 10 client.0 _ll_register_callbacks cb 0x7f6eac003d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:10.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.400+0000 7f6ebaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea103d13b0 0x55ea103e7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.400+0000 7f6ebaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea103d13b0 0x55ea103e7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52340/0 (socket says 172.21.15.12:52340) 2024-04-22T13:29:10.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.400+0000 7f6ebaffd640 1 -- 172.21.15.12:0/1529888670 learned_addr learned my addr 172.21.15.12:0/1529888670 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:10.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.400+0000 7f6ebb7fe640 1 --2- 172.21.15.12:0/1529888670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea103d1d60 0x55ea103dff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.400+0000 7f6ebaffd640 1 -- 172.21.15.12:0/1529888670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea103d1d60 msgr2=0x55ea103dff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.400+0000 7f6ebaffd640 1 --2- 172.21.15.12:0/1529888670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea103d1d60 0x55ea103dff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.400+0000 7f6ebaffd640 1 -- 172.21.15.12:0/1529888670 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea103dae60 msgr2=0x55ea103e04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.400+0000 7f6ebaffd640 1 --2- 172.21.15.12:0/1529888670 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea103dae60 0x55ea103e04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.401+0000 7f6ebaffd640 1 -- 172.21.15.12:0/1529888670 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea103e0ba0 con 0x55ea103d13b0 2024-04-22T13:29:10.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.401+0000 7f6ebb7fe640 1 --2- 172.21.15.12:0/1529888670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea103d1d60 0x55ea103dff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:10.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.401+0000 7f6ebaffd640 1 --2- 172.21.15.12:0/1529888670 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea103d13b0 0x55ea103e7540 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0012760 tx=0x7f6eb000a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad20524f95931ec server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:10.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.401+0000 7f6e9affd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:10.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.401+0000 7f6e9affd640 1 -- 172.21.15.12:0/1529888670 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0031060 con 0x55ea103d13b0 2024-04-22T13:29:10.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.401+0000 7f6e9affd640 1 -- 172.21.15.12:0/1529888670 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f6eb0004d30 con 0x55ea103d13b0 2024-04-22T13:29:10.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.401+0000 7f6ec4bed740 1 -- 172.21.15.12:0/1529888670 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6eb00047a0 con 0x55ea103d13b0 2024-04-22T13:29:10.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.401+0000 7f6e9affd640 1 -- 172.21.15.12:0/1529888670 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6eb001c040 con 0x55ea103d13b0 2024-04-22T13:29:10.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.401+0000 7f6ec4bed740 10 client.17649 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:10.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.401+0000 7f6ec4bed740 1 -- 172.21.15.12:0/1529888670 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ea10455160 con 0x55ea103d13b0 2024-04-22T13:29:10.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.401+0000 7f6e9affd640 1 -- 172.21.15.12:0/1529888670 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eb00047a0 con 0x55ea103d13b0 2024-04-22T13:29:10.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.401+0000 7f6e9affd640 1 -- 172.21.15.12:0/1529888670 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6eb001f070 con 0x55ea103d13b0 2024-04-22T13:29:10.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.401+0000 7f6ec4bed740 10 client.17649 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:10.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.401+0000 7f6ec4bed740 10 client.17649 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:29:10.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.401+0000 7f6ec4bed740 20 client.17649 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:10.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.401+0000 7f6ec4bed740 1 --2- 172.21.15.12:0/1529888670 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55ea10455890 0x55ea10457cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.401+0000 7f6ec4bed740 4 client.17649 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:29:10.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.401+0000 7f6ec4bed740 1 -- 172.21.15.12:0/1529888670 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ea10343c10 con 0x55ea10455890 2024-04-22T13:29:10.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.402+0000 7f6ebb7fe640 1 --2- 172.21.15.12:0/1529888670 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55ea10455890 0x55ea10457cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.402+0000 7f6e9affd640 10 client.17649 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:29:10.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.402+0000 7f6ebb7fe640 1 --2- 172.21.15.12:0/1529888670 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55ea10455890 0x55ea10457cb0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a50fbd8ec4923f6 server_cookie=43b7664d6c5e4550 in_seq=0 out_seq=0 2024-04-22T13:29:10.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.402+0000 7f6e9affd640 1 -- 172.21.15.12:0/1529888670 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ea10343c10 con 0x55ea10455890 2024-04-22T13:29:10.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.402+0000 7f6e9affd640 10 client.17649 handle_command_reply: tid=0 2024-04-22T13:29:10.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.402+0000 7f6ec4bed740 1 client.17649 shutdown 2024-04-22T13:29:10.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.402+0000 7f6ec4bed740 1 -- 172.21.15.12:0/1529888670 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea103d13b0 msgr2=0x55ea103e7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.402+0000 7f6ec4bed740 1 --2- 172.21.15.12:0/1529888670 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea103d13b0 0x55ea103e7540 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0012760 tx=0x7f6eb000a420 comp rx=0 tx=0).stop 2024-04-22T13:29:10.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.402+0000 7f6ec4bed740 1 -- 172.21.15.12:0/1529888670 shutdown_connections 2024-04-22T13:29:10.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.402+0000 7f6ec4bed740 1 --2- 172.21.15.12:0/1529888670 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55ea10455890 0x55ea10457cb0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.402+0000 7f6ec4bed740 1 --2- 172.21.15.12:0/1529888670 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ea103dae60 0x55ea103e04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.402+0000 7f6ec4bed740 1 --2- 172.21.15.12:0/1529888670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ea103d1d60 0x55ea103dff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.402+0000 7f6ec4bed740 1 --2- 172.21.15.12:0/1529888670 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea103d13b0 0x55ea103e7540 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.402+0000 7f6e9affd640 0 client.17649 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:29:10.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.402+0000 7f6ec4bed740 1 -- 172.21.15.12:0/1529888670 >> 172.21.15.12:0/1529888670 conn(0x55ea103cbba0 msgr2=0x55ea103ca370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:10.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.403+0000 7f6ec4bed740 1 -- 172.21.15.12:0/1529888670 shutdown_connections 2024-04-22T13:29:10.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.403+0000 7f6ec4bed740 1 -- 172.21.15.12:0/1529888670 wait complete. 2024-04-22T13:29:10.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.403+0000 7f6ec4bed740 20 client.17649 trim_cache size 0 max 16384 2024-04-22T13:29:10.405 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:29:10.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.403+0000 7f6ec1f23640 1 -- 172.21.15.12:0/2827249664 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6ea40a3d20 msgr2=0x7f6ea40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.403+0000 7f6ec1f23640 1 --2- 172.21.15.12:0/2827249664 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6ea40a3d20 0x7f6ea40a61e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6eb40f01d0 tx=0x7f6eb40df040 comp rx=0 tx=0).stop 2024-04-22T13:29:10.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.403+0000 7f6ec1f23640 1 -- 172.21.15.12:0/2827249664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ebc199f30 msgr2=0x7f6ebc1a44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.403+0000 7f6ec1f23640 1 --2- 172.21.15.12:0/2827249664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ebc199f30 0x7f6ebc1a44e0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4005e30 tx=0x7f6eb4034760 comp rx=0 tx=0).stop 2024-04-22T13:29:10.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.403+0000 7f6ec1f23640 1 -- 172.21.15.12:0/2827249664 shutdown_connections 2024-04-22T13:29:10.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.403+0000 7f6ec1f23640 1 --2- 172.21.15.12:0/2827249664 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6ea40a3d20 0x7f6ea40a61e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.403+0000 7f6ec1f23640 1 --2- 172.21.15.12:0/2827249664 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6ebc199f30 0x7f6ebc1a44e0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.403+0000 7f6ec1f23640 1 --2- 172.21.15.12:0/2827249664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ebc106170 0x7f6ebc1999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.403+0000 7f6ec1f23640 1 --2- 172.21.15.12:0/2827249664 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ebc1010a0 0x7f6ebc199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.403+0000 7f6ec1f23640 1 -- 172.21.15.12:0/2827249664 >> 172.21.15.12:0/2827249664 conn(0x7f6ebc077eb0 msgr2=0x7f6ebc0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:10.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.403+0000 7f6ec1f23640 1 -- 172.21.15.12:0/2827249664 shutdown_connections 2024-04-22T13:29:10.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.403+0000 7f6ec1f23640 1 -- 172.21.15.12:0/2827249664 wait complete. 2024-04-22T13:29:10.410 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:29:10.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.518+0000 7fc33d650640 1 -- 172.21.15.12:0/3446575733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc338107260 msgr2=0x7fc33810ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.518+0000 7fc33d650640 1 --2- 172.21.15.12:0/3446575733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc338107260 0x7fc33810ee30 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fc330009ca0 tx=0x7fc33002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.518+0000 7fc33d650640 1 -- 172.21.15.12:0/3446575733 shutdown_connections 2024-04-22T13:29:10.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.518+0000 7fc33d650640 1 --2- 172.21.15.12:0/3446575733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc338107260 0x7fc33810ee30 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.518+0000 7fc33d650640 1 --2- 172.21.15.12:0/3446575733 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc338102b40 0x7fc338106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.518+0000 7fc33d650640 1 --2- 172.21.15.12:0/3446575733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc338102190 0x7fc338102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.518+0000 7fc33d650640 1 -- 172.21.15.12:0/3446575733 >> 172.21.15.12:0/3446575733 conn(0x7fc3380fd990 msgr2=0x7fc3380ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:10.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.518+0000 7fc33d650640 1 -- 172.21.15.12:0/3446575733 shutdown_connections 2024-04-22T13:29:10.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.518+0000 7fc33d650640 1 -- 172.21.15.12:0/3446575733 wait complete. 2024-04-22T13:29:10.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.519+0000 7fc33d650640 1 Processor -- start 2024-04-22T13:29:10.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.519+0000 7fc33d650640 1 -- start start 2024-04-22T13:29:10.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.519+0000 7fc33d650640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc338102190 0x7fc338198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.519+0000 7fc33d650640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc338107260 0x7fc338198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.519+0000 7fc33d650640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc33819cff0 0x7fc33819d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.519+0000 7fc33d650640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc3381121b0 con 0x7fc338107260 2024-04-22T13:29:10.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.519+0000 7fc33d650640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc338112030 con 0x7fc338102190 2024-04-22T13:29:10.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.519+0000 7fc33d650640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc338112330 con 0x7fc33819cff0 2024-04-22T13:29:10.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.519+0000 7fc3367fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc338107260 0x7fc338198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.519+0000 7fc3367fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc338107260 0x7fc338198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52370/0 (socket says 172.21.15.12:52370) 2024-04-22T13:29:10.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.519+0000 7fc3367fc640 1 -- 172.21.15.12:0/1962393237 learned_addr learned my addr 172.21.15.12:0/1962393237 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:10.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.519+0000 7fc3377fe640 1 --2- 172.21.15.12:0/1962393237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc33819cff0 0x7fc33819d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.519+0000 7fc336ffd640 1 --2- 172.21.15.12:0/1962393237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc338102190 0x7fc338198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.519+0000 7fc3367fc640 1 -- 172.21.15.12:0/1962393237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc33819cff0 msgr2=0x7fc33819d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.519+0000 7fc3367fc640 1 --2- 172.21.15.12:0/1962393237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc33819cff0 0x7fc33819d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.519+0000 7fc3367fc640 1 -- 172.21.15.12:0/1962393237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc338102190 msgr2=0x7fc338198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.519+0000 7fc3367fc640 1 --2- 172.21.15.12:0/1962393237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc338102190 0x7fc338198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.520+0000 7fc3367fc640 1 -- 172.21.15.12:0/1962393237 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc33819dae0 con 0x7fc338107260 2024-04-22T13:29:10.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.520+0000 7fc336ffd640 1 --2- 172.21.15.12:0/1962393237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc338102190 0x7fc338198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:10.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.520+0000 7fc3367fc640 1 --2- 172.21.15.12:0/1962393237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc338107260 0x7fc338198c70 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fc32c00d770 tx=0x7fc32c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:10.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.520+0000 7fc31ffff640 1 -- 172.21.15.12:0/1962393237 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c017070 con 0x7fc338107260 2024-04-22T13:29:10.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.520+0000 7fc33d650640 1 -- 172.21.15.12:0/1962393237 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc33819ddd0 con 0x7fc338107260 2024-04-22T13:29:10.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.520+0000 7fc31ffff640 1 -- 172.21.15.12:0/1962393237 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc32c009c30 con 0x7fc338107260 2024-04-22T13:29:10.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.520+0000 7fc31ffff640 1 -- 172.21.15.12:0/1962393237 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c00f620 con 0x7fc338107260 2024-04-22T13:29:10.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.520+0000 7fc33d650640 1 -- 172.21.15.12:0/1962393237 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3381a55c0 con 0x7fc338107260 2024-04-22T13:29:10.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.522+0000 7fc31ffff640 1 -- 172.21.15.12:0/1962393237 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc32c01a070 con 0x7fc338107260 2024-04-22T13:29:10.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.523+0000 7fc34023d740 1 Processor -- start 2024-04-22T13:29:10.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.523+0000 7fc34023d740 1 -- start start 2024-04-22T13:29:10.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.523+0000 7fc34023d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b6a0b14d0 0x561b6a0b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.523+0000 7fc31ffff640 1 --2- 172.21.15.12:0/1962393237 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc3200a3d20 0x7fc3200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.523+0000 7fc34023d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b6a0b1e80 0x561b6a0ba430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.523+0000 7fc34023d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b6a0baa00 0x561b6a0bce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.523+0000 7fc34023d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561b6a0161f0 con 0x561b6a0b1e80 2024-04-22T13:29:10.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.523+0000 7fc34023d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561b69fb34b0 con 0x561b6a0b14d0 2024-04-22T13:29:10.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.523+0000 7fc34023d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561b6a0121b0 con 0x561b6a0baa00 2024-04-22T13:29:10.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.523+0000 7fc3377fe640 1 --2- 172.21.15.12:0/1962393237 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc3200a3d20 0x7fc3200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.523+0000 7fc3367fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b6a0b1e80 0x561b6a0ba430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.523+0000 7fc3367fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b6a0b1e80 0x561b6a0ba430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52372/0 (socket says 172.21.15.12:52372) 2024-04-22T13:29:10.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.523+0000 7fc3367fc640 1 -- 172.21.15.12:0/4019543187 learned_addr learned my addr 172.21.15.12:0/4019543187 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:10.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.523+0000 7fc336ffd640 1 --2- 172.21.15.12:0/4019543187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b6a0b14d0 0x561b6a0b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.523+0000 7fc31ffff640 1 -- 172.21.15.12:0/1962393237 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fc32c0d0280 con 0x7fc338107260 2024-04-22T13:29:10.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.523+0000 7fc3377fe640 1 --2- 172.21.15.12:0/4019543187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b6a0baa00 0x561b6a0bce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.523+0000 7fc3367fc640 1 -- 172.21.15.12:0/4019543187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b6a0baa00 msgr2=0x561b6a0bce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.523+0000 7fc3367fc640 1 --2- 172.21.15.12:0/4019543187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b6a0baa00 0x561b6a0bce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.523+0000 7fc3367fc640 1 -- 172.21.15.12:0/4019543187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b6a0b14d0 msgr2=0x561b6a0b18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.523+0000 7fc3367fc640 1 --2- 172.21.15.12:0/4019543187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b6a0b14d0 0x561b6a0b18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.523+0000 7fc3367fc640 1 -- 172.21.15.12:0/4019543187 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b69fa51b0 con 0x561b6a0b1e80 2024-04-22T13:29:10.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.524+0000 7fc3377fe640 1 --2- 172.21.15.12:0/1962393237 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc3200a3d20 0x7fc3200a61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc330004af0 tx=0x7fc33002cf50 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:10.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.524+0000 7fc3367fc640 1 --2- 172.21.15.12:0/4019543187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b6a0b1e80 0x561b6a0ba430 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fc32c098c40 tx=0x7fc32c00df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170b7eaf8b26c685 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:10.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.524+0000 7fc31d7fa640 1 -- 172.21.15.12:0/4019543187 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c0cf070 con 0x561b6a0b1e80 2024-04-22T13:29:10.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.524+0000 7fc31d7fa640 1 -- 172.21.15.12:0/4019543187 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc32c09c970 con 0x561b6a0b1e80 2024-04-22T13:29:10.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.524+0000 7fc31d7fa640 1 -- 172.21.15.12:0/4019543187 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c021040 con 0x561b6a0b1e80 2024-04-22T13:29:10.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.524+0000 7fc34023d740 1 -- 172.21.15.12:0/4019543187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b6a0b1e80 msgr2=0x561b6a0ba430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.524+0000 7fc34023d740 1 --2- 172.21.15.12:0/4019543187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b6a0b1e80 0x561b6a0ba430 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fc32c098c40 tx=0x7fc32c00df10 comp rx=0 tx=0).stop 2024-04-22T13:29:10.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.524+0000 7fc34023d740 1 -- 172.21.15.12:0/4019543187 shutdown_connections 2024-04-22T13:29:10.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.524+0000 7fc34023d740 1 --2- 172.21.15.12:0/4019543187 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b6a0baa00 0x561b6a0bce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.524+0000 7fc34023d740 1 --2- 172.21.15.12:0/4019543187 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b6a0b1e80 0x561b6a0ba430 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.524+0000 7fc34023d740 1 --2- 172.21.15.12:0/4019543187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b6a0b14d0 0x561b6a0b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.524+0000 7fc34023d740 1 -- 172.21.15.12:0/4019543187 >> 172.21.15.12:0/4019543187 conn(0x561b6a0abba0 msgr2=0x561b6a0abf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:10.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.524+0000 7fc34023d740 1 -- 172.21.15.12:0/4019543187 shutdown_connections 2024-04-22T13:29:10.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.524+0000 7fc34023d740 1 -- 172.21.15.12:0/4019543187 wait complete. 2024-04-22T13:29:10.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.525+0000 7fc34023d740 1 -- start start 2024-04-22T13:29:10.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.525+0000 7fc34023d740 1 Processor -- start 2024-04-22T13:29:10.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.525+0000 7fc34023d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b6a0b14d0 0x561b6a0ba350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.525+0000 7fc34023d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b6a0b1e80 0x561b6a0b76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.525+0000 7fc34023d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b6a0baa00 0x561b6a0b7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.525+0000 7fc34023d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561b6a0bd970 con 0x561b6a0baa00 2024-04-22T13:29:10.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.525+0000 7fc34023d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561b69fcf500 con 0x561b6a0b1e80 2024-04-22T13:29:10.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.525+0000 7fc34023d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561b6a0bdaf0 con 0x561b6a0b14d0 2024-04-22T13:29:10.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.525+0000 7fc34023d740 10 client.0 _ll_register_callbacks cb 0x561b69f9db80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:10.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.525+0000 7fc3377fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b6a0baa00 0x561b6a0b7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.525+0000 7fc3377fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b6a0baa00 0x561b6a0b7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52374/0 (socket says 172.21.15.12:52374) 2024-04-22T13:29:10.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.525+0000 7fc3377fe640 1 -- 172.21.15.12:0/1193798760 learned_addr learned my addr 172.21.15.12:0/1193798760 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:10.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.525+0000 7fc336ffd640 1 --2- 172.21.15.12:0/1193798760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b6a0b14d0 0x561b6a0ba350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.525+0000 7fc3367fc640 1 --2- 172.21.15.12:0/1193798760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b6a0b1e80 0x561b6a0b76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.526+0000 7fc3377fe640 1 -- 172.21.15.12:0/1193798760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b6a0b14d0 msgr2=0x561b6a0ba350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.526+0000 7fc3377fe640 1 --2- 172.21.15.12:0/1193798760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b6a0b14d0 0x561b6a0ba350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.526+0000 7fc3377fe640 1 -- 172.21.15.12:0/1193798760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b6a0b1e80 msgr2=0x561b6a0b76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.526+0000 7fc3377fe640 1 --2- 172.21.15.12:0/1193798760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b6a0b1e80 0x561b6a0b76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.526+0000 7fc3377fe640 1 -- 172.21.15.12:0/1193798760 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b69d283f0 con 0x561b6a0baa00 2024-04-22T13:29:10.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.526+0000 7fc3377fe640 1 --2- 172.21.15.12:0/1193798760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b6a0baa00 0x561b6a0b7cf0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fc33002dfd0 tx=0x7fc33002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f1b8a4d15e488f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:10.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.526+0000 7fc31aff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:10.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.526+0000 7fc31aff5640 1 -- 172.21.15.12:0/1193798760 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc330052060 con 0x561b6a0baa00 2024-04-22T13:29:10.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.526+0000 7fc31aff5640 1 -- 172.21.15.12:0/1193798760 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fc330004240 con 0x561b6a0baa00 2024-04-22T13:29:10.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.526+0000 7fc34023d740 1 -- 172.21.15.12:0/1193798760 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc32c09c970 con 0x561b6a0baa00 2024-04-22T13:29:10.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.526+0000 7fc31aff5640 1 -- 172.21.15.12:0/1193798760 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc330039040 con 0x561b6a0baa00 2024-04-22T13:29:10.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.526+0000 7fc34023d740 10 client.17661 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:10.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.526+0000 7fc34023d740 1 -- 172.21.15.12:0/1193798760 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561b69df27d0 con 0x561b6a0baa00 2024-04-22T13:29:10.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.526+0000 7fc31aff5640 1 -- 172.21.15.12:0/1193798760 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc32c09c970 con 0x561b6a0baa00 2024-04-22T13:29:10.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.526+0000 7fc31aff5640 1 -- 172.21.15.12:0/1193798760 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc33003f070 con 0x561b6a0baa00 2024-04-22T13:29:10.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.526+0000 7fc34023d740 10 client.17661 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:10.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.526+0000 7fc34023d740 10 client.17661 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:29:10.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.526+0000 7fc34023d740 20 client.17661 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:10.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.527+0000 7fc34023d740 1 --2- 172.21.15.12:0/1193798760 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x561b6a0caca0 0x561b6a0cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.527+0000 7fc34023d740 4 client.17661 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:10.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.527+0000 7fc34023d740 1 -- 172.21.15.12:0/1193798760 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b69fb5930 con 0x561b6a0caca0 2024-04-22T13:29:10.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.527+0000 7fc336ffd640 1 --2- 172.21.15.12:0/1193798760 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x561b6a0caca0 0x561b6a0cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.527+0000 7fc31aff5640 10 client.17661 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:29:10.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.527+0000 7fc336ffd640 1 --2- 172.21.15.12:0/1193798760 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x561b6a0caca0 0x561b6a0cd090 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a184fd31be0437ad server_cookie=53a19e73e0cf8c9 in_seq=0 out_seq=0 2024-04-22T13:29:10.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.528+0000 7fc31aff5640 1 -- 172.21.15.12:0/1193798760 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561b69fb5930 con 0x561b6a0caca0 2024-04-22T13:29:10.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.528+0000 7fc31aff5640 10 client.17661 handle_command_reply: tid=0 2024-04-22T13:29:10.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.528+0000 7fc34023d740 1 client.17661 shutdown 2024-04-22T13:29:10.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.528+0000 7fc34023d740 1 -- 172.21.15.12:0/1193798760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b6a0baa00 msgr2=0x561b6a0b7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.528+0000 7fc34023d740 1 --2- 172.21.15.12:0/1193798760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b6a0baa00 0x561b6a0b7cf0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fc33002dfd0 tx=0x7fc33002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.528+0000 7fc34023d740 1 -- 172.21.15.12:0/1193798760 shutdown_connections 2024-04-22T13:29:10.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.528+0000 7fc34023d740 1 --2- 172.21.15.12:0/1193798760 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x561b6a0caca0 0x561b6a0cd090 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.528+0000 7fc34023d740 1 --2- 172.21.15.12:0/1193798760 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b6a0baa00 0x561b6a0b7cf0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.528+0000 7fc31aff5640 0 client.17661 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:29:10.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.528+0000 7fc34023d740 1 --2- 172.21.15.12:0/1193798760 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b6a0b1e80 0x561b6a0b76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.528+0000 7fc34023d740 1 --2- 172.21.15.12:0/1193798760 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b6a0b14d0 0x561b6a0ba350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.528+0000 7fc34023d740 1 -- 172.21.15.12:0/1193798760 >> 172.21.15.12:0/1193798760 conn(0x561b6a0abba0 msgr2=0x561b6a0ad8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:10.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.528+0000 7fc34023d740 1 -- 172.21.15.12:0/1193798760 shutdown_connections 2024-04-22T13:29:10.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.528+0000 7fc34023d740 1 -- 172.21.15.12:0/1193798760 wait complete. 2024-04-22T13:29:10.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.528+0000 7fc34023d740 20 client.17661 trim_cache size 0 max 16384 2024-04-22T13:29:10.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.541+0000 7fc34023d740 1 Processor -- start 2024-04-22T13:29:10.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.542+0000 7fc34023d740 1 -- start start 2024-04-22T13:29:10.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.542+0000 7fc34023d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b6a0b14d0 0x561b6a0c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.542+0000 7fc34023d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b6a0b1e80 0x561b6a0ad630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.542+0000 7fc34023d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b6a0baa00 0x561b6a0b9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.542+0000 7fc34023d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc3381adb10 con 0x561b6a0b1e80 2024-04-22T13:29:10.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.542+0000 7fc34023d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc3381af650 con 0x561b6a0baa00 2024-04-22T13:29:10.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.542+0000 7fc34023d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc3381ae1e0 con 0x561b6a0b14d0 2024-04-22T13:29:10.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.542+0000 7fc3367fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b6a0b1e80 0x561b6a0ad630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.542+0000 7fc3367fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b6a0b1e80 0x561b6a0ad630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52378/0 (socket says 172.21.15.12:52378) 2024-04-22T13:29:10.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.542+0000 7fc3367fc640 1 -- 172.21.15.12:0/2902863843 learned_addr learned my addr 172.21.15.12:0/2902863843 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:10.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.542+0000 7fc3377fe640 1 --2- 172.21.15.12:0/2902863843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b6a0baa00 0x561b6a0b9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.542+0000 7fc336ffd640 1 --2- 172.21.15.12:0/2902863843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b6a0b14d0 0x561b6a0c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.542+0000 7fc3367fc640 1 -- 172.21.15.12:0/2902863843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b6a0b14d0 msgr2=0x561b6a0c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.542+0000 7fc3367fc640 1 --2- 172.21.15.12:0/2902863843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b6a0b14d0 0x561b6a0c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.542+0000 7fc3367fc640 1 -- 172.21.15.12:0/2902863843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b6a0baa00 msgr2=0x561b6a0b9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.542+0000 7fc3367fc640 1 --2- 172.21.15.12:0/2902863843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b6a0baa00 0x561b6a0b9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.542+0000 7fc3367fc640 1 -- 172.21.15.12:0/2902863843 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b6a0ab380 con 0x561b6a0b1e80 2024-04-22T13:29:10.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.542+0000 7fc336ffd640 1 --2- 172.21.15.12:0/2902863843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b6a0b14d0 0x561b6a0c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:10.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.542+0000 7fc3367fc640 1 --2- 172.21.15.12:0/2902863843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b6a0b1e80 0x561b6a0ad630 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fc32c005860 tx=0x7fc32c09c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de02306b4b6c75ca server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:10.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.542+0000 7fc31cff9640 1 -- 172.21.15.12:0/2902863843 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c021040 con 0x561b6a0b1e80 2024-04-22T13:29:10.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.542+0000 7fc31cff9640 1 -- 172.21.15.12:0/2902863843 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc32c009de0 con 0x561b6a0b1e80 2024-04-22T13:29:10.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.542+0000 7fc31cff9640 1 -- 172.21.15.12:0/2902863843 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c0cf040 con 0x561b6a0b1e80 2024-04-22T13:29:10.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.543+0000 7fc34023d740 1 -- 172.21.15.12:0/2902863843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b6a0b1e80 msgr2=0x561b6a0ad630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.543+0000 7fc34023d740 1 --2- 172.21.15.12:0/2902863843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b6a0b1e80 0x561b6a0ad630 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fc32c005860 tx=0x7fc32c09c270 comp rx=0 tx=0).stop 2024-04-22T13:29:10.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.543+0000 7fc34023d740 1 -- 172.21.15.12:0/2902863843 shutdown_connections 2024-04-22T13:29:10.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.543+0000 7fc34023d740 1 --2- 172.21.15.12:0/2902863843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b6a0baa00 0x561b6a0b9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.543+0000 7fc34023d740 1 --2- 172.21.15.12:0/2902863843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b6a0b1e80 0x561b6a0ad630 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.543+0000 7fc34023d740 1 --2- 172.21.15.12:0/2902863843 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b6a0b14d0 0x561b6a0c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.543+0000 7fc34023d740 1 -- 172.21.15.12:0/2902863843 >> 172.21.15.12:0/2902863843 conn(0x561b6a0abba0 msgr2=0x561b6a0acf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:10.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.543+0000 7fc34023d740 1 -- 172.21.15.12:0/2902863843 shutdown_connections 2024-04-22T13:29:10.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.543+0000 7fc34023d740 1 -- 172.21.15.12:0/2902863843 wait complete. 2024-04-22T13:29:10.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.543+0000 7fc34023d740 1 -- start start 2024-04-22T13:29:10.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.543+0000 7fc34023d740 1 Processor -- start 2024-04-22T13:29:10.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.544+0000 7fc34023d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b6a0b14d0 0x561b6a0c30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.544+0000 7fc34023d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b6a0b1e80 0x561b6a0be410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.544+0000 7fc34023d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b6a0baa00 0x561b6a0be950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.544+0000 7fc34023d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc3381aef80 con 0x561b6a0b1e80 2024-04-22T13:29:10.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.544+0000 7fc34023d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc3381ae8b0 con 0x561b6a0b14d0 2024-04-22T13:29:10.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.544+0000 7fc34023d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc3381abb10 con 0x561b6a0baa00 2024-04-22T13:29:10.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.544+0000 7fc34023d740 10 client.0 _ll_register_callbacks cb 0x7fc3300328a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:10.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.544+0000 7fc3367fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b6a0b1e80 0x561b6a0be410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.544+0000 7fc3367fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b6a0b1e80 0x561b6a0be410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52388/0 (socket says 172.21.15.12:52388) 2024-04-22T13:29:10.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.544+0000 7fc3367fc640 1 -- 172.21.15.12:0/3961453256 learned_addr learned my addr 172.21.15.12:0/3961453256 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:10.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.544+0000 7fc336ffd640 1 --2- 172.21.15.12:0/3961453256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b6a0b14d0 0x561b6a0c30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.544+0000 7fc3377fe640 1 --2- 172.21.15.12:0/3961453256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b6a0baa00 0x561b6a0be950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.544+0000 7fc3367fc640 1 -- 172.21.15.12:0/3961453256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b6a0baa00 msgr2=0x561b6a0be950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.544+0000 7fc3367fc640 1 --2- 172.21.15.12:0/3961453256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b6a0baa00 0x561b6a0be950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.544+0000 7fc3367fc640 1 -- 172.21.15.12:0/3961453256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b6a0b14d0 msgr2=0x561b6a0c30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.544+0000 7fc3367fc640 1 --2- 172.21.15.12:0/3961453256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b6a0b14d0 0x561b6a0c30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.544+0000 7fc3367fc640 1 -- 172.21.15.12:0/3961453256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b6a0bf120 con 0x561b6a0b1e80 2024-04-22T13:29:10.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.544+0000 7fc3367fc640 1 --2- 172.21.15.12:0/3961453256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b6a0b1e80 0x561b6a0be410 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fc32c098c40 tx=0x7fc32c010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f03503068fe2fbe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:10.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.544+0000 7fc31aff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:10.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.544+0000 7fc31aff5640 1 -- 172.21.15.12:0/3961453256 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c08f030 con 0x561b6a0b1e80 2024-04-22T13:29:10.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.544+0000 7fc31aff5640 1 -- 172.21.15.12:0/3961453256 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fc32c09a570 con 0x561b6a0b1e80 2024-04-22T13:29:10.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.544+0000 7fc34023d740 1 -- 172.21.15.12:0/3961453256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc32c009de0 con 0x561b6a0b1e80 2024-04-22T13:29:10.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.544+0000 7fc31aff5640 1 -- 172.21.15.12:0/3961453256 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc32c021050 con 0x561b6a0b1e80 2024-04-22T13:29:10.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.544+0000 7fc34023d740 10 client.17667 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:10.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.544+0000 7fc34023d740 1 -- 172.21.15.12:0/3961453256 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561b6a1310c0 con 0x561b6a0b1e80 2024-04-22T13:29:10.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.545+0000 7fc31aff5640 1 -- 172.21.15.12:0/3961453256 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc32c009de0 con 0x561b6a0b1e80 2024-04-22T13:29:10.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.545+0000 7fc31aff5640 1 -- 172.21.15.12:0/3961453256 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc32c097fc0 con 0x561b6a0b1e80 2024-04-22T13:29:10.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.545+0000 7fc34023d740 10 client.17667 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:10.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.545+0000 7fc34023d740 10 client.17667 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:29:10.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.545+0000 7fc34023d740 20 client.17667 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:10.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.545+0000 7fc34023d740 1 --2- 172.21.15.12:0/3961453256 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x561b6a1312e0 0x561b6a1336d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.545+0000 7fc34023d740 4 client.17667 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:29:10.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.545+0000 7fc34023d740 1 -- 172.21.15.12:0/3961453256 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561b6a023c10 con 0x561b6a1312e0 2024-04-22T13:29:10.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.545+0000 7fc336ffd640 1 --2- 172.21.15.12:0/3961453256 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x561b6a1312e0 0x561b6a1336d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.546+0000 7fc31aff5640 10 client.17667 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:29:10.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.546+0000 7fc336ffd640 1 --2- 172.21.15.12:0/3961453256 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x561b6a1312e0 0x561b6a1336d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9c241e799e0ba8f9 server_cookie=a61e09896392942f in_seq=0 out_seq=0 2024-04-22T13:29:10.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.546+0000 7fc31aff5640 1 -- 172.21.15.12:0/3961453256 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561b6a023c10 con 0x561b6a1312e0 2024-04-22T13:29:10.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.546+0000 7fc31aff5640 10 client.17667 handle_command_reply: tid=0 2024-04-22T13:29:10.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.546+0000 7fc34023d740 1 client.17667 shutdown 2024-04-22T13:29:10.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.546+0000 7fc34023d740 1 -- 172.21.15.12:0/3961453256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b6a0b1e80 msgr2=0x561b6a0be410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.546+0000 7fc34023d740 1 --2- 172.21.15.12:0/3961453256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b6a0b1e80 0x561b6a0be410 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fc32c098c40 tx=0x7fc32c010f90 comp rx=0 tx=0).stop 2024-04-22T13:29:10.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.546+0000 7fc34023d740 1 -- 172.21.15.12:0/3961453256 shutdown_connections 2024-04-22T13:29:10.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.546+0000 7fc34023d740 1 --2- 172.21.15.12:0/3961453256 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561b6a0baa00 0x561b6a0be950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.546+0000 7fc34023d740 1 --2- 172.21.15.12:0/3961453256 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x561b6a1312e0 0x561b6a1336d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.546+0000 7fc34023d740 1 --2- 172.21.15.12:0/3961453256 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561b6a0b1e80 0x561b6a0be410 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.546+0000 7fc31aff5640 0 client.17667 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:29:10.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.546+0000 7fc34023d740 1 --2- 172.21.15.12:0/3961453256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561b6a0b14d0 0x561b6a0c30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.546+0000 7fc34023d740 1 -- 172.21.15.12:0/3961453256 >> 172.21.15.12:0/3961453256 conn(0x561b6a0abba0 msgr2=0x561b6a0aac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:10.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.546+0000 7fc34023d740 1 -- 172.21.15.12:0/3961453256 shutdown_connections 2024-04-22T13:29:10.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.546+0000 7fc34023d740 1 -- 172.21.15.12:0/3961453256 wait complete. 2024-04-22T13:29:10.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.546+0000 7fc34023d740 20 client.17667 trim_cache size 0 max 16384 2024-04-22T13:29:10.549 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:29:10.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.546+0000 7fc33d650640 1 -- 172.21.15.12:0/1962393237 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc3200a3d20 msgr2=0x7fc3200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.546+0000 7fc33d650640 1 --2- 172.21.15.12:0/1962393237 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc3200a3d20 0x7fc3200a61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc330004af0 tx=0x7fc33002cf50 comp rx=0 tx=0).stop 2024-04-22T13:29:10.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.546+0000 7fc33d650640 1 -- 172.21.15.12:0/1962393237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc338107260 msgr2=0x7fc338198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.546+0000 7fc33d650640 1 --2- 172.21.15.12:0/1962393237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc338107260 0x7fc338198c70 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fc32c00d770 tx=0x7fc32c00dc30 comp rx=0 tx=0).stop 2024-04-22T13:29:10.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.547+0000 7fc33d650640 1 -- 172.21.15.12:0/1962393237 shutdown_connections 2024-04-22T13:29:10.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.547+0000 7fc33d650640 1 --2- 172.21.15.12:0/1962393237 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc3200a3d20 0x7fc3200a61e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.547+0000 7fc33d650640 1 --2- 172.21.15.12:0/1962393237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc33819cff0 0x7fc33819d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.547+0000 7fc33d650640 1 --2- 172.21.15.12:0/1962393237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc338107260 0x7fc338198c70 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.547+0000 7fc33d650640 1 --2- 172.21.15.12:0/1962393237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc338102190 0x7fc338198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.547+0000 7fc33d650640 1 -- 172.21.15.12:0/1962393237 >> 172.21.15.12:0/1962393237 conn(0x7fc3380fd990 msgr2=0x7fc3380ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:10.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.547+0000 7fc33d650640 1 -- 172.21.15.12:0/1962393237 shutdown_connections 2024-04-22T13:29:10.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.547+0000 7fc33d650640 1 -- 172.21.15.12:0/1962393237 wait complete. 2024-04-22T13:29:10.553 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:29:10.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.664+0000 7faa382e4640 1 -- 172.21.15.12:0/2104918840 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa30101a40 msgr2=0x7faa30105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.664+0000 7faa382e4640 1 --2- 172.21.15.12:0/2104918840 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa30101a40 0x7faa30105a80 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7faa20009900 tx=0x7faa2002bc30 comp rx=0 tx=0).stop 2024-04-22T13:29:10.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.664+0000 7faa382e4640 1 -- 172.21.15.12:0/2104918840 shutdown_connections 2024-04-22T13:29:10.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.664+0000 7faa382e4640 1 --2- 172.21.15.12:0/2104918840 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa30106160 0x7faa3010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.664+0000 7faa382e4640 1 --2- 172.21.15.12:0/2104918840 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa30101a40 0x7faa30105a80 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.664+0000 7faa382e4640 1 --2- 172.21.15.12:0/2104918840 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa30101090 0x7faa30101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.664+0000 7faa382e4640 1 -- 172.21.15.12:0/2104918840 >> 172.21.15.12:0/2104918840 conn(0x7faa30077f50 msgr2=0x7faa30078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:10.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.664+0000 7faa382e4640 1 -- 172.21.15.12:0/2104918840 shutdown_connections 2024-04-22T13:29:10.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.664+0000 7faa382e4640 1 -- 172.21.15.12:0/2104918840 wait complete. 2024-04-22T13:29:10.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.665+0000 7faa382e4640 1 Processor -- start 2024-04-22T13:29:10.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.665+0000 7faa382e4640 1 -- start start 2024-04-22T13:29:10.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.665+0000 7faa382e4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa30101090 0x7faa30102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.665+0000 7faa382e4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa30106160 0x7faa30103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.665+0000 7faa382e4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa301047c0 0x7faa301035a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.665+0000 7faa382e4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7faa301110f0 con 0x7faa30101090 2024-04-22T13:29:10.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.665+0000 7faa382e4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faa30110f70 con 0x7faa301047c0 2024-04-22T13:29:10.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.665+0000 7faa382e4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faa30111270 con 0x7faa30106160 2024-04-22T13:29:10.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.666+0000 7faa36080640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa30101090 0x7faa30102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.666+0000 7faa36080640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa30101090 0x7faa30102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52404/0 (socket says 172.21.15.12:52404) 2024-04-22T13:29:10.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.666+0000 7faa36080640 1 -- 172.21.15.12:0/2154695639 learned_addr learned my addr 172.21.15.12:0/2154695639 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:10.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.666+0000 7faa36881640 1 --2- 172.21.15.12:0/2154695639 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa301047c0 0x7faa301035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.666+0000 7faa3587f640 1 --2- 172.21.15.12:0/2154695639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa30106160 0x7faa30103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.666+0000 7faa36080640 1 -- 172.21.15.12:0/2154695639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa30106160 msgr2=0x7faa30103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.666+0000 7faa36080640 1 --2- 172.21.15.12:0/2154695639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa30106160 0x7faa30103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.666+0000 7faa36080640 1 -- 172.21.15.12:0/2154695639 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa301047c0 msgr2=0x7faa301035a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.666+0000 7faa36080640 1 --2- 172.21.15.12:0/2154695639 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa301047c0 0x7faa301035a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.666+0000 7faa36080640 1 -- 172.21.15.12:0/2154695639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa301ab420 con 0x7faa30101090 2024-04-22T13:29:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.666+0000 7faa3587f640 1 --2- 172.21.15.12:0/2154695639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa30106160 0x7faa30103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.666+0000 7faa36080640 1 --2- 172.21.15.12:0/2154695639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa30101090 0x7faa30102b20 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00e9f0 tx=0x7faa2c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:10.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.668+0000 7faa277fe640 1 -- 172.21.15.12:0/2154695639 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2c00cd80 con 0x7faa30101090 2024-04-22T13:29:10.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.668+0000 7faa277fe640 1 -- 172.21.15.12:0/2154695639 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa2c0108e0 con 0x7faa30101090 2024-04-22T13:29:10.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.668+0000 7faa277fe640 1 -- 172.21.15.12:0/2154695639 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2c018980 con 0x7faa30101090 2024-04-22T13:29:10.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.668+0000 7faa382e4640 1 -- 172.21.15.12:0/2154695639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa301ab710 con 0x7faa30101090 2024-04-22T13:29:10.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.668+0000 7faa382e4640 1 -- 172.21.15.12:0/2154695639 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa301abc30 con 0x7faa30101090 2024-04-22T13:29:10.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.671+0000 7faa277fe640 1 -- 172.21.15.12:0/2154695639 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faa2c010430 con 0x7faa30101090 2024-04-22T13:29:10.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.671+0000 7faa3afb4740 1 Processor -- start 2024-04-22T13:29:10.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.671+0000 7faa3afb4740 1 -- start start 2024-04-22T13:29:10.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.671+0000 7faa3afb4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610a34854d0 0x5610a34858b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.671+0000 7faa3afb4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610a3485e80 0x5610a348e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.671+0000 7faa3afb4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610a348ea00 0x5610a3490e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.671+0000 7faa3afb4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5610a33ea1f0 con 0x5610a348ea00 2024-04-22T13:29:10.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.671+0000 7faa3afb4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5610a33874b0 con 0x5610a3485e80 2024-04-22T13:29:10.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.671+0000 7faa3afb4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5610a33e61b0 con 0x5610a34854d0 2024-04-22T13:29:10.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.672+0000 7faa3587f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610a348ea00 0x5610a3490e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.672+0000 7faa36881640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610a34854d0 0x5610a34858b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.672+0000 7faa277fe640 1 --2- 172.21.15.12:0/2154695639 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7faa1c0a3d20 0x7faa1c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.672+0000 7faa3587f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610a348ea00 0x5610a3490e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52408/0 (socket says 172.21.15.12:52408) 2024-04-22T13:29:10.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.672+0000 7faa3587f640 1 -- 172.21.15.12:0/1237958912 learned_addr learned my addr 172.21.15.12:0/1237958912 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:10.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.672+0000 7faa36080640 1 --2- 172.21.15.12:0/1237958912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610a3485e80 0x5610a348e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.672+0000 7faa36881640 1 --2- 172.21.15.12:0/2154695639 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7faa1c0a3d20 0x7faa1c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.672+0000 7faa277fe640 1 -- 172.21.15.12:0/2154695639 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7faa2c0d0b40 con 0x7faa30101090 2024-04-22T13:29:10.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.672+0000 7faa3587f640 1 -- 172.21.15.12:0/1237958912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610a34854d0 msgr2=0x5610a34858b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.672+0000 7faa3587f640 1 --2- 172.21.15.12:0/1237958912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610a34854d0 0x5610a34858b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.672+0000 7faa3587f640 1 -- 172.21.15.12:0/1237958912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610a3485e80 msgr2=0x5610a348e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.672+0000 7faa3587f640 1 --2- 172.21.15.12:0/1237958912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610a3485e80 0x5610a348e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.672+0000 7faa3587f640 1 -- 172.21.15.12:0/1237958912 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610a33791b0 con 0x5610a348ea00 2024-04-22T13:29:10.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.672+0000 7faa36881640 1 --2- 172.21.15.12:0/1237958912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610a34854d0 0x5610a34858b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:29:10.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.672+0000 7faa3587f640 1 --2- 172.21.15.12:0/1237958912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610a348ea00 0x5610a3490e50 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7faa20000c00 tx=0x7faa2002c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec7f509b831541e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:10.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.672+0000 7faa36881640 1 --2- 172.21.15.12:0/2154695639 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7faa1c0a3d20 0x7faa1c0a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faa28009fd0 tx=0x7faa28004ab0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:10.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.672+0000 7faa24bfe640 1 -- 172.21.15.12:0/1237958912 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa20044020 con 0x5610a348ea00 2024-04-22T13:29:10.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.672+0000 7faa24bfe640 1 -- 172.21.15.12:0/1237958912 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa2002cc90 con 0x5610a348ea00 2024-04-22T13:29:10.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.672+0000 7faa24bfe640 1 -- 172.21.15.12:0/1237958912 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa200046c0 con 0x5610a348ea00 2024-04-22T13:29:10.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.673+0000 7faa3afb4740 1 -- 172.21.15.12:0/1237958912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610a348ea00 msgr2=0x5610a3490e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.673+0000 7faa3afb4740 1 --2- 172.21.15.12:0/1237958912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610a348ea00 0x5610a3490e50 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7faa20000c00 tx=0x7faa2002c5d0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.673+0000 7faa3afb4740 1 -- 172.21.15.12:0/1237958912 shutdown_connections 2024-04-22T13:29:10.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.673+0000 7faa3afb4740 1 --2- 172.21.15.12:0/1237958912 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610a348ea00 0x5610a3490e50 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.673+0000 7faa3afb4740 1 --2- 172.21.15.12:0/1237958912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610a3485e80 0x5610a348e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.673+0000 7faa3afb4740 1 --2- 172.21.15.12:0/1237958912 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610a34854d0 0x5610a34858b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.673+0000 7faa3afb4740 1 -- 172.21.15.12:0/1237958912 >> 172.21.15.12:0/1237958912 conn(0x5610a347fba0 msgr2=0x5610a347ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:10.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.673+0000 7faa3afb4740 1 -- 172.21.15.12:0/1237958912 shutdown_connections 2024-04-22T13:29:10.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.673+0000 7faa3afb4740 1 -- 172.21.15.12:0/1237958912 wait complete. 2024-04-22T13:29:10.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.674+0000 7faa3afb4740 1 -- start start 2024-04-22T13:29:10.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.674+0000 7faa3afb4740 1 Processor -- start 2024-04-22T13:29:10.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.674+0000 7faa3afb4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610a34854d0 0x5610a348e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.674+0000 7faa3afb4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610a3485e80 0x5610a348b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.674+0000 7faa3afb4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610a348ea00 0x5610a348bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.674+0000 7faa3afb4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5610a3491910 con 0x5610a3485e80 2024-04-22T13:29:10.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.674+0000 7faa3afb4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5610a33a3500 con 0x5610a348ea00 2024-04-22T13:29:10.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.674+0000 7faa3afb4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5610a3491a90 con 0x5610a34854d0 2024-04-22T13:29:10.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.674+0000 7faa3afb4740 10 client.0 _ll_register_callbacks cb 0x5610a3371b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:10.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.674+0000 7faa3587f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610a3485e80 0x5610a348b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.674+0000 7faa3587f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610a3485e80 0x5610a348b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52412/0 (socket says 172.21.15.12:52412) 2024-04-22T13:29:10.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.674+0000 7faa3587f640 1 -- 172.21.15.12:0/642434390 learned_addr learned my addr 172.21.15.12:0/642434390 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:10.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.674+0000 7faa36881640 1 --2- 172.21.15.12:0/642434390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610a348ea00 0x5610a348bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.674+0000 7faa36080640 1 --2- 172.21.15.12:0/642434390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610a34854d0 0x5610a348e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.674+0000 7faa3587f640 1 -- 172.21.15.12:0/642434390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610a34854d0 msgr2=0x5610a348e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.674+0000 7faa3587f640 1 --2- 172.21.15.12:0/642434390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610a34854d0 0x5610a348e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.674+0000 7faa3587f640 1 -- 172.21.15.12:0/642434390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610a348ea00 msgr2=0x5610a348bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.674+0000 7faa3587f640 1 --2- 172.21.15.12:0/642434390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610a348ea00 0x5610a348bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.674+0000 7faa3587f640 1 -- 172.21.15.12:0/642434390 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610a30fc3f0 con 0x5610a3485e80 2024-04-22T13:29:10.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.675+0000 7faa3587f640 1 --2- 172.21.15.12:0/642434390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610a3485e80 0x5610a348b5f0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7faa2002c780 tx=0x7faa2002c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ddec4c6e1187fe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:10.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.675+0000 7faa19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:10.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.675+0000 7faa19ffb640 1 -- 172.21.15.12:0/642434390 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2004f070 con 0x5610a3485e80 2024-04-22T13:29:10.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.675+0000 7faa3afb4740 1 -- 172.21.15.12:0/642434390 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa2002cc90 con 0x5610a3485e80 2024-04-22T13:29:10.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.675+0000 7faa19ffb640 1 -- 172.21.15.12:0/642434390 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7faa200035f0 con 0x5610a3485e80 2024-04-22T13:29:10.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.675+0000 7faa19ffb640 1 -- 172.21.15.12:0/642434390 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faa2005b050 con 0x5610a3485e80 2024-04-22T13:29:10.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.675+0000 7faa3afb4740 10 client.17679 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:10.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.675+0000 7faa3afb4740 1 -- 172.21.15.12:0/642434390 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5610a31c67d0 con 0x5610a3485e80 2024-04-22T13:29:10.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.675+0000 7faa19ffb640 1 -- 172.21.15.12:0/642434390 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa2002cc90 con 0x5610a3485e80 2024-04-22T13:29:10.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.675+0000 7faa19ffb640 1 -- 172.21.15.12:0/642434390 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7faa20004850 con 0x5610a3485e80 2024-04-22T13:29:10.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.675+0000 7faa3afb4740 10 client.17679 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:10.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.675+0000 7faa3afb4740 10 client.17679 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:29:10.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.675+0000 7faa3afb4740 20 client.17679 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:10.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.675+0000 7faa3afb4740 1 --2- 172.21.15.12:0/642434390 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5610a349ebf0 0x5610a34a1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.675+0000 7faa3afb4740 4 client.17679 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:10.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.675+0000 7faa3afb4740 1 -- 172.21.15.12:0/642434390 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610a3389930 con 0x5610a349ebf0 2024-04-22T13:29:10.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.676+0000 7faa36080640 1 --2- 172.21.15.12:0/642434390 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5610a349ebf0 0x5610a34a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.676+0000 7faa36080640 1 --2- 172.21.15.12:0/642434390 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5610a349ebf0 0x5610a34a1040 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4fae3e9e326a4c00 server_cookie=4e07b746010522df in_seq=0 out_seq=0 2024-04-22T13:29:10.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.676+0000 7faa19ffb640 10 client.17679 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:29:10.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.677+0000 7faa19ffb640 1 -- 172.21.15.12:0/642434390 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5610a3389930 con 0x5610a349ebf0 2024-04-22T13:29:10.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.677+0000 7faa19ffb640 10 client.17679 handle_command_reply: tid=0 2024-04-22T13:29:10.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.677+0000 7faa3afb4740 1 client.17679 shutdown 2024-04-22T13:29:10.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.677+0000 7faa3afb4740 1 -- 172.21.15.12:0/642434390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610a3485e80 msgr2=0x5610a348b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.677+0000 7faa3afb4740 1 --2- 172.21.15.12:0/642434390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610a3485e80 0x5610a348b5f0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7faa2002c780 tx=0x7faa2002c360 comp rx=0 tx=0).stop 2024-04-22T13:29:10.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.677+0000 7faa3afb4740 1 -- 172.21.15.12:0/642434390 shutdown_connections 2024-04-22T13:29:10.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.677+0000 7faa3afb4740 1 --2- 172.21.15.12:0/642434390 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5610a349ebf0 0x5610a34a1040 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.677+0000 7faa3afb4740 1 --2- 172.21.15.12:0/642434390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610a348ea00 0x5610a348bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.677+0000 7faa3afb4740 1 --2- 172.21.15.12:0/642434390 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610a3485e80 0x5610a348b5f0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.677+0000 7faa3afb4740 1 --2- 172.21.15.12:0/642434390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610a34854d0 0x5610a348e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.677+0000 7faa3afb4740 1 -- 172.21.15.12:0/642434390 >> 172.21.15.12:0/642434390 conn(0x5610a347fba0 msgr2=0x5610a34818f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:10.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.677+0000 7faa19ffb640 0 client.17679 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:29:10.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.677+0000 7faa3afb4740 1 -- 172.21.15.12:0/642434390 shutdown_connections 2024-04-22T13:29:10.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.677+0000 7faa3afb4740 1 -- 172.21.15.12:0/642434390 wait complete. 2024-04-22T13:29:10.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.677+0000 7faa3afb4740 20 client.17679 trim_cache size 0 max 16384 2024-04-22T13:29:10.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.690+0000 7faa3afb4740 1 Processor -- start 2024-04-22T13:29:10.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.690+0000 7faa3afb4740 1 -- start start 2024-04-22T13:29:10.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.690+0000 7faa3afb4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610a34854d0 0x5610a349a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.690+0000 7faa3afb4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610a3485e80 0x5610a348ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.691+0000 7faa3afb4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610a348ea00 0x5610a34816c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.691+0000 7faa3afb4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7faa301b40d0 con 0x5610a34854d0 2024-04-22T13:29:10.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.691+0000 7faa3afb4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faa301b5c10 con 0x5610a3485e80 2024-04-22T13:29:10.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.691+0000 7faa3afb4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faa301b47a0 con 0x5610a348ea00 2024-04-22T13:29:10.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.691+0000 7faa36080640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610a34854d0 0x5610a349a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.691+0000 7faa36080640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610a34854d0 0x5610a349a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52420/0 (socket says 172.21.15.12:52420) 2024-04-22T13:29:10.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.691+0000 7faa36080640 1 -- 172.21.15.12:0/2690578024 learned_addr learned my addr 172.21.15.12:0/2690578024 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:10.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.691+0000 7faa36080640 1 -- 172.21.15.12:0/2690578024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610a348ea00 msgr2=0x5610a34816c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.691+0000 7faa3587f640 1 --2- 172.21.15.12:0/2690578024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610a3485e80 0x5610a348ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.691+0000 7faa36080640 1 --2- 172.21.15.12:0/2690578024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610a348ea00 0x5610a34816c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.691+0000 7faa36080640 1 -- 172.21.15.12:0/2690578024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610a3485e80 msgr2=0x5610a348ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.691+0000 7faa36080640 1 --2- 172.21.15.12:0/2690578024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610a3485e80 0x5610a348ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.691+0000 7faa36080640 1 -- 172.21.15.12:0/2690578024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610a3485be0 con 0x5610a34854d0 2024-04-22T13:29:10.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.691+0000 7faa3587f640 1 --2- 172.21.15.12:0/2690578024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610a3485e80 0x5610a348ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:10.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.691+0000 7faa36080640 1 --2- 172.21.15.12:0/2690578024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610a34854d0 0x5610a349a7d0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7faa2c0ce390 tx=0x7faa2c013700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa78f0812a43325 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:10.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.691+0000 7faa1bfff640 1 -- 172.21.15.12:0/2690578024 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2c09de60 con 0x5610a34854d0 2024-04-22T13:29:10.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.691+0000 7faa1bfff640 1 -- 172.21.15.12:0/2690578024 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa2c09cb30 con 0x5610a34854d0 2024-04-22T13:29:10.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.691+0000 7faa1bfff640 1 -- 172.21.15.12:0/2690578024 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2c010a90 con 0x5610a34854d0 2024-04-22T13:29:10.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.692+0000 7faa3afb4740 1 -- 172.21.15.12:0/2690578024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610a34854d0 msgr2=0x5610a349a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.692+0000 7faa3afb4740 1 --2- 172.21.15.12:0/2690578024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610a34854d0 0x5610a349a7d0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7faa2c0ce390 tx=0x7faa2c013700 comp rx=0 tx=0).stop 2024-04-22T13:29:10.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.692+0000 7faa3afb4740 1 -- 172.21.15.12:0/2690578024 shutdown_connections 2024-04-22T13:29:10.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.692+0000 7faa3afb4740 1 --2- 172.21.15.12:0/2690578024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610a348ea00 0x5610a34816c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.692+0000 7faa3afb4740 1 --2- 172.21.15.12:0/2690578024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610a3485e80 0x5610a348ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.692+0000 7faa3afb4740 1 --2- 172.21.15.12:0/2690578024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610a34854d0 0x5610a349a7d0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.692+0000 7faa3afb4740 1 -- 172.21.15.12:0/2690578024 >> 172.21.15.12:0/2690578024 conn(0x5610a347fba0 msgr2=0x5610a3480fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:10.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.692+0000 7faa3afb4740 1 -- 172.21.15.12:0/2690578024 shutdown_connections 2024-04-22T13:29:10.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.692+0000 7faa3afb4740 1 -- 172.21.15.12:0/2690578024 wait complete. 2024-04-22T13:29:10.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.692+0000 7faa3afb4740 1 -- start start 2024-04-22T13:29:10.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.692+0000 7faa3afb4740 1 Processor -- start 2024-04-22T13:29:10.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.692+0000 7faa3afb4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610a34854d0 0x5610a3491e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.692+0000 7faa3afb4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610a3485e80 0x5610a34923d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.692+0000 7faa3afb4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610a348ea00 0x5610a35031b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.692+0000 7faa3afb4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7faa301b5540 con 0x5610a3485e80 2024-04-22T13:29:10.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.692+0000 7faa3afb4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faa301b4e70 con 0x5610a348ea00 2024-04-22T13:29:10.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.692+0000 7faa3afb4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faa301b20d0 con 0x5610a34854d0 2024-04-22T13:29:10.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.692+0000 7faa3afb4740 10 client.0 _ll_register_callbacks cb 0x7faa20009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:10.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.692+0000 7faa3587f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610a3485e80 0x5610a34923d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.692+0000 7faa3587f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610a3485e80 0x5610a34923d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52430/0 (socket says 172.21.15.12:52430) 2024-04-22T13:29:10.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.692+0000 7faa3587f640 1 -- 172.21.15.12:0/3110801600 learned_addr learned my addr 172.21.15.12:0/3110801600 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:10.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.693+0000 7faa36080640 1 --2- 172.21.15.12:0/3110801600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610a34854d0 0x5610a3491e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.693+0000 7faa3587f640 1 -- 172.21.15.12:0/3110801600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610a34854d0 msgr2=0x5610a3491e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.693+0000 7faa3587f640 1 --2- 172.21.15.12:0/3110801600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610a34854d0 0x5610a3491e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.693+0000 7faa3587f640 1 -- 172.21.15.12:0/3110801600 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610a348ea00 msgr2=0x5610a35031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.693+0000 7faa3587f640 1 --2- 172.21.15.12:0/3110801600 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610a348ea00 0x5610a35031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.693+0000 7faa3587f640 1 -- 172.21.15.12:0/3110801600 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610a35036f0 con 0x5610a3485e80 2024-04-22T13:29:10.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.693+0000 7faa3587f640 1 --2- 172.21.15.12:0/3110801600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610a3485e80 0x5610a34923d0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7faa20006010 tx=0x7faa20005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be5cd09085b3aa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:10.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.693+0000 7faa19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:10.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.693+0000 7faa19ffb640 1 -- 172.21.15.12:0/3110801600 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2004f040 con 0x5610a3485e80 2024-04-22T13:29:10.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.693+0000 7faa19ffb640 1 -- 172.21.15.12:0/3110801600 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7faa200046c0 con 0x5610a3485e80 2024-04-22T13:29:10.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.693+0000 7faa3afb4740 1 -- 172.21.15.12:0/3110801600 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa2c09cb30 con 0x5610a3485e80 2024-04-22T13:29:10.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.693+0000 7faa19ffb640 1 -- 172.21.15.12:0/3110801600 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faa20044040 con 0x5610a3485e80 2024-04-22T13:29:10.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.693+0000 7faa3afb4740 10 client.17685 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:10.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.693+0000 7faa3afb4740 1 -- 172.21.15.12:0/3110801600 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5610a3504940 con 0x5610a3485e80 2024-04-22T13:29:10.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.693+0000 7faa19ffb640 1 -- 172.21.15.12:0/3110801600 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa2c09cb30 con 0x5610a3485e80 2024-04-22T13:29:10.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.693+0000 7faa19ffb640 1 -- 172.21.15.12:0/3110801600 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7faa20003b30 con 0x5610a3485e80 2024-04-22T13:29:10.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.693+0000 7faa3afb4740 10 client.17685 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:10.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.693+0000 7faa3afb4740 10 client.17685 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:29:10.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.694+0000 7faa3afb4740 20 client.17685 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:10.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.694+0000 7faa3afb4740 1 --2- 172.21.15.12:0/3110801600 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5610a3505090 0x5610a35074b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.694+0000 7faa3afb4740 4 client.17685 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:29:10.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.694+0000 7faa3afb4740 1 -- 172.21.15.12:0/3110801600 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5610a33f7c10 con 0x5610a3505090 2024-04-22T13:29:10.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.694+0000 7faa36080640 1 --2- 172.21.15.12:0/3110801600 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5610a3505090 0x5610a35074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.694+0000 7faa19ffb640 10 client.17685 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:29:10.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.694+0000 7faa36080640 1 --2- 172.21.15.12:0/3110801600 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5610a3505090 0x5610a35074b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8a6b79c182618ff6 server_cookie=264054b779f30e14 in_seq=0 out_seq=0 2024-04-22T13:29:10.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.694+0000 7faa19ffb640 1 -- 172.21.15.12:0/3110801600 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5610a33f7c10 con 0x5610a3505090 2024-04-22T13:29:10.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.694+0000 7faa19ffb640 10 client.17685 handle_command_reply: tid=0 2024-04-22T13:29:10.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.694+0000 7faa3afb4740 1 client.17685 shutdown 2024-04-22T13:29:10.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.694+0000 7faa3afb4740 1 -- 172.21.15.12:0/3110801600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610a3485e80 msgr2=0x5610a34923d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.694+0000 7faa3afb4740 1 --2- 172.21.15.12:0/3110801600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610a3485e80 0x5610a34923d0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7faa20006010 tx=0x7faa20005720 comp rx=0 tx=0).stop 2024-04-22T13:29:10.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.694+0000 7faa3afb4740 1 -- 172.21.15.12:0/3110801600 shutdown_connections 2024-04-22T13:29:10.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.694+0000 7faa3afb4740 1 --2- 172.21.15.12:0/3110801600 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5610a3505090 0x5610a35074b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.694+0000 7faa3afb4740 1 --2- 172.21.15.12:0/3110801600 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610a348ea00 0x5610a35031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.694+0000 7faa3afb4740 1 --2- 172.21.15.12:0/3110801600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610a3485e80 0x5610a34923d0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.694+0000 7faa19ffb640 0 client.17685 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:29:10.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.694+0000 7faa3afb4740 1 --2- 172.21.15.12:0/3110801600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5610a34854d0 0x5610a3491e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.694+0000 7faa3afb4740 1 -- 172.21.15.12:0/3110801600 >> 172.21.15.12:0/3110801600 conn(0x5610a347fba0 msgr2=0x5610a3486af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:10.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.695+0000 7faa3afb4740 1 -- 172.21.15.12:0/3110801600 shutdown_connections 2024-04-22T13:29:10.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.695+0000 7faa3afb4740 1 -- 172.21.15.12:0/3110801600 wait complete. 2024-04-22T13:29:10.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.695+0000 7faa3afb4740 20 client.17685 trim_cache size 0 max 16384 2024-04-22T13:29:10.699 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:29:10.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.695+0000 7faa382e4640 1 -- 172.21.15.12:0/2154695639 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7faa1c0a3d20 msgr2=0x7faa1c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.695+0000 7faa382e4640 1 --2- 172.21.15.12:0/2154695639 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7faa1c0a3d20 0x7faa1c0a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faa28009fd0 tx=0x7faa28004ab0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.695+0000 7faa382e4640 1 -- 172.21.15.12:0/2154695639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa30101090 msgr2=0x7faa30102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.695+0000 7faa382e4640 1 --2- 172.21.15.12:0/2154695639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa30101090 0x7faa30102b20 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00e9f0 tx=0x7faa2c00eeb0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.695+0000 7faa382e4640 1 -- 172.21.15.12:0/2154695639 shutdown_connections 2024-04-22T13:29:10.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.695+0000 7faa382e4640 1 --2- 172.21.15.12:0/2154695639 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7faa1c0a3d20 0x7faa1c0a61e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.695+0000 7faa382e4640 1 --2- 172.21.15.12:0/2154695639 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa301047c0 0x7faa301035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.695+0000 7faa382e4640 1 --2- 172.21.15.12:0/2154695639 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa30106160 0x7faa30103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.695+0000 7faa382e4640 1 --2- 172.21.15.12:0/2154695639 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faa30101090 0x7faa30102b20 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.695+0000 7faa382e4640 1 -- 172.21.15.12:0/2154695639 >> 172.21.15.12:0/2154695639 conn(0x7faa30077f50 msgr2=0x7faa30069480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:10.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.695+0000 7faa382e4640 1 -- 172.21.15.12:0/2154695639 shutdown_connections 2024-04-22T13:29:10.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.695+0000 7faa382e4640 1 -- 172.21.15.12:0/2154695639 wait complete. 2024-04-22T13:29:10.701 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:29:10.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.810+0000 7faffaff4640 1 -- 172.21.15.12:0/2758878447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faff4107270 msgr2=0x7faff410ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.810+0000 7faffaff4640 1 --2- 172.21.15.12:0/2758878447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faff4107270 0x7faff410ee40 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fafec009ca0 tx=0x7fafec02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.810+0000 7faffaff4640 1 -- 172.21.15.12:0/2758878447 shutdown_connections 2024-04-22T13:29:10.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.810+0000 7faffaff4640 1 --2- 172.21.15.12:0/2758878447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faff4107270 0x7faff410ee40 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.810+0000 7faffaff4640 1 --2- 172.21.15.12:0/2758878447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faff406b1f0 0x7faff4106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.810+0000 7faffaff4640 1 --2- 172.21.15.12:0/2758878447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faff406a840 0x7faff406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.810+0000 7faffaff4640 1 -- 172.21.15.12:0/2758878447 >> 172.21.15.12:0/2758878447 conn(0x7faff4077eb0 msgr2=0x7faff40782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:10.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.810+0000 7faffaff4640 1 -- 172.21.15.12:0/2758878447 shutdown_connections 2024-04-22T13:29:10.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.810+0000 7faffaff4640 1 -- 172.21.15.12:0/2758878447 wait complete. 2024-04-22T13:29:10.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.810+0000 7faffaff4640 1 Processor -- start 2024-04-22T13:29:10.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.811+0000 7faffaff4640 1 -- start start 2024-04-22T13:29:10.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.811+0000 7faffaff4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faff406a840 0x7faff41986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.811+0000 7faffaff4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faff4107270 0x7faff4198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.811+0000 7faffaff4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faff419cf60 0x7faff419d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.811+0000 7faffaff4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7faff41121e0 con 0x7faff419cf60 2024-04-22T13:29:10.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.811+0000 7faffaff4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faff4112060 con 0x7faff4107270 2024-04-22T13:29:10.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.811+0000 7faffaff4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faff4112360 con 0x7faff406a840 2024-04-22T13:29:10.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.811+0000 7faff9591640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faff419cf60 0x7faff419d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.811+0000 7faff9591640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faff419cf60 0x7faff419d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52444/0 (socket says 172.21.15.12:52444) 2024-04-22T13:29:10.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.811+0000 7faff9591640 1 -- 172.21.15.12:0/1139689672 learned_addr learned my addr 172.21.15.12:0/1139689672 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:10.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.811+0000 7faff8d90640 1 --2- 172.21.15.12:0/1139689672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faff406a840 0x7faff41986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.811+0000 7fafebfff640 1 --2- 172.21.15.12:0/1139689672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faff4107270 0x7faff4198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.811+0000 7faff9591640 1 -- 172.21.15.12:0/1139689672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faff406a840 msgr2=0x7faff41986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.811+0000 7faff9591640 1 --2- 172.21.15.12:0/1139689672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faff406a840 0x7faff41986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.811+0000 7faff9591640 1 -- 172.21.15.12:0/1139689672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faff4107270 msgr2=0x7faff4198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.811+0000 7faff9591640 1 --2- 172.21.15.12:0/1139689672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faff4107270 0x7faff4198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.811+0000 7faff9591640 1 -- 172.21.15.12:0/1139689672 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff419da50 con 0x7faff419cf60 2024-04-22T13:29:10.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.811+0000 7faff9591640 1 --2- 172.21.15.12:0/1139689672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faff419cf60 0x7faff419d410 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fafec005e30 tx=0x7fafec0346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:10.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.811+0000 7fafe9ffb640 1 -- 172.21.15.12:0/1139689672 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec041020 con 0x7faff419cf60 2024-04-22T13:29:10.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.812+0000 7fafe9ffb640 1 -- 172.21.15.12:0/1139689672 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafec032970 con 0x7faff419cf60 2024-04-22T13:29:10.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.812+0000 7faffaff4640 1 -- 172.21.15.12:0/1139689672 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff419dce0 con 0x7faff419cf60 2024-04-22T13:29:10.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.812+0000 7fafe9ffb640 1 -- 172.21.15.12:0/1139689672 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafec039040 con 0x7faff419cf60 2024-04-22T13:29:10.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.812+0000 7faffaff4640 1 -- 172.21.15.12:0/1139689672 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff41a54a0 con 0x7faff419cf60 2024-04-22T13:29:10.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.815+0000 7fafe9ffb640 1 -- 172.21.15.12:0/1139689672 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fafec044020 con 0x7faff419cf60 2024-04-22T13:29:10.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.815+0000 7faffdbe6740 1 Processor -- start 2024-04-22T13:29:10.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.815+0000 7faffdbe6740 1 -- start start 2024-04-22T13:29:10.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.815+0000 7faffdbe6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f201bb4d0 0x555f201bb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.815+0000 7faffdbe6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f201bbe80 0x555f201c4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.815+0000 7faffdbe6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f201c4a00 0x555f201c6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.815+0000 7faffdbe6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555f201201f0 con 0x555f201c4a00 2024-04-22T13:29:10.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.815+0000 7faffdbe6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555f200bd4b0 con 0x555f201bb4d0 2024-04-22T13:29:10.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.815+0000 7faffdbe6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555f2011c1b0 con 0x555f201bbe80 2024-04-22T13:29:10.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.815+0000 7fafebfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f201c4a00 0x555f201c6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.815+0000 7fafebfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f201c4a00 0x555f201c6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52460/0 (socket says 172.21.15.12:52460) 2024-04-22T13:29:10.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.815+0000 7fafebfff640 1 -- 172.21.15.12:0/2118327027 learned_addr learned my addr 172.21.15.12:0/2118327027 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:10.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.815+0000 7faff8d90640 1 --2- 172.21.15.12:0/2118327027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f201bbe80 0x555f201c4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.815+0000 7fafebfff640 1 --2- 172.21.15.12:0/2118327027 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f201bb4d0 0x555f201bb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.815+0000 7fafe9ffb640 1 --2- 172.21.15.12:0/1139689672 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fafdc0a3d20 0x7fafdc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.815+0000 7faff9591640 1 --2- 172.21.15.12:0/1139689672 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fafdc0a3d20 0x7fafdc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.815+0000 7fafebfff640 1 -- 172.21.15.12:0/2118327027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f201bbe80 msgr2=0x555f201c4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.815+0000 7fafebfff640 1 --2- 172.21.15.12:0/2118327027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f201bbe80 0x555f201c4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.815+0000 7fafebfff640 1 -- 172.21.15.12:0/2118327027 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f201bb4d0 msgr2=0x555f201bb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.816+0000 7fafebfff640 1 --2- 172.21.15.12:0/2118327027 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f201bb4d0 0x555f201bb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.816+0000 7fafebfff640 1 -- 172.21.15.12:0/2118327027 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f200af1b0 con 0x555f201c4a00 2024-04-22T13:29:10.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.816+0000 7fafe9ffb640 1 -- 172.21.15.12:0/1139689672 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fafec0f22a0 con 0x7faff419cf60 2024-04-22T13:29:10.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.816+0000 7fafebfff640 1 --2- 172.21.15.12:0/2118327027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f201c4a00 0x555f201c6e50 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fafe4011740 tx=0x7fafe4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad903232f5615af7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:10.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.816+0000 7fafdaffd640 1 -- 172.21.15.12:0/2118327027 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe4004050 con 0x555f201c4a00 2024-04-22T13:29:10.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.816+0000 7fafdaffd640 1 -- 172.21.15.12:0/2118327027 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafe400fc70 con 0x555f201c4a00 2024-04-22T13:29:10.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.816+0000 7fafdaffd640 1 -- 172.21.15.12:0/2118327027 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe40038e0 con 0x555f201c4a00 2024-04-22T13:29:10.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.816+0000 7faffdbe6740 1 -- 172.21.15.12:0/2118327027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f201c4a00 msgr2=0x555f201c6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.816+0000 7faff9591640 1 --2- 172.21.15.12:0/1139689672 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fafdc0a3d20 0x7fafdc0a61e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fafec0f0040 tx=0x7fafec02e040 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:10.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.816+0000 7faffdbe6740 1 --2- 172.21.15.12:0/2118327027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f201c4a00 0x555f201c6e50 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fafe4011740 tx=0x7fafe4004730 comp rx=0 tx=0).stop 2024-04-22T13:29:10.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.817+0000 7faffdbe6740 1 -- 172.21.15.12:0/2118327027 shutdown_connections 2024-04-22T13:29:10.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.817+0000 7faffdbe6740 1 --2- 172.21.15.12:0/2118327027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f201c4a00 0x555f201c6e50 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.817+0000 7faffdbe6740 1 --2- 172.21.15.12:0/2118327027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f201bbe80 0x555f201c4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.817+0000 7faffdbe6740 1 --2- 172.21.15.12:0/2118327027 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f201bb4d0 0x555f201bb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.817+0000 7faffdbe6740 1 -- 172.21.15.12:0/2118327027 >> 172.21.15.12:0/2118327027 conn(0x555f201b5ba0 msgr2=0x555f201b5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:10.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.817+0000 7faffdbe6740 1 -- 172.21.15.12:0/2118327027 shutdown_connections 2024-04-22T13:29:10.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.817+0000 7faffdbe6740 1 -- 172.21.15.12:0/2118327027 wait complete. 2024-04-22T13:29:10.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.817+0000 7faffdbe6740 1 -- start start 2024-04-22T13:29:10.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.817+0000 7faffdbe6740 1 Processor -- start 2024-04-22T13:29:10.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.817+0000 7faffdbe6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f201bb4d0 0x555f201c4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.817+0000 7faffdbe6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f201bbe80 0x555f201c1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.817+0000 7faffdbe6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f201c4a00 0x555f201c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.817+0000 7faffdbe6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555f201c7970 con 0x555f201bbe80 2024-04-22T13:29:10.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.817+0000 7faffdbe6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555f200d9500 con 0x555f201bb4d0 2024-04-22T13:29:10.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.817+0000 7faffdbe6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555f201c7af0 con 0x555f201c4a00 2024-04-22T13:29:10.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.817+0000 7faffdbe6740 10 client.0 _ll_register_callbacks cb 0x555f200a7b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:10.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.818+0000 7faff8d90640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f201bbe80 0x555f201c1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.818+0000 7faff8d90640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f201bbe80 0x555f201c1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52472/0 (socket says 172.21.15.12:52472) 2024-04-22T13:29:10.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.818+0000 7faff8d90640 1 -- 172.21.15.12:0/466325909 learned_addr learned my addr 172.21.15.12:0/466325909 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:10.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.818+0000 7fafebfff640 1 --2- 172.21.15.12:0/466325909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f201c4a00 0x555f201c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.818+0000 7fafebfff640 1 --2- 172.21.15.12:0/466325909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f201bb4d0 0x555f201c4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.818+0000 7faff8d90640 1 -- 172.21.15.12:0/466325909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f201c4a00 msgr2=0x555f201c1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.818+0000 7faff8d90640 1 --2- 172.21.15.12:0/466325909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f201c4a00 0x555f201c1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.818+0000 7faff8d90640 1 -- 172.21.15.12:0/466325909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f201bb4d0 msgr2=0x555f201c4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.818+0000 7faff8d90640 1 --2- 172.21.15.12:0/466325909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f201bb4d0 0x555f201c4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.818+0000 7faff8d90640 1 -- 172.21.15.12:0/466325909 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f1fe323f0 con 0x555f201bbe80 2024-04-22T13:29:10.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.818+0000 7fafebfff640 1 --2- 172.21.15.12:0/466325909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f201bb4d0 0x555f201c4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:10.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.818+0000 7faff8d90640 1 --2- 172.21.15.12:0/466325909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f201bbe80 0x555f201c1690 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7faff00126e0 tx=0x7faff0012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa829a5b3ed64b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:10.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.818+0000 7fafd3fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:10.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.818+0000 7fafd3fff640 1 -- 172.21.15.12:0/466325909 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff0012190 con 0x555f201bbe80 2024-04-22T13:29:10.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.818+0000 7fafd3fff640 1 -- 172.21.15.12:0/466325909 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7faff000f920 con 0x555f201bbe80 2024-04-22T13:29:10.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.818+0000 7faffdbe6740 1 -- 172.21.15.12:0/466325909 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafe400fc70 con 0x555f201bbe80 2024-04-22T13:29:10.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.818+0000 7fafd3fff640 1 -- 172.21.15.12:0/466325909 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faff000fb20 con 0x555f201bbe80 2024-04-22T13:29:10.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.818+0000 7faffdbe6740 10 client.17697 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:10.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.818+0000 7faffdbe6740 1 -- 172.21.15.12:0/466325909 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555f1fefc7d0 con 0x555f201bbe80 2024-04-22T13:29:10.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.819+0000 7fafd3fff640 1 -- 172.21.15.12:0/466325909 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafe400fc70 con 0x555f201bbe80 2024-04-22T13:29:10.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.819+0000 7fafd3fff640 1 -- 172.21.15.12:0/466325909 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7faff0010cb0 con 0x555f201bbe80 2024-04-22T13:29:10.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.819+0000 7faffdbe6740 10 client.17697 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:10.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.819+0000 7faffdbe6740 10 client.17697 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:10.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.819+0000 7faffdbe6740 20 client.17697 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:10.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.819+0000 7faffdbe6740 1 --2- 172.21.15.12:0/466325909 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555f201d4d70 0x555f201d7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.819+0000 7faffdbe6740 4 client.17697 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:10.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.819+0000 7faffdbe6740 1 -- 172.21.15.12:0/466325909 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f200bf930 con 0x555f201d4d70 2024-04-22T13:29:10.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.819+0000 7fafebfff640 1 --2- 172.21.15.12:0/466325909 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555f201d4d70 0x555f201d7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.819+0000 7fafebfff640 1 --2- 172.21.15.12:0/466325909 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555f201d4d70 0x555f201d7190 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ab4e75e0ab0bab0 server_cookie=831d682172cb4e92 in_seq=0 out_seq=0 2024-04-22T13:29:10.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.819+0000 7fafd3fff640 10 client.17697 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:10.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.820+0000 7fafd3fff640 1 -- 172.21.15.12:0/466325909 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555f200bf930 con 0x555f201d4d70 2024-04-22T13:29:10.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.820+0000 7fafd3fff640 10 client.17697 handle_command_reply: tid=0 2024-04-22T13:29:10.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.820+0000 7faffdbe6740 1 client.17697 shutdown 2024-04-22T13:29:10.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.820+0000 7faffdbe6740 1 -- 172.21.15.12:0/466325909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f201bbe80 msgr2=0x555f201c1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.820+0000 7faffdbe6740 1 --2- 172.21.15.12:0/466325909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f201bbe80 0x555f201c1690 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7faff00126e0 tx=0x7faff0012ba0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.820+0000 7faffdbe6740 1 -- 172.21.15.12:0/466325909 shutdown_connections 2024-04-22T13:29:10.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.820+0000 7faffdbe6740 1 --2- 172.21.15.12:0/466325909 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555f201d4d70 0x555f201d7190 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.820+0000 7faffdbe6740 1 --2- 172.21.15.12:0/466325909 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f201c4a00 0x555f201c1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.820+0000 7faffdbe6740 1 --2- 172.21.15.12:0/466325909 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f201bbe80 0x555f201c1690 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.820+0000 7faffdbe6740 1 --2- 172.21.15.12:0/466325909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f201bb4d0 0x555f201c4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.820+0000 7fafd3fff640 0 client.17697 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:10.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.820+0000 7faffdbe6740 1 -- 172.21.15.12:0/466325909 >> 172.21.15.12:0/466325909 conn(0x555f201b5ba0 msgr2=0x555f201b78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:10.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.820+0000 7faffdbe6740 1 -- 172.21.15.12:0/466325909 shutdown_connections 2024-04-22T13:29:10.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.820+0000 7faffdbe6740 1 -- 172.21.15.12:0/466325909 wait complete. 2024-04-22T13:29:10.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.821+0000 7faffdbe6740 20 client.17697 trim_cache size 0 max 16384 2024-04-22T13:29:10.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.834+0000 7faffdbe6740 1 Processor -- start 2024-04-22T13:29:10.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.834+0000 7faffdbe6740 1 -- start start 2024-04-22T13:29:10.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.834+0000 7faffdbe6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f201bb4d0 0x555f201d0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.834+0000 7faffdbe6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f201bbe80 0x555f201c3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.834+0000 7faffdbe6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f201c4a00 0x555f201c4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.834+0000 7faffdbe6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7faff41ad990 con 0x555f201bbe80 2024-04-22T13:29:10.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.834+0000 7faffdbe6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faff41af4d0 con 0x555f201c4a00 2024-04-22T13:29:10.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.834+0000 7faffdbe6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faff41ae060 con 0x555f201bb4d0 2024-04-22T13:29:10.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.834+0000 7faff8d90640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f201bbe80 0x555f201c3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.834+0000 7faff8d90640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f201bbe80 0x555f201c3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52476/0 (socket says 172.21.15.12:52476) 2024-04-22T13:29:10.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.834+0000 7faff8d90640 1 -- 172.21.15.12:0/4069411849 learned_addr learned my addr 172.21.15.12:0/4069411849 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:10.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.834+0000 7fafebfff640 1 --2- 172.21.15.12:0/4069411849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f201c4a00 0x555f201c4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.834+0000 7fafebfff640 1 --2- 172.21.15.12:0/4069411849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f201bb4d0 0x555f201d0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.834+0000 7faff8d90640 1 -- 172.21.15.12:0/4069411849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f201bb4d0 msgr2=0x555f201d0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.834+0000 7faff8d90640 1 --2- 172.21.15.12:0/4069411849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f201bb4d0 0x555f201d0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.834+0000 7faff8d90640 1 -- 172.21.15.12:0/4069411849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f201c4a00 msgr2=0x555f201c4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.834+0000 7faff8d90640 1 --2- 172.21.15.12:0/4069411849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f201c4a00 0x555f201c4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.835+0000 7faff8d90640 1 -- 172.21.15.12:0/4069411849 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f201d3870 con 0x555f201bbe80 2024-04-22T13:29:10.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.835+0000 7faff8d90640 1 --2- 172.21.15.12:0/4069411849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f201bbe80 0x555f201c3dc0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7faff00098f0 tx=0x7faff000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dbb83bb8c9eaf17 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:10.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.835+0000 7fafda7fc640 1 -- 172.21.15.12:0/4069411849 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff0029070 con 0x555f201bbe80 2024-04-22T13:29:10.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.835+0000 7fafda7fc640 1 -- 172.21.15.12:0/4069411849 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faff000fcc0 con 0x555f201bbe80 2024-04-22T13:29:10.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.835+0000 7fafda7fc640 1 -- 172.21.15.12:0/4069411849 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff00054b0 con 0x555f201bbe80 2024-04-22T13:29:10.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.835+0000 7faffdbe6740 1 -- 172.21.15.12:0/4069411849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f201bbe80 msgr2=0x555f201c3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.835+0000 7faffdbe6740 1 --2- 172.21.15.12:0/4069411849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f201bbe80 0x555f201c3dc0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7faff00098f0 tx=0x7faff000fab0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.835+0000 7faffdbe6740 1 -- 172.21.15.12:0/4069411849 shutdown_connections 2024-04-22T13:29:10.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.835+0000 7faffdbe6740 1 --2- 172.21.15.12:0/4069411849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f201c4a00 0x555f201c4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.835+0000 7faffdbe6740 1 --2- 172.21.15.12:0/4069411849 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f201bbe80 0x555f201c3dc0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.835+0000 7faffdbe6740 1 --2- 172.21.15.12:0/4069411849 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f201bb4d0 0x555f201d0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.835+0000 7faffdbe6740 1 -- 172.21.15.12:0/4069411849 >> 172.21.15.12:0/4069411849 conn(0x555f201b5ba0 msgr2=0x555f201bcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:10.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.835+0000 7faffdbe6740 1 -- 172.21.15.12:0/4069411849 shutdown_connections 2024-04-22T13:29:10.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.836+0000 7faffdbe6740 1 -- 172.21.15.12:0/4069411849 wait complete. 2024-04-22T13:29:10.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.836+0000 7faffdbe6740 1 -- start start 2024-04-22T13:29:10.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.836+0000 7faffdbe6740 1 Processor -- start 2024-04-22T13:29:10.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.836+0000 7faffdbe6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f201bb4d0 0x555f201cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.836+0000 7faffdbe6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f201bbe80 0x555f201be180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.836+0000 7faffdbe6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f201c4a00 0x555f201be8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.836+0000 7faffdbe6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7faff41aee00 con 0x555f201c4a00 2024-04-22T13:29:10.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.836+0000 7faffdbe6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faff41ae730 con 0x555f201bbe80 2024-04-22T13:29:10.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.836+0000 7faffdbe6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faff41ab990 con 0x555f201bb4d0 2024-04-22T13:29:10.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.836+0000 7faffdbe6740 10 client.0 _ll_register_callbacks cb 0x7faff0009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:10.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.836+0000 7fafebfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f201c4a00 0x555f201be8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.836+0000 7fafebfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f201c4a00 0x555f201be8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:52492/0 (socket says 172.21.15.12:52492) 2024-04-22T13:29:10.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.836+0000 7fafebfff640 1 -- 172.21.15.12:0/1614645036 learned_addr learned my addr 172.21.15.12:0/1614645036 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:10.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.837+0000 7fafebfff640 1 --2- 172.21.15.12:0/1614645036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f201bb4d0 0x555f201cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.837+0000 7faff8d90640 1 --2- 172.21.15.12:0/1614645036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f201bbe80 0x555f201be180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.837+0000 7fafebfff640 1 -- 172.21.15.12:0/1614645036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f201bb4d0 msgr2=0x555f201cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.837+0000 7fafebfff640 1 --2- 172.21.15.12:0/1614645036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f201bb4d0 0x555f201cd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.837+0000 7fafebfff640 1 -- 172.21.15.12:0/1614645036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f201bbe80 msgr2=0x555f201be180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:10.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.837+0000 7fafebfff640 1 --2- 172.21.15.12:0/1614645036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f201bbe80 0x555f201be180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.837+0000 7fafebfff640 1 -- 172.21.15.12:0/1614645036 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f2023a2d0 con 0x555f201c4a00 2024-04-22T13:29:10.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.837+0000 7faff8d90640 1 --2- 172.21.15.12:0/1614645036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f201bbe80 0x555f201be180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:10.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.837+0000 7fafebfff640 1 --2- 172.21.15.12:0/1614645036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f201c4a00 0x555f201be8c0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fafe4004620 tx=0x7faff0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7705fa7276e648e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:10.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.837+0000 7fafd3fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:10.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.837+0000 7fafd3fff640 1 -- 172.21.15.12:0/1614645036 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe400f840 con 0x555f201c4a00 2024-04-22T13:29:10.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.837+0000 7fafd3fff640 1 -- 172.21.15.12:0/1614645036 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fafe400f9e0 con 0x555f201c4a00 2024-04-22T13:29:10.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.837+0000 7faffdbe6740 1 -- 172.21.15.12:0/1614645036 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faff000fcc0 con 0x555f201c4a00 2024-04-22T13:29:10.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.837+0000 7fafd3fff640 1 -- 172.21.15.12:0/1614645036 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fafe400fbe0 con 0x555f201c4a00 2024-04-22T13:29:10.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.837+0000 7faffdbe6740 10 client.17703 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:10.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.837+0000 7faffdbe6740 1 -- 172.21.15.12:0/1614645036 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555f2023b3b0 con 0x555f201c4a00 2024-04-22T13:29:10.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.837+0000 7fafd3fff640 1 -- 172.21.15.12:0/1614645036 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faff000fcc0 con 0x555f201c4a00 2024-04-22T13:29:10.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.837+0000 7fafd3fff640 1 -- 172.21.15.12:0/1614645036 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fafe401f670 con 0x555f201c4a00 2024-04-22T13:29:10.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.838+0000 7faffdbe6740 10 client.17703 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:10.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.838+0000 7faffdbe6740 10 client.17703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:10.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.838+0000 7faffdbe6740 20 client.17703 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:10.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.838+0000 7faffdbe6740 1 --2- 172.21.15.12:0/1614645036 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555f2023b8b0 0x555f2023dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:10.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.838+0000 7faffdbe6740 4 client.17703 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:29:10.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.838+0000 7faffdbe6740 1 -- 172.21.15.12:0/1614645036 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555f2012dc10 con 0x555f2023b8b0 2024-04-22T13:29:10.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.838+0000 7faff8d90640 1 --2- 172.21.15.12:0/1614645036 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555f2023b8b0 0x555f2023dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:10.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.838+0000 7faff8d90640 1 --2- 172.21.15.12:0/1614645036 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555f2023b8b0 0x555f2023dcd0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=241b7784fe1a43c8 server_cookie=2b0f8023e10e25e0 in_seq=0 out_seq=0 2024-04-22T13:29:10.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.838+0000 7fafd3fff640 10 client.17703 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:10.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.838+0000 7fafd3fff640 1 -- 172.21.15.12:0/1614645036 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555f2012dc10 con 0x555f2023b8b0 2024-04-22T13:29:10.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.838+0000 7fafd3fff640 10 client.17703 handle_command_reply: tid=0 2024-04-22T13:29:10.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.838+0000 7faffdbe6740 1 client.17703 shutdown 2024-04-22T13:29:10.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.838+0000 7faffdbe6740 1 -- 172.21.15.12:0/1614645036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f201c4a00 msgr2=0x555f201be8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.838+0000 7faffdbe6740 1 --2- 172.21.15.12:0/1614645036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f201c4a00 0x555f201be8c0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fafe4004620 tx=0x7faff0009e30 comp rx=0 tx=0).stop 2024-04-22T13:29:10.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.839+0000 7faffdbe6740 1 -- 172.21.15.12:0/1614645036 shutdown_connections 2024-04-22T13:29:10.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.839+0000 7faffdbe6740 1 --2- 172.21.15.12:0/1614645036 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555f2023b8b0 0x555f2023dcd0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.839+0000 7faffdbe6740 1 --2- 172.21.15.12:0/1614645036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555f201c4a00 0x555f201be8c0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.839+0000 7faffdbe6740 1 --2- 172.21.15.12:0/1614645036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555f201bbe80 0x555f201be180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.839+0000 7fafd3fff640 0 client.17703 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:10.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.839+0000 7faffdbe6740 1 --2- 172.21.15.12:0/1614645036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555f201bb4d0 0x555f201cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.839+0000 7faffdbe6740 1 -- 172.21.15.12:0/1614645036 >> 172.21.15.12:0/1614645036 conn(0x555f201b5ba0 msgr2=0x555f201b4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:10.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.839+0000 7faffdbe6740 1 -- 172.21.15.12:0/1614645036 shutdown_connections 2024-04-22T13:29:10.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.839+0000 7faffdbe6740 1 -- 172.21.15.12:0/1614645036 wait complete. 2024-04-22T13:29:10.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.839+0000 7faffdbe6740 20 client.17703 trim_cache size 0 max 16384 2024-04-22T13:29:10.841 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:29:10.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.839+0000 7faffaff4640 1 -- 172.21.15.12:0/1139689672 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fafdc0a3d20 msgr2=0x7fafdc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.839+0000 7faffaff4640 1 --2- 172.21.15.12:0/1139689672 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fafdc0a3d20 0x7fafdc0a61e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fafec0f0040 tx=0x7fafec02e040 comp rx=0 tx=0).stop 2024-04-22T13:29:10.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.839+0000 7faffaff4640 1 -- 172.21.15.12:0/1139689672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faff419cf60 msgr2=0x7faff419d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:10.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.839+0000 7faffaff4640 1 --2- 172.21.15.12:0/1139689672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faff419cf60 0x7faff419d410 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fafec005e30 tx=0x7fafec0346d0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.839+0000 7faffaff4640 1 -- 172.21.15.12:0/1139689672 shutdown_connections 2024-04-22T13:29:10.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.839+0000 7faffaff4640 1 --2- 172.21.15.12:0/1139689672 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fafdc0a3d20 0x7fafdc0a61e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.839+0000 7faffaff4640 1 --2- 172.21.15.12:0/1139689672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faff419cf60 0x7faff419d410 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.839+0000 7faffaff4640 1 --2- 172.21.15.12:0/1139689672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faff4107270 0x7faff4198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.839+0000 7faffaff4640 1 --2- 172.21.15.12:0/1139689672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faff406a840 0x7faff41986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:10.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.839+0000 7faffaff4640 1 -- 172.21.15.12:0/1139689672 >> 172.21.15.12:0/1139689672 conn(0x7faff4077eb0 msgr2=0x7faff40fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:10.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.839+0000 7faffaff4640 1 -- 172.21.15.12:0/1139689672 shutdown_connections 2024-04-22T13:29:10.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:10.839+0000 7faffaff4640 1 -- 172.21.15.12:0/1139689672 wait complete. 2024-04-22T13:29:10.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:10 smithi134 ceph-mon[31338]: scrub queued for path: / 2024-04-22T13:29:10.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:10 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:29:10.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:10 smithi134 ceph-mon[31338]: scrub summary: active paths [/] 2024-04-22T13:29:10.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:10 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:29:10.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:10 smithi134 ceph-mon[31338]: scrub summary: idle 2024-04-22T13:29:10.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:10 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2054897409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:29:10.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:10 smithi111 ceph-mon[31361]: scrub queued for path: / 2024-04-22T13:29:10.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:10 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:29:10.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:10 smithi111 ceph-mon[31361]: scrub summary: active paths [/] 2024-04-22T13:29:10.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:10 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:29:10.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:10 smithi111 ceph-mon[31361]: scrub summary: idle 2024-04-22T13:29:10.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:10 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2054897409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:29:11.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:10 smithi012 ceph-mon[28265]: scrub queued for path: / 2024-04-22T13:29:11.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:10 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:29:11.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:10 smithi012 ceph-mon[28265]: scrub summary: active paths [/] 2024-04-22T13:29:11.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:10 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:29:11.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:10 smithi012 ceph-mon[28265]: scrub summary: idle 2024-04-22T13:29:11.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:10 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2054897409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:29:11.846 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:29:11.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:29:11.847 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:29:11.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:11 smithi134 ceph-mon[31338]: pgmap v155: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:29:11.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.971+0000 7f4abbdab640 1 -- 172.21.15.12:0/3636918201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ab406b230 msgr2=0x7f4ab4106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:11.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.971+0000 7f4abbdab640 1 --2- 172.21.15.12:0/3636918201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ab406b230 0x7f4ab4106b90 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4009900 tx=0x7f4aa402bc30 comp rx=0 tx=0).stop 2024-04-22T13:29:11.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.971+0000 7f4abbdab640 1 -- 172.21.15.12:0/3636918201 shutdown_connections 2024-04-22T13:29:11.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.971+0000 7f4abbdab640 1 --2- 172.21.15.12:0/3636918201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ab4107270 0x7f4ab410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:11.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.971+0000 7f4abbdab640 1 --2- 172.21.15.12:0/3636918201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ab406b230 0x7f4ab4106b90 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:11.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.971+0000 7f4abbdab640 1 --2- 172.21.15.12:0/3636918201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ab406a880 0x7f4ab406ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:11.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.971+0000 7f4abbdab640 1 -- 172.21.15.12:0/3636918201 >> 172.21.15.12:0/3636918201 conn(0x7f4ab4077eb0 msgr2=0x7f4ab40782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:11.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.971+0000 7f4abbdab640 1 -- 172.21.15.12:0/3636918201 shutdown_connections 2024-04-22T13:29:11.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.971+0000 7f4abbdab640 1 -- 172.21.15.12:0/3636918201 wait complete. 2024-04-22T13:29:11.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.971+0000 7f4abbdab640 1 Processor -- start 2024-04-22T13:29:11.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.972+0000 7f4abbdab640 1 -- start start 2024-04-22T13:29:11.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.972+0000 7f4abbdab640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ab406a880 0x7f4ab4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:11.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.972+0000 7f4abbdab640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ab4107270 0x7f4ab4198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:11.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.972+0000 7f4abbdab640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ab419cff0 0x7f4ab419d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:11.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.972+0000 7f4abbdab640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab41121b0 con 0x7f4ab4107270 2024-04-22T13:29:11.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.972+0000 7f4abbdab640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab4112030 con 0x7f4ab419cff0 2024-04-22T13:29:11.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.972+0000 7f4abbdab640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab4112330 con 0x7f4ab406a880 2024-04-22T13:29:11.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.972+0000 7f4ab9346640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ab4107270 0x7f4ab4198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:11.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.972+0000 7f4ab9346640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ab4107270 0x7f4ab4198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38018/0 (socket says 172.21.15.12:38018) 2024-04-22T13:29:11.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.972+0000 7f4ab9346640 1 -- 172.21.15.12:0/1138066873 learned_addr learned my addr 172.21.15.12:0/1138066873 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:11.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.972+0000 7f4ab9b47640 1 --2- 172.21.15.12:0/1138066873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ab406a880 0x7f4ab4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:11.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.972+0000 7f4aba348640 1 --2- 172.21.15.12:0/1138066873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ab419cff0 0x7f4ab419d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:11.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.972+0000 7f4ab9346640 1 -- 172.21.15.12:0/1138066873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ab406a880 msgr2=0x7f4ab4198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:11.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.972+0000 7f4ab9346640 1 --2- 172.21.15.12:0/1138066873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ab406a880 0x7f4ab4198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:11.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.972+0000 7f4ab9346640 1 -- 172.21.15.12:0/1138066873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ab419cff0 msgr2=0x7f4ab419d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:11.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.972+0000 7f4ab9346640 1 --2- 172.21.15.12:0/1138066873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ab419cff0 0x7f4ab419d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:11.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.972+0000 7f4ab9346640 1 -- 172.21.15.12:0/1138066873 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ab419dae0 con 0x7f4ab4107270 2024-04-22T13:29:11.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.972+0000 7f4ab9346640 1 --2- 172.21.15.12:0/1138066873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ab4107270 0x7f4ab4198c70 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4036040 tx=0x7f4aa40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:11.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.973+0000 7f4aaaffd640 1 -- 172.21.15.12:0/1138066873 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4043020 con 0x7f4ab4107270 2024-04-22T13:29:11.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.973+0000 7f4abbdab640 1 -- 172.21.15.12:0/1138066873 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ab419dd70 con 0x7f4ab4107270 2024-04-22T13:29:11.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.973+0000 7f4aaaffd640 1 -- 172.21.15.12:0/1138066873 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4aa402cb70 con 0x7f4ab4107270 2024-04-22T13:29:11.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.973+0000 7f4aaaffd640 1 -- 172.21.15.12:0/1138066873 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4003bf0 con 0x7f4ab4107270 2024-04-22T13:29:11.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.973+0000 7f4abbdab640 1 -- 172.21.15.12:0/1138066873 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ab41a5570 con 0x7f4ab4107270 2024-04-22T13:29:11.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:11 smithi111 ceph-mon[31361]: pgmap v155: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:29:11.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.975+0000 7f4aaaffd640 1 -- 172.21.15.12:0/1138066873 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4aa4046020 con 0x7f4ab4107270 2024-04-22T13:29:11.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.975+0000 7f4abe99d740 1 Processor -- start 2024-04-22T13:29:11.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.976+0000 7f4abe99d740 1 -- start start 2024-04-22T13:29:11.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.976+0000 7f4abe99d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d330448e0 0x555d33044cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:11.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.976+0000 7f4abe99d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d33045290 0x555d3304d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:11.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.976+0000 7f4abe99d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d3304de10 0x555d33050260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:11.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.976+0000 7f4abe99d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555d32fa8f00 con 0x555d3304de10 2024-04-22T13:29:11.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.976+0000 7f4abe99d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555d32f46590 con 0x555d33045290 2024-04-22T13:29:11.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.976+0000 7f4abe99d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555d32fa4ec0 con 0x555d330448e0 2024-04-22T13:29:11.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.976+0000 7f4aaaffd640 1 --2- 172.21.15.12:0/1138066873 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4aa00a3d20 0x7f4aa00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:11.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.976+0000 7f4aba348640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d330448e0 0x555d33044cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:11.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.976+0000 7f4aba348640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d330448e0 0x555d33044cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.12:47238/0 (socket says 172.21.15.12:47238) 2024-04-22T13:29:11.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.976+0000 7f4aba348640 1 -- 172.21.15.12:0/2403220012 learned_addr learned my addr 172.21.15.12:0/2403220012 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:11.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.976+0000 7f4ab9346640 1 --2- 172.21.15.12:0/2403220012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d3304de10 0x555d33050260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:11.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.976+0000 7f4ab9b47640 1 --2- 172.21.15.12:0/2403220012 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d33045290 0x555d3304d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:11.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.977+0000 7f4aba348640 1 --2- 172.21.15.12:0/1138066873 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4aa00a3d20 0x7f4aa00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:11.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.977+0000 7f4aba348640 1 -- 172.21.15.12:0/2403220012 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d33045290 msgr2=0x555d3304d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:11.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.977+0000 7f4aba348640 1 --2- 172.21.15.12:0/2403220012 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d33045290 0x555d3304d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:11.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.977+0000 7f4aaaffd640 1 -- 172.21.15.12:0/1138066873 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f4aa40f28a0 con 0x7f4ab4107270 2024-04-22T13:29:11.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.977+0000 7f4aba348640 1 -- 172.21.15.12:0/2403220012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d3304de10 msgr2=0x555d33050260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:11.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.977+0000 7f4aba348640 1 --2- 172.21.15.12:0/2403220012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d3304de10 0x555d33050260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:11.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.977+0000 7f4aba348640 1 -- 172.21.15.12:0/2403220012 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d32d837e0 con 0x555d330448e0 2024-04-22T13:29:11.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.977+0000 7f4aba348640 1 --2- 172.21.15.12:0/2403220012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d330448e0 0x555d33044cc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4aac0129f0 tx=0x7f4aac012eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cee232230d0dcbd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:11.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.977+0000 7f4aba348640 1 --2- 172.21.15.12:0/1138066873 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4aa00a3d20 0x7f4aa00a61e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4aac01c000 tx=0x7f4aac020000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:11.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.977+0000 7f4a9ffff640 1 -- 172.21.15.12:0/2403220012 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac0120d0 con 0x555d330448e0 2024-04-22T13:29:11.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.977+0000 7f4a9ffff640 1 -- 172.21.15.12:0/2403220012 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4aac023020 con 0x555d330448e0 2024-04-22T13:29:11.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.977+0000 7f4a9ffff640 1 -- 172.21.15.12:0/2403220012 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac004030 con 0x555d330448e0 2024-04-22T13:29:11.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.978+0000 7f4abe99d740 1 -- 172.21.15.12:0/2403220012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d330448e0 msgr2=0x555d33044cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:11.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.978+0000 7f4abe99d740 1 --2- 172.21.15.12:0/2403220012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d330448e0 0x555d33044cc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4aac0129f0 tx=0x7f4aac012eb0 comp rx=0 tx=0).stop 2024-04-22T13:29:11.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.978+0000 7f4abe99d740 1 -- 172.21.15.12:0/2403220012 shutdown_connections 2024-04-22T13:29:11.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.978+0000 7f4abe99d740 1 --2- 172.21.15.12:0/2403220012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d3304de10 0x555d33050260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:11.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.978+0000 7f4abe99d740 1 --2- 172.21.15.12:0/2403220012 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d33045290 0x555d3304d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:11.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.978+0000 7f4abe99d740 1 --2- 172.21.15.12:0/2403220012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d330448e0 0x555d33044cc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:11.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.978+0000 7f4abe99d740 1 -- 172.21.15.12:0/2403220012 >> 172.21.15.12:0/2403220012 conn(0x555d3303efb0 msgr2=0x555d3303f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:11.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.978+0000 7f4abe99d740 1 -- 172.21.15.12:0/2403220012 shutdown_connections 2024-04-22T13:29:11.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.978+0000 7f4abe99d740 1 -- 172.21.15.12:0/2403220012 wait complete. 2024-04-22T13:29:11.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.978+0000 7f4abe99d740 1 -- start start 2024-04-22T13:29:11.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.978+0000 7f4abe99d740 1 Processor -- start 2024-04-22T13:29:11.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.979+0000 7f4abe99d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d330448e0 0x555d3304ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:11.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.979+0000 7f4abe99d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d33045290 0x555d3304b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:11.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.979+0000 7f4abe99d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d3304de10 0x555d3304b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:11.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.979+0000 7f4abe99d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555d33050cc0 con 0x555d3304de10 2024-04-22T13:29:11.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.979+0000 7f4abe99d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555d32f62580 con 0x555d330448e0 2024-04-22T13:29:11.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.979+0000 7f4abe99d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555d33050e40 con 0x555d33045290 2024-04-22T13:29:11.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.979+0000 7f4abe99d740 10 client.0 _ll_register_callbacks cb 0x555d32f68650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:11.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.979+0000 7f4aba348640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d3304de10 0x555d3304b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:11.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.979+0000 7f4aba348640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d3304de10 0x555d3304b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38038/0 (socket says 172.21.15.12:38038) 2024-04-22T13:29:11.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.979+0000 7f4aba348640 1 -- 172.21.15.12:0/822757830 learned_addr learned my addr 172.21.15.12:0/822757830 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:11.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.979+0000 7f4ab9b47640 1 --2- 172.21.15.12:0/822757830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d330448e0 0x555d3304ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:11.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.979+0000 7f4ab9346640 1 --2- 172.21.15.12:0/822757830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d33045290 0x555d3304b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:11.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.979+0000 7f4aba348640 1 -- 172.21.15.12:0/822757830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d33045290 msgr2=0x555d3304b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:11.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.979+0000 7f4aba348640 1 --2- 172.21.15.12:0/822757830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d33045290 0x555d3304b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:11.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.979+0000 7f4aba348640 1 -- 172.21.15.12:0/822757830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d330448e0 msgr2=0x555d3304ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:11.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.979+0000 7f4aba348640 1 --2- 172.21.15.12:0/822757830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d330448e0 0x555d3304ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:11.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.979+0000 7f4aba348640 1 -- 172.21.15.12:0/822757830 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d32de3f70 con 0x555d3304de10 2024-04-22T13:29:11.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.980+0000 7f4aba348640 1 --2- 172.21.15.12:0/822757830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d3304de10 0x555d3304b8c0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f4aac0123c0 tx=0x7f4aac009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a9ce688fb8f5ef server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:11.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.980+0000 7f4a9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:11.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.980+0000 7f4a9d7fa640 1 -- 172.21.15.12:0/822757830 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac01f980 con 0x555d3304de10 2024-04-22T13:29:11.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.980+0000 7f4a9d7fa640 1 -- 172.21.15.12:0/822757830 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f4aac01fb20 con 0x555d3304de10 2024-04-22T13:29:11.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.980+0000 7f4abe99d740 1 -- 172.21.15.12:0/822757830 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4aac023020 con 0x555d3304de10 2024-04-22T13:29:11.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.980+0000 7f4a9d7fa640 1 -- 172.21.15.12:0/822757830 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4aac035020 con 0x555d3304de10 2024-04-22T13:29:11.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.980+0000 7f4abe99d740 10 client.17715 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:11.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.980+0000 7f4abe99d740 1 -- 172.21.15.12:0/822757830 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555d32c7b170 con 0x555d3304de10 2024-04-22T13:29:11.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.980+0000 7f4a9d7fa640 1 -- 172.21.15.12:0/822757830 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4aac023020 con 0x555d3304de10 2024-04-22T13:29:11.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.980+0000 7f4a9d7fa640 1 -- 172.21.15.12:0/822757830 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4aac01e070 con 0x555d3304de10 2024-04-22T13:29:11.984 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.980+0000 7f4abe99d740 10 client.17715 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:11.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.980+0000 7f4abe99d740 10 client.17715 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:11.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.980+0000 7f4abe99d740 20 client.17715 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:11.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.980+0000 7f4abe99d740 1 --2- 172.21.15.12:0/822757830 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555d3305e2b0 0x555d330606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:11.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.980+0000 7f4abe99d740 4 client.17715 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:11.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.980+0000 7f4abe99d740 1 -- 172.21.15.12:0/822757830 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d32fb6920 con 0x555d3305e2b0 2024-04-22T13:29:11.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.981+0000 7f4ab9b47640 1 --2- 172.21.15.12:0/822757830 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555d3305e2b0 0x555d330606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:11.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.981+0000 7f4ab9b47640 1 --2- 172.21.15.12:0/822757830 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555d3305e2b0 0x555d330606a0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25af9593be4c6117 server_cookie=538e5c7a007c92d in_seq=0 out_seq=0 2024-04-22T13:29:11.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.981+0000 7f4a9d7fa640 10 client.17715 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:11.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.982+0000 7f4a9d7fa640 1 -- 172.21.15.12:0/822757830 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555d32fb6920 con 0x555d3305e2b0 2024-04-22T13:29:11.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.982+0000 7f4a9d7fa640 10 client.17715 handle_command_reply: tid=0 2024-04-22T13:29:11.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.982+0000 7f4abe99d740 1 client.17715 shutdown 2024-04-22T13:29:11.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.982+0000 7f4abe99d740 1 -- 172.21.15.12:0/822757830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d3304de10 msgr2=0x555d3304b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:11.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.982+0000 7f4abe99d740 1 --2- 172.21.15.12:0/822757830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d3304de10 0x555d3304b8c0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f4aac0123c0 tx=0x7f4aac009480 comp rx=0 tx=0).stop 2024-04-22T13:29:11.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.982+0000 7f4abe99d740 1 -- 172.21.15.12:0/822757830 shutdown_connections 2024-04-22T13:29:11.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.982+0000 7f4abe99d740 1 --2- 172.21.15.12:0/822757830 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555d3305e2b0 0x555d330606a0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:11.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.982+0000 7f4abe99d740 1 --2- 172.21.15.12:0/822757830 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d3304de10 0x555d3304b8c0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:11.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.982+0000 7f4abe99d740 1 --2- 172.21.15.12:0/822757830 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d33045290 0x555d3304b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:11.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.982+0000 7f4abe99d740 1 --2- 172.21.15.12:0/822757830 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d330448e0 0x555d3304ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:11.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.982+0000 7f4a9d7fa640 0 client.17715 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:11.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.982+0000 7f4abe99d740 1 -- 172.21.15.12:0/822757830 >> 172.21.15.12:0/822757830 conn(0x555d3303efb0 msgr2=0x555d33040d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:11.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.982+0000 7f4abe99d740 1 -- 172.21.15.12:0/822757830 shutdown_connections 2024-04-22T13:29:11.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.982+0000 7f4abe99d740 1 -- 172.21.15.12:0/822757830 wait complete. 2024-04-22T13:29:11.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:11.982+0000 7f4abe99d740 20 client.17715 trim_cache size 0 max 16384 2024-04-22T13:29:12.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.000+0000 7f4abe99d740 1 Processor -- start 2024-04-22T13:29:12.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.000+0000 7f4abe99d740 1 -- start start 2024-04-22T13:29:12.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.000+0000 7f4abe99d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d330448e0 0x555d3304d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:12.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.000+0000 7f4abe99d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d33045290 0x555d33059bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:12.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.000+0000 7f4abe99d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d3304de10 0x555d33040a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:12.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.000+0000 7f4abe99d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab41ad9b0 con 0x555d3304de10 2024-04-22T13:29:12.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.000+0000 7f4abe99d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab41af4f0 con 0x555d33045290 2024-04-22T13:29:12.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.000+0000 7f4abe99d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab41ae080 con 0x555d330448e0 2024-04-22T13:29:12.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.000+0000 7f4aba348640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d3304de10 0x555d33040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:12.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.000+0000 7f4aba348640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d3304de10 0x555d33040a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38052/0 (socket says 172.21.15.12:38052) 2024-04-22T13:29:12.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.000+0000 7f4aba348640 1 -- 172.21.15.12:0/1914603109 learned_addr learned my addr 172.21.15.12:0/1914603109 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:12.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.000+0000 7f4ab9b47640 1 --2- 172.21.15.12:0/1914603109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d330448e0 0x555d3304d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:12.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.001+0000 7f4ab9346640 1 --2- 172.21.15.12:0/1914603109 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d33045290 0x555d33059bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:12.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.001+0000 7f4aba348640 1 -- 172.21.15.12:0/1914603109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d330448e0 msgr2=0x555d3304d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:12.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.001+0000 7f4aba348640 1 --2- 172.21.15.12:0/1914603109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d330448e0 0x555d3304d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.001+0000 7f4aba348640 1 -- 172.21.15.12:0/1914603109 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d33045290 msgr2=0x555d33059bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:12.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.001+0000 7f4aba348640 1 --2- 172.21.15.12:0/1914603109 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d33045290 0x555d33059bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.001+0000 7f4aba348640 1 -- 172.21.15.12:0/1914603109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d33049390 con 0x555d3304de10 2024-04-22T13:29:12.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.001+0000 7f4ab9346640 1 --2- 172.21.15.12:0/1914603109 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d33045290 0x555d33059bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:12.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.001+0000 7f4aba348640 1 --2- 172.21.15.12:0/1914603109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d3304de10 0x555d33040a90 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f4aac01ffd0 tx=0x7f4aac0126a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb175c210922463 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:12.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.001+0000 7f4a9f7fe640 1 -- 172.21.15.12:0/1914603109 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac0337d0 con 0x555d3304de10 2024-04-22T13:29:12.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.001+0000 7f4a9f7fe640 1 -- 172.21.15.12:0/1914603109 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4aac02c030 con 0x555d3304de10 2024-04-22T13:29:12.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.001+0000 7f4a9f7fe640 1 -- 172.21.15.12:0/1914603109 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac033ab0 con 0x555d3304de10 2024-04-22T13:29:12.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.002+0000 7f4abe99d740 1 -- 172.21.15.12:0/1914603109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d3304de10 msgr2=0x555d33040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:12.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.002+0000 7f4abe99d740 1 --2- 172.21.15.12:0/1914603109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d3304de10 0x555d33040a90 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f4aac01ffd0 tx=0x7f4aac0126a0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.002+0000 7f4abe99d740 1 -- 172.21.15.12:0/1914603109 shutdown_connections 2024-04-22T13:29:12.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.002+0000 7f4abe99d740 1 --2- 172.21.15.12:0/1914603109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d3304de10 0x555d33040a90 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.002+0000 7f4abe99d740 1 --2- 172.21.15.12:0/1914603109 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d33045290 0x555d33059bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.002+0000 7f4abe99d740 1 --2- 172.21.15.12:0/1914603109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d330448e0 0x555d3304d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.002+0000 7f4abe99d740 1 -- 172.21.15.12:0/1914603109 >> 172.21.15.12:0/1914603109 conn(0x555d3303efb0 msgr2=0x555d33040380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:12.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.002+0000 7f4abe99d740 1 -- 172.21.15.12:0/1914603109 shutdown_connections 2024-04-22T13:29:12.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.002+0000 7f4abe99d740 1 -- 172.21.15.12:0/1914603109 wait complete. 2024-04-22T13:29:12.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.002+0000 7f4abe99d740 1 -- start start 2024-04-22T13:29:12.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.002+0000 7f4abe99d740 1 Processor -- start 2024-04-22T13:29:12.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.003+0000 7f4abe99d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d330448e0 0x555d33051ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:12.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.003+0000 7f4abe99d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d33045290 0x555d33051ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:12.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.003+0000 7f4abe99d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d3304de10 0x555d33052530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:12.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.003+0000 7f4abe99d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab41aee20 con 0x555d330448e0 2024-04-22T13:29:12.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.003+0000 7f4abe99d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab41ae750 con 0x555d3304de10 2024-04-22T13:29:12.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.003+0000 7f4abe99d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab41ab9b0 con 0x555d33045290 2024-04-22T13:29:12.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.003+0000 7f4abe99d740 10 client.0 _ll_register_callbacks cb 0x7f4aa40ba4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:12.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.003+0000 7f4ab9b47640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d330448e0 0x555d33051ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:12.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.003+0000 7f4ab9b47640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d330448e0 0x555d33051ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38062/0 (socket says 172.21.15.12:38062) 2024-04-22T13:29:12.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.003+0000 7f4ab9b47640 1 -- 172.21.15.12:0/1540771175 learned_addr learned my addr 172.21.15.12:0/1540771175 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:12.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.003+0000 7f4aba348640 1 --2- 172.21.15.12:0/1540771175 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d3304de10 0x555d33052530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:12.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.003+0000 7f4ab9346640 1 --2- 172.21.15.12:0/1540771175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d33045290 0x555d33051ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:12.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.003+0000 7f4ab9b47640 1 -- 172.21.15.12:0/1540771175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d33045290 msgr2=0x555d33051ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:12.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.003+0000 7f4ab9b47640 1 --2- 172.21.15.12:0/1540771175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d33045290 0x555d33051ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.003+0000 7f4ab9b47640 1 -- 172.21.15.12:0/1540771175 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d3304de10 msgr2=0x555d33052530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:12.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.003+0000 7f4ab9b47640 1 --2- 172.21.15.12:0/1540771175 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d3304de10 0x555d33052530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.003+0000 7f4ab9b47640 1 -- 172.21.15.12:0/1540771175 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d330c3700 con 0x555d330448e0 2024-04-22T13:29:12.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.003+0000 7f4ab9b47640 1 --2- 172.21.15.12:0/1540771175 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d330448e0 0x555d33051ab0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x555d330443b0 tx=0x7f4ab000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f97b18dc1dc181 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:12.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.003+0000 7f4a9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:12.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.003+0000 7f4a9d7fa640 1 -- 172.21.15.12:0/1540771175 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0005020 con 0x555d330448e0 2024-04-22T13:29:12.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.003+0000 7f4a9d7fa640 1 -- 172.21.15.12:0/1540771175 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f4ab00051c0 con 0x555d330448e0 2024-04-22T13:29:12.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.003+0000 7f4abe99d740 1 -- 172.21.15.12:0/1540771175 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4aac02c030 con 0x555d330448e0 2024-04-22T13:29:12.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.004+0000 7f4a9d7fa640 1 -- 172.21.15.12:0/1540771175 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4ab002f050 con 0x555d330448e0 2024-04-22T13:29:12.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.004+0000 7f4abe99d740 10 client.17721 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:12.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.004+0000 7f4abe99d740 1 -- 172.21.15.12:0/1540771175 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555d330c4a70 con 0x555d330448e0 2024-04-22T13:29:12.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.004+0000 7f4a9d7fa640 1 -- 172.21.15.12:0/1540771175 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4aac02c030 con 0x555d330448e0 2024-04-22T13:29:12.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.004+0000 7f4a9d7fa640 1 -- 172.21.15.12:0/1540771175 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4ab00200e0 con 0x555d330448e0 2024-04-22T13:29:12.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.004+0000 7f4abe99d740 10 client.17721 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:12.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.004+0000 7f4abe99d740 10 client.17721 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:12.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.004+0000 7f4abe99d740 20 client.17721 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:12.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.004+0000 7f4abe99d740 1 --2- 172.21.15.12:0/1540771175 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555d330c5230 0x555d330c7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:12.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.004+0000 7f4abe99d740 4 client.17721 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:29:12.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.004+0000 7f4abe99d740 1 -- 172.21.15.12:0/1540771175 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555d32fb0790 con 0x555d330c5230 2024-04-22T13:29:12.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.004+0000 7f4ab9346640 1 --2- 172.21.15.12:0/1540771175 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555d330c5230 0x555d330c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:12.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.005+0000 7f4a9d7fa640 10 client.17721 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:12.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.005+0000 7f4ab9346640 1 --2- 172.21.15.12:0/1540771175 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555d330c5230 0x555d330c7620 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a795985ffe5389b server_cookie=ad87513e67b336c1 in_seq=0 out_seq=0 2024-04-22T13:29:12.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.006+0000 7f4a9d7fa640 1 -- 172.21.15.12:0/1540771175 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555d32fb0790 con 0x555d330c5230 2024-04-22T13:29:12.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.006+0000 7f4a9d7fa640 10 client.17721 handle_command_reply: tid=0 2024-04-22T13:29:12.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.006+0000 7f4abe99d740 1 client.17721 shutdown 2024-04-22T13:29:12.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.006+0000 7f4abe99d740 1 -- 172.21.15.12:0/1540771175 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d330448e0 msgr2=0x555d33051ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:12.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.006+0000 7f4abe99d740 1 --2- 172.21.15.12:0/1540771175 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d330448e0 0x555d33051ab0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x555d330443b0 tx=0x7f4ab000be60 comp rx=0 tx=0).stop 2024-04-22T13:29:12.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.006+0000 7f4abe99d740 1 -- 172.21.15.12:0/1540771175 shutdown_connections 2024-04-22T13:29:12.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.006+0000 7f4abe99d740 1 --2- 172.21.15.12:0/1540771175 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555d330c5230 0x555d330c7620 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.006+0000 7f4abe99d740 1 --2- 172.21.15.12:0/1540771175 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d3304de10 0x555d33052530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.006+0000 7f4abe99d740 1 --2- 172.21.15.12:0/1540771175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555d33045290 0x555d33051ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.006+0000 7f4abe99d740 1 --2- 172.21.15.12:0/1540771175 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d330448e0 0x555d33051ab0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.006+0000 7f4abe99d740 1 -- 172.21.15.12:0/1540771175 >> 172.21.15.12:0/1540771175 conn(0x555d3303efb0 msgr2=0x555d33040be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:12.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.006+0000 7f4a9d7fa640 0 client.17721 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:12.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.006+0000 7f4abe99d740 1 -- 172.21.15.12:0/1540771175 shutdown_connections 2024-04-22T13:29:12.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.006+0000 7f4abe99d740 1 -- 172.21.15.12:0/1540771175 wait complete. 2024-04-22T13:29:12.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.006+0000 7f4abe99d740 20 client.17721 trim_cache size 0 max 16384 2024-04-22T13:29:12.010 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:29:12.010 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-04-22T13:29:12.010 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "cce66f85-2351-42d4-ab0f-e77412324805", 2024-04-22T13:29:12.010 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-04-22T13:29:12.010 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:29:12.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.007+0000 7f4abbdab640 1 -- 172.21.15.12:0/1138066873 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4aa00a3d20 msgr2=0x7f4aa00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:12.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.007+0000 7f4abbdab640 1 --2- 172.21.15.12:0/1138066873 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4aa00a3d20 0x7f4aa00a61e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4aac01c000 tx=0x7f4aac020000 comp rx=0 tx=0).stop 2024-04-22T13:29:12.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.007+0000 7f4abbdab640 1 -- 172.21.15.12:0/1138066873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ab4107270 msgr2=0x7f4ab4198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:12.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.007+0000 7f4abbdab640 1 --2- 172.21.15.12:0/1138066873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ab4107270 0x7f4ab4198c70 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4036040 tx=0x7f4aa40045e0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.007+0000 7f4abbdab640 1 -- 172.21.15.12:0/1138066873 shutdown_connections 2024-04-22T13:29:12.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.007+0000 7f4abbdab640 1 --2- 172.21.15.12:0/1138066873 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4aa00a3d20 0x7f4aa00a61e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.007+0000 7f4abbdab640 1 --2- 172.21.15.12:0/1138066873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ab419cff0 0x7f4ab419d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.007+0000 7f4abbdab640 1 --2- 172.21.15.12:0/1138066873 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4ab4107270 0x7f4ab4198c70 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.007+0000 7f4abbdab640 1 --2- 172.21.15.12:0/1138066873 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ab406a880 0x7f4ab4198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.007+0000 7f4abbdab640 1 -- 172.21.15.12:0/1138066873 >> 172.21.15.12:0/1138066873 conn(0x7f4ab4077eb0 msgr2=0x7f4ab4108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:12.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.007+0000 7f4abbdab640 1 -- 172.21.15.12:0/1138066873 shutdown_connections 2024-04-22T13:29:12.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.007+0000 7f4abbdab640 1 -- 172.21.15.12:0/1138066873 wait complete. 2024-04-22T13:29:12.013 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:11 smithi012 ceph-mon[28265]: pgmap v155: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:29:12.014 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:29:12.014 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:29:12.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.123+0000 7fa96ee4e640 1 -- 172.21.15.12:0/1649009559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa968101a30 msgr2=0x7fa968105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:12.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.123+0000 7fa96ee4e640 1 --2- 172.21.15.12:0/1649009559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa968101a30 0x7fa968105a70 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fa95c009900 tx=0x7fa95c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:29:12.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.123+0000 7fa96ee4e640 1 -- 172.21.15.12:0/1649009559 shutdown_connections 2024-04-22T13:29:12.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.123+0000 7fa96ee4e640 1 --2- 172.21.15.12:0/1649009559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa968106150 0x7fa96810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.123+0000 7fa96ee4e640 1 --2- 172.21.15.12:0/1649009559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa968101a30 0x7fa968105a70 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.123+0000 7fa96ee4e640 1 --2- 172.21.15.12:0/1649009559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa968101080 0x7fa968101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.123+0000 7fa96ee4e640 1 -- 172.21.15.12:0/1649009559 >> 172.21.15.12:0/1649009559 conn(0x7fa968078000 msgr2=0x7fa968078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:12.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.123+0000 7fa96ee4e640 1 -- 172.21.15.12:0/1649009559 shutdown_connections 2024-04-22T13:29:12.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.123+0000 7fa96ee4e640 1 -- 172.21.15.12:0/1649009559 wait complete. 2024-04-22T13:29:12.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.124+0000 7fa96ee4e640 1 Processor -- start 2024-04-22T13:29:12.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.124+0000 7fa96ee4e640 1 -- start start 2024-04-22T13:29:12.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.124+0000 7fa96ee4e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa968101080 0x7fa9681986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:12.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.124+0000 7fa96ee4e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa968106150 0x7fa968198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:12.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.124+0000 7fa96ee4e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa96819cfb0 0x7fa96819d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:12.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.124+0000 7fa96ee4e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681121d0 con 0x7fa968106150 2024-04-22T13:29:12.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.124+0000 7fa96ee4e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa968112050 con 0x7fa96819cfb0 2024-04-22T13:29:12.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.124+0000 7fa96ee4e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa968112350 con 0x7fa968101080 2024-04-22T13:29:12.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.124+0000 7fa967fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa968106150 0x7fa968198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:12.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.124+0000 7fa967fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa968106150 0x7fa968198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38070/0 (socket says 172.21.15.12:38070) 2024-04-22T13:29:12.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.124+0000 7fa967fff640 1 -- 172.21.15.12:0/3333775399 learned_addr learned my addr 172.21.15.12:0/3333775399 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:12.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.124+0000 7fa96cbea640 1 --2- 172.21.15.12:0/3333775399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa968101080 0x7fa9681986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:12.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.125+0000 7fa96d3eb640 1 --2- 172.21.15.12:0/3333775399 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa96819cfb0 0x7fa96819d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:12.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.125+0000 7fa967fff640 1 -- 172.21.15.12:0/3333775399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa968101080 msgr2=0x7fa9681986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:12.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.125+0000 7fa967fff640 1 --2- 172.21.15.12:0/3333775399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa968101080 0x7fa9681986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.125+0000 7fa967fff640 1 -- 172.21.15.12:0/3333775399 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa96819cfb0 msgr2=0x7fa96819d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:12.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.125+0000 7fa967fff640 1 --2- 172.21.15.12:0/3333775399 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa96819cfb0 0x7fa96819d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.125+0000 7fa967fff640 1 -- 172.21.15.12:0/3333775399 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96819daa0 con 0x7fa968106150 2024-04-22T13:29:12.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.125+0000 7fa967fff640 1 --2- 172.21.15.12:0/3333775399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa968106150 0x7fa968198c30 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fa95c036040 tx=0x7fa95c02c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:12.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.125+0000 7fa965ffb640 1 -- 172.21.15.12:0/3333775399 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c043020 con 0x7fa968106150 2024-04-22T13:29:12.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.125+0000 7fa965ffb640 1 -- 172.21.15.12:0/3333775399 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa95c02cbe0 con 0x7fa968106150 2024-04-22T13:29:12.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.125+0000 7fa96ee4e640 1 -- 172.21.15.12:0/3333775399 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa96819dd30 con 0x7fa968106150 2024-04-22T13:29:12.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.125+0000 7fa965ffb640 1 -- 172.21.15.12:0/3333775399 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c003bf0 con 0x7fa968106150 2024-04-22T13:29:12.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.125+0000 7fa96ee4e640 1 -- 172.21.15.12:0/3333775399 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9681a5550 con 0x7fa968106150 2024-04-22T13:29:12.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.128+0000 7fa965ffb640 1 -- 172.21.15.12:0/3333775399 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa95c046020 con 0x7fa968106150 2024-04-22T13:29:12.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.128+0000 7fa971a44740 1 Processor -- start 2024-04-22T13:29:12.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.128+0000 7fa971a44740 1 -- start start 2024-04-22T13:29:12.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.129+0000 7fa971a44740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630e43e82b0 0x5630e43e8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:12.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.129+0000 7fa971a44740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630e43e8bd0 0x5630e43f5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:12.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.129+0000 7fa971a44740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630e43f5e90 0x5630e43f8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:12.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.129+0000 7fa971a44740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5630e434d1e0 con 0x5630e43e82b0 2024-04-22T13:29:12.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.129+0000 7fa971a44740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5630e42ea390 con 0x5630e43e8bd0 2024-04-22T13:29:12.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.129+0000 7fa971a44740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5630e43491a0 con 0x5630e43f5e90 2024-04-22T13:29:12.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.129+0000 7fa96d3eb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630e43e82b0 0x5630e43e8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:12.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.129+0000 7fa96d3eb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630e43e82b0 0x5630e43e8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38074/0 (socket says 172.21.15.12:38074) 2024-04-22T13:29:12.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.129+0000 7fa96d3eb640 1 -- 172.21.15.12:0/1947924377 learned_addr learned my addr 172.21.15.12:0/1947924377 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:12.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.129+0000 7fa96cbea640 1 --2- 172.21.15.12:0/1947924377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630e43e8bd0 0x5630e43f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:12.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.129+0000 7fa967fff640 1 --2- 172.21.15.12:0/1947924377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630e43f5e90 0x5630e43f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:12.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.129+0000 7fa965ffb640 1 --2- 172.21.15.12:0/3333775399 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa9500a3d20 0x7fa9500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:12.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.129+0000 7fa96d3eb640 1 -- 172.21.15.12:0/1947924377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630e43f5e90 msgr2=0x5630e43f8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:12.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.129+0000 7fa96d3eb640 1 --2- 172.21.15.12:0/1947924377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630e43f5e90 0x5630e43f8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.129+0000 7fa96d3eb640 1 -- 172.21.15.12:0/1947924377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630e43e8bd0 msgr2=0x5630e43f5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:12.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.129+0000 7fa96d3eb640 1 --2- 172.21.15.12:0/1947924377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630e43e8bd0 0x5630e43f5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.129+0000 7fa96d3eb640 1 -- 172.21.15.12:0/1947924377 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630e41297d0 con 0x5630e43e82b0 2024-04-22T13:29:12.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.129+0000 7fa967fff640 1 --2- 172.21.15.12:0/1947924377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630e43f5e90 0x5630e43f8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:12.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.129+0000 7fa96cbea640 1 --2- 172.21.15.12:0/1947924377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630e43e8bd0 0x5630e43f5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:12.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.129+0000 7fa965ffb640 1 -- 172.21.15.12:0/3333775399 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fa95c0f2890 con 0x7fa968106150 2024-04-22T13:29:12.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.129+0000 7fa96d3eb640 1 --2- 172.21.15.12:0/1947924377 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630e43e82b0 0x5630e43e8690 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fa960009f90 tx=0x7fa960010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09f98e1196005fd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:12.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.129+0000 7fa96d3eb640 1 --2- 172.21.15.12:0/3333775399 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa9500a3d20 0x7fa9500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:12.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.129+0000 7fa94effd640 1 -- 172.21.15.12:0/1947924377 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96001f040 con 0x5630e43e82b0 2024-04-22T13:29:12.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.129+0000 7fa94effd640 1 -- 172.21.15.12:0/1947924377 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa96001a4b0 con 0x5630e43e82b0 2024-04-22T13:29:12.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.130+0000 7fa94effd640 1 -- 172.21.15.12:0/1947924377 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9600196f0 con 0x5630e43e82b0 2024-04-22T13:29:12.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.130+0000 7fa971a44740 1 -- 172.21.15.12:0/1947924377 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630e43e82b0 msgr2=0x5630e43e8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:12.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.130+0000 7fa971a44740 1 --2- 172.21.15.12:0/1947924377 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630e43e82b0 0x5630e43e8690 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fa960009f90 tx=0x7fa960010860 comp rx=0 tx=0).stop 2024-04-22T13:29:12.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.130+0000 7fa96d3eb640 1 --2- 172.21.15.12:0/3333775399 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa9500a3d20 0x7fa9500a61e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa960011010 tx=0x7fa960019e10 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:12.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.130+0000 7fa971a44740 1 -- 172.21.15.12:0/1947924377 shutdown_connections 2024-04-22T13:29:12.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.130+0000 7fa971a44740 1 --2- 172.21.15.12:0/1947924377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630e43f5e90 0x5630e43f8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.130+0000 7fa971a44740 1 --2- 172.21.15.12:0/1947924377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630e43e8bd0 0x5630e43f5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.130+0000 7fa971a44740 1 --2- 172.21.15.12:0/1947924377 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630e43e82b0 0x5630e43e8690 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.130+0000 7fa971a44740 1 -- 172.21.15.12:0/1947924377 >> 172.21.15.12:0/1947924377 conn(0x5630e43e2cb0 msgr2=0x5630e43e3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:12.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.130+0000 7fa971a44740 1 -- 172.21.15.12:0/1947924377 shutdown_connections 2024-04-22T13:29:12.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.130+0000 7fa971a44740 1 -- 172.21.15.12:0/1947924377 wait complete. 2024-04-22T13:29:12.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.131+0000 7fa971a44740 1 -- start start 2024-04-22T13:29:12.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.131+0000 7fa971a44740 1 Processor -- start 2024-04-22T13:29:12.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.131+0000 7fa971a44740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630e43e82b0 0x5630e43f2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:12.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.131+0000 7fa971a44740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630e43e8bd0 0x5630e43f33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:12.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.131+0000 7fa971a44740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630e43f5e90 0x5630e43f38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:12.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.131+0000 7fa971a44740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5630e43f8c50 con 0x5630e43e82b0 2024-04-22T13:29:12.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.131+0000 7fa971a44740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5630e43062f0 con 0x5630e43e8bd0 2024-04-22T13:29:12.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.131+0000 7fa971a44740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5630e43f8dd0 con 0x5630e43f5e90 2024-04-22T13:29:12.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.131+0000 7fa971a44740 10 client.0 _ll_register_callbacks cb 0x5630e4313580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:12.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.131+0000 7fa96cbea640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630e43e82b0 0x5630e43f2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:12.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.131+0000 7fa96cbea640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630e43e82b0 0x5630e43f2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38080/0 (socket says 172.21.15.12:38080) 2024-04-22T13:29:12.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.131+0000 7fa96cbea640 1 -- 172.21.15.12:0/4163092794 learned_addr learned my addr 172.21.15.12:0/4163092794 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:12.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.131+0000 7fa967fff640 1 --2- 172.21.15.12:0/4163092794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630e43e8bd0 0x5630e43f33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:12.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.131+0000 7fa96d3eb640 1 --2- 172.21.15.12:0/4163092794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630e43f5e90 0x5630e43f38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:12.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.131+0000 7fa96cbea640 1 -- 172.21.15.12:0/4163092794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630e43f5e90 msgr2=0x5630e43f38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:12.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.131+0000 7fa96cbea640 1 --2- 172.21.15.12:0/4163092794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630e43f5e90 0x5630e43f38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.131+0000 7fa96cbea640 1 -- 172.21.15.12:0/4163092794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630e43e8bd0 msgr2=0x5630e43f33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:12.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.131+0000 7fa96cbea640 1 --2- 172.21.15.12:0/4163092794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630e43e8bd0 0x5630e43f33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.131+0000 7fa96cbea640 1 -- 172.21.15.12:0/4163092794 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630e405f3f0 con 0x5630e43e82b0 2024-04-22T13:29:12.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.132+0000 7fa96d3eb640 1 --2- 172.21.15.12:0/4163092794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630e43f5e90 0x5630e43f38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:12.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.132+0000 7fa96cbea640 1 --2- 172.21.15.12:0/4163092794 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630e43e82b0 0x5630e43f2e60 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fa95800bad0 tx=0x7fa95800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345cf4cc342421a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:12.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.132+0000 7fa947fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:12.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.132+0000 7fa947fff640 1 -- 172.21.15.12:0/4163092794 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa958003ea0 con 0x5630e43e82b0 2024-04-22T13:29:12.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.132+0000 7fa947fff640 1 -- 172.21.15.12:0/4163092794 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fa958011440 con 0x5630e43e82b0 2024-04-22T13:29:12.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.132+0000 7fa971a44740 1 -- 172.21.15.12:0/4163092794 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa96001a4b0 con 0x5630e43e82b0 2024-04-22T13:29:12.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.132+0000 7fa947fff640 1 -- 172.21.15.12:0/4163092794 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa95801b640 con 0x5630e43e82b0 2024-04-22T13:29:12.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.132+0000 7fa971a44740 10 client.17733 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:12.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.132+0000 7fa971a44740 1 -- 172.21.15.12:0/4163092794 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5630e401f0f0 con 0x5630e43e82b0 2024-04-22T13:29:12.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.132+0000 7fa947fff640 1 -- 172.21.15.12:0/4163092794 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa96001a4b0 con 0x5630e43e82b0 2024-04-22T13:29:12.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.132+0000 7fa947fff640 1 -- 172.21.15.12:0/4163092794 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa958019020 con 0x5630e43e82b0 2024-04-22T13:29:12.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.132+0000 7fa971a44740 10 client.17733 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:12.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.132+0000 7fa971a44740 10 client.17733 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:12.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.132+0000 7fa971a44740 20 client.17733 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:12.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.132+0000 7fa971a44740 1 --2- 172.21.15.12:0/4163092794 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5630e4405da0 0x5630e4408240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:12.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.132+0000 7fa971a44740 4 client.17733 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:12.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.132+0000 7fa971a44740 1 -- 172.21.15.12:0/4163092794 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630e435ac00 con 0x5630e4405da0 2024-04-22T13:29:12.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.133+0000 7fa967fff640 1 --2- 172.21.15.12:0/4163092794 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5630e4405da0 0x5630e4408240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:12.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.133+0000 7fa967fff640 1 --2- 172.21.15.12:0/4163092794 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5630e4405da0 0x5630e4408240 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ff6e783b5d8f19a server_cookie=cbf48e15196932c4 in_seq=0 out_seq=0 2024-04-22T13:29:12.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.133+0000 7fa947fff640 10 client.17733 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:12.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.134+0000 7fa947fff640 1 -- 172.21.15.12:0/4163092794 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5630e435ac00 con 0x5630e4405da0 2024-04-22T13:29:12.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.134+0000 7fa947fff640 10 client.17733 handle_command_reply: tid=0 2024-04-22T13:29:12.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.134+0000 7fa971a44740 1 client.17733 shutdown 2024-04-22T13:29:12.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.134+0000 7fa971a44740 1 -- 172.21.15.12:0/4163092794 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630e43e82b0 msgr2=0x5630e43f2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:12.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.134+0000 7fa971a44740 1 --2- 172.21.15.12:0/4163092794 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630e43e82b0 0x5630e43f2e60 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fa95800bad0 tx=0x7fa95800bf90 comp rx=0 tx=0).stop 2024-04-22T13:29:12.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.134+0000 7fa971a44740 1 -- 172.21.15.12:0/4163092794 shutdown_connections 2024-04-22T13:29:12.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.134+0000 7fa971a44740 1 --2- 172.21.15.12:0/4163092794 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5630e4405da0 0x5630e4408240 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.134+0000 7fa971a44740 1 --2- 172.21.15.12:0/4163092794 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630e43f5e90 0x5630e43f38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.134+0000 7fa971a44740 1 --2- 172.21.15.12:0/4163092794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630e43e8bd0 0x5630e43f33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.134+0000 7fa947fff640 0 client.17733 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:12.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.134+0000 7fa971a44740 1 --2- 172.21.15.12:0/4163092794 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630e43e82b0 0x5630e43f2e60 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.134+0000 7fa971a44740 1 -- 172.21.15.12:0/4163092794 >> 172.21.15.12:0/4163092794 conn(0x5630e43e2cb0 msgr2=0x5630e43e4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:12.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.134+0000 7fa971a44740 1 -- 172.21.15.12:0/4163092794 shutdown_connections 2024-04-22T13:29:12.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.134+0000 7fa971a44740 1 -- 172.21.15.12:0/4163092794 wait complete. 2024-04-22T13:29:12.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.134+0000 7fa971a44740 20 client.17733 trim_cache size 0 max 16384 2024-04-22T13:29:12.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.151+0000 7fa971a44740 1 Processor -- start 2024-04-22T13:29:12.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.152+0000 7fa971a44740 1 -- start start 2024-04-22T13:29:12.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.152+0000 7fa971a44740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630e43e82b0 0x5630e43f5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:12.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.152+0000 7fa971a44740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630e43e8bd0 0x5630e4401b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:12.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.152+0000 7fa971a44740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630e43f5e90 0x5630e43f2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:12.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.152+0000 7fa971a44740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681ab7e0 con 0x5630e43e82b0 2024-04-22T13:29:12.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.152+0000 7fa971a44740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681ad320 con 0x5630e43f5e90 2024-04-22T13:29:12.153 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.152+0000 7fa971a44740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681abeb0 con 0x5630e43e8bd0 2024-04-22T13:29:12.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.152+0000 7fa96cbea640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630e43e82b0 0x5630e43f5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:12.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.152+0000 7fa96cbea640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630e43e82b0 0x5630e43f5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38096/0 (socket says 172.21.15.12:38096) 2024-04-22T13:29:12.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.152+0000 7fa96cbea640 1 -- 172.21.15.12:0/4046620300 learned_addr learned my addr 172.21.15.12:0/4046620300 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:12.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.152+0000 7fa96d3eb640 1 --2- 172.21.15.12:0/4046620300 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630e43f5e90 0x5630e43f2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:12.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.152+0000 7fa96cbea640 1 -- 172.21.15.12:0/4046620300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630e43e8bd0 msgr2=0x5630e4401b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:12.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.152+0000 7fa96cbea640 1 --2- 172.21.15.12:0/4046620300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630e43e8bd0 0x5630e4401b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.152+0000 7fa96cbea640 1 -- 172.21.15.12:0/4046620300 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630e43f5e90 msgr2=0x5630e43f2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:12.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.152+0000 7fa96cbea640 1 --2- 172.21.15.12:0/4046620300 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630e43f5e90 0x5630e43f2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.152+0000 7fa96cbea640 1 -- 172.21.15.12:0/4046620300 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630e43e8a00 con 0x5630e43e82b0 2024-04-22T13:29:12.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.152+0000 7fa96d3eb640 1 --2- 172.21.15.12:0/4046620300 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630e43f5e90 0x5630e43f2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:12.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.152+0000 7fa96cbea640 1 --2- 172.21.15.12:0/4046620300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630e43e82b0 0x5630e43f5590 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fa958006010 tx=0x7fa958004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535e62f1e5ebcc11 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:12.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.152+0000 7fa94e7fc640 1 -- 172.21.15.12:0/4046620300 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9580039c0 con 0x5630e43e82b0 2024-04-22T13:29:12.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.153+0000 7fa94e7fc640 1 -- 172.21.15.12:0/4046620300 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa958005850 con 0x5630e43e82b0 2024-04-22T13:29:12.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.153+0000 7fa94e7fc640 1 -- 172.21.15.12:0/4046620300 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95801bdf0 con 0x5630e43e82b0 2024-04-22T13:29:12.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.153+0000 7fa971a44740 1 -- 172.21.15.12:0/4046620300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630e43e82b0 msgr2=0x5630e43f5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:12.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.153+0000 7fa971a44740 1 --2- 172.21.15.12:0/4046620300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630e43e82b0 0x5630e43f5590 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fa958006010 tx=0x7fa958004440 comp rx=0 tx=0).stop 2024-04-22T13:29:12.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.153+0000 7fa971a44740 1 -- 172.21.15.12:0/4046620300 shutdown_connections 2024-04-22T13:29:12.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.153+0000 7fa971a44740 1 --2- 172.21.15.12:0/4046620300 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630e43f5e90 0x5630e43f2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.153+0000 7fa971a44740 1 --2- 172.21.15.12:0/4046620300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630e43e8bd0 0x5630e4401b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.153+0000 7fa971a44740 1 --2- 172.21.15.12:0/4046620300 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630e43e82b0 0x5630e43f5590 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.153+0000 7fa971a44740 1 -- 172.21.15.12:0/4046620300 >> 172.21.15.12:0/4046620300 conn(0x5630e43e2cb0 msgr2=0x5630e43e14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:12.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.153+0000 7fa971a44740 1 -- 172.21.15.12:0/4046620300 shutdown_connections 2024-04-22T13:29:12.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.153+0000 7fa971a44740 1 -- 172.21.15.12:0/4046620300 wait complete. 2024-04-22T13:29:12.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.153+0000 7fa971a44740 1 -- start start 2024-04-22T13:29:12.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.153+0000 7fa971a44740 1 Processor -- start 2024-04-22T13:29:12.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.154+0000 7fa971a44740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630e43e82b0 0x5630e43efc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:12.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.154+0000 7fa971a44740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630e43e8bd0 0x5630e43f0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:12.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.154+0000 7fa971a44740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630e43f5e90 0x5630e43f06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:12.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.154+0000 7fa971a44740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681acc50 con 0x5630e43f5e90 2024-04-22T13:29:12.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.154+0000 7fa971a44740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681ac580 con 0x5630e43e82b0 2024-04-22T13:29:12.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.154+0000 7fa971a44740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681a97e0 con 0x5630e43e8bd0 2024-04-22T13:29:12.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.154+0000 7fa971a44740 10 client.0 _ll_register_callbacks cb 0x7fa958004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:12.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.154+0000 7fa96d3eb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630e43f5e90 0x5630e43f06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:12.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.154+0000 7fa96d3eb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630e43f5e90 0x5630e43f06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:38106/0 (socket says 172.21.15.12:38106) 2024-04-22T13:29:12.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.154+0000 7fa96d3eb640 1 -- 172.21.15.12:0/3302012614 learned_addr learned my addr 172.21.15.12:0/3302012614 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:12.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.154+0000 7fa96cbea640 1 --2- 172.21.15.12:0/3302012614 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630e43e82b0 0x5630e43efc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:12.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.154+0000 7fa967fff640 1 --2- 172.21.15.12:0/3302012614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630e43e8bd0 0x5630e43f0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:12.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.154+0000 7fa96d3eb640 1 -- 172.21.15.12:0/3302012614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630e43e8bd0 msgr2=0x5630e43f0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:12.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.154+0000 7fa96d3eb640 1 --2- 172.21.15.12:0/3302012614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630e43e8bd0 0x5630e43f0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.154+0000 7fa96d3eb640 1 -- 172.21.15.12:0/3302012614 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630e43e82b0 msgr2=0x5630e43efc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:12.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.154+0000 7fa96d3eb640 1 --2- 172.21.15.12:0/3302012614 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630e43e82b0 0x5630e43efc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.154+0000 7fa96d3eb640 1 -- 172.21.15.12:0/3302012614 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630e446b450 con 0x5630e43f5e90 2024-04-22T13:29:12.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.154+0000 7fa96cbea640 1 --2- 172.21.15.12:0/3302012614 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630e43e82b0 0x5630e43efc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:29:12.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.155+0000 7fa96d3eb640 1 --2- 172.21.15.12:0/3302012614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630e43f5e90 0x5630e43f06c0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fa960009910 tx=0x7fa960023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=696bf85d394f1643 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:12.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.155+0000 7fa947fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:12.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.155+0000 7fa947fff640 1 -- 172.21.15.12:0/3302012614 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa960011ac0 con 0x5630e43f5e90 2024-04-22T13:29:12.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.155+0000 7fa947fff640 1 -- 172.21.15.12:0/3302012614 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fa960011c60 con 0x5630e43f5e90 2024-04-22T13:29:12.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.155+0000 7fa971a44740 1 -- 172.21.15.12:0/3302012614 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa958005850 con 0x5630e43f5e90 2024-04-22T13:29:12.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.155+0000 7fa947fff640 1 -- 172.21.15.12:0/3302012614 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa96003f050 con 0x5630e43f5e90 2024-04-22T13:29:12.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.155+0000 7fa971a44740 10 client.17739 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:12.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.155+0000 7fa971a44740 1 -- 172.21.15.12:0/3302012614 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5630e446c6d0 con 0x5630e43f5e90 2024-04-22T13:29:12.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.155+0000 7fa947fff640 1 -- 172.21.15.12:0/3302012614 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa958005850 con 0x5630e43f5e90 2024-04-22T13:29:12.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.155+0000 7fa947fff640 1 -- 172.21.15.12:0/3302012614 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa960018b60 con 0x5630e43f5e90 2024-04-22T13:29:12.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.155+0000 7fa971a44740 10 client.17739 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:12.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.155+0000 7fa971a44740 10 client.17739 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:12.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.155+0000 7fa971a44740 20 client.17739 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:12.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.155+0000 7fa971a44740 1 --2- 172.21.15.12:0/3302012614 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5630e446cd00 0x5630e446f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:12.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.155+0000 7fa971a44740 4 client.17739 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:29:12.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.155+0000 7fa971a44740 1 -- 172.21.15.12:0/3302012614 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5630e4354a70 con 0x5630e446cd00 2024-04-22T13:29:12.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.155+0000 7fa96cbea640 1 --2- 172.21.15.12:0/3302012614 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5630e446cd00 0x5630e446f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:12.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.156+0000 7fa96cbea640 1 --2- 172.21.15.12:0/3302012614 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5630e446cd00 0x5630e446f120 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdfd59ed904398df server_cookie=e07801a54b8ba114 in_seq=0 out_seq=0 2024-04-22T13:29:12.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.156+0000 7fa947fff640 10 client.17739 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:12.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.156+0000 7fa947fff640 1 -- 172.21.15.12:0/3302012614 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5630e4354a70 con 0x5630e446cd00 2024-04-22T13:29:12.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.156+0000 7fa947fff640 10 client.17739 handle_command_reply: tid=0 2024-04-22T13:29:12.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.156+0000 7fa971a44740 1 client.17739 shutdown 2024-04-22T13:29:12.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.156+0000 7fa971a44740 1 -- 172.21.15.12:0/3302012614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630e43f5e90 msgr2=0x5630e43f06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:12.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.156+0000 7fa971a44740 1 --2- 172.21.15.12:0/3302012614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630e43f5e90 0x5630e43f06c0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fa960009910 tx=0x7fa960023000 comp rx=0 tx=0).stop 2024-04-22T13:29:12.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.156+0000 7fa971a44740 1 -- 172.21.15.12:0/3302012614 shutdown_connections 2024-04-22T13:29:12.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.156+0000 7fa971a44740 1 --2- 172.21.15.12:0/3302012614 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5630e446cd00 0x5630e446f120 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.156+0000 7fa971a44740 1 --2- 172.21.15.12:0/3302012614 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630e43f5e90 0x5630e43f06c0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.156+0000 7fa971a44740 1 --2- 172.21.15.12:0/3302012614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630e43e8bd0 0x5630e43f0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.156+0000 7fa947fff640 0 client.17739 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:12.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.156+0000 7fa971a44740 1 --2- 172.21.15.12:0/3302012614 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630e43e82b0 0x5630e43efc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.156+0000 7fa971a44740 1 -- 172.21.15.12:0/3302012614 >> 172.21.15.12:0/3302012614 conn(0x5630e43e2cb0 msgr2=0x5630e43f1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:12.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.156+0000 7fa971a44740 1 -- 172.21.15.12:0/3302012614 shutdown_connections 2024-04-22T13:29:12.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.156+0000 7fa971a44740 1 -- 172.21.15.12:0/3302012614 wait complete. 2024-04-22T13:29:12.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.156+0000 7fa971a44740 20 client.17739 trim_cache size 0 max 16384 2024-04-22T13:29:12.159 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:29:12.159 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:29:12.159 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-04-22T13:29:12.159 INFO:teuthology.orchestra.run.smithi012.stdout: "cce66f85-2351-42d4-ab0f-e77412324805": { 2024-04-22T13:29:12.159 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-04-22T13:29:12.160 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "cce66f85-2351-42d4-ab0f-e77412324805", 2024-04-22T13:29:12.160 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-04-22T13:29:12.160 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:29:12.160 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:29:12.160 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:29:12.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.157+0000 7fa96ee4e640 1 -- 172.21.15.12:0/3333775399 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa9500a3d20 msgr2=0x7fa9500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:12.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.157+0000 7fa96ee4e640 1 --2- 172.21.15.12:0/3333775399 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa9500a3d20 0x7fa9500a61e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa960011010 tx=0x7fa960019e10 comp rx=0 tx=0).stop 2024-04-22T13:29:12.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.157+0000 7fa96ee4e640 1 -- 172.21.15.12:0/3333775399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa968106150 msgr2=0x7fa968198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:12.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.157+0000 7fa96ee4e640 1 --2- 172.21.15.12:0/3333775399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa968106150 0x7fa968198c30 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fa95c036040 tx=0x7fa95c02c7e0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.157+0000 7fa96ee4e640 1 -- 172.21.15.12:0/3333775399 shutdown_connections 2024-04-22T13:29:12.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.157+0000 7fa96ee4e640 1 --2- 172.21.15.12:0/3333775399 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa9500a3d20 0x7fa9500a61e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.157+0000 7fa96ee4e640 1 --2- 172.21.15.12:0/3333775399 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa96819cfb0 0x7fa96819d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.157+0000 7fa96ee4e640 1 --2- 172.21.15.12:0/3333775399 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa968106150 0x7fa968198c30 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.157+0000 7fa96ee4e640 1 --2- 172.21.15.12:0/3333775399 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa968101080 0x7fa9681986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:12.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.157+0000 7fa96ee4e640 1 -- 172.21.15.12:0/3333775399 >> 172.21.15.12:0/3333775399 conn(0x7fa968078000 msgr2=0x7fa9680feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:12.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.157+0000 7fa96ee4e640 1 -- 172.21.15.12:0/3333775399 shutdown_connections 2024-04-22T13:29:12.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:12.157+0000 7fa96ee4e640 1 -- 172.21.15.12:0/3333775399 wait complete. 2024-04-22T13:29:12.163 INFO:tasks.cephfs.filesystem:scrub status for tag:cce66f85-2351-42d4-ab0f-e77412324805 - {'path': '/', 'tag': 'cce66f85-2351-42d4-ab0f-e77412324805', 'options': 'recursive,force'} 2024-04-22T13:29:13.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:13 smithi134 ceph-mon[31338]: scrub queued for path: / 2024-04-22T13:29:13.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:13 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:29:13.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:13 smithi134 ceph-mon[31338]: scrub summary: active paths [/] 2024-04-22T13:29:13.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:13 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:29:13.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:13 smithi134 ceph-mon[31338]: pgmap v156: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 29 op/s 2024-04-22T13:29:13.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:13 smithi111 ceph-mon[31361]: scrub queued for path: / 2024-04-22T13:29:13.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:13 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:29:13.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:13 smithi111 ceph-mon[31361]: scrub summary: active paths [/] 2024-04-22T13:29:13.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:13 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:29:13.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:13 smithi111 ceph-mon[31361]: pgmap v156: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 29 op/s 2024-04-22T13:29:14.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:13 smithi012 ceph-mon[28265]: scrub queued for path: / 2024-04-22T13:29:14.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:13 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:29:14.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:13 smithi012 ceph-mon[28265]: scrub summary: active paths [/] 2024-04-22T13:29:14.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:13 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:29:14.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:13 smithi012 ceph-mon[28265]: pgmap v156: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 29 op/s 2024-04-22T13:29:15.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:15 smithi134 ceph-mon[31338]: pgmap v157: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-22T13:29:15.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:15 smithi134 ceph-mon[31338]: scrub summary: idle 2024-04-22T13:29:15.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:15 smithi111 ceph-mon[31361]: pgmap v157: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-22T13:29:15.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:15 smithi111 ceph-mon[31361]: scrub summary: idle 2024-04-22T13:29:16.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:15 smithi012 ceph-mon[28265]: pgmap v157: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-22T13:29:16.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:15 smithi012 ceph-mon[28265]: scrub summary: idle 2024-04-22T13:29:16.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:16 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:29:16.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:16 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:29:16.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:16 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:29:16.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:16 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:29:17.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:16 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:29:17.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:16 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:29:17.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:16 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:29:17.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:16 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:29:17.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:16 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:29:17.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:16 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:29:17.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:16 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:29:17.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:16 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:29:17.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:17 smithi111 ceph-mon[31361]: pgmap v158: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-22T13:29:18.078 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:17 smithi134 ceph-mon[31338]: pgmap v158: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-22T13:29:18.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:17 smithi012 ceph-mon[28265]: pgmap v158: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-22T13:29:19.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:19 smithi111 ceph-mon[31361]: pgmap v159: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 170 B/s wr, 14 op/s 2024-04-22T13:29:20.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:19 smithi012 ceph-mon[28265]: pgmap v159: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 170 B/s wr, 14 op/s 2024-04-22T13:29:20.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:19 smithi134 ceph-mon[31338]: pgmap v159: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 170 B/s wr, 14 op/s 2024-04-22T13:29:21.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:21 smithi111 ceph-mon[31361]: pgmap v160: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-22T13:29:22.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:21 smithi012 ceph-mon[28265]: pgmap v160: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-22T13:29:22.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:21 smithi134 ceph-mon[31338]: pgmap v160: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-22T13:29:23.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:23 smithi111 ceph-mon[31361]: pgmap v161: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 85 B/s wr, 14 op/s 2024-04-22T13:29:24.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:23 smithi012 ceph-mon[28265]: pgmap v161: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 85 B/s wr, 14 op/s 2024-04-22T13:29:24.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:23 smithi134 ceph-mon[31338]: pgmap v161: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 85 B/s wr, 14 op/s 2024-04-22T13:29:25.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:25 smithi111 ceph-mon[31361]: pgmap v162: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:29:26.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:25 smithi012 ceph-mon[28265]: pgmap v162: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:29:26.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:25 smithi134 ceph-mon[31338]: pgmap v162: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:29:27.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:27 smithi111 ceph-mon[31361]: pgmap v163: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:29:28.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:27 smithi012 ceph-mon[28265]: pgmap v163: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:29:28.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:27 smithi134 ceph-mon[31338]: pgmap v163: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:29:29.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:29 smithi111 ceph-mon[31361]: pgmap v164: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:29:30.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:29 smithi012 ceph-mon[28265]: pgmap v164: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:29:30.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:29 smithi134 ceph-mon[31338]: pgmap v164: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:29:31.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:31 smithi111 ceph-mon[31361]: pgmap v165: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:29:32.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:31 smithi012 ceph-mon[28265]: pgmap v165: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:29:32.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:31 smithi134 ceph-mon[31338]: pgmap v165: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:29:33.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:33 smithi111 ceph-mon[31361]: pgmap v166: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:29:34.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:33 smithi012 ceph-mon[28265]: pgmap v166: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:29:34.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:33 smithi134 ceph-mon[31338]: pgmap v166: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:29:35.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:35 smithi111 ceph-mon[31361]: pgmap v167: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:29:36.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:35 smithi012 ceph-mon[28265]: pgmap v167: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:29:36.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:35 smithi134 ceph-mon[31338]: pgmap v167: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:29:37.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:37 smithi111 ceph-mon[31361]: pgmap v168: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:29:38.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:37 smithi012 ceph-mon[28265]: pgmap v168: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:29:38.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:37 smithi134 ceph-mon[31338]: pgmap v168: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:29:39.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:39 smithi111 ceph-mon[31361]: pgmap v169: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:29:40.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:39 smithi012 ceph-mon[28265]: pgmap v169: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:29:40.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:39 smithi134 ceph-mon[31338]: pgmap v169: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:29:41.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:41 smithi111 ceph-mon[31361]: pgmap v170: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:29:42.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:41 smithi012 ceph-mon[28265]: pgmap v170: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:29:42.164 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:29:42.165 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:29:42.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:41 smithi134 ceph-mon[31338]: pgmap v170: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:29:42.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.288+0000 7f5e8e184640 1 -- 172.21.15.12:0/1549180823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e88101080 msgr2=0x7f5e88101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.288+0000 7f5e8e184640 1 --2- 172.21.15.12:0/1549180823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e88101080 0x7f5e88101460 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009900 tx=0x7f5e7802bc30 comp rx=0 tx=0).stop 2024-04-22T13:29:42.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.288+0000 7f5e8e184640 1 -- 172.21.15.12:0/1549180823 shutdown_connections 2024-04-22T13:29:42.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.288+0000 7f5e8e184640 1 --2- 172.21.15.12:0/1549180823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e88106150 0x7f5e8810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.288+0000 7f5e8e184640 1 --2- 172.21.15.12:0/1549180823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e88101a30 0x7f5e88105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.288+0000 7f5e8e184640 1 --2- 172.21.15.12:0/1549180823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e88101080 0x7f5e88101460 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.288+0000 7f5e8e184640 1 -- 172.21.15.12:0/1549180823 >> 172.21.15.12:0/1549180823 conn(0x7f5e88078000 msgr2=0x7f5e88078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:42.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.288+0000 7f5e8e184640 1 -- 172.21.15.12:0/1549180823 shutdown_connections 2024-04-22T13:29:42.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.288+0000 7f5e8e184640 1 -- 172.21.15.12:0/1549180823 wait complete. 2024-04-22T13:29:42.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.289+0000 7f5e8e184640 1 Processor -- start 2024-04-22T13:29:42.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.289+0000 7f5e8e184640 1 -- start start 2024-04-22T13:29:42.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.289+0000 7f5e8e184640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e88101080 0x7f5e88102b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.289+0000 7f5e8e184640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e88106150 0x7f5e88103050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.289+0000 7f5e8e184640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e88103590 0x7f5e881a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.289+0000 7f5e8e184640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5e88111110 con 0x7f5e88101080 2024-04-22T13:29:42.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.289+0000 7f5e8e184640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5e88110f90 con 0x7f5e88103590 2024-04-22T13:29:42.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.289+0000 7f5e8e184640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5e88111290 con 0x7f5e88106150 2024-04-22T13:29:42.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.289+0000 7f5e877fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e88101080 0x7f5e88102b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.289+0000 7f5e877fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e88101080 0x7f5e88102b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43272/0 (socket says 172.21.15.12:43272) 2024-04-22T13:29:42.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.289+0000 7f5e877fe640 1 -- 172.21.15.12:0/3509988534 learned_addr learned my addr 172.21.15.12:0/3509988534 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:42.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.289+0000 7f5e87fff640 1 --2- 172.21.15.12:0/3509988534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e88103590 0x7f5e881a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.289+0000 7f5e86ffd640 1 --2- 172.21.15.12:0/3509988534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e88106150 0x7f5e88103050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.290+0000 7f5e877fe640 1 -- 172.21.15.12:0/3509988534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e88106150 msgr2=0x7f5e88103050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.290+0000 7f5e877fe640 1 --2- 172.21.15.12:0/3509988534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e88106150 0x7f5e88103050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.290+0000 7f5e877fe640 1 -- 172.21.15.12:0/3509988534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e88103590 msgr2=0x7f5e881a6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.290+0000 7f5e877fe640 1 --2- 172.21.15.12:0/3509988534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e88103590 0x7f5e881a6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.290+0000 7f5e877fe640 1 -- 172.21.15.12:0/3509988534 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e881a7010 con 0x7f5e88101080 2024-04-22T13:29:42.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.290+0000 7f5e877fe640 1 --2- 172.21.15.12:0/3509988534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e88101080 0x7f5e88102b10 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f5e78000c00 tx=0x7f5e7802c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:42.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.290+0000 7f5e84ff9640 1 -- 172.21.15.12:0/3509988534 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78043020 con 0x7f5e88101080 2024-04-22T13:29:42.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.290+0000 7f5e8e184640 1 -- 172.21.15.12:0/3509988534 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e881a72a0 con 0x7f5e88101080 2024-04-22T13:29:42.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.290+0000 7f5e84ff9640 1 -- 172.21.15.12:0/3509988534 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e7802cbb0 con 0x7f5e88101080 2024-04-22T13:29:42.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.290+0000 7f5e84ff9640 1 -- 172.21.15.12:0/3509988534 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78003ac0 con 0x7f5e88101080 2024-04-22T13:29:42.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.290+0000 7f5e8e184640 1 -- 172.21.15.12:0/3509988534 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e881a7840 con 0x7f5e88101080 2024-04-22T13:29:42.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.292+0000 7f5e84ff9640 1 -- 172.21.15.12:0/3509988534 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5e78046020 con 0x7f5e88101080 2024-04-22T13:29:42.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.293+0000 7f5e90d7a740 1 Processor -- start 2024-04-22T13:29:42.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.293+0000 7f5e90d7a740 1 -- start start 2024-04-22T13:29:42.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.293+0000 7f5e90d7a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ecf73d580 0x555ecf73d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.293+0000 7f5e90d7a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ecf73dea0 0x555ecf746540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.293+0000 7f5e90d7a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ecf746a80 0x555ecf748e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.293+0000 7f5e90d7a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555ecf6a21e0 con 0x555ecf746a80 2024-04-22T13:29:42.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.293+0000 7f5e90d7a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555ecf63f390 con 0x555ecf73dea0 2024-04-22T13:29:42.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.293+0000 7f5e90d7a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555ecf69e1a0 con 0x555ecf73d580 2024-04-22T13:29:42.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.293+0000 7f5e84ff9640 1 --2- 172.21.15.12:0/3509988534 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5e700a3d50 0x7f5e700a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.293+0000 7f5e86ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ecf746a80 0x555ecf748e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.293+0000 7f5e86ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ecf746a80 0x555ecf748e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43276/0 (socket says 172.21.15.12:43276) 2024-04-22T13:29:42.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.293+0000 7f5e86ffd640 1 -- 172.21.15.12:0/3568591057 learned_addr learned my addr 172.21.15.12:0/3568591057 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:42.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.293+0000 7f5e877fe640 1 --2- 172.21.15.12:0/3568591057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ecf73dea0 0x555ecf746540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.293+0000 7f5e87fff640 1 --2- 172.21.15.12:0/3568591057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ecf73d580 0x555ecf73d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.293+0000 7f5e87fff640 1 --2- 172.21.15.12:0/3509988534 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5e700a3d50 0x7f5e700a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.293+0000 7f5e86ffd640 1 -- 172.21.15.12:0/3568591057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ecf73d580 msgr2=0x555ecf73d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:42.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.293+0000 7f5e86ffd640 1 --2- 172.21.15.12:0/3568591057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ecf73d580 0x555ecf73d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.293+0000 7f5e84ff9640 1 -- 172.21.15.12:0/3509988534 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f5e780f2710 con 0x7f5e88101080 2024-04-22T13:29:42.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.293+0000 7f5e86ffd640 1 -- 172.21.15.12:0/3568591057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ecf73dea0 msgr2=0x555ecf746540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:42.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.293+0000 7f5e86ffd640 1 --2- 172.21.15.12:0/3568591057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ecf73dea0 0x555ecf746540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.293+0000 7f5e86ffd640 1 -- 172.21.15.12:0/3568591057 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ecf47e7d0 con 0x555ecf746a80 2024-04-22T13:29:42.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.294+0000 7f5e86ffd640 1 --2- 172.21.15.12:0/3568591057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ecf746a80 0x555ecf748e70 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00abb0 tx=0x7f5e7c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a30a8b4a86561777 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:42.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.294+0000 7f5e87fff640 1 --2- 172.21.15.12:0/3509988534 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5e700a3d50 0x7f5e700a6210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5e80005fd0 tx=0x7f5e80009500 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:42.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.294+0000 7f5e6dffb640 1 -- 172.21.15.12:0/3568591057 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c01d070 con 0x555ecf746a80 2024-04-22T13:29:42.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.294+0000 7f5e6dffb640 1 -- 172.21.15.12:0/3568591057 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e7c018af0 con 0x555ecf746a80 2024-04-22T13:29:42.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.294+0000 7f5e6dffb640 1 -- 172.21.15.12:0/3568591057 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c010a50 con 0x555ecf746a80 2024-04-22T13:29:42.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.295+0000 7f5e90d7a740 1 -- 172.21.15.12:0/3568591057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ecf746a80 msgr2=0x555ecf748e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.295+0000 7f5e90d7a740 1 --2- 172.21.15.12:0/3568591057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ecf746a80 0x555ecf748e70 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00abb0 tx=0x7f5e7c00f790 comp rx=0 tx=0).stop 2024-04-22T13:29:42.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.295+0000 7f5e90d7a740 1 -- 172.21.15.12:0/3568591057 shutdown_connections 2024-04-22T13:29:42.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.295+0000 7f5e90d7a740 1 --2- 172.21.15.12:0/3568591057 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ecf746a80 0x555ecf748e70 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.295+0000 7f5e90d7a740 1 --2- 172.21.15.12:0/3568591057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ecf73dea0 0x555ecf746540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.295+0000 7f5e90d7a740 1 --2- 172.21.15.12:0/3568591057 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ecf73d580 0x555ecf73d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.295+0000 7f5e90d7a740 1 -- 172.21.15.12:0/3568591057 >> 172.21.15.12:0/3568591057 conn(0x555ecf737cb0 msgr2=0x555ecf738090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:42.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.295+0000 7f5e90d7a740 1 -- 172.21.15.12:0/3568591057 shutdown_connections 2024-04-22T13:29:42.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.295+0000 7f5e90d7a740 1 -- 172.21.15.12:0/3568591057 wait complete. 2024-04-22T13:29:42.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.295+0000 7f5e90d7a740 1 -- start start 2024-04-22T13:29:42.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.295+0000 7f5e90d7a740 1 Processor -- start 2024-04-22T13:29:42.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.295+0000 7f5e90d7a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ecf73d580 0x555ecf743a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.296+0000 7f5e90d7a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ecf73dea0 0x555ecf743fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.296+0000 7f5e90d7a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ecf746a80 0x555ecf7444f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.296+0000 7f5e90d7a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555ecf749870 con 0x555ecf746a80 2024-04-22T13:29:42.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.296+0000 7f5e90d7a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555ecf65b2f0 con 0x555ecf73dea0 2024-04-22T13:29:42.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.296+0000 7f5e90d7a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555ecf7499f0 con 0x555ecf73d580 2024-04-22T13:29:42.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.296+0000 7f5e90d7a740 10 client.0 _ll_register_callbacks cb 0x555ecf668580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:42.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.296+0000 7f5e87fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ecf746a80 0x555ecf7444f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.296+0000 7f5e87fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ecf746a80 0x555ecf7444f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43282/0 (socket says 172.21.15.12:43282) 2024-04-22T13:29:42.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.296+0000 7f5e87fff640 1 -- 172.21.15.12:0/2924779549 learned_addr learned my addr 172.21.15.12:0/2924779549 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:42.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.296+0000 7f5e877fe640 1 --2- 172.21.15.12:0/2924779549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ecf73d580 0x555ecf743a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.296+0000 7f5e86ffd640 1 --2- 172.21.15.12:0/2924779549 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ecf73dea0 0x555ecf743fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.296+0000 7f5e87fff640 1 -- 172.21.15.12:0/2924779549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ecf73d580 msgr2=0x555ecf743a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:42.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.296+0000 7f5e87fff640 1 --2- 172.21.15.12:0/2924779549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ecf73d580 0x555ecf743a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.296+0000 7f5e87fff640 1 -- 172.21.15.12:0/2924779549 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ecf73dea0 msgr2=0x555ecf743fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:42.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.296+0000 7f5e87fff640 1 --2- 172.21.15.12:0/2924779549 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ecf73dea0 0x555ecf743fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.296+0000 7f5e87fff640 1 -- 172.21.15.12:0/2924779549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ecf3b43f0 con 0x555ecf746a80 2024-04-22T13:29:42.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.296+0000 7f5e86ffd640 1 --2- 172.21.15.12:0/2924779549 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ecf73dea0 0x555ecf743fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:42.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.296+0000 7f5e87fff640 1 --2- 172.21.15.12:0/2924779549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ecf746a80 0x555ecf7444f0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f5e80020b80 tx=0x7f5e80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3fbcf3b41d7cd73 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:42.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.296+0000 7f5e66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:42.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.296+0000 7f5e66ffd640 1 -- 172.21.15.12:0/2924779549 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80010cc0 con 0x555ecf746a80 2024-04-22T13:29:42.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.297+0000 7f5e66ffd640 1 -- 172.21.15.12:0/2924779549 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f5e8001e3f0 con 0x555ecf746a80 2024-04-22T13:29:42.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.297+0000 7f5e90d7a740 1 -- 172.21.15.12:0/2924779549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e7c018af0 con 0x555ecf746a80 2024-04-22T13:29:42.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.297+0000 7f5e66ffd640 1 -- 172.21.15.12:0/2924779549 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5e80010e60 con 0x555ecf746a80 2024-04-22T13:29:42.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.297+0000 7f5e90d7a740 10 client.17751 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:42.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.297+0000 7f5e90d7a740 1 -- 172.21.15.12:0/2924779549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555ecf3740f0 con 0x555ecf746a80 2024-04-22T13:29:42.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.297+0000 7f5e66ffd640 1 -- 172.21.15.12:0/2924779549 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e7c018af0 con 0x555ecf746a80 2024-04-22T13:29:42.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.297+0000 7f5e66ffd640 1 -- 172.21.15.12:0/2924779549 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5e80027070 con 0x555ecf746a80 2024-04-22T13:29:42.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.297+0000 7f5e90d7a740 10 client.17751 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:42.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.297+0000 7f5e90d7a740 10 client.17751 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:42.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.297+0000 7f5e90d7a740 20 client.17751 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:42.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.297+0000 7f5e90d7a740 1 --2- 172.21.15.12:0/2924779549 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555ecf756b00 0x555ecf758f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.297+0000 7f5e90d7a740 4 client.17751 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:42.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.297+0000 7f5e90d7a740 1 -- 172.21.15.12:0/2924779549 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ecf6afc00 con 0x555ecf756b00 2024-04-22T13:29:42.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.297+0000 7f5e877fe640 1 --2- 172.21.15.12:0/2924779549 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555ecf756b00 0x555ecf758f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.298+0000 7f5e66ffd640 10 client.17751 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:42.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.298+0000 7f5e877fe640 1 --2- 172.21.15.12:0/2924779549 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555ecf756b00 0x555ecf758f80 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57cdce6fb2225d0a server_cookie=f15593708d32f9b4 in_seq=0 out_seq=0 2024-04-22T13:29:42.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.298+0000 7f5e66ffd640 1 -- 172.21.15.12:0/2924779549 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555ecf6afc00 con 0x555ecf756b00 2024-04-22T13:29:42.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.298+0000 7f5e66ffd640 10 client.17751 handle_command_reply: tid=0 2024-04-22T13:29:42.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.299+0000 7f5e90d7a740 1 client.17751 shutdown 2024-04-22T13:29:42.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.299+0000 7f5e90d7a740 1 -- 172.21.15.12:0/2924779549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ecf746a80 msgr2=0x555ecf7444f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.299+0000 7f5e90d7a740 1 --2- 172.21.15.12:0/2924779549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ecf746a80 0x555ecf7444f0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f5e80020b80 tx=0x7f5e80004a40 comp rx=0 tx=0).stop 2024-04-22T13:29:42.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.299+0000 7f5e90d7a740 1 -- 172.21.15.12:0/2924779549 shutdown_connections 2024-04-22T13:29:42.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.299+0000 7f5e90d7a740 1 --2- 172.21.15.12:0/2924779549 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555ecf756b00 0x555ecf758f80 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.299+0000 7f5e90d7a740 1 --2- 172.21.15.12:0/2924779549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ecf746a80 0x555ecf7444f0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.299+0000 7f5e66ffd640 0 client.17751 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:42.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.299+0000 7f5e90d7a740 1 --2- 172.21.15.12:0/2924779549 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ecf73dea0 0x555ecf743fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.299+0000 7f5e90d7a740 1 --2- 172.21.15.12:0/2924779549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ecf73d580 0x555ecf743a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.299+0000 7f5e90d7a740 1 -- 172.21.15.12:0/2924779549 >> 172.21.15.12:0/2924779549 conn(0x555ecf737cb0 msgr2=0x555ecf739a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:42.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.299+0000 7f5e90d7a740 1 -- 172.21.15.12:0/2924779549 shutdown_connections 2024-04-22T13:29:42.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.299+0000 7f5e90d7a740 1 -- 172.21.15.12:0/2924779549 wait complete. 2024-04-22T13:29:42.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.299+0000 7f5e90d7a740 20 client.17751 trim_cache size 0 max 16384 2024-04-22T13:29:42.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.317+0000 7f5e90d7a740 1 Processor -- start 2024-04-22T13:29:42.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.317+0000 7f5e90d7a740 1 -- start start 2024-04-22T13:29:42.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.317+0000 7f5e90d7a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ecf73d580 0x555ecf7461a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.317+0000 7f5e90d7a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ecf73dea0 0x555ecf752760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.317+0000 7f5e90d7a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ecf746a80 0x555ecf75e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.317+0000 7f5e90d7a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5e881afcc0 con 0x555ecf73d580 2024-04-22T13:29:42.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.317+0000 7f5e90d7a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5e881b1800 con 0x555ecf73dea0 2024-04-22T13:29:42.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.317+0000 7f5e90d7a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5e881b0390 con 0x555ecf746a80 2024-04-22T13:29:42.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.317+0000 7f5e877fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ecf73d580 0x555ecf7461a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.317+0000 7f5e877fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ecf73d580 0x555ecf7461a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43296/0 (socket says 172.21.15.12:43296) 2024-04-22T13:29:42.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.317+0000 7f5e877fe640 1 -- 172.21.15.12:0/210887887 learned_addr learned my addr 172.21.15.12:0/210887887 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:42.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.317+0000 7f5e87fff640 1 --2- 172.21.15.12:0/210887887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ecf746a80 0x555ecf75e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.317+0000 7f5e86ffd640 1 --2- 172.21.15.12:0/210887887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ecf73dea0 0x555ecf752760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.317+0000 7f5e877fe640 1 -- 172.21.15.12:0/210887887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ecf746a80 msgr2=0x555ecf75e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:42.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.317+0000 7f5e877fe640 1 --2- 172.21.15.12:0/210887887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ecf746a80 0x555ecf75e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.317+0000 7f5e877fe640 1 -- 172.21.15.12:0/210887887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ecf73dea0 msgr2=0x555ecf752760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:42.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.317+0000 7f5e877fe640 1 --2- 172.21.15.12:0/210887887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ecf73dea0 0x555ecf752760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.317+0000 7f5e877fe640 1 -- 172.21.15.12:0/210887887 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ecf737ae0 con 0x555ecf73d580 2024-04-22T13:29:42.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.317+0000 7f5e86ffd640 1 --2- 172.21.15.12:0/210887887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ecf73dea0 0x555ecf752760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:42.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.317+0000 7f5e877fe640 1 --2- 172.21.15.12:0/210887887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ecf73d580 0x555ecf7461a0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f5e780be860 tx=0x7f5e780f4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc7c15eafb196c0d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:42.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.317+0000 7f5e6d7fa640 1 -- 172.21.15.12:0/210887887 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e780be8d0 con 0x555ecf73d580 2024-04-22T13:29:42.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.318+0000 7f5e6d7fa640 1 -- 172.21.15.12:0/210887887 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e780c2c80 con 0x555ecf73d580 2024-04-22T13:29:42.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.318+0000 7f5e6d7fa640 1 -- 172.21.15.12:0/210887887 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78003c60 con 0x555ecf73d580 2024-04-22T13:29:42.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.318+0000 7f5e90d7a740 1 -- 172.21.15.12:0/210887887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ecf73d580 msgr2=0x555ecf7461a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.318+0000 7f5e90d7a740 1 --2- 172.21.15.12:0/210887887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ecf73d580 0x555ecf7461a0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f5e780be860 tx=0x7f5e780f4040 comp rx=0 tx=0).stop 2024-04-22T13:29:42.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.318+0000 7f5e90d7a740 1 -- 172.21.15.12:0/210887887 shutdown_connections 2024-04-22T13:29:42.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.318+0000 7f5e90d7a740 1 --2- 172.21.15.12:0/210887887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ecf746a80 0x555ecf75e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.318+0000 7f5e90d7a740 1 --2- 172.21.15.12:0/210887887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ecf73dea0 0x555ecf752760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.318+0000 7f5e90d7a740 1 --2- 172.21.15.12:0/210887887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ecf73d580 0x555ecf7461a0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.318+0000 7f5e90d7a740 1 -- 172.21.15.12:0/210887887 >> 172.21.15.12:0/210887887 conn(0x555ecf737cb0 msgr2=0x555ecf736d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:42.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.318+0000 7f5e90d7a740 1 -- 172.21.15.12:0/210887887 shutdown_connections 2024-04-22T13:29:42.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.318+0000 7f5e90d7a740 1 -- 172.21.15.12:0/210887887 wait complete. 2024-04-22T13:29:42.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.318+0000 7f5e90d7a740 1 -- start start 2024-04-22T13:29:42.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.319+0000 7f5e90d7a740 1 Processor -- start 2024-04-22T13:29:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.319+0000 7f5e90d7a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ecf73d580 0x555ecf74ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.319+0000 7f5e90d7a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ecf73dea0 0x555ecf74b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.319+0000 7f5e90d7a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ecf746a80 0x555ecf74b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.319+0000 7f5e90d7a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5e881b1130 con 0x555ecf73d580 2024-04-22T13:29:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.319+0000 7f5e90d7a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5e881b0a60 con 0x555ecf746a80 2024-04-22T13:29:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.319+0000 7f5e90d7a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5e881adcc0 con 0x555ecf73dea0 2024-04-22T13:29:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.319+0000 7f5e90d7a740 10 client.0 _ll_register_callbacks cb 0x7f5e80011370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.319+0000 7f5e877fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ecf73d580 0x555ecf74ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.319+0000 7f5e877fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ecf73d580 0x555ecf74ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43304/0 (socket says 172.21.15.12:43304) 2024-04-22T13:29:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.319+0000 7f5e877fe640 1 -- 172.21.15.12:0/6908154 learned_addr learned my addr 172.21.15.12:0/6908154 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.319+0000 7f5e86ffd640 1 --2- 172.21.15.12:0/6908154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ecf73dea0 0x555ecf74b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.319+0000 7f5e87fff640 1 --2- 172.21.15.12:0/6908154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ecf746a80 0x555ecf74b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.319+0000 7f5e877fe640 1 -- 172.21.15.12:0/6908154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ecf73dea0 msgr2=0x555ecf74b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:42.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.319+0000 7f5e877fe640 1 --2- 172.21.15.12:0/6908154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ecf73dea0 0x555ecf74b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.319+0000 7f5e877fe640 1 -- 172.21.15.12:0/6908154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ecf746a80 msgr2=0x555ecf74b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:42.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.319+0000 7f5e877fe640 1 --2- 172.21.15.12:0/6908154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ecf746a80 0x555ecf74b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.319+0000 7f5e877fe640 1 -- 172.21.15.12:0/6908154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ecf74bb00 con 0x555ecf73d580 2024-04-22T13:29:42.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.320+0000 7f5e877fe640 1 --2- 172.21.15.12:0/6908154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ecf73d580 0x555ecf74ed00 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f5e7804aff0 tx=0x7f5e7802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348b9b0ea0b3f6e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:42.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.320+0000 7f5e66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:42.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.320+0000 7f5e66ffd640 1 -- 172.21.15.12:0/6908154 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78005420 con 0x555ecf73d580 2024-04-22T13:29:42.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.320+0000 7f5e66ffd640 1 -- 172.21.15.12:0/6908154 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f5e78003c60 con 0x555ecf73d580 2024-04-22T13:29:42.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.320+0000 7f5e90d7a740 1 -- 172.21.15.12:0/6908154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e780c2c80 con 0x555ecf73d580 2024-04-22T13:29:42.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.320+0000 7f5e66ffd640 1 -- 172.21.15.12:0/6908154 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5e78003e60 con 0x555ecf73d580 2024-04-22T13:29:42.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.320+0000 7f5e90d7a740 10 client.17757 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:42.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.320+0000 7f5e90d7a740 1 -- 172.21.15.12:0/6908154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555ecf7bdd20 con 0x555ecf73d580 2024-04-22T13:29:42.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.320+0000 7f5e66ffd640 1 -- 172.21.15.12:0/6908154 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e780c2c80 con 0x555ecf73d580 2024-04-22T13:29:42.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.320+0000 7f5e66ffd640 1 -- 172.21.15.12:0/6908154 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5e780c7140 con 0x555ecf73d580 2024-04-22T13:29:42.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.320+0000 7f5e90d7a740 10 client.17757 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:42.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.320+0000 7f5e90d7a740 10 client.17757 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:42.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.320+0000 7f5e90d7a740 20 client.17757 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:42.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.320+0000 7f5e90d7a740 1 --2- 172.21.15.12:0/6908154 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555ecf7be380 0x555ecf7c0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.320+0000 7f5e90d7a740 4 client.17757 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:29:42.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.320+0000 7f5e90d7a740 1 -- 172.21.15.12:0/6908154 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555ecf6a9a70 con 0x555ecf7be380 2024-04-22T13:29:42.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.321+0000 7f5e86ffd640 1 --2- 172.21.15.12:0/6908154 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555ecf7be380 0x555ecf7c0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.321+0000 7f5e86ffd640 1 --2- 172.21.15.12:0/6908154 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555ecf7be380 0x555ecf7c0770 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49d8314179b413c7 server_cookie=7051c3d94edca4d4 in_seq=0 out_seq=0 2024-04-22T13:29:42.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.321+0000 7f5e66ffd640 10 client.17757 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:42.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.321+0000 7f5e66ffd640 1 -- 172.21.15.12:0/6908154 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555ecf6a9a70 con 0x555ecf7be380 2024-04-22T13:29:42.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.321+0000 7f5e66ffd640 10 client.17757 handle_command_reply: tid=0 2024-04-22T13:29:42.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.321+0000 7f5e90d7a740 1 client.17757 shutdown 2024-04-22T13:29:42.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.321+0000 7f5e90d7a740 1 -- 172.21.15.12:0/6908154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ecf73d580 msgr2=0x555ecf74ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.323 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.321+0000 7f5e90d7a740 1 --2- 172.21.15.12:0/6908154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ecf73d580 0x555ecf74ed00 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f5e7804aff0 tx=0x7f5e7802bc30 comp rx=0 tx=0).stop 2024-04-22T13:29:42.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.321+0000 7f5e90d7a740 1 -- 172.21.15.12:0/6908154 shutdown_connections 2024-04-22T13:29:42.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.321+0000 7f5e90d7a740 1 --2- 172.21.15.12:0/6908154 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555ecf7be380 0x555ecf7c0770 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.321+0000 7f5e90d7a740 1 --2- 172.21.15.12:0/6908154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ecf746a80 0x555ecf74b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.321+0000 7f5e90d7a740 1 --2- 172.21.15.12:0/6908154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555ecf73dea0 0x555ecf74b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.321+0000 7f5e66ffd640 0 client.17757 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:42.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.321+0000 7f5e90d7a740 1 --2- 172.21.15.12:0/6908154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555ecf73d580 0x555ecf74ed00 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.321+0000 7f5e90d7a740 1 -- 172.21.15.12:0/6908154 >> 172.21.15.12:0/6908154 conn(0x555ecf737cb0 msgr2=0x555ecf7364e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:42.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.321+0000 7f5e90d7a740 1 -- 172.21.15.12:0/6908154 shutdown_connections 2024-04-22T13:29:42.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.321+0000 7f5e90d7a740 1 -- 172.21.15.12:0/6908154 wait complete. 2024-04-22T13:29:42.324 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.322+0000 7f5e90d7a740 20 client.17757 trim_cache size 0 max 16384 2024-04-22T13:29:42.325 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:29:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-04-22T13:29:42.325 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-04-22T13:29:42.325 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:29:42.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.322+0000 7f5e8e184640 1 -- 172.21.15.12:0/3509988534 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5e700a3d50 msgr2=0x7f5e700a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.322+0000 7f5e8e184640 1 --2- 172.21.15.12:0/3509988534 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5e700a3d50 0x7f5e700a6210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5e80005fd0 tx=0x7f5e80009500 comp rx=0 tx=0).stop 2024-04-22T13:29:42.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.322+0000 7f5e8e184640 1 -- 172.21.15.12:0/3509988534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e88101080 msgr2=0x7f5e88102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.322+0000 7f5e8e184640 1 --2- 172.21.15.12:0/3509988534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e88101080 0x7f5e88102b10 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f5e78000c00 tx=0x7f5e7802c6c0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.322+0000 7f5e8e184640 1 -- 172.21.15.12:0/3509988534 shutdown_connections 2024-04-22T13:29:42.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.322+0000 7f5e8e184640 1 --2- 172.21.15.12:0/3509988534 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5e700a3d50 0x7f5e700a6210 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.322+0000 7f5e8e184640 1 --2- 172.21.15.12:0/3509988534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e88103590 0x7f5e881a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.322+0000 7f5e8e184640 1 --2- 172.21.15.12:0/3509988534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e88106150 0x7f5e88103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.322+0000 7f5e8e184640 1 --2- 172.21.15.12:0/3509988534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5e88101080 0x7f5e88102b10 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.322+0000 7f5e8e184640 1 -- 172.21.15.12:0/3509988534 >> 172.21.15.12:0/3509988534 conn(0x7f5e88078000 msgr2=0x7f5e88069490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:42.326 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.322+0000 7f5e8e184640 1 -- 172.21.15.12:0/3509988534 shutdown_connections 2024-04-22T13:29:42.327 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.322+0000 7f5e8e184640 1 -- 172.21.15.12:0/3509988534 wait complete. 2024-04-22T13:29:42.329 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:29:42.329 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:29:42.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.435+0000 7ff7b6050640 1 -- 172.21.15.12:0/3954645335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7b0101110 msgr2=0x7ff7b0101570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.435+0000 7ff7b6050640 1 --2- 172.21.15.12:0/3954645335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7b0101110 0x7ff7b0101570 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4009900 tx=0x7ff7a402bc30 comp rx=0 tx=0).stop 2024-04-22T13:29:42.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.436+0000 7ff7b6050640 1 -- 172.21.15.12:0/3954645335 shutdown_connections 2024-04-22T13:29:42.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.436+0000 7ff7b6050640 1 --2- 172.21.15.12:0/3954645335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7b0101ab0 0x7ff7b010dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.436+0000 7ff7b6050640 1 --2- 172.21.15.12:0/3954645335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7b0101110 0x7ff7b0101570 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.436+0000 7ff7b6050640 1 --2- 172.21.15.12:0/3954645335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7b0107010 0x7ff7b01073f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.436+0000 7ff7b6050640 1 -- 172.21.15.12:0/3954645335 >> 172.21.15.12:0/3954645335 conn(0x7ff7b0077e30 msgr2=0x7ff7b0078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:42.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.436+0000 7ff7b6050640 1 -- 172.21.15.12:0/3954645335 shutdown_connections 2024-04-22T13:29:42.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.436+0000 7ff7b6050640 1 -- 172.21.15.12:0/3954645335 wait complete. 2024-04-22T13:29:42.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.436+0000 7ff7b6050640 1 Processor -- start 2024-04-22T13:29:42.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.436+0000 7ff7b6050640 1 -- start start 2024-04-22T13:29:42.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.436+0000 7ff7b6050640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7b0101110 0x7ff7b01986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.436+0000 7ff7b6050640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7b0101ab0 0x7ff7b0198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.434 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.437+0000 7ff7b6050640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7b0107010 0x7ff7b019cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.437+0000 7ff7b6050640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b0112230 con 0x7ff7b0101110 2024-04-22T13:29:42.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.437+0000 7ff7b6050640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b01120b0 con 0x7ff7b0101ab0 2024-04-22T13:29:42.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.437+0000 7ff7b6050640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b01123b0 con 0x7ff7b0107010 2024-04-22T13:29:42.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.437+0000 7ff7af7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7b0101110 0x7ff7b01986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.437+0000 7ff7af7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7b0101110 0x7ff7b01986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43324/0 (socket says 172.21.15.12:43324) 2024-04-22T13:29:42.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.437+0000 7ff7af7fe640 1 -- 172.21.15.12:0/1068026995 learned_addr learned my addr 172.21.15.12:0/1068026995 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:42.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.437+0000 7ff7affff640 1 --2- 172.21.15.12:0/1068026995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7b0107010 0x7ff7b019cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.437+0000 7ff7aeffd640 1 --2- 172.21.15.12:0/1068026995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7b0101ab0 0x7ff7b0198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.437+0000 7ff7af7fe640 1 -- 172.21.15.12:0/1068026995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7b0107010 msgr2=0x7ff7b019cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.437+0000 7ff7af7fe640 1 --2- 172.21.15.12:0/1068026995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7b0107010 0x7ff7b019cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.437+0000 7ff7af7fe640 1 -- 172.21.15.12:0/1068026995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7b0101ab0 msgr2=0x7ff7b0198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.437+0000 7ff7af7fe640 1 --2- 172.21.15.12:0/1068026995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7b0101ab0 0x7ff7b0198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.437+0000 7ff7af7fe640 1 -- 172.21.15.12:0/1068026995 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b019d6e0 con 0x7ff7b0101110 2024-04-22T13:29:42.435 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.437+0000 7ff7affff640 1 --2- 172.21.15.12:0/1068026995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7b0107010 0x7ff7b019cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:42.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.437+0000 7ff7af7fe640 1 --2- 172.21.15.12:0/1068026995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7b0101110 0x7ff7b01986f0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000c890 tx=0x7ff7a000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:42.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.437+0000 7ff7acff9640 1 -- 172.21.15.12:0/1068026995 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0004220 con 0x7ff7b0101110 2024-04-22T13:29:42.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.437+0000 7ff7acff9640 1 -- 172.21.15.12:0/1068026995 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7a00043c0 con 0x7ff7b0101110 2024-04-22T13:29:42.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.437+0000 7ff7acff9640 1 -- 172.21.15.12:0/1068026995 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0004a00 con 0x7ff7b0101110 2024-04-22T13:29:42.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.437+0000 7ff7b6050640 1 -- 172.21.15.12:0/1068026995 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b019d9d0 con 0x7ff7b0101110 2024-04-22T13:29:42.436 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.437+0000 7ff7b6050640 1 -- 172.21.15.12:0/1068026995 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b01a5260 con 0x7ff7b0101110 2024-04-22T13:29:42.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.438+0000 7ff7b6050640 1 -- 172.21.15.12:0/1068026995 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7b0069230 con 0x7ff7b0101110 2024-04-22T13:29:42.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.439+0000 7ff7acff9640 1 -- 172.21.15.12:0/1068026995 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff7a00079b0 con 0x7ff7b0101110 2024-04-22T13:29:42.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.440+0000 7ff7acff9640 1 --2- 172.21.15.12:0/1068026995 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff7980a3d20 0x7ff7980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.440+0000 7ff7aeffd640 1 --2- 172.21.15.12:0/1068026995 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff7980a3d20 0x7ff7980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.440+0000 7ff7acff9640 1 -- 172.21.15.12:0/1068026995 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7ff7a00d0500 con 0x7ff7b0101110 2024-04-22T13:29:42.437 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.441+0000 7ff7aeffd640 1 --2- 172.21.15.12:0/1068026995 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff7980a3d20 0x7ff7980a61e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4006010 tx=0x7ff7a4005e40 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:42.440 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.445+0000 7ff7acff9640 1 -- 172.21.15.12:0/1068026995 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff7a0014030 con 0x7ff7b0101110 2024-04-22T13:29:42.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.670+0000 7ff7b6050640 1 -- 172.21.15.12:0/1068026995 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7b0109300 con 0x7ff7b0101110 2024-04-22T13:29:42.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.670+0000 7ff7acff9640 1 -- 172.21.15.12:0/1068026995 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7ff7a009cbf0 con 0x7ff7b0101110 2024-04-22T13:29:42.666 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:29:42.666 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:29:42.667 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:29:42.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.672+0000 7ff7b6050640 1 -- 172.21.15.12:0/1068026995 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff7980a3d20 msgr2=0x7ff7980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.672+0000 7ff7b6050640 1 --2- 172.21.15.12:0/1068026995 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff7980a3d20 0x7ff7980a61e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4006010 tx=0x7ff7a4005e40 comp rx=0 tx=0).stop 2024-04-22T13:29:42.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.672+0000 7ff7b6050640 1 -- 172.21.15.12:0/1068026995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7b0101110 msgr2=0x7ff7b01986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.672+0000 7ff7b6050640 1 --2- 172.21.15.12:0/1068026995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7b0101110 0x7ff7b01986f0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000c890 tx=0x7ff7a000cd50 comp rx=0 tx=0).stop 2024-04-22T13:29:42.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.672+0000 7ff7b6050640 1 -- 172.21.15.12:0/1068026995 shutdown_connections 2024-04-22T13:29:42.668 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.672+0000 7ff7b6050640 1 --2- 172.21.15.12:0/1068026995 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff7980a3d20 0x7ff7980a61e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.672+0000 7ff7b6050640 1 --2- 172.21.15.12:0/1068026995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff7b0107010 0x7ff7b019cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.672+0000 7ff7b6050640 1 --2- 172.21.15.12:0/1068026995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7b0101ab0 0x7ff7b0198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.672+0000 7ff7b6050640 1 --2- 172.21.15.12:0/1068026995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff7b0101110 0x7ff7b01986f0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.672+0000 7ff7b6050640 1 -- 172.21.15.12:0/1068026995 >> 172.21.15.12:0/1068026995 conn(0x7ff7b0077e30 msgr2=0x7ff7b00fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:42.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.672+0000 7ff7b6050640 1 -- 172.21.15.12:0/1068026995 shutdown_connections 2024-04-22T13:29:42.669 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.672+0000 7ff7b6050640 1 -- 172.21.15.12:0/1068026995 wait complete. 2024-04-22T13:29:42.679 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:29:42.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.791+0000 7f0308c25640 1 -- 172.21.15.12:0/1722992371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f030406a840 msgr2=0x7f030406ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.791+0000 7f0308c25640 1 --2- 172.21.15.12:0/1722992371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f030406a840 0x7f030406ac20 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f02f4009900 tx=0x7f02f402bc30 comp rx=0 tx=0).stop 2024-04-22T13:29:42.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.792+0000 7f0308c25640 1 -- 172.21.15.12:0/1722992371 shutdown_connections 2024-04-22T13:29:42.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.792+0000 7f0308c25640 1 --2- 172.21.15.12:0/1722992371 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0304107270 0x7f030410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.792+0000 7f0308c25640 1 --2- 172.21.15.12:0/1722992371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f030406b1f0 0x7f0304106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.792+0000 7f0308c25640 1 --2- 172.21.15.12:0/1722992371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f030406a840 0x7f030406ac20 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.792+0000 7f0308c25640 1 -- 172.21.15.12:0/1722992371 >> 172.21.15.12:0/1722992371 conn(0x7f0304077eb0 msgr2=0x7f03040782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:42.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.792+0000 7f0308c25640 1 -- 172.21.15.12:0/1722992371 shutdown_connections 2024-04-22T13:29:42.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.792+0000 7f0308c25640 1 -- 172.21.15.12:0/1722992371 wait complete. 2024-04-22T13:29:42.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.792+0000 7f0308c25640 1 Processor -- start 2024-04-22T13:29:42.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.792+0000 7f0308c25640 1 -- start start 2024-04-22T13:29:42.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.792+0000 7f0308c25640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f030406a840 0x7f03041986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.792+0000 7f0308c25640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0304107270 0x7f0304198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.792+0000 7f0308c25640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f030419cf60 0x7f030419d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.793+0000 7f0308c25640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f03041121b0 con 0x7f030419cf60 2024-04-22T13:29:42.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.793+0000 7f0308c25640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0304112030 con 0x7f0304107270 2024-04-22T13:29:42.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.793+0000 7f0308c25640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0304112330 con 0x7f030406a840 2024-04-22T13:29:42.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.793+0000 7f0302d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f030419cf60 0x7f030419d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.793+0000 7f0302d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f030419cf60 0x7f030419d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43344/0 (socket says 172.21.15.12:43344) 2024-04-22T13:29:42.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.793+0000 7f0302d9d640 1 -- 172.21.15.12:0/1731236982 learned_addr learned my addr 172.21.15.12:0/1731236982 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:42.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.793+0000 7f030259c640 1 --2- 172.21.15.12:0/1731236982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f030406a840 0x7f03041986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.793+0000 7f0301d9b640 1 --2- 172.21.15.12:0/1731236982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0304107270 0x7f0304198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.793+0000 7f0302d9d640 1 -- 172.21.15.12:0/1731236982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f030406a840 msgr2=0x7f03041986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.793+0000 7f0302d9d640 1 --2- 172.21.15.12:0/1731236982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f030406a840 0x7f03041986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.793+0000 7f0302d9d640 1 -- 172.21.15.12:0/1731236982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0304107270 msgr2=0x7f0304198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.793+0000 7f0302d9d640 1 --2- 172.21.15.12:0/1731236982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0304107270 0x7f0304198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.793+0000 7f0302d9d640 1 -- 172.21.15.12:0/1731236982 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030419da50 con 0x7f030419cf60 2024-04-22T13:29:42.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.793+0000 7f0301d9b640 1 --2- 172.21.15.12:0/1731236982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0304107270 0x7f0304198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:42.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.793+0000 7f0302d9d640 1 --2- 172.21.15.12:0/1731236982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f030419cf60 0x7f030419d410 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f02fc00cce0 tx=0x7f02fc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:42.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.793+0000 7f02f37fe640 1 -- 172.21.15.12:0/1731236982 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc00f450 con 0x7f030419cf60 2024-04-22T13:29:42.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.793+0000 7f02f37fe640 1 -- 172.21.15.12:0/1731236982 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02fc00faa0 con 0x7f030419cf60 2024-04-22T13:29:42.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.793+0000 7f0308c25640 1 -- 172.21.15.12:0/1731236982 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f030419dd40 con 0x7f030419cf60 2024-04-22T13:29:42.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.793+0000 7f02f37fe640 1 -- 172.21.15.12:0/1731236982 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc018b70 con 0x7f030419cf60 2024-04-22T13:29:42.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.793+0000 7f0308c25640 1 -- 172.21.15.12:0/1731236982 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03041a5530 con 0x7f030419cf60 2024-04-22T13:29:42.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.796+0000 7f02f37fe640 1 -- 172.21.15.12:0/1731236982 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f02fc018d10 con 0x7f030419cf60 2024-04-22T13:29:42.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.796+0000 7f030b811740 1 Processor -- start 2024-04-22T13:29:42.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.797+0000 7f030b811740 1 -- start start 2024-04-22T13:29:42.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.797+0000 7f030b811740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccac2044d0 0x55ccac2048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.797+0000 7f030b811740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccac204e80 0x55ccac20d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.797+0000 7f030b811740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ccac20da00 0x55ccac20fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.797+0000 7f030b811740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ccac1691f0 con 0x55ccac204e80 2024-04-22T13:29:42.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.797+0000 7f030b811740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ccac1064b0 con 0x55ccac20da00 2024-04-22T13:29:42.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.797+0000 7f030b811740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ccac1651b0 con 0x55ccac2044d0 2024-04-22T13:29:42.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.797+0000 7f030259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccac204e80 0x55ccac20d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.797+0000 7f030259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccac204e80 0x55ccac20d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43352/0 (socket says 172.21.15.12:43352) 2024-04-22T13:29:42.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.797+0000 7f030259c640 1 -- 172.21.15.12:0/2574065663 learned_addr learned my addr 172.21.15.12:0/2574065663 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:42.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.797+0000 7f0301d9b640 1 --2- 172.21.15.12:0/2574065663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccac2044d0 0x55ccac2048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.797+0000 7f0301d9b640 1 --2- 172.21.15.12:0/2574065663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ccac20da00 0x55ccac20fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.797+0000 7f02f37fe640 1 --2- 172.21.15.12:0/1731236982 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f02ec0a3da0 0x7f02ec0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.797+0000 7f030259c640 1 -- 172.21.15.12:0/2574065663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccac2044d0 msgr2=0x55ccac2048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:42.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.797+0000 7f030259c640 1 --2- 172.21.15.12:0/2574065663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccac2044d0 0x55ccac2048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.797+0000 7f030259c640 1 -- 172.21.15.12:0/2574065663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ccac20da00 msgr2=0x55ccac20fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:42.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.797+0000 7f030259c640 1 --2- 172.21.15.12:0/2574065663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ccac20da00 0x55ccac20fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.797+0000 7f030259c640 1 -- 172.21.15.12:0/2574065663 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccac0f81b0 con 0x55ccac204e80 2024-04-22T13:29:42.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.797+0000 7f0301d9b640 1 --2- 172.21.15.12:0/2574065663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccac2044d0 0x55ccac2048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:29:42.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.797+0000 7f0302d9d640 1 --2- 172.21.15.12:0/1731236982 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f02ec0a3da0 0x7f02ec0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.797+0000 7f02f37fe640 1 -- 172.21.15.12:0/1731236982 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f02fc0d0b20 con 0x7f030419cf60 2024-04-22T13:29:42.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.798+0000 7f030259c640 1 --2- 172.21.15.12:0/2574065663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccac204e80 0x55ccac20d430 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f02f4009900 tx=0x7f02f4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76db8fc3b60a9cd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:42.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.798+0000 7f02f0ff9640 1 -- 172.21.15.12:0/2574065663 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f4045020 con 0x55ccac204e80 2024-04-22T13:29:42.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.798+0000 7f02f0ff9640 1 -- 172.21.15.12:0/2574065663 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02f402cc90 con 0x55ccac204e80 2024-04-22T13:29:42.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.798+0000 7f02f0ff9640 1 -- 172.21.15.12:0/2574065663 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f40046c0 con 0x55ccac204e80 2024-04-22T13:29:42.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.798+0000 7f030b811740 1 -- 172.21.15.12:0/2574065663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccac204e80 msgr2=0x55ccac20d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.798+0000 7f0302d9d640 1 --2- 172.21.15.12:0/1731236982 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f02ec0a3da0 0x7f02ec0a6260 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f02fc0ce370 tx=0x7f02fc06b4c0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:42.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.798+0000 7f030b811740 1 --2- 172.21.15.12:0/2574065663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccac204e80 0x55ccac20d430 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f02f4009900 tx=0x7f02f4004040 comp rx=0 tx=0).stop 2024-04-22T13:29:42.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.799+0000 7f030b811740 1 -- 172.21.15.12:0/2574065663 shutdown_connections 2024-04-22T13:29:42.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.799+0000 7f030b811740 1 --2- 172.21.15.12:0/2574065663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ccac20da00 0x55ccac20fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.799+0000 7f030b811740 1 --2- 172.21.15.12:0/2574065663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccac204e80 0x55ccac20d430 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.799+0000 7f030b811740 1 --2- 172.21.15.12:0/2574065663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccac2044d0 0x55ccac2048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.799+0000 7f030b811740 1 -- 172.21.15.12:0/2574065663 >> 172.21.15.12:0/2574065663 conn(0x55ccac1feba0 msgr2=0x55ccac1fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:42.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.799+0000 7f030b811740 1 -- 172.21.15.12:0/2574065663 shutdown_connections 2024-04-22T13:29:42.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.799+0000 7f030b811740 1 -- 172.21.15.12:0/2574065663 wait complete. 2024-04-22T13:29:42.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.799+0000 7f030b811740 1 -- start start 2024-04-22T13:29:42.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.799+0000 7f030b811740 1 Processor -- start 2024-04-22T13:29:42.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.799+0000 7f030b811740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccac2044d0 0x55ccac20d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.799+0000 7f030b811740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ccac204e80 0x55ccac20a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.799+0000 7f030b811740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccac20da00 0x55ccac20abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.799+0000 7f030b811740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ccac210910 con 0x55ccac2044d0 2024-04-22T13:29:42.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.799+0000 7f030b811740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ccac122500 con 0x55ccac204e80 2024-04-22T13:29:42.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.799+0000 7f030b811740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ccac210a90 con 0x55ccac20da00 2024-04-22T13:29:42.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.799+0000 7f030b811740 10 client.0 _ll_register_callbacks cb 0x55ccac0f0b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:42.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.799+0000 7f0301d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccac2044d0 0x55ccac20d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.799+0000 7f0301d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccac2044d0 0x55ccac20d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43358/0 (socket says 172.21.15.12:43358) 2024-04-22T13:29:42.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.799+0000 7f0301d9b640 1 -- 172.21.15.12:0/233334600 learned_addr learned my addr 172.21.15.12:0/233334600 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:42.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.800+0000 7f030259c640 1 --2- 172.21.15.12:0/233334600 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ccac204e80 0x55ccac20a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.800+0000 7f0301d9b640 1 --2- 172.21.15.12:0/233334600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccac20da00 0x55ccac20abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.800+0000 7f0301d9b640 1 -- 172.21.15.12:0/233334600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccac20da00 msgr2=0x55ccac20abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:42.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.800+0000 7f0301d9b640 1 --2- 172.21.15.12:0/233334600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccac20da00 0x55ccac20abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.800+0000 7f0301d9b640 1 -- 172.21.15.12:0/233334600 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ccac204e80 msgr2=0x55ccac20a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:42.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.800+0000 7f0301d9b640 1 --2- 172.21.15.12:0/233334600 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ccac204e80 0x55ccac20a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.800+0000 7f0301d9b640 1 -- 172.21.15.12:0/233334600 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccabe7b3f0 con 0x55ccac2044d0 2024-04-22T13:29:42.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.800+0000 7f0301d9b640 1 --2- 172.21.15.12:0/233334600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccac2044d0 0x55ccac20d2e0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f02f8015f70 tx=0x7f02f8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486433b2b36c99f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:42.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.800+0000 7f02e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:42.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.800+0000 7f02e9ffb640 1 -- 172.21.15.12:0/233334600 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f8004030 con 0x55ccac2044d0 2024-04-22T13:29:42.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.800+0000 7f02e9ffb640 1 -- 172.21.15.12:0/233334600 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f02f80041d0 con 0x55ccac2044d0 2024-04-22T13:29:42.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.800+0000 7f030b811740 1 -- 172.21.15.12:0/233334600 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02f402cc90 con 0x55ccac2044d0 2024-04-22T13:29:42.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.800+0000 7f02e9ffb640 1 -- 172.21.15.12:0/233334600 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f02f80043f0 con 0x55ccac2044d0 2024-04-22T13:29:42.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.800+0000 7f030b811740 10 client.17775 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:42.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.800+0000 7f030b811740 1 -- 172.21.15.12:0/233334600 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ccabf457d0 con 0x55ccac2044d0 2024-04-22T13:29:42.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.800+0000 7f02e9ffb640 1 -- 172.21.15.12:0/233334600 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02f402cc90 con 0x55ccac2044d0 2024-04-22T13:29:42.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.801+0000 7f02e9ffb640 1 -- 172.21.15.12:0/233334600 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f02f8003b60 con 0x55ccac2044d0 2024-04-22T13:29:42.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.801+0000 7f030b811740 10 client.17775 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:42.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.801+0000 7f030b811740 10 client.17775 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:29:42.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.801+0000 7f030b811740 20 client.17775 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:42.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.801+0000 7f030b811740 1 --2- 172.21.15.12:0/233334600 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55ccac21dbc0 0x55ccac21ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.801+0000 7f030b811740 4 client.17775 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:42.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.801+0000 7f030b811740 1 -- 172.21.15.12:0/233334600 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccac108930 con 0x55ccac21dbc0 2024-04-22T13:29:42.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.801+0000 7f030259c640 1 --2- 172.21.15.12:0/233334600 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55ccac21dbc0 0x55ccac21ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.801+0000 7f02e9ffb640 10 client.17775 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:29:42.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.801+0000 7f030259c640 1 --2- 172.21.15.12:0/233334600 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55ccac21dbc0 0x55ccac21ffb0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=42a4c0f7925fb900 server_cookie=b0264e820541f974 in_seq=0 out_seq=0 2024-04-22T13:29:42.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.802+0000 7f02e9ffb640 1 -- 172.21.15.12:0/233334600 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ccac108930 con 0x55ccac21dbc0 2024-04-22T13:29:42.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.802+0000 7f02e9ffb640 10 client.17775 handle_command_reply: tid=0 2024-04-22T13:29:42.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.802+0000 7f030b811740 1 client.17775 shutdown 2024-04-22T13:29:42.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.803+0000 7f030b811740 1 -- 172.21.15.12:0/233334600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccac2044d0 msgr2=0x55ccac20d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.803+0000 7f030b811740 1 --2- 172.21.15.12:0/233334600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccac2044d0 0x55ccac20d2e0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f02f8015f70 tx=0x7f02f8005850 comp rx=0 tx=0).stop 2024-04-22T13:29:42.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.803+0000 7f030b811740 1 -- 172.21.15.12:0/233334600 shutdown_connections 2024-04-22T13:29:42.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.803+0000 7f030b811740 1 --2- 172.21.15.12:0/233334600 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55ccac21dbc0 0x55ccac21ffb0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.803+0000 7f030b811740 1 --2- 172.21.15.12:0/233334600 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccac20da00 0x55ccac20abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.803+0000 7f030b811740 1 --2- 172.21.15.12:0/233334600 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ccac204e80 0x55ccac20a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.803+0000 7f02e9ffb640 0 client.17775 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:29:42.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.803+0000 7f030b811740 1 --2- 172.21.15.12:0/233334600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccac2044d0 0x55ccac20d2e0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.803+0000 7f030b811740 1 -- 172.21.15.12:0/233334600 >> 172.21.15.12:0/233334600 conn(0x55ccac1feba0 msgr2=0x55ccac2008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:42.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.803+0000 7f030b811740 1 -- 172.21.15.12:0/233334600 shutdown_connections 2024-04-22T13:29:42.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.803+0000 7f030b811740 1 -- 172.21.15.12:0/233334600 wait complete. 2024-04-22T13:29:42.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.803+0000 7f030b811740 20 client.17775 trim_cache size 0 max 16384 2024-04-22T13:29:42.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.816+0000 7f030b811740 1 Processor -- start 2024-04-22T13:29:42.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.816+0000 7f030b811740 1 -- start start 2024-04-22T13:29:42.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.816+0000 7f030b811740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccac2044d0 0x55ccac2197d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.816+0000 7f030b811740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccac204e80 0x55ccac20cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.816+0000 7f030b811740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ccac20da00 0x55ccac20d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.816+0000 7f030b811740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f03041ad950 con 0x55ccac2044d0 2024-04-22T13:29:42.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.816+0000 7f030b811740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f03041af490 con 0x55ccac20da00 2024-04-22T13:29:42.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.816+0000 7f030b811740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f03041ae020 con 0x55ccac204e80 2024-04-22T13:29:42.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.816+0000 7f0301d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccac2044d0 0x55ccac2197d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.816+0000 7f0301d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccac2044d0 0x55ccac2197d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43374/0 (socket says 172.21.15.12:43374) 2024-04-22T13:29:42.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.816+0000 7f0301d9b640 1 -- 172.21.15.12:0/3890813671 learned_addr learned my addr 172.21.15.12:0/3890813671 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:42.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.816+0000 7f030259c640 1 --2- 172.21.15.12:0/3890813671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccac204e80 0x55ccac20cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.817+0000 7f0301d9b640 1 -- 172.21.15.12:0/3890813671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccac204e80 msgr2=0x55ccac20cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:42.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.817+0000 7f0301d9b640 1 --2- 172.21.15.12:0/3890813671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccac204e80 0x55ccac20cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.817+0000 7f0301d9b640 1 -- 172.21.15.12:0/3890813671 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ccac20da00 msgr2=0x55ccac20d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:42.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.817+0000 7f0301d9b640 1 --2- 172.21.15.12:0/3890813671 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ccac20da00 0x55ccac20d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.817+0000 7f0301d9b640 1 -- 172.21.15.12:0/3890813671 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccac220540 con 0x55ccac2044d0 2024-04-22T13:29:42.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.817+0000 7f030259c640 1 --2- 172.21.15.12:0/3890813671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccac204e80 0x55ccac20cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:42.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.817+0000 7f0301d9b640 1 --2- 172.21.15.12:0/3890813671 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccac2044d0 0x55ccac2197d0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f02f8001eb0 tx=0x7f02f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a664e5bcbabc95db server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:42.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.817+0000 7f02ebfff640 1 -- 172.21.15.12:0/3890813671 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f8030070 con 0x55ccac2044d0 2024-04-22T13:29:42.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.817+0000 7f02ebfff640 1 -- 172.21.15.12:0/3890813671 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02f800c3d0 con 0x55ccac2044d0 2024-04-22T13:29:42.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.817+0000 7f02ebfff640 1 -- 172.21.15.12:0/3890813671 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f8003d00 con 0x55ccac2044d0 2024-04-22T13:29:42.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.817+0000 7f030b811740 1 -- 172.21.15.12:0/3890813671 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccac2044d0 msgr2=0x55ccac2197d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.817+0000 7f030b811740 1 --2- 172.21.15.12:0/3890813671 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccac2044d0 0x55ccac2197d0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f02f8001eb0 tx=0x7f02f80046c0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.817+0000 7f030b811740 1 -- 172.21.15.12:0/3890813671 shutdown_connections 2024-04-22T13:29:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.817+0000 7f030b811740 1 --2- 172.21.15.12:0/3890813671 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ccac20da00 0x55ccac20d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.817+0000 7f030b811740 1 --2- 172.21.15.12:0/3890813671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccac204e80 0x55ccac20cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.817+0000 7f030b811740 1 --2- 172.21.15.12:0/3890813671 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccac2044d0 0x55ccac2197d0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.817+0000 7f030b811740 1 -- 172.21.15.12:0/3890813671 >> 172.21.15.12:0/3890813671 conn(0x55ccac1feba0 msgr2=0x55ccac1fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.817+0000 7f030b811740 1 -- 172.21.15.12:0/3890813671 shutdown_connections 2024-04-22T13:29:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.818+0000 7f030b811740 1 -- 172.21.15.12:0/3890813671 wait complete. 2024-04-22T13:29:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.818+0000 7f030b811740 1 -- start start 2024-04-22T13:29:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.818+0000 7f030b811740 1 Processor -- start 2024-04-22T13:29:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.818+0000 7f030b811740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccac2044d0 0x55ccac211dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.818+0000 7f030b811740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ccac204e80 0x55ccac212310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.818+0000 7f030b811740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccac20da00 0x55ccac282fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.818+0000 7f030b811740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f03041aedc0 con 0x55ccac2044d0 2024-04-22T13:29:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.818+0000 7f030b811740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f03041ae6f0 con 0x55ccac204e80 2024-04-22T13:29:42.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.818+0000 7f030b811740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f03041ab950 con 0x55ccac20da00 2024-04-22T13:29:42.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.818+0000 7f030b811740 10 client.0 _ll_register_callbacks cb 0x7f02f8015bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:42.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.818+0000 7f0301d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccac2044d0 0x55ccac211dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.818+0000 7f0301d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccac2044d0 0x55ccac211dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43380/0 (socket says 172.21.15.12:43380) 2024-04-22T13:29:42.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.818+0000 7f0301d9b640 1 -- 172.21.15.12:0/2265165773 learned_addr learned my addr 172.21.15.12:0/2265165773 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:42.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.818+0000 7f030259c640 1 --2- 172.21.15.12:0/2265165773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ccac204e80 0x55ccac212310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.818+0000 7f0301d9b640 1 -- 172.21.15.12:0/2265165773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccac20da00 msgr2=0x55ccac282fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:42.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.818+0000 7f0301d9b640 1 --2- 172.21.15.12:0/2265165773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccac20da00 0x55ccac282fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.818+0000 7f0301d9b640 1 -- 172.21.15.12:0/2265165773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ccac204e80 msgr2=0x55ccac212310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:42.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.818+0000 7f0301d9b640 1 --2- 172.21.15.12:0/2265165773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ccac204e80 0x55ccac212310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.818+0000 7f0301d9b640 1 -- 172.21.15.12:0/2265165773 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccac283520 con 0x55ccac2044d0 2024-04-22T13:29:42.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.819+0000 7f0301d9b640 1 --2- 172.21.15.12:0/2265165773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccac2044d0 0x55ccac211dd0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f02f8005fb0 tx=0x7f02f8011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6b66c54aa49089 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:42.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.819+0000 7f02e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:42.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.819+0000 7f02e9ffb640 1 -- 172.21.15.12:0/2265165773 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f8030040 con 0x55ccac2044d0 2024-04-22T13:29:42.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.819+0000 7f02e9ffb640 1 -- 172.21.15.12:0/2265165773 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f02f80035f0 con 0x55ccac2044d0 2024-04-22T13:29:42.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.819+0000 7f030b811740 1 -- 172.21.15.12:0/2265165773 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02f800c3d0 con 0x55ccac2044d0 2024-04-22T13:29:42.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.819+0000 7f02e9ffb640 1 -- 172.21.15.12:0/2265165773 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f02f803a020 con 0x55ccac2044d0 2024-04-22T13:29:42.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.819+0000 7f030b811740 10 client.17781 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:42.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.819+0000 7f030b811740 1 -- 172.21.15.12:0/2265165773 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ccac2847c0 con 0x55ccac2044d0 2024-04-22T13:29:42.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.819+0000 7f02e9ffb640 1 -- 172.21.15.12:0/2265165773 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02f800c3d0 con 0x55ccac2044d0 2024-04-22T13:29:42.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.819+0000 7f02e9ffb640 1 -- 172.21.15.12:0/2265165773 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f02f80128a0 con 0x55ccac2044d0 2024-04-22T13:29:42.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.819+0000 7f030b811740 10 client.17781 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:42.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.819+0000 7f030b811740 10 client.17781 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:29:42.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.819+0000 7f030b811740 20 client.17781 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:42.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.819+0000 7f030b811740 1 --2- 172.21.15.12:0/2265165773 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55ccac284e20 0x55ccac287240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.819+0000 7f030b811740 4 client.17781 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:29:42.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.819+0000 7f030b811740 1 -- 172.21.15.12:0/2265165773 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ccac176c10 con 0x55ccac284e20 2024-04-22T13:29:42.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.820+0000 7f030259c640 1 --2- 172.21.15.12:0/2265165773 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55ccac284e20 0x55ccac287240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.820+0000 7f030259c640 1 --2- 172.21.15.12:0/2265165773 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55ccac284e20 0x55ccac287240 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2404e7c26c03ec52 server_cookie=b41e17187686fc87 in_seq=0 out_seq=0 2024-04-22T13:29:42.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.820+0000 7f02e9ffb640 10 client.17781 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:29:42.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.820+0000 7f02e9ffb640 1 -- 172.21.15.12:0/2265165773 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ccac176c10 con 0x55ccac284e20 2024-04-22T13:29:42.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.820+0000 7f02e9ffb640 10 client.17781 handle_command_reply: tid=0 2024-04-22T13:29:42.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.820+0000 7f030b811740 1 client.17781 shutdown 2024-04-22T13:29:42.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.820+0000 7f030b811740 1 -- 172.21.15.12:0/2265165773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccac2044d0 msgr2=0x55ccac211dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.820+0000 7f030b811740 1 --2- 172.21.15.12:0/2265165773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccac2044d0 0x55ccac211dd0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f02f8005fb0 tx=0x7f02f8011d60 comp rx=0 tx=0).stop 2024-04-22T13:29:42.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.820+0000 7f030b811740 1 -- 172.21.15.12:0/2265165773 shutdown_connections 2024-04-22T13:29:42.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.820+0000 7f030b811740 1 --2- 172.21.15.12:0/2265165773 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55ccac284e20 0x55ccac287240 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.820+0000 7f030b811740 1 --2- 172.21.15.12:0/2265165773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ccac20da00 0x55ccac282fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.820+0000 7f030b811740 1 --2- 172.21.15.12:0/2265165773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ccac204e80 0x55ccac212310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.820+0000 7f02e9ffb640 0 client.17781 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:29:42.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.820+0000 7f030b811740 1 --2- 172.21.15.12:0/2265165773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ccac2044d0 0x55ccac211dd0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.820+0000 7f030b811740 1 -- 172.21.15.12:0/2265165773 >> 172.21.15.12:0/2265165773 conn(0x55ccac1feba0 msgr2=0x55ccac1fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:42.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.820+0000 7f030b811740 1 -- 172.21.15.12:0/2265165773 shutdown_connections 2024-04-22T13:29:42.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.821+0000 7f030b811740 1 -- 172.21.15.12:0/2265165773 wait complete. 2024-04-22T13:29:42.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.821+0000 7f030b811740 20 client.17781 trim_cache size 0 max 16384 2024-04-22T13:29:42.823 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:29:42.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.821+0000 7f0308c25640 1 -- 172.21.15.12:0/1731236982 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f02ec0a3da0 msgr2=0x7f02ec0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.821+0000 7f0308c25640 1 --2- 172.21.15.12:0/1731236982 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f02ec0a3da0 0x7f02ec0a6260 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f02fc0ce370 tx=0x7f02fc06b4c0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.821+0000 7f0308c25640 1 -- 172.21.15.12:0/1731236982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f030419cf60 msgr2=0x7f030419d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.821+0000 7f0308c25640 1 --2- 172.21.15.12:0/1731236982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f030419cf60 0x7f030419d410 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f02fc00cce0 tx=0x7f02fc00a620 comp rx=0 tx=0).stop 2024-04-22T13:29:42.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.821+0000 7f0308c25640 1 -- 172.21.15.12:0/1731236982 shutdown_connections 2024-04-22T13:29:42.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.821+0000 7f0308c25640 1 --2- 172.21.15.12:0/1731236982 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f02ec0a3da0 0x7f02ec0a6260 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.821+0000 7f0308c25640 1 --2- 172.21.15.12:0/1731236982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f030419cf60 0x7f030419d410 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.821+0000 7f0308c25640 1 --2- 172.21.15.12:0/1731236982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0304107270 0x7f0304198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.821+0000 7f0308c25640 1 --2- 172.21.15.12:0/1731236982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f030406a840 0x7f03041986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.821+0000 7f0308c25640 1 -- 172.21.15.12:0/1731236982 >> 172.21.15.12:0/1731236982 conn(0x7f0304077eb0 msgr2=0x7f0304100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:42.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.821+0000 7f0308c25640 1 -- 172.21.15.12:0/1731236982 shutdown_connections 2024-04-22T13:29:42.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.821+0000 7f0308c25640 1 -- 172.21.15.12:0/1731236982 wait complete. 2024-04-22T13:29:42.828 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:29:42.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.936+0000 7fe9a96f2640 1 -- 172.21.15.12:0/2456389423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9a41010a0 msgr2=0x7fe9a4101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.936+0000 7fe9a96f2640 1 --2- 172.21.15.12:0/2456389423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9a41010a0 0x7fe9a4101480 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fe994009900 tx=0x7fe99402bc30 comp rx=0 tx=0).stop 2024-04-22T13:29:42.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.936+0000 7fe9a96f2640 1 -- 172.21.15.12:0/2456389423 shutdown_connections 2024-04-22T13:29:42.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.936+0000 7fe9a96f2640 1 --2- 172.21.15.12:0/2456389423 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9a4106170 0x7fe9a410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.936+0000 7fe9a96f2640 1 --2- 172.21.15.12:0/2456389423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9a4101a50 0x7fe9a4105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.936+0000 7fe9a96f2640 1 --2- 172.21.15.12:0/2456389423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9a41010a0 0x7fe9a4101480 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.936+0000 7fe9a96f2640 1 -- 172.21.15.12:0/2456389423 >> 172.21.15.12:0/2456389423 conn(0x7fe9a4077eb0 msgr2=0x7fe9a40782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:42.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.936+0000 7fe9a96f2640 1 -- 172.21.15.12:0/2456389423 shutdown_connections 2024-04-22T13:29:42.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.936+0000 7fe9a96f2640 1 -- 172.21.15.12:0/2456389423 wait complete. 2024-04-22T13:29:42.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.937+0000 7fe9a96f2640 1 Processor -- start 2024-04-22T13:29:42.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.937+0000 7fe9a96f2640 1 -- start start 2024-04-22T13:29:42.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.937+0000 7fe9a96f2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9a41010a0 0x7fe9a4198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.937+0000 7fe9a96f2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9a4106170 0x7fe9a4198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.937+0000 7fe9a96f2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9a419cf90 0x7fe9a419d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.937+0000 7fe9a96f2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a41110b0 con 0x7fe9a419cf90 2024-04-22T13:29:42.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.937+0000 7fe9a96f2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a4110f30 con 0x7fe9a4106170 2024-04-22T13:29:42.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.937+0000 7fe9a96f2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a4111230 con 0x7fe9a41010a0 2024-04-22T13:29:42.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.938+0000 7fe9a37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9a419cf90 0x7fe9a419d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.938+0000 7fe9a37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9a419cf90 0x7fe9a419d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43394/0 (socket says 172.21.15.12:43394) 2024-04-22T13:29:42.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.938+0000 7fe9a37fe640 1 -- 172.21.15.12:0/2390470378 learned_addr learned my addr 172.21.15.12:0/2390470378 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:42.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.938+0000 7fe9a2ffd640 1 --2- 172.21.15.12:0/2390470378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9a41010a0 0x7fe9a4198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.938+0000 7fe9a27fc640 1 --2- 172.21.15.12:0/2390470378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9a4106170 0x7fe9a4198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.938+0000 7fe9a37fe640 1 -- 172.21.15.12:0/2390470378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9a41010a0 msgr2=0x7fe9a4198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.938+0000 7fe9a37fe640 1 --2- 172.21.15.12:0/2390470378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9a41010a0 0x7fe9a4198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.938+0000 7fe9a37fe640 1 -- 172.21.15.12:0/2390470378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9a4106170 msgr2=0x7fe9a4198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.938+0000 7fe9a37fe640 1 --2- 172.21.15.12:0/2390470378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9a4106170 0x7fe9a4198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.938+0000 7fe9a37fe640 1 -- 172.21.15.12:0/2390470378 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a419da80 con 0x7fe9a419cf90 2024-04-22T13:29:42.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.938+0000 7fe9a2ffd640 1 --2- 172.21.15.12:0/2390470378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9a41010a0 0x7fe9a4198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:42.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.938+0000 7fe9a27fc640 1 --2- 172.21.15.12:0/2390470378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9a4106170 0x7fe9a4198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:42.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.938+0000 7fe9a37fe640 1 --2- 172.21.15.12:0/2390470378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9a419cf90 0x7fe9a419d440 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009fd0 tx=0x7fe99c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:42.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.938+0000 7fe98bfff640 1 -- 172.21.15.12:0/2390470378 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c017070 con 0x7fe9a419cf90 2024-04-22T13:29:42.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.938+0000 7fe9a96f2640 1 -- 172.21.15.12:0/2390470378 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a419dd70 con 0x7fe9a419cf90 2024-04-22T13:29:42.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.938+0000 7fe98bfff640 1 -- 172.21.15.12:0/2390470378 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe99c003e20 con 0x7fe9a419cf90 2024-04-22T13:29:42.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.938+0000 7fe98bfff640 1 -- 172.21.15.12:0/2390470378 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c010460 con 0x7fe9a419cf90 2024-04-22T13:29:42.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.938+0000 7fe9a96f2640 1 -- 172.21.15.12:0/2390470378 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a41a5510 con 0x7fe9a419cf90 2024-04-22T13:29:42.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.941+0000 7fe9ac3be740 1 Processor -- start 2024-04-22T13:29:42.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.941+0000 7fe98bfff640 1 -- 172.21.15.12:0/2390470378 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe99c003970 con 0x7fe9a419cf90 2024-04-22T13:29:42.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.941+0000 7fe9ac3be740 1 -- start start 2024-04-22T13:29:42.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.941+0000 7fe9ac3be740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560b273e94d0 0x560b273e98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.941+0000 7fe9ac3be740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560b273e9e80 0x560b273f2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.941+0000 7fe9ac3be740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b273f2a00 0x560b273f4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.941+0000 7fe9ac3be740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560b2734e1f0 con 0x560b273e94d0 2024-04-22T13:29:42.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.941+0000 7fe9ac3be740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560b272eb4b0 con 0x560b273e9e80 2024-04-22T13:29:42.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.941+0000 7fe9ac3be740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560b2734a1b0 con 0x560b273f2a00 2024-04-22T13:29:42.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.941+0000 7fe9a27fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560b273e94d0 0x560b273e98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.941+0000 7fe9a27fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560b273e94d0 0x560b273e98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43408/0 (socket says 172.21.15.12:43408) 2024-04-22T13:29:42.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.941+0000 7fe9a27fc640 1 -- 172.21.15.12:0/2444650953 learned_addr learned my addr 172.21.15.12:0/2444650953 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:42.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.941+0000 7fe9a2ffd640 1 --2- 172.21.15.12:0/2444650953 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560b273e9e80 0x560b273f2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.941+0000 7fe9a27fc640 1 --2- 172.21.15.12:0/2444650953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b273f2a00 0x560b273f4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.941+0000 7fe98bfff640 1 --2- 172.21.15.12:0/2390470378 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe98c0a3d20 0x7fe98c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.941+0000 7fe9a27fc640 1 -- 172.21.15.12:0/2444650953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b273f2a00 msgr2=0x560b273f4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:42.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.941+0000 7fe9a27fc640 1 --2- 172.21.15.12:0/2444650953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b273f2a00 0x560b273f4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.941+0000 7fe9a27fc640 1 -- 172.21.15.12:0/2444650953 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560b273e9e80 msgr2=0x560b273f2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:42.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.941+0000 7fe9a27fc640 1 --2- 172.21.15.12:0/2444650953 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560b273e9e80 0x560b273f2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.941+0000 7fe9a27fc640 1 -- 172.21.15.12:0/2444650953 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b272dd1b0 con 0x560b273e94d0 2024-04-22T13:29:42.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.941+0000 7fe9a37fe640 1 --2- 172.21.15.12:0/2390470378 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe98c0a3d20 0x7fe98c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.942+0000 7fe98bfff640 1 -- 172.21.15.12:0/2390470378 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fe99c0d05a0 con 0x7fe9a419cf90 2024-04-22T13:29:42.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.942+0000 7fe9a27fc640 1 --2- 172.21.15.12:0/2444650953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560b273e94d0 0x560b273e98b0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fe998015520 tx=0x7fe9980095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e58212e3eb23ef server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:42.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.942+0000 7fe9897fa640 1 -- 172.21.15.12:0/2444650953 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9980150f0 con 0x560b273e94d0 2024-04-22T13:29:42.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.942+0000 7fe9897fa640 1 -- 172.21.15.12:0/2444650953 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe998005020 con 0x560b273e94d0 2024-04-22T13:29:42.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.942+0000 7fe9897fa640 1 -- 172.21.15.12:0/2444650953 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe998004a00 con 0x560b273e94d0 2024-04-22T13:29:42.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.942+0000 7fe9ac3be740 1 -- 172.21.15.12:0/2444650953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560b273e94d0 msgr2=0x560b273e98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.942+0000 7fe9a37fe640 1 --2- 172.21.15.12:0/2390470378 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe98c0a3d20 0x7fe98c0a61e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe99c0ce340 tx=0x7fe99c091a20 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:42.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.942+0000 7fe9ac3be740 1 --2- 172.21.15.12:0/2444650953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560b273e94d0 0x560b273e98b0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fe998015520 tx=0x7fe9980095d0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.943+0000 7fe9ac3be740 1 -- 172.21.15.12:0/2444650953 shutdown_connections 2024-04-22T13:29:42.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.943+0000 7fe9ac3be740 1 --2- 172.21.15.12:0/2444650953 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b273f2a00 0x560b273f4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.943+0000 7fe9ac3be740 1 --2- 172.21.15.12:0/2444650953 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560b273e9e80 0x560b273f2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.943+0000 7fe9ac3be740 1 --2- 172.21.15.12:0/2444650953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560b273e94d0 0x560b273e98b0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.943+0000 7fe9ac3be740 1 -- 172.21.15.12:0/2444650953 >> 172.21.15.12:0/2444650953 conn(0x560b273e3ba0 msgr2=0x560b273e3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:42.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.943+0000 7fe9ac3be740 1 -- 172.21.15.12:0/2444650953 shutdown_connections 2024-04-22T13:29:42.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.943+0000 7fe9ac3be740 1 -- 172.21.15.12:0/2444650953 wait complete. 2024-04-22T13:29:42.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.943+0000 7fe9ac3be740 1 -- start start 2024-04-22T13:29:42.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.943+0000 7fe9ac3be740 1 Processor -- start 2024-04-22T13:29:42.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.943+0000 7fe9ac3be740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560b273e94d0 0x560b273f2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.943+0000 7fe9ac3be740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560b273e9e80 0x560b273ef690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.943+0000 7fe9ac3be740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b273f2a00 0x560b273efc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.943+0000 7fe9ac3be740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560b273f5970 con 0x560b273e94d0 2024-04-22T13:29:42.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.943+0000 7fe9ac3be740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560b27307500 con 0x560b273e9e80 2024-04-22T13:29:42.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.943+0000 7fe9ac3be740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560b273f5af0 con 0x560b273f2a00 2024-04-22T13:29:42.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.943+0000 7fe9ac3be740 10 client.0 _ll_register_callbacks cb 0x560b272d5b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:42.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.943+0000 7fe9a27fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560b273e94d0 0x560b273f2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.943+0000 7fe9a27fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560b273e94d0 0x560b273f2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43410/0 (socket says 172.21.15.12:43410) 2024-04-22T13:29:42.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.943+0000 7fe9a27fc640 1 -- 172.21.15.12:0/1421376911 learned_addr learned my addr 172.21.15.12:0/1421376911 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:42.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.943+0000 7fe9a2ffd640 1 --2- 172.21.15.12:0/1421376911 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560b273e9e80 0x560b273ef690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.944+0000 7fe9a27fc640 1 --2- 172.21.15.12:0/1421376911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b273f2a00 0x560b273efc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.944+0000 7fe9a27fc640 1 -- 172.21.15.12:0/1421376911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b273f2a00 msgr2=0x560b273efc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:42.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.944+0000 7fe9a27fc640 1 --2- 172.21.15.12:0/1421376911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b273f2a00 0x560b273efc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.944+0000 7fe9a27fc640 1 -- 172.21.15.12:0/1421376911 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560b273e9e80 msgr2=0x560b273ef690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:42.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.944+0000 7fe9a27fc640 1 --2- 172.21.15.12:0/1421376911 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560b273e9e80 0x560b273ef690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.944+0000 7fe9a27fc640 1 -- 172.21.15.12:0/1421376911 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b270603f0 con 0x560b273e94d0 2024-04-22T13:29:42.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.944+0000 7fe9a27fc640 1 --2- 172.21.15.12:0/1421376911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560b273e94d0 0x560b273f2300 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fe998015e70 tx=0x7fe998009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e6d0b7bb89ea1d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:42.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.944+0000 7fe986ff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:42.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.944+0000 7fe986ff5640 1 -- 172.21.15.12:0/1421376911 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9980051d0 con 0x560b273e94d0 2024-04-22T13:29:42.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.944+0000 7fe9ac3be740 1 -- 172.21.15.12:0/1421376911 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe998005020 con 0x560b273e94d0 2024-04-22T13:29:42.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.944+0000 7fe986ff5640 1 -- 172.21.15.12:0/1421376911 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fe998005370 con 0x560b273e94d0 2024-04-22T13:29:42.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.944+0000 7fe986ff5640 1 -- 172.21.15.12:0/1421376911 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe998005570 con 0x560b273e94d0 2024-04-22T13:29:42.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.944+0000 7fe9ac3be740 10 client.17793 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:42.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.944+0000 7fe9ac3be740 1 -- 172.21.15.12:0/1421376911 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560b2712a7d0 con 0x560b273e94d0 2024-04-22T13:29:42.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.944+0000 7fe986ff5640 1 -- 172.21.15.12:0/1421376911 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe998005020 con 0x560b273e94d0 2024-04-22T13:29:42.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.945+0000 7fe986ff5640 1 -- 172.21.15.12:0/1421376911 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe99801e070 con 0x560b273e94d0 2024-04-22T13:29:42.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.945+0000 7fe9ac3be740 10 client.17793 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:42.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.945+0000 7fe9ac3be740 10 client.17793 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:29:42.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.945+0000 7fe9ac3be740 20 client.17793 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:42.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.945+0000 7fe9ac3be740 1 --2- 172.21.15.12:0/1421376911 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x560b27402a80 0x560b27404e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.945+0000 7fe9ac3be740 4 client.17793 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:42.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.945+0000 7fe9ac3be740 1 -- 172.21.15.12:0/1421376911 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b272ed930 con 0x560b27402a80 2024-04-22T13:29:42.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.945+0000 7fe9a2ffd640 1 --2- 172.21.15.12:0/1421376911 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x560b27402a80 0x560b27404e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.946+0000 7fe986ff5640 10 client.17793 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:29:42.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.946+0000 7fe9a2ffd640 1 --2- 172.21.15.12:0/1421376911 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x560b27402a80 0x560b27404e70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=afdc37cb863804e7 server_cookie=14d685574c3ea8d2 in_seq=0 out_seq=0 2024-04-22T13:29:42.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.946+0000 7fe986ff5640 1 -- 172.21.15.12:0/1421376911 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560b272ed930 con 0x560b27402a80 2024-04-22T13:29:42.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.946+0000 7fe986ff5640 10 client.17793 handle_command_reply: tid=0 2024-04-22T13:29:42.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.946+0000 7fe9ac3be740 1 client.17793 shutdown 2024-04-22T13:29:42.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.946+0000 7fe9ac3be740 1 -- 172.21.15.12:0/1421376911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560b273e94d0 msgr2=0x560b273f2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.946+0000 7fe9ac3be740 1 --2- 172.21.15.12:0/1421376911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560b273e94d0 0x560b273f2300 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fe998015e70 tx=0x7fe998009d00 comp rx=0 tx=0).stop 2024-04-22T13:29:42.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.946+0000 7fe9ac3be740 1 -- 172.21.15.12:0/1421376911 shutdown_connections 2024-04-22T13:29:42.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.946+0000 7fe9ac3be740 1 --2- 172.21.15.12:0/1421376911 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x560b27402a80 0x560b27404e70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.946+0000 7fe9ac3be740 1 --2- 172.21.15.12:0/1421376911 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b273f2a00 0x560b273efc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.946+0000 7fe9ac3be740 1 --2- 172.21.15.12:0/1421376911 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560b273e9e80 0x560b273ef690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.946+0000 7fe986ff5640 0 client.17793 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:29:42.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.946+0000 7fe9ac3be740 1 --2- 172.21.15.12:0/1421376911 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560b273e94d0 0x560b273f2300 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.946+0000 7fe9ac3be740 1 -- 172.21.15.12:0/1421376911 >> 172.21.15.12:0/1421376911 conn(0x560b273e3ba0 msgr2=0x560b273e58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:42.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.947+0000 7fe9ac3be740 1 -- 172.21.15.12:0/1421376911 shutdown_connections 2024-04-22T13:29:42.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.947+0000 7fe9ac3be740 1 -- 172.21.15.12:0/1421376911 wait complete. 2024-04-22T13:29:42.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.947+0000 7fe9ac3be740 20 client.17793 trim_cache size 0 max 16384 2024-04-22T13:29:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.960+0000 7fe9ac3be740 1 Processor -- start 2024-04-22T13:29:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.960+0000 7fe9ac3be740 1 -- start start 2024-04-22T13:29:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.960+0000 7fe9ac3be740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b273e94d0 0x560b273fe810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.960+0000 7fe9ac3be740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560b273e9e80 0x560b273eee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.960+0000 7fe9ac3be740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560b273f2a00 0x560b273ef380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.960+0000 7fe9ac3be740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a41ada60 con 0x560b273e9e80 2024-04-22T13:29:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.960+0000 7fe9ac3be740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a41af5a0 con 0x560b273f2a00 2024-04-22T13:29:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.960+0000 7fe9ac3be740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a41ae130 con 0x560b273e94d0 2024-04-22T13:29:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.960+0000 7fe9a2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560b273e9e80 0x560b273eee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.960+0000 7fe9a2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560b273e9e80 0x560b273eee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43424/0 (socket says 172.21.15.12:43424) 2024-04-22T13:29:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.960+0000 7fe9a2ffd640 1 -- 172.21.15.12:0/516862481 learned_addr learned my addr 172.21.15.12:0/516862481 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.960+0000 7fe9a27fc640 1 --2- 172.21.15.12:0/516862481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b273e94d0 0x560b273fe810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.960+0000 7fe9a2ffd640 1 -- 172.21.15.12:0/516862481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b273e94d0 msgr2=0x560b273fe810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.960+0000 7fe9a2ffd640 1 --2- 172.21.15.12:0/516862481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b273e94d0 0x560b273fe810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.960+0000 7fe9a2ffd640 1 -- 172.21.15.12:0/516862481 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560b273f2a00 msgr2=0x560b273ef380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.960+0000 7fe9a2ffd640 1 --2- 172.21.15.12:0/516862481 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560b273f2a00 0x560b273ef380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.960+0000 7fe9a2ffd640 1 -- 172.21.15.12:0/516862481 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b273e8560 con 0x560b273e9e80 2024-04-22T13:29:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.960+0000 7fe9a27fc640 1 --2- 172.21.15.12:0/516862481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b273e94d0 0x560b273fe810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.960+0000 7fe9a2ffd640 1 --2- 172.21.15.12:0/516862481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560b273e9e80 0x560b273eee40 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fe99402c170 tx=0x7fe994005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f0e4353e6bba3c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.961+0000 7fe988ff9640 1 -- 172.21.15.12:0/516862481 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe994048050 con 0x560b273e9e80 2024-04-22T13:29:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.961+0000 7fe988ff9640 1 -- 172.21.15.12:0/516862481 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe99402cc90 con 0x560b273e9e80 2024-04-22T13:29:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.961+0000 7fe988ff9640 1 -- 172.21.15.12:0/516862481 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9940053c0 con 0x560b273e9e80 2024-04-22T13:29:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.961+0000 7fe9ac3be740 1 -- 172.21.15.12:0/516862481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560b273e9e80 msgr2=0x560b273eee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.961+0000 7fe9ac3be740 1 --2- 172.21.15.12:0/516862481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560b273e9e80 0x560b273eee40 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fe99402c170 tx=0x7fe994005b50 comp rx=0 tx=0).stop 2024-04-22T13:29:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.961+0000 7fe9ac3be740 1 -- 172.21.15.12:0/516862481 shutdown_connections 2024-04-22T13:29:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.961+0000 7fe9ac3be740 1 --2- 172.21.15.12:0/516862481 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560b273f2a00 0x560b273ef380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.961+0000 7fe9ac3be740 1 --2- 172.21.15.12:0/516862481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560b273e9e80 0x560b273eee40 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.961+0000 7fe9ac3be740 1 --2- 172.21.15.12:0/516862481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b273e94d0 0x560b273fe810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.961+0000 7fe9ac3be740 1 -- 172.21.15.12:0/516862481 >> 172.21.15.12:0/516862481 conn(0x560b273e3ba0 msgr2=0x560b273e5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:42.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.961+0000 7fe9ac3be740 1 -- 172.21.15.12:0/516862481 shutdown_connections 2024-04-22T13:29:42.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.961+0000 7fe9ac3be740 1 -- 172.21.15.12:0/516862481 wait complete. 2024-04-22T13:29:42.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.961+0000 7fe9ac3be740 1 -- start start 2024-04-22T13:29:42.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.962+0000 7fe9ac3be740 1 Processor -- start 2024-04-22T13:29:42.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.962+0000 7fe9ac3be740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560b273e94d0 0x560b274081d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.962+0000 7fe9ac3be740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b273e9e80 0x560b27405660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.962+0000 7fe9ac3be740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560b273f2a00 0x560b27405ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.962+0000 7fe9ac3be740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a41aeed0 con 0x560b273e94d0 2024-04-22T13:29:42.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.962+0000 7fe9ac3be740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a41ae800 con 0x560b273f2a00 2024-04-22T13:29:42.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.962+0000 7fe9ac3be740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a41aba60 con 0x560b273e9e80 2024-04-22T13:29:42.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.962+0000 7fe9ac3be740 10 client.0 _ll_register_callbacks cb 0x7fe998004020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:42.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.962+0000 7fe9a27fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560b273e94d0 0x560b274081d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.962+0000 7fe9a27fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560b273e94d0 0x560b274081d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43426/0 (socket says 172.21.15.12:43426) 2024-04-22T13:29:42.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.962+0000 7fe9a27fc640 1 -- 172.21.15.12:0/1521497810 learned_addr learned my addr 172.21.15.12:0/1521497810 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:42.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.962+0000 7fe9a2ffd640 1 --2- 172.21.15.12:0/1521497810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b273e9e80 0x560b27405660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.962+0000 7fe9a27fc640 1 -- 172.21.15.12:0/1521497810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b273e9e80 msgr2=0x560b27405660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:42.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.962+0000 7fe9a27fc640 1 --2- 172.21.15.12:0/1521497810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b273e9e80 0x560b27405660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.962+0000 7fe9a27fc640 1 -- 172.21.15.12:0/1521497810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560b273f2a00 msgr2=0x560b27405ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:42.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.962+0000 7fe9a27fc640 1 --2- 172.21.15.12:0/1521497810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560b273f2a00 0x560b27405ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.962+0000 7fe9a27fc640 1 -- 172.21.15.12:0/1521497810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b274062e0 con 0x560b273e94d0 2024-04-22T13:29:42.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.962+0000 7fe9a27fc640 1 --2- 172.21.15.12:0/1521497810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560b273e94d0 0x560b274081d0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fe9980096f0 tx=0x7fe998003900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4e5e792701ea93 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:42.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.962+0000 7fe986ff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:42.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.962+0000 7fe986ff5640 1 -- 172.21.15.12:0/1521497810 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe998004490 con 0x560b273e94d0 2024-04-22T13:29:42.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.963+0000 7fe9ac3be740 1 -- 172.21.15.12:0/1521497810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe99402cc90 con 0x560b273e94d0 2024-04-22T13:29:42.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.963+0000 7fe986ff5640 1 -- 172.21.15.12:0/1521497810 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fe998003d60 con 0x560b273e94d0 2024-04-22T13:29:42.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.963+0000 7fe986ff5640 1 -- 172.21.15.12:0/1521497810 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe9980040d0 con 0x560b273e94d0 2024-04-22T13:29:42.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.963+0000 7fe9ac3be740 10 client.17799 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:42.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.963+0000 7fe9ac3be740 1 -- 172.21.15.12:0/1521497810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560b27468bb0 con 0x560b273e94d0 2024-04-22T13:29:42.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.963+0000 7fe986ff5640 1 -- 172.21.15.12:0/1521497810 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe99402cc90 con 0x560b273e94d0 2024-04-22T13:29:42.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.963+0000 7fe986ff5640 1 -- 172.21.15.12:0/1521497810 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe99801e070 con 0x560b273e94d0 2024-04-22T13:29:42.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.963+0000 7fe9ac3be740 10 client.17799 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:42.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.963+0000 7fe9ac3be740 10 client.17799 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:29:42.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.963+0000 7fe9ac3be740 20 client.17799 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:42.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.963+0000 7fe9ac3be740 1 --2- 172.21.15.12:0/1521497810 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x560b27469300 0x560b2746b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:42.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.963+0000 7fe9ac3be740 4 client.17799 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:29:42.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.963+0000 7fe9ac3be740 1 -- 172.21.15.12:0/1521497810 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560b2735bc10 con 0x560b27469300 2024-04-22T13:29:42.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.963+0000 7fe9a2ffd640 1 --2- 172.21.15.12:0/1521497810 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x560b27469300 0x560b2746b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:42.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.964+0000 7fe986ff5640 10 client.17799 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:29:42.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.964+0000 7fe9a2ffd640 1 --2- 172.21.15.12:0/1521497810 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x560b27469300 0x560b2746b720 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5b4e464cf29b1d83 server_cookie=ed14f269401d92ac in_seq=0 out_seq=0 2024-04-22T13:29:42.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.964+0000 7fe986ff5640 1 -- 172.21.15.12:0/1521497810 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560b2735bc10 con 0x560b27469300 2024-04-22T13:29:42.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.964+0000 7fe986ff5640 10 client.17799 handle_command_reply: tid=0 2024-04-22T13:29:42.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.964+0000 7fe9ac3be740 1 client.17799 shutdown 2024-04-22T13:29:42.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.964+0000 7fe9ac3be740 1 -- 172.21.15.12:0/1521497810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560b273e94d0 msgr2=0x560b274081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.964+0000 7fe9ac3be740 1 --2- 172.21.15.12:0/1521497810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560b273e94d0 0x560b274081d0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fe9980096f0 tx=0x7fe998003900 comp rx=0 tx=0).stop 2024-04-22T13:29:42.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.964+0000 7fe9ac3be740 1 -- 172.21.15.12:0/1521497810 shutdown_connections 2024-04-22T13:29:42.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.964+0000 7fe9ac3be740 1 --2- 172.21.15.12:0/1521497810 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x560b27469300 0x560b2746b720 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.964+0000 7fe9ac3be740 1 --2- 172.21.15.12:0/1521497810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560b273f2a00 0x560b27405ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.964+0000 7fe9ac3be740 1 --2- 172.21.15.12:0/1521497810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560b273e9e80 0x560b27405660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.964+0000 7fe986ff5640 0 client.17799 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:29:42.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.964+0000 7fe9ac3be740 1 --2- 172.21.15.12:0/1521497810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560b273e94d0 0x560b274081d0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.964+0000 7fe9ac3be740 1 -- 172.21.15.12:0/1521497810 >> 172.21.15.12:0/1521497810 conn(0x560b273e3ba0 msgr2=0x560b273ea260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:42.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.964+0000 7fe9ac3be740 1 -- 172.21.15.12:0/1521497810 shutdown_connections 2024-04-22T13:29:42.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.964+0000 7fe9ac3be740 1 -- 172.21.15.12:0/1521497810 wait complete. 2024-04-22T13:29:42.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.964+0000 7fe9ac3be740 20 client.17799 trim_cache size 0 max 16384 2024-04-22T13:29:42.967 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:29:42.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.965+0000 7fe9a96f2640 1 -- 172.21.15.12:0/2390470378 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe98c0a3d20 msgr2=0x7fe98c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.965+0000 7fe9a96f2640 1 --2- 172.21.15.12:0/2390470378 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe98c0a3d20 0x7fe98c0a61e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe99c0ce340 tx=0x7fe99c091a20 comp rx=0 tx=0).stop 2024-04-22T13:29:42.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.965+0000 7fe9a96f2640 1 -- 172.21.15.12:0/2390470378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9a419cf90 msgr2=0x7fe9a419d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:42.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.965+0000 7fe9a96f2640 1 --2- 172.21.15.12:0/2390470378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9a419cf90 0x7fe9a419d440 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009fd0 tx=0x7fe99c00b260 comp rx=0 tx=0).stop 2024-04-22T13:29:42.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.965+0000 7fe9a96f2640 1 -- 172.21.15.12:0/2390470378 shutdown_connections 2024-04-22T13:29:42.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.965+0000 7fe9a96f2640 1 --2- 172.21.15.12:0/2390470378 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe98c0a3d20 0x7fe98c0a61e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.965+0000 7fe9a96f2640 1 --2- 172.21.15.12:0/2390470378 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9a419cf90 0x7fe9a419d440 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.965+0000 7fe9a96f2640 1 --2- 172.21.15.12:0/2390470378 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9a4106170 0x7fe9a4198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.965+0000 7fe9a96f2640 1 --2- 172.21.15.12:0/2390470378 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9a41010a0 0x7fe9a4198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:42.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.965+0000 7fe9a96f2640 1 -- 172.21.15.12:0/2390470378 >> 172.21.15.12:0/2390470378 conn(0x7fe9a4077eb0 msgr2=0x7fe9a406aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:42.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.965+0000 7fe9a96f2640 1 -- 172.21.15.12:0/2390470378 shutdown_connections 2024-04-22T13:29:42.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:42.965+0000 7fe9a96f2640 1 -- 172.21.15.12:0/2390470378 wait complete. 2024-04-22T13:29:42.970 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:42 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1068026995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:29:42.971 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:29:43.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.080+0000 7f695b9db640 1 -- 172.21.15.12:0/1146240396 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f695406a840 msgr2=0x7f695406ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:43.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.080+0000 7f695b9db640 1 --2- 172.21.15.12:0/1146240396 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f695406a840 0x7f695406ac20 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f6950009900 tx=0x7f695002bc30 comp rx=0 tx=0).stop 2024-04-22T13:29:43.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.080+0000 7f695b9db640 1 -- 172.21.15.12:0/1146240396 shutdown_connections 2024-04-22T13:29:43.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.080+0000 7f695b9db640 1 --2- 172.21.15.12:0/1146240396 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6954107270 0x7f695410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.080+0000 7f695b9db640 1 --2- 172.21.15.12:0/1146240396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f695406b1f0 0x7f6954106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.080+0000 7f695b9db640 1 --2- 172.21.15.12:0/1146240396 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f695406a840 0x7f695406ac20 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.078 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.080+0000 7f695b9db640 1 -- 172.21.15.12:0/1146240396 >> 172.21.15.12:0/1146240396 conn(0x7f6954077eb0 msgr2=0x7f69540782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:43.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.080+0000 7f695b9db640 1 -- 172.21.15.12:0/1146240396 shutdown_connections 2024-04-22T13:29:43.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.080+0000 7f695b9db640 1 -- 172.21.15.12:0/1146240396 wait complete. 2024-04-22T13:29:43.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.081+0000 7f695b9db640 1 Processor -- start 2024-04-22T13:29:43.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.081+0000 7f695b9db640 1 -- start start 2024-04-22T13:29:43.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.081+0000 7f695b9db640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f695406a840 0x7f6954100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.081+0000 7f695b9db640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6954107270 0x7f69541013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.081+0000 7f695b9db640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6954101900 0x7f69541a6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.081+0000 7f695b9db640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f69541121b0 con 0x7f6954101900 2024-04-22T13:29:43.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.081+0000 7f695b9db640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6954112030 con 0x7f695406a840 2024-04-22T13:29:43.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.081+0000 7f695b9db640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6954112330 con 0x7f6954107270 2024-04-22T13:29:43.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.081+0000 7f6959f78640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6954101900 0x7f69541a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.081+0000 7f6959f78640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6954101900 0x7f69541a6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43442/0 (socket says 172.21.15.12:43442) 2024-04-22T13:29:43.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.081+0000 7f6959777640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f695406a840 0x7f6954100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.081+0000 7f6959f78640 1 -- 172.21.15.12:0/330858437 learned_addr learned my addr 172.21.15.12:0/330858437 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:43.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.081+0000 7f6958f76640 1 --2- 172.21.15.12:0/330858437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6954107270 0x7f69541013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.082+0000 7f6959f78640 1 -- 172.21.15.12:0/330858437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6954107270 msgr2=0x7f69541013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:43.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.082+0000 7f6959f78640 1 --2- 172.21.15.12:0/330858437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6954107270 0x7f69541013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.082+0000 7f6959f78640 1 -- 172.21.15.12:0/330858437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f695406a840 msgr2=0x7f6954100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:43.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.082+0000 7f6959f78640 1 --2- 172.21.15.12:0/330858437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f695406a840 0x7f6954100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.082+0000 7f6959f78640 1 -- 172.21.15.12:0/330858437 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69541a7000 con 0x7f6954101900 2024-04-22T13:29:43.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.082+0000 7f6959f78640 1 --2- 172.21.15.12:0/330858437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6954101900 0x7f69541a6900 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f694c00cad0 tx=0x7f694c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:43.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.082+0000 7f694a7fc640 1 -- 172.21.15.12:0/330858437 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c00ae20 con 0x7f6954101900 2024-04-22T13:29:43.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.082+0000 7f695b9db640 1 -- 172.21.15.12:0/330858437 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69541a72f0 con 0x7f6954101900 2024-04-22T13:29:43.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.082+0000 7f694a7fc640 1 -- 172.21.15.12:0/330858437 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f694c005020 con 0x7f6954101900 2024-04-22T13:29:43.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.082+0000 7f694a7fc640 1 -- 172.21.15.12:0/330858437 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c00f6b0 con 0x7f6954101900 2024-04-22T13:29:43.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.082+0000 7f695b9db640 1 -- 172.21.15.12:0/330858437 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69541a7840 con 0x7f6954101900 2024-04-22T13:29:43.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.084+0000 7f694a7fc640 1 -- 172.21.15.12:0/330858437 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f694c0051d0 con 0x7f6954101900 2024-04-22T13:29:43.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.085+0000 7f695e5d0740 1 Processor -- start 2024-04-22T13:29:43.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.085+0000 7f695e5d0740 1 -- start start 2024-04-22T13:29:43.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.085+0000 7f695e5d0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c81ace84d0 0x55c81ace88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.085+0000 7f695e5d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c81ace8e80 0x55c81acf1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.085+0000 7f695e5d0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c81acf1a00 0x55c81acf3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.085+0000 7f695e5d0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c81ac4d1f0 con 0x55c81ace84d0 2024-04-22T13:29:43.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.085+0000 7f695e5d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c81abea4b0 con 0x55c81ace8e80 2024-04-22T13:29:43.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.085+0000 7f695e5d0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c81ac491b0 con 0x55c81acf1a00 2024-04-22T13:29:43.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.085+0000 7f6958f76640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c81ace84d0 0x55c81ace88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.085+0000 7f6958f76640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c81ace84d0 0x55c81ace88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43450/0 (socket says 172.21.15.12:43450) 2024-04-22T13:29:43.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.085+0000 7f6958f76640 1 -- 172.21.15.12:0/1069908888 learned_addr learned my addr 172.21.15.12:0/1069908888 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:43.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.085+0000 7f694a7fc640 1 --2- 172.21.15.12:0/330858437 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f69400a3d50 0x7f69400a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.085+0000 7f6958f76640 1 -- 172.21.15.12:0/1069908888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c81acf1a00 msgr2=0x55c81acf3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:43.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.085+0000 7f6958f76640 1 --2- 172.21.15.12:0/1069908888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c81acf1a00 0x55c81acf3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.085+0000 7f6959777640 1 --2- 172.21.15.12:0/1069908888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c81ace8e80 0x55c81acf1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.085+0000 7f6958f76640 1 -- 172.21.15.12:0/1069908888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c81ace8e80 msgr2=0x55c81acf1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:43.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.085+0000 7f6958f76640 1 --2- 172.21.15.12:0/1069908888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c81ace8e80 0x55c81acf1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.085+0000 7f6958f76640 1 -- 172.21.15.12:0/1069908888 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c81abdc1b0 con 0x55c81ace84d0 2024-04-22T13:29:43.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.085+0000 7f6959777640 1 --2- 172.21.15.12:0/1069908888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c81ace8e80 0x55c81acf1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:43.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.085+0000 7f6959f78640 1 --2- 172.21.15.12:0/330858437 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f69400a3d50 0x7f69400a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.086+0000 7f6958f76640 1 --2- 172.21.15.12:0/1069908888 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c81ace84d0 0x55c81ace88b0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f6944009d90 tx=0x7f6944013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646827dbd282872f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:43.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.086+0000 7f694a7fc640 1 -- 172.21.15.12:0/330858437 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f694c0d0720 con 0x7f6954101900 2024-04-22T13:29:43.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.086+0000 7f693f7fe640 1 -- 172.21.15.12:0/1069908888 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6944004050 con 0x55c81ace84d0 2024-04-22T13:29:43.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.086+0000 7f693f7fe640 1 -- 172.21.15.12:0/1069908888 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6944017030 con 0x55c81ace84d0 2024-04-22T13:29:43.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.086+0000 7f693f7fe640 1 -- 172.21.15.12:0/1069908888 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6944010430 con 0x55c81ace84d0 2024-04-22T13:29:43.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.086+0000 7f695e5d0740 1 -- 172.21.15.12:0/1069908888 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c81ace84d0 msgr2=0x55c81ace88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:43.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.086+0000 7f6959f78640 1 --2- 172.21.15.12:0/330858437 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f69400a3d50 0x7f69400a6210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f694c0cdf70 tx=0x7f694c005890 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:43.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.086+0000 7f695e5d0740 1 --2- 172.21.15.12:0/1069908888 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c81ace84d0 0x55c81ace88b0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f6944009d90 tx=0x7f6944013830 comp rx=0 tx=0).stop 2024-04-22T13:29:43.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.087+0000 7f695e5d0740 1 -- 172.21.15.12:0/1069908888 shutdown_connections 2024-04-22T13:29:43.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.087+0000 7f695e5d0740 1 --2- 172.21.15.12:0/1069908888 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c81acf1a00 0x55c81acf3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.087+0000 7f695e5d0740 1 --2- 172.21.15.12:0/1069908888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c81ace8e80 0x55c81acf1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.087+0000 7f695e5d0740 1 --2- 172.21.15.12:0/1069908888 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c81ace84d0 0x55c81ace88b0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.087+0000 7f695e5d0740 1 -- 172.21.15.12:0/1069908888 >> 172.21.15.12:0/1069908888 conn(0x55c81ace2ba0 msgr2=0x55c81ace2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:43.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.087+0000 7f695e5d0740 1 -- 172.21.15.12:0/1069908888 shutdown_connections 2024-04-22T13:29:43.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.087+0000 7f695e5d0740 1 -- 172.21.15.12:0/1069908888 wait complete. 2024-04-22T13:29:43.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.087+0000 7f695e5d0740 1 -- start start 2024-04-22T13:29:43.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.087+0000 7f695e5d0740 1 Processor -- start 2024-04-22T13:29:43.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.087+0000 7f695e5d0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c81ace84d0 0x55c81acf12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.087+0000 7f695e5d0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c81ace8e80 0x55c81acee630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.087+0000 7f695e5d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c81acf1a00 0x55c81aceeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.087+0000 7f695e5d0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c81acf4970 con 0x55c81ace8e80 2024-04-22T13:29:43.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.087+0000 7f695e5d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c81ac06500 con 0x55c81acf1a00 2024-04-22T13:29:43.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.087+0000 7f695e5d0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c81acf4af0 con 0x55c81ace84d0 2024-04-22T13:29:43.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.087+0000 7f695e5d0740 10 client.0 _ll_register_callbacks cb 0x55c81abd4b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:43.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.087+0000 7f6959777640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c81ace8e80 0x55c81acee630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.088+0000 7f6959777640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c81ace8e80 0x55c81acee630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43466/0 (socket says 172.21.15.12:43466) 2024-04-22T13:29:43.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.088+0000 7f6959777640 1 -- 172.21.15.12:0/3656069202 learned_addr learned my addr 172.21.15.12:0/3656069202 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:43.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.088+0000 7f6958f76640 1 --2- 172.21.15.12:0/3656069202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c81ace84d0 0x55c81acf12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.088+0000 7f6958f76640 1 --2- 172.21.15.12:0/3656069202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c81acf1a00 0x55c81aceeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.088+0000 7f6959777640 1 -- 172.21.15.12:0/3656069202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c81ace84d0 msgr2=0x55c81acf12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:43.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.088+0000 7f6959777640 1 --2- 172.21.15.12:0/3656069202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c81ace84d0 0x55c81acf12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.088+0000 7f6959777640 1 -- 172.21.15.12:0/3656069202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c81acf1a00 msgr2=0x55c81aceeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:43.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.088+0000 7f6959777640 1 --2- 172.21.15.12:0/3656069202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c81acf1a00 0x55c81aceeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.088+0000 7f6959777640 1 -- 172.21.15.12:0/3656069202 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c81a95f3f0 con 0x55c81ace8e80 2024-04-22T13:29:43.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.088+0000 7f6958f76640 1 --2- 172.21.15.12:0/3656069202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c81ace84d0 0x55c81acf12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:43.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.088+0000 7f6959777640 1 --2- 172.21.15.12:0/3656069202 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c81ace8e80 0x55c81acee630 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f695002c170 tx=0x7f695002c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f08f7c90f8e6e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:43.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.088+0000 7f693cff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:43.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.088+0000 7f693cff9640 1 -- 172.21.15.12:0/3656069202 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695002c610 con 0x55c81ace8e80 2024-04-22T13:29:43.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.088+0000 7f693cff9640 1 -- 172.21.15.12:0/3656069202 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f6950005030 con 0x55c81ace8e80 2024-04-22T13:29:43.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.088+0000 7f695e5d0740 1 -- 172.21.15.12:0/3656069202 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6944017030 con 0x55c81ace8e80 2024-04-22T13:29:43.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.088+0000 7f693cff9640 1 -- 172.21.15.12:0/3656069202 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6950005230 con 0x55c81ace8e80 2024-04-22T13:29:43.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.088+0000 7f695e5d0740 10 client.17811 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:43.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.088+0000 7f695e5d0740 1 -- 172.21.15.12:0/3656069202 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c81aa297d0 con 0x55c81ace8e80 2024-04-22T13:29:43.091 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.089+0000 7f693cff9640 1 -- 172.21.15.12:0/3656069202 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6944017030 con 0x55c81ace8e80 2024-04-22T13:29:43.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.089+0000 7f693cff9640 1 -- 172.21.15.12:0/3656069202 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6950003680 con 0x55c81ace8e80 2024-04-22T13:29:43.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.089+0000 7f695e5d0740 10 client.17811 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:43.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.089+0000 7f695e5d0740 10 client.17811 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:29:43.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.089+0000 7f695e5d0740 20 client.17811 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:43.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.089+0000 7f695e5d0740 1 --2- 172.21.15.12:0/3656069202 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55c81ad01b20 0x55c81ad03f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.089+0000 7f695e5d0740 4 client.17811 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:43.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.089+0000 7f695e5d0740 1 -- 172.21.15.12:0/3656069202 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c81abec930 con 0x55c81ad01b20 2024-04-22T13:29:43.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.089+0000 7f6958f76640 1 --2- 172.21.15.12:0/3656069202 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55c81ad01b20 0x55c81ad03f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.089+0000 7f693cff9640 10 client.17811 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:29:43.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.089+0000 7f6958f76640 1 --2- 172.21.15.12:0/3656069202 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55c81ad01b20 0x55c81ad03f10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a2a3258eff63bc5a server_cookie=6f30baf013c093ac in_seq=0 out_seq=0 2024-04-22T13:29:43.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.090+0000 7f693cff9640 1 -- 172.21.15.12:0/3656069202 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c81abec930 con 0x55c81ad01b20 2024-04-22T13:29:43.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.090+0000 7f693cff9640 10 client.17811 handle_command_reply: tid=0 2024-04-22T13:29:43.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.090+0000 7f695e5d0740 1 client.17811 shutdown 2024-04-22T13:29:43.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.090+0000 7f695e5d0740 1 -- 172.21.15.12:0/3656069202 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c81ace8e80 msgr2=0x55c81acee630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:43.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.090+0000 7f695e5d0740 1 --2- 172.21.15.12:0/3656069202 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c81ace8e80 0x55c81acee630 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f695002c170 tx=0x7f695002c9f0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.090+0000 7f695e5d0740 1 -- 172.21.15.12:0/3656069202 shutdown_connections 2024-04-22T13:29:43.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.090+0000 7f695e5d0740 1 --2- 172.21.15.12:0/3656069202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c81acf1a00 0x55c81aceeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.090+0000 7f695e5d0740 1 --2- 172.21.15.12:0/3656069202 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55c81ad01b20 0x55c81ad03f10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.090+0000 7f695e5d0740 1 --2- 172.21.15.12:0/3656069202 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c81ace8e80 0x55c81acee630 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.090+0000 7f695e5d0740 1 --2- 172.21.15.12:0/3656069202 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c81ace84d0 0x55c81acf12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.090+0000 7f693cff9640 0 client.17811 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:29:43.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.090+0000 7f695e5d0740 1 -- 172.21.15.12:0/3656069202 >> 172.21.15.12:0/3656069202 conn(0x55c81ace2ba0 msgr2=0x55c81ace48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:43.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.090+0000 7f695e5d0740 1 -- 172.21.15.12:0/3656069202 shutdown_connections 2024-04-22T13:29:43.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.090+0000 7f695e5d0740 1 -- 172.21.15.12:0/3656069202 wait complete. 2024-04-22T13:29:43.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.091+0000 7f695e5d0740 20 client.17811 trim_cache size 0 max 16384 2024-04-22T13:29:43.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.104+0000 7f695e5d0740 1 Processor -- start 2024-04-22T13:29:43.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.104+0000 7f695e5d0740 1 -- start start 2024-04-22T13:29:43.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.104+0000 7f695e5d0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c81ace2ba0 0x55c81acfd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.104+0000 7f695e5d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c81ace84d0 0x55c81ace1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.104+0000 7f695e5d0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c81ace8e80 0x55c81acf0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.104+0000 7f695e5d0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f69541afcb0 con 0x55c81ace8e80 2024-04-22T13:29:43.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.104+0000 7f695e5d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f69541b17f0 con 0x55c81ace84d0 2024-04-22T13:29:43.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.104+0000 7f695e5d0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f69541b0380 con 0x55c81ace2ba0 2024-04-22T13:29:43.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.104+0000 7f6958f76640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c81ace8e80 0x55c81acf0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.104+0000 7f6958f76640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c81ace8e80 0x55c81acf0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43468/0 (socket says 172.21.15.12:43468) 2024-04-22T13:29:43.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.104+0000 7f6958f76640 1 -- 172.21.15.12:0/595463186 learned_addr learned my addr 172.21.15.12:0/595463186 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:43.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.104+0000 7f6958f76640 1 --2- 172.21.15.12:0/595463186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c81ace2ba0 0x55c81acfd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.104+0000 7f6959777640 1 --2- 172.21.15.12:0/595463186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c81ace84d0 0x55c81ace1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.105+0000 7f6958f76640 1 -- 172.21.15.12:0/595463186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c81ace2ba0 msgr2=0x55c81acfd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:43.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.105+0000 7f6958f76640 1 --2- 172.21.15.12:0/595463186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c81ace2ba0 0x55c81acfd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.105+0000 7f6958f76640 1 -- 172.21.15.12:0/595463186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c81ace84d0 msgr2=0x55c81ace1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:43.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.105+0000 7f6958f76640 1 --2- 172.21.15.12:0/595463186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c81ace84d0 0x55c81ace1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.105+0000 7f6958f76640 1 -- 172.21.15.12:0/595463186 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c81ace8c20 con 0x55c81ace8e80 2024-04-22T13:29:43.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.105+0000 7f6958f76640 1 --2- 172.21.15.12:0/595463186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c81ace8e80 0x55c81acf0d60 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f6944009250 tx=0x7f6944012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1682eeb181fe2c8d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:43.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.105+0000 7f693effd640 1 -- 172.21.15.12:0/595463186 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6944003740 con 0x55c81ace8e80 2024-04-22T13:29:43.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.105+0000 7f693effd640 1 -- 172.21.15.12:0/595463186 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6944017070 con 0x55c81ace8e80 2024-04-22T13:29:43.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.105+0000 7f693effd640 1 -- 172.21.15.12:0/595463186 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694402d290 con 0x55c81ace8e80 2024-04-22T13:29:43.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.105+0000 7f695e5d0740 1 -- 172.21.15.12:0/595463186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c81ace8e80 msgr2=0x55c81acf0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:43.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.105+0000 7f695e5d0740 1 --2- 172.21.15.12:0/595463186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c81ace8e80 0x55c81acf0d60 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f6944009250 tx=0x7f6944012040 comp rx=0 tx=0).stop 2024-04-22T13:29:43.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.105+0000 7f695e5d0740 1 -- 172.21.15.12:0/595463186 shutdown_connections 2024-04-22T13:29:43.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.105+0000 7f695e5d0740 1 --2- 172.21.15.12:0/595463186 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c81ace8e80 0x55c81acf0d60 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.105+0000 7f695e5d0740 1 --2- 172.21.15.12:0/595463186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c81ace84d0 0x55c81ace1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.105+0000 7f695e5d0740 1 --2- 172.21.15.12:0/595463186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c81ace2ba0 0x55c81acfd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.105+0000 7f695e5d0740 1 -- 172.21.15.12:0/595463186 >> 172.21.15.12:0/595463186 conn(0x55c81aceecd0 msgr2=0x55c81ace1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:43.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.106+0000 7f695e5d0740 1 -- 172.21.15.12:0/595463186 shutdown_connections 2024-04-22T13:29:43.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.106+0000 7f695e5d0740 1 -- 172.21.15.12:0/595463186 wait complete. 2024-04-22T13:29:43.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.106+0000 7f695e5d0740 1 -- start start 2024-04-22T13:29:43.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.106+0000 7f695e5d0740 1 Processor -- start 2024-04-22T13:29:43.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.106+0000 7f695e5d0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c81ace2ba0 0x55c81acf9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.106+0000 7f695e5d0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c81ace84d0 0x55c81acf5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.106+0000 7f695e5d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c81ace8e80 0x55c81acf5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.106+0000 7f695e5d0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f69541b1120 con 0x55c81ace84d0 2024-04-22T13:29:43.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.106+0000 7f695e5d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f69541b0a50 con 0x55c81ace8e80 2024-04-22T13:29:43.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.106+0000 7f695e5d0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f69541adcb0 con 0x55c81ace2ba0 2024-04-22T13:29:43.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.106+0000 7f695e5d0740 10 client.0 _ll_register_callbacks cb 0x7f6950004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:43.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.106+0000 7f6959777640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c81ace84d0 0x55c81acf5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.106+0000 7f6959777640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c81ace84d0 0x55c81acf5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43484/0 (socket says 172.21.15.12:43484) 2024-04-22T13:29:43.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.106+0000 7f6959777640 1 -- 172.21.15.12:0/3436741 learned_addr learned my addr 172.21.15.12:0/3436741 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:43.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.107+0000 7f6958f76640 1 --2- 172.21.15.12:0/3436741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c81ace2ba0 0x55c81acf9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.107+0000 7f6959777640 1 -- 172.21.15.12:0/3436741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c81ace2ba0 msgr2=0x55c81acf9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:43.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.107+0000 7f6959777640 1 --2- 172.21.15.12:0/3436741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c81ace2ba0 0x55c81acf9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.107+0000 7f6959777640 1 -- 172.21.15.12:0/3436741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c81ace8e80 msgr2=0x55c81acf5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:43.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.107+0000 7f6958f76640 1 --2- 172.21.15.12:0/3436741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c81ace8e80 0x55c81acf5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.107+0000 7f6959777640 1 --2- 172.21.15.12:0/3436741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c81ace8e80 0x55c81acf5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.107+0000 7f6959777640 1 -- 172.21.15.12:0/3436741 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c81acf5e00 con 0x55c81ace84d0 2024-04-22T13:29:43.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.107+0000 7f6959777640 1 --2- 172.21.15.12:0/3436741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c81ace84d0 0x55c81acf5300 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x55c81acefc00 tx=0x7f6950005600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd7c0ba95fd67ec3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:43.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.107+0000 7f693cff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:43.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.107+0000 7f693cff9640 1 -- 172.21.15.12:0/3436741 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695002c610 con 0x55c81ace84d0 2024-04-22T13:29:43.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.107+0000 7f693cff9640 1 -- 172.21.15.12:0/3436741 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f695003f510 con 0x55c81ace84d0 2024-04-22T13:29:43.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.107+0000 7f695e5d0740 1 -- 172.21.15.12:0/3436741 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6944017070 con 0x55c81ace84d0 2024-04-22T13:29:43.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.107+0000 7f693cff9640 1 -- 172.21.15.12:0/3436741 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f695003f710 con 0x55c81ace84d0 2024-04-22T13:29:43.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.107+0000 7f695e5d0740 10 client.17817 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:43.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.107+0000 7f695e5d0740 1 -- 172.21.15.12:0/3436741 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c81ad67de0 con 0x55c81ace84d0 2024-04-22T13:29:43.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.107+0000 7f693cff9640 1 -- 172.21.15.12:0/3436741 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6944017070 con 0x55c81ace84d0 2024-04-22T13:29:43.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.108+0000 7f693cff9640 1 -- 172.21.15.12:0/3436741 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6950004960 con 0x55c81ace84d0 2024-04-22T13:29:43.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.108+0000 7f695e5d0740 10 client.17817 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:43.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.108+0000 7f695e5d0740 10 client.17817 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:29:43.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.108+0000 7f695e5d0740 20 client.17817 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:43.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.108+0000 7f695e5d0740 1 --2- 172.21.15.12:0/3436741 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55c81acf1a00 0x55c81ad6a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.108+0000 7f695e5d0740 4 client.17817 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:29:43.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.108+0000 7f695e5d0740 1 -- 172.21.15.12:0/3436741 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c81ac5ac10 con 0x55c81acf1a00 2024-04-22T13:29:43.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.108+0000 7f6958f76640 1 --2- 172.21.15.12:0/3436741 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55c81acf1a00 0x55c81ad6a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.108+0000 7f693cff9640 10 client.17817 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:29:43.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.108+0000 7f6958f76640 1 --2- 172.21.15.12:0/3436741 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55c81acf1a00 0x55c81ad6a5d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3f9c6d7d6f3fa353 server_cookie=7a09dc02d270e74b in_seq=0 out_seq=0 2024-04-22T13:29:43.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.108+0000 7f693cff9640 1 -- 172.21.15.12:0/3436741 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c81ac5ac10 con 0x55c81acf1a00 2024-04-22T13:29:43.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.108+0000 7f693cff9640 10 client.17817 handle_command_reply: tid=0 2024-04-22T13:29:43.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.108+0000 7f695e5d0740 1 client.17817 shutdown 2024-04-22T13:29:43.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.109+0000 7f695e5d0740 1 -- 172.21.15.12:0/3436741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c81ace84d0 msgr2=0x55c81acf5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:43.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.109+0000 7f695e5d0740 1 --2- 172.21.15.12:0/3436741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c81ace84d0 0x55c81acf5300 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x55c81acefc00 tx=0x7f6950005600 comp rx=0 tx=0).stop 2024-04-22T13:29:43.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.109+0000 7f695e5d0740 1 -- 172.21.15.12:0/3436741 shutdown_connections 2024-04-22T13:29:43.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.109+0000 7f695e5d0740 1 --2- 172.21.15.12:0/3436741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c81ace8e80 0x55c81acf5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.109+0000 7f695e5d0740 1 --2- 172.21.15.12:0/3436741 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55c81acf1a00 0x55c81ad6a5d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.109+0000 7f695e5d0740 1 --2- 172.21.15.12:0/3436741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c81ace84d0 0x55c81acf5300 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.109+0000 7f695e5d0740 1 --2- 172.21.15.12:0/3436741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c81ace2ba0 0x55c81acf9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.109+0000 7f693cff9640 0 client.17817 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:29:43.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.109+0000 7f695e5d0740 1 -- 172.21.15.12:0/3436741 >> 172.21.15.12:0/3436741 conn(0x55c81aceecd0 msgr2=0x55c81acecac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:43.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.109+0000 7f695e5d0740 1 -- 172.21.15.12:0/3436741 shutdown_connections 2024-04-22T13:29:43.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.109+0000 7f695e5d0740 1 -- 172.21.15.12:0/3436741 wait complete. 2024-04-22T13:29:43.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.109+0000 7f695e5d0740 20 client.17817 trim_cache size 0 max 16384 2024-04-22T13:29:43.115 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:29:43.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.109+0000 7f695b9db640 1 -- 172.21.15.12:0/330858437 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f69400a3d50 msgr2=0x7f69400a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:43.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.109+0000 7f695b9db640 1 --2- 172.21.15.12:0/330858437 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f69400a3d50 0x7f69400a6210 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f694c0cdf70 tx=0x7f694c005890 comp rx=0 tx=0).stop 2024-04-22T13:29:43.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.109+0000 7f695b9db640 1 -- 172.21.15.12:0/330858437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6954101900 msgr2=0x7f69541a6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:43.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.109+0000 7f695b9db640 1 --2- 172.21.15.12:0/330858437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6954101900 0x7f69541a6900 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f694c00cad0 tx=0x7f694c00cf90 comp rx=0 tx=0).stop 2024-04-22T13:29:43.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.109+0000 7f695b9db640 1 -- 172.21.15.12:0/330858437 shutdown_connections 2024-04-22T13:29:43.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.109+0000 7f695b9db640 1 --2- 172.21.15.12:0/330858437 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f69400a3d50 0x7f69400a6210 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.109+0000 7f695b9db640 1 --2- 172.21.15.12:0/330858437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6954101900 0x7f69541a6900 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.109+0000 7f695b9db640 1 --2- 172.21.15.12:0/330858437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6954107270 0x7f69541013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.109+0000 7f695b9db640 1 --2- 172.21.15.12:0/330858437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f695406a840 0x7f6954100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.109+0000 7f695b9db640 1 -- 172.21.15.12:0/330858437 >> 172.21.15.12:0/330858437 conn(0x7f6954077eb0 msgr2=0x7f6954108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:43.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.109+0000 7f695b9db640 1 -- 172.21.15.12:0/330858437 shutdown_connections 2024-04-22T13:29:43.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.109+0000 7f695b9db640 1 -- 172.21.15.12:0/330858437 wait complete. 2024-04-22T13:29:43.118 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:29:43.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:42 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1068026995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:29:43.229 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:42 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1068026995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:29:43.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.226+0000 7f62695fb640 1 -- 172.21.15.12:0/942987744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62641031d0 msgr2=0x7f62641035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:43.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.226+0000 7f62695fb640 1 --2- 172.21.15.12:0/942987744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62641031d0 0x7f62641035b0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f6254009900 tx=0x7f625402bd60 comp rx=0 tx=0).stop 2024-04-22T13:29:43.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.226+0000 7f62695fb640 1 -- 172.21.15.12:0/942987744 shutdown_connections 2024-04-22T13:29:43.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.226+0000 7f62695fb640 1 --2- 172.21.15.12:0/942987744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f626410c510 0x7f626410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.226+0000 7f62695fb640 1 --2- 172.21.15.12:0/942987744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6264103af0 0x7f626410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.226+0000 7f62695fb640 1 --2- 172.21.15.12:0/942987744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62641031d0 0x7f62641035b0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.226+0000 7f62695fb640 1 -- 172.21.15.12:0/942987744 >> 172.21.15.12:0/942987744 conn(0x7f6264077f50 msgr2=0x7f6264078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:43.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.226+0000 7f62695fb640 1 -- 172.21.15.12:0/942987744 shutdown_connections 2024-04-22T13:29:43.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.226+0000 7f62695fb640 1 -- 172.21.15.12:0/942987744 wait complete. 2024-04-22T13:29:43.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.227+0000 7f62695fb640 1 Processor -- start 2024-04-22T13:29:43.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.227+0000 7f62695fb640 1 -- start start 2024-04-22T13:29:43.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.227+0000 7f62695fb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62641031d0 0x7f626419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.227+0000 7f62695fb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6264103af0 0x7f626419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.227+0000 7f62695fb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f626410c510 0x7f62641985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.227+0000 7f62695fb640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6264112180 con 0x7f626410c510 2024-04-22T13:29:43.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.227+0000 7f62695fb640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6264112000 con 0x7f62641031d0 2024-04-22T13:29:43.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.227+0000 7f62695fb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6264112300 con 0x7f6264103af0 2024-04-22T13:29:43.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.227+0000 7f62637fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f626410c510 0x7f62641985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.227+0000 7f62637fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f626410c510 0x7f62641985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43506/0 (socket says 172.21.15.12:43506) 2024-04-22T13:29:43.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.227+0000 7f62637fe640 1 -- 172.21.15.12:0/2953825521 learned_addr learned my addr 172.21.15.12:0/2953825521 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:43.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.227+0000 7f6262ffd640 1 --2- 172.21.15.12:0/2953825521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62641031d0 0x7f626419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.228+0000 7f62627fc640 1 --2- 172.21.15.12:0/2953825521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6264103af0 0x7f626419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.228+0000 7f62637fe640 1 -- 172.21.15.12:0/2953825521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6264103af0 msgr2=0x7f626419e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:43.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.228+0000 7f62637fe640 1 --2- 172.21.15.12:0/2953825521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6264103af0 0x7f626419e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.228+0000 7f62637fe640 1 -- 172.21.15.12:0/2953825521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62641031d0 msgr2=0x7f626419e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:43.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.228+0000 7f62637fe640 1 --2- 172.21.15.12:0/2953825521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62641031d0 0x7f626419e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.228+0000 7f62637fe640 1 -- 172.21.15.12:0/2953825521 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6264198d20 con 0x7f626410c510 2024-04-22T13:29:43.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.228+0000 7f62637fe640 1 --2- 172.21.15.12:0/2953825521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f626410c510 0x7f62641985a0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f625c009fd0 tx=0x7f625c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:43.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.228+0000 7f624bfff640 1 -- 172.21.15.12:0/2953825521 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c017070 con 0x7f626410c510 2024-04-22T13:29:43.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.228+0000 7f62695fb640 1 -- 172.21.15.12:0/2953825521 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6264199010 con 0x7f626410c510 2024-04-22T13:29:43.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.228+0000 7f624bfff640 1 -- 172.21.15.12:0/2953825521 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f625c003e20 con 0x7f626410c510 2024-04-22T13:29:43.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.228+0000 7f624bfff640 1 -- 172.21.15.12:0/2953825521 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c010460 con 0x7f626410c510 2024-04-22T13:29:43.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.228+0000 7f62695fb640 1 -- 172.21.15.12:0/2953825521 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62641a52b0 con 0x7f626410c510 2024-04-22T13:29:43.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.231+0000 7f624bfff640 1 -- 172.21.15.12:0/2953825521 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f625c003970 con 0x7f626410c510 2024-04-22T13:29:43.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.231+0000 7f626c1e1740 1 Processor -- start 2024-04-22T13:29:43.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.231+0000 7f626c1e1740 1 -- start start 2024-04-22T13:29:43.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.231+0000 7f626c1e1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14f3184d0 0x55c14f3188b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.231+0000 7f626c1e1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14f318e80 0x55c14f321430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.231+0000 7f626c1e1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14f321a00 0x55c14f323e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.231+0000 7f626c1e1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c14f27d1f0 con 0x55c14f3184d0 2024-04-22T13:29:43.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.231+0000 7f626c1e1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c14f21a4b0 con 0x55c14f318e80 2024-04-22T13:29:43.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.231+0000 7f626c1e1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c14f2791b0 con 0x55c14f321a00 2024-04-22T13:29:43.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.231+0000 7f62627fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14f3184d0 0x55c14f3188b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.231+0000 7f62627fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14f3184d0 0x55c14f3188b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43520/0 (socket says 172.21.15.12:43520) 2024-04-22T13:29:43.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.231+0000 7f62627fc640 1 -- 172.21.15.12:0/553428856 learned_addr learned my addr 172.21.15.12:0/553428856 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:43.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.231+0000 7f6262ffd640 1 --2- 172.21.15.12:0/553428856 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14f318e80 0x55c14f321430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.231+0000 7f624bfff640 1 --2- 172.21.15.12:0/2953825521 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f624c0a3d20 0x7f624c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.231+0000 7f62627fc640 1 -- 172.21.15.12:0/553428856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14f321a00 msgr2=0x55c14f323e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:43.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.231+0000 7f62627fc640 1 --2- 172.21.15.12:0/553428856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14f321a00 0x55c14f323e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.231+0000 7f62627fc640 1 -- 172.21.15.12:0/553428856 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14f318e80 msgr2=0x55c14f321430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:43.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.231+0000 7f62627fc640 1 --2- 172.21.15.12:0/553428856 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14f318e80 0x55c14f321430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.231+0000 7f62627fc640 1 -- 172.21.15.12:0/553428856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c14f20c1b0 con 0x55c14f3184d0 2024-04-22T13:29:43.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.232+0000 7f62637fe640 1 --2- 172.21.15.12:0/2953825521 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f624c0a3d20 0x7f624c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.232+0000 7f624bfff640 1 -- 172.21.15.12:0/2953825521 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f625c0d04d0 con 0x7f626410c510 2024-04-22T13:29:43.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.232+0000 7f62627fc640 1 --2- 172.21.15.12:0/553428856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14f3184d0 0x55c14f3188b0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f6258009c90 tx=0x7f62580137d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf7b557616a0769 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:43.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.232+0000 7f62497fa640 1 -- 172.21.15.12:0/553428856 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6258013070 con 0x55c14f3184d0 2024-04-22T13:29:43.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.232+0000 7f62497fa640 1 -- 172.21.15.12:0/553428856 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6258017030 con 0x55c14f3184d0 2024-04-22T13:29:43.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.232+0000 7f62497fa640 1 -- 172.21.15.12:0/553428856 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6258011680 con 0x55c14f3184d0 2024-04-22T13:29:43.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.232+0000 7f626c1e1740 1 -- 172.21.15.12:0/553428856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14f3184d0 msgr2=0x55c14f3188b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:43.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.232+0000 7f62637fe640 1 --2- 172.21.15.12:0/2953825521 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f624c0a3d20 0x7f624c0a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f625c0ce270 tx=0x7f625c06bda0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:43.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.232+0000 7f626c1e1740 1 --2- 172.21.15.12:0/553428856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14f3184d0 0x55c14f3188b0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f6258009c90 tx=0x7f62580137d0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.233+0000 7f626c1e1740 1 -- 172.21.15.12:0/553428856 shutdown_connections 2024-04-22T13:29:43.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.233+0000 7f626c1e1740 1 --2- 172.21.15.12:0/553428856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14f321a00 0x55c14f323e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.233+0000 7f626c1e1740 1 --2- 172.21.15.12:0/553428856 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14f318e80 0x55c14f321430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.233+0000 7f626c1e1740 1 --2- 172.21.15.12:0/553428856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14f3184d0 0x55c14f3188b0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.233+0000 7f626c1e1740 1 -- 172.21.15.12:0/553428856 >> 172.21.15.12:0/553428856 conn(0x55c14f312ba0 msgr2=0x55c14f312f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:43.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.233+0000 7f626c1e1740 1 -- 172.21.15.12:0/553428856 shutdown_connections 2024-04-22T13:29:43.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.233+0000 7f626c1e1740 1 -- 172.21.15.12:0/553428856 wait complete. 2024-04-22T13:29:43.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.233+0000 7f626c1e1740 1 -- start start 2024-04-22T13:29:43.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.233+0000 7f626c1e1740 1 Processor -- start 2024-04-22T13:29:43.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.233+0000 7f626c1e1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14f3184d0 0x55c14f321300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.233+0000 7f626c1e1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14f318e80 0x55c14f31e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.233+0000 7f626c1e1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14f321a00 0x55c14f31ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.233+0000 7f626c1e1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c14f324970 con 0x55c14f321a00 2024-04-22T13:29:43.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.233+0000 7f626c1e1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c14f236500 con 0x55c14f3184d0 2024-04-22T13:29:43.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.233+0000 7f626c1e1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c14f324af0 con 0x55c14f318e80 2024-04-22T13:29:43.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.233+0000 7f626c1e1740 10 client.0 _ll_register_callbacks cb 0x55c14f204b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:43.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.233+0000 7f62627fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14f321a00 0x55c14f31ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.233+0000 7f62627fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14f321a00 0x55c14f31ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43524/0 (socket says 172.21.15.12:43524) 2024-04-22T13:29:43.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.233+0000 7f62627fc640 1 -- 172.21.15.12:0/2683196527 learned_addr learned my addr 172.21.15.12:0/2683196527 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:43.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.234+0000 7f6262ffd640 1 --2- 172.21.15.12:0/2683196527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14f318e80 0x55c14f31e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.234+0000 7f62627fc640 1 -- 172.21.15.12:0/2683196527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14f318e80 msgr2=0x55c14f31e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:43.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.234+0000 7f62627fc640 1 --2- 172.21.15.12:0/2683196527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14f318e80 0x55c14f31e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.234+0000 7f62627fc640 1 -- 172.21.15.12:0/2683196527 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14f3184d0 msgr2=0x55c14f321300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:43.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.234+0000 7f62627fc640 1 --2- 172.21.15.12:0/2683196527 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14f3184d0 0x55c14f321300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.234+0000 7f62627fc640 1 -- 172.21.15.12:0/2683196527 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c14ef8f3f0 con 0x55c14f321a00 2024-04-22T13:29:43.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.234+0000 7f62627fc640 1 --2- 172.21.15.12:0/2683196527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14f321a00 0x55c14f31ec10 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f6258009c60 tx=0x7f6258005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99124c069c33f795 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:43.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.234+0000 7f6246ff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:43.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.234+0000 7f6246ff5640 1 -- 172.21.15.12:0/2683196527 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6258005460 con 0x55c14f321a00 2024-04-22T13:29:43.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.234+0000 7f6246ff5640 1 -- 172.21.15.12:0/2683196527 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f625800f440 con 0x55c14f321a00 2024-04-22T13:29:43.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.234+0000 7f626c1e1740 1 -- 172.21.15.12:0/2683196527 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6258017030 con 0x55c14f321a00 2024-04-22T13:29:43.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.234+0000 7f6246ff5640 1 -- 172.21.15.12:0/2683196527 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f625802b020 con 0x55c14f321a00 2024-04-22T13:29:43.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.234+0000 7f626c1e1740 10 client.17829 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:43.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.234+0000 7f626c1e1740 1 -- 172.21.15.12:0/2683196527 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c14f0597d0 con 0x55c14f321a00 2024-04-22T13:29:43.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.234+0000 7f6246ff5640 1 -- 172.21.15.12:0/2683196527 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6258017030 con 0x55c14f321a00 2024-04-22T13:29:43.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.235+0000 7f6246ff5640 1 -- 172.21.15.12:0/2683196527 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6258004ad0 con 0x55c14f321a00 2024-04-22T13:29:43.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.235+0000 7f626c1e1740 10 client.17829 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:43.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.235+0000 7f626c1e1740 10 client.17829 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:29:43.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.235+0000 7f626c1e1740 20 client.17829 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:43.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.235+0000 7f626c1e1740 1 --2- 172.21.15.12:0/2683196527 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55c14f331a80 0x55c14f333e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.235+0000 7f626c1e1740 4 client.17829 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:43.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.235+0000 7f626c1e1740 1 -- 172.21.15.12:0/2683196527 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c14f21c930 con 0x55c14f331a80 2024-04-22T13:29:43.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.235+0000 7f6262ffd640 1 --2- 172.21.15.12:0/2683196527 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55c14f331a80 0x55c14f333e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.235+0000 7f6262ffd640 1 --2- 172.21.15.12:0/2683196527 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55c14f331a80 0x55c14f333e70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a0122ed6933bfee3 server_cookie=76ec71a6de831181 in_seq=0 out_seq=0 2024-04-22T13:29:43.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.235+0000 7f6246ff5640 10 client.17829 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:29:43.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.236+0000 7f6246ff5640 1 -- 172.21.15.12:0/2683196527 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c14f21c930 con 0x55c14f331a80 2024-04-22T13:29:43.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.236+0000 7f6246ff5640 10 client.17829 handle_command_reply: tid=0 2024-04-22T13:29:43.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.236+0000 7f626c1e1740 1 client.17829 shutdown 2024-04-22T13:29:43.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.236+0000 7f626c1e1740 1 -- 172.21.15.12:0/2683196527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14f321a00 msgr2=0x55c14f31ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:43.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.236+0000 7f626c1e1740 1 --2- 172.21.15.12:0/2683196527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14f321a00 0x55c14f31ec10 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f6258009c60 tx=0x7f6258005750 comp rx=0 tx=0).stop 2024-04-22T13:29:43.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.236+0000 7f626c1e1740 1 -- 172.21.15.12:0/2683196527 shutdown_connections 2024-04-22T13:29:43.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.236+0000 7f626c1e1740 1 --2- 172.21.15.12:0/2683196527 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55c14f331a80 0x55c14f333e70 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.236+0000 7f626c1e1740 1 --2- 172.21.15.12:0/2683196527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14f321a00 0x55c14f31ec10 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.236+0000 7f626c1e1740 1 --2- 172.21.15.12:0/2683196527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14f318e80 0x55c14f31e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.236+0000 7f6246ff5640 0 client.17829 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:29:43.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.236+0000 7f626c1e1740 1 --2- 172.21.15.12:0/2683196527 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14f3184d0 0x55c14f321300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.236+0000 7f626c1e1740 1 -- 172.21.15.12:0/2683196527 >> 172.21.15.12:0/2683196527 conn(0x55c14f312ba0 msgr2=0x55c14f3148f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:43.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.236+0000 7f626c1e1740 1 -- 172.21.15.12:0/2683196527 shutdown_connections 2024-04-22T13:29:43.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.236+0000 7f626c1e1740 1 -- 172.21.15.12:0/2683196527 wait complete. 2024-04-22T13:29:43.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.237+0000 7f626c1e1740 20 client.17829 trim_cache size 0 max 16384 2024-04-22T13:29:43.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.250+0000 7f626c1e1740 1 Processor -- start 2024-04-22T13:29:43.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.250+0000 7f626c1e1740 1 -- start start 2024-04-22T13:29:43.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.250+0000 7f626c1e1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14f3184d0 0x55c14f311370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.250+0000 7f626c1e1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14f318e80 0x55c14f31de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.250+0000 7f626c1e1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14f321a00 0x55c14f320dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.250+0000 7f626c1e1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f62641adb50 con 0x55c14f318e80 2024-04-22T13:29:43.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.250+0000 7f626c1e1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f62641af690 con 0x55c14f321a00 2024-04-22T13:29:43.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.250+0000 7f626c1e1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f62641ae220 con 0x55c14f3184d0 2024-04-22T13:29:43.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.250+0000 7f6262ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14f318e80 0x55c14f31de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.250+0000 7f6262ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14f318e80 0x55c14f31de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43538/0 (socket says 172.21.15.12:43538) 2024-04-22T13:29:43.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.250+0000 7f6262ffd640 1 -- 172.21.15.12:0/1128320580 learned_addr learned my addr 172.21.15.12:0/1128320580 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:43.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.250+0000 7f62627fc640 1 --2- 172.21.15.12:0/1128320580 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14f321a00 0x55c14f320dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.250+0000 7f62627fc640 1 --2- 172.21.15.12:0/1128320580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14f3184d0 0x55c14f311370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.250+0000 7f6262ffd640 1 -- 172.21.15.12:0/1128320580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14f3184d0 msgr2=0x55c14f311370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:43.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.250+0000 7f6262ffd640 1 --2- 172.21.15.12:0/1128320580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14f3184d0 0x55c14f311370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.250+0000 7f6262ffd640 1 -- 172.21.15.12:0/1128320580 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14f321a00 msgr2=0x55c14f320dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:43.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.250+0000 7f6262ffd640 1 --2- 172.21.15.12:0/1128320580 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14f321a00 0x55c14f320dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.250+0000 7f6262ffd640 1 -- 172.21.15.12:0/1128320580 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c14f31a270 con 0x55c14f318e80 2024-04-22T13:29:43.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.251+0000 7f6262ffd640 1 --2- 172.21.15.12:0/1128320580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14f318e80 0x55c14f31de40 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f625402c2a0 tx=0x7f6254009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194b85f4f561736b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:43.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.251+0000 7f6248ff9640 1 -- 172.21.15.12:0/1128320580 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6254004140 con 0x55c14f318e80 2024-04-22T13:29:43.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.251+0000 7f6248ff9640 1 -- 172.21.15.12:0/1128320580 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f625402cdc0 con 0x55c14f318e80 2024-04-22T13:29:43.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.251+0000 7f6248ff9640 1 -- 172.21.15.12:0/1128320580 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62540046c0 con 0x55c14f318e80 2024-04-22T13:29:43.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.251+0000 7f626c1e1740 1 -- 172.21.15.12:0/1128320580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14f318e80 msgr2=0x55c14f31de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:43.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.251+0000 7f626c1e1740 1 --2- 172.21.15.12:0/1128320580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14f318e80 0x55c14f31de40 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f625402c2a0 tx=0x7f6254009260 comp rx=0 tx=0).stop 2024-04-22T13:29:43.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.251+0000 7f626c1e1740 1 -- 172.21.15.12:0/1128320580 shutdown_connections 2024-04-22T13:29:43.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.251+0000 7f626c1e1740 1 --2- 172.21.15.12:0/1128320580 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14f321a00 0x55c14f320dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.251+0000 7f626c1e1740 1 --2- 172.21.15.12:0/1128320580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14f318e80 0x55c14f31de40 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.251+0000 7f626c1e1740 1 --2- 172.21.15.12:0/1128320580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14f3184d0 0x55c14f311370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.251+0000 7f626c1e1740 1 -- 172.21.15.12:0/1128320580 >> 172.21.15.12:0/1128320580 conn(0x55c14f312ba0 msgr2=0x55c14f314730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:43.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.251+0000 7f626c1e1740 1 -- 172.21.15.12:0/1128320580 shutdown_connections 2024-04-22T13:29:43.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.251+0000 7f626c1e1740 1 -- 172.21.15.12:0/1128320580 wait complete. 2024-04-22T13:29:43.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.252+0000 7f626c1e1740 1 -- start start 2024-04-22T13:29:43.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.252+0000 7f626c1e1740 1 Processor -- start 2024-04-22T13:29:43.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.252+0000 7f626c1e1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14f3184d0 0x55c14f329fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.252+0000 7f626c1e1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14f318e80 0x55c14f3253a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.252+0000 7f626c1e1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14f321a00 0x55c14f3258e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.252+0000 7f626c1e1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f62641aefc0 con 0x55c14f318e80 2024-04-22T13:29:43.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.252+0000 7f626c1e1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f62641ae8f0 con 0x55c14f3184d0 2024-04-22T13:29:43.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.252+0000 7f626c1e1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f62641abb30 con 0x55c14f321a00 2024-04-22T13:29:43.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.252+0000 7f626c1e1740 10 client.0 _ll_register_callbacks cb 0x7f6258013050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:43.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.252+0000 7f6262ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14f318e80 0x55c14f3253a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.252+0000 7f6262ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14f318e80 0x55c14f3253a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43548/0 (socket says 172.21.15.12:43548) 2024-04-22T13:29:43.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.252+0000 7f6262ffd640 1 -- 172.21.15.12:0/1218305227 learned_addr learned my addr 172.21.15.12:0/1218305227 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:43.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.252+0000 7f62627fc640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14f3184d0 0x55c14f329fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.252+0000 7f62627fc640 1 --2- 172.21.15.12:0/1218305227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14f321a00 0x55c14f3258e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.252+0000 7f6262ffd640 1 -- 172.21.15.12:0/1218305227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14f321a00 msgr2=0x55c14f3258e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:43.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.252+0000 7f6262ffd640 1 --2- 172.21.15.12:0/1218305227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14f321a00 0x55c14f3258e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.252+0000 7f6262ffd640 1 -- 172.21.15.12:0/1218305227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14f3184d0 msgr2=0x55c14f329fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:43.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.252+0000 7f6262ffd640 1 --2- 172.21.15.12:0/1218305227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14f3184d0 0x55c14f329fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.252+0000 7f6262ffd640 1 -- 172.21.15.12:0/1218305227 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c14f325f80 con 0x55c14f318e80 2024-04-22T13:29:43.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.252+0000 7f62627fc640 1 --2- 172.21.15.12:0/1218305227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14f321a00 0x55c14f3258e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:43.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.253+0000 7f6262ffd640 1 --2- 172.21.15.12:0/1218305227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14f318e80 0x55c14f3253a0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f6254036040 tx=0x7f62540051a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6c0c285f76ac47 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:43.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.253+0000 7f6246ff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:43.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.253+0000 7f6246ff5640 1 -- 172.21.15.12:0/1218305227 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625404f070 con 0x55c14f318e80 2024-04-22T13:29:43.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.253+0000 7f6246ff5640 1 -- 172.21.15.12:0/1218305227 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f625402cb10 con 0x55c14f318e80 2024-04-22T13:29:43.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.253+0000 7f626c1e1740 1 -- 172.21.15.12:0/1218305227 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f625402cdc0 con 0x55c14f318e80 2024-04-22T13:29:43.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.253+0000 7f6246ff5640 1 -- 172.21.15.12:0/1218305227 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6254004140 con 0x55c14f318e80 2024-04-22T13:29:43.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.253+0000 7f626c1e1740 10 client.17835 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:43.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.253+0000 7f626c1e1740 1 -- 172.21.15.12:0/1218305227 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c14f397b60 con 0x55c14f318e80 2024-04-22T13:29:43.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.253+0000 7f6246ff5640 1 -- 172.21.15.12:0/1218305227 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f625402cdc0 con 0x55c14f318e80 2024-04-22T13:29:43.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.253+0000 7f6246ff5640 1 -- 172.21.15.12:0/1218305227 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6254047070 con 0x55c14f318e80 2024-04-22T13:29:43.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.253+0000 7f626c1e1740 10 client.17835 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:43.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.253+0000 7f626c1e1740 10 client.17835 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:29:43.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.253+0000 7f626c1e1740 20 client.17835 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:43.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.253+0000 7f626c1e1740 1 --2- 172.21.15.12:0/1218305227 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55c14f3982c0 0x55c14f39a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.253+0000 7f626c1e1740 4 client.17835 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:29:43.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.253+0000 7f626c1e1740 1 -- 172.21.15.12:0/1218305227 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c14f28ac10 con 0x55c14f3982c0 2024-04-22T13:29:43.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.253+0000 7f62627fc640 1 --2- 172.21.15.12:0/1218305227 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55c14f3982c0 0x55c14f39a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.254+0000 7f6246ff5640 10 client.17835 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:29:43.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.254+0000 7f62627fc640 1 --2- 172.21.15.12:0/1218305227 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55c14f3982c0 0x55c14f39a6e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=792282ae2cdad671 server_cookie=d4d0b68851c007be in_seq=0 out_seq=0 2024-04-22T13:29:43.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.254+0000 7f6246ff5640 1 -- 172.21.15.12:0/1218305227 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c14f28ac10 con 0x55c14f3982c0 2024-04-22T13:29:43.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.254+0000 7f6246ff5640 10 client.17835 handle_command_reply: tid=0 2024-04-22T13:29:43.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.254+0000 7f626c1e1740 1 client.17835 shutdown 2024-04-22T13:29:43.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.254+0000 7f626c1e1740 1 -- 172.21.15.12:0/1218305227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14f318e80 msgr2=0x55c14f3253a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:43.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.254+0000 7f626c1e1740 1 --2- 172.21.15.12:0/1218305227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14f318e80 0x55c14f3253a0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f6254036040 tx=0x7f62540051a0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.254+0000 7f626c1e1740 1 -- 172.21.15.12:0/1218305227 shutdown_connections 2024-04-22T13:29:43.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.254+0000 7f626c1e1740 1 --2- 172.21.15.12:0/1218305227 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55c14f3982c0 0x55c14f39a6e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.254+0000 7f626c1e1740 1 --2- 172.21.15.12:0/1218305227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c14f321a00 0x55c14f3258e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.254+0000 7f626c1e1740 1 --2- 172.21.15.12:0/1218305227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c14f318e80 0x55c14f3253a0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.254+0000 7f6246ff5640 0 client.17835 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:29:43.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.254+0000 7f626c1e1740 1 --2- 172.21.15.12:0/1218305227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c14f3184d0 0x55c14f329fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.254+0000 7f626c1e1740 1 -- 172.21.15.12:0/1218305227 >> 172.21.15.12:0/1218305227 conn(0x55c14f312ba0 msgr2=0x55c14f313e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:43.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.254+0000 7f626c1e1740 1 -- 172.21.15.12:0/1218305227 shutdown_connections 2024-04-22T13:29:43.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.254+0000 7f626c1e1740 1 -- 172.21.15.12:0/1218305227 wait complete. 2024-04-22T13:29:43.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.254+0000 7f626c1e1740 20 client.17835 trim_cache size 0 max 16384 2024-04-22T13:29:43.275 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:29:43.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.255+0000 7f62695fb640 1 -- 172.21.15.12:0/2953825521 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f624c0a3d20 msgr2=0x7f624c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:43.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.255+0000 7f62695fb640 1 --2- 172.21.15.12:0/2953825521 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f624c0a3d20 0x7f624c0a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f625c0ce270 tx=0x7f625c06bda0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.255+0000 7f62695fb640 1 -- 172.21.15.12:0/2953825521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f626410c510 msgr2=0x7f62641985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:43.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.255+0000 7f62695fb640 1 --2- 172.21.15.12:0/2953825521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f626410c510 0x7f62641985a0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f625c009fd0 tx=0x7f625c00b260 comp rx=0 tx=0).stop 2024-04-22T13:29:43.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.255+0000 7f62695fb640 1 -- 172.21.15.12:0/2953825521 shutdown_connections 2024-04-22T13:29:43.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.255+0000 7f62695fb640 1 --2- 172.21.15.12:0/2953825521 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f624c0a3d20 0x7f624c0a61e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.255+0000 7f62695fb640 1 --2- 172.21.15.12:0/2953825521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f626410c510 0x7f62641985a0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.255+0000 7f62695fb640 1 --2- 172.21.15.12:0/2953825521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6264103af0 0x7f626419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.255+0000 7f62695fb640 1 --2- 172.21.15.12:0/2953825521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62641031d0 0x7f626419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.255+0000 7f62695fb640 1 -- 172.21.15.12:0/2953825521 >> 172.21.15.12:0/2953825521 conn(0x7f6264077f50 msgr2=0x7f62640fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:43.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.255+0000 7f62695fb640 1 -- 172.21.15.12:0/2953825521 shutdown_connections 2024-04-22T13:29:43.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.255+0000 7f62695fb640 1 -- 172.21.15.12:0/2953825521 wait complete. 2024-04-22T13:29:43.278 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:29:43.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.387+0000 7fe953559640 1 -- 172.21.15.12:0/1945445679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe94c1031b0 msgr2=0x7fe94c103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:43.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.387+0000 7fe953559640 1 --2- 172.21.15.12:0/1945445679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe94c1031b0 0x7fe94c103590 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fe948009900 tx=0x7fe94802bcd0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.387+0000 7fe953559640 1 -- 172.21.15.12:0/1945445679 shutdown_connections 2024-04-22T13:29:43.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.387+0000 7fe953559640 1 --2- 172.21.15.12:0/1945445679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe94c10c580 0x7fe94c10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.387+0000 7fe953559640 1 --2- 172.21.15.12:0/1945445679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe94c103ad0 0x7fe94c10bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.387+0000 7fe953559640 1 --2- 172.21.15.12:0/1945445679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe94c1031b0 0x7fe94c103590 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.387+0000 7fe953559640 1 -- 172.21.15.12:0/1945445679 >> 172.21.15.12:0/1945445679 conn(0x7fe94c0771d0 msgr2=0x7fe94c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:43.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.387+0000 7fe953559640 1 -- 172.21.15.12:0/1945445679 shutdown_connections 2024-04-22T13:29:43.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.387+0000 7fe953559640 1 -- 172.21.15.12:0/1945445679 wait complete. 2024-04-22T13:29:43.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.388+0000 7fe953559640 1 Processor -- start 2024-04-22T13:29:43.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.388+0000 7fe953559640 1 -- start start 2024-04-22T13:29:43.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.388+0000 7fe953559640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe94c1031b0 0x7fe94c19e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.388+0000 7fe953559640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe94c103ad0 0x7fe94c19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.388+0000 7fe953559640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe94c10c580 0x7fe94c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.388+0000 7fe953559640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c1121d0 con 0x7fe94c1031b0 2024-04-22T13:29:43.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.388+0000 7fe953559640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c112050 con 0x7fe94c10c580 2024-04-22T13:29:43.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.388+0000 7fe953559640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c112350 con 0x7fe94c103ad0 2024-04-22T13:29:43.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.388+0000 7fe9512f5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe94c1031b0 0x7fe94c19e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.388+0000 7fe9512f5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe94c1031b0 0x7fe94c19e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43564/0 (socket says 172.21.15.12:43564) 2024-04-22T13:29:43.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.388+0000 7fe9512f5640 1 -- 172.21.15.12:0/3923268416 learned_addr learned my addr 172.21.15.12:0/3923268416 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:43.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.388+0000 7fe951af6640 1 --2- 172.21.15.12:0/3923268416 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe94c10c580 0x7fe94c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.388+0000 7fe950af4640 1 --2- 172.21.15.12:0/3923268416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe94c103ad0 0x7fe94c19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.388+0000 7fe9512f5640 1 -- 172.21.15.12:0/3923268416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe94c103ad0 msgr2=0x7fe94c19e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:43.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.388+0000 7fe9512f5640 1 --2- 172.21.15.12:0/3923268416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe94c103ad0 0x7fe94c19e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.388+0000 7fe9512f5640 1 -- 172.21.15.12:0/3923268416 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe94c10c580 msgr2=0x7fe94c1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:43.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.388+0000 7fe9512f5640 1 --2- 172.21.15.12:0/3923268416 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe94c10c580 0x7fe94c1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.388+0000 7fe9512f5640 1 -- 172.21.15.12:0/3923268416 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe94c198d30 con 0x7fe94c1031b0 2024-04-22T13:29:43.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.389+0000 7fe9512f5640 1 --2- 172.21.15.12:0/3923268416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe94c1031b0 0x7fe94c19e400 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fe948005940 tx=0x7fe94802d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:43.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.389+0000 7fe9427fc640 1 -- 172.21.15.12:0/3923268416 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe948043020 con 0x7fe94c1031b0 2024-04-22T13:29:43.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.389+0000 7fe9427fc640 1 -- 172.21.15.12:0/3923268416 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe94802dc90 con 0x7fe94c1031b0 2024-04-22T13:29:43.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.389+0000 7fe953559640 1 -- 172.21.15.12:0/3923268416 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe94c198fc0 con 0x7fe94c1031b0 2024-04-22T13:29:43.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.389+0000 7fe9427fc640 1 -- 172.21.15.12:0/3923268416 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe948003ab0 con 0x7fe94c1031b0 2024-04-22T13:29:43.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.389+0000 7fe953559640 1 -- 172.21.15.12:0/3923268416 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe94c1a52c0 con 0x7fe94c1031b0 2024-04-22T13:29:43.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.391+0000 7fe9427fc640 1 -- 172.21.15.12:0/3923268416 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe948046020 con 0x7fe94c1031b0 2024-04-22T13:29:43.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.391+0000 7fe95622c740 1 Processor -- start 2024-04-22T13:29:43.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.392+0000 7fe95622c740 1 -- start start 2024-04-22T13:29:43.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.392+0000 7fe95622c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563225097330 0x563225097710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.392+0000 7fe95622c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563225097ce0 0x5632250a0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.392+0000 7fe95622c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632250a0e40 0x5632250a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.392+0000 7fe95622c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563224ffc1f0 con 0x5632250a0e40 2024-04-22T13:29:43.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.392+0000 7fe95622c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x563224f994b0 con 0x563225097330 2024-04-22T13:29:43.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.392+0000 7fe95622c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563224ff81b0 con 0x563225097ce0 2024-04-22T13:29:43.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.392+0000 7fe9427fc640 1 --2- 172.21.15.12:0/3923268416 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe9380a3da0 0x7fe9380a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.392+0000 7fe950af4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632250a0e40 0x5632250a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.392+0000 7fe950af4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632250a0e40 0x5632250a12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43566/0 (socket says 172.21.15.12:43566) 2024-04-22T13:29:43.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.392+0000 7fe950af4640 1 -- 172.21.15.12:0/2819668248 learned_addr learned my addr 172.21.15.12:0/2819668248 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:43.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.392+0000 7fe9512f5640 1 --2- 172.21.15.12:0/2819668248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563225097ce0 0x5632250a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.392+0000 7fe951af6640 1 --2- 172.21.15.12:0/3923268416 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe9380a3da0 0x7fe9380a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.392+0000 7fe951af6640 1 --2- 172.21.15.12:0/2819668248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563225097330 0x563225097710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.392+0000 7fe9427fc640 1 -- 172.21.15.12:0/3923268416 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fe9480f27f0 con 0x7fe94c1031b0 2024-04-22T13:29:43.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.392+0000 7fe950af4640 1 -- 172.21.15.12:0/2819668248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563225097ce0 msgr2=0x5632250a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:43.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.392+0000 7fe950af4640 1 --2- 172.21.15.12:0/2819668248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563225097ce0 0x5632250a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.392+0000 7fe950af4640 1 -- 172.21.15.12:0/2819668248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563225097330 msgr2=0x563225097710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:43.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.392+0000 7fe950af4640 1 --2- 172.21.15.12:0/2819668248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563225097330 0x563225097710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.392+0000 7fe950af4640 1 -- 172.21.15.12:0/2819668248 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563224f8b1b0 con 0x5632250a0e40 2024-04-22T13:29:43.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.392+0000 7fe9512f5640 1 --2- 172.21.15.12:0/2819668248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563225097ce0 0x5632250a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:43.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.392+0000 7fe951af6640 1 --2- 172.21.15.12:0/2819668248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563225097330 0x563225097710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:43.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.392+0000 7fe950af4640 1 --2- 172.21.15.12:0/2819668248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632250a0e40 0x5632250a12c0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00bc50 tx=0x7fe93c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316d27c924ea1119 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:43.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.393+0000 7fe951af6640 1 --2- 172.21.15.12:0/3923268416 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe9380a3da0 0x7fe9380a6260 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe944007fd0 tx=0x7fe944007a20 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:43.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.393+0000 7fe9377fe640 1 -- 172.21.15.12:0/2819668248 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c015070 con 0x5632250a0e40 2024-04-22T13:29:43.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.393+0000 7fe9377fe640 1 -- 172.21.15.12:0/2819668248 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe93c01a910 con 0x5632250a0e40 2024-04-22T13:29:43.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.393+0000 7fe9377fe640 1 -- 172.21.15.12:0/2819668248 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c011940 con 0x5632250a0e40 2024-04-22T13:29:43.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.393+0000 7fe95622c740 1 -- 172.21.15.12:0/2819668248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632250a0e40 msgr2=0x5632250a12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:43.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.393+0000 7fe95622c740 1 --2- 172.21.15.12:0/2819668248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632250a0e40 0x5632250a12c0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00bc50 tx=0x7fe93c00f730 comp rx=0 tx=0).stop 2024-04-22T13:29:43.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.393+0000 7fe95622c740 1 -- 172.21.15.12:0/2819668248 shutdown_connections 2024-04-22T13:29:43.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.393+0000 7fe95622c740 1 --2- 172.21.15.12:0/2819668248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632250a0e40 0x5632250a12c0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.393+0000 7fe95622c740 1 --2- 172.21.15.12:0/2819668248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563225097ce0 0x5632250a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.393+0000 7fe95622c740 1 --2- 172.21.15.12:0/2819668248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563225097330 0x563225097710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.393+0000 7fe95622c740 1 -- 172.21.15.12:0/2819668248 >> 172.21.15.12:0/2819668248 conn(0x563225091ba0 msgr2=0x563225091f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:43.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.393+0000 7fe95622c740 1 -- 172.21.15.12:0/2819668248 shutdown_connections 2024-04-22T13:29:43.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.393+0000 7fe95622c740 1 -- 172.21.15.12:0/2819668248 wait complete. 2024-04-22T13:29:43.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.394+0000 7fe95622c740 1 -- start start 2024-04-22T13:29:43.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.394+0000 7fe95622c740 1 Processor -- start 2024-04-22T13:29:43.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.394+0000 7fe95622c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563225097330 0x5632250a71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.394+0000 7fe95622c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563225097ce0 0x5632250a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.394+0000 7fe95622c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632250a0e40 0x5632250a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.394+0000 7fe95622c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5632250a7dc0 con 0x5632250a0e40 2024-04-22T13:29:43.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.394+0000 7fe95622c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x563224fb5500 con 0x563225097330 2024-04-22T13:29:43.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.394+0000 7fe95622c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5632250a7f40 con 0x563225097ce0 2024-04-22T13:29:43.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.394+0000 7fe95622c740 10 client.0 _ll_register_callbacks cb 0x563224f83b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:43.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.394+0000 7fe951af6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632250a0e40 0x5632250a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.394+0000 7fe951af6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632250a0e40 0x5632250a4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43570/0 (socket says 172.21.15.12:43570) 2024-04-22T13:29:43.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.394+0000 7fe951af6640 1 -- 172.21.15.12:0/2031263960 learned_addr learned my addr 172.21.15.12:0/2031263960 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:43.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.394+0000 7fe950af4640 1 --2- 172.21.15.12:0/2031263960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563225097ce0 0x5632250a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.395+0000 7fe9512f5640 1 --2- 172.21.15.12:0/2031263960 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563225097330 0x5632250a71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.395+0000 7fe951af6640 1 -- 172.21.15.12:0/2031263960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563225097ce0 msgr2=0x5632250a45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:43.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.395+0000 7fe951af6640 1 --2- 172.21.15.12:0/2031263960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563225097ce0 0x5632250a45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.395+0000 7fe951af6640 1 -- 172.21.15.12:0/2031263960 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563225097330 msgr2=0x5632250a71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:43.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.395+0000 7fe951af6640 1 --2- 172.21.15.12:0/2031263960 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563225097330 0x5632250a71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.395+0000 7fe951af6640 1 -- 172.21.15.12:0/2031263960 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563224d0e3f0 con 0x5632250a0e40 2024-04-22T13:29:43.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.395+0000 7fe9512f5640 1 --2- 172.21.15.12:0/2031263960 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563225097330 0x5632250a71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:43.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.395+0000 7fe951af6640 1 --2- 172.21.15.12:0/2031263960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632250a0e40 0x5632250a4b00 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fe94400fe00 tx=0x7fe944004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31cf0e9cda5546c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:43.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.395+0000 7fe934ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:43.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.395+0000 7fe934ff9640 1 -- 172.21.15.12:0/2031263960 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe944024030 con 0x5632250a0e40 2024-04-22T13:29:43.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.395+0000 7fe934ff9640 1 -- 172.21.15.12:0/2031263960 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fe944004b20 con 0x5632250a0e40 2024-04-22T13:29:43.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.395+0000 7fe95622c740 1 -- 172.21.15.12:0/2031263960 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe93c01a910 con 0x5632250a0e40 2024-04-22T13:29:43.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.395+0000 7fe934ff9640 1 -- 172.21.15.12:0/2031263960 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe944030050 con 0x5632250a0e40 2024-04-22T13:29:43.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.395+0000 7fe95622c740 10 client.17847 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:43.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.395+0000 7fe95622c740 1 -- 172.21.15.12:0/2031263960 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563224dd87d0 con 0x5632250a0e40 2024-04-22T13:29:43.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.395+0000 7fe934ff9640 1 -- 172.21.15.12:0/2031263960 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe93c01a910 con 0x5632250a0e40 2024-04-22T13:29:43.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.396+0000 7fe934ff9640 1 -- 172.21.15.12:0/2031263960 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe94401d050 con 0x5632250a0e40 2024-04-22T13:29:43.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.396+0000 7fe95622c740 10 client.17847 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:43.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.396+0000 7fe95622c740 10 client.17847 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:43.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.396+0000 7fe95622c740 20 client.17847 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:43.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.396+0000 7fe95622c740 1 --2- 172.21.15.12:0/2031263960 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5632250b5010 0x5632250b7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.396+0000 7fe95622c740 4 client.17847 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:43.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.396+0000 7fe95622c740 1 -- 172.21.15.12:0/2031263960 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563224f9b930 con 0x5632250b5010 2024-04-22T13:29:43.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.396+0000 7fe9512f5640 1 --2- 172.21.15.12:0/2031263960 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5632250b5010 0x5632250b7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.396+0000 7fe934ff9640 10 client.17847 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:43.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.396+0000 7fe9512f5640 1 --2- 172.21.15.12:0/2031263960 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5632250b5010 0x5632250b7490 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3038210aa924f03 server_cookie=3c0954f2212b667b in_seq=0 out_seq=0 2024-04-22T13:29:43.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.397+0000 7fe934ff9640 1 -- 172.21.15.12:0/2031263960 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563224f9b930 con 0x5632250b5010 2024-04-22T13:29:43.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.397+0000 7fe934ff9640 10 client.17847 handle_command_reply: tid=0 2024-04-22T13:29:43.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.397+0000 7fe95622c740 1 client.17847 shutdown 2024-04-22T13:29:43.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.397+0000 7fe95622c740 1 -- 172.21.15.12:0/2031263960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632250a0e40 msgr2=0x5632250a4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:43.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.397+0000 7fe95622c740 1 --2- 172.21.15.12:0/2031263960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632250a0e40 0x5632250a4b00 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fe94400fe00 tx=0x7fe944004a40 comp rx=0 tx=0).stop 2024-04-22T13:29:43.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.397+0000 7fe95622c740 1 -- 172.21.15.12:0/2031263960 shutdown_connections 2024-04-22T13:29:43.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.397+0000 7fe95622c740 1 --2- 172.21.15.12:0/2031263960 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5632250b5010 0x5632250b7490 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.397+0000 7fe95622c740 1 --2- 172.21.15.12:0/2031263960 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632250a0e40 0x5632250a4b00 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.397+0000 7fe95622c740 1 --2- 172.21.15.12:0/2031263960 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563225097ce0 0x5632250a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.397+0000 7fe95622c740 1 --2- 172.21.15.12:0/2031263960 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563225097330 0x5632250a71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.397+0000 7fe934ff9640 0 client.17847 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:43.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.397+0000 7fe95622c740 1 -- 172.21.15.12:0/2031263960 >> 172.21.15.12:0/2031263960 conn(0x563225091ba0 msgr2=0x5632250938f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:43.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.397+0000 7fe95622c740 1 -- 172.21.15.12:0/2031263960 shutdown_connections 2024-04-22T13:29:43.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.397+0000 7fe95622c740 1 -- 172.21.15.12:0/2031263960 wait complete. 2024-04-22T13:29:43.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.397+0000 7fe95622c740 20 client.17847 trim_cache size 0 max 16384 2024-04-22T13:29:43.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.411+0000 7fe95622c740 1 Processor -- start 2024-04-22T13:29:43.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.411+0000 7fe95622c740 1 -- start start 2024-04-22T13:29:43.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.411+0000 7fe95622c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563225097330 0x5632250b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.411+0000 7fe95622c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563225097ce0 0x563225090a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.411+0000 7fe95622c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632250a0e40 0x5632250a6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.411+0000 7fe95622c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c1ab7f0 con 0x5632250a0e40 2024-04-22T13:29:43.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.411+0000 7fe95622c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c1ad330 con 0x563225097330 2024-04-22T13:29:43.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.411+0000 7fe95622c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c1abec0 con 0x563225097ce0 2024-04-22T13:29:43.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.411+0000 7fe951af6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632250a0e40 0x5632250a6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.411+0000 7fe951af6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632250a0e40 0x5632250a6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43580/0 (socket says 172.21.15.12:43580) 2024-04-22T13:29:43.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.411+0000 7fe951af6640 1 -- 172.21.15.12:0/3585700329 learned_addr learned my addr 172.21.15.12:0/3585700329 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:43.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.411+0000 7fe9512f5640 1 --2- 172.21.15.12:0/3585700329 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563225097330 0x5632250b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.411+0000 7fe950af4640 1 --2- 172.21.15.12:0/3585700329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563225097ce0 0x563225090a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.411+0000 7fe951af6640 1 -- 172.21.15.12:0/3585700329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563225097ce0 msgr2=0x563225090a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:43.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.412+0000 7fe951af6640 1 --2- 172.21.15.12:0/3585700329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563225097ce0 0x563225090a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.412+0000 7fe951af6640 1 -- 172.21.15.12:0/3585700329 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563225097330 msgr2=0x5632250b0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:43.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.412+0000 7fe951af6640 1 --2- 172.21.15.12:0/3585700329 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563225097330 0x5632250b0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.412+0000 7fe951af6640 1 -- 172.21.15.12:0/3585700329 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563225096560 con 0x5632250a0e40 2024-04-22T13:29:43.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.412+0000 7fe951af6640 1 --2- 172.21.15.12:0/3585700329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632250a0e40 0x5632250a6ca0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fe944005ae0 tx=0x7fe944007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d1ed72a533f97f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:43.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.412+0000 7fe936ffd640 1 -- 172.21.15.12:0/3585700329 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe944024040 con 0x5632250a0e40 2024-04-22T13:29:43.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.412+0000 7fe936ffd640 1 -- 172.21.15.12:0/3585700329 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe944020930 con 0x5632250a0e40 2024-04-22T13:29:43.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.412+0000 7fe936ffd640 1 -- 172.21.15.12:0/3585700329 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9440298c0 con 0x5632250a0e40 2024-04-22T13:29:43.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.412+0000 7fe95622c740 1 -- 172.21.15.12:0/3585700329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632250a0e40 msgr2=0x5632250a6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:43.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.412+0000 7fe95622c740 1 --2- 172.21.15.12:0/3585700329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632250a0e40 0x5632250a6ca0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fe944005ae0 tx=0x7fe944007d20 comp rx=0 tx=0).stop 2024-04-22T13:29:43.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.412+0000 7fe95622c740 1 -- 172.21.15.12:0/3585700329 shutdown_connections 2024-04-22T13:29:43.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.412+0000 7fe95622c740 1 --2- 172.21.15.12:0/3585700329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632250a0e40 0x5632250a6ca0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.412+0000 7fe95622c740 1 --2- 172.21.15.12:0/3585700329 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563225097ce0 0x563225090a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.412+0000 7fe95622c740 1 --2- 172.21.15.12:0/3585700329 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563225097330 0x5632250b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.412+0000 7fe95622c740 1 -- 172.21.15.12:0/3585700329 >> 172.21.15.12:0/3585700329 conn(0x563225091ba0 msgr2=0x563225090370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:43.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.413+0000 7fe95622c740 1 -- 172.21.15.12:0/3585700329 shutdown_connections 2024-04-22T13:29:43.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.413+0000 7fe95622c740 1 -- 172.21.15.12:0/3585700329 wait complete. 2024-04-22T13:29:43.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.413+0000 7fe95622c740 1 -- start start 2024-04-22T13:29:43.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.413+0000 7fe95622c740 1 Processor -- start 2024-04-22T13:29:43.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.413+0000 7fe95622c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563225097330 0x5632250a6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.413+0000 7fe95622c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563225097ce0 0x5632250a6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.413+0000 7fe95622c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632250a0e40 0x5632250a6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.413+0000 7fe95622c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c1acc60 con 0x563225097330 2024-04-22T13:29:43.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.413+0000 7fe95622c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c1ac590 con 0x563225097ce0 2024-04-22T13:29:43.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.413+0000 7fe95622c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c1a97d0 con 0x5632250a0e40 2024-04-22T13:29:43.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.413+0000 7fe95622c740 10 client.0 _ll_register_callbacks cb 0x7fe9440078a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:43.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.413+0000 7fe9512f5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563225097330 0x5632250a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.413+0000 7fe9512f5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563225097330 0x5632250a6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43586/0 (socket says 172.21.15.12:43586) 2024-04-22T13:29:43.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.414+0000 7fe9512f5640 1 -- 172.21.15.12:0/2934451283 learned_addr learned my addr 172.21.15.12:0/2934451283 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:43.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.414+0000 7fe951af6640 1 --2- 172.21.15.12:0/2934451283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632250a0e40 0x5632250a6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.414+0000 7fe950af4640 1 --2- 172.21.15.12:0/2934451283 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563225097ce0 0x5632250a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.414+0000 7fe9512f5640 1 -- 172.21.15.12:0/2934451283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632250a0e40 msgr2=0x5632250a6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:43.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.414+0000 7fe9512f5640 1 --2- 172.21.15.12:0/2934451283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632250a0e40 0x5632250a6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.414+0000 7fe9512f5640 1 -- 172.21.15.12:0/2934451283 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563225097ce0 msgr2=0x5632250a6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:43.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.414+0000 7fe9512f5640 1 --2- 172.21.15.12:0/2934451283 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563225097ce0 0x5632250a6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.414+0000 7fe9512f5640 1 -- 172.21.15.12:0/2934451283 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56322511a680 con 0x563225097330 2024-04-22T13:29:43.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.414+0000 7fe9512f5640 1 --2- 172.21.15.12:0/2934451283 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563225097330 0x5632250a6220 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fe9480be590 tx=0x7fe94802bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceed320e313106ca server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:43.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.414+0000 7fe934ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:43.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.414+0000 7fe934ff9640 1 -- 172.21.15.12:0/2934451283 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9480c2c40 con 0x563225097330 2024-04-22T13:29:43.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.414+0000 7fe934ff9640 1 -- 172.21.15.12:0/2934451283 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fe948005340 con 0x563225097330 2024-04-22T13:29:43.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.414+0000 7fe95622c740 1 -- 172.21.15.12:0/2934451283 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe944020930 con 0x563225097330 2024-04-22T13:29:43.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.414+0000 7fe934ff9640 1 -- 172.21.15.12:0/2934451283 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe9480c7500 con 0x563225097330 2024-04-22T13:29:43.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.414+0000 7fe95622c740 10 client.17853 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:43.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.414+0000 7fe95622c740 1 -- 172.21.15.12:0/2934451283 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56322511b960 con 0x563225097330 2024-04-22T13:29:43.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.414+0000 7fe934ff9640 1 -- 172.21.15.12:0/2934451283 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe944020930 con 0x563225097330 2024-04-22T13:29:43.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.415+0000 7fe934ff9640 1 -- 172.21.15.12:0/2934451283 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe948003c50 con 0x563225097330 2024-04-22T13:29:43.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.415+0000 7fe95622c740 10 client.17853 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:43.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.415+0000 7fe95622c740 10 client.17853 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:43.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.415+0000 7fe95622c740 20 client.17853 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:43.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.415+0000 7fe95622c740 1 --2- 172.21.15.12:0/2934451283 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56322511c090 0x56322511e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:43.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.415+0000 7fe95622c740 4 client.17853 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:29:43.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.415+0000 7fe95622c740 1 -- 172.21.15.12:0/2934451283 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563225009c10 con 0x56322511c090 2024-04-22T13:29:43.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.415+0000 7fe950af4640 1 --2- 172.21.15.12:0/2934451283 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56322511c090 0x56322511e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:43.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.415+0000 7fe934ff9640 10 client.17853 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:43.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.415+0000 7fe950af4640 1 --2- 172.21.15.12:0/2934451283 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56322511c090 0x56322511e4b0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90c012d6212b53e1 server_cookie=a06a42833ec2f49a in_seq=0 out_seq=0 2024-04-22T13:29:43.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.416+0000 7fe934ff9640 1 -- 172.21.15.12:0/2934451283 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563225009c10 con 0x56322511c090 2024-04-22T13:29:43.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.416+0000 7fe934ff9640 10 client.17853 handle_command_reply: tid=0 2024-04-22T13:29:43.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.416+0000 7fe95622c740 1 client.17853 shutdown 2024-04-22T13:29:43.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.416+0000 7fe95622c740 1 -- 172.21.15.12:0/2934451283 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563225097330 msgr2=0x5632250a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:43.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.416+0000 7fe95622c740 1 --2- 172.21.15.12:0/2934451283 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563225097330 0x5632250a6220 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fe9480be590 tx=0x7fe94802bcd0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.416+0000 7fe95622c740 1 -- 172.21.15.12:0/2934451283 shutdown_connections 2024-04-22T13:29:43.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.416+0000 7fe95622c740 1 --2- 172.21.15.12:0/2934451283 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56322511c090 0x56322511e4b0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.416+0000 7fe95622c740 1 --2- 172.21.15.12:0/2934451283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5632250a0e40 0x5632250a6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.416+0000 7fe934ff9640 0 client.17853 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:43.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.416+0000 7fe95622c740 1 --2- 172.21.15.12:0/2934451283 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563225097ce0 0x5632250a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.416+0000 7fe95622c740 1 --2- 172.21.15.12:0/2934451283 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563225097330 0x5632250a6220 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.416+0000 7fe95622c740 1 -- 172.21.15.12:0/2934451283 >> 172.21.15.12:0/2934451283 conn(0x563225091ba0 msgr2=0x563225090c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:43.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.416+0000 7fe95622c740 1 -- 172.21.15.12:0/2934451283 shutdown_connections 2024-04-22T13:29:43.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.416+0000 7fe95622c740 1 -- 172.21.15.12:0/2934451283 wait complete. 2024-04-22T13:29:43.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.416+0000 7fe95622c740 20 client.17853 trim_cache size 0 max 16384 2024-04-22T13:29:43.418 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:29:43.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.416+0000 7fe953559640 1 -- 172.21.15.12:0/3923268416 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe9380a3da0 msgr2=0x7fe9380a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:43.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.417+0000 7fe953559640 1 --2- 172.21.15.12:0/3923268416 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe9380a3da0 0x7fe9380a6260 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe944007fd0 tx=0x7fe944007a20 comp rx=0 tx=0).stop 2024-04-22T13:29:43.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.417+0000 7fe953559640 1 -- 172.21.15.12:0/3923268416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe94c1031b0 msgr2=0x7fe94c19e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:43.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.417+0000 7fe953559640 1 --2- 172.21.15.12:0/3923268416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe94c1031b0 0x7fe94c19e400 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fe948005940 tx=0x7fe94802d890 comp rx=0 tx=0).stop 2024-04-22T13:29:43.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.417+0000 7fe953559640 1 -- 172.21.15.12:0/3923268416 shutdown_connections 2024-04-22T13:29:43.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.417+0000 7fe953559640 1 --2- 172.21.15.12:0/3923268416 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe9380a3da0 0x7fe9380a6260 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.417+0000 7fe953559640 1 --2- 172.21.15.12:0/3923268416 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe94c10c580 0x7fe94c1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.417+0000 7fe953559640 1 --2- 172.21.15.12:0/3923268416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe94c103ad0 0x7fe94c19e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.417+0000 7fe953559640 1 --2- 172.21.15.12:0/3923268416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe94c1031b0 0x7fe94c19e400 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:43.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.417+0000 7fe953559640 1 -- 172.21.15.12:0/3923268416 >> 172.21.15.12:0/3923268416 conn(0x7fe94c0771d0 msgr2=0x7fe94c0ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:43.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.417+0000 7fe953559640 1 -- 172.21.15.12:0/3923268416 shutdown_connections 2024-04-22T13:29:43.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:43.417+0000 7fe953559640 1 -- 172.21.15.12:0/3923268416 wait complete. 2024-04-22T13:29:44.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:43 smithi012 ceph-mon[28265]: pgmap v171: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:29:44.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:43 smithi134 ceph-mon[31338]: pgmap v171: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:29:44.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:43 smithi111 ceph-mon[31361]: pgmap v171: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:29:44.423 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:29:44.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:29:44.423 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:29:44.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.546+0000 7f20866ce640 1 -- 172.21.15.12:0/3730164817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2080106170 msgr2=0x7f208010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:44.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.546+0000 7f20866ce640 1 --2- 172.21.15.12:0/3730164817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2080106170 0x7f208010dd40 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f2078009ca0 tx=0x7f207802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.546+0000 7f20866ce640 1 -- 172.21.15.12:0/3730164817 shutdown_connections 2024-04-22T13:29:44.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.546+0000 7f20866ce640 1 --2- 172.21.15.12:0/3730164817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2080106170 0x7f208010dd40 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.546+0000 7f20866ce640 1 --2- 172.21.15.12:0/3730164817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2080101a50 0x7f2080105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.546+0000 7f20866ce640 1 --2- 172.21.15.12:0/3730164817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20801010a0 0x7f2080101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.546+0000 7f20866ce640 1 -- 172.21.15.12:0/3730164817 >> 172.21.15.12:0/3730164817 conn(0x7f2080077eb0 msgr2=0x7f20800782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:44.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.546+0000 7f20866ce640 1 -- 172.21.15.12:0/3730164817 shutdown_connections 2024-04-22T13:29:44.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.546+0000 7f20866ce640 1 -- 172.21.15.12:0/3730164817 wait complete. 2024-04-22T13:29:44.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.547+0000 7f20866ce640 1 Processor -- start 2024-04-22T13:29:44.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.547+0000 7f20866ce640 1 -- start start 2024-04-22T13:29:44.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.547+0000 7f20866ce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20801010a0 0x7f20801986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.547+0000 7f20866ce640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2080106170 0x7f2080198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.547+0000 7f20866ce640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f208019cf60 0x7f208019d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.547+0000 7f20866ce640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f20801121f0 con 0x7f2080106170 2024-04-22T13:29:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.547+0000 7f20866ce640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2080112070 con 0x7f20801010a0 2024-04-22T13:29:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.547+0000 7f20866ce640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2080112370 con 0x7f208019cf60 2024-04-22T13:29:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.547+0000 7f207f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2080106170 0x7f2080198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.547+0000 7f207f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2080106170 0x7f2080198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43600/0 (socket says 172.21.15.12:43600) 2024-04-22T13:29:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.547+0000 7f207f7fe640 1 -- 172.21.15.12:0/2267275280 learned_addr learned my addr 172.21.15.12:0/2267275280 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.547+0000 7f2084c6b640 1 --2- 172.21.15.12:0/2267275280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f208019cf60 0x7f208019d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.547+0000 7f207ffff640 1 --2- 172.21.15.12:0/2267275280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20801010a0 0x7f20801986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.548+0000 7f207f7fe640 1 -- 172.21.15.12:0/2267275280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f208019cf60 msgr2=0x7f208019d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.548+0000 7f207f7fe640 1 --2- 172.21.15.12:0/2267275280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f208019cf60 0x7f208019d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.548+0000 7f207f7fe640 1 -- 172.21.15.12:0/2267275280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20801010a0 msgr2=0x7f20801986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.548+0000 7f207f7fe640 1 --2- 172.21.15.12:0/2267275280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20801010a0 0x7f20801986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.548+0000 7f207f7fe640 1 -- 172.21.15.12:0/2267275280 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f208019da50 con 0x7f2080106170 2024-04-22T13:29:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.548+0000 7f207ffff640 1 --2- 172.21.15.12:0/2267275280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20801010a0 0x7f20801986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:29:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.548+0000 7f207f7fe640 1 --2- 172.21.15.12:0/2267275280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2080106170 0x7f2080198c30 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f207400e9f0 tx=0x7f207400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.548+0000 7f207d7fa640 1 -- 172.21.15.12:0/2267275280 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207400ccc0 con 0x7f2080106170 2024-04-22T13:29:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.548+0000 7f20866ce640 1 -- 172.21.15.12:0/2267275280 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f208019dd40 con 0x7f2080106170 2024-04-22T13:29:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.548+0000 7f207d7fa640 1 -- 172.21.15.12:0/2267275280 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20740054d0 con 0x7f2080106170 2024-04-22T13:29:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.548+0000 7f207d7fa640 1 -- 172.21.15.12:0/2267275280 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2074010430 con 0x7f2080106170 2024-04-22T13:29:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.548+0000 7f20866ce640 1 -- 172.21.15.12:0/2267275280 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20801a54e0 con 0x7f2080106170 2024-04-22T13:29:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.550+0000 7f207d7fa640 1 -- 172.21.15.12:0/2267275280 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f20740046c0 con 0x7f2080106170 2024-04-22T13:29:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.551+0000 7f2089395740 1 Processor -- start 2024-04-22T13:29:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.551+0000 7f2089395740 1 -- start start 2024-04-22T13:29:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.551+0000 7f2089395740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcab41a8e0 0x55fcab41acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.551+0000 7f2089395740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fcab41b290 0x55fcab423840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.551+0000 7f2089395740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcab423e10 0x55fcab426260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.551+0000 7f2089395740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fcab37ef00 con 0x55fcab423e10 2024-04-22T13:29:44.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.551+0000 7f2089395740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fcab31c590 con 0x55fcab41a8e0 2024-04-22T13:29:44.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.551+0000 7f2089395740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fcab37aec0 con 0x55fcab41b290 2024-04-22T13:29:44.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.551+0000 7f207d7fa640 1 --2- 172.21.15.12:0/2267275280 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f20680a3d20 0x7f20680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.551+0000 7f207f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcab423e10 0x55fcab426260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.551+0000 7f2084c6b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcab41a8e0 0x55fcab41acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.551+0000 7f207ffff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fcab41b290 0x55fcab423840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.551+0000 7f2084c6b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcab41a8e0 0x55fcab41acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:37020/0 (socket says 172.21.15.12:37020) 2024-04-22T13:29:44.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.551+0000 7f2084c6b640 1 -- 172.21.15.12:0/3224973487 learned_addr learned my addr 172.21.15.12:0/3224973487 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:44.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.551+0000 7f207d7fa640 1 -- 172.21.15.12:0/2267275280 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f20740d0b90 con 0x7f2080106170 2024-04-22T13:29:44.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.551+0000 7f2084c6b640 1 --2- 172.21.15.12:0/2267275280 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f20680a3d20 0x7f20680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.551+0000 7f207ffff640 1 -- 172.21.15.12:0/3224973487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcab41a8e0 msgr2=0x55fcab41acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:44.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.551+0000 7f207ffff640 1 --2- 172.21.15.12:0/3224973487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcab41a8e0 0x55fcab41acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.551+0000 7f207ffff640 1 -- 172.21.15.12:0/3224973487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcab423e10 msgr2=0x55fcab426260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:44.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.551+0000 7f207ffff640 1 --2- 172.21.15.12:0/3224973487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcab423e10 0x55fcab426260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.551+0000 7f207ffff640 1 -- 172.21.15.12:0/3224973487 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcab1597e0 con 0x55fcab41b290 2024-04-22T13:29:44.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.552+0000 7f2084c6b640 1 --2- 172.21.15.12:0/2267275280 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f20680a3d20 0x7f20680a61e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2078009a60 tx=0x7f2078005c50 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:44.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.552+0000 7f207ffff640 1 --2- 172.21.15.12:0/3224973487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fcab41b290 0x55fcab423840 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2070009c30 tx=0x7f20700117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d78481b71ab19174 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:44.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.552+0000 7f20667fc640 1 -- 172.21.15.12:0/3224973487 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2070012450 con 0x55fcab41b290 2024-04-22T13:29:44.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.552+0000 7f20667fc640 1 -- 172.21.15.12:0/3224973487 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2070012aa0 con 0x55fcab41b290 2024-04-22T13:29:44.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.552+0000 7f20667fc640 1 -- 172.21.15.12:0/3224973487 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207000f9c0 con 0x55fcab41b290 2024-04-22T13:29:44.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.553+0000 7f2089395740 1 -- 172.21.15.12:0/3224973487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fcab41b290 msgr2=0x55fcab423840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:44.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.553+0000 7f2089395740 1 --2- 172.21.15.12:0/3224973487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fcab41b290 0x55fcab423840 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2070009c30 tx=0x7f20700117b0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.553+0000 7f2089395740 1 -- 172.21.15.12:0/3224973487 shutdown_connections 2024-04-22T13:29:44.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.553+0000 7f2089395740 1 --2- 172.21.15.12:0/3224973487 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcab423e10 0x55fcab426260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.553+0000 7f2089395740 1 --2- 172.21.15.12:0/3224973487 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fcab41b290 0x55fcab423840 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.553+0000 7f2089395740 1 --2- 172.21.15.12:0/3224973487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcab41a8e0 0x55fcab41acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.553+0000 7f2089395740 1 -- 172.21.15.12:0/3224973487 >> 172.21.15.12:0/3224973487 conn(0x55fcab414fb0 msgr2=0x55fcab415390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:44.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.553+0000 7f2089395740 1 -- 172.21.15.12:0/3224973487 shutdown_connections 2024-04-22T13:29:44.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.553+0000 7f2089395740 1 -- 172.21.15.12:0/3224973487 wait complete. 2024-04-22T13:29:44.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.553+0000 7f2089395740 1 -- start start 2024-04-22T13:29:44.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.553+0000 7f2089395740 1 Processor -- start 2024-04-22T13:29:44.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.554+0000 7f2089395740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcab41a8e0 0x55fcab420da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.554+0000 7f2089395740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcab41b290 0x55fcab4212e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.554+0000 7f2089395740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fcab423e10 0x55fcab421820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.554+0000 7f2089395740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fcab426d80 con 0x55fcab41a8e0 2024-04-22T13:29:44.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.554+0000 7f2089395740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fcab338580 con 0x55fcab41b290 2024-04-22T13:29:44.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.554+0000 7f2089395740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fcab426f00 con 0x55fcab423e10 2024-04-22T13:29:44.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.554+0000 7f2089395740 10 client.0 _ll_register_callbacks cb 0x55fcab33e650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:44.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.554+0000 7f207ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcab41a8e0 0x55fcab420da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.554+0000 7f207ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcab41a8e0 0x55fcab420da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43620/0 (socket says 172.21.15.12:43620) 2024-04-22T13:29:44.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.554+0000 7f207ffff640 1 -- 172.21.15.12:0/914479174 learned_addr learned my addr 172.21.15.12:0/914479174 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:44.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.554+0000 7f2084c6b640 1 --2- 172.21.15.12:0/914479174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fcab423e10 0x55fcab421820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.554+0000 7f207f7fe640 1 --2- 172.21.15.12:0/914479174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcab41b290 0x55fcab4212e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.554+0000 7f207ffff640 1 -- 172.21.15.12:0/914479174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fcab423e10 msgr2=0x55fcab421820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:44.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.554+0000 7f207ffff640 1 --2- 172.21.15.12:0/914479174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fcab423e10 0x55fcab421820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.554+0000 7f207ffff640 1 -- 172.21.15.12:0/914479174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcab41b290 msgr2=0x55fcab4212e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:44.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.554+0000 7f207ffff640 1 --2- 172.21.15.12:0/914479174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcab41b290 0x55fcab4212e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.554+0000 7f207ffff640 1 -- 172.21.15.12:0/914479174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcab1b9f70 con 0x55fcab41a8e0 2024-04-22T13:29:44.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.554+0000 7f207ffff640 1 --2- 172.21.15.12:0/914479174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcab41a8e0 0x55fcab420da0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f2070000c00 tx=0x7f2070012f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4582ff9420a7eaae server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:44.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.554+0000 7f205f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:44.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.555+0000 7f205f7fe640 1 -- 172.21.15.12:0/914479174 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2070029020 con 0x55fcab41a8e0 2024-04-22T13:29:44.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.555+0000 7f205f7fe640 1 -- 172.21.15.12:0/914479174 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f20700126e0 con 0x55fcab41a8e0 2024-04-22T13:29:44.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.555+0000 7f2089395740 1 -- 172.21.15.12:0/914479174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2070012aa0 con 0x55fcab41a8e0 2024-04-22T13:29:44.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.555+0000 7f205f7fe640 1 -- 172.21.15.12:0/914479174 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2070034050 con 0x55fcab41a8e0 2024-04-22T13:29:44.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.555+0000 7f2089395740 10 client.17865 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:44.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.555+0000 7f2089395740 1 -- 172.21.15.12:0/914479174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fcab051170 con 0x55fcab41a8e0 2024-04-22T13:29:44.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.555+0000 7f205f7fe640 1 -- 172.21.15.12:0/914479174 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2070012aa0 con 0x55fcab41a8e0 2024-04-22T13:29:44.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.555+0000 7f205f7fe640 1 -- 172.21.15.12:0/914479174 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2070004c50 con 0x55fcab41a8e0 2024-04-22T13:29:44.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.555+0000 7f2089395740 10 client.17865 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:44.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.555+0000 7f2089395740 10 client.17865 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:44.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.555+0000 7f2089395740 20 client.17865 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:44.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.555+0000 7f2089395740 1 --2- 172.21.15.12:0/914479174 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55fcab4342c0 0x55fcab4366b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.555+0000 7f2089395740 4 client.17865 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:44.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.555+0000 7f2089395740 1 -- 172.21.15.12:0/914479174 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcab38c920 con 0x55fcab4342c0 2024-04-22T13:29:44.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.555+0000 7f207f7fe640 1 --2- 172.21.15.12:0/914479174 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55fcab4342c0 0x55fcab4366b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.556+0000 7f207f7fe640 1 --2- 172.21.15.12:0/914479174 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55fcab4342c0 0x55fcab4366b0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a6c9e9fb8942432 server_cookie=8bdf6313be487c8 in_seq=0 out_seq=0 2024-04-22T13:29:44.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.556+0000 7f205f7fe640 10 client.17865 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:44.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.556+0000 7f205f7fe640 1 -- 172.21.15.12:0/914479174 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fcab38c920 con 0x55fcab4342c0 2024-04-22T13:29:44.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.556+0000 7f205f7fe640 10 client.17865 handle_command_reply: tid=0 2024-04-22T13:29:44.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.557+0000 7f2089395740 1 client.17865 shutdown 2024-04-22T13:29:44.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.557+0000 7f2089395740 1 -- 172.21.15.12:0/914479174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcab41a8e0 msgr2=0x55fcab420da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:44.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.557+0000 7f2089395740 1 --2- 172.21.15.12:0/914479174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcab41a8e0 0x55fcab420da0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f2070000c00 tx=0x7f2070012f00 comp rx=0 tx=0).stop 2024-04-22T13:29:44.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.557+0000 7f2089395740 1 -- 172.21.15.12:0/914479174 shutdown_connections 2024-04-22T13:29:44.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.557+0000 7f2089395740 1 --2- 172.21.15.12:0/914479174 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55fcab4342c0 0x55fcab4366b0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.557+0000 7f2089395740 1 --2- 172.21.15.12:0/914479174 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fcab423e10 0x55fcab421820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.557+0000 7f2089395740 1 --2- 172.21.15.12:0/914479174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcab41b290 0x55fcab4212e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.557+0000 7f205f7fe640 0 client.17865 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:44.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.557+0000 7f2089395740 1 --2- 172.21.15.12:0/914479174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcab41a8e0 0x55fcab420da0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.557+0000 7f2089395740 1 -- 172.21.15.12:0/914479174 >> 172.21.15.12:0/914479174 conn(0x55fcab414fb0 msgr2=0x55fcab416d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:44.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.557+0000 7f2089395740 1 -- 172.21.15.12:0/914479174 shutdown_connections 2024-04-22T13:29:44.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.557+0000 7f2089395740 1 -- 172.21.15.12:0/914479174 wait complete. 2024-04-22T13:29:44.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.557+0000 7f2089395740 20 client.17865 trim_cache size 0 max 16384 2024-04-22T13:29:44.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.575+0000 7f2089395740 1 Processor -- start 2024-04-22T13:29:44.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.575+0000 7f2089395740 1 -- start start 2024-04-22T13:29:44.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.575+0000 7f2089395740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcab41a8e0 0x55fcab42fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.575+0000 7f2089395740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fcab41b290 0x55fcab42c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.575+0000 7f2089395740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcab423e10 0x55fcab420550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.575+0000 7f2089395740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f20801adab0 con 0x55fcab41a8e0 2024-04-22T13:29:44.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.575+0000 7f2089395740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f20801af5f0 con 0x55fcab423e10 2024-04-22T13:29:44.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.575+0000 7f2089395740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f20801ae180 con 0x55fcab41b290 2024-04-22T13:29:44.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.575+0000 7f207ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcab41a8e0 0x55fcab42fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.575+0000 7f207ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcab41a8e0 0x55fcab42fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43632/0 (socket says 172.21.15.12:43632) 2024-04-22T13:29:44.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.575+0000 7f207ffff640 1 -- 172.21.15.12:0/876253377 learned_addr learned my addr 172.21.15.12:0/876253377 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:44.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.575+0000 7f207f7fe640 1 --2- 172.21.15.12:0/876253377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fcab41b290 0x55fcab42c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.575+0000 7f207ffff640 1 -- 172.21.15.12:0/876253377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fcab41b290 msgr2=0x55fcab42c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:44.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.575+0000 7f207ffff640 1 --2- 172.21.15.12:0/876253377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fcab41b290 0x55fcab42c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.575+0000 7f207ffff640 1 -- 172.21.15.12:0/876253377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcab423e10 msgr2=0x55fcab420550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:44.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.575+0000 7f2084c6b640 1 --2- 172.21.15.12:0/876253377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcab423e10 0x55fcab420550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.575+0000 7f207ffff640 1 --2- 172.21.15.12:0/876253377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcab423e10 0x55fcab420550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.575+0000 7f207ffff640 1 -- 172.21.15.12:0/876253377 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcab414de0 con 0x55fcab41a8e0 2024-04-22T13:29:44.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.575+0000 7f207ffff640 1 --2- 172.21.15.12:0/876253377 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcab41a8e0 0x55fcab42fc70 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f2070012fd0 tx=0x7f2070009a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b9ca07e818de4f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:44.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.575+0000 7f2065ffb640 1 -- 172.21.15.12:0/876253377 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207003d070 con 0x55fcab41a8e0 2024-04-22T13:29:44.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.576+0000 7f2065ffb640 1 -- 172.21.15.12:0/876253377 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20700092b0 con 0x55fcab41a8e0 2024-04-22T13:29:44.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.576+0000 7f2065ffb640 1 -- 172.21.15.12:0/876253377 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2070029070 con 0x55fcab41a8e0 2024-04-22T13:29:44.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.576+0000 7f2089395740 1 -- 172.21.15.12:0/876253377 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcab41a8e0 msgr2=0x55fcab42fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:44.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.576+0000 7f2089395740 1 --2- 172.21.15.12:0/876253377 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcab41a8e0 0x55fcab42fc70 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f2070012fd0 tx=0x7f2070009a80 comp rx=0 tx=0).stop 2024-04-22T13:29:44.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.576+0000 7f2089395740 1 -- 172.21.15.12:0/876253377 shutdown_connections 2024-04-22T13:29:44.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.576+0000 7f2089395740 1 --2- 172.21.15.12:0/876253377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcab423e10 0x55fcab420550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.576+0000 7f2089395740 1 --2- 172.21.15.12:0/876253377 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fcab41b290 0x55fcab42c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.576+0000 7f2089395740 1 --2- 172.21.15.12:0/876253377 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcab41a8e0 0x55fcab42fc70 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.576+0000 7f2089395740 1 -- 172.21.15.12:0/876253377 >> 172.21.15.12:0/876253377 conn(0x55fcab414fb0 msgr2=0x55fcab413780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:44.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.576+0000 7f2089395740 1 -- 172.21.15.12:0/876253377 shutdown_connections 2024-04-22T13:29:44.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.576+0000 7f2089395740 1 -- 172.21.15.12:0/876253377 wait complete. 2024-04-22T13:29:44.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.576+0000 7f2089395740 1 -- start start 2024-04-22T13:29:44.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.577+0000 7f2089395740 1 Processor -- start 2024-04-22T13:29:44.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.577+0000 7f2089395740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcab41a8e0 0x55fcab497280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.577+0000 7f2089395740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fcab41b290 0x55fcab4977c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.577+0000 7f2089395740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcab423e10 0x55fcab43b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.577+0000 7f2089395740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f20801aef20 con 0x55fcab423e10 2024-04-22T13:29:44.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.577+0000 7f2089395740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f20801ae850 con 0x55fcab41a8e0 2024-04-22T13:29:44.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.577+0000 7f2089395740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f20801aba90 con 0x55fcab41b290 2024-04-22T13:29:44.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.577+0000 7f2089395740 10 client.0 _ll_register_callbacks cb 0x7f207400cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:44.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.577+0000 7f2084c6b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcab423e10 0x55fcab43b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.577+0000 7f2084c6b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcab423e10 0x55fcab43b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43644/0 (socket says 172.21.15.12:43644) 2024-04-22T13:29:44.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.577+0000 7f2084c6b640 1 -- 172.21.15.12:0/2171964829 learned_addr learned my addr 172.21.15.12:0/2171964829 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:44.577 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.577+0000 7f207f7fe640 1 --2- 172.21.15.12:0/2171964829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fcab41b290 0x55fcab4977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.577+0000 7f207ffff640 1 --2- 172.21.15.12:0/2171964829 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcab41a8e0 0x55fcab497280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.577+0000 7f2084c6b640 1 -- 172.21.15.12:0/2171964829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fcab41b290 msgr2=0x55fcab4977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:44.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.577+0000 7f2084c6b640 1 --2- 172.21.15.12:0/2171964829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fcab41b290 0x55fcab4977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.577+0000 7f2084c6b640 1 -- 172.21.15.12:0/2171964829 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcab41a8e0 msgr2=0x55fcab497280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:44.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.577+0000 7f2084c6b640 1 --2- 172.21.15.12:0/2171964829 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcab41a8e0 0x55fcab497280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.577+0000 7f2084c6b640 1 -- 172.21.15.12:0/2171964829 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcab43c0c0 con 0x55fcab423e10 2024-04-22T13:29:44.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.578+0000 7f2084c6b640 1 --2- 172.21.15.12:0/2171964829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcab423e10 0x55fcab43b8c0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f2078037040 tx=0x7f207802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a007fb5257ec4d0b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:44.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.578+0000 7f205f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:44.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.578+0000 7f205f7fe640 1 -- 172.21.15.12:0/2171964829 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2078056060 con 0x55fcab423e10 2024-04-22T13:29:44.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.578+0000 7f205f7fe640 1 -- 172.21.15.12:0/2171964829 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f2078004040 con 0x55fcab423e10 2024-04-22T13:29:44.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.578+0000 7f2089395740 1 -- 172.21.15.12:0/2171964829 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20700092b0 con 0x55fcab423e10 2024-04-22T13:29:44.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.578+0000 7f205f7fe640 1 -- 172.21.15.12:0/2171964829 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2078039040 con 0x55fcab423e10 2024-04-22T13:29:44.578 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.578+0000 7f2089395740 10 client.17871 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:44.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.578+0000 7f2089395740 1 -- 172.21.15.12:0/2171964829 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fcab49ea70 con 0x55fcab423e10 2024-04-22T13:29:44.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.578+0000 7f205f7fe640 1 -- 172.21.15.12:0/2171964829 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20700092b0 con 0x55fcab423e10 2024-04-22T13:29:44.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.578+0000 7f205f7fe640 1 -- 172.21.15.12:0/2171964829 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2078051bc0 con 0x55fcab423e10 2024-04-22T13:29:44.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.578+0000 7f2089395740 10 client.17871 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:44.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.578+0000 7f2089395740 10 client.17871 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:44.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.578+0000 7f2089395740 20 client.17871 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:44.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.578+0000 7f2089395740 1 --2- 172.21.15.12:0/2171964829 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55fcab49f1c0 0x55fcab4a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.578+0000 7f2089395740 4 client.17871 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:29:44.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.578+0000 7f2089395740 1 -- 172.21.15.12:0/2171964829 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fcab386790 con 0x55fcab49f1c0 2024-04-22T13:29:44.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.579+0000 7f207ffff640 1 --2- 172.21.15.12:0/2171964829 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55fcab49f1c0 0x55fcab4a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.579+0000 7f205f7fe640 10 client.17871 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:44.579 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.579+0000 7f207ffff640 1 --2- 172.21.15.12:0/2171964829 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55fcab49f1c0 0x55fcab4a15e0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6529083fbd1690dd server_cookie=de6c75f7cbbb7654 in_seq=0 out_seq=0 2024-04-22T13:29:44.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.580+0000 7f205f7fe640 1 -- 172.21.15.12:0/2171964829 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fcab386790 con 0x55fcab49f1c0 2024-04-22T13:29:44.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.580+0000 7f205f7fe640 10 client.17871 handle_command_reply: tid=0 2024-04-22T13:29:44.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.580+0000 7f2089395740 1 client.17871 shutdown 2024-04-22T13:29:44.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.580+0000 7f2089395740 1 -- 172.21.15.12:0/2171964829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcab423e10 msgr2=0x55fcab43b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:44.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.580+0000 7f2089395740 1 --2- 172.21.15.12:0/2171964829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcab423e10 0x55fcab43b8c0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f2078037040 tx=0x7f207802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.580+0000 7f2089395740 1 -- 172.21.15.12:0/2171964829 shutdown_connections 2024-04-22T13:29:44.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.580+0000 7f2089395740 1 --2- 172.21.15.12:0/2171964829 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55fcab49f1c0 0x55fcab4a15e0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.580+0000 7f2089395740 1 --2- 172.21.15.12:0/2171964829 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fcab423e10 0x55fcab43b8c0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.580+0000 7f205f7fe640 0 client.17871 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:44.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.580+0000 7f2089395740 1 --2- 172.21.15.12:0/2171964829 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fcab41b290 0x55fcab4977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.580+0000 7f2089395740 1 --2- 172.21.15.12:0/2171964829 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fcab41a8e0 0x55fcab497280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.580+0000 7f2089395740 1 -- 172.21.15.12:0/2171964829 >> 172.21.15.12:0/2171964829 conn(0x55fcab414fb0 msgr2=0x55fcab414010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:44.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.581+0000 7f2089395740 1 -- 172.21.15.12:0/2171964829 shutdown_connections 2024-04-22T13:29:44.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.581+0000 7f2089395740 1 -- 172.21.15.12:0/2171964829 wait complete. 2024-04-22T13:29:44.582 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.581+0000 7f2089395740 20 client.17871 trim_cache size 0 max 16384 2024-04-22T13:29:44.582 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:29:44.582 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-04-22T13:29:44.582 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "eb5de7d9-71dd-4c2a-9a1f-b60c2dd18b48", 2024-04-22T13:29:44.582 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-04-22T13:29:44.582 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:29:44.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.581+0000 7f20866ce640 1 -- 172.21.15.12:0/2267275280 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f20680a3d20 msgr2=0x7f20680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:44.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.581+0000 7f20866ce640 1 --2- 172.21.15.12:0/2267275280 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f20680a3d20 0x7f20680a61e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2078009a60 tx=0x7f2078005c50 comp rx=0 tx=0).stop 2024-04-22T13:29:44.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.581+0000 7f20866ce640 1 -- 172.21.15.12:0/2267275280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2080106170 msgr2=0x7f2080198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:44.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.581+0000 7f20866ce640 1 --2- 172.21.15.12:0/2267275280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2080106170 0x7f2080198c30 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f207400e9f0 tx=0x7f207400eeb0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.581+0000 7f20866ce640 1 -- 172.21.15.12:0/2267275280 shutdown_connections 2024-04-22T13:29:44.583 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.581+0000 7f20866ce640 1 --2- 172.21.15.12:0/2267275280 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f20680a3d20 0x7f20680a61e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.581+0000 7f20866ce640 1 --2- 172.21.15.12:0/2267275280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f208019cf60 0x7f208019d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.581+0000 7f20866ce640 1 --2- 172.21.15.12:0/2267275280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2080106170 0x7f2080198c30 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.581+0000 7f20866ce640 1 --2- 172.21.15.12:0/2267275280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20801010a0 0x7f20801986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.581+0000 7f20866ce640 1 -- 172.21.15.12:0/2267275280 >> 172.21.15.12:0/2267275280 conn(0x7f2080077eb0 msgr2=0x7f2080100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:44.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.581+0000 7f20866ce640 1 -- 172.21.15.12:0/2267275280 shutdown_connections 2024-04-22T13:29:44.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.581+0000 7f20866ce640 1 -- 172.21.15.12:0/2267275280 wait complete. 2024-04-22T13:29:44.588 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:29:44.588 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:29:44.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.696+0000 7f8f4457e640 1 -- 172.21.15.12:0/2664815684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f3c06aed0 msgr2=0x7f8f3c10e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:44.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.696+0000 7f8f4457e640 1 --2- 172.21.15.12:0/2664815684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f3c06aed0 0x7f8f3c10e710 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c009930 tx=0x7f8f2c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:29:44.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.696+0000 7f8f4457e640 1 -- 172.21.15.12:0/2664815684 shutdown_connections 2024-04-22T13:29:44.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.696+0000 7f8f4457e640 1 --2- 172.21.15.12:0/2664815684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f3c10ec50 0x7f8f3c111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.696+0000 7f8f4457e640 1 --2- 172.21.15.12:0/2664815684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f3c06aed0 0x7f8f3c10e710 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.696+0000 7f8f4457e640 1 --2- 172.21.15.12:0/2664815684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f3c06a5b0 0x7f8f3c06a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.696+0000 7f8f4457e640 1 -- 172.21.15.12:0/2664815684 >> 172.21.15.12:0/2664815684 conn(0x7f8f3c077c40 msgr2=0x7f8f3c078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:44.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.696+0000 7f8f4457e640 1 -- 172.21.15.12:0/2664815684 shutdown_connections 2024-04-22T13:29:44.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.696+0000 7f8f4457e640 1 -- 172.21.15.12:0/2664815684 wait complete. 2024-04-22T13:29:44.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.697+0000 7f8f4457e640 1 Processor -- start 2024-04-22T13:29:44.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.697+0000 7f8f4457e640 1 -- start start 2024-04-22T13:29:44.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.697+0000 7f8f4457e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f3c06a5b0 0x7f8f3c1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.697+0000 7f8f4457e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f3c06aed0 0x7f8f3c1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.697+0000 7f8f4457e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f3c10ec50 0x7f8f3c19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.697+0000 7f8f4457e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c1143d0 con 0x7f8f3c10ec50 2024-04-22T13:29:44.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.697+0000 7f8f4457e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c114250 con 0x7f8f3c06a5b0 2024-04-22T13:29:44.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.697+0000 7f8f4457e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c114550 con 0x7f8f3c06aed0 2024-04-22T13:29:44.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.697+0000 7f8f42b1b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f3c10ec50 0x7f8f3c19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.697+0000 7f8f42b1b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f3c10ec50 0x7f8f3c19a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43674/0 (socket says 172.21.15.12:43674) 2024-04-22T13:29:44.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.697+0000 7f8f42b1b640 1 -- 172.21.15.12:0/1977238330 learned_addr learned my addr 172.21.15.12:0/1977238330 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:44.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.697+0000 7f8f41b19640 1 --2- 172.21.15.12:0/1977238330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f3c06aed0 0x7f8f3c1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.697+0000 7f8f4231a640 1 --2- 172.21.15.12:0/1977238330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f3c06a5b0 0x7f8f3c1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.698+0000 7f8f42b1b640 1 -- 172.21.15.12:0/1977238330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f3c06aed0 msgr2=0x7f8f3c1a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:44.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.698+0000 7f8f42b1b640 1 --2- 172.21.15.12:0/1977238330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f3c06aed0 0x7f8f3c1a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.698+0000 7f8f42b1b640 1 -- 172.21.15.12:0/1977238330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f3c06a5b0 msgr2=0x7f8f3c1a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:44.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.698+0000 7f8f42b1b640 1 --2- 172.21.15.12:0/1977238330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f3c06a5b0 0x7f8f3c1a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.698+0000 7f8f42b1b640 1 -- 172.21.15.12:0/1977238330 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3c19b010 con 0x7f8f3c10ec50 2024-04-22T13:29:44.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.698+0000 7f8f42b1b640 1 --2- 172.21.15.12:0/1977238330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f3c10ec50 0x7f8f3c19a750 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009c90 tx=0x7f8f3400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:44.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.698+0000 7f8f337fe640 1 -- 172.21.15.12:0/1977238330 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f34003b70 con 0x7f8f3c10ec50 2024-04-22T13:29:44.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.698+0000 7f8f337fe640 1 -- 172.21.15.12:0/1977238330 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f34003d10 con 0x7f8f3c10ec50 2024-04-22T13:29:44.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.698+0000 7f8f4457e640 1 -- 172.21.15.12:0/1977238330 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f3c19b300 con 0x7f8f3c10ec50 2024-04-22T13:29:44.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.698+0000 7f8f337fe640 1 -- 172.21.15.12:0/1977238330 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f34010850 con 0x7f8f3c10ec50 2024-04-22T13:29:44.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.698+0000 7f8f4457e640 1 -- 172.21.15.12:0/1977238330 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f3c1a75c0 con 0x7f8f3c10ec50 2024-04-22T13:29:44.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.700+0000 7f8f337fe640 1 -- 172.21.15.12:0/1977238330 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8f34004020 con 0x7f8f3c10ec50 2024-04-22T13:29:44.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.701+0000 7f8f4716c740 1 Processor -- start 2024-04-22T13:29:44.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.701+0000 7f8f4716c740 1 -- start start 2024-04-22T13:29:44.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.701+0000 7f8f4716c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1389e5580 0x55c1389e5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.701+0000 7f8f4716c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1389e5ea0 0x55c1389ee540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.701+0000 7f8f4716c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1389eea80 0x55c1389f0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.701+0000 7f8f4716c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c13894a1e0 con 0x55c1389e5ea0 2024-04-22T13:29:44.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.701+0000 7f8f4716c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c1388e7390 con 0x55c1389eea80 2024-04-22T13:29:44.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.701+0000 7f8f4716c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c1389461a0 con 0x55c1389e5580 2024-04-22T13:29:44.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.701+0000 7f8f337fe640 1 --2- 172.21.15.12:0/1977238330 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f8f280a3d20 0x7f8f280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.701+0000 7f8f4231a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1389e5ea0 0x55c1389ee540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.701+0000 7f8f4231a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1389e5ea0 0x55c1389ee540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43676/0 (socket says 172.21.15.12:43676) 2024-04-22T13:29:44.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.701+0000 7f8f4231a640 1 -- 172.21.15.12:0/93951232 learned_addr learned my addr 172.21.15.12:0/93951232 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:44.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.701+0000 7f8f41b19640 1 --2- 172.21.15.12:0/93951232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1389e5580 0x55c1389e5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.701+0000 7f8f42b1b640 1 --2- 172.21.15.12:0/1977238330 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f8f280a3d20 0x7f8f280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.701+0000 7f8f41b19640 1 --2- 172.21.15.12:0/93951232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1389eea80 0x55c1389f0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.702+0000 7f8f337fe640 1 -- 172.21.15.12:0/1977238330 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f8f340d0de0 con 0x7f8f3c10ec50 2024-04-22T13:29:44.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.702+0000 7f8f4231a640 1 -- 172.21.15.12:0/93951232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1389e5580 msgr2=0x55c1389e5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:44.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.702+0000 7f8f4231a640 1 --2- 172.21.15.12:0/93951232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1389e5580 0x55c1389e5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.702+0000 7f8f4231a640 1 -- 172.21.15.12:0/93951232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1389eea80 msgr2=0x55c1389f0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:44.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.702+0000 7f8f4231a640 1 --2- 172.21.15.12:0/93951232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1389eea80 0x55c1389f0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.702+0000 7f8f4231a640 1 -- 172.21.15.12:0/93951232 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1387267d0 con 0x55c1389e5ea0 2024-04-22T13:29:44.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.702+0000 7f8f4231a640 1 --2- 172.21.15.12:0/93951232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1389e5ea0 0x55c1389ee540 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009cf0 tx=0x7f8f38011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739c73edfa8e0ba1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:44.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.702+0000 7f8f42b1b640 1 --2- 172.21.15.12:0/1977238330 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f8f280a3d20 0x7f8f280a61e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8f3409cd00 tx=0x7f8f34009f90 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:44.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.702+0000 7f8f30bfe640 1 -- 172.21.15.12:0/93951232 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3801e070 con 0x55c1389e5ea0 2024-04-22T13:29:44.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.702+0000 7f8f30bfe640 1 -- 172.21.15.12:0/93951232 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f38009de0 con 0x55c1389e5ea0 2024-04-22T13:29:44.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.702+0000 7f8f30bfe640 1 -- 172.21.15.12:0/93951232 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38019650 con 0x55c1389e5ea0 2024-04-22T13:29:44.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.703+0000 7f8f4716c740 1 -- 172.21.15.12:0/93951232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1389e5ea0 msgr2=0x55c1389ee540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:44.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.703+0000 7f8f4716c740 1 --2- 172.21.15.12:0/93951232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1389e5ea0 0x55c1389ee540 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009cf0 tx=0x7f8f38011810 comp rx=0 tx=0).stop 2024-04-22T13:29:44.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.703+0000 7f8f4716c740 1 -- 172.21.15.12:0/93951232 shutdown_connections 2024-04-22T13:29:44.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.703+0000 7f8f4716c740 1 --2- 172.21.15.12:0/93951232 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1389eea80 0x55c1389f0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.703+0000 7f8f4716c740 1 --2- 172.21.15.12:0/93951232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1389e5ea0 0x55c1389ee540 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.703+0000 7f8f4716c740 1 --2- 172.21.15.12:0/93951232 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1389e5580 0x55c1389e5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.703+0000 7f8f4716c740 1 -- 172.21.15.12:0/93951232 >> 172.21.15.12:0/93951232 conn(0x55c1389dfcb0 msgr2=0x55c1389e0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:44.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.703+0000 7f8f4716c740 1 -- 172.21.15.12:0/93951232 shutdown_connections 2024-04-22T13:29:44.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.703+0000 7f8f4716c740 1 -- 172.21.15.12:0/93951232 wait complete. 2024-04-22T13:29:44.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.703+0000 7f8f4716c740 1 -- start start 2024-04-22T13:29:44.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.703+0000 7f8f4716c740 1 Processor -- start 2024-04-22T13:29:44.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.703+0000 7f8f4716c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1389e5580 0x55c1389eba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.703+0000 7f8f4716c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1389e5ea0 0x55c1389ebfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.703+0000 7f8f4716c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1389eea80 0x55c1389ec4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.703+0000 7f8f4716c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c1389f1870 con 0x55c1389e5ea0 2024-04-22T13:29:44.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.703+0000 7f8f4716c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c1389032f0 con 0x55c1389e5580 2024-04-22T13:29:44.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.703+0000 7f8f4716c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c1389f19f0 con 0x55c1389eea80 2024-04-22T13:29:44.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.704+0000 7f8f4716c740 10 client.0 _ll_register_callbacks cb 0x55c138910580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:44.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.704+0000 7f8f4231a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1389e5ea0 0x55c1389ebfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.704+0000 7f8f4231a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1389e5ea0 0x55c1389ebfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43680/0 (socket says 172.21.15.12:43680) 2024-04-22T13:29:44.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.704+0000 7f8f4231a640 1 -- 172.21.15.12:0/2893746684 learned_addr learned my addr 172.21.15.12:0/2893746684 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:44.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.704+0000 7f8f41b19640 1 --2- 172.21.15.12:0/2893746684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1389e5580 0x55c1389eba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.704+0000 7f8f41b19640 1 --2- 172.21.15.12:0/2893746684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1389eea80 0x55c1389ec4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.704+0000 7f8f4231a640 1 -- 172.21.15.12:0/2893746684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1389eea80 msgr2=0x55c1389ec4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:44.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.704+0000 7f8f4231a640 1 --2- 172.21.15.12:0/2893746684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1389eea80 0x55c1389ec4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.704+0000 7f8f4231a640 1 -- 172.21.15.12:0/2893746684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1389e5580 msgr2=0x55c1389eba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:44.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.704+0000 7f8f4231a640 1 --2- 172.21.15.12:0/2893746684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1389e5580 0x55c1389eba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.704+0000 7f8f4231a640 1 -- 172.21.15.12:0/2893746684 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c13865c3f0 con 0x55c1389e5ea0 2024-04-22T13:29:44.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.704+0000 7f8f41b19640 1 --2- 172.21.15.12:0/2893746684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1389eea80 0x55c1389ec4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:29:44.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.704+0000 7f8f4231a640 1 --2- 172.21.15.12:0/2893746684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1389e5ea0 0x55c1389ebfa0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f8f380058c0 tx=0x7f8f380052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe8cb2bd144fc3f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:44.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.704+0000 7f8f25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:44.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.704+0000 7f8f25ffb640 1 -- 172.21.15.12:0/2893746684 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3801e040 con 0x55c1389e5ea0 2024-04-22T13:29:44.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.704+0000 7f8f25ffb640 1 -- 172.21.15.12:0/2893746684 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f8f38016070 con 0x55c1389e5ea0 2024-04-22T13:29:44.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.704+0000 7f8f4716c740 1 -- 172.21.15.12:0/2893746684 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f38009de0 con 0x55c1389e5ea0 2024-04-22T13:29:44.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.704+0000 7f8f25ffb640 1 -- 172.21.15.12:0/2893746684 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8f380054e0 con 0x55c1389e5ea0 2024-04-22T13:29:44.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.704+0000 7f8f4716c740 10 client.17883 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:44.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.704+0000 7f8f4716c740 1 -- 172.21.15.12:0/2893746684 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c13861c0f0 con 0x55c1389e5ea0 2024-04-22T13:29:44.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.705+0000 7f8f25ffb640 1 -- 172.21.15.12:0/2893746684 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f38009de0 con 0x55c1389e5ea0 2024-04-22T13:29:44.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.705+0000 7f8f25ffb640 1 -- 172.21.15.12:0/2893746684 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8f38021070 con 0x55c1389e5ea0 2024-04-22T13:29:44.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.705+0000 7f8f4716c740 10 client.17883 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:44.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.705+0000 7f8f4716c740 10 client.17883 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:44.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.705+0000 7f8f4716c740 20 client.17883 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:44.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.705+0000 7f8f4716c740 1 --2- 172.21.15.12:0/2893746684 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c1389fe9e0 0x55c138a00dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.705+0000 7f8f4716c740 4 client.17883 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:29:44.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.705+0000 7f8f4716c740 1 -- 172.21.15.12:0/2893746684 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c138957c00 con 0x55c1389fe9e0 2024-04-22T13:29:44.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.705+0000 7f8f41b19640 1 --2- 172.21.15.12:0/2893746684 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c1389fe9e0 0x55c138a00dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.705+0000 7f8f25ffb640 10 client.17883 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:44.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.705+0000 7f8f41b19640 1 --2- 172.21.15.12:0/2893746684 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c1389fe9e0 0x55c138a00dd0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0d339c313a02d86 server_cookie=921835f49e9e17a in_seq=0 out_seq=0 2024-04-22T13:29:44.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.706+0000 7f8f25ffb640 1 -- 172.21.15.12:0/2893746684 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c138957c00 con 0x55c1389fe9e0 2024-04-22T13:29:44.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.706+0000 7f8f25ffb640 10 client.17883 handle_command_reply: tid=0 2024-04-22T13:29:44.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.706+0000 7f8f4716c740 1 client.17883 shutdown 2024-04-22T13:29:44.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.706+0000 7f8f4716c740 1 -- 172.21.15.12:0/2893746684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1389e5ea0 msgr2=0x55c1389ebfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:44.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.706+0000 7f8f4716c740 1 --2- 172.21.15.12:0/2893746684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1389e5ea0 0x55c1389ebfa0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f8f380058c0 tx=0x7f8f380052f0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.706+0000 7f8f4716c740 1 -- 172.21.15.12:0/2893746684 shutdown_connections 2024-04-22T13:29:44.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.706+0000 7f8f4716c740 1 --2- 172.21.15.12:0/2893746684 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c1389fe9e0 0x55c138a00dd0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.706+0000 7f8f4716c740 1 --2- 172.21.15.12:0/2893746684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1389eea80 0x55c1389ec4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.706+0000 7f8f4716c740 1 --2- 172.21.15.12:0/2893746684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1389e5ea0 0x55c1389ebfa0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.706+0000 7f8f25ffb640 0 client.17883 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:44.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.706+0000 7f8f4716c740 1 --2- 172.21.15.12:0/2893746684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1389e5580 0x55c1389eba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.706+0000 7f8f4716c740 1 -- 172.21.15.12:0/2893746684 >> 172.21.15.12:0/2893746684 conn(0x55c1389dfcb0 msgr2=0x55c1389e1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:44.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.706+0000 7f8f4716c740 1 -- 172.21.15.12:0/2893746684 shutdown_connections 2024-04-22T13:29:44.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.706+0000 7f8f4716c740 1 -- 172.21.15.12:0/2893746684 wait complete. 2024-04-22T13:29:44.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.707+0000 7f8f4716c740 20 client.17883 trim_cache size 0 max 16384 2024-04-22T13:29:44.725 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.724+0000 7f8f4716c740 1 Processor -- start 2024-04-22T13:29:44.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.724+0000 7f8f4716c740 1 -- start start 2024-04-22T13:29:44.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.724+0000 7f8f4716c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1389e5580 0x55c1389fa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.724+0000 7f8f4716c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1389e5ea0 0x55c1389ed890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.724+0000 7f8f4716c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1389eea80 0x55c138a06580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.724+0000 7f8f4716c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c1afc80 con 0x55c1389eea80 2024-04-22T13:29:44.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.724+0000 7f8f4716c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c1b17c0 con 0x55c1389e5580 2024-04-22T13:29:44.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.724+0000 7f8f4716c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c1b0350 con 0x55c1389e5ea0 2024-04-22T13:29:44.726 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.724+0000 7f8f41b19640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1389eea80 0x55c138a06580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.724+0000 7f8f41b19640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1389eea80 0x55c138a06580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43682/0 (socket says 172.21.15.12:43682) 2024-04-22T13:29:44.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.724+0000 7f8f41b19640 1 -- 172.21.15.12:0/532174443 learned_addr learned my addr 172.21.15.12:0/532174443 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:44.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.724+0000 7f8f41b19640 1 -- 172.21.15.12:0/532174443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1389e5ea0 msgr2=0x55c1389ed890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:44.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.724+0000 7f8f41b19640 1 --2- 172.21.15.12:0/532174443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1389e5ea0 0x55c1389ed890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.725+0000 7f8f41b19640 1 -- 172.21.15.12:0/532174443 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1389e5580 msgr2=0x55c1389fa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:44.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.725+0000 7f8f41b19640 1 --2- 172.21.15.12:0/532174443 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1389e5580 0x55c1389fa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.725+0000 7f8f41b19640 1 -- 172.21.15.12:0/532174443 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1389e4d20 con 0x55c1389eea80 2024-04-22T13:29:44.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.725+0000 7f8f41b19640 1 --2- 172.21.15.12:0/532174443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1389eea80 0x55c138a06580 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c005ab0 tx=0x7f8f2c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6196adf450c5d26 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:44.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.725+0000 7f8f27fff640 1 -- 172.21.15.12:0/532174443 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c050070 con 0x55c1389eea80 2024-04-22T13:29:44.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.725+0000 7f8f27fff640 1 -- 172.21.15.12:0/532174443 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f2c009780 con 0x55c1389eea80 2024-04-22T13:29:44.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.725+0000 7f8f27fff640 1 -- 172.21.15.12:0/532174443 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c0052a0 con 0x55c1389eea80 2024-04-22T13:29:44.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.725+0000 7f8f4716c740 1 -- 172.21.15.12:0/532174443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1389eea80 msgr2=0x55c138a06580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:44.727 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.725+0000 7f8f4716c740 1 --2- 172.21.15.12:0/532174443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1389eea80 0x55c138a06580 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c005ab0 tx=0x7f8f2c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:29:44.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.725+0000 7f8f4716c740 1 -- 172.21.15.12:0/532174443 shutdown_connections 2024-04-22T13:29:44.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.725+0000 7f8f4716c740 1 --2- 172.21.15.12:0/532174443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1389eea80 0x55c138a06580 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.725+0000 7f8f4716c740 1 --2- 172.21.15.12:0/532174443 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1389e5ea0 0x55c1389ed890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.725+0000 7f8f4716c740 1 --2- 172.21.15.12:0/532174443 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1389e5580 0x55c1389fa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.725+0000 7f8f4716c740 1 -- 172.21.15.12:0/532174443 >> 172.21.15.12:0/532174443 conn(0x55c1389dfcb0 msgr2=0x55c1389ed180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:44.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.725+0000 7f8f4716c740 1 -- 172.21.15.12:0/532174443 shutdown_connections 2024-04-22T13:29:44.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.726+0000 7f8f4716c740 1 -- 172.21.15.12:0/532174443 wait complete. 2024-04-22T13:29:44.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.726+0000 7f8f4716c740 1 -- start start 2024-04-22T13:29:44.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.726+0000 7f8f4716c740 1 Processor -- start 2024-04-22T13:29:44.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.726+0000 7f8f4716c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1389e5580 0x55c1389f2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.726+0000 7f8f4716c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1389e5ea0 0x55c1389f2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.726+0000 7f8f4716c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1389eea80 0x55c1389f3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.726+0000 7f8f4716c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c1b10f0 con 0x55c1389eea80 2024-04-22T13:29:44.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.726+0000 7f8f4716c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c1b0a20 con 0x55c1389e5ea0 2024-04-22T13:29:44.728 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.726+0000 7f8f4716c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c1adc60 con 0x55c1389e5580 2024-04-22T13:29:44.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.726+0000 7f8f4716c740 10 client.0 _ll_register_callbacks cb 0x7f8f3801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:29:44.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.726+0000 7f8f41b19640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1389eea80 0x55c1389f3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.726+0000 7f8f41b19640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1389eea80 0x55c1389f3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:43696/0 (socket says 172.21.15.12:43696) 2024-04-22T13:29:44.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.726+0000 7f8f41b19640 1 -- 172.21.15.12:0/1527659800 learned_addr learned my addr 172.21.15.12:0/1527659800 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:29:44.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.726+0000 7f8f4231a640 1 --2- 172.21.15.12:0/1527659800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1389e5ea0 0x55c1389f2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.726+0000 7f8f41b19640 1 --2- 172.21.15.12:0/1527659800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1389e5580 0x55c1389f2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.726+0000 7f8f41b19640 1 -- 172.21.15.12:0/1527659800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1389e5580 msgr2=0x55c1389f2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:44.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.726+0000 7f8f41b19640 1 --2- 172.21.15.12:0/1527659800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1389e5580 0x55c1389f2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.726+0000 7f8f41b19640 1 -- 172.21.15.12:0/1527659800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1389e5ea0 msgr2=0x55c1389f2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:29:44.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.726+0000 7f8f41b19640 1 --2- 172.21.15.12:0/1527659800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1389e5ea0 0x55c1389f2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.727+0000 7f8f41b19640 1 -- 172.21.15.12:0/1527659800 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c138a63ef0 con 0x55c1389eea80 2024-04-22T13:29:44.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.727+0000 7f8f41b19640 1 --2- 172.21.15.12:0/1527659800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1389eea80 0x55c1389f3210 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c001eb0 tx=0x7f8f2c045040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8a2f2b021658f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:29:44.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.727+0000 7f8f25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:29:44.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.727+0000 7f8f25ffb640 1 -- 172.21.15.12:0/1527659800 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c050040 con 0x55c1389eea80 2024-04-22T13:29:44.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.727+0000 7f8f25ffb640 1 -- 172.21.15.12:0/1527659800 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f8f2c02cba0 con 0x55c1389eea80 2024-04-22T13:29:44.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.727+0000 7f8f4716c740 1 -- 172.21.15.12:0/1527659800 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f2c009780 con 0x55c1389eea80 2024-04-22T13:29:44.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.727+0000 7f8f25ffb640 1 -- 172.21.15.12:0/1527659800 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8f2c005030 con 0x55c1389eea80 2024-04-22T13:29:44.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.727+0000 7f8f4716c740 10 client.17889 fetch_fsmap learned FSMap version 28 2024-04-22T13:29:44.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.727+0000 7f8f4716c740 1 -- 172.21.15.12:0/1527659800 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c138a65200 con 0x55c1389eea80 2024-04-22T13:29:44.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.727+0000 7f8f25ffb640 1 -- 172.21.15.12:0/1527659800 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f2c009780 con 0x55c1389eea80 2024-04-22T13:29:44.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.727+0000 7f8f25ffb640 1 -- 172.21.15.12:0/1527659800 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8f2c004c20 con 0x55c1389eea80 2024-04-22T13:29:44.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.727+0000 7f8f4716c740 10 client.17889 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:29:44.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.727+0000 7f8f4716c740 10 client.17889 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:29:44.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.727+0000 7f8f4716c740 20 client.17889 populate_metadata read hostname 'smithi012' 2024-04-22T13:29:44.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.727+0000 7f8f4716c740 1 --2- 172.21.15.12:0/1527659800 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c138a65870 0x55c138a67c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:29:44.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.727+0000 7f8f4716c740 4 client.17889 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:29:44.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.727+0000 7f8f4716c740 1 -- 172.21.15.12:0/1527659800 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c138951a70 con 0x55c138a65870 2024-04-22T13:29:44.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.728+0000 7f8f4231a640 1 --2- 172.21.15.12:0/1527659800 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c138a65870 0x55c138a67c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:29:44.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.728+0000 7f8f4231a640 1 --2- 172.21.15.12:0/1527659800 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c138a65870 0x55c138a67c60 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56c24e09821acad6 server_cookie=a012d4b0684a3c42 in_seq=0 out_seq=0 2024-04-22T13:29:44.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.728+0000 7f8f25ffb640 10 client.17889 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:44.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.728+0000 7f8f25ffb640 1 -- 172.21.15.12:0/1527659800 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c138951a70 con 0x55c138a65870 2024-04-22T13:29:44.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.728+0000 7f8f25ffb640 10 client.17889 handle_command_reply: tid=0 2024-04-22T13:29:44.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.728+0000 7f8f4716c740 1 client.17889 shutdown 2024-04-22T13:29:44.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.728+0000 7f8f4716c740 1 -- 172.21.15.12:0/1527659800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1389eea80 msgr2=0x55c1389f3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:44.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.728+0000 7f8f4716c740 1 --2- 172.21.15.12:0/1527659800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1389eea80 0x55c1389f3210 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c001eb0 tx=0x7f8f2c045040 comp rx=0 tx=0).stop 2024-04-22T13:29:44.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.728+0000 7f8f4716c740 1 -- 172.21.15.12:0/1527659800 shutdown_connections 2024-04-22T13:29:44.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.728+0000 7f8f4716c740 1 --2- 172.21.15.12:0/1527659800 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55c138a65870 0x55c138a67c60 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.728+0000 7f8f4716c740 1 --2- 172.21.15.12:0/1527659800 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c1389eea80 0x55c1389f3210 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.728+0000 7f8f4716c740 1 --2- 172.21.15.12:0/1527659800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c1389e5ea0 0x55c1389f2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.728+0000 7f8f25ffb640 0 client.17889 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:29:44.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.728+0000 7f8f4716c740 1 --2- 172.21.15.12:0/1527659800 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c1389e5580 0x55c1389f2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.728+0000 7f8f4716c740 1 -- 172.21.15.12:0/1527659800 >> 172.21.15.12:0/1527659800 conn(0x55c1389dfcb0 msgr2=0x55c1389ec290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:44.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.728+0000 7f8f4716c740 1 -- 172.21.15.12:0/1527659800 shutdown_connections 2024-04-22T13:29:44.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.728+0000 7f8f4716c740 1 -- 172.21.15.12:0/1527659800 wait complete. 2024-04-22T13:29:44.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.729+0000 7f8f4716c740 20 client.17889 trim_cache size 0 max 16384 2024-04-22T13:29:44.733 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:29:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:29:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-04-22T13:29:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: "eb5de7d9-71dd-4c2a-9a1f-b60c2dd18b48": { 2024-04-22T13:29:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-04-22T13:29:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "eb5de7d9-71dd-4c2a-9a1f-b60c2dd18b48", 2024-04-22T13:29:44.733 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-04-22T13:29:44.734 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:29:44.734 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:29:44.734 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:29:44.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.729+0000 7f8f4457e640 1 -- 172.21.15.12:0/1977238330 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f8f280a3d20 msgr2=0x7f8f280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:44.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.729+0000 7f8f4457e640 1 --2- 172.21.15.12:0/1977238330 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f8f280a3d20 0x7f8f280a61e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8f3409cd00 tx=0x7f8f34009f90 comp rx=0 tx=0).stop 2024-04-22T13:29:44.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.729+0000 7f8f4457e640 1 -- 172.21.15.12:0/1977238330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f3c10ec50 msgr2=0x7f8f3c19a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:29:44.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.729+0000 7f8f4457e640 1 --2- 172.21.15.12:0/1977238330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f3c10ec50 0x7f8f3c19a750 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009c90 tx=0x7f8f3400c6a0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.729+0000 7f8f4457e640 1 -- 172.21.15.12:0/1977238330 shutdown_connections 2024-04-22T13:29:44.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.729+0000 7f8f4457e640 1 --2- 172.21.15.12:0/1977238330 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f8f280a3d20 0x7f8f280a61e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.729+0000 7f8f4457e640 1 --2- 172.21.15.12:0/1977238330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8f3c10ec50 0x7f8f3c19a750 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.729+0000 7f8f4457e640 1 --2- 172.21.15.12:0/1977238330 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f3c06aed0 0x7f8f3c1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.729+0000 7f8f4457e640 1 --2- 172.21.15.12:0/1977238330 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f3c06a5b0 0x7f8f3c1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:29:44.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.729+0000 7f8f4457e640 1 -- 172.21.15.12:0/1977238330 >> 172.21.15.12:0/1977238330 conn(0x7f8f3c077c40 msgr2=0x7f8f3c110260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:29:44.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.729+0000 7f8f4457e640 1 -- 172.21.15.12:0/1977238330 shutdown_connections 2024-04-22T13:29:44.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:29:44.729+0000 7f8f4457e640 1 -- 172.21.15.12:0/1977238330 wait complete. 2024-04-22T13:29:44.738 INFO:tasks.cephfs.filesystem:scrub status for tag:eb5de7d9-71dd-4c2a-9a1f-b60c2dd18b48 - {'path': '/', 'tag': 'eb5de7d9-71dd-4c2a-9a1f-b60c2dd18b48', 'options': 'recursive,force'} 2024-04-22T13:29:46.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:45 smithi012 ceph-mon[28265]: pgmap v172: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:29:46.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:45 smithi012 ceph-mon[28265]: scrub queued for path: / 2024-04-22T13:29:46.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:45 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:29:46.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:45 smithi012 ceph-mon[28265]: scrub summary: active paths [/] 2024-04-22T13:29:46.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:45 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:29:46.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:45 smithi012 ceph-mon[28265]: scrub summary: idle 2024-04-22T13:29:46.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:45 smithi134 ceph-mon[31338]: pgmap v172: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:29:46.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:45 smithi134 ceph-mon[31338]: scrub queued for path: / 2024-04-22T13:29:46.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:45 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:29:46.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:45 smithi134 ceph-mon[31338]: scrub summary: active paths [/] 2024-04-22T13:29:46.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:45 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:29:46.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:45 smithi134 ceph-mon[31338]: scrub summary: idle 2024-04-22T13:29:46.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:45 smithi111 ceph-mon[31361]: pgmap v172: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:29:46.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:45 smithi111 ceph-mon[31361]: scrub queued for path: / 2024-04-22T13:29:46.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:45 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:29:46.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:45 smithi111 ceph-mon[31361]: scrub summary: active paths [/] 2024-04-22T13:29:46.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:45 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:29:46.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:45 smithi111 ceph-mon[31361]: scrub summary: idle 2024-04-22T13:29:48.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:47 smithi012 ceph-mon[28265]: pgmap v173: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:29:48.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:47 smithi134 ceph-mon[31338]: pgmap v173: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:29:48.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:47 smithi111 ceph-mon[31361]: pgmap v173: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:29:50.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:49 smithi012 ceph-mon[28265]: pgmap v174: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:29:50.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:49 smithi134 ceph-mon[31338]: pgmap v174: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:29:50.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:49 smithi111 ceph-mon[31361]: pgmap v174: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:29:52.028 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:51 smithi111 ceph-mon[31361]: pgmap v175: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:29:52.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:51 smithi012 ceph-mon[28265]: pgmap v175: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:29:52.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:51 smithi134 ceph-mon[31338]: pgmap v175: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:29:54.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:53 smithi012 ceph-mon[28265]: pgmap v176: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:29:54.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:53 smithi134 ceph-mon[31338]: pgmap v176: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:29:54.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:53 smithi111 ceph-mon[31361]: pgmap v176: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:29:56.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:55 smithi012 ceph-mon[28265]: pgmap v177: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:29:56.178 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:55 smithi134 ceph-mon[31338]: pgmap v177: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:29:56.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:55 smithi111 ceph-mon[31361]: pgmap v177: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:29:58.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:57 smithi012 ceph-mon[28265]: pgmap v178: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:29:58.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:57 smithi134 ceph-mon[31338]: pgmap v178: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:29:58.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:57 smithi111 ceph-mon[31361]: pgmap v178: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:30:00.005 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:29:59 smithi012 ceph-mon[28265]: pgmap v179: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:30:00.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:29:59 smithi134 ceph-mon[31338]: pgmap v179: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:30:00.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:29:59 smithi111 ceph-mon[31361]: pgmap v179: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:30:01.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:00 smithi012 ceph-mon[28265]: overall HEALTH_OK 2024-04-22T13:30:01.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:00 smithi134 ceph-mon[31338]: overall HEALTH_OK 2024-04-22T13:30:01.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:00 smithi111 ceph-mon[31361]: overall HEALTH_OK 2024-04-22T13:30:02.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:01 smithi012 ceph-mon[28265]: pgmap v180: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:02.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:01 smithi134 ceph-mon[31338]: pgmap v180: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:02.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:01 smithi111 ceph-mon[31361]: pgmap v180: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:04.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:03 smithi012 ceph-mon[28265]: pgmap v181: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-22T13:30:04.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:03 smithi134 ceph-mon[31338]: pgmap v181: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-22T13:30:04.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:03 smithi111 ceph-mon[31361]: pgmap v181: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-22T13:30:06.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:05 smithi012 ceph-mon[28265]: pgmap v182: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:30:06.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:05 smithi134 ceph-mon[31338]: pgmap v182: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:30:06.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:05 smithi111 ceph-mon[31361]: pgmap v182: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:30:08.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:07 smithi012 ceph-mon[28265]: pgmap v183: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:30:08.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:07 smithi134 ceph-mon[31338]: pgmap v183: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:30:08.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:07 smithi111 ceph-mon[31361]: pgmap v183: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:30:10.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:09 smithi012 ceph-mon[28265]: pgmap v184: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-22T13:30:10.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:09 smithi134 ceph-mon[31338]: pgmap v184: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-22T13:30:10.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:09 smithi111 ceph-mon[31361]: pgmap v184: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-22T13:30:12.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:11 smithi012 ceph-mon[28265]: pgmap v185: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:30:12.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:11 smithi134 ceph-mon[31338]: pgmap v185: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:30:12.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:11 smithi111 ceph-mon[31361]: pgmap v185: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-22T13:30:14.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:13 smithi012 ceph-mon[28265]: pgmap v186: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-22T13:30:14.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:13 smithi134 ceph-mon[31338]: pgmap v186: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-22T13:30:14.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:13 smithi111 ceph-mon[31361]: pgmap v186: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-22T13:30:14.739 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:30:14.739 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:30:14.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.861+0000 7ffbbfe06640 1 -- 172.21.15.12:0/1943272428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbb806af50 msgr2=0x7ffbb810ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:14.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.861+0000 7ffbbfe06640 1 --2- 172.21.15.12:0/1943272428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbb806af50 0x7ffbb810ede0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0009ca0 tx=0x7ffbb002c180 comp rx=0 tx=0).stop 2024-04-22T13:30:14.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.861+0000 7ffbbfe06640 1 -- 172.21.15.12:0/1943272428 shutdown_connections 2024-04-22T13:30:14.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.861+0000 7ffbbfe06640 1 --2- 172.21.15.12:0/1943272428 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbb806af50 0x7ffbb810ede0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.861+0000 7ffbbfe06640 1 --2- 172.21.15.12:0/1943272428 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffbb806a5b0 0x7ffbb806aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.860 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.861+0000 7ffbbfe06640 1 --2- 172.21.15.12:0/1943272428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbb8108040 0x7ffbb8108420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.861+0000 7ffbbfe06640 1 -- 172.21.15.12:0/1943272428 >> 172.21.15.12:0/1943272428 conn(0x7ffbb8077c40 msgr2=0x7ffbb8078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:14.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.861+0000 7ffbbfe06640 1 -- 172.21.15.12:0/1943272428 shutdown_connections 2024-04-22T13:30:14.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.861+0000 7ffbbfe06640 1 -- 172.21.15.12:0/1943272428 wait complete. 2024-04-22T13:30:14.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.862+0000 7ffbbfe06640 1 Processor -- start 2024-04-22T13:30:14.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.862+0000 7ffbbfe06640 1 -- start start 2024-04-22T13:30:14.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.862+0000 7ffbbfe06640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbb806a5b0 0x7ffbb8198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:14.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.862+0000 7ffbbfe06640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffbb806af50 0x7ffbb8198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:14.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.862+0000 7ffbbfe06640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbb8108040 0x7ffbb819cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:14.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.862+0000 7ffbbfe06640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb8112160 con 0x7ffbb806a5b0 2024-04-22T13:30:14.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.862+0000 7ffbbfe06640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb8111fe0 con 0x7ffbb806af50 2024-04-22T13:30:14.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.862+0000 7ffbbfe06640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb81122e0 con 0x7ffbb8108040 2024-04-22T13:30:14.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.862+0000 7ffbbdba2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbb806a5b0 0x7ffbb8198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:14.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.862+0000 7ffbbdba2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbb806a5b0 0x7ffbb8198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51412/0 (socket says 172.21.15.12:51412) 2024-04-22T13:30:14.861 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.862+0000 7ffbbdba2640 1 -- 172.21.15.12:0/310008393 learned_addr learned my addr 172.21.15.12:0/310008393 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:14.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.863+0000 7ffbbe3a3640 1 --2- 172.21.15.12:0/310008393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbb8108040 0x7ffbb819cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:14.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.863+0000 7ffbbd3a1640 1 --2- 172.21.15.12:0/310008393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffbb806af50 0x7ffbb8198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:14.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.863+0000 7ffbbdba2640 1 -- 172.21.15.12:0/310008393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbb8108040 msgr2=0x7ffbb819cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:14.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.863+0000 7ffbbdba2640 1 --2- 172.21.15.12:0/310008393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbb8108040 0x7ffbb819cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.863+0000 7ffbbdba2640 1 -- 172.21.15.12:0/310008393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffbb806af50 msgr2=0x7ffbb8198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:14.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.863+0000 7ffbbdba2640 1 --2- 172.21.15.12:0/310008393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffbb806af50 0x7ffbb8198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.863+0000 7ffbbdba2640 1 -- 172.21.15.12:0/310008393 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb819d5f0 con 0x7ffbb806a5b0 2024-04-22T13:30:14.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.863+0000 7ffbbd3a1640 1 --2- 172.21.15.12:0/310008393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffbb806af50 0x7ffbb8198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:30:14.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.863+0000 7ffbbdba2640 1 --2- 172.21.15.12:0/310008393 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbb806a5b0 0x7ffbb8198600 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400d950 tx=0x7ffbb400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:14.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.863+0000 7ffbaeffd640 1 -- 172.21.15.12:0/310008393 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4017070 con 0x7ffbb806a5b0 2024-04-22T13:30:14.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.863+0000 7ffbbfe06640 1 -- 172.21.15.12:0/310008393 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb819d8e0 con 0x7ffbb806a5b0 2024-04-22T13:30:14.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.863+0000 7ffbaeffd640 1 -- 172.21.15.12:0/310008393 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbb4009c30 con 0x7ffbb806a5b0 2024-04-22T13:30:14.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.863+0000 7ffbaeffd640 1 -- 172.21.15.12:0/310008393 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb400f620 con 0x7ffbb806a5b0 2024-04-22T13:30:14.862 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.863+0000 7ffbbfe06640 1 -- 172.21.15.12:0/310008393 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb81a5170 con 0x7ffbb806a5b0 2024-04-22T13:30:14.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.865+0000 7ffbaeffd640 1 -- 172.21.15.12:0/310008393 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ffbb4010460 con 0x7ffbb806a5b0 2024-04-22T13:30:14.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.866+0000 7ffbc29fb740 1 Processor -- start 2024-04-22T13:30:14.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.866+0000 7ffbc29fb740 1 -- start start 2024-04-22T13:30:14.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.866+0000 7ffbaeffd640 1 --2- 172.21.15.12:0/310008393 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ffba40a3d20 0x7ffba40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:14.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.866+0000 7ffbc29fb740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a116b580 0x5609a116b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:14.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.866+0000 7ffbc29fb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609a116bea0 0x5609a1174540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:14.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.866+0000 7ffbc29fb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609a1174a80 0x5609a1176e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:14.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.866+0000 7ffbc29fb740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5609a10d01e0 con 0x5609a116b580 2024-04-22T13:30:14.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.867+0000 7ffbc29fb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5609a106d390 con 0x5609a116bea0 2024-04-22T13:30:14.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.867+0000 7ffbc29fb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5609a10cc1a0 con 0x5609a1174a80 2024-04-22T13:30:14.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.867+0000 7ffbbdba2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a116b580 0x5609a116b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:14.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.867+0000 7ffbbe3a3640 1 --2- 172.21.15.12:0/310008393 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ffba40a3d20 0x7ffba40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:14.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.867+0000 7ffbaeffd640 1 -- 172.21.15.12:0/310008393 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7ffbb40d0410 con 0x7ffbb806a5b0 2024-04-22T13:30:14.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.867+0000 7ffbbdba2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a116b580 0x5609a116b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51422/0 (socket says 172.21.15.12:51422) 2024-04-22T13:30:14.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.867+0000 7ffbbdba2640 1 -- 172.21.15.12:0/1298019255 learned_addr learned my addr 172.21.15.12:0/1298019255 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:14.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.867+0000 7ffbbe3a3640 1 --2- 172.21.15.12:0/1298019255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609a1174a80 0x5609a1176e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:14.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.867+0000 7ffbbd3a1640 1 --2- 172.21.15.12:0/1298019255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609a116bea0 0x5609a1174540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:14.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.867+0000 7ffbbdba2640 1 -- 172.21.15.12:0/1298019255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609a1174a80 msgr2=0x5609a1176e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:14.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.867+0000 7ffbbdba2640 1 --2- 172.21.15.12:0/1298019255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609a1174a80 0x5609a1176e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.867+0000 7ffbbdba2640 1 -- 172.21.15.12:0/1298019255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609a116bea0 msgr2=0x5609a1174540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:14.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.867+0000 7ffbbdba2640 1 --2- 172.21.15.12:0/1298019255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609a116bea0 0x5609a1174540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.867+0000 7ffbbdba2640 1 -- 172.21.15.12:0/1298019255 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609a0eac7d0 con 0x5609a116b580 2024-04-22T13:30:14.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.867+0000 7ffbbd3a1640 1 --2- 172.21.15.12:0/1298019255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609a116bea0 0x5609a1174540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:30:14.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.867+0000 7ffbbdba2640 1 --2- 172.21.15.12:0/1298019255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a116b580 0x5609a116b960 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40cdc10 tx=0x7ffbb409c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277d046e63687f3f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:14.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.867+0000 7ffbbe3a3640 1 --2- 172.21.15.12:0/310008393 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ffba40a3d20 0x7ffba40a61e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0005cc0 tx=0x7ffbb0003af0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:14.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.867+0000 7ffba3fff640 1 -- 172.21.15.12:0/1298019255 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4020070 con 0x5609a116b580 2024-04-22T13:30:14.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.867+0000 7ffba3fff640 1 -- 172.21.15.12:0/1298019255 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbb4009de0 con 0x5609a116b580 2024-04-22T13:30:14.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.867+0000 7ffba3fff640 1 -- 172.21.15.12:0/1298019255 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb40cde30 con 0x5609a116b580 2024-04-22T13:30:14.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.868+0000 7ffbc29fb740 1 -- 172.21.15.12:0/1298019255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a116b580 msgr2=0x5609a116b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:14.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.868+0000 7ffbc29fb740 1 --2- 172.21.15.12:0/1298019255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a116b580 0x5609a116b960 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40cdc10 tx=0x7ffbb409c300 comp rx=0 tx=0).stop 2024-04-22T13:30:14.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.868+0000 7ffbc29fb740 1 -- 172.21.15.12:0/1298019255 shutdown_connections 2024-04-22T13:30:14.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.868+0000 7ffbc29fb740 1 --2- 172.21.15.12:0/1298019255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609a1174a80 0x5609a1176e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.868+0000 7ffbc29fb740 1 --2- 172.21.15.12:0/1298019255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609a116bea0 0x5609a1174540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.868+0000 7ffbc29fb740 1 --2- 172.21.15.12:0/1298019255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a116b580 0x5609a116b960 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.868+0000 7ffbc29fb740 1 -- 172.21.15.12:0/1298019255 >> 172.21.15.12:0/1298019255 conn(0x5609a1165cb0 msgr2=0x5609a1166090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:14.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.868+0000 7ffbc29fb740 1 -- 172.21.15.12:0/1298019255 shutdown_connections 2024-04-22T13:30:14.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.868+0000 7ffbc29fb740 1 -- 172.21.15.12:0/1298019255 wait complete. 2024-04-22T13:30:14.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.869+0000 7ffbc29fb740 1 -- start start 2024-04-22T13:30:14.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.869+0000 7ffbc29fb740 1 Processor -- start 2024-04-22T13:30:14.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.869+0000 7ffbc29fb740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a116b580 0x5609a1171ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:14.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.869+0000 7ffbc29fb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609a116bea0 0x5609a1172010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:14.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.869+0000 7ffbc29fb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609a1174a80 0x5609a1172550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:14.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.869+0000 7ffbc29fb740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5609a1177810 con 0x5609a116b580 2024-04-22T13:30:14.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.869+0000 7ffbc29fb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5609a10892f0 con 0x5609a116bea0 2024-04-22T13:30:14.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.869+0000 7ffbc29fb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5609a1177990 con 0x5609a1174a80 2024-04-22T13:30:14.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.869+0000 7ffbc29fb740 10 client.0 _ll_register_callbacks cb 0x5609a1096580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:14.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.869+0000 7ffbbdba2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a116b580 0x5609a1171ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:14.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.869+0000 7ffbbdba2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a116b580 0x5609a1171ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51428/0 (socket says 172.21.15.12:51428) 2024-04-22T13:30:14.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.869+0000 7ffbbdba2640 1 -- 172.21.15.12:0/4033879145 learned_addr learned my addr 172.21.15.12:0/4033879145 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:14.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.869+0000 7ffbbd3a1640 1 --2- 172.21.15.12:0/4033879145 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609a116bea0 0x5609a1172010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:14.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.869+0000 7ffbbe3a3640 1 --2- 172.21.15.12:0/4033879145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609a1174a80 0x5609a1172550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:14.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.870+0000 7ffbbdba2640 1 -- 172.21.15.12:0/4033879145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609a1174a80 msgr2=0x5609a1172550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:14.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.870+0000 7ffbbdba2640 1 --2- 172.21.15.12:0/4033879145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609a1174a80 0x5609a1172550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.870+0000 7ffbbdba2640 1 -- 172.21.15.12:0/4033879145 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609a116bea0 msgr2=0x5609a1172010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:14.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.870+0000 7ffbbdba2640 1 --2- 172.21.15.12:0/4033879145 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609a116bea0 0x5609a1172010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.870+0000 7ffbbdba2640 1 -- 172.21.15.12:0/4033879145 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609a0de23f0 con 0x5609a116b580 2024-04-22T13:30:14.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.870+0000 7ffbbdba2640 1 --2- 172.21.15.12:0/4033879145 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a116b580 0x5609a1171ad0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4009f90 tx=0x7ffbb400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9abe79d6336a65c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:14.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.870+0000 7ffba17fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:14.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.870+0000 7ffba17fa640 1 -- 172.21.15.12:0/4033879145 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4020070 con 0x5609a116b580 2024-04-22T13:30:14.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.870+0000 7ffba17fa640 1 -- 172.21.15.12:0/4033879145 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7ffbb4098a70 con 0x5609a116b580 2024-04-22T13:30:14.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.870+0000 7ffbc29fb740 1 -- 172.21.15.12:0/4033879145 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbb4009de0 con 0x5609a116b580 2024-04-22T13:30:14.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.870+0000 7ffba17fa640 1 -- 172.21.15.12:0/4033879145 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffbb40b4030 con 0x5609a116b580 2024-04-22T13:30:14.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.870+0000 7ffbc29fb740 10 client.17901 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:14.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.870+0000 7ffbc29fb740 1 -- 172.21.15.12:0/4033879145 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5609a0da20f0 con 0x5609a116b580 2024-04-22T13:30:14.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.870+0000 7ffba17fa640 1 -- 172.21.15.12:0/4033879145 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbb4009de0 con 0x5609a116b580 2024-04-22T13:30:14.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.870+0000 7ffba17fa640 1 -- 172.21.15.12:0/4033879145 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ffbb40a4be0 con 0x5609a116b580 2024-04-22T13:30:14.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.870+0000 7ffbc29fb740 10 client.17901 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:14.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.871+0000 7ffbc29fb740 10 client.17901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:30:14.877 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.871+0000 7ffbc29fb740 20 client.17901 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:14.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.871+0000 7ffbc29fb740 1 --2- 172.21.15.12:0/4033879145 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5609a1184830 0x5609a1186c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:14.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.871+0000 7ffbc29fb740 4 client.17901 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:30:14.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.871+0000 7ffbc29fb740 1 -- 172.21.15.12:0/4033879145 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609a10ddc00 con 0x5609a1184830 2024-04-22T13:30:14.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.871+0000 7ffbbd3a1640 1 --2- 172.21.15.12:0/4033879145 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5609a1184830 0x5609a1186c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:14.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.871+0000 7ffbbd3a1640 1 --2- 172.21.15.12:0/4033879145 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5609a1184830 0x5609a1186c20 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7abfb73e9631fb5e server_cookie=5c7f0f2dc18052c6 in_seq=0 out_seq=0 2024-04-22T13:30:14.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.871+0000 7ffba17fa640 10 client.17901 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:14.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.872+0000 7ffba17fa640 1 -- 172.21.15.12:0/4033879145 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5609a10ddc00 con 0x5609a1184830 2024-04-22T13:30:14.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.872+0000 7ffba17fa640 10 client.17901 handle_command_reply: tid=0 2024-04-22T13:30:14.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.872+0000 7ffbc29fb740 1 client.17901 shutdown 2024-04-22T13:30:14.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.872+0000 7ffbc29fb740 1 -- 172.21.15.12:0/4033879145 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a116b580 msgr2=0x5609a1171ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:14.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.872+0000 7ffbc29fb740 1 --2- 172.21.15.12:0/4033879145 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a116b580 0x5609a1171ad0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4009f90 tx=0x7ffbb400bf60 comp rx=0 tx=0).stop 2024-04-22T13:30:14.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.872+0000 7ffbc29fb740 1 -- 172.21.15.12:0/4033879145 shutdown_connections 2024-04-22T13:30:14.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.872+0000 7ffbc29fb740 1 --2- 172.21.15.12:0/4033879145 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5609a1184830 0x5609a1186c20 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.872+0000 7ffbc29fb740 1 --2- 172.21.15.12:0/4033879145 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609a1174a80 0x5609a1172550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.872+0000 7ffbc29fb740 1 --2- 172.21.15.12:0/4033879145 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609a116bea0 0x5609a1172010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.872+0000 7ffba17fa640 0 client.17901 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:14.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.872+0000 7ffbc29fb740 1 --2- 172.21.15.12:0/4033879145 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a116b580 0x5609a1171ad0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.872+0000 7ffbc29fb740 1 -- 172.21.15.12:0/4033879145 >> 172.21.15.12:0/4033879145 conn(0x5609a1165cb0 msgr2=0x5609a1167a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:14.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.872+0000 7ffbc29fb740 1 -- 172.21.15.12:0/4033879145 shutdown_connections 2024-04-22T13:30:14.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.872+0000 7ffbc29fb740 1 -- 172.21.15.12:0/4033879145 wait complete. 2024-04-22T13:30:14.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.872+0000 7ffbc29fb740 20 client.17901 trim_cache size 0 max 16384 2024-04-22T13:30:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.890+0000 7ffbc29fb740 1 Processor -- start 2024-04-22T13:30:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.890+0000 7ffbc29fb740 1 -- start start 2024-04-22T13:30:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.890+0000 7ffbc29fb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609a116b580 0x5609a1174200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.890+0000 7ffbc29fb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609a116bea0 0x5609a116c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.890+0000 7ffbc29fb740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a1174a80 0x5609a11806d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.890+0000 7ffbc29fb740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb81ada20 con 0x5609a1174a80 2024-04-22T13:30:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.890+0000 7ffbc29fb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb81af560 con 0x5609a116bea0 2024-04-22T13:30:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.890+0000 7ffbc29fb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb81ae0f0 con 0x5609a116b580 2024-04-22T13:30:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.890+0000 7ffbbe3a3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a1174a80 0x5609a11806d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.890+0000 7ffbbe3a3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a1174a80 0x5609a11806d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51444/0 (socket says 172.21.15.12:51444) 2024-04-22T13:30:14.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.890+0000 7ffbbe3a3640 1 -- 172.21.15.12:0/1020217193 learned_addr learned my addr 172.21.15.12:0/1020217193 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.890+0000 7ffbbd3a1640 1 --2- 172.21.15.12:0/1020217193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609a116bea0 0x5609a116c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.891+0000 7ffbbdba2640 1 --2- 172.21.15.12:0/1020217193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609a116b580 0x5609a1174200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.891+0000 7ffbbe3a3640 1 -- 172.21.15.12:0/1020217193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609a116b580 msgr2=0x5609a1174200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.891+0000 7ffbbe3a3640 1 --2- 172.21.15.12:0/1020217193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609a116b580 0x5609a1174200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.891+0000 7ffbbe3a3640 1 -- 172.21.15.12:0/1020217193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609a116bea0 msgr2=0x5609a116c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.891+0000 7ffbbe3a3640 1 --2- 172.21.15.12:0/1020217193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609a116bea0 0x5609a116c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.891+0000 7ffbbe3a3640 1 -- 172.21.15.12:0/1020217193 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609a1170a80 con 0x5609a1174a80 2024-04-22T13:30:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.891+0000 7ffbbd3a1640 1 --2- 172.21.15.12:0/1020217193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609a116bea0 0x5609a116c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.891+0000 7ffbbdba2640 1 --2- 172.21.15.12:0/1020217193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609a116b580 0x5609a1174200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:30:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.891+0000 7ffbbe3a3640 1 --2- 172.21.15.12:0/1020217193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a1174a80 0x5609a11806d0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0009ca0 tx=0x7ffbb0044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eabdc16ef4f1d6a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.891+0000 7ffba37fe640 1 -- 172.21.15.12:0/1020217193 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0053070 con 0x5609a1174a80 2024-04-22T13:30:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.891+0000 7ffba37fe640 1 -- 172.21.15.12:0/1020217193 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbb0034440 con 0x5609a1174a80 2024-04-22T13:30:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.891+0000 7ffba37fe640 1 -- 172.21.15.12:0/1020217193 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0039040 con 0x5609a1174a80 2024-04-22T13:30:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.891+0000 7ffbc29fb740 1 -- 172.21.15.12:0/1020217193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a1174a80 msgr2=0x5609a11806d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:14.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.891+0000 7ffbc29fb740 1 --2- 172.21.15.12:0/1020217193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a1174a80 0x5609a11806d0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0009ca0 tx=0x7ffbb0044000 comp rx=0 tx=0).stop 2024-04-22T13:30:14.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.892+0000 7ffbc29fb740 1 -- 172.21.15.12:0/1020217193 shutdown_connections 2024-04-22T13:30:14.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.892+0000 7ffbc29fb740 1 --2- 172.21.15.12:0/1020217193 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a1174a80 0x5609a11806d0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.892+0000 7ffbc29fb740 1 --2- 172.21.15.12:0/1020217193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609a116bea0 0x5609a116c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.892+0000 7ffbc29fb740 1 --2- 172.21.15.12:0/1020217193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609a116b580 0x5609a1174200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.892+0000 7ffbc29fb740 1 -- 172.21.15.12:0/1020217193 >> 172.21.15.12:0/1020217193 conn(0x5609a1165cb0 msgr2=0x5609a1171a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:14.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.892+0000 7ffbc29fb740 1 -- 172.21.15.12:0/1020217193 shutdown_connections 2024-04-22T13:30:14.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.892+0000 7ffbc29fb740 1 -- 172.21.15.12:0/1020217193 wait complete. 2024-04-22T13:30:14.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.892+0000 7ffbc29fb740 1 -- start start 2024-04-22T13:30:14.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.892+0000 7ffbc29fb740 1 Processor -- start 2024-04-22T13:30:14.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.892+0000 7ffbc29fb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609a116b580 0x5609a116e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:14.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.892+0000 7ffbc29fb740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a116bea0 0x5609a116ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:14.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.892+0000 7ffbc29fb740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609a1174a80 0x5609a116f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:14.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.892+0000 7ffbc29fb740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb81aee90 con 0x5609a116bea0 2024-04-22T13:30:14.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.892+0000 7ffbc29fb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb81ae7c0 con 0x5609a116b580 2024-04-22T13:30:14.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.892+0000 7ffbc29fb740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb81aba00 con 0x5609a1174a80 2024-04-22T13:30:14.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.892+0000 7ffbc29fb740 10 client.0 _ll_register_callbacks cb 0x7ffbb409c1d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:14.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.892+0000 7ffbbd3a1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a116bea0 0x5609a116ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:14.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.892+0000 7ffbbd3a1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a116bea0 0x5609a116ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51460/0 (socket says 172.21.15.12:51460) 2024-04-22T13:30:14.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.892+0000 7ffbbd3a1640 1 -- 172.21.15.12:0/579013510 learned_addr learned my addr 172.21.15.12:0/579013510 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:14.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.893+0000 7ffbbdba2640 1 --2- 172.21.15.12:0/579013510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609a116b580 0x5609a116e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:14.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.893+0000 7ffbbe3a3640 1 --2- 172.21.15.12:0/579013510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609a1174a80 0x5609a116f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:14.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.893+0000 7ffbbd3a1640 1 -- 172.21.15.12:0/579013510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609a1174a80 msgr2=0x5609a116f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:14.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.893+0000 7ffbbd3a1640 1 --2- 172.21.15.12:0/579013510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609a1174a80 0x5609a116f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.893+0000 7ffbbd3a1640 1 -- 172.21.15.12:0/579013510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609a116b580 msgr2=0x5609a116e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:14.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.893+0000 7ffbbd3a1640 1 --2- 172.21.15.12:0/579013510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609a116b580 0x5609a116e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.893+0000 7ffbbd3a1640 1 -- 172.21.15.12:0/579013510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609a11e9ab0 con 0x5609a116bea0 2024-04-22T13:30:14.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.893+0000 7ffbbe3a3640 1 --2- 172.21.15.12:0/579013510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609a1174a80 0x5609a116f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.893+0000 7ffbbd3a1640 1 --2- 172.21.15.12:0/579013510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a116bea0 0x5609a116ebd0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x5609a116b050 tx=0x7ffba801af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83e7507b7f3b67e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.893+0000 7ffba17fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.893+0000 7ffba17fa640 1 -- 172.21.15.12:0/579013510 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba801a290 con 0x5609a116bea0 2024-04-22T13:30:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.893+0000 7ffba17fa640 1 -- 172.21.15.12:0/579013510 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7ffba801a430 con 0x5609a116bea0 2024-04-22T13:30:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.893+0000 7ffbc29fb740 1 -- 172.21.15.12:0/579013510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbb0034440 con 0x5609a116bea0 2024-04-22T13:30:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.893+0000 7ffba17fa640 1 -- 172.21.15.12:0/579013510 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffba801a630 con 0x5609a116bea0 2024-04-22T13:30:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.893+0000 7ffbc29fb740 10 client.17907 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.893+0000 7ffbc29fb740 1 -- 172.21.15.12:0/579013510 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5609a11eacc0 con 0x5609a116bea0 2024-04-22T13:30:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.893+0000 7ffba17fa640 1 -- 172.21.15.12:0/579013510 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbb0034440 con 0x5609a116bea0 2024-04-22T13:30:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.894+0000 7ffba17fa640 1 -- 172.21.15.12:0/579013510 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ffba802c430 con 0x5609a116bea0 2024-04-22T13:30:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.894+0000 7ffbc29fb740 10 client.17907 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.894+0000 7ffbc29fb740 10 client.17907 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:30:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.894+0000 7ffbc29fb740 20 client.17907 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:14.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.894+0000 7ffbc29fb740 1 --2- 172.21.15.12:0/579013510 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5609a11eb3d0 0x5609a11ed7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.894+0000 7ffbc29fb740 4 client.17907 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:30:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.894+0000 7ffbc29fb740 1 -- 172.21.15.12:0/579013510 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5609a10d7a70 con 0x5609a11eb3d0 2024-04-22T13:30:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.894+0000 7ffbbdba2640 1 --2- 172.21.15.12:0/579013510 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5609a11eb3d0 0x5609a11ed7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.894+0000 7ffba17fa640 10 client.17907 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.894+0000 7ffbbdba2640 1 --2- 172.21.15.12:0/579013510 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5609a11eb3d0 0x5609a11ed7f0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d90ca62c36b9a02 server_cookie=a71c259eb565b3eb in_seq=0 out_seq=0 2024-04-22T13:30:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.894+0000 7ffba17fa640 1 -- 172.21.15.12:0/579013510 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5609a10d7a70 con 0x5609a11eb3d0 2024-04-22T13:30:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.894+0000 7ffba17fa640 10 client.17907 handle_command_reply: tid=0 2024-04-22T13:30:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.895+0000 7ffbc29fb740 1 client.17907 shutdown 2024-04-22T13:30:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.895+0000 7ffbc29fb740 1 -- 172.21.15.12:0/579013510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a116bea0 msgr2=0x5609a116ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.895+0000 7ffbc29fb740 1 --2- 172.21.15.12:0/579013510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a116bea0 0x5609a116ebd0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x5609a116b050 tx=0x7ffba801af90 comp rx=0 tx=0).stop 2024-04-22T13:30:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.895+0000 7ffbc29fb740 1 -- 172.21.15.12:0/579013510 shutdown_connections 2024-04-22T13:30:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.895+0000 7ffbc29fb740 1 --2- 172.21.15.12:0/579013510 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5609a11eb3d0 0x5609a11ed7f0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.895+0000 7ffbc29fb740 1 --2- 172.21.15.12:0/579013510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5609a1174a80 0x5609a116f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.895+0000 7ffbc29fb740 1 --2- 172.21.15.12:0/579013510 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5609a116bea0 0x5609a116ebd0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.895+0000 7ffbc29fb740 1 --2- 172.21.15.12:0/579013510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609a116b580 0x5609a116e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.895+0000 7ffba17fa640 0 client.17907 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.895+0000 7ffbc29fb740 1 -- 172.21.15.12:0/579013510 >> 172.21.15.12:0/579013510 conn(0x5609a1165cb0 msgr2=0x5609a1167840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.895+0000 7ffbc29fb740 1 -- 172.21.15.12:0/579013510 shutdown_connections 2024-04-22T13:30:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.895+0000 7ffbc29fb740 1 -- 172.21.15.12:0/579013510 wait complete. 2024-04-22T13:30:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.895+0000 7ffbc29fb740 20 client.17907 trim_cache size 0 max 16384 2024-04-22T13:30:14.898 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:30:14.898 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-04-22T13:30:14.898 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-04-22T13:30:14.898 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:30:14.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.895+0000 7ffbbfe06640 1 -- 172.21.15.12:0/310008393 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ffba40a3d20 msgr2=0x7ffba40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:14.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.895+0000 7ffbbfe06640 1 --2- 172.21.15.12:0/310008393 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ffba40a3d20 0x7ffba40a61e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0005cc0 tx=0x7ffbb0003af0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.895+0000 7ffbbfe06640 1 -- 172.21.15.12:0/310008393 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbb806a5b0 msgr2=0x7ffbb8198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:14.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.895+0000 7ffbbfe06640 1 --2- 172.21.15.12:0/310008393 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbb806a5b0 0x7ffbb8198600 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400d950 tx=0x7ffbb400de10 comp rx=0 tx=0).stop 2024-04-22T13:30:14.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.895+0000 7ffbbfe06640 1 -- 172.21.15.12:0/310008393 shutdown_connections 2024-04-22T13:30:14.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.895+0000 7ffbbfe06640 1 --2- 172.21.15.12:0/310008393 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ffba40a3d20 0x7ffba40a61e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.895+0000 7ffbbfe06640 1 --2- 172.21.15.12:0/310008393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbb8108040 0x7ffbb819cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.895+0000 7ffbbfe06640 1 --2- 172.21.15.12:0/310008393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffbb806af50 0x7ffbb8198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.895+0000 7ffbbfe06640 1 --2- 172.21.15.12:0/310008393 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbb806a5b0 0x7ffbb8198600 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:14.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.895+0000 7ffbbfe06640 1 -- 172.21.15.12:0/310008393 >> 172.21.15.12:0/310008393 conn(0x7ffbb8077c40 msgr2=0x7ffbb80ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:14.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.896+0000 7ffbbfe06640 1 -- 172.21.15.12:0/310008393 shutdown_connections 2024-04-22T13:30:14.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:14.896+0000 7ffbbfe06640 1 -- 172.21.15.12:0/310008393 wait complete. 2024-04-22T13:30:14.903 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:30:14.903 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:30:15.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.009+0000 7fb579274640 1 -- 172.21.15.12:0/2872229554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb574102030 msgr2=0x7fb57410dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.008 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.009+0000 7fb579274640 1 --2- 172.21.15.12:0/2872229554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb574102030 0x7fb57410dd50 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fb56c009ca0 tx=0x7fb56c02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.009+0000 7fb579274640 1 -- 172.21.15.12:0/2872229554 shutdown_connections 2024-04-22T13:30:15.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.009+0000 7fb579274640 1 --2- 172.21.15.12:0/2872229554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb574102030 0x7fb57410dd50 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.009+0000 7fb579274640 1 --2- 172.21.15.12:0/2872229554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb574101670 0x7fb574101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.009+0000 7fb579274640 1 --2- 172.21.15.12:0/2872229554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb574105ec0 0x7fb5741062a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.009+0000 7fb579274640 1 -- 172.21.15.12:0/2872229554 >> 172.21.15.12:0/2872229554 conn(0x7fb574077480 msgr2=0x7fb5740fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:15.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.009+0000 7fb579274640 1 -- 172.21.15.12:0/2872229554 shutdown_connections 2024-04-22T13:30:15.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.009+0000 7fb579274640 1 -- 172.21.15.12:0/2872229554 wait complete. 2024-04-22T13:30:15.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.010+0000 7fb579274640 1 Processor -- start 2024-04-22T13:30:15.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.010+0000 7fb579274640 1 -- start start 2024-04-22T13:30:15.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.010+0000 7fb579274640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb574101670 0x7fb5740fc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.010+0000 7fb579274640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb574102030 0x7fb5740fce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.010+0000 7fb579274640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb574105ec0 0x7fb5740fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.010+0000 7fb579274640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb5741110d0 con 0x7fb574101670 2024-04-22T13:30:15.009 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.010+0000 7fb579274640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb574110f50 con 0x7fb574102030 2024-04-22T13:30:15.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.010+0000 7fb579274640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb574111250 con 0x7fb574105ec0 2024-04-22T13:30:15.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.010+0000 7fb572ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb574101670 0x7fb5740fc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.010+0000 7fb572ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb574101670 0x7fb5740fc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51478/0 (socket says 172.21.15.12:51478) 2024-04-22T13:30:15.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.010+0000 7fb572ffd640 1 -- 172.21.15.12:0/2983305822 learned_addr learned my addr 172.21.15.12:0/2983305822 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:15.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.010+0000 7fb5727fc640 1 --2- 172.21.15.12:0/2983305822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb574102030 0x7fb5740fce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.010+0000 7fb5737fe640 1 --2- 172.21.15.12:0/2983305822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb574105ec0 0x7fb5740fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.010+0000 7fb572ffd640 1 -- 172.21.15.12:0/2983305822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb574105ec0 msgr2=0x7fb5740fd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.010+0000 7fb572ffd640 1 --2- 172.21.15.12:0/2983305822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb574105ec0 0x7fb5740fd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.010+0000 7fb572ffd640 1 -- 172.21.15.12:0/2983305822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb574102030 msgr2=0x7fb5740fce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.010+0000 7fb572ffd640 1 --2- 172.21.15.12:0/2983305822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb574102030 0x7fb5740fce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.010 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.010+0000 7fb572ffd640 1 -- 172.21.15.12:0/2983305822 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5741a6a80 con 0x7fb574101670 2024-04-22T13:30:15.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.011+0000 7fb572ffd640 1 --2- 172.21.15.12:0/2983305822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb574101670 0x7fb5740fc930 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fb56400d950 tx=0x7fb56400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:15.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.011+0000 7fb55bfff640 1 -- 172.21.15.12:0/2983305822 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb564017030 con 0x7fb574101670 2024-04-22T13:30:15.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.011+0000 7fb579274640 1 -- 172.21.15.12:0/2983305822 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5741a6d70 con 0x7fb574101670 2024-04-22T13:30:15.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.011+0000 7fb55bfff640 1 -- 172.21.15.12:0/2983305822 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb564004020 con 0x7fb574101670 2024-04-22T13:30:15.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.011+0000 7fb55bfff640 1 -- 172.21.15.12:0/2983305822 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5640047a0 con 0x7fb574101670 2024-04-22T13:30:15.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.011+0000 7fb579274640 1 -- 172.21.15.12:0/2983305822 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5741a7260 con 0x7fb574101670 2024-04-22T13:30:15.011 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.012+0000 7fb579274640 1 -- 172.21.15.12:0/2983305822 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb574069230 con 0x7fb574101670 2024-04-22T13:30:15.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.013+0000 7fb55bfff640 1 -- 172.21.15.12:0/2983305822 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb564009a30 con 0x7fb574101670 2024-04-22T13:30:15.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.014+0000 7fb55bfff640 1 --2- 172.21.15.12:0/2983305822 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb55c0a3d50 0x7fb55c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.012 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.014+0000 7fb5727fc640 1 --2- 172.21.15.12:0/2983305822 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb55c0a3d50 0x7fb55c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.014+0000 7fb55bfff640 1 -- 172.21.15.12:0/2983305822 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fb5640d08a0 con 0x7fb574101670 2024-04-22T13:30:15.013 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.014+0000 7fb5727fc640 1 --2- 172.21.15.12:0/2983305822 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb55c0a3d50 0x7fb55c0a6210 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb568005e10 tx=0x7fb568005da0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:15.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.018+0000 7fb55bfff640 1 -- 172.21.15.12:0/2983305822 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb56400e350 con 0x7fb574101670 2024-04-22T13:30:15.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.244+0000 7fb579274640 1 -- 172.21.15.12:0/2983305822 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5740fde10 con 0x7fb574101670 2024-04-22T13:30:15.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.245+0000 7fb55bfff640 1 -- 172.21.15.12:0/2983305822 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7fb56409cf30 con 0x7fb574101670 2024-04-22T13:30:15.242 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:30:15.242 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:30:15.242 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:30:15.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.247+0000 7fb579274640 1 -- 172.21.15.12:0/2983305822 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb55c0a3d50 msgr2=0x7fb55c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.247+0000 7fb579274640 1 --2- 172.21.15.12:0/2983305822 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb55c0a3d50 0x7fb55c0a6210 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb568005e10 tx=0x7fb568005da0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.247+0000 7fb579274640 1 -- 172.21.15.12:0/2983305822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb574101670 msgr2=0x7fb5740fc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.247+0000 7fb579274640 1 --2- 172.21.15.12:0/2983305822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb574101670 0x7fb5740fc930 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fb56400d950 tx=0x7fb56400de10 comp rx=0 tx=0).stop 2024-04-22T13:30:15.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.247+0000 7fb579274640 1 -- 172.21.15.12:0/2983305822 shutdown_connections 2024-04-22T13:30:15.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.247+0000 7fb579274640 1 --2- 172.21.15.12:0/2983305822 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb55c0a3d50 0x7fb55c0a6210 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.247+0000 7fb579274640 1 --2- 172.21.15.12:0/2983305822 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb574105ec0 0x7fb5740fd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.244 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.247+0000 7fb579274640 1 --2- 172.21.15.12:0/2983305822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb574102030 0x7fb5740fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.247+0000 7fb579274640 1 --2- 172.21.15.12:0/2983305822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb574101670 0x7fb5740fc930 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.247+0000 7fb579274640 1 -- 172.21.15.12:0/2983305822 >> 172.21.15.12:0/2983305822 conn(0x7fb574077480 msgr2=0x7fb5741007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:15.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.247+0000 7fb579274640 1 -- 172.21.15.12:0/2983305822 shutdown_connections 2024-04-22T13:30:15.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.247+0000 7fb579274640 1 -- 172.21.15.12:0/2983305822 wait complete. 2024-04-22T13:30:15.255 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:30:15.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.363+0000 7f2b622f2640 1 -- 172.21.15.12:0/3809395080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b5c06a840 msgr2=0x7f2b5c06ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.363+0000 7f2b622f2640 1 --2- 172.21.15.12:0/3809395080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b5c06a840 0x7f2b5c06ac20 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009900 tx=0x7f2b4402bc30 comp rx=0 tx=0).stop 2024-04-22T13:30:15.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.363+0000 7f2b622f2640 1 -- 172.21.15.12:0/3809395080 shutdown_connections 2024-04-22T13:30:15.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.363+0000 7f2b622f2640 1 --2- 172.21.15.12:0/3809395080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b5c107270 0x7f2b5c10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.363+0000 7f2b622f2640 1 --2- 172.21.15.12:0/3809395080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b5c06b1f0 0x7f2b5c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.363+0000 7f2b622f2640 1 --2- 172.21.15.12:0/3809395080 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b5c06a840 0x7f2b5c06ac20 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.363+0000 7f2b622f2640 1 -- 172.21.15.12:0/3809395080 >> 172.21.15.12:0/3809395080 conn(0x7f2b5c077eb0 msgr2=0x7f2b5c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:15.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.363+0000 7f2b622f2640 1 -- 172.21.15.12:0/3809395080 shutdown_connections 2024-04-22T13:30:15.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.363+0000 7f2b622f2640 1 -- 172.21.15.12:0/3809395080 wait complete. 2024-04-22T13:30:15.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.364+0000 7f2b622f2640 1 Processor -- start 2024-04-22T13:30:15.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.364+0000 7f2b622f2640 1 -- start start 2024-04-22T13:30:15.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.364+0000 7f2b622f2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b5c06a840 0x7f2b5c1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.364+0000 7f2b622f2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b5c107270 0x7f2b5c198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.364+0000 7f2b622f2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b5c19cf70 0x7f2b5c19d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.364+0000 7f2b622f2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5c1121d0 con 0x7f2b5c107270 2024-04-22T13:30:15.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.364+0000 7f2b622f2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5c112050 con 0x7f2b5c06a840 2024-04-22T13:30:15.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.364+0000 7f2b622f2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5c112350 con 0x7f2b5c19cf70 2024-04-22T13:30:15.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.364+0000 7f2b5b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b5c107270 0x7f2b5c198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.364+0000 7f2b5b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b5c107270 0x7f2b5c198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51498/0 (socket says 172.21.15.12:51498) 2024-04-22T13:30:15.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.364+0000 7f2b5b7fe640 1 -- 172.21.15.12:0/3799227338 learned_addr learned my addr 172.21.15.12:0/3799227338 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:15.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.364+0000 7f2b6088f640 1 --2- 172.21.15.12:0/3799227338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b5c19cf70 0x7f2b5c19d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.365+0000 7f2b5bfff640 1 --2- 172.21.15.12:0/3799227338 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b5c06a840 0x7f2b5c1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.365+0000 7f2b5b7fe640 1 -- 172.21.15.12:0/3799227338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b5c19cf70 msgr2=0x7f2b5c19d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.365+0000 7f2b5b7fe640 1 --2- 172.21.15.12:0/3799227338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b5c19cf70 0x7f2b5c19d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.365+0000 7f2b5b7fe640 1 -- 172.21.15.12:0/3799227338 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b5c06a840 msgr2=0x7f2b5c1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.365+0000 7f2b5b7fe640 1 --2- 172.21.15.12:0/3799227338 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b5c06a840 0x7f2b5c1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.365+0000 7f2b5b7fe640 1 -- 172.21.15.12:0/3799227338 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b5c19da60 con 0x7f2b5c107270 2024-04-22T13:30:15.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.365+0000 7f2b5bfff640 1 --2- 172.21.15.12:0/3799227338 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b5c06a840 0x7f2b5c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:30:15.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.365+0000 7f2b5b7fe640 1 --2- 172.21.15.12:0/3799227338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b5c107270 0x7f2b5c198bf0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c00e9f0 tx=0x7f2b4c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:15.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.365+0000 7f2b597fa640 1 -- 172.21.15.12:0/3799227338 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c00ccc0 con 0x7f2b5c107270 2024-04-22T13:30:15.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.365+0000 7f2b622f2640 1 -- 172.21.15.12:0/3799227338 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b5c19dd50 con 0x7f2b5c107270 2024-04-22T13:30:15.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.365+0000 7f2b597fa640 1 -- 172.21.15.12:0/3799227338 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b4c0054d0 con 0x7f2b5c107270 2024-04-22T13:30:15.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.365+0000 7f2b597fa640 1 -- 172.21.15.12:0/3799227338 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c010430 con 0x7f2b5c107270 2024-04-22T13:30:15.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.365+0000 7f2b622f2640 1 -- 172.21.15.12:0/3799227338 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b5c1a5540 con 0x7f2b5c107270 2024-04-22T13:30:15.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.367+0000 7f2b64fc1740 1 Processor -- start 2024-04-22T13:30:15.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.367+0000 7f2b597fa640 1 -- 172.21.15.12:0/3799227338 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2b4c0046c0 con 0x7f2b5c107270 2024-04-22T13:30:15.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.367+0000 7f2b64fc1740 1 -- start start 2024-04-22T13:30:15.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.368+0000 7f2b64fc1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d669544d0 0x563d669548b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.368+0000 7f2b64fc1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d66954e80 0x563d6695d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.368+0000 7f2b64fc1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d6695da00 0x563d6695fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.368+0000 7f2b64fc1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563d668b91f0 con 0x563d669544d0 2024-04-22T13:30:15.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.368+0000 7f2b64fc1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x563d668564b0 con 0x563d66954e80 2024-04-22T13:30:15.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.368+0000 7f2b64fc1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563d668b51b0 con 0x563d6695da00 2024-04-22T13:30:15.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.368+0000 7f2b6088f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d669544d0 0x563d669548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.368+0000 7f2b6088f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d669544d0 0x563d669548b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51502/0 (socket says 172.21.15.12:51502) 2024-04-22T13:30:15.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.368+0000 7f2b6088f640 1 -- 172.21.15.12:0/1682039339 learned_addr learned my addr 172.21.15.12:0/1682039339 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:15.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.368+0000 7f2b5b7fe640 1 --2- 172.21.15.12:0/1682039339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d6695da00 0x563d6695fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.368+0000 7f2b5bfff640 1 --2- 172.21.15.12:0/1682039339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d66954e80 0x563d6695d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.368+0000 7f2b597fa640 1 --2- 172.21.15.12:0/3799227338 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2b480a3d20 0x7f2b480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.368+0000 7f2b6088f640 1 -- 172.21.15.12:0/1682039339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d6695da00 msgr2=0x563d6695fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.368+0000 7f2b6088f640 1 --2- 172.21.15.12:0/1682039339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d6695da00 0x563d6695fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.368+0000 7f2b6088f640 1 -- 172.21.15.12:0/1682039339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d66954e80 msgr2=0x563d6695d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.368+0000 7f2b6088f640 1 --2- 172.21.15.12:0/1682039339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d66954e80 0x563d6695d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.368+0000 7f2b6088f640 1 -- 172.21.15.12:0/1682039339 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d668481b0 con 0x563d669544d0 2024-04-22T13:30:15.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.368+0000 7f2b5b7fe640 1 --2- 172.21.15.12:0/1682039339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d6695da00 0x563d6695fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:30:15.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.368+0000 7f2b6088f640 1 --2- 172.21.15.12:0/3799227338 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2b480a3d20 0x7f2b480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.368+0000 7f2b597fa640 1 -- 172.21.15.12:0/3799227338 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f2b4c0d0bc0 con 0x7f2b5c107270 2024-04-22T13:30:15.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.368+0000 7f2b6088f640 1 --2- 172.21.15.12:0/1682039339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d669544d0 0x563d669548b0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f2b540072e0 tx=0x7f2b5400f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5eb4877bc1bc69b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:15.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.369+0000 7f2b527fc640 1 -- 172.21.15.12:0/1682039339 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54019870 con 0x563d669544d0 2024-04-22T13:30:15.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.369+0000 7f2b527fc640 1 -- 172.21.15.12:0/1682039339 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b54010430 con 0x563d669544d0 2024-04-22T13:30:15.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.369+0000 7f2b527fc640 1 -- 172.21.15.12:0/1682039339 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5401a610 con 0x563d669544d0 2024-04-22T13:30:15.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.369+0000 7f2b64fc1740 1 -- 172.21.15.12:0/1682039339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d669544d0 msgr2=0x563d669548b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.369+0000 7f2b6088f640 1 --2- 172.21.15.12:0/3799227338 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2b480a3d20 0x7f2b480a61e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2b54020000 tx=0x7f2b5401d000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:15.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.369+0000 7f2b64fc1740 1 --2- 172.21.15.12:0/1682039339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d669544d0 0x563d669548b0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f2b540072e0 tx=0x7f2b5400f850 comp rx=0 tx=0).stop 2024-04-22T13:30:15.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.369+0000 7f2b64fc1740 1 -- 172.21.15.12:0/1682039339 shutdown_connections 2024-04-22T13:30:15.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.369+0000 7f2b64fc1740 1 --2- 172.21.15.12:0/1682039339 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d6695da00 0x563d6695fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.369+0000 7f2b64fc1740 1 --2- 172.21.15.12:0/1682039339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d66954e80 0x563d6695d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.369+0000 7f2b64fc1740 1 --2- 172.21.15.12:0/1682039339 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d669544d0 0x563d669548b0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.369+0000 7f2b64fc1740 1 -- 172.21.15.12:0/1682039339 >> 172.21.15.12:0/1682039339 conn(0x563d6694eba0 msgr2=0x563d6694ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:15.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.369+0000 7f2b64fc1740 1 -- 172.21.15.12:0/1682039339 shutdown_connections 2024-04-22T13:30:15.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.369+0000 7f2b64fc1740 1 -- 172.21.15.12:0/1682039339 wait complete. 2024-04-22T13:30:15.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.370+0000 7f2b64fc1740 1 -- start start 2024-04-22T13:30:15.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.370+0000 7f2b64fc1740 1 Processor -- start 2024-04-22T13:30:15.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.370+0000 7f2b64fc1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d669544d0 0x563d6695d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.370+0000 7f2b64fc1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d66954e80 0x563d6695a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.370+0000 7f2b64fc1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d6695da00 0x563d6695acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.370+0000 7f2b64fc1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563d66960970 con 0x563d669544d0 2024-04-22T13:30:15.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.370+0000 7f2b64fc1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x563d66872500 con 0x563d66954e80 2024-04-22T13:30:15.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.370+0000 7f2b64fc1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563d66960af0 con 0x563d6695da00 2024-04-22T13:30:15.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.370+0000 7f2b64fc1740 10 client.0 _ll_register_callbacks cb 0x563d66840b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:15.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.370+0000 7f2b5bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d669544d0 0x563d6695d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.370+0000 7f2b5bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d669544d0 0x563d6695d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51510/0 (socket says 172.21.15.12:51510) 2024-04-22T13:30:15.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.370+0000 7f2b5bfff640 1 -- 172.21.15.12:0/287252784 learned_addr learned my addr 172.21.15.12:0/287252784 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:15.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.370+0000 7f2b6088f640 1 --2- 172.21.15.12:0/287252784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d6695da00 0x563d6695acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.370+0000 7f2b5b7fe640 1 --2- 172.21.15.12:0/287252784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d66954e80 0x563d6695a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.371+0000 7f2b5bfff640 1 -- 172.21.15.12:0/287252784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d6695da00 msgr2=0x563d6695acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.371+0000 7f2b5bfff640 1 --2- 172.21.15.12:0/287252784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d6695da00 0x563d6695acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.371+0000 7f2b5bfff640 1 -- 172.21.15.12:0/287252784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d66954e80 msgr2=0x563d6695a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.371+0000 7f2b5bfff640 1 --2- 172.21.15.12:0/287252784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d66954e80 0x563d6695a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.371+0000 7f2b5bfff640 1 -- 172.21.15.12:0/287252784 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d665cb3f0 con 0x563d669544d0 2024-04-22T13:30:15.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.371+0000 7f2b5bfff640 1 --2- 172.21.15.12:0/287252784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d669544d0 0x563d6695d350 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009230 tx=0x7f2b4402ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f009c1a061079ae server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:15.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.371+0000 7f2b3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:15.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.371+0000 7f2b3b7fe640 1 -- 172.21.15.12:0/287252784 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b44005030 con 0x563d669544d0 2024-04-22T13:30:15.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.371+0000 7f2b3b7fe640 1 -- 172.21.15.12:0/287252784 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f2b440051d0 con 0x563d669544d0 2024-04-22T13:30:15.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.371+0000 7f2b64fc1740 1 -- 172.21.15.12:0/287252784 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b54010430 con 0x563d669544d0 2024-04-22T13:30:15.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.371+0000 7f2b3b7fe640 1 -- 172.21.15.12:0/287252784 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2b44050050 con 0x563d669544d0 2024-04-22T13:30:15.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.371+0000 7f2b64fc1740 10 client.17925 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:15.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.371+0000 7f2b64fc1740 1 -- 172.21.15.12:0/287252784 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563d666957d0 con 0x563d669544d0 2024-04-22T13:30:15.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.371+0000 7f2b3b7fe640 1 -- 172.21.15.12:0/287252784 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b54010430 con 0x563d669544d0 2024-04-22T13:30:15.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.371+0000 7f2b3b7fe640 1 -- 172.21.15.12:0/287252784 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2b440035f0 con 0x563d669544d0 2024-04-22T13:30:15.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.371+0000 7f2b64fc1740 10 client.17925 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:15.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.371+0000 7f2b64fc1740 10 client.17925 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:30:15.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.372+0000 7f2b64fc1740 20 client.17925 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:15.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.372+0000 7f2b64fc1740 1 --2- 172.21.15.12:0/287252784 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x563d6696dc50 0x563d66970080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.372+0000 7f2b64fc1740 4 client.17925 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:30:15.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.372+0000 7f2b64fc1740 1 -- 172.21.15.12:0/287252784 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d66858930 con 0x563d6696dc50 2024-04-22T13:30:15.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.372+0000 7f2b5b7fe640 1 --2- 172.21.15.12:0/287252784 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x563d6696dc50 0x563d66970080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.372+0000 7f2b5b7fe640 1 --2- 172.21.15.12:0/287252784 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x563d6696dc50 0x563d66970080 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c7d9fa0c4ff9c8f4 server_cookie=628144ba9f8afab2 in_seq=0 out_seq=0 2024-04-22T13:30:15.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.372+0000 7f2b3b7fe640 10 client.17925 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:30:15.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.373+0000 7f2b3b7fe640 1 -- 172.21.15.12:0/287252784 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563d66858930 con 0x563d6696dc50 2024-04-22T13:30:15.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.373+0000 7f2b3b7fe640 10 client.17925 handle_command_reply: tid=0 2024-04-22T13:30:15.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.373+0000 7f2b64fc1740 1 client.17925 shutdown 2024-04-22T13:30:15.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.373+0000 7f2b64fc1740 1 -- 172.21.15.12:0/287252784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d669544d0 msgr2=0x563d6695d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.373+0000 7f2b64fc1740 1 --2- 172.21.15.12:0/287252784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d669544d0 0x563d6695d350 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009230 tx=0x7f2b4402ca10 comp rx=0 tx=0).stop 2024-04-22T13:30:15.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.373+0000 7f2b64fc1740 1 -- 172.21.15.12:0/287252784 shutdown_connections 2024-04-22T13:30:15.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.373+0000 7f2b64fc1740 1 --2- 172.21.15.12:0/287252784 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x563d6696dc50 0x563d66970080 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.373+0000 7f2b64fc1740 1 --2- 172.21.15.12:0/287252784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d6695da00 0x563d6695acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.373+0000 7f2b3b7fe640 0 client.17925 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:30:15.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.373+0000 7f2b64fc1740 1 --2- 172.21.15.12:0/287252784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d66954e80 0x563d6695a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.373+0000 7f2b64fc1740 1 --2- 172.21.15.12:0/287252784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d669544d0 0x563d6695d350 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.373+0000 7f2b64fc1740 1 -- 172.21.15.12:0/287252784 >> 172.21.15.12:0/287252784 conn(0x563d6694eba0 msgr2=0x563d669508f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:15.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.373+0000 7f2b64fc1740 1 -- 172.21.15.12:0/287252784 shutdown_connections 2024-04-22T13:30:15.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.373+0000 7f2b64fc1740 1 -- 172.21.15.12:0/287252784 wait complete. 2024-04-22T13:30:15.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.373+0000 7f2b64fc1740 20 client.17925 trim_cache size 0 max 16384 2024-04-22T13:30:15.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.386+0000 7f2b64fc1740 1 Processor -- start 2024-04-22T13:30:15.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.386+0000 7f2b64fc1740 1 -- start start 2024-04-22T13:30:15.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.387+0000 7f2b64fc1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d669544d0 0x563d66969830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.387+0000 7f2b64fc1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d66954e80 0x563d6694da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.387+0000 7f2b64fc1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d6695da00 0x563d6695ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.387+0000 7f2b64fc1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5c1ad9f0 con 0x563d6695da00 2024-04-22T13:30:15.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.387+0000 7f2b64fc1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5c1af530 con 0x563d669544d0 2024-04-22T13:30:15.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.387+0000 7f2b64fc1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5c1ae0c0 con 0x563d66954e80 2024-04-22T13:30:15.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.387+0000 7f2b6088f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d6695da00 0x563d6695ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.387+0000 7f2b6088f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d6695da00 0x563d6695ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51522/0 (socket says 172.21.15.12:51522) 2024-04-22T13:30:15.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.387+0000 7f2b6088f640 1 -- 172.21.15.12:0/3972172732 learned_addr learned my addr 172.21.15.12:0/3972172732 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:15.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.387+0000 7f2b5bfff640 1 --2- 172.21.15.12:0/3972172732 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d669544d0 0x563d66969830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.387+0000 7f2b6088f640 1 -- 172.21.15.12:0/3972172732 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d66954e80 msgr2=0x563d6694da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.387+0000 7f2b5b7fe640 1 --2- 172.21.15.12:0/3972172732 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d66954e80 0x563d6694da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.387+0000 7f2b6088f640 1 --2- 172.21.15.12:0/3972172732 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d66954e80 0x563d6694da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.387+0000 7f2b6088f640 1 -- 172.21.15.12:0/3972172732 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d669544d0 msgr2=0x563d66969830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.387+0000 7f2b6088f640 1 --2- 172.21.15.12:0/3972172732 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d669544d0 0x563d66969830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.387+0000 7f2b6088f640 1 -- 172.21.15.12:0/3972172732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d6694f4e0 con 0x563d6695da00 2024-04-22T13:30:15.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.387+0000 7f2b5bfff640 1 --2- 172.21.15.12:0/3972172732 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d669544d0 0x563d66969830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:15.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.387+0000 7f2b6088f640 1 --2- 172.21.15.12:0/3972172732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d6695da00 0x563d6695ce10 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f2b54010de0 tx=0x7f2b54010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7085176b21764197 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:15.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.387+0000 7f2b51ffb640 1 -- 172.21.15.12:0/3972172732 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54004980 con 0x563d6695da00 2024-04-22T13:30:15.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.388+0000 7f2b51ffb640 1 -- 172.21.15.12:0/3972172732 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b5401add0 con 0x563d6695da00 2024-04-22T13:30:15.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.388+0000 7f2b51ffb640 1 -- 172.21.15.12:0/3972172732 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b540197f0 con 0x563d6695da00 2024-04-22T13:30:15.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.388+0000 7f2b64fc1740 1 -- 172.21.15.12:0/3972172732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d6695da00 msgr2=0x563d6695ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.388+0000 7f2b64fc1740 1 --2- 172.21.15.12:0/3972172732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d6695da00 0x563d6695ce10 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f2b54010de0 tx=0x7f2b54010d70 comp rx=0 tx=0).stop 2024-04-22T13:30:15.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.388+0000 7f2b64fc1740 1 -- 172.21.15.12:0/3972172732 shutdown_connections 2024-04-22T13:30:15.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.388+0000 7f2b64fc1740 1 --2- 172.21.15.12:0/3972172732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d6695da00 0x563d6695ce10 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.388+0000 7f2b64fc1740 1 --2- 172.21.15.12:0/3972172732 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d66954e80 0x563d6694da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.388+0000 7f2b64fc1740 1 --2- 172.21.15.12:0/3972172732 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d669544d0 0x563d66969830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.388+0000 7f2b64fc1740 1 -- 172.21.15.12:0/3972172732 >> 172.21.15.12:0/3972172732 conn(0x563d6694eba0 msgr2=0x563d6694d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:15.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.388+0000 7f2b64fc1740 1 -- 172.21.15.12:0/3972172732 shutdown_connections 2024-04-22T13:30:15.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.388+0000 7f2b64fc1740 1 -- 172.21.15.12:0/3972172732 wait complete. 2024-04-22T13:30:15.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.388+0000 7f2b64fc1740 1 -- start start 2024-04-22T13:30:15.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.389+0000 7f2b64fc1740 1 Processor -- start 2024-04-22T13:30:15.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.389+0000 7f2b64fc1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d669544d0 0x563d66966200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.389+0000 7f2b64fc1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d66954e80 0x563d66957100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.389+0000 7f2b64fc1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d6695da00 0x563d669576a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.389+0000 7f2b64fc1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5c1aee60 con 0x563d6695da00 2024-04-22T13:30:15.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.389+0000 7f2b64fc1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5c1ae790 con 0x563d669544d0 2024-04-22T13:30:15.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.389+0000 7f2b64fc1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5c1ab9f0 con 0x563d66954e80 2024-04-22T13:30:15.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.389+0000 7f2b64fc1740 10 client.0 _ll_register_callbacks cb 0x7f2b44004490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:15.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.389+0000 7f2b6088f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d6695da00 0x563d669576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.389+0000 7f2b6088f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d6695da00 0x563d669576a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51538/0 (socket says 172.21.15.12:51538) 2024-04-22T13:30:15.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.389+0000 7f2b6088f640 1 -- 172.21.15.12:0/1429207988 learned_addr learned my addr 172.21.15.12:0/1429207988 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:15.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.389+0000 7f2b5bfff640 1 --2- 172.21.15.12:0/1429207988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d669544d0 0x563d66966200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.389+0000 7f2b5b7fe640 1 --2- 172.21.15.12:0/1429207988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d66954e80 0x563d66957100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.389+0000 7f2b6088f640 1 -- 172.21.15.12:0/1429207988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d66954e80 msgr2=0x563d66957100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.389+0000 7f2b6088f640 1 --2- 172.21.15.12:0/1429207988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d66954e80 0x563d66957100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.389+0000 7f2b6088f640 1 -- 172.21.15.12:0/1429207988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d669544d0 msgr2=0x563d66966200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.389+0000 7f2b6088f640 1 --2- 172.21.15.12:0/1429207988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d669544d0 0x563d66966200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.389+0000 7f2b6088f640 1 -- 172.21.15.12:0/1429207988 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d66957ed0 con 0x563d6695da00 2024-04-22T13:30:15.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.389+0000 7f2b5b7fe640 1 --2- 172.21.15.12:0/1429207988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d66954e80 0x563d66957100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:15.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.390+0000 7f2b6088f640 1 --2- 172.21.15.12:0/1429207988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d6695da00 0x563d669576a0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f2b54012090 tx=0x7f2b54007a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d21b038a8eb819b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:15.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.390+0000 7f2b3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:15.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.390+0000 7f2b3b7fe640 1 -- 172.21.15.12:0/1429207988 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5401a920 con 0x563d6695da00 2024-04-22T13:30:15.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.390+0000 7f2b3b7fe640 1 -- 172.21.15.12:0/1429207988 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f2b5401aac0 con 0x563d6695da00 2024-04-22T13:30:15.417 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.390+0000 7f2b64fc1740 1 -- 172.21.15.12:0/1429207988 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b5401add0 con 0x563d6695da00 2024-04-22T13:30:15.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.390+0000 7f2b3b7fe640 1 -- 172.21.15.12:0/1429207988 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2b5403d020 con 0x563d6695da00 2024-04-22T13:30:15.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.390+0000 7f2b64fc1740 10 client.17931 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:15.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.390+0000 7f2b64fc1740 1 -- 172.21.15.12:0/1429207988 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563d669d4280 con 0x563d6695da00 2024-04-22T13:30:15.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.390+0000 7f2b3b7fe640 1 -- 172.21.15.12:0/1429207988 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b5401add0 con 0x563d6695da00 2024-04-22T13:30:15.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.390+0000 7f2b3b7fe640 1 -- 172.21.15.12:0/1429207988 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2b54010530 con 0x563d6695da00 2024-04-22T13:30:15.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.390+0000 7f2b64fc1740 10 client.17931 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:15.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.390+0000 7f2b64fc1740 10 client.17931 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:30:15.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.390+0000 7f2b64fc1740 20 client.17931 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:15.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.390+0000 7f2b64fc1740 1 --2- 172.21.15.12:0/1429207988 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x563d669d4740 0x563d669d6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.390+0000 7f2b64fc1740 4 client.17931 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:30:15.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.390+0000 7f2b64fc1740 1 -- 172.21.15.12:0/1429207988 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563d668c6c10 con 0x563d669d4740 2024-04-22T13:30:15.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.391+0000 7f2b5bfff640 1 --2- 172.21.15.12:0/1429207988 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x563d669d4740 0x563d669d6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.391+0000 7f2b3b7fe640 10 client.17931 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:30:15.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.391+0000 7f2b5bfff640 1 --2- 172.21.15.12:0/1429207988 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x563d669d4740 0x563d669d6b30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2de797e847ae5372 server_cookie=674b8d95054bde45 in_seq=0 out_seq=0 2024-04-22T13:30:15.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.391+0000 7f2b3b7fe640 1 -- 172.21.15.12:0/1429207988 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563d668c6c10 con 0x563d669d4740 2024-04-22T13:30:15.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.391+0000 7f2b3b7fe640 10 client.17931 handle_command_reply: tid=0 2024-04-22T13:30:15.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.391+0000 7f2b64fc1740 1 client.17931 shutdown 2024-04-22T13:30:15.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.391+0000 7f2b64fc1740 1 -- 172.21.15.12:0/1429207988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d6695da00 msgr2=0x563d669576a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.391+0000 7f2b64fc1740 1 --2- 172.21.15.12:0/1429207988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d6695da00 0x563d669576a0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f2b54012090 tx=0x7f2b54007a80 comp rx=0 tx=0).stop 2024-04-22T13:30:15.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.391+0000 7f2b64fc1740 1 -- 172.21.15.12:0/1429207988 shutdown_connections 2024-04-22T13:30:15.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.391+0000 7f2b64fc1740 1 --2- 172.21.15.12:0/1429207988 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x563d669d4740 0x563d669d6b30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.391+0000 7f2b64fc1740 1 --2- 172.21.15.12:0/1429207988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563d6695da00 0x563d669576a0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.391+0000 7f2b64fc1740 1 --2- 172.21.15.12:0/1429207988 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563d66954e80 0x563d66957100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.391+0000 7f2b64fc1740 1 --2- 172.21.15.12:0/1429207988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563d669544d0 0x563d66966200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.391+0000 7f2b3b7fe640 0 client.17931 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:30:15.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.391+0000 7f2b64fc1740 1 -- 172.21.15.12:0/1429207988 >> 172.21.15.12:0/1429207988 conn(0x563d6694eba0 msgr2=0x563d669507b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:15.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.391+0000 7f2b64fc1740 1 -- 172.21.15.12:0/1429207988 shutdown_connections 2024-04-22T13:30:15.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.391+0000 7f2b64fc1740 1 -- 172.21.15.12:0/1429207988 wait complete. 2024-04-22T13:30:15.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.392+0000 7f2b64fc1740 20 client.17931 trim_cache size 0 max 16384 2024-04-22T13:30:15.423 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:30:15.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.392+0000 7f2b622f2640 1 -- 172.21.15.12:0/3799227338 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2b480a3d20 msgr2=0x7f2b480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.392+0000 7f2b622f2640 1 --2- 172.21.15.12:0/3799227338 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2b480a3d20 0x7f2b480a61e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2b54020000 tx=0x7f2b5401d000 comp rx=0 tx=0).stop 2024-04-22T13:30:15.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.392+0000 7f2b622f2640 1 -- 172.21.15.12:0/3799227338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b5c107270 msgr2=0x7f2b5c198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.392+0000 7f2b622f2640 1 --2- 172.21.15.12:0/3799227338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b5c107270 0x7f2b5c198bf0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c00e9f0 tx=0x7f2b4c00eeb0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.392+0000 7f2b622f2640 1 -- 172.21.15.12:0/3799227338 shutdown_connections 2024-04-22T13:30:15.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.392+0000 7f2b622f2640 1 --2- 172.21.15.12:0/3799227338 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2b480a3d20 0x7f2b480a61e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.392+0000 7f2b622f2640 1 --2- 172.21.15.12:0/3799227338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b5c19cf70 0x7f2b5c19d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.392+0000 7f2b622f2640 1 --2- 172.21.15.12:0/3799227338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2b5c107270 0x7f2b5c198bf0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.392+0000 7f2b622f2640 1 --2- 172.21.15.12:0/3799227338 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b5c06a840 0x7f2b5c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.392+0000 7f2b622f2640 1 -- 172.21.15.12:0/3799227338 >> 172.21.15.12:0/3799227338 conn(0x7f2b5c077eb0 msgr2=0x7f2b5c100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:15.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.392+0000 7f2b622f2640 1 -- 172.21.15.12:0/3799227338 shutdown_connections 2024-04-22T13:30:15.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.392+0000 7f2b622f2640 1 -- 172.21.15.12:0/3799227338 wait complete. 2024-04-22T13:30:15.426 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:30:15.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.534+0000 7f73f14d5640 1 -- 172.21.15.12:0/799988215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73ec106170 msgr2=0x7f73ec10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.534+0000 7f73f14d5640 1 --2- 172.21.15.12:0/799988215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73ec106170 0x7f73ec10dd40 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f73e4009ca0 tx=0x7f73e402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.534+0000 7f73f14d5640 1 -- 172.21.15.12:0/799988215 shutdown_connections 2024-04-22T13:30:15.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.534+0000 7f73f14d5640 1 --2- 172.21.15.12:0/799988215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73ec106170 0x7f73ec10dd40 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.534+0000 7f73f14d5640 1 --2- 172.21.15.12:0/799988215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73ec101a50 0x7f73ec105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.534+0000 7f73f14d5640 1 --2- 172.21.15.12:0/799988215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73ec1010a0 0x7f73ec101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.534+0000 7f73f14d5640 1 -- 172.21.15.12:0/799988215 >> 172.21.15.12:0/799988215 conn(0x7f73ec077eb0 msgr2=0x7f73ec0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:15.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.534+0000 7f73f14d5640 1 -- 172.21.15.12:0/799988215 shutdown_connections 2024-04-22T13:30:15.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.534+0000 7f73f14d5640 1 -- 172.21.15.12:0/799988215 wait complete. 2024-04-22T13:30:15.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.534+0000 7f73f14d5640 1 Processor -- start 2024-04-22T13:30:15.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.535+0000 7f73f14d5640 1 -- start start 2024-04-22T13:30:15.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.535+0000 7f73f14d5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73ec1010a0 0x7f73ec1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.535+0000 7f73f14d5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73ec106170 0x7f73ec198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.535+0000 7f73f14d5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73ec19cf90 0x7f73ec19d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.535+0000 7f73f14d5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1110e0 con 0x7f73ec1010a0 2024-04-22T13:30:15.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.535+0000 7f73f14d5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec110f60 con 0x7f73ec106170 2024-04-22T13:30:15.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.535+0000 7f73f14d5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec111260 con 0x7f73ec19cf90 2024-04-22T13:30:15.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.535+0000 7f73eaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73ec1010a0 0x7f73ec1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.535+0000 7f73eaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73ec1010a0 0x7f73ec1986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51570/0 (socket says 172.21.15.12:51570) 2024-04-22T13:30:15.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.535+0000 7f73eaffd640 1 -- 172.21.15.12:0/1365039335 learned_addr learned my addr 172.21.15.12:0/1365039335 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:15.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.535+0000 7f73e27fc640 1 --2- 172.21.15.12:0/1365039335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73ec106170 0x7f73ec198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.535+0000 7f73eb7fe640 1 --2- 172.21.15.12:0/1365039335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73ec19cf90 0x7f73ec19d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.535+0000 7f73eaffd640 1 -- 172.21.15.12:0/1365039335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73ec19cf90 msgr2=0x7f73ec19d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.535+0000 7f73eaffd640 1 --2- 172.21.15.12:0/1365039335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73ec19cf90 0x7f73ec19d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.535+0000 7f73eaffd640 1 -- 172.21.15.12:0/1365039335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73ec106170 msgr2=0x7f73ec198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.535+0000 7f73eaffd640 1 --2- 172.21.15.12:0/1365039335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73ec106170 0x7f73ec198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.535+0000 7f73eaffd640 1 -- 172.21.15.12:0/1365039335 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ec19da80 con 0x7f73ec1010a0 2024-04-22T13:30:15.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.535+0000 7f73eaffd640 1 --2- 172.21.15.12:0/1365039335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73ec1010a0 0x7f73ec1986d0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f73dc00c890 tx=0x7f73dc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:15.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.536+0000 7f73e8ff9640 1 -- 172.21.15.12:0/1365039335 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc004220 con 0x7f73ec1010a0 2024-04-22T13:30:15.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.536+0000 7f73e8ff9640 1 -- 172.21.15.12:0/1365039335 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73dc0043c0 con 0x7f73ec1010a0 2024-04-22T13:30:15.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.536+0000 7f73f14d5640 1 -- 172.21.15.12:0/1365039335 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73ec19dd70 con 0x7f73ec1010a0 2024-04-22T13:30:15.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.536+0000 7f73e8ff9640 1 -- 172.21.15.12:0/1365039335 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc004a00 con 0x7f73ec1010a0 2024-04-22T13:30:15.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.536+0000 7f73f14d5640 1 -- 172.21.15.12:0/1365039335 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73ec1a55b0 con 0x7f73ec1010a0 2024-04-22T13:30:15.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.538+0000 7f73e8ff9640 1 -- 172.21.15.12:0/1365039335 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f73dc0079b0 con 0x7f73ec1010a0 2024-04-22T13:30:15.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.538+0000 7f73f419c740 1 Processor -- start 2024-04-22T13:30:15.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.538+0000 7f73f419c740 1 -- start start 2024-04-22T13:30:15.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.538+0000 7f73f419c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c731894d0 0x559c731898b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.538+0000 7f73f419c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c73189e80 0x559c73192430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.538+0000 7f73f419c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c73192a00 0x559c73194e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.538+0000 7f73f419c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559c730ee1f0 con 0x559c73189e80 2024-04-22T13:30:15.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.538+0000 7f73f419c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559c7308b4b0 con 0x559c73192a00 2024-04-22T13:30:15.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.538+0000 7f73f419c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559c730ea1b0 con 0x559c731894d0 2024-04-22T13:30:15.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.539+0000 7f73eaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c73189e80 0x559c73192430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.539+0000 7f73eaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c73189e80 0x559c73192430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51578/0 (socket says 172.21.15.12:51578) 2024-04-22T13:30:15.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.539+0000 7f73eaffd640 1 -- 172.21.15.12:0/4283200712 learned_addr learned my addr 172.21.15.12:0/4283200712 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:15.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.539+0000 7f73e8ff9640 1 --2- 172.21.15.12:0/1365039335 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f73d80a3da0 0x7f73d80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.539+0000 7f73eb7fe640 1 --2- 172.21.15.12:0/4283200712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c731894d0 0x559c731898b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.539+0000 7f73eb7fe640 1 --2- 172.21.15.12:0/1365039335 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f73d80a3da0 0x7f73d80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.539+0000 7f73e27fc640 1 --2- 172.21.15.12:0/4283200712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c73192a00 0x559c73194e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.539+0000 7f73e8ff9640 1 -- 172.21.15.12:0/1365039335 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f73dc0d03d0 con 0x7f73ec1010a0 2024-04-22T13:30:15.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.539+0000 7f73eaffd640 1 -- 172.21.15.12:0/4283200712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c731894d0 msgr2=0x559c731898b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.539+0000 7f73eaffd640 1 --2- 172.21.15.12:0/4283200712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c731894d0 0x559c731898b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.539+0000 7f73eaffd640 1 -- 172.21.15.12:0/4283200712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c73192a00 msgr2=0x559c73194e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.539+0000 7f73eaffd640 1 --2- 172.21.15.12:0/4283200712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c73192a00 0x559c73194e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.539+0000 7f73eaffd640 1 -- 172.21.15.12:0/4283200712 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c7307d1b0 con 0x559c73189e80 2024-04-22T13:30:15.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.539+0000 7f73eb7fe640 1 --2- 172.21.15.12:0/4283200712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c731894d0 0x559c731898b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:30:15.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.539+0000 7f73eaffd640 1 --2- 172.21.15.12:0/4283200712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c73189e80 0x559c73192430 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f73dc09c400 tx=0x7f73dc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66f7e8eea011e5f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:15.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.539+0000 7f73e17fa640 1 -- 172.21.15.12:0/4283200712 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc01d050 con 0x559c73189e80 2024-04-22T13:30:15.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.539+0000 7f73e17fa640 1 -- 172.21.15.12:0/4283200712 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73dc005290 con 0x559c73189e80 2024-04-22T13:30:15.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.539+0000 7f73e17fa640 1 -- 172.21.15.12:0/4283200712 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc09a800 con 0x559c73189e80 2024-04-22T13:30:15.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.540+0000 7f73f419c740 1 -- 172.21.15.12:0/4283200712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c73189e80 msgr2=0x559c73192430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.540+0000 7f73eb7fe640 1 --2- 172.21.15.12:0/1365039335 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f73d80a3da0 0x7f73d80a6260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f73e4004af0 tx=0x7f73e4005c50 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:15.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.540+0000 7f73f419c740 1 --2- 172.21.15.12:0/4283200712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c73189e80 0x559c73192430 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f73dc09c400 tx=0x7f73dc000bc0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.540+0000 7f73f419c740 1 -- 172.21.15.12:0/4283200712 shutdown_connections 2024-04-22T13:30:15.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.540+0000 7f73f419c740 1 --2- 172.21.15.12:0/4283200712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c73192a00 0x559c73194e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.540+0000 7f73f419c740 1 --2- 172.21.15.12:0/4283200712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c73189e80 0x559c73192430 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.540+0000 7f73f419c740 1 --2- 172.21.15.12:0/4283200712 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c731894d0 0x559c731898b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.540+0000 7f73f419c740 1 -- 172.21.15.12:0/4283200712 >> 172.21.15.12:0/4283200712 conn(0x559c73183ba0 msgr2=0x559c73183f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:15.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.540+0000 7f73f419c740 1 -- 172.21.15.12:0/4283200712 shutdown_connections 2024-04-22T13:30:15.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.540+0000 7f73f419c740 1 -- 172.21.15.12:0/4283200712 wait complete. 2024-04-22T13:30:15.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.541+0000 7f73f419c740 1 -- start start 2024-04-22T13:30:15.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.541+0000 7f73f419c740 1 Processor -- start 2024-04-22T13:30:15.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.541+0000 7f73f419c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c731894d0 0x559c73192380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.541+0000 7f73f419c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c73189e80 0x559c7318f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.541+0000 7f73f419c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c73192a00 0x559c7318fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.541+0000 7f73f419c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559c731958b0 con 0x559c731894d0 2024-04-22T13:30:15.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.541+0000 7f73f419c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559c730a7500 con 0x559c73192a00 2024-04-22T13:30:15.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.541+0000 7f73f419c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559c73195a30 con 0x559c73189e80 2024-04-22T13:30:15.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.541+0000 7f73f419c740 10 client.0 _ll_register_callbacks cb 0x559c73075b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:15.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.541+0000 7f73eaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c731894d0 0x559c73192380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.541+0000 7f73eaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c731894d0 0x559c73192380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51590/0 (socket says 172.21.15.12:51590) 2024-04-22T13:30:15.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.541+0000 7f73eaffd640 1 -- 172.21.15.12:0/2135316940 learned_addr learned my addr 172.21.15.12:0/2135316940 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:15.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.541+0000 7f73e27fc640 1 --2- 172.21.15.12:0/2135316940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c73189e80 0x559c7318f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.541+0000 7f73eb7fe640 1 --2- 172.21.15.12:0/2135316940 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c73192a00 0x559c7318fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.541+0000 7f73eaffd640 1 -- 172.21.15.12:0/2135316940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c73189e80 msgr2=0x559c7318f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.541+0000 7f73eaffd640 1 --2- 172.21.15.12:0/2135316940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c73189e80 0x559c7318f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.541+0000 7f73eaffd640 1 -- 172.21.15.12:0/2135316940 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c73192a00 msgr2=0x559c7318fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.541+0000 7f73eaffd640 1 --2- 172.21.15.12:0/2135316940 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c73192a00 0x559c7318fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.541+0000 7f73eaffd640 1 -- 172.21.15.12:0/2135316940 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c72e003f0 con 0x559c731894d0 2024-04-22T13:30:15.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.542+0000 7f73eaffd640 1 --2- 172.21.15.12:0/2135316940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c731894d0 0x559c73192380 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f73dc0cdc20 tx=0x7f73dc0d3c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72cc6d9fcb35207 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:15.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.542+0000 7f73d27fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:15.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.542+0000 7f73d27fc640 1 -- 172.21.15.12:0/2135316940 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc0b0e60 con 0x559c731894d0 2024-04-22T13:30:15.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.542+0000 7f73d27fc640 1 -- 172.21.15.12:0/2135316940 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f73dc098ae0 con 0x559c731894d0 2024-04-22T13:30:15.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.542+0000 7f73f419c740 1 -- 172.21.15.12:0/2135316940 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559c7308d930 con 0x559c731894d0 2024-04-22T13:30:15.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.542+0000 7f73d27fc640 1 -- 172.21.15.12:0/2135316940 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f73dc0b4030 con 0x559c731894d0 2024-04-22T13:30:15.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.542+0000 7f73f419c740 10 client.17943 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:15.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.542+0000 7f73f419c740 1 -- 172.21.15.12:0/2135316940 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559c72eca7d0 con 0x559c731894d0 2024-04-22T13:30:15.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.542+0000 7f73d27fc640 1 -- 172.21.15.12:0/2135316940 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559c7308d930 con 0x559c731894d0 2024-04-22T13:30:15.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.542+0000 7f73d27fc640 1 -- 172.21.15.12:0/2135316940 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f73dc0981c0 con 0x559c731894d0 2024-04-22T13:30:15.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.542+0000 7f73f419c740 10 client.17943 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:15.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.542+0000 7f73f419c740 10 client.17943 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:30:15.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.542+0000 7f73f419c740 20 client.17943 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:15.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.542+0000 7f73f419c740 1 --2- 172.21.15.12:0/2135316940 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x559c731a29d0 0x559c731a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.542+0000 7f73f419c740 4 client.17943 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:30:15.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.542+0000 7f73f419c740 1 -- 172.21.15.12:0/2135316940 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c730fbc10 con 0x559c731a29d0 2024-04-22T13:30:15.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.543+0000 7f73e27fc640 1 --2- 172.21.15.12:0/2135316940 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x559c731a29d0 0x559c731a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.543+0000 7f73d27fc640 10 client.17943 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:30:15.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.543+0000 7f73e27fc640 1 --2- 172.21.15.12:0/2135316940 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x559c731a29d0 0x559c731a4dc0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8f81d52e6ba0c9b0 server_cookie=b9c52792163ff1f2 in_seq=0 out_seq=0 2024-04-22T13:30:15.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.544+0000 7f73d27fc640 1 -- 172.21.15.12:0/2135316940 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559c730fbc10 con 0x559c731a29d0 2024-04-22T13:30:15.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.544+0000 7f73d27fc640 10 client.17943 handle_command_reply: tid=0 2024-04-22T13:30:15.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.544+0000 7f73f419c740 1 client.17943 shutdown 2024-04-22T13:30:15.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.544+0000 7f73f419c740 1 -- 172.21.15.12:0/2135316940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c731894d0 msgr2=0x559c73192380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.544+0000 7f73f419c740 1 --2- 172.21.15.12:0/2135316940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c731894d0 0x559c73192380 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f73dc0cdc20 tx=0x7f73dc0d3c20 comp rx=0 tx=0).stop 2024-04-22T13:30:15.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.544+0000 7f73f419c740 1 -- 172.21.15.12:0/2135316940 shutdown_connections 2024-04-22T13:30:15.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.544+0000 7f73f419c740 1 --2- 172.21.15.12:0/2135316940 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x559c731a29d0 0x559c731a4dc0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.544+0000 7f73f419c740 1 --2- 172.21.15.12:0/2135316940 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c73192a00 0x559c7318fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.544+0000 7f73f419c740 1 --2- 172.21.15.12:0/2135316940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c73189e80 0x559c7318f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.544+0000 7f73f419c740 1 --2- 172.21.15.12:0/2135316940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c731894d0 0x559c73192380 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.544+0000 7f73d27fc640 0 client.17943 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:30:15.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.544+0000 7f73f419c740 1 -- 172.21.15.12:0/2135316940 >> 172.21.15.12:0/2135316940 conn(0x559c73183ba0 msgr2=0x559c731858f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:15.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.544+0000 7f73f419c740 1 -- 172.21.15.12:0/2135316940 shutdown_connections 2024-04-22T13:30:15.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.544+0000 7f73f419c740 1 -- 172.21.15.12:0/2135316940 wait complete. 2024-04-22T13:30:15.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.544+0000 7f73f419c740 20 client.17943 trim_cache size 0 max 16384 2024-04-22T13:30:15.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.557+0000 7f73f419c740 1 Processor -- start 2024-04-22T13:30:15.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.557+0000 7f73f419c740 1 -- start start 2024-04-22T13:30:15.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.558+0000 7f73f419c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c731894d0 0x559c73191e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.558+0000 7f73f419c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c73189e80 0x559c73192380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.558+0000 7f73f419c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c73192a00 0x559c731a7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.558+0000 7f73f419c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1adab0 con 0x559c73192a00 2024-04-22T13:30:15.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.558+0000 7f73f419c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1af5f0 con 0x559c731894d0 2024-04-22T13:30:15.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.558+0000 7f73f419c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1ae180 con 0x559c73189e80 2024-04-22T13:30:15.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.558+0000 7f73eb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c73192a00 0x559c731a7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.558+0000 7f73eb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c73192a00 0x559c731a7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51602/0 (socket says 172.21.15.12:51602) 2024-04-22T13:30:15.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.558+0000 7f73eb7fe640 1 -- 172.21.15.12:0/1989865385 learned_addr learned my addr 172.21.15.12:0/1989865385 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:15.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.558+0000 7f73eaffd640 1 --2- 172.21.15.12:0/1989865385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c731894d0 0x559c73191e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.558+0000 7f73e27fc640 1 --2- 172.21.15.12:0/1989865385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c73189e80 0x559c73192380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.558+0000 7f73eb7fe640 1 -- 172.21.15.12:0/1989865385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c73189e80 msgr2=0x559c73192380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.558+0000 7f73eb7fe640 1 --2- 172.21.15.12:0/1989865385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c73189e80 0x559c73192380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.558+0000 7f73eb7fe640 1 -- 172.21.15.12:0/1989865385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c731894d0 msgr2=0x559c73191e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.558+0000 7f73eb7fe640 1 --2- 172.21.15.12:0/1989865385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c731894d0 0x559c73191e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.558+0000 7f73eb7fe640 1 -- 172.21.15.12:0/1989865385 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c73190090 con 0x559c73192a00 2024-04-22T13:30:15.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.558+0000 7f73e27fc640 1 --2- 172.21.15.12:0/1989865385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c73189e80 0x559c73192380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:15.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.558+0000 7f73eb7fe640 1 --2- 172.21.15.12:0/1989865385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c73192a00 0x559c731a7de0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f73e4009ca0 tx=0x7f73e402e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed9a6d86b4bb770 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:15.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.558+0000 7f73e0ff9640 1 -- 172.21.15.12:0/1989865385 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4050070 con 0x559c73192a00 2024-04-22T13:30:15.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.558+0000 7f73e0ff9640 1 -- 172.21.15.12:0/1989865385 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73e4003b70 con 0x559c73192a00 2024-04-22T13:30:15.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.559+0000 7f73e0ff9640 1 -- 172.21.15.12:0/1989865385 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e4044070 con 0x559c73192a00 2024-04-22T13:30:15.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.559+0000 7f73f419c740 1 -- 172.21.15.12:0/1989865385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c73192a00 msgr2=0x559c731a7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.559+0000 7f73f419c740 1 --2- 172.21.15.12:0/1989865385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c73192a00 0x559c731a7de0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f73e4009ca0 tx=0x7f73e402e040 comp rx=0 tx=0).stop 2024-04-22T13:30:15.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.559+0000 7f73f419c740 1 -- 172.21.15.12:0/1989865385 shutdown_connections 2024-04-22T13:30:15.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.559+0000 7f73f419c740 1 --2- 172.21.15.12:0/1989865385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c73192a00 0x559c731a7de0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.559+0000 7f73f419c740 1 --2- 172.21.15.12:0/1989865385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c73189e80 0x559c73192380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.559+0000 7f73f419c740 1 --2- 172.21.15.12:0/1989865385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c731894d0 0x559c73191e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.559+0000 7f73f419c740 1 -- 172.21.15.12:0/1989865385 >> 172.21.15.12:0/1989865385 conn(0x559c73183ba0 msgr2=0x559c7319e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:15.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.559+0000 7f73f419c740 1 -- 172.21.15.12:0/1989865385 shutdown_connections 2024-04-22T13:30:15.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.559+0000 7f73f419c740 1 -- 172.21.15.12:0/1989865385 wait complete. 2024-04-22T13:30:15.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.559+0000 7f73f419c740 1 -- start start 2024-04-22T13:30:15.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.560+0000 7f73f419c740 1 Processor -- start 2024-04-22T13:30:15.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.560+0000 7f73f419c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c731894d0 0x559c731a5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.560+0000 7f73f419c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c73189e80 0x559c731a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.560+0000 7f73f419c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c73192a00 0x559c731a6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.560+0000 7f73f419c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1aef20 con 0x559c731894d0 2024-04-22T13:30:15.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.560+0000 7f73f419c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1ae850 con 0x559c73189e80 2024-04-22T13:30:15.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.560+0000 7f73f419c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec1aba90 con 0x559c73192a00 2024-04-22T13:30:15.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.560+0000 7f73f419c740 10 client.0 _ll_register_callbacks cb 0x7f73dc09c470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:15.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.560+0000 7f73eaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c731894d0 0x559c731a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.560+0000 7f73eaffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c731894d0 0x559c731a5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51614/0 (socket says 172.21.15.12:51614) 2024-04-22T13:30:15.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.560+0000 7f73eaffd640 1 -- 172.21.15.12:0/3917089440 learned_addr learned my addr 172.21.15.12:0/3917089440 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:15.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.560+0000 7f73e27fc640 1 --2- 172.21.15.12:0/3917089440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c73189e80 0x559c731a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.560+0000 7f73eb7fe640 1 --2- 172.21.15.12:0/3917089440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c73192a00 0x559c731a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.560+0000 7f73eaffd640 1 -- 172.21.15.12:0/3917089440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c73192a00 msgr2=0x559c731a6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.560+0000 7f73eaffd640 1 --2- 172.21.15.12:0/3917089440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c73192a00 0x559c731a6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.560+0000 7f73eaffd640 1 -- 172.21.15.12:0/3917089440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c73189e80 msgr2=0x559c731a5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.560+0000 7f73eaffd640 1 --2- 172.21.15.12:0/3917089440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c73189e80 0x559c731a5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.560+0000 7f73eaffd640 1 -- 172.21.15.12:0/3917089440 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c7320bfc0 con 0x559c731894d0 2024-04-22T13:30:15.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.560+0000 7f73eaffd640 1 --2- 172.21.15.12:0/3917089440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c731894d0 0x559c731a5780 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f73dc005fd0 tx=0x7f73dc0d3a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44af36190355bb47 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:15.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.560+0000 7f73d27fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:15.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.560+0000 7f73d27fc640 1 -- 172.21.15.12:0/3917089440 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc01d040 con 0x559c731894d0 2024-04-22T13:30:15.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.560+0000 7f73d27fc640 1 -- 172.21.15.12:0/3917089440 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f73dc09b590 con 0x559c731894d0 2024-04-22T13:30:15.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.560+0000 7f73f419c740 1 -- 172.21.15.12:0/3917089440 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73e4003b70 con 0x559c731894d0 2024-04-22T13:30:15.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.561+0000 7f73d27fc640 1 -- 172.21.15.12:0/3917089440 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f73dc0b0e60 con 0x559c731894d0 2024-04-22T13:30:15.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.561+0000 7f73f419c740 10 client.17949 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:15.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.561+0000 7f73f419c740 1 -- 172.21.15.12:0/3917089440 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559c7320d200 con 0x559c731894d0 2024-04-22T13:30:15.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.561+0000 7f73d27fc640 1 -- 172.21.15.12:0/3917089440 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73e4003b70 con 0x559c731894d0 2024-04-22T13:30:15.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.561+0000 7f73d27fc640 1 -- 172.21.15.12:0/3917089440 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f73dc0981c0 con 0x559c731894d0 2024-04-22T13:30:15.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.561+0000 7f73f419c740 10 client.17949 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:15.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.561+0000 7f73f419c740 10 client.17949 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:30:15.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.561+0000 7f73f419c740 20 client.17949 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:15.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.561+0000 7f73f419c740 1 --2- 172.21.15.12:0/3917089440 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x559c7320d730 0x559c7320fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.561+0000 7f73f419c740 4 client.17949 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:30:15.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.561+0000 7f73f419c740 1 -- 172.21.15.12:0/3917089440 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559c730f5a80 con 0x559c7320d730 2024-04-22T13:30:15.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.561+0000 7f73e27fc640 1 --2- 172.21.15.12:0/3917089440 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x559c7320d730 0x559c7320fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.562+0000 7f73d27fc640 10 client.17949 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:30:15.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.562+0000 7f73e27fc640 1 --2- 172.21.15.12:0/3917089440 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x559c7320d730 0x559c7320fb20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=439339105ec2fa89 server_cookie=874fc68d6fac23be in_seq=0 out_seq=0 2024-04-22T13:30:15.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.562+0000 7f73d27fc640 1 -- 172.21.15.12:0/3917089440 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559c730f5a80 con 0x559c7320d730 2024-04-22T13:30:15.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.562+0000 7f73d27fc640 10 client.17949 handle_command_reply: tid=0 2024-04-22T13:30:15.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.562+0000 7f73f419c740 1 client.17949 shutdown 2024-04-22T13:30:15.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.562+0000 7f73f419c740 1 -- 172.21.15.12:0/3917089440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c731894d0 msgr2=0x559c731a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.562+0000 7f73f419c740 1 --2- 172.21.15.12:0/3917089440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c731894d0 0x559c731a5780 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f73dc005fd0 tx=0x7f73dc0d3a30 comp rx=0 tx=0).stop 2024-04-22T13:30:15.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.562+0000 7f73f419c740 1 -- 172.21.15.12:0/3917089440 shutdown_connections 2024-04-22T13:30:15.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.562+0000 7f73f419c740 1 --2- 172.21.15.12:0/3917089440 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x559c7320d730 0x559c7320fb20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.562+0000 7f73f419c740 1 --2- 172.21.15.12:0/3917089440 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c73192a00 0x559c731a6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.562+0000 7f73d27fc640 0 client.17949 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:30:15.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.562+0000 7f73f419c740 1 --2- 172.21.15.12:0/3917089440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c73189e80 0x559c731a5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.562+0000 7f73f419c740 1 --2- 172.21.15.12:0/3917089440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559c731894d0 0x559c731a5780 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.562+0000 7f73f419c740 1 -- 172.21.15.12:0/3917089440 >> 172.21.15.12:0/3917089440 conn(0x559c73183ba0 msgr2=0x559c73182370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:15.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.562+0000 7f73f419c740 1 -- 172.21.15.12:0/3917089440 shutdown_connections 2024-04-22T13:30:15.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.562+0000 7f73f419c740 1 -- 172.21.15.12:0/3917089440 wait complete. 2024-04-22T13:30:15.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.562+0000 7f73f419c740 20 client.17949 trim_cache size 0 max 16384 2024-04-22T13:30:15.566 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:30:15.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.562+0000 7f73f14d5640 1 -- 172.21.15.12:0/1365039335 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f73d80a3da0 msgr2=0x7f73d80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.562+0000 7f73f14d5640 1 --2- 172.21.15.12:0/1365039335 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f73d80a3da0 0x7f73d80a6260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f73e4004af0 tx=0x7f73e4005c50 comp rx=0 tx=0).stop 2024-04-22T13:30:15.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.562+0000 7f73f14d5640 1 -- 172.21.15.12:0/1365039335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73ec1010a0 msgr2=0x7f73ec1986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.562+0000 7f73f14d5640 1 --2- 172.21.15.12:0/1365039335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73ec1010a0 0x7f73ec1986d0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f73dc00c890 tx=0x7f73dc00cd50 comp rx=0 tx=0).stop 2024-04-22T13:30:15.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.563+0000 7f73f14d5640 1 -- 172.21.15.12:0/1365039335 shutdown_connections 2024-04-22T13:30:15.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.563+0000 7f73f14d5640 1 --2- 172.21.15.12:0/1365039335 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f73d80a3da0 0x7f73d80a6260 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.563+0000 7f73f14d5640 1 --2- 172.21.15.12:0/1365039335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73ec19cf90 0x7f73ec19d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.563+0000 7f73f14d5640 1 --2- 172.21.15.12:0/1365039335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73ec106170 0x7f73ec198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.563+0000 7f73f14d5640 1 --2- 172.21.15.12:0/1365039335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f73ec1010a0 0x7f73ec1986d0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.563+0000 7f73f14d5640 1 -- 172.21.15.12:0/1365039335 >> 172.21.15.12:0/1365039335 conn(0x7f73ec077eb0 msgr2=0x7f73ec06a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:15.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.563+0000 7f73f14d5640 1 -- 172.21.15.12:0/1365039335 shutdown_connections 2024-04-22T13:30:15.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.563+0000 7f73f14d5640 1 -- 172.21.15.12:0/1365039335 wait complete. 2024-04-22T13:30:15.570 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:30:15.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.678+0000 7fd0cd80f640 1 -- 172.21.15.12:0/4148656752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0c8107260 msgr2=0x7fd0c810ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.678+0000 7fd0cd80f640 1 --2- 172.21.15.12:0/4148656752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0c8107260 0x7fd0c810ee30 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0009ca0 tx=0x7fd0c002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.679+0000 7fd0cd80f640 1 -- 172.21.15.12:0/4148656752 shutdown_connections 2024-04-22T13:30:15.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.679+0000 7fd0cd80f640 1 --2- 172.21.15.12:0/4148656752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0c8107260 0x7fd0c810ee30 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.679+0000 7fd0cd80f640 1 --2- 172.21.15.12:0/4148656752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0c8102b40 0x7fd0c8106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.679+0000 7fd0cd80f640 1 --2- 172.21.15.12:0/4148656752 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0c8102190 0x7fd0c8102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.679+0000 7fd0cd80f640 1 -- 172.21.15.12:0/4148656752 >> 172.21.15.12:0/4148656752 conn(0x7fd0c80fd990 msgr2=0x7fd0c80ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:15.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.679+0000 7fd0cd80f640 1 -- 172.21.15.12:0/4148656752 shutdown_connections 2024-04-22T13:30:15.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.679+0000 7fd0cd80f640 1 -- 172.21.15.12:0/4148656752 wait complete. 2024-04-22T13:30:15.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.679+0000 7fd0cd80f640 1 Processor -- start 2024-04-22T13:30:15.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.679+0000 7fd0cd80f640 1 -- start start 2024-04-22T13:30:15.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.679+0000 7fd0cd80f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0c8102190 0x7fd0c81a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.679+0000 7fd0cd80f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0c8107260 0x7fd0c81a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.679+0000 7fd0cd80f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0c819a750 0x7fd0c819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.679+0000 7fd0cd80f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c81121d0 con 0x7fd0c8102190 2024-04-22T13:30:15.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.679+0000 7fd0cd80f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c8112050 con 0x7fd0c819a750 2024-04-22T13:30:15.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.679+0000 7fd0cd80f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c8112350 con 0x7fd0c8107260 2024-04-22T13:30:15.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.679+0000 7fd0c6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0c8102190 0x7fd0c81a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.680+0000 7fd0c6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0c8102190 0x7fd0c81a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51622/0 (socket says 172.21.15.12:51622) 2024-04-22T13:30:15.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.680+0000 7fd0c6ffd640 1 -- 172.21.15.12:0/1014138307 learned_addr learned my addr 172.21.15.12:0/1014138307 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:15.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.680+0000 7fd0bffff640 1 --2- 172.21.15.12:0/1014138307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0c8107260 0x7fd0c81a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.680+0000 7fd0c77fe640 1 --2- 172.21.15.12:0/1014138307 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0c819a750 0x7fd0c819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.680+0000 7fd0c6ffd640 1 -- 172.21.15.12:0/1014138307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0c8107260 msgr2=0x7fd0c81a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.680+0000 7fd0c6ffd640 1 --2- 172.21.15.12:0/1014138307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0c8107260 0x7fd0c81a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.680+0000 7fd0c6ffd640 1 -- 172.21.15.12:0/1014138307 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0c819a750 msgr2=0x7fd0c819abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.680+0000 7fd0c6ffd640 1 --2- 172.21.15.12:0/1014138307 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0c819a750 0x7fd0c819abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.680+0000 7fd0c6ffd640 1 -- 172.21.15.12:0/1014138307 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c819b350 con 0x7fd0c8102190 2024-04-22T13:30:15.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.680+0000 7fd0bffff640 1 --2- 172.21.15.12:0/1014138307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0c8107260 0x7fd0c81a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:15.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.680+0000 7fd0c6ffd640 1 --2- 172.21.15.12:0/1014138307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0c8102190 0x7fd0c81a0690 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800d950 tx=0x7fd0b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:15.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.680+0000 7fd0cc80d640 1 -- 172.21.15.12:0/1014138307 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b8017070 con 0x7fd0c8102190 2024-04-22T13:30:15.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.680+0000 7fd0cc80d640 1 -- 172.21.15.12:0/1014138307 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0b8009c30 con 0x7fd0c8102190 2024-04-22T13:30:15.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.680+0000 7fd0cd80f640 1 -- 172.21.15.12:0/1014138307 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0c81a7430 con 0x7fd0c8102190 2024-04-22T13:30:15.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.680+0000 7fd0cc80d640 1 -- 172.21.15.12:0/1014138307 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b800f620 con 0x7fd0c8102190 2024-04-22T13:30:15.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.680+0000 7fd0cd80f640 1 -- 172.21.15.12:0/1014138307 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0c81a7830 con 0x7fd0c8102190 2024-04-22T13:30:15.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.682+0000 7fd0cc80d640 1 -- 172.21.15.12:0/1014138307 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd0b8010460 con 0x7fd0c8102190 2024-04-22T13:30:15.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.682+0000 7fd0d03fd740 1 Processor -- start 2024-04-22T13:30:15.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.683+0000 7fd0d03fd740 1 -- start start 2024-04-22T13:30:15.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.683+0000 7fd0d03fd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6150b64d0 0x55b6150b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.683+0000 7fd0d03fd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6150b6e80 0x55b6150bf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.683+0000 7fd0d03fd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6150bfa00 0x55b6150c1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.683+0000 7fd0d03fd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b61501b1f0 con 0x55b6150b6e80 2024-04-22T13:30:15.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.683+0000 7fd0d03fd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b614fb84b0 con 0x55b6150bfa00 2024-04-22T13:30:15.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.683+0000 7fd0d03fd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b6150171b0 con 0x55b6150b64d0 2024-04-22T13:30:15.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.683+0000 7fd0c6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6150b6e80 0x55b6150bf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.683+0000 7fd0cc80d640 1 --2- 172.21.15.12:0/1014138307 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd0b40a3d20 0x7fd0b40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.683+0000 7fd0c6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6150b6e80 0x55b6150bf430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51630/0 (socket says 172.21.15.12:51630) 2024-04-22T13:30:15.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.683+0000 7fd0c6ffd640 1 -- 172.21.15.12:0/344354761 learned_addr learned my addr 172.21.15.12:0/344354761 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:15.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.683+0000 7fd0c77fe640 1 --2- 172.21.15.12:0/344354761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6150b64d0 0x55b6150b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.683+0000 7fd0bffff640 1 --2- 172.21.15.12:0/344354761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6150bfa00 0x55b6150c1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.683+0000 7fd0c77fe640 1 --2- 172.21.15.12:0/1014138307 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd0b40a3d20 0x7fd0b40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.683+0000 7fd0cc80d640 1 -- 172.21.15.12:0/1014138307 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fd0b80d04c0 con 0x7fd0c8102190 2024-04-22T13:30:15.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.683+0000 7fd0c6ffd640 1 -- 172.21.15.12:0/344354761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6150b64d0 msgr2=0x55b6150b68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.683+0000 7fd0c6ffd640 1 --2- 172.21.15.12:0/344354761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6150b64d0 0x55b6150b68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.683+0000 7fd0c6ffd640 1 -- 172.21.15.12:0/344354761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6150bfa00 msgr2=0x55b6150c1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.683+0000 7fd0c6ffd640 1 --2- 172.21.15.12:0/344354761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6150bfa00 0x55b6150c1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.683+0000 7fd0c6ffd640 1 -- 172.21.15.12:0/344354761 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b614faa1b0 con 0x55b6150b6e80 2024-04-22T13:30:15.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.683+0000 7fd0c77fe640 1 --2- 172.21.15.12:0/344354761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6150b64d0 0x55b6150b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:15.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.683+0000 7fd0bffff640 1 --2- 172.21.15.12:0/344354761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6150bfa00 0x55b6150c1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:15.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.684+0000 7fd0c6ffd640 1 --2- 172.21.15.12:0/344354761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6150b6e80 0x55b6150bf430 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fd0b809c440 tx=0x7fd0b8010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d086a1f92bd0a35c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:15.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.684+0000 7fd0c77fe640 1 --2- 172.21.15.12:0/1014138307 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd0b40a3d20 0x7fd0b40a61e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0009430 tx=0x7fd0c00095c0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:15.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.684+0000 7fd0bdffb640 1 -- 172.21.15.12:0/344354761 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b8020070 con 0x55b6150b6e80 2024-04-22T13:30:15.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.684+0000 7fd0bdffb640 1 -- 172.21.15.12:0/344354761 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0b80a5120 con 0x55b6150b6e80 2024-04-22T13:30:15.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.684+0000 7fd0bdffb640 1 -- 172.21.15.12:0/344354761 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b80a4c90 con 0x55b6150b6e80 2024-04-22T13:30:15.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.684+0000 7fd0d03fd740 1 -- 172.21.15.12:0/344354761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6150b6e80 msgr2=0x55b6150bf430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.684+0000 7fd0d03fd740 1 --2- 172.21.15.12:0/344354761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6150b6e80 0x55b6150bf430 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fd0b809c440 tx=0x7fd0b8010cb0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.684+0000 7fd0d03fd740 1 -- 172.21.15.12:0/344354761 shutdown_connections 2024-04-22T13:30:15.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.684+0000 7fd0d03fd740 1 --2- 172.21.15.12:0/344354761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6150bfa00 0x55b6150c1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.684+0000 7fd0d03fd740 1 --2- 172.21.15.12:0/344354761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6150b6e80 0x55b6150bf430 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.684+0000 7fd0d03fd740 1 --2- 172.21.15.12:0/344354761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6150b64d0 0x55b6150b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.684+0000 7fd0d03fd740 1 -- 172.21.15.12:0/344354761 >> 172.21.15.12:0/344354761 conn(0x55b6150b0ba0 msgr2=0x55b6150b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:15.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.684+0000 7fd0d03fd740 1 -- 172.21.15.12:0/344354761 shutdown_connections 2024-04-22T13:30:15.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.684+0000 7fd0d03fd740 1 -- 172.21.15.12:0/344354761 wait complete. 2024-04-22T13:30:15.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.685+0000 7fd0d03fd740 1 -- start start 2024-04-22T13:30:15.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.685+0000 7fd0d03fd740 1 Processor -- start 2024-04-22T13:30:15.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.685+0000 7fd0d03fd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6150b64d0 0x55b6150bf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.685+0000 7fd0d03fd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6150b6e80 0x55b6150bc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.685+0000 7fd0d03fd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6150bfa00 0x55b6150bcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.685+0000 7fd0d03fd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b6150c2910 con 0x55b6150bfa00 2024-04-22T13:30:15.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.685+0000 7fd0d03fd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b614fd4500 con 0x55b6150b6e80 2024-04-22T13:30:15.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.685+0000 7fd0d03fd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b6150c2a90 con 0x55b6150b64d0 2024-04-22T13:30:15.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.685+0000 7fd0d03fd740 10 client.0 _ll_register_callbacks cb 0x55b614fa2b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:15.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.685+0000 7fd0c77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6150bfa00 0x55b6150bcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.685+0000 7fd0c77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6150bfa00 0x55b6150bcb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51632/0 (socket says 172.21.15.12:51632) 2024-04-22T13:30:15.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.685+0000 7fd0c77fe640 1 -- 172.21.15.12:0/2961795147 learned_addr learned my addr 172.21.15.12:0/2961795147 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:15.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.686+0000 7fd0c6ffd640 1 --2- 172.21.15.12:0/2961795147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6150b64d0 0x55b6150bf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.686+0000 7fd0bffff640 1 --2- 172.21.15.12:0/2961795147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6150b6e80 0x55b6150bc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.686+0000 7fd0c77fe640 1 -- 172.21.15.12:0/2961795147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6150b64d0 msgr2=0x55b6150bf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.686+0000 7fd0c77fe640 1 --2- 172.21.15.12:0/2961795147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6150b64d0 0x55b6150bf2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.686+0000 7fd0c77fe640 1 -- 172.21.15.12:0/2961795147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6150b6e80 msgr2=0x55b6150bc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.686+0000 7fd0c77fe640 1 --2- 172.21.15.12:0/2961795147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6150b6e80 0x55b6150bc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.686+0000 7fd0c77fe640 1 -- 172.21.15.12:0/2961795147 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b614d2d3f0 con 0x55b6150bfa00 2024-04-22T13:30:15.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.686+0000 7fd0bffff640 1 --2- 172.21.15.12:0/2961795147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6150b6e80 0x55b6150bc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:15.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.686+0000 7fd0c77fe640 1 --2- 172.21.15.12:0/2961795147 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6150bfa00 0x55b6150bcb70 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0009ca0 tx=0x7fd0c002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739e09744a871804 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:15.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.686+0000 7fd0a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:15.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.686+0000 7fd0a6ffd640 1 -- 172.21.15.12:0/2961795147 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c0052060 con 0x55b6150bfa00 2024-04-22T13:30:15.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.686+0000 7fd0a6ffd640 1 -- 172.21.15.12:0/2961795147 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fd0c0003d00 con 0x55b6150bfa00 2024-04-22T13:30:15.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.686+0000 7fd0d03fd740 1 -- 172.21.15.12:0/2961795147 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0b80a5120 con 0x55b6150bfa00 2024-04-22T13:30:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.686+0000 7fd0a6ffd640 1 -- 172.21.15.12:0/2961795147 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd0c0039040 con 0x55b6150bfa00 2024-04-22T13:30:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.686+0000 7fd0d03fd740 10 client.17961 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.686+0000 7fd0d03fd740 1 -- 172.21.15.12:0/2961795147 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b614df77d0 con 0x55b6150bfa00 2024-04-22T13:30:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.686+0000 7fd0a6ffd640 1 -- 172.21.15.12:0/2961795147 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0b80a5120 con 0x55b6150bfa00 2024-04-22T13:30:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.687+0000 7fd0a6ffd640 1 -- 172.21.15.12:0/2961795147 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd0c0057660 con 0x55b6150bfa00 2024-04-22T13:30:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.687+0000 7fd0d03fd740 10 client.17961 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.687+0000 7fd0d03fd740 10 client.17961 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:30:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.687+0000 7fd0d03fd740 20 client.17961 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.687+0000 7fd0d03fd740 1 --2- 172.21.15.12:0/2961795147 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55b6150cfb50 0x55b6150d1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.687+0000 7fd0d03fd740 4 client.17961 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:30:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.687+0000 7fd0d03fd740 1 -- 172.21.15.12:0/2961795147 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b614fba930 con 0x55b6150cfb50 2024-04-22T13:30:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.687+0000 7fd0c6ffd640 1 --2- 172.21.15.12:0/2961795147 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55b6150cfb50 0x55b6150d1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.687+0000 7fd0a6ffd640 10 client.17961 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:30:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.687+0000 7fd0c6ffd640 1 --2- 172.21.15.12:0/2961795147 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55b6150cfb50 0x55b6150d1f90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4738efdd3934bc89 server_cookie=195bcce9ce65554e in_seq=0 out_seq=0 2024-04-22T13:30:15.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.688+0000 7fd0a6ffd640 1 -- 172.21.15.12:0/2961795147 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b614fba930 con 0x55b6150cfb50 2024-04-22T13:30:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.688+0000 7fd0a6ffd640 10 client.17961 handle_command_reply: tid=0 2024-04-22T13:30:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.688+0000 7fd0d03fd740 1 client.17961 shutdown 2024-04-22T13:30:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.688+0000 7fd0d03fd740 1 -- 172.21.15.12:0/2961795147 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6150bfa00 msgr2=0x55b6150bcb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.688+0000 7fd0d03fd740 1 --2- 172.21.15.12:0/2961795147 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6150bfa00 0x55b6150bcb70 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0009ca0 tx=0x7fd0c002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.688+0000 7fd0d03fd740 1 -- 172.21.15.12:0/2961795147 shutdown_connections 2024-04-22T13:30:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.688+0000 7fd0d03fd740 1 --2- 172.21.15.12:0/2961795147 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55b6150cfb50 0x55b6150d1f90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.688+0000 7fd0d03fd740 1 --2- 172.21.15.12:0/2961795147 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6150bfa00 0x55b6150bcb70 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.688+0000 7fd0d03fd740 1 --2- 172.21.15.12:0/2961795147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6150b6e80 0x55b6150bc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.688+0000 7fd0a6ffd640 0 client.17961 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:30:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.688+0000 7fd0d03fd740 1 --2- 172.21.15.12:0/2961795147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6150b64d0 0x55b6150bf2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.688+0000 7fd0d03fd740 1 -- 172.21.15.12:0/2961795147 >> 172.21.15.12:0/2961795147 conn(0x55b6150b0ba0 msgr2=0x55b6150b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.689+0000 7fd0d03fd740 1 -- 172.21.15.12:0/2961795147 shutdown_connections 2024-04-22T13:30:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.689+0000 7fd0d03fd740 1 -- 172.21.15.12:0/2961795147 wait complete. 2024-04-22T13:30:15.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.689+0000 7fd0d03fd740 20 client.17961 trim_cache size 0 max 16384 2024-04-22T13:30:15.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.702+0000 7fd0d03fd740 1 Processor -- start 2024-04-22T13:30:15.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.702+0000 7fd0d03fd740 1 -- start start 2024-04-22T13:30:15.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.702+0000 7fd0d03fd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6150b64d0 0x55b6150cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.702+0000 7fd0d03fd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6150b6e80 0x55b6150afa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.702+0000 7fd0d03fd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6150bfa00 0x55b6150bed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.702+0000 7fd0d03fd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c81afd90 con 0x55b6150b64d0 2024-04-22T13:30:15.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.702+0000 7fd0d03fd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c81b18d0 con 0x55b6150bfa00 2024-04-22T13:30:15.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.702+0000 7fd0d03fd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c81b0460 con 0x55b6150b6e80 2024-04-22T13:30:15.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.702+0000 7fd0c6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6150b64d0 0x55b6150cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.702+0000 7fd0c6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6150b64d0 0x55b6150cb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51634/0 (socket says 172.21.15.12:51634) 2024-04-22T13:30:15.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.702+0000 7fd0c6ffd640 1 -- 172.21.15.12:0/124134940 learned_addr learned my addr 172.21.15.12:0/124134940 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:15.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.702+0000 7fd0bffff640 1 --2- 172.21.15.12:0/124134940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6150b6e80 0x55b6150afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.702+0000 7fd0c77fe640 1 --2- 172.21.15.12:0/124134940 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6150bfa00 0x55b6150bed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.702+0000 7fd0c6ffd640 1 -- 172.21.15.12:0/124134940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6150b6e80 msgr2=0x55b6150afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.702+0000 7fd0c6ffd640 1 --2- 172.21.15.12:0/124134940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6150b6e80 0x55b6150afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.702+0000 7fd0c6ffd640 1 -- 172.21.15.12:0/124134940 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6150bfa00 msgr2=0x55b6150bed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.702+0000 7fd0c6ffd640 1 --2- 172.21.15.12:0/124134940 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6150bfa00 0x55b6150bed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.703+0000 7fd0c6ffd640 1 -- 172.21.15.12:0/124134940 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6150b6c20 con 0x55b6150b64d0 2024-04-22T13:30:15.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.703+0000 7fd0c77fe640 1 --2- 172.21.15.12:0/124134940 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6150bfa00 0x55b6150bed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:30:15.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.703+0000 7fd0c6ffd640 1 --2- 172.21.15.12:0/124134940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6150b64d0 0x55b6150cb7d0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fd0b809c440 tx=0x7fd0b800d8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3919110e5536de4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:15.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.703+0000 7fd0bd7fa640 1 -- 172.21.15.12:0/124134940 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b8020040 con 0x55b6150b64d0 2024-04-22T13:30:15.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.703+0000 7fd0bd7fa640 1 -- 172.21.15.12:0/124134940 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0b80a0860 con 0x55b6150b64d0 2024-04-22T13:30:15.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.703+0000 7fd0bd7fa640 1 -- 172.21.15.12:0/124134940 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b80a4ef0 con 0x55b6150b64d0 2024-04-22T13:30:15.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.703+0000 7fd0d03fd740 1 -- 172.21.15.12:0/124134940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6150b64d0 msgr2=0x55b6150cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.703+0000 7fd0d03fd740 1 --2- 172.21.15.12:0/124134940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6150b64d0 0x55b6150cb7d0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fd0b809c440 tx=0x7fd0b800d8d0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.703+0000 7fd0d03fd740 1 -- 172.21.15.12:0/124134940 shutdown_connections 2024-04-22T13:30:15.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.703+0000 7fd0d03fd740 1 --2- 172.21.15.12:0/124134940 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6150bfa00 0x55b6150bed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.703+0000 7fd0d03fd740 1 --2- 172.21.15.12:0/124134940 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6150b6e80 0x55b6150afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.703+0000 7fd0d03fd740 1 --2- 172.21.15.12:0/124134940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6150b64d0 0x55b6150cb7d0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.703+0000 7fd0d03fd740 1 -- 172.21.15.12:0/124134940 >> 172.21.15.12:0/124134940 conn(0x55b6150b0ba0 msgr2=0x55b6150af370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:15.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.703+0000 7fd0d03fd740 1 -- 172.21.15.12:0/124134940 shutdown_connections 2024-04-22T13:30:15.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.703+0000 7fd0d03fd740 1 -- 172.21.15.12:0/124134940 wait complete. 2024-04-22T13:30:15.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.704+0000 7fd0d03fd740 1 -- start start 2024-04-22T13:30:15.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.704+0000 7fd0d03fd740 1 Processor -- start 2024-04-22T13:30:15.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.704+0000 7fd0d03fd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6150b64d0 0x55b6150c8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.704+0000 7fd0d03fd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6150b6e80 0x55b6150c33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.704+0000 7fd0d03fd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6150bfa00 0x55b6150c3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.704+0000 7fd0d03fd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c81b1200 con 0x55b6150b6e80 2024-04-22T13:30:15.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.704+0000 7fd0d03fd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c81b0b30 con 0x55b6150b64d0 2024-04-22T13:30:15.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.704+0000 7fd0d03fd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c81add70 con 0x55b6150bfa00 2024-04-22T13:30:15.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.704+0000 7fd0d03fd740 10 client.0 _ll_register_callbacks cb 0x7fd0c0009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:15.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.704+0000 7fd0bffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6150b6e80 0x55b6150c33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.704+0000 7fd0bffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6150b6e80 0x55b6150c33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51644/0 (socket says 172.21.15.12:51644) 2024-04-22T13:30:15.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.704+0000 7fd0bffff640 1 -- 172.21.15.12:0/1719552841 learned_addr learned my addr 172.21.15.12:0/1719552841 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:15.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.704+0000 7fd0c6ffd640 1 --2- 172.21.15.12:0/1719552841 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6150b64d0 0x55b6150c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.704+0000 7fd0c77fe640 1 --2- 172.21.15.12:0/1719552841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6150bfa00 0x55b6150c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.705+0000 7fd0bffff640 1 -- 172.21.15.12:0/1719552841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6150bfa00 msgr2=0x55b6150c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.705+0000 7fd0bffff640 1 --2- 172.21.15.12:0/1719552841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6150bfa00 0x55b6150c3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.705+0000 7fd0bffff640 1 -- 172.21.15.12:0/1719552841 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6150b64d0 msgr2=0x55b6150c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.705+0000 7fd0bffff640 1 --2- 172.21.15.12:0/1719552841 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6150b64d0 0x55b6150c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.705+0000 7fd0bffff640 1 -- 172.21.15.12:0/1719552841 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6150c4130 con 0x55b6150b6e80 2024-04-22T13:30:15.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.705+0000 7fd0bffff640 1 --2- 172.21.15.12:0/1719552841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6150b6e80 0x55b6150c33c0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fd0c819bfa0 tx=0x7fd0b0013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ea212052a4ac17 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:15.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.705+0000 7fd0a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:15.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.705+0000 7fd0a6ffd640 1 -- 172.21.15.12:0/1719552841 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0005020 con 0x55b6150b6e80 2024-04-22T13:30:15.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.705+0000 7fd0a6ffd640 1 -- 172.21.15.12:0/1719552841 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fd0b00051c0 con 0x55b6150b6e80 2024-04-22T13:30:15.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.705+0000 7fd0d03fd740 1 -- 172.21.15.12:0/1719552841 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0b80a0860 con 0x55b6150b6e80 2024-04-22T13:30:15.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.705+0000 7fd0a6ffd640 1 -- 172.21.15.12:0/1719552841 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd0b00053c0 con 0x55b6150b6e80 2024-04-22T13:30:15.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.705+0000 7fd0d03fd740 10 client.17967 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:15.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.705+0000 7fd0d03fd740 1 -- 172.21.15.12:0/1719552841 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b615136010 con 0x55b6150b6e80 2024-04-22T13:30:15.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.705+0000 7fd0a6ffd640 1 -- 172.21.15.12:0/1719552841 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0b80a0860 con 0x55b6150b6e80 2024-04-22T13:30:15.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.705+0000 7fd0a6ffd640 1 -- 172.21.15.12:0/1719552841 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd0b00118f0 con 0x55b6150b6e80 2024-04-22T13:30:15.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.705+0000 7fd0d03fd740 10 client.17967 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:15.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.705+0000 7fd0d03fd740 10 client.17967 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:30:15.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.706+0000 7fd0d03fd740 20 client.17967 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:15.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.706+0000 7fd0d03fd740 1 --2- 172.21.15.12:0/1719552841 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55b6151367f0 0x55b615138c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.706+0000 7fd0d03fd740 4 client.17967 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:30:15.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.706+0000 7fd0d03fd740 1 -- 172.21.15.12:0/1719552841 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b6150bcf00 con 0x55b6151367f0 2024-04-22T13:30:15.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.706+0000 7fd0c6ffd640 1 --2- 172.21.15.12:0/1719552841 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55b6151367f0 0x55b615138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.706+0000 7fd0c6ffd640 1 --2- 172.21.15.12:0/1719552841 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55b6151367f0 0x55b615138c10 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c601e1ba3310c95c server_cookie=ab48fdee5d0d2f54 in_seq=0 out_seq=0 2024-04-22T13:30:15.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.706+0000 7fd0a6ffd640 10 client.17967 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:30:15.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.706+0000 7fd0a6ffd640 1 -- 172.21.15.12:0/1719552841 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b6150bcf00 con 0x55b6151367f0 2024-04-22T13:30:15.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.706+0000 7fd0a6ffd640 10 client.17967 handle_command_reply: tid=0 2024-04-22T13:30:15.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.706+0000 7fd0d03fd740 1 client.17967 shutdown 2024-04-22T13:30:15.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.707+0000 7fd0d03fd740 1 -- 172.21.15.12:0/1719552841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6150b6e80 msgr2=0x55b6150c33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.707+0000 7fd0d03fd740 1 --2- 172.21.15.12:0/1719552841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6150b6e80 0x55b6150c33c0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fd0c819bfa0 tx=0x7fd0b0013830 comp rx=0 tx=0).stop 2024-04-22T13:30:15.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.707+0000 7fd0d03fd740 1 -- 172.21.15.12:0/1719552841 shutdown_connections 2024-04-22T13:30:15.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.707+0000 7fd0d03fd740 1 --2- 172.21.15.12:0/1719552841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6150bfa00 0x55b6150c3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.707+0000 7fd0d03fd740 1 --2- 172.21.15.12:0/1719552841 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55b6151367f0 0x55b615138c10 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.707+0000 7fd0d03fd740 1 --2- 172.21.15.12:0/1719552841 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b6150b6e80 0x55b6150c33c0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.707+0000 7fd0d03fd740 1 --2- 172.21.15.12:0/1719552841 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b6150b64d0 0x55b6150c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.707+0000 7fd0a6ffd640 0 client.17967 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:30:15.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.707+0000 7fd0d03fd740 1 -- 172.21.15.12:0/1719552841 >> 172.21.15.12:0/1719552841 conn(0x55b6150b0ba0 msgr2=0x55b6150afc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:15.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.707+0000 7fd0d03fd740 1 -- 172.21.15.12:0/1719552841 shutdown_connections 2024-04-22T13:30:15.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.707+0000 7fd0d03fd740 1 -- 172.21.15.12:0/1719552841 wait complete. 2024-04-22T13:30:15.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.707+0000 7fd0d03fd740 20 client.17967 trim_cache size 0 max 16384 2024-04-22T13:30:15.711 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:30:15.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.707+0000 7fd0cd80f640 1 -- 172.21.15.12:0/1014138307 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd0b40a3d20 msgr2=0x7fd0b40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.707+0000 7fd0cd80f640 1 --2- 172.21.15.12:0/1014138307 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd0b40a3d20 0x7fd0b40a61e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0009430 tx=0x7fd0c00095c0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.707+0000 7fd0cd80f640 1 -- 172.21.15.12:0/1014138307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0c8102190 msgr2=0x7fd0c81a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.707+0000 7fd0cd80f640 1 --2- 172.21.15.12:0/1014138307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0c8102190 0x7fd0c81a0690 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800d950 tx=0x7fd0b800de10 comp rx=0 tx=0).stop 2024-04-22T13:30:15.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.707+0000 7fd0cd80f640 1 -- 172.21.15.12:0/1014138307 shutdown_connections 2024-04-22T13:30:15.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.707+0000 7fd0cd80f640 1 --2- 172.21.15.12:0/1014138307 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd0b40a3d20 0x7fd0b40a61e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.707+0000 7fd0cd80f640 1 --2- 172.21.15.12:0/1014138307 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0c819a750 0x7fd0c819abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.707+0000 7fd0cd80f640 1 --2- 172.21.15.12:0/1014138307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0c8107260 0x7fd0c81a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.707+0000 7fd0cd80f640 1 --2- 172.21.15.12:0/1014138307 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0c8102190 0x7fd0c81a0690 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.707+0000 7fd0cd80f640 1 -- 172.21.15.12:0/1014138307 >> 172.21.15.12:0/1014138307 conn(0x7fd0c80fd990 msgr2=0x7fd0c80ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:15.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.707+0000 7fd0cd80f640 1 -- 172.21.15.12:0/1014138307 shutdown_connections 2024-04-22T13:30:15.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.707+0000 7fd0cd80f640 1 -- 172.21.15.12:0/1014138307 wait complete. 2024-04-22T13:30:15.714 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:30:15.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.822+0000 7fe36fb59640 1 -- 172.21.15.12:0/104010943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe36810e960 msgr2=0x7fe368110d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.822+0000 7fe36fb59640 1 --2- 172.21.15.12:0/104010943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe36810e960 0x7fe368110d50 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fe360009ca0 tx=0x7fe36002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.822+0000 7fe36fb59640 1 -- 172.21.15.12:0/104010943 shutdown_connections 2024-04-22T13:30:15.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.822+0000 7fe36fb59640 1 --2- 172.21.15.12:0/104010943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe36810e960 0x7fe368110d50 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.822+0000 7fe36fb59640 1 --2- 172.21.15.12:0/104010943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe36806aef0 0x7fe36810e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.822+0000 7fe36fb59640 1 --2- 172.21.15.12:0/104010943 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe36806a5d0 0x7fe36806a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.822+0000 7fe36fb59640 1 -- 172.21.15.12:0/104010943 >> 172.21.15.12:0/104010943 conn(0x7fe368077eb0 msgr2=0x7fe3680782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:15.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.822+0000 7fe36fb59640 1 -- 172.21.15.12:0/104010943 shutdown_connections 2024-04-22T13:30:15.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.822+0000 7fe36fb59640 1 -- 172.21.15.12:0/104010943 wait complete. 2024-04-22T13:30:15.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.823+0000 7fe36fb59640 1 Processor -- start 2024-04-22T13:30:15.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.823+0000 7fe36fb59640 1 -- start start 2024-04-22T13:30:15.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.823+0000 7fe36fb59640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe36806a5d0 0x7fe3681a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.823+0000 7fe36fb59640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe36806aef0 0x7fe3681a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.823+0000 7fe36fb59640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe36810e960 0x7fe36819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.823+0000 7fe36fb59640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe3681143b0 con 0x7fe36806a5d0 2024-04-22T13:30:15.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.823+0000 7fe36fb59640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe368114230 con 0x7fe36810e960 2024-04-22T13:30:15.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.823+0000 7fe36fb59640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe368114530 con 0x7fe36806aef0 2024-04-22T13:30:15.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.823+0000 7fe36d8f5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe36806a5d0 0x7fe3681a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.823+0000 7fe36d8f5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe36806a5d0 0x7fe3681a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51654/0 (socket says 172.21.15.12:51654) 2024-04-22T13:30:15.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.823+0000 7fe36d8f5640 1 -- 172.21.15.12:0/3261704961 learned_addr learned my addr 172.21.15.12:0/3261704961 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:15.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.823+0000 7fe36d0f4640 1 --2- 172.21.15.12:0/3261704961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe36806aef0 0x7fe3681a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.823+0000 7fe36e0f6640 1 --2- 172.21.15.12:0/3261704961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe36810e960 0x7fe36819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.824+0000 7fe36d8f5640 1 -- 172.21.15.12:0/3261704961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe36806aef0 msgr2=0x7fe3681a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.824+0000 7fe36d8f5640 1 --2- 172.21.15.12:0/3261704961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe36806aef0 0x7fe3681a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.824+0000 7fe36d8f5640 1 -- 172.21.15.12:0/3261704961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe36810e960 msgr2=0x7fe36819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.824+0000 7fe36d8f5640 1 --2- 172.21.15.12:0/3261704961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe36810e960 0x7fe36819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.824+0000 7fe36d8f5640 1 -- 172.21.15.12:0/3261704961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe36819afb0 con 0x7fe36806a5d0 2024-04-22T13:30:15.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.824+0000 7fe36e0f6640 1 --2- 172.21.15.12:0/3261704961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe36810e960 0x7fe36819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:15.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.824+0000 7fe36d8f5640 1 --2- 172.21.15.12:0/3261704961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe36806a5d0 0x7fe3681a0630 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fe36400d770 tx=0x7fe36400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:15.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.824+0000 7fe35effd640 1 -- 172.21.15.12:0/3261704961 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364017070 con 0x7fe36806a5d0 2024-04-22T13:30:15.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.824+0000 7fe35effd640 1 -- 172.21.15.12:0/3261704961 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3640054d0 con 0x7fe36806a5d0 2024-04-22T13:30:15.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.824+0000 7fe36fb59640 1 -- 172.21.15.12:0/3261704961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe36819b2a0 con 0x7fe36806a5d0 2024-04-22T13:30:15.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.824+0000 7fe35effd640 1 -- 172.21.15.12:0/3261704961 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364004c30 con 0x7fe36806a5d0 2024-04-22T13:30:15.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.824+0000 7fe36fb59640 1 -- 172.21.15.12:0/3261704961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3681a75b0 con 0x7fe36806a5d0 2024-04-22T13:30:15.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.826+0000 7fe35effd640 1 -- 172.21.15.12:0/3261704961 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe364004e10 con 0x7fe36806a5d0 2024-04-22T13:30:15.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.827+0000 7fe37282c740 1 Processor -- start 2024-04-22T13:30:15.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.827+0000 7fe37282c740 1 -- start start 2024-04-22T13:30:15.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.827+0000 7fe37282c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d9fb9f4d0 0x564d9fb9f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.827+0000 7fe37282c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d9fb9fe80 0x564d9fba8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.827+0000 7fe37282c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564d9fba8a00 0x564d9fbaae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.827+0000 7fe37282c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564d9fb041f0 con 0x564d9fb9fe80 2024-04-22T13:30:15.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.827+0000 7fe37282c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564d9faa14b0 con 0x564d9fba8a00 2024-04-22T13:30:15.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.827+0000 7fe37282c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564d9fb001b0 con 0x564d9fb9f4d0 2024-04-22T13:30:15.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.827+0000 7fe35effd640 1 --2- 172.21.15.12:0/3261704961 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe3540a3da0 0x7fe3540a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.827+0000 7fe36d8f5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d9fb9fe80 0x564d9fba8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.827+0000 7fe36d8f5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d9fb9fe80 0x564d9fba8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51666/0 (socket says 172.21.15.12:51666) 2024-04-22T13:30:15.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.827+0000 7fe36d8f5640 1 -- 172.21.15.12:0/708621435 learned_addr learned my addr 172.21.15.12:0/708621435 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:15.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.827+0000 7fe36d0f4640 1 --2- 172.21.15.12:0/708621435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564d9fba8a00 0x564d9fbaae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.827+0000 7fe36e0f6640 1 --2- 172.21.15.12:0/3261704961 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe3540a3da0 0x7fe3540a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.827+0000 7fe36e0f6640 1 --2- 172.21.15.12:0/708621435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d9fb9f4d0 0x564d9fb9f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.828+0000 7fe35effd640 1 -- 172.21.15.12:0/3261704961 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fe3640d0260 con 0x7fe36806a5d0 2024-04-22T13:30:15.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.828+0000 7fe36d8f5640 1 -- 172.21.15.12:0/708621435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d9fb9f4d0 msgr2=0x564d9fb9f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.828+0000 7fe36d8f5640 1 --2- 172.21.15.12:0/708621435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d9fb9f4d0 0x564d9fb9f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.828+0000 7fe36d8f5640 1 -- 172.21.15.12:0/708621435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564d9fba8a00 msgr2=0x564d9fbaae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.828+0000 7fe36d8f5640 1 --2- 172.21.15.12:0/708621435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564d9fba8a00 0x564d9fbaae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.828+0000 7fe36d8f5640 1 -- 172.21.15.12:0/708621435 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d9fa931b0 con 0x564d9fb9fe80 2024-04-22T13:30:15.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.828+0000 7fe36e0f6640 1 --2- 172.21.15.12:0/708621435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d9fb9f4d0 0x564d9fb9f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:15.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.828+0000 7fe36d0f4640 1 --2- 172.21.15.12:0/708621435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564d9fba8a00 0x564d9fbaae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:30:15.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.828+0000 7fe36d8f5640 1 --2- 172.21.15.12:0/708621435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d9fb9fe80 0x564d9fba8430 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fe3640cd940 tx=0x7fe36401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e657351e35677314 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:15.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.828+0000 7fe36e0f6640 1 --2- 172.21.15.12:0/3261704961 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe3540a3da0 0x7fe3540a6260 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe360006010 tx=0x7fe36002e040 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:15.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.828+0000 7fe353fff640 1 -- 172.21.15.12:0/708621435 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364020070 con 0x564d9fb9fe80 2024-04-22T13:30:15.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.828+0000 7fe353fff640 1 -- 172.21.15.12:0/708621435 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe36409c8f0 con 0x564d9fb9fe80 2024-04-22T13:30:15.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.828+0000 7fe353fff640 1 -- 172.21.15.12:0/708621435 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36409a5a0 con 0x564d9fb9fe80 2024-04-22T13:30:15.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.829+0000 7fe37282c740 1 -- 172.21.15.12:0/708621435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d9fb9fe80 msgr2=0x564d9fba8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.829+0000 7fe37282c740 1 --2- 172.21.15.12:0/708621435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d9fb9fe80 0x564d9fba8430 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fe3640cd940 tx=0x7fe36401e000 comp rx=0 tx=0).stop 2024-04-22T13:30:15.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.829+0000 7fe37282c740 1 -- 172.21.15.12:0/708621435 shutdown_connections 2024-04-22T13:30:15.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.829+0000 7fe37282c740 1 --2- 172.21.15.12:0/708621435 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564d9fba8a00 0x564d9fbaae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.829+0000 7fe37282c740 1 --2- 172.21.15.12:0/708621435 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d9fb9fe80 0x564d9fba8430 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.829+0000 7fe37282c740 1 --2- 172.21.15.12:0/708621435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d9fb9f4d0 0x564d9fb9f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.829+0000 7fe37282c740 1 -- 172.21.15.12:0/708621435 >> 172.21.15.12:0/708621435 conn(0x564d9fb99ba0 msgr2=0x564d9fb99f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:15.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.829+0000 7fe37282c740 1 -- 172.21.15.12:0/708621435 shutdown_connections 2024-04-22T13:30:15.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.829+0000 7fe37282c740 1 -- 172.21.15.12:0/708621435 wait complete. 2024-04-22T13:30:15.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.829+0000 7fe37282c740 1 -- start start 2024-04-22T13:30:15.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.829+0000 7fe37282c740 1 Processor -- start 2024-04-22T13:30:15.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.830+0000 7fe37282c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d9fb9f4d0 0x564d9fba8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.830+0000 7fe37282c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d9fb9fe80 0x564d9fba56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.830+0000 7fe37282c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564d9fba8a00 0x564d9fba5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.830+0000 7fe37282c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564d9fbab8b0 con 0x564d9fb9fe80 2024-04-22T13:30:15.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.830+0000 7fe37282c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564d9fabd500 con 0x564d9fba8a00 2024-04-22T13:30:15.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.830+0000 7fe37282c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564d9fbaba30 con 0x564d9fb9f4d0 2024-04-22T13:30:15.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.830+0000 7fe37282c740 10 client.0 _ll_register_callbacks cb 0x564d9fa8bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:15.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.830+0000 7fe36d0f4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d9fb9fe80 0x564d9fba56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.830+0000 7fe36d0f4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d9fb9fe80 0x564d9fba56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51672/0 (socket says 172.21.15.12:51672) 2024-04-22T13:30:15.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.830+0000 7fe36d0f4640 1 -- 172.21.15.12:0/4143473472 learned_addr learned my addr 172.21.15.12:0/4143473472 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:15.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.830+0000 7fe36d8f5640 1 --2- 172.21.15.12:0/4143473472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d9fb9f4d0 0x564d9fba8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.850 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.830+0000 7fe36e0f6640 1 --2- 172.21.15.12:0/4143473472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564d9fba8a00 0x564d9fba5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.830+0000 7fe36d0f4640 1 -- 172.21.15.12:0/4143473472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d9fb9f4d0 msgr2=0x564d9fba8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.830+0000 7fe36d0f4640 1 --2- 172.21.15.12:0/4143473472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d9fb9f4d0 0x564d9fba8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.830+0000 7fe36d0f4640 1 -- 172.21.15.12:0/4143473472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564d9fba8a00 msgr2=0x564d9fba5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.830+0000 7fe36d0f4640 1 --2- 172.21.15.12:0/4143473472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564d9fba8a00 0x564d9fba5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.830+0000 7fe36d0f4640 1 -- 172.21.15.12:0/4143473472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d9f8163f0 con 0x564d9fb9fe80 2024-04-22T13:30:15.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.830+0000 7fe36d0f4640 1 --2- 172.21.15.12:0/4143473472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d9fb9fe80 0x564d9fba56f0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fe35800b9f0 tx=0x7fe35800beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72fbacfa8645ee19 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:15.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.830+0000 7fe3517fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:15.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.830+0000 7fe3517fa640 1 -- 172.21.15.12:0/4143473472 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe358003e20 con 0x564d9fb9fe80 2024-04-22T13:30:15.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.830+0000 7fe3517fa640 1 -- 172.21.15.12:0/4143473472 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fe358005020 con 0x564d9fb9fe80 2024-04-22T13:30:15.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.830+0000 7fe37282c740 1 -- 172.21.15.12:0/4143473472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe36409c8f0 con 0x564d9fb9fe80 2024-04-22T13:30:15.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.831+0000 7fe3517fa640 1 -- 172.21.15.12:0/4143473472 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe358005220 con 0x564d9fb9fe80 2024-04-22T13:30:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.831+0000 7fe37282c740 10 client.17979 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.831+0000 7fe37282c740 1 -- 172.21.15.12:0/4143473472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564d9f8e07d0 con 0x564d9fb9fe80 2024-04-22T13:30:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.831+0000 7fe3517fa640 1 -- 172.21.15.12:0/4143473472 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe36409c8f0 con 0x564d9fb9fe80 2024-04-22T13:30:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.831+0000 7fe3517fa640 1 -- 172.21.15.12:0/4143473472 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe35801b500 con 0x564d9fb9fe80 2024-04-22T13:30:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.831+0000 7fe37282c740 10 client.17979 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.831+0000 7fe37282c740 10 client.17979 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:30:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.831+0000 7fe37282c740 20 client.17979 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:15.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.831+0000 7fe37282c740 1 --2- 172.21.15.12:0/4143473472 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x564d9fbb8b40 0x564d9fbbaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.831+0000 7fe37282c740 4 client.17979 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:30:15.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.831+0000 7fe37282c740 1 -- 172.21.15.12:0/4143473472 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d9faa3930 con 0x564d9fbb8b40 2024-04-22T13:30:15.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.831+0000 7fe36d8f5640 1 --2- 172.21.15.12:0/4143473472 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x564d9fbb8b40 0x564d9fbbaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.832+0000 7fe3517fa640 10 client.17979 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:30:15.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.832+0000 7fe36d8f5640 1 --2- 172.21.15.12:0/4143473472 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x564d9fbb8b40 0x564d9fbbaf70 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=920c54a886e5138f server_cookie=cf863a9cacce6c18 in_seq=0 out_seq=0 2024-04-22T13:30:15.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.832+0000 7fe3517fa640 1 -- 172.21.15.12:0/4143473472 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564d9faa3930 con 0x564d9fbb8b40 2024-04-22T13:30:15.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.832+0000 7fe3517fa640 10 client.17979 handle_command_reply: tid=0 2024-04-22T13:30:15.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.833+0000 7fe37282c740 1 client.17979 shutdown 2024-04-22T13:30:15.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.833+0000 7fe37282c740 1 -- 172.21.15.12:0/4143473472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d9fb9fe80 msgr2=0x564d9fba56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.833+0000 7fe37282c740 1 --2- 172.21.15.12:0/4143473472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d9fb9fe80 0x564d9fba56f0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fe35800b9f0 tx=0x7fe35800beb0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.833+0000 7fe37282c740 1 -- 172.21.15.12:0/4143473472 shutdown_connections 2024-04-22T13:30:15.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.833+0000 7fe37282c740 1 --2- 172.21.15.12:0/4143473472 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x564d9fbb8b40 0x564d9fbbaf70 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.833+0000 7fe37282c740 1 --2- 172.21.15.12:0/4143473472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564d9fba8a00 0x564d9fba5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.833+0000 7fe37282c740 1 --2- 172.21.15.12:0/4143473472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d9fb9fe80 0x564d9fba56f0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.833+0000 7fe3517fa640 0 client.17979 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:30:15.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.833+0000 7fe37282c740 1 --2- 172.21.15.12:0/4143473472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d9fb9f4d0 0x564d9fba8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.833+0000 7fe37282c740 1 -- 172.21.15.12:0/4143473472 >> 172.21.15.12:0/4143473472 conn(0x564d9fb99ba0 msgr2=0x564d9fb9b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:15.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.833+0000 7fe37282c740 1 -- 172.21.15.12:0/4143473472 shutdown_connections 2024-04-22T13:30:15.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.833+0000 7fe37282c740 1 -- 172.21.15.12:0/4143473472 wait complete. 2024-04-22T13:30:15.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.833+0000 7fe37282c740 20 client.17979 trim_cache size 0 max 16384 2024-04-22T13:30:15.857 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:15 smithi012 ceph-mon[28265]: pgmap v187: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-22T13:30:15.857 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:15 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2983305822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:30:15.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.847+0000 7fe37282c740 1 Processor -- start 2024-04-22T13:30:15.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.847+0000 7fe37282c740 1 -- start start 2024-04-22T13:30:15.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.847+0000 7fe37282c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564d9fb9f4d0 0x564d9fbb4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.847+0000 7fe37282c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d9fb9fe80 0x564d9fba7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.866 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.847+0000 7fe37282c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d9fba8a00 0x564d9fba8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.847+0000 7fe37282c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe3681afce0 con 0x564d9fb9fe80 2024-04-22T13:30:15.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.847+0000 7fe37282c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe3681b1820 con 0x564d9fb9f4d0 2024-04-22T13:30:15.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.847+0000 7fe37282c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe3681b03b0 con 0x564d9fba8a00 2024-04-22T13:30:15.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.847+0000 7fe36d0f4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d9fb9fe80 0x564d9fba7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.847+0000 7fe36d0f4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d9fb9fe80 0x564d9fba7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51678/0 (socket says 172.21.15.12:51678) 2024-04-22T13:30:15.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.847+0000 7fe36d0f4640 1 -- 172.21.15.12:0/3709665982 learned_addr learned my addr 172.21.15.12:0/3709665982 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:15.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.847+0000 7fe36e0f6640 1 --2- 172.21.15.12:0/3709665982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d9fba8a00 0x564d9fba8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.847+0000 7fe36d8f5640 1 --2- 172.21.15.12:0/3709665982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564d9fb9f4d0 0x564d9fbb4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.847+0000 7fe36d0f4640 1 -- 172.21.15.12:0/3709665982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d9fba8a00 msgr2=0x564d9fba8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.847+0000 7fe36d0f4640 1 --2- 172.21.15.12:0/3709665982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d9fba8a00 0x564d9fba8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.847+0000 7fe36d0f4640 1 -- 172.21.15.12:0/3709665982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564d9fb9f4d0 msgr2=0x564d9fbb4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.847+0000 7fe36d0f4640 1 --2- 172.21.15.12:0/3709665982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564d9fb9f4d0 0x564d9fbb4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.847+0000 7fe36d0f4640 1 -- 172.21.15.12:0/3709665982 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d9fb9e800 con 0x564d9fb9fe80 2024-04-22T13:30:15.867 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.847+0000 7fe36d8f5640 1 --2- 172.21.15.12:0/3709665982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564d9fb9f4d0 0x564d9fbb4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:30:15.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.848+0000 7fe36d0f4640 1 --2- 172.21.15.12:0/3709665982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d9fb9fe80 0x564d9fba7e20 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fe35801bac0 tx=0x7fe358003e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e0cd9f14351722 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:15.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.848+0000 7fe3537fe640 1 -- 172.21.15.12:0/3709665982 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35800b480 con 0x564d9fb9fe80 2024-04-22T13:30:15.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.848+0000 7fe3537fe640 1 -- 172.21.15.12:0/3709665982 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe35801b470 con 0x564d9fb9fe80 2024-04-22T13:30:15.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.848+0000 7fe3537fe640 1 -- 172.21.15.12:0/3709665982 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe358004b20 con 0x564d9fb9fe80 2024-04-22T13:30:15.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.848+0000 7fe37282c740 1 -- 172.21.15.12:0/3709665982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d9fb9fe80 msgr2=0x564d9fba7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.848+0000 7fe37282c740 1 --2- 172.21.15.12:0/3709665982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d9fb9fe80 0x564d9fba7e20 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fe35801bac0 tx=0x7fe358003e20 comp rx=0 tx=0).stop 2024-04-22T13:30:15.868 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.848+0000 7fe37282c740 1 -- 172.21.15.12:0/3709665982 shutdown_connections 2024-04-22T13:30:15.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.848+0000 7fe37282c740 1 --2- 172.21.15.12:0/3709665982 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d9fba8a00 0x564d9fba8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.848+0000 7fe37282c740 1 --2- 172.21.15.12:0/3709665982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d9fb9fe80 0x564d9fba7e20 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.848+0000 7fe37282c740 1 --2- 172.21.15.12:0/3709665982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564d9fb9f4d0 0x564d9fbb4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.848+0000 7fe37282c740 1 -- 172.21.15.12:0/3709665982 >> 172.21.15.12:0/3709665982 conn(0x564d9fb99ba0 msgr2=0x564d9fb98c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:15.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.848+0000 7fe37282c740 1 -- 172.21.15.12:0/3709665982 shutdown_connections 2024-04-22T13:30:15.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.848+0000 7fe37282c740 1 -- 172.21.15.12:0/3709665982 wait complete. 2024-04-22T13:30:15.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.849+0000 7fe37282c740 1 -- start start 2024-04-22T13:30:15.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.849+0000 7fe37282c740 1 Processor -- start 2024-04-22T13:30:15.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.849+0000 7fe37282c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d9fb9f4d0 0x564d9fba23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.849+0000 7fe37282c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d9fb9fe80 0x564d9fba28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.849+0000 7fe37282c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564d9fba8a00 0x564d9fba2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.849+0000 7fe37282c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe3681b1150 con 0x564d9fb9f4d0 2024-04-22T13:30:15.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.849+0000 7fe37282c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe3681b0a80 con 0x564d9fba8a00 2024-04-22T13:30:15.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.849+0000 7fe37282c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe3681adcc0 con 0x564d9fb9fe80 2024-04-22T13:30:15.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.849+0000 7fe37282c740 10 client.0 _ll_register_callbacks cb 0x7fe3580044e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:15.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.849+0000 7fe36d8f5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d9fb9f4d0 0x564d9fba23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.849+0000 7fe36d8f5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d9fb9f4d0 0x564d9fba23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51686/0 (socket says 172.21.15.12:51686) 2024-04-22T13:30:15.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.849+0000 7fe36d8f5640 1 -- 172.21.15.12:0/3216502021 learned_addr learned my addr 172.21.15.12:0/3216502021 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:15.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.849+0000 7fe36d0f4640 1 --2- 172.21.15.12:0/3216502021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d9fb9fe80 0x564d9fba28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.849+0000 7fe36e0f6640 1 --2- 172.21.15.12:0/3216502021 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564d9fba8a00 0x564d9fba2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.850+0000 7fe36d8f5640 1 -- 172.21.15.12:0/3216502021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d9fb9fe80 msgr2=0x564d9fba28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.850+0000 7fe36d8f5640 1 --2- 172.21.15.12:0/3216502021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d9fb9fe80 0x564d9fba28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.850+0000 7fe36d8f5640 1 -- 172.21.15.12:0/3216502021 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564d9fba8a00 msgr2=0x564d9fba2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.850+0000 7fe36d8f5640 1 --2- 172.21.15.12:0/3216502021 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564d9fba8a00 0x564d9fba2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.850+0000 7fe36d8f5640 1 -- 172.21.15.12:0/3216502021 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d9fc1e0b0 con 0x564d9fb9f4d0 2024-04-22T13:30:15.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.850+0000 7fe36e0f6640 1 --2- 172.21.15.12:0/3216502021 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564d9fba8a00 0x564d9fba2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:15.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.850+0000 7fe36d8f5640 1 --2- 172.21.15.12:0/3216502021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d9fb9f4d0 0x564d9fba23a0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fe36409bef0 tx=0x7fe36409bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5bf23147836fcc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:15.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.850+0000 7fe3517fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:15.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.850+0000 7fe3517fa640 1 -- 172.21.15.12:0/3216502021 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364020040 con 0x564d9fb9f4d0 2024-04-22T13:30:15.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.850+0000 7fe3517fa640 1 -- 172.21.15.12:0/3216502021 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fe3640cdc20 con 0x564d9fb9f4d0 2024-04-22T13:30:15.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.850+0000 7fe37282c740 1 -- 172.21.15.12:0/3216502021 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe35801b470 con 0x564d9fb9f4d0 2024-04-22T13:30:15.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.850+0000 7fe3517fa640 1 -- 172.21.15.12:0/3216502021 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe36409c8f0 con 0x564d9fb9f4d0 2024-04-22T13:30:15.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.850+0000 7fe37282c740 10 client.17985 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:15.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.850+0000 7fe37282c740 1 -- 172.21.15.12:0/3216502021 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564d9fc1f2d0 con 0x564d9fb9f4d0 2024-04-22T13:30:15.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.850+0000 7fe3517fa640 1 -- 172.21.15.12:0/3216502021 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe35801b470 con 0x564d9fb9f4d0 2024-04-22T13:30:15.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.850+0000 7fe3517fa640 1 -- 172.21.15.12:0/3216502021 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe36409c450 con 0x564d9fb9f4d0 2024-04-22T13:30:15.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.850+0000 7fe37282c740 10 client.17985 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:15.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.850+0000 7fe37282c740 10 client.17985 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:30:15.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.850+0000 7fe37282c740 20 client.17985 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:15.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.850+0000 7fe37282c740 1 --2- 172.21.15.12:0/3216502021 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x564d9fc1f5c0 0x564d9fc219b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.851+0000 7fe37282c740 4 client.17985 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:30:15.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.851+0000 7fe37282c740 1 -- 172.21.15.12:0/3216502021 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564d9fb11c10 con 0x564d9fc1f5c0 2024-04-22T13:30:15.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.851+0000 7fe36d0f4640 1 --2- 172.21.15.12:0/3216502021 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x564d9fc1f5c0 0x564d9fc219b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.851+0000 7fe3517fa640 10 client.17985 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:30:15.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.851+0000 7fe36d0f4640 1 --2- 172.21.15.12:0/3216502021 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x564d9fc1f5c0 0x564d9fc219b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e2ee73df9e3c9804 server_cookie=238ca28fbdd713be in_seq=0 out_seq=0 2024-04-22T13:30:15.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.851+0000 7fe3517fa640 1 -- 172.21.15.12:0/3216502021 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564d9fb11c10 con 0x564d9fc1f5c0 2024-04-22T13:30:15.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.851+0000 7fe3517fa640 10 client.17985 handle_command_reply: tid=0 2024-04-22T13:30:15.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.851+0000 7fe37282c740 1 client.17985 shutdown 2024-04-22T13:30:15.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.851+0000 7fe37282c740 1 -- 172.21.15.12:0/3216502021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d9fb9f4d0 msgr2=0x564d9fba23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.851+0000 7fe37282c740 1 --2- 172.21.15.12:0/3216502021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d9fb9f4d0 0x564d9fba23a0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fe36409bef0 tx=0x7fe36409bc50 comp rx=0 tx=0).stop 2024-04-22T13:30:15.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.851+0000 7fe37282c740 1 -- 172.21.15.12:0/3216502021 shutdown_connections 2024-04-22T13:30:15.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.851+0000 7fe37282c740 1 --2- 172.21.15.12:0/3216502021 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x564d9fc1f5c0 0x564d9fc219b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.851+0000 7fe37282c740 1 --2- 172.21.15.12:0/3216502021 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564d9fba8a00 0x564d9fba2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.851+0000 7fe37282c740 1 --2- 172.21.15.12:0/3216502021 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564d9fb9fe80 0x564d9fba28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.851+0000 7fe3517fa640 0 client.17985 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:30:15.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.851+0000 7fe37282c740 1 --2- 172.21.15.12:0/3216502021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564d9fb9f4d0 0x564d9fba23a0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.852+0000 7fe37282c740 1 -- 172.21.15.12:0/3216502021 >> 172.21.15.12:0/3216502021 conn(0x564d9fb99ba0 msgr2=0x564d9fba0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:15.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.852+0000 7fe37282c740 1 -- 172.21.15.12:0/3216502021 shutdown_connections 2024-04-22T13:30:15.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.852+0000 7fe37282c740 1 -- 172.21.15.12:0/3216502021 wait complete. 2024-04-22T13:30:15.876 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.852+0000 7fe37282c740 20 client.17985 trim_cache size 0 max 16384 2024-04-22T13:30:15.877 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:30:15.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.852+0000 7fe36fb59640 1 -- 172.21.15.12:0/3261704961 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe3540a3da0 msgr2=0x7fe3540a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.852+0000 7fe36fb59640 1 --2- 172.21.15.12:0/3261704961 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe3540a3da0 0x7fe3540a6260 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe360006010 tx=0x7fe36002e040 comp rx=0 tx=0).stop 2024-04-22T13:30:15.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.852+0000 7fe36fb59640 1 -- 172.21.15.12:0/3261704961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe36806a5d0 msgr2=0x7fe3681a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.852+0000 7fe36fb59640 1 --2- 172.21.15.12:0/3261704961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe36806a5d0 0x7fe3681a0630 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fe36400d770 tx=0x7fe36400dc30 comp rx=0 tx=0).stop 2024-04-22T13:30:15.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.852+0000 7fe36fb59640 1 -- 172.21.15.12:0/3261704961 shutdown_connections 2024-04-22T13:30:15.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.852+0000 7fe36fb59640 1 --2- 172.21.15.12:0/3261704961 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe3540a3da0 0x7fe3540a6260 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.852+0000 7fe36fb59640 1 --2- 172.21.15.12:0/3261704961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe36810e960 0x7fe36819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.852+0000 7fe36fb59640 1 --2- 172.21.15.12:0/3261704961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe36806aef0 0x7fe3681a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.878 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.852+0000 7fe36fb59640 1 --2- 172.21.15.12:0/3261704961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe36806a5d0 0x7fe3681a0630 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.852+0000 7fe36fb59640 1 -- 172.21.15.12:0/3261704961 >> 172.21.15.12:0/3261704961 conn(0x7fe368077eb0 msgr2=0x7fe36810fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:15.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.852+0000 7fe36fb59640 1 -- 172.21.15.12:0/3261704961 shutdown_connections 2024-04-22T13:30:15.879 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.852+0000 7fe36fb59640 1 -- 172.21.15.12:0/3261704961 wait complete. 2024-04-22T13:30:15.879 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:30:15.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.987+0000 7fd9be2c0640 1 -- 172.21.15.12:0/2762586218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9b8107270 msgr2=0x7fd9b810ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.987+0000 7fd9be2c0640 1 --2- 172.21.15.12:0/2762586218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9b8107270 0x7fd9b810ee40 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0009ca0 tx=0x7fd9b002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.987+0000 7fd9be2c0640 1 -- 172.21.15.12:0/2762586218 shutdown_connections 2024-04-22T13:30:15.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.987+0000 7fd9be2c0640 1 --2- 172.21.15.12:0/2762586218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9b8107270 0x7fd9b810ee40 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.987+0000 7fd9be2c0640 1 --2- 172.21.15.12:0/2762586218 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b8102b50 0x7fd9b8106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.987+0000 7fd9be2c0640 1 --2- 172.21.15.12:0/2762586218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9b81021a0 0x7fd9b8102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.987+0000 7fd9be2c0640 1 -- 172.21.15.12:0/2762586218 >> 172.21.15.12:0/2762586218 conn(0x7fd9b8079350 msgr2=0x7fd9b8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:15.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.987+0000 7fd9be2c0640 1 -- 172.21.15.12:0/2762586218 shutdown_connections 2024-04-22T13:30:15.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.987+0000 7fd9be2c0640 1 -- 172.21.15.12:0/2762586218 wait complete. 2024-04-22T13:30:15.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.988+0000 7fd9be2c0640 1 Processor -- start 2024-04-22T13:30:15.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.988+0000 7fd9be2c0640 1 -- start start 2024-04-22T13:30:15.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.988+0000 7fd9be2c0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b81021a0 0x7fd9b8198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.988+0000 7fd9be2c0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9b8107270 0x7fd9b8198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.988+0000 7fd9be2c0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9b819d000 0x7fd9b819d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.988+0000 7fd9be2c0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b81121e0 con 0x7fd9b819d000 2024-04-22T13:30:15.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.988+0000 7fd9be2c0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b8112060 con 0x7fd9b8107270 2024-04-22T13:30:15.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.988+0000 7fd9be2c0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b8112360 con 0x7fd9b81021a0 2024-04-22T13:30:15.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.988+0000 7fd9bc85d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9b819d000 0x7fd9b819d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.988+0000 7fd9bc85d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9b819d000 0x7fd9b819d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51702/0 (socket says 172.21.15.12:51702) 2024-04-22T13:30:15.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.988+0000 7fd9bc85d640 1 -- 172.21.15.12:0/544952212 learned_addr learned my addr 172.21.15.12:0/544952212 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:15.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.988+0000 7fd9b77fe640 1 --2- 172.21.15.12:0/544952212 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9b8107270 0x7fd9b8198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.988+0000 7fd9b7fff640 1 --2- 172.21.15.12:0/544952212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b81021a0 0x7fd9b8198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.988+0000 7fd9bc85d640 1 -- 172.21.15.12:0/544952212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b81021a0 msgr2=0x7fd9b8198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.988+0000 7fd9bc85d640 1 --2- 172.21.15.12:0/544952212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b81021a0 0x7fd9b8198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.988+0000 7fd9bc85d640 1 -- 172.21.15.12:0/544952212 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9b8107270 msgr2=0x7fd9b8198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:15.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.988+0000 7fd9bc85d640 1 --2- 172.21.15.12:0/544952212 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9b8107270 0x7fd9b8198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.988+0000 7fd9bc85d640 1 -- 172.21.15.12:0/544952212 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b819daf0 con 0x7fd9b819d000 2024-04-22T13:30:15.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.989+0000 7fd9bc85d640 1 --2- 172.21.15.12:0/544952212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9b819d000 0x7fd9b819d4b0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0005e30 tx=0x7fd9b0032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:15.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.989+0000 7fd9b57fa640 1 -- 172.21.15.12:0/544952212 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0041020 con 0x7fd9b819d000 2024-04-22T13:30:15.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.989+0000 7fd9b57fa640 1 -- 172.21.15.12:0/544952212 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9b0032970 con 0x7fd9b819d000 2024-04-22T13:30:15.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.989+0000 7fd9be2c0640 1 -- 172.21.15.12:0/544952212 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9b819dd80 con 0x7fd9b819d000 2024-04-22T13:30:15.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.989+0000 7fd9b57fa640 1 -- 172.21.15.12:0/544952212 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0039040 con 0x7fd9b819d000 2024-04-22T13:30:15.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.989+0000 7fd9be2c0640 1 -- 172.21.15.12:0/544952212 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9b81a5540 con 0x7fd9b819d000 2024-04-22T13:30:15.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.991+0000 7fd9c0f82740 1 Processor -- start 2024-04-22T13:30:15.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.991+0000 7fd9b57fa640 1 -- 172.21.15.12:0/544952212 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd9b0044020 con 0x7fd9b819d000 2024-04-22T13:30:15.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.991+0000 7fd9c0f82740 1 -- start start 2024-04-22T13:30:15.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.991+0000 7fd9c0f82740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585b478e3b0 0x5585b478e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.991+0000 7fd9c0f82740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585b478ed60 0x5585b4797890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.991+0000 7fd9c0f82740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5585b4797e60 0x5585b47982c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.991+0000 7fd9c0f82740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5585b46f31f0 con 0x5585b478e3b0 2024-04-22T13:30:15.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.992+0000 7fd9c0f82740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5585b46904b0 con 0x5585b4797e60 2024-04-22T13:30:15.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.992+0000 7fd9c0f82740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5585b46ef1b0 con 0x5585b478ed60 2024-04-22T13:30:15.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.992+0000 7fd9b77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585b478e3b0 0x5585b478e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.992+0000 7fd9b77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585b478e3b0 0x5585b478e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51708/0 (socket says 172.21.15.12:51708) 2024-04-22T13:30:15.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.992+0000 7fd9b77fe640 1 -- 172.21.15.12:0/1079831785 learned_addr learned my addr 172.21.15.12:0/1079831785 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:15.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.992+0000 7fd9b57fa640 1 --2- 172.21.15.12:0/544952212 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd9a40a3d20 0x7fd9a40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:15.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.992+0000 7fd9b77fe640 1 --2- 172.21.15.12:0/1079831785 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5585b4797e60 0x5585b47982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.992+0000 7fd9b7fff640 1 --2- 172.21.15.12:0/1079831785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585b478ed60 0x5585b4797890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.992+0000 7fd9bc85d640 1 --2- 172.21.15.12:0/544952212 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd9a40a3d20 0x7fd9a40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:15.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.992+0000 7fd9b77fe640 1 -- 172.21.15.12:0/1079831785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585b478ed60 msgr2=0x5585b4797890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.992+0000 7fd9b77fe640 1 --2- 172.21.15.12:0/1079831785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585b478ed60 0x5585b4797890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.992+0000 7fd9b77fe640 1 -- 172.21.15.12:0/1079831785 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5585b4797e60 msgr2=0x5585b47982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:15.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.992+0000 7fd9b77fe640 1 --2- 172.21.15.12:0/1079831785 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5585b4797e60 0x5585b47982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:15.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.992+0000 7fd9b77fe640 1 -- 172.21.15.12:0/1079831785 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585b46821b0 con 0x5585b478e3b0 2024-04-22T13:30:15.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.992+0000 7fd9b57fa640 1 -- 172.21.15.12:0/544952212 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fd9b00f2470 con 0x7fd9b819d000 2024-04-22T13:30:15.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.992+0000 7fd9b77fe640 1 --2- 172.21.15.12:0/1079831785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585b478e3b0 0x5585b478e790 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800bb40 tx=0x7fd9a8004190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af4accbff28c175 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:15.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.992+0000 7fd9ae7fc640 1 -- 172.21.15.12:0/1079831785 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8005020 con 0x5585b478e3b0 2024-04-22T13:30:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.992+0000 7fd9ae7fc640 1 -- 172.21.15.12:0/1079831785 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9a8005670 con 0x5585b478e3b0 2024-04-22T13:30:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.992+0000 7fd9ae7fc640 1 -- 172.21.15.12:0/1079831785 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a80106c0 con 0x5585b478e3b0 2024-04-22T13:30:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.993+0000 7fd9c0f82740 1 -- 172.21.15.12:0/1079831785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585b478e3b0 msgr2=0x5585b478e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.993+0000 7fd9bc85d640 1 --2- 172.21.15.12:0/544952212 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd9a40a3d20 0x7fd9a40a61e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd9b00f0210 tx=0x7fd9b00ba170 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.993+0000 7fd9c0f82740 1 --2- 172.21.15.12:0/1079831785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585b478e3b0 0x5585b478e790 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800bb40 tx=0x7fd9a8004190 comp rx=0 tx=0).stop 2024-04-22T13:30:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.993+0000 7fd9c0f82740 1 -- 172.21.15.12:0/1079831785 shutdown_connections 2024-04-22T13:30:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.993+0000 7fd9c0f82740 1 --2- 172.21.15.12:0/1079831785 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5585b4797e60 0x5585b47982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.993+0000 7fd9c0f82740 1 --2- 172.21.15.12:0/1079831785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585b478ed60 0x5585b4797890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:16.000 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.993+0000 7fd9c0f82740 1 --2- 172.21.15.12:0/1079831785 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585b478e3b0 0x5585b478e790 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:16.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.993+0000 7fd9c0f82740 1 -- 172.21.15.12:0/1079831785 >> 172.21.15.12:0/1079831785 conn(0x5585b4788ba0 msgr2=0x5585b4788f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:16.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.993+0000 7fd9c0f82740 1 -- 172.21.15.12:0/1079831785 shutdown_connections 2024-04-22T13:30:16.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.993+0000 7fd9c0f82740 1 -- 172.21.15.12:0/1079831785 wait complete. 2024-04-22T13:30:16.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.993+0000 7fd9c0f82740 1 -- start start 2024-04-22T13:30:16.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.994+0000 7fd9c0f82740 1 Processor -- start 2024-04-22T13:30:16.001 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.994+0000 7fd9c0f82740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585b478e3b0 0x5585b479e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:16.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.994+0000 7fd9c0f82740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585b478ed60 0x5585b479b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:16.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.994+0000 7fd9c0f82740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5585b4797e60 0x5585b479ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:16.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.994+0000 7fd9c0f82740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5585b479ed40 con 0x5585b478e3b0 2024-04-22T13:30:16.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.994+0000 7fd9c0f82740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5585b46ac500 con 0x5585b4797e60 2024-04-22T13:30:16.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.994+0000 7fd9c0f82740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5585b479eec0 con 0x5585b478ed60 2024-04-22T13:30:16.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.994+0000 7fd9c0f82740 10 client.0 _ll_register_callbacks cb 0x5585b467ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:16.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.994+0000 7fd9b77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585b478e3b0 0x5585b479e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:16.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.994+0000 7fd9b77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585b478e3b0 0x5585b479e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51724/0 (socket says 172.21.15.12:51724) 2024-04-22T13:30:16.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.994+0000 7fd9b77fe640 1 -- 172.21.15.12:0/521598738 learned_addr learned my addr 172.21.15.12:0/521598738 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:16.002 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.994+0000 7fd9b7fff640 1 --2- 172.21.15.12:0/521598738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585b478ed60 0x5585b479b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:16.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.994+0000 7fd9b77fe640 1 --2- 172.21.15.12:0/521598738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5585b4797e60 0x5585b479ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:16.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.994+0000 7fd9b77fe640 1 -- 172.21.15.12:0/521598738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585b478ed60 msgr2=0x5585b479b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:16.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.994+0000 7fd9b77fe640 1 --2- 172.21.15.12:0/521598738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585b478ed60 0x5585b479b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:16.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.994+0000 7fd9b77fe640 1 -- 172.21.15.12:0/521598738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5585b4797e60 msgr2=0x5585b479ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:16.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.994+0000 7fd9b77fe640 1 --2- 172.21.15.12:0/521598738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5585b4797e60 0x5585b479ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:16.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.994+0000 7fd9b77fe640 1 -- 172.21.15.12:0/521598738 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585b44053f0 con 0x5585b478e3b0 2024-04-22T13:30:16.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.995+0000 7fd9b77fe640 1 --2- 172.21.15.12:0/521598738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585b478e3b0 0x5585b479e1c0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800b850 tx=0x7fd9a8004d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b1a873848df186a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:16.003 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.995+0000 7fd9977fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:16.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.995+0000 7fd9977fe640 1 -- 172.21.15.12:0/521598738 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8005200 con 0x5585b478e3b0 2024-04-22T13:30:16.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.995+0000 7fd9977fe640 1 -- 172.21.15.12:0/521598738 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fd9a80053a0 con 0x5585b478e3b0 2024-04-22T13:30:16.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.995+0000 7fd9c0f82740 1 -- 172.21.15.12:0/521598738 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9a8005670 con 0x5585b478e3b0 2024-04-22T13:30:16.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.995+0000 7fd9977fe640 1 -- 172.21.15.12:0/521598738 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd9a8020480 con 0x5585b478e3b0 2024-04-22T13:30:16.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.995+0000 7fd9c0f82740 10 client.17997 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:16.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.995+0000 7fd9c0f82740 1 -- 172.21.15.12:0/521598738 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5585b44cf7d0 con 0x5585b478e3b0 2024-04-22T13:30:16.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.995+0000 7fd9977fe640 1 -- 172.21.15.12:0/521598738 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9a8005670 con 0x5585b478e3b0 2024-04-22T13:30:16.004 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.995+0000 7fd9977fe640 1 -- 172.21.15.12:0/521598738 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd9a801f470 con 0x5585b478e3b0 2024-04-22T13:30:16.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.995+0000 7fd9c0f82740 10 client.17997 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:16.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.995+0000 7fd9c0f82740 10 client.17997 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:30:16.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.995+0000 7fd9c0f82740 20 client.17997 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:16.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.995+0000 7fd9c0f82740 1 --2- 172.21.15.12:0/521598738 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5585b47abeb0 0x5585b47ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:16.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.995+0000 7fd9c0f82740 4 client.17997 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:30:16.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.995+0000 7fd9c0f82740 1 -- 172.21.15.12:0/521598738 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585b4692930 con 0x5585b47abeb0 2024-04-22T13:30:16.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.996+0000 7fd9b7fff640 1 --2- 172.21.15.12:0/521598738 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5585b47abeb0 0x5585b47ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:16.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.996+0000 7fd9b7fff640 1 --2- 172.21.15.12:0/521598738 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5585b47abeb0 0x5585b47ae2a0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14643c082fb20a87 server_cookie=a5937cdb5588df55 in_seq=0 out_seq=0 2024-04-22T13:30:16.005 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.996+0000 7fd9977fe640 10 client.17997 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:16.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.997+0000 7fd9977fe640 1 -- 172.21.15.12:0/521598738 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5585b4692930 con 0x5585b47abeb0 2024-04-22T13:30:16.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.997+0000 7fd9977fe640 10 client.17997 handle_command_reply: tid=0 2024-04-22T13:30:16.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.997+0000 7fd9c0f82740 1 client.17997 shutdown 2024-04-22T13:30:16.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.997+0000 7fd9c0f82740 1 -- 172.21.15.12:0/521598738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585b478e3b0 msgr2=0x5585b479e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:16.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.997+0000 7fd9c0f82740 1 --2- 172.21.15.12:0/521598738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585b478e3b0 0x5585b479e1c0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800b850 tx=0x7fd9a8004d80 comp rx=0 tx=0).stop 2024-04-22T13:30:16.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.997+0000 7fd9c0f82740 1 -- 172.21.15.12:0/521598738 shutdown_connections 2024-04-22T13:30:16.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.997+0000 7fd9c0f82740 1 --2- 172.21.15.12:0/521598738 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5585b47abeb0 0x5585b47ae2a0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:16.006 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.997+0000 7fd9c0f82740 1 --2- 172.21.15.12:0/521598738 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5585b4797e60 0x5585b479ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:16.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.997+0000 7fd9c0f82740 1 --2- 172.21.15.12:0/521598738 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585b478ed60 0x5585b479b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:16.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.997+0000 7fd9977fe640 0 client.17997 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:16.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.997+0000 7fd9c0f82740 1 --2- 172.21.15.12:0/521598738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585b478e3b0 0x5585b479e1c0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:16.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.997+0000 7fd9c0f82740 1 -- 172.21.15.12:0/521598738 >> 172.21.15.12:0/521598738 conn(0x5585b4788ba0 msgr2=0x5585b478a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:16.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.997+0000 7fd9c0f82740 1 -- 172.21.15.12:0/521598738 shutdown_connections 2024-04-22T13:30:16.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.997+0000 7fd9c0f82740 1 -- 172.21.15.12:0/521598738 wait complete. 2024-04-22T13:30:16.007 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:15.997+0000 7fd9c0f82740 20 client.17997 trim_cache size 0 max 16384 2024-04-22T13:30:16.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.010+0000 7fd9c0f82740 1 Processor -- start 2024-04-22T13:30:16.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.010+0000 7fd9c0f82740 1 -- start start 2024-04-22T13:30:16.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.010+0000 7fd9c0f82740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585b478e3b0 0x5585b47a7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:16.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.010+0000 7fd9c0f82740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5585b478ed60 0x5585b479dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:16.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.010+0000 7fd9c0f82740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585b4797e60 0x5585b479e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:16.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.010+0000 7fd9c0f82740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b81ab7f0 con 0x5585b478e3b0 2024-04-22T13:30:16.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.010+0000 7fd9c0f82740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b81ad330 con 0x5585b478ed60 2024-04-22T13:30:16.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.010+0000 7fd9c0f82740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b81abec0 con 0x5585b4797e60 2024-04-22T13:30:16.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.011+0000 7fd9b77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585b478e3b0 0x5585b47a7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:16.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.011+0000 7fd9b77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585b478e3b0 0x5585b47a7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51728/0 (socket says 172.21.15.12:51728) 2024-04-22T13:30:16.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.011+0000 7fd9b77fe640 1 -- 172.21.15.12:0/4186263380 learned_addr learned my addr 172.21.15.12:0/4186263380 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:16.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.011+0000 7fd9b77fe640 1 -- 172.21.15.12:0/4186263380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585b4797e60 msgr2=0x5585b479e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:16.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.011+0000 7fd9b77fe640 1 --2- 172.21.15.12:0/4186263380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585b4797e60 0x5585b479e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:16.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.011+0000 7fd9b77fe640 1 -- 172.21.15.12:0/4186263380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5585b478ed60 msgr2=0x5585b479dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:16.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.011+0000 7fd9b77fe640 1 --2- 172.21.15.12:0/4186263380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5585b478ed60 0x5585b479dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:16.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.011+0000 7fd9b77fe640 1 -- 172.21.15.12:0/4186263380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585b478f1d0 con 0x5585b478e3b0 2024-04-22T13:30:16.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.011+0000 7fd9b77fe640 1 --2- 172.21.15.12:0/4186263380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585b478e3b0 0x5585b47a7ca0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800b950 tx=0x7fd9a8004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbc920f8b417f5e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:16.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.011+0000 7fd9adffb640 1 -- 172.21.15.12:0/4186263380 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a800b390 con 0x5585b478e3b0 2024-04-22T13:30:16.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.011+0000 7fd9adffb640 1 -- 172.21.15.12:0/4186263380 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9a8034020 con 0x5585b478e3b0 2024-04-22T13:30:16.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.011+0000 7fd9adffb640 1 -- 172.21.15.12:0/4186263380 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a801fcc0 con 0x5585b478e3b0 2024-04-22T13:30:16.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.012+0000 7fd9c0f82740 1 -- 172.21.15.12:0/4186263380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585b478e3b0 msgr2=0x5585b47a7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:16.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.012+0000 7fd9c0f82740 1 --2- 172.21.15.12:0/4186263380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585b478e3b0 0x5585b47a7ca0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800b950 tx=0x7fd9a8004260 comp rx=0 tx=0).stop 2024-04-22T13:30:16.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.012+0000 7fd9c0f82740 1 -- 172.21.15.12:0/4186263380 shutdown_connections 2024-04-22T13:30:16.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.012+0000 7fd9c0f82740 1 --2- 172.21.15.12:0/4186263380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585b4797e60 0x5585b479e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:16.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.012+0000 7fd9c0f82740 1 --2- 172.21.15.12:0/4186263380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5585b478ed60 0x5585b479dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:16.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.012+0000 7fd9c0f82740 1 --2- 172.21.15.12:0/4186263380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585b478e3b0 0x5585b47a7ca0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:16.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.012+0000 7fd9c0f82740 1 -- 172.21.15.12:0/4186263380 >> 172.21.15.12:0/4186263380 conn(0x5585b4788ba0 msgr2=0x5585b4787c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:16.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.012+0000 7fd9c0f82740 1 -- 172.21.15.12:0/4186263380 shutdown_connections 2024-04-22T13:30:16.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.012+0000 7fd9c0f82740 1 -- 172.21.15.12:0/4186263380 wait complete. 2024-04-22T13:30:16.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.012+0000 7fd9c0f82740 1 -- start start 2024-04-22T13:30:16.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.012+0000 7fd9c0f82740 1 Processor -- start 2024-04-22T13:30:16.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.012+0000 7fd9c0f82740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585b478e3b0 0x5585b47a4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:16.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.012+0000 7fd9c0f82740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5585b478ed60 0x5585b479d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:16.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.012+0000 7fd9c0f82740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585b4797e60 0x5585b479d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:16.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.012+0000 7fd9c0f82740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b81acc60 con 0x5585b478e3b0 2024-04-22T13:30:16.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.012+0000 7fd9c0f82740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b81ac590 con 0x5585b478ed60 2024-04-22T13:30:16.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.012+0000 7fd9c0f82740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b81a9810 con 0x5585b4797e60 2024-04-22T13:30:16.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.012+0000 7fd9c0f82740 10 client.0 _ll_register_callbacks cb 0x7fd9a8003df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:16.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.013+0000 7fd9b77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585b478e3b0 0x5585b47a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:16.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.013+0000 7fd9b77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585b478e3b0 0x5585b47a4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51736/0 (socket says 172.21.15.12:51736) 2024-04-22T13:30:16.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.013+0000 7fd9b77fe640 1 -- 172.21.15.12:0/2036853156 learned_addr learned my addr 172.21.15.12:0/2036853156 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:16.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.013+0000 7fd9b7fff640 1 --2- 172.21.15.12:0/2036853156 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5585b478ed60 0x5585b479d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:16.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.013+0000 7fd9b77fe640 1 -- 172.21.15.12:0/2036853156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585b4797e60 msgr2=0x5585b479d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:16.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.013+0000 7fd9b77fe640 1 --2- 172.21.15.12:0/2036853156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585b4797e60 0x5585b479d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:16.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.013+0000 7fd9b77fe640 1 -- 172.21.15.12:0/2036853156 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5585b478ed60 msgr2=0x5585b479d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:16.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.013+0000 7fd9b77fe640 1 --2- 172.21.15.12:0/2036853156 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5585b478ed60 0x5585b479d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:16.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.013+0000 7fd9b77fe640 1 -- 172.21.15.12:0/2036853156 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585b479dda0 con 0x5585b478e3b0 2024-04-22T13:30:16.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.013+0000 7fd9b7fff640 1 --2- 172.21.15.12:0/2036853156 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5585b478ed60 0x5585b479d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:16.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.013+0000 7fd9b77fe640 1 --2- 172.21.15.12:0/2036853156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585b478e3b0 0x5585b47a4620 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8003dc0 tx=0x7fd9a8003b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=479389be30235af0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:16.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.013+0000 7fd9977fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:16.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.013+0000 7fd9977fe640 1 -- 172.21.15.12:0/2036853156 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a800b390 con 0x5585b478e3b0 2024-04-22T13:30:16.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.013+0000 7fd9977fe640 1 -- 172.21.15.12:0/2036853156 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fd9a801fa60 con 0x5585b478e3b0 2024-04-22T13:30:16.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.013+0000 7fd9c0f82740 1 -- 172.21.15.12:0/2036853156 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9a8034020 con 0x5585b478e3b0 2024-04-22T13:30:16.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.013+0000 7fd9977fe640 1 -- 172.21.15.12:0/2036853156 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd9a801fc60 con 0x5585b478e3b0 2024-04-22T13:30:16.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.013+0000 7fd9c0f82740 10 client.18003 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:16.022 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.013+0000 7fd9c0f82740 1 -- 172.21.15.12:0/2036853156 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5585b4812290 con 0x5585b478e3b0 2024-04-22T13:30:16.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.014+0000 7fd9977fe640 1 -- 172.21.15.12:0/2036853156 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9a8034020 con 0x5585b478e3b0 2024-04-22T13:30:16.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.014+0000 7fd9977fe640 1 -- 172.21.15.12:0/2036853156 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd9a8038070 con 0x5585b478e3b0 2024-04-22T13:30:16.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.014+0000 7fd9c0f82740 10 client.18003 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:16.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.014+0000 7fd9c0f82740 10 client.18003 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:30:16.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.014+0000 7fd9c0f82740 20 client.18003 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:16.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.014+0000 7fd9c0f82740 1 --2- 172.21.15.12:0/2036853156 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5585b48128a0 0x5585b4814cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:16.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.014+0000 7fd9c0f82740 4 client.18003 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:30:16.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.014+0000 7fd9c0f82740 1 -- 172.21.15.12:0/2036853156 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5585b479be20 con 0x5585b48128a0 2024-04-22T13:30:16.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.014+0000 7fd9b7fff640 1 --2- 172.21.15.12:0/2036853156 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5585b48128a0 0x5585b4814cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:16.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.014+0000 7fd9977fe640 10 client.18003 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:16.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.014+0000 7fd9b7fff640 1 --2- 172.21.15.12:0/2036853156 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5585b48128a0 0x5585b4814cc0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb245f77356192ca server_cookie=d26fe31c15926270 in_seq=0 out_seq=0 2024-04-22T13:30:16.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.015+0000 7fd9977fe640 1 -- 172.21.15.12:0/2036853156 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5585b479be20 con 0x5585b48128a0 2024-04-22T13:30:16.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.015+0000 7fd9977fe640 10 client.18003 handle_command_reply: tid=0 2024-04-22T13:30:16.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.015+0000 7fd9c0f82740 1 client.18003 shutdown 2024-04-22T13:30:16.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.015+0000 7fd9c0f82740 1 -- 172.21.15.12:0/2036853156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585b478e3b0 msgr2=0x5585b47a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:16.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.015+0000 7fd9c0f82740 1 --2- 172.21.15.12:0/2036853156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585b478e3b0 0x5585b47a4620 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8003dc0 tx=0x7fd9a8003b10 comp rx=0 tx=0).stop 2024-04-22T13:30:16.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.015+0000 7fd9c0f82740 1 -- 172.21.15.12:0/2036853156 shutdown_connections 2024-04-22T13:30:16.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.015+0000 7fd9c0f82740 1 --2- 172.21.15.12:0/2036853156 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5585b48128a0 0x5585b4814cc0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:16.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.015+0000 7fd9c0f82740 1 --2- 172.21.15.12:0/2036853156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5585b4797e60 0x5585b479d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:16.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.015+0000 7fd9c0f82740 1 --2- 172.21.15.12:0/2036853156 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5585b478ed60 0x5585b479d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:16.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.015+0000 7fd9c0f82740 1 --2- 172.21.15.12:0/2036853156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5585b478e3b0 0x5585b47a4620 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:16.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.015+0000 7fd9977fe640 0 client.18003 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:16.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.015+0000 7fd9c0f82740 1 -- 172.21.15.12:0/2036853156 >> 172.21.15.12:0/2036853156 conn(0x5585b4788ba0 msgr2=0x5585b4799d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:16.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.015+0000 7fd9c0f82740 1 -- 172.21.15.12:0/2036853156 shutdown_connections 2024-04-22T13:30:16.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.015+0000 7fd9c0f82740 1 -- 172.21.15.12:0/2036853156 wait complete. 2024-04-22T13:30:16.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.015+0000 7fd9c0f82740 20 client.18003 trim_cache size 0 max 16384 2024-04-22T13:30:16.026 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:30:16.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.015+0000 7fd9be2c0640 1 -- 172.21.15.12:0/544952212 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd9a40a3d20 msgr2=0x7fd9a40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:16.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.015+0000 7fd9be2c0640 1 --2- 172.21.15.12:0/544952212 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd9a40a3d20 0x7fd9a40a61e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd9b00f0210 tx=0x7fd9b00ba170 comp rx=0 tx=0).stop 2024-04-22T13:30:16.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.015+0000 7fd9be2c0640 1 -- 172.21.15.12:0/544952212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9b819d000 msgr2=0x7fd9b819d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:16.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.015+0000 7fd9be2c0640 1 --2- 172.21.15.12:0/544952212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9b819d000 0x7fd9b819d4b0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0005e30 tx=0x7fd9b0032420 comp rx=0 tx=0).stop 2024-04-22T13:30:16.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.015+0000 7fd9be2c0640 1 -- 172.21.15.12:0/544952212 shutdown_connections 2024-04-22T13:30:16.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.015+0000 7fd9be2c0640 1 --2- 172.21.15.12:0/544952212 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd9a40a3d20 0x7fd9a40a61e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:16.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.015+0000 7fd9be2c0640 1 --2- 172.21.15.12:0/544952212 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9b819d000 0x7fd9b819d4b0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:16.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.015+0000 7fd9be2c0640 1 --2- 172.21.15.12:0/544952212 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9b8107270 0x7fd9b8198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:16.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.016+0000 7fd9be2c0640 1 --2- 172.21.15.12:0/544952212 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b81021a0 0x7fd9b8198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:16.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.016+0000 7fd9be2c0640 1 -- 172.21.15.12:0/544952212 >> 172.21.15.12:0/544952212 conn(0x7fd9b8079350 msgr2=0x7fd9b8100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:16.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.016+0000 7fd9be2c0640 1 -- 172.21.15.12:0/544952212 shutdown_connections 2024-04-22T13:30:16.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:16.016+0000 7fd9be2c0640 1 -- 172.21.15.12:0/544952212 wait complete. 2024-04-22T13:30:16.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:15 smithi134 ceph-mon[31338]: pgmap v187: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-22T13:30:16.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:15 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2983305822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:30:16.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:15 smithi111 ceph-mon[31361]: pgmap v187: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-22T13:30:16.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:15 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2983305822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:30:17.031 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:30:17.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:30:17.032 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:30:17.054 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:16 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:30:17.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.145+0000 7f2aca5a4640 1 -- 172.21.15.12:0/1726941105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ac4106160 msgr2=0x7f2ac410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:17.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.145+0000 7f2aca5a4640 1 --2- 172.21.15.12:0/1726941105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ac4106160 0x7f2ac410dd30 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f2abc009ca0 tx=0x7f2abc02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.145+0000 7f2aca5a4640 1 -- 172.21.15.12:0/1726941105 shutdown_connections 2024-04-22T13:30:17.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.145+0000 7f2aca5a4640 1 --2- 172.21.15.12:0/1726941105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ac4106160 0x7f2ac410dd30 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.145+0000 7f2aca5a4640 1 --2- 172.21.15.12:0/1726941105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ac4101a40 0x7f2ac4105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.145+0000 7f2aca5a4640 1 --2- 172.21.15.12:0/1726941105 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ac4101090 0x7f2ac4101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.145+0000 7f2aca5a4640 1 -- 172.21.15.12:0/1726941105 >> 172.21.15.12:0/1726941105 conn(0x7f2ac4077f50 msgr2=0x7f2ac4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:17.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.145+0000 7f2aca5a4640 1 -- 172.21.15.12:0/1726941105 shutdown_connections 2024-04-22T13:30:17.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.145+0000 7f2aca5a4640 1 -- 172.21.15.12:0/1726941105 wait complete. 2024-04-22T13:30:17.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.146+0000 7f2aca5a4640 1 Processor -- start 2024-04-22T13:30:17.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.146+0000 7f2aca5a4640 1 -- start start 2024-04-22T13:30:17.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.146+0000 7f2aca5a4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ac4101090 0x7f2ac4198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.146+0000 7f2aca5a4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ac4106160 0x7f2ac4198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.146+0000 7f2aca5a4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ac419cfe0 0x7f2ac419d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.146+0000 7f2aca5a4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac41110b0 con 0x7f2ac4101090 2024-04-22T13:30:17.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.146+0000 7f2aca5a4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac4110f30 con 0x7f2ac419cfe0 2024-04-22T13:30:17.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.146+0000 7f2aca5a4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac4111230 con 0x7f2ac4106160 2024-04-22T13:30:17.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.146+0000 7f2ac3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ac4101090 0x7f2ac4198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.147+0000 7f2ac3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ac4101090 0x7f2ac4198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51760/0 (socket says 172.21.15.12:51760) 2024-04-22T13:30:17.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.147+0000 7f2ac3fff640 1 -- 172.21.15.12:0/3744595514 learned_addr learned my addr 172.21.15.12:0/3744595514 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:17.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.147+0000 7f2ac8b41640 1 --2- 172.21.15.12:0/3744595514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ac419cfe0 0x7f2ac419d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.147+0000 7f2ac37fe640 1 --2- 172.21.15.12:0/3744595514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ac4106160 0x7f2ac4198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.147+0000 7f2ac3fff640 1 -- 172.21.15.12:0/3744595514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ac4106160 msgr2=0x7f2ac4198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:17.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.147+0000 7f2ac3fff640 1 --2- 172.21.15.12:0/3744595514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ac4106160 0x7f2ac4198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.147+0000 7f2ac3fff640 1 -- 172.21.15.12:0/3744595514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ac419cfe0 msgr2=0x7f2ac419d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:17.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.147+0000 7f2ac3fff640 1 --2- 172.21.15.12:0/3744595514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ac419cfe0 0x7f2ac419d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.147+0000 7f2ac3fff640 1 -- 172.21.15.12:0/3744595514 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac419dad0 con 0x7f2ac4101090 2024-04-22T13:30:17.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.147+0000 7f2ac37fe640 1 --2- 172.21.15.12:0/3744595514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ac4106160 0x7f2ac4198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:17.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.147+0000 7f2ac3fff640 1 --2- 172.21.15.12:0/3744595514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ac4101090 0x7f2ac4198720 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00d950 tx=0x7f2aac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:17.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.147+0000 7f2ac17fa640 1 -- 172.21.15.12:0/3744595514 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac017070 con 0x7f2ac4101090 2024-04-22T13:30:17.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.147+0000 7f2ac17fa640 1 -- 172.21.15.12:0/3744595514 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aac009c30 con 0x7f2ac4101090 2024-04-22T13:30:17.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.147+0000 7f2aca5a4640 1 -- 172.21.15.12:0/3744595514 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ac419ddc0 con 0x7f2ac4101090 2024-04-22T13:30:17.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.147+0000 7f2ac17fa640 1 -- 172.21.15.12:0/3744595514 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac00f620 con 0x7f2ac4101090 2024-04-22T13:30:17.147 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.147+0000 7f2aca5a4640 1 -- 172.21.15.12:0/3744595514 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ac41a55b0 con 0x7f2ac4101090 2024-04-22T13:30:17.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.149+0000 7f2ac17fa640 1 -- 172.21.15.12:0/3744595514 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2aac010460 con 0x7f2ac4101090 2024-04-22T13:30:17.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.150+0000 7f2acd193740 1 Processor -- start 2024-04-22T13:30:17.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.150+0000 7f2acd193740 1 -- start start 2024-04-22T13:30:17.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.150+0000 7f2acd193740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604512508e0 0x560451250cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.150+0000 7f2acd193740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560451251290 0x560451259840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.150+0000 7f2acd193740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560451259e10 0x56045125c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.150+0000 7f2acd193740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5604511b4f00 con 0x5604512508e0 2024-04-22T13:30:17.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.150+0000 7f2acd193740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560451152590 con 0x560451251290 2024-04-22T13:30:17.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.150+0000 7f2acd193740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5604511b0ec0 con 0x560451259e10 2024-04-22T13:30:17.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.150+0000 7f2ac8b41640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604512508e0 0x560451250cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.150+0000 7f2ac8b41640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604512508e0 0x560451250cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51772/0 (socket says 172.21.15.12:51772) 2024-04-22T13:30:17.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.150+0000 7f2ac8b41640 1 -- 172.21.15.12:0/3182231647 learned_addr learned my addr 172.21.15.12:0/3182231647 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:17.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.150+0000 7f2ac37fe640 1 --2- 172.21.15.12:0/3182231647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560451259e10 0x56045125c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.150+0000 7f2ac3fff640 1 --2- 172.21.15.12:0/3182231647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560451251290 0x560451259840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.150+0000 7f2ac8b41640 1 -- 172.21.15.12:0/3182231647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560451259e10 msgr2=0x56045125c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:17.156 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.150+0000 7f2ac8b41640 1 --2- 172.21.15.12:0/3182231647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560451259e10 0x56045125c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.150+0000 7f2ac8b41640 1 -- 172.21.15.12:0/3182231647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560451251290 msgr2=0x560451259840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:17.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.150+0000 7f2ac8b41640 1 --2- 172.21.15.12:0/3182231647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560451251290 0x560451259840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.150+0000 7f2ac8b41640 1 -- 172.21.15.12:0/3182231647 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560450f8f7e0 con 0x5604512508e0 2024-04-22T13:30:17.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.150+0000 7f2ac17fa640 1 --2- 172.21.15.12:0/3744595514 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2ab00a3d20 0x7f2ab00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.150+0000 7f2ac8b41640 1 --2- 172.21.15.12:0/3182231647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604512508e0 0x560451250cc0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f2abc009ca0 tx=0x7f2abc005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=664decf1f2c80523 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:17.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.150+0000 7f2ac3fff640 1 --2- 172.21.15.12:0/3744595514 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2ab00a3d20 0x7f2ab00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.151+0000 7f2aba7fc640 1 -- 172.21.15.12:0/3182231647 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc043070 con 0x5604512508e0 2024-04-22T13:30:17.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.151+0000 7f2ac17fa640 1 -- 172.21.15.12:0/3744595514 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f2aac0d03f0 con 0x7f2ac4101090 2024-04-22T13:30:17.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.151+0000 7f2aba7fc640 1 -- 172.21.15.12:0/3182231647 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2abc032970 con 0x5604512508e0 2024-04-22T13:30:17.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.151+0000 7f2aba7fc640 1 -- 172.21.15.12:0/3182231647 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc039040 con 0x5604512508e0 2024-04-22T13:30:17.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.151+0000 7f2acd193740 1 -- 172.21.15.12:0/3182231647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604512508e0 msgr2=0x560451250cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:17.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.151+0000 7f2acd193740 1 --2- 172.21.15.12:0/3182231647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604512508e0 0x560451250cc0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f2abc009ca0 tx=0x7f2abc005f70 comp rx=0 tx=0).stop 2024-04-22T13:30:17.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.151+0000 7f2ac3fff640 1 --2- 172.21.15.12:0/3744595514 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2ab00a3d20 0x7f2ab00a61e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2aac0cdc40 tx=0x7f2aac010cb0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:17.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.151+0000 7f2acd193740 1 -- 172.21.15.12:0/3182231647 shutdown_connections 2024-04-22T13:30:17.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.151+0000 7f2acd193740 1 --2- 172.21.15.12:0/3182231647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560451259e10 0x56045125c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.151+0000 7f2acd193740 1 --2- 172.21.15.12:0/3182231647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560451251290 0x560451259840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.151+0000 7f2acd193740 1 --2- 172.21.15.12:0/3182231647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604512508e0 0x560451250cc0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.151+0000 7f2acd193740 1 -- 172.21.15.12:0/3182231647 >> 172.21.15.12:0/3182231647 conn(0x56045124afb0 msgr2=0x56045124b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:17.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.151+0000 7f2acd193740 1 -- 172.21.15.12:0/3182231647 shutdown_connections 2024-04-22T13:30:17.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.151+0000 7f2acd193740 1 -- 172.21.15.12:0/3182231647 wait complete. 2024-04-22T13:30:17.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.152+0000 7f2acd193740 1 -- start start 2024-04-22T13:30:17.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.152+0000 7f2acd193740 1 Processor -- start 2024-04-22T13:30:17.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.152+0000 7f2acd193740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604512508e0 0x560451256d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.152+0000 7f2acd193740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560451251290 0x560451257290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.152+0000 7f2acd193740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560451259e10 0x5604512577d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.152+0000 7f2acd193740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56045125cd20 con 0x560451251290 2024-04-22T13:30:17.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.152+0000 7f2acd193740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56045116e580 con 0x5604512508e0 2024-04-22T13:30:17.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.152+0000 7f2acd193740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56045125cea0 con 0x560451259e10 2024-04-22T13:30:17.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.152+0000 7f2acd193740 10 client.0 _ll_register_callbacks cb 0x560451174650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:17.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.152+0000 7f2ac37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560451251290 0x560451257290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.152+0000 7f2ac37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560451251290 0x560451257290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51780/0 (socket says 172.21.15.12:51780) 2024-04-22T13:30:17.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.152+0000 7f2ac37fe640 1 -- 172.21.15.12:0/1877678779 learned_addr learned my addr 172.21.15.12:0/1877678779 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:17.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.152+0000 7f2ac8b41640 1 --2- 172.21.15.12:0/1877678779 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604512508e0 0x560451256d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.153+0000 7f2ac8b41640 1 --2- 172.21.15.12:0/1877678779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560451259e10 0x5604512577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.153+0000 7f2ac37fe640 1 -- 172.21.15.12:0/1877678779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560451259e10 msgr2=0x5604512577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:17.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.153+0000 7f2ac37fe640 1 --2- 172.21.15.12:0/1877678779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560451259e10 0x5604512577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.153+0000 7f2ac37fe640 1 -- 172.21.15.12:0/1877678779 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604512508e0 msgr2=0x560451256d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:17.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.153+0000 7f2ac37fe640 1 --2- 172.21.15.12:0/1877678779 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604512508e0 0x560451256d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.153+0000 7f2ac37fe640 1 -- 172.21.15.12:0/1877678779 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560450feff70 con 0x560451251290 2024-04-22T13:30:17.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.153+0000 7f2ac8b41640 1 --2- 172.21.15.12:0/1877678779 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604512508e0 0x560451256d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:17.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.153+0000 7f2ac37fe640 1 --2- 172.21.15.12:0/1877678779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560451251290 0x560451257290 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f2ab400da00 tx=0x7f2ab400dec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc5663e1e61e9c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:17.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.153+0000 7f2aa37fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:17.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.153+0000 7f2aa37fe640 1 -- 172.21.15.12:0/1877678779 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab401b440 con 0x560451251290 2024-04-22T13:30:17.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.153+0000 7f2aa37fe640 1 -- 172.21.15.12:0/1877678779 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f2ab4011440 con 0x560451251290 2024-04-22T13:30:17.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.153+0000 7f2acd193740 1 -- 172.21.15.12:0/1877678779 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2abc032970 con 0x560451251290 2024-04-22T13:30:17.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.153+0000 7f2aa37fe640 1 -- 172.21.15.12:0/1877678779 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2ab4011640 con 0x560451251290 2024-04-22T13:30:17.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.153+0000 7f2acd193740 10 client.18015 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:17.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.153+0000 7f2acd193740 1 -- 172.21.15.12:0/1877678779 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560450e87170 con 0x560451251290 2024-04-22T13:30:17.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.153+0000 7f2aa37fe640 1 -- 172.21.15.12:0/1877678779 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2abc032970 con 0x560451251290 2024-04-22T13:30:17.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.154+0000 7f2aa37fe640 1 -- 172.21.15.12:0/1877678779 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2ab4019070 con 0x560451251290 2024-04-22T13:30:17.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.154+0000 7f2acd193740 10 client.18015 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:17.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.154+0000 7f2acd193740 10 client.18015 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:30:17.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.154+0000 7f2acd193740 20 client.18015 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:17.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.154+0000 7f2acd193740 1 --2- 172.21.15.12:0/1877678779 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56045126a300 0x56045126c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.154+0000 7f2acd193740 4 client.18015 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:30:17.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.154+0000 7f2acd193740 1 -- 172.21.15.12:0/1877678779 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604511c2920 con 0x56045126a300 2024-04-22T13:30:17.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.154+0000 7f2ac8b41640 1 --2- 172.21.15.12:0/1877678779 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56045126a300 0x56045126c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.154+0000 7f2ac8b41640 1 --2- 172.21.15.12:0/1877678779 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56045126a300 0x56045126c780 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b356e081cc0230ba server_cookie=1def53b29f032b1a in_seq=0 out_seq=0 2024-04-22T13:30:17.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.154+0000 7f2aa37fe640 10 client.18015 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:17.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.155+0000 7f2aa37fe640 1 -- 172.21.15.12:0/1877678779 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5604511c2920 con 0x56045126a300 2024-04-22T13:30:17.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.155+0000 7f2aa37fe640 10 client.18015 handle_command_reply: tid=0 2024-04-22T13:30:17.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.155+0000 7f2acd193740 1 client.18015 shutdown 2024-04-22T13:30:17.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.155+0000 7f2acd193740 1 -- 172.21.15.12:0/1877678779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560451251290 msgr2=0x560451257290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:17.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.155+0000 7f2acd193740 1 --2- 172.21.15.12:0/1877678779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560451251290 0x560451257290 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f2ab400da00 tx=0x7f2ab400dec0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.155+0000 7f2acd193740 1 -- 172.21.15.12:0/1877678779 shutdown_connections 2024-04-22T13:30:17.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.155+0000 7f2acd193740 1 --2- 172.21.15.12:0/1877678779 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x56045126a300 0x56045126c780 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.155+0000 7f2acd193740 1 --2- 172.21.15.12:0/1877678779 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560451259e10 0x5604512577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.155+0000 7f2acd193740 1 --2- 172.21.15.12:0/1877678779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560451251290 0x560451257290 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.155+0000 7f2aa37fe640 0 client.18015 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:17.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.155+0000 7f2acd193740 1 --2- 172.21.15.12:0/1877678779 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604512508e0 0x560451256d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.155+0000 7f2acd193740 1 -- 172.21.15.12:0/1877678779 >> 172.21.15.12:0/1877678779 conn(0x56045124afb0 msgr2=0x56045124cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:17.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.155+0000 7f2acd193740 1 -- 172.21.15.12:0/1877678779 shutdown_connections 2024-04-22T13:30:17.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.155+0000 7f2acd193740 1 -- 172.21.15.12:0/1877678779 wait complete. 2024-04-22T13:30:17.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.155+0000 7f2acd193740 20 client.18015 trim_cache size 0 max 16384 2024-04-22T13:30:17.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:16 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:30:17.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.176+0000 7f2acd193740 1 Processor -- start 2024-04-22T13:30:17.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.176+0000 7f2acd193740 1 -- start start 2024-04-22T13:30:17.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.176+0000 7f2acd193740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604512508e0 0x560451259480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.176+0000 7f2acd193740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560451251290 0x560451265be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.176+0000 7f2acd193740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560451259e10 0x560451249e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.176+0000 7f2acd193740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac41adb30 con 0x5604512508e0 2024-04-22T13:30:17.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.176+0000 7f2acd193740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac41af670 con 0x560451259e10 2024-04-22T13:30:17.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.176+0000 7f2acd193740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac41ae200 con 0x560451251290 2024-04-22T13:30:17.183 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.176+0000 7f2ac8b41640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604512508e0 0x560451259480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.176+0000 7f2ac8b41640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604512508e0 0x560451259480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51786/0 (socket says 172.21.15.12:51786) 2024-04-22T13:30:17.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.176+0000 7f2ac8b41640 1 -- 172.21.15.12:0/3922315899 learned_addr learned my addr 172.21.15.12:0/3922315899 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:17.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.176+0000 7f2ac37fe640 1 --2- 172.21.15.12:0/3922315899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560451251290 0x560451265be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.176+0000 7f2ac8b41640 1 --2- 172.21.15.12:0/3922315899 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560451259e10 0x560451249e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.176+0000 7f2ac8b41640 1 -- 172.21.15.12:0/3922315899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560451251290 msgr2=0x560451265be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:17.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.176+0000 7f2ac8b41640 1 --2- 172.21.15.12:0/3922315899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560451251290 0x560451265be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.184 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.176+0000 7f2ac8b41640 1 -- 172.21.15.12:0/3922315899 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560451259e10 msgr2=0x560451249e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:17.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.176+0000 7f2ac8b41640 1 --2- 172.21.15.12:0/3922315899 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560451259e10 0x560451249e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.176+0000 7f2ac8b41640 1 -- 172.21.15.12:0/3922315899 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560451268ea0 con 0x5604512508e0 2024-04-22T13:30:17.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.177+0000 7f2ac8b41640 1 --2- 172.21.15.12:0/3922315899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604512508e0 0x560451259480 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f2abc003b20 tx=0x7f2abc005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97a2062fbe2c0a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:17.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.177+0000 7f2ab9ffb640 1 -- 172.21.15.12:0/3922315899 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc043040 con 0x5604512508e0 2024-04-22T13:30:17.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.177+0000 7f2ab9ffb640 1 -- 172.21.15.12:0/3922315899 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2abc032970 con 0x5604512508e0 2024-04-22T13:30:17.185 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.177+0000 7f2ab9ffb640 1 -- 172.21.15.12:0/3922315899 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc039040 con 0x5604512508e0 2024-04-22T13:30:17.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.178+0000 7f2acd193740 1 -- 172.21.15.12:0/3922315899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604512508e0 msgr2=0x560451259480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:17.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.178+0000 7f2acd193740 1 --2- 172.21.15.12:0/3922315899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604512508e0 0x560451259480 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f2abc003b20 tx=0x7f2abc005c50 comp rx=0 tx=0).stop 2024-04-22T13:30:17.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.178+0000 7f2acd193740 1 -- 172.21.15.12:0/3922315899 shutdown_connections 2024-04-22T13:30:17.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.178+0000 7f2acd193740 1 --2- 172.21.15.12:0/3922315899 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560451259e10 0x560451249e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.178+0000 7f2acd193740 1 --2- 172.21.15.12:0/3922315899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560451251290 0x560451265be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.178+0000 7f2acd193740 1 --2- 172.21.15.12:0/3922315899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604512508e0 0x560451259480 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.178+0000 7f2acd193740 1 -- 172.21.15.12:0/3922315899 >> 172.21.15.12:0/3922315899 conn(0x56045124afb0 msgr2=0x560451249780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.178+0000 7f2acd193740 1 -- 172.21.15.12:0/3922315899 shutdown_connections 2024-04-22T13:30:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.178+0000 7f2acd193740 1 -- 172.21.15.12:0/3922315899 wait complete. 2024-04-22T13:30:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.178+0000 7f2acd193740 1 -- start start 2024-04-22T13:30:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.179+0000 7f2acd193740 1 Processor -- start 2024-04-22T13:30:17.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.179+0000 7f2acd193740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604512508e0 0x5604512620b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.179+0000 7f2acd193740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560451251290 0x5604512625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.179+0000 7f2acd193740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560451259e10 0x56045125e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.179+0000 7f2acd193740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac41aefa0 con 0x560451251290 2024-04-22T13:30:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.179+0000 7f2acd193740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac41ae8d0 con 0x560451259e10 2024-04-22T13:30:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.179+0000 7f2acd193740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac41abb30 con 0x5604512508e0 2024-04-22T13:30:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.179+0000 7f2acd193740 10 client.0 _ll_register_callbacks cb 0x7f2ab40040c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:17.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.179+0000 7f2ac37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560451251290 0x5604512625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.179+0000 7f2ac8b41640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604512508e0 0x5604512620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.179+0000 7f2ac37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560451251290 0x5604512625f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51788/0 (socket says 172.21.15.12:51788) 2024-04-22T13:30:17.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.179+0000 7f2ac37fe640 1 -- 172.21.15.12:0/1223537035 learned_addr learned my addr 172.21.15.12:0/1223537035 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:17.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.179+0000 7f2ac8b41640 1 --2- 172.21.15.12:0/1223537035 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560451259e10 0x56045125e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.179+0000 7f2ac37fe640 1 -- 172.21.15.12:0/1223537035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604512508e0 msgr2=0x5604512620b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:17.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.179+0000 7f2ac37fe640 1 --2- 172.21.15.12:0/1223537035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604512508e0 0x5604512620b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.179+0000 7f2ac37fe640 1 -- 172.21.15.12:0/1223537035 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560451259e10 msgr2=0x56045125e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:17.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.179+0000 7f2ac37fe640 1 --2- 172.21.15.12:0/1223537035 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560451259e10 0x56045125e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.179+0000 7f2ac37fe640 1 -- 172.21.15.12:0/1223537035 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56045125ee30 con 0x560451251290 2024-04-22T13:30:17.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.179+0000 7f2ac8b41640 1 --2- 172.21.15.12:0/1223537035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604512508e0 0x5604512620b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:30:17.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.179+0000 7f2ac37fe640 1 --2- 172.21.15.12:0/1223537035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560451251290 0x5604512625f0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x560451258320 tx=0x7f2ab40039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d47583e70c7ac031 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:17.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.179+0000 7f2aa37fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:17.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.180+0000 7f2aa37fe640 1 -- 172.21.15.12:0/1223537035 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab4028040 con 0x560451251290 2024-04-22T13:30:17.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.180+0000 7f2aa37fe640 1 -- 172.21.15.12:0/1223537035 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f2ab40117e0 con 0x560451251290 2024-04-22T13:30:17.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.180+0000 7f2acd193740 1 -- 172.21.15.12:0/1223537035 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2abc032970 con 0x560451251290 2024-04-22T13:30:17.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.180+0000 7f2aa37fe640 1 -- 172.21.15.12:0/1223537035 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2ab40119e0 con 0x560451251290 2024-04-22T13:30:17.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.180+0000 7f2acd193740 10 client.18021 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:17.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.180+0000 7f2acd193740 1 -- 172.21.15.12:0/1223537035 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5604512d1520 con 0x560451251290 2024-04-22T13:30:17.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.180+0000 7f2aa37fe640 1 -- 172.21.15.12:0/1223537035 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2abc032970 con 0x560451251290 2024-04-22T13:30:17.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.180+0000 7f2aa37fe640 1 -- 172.21.15.12:0/1223537035 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2ab402b070 con 0x560451251290 2024-04-22T13:30:17.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.180+0000 7f2acd193740 10 client.18021 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:17.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.180+0000 7f2acd193740 10 client.18021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:30:17.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.180+0000 7f2acd193740 20 client.18021 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:17.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.180+0000 7f2acd193740 1 --2- 172.21.15.12:0/1223537035 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5604512d1c70 0x5604512d4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.180+0000 7f2acd193740 4 client.18021 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:30:17.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.180+0000 7f2acd193740 1 -- 172.21.15.12:0/1223537035 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5604511bc790 con 0x5604512d1c70 2024-04-22T13:30:17.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.181+0000 7f2ac8b41640 1 --2- 172.21.15.12:0/1223537035 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5604512d1c70 0x5604512d4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.181+0000 7f2ac8b41640 1 --2- 172.21.15.12:0/1223537035 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5604512d1c70 0x5604512d4090 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4991092e7bc77df server_cookie=a809807eef82db44 in_seq=0 out_seq=0 2024-04-22T13:30:17.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.181+0000 7f2aa37fe640 10 client.18021 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:17.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.182+0000 7f2aa37fe640 1 -- 172.21.15.12:0/1223537035 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5604511bc790 con 0x5604512d1c70 2024-04-22T13:30:17.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.182+0000 7f2aa37fe640 10 client.18021 handle_command_reply: tid=0 2024-04-22T13:30:17.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.182+0000 7f2acd193740 1 client.18021 shutdown 2024-04-22T13:30:17.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.182+0000 7f2acd193740 1 -- 172.21.15.12:0/1223537035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560451251290 msgr2=0x5604512625f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:17.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.182+0000 7f2acd193740 1 --2- 172.21.15.12:0/1223537035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560451251290 0x5604512625f0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x560451258320 tx=0x7f2ab40039e0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.182+0000 7f2acd193740 1 -- 172.21.15.12:0/1223537035 shutdown_connections 2024-04-22T13:30:17.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.182+0000 7f2acd193740 1 --2- 172.21.15.12:0/1223537035 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5604512d1c70 0x5604512d4090 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.182+0000 7f2acd193740 1 --2- 172.21.15.12:0/1223537035 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560451259e10 0x56045125e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.182+0000 7f2acd193740 1 --2- 172.21.15.12:0/1223537035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560451251290 0x5604512625f0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.182+0000 7f2aa37fe640 0 client.18021 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:17.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.182+0000 7f2acd193740 1 --2- 172.21.15.12:0/1223537035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604512508e0 0x5604512620b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.182+0000 7f2acd193740 1 -- 172.21.15.12:0/1223537035 >> 172.21.15.12:0/1223537035 conn(0x56045124afb0 msgr2=0x56045124a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:17.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.183+0000 7f2acd193740 1 -- 172.21.15.12:0/1223537035 shutdown_connections 2024-04-22T13:30:17.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.183+0000 7f2acd193740 1 -- 172.21.15.12:0/1223537035 wait complete. 2024-04-22T13:30:17.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.183+0000 7f2acd193740 20 client.18021 trim_cache size 0 max 16384 2024-04-22T13:30:17.197 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:30:17.197 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-04-22T13:30:17.197 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "63719491-526f-4912-b7f5-f6a6d05ef137", 2024-04-22T13:30:17.197 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-04-22T13:30:17.197 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:30:17.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.183+0000 7f2aca5a4640 1 -- 172.21.15.12:0/3744595514 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2ab00a3d20 msgr2=0x7f2ab00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.183+0000 7f2aca5a4640 1 --2- 172.21.15.12:0/3744595514 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2ab00a3d20 0x7f2ab00a61e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2aac0cdc40 tx=0x7f2aac010cb0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.183+0000 7f2aca5a4640 1 -- 172.21.15.12:0/3744595514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ac4101090 msgr2=0x7f2ac4198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.183+0000 7f2aca5a4640 1 --2- 172.21.15.12:0/3744595514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ac4101090 0x7f2ac4198720 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00d950 tx=0x7f2aac00de10 comp rx=0 tx=0).stop 2024-04-22T13:30:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.183+0000 7f2aca5a4640 1 -- 172.21.15.12:0/3744595514 shutdown_connections 2024-04-22T13:30:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.183+0000 7f2aca5a4640 1 --2- 172.21.15.12:0/3744595514 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2ab00a3d20 0x7f2ab00a61e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.183+0000 7f2aca5a4640 1 --2- 172.21.15.12:0/3744595514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ac419cfe0 0x7f2ac419d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.183+0000 7f2aca5a4640 1 --2- 172.21.15.12:0/3744595514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ac4106160 0x7f2ac4198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.183+0000 7f2aca5a4640 1 --2- 172.21.15.12:0/3744595514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2ac4101090 0x7f2ac4198720 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.183+0000 7f2aca5a4640 1 -- 172.21.15.12:0/3744595514 >> 172.21.15.12:0/3744595514 conn(0x7f2ac4077f50 msgr2=0x7f2ac4069420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.184+0000 7f2aca5a4640 1 -- 172.21.15.12:0/3744595514 shutdown_connections 2024-04-22T13:30:17.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.184+0000 7f2aca5a4640 1 -- 172.21.15.12:0/3744595514 wait complete. 2024-04-22T13:30:17.200 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:30:17.200 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:30:17.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:16 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:30:17.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.347+0000 7f2e2c515640 1 -- 172.21.15.12:0/1925450222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e24101a50 msgr2=0x7f2e24105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:17.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.347+0000 7f2e2c515640 1 --2- 172.21.15.12:0/1925450222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e24101a50 0x7f2e24105a90 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009900 tx=0x7f2e1402bc30 comp rx=0 tx=0).stop 2024-04-22T13:30:17.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.347+0000 7f2e2c515640 1 -- 172.21.15.12:0/1925450222 shutdown_connections 2024-04-22T13:30:17.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.347+0000 7f2e2c515640 1 --2- 172.21.15.12:0/1925450222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e24106170 0x7f2e2410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.347+0000 7f2e2c515640 1 --2- 172.21.15.12:0/1925450222 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e24101a50 0x7f2e24105a90 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.347+0000 7f2e2c515640 1 --2- 172.21.15.12:0/1925450222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e241010a0 0x7f2e24101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.347+0000 7f2e2c515640 1 -- 172.21.15.12:0/1925450222 >> 172.21.15.12:0/1925450222 conn(0x7f2e24077f70 msgr2=0x7f2e24078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:17.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.348+0000 7f2e2c515640 1 -- 172.21.15.12:0/1925450222 shutdown_connections 2024-04-22T13:30:17.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.348+0000 7f2e2c515640 1 -- 172.21.15.12:0/1925450222 wait complete. 2024-04-22T13:30:17.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.348+0000 7f2e2c515640 1 Processor -- start 2024-04-22T13:30:17.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.348+0000 7f2e2c515640 1 -- start start 2024-04-22T13:30:17.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.348+0000 7f2e2c515640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e241010a0 0x7f2e24197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.348+0000 7f2e2c515640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e24106170 0x7f2e24197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.348+0000 7f2e2c515640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e2419bf50 0x7f2e2419c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.348+0000 7f2e2c515640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241110b0 con 0x7f2e24106170 2024-04-22T13:30:17.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.348+0000 7f2e2c515640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2e24110f30 con 0x7f2e2419bf50 2024-04-22T13:30:17.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.348+0000 7f2e2c515640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2e24111230 con 0x7f2e241010a0 2024-04-22T13:30:17.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.348+0000 7f2e29ab0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e24106170 0x7f2e24197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.347 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.348+0000 7f2e29ab0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e24106170 0x7f2e24197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51812/0 (socket says 172.21.15.12:51812) 2024-04-22T13:30:17.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.348+0000 7f2e29ab0640 1 -- 172.21.15.12:0/2624043679 learned_addr learned my addr 172.21.15.12:0/2624043679 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:17.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.348+0000 7f2e2a2b1640 1 --2- 172.21.15.12:0/2624043679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e241010a0 0x7f2e24197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.349+0000 7f2e2aab2640 1 --2- 172.21.15.12:0/2624043679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e2419bf50 0x7f2e2419c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.349+0000 7f2e29ab0640 1 -- 172.21.15.12:0/2624043679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e241010a0 msgr2=0x7f2e24197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:17.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.349+0000 7f2e29ab0640 1 --2- 172.21.15.12:0/2624043679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e241010a0 0x7f2e24197690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.349+0000 7f2e29ab0640 1 -- 172.21.15.12:0/2624043679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e2419bf50 msgr2=0x7f2e2419c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:17.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.349+0000 7f2e29ab0640 1 --2- 172.21.15.12:0/2624043679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e2419bf50 0x7f2e2419c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.349+0000 7f2e29ab0640 1 -- 172.21.15.12:0/2624043679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e2419ca40 con 0x7f2e24106170 2024-04-22T13:30:17.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.349+0000 7f2e29ab0640 1 --2- 172.21.15.12:0/2624043679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e24106170 0x7f2e24197bd0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f2e14036040 tx=0x7f2e140045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:17.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.349+0000 7f2e1b7fe640 1 -- 172.21.15.12:0/2624043679 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e14043020 con 0x7f2e24106170 2024-04-22T13:30:17.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.349+0000 7f2e2c515640 1 -- 172.21.15.12:0/2624043679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e2419ccd0 con 0x7f2e24106170 2024-04-22T13:30:17.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.349+0000 7f2e1b7fe640 1 -- 172.21.15.12:0/2624043679 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e1402ca70 con 0x7f2e24106170 2024-04-22T13:30:17.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.349+0000 7f2e1b7fe640 1 -- 172.21.15.12:0/2624043679 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e14003ac0 con 0x7f2e24106170 2024-04-22T13:30:17.348 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.349+0000 7f2e2c515640 1 -- 172.21.15.12:0/2624043679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e241a44d0 con 0x7f2e24106170 2024-04-22T13:30:17.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.351+0000 7f2e1b7fe640 1 -- 172.21.15.12:0/2624043679 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2e14046020 con 0x7f2e24106170 2024-04-22T13:30:17.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.352+0000 7f2e2f1e4740 1 Processor -- start 2024-04-22T13:30:17.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.352+0000 7f2e2f1e4740 1 -- start start 2024-04-22T13:30:17.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.352+0000 7f2e2f1e4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d57609df60 0x55d57609e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.352+0000 7f2e2f1e4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d57609e910 0x55d5760a9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.352+0000 7f2e2f1e4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5760a9c90 0x55d5760ac080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.352+0000 7f2e2f1e4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d5760031e0 con 0x55d5760a9c90 2024-04-22T13:30:17.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.352+0000 7f2e2f1e4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d575fa0390 con 0x55d57609e910 2024-04-22T13:30:17.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.352+0000 7f2e2f1e4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d575fff1a0 con 0x55d57609df60 2024-04-22T13:30:17.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.352+0000 7f2e1b7fe640 1 --2- 172.21.15.12:0/2624043679 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2e100a3d20 0x7f2e100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.352+0000 7f2e29ab0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5760a9c90 0x55d5760ac080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.352+0000 7f2e29ab0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5760a9c90 0x55d5760ac080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51824/0 (socket says 172.21.15.12:51824) 2024-04-22T13:30:17.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.353+0000 7f2e29ab0640 1 -- 172.21.15.12:0/3261305730 learned_addr learned my addr 172.21.15.12:0/3261305730 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:17.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.353+0000 7f2e2a2b1640 1 --2- 172.21.15.12:0/3261305730 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d57609e910 0x55d5760a9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.353+0000 7f2e2aab2640 1 --2- 172.21.15.12:0/2624043679 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2e100a3d20 0x7f2e100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.353+0000 7f2e2aab2640 1 --2- 172.21.15.12:0/3261305730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d57609df60 0x55d57609e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.353+0000 7f2e1b7fe640 1 -- 172.21.15.12:0/2624043679 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f2e140f2780 con 0x7f2e24106170 2024-04-22T13:30:17.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.353+0000 7f2e29ab0640 1 -- 172.21.15.12:0/3261305730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d57609df60 msgr2=0x55d57609e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:17.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.353+0000 7f2e29ab0640 1 --2- 172.21.15.12:0/3261305730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d57609df60 0x55d57609e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.353+0000 7f2e29ab0640 1 -- 172.21.15.12:0/3261305730 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d57609e910 msgr2=0x55d5760a9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:17.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.353+0000 7f2e29ab0640 1 --2- 172.21.15.12:0/3261305730 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d57609e910 0x55d5760a9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.353+0000 7f2e29ab0640 1 -- 172.21.15.12:0/3261305730 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d575ddf7d0 con 0x55d5760a9c90 2024-04-22T13:30:17.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.353+0000 7f2e2a2b1640 1 --2- 172.21.15.12:0/3261305730 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d57609e910 0x55d5760a9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:30:17.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.353+0000 7f2e29ab0640 1 --2- 172.21.15.12:0/3261305730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5760a9c90 0x55d5760ac080 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f2e14006010 tx=0x7f2e1402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d8ee639eb467c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:17.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.353+0000 7f2e2aab2640 1 --2- 172.21.15.12:0/2624043679 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2e100a3d20 0x7f2e100a61e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c005fd0 tx=0x7f2e1c009b60 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:17.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.353+0000 7f2e18bfe640 1 -- 172.21.15.12:0/3261305730 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e140bef20 con 0x55d5760a9c90 2024-04-22T13:30:17.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.353+0000 7f2e18bfe640 1 -- 172.21.15.12:0/3261305730 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e1402cc20 con 0x55d5760a9c90 2024-04-22T13:30:17.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.353+0000 7f2e18bfe640 1 -- 172.21.15.12:0/3261305730 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e140053d0 con 0x55d5760a9c90 2024-04-22T13:30:17.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.354+0000 7f2e2f1e4740 1 -- 172.21.15.12:0/3261305730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5760a9c90 msgr2=0x55d5760ac080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:17.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.354+0000 7f2e2f1e4740 1 --2- 172.21.15.12:0/3261305730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5760a9c90 0x55d5760ac080 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f2e14006010 tx=0x7f2e1402bc30 comp rx=0 tx=0).stop 2024-04-22T13:30:17.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.354+0000 7f2e2f1e4740 1 -- 172.21.15.12:0/3261305730 shutdown_connections 2024-04-22T13:30:17.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.354+0000 7f2e2f1e4740 1 --2- 172.21.15.12:0/3261305730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5760a9c90 0x55d5760ac080 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.354+0000 7f2e2f1e4740 1 --2- 172.21.15.12:0/3261305730 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d57609e910 0x55d5760a9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.354+0000 7f2e2f1e4740 1 --2- 172.21.15.12:0/3261305730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d57609df60 0x55d57609e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.354+0000 7f2e2f1e4740 1 -- 172.21.15.12:0/3261305730 >> 172.21.15.12:0/3261305730 conn(0x55d576098cb0 msgr2=0x55d576099090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:17.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.354+0000 7f2e2f1e4740 1 -- 172.21.15.12:0/3261305730 shutdown_connections 2024-04-22T13:30:17.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.354+0000 7f2e2f1e4740 1 -- 172.21.15.12:0/3261305730 wait complete. 2024-04-22T13:30:17.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.354+0000 7f2e2f1e4740 1 -- start start 2024-04-22T13:30:17.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.355+0000 7f2e2f1e4740 1 Processor -- start 2024-04-22T13:30:17.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.355+0000 7f2e2f1e4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d57609df60 0x55d5760a6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.355+0000 7f2e2f1e4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d57609e910 0x55d5760a7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.355+0000 7f2e2f1e4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5760a9c90 0x55d5760a7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.355+0000 7f2e2f1e4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d5760ac810 con 0x55d5760a9c90 2024-04-22T13:30:17.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.355+0000 7f2e2f1e4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d575fbc2f0 con 0x55d57609df60 2024-04-22T13:30:17.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.355+0000 7f2e2f1e4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d5760ac990 con 0x55d57609e910 2024-04-22T13:30:17.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.355+0000 7f2e2f1e4740 10 client.0 _ll_register_callbacks cb 0x55d575fc9580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:17.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.355+0000 7f2e2aab2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5760a9c90 0x55d5760a7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.355+0000 7f2e2aab2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5760a9c90 0x55d5760a7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51838/0 (socket says 172.21.15.12:51838) 2024-04-22T13:30:17.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.355+0000 7f2e2aab2640 1 -- 172.21.15.12:0/1296910014 learned_addr learned my addr 172.21.15.12:0/1296910014 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:17.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.355+0000 7f2e29ab0640 1 --2- 172.21.15.12:0/1296910014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d57609e910 0x55d5760a7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.355+0000 7f2e2a2b1640 1 --2- 172.21.15.12:0/1296910014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d57609df60 0x55d5760a6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.355+0000 7f2e2aab2640 1 -- 172.21.15.12:0/1296910014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d57609e910 msgr2=0x55d5760a7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:17.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.355+0000 7f2e2aab2640 1 --2- 172.21.15.12:0/1296910014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d57609e910 0x55d5760a7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.355+0000 7f2e2aab2640 1 -- 172.21.15.12:0/1296910014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d57609df60 msgr2=0x55d5760a6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:17.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.355+0000 7f2e2aab2640 1 --2- 172.21.15.12:0/1296910014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d57609df60 0x55d5760a6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.355+0000 7f2e2aab2640 1 -- 172.21.15.12:0/1296910014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d575d153f0 con 0x55d5760a9c90 2024-04-22T13:30:17.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.355+0000 7f2e2a2b1640 1 --2- 172.21.15.12:0/1296910014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d57609df60 0x55d5760a6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.355+0000 7f2e2aab2640 1 --2- 172.21.15.12:0/1296910014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5760a9c90 0x55d5760a7750 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00fe70 tx=0x7f2e1c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb69afbda17e88e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.355+0000 7f2e0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.356+0000 7f2e0dffb640 1 -- 172.21.15.12:0/1296910014 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c025070 con 0x55d5760a9c90 2024-04-22T13:30:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.356+0000 7f2e2f1e4740 1 -- 172.21.15.12:0/1296910014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e1402cc20 con 0x55d5760a9c90 2024-04-22T13:30:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.356+0000 7f2e0dffb640 1 -- 172.21.15.12:0/1296910014 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f2e1c004b20 con 0x55d5760a9c90 2024-04-22T13:30:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.356+0000 7f2e0dffb640 1 -- 172.21.15.12:0/1296910014 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2e1c00fea0 con 0x55d5760a9c90 2024-04-22T13:30:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.356+0000 7f2e2f1e4740 10 client.18033 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.356+0000 7f2e2f1e4740 1 -- 172.21.15.12:0/1296910014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d575cd50f0 con 0x55d5760a9c90 2024-04-22T13:30:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.356+0000 7f2e0dffb640 1 -- 172.21.15.12:0/1296910014 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e1402cc20 con 0x55d5760a9c90 2024-04-22T13:30:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.356+0000 7f2e0dffb640 1 -- 172.21.15.12:0/1296910014 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2e1c033480 con 0x55d5760a9c90 2024-04-22T13:30:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.356+0000 7f2e2f1e4740 10 client.18033 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.356+0000 7f2e2f1e4740 10 client.18033 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:30:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.356+0000 7f2e2f1e4740 20 client.18033 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:17.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.356+0000 7f2e2f1e4740 1 --2- 172.21.15.12:0/1296910014 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d5760b9a00 0x55d5760bbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.356+0000 7f2e2f1e4740 4 client.18033 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:30:17.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.356+0000 7f2e2f1e4740 1 -- 172.21.15.12:0/1296910014 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d576010c00 con 0x55d5760b9a00 2024-04-22T13:30:17.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.357+0000 7f2e2a2b1640 1 --2- 172.21.15.12:0/1296910014 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d5760b9a00 0x55d5760bbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.357+0000 7f2e2a2b1640 1 --2- 172.21.15.12:0/1296910014 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d5760b9a00 0x55d5760bbe60 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94d7e9cc030f5d30 server_cookie=f0333ecfa7f9a8f in_seq=0 out_seq=0 2024-04-22T13:30:17.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.357+0000 7f2e0dffb640 10 client.18033 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:17.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.357+0000 7f2e0dffb640 1 -- 172.21.15.12:0/1296910014 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d576010c00 con 0x55d5760b9a00 2024-04-22T13:30:17.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.358+0000 7f2e0dffb640 10 client.18033 handle_command_reply: tid=0 2024-04-22T13:30:17.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.358+0000 7f2e2f1e4740 1 client.18033 shutdown 2024-04-22T13:30:17.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.358+0000 7f2e2f1e4740 1 -- 172.21.15.12:0/1296910014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5760a9c90 msgr2=0x55d5760a7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:17.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.358+0000 7f2e2f1e4740 1 --2- 172.21.15.12:0/1296910014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5760a9c90 0x55d5760a7750 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00fe70 tx=0x7f2e1c004ab0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.358+0000 7f2e2f1e4740 1 -- 172.21.15.12:0/1296910014 shutdown_connections 2024-04-22T13:30:17.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.358+0000 7f2e2f1e4740 1 --2- 172.21.15.12:0/1296910014 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d5760b9a00 0x55d5760bbe60 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.358+0000 7f2e2f1e4740 1 --2- 172.21.15.12:0/1296910014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5760a9c90 0x55d5760a7750 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.358+0000 7f2e2f1e4740 1 --2- 172.21.15.12:0/1296910014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d57609e910 0x55d5760a7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.358+0000 7f2e0dffb640 0 client.18033 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:17.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.358+0000 7f2e2f1e4740 1 --2- 172.21.15.12:0/1296910014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d57609df60 0x55d5760a6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.358+0000 7f2e2f1e4740 1 -- 172.21.15.12:0/1296910014 >> 172.21.15.12:0/1296910014 conn(0x55d576098cb0 msgr2=0x55d57609aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:17.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.358+0000 7f2e2f1e4740 1 -- 172.21.15.12:0/1296910014 shutdown_connections 2024-04-22T13:30:17.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.358+0000 7f2e2f1e4740 1 -- 172.21.15.12:0/1296910014 wait complete. 2024-04-22T13:30:17.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.358+0000 7f2e2f1e4740 20 client.18033 trim_cache size 0 max 16384 2024-04-22T13:30:17.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.375+0000 7f2e2f1e4740 1 Processor -- start 2024-04-22T13:30:17.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.375+0000 7f2e2f1e4740 1 -- start start 2024-04-22T13:30:17.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.375+0000 7f2e2f1e4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d57609df60 0x55d5760a9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.375+0000 7f2e2f1e4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d57609e910 0x55d5760b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.376+0000 7f2e2f1e4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5760a9c90 0x55d57609a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.376+0000 7f2e2f1e4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241ab7f0 con 0x55d57609e910 2024-04-22T13:30:17.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.376+0000 7f2e2f1e4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241ad330 con 0x55d57609df60 2024-04-22T13:30:17.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.376+0000 7f2e2f1e4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241abec0 con 0x55d5760a9c90 2024-04-22T13:30:17.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.376+0000 7f2e29ab0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d57609e910 0x55d5760b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.376+0000 7f2e29ab0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d57609e910 0x55d5760b56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51844/0 (socket says 172.21.15.12:51844) 2024-04-22T13:30:17.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.376+0000 7f2e29ab0640 1 -- 172.21.15.12:0/1020536649 learned_addr learned my addr 172.21.15.12:0/1020536649 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:17.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.376+0000 7f2e2a2b1640 1 --2- 172.21.15.12:0/1020536649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d57609df60 0x55d5760a9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.376+0000 7f2e2aab2640 1 --2- 172.21.15.12:0/1020536649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5760a9c90 0x55d57609a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.376+0000 7f2e29ab0640 1 -- 172.21.15.12:0/1020536649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5760a9c90 msgr2=0x55d57609a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:17.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.376+0000 7f2e29ab0640 1 --2- 172.21.15.12:0/1020536649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5760a9c90 0x55d57609a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.376+0000 7f2e29ab0640 1 -- 172.21.15.12:0/1020536649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d57609df60 msgr2=0x55d5760a9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:17.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.376+0000 7f2e29ab0640 1 --2- 172.21.15.12:0/1020536649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d57609df60 0x55d5760a9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.376+0000 7f2e29ab0640 1 -- 172.21.15.12:0/1020536649 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5760b8600 con 0x55d57609e910 2024-04-22T13:30:17.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.376+0000 7f2e29ab0640 1 --2- 172.21.15.12:0/1020536649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d57609e910 0x55d5760b56d0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f2e140be520 tx=0x7f2e14009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f955052b943c92f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:17.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.376+0000 7f2e0ffff640 1 -- 172.21.15.12:0/1020536649 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e140c2bd0 con 0x55d57609e910 2024-04-22T13:30:17.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.376+0000 7f2e0ffff640 1 -- 172.21.15.12:0/1020536649 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e140bef20 con 0x55d57609e910 2024-04-22T13:30:17.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.376+0000 7f2e0ffff640 1 -- 172.21.15.12:0/1020536649 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e140baff0 con 0x55d57609e910 2024-04-22T13:30:17.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.377+0000 7f2e2f1e4740 1 -- 172.21.15.12:0/1020536649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d57609e910 msgr2=0x55d5760b56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:17.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.377+0000 7f2e2f1e4740 1 --2- 172.21.15.12:0/1020536649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d57609e910 0x55d5760b56d0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f2e140be520 tx=0x7f2e14009260 comp rx=0 tx=0).stop 2024-04-22T13:30:17.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.377+0000 7f2e2f1e4740 1 -- 172.21.15.12:0/1020536649 shutdown_connections 2024-04-22T13:30:17.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.377+0000 7f2e2f1e4740 1 --2- 172.21.15.12:0/1020536649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d5760a9c90 0x55d57609a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.377+0000 7f2e2f1e4740 1 --2- 172.21.15.12:0/1020536649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d57609e910 0x55d5760b56d0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.377+0000 7f2e2f1e4740 1 --2- 172.21.15.12:0/1020536649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d57609df60 0x55d5760a9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.377+0000 7f2e2f1e4740 1 -- 172.21.15.12:0/1020536649 >> 172.21.15.12:0/1020536649 conn(0x55d576098cb0 msgr2=0x55d57609a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:17.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.377+0000 7f2e2f1e4740 1 -- 172.21.15.12:0/1020536649 shutdown_connections 2024-04-22T13:30:17.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.377+0000 7f2e2f1e4740 1 -- 172.21.15.12:0/1020536649 wait complete. 2024-04-22T13:30:17.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.377+0000 7f2e2f1e4740 1 -- start start 2024-04-22T13:30:17.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.377+0000 7f2e2f1e4740 1 Processor -- start 2024-04-22T13:30:17.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.377+0000 7f2e2f1e4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d57609df60 0x55d5760ada10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.377+0000 7f2e2f1e4740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d57609e910 0x55d5760adf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.377+0000 7f2e2f1e4740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5760a9c90 0x55d57611efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.377+0000 7f2e2f1e4740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241acc60 con 0x55d5760a9c90 2024-04-22T13:30:17.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.377+0000 7f2e2f1e4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241ac590 con 0x55d57609df60 2024-04-22T13:30:17.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.377+0000 7f2e2f1e4740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241a97d0 con 0x55d57609e910 2024-04-22T13:30:17.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.377+0000 7f2e2f1e4740 10 client.0 _ll_register_callbacks cb 0x7f2e1c009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:17.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.378+0000 7f2e2aab2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5760a9c90 0x55d57611efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.378+0000 7f2e2aab2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5760a9c90 0x55d57611efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51860/0 (socket says 172.21.15.12:51860) 2024-04-22T13:30:17.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.378+0000 7f2e2aab2640 1 -- 172.21.15.12:0/2079352359 learned_addr learned my addr 172.21.15.12:0/2079352359 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:17.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.378+0000 7f2e2a2b1640 1 --2- 172.21.15.12:0/2079352359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d57609df60 0x55d5760ada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.378+0000 7f2e29ab0640 1 --2- 172.21.15.12:0/2079352359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d57609e910 0x55d5760adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.378+0000 7f2e2aab2640 1 -- 172.21.15.12:0/2079352359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d57609e910 msgr2=0x55d5760adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:17.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.378+0000 7f2e2aab2640 1 --2- 172.21.15.12:0/2079352359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d57609e910 0x55d5760adf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.378+0000 7f2e2aab2640 1 -- 172.21.15.12:0/2079352359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d57609df60 msgr2=0x55d5760ada10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:17.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.378+0000 7f2e2aab2640 1 --2- 172.21.15.12:0/2079352359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d57609df60 0x55d5760ada10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.378+0000 7f2e2aab2640 1 -- 172.21.15.12:0/2079352359 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d57611f4e0 con 0x55d5760a9c90 2024-04-22T13:30:17.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.378+0000 7f2e2aab2640 1 --2- 172.21.15.12:0/2079352359 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5760a9c90 0x55d57611efa0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c000f80 tx=0x7f2e1c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=105b181e57ace6bf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:17.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.378+0000 7f2e0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:17.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.378+0000 7f2e0dffb640 1 -- 172.21.15.12:0/2079352359 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c025070 con 0x55d5760a9c90 2024-04-22T13:30:17.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.378+0000 7f2e0dffb640 1 -- 172.21.15.12:0/2079352359 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f2e1c033ac0 con 0x55d5760a9c90 2024-04-22T13:30:17.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.378+0000 7f2e2f1e4740 1 -- 172.21.15.12:0/2079352359 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e140bef20 con 0x55d5760a9c90 2024-04-22T13:30:17.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.378+0000 7f2e0dffb640 1 -- 172.21.15.12:0/2079352359 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2e1c00ec40 con 0x55d5760a9c90 2024-04-22T13:30:17.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.378+0000 7f2e2f1e4740 10 client.18039 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:17.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.378+0000 7f2e2f1e4740 1 -- 172.21.15.12:0/2079352359 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d5761206b0 con 0x55d5760a9c90 2024-04-22T13:30:17.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.378+0000 7f2e0dffb640 1 -- 172.21.15.12:0/2079352359 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e140bef20 con 0x55d5760a9c90 2024-04-22T13:30:17.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.379+0000 7f2e0dffb640 1 -- 172.21.15.12:0/2079352359 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2e1c0044f0 con 0x55d5760a9c90 2024-04-22T13:30:17.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.379+0000 7f2e2f1e4740 10 client.18039 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:17.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.379+0000 7f2e2f1e4740 10 client.18039 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:30:17.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.379+0000 7f2e2f1e4740 20 client.18039 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:17.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.379+0000 7f2e2f1e4740 1 --2- 172.21.15.12:0/2079352359 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d576120e50 0x55d576123270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:17.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.379+0000 7f2e2f1e4740 4 client.18039 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:30:17.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.379+0000 7f2e2f1e4740 1 -- 172.21.15.12:0/2079352359 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d57600aa70 con 0x55d576120e50 2024-04-22T13:30:17.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.379+0000 7f2e2a2b1640 1 --2- 172.21.15.12:0/2079352359 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d576120e50 0x55d576123270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:17.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.379+0000 7f2e0dffb640 10 client.18039 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:17.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.379+0000 7f2e2a2b1640 1 --2- 172.21.15.12:0/2079352359 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d576120e50 0x55d576123270 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2281a3c0d0457dd6 server_cookie=9ab90818e8a38c7a in_seq=0 out_seq=0 2024-04-22T13:30:17.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.379+0000 7f2e0dffb640 1 -- 172.21.15.12:0/2079352359 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d57600aa70 con 0x55d576120e50 2024-04-22T13:30:17.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.379+0000 7f2e0dffb640 10 client.18039 handle_command_reply: tid=0 2024-04-22T13:30:17.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.379+0000 7f2e2f1e4740 1 client.18039 shutdown 2024-04-22T13:30:17.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.379+0000 7f2e2f1e4740 1 -- 172.21.15.12:0/2079352359 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5760a9c90 msgr2=0x55d57611efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:17.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.379+0000 7f2e2f1e4740 1 --2- 172.21.15.12:0/2079352359 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5760a9c90 0x55d57611efa0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c000f80 tx=0x7f2e1c003ce0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.379+0000 7f2e2f1e4740 1 -- 172.21.15.12:0/2079352359 shutdown_connections 2024-04-22T13:30:17.383 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.379+0000 7f2e2f1e4740 1 --2- 172.21.15.12:0/2079352359 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55d576120e50 0x55d576123270 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.379+0000 7f2e2f1e4740 1 --2- 172.21.15.12:0/2079352359 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d5760a9c90 0x55d57611efa0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.379+0000 7f2e0dffb640 0 client.18039 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:17.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.379+0000 7f2e2f1e4740 1 --2- 172.21.15.12:0/2079352359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d57609e910 0x55d5760adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.379+0000 7f2e2f1e4740 1 --2- 172.21.15.12:0/2079352359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d57609df60 0x55d5760ada10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.379+0000 7f2e2f1e4740 1 -- 172.21.15.12:0/2079352359 >> 172.21.15.12:0/2079352359 conn(0x55d576098cb0 msgr2=0x55d5760a7520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:17.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.380+0000 7f2e2f1e4740 1 -- 172.21.15.12:0/2079352359 shutdown_connections 2024-04-22T13:30:17.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.380+0000 7f2e2f1e4740 1 -- 172.21.15.12:0/2079352359 wait complete. 2024-04-22T13:30:17.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.380+0000 7f2e2f1e4740 20 client.18039 trim_cache size 0 max 16384 2024-04-22T13:30:17.385 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:30:17.385 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:30:17.385 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-04-22T13:30:17.385 INFO:teuthology.orchestra.run.smithi012.stdout: "63719491-526f-4912-b7f5-f6a6d05ef137": { 2024-04-22T13:30:17.385 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-04-22T13:30:17.385 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "63719491-526f-4912-b7f5-f6a6d05ef137", 2024-04-22T13:30:17.385 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-04-22T13:30:17.385 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:30:17.385 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:30:17.385 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:30:17.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.380+0000 7f2e2c515640 1 -- 172.21.15.12:0/2624043679 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2e100a3d20 msgr2=0x7f2e100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:17.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.380+0000 7f2e2c515640 1 --2- 172.21.15.12:0/2624043679 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2e100a3d20 0x7f2e100a61e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c005fd0 tx=0x7f2e1c009b60 comp rx=0 tx=0).stop 2024-04-22T13:30:17.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.380+0000 7f2e2c515640 1 -- 172.21.15.12:0/2624043679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e24106170 msgr2=0x7f2e24197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:17.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.380+0000 7f2e2c515640 1 --2- 172.21.15.12:0/2624043679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e24106170 0x7f2e24197bd0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f2e14036040 tx=0x7f2e140045e0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.380+0000 7f2e2c515640 1 -- 172.21.15.12:0/2624043679 shutdown_connections 2024-04-22T13:30:17.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.380+0000 7f2e2c515640 1 --2- 172.21.15.12:0/2624043679 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2e100a3d20 0x7f2e100a61e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.380+0000 7f2e2c515640 1 --2- 172.21.15.12:0/2624043679 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e2419bf50 0x7f2e2419c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.380+0000 7f2e2c515640 1 --2- 172.21.15.12:0/2624043679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e24106170 0x7f2e24197bd0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.380+0000 7f2e2c515640 1 --2- 172.21.15.12:0/2624043679 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e241010a0 0x7f2e24197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:17.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.380+0000 7f2e2c515640 1 -- 172.21.15.12:0/2624043679 >> 172.21.15.12:0/2624043679 conn(0x7f2e24077f70 msgr2=0x7f2e240ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:17.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.380+0000 7f2e2c515640 1 -- 172.21.15.12:0/2624043679 shutdown_connections 2024-04-22T13:30:17.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:17.380+0000 7f2e2c515640 1 -- 172.21.15.12:0/2624043679 wait complete. 2024-04-22T13:30:17.388 INFO:tasks.cephfs.filesystem:scrub status for tag:63719491-526f-4912-b7f5-f6a6d05ef137 - {'path': '/', 'tag': '63719491-526f-4912-b7f5-f6a6d05ef137', 'options': 'recursive,force'} 2024-04-22T13:30:18.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:17 smithi012 ceph-mon[28265]: pgmap v188: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-22T13:30:18.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:17 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:30:18.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:17 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:30:18.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:17 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:30:18.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:17 smithi134 ceph-mon[31338]: pgmap v188: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-22T13:30:18.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:17 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:30:18.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:17 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:30:18.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:17 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:30:18.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:17 smithi111 ceph-mon[31361]: pgmap v188: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-22T13:30:18.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:17 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:30:18.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:17 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:30:18.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:17 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:30:19.078 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:18 smithi134 ceph-mon[31338]: scrub queued for path: / 2024-04-22T13:30:19.078 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:18 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:30:19.078 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:18 smithi134 ceph-mon[31338]: scrub summary: active paths [/] 2024-04-22T13:30:19.078 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:18 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:30:19.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:18 smithi012 ceph-mon[28265]: scrub queued for path: / 2024-04-22T13:30:19.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:18 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:30:19.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:18 smithi012 ceph-mon[28265]: scrub summary: active paths [/] 2024-04-22T13:30:19.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:18 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:30:19.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:18 smithi111 ceph-mon[31361]: scrub queued for path: / 2024-04-22T13:30:19.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:18 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:30:19.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:18 smithi111 ceph-mon[31361]: scrub summary: active paths [/] 2024-04-22T13:30:19.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:18 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:30:20.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:19 smithi012 ceph-mon[28265]: pgmap v189: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-22T13:30:20.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:19 smithi012 ceph-mon[28265]: scrub summary: idle 2024-04-22T13:30:20.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:19 smithi134 ceph-mon[31338]: pgmap v189: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-22T13:30:20.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:19 smithi134 ceph-mon[31338]: scrub summary: idle 2024-04-22T13:30:20.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:19 smithi111 ceph-mon[31361]: pgmap v189: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-22T13:30:20.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:19 smithi111 ceph-mon[31361]: scrub summary: idle 2024-04-22T13:30:22.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:21 smithi012 ceph-mon[28265]: pgmap v190: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T13:30:22.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:21 smithi134 ceph-mon[31338]: pgmap v190: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T13:30:22.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:21 smithi111 ceph-mon[31361]: pgmap v190: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-22T13:30:24.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:23 smithi012 ceph-mon[28265]: pgmap v191: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-22T13:30:24.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:23 smithi134 ceph-mon[31338]: pgmap v191: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-22T13:30:24.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:23 smithi111 ceph-mon[31361]: pgmap v191: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-22T13:30:26.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:25 smithi012 ceph-mon[28265]: pgmap v192: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:30:26.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:25 smithi134 ceph-mon[31338]: pgmap v192: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:30:26.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:25 smithi111 ceph-mon[31361]: pgmap v192: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:30:28.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:27 smithi012 ceph-mon[28265]: pgmap v193: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:30:28.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:27 smithi134 ceph-mon[31338]: pgmap v193: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:30:28.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:27 smithi111 ceph-mon[31361]: pgmap v193: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:30:30.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:29 smithi012 ceph-mon[28265]: pgmap v194: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:30:30.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:29 smithi134 ceph-mon[31338]: pgmap v194: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:30:30.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:29 smithi111 ceph-mon[31361]: pgmap v194: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:30:32.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:31 smithi012 ceph-mon[28265]: pgmap v195: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:32.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:31 smithi134 ceph-mon[31338]: pgmap v195: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:32.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:31 smithi111 ceph-mon[31361]: pgmap v195: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:34.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:33 smithi012 ceph-mon[28265]: pgmap v196: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:30:34.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:33 smithi134 ceph-mon[31338]: pgmap v196: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:30:34.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:33 smithi111 ceph-mon[31361]: pgmap v196: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:30:36.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:35 smithi012 ceph-mon[28265]: pgmap v197: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:36.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:35 smithi134 ceph-mon[31338]: pgmap v197: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:36.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:35 smithi111 ceph-mon[31361]: pgmap v197: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:38.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:37 smithi012 ceph-mon[28265]: pgmap v198: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:38.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:37 smithi134 ceph-mon[31338]: pgmap v198: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:38.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:37 smithi111 ceph-mon[31361]: pgmap v198: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:40.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:39 smithi012 ceph-mon[28265]: pgmap v199: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:30:40.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:39 smithi134 ceph-mon[31338]: pgmap v199: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:30:40.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:39 smithi111 ceph-mon[31361]: pgmap v199: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:30:42.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:41 smithi012 ceph-mon[28265]: pgmap v200: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:42.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:41 smithi134 ceph-mon[31338]: pgmap v200: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:42.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:41 smithi111 ceph-mon[31361]: pgmap v200: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:44.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:43 smithi012 ceph-mon[28265]: pgmap v201: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:30:44.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:43 smithi134 ceph-mon[31338]: pgmap v201: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:30:44.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:43 smithi111 ceph-mon[31361]: pgmap v201: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:30:46.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:45 smithi012 ceph-mon[28265]: pgmap v202: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:46.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:45 smithi134 ceph-mon[31338]: pgmap v202: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:46.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:45 smithi111 ceph-mon[31361]: pgmap v202: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:47.389 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:30:47.390 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:30:47.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.515+0000 7f71d17a4640 1 -- 172.21.15.12:0/3311132442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71cc1031c0 msgr2=0x7f71cc1035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:47.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.515+0000 7f71d17a4640 1 --2- 172.21.15.12:0/3311132442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71cc1031c0 0x7f71cc1035a0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009900 tx=0x7f71bc02bd60 comp rx=0 tx=0).stop 2024-04-22T13:30:47.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.515+0000 7f71d17a4640 1 -- 172.21.15.12:0/3311132442 shutdown_connections 2024-04-22T13:30:47.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.515+0000 7f71d17a4640 1 --2- 172.21.15.12:0/3311132442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71cc10c500 0x7f71cc10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.515+0000 7f71d17a4640 1 --2- 172.21.15.12:0/3311132442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71cc103ae0 0x7f71cc10bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.515+0000 7f71d17a4640 1 --2- 172.21.15.12:0/3311132442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71cc1031c0 0x7f71cc1035a0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.515+0000 7f71d17a4640 1 -- 172.21.15.12:0/3311132442 >> 172.21.15.12:0/3311132442 conn(0x7f71cc078000 msgr2=0x7f71cc078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:47.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.515+0000 7f71d17a4640 1 -- 172.21.15.12:0/3311132442 shutdown_connections 2024-04-22T13:30:47.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.515+0000 7f71d17a4640 1 -- 172.21.15.12:0/3311132442 wait complete. 2024-04-22T13:30:47.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.515+0000 7f71d17a4640 1 Processor -- start 2024-04-22T13:30:47.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.516+0000 7f71d17a4640 1 -- start start 2024-04-22T13:30:47.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.516+0000 7f71d17a4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71cc1031c0 0x7f71cc198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:47.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.516+0000 7f71d17a4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71cc103ae0 0x7f71cc198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:47.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.516+0000 7f71d17a4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71cc10c500 0x7f71cc19cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:47.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.516+0000 7f71d17a4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc112170 con 0x7f71cc103ae0 2024-04-22T13:30:47.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.516+0000 7f71d17a4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc111ff0 con 0x7f71cc10c500 2024-04-22T13:30:47.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.516+0000 7f71d17a4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1122f0 con 0x7f71cc1031c0 2024-04-22T13:30:47.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.516+0000 7f71ca7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71cc103ae0 0x7f71cc198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:47.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.516+0000 7f71ca7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71cc103ae0 0x7f71cc198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42220/0 (socket says 172.21.15.12:42220) 2024-04-22T13:30:47.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.516+0000 7f71caffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71cc1031c0 0x7f71cc198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:47.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.516+0000 7f71ca7fc640 1 -- 172.21.15.12:0/336352916 learned_addr learned my addr 172.21.15.12:0/336352916 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:47.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.516+0000 7f71cb7fe640 1 --2- 172.21.15.12:0/336352916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71cc10c500 0x7f71cc19cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:47.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.516+0000 7f71ca7fc640 1 -- 172.21.15.12:0/336352916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71cc1031c0 msgr2=0x7f71cc198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:47.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.516+0000 7f71ca7fc640 1 --2- 172.21.15.12:0/336352916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71cc1031c0 0x7f71cc198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.516+0000 7f71ca7fc640 1 -- 172.21.15.12:0/336352916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71cc10c500 msgr2=0x7f71cc19cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:47.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.516+0000 7f71ca7fc640 1 --2- 172.21.15.12:0/336352916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71cc10c500 0x7f71cc19cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.516+0000 7f71ca7fc640 1 -- 172.21.15.12:0/336352916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71cc19d6b0 con 0x7f71cc103ae0 2024-04-22T13:30:47.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.516+0000 7f71cb7fe640 1 --2- 172.21.15.12:0/336352916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71cc10c500 0x7f71cc19cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:47.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.516+0000 7f71ca7fc640 1 --2- 172.21.15.12:0/336352916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71cc103ae0 0x7f71cc198c50 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f71c000d950 tx=0x7f71c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:47.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.517+0000 7f71b3fff640 1 -- 172.21.15.12:0/336352916 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c0017070 con 0x7f71cc103ae0 2024-04-22T13:30:47.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.517+0000 7f71b3fff640 1 -- 172.21.15.12:0/336352916 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71c0009c30 con 0x7f71cc103ae0 2024-04-22T13:30:47.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.517+0000 7f71d17a4640 1 -- 172.21.15.12:0/336352916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71cc19d9a0 con 0x7f71cc103ae0 2024-04-22T13:30:47.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.517+0000 7f71b3fff640 1 -- 172.21.15.12:0/336352916 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c000f620 con 0x7f71cc103ae0 2024-04-22T13:30:47.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.517+0000 7f71d17a4640 1 -- 172.21.15.12:0/336352916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71cc1a5230 con 0x7f71cc103ae0 2024-04-22T13:30:47.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.519+0000 7f71b3fff640 1 -- 172.21.15.12:0/336352916 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f71c0010460 con 0x7f71cc103ae0 2024-04-22T13:30:47.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.519+0000 7f71d4394740 1 Processor -- start 2024-04-22T13:30:47.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.519+0000 7f71d4394740 1 -- start start 2024-04-22T13:30:47.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.520+0000 7f71d4394740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f490a07580 0x55f490a07960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:47.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.520+0000 7f71d4394740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f490a07ea0 0x55f490a10540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:47.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.520+0000 7f71d4394740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f490a10a80 0x55f490a12e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:47.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.520+0000 7f71d4394740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f49096c1e0 con 0x55f490a07580 2024-04-22T13:30:47.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.520+0000 7f71d4394740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f490909390 con 0x55f490a10a80 2024-04-22T13:30:47.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.520+0000 7f71d4394740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f4909681a0 con 0x55f490a07ea0 2024-04-22T13:30:47.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.520+0000 7f71b3fff640 1 --2- 172.21.15.12:0/336352916 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f71b40a3d20 0x7f71b40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:47.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.520+0000 7f71cb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f490a07580 0x55f490a07960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:47.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.520+0000 7f71cb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f490a07580 0x55f490a07960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42230/0 (socket says 172.21.15.12:42230) 2024-04-22T13:30:47.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.520+0000 7f71cb7fe640 1 -- 172.21.15.12:0/3132182884 learned_addr learned my addr 172.21.15.12:0/3132182884 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:47.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.520+0000 7f71cb7fe640 1 --2- 172.21.15.12:0/336352916 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f71b40a3d20 0x7f71b40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:47.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.520+0000 7f71b3fff640 1 -- 172.21.15.12:0/336352916 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f71c00d0410 con 0x7f71cc103ae0 2024-04-22T13:30:47.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.520+0000 7f71caffd640 1 --2- 172.21.15.12:0/3132182884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f490a07ea0 0x55f490a10540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:47.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.520+0000 7f71ca7fc640 1 --2- 172.21.15.12:0/3132182884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f490a10a80 0x55f490a12e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:47.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.520+0000 7f71cb7fe640 1 -- 172.21.15.12:0/3132182884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f490a07ea0 msgr2=0x55f490a10540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:47.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.520+0000 7f71cb7fe640 1 --2- 172.21.15.12:0/3132182884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f490a07ea0 0x55f490a10540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.520+0000 7f71cb7fe640 1 -- 172.21.15.12:0/3132182884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f490a10a80 msgr2=0x55f490a12e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:47.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.520+0000 7f71cb7fe640 1 --2- 172.21.15.12:0/3132182884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f490a10a80 0x55f490a12e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.520+0000 7f71cb7fe640 1 -- 172.21.15.12:0/3132182884 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4907487d0 con 0x55f490a07580 2024-04-22T13:30:47.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.520+0000 7f71cb7fe640 1 --2- 172.21.15.12:0/3132182884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f490a07580 0x55f490a07960 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f71c4012cf0 tx=0x7f71c4010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ffa5608f87ed99 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:47.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.520+0000 7f71cb7fe640 1 --2- 172.21.15.12:0/336352916 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f71b40a3d20 0x7f71b40a61e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f71c4010c40 tx=0x7f71c4017000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:47.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.521+0000 7f71b17fa640 1 -- 172.21.15.12:0/3132182884 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c4022070 con 0x55f490a07580 2024-04-22T13:30:47.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.521+0000 7f71b17fa640 1 -- 172.21.15.12:0/3132182884 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71c4009cb0 con 0x55f490a07580 2024-04-22T13:30:47.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.521+0000 7f71b17fa640 1 -- 172.21.15.12:0/3132182884 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c400cdb0 con 0x55f490a07580 2024-04-22T13:30:47.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.521+0000 7f71d4394740 1 -- 172.21.15.12:0/3132182884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f490a07580 msgr2=0x55f490a07960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:47.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.521+0000 7f71d4394740 1 --2- 172.21.15.12:0/3132182884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f490a07580 0x55f490a07960 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f71c4012cf0 tx=0x7f71c4010740 comp rx=0 tx=0).stop 2024-04-22T13:30:47.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.521+0000 7f71d4394740 1 -- 172.21.15.12:0/3132182884 shutdown_connections 2024-04-22T13:30:47.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.521+0000 7f71d4394740 1 --2- 172.21.15.12:0/3132182884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f490a10a80 0x55f490a12e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.521+0000 7f71d4394740 1 --2- 172.21.15.12:0/3132182884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f490a07ea0 0x55f490a10540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.521+0000 7f71d4394740 1 --2- 172.21.15.12:0/3132182884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f490a07580 0x55f490a07960 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.521+0000 7f71d4394740 1 -- 172.21.15.12:0/3132182884 >> 172.21.15.12:0/3132182884 conn(0x55f490a01cb0 msgr2=0x55f490a02090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:47.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.521+0000 7f71d4394740 1 -- 172.21.15.12:0/3132182884 shutdown_connections 2024-04-22T13:30:47.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.521+0000 7f71d4394740 1 -- 172.21.15.12:0/3132182884 wait complete. 2024-04-22T13:30:47.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.522+0000 7f71d4394740 1 -- start start 2024-04-22T13:30:47.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.522+0000 7f71d4394740 1 Processor -- start 2024-04-22T13:30:47.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.522+0000 7f71d4394740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f490a07580 0x55f490a0db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:47.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.522+0000 7f71d4394740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f490a07ea0 0x55f490a0e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:47.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.522+0000 7f71d4394740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f490a10a80 0x55f490a0e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:47.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.522+0000 7f71d4394740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f490a13810 con 0x55f490a10a80 2024-04-22T13:30:47.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.522+0000 7f71d4394740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f4909252f0 con 0x55f490a07580 2024-04-22T13:30:47.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.522+0000 7f71d4394740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f490a13990 con 0x55f490a07ea0 2024-04-22T13:30:47.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.522+0000 7f71d4394740 10 client.0 _ll_register_callbacks cb 0x55f490932580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:47.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.522+0000 7f71cb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f490a10a80 0x55f490a0e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:47.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.522+0000 7f71cb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f490a10a80 0x55f490a0e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42238/0 (socket says 172.21.15.12:42238) 2024-04-22T13:30:47.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.522+0000 7f71cb7fe640 1 -- 172.21.15.12:0/3661366109 learned_addr learned my addr 172.21.15.12:0/3661366109 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:47.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.522+0000 7f71caffd640 1 --2- 172.21.15.12:0/3661366109 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f490a07580 0x55f490a0db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:47.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.522+0000 7f71ca7fc640 1 --2- 172.21.15.12:0/3661366109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f490a07ea0 0x55f490a0e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:47.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.523+0000 7f71cb7fe640 1 -- 172.21.15.12:0/3661366109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f490a07ea0 msgr2=0x55f490a0e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:47.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.523+0000 7f71cb7fe640 1 --2- 172.21.15.12:0/3661366109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f490a07ea0 0x55f490a0e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.523+0000 7f71cb7fe640 1 -- 172.21.15.12:0/3661366109 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f490a07580 msgr2=0x55f490a0db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:47.526 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.523+0000 7f71cb7fe640 1 --2- 172.21.15.12:0/3661366109 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f490a07580 0x55f490a0db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.523+0000 7f71cb7fe640 1 -- 172.21.15.12:0/3661366109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f49067e3f0 con 0x55f490a10a80 2024-04-22T13:30:47.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.523+0000 7f71cb7fe640 1 --2- 172.21.15.12:0/3661366109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f490a10a80 0x55f490a0e5c0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f71c4012410 tx=0x7f71c4012ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580f980def527ef9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:47.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.523+0000 7f71aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:47.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.523+0000 7f71aeff5640 1 -- 172.21.15.12:0/3661366109 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c4022050 con 0x55f490a10a80 2024-04-22T13:30:47.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.523+0000 7f71aeff5640 1 -- 172.21.15.12:0/3661366109 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f71c400cd50 con 0x55f490a10a80 2024-04-22T13:30:47.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.523+0000 7f71d4394740 1 -- 172.21.15.12:0/3661366109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71c4009cb0 con 0x55f490a10a80 2024-04-22T13:30:47.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.523+0000 7f71aeff5640 1 -- 172.21.15.12:0/3661366109 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f71c4033020 con 0x55f490a10a80 2024-04-22T13:30:47.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.523+0000 7f71d4394740 10 client.18051 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:47.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.523+0000 7f71d4394740 1 -- 172.21.15.12:0/3661366109 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f49063e0f0 con 0x55f490a10a80 2024-04-22T13:30:47.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.523+0000 7f71aeff5640 1 -- 172.21.15.12:0/3661366109 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71c4009cb0 con 0x55f490a10a80 2024-04-22T13:30:47.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.523+0000 7f71aeff5640 1 -- 172.21.15.12:0/3661366109 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f71c4005540 con 0x55f490a10a80 2024-04-22T13:30:47.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.524+0000 7f71d4394740 10 client.18051 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:47.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.524+0000 7f71d4394740 10 client.18051 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:30:47.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.524+0000 7f71d4394740 20 client.18051 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:47.527 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.524+0000 7f71d4394740 1 --2- 172.21.15.12:0/3661366109 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f490a209f0 0x55f490a22de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:47.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.524+0000 7f71d4394740 4 client.18051 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:30:47.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.524+0000 7f71d4394740 1 -- 172.21.15.12:0/3661366109 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f490979c00 con 0x55f490a209f0 2024-04-22T13:30:47.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.524+0000 7f71caffd640 1 --2- 172.21.15.12:0/3661366109 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f490a209f0 0x55f490a22de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:47.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.524+0000 7f71caffd640 1 --2- 172.21.15.12:0/3661366109 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f490a209f0 0x55f490a22de0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9eb6d5e570816c7 server_cookie=8a562ef216f309b in_seq=0 out_seq=0 2024-04-22T13:30:47.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.524+0000 7f71aeff5640 10 client.18051 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:47.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.525+0000 7f71aeff5640 1 -- 172.21.15.12:0/3661366109 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f490979c00 con 0x55f490a209f0 2024-04-22T13:30:47.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.525+0000 7f71aeff5640 10 client.18051 handle_command_reply: tid=0 2024-04-22T13:30:47.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.525+0000 7f71d4394740 1 client.18051 shutdown 2024-04-22T13:30:47.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.525+0000 7f71d4394740 1 -- 172.21.15.12:0/3661366109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f490a10a80 msgr2=0x55f490a0e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:47.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.525+0000 7f71d4394740 1 --2- 172.21.15.12:0/3661366109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f490a10a80 0x55f490a0e5c0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f71c4012410 tx=0x7f71c4012ea0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.525+0000 7f71d4394740 1 -- 172.21.15.12:0/3661366109 shutdown_connections 2024-04-22T13:30:47.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.525+0000 7f71d4394740 1 --2- 172.21.15.12:0/3661366109 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f490a209f0 0x55f490a22de0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.525+0000 7f71d4394740 1 --2- 172.21.15.12:0/3661366109 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f490a10a80 0x55f490a0e5c0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.525+0000 7f71d4394740 1 --2- 172.21.15.12:0/3661366109 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f490a07ea0 0x55f490a0e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.525+0000 7f71aeff5640 0 client.18051 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:47.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.525+0000 7f71d4394740 1 --2- 172.21.15.12:0/3661366109 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f490a07580 0x55f490a0db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.525+0000 7f71d4394740 1 -- 172.21.15.12:0/3661366109 >> 172.21.15.12:0/3661366109 conn(0x55f490a01cb0 msgr2=0x55f490a03a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:47.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.525+0000 7f71d4394740 1 -- 172.21.15.12:0/3661366109 shutdown_connections 2024-04-22T13:30:47.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.525+0000 7f71d4394740 1 -- 172.21.15.12:0/3661366109 wait complete. 2024-04-22T13:30:47.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.525+0000 7f71d4394740 20 client.18051 trim_cache size 0 max 16384 2024-04-22T13:30:47.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.543+0000 7f71d4394740 1 Processor -- start 2024-04-22T13:30:47.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.543+0000 7f71d4394740 1 -- start start 2024-04-22T13:30:47.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.543+0000 7f71d4394740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f490a07580 0x55f490a10270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:47.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.543+0000 7f71d4394740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f490a07ea0 0x55f490a0e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:47.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.543+0000 7f71d4394740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f490a10a80 0x55f490a18fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:47.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.543+0000 7f71d4394740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1ada50 con 0x55f490a07ea0 2024-04-22T13:30:47.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.543+0000 7f71d4394740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1af590 con 0x55f490a07580 2024-04-22T13:30:47.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.543+0000 7f71d4394740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1ae120 con 0x55f490a10a80 2024-04-22T13:30:47.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.543+0000 7f71ca7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f490a07ea0 0x55f490a0e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:47.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.543+0000 7f71ca7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f490a07ea0 0x55f490a0e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42254/0 (socket says 172.21.15.12:42254) 2024-04-22T13:30:47.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.543+0000 7f71ca7fc640 1 -- 172.21.15.12:0/853960359 learned_addr learned my addr 172.21.15.12:0/853960359 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:47.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.544+0000 7f71caffd640 1 --2- 172.21.15.12:0/853960359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f490a07580 0x55f490a10270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:47.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.544+0000 7f71cb7fe640 1 --2- 172.21.15.12:0/853960359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f490a10a80 0x55f490a18fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:47.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.544+0000 7f71ca7fc640 1 -- 172.21.15.12:0/853960359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f490a10a80 msgr2=0x55f490a18fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:47.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.544+0000 7f71ca7fc640 1 --2- 172.21.15.12:0/853960359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f490a10a80 0x55f490a18fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.544+0000 7f71ca7fc640 1 -- 172.21.15.12:0/853960359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f490a07580 msgr2=0x55f490a10270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:47.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.544+0000 7f71ca7fc640 1 --2- 172.21.15.12:0/853960359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f490a07580 0x55f490a10270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.544+0000 7f71ca7fc640 1 -- 172.21.15.12:0/853960359 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f490a23150 con 0x55f490a07ea0 2024-04-22T13:30:47.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.544+0000 7f71caffd640 1 --2- 172.21.15.12:0/853960359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f490a07580 0x55f490a10270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:30:47.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.544+0000 7f71cb7fe640 1 --2- 172.21.15.12:0/853960359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f490a10a80 0x55f490a18fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:30:47.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.544+0000 7f71ca7fc640 1 --2- 172.21.15.12:0/853960359 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f490a07ea0 0x55f490a0e390 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f71c0098dd0 tx=0x7f71c009cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b703c375d679e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:47.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.544+0000 7f71b0ff9640 1 -- 172.21.15.12:0/853960359 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c00b4020 con 0x55f490a07ea0 2024-04-22T13:30:47.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.544+0000 7f71b0ff9640 1 -- 172.21.15.12:0/853960359 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71c009c400 con 0x55f490a07ea0 2024-04-22T13:30:47.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.544+0000 7f71b0ff9640 1 -- 172.21.15.12:0/853960359 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c0020040 con 0x55f490a07ea0 2024-04-22T13:30:47.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.545+0000 7f71d4394740 1 -- 172.21.15.12:0/853960359 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f490a07ea0 msgr2=0x55f490a0e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:47.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.545+0000 7f71d4394740 1 --2- 172.21.15.12:0/853960359 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f490a07ea0 0x55f490a0e390 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f71c0098dd0 tx=0x7f71c009cb00 comp rx=0 tx=0).stop 2024-04-22T13:30:47.547 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.545+0000 7f71d4394740 1 -- 172.21.15.12:0/853960359 shutdown_connections 2024-04-22T13:30:47.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.545+0000 7f71d4394740 1 --2- 172.21.15.12:0/853960359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f490a10a80 0x55f490a18fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.545+0000 7f71d4394740 1 --2- 172.21.15.12:0/853960359 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f490a07ea0 0x55f490a0e390 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.545+0000 7f71d4394740 1 --2- 172.21.15.12:0/853960359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f490a07580 0x55f490a10270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.545+0000 7f71d4394740 1 -- 172.21.15.12:0/853960359 >> 172.21.15.12:0/853960359 conn(0x55f490a01cb0 msgr2=0x55f490a02fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:47.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.545+0000 7f71d4394740 1 -- 172.21.15.12:0/853960359 shutdown_connections 2024-04-22T13:30:47.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.545+0000 7f71d4394740 1 -- 172.21.15.12:0/853960359 wait complete. 2024-04-22T13:30:47.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.545+0000 7f71d4394740 1 -- start start 2024-04-22T13:30:47.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.545+0000 7f71d4394740 1 Processor -- start 2024-04-22T13:30:47.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.545+0000 7f71d4394740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f490a07580 0x55f490a7efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:47.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.545+0000 7f71d4394740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f490a07ea0 0x55f490a7f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:47.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.545+0000 7f71d4394740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f490a10a80 0x55f490a83f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:47.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.545+0000 7f71d4394740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1aeec0 con 0x55f490a07580 2024-04-22T13:30:47.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.545+0000 7f71d4394740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1ae7f0 con 0x55f490a10a80 2024-04-22T13:30:47.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.545+0000 7f71d4394740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc1aba30 con 0x55f490a07ea0 2024-04-22T13:30:47.548 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.545+0000 7f71d4394740 10 client.0 _ll_register_callbacks cb 0x7f71c000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:47.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.545+0000 7f71caffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f490a07580 0x55f490a7efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:47.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.545+0000 7f71caffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f490a07580 0x55f490a7efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42266/0 (socket says 172.21.15.12:42266) 2024-04-22T13:30:47.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.545+0000 7f71caffd640 1 -- 172.21.15.12:0/962879958 learned_addr learned my addr 172.21.15.12:0/962879958 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:47.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.546+0000 7f71cb7fe640 1 --2- 172.21.15.12:0/962879958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f490a10a80 0x55f490a83f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:47.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.546+0000 7f71ca7fc640 1 --2- 172.21.15.12:0/962879958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f490a07ea0 0x55f490a7f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:47.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.546+0000 7f71caffd640 1 -- 172.21.15.12:0/962879958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f490a07ea0 msgr2=0x55f490a7f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:47.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.546+0000 7f71caffd640 1 --2- 172.21.15.12:0/962879958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f490a07ea0 0x55f490a7f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.546+0000 7f71caffd640 1 -- 172.21.15.12:0/962879958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f490a10a80 msgr2=0x55f490a83f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:47.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.546+0000 7f71caffd640 1 --2- 172.21.15.12:0/962879958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f490a10a80 0x55f490a83f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.546+0000 7f71caffd640 1 -- 172.21.15.12:0/962879958 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f490a07cd0 con 0x55f490a07580 2024-04-22T13:30:47.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.546+0000 7f71caffd640 1 --2- 172.21.15.12:0/962879958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f490a07580 0x55f490a7efe0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f71bc02c2a0 tx=0x7f71bc0044a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fead0edfaca6c41f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:47.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.546+0000 7f71aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:47.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.546+0000 7f71aeff5640 1 -- 172.21.15.12:0/962879958 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71bc04f060 con 0x55f490a07580 2024-04-22T13:30:47.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.546+0000 7f71aeff5640 1 -- 172.21.15.12:0/962879958 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f71bc03a040 con 0x55f490a07580 2024-04-22T13:30:47.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.546+0000 7f71d4394740 1 -- 172.21.15.12:0/962879958 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71c009c400 con 0x55f490a07580 2024-04-22T13:30:47.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.546+0000 7f71aeff5640 1 -- 172.21.15.12:0/962879958 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f71bc02cb20 con 0x55f490a07580 2024-04-22T13:30:47.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.546+0000 7f71d4394740 10 client.18057 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:47.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.546+0000 7f71d4394740 1 -- 172.21.15.12:0/962879958 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f490a8b670 con 0x55f490a07580 2024-04-22T13:30:47.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.546+0000 7f71aeff5640 1 -- 172.21.15.12:0/962879958 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71c009c400 con 0x55f490a07580 2024-04-22T13:30:47.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.547+0000 7f71aeff5640 1 -- 172.21.15.12:0/962879958 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f71bc049070 con 0x55f490a07580 2024-04-22T13:30:47.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.547+0000 7f71d4394740 10 client.18057 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:47.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.547+0000 7f71d4394740 10 client.18057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:30:47.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.547+0000 7f71d4394740 20 client.18057 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:47.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.547+0000 7f71d4394740 1 --2- 172.21.15.12:0/962879958 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f490a8bda0 0x55f490a8e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:47.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.547+0000 7f71d4394740 4 client.18057 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:30:47.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.547+0000 7f71d4394740 1 -- 172.21.15.12:0/962879958 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f490973a70 con 0x55f490a8bda0 2024-04-22T13:30:47.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.547+0000 7f71ca7fc640 1 --2- 172.21.15.12:0/962879958 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f490a8bda0 0x55f490a8e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:47.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.547+0000 7f71aeff5640 10 client.18057 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:47.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.547+0000 7f71ca7fc640 1 --2- 172.21.15.12:0/962879958 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f490a8bda0 0x55f490a8e1c0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9edfbc7d6a232349 server_cookie=266b95742895c43 in_seq=0 out_seq=0 2024-04-22T13:30:47.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.547+0000 7f71aeff5640 1 -- 172.21.15.12:0/962879958 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f490973a70 con 0x55f490a8bda0 2024-04-22T13:30:47.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.547+0000 7f71aeff5640 10 client.18057 handle_command_reply: tid=0 2024-04-22T13:30:47.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.548+0000 7f71d4394740 1 client.18057 shutdown 2024-04-22T13:30:47.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.548+0000 7f71d4394740 1 -- 172.21.15.12:0/962879958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f490a07580 msgr2=0x55f490a7efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:47.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.548+0000 7f71d4394740 1 --2- 172.21.15.12:0/962879958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f490a07580 0x55f490a7efe0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f71bc02c2a0 tx=0x7f71bc0044a0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.548+0000 7f71d4394740 1 -- 172.21.15.12:0/962879958 shutdown_connections 2024-04-22T13:30:47.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.548+0000 7f71d4394740 1 --2- 172.21.15.12:0/962879958 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55f490a8bda0 0x55f490a8e1c0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.548+0000 7f71d4394740 1 --2- 172.21.15.12:0/962879958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f490a10a80 0x55f490a83f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.548+0000 7f71aeff5640 0 client.18057 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:47.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.548+0000 7f71d4394740 1 --2- 172.21.15.12:0/962879958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f490a07ea0 0x55f490a7f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.548+0000 7f71d4394740 1 --2- 172.21.15.12:0/962879958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f490a07580 0x55f490a7efe0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.548+0000 7f71d4394740 1 -- 172.21.15.12:0/962879958 >> 172.21.15.12:0/962879958 conn(0x55f490a01cb0 msgr2=0x55f490a0f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:47.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.548+0000 7f71d4394740 1 -- 172.21.15.12:0/962879958 shutdown_connections 2024-04-22T13:30:47.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.548+0000 7f71d4394740 1 -- 172.21.15.12:0/962879958 wait complete. 2024-04-22T13:30:47.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.548+0000 7f71d4394740 20 client.18057 trim_cache size 0 max 16384 2024-04-22T13:30:47.552 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:30:47.552 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-04-22T13:30:47.552 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-04-22T13:30:47.552 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:30:47.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.548+0000 7f71d17a4640 1 -- 172.21.15.12:0/336352916 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f71b40a3d20 msgr2=0x7f71b40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:47.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.548+0000 7f71d17a4640 1 --2- 172.21.15.12:0/336352916 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f71b40a3d20 0x7f71b40a61e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f71c4010c40 tx=0x7f71c4017000 comp rx=0 tx=0).stop 2024-04-22T13:30:47.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.548+0000 7f71d17a4640 1 -- 172.21.15.12:0/336352916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71cc103ae0 msgr2=0x7f71cc198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:47.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.548+0000 7f71d17a4640 1 --2- 172.21.15.12:0/336352916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71cc103ae0 0x7f71cc198c50 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f71c000d950 tx=0x7f71c000de10 comp rx=0 tx=0).stop 2024-04-22T13:30:47.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.548+0000 7f71d17a4640 1 -- 172.21.15.12:0/336352916 shutdown_connections 2024-04-22T13:30:47.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.548+0000 7f71d17a4640 1 --2- 172.21.15.12:0/336352916 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f71b40a3d20 0x7f71b40a61e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.548+0000 7f71d17a4640 1 --2- 172.21.15.12:0/336352916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71cc10c500 0x7f71cc19cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.548+0000 7f71d17a4640 1 --2- 172.21.15.12:0/336352916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71cc103ae0 0x7f71cc198c50 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.548+0000 7f71d17a4640 1 --2- 172.21.15.12:0/336352916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71cc1031c0 0x7f71cc198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.548+0000 7f71d17a4640 1 -- 172.21.15.12:0/336352916 >> 172.21.15.12:0/336352916 conn(0x7f71cc078000 msgr2=0x7f71cc0fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:47.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.548+0000 7f71d17a4640 1 -- 172.21.15.12:0/336352916 shutdown_connections 2024-04-22T13:30:47.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.548+0000 7f71d17a4640 1 -- 172.21.15.12:0/336352916 wait complete. 2024-04-22T13:30:47.555 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:30:47.555 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:30:47.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.662+0000 7f3bd84e0640 1 -- 172.21.15.12:0/3614841968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3bd0101b60 msgr2=0x7f3bd010dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:47.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.662+0000 7f3bd84e0640 1 --2- 172.21.15.12:0/3614841968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3bd0101b60 0x7f3bd010dc90 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8009cd0 tx=0x7f3bc802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.662+0000 7f3bd84e0640 1 -- 172.21.15.12:0/3614841968 shutdown_connections 2024-04-22T13:30:47.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.662+0000 7f3bd84e0640 1 --2- 172.21.15.12:0/3614841968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3bd0101b60 0x7f3bd010dc90 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.662+0000 7f3bd84e0640 1 --2- 172.21.15.12:0/3614841968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bd01011a0 0x7f3bd0101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.662+0000 7f3bd84e0640 1 --2- 172.21.15.12:0/3614841968 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bd0105e30 0x7f3bd0106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.662+0000 7f3bd84e0640 1 -- 172.21.15.12:0/3614841968 >> 172.21.15.12:0/3614841968 conn(0x7f3bd0077e30 msgr2=0x7f3bd0078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:47.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.662+0000 7f3bd84e0640 1 -- 172.21.15.12:0/3614841968 shutdown_connections 2024-04-22T13:30:47.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.662+0000 7f3bd84e0640 1 -- 172.21.15.12:0/3614841968 wait complete. 2024-04-22T13:30:47.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.663+0000 7f3bd84e0640 1 Processor -- start 2024-04-22T13:30:47.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.663+0000 7f3bd84e0640 1 -- start start 2024-04-22T13:30:47.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.663+0000 7f3bd84e0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bd01011a0 0x7f3bd01995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:47.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.663+0000 7f3bd84e0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bd0101b60 0x7f3bd0199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:47.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.663+0000 7f3bd84e0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3bd0105e30 0x7f3bd01a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:47.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.663+0000 7f3bd84e0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd01110e0 con 0x7f3bd0105e30 2024-04-22T13:30:47.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.663+0000 7f3bd84e0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd0110f60 con 0x7f3bd0101b60 2024-04-22T13:30:47.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.663+0000 7f3bd84e0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd0111260 con 0x7f3bd01011a0 2024-04-22T13:30:47.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.663+0000 7f3bd6a7d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3bd0105e30 0x7f3bd01a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:47.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.663+0000 7f3bd6a7d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3bd0105e30 0x7f3bd01a4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42296/0 (socket says 172.21.15.12:42296) 2024-04-22T13:30:47.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.663+0000 7f3bd6a7d640 1 -- 172.21.15.12:0/2965967113 learned_addr learned my addr 172.21.15.12:0/2965967113 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:47.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.663+0000 7f3bd5a7b640 1 --2- 172.21.15.12:0/2965967113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bd0101b60 0x7f3bd0199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:47.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.663+0000 7f3bd627c640 1 --2- 172.21.15.12:0/2965967113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bd01011a0 0x7f3bd01995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:47.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.663+0000 7f3bd6a7d640 1 -- 172.21.15.12:0/2965967113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bd01011a0 msgr2=0x7f3bd01995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:47.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.663+0000 7f3bd6a7d640 1 --2- 172.21.15.12:0/2965967113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bd01011a0 0x7f3bd01995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.663+0000 7f3bd6a7d640 1 -- 172.21.15.12:0/2965967113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bd0101b60 msgr2=0x7f3bd0199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:47.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.663+0000 7f3bd6a7d640 1 --2- 172.21.15.12:0/2965967113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bd0101b60 0x7f3bd0199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.663+0000 7f3bd6a7d640 1 -- 172.21.15.12:0/2965967113 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bd01a4b60 con 0x7f3bd0105e30 2024-04-22T13:30:47.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.663+0000 7f3bd5a7b640 1 --2- 172.21.15.12:0/2965967113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bd0101b60 0x7f3bd0199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:47.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.663+0000 7f3bd6a7d640 1 --2- 172.21.15.12:0/2965967113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3bd0105e30 0x7f3bd01a4620 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8004af0 tx=0x7f3bc8034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:47.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.664+0000 7f3bc77fe640 1 -- 172.21.15.12:0/2965967113 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8041020 con 0x7f3bd0105e30 2024-04-22T13:30:47.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.664+0000 7f3bc77fe640 1 -- 172.21.15.12:0/2965967113 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bc8034ca0 con 0x7f3bd0105e30 2024-04-22T13:30:47.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.664+0000 7f3bd84e0640 1 -- 172.21.15.12:0/2965967113 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bd01a4d30 con 0x7f3bd0105e30 2024-04-22T13:30:47.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.664+0000 7f3bc77fe640 1 -- 172.21.15.12:0/2965967113 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8039040 con 0x7f3bd0105e30 2024-04-22T13:30:47.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.664+0000 7f3bd84e0640 1 -- 172.21.15.12:0/2965967113 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bd01a5130 con 0x7f3bd0105e30 2024-04-22T13:30:47.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.665+0000 7f3bd84e0640 1 -- 172.21.15.12:0/2965967113 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bd0068de0 con 0x7f3bd0105e30 2024-04-22T13:30:47.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.666+0000 7f3bc77fe640 1 -- 172.21.15.12:0/2965967113 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3bc8044020 con 0x7f3bd0105e30 2024-04-22T13:30:47.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.666+0000 7f3bc77fe640 1 --2- 172.21.15.12:0/2965967113 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3bbc0a3d20 0x7f3bbc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:47.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.667+0000 7f3bd627c640 1 --2- 172.21.15.12:0/2965967113 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3bbc0a3d20 0x7f3bbc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:47.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.667+0000 7f3bc77fe640 1 -- 172.21.15.12:0/2965967113 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f3bc80f2540 con 0x7f3bd0105e30 2024-04-22T13:30:47.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.667+0000 7f3bd627c640 1 --2- 172.21.15.12:0/2965967113 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3bbc0a3d20 0x7f3bbc0a61e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc005fd0 tx=0x7f3bcc00b510 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:47.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.672+0000 7f3bc77fe640 1 -- 172.21.15.12:0/2965967113 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3bc804b020 con 0x7f3bd0105e30 2024-04-22T13:30:47.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.898+0000 7f3bd84e0640 1 -- 172.21.15.12:0/2965967113 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3bd0109350 con 0x7f3bd0105e30 2024-04-22T13:30:47.894 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.899+0000 7f3bc77fe640 1 -- 172.21.15.12:0/2965967113 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7f3bc80ba030 con 0x7f3bd0105e30 2024-04-22T13:30:47.895 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:30:47.895 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:30:47.895 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:30:47.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.901+0000 7f3bd84e0640 1 -- 172.21.15.12:0/2965967113 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3bbc0a3d20 msgr2=0x7f3bbc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:47.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.901+0000 7f3bd84e0640 1 --2- 172.21.15.12:0/2965967113 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3bbc0a3d20 0x7f3bbc0a61e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc005fd0 tx=0x7f3bcc00b510 comp rx=0 tx=0).stop 2024-04-22T13:30:47.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.901+0000 7f3bd84e0640 1 -- 172.21.15.12:0/2965967113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3bd0105e30 msgr2=0x7f3bd01a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:47.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.901+0000 7f3bd84e0640 1 --2- 172.21.15.12:0/2965967113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3bd0105e30 0x7f3bd01a4620 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8004af0 tx=0x7f3bc8034670 comp rx=0 tx=0).stop 2024-04-22T13:30:47.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.901+0000 7f3bd84e0640 1 -- 172.21.15.12:0/2965967113 shutdown_connections 2024-04-22T13:30:47.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.901+0000 7f3bd84e0640 1 --2- 172.21.15.12:0/2965967113 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3bbc0a3d20 0x7f3bbc0a61e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.901+0000 7f3bd84e0640 1 --2- 172.21.15.12:0/2965967113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3bd0105e30 0x7f3bd01a4620 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.901+0000 7f3bd84e0640 1 --2- 172.21.15.12:0/2965967113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3bd0101b60 0x7f3bd0199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.901+0000 7f3bd84e0640 1 --2- 172.21.15.12:0/2965967113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3bd01011a0 0x7f3bd01995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:47.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.901+0000 7f3bd84e0640 1 -- 172.21.15.12:0/2965967113 >> 172.21.15.12:0/2965967113 conn(0x7f3bd0077e30 msgr2=0x7f3bd00fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:47.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.901+0000 7f3bd84e0640 1 -- 172.21.15.12:0/2965967113 shutdown_connections 2024-04-22T13:30:47.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:47.901+0000 7f3bd84e0640 1 -- 172.21.15.12:0/2965967113 wait complete. 2024-04-22T13:30:47.908 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:30:48.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.016+0000 7fd46c1a4640 1 -- 172.21.15.12:0/3105161395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd464101a40 msgr2=0x7fd464105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.016+0000 7fd46c1a4640 1 --2- 172.21.15.12:0/3105161395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd464101a40 0x7fd464105a80 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fd454009900 tx=0x7fd45402bc30 comp rx=0 tx=0).stop 2024-04-22T13:30:48.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.016+0000 7fd46c1a4640 1 -- 172.21.15.12:0/3105161395 shutdown_connections 2024-04-22T13:30:48.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.017+0000 7fd46c1a4640 1 --2- 172.21.15.12:0/3105161395 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd464106160 0x7fd46410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.017+0000 7fd46c1a4640 1 --2- 172.21.15.12:0/3105161395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd464101a40 0x7fd464105a80 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.017+0000 7fd46c1a4640 1 --2- 172.21.15.12:0/3105161395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd464101090 0x7fd464101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.017+0000 7fd46c1a4640 1 -- 172.21.15.12:0/3105161395 >> 172.21.15.12:0/3105161395 conn(0x7fd464077f50 msgr2=0x7fd464078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:48.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.017+0000 7fd46c1a4640 1 -- 172.21.15.12:0/3105161395 shutdown_connections 2024-04-22T13:30:48.014 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.017+0000 7fd46c1a4640 1 -- 172.21.15.12:0/3105161395 wait complete. 2024-04-22T13:30:48.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.017+0000 7fd46c1a4640 1 Processor -- start 2024-04-22T13:30:48.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.017+0000 7fd46c1a4640 1 -- start start 2024-04-22T13:30:48.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.017+0000 7fd46c1a4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd464101090 0x7fd4641986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.017+0000 7fd46c1a4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd464106160 0x7fd464198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.017+0000 7fd46c1a4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd46419cfa0 0x7fd46419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.017+0000 7fd46c1a4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd4641121a0 con 0x7fd46419cfa0 2024-04-22T13:30:48.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.017+0000 7fd46c1a4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd464112020 con 0x7fd464101090 2024-04-22T13:30:48.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.017+0000 7fd46c1a4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd464112320 con 0x7fd464106160 2024-04-22T13:30:48.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.017+0000 7fd46a741640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd46419cfa0 0x7fd46419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.018+0000 7fd46a741640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd46419cfa0 0x7fd46419d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42320/0 (socket says 172.21.15.12:42320) 2024-04-22T13:30:48.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.018+0000 7fd46a741640 1 -- 172.21.15.12:0/149760952 learned_addr learned my addr 172.21.15.12:0/149760952 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:48.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.018+0000 7fd469f40640 1 --2- 172.21.15.12:0/149760952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd464101090 0x7fd4641986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.018+0000 7fd46973f640 1 --2- 172.21.15.12:0/149760952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd464106160 0x7fd464198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.018+0000 7fd46a741640 1 -- 172.21.15.12:0/149760952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd464106160 msgr2=0x7fd464198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.018+0000 7fd46a741640 1 --2- 172.21.15.12:0/149760952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd464106160 0x7fd464198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.018+0000 7fd46a741640 1 -- 172.21.15.12:0/149760952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd464101090 msgr2=0x7fd4641986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.018+0000 7fd46a741640 1 --2- 172.21.15.12:0/149760952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd464101090 0x7fd4641986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.018+0000 7fd46a741640 1 -- 172.21.15.12:0/149760952 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd46419da90 con 0x7fd46419cfa0 2024-04-22T13:30:48.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.018+0000 7fd46a741640 1 --2- 172.21.15.12:0/149760952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd46419cfa0 0x7fd46419d450 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fd45c00cad0 tx=0x7fd45c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:48.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.018+0000 7fd45affd640 1 -- 172.21.15.12:0/149760952 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c00ae20 con 0x7fd46419cfa0 2024-04-22T13:30:48.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.018+0000 7fd45affd640 1 -- 172.21.15.12:0/149760952 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd45c005020 con 0x7fd46419cfa0 2024-04-22T13:30:48.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.018+0000 7fd46c1a4640 1 -- 172.21.15.12:0/149760952 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd46419dd80 con 0x7fd46419cfa0 2024-04-22T13:30:48.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.018+0000 7fd45affd640 1 -- 172.21.15.12:0/149760952 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c00f6b0 con 0x7fd46419cfa0 2024-04-22T13:30:48.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.018+0000 7fd46c1a4640 1 -- 172.21.15.12:0/149760952 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4641a54f0 con 0x7fd46419cfa0 2024-04-22T13:30:48.023 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.021+0000 7fd45affd640 1 -- 172.21.15.12:0/149760952 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd45c0051d0 con 0x7fd46419cfa0 2024-04-22T13:30:48.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.021+0000 7fd46ed93740 1 Processor -- start 2024-04-22T13:30:48.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.021+0000 7fd46ed93740 1 -- start start 2024-04-22T13:30:48.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.021+0000 7fd46ed93740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578628b84d0 0x5578628b88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.021+0000 7fd46ed93740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578628b8e80 0x5578628c1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.021+0000 7fd46ed93740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578628c1a00 0x5578628c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.021+0000 7fd46ed93740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55786281d1f0 con 0x5578628c1a00 2024-04-22T13:30:48.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.021+0000 7fd46ed93740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5578627ba4b0 con 0x5578628b8e80 2024-04-22T13:30:48.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.021+0000 7fd46ed93740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5578628191b0 con 0x5578628b84d0 2024-04-22T13:30:48.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.021+0000 7fd46973f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578628c1a00 0x5578628c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.024 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.021+0000 7fd46973f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578628c1a00 0x5578628c3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42322/0 (socket says 172.21.15.12:42322) 2024-04-22T13:30:48.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.021+0000 7fd46973f640 1 -- 172.21.15.12:0/1345837011 learned_addr learned my addr 172.21.15.12:0/1345837011 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:48.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.021+0000 7fd46973f640 1 --2- 172.21.15.12:0/1345837011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578628b84d0 0x5578628b88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.022+0000 7fd469f40640 1 --2- 172.21.15.12:0/1345837011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578628b8e80 0x5578628c1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.022+0000 7fd45affd640 1 --2- 172.21.15.12:0/149760952 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd4500a3d20 0x7fd4500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.022+0000 7fd46a741640 1 --2- 172.21.15.12:0/149760952 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd4500a3d20 0x7fd4500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.022+0000 7fd46973f640 1 -- 172.21.15.12:0/1345837011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578628b84d0 msgr2=0x5578628b88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.022+0000 7fd46973f640 1 --2- 172.21.15.12:0/1345837011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578628b84d0 0x5578628b88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.022+0000 7fd46973f640 1 -- 172.21.15.12:0/1345837011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578628b8e80 msgr2=0x5578628c1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.022+0000 7fd46973f640 1 --2- 172.21.15.12:0/1345837011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578628b8e80 0x5578628c1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.022+0000 7fd46973f640 1 -- 172.21.15.12:0/1345837011 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578627ac1b0 con 0x5578628c1a00 2024-04-22T13:30:48.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.022+0000 7fd45affd640 1 -- 172.21.15.12:0/149760952 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fd45c0d08c0 con 0x7fd46419cfa0 2024-04-22T13:30:48.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.022+0000 7fd46973f640 1 --2- 172.21.15.12:0/1345837011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578628c1a00 0x5578628c3e50 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fd454005ec0 tx=0x7fd454005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e563bc51d835e17d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:48.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.022+0000 7fd44ffff640 1 -- 172.21.15.12:0/1345837011 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd454049070 con 0x5578628c1a00 2024-04-22T13:30:48.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.022+0000 7fd46a741640 1 --2- 172.21.15.12:0/149760952 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd4500a3d20 0x7fd4500a61e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0ce110 tx=0x7fd45c087040 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:48.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.022+0000 7fd44ffff640 1 -- 172.21.15.12:0/1345837011 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd45402cb20 con 0x5578628c1a00 2024-04-22T13:30:48.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.022+0000 7fd44ffff640 1 -- 172.21.15.12:0/1345837011 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4540436b0 con 0x5578628c1a00 2024-04-22T13:30:48.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.023+0000 7fd46ed93740 1 -- 172.21.15.12:0/1345837011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578628c1a00 msgr2=0x5578628c3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.023+0000 7fd46ed93740 1 --2- 172.21.15.12:0/1345837011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578628c1a00 0x5578628c3e50 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fd454005ec0 tx=0x7fd454005e30 comp rx=0 tx=0).stop 2024-04-22T13:30:48.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.023+0000 7fd46ed93740 1 -- 172.21.15.12:0/1345837011 shutdown_connections 2024-04-22T13:30:48.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.023+0000 7fd46ed93740 1 --2- 172.21.15.12:0/1345837011 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578628c1a00 0x5578628c3e50 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.023+0000 7fd46ed93740 1 --2- 172.21.15.12:0/1345837011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578628b8e80 0x5578628c1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.023+0000 7fd46ed93740 1 --2- 172.21.15.12:0/1345837011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578628b84d0 0x5578628b88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.023+0000 7fd46ed93740 1 -- 172.21.15.12:0/1345837011 >> 172.21.15.12:0/1345837011 conn(0x5578628b2ba0 msgr2=0x5578628b2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:48.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.023+0000 7fd46ed93740 1 -- 172.21.15.12:0/1345837011 shutdown_connections 2024-04-22T13:30:48.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.023+0000 7fd46ed93740 1 -- 172.21.15.12:0/1345837011 wait complete. 2024-04-22T13:30:48.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.023+0000 7fd46ed93740 1 -- start start 2024-04-22T13:30:48.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.024+0000 7fd46ed93740 1 Processor -- start 2024-04-22T13:30:48.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.024+0000 7fd46ed93740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578628b84d0 0x5578628c1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.024+0000 7fd46ed93740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578628b8e80 0x5578628be6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.024+0000 7fd46ed93740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578628c1a00 0x5578628bed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.024+0000 7fd46ed93740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5578628c4910 con 0x5578628b84d0 2024-04-22T13:30:48.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.024+0000 7fd46ed93740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5578627d6500 con 0x5578628b8e80 2024-04-22T13:30:48.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.024+0000 7fd46ed93740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5578628c4a90 con 0x5578628c1a00 2024-04-22T13:30:48.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.024+0000 7fd46ed93740 10 client.0 _ll_register_callbacks cb 0x5578627a4b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:48.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.024+0000 7fd46973f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578628b84d0 0x5578628c1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.024+0000 7fd46973f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578628b84d0 0x5578628c1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42336/0 (socket says 172.21.15.12:42336) 2024-04-22T13:30:48.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.024+0000 7fd46973f640 1 -- 172.21.15.12:0/3881458233 learned_addr learned my addr 172.21.15.12:0/3881458233 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:48.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.024+0000 7fd469f40640 1 --2- 172.21.15.12:0/3881458233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578628b8e80 0x5578628be6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.024+0000 7fd46973f640 1 --2- 172.21.15.12:0/3881458233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578628c1a00 0x5578628bed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.024+0000 7fd46973f640 1 -- 172.21.15.12:0/3881458233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578628c1a00 msgr2=0x5578628bed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.024+0000 7fd46973f640 1 --2- 172.21.15.12:0/3881458233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578628c1a00 0x5578628bed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.024+0000 7fd46973f640 1 -- 172.21.15.12:0/3881458233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578628b8e80 msgr2=0x5578628be6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.024+0000 7fd46973f640 1 --2- 172.21.15.12:0/3881458233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578628b8e80 0x5578628be6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.024+0000 7fd46973f640 1 -- 172.21.15.12:0/3881458233 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55786252f3f0 con 0x5578628b84d0 2024-04-22T13:30:48.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.025+0000 7fd46973f640 1 --2- 172.21.15.12:0/3881458233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578628b84d0 0x5578628c1360 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fd4540096c0 tx=0x7fd45402c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49153e62ff44c9a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:48.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.025+0000 7fd44d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:48.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.025+0000 7fd44d7fa640 1 -- 172.21.15.12:0/3881458233 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd454049050 con 0x5578628b84d0 2024-04-22T13:30:48.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.025+0000 7fd44d7fa640 1 -- 172.21.15.12:0/3881458233 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fd454005020 con 0x5578628b84d0 2024-04-22T13:30:48.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.025+0000 7fd46ed93740 1 -- 172.21.15.12:0/3881458233 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd45402cb20 con 0x5578628b84d0 2024-04-22T13:30:48.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.025+0000 7fd44d7fa640 1 -- 172.21.15.12:0/3881458233 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd454005220 con 0x5578628b84d0 2024-04-22T13:30:48.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.025+0000 7fd46ed93740 10 client.18075 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:48.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.025+0000 7fd46ed93740 1 -- 172.21.15.12:0/3881458233 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5578625f97d0 con 0x5578628b84d0 2024-04-22T13:30:48.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.025+0000 7fd44d7fa640 1 -- 172.21.15.12:0/3881458233 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd45402cb20 con 0x5578628b84d0 2024-04-22T13:30:48.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.025+0000 7fd44d7fa640 1 -- 172.21.15.12:0/3881458233 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd454040040 con 0x5578628b84d0 2024-04-22T13:30:48.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.025+0000 7fd46ed93740 10 client.18075 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:48.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.025+0000 7fd46ed93740 10 client.18075 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:30:48.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.025+0000 7fd46ed93740 20 client.18075 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:48.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.025+0000 7fd46ed93740 1 --2- 172.21.15.12:0/3881458233 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5578628d1a80 0x5578628d3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.025+0000 7fd46ed93740 4 client.18075 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:30:48.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.025+0000 7fd46ed93740 1 -- 172.21.15.12:0/3881458233 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578627bc930 con 0x5578628d1a80 2024-04-22T13:30:48.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.026+0000 7fd469f40640 1 --2- 172.21.15.12:0/3881458233 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5578628d1a80 0x5578628d3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.026+0000 7fd44d7fa640 10 client.18075 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:30:48.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.026+0000 7fd469f40640 1 --2- 172.21.15.12:0/3881458233 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5578628d1a80 0x5578628d3e70 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ee7030ccd56ac8f9 server_cookie=859cf5a671158a3a in_seq=0 out_seq=0 2024-04-22T13:30:48.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.027+0000 7fd44d7fa640 1 -- 172.21.15.12:0/3881458233 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5578627bc930 con 0x5578628d1a80 2024-04-22T13:30:48.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.027+0000 7fd44d7fa640 10 client.18075 handle_command_reply: tid=0 2024-04-22T13:30:48.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.027+0000 7fd46ed93740 1 client.18075 shutdown 2024-04-22T13:30:48.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.027+0000 7fd46ed93740 1 -- 172.21.15.12:0/3881458233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578628b84d0 msgr2=0x5578628c1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.027+0000 7fd46ed93740 1 --2- 172.21.15.12:0/3881458233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578628b84d0 0x5578628c1360 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fd4540096c0 tx=0x7fd45402c5d0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.027+0000 7fd46ed93740 1 -- 172.21.15.12:0/3881458233 shutdown_connections 2024-04-22T13:30:48.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.027+0000 7fd46ed93740 1 --2- 172.21.15.12:0/3881458233 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5578628d1a80 0x5578628d3e70 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.027+0000 7fd46ed93740 1 --2- 172.21.15.12:0/3881458233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578628c1a00 0x5578628bed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.027+0000 7fd46ed93740 1 --2- 172.21.15.12:0/3881458233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578628b8e80 0x5578628be6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.027+0000 7fd44d7fa640 0 client.18075 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:30:48.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.027+0000 7fd46ed93740 1 --2- 172.21.15.12:0/3881458233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578628b84d0 0x5578628c1360 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.027+0000 7fd46ed93740 1 -- 172.21.15.12:0/3881458233 >> 172.21.15.12:0/3881458233 conn(0x5578628b2ba0 msgr2=0x5578628b48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:48.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.027+0000 7fd46ed93740 1 -- 172.21.15.12:0/3881458233 shutdown_connections 2024-04-22T13:30:48.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.027+0000 7fd46ed93740 1 -- 172.21.15.12:0/3881458233 wait complete. 2024-04-22T13:30:48.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.027+0000 7fd46ed93740 20 client.18075 trim_cache size 0 max 16384 2024-04-22T13:30:48.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.040+0000 7fd46ed93740 1 Processor -- start 2024-04-22T13:30:48.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.040+0000 7fd46ed93740 1 -- start start 2024-04-22T13:30:48.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.041+0000 7fd46ed93740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578628b84d0 0x5578628c0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.041+0000 7fd46ed93740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578628b8e80 0x5578628c1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.041+0000 7fd46ed93740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578628c1a00 0x5578628bb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.041+0000 7fd46ed93740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd4641ad9a0 con 0x5578628b8e80 2024-04-22T13:30:48.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.041+0000 7fd46ed93740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd4641af4e0 con 0x5578628c1a00 2024-04-22T13:30:48.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.041+0000 7fd46ed93740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd4641ae070 con 0x5578628b84d0 2024-04-22T13:30:48.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.041+0000 7fd469f40640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578628b8e80 0x5578628c1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.041+0000 7fd469f40640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578628b8e80 0x5578628c1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42344/0 (socket says 172.21.15.12:42344) 2024-04-22T13:30:48.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.041+0000 7fd469f40640 1 -- 172.21.15.12:0/3051296826 learned_addr learned my addr 172.21.15.12:0/3051296826 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:48.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.041+0000 7fd46973f640 1 --2- 172.21.15.12:0/3051296826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578628b84d0 0x5578628c0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.041+0000 7fd46973f640 1 --2- 172.21.15.12:0/3051296826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578628c1a00 0x5578628bb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.041+0000 7fd469f40640 1 -- 172.21.15.12:0/3051296826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578628b84d0 msgr2=0x5578628c0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.041+0000 7fd469f40640 1 --2- 172.21.15.12:0/3051296826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578628b84d0 0x5578628c0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.041+0000 7fd469f40640 1 -- 172.21.15.12:0/3051296826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578628c1a00 msgr2=0x5578628bb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.041+0000 7fd469f40640 1 --2- 172.21.15.12:0/3051296826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578628c1a00 0x5578628bb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.041+0000 7fd469f40640 1 -- 172.21.15.12:0/3051296826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578628b8c20 con 0x5578628b8e80 2024-04-22T13:30:48.039 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.041+0000 7fd469f40640 1 --2- 172.21.15.12:0/3051296826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578628b8e80 0x5578628c1360 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x5578628b3c10 tx=0x7fd46000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a325f7bb96d1ee7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:48.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.041+0000 7fd44f7fe640 1 -- 172.21.15.12:0/3051296826 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd460022070 con 0x5578628b8e80 2024-04-22T13:30:48.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.041+0000 7fd44f7fe640 1 -- 172.21.15.12:0/3051296826 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd46001c920 con 0x5578628b8e80 2024-04-22T13:30:48.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.041+0000 7fd44f7fe640 1 -- 172.21.15.12:0/3051296826 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd460010c60 con 0x5578628b8e80 2024-04-22T13:30:48.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.042+0000 7fd46ed93740 1 -- 172.21.15.12:0/3051296826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578628b8e80 msgr2=0x5578628c1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.042+0000 7fd46ed93740 1 --2- 172.21.15.12:0/3051296826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578628b8e80 0x5578628c1360 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x5578628b3c10 tx=0x7fd46000cd80 comp rx=0 tx=0).stop 2024-04-22T13:30:48.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.042+0000 7fd46ed93740 1 -- 172.21.15.12:0/3051296826 shutdown_connections 2024-04-22T13:30:48.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.042+0000 7fd46ed93740 1 --2- 172.21.15.12:0/3051296826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578628c1a00 0x5578628bb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.042+0000 7fd46ed93740 1 --2- 172.21.15.12:0/3051296826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578628b8e80 0x5578628c1360 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.042+0000 7fd46ed93740 1 --2- 172.21.15.12:0/3051296826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578628b84d0 0x5578628c0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.042+0000 7fd46ed93740 1 -- 172.21.15.12:0/3051296826 >> 172.21.15.12:0/3051296826 conn(0x5578628b2ba0 msgr2=0x5578628b1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:48.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.042+0000 7fd46ed93740 1 -- 172.21.15.12:0/3051296826 shutdown_connections 2024-04-22T13:30:48.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.042+0000 7fd46ed93740 1 -- 172.21.15.12:0/3051296826 wait complete. 2024-04-22T13:30:48.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.042+0000 7fd46ed93740 1 -- start start 2024-04-22T13:30:48.040 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.042+0000 7fd46ed93740 1 Processor -- start 2024-04-22T13:30:48.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.043+0000 7fd46ed93740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578628b84d0 0x5578628d6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.043+0000 7fd46ed93740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578628b8e80 0x5578628d70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.043+0000 7fd46ed93740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578628c1a00 0x5578628d40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.043+0000 7fd46ed93740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd4641aee10 con 0x5578628c1a00 2024-04-22T13:30:48.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.043+0000 7fd46ed93740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd4641ae740 con 0x5578628b84d0 2024-04-22T13:30:48.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.043+0000 7fd46ed93740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd4641ab9a0 con 0x5578628b8e80 2024-04-22T13:30:48.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.043+0000 7fd46ed93740 10 client.0 _ll_register_callbacks cb 0x7fd454004320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:48.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.043+0000 7fd46973f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578628c1a00 0x5578628d40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.043+0000 7fd46973f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578628c1a00 0x5578628d40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42348/0 (socket says 172.21.15.12:42348) 2024-04-22T13:30:48.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.043+0000 7fd46973f640 1 -- 172.21.15.12:0/1963312626 learned_addr learned my addr 172.21.15.12:0/1963312626 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:48.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.043+0000 7fd46973f640 1 --2- 172.21.15.12:0/1963312626 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578628b84d0 0x5578628d6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.043+0000 7fd469f40640 1 --2- 172.21.15.12:0/1963312626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578628b8e80 0x5578628d70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.043+0000 7fd46973f640 1 -- 172.21.15.12:0/1963312626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578628b8e80 msgr2=0x5578628d70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.043+0000 7fd46973f640 1 --2- 172.21.15.12:0/1963312626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578628b8e80 0x5578628d70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.043+0000 7fd46973f640 1 -- 172.21.15.12:0/1963312626 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578628b84d0 msgr2=0x5578628d6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.043+0000 7fd46973f640 1 --2- 172.21.15.12:0/1963312626 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578628b84d0 0x5578628d6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.043+0000 7fd46973f640 1 -- 172.21.15.12:0/1963312626 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578628d47d0 con 0x5578628c1a00 2024-04-22T13:30:48.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.043+0000 7fd469f40640 1 --2- 172.21.15.12:0/1963312626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578628b8e80 0x5578628d70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:48.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.043+0000 7fd46973f640 1 --2- 172.21.15.12:0/1963312626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578628c1a00 0x5578628d40e0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fd454009900 tx=0x7fd45402c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b505d55a4fdfc289 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:48.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.043+0000 7fd44d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:48.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.044+0000 7fd44d7fa640 1 -- 172.21.15.12:0/1963312626 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45405e070 con 0x5578628c1a00 2024-04-22T13:30:48.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.044+0000 7fd44d7fa640 1 -- 172.21.15.12:0/1963312626 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fd45402c920 con 0x5578628c1a00 2024-04-22T13:30:48.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.044+0000 7fd46ed93740 1 -- 172.21.15.12:0/1963312626 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55786282ac10 con 0x5578628c1a00 2024-04-22T13:30:48.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.044+0000 7fd44d7fa640 1 -- 172.21.15.12:0/1963312626 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd454049040 con 0x5578628c1a00 2024-04-22T13:30:48.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.044+0000 7fd46ed93740 10 client.18081 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:48.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.044+0000 7fd46ed93740 1 -- 172.21.15.12:0/1963312626 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5578629376e0 con 0x5578628c1a00 2024-04-22T13:30:48.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.044+0000 7fd44d7fa640 1 -- 172.21.15.12:0/1963312626 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55786282ac10 con 0x5578628c1a00 2024-04-22T13:30:48.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.044+0000 7fd44d7fa640 1 -- 172.21.15.12:0/1963312626 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd454064060 con 0x5578628c1a00 2024-04-22T13:30:48.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.044+0000 7fd46ed93740 10 client.18081 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:48.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.044+0000 7fd46ed93740 10 client.18081 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:30:48.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.044+0000 7fd46ed93740 20 client.18081 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:48.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.044+0000 7fd46ed93740 1 --2- 172.21.15.12:0/1963312626 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x557862937d70 0x55786293a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.044+0000 7fd46ed93740 4 client.18081 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:30:48.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.044+0000 7fd46ed93740 1 -- 172.21.15.12:0/1963312626 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557862824a80 con 0x557862937d70 2024-04-22T13:30:48.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.045+0000 7fd469f40640 1 --2- 172.21.15.12:0/1963312626 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x557862937d70 0x55786293a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.045+0000 7fd44d7fa640 10 client.18081 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:30:48.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.045+0000 7fd469f40640 1 --2- 172.21.15.12:0/1963312626 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x557862937d70 0x55786293a190 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b46448987625cbcf server_cookie=b895e82991252306 in_seq=0 out_seq=0 2024-04-22T13:30:48.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.045+0000 7fd44d7fa640 1 -- 172.21.15.12:0/1963312626 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557862824a80 con 0x557862937d70 2024-04-22T13:30:48.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.045+0000 7fd44d7fa640 10 client.18081 handle_command_reply: tid=0 2024-04-22T13:30:48.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.045+0000 7fd46ed93740 1 client.18081 shutdown 2024-04-22T13:30:48.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.045+0000 7fd46ed93740 1 -- 172.21.15.12:0/1963312626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578628c1a00 msgr2=0x5578628d40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.045+0000 7fd46ed93740 1 --2- 172.21.15.12:0/1963312626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578628c1a00 0x5578628d40e0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fd454009900 tx=0x7fd45402c5d0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.045+0000 7fd46ed93740 1 -- 172.21.15.12:0/1963312626 shutdown_connections 2024-04-22T13:30:48.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.045+0000 7fd46ed93740 1 --2- 172.21.15.12:0/1963312626 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x557862937d70 0x55786293a190 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.045+0000 7fd46ed93740 1 --2- 172.21.15.12:0/1963312626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5578628c1a00 0x5578628d40e0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.045+0000 7fd44d7fa640 0 client.18081 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:30:48.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.045+0000 7fd46ed93740 1 --2- 172.21.15.12:0/1963312626 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5578628b8e80 0x5578628d70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.045+0000 7fd46ed93740 1 --2- 172.21.15.12:0/1963312626 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5578628b84d0 0x5578628d6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.045+0000 7fd46ed93740 1 -- 172.21.15.12:0/1963312626 >> 172.21.15.12:0/1963312626 conn(0x5578628b2ba0 msgr2=0x5578628ccce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:48.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.046+0000 7fd46ed93740 1 -- 172.21.15.12:0/1963312626 shutdown_connections 2024-04-22T13:30:48.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.046+0000 7fd46ed93740 1 -- 172.21.15.12:0/1963312626 wait complete. 2024-04-22T13:30:48.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.046+0000 7fd46ed93740 20 client.18081 trim_cache size 0 max 16384 2024-04-22T13:30:48.048 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:30:48.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.046+0000 7fd46c1a4640 1 -- 172.21.15.12:0/149760952 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd4500a3d20 msgr2=0x7fd4500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.046+0000 7fd46c1a4640 1 --2- 172.21.15.12:0/149760952 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd4500a3d20 0x7fd4500a61e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0ce110 tx=0x7fd45c087040 comp rx=0 tx=0).stop 2024-04-22T13:30:48.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.046+0000 7fd46c1a4640 1 -- 172.21.15.12:0/149760952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd46419cfa0 msgr2=0x7fd46419d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.046+0000 7fd46c1a4640 1 --2- 172.21.15.12:0/149760952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd46419cfa0 0x7fd46419d450 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fd45c00cad0 tx=0x7fd45c00cf90 comp rx=0 tx=0).stop 2024-04-22T13:30:48.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.047+0000 7fd46c1a4640 1 -- 172.21.15.12:0/149760952 shutdown_connections 2024-04-22T13:30:48.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.047+0000 7fd46c1a4640 1 --2- 172.21.15.12:0/149760952 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd4500a3d20 0x7fd4500a61e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.047+0000 7fd46c1a4640 1 --2- 172.21.15.12:0/149760952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd46419cfa0 0x7fd46419d450 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.047+0000 7fd46c1a4640 1 --2- 172.21.15.12:0/149760952 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd464106160 0x7fd464198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.047+0000 7fd46c1a4640 1 --2- 172.21.15.12:0/149760952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd464101090 0x7fd4641986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.047+0000 7fd46c1a4640 1 -- 172.21.15.12:0/149760952 >> 172.21.15.12:0/149760952 conn(0x7fd464077f50 msgr2=0x7fd4640feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:48.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.047+0000 7fd46c1a4640 1 -- 172.21.15.12:0/149760952 shutdown_connections 2024-04-22T13:30:48.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.047+0000 7fd46c1a4640 1 -- 172.21.15.12:0/149760952 wait complete. 2024-04-22T13:30:48.052 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:30:48.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.161+0000 7f6247eeb640 1 -- 172.21.15.12:0/1499208009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6240106170 msgr2=0x7f624010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.161+0000 7f6247eeb640 1 --2- 172.21.15.12:0/1499208009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6240106170 0x7f624010dd40 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f6230009ca0 tx=0x7f623002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.161+0000 7f6247eeb640 1 -- 172.21.15.12:0/1499208009 shutdown_connections 2024-04-22T13:30:48.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.161+0000 7f6247eeb640 1 --2- 172.21.15.12:0/1499208009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6240106170 0x7f624010dd40 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.161+0000 7f6247eeb640 1 --2- 172.21.15.12:0/1499208009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6240101a50 0x7f6240105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.161+0000 7f6247eeb640 1 --2- 172.21.15.12:0/1499208009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62401010a0 0x7f6240101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.161+0000 7f6247eeb640 1 -- 172.21.15.12:0/1499208009 >> 172.21.15.12:0/1499208009 conn(0x7f6240077eb0 msgr2=0x7f62400782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:48.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.161+0000 7f6247eeb640 1 -- 172.21.15.12:0/1499208009 shutdown_connections 2024-04-22T13:30:48.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.161+0000 7f6247eeb640 1 -- 172.21.15.12:0/1499208009 wait complete. 2024-04-22T13:30:48.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.162+0000 7f6247eeb640 1 Processor -- start 2024-04-22T13:30:48.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.162+0000 7f6247eeb640 1 -- start start 2024-04-22T13:30:48.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.162+0000 7f6247eeb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62401010a0 0x7f6240102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.162+0000 7f6247eeb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6240106170 0x7f6240103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.162+0000 7f6247eeb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62401035b0 0x7f62401a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.162+0000 7f6247eeb640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f62401110c0 con 0x7f62401035b0 2024-04-22T13:30:48.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.162+0000 7f6247eeb640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6240110f40 con 0x7f62401010a0 2024-04-22T13:30:48.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.162+0000 7f6247eeb640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6240111240 con 0x7f6240106170 2024-04-22T13:30:48.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.162+0000 7f6246488640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62401035b0 0x7f62401a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.162+0000 7f6246488640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62401035b0 0x7f62401a68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42370/0 (socket says 172.21.15.12:42370) 2024-04-22T13:30:48.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.162+0000 7f6246488640 1 -- 172.21.15.12:0/2981957182 learned_addr learned my addr 172.21.15.12:0/2981957182 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:48.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.163+0000 7f6245c87640 1 --2- 172.21.15.12:0/2981957182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62401010a0 0x7f6240102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.163+0000 7f6246488640 1 -- 172.21.15.12:0/2981957182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6240106170 msgr2=0x7f6240103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.163+0000 7f6245486640 1 --2- 172.21.15.12:0/2981957182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6240106170 0x7f6240103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.163+0000 7f6246488640 1 --2- 172.21.15.12:0/2981957182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6240106170 0x7f6240103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.163+0000 7f6246488640 1 -- 172.21.15.12:0/2981957182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62401010a0 msgr2=0x7f6240102b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.163+0000 7f6246488640 1 --2- 172.21.15.12:0/2981957182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62401010a0 0x7f6240102b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.163+0000 7f6246488640 1 -- 172.21.15.12:0/2981957182 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62401a6fc0 con 0x7f62401035b0 2024-04-22T13:30:48.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.163+0000 7f6245c87640 1 --2- 172.21.15.12:0/2981957182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62401010a0 0x7f6240102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:30:48.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.163+0000 7f6246488640 1 --2- 172.21.15.12:0/2981957182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62401035b0 0x7f62401a68c0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f6230005e30 tx=0x7f6230034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:48.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.163+0000 7f623effd640 1 -- 172.21.15.12:0/2981957182 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6230039040 con 0x7f62401035b0 2024-04-22T13:30:48.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.163+0000 7f623effd640 1 -- 172.21.15.12:0/2981957182 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6230044070 con 0x7f62401035b0 2024-04-22T13:30:48.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.163+0000 7f6247eeb640 1 -- 172.21.15.12:0/2981957182 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62401a7250 con 0x7f62401035b0 2024-04-22T13:30:48.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.163+0000 7f623effd640 1 -- 172.21.15.12:0/2981957182 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6230003e60 con 0x7f62401035b0 2024-04-22T13:30:48.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.163+0000 7f6247eeb640 1 -- 172.21.15.12:0/2981957182 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62401a7790 con 0x7f62401035b0 2024-04-22T13:30:48.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.165+0000 7f624abb8740 1 Processor -- start 2024-04-22T13:30:48.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.165+0000 7f623effd640 1 -- 172.21.15.12:0/2981957182 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f62300324c0 con 0x7f62401035b0 2024-04-22T13:30:48.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.166+0000 7f624abb8740 1 -- start start 2024-04-22T13:30:48.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.166+0000 7f624abb8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621850fe4d0 0x5621850fe8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.166+0000 7f624abb8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5621850fee80 0x562185107430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.166+0000 7f624abb8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562185107a00 0x562185109e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.166+0000 7f624abb8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5621850631f0 con 0x5621850fee80 2024-04-22T13:30:48.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.166+0000 7f624abb8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5621850004b0 con 0x562185107a00 2024-04-22T13:30:48.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.166+0000 7f624abb8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56218505f1b0 con 0x5621850fe4d0 2024-04-22T13:30:48.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.166+0000 7f6245c87640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5621850fee80 0x562185107430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.166+0000 7f6245c87640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5621850fee80 0x562185107430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42386/0 (socket says 172.21.15.12:42386) 2024-04-22T13:30:48.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.166+0000 7f6245c87640 1 -- 172.21.15.12:0/521586552 learned_addr learned my addr 172.21.15.12:0/521586552 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:48.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.166+0000 7f6245486640 1 --2- 172.21.15.12:0/521586552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621850fe4d0 0x5621850fe8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.166+0000 7f6245486640 1 --2- 172.21.15.12:0/521586552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562185107a00 0x562185109e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.166+0000 7f623effd640 1 --2- 172.21.15.12:0/2981957182 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f62280a3dd0 0x7f62280a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.166+0000 7f6245c87640 1 -- 172.21.15.12:0/521586552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621850fe4d0 msgr2=0x5621850fe8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.166+0000 7f6245c87640 1 --2- 172.21.15.12:0/521586552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621850fe4d0 0x5621850fe8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.166+0000 7f6245c87640 1 -- 172.21.15.12:0/521586552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562185107a00 msgr2=0x562185109e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.166+0000 7f6245c87640 1 --2- 172.21.15.12:0/521586552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562185107a00 0x562185109e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.166+0000 7f6245c87640 1 -- 172.21.15.12:0/521586552 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562184ff21b0 con 0x5621850fee80 2024-04-22T13:30:48.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.166+0000 7f6246488640 1 --2- 172.21.15.12:0/2981957182 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f62280a3dd0 0x7f62280a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.166+0000 7f6245486640 1 --2- 172.21.15.12:0/521586552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621850fe4d0 0x5621850fe8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:30:48.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.167+0000 7f623effd640 1 -- 172.21.15.12:0/2981957182 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f62300f2980 con 0x7f62401035b0 2024-04-22T13:30:48.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.167+0000 7f6245c87640 1 --2- 172.21.15.12:0/521586552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5621850fee80 0x562185107430 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f6238005fd0 tx=0x7f623800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b058b4c25e2c6fc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:48.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.167+0000 7f6227fff640 1 -- 172.21.15.12:0/521586552 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6238016070 con 0x5621850fee80 2024-04-22T13:30:48.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.167+0000 7f6227fff640 1 -- 172.21.15.12:0/521586552 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62380054d0 con 0x5621850fee80 2024-04-22T13:30:48.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.167+0000 7f6227fff640 1 -- 172.21.15.12:0/521586552 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6238011470 con 0x5621850fee80 2024-04-22T13:30:48.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.168+0000 7f624abb8740 1 -- 172.21.15.12:0/521586552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5621850fee80 msgr2=0x562185107430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.168+0000 7f6246488640 1 --2- 172.21.15.12:0/2981957182 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f62280a3dd0 0x7f62280a6290 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f62300f02e0 tx=0x7f6230049000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:48.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.168+0000 7f624abb8740 1 --2- 172.21.15.12:0/521586552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5621850fee80 0x562185107430 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f6238005fd0 tx=0x7f623800e6c0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.168+0000 7f624abb8740 1 -- 172.21.15.12:0/521586552 shutdown_connections 2024-04-22T13:30:48.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.168+0000 7f624abb8740 1 --2- 172.21.15.12:0/521586552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562185107a00 0x562185109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.168+0000 7f624abb8740 1 --2- 172.21.15.12:0/521586552 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5621850fee80 0x562185107430 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.168+0000 7f624abb8740 1 --2- 172.21.15.12:0/521586552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621850fe4d0 0x5621850fe8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.168+0000 7f624abb8740 1 -- 172.21.15.12:0/521586552 >> 172.21.15.12:0/521586552 conn(0x5621850f8ba0 msgr2=0x5621850f8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:48.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.168+0000 7f624abb8740 1 -- 172.21.15.12:0/521586552 shutdown_connections 2024-04-22T13:30:48.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.168+0000 7f624abb8740 1 -- 172.21.15.12:0/521586552 wait complete. 2024-04-22T13:30:48.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.168+0000 7f624abb8740 1 -- start start 2024-04-22T13:30:48.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.168+0000 7f624abb8740 1 Processor -- start 2024-04-22T13:30:48.171 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.169+0000 7f624abb8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5621850fe4d0 0x5621851072a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.169+0000 7f624abb8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5621850fee80 0x562185104630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.169+0000 7f624abb8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562185107a00 0x562185104b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.169+0000 7f624abb8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56218510a970 con 0x5621850fe4d0 2024-04-22T13:30:48.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.169+0000 7f624abb8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56218501c500 con 0x5621850fee80 2024-04-22T13:30:48.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.169+0000 7f624abb8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56218510aaf0 con 0x562185107a00 2024-04-22T13:30:48.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.169+0000 7f624abb8740 10 client.0 _ll_register_callbacks cb 0x562184feab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:48.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.169+0000 7f6245486640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5621850fe4d0 0x5621851072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.169+0000 7f6245486640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5621850fe4d0 0x5621851072a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42392/0 (socket says 172.21.15.12:42392) 2024-04-22T13:30:48.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.169+0000 7f6245486640 1 -- 172.21.15.12:0/1929578144 learned_addr learned my addr 172.21.15.12:0/1929578144 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:48.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.169+0000 7f6245c87640 1 --2- 172.21.15.12:0/1929578144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5621850fee80 0x562185104630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.169+0000 7f6245486640 1 --2- 172.21.15.12:0/1929578144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562185107a00 0x562185104b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.169+0000 7f6245486640 1 -- 172.21.15.12:0/1929578144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562185107a00 msgr2=0x562185104b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.169+0000 7f6245486640 1 --2- 172.21.15.12:0/1929578144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562185107a00 0x562185104b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.169+0000 7f6245486640 1 -- 172.21.15.12:0/1929578144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5621850fee80 msgr2=0x562185104630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.169+0000 7f6245486640 1 --2- 172.21.15.12:0/1929578144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5621850fee80 0x562185104630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.169+0000 7f6245486640 1 -- 172.21.15.12:0/1929578144 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562184d753f0 con 0x5621850fe4d0 2024-04-22T13:30:48.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.169+0000 7f6245486640 1 --2- 172.21.15.12:0/1929578144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5621850fe4d0 0x5621851072a0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f6234014e20 tx=0x7f6234005ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58804160b688ea6b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:48.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.169+0000 7f62257fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:48.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.169+0000 7f62257fa640 1 -- 172.21.15.12:0/1929578144 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62340140e0 con 0x5621850fe4d0 2024-04-22T13:30:48.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.169+0000 7f62257fa640 1 -- 172.21.15.12:0/1929578144 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f6234005020 con 0x5621850fe4d0 2024-04-22T13:30:48.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.169+0000 7f624abb8740 1 -- 172.21.15.12:0/1929578144 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62380054d0 con 0x5621850fe4d0 2024-04-22T13:30:48.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.170+0000 7f62257fa640 1 -- 172.21.15.12:0/1929578144 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6234027050 con 0x5621850fe4d0 2024-04-22T13:30:48.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.170+0000 7f624abb8740 10 client.18093 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:48.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.170+0000 7f624abb8740 1 -- 172.21.15.12:0/1929578144 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562184e3f7d0 con 0x5621850fe4d0 2024-04-22T13:30:48.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.170+0000 7f62257fa640 1 -- 172.21.15.12:0/1929578144 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62380054d0 con 0x5621850fe4d0 2024-04-22T13:30:48.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.170+0000 7f62257fa640 1 -- 172.21.15.12:0/1929578144 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f623400f040 con 0x5621850fe4d0 2024-04-22T13:30:48.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.170+0000 7f624abb8740 10 client.18093 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:48.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.170+0000 7f624abb8740 10 client.18093 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:30:48.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.170+0000 7f624abb8740 20 client.18093 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:48.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.170+0000 7f624abb8740 1 --2- 172.21.15.12:0/1929578144 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x562185117b80 0x562185119fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.170+0000 7f624abb8740 4 client.18093 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:30:48.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.170+0000 7f624abb8740 1 -- 172.21.15.12:0/1929578144 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562185002930 con 0x562185117b80 2024-04-22T13:30:48.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.170+0000 7f6245c87640 1 --2- 172.21.15.12:0/1929578144 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x562185117b80 0x562185119fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.171+0000 7f62257fa640 10 client.18093 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:30:48.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.171+0000 7f6245c87640 1 --2- 172.21.15.12:0/1929578144 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x562185117b80 0x562185119fb0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cd9efefe4c755479 server_cookie=43edbc9aca3ce06f in_seq=0 out_seq=0 2024-04-22T13:30:48.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.171+0000 7f62257fa640 1 -- 172.21.15.12:0/1929578144 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562185002930 con 0x562185117b80 2024-04-22T13:30:48.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.171+0000 7f62257fa640 10 client.18093 handle_command_reply: tid=0 2024-04-22T13:30:48.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.171+0000 7f624abb8740 1 client.18093 shutdown 2024-04-22T13:30:48.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.171+0000 7f624abb8740 1 -- 172.21.15.12:0/1929578144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5621850fe4d0 msgr2=0x5621851072a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.171+0000 7f624abb8740 1 --2- 172.21.15.12:0/1929578144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5621850fe4d0 0x5621851072a0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f6234014e20 tx=0x7f6234005ed0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.172+0000 7f624abb8740 1 -- 172.21.15.12:0/1929578144 shutdown_connections 2024-04-22T13:30:48.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.172+0000 7f624abb8740 1 --2- 172.21.15.12:0/1929578144 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x562185117b80 0x562185119fb0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.172+0000 7f624abb8740 1 --2- 172.21.15.12:0/1929578144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562185107a00 0x562185104b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.174 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.172+0000 7f62257fa640 0 client.18093 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:30:48.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.172+0000 7f624abb8740 1 --2- 172.21.15.12:0/1929578144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5621850fee80 0x562185104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.172+0000 7f624abb8740 1 --2- 172.21.15.12:0/1929578144 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5621850fe4d0 0x5621851072a0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.172+0000 7f624abb8740 1 -- 172.21.15.12:0/1929578144 >> 172.21.15.12:0/1929578144 conn(0x5621850f8ba0 msgr2=0x5621850fa8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:48.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.172+0000 7f624abb8740 1 -- 172.21.15.12:0/1929578144 shutdown_connections 2024-04-22T13:30:48.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.172+0000 7f624abb8740 1 -- 172.21.15.12:0/1929578144 wait complete. 2024-04-22T13:30:48.175 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.172+0000 7f624abb8740 20 client.18093 trim_cache size 0 max 16384 2024-04-22T13:30:48.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:47 smithi134 ceph-mon[31338]: pgmap v203: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:48.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.185+0000 7f624abb8740 1 Processor -- start 2024-04-22T13:30:48.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.185+0000 7f624abb8740 1 -- start start 2024-04-22T13:30:48.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.185+0000 7f624abb8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5621850fe4d0 0x562185113860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.185+0000 7f624abb8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621850fee80 0x5621850ffb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.185+0000 7f624abb8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562185107a00 0x562185106d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.185+0000 7f624abb8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f62401afd50 con 0x5621850fe4d0 2024-04-22T13:30:48.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.185+0000 7f624abb8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f62401b1890 con 0x562185107a00 2024-04-22T13:30:48.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.185+0000 7f624abb8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f62401b0420 con 0x5621850fee80 2024-04-22T13:30:48.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.185+0000 7f6245486640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5621850fe4d0 0x562185113860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.185+0000 7f6245486640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5621850fe4d0 0x562185113860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42406/0 (socket says 172.21.15.12:42406) 2024-04-22T13:30:48.186 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.185+0000 7f6245486640 1 -- 172.21.15.12:0/1998874035 learned_addr learned my addr 172.21.15.12:0/1998874035 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:48.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.186+0000 7f6245c87640 1 --2- 172.21.15.12:0/1998874035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621850fee80 0x5621850ffb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.186+0000 7f6245486640 1 -- 172.21.15.12:0/1998874035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621850fee80 msgr2=0x5621850ffb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.186+0000 7f6245486640 1 --2- 172.21.15.12:0/1998874035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621850fee80 0x5621850ffb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.186+0000 7f6245486640 1 -- 172.21.15.12:0/1998874035 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562185107a00 msgr2=0x562185106d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.186+0000 7f6245486640 1 --2- 172.21.15.12:0/1998874035 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562185107a00 0x562185106d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.186+0000 7f6245486640 1 -- 172.21.15.12:0/1998874035 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562185109df0 con 0x5621850fe4d0 2024-04-22T13:30:48.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.186+0000 7f6245c87640 1 --2- 172.21.15.12:0/1998874035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621850fee80 0x5621850ffb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:30:48.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.186+0000 7f6245486640 1 --2- 172.21.15.12:0/1998874035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5621850fe4d0 0x562185113860 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f6234014fd0 tx=0x7f623400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b942ee92cd92511 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:48.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.186+0000 7f62277fe640 1 -- 172.21.15.12:0/1998874035 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6234024070 con 0x5621850fe4d0 2024-04-22T13:30:48.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.186+0000 7f62277fe640 1 -- 172.21.15.12:0/1998874035 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6234012450 con 0x5621850fe4d0 2024-04-22T13:30:48.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.186+0000 7f62277fe640 1 -- 172.21.15.12:0/1998874035 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6234012a10 con 0x5621850fe4d0 2024-04-22T13:30:48.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.187+0000 7f624abb8740 1 -- 172.21.15.12:0/1998874035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5621850fe4d0 msgr2=0x562185113860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.187+0000 7f624abb8740 1 --2- 172.21.15.12:0/1998874035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5621850fe4d0 0x562185113860 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f6234014fd0 tx=0x7f623400ebf0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.187+0000 7f624abb8740 1 -- 172.21.15.12:0/1998874035 shutdown_connections 2024-04-22T13:30:48.187 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.187+0000 7f624abb8740 1 --2- 172.21.15.12:0/1998874035 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562185107a00 0x562185106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.187+0000 7f624abb8740 1 --2- 172.21.15.12:0/1998874035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621850fee80 0x5621850ffb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.187+0000 7f624abb8740 1 --2- 172.21.15.12:0/1998874035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5621850fe4d0 0x562185113860 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.187+0000 7f624abb8740 1 -- 172.21.15.12:0/1998874035 >> 172.21.15.12:0/1998874035 conn(0x5621850f8ba0 msgr2=0x5621850f7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:48.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.187+0000 7f624abb8740 1 -- 172.21.15.12:0/1998874035 shutdown_connections 2024-04-22T13:30:48.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.187+0000 7f624abb8740 1 -- 172.21.15.12:0/1998874035 wait complete. 2024-04-22T13:30:48.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.187+0000 7f624abb8740 1 -- start start 2024-04-22T13:30:48.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.187+0000 7f624abb8740 1 Processor -- start 2024-04-22T13:30:48.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.187+0000 7f624abb8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5621850fe4d0 0x562185110110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.187+0000 7f624abb8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5621850fee80 0x56218510b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.188+0000 7f624abb8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562185107a00 0x56218510b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.188+0000 7f624abb8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f62401b11c0 con 0x5621850fee80 2024-04-22T13:30:48.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.188+0000 7f624abb8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f62401b0af0 con 0x5621850fe4d0 2024-04-22T13:30:48.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.188+0000 7f624abb8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f62401add50 con 0x562185107a00 2024-04-22T13:30:48.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.188+0000 7f624abb8740 10 client.0 _ll_register_callbacks cb 0x7f6234014940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:48.188 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.188+0000 7f6245c87640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5621850fee80 0x56218510b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.188+0000 7f6245c87640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5621850fee80 0x56218510b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42418/0 (socket says 172.21.15.12:42418) 2024-04-22T13:30:48.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.188+0000 7f6245c87640 1 -- 172.21.15.12:0/1524346589 learned_addr learned my addr 172.21.15.12:0/1524346589 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:48.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.188+0000 7f6245486640 1 --2- 172.21.15.12:0/1524346589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562185107a00 0x56218510b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.188+0000 7f6245486640 1 --2- 172.21.15.12:0/1524346589 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5621850fe4d0 0x562185110110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.188+0000 7f6245c87640 1 -- 172.21.15.12:0/1524346589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562185107a00 msgr2=0x56218510b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.188+0000 7f6245c87640 1 --2- 172.21.15.12:0/1524346589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562185107a00 0x56218510b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.188+0000 7f6245c87640 1 -- 172.21.15.12:0/1524346589 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5621850fe4d0 msgr2=0x562185110110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.188+0000 7f6245c87640 1 --2- 172.21.15.12:0/1524346589 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5621850fe4d0 0x562185110110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.188+0000 7f6245c87640 1 -- 172.21.15.12:0/1524346589 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56218510c1f0 con 0x5621850fee80 2024-04-22T13:30:48.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.188+0000 7f6245c87640 1 --2- 172.21.15.12:0/1524346589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5621850fee80 0x56218510b480 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x5621850fdfa0 tx=0x7f6238009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317066d04f1b1fc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:48.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.188+0000 7f62257fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:48.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.188+0000 7f62257fa640 1 -- 172.21.15.12:0/1524346589 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623802f060 con 0x5621850fee80 2024-04-22T13:30:48.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.188+0000 7f62257fa640 1 -- 172.21.15.12:0/1524346589 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f6238005040 con 0x5621850fee80 2024-04-22T13:30:48.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.188+0000 7f624abb8740 1 -- 172.21.15.12:0/1524346589 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6234012450 con 0x5621850fee80 2024-04-22T13:30:48.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.188+0000 7f62257fa640 1 -- 172.21.15.12:0/1524346589 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6238016040 con 0x5621850fee80 2024-04-22T13:30:48.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.188+0000 7f624abb8740 10 client.18099 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:48.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.188+0000 7f624abb8740 1 -- 172.21.15.12:0/1524346589 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56218517e080 con 0x5621850fee80 2024-04-22T13:30:48.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.189+0000 7f62257fa640 1 -- 172.21.15.12:0/1524346589 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6234012450 con 0x5621850fee80 2024-04-22T13:30:48.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.189+0000 7f62257fa640 1 -- 172.21.15.12:0/1524346589 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f623800f040 con 0x5621850fee80 2024-04-22T13:30:48.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.189+0000 7f624abb8740 10 client.18099 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:48.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.189+0000 7f624abb8740 10 client.18099 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:30:48.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.189+0000 7f624abb8740 20 client.18099 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:48.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.189+0000 7f624abb8740 1 --2- 172.21.15.12:0/1524346589 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x56218517e520 0x562185180910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.189+0000 7f624abb8740 4 client.18099 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:30:48.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.189+0000 7f624abb8740 1 -- 172.21.15.12:0/1524346589 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562185070c10 con 0x56218517e520 2024-04-22T13:30:48.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.189+0000 7f6245486640 1 --2- 172.21.15.12:0/1524346589 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x56218517e520 0x562185180910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.189+0000 7f6245486640 1 --2- 172.21.15.12:0/1524346589 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x56218517e520 0x562185180910 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=44a999ff22d54dbd server_cookie=6714dbc5787f49a in_seq=0 out_seq=0 2024-04-22T13:30:48.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.189+0000 7f62257fa640 10 client.18099 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:30:48.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.190+0000 7f62257fa640 1 -- 172.21.15.12:0/1524346589 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562185070c10 con 0x56218517e520 2024-04-22T13:30:48.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.190+0000 7f62257fa640 10 client.18099 handle_command_reply: tid=0 2024-04-22T13:30:48.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.190+0000 7f624abb8740 1 client.18099 shutdown 2024-04-22T13:30:48.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.190+0000 7f624abb8740 1 -- 172.21.15.12:0/1524346589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5621850fee80 msgr2=0x56218510b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.190+0000 7f624abb8740 1 --2- 172.21.15.12:0/1524346589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5621850fee80 0x56218510b480 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x5621850fdfa0 tx=0x7f6238009e70 comp rx=0 tx=0).stop 2024-04-22T13:30:48.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.190+0000 7f624abb8740 1 -- 172.21.15.12:0/1524346589 shutdown_connections 2024-04-22T13:30:48.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.190+0000 7f624abb8740 1 --2- 172.21.15.12:0/1524346589 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x56218517e520 0x562185180910 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.190+0000 7f624abb8740 1 --2- 172.21.15.12:0/1524346589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562185107a00 0x56218510b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.190+0000 7f624abb8740 1 --2- 172.21.15.12:0/1524346589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5621850fee80 0x56218510b480 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.190+0000 7f624abb8740 1 --2- 172.21.15.12:0/1524346589 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5621850fe4d0 0x562185110110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.190+0000 7f62257fa640 0 client.18099 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:30:48.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.190+0000 7f624abb8740 1 -- 172.21.15.12:0/1524346589 >> 172.21.15.12:0/1524346589 conn(0x5621850f8ba0 msgr2=0x5621850ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:48.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.190+0000 7f624abb8740 1 -- 172.21.15.12:0/1524346589 shutdown_connections 2024-04-22T13:30:48.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.190+0000 7f624abb8740 1 -- 172.21.15.12:0/1524346589 wait complete. 2024-04-22T13:30:48.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.190+0000 7f624abb8740 20 client.18099 trim_cache size 0 max 16384 2024-04-22T13:30:48.192 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:30:48.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.190+0000 7f6247eeb640 1 -- 172.21.15.12:0/2981957182 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f62280a3dd0 msgr2=0x7f62280a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.190+0000 7f6247eeb640 1 --2- 172.21.15.12:0/2981957182 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f62280a3dd0 0x7f62280a6290 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f62300f02e0 tx=0x7f6230049000 comp rx=0 tx=0).stop 2024-04-22T13:30:48.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.190+0000 7f6247eeb640 1 -- 172.21.15.12:0/2981957182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62401035b0 msgr2=0x7f62401a68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.190+0000 7f6247eeb640 1 --2- 172.21.15.12:0/2981957182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62401035b0 0x7f62401a68c0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f6230005e30 tx=0x7f6230034760 comp rx=0 tx=0).stop 2024-04-22T13:30:48.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.190+0000 7f6247eeb640 1 -- 172.21.15.12:0/2981957182 shutdown_connections 2024-04-22T13:30:48.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.190+0000 7f6247eeb640 1 --2- 172.21.15.12:0/2981957182 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f62280a3dd0 0x7f62280a6290 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.190+0000 7f6247eeb640 1 --2- 172.21.15.12:0/2981957182 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f62401035b0 0x7f62401a68c0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.190+0000 7f6247eeb640 1 --2- 172.21.15.12:0/2981957182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6240106170 0x7f6240103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.190+0000 7f6247eeb640 1 --2- 172.21.15.12:0/2981957182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f62401010a0 0x7f6240102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.190+0000 7f6247eeb640 1 -- 172.21.15.12:0/2981957182 >> 172.21.15.12:0/2981957182 conn(0x7f6240077eb0 msgr2=0x7f624006a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:48.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.191+0000 7f6247eeb640 1 -- 172.21.15.12:0/2981957182 shutdown_connections 2024-04-22T13:30:48.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.191+0000 7f6247eeb640 1 -- 172.21.15.12:0/2981957182 wait complete. 2024-04-22T13:30:48.195 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:47 smithi012 ceph-mon[28265]: pgmap v203: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:48.196 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:30:48.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:47 smithi111 ceph-mon[31361]: pgmap v203: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:30:48.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.306+0000 7f55b6f24640 1 -- 172.21.15.12:0/928296054 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55b0107270 msgr2=0x7f55b010ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.306+0000 7f55b6f24640 1 --2- 172.21.15.12:0/928296054 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55b0107270 0x7f55b010ee40 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009ca0 tx=0x7f55a802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.306+0000 7f55b6f24640 1 -- 172.21.15.12:0/928296054 shutdown_connections 2024-04-22T13:30:48.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.306+0000 7f55b6f24640 1 --2- 172.21.15.12:0/928296054 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55b0107270 0x7f55b010ee40 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.306+0000 7f55b6f24640 1 --2- 172.21.15.12:0/928296054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55b006b1f0 0x7f55b0106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.306+0000 7f55b6f24640 1 --2- 172.21.15.12:0/928296054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55b006a840 0x7f55b006ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.306+0000 7f55b6f24640 1 -- 172.21.15.12:0/928296054 >> 172.21.15.12:0/928296054 conn(0x7f55b0077eb0 msgr2=0x7f55b00782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:48.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.306+0000 7f55b6f24640 1 -- 172.21.15.12:0/928296054 shutdown_connections 2024-04-22T13:30:48.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.306+0000 7f55b6f24640 1 -- 172.21.15.12:0/928296054 wait complete. 2024-04-22T13:30:48.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.307+0000 7f55b6f24640 1 Processor -- start 2024-04-22T13:30:48.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.307+0000 7f55b6f24640 1 -- start start 2024-04-22T13:30:48.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.307+0000 7f55b6f24640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55b006a840 0x7f55b01986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.307+0000 7f55b6f24640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55b0107270 0x7f55b0198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.307+0000 7f55b6f24640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55b019cf90 0x7f55b019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.307+0000 7f55b6f24640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f55b01121d0 con 0x7f55b006a840 2024-04-22T13:30:48.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.307+0000 7f55b6f24640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f55b0112050 con 0x7f55b0107270 2024-04-22T13:30:48.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.307+0000 7f55b6f24640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f55b0112350 con 0x7f55b019cf90 2024-04-22T13:30:48.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.307+0000 7f55b4cc0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55b006a840 0x7f55b01986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.307+0000 7f55b4cc0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55b006a840 0x7f55b01986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42426/0 (socket says 172.21.15.12:42426) 2024-04-22T13:30:48.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.307+0000 7f55b4cc0640 1 -- 172.21.15.12:0/2999118666 learned_addr learned my addr 172.21.15.12:0/2999118666 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:48.304 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.307+0000 7f55affff640 1 --2- 172.21.15.12:0/2999118666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55b0107270 0x7f55b0198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.305 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.307+0000 7f55b54c1640 1 --2- 172.21.15.12:0/2999118666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55b019cf90 0x7f55b019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.307+0000 7f55b4cc0640 1 -- 172.21.15.12:0/2999118666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55b019cf90 msgr2=0x7f55b019d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.307+0000 7f55b4cc0640 1 --2- 172.21.15.12:0/2999118666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55b019cf90 0x7f55b019d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.307+0000 7f55b4cc0640 1 -- 172.21.15.12:0/2999118666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55b0107270 msgr2=0x7f55b0198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.307+0000 7f55b4cc0640 1 --2- 172.21.15.12:0/2999118666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55b0107270 0x7f55b0198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.307+0000 7f55b4cc0640 1 -- 172.21.15.12:0/2999118666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b019da80 con 0x7f55b006a840 2024-04-22T13:30:48.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.307+0000 7f55b4cc0640 1 --2- 172.21.15.12:0/2999118666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55b006a840 0x7f55b01986d0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f55a400d950 tx=0x7f55a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:48.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.308+0000 7f55adffb640 1 -- 172.21.15.12:0/2999118666 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a4017030 con 0x7f55b006a840 2024-04-22T13:30:48.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.308+0000 7f55adffb640 1 -- 172.21.15.12:0/2999118666 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55a4004020 con 0x7f55b006a840 2024-04-22T13:30:48.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.308+0000 7f55b6f24640 1 -- 172.21.15.12:0/2999118666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55b019dd70 con 0x7f55b006a840 2024-04-22T13:30:48.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.308+0000 7f55adffb640 1 -- 172.21.15.12:0/2999118666 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a40047a0 con 0x7f55b006a840 2024-04-22T13:30:48.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.308+0000 7f55b6f24640 1 -- 172.21.15.12:0/2999118666 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55b01a55b0 con 0x7f55b006a840 2024-04-22T13:30:48.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.310+0000 7f55adffb640 1 -- 172.21.15.12:0/2999118666 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f55a4009a30 con 0x7f55b006a840 2024-04-22T13:30:48.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.310+0000 7f55b9bf2740 1 Processor -- start 2024-04-22T13:30:48.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.310+0000 7f55b9bf2740 1 -- start start 2024-04-22T13:30:48.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.310+0000 7f55b9bf2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556b78384d0 0x5556b78388b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.310+0000 7f55adffb640 1 --2- 172.21.15.12:0/2999118666 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f55a00a3da0 0x7f55a00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.311+0000 7f55b9bf2740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556b7838e80 0x5556b7841430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.311+0000 7f55b9bf2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556b7841a00 0x5556b7843e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.311+0000 7f55b9bf2740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5556b779d1f0 con 0x5556b7838e80 2024-04-22T13:30:48.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.311+0000 7f55b9bf2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5556b773a4b0 con 0x5556b78384d0 2024-04-22T13:30:48.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.311+0000 7f55b9bf2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5556b77991b0 con 0x5556b7841a00 2024-04-22T13:30:48.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.311+0000 7f55affff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556b7838e80 0x5556b7841430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.311+0000 7f55b4cc0640 1 --2- 172.21.15.12:0/2999118666 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f55a00a3da0 0x7f55a00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.311+0000 7f55affff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556b7838e80 0x5556b7841430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42430/0 (socket says 172.21.15.12:42430) 2024-04-22T13:30:48.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.311+0000 7f55affff640 1 -- 172.21.15.12:0/2870194370 learned_addr learned my addr 172.21.15.12:0/2870194370 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:48.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.311+0000 7f55adffb640 1 -- 172.21.15.12:0/2999118666 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f55a40d00f0 con 0x7f55b006a840 2024-04-22T13:30:48.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.311+0000 7f55b54c1640 1 --2- 172.21.15.12:0/2870194370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556b7841a00 0x5556b7843e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.311+0000 7f55b54c1640 1 --2- 172.21.15.12:0/2870194370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556b78384d0 0x5556b78388b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.311+0000 7f55affff640 1 -- 172.21.15.12:0/2870194370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556b7841a00 msgr2=0x5556b7843e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.311+0000 7f55affff640 1 --2- 172.21.15.12:0/2870194370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556b7841a00 0x5556b7843e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.311+0000 7f55affff640 1 -- 172.21.15.12:0/2870194370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556b78384d0 msgr2=0x5556b78388b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.311+0000 7f55affff640 1 --2- 172.21.15.12:0/2870194370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556b78384d0 0x5556b78388b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.311+0000 7f55affff640 1 -- 172.21.15.12:0/2870194370 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556b772c1b0 con 0x5556b7838e80 2024-04-22T13:30:48.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.311+0000 7f55b54c1640 1 --2- 172.21.15.12:0/2870194370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556b78384d0 0x5556b78388b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:30:48.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.311+0000 7f55b4cc0640 1 --2- 172.21.15.12:0/2999118666 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f55a00a3da0 0x7f55a00a6260 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f55a4098ab0 tx=0x7f55a40046c0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:48.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.311+0000 7f55affff640 1 --2- 172.21.15.12:0/2870194370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556b7838e80 0x5556b7841430 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f559c009c00 tx=0x7f559c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c217e99cd777f451 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:48.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.311+0000 7f559affd640 1 -- 172.21.15.12:0/2870194370 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c004230 con 0x5556b7838e80 2024-04-22T13:30:48.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.311+0000 7f559affd640 1 -- 172.21.15.12:0/2870194370 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f559c0043d0 con 0x5556b7838e80 2024-04-22T13:30:48.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.311+0000 7f559affd640 1 -- 172.21.15.12:0/2870194370 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c019640 con 0x5556b7838e80 2024-04-22T13:30:48.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.312+0000 7f55b9bf2740 1 -- 172.21.15.12:0/2870194370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556b7838e80 msgr2=0x5556b7841430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.312+0000 7f55b9bf2740 1 --2- 172.21.15.12:0/2870194370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556b7838e80 0x5556b7841430 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f559c009c00 tx=0x7f559c011810 comp rx=0 tx=0).stop 2024-04-22T13:30:48.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.312+0000 7f55b9bf2740 1 -- 172.21.15.12:0/2870194370 shutdown_connections 2024-04-22T13:30:48.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.312+0000 7f55b9bf2740 1 --2- 172.21.15.12:0/2870194370 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556b7841a00 0x5556b7843e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.312+0000 7f55b9bf2740 1 --2- 172.21.15.12:0/2870194370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556b7838e80 0x5556b7841430 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.312+0000 7f55b9bf2740 1 --2- 172.21.15.12:0/2870194370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556b78384d0 0x5556b78388b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.312+0000 7f55b9bf2740 1 -- 172.21.15.12:0/2870194370 >> 172.21.15.12:0/2870194370 conn(0x5556b7832ba0 msgr2=0x5556b7832f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:48.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.312+0000 7f55b9bf2740 1 -- 172.21.15.12:0/2870194370 shutdown_connections 2024-04-22T13:30:48.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.312+0000 7f55b9bf2740 1 -- 172.21.15.12:0/2870194370 wait complete. 2024-04-22T13:30:48.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.312+0000 7f55b9bf2740 1 -- start start 2024-04-22T13:30:48.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.313+0000 7f55b9bf2740 1 Processor -- start 2024-04-22T13:30:48.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.313+0000 7f55b9bf2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556b78384d0 0x5556b78412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.313+0000 7f55b9bf2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556b7838e80 0x5556b783e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.313+0000 7f55b9bf2740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556b7841a00 0x5556b783ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.313+0000 7f55b9bf2740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5556b7844970 con 0x5556b7841a00 2024-04-22T13:30:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.313+0000 7f55b9bf2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5556b7756500 con 0x5556b7838e80 2024-04-22T13:30:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.313+0000 7f55b9bf2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5556b7844af0 con 0x5556b78384d0 2024-04-22T13:30:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.313+0000 7f55b9bf2740 10 client.0 _ll_register_callbacks cb 0x5556b7724b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.313+0000 7f55b54c1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556b7841a00 0x5556b783ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.313+0000 7f55b54c1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556b7841a00 0x5556b783ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42446/0 (socket says 172.21.15.12:42446) 2024-04-22T13:30:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.313+0000 7f55b54c1640 1 -- 172.21.15.12:0/3615931090 learned_addr learned my addr 172.21.15.12:0/3615931090 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.313+0000 7f55b54c1640 1 --2- 172.21.15.12:0/3615931090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556b78384d0 0x5556b78412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.313+0000 7f55affff640 1 --2- 172.21.15.12:0/3615931090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556b7838e80 0x5556b783e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.313+0000 7f55b54c1640 1 -- 172.21.15.12:0/3615931090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556b78384d0 msgr2=0x5556b78412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.313+0000 7f55b54c1640 1 --2- 172.21.15.12:0/3615931090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556b78384d0 0x5556b78412e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.313+0000 7f55b54c1640 1 -- 172.21.15.12:0/3615931090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556b7838e80 msgr2=0x5556b783e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.313+0000 7f55b54c1640 1 --2- 172.21.15.12:0/3615931090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556b7838e80 0x5556b783e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.313+0000 7f55b54c1640 1 -- 172.21.15.12:0/3615931090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556b74af3f0 con 0x5556b7841a00 2024-04-22T13:30:48.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.314+0000 7f55b54c1640 1 --2- 172.21.15.12:0/3615931090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556b7841a00 0x5556b783ebd0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009890 tx=0x7f55a802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b956159e1ff5269e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:48.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.314+0000 7f558ffff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:48.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.314+0000 7f558ffff640 1 -- 172.21.15.12:0/3615931090 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a804b060 con 0x5556b7841a00 2024-04-22T13:30:48.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.314+0000 7f558ffff640 1 -- 172.21.15.12:0/3615931090 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f55a8003970 con 0x5556b7841a00 2024-04-22T13:30:48.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.314+0000 7f55b9bf2740 1 -- 172.21.15.12:0/3615931090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f559c0043d0 con 0x5556b7841a00 2024-04-22T13:30:48.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.314+0000 7f558ffff640 1 -- 172.21.15.12:0/3615931090 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f55a8039040 con 0x5556b7841a00 2024-04-22T13:30:48.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.314+0000 7f55b9bf2740 10 client.18111 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:48.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.314+0000 7f55b9bf2740 1 -- 172.21.15.12:0/3615931090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5556b75797d0 con 0x5556b7841a00 2024-04-22T13:30:48.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.314+0000 7f558ffff640 1 -- 172.21.15.12:0/3615931090 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f559c0043d0 con 0x5556b7841a00 2024-04-22T13:30:48.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.314+0000 7f558ffff640 1 -- 172.21.15.12:0/3615931090 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f55a803f070 con 0x5556b7841a00 2024-04-22T13:30:48.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.314+0000 7f55b9bf2740 10 client.18111 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:48.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.314+0000 7f55b9bf2740 10 client.18111 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:30:48.320 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.314+0000 7f55b9bf2740 20 client.18111 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:48.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.314+0000 7f55b9bf2740 1 --2- 172.21.15.12:0/3615931090 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x5556b7851bc0 0x5556b7853fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.314+0000 7f55b9bf2740 4 client.18111 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:30:48.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.314+0000 7f55b9bf2740 1 -- 172.21.15.12:0/3615931090 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556b773c930 con 0x5556b7851bc0 2024-04-22T13:30:48.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.315+0000 7f55affff640 1 --2- 172.21.15.12:0/3615931090 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x5556b7851bc0 0x5556b7853fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.315+0000 7f558ffff640 10 client.18111 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:30:48.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.315+0000 7f55affff640 1 --2- 172.21.15.12:0/3615931090 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x5556b7851bc0 0x5556b7853fb0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bc587f633ac78ba6 server_cookie=754438a94eb8de07 in_seq=0 out_seq=0 2024-04-22T13:30:48.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.316+0000 7f558ffff640 1 -- 172.21.15.12:0/3615931090 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5556b773c930 con 0x5556b7851bc0 2024-04-22T13:30:48.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.316+0000 7f558ffff640 10 client.18111 handle_command_reply: tid=0 2024-04-22T13:30:48.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.316+0000 7f55b9bf2740 1 client.18111 shutdown 2024-04-22T13:30:48.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.316+0000 7f55b9bf2740 1 -- 172.21.15.12:0/3615931090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556b7841a00 msgr2=0x5556b783ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.316+0000 7f55b9bf2740 1 --2- 172.21.15.12:0/3615931090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556b7841a00 0x5556b783ebd0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009890 tx=0x7f55a802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.316+0000 7f55b9bf2740 1 -- 172.21.15.12:0/3615931090 shutdown_connections 2024-04-22T13:30:48.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.316+0000 7f55b9bf2740 1 --2- 172.21.15.12:0/3615931090 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x5556b7851bc0 0x5556b7853fb0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.316+0000 7f55b9bf2740 1 --2- 172.21.15.12:0/3615931090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556b7841a00 0x5556b783ebd0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.316+0000 7f558ffff640 0 client.18111 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:30:48.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.316+0000 7f55b9bf2740 1 --2- 172.21.15.12:0/3615931090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556b7838e80 0x5556b783e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.316+0000 7f55b9bf2740 1 --2- 172.21.15.12:0/3615931090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556b78384d0 0x5556b78412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.316+0000 7f55b9bf2740 1 -- 172.21.15.12:0/3615931090 >> 172.21.15.12:0/3615931090 conn(0x5556b7832ba0 msgr2=0x5556b78348f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:48.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.316+0000 7f55b9bf2740 1 -- 172.21.15.12:0/3615931090 shutdown_connections 2024-04-22T13:30:48.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.316+0000 7f55b9bf2740 1 -- 172.21.15.12:0/3615931090 wait complete. 2024-04-22T13:30:48.322 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.316+0000 7f55b9bf2740 20 client.18111 trim_cache size 0 max 16384 2024-04-22T13:30:48.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.329+0000 7f55b9bf2740 1 Processor -- start 2024-04-22T13:30:48.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.329+0000 7f55b9bf2740 1 -- start start 2024-04-22T13:30:48.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.330+0000 7f55b9bf2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556b78384d0 0x5556b784d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.330+0000 7f55b9bf2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556b7838e80 0x5556b784a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.330+0000 7f55b9bf2740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556b7841a00 0x5556b783d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.330+0000 7f55b9bf2740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f55b01ad940 con 0x5556b7841a00 2024-04-22T13:30:48.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.330+0000 7f55b9bf2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f55b01af480 con 0x5556b7838e80 2024-04-22T13:30:48.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.330+0000 7f55b9bf2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f55b01ae010 con 0x5556b78384d0 2024-04-22T13:30:48.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.330+0000 7f55b54c1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556b7841a00 0x5556b783d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.330+0000 7f55b54c1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556b7841a00 0x5556b783d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42450/0 (socket says 172.21.15.12:42450) 2024-04-22T13:30:48.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.330+0000 7f55b54c1640 1 -- 172.21.15.12:0/3503907808 learned_addr learned my addr 172.21.15.12:0/3503907808 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:48.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.330+0000 7f55b54c1640 1 --2- 172.21.15.12:0/3503907808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556b78384d0 0x5556b784d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.330+0000 7f55b54c1640 1 -- 172.21.15.12:0/3503907808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556b78384d0 msgr2=0x5556b784d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.330+0000 7f55b54c1640 1 --2- 172.21.15.12:0/3503907808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556b78384d0 0x5556b784d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.330+0000 7f55b54c1640 1 -- 172.21.15.12:0/3503907808 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556b7838e80 msgr2=0x5556b784a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.330+0000 7f55b54c1640 1 --2- 172.21.15.12:0/3503907808 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556b7838e80 0x5556b784a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.330+0000 7f55b54c1640 1 -- 172.21.15.12:0/3503907808 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556b783cbc0 con 0x5556b7841a00 2024-04-22T13:30:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.330+0000 7f55b54c1640 1 --2- 172.21.15.12:0/3503907808 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556b7841a00 0x5556b783d7d0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009860 tx=0x7f55a8032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a11097ea0c9cca3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.330+0000 7f559a7fc640 1 -- 172.21.15.12:0/3503907808 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a804b040 con 0x5556b7841a00 2024-04-22T13:30:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.330+0000 7f559a7fc640 1 -- 172.21.15.12:0/3503907808 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55a8032e80 con 0x5556b7841a00 2024-04-22T13:30:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.330+0000 7f559a7fc640 1 -- 172.21.15.12:0/3503907808 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a8003970 con 0x5556b7841a00 2024-04-22T13:30:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.331+0000 7f55b9bf2740 1 -- 172.21.15.12:0/3503907808 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556b7841a00 msgr2=0x5556b783d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.331+0000 7f55b9bf2740 1 --2- 172.21.15.12:0/3503907808 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556b7841a00 0x5556b783d7d0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009860 tx=0x7f55a8032340 comp rx=0 tx=0).stop 2024-04-22T13:30:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.331+0000 7f55b9bf2740 1 -- 172.21.15.12:0/3503907808 shutdown_connections 2024-04-22T13:30:48.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.331+0000 7f55b9bf2740 1 --2- 172.21.15.12:0/3503907808 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556b7841a00 0x5556b783d7d0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.331+0000 7f55b9bf2740 1 --2- 172.21.15.12:0/3503907808 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556b7838e80 0x5556b784a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.331+0000 7f55b9bf2740 1 --2- 172.21.15.12:0/3503907808 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556b78384d0 0x5556b784d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.331+0000 7f55b9bf2740 1 -- 172.21.15.12:0/3503907808 >> 172.21.15.12:0/3503907808 conn(0x5556b7832ba0 msgr2=0x5556b7833ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:48.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.331+0000 7f55b9bf2740 1 -- 172.21.15.12:0/3503907808 shutdown_connections 2024-04-22T13:30:48.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.331+0000 7f55b9bf2740 1 -- 172.21.15.12:0/3503907808 wait complete. 2024-04-22T13:30:48.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.331+0000 7f55b9bf2740 1 -- start start 2024-04-22T13:30:48.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.331+0000 7f55b9bf2740 1 Processor -- start 2024-04-22T13:30:48.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.332+0000 7f55b9bf2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556b78384d0 0x5556b7840670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.332+0000 7f55b9bf2740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556b7838e80 0x5556b7840bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.332+0000 7f55b9bf2740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556b7841a00 0x5556b78b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.332+0000 7f55b9bf2740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f55b01aedb0 con 0x5556b7841a00 2024-04-22T13:30:48.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.332+0000 7f55b9bf2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f55b01ae6e0 con 0x5556b78384d0 2024-04-22T13:30:48.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.332+0000 7f55b9bf2740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f55b01ab940 con 0x5556b7838e80 2024-04-22T13:30:48.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.332+0000 7f55b9bf2740 10 client.0 _ll_register_callbacks cb 0x7f55a8032dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:48.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.332+0000 7f55b54c1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556b7841a00 0x5556b78b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.332+0000 7f55b54c1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556b7841a00 0x5556b78b34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42460/0 (socket says 172.21.15.12:42460) 2024-04-22T13:30:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.332+0000 7f55b54c1640 1 -- 172.21.15.12:0/3764434795 learned_addr learned my addr 172.21.15.12:0/3764434795 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.332+0000 7f55b54c1640 1 --2- 172.21.15.12:0/3764434795 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556b78384d0 0x5556b7840670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.332+0000 7f55affff640 1 --2- 172.21.15.12:0/3764434795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556b7838e80 0x5556b7840bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.332+0000 7f55b54c1640 1 -- 172.21.15.12:0/3764434795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556b7838e80 msgr2=0x5556b7840bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.332+0000 7f55b54c1640 1 --2- 172.21.15.12:0/3764434795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556b7838e80 0x5556b7840bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.332+0000 7f55b54c1640 1 -- 172.21.15.12:0/3764434795 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556b78384d0 msgr2=0x5556b7840670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.332+0000 7f55b54c1640 1 --2- 172.21.15.12:0/3764434795 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556b78384d0 0x5556b7840670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.332+0000 7f55b54c1640 1 -- 172.21.15.12:0/3764434795 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556b78b3b20 con 0x5556b7841a00 2024-04-22T13:30:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.332+0000 7f55b54c1640 1 --2- 172.21.15.12:0/3764434795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556b7841a00 0x5556b78b34b0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x5556b7841690 tx=0x7f55a802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db48924785fbf5c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.332+0000 7f558ffff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.332+0000 7f558ffff640 1 -- 172.21.15.12:0/3764434795 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a804b030 con 0x5556b7841a00 2024-04-22T13:30:48.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.333+0000 7f558ffff640 1 -- 172.21.15.12:0/3764434795 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f55a8003d00 con 0x5556b7841a00 2024-04-22T13:30:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.333+0000 7f55b9bf2740 1 -- 172.21.15.12:0/3764434795 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55a8032e80 con 0x5556b7841a00 2024-04-22T13:30:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.333+0000 7f558ffff640 1 -- 172.21.15.12:0/3764434795 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f55a8034e20 con 0x5556b7841a00 2024-04-22T13:30:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.333+0000 7f55b9bf2740 10 client.18117 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.333+0000 7f55b9bf2740 1 -- 172.21.15.12:0/3764434795 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5556b78bc0c0 con 0x5556b7841a00 2024-04-22T13:30:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.333+0000 7f558ffff640 1 -- 172.21.15.12:0/3764434795 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55a8032e80 con 0x5556b7841a00 2024-04-22T13:30:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.333+0000 7f558ffff640 1 -- 172.21.15.12:0/3764434795 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f55a803f070 con 0x5556b7841a00 2024-04-22T13:30:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.333+0000 7f55b9bf2740 10 client.18117 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.333+0000 7f55b9bf2740 10 client.18117 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:30:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.333+0000 7f55b9bf2740 20 client.18117 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.333+0000 7f55b9bf2740 1 --2- 172.21.15.12:0/3764434795 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x5556b78bc6c0 0x5556b78beab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.333+0000 7f55b9bf2740 4 client.18117 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:30:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.333+0000 7f55b9bf2740 1 -- 172.21.15.12:0/3764434795 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5556b77aac10 con 0x5556b78bc6c0 2024-04-22T13:30:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.333+0000 7f55affff640 1 --2- 172.21.15.12:0/3764434795 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x5556b78bc6c0 0x5556b78beab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.334+0000 7f558ffff640 10 client.18117 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:30:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.334+0000 7f55affff640 1 --2- 172.21.15.12:0/3764434795 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x5556b78bc6c0 0x5556b78beab0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f94c1dc6834100bb server_cookie=18770a307ca01e44 in_seq=0 out_seq=0 2024-04-22T13:30:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.334+0000 7f558ffff640 1 -- 172.21.15.12:0/3764434795 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5556b77aac10 con 0x5556b78bc6c0 2024-04-22T13:30:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.334+0000 7f558ffff640 10 client.18117 handle_command_reply: tid=0 2024-04-22T13:30:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.334+0000 7f55b9bf2740 1 client.18117 shutdown 2024-04-22T13:30:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.334+0000 7f55b9bf2740 1 -- 172.21.15.12:0/3764434795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556b7841a00 msgr2=0x5556b78b34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.334+0000 7f55b9bf2740 1 --2- 172.21.15.12:0/3764434795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556b7841a00 0x5556b78b34b0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x5556b7841690 tx=0x7f55a802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.334+0000 7f55b9bf2740 1 -- 172.21.15.12:0/3764434795 shutdown_connections 2024-04-22T13:30:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.334+0000 7f55b9bf2740 1 --2- 172.21.15.12:0/3764434795 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x5556b78bc6c0 0x5556b78beab0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.334+0000 7f55b9bf2740 1 --2- 172.21.15.12:0/3764434795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5556b7841a00 0x5556b78b34b0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.334+0000 7f55b9bf2740 1 --2- 172.21.15.12:0/3764434795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5556b7838e80 0x5556b7840bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.334+0000 7f558ffff640 0 client.18117 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:30:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.334+0000 7f55b9bf2740 1 --2- 172.21.15.12:0/3764434795 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5556b78384d0 0x5556b7840670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.334+0000 7f55b9bf2740 1 -- 172.21.15.12:0/3764434795 >> 172.21.15.12:0/3764434795 conn(0x5556b7832ba0 msgr2=0x5556b7833ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.334+0000 7f55b9bf2740 1 -- 172.21.15.12:0/3764434795 shutdown_connections 2024-04-22T13:30:48.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.334+0000 7f55b9bf2740 1 -- 172.21.15.12:0/3764434795 wait complete. 2024-04-22T13:30:48.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.334+0000 7f55b9bf2740 20 client.18117 trim_cache size 0 max 16384 2024-04-22T13:30:48.337 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:30:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.335+0000 7f55b6f24640 1 -- 172.21.15.12:0/2999118666 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f55a00a3da0 msgr2=0x7f55a00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.335+0000 7f55b6f24640 1 --2- 172.21.15.12:0/2999118666 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f55a00a3da0 0x7f55a00a6260 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f55a4098ab0 tx=0x7f55a40046c0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.335+0000 7f55b6f24640 1 -- 172.21.15.12:0/2999118666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55b006a840 msgr2=0x7f55b01986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.335+0000 7f55b6f24640 1 --2- 172.21.15.12:0/2999118666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55b006a840 0x7f55b01986d0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f55a400d950 tx=0x7f55a400de10 comp rx=0 tx=0).stop 2024-04-22T13:30:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.335+0000 7f55b6f24640 1 -- 172.21.15.12:0/2999118666 shutdown_connections 2024-04-22T13:30:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.335+0000 7f55b6f24640 1 --2- 172.21.15.12:0/2999118666 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f55a00a3da0 0x7f55a00a6260 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.335+0000 7f55b6f24640 1 --2- 172.21.15.12:0/2999118666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55b019cf90 0x7f55b019d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.335+0000 7f55b6f24640 1 --2- 172.21.15.12:0/2999118666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55b0107270 0x7f55b0198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.335+0000 7f55b6f24640 1 --2- 172.21.15.12:0/2999118666 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f55b006a840 0x7f55b01986d0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.335+0000 7f55b6f24640 1 -- 172.21.15.12:0/2999118666 >> 172.21.15.12:0/2999118666 conn(0x7f55b0077eb0 msgr2=0x7f55b0100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:48.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.335+0000 7f55b6f24640 1 -- 172.21.15.12:0/2999118666 shutdown_connections 2024-04-22T13:30:48.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.335+0000 7f55b6f24640 1 -- 172.21.15.12:0/2999118666 wait complete. 2024-04-22T13:30:48.341 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:30:48.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.449+0000 7f4f4a0e4640 1 -- 172.21.15.12:0/2231301015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f44107260 msgr2=0x7f4f4410ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.449+0000 7f4f4a0e4640 1 --2- 172.21.15.12:0/2231301015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f44107260 0x7f4f4410ee30 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009ca0 tx=0x7f4f3c02c100 comp rx=0 tx=0).stop 2024-04-22T13:30:48.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.449+0000 7f4f4a0e4640 1 -- 172.21.15.12:0/2231301015 shutdown_connections 2024-04-22T13:30:48.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.449+0000 7f4f4a0e4640 1 --2- 172.21.15.12:0/2231301015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f44107260 0x7f4f4410ee30 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.449+0000 7f4f4a0e4640 1 --2- 172.21.15.12:0/2231301015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f44102b40 0x7f4f44106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.449+0000 7f4f4a0e4640 1 --2- 172.21.15.12:0/2231301015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f44102190 0x7f4f44102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.449+0000 7f4f4a0e4640 1 -- 172.21.15.12:0/2231301015 >> 172.21.15.12:0/2231301015 conn(0x7f4f440fd990 msgr2=0x7f4f440ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:48.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.449+0000 7f4f4a0e4640 1 -- 172.21.15.12:0/2231301015 shutdown_connections 2024-04-22T13:30:48.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.449+0000 7f4f4a0e4640 1 -- 172.21.15.12:0/2231301015 wait complete. 2024-04-22T13:30:48.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.450+0000 7f4f4a0e4640 1 Processor -- start 2024-04-22T13:30:48.446 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.450+0000 7f4f4a0e4640 1 -- start start 2024-04-22T13:30:48.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.450+0000 7f4f4a0e4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f44102190 0x7f4f441985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.450+0000 7f4f4a0e4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f44107260 0x7f4f44198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.450+0000 7f4f4a0e4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f4419ce90 0x7f4f4419d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.450+0000 7f4f4a0e4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4f441121b0 con 0x7f4f4419ce90 2024-04-22T13:30:48.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.450+0000 7f4f4a0e4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4f44112030 con 0x7f4f44102190 2024-04-22T13:30:48.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.450+0000 7f4f4a0e4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4f44112330 con 0x7f4f44107260 2024-04-22T13:30:48.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.451+0000 7f4f43fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f4419ce90 0x7f4f4419d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.451+0000 7f4f43fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f4419ce90 0x7f4f4419d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42488/0 (socket says 172.21.15.12:42488) 2024-04-22T13:30:48.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.451+0000 7f4f43fff640 1 -- 172.21.15.12:0/89165936 learned_addr learned my addr 172.21.15.12:0/89165936 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:48.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.451+0000 7f4f437fe640 1 --2- 172.21.15.12:0/89165936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f44102190 0x7f4f441985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.451+0000 7f4f42ffd640 1 --2- 172.21.15.12:0/89165936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f44107260 0x7f4f44198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.451+0000 7f4f43fff640 1 -- 172.21.15.12:0/89165936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f44107260 msgr2=0x7f4f44198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.451+0000 7f4f43fff640 1 --2- 172.21.15.12:0/89165936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f44107260 0x7f4f44198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.451+0000 7f4f43fff640 1 -- 172.21.15.12:0/89165936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f44102190 msgr2=0x7f4f441985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.451+0000 7f4f43fff640 1 --2- 172.21.15.12:0/89165936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f44102190 0x7f4f441985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.451+0000 7f4f43fff640 1 -- 172.21.15.12:0/89165936 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4419d980 con 0x7f4f4419ce90 2024-04-22T13:30:48.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.451+0000 7f4f43fff640 1 --2- 172.21.15.12:0/89165936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f4419ce90 0x7f4f4419d340 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0098d0 tx=0x7f4f3c035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:48.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.451+0000 7f4f40ff9640 1 -- 172.21.15.12:0/89165936 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c039040 con 0x7f4f4419ce90 2024-04-22T13:30:48.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.451+0000 7f4f40ff9640 1 -- 172.21.15.12:0/89165936 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f3c044070 con 0x7f4f4419ce90 2024-04-22T13:30:48.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.451+0000 7f4f40ff9640 1 -- 172.21.15.12:0/89165936 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c004540 con 0x7f4f4419ce90 2024-04-22T13:30:48.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.451+0000 7f4f4a0e4640 1 -- 172.21.15.12:0/89165936 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f4419dc10 con 0x7f4f4419ce90 2024-04-22T13:30:48.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.451+0000 7f4f4a0e4640 1 -- 172.21.15.12:0/89165936 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f441a5430 con 0x7f4f4419ce90 2024-04-22T13:30:48.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.454+0000 7f4f40ff9640 1 -- 172.21.15.12:0/89165936 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4f3c0046e0 con 0x7f4f4419ce90 2024-04-22T13:30:48.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.454+0000 7f4f4cdb3740 1 Processor -- start 2024-04-22T13:30:48.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.454+0000 7f4f4cdb3740 1 -- start start 2024-04-22T13:30:48.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.454+0000 7f4f4cdb3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598447c74d0 0x5598447c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.454+0000 7f4f4cdb3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598447c7e80 0x5598447d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.454+0000 7f4f4cdb3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5598447d0a00 0x5598447d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.454+0000 7f4f4cdb3740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55984472c1f0 con 0x5598447c7e80 2024-04-22T13:30:48.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.454+0000 7f4f4cdb3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5598446c94b0 con 0x5598447d0a00 2024-04-22T13:30:48.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.454+0000 7f4f4cdb3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5598447281b0 con 0x5598447c74d0 2024-04-22T13:30:48.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.454+0000 7f4f437fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598447c7e80 0x5598447d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.454+0000 7f4f437fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598447c7e80 0x5598447d0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42502/0 (socket says 172.21.15.12:42502) 2024-04-22T13:30:48.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.454+0000 7f4f437fe640 1 -- 172.21.15.12:0/2163167887 learned_addr learned my addr 172.21.15.12:0/2163167887 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:48.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.454+0000 7f4f42ffd640 1 --2- 172.21.15.12:0/2163167887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5598447d0a00 0x5598447d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.454+0000 7f4f42ffd640 1 --2- 172.21.15.12:0/2163167887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598447c74d0 0x5598447c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.454+0000 7f4f40ff9640 1 --2- 172.21.15.12:0/89165936 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4f2c0a3da0 0x7f4f2c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.454+0000 7f4f437fe640 1 -- 172.21.15.12:0/2163167887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598447c74d0 msgr2=0x5598447c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.454+0000 7f4f437fe640 1 --2- 172.21.15.12:0/2163167887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598447c74d0 0x5598447c78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.457 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.455+0000 7f4f437fe640 1 -- 172.21.15.12:0/2163167887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5598447d0a00 msgr2=0x5598447d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.455+0000 7f4f437fe640 1 --2- 172.21.15.12:0/2163167887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5598447d0a00 0x5598447d2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.455+0000 7f4f437fe640 1 -- 172.21.15.12:0/2163167887 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598446bb1b0 con 0x5598447c7e80 2024-04-22T13:30:48.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.455+0000 7f4f42ffd640 1 --2- 172.21.15.12:0/2163167887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5598447d0a00 0x5598447d2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:30:48.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.455+0000 7f4f43fff640 1 --2- 172.21.15.12:0/89165936 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4f2c0a3da0 0x7f4f2c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.455+0000 7f4f40ff9640 1 -- 172.21.15.12:0/89165936 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f4f3c0f2f10 con 0x7f4f4419ce90 2024-04-22T13:30:48.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.455+0000 7f4f437fe640 1 --2- 172.21.15.12:0/2163167887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598447c7e80 0x5598447d0430 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f4f3400bb10 tx=0x7f4f3400d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=226dd01b253c9f6b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:48.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.455+0000 7f4f29ffb640 1 -- 172.21.15.12:0/2163167887 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34017420 con 0x5598447c7e80 2024-04-22T13:30:48.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.455+0000 7f4f29ffb640 1 -- 172.21.15.12:0/2163167887 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f34017a70 con 0x5598447c7e80 2024-04-22T13:30:48.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.455+0000 7f4f29ffb640 1 -- 172.21.15.12:0/2163167887 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3400f980 con 0x5598447c7e80 2024-04-22T13:30:48.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.456+0000 7f4f4cdb3740 1 -- 172.21.15.12:0/2163167887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598447c7e80 msgr2=0x5598447d0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.456+0000 7f4f43fff640 1 --2- 172.21.15.12:0/89165936 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4f2c0a3da0 0x7f4f2c0a6260 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0f0870 tx=0x7f4f3c09d040 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:48.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.456+0000 7f4f4cdb3740 1 --2- 172.21.15.12:0/2163167887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598447c7e80 0x5598447d0430 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f4f3400bb10 tx=0x7f4f3400d340 comp rx=0 tx=0).stop 2024-04-22T13:30:48.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.456+0000 7f4f4cdb3740 1 -- 172.21.15.12:0/2163167887 shutdown_connections 2024-04-22T13:30:48.458 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.456+0000 7f4f4cdb3740 1 --2- 172.21.15.12:0/2163167887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5598447d0a00 0x5598447d2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.456+0000 7f4f4cdb3740 1 --2- 172.21.15.12:0/2163167887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598447c7e80 0x5598447d0430 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.456+0000 7f4f4cdb3740 1 --2- 172.21.15.12:0/2163167887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598447c74d0 0x5598447c78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.456+0000 7f4f4cdb3740 1 -- 172.21.15.12:0/2163167887 >> 172.21.15.12:0/2163167887 conn(0x5598447c1ba0 msgr2=0x5598447c1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:48.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.456+0000 7f4f4cdb3740 1 -- 172.21.15.12:0/2163167887 shutdown_connections 2024-04-22T13:30:48.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.456+0000 7f4f4cdb3740 1 -- 172.21.15.12:0/2163167887 wait complete. 2024-04-22T13:30:48.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.456+0000 7f4f4cdb3740 1 -- start start 2024-04-22T13:30:48.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.456+0000 7f4f4cdb3740 1 Processor -- start 2024-04-22T13:30:48.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.456+0000 7f4f4cdb3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5598447c74d0 0x5598447d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.456+0000 7f4f4cdb3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598447c7e80 0x5598447cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.456+0000 7f4f4cdb3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598447d0a00 0x5598447cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.456+0000 7f4f4cdb3740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5598447d3970 con 0x5598447d0a00 2024-04-22T13:30:48.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.456+0000 7f4f4cdb3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5598446e5500 con 0x5598447c74d0 2024-04-22T13:30:48.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.456+0000 7f4f4cdb3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5598447d3af0 con 0x5598447c7e80 2024-04-22T13:30:48.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.457+0000 7f4f4cdb3740 10 client.0 _ll_register_callbacks cb 0x5598446b3b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:48.459 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.457+0000 7f4f42ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598447d0a00 0x5598447cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.457+0000 7f4f42ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598447d0a00 0x5598447cdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42504/0 (socket says 172.21.15.12:42504) 2024-04-22T13:30:48.460 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.457+0000 7f4f42ffd640 1 -- 172.21.15.12:0/4046942374 learned_addr learned my addr 172.21.15.12:0/4046942374 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:48.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.457+0000 7f4f437fe640 1 --2- 172.21.15.12:0/4046942374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598447c7e80 0x5598447cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.457+0000 7f4f42ffd640 1 --2- 172.21.15.12:0/4046942374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5598447c74d0 0x5598447d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.457+0000 7f4f42ffd640 1 -- 172.21.15.12:0/4046942374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598447c7e80 msgr2=0x5598447cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.457+0000 7f4f42ffd640 1 --2- 172.21.15.12:0/4046942374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598447c7e80 0x5598447cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.457+0000 7f4f42ffd640 1 -- 172.21.15.12:0/4046942374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5598447c74d0 msgr2=0x5598447d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.457+0000 7f4f42ffd640 1 --2- 172.21.15.12:0/4046942374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5598447c74d0 0x5598447d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.457+0000 7f4f42ffd640 1 -- 172.21.15.12:0/4046942374 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55984443e3f0 con 0x5598447d0a00 2024-04-22T13:30:48.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.457+0000 7f4f42ffd640 1 --2- 172.21.15.12:0/4046942374 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598447d0a00 0x5598447cdb60 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f4f38012ba0 tx=0x7f4f38004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68adbbdd295c1880 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:48.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.457+0000 7f4f22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:48.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.457+0000 7f4f22ffd640 1 -- 172.21.15.12:0/4046942374 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3800dd10 con 0x5598447d0a00 2024-04-22T13:30:48.462 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.457+0000 7f4f22ffd640 1 -- 172.21.15.12:0/4046942374 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f4f3801f430 con 0x5598447d0a00 2024-04-22T13:30:48.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.457+0000 7f4f4cdb3740 1 -- 172.21.15.12:0/4046942374 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f34017a70 con 0x5598447d0a00 2024-04-22T13:30:48.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.457+0000 7f4f22ffd640 1 -- 172.21.15.12:0/4046942374 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4f3801f630 con 0x5598447d0a00 2024-04-22T13:30:48.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.457+0000 7f4f4cdb3740 10 client.18129 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:48.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.457+0000 7f4f4cdb3740 1 -- 172.21.15.12:0/4046942374 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5598445087d0 con 0x5598447d0a00 2024-04-22T13:30:48.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.458+0000 7f4f22ffd640 1 -- 172.21.15.12:0/4046942374 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f34017a70 con 0x5598447d0a00 2024-04-22T13:30:48.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.458+0000 7f4f22ffd640 1 -- 172.21.15.12:0/4046942374 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4f38010560 con 0x5598447d0a00 2024-04-22T13:30:48.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.458+0000 7f4f4cdb3740 10 client.18129 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:48.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.458+0000 7f4f4cdb3740 10 client.18129 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:30:48.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.458+0000 7f4f4cdb3740 20 client.18129 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:48.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.458+0000 7f4f4cdb3740 1 --2- 172.21.15.12:0/4046942374 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5598447e0be0 0x5598447e3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.458+0000 7f4f4cdb3740 4 client.18129 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:30:48.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.458+0000 7f4f4cdb3740 1 -- 172.21.15.12:0/4046942374 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598446cb930 con 0x5598447e0be0 2024-04-22T13:30:48.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.458+0000 7f4f437fe640 1 --2- 172.21.15.12:0/4046942374 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5598447e0be0 0x5598447e3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.463 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.459+0000 7f4f22ffd640 10 client.18129 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:30:48.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.459+0000 7f4f437fe640 1 --2- 172.21.15.12:0/4046942374 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5598447e0be0 0x5598447e3040 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=61344b61dbb25aa2 server_cookie=1f96df79d5f86952 in_seq=0 out_seq=0 2024-04-22T13:30:48.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.459+0000 7f4f22ffd640 1 -- 172.21.15.12:0/4046942374 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5598446cb930 con 0x5598447e0be0 2024-04-22T13:30:48.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.459+0000 7f4f22ffd640 10 client.18129 handle_command_reply: tid=0 2024-04-22T13:30:48.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.459+0000 7f4f4cdb3740 1 client.18129 shutdown 2024-04-22T13:30:48.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.460+0000 7f4f4cdb3740 1 -- 172.21.15.12:0/4046942374 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598447d0a00 msgr2=0x5598447cdb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.460+0000 7f4f4cdb3740 1 --2- 172.21.15.12:0/4046942374 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598447d0a00 0x5598447cdb60 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f4f38012ba0 tx=0x7f4f38004730 comp rx=0 tx=0).stop 2024-04-22T13:30:48.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.460+0000 7f4f4cdb3740 1 -- 172.21.15.12:0/4046942374 shutdown_connections 2024-04-22T13:30:48.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.460+0000 7f4f4cdb3740 1 --2- 172.21.15.12:0/4046942374 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5598447e0be0 0x5598447e3040 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.460+0000 7f4f4cdb3740 1 --2- 172.21.15.12:0/4046942374 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598447d0a00 0x5598447cdb60 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.460+0000 7f4f4cdb3740 1 --2- 172.21.15.12:0/4046942374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598447c7e80 0x5598447cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.460+0000 7f4f22ffd640 0 client.18129 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:30:48.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.460+0000 7f4f4cdb3740 1 --2- 172.21.15.12:0/4046942374 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5598447c74d0 0x5598447d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.460+0000 7f4f4cdb3740 1 -- 172.21.15.12:0/4046942374 >> 172.21.15.12:0/4046942374 conn(0x5598447c1ba0 msgr2=0x5598447c38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:48.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.460+0000 7f4f4cdb3740 1 -- 172.21.15.12:0/4046942374 shutdown_connections 2024-04-22T13:30:48.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.460+0000 7f4f4cdb3740 1 -- 172.21.15.12:0/4046942374 wait complete. 2024-04-22T13:30:48.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.460+0000 7f4f4cdb3740 20 client.18129 trim_cache size 0 max 16384 2024-04-22T13:30:48.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.473+0000 7f4f4cdb3740 1 Processor -- start 2024-04-22T13:30:48.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.473+0000 7f4f4cdb3740 1 -- start start 2024-04-22T13:30:48.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.473+0000 7f4f4cdb3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598447c1ba0 0x5598447dc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.473+0000 7f4f4cdb3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598447c74d0 0x5598447c36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.473+0000 7f4f4cdb3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5598447c7e80 0x5598447cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.473+0000 7f4f4cdb3740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4f441ad970 con 0x5598447c74d0 2024-04-22T13:30:48.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.473+0000 7f4f4cdb3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4f441af4b0 con 0x5598447c7e80 2024-04-22T13:30:48.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.473+0000 7f4f4cdb3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4f441ae040 con 0x5598447c1ba0 2024-04-22T13:30:48.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.473+0000 7f4f437fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598447c74d0 0x5598447c36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.473+0000 7f4f437fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598447c74d0 0x5598447c36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42514/0 (socket says 172.21.15.12:42514) 2024-04-22T13:30:48.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.473+0000 7f4f437fe640 1 -- 172.21.15.12:0/2457122215 learned_addr learned my addr 172.21.15.12:0/2457122215 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:48.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.473+0000 7f4f42ffd640 1 --2- 172.21.15.12:0/2457122215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598447c1ba0 0x5598447dc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.473+0000 7f4f437fe640 1 -- 172.21.15.12:0/2457122215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598447c1ba0 msgr2=0x5598447dc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.473+0000 7f4f437fe640 1 --2- 172.21.15.12:0/2457122215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598447c1ba0 0x5598447dc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.473+0000 7f4f437fe640 1 -- 172.21.15.12:0/2457122215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5598447c7e80 msgr2=0x5598447cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.473+0000 7f4f437fe640 1 --2- 172.21.15.12:0/2457122215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5598447c7e80 0x5598447cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.473+0000 7f4f437fe640 1 -- 172.21.15.12:0/2457122215 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598447cbb70 con 0x5598447c74d0 2024-04-22T13:30:48.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.474+0000 7f4f42ffd640 1 --2- 172.21.15.12:0/2457122215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598447c1ba0 0x5598447dc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:30:48.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.474+0000 7f4f437fe640 1 --2- 172.21.15.12:0/2457122215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598447c74d0 0x5598447c36b0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f4f34019040 tx=0x7f4f3400d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2aeb3302c34b40f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:48.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.474+0000 7f4f297fa640 1 -- 172.21.15.12:0/2457122215 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34028d50 con 0x5598447c74d0 2024-04-22T13:30:48.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.474+0000 7f4f297fa640 1 -- 172.21.15.12:0/2457122215 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f3400de50 con 0x5598447c74d0 2024-04-22T13:30:48.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.474+0000 7f4f297fa640 1 -- 172.21.15.12:0/2457122215 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34028920 con 0x5598447c74d0 2024-04-22T13:30:48.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.474+0000 7f4f4cdb3740 1 -- 172.21.15.12:0/2457122215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598447c74d0 msgr2=0x5598447c36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.474+0000 7f4f4cdb3740 1 --2- 172.21.15.12:0/2457122215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598447c74d0 0x5598447c36b0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f4f34019040 tx=0x7f4f3400d340 comp rx=0 tx=0).stop 2024-04-22T13:30:48.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.474+0000 7f4f4cdb3740 1 -- 172.21.15.12:0/2457122215 shutdown_connections 2024-04-22T13:30:48.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.474+0000 7f4f4cdb3740 1 --2- 172.21.15.12:0/2457122215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5598447c7e80 0x5598447cfd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.474+0000 7f4f4cdb3740 1 --2- 172.21.15.12:0/2457122215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598447c74d0 0x5598447c36b0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.474+0000 7f4f4cdb3740 1 --2- 172.21.15.12:0/2457122215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598447c1ba0 0x5598447dc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.474+0000 7f4f4cdb3740 1 -- 172.21.15.12:0/2457122215 >> 172.21.15.12:0/2457122215 conn(0x5598447cdcc0 msgr2=0x5598447c2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:48.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.474+0000 7f4f4cdb3740 1 -- 172.21.15.12:0/2457122215 shutdown_connections 2024-04-22T13:30:48.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.474+0000 7f4f4cdb3740 1 -- 172.21.15.12:0/2457122215 wait complete. 2024-04-22T13:30:48.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.475+0000 7f4f4cdb3740 1 -- start start 2024-04-22T13:30:48.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.475+0000 7f4f4cdb3740 1 Processor -- start 2024-04-22T13:30:48.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.475+0000 7f4f4cdb3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598447c1ba0 0x5598447d9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.475+0000 7f4f4cdb3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598447c74d0 0x5598447d4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.475+0000 7f4f4cdb3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5598447c7e80 0x5598447d4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.475+0000 7f4f4cdb3740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4f441aede0 con 0x5598447c1ba0 2024-04-22T13:30:48.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.475+0000 7f4f4cdb3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4f441ae710 con 0x5598447c7e80 2024-04-22T13:30:48.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.475+0000 7f4f4cdb3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4f441ab950 con 0x5598447c74d0 2024-04-22T13:30:48.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.475+0000 7f4f4cdb3740 10 client.0 _ll_register_callbacks cb 0x7f4f38012470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:48.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.475+0000 7f4f42ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598447c1ba0 0x5598447d9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.475+0000 7f4f42ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598447c1ba0 0x5598447d9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42522/0 (socket says 172.21.15.12:42522) 2024-04-22T13:30:48.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.475+0000 7f4f42ffd640 1 -- 172.21.15.12:0/1440602337 learned_addr learned my addr 172.21.15.12:0/1440602337 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.475+0000 7f4f437fe640 1 --2- 172.21.15.12:0/1440602337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598447c74d0 0x5598447d4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.475+0000 7f4f42ffd640 1 --2- 172.21.15.12:0/1440602337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5598447c7e80 0x5598447d4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.476+0000 7f4f42ffd640 1 -- 172.21.15.12:0/1440602337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598447c74d0 msgr2=0x5598447d4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.476+0000 7f4f42ffd640 1 --2- 172.21.15.12:0/1440602337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598447c74d0 0x5598447d4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.476+0000 7f4f42ffd640 1 -- 172.21.15.12:0/1440602337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5598447c7e80 msgr2=0x5598447d4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.476+0000 7f4f42ffd640 1 --2- 172.21.15.12:0/1440602337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5598447c7e80 0x5598447d4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.476+0000 7f4f42ffd640 1 -- 172.21.15.12:0/1440602337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598447d5240 con 0x5598447c1ba0 2024-04-22T13:30:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.476+0000 7f4f42ffd640 1 --2- 172.21.15.12:0/1440602337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598447c1ba0 0x5598447d9190 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f4f38012ba0 tx=0x7f4f3800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f58f1b573681d8c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.476+0000 7f4f22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.476+0000 7f4f22ffd640 1 -- 172.21.15.12:0/1440602337 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f38005e30 con 0x5598447c1ba0 2024-04-22T13:30:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.476+0000 7f4f4cdb3740 1 -- 172.21.15.12:0/1440602337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f3400de50 con 0x5598447c1ba0 2024-04-22T13:30:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.476+0000 7f4f22ffd640 1 -- 172.21.15.12:0/1440602337 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f4f38010db0 con 0x5598447c1ba0 2024-04-22T13:30:48.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.476+0000 7f4f22ffd640 1 -- 172.21.15.12:0/1440602337 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4f380103d0 con 0x5598447c1ba0 2024-04-22T13:30:48.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.476+0000 7f4f4cdb3740 10 client.18135 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:48.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.476+0000 7f4f4cdb3740 1 -- 172.21.15.12:0/1440602337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559844847120 con 0x5598447c1ba0 2024-04-22T13:30:48.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.476+0000 7f4f22ffd640 1 -- 172.21.15.12:0/1440602337 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f3400de50 con 0x5598447c1ba0 2024-04-22T13:30:48.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.476+0000 7f4f22ffd640 1 -- 172.21.15.12:0/1440602337 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4f380047a0 con 0x5598447c1ba0 2024-04-22T13:30:48.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.476+0000 7f4f4cdb3740 10 client.18135 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:48.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.476+0000 7f4f4cdb3740 10 client.18135 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:30:48.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.476+0000 7f4f4cdb3740 20 client.18135 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:48.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.476+0000 7f4f4cdb3740 1 --2- 172.21.15.12:0/1440602337 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5598447d0a00 0x5598448497a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.476+0000 7f4f4cdb3740 4 client.18135 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:30:48.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.476+0000 7f4f4cdb3740 1 -- 172.21.15.12:0/1440602337 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559844739c10 con 0x5598447d0a00 2024-04-22T13:30:48.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.477+0000 7f4f437fe640 1 --2- 172.21.15.12:0/1440602337 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5598447d0a00 0x5598448497a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.477+0000 7f4f437fe640 1 --2- 172.21.15.12:0/1440602337 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5598447d0a00 0x5598448497a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6f5c78dc8296031 server_cookie=ed79b8c6eec10440 in_seq=0 out_seq=0 2024-04-22T13:30:48.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.477+0000 7f4f22ffd640 10 client.18135 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:30:48.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.477+0000 7f4f22ffd640 1 -- 172.21.15.12:0/1440602337 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559844739c10 con 0x5598447d0a00 2024-04-22T13:30:48.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.477+0000 7f4f22ffd640 10 client.18135 handle_command_reply: tid=0 2024-04-22T13:30:48.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.477+0000 7f4f4cdb3740 1 client.18135 shutdown 2024-04-22T13:30:48.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.478+0000 7f4f4cdb3740 1 -- 172.21.15.12:0/1440602337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598447c1ba0 msgr2=0x5598447d9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.478+0000 7f4f4cdb3740 1 --2- 172.21.15.12:0/1440602337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598447c1ba0 0x5598447d9190 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f4f38012ba0 tx=0x7f4f3800dbe0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.478+0000 7f4f4cdb3740 1 -- 172.21.15.12:0/1440602337 shutdown_connections 2024-04-22T13:30:48.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.478+0000 7f4f4cdb3740 1 --2- 172.21.15.12:0/1440602337 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5598447d0a00 0x5598448497a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.478+0000 7f4f4cdb3740 1 --2- 172.21.15.12:0/1440602337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5598447c7e80 0x5598447d4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.478+0000 7f4f4cdb3740 1 --2- 172.21.15.12:0/1440602337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5598447c74d0 0x5598447d4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.478+0000 7f4f22ffd640 0 client.18135 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:30:48.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.478+0000 7f4f4cdb3740 1 --2- 172.21.15.12:0/1440602337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5598447c1ba0 0x5598447d9190 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.478+0000 7f4f4cdb3740 1 -- 172.21.15.12:0/1440602337 >> 172.21.15.12:0/1440602337 conn(0x5598447cdcc0 msgr2=0x5598447c0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:48.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.478+0000 7f4f4cdb3740 1 -- 172.21.15.12:0/1440602337 shutdown_connections 2024-04-22T13:30:48.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.478+0000 7f4f4cdb3740 1 -- 172.21.15.12:0/1440602337 wait complete. 2024-04-22T13:30:48.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.478+0000 7f4f4cdb3740 20 client.18135 trim_cache size 0 max 16384 2024-04-22T13:30:48.481 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:30:48.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.478+0000 7f4f4a0e4640 1 -- 172.21.15.12:0/89165936 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4f2c0a3da0 msgr2=0x7f4f2c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.478+0000 7f4f4a0e4640 1 --2- 172.21.15.12:0/89165936 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4f2c0a3da0 0x7f4f2c0a6260 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0f0870 tx=0x7f4f3c09d040 comp rx=0 tx=0).stop 2024-04-22T13:30:48.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.478+0000 7f4f4a0e4640 1 -- 172.21.15.12:0/89165936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f4419ce90 msgr2=0x7f4f4419d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.478+0000 7f4f4a0e4640 1 --2- 172.21.15.12:0/89165936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f4419ce90 0x7f4f4419d340 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0098d0 tx=0x7f4f3c035d60 comp rx=0 tx=0).stop 2024-04-22T13:30:48.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.478+0000 7f4f4a0e4640 1 -- 172.21.15.12:0/89165936 shutdown_connections 2024-04-22T13:30:48.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.478+0000 7f4f4a0e4640 1 --2- 172.21.15.12:0/89165936 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4f2c0a3da0 0x7f4f2c0a6260 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.478+0000 7f4f4a0e4640 1 --2- 172.21.15.12:0/89165936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f4419ce90 0x7f4f4419d340 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.478+0000 7f4f4a0e4640 1 --2- 172.21.15.12:0/89165936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4f44107260 0x7f4f44198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.478+0000 7f4f4a0e4640 1 --2- 172.21.15.12:0/89165936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4f44102190 0x7f4f441985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.478+0000 7f4f4a0e4640 1 -- 172.21.15.12:0/89165936 >> 172.21.15.12:0/89165936 conn(0x7f4f440fd990 msgr2=0x7f4f440fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:48.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.479+0000 7f4f4a0e4640 1 -- 172.21.15.12:0/89165936 shutdown_connections 2024-04-22T13:30:48.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.479+0000 7f4f4a0e4640 1 -- 172.21.15.12:0/89165936 wait complete. 2024-04-22T13:30:48.485 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:30:48.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.593+0000 7f54adae0640 1 -- 172.21.15.12:0/530689389 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54a8107270 msgr2=0x7f54a810ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.593+0000 7f54adae0640 1 --2- 172.21.15.12:0/530689389 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54a8107270 0x7f54a810ee40 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f54a0009ca0 tx=0x7f54a002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.593+0000 7f54adae0640 1 -- 172.21.15.12:0/530689389 shutdown_connections 2024-04-22T13:30:48.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.593+0000 7f54adae0640 1 --2- 172.21.15.12:0/530689389 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54a8107270 0x7f54a810ee40 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.589 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.593+0000 7f54adae0640 1 --2- 172.21.15.12:0/530689389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54a806b1f0 0x7f54a8106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.593+0000 7f54adae0640 1 --2- 172.21.15.12:0/530689389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54a806a840 0x7f54a806ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.593+0000 7f54adae0640 1 -- 172.21.15.12:0/530689389 >> 172.21.15.12:0/530689389 conn(0x7f54a8077eb0 msgr2=0x7f54a80782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:48.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.593+0000 7f54adae0640 1 -- 172.21.15.12:0/530689389 shutdown_connections 2024-04-22T13:30:48.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.593+0000 7f54adae0640 1 -- 172.21.15.12:0/530689389 wait complete. 2024-04-22T13:30:48.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.594+0000 7f54adae0640 1 Processor -- start 2024-04-22T13:30:48.590 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.594+0000 7f54adae0640 1 -- start start 2024-04-22T13:30:48.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.594+0000 7f54adae0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54a806a840 0x7f54a8100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.594+0000 7f54adae0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54a8107270 0x7f54a81013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.594+0000 7f54adae0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54a8101900 0x7f54a81a68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.594+0000 7f54adae0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f54a81121b0 con 0x7f54a8107270 2024-04-22T13:30:48.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.594+0000 7f54adae0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f54a8112030 con 0x7f54a806a840 2024-04-22T13:30:48.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.594+0000 7f54adae0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54a8112330 con 0x7f54a8101900 2024-04-22T13:30:48.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.595+0000 7f54a6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54a8107270 0x7f54a81013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.595+0000 7f54a6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54a8107270 0x7f54a81013c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42536/0 (socket says 172.21.15.12:42536) 2024-04-22T13:30:48.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.595+0000 7f54a6ffd640 1 -- 172.21.15.12:0/2158704264 learned_addr learned my addr 172.21.15.12:0/2158704264 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:48.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.595+0000 7f54a7fff640 1 --2- 172.21.15.12:0/2158704264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54a8101900 0x7f54a81a68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.595+0000 7f54a6ffd640 1 -- 172.21.15.12:0/2158704264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54a8101900 msgr2=0x7f54a81a68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.595+0000 7f54a6ffd640 1 --2- 172.21.15.12:0/2158704264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54a8101900 0x7f54a81a68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.595+0000 7f54a6ffd640 1 -- 172.21.15.12:0/2158704264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54a806a840 msgr2=0x7f54a8100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.595+0000 7f54a77fe640 1 --2- 172.21.15.12:0/2158704264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54a806a840 0x7f54a8100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.595+0000 7f54a6ffd640 1 --2- 172.21.15.12:0/2158704264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54a806a840 0x7f54a8100e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.595+0000 7f54a6ffd640 1 -- 172.21.15.12:0/2158704264 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a81a6f40 con 0x7f54a8107270 2024-04-22T13:30:48.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.595+0000 7f54a6ffd640 1 --2- 172.21.15.12:0/2158704264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54a8107270 0x7f54a81013c0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f549c005fd0 tx=0x7f549c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:48.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.595+0000 7f54a4ff9640 1 -- 172.21.15.12:0/2158704264 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c017070 con 0x7f54a8107270 2024-04-22T13:30:48.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.595+0000 7f54a4ff9640 1 -- 172.21.15.12:0/2158704264 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f549c009c30 con 0x7f54a8107270 2024-04-22T13:30:48.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.595+0000 7f54a4ff9640 1 -- 172.21.15.12:0/2158704264 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c003d30 con 0x7f54a8107270 2024-04-22T13:30:48.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.595+0000 7f54adae0640 1 -- 172.21.15.12:0/2158704264 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54a81a7230 con 0x7f54a8107270 2024-04-22T13:30:48.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.595+0000 7f54adae0640 1 -- 172.21.15.12:0/2158704264 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54a81a7730 con 0x7f54a8107270 2024-04-22T13:30:48.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.597+0000 7f54a4ff9640 1 -- 172.21.15.12:0/2158704264 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f549c003ed0 con 0x7f54a8107270 2024-04-22T13:30:48.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.598+0000 7f54b07af740 1 Processor -- start 2024-04-22T13:30:48.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.598+0000 7f54b07af740 1 -- start start 2024-04-22T13:30:48.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.598+0000 7f54b07af740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623b92e94d0 0x5623b92e98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.598+0000 7f54b07af740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623b92e9e80 0x5623b92f2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.598+0000 7f54b07af740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623b92f2a00 0x5623b92f4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.598+0000 7f54b07af740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5623b924e1f0 con 0x5623b92f2a00 2024-04-22T13:30:48.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.598+0000 7f54b07af740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5623b91eb4b0 con 0x5623b92e9e80 2024-04-22T13:30:48.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.598+0000 7f54b07af740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5623b924a1b0 con 0x5623b92e94d0 2024-04-22T13:30:48.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.598+0000 7f54a6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623b92f2a00 0x5623b92f4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.598+0000 7f54a6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623b92f2a00 0x5623b92f4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42538/0 (socket says 172.21.15.12:42538) 2024-04-22T13:30:48.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.598+0000 7f54a6ffd640 1 -- 172.21.15.12:0/2172884135 learned_addr learned my addr 172.21.15.12:0/2172884135 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:48.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.598+0000 7f54a4ff9640 1 --2- 172.21.15.12:0/2158704264 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f54900a3da0 0x7f54900a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.598+0000 7f54a7fff640 1 --2- 172.21.15.12:0/2172884135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623b92e94d0 0x5623b92e98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.598+0000 7f54a77fe640 1 --2- 172.21.15.12:0/2172884135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623b92e9e80 0x5623b92f2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.598+0000 7f54a7fff640 1 --2- 172.21.15.12:0/2158704264 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f54900a3da0 0x7f54900a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.598+0000 7f54a4ff9640 1 -- 172.21.15.12:0/2158704264 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f549c0d0460 con 0x7f54a8107270 2024-04-22T13:30:48.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.598+0000 7f54a6ffd640 1 -- 172.21.15.12:0/2172884135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623b92e94d0 msgr2=0x5623b92e98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.598+0000 7f54a6ffd640 1 --2- 172.21.15.12:0/2172884135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623b92e94d0 0x5623b92e98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.598+0000 7f54a6ffd640 1 -- 172.21.15.12:0/2172884135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623b92e9e80 msgr2=0x5623b92f2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.598+0000 7f54a6ffd640 1 --2- 172.21.15.12:0/2172884135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623b92e9e80 0x5623b92f2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.598+0000 7f54a6ffd640 1 -- 172.21.15.12:0/2172884135 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623b91dd1b0 con 0x5623b92f2a00 2024-04-22T13:30:48.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.598+0000 7f54a77fe640 1 --2- 172.21.15.12:0/2172884135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623b92e9e80 0x5623b92f2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:48.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.598+0000 7f54a7fff640 1 --2- 172.21.15.12:0/2172884135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623b92e94d0 0x5623b92e98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:48.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.599+0000 7f54a6ffd640 1 --2- 172.21.15.12:0/2172884135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623b92f2a00 0x5623b92f4e50 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f549c0cdb10 tx=0x7f549c00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0ffb2b5e284ac0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:48.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.599+0000 7f548dffb640 1 -- 172.21.15.12:0/2172884135 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c09c3f0 con 0x5623b92f2a00 2024-04-22T13:30:48.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.599+0000 7f54a7fff640 1 --2- 172.21.15.12:0/2158704264 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f54900a3da0 0x7f54900a6260 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f54a0009430 tx=0x7f54a00095c0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:48.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.599+0000 7f548dffb640 1 -- 172.21.15.12:0/2172884135 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f549c0a07a0 con 0x5623b92f2a00 2024-04-22T13:30:48.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.599+0000 7f548dffb640 1 -- 172.21.15.12:0/2172884135 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c0cde20 con 0x5623b92f2a00 2024-04-22T13:30:48.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.600+0000 7f54b07af740 1 -- 172.21.15.12:0/2172884135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623b92f2a00 msgr2=0x5623b92f4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.600+0000 7f54b07af740 1 --2- 172.21.15.12:0/2172884135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623b92f2a00 0x5623b92f4e50 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f549c0cdb10 tx=0x7f549c00bf60 comp rx=0 tx=0).stop 2024-04-22T13:30:48.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.600+0000 7f54b07af740 1 -- 172.21.15.12:0/2172884135 shutdown_connections 2024-04-22T13:30:48.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.600+0000 7f54b07af740 1 --2- 172.21.15.12:0/2172884135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623b92f2a00 0x5623b92f4e50 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.600+0000 7f54b07af740 1 --2- 172.21.15.12:0/2172884135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623b92e9e80 0x5623b92f2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.600+0000 7f54b07af740 1 --2- 172.21.15.12:0/2172884135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623b92e94d0 0x5623b92e98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.600+0000 7f54b07af740 1 -- 172.21.15.12:0/2172884135 >> 172.21.15.12:0/2172884135 conn(0x5623b92e3ba0 msgr2=0x5623b92e3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:48.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.600+0000 7f54b07af740 1 -- 172.21.15.12:0/2172884135 shutdown_connections 2024-04-22T13:30:48.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.600+0000 7f54b07af740 1 -- 172.21.15.12:0/2172884135 wait complete. 2024-04-22T13:30:48.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.600+0000 7f54b07af740 1 -- start start 2024-04-22T13:30:48.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.600+0000 7f54b07af740 1 Processor -- start 2024-04-22T13:30:48.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.600+0000 7f54b07af740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623b92e94d0 0x5623b92f22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.600+0000 7f54b07af740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623b92e9e80 0x5623b92ef680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.600+0000 7f54b07af740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623b92f2a00 0x5623b92efc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.600+0000 7f54b07af740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5623b92f5910 con 0x5623b92e9e80 2024-04-22T13:30:48.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.600+0000 7f54b07af740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5623b9207500 con 0x5623b92f2a00 2024-04-22T13:30:48.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.600+0000 7f54b07af740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5623b92f5a90 con 0x5623b92e94d0 2024-04-22T13:30:48.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.600+0000 7f54b07af740 10 client.0 _ll_register_callbacks cb 0x5623b91d5b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:48.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.601+0000 7f54a6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623b92e9e80 0x5623b92ef680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.601+0000 7f54a6ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623b92e9e80 0x5623b92ef680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42552/0 (socket says 172.21.15.12:42552) 2024-04-22T13:30:48.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.601+0000 7f54a6ffd640 1 -- 172.21.15.12:0/3548362632 learned_addr learned my addr 172.21.15.12:0/3548362632 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:48.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.601+0000 7f54a77fe640 1 --2- 172.21.15.12:0/3548362632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623b92e94d0 0x5623b92f22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.601+0000 7f54a7fff640 1 --2- 172.21.15.12:0/3548362632 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623b92f2a00 0x5623b92efc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.601+0000 7f54a6ffd640 1 -- 172.21.15.12:0/3548362632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623b92e94d0 msgr2=0x5623b92f22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.601+0000 7f54a6ffd640 1 --2- 172.21.15.12:0/3548362632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623b92e94d0 0x5623b92f22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.601+0000 7f54a6ffd640 1 -- 172.21.15.12:0/3548362632 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623b92f2a00 msgr2=0x5623b92efc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.601+0000 7f54a6ffd640 1 --2- 172.21.15.12:0/3548362632 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623b92f2a00 0x5623b92efc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.601+0000 7f54a6ffd640 1 -- 172.21.15.12:0/3548362632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623b8f603f0 con 0x5623b92e9e80 2024-04-22T13:30:48.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.601+0000 7f54a77fe640 1 --2- 172.21.15.12:0/3548362632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623b92e94d0 0x5623b92f22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:48.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.601+0000 7f54a7fff640 1 --2- 172.21.15.12:0/3548362632 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623b92f2a00 0x5623b92efc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:30:48.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.601+0000 7f54a6ffd640 1 --2- 172.21.15.12:0/3548362632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623b92e9e80 0x5623b92ef680 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f549c012040 tx=0x7f549c0d3d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42183891fff45a99 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:48.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.601+0000 7f5486ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:48.605 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.601+0000 7f5486ffd640 1 -- 172.21.15.12:0/3548362632 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c0d3e60 con 0x5623b92e9e80 2024-04-22T13:30:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.601+0000 7f5486ffd640 1 -- 172.21.15.12:0/3548362632 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f549c098a60 con 0x5623b92e9e80 2024-04-22T13:30:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.601+0000 7f54b07af740 1 -- 172.21.15.12:0/3548362632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f549c0a07a0 con 0x5623b92e9e80 2024-04-22T13:30:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.601+0000 7f5486ffd640 1 -- 172.21.15.12:0/3548362632 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f549c0cde20 con 0x5623b92e9e80 2024-04-22T13:30:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.601+0000 7f54b07af740 10 client.18147 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.601+0000 7f54b07af740 1 -- 172.21.15.12:0/3548362632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5623b902a7d0 con 0x5623b92e9e80 2024-04-22T13:30:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.601+0000 7f5486ffd640 1 -- 172.21.15.12:0/3548362632 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f549c0a07a0 con 0x5623b92e9e80 2024-04-22T13:30:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.602+0000 7f5486ffd640 1 -- 172.21.15.12:0/3548362632 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f549c0cf070 con 0x5623b92e9e80 2024-04-22T13:30:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.602+0000 7f54b07af740 10 client.18147 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.602+0000 7f54b07af740 10 client.18147 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:30:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.602+0000 7f54b07af740 20 client.18147 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.602+0000 7f54b07af740 1 --2- 172.21.15.12:0/3548362632 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5623b9302a20 0x5623b9304e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.602+0000 7f54b07af740 4 client.18147 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:30:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.602+0000 7f54b07af740 1 -- 172.21.15.12:0/3548362632 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623b91ed930 con 0x5623b9302a20 2024-04-22T13:30:48.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.602+0000 7f54a77fe640 1 --2- 172.21.15.12:0/3548362632 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5623b9302a20 0x5623b9304e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.602+0000 7f5486ffd640 10 client.18147 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.602+0000 7f54a77fe640 1 --2- 172.21.15.12:0/3548362632 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5623b9302a20 0x5623b9304e10 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82669eb8b4e49c22 server_cookie=dcf6e24135285e94 in_seq=0 out_seq=0 2024-04-22T13:30:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.603+0000 7f5486ffd640 1 -- 172.21.15.12:0/3548362632 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5623b91ed930 con 0x5623b9302a20 2024-04-22T13:30:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.603+0000 7f5486ffd640 10 client.18147 handle_command_reply: tid=0 2024-04-22T13:30:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.603+0000 7f54b07af740 1 client.18147 shutdown 2024-04-22T13:30:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.604+0000 7f54b07af740 1 -- 172.21.15.12:0/3548362632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623b92e9e80 msgr2=0x5623b92ef680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.604+0000 7f54b07af740 1 --2- 172.21.15.12:0/3548362632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623b92e9e80 0x5623b92ef680 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f549c012040 tx=0x7f549c0d3d70 comp rx=0 tx=0).stop 2024-04-22T13:30:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.604+0000 7f54b07af740 1 -- 172.21.15.12:0/3548362632 shutdown_connections 2024-04-22T13:30:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.604+0000 7f54b07af740 1 --2- 172.21.15.12:0/3548362632 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5623b9302a20 0x5623b9304e10 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.604+0000 7f54b07af740 1 --2- 172.21.15.12:0/3548362632 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623b92f2a00 0x5623b92efc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.604+0000 7f54b07af740 1 --2- 172.21.15.12:0/3548362632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623b92e9e80 0x5623b92ef680 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.604+0000 7f5486ffd640 0 client.18147 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.604+0000 7f54b07af740 1 --2- 172.21.15.12:0/3548362632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623b92e94d0 0x5623b92f22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.604+0000 7f54b07af740 1 -- 172.21.15.12:0/3548362632 >> 172.21.15.12:0/3548362632 conn(0x5623b92e3ba0 msgr2=0x5623b92e58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:48.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.604+0000 7f54b07af740 1 -- 172.21.15.12:0/3548362632 shutdown_connections 2024-04-22T13:30:48.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.604+0000 7f54b07af740 1 -- 172.21.15.12:0/3548362632 wait complete. 2024-04-22T13:30:48.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.604+0000 7f54b07af740 20 client.18147 trim_cache size 0 max 16384 2024-04-22T13:30:48.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.617+0000 7f54b07af740 1 Processor -- start 2024-04-22T13:30:48.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.617+0000 7f54b07af740 1 -- start start 2024-04-22T13:30:48.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.617+0000 7f54b07af740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623b92e94d0 0x5623b92f1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.617+0000 7f54b07af740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623b92e9e80 0x5623b92f22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.617+0000 7f54b07af740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623b92f2a00 0x5623b9307e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.617+0000 7f54b07af740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f54a81afce0 con 0x5623b92f2a00 2024-04-22T13:30:48.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.617+0000 7f54b07af740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f54a81b1820 con 0x5623b92e94d0 2024-04-22T13:30:48.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.617+0000 7f54b07af740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54a81b03b0 con 0x5623b92e9e80 2024-04-22T13:30:48.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.617+0000 7f54a7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623b92f2a00 0x5623b9307e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.617+0000 7f54a7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623b92f2a00 0x5623b9307e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42564/0 (socket says 172.21.15.12:42564) 2024-04-22T13:30:48.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.617+0000 7f54a7fff640 1 -- 172.21.15.12:0/1133740086 learned_addr learned my addr 172.21.15.12:0/1133740086 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:48.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.617+0000 7f54a77fe640 1 --2- 172.21.15.12:0/1133740086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623b92e94d0 0x5623b92f1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.618+0000 7f54a6ffd640 1 --2- 172.21.15.12:0/1133740086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623b92e9e80 0x5623b92f22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.618+0000 7f54a7fff640 1 -- 172.21.15.12:0/1133740086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623b92e9e80 msgr2=0x5623b92f22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.618+0000 7f54a7fff640 1 --2- 172.21.15.12:0/1133740086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623b92e9e80 0x5623b92f22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.618+0000 7f54a7fff640 1 -- 172.21.15.12:0/1133740086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623b92e94d0 msgr2=0x5623b92f1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.618+0000 7f54a7fff640 1 --2- 172.21.15.12:0/1133740086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623b92e94d0 0x5623b92f1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.618+0000 7f54a7fff640 1 -- 172.21.15.12:0/1133740086 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623b92ff340 con 0x5623b92f2a00 2024-04-22T13:30:48.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.618+0000 7f54a6ffd640 1 --2- 172.21.15.12:0/1133740086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623b92e9e80 0x5623b92f22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:30:48.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.618+0000 7f54a77fe640 1 --2- 172.21.15.12:0/1133740086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623b92e94d0 0x5623b92f1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:48.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.618+0000 7f54a7fff640 1 --2- 172.21.15.12:0/1133740086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623b92f2a00 0x5623b9307e30 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f54a0009ca0 tx=0x7f54a003a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8732308dc50b16 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:48.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.618+0000 7f548d7fa640 1 -- 172.21.15.12:0/1133740086 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a0052070 con 0x5623b92f2a00 2024-04-22T13:30:48.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.618+0000 7f548d7fa640 1 -- 172.21.15.12:0/1133740086 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54a0003e20 con 0x5623b92f2a00 2024-04-22T13:30:48.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.618+0000 7f548d7fa640 1 -- 172.21.15.12:0/1133740086 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a0039040 con 0x5623b92f2a00 2024-04-22T13:30:48.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.618+0000 7f54b07af740 1 -- 172.21.15.12:0/1133740086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623b92f2a00 msgr2=0x5623b9307e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.618+0000 7f54b07af740 1 --2- 172.21.15.12:0/1133740086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623b92f2a00 0x5623b9307e30 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f54a0009ca0 tx=0x7f54a003a000 comp rx=0 tx=0).stop 2024-04-22T13:30:48.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.618+0000 7f54b07af740 1 -- 172.21.15.12:0/1133740086 shutdown_connections 2024-04-22T13:30:48.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.618+0000 7f54b07af740 1 --2- 172.21.15.12:0/1133740086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623b92f2a00 0x5623b9307e30 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.618+0000 7f54b07af740 1 --2- 172.21.15.12:0/1133740086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623b92e9e80 0x5623b92f22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.618+0000 7f54b07af740 1 --2- 172.21.15.12:0/1133740086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623b92e94d0 0x5623b92f1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.618+0000 7f54b07af740 1 -- 172.21.15.12:0/1133740086 >> 172.21.15.12:0/1133740086 conn(0x5623b92e3ba0 msgr2=0x5623b92e5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:48.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.619+0000 7f54b07af740 1 -- 172.21.15.12:0/1133740086 shutdown_connections 2024-04-22T13:30:48.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.619+0000 7f54b07af740 1 -- 172.21.15.12:0/1133740086 wait complete. 2024-04-22T13:30:48.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.619+0000 7f54b07af740 1 -- start start 2024-04-22T13:30:48.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.619+0000 7f54b07af740 1 Processor -- start 2024-04-22T13:30:48.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.619+0000 7f54b07af740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623b92e94d0 0x5623b9305b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.619+0000 7f54b07af740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623b92e9e80 0x5623b9306090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.619+0000 7f54b07af740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623b92f2a00 0x5623b93065d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.619+0000 7f54b07af740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f54a81b1150 con 0x5623b92e94d0 2024-04-22T13:30:48.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.619+0000 7f54b07af740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f54a81b0a80 con 0x5623b92e9e80 2024-04-22T13:30:48.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.619+0000 7f54b07af740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54a81adcc0 con 0x5623b92f2a00 2024-04-22T13:30:48.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.619+0000 7f54b07af740 10 client.0 _ll_register_callbacks cb 0x7f549c0a5060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:48.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.619+0000 7f54a77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623b92e94d0 0x5623b9305b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.619+0000 7f54a77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623b92e94d0 0x5623b9305b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42578/0 (socket says 172.21.15.12:42578) 2024-04-22T13:30:48.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.619+0000 7f54a77fe640 1 -- 172.21.15.12:0/3794066763 learned_addr learned my addr 172.21.15.12:0/3794066763 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:48.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.620+0000 7f54a6ffd640 1 --2- 172.21.15.12:0/3794066763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623b92e9e80 0x5623b9306090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.620+0000 7f54a7fff640 1 --2- 172.21.15.12:0/3794066763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623b92f2a00 0x5623b93065d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.620+0000 7f54a77fe640 1 -- 172.21.15.12:0/3794066763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623b92f2a00 msgr2=0x5623b93065d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.620+0000 7f54a77fe640 1 --2- 172.21.15.12:0/3794066763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623b92f2a00 0x5623b93065d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.620+0000 7f54a77fe640 1 -- 172.21.15.12:0/3794066763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623b92e9e80 msgr2=0x5623b9306090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:48.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.620+0000 7f54a77fe640 1 --2- 172.21.15.12:0/3794066763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623b92e9e80 0x5623b9306090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.620+0000 7f54a77fe640 1 -- 172.21.15.12:0/3794066763 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623b9367ea0 con 0x5623b92e94d0 2024-04-22T13:30:48.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.620+0000 7f54a77fe640 1 --2- 172.21.15.12:0/3794066763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623b92e94d0 0x5623b9305b50 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f5498005fd0 tx=0x7f549801af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84d894453ab3a13 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:48.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.620+0000 7f5486ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:48.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.620+0000 7f5486ffd640 1 -- 172.21.15.12:0/3794066763 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5498013b90 con 0x5623b92e94d0 2024-04-22T13:30:48.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.620+0000 7f5486ffd640 1 -- 172.21.15.12:0/3794066763 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f5498013d30 con 0x5623b92e94d0 2024-04-22T13:30:48.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.620+0000 7f54b07af740 1 -- 172.21.15.12:0/3794066763 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54a0003e20 con 0x5623b92e94d0 2024-04-22T13:30:48.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.620+0000 7f5486ffd640 1 -- 172.21.15.12:0/3794066763 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f549801a290 con 0x5623b92e94d0 2024-04-22T13:30:48.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.620+0000 7f54b07af740 10 client.18153 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:48.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.620+0000 7f54b07af740 1 -- 172.21.15.12:0/3794066763 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5623b9369150 con 0x5623b92e94d0 2024-04-22T13:30:48.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.620+0000 7f5486ffd640 1 -- 172.21.15.12:0/3794066763 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54a0003e20 con 0x5623b92e94d0 2024-04-22T13:30:48.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.621+0000 7f5486ffd640 1 -- 172.21.15.12:0/3794066763 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f549800f040 con 0x5623b92e94d0 2024-04-22T13:30:48.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.621+0000 7f54b07af740 10 client.18153 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:48.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.621+0000 7f54b07af740 10 client.18153 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:30:48.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.621+0000 7f54b07af740 20 client.18153 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:48.624 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.621+0000 7f54b07af740 1 --2- 172.21.15.12:0/3794066763 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5623b93698a0 0x5623b936bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:48.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.621+0000 7f54b07af740 4 client.18153 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:30:48.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.621+0000 7f54b07af740 1 -- 172.21.15.12:0/3794066763 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5623b92efa10 con 0x5623b93698a0 2024-04-22T13:30:48.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.621+0000 7f54a6ffd640 1 --2- 172.21.15.12:0/3794066763 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5623b93698a0 0x5623b936bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:48.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.621+0000 7f5486ffd640 10 client.18153 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:48.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.621+0000 7f54a6ffd640 1 --2- 172.21.15.12:0/3794066763 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5623b93698a0 0x5623b936bcc0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d50b1214bf776417 server_cookie=84a8e8ce47f7092a in_seq=0 out_seq=0 2024-04-22T13:30:48.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.622+0000 7f5486ffd640 1 -- 172.21.15.12:0/3794066763 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5623b92efa10 con 0x5623b93698a0 2024-04-22T13:30:48.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.622+0000 7f5486ffd640 10 client.18153 handle_command_reply: tid=0 2024-04-22T13:30:48.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.622+0000 7f54b07af740 1 client.18153 shutdown 2024-04-22T13:30:48.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.622+0000 7f54b07af740 1 -- 172.21.15.12:0/3794066763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623b92e94d0 msgr2=0x5623b9305b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.622+0000 7f54b07af740 1 --2- 172.21.15.12:0/3794066763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623b92e94d0 0x5623b9305b50 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f5498005fd0 tx=0x7f549801af90 comp rx=0 tx=0).stop 2024-04-22T13:30:48.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.622+0000 7f54b07af740 1 -- 172.21.15.12:0/3794066763 shutdown_connections 2024-04-22T13:30:48.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.622+0000 7f54b07af740 1 --2- 172.21.15.12:0/3794066763 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5623b93698a0 0x5623b936bcc0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.622+0000 7f54b07af740 1 --2- 172.21.15.12:0/3794066763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5623b92f2a00 0x5623b93065d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.622+0000 7f5486ffd640 0 client.18153 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:48.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.622+0000 7f54b07af740 1 --2- 172.21.15.12:0/3794066763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623b92e9e80 0x5623b9306090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.622+0000 7f54b07af740 1 --2- 172.21.15.12:0/3794066763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623b92e94d0 0x5623b9305b50 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.622+0000 7f54b07af740 1 -- 172.21.15.12:0/3794066763 >> 172.21.15.12:0/3794066763 conn(0x5623b92e3ba0 msgr2=0x5623b92ea350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:48.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.622+0000 7f54b07af740 1 -- 172.21.15.12:0/3794066763 shutdown_connections 2024-04-22T13:30:48.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.622+0000 7f54b07af740 1 -- 172.21.15.12:0/3794066763 wait complete. 2024-04-22T13:30:48.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.622+0000 7f54b07af740 20 client.18153 trim_cache size 0 max 16384 2024-04-22T13:30:48.626 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:30:48.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.622+0000 7f54adae0640 1 -- 172.21.15.12:0/2158704264 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f54900a3da0 msgr2=0x7f54900a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.622+0000 7f54adae0640 1 --2- 172.21.15.12:0/2158704264 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f54900a3da0 0x7f54900a6260 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f54a0009430 tx=0x7f54a00095c0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.622+0000 7f54adae0640 1 -- 172.21.15.12:0/2158704264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54a8107270 msgr2=0x7f54a81013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:48.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.622+0000 7f54adae0640 1 --2- 172.21.15.12:0/2158704264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54a8107270 0x7f54a81013c0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f549c005fd0 tx=0x7f549c00dde0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.622+0000 7f54adae0640 1 -- 172.21.15.12:0/2158704264 shutdown_connections 2024-04-22T13:30:48.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.622+0000 7f54adae0640 1 --2- 172.21.15.12:0/2158704264 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f54900a3da0 0x7f54900a6260 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.622+0000 7f54adae0640 1 --2- 172.21.15.12:0/2158704264 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54a8101900 0x7f54a81a68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.622+0000 7f54adae0640 1 --2- 172.21.15.12:0/2158704264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f54a8107270 0x7f54a81013c0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.622+0000 7f54adae0640 1 --2- 172.21.15.12:0/2158704264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54a806a840 0x7f54a8100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:48.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.622+0000 7f54adae0640 1 -- 172.21.15.12:0/2158704264 >> 172.21.15.12:0/2158704264 conn(0x7f54a8077eb0 msgr2=0x7f54a80fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:48.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.623+0000 7f54adae0640 1 -- 172.21.15.12:0/2158704264 shutdown_connections 2024-04-22T13:30:48.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:48.623+0000 7f54adae0640 1 -- 172.21.15.12:0/2158704264 wait complete. 2024-04-22T13:30:49.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:48 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2965967113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:30:49.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:48 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2965967113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:30:49.336 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:48 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2965967113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:30:49.630 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:30:51.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:30:51.076 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:30:51.077 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:49 smithi012 ceph-mon[28265]: pgmap v204: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:30:51.078 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:49 smithi111 ceph-mon[31361]: pgmap v204: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:30:51.078 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:49 smithi134 ceph-mon[31338]: pgmap v204: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:30:51.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.203+0000 7fcd15c26640 1 -- 172.21.15.12:0/500487127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd10106160 msgr2=0x7fcd1010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:51.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.203+0000 7fcd15c26640 1 --2- 172.21.15.12:0/500487127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd10106160 0x7fcd1010dd30 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fcd08009ca0 tx=0x7fcd0802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.203+0000 7fcd15c26640 1 -- 172.21.15.12:0/500487127 shutdown_connections 2024-04-22T13:30:51.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.203+0000 7fcd15c26640 1 --2- 172.21.15.12:0/500487127 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd10106160 0x7fcd1010dd30 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.203+0000 7fcd15c26640 1 --2- 172.21.15.12:0/500487127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd10101a40 0x7fcd10105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.203+0000 7fcd15c26640 1 --2- 172.21.15.12:0/500487127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd10101090 0x7fcd10101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.203+0000 7fcd15c26640 1 -- 172.21.15.12:0/500487127 >> 172.21.15.12:0/500487127 conn(0x7fcd10077f50 msgr2=0x7fcd10078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:51.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.203+0000 7fcd15c26640 1 -- 172.21.15.12:0/500487127 shutdown_connections 2024-04-22T13:30:51.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.203+0000 7fcd15c26640 1 -- 172.21.15.12:0/500487127 wait complete. 2024-04-22T13:30:51.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.203+0000 7fcd15c26640 1 Processor -- start 2024-04-22T13:30:51.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.204+0000 7fcd15c26640 1 -- start start 2024-04-22T13:30:51.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.204+0000 7fcd15c26640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd10101090 0x7fcd101986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.204+0000 7fcd15c26640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd10106160 0x7fcd10198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.204+0000 7fcd15c26640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd1019cfb0 0x7fcd1019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.204+0000 7fcd15c26640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcd101110b0 con 0x7fcd10106160 2024-04-22T13:30:51.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.204+0000 7fcd15c26640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcd10110f30 con 0x7fcd1019cfb0 2024-04-22T13:30:51.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.204+0000 7fcd15c26640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcd10111230 con 0x7fcd10101090 2024-04-22T13:30:51.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.204+0000 7fcd0effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd10106160 0x7fcd10198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.204+0000 7fcd0effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd10106160 0x7fcd10198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42592/0 (socket says 172.21.15.12:42592) 2024-04-22T13:30:51.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.204+0000 7fcd0effd640 1 -- 172.21.15.12:0/1158874458 learned_addr learned my addr 172.21.15.12:0/1158874458 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:51.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.204+0000 7fcd0f7fe640 1 --2- 172.21.15.12:0/1158874458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd10101090 0x7fcd101986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.204+0000 7fcd0ffff640 1 --2- 172.21.15.12:0/1158874458 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd1019cfb0 0x7fcd1019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.204+0000 7fcd0effd640 1 -- 172.21.15.12:0/1158874458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd10101090 msgr2=0x7fcd101986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:51.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.204+0000 7fcd0effd640 1 --2- 172.21.15.12:0/1158874458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd10101090 0x7fcd101986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.204+0000 7fcd0effd640 1 -- 172.21.15.12:0/1158874458 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd1019cfb0 msgr2=0x7fcd1019d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:51.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.204+0000 7fcd0effd640 1 --2- 172.21.15.12:0/1158874458 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd1019cfb0 0x7fcd1019d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.204+0000 7fcd0effd640 1 -- 172.21.15.12:0/1158874458 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd1019daa0 con 0x7fcd10106160 2024-04-22T13:30:51.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.204+0000 7fcd0ffff640 1 --2- 172.21.15.12:0/1158874458 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd1019cfb0 0x7fcd1019d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:51.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.205+0000 7fcd0effd640 1 --2- 172.21.15.12:0/1158874458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd10106160 0x7fcd10198c30 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fcd0400c970 tx=0x7fcd0400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:51.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.205+0000 7fcd0cff9640 1 -- 172.21.15.12:0/1158874458 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd04007bb0 con 0x7fcd10106160 2024-04-22T13:30:51.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.205+0000 7fcd0cff9640 1 -- 172.21.15.12:0/1158874458 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd04007d50 con 0x7fcd10106160 2024-04-22T13:30:51.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.205+0000 7fcd15c26640 1 -- 172.21.15.12:0/1158874458 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd1019dd90 con 0x7fcd10106160 2024-04-22T13:30:51.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.205+0000 7fcd0cff9640 1 -- 172.21.15.12:0/1158874458 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0400f450 con 0x7fcd10106160 2024-04-22T13:30:51.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.205+0000 7fcd15c26640 1 -- 172.21.15.12:0/1158874458 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd101a55d0 con 0x7fcd10106160 2024-04-22T13:30:51.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.208+0000 7fcd0cff9640 1 -- 172.21.15.12:0/1158874458 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcd04004f20 con 0x7fcd10106160 2024-04-22T13:30:51.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.208+0000 7fcd1880f740 1 Processor -- start 2024-04-22T13:30:51.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.208+0000 7fcd1880f740 1 -- start start 2024-04-22T13:30:51.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.208+0000 7fcd1880f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622df23b8e0 0x5622df23bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.208+0000 7fcd1880f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5622df23c290 0x5622df244840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.208+0000 7fcd1880f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622df244e10 0x5622df247260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.208+0000 7fcd1880f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5622df19ff00 con 0x5622df23b8e0 2024-04-22T13:30:51.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.208+0000 7fcd1880f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5622df13d590 con 0x5622df23c290 2024-04-22T13:30:51.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.208+0000 7fcd1880f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5622df19bec0 con 0x5622df244e10 2024-04-22T13:30:51.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.208+0000 7fcd0ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622df23b8e0 0x5622df23bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.208+0000 7fcd0ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622df23b8e0 0x5622df23bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42598/0 (socket says 172.21.15.12:42598) 2024-04-22T13:30:51.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.208+0000 7fcd0ffff640 1 -- 172.21.15.12:0/3704703082 learned_addr learned my addr 172.21.15.12:0/3704703082 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:51.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.208+0000 7fcd0cff9640 1 --2- 172.21.15.12:0/1158874458 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fccf80a3da0 0x7fccf80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.209+0000 7fcd0f7fe640 1 --2- 172.21.15.12:0/3704703082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5622df23c290 0x5622df244840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.209+0000 7fcd0effd640 1 --2- 172.21.15.12:0/3704703082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622df244e10 0x5622df247260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.209+0000 7fcd0ffff640 1 --2- 172.21.15.12:0/1158874458 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fccf80a3da0 0x7fccf80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.209+0000 7fcd0cff9640 1 -- 172.21.15.12:0/1158874458 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fcd040d0700 con 0x7fcd10106160 2024-04-22T13:30:51.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.209+0000 7fcd0ffff640 1 -- 172.21.15.12:0/3704703082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622df244e10 msgr2=0x5622df247260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:51.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.209+0000 7fcd0ffff640 1 --2- 172.21.15.12:0/3704703082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622df244e10 0x5622df247260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.209+0000 7fcd0ffff640 1 -- 172.21.15.12:0/3704703082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5622df23c290 msgr2=0x5622df244840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:51.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.209+0000 7fcd0ffff640 1 --2- 172.21.15.12:0/3704703082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5622df23c290 0x5622df244840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.209+0000 7fcd0ffff640 1 -- 172.21.15.12:0/3704703082 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622def7a7e0 con 0x5622df23b8e0 2024-04-22T13:30:51.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.209+0000 7fcd0ffff640 1 --2- 172.21.15.12:0/3704703082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622df23b8e0 0x5622df23bcc0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fcd08009ca0 tx=0x7fcd08032d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c325e312cf8652 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:51.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.209+0000 7fccf5ffb640 1 -- 172.21.15.12:0/3704703082 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd08039040 con 0x5622df23b8e0 2024-04-22T13:30:51.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.209+0000 7fccf5ffb640 1 -- 172.21.15.12:0/3704703082 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd080327c0 con 0x5622df23b8e0 2024-04-22T13:30:51.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.209+0000 7fccf5ffb640 1 -- 172.21.15.12:0/3704703082 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd080346a0 con 0x5622df23b8e0 2024-04-22T13:30:51.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.209+0000 7fcd0ffff640 1 --2- 172.21.15.12:0/1158874458 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fccf80a3da0 0x7fccf80a6260 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcd0804c000 tx=0x7fcd08048000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:51.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.210+0000 7fcd1880f740 1 -- 172.21.15.12:0/3704703082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622df23b8e0 msgr2=0x5622df23bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:51.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.210+0000 7fcd1880f740 1 --2- 172.21.15.12:0/3704703082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622df23b8e0 0x5622df23bcc0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fcd08009ca0 tx=0x7fcd08032d70 comp rx=0 tx=0).stop 2024-04-22T13:30:51.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.210+0000 7fcd1880f740 1 -- 172.21.15.12:0/3704703082 shutdown_connections 2024-04-22T13:30:51.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.210+0000 7fcd1880f740 1 --2- 172.21.15.12:0/3704703082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622df244e10 0x5622df247260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.210+0000 7fcd1880f740 1 --2- 172.21.15.12:0/3704703082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5622df23c290 0x5622df244840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.210+0000 7fcd1880f740 1 --2- 172.21.15.12:0/3704703082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622df23b8e0 0x5622df23bcc0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.210+0000 7fcd1880f740 1 -- 172.21.15.12:0/3704703082 >> 172.21.15.12:0/3704703082 conn(0x5622df235fb0 msgr2=0x5622df236390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:51.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.210+0000 7fcd1880f740 1 -- 172.21.15.12:0/3704703082 shutdown_connections 2024-04-22T13:30:51.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.210+0000 7fcd1880f740 1 -- 172.21.15.12:0/3704703082 wait complete. 2024-04-22T13:30:51.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.210+0000 7fcd1880f740 1 -- start start 2024-04-22T13:30:51.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.211+0000 7fcd1880f740 1 Processor -- start 2024-04-22T13:30:51.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.211+0000 7fcd1880f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622df23b8e0 0x5622df241de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.211+0000 7fcd1880f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622df23c290 0x5622df242320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.211+0000 7fcd1880f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5622df244e10 0x5622df242860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.211+0000 7fcd1880f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5622df247d80 con 0x5622df23b8e0 2024-04-22T13:30:51.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.211+0000 7fcd1880f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5622df159580 con 0x5622df244e10 2024-04-22T13:30:51.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.211+0000 7fcd1880f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5622df247f00 con 0x5622df23c290 2024-04-22T13:30:51.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.211+0000 7fcd1880f740 10 client.0 _ll_register_callbacks cb 0x5622df15f650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:51.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.211+0000 7fcd0f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622df23b8e0 0x5622df241de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.211+0000 7fcd0f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622df23b8e0 0x5622df241de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42612/0 (socket says 172.21.15.12:42612) 2024-04-22T13:30:51.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.211+0000 7fcd0f7fe640 1 -- 172.21.15.12:0/1336390562 learned_addr learned my addr 172.21.15.12:0/1336390562 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:51.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.211+0000 7fcd0ffff640 1 --2- 172.21.15.12:0/1336390562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5622df244e10 0x5622df242860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.211+0000 7fcd0effd640 1 --2- 172.21.15.12:0/1336390562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622df23c290 0x5622df242320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.211+0000 7fcd0f7fe640 1 -- 172.21.15.12:0/1336390562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622df23c290 msgr2=0x5622df242320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:51.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.211+0000 7fcd0f7fe640 1 --2- 172.21.15.12:0/1336390562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622df23c290 0x5622df242320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.211+0000 7fcd0f7fe640 1 -- 172.21.15.12:0/1336390562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5622df244e10 msgr2=0x5622df242860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:51.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.211+0000 7fcd0f7fe640 1 --2- 172.21.15.12:0/1336390562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5622df244e10 0x5622df242860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.212+0000 7fcd0f7fe640 1 -- 172.21.15.12:0/1336390562 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622defdaf70 con 0x5622df23b8e0 2024-04-22T13:30:51.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.212+0000 7fcd0f7fe640 1 --2- 172.21.15.12:0/1336390562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622df23b8e0 0x5622df241de0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000bf50 tx=0x7fcd00011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d3c6270f9d4f95d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:51.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.212+0000 7fcceeffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:51.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.212+0000 7fcceeffd640 1 -- 172.21.15.12:0/1336390562 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd00004020 con 0x5622df23b8e0 2024-04-22T13:30:51.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.212+0000 7fcceeffd640 1 -- 172.21.15.12:0/1336390562 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fcd000041c0 con 0x5622df23b8e0 2024-04-22T13:30:51.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.212+0000 7fcd1880f740 1 -- 172.21.15.12:0/1336390562 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd080327c0 con 0x5622df23b8e0 2024-04-22T13:30:51.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.212+0000 7fcceeffd640 1 -- 172.21.15.12:0/1336390562 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcd000043c0 con 0x5622df23b8e0 2024-04-22T13:30:51.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.212+0000 7fcd1880f740 10 client.18165 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:51.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.212+0000 7fcd1880f740 1 -- 172.21.15.12:0/1336390562 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5622dee72170 con 0x5622df23b8e0 2024-04-22T13:30:51.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.212+0000 7fcceeffd640 1 -- 172.21.15.12:0/1336390562 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd080327c0 con 0x5622df23b8e0 2024-04-22T13:30:51.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.212+0000 7fcceeffd640 1 -- 172.21.15.12:0/1336390562 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fcd00003920 con 0x5622df23b8e0 2024-04-22T13:30:51.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.213+0000 7fcd1880f740 10 client.18165 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:51.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.213+0000 7fcd1880f740 10 client.18165 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:30:51.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.213+0000 7fcd1880f740 20 client.18165 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:51.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.213+0000 7fcd1880f740 1 --2- 172.21.15.12:0/1336390562 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5622df255300 0x5622df257750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.213+0000 7fcd1880f740 4 client.18165 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:30:51.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.213+0000 7fcd1880f740 1 -- 172.21.15.12:0/1336390562 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622df1ad920 con 0x5622df255300 2024-04-22T13:30:51.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.213+0000 7fcd0effd640 1 --2- 172.21.15.12:0/1336390562 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5622df255300 0x5622df257750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.213+0000 7fcd0effd640 1 --2- 172.21.15.12:0/1336390562 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5622df255300 0x5622df257750 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1402567f7af09588 server_cookie=d199a6cdb6c8c763 in_seq=0 out_seq=0 2024-04-22T13:30:51.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.213+0000 7fcceeffd640 10 client.18165 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:51.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.214+0000 7fcceeffd640 1 -- 172.21.15.12:0/1336390562 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5622df1ad920 con 0x5622df255300 2024-04-22T13:30:51.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.214+0000 7fcceeffd640 10 client.18165 handle_command_reply: tid=0 2024-04-22T13:30:51.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.214+0000 7fcd1880f740 1 client.18165 shutdown 2024-04-22T13:30:51.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.214+0000 7fcd1880f740 1 -- 172.21.15.12:0/1336390562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622df23b8e0 msgr2=0x5622df241de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:51.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.214+0000 7fcd1880f740 1 --2- 172.21.15.12:0/1336390562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622df23b8e0 0x5622df241de0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000bf50 tx=0x7fcd00011810 comp rx=0 tx=0).stop 2024-04-22T13:30:51.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.214+0000 7fcd1880f740 1 -- 172.21.15.12:0/1336390562 shutdown_connections 2024-04-22T13:30:51.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.214+0000 7fcd1880f740 1 --2- 172.21.15.12:0/1336390562 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5622df255300 0x5622df257750 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.214+0000 7fcd1880f740 1 --2- 172.21.15.12:0/1336390562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5622df244e10 0x5622df242860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.214+0000 7fcd1880f740 1 --2- 172.21.15.12:0/1336390562 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622df23c290 0x5622df242320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.214+0000 7fcceeffd640 0 client.18165 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:51.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.214+0000 7fcd1880f740 1 --2- 172.21.15.12:0/1336390562 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622df23b8e0 0x5622df241de0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.214+0000 7fcd1880f740 1 -- 172.21.15.12:0/1336390562 >> 172.21.15.12:0/1336390562 conn(0x5622df235fb0 msgr2=0x5622df237d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:51.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.214+0000 7fcd1880f740 1 -- 172.21.15.12:0/1336390562 shutdown_connections 2024-04-22T13:30:51.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.214+0000 7fcd1880f740 1 -- 172.21.15.12:0/1336390562 wait complete. 2024-04-22T13:30:51.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.214+0000 7fcd1880f740 20 client.18165 trim_cache size 0 max 16384 2024-04-22T13:30:51.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.232+0000 7fcd1880f740 1 Processor -- start 2024-04-22T13:30:51.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.232+0000 7fcd1880f740 1 -- start start 2024-04-22T13:30:51.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.232+0000 7fcd1880f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622df23b8e0 0x5622df244510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.232+0000 7fcd1880f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622df23c290 0x5622df250c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.232+0000 7fcd1880f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5622df244e10 0x5622df237ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.232+0000 7fcd1880f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcd101adb20 con 0x5622df23b8e0 2024-04-22T13:30:51.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.232+0000 7fcd1880f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcd101af660 con 0x5622df244e10 2024-04-22T13:30:51.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.232+0000 7fcd1880f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcd101ae1f0 con 0x5622df23c290 2024-04-22T13:30:51.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.233+0000 7fcd0f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622df23b8e0 0x5622df244510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.233+0000 7fcd0f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622df23b8e0 0x5622df244510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42620/0 (socket says 172.21.15.12:42620) 2024-04-22T13:30:51.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.233+0000 7fcd0f7fe640 1 -- 172.21.15.12:0/4056970861 learned_addr learned my addr 172.21.15.12:0/4056970861 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:51.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.233+0000 7fcd0effd640 1 --2- 172.21.15.12:0/4056970861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622df23c290 0x5622df250c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.233+0000 7fcd0f7fe640 1 -- 172.21.15.12:0/4056970861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622df23c290 msgr2=0x5622df250c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:51.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.233+0000 7fcd0f7fe640 1 --2- 172.21.15.12:0/4056970861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622df23c290 0x5622df250c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.233+0000 7fcd0f7fe640 1 -- 172.21.15.12:0/4056970861 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5622df244e10 msgr2=0x5622df237ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:51.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.233+0000 7fcd0f7fe640 1 --2- 172.21.15.12:0/4056970861 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5622df244e10 0x5622df237ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.233+0000 7fcd0f7fe640 1 -- 172.21.15.12:0/4056970861 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622df240330 con 0x5622df23b8e0 2024-04-22T13:30:51.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.233+0000 7fcd0effd640 1 --2- 172.21.15.12:0/4056970861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622df23c290 0x5622df250c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:30:51.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.233+0000 7fcd0f7fe640 1 --2- 172.21.15.12:0/4056970861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622df23b8e0 0x5622df244510 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fcd00001eb0 tx=0x7fcd00004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0f32cf23659f63 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:51.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.233+0000 7fccf57fa640 1 -- 172.21.15.12:0/4056970861 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd00000e60 con 0x5622df23b8e0 2024-04-22T13:30:51.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.233+0000 7fccf57fa640 1 -- 172.21.15.12:0/4056970861 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd00003aa0 con 0x5622df23b8e0 2024-04-22T13:30:51.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.233+0000 7fccf57fa640 1 -- 172.21.15.12:0/4056970861 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd000052a0 con 0x5622df23b8e0 2024-04-22T13:30:51.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.234+0000 7fcd1880f740 1 -- 172.21.15.12:0/4056970861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622df23b8e0 msgr2=0x5622df244510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:51.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.234+0000 7fcd1880f740 1 --2- 172.21.15.12:0/4056970861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622df23b8e0 0x5622df244510 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fcd00001eb0 tx=0x7fcd00004560 comp rx=0 tx=0).stop 2024-04-22T13:30:51.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.234+0000 7fcd1880f740 1 -- 172.21.15.12:0/4056970861 shutdown_connections 2024-04-22T13:30:51.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.234+0000 7fcd1880f740 1 --2- 172.21.15.12:0/4056970861 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5622df244e10 0x5622df237ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.234+0000 7fcd1880f740 1 --2- 172.21.15.12:0/4056970861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622df23c290 0x5622df250c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.234+0000 7fcd1880f740 1 --2- 172.21.15.12:0/4056970861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622df23b8e0 0x5622df244510 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.234+0000 7fcd1880f740 1 -- 172.21.15.12:0/4056970861 >> 172.21.15.12:0/4056970861 conn(0x5622df235fb0 msgr2=0x5622df2373d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:51.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.234+0000 7fcd1880f740 1 -- 172.21.15.12:0/4056970861 shutdown_connections 2024-04-22T13:30:51.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.234+0000 7fcd1880f740 1 -- 172.21.15.12:0/4056970861 wait complete. 2024-04-22T13:30:51.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.234+0000 7fcd1880f740 1 -- start start 2024-04-22T13:30:51.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.234+0000 7fcd1880f740 1 Processor -- start 2024-04-22T13:30:51.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.235+0000 7fcd1880f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5622df23b8e0 0x5622df24cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.235+0000 7fcd1880f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622df23c290 0x5622df24d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.235+0000 7fcd1880f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622df244e10 0x5622df249280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.235+0000 7fcd1880f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcd101aef90 con 0x5622df244e10 2024-04-22T13:30:51.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.235+0000 7fcd1880f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcd101ae8c0 con 0x5622df23b8e0 2024-04-22T13:30:51.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.235+0000 7fcd1880f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcd101abb20 con 0x5622df23c290 2024-04-22T13:30:51.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.235+0000 7fcd1880f740 10 client.0 _ll_register_callbacks cb 0x7fcd0000bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:51.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.235+0000 7fcd0ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622df244e10 0x5622df249280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.235+0000 7fcd0ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622df244e10 0x5622df249280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42636/0 (socket says 172.21.15.12:42636) 2024-04-22T13:30:51.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.235+0000 7fcd0f7fe640 1 --2- 172.21.15.12:0/1158234726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5622df23b8e0 0x5622df24cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.235+0000 7fcd0ffff640 1 -- 172.21.15.12:0/1158234726 learned_addr learned my addr 172.21.15.12:0/1158234726 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:51.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.235+0000 7fcd0effd640 1 --2- 172.21.15.12:0/1158234726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622df23c290 0x5622df24d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.235+0000 7fcd0ffff640 1 -- 172.21.15.12:0/1158234726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622df23c290 msgr2=0x5622df24d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:51.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.235+0000 7fcd0ffff640 1 --2- 172.21.15.12:0/1158234726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622df23c290 0x5622df24d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.235+0000 7fcd0ffff640 1 -- 172.21.15.12:0/1158234726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5622df23b8e0 msgr2=0x5622df24cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:51.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.235+0000 7fcd0ffff640 1 --2- 172.21.15.12:0/1158234726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5622df23b8e0 0x5622df24cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.235+0000 7fcd0ffff640 1 -- 172.21.15.12:0/1158234726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622df2499c0 con 0x5622df244e10 2024-04-22T13:30:51.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.235+0000 7fcd0effd640 1 --2- 172.21.15.12:0/1158234726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622df23c290 0x5622df24d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:30:51.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.235+0000 7fcd0ffff640 1 --2- 172.21.15.12:0/1158234726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622df244e10 0x5622df249280 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fcd0802c700 tx=0x7fcd08005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb19f3d5dd49caf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:51.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.235+0000 7fcceeffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:51.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.235+0000 7fcceeffd640 1 -- 172.21.15.12:0/1158234726 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd08004830 con 0x5622df244e10 2024-04-22T13:30:51.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.235+0000 7fcceeffd640 1 -- 172.21.15.12:0/1158234726 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fcd080327c0 con 0x5622df244e10 2024-04-22T13:30:51.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.235+0000 7fcd1880f740 1 -- 172.21.15.12:0/1158234726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd00003aa0 con 0x5622df244e10 2024-04-22T13:30:51.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.236+0000 7fcceeffd640 1 -- 172.21.15.12:0/1158234726 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcd0802cdf0 con 0x5622df244e10 2024-04-22T13:30:51.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.236+0000 7fcd1880f740 10 client.18171 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:51.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.236+0000 7fcd1880f740 1 -- 172.21.15.12:0/1158234726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5622df2bc250 con 0x5622df244e10 2024-04-22T13:30:51.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.236+0000 7fcceeffd640 1 -- 172.21.15.12:0/1158234726 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd00003aa0 con 0x5622df244e10 2024-04-22T13:30:51.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.236+0000 7fcceeffd640 1 -- 172.21.15.12:0/1158234726 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fcd08004040 con 0x5622df244e10 2024-04-22T13:30:51.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.236+0000 7fcd1880f740 10 client.18171 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:51.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.236+0000 7fcd1880f740 10 client.18171 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:30:51.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.236+0000 7fcd1880f740 20 client.18171 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:51.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.236+0000 7fcd1880f740 1 --2- 172.21.15.12:0/1158234726 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5622df2bc530 0x5622df2be920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.236+0000 7fcd1880f740 4 client.18171 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:30:51.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.236+0000 7fcd1880f740 1 -- 172.21.15.12:0/1158234726 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5622df23cc10 con 0x5622df2bc530 2024-04-22T13:30:51.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.236+0000 7fcd0f7fe640 1 --2- 172.21.15.12:0/1158234726 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5622df2bc530 0x5622df2be920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.237+0000 7fcd0f7fe640 1 --2- 172.21.15.12:0/1158234726 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5622df2bc530 0x5622df2be920 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=692eca5cd915885f server_cookie=762d58405d46459c in_seq=0 out_seq=0 2024-04-22T13:30:51.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.237+0000 7fcceeffd640 10 client.18171 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:51.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.238+0000 7fcceeffd640 1 -- 172.21.15.12:0/1158234726 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5622df23cc10 con 0x5622df2bc530 2024-04-22T13:30:51.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.238+0000 7fcceeffd640 10 client.18171 handle_command_reply: tid=0 2024-04-22T13:30:51.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.238+0000 7fcd1880f740 1 client.18171 shutdown 2024-04-22T13:30:51.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.238+0000 7fcd1880f740 1 -- 172.21.15.12:0/1158234726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622df244e10 msgr2=0x5622df249280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:51.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.238+0000 7fcd1880f740 1 --2- 172.21.15.12:0/1158234726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622df244e10 0x5622df249280 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fcd0802c700 tx=0x7fcd08005c50 comp rx=0 tx=0).stop 2024-04-22T13:30:51.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.238+0000 7fcd1880f740 1 -- 172.21.15.12:0/1158234726 shutdown_connections 2024-04-22T13:30:51.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.238+0000 7fcd1880f740 1 --2- 172.21.15.12:0/1158234726 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5622df2bc530 0x5622df2be920 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.238+0000 7fcd1880f740 1 --2- 172.21.15.12:0/1158234726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5622df244e10 0x5622df249280 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.238+0000 7fcd1880f740 1 --2- 172.21.15.12:0/1158234726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5622df23c290 0x5622df24d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.238+0000 7fcd1880f740 1 --2- 172.21.15.12:0/1158234726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5622df23b8e0 0x5622df24cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.238+0000 7fcd1880f740 1 -- 172.21.15.12:0/1158234726 >> 172.21.15.12:0/1158234726 conn(0x5622df235fb0 msgr2=0x5622df237c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:51.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.238+0000 7fcceeffd640 0 client.18171 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:51.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.238+0000 7fcd1880f740 1 -- 172.21.15.12:0/1158234726 shutdown_connections 2024-04-22T13:30:51.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.238+0000 7fcd1880f740 1 -- 172.21.15.12:0/1158234726 wait complete. 2024-04-22T13:30:51.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.238+0000 7fcd1880f740 20 client.18171 trim_cache size 0 max 16384 2024-04-22T13:30:51.241 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:30:51.241 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-04-22T13:30:51.241 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "2fdfb73e-ada3-48a6-a0ab-5460ecb4ae20", 2024-04-22T13:30:51.241 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-04-22T13:30:51.241 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:30:51.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.239+0000 7fcd15c26640 1 -- 172.21.15.12:0/1158874458 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fccf80a3da0 msgr2=0x7fccf80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:51.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.239+0000 7fcd15c26640 1 --2- 172.21.15.12:0/1158874458 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fccf80a3da0 0x7fccf80a6260 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcd0804c000 tx=0x7fcd08048000 comp rx=0 tx=0).stop 2024-04-22T13:30:51.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.239+0000 7fcd15c26640 1 -- 172.21.15.12:0/1158874458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd10106160 msgr2=0x7fcd10198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:51.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.239+0000 7fcd15c26640 1 --2- 172.21.15.12:0/1158874458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd10106160 0x7fcd10198c30 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fcd0400c970 tx=0x7fcd0400ce30 comp rx=0 tx=0).stop 2024-04-22T13:30:51.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.239+0000 7fcd15c26640 1 -- 172.21.15.12:0/1158874458 shutdown_connections 2024-04-22T13:30:51.242 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.239+0000 7fcd15c26640 1 --2- 172.21.15.12:0/1158874458 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fccf80a3da0 0x7fccf80a6260 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.239+0000 7fcd15c26640 1 --2- 172.21.15.12:0/1158874458 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd1019cfb0 0x7fcd1019d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.239+0000 7fcd15c26640 1 --2- 172.21.15.12:0/1158874458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcd10106160 0x7fcd10198c30 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.239+0000 7fcd15c26640 1 --2- 172.21.15.12:0/1158874458 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd10101090 0x7fcd101986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.239+0000 7fcd15c26640 1 -- 172.21.15.12:0/1158874458 >> 172.21.15.12:0/1158874458 conn(0x7fcd10077f50 msgr2=0x7fcd10069480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:51.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.239+0000 7fcd15c26640 1 -- 172.21.15.12:0/1158874458 shutdown_connections 2024-04-22T13:30:51.243 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.239+0000 7fcd15c26640 1 -- 172.21.15.12:0/1158874458 wait complete. 2024-04-22T13:30:51.246 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:30:51.246 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:30:51.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.355+0000 7f84f7f75640 1 -- 172.21.15.12:0/2785353165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84f0102b50 msgr2=0x7f84f0106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:51.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.355+0000 7f84f7f75640 1 --2- 172.21.15.12:0/2785353165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84f0102b50 0x7f84f0106b90 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f84e0009900 tx=0x7f84e002bc30 comp rx=0 tx=0).stop 2024-04-22T13:30:51.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.355+0000 7f84f7f75640 1 -- 172.21.15.12:0/2785353165 shutdown_connections 2024-04-22T13:30:51.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.355+0000 7f84f7f75640 1 --2- 172.21.15.12:0/2785353165 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84f0107270 0x7f84f010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.355+0000 7f84f7f75640 1 --2- 172.21.15.12:0/2785353165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84f0102b50 0x7f84f0106b90 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.355+0000 7f84f7f75640 1 --2- 172.21.15.12:0/2785353165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84f01021a0 0x7f84f0102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.355+0000 7f84f7f75640 1 -- 172.21.15.12:0/2785353165 >> 172.21.15.12:0/2785353165 conn(0x7f84f00fda40 msgr2=0x7f84f00ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:51.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.355+0000 7f84f7f75640 1 -- 172.21.15.12:0/2785353165 shutdown_connections 2024-04-22T13:30:51.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.355+0000 7f84f7f75640 1 -- 172.21.15.12:0/2785353165 wait complete. 2024-04-22T13:30:51.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.355+0000 7f84f7f75640 1 Processor -- start 2024-04-22T13:30:51.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.356+0000 7f84f7f75640 1 -- start start 2024-04-22T13:30:51.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.356+0000 7f84f7f75640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84f01021a0 0x7f84f019e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.356+0000 7f84f7f75640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84f0107270 0x7f84f019e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.356+0000 7f84f7f75640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84f0198670 0x7f84f0198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.356+0000 7f84f7f75640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f84f01121b0 con 0x7f84f0198670 2024-04-22T13:30:51.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.356+0000 7f84f7f75640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f84f0112030 con 0x7f84f0107270 2024-04-22T13:30:51.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.356+0000 7f84f7f75640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f84f0112330 con 0x7f84f01021a0 2024-04-22T13:30:51.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.356+0000 7f84f6512640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84f0198670 0x7f84f0198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.356+0000 7f84f6512640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84f0198670 0x7f84f0198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42654/0 (socket says 172.21.15.12:42654) 2024-04-22T13:30:51.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.356+0000 7f84f6512640 1 -- 172.21.15.12:0/2910102347 learned_addr learned my addr 172.21.15.12:0/2910102347 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:51.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.356+0000 7f84f5510640 1 --2- 172.21.15.12:0/2910102347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84f0107270 0x7f84f019e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.356+0000 7f84f5d11640 1 --2- 172.21.15.12:0/2910102347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84f01021a0 0x7f84f019e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.356+0000 7f84f6512640 1 -- 172.21.15.12:0/2910102347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84f01021a0 msgr2=0x7f84f019e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:51.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.356+0000 7f84f6512640 1 --2- 172.21.15.12:0/2910102347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84f01021a0 0x7f84f019e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.356+0000 7f84f6512640 1 -- 172.21.15.12:0/2910102347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84f0107270 msgr2=0x7f84f019e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:51.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.356+0000 7f84f6512640 1 --2- 172.21.15.12:0/2910102347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84f0107270 0x7f84f019e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.356+0000 7f84f6512640 1 -- 172.21.15.12:0/2910102347 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f0199190 con 0x7f84f0198670 2024-04-22T13:30:51.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.356+0000 7f84f6512640 1 --2- 172.21.15.12:0/2910102347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84f0198670 0x7f84f0198b20 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f84e800eb10 tx=0x7f84e800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:51.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.356+0000 7f84e6ffd640 1 -- 172.21.15.12:0/2910102347 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8004b20 con 0x7f84f0198670 2024-04-22T13:30:51.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.357+0000 7f84e6ffd640 1 -- 172.21.15.12:0/2910102347 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84e8004cc0 con 0x7f84f0198670 2024-04-22T13:30:51.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.357+0000 7f84f7f75640 1 -- 172.21.15.12:0/2910102347 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84f0199480 con 0x7f84f0198670 2024-04-22T13:30:51.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.357+0000 7f84e6ffd640 1 -- 172.21.15.12:0/2910102347 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8010670 con 0x7f84f0198670 2024-04-22T13:30:51.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.357+0000 7f84f7f75640 1 -- 172.21.15.12:0/2910102347 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84f01a5630 con 0x7f84f0198670 2024-04-22T13:30:51.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.359+0000 7f84fab65740 1 Processor -- start 2024-04-22T13:30:51.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.360+0000 7f84e6ffd640 1 -- 172.21.15.12:0/2910102347 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f84e8005020 con 0x7f84f0198670 2024-04-22T13:30:51.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.360+0000 7f84fab65740 1 -- start start 2024-04-22T13:30:51.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.360+0000 7f84fab65740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec8c5aaf60 0x55ec8c5ab340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.360+0000 7f84fab65740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec8c5ab910 0x55ec8c5b8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.360+0000 7f84fab65740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec8c5b8e90 0x55ec8c5bb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.360+0000 7f84fab65740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ec8c5101e0 con 0x55ec8c5ab910 2024-04-22T13:30:51.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.360+0000 7f84fab65740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ec8c4ad390 con 0x55ec8c5b8e90 2024-04-22T13:30:51.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.360+0000 7f84fab65740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ec8c50c1a0 con 0x55ec8c5aaf60 2024-04-22T13:30:51.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.360+0000 7f84f5d11640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec8c5ab910 0x55ec8c5b8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.360+0000 7f84f5d11640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec8c5ab910 0x55ec8c5b8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42668/0 (socket says 172.21.15.12:42668) 2024-04-22T13:30:51.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.360+0000 7f84f5d11640 1 -- 172.21.15.12:0/1559073936 learned_addr learned my addr 172.21.15.12:0/1559073936 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:51.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.360+0000 7f84e6ffd640 1 --2- 172.21.15.12:0/2910102347 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f84dc0a3d20 0x7f84dc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.360+0000 7f84f5510640 1 --2- 172.21.15.12:0/1559073936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec8c5aaf60 0x55ec8c5ab340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.360+0000 7f84f5510640 1 --2- 172.21.15.12:0/1559073936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec8c5b8e90 0x55ec8c5bb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.361+0000 7f84f6512640 1 --2- 172.21.15.12:0/2910102347 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f84dc0a3d20 0x7f84dc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.361+0000 7f84f5d11640 1 -- 172.21.15.12:0/1559073936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec8c5aaf60 msgr2=0x55ec8c5ab340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:51.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.361+0000 7f84f5d11640 1 --2- 172.21.15.12:0/1559073936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec8c5aaf60 0x55ec8c5ab340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.361+0000 7f84f5d11640 1 -- 172.21.15.12:0/1559073936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec8c5b8e90 msgr2=0x55ec8c5bb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:51.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.361+0000 7f84f5d11640 1 --2- 172.21.15.12:0/1559073936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec8c5b8e90 0x55ec8c5bb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.361+0000 7f84f5d11640 1 -- 172.21.15.12:0/1559073936 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec8c2ec7d0 con 0x55ec8c5ab910 2024-04-22T13:30:51.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.361+0000 7f84e6ffd640 1 -- 172.21.15.12:0/2910102347 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f84e80d0d60 con 0x7f84f0198670 2024-04-22T13:30:51.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.361+0000 7f84f5510640 1 --2- 172.21.15.12:0/1559073936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec8c5aaf60 0x55ec8c5ab340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:51.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.361+0000 7f84f5d11640 1 --2- 172.21.15.12:0/1559073936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec8c5ab910 0x55ec8c5b8950 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00bcf0 tx=0x7f84ec00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac75b1c014276df server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:51.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.361+0000 7f84dbfff640 1 -- 172.21.15.12:0/1559073936 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ec015070 con 0x55ec8c5ab910 2024-04-22T13:30:51.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.361+0000 7f84f6512640 1 --2- 172.21.15.12:0/2910102347 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f84dc0a3d20 0x7f84dc0a61e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f84e80ce5b0 tx=0x7f84e801bb30 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:51.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.361+0000 7f84dbfff640 1 -- 172.21.15.12:0/1559073936 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84ec0118f0 con 0x55ec8c5ab910 2024-04-22T13:30:51.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.361+0000 7f84dbfff640 1 -- 172.21.15.12:0/1559073936 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ec0107f0 con 0x55ec8c5ab910 2024-04-22T13:30:51.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.362+0000 7f84fab65740 1 -- 172.21.15.12:0/1559073936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec8c5ab910 msgr2=0x55ec8c5b8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:51.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.362+0000 7f84fab65740 1 --2- 172.21.15.12:0/1559073936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec8c5ab910 0x55ec8c5b8950 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00bcf0 tx=0x7f84ec00f730 comp rx=0 tx=0).stop 2024-04-22T13:30:51.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.362+0000 7f84fab65740 1 -- 172.21.15.12:0/1559073936 shutdown_connections 2024-04-22T13:30:51.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.362+0000 7f84fab65740 1 --2- 172.21.15.12:0/1559073936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec8c5b8e90 0x55ec8c5bb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.362+0000 7f84fab65740 1 --2- 172.21.15.12:0/1559073936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec8c5ab910 0x55ec8c5b8950 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.362+0000 7f84fab65740 1 --2- 172.21.15.12:0/1559073936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec8c5aaf60 0x55ec8c5ab340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.362+0000 7f84fab65740 1 -- 172.21.15.12:0/1559073936 >> 172.21.15.12:0/1559073936 conn(0x55ec8c5a5cb0 msgr2=0x55ec8c5a6090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:51.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.362+0000 7f84fab65740 1 -- 172.21.15.12:0/1559073936 shutdown_connections 2024-04-22T13:30:51.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.362+0000 7f84fab65740 1 -- 172.21.15.12:0/1559073936 wait complete. 2024-04-22T13:30:51.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.362+0000 7f84fab65740 1 -- start start 2024-04-22T13:30:51.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.362+0000 7f84fab65740 1 Processor -- start 2024-04-22T13:30:51.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.363+0000 7f84fab65740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec8c5aaf60 0x55ec8c5b5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.363+0000 7f84fab65740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec8c5ab910 0x55ec8c5b6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.363+0000 7f84fab65740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec8c5b8e90 0x55ec8c5b6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.363+0000 7f84fab65740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ec8c5bbc50 con 0x55ec8c5b8e90 2024-04-22T13:30:51.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.363+0000 7f84fab65740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ec8c4c92f0 con 0x55ec8c5ab910 2024-04-22T13:30:51.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.363+0000 7f84fab65740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ec8c5bbdd0 con 0x55ec8c5aaf60 2024-04-22T13:30:51.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.363+0000 7f84fab65740 10 client.0 _ll_register_callbacks cb 0x55ec8c4d6580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:51.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.363+0000 7f84f5510640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec8c5b8e90 0x55ec8c5b6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.363+0000 7f84f5510640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec8c5b8e90 0x55ec8c5b6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42674/0 (socket says 172.21.15.12:42674) 2024-04-22T13:30:51.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.363+0000 7f84f5510640 1 -- 172.21.15.12:0/2589297083 learned_addr learned my addr 172.21.15.12:0/2589297083 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:51.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.363+0000 7f84f5510640 1 --2- 172.21.15.12:0/2589297083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec8c5aaf60 0x55ec8c5b5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.363+0000 7f84f5d11640 1 --2- 172.21.15.12:0/2589297083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec8c5ab910 0x55ec8c5b6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.363+0000 7f84f5510640 1 -- 172.21.15.12:0/2589297083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec8c5aaf60 msgr2=0x55ec8c5b5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:51.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.363+0000 7f84f5510640 1 --2- 172.21.15.12:0/2589297083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec8c5aaf60 0x55ec8c5b5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.363+0000 7f84f5510640 1 -- 172.21.15.12:0/2589297083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec8c5ab910 msgr2=0x55ec8c5b6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:51.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.363+0000 7f84f5510640 1 --2- 172.21.15.12:0/2589297083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec8c5ab910 0x55ec8c5b6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.363+0000 7f84f5510640 1 -- 172.21.15.12:0/2589297083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec8c2223f0 con 0x55ec8c5b8e90 2024-04-22T13:30:51.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.363+0000 7f84f5510640 1 --2- 172.21.15.12:0/2589297083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec8c5b8e90 0x55ec8c5b6940 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f84e0009900 tx=0x7f84e0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78857f76a6d28145 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:51.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.363+0000 7f84d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:51.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.363+0000 7f84d97fa640 1 -- 172.21.15.12:0/2589297083 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e0005250 con 0x55ec8c5b8e90 2024-04-22T13:30:51.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.363+0000 7f84fab65740 1 -- 172.21.15.12:0/2589297083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84ec0118f0 con 0x55ec8c5b8e90 2024-04-22T13:30:51.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.363+0000 7f84d97fa640 1 -- 172.21.15.12:0/2589297083 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f84e00053f0 con 0x55ec8c5b8e90 2024-04-22T13:30:51.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.364+0000 7f84d97fa640 1 -- 172.21.15.12:0/2589297083 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f84e0055050 con 0x55ec8c5b8e90 2024-04-22T13:30:51.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.364+0000 7f84fab65740 10 client.18183 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:51.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.364+0000 7f84fab65740 1 -- 172.21.15.12:0/2589297083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ec8c1e20f0 con 0x55ec8c5b8e90 2024-04-22T13:30:51.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.364+0000 7f84d97fa640 1 -- 172.21.15.12:0/2589297083 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84ec0118f0 con 0x55ec8c5b8e90 2024-04-22T13:30:51.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.364+0000 7f84d97fa640 1 -- 172.21.15.12:0/2589297083 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f84e0003680 con 0x55ec8c5b8e90 2024-04-22T13:30:51.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.364+0000 7f84fab65740 10 client.18183 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:51.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.364+0000 7f84fab65740 10 client.18183 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:30:51.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.364+0000 7f84fab65740 20 client.18183 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:51.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.364+0000 7f84fab65740 1 --2- 172.21.15.12:0/2589297083 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55ec8c5c8de0 0x55ec8c5cb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.364+0000 7f84fab65740 4 client.18183 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:30:51.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.364+0000 7f84fab65740 1 -- 172.21.15.12:0/2589297083 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec8c51dc00 con 0x55ec8c5c8de0 2024-04-22T13:30:51.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.364+0000 7f84f5d11640 1 --2- 172.21.15.12:0/2589297083 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55ec8c5c8de0 0x55ec8c5cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.365+0000 7f84f5d11640 1 --2- 172.21.15.12:0/2589297083 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55ec8c5c8de0 0x55ec8c5cb240 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=353747b42bacd190 server_cookie=74ed6b63ca96f717 in_seq=0 out_seq=0 2024-04-22T13:30:51.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.365+0000 7f84d97fa640 10 client.18183 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:51.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.365+0000 7f84d97fa640 1 -- 172.21.15.12:0/2589297083 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ec8c51dc00 con 0x55ec8c5c8de0 2024-04-22T13:30:51.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.365+0000 7f84d97fa640 10 client.18183 handle_command_reply: tid=0 2024-04-22T13:30:51.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.365+0000 7f84fab65740 1 client.18183 shutdown 2024-04-22T13:30:51.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.366+0000 7f84fab65740 1 -- 172.21.15.12:0/2589297083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec8c5b8e90 msgr2=0x55ec8c5b6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:51.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.366+0000 7f84fab65740 1 --2- 172.21.15.12:0/2589297083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec8c5b8e90 0x55ec8c5b6940 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f84e0009900 tx=0x7f84e0009260 comp rx=0 tx=0).stop 2024-04-22T13:30:51.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.366+0000 7f84fab65740 1 -- 172.21.15.12:0/2589297083 shutdown_connections 2024-04-22T13:30:51.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.366+0000 7f84fab65740 1 --2- 172.21.15.12:0/2589297083 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55ec8c5c8de0 0x55ec8c5cb240 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.366+0000 7f84fab65740 1 --2- 172.21.15.12:0/2589297083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec8c5b8e90 0x55ec8c5b6940 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.366+0000 7f84fab65740 1 --2- 172.21.15.12:0/2589297083 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec8c5ab910 0x55ec8c5b6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.366+0000 7f84d97fa640 0 client.18183 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:51.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.366+0000 7f84fab65740 1 --2- 172.21.15.12:0/2589297083 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec8c5aaf60 0x55ec8c5b5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.366+0000 7f84fab65740 1 -- 172.21.15.12:0/2589297083 >> 172.21.15.12:0/2589297083 conn(0x55ec8c5a5cb0 msgr2=0x55ec8c5a7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:51.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.366+0000 7f84fab65740 1 -- 172.21.15.12:0/2589297083 shutdown_connections 2024-04-22T13:30:51.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.366+0000 7f84fab65740 1 -- 172.21.15.12:0/2589297083 wait complete. 2024-04-22T13:30:51.381 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.366+0000 7f84fab65740 20 client.18183 trim_cache size 0 max 16384 2024-04-22T13:30:51.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.384+0000 7f84fab65740 1 Processor -- start 2024-04-22T13:30:51.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.384+0000 7f84fab65740 1 -- start start 2024-04-22T13:30:51.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.384+0000 7f84fab65740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec8c5aaf60 0x55ec8c5b85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.384+0000 7f84fab65740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec8c5ab910 0x55ec8c5c4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.384+0000 7f84fab65740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec8c5b8e90 0x55ec8c5b5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.384+0000 7f84fab65740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f84f01ab940 con 0x55ec8c5b8e90 2024-04-22T13:30:51.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.384+0000 7f84fab65740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f84f01ad480 con 0x55ec8c5aaf60 2024-04-22T13:30:51.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.384+0000 7f84fab65740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f84f01ac010 con 0x55ec8c5ab910 2024-04-22T13:30:51.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.384+0000 7f84f5510640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec8c5b8e90 0x55ec8c5b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.384+0000 7f84f5510640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec8c5b8e90 0x55ec8c5b5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42676/0 (socket says 172.21.15.12:42676) 2024-04-22T13:30:51.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.384+0000 7f84f5510640 1 -- 172.21.15.12:0/3052918293 learned_addr learned my addr 172.21.15.12:0/3052918293 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:51.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.384+0000 7f84f5d11640 1 --2- 172.21.15.12:0/3052918293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec8c5ab910 0x55ec8c5c4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.384+0000 7f84f5510640 1 --2- 172.21.15.12:0/3052918293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec8c5aaf60 0x55ec8c5b85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.384+0000 7f84f5510640 1 -- 172.21.15.12:0/3052918293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec8c5ab910 msgr2=0x55ec8c5c4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:51.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.384+0000 7f84f5510640 1 --2- 172.21.15.12:0/3052918293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec8c5ab910 0x55ec8c5c4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.384+0000 7f84f5510640 1 -- 172.21.15.12:0/3052918293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec8c5aaf60 msgr2=0x55ec8c5b85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:51.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.384+0000 7f84f5510640 1 --2- 172.21.15.12:0/3052918293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec8c5aaf60 0x55ec8c5b85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.384+0000 7f84f5510640 1 -- 172.21.15.12:0/3052918293 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec8c5aa670 con 0x55ec8c5b8e90 2024-04-22T13:30:51.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.384+0000 7f84f5d11640 1 --2- 172.21.15.12:0/3052918293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec8c5ab910 0x55ec8c5c4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:30:51.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.385+0000 7f84f5510640 1 --2- 172.21.15.12:0/3052918293 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec8c5b8e90 0x55ec8c5b5040 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f84e0009850 tx=0x7f84e0036040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7815b60cd2710a2e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:51.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.385+0000 7f84db7fe640 1 -- 172.21.15.12:0/3052918293 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e002cca0 con 0x55ec8c5b8e90 2024-04-22T13:30:51.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.385+0000 7f84db7fe640 1 -- 172.21.15.12:0/3052918293 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84e0005700 con 0x55ec8c5b8e90 2024-04-22T13:30:51.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.385+0000 7f84db7fe640 1 -- 172.21.15.12:0/3052918293 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e0004ae0 con 0x55ec8c5b8e90 2024-04-22T13:30:51.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.385+0000 7f84fab65740 1 -- 172.21.15.12:0/3052918293 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec8c5b8e90 msgr2=0x55ec8c5b5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:51.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.385+0000 7f84fab65740 1 --2- 172.21.15.12:0/3052918293 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec8c5b8e90 0x55ec8c5b5040 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f84e0009850 tx=0x7f84e0036040 comp rx=0 tx=0).stop 2024-04-22T13:30:51.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.385+0000 7f84fab65740 1 -- 172.21.15.12:0/3052918293 shutdown_connections 2024-04-22T13:30:51.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.385+0000 7f84fab65740 1 --2- 172.21.15.12:0/3052918293 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec8c5b8e90 0x55ec8c5b5040 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.385+0000 7f84fab65740 1 --2- 172.21.15.12:0/3052918293 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec8c5ab910 0x55ec8c5c4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.385+0000 7f84fab65740 1 --2- 172.21.15.12:0/3052918293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec8c5aaf60 0x55ec8c5b85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.385+0000 7f84fab65740 1 -- 172.21.15.12:0/3052918293 >> 172.21.15.12:0/3052918293 conn(0x55ec8c5a5cb0 msgr2=0x55ec8c5b5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:51.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.385+0000 7f84fab65740 1 -- 172.21.15.12:0/3052918293 shutdown_connections 2024-04-22T13:30:51.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.386+0000 7f84fab65740 1 -- 172.21.15.12:0/3052918293 wait complete. 2024-04-22T13:30:51.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.386+0000 7f84fab65740 1 -- start start 2024-04-22T13:30:51.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.386+0000 7f84fab65740 1 Processor -- start 2024-04-22T13:30:51.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.386+0000 7f84fab65740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec8c5aaf60 0x55ec8c5c1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.386+0000 7f84fab65740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec8c5ab910 0x55ec8c5b2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.386+0000 7f84fab65740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec8c5b8e90 0x55ec8c5b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.386+0000 7f84fab65740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f84f01acdb0 con 0x55ec8c5b8e90 2024-04-22T13:30:51.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.386+0000 7f84fab65740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f84f01ac6e0 con 0x55ec8c5aaf60 2024-04-22T13:30:51.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.386+0000 7f84fab65740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f84f01a9960 con 0x55ec8c5ab910 2024-04-22T13:30:51.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.386+0000 7f84fab65740 10 client.0 _ll_register_callbacks cb 0x7f84e0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:30:51.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.386+0000 7f84f5510640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec8c5b8e90 0x55ec8c5b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.386+0000 7f84f5510640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec8c5b8e90 0x55ec8c5b2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:42686/0 (socket says 172.21.15.12:42686) 2024-04-22T13:30:51.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.386+0000 7f84f5510640 1 -- 172.21.15.12:0/1317435146 learned_addr learned my addr 172.21.15.12:0/1317435146 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:30:51.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.387+0000 7f84f5d11640 1 --2- 172.21.15.12:0/1317435146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec8c5ab910 0x55ec8c5b2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.387+0000 7f84f5510640 1 --2- 172.21.15.12:0/1317435146 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec8c5aaf60 0x55ec8c5c1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.387+0000 7f84f5510640 1 -- 172.21.15.12:0/1317435146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec8c5ab910 msgr2=0x55ec8c5b2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:51.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.387+0000 7f84f5510640 1 --2- 172.21.15.12:0/1317435146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec8c5ab910 0x55ec8c5b2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.397 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.387+0000 7f84f5510640 1 -- 172.21.15.12:0/1317435146 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec8c5aaf60 msgr2=0x55ec8c5c1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:30:51.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.387+0000 7f84f5510640 1 --2- 172.21.15.12:0/1317435146 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec8c5aaf60 0x55ec8c5c1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.387+0000 7f84f5510640 1 -- 172.21.15.12:0/1317435146 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec8c5bb280 con 0x55ec8c5b8e90 2024-04-22T13:30:51.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.387+0000 7f84f5510640 1 --2- 172.21.15.12:0/1317435146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec8c5b8e90 0x55ec8c5b2e50 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f84e002cc00 tx=0x7f84e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b542568d578df38 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:30:51.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.387+0000 7f84d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:30:51.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.387+0000 7f84d97fa640 1 -- 172.21.15.12:0/1317435146 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e00035f0 con 0x55ec8c5b8e90 2024-04-22T13:30:51.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.387+0000 7f84d97fa640 1 -- 172.21.15.12:0/1317435146 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f84e0003790 con 0x55ec8c5b8e90 2024-04-22T13:30:51.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.387+0000 7f84fab65740 1 -- 172.21.15.12:0/1317435146 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84e0005700 con 0x55ec8c5b8e90 2024-04-22T13:30:51.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.387+0000 7f84d97fa640 1 -- 172.21.15.12:0/1317435146 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f84e0004860 con 0x55ec8c5b8e90 2024-04-22T13:30:51.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.387+0000 7f84fab65740 10 client.18189 fetch_fsmap learned FSMap version 28 2024-04-22T13:30:51.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.387+0000 7f84fab65740 1 -- 172.21.15.12:0/1317435146 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ec8c62f210 con 0x55ec8c5b8e90 2024-04-22T13:30:51.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.387+0000 7f84d97fa640 1 -- 172.21.15.12:0/1317435146 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84e0005700 con 0x55ec8c5b8e90 2024-04-22T13:30:51.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.388+0000 7f84d97fa640 1 -- 172.21.15.12:0/1317435146 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f84e00347a0 con 0x55ec8c5b8e90 2024-04-22T13:30:51.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.388+0000 7f84fab65740 10 client.18189 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:30:51.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.388+0000 7f84fab65740 10 client.18189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:30:51.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.388+0000 7f84fab65740 20 client.18189 populate_metadata read hostname 'smithi012' 2024-04-22T13:30:51.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.388+0000 7f84fab65740 1 --2- 172.21.15.12:0/1317435146 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55ec8c62f720 0x55ec8c631b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:30:51.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.388+0000 7f84fab65740 4 client.18189 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:30:51.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.388+0000 7f84fab65740 1 -- 172.21.15.12:0/1317435146 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ec8c517a70 con 0x55ec8c62f720 2024-04-22T13:30:51.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.388+0000 7f84f5d11640 1 --2- 172.21.15.12:0/1317435146 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55ec8c62f720 0x55ec8c631b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:30:51.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.388+0000 7f84f5d11640 1 --2- 172.21.15.12:0/1317435146 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55ec8c62f720 0x55ec8c631b40 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a2886050c4d11fe server_cookie=b00841bf9c0ac19c in_seq=0 out_seq=0 2024-04-22T13:30:51.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.388+0000 7f84d97fa640 10 client.18189 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:51.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.389+0000 7f84d97fa640 1 -- 172.21.15.12:0/1317435146 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ec8c517a70 con 0x55ec8c62f720 2024-04-22T13:30:51.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.389+0000 7f84d97fa640 10 client.18189 handle_command_reply: tid=0 2024-04-22T13:30:51.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.389+0000 7f84fab65740 1 client.18189 shutdown 2024-04-22T13:30:51.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.389+0000 7f84fab65740 1 -- 172.21.15.12:0/1317435146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec8c5b8e90 msgr2=0x55ec8c5b2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:51.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.389+0000 7f84fab65740 1 --2- 172.21.15.12:0/1317435146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec8c5b8e90 0x55ec8c5b2e50 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f84e002cc00 tx=0x7f84e00046c0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.389+0000 7f84fab65740 1 -- 172.21.15.12:0/1317435146 shutdown_connections 2024-04-22T13:30:51.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.389+0000 7f84fab65740 1 --2- 172.21.15.12:0/1317435146 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55ec8c62f720 0x55ec8c631b40 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.389+0000 7f84fab65740 1 --2- 172.21.15.12:0/1317435146 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ec8c5b8e90 0x55ec8c5b2e50 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.389+0000 7f84fab65740 1 --2- 172.21.15.12:0/1317435146 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ec8c5ab910 0x55ec8c5b2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.389+0000 7f84d97fa640 0 client.18189 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:30:51.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.389+0000 7f84fab65740 1 --2- 172.21.15.12:0/1317435146 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ec8c5aaf60 0x55ec8c5c1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.389+0000 7f84fab65740 1 -- 172.21.15.12:0/1317435146 >> 172.21.15.12:0/1317435146 conn(0x55ec8c5a5cb0 msgr2=0x55ec8c5a4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:51.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.389+0000 7f84fab65740 1 -- 172.21.15.12:0/1317435146 shutdown_connections 2024-04-22T13:30:51.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.389+0000 7f84fab65740 1 -- 172.21.15.12:0/1317435146 wait complete. 2024-04-22T13:30:51.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.389+0000 7f84fab65740 20 client.18189 trim_cache size 0 max 16384 2024-04-22T13:30:51.403 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:30:51.403 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:30:51.403 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-04-22T13:30:51.403 INFO:teuthology.orchestra.run.smithi012.stdout: "2fdfb73e-ada3-48a6-a0ab-5460ecb4ae20": { 2024-04-22T13:30:51.403 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-04-22T13:30:51.403 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "2fdfb73e-ada3-48a6-a0ab-5460ecb4ae20", 2024-04-22T13:30:51.404 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-04-22T13:30:51.404 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:30:51.404 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:30:51.404 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:30:51.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.389+0000 7f84f7f75640 1 -- 172.21.15.12:0/2910102347 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f84dc0a3d20 msgr2=0x7f84dc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:51.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.389+0000 7f84f7f75640 1 --2- 172.21.15.12:0/2910102347 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f84dc0a3d20 0x7f84dc0a61e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f84e80ce5b0 tx=0x7f84e801bb30 comp rx=0 tx=0).stop 2024-04-22T13:30:51.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.389+0000 7f84f7f75640 1 -- 172.21.15.12:0/2910102347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84f0198670 msgr2=0x7f84f0198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:30:51.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.389+0000 7f84f7f75640 1 --2- 172.21.15.12:0/2910102347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84f0198670 0x7f84f0198b20 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f84e800eb10 tx=0x7f84e800c2d0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.389+0000 7f84f7f75640 1 -- 172.21.15.12:0/2910102347 shutdown_connections 2024-04-22T13:30:51.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.389+0000 7f84f7f75640 1 --2- 172.21.15.12:0/2910102347 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f84dc0a3d20 0x7f84dc0a61e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.389+0000 7f84f7f75640 1 --2- 172.21.15.12:0/2910102347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84f0198670 0x7f84f0198b20 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.389+0000 7f84f7f75640 1 --2- 172.21.15.12:0/2910102347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84f0107270 0x7f84f019e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.389+0000 7f84f7f75640 1 --2- 172.21.15.12:0/2910102347 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84f01021a0 0x7f84f019e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:30:51.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.389+0000 7f84f7f75640 1 -- 172.21.15.12:0/2910102347 >> 172.21.15.12:0/2910102347 conn(0x7f84f00fda40 msgr2=0x7f84f00fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:30:51.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.390+0000 7f84f7f75640 1 -- 172.21.15.12:0/2910102347 shutdown_connections 2024-04-22T13:30:51.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:30:51.390+0000 7f84f7f75640 1 -- 172.21.15.12:0/2910102347 wait complete. 2024-04-22T13:30:51.407 INFO:tasks.cephfs.filesystem:scrub status for tag:2fdfb73e-ada3-48a6-a0ab-5460ecb4ae20 - {'path': '/', 'tag': '2fdfb73e-ada3-48a6-a0ab-5460ecb4ae20', 'options': 'recursive,force'} 2024-04-22T13:30:52.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:51 smithi134 ceph-mon[31338]: pgmap v205: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:30:52.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:51 smithi134 ceph-mon[31338]: scrub queued for path: / 2024-04-22T13:30:52.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:51 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:30:52.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:51 smithi134 ceph-mon[31338]: scrub summary: active paths [/] 2024-04-22T13:30:52.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:51 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:30:52.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:51 smithi111 ceph-mon[31361]: pgmap v205: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:30:52.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:51 smithi111 ceph-mon[31361]: scrub queued for path: / 2024-04-22T13:30:52.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:51 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:30:52.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:51 smithi111 ceph-mon[31361]: scrub summary: active paths [/] 2024-04-22T13:30:52.227 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:51 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:30:52.337 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:51 smithi012 ceph-mon[28265]: pgmap v205: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:30:52.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:51 smithi012 ceph-mon[28265]: scrub queued for path: / 2024-04-22T13:30:52.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:51 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:30:52.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:51 smithi012 ceph-mon[28265]: scrub summary: active paths [/] 2024-04-22T13:30:52.338 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:51 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:30:54.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:53 smithi134 ceph-mon[31338]: pgmap v206: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 13 op/s 2024-04-22T13:30:54.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:53 smithi111 ceph-mon[31361]: pgmap v206: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 13 op/s 2024-04-22T13:30:54.336 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:53 smithi012 ceph-mon[28265]: pgmap v206: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 13 op/s 2024-04-22T13:30:55.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:54 smithi134 ceph-mon[31338]: scrub summary: idle 2024-04-22T13:30:55.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:54 smithi111 ceph-mon[31361]: scrub summary: idle 2024-04-22T13:30:55.336 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:54 smithi012 ceph-mon[28265]: scrub summary: idle 2024-04-22T13:30:56.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:55 smithi134 ceph-mon[31338]: pgmap v207: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:30:56.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:55 smithi111 ceph-mon[31361]: pgmap v207: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:30:56.336 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:55 smithi012 ceph-mon[28265]: pgmap v207: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:30:58.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:57 smithi134 ceph-mon[31338]: pgmap v208: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:30:58.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:57 smithi111 ceph-mon[31361]: pgmap v208: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:30:58.336 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:57 smithi012 ceph-mon[28265]: pgmap v208: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:30:59.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:30:58 smithi134 ceph-mon[31338]: pgmap v209: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 13 op/s 2024-04-22T13:30:59.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:30:58 smithi111 ceph-mon[31361]: pgmap v209: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 13 op/s 2024-04-22T13:30:59.336 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:30:58 smithi012 ceph-mon[28265]: pgmap v209: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 13 op/s 2024-04-22T13:31:01.586 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:01 smithi012 ceph-mon[28265]: pgmap v210: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:31:01.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:01 smithi134 ceph-mon[31338]: pgmap v210: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:31:01.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:01 smithi111 ceph-mon[31361]: pgmap v210: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:31:03.586 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:03 smithi012 ceph-mon[28265]: pgmap v211: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-22T13:31:03.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:03 smithi134 ceph-mon[31338]: pgmap v211: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-22T13:31:03.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:03 smithi111 ceph-mon[31361]: pgmap v211: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-22T13:31:05.586 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:05 smithi012 ceph-mon[28265]: pgmap v212: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:31:05.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:05 smithi134 ceph-mon[31338]: pgmap v212: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:31:05.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:05 smithi111 ceph-mon[31361]: pgmap v212: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:31:07.586 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:07 smithi012 ceph-mon[28265]: pgmap v213: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:31:07.676 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:07 smithi134 ceph-mon[31338]: pgmap v213: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:31:07.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:07 smithi111 ceph-mon[31361]: pgmap v213: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:31:09.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:09 smithi134 ceph-mon[31338]: pgmap v214: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-22T13:31:09.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:09 smithi111 ceph-mon[31361]: pgmap v214: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-22T13:31:09.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:09 smithi012 ceph-mon[28265]: pgmap v214: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-22T13:31:11.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:11 smithi134 ceph-mon[31338]: pgmap v215: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:31:11.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:11 smithi134 ceph-mon[31338]: osdmap e91: 12 total, 12 up, 12 in 2024-04-22T13:31:11.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:11 smithi111 ceph-mon[31361]: pgmap v215: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:31:11.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:11 smithi111 ceph-mon[31361]: osdmap e91: 12 total, 12 up, 12 in 2024-04-22T13:31:11.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:11 smithi012 ceph-mon[28265]: pgmap v215: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:31:11.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:11 smithi012 ceph-mon[28265]: osdmap e91: 12 total, 12 up, 12 in 2024-04-22T13:31:13.677 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:13 smithi134 ceph-mon[31338]: pgmap v217: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:31:13.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:13 smithi111 ceph-mon[31361]: pgmap v217: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:31:13.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:13 smithi012 ceph-mon[28265]: pgmap v217: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:31:15.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:15 smithi012 ceph-mon[28265]: pgmap v218: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:31:15.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:15 smithi012 ceph-mon[28265]: osdmap e92: 12 total, 12 up, 12 in 2024-04-22T13:31:15.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:15 smithi134 ceph-mon[31338]: pgmap v218: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:31:15.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:15 smithi134 ceph-mon[31338]: osdmap e92: 12 total, 12 up, 12 in 2024-04-22T13:31:15.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:15 smithi111 ceph-mon[31361]: pgmap v218: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:31:15.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:15 smithi111 ceph-mon[31361]: osdmap e92: 12 total, 12 up, 12 in 2024-04-22T13:31:17.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:17 smithi012 ceph-mon[28265]: pgmap v220: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-22T13:31:17.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:17 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:31:17.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:17 smithi134 ceph-mon[31338]: pgmap v220: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-22T13:31:17.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:17 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:31:17.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:17 smithi111 ceph-mon[31361]: pgmap v220: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-22T13:31:17.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:17 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:31:18.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:18 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:31:18.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:18 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:31:18.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:18 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:31:18.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:18 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:31:18.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:18 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:31:18.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:18 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:31:18.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:18 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:31:18.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:18 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:31:18.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:18 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:31:19.828 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:19 smithi134 ceph-mon[31338]: pgmap v221: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-22T13:31:19.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:19 smithi012 ceph-mon[28265]: pgmap v221: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-22T13:31:19.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:19 smithi111 ceph-mon[31361]: pgmap v221: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-22T13:31:21.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:31:21.409 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:31:21.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.532+0000 7f7c45915640 1 -- 172.21.15.12:0/388688855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c40105dd0 msgr2=0x7f7c401061b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:21.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.532+0000 7f7c45915640 1 --2- 172.21.15.12:0/388688855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c40105dd0 0x7f7c401061b0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009900 tx=0x7f7c3002bc30 comp rx=0 tx=0).stop 2024-04-22T13:31:21.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.532+0000 7f7c45915640 1 -- 172.21.15.12:0/388688855 shutdown_connections 2024-04-22T13:31:21.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.532+0000 7f7c45915640 1 --2- 172.21.15.12:0/388688855 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c40101a40 0x7f7c4010dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.532+0000 7f7c45915640 1 --2- 172.21.15.12:0/388688855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c401010a0 0x7f7c40101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.532+0000 7f7c45915640 1 --2- 172.21.15.12:0/388688855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c40105dd0 0x7f7c401061b0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.532+0000 7f7c45915640 1 -- 172.21.15.12:0/388688855 >> 172.21.15.12:0/388688855 conn(0x7f7c40077c40 msgr2=0x7f7c40078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:21.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.532+0000 7f7c45915640 1 -- 172.21.15.12:0/388688855 shutdown_connections 2024-04-22T13:31:21.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.532+0000 7f7c45915640 1 -- 172.21.15.12:0/388688855 wait complete. 2024-04-22T13:31:21.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.533+0000 7f7c45915640 1 Processor -- start 2024-04-22T13:31:21.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.533+0000 7f7c45915640 1 -- start start 2024-04-22T13:31:21.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.533+0000 7f7c45915640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c401010a0 0x7f7c400feba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:21.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.533+0000 7f7c45915640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c40101a40 0x7f7c400ff0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:21.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.533+0000 7f7c45915640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c40105dd0 0x7f7c400ff620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:21.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.533+0000 7f7c45915640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7c40111150 con 0x7f7c401010a0 2024-04-22T13:31:21.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.533+0000 7f7c45915640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7c40110fd0 con 0x7f7c40105dd0 2024-04-22T13:31:21.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.533+0000 7f7c45915640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7c401112d0 con 0x7f7c40101a40 2024-04-22T13:31:21.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.533+0000 7f7c3effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c401010a0 0x7f7c400feba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:21.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.533+0000 7f7c3effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c401010a0 0x7f7c400feba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34792/0 (socket says 172.21.15.12:34792) 2024-04-22T13:31:21.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.533+0000 7f7c3effd640 1 -- 172.21.15.12:0/390441994 learned_addr learned my addr 172.21.15.12:0/390441994 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:21.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.534+0000 7f7c3f7fe640 1 --2- 172.21.15.12:0/390441994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c40105dd0 0x7f7c400ff620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:21.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.534+0000 7f7c3e7fc640 1 --2- 172.21.15.12:0/390441994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c40101a40 0x7f7c400ff0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:21.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.534+0000 7f7c3effd640 1 -- 172.21.15.12:0/390441994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c40101a40 msgr2=0x7f7c400ff0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:21.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.534+0000 7f7c3effd640 1 --2- 172.21.15.12:0/390441994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c40101a40 0x7f7c400ff0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.534+0000 7f7c3effd640 1 -- 172.21.15.12:0/390441994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c40105dd0 msgr2=0x7f7c400ff620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:21.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.534+0000 7f7c3effd640 1 --2- 172.21.15.12:0/390441994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c40105dd0 0x7f7c400ff620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.534+0000 7f7c3effd640 1 -- 172.21.15.12:0/390441994 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c401a6c70 con 0x7f7c401010a0 2024-04-22T13:31:21.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.534+0000 7f7c3e7fc640 1 --2- 172.21.15.12:0/390441994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c40101a40 0x7f7c400ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:21.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.534+0000 7f7c3effd640 1 --2- 172.21.15.12:0/390441994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c401010a0 0x7f7c400feba0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f7c30005940 tx=0x7f7c3002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:21.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.534+0000 7f7c44913640 1 -- 172.21.15.12:0/390441994 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30043020 con 0x7f7c401010a0 2024-04-22T13:31:21.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.534+0000 7f7c44913640 1 -- 172.21.15.12:0/390441994 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c3002cbe0 con 0x7f7c401010a0 2024-04-22T13:31:21.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.534+0000 7f7c44913640 1 -- 172.21.15.12:0/390441994 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30003bf0 con 0x7f7c401010a0 2024-04-22T13:31:21.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.534+0000 7f7c45915640 1 -- 172.21.15.12:0/390441994 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c401a6f00 con 0x7f7c401010a0 2024-04-22T13:31:21.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.534+0000 7f7c45915640 1 -- 172.21.15.12:0/390441994 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c401a7370 con 0x7f7c401010a0 2024-04-22T13:31:21.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.536+0000 7f7c44913640 1 -- 172.21.15.12:0/390441994 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7c30046020 con 0x7f7c401010a0 2024-04-22T13:31:21.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.537+0000 7f7c485de740 1 Processor -- start 2024-04-22T13:31:21.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.537+0000 7f7c485de740 1 -- start start 2024-04-22T13:31:21.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.537+0000 7f7c485de740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e391c7580 0x555e391c7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:21.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.537+0000 7f7c485de740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e391c7ea0 0x555e391d0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:21.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.537+0000 7f7c485de740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e391d0a80 0x555e391d2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:21.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.537+0000 7f7c485de740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555e3912c1e0 con 0x555e391c7580 2024-04-22T13:31:21.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.537+0000 7f7c485de740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555e390c9390 con 0x555e391d0a80 2024-04-22T13:31:21.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.537+0000 7f7c44913640 1 --2- 172.21.15.12:0/390441994 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7c280a3d20 0x7f7c280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:21.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.537+0000 7f7c485de740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555e391281a0 con 0x555e391c7ea0 2024-04-22T13:31:21.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.537+0000 7f7c3f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e391c7580 0x555e391c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:21.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.537+0000 7f7c3f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e391c7580 0x555e391c7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34808/0 (socket says 172.21.15.12:34808) 2024-04-22T13:31:21.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.537+0000 7f7c3f7fe640 1 -- 172.21.15.12:0/1873085663 learned_addr learned my addr 172.21.15.12:0/1873085663 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:21.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.537+0000 7f7c3f7fe640 1 --2- 172.21.15.12:0/390441994 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7c280a3d20 0x7f7c280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:21.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.537+0000 7f7c44913640 1 -- 172.21.15.12:0/390441994 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f7c300f2910 con 0x7f7c401010a0 2024-04-22T13:31:21.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.537+0000 7f7c3e7fc640 1 --2- 172.21.15.12:0/1873085663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e391d0a80 0x555e391d2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:21.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.537+0000 7f7c3effd640 1 --2- 172.21.15.12:0/1873085663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e391c7ea0 0x555e391d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:21.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.538+0000 7f7c3f7fe640 1 -- 172.21.15.12:0/1873085663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e391c7ea0 msgr2=0x555e391d0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:21.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.538+0000 7f7c3f7fe640 1 --2- 172.21.15.12:0/1873085663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e391c7ea0 0x555e391d0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.538+0000 7f7c3f7fe640 1 -- 172.21.15.12:0/1873085663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e391d0a80 msgr2=0x555e391d2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:21.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.538+0000 7f7c3f7fe640 1 --2- 172.21.15.12:0/1873085663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e391d0a80 0x555e391d2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.538+0000 7f7c3f7fe640 1 -- 172.21.15.12:0/1873085663 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e38f087d0 con 0x555e391c7580 2024-04-22T13:31:21.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.538+0000 7f7c3effd640 1 --2- 172.21.15.12:0/1873085663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e391c7ea0 0x555e391d0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:21.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.538+0000 7f7c3e7fc640 1 --2- 172.21.15.12:0/1873085663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e391d0a80 0x555e391d2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:21.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.538+0000 7f7c3f7fe640 1 --2- 172.21.15.12:0/1873085663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e391c7580 0x555e391c7960 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f7c38012d00 tx=0x7f7c38010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1a829fb0851483 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:21.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.538+0000 7f7c3f7fe640 1 --2- 172.21.15.12:0/390441994 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7c280a3d20 0x7f7c280a61e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7c38010c40 tx=0x7f7c3800c000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:21.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.538+0000 7f7c25ffb640 1 -- 172.21.15.12:0/1873085663 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c38024070 con 0x555e391c7580 2024-04-22T13:31:21.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.538+0000 7f7c25ffb640 1 -- 172.21.15.12:0/1873085663 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c38009d20 con 0x555e391c7580 2024-04-22T13:31:21.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.538+0000 7f7c25ffb640 1 -- 172.21.15.12:0/1873085663 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c38005280 con 0x555e391c7580 2024-04-22T13:31:21.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.539+0000 7f7c485de740 1 -- 172.21.15.12:0/1873085663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e391c7580 msgr2=0x555e391c7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:21.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.539+0000 7f7c485de740 1 --2- 172.21.15.12:0/1873085663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e391c7580 0x555e391c7960 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f7c38012d00 tx=0x7f7c38010740 comp rx=0 tx=0).stop 2024-04-22T13:31:21.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.539+0000 7f7c485de740 1 -- 172.21.15.12:0/1873085663 shutdown_connections 2024-04-22T13:31:21.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.539+0000 7f7c485de740 1 --2- 172.21.15.12:0/1873085663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e391d0a80 0x555e391d2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.539+0000 7f7c485de740 1 --2- 172.21.15.12:0/1873085663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e391c7ea0 0x555e391d0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.539+0000 7f7c485de740 1 --2- 172.21.15.12:0/1873085663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e391c7580 0x555e391c7960 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.539+0000 7f7c485de740 1 -- 172.21.15.12:0/1873085663 >> 172.21.15.12:0/1873085663 conn(0x555e391c1cb0 msgr2=0x555e391c2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:21.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.539+0000 7f7c485de740 1 -- 172.21.15.12:0/1873085663 shutdown_connections 2024-04-22T13:31:21.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.539+0000 7f7c485de740 1 -- 172.21.15.12:0/1873085663 wait complete. 2024-04-22T13:31:21.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.539+0000 7f7c485de740 1 -- start start 2024-04-22T13:31:21.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.540+0000 7f7c485de740 1 Processor -- start 2024-04-22T13:31:21.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.540+0000 7f7c485de740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e391c7580 0x555e391cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:21.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.540+0000 7f7c485de740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e391c7ea0 0x555e391ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:21.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.540+0000 7f7c485de740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e391d0a80 0x555e391ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:21.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.540+0000 7f7c485de740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555e391d3810 con 0x555e391d0a80 2024-04-22T13:31:21.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.540+0000 7f7c485de740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555e390e52f0 con 0x555e391c7ea0 2024-04-22T13:31:21.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.540+0000 7f7c485de740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555e391d3990 con 0x555e391c7580 2024-04-22T13:31:21.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.540+0000 7f7c485de740 10 client.0 _ll_register_callbacks cb 0x555e390f2580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:21.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.540+0000 7f7c3f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e391d0a80 0x555e391ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:21.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.540+0000 7f7c3f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e391d0a80 0x555e391ce600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34824/0 (socket says 172.21.15.12:34824) 2024-04-22T13:31:21.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.540+0000 7f7c3f7fe640 1 -- 172.21.15.12:0/813470641 learned_addr learned my addr 172.21.15.12:0/813470641 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:21.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.540+0000 7f7c3effd640 1 --2- 172.21.15.12:0/813470641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e391c7580 0x555e391cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:21.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.540+0000 7f7c3e7fc640 1 --2- 172.21.15.12:0/813470641 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e391c7ea0 0x555e391ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:21.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.540+0000 7f7c3f7fe640 1 -- 172.21.15.12:0/813470641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e391c7580 msgr2=0x555e391cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:21.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.540+0000 7f7c3f7fe640 1 --2- 172.21.15.12:0/813470641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e391c7580 0x555e391cdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.540+0000 7f7c3f7fe640 1 -- 172.21.15.12:0/813470641 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e391c7ea0 msgr2=0x555e391ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:21.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.540+0000 7f7c3f7fe640 1 --2- 172.21.15.12:0/813470641 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e391c7ea0 0x555e391ce0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.540+0000 7f7c3f7fe640 1 -- 172.21.15.12:0/813470641 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e38e3e3f0 con 0x555e391d0a80 2024-04-22T13:31:21.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.540+0000 7f7c3f7fe640 1 --2- 172.21.15.12:0/813470641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e391d0a80 0x555e391ce600 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f7c38009a40 tx=0x7f7c38012880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=140628eef0ba7847 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:21.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.540+0000 7f7c1effd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:21.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.541+0000 7f7c1effd640 1 -- 172.21.15.12:0/813470641 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c38024040 con 0x555e391d0a80 2024-04-22T13:31:21.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.541+0000 7f7c1effd640 1 -- 172.21.15.12:0/813470641 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f7c38004c90 con 0x555e391d0a80 2024-04-22T13:31:21.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.541+0000 7f7c485de740 1 -- 172.21.15.12:0/813470641 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555e39139c00 con 0x555e391d0a80 2024-04-22T13:31:21.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.541+0000 7f7c1effd640 1 -- 172.21.15.12:0/813470641 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7c38020430 con 0x555e391d0a80 2024-04-22T13:31:21.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.541+0000 7f7c485de740 10 client.18201 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:21.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.541+0000 7f7c485de740 1 -- 172.21.15.12:0/813470641 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555e38dfe0f0 con 0x555e391d0a80 2024-04-22T13:31:21.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.541+0000 7f7c1effd640 1 -- 172.21.15.12:0/813470641 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555e39139c00 con 0x555e391d0a80 2024-04-22T13:31:21.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.541+0000 7f7c1effd640 1 -- 172.21.15.12:0/813470641 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7c38037030 con 0x555e391d0a80 2024-04-22T13:31:21.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.541+0000 7f7c485de740 10 client.18201 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:21.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.541+0000 7f7c485de740 10 client.18201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:21.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.541+0000 7f7c485de740 20 client.18201 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:21.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.541+0000 7f7c485de740 1 --2- 172.21.15.12:0/813470641 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555e391e0a00 0x555e391e2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:21.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.541+0000 7f7c485de740 4 client.18201 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:21.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.541+0000 7f7c485de740 1 -- 172.21.15.12:0/813470641 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e39133a70 con 0x555e391e0a00 2024-04-22T13:31:21.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.542+0000 7f7c3effd640 1 --2- 172.21.15.12:0/813470641 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555e391e0a00 0x555e391e2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:21.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.542+0000 7f7c3effd640 1 --2- 172.21.15.12:0/813470641 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555e391e0a00 0x555e391e2df0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9148874a1c35a24 server_cookie=9af26389d302ce5f in_seq=0 out_seq=0 2024-04-22T13:31:21.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.542+0000 7f7c1effd640 10 client.18201 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:21.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.543+0000 7f7c1effd640 1 -- 172.21.15.12:0/813470641 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555e39133a70 con 0x555e391e0a00 2024-04-22T13:31:21.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.543+0000 7f7c1effd640 10 client.18201 handle_command_reply: tid=0 2024-04-22T13:31:21.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.543+0000 7f7c485de740 1 client.18201 shutdown 2024-04-22T13:31:21.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.543+0000 7f7c485de740 1 -- 172.21.15.12:0/813470641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e391d0a80 msgr2=0x555e391ce600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:21.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.543+0000 7f7c485de740 1 --2- 172.21.15.12:0/813470641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e391d0a80 0x555e391ce600 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f7c38009a40 tx=0x7f7c38012880 comp rx=0 tx=0).stop 2024-04-22T13:31:21.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.543+0000 7f7c485de740 1 -- 172.21.15.12:0/813470641 shutdown_connections 2024-04-22T13:31:21.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.543+0000 7f7c485de740 1 --2- 172.21.15.12:0/813470641 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555e391e0a00 0x555e391e2df0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.543+0000 7f7c485de740 1 --2- 172.21.15.12:0/813470641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e391d0a80 0x555e391ce600 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.543+0000 7f7c485de740 1 --2- 172.21.15.12:0/813470641 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e391c7ea0 0x555e391ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.543+0000 7f7c485de740 1 --2- 172.21.15.12:0/813470641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e391c7580 0x555e391cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.543+0000 7f7c1effd640 0 client.18201 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:21.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.543+0000 7f7c485de740 1 -- 172.21.15.12:0/813470641 >> 172.21.15.12:0/813470641 conn(0x555e391c1cb0 msgr2=0x555e391c3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:21.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.543+0000 7f7c485de740 1 -- 172.21.15.12:0/813470641 shutdown_connections 2024-04-22T13:31:21.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.543+0000 7f7c485de740 1 -- 172.21.15.12:0/813470641 wait complete. 2024-04-22T13:31:21.546 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.543+0000 7f7c485de740 20 client.18201 trim_cache size 0 max 16384 2024-04-22T13:31:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.561+0000 7f7c485de740 1 Processor -- start 2024-04-22T13:31:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.561+0000 7f7c485de740 1 -- start start 2024-04-22T13:31:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.561+0000 7f7c485de740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e391c7580 0x555e391d02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.561+0000 7f7c485de740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e391c7ea0 0x555e391dc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.561+0000 7f7c485de740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e391d0a80 0x555e391ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.561+0000 7f7c485de740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7c401afd80 con 0x555e391d0a80 2024-04-22T13:31:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.561+0000 7f7c485de740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7c401b18c0 con 0x555e391c7ea0 2024-04-22T13:31:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.561+0000 7f7c485de740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7c401b0450 con 0x555e391c7580 2024-04-22T13:31:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.561+0000 7f7c3f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e391d0a80 0x555e391ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.561+0000 7f7c3f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e391d0a80 0x555e391ccce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34830/0 (socket says 172.21.15.12:34830) 2024-04-22T13:31:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.561+0000 7f7c3f7fe640 1 -- 172.21.15.12:0/2695893585 learned_addr learned my addr 172.21.15.12:0/2695893585 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:21.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.561+0000 7f7c3effd640 1 --2- 172.21.15.12:0/2695893585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e391c7580 0x555e391d02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.562+0000 7f7c3e7fc640 1 --2- 172.21.15.12:0/2695893585 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e391c7ea0 0x555e391dc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.562+0000 7f7c3f7fe640 1 -- 172.21.15.12:0/2695893585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e391c7580 msgr2=0x555e391d02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.562+0000 7f7c3f7fe640 1 --2- 172.21.15.12:0/2695893585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e391c7580 0x555e391d02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.562+0000 7f7c3f7fe640 1 -- 172.21.15.12:0/2695893585 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e391c7ea0 msgr2=0x555e391dc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.562+0000 7f7c3f7fe640 1 --2- 172.21.15.12:0/2695893585 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e391c7ea0 0x555e391dc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.562+0000 7f7c3f7fe640 1 -- 172.21.15.12:0/2695893585 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e391c7cd0 con 0x555e391d0a80 2024-04-22T13:31:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.562+0000 7f7c3f7fe640 1 --2- 172.21.15.12:0/2695893585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e391d0a80 0x555e391ccce0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f7c38002230 tx=0x7f7c38010cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fab5991b65eb0e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.562+0000 7f7c257fa640 1 -- 172.21.15.12:0/2695893585 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c38024040 con 0x555e391d0a80 2024-04-22T13:31:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.562+0000 7f7c257fa640 1 -- 172.21.15.12:0/2695893585 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c38004c90 con 0x555e391d0a80 2024-04-22T13:31:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.562+0000 7f7c257fa640 1 -- 172.21.15.12:0/2695893585 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c38030c20 con 0x555e391d0a80 2024-04-22T13:31:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.563+0000 7f7c485de740 1 -- 172.21.15.12:0/2695893585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e391d0a80 msgr2=0x555e391ccce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.563+0000 7f7c485de740 1 --2- 172.21.15.12:0/2695893585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e391d0a80 0x555e391ccce0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f7c38002230 tx=0x7f7c38010cf0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.563+0000 7f7c485de740 1 -- 172.21.15.12:0/2695893585 shutdown_connections 2024-04-22T13:31:21.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.563+0000 7f7c485de740 1 --2- 172.21.15.12:0/2695893585 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e391d0a80 0x555e391ccce0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.563+0000 7f7c485de740 1 --2- 172.21.15.12:0/2695893585 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e391c7ea0 0x555e391dc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.563+0000 7f7c485de740 1 --2- 172.21.15.12:0/2695893585 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e391c7580 0x555e391d02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.563+0000 7f7c485de740 1 -- 172.21.15.12:0/2695893585 >> 172.21.15.12:0/2695893585 conn(0x555e391c1cb0 msgr2=0x555e391c3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.563+0000 7f7c485de740 1 -- 172.21.15.12:0/2695893585 shutdown_connections 2024-04-22T13:31:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.563+0000 7f7c485de740 1 -- 172.21.15.12:0/2695893585 wait complete. 2024-04-22T13:31:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.563+0000 7f7c485de740 1 -- start start 2024-04-22T13:31:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.563+0000 7f7c485de740 1 Processor -- start 2024-04-22T13:31:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.563+0000 7f7c485de740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e391c7580 0x555e391d8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.563+0000 7f7c485de740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e391c7ea0 0x555e391d4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.563+0000 7f7c485de740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e391d0a80 0x555e391d4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.563+0000 7f7c485de740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7c401b11f0 con 0x555e391c7ea0 2024-04-22T13:31:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.563+0000 7f7c485de740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7c401b0b20 con 0x555e391c7580 2024-04-22T13:31:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.563+0000 7f7c485de740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7c401add80 con 0x555e391d0a80 2024-04-22T13:31:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.564+0000 7f7c485de740 10 client.0 _ll_register_callbacks cb 0x7f7c38012b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:21.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.564+0000 7f7c3e7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e391c7ea0 0x555e391d4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:21.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.564+0000 7f7c3e7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e391c7ea0 0x555e391d4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34840/0 (socket says 172.21.15.12:34840) 2024-04-22T13:31:21.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.564+0000 7f7c3e7fc640 1 -- 172.21.15.12:0/2880623778 learned_addr learned my addr 172.21.15.12:0/2880623778 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:21.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.564+0000 7f7c3effd640 1 --2- 172.21.15.12:0/2880623778 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e391c7580 0x555e391d8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:21.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.564+0000 7f7c3f7fe640 1 --2- 172.21.15.12:0/2880623778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e391d0a80 0x555e391d4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:21.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.564+0000 7f7c3e7fc640 1 -- 172.21.15.12:0/2880623778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e391d0a80 msgr2=0x555e391d4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:21.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.564+0000 7f7c3e7fc640 1 --2- 172.21.15.12:0/2880623778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e391d0a80 0x555e391d4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.564+0000 7f7c3e7fc640 1 -- 172.21.15.12:0/2880623778 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e391c7580 msgr2=0x555e391d8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:21.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.564+0000 7f7c3e7fc640 1 --2- 172.21.15.12:0/2880623778 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e391c7580 0x555e391d8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.564+0000 7f7c3e7fc640 1 -- 172.21.15.12:0/2880623778 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e391d4cc0 con 0x555e391c7ea0 2024-04-22T13:31:21.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.564+0000 7f7c3e7fc640 1 --2- 172.21.15.12:0/2880623778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e391c7ea0 0x555e391d4140 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f7c40100630 tx=0x7f7c34012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a12fc85c0b7fa52 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:21.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.564+0000 7f7c1effd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:21.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.564+0000 7f7c1effd640 1 -- 172.21.15.12:0/2880623778 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c34005020 con 0x555e391c7ea0 2024-04-22T13:31:21.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.564+0000 7f7c1effd640 1 -- 172.21.15.12:0/2880623778 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f7c340051c0 con 0x555e391c7ea0 2024-04-22T13:31:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.564+0000 7f7c485de740 1 -- 172.21.15.12:0/2880623778 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c38004c90 con 0x555e391c7ea0 2024-04-22T13:31:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.564+0000 7f7c1effd640 1 -- 172.21.15.12:0/2880623778 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7c34029050 con 0x555e391c7ea0 2024-04-22T13:31:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.564+0000 7f7c485de740 10 client.18207 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.564+0000 7f7c485de740 1 -- 172.21.15.12:0/2880623778 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555e39246be0 con 0x555e391c7ea0 2024-04-22T13:31:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.565+0000 7f7c1effd640 1 -- 172.21.15.12:0/2880623778 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c38004c90 con 0x555e391c7ea0 2024-04-22T13:31:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.565+0000 7f7c1effd640 1 -- 172.21.15.12:0/2880623778 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7c34011650 con 0x555e391c7ea0 2024-04-22T13:31:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.565+0000 7f7c485de740 10 client.18207 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.565+0000 7f7c485de740 10 client.18207 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.565+0000 7f7c485de740 20 client.18207 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.565+0000 7f7c485de740 1 --2- 172.21.15.12:0/2880623778 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555e39247410 0x555e39249830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.565+0000 7f7c485de740 4 client.18207 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:31:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.565+0000 7f7c485de740 1 -- 172.21.15.12:0/2880623778 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555e390af590 con 0x555e39247410 2024-04-22T13:31:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.565+0000 7f7c3effd640 1 --2- 172.21.15.12:0/2880623778 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555e39247410 0x555e39249830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:21.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.565+0000 7f7c3effd640 1 --2- 172.21.15.12:0/2880623778 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555e39247410 0x555e39249830 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80a73d9ec4e23ad3 server_cookie=e194175bfcc68e1e in_seq=0 out_seq=0 2024-04-22T13:31:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.565+0000 7f7c1effd640 10 client.18207 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.566+0000 7f7c1effd640 1 -- 172.21.15.12:0/2880623778 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555e390af590 con 0x555e39247410 2024-04-22T13:31:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.566+0000 7f7c1effd640 10 client.18207 handle_command_reply: tid=0 2024-04-22T13:31:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.566+0000 7f7c485de740 1 client.18207 shutdown 2024-04-22T13:31:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.566+0000 7f7c485de740 1 -- 172.21.15.12:0/2880623778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e391c7ea0 msgr2=0x555e391d4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.566+0000 7f7c485de740 1 --2- 172.21.15.12:0/2880623778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e391c7ea0 0x555e391d4140 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f7c40100630 tx=0x7f7c34012820 comp rx=0 tx=0).stop 2024-04-22T13:31:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.566+0000 7f7c485de740 1 -- 172.21.15.12:0/2880623778 shutdown_connections 2024-04-22T13:31:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.566+0000 7f7c485de740 1 --2- 172.21.15.12:0/2880623778 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555e39247410 0x555e39249830 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.566+0000 7f7c485de740 1 --2- 172.21.15.12:0/2880623778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e391d0a80 0x555e391d4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.566+0000 7f7c485de740 1 --2- 172.21.15.12:0/2880623778 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e391c7ea0 0x555e391d4140 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.566+0000 7f7c485de740 1 --2- 172.21.15.12:0/2880623778 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e391c7580 0x555e391d8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.566+0000 7f7c1effd640 0 client.18207 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.566+0000 7f7c485de740 1 -- 172.21.15.12:0/2880623778 >> 172.21.15.12:0/2880623778 conn(0x555e391c1cb0 msgr2=0x555e391c3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:21.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.566+0000 7f7c485de740 1 -- 172.21.15.12:0/2880623778 shutdown_connections 2024-04-22T13:31:21.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.566+0000 7f7c485de740 1 -- 172.21.15.12:0/2880623778 wait complete. 2024-04-22T13:31:21.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.566+0000 7f7c485de740 20 client.18207 trim_cache size 0 max 16384 2024-04-22T13:31:21.569 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:31:21.569 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-04-22T13:31:21.569 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-04-22T13:31:21.569 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:31:21.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.567+0000 7f7c45915640 1 -- 172.21.15.12:0/390441994 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7c280a3d20 msgr2=0x7f7c280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:21.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.567+0000 7f7c45915640 1 --2- 172.21.15.12:0/390441994 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7c280a3d20 0x7f7c280a61e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7c38010c40 tx=0x7f7c3800c000 comp rx=0 tx=0).stop 2024-04-22T13:31:21.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.567+0000 7f7c45915640 1 -- 172.21.15.12:0/390441994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c401010a0 msgr2=0x7f7c400feba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:21.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.567+0000 7f7c45915640 1 --2- 172.21.15.12:0/390441994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c401010a0 0x7f7c400feba0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f7c30005940 tx=0x7f7c3002c7e0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.567+0000 7f7c45915640 1 -- 172.21.15.12:0/390441994 shutdown_connections 2024-04-22T13:31:21.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.567+0000 7f7c45915640 1 --2- 172.21.15.12:0/390441994 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7c280a3d20 0x7f7c280a61e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.567+0000 7f7c45915640 1 --2- 172.21.15.12:0/390441994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c40105dd0 0x7f7c400ff620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.567+0000 7f7c45915640 1 --2- 172.21.15.12:0/390441994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c40101a40 0x7f7c400ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.567+0000 7f7c45915640 1 --2- 172.21.15.12:0/390441994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7c401010a0 0x7f7c400feba0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.567+0000 7f7c45915640 1 -- 172.21.15.12:0/390441994 >> 172.21.15.12:0/390441994 conn(0x7f7c40077c40 msgr2=0x7f7c4006a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:21.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.567+0000 7f7c45915640 1 -- 172.21.15.12:0/390441994 shutdown_connections 2024-04-22T13:31:21.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.567+0000 7f7c45915640 1 -- 172.21.15.12:0/390441994 wait complete. 2024-04-22T13:31:21.573 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:31:21.573 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:31:21.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.680+0000 7f49bd76d640 1 -- 172.21.15.12:0/1887726710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49b8105e30 msgr2=0x7f49b8106210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:21.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.680+0000 7f49bd76d640 1 --2- 172.21.15.12:0/1887726710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49b8105e30 0x7f49b8106210 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f49a8009cd0 tx=0x7f49a802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.680+0000 7f49bd76d640 1 -- 172.21.15.12:0/1887726710 shutdown_connections 2024-04-22T13:31:21.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.680+0000 7f49bd76d640 1 --2- 172.21.15.12:0/1887726710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49b8101b60 0x7f49b810dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.680+0000 7f49bd76d640 1 --2- 172.21.15.12:0/1887726710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49b81011a0 0x7f49b8101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.680+0000 7f49bd76d640 1 --2- 172.21.15.12:0/1887726710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49b8105e30 0x7f49b8106210 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.680+0000 7f49bd76d640 1 -- 172.21.15.12:0/1887726710 >> 172.21.15.12:0/1887726710 conn(0x7f49b8077e30 msgr2=0x7f49b8078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:21.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.681+0000 7f49bd76d640 1 -- 172.21.15.12:0/1887726710 shutdown_connections 2024-04-22T13:31:21.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.681+0000 7f49bd76d640 1 -- 172.21.15.12:0/1887726710 wait complete. 2024-04-22T13:31:21.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.681+0000 7f49bd76d640 1 Processor -- start 2024-04-22T13:31:21.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.681+0000 7f49bd76d640 1 -- start start 2024-04-22T13:31:21.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.681+0000 7f49bd76d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49b81011a0 0x7f49b80fecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:21.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.681+0000 7f49bd76d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49b8101b60 0x7f49b80ff200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:21.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.681+0000 7f49bd76d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49b8105e30 0x7f49b80ff740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:21.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.681+0000 7f49bd76d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f49b81110c0 con 0x7f49b8101b60 2024-04-22T13:31:21.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.681+0000 7f49bd76d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f49b8110f40 con 0x7f49b81011a0 2024-04-22T13:31:21.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.682+0000 7f49bd76d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f49b8111240 con 0x7f49b8105e30 2024-04-22T13:31:21.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.682+0000 7f49b67fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49b8101b60 0x7f49b80ff200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.682+0000 7f49b67fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49b8101b60 0x7f49b80ff200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44522/0 (socket says 172.21.15.12:44522) 2024-04-22T13:31:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.682+0000 7f49b67fc640 1 -- 172.21.15.12:0/1318687060 learned_addr learned my addr 172.21.15.12:0/1318687060 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.682+0000 7f49b6ffd640 1 --2- 172.21.15.12:0/1318687060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49b81011a0 0x7f49b80fecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.682+0000 7f49b77fe640 1 --2- 172.21.15.12:0/1318687060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49b8105e30 0x7f49b80ff740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.682+0000 7f49b67fc640 1 -- 172.21.15.12:0/1318687060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49b8105e30 msgr2=0x7f49b80ff740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.682+0000 7f49b67fc640 1 --2- 172.21.15.12:0/1318687060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49b8105e30 0x7f49b80ff740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.682+0000 7f49b67fc640 1 -- 172.21.15.12:0/1318687060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49b81011a0 msgr2=0x7f49b80fecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.682+0000 7f49b67fc640 1 --2- 172.21.15.12:0/1318687060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49b81011a0 0x7f49b80fecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.682+0000 7f49b67fc640 1 -- 172.21.15.12:0/1318687060 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b81a6c10 con 0x7f49b8101b60 2024-04-22T13:31:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.682+0000 7f49b67fc640 1 --2- 172.21.15.12:0/1318687060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49b8101b60 0x7f49b80ff200 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00e9f0 tx=0x7f49ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.682+0000 7f499ffff640 1 -- 172.21.15.12:0/1318687060 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac00cd80 con 0x7f49b8101b60 2024-04-22T13:31:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.682+0000 7f49bd76d640 1 -- 172.21.15.12:0/1318687060 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49b81a6f00 con 0x7f49b8101b60 2024-04-22T13:31:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.682+0000 7f499ffff640 1 -- 172.21.15.12:0/1318687060 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49ac0108e0 con 0x7f49b8101b60 2024-04-22T13:31:21.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.682+0000 7f499ffff640 1 -- 172.21.15.12:0/1318687060 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac018980 con 0x7f49b8101b60 2024-04-22T13:31:21.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.682+0000 7f49bd76d640 1 -- 172.21.15.12:0/1318687060 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49b81a7410 con 0x7f49b8101b60 2024-04-22T13:31:21.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.683+0000 7f49bd76d640 1 -- 172.21.15.12:0/1318687060 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49b8068de0 con 0x7f49b8101b60 2024-04-22T13:31:21.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.684+0000 7f499ffff640 1 -- 172.21.15.12:0/1318687060 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f49ac010430 con 0x7f49b8101b60 2024-04-22T13:31:21.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.685+0000 7f499ffff640 1 --2- 172.21.15.12:0/1318687060 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f49a00a3d50 0x7f49a00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:21.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.685+0000 7f49b6ffd640 1 --2- 172.21.15.12:0/1318687060 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f49a00a3d50 0x7f49a00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:21.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.685+0000 7f499ffff640 1 -- 172.21.15.12:0/1318687060 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f49ac0d0bb0 con 0x7f49b8101b60 2024-04-22T13:31:21.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.686+0000 7f49b6ffd640 1 --2- 172.21.15.12:0/1318687060 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f49a00a3d50 0x7f49a00a6210 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f49a8006010 tx=0x7f49a8005c30 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:21.685 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.690+0000 7f499ffff640 1 -- 172.21.15.12:0/1318687060 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f49ac09d270 con 0x7f49b8101b60 2024-04-22T13:31:21.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:21 smithi012 ceph-mon[28265]: pgmap v222: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s 2024-04-22T13:31:21.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.918+0000 7f49bd76d640 1 -- 172.21.15.12:0/1318687060 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49b810a340 con 0x7f49b8101b60 2024-04-22T13:31:21.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.919+0000 7f499ffff640 1 -- 172.21.15.12:0/1318687060 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7f49ac09cb70 con 0x7f49b8101b60 2024-04-22T13:31:21.915 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:31:21.915 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:31:21.915 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:31:21.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.921+0000 7f49bd76d640 1 -- 172.21.15.12:0/1318687060 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f49a00a3d50 msgr2=0x7f49a00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:21.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.921+0000 7f49bd76d640 1 --2- 172.21.15.12:0/1318687060 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f49a00a3d50 0x7f49a00a6210 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f49a8006010 tx=0x7f49a8005c30 comp rx=0 tx=0).stop 2024-04-22T13:31:21.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.921+0000 7f49bd76d640 1 -- 172.21.15.12:0/1318687060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49b8101b60 msgr2=0x7f49b80ff200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:21.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.921+0000 7f49bd76d640 1 --2- 172.21.15.12:0/1318687060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49b8101b60 0x7f49b80ff200 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00e9f0 tx=0x7f49ac00eeb0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.921+0000 7f49bd76d640 1 -- 172.21.15.12:0/1318687060 shutdown_connections 2024-04-22T13:31:21.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.921+0000 7f49bd76d640 1 --2- 172.21.15.12:0/1318687060 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f49a00a3d50 0x7f49a00a6210 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.921+0000 7f49bd76d640 1 --2- 172.21.15.12:0/1318687060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49b8105e30 0x7f49b80ff740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.921+0000 7f49bd76d640 1 --2- 172.21.15.12:0/1318687060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49b8101b60 0x7f49b80ff200 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.921+0000 7f49bd76d640 1 --2- 172.21.15.12:0/1318687060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f49b81011a0 0x7f49b80fecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:21.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.921+0000 7f49bd76d640 1 -- 172.21.15.12:0/1318687060 >> 172.21.15.12:0/1318687060 conn(0x7f49b8077e30 msgr2=0x7f49b80694c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:21.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.921+0000 7f49bd76d640 1 -- 172.21.15.12:0/1318687060 shutdown_connections 2024-04-22T13:31:21.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:21.921+0000 7f49bd76d640 1 -- 172.21.15.12:0/1318687060 wait complete. 2024-04-22T13:31:21.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:21 smithi134 ceph-mon[31338]: pgmap v222: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s 2024-04-22T13:31:21.928 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:31:21.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:21 smithi111 ceph-mon[31361]: pgmap v222: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s 2024-04-22T13:31:22.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.037+0000 7f0b68167640 1 -- 172.21.15.12:0/3667778118 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b60101a50 msgr2=0x7f0b60105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.037+0000 7f0b68167640 1 --2- 172.21.15.12:0/3667778118 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b60101a50 0x7f0b60105a90 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f0b50009900 tx=0x7f0b5002bc30 comp rx=0 tx=0).stop 2024-04-22T13:31:22.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.037+0000 7f0b68167640 1 -- 172.21.15.12:0/3667778118 shutdown_connections 2024-04-22T13:31:22.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.037+0000 7f0b68167640 1 --2- 172.21.15.12:0/3667778118 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b60106170 0x7f0b6010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.037+0000 7f0b68167640 1 --2- 172.21.15.12:0/3667778118 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b60101a50 0x7f0b60105a90 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.037+0000 7f0b68167640 1 --2- 172.21.15.12:0/3667778118 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b601010a0 0x7f0b60101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.037+0000 7f0b68167640 1 -- 172.21.15.12:0/3667778118 >> 172.21.15.12:0/3667778118 conn(0x7f0b600771d0 msgr2=0x7f0b600775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:22.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.037+0000 7f0b68167640 1 -- 172.21.15.12:0/3667778118 shutdown_connections 2024-04-22T13:31:22.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.037+0000 7f0b68167640 1 -- 172.21.15.12:0/3667778118 wait complete. 2024-04-22T13:31:22.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.038+0000 7f0b68167640 1 Processor -- start 2024-04-22T13:31:22.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.038+0000 7f0b68167640 1 -- start start 2024-04-22T13:31:22.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.038+0000 7f0b68167640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b601010a0 0x7f0b601995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.038+0000 7f0b68167640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b60106170 0x7f0b60199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.038+0000 7f0b68167640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b6019a080 0x7f0b601a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.038+0000 7f0b68167640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0b601110b0 con 0x7f0b601010a0 2024-04-22T13:31:22.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.038+0000 7f0b68167640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0b60110f30 con 0x7f0b6019a080 2024-04-22T13:31:22.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.038+0000 7f0b68167640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b60111230 con 0x7f0b60106170 2024-04-22T13:31:22.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.038+0000 7f0b65f03640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b601010a0 0x7f0b601995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.038+0000 7f0b65f03640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b601010a0 0x7f0b601995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44540/0 (socket says 172.21.15.12:44540) 2024-04-22T13:31:22.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.038+0000 7f0b65f03640 1 -- 172.21.15.12:0/3224655958 learned_addr learned my addr 172.21.15.12:0/3224655958 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:22.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.038+0000 7f0b65702640 1 --2- 172.21.15.12:0/3224655958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b60106170 0x7f0b60199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.038+0000 7f0b66704640 1 --2- 172.21.15.12:0/3224655958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b6019a080 0x7f0b601a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.038+0000 7f0b65f03640 1 -- 172.21.15.12:0/3224655958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b60106170 msgr2=0x7f0b60199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.038+0000 7f0b65f03640 1 --2- 172.21.15.12:0/3224655958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b60106170 0x7f0b60199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.038+0000 7f0b65f03640 1 -- 172.21.15.12:0/3224655958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b6019a080 msgr2=0x7f0b601a4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.038+0000 7f0b65f03640 1 --2- 172.21.15.12:0/3224655958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b6019a080 0x7f0b601a4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.037 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.038+0000 7f0b65f03640 1 -- 172.21.15.12:0/3224655958 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b601a4e00 con 0x7f0b601010a0 2024-04-22T13:31:22.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.038+0000 7f0b65702640 1 --2- 172.21.15.12:0/3224655958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b60106170 0x7f0b60199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:22.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.039+0000 7f0b65f03640 1 --2- 172.21.15.12:0/3224655958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b601010a0 0x7f0b601995b0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00e9f0 tx=0x7f0b5c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:22.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.039+0000 7f0b56ffd640 1 -- 172.21.15.12:0/3224655958 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c00cd80 con 0x7f0b601010a0 2024-04-22T13:31:22.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.039+0000 7f0b68167640 1 -- 172.21.15.12:0/3224655958 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b601a50f0 con 0x7f0b601010a0 2024-04-22T13:31:22.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.039+0000 7f0b56ffd640 1 -- 172.21.15.12:0/3224655958 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b5c0108e0 con 0x7f0b601010a0 2024-04-22T13:31:22.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.039+0000 7f0b56ffd640 1 -- 172.21.15.12:0/3224655958 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c018980 con 0x7f0b601010a0 2024-04-22T13:31:22.038 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.039+0000 7f0b68167640 1 -- 172.21.15.12:0/3224655958 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b601a5660 con 0x7f0b601010a0 2024-04-22T13:31:22.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.041+0000 7f0b56ffd640 1 -- 172.21.15.12:0/3224655958 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0b5c010430 con 0x7f0b601010a0 2024-04-22T13:31:22.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.041+0000 7f0b6ae38740 1 Processor -- start 2024-04-22T13:31:22.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.041+0000 7f0b6ae38740 1 -- start start 2024-04-22T13:31:22.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.041+0000 7f0b6ae38740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adeff13b0 0x557adeff1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.042+0000 7f0b6ae38740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557adeff1d60 0x557adeffa890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.042+0000 7f0b6ae38740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557adeffae60 0x557adeffb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.042+0000 7f0b6ae38740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557adef561f0 con 0x557adeff1d60 2024-04-22T13:31:22.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.042+0000 7f0b6ae38740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557adeef34b0 con 0x557adeffae60 2024-04-22T13:31:22.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.042+0000 7f0b6ae38740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557adef521b0 con 0x557adeff13b0 2024-04-22T13:31:22.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.042+0000 7f0b65f03640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557adeff1d60 0x557adeffa890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.042+0000 7f0b65f03640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557adeff1d60 0x557adeffa890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44556/0 (socket says 172.21.15.12:44556) 2024-04-22T13:31:22.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.042+0000 7f0b65f03640 1 -- 172.21.15.12:0/2153889224 learned_addr learned my addr 172.21.15.12:0/2153889224 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:22.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.042+0000 7f0b56ffd640 1 --2- 172.21.15.12:0/3224655958 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f0b4c0a3d20 0x7f0b4c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.042+0000 7f0b65702640 1 --2- 172.21.15.12:0/2153889224 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557adeffae60 0x557adeffb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.042+0000 7f0b66704640 1 --2- 172.21.15.12:0/3224655958 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f0b4c0a3d20 0x7f0b4c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.042+0000 7f0b66704640 1 --2- 172.21.15.12:0/2153889224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adeff13b0 0x557adeff1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.042+0000 7f0b56ffd640 1 -- 172.21.15.12:0/3224655958 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f0b5c0d0c20 con 0x7f0b601010a0 2024-04-22T13:31:22.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.042+0000 7f0b65f03640 1 -- 172.21.15.12:0/2153889224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adeff13b0 msgr2=0x557adeff1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.042+0000 7f0b65f03640 1 --2- 172.21.15.12:0/2153889224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adeff13b0 0x557adeff1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.042+0000 7f0b65f03640 1 -- 172.21.15.12:0/2153889224 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557adeffae60 msgr2=0x557adeffb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.042+0000 7f0b65f03640 1 --2- 172.21.15.12:0/2153889224 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557adeffae60 0x557adeffb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.042+0000 7f0b65f03640 1 -- 172.21.15.12:0/2153889224 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557adeee51b0 con 0x557adeff1d60 2024-04-22T13:31:22.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.042+0000 7f0b65702640 1 --2- 172.21.15.12:0/2153889224 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557adeffae60 0x557adeffb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:22.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.042+0000 7f0b65f03640 1 --2- 172.21.15.12:0/2153889224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557adeff1d60 0x557adeffa890 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c09cb70 tx=0x7f0b5c018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bcaf62404a14938 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:22.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.042+0000 7f0b66704640 1 --2- 172.21.15.12:0/3224655958 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f0b4c0a3d20 0x7f0b4c0a61e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0b58009fd0 tx=0x7f0b58009500 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:22.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.042+0000 7f0b4bfff640 1 -- 172.21.15.12:0/2153889224 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c09cbe0 con 0x557adeff1d60 2024-04-22T13:31:22.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.043+0000 7f0b4bfff640 1 -- 172.21.15.12:0/2153889224 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b5c0a0f90 con 0x557adeff1d60 2024-04-22T13:31:22.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.043+0000 7f0b4bfff640 1 -- 172.21.15.12:0/2153889224 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c099250 con 0x557adeff1d60 2024-04-22T13:31:22.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.043+0000 7f0b6ae38740 1 -- 172.21.15.12:0/2153889224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557adeff1d60 msgr2=0x557adeffa890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.043+0000 7f0b6ae38740 1 --2- 172.21.15.12:0/2153889224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557adeff1d60 0x557adeffa890 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c09cb70 tx=0x7f0b5c018420 comp rx=0 tx=0).stop 2024-04-22T13:31:22.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.043+0000 7f0b6ae38740 1 -- 172.21.15.12:0/2153889224 shutdown_connections 2024-04-22T13:31:22.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.043+0000 7f0b6ae38740 1 --2- 172.21.15.12:0/2153889224 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557adeffae60 0x557adeffb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.048 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.043+0000 7f0b6ae38740 1 --2- 172.21.15.12:0/2153889224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557adeff1d60 0x557adeffa890 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.043+0000 7f0b6ae38740 1 --2- 172.21.15.12:0/2153889224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adeff13b0 0x557adeff1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.043+0000 7f0b6ae38740 1 -- 172.21.15.12:0/2153889224 >> 172.21.15.12:0/2153889224 conn(0x557adefebba0 msgr2=0x557adefebf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:22.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.043+0000 7f0b6ae38740 1 -- 172.21.15.12:0/2153889224 shutdown_connections 2024-04-22T13:31:22.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.043+0000 7f0b6ae38740 1 -- 172.21.15.12:0/2153889224 wait complete. 2024-04-22T13:31:22.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.044+0000 7f0b6ae38740 1 -- start start 2024-04-22T13:31:22.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.044+0000 7f0b6ae38740 1 Processor -- start 2024-04-22T13:31:22.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.044+0000 7f0b6ae38740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557adeff13b0 0x557adf001310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.044+0000 7f0b6ae38740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557adeff1d60 0x557adeffe6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.044+0000 7f0b6ae38740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adeffae60 0x557adeffed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.044+0000 7f0b6ae38740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557adf001ce0 con 0x557adeff13b0 2024-04-22T13:31:22.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.044+0000 7f0b6ae38740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557adef0f500 con 0x557adeff1d60 2024-04-22T13:31:22.049 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.044+0000 7f0b6ae38740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557adf001e60 con 0x557adeffae60 2024-04-22T13:31:22.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.044+0000 7f0b6ae38740 10 client.0 _ll_register_callbacks cb 0x557adeeddb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:22.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.044+0000 7f0b65f03640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557adeff13b0 0x557adf001310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.044+0000 7f0b65f03640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557adeff13b0 0x557adf001310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44572/0 (socket says 172.21.15.12:44572) 2024-04-22T13:31:22.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.044+0000 7f0b65f03640 1 -- 172.21.15.12:0/442227224 learned_addr learned my addr 172.21.15.12:0/442227224 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:22.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.044+0000 7f0b65702640 1 --2- 172.21.15.12:0/442227224 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557adeff1d60 0x557adeffe6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.045+0000 7f0b66704640 1 --2- 172.21.15.12:0/442227224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adeffae60 0x557adeffed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.045+0000 7f0b65f03640 1 -- 172.21.15.12:0/442227224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adeffae60 msgr2=0x557adeffed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.045+0000 7f0b65f03640 1 --2- 172.21.15.12:0/442227224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adeffae60 0x557adeffed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.045+0000 7f0b65f03640 1 -- 172.21.15.12:0/442227224 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557adeff1d60 msgr2=0x557adeffe6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.045+0000 7f0b65f03640 1 --2- 172.21.15.12:0/442227224 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557adeff1d60 0x557adeffe6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.045+0000 7f0b65f03640 1 -- 172.21.15.12:0/442227224 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557adec683f0 con 0x557adeff13b0 2024-04-22T13:31:22.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.045+0000 7f0b65702640 1 --2- 172.21.15.12:0/442227224 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557adeff1d60 0x557adeffe6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:22.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.045+0000 7f0b65f03640 1 --2- 172.21.15.12:0/442227224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557adeff13b0 0x557adf001310 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0995b0 tx=0x7f0b5c0d4480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eae42a90a7e6fcf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:22.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.045+0000 7f0b497fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:22.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.045+0000 7f0b497fa640 1 -- 172.21.15.12:0/442227224 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c0d3070 con 0x557adeff13b0 2024-04-22T13:31:22.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.045+0000 7f0b497fa640 1 -- 172.21.15.12:0/442227224 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f0b5c0bd070 con 0x557adeff13b0 2024-04-22T13:31:22.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.045+0000 7f0b6ae38740 1 -- 172.21.15.12:0/442227224 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b5c0a0f90 con 0x557adeff13b0 2024-04-22T13:31:22.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.045+0000 7f0b497fa640 1 -- 172.21.15.12:0/442227224 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0b5c0b6120 con 0x557adeff13b0 2024-04-22T13:31:22.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.045+0000 7f0b6ae38740 10 client.18225 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:22.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.045+0000 7f0b6ae38740 1 -- 172.21.15.12:0/442227224 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557aded327d0 con 0x557adeff13b0 2024-04-22T13:31:22.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.045+0000 7f0b497fa640 1 -- 172.21.15.12:0/442227224 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b5c0a0f90 con 0x557adeff13b0 2024-04-22T13:31:22.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.045+0000 7f0b497fa640 1 -- 172.21.15.12:0/442227224 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0b5c0d6070 con 0x557adeff13b0 2024-04-22T13:31:22.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.045+0000 7f0b6ae38740 10 client.18225 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:22.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.046+0000 7f0b6ae38740 10 client.18225 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:31:22.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.046+0000 7f0b6ae38740 20 client.18225 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:22.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.046+0000 7f0b6ae38740 1 --2- 172.21.15.12:0/442227224 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x557adf00ee70 0x557adf011260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.046+0000 7f0b6ae38740 4 client.18225 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:22.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.046+0000 7f0b6ae38740 1 -- 172.21.15.12:0/442227224 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557adeef5930 con 0x557adf00ee70 2024-04-22T13:31:22.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.046+0000 7f0b65702640 1 --2- 172.21.15.12:0/442227224 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x557adf00ee70 0x557adf011260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.046+0000 7f0b65702640 1 --2- 172.21.15.12:0/442227224 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x557adf00ee70 0x557adf011260 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=273879953c8c164 server_cookie=3102e48aa5cbf40 in_seq=0 out_seq=0 2024-04-22T13:31:22.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.046+0000 7f0b497fa640 10 client.18225 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:31:22.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.047+0000 7f0b497fa640 1 -- 172.21.15.12:0/442227224 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557adeef5930 con 0x557adf00ee70 2024-04-22T13:31:22.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.047+0000 7f0b497fa640 10 client.18225 handle_command_reply: tid=0 2024-04-22T13:31:22.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.047+0000 7f0b6ae38740 1 client.18225 shutdown 2024-04-22T13:31:22.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.047+0000 7f0b6ae38740 1 -- 172.21.15.12:0/442227224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557adeff13b0 msgr2=0x557adf001310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.047+0000 7f0b6ae38740 1 --2- 172.21.15.12:0/442227224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557adeff13b0 0x557adf001310 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0995b0 tx=0x7f0b5c0d4480 comp rx=0 tx=0).stop 2024-04-22T13:31:22.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.047+0000 7f0b6ae38740 1 -- 172.21.15.12:0/442227224 shutdown_connections 2024-04-22T13:31:22.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.047+0000 7f0b6ae38740 1 --2- 172.21.15.12:0/442227224 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x557adf00ee70 0x557adf011260 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.047+0000 7f0b6ae38740 1 --2- 172.21.15.12:0/442227224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adeffae60 0x557adeffed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.047+0000 7f0b497fa640 0 client.18225 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:31:22.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.047+0000 7f0b6ae38740 1 --2- 172.21.15.12:0/442227224 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557adeff1d60 0x557adeffe6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.047+0000 7f0b6ae38740 1 --2- 172.21.15.12:0/442227224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557adeff13b0 0x557adf001310 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.047+0000 7f0b6ae38740 1 -- 172.21.15.12:0/442227224 >> 172.21.15.12:0/442227224 conn(0x557adefebba0 msgr2=0x557adefed8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:22.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.047+0000 7f0b6ae38740 1 -- 172.21.15.12:0/442227224 shutdown_connections 2024-04-22T13:31:22.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.047+0000 7f0b6ae38740 1 -- 172.21.15.12:0/442227224 wait complete. 2024-04-22T13:31:22.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.047+0000 7f0b6ae38740 20 client.18225 trim_cache size 0 max 16384 2024-04-22T13:31:22.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.061+0000 7f0b6ae38740 1 Processor -- start 2024-04-22T13:31:22.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.061+0000 7f0b6ae38740 1 -- start start 2024-04-22T13:31:22.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.062+0000 7f0b6ae38740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557adeff13b0 0x557adeffc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.062+0000 7f0b6ae38740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adeff1d60 0x557adf000dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.062+0000 7f0b6ae38740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557adeffae60 0x557adf001310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.062+0000 7f0b6ae38740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0b601ab9c0 con 0x557adeffae60 2024-04-22T13:31:22.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.062+0000 7f0b6ae38740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0b601ad500 con 0x557adeff13b0 2024-04-22T13:31:22.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.062+0000 7f0b6ae38740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b601ac090 con 0x557adeff1d60 2024-04-22T13:31:22.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.062+0000 7f0b66704640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557adeffae60 0x557adf001310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.062+0000 7f0b66704640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557adeffae60 0x557adf001310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44588/0 (socket says 172.21.15.12:44588) 2024-04-22T13:31:22.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.062+0000 7f0b66704640 1 -- 172.21.15.12:0/2782942357 learned_addr learned my addr 172.21.15.12:0/2782942357 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:22.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.062+0000 7f0b65702640 1 --2- 172.21.15.12:0/2782942357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adeff1d60 0x557adf000dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.062+0000 7f0b65f03640 1 --2- 172.21.15.12:0/2782942357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557adeff13b0 0x557adeffc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.062+0000 7f0b66704640 1 -- 172.21.15.12:0/2782942357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adeff1d60 msgr2=0x557adf000dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.062+0000 7f0b66704640 1 --2- 172.21.15.12:0/2782942357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adeff1d60 0x557adf000dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.062+0000 7f0b66704640 1 -- 172.21.15.12:0/2782942357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557adeff13b0 msgr2=0x557adeffc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.062+0000 7f0b66704640 1 --2- 172.21.15.12:0/2782942357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557adeff13b0 0x557adeffc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.062+0000 7f0b66704640 1 -- 172.21.15.12:0/2782942357 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557adf001c80 con 0x557adeffae60 2024-04-22T13:31:22.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.062+0000 7f0b65f03640 1 --2- 172.21.15.12:0/2782942357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557adeff13b0 0x557adeffc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:22.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.062+0000 7f0b66704640 1 --2- 172.21.15.12:0/2782942357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557adeffae60 0x557adf001310 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f0b58012600 tx=0x7f0b58005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02e7c33d7019701 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:22.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.062+0000 7f0b4b7fe640 1 -- 172.21.15.12:0/2782942357 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b58010cf0 con 0x557adeffae60 2024-04-22T13:31:22.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.063+0000 7f0b4b7fe640 1 -- 172.21.15.12:0/2782942357 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b58010e90 con 0x557adeffae60 2024-04-22T13:31:22.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.063+0000 7f0b4b7fe640 1 -- 172.21.15.12:0/2782942357 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b58028bb0 con 0x557adeffae60 2024-04-22T13:31:22.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.063+0000 7f0b6ae38740 1 -- 172.21.15.12:0/2782942357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557adeffae60 msgr2=0x557adf001310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.063+0000 7f0b6ae38740 1 --2- 172.21.15.12:0/2782942357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557adeffae60 0x557adf001310 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f0b58012600 tx=0x7f0b58005bd0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.063+0000 7f0b6ae38740 1 -- 172.21.15.12:0/2782942357 shutdown_connections 2024-04-22T13:31:22.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.063+0000 7f0b6ae38740 1 --2- 172.21.15.12:0/2782942357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557adeffae60 0x557adf001310 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.063+0000 7f0b6ae38740 1 --2- 172.21.15.12:0/2782942357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adeff1d60 0x557adf000dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.063+0000 7f0b6ae38740 1 --2- 172.21.15.12:0/2782942357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557adeff13b0 0x557adeffc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.063+0000 7f0b6ae38740 1 -- 172.21.15.12:0/2782942357 >> 172.21.15.12:0/2782942357 conn(0x557adefebba0 msgr2=0x557adefeac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:22.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.063+0000 7f0b6ae38740 1 -- 172.21.15.12:0/2782942357 shutdown_connections 2024-04-22T13:31:22.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.063+0000 7f0b6ae38740 1 -- 172.21.15.12:0/2782942357 wait complete. 2024-04-22T13:31:22.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.064+0000 7f0b6ae38740 1 -- start start 2024-04-22T13:31:22.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.064+0000 7f0b6ae38740 1 Processor -- start 2024-04-22T13:31:22.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.064+0000 7f0b6ae38740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557adeff13b0 0x557adf011e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.064+0000 7f0b6ae38740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557adeff1d60 0x557adf0123a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.064+0000 7f0b6ae38740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adeffae60 0x557adf0128e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.064+0000 7f0b6ae38740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0b601ace30 con 0x557adeff13b0 2024-04-22T13:31:22.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.064+0000 7f0b6ae38740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0b601ac760 con 0x557adeff1d60 2024-04-22T13:31:22.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.064+0000 7f0b6ae38740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b601a99c0 con 0x557adeffae60 2024-04-22T13:31:22.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.064+0000 7f0b6ae38740 10 client.0 _ll_register_callbacks cb 0x7f0b5c0ce300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:22.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.064+0000 7f0b65f03640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557adeff13b0 0x557adf011e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.064+0000 7f0b65f03640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557adeff13b0 0x557adf011e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44602/0 (socket says 172.21.15.12:44602) 2024-04-22T13:31:22.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.064+0000 7f0b65f03640 1 -- 172.21.15.12:0/95292910 learned_addr learned my addr 172.21.15.12:0/95292910 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:22.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.064+0000 7f0b65702640 1 --2- 172.21.15.12:0/95292910 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557adeff1d60 0x557adf0123a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.064+0000 7f0b66704640 1 --2- 172.21.15.12:0/95292910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adeffae60 0x557adf0128e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.064+0000 7f0b65f03640 1 -- 172.21.15.12:0/95292910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adeffae60 msgr2=0x557adf0128e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.064+0000 7f0b65f03640 1 --2- 172.21.15.12:0/95292910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adeffae60 0x557adf0128e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.064+0000 7f0b65f03640 1 -- 172.21.15.12:0/95292910 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557adeff1d60 msgr2=0x557adf0123a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.064+0000 7f0b65f03640 1 --2- 172.21.15.12:0/95292910 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557adeff1d60 0x557adf0123a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.065+0000 7f0b65f03640 1 -- 172.21.15.12:0/95292910 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557adf074220 con 0x557adeff13b0 2024-04-22T13:31:22.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.065+0000 7f0b65f03640 1 --2- 172.21.15.12:0/95292910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557adeff13b0 0x557adf011e60 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c09cb70 tx=0x7f0b5c0bd2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb84191b4d482a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:22.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.065+0000 7f0b497fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:22.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.065+0000 7f0b497fa640 1 -- 172.21.15.12:0/95292910 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c0d3040 con 0x557adeff13b0 2024-04-22T13:31:22.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.065+0000 7f0b497fa640 1 -- 172.21.15.12:0/95292910 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f0b5c09c240 con 0x557adeff13b0 2024-04-22T13:31:22.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.065+0000 7f0b6ae38740 1 -- 172.21.15.12:0/95292910 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b58010e90 con 0x557adeff13b0 2024-04-22T13:31:22.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.065+0000 7f0b497fa640 1 -- 172.21.15.12:0/95292910 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0b5c098e80 con 0x557adeff13b0 2024-04-22T13:31:22.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.065+0000 7f0b6ae38740 10 client.18231 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:22.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.065+0000 7f0b6ae38740 1 -- 172.21.15.12:0/95292910 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557adf0754e0 con 0x557adeff13b0 2024-04-22T13:31:22.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.065+0000 7f0b497fa640 1 -- 172.21.15.12:0/95292910 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b58010e90 con 0x557adeff13b0 2024-04-22T13:31:22.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.065+0000 7f0b497fa640 1 -- 172.21.15.12:0/95292910 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0b5c0d6040 con 0x557adeff13b0 2024-04-22T13:31:22.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.065+0000 7f0b6ae38740 10 client.18231 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:22.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.065+0000 7f0b6ae38740 10 client.18231 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:31:22.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.065+0000 7f0b6ae38740 20 client.18231 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:22.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.065+0000 7f0b6ae38740 1 --2- 172.21.15.12:0/95292910 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x557adf075ac0 0x557adf077eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.065+0000 7f0b6ae38740 4 client.18231 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:31:22.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.065+0000 7f0b6ae38740 1 -- 172.21.15.12:0/95292910 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557adef63c10 con 0x557adf075ac0 2024-04-22T13:31:22.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.066+0000 7f0b65702640 1 --2- 172.21.15.12:0/95292910 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x557adf075ac0 0x557adf077eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.066+0000 7f0b497fa640 10 client.18231 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:31:22.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.066+0000 7f0b65702640 1 --2- 172.21.15.12:0/95292910 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x557adf075ac0 0x557adf077eb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a38072fd7dfbaaae server_cookie=704fcfd94e701cd2 in_seq=0 out_seq=0 2024-04-22T13:31:22.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.066+0000 7f0b497fa640 1 -- 172.21.15.12:0/95292910 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557adef63c10 con 0x557adf075ac0 2024-04-22T13:31:22.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.066+0000 7f0b497fa640 10 client.18231 handle_command_reply: tid=0 2024-04-22T13:31:22.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.066+0000 7f0b6ae38740 1 client.18231 shutdown 2024-04-22T13:31:22.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.066+0000 7f0b6ae38740 1 -- 172.21.15.12:0/95292910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557adeff13b0 msgr2=0x557adf011e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.066+0000 7f0b6ae38740 1 --2- 172.21.15.12:0/95292910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557adeff13b0 0x557adf011e60 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c09cb70 tx=0x7f0b5c0bd2e0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.066+0000 7f0b6ae38740 1 -- 172.21.15.12:0/95292910 shutdown_connections 2024-04-22T13:31:22.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.066+0000 7f0b6ae38740 1 --2- 172.21.15.12:0/95292910 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x557adf075ac0 0x557adf077eb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.066+0000 7f0b6ae38740 1 --2- 172.21.15.12:0/95292910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557adeffae60 0x557adf0128e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.066+0000 7f0b6ae38740 1 --2- 172.21.15.12:0/95292910 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557adeff1d60 0x557adf0123a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.066+0000 7f0b6ae38740 1 --2- 172.21.15.12:0/95292910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557adeff13b0 0x557adf011e60 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.066+0000 7f0b497fa640 0 client.18231 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:31:22.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.066+0000 7f0b6ae38740 1 -- 172.21.15.12:0/95292910 >> 172.21.15.12:0/95292910 conn(0x557adefebba0 msgr2=0x557adefeac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:22.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.067+0000 7f0b6ae38740 1 -- 172.21.15.12:0/95292910 shutdown_connections 2024-04-22T13:31:22.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.067+0000 7f0b6ae38740 1 -- 172.21.15.12:0/95292910 wait complete. 2024-04-22T13:31:22.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.067+0000 7f0b6ae38740 20 client.18231 trim_cache size 0 max 16384 2024-04-22T13:31:22.071 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:31:22.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.067+0000 7f0b68167640 1 -- 172.21.15.12:0/3224655958 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f0b4c0a3d20 msgr2=0x7f0b4c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.067+0000 7f0b68167640 1 --2- 172.21.15.12:0/3224655958 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f0b4c0a3d20 0x7f0b4c0a61e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0b58009fd0 tx=0x7f0b58009500 comp rx=0 tx=0).stop 2024-04-22T13:31:22.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.067+0000 7f0b68167640 1 -- 172.21.15.12:0/3224655958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b601010a0 msgr2=0x7f0b601995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.067+0000 7f0b68167640 1 --2- 172.21.15.12:0/3224655958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b601010a0 0x7f0b601995b0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00e9f0 tx=0x7f0b5c00eeb0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.067+0000 7f0b68167640 1 -- 172.21.15.12:0/3224655958 shutdown_connections 2024-04-22T13:31:22.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.067+0000 7f0b68167640 1 --2- 172.21.15.12:0/3224655958 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f0b4c0a3d20 0x7f0b4c0a61e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.067+0000 7f0b68167640 1 --2- 172.21.15.12:0/3224655958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b6019a080 0x7f0b601a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.067+0000 7f0b68167640 1 --2- 172.21.15.12:0/3224655958 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b60106170 0x7f0b60199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.067+0000 7f0b68167640 1 --2- 172.21.15.12:0/3224655958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b601010a0 0x7f0b601995b0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.067+0000 7f0b68167640 1 -- 172.21.15.12:0/3224655958 >> 172.21.15.12:0/3224655958 conn(0x7f0b600771d0 msgr2=0x7f0b600feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:22.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.067+0000 7f0b68167640 1 -- 172.21.15.12:0/3224655958 shutdown_connections 2024-04-22T13:31:22.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.067+0000 7f0b68167640 1 -- 172.21.15.12:0/3224655958 wait complete. 2024-04-22T13:31:22.074 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:31:22.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.182+0000 7fb403606640 1 -- 172.21.15.12:0/1530489328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3fc101a40 msgr2=0x7fb3fc105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.182+0000 7fb403606640 1 --2- 172.21.15.12:0/1530489328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3fc101a40 0x7fb3fc105a80 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec009900 tx=0x7fb3ec02bc30 comp rx=0 tx=0).stop 2024-04-22T13:31:22.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.182+0000 7fb403606640 1 -- 172.21.15.12:0/1530489328 shutdown_connections 2024-04-22T13:31:22.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.182+0000 7fb403606640 1 --2- 172.21.15.12:0/1530489328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3fc106160 0x7fb3fc10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.183+0000 7fb403606640 1 --2- 172.21.15.12:0/1530489328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3fc101a40 0x7fb3fc105a80 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.183+0000 7fb403606640 1 --2- 172.21.15.12:0/1530489328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3fc101090 0x7fb3fc101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.183+0000 7fb403606640 1 -- 172.21.15.12:0/1530489328 >> 172.21.15.12:0/1530489328 conn(0x7fb3fc077f50 msgr2=0x7fb3fc078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:22.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.183+0000 7fb403606640 1 -- 172.21.15.12:0/1530489328 shutdown_connections 2024-04-22T13:31:22.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.183+0000 7fb403606640 1 -- 172.21.15.12:0/1530489328 wait complete. 2024-04-22T13:31:22.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.183+0000 7fb403606640 1 Processor -- start 2024-04-22T13:31:22.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.183+0000 7fb403606640 1 -- start start 2024-04-22T13:31:22.180 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.183+0000 7fb403606640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3fc101090 0x7fb3fc198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.183+0000 7fb403606640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3fc106160 0x7fb3fc198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.183+0000 7fb403606640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3fc19d000 0x7fb3fc19d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.183+0000 7fb403606640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb3fc1121e0 con 0x7fb3fc19d000 2024-04-22T13:31:22.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.183+0000 7fb403606640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb3fc112060 con 0x7fb3fc101090 2024-04-22T13:31:22.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.183+0000 7fb403606640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb3fc112360 con 0x7fb3fc106160 2024-04-22T13:31:22.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.184+0000 7fb401ba3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3fc19d000 0x7fb3fc19d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.184+0000 7fb401ba3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3fc19d000 0x7fb3fc19d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44624/0 (socket says 172.21.15.12:44624) 2024-04-22T13:31:22.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.184+0000 7fb401ba3640 1 -- 172.21.15.12:0/1143930015 learned_addr learned my addr 172.21.15.12:0/1143930015 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:22.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.184+0000 7fb4013a2640 1 --2- 172.21.15.12:0/1143930015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3fc101090 0x7fb3fc198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.184+0000 7fb400ba1640 1 --2- 172.21.15.12:0/1143930015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3fc106160 0x7fb3fc198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.184+0000 7fb401ba3640 1 -- 172.21.15.12:0/1143930015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3fc106160 msgr2=0x7fb3fc198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.184+0000 7fb401ba3640 1 --2- 172.21.15.12:0/1143930015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3fc106160 0x7fb3fc198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.184+0000 7fb401ba3640 1 -- 172.21.15.12:0/1143930015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3fc101090 msgr2=0x7fb3fc198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.181 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.184+0000 7fb401ba3640 1 --2- 172.21.15.12:0/1143930015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3fc101090 0x7fb3fc198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.184+0000 7fb401ba3640 1 -- 172.21.15.12:0/1143930015 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3fc19daf0 con 0x7fb3fc19d000 2024-04-22T13:31:22.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.184+0000 7fb401ba3640 1 --2- 172.21.15.12:0/1143930015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3fc19d000 0x7fb3fc19d4b0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400d750 tx=0x7fb3f400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:22.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.184+0000 7fb3f27fc640 1 -- 172.21.15.12:0/1143930015 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4017070 con 0x7fb3fc19d000 2024-04-22T13:31:22.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.184+0000 7fb403606640 1 -- 172.21.15.12:0/1143930015 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3fc19dde0 con 0x7fb3fc19d000 2024-04-22T13:31:22.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.184+0000 7fb3f27fc640 1 -- 172.21.15.12:0/1143930015 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3f40054d0 con 0x7fb3fc19d000 2024-04-22T13:31:22.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.184+0000 7fb3f27fc640 1 -- 172.21.15.12:0/1143930015 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4004c60 con 0x7fb3fc19d000 2024-04-22T13:31:22.182 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.184+0000 7fb403606640 1 -- 172.21.15.12:0/1143930015 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3fc1a5550 con 0x7fb3fc19d000 2024-04-22T13:31:22.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.187+0000 7fb3f27fc640 1 -- 172.21.15.12:0/1143930015 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb3f4009bd0 con 0x7fb3fc19d000 2024-04-22T13:31:22.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.187+0000 7fb4061fa740 1 Processor -- start 2024-04-22T13:31:22.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.187+0000 7fb4061fa740 1 -- start start 2024-04-22T13:31:22.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.187+0000 7fb4061fa740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b0764934d0 0x55b0764938b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.187+0000 7fb4061fa740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b076493e80 0x55b07649c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.187+0000 7fb4061fa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b07649ca00 0x55b07649ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.187+0000 7fb4061fa740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b0763f81f0 con 0x55b0764934d0 2024-04-22T13:31:22.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.187+0000 7fb4061fa740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b0763954b0 con 0x55b076493e80 2024-04-22T13:31:22.189 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.187+0000 7fb4061fa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b0763f41b0 con 0x55b07649ca00 2024-04-22T13:31:22.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.188+0000 7fb400ba1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b0764934d0 0x55b0764938b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.188+0000 7fb400ba1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b0764934d0 0x55b0764938b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44640/0 (socket says 172.21.15.12:44640) 2024-04-22T13:31:22.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.188+0000 7fb400ba1640 1 -- 172.21.15.12:0/1725773433 learned_addr learned my addr 172.21.15.12:0/1725773433 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:22.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.188+0000 7fb3f27fc640 1 --2- 172.21.15.12:0/1143930015 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb3e80a3d20 0x7fb3e80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.188+0000 7fb4013a2640 1 --2- 172.21.15.12:0/1725773433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b076493e80 0x55b07649c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.188+0000 7fb400ba1640 1 --2- 172.21.15.12:0/1725773433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b07649ca00 0x55b07649ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.188+0000 7fb401ba3640 1 --2- 172.21.15.12:0/1143930015 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb3e80a3d20 0x7fb3e80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.188+0000 7fb400ba1640 1 -- 172.21.15.12:0/1725773433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b07649ca00 msgr2=0x55b07649ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.188+0000 7fb400ba1640 1 --2- 172.21.15.12:0/1725773433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b07649ca00 0x55b07649ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.188+0000 7fb400ba1640 1 -- 172.21.15.12:0/1725773433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b076493e80 msgr2=0x55b07649c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.188+0000 7fb400ba1640 1 --2- 172.21.15.12:0/1725773433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b076493e80 0x55b07649c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.188+0000 7fb400ba1640 1 -- 172.21.15.12:0/1725773433 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0763871b0 con 0x55b0764934d0 2024-04-22T13:31:22.190 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.188+0000 7fb4013a2640 1 --2- 172.21.15.12:0/1725773433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b076493e80 0x55b07649c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:22.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.188+0000 7fb3f27fc640 1 -- 172.21.15.12:0/1143930015 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fb3f40d00a0 con 0x7fb3fc19d000 2024-04-22T13:31:22.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.188+0000 7fb400ba1640 1 --2- 172.21.15.12:0/1725773433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b0764934d0 0x55b0764938b0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec005940 tx=0x7fb3ec0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d62f54c02211b5a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:22.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.188+0000 7fb401ba3640 1 --2- 172.21.15.12:0/1143930015 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb3e80a3d20 0x7fb3e80a61e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40cd960 tx=0x7fb3f4076040 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:22.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.188+0000 7fb3e77fe640 1 -- 172.21.15.12:0/1725773433 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec02c7e0 con 0x55b0764934d0 2024-04-22T13:31:22.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.188+0000 7fb3e77fe640 1 -- 172.21.15.12:0/1725773433 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3ec048070 con 0x55b0764934d0 2024-04-22T13:31:22.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.188+0000 7fb3e77fe640 1 -- 172.21.15.12:0/1725773433 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec004970 con 0x55b0764934d0 2024-04-22T13:31:22.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.189+0000 7fb4061fa740 1 -- 172.21.15.12:0/1725773433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b0764934d0 msgr2=0x55b0764938b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.189+0000 7fb4061fa740 1 --2- 172.21.15.12:0/1725773433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b0764934d0 0x55b0764938b0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec005940 tx=0x7fb3ec0058b0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.189+0000 7fb4061fa740 1 -- 172.21.15.12:0/1725773433 shutdown_connections 2024-04-22T13:31:22.191 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.189+0000 7fb4061fa740 1 --2- 172.21.15.12:0/1725773433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b07649ca00 0x55b07649ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.189+0000 7fb4061fa740 1 --2- 172.21.15.12:0/1725773433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b076493e80 0x55b07649c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.189+0000 7fb4061fa740 1 --2- 172.21.15.12:0/1725773433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b0764934d0 0x55b0764938b0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.189+0000 7fb4061fa740 1 -- 172.21.15.12:0/1725773433 >> 172.21.15.12:0/1725773433 conn(0x55b07648dba0 msgr2=0x55b07648df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:22.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.189+0000 7fb4061fa740 1 -- 172.21.15.12:0/1725773433 shutdown_connections 2024-04-22T13:31:22.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.189+0000 7fb4061fa740 1 -- 172.21.15.12:0/1725773433 wait complete. 2024-04-22T13:31:22.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.190+0000 7fb4061fa740 1 -- start start 2024-04-22T13:31:22.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.190+0000 7fb4061fa740 1 Processor -- start 2024-04-22T13:31:22.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.190+0000 7fb4061fa740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b0764934d0 0x55b07649c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.190+0000 7fb4061fa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b076493e80 0x55b076499660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.190+0000 7fb4061fa740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b07649ca00 0x55b076499bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.190+0000 7fb4061fa740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b07649f910 con 0x55b0764934d0 2024-04-22T13:31:22.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.190+0000 7fb4061fa740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b0763b1500 con 0x55b07649ca00 2024-04-22T13:31:22.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.190+0000 7fb4061fa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b07649fa90 con 0x55b076493e80 2024-04-22T13:31:22.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.190+0000 7fb4061fa740 10 client.0 _ll_register_callbacks cb 0x55b07637fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:22.192 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.190+0000 7fb400ba1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b0764934d0 0x55b07649c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.190+0000 7fb400ba1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b0764934d0 0x55b07649c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44650/0 (socket says 172.21.15.12:44650) 2024-04-22T13:31:22.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.190+0000 7fb400ba1640 1 -- 172.21.15.12:0/3314670559 learned_addr learned my addr 172.21.15.12:0/3314670559 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:22.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.190+0000 7fb4013a2640 1 --2- 172.21.15.12:0/3314670559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b076493e80 0x55b076499660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.190+0000 7fb400ba1640 1 -- 172.21.15.12:0/3314670559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b076493e80 msgr2=0x55b076499660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.190+0000 7fb400ba1640 1 --2- 172.21.15.12:0/3314670559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b076493e80 0x55b076499660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.190+0000 7fb400ba1640 1 -- 172.21.15.12:0/3314670559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b07649ca00 msgr2=0x55b076499bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.190+0000 7fb400ba1640 1 --2- 172.21.15.12:0/3314670559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b07649ca00 0x55b076499bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.190+0000 7fb400ba1640 1 -- 172.21.15.12:0/3314670559 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b07610a3f0 con 0x55b0764934d0 2024-04-22T13:31:22.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.191+0000 7fb400ba1640 1 --2- 172.21.15.12:0/3314670559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b0764934d0 0x55b07649c2d0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec036040 tx=0x7fb3ec02ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aec25ee9fada7f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:22.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.191+0000 7fb3e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:22.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.191+0000 7fb3e4ff9640 1 -- 172.21.15.12:0/3314670559 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec051070 con 0x55b0764934d0 2024-04-22T13:31:22.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.191+0000 7fb3e4ff9640 1 -- 172.21.15.12:0/3314670559 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fb3ec005450 con 0x55b0764934d0 2024-04-22T13:31:22.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.191+0000 7fb4061fa740 1 -- 172.21.15.12:0/3314670559 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3ec048070 con 0x55b0764934d0 2024-04-22T13:31:22.193 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.191+0000 7fb3e4ff9640 1 -- 172.21.15.12:0/3314670559 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb3ec05d050 con 0x55b0764934d0 2024-04-22T13:31:22.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.191+0000 7fb4061fa740 10 client.18243 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:22.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.191+0000 7fb4061fa740 1 -- 172.21.15.12:0/3314670559 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b0761d47d0 con 0x55b0764934d0 2024-04-22T13:31:22.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.191+0000 7fb3e4ff9640 1 -- 172.21.15.12:0/3314670559 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3ec048070 con 0x55b0764934d0 2024-04-22T13:31:22.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.191+0000 7fb3e4ff9640 1 -- 172.21.15.12:0/3314670559 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb3ec003890 con 0x55b0764934d0 2024-04-22T13:31:22.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.191+0000 7fb4061fa740 10 client.18243 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:22.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.191+0000 7fb4061fa740 10 client.18243 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:31:22.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.191+0000 7fb4061fa740 20 client.18243 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:22.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.191+0000 7fb4061fa740 1 --2- 172.21.15.12:0/3314670559 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55b0764aca00 0x55b0764aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.191+0000 7fb4061fa740 4 client.18243 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:22.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.191+0000 7fb4061fa740 1 -- 172.21.15.12:0/3314670559 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b076397930 con 0x55b0764aca00 2024-04-22T13:31:22.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.192+0000 7fb4013a2640 1 --2- 172.21.15.12:0/3314670559 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55b0764aca00 0x55b0764aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.192+0000 7fb3e4ff9640 10 client.18243 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:31:22.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.192+0000 7fb4013a2640 1 --2- 172.21.15.12:0/3314670559 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55b0764aca00 0x55b0764aedf0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=136d54068f99f2f5 server_cookie=1ce343c5256006e0 in_seq=0 out_seq=0 2024-04-22T13:31:22.194 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.193+0000 7fb3e4ff9640 1 -- 172.21.15.12:0/3314670559 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b076397930 con 0x55b0764aca00 2024-04-22T13:31:22.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.193+0000 7fb3e4ff9640 10 client.18243 handle_command_reply: tid=0 2024-04-22T13:31:22.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.193+0000 7fb4061fa740 1 client.18243 shutdown 2024-04-22T13:31:22.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.193+0000 7fb4061fa740 1 -- 172.21.15.12:0/3314670559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b0764934d0 msgr2=0x55b07649c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.193+0000 7fb4061fa740 1 --2- 172.21.15.12:0/3314670559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b0764934d0 0x55b07649c2d0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec036040 tx=0x7fb3ec02ca80 comp rx=0 tx=0).stop 2024-04-22T13:31:22.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.193+0000 7fb4061fa740 1 -- 172.21.15.12:0/3314670559 shutdown_connections 2024-04-22T13:31:22.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.193+0000 7fb4061fa740 1 --2- 172.21.15.12:0/3314670559 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55b0764aca00 0x55b0764aedf0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.193+0000 7fb4061fa740 1 --2- 172.21.15.12:0/3314670559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b07649ca00 0x55b076499bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.193+0000 7fb3e4ff9640 0 client.18243 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:31:22.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.193+0000 7fb4061fa740 1 --2- 172.21.15.12:0/3314670559 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b076493e80 0x55b076499660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.193+0000 7fb4061fa740 1 --2- 172.21.15.12:0/3314670559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b0764934d0 0x55b07649c2d0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.193+0000 7fb4061fa740 1 -- 172.21.15.12:0/3314670559 >> 172.21.15.12:0/3314670559 conn(0x55b07648dba0 msgr2=0x55b07648f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:22.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.193+0000 7fb4061fa740 1 -- 172.21.15.12:0/3314670559 shutdown_connections 2024-04-22T13:31:22.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.193+0000 7fb4061fa740 1 -- 172.21.15.12:0/3314670559 wait complete. 2024-04-22T13:31:22.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.193+0000 7fb4061fa740 20 client.18243 trim_cache size 0 max 16384 2024-04-22T13:31:22.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.207+0000 7fb4061fa740 1 Processor -- start 2024-04-22T13:31:22.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.207+0000 7fb4061fa740 1 -- start start 2024-04-22T13:31:22.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.207+0000 7fb4061fa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0764934d0 0x55b0764a87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.207+0000 7fb4061fa740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b076493e80 0x55b076497e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.207+0000 7fb4061fa740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b07649ca00 0x55b07649bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.207+0000 7fb4061fa740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb3fc1ada30 con 0x55b07649ca00 2024-04-22T13:31:22.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.207+0000 7fb4061fa740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb3fc1af570 con 0x55b076493e80 2024-04-22T13:31:22.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.207+0000 7fb4061fa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb3fc1ae100 con 0x55b0764934d0 2024-04-22T13:31:22.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.207+0000 7fb400ba1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b07649ca00 0x55b07649bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.208+0000 7fb4013a2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b076493e80 0x55b076497e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.208+0000 7fb400ba1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b07649ca00 0x55b07649bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44660/0 (socket says 172.21.15.12:44660) 2024-04-22T13:31:22.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.208+0000 7fb400ba1640 1 -- 172.21.15.12:0/2989141230 learned_addr learned my addr 172.21.15.12:0/2989141230 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:22.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.208+0000 7fb400ba1640 1 --2- 172.21.15.12:0/2989141230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0764934d0 0x55b0764a87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.208+0000 7fb4013a2640 1 -- 172.21.15.12:0/2989141230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0764934d0 msgr2=0x55b0764a87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.208+0000 7fb4013a2640 1 --2- 172.21.15.12:0/2989141230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0764934d0 0x55b0764a87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.208+0000 7fb4013a2640 1 -- 172.21.15.12:0/2989141230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b07649ca00 msgr2=0x55b07649bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.208+0000 7fb4013a2640 1 --2- 172.21.15.12:0/2989141230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b07649ca00 0x55b07649bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.208+0000 7fb4013a2640 1 -- 172.21.15.12:0/2989141230 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b076497bb0 con 0x55b076493e80 2024-04-22T13:31:22.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.208+0000 7fb400ba1640 1 --2- 172.21.15.12:0/2989141230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b07649ca00 0x55b07649bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:31:22.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.208+0000 7fb4013a2640 1 --2- 172.21.15.12:0/2989141230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b076493e80 0x55b076497e00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x55b076498ec0 tx=0x7fb3f8012bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fe0d57fe43ff201 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:22.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.208+0000 7fb3e6ffd640 1 -- 172.21.15.12:0/2989141230 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8004b20 con 0x55b076493e80 2024-04-22T13:31:22.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.208+0000 7fb3e6ffd640 1 -- 172.21.15.12:0/2989141230 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3f8022030 con 0x55b076493e80 2024-04-22T13:31:22.205 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.208+0000 7fb3e6ffd640 1 -- 172.21.15.12:0/2989141230 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f800fbb0 con 0x55b076493e80 2024-04-22T13:31:22.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.209+0000 7fb4061fa740 1 -- 172.21.15.12:0/2989141230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b076493e80 msgr2=0x55b076497e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.209+0000 7fb4061fa740 1 --2- 172.21.15.12:0/2989141230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b076493e80 0x55b076497e00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x55b076498ec0 tx=0x7fb3f8012bd0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.209+0000 7fb4061fa740 1 -- 172.21.15.12:0/2989141230 shutdown_connections 2024-04-22T13:31:22.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.209+0000 7fb4061fa740 1 --2- 172.21.15.12:0/2989141230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b07649ca00 0x55b07649bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.209+0000 7fb4061fa740 1 --2- 172.21.15.12:0/2989141230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b076493e80 0x55b076497e00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.209+0000 7fb4061fa740 1 --2- 172.21.15.12:0/2989141230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b0764934d0 0x55b0764a87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.209+0000 7fb4061fa740 1 -- 172.21.15.12:0/2989141230 >> 172.21.15.12:0/2989141230 conn(0x55b07648dba0 msgr2=0x55b076494b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:22.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.209+0000 7fb4061fa740 1 -- 172.21.15.12:0/2989141230 shutdown_connections 2024-04-22T13:31:22.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.209+0000 7fb4061fa740 1 -- 172.21.15.12:0/2989141230 wait complete. 2024-04-22T13:31:22.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.209+0000 7fb4061fa740 1 -- start start 2024-04-22T13:31:22.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.210+0000 7fb4061fa740 1 Processor -- start 2024-04-22T13:31:22.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.210+0000 7fb4061fa740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b0764934d0 0x55b0764a0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.210+0000 7fb4061fa740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b076493e80 0x55b0764a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.210+0000 7fb4061fa740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b07649ca00 0x55b0764a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.210+0000 7fb4061fa740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb3fc1aeea0 con 0x55b07649ca00 2024-04-22T13:31:22.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.210+0000 7fb4061fa740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb3fc1ae7d0 con 0x55b0764934d0 2024-04-22T13:31:22.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.210+0000 7fb4061fa740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb3fc1aba30 con 0x55b076493e80 2024-04-22T13:31:22.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.210+0000 7fb4061fa740 10 client.0 _ll_register_callbacks cb 0x7fb3ec004450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:22.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.210+0000 7fb400ba1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b07649ca00 0x55b0764a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.210+0000 7fb400ba1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b07649ca00 0x55b0764a47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44674/0 (socket says 172.21.15.12:44674) 2024-04-22T13:31:22.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.210+0000 7fb400ba1640 1 -- 172.21.15.12:0/2439037512 learned_addr learned my addr 172.21.15.12:0/2439037512 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:22.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.210+0000 7fb400ba1640 1 --2- 172.21.15.12:0/2439037512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b0764934d0 0x55b0764a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.210+0000 7fb4013a2640 1 --2- 172.21.15.12:0/2439037512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b076493e80 0x55b0764a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.210+0000 7fb400ba1640 1 -- 172.21.15.12:0/2439037512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b076493e80 msgr2=0x55b0764a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.210+0000 7fb400ba1640 1 --2- 172.21.15.12:0/2439037512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b076493e80 0x55b0764a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.210+0000 7fb400ba1640 1 -- 172.21.15.12:0/2439037512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b0764934d0 msgr2=0x55b0764a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.210+0000 7fb400ba1640 1 --2- 172.21.15.12:0/2439037512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b0764934d0 0x55b0764a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.210+0000 7fb400ba1640 1 -- 172.21.15.12:0/2439037512 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0764a4e10 con 0x55b07649ca00 2024-04-22T13:31:22.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.211+0000 7fb4013a2640 1 --2- 172.21.15.12:0/2439037512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b076493e80 0x55b0764a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:22.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.211+0000 7fb400ba1640 1 --2- 172.21.15.12:0/2439037512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b07649ca00 0x55b0764a47a0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec003ce0 tx=0x7fb3ec005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1bf5e5b8e9d8096 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:22.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.211+0000 7fb3e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:22.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.211+0000 7fb3e4ff9640 1 -- 172.21.15.12:0/2439037512 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec051040 con 0x55b07649ca00 2024-04-22T13:31:22.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.211+0000 7fb3e4ff9640 1 -- 172.21.15.12:0/2439037512 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fb3ec0037f0 con 0x55b07649ca00 2024-04-22T13:31:22.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.211+0000 7fb4061fa740 1 -- 172.21.15.12:0/2439037512 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3f8022030 con 0x55b07649ca00 2024-04-22T13:31:22.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.211+0000 7fb3e4ff9640 1 -- 172.21.15.12:0/2439037512 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb3ec005d50 con 0x55b07649ca00 2024-04-22T13:31:22.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.211+0000 7fb4061fa740 10 client.18249 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:22.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.211+0000 7fb4061fa740 1 -- 172.21.15.12:0/2439037512 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b0765127c0 con 0x55b07649ca00 2024-04-22T13:31:22.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.211+0000 7fb3e4ff9640 1 -- 172.21.15.12:0/2439037512 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3f8022030 con 0x55b07649ca00 2024-04-22T13:31:22.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.211+0000 7fb3e4ff9640 1 -- 172.21.15.12:0/2439037512 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb3ec043690 con 0x55b07649ca00 2024-04-22T13:31:22.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.212+0000 7fb4061fa740 10 client.18249 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:22.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.212+0000 7fb4061fa740 10 client.18249 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:31:22.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.212+0000 7fb4061fa740 20 client.18249 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:22.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.212+0000 7fb4061fa740 1 --2- 172.21.15.12:0/2439037512 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55b076512e20 0x55b076515210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.212+0000 7fb4061fa740 4 client.18249 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:31:22.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.212+0000 7fb4061fa740 1 -- 172.21.15.12:0/2439037512 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b076405c10 con 0x55b076512e20 2024-04-22T13:31:22.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.212+0000 7fb4013a2640 1 --2- 172.21.15.12:0/2439037512 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55b076512e20 0x55b076515210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.212+0000 7fb3e4ff9640 10 client.18249 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:31:22.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.212+0000 7fb4013a2640 1 --2- 172.21.15.12:0/2439037512 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55b076512e20 0x55b076515210 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8c621ba54883c83c server_cookie=a792eb559ee7804b in_seq=0 out_seq=0 2024-04-22T13:31:22.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.213+0000 7fb3e4ff9640 1 -- 172.21.15.12:0/2439037512 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b076405c10 con 0x55b076512e20 2024-04-22T13:31:22.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.213+0000 7fb3e4ff9640 10 client.18249 handle_command_reply: tid=0 2024-04-22T13:31:22.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.213+0000 7fb4061fa740 1 client.18249 shutdown 2024-04-22T13:31:22.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.213+0000 7fb4061fa740 1 -- 172.21.15.12:0/2439037512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b07649ca00 msgr2=0x55b0764a47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.213+0000 7fb4061fa740 1 --2- 172.21.15.12:0/2439037512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b07649ca00 0x55b0764a47a0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec003ce0 tx=0x7fb3ec005a20 comp rx=0 tx=0).stop 2024-04-22T13:31:22.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.213+0000 7fb4061fa740 1 -- 172.21.15.12:0/2439037512 shutdown_connections 2024-04-22T13:31:22.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.213+0000 7fb4061fa740 1 --2- 172.21.15.12:0/2439037512 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55b076512e20 0x55b076515210 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.213+0000 7fb4061fa740 1 --2- 172.21.15.12:0/2439037512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b07649ca00 0x55b0764a47a0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.213+0000 7fb4061fa740 1 --2- 172.21.15.12:0/2439037512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b076493e80 0x55b0764a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.213+0000 7fb4061fa740 1 --2- 172.21.15.12:0/2439037512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b0764934d0 0x55b0764a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.213+0000 7fb3e4ff9640 0 client.18249 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:31:22.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.213+0000 7fb4061fa740 1 -- 172.21.15.12:0/2439037512 >> 172.21.15.12:0/2439037512 conn(0x55b07648dba0 msgr2=0x55b07648c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:22.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.213+0000 7fb4061fa740 1 -- 172.21.15.12:0/2439037512 shutdown_connections 2024-04-22T13:31:22.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.213+0000 7fb4061fa740 1 -- 172.21.15.12:0/2439037512 wait complete. 2024-04-22T13:31:22.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.213+0000 7fb4061fa740 20 client.18249 trim_cache size 0 max 16384 2024-04-22T13:31:22.220 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:31:22.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.213+0000 7fb403606640 1 -- 172.21.15.12:0/1143930015 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb3e80a3d20 msgr2=0x7fb3e80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.213+0000 7fb403606640 1 --2- 172.21.15.12:0/1143930015 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb3e80a3d20 0x7fb3e80a61e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40cd960 tx=0x7fb3f4076040 comp rx=0 tx=0).stop 2024-04-22T13:31:22.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.213+0000 7fb403606640 1 -- 172.21.15.12:0/1143930015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3fc19d000 msgr2=0x7fb3fc19d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.213+0000 7fb403606640 1 --2- 172.21.15.12:0/1143930015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3fc19d000 0x7fb3fc19d4b0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400d750 tx=0x7fb3f400dc10 comp rx=0 tx=0).stop 2024-04-22T13:31:22.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.214+0000 7fb403606640 1 -- 172.21.15.12:0/1143930015 shutdown_connections 2024-04-22T13:31:22.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.214+0000 7fb403606640 1 --2- 172.21.15.12:0/1143930015 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb3e80a3d20 0x7fb3e80a61e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.214+0000 7fb403606640 1 --2- 172.21.15.12:0/1143930015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3fc19d000 0x7fb3fc19d4b0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.214+0000 7fb403606640 1 --2- 172.21.15.12:0/1143930015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3fc106160 0x7fb3fc198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.214+0000 7fb403606640 1 --2- 172.21.15.12:0/1143930015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3fc101090 0x7fb3fc198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.214+0000 7fb403606640 1 -- 172.21.15.12:0/1143930015 >> 172.21.15.12:0/1143930015 conn(0x7fb3fc077f50 msgr2=0x7fb3fc0ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:22.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.214+0000 7fb403606640 1 -- 172.21.15.12:0/1143930015 shutdown_connections 2024-04-22T13:31:22.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.214+0000 7fb403606640 1 -- 172.21.15.12:0/1143930015 wait complete. 2024-04-22T13:31:22.222 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:31:22.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.332+0000 7eff5533b640 1 -- 172.21.15.12:0/2525332543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff50101a50 msgr2=0x7eff50105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.332+0000 7eff5533b640 1 --2- 172.21.15.12:0/2525332543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff50101a50 0x7eff50105a90 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7eff44009900 tx=0x7eff4402bc30 comp rx=0 tx=0).stop 2024-04-22T13:31:22.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.332+0000 7eff5533b640 1 -- 172.21.15.12:0/2525332543 shutdown_connections 2024-04-22T13:31:22.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.332+0000 7eff5533b640 1 --2- 172.21.15.12:0/2525332543 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff50106170 0x7eff5010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.332+0000 7eff5533b640 1 --2- 172.21.15.12:0/2525332543 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff50101a50 0x7eff50105a90 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.332+0000 7eff5533b640 1 --2- 172.21.15.12:0/2525332543 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff501010a0 0x7eff50101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.332+0000 7eff5533b640 1 -- 172.21.15.12:0/2525332543 >> 172.21.15.12:0/2525332543 conn(0x7eff50077eb0 msgr2=0x7eff500782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:22.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.332+0000 7eff5533b640 1 -- 172.21.15.12:0/2525332543 shutdown_connections 2024-04-22T13:31:22.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.332+0000 7eff5533b640 1 -- 172.21.15.12:0/2525332543 wait complete. 2024-04-22T13:31:22.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.333+0000 7eff5533b640 1 Processor -- start 2024-04-22T13:31:22.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.333+0000 7eff5533b640 1 -- start start 2024-04-22T13:31:22.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.333+0000 7eff5533b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff501010a0 0x7eff501a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.333+0000 7eff5533b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff50106170 0x7eff501a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.333+0000 7eff5533b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff5019a790 0x7eff5019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.333+0000 7eff5533b640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7eff501110b0 con 0x7eff5019a790 2024-04-22T13:31:22.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.333+0000 7eff5533b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7eff50110f30 con 0x7eff50106170 2024-04-22T13:31:22.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.333+0000 7eff5533b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7eff50111230 con 0x7eff501010a0 2024-04-22T13:31:22.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.333+0000 7eff4f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff5019a790 0x7eff5019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.333+0000 7eff4f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff5019a790 0x7eff5019ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44704/0 (socket says 172.21.15.12:44704) 2024-04-22T13:31:22.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.333+0000 7eff4f7fe640 1 -- 172.21.15.12:0/3727717514 learned_addr learned my addr 172.21.15.12:0/3727717514 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:22.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.333+0000 7eff4effd640 1 --2- 172.21.15.12:0/3727717514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff501010a0 0x7eff501a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.333+0000 7eff4e7fc640 1 --2- 172.21.15.12:0/3727717514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff50106170 0x7eff501a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.334+0000 7eff4f7fe640 1 -- 172.21.15.12:0/3727717514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff501010a0 msgr2=0x7eff501a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.334+0000 7eff4f7fe640 1 --2- 172.21.15.12:0/3727717514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff501010a0 0x7eff501a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.334+0000 7eff4f7fe640 1 -- 172.21.15.12:0/3727717514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff50106170 msgr2=0x7eff501a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.334+0000 7eff4f7fe640 1 --2- 172.21.15.12:0/3727717514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff50106170 0x7eff501a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.334+0000 7eff4f7fe640 1 -- 172.21.15.12:0/3727717514 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5019b370 con 0x7eff5019a790 2024-04-22T13:31:22.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.334+0000 7eff4f7fe640 1 --2- 172.21.15.12:0/3727717514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff5019a790 0x7eff5019ac40 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7eff4800eaa0 tx=0x7eff4800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:22.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.334+0000 7eff37fff640 1 -- 172.21.15.12:0/3727717514 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff48004d20 con 0x7eff5019a790 2024-04-22T13:31:22.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.334+0000 7eff37fff640 1 -- 172.21.15.12:0/3727717514 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff48005020 con 0x7eff5019a790 2024-04-22T13:31:22.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.334+0000 7eff5533b640 1 -- 172.21.15.12:0/3727717514 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff501a7450 con 0x7eff5019a790 2024-04-22T13:31:22.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.334+0000 7eff37fff640 1 -- 172.21.15.12:0/3727717514 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff480116b0 con 0x7eff5019a790 2024-04-22T13:31:22.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.334+0000 7eff5533b640 1 -- 172.21.15.12:0/3727717514 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff501a7800 con 0x7eff5019a790 2024-04-22T13:31:22.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.336+0000 7eff37fff640 1 -- 172.21.15.12:0/3727717514 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7eff480051d0 con 0x7eff5019a790 2024-04-22T13:31:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.336+0000 7eff58005740 1 Processor -- start 2024-04-22T13:31:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.336+0000 7eff58005740 1 -- start start 2024-04-22T13:31:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.337+0000 7eff58005740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d6cb364d0 0x559d6cb368b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.337+0000 7eff58005740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d6cb36e80 0x559d6cb3f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.337+0000 7eff58005740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d6cb3fa00 0x559d6cb41e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.337+0000 7eff58005740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559d6ca9b1f0 con 0x559d6cb364d0 2024-04-22T13:31:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.337+0000 7eff58005740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559d6ca384b0 con 0x559d6cb36e80 2024-04-22T13:31:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.337+0000 7eff58005740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559d6ca971b0 con 0x559d6cb3fa00 2024-04-22T13:31:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.337+0000 7eff4e7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d6cb364d0 0x559d6cb368b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.337+0000 7eff4e7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d6cb364d0 0x559d6cb368b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44708/0 (socket says 172.21.15.12:44708) 2024-04-22T13:31:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.337+0000 7eff4e7fc640 1 -- 172.21.15.12:0/1027821017 learned_addr learned my addr 172.21.15.12:0/1027821017 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.337+0000 7eff37fff640 1 --2- 172.21.15.12:0/3727717514 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7eff380a3d20 0x7eff380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.337+0000 7eff4e7fc640 1 --2- 172.21.15.12:0/1027821017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d6cb3fa00 0x559d6cb41e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.337+0000 7eff4effd640 1 --2- 172.21.15.12:0/1027821017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d6cb36e80 0x559d6cb3f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.337+0000 7eff4e7fc640 1 -- 172.21.15.12:0/1027821017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d6cb3fa00 msgr2=0x559d6cb41e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.337+0000 7eff4e7fc640 1 --2- 172.21.15.12:0/1027821017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d6cb3fa00 0x559d6cb41e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.337+0000 7eff4e7fc640 1 -- 172.21.15.12:0/1027821017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d6cb36e80 msgr2=0x559d6cb3f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.337+0000 7eff4e7fc640 1 --2- 172.21.15.12:0/1027821017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d6cb36e80 0x559d6cb3f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.337+0000 7eff4e7fc640 1 -- 172.21.15.12:0/1027821017 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d6ca2a1b0 con 0x559d6cb364d0 2024-04-22T13:31:22.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.337+0000 7eff4effd640 1 --2- 172.21.15.12:0/1027821017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d6cb36e80 0x559d6cb3f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:22.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.337+0000 7eff4f7fe640 1 --2- 172.21.15.12:0/3727717514 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7eff380a3d20 0x7eff380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.337+0000 7eff37fff640 1 -- 172.21.15.12:0/3727717514 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7eff480d0900 con 0x7eff5019a790 2024-04-22T13:31:22.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.337+0000 7eff4e7fc640 1 --2- 172.21.15.12:0/1027821017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d6cb364d0 0x559d6cb368b0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7eff5006a420 tx=0x7eff44009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518d861afe8b790 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:22.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.338+0000 7eff357fa640 1 -- 172.21.15.12:0/1027821017 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff44047020 con 0x559d6cb364d0 2024-04-22T13:31:22.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.338+0000 7eff4f7fe640 1 --2- 172.21.15.12:0/3727717514 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7eff380a3d20 0x7eff380a61e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7eff48018000 tx=0x7eff4806f350 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:22.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.338+0000 7eff357fa640 1 -- 172.21.15.12:0/1027821017 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff4402cc90 con 0x559d6cb364d0 2024-04-22T13:31:22.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.338+0000 7eff357fa640 1 -- 172.21.15.12:0/1027821017 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff440049e0 con 0x559d6cb364d0 2024-04-22T13:31:22.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.338+0000 7eff58005740 1 -- 172.21.15.12:0/1027821017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d6cb364d0 msgr2=0x559d6cb368b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.338+0000 7eff58005740 1 --2- 172.21.15.12:0/1027821017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d6cb364d0 0x559d6cb368b0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7eff5006a420 tx=0x7eff44009560 comp rx=0 tx=0).stop 2024-04-22T13:31:22.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.338+0000 7eff58005740 1 -- 172.21.15.12:0/1027821017 shutdown_connections 2024-04-22T13:31:22.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.338+0000 7eff58005740 1 --2- 172.21.15.12:0/1027821017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d6cb3fa00 0x559d6cb41e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.338+0000 7eff58005740 1 --2- 172.21.15.12:0/1027821017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d6cb36e80 0x559d6cb3f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.338+0000 7eff58005740 1 --2- 172.21.15.12:0/1027821017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d6cb364d0 0x559d6cb368b0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.338+0000 7eff58005740 1 -- 172.21.15.12:0/1027821017 >> 172.21.15.12:0/1027821017 conn(0x559d6cb30ba0 msgr2=0x559d6cb30f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:22.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.338+0000 7eff58005740 1 -- 172.21.15.12:0/1027821017 shutdown_connections 2024-04-22T13:31:22.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.338+0000 7eff58005740 1 -- 172.21.15.12:0/1027821017 wait complete. 2024-04-22T13:31:22.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.339+0000 7eff58005740 1 -- start start 2024-04-22T13:31:22.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.339+0000 7eff58005740 1 Processor -- start 2024-04-22T13:31:22.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.339+0000 7eff58005740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d6cb364d0 0x559d6cb3f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.339+0000 7eff58005740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d6cb36e80 0x559d6cb3c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.339+0000 7eff58005740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d6cb3fa00 0x559d6cb3cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.339+0000 7eff58005740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559d6cb42910 con 0x559d6cb36e80 2024-04-22T13:31:22.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.339+0000 7eff58005740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559d6ca54500 con 0x559d6cb364d0 2024-04-22T13:31:22.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.339+0000 7eff58005740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559d6cb42a90 con 0x559d6cb3fa00 2024-04-22T13:31:22.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.339+0000 7eff58005740 10 client.0 _ll_register_callbacks cb 0x559d6ca22b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:22.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.339+0000 7eff4effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d6cb36e80 0x559d6cb3c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.339+0000 7eff4effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d6cb36e80 0x559d6cb3c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44710/0 (socket says 172.21.15.12:44710) 2024-04-22T13:31:22.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.339+0000 7eff4effd640 1 -- 172.21.15.12:0/376839995 learned_addr learned my addr 172.21.15.12:0/376839995 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:22.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.339+0000 7eff4e7fc640 1 --2- 172.21.15.12:0/376839995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d6cb3fa00 0x559d6cb3cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.339+0000 7eff4e7fc640 1 --2- 172.21.15.12:0/376839995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d6cb364d0 0x559d6cb3f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.339+0000 7eff4effd640 1 -- 172.21.15.12:0/376839995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d6cb3fa00 msgr2=0x559d6cb3cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.339+0000 7eff4effd640 1 --2- 172.21.15.12:0/376839995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d6cb3fa00 0x559d6cb3cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.339+0000 7eff4effd640 1 -- 172.21.15.12:0/376839995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d6cb364d0 msgr2=0x559d6cb3f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.339+0000 7eff4effd640 1 --2- 172.21.15.12:0/376839995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d6cb364d0 0x559d6cb3f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.339+0000 7eff4effd640 1 -- 172.21.15.12:0/376839995 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d6c7ad3f0 con 0x559d6cb36e80 2024-04-22T13:31:22.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.340+0000 7eff4effd640 1 --2- 172.21.15.12:0/376839995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d6cb36e80 0x559d6cb3c720 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7eff400117d0 tx=0x7eff40011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36670aa4f43c3dcc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:22.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.340+0000 7eff32ff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:22.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.340+0000 7eff32ff5640 1 -- 172.21.15.12:0/376839995 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4001f060 con 0x559d6cb36e80 2024-04-22T13:31:22.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.340+0000 7eff32ff5640 1 -- 172.21.15.12:0/376839995 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7eff4000fcd0 con 0x559d6cb36e80 2024-04-22T13:31:22.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.340+0000 7eff58005740 1 -- 172.21.15.12:0/376839995 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff4402cc90 con 0x559d6cb36e80 2024-04-22T13:31:22.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.340+0000 7eff32ff5640 1 -- 172.21.15.12:0/376839995 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7eff40012450 con 0x559d6cb36e80 2024-04-22T13:31:22.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.340+0000 7eff58005740 10 client.18261 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:22.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.340+0000 7eff58005740 1 -- 172.21.15.12:0/376839995 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559d6c8777d0 con 0x559d6cb36e80 2024-04-22T13:31:22.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.340+0000 7eff32ff5640 1 -- 172.21.15.12:0/376839995 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff4402cc90 con 0x559d6cb36e80 2024-04-22T13:31:22.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.340+0000 7eff32ff5640 1 -- 172.21.15.12:0/376839995 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7eff400225c0 con 0x559d6cb36e80 2024-04-22T13:31:22.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.340+0000 7eff58005740 10 client.18261 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:22.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.340+0000 7eff58005740 10 client.18261 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:31:22.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.341+0000 7eff58005740 20 client.18261 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:22.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.341+0000 7eff58005740 1 --2- 172.21.15.12:0/376839995 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x559d6cb4fc00 0x559d6cb51ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.341+0000 7eff58005740 4 client.18261 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:22.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.341+0000 7eff58005740 1 -- 172.21.15.12:0/376839995 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d6ca3a930 con 0x559d6cb4fc00 2024-04-22T13:31:22.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.341+0000 7eff4e7fc640 1 --2- 172.21.15.12:0/376839995 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x559d6cb4fc00 0x559d6cb51ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.341+0000 7eff4e7fc640 1 --2- 172.21.15.12:0/376839995 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x559d6cb4fc00 0x559d6cb51ff0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f1635c279efcbea server_cookie=7343f26496af3d07 in_seq=0 out_seq=0 2024-04-22T13:31:22.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.341+0000 7eff32ff5640 10 client.18261 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:31:22.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.342+0000 7eff32ff5640 1 -- 172.21.15.12:0/376839995 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559d6ca3a930 con 0x559d6cb4fc00 2024-04-22T13:31:22.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.342+0000 7eff32ff5640 10 client.18261 handle_command_reply: tid=0 2024-04-22T13:31:22.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.342+0000 7eff58005740 1 client.18261 shutdown 2024-04-22T13:31:22.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.342+0000 7eff58005740 1 -- 172.21.15.12:0/376839995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d6cb36e80 msgr2=0x559d6cb3c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.342+0000 7eff58005740 1 --2- 172.21.15.12:0/376839995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d6cb36e80 0x559d6cb3c720 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7eff400117d0 tx=0x7eff40011c90 comp rx=0 tx=0).stop 2024-04-22T13:31:22.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.342+0000 7eff58005740 1 -- 172.21.15.12:0/376839995 shutdown_connections 2024-04-22T13:31:22.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.342+0000 7eff58005740 1 --2- 172.21.15.12:0/376839995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d6cb3fa00 0x559d6cb3cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.342+0000 7eff58005740 1 --2- 172.21.15.12:0/376839995 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x559d6cb4fc00 0x559d6cb51ff0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.342+0000 7eff58005740 1 --2- 172.21.15.12:0/376839995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d6cb36e80 0x559d6cb3c720 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.342+0000 7eff32ff5640 0 client.18261 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:31:22.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.342+0000 7eff58005740 1 --2- 172.21.15.12:0/376839995 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d6cb364d0 0x559d6cb3f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.342+0000 7eff58005740 1 -- 172.21.15.12:0/376839995 >> 172.21.15.12:0/376839995 conn(0x559d6cb30ba0 msgr2=0x559d6cb328f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:22.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.342+0000 7eff58005740 1 -- 172.21.15.12:0/376839995 shutdown_connections 2024-04-22T13:31:22.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.342+0000 7eff58005740 1 -- 172.21.15.12:0/376839995 wait complete. 2024-04-22T13:31:22.346 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.342+0000 7eff58005740 20 client.18261 trim_cache size 0 max 16384 2024-04-22T13:31:22.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.355+0000 7eff58005740 1 Processor -- start 2024-04-22T13:31:22.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.355+0000 7eff58005740 1 -- start start 2024-04-22T13:31:22.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.356+0000 7eff58005740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d6cb364d0 0x559d6cb4b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.356+0000 7eff58005740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d6cb36e80 0x559d6cb3ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.356+0000 7eff58005740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d6cb3fa00 0x559d6cb3f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.356+0000 7eff58005740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7eff501afbe0 con 0x559d6cb364d0 2024-04-22T13:31:22.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.356+0000 7eff58005740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7eff501b1720 con 0x559d6cb36e80 2024-04-22T13:31:22.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.356+0000 7eff58005740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7eff501b02b0 con 0x559d6cb3fa00 2024-04-22T13:31:22.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.356+0000 7eff4e7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d6cb364d0 0x559d6cb4b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.356+0000 7eff4e7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d6cb364d0 0x559d6cb4b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44726/0 (socket says 172.21.15.12:44726) 2024-04-22T13:31:22.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.356+0000 7eff4e7fc640 1 -- 172.21.15.12:0/557218753 learned_addr learned my addr 172.21.15.12:0/557218753 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:22.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.356+0000 7eff4e7fc640 1 --2- 172.21.15.12:0/557218753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d6cb3fa00 0x559d6cb3f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.356+0000 7eff4effd640 1 --2- 172.21.15.12:0/557218753 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d6cb36e80 0x559d6cb3ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.356+0000 7eff4e7fc640 1 -- 172.21.15.12:0/557218753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d6cb3fa00 msgr2=0x559d6cb3f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.356+0000 7eff4e7fc640 1 --2- 172.21.15.12:0/557218753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d6cb3fa00 0x559d6cb3f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.356+0000 7eff4e7fc640 1 -- 172.21.15.12:0/557218753 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d6cb36e80 msgr2=0x559d6cb3ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.356+0000 7eff4e7fc640 1 --2- 172.21.15.12:0/557218753 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d6cb36e80 0x559d6cb3ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.356+0000 7eff4e7fc640 1 -- 172.21.15.12:0/557218753 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d6cb306d0 con 0x559d6cb364d0 2024-04-22T13:31:22.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.356+0000 7eff4e7fc640 1 --2- 172.21.15.12:0/557218753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d6cb364d0 0x559d6cb4b7d0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7eff44009900 tx=0x7eff4402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a14929090f4b8dc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:22.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.356+0000 7eff34ff9640 1 -- 172.21.15.12:0/557218753 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff44047040 con 0x559d6cb364d0 2024-04-22T13:31:22.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.357+0000 7eff34ff9640 1 -- 172.21.15.12:0/557218753 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff440054f0 con 0x559d6cb364d0 2024-04-22T13:31:22.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.357+0000 7eff34ff9640 1 -- 172.21.15.12:0/557218753 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff44004920 con 0x559d6cb364d0 2024-04-22T13:31:22.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.357+0000 7eff58005740 1 -- 172.21.15.12:0/557218753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d6cb364d0 msgr2=0x559d6cb4b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.357+0000 7eff58005740 1 --2- 172.21.15.12:0/557218753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d6cb364d0 0x559d6cb4b7d0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7eff44009900 tx=0x7eff4402bc30 comp rx=0 tx=0).stop 2024-04-22T13:31:22.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.357+0000 7eff58005740 1 -- 172.21.15.12:0/557218753 shutdown_connections 2024-04-22T13:31:22.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.357+0000 7eff58005740 1 --2- 172.21.15.12:0/557218753 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d6cb3fa00 0x559d6cb3f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.357+0000 7eff58005740 1 --2- 172.21.15.12:0/557218753 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d6cb36e80 0x559d6cb3ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.357+0000 7eff58005740 1 --2- 172.21.15.12:0/557218753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d6cb364d0 0x559d6cb4b7d0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.357+0000 7eff58005740 1 -- 172.21.15.12:0/557218753 >> 172.21.15.12:0/557218753 conn(0x559d6cb30ba0 msgr2=0x559d6cb37c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:22.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.357+0000 7eff58005740 1 -- 172.21.15.12:0/557218753 shutdown_connections 2024-04-22T13:31:22.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.357+0000 7eff58005740 1 -- 172.21.15.12:0/557218753 wait complete. 2024-04-22T13:31:22.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.357+0000 7eff58005740 1 -- start start 2024-04-22T13:31:22.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.357+0000 7eff58005740 1 Processor -- start 2024-04-22T13:31:22.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.358+0000 7eff58005740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d6cb364d0 0x559d6cb39640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.358+0000 7eff58005740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d6cb36e80 0x559d6cb39b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.358+0000 7eff58005740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d6cb3fa00 0x559d6cb3a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.358+0000 7eff58005740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7eff501b1050 con 0x559d6cb3fa00 2024-04-22T13:31:22.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.358+0000 7eff58005740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7eff501b0980 con 0x559d6cb364d0 2024-04-22T13:31:22.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.358+0000 7eff58005740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7eff501adbe0 con 0x559d6cb36e80 2024-04-22T13:31:22.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.358+0000 7eff58005740 10 client.0 _ll_register_callbacks cb 0x7eff400113e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:22.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.358+0000 7eff4e7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d6cb3fa00 0x559d6cb3a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.358+0000 7eff4e7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d6cb3fa00 0x559d6cb3a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44736/0 (socket says 172.21.15.12:44736) 2024-04-22T13:31:22.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.358+0000 7eff4e7fc640 1 -- 172.21.15.12:0/13373874 learned_addr learned my addr 172.21.15.12:0/13373874 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:22.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.358+0000 7eff4effd640 1 --2- 172.21.15.12:0/13373874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d6cb36e80 0x559d6cb39b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.358+0000 7eff4e7fc640 1 --2- 172.21.15.12:0/13373874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d6cb364d0 0x559d6cb39640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.358+0000 7eff4e7fc640 1 -- 172.21.15.12:0/13373874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d6cb36e80 msgr2=0x559d6cb39b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.358+0000 7eff4e7fc640 1 --2- 172.21.15.12:0/13373874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d6cb36e80 0x559d6cb39b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.358+0000 7eff4e7fc640 1 -- 172.21.15.12:0/13373874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d6cb364d0 msgr2=0x559d6cb39640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.358+0000 7eff4e7fc640 1 --2- 172.21.15.12:0/13373874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d6cb364d0 0x559d6cb39640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.358+0000 7eff4e7fc640 1 -- 172.21.15.12:0/13373874 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d6cbb5380 con 0x559d6cb3fa00 2024-04-22T13:31:22.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.358+0000 7eff4e7fc640 1 --2- 172.21.15.12:0/13373874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d6cb3fa00 0x559d6cb3a0c0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7eff44059ab0 tx=0x7eff440597d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d721727e4d59003 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:22.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.358+0000 7eff32ff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:22.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.359+0000 7eff32ff5640 1 -- 172.21.15.12:0/13373874 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff44047040 con 0x559d6cb3fa00 2024-04-22T13:31:22.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.359+0000 7eff32ff5640 1 -- 172.21.15.12:0/13373874 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7eff440592d0 con 0x559d6cb3fa00 2024-04-22T13:31:22.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.359+0000 7eff58005740 1 -- 172.21.15.12:0/13373874 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff440054f0 con 0x559d6cb3fa00 2024-04-22T13:31:22.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.359+0000 7eff32ff5640 1 -- 172.21.15.12:0/13373874 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7eff440594d0 con 0x559d6cb3fa00 2024-04-22T13:31:22.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.359+0000 7eff58005740 10 client.18267 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:22.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.359+0000 7eff58005740 1 -- 172.21.15.12:0/13373874 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559d6cbb6660 con 0x559d6cb3fa00 2024-04-22T13:31:22.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.359+0000 7eff32ff5640 1 -- 172.21.15.12:0/13373874 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff440054f0 con 0x559d6cb3fa00 2024-04-22T13:31:22.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.359+0000 7eff32ff5640 1 -- 172.21.15.12:0/13373874 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7eff44040040 con 0x559d6cb3fa00 2024-04-22T13:31:22.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.359+0000 7eff58005740 10 client.18267 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:22.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.359+0000 7eff58005740 10 client.18267 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:31:22.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.359+0000 7eff58005740 20 client.18267 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:22.361 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.359+0000 7eff58005740 1 --2- 172.21.15.12:0/13373874 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x559d6cbb6830 0x559d6cbb8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.359+0000 7eff58005740 4 client.18267 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:31:22.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.359+0000 7eff58005740 1 -- 172.21.15.12:0/13373874 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559d6caa8c10 con 0x559d6cbb6830 2024-04-22T13:31:22.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.359+0000 7eff4effd640 1 --2- 172.21.15.12:0/13373874 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x559d6cbb6830 0x559d6cbb8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.360+0000 7eff32ff5640 10 client.18267 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:31:22.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.360+0000 7eff4effd640 1 --2- 172.21.15.12:0/13373874 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x559d6cbb6830 0x559d6cbb8c20 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=335f21d407cffb46 server_cookie=1a2f09e723aaafee in_seq=0 out_seq=0 2024-04-22T13:31:22.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.360+0000 7eff32ff5640 1 -- 172.21.15.12:0/13373874 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559d6caa8c10 con 0x559d6cbb6830 2024-04-22T13:31:22.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.360+0000 7eff32ff5640 10 client.18267 handle_command_reply: tid=0 2024-04-22T13:31:22.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.360+0000 7eff58005740 1 client.18267 shutdown 2024-04-22T13:31:22.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.360+0000 7eff58005740 1 -- 172.21.15.12:0/13373874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d6cb3fa00 msgr2=0x559d6cb3a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.360+0000 7eff58005740 1 --2- 172.21.15.12:0/13373874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d6cb3fa00 0x559d6cb3a0c0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7eff44059ab0 tx=0x7eff440597d0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.360+0000 7eff58005740 1 -- 172.21.15.12:0/13373874 shutdown_connections 2024-04-22T13:31:22.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.360+0000 7eff58005740 1 --2- 172.21.15.12:0/13373874 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x559d6cbb6830 0x559d6cbb8c20 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.360+0000 7eff58005740 1 --2- 172.21.15.12:0/13373874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d6cb3fa00 0x559d6cb3a0c0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.362 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.360+0000 7eff32ff5640 0 client.18267 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:31:22.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.360+0000 7eff58005740 1 --2- 172.21.15.12:0/13373874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d6cb36e80 0x559d6cb39b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.360+0000 7eff58005740 1 --2- 172.21.15.12:0/13373874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d6cb364d0 0x559d6cb39640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.360+0000 7eff58005740 1 -- 172.21.15.12:0/13373874 >> 172.21.15.12:0/13373874 conn(0x559d6cb30ba0 msgr2=0x559d6cb37360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:22.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.360+0000 7eff58005740 1 -- 172.21.15.12:0/13373874 shutdown_connections 2024-04-22T13:31:22.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.360+0000 7eff58005740 1 -- 172.21.15.12:0/13373874 wait complete. 2024-04-22T13:31:22.363 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.360+0000 7eff58005740 20 client.18267 trim_cache size 0 max 16384 2024-04-22T13:31:22.363 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:31:22.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.361+0000 7eff5533b640 1 -- 172.21.15.12:0/3727717514 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7eff380a3d20 msgr2=0x7eff380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.361+0000 7eff5533b640 1 --2- 172.21.15.12:0/3727717514 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7eff380a3d20 0x7eff380a61e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7eff48018000 tx=0x7eff4806f350 comp rx=0 tx=0).stop 2024-04-22T13:31:22.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.361+0000 7eff5533b640 1 -- 172.21.15.12:0/3727717514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff5019a790 msgr2=0x7eff5019ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.361+0000 7eff5533b640 1 --2- 172.21.15.12:0/3727717514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff5019a790 0x7eff5019ac40 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7eff4800eaa0 tx=0x7eff4800eea0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.364 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.361+0000 7eff5533b640 1 -- 172.21.15.12:0/3727717514 shutdown_connections 2024-04-22T13:31:22.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.361+0000 7eff5533b640 1 --2- 172.21.15.12:0/3727717514 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7eff380a3d20 0x7eff380a61e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.361+0000 7eff5533b640 1 --2- 172.21.15.12:0/3727717514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7eff5019a790 0x7eff5019ac40 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.361+0000 7eff5533b640 1 --2- 172.21.15.12:0/3727717514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff50106170 0x7eff501a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.361+0000 7eff5533b640 1 --2- 172.21.15.12:0/3727717514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff501010a0 0x7eff501a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.361+0000 7eff5533b640 1 -- 172.21.15.12:0/3727717514 >> 172.21.15.12:0/3727717514 conn(0x7eff50077eb0 msgr2=0x7eff5010c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:22.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.361+0000 7eff5533b640 1 -- 172.21.15.12:0/3727717514 shutdown_connections 2024-04-22T13:31:22.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.361+0000 7eff5533b640 1 -- 172.21.15.12:0/3727717514 wait complete. 2024-04-22T13:31:22.367 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:31:22.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.476+0000 7f7ba3fd5640 1 -- 172.21.15.12:0/2244121517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b9c107270 msgr2=0x7f7b9c10ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.476+0000 7f7ba3fd5640 1 --2- 172.21.15.12:0/2244121517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b9c107270 0x7f7b9c10ee40 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009ca0 tx=0x7f7b9402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.477+0000 7f7ba3fd5640 1 -- 172.21.15.12:0/2244121517 shutdown_connections 2024-04-22T13:31:22.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.477+0000 7f7ba3fd5640 1 --2- 172.21.15.12:0/2244121517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b9c107270 0x7f7b9c10ee40 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.477+0000 7f7ba3fd5640 1 --2- 172.21.15.12:0/2244121517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b9c06b1f0 0x7f7b9c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.477+0000 7f7ba3fd5640 1 --2- 172.21.15.12:0/2244121517 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b9c06a840 0x7f7b9c06ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.477+0000 7f7ba3fd5640 1 -- 172.21.15.12:0/2244121517 >> 172.21.15.12:0/2244121517 conn(0x7f7b9c077eb0 msgr2=0x7f7b9c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:22.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.477+0000 7f7ba3fd5640 1 -- 172.21.15.12:0/2244121517 shutdown_connections 2024-04-22T13:31:22.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.477+0000 7f7ba3fd5640 1 -- 172.21.15.12:0/2244121517 wait complete. 2024-04-22T13:31:22.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.477+0000 7f7ba3fd5640 1 Processor -- start 2024-04-22T13:31:22.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.477+0000 7f7ba3fd5640 1 -- start start 2024-04-22T13:31:22.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.477+0000 7f7ba3fd5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b9c06a840 0x7f7b9c198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.477+0000 7f7ba3fd5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b9c107270 0x7f7b9c198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.477+0000 7f7ba3fd5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b9c19cf50 0x7f7b9c19d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.477+0000 7f7ba3fd5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c1121b0 con 0x7f7b9c107270 2024-04-22T13:31:22.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.477+0000 7f7ba3fd5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c112030 con 0x7f7b9c19cf50 2024-04-22T13:31:22.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.477+0000 7f7ba3fd5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c112330 con 0x7f7b9c06a840 2024-04-22T13:31:22.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.478+0000 7f7ba1570640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b9c107270 0x7f7b9c198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.478+0000 7f7ba1570640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b9c107270 0x7f7b9c198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44750/0 (socket says 172.21.15.12:44750) 2024-04-22T13:31:22.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.478+0000 7f7ba1570640 1 -- 172.21.15.12:0/2223696220 learned_addr learned my addr 172.21.15.12:0/2223696220 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:22.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.478+0000 7f7ba1d71640 1 --2- 172.21.15.12:0/2223696220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b9c06a840 0x7f7b9c198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.478+0000 7f7ba1570640 1 -- 172.21.15.12:0/2223696220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b9c06a840 msgr2=0x7f7b9c198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.478+0000 7f7ba1570640 1 --2- 172.21.15.12:0/2223696220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b9c06a840 0x7f7b9c198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.478+0000 7f7ba1570640 1 -- 172.21.15.12:0/2223696220 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b9c19cf50 msgr2=0x7f7b9c19d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.478+0000 7f7ba2572640 1 --2- 172.21.15.12:0/2223696220 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b9c19cf50 0x7f7b9c19d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.478+0000 7f7ba1570640 1 --2- 172.21.15.12:0/2223696220 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b9c19cf50 0x7f7b9c19d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.478+0000 7f7ba1570640 1 -- 172.21.15.12:0/2223696220 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9c19da40 con 0x7f7b9c107270 2024-04-22T13:31:22.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.478+0000 7f7ba1570640 1 --2- 172.21.15.12:0/2223696220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b9c107270 0x7f7b9c198bd0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00c970 tx=0x7f7b8c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:22.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.478+0000 7f7b92ffd640 1 -- 172.21.15.12:0/2223696220 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c014070 con 0x7f7b9c107270 2024-04-22T13:31:22.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.478+0000 7f7b92ffd640 1 -- 172.21.15.12:0/2223696220 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b8c004440 con 0x7f7b9c107270 2024-04-22T13:31:22.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.478+0000 7f7ba3fd5640 1 -- 172.21.15.12:0/2223696220 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b9c19dd30 con 0x7f7b9c107270 2024-04-22T13:31:22.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.478+0000 7f7b92ffd640 1 -- 172.21.15.12:0/2223696220 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c003e50 con 0x7f7b9c107270 2024-04-22T13:31:22.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.478+0000 7f7ba3fd5640 1 -- 172.21.15.12:0/2223696220 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b9c1a5570 con 0x7f7b9c107270 2024-04-22T13:31:22.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.481+0000 7f7b92ffd640 1 -- 172.21.15.12:0/2223696220 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7b8c017070 con 0x7f7b9c107270 2024-04-22T13:31:22.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.481+0000 7f7ba6bc8740 1 Processor -- start 2024-04-22T13:31:22.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.482+0000 7f7ba6bc8740 1 -- start start 2024-04-22T13:31:22.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.482+0000 7f7ba6bc8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574b3345250 0x5574b3345630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.482+0000 7f7ba6bc8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5574b3345c00 0x5574b3352840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.482+0000 7f7ba6bc8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574b3352e10 0x5574b3355220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.482+0000 7f7ba6bc8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5574b32aa1f0 con 0x5574b3352e10 2024-04-22T13:31:22.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.482+0000 7f7ba6bc8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5574b32474b0 con 0x5574b3345c00 2024-04-22T13:31:22.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.482+0000 7f7ba6bc8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5574b32a61b0 con 0x5574b3345250 2024-04-22T13:31:22.482 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.482+0000 7f7ba1570640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574b3352e10 0x5574b3355220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.482+0000 7f7ba2572640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574b3345250 0x5574b3345630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.482+0000 7f7ba1570640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574b3352e10 0x5574b3355220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44764/0 (socket says 172.21.15.12:44764) 2024-04-22T13:31:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.482+0000 7f7ba1d71640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5574b3345c00 0x5574b3352840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.482+0000 7f7ba1570640 1 -- 172.21.15.12:0/4084099337 learned_addr learned my addr 172.21.15.12:0/4084099337 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.482+0000 7f7b92ffd640 1 --2- 172.21.15.12:0/2223696220 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7b880a3f40 0x7f7b880a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.482+0000 7f7ba2572640 1 --2- 172.21.15.12:0/2223696220 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7b880a3f40 0x7f7b880a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.482+0000 7f7b92ffd640 1 -- 172.21.15.12:0/2223696220 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f7b8c0d0350 con 0x7f7b9c107270 2024-04-22T13:31:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.483+0000 7f7ba1570640 1 -- 172.21.15.12:0/4084099337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574b3345250 msgr2=0x5574b3345630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.483+0000 7f7ba1570640 1 --2- 172.21.15.12:0/4084099337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574b3345250 0x5574b3345630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.483+0000 7f7ba1570640 1 -- 172.21.15.12:0/4084099337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5574b3345c00 msgr2=0x5574b3352840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.483+0000 7f7ba1570640 1 --2- 172.21.15.12:0/4084099337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5574b3345c00 0x5574b3352840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.483+0000 7f7ba1570640 1 -- 172.21.15.12:0/4084099337 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574b32391b0 con 0x5574b3352e10 2024-04-22T13:31:22.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.483+0000 7f7ba1570640 1 --2- 172.21.15.12:0/4084099337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574b3352e10 0x5574b3355220 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0a4a50 tx=0x7f7b8c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f82c583e6e202e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.483+0000 7f7ba2572640 1 --2- 172.21.15.12:0/2223696220 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7b880a3f40 0x7f7b880a6400 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009430 tx=0x7f7b9402c0f0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.483+0000 7f7b87fff640 1 -- 172.21.15.12:0/4084099337 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c09ca10 con 0x5574b3352e10 2024-04-22T13:31:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.483+0000 7f7b87fff640 1 -- 172.21.15.12:0/4084099337 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b8c00de20 con 0x5574b3352e10 2024-04-22T13:31:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.483+0000 7f7b87fff640 1 -- 172.21.15.12:0/4084099337 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c0d3cf0 con 0x5574b3352e10 2024-04-22T13:31:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.484+0000 7f7ba6bc8740 1 -- 172.21.15.12:0/4084099337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574b3352e10 msgr2=0x5574b3355220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.484+0000 7f7ba6bc8740 1 --2- 172.21.15.12:0/4084099337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574b3352e10 0x5574b3355220 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0a4a50 tx=0x7f7b8c000bc0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.484+0000 7f7ba6bc8740 1 -- 172.21.15.12:0/4084099337 shutdown_connections 2024-04-22T13:31:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.484+0000 7f7ba6bc8740 1 --2- 172.21.15.12:0/4084099337 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574b3352e10 0x5574b3355220 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.484+0000 7f7ba6bc8740 1 --2- 172.21.15.12:0/4084099337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5574b3345c00 0x5574b3352840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.484+0000 7f7ba6bc8740 1 --2- 172.21.15.12:0/4084099337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574b3345250 0x5574b3345630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.484+0000 7f7ba6bc8740 1 -- 172.21.15.12:0/4084099337 >> 172.21.15.12:0/4084099337 conn(0x5574b333fba0 msgr2=0x5574b333ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.484+0000 7f7ba6bc8740 1 -- 172.21.15.12:0/4084099337 shutdown_connections 2024-04-22T13:31:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.484+0000 7f7ba6bc8740 1 -- 172.21.15.12:0/4084099337 wait complete. 2024-04-22T13:31:22.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.484+0000 7f7ba6bc8740 1 -- start start 2024-04-22T13:31:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.484+0000 7f7ba6bc8740 1 Processor -- start 2024-04-22T13:31:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.485+0000 7f7ba6bc8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574b3345250 0x5574b3352700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.485+0000 7f7ba6bc8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5574b3345c00 0x5574b334fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.485+0000 7f7ba6bc8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574b3352e10 0x5574b3350010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.485+0000 7f7ba6bc8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5574b3355ce0 con 0x5574b3352e10 2024-04-22T13:31:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.485+0000 7f7ba6bc8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5574b3263500 con 0x5574b3345c00 2024-04-22T13:31:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.485+0000 7f7ba6bc8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5574b3355e60 con 0x5574b3345250 2024-04-22T13:31:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.485+0000 7f7ba6bc8740 10 client.0 _ll_register_callbacks cb 0x5574b3231b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.485+0000 7f7ba2572640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574b3352e10 0x5574b3350010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.485+0000 7f7ba2572640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574b3352e10 0x5574b3350010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44776/0 (socket says 172.21.15.12:44776) 2024-04-22T13:31:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.485+0000 7f7ba2572640 1 -- 172.21.15.12:0/736893385 learned_addr learned my addr 172.21.15.12:0/736893385 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.485+0000 7f7ba1d71640 1 --2- 172.21.15.12:0/736893385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574b3345250 0x5574b3352700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.485 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.485+0000 7f7ba1570640 1 --2- 172.21.15.12:0/736893385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5574b3345c00 0x5574b334fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.485+0000 7f7ba2572640 1 -- 172.21.15.12:0/736893385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574b3345250 msgr2=0x5574b3352700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.485+0000 7f7ba2572640 1 --2- 172.21.15.12:0/736893385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574b3345250 0x5574b3352700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.485+0000 7f7ba2572640 1 -- 172.21.15.12:0/736893385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5574b3345c00 msgr2=0x5574b334fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.485+0000 7f7ba2572640 1 --2- 172.21.15.12:0/736893385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5574b3345c00 0x5574b334fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.485+0000 7f7ba2572640 1 -- 172.21.15.12:0/736893385 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574b2fbc3f0 con 0x5574b3352e10 2024-04-22T13:31:22.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.485+0000 7f7ba2572640 1 --2- 172.21.15.12:0/736893385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574b3352e10 0x5574b3350010 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009ca0 tx=0x7f7b94005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1f239544394e75 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:22.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.485+0000 7f7b857fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:22.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.485+0000 7f7b857fa640 1 -- 172.21.15.12:0/736893385 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b94051020 con 0x5574b3352e10 2024-04-22T13:31:22.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.485+0000 7f7b857fa640 1 -- 172.21.15.12:0/736893385 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f7b94003cb0 con 0x5574b3352e10 2024-04-22T13:31:22.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.485+0000 7f7ba6bc8740 1 -- 172.21.15.12:0/736893385 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b8c00de20 con 0x5574b3352e10 2024-04-22T13:31:22.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.486+0000 7f7b857fa640 1 -- 172.21.15.12:0/736893385 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7b9405c050 con 0x5574b3352e10 2024-04-22T13:31:22.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.486+0000 7f7ba6bc8740 10 client.18279 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:22.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.486+0000 7f7ba6bc8740 1 -- 172.21.15.12:0/736893385 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5574b30867d0 con 0x5574b3352e10 2024-04-22T13:31:22.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.486+0000 7f7b857fa640 1 -- 172.21.15.12:0/736893385 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b8c00de20 con 0x5574b3352e10 2024-04-22T13:31:22.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.486+0000 7f7b857fa640 1 -- 172.21.15.12:0/736893385 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7b9404c030 con 0x5574b3352e10 2024-04-22T13:31:22.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.486+0000 7f7ba6bc8740 10 client.18279 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:22.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.486+0000 7f7ba6bc8740 10 client.18279 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:31:22.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.486+0000 7f7ba6bc8740 20 client.18279 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:22.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.486+0000 7f7ba6bc8740 1 --2- 172.21.15.12:0/736893385 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5574b3363070 0x5574b33654d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.486+0000 7f7ba6bc8740 4 client.18279 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:22.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.486+0000 7f7ba6bc8740 1 -- 172.21.15.12:0/736893385 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574b3249930 con 0x5574b3363070 2024-04-22T13:31:22.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.486+0000 7f7ba1d71640 1 --2- 172.21.15.12:0/736893385 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5574b3363070 0x5574b33654d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.487+0000 7f7b857fa640 10 client.18279 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:31:22.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.487+0000 7f7ba1d71640 1 --2- 172.21.15.12:0/736893385 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5574b3363070 0x5574b33654d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b0a4d790e9f8454c server_cookie=11f69aa6e26c7d5a in_seq=0 out_seq=0 2024-04-22T13:31:22.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.487+0000 7f7b857fa640 1 -- 172.21.15.12:0/736893385 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5574b3249930 con 0x5574b3363070 2024-04-22T13:31:22.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.487+0000 7f7b857fa640 10 client.18279 handle_command_reply: tid=0 2024-04-22T13:31:22.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.487+0000 7f7ba6bc8740 1 client.18279 shutdown 2024-04-22T13:31:22.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.487+0000 7f7ba6bc8740 1 -- 172.21.15.12:0/736893385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574b3352e10 msgr2=0x5574b3350010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.487+0000 7f7ba6bc8740 1 --2- 172.21.15.12:0/736893385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574b3352e10 0x5574b3350010 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009ca0 tx=0x7f7b94005dc0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.488+0000 7f7ba6bc8740 1 -- 172.21.15.12:0/736893385 shutdown_connections 2024-04-22T13:31:22.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.488+0000 7f7ba6bc8740 1 --2- 172.21.15.12:0/736893385 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5574b3363070 0x5574b33654d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.488+0000 7f7ba6bc8740 1 --2- 172.21.15.12:0/736893385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574b3352e10 0x5574b3350010 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.488+0000 7f7ba6bc8740 1 --2- 172.21.15.12:0/736893385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5574b3345c00 0x5574b334fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.488+0000 7f7b857fa640 0 client.18279 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:31:22.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.488+0000 7f7ba6bc8740 1 --2- 172.21.15.12:0/736893385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574b3345250 0x5574b3352700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.488+0000 7f7ba6bc8740 1 -- 172.21.15.12:0/736893385 >> 172.21.15.12:0/736893385 conn(0x5574b333fba0 msgr2=0x5574b33418f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:22.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.488+0000 7f7ba6bc8740 1 -- 172.21.15.12:0/736893385 shutdown_connections 2024-04-22T13:31:22.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.488+0000 7f7ba6bc8740 1 -- 172.21.15.12:0/736893385 wait complete. 2024-04-22T13:31:22.490 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.488+0000 7f7ba6bc8740 20 client.18279 trim_cache size 0 max 16384 2024-04-22T13:31:22.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.505+0000 7f7ba6bc8740 1 Processor -- start 2024-04-22T13:31:22.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.505+0000 7f7ba6bc8740 1 -- start start 2024-04-22T13:31:22.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.505+0000 7f7ba6bc8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574b3345250 0x5574b335ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.505+0000 7f7ba6bc8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5574b3345c00 0x5574b33521c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.505+0000 7f7ba6bc8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574b3352e10 0x5574b3352700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.505+0000 7f7ba6bc8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c1ab870 con 0x5574b3345250 2024-04-22T13:31:22.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.505+0000 7f7ba6bc8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c1ad3b0 con 0x5574b3345c00 2024-04-22T13:31:22.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.505+0000 7f7ba6bc8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c1abf40 con 0x5574b3352e10 2024-04-22T13:31:22.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.505+0000 7f7ba1d71640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574b3345250 0x5574b335ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.505+0000 7f7ba1d71640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574b3345250 0x5574b335ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44780/0 (socket says 172.21.15.12:44780) 2024-04-22T13:31:22.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.505+0000 7f7ba1d71640 1 -- 172.21.15.12:0/182757346 learned_addr learned my addr 172.21.15.12:0/182757346 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:22.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.506+0000 7f7ba1d71640 1 -- 172.21.15.12:0/182757346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574b3352e10 msgr2=0x5574b3352700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.506+0000 7f7ba2572640 1 --2- 172.21.15.12:0/182757346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574b3352e10 0x5574b3352700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.506+0000 7f7ba1570640 1 --2- 172.21.15.12:0/182757346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5574b3345c00 0x5574b33521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.506+0000 7f7ba1d71640 1 --2- 172.21.15.12:0/182757346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574b3352e10 0x5574b3352700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.506+0000 7f7ba1d71640 1 -- 172.21.15.12:0/182757346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5574b3345c00 msgr2=0x5574b33521c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.506+0000 7f7ba1d71640 1 --2- 172.21.15.12:0/182757346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5574b3345c00 0x5574b33521c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.506+0000 7f7ba1d71640 1 -- 172.21.15.12:0/182757346 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574b3344560 con 0x5574b3345250 2024-04-22T13:31:22.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.506+0000 7f7ba2572640 1 --2- 172.21.15.12:0/182757346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574b3352e10 0x5574b3352700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:22.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.506+0000 7f7ba1570640 1 --2- 172.21.15.12:0/182757346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5574b3345c00 0x5574b33521c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:22.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.506+0000 7f7ba1d71640 1 --2- 172.21.15.12:0/182757346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574b3345250 0x5574b335ec10 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f7b98000c00 tx=0x7f7b980118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f882c55ac0ce6e5a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:22.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.506+0000 7f7b877fe640 1 -- 172.21.15.12:0/182757346 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9801d6e0 con 0x5574b3345250 2024-04-22T13:31:22.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.506+0000 7f7b877fe640 1 -- 172.21.15.12:0/182757346 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b9801d880 con 0x5574b3345250 2024-04-22T13:31:22.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.506+0000 7f7b877fe640 1 -- 172.21.15.12:0/182757346 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9801c3a0 con 0x5574b3345250 2024-04-22T13:31:22.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.507+0000 7f7ba6bc8740 1 -- 172.21.15.12:0/182757346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574b3345250 msgr2=0x5574b335ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.507+0000 7f7ba6bc8740 1 --2- 172.21.15.12:0/182757346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574b3345250 0x5574b335ec10 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f7b98000c00 tx=0x7f7b980118d0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.507+0000 7f7ba6bc8740 1 -- 172.21.15.12:0/182757346 shutdown_connections 2024-04-22T13:31:22.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.507+0000 7f7ba6bc8740 1 --2- 172.21.15.12:0/182757346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574b3352e10 0x5574b3352700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.507+0000 7f7ba6bc8740 1 --2- 172.21.15.12:0/182757346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5574b3345c00 0x5574b33521c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.507+0000 7f7ba6bc8740 1 --2- 172.21.15.12:0/182757346 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574b3345250 0x5574b335ec10 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.507+0000 7f7ba6bc8740 1 -- 172.21.15.12:0/182757346 >> 172.21.15.12:0/182757346 conn(0x5574b333fba0 msgr2=0x5574b3341880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:22.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.507+0000 7f7ba6bc8740 1 -- 172.21.15.12:0/182757346 shutdown_connections 2024-04-22T13:31:22.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.507+0000 7f7ba6bc8740 1 -- 172.21.15.12:0/182757346 wait complete. 2024-04-22T13:31:22.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.507+0000 7f7ba6bc8740 1 -- start start 2024-04-22T13:31:22.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.507+0000 7f7ba6bc8740 1 Processor -- start 2024-04-22T13:31:22.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.508+0000 7f7ba6bc8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5574b3345250 0x5574b334cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.508+0000 7f7ba6bc8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574b3345c00 0x5574b334d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.508+0000 7f7ba6bc8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574b3352e10 0x5574b33c8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.508+0000 7f7ba6bc8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c1acce0 con 0x5574b3345c00 2024-04-22T13:31:22.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.508+0000 7f7ba6bc8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c1ac610 con 0x5574b3345250 2024-04-22T13:31:22.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.508+0000 7f7ba6bc8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c1a9870 con 0x5574b3352e10 2024-04-22T13:31:22.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.508+0000 7f7ba6bc8740 10 client.0 _ll_register_callbacks cb 0x7f7b940328a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:22.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.508+0000 7f7ba1570640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574b3345c00 0x5574b334d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.508+0000 7f7ba1570640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574b3345c00 0x5574b334d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44796/0 (socket says 172.21.15.12:44796) 2024-04-22T13:31:22.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.508+0000 7f7ba1570640 1 -- 172.21.15.12:0/3333396813 learned_addr learned my addr 172.21.15.12:0/3333396813 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:22.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.508+0000 7f7ba1d71640 1 --2- 172.21.15.12:0/3333396813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5574b3345250 0x5574b334cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.508+0000 7f7ba2572640 1 --2- 172.21.15.12:0/3333396813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574b3352e10 0x5574b33c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.508+0000 7f7ba1570640 1 -- 172.21.15.12:0/3333396813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574b3352e10 msgr2=0x5574b33c8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.508+0000 7f7ba1570640 1 --2- 172.21.15.12:0/3333396813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574b3352e10 0x5574b33c8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.508+0000 7f7ba1570640 1 -- 172.21.15.12:0/3333396813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5574b3345250 msgr2=0x5574b334cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.508+0000 7f7ba1570640 1 --2- 172.21.15.12:0/3333396813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5574b3345250 0x5574b334cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.508+0000 7f7ba1570640 1 -- 172.21.15.12:0/3333396813 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574b33c8b50 con 0x5574b3345c00 2024-04-22T13:31:22.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.508+0000 7f7ba2572640 1 --2- 172.21.15.12:0/3333396813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574b3352e10 0x5574b33c8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:22.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.508+0000 7f7ba1570640 1 --2- 172.21.15.12:0/3333396813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574b3345c00 0x5574b334d0a0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c098770 tx=0x7f7b8c0d06c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d553e7cb99e18ea2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:22.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.508+0000 7f7b857fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:22.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.508+0000 7f7b857fa640 1 -- 172.21.15.12:0/3333396813 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c0b5060 con 0x5574b3345c00 2024-04-22T13:31:22.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.508+0000 7f7b857fa640 1 -- 172.21.15.12:0/3333396813 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f7b8c09b760 con 0x5574b3345c00 2024-04-22T13:31:22.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.508+0000 7f7ba6bc8740 1 -- 172.21.15.12:0/3333396813 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b9801d880 con 0x5574b3345c00 2024-04-22T13:31:22.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.509+0000 7f7b857fa640 1 -- 172.21.15.12:0/3333396813 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7b8c09bc70 con 0x5574b3345c00 2024-04-22T13:31:22.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.509+0000 7f7ba6bc8740 10 client.18285 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:22.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.509+0000 7f7ba6bc8740 1 -- 172.21.15.12:0/3333396813 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5574b33c9c80 con 0x5574b3345c00 2024-04-22T13:31:22.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.509+0000 7f7b857fa640 1 -- 172.21.15.12:0/3333396813 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b9801d880 con 0x5574b3345c00 2024-04-22T13:31:22.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.509+0000 7f7b857fa640 1 -- 172.21.15.12:0/3333396813 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7b8c098060 con 0x5574b3345c00 2024-04-22T13:31:22.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.509+0000 7f7ba6bc8740 10 client.18285 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:22.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.509+0000 7f7ba6bc8740 10 client.18285 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:31:22.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.509+0000 7f7ba6bc8740 20 client.18285 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:22.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.509+0000 7f7ba6bc8740 1 --2- 172.21.15.12:0/3333396813 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5574b33c9fc0 0x5574b33cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.509+0000 7f7ba6bc8740 4 client.18285 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:31:22.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.509+0000 7f7ba6bc8740 1 -- 172.21.15.12:0/3333396813 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5574b32b7c10 con 0x5574b33c9fc0 2024-04-22T13:31:22.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.509+0000 7f7ba1d71640 1 --2- 172.21.15.12:0/3333396813 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5574b33c9fc0 0x5574b33cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.510+0000 7f7ba1d71640 1 --2- 172.21.15.12:0/3333396813 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5574b33c9fc0 0x5574b33cc3b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8db744343ee2aff0 server_cookie=1540dccb55bc4cc7 in_seq=0 out_seq=0 2024-04-22T13:31:22.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.510+0000 7f7b857fa640 10 client.18285 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:31:22.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.510+0000 7f7b857fa640 1 -- 172.21.15.12:0/3333396813 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5574b32b7c10 con 0x5574b33c9fc0 2024-04-22T13:31:22.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.510+0000 7f7b857fa640 10 client.18285 handle_command_reply: tid=0 2024-04-22T13:31:22.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.510+0000 7f7ba6bc8740 1 client.18285 shutdown 2024-04-22T13:31:22.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.510+0000 7f7ba6bc8740 1 -- 172.21.15.12:0/3333396813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574b3345c00 msgr2=0x5574b334d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.510+0000 7f7ba6bc8740 1 --2- 172.21.15.12:0/3333396813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574b3345c00 0x5574b334d0a0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c098770 tx=0x7f7b8c0d06c0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.510+0000 7f7ba6bc8740 1 -- 172.21.15.12:0/3333396813 shutdown_connections 2024-04-22T13:31:22.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.510+0000 7f7ba6bc8740 1 --2- 172.21.15.12:0/3333396813 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5574b33c9fc0 0x5574b33cc3b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.510+0000 7f7ba6bc8740 1 --2- 172.21.15.12:0/3333396813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5574b3352e10 0x5574b33c8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.510+0000 7f7ba6bc8740 1 --2- 172.21.15.12:0/3333396813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5574b3345c00 0x5574b334d0a0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.510+0000 7f7b857fa640 0 client.18285 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:31:22.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.510+0000 7f7ba6bc8740 1 --2- 172.21.15.12:0/3333396813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5574b3345250 0x5574b334cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.510+0000 7f7ba6bc8740 1 -- 172.21.15.12:0/3333396813 >> 172.21.15.12:0/3333396813 conn(0x5574b333fba0 msgr2=0x5574b334a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:22.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.510+0000 7f7ba6bc8740 1 -- 172.21.15.12:0/3333396813 shutdown_connections 2024-04-22T13:31:22.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.510+0000 7f7ba6bc8740 1 -- 172.21.15.12:0/3333396813 wait complete. 2024-04-22T13:31:22.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.510+0000 7f7ba6bc8740 20 client.18285 trim_cache size 0 max 16384 2024-04-22T13:31:22.513 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:31:22.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.511+0000 7f7ba3fd5640 1 -- 172.21.15.12:0/2223696220 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7b880a3f40 msgr2=0x7f7b880a6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.511+0000 7f7ba3fd5640 1 --2- 172.21.15.12:0/2223696220 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7b880a3f40 0x7f7b880a6400 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009430 tx=0x7f7b9402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.511+0000 7f7ba3fd5640 1 -- 172.21.15.12:0/2223696220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b9c107270 msgr2=0x7f7b9c198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.511+0000 7f7ba3fd5640 1 --2- 172.21.15.12:0/2223696220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b9c107270 0x7f7b9c198bd0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00c970 tx=0x7f7b8c00ce30 comp rx=0 tx=0).stop 2024-04-22T13:31:22.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.511+0000 7f7ba3fd5640 1 -- 172.21.15.12:0/2223696220 shutdown_connections 2024-04-22T13:31:22.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.511+0000 7f7ba3fd5640 1 --2- 172.21.15.12:0/2223696220 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7b880a3f40 0x7f7b880a6400 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.511+0000 7f7ba3fd5640 1 --2- 172.21.15.12:0/2223696220 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b9c19cf50 0x7f7b9c19d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.511+0000 7f7ba3fd5640 1 --2- 172.21.15.12:0/2223696220 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b9c107270 0x7f7b9c198bd0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.511+0000 7f7ba3fd5640 1 --2- 172.21.15.12:0/2223696220 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b9c06a840 0x7f7b9c198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.511+0000 7f7ba3fd5640 1 -- 172.21.15.12:0/2223696220 >> 172.21.15.12:0/2223696220 conn(0x7f7b9c077eb0 msgr2=0x7f7b9c0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:22.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.511+0000 7f7ba3fd5640 1 -- 172.21.15.12:0/2223696220 shutdown_connections 2024-04-22T13:31:22.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.511+0000 7f7ba3fd5640 1 -- 172.21.15.12:0/2223696220 wait complete. 2024-04-22T13:31:22.517 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:31:22.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.626+0000 7f1a31b96640 1 -- 172.21.15.12:0/2471253069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a2c101090 msgr2=0x7f1a2c101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.626+0000 7f1a31b96640 1 --2- 172.21.15.12:0/2471253069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a2c101090 0x7f1a2c101470 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009900 tx=0x7f1a1c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:31:22.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.627+0000 7f1a31b96640 1 -- 172.21.15.12:0/2471253069 shutdown_connections 2024-04-22T13:31:22.625 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.627+0000 7f1a31b96640 1 --2- 172.21.15.12:0/2471253069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a2c106160 0x7f1a2c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.627+0000 7f1a31b96640 1 --2- 172.21.15.12:0/2471253069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a2c101a40 0x7f1a2c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.627+0000 7f1a31b96640 1 --2- 172.21.15.12:0/2471253069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a2c101090 0x7f1a2c101470 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.627+0000 7f1a31b96640 1 -- 172.21.15.12:0/2471253069 >> 172.21.15.12:0/2471253069 conn(0x7f1a2c077f50 msgr2=0x7f1a2c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:22.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.627+0000 7f1a31b96640 1 -- 172.21.15.12:0/2471253069 shutdown_connections 2024-04-22T13:31:22.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.627+0000 7f1a31b96640 1 -- 172.21.15.12:0/2471253069 wait complete. 2024-04-22T13:31:22.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.627+0000 7f1a31b96640 1 Processor -- start 2024-04-22T13:31:22.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.627+0000 7f1a31b96640 1 -- start start 2024-04-22T13:31:22.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.628+0000 7f1a31b96640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a2c101090 0x7f1a2c19e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.628+0000 7f1a31b96640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a2c106160 0x7f1a2c19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.628+0000 7f1a31b96640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a2c1985b0 0x7f1a2c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.628+0000 7f1a31b96640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c1110c0 con 0x7f1a2c106160 2024-04-22T13:31:22.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.628+0000 7f1a31b96640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c110f40 con 0x7f1a2c101090 2024-04-22T13:31:22.626 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.628+0000 7f1a31b96640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c111240 con 0x7f1a2c1985b0 2024-04-22T13:31:22.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.628+0000 7f1a2affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a2c106160 0x7f1a2c19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.628+0000 7f1a2affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a2c106160 0x7f1a2c19e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44820/0 (socket says 172.21.15.12:44820) 2024-04-22T13:31:22.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.628+0000 7f1a2affd640 1 -- 172.21.15.12:0/4018277904 learned_addr learned my addr 172.21.15.12:0/4018277904 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:22.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.628+0000 7f1a2bfff640 1 --2- 172.21.15.12:0/4018277904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a2c1985b0 0x7f1a2c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.628+0000 7f1a2b7fe640 1 --2- 172.21.15.12:0/4018277904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a2c101090 0x7f1a2c19e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.628+0000 7f1a2affd640 1 -- 172.21.15.12:0/4018277904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a2c1985b0 msgr2=0x7f1a2c198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.628+0000 7f1a2affd640 1 --2- 172.21.15.12:0/4018277904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a2c1985b0 0x7f1a2c198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.628+0000 7f1a2affd640 1 -- 172.21.15.12:0/4018277904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a2c101090 msgr2=0x7f1a2c19e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.628+0000 7f1a2affd640 1 --2- 172.21.15.12:0/4018277904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a2c101090 0x7f1a2c19e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.628+0000 7f1a2affd640 1 -- 172.21.15.12:0/4018277904 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a2c1990d0 con 0x7f1a2c106160 2024-04-22T13:31:22.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.628+0000 7f1a2affd640 1 --2- 172.21.15.12:0/4018277904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a2c106160 0x7f1a2c19e940 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000d740 tx=0x7f1a2000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:22.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.628+0000 7f1a28ff9640 1 -- 172.21.15.12:0/4018277904 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20017070 con 0x7f1a2c106160 2024-04-22T13:31:22.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.628+0000 7f1a31b96640 1 -- 172.21.15.12:0/4018277904 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a2c1993c0 con 0x7f1a2c106160 2024-04-22T13:31:22.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.628+0000 7f1a28ff9640 1 -- 172.21.15.12:0/4018277904 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a2000de50 con 0x7f1a2c106160 2024-04-22T13:31:22.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.628+0000 7f1a28ff9640 1 -- 172.21.15.12:0/4018277904 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20004ce0 con 0x7f1a2c106160 2024-04-22T13:31:22.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.628+0000 7f1a31b96640 1 -- 172.21.15.12:0/4018277904 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a2c1a5610 con 0x7f1a2c106160 2024-04-22T13:31:22.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.631+0000 7f1a28ff9640 1 -- 172.21.15.12:0/4018277904 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1a20004e80 con 0x7f1a2c106160 2024-04-22T13:31:22.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.631+0000 7f1a34788740 1 Processor -- start 2024-04-22T13:31:22.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.631+0000 7f1a34788740 1 -- start start 2024-04-22T13:31:22.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.631+0000 7f1a34788740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604855394d0 0x5604855398b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.631+0000 7f1a34788740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560485539e80 0x560485542430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.631+0000 7f1a34788740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560485542a00 0x560485544e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.631+0000 7f1a34788740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56048549e1f0 con 0x560485542a00 2024-04-22T13:31:22.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.631+0000 7f1a34788740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56048543b4b0 con 0x5604855394d0 2024-04-22T13:31:22.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.631+0000 7f1a34788740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56048549a1b0 con 0x560485539e80 2024-04-22T13:31:22.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.632+0000 7f1a2affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560485542a00 0x560485544e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.632+0000 7f1a28ff9640 1 --2- 172.21.15.12:0/4018277904 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f1a140a3d20 0x7f1a140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.632+0000 7f1a2affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560485542a00 0x560485544e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44836/0 (socket says 172.21.15.12:44836) 2024-04-22T13:31:22.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.632+0000 7f1a2affd640 1 -- 172.21.15.12:0/3118111817 learned_addr learned my addr 172.21.15.12:0/3118111817 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:22.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.632+0000 7f1a2bfff640 1 --2- 172.21.15.12:0/3118111817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604855394d0 0x5604855398b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.632+0000 7f1a2b7fe640 1 --2- 172.21.15.12:0/3118111817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560485539e80 0x560485542430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.632+0000 7f1a2bfff640 1 --2- 172.21.15.12:0/4018277904 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f1a140a3d20 0x7f1a140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.632+0000 7f1a28ff9640 1 -- 172.21.15.12:0/4018277904 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f1a200d02a0 con 0x7f1a2c106160 2024-04-22T13:31:22.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.632+0000 7f1a2affd640 1 -- 172.21.15.12:0/3118111817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560485539e80 msgr2=0x560485542430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.632+0000 7f1a2affd640 1 --2- 172.21.15.12:0/3118111817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560485539e80 0x560485542430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.632+0000 7f1a2affd640 1 -- 172.21.15.12:0/3118111817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604855394d0 msgr2=0x5604855398b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.632+0000 7f1a2affd640 1 --2- 172.21.15.12:0/3118111817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604855394d0 0x5604855398b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.632+0000 7f1a2affd640 1 -- 172.21.15.12:0/3118111817 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56048542d1b0 con 0x560485542a00 2024-04-22T13:31:22.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.632+0000 7f1a2bfff640 1 --2- 172.21.15.12:0/3118111817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604855394d0 0x5604855398b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:22.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.632+0000 7f1a2affd640 1 --2- 172.21.15.12:0/3118111817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560485542a00 0x560485544e50 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f1a20098bd0 tx=0x7f1a2009c100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5dba318b5f65c15 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:22.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.632+0000 7f1a2bfff640 1 --2- 172.21.15.12:0/4018277904 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f1a140a3d20 0x7f1a140a61e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009370 tx=0x7f1a24009240 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:22.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.632+0000 7f1a11ffb640 1 -- 172.21.15.12:0/3118111817 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20020070 con 0x560485542a00 2024-04-22T13:31:22.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.632+0000 7f1a11ffb640 1 -- 172.21.15.12:0/3118111817 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a2009c900 con 0x560485542a00 2024-04-22T13:31:22.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.632+0000 7f1a11ffb640 1 -- 172.21.15.12:0/3118111817 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a200a49e0 con 0x560485542a00 2024-04-22T13:31:22.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.633+0000 7f1a34788740 1 -- 172.21.15.12:0/3118111817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560485542a00 msgr2=0x560485544e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.633+0000 7f1a34788740 1 --2- 172.21.15.12:0/3118111817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560485542a00 0x560485544e50 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f1a20098bd0 tx=0x7f1a2009c100 comp rx=0 tx=0).stop 2024-04-22T13:31:22.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.633+0000 7f1a34788740 1 -- 172.21.15.12:0/3118111817 shutdown_connections 2024-04-22T13:31:22.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.633+0000 7f1a34788740 1 --2- 172.21.15.12:0/3118111817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560485542a00 0x560485544e50 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.633+0000 7f1a34788740 1 --2- 172.21.15.12:0/3118111817 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560485539e80 0x560485542430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.633+0000 7f1a34788740 1 --2- 172.21.15.12:0/3118111817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604855394d0 0x5604855398b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.633+0000 7f1a34788740 1 -- 172.21.15.12:0/3118111817 >> 172.21.15.12:0/3118111817 conn(0x560485533ba0 msgr2=0x560485533f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:22.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.633+0000 7f1a34788740 1 -- 172.21.15.12:0/3118111817 shutdown_connections 2024-04-22T13:31:22.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.633+0000 7f1a34788740 1 -- 172.21.15.12:0/3118111817 wait complete. 2024-04-22T13:31:22.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.633+0000 7f1a34788740 1 -- start start 2024-04-22T13:31:22.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.634+0000 7f1a34788740 1 Processor -- start 2024-04-22T13:31:22.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.634+0000 7f1a34788740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604855394d0 0x560485542350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.634+0000 7f1a34788740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560485539e80 0x56048553f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.634+0000 7f1a34788740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560485542a00 0x56048553fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.634+0000 7f1a34788740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560485545910 con 0x560485542a00 2024-04-22T13:31:22.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.634+0000 7f1a34788740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560485457500 con 0x560485539e80 2024-04-22T13:31:22.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.634+0000 7f1a34788740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560485545a90 con 0x5604855394d0 2024-04-22T13:31:22.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.634+0000 7f1a34788740 10 client.0 _ll_register_callbacks cb 0x560485425b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:22.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.634+0000 7f1a2bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560485542a00 0x56048553fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.634+0000 7f1a2bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560485542a00 0x56048553fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44844/0 (socket says 172.21.15.12:44844) 2024-04-22T13:31:22.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.634+0000 7f1a2bfff640 1 -- 172.21.15.12:0/1268719496 learned_addr learned my addr 172.21.15.12:0/1268719496 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:22.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.634+0000 7f1a2affd640 1 --2- 172.21.15.12:0/1268719496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560485539e80 0x56048553f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.634+0000 7f1a2b7fe640 1 --2- 172.21.15.12:0/1268719496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604855394d0 0x560485542350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.634+0000 7f1a2bfff640 1 -- 172.21.15.12:0/1268719496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604855394d0 msgr2=0x560485542350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.634+0000 7f1a2bfff640 1 --2- 172.21.15.12:0/1268719496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604855394d0 0x560485542350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.634+0000 7f1a2bfff640 1 -- 172.21.15.12:0/1268719496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560485539e80 msgr2=0x56048553f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.634+0000 7f1a2bfff640 1 --2- 172.21.15.12:0/1268719496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560485539e80 0x56048553f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.634+0000 7f1a2bfff640 1 -- 172.21.15.12:0/1268719496 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604851b03f0 con 0x560485542a00 2024-04-22T13:31:22.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.635+0000 7f1a2bfff640 1 --2- 172.21.15.12:0/1268719496 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560485542a00 0x56048553fcf0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f1a24013c50 tx=0x7f1a24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62039324a955e2d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:22.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.635+0000 7f1a0affd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:22.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.635+0000 7f1a0affd640 1 -- 172.21.15.12:0/1268719496 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a24023020 con 0x560485542a00 2024-04-22T13:31:22.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.635+0000 7f1a0affd640 1 -- 172.21.15.12:0/1268719496 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f1a24011470 con 0x560485542a00 2024-04-22T13:31:22.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.635+0000 7f1a34788740 1 -- 172.21.15.12:0/1268719496 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a2009c900 con 0x560485542a00 2024-04-22T13:31:22.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.635+0000 7f1a0affd640 1 -- 172.21.15.12:0/1268719496 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1a2402e050 con 0x560485542a00 2024-04-22T13:31:22.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.635+0000 7f1a34788740 10 client.18297 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:22.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.635+0000 7f1a34788740 1 -- 172.21.15.12:0/1268719496 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56048527a7d0 con 0x560485542a00 2024-04-22T13:31:22.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.635+0000 7f1a0affd640 1 -- 172.21.15.12:0/1268719496 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a2009c900 con 0x560485542a00 2024-04-22T13:31:22.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.635+0000 7f1a0affd640 1 -- 172.21.15.12:0/1268719496 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1a2401f4a0 con 0x560485542a00 2024-04-22T13:31:22.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.635+0000 7f1a34788740 10 client.18297 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.635+0000 7f1a34788740 10 client.18297 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.635+0000 7f1a34788740 20 client.18297 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.635+0000 7f1a34788740 1 --2- 172.21.15.12:0/1268719496 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560485552ca0 0x560485555100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.635+0000 7f1a34788740 4 client.18297 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.635+0000 7f1a34788740 1 -- 172.21.15.12:0/1268719496 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56048543d930 con 0x560485552ca0 2024-04-22T13:31:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.636+0000 7f1a2b7fe640 1 --2- 172.21.15.12:0/1268719496 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560485552ca0 0x560485555100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.636+0000 7f1a2b7fe640 1 --2- 172.21.15.12:0/1268719496 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560485552ca0 0x560485555100 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9992160ca5856fd4 server_cookie=e762977fd609c8d5 in_seq=0 out_seq=0 2024-04-22T13:31:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.636+0000 7f1a0affd640 10 client.18297 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.637+0000 7f1a0affd640 1 -- 172.21.15.12:0/1268719496 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56048543d930 con 0x560485552ca0 2024-04-22T13:31:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.637+0000 7f1a0affd640 10 client.18297 handle_command_reply: tid=0 2024-04-22T13:31:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.637+0000 7f1a34788740 1 client.18297 shutdown 2024-04-22T13:31:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.637+0000 7f1a34788740 1 -- 172.21.15.12:0/1268719496 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560485542a00 msgr2=0x56048553fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.637+0000 7f1a34788740 1 --2- 172.21.15.12:0/1268719496 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560485542a00 0x56048553fcf0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f1a24013c50 tx=0x7f1a24004a40 comp rx=0 tx=0).stop 2024-04-22T13:31:22.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.637+0000 7f1a34788740 1 -- 172.21.15.12:0/1268719496 shutdown_connections 2024-04-22T13:31:22.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.637+0000 7f1a34788740 1 --2- 172.21.15.12:0/1268719496 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560485552ca0 0x560485555100 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.637+0000 7f1a34788740 1 --2- 172.21.15.12:0/1268719496 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560485542a00 0x56048553fcf0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.637+0000 7f1a34788740 1 --2- 172.21.15.12:0/1268719496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560485539e80 0x56048553f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.637+0000 7f1a0affd640 0 client.18297 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:22.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.637+0000 7f1a34788740 1 --2- 172.21.15.12:0/1268719496 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604855394d0 0x560485542350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.637+0000 7f1a34788740 1 -- 172.21.15.12:0/1268719496 >> 172.21.15.12:0/1268719496 conn(0x560485533ba0 msgr2=0x5604855358f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:22.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.637+0000 7f1a34788740 1 -- 172.21.15.12:0/1268719496 shutdown_connections 2024-04-22T13:31:22.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.637+0000 7f1a34788740 1 -- 172.21.15.12:0/1268719496 wait complete. 2024-04-22T13:31:22.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.637+0000 7f1a34788740 20 client.18297 trim_cache size 0 max 16384 2024-04-22T13:31:22.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.651+0000 7f1a34788740 1 Processor -- start 2024-04-22T13:31:22.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.651+0000 7f1a34788740 1 -- start start 2024-04-22T13:31:22.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.651+0000 7f1a34788740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604855394d0 0x56048554e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.651+0000 7f1a34788740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560485539e80 0x56048554b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.651+0000 7f1a34788740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560485542a00 0x560485532a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.651+0000 7f1a34788740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c1adba0 con 0x5604855394d0 2024-04-22T13:31:22.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.651+0000 7f1a34788740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c1af6e0 con 0x560485539e80 2024-04-22T13:31:22.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.651+0000 7f1a34788740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c1ae270 con 0x560485542a00 2024-04-22T13:31:22.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.651+0000 7f1a2b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604855394d0 0x56048554e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.651+0000 7f1a2b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604855394d0 0x56048554e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44850/0 (socket says 172.21.15.12:44850) 2024-04-22T13:31:22.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.651+0000 7f1a2b7fe640 1 -- 172.21.15.12:0/35580027 learned_addr learned my addr 172.21.15.12:0/35580027 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:22.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.651+0000 7f1a2affd640 1 --2- 172.21.15.12:0/35580027 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560485539e80 0x56048554b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.651+0000 7f1a2bfff640 1 --2- 172.21.15.12:0/35580027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560485542a00 0x560485532a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.651+0000 7f1a2b7fe640 1 -- 172.21.15.12:0/35580027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560485542a00 msgr2=0x560485532a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.651+0000 7f1a2b7fe640 1 --2- 172.21.15.12:0/35580027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560485542a00 0x560485532a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.651+0000 7f1a2b7fe640 1 -- 172.21.15.12:0/35580027 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560485539e80 msgr2=0x56048554b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.651+0000 7f1a2b7fe640 1 --2- 172.21.15.12:0/35580027 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560485539e80 0x56048554b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.651+0000 7f1a2b7fe640 1 -- 172.21.15.12:0/35580027 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560485551840 con 0x5604855394d0 2024-04-22T13:31:22.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.651+0000 7f1a2affd640 1 --2- 172.21.15.12:0/35580027 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560485539e80 0x56048554b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:22.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.651+0000 7f1a2b7fe640 1 --2- 172.21.15.12:0/35580027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604855394d0 0x56048554e7d0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c02c170 tx=0x7f1a1c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e57dc98db7e5aa server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:22.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.651+0000 7f1a117fa640 1 -- 172.21.15.12:0/35580027 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c047050 con 0x5604855394d0 2024-04-22T13:31:22.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.652+0000 7f1a117fa640 1 -- 172.21.15.12:0/35580027 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a1c02cc90 con 0x5604855394d0 2024-04-22T13:31:22.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.652+0000 7f1a117fa640 1 -- 172.21.15.12:0/35580027 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c005410 con 0x5604855394d0 2024-04-22T13:31:22.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.652+0000 7f1a34788740 1 -- 172.21.15.12:0/35580027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604855394d0 msgr2=0x56048554e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.652+0000 7f1a34788740 1 --2- 172.21.15.12:0/35580027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604855394d0 0x56048554e7d0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c02c170 tx=0x7f1a1c009260 comp rx=0 tx=0).stop 2024-04-22T13:31:22.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.652+0000 7f1a34788740 1 -- 172.21.15.12:0/35580027 shutdown_connections 2024-04-22T13:31:22.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.652+0000 7f1a34788740 1 --2- 172.21.15.12:0/35580027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560485542a00 0x560485532a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.652+0000 7f1a34788740 1 --2- 172.21.15.12:0/35580027 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560485539e80 0x56048554b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.652+0000 7f1a34788740 1 --2- 172.21.15.12:0/35580027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5604855394d0 0x56048554e7d0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.652+0000 7f1a34788740 1 -- 172.21.15.12:0/35580027 >> 172.21.15.12:0/35580027 conn(0x560485533ba0 msgr2=0x560485532370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:22.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.652+0000 7f1a34788740 1 -- 172.21.15.12:0/35580027 shutdown_connections 2024-04-22T13:31:22.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.652+0000 7f1a34788740 1 -- 172.21.15.12:0/35580027 wait complete. 2024-04-22T13:31:22.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.652+0000 7f1a34788740 1 -- start start 2024-04-22T13:31:22.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.653+0000 7f1a34788740 1 Processor -- start 2024-04-22T13:31:22.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.653+0000 7f1a34788740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604855394d0 0x56048553bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.653+0000 7f1a34788740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560485539e80 0x56048553c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.653+0000 7f1a34788740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560485542a00 0x56048553c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.653+0000 7f1a34788740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c1af010 con 0x560485539e80 2024-04-22T13:31:22.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.653+0000 7f1a34788740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c1ae940 con 0x560485542a00 2024-04-22T13:31:22.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.653+0000 7f1a34788740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c1abba0 con 0x5604855394d0 2024-04-22T13:31:22.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.653+0000 7f1a34788740 10 client.0 _ll_register_callbacks cb 0x7f1a240138d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:22.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.653+0000 7f1a2bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560485542a00 0x56048553c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.653+0000 7f1a2affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560485539e80 0x56048553c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.654 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.653+0000 7f1a2bfff640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560485542a00 0x56048553c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:52172/0 (socket says 172.21.15.12:52172) 2024-04-22T13:31:22.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.653+0000 7f1a2bfff640 1 -- 172.21.15.12:0/782044871 learned_addr learned my addr 172.21.15.12:0/782044871 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:22.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.653+0000 7f1a2affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560485539e80 0x56048553c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44866/0 (socket says 172.21.15.12:44866) 2024-04-22T13:31:22.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.653+0000 7f1a2b7fe640 1 --2- 172.21.15.12:0/782044871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604855394d0 0x56048553bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.653+0000 7f1a2affd640 1 -- 172.21.15.12:0/782044871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604855394d0 msgr2=0x56048553bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.653+0000 7f1a2affd640 1 --2- 172.21.15.12:0/782044871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604855394d0 0x56048553bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.654+0000 7f1a2affd640 1 -- 172.21.15.12:0/782044871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560485542a00 msgr2=0x56048553c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:22.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.654+0000 7f1a2affd640 1 --2- 172.21.15.12:0/782044871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560485542a00 0x56048553c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.654+0000 7f1a2affd640 1 -- 172.21.15.12:0/782044871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560485545120 con 0x560485539e80 2024-04-22T13:31:22.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.654+0000 7f1a2affd640 1 --2- 172.21.15.12:0/782044871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560485539e80 0x56048553c080 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f1a2009cab0 tx=0x7f1a200a06e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c530b408ec38005c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:22.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.654+0000 7f1a0affd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:22.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.654+0000 7f1a0affd640 1 -- 172.21.15.12:0/782044871 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20020040 con 0x560485539e80 2024-04-22T13:31:22.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.654+0000 7f1a0affd640 1 -- 172.21.15.12:0/782044871 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f1a200cdc30 con 0x560485539e80 2024-04-22T13:31:22.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.654+0000 7f1a34788740 1 -- 172.21.15.12:0/782044871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a1c02cc90 con 0x560485539e80 2024-04-22T13:31:22.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.654+0000 7f1a0affd640 1 -- 172.21.15.12:0/782044871 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1a200cde30 con 0x560485539e80 2024-04-22T13:31:22.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.654+0000 7f1a34788740 10 client.18303 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:22.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.654+0000 7f1a34788740 1 -- 172.21.15.12:0/782044871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5604855bd900 con 0x560485539e80 2024-04-22T13:31:22.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.654+0000 7f1a0affd640 1 -- 172.21.15.12:0/782044871 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a1c02cc90 con 0x560485539e80 2024-04-22T13:31:22.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.654+0000 7f1a0affd640 1 -- 172.21.15.12:0/782044871 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1a200a00b0 con 0x560485539e80 2024-04-22T13:31:22.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.654+0000 7f1a34788740 10 client.18303 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:22.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.654+0000 7f1a34788740 10 client.18303 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:22.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.654+0000 7f1a34788740 20 client.18303 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:22.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.654+0000 7f1a34788740 1 --2- 172.21.15.12:0/782044871 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5604855bdbe0 0x5604855bffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:22.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.654+0000 7f1a34788740 4 client.18303 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:31:22.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.654+0000 7f1a34788740 1 -- 172.21.15.12:0/782044871 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5604854abc10 con 0x5604855bdbe0 2024-04-22T13:31:22.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.655+0000 7f1a2b7fe640 1 --2- 172.21.15.12:0/782044871 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5604855bdbe0 0x5604855bffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:22.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.655+0000 7f1a0affd640 10 client.18303 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:22.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.655+0000 7f1a2b7fe640 1 --2- 172.21.15.12:0/782044871 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5604855bdbe0 0x5604855bffd0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54140b9e8fefd0 server_cookie=3f48c4c987bd049b in_seq=0 out_seq=0 2024-04-22T13:31:22.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.655+0000 7f1a0affd640 1 -- 172.21.15.12:0/782044871 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5604854abc10 con 0x5604855bdbe0 2024-04-22T13:31:22.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.655+0000 7f1a0affd640 10 client.18303 handle_command_reply: tid=0 2024-04-22T13:31:22.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.655+0000 7f1a34788740 1 client.18303 shutdown 2024-04-22T13:31:22.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.655+0000 7f1a34788740 1 -- 172.21.15.12:0/782044871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560485539e80 msgr2=0x56048553c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.655+0000 7f1a34788740 1 --2- 172.21.15.12:0/782044871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560485539e80 0x56048553c080 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f1a2009cab0 tx=0x7f1a200a06e0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.655+0000 7f1a34788740 1 -- 172.21.15.12:0/782044871 shutdown_connections 2024-04-22T13:31:22.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.655+0000 7f1a34788740 1 --2- 172.21.15.12:0/782044871 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5604855bdbe0 0x5604855bffd0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.655+0000 7f1a34788740 1 --2- 172.21.15.12:0/782044871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560485542a00 0x56048553c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.655+0000 7f1a34788740 1 --2- 172.21.15.12:0/782044871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560485539e80 0x56048553c080 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.655+0000 7f1a34788740 1 --2- 172.21.15.12:0/782044871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5604855394d0 0x56048553bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.655+0000 7f1a0affd640 0 client.18303 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:22.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.655+0000 7f1a34788740 1 -- 172.21.15.12:0/782044871 >> 172.21.15.12:0/782044871 conn(0x560485533ba0 msgr2=0x5604855357b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:22.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.655+0000 7f1a34788740 1 -- 172.21.15.12:0/782044871 shutdown_connections 2024-04-22T13:31:22.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.655+0000 7f1a34788740 1 -- 172.21.15.12:0/782044871 wait complete. 2024-04-22T13:31:22.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.656+0000 7f1a34788740 20 client.18303 trim_cache size 0 max 16384 2024-04-22T13:31:22.658 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:31:22.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.656+0000 7f1a31b96640 1 -- 172.21.15.12:0/4018277904 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f1a140a3d20 msgr2=0x7f1a140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.656+0000 7f1a31b96640 1 --2- 172.21.15.12:0/4018277904 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f1a140a3d20 0x7f1a140a61e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009370 tx=0x7f1a24009240 comp rx=0 tx=0).stop 2024-04-22T13:31:22.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.656+0000 7f1a31b96640 1 -- 172.21.15.12:0/4018277904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a2c106160 msgr2=0x7f1a2c19e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:22.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.656+0000 7f1a31b96640 1 --2- 172.21.15.12:0/4018277904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a2c106160 0x7f1a2c19e940 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000d740 tx=0x7f1a2000dc00 comp rx=0 tx=0).stop 2024-04-22T13:31:22.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.656+0000 7f1a31b96640 1 -- 172.21.15.12:0/4018277904 shutdown_connections 2024-04-22T13:31:22.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.656+0000 7f1a31b96640 1 --2- 172.21.15.12:0/4018277904 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f1a140a3d20 0x7f1a140a61e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.656+0000 7f1a31b96640 1 --2- 172.21.15.12:0/4018277904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a2c1985b0 0x7f1a2c198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.656+0000 7f1a31b96640 1 --2- 172.21.15.12:0/4018277904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1a2c106160 0x7f1a2c19e940 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.656+0000 7f1a31b96640 1 --2- 172.21.15.12:0/4018277904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1a2c101090 0x7f1a2c19e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:22.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.656+0000 7f1a31b96640 1 -- 172.21.15.12:0/4018277904 >> 172.21.15.12:0/4018277904 conn(0x7f1a2c077f50 msgr2=0x7f1a2c069510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:22.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.656+0000 7f1a31b96640 1 -- 172.21.15.12:0/4018277904 shutdown_connections 2024-04-22T13:31:22.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:22.656+0000 7f1a31b96640 1 -- 172.21.15.12:0/4018277904 wait complete. 2024-04-22T13:31:22.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:22 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1318687060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:31:22.876 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:22 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1318687060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:31:22.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:22 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1318687060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:31:23.662 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:31:23.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:31:23.663 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:31:23.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.787+0000 7fd9ba9f4640 1 -- 172.21.15.12:0/2086789834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9b406a840 msgr2=0x7fd9b406ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:23.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.787+0000 7fd9ba9f4640 1 --2- 172.21.15.12:0/2086789834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9b406a840 0x7fd9b406ac20 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009900 tx=0x7fd9a402bc30 comp rx=0 tx=0).stop 2024-04-22T13:31:23.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.787+0000 7fd9ba9f4640 1 -- 172.21.15.12:0/2086789834 shutdown_connections 2024-04-22T13:31:23.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.787+0000 7fd9ba9f4640 1 --2- 172.21.15.12:0/2086789834 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9b4107270 0x7fd9b410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.787+0000 7fd9ba9f4640 1 --2- 172.21.15.12:0/2086789834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b406b1f0 0x7fd9b4106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.787+0000 7fd9ba9f4640 1 --2- 172.21.15.12:0/2086789834 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9b406a840 0x7fd9b406ac20 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.787+0000 7fd9ba9f4640 1 -- 172.21.15.12:0/2086789834 >> 172.21.15.12:0/2086789834 conn(0x7fd9b4077eb0 msgr2=0x7fd9b40782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:23.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.787+0000 7fd9ba9f4640 1 -- 172.21.15.12:0/2086789834 shutdown_connections 2024-04-22T13:31:23.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.787+0000 7fd9ba9f4640 1 -- 172.21.15.12:0/2086789834 wait complete. 2024-04-22T13:31:23.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.788+0000 7fd9ba9f4640 1 Processor -- start 2024-04-22T13:31:23.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.788+0000 7fd9ba9f4640 1 -- start start 2024-04-22T13:31:23.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.788+0000 7fd9ba9f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b406a840 0x7fd9b4100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.788+0000 7fd9ba9f4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9b4107270 0x7fd9b41013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.788+0000 7fd9ba9f4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9b4101900 0x7fd9b41a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.788+0000 7fd9ba9f4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41121d0 con 0x7fd9b4107270 2024-04-22T13:31:23.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.788+0000 7fd9ba9f4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b4112050 con 0x7fd9b4101900 2024-04-22T13:31:23.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.788+0000 7fd9ba9f4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b4112350 con 0x7fd9b406a840 2024-04-22T13:31:23.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.788+0000 7fd9b37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9b4107270 0x7fd9b41013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.788+0000 7fd9b37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9b4107270 0x7fd9b41013c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44884/0 (socket says 172.21.15.12:44884) 2024-04-22T13:31:23.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.788+0000 7fd9b37fe640 1 -- 172.21.15.12:0/632252249 learned_addr learned my addr 172.21.15.12:0/632252249 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:23.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.789+0000 7fd9b3fff640 1 --2- 172.21.15.12:0/632252249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b406a840 0x7fd9b4100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.789+0000 7fd9b8f91640 1 --2- 172.21.15.12:0/632252249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9b4101900 0x7fd9b41a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.789+0000 7fd9b37fe640 1 -- 172.21.15.12:0/632252249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b406a840 msgr2=0x7fd9b4100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:23.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.789+0000 7fd9b37fe640 1 --2- 172.21.15.12:0/632252249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b406a840 0x7fd9b4100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.789+0000 7fd9b37fe640 1 -- 172.21.15.12:0/632252249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9b4101900 msgr2=0x7fd9b41a68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:23.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.789+0000 7fd9b37fe640 1 --2- 172.21.15.12:0/632252249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9b4101900 0x7fd9b41a68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.789+0000 7fd9b37fe640 1 -- 172.21.15.12:0/632252249 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b41a6fe0 con 0x7fd9b4107270 2024-04-22T13:31:23.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.789+0000 7fd9b37fe640 1 --2- 172.21.15.12:0/632252249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9b4107270 0x7fd9b41013c0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800d740 tx=0x7fd9a800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:23.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.789+0000 7fd9b17fa640 1 -- 172.21.15.12:0/632252249 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8017070 con 0x7fd9b4107270 2024-04-22T13:31:23.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.789+0000 7fd9b17fa640 1 -- 172.21.15.12:0/632252249 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9a800de50 con 0x7fd9b4107270 2024-04-22T13:31:23.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.789+0000 7fd9ba9f4640 1 -- 172.21.15.12:0/632252249 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9b41a72d0 con 0x7fd9b4107270 2024-04-22T13:31:23.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.789+0000 7fd9b17fa640 1 -- 172.21.15.12:0/632252249 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8004ce0 con 0x7fd9b4107270 2024-04-22T13:31:23.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.789+0000 7fd9ba9f4640 1 -- 172.21.15.12:0/632252249 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9b41a77d0 con 0x7fd9b4107270 2024-04-22T13:31:23.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.792+0000 7fd9b17fa640 1 -- 172.21.15.12:0/632252249 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd9a8004e80 con 0x7fd9b4107270 2024-04-22T13:31:23.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.792+0000 7fd9bd5e3740 1 Processor -- start 2024-04-22T13:31:23.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.792+0000 7fd9bd5e3740 1 -- start start 2024-04-22T13:31:23.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.792+0000 7fd9bd5e3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ef06a418e0 0x55ef06a41cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.792+0000 7fd9bd5e3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef06a42290 0x55ef06a4a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.792+0000 7fd9bd5e3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef06a4ae10 0x55ef06a4d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.792+0000 7fd9bd5e3740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ef069a5f00 con 0x55ef06a4ae10 2024-04-22T13:31:23.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.792+0000 7fd9bd5e3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ef06943590 con 0x55ef06a418e0 2024-04-22T13:31:23.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.792+0000 7fd9bd5e3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ef069a1ec0 con 0x55ef06a42290 2024-04-22T13:31:23.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.792+0000 7fd9b17fa640 1 --2- 172.21.15.12:0/632252249 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd99c0a3d50 0x7fd99c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.793+0000 7fd9b37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef06a4ae10 0x55ef06a4d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.793+0000 7fd9b37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef06a4ae10 0x55ef06a4d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44896/0 (socket says 172.21.15.12:44896) 2024-04-22T13:31:23.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.793+0000 7fd9b37fe640 1 -- 172.21.15.12:0/2690206165 learned_addr learned my addr 172.21.15.12:0/2690206165 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:23.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.793+0000 7fd9b3fff640 1 --2- 172.21.15.12:0/2690206165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef06a42290 0x55ef06a4a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.793+0000 7fd9b8f91640 1 --2- 172.21.15.12:0/632252249 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd99c0a3d50 0x7fd99c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.793+0000 7fd9b17fa640 1 -- 172.21.15.12:0/632252249 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fd9a80d0210 con 0x7fd9b4107270 2024-04-22T13:31:23.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.793+0000 7fd9b8f91640 1 --2- 172.21.15.12:0/2690206165 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ef06a418e0 0x55ef06a41cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.793+0000 7fd9b37fe640 1 -- 172.21.15.12:0/2690206165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef06a42290 msgr2=0x55ef06a4a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:23.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.793+0000 7fd9b37fe640 1 --2- 172.21.15.12:0/2690206165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef06a42290 0x55ef06a4a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.793+0000 7fd9b37fe640 1 -- 172.21.15.12:0/2690206165 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ef06a418e0 msgr2=0x55ef06a41cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:23.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.793+0000 7fd9b37fe640 1 --2- 172.21.15.12:0/2690206165 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ef06a418e0 0x55ef06a41cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.793+0000 7fd9b37fe640 1 -- 172.21.15.12:0/2690206165 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef067807e0 con 0x55ef06a4ae10 2024-04-22T13:31:23.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.793+0000 7fd9b8f91640 1 --2- 172.21.15.12:0/2690206165 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ef06a418e0 0x55ef06a41cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:23.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.793+0000 7fd9b37fe640 1 --2- 172.21.15.12:0/2690206165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef06a4ae10 0x55ef06a4d260 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8005d00 tx=0x7fd9a809bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=716e8d3adc3bc0a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:23.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.793+0000 7fd9b8f91640 1 --2- 172.21.15.12:0/632252249 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd99c0a3d50 0x7fd99c0a6210 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac005cc0 tx=0x7fd9ac00a340 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:23.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.793+0000 7fd99a7fc640 1 -- 172.21.15.12:0/2690206165 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a80d0020 con 0x55ef06a4ae10 2024-04-22T13:31:23.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.793+0000 7fd99a7fc640 1 -- 172.21.15.12:0/2690206165 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9a809c8d0 con 0x55ef06a4ae10 2024-04-22T13:31:23.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.793+0000 7fd99a7fc640 1 -- 172.21.15.12:0/2690206165 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8020070 con 0x55ef06a4ae10 2024-04-22T13:31:23.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.794+0000 7fd9bd5e3740 1 -- 172.21.15.12:0/2690206165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef06a4ae10 msgr2=0x55ef06a4d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:23.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.794+0000 7fd9bd5e3740 1 --2- 172.21.15.12:0/2690206165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef06a4ae10 0x55ef06a4d260 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8005d00 tx=0x7fd9a809bdd0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.794+0000 7fd9bd5e3740 1 -- 172.21.15.12:0/2690206165 shutdown_connections 2024-04-22T13:31:23.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.794+0000 7fd9bd5e3740 1 --2- 172.21.15.12:0/2690206165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef06a4ae10 0x55ef06a4d260 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.794+0000 7fd9bd5e3740 1 --2- 172.21.15.12:0/2690206165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef06a42290 0x55ef06a4a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.794+0000 7fd9bd5e3740 1 --2- 172.21.15.12:0/2690206165 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ef06a418e0 0x55ef06a41cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.794+0000 7fd9bd5e3740 1 -- 172.21.15.12:0/2690206165 >> 172.21.15.12:0/2690206165 conn(0x55ef06a3bfb0 msgr2=0x55ef06a3c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:23.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.794+0000 7fd9bd5e3740 1 -- 172.21.15.12:0/2690206165 shutdown_connections 2024-04-22T13:31:23.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.794+0000 7fd9bd5e3740 1 -- 172.21.15.12:0/2690206165 wait complete. 2024-04-22T13:31:23.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.794+0000 7fd9bd5e3740 1 -- start start 2024-04-22T13:31:23.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.795+0000 7fd9bd5e3740 1 Processor -- start 2024-04-22T13:31:23.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.795+0000 7fd9bd5e3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef06a418e0 0x55ef06a47da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.795+0000 7fd9bd5e3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ef06a42290 0x55ef06a482e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.795+0000 7fd9bd5e3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef06a4ae10 0x55ef06a48820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.795+0000 7fd9bd5e3740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55ef06a4dcc0 con 0x55ef06a418e0 2024-04-22T13:31:23.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.795+0000 7fd9bd5e3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ef0695f580 con 0x55ef06a42290 2024-04-22T13:31:23.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.795+0000 7fd9bd5e3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ef06a4de40 con 0x55ef06a4ae10 2024-04-22T13:31:23.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.795+0000 7fd9bd5e3740 10 client.0 _ll_register_callbacks cb 0x55ef06965650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:23.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.795+0000 7fd9b3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef06a418e0 0x55ef06a47da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.795+0000 7fd9b3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef06a418e0 0x55ef06a47da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44910/0 (socket says 172.21.15.12:44910) 2024-04-22T13:31:23.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.795+0000 7fd9b3fff640 1 -- 172.21.15.12:0/2302017625 learned_addr learned my addr 172.21.15.12:0/2302017625 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:23.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.795+0000 7fd9b8f91640 1 --2- 172.21.15.12:0/2302017625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef06a4ae10 0x55ef06a48820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.795+0000 7fd9b37fe640 1 --2- 172.21.15.12:0/2302017625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ef06a42290 0x55ef06a482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.795+0000 7fd9b3fff640 1 -- 172.21.15.12:0/2302017625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef06a4ae10 msgr2=0x55ef06a48820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:23.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.795+0000 7fd9b3fff640 1 --2- 172.21.15.12:0/2302017625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef06a4ae10 0x55ef06a48820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.795+0000 7fd9b3fff640 1 -- 172.21.15.12:0/2302017625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ef06a42290 msgr2=0x55ef06a482e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:23.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.795+0000 7fd9b3fff640 1 --2- 172.21.15.12:0/2302017625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ef06a42290 0x55ef06a482e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.795+0000 7fd9b3fff640 1 -- 172.21.15.12:0/2302017625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef067e0f70 con 0x55ef06a418e0 2024-04-22T13:31:23.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.795+0000 7fd9b37fe640 1 --2- 172.21.15.12:0/2302017625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ef06a42290 0x55ef06a482e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:23.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.796+0000 7fd9b3fff640 1 --2- 172.21.15.12:0/2302017625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef06a418e0 0x55ef06a47da0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fd9a402c170 tx=0x7fd9a402ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7ade4bd1468a67 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:23.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.796+0000 7fd9937fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:23.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.796+0000 7fd9937fe640 1 -- 172.21.15.12:0/2302017625 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4005230 con 0x55ef06a418e0 2024-04-22T13:31:23.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.796+0000 7fd9937fe640 1 -- 172.21.15.12:0/2302017625 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fd9a40053d0 con 0x55ef06a418e0 2024-04-22T13:31:23.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.796+0000 7fd9bd5e3740 1 -- 172.21.15.12:0/2302017625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9a809c8d0 con 0x55ef06a418e0 2024-04-22T13:31:23.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.796+0000 7fd9937fe640 1 -- 172.21.15.12:0/2302017625 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd9a40055d0 con 0x55ef06a418e0 2024-04-22T13:31:23.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.796+0000 7fd9bd5e3740 10 client.18315 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:23.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.796+0000 7fd9bd5e3740 1 -- 172.21.15.12:0/2302017625 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ef06678170 con 0x55ef06a418e0 2024-04-22T13:31:23.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.796+0000 7fd9937fe640 1 -- 172.21.15.12:0/2302017625 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9a809c8d0 con 0x55ef06a418e0 2024-04-22T13:31:23.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.796+0000 7fd9937fe640 1 -- 172.21.15.12:0/2302017625 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd9a4050020 con 0x55ef06a418e0 2024-04-22T13:31:23.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.796+0000 7fd9bd5e3740 10 client.18315 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:23.798 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.796+0000 7fd9bd5e3740 10 client.18315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:23.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.796+0000 7fd9bd5e3740 20 client.18315 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:23.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.796+0000 7fd9bd5e3740 1 --2- 172.21.15.12:0/2302017625 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55ef06a5b1d0 0x55ef06a5d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.797+0000 7fd9bd5e3740 4 client.18315 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:23.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.797+0000 7fd9bd5e3740 1 -- 172.21.15.12:0/2302017625 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef069b3920 con 0x55ef06a5b1d0 2024-04-22T13:31:23.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.797+0000 7fd9b37fe640 1 --2- 172.21.15.12:0/2302017625 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55ef06a5b1d0 0x55ef06a5d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.797+0000 7fd9937fe640 10 client.18315 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:23.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.797+0000 7fd9b37fe640 1 --2- 172.21.15.12:0/2302017625 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55ef06a5b1d0 0x55ef06a5d640 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a473a486b6b453f server_cookie=5d597297c183622 in_seq=0 out_seq=0 2024-04-22T13:31:23.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.798+0000 7fd9937fe640 1 -- 172.21.15.12:0/2302017625 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ef069b3920 con 0x55ef06a5b1d0 2024-04-22T13:31:23.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.798+0000 7fd9937fe640 10 client.18315 handle_command_reply: tid=0 2024-04-22T13:31:23.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.798+0000 7fd9bd5e3740 1 client.18315 shutdown 2024-04-22T13:31:23.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.798+0000 7fd9bd5e3740 1 -- 172.21.15.12:0/2302017625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef06a418e0 msgr2=0x55ef06a47da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:23.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.798+0000 7fd9bd5e3740 1 --2- 172.21.15.12:0/2302017625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef06a418e0 0x55ef06a47da0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fd9a402c170 tx=0x7fd9a402ca10 comp rx=0 tx=0).stop 2024-04-22T13:31:23.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.798+0000 7fd9bd5e3740 1 -- 172.21.15.12:0/2302017625 shutdown_connections 2024-04-22T13:31:23.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.798+0000 7fd9bd5e3740 1 --2- 172.21.15.12:0/2302017625 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55ef06a5b1d0 0x55ef06a5d640 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.799 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.798+0000 7fd9bd5e3740 1 --2- 172.21.15.12:0/2302017625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef06a4ae10 0x55ef06a48820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.798+0000 7fd9bd5e3740 1 --2- 172.21.15.12:0/2302017625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ef06a42290 0x55ef06a482e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.798+0000 7fd9937fe640 0 client.18315 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:23.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.798+0000 7fd9bd5e3740 1 --2- 172.21.15.12:0/2302017625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef06a418e0 0x55ef06a47da0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.798+0000 7fd9bd5e3740 1 -- 172.21.15.12:0/2302017625 >> 172.21.15.12:0/2302017625 conn(0x55ef06a3bfb0 msgr2=0x55ef06a3dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:23.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.798+0000 7fd9bd5e3740 1 -- 172.21.15.12:0/2302017625 shutdown_connections 2024-04-22T13:31:23.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.798+0000 7fd9bd5e3740 1 -- 172.21.15.12:0/2302017625 wait complete. 2024-04-22T13:31:23.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.798+0000 7fd9bd5e3740 20 client.18315 trim_cache size 0 max 16384 2024-04-22T13:31:23.816 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.816+0000 7fd9bd5e3740 1 Processor -- start 2024-04-22T13:31:23.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.816+0000 7fd9bd5e3740 1 -- start start 2024-04-22T13:31:23.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.816+0000 7fd9bd5e3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ef06a418e0 0x55ef06a4a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.816+0000 7fd9bd5e3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef06a42290 0x55ef06a56b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.816+0000 7fd9bd5e3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef06a4ae10 0x55ef06a3ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.816+0000 7fd9bd5e3740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41afd00 con 0x55ef06a4ae10 2024-04-22T13:31:23.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.816+0000 7fd9bd5e3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41b1840 con 0x55ef06a418e0 2024-04-22T13:31:23.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.816+0000 7fd9bd5e3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41b03d0 con 0x55ef06a42290 2024-04-22T13:31:23.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.816+0000 7fd9b8f91640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef06a4ae10 0x55ef06a3ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.817+0000 7fd9b8f91640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef06a4ae10 0x55ef06a3ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44914/0 (socket says 172.21.15.12:44914) 2024-04-22T13:31:23.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.817+0000 7fd9b8f91640 1 -- 172.21.15.12:0/3561162826 learned_addr learned my addr 172.21.15.12:0/3561162826 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:23.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.817+0000 7fd9b37fe640 1 --2- 172.21.15.12:0/3561162826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef06a42290 0x55ef06a56b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.817+0000 7fd9b3fff640 1 --2- 172.21.15.12:0/3561162826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ef06a418e0 0x55ef06a4a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.817+0000 7fd9b8f91640 1 -- 172.21.15.12:0/3561162826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef06a42290 msgr2=0x55ef06a56b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:23.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.817+0000 7fd9b8f91640 1 --2- 172.21.15.12:0/3561162826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef06a42290 0x55ef06a56b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.817+0000 7fd9b8f91640 1 -- 172.21.15.12:0/3561162826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ef06a418e0 msgr2=0x55ef06a4a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:23.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.817+0000 7fd9b8f91640 1 --2- 172.21.15.12:0/3561162826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ef06a418e0 0x55ef06a4a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.817+0000 7fd9b8f91640 1 -- 172.21.15.12:0/3561162826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef06a462f0 con 0x55ef06a4ae10 2024-04-22T13:31:23.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.817+0000 7fd9b8f91640 1 --2- 172.21.15.12:0/3561162826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef06a4ae10 0x55ef06a3ae90 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac013230 tx=0x7fd9ac005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6466ebbdb063cc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:23.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.817+0000 7fd999ffb640 1 -- 172.21.15.12:0/3561162826 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac02a070 con 0x55ef06a4ae10 2024-04-22T13:31:23.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.817+0000 7fd999ffb640 1 -- 172.21.15.12:0/3561162826 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9ac0118f0 con 0x55ef06a4ae10 2024-04-22T13:31:23.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.817+0000 7fd999ffb640 1 -- 172.21.15.12:0/3561162826 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac010830 con 0x55ef06a4ae10 2024-04-22T13:31:23.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.818+0000 7fd9bd5e3740 1 -- 172.21.15.12:0/3561162826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef06a4ae10 msgr2=0x55ef06a3ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:23.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.818+0000 7fd9bd5e3740 1 --2- 172.21.15.12:0/3561162826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef06a4ae10 0x55ef06a3ae90 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac013230 tx=0x7fd9ac005bd0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.818+0000 7fd9bd5e3740 1 -- 172.21.15.12:0/3561162826 shutdown_connections 2024-04-22T13:31:23.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.818+0000 7fd9bd5e3740 1 --2- 172.21.15.12:0/3561162826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef06a4ae10 0x55ef06a3ae90 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.818+0000 7fd9bd5e3740 1 --2- 172.21.15.12:0/3561162826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef06a42290 0x55ef06a56b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.818+0000 7fd9bd5e3740 1 --2- 172.21.15.12:0/3561162826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ef06a418e0 0x55ef06a4a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.818+0000 7fd9bd5e3740 1 -- 172.21.15.12:0/3561162826 >> 172.21.15.12:0/3561162826 conn(0x55ef06a3bfb0 msgr2=0x55ef06a3a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:23.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.818+0000 7fd9bd5e3740 1 -- 172.21.15.12:0/3561162826 shutdown_connections 2024-04-22T13:31:23.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.818+0000 7fd9bd5e3740 1 -- 172.21.15.12:0/3561162826 wait complete. 2024-04-22T13:31:23.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.818+0000 7fd9bd5e3740 1 -- start start 2024-04-22T13:31:23.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.818+0000 7fd9bd5e3740 1 Processor -- start 2024-04-22T13:31:23.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.819+0000 7fd9bd5e3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef06a418e0 0x55ef06a4ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.819+0000 7fd9bd5e3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ef06a42290 0x55ef06a4f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.819+0000 7fd9bd5e3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef06a4ae10 0x55ef06a4f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.819+0000 7fd9bd5e3740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41b1170 con 0x55ef06a418e0 2024-04-22T13:31:23.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.819+0000 7fd9bd5e3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41b0aa0 con 0x55ef06a42290 2024-04-22T13:31:23.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.819+0000 7fd9bd5e3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41add00 con 0x55ef06a4ae10 2024-04-22T13:31:23.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.819+0000 7fd9bd5e3740 10 client.0 _ll_register_callbacks cb 0x7fd9a4005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:23.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.819+0000 7fd9b3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef06a418e0 0x55ef06a4ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.819+0000 7fd9b3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef06a418e0 0x55ef06a4ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44926/0 (socket says 172.21.15.12:44926) 2024-04-22T13:31:23.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.819+0000 7fd9b3fff640 1 -- 172.21.15.12:0/2861254832 learned_addr learned my addr 172.21.15.12:0/2861254832 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:23.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.819+0000 7fd9b37fe640 1 --2- 172.21.15.12:0/2861254832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ef06a42290 0x55ef06a4f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.819+0000 7fd9b8f91640 1 --2- 172.21.15.12:0/2861254832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef06a4ae10 0x55ef06a4f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.819+0000 7fd9b3fff640 1 -- 172.21.15.12:0/2861254832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef06a4ae10 msgr2=0x55ef06a4f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:23.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.819+0000 7fd9b3fff640 1 --2- 172.21.15.12:0/2861254832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef06a4ae10 0x55ef06a4f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.819+0000 7fd9b3fff640 1 -- 172.21.15.12:0/2861254832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ef06a42290 msgr2=0x55ef06a4f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:23.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.819+0000 7fd9b3fff640 1 --2- 172.21.15.12:0/2861254832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ef06a42290 0x55ef06a4f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.819+0000 7fd9b3fff640 1 -- 172.21.15.12:0/2861254832 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef06ac0aa0 con 0x55ef06a418e0 2024-04-22T13:31:23.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.819+0000 7fd9b8f91640 1 --2- 172.21.15.12:0/2861254832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef06a4ae10 0x55ef06a4f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:23.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.819+0000 7fd9b3fff640 1 --2- 172.21.15.12:0/2861254832 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef06a418e0 0x55ef06a4ed60 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4003a50 tx=0x7fd9a4003fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0bb7d8f8776830c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:23.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.819+0000 7fd9937fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:23.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.819+0000 7fd9937fe640 1 -- 172.21.15.12:0/2861254832 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4005410 con 0x55ef06a418e0 2024-04-22T13:31:23.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.819+0000 7fd9937fe640 1 -- 172.21.15.12:0/2861254832 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fd9a40046c0 con 0x55ef06a418e0 2024-04-22T13:31:23.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.819+0000 7fd9bd5e3740 1 -- 172.21.15.12:0/2861254832 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9ac0118f0 con 0x55ef06a418e0 2024-04-22T13:31:23.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.820+0000 7fd9937fe640 1 -- 172.21.15.12:0/2861254832 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd9a402c5d0 con 0x55ef06a418e0 2024-04-22T13:31:23.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.820+0000 7fd9bd5e3740 10 client.18321 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:23.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.820+0000 7fd9bd5e3740 1 -- 172.21.15.12:0/2861254832 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ef06ac1d80 con 0x55ef06a418e0 2024-04-22T13:31:23.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.820+0000 7fd9937fe640 1 -- 172.21.15.12:0/2861254832 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9ac0118f0 con 0x55ef06a418e0 2024-04-22T13:31:23.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.820+0000 7fd9937fe640 1 -- 172.21.15.12:0/2861254832 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd9a4063020 con 0x55ef06a418e0 2024-04-22T13:31:23.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.820+0000 7fd9bd5e3740 10 client.18321 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:23.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.820+0000 7fd9bd5e3740 10 client.18321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:23.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.820+0000 7fd9bd5e3740 20 client.18321 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:23.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.820+0000 7fd9bd5e3740 1 --2- 172.21.15.12:0/2861254832 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55ef06ac24a0 0x55ef06ac48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.820+0000 7fd9bd5e3740 4 client.18321 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:31:23.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.820+0000 7fd9bd5e3740 1 -- 172.21.15.12:0/2861254832 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ef069ad790 con 0x55ef06ac24a0 2024-04-22T13:31:23.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.820+0000 7fd9b37fe640 1 --2- 172.21.15.12:0/2861254832 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55ef06ac24a0 0x55ef06ac48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.821+0000 7fd9937fe640 10 client.18321 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:23.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.821+0000 7fd9b37fe640 1 --2- 172.21.15.12:0/2861254832 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55ef06ac24a0 0x55ef06ac48c0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c64a39514ccc604c server_cookie=870beb3422cfef19 in_seq=0 out_seq=0 2024-04-22T13:31:23.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.822+0000 7fd9937fe640 1 -- 172.21.15.12:0/2861254832 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ef069ad790 con 0x55ef06ac24a0 2024-04-22T13:31:23.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.822+0000 7fd9937fe640 10 client.18321 handle_command_reply: tid=0 2024-04-22T13:31:23.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.822+0000 7fd9bd5e3740 1 client.18321 shutdown 2024-04-22T13:31:23.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.822+0000 7fd9bd5e3740 1 -- 172.21.15.12:0/2861254832 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef06a418e0 msgr2=0x55ef06a4ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:23.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.822+0000 7fd9bd5e3740 1 --2- 172.21.15.12:0/2861254832 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef06a418e0 0x55ef06a4ed60 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4003a50 tx=0x7fd9a4003fd0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.822+0000 7fd9bd5e3740 1 -- 172.21.15.12:0/2861254832 shutdown_connections 2024-04-22T13:31:23.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.822+0000 7fd9bd5e3740 1 --2- 172.21.15.12:0/2861254832 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55ef06ac24a0 0x55ef06ac48c0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.822+0000 7fd9bd5e3740 1 --2- 172.21.15.12:0/2861254832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ef06a4ae10 0x55ef06a4f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.822+0000 7fd9bd5e3740 1 --2- 172.21.15.12:0/2861254832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ef06a42290 0x55ef06a4f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.822+0000 7fd9bd5e3740 1 --2- 172.21.15.12:0/2861254832 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef06a418e0 0x55ef06a4ed60 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.822+0000 7fd9bd5e3740 1 -- 172.21.15.12:0/2861254832 >> 172.21.15.12:0/2861254832 conn(0x55ef06a3bfb0 msgr2=0x55ef06a485d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:23.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.822+0000 7fd9937fe640 0 client.18321 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:23.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.822+0000 7fd9bd5e3740 1 -- 172.21.15.12:0/2861254832 shutdown_connections 2024-04-22T13:31:23.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.822+0000 7fd9bd5e3740 1 -- 172.21.15.12:0/2861254832 wait complete. 2024-04-22T13:31:23.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.822+0000 7fd9bd5e3740 20 client.18321 trim_cache size 0 max 16384 2024-04-22T13:31:23.825 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:31:23.825 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-04-22T13:31:23.825 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "4f57d9e4-838d-457e-9dee-166f8b4853ce", 2024-04-22T13:31:23.825 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-04-22T13:31:23.825 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:31:23.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.823+0000 7fd9ba9f4640 1 -- 172.21.15.12:0/632252249 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd99c0a3d50 msgr2=0x7fd99c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:23.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.823+0000 7fd9ba9f4640 1 --2- 172.21.15.12:0/632252249 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd99c0a3d50 0x7fd99c0a6210 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac005cc0 tx=0x7fd9ac00a340 comp rx=0 tx=0).stop 2024-04-22T13:31:23.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.823+0000 7fd9ba9f4640 1 -- 172.21.15.12:0/632252249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9b4107270 msgr2=0x7fd9b41013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:23.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.823+0000 7fd9ba9f4640 1 --2- 172.21.15.12:0/632252249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9b4107270 0x7fd9b41013c0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800d740 tx=0x7fd9a800dc00 comp rx=0 tx=0).stop 2024-04-22T13:31:23.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.823+0000 7fd9ba9f4640 1 -- 172.21.15.12:0/632252249 shutdown_connections 2024-04-22T13:31:23.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.823+0000 7fd9ba9f4640 1 --2- 172.21.15.12:0/632252249 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd99c0a3d50 0x7fd99c0a6210 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.823+0000 7fd9ba9f4640 1 --2- 172.21.15.12:0/632252249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9b4101900 0x7fd9b41a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.823+0000 7fd9ba9f4640 1 --2- 172.21.15.12:0/632252249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd9b4107270 0x7fd9b41013c0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.823+0000 7fd9ba9f4640 1 --2- 172.21.15.12:0/632252249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd9b406a840 0x7fd9b4100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.823+0000 7fd9ba9f4640 1 -- 172.21.15.12:0/632252249 >> 172.21.15.12:0/632252249 conn(0x7fd9b4077eb0 msgr2=0x7fd9b4108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:23.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.823+0000 7fd9ba9f4640 1 -- 172.21.15.12:0/632252249 shutdown_connections 2024-04-22T13:31:23.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.823+0000 7fd9ba9f4640 1 -- 172.21.15.12:0/632252249 wait complete. 2024-04-22T13:31:23.828 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:23 smithi012 ceph-mon[28265]: pgmap v223: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-22T13:31:23.829 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:31:23.829 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:31:23.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:23 smithi134 ceph-mon[31338]: pgmap v223: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-22T13:31:23.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.938+0000 7f6619cc1640 1 -- 172.21.15.12:0/2446372165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f661406af70 msgr2=0x7f661410ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:23.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.938+0000 7f6619cc1640 1 --2- 172.21.15.12:0/2446372165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f661406af70 0x7f661410ede0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f660c009cd0 tx=0x7f660c02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.938+0000 7f6619cc1640 1 -- 172.21.15.12:0/2446372165 shutdown_connections 2024-04-22T13:31:23.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.938+0000 7f6619cc1640 1 --2- 172.21.15.12:0/2446372165 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f661406af70 0x7f661410ede0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.938+0000 7f6619cc1640 1 --2- 172.21.15.12:0/2446372165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f661406a5b0 0x7f661406aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.938+0000 7f6619cc1640 1 --2- 172.21.15.12:0/2446372165 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6614108060 0x7f6614108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.938+0000 7f6619cc1640 1 -- 172.21.15.12:0/2446372165 >> 172.21.15.12:0/2446372165 conn(0x7f6614077c40 msgr2=0x7f6614078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:23.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.938+0000 7f6619cc1640 1 -- 172.21.15.12:0/2446372165 shutdown_connections 2024-04-22T13:31:23.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.938+0000 7f6619cc1640 1 -- 172.21.15.12:0/2446372165 wait complete. 2024-04-22T13:31:23.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.939+0000 7f6619cc1640 1 Processor -- start 2024-04-22T13:31:23.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.939+0000 7f6619cc1640 1 -- start start 2024-04-22T13:31:23.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.939+0000 7f6619cc1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f661406a5b0 0x7f66141986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.939+0000 7f6619cc1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f661406af70 0x7f6614198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.939+0000 7f6619cc1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6614108060 0x7f661419cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.939+0000 7f6619cc1640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f66141121a0 con 0x7f661406a5b0 2024-04-22T13:31:23.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.939+0000 7f6619cc1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6614112020 con 0x7f6614108060 2024-04-22T13:31:23.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.939+0000 7f6619cc1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6614112320 con 0x7f661406af70 2024-04-22T13:31:23.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.939+0000 7f66137fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f661406a5b0 0x7f66141986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.939+0000 7f66137fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f661406a5b0 0x7f66141986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44952/0 (socket says 172.21.15.12:44952) 2024-04-22T13:31:23.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.939+0000 7f66137fe640 1 -- 172.21.15.12:0/2931984075 learned_addr learned my addr 172.21.15.12:0/2931984075 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:23.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.939+0000 7f6612ffd640 1 --2- 172.21.15.12:0/2931984075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f661406af70 0x7f6614198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.939+0000 7f6613fff640 1 --2- 172.21.15.12:0/2931984075 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6614108060 0x7f661419cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.939+0000 7f66137fe640 1 -- 172.21.15.12:0/2931984075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f661406af70 msgr2=0x7f6614198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:23.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.939+0000 7f66137fe640 1 --2- 172.21.15.12:0/2931984075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f661406af70 0x7f6614198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.939+0000 7f66137fe640 1 -- 172.21.15.12:0/2931984075 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6614108060 msgr2=0x7f661419cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:23.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.939+0000 7f66137fe640 1 --2- 172.21.15.12:0/2931984075 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6614108060 0x7f661419cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.940+0000 7f66137fe640 1 -- 172.21.15.12:0/2931984075 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661419d6b0 con 0x7f661406a5b0 2024-04-22T13:31:23.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.940+0000 7f66137fe640 1 --2- 172.21.15.12:0/2931984075 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f661406a5b0 0x7f66141986c0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f660800c890 tx=0x7f660800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:23.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.940+0000 7f6610ff9640 1 -- 172.21.15.12:0/2931984075 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6608004220 con 0x7f661406a5b0 2024-04-22T13:31:23.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.940+0000 7f6610ff9640 1 -- 172.21.15.12:0/2931984075 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66080043c0 con 0x7f661406a5b0 2024-04-22T13:31:23.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.940+0000 7f6619cc1640 1 -- 172.21.15.12:0/2931984075 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f661419d9a0 con 0x7f661406a5b0 2024-04-22T13:31:23.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.940+0000 7f6610ff9640 1 -- 172.21.15.12:0/2931984075 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6608004a00 con 0x7f661406a5b0 2024-04-22T13:31:23.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.940+0000 7f6619cc1640 1 -- 172.21.15.12:0/2931984075 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66141a5230 con 0x7f661406a5b0 2024-04-22T13:31:23.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.943+0000 7f661c98d740 1 Processor -- start 2024-04-22T13:31:23.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.943+0000 7f6610ff9640 1 -- 172.21.15.12:0/2931984075 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f66080079b0 con 0x7f661406a5b0 2024-04-22T13:31:23.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.943+0000 7f661c98d740 1 -- start start 2024-04-22T13:31:23.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.943+0000 7f661c98d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640a23c92b0 0x5640a23c9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.943+0000 7f661c98d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640a23c9bd0 0x5640a23d6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.943+0000 7f661c98d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640a23d6e90 0x5640a23d9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.943+0000 7f661c98d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5640a232e1e0 con 0x5640a23c9bd0 2024-04-22T13:31:23.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.943+0000 7f661c98d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5640a22cb390 con 0x5640a23d6e90 2024-04-22T13:31:23.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.943+0000 7f661c98d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5640a232a1a0 con 0x5640a23c92b0 2024-04-22T13:31:23.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.943+0000 7f66137fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640a23c9bd0 0x5640a23d6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.943+0000 7f66137fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640a23c9bd0 0x5640a23d6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44966/0 (socket says 172.21.15.12:44966) 2024-04-22T13:31:23.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.943+0000 7f66137fe640 1 -- 172.21.15.12:0/2541150125 learned_addr learned my addr 172.21.15.12:0/2541150125 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:23.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.943+0000 7f6613fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640a23c92b0 0x5640a23c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.943+0000 7f6612ffd640 1 --2- 172.21.15.12:0/2541150125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640a23d6e90 0x5640a23d9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.943+0000 7f6610ff9640 1 --2- 172.21.15.12:0/2931984075 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f65fc0a3d20 0x7f65fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.943+0000 7f66137fe640 1 -- 172.21.15.12:0/2541150125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640a23c92b0 msgr2=0x5640a23c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:23.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.943+0000 7f66137fe640 1 --2- 172.21.15.12:0/2541150125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640a23c92b0 0x5640a23c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.945 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.943+0000 7f66137fe640 1 -- 172.21.15.12:0/2541150125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640a23d6e90 msgr2=0x5640a23d9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:23.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.943+0000 7f66137fe640 1 --2- 172.21.15.12:0/2541150125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640a23d6e90 0x5640a23d9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.943+0000 7f66137fe640 1 -- 172.21.15.12:0/2541150125 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640a210a7d0 con 0x5640a23c9bd0 2024-04-22T13:31:23.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.943+0000 7f6612ffd640 1 --2- 172.21.15.12:0/2541150125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640a23d6e90 0x5640a23d9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:23.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.943+0000 7f6613fff640 1 --2- 172.21.15.12:0/2931984075 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f65fc0a3d20 0x7f65fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.944+0000 7f6610ff9640 1 -- 172.21.15.12:0/2931984075 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f66080d0570 con 0x7f661406a5b0 2024-04-22T13:31:23.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.944+0000 7f66137fe640 1 --2- 172.21.15.12:0/2541150125 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640a23c9bd0 0x5640a23d6950 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f66080cdc50 tx=0x7f6608013840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc753856f5baa33e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:23.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.944+0000 7f65f9ffb640 1 -- 172.21.15.12:0/2541150125 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660809c070 con 0x5640a23c9bd0 2024-04-22T13:31:23.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.944+0000 7f65f9ffb640 1 -- 172.21.15.12:0/2541150125 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6608005290 con 0x5640a23c9bd0 2024-04-22T13:31:23.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.944+0000 7f65f9ffb640 1 -- 172.21.15.12:0/2541150125 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660806e220 con 0x5640a23c9bd0 2024-04-22T13:31:23.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.944+0000 7f661c98d740 1 -- 172.21.15.12:0/2541150125 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640a23c9bd0 msgr2=0x5640a23d6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:23.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.944+0000 7f6613fff640 1 --2- 172.21.15.12:0/2931984075 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f65fc0a3d20 0x7f65fc0a61e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f660c006010 tx=0x7f660c02e040 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:23.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.944+0000 7f661c98d740 1 --2- 172.21.15.12:0/2541150125 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640a23c9bd0 0x5640a23d6950 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f66080cdc50 tx=0x7f6608013840 comp rx=0 tx=0).stop 2024-04-22T13:31:23.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.945+0000 7f661c98d740 1 -- 172.21.15.12:0/2541150125 shutdown_connections 2024-04-22T13:31:23.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.945+0000 7f661c98d740 1 --2- 172.21.15.12:0/2541150125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640a23d6e90 0x5640a23d9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.945+0000 7f661c98d740 1 --2- 172.21.15.12:0/2541150125 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640a23c9bd0 0x5640a23d6950 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.945+0000 7f661c98d740 1 --2- 172.21.15.12:0/2541150125 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640a23c92b0 0x5640a23c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.945+0000 7f661c98d740 1 -- 172.21.15.12:0/2541150125 >> 172.21.15.12:0/2541150125 conn(0x5640a23c3cb0 msgr2=0x5640a23c4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:23.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.945+0000 7f661c98d740 1 -- 172.21.15.12:0/2541150125 shutdown_connections 2024-04-22T13:31:23.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.945+0000 7f661c98d740 1 -- 172.21.15.12:0/2541150125 wait complete. 2024-04-22T13:31:23.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.945+0000 7f661c98d740 1 -- start start 2024-04-22T13:31:23.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.945+0000 7f661c98d740 1 Processor -- start 2024-04-22T13:31:23.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.945+0000 7f661c98d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640a23c92b0 0x5640a23d3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.945+0000 7f661c98d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640a23c9bd0 0x5640a23d43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.945+0000 7f661c98d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640a23d6e90 0x5640a23d48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.945+0000 7f661c98d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5640a23d9bf0 con 0x5640a23c92b0 2024-04-22T13:31:23.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.945+0000 7f661c98d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5640a22e72f0 con 0x5640a23c9bd0 2024-04-22T13:31:23.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.945+0000 7f661c98d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5640a23d9d70 con 0x5640a23d6e90 2024-04-22T13:31:23.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.945+0000 7f661c98d740 10 client.0 _ll_register_callbacks cb 0x5640a22f4580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:23.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.945+0000 7f66137fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640a23c92b0 0x5640a23d3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.946+0000 7f66137fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640a23c92b0 0x5640a23d3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44982/0 (socket says 172.21.15.12:44982) 2024-04-22T13:31:23.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.946+0000 7f66137fe640 1 -- 172.21.15.12:0/4035787701 learned_addr learned my addr 172.21.15.12:0/4035787701 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:23.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.946+0000 7f66137fe640 1 -- 172.21.15.12:0/4035787701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640a23d6e90 msgr2=0x5640a23d48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:23.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.946+0000 7f66137fe640 1 --2- 172.21.15.12:0/4035787701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640a23d6e90 0x5640a23d48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.946+0000 7f66137fe640 1 -- 172.21.15.12:0/4035787701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640a23c9bd0 msgr2=0x5640a23d43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:23.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.946+0000 7f66137fe640 1 --2- 172.21.15.12:0/4035787701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640a23c9bd0 0x5640a23d43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.946+0000 7f66137fe640 1 -- 172.21.15.12:0/4035787701 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640a20403f0 con 0x5640a23c92b0 2024-04-22T13:31:23.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.946+0000 7f66137fe640 1 --2- 172.21.15.12:0/4035787701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640a23c92b0 0x5640a23d3e70 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f66080058e0 tx=0x7f6608007730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446a2b901afc9466 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:23.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.946+0000 7f65f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:23.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.946+0000 7f65f2ffd640 1 -- 172.21.15.12:0/4035787701 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660809be90 con 0x5640a23c92b0 2024-04-22T13:31:23.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.946+0000 7f65f2ffd640 1 -- 172.21.15.12:0/4035787701 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f660809a830 con 0x5640a23c92b0 2024-04-22T13:31:23.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.946+0000 7f661c98d740 1 -- 172.21.15.12:0/4035787701 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5640a233bc00 con 0x5640a23c92b0 2024-04-22T13:31:23.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.946+0000 7f65f2ffd640 1 -- 172.21.15.12:0/4035787701 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f660809c070 con 0x5640a23c92b0 2024-04-22T13:31:23.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.946+0000 7f661c98d740 10 client.18333 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:23.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.946+0000 7f661c98d740 1 -- 172.21.15.12:0/4035787701 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5640a20000f0 con 0x5640a23c92b0 2024-04-22T13:31:23.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.946+0000 7f65f2ffd640 1 -- 172.21.15.12:0/4035787701 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5640a233bc00 con 0x5640a23c92b0 2024-04-22T13:31:23.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.947+0000 7f65f2ffd640 1 -- 172.21.15.12:0/4035787701 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f66080cdf60 con 0x5640a23c92b0 2024-04-22T13:31:23.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.947+0000 7f661c98d740 10 client.18333 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:23.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.947+0000 7f661c98d740 10 client.18333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:23.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.947+0000 7f661c98d740 20 client.18333 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:23.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.947+0000 7f661c98d740 1 --2- 172.21.15.12:0/4035787701 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5640a23e6ca0 0x5640a23e9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.947+0000 7f661c98d740 4 client.18333 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:23.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.947+0000 7f661c98d740 1 -- 172.21.15.12:0/4035787701 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640a2335a70 con 0x5640a23e6ca0 2024-04-22T13:31:23.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.947+0000 7f6612ffd640 1 --2- 172.21.15.12:0/4035787701 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5640a23e6ca0 0x5640a23e9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.947+0000 7f65f2ffd640 10 client.18333 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:23.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.947+0000 7f6612ffd640 1 --2- 172.21.15.12:0/4035787701 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5640a23e6ca0 0x5640a23e9090 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5230a061242159f1 server_cookie=64c467ce21e61b27 in_seq=0 out_seq=0 2024-04-22T13:31:23.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.948+0000 7f65f2ffd640 1 -- 172.21.15.12:0/4035787701 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5640a2335a70 con 0x5640a23e6ca0 2024-04-22T13:31:23.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.948+0000 7f65f2ffd640 10 client.18333 handle_command_reply: tid=0 2024-04-22T13:31:23.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.948+0000 7f661c98d740 1 client.18333 shutdown 2024-04-22T13:31:23.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.948+0000 7f661c98d740 1 -- 172.21.15.12:0/4035787701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640a23c92b0 msgr2=0x5640a23d3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:23.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.948+0000 7f661c98d740 1 --2- 172.21.15.12:0/4035787701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640a23c92b0 0x5640a23d3e70 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f66080058e0 tx=0x7f6608007730 comp rx=0 tx=0).stop 2024-04-22T13:31:23.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.948+0000 7f661c98d740 1 -- 172.21.15.12:0/4035787701 shutdown_connections 2024-04-22T13:31:23.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.948+0000 7f661c98d740 1 --2- 172.21.15.12:0/4035787701 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5640a23e6ca0 0x5640a23e9090 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.948+0000 7f661c98d740 1 --2- 172.21.15.12:0/4035787701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640a23d6e90 0x5640a23d48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.948+0000 7f65f2ffd640 0 client.18333 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:23.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.948+0000 7f661c98d740 1 --2- 172.21.15.12:0/4035787701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640a23c9bd0 0x5640a23d43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.948+0000 7f661c98d740 1 --2- 172.21.15.12:0/4035787701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640a23c92b0 0x5640a23d3e70 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.948+0000 7f661c98d740 1 -- 172.21.15.12:0/4035787701 >> 172.21.15.12:0/4035787701 conn(0x5640a23c3cb0 msgr2=0x5640a23c5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:23.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.948+0000 7f661c98d740 1 -- 172.21.15.12:0/4035787701 shutdown_connections 2024-04-22T13:31:23.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.948+0000 7f661c98d740 1 -- 172.21.15.12:0/4035787701 wait complete. 2024-04-22T13:31:23.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.948+0000 7f661c98d740 20 client.18333 trim_cache size 0 max 16384 2024-04-22T13:31:23.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.966+0000 7f661c98d740 1 Processor -- start 2024-04-22T13:31:23.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.966+0000 7f661c98d740 1 -- start start 2024-04-22T13:31:23.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.966+0000 7f661c98d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640a23c92b0 0x5640a23d65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.966+0000 7f661c98d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640a23c9bd0 0x5640a23d2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.966+0000 7f661c98d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640a23d6e90 0x5640a23e2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.966+0000 7f661c98d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f66141ab9f0 con 0x5640a23c92b0 2024-04-22T13:31:23.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.966+0000 7f661c98d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f66141ad530 con 0x5640a23c9bd0 2024-04-22T13:31:23.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.966+0000 7f661c98d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66141ac0c0 con 0x5640a23d6e90 2024-04-22T13:31:23.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.966+0000 7f66137fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640a23c92b0 0x5640a23d65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.966+0000 7f66137fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640a23c92b0 0x5640a23d65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44986/0 (socket says 172.21.15.12:44986) 2024-04-22T13:31:23.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.966+0000 7f66137fe640 1 -- 172.21.15.12:0/171830582 learned_addr learned my addr 172.21.15.12:0/171830582 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:23.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.966+0000 7f6612ffd640 1 --2- 172.21.15.12:0/171830582 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640a23c9bd0 0x5640a23d2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.966+0000 7f66137fe640 1 -- 172.21.15.12:0/171830582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640a23d6e90 msgr2=0x5640a23e2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:23.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.966+0000 7f66137fe640 1 --2- 172.21.15.12:0/171830582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640a23d6e90 0x5640a23e2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.966+0000 7f66137fe640 1 -- 172.21.15.12:0/171830582 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640a23c9bd0 msgr2=0x5640a23d2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:23.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.966+0000 7f66137fe640 1 --2- 172.21.15.12:0/171830582 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640a23c9bd0 0x5640a23d2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.966+0000 7f66137fe640 1 -- 172.21.15.12:0/171830582 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640a23ca2d0 con 0x5640a23c92b0 2024-04-22T13:31:23.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.967+0000 7f66137fe640 1 --2- 172.21.15.12:0/171830582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640a23c92b0 0x5640a23d65a0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f6608005fd0 tx=0x7f6608098690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e873417aabd65d7b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:23.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.967+0000 7f65f97fa640 1 -- 172.21.15.12:0/171830582 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660809c300 con 0x5640a23c92b0 2024-04-22T13:31:23.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.967+0000 7f65f97fa640 1 -- 172.21.15.12:0/171830582 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f660809cc30 con 0x5640a23c92b0 2024-04-22T13:31:23.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.967+0000 7f65f97fa640 1 -- 172.21.15.12:0/171830582 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66080ce0c0 con 0x5640a23c92b0 2024-04-22T13:31:23.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.967+0000 7f661c98d740 1 -- 172.21.15.12:0/171830582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640a23c92b0 msgr2=0x5640a23d65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:23.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.967+0000 7f661c98d740 1 --2- 172.21.15.12:0/171830582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640a23c92b0 0x5640a23d65a0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f6608005fd0 tx=0x7f6608098690 comp rx=0 tx=0).stop 2024-04-22T13:31:23.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.967+0000 7f661c98d740 1 -- 172.21.15.12:0/171830582 shutdown_connections 2024-04-22T13:31:23.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.967+0000 7f661c98d740 1 --2- 172.21.15.12:0/171830582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640a23d6e90 0x5640a23e2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.967+0000 7f661c98d740 1 --2- 172.21.15.12:0/171830582 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640a23c9bd0 0x5640a23d2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.967+0000 7f661c98d740 1 --2- 172.21.15.12:0/171830582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640a23c92b0 0x5640a23d65a0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.967+0000 7f661c98d740 1 -- 172.21.15.12:0/171830582 >> 172.21.15.12:0/171830582 conn(0x5640a23c3cb0 msgr2=0x5640a23c2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:23.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.967+0000 7f661c98d740 1 -- 172.21.15.12:0/171830582 shutdown_connections 2024-04-22T13:31:23.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.967+0000 7f661c98d740 1 -- 172.21.15.12:0/171830582 wait complete. 2024-04-22T13:31:23.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.968+0000 7f661c98d740 1 -- start start 2024-04-22T13:31:23.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.968+0000 7f661c98d740 1 Processor -- start 2024-04-22T13:31:23.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.968+0000 7f661c98d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640a23c92b0 0x5640a23daa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.968+0000 7f661c98d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640a23c9bd0 0x5640a23daf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.968+0000 7f661c98d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640a23d6e90 0x5640a23db480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.968+0000 7f661c98d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f66141ace60 con 0x5640a23d6e90 2024-04-22T13:31:23.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.968+0000 7f661c98d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f66141ac790 con 0x5640a23c9bd0 2024-04-22T13:31:23.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.968+0000 7f661c98d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66141a99f0 con 0x5640a23c92b0 2024-04-22T13:31:23.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.968+0000 7f661c98d740 10 client.0 _ll_register_callbacks cb 0x7f660c037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:23.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.968+0000 7f6613fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640a23d6e90 0x5640a23db480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.968+0000 7f6613fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640a23d6e90 0x5640a23db480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44988/0 (socket says 172.21.15.12:44988) 2024-04-22T13:31:23.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.968+0000 7f6613fff640 1 -- 172.21.15.12:0/971094019 learned_addr learned my addr 172.21.15.12:0/971094019 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:23.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.968+0000 7f6612ffd640 1 --2- 172.21.15.12:0/971094019 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640a23c9bd0 0x5640a23daf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.968+0000 7f66137fe640 1 --2- 172.21.15.12:0/971094019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640a23c92b0 0x5640a23daa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.968+0000 7f6613fff640 1 -- 172.21.15.12:0/971094019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640a23c92b0 msgr2=0x5640a23daa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:23.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.968+0000 7f6613fff640 1 --2- 172.21.15.12:0/971094019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640a23c92b0 0x5640a23daa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.968+0000 7f6613fff640 1 -- 172.21.15.12:0/971094019 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640a23c9bd0 msgr2=0x5640a23daf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:23.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.969+0000 7f6613fff640 1 --2- 172.21.15.12:0/971094019 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640a23c9bd0 0x5640a23daf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.969+0000 7f6613fff640 1 -- 172.21.15.12:0/971094019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640a244bf30 con 0x5640a23d6e90 2024-04-22T13:31:23.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.969+0000 7f66137fe640 1 --2- 172.21.15.12:0/971094019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640a23c92b0 0x5640a23daa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:23.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.969+0000 7f6613fff640 1 --2- 172.21.15.12:0/971094019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640a23d6e90 0x5640a23db480 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f660c02c700 tx=0x7f660c009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467eb415b6c85d9f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:23.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.969+0000 7f65f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:23.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.969+0000 7f65f2ffd640 1 -- 172.21.15.12:0/971094019 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660c050070 con 0x5640a23d6e90 2024-04-22T13:31:23.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.969+0000 7f65f2ffd640 1 -- 172.21.15.12:0/971094019 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f660c003b70 con 0x5640a23d6e90 2024-04-22T13:31:23.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.969+0000 7f661c98d740 1 -- 172.21.15.12:0/971094019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f660809cc30 con 0x5640a23d6e90 2024-04-22T13:31:23.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.969+0000 7f65f2ffd640 1 -- 172.21.15.12:0/971094019 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f660c039040 con 0x5640a23d6e90 2024-04-22T13:31:23.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.969+0000 7f661c98d740 10 client.18339 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:23.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.969+0000 7f661c98d740 1 -- 172.21.15.12:0/971094019 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5640a244d1e0 con 0x5640a23d6e90 2024-04-22T13:31:23.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.969+0000 7f65f2ffd640 1 -- 172.21.15.12:0/971094019 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f660809cc30 con 0x5640a23d6e90 2024-04-22T13:31:23.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.969+0000 7f65f2ffd640 1 -- 172.21.15.12:0/971094019 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f660c053020 con 0x5640a23d6e90 2024-04-22T13:31:23.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.969+0000 7f661c98d740 10 client.18339 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:23.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.969+0000 7f661c98d740 10 client.18339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:23.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.969+0000 7f661c98d740 20 client.18339 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:23.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.969+0000 7f661c98d740 1 --2- 172.21.15.12:0/971094019 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5640a244d950 0x5640a244fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:23.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.969+0000 7f661c98d740 4 client.18339 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:31:23.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.969+0000 7f661c98d740 1 -- 172.21.15.12:0/971094019 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5640a22b1590 con 0x5640a244d950 2024-04-22T13:31:23.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.970+0000 7f66137fe640 1 --2- 172.21.15.12:0/971094019 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5640a244d950 0x5640a244fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:23.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.970+0000 7f65f2ffd640 10 client.18339 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:23.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.970+0000 7f66137fe640 1 --2- 172.21.15.12:0/971094019 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5640a244d950 0x5640a244fd40 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bd8854a18cd6e51 server_cookie=488abe2c776d0f5f in_seq=0 out_seq=0 2024-04-22T13:31:23.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.970+0000 7f65f2ffd640 1 -- 172.21.15.12:0/971094019 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5640a22b1590 con 0x5640a244d950 2024-04-22T13:31:23.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.970+0000 7f65f2ffd640 10 client.18339 handle_command_reply: tid=0 2024-04-22T13:31:23.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.970+0000 7f661c98d740 1 client.18339 shutdown 2024-04-22T13:31:23.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.970+0000 7f661c98d740 1 -- 172.21.15.12:0/971094019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640a23d6e90 msgr2=0x5640a23db480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:23.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.970+0000 7f661c98d740 1 --2- 172.21.15.12:0/971094019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640a23d6e90 0x5640a23db480 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f660c02c700 tx=0x7f660c009630 comp rx=0 tx=0).stop 2024-04-22T13:31:23.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.970+0000 7f661c98d740 1 -- 172.21.15.12:0/971094019 shutdown_connections 2024-04-22T13:31:23.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.970+0000 7f661c98d740 1 --2- 172.21.15.12:0/971094019 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5640a244d950 0x5640a244fd40 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.970+0000 7f661c98d740 1 --2- 172.21.15.12:0/971094019 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5640a23d6e90 0x5640a23db480 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.970+0000 7f661c98d740 1 --2- 172.21.15.12:0/971094019 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640a23c9bd0 0x5640a23daf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.970+0000 7f65f2ffd640 0 client.18339 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:23.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.970+0000 7f661c98d740 1 --2- 172.21.15.12:0/971094019 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640a23c92b0 0x5640a23daa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.970+0000 7f661c98d740 1 -- 172.21.15.12:0/971094019 >> 172.21.15.12:0/971094019 conn(0x5640a23c3cb0 msgr2=0x5640a23c24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:23.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.971+0000 7f661c98d740 1 -- 172.21.15.12:0/971094019 shutdown_connections 2024-04-22T13:31:23.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.971+0000 7f661c98d740 1 -- 172.21.15.12:0/971094019 wait complete. 2024-04-22T13:31:23.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.971+0000 7f661c98d740 20 client.18339 trim_cache size 0 max 16384 2024-04-22T13:31:23.973 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:31:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:31:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-04-22T13:31:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "4f57d9e4-838d-457e-9dee-166f8b4853ce": { 2024-04-22T13:31:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-04-22T13:31:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "4f57d9e4-838d-457e-9dee-166f8b4853ce", 2024-04-22T13:31:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-04-22T13:31:23.973 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:31:23.974 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:31:23.974 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:31:23.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.971+0000 7f6619cc1640 1 -- 172.21.15.12:0/2931984075 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f65fc0a3d20 msgr2=0x7f65fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:23.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.971+0000 7f6619cc1640 1 --2- 172.21.15.12:0/2931984075 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f65fc0a3d20 0x7f65fc0a61e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f660c006010 tx=0x7f660c02e040 comp rx=0 tx=0).stop 2024-04-22T13:31:23.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.971+0000 7f6619cc1640 1 -- 172.21.15.12:0/2931984075 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f661406a5b0 msgr2=0x7f66141986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:23.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.971+0000 7f6619cc1640 1 --2- 172.21.15.12:0/2931984075 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f661406a5b0 0x7f66141986c0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f660800c890 tx=0x7f660800cd50 comp rx=0 tx=0).stop 2024-04-22T13:31:23.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.971+0000 7f6619cc1640 1 -- 172.21.15.12:0/2931984075 shutdown_connections 2024-04-22T13:31:23.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.971+0000 7f6619cc1640 1 --2- 172.21.15.12:0/2931984075 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f65fc0a3d20 0x7f65fc0a61e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.971+0000 7f6619cc1640 1 --2- 172.21.15.12:0/2931984075 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6614108060 0x7f661419cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.971+0000 7f6619cc1640 1 --2- 172.21.15.12:0/2931984075 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f661406af70 0x7f6614198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.971+0000 7f6619cc1640 1 --2- 172.21.15.12:0/2931984075 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f661406a5b0 0x7f66141986c0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:23.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.971+0000 7f6619cc1640 1 -- 172.21.15.12:0/2931984075 >> 172.21.15.12:0/2931984075 conn(0x7f6614077c40 msgr2=0x7f66140ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:23.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.971+0000 7f6619cc1640 1 -- 172.21.15.12:0/2931984075 shutdown_connections 2024-04-22T13:31:23.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:23.971+0000 7f6619cc1640 1 -- 172.21.15.12:0/2931984075 wait complete. 2024-04-22T13:31:23.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:23 smithi111 ceph-mon[31361]: pgmap v223: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-22T13:31:23.977 INFO:tasks.cephfs.filesystem:scrub status for tag:4f57d9e4-838d-457e-9dee-166f8b4853ce - {'path': '/', 'tag': '4f57d9e4-838d-457e-9dee-166f8b4853ce', 'options': 'recursive,force'} 2024-04-22T13:31:25.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:25 smithi012 ceph-mon[28265]: scrub queued for path: / 2024-04-22T13:31:25.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:25 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:31:25.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:25 smithi012 ceph-mon[28265]: scrub summary: active paths [/] 2024-04-22T13:31:25.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:25 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:31:25.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:25 smithi012 ceph-mon[28265]: pgmap v224: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-22T13:31:25.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:25 smithi012 ceph-mon[28265]: scrub summary: idle 2024-04-22T13:31:25.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:25 smithi134 ceph-mon[31338]: scrub queued for path: / 2024-04-22T13:31:25.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:25 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:31:25.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:25 smithi134 ceph-mon[31338]: scrub summary: active paths [/] 2024-04-22T13:31:25.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:25 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:31:25.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:25 smithi134 ceph-mon[31338]: pgmap v224: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-22T13:31:25.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:25 smithi134 ceph-mon[31338]: scrub summary: idle 2024-04-22T13:31:25.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:25 smithi111 ceph-mon[31361]: scrub queued for path: / 2024-04-22T13:31:25.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:25 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:31:25.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:25 smithi111 ceph-mon[31361]: scrub summary: active paths [/] 2024-04-22T13:31:25.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:25 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:31:25.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:25 smithi111 ceph-mon[31361]: pgmap v224: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-22T13:31:25.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:25 smithi111 ceph-mon[31361]: scrub summary: idle 2024-04-22T13:31:27.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:27 smithi012 ceph-mon[28265]: pgmap v225: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 86 B/s wr, 8 op/s 2024-04-22T13:31:27.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:27 smithi134 ceph-mon[31338]: pgmap v225: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 86 B/s wr, 8 op/s 2024-04-22T13:31:27.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:27 smithi111 ceph-mon[31361]: pgmap v225: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 86 B/s wr, 8 op/s 2024-04-22T13:31:29.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:29 smithi012 ceph-mon[28265]: pgmap v226: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-22T13:31:29.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:29 smithi134 ceph-mon[31338]: pgmap v226: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-22T13:31:29.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:29 smithi111 ceph-mon[31361]: pgmap v226: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-22T13:31:31.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:31 smithi012 ceph-mon[28265]: pgmap v227: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:31:31.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:31 smithi134 ceph-mon[31338]: pgmap v227: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:31:31.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:31 smithi111 ceph-mon[31361]: pgmap v227: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:31:33.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:33 smithi012 ceph-mon[28265]: pgmap v228: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:31:33.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:33 smithi134 ceph-mon[31338]: pgmap v228: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:31:33.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:33 smithi111 ceph-mon[31361]: pgmap v228: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:31:35.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:35 smithi012 ceph-mon[28265]: pgmap v229: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:31:35.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:35 smithi134 ceph-mon[31338]: pgmap v229: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:31:35.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:35 smithi111 ceph-mon[31361]: pgmap v229: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:31:37.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:37 smithi012 ceph-mon[28265]: pgmap v230: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:31:37.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:37 smithi134 ceph-mon[31338]: pgmap v230: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:31:37.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:37 smithi111 ceph-mon[31361]: pgmap v230: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:31:39.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:39 smithi012 ceph-mon[28265]: pgmap v231: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 13 op/s 2024-04-22T13:31:39.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:39 smithi134 ceph-mon[31338]: pgmap v231: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 13 op/s 2024-04-22T13:31:39.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:39 smithi111 ceph-mon[31361]: pgmap v231: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 13 op/s 2024-04-22T13:31:41.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:41 smithi012 ceph-mon[28265]: pgmap v232: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:31:41.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:41 smithi134 ceph-mon[31338]: pgmap v232: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:31:41.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:41 smithi111 ceph-mon[31361]: pgmap v232: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:31:43.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:43 smithi012 ceph-mon[28265]: pgmap v233: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:31:43.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:43 smithi134 ceph-mon[31338]: pgmap v233: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:31:43.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:43 smithi111 ceph-mon[31361]: pgmap v233: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:31:45.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:45 smithi012 ceph-mon[28265]: pgmap v234: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:31:45.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:45 smithi134 ceph-mon[31338]: pgmap v234: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:31:45.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:45 smithi111 ceph-mon[31361]: pgmap v234: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:31:47.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:47 smithi012 ceph-mon[28265]: pgmap v235: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:31:47.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:47 smithi134 ceph-mon[31338]: pgmap v235: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:31:47.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:47 smithi111 ceph-mon[31361]: pgmap v235: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:31:49.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:49 smithi012 ceph-mon[28265]: pgmap v236: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-04-22T13:31:49.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:49 smithi134 ceph-mon[31338]: pgmap v236: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-04-22T13:31:49.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:49 smithi111 ceph-mon[31361]: pgmap v236: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-04-22T13:31:51.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:51 smithi012 ceph-mon[28265]: pgmap v237: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:31:51.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:51 smithi134 ceph-mon[31338]: pgmap v237: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:31:51.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:51 smithi111 ceph-mon[31361]: pgmap v237: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:31:53.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:53 smithi012 ceph-mon[28265]: pgmap v238: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:31:53.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:53 smithi134 ceph-mon[31338]: pgmap v238: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:31:53.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:53 smithi111 ceph-mon[31361]: pgmap v238: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:31:53.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:31:53.979 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:31:54.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.101+0000 7fbd18d0d640 1 -- 172.21.15.12:0/48527043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd14100d40 msgr2=0x7fbd14101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.101+0000 7fbd18d0d640 1 --2- 172.21.15.12:0/48527043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd14100d40 0x7fbd14101120 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fbd04009900 tx=0x7fbd0402bc30 comp rx=0 tx=0).stop 2024-04-22T13:31:54.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.101+0000 7fbd18d0d640 1 -- 172.21.15.12:0/48527043 shutdown_connections 2024-04-22T13:31:54.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.101+0000 7fbd18d0d640 1 --2- 172.21.15.12:0/48527043 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd1410e8f0 0x7fbd14110ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.101+0000 7fbd18d0d640 1 --2- 172.21.15.12:0/48527043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd14101660 0x7fbd1410e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.101+0000 7fbd18d0d640 1 --2- 172.21.15.12:0/48527043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd14100d40 0x7fbd14101120 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.101+0000 7fbd18d0d640 1 -- 172.21.15.12:0/48527043 >> 172.21.15.12:0/48527043 conn(0x7fbd14078000 msgr2=0x7fbd14078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:54.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.101+0000 7fbd18d0d640 1 -- 172.21.15.12:0/48527043 shutdown_connections 2024-04-22T13:31:54.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.101+0000 7fbd18d0d640 1 -- 172.21.15.12:0/48527043 wait complete. 2024-04-22T13:31:54.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.102+0000 7fbd18d0d640 1 Processor -- start 2024-04-22T13:31:54.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.102+0000 7fbd18d0d640 1 -- start start 2024-04-22T13:31:54.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.102+0000 7fbd18d0d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd14100d40 0x7fbd141a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.102+0000 7fbd18d0d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd14101660 0x7fbd141a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.102+0000 7fbd18d0d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd1410e8f0 0x7fbd1419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.102+0000 7fbd18d0d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbd141143b0 con 0x7fbd14100d40 2024-04-22T13:31:54.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.102+0000 7fbd18d0d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbd14114230 con 0x7fbd14101660 2024-04-22T13:31:54.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.102+0000 7fbd18d0d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbd14114530 con 0x7fbd1410e8f0 2024-04-22T13:31:54.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.102+0000 7fbd1259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd14100d40 0x7fbd141a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.102+0000 7fbd1259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd14100d40 0x7fbd141a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60720/0 (socket says 172.21.15.12:60720) 2024-04-22T13:31:54.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.102+0000 7fbd1259c640 1 -- 172.21.15.12:0/3113801024 learned_addr learned my addr 172.21.15.12:0/3113801024 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:54.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.102+0000 7fbd11d9b640 1 --2- 172.21.15.12:0/3113801024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd14101660 0x7fbd141a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.102+0000 7fbd12d9d640 1 --2- 172.21.15.12:0/3113801024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd1410e8f0 0x7fbd1419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.102+0000 7fbd1259c640 1 -- 172.21.15.12:0/3113801024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd1410e8f0 msgr2=0x7fbd1419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.102+0000 7fbd1259c640 1 --2- 172.21.15.12:0/3113801024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd1410e8f0 0x7fbd1419a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.102+0000 7fbd1259c640 1 -- 172.21.15.12:0/3113801024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd14101660 msgr2=0x7fbd141a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.102+0000 7fbd1259c640 1 --2- 172.21.15.12:0/3113801024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd14101660 0x7fbd141a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.102+0000 7fbd1259c640 1 -- 172.21.15.12:0/3113801024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd1419af90 con 0x7fbd14100d40 2024-04-22T13:31:54.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.103+0000 7fbd1259c640 1 --2- 172.21.15.12:0/3113801024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd14100d40 0x7fbd141a0610 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fbd04005940 tx=0x7fbd040040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:54.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.103+0000 7fbd037fe640 1 -- 172.21.15.12:0/3113801024 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd04043020 con 0x7fbd14100d40 2024-04-22T13:31:54.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.103+0000 7fbd037fe640 1 -- 172.21.15.12:0/3113801024 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd040052b0 con 0x7fbd14100d40 2024-04-22T13:31:54.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.103+0000 7fbd18d0d640 1 -- 172.21.15.12:0/3113801024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd1419b220 con 0x7fbd14100d40 2024-04-22T13:31:54.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.103+0000 7fbd037fe640 1 -- 172.21.15.12:0/3113801024 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd04003c00 con 0x7fbd14100d40 2024-04-22T13:31:54.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.103+0000 7fbd18d0d640 1 -- 172.21.15.12:0/3113801024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd141a7590 con 0x7fbd14100d40 2024-04-22T13:31:54.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.105+0000 7fbd037fe640 1 -- 172.21.15.12:0/3113801024 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbd04046020 con 0x7fbd14100d40 2024-04-22T13:31:54.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.105+0000 7fbd1b9d0740 1 Processor -- start 2024-04-22T13:31:54.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.106+0000 7fbd1b9d0740 1 -- start start 2024-04-22T13:31:54.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.106+0000 7fbd1b9d0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd482c1580 0x55bd482c1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.106+0000 7fbd1b9d0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd482c1ea0 0x55bd482ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.106+0000 7fbd1b9d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bd482caa80 0x55bd482cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.106+0000 7fbd1b9d0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55bd482261e0 con 0x55bd482c1ea0 2024-04-22T13:31:54.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.106+0000 7fbd1b9d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55bd481c3390 con 0x55bd482caa80 2024-04-22T13:31:54.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.106+0000 7fbd1b9d0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bd482221a0 con 0x55bd482c1580 2024-04-22T13:31:54.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.106+0000 7fbd037fe640 1 --2- 172.21.15.12:0/3113801024 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbcfc0a3d20 0x7fbcfc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.106+0000 7fbd1259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd482c1ea0 0x55bd482ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.106+0000 7fbd1259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd482c1ea0 0x55bd482ca540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60726/0 (socket says 172.21.15.12:60726) 2024-04-22T13:31:54.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.106+0000 7fbd1259c640 1 -- 172.21.15.12:0/1501835457 learned_addr learned my addr 172.21.15.12:0/1501835457 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:54.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.106+0000 7fbd11d9b640 1 --2- 172.21.15.12:0/1501835457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bd482caa80 0x55bd482cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.106+0000 7fbd12d9d640 1 --2- 172.21.15.12:0/1501835457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd482c1580 0x55bd482c1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.106+0000 7fbd12d9d640 1 --2- 172.21.15.12:0/3113801024 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbcfc0a3d20 0x7fbcfc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.106+0000 7fbd037fe640 1 -- 172.21.15.12:0/3113801024 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fbd040f2ac0 con 0x7fbd14100d40 2024-04-22T13:31:54.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.106+0000 7fbd11d9b640 1 -- 172.21.15.12:0/1501835457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd482c1580 msgr2=0x55bd482c1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.106+0000 7fbd11d9b640 1 --2- 172.21.15.12:0/1501835457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd482c1580 0x55bd482c1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.106+0000 7fbd11d9b640 1 -- 172.21.15.12:0/1501835457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd482c1ea0 msgr2=0x55bd482ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.106+0000 7fbd11d9b640 1 --2- 172.21.15.12:0/1501835457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd482c1ea0 0x55bd482ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.106+0000 7fbd11d9b640 1 -- 172.21.15.12:0/1501835457 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd480027d0 con 0x55bd482caa80 2024-04-22T13:31:54.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.106+0000 7fbd1259c640 1 --2- 172.21.15.12:0/1501835457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd482c1ea0 0x55bd482ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:31:54.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.107+0000 7fbd12d9d640 1 --2- 172.21.15.12:0/3113801024 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbcfc0a3d20 0x7fbcfc0a61e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c005cc0 tx=0x7fbd0c00a270 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:54.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.107+0000 7fbd11d9b640 1 --2- 172.21.15.12:0/1501835457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bd482caa80 0x55bd482cce70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbd0800abc0 tx=0x7fbd0800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=145ef1f8f944b386 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:54.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.107+0000 7fbd00ff9640 1 -- 172.21.15.12:0/1501835457 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd08005220 con 0x55bd482caa80 2024-04-22T13:31:54.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.107+0000 7fbd00ff9640 1 -- 172.21.15.12:0/1501835457 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd08004c50 con 0x55bd482caa80 2024-04-22T13:31:54.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.107+0000 7fbd00ff9640 1 -- 172.21.15.12:0/1501835457 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd08018840 con 0x55bd482caa80 2024-04-22T13:31:54.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.108+0000 7fbd1b9d0740 1 -- 172.21.15.12:0/1501835457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bd482caa80 msgr2=0x55bd482cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.108+0000 7fbd1b9d0740 1 --2- 172.21.15.12:0/1501835457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bd482caa80 0x55bd482cce70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbd0800abc0 tx=0x7fbd0800f790 comp rx=0 tx=0).stop 2024-04-22T13:31:54.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.108+0000 7fbd1b9d0740 1 -- 172.21.15.12:0/1501835457 shutdown_connections 2024-04-22T13:31:54.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.108+0000 7fbd1b9d0740 1 --2- 172.21.15.12:0/1501835457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bd482caa80 0x55bd482cce70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.108+0000 7fbd1b9d0740 1 --2- 172.21.15.12:0/1501835457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd482c1ea0 0x55bd482ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.108+0000 7fbd1b9d0740 1 --2- 172.21.15.12:0/1501835457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd482c1580 0x55bd482c1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.108+0000 7fbd1b9d0740 1 -- 172.21.15.12:0/1501835457 >> 172.21.15.12:0/1501835457 conn(0x55bd482bbcb0 msgr2=0x55bd482bc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:54.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.108+0000 7fbd1b9d0740 1 -- 172.21.15.12:0/1501835457 shutdown_connections 2024-04-22T13:31:54.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.108+0000 7fbd1b9d0740 1 -- 172.21.15.12:0/1501835457 wait complete. 2024-04-22T13:31:54.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.108+0000 7fbd1b9d0740 1 -- start start 2024-04-22T13:31:54.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.108+0000 7fbd1b9d0740 1 Processor -- start 2024-04-22T13:31:54.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.108+0000 7fbd1b9d0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd482c1580 0x55bd482c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.108+0000 7fbd1b9d0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd482c1ea0 0x55bd482c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.108+0000 7fbd1b9d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bd482caa80 0x55bd482c8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.109+0000 7fbd1b9d0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55bd482cd870 con 0x55bd482c1580 2024-04-22T13:31:54.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.109+0000 7fbd1b9d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55bd481df2f0 con 0x55bd482caa80 2024-04-22T13:31:54.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.109+0000 7fbd1b9d0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bd482cd9f0 con 0x55bd482c1ea0 2024-04-22T13:31:54.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.109+0000 7fbd1b9d0740 10 client.0 _ll_register_callbacks cb 0x55bd481ec580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:54.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.109+0000 7fbd1259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd482c1580 0x55bd482c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.109+0000 7fbd1259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd482c1580 0x55bd482c7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60738/0 (socket says 172.21.15.12:60738) 2024-04-22T13:31:54.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.109+0000 7fbd1259c640 1 -- 172.21.15.12:0/222466441 learned_addr learned my addr 172.21.15.12:0/222466441 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:54.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.109+0000 7fbd12d9d640 1 --2- 172.21.15.12:0/222466441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bd482caa80 0x55bd482c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.109+0000 7fbd11d9b640 1 --2- 172.21.15.12:0/222466441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd482c1ea0 0x55bd482c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.109+0000 7fbd1259c640 1 -- 172.21.15.12:0/222466441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd482c1ea0 msgr2=0x55bd482c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.109+0000 7fbd1259c640 1 --2- 172.21.15.12:0/222466441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd482c1ea0 0x55bd482c7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.109+0000 7fbd1259c640 1 -- 172.21.15.12:0/222466441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bd482caa80 msgr2=0x55bd482c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.109+0000 7fbd1259c640 1 --2- 172.21.15.12:0/222466441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bd482caa80 0x55bd482c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.109+0000 7fbd1259c640 1 -- 172.21.15.12:0/222466441 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd47f383f0 con 0x55bd482c1580 2024-04-22T13:31:54.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.109+0000 7fbd1259c640 1 --2- 172.21.15.12:0/222466441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd482c1580 0x55bd482c7a00 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fbd04005ec0 tx=0x7fbd04005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6a8d401b163e9e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:54.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.109+0000 7fbcf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:54.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.109+0000 7fbcf9ffb640 1 -- 172.21.15.12:0/222466441 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd040be4f0 con 0x55bd482c1580 2024-04-22T13:31:54.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.109+0000 7fbcf9ffb640 1 -- 172.21.15.12:0/222466441 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fbd040df0b0 con 0x55bd482c1580 2024-04-22T13:31:54.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.109+0000 7fbd1b9d0740 1 -- 172.21.15.12:0/222466441 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd08004c50 con 0x55bd482c1580 2024-04-22T13:31:54.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.109+0000 7fbcf9ffb640 1 -- 172.21.15.12:0/222466441 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbd040beb90 con 0x55bd482c1580 2024-04-22T13:31:54.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.109+0000 7fbd1b9d0740 10 client.18351 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:54.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.110+0000 7fbd1b9d0740 1 -- 172.21.15.12:0/222466441 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bd47ef80f0 con 0x55bd482c1580 2024-04-22T13:31:54.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.110+0000 7fbcf9ffb640 1 -- 172.21.15.12:0/222466441 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd08004c50 con 0x55bd482c1580 2024-04-22T13:31:54.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.110+0000 7fbcf9ffb640 1 -- 172.21.15.12:0/222466441 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbd040bb200 con 0x55bd482c1580 2024-04-22T13:31:54.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.110+0000 7fbd1b9d0740 10 client.18351 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:54.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.110+0000 7fbd1b9d0740 10 client.18351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:54.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.110+0000 7fbd1b9d0740 20 client.18351 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:54.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.110+0000 7fbd1b9d0740 1 --2- 172.21.15.12:0/222466441 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55bd482da9f0 0x55bd482dceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.110+0000 7fbd1b9d0740 4 client.18351 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:54.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.110+0000 7fbd1b9d0740 1 -- 172.21.15.12:0/222466441 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd48233c00 con 0x55bd482da9f0 2024-04-22T13:31:54.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.110+0000 7fbd11d9b640 1 --2- 172.21.15.12:0/222466441 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55bd482da9f0 0x55bd482dceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.111+0000 7fbd11d9b640 1 --2- 172.21.15.12:0/222466441 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55bd482da9f0 0x55bd482dceb0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67cdaf35c21eb8af server_cookie=8f701a5a53e64c56 in_seq=0 out_seq=0 2024-04-22T13:31:54.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.111+0000 7fbcf9ffb640 10 client.18351 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:54.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.111+0000 7fbcf9ffb640 1 -- 172.21.15.12:0/222466441 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bd48233c00 con 0x55bd482da9f0 2024-04-22T13:31:54.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.111+0000 7fbcf9ffb640 10 client.18351 handle_command_reply: tid=0 2024-04-22T13:31:54.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.111+0000 7fbd1b9d0740 1 client.18351 shutdown 2024-04-22T13:31:54.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.111+0000 7fbd1b9d0740 1 -- 172.21.15.12:0/222466441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd482c1580 msgr2=0x55bd482c7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.111+0000 7fbd1b9d0740 1 --2- 172.21.15.12:0/222466441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd482c1580 0x55bd482c7a00 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fbd04005ec0 tx=0x7fbd04005b50 comp rx=0 tx=0).stop 2024-04-22T13:31:54.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.111+0000 7fbd1b9d0740 1 -- 172.21.15.12:0/222466441 shutdown_connections 2024-04-22T13:31:54.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.111+0000 7fbd1b9d0740 1 --2- 172.21.15.12:0/222466441 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55bd482da9f0 0x55bd482dceb0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.111+0000 7fbd1b9d0740 1 --2- 172.21.15.12:0/222466441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bd482caa80 0x55bd482c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.111+0000 7fbd1b9d0740 1 --2- 172.21.15.12:0/222466441 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd482c1ea0 0x55bd482c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.111+0000 7fbcf9ffb640 0 client.18351 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:54.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.111+0000 7fbd1b9d0740 1 --2- 172.21.15.12:0/222466441 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd482c1580 0x55bd482c7a00 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.111+0000 7fbd1b9d0740 1 -- 172.21.15.12:0/222466441 >> 172.21.15.12:0/222466441 conn(0x55bd482bbcb0 msgr2=0x55bd482bda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:54.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.112+0000 7fbd1b9d0740 1 -- 172.21.15.12:0/222466441 shutdown_connections 2024-04-22T13:31:54.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.112+0000 7fbd1b9d0740 1 -- 172.21.15.12:0/222466441 wait complete. 2024-04-22T13:31:54.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.112+0000 7fbd1b9d0740 20 client.18351 trim_cache size 0 max 16384 2024-04-22T13:31:54.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.130+0000 7fbd1b9d0740 1 Processor -- start 2024-04-22T13:31:54.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.130+0000 7fbd1b9d0740 1 -- start start 2024-04-22T13:31:54.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.130+0000 7fbd1b9d0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd482c1580 0x55bd482d6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.130+0000 7fbd1b9d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bd482c1ea0 0x55bd482c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.130+0000 7fbd1b9d0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd482caa80 0x55bd482c8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.130+0000 7fbd1b9d0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbd141afb70 con 0x55bd482caa80 2024-04-22T13:31:54.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.130+0000 7fbd1b9d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbd141b16b0 con 0x55bd482c1ea0 2024-04-22T13:31:54.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.130+0000 7fbd1b9d0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbd141b0240 con 0x55bd482c1580 2024-04-22T13:31:54.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.130+0000 7fbd12d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd482caa80 0x55bd482c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.130+0000 7fbd12d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd482caa80 0x55bd482c8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60754/0 (socket says 172.21.15.12:60754) 2024-04-22T13:31:54.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.130+0000 7fbd1259c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd482c1580 0x55bd482d6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.130+0000 7fbd12d9d640 1 -- 172.21.15.12:0/13997655 learned_addr learned my addr 172.21.15.12:0/13997655 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:54.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.130+0000 7fbd11d9b640 1 --2- 172.21.15.12:0/13997655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bd482c1ea0 0x55bd482c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.130+0000 7fbd12d9d640 1 -- 172.21.15.12:0/13997655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd482c1580 msgr2=0x55bd482d6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.130+0000 7fbd12d9d640 1 --2- 172.21.15.12:0/13997655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd482c1580 0x55bd482d6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.130+0000 7fbd12d9d640 1 -- 172.21.15.12:0/13997655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bd482c1ea0 msgr2=0x55bd482c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.130+0000 7fbd12d9d640 1 --2- 172.21.15.12:0/13997655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bd482c1ea0 0x55bd482c80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.130+0000 7fbd12d9d640 1 -- 172.21.15.12:0/13997655 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd482c5f50 con 0x55bd482caa80 2024-04-22T13:31:54.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.130+0000 7fbd11d9b640 1 --2- 172.21.15.12:0/13997655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bd482c1ea0 0x55bd482c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:54.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.130+0000 7fbd1259c640 1 --2- 172.21.15.12:0/13997655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd482c1580 0x55bd482d6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:31:54.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.131+0000 7fbd12d9d640 1 --2- 172.21.15.12:0/13997655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd482caa80 0x55bd482c8610 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c005fd0 tx=0x7fbd0c00a500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107c513b61814043 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:54.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.131+0000 7fbcfbfff640 1 -- 172.21.15.12:0/13997655 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c004030 con 0x55bd482caa80 2024-04-22T13:31:54.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.131+0000 7fbcfbfff640 1 -- 172.21.15.12:0/13997655 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd0c004680 con 0x55bd482caa80 2024-04-22T13:31:54.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.131+0000 7fbcfbfff640 1 -- 172.21.15.12:0/13997655 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c021cd0 con 0x55bd482caa80 2024-04-22T13:31:54.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.131+0000 7fbd1b9d0740 1 -- 172.21.15.12:0/13997655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd482caa80 msgr2=0x55bd482c8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.131+0000 7fbd1b9d0740 1 --2- 172.21.15.12:0/13997655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd482caa80 0x55bd482c8610 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c005fd0 tx=0x7fbd0c00a500 comp rx=0 tx=0).stop 2024-04-22T13:31:54.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.131+0000 7fbd1b9d0740 1 -- 172.21.15.12:0/13997655 shutdown_connections 2024-04-22T13:31:54.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.131+0000 7fbd1b9d0740 1 --2- 172.21.15.12:0/13997655 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd482caa80 0x55bd482c8610 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.131+0000 7fbd1b9d0740 1 --2- 172.21.15.12:0/13997655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bd482c1ea0 0x55bd482c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.131+0000 7fbd1b9d0740 1 --2- 172.21.15.12:0/13997655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd482c1580 0x55bd482d6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.131+0000 7fbd1b9d0740 1 -- 172.21.15.12:0/13997655 >> 172.21.15.12:0/13997655 conn(0x55bd482bbcb0 msgr2=0x55bd482c79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:54.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.131+0000 7fbd1b9d0740 1 -- 172.21.15.12:0/13997655 shutdown_connections 2024-04-22T13:31:54.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.131+0000 7fbd1b9d0740 1 -- 172.21.15.12:0/13997655 wait complete. 2024-04-22T13:31:54.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.132+0000 7fbd1b9d0740 1 -- start start 2024-04-22T13:31:54.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.132+0000 7fbd1b9d0740 1 Processor -- start 2024-04-22T13:31:54.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.132+0000 7fbd1b9d0740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd482c1580 0x55bd482cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.132+0000 7fbd1b9d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bd482c1ea0 0x55bd482cf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.132+0000 7fbd1b9d0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd482caa80 0x55bd48340030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.132+0000 7fbd1b9d0740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbd141b0fe0 con 0x55bd482c1580 2024-04-22T13:31:54.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.132+0000 7fbd1b9d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbd141b0910 con 0x55bd482c1ea0 2024-04-22T13:31:54.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.132+0000 7fbd1b9d0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbd141adb70 con 0x55bd482caa80 2024-04-22T13:31:54.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.132+0000 7fbd1b9d0740 10 client.0 _ll_register_callbacks cb 0x7fbd040be7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:54.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.132+0000 7fbd1259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd482c1580 0x55bd482cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.132+0000 7fbd1259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd482c1580 0x55bd482cedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60760/0 (socket says 172.21.15.12:60760) 2024-04-22T13:31:54.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.132+0000 7fbd1259c640 1 -- 172.21.15.12:0/129271426 learned_addr learned my addr 172.21.15.12:0/129271426 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:54.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.132+0000 7fbd12d9d640 1 --2- 172.21.15.12:0/129271426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd482caa80 0x55bd48340030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.132+0000 7fbd11d9b640 1 --2- 172.21.15.12:0/129271426 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bd482c1ea0 0x55bd482cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.132+0000 7fbd1259c640 1 -- 172.21.15.12:0/129271426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd482caa80 msgr2=0x55bd48340030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.132+0000 7fbd1259c640 1 --2- 172.21.15.12:0/129271426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd482caa80 0x55bd48340030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.132+0000 7fbd1259c640 1 -- 172.21.15.12:0/129271426 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bd482c1ea0 msgr2=0x55bd482cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.132+0000 7fbd1259c640 1 --2- 172.21.15.12:0/129271426 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bd482c1ea0 0x55bd482cf300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.132+0000 7fbd1259c640 1 -- 172.21.15.12:0/129271426 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd48340570 con 0x55bd482c1580 2024-04-22T13:31:54.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.132+0000 7fbd11d9b640 1 --2- 172.21.15.12:0/129271426 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bd482c1ea0 0x55bd482cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:54.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.133+0000 7fbd1259c640 1 --2- 172.21.15.12:0/129271426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd482c1580 0x55bd482cedc0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fbd04003fd0 tx=0x7fbd040c7370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c28c4e620b82d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:54.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.133+0000 7fbcf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:54.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.133+0000 7fbcf9ffb640 1 -- 172.21.15.12:0/129271426 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd040bf290 con 0x55bd482c1580 2024-04-22T13:31:54.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.133+0000 7fbcf9ffb640 1 -- 172.21.15.12:0/129271426 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fbd040df0d0 con 0x55bd482c1580 2024-04-22T13:31:54.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.133+0000 7fbd1b9d0740 1 -- 172.21.15.12:0/129271426 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd0c004680 con 0x55bd482c1580 2024-04-22T13:31:54.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.133+0000 7fbcf9ffb640 1 -- 172.21.15.12:0/129271426 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbd040c2f40 con 0x55bd482c1580 2024-04-22T13:31:54.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.133+0000 7fbd1b9d0740 10 client.18357 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:54.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.133+0000 7fbd1b9d0740 1 -- 172.21.15.12:0/129271426 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bd48340570 con 0x55bd482c1580 2024-04-22T13:31:54.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.133+0000 7fbcf9ffb640 1 -- 172.21.15.12:0/129271426 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd0c004680 con 0x55bd482c1580 2024-04-22T13:31:54.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.133+0000 7fbcf9ffb640 1 -- 172.21.15.12:0/129271426 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbd040ba8e0 con 0x55bd482c1580 2024-04-22T13:31:54.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.133+0000 7fbd1b9d0740 10 client.18357 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:54.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.133+0000 7fbd1b9d0740 10 client.18357 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:54.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.133+0000 7fbd1b9d0740 20 client.18357 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:54.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.133+0000 7fbd1b9d0740 1 --2- 172.21.15.12:0/129271426 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55bd48341be0 0x55bd48344000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.133+0000 7fbd1b9d0740 4 client.18357 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:31:54.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.133+0000 7fbd1b9d0740 1 -- 172.21.15.12:0/129271426 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bd4822da70 con 0x55bd48341be0 2024-04-22T13:31:54.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.134+0000 7fbd11d9b640 1 --2- 172.21.15.12:0/129271426 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55bd48341be0 0x55bd48344000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.134+0000 7fbd11d9b640 1 --2- 172.21.15.12:0/129271426 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55bd48341be0 0x55bd48344000 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e8a5e1be2295223 server_cookie=ed516965c27b2d67 in_seq=0 out_seq=0 2024-04-22T13:31:54.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.134+0000 7fbcf9ffb640 10 client.18357 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:54.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.134+0000 7fbcf9ffb640 1 -- 172.21.15.12:0/129271426 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bd4822da70 con 0x55bd48341be0 2024-04-22T13:31:54.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.134+0000 7fbcf9ffb640 10 client.18357 handle_command_reply: tid=0 2024-04-22T13:31:54.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.134+0000 7fbd1b9d0740 1 client.18357 shutdown 2024-04-22T13:31:54.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.134+0000 7fbd1b9d0740 1 -- 172.21.15.12:0/129271426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd482c1580 msgr2=0x55bd482cedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.134+0000 7fbd1b9d0740 1 --2- 172.21.15.12:0/129271426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd482c1580 0x55bd482cedc0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fbd04003fd0 tx=0x7fbd040c7370 comp rx=0 tx=0).stop 2024-04-22T13:31:54.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.134+0000 7fbd1b9d0740 1 -- 172.21.15.12:0/129271426 shutdown_connections 2024-04-22T13:31:54.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.134+0000 7fbd1b9d0740 1 --2- 172.21.15.12:0/129271426 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55bd48341be0 0x55bd48344000 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.134+0000 7fbd1b9d0740 1 --2- 172.21.15.12:0/129271426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bd482caa80 0x55bd48340030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.134+0000 7fbd1b9d0740 1 --2- 172.21.15.12:0/129271426 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bd482c1ea0 0x55bd482cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.134+0000 7fbd1b9d0740 1 --2- 172.21.15.12:0/129271426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bd482c1580 0x55bd482cedc0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.134+0000 7fbcf9ffb640 0 client.18357 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:54.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.134+0000 7fbd1b9d0740 1 -- 172.21.15.12:0/129271426 >> 172.21.15.12:0/129271426 conn(0x55bd482bbcb0 msgr2=0x55bd482bad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:54.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.134+0000 7fbd1b9d0740 1 -- 172.21.15.12:0/129271426 shutdown_connections 2024-04-22T13:31:54.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.134+0000 7fbd1b9d0740 1 -- 172.21.15.12:0/129271426 wait complete. 2024-04-22T13:31:54.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.134+0000 7fbd1b9d0740 20 client.18357 trim_cache size 0 max 16384 2024-04-22T13:31:54.138 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:31:54.138 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-04-22T13:31:54.138 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-04-22T13:31:54.139 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:31:54.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.135+0000 7fbd18d0d640 1 -- 172.21.15.12:0/3113801024 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbcfc0a3d20 msgr2=0x7fbcfc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.135+0000 7fbd18d0d640 1 --2- 172.21.15.12:0/3113801024 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbcfc0a3d20 0x7fbcfc0a61e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c005cc0 tx=0x7fbd0c00a270 comp rx=0 tx=0).stop 2024-04-22T13:31:54.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.135+0000 7fbd18d0d640 1 -- 172.21.15.12:0/3113801024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd14100d40 msgr2=0x7fbd141a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.135+0000 7fbd18d0d640 1 --2- 172.21.15.12:0/3113801024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd14100d40 0x7fbd141a0610 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fbd04005940 tx=0x7fbd040040e0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.135+0000 7fbd18d0d640 1 -- 172.21.15.12:0/3113801024 shutdown_connections 2024-04-22T13:31:54.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.135+0000 7fbd18d0d640 1 --2- 172.21.15.12:0/3113801024 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbcfc0a3d20 0x7fbcfc0a61e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.135+0000 7fbd18d0d640 1 --2- 172.21.15.12:0/3113801024 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd1410e8f0 0x7fbd1419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.135+0000 7fbd18d0d640 1 --2- 172.21.15.12:0/3113801024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd14101660 0x7fbd141a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.135+0000 7fbd18d0d640 1 --2- 172.21.15.12:0/3113801024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd14100d40 0x7fbd141a0610 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.135+0000 7fbd18d0d640 1 -- 172.21.15.12:0/3113801024 >> 172.21.15.12:0/3113801024 conn(0x7fbd14078000 msgr2=0x7fbd1410ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:54.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.135+0000 7fbd18d0d640 1 -- 172.21.15.12:0/3113801024 shutdown_connections 2024-04-22T13:31:54.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.135+0000 7fbd18d0d640 1 -- 172.21.15.12:0/3113801024 wait complete. 2024-04-22T13:31:54.141 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:31:54.142 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:31:54.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.249+0000 7f5c51b66640 1 -- 172.21.15.12:0/3629625691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c4c102b40 msgr2=0x7f5c4c106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.249+0000 7f5c51b66640 1 --2- 172.21.15.12:0/3629625691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c4c102b40 0x7f5c4c106b80 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f5c40009900 tx=0x7f5c4002bc30 comp rx=0 tx=0).stop 2024-04-22T13:31:54.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.249+0000 7f5c51b66640 1 -- 172.21.15.12:0/3629625691 shutdown_connections 2024-04-22T13:31:54.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.249+0000 7f5c51b66640 1 --2- 172.21.15.12:0/3629625691 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c4c107260 0x7f5c4c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.249+0000 7f5c51b66640 1 --2- 172.21.15.12:0/3629625691 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c4c102b40 0x7f5c4c106b80 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.249+0000 7f5c51b66640 1 --2- 172.21.15.12:0/3629625691 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c4c102190 0x7f5c4c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.245 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.249+0000 7f5c51b66640 1 -- 172.21.15.12:0/3629625691 >> 172.21.15.12:0/3629625691 conn(0x7f5c4c0793e0 msgr2=0x7f5c4c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:54.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.249+0000 7f5c51b66640 1 -- 172.21.15.12:0/3629625691 shutdown_connections 2024-04-22T13:31:54.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.249+0000 7f5c51b66640 1 -- 172.21.15.12:0/3629625691 wait complete. 2024-04-22T13:31:54.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.250+0000 7f5c51b66640 1 Processor -- start 2024-04-22T13:31:54.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.250+0000 7f5c51b66640 1 -- start start 2024-04-22T13:31:54.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.250+0000 7f5c51b66640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c4c102190 0x7f5c4c198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.250+0000 7f5c51b66640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c4c107260 0x7f5c4c198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.250+0000 7f5c51b66640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c4c19d020 0x7f5c4c19d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.250+0000 7f5c51b66640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4c1121a0 con 0x7f5c4c102190 2024-04-22T13:31:54.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.250+0000 7f5c51b66640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4c112020 con 0x7f5c4c107260 2024-04-22T13:31:54.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.250+0000 7f5c51b66640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4c112320 con 0x7f5c4c19d020 2024-04-22T13:31:54.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.250+0000 7f5c4b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c4c102190 0x7f5c4c198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.250+0000 7f5c4b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c4c102190 0x7f5c4c198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60772/0 (socket says 172.21.15.12:60772) 2024-04-22T13:31:54.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.250+0000 7f5c4b7fe640 1 -- 172.21.15.12:0/824132446 learned_addr learned my addr 172.21.15.12:0/824132446 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:54.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.250+0000 7f5c4affd640 1 --2- 172.21.15.12:0/824132446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c4c107260 0x7f5c4c198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.251+0000 7f5c4bfff640 1 --2- 172.21.15.12:0/824132446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c4c19d020 0x7f5c4c19d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.251+0000 7f5c4b7fe640 1 -- 172.21.15.12:0/824132446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c4c19d020 msgr2=0x7f5c4c19d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.251+0000 7f5c4b7fe640 1 --2- 172.21.15.12:0/824132446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c4c19d020 0x7f5c4c19d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.251+0000 7f5c4b7fe640 1 -- 172.21.15.12:0/824132446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c4c107260 msgr2=0x7f5c4c198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.251+0000 7f5c4b7fe640 1 --2- 172.21.15.12:0/824132446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c4c107260 0x7f5c4c198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.251+0000 7f5c4b7fe640 1 -- 172.21.15.12:0/824132446 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c4c19db10 con 0x7f5c4c102190 2024-04-22T13:31:54.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.251+0000 7f5c4affd640 1 --2- 172.21.15.12:0/824132446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c4c107260 0x7f5c4c198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:54.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.251+0000 7f5c4b7fe640 1 --2- 172.21.15.12:0/824132446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c4c102190 0x7f5c4c198760 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00d950 tx=0x7f5c3c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:54.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.251+0000 7f5c48ff9640 1 -- 172.21.15.12:0/824132446 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c017030 con 0x7f5c4c102190 2024-04-22T13:31:54.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.251+0000 7f5c51b66640 1 -- 172.21.15.12:0/824132446 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c4c19de00 con 0x7f5c4c102190 2024-04-22T13:31:54.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.251+0000 7f5c48ff9640 1 -- 172.21.15.12:0/824132446 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c3c004020 con 0x7f5c4c102190 2024-04-22T13:31:54.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.251+0000 7f5c48ff9640 1 -- 172.21.15.12:0/824132446 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c0047a0 con 0x7f5c4c102190 2024-04-22T13:31:54.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.251+0000 7f5c51b66640 1 -- 172.21.15.12:0/824132446 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c4c1a55f0 con 0x7f5c4c102190 2024-04-22T13:31:54.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.252+0000 7f5c51b66640 1 -- 172.21.15.12:0/824132446 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c4c069230 con 0x7f5c4c102190 2024-04-22T13:31:54.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.253+0000 7f5c48ff9640 1 -- 172.21.15.12:0/824132446 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5c3c009a30 con 0x7f5c4c102190 2024-04-22T13:31:54.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.254+0000 7f5c48ff9640 1 --2- 172.21.15.12:0/824132446 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5c340a3d20 0x7f5c340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.254+0000 7f5c4affd640 1 --2- 172.21.15.12:0/824132446 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5c340a3d20 0x7f5c340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.254+0000 7f5c48ff9640 1 -- 172.21.15.12:0/824132446 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f5c3c0d02d0 con 0x7f5c4c102190 2024-04-22T13:31:54.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.254+0000 7f5c4affd640 1 --2- 172.21.15.12:0/824132446 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5c340a3d20 0x7f5c340a61e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5c40036040 tx=0x7f5c40000bc0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:54.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.258+0000 7f5c48ff9640 1 -- 172.21.15.12:0/824132446 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5c3c09c990 con 0x7f5c4c102190 2024-04-22T13:31:54.483 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.488+0000 7f5c51b66640 1 -- 172.21.15.12:0/824132446 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c4c199ab0 con 0x7f5c4c102190 2024-04-22T13:31:54.484 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.488+0000 7f5c48ff9640 1 -- 172.21.15.12:0/824132446 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7f5c3c09c290 con 0x7f5c4c102190 2024-04-22T13:31:54.484 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:31:54.485 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:31:54.485 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:31:54.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.490+0000 7f5c51b66640 1 -- 172.21.15.12:0/824132446 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5c340a3d20 msgr2=0x7f5c340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.491+0000 7f5c51b66640 1 --2- 172.21.15.12:0/824132446 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5c340a3d20 0x7f5c340a61e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5c40036040 tx=0x7f5c40000bc0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.491+0000 7f5c51b66640 1 -- 172.21.15.12:0/824132446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c4c102190 msgr2=0x7f5c4c198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.491+0000 7f5c51b66640 1 --2- 172.21.15.12:0/824132446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c4c102190 0x7f5c4c198760 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00d950 tx=0x7f5c3c00de10 comp rx=0 tx=0).stop 2024-04-22T13:31:54.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.491+0000 7f5c51b66640 1 -- 172.21.15.12:0/824132446 shutdown_connections 2024-04-22T13:31:54.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.491+0000 7f5c51b66640 1 --2- 172.21.15.12:0/824132446 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5c340a3d20 0x7f5c340a61e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.491+0000 7f5c51b66640 1 --2- 172.21.15.12:0/824132446 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5c4c19d020 0x7f5c4c19d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.491+0000 7f5c51b66640 1 --2- 172.21.15.12:0/824132446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c4c107260 0x7f5c4c198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.491+0000 7f5c51b66640 1 --2- 172.21.15.12:0/824132446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5c4c102190 0x7f5c4c198760 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.491+0000 7f5c51b66640 1 -- 172.21.15.12:0/824132446 >> 172.21.15.12:0/824132446 conn(0x7f5c4c0793e0 msgr2=0x7f5c4c1000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:54.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.491+0000 7f5c51b66640 1 -- 172.21.15.12:0/824132446 shutdown_connections 2024-04-22T13:31:54.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.491+0000 7f5c51b66640 1 -- 172.21.15.12:0/824132446 wait complete. 2024-04-22T13:31:54.498 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:31:54.606 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.608+0000 7f4d4505c640 1 -- 172.21.15.12:0/3981660500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d40106170 msgr2=0x7f4d4010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.608+0000 7f4d4505c640 1 --2- 172.21.15.12:0/3981660500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d40106170 0x7f4d4010dd40 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009ca0 tx=0x7f4d3802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.608+0000 7f4d4505c640 1 -- 172.21.15.12:0/3981660500 shutdown_connections 2024-04-22T13:31:54.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.608+0000 7f4d4505c640 1 --2- 172.21.15.12:0/3981660500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d40106170 0x7f4d4010dd40 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.608+0000 7f4d4505c640 1 --2- 172.21.15.12:0/3981660500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d40101a50 0x7f4d40105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.608+0000 7f4d4505c640 1 --2- 172.21.15.12:0/3981660500 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d401010a0 0x7f4d40101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.608+0000 7f4d4505c640 1 -- 172.21.15.12:0/3981660500 >> 172.21.15.12:0/3981660500 conn(0x7f4d40077eb0 msgr2=0x7f4d400782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:54.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.608+0000 7f4d4505c640 1 -- 172.21.15.12:0/3981660500 shutdown_connections 2024-04-22T13:31:54.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.608+0000 7f4d4505c640 1 -- 172.21.15.12:0/3981660500 wait complete. 2024-04-22T13:31:54.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.609+0000 7f4d4505c640 1 Processor -- start 2024-04-22T13:31:54.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.609+0000 7f4d4505c640 1 -- start start 2024-04-22T13:31:54.607 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.609+0000 7f4d4505c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d401010a0 0x7f4d40198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.609+0000 7f4d4505c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d40106170 0x7f4d40198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.609+0000 7f4d4505c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d4019cfe0 0x7f4d4019d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.609+0000 7f4d4505c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4d40112250 con 0x7f4d4019cfe0 2024-04-22T13:31:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.609+0000 7f4d4505c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4d401120d0 con 0x7f4d40106170 2024-04-22T13:31:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.609+0000 7f4d4505c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4d401123d0 con 0x7f4d401010a0 2024-04-22T13:31:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.609+0000 7f4d3f59e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d4019cfe0 0x7f4d4019d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.609+0000 7f4d3f59e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d4019cfe0 0x7f4d4019d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60790/0 (socket says 172.21.15.12:60790) 2024-04-22T13:31:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.609+0000 7f4d3f59e640 1 -- 172.21.15.12:0/4094495069 learned_addr learned my addr 172.21.15.12:0/4094495069 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.609+0000 7f4d3ed9d640 1 --2- 172.21.15.12:0/4094495069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d401010a0 0x7f4d40198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.610+0000 7f4d3659c640 1 --2- 172.21.15.12:0/4094495069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d40106170 0x7f4d40198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.610+0000 7f4d3f59e640 1 -- 172.21.15.12:0/4094495069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d401010a0 msgr2=0x7f4d40198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.610+0000 7f4d3f59e640 1 --2- 172.21.15.12:0/4094495069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d401010a0 0x7f4d40198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.610+0000 7f4d3f59e640 1 -- 172.21.15.12:0/4094495069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d40106170 msgr2=0x7f4d40198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.610+0000 7f4d3f59e640 1 --2- 172.21.15.12:0/4094495069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d40106170 0x7f4d40198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.608 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.610+0000 7f4d3f59e640 1 -- 172.21.15.12:0/4094495069 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4019dad0 con 0x7f4d4019cfe0 2024-04-22T13:31:54.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.610+0000 7f4d3ed9d640 1 --2- 172.21.15.12:0/4094495069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d401010a0 0x7f4d40198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:54.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.610+0000 7f4d3659c640 1 --2- 172.21.15.12:0/4094495069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d40106170 0x7f4d40198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:54.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.610+0000 7f4d3f59e640 1 --2- 172.21.15.12:0/4094495069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d4019cfe0 0x7f4d4019d490 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f4d38005e30 tx=0x7f4d38034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:54.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.610+0000 7f4d3cd99640 1 -- 172.21.15.12:0/4094495069 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d38039040 con 0x7f4d4019cfe0 2024-04-22T13:31:54.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.610+0000 7f4d4505c640 1 -- 172.21.15.12:0/4094495069 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d4019dd60 con 0x7f4d4019cfe0 2024-04-22T13:31:54.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.610+0000 7f4d3cd99640 1 -- 172.21.15.12:0/4094495069 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d38044070 con 0x7f4d4019cfe0 2024-04-22T13:31:54.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.610+0000 7f4d3cd99640 1 -- 172.21.15.12:0/4094495069 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d38003e60 con 0x7f4d4019cfe0 2024-04-22T13:31:54.609 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.610+0000 7f4d4505c640 1 -- 172.21.15.12:0/4094495069 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d401a5520 con 0x7f4d4019cfe0 2024-04-22T13:31:54.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.612+0000 7f4d3cd99640 1 -- 172.21.15.12:0/4094495069 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4d380324c0 con 0x7f4d4019cfe0 2024-04-22T13:31:54.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.613+0000 7f4d47c54740 1 Processor -- start 2024-04-22T13:31:54.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.613+0000 7f4d47c54740 1 -- start start 2024-04-22T13:31:54.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.613+0000 7f4d47c54740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fb83254d0 0x562fb83258b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.613+0000 7f4d47c54740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562fb8325e80 0x562fb832e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.613+0000 7f4d47c54740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fb832ea00 0x562fb8330e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.613+0000 7f4d47c54740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x562fb828a1f0 con 0x562fb8325e80 2024-04-22T13:31:54.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.613+0000 7f4d47c54740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x562fb82274b0 con 0x562fb832ea00 2024-04-22T13:31:54.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.613+0000 7f4d47c54740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562fb82861b0 con 0x562fb83254d0 2024-04-22T13:31:54.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.613+0000 7f4d3ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562fb8325e80 0x562fb832e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.613+0000 7f4d3ed9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562fb8325e80 0x562fb832e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60796/0 (socket says 172.21.15.12:60796) 2024-04-22T13:31:54.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.613+0000 7f4d3ed9d640 1 -- 172.21.15.12:0/1677229038 learned_addr learned my addr 172.21.15.12:0/1677229038 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:54.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.613+0000 7f4d3cd99640 1 --2- 172.21.15.12:0/4094495069 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4d2c0a3d20 0x7f4d2c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.613+0000 7f4d3659c640 1 --2- 172.21.15.12:0/1677229038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fb83254d0 0x562fb83258b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.613+0000 7f4d3659c640 1 --2- 172.21.15.12:0/1677229038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fb832ea00 0x562fb8330e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.613+0000 7f4d3f59e640 1 --2- 172.21.15.12:0/4094495069 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4d2c0a3d20 0x7f4d2c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.613+0000 7f4d3ed9d640 1 -- 172.21.15.12:0/1677229038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fb83254d0 msgr2=0x562fb83258b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.613+0000 7f4d3ed9d640 1 --2- 172.21.15.12:0/1677229038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fb83254d0 0x562fb83258b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.613+0000 7f4d3ed9d640 1 -- 172.21.15.12:0/1677229038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fb832ea00 msgr2=0x562fb8330e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.613+0000 7f4d3ed9d640 1 --2- 172.21.15.12:0/1677229038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fb832ea00 0x562fb8330e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.613+0000 7f4d3ed9d640 1 -- 172.21.15.12:0/1677229038 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fb82191b0 con 0x562fb8325e80 2024-04-22T13:31:54.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.613+0000 7f4d3659c640 1 --2- 172.21.15.12:0/1677229038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fb83254d0 0x562fb83258b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:54.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.614+0000 7f4d3cd99640 1 -- 172.21.15.12:0/4094495069 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f4d380f2ac0 con 0x7f4d4019cfe0 2024-04-22T13:31:54.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.614+0000 7f4d3ed9d640 1 --2- 172.21.15.12:0/1677229038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562fb8325e80 0x562fb832e430 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f4d30007850 tx=0x7f4d30007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d31d69f313c6aad server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:54.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.614+0000 7f4d3f59e640 1 --2- 172.21.15.12:0/4094495069 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4d2c0a3d20 0x7f4d2c0a61e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4d380be830 tx=0x7f4d380095c0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:54.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.614+0000 7f4d3559a640 1 -- 172.21.15.12:0/1677229038 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3000d950 con 0x562fb8325e80 2024-04-22T13:31:54.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.614+0000 7f4d3559a640 1 -- 172.21.15.12:0/1677229038 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d30018070 con 0x562fb8325e80 2024-04-22T13:31:54.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.614+0000 7f4d3559a640 1 -- 172.21.15.12:0/1677229038 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d30013de0 con 0x562fb8325e80 2024-04-22T13:31:54.617 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.615+0000 7f4d47c54740 1 -- 172.21.15.12:0/1677229038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562fb8325e80 msgr2=0x562fb832e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.615+0000 7f4d47c54740 1 --2- 172.21.15.12:0/1677229038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562fb8325e80 0x562fb832e430 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f4d30007850 tx=0x7f4d30007d10 comp rx=0 tx=0).stop 2024-04-22T13:31:54.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.615+0000 7f4d47c54740 1 -- 172.21.15.12:0/1677229038 shutdown_connections 2024-04-22T13:31:54.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.615+0000 7f4d47c54740 1 --2- 172.21.15.12:0/1677229038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fb832ea00 0x562fb8330e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.615+0000 7f4d47c54740 1 --2- 172.21.15.12:0/1677229038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562fb8325e80 0x562fb832e430 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.615+0000 7f4d47c54740 1 --2- 172.21.15.12:0/1677229038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fb83254d0 0x562fb83258b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.615+0000 7f4d47c54740 1 -- 172.21.15.12:0/1677229038 >> 172.21.15.12:0/1677229038 conn(0x562fb831fba0 msgr2=0x562fb831ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:54.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.615+0000 7f4d47c54740 1 -- 172.21.15.12:0/1677229038 shutdown_connections 2024-04-22T13:31:54.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.615+0000 7f4d47c54740 1 -- 172.21.15.12:0/1677229038 wait complete. 2024-04-22T13:31:54.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.615+0000 7f4d47c54740 1 -- start start 2024-04-22T13:31:54.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.615+0000 7f4d47c54740 1 Processor -- start 2024-04-22T13:31:54.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.615+0000 7f4d47c54740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fb83254d0 0x562fb832e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.615+0000 7f4d47c54740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fb8325e80 0x562fb832b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.615+0000 7f4d47c54740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562fb832ea00 0x562fb832bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.615+0000 7f4d47c54740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x562fb8331970 con 0x562fb832ea00 2024-04-22T13:31:54.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.615+0000 7f4d47c54740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x562fb8243500 con 0x562fb83254d0 2024-04-22T13:31:54.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.615+0000 7f4d47c54740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562fb8331af0 con 0x562fb8325e80 2024-04-22T13:31:54.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.615+0000 7f4d47c54740 10 client.0 _ll_register_callbacks cb 0x562fb8211b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:54.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.616+0000 7f4d3659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562fb832ea00 0x562fb832bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.616+0000 7f4d3659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562fb832ea00 0x562fb832bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60802/0 (socket says 172.21.15.12:60802) 2024-04-22T13:31:54.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.616+0000 7f4d3659c640 1 -- 172.21.15.12:0/1830531663 learned_addr learned my addr 172.21.15.12:0/1830531663 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:54.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.616+0000 7f4d3ed9d640 1 --2- 172.21.15.12:0/1830531663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fb8325e80 0x562fb832b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.616+0000 7f4d3659c640 1 --2- 172.21.15.12:0/1830531663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fb83254d0 0x562fb832e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.616+0000 7f4d3659c640 1 -- 172.21.15.12:0/1830531663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fb8325e80 msgr2=0x562fb832b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.616+0000 7f4d3659c640 1 --2- 172.21.15.12:0/1830531663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fb8325e80 0x562fb832b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.616+0000 7f4d3659c640 1 -- 172.21.15.12:0/1830531663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fb83254d0 msgr2=0x562fb832e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.616+0000 7f4d3659c640 1 --2- 172.21.15.12:0/1830531663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fb83254d0 0x562fb832e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.616+0000 7f4d3659c640 1 -- 172.21.15.12:0/1830531663 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fb7f9c3f0 con 0x562fb832ea00 2024-04-22T13:31:54.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.616+0000 7f4d3659c640 1 --2- 172.21.15.12:0/1830531663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562fb832ea00 0x562fb832bb60 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f4d28013a30 tx=0x7f4d280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78cabfcaaac35e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:54.619 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.616+0000 7f4d267fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:54.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.616+0000 7f4d267fc640 1 -- 172.21.15.12:0/1830531663 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28011db0 con 0x562fb832ea00 2024-04-22T13:31:54.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.616+0000 7f4d267fc640 1 -- 172.21.15.12:0/1830531663 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f4d28010450 con 0x562fb832ea00 2024-04-22T13:31:54.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.616+0000 7f4d47c54740 1 -- 172.21.15.12:0/1830531663 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d30018070 con 0x562fb832ea00 2024-04-22T13:31:54.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.616+0000 7f4d267fc640 1 -- 172.21.15.12:0/1830531663 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4d28010650 con 0x562fb832ea00 2024-04-22T13:31:54.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.616+0000 7f4d47c54740 10 client.18375 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:54.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.616+0000 7f4d47c54740 1 -- 172.21.15.12:0/1830531663 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562fb80667d0 con 0x562fb832ea00 2024-04-22T13:31:54.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.616+0000 7f4d267fc640 1 -- 172.21.15.12:0/1830531663 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d30018070 con 0x562fb832ea00 2024-04-22T13:31:54.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.617+0000 7f4d267fc640 1 -- 172.21.15.12:0/1830531663 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4d28025020 con 0x562fb832ea00 2024-04-22T13:31:54.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.617+0000 7f4d47c54740 10 client.18375 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:54.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.617+0000 7f4d47c54740 10 client.18375 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:31:54.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.617+0000 7f4d47c54740 20 client.18375 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:54.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.617+0000 7f4d47c54740 1 --2- 172.21.15.12:0/1830531663 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x562fb833eb50 0x562fb8340f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.617+0000 7f4d47c54740 4 client.18375 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:54.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.617+0000 7f4d47c54740 1 -- 172.21.15.12:0/1830531663 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fb8229930 con 0x562fb833eb50 2024-04-22T13:31:54.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.617+0000 7f4d3ed9d640 1 --2- 172.21.15.12:0/1830531663 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x562fb833eb50 0x562fb8340f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.618+0000 7f4d267fc640 10 client.18375 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:31:54.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.618+0000 7f4d3ed9d640 1 --2- 172.21.15.12:0/1830531663 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x562fb833eb50 0x562fb8340f80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a66b74e397e2ec28 server_cookie=be700720e00ac312 in_seq=0 out_seq=0 2024-04-22T13:31:54.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.618+0000 7f4d267fc640 1 -- 172.21.15.12:0/1830531663 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562fb8229930 con 0x562fb833eb50 2024-04-22T13:31:54.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.618+0000 7f4d267fc640 10 client.18375 handle_command_reply: tid=0 2024-04-22T13:31:54.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.618+0000 7f4d47c54740 1 client.18375 shutdown 2024-04-22T13:31:54.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.619+0000 7f4d47c54740 1 -- 172.21.15.12:0/1830531663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562fb832ea00 msgr2=0x562fb832bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.619+0000 7f4d47c54740 1 --2- 172.21.15.12:0/1830531663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562fb832ea00 0x562fb832bb60 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f4d28013a30 tx=0x7f4d280046c0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.619+0000 7f4d47c54740 1 -- 172.21.15.12:0/1830531663 shutdown_connections 2024-04-22T13:31:54.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.619+0000 7f4d47c54740 1 --2- 172.21.15.12:0/1830531663 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x562fb833eb50 0x562fb8340f80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.619+0000 7f4d47c54740 1 --2- 172.21.15.12:0/1830531663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562fb832ea00 0x562fb832bb60 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.619+0000 7f4d47c54740 1 --2- 172.21.15.12:0/1830531663 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fb8325e80 0x562fb832b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.619+0000 7f4d267fc640 0 client.18375 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:31:54.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.619+0000 7f4d47c54740 1 --2- 172.21.15.12:0/1830531663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fb83254d0 0x562fb832e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.619+0000 7f4d47c54740 1 -- 172.21.15.12:0/1830531663 >> 172.21.15.12:0/1830531663 conn(0x562fb831fba0 msgr2=0x562fb83218f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:54.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.619+0000 7f4d47c54740 1 -- 172.21.15.12:0/1830531663 shutdown_connections 2024-04-22T13:31:54.621 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.619+0000 7f4d47c54740 1 -- 172.21.15.12:0/1830531663 wait complete. 2024-04-22T13:31:54.622 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.619+0000 7f4d47c54740 20 client.18375 trim_cache size 0 max 16384 2024-04-22T13:31:54.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.632+0000 7f4d47c54740 1 Processor -- start 2024-04-22T13:31:54.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.632+0000 7f4d47c54740 1 -- start start 2024-04-22T13:31:54.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.632+0000 7f4d47c54740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fb83254d0 0x562fb833a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.632+0000 7f4d47c54740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fb8325e80 0x562fb832a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.632+0000 7f4d47c54740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562fb832ea00 0x562fb831ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.632+0000 7f4d47c54740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4d401adb40 con 0x562fb832ea00 2024-04-22T13:31:54.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.632+0000 7f4d47c54740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4d401af680 con 0x562fb8325e80 2024-04-22T13:31:54.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.632+0000 7f4d47c54740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4d401ae210 con 0x562fb83254d0 2024-04-22T13:31:54.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.632+0000 7f4d3659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562fb832ea00 0x562fb831ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.633+0000 7f4d3659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562fb832ea00 0x562fb831ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60806/0 (socket says 172.21.15.12:60806) 2024-04-22T13:31:54.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.633+0000 7f4d3659c640 1 -- 172.21.15.12:0/1021543049 learned_addr learned my addr 172.21.15.12:0/1021543049 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:54.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.633+0000 7f4d3ed9d640 1 --2- 172.21.15.12:0/1021543049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fb8325e80 0x562fb832a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.633+0000 7f4d3659c640 1 -- 172.21.15.12:0/1021543049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fb83254d0 msgr2=0x562fb833a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.633+0000 7f4d3659c640 1 --2- 172.21.15.12:0/1021543049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fb83254d0 0x562fb833a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.633+0000 7f4d3659c640 1 -- 172.21.15.12:0/1021543049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fb8325e80 msgr2=0x562fb832a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.633+0000 7f4d3659c640 1 --2- 172.21.15.12:0/1021543049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fb8325e80 0x562fb832a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.629 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.633+0000 7f4d3659c640 1 -- 172.21.15.12:0/1021543049 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fb8331850 con 0x562fb832ea00 2024-04-22T13:31:54.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.633+0000 7f4d3659c640 1 --2- 172.21.15.12:0/1021543049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562fb832ea00 0x562fb831ea80 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f4d280136d0 tx=0x7f4d28027ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f0fcc5c902cbce server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:54.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.633+0000 7f4d34d99640 1 -- 172.21.15.12:0/1021543049 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2800d900 con 0x562fb832ea00 2024-04-22T13:31:54.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.633+0000 7f4d34d99640 1 -- 172.21.15.12:0/1021543049 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d28027460 con 0x562fb832ea00 2024-04-22T13:31:54.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.633+0000 7f4d34d99640 1 -- 172.21.15.12:0/1021543049 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28011bb0 con 0x562fb832ea00 2024-04-22T13:31:54.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.634+0000 7f4d47c54740 1 -- 172.21.15.12:0/1021543049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562fb832ea00 msgr2=0x562fb831ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.634+0000 7f4d47c54740 1 --2- 172.21.15.12:0/1021543049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562fb832ea00 0x562fb831ea80 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f4d280136d0 tx=0x7f4d28027ec0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.634+0000 7f4d47c54740 1 -- 172.21.15.12:0/1021543049 shutdown_connections 2024-04-22T13:31:54.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.634+0000 7f4d47c54740 1 --2- 172.21.15.12:0/1021543049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562fb832ea00 0x562fb831ea80 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.634+0000 7f4d47c54740 1 --2- 172.21.15.12:0/1021543049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fb8325e80 0x562fb832a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.634+0000 7f4d47c54740 1 --2- 172.21.15.12:0/1021543049 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fb83254d0 0x562fb833a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.634+0000 7f4d47c54740 1 -- 172.21.15.12:0/1021543049 >> 172.21.15.12:0/1021543049 conn(0x562fb831fba0 msgr2=0x562fb831e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:54.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.634+0000 7f4d47c54740 1 -- 172.21.15.12:0/1021543049 shutdown_connections 2024-04-22T13:31:54.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.634+0000 7f4d47c54740 1 -- 172.21.15.12:0/1021543049 wait complete. 2024-04-22T13:31:54.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.634+0000 7f4d47c54740 1 -- start start 2024-04-22T13:31:54.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.634+0000 7f4d47c54740 1 Processor -- start 2024-04-22T13:31:54.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.634+0000 7f4d47c54740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562fb83254d0 0x562fb8336a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.635+0000 7f4d47c54740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fb8325e80 0x562fb8336f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.635+0000 7f4d47c54740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fb832ea00 0x562fb8331ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.635+0000 7f4d47c54740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f4d401aefb0 con 0x562fb83254d0 2024-04-22T13:31:54.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.635+0000 7f4d47c54740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4d401ae8e0 con 0x562fb8325e80 2024-04-22T13:31:54.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.635+0000 7f4d47c54740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4d401abb40 con 0x562fb832ea00 2024-04-22T13:31:54.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.635+0000 7f4d47c54740 10 client.0 _ll_register_callbacks cb 0x7f4d280135c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:54.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.635+0000 7f4d3659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562fb83254d0 0x562fb8336a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.635+0000 7f4d3659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562fb83254d0 0x562fb8336a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60818/0 (socket says 172.21.15.12:60818) 2024-04-22T13:31:54.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.635+0000 7f4d3659c640 1 -- 172.21.15.12:0/353327576 learned_addr learned my addr 172.21.15.12:0/353327576 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:54.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.635+0000 7f4d3ed9d640 1 --2- 172.21.15.12:0/353327576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fb8325e80 0x562fb8336f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.635+0000 7f4d3659c640 1 -- 172.21.15.12:0/353327576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fb832ea00 msgr2=0x562fb8331ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.635+0000 7f4d3659c640 1 --2- 172.21.15.12:0/353327576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fb832ea00 0x562fb8331ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.635+0000 7f4d3659c640 1 -- 172.21.15.12:0/353327576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fb8325e80 msgr2=0x562fb8336f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.635+0000 7f4d3659c640 1 --2- 172.21.15.12:0/353327576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fb8325e80 0x562fb8336f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.635+0000 7f4d3659c640 1 -- 172.21.15.12:0/353327576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fb8332600 con 0x562fb83254d0 2024-04-22T13:31:54.636 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.635+0000 7f4d3ed9d640 1 --2- 172.21.15.12:0/353327576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fb8325e80 0x562fb8336f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:54.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.635+0000 7f4d3659c640 1 --2- 172.21.15.12:0/353327576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562fb83254d0 0x562fb8336a00 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800d560 tx=0x7f4d28005860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4501562ba17dc0c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:54.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.635+0000 7f4d267fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:54.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.635+0000 7f4d267fc640 1 -- 172.21.15.12:0/353327576 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28036060 con 0x562fb83254d0 2024-04-22T13:31:54.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.635+0000 7f4d267fc640 1 -- 172.21.15.12:0/353327576 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f4d28011bb0 con 0x562fb83254d0 2024-04-22T13:31:54.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.635+0000 7f4d47c54740 1 -- 172.21.15.12:0/353327576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d28027460 con 0x562fb83254d0 2024-04-22T13:31:54.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.635+0000 7f4d267fc640 1 -- 172.21.15.12:0/353327576 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4d2800d900 con 0x562fb83254d0 2024-04-22T13:31:54.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.635+0000 7f4d47c54740 10 client.18381 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:54.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.635+0000 7f4d47c54740 1 -- 172.21.15.12:0/353327576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562fb83a4ac0 con 0x562fb83254d0 2024-04-22T13:31:54.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.636+0000 7f4d267fc640 1 -- 172.21.15.12:0/353327576 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d28027460 con 0x562fb83254d0 2024-04-22T13:31:54.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.636+0000 7f4d267fc640 1 -- 172.21.15.12:0/353327576 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4d28025070 con 0x562fb83254d0 2024-04-22T13:31:54.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.636+0000 7f4d47c54740 10 client.18381 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:54.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.636+0000 7f4d47c54740 10 client.18381 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:31:54.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.636+0000 7f4d47c54740 20 client.18381 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:54.637 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.636+0000 7f4d47c54740 1 --2- 172.21.15.12:0/353327576 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x562fb83a5120 0x562fb83a7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.636+0000 7f4d47c54740 4 client.18381 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:31:54.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.636+0000 7f4d47c54740 1 -- 172.21.15.12:0/353327576 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562fb8297c10 con 0x562fb83a5120 2024-04-22T13:31:54.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.636+0000 7f4d3ed9d640 1 --2- 172.21.15.12:0/353327576 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x562fb83a5120 0x562fb83a7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.637+0000 7f4d267fc640 10 client.18381 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:31:54.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.637+0000 7f4d3ed9d640 1 --2- 172.21.15.12:0/353327576 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x562fb83a5120 0x562fb83a7510 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=22dd8844f0c5185c server_cookie=3d1b350b96033b73 in_seq=0 out_seq=0 2024-04-22T13:31:54.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.637+0000 7f4d267fc640 1 -- 172.21.15.12:0/353327576 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562fb8297c10 con 0x562fb83a5120 2024-04-22T13:31:54.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.637+0000 7f4d267fc640 10 client.18381 handle_command_reply: tid=0 2024-04-22T13:31:54.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.637+0000 7f4d47c54740 1 client.18381 shutdown 2024-04-22T13:31:54.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.637+0000 7f4d47c54740 1 -- 172.21.15.12:0/353327576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562fb83254d0 msgr2=0x562fb8336a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.637+0000 7f4d47c54740 1 --2- 172.21.15.12:0/353327576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562fb83254d0 0x562fb8336a00 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800d560 tx=0x7f4d28005860 comp rx=0 tx=0).stop 2024-04-22T13:31:54.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.637+0000 7f4d47c54740 1 -- 172.21.15.12:0/353327576 shutdown_connections 2024-04-22T13:31:54.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.637+0000 7f4d47c54740 1 --2- 172.21.15.12:0/353327576 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x562fb83a5120 0x562fb83a7510 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.637+0000 7f4d47c54740 1 --2- 172.21.15.12:0/353327576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fb832ea00 0x562fb8331ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.638 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.637+0000 7f4d47c54740 1 --2- 172.21.15.12:0/353327576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562fb8325e80 0x562fb8336f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.637+0000 7f4d267fc640 0 client.18381 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:31:54.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.637+0000 7f4d47c54740 1 --2- 172.21.15.12:0/353327576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562fb83254d0 0x562fb8336a00 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.637+0000 7f4d47c54740 1 -- 172.21.15.12:0/353327576 >> 172.21.15.12:0/353327576 conn(0x562fb831fba0 msgr2=0x562fb831e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:54.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.637+0000 7f4d47c54740 1 -- 172.21.15.12:0/353327576 shutdown_connections 2024-04-22T13:31:54.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.637+0000 7f4d47c54740 1 -- 172.21.15.12:0/353327576 wait complete. 2024-04-22T13:31:54.639 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.637+0000 7f4d47c54740 20 client.18381 trim_cache size 0 max 16384 2024-04-22T13:31:54.639 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:31:54.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.637+0000 7f4d4505c640 1 -- 172.21.15.12:0/4094495069 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4d2c0a3d20 msgr2=0x7f4d2c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.638+0000 7f4d4505c640 1 --2- 172.21.15.12:0/4094495069 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4d2c0a3d20 0x7f4d2c0a61e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4d380be830 tx=0x7f4d380095c0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.638+0000 7f4d4505c640 1 -- 172.21.15.12:0/4094495069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d4019cfe0 msgr2=0x7f4d4019d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.638+0000 7f4d4505c640 1 --2- 172.21.15.12:0/4094495069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d4019cfe0 0x7f4d4019d490 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f4d38005e30 tx=0x7f4d38034760 comp rx=0 tx=0).stop 2024-04-22T13:31:54.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.638+0000 7f4d4505c640 1 -- 172.21.15.12:0/4094495069 shutdown_connections 2024-04-22T13:31:54.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.638+0000 7f4d4505c640 1 --2- 172.21.15.12:0/4094495069 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f4d2c0a3d20 0x7f4d2c0a61e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.638+0000 7f4d4505c640 1 --2- 172.21.15.12:0/4094495069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4d4019cfe0 0x7f4d4019d490 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.638+0000 7f4d4505c640 1 --2- 172.21.15.12:0/4094495069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d40106170 0x7f4d40198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.638+0000 7f4d4505c640 1 --2- 172.21.15.12:0/4094495069 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4d401010a0 0x7f4d40198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.638+0000 7f4d4505c640 1 -- 172.21.15.12:0/4094495069 >> 172.21.15.12:0/4094495069 conn(0x7f4d40077eb0 msgr2=0x7f4d40100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:54.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.638+0000 7f4d4505c640 1 -- 172.21.15.12:0/4094495069 shutdown_connections 2024-04-22T13:31:54.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.638+0000 7f4d4505c640 1 -- 172.21.15.12:0/4094495069 wait complete. 2024-04-22T13:31:54.644 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:31:54.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.751+0000 7f85c226d640 1 -- 172.21.15.12:0/2868564438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85bc107270 msgr2=0x7f85bc10ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.751+0000 7f85c226d640 1 --2- 172.21.15.12:0/2868564438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85bc107270 0x7f85bc10ee40 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f85b4009ca0 tx=0x7f85b402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.751+0000 7f85c226d640 1 -- 172.21.15.12:0/2868564438 shutdown_connections 2024-04-22T13:31:54.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.751+0000 7f85c226d640 1 --2- 172.21.15.12:0/2868564438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85bc107270 0x7f85bc10ee40 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.751+0000 7f85c226d640 1 --2- 172.21.15.12:0/2868564438 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85bc06b1f0 0x7f85bc106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.751+0000 7f85c226d640 1 --2- 172.21.15.12:0/2868564438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85bc06a840 0x7f85bc06ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.751+0000 7f85c226d640 1 -- 172.21.15.12:0/2868564438 >> 172.21.15.12:0/2868564438 conn(0x7f85bc077eb0 msgr2=0x7f85bc0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:54.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.751+0000 7f85c226d640 1 -- 172.21.15.12:0/2868564438 shutdown_connections 2024-04-22T13:31:54.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.752+0000 7f85c226d640 1 -- 172.21.15.12:0/2868564438 wait complete. 2024-04-22T13:31:54.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.752+0000 7f85c226d640 1 Processor -- start 2024-04-22T13:31:54.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.752+0000 7f85c226d640 1 -- start start 2024-04-22T13:31:54.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.752+0000 7f85c226d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85bc06a840 0x7f85bc198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.752+0000 7f85c226d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85bc107270 0x7f85bc198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.752+0000 7f85c226d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85bc19cef0 0x7f85bc19d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.752+0000 7f85c226d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f85bc1121d0 con 0x7f85bc19cef0 2024-04-22T13:31:54.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.752+0000 7f85c226d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f85bc112050 con 0x7f85bc107270 2024-04-22T13:31:54.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.753+0000 7f85c226d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f85bc112350 con 0x7f85bc06a840 2024-04-22T13:31:54.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.753+0000 7f85c080a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85bc19cef0 0x7f85bc19d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.753+0000 7f85c080a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85bc19cef0 0x7f85bc19d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60824/0 (socket says 172.21.15.12:60824) 2024-04-22T13:31:54.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.753+0000 7f85c080a640 1 -- 172.21.15.12:0/167771269 learned_addr learned my addr 172.21.15.12:0/167771269 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:54.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.753+0000 7f85bbfff640 1 --2- 172.21.15.12:0/167771269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85bc06a840 0x7f85bc198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.753+0000 7f85bb7fe640 1 --2- 172.21.15.12:0/167771269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85bc107270 0x7f85bc198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.753+0000 7f85c080a640 1 -- 172.21.15.12:0/167771269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85bc06a840 msgr2=0x7f85bc198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.753+0000 7f85c080a640 1 --2- 172.21.15.12:0/167771269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85bc06a840 0x7f85bc198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.753+0000 7f85c080a640 1 -- 172.21.15.12:0/167771269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85bc107270 msgr2=0x7f85bc198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.753+0000 7f85c080a640 1 --2- 172.21.15.12:0/167771269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85bc107270 0x7f85bc198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.753+0000 7f85c080a640 1 -- 172.21.15.12:0/167771269 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85bc19d9e0 con 0x7f85bc19cef0 2024-04-22T13:31:54.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.753+0000 7f85bbfff640 1 --2- 172.21.15.12:0/167771269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85bc06a840 0x7f85bc198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:54.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.753+0000 7f85c080a640 1 --2- 172.21.15.12:0/167771269 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85bc19cef0 0x7f85bc19d3a0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f85b4005e30 tx=0x7f85b4034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:54.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.753+0000 7f85b97fa640 1 -- 172.21.15.12:0/167771269 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b4041020 con 0x7f85bc19cef0 2024-04-22T13:31:54.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.753+0000 7f85b97fa640 1 -- 172.21.15.12:0/167771269 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85b4032970 con 0x7f85bc19cef0 2024-04-22T13:31:54.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.753+0000 7f85c226d640 1 -- 172.21.15.12:0/167771269 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85bc19dc70 con 0x7f85bc19cef0 2024-04-22T13:31:54.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.753+0000 7f85b97fa640 1 -- 172.21.15.12:0/167771269 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b4039040 con 0x7f85bc19cef0 2024-04-22T13:31:54.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.753+0000 7f85c226d640 1 -- 172.21.15.12:0/167771269 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85bc1a5490 con 0x7f85bc19cef0 2024-04-22T13:31:54.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.756+0000 7f85b97fa640 1 -- 172.21.15.12:0/167771269 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f85b4044020 con 0x7f85bc19cef0 2024-04-22T13:31:54.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.756+0000 7f85c4e66740 1 Processor -- start 2024-04-22T13:31:54.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.756+0000 7f85c4e66740 1 -- start start 2024-04-22T13:31:54.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.756+0000 7f85c4e66740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b56b334d0 0x556b56b338b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.756+0000 7f85c4e66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b56b33e80 0x556b56b3c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.756+0000 7f85c4e66740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b56b3ca00 0x556b56b3ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.756+0000 7f85c4e66740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556b56a981f0 con 0x556b56b3ca00 2024-04-22T13:31:54.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.756+0000 7f85c4e66740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556b56a354b0 con 0x556b56b334d0 2024-04-22T13:31:54.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.756+0000 7f85c4e66740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556b56a941b0 con 0x556b56b33e80 2024-04-22T13:31:54.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.756+0000 7f85b97fa640 1 --2- 172.21.15.12:0/167771269 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f85a80a3da0 0x7f85a80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.756+0000 7f85bb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b56b3ca00 0x556b56b3ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.756+0000 7f85bb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b56b3ca00 0x556b56b3ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60832/0 (socket says 172.21.15.12:60832) 2024-04-22T13:31:54.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.756+0000 7f85bb7fe640 1 -- 172.21.15.12:0/3627316448 learned_addr learned my addr 172.21.15.12:0/3627316448 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:54.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.756+0000 7f85bbfff640 1 --2- 172.21.15.12:0/3627316448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b56b33e80 0x556b56b3c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.756+0000 7f85c080a640 1 --2- 172.21.15.12:0/167771269 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f85a80a3da0 0x7f85a80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.756+0000 7f85bb7fe640 1 --2- 172.21.15.12:0/3627316448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b56b334d0 0x556b56b338b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.757+0000 7f85b97fa640 1 -- 172.21.15.12:0/167771269 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f85b40f2510 con 0x7f85bc19cef0 2024-04-22T13:31:54.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.757+0000 7f85bb7fe640 1 -- 172.21.15.12:0/3627316448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b56b33e80 msgr2=0x556b56b3c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.757+0000 7f85bb7fe640 1 --2- 172.21.15.12:0/3627316448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b56b33e80 0x556b56b3c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.757+0000 7f85bb7fe640 1 -- 172.21.15.12:0/3627316448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b56b334d0 msgr2=0x556b56b338b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.757+0000 7f85bb7fe640 1 --2- 172.21.15.12:0/3627316448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b56b334d0 0x556b56b338b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.757+0000 7f85bb7fe640 1 -- 172.21.15.12:0/3627316448 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b56a271b0 con 0x556b56b3ca00 2024-04-22T13:31:54.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.757+0000 7f85c080a640 1 --2- 172.21.15.12:0/167771269 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f85a80a3da0 0x7f85a80a6260 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f85b40be280 tx=0x7f85b40b6040 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:54.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.757+0000 7f85bb7fe640 1 --2- 172.21.15.12:0/3627316448 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b56b3ca00 0x556b56b3ee50 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f85a4005d80 tx=0x7f85a400e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfbc0e7f27ca4138 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:54.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.757+0000 7f85b27fc640 1 -- 172.21.15.12:0/3627316448 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a4005020 con 0x556b56b3ca00 2024-04-22T13:31:54.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.757+0000 7f85b27fc640 1 -- 172.21.15.12:0/3627316448 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85a4017070 con 0x556b56b3ca00 2024-04-22T13:31:54.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.757+0000 7f85b27fc640 1 -- 172.21.15.12:0/3627316448 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a400b660 con 0x556b56b3ca00 2024-04-22T13:31:54.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.758+0000 7f85c4e66740 1 -- 172.21.15.12:0/3627316448 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b56b3ca00 msgr2=0x556b56b3ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.758+0000 7f85c4e66740 1 --2- 172.21.15.12:0/3627316448 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b56b3ca00 0x556b56b3ee50 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f85a4005d80 tx=0x7f85a400e350 comp rx=0 tx=0).stop 2024-04-22T13:31:54.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.758+0000 7f85c4e66740 1 -- 172.21.15.12:0/3627316448 shutdown_connections 2024-04-22T13:31:54.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.758+0000 7f85c4e66740 1 --2- 172.21.15.12:0/3627316448 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b56b3ca00 0x556b56b3ee50 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.758+0000 7f85c4e66740 1 --2- 172.21.15.12:0/3627316448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b56b33e80 0x556b56b3c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.758+0000 7f85c4e66740 1 --2- 172.21.15.12:0/3627316448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b56b334d0 0x556b56b338b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.758+0000 7f85c4e66740 1 -- 172.21.15.12:0/3627316448 >> 172.21.15.12:0/3627316448 conn(0x556b56b2dba0 msgr2=0x556b56b2df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:54.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.758+0000 7f85c4e66740 1 -- 172.21.15.12:0/3627316448 shutdown_connections 2024-04-22T13:31:54.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.758+0000 7f85c4e66740 1 -- 172.21.15.12:0/3627316448 wait complete. 2024-04-22T13:31:54.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.758+0000 7f85c4e66740 1 -- start start 2024-04-22T13:31:54.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.758+0000 7f85c4e66740 1 Processor -- start 2024-04-22T13:31:54.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.759+0000 7f85c4e66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b56b334d0 0x556b56b3c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.759+0000 7f85c4e66740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b56b33e80 0x556b56b39640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.759+0000 7f85c4e66740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b56b3ca00 0x556b56b39b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.759+0000 7f85c4e66740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x556b56b3f970 con 0x556b56b3ca00 2024-04-22T13:31:54.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.759+0000 7f85c4e66740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556b56a51500 con 0x556b56b33e80 2024-04-22T13:31:54.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.759+0000 7f85c4e66740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556b56b3faf0 con 0x556b56b334d0 2024-04-22T13:31:54.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.759+0000 7f85c4e66740 10 client.0 _ll_register_callbacks cb 0x556b56a1fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:54.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.759+0000 7f85bb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b56b3ca00 0x556b56b39b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.759+0000 7f85bb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b56b3ca00 0x556b56b39b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60838/0 (socket says 172.21.15.12:60838) 2024-04-22T13:31:54.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.759+0000 7f85bb7fe640 1 -- 172.21.15.12:0/492147020 learned_addr learned my addr 172.21.15.12:0/492147020 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:54.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.759+0000 7f85bbfff640 1 --2- 172.21.15.12:0/492147020 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b56b33e80 0x556b56b39640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.759+0000 7f85bb7fe640 1 --2- 172.21.15.12:0/492147020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b56b334d0 0x556b56b3c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.759+0000 7f85bb7fe640 1 -- 172.21.15.12:0/492147020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b56b334d0 msgr2=0x556b56b3c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.759+0000 7f85bb7fe640 1 --2- 172.21.15.12:0/492147020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b56b334d0 0x556b56b3c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.759+0000 7f85bb7fe640 1 -- 172.21.15.12:0/492147020 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b56b33e80 msgr2=0x556b56b39640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.759+0000 7f85bb7fe640 1 --2- 172.21.15.12:0/492147020 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b56b33e80 0x556b56b39640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.759+0000 7f85bb7fe640 1 -- 172.21.15.12:0/492147020 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b567aa3f0 con 0x556b56b3ca00 2024-04-22T13:31:54.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.759+0000 7f85bb7fe640 1 --2- 172.21.15.12:0/492147020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b56b3ca00 0x556b56b39b80 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f85a4001eb0 tx=0x7f85a4015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d257b96b3140c22d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:54.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.759+0000 7f859b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:54.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.759+0000 7f859b7fe640 1 -- 172.21.15.12:0/492147020 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a400b660 con 0x556b56b3ca00 2024-04-22T13:31:54.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.759+0000 7f859b7fe640 1 -- 172.21.15.12:0/492147020 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f85a400b800 con 0x556b56b3ca00 2024-04-22T13:31:54.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.759+0000 7f85c4e66740 1 -- 172.21.15.12:0/492147020 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85a4017070 con 0x556b56b3ca00 2024-04-22T13:31:54.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.760+0000 7f859b7fe640 1 -- 172.21.15.12:0/492147020 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f85a400ba00 con 0x556b56b3ca00 2024-04-22T13:31:54.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.760+0000 7f85c4e66740 10 client.18393 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:54.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.760+0000 7f85c4e66740 1 -- 172.21.15.12:0/492147020 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556b568747d0 con 0x556b56b3ca00 2024-04-22T13:31:54.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.760+0000 7f859b7fe640 1 -- 172.21.15.12:0/492147020 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85a4017070 con 0x556b56b3ca00 2024-04-22T13:31:54.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.760+0000 7f859b7fe640 1 -- 172.21.15.12:0/492147020 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f85a4034020 con 0x556b56b3ca00 2024-04-22T13:31:54.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.760+0000 7f85c4e66740 10 client.18393 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:54.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.760+0000 7f85c4e66740 10 client.18393 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:31:54.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.760+0000 7f85c4e66740 20 client.18393 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:54.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.760+0000 7f85c4e66740 1 --2- 172.21.15.12:0/492147020 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x556b56b50eb0 0x556b56b532a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.760+0000 7f85c4e66740 4 client.18393 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:54.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.760+0000 7f85c4e66740 1 -- 172.21.15.12:0/492147020 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b56a37930 con 0x556b56b50eb0 2024-04-22T13:31:54.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.760+0000 7f85bbfff640 1 --2- 172.21.15.12:0/492147020 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x556b56b50eb0 0x556b56b532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.761+0000 7f85bbfff640 1 --2- 172.21.15.12:0/492147020 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x556b56b50eb0 0x556b56b532a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=98fe4e0bdb751bf2 server_cookie=8a4121734af3dd99 in_seq=0 out_seq=0 2024-04-22T13:31:54.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.761+0000 7f859b7fe640 10 client.18393 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:31:54.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.762+0000 7f859b7fe640 1 -- 172.21.15.12:0/492147020 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556b56a37930 con 0x556b56b50eb0 2024-04-22T13:31:54.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.762+0000 7f859b7fe640 10 client.18393 handle_command_reply: tid=0 2024-04-22T13:31:54.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.762+0000 7f85c4e66740 1 client.18393 shutdown 2024-04-22T13:31:54.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.762+0000 7f85c4e66740 1 -- 172.21.15.12:0/492147020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b56b3ca00 msgr2=0x556b56b39b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.762+0000 7f85c4e66740 1 --2- 172.21.15.12:0/492147020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b56b3ca00 0x556b56b39b80 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f85a4001eb0 tx=0x7f85a4015040 comp rx=0 tx=0).stop 2024-04-22T13:31:54.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.762+0000 7f85c4e66740 1 -- 172.21.15.12:0/492147020 shutdown_connections 2024-04-22T13:31:54.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.762+0000 7f85c4e66740 1 --2- 172.21.15.12:0/492147020 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x556b56b50eb0 0x556b56b532a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.762+0000 7f85c4e66740 1 --2- 172.21.15.12:0/492147020 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b56b3ca00 0x556b56b39b80 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.762+0000 7f859b7fe640 0 client.18393 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:31:54.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.762+0000 7f85c4e66740 1 --2- 172.21.15.12:0/492147020 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b56b33e80 0x556b56b39640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.762+0000 7f85c4e66740 1 --2- 172.21.15.12:0/492147020 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b56b334d0 0x556b56b3c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.762+0000 7f85c4e66740 1 -- 172.21.15.12:0/492147020 >> 172.21.15.12:0/492147020 conn(0x556b56b2dba0 msgr2=0x556b56b2f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:54.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.762+0000 7f85c4e66740 1 -- 172.21.15.12:0/492147020 shutdown_connections 2024-04-22T13:31:54.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.762+0000 7f85c4e66740 1 -- 172.21.15.12:0/492147020 wait complete. 2024-04-22T13:31:54.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.762+0000 7f85c4e66740 20 client.18393 trim_cache size 0 max 16384 2024-04-22T13:31:54.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.775+0000 7f85c4e66740 1 Processor -- start 2024-04-22T13:31:54.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.775+0000 7f85c4e66740 1 -- start start 2024-04-22T13:31:54.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.775+0000 7f85c4e66740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b56b334d0 0x556b56b2c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.775+0000 7f85c4e66740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b56b33e80 0x556b56b3bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.775+0000 7f85c4e66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b56b3ca00 0x556b56b3c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.775+0000 7f85c4e66740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f85bc1ab860 con 0x556b56b33e80 2024-04-22T13:31:54.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.775+0000 7f85c4e66740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f85bc1ad3a0 con 0x556b56b334d0 2024-04-22T13:31:54.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.775+0000 7f85c4e66740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f85bc1abf30 con 0x556b56b3ca00 2024-04-22T13:31:54.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.775+0000 7f85bbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b56b33e80 0x556b56b3bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.775+0000 7f85bbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b56b33e80 0x556b56b3bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60852/0 (socket says 172.21.15.12:60852) 2024-04-22T13:31:54.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.775+0000 7f85bbfff640 1 -- 172.21.15.12:0/2085923421 learned_addr learned my addr 172.21.15.12:0/2085923421 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:54.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.776+0000 7f85bb7fe640 1 --2- 172.21.15.12:0/2085923421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b56b3ca00 0x556b56b3c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.776+0000 7f85bbfff640 1 -- 172.21.15.12:0/2085923421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b56b3ca00 msgr2=0x556b56b3c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.776+0000 7f85bbfff640 1 --2- 172.21.15.12:0/2085923421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b56b3ca00 0x556b56b3c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.776+0000 7f85bbfff640 1 -- 172.21.15.12:0/2085923421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b56b334d0 msgr2=0x556b56b2c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.776+0000 7f85bbfff640 1 --2- 172.21.15.12:0/2085923421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b56b334d0 0x556b56b2c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.776+0000 7f85bbfff640 1 -- 172.21.15.12:0/2085923421 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b56b33090 con 0x556b56b33e80 2024-04-22T13:31:54.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.776+0000 7f85bb7fe640 1 --2- 172.21.15.12:0/2085923421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b56b3ca00 0x556b56b3c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:54.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.776+0000 7f85bbfff640 1 --2- 172.21.15.12:0/2085923421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b56b33e80 0x556b56b3bd70 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x556b56b34280 tx=0x7f85ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd030195acb1066e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:54.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.776+0000 7f85b1ffb640 1 -- 172.21.15.12:0/2085923421 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac01c3b0 con 0x556b56b33e80 2024-04-22T13:31:54.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.776+0000 7f85b1ffb640 1 -- 172.21.15.12:0/2085923421 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85ac005020 con 0x556b56b33e80 2024-04-22T13:31:54.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.776+0000 7f85b1ffb640 1 -- 172.21.15.12:0/2085923421 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac004b20 con 0x556b56b33e80 2024-04-22T13:31:54.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.776+0000 7f85c4e66740 1 -- 172.21.15.12:0/2085923421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b56b33e80 msgr2=0x556b56b3bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.776+0000 7f85c4e66740 1 --2- 172.21.15.12:0/2085923421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b56b33e80 0x556b56b3bd70 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x556b56b34280 tx=0x7f85ac0046c0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.777+0000 7f85c4e66740 1 -- 172.21.15.12:0/2085923421 shutdown_connections 2024-04-22T13:31:54.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.777+0000 7f85c4e66740 1 --2- 172.21.15.12:0/2085923421 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b56b3ca00 0x556b56b3c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.777+0000 7f85c4e66740 1 --2- 172.21.15.12:0/2085923421 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b56b33e80 0x556b56b3bd70 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.777+0000 7f85c4e66740 1 --2- 172.21.15.12:0/2085923421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b56b334d0 0x556b56b2c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.777+0000 7f85c4e66740 1 -- 172.21.15.12:0/2085923421 >> 172.21.15.12:0/2085923421 conn(0x556b56b2dba0 msgr2=0x556b56b2f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:54.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.777+0000 7f85c4e66740 1 -- 172.21.15.12:0/2085923421 shutdown_connections 2024-04-22T13:31:54.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.777+0000 7f85c4e66740 1 -- 172.21.15.12:0/2085923421 wait complete. 2024-04-22T13:31:54.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.777+0000 7f85c4e66740 1 -- start start 2024-04-22T13:31:54.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.777+0000 7f85c4e66740 1 Processor -- start 2024-04-22T13:31:54.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.777+0000 7f85c4e66740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b56b334d0 0x556b56b49680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.777+0000 7f85c4e66740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b56b33e80 0x556b56b36130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.777+0000 7f85c4e66740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b56b3ca00 0x556b56b36690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.777+0000 7f85c4e66740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f85bc1accd0 con 0x556b56b3ca00 2024-04-22T13:31:54.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.777+0000 7f85c4e66740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f85bc1ac600 con 0x556b56b334d0 2024-04-22T13:31:54.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.777+0000 7f85c4e66740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f85bc1a9880 con 0x556b56b33e80 2024-04-22T13:31:54.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.777+0000 7f85c4e66740 10 client.0 _ll_register_callbacks cb 0x7f85a40136c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:54.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.777+0000 7f85bb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b56b3ca00 0x556b56b36690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.778+0000 7f85bb7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b56b3ca00 0x556b56b36690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60864/0 (socket says 172.21.15.12:60864) 2024-04-22T13:31:54.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.778+0000 7f85bb7fe640 1 -- 172.21.15.12:0/713082627 learned_addr learned my addr 172.21.15.12:0/713082627 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:54.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.778+0000 7f85bb7fe640 1 --2- 172.21.15.12:0/713082627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b56b334d0 0x556b56b49680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.783 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.778+0000 7f85bbfff640 1 --2- 172.21.15.12:0/713082627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b56b33e80 0x556b56b36130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.778+0000 7f85bb7fe640 1 -- 172.21.15.12:0/713082627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b56b33e80 msgr2=0x556b56b36130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.778+0000 7f85bb7fe640 1 --2- 172.21.15.12:0/713082627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b56b33e80 0x556b56b36130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.778+0000 7f85bb7fe640 1 -- 172.21.15.12:0/713082627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b56b334d0 msgr2=0x556b56b49680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.778+0000 7f85bb7fe640 1 --2- 172.21.15.12:0/713082627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b56b334d0 0x556b56b49680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.778+0000 7f85bb7fe640 1 -- 172.21.15.12:0/713082627 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b56b36ec0 con 0x556b56b3ca00 2024-04-22T13:31:54.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.778+0000 7f85bb7fe640 1 --2- 172.21.15.12:0/713082627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b56b3ca00 0x556b56b36690 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x556b56b2d510 tx=0x7f85a4004ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a7b5151e2432694 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:54.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.778+0000 7f859b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:54.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.778+0000 7f859b7fe640 1 -- 172.21.15.12:0/713082627 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a403c060 con 0x556b56b3ca00 2024-04-22T13:31:54.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.778+0000 7f859b7fe640 1 -- 172.21.15.12:0/713082627 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f85a4013070 con 0x556b56b3ca00 2024-04-22T13:31:54.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.778+0000 7f85c4e66740 1 -- 172.21.15.12:0/713082627 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85ac005020 con 0x556b56b3ca00 2024-04-22T13:31:54.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.778+0000 7f859b7fe640 1 -- 172.21.15.12:0/713082627 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f85a4005bf0 con 0x556b56b3ca00 2024-04-22T13:31:54.784 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.778+0000 7f85c4e66740 10 client.18399 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:54.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.778+0000 7f85c4e66740 1 -- 172.21.15.12:0/713082627 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556b56bb7140 con 0x556b56b3ca00 2024-04-22T13:31:54.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.779+0000 7f859b7fe640 1 -- 172.21.15.12:0/713082627 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85ac005020 con 0x556b56b3ca00 2024-04-22T13:31:54.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.779+0000 7f859b7fe640 1 -- 172.21.15.12:0/713082627 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f85a4034030 con 0x556b56b3ca00 2024-04-22T13:31:54.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.779+0000 7f85c4e66740 10 client.18399 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:54.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.779+0000 7f85c4e66740 10 client.18399 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:31:54.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.779+0000 7f85c4e66740 20 client.18399 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:54.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.779+0000 7f85c4e66740 1 --2- 172.21.15.12:0/713082627 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x556b56bb7920 0x556b56bb9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.779+0000 7f85c4e66740 4 client.18399 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:31:54.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.779+0000 7f85c4e66740 1 -- 172.21.15.12:0/713082627 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b56b2d220 con 0x556b56bb7920 2024-04-22T13:31:54.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.779+0000 7f85bbfff640 1 --2- 172.21.15.12:0/713082627 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x556b56bb7920 0x556b56bb9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.779+0000 7f85bbfff640 1 --2- 172.21.15.12:0/713082627 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x556b56bb7920 0x556b56bb9d40 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f0874e3fc42b275c server_cookie=b7ddb8569b6e564b in_seq=0 out_seq=0 2024-04-22T13:31:54.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.779+0000 7f859b7fe640 10 client.18399 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:31:54.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.780+0000 7f859b7fe640 1 -- 172.21.15.12:0/713082627 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b56b2d220 con 0x556b56bb7920 2024-04-22T13:31:54.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.780+0000 7f859b7fe640 10 client.18399 handle_command_reply: tid=0 2024-04-22T13:31:54.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.780+0000 7f85c4e66740 1 client.18399 shutdown 2024-04-22T13:31:54.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.780+0000 7f85c4e66740 1 -- 172.21.15.12:0/713082627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b56b3ca00 msgr2=0x556b56b36690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.780+0000 7f85c4e66740 1 --2- 172.21.15.12:0/713082627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b56b3ca00 0x556b56b36690 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x556b56b2d510 tx=0x7f85a4004ba0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.780+0000 7f85c4e66740 1 -- 172.21.15.12:0/713082627 shutdown_connections 2024-04-22T13:31:54.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.780+0000 7f85c4e66740 1 --2- 172.21.15.12:0/713082627 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x556b56bb7920 0x556b56bb9d40 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.780+0000 7f85c4e66740 1 --2- 172.21.15.12:0/713082627 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556b56b3ca00 0x556b56b36690 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.780+0000 7f85c4e66740 1 --2- 172.21.15.12:0/713082627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556b56b33e80 0x556b56b36130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.780+0000 7f859b7fe640 0 client.18399 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:31:54.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.780+0000 7f85c4e66740 1 --2- 172.21.15.12:0/713082627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b56b334d0 0x556b56b49680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.780+0000 7f85c4e66740 1 -- 172.21.15.12:0/713082627 >> 172.21.15.12:0/713082627 conn(0x556b56b2dba0 msgr2=0x556b56b342b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:54.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.780+0000 7f85c4e66740 1 -- 172.21.15.12:0/713082627 shutdown_connections 2024-04-22T13:31:54.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.780+0000 7f85c4e66740 1 -- 172.21.15.12:0/713082627 wait complete. 2024-04-22T13:31:54.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.780+0000 7f85c4e66740 20 client.18399 trim_cache size 0 max 16384 2024-04-22T13:31:54.788 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:31:54.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.780+0000 7f85c226d640 1 -- 172.21.15.12:0/167771269 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f85a80a3da0 msgr2=0x7f85a80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.780+0000 7f85c226d640 1 --2- 172.21.15.12:0/167771269 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f85a80a3da0 0x7f85a80a6260 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f85b40be280 tx=0x7f85b40b6040 comp rx=0 tx=0).stop 2024-04-22T13:31:54.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.780+0000 7f85c226d640 1 -- 172.21.15.12:0/167771269 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85bc19cef0 msgr2=0x7f85bc19d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.780+0000 7f85c226d640 1 --2- 172.21.15.12:0/167771269 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85bc19cef0 0x7f85bc19d3a0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f85b4005e30 tx=0x7f85b4034670 comp rx=0 tx=0).stop 2024-04-22T13:31:54.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.781+0000 7f85c226d640 1 -- 172.21.15.12:0/167771269 shutdown_connections 2024-04-22T13:31:54.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.781+0000 7f85c226d640 1 --2- 172.21.15.12:0/167771269 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f85a80a3da0 0x7f85a80a6260 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.781+0000 7f85c226d640 1 --2- 172.21.15.12:0/167771269 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f85bc19cef0 0x7f85bc19d3a0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.781+0000 7f85c226d640 1 --2- 172.21.15.12:0/167771269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85bc107270 0x7f85bc198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.781+0000 7f85c226d640 1 --2- 172.21.15.12:0/167771269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85bc06a840 0x7f85bc198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.781+0000 7f85c226d640 1 -- 172.21.15.12:0/167771269 >> 172.21.15.12:0/167771269 conn(0x7f85bc077eb0 msgr2=0x7f85bc0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:54.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.781+0000 7f85c226d640 1 -- 172.21.15.12:0/167771269 shutdown_connections 2024-04-22T13:31:54.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.781+0000 7f85c226d640 1 -- 172.21.15.12:0/167771269 wait complete. 2024-04-22T13:31:54.791 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:54 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/824132446' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:31:54.792 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:31:54.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.901+0000 7fc85b493640 1 -- 172.21.15.12:0/3479560029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc854074300 msgr2=0x7fc8540746e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.901+0000 7fc85b493640 1 --2- 172.21.15.12:0/3479560029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc854074300 0x7fc8540746e0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fc850009900 tx=0x7fc85002bcd0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.901+0000 7fc85b493640 1 -- 172.21.15.12:0/3479560029 shutdown_connections 2024-04-22T13:31:54.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.901+0000 7fc85b493640 1 --2- 172.21.15.12:0/3479560029 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc85410e7e0 0x7fc854110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.901+0000 7fc85b493640 1 --2- 172.21.15.12:0/3479560029 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc854074c20 0x7fc85410e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.901+0000 7fc85b493640 1 --2- 172.21.15.12:0/3479560029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc854074300 0x7fc8540746e0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.901+0000 7fc85b493640 1 -- 172.21.15.12:0/3479560029 >> 172.21.15.12:0/3479560029 conn(0x7fc8540fd990 msgr2=0x7fc8540ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:54.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.901+0000 7fc85b493640 1 -- 172.21.15.12:0/3479560029 shutdown_connections 2024-04-22T13:31:54.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.901+0000 7fc85b493640 1 -- 172.21.15.12:0/3479560029 wait complete. 2024-04-22T13:31:54.899 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.902+0000 7fc85b493640 1 Processor -- start 2024-04-22T13:31:54.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.902+0000 7fc85b493640 1 -- start start 2024-04-22T13:31:54.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.902+0000 7fc85b493640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc854074300 0x7fc8541a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.902+0000 7fc85b493640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc854074c20 0x7fc8541a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.902+0000 7fc85b493640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc85410e7e0 0x7fc85419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.902+0000 7fc85b493640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc8541143a0 con 0x7fc854074c20 2024-04-22T13:31:54.900 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.902+0000 7fc85b493640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc854114220 con 0x7fc854074300 2024-04-22T13:31:54.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.902+0000 7fc85b493640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc854114520 con 0x7fc85410e7e0 2024-04-22T13:31:54.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.902+0000 7fc858a2e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc854074c20 0x7fc8541a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.902+0000 7fc858a2e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc854074c20 0x7fc8541a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60880/0 (socket says 172.21.15.12:60880) 2024-04-22T13:31:54.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.902+0000 7fc858a2e640 1 -- 172.21.15.12:0/496132394 learned_addr learned my addr 172.21.15.12:0/496132394 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:54.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.902+0000 7fc85922f640 1 --2- 172.21.15.12:0/496132394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc854074300 0x7fc8541a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.901 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.903+0000 7fc859a30640 1 --2- 172.21.15.12:0/496132394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc85410e7e0 0x7fc85419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.903+0000 7fc858a2e640 1 -- 172.21.15.12:0/496132394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc85410e7e0 msgr2=0x7fc85419a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.903+0000 7fc858a2e640 1 --2- 172.21.15.12:0/496132394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc85410e7e0 0x7fc85419a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.903+0000 7fc858a2e640 1 -- 172.21.15.12:0/496132394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc854074300 msgr2=0x7fc8541a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.903+0000 7fc858a2e640 1 --2- 172.21.15.12:0/496132394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc854074300 0x7fc8541a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.903+0000 7fc858a2e640 1 -- 172.21.15.12:0/496132394 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc85419af90 con 0x7fc854074c20 2024-04-22T13:31:54.902 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.903+0000 7fc859a30640 1 --2- 172.21.15.12:0/496132394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc85410e7e0 0x7fc85419a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:54.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.903+0000 7fc85922f640 1 --2- 172.21.15.12:0/496132394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc854074300 0x7fc8541a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:54.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.903+0000 7fc858a2e640 1 --2- 172.21.15.12:0/496132394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc854074c20 0x7fc8541a0b90 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fc84400d770 tx=0x7fc84400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:54.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.903+0000 7fc84a7fc640 1 -- 172.21.15.12:0/496132394 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc844017070 con 0x7fc854074c20 2024-04-22T13:31:54.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.903+0000 7fc84a7fc640 1 -- 172.21.15.12:0/496132394 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8440054d0 con 0x7fc854074c20 2024-04-22T13:31:54.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.903+0000 7fc85b493640 1 -- 172.21.15.12:0/496132394 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc85419b280 con 0x7fc854074c20 2024-04-22T13:31:54.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.903+0000 7fc84a7fc640 1 -- 172.21.15.12:0/496132394 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc844004c30 con 0x7fc854074c20 2024-04-22T13:31:54.903 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.903+0000 7fc85b493640 1 -- 172.21.15.12:0/496132394 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8541a74a0 con 0x7fc854074c20 2024-04-22T13:31:54.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.905+0000 7fc84a7fc640 1 -- 172.21.15.12:0/496132394 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc844004e10 con 0x7fc854074c20 2024-04-22T13:31:54.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.906+0000 7fc85e165740 1 Processor -- start 2024-04-22T13:31:54.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.906+0000 7fc85e165740 1 -- start start 2024-04-22T13:31:54.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.906+0000 7fc85e165740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e53ec44d0 0x555e53ec48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.906+0000 7fc85e165740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e53ec4e80 0x555e53ecd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.906+0000 7fc85e165740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e53ecda00 0x555e53ecfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.906+0000 7fc85e165740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555e53e291f0 con 0x555e53ec44d0 2024-04-22T13:31:54.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.906+0000 7fc85e165740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555e53dc64b0 con 0x555e53ec4e80 2024-04-22T13:31:54.910 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.906+0000 7fc85e165740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555e53e251b0 con 0x555e53ecda00 2024-04-22T13:31:54.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.906+0000 7fc84a7fc640 1 --2- 172.21.15.12:0/496132394 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc8400a3da0 0x7fc8400a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.906+0000 7fc859a30640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e53ec44d0 0x555e53ec48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.906+0000 7fc859a30640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e53ec44d0 0x555e53ec48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60894/0 (socket says 172.21.15.12:60894) 2024-04-22T13:31:54.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.906+0000 7fc859a30640 1 -- 172.21.15.12:0/3101089153 learned_addr learned my addr 172.21.15.12:0/3101089153 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:54.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.906+0000 7fc858a2e640 1 --2- 172.21.15.12:0/3101089153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e53ecda00 0x555e53ecfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.906+0000 7fc85922f640 1 --2- 172.21.15.12:0/3101089153 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e53ec4e80 0x555e53ecd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.911 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.906+0000 7fc859a30640 1 --2- 172.21.15.12:0/496132394 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc8400a3da0 0x7fc8400a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.906+0000 7fc84a7fc640 1 -- 172.21.15.12:0/496132394 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fc8440d02e0 con 0x7fc854074c20 2024-04-22T13:31:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.906+0000 7fc859a30640 1 -- 172.21.15.12:0/3101089153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e53ecda00 msgr2=0x555e53ecfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.906+0000 7fc859a30640 1 --2- 172.21.15.12:0/3101089153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e53ecda00 0x555e53ecfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.906+0000 7fc859a30640 1 -- 172.21.15.12:0/3101089153 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e53ec4e80 msgr2=0x555e53ecd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.906+0000 7fc859a30640 1 --2- 172.21.15.12:0/3101089153 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e53ec4e80 0x555e53ecd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.906+0000 7fc859a30640 1 -- 172.21.15.12:0/3101089153 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e53db81b0 con 0x555e53ec44d0 2024-04-22T13:31:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.907+0000 7fc859a30640 1 --2- 172.21.15.12:0/3101089153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e53ec44d0 0x555e53ec48b0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fc84c014930 tx=0x7fc84c014d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd035039d505add6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.907+0000 7fc83f7fe640 1 -- 172.21.15.12:0/3101089153 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c021070 con 0x555e53ec44d0 2024-04-22T13:31:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.907+0000 7fc83f7fe640 1 -- 172.21.15.12:0/3101089153 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc84c008df0 con 0x555e53ec44d0 2024-04-22T13:31:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.907+0000 7fc83f7fe640 1 -- 172.21.15.12:0/3101089153 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c013d10 con 0x555e53ec44d0 2024-04-22T13:31:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.907+0000 7fc85e165740 1 -- 172.21.15.12:0/3101089153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e53ec44d0 msgr2=0x555e53ec48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.907+0000 7fc85e165740 1 --2- 172.21.15.12:0/3101089153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e53ec44d0 0x555e53ec48b0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fc84c014930 tx=0x7fc84c014d30 comp rx=0 tx=0).stop 2024-04-22T13:31:54.912 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.907+0000 7fc859a30640 1 --2- 172.21.15.12:0/496132394 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc8400a3da0 0x7fc8400a6260 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc84c01e000 tx=0x7fc84c01a000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:54.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.907+0000 7fc85e165740 1 -- 172.21.15.12:0/3101089153 shutdown_connections 2024-04-22T13:31:54.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.908+0000 7fc85e165740 1 --2- 172.21.15.12:0/3101089153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e53ecda00 0x555e53ecfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.908+0000 7fc85e165740 1 --2- 172.21.15.12:0/3101089153 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e53ec4e80 0x555e53ecd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.908+0000 7fc85e165740 1 --2- 172.21.15.12:0/3101089153 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e53ec44d0 0x555e53ec48b0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.908+0000 7fc85e165740 1 -- 172.21.15.12:0/3101089153 >> 172.21.15.12:0/3101089153 conn(0x555e53ebeba0 msgr2=0x555e53ebef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:54.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.908+0000 7fc85e165740 1 -- 172.21.15.12:0/3101089153 shutdown_connections 2024-04-22T13:31:54.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.908+0000 7fc85e165740 1 -- 172.21.15.12:0/3101089153 wait complete. 2024-04-22T13:31:54.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.908+0000 7fc85e165740 1 -- start start 2024-04-22T13:31:54.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.908+0000 7fc85e165740 1 Processor -- start 2024-04-22T13:31:54.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.908+0000 7fc85e165740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e53ec44d0 0x555e53ecd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.908+0000 7fc85e165740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e53ec4e80 0x555e53eca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.908+0000 7fc85e165740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e53ecda00 0x555e53ecace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.908+0000 7fc85e165740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555e53ed0970 con 0x555e53ec44d0 2024-04-22T13:31:54.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.908+0000 7fc85e165740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555e53de2500 con 0x555e53ec4e80 2024-04-22T13:31:54.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.908+0000 7fc85e165740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555e53ed0af0 con 0x555e53ecda00 2024-04-22T13:31:54.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.908+0000 7fc85e165740 10 client.0 _ll_register_callbacks cb 0x555e53db0b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:54.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.908+0000 7fc85922f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e53ec44d0 0x555e53ecd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.909+0000 7fc85922f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e53ec44d0 0x555e53ecd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60906/0 (socket says 172.21.15.12:60906) 2024-04-22T13:31:54.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.909+0000 7fc85922f640 1 -- 172.21.15.12:0/2211077478 learned_addr learned my addr 172.21.15.12:0/2211077478 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:54.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.909+0000 7fc859a30640 1 --2- 172.21.15.12:0/2211077478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e53ecda00 0x555e53ecace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.909+0000 7fc858a2e640 1 --2- 172.21.15.12:0/2211077478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e53ec4e80 0x555e53eca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.909+0000 7fc85922f640 1 -- 172.21.15.12:0/2211077478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e53ecda00 msgr2=0x555e53ecace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.909+0000 7fc85922f640 1 --2- 172.21.15.12:0/2211077478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e53ecda00 0x555e53ecace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.909+0000 7fc85922f640 1 -- 172.21.15.12:0/2211077478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e53ec4e80 msgr2=0x555e53eca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.909+0000 7fc85922f640 1 --2- 172.21.15.12:0/2211077478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e53ec4e80 0x555e53eca6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.909+0000 7fc85922f640 1 -- 172.21.15.12:0/2211077478 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e53b3b3f0 con 0x555e53ec44d0 2024-04-22T13:31:54.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.909+0000 7fc85922f640 1 --2- 172.21.15.12:0/2211077478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e53ec44d0 0x555e53ecd340 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fc85002c210 tx=0x7fc850005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d16bc65eafd03c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:54.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.909+0000 7fc83cff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:54.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.909+0000 7fc83cff9640 1 -- 172.21.15.12:0/2211077478 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc850043070 con 0x555e53ec44d0 2024-04-22T13:31:54.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.909+0000 7fc83cff9640 1 -- 172.21.15.12:0/2211077478 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fc85002ca80 con 0x555e53ec44d0 2024-04-22T13:31:54.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.909+0000 7fc85e165740 1 -- 172.21.15.12:0/2211077478 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc84c008df0 con 0x555e53ec44d0 2024-04-22T13:31:54.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.909+0000 7fc83cff9640 1 -- 172.21.15.12:0/2211077478 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc85004f050 con 0x555e53ec44d0 2024-04-22T13:31:54.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.909+0000 7fc85e165740 10 client.18411 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:54.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.909+0000 7fc85e165740 1 -- 172.21.15.12:0/2211077478 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555e53c057d0 con 0x555e53ec44d0 2024-04-22T13:31:54.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.910+0000 7fc83cff9640 1 -- 172.21.15.12:0/2211077478 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc84c008df0 con 0x555e53ec44d0 2024-04-22T13:31:54.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.910+0000 7fc83cff9640 1 -- 172.21.15.12:0/2211077478 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc8500035f0 con 0x555e53ec44d0 2024-04-22T13:31:54.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.910+0000 7fc85e165740 10 client.18411 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:54.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.910+0000 7fc85e165740 10 client.18411 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:31:54.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.910+0000 7fc85e165740 20 client.18411 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:54.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.910+0000 7fc85e165740 1 --2- 172.21.15.12:0/2211077478 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x555e53eddbd0 0x555e53ee0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.910+0000 7fc85e165740 4 client.18411 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:54.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.910+0000 7fc85e165740 1 -- 172.21.15.12:0/2211077478 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e53dc8930 con 0x555e53eddbd0 2024-04-22T13:31:54.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.910+0000 7fc858a2e640 1 --2- 172.21.15.12:0/2211077478 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x555e53eddbd0 0x555e53ee0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.910+0000 7fc83cff9640 10 client.18411 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:31:54.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.910+0000 7fc858a2e640 1 --2- 172.21.15.12:0/2211077478 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x555e53eddbd0 0x555e53ee0000 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a75c48d119f2b846 server_cookie=b5281fac65d13460 in_seq=0 out_seq=0 2024-04-22T13:31:54.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.911+0000 7fc83cff9640 1 -- 172.21.15.12:0/2211077478 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555e53dc8930 con 0x555e53eddbd0 2024-04-22T13:31:54.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.911+0000 7fc83cff9640 10 client.18411 handle_command_reply: tid=0 2024-04-22T13:31:54.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.911+0000 7fc85e165740 1 client.18411 shutdown 2024-04-22T13:31:54.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.911+0000 7fc85e165740 1 -- 172.21.15.12:0/2211077478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e53ec44d0 msgr2=0x555e53ecd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.911+0000 7fc85e165740 1 --2- 172.21.15.12:0/2211077478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e53ec44d0 0x555e53ecd340 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fc85002c210 tx=0x7fc850005250 comp rx=0 tx=0).stop 2024-04-22T13:31:54.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.911+0000 7fc85e165740 1 -- 172.21.15.12:0/2211077478 shutdown_connections 2024-04-22T13:31:54.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.911+0000 7fc85e165740 1 --2- 172.21.15.12:0/2211077478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e53ecda00 0x555e53ecace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.911+0000 7fc85e165740 1 --2- 172.21.15.12:0/2211077478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e53ec4e80 0x555e53eca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.911+0000 7fc85e165740 1 --2- 172.21.15.12:0/2211077478 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x555e53eddbd0 0x555e53ee0000 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.911+0000 7fc85e165740 1 --2- 172.21.15.12:0/2211077478 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e53ec44d0 0x555e53ecd340 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.911+0000 7fc85e165740 1 -- 172.21.15.12:0/2211077478 >> 172.21.15.12:0/2211077478 conn(0x555e53ebeba0 msgr2=0x555e53ec08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:54.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.911+0000 7fc83cff9640 0 client.18411 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:31:54.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.912+0000 7fc85e165740 1 -- 172.21.15.12:0/2211077478 shutdown_connections 2024-04-22T13:31:54.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.912+0000 7fc85e165740 1 -- 172.21.15.12:0/2211077478 wait complete. 2024-04-22T13:31:54.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.912+0000 7fc85e165740 20 client.18411 trim_cache size 0 max 16384 2024-04-22T13:31:54.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.925+0000 7fc85e165740 1 Processor -- start 2024-04-22T13:31:54.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.925+0000 7fc85e165740 1 -- start start 2024-04-22T13:31:54.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.925+0000 7fc85e165740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e53ec44d0 0x555e53ed9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.925+0000 7fc85e165740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e53ec4e80 0x555e53ec05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.925+0000 7fc85e165740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e53ecda00 0x555e53ec0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.925+0000 7fc85e165740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc8541afd00 con 0x555e53ec44d0 2024-04-22T13:31:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.925+0000 7fc85e165740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc8541b1840 con 0x555e53ecda00 2024-04-22T13:31:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.925+0000 7fc85e165740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc8541b03d0 con 0x555e53ec4e80 2024-04-22T13:31:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.925+0000 7fc85922f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e53ec44d0 0x555e53ed9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.925+0000 7fc85922f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e53ec44d0 0x555e53ed9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60914/0 (socket says 172.21.15.12:60914) 2024-04-22T13:31:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.925+0000 7fc85922f640 1 -- 172.21.15.12:0/3449542223 learned_addr learned my addr 172.21.15.12:0/3449542223 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.926+0000 7fc858a2e640 1 --2- 172.21.15.12:0/3449542223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e53ec4e80 0x555e53ec05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.926+0000 7fc85922f640 1 -- 172.21.15.12:0/3449542223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e53ec4e80 msgr2=0x555e53ec05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.926+0000 7fc85922f640 1 --2- 172.21.15.12:0/3449542223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e53ec4e80 0x555e53ec05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.926+0000 7fc85922f640 1 -- 172.21.15.12:0/3449542223 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e53ecda00 msgr2=0x555e53ec0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.926+0000 7fc85922f640 1 --2- 172.21.15.12:0/3449542223 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e53ecda00 0x555e53ec0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.926+0000 7fc85922f640 1 -- 172.21.15.12:0/3449542223 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e53ecb2e0 con 0x555e53ec44d0 2024-04-22T13:31:54.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.926+0000 7fc858a2e640 1 --2- 172.21.15.12:0/3449542223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e53ec4e80 0x555e53ec05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.926+0000 7fc85922f640 1 --2- 172.21.15.12:0/3449542223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e53ec44d0 0x555e53ed9830 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fc85002f020 tx=0x7fc85002bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299290da047e1af5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.926+0000 7fc83effd640 1 -- 172.21.15.12:0/3449542223 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc850043040 con 0x555e53ec44d0 2024-04-22T13:31:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.926+0000 7fc83effd640 1 -- 172.21.15.12:0/3449542223 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc850003d40 con 0x555e53ec44d0 2024-04-22T13:31:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.926+0000 7fc83effd640 1 -- 172.21.15.12:0/3449542223 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc850004920 con 0x555e53ec44d0 2024-04-22T13:31:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.927+0000 7fc85e165740 1 -- 172.21.15.12:0/3449542223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e53ec44d0 msgr2=0x555e53ed9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.927+0000 7fc85e165740 1 --2- 172.21.15.12:0/3449542223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e53ec44d0 0x555e53ed9830 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fc85002f020 tx=0x7fc85002bcd0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.927+0000 7fc85e165740 1 -- 172.21.15.12:0/3449542223 shutdown_connections 2024-04-22T13:31:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.927+0000 7fc85e165740 1 --2- 172.21.15.12:0/3449542223 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e53ecda00 0x555e53ec0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.927+0000 7fc85e165740 1 --2- 172.21.15.12:0/3449542223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e53ec4e80 0x555e53ec05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.927+0000 7fc85e165740 1 --2- 172.21.15.12:0/3449542223 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e53ec44d0 0x555e53ed9830 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.927+0000 7fc85e165740 1 -- 172.21.15.12:0/3449542223 >> 172.21.15.12:0/3449542223 conn(0x555e53ebeba0 msgr2=0x555e53ec5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.927+0000 7fc85e165740 1 -- 172.21.15.12:0/3449542223 shutdown_connections 2024-04-22T13:31:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.927+0000 7fc85e165740 1 -- 172.21.15.12:0/3449542223 wait complete. 2024-04-22T13:31:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.927+0000 7fc85e165740 1 -- start start 2024-04-22T13:31:54.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.927+0000 7fc85e165740 1 Processor -- start 2024-04-22T13:31:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.927+0000 7fc85e165740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e53ec44d0 0x555e53ed6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.927+0000 7fc85e165740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e53ec4e80 0x555e53ed1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.927+0000 7fc85e165740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e53ecda00 0x555e53ed1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.927+0000 7fc85e165740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc8541b1170 con 0x555e53ec44d0 2024-04-22T13:31:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.927+0000 7fc85e165740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc8541b0aa0 con 0x555e53ec4e80 2024-04-22T13:31:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.927+0000 7fc85e165740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc8541add00 con 0x555e53ecda00 2024-04-22T13:31:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.928+0000 7fc85e165740 10 client.0 _ll_register_callbacks cb 0x7fc8500043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.928+0000 7fc85922f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e53ec44d0 0x555e53ed6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.928+0000 7fc85922f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e53ec44d0 0x555e53ed6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60926/0 (socket says 172.21.15.12:60926) 2024-04-22T13:31:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.928+0000 7fc85922f640 1 -- 172.21.15.12:0/647797517 learned_addr learned my addr 172.21.15.12:0/647797517 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.928+0000 7fc859a30640 1 --2- 172.21.15.12:0/647797517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e53ecda00 0x555e53ed1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.928+0000 7fc858a2e640 1 --2- 172.21.15.12:0/647797517 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e53ec4e80 0x555e53ed1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.928+0000 7fc85922f640 1 -- 172.21.15.12:0/647797517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e53ecda00 msgr2=0x555e53ed1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.932 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.928+0000 7fc85922f640 1 --2- 172.21.15.12:0/647797517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e53ecda00 0x555e53ed1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.928+0000 7fc85922f640 1 -- 172.21.15.12:0/647797517 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e53ec4e80 msgr2=0x555e53ed1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.928+0000 7fc85922f640 1 --2- 172.21.15.12:0/647797517 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e53ec4e80 0x555e53ed1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.928+0000 7fc85922f640 1 -- 172.21.15.12:0/647797517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e53f43140 con 0x555e53ec44d0 2024-04-22T13:31:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.928+0000 7fc858a2e640 1 --2- 172.21.15.12:0/647797517 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e53ec4e80 0x555e53ed1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.928+0000 7fc85922f640 1 --2- 172.21.15.12:0/647797517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e53ec44d0 0x555e53ed6290 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fc85002c210 tx=0x7fc850005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac17f2b0001ef497 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.928+0000 7fc83cff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.928+0000 7fc83cff9640 1 -- 172.21.15.12:0/647797517 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc850043040 con 0x555e53ec44d0 2024-04-22T13:31:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.928+0000 7fc83cff9640 1 -- 172.21.15.12:0/647797517 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fc85002cc90 con 0x555e53ec44d0 2024-04-22T13:31:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.928+0000 7fc85e165740 1 -- 172.21.15.12:0/647797517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc850003d40 con 0x555e53ec44d0 2024-04-22T13:31:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.928+0000 7fc83cff9640 1 -- 172.21.15.12:0/647797517 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc8500046c0 con 0x555e53ec44d0 2024-04-22T13:31:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.928+0000 7fc85e165740 10 client.18417 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.928+0000 7fc85e165740 1 -- 172.21.15.12:0/647797517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555e53f44250 con 0x555e53ec44d0 2024-04-22T13:31:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.929+0000 7fc83cff9640 1 -- 172.21.15.12:0/647797517 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc850003d40 con 0x555e53ec44d0 2024-04-22T13:31:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.929+0000 7fc83cff9640 1 -- 172.21.15.12:0/647797517 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc850063070 con 0x555e53ec44d0 2024-04-22T13:31:54.933 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.929+0000 7fc85e165740 10 client.18417 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.929+0000 7fc85e165740 10 client.18417 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:31:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.929+0000 7fc85e165740 20 client.18417 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.929+0000 7fc85e165740 1 --2- 172.21.15.12:0/647797517 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x555e53f448b0 0x555e53f46cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.929+0000 7fc85e165740 4 client.18417 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:31:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.929+0000 7fc85e165740 1 -- 172.21.15.12:0/647797517 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555e53e36c10 con 0x555e53f448b0 2024-04-22T13:31:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.929+0000 7fc858a2e640 1 --2- 172.21.15.12:0/647797517 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x555e53f448b0 0x555e53f46cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.929+0000 7fc83cff9640 10 client.18417 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:31:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.929+0000 7fc858a2e640 1 --2- 172.21.15.12:0/647797517 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x555e53f448b0 0x555e53f46cd0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=52dc471d2c11c0f9 server_cookie=40537108caf74e89 in_seq=0 out_seq=0 2024-04-22T13:31:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.930+0000 7fc83cff9640 1 -- 172.21.15.12:0/647797517 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555e53e36c10 con 0x555e53f448b0 2024-04-22T13:31:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.930+0000 7fc83cff9640 10 client.18417 handle_command_reply: tid=0 2024-04-22T13:31:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.930+0000 7fc85e165740 1 client.18417 shutdown 2024-04-22T13:31:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.930+0000 7fc85e165740 1 -- 172.21.15.12:0/647797517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e53ec44d0 msgr2=0x555e53ed6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.930+0000 7fc85e165740 1 --2- 172.21.15.12:0/647797517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e53ec44d0 0x555e53ed6290 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fc85002c210 tx=0x7fc850005250 comp rx=0 tx=0).stop 2024-04-22T13:31:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.930+0000 7fc85e165740 1 -- 172.21.15.12:0/647797517 shutdown_connections 2024-04-22T13:31:54.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.930+0000 7fc85e165740 1 --2- 172.21.15.12:0/647797517 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555e53ecda00 0x555e53ed1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.930+0000 7fc85e165740 1 --2- 172.21.15.12:0/647797517 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e53ec4e80 0x555e53ed1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.930+0000 7fc85e165740 1 --2- 172.21.15.12:0/647797517 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x555e53f448b0 0x555e53f46cd0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.930+0000 7fc85e165740 1 --2- 172.21.15.12:0/647797517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555e53ec44d0 0x555e53ed6290 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.930+0000 7fc83cff9640 0 client.18417 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:31:54.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.930+0000 7fc85e165740 1 -- 172.21.15.12:0/647797517 >> 172.21.15.12:0/647797517 conn(0x555e53ebeba0 msgr2=0x555e53ec05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:54.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.930+0000 7fc85e165740 1 -- 172.21.15.12:0/647797517 shutdown_connections 2024-04-22T13:31:54.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.930+0000 7fc85e165740 1 -- 172.21.15.12:0/647797517 wait complete. 2024-04-22T13:31:54.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.930+0000 7fc85e165740 20 client.18417 trim_cache size 0 max 16384 2024-04-22T13:31:54.935 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:31:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.930+0000 7fc85b493640 1 -- 172.21.15.12:0/496132394 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc8400a3da0 msgr2=0x7fc8400a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.931+0000 7fc85b493640 1 --2- 172.21.15.12:0/496132394 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc8400a3da0 0x7fc8400a6260 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc84c01e000 tx=0x7fc84c01a000 comp rx=0 tx=0).stop 2024-04-22T13:31:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.931+0000 7fc85b493640 1 -- 172.21.15.12:0/496132394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc854074c20 msgr2=0x7fc8541a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.931+0000 7fc85b493640 1 --2- 172.21.15.12:0/496132394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc854074c20 0x7fc8541a0b90 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fc84400d770 tx=0x7fc84400dc30 comp rx=0 tx=0).stop 2024-04-22T13:31:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.931+0000 7fc85b493640 1 -- 172.21.15.12:0/496132394 shutdown_connections 2024-04-22T13:31:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.931+0000 7fc85b493640 1 --2- 172.21.15.12:0/496132394 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc8400a3da0 0x7fc8400a6260 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.931+0000 7fc85b493640 1 --2- 172.21.15.12:0/496132394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc85410e7e0 0x7fc85419a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.931+0000 7fc85b493640 1 --2- 172.21.15.12:0/496132394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc854074c20 0x7fc8541a0b90 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.931+0000 7fc85b493640 1 --2- 172.21.15.12:0/496132394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc854074300 0x7fc8541a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:54.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.931+0000 7fc85b493640 1 -- 172.21.15.12:0/496132394 >> 172.21.15.12:0/496132394 conn(0x7fc8540fd990 msgr2=0x7fc8540fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:54.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.931+0000 7fc85b493640 1 -- 172.21.15.12:0/496132394 shutdown_connections 2024-04-22T13:31:54.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:54.931+0000 7fc85b493640 1 -- 172.21.15.12:0/496132394 wait complete. 2024-04-22T13:31:54.938 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:31:54.938 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:54 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/824132446' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:31:54.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:54 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/824132446' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:31:55.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.047+0000 7f68aaf15640 1 -- 172.21.15.12:0/696808096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68a410e900 msgr2=0x7f68a4110cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:55.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.047+0000 7f68aaf15640 1 --2- 172.21.15.12:0/696808096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68a410e900 0x7f68a4110cf0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f689c009ca0 tx=0x7f689c02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.058 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.047+0000 7f68aaf15640 1 -- 172.21.15.12:0/696808096 shutdown_connections 2024-04-22T13:31:55.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.047+0000 7f68aaf15640 1 --2- 172.21.15.12:0/696808096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68a410e900 0x7f68a4110cf0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.047+0000 7f68aaf15640 1 --2- 172.21.15.12:0/696808096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68a4101670 0x7f68a410e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.047+0000 7f68aaf15640 1 --2- 172.21.15.12:0/696808096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68a4100d50 0x7f68a4101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.047+0000 7f68aaf15640 1 -- 172.21.15.12:0/696808096 >> 172.21.15.12:0/696808096 conn(0x7f68a4077f50 msgr2=0x7f68a4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:55.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.047+0000 7f68aaf15640 1 -- 172.21.15.12:0/696808096 shutdown_connections 2024-04-22T13:31:55.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.047+0000 7f68aaf15640 1 -- 172.21.15.12:0/696808096 wait complete. 2024-04-22T13:31:55.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.047+0000 7f68aaf15640 1 Processor -- start 2024-04-22T13:31:55.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.047+0000 7f68aaf15640 1 -- start start 2024-04-22T13:31:55.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.047+0000 7f68aaf15640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68a4100d50 0x7f68a41a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.047+0000 7f68aaf15640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68a4101670 0x7f68a41a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.047+0000 7f68aaf15640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68a410e900 0x7f68a419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.047+0000 7f68aaf15640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41143a0 con 0x7f68a4100d50 2024-04-22T13:31:55.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.047+0000 7f68aaf15640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f68a4114220 con 0x7f68a4101670 2024-04-22T13:31:55.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.047+0000 7f68aaf15640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f68a4114520 con 0x7f68a410e900 2024-04-22T13:31:55.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.048+0000 7f68a8cb1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68a4100d50 0x7f68a41a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.048+0000 7f68a8cb1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68a4100d50 0x7f68a41a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60944/0 (socket says 172.21.15.12:60944) 2024-04-22T13:31:55.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.048+0000 7f68a8cb1640 1 -- 172.21.15.12:0/232274062 learned_addr learned my addr 172.21.15.12:0/232274062 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:55.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.048+0000 7f68a8cb1640 1 -- 172.21.15.12:0/232274062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68a410e900 msgr2=0x7f68a419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:55.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.048+0000 7f68a8cb1640 1 --2- 172.21.15.12:0/232274062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68a410e900 0x7f68a419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.048+0000 7f68a8cb1640 1 -- 172.21.15.12:0/232274062 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68a4101670 msgr2=0x7f68a41a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:55.061 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.048+0000 7f68a3fff640 1 --2- 172.21.15.12:0/232274062 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68a4101670 0x7f68a41a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.048+0000 7f68a8cb1640 1 --2- 172.21.15.12:0/232274062 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68a4101670 0x7f68a41a0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.048+0000 7f68a8cb1640 1 -- 172.21.15.12:0/232274062 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a419af60 con 0x7f68a4100d50 2024-04-22T13:31:55.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.048+0000 7f68a8cb1640 1 --2- 172.21.15.12:0/232274062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68a4100d50 0x7f68a41a05e0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f689400c920 tx=0x7f689400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:55.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.048+0000 7f68a1ffb640 1 -- 172.21.15.12:0/232274062 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6894007c70 con 0x7f68a4100d50 2024-04-22T13:31:55.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.048+0000 7f68a1ffb640 1 -- 172.21.15.12:0/232274062 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6894007e10 con 0x7f68a4100d50 2024-04-22T13:31:55.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.048+0000 7f68aaf15640 1 -- 172.21.15.12:0/232274062 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68a419b250 con 0x7f68a4100d50 2024-04-22T13:31:55.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.048+0000 7f68a1ffb640 1 -- 172.21.15.12:0/232274062 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6894018820 con 0x7f68a4100d50 2024-04-22T13:31:55.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.048+0000 7f68aaf15640 1 -- 172.21.15.12:0/232274062 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68a41a7560 con 0x7f68a4100d50 2024-04-22T13:31:55.062 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.051+0000 7f68a1ffb640 1 -- 172.21.15.12:0/232274062 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f689400f450 con 0x7f68a4100d50 2024-04-22T13:31:55.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.051+0000 7f68adbde740 1 Processor -- start 2024-04-22T13:31:55.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.051+0000 7f68adbde740 1 -- start start 2024-04-22T13:31:55.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.051+0000 7f68adbde740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb1ad424d0 0x55eb1ad428b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.051+0000 7f68adbde740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb1ad42e80 0x55eb1ad4b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.051+0000 7f68adbde740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55eb1ad4ba00 0x55eb1ad4de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.051+0000 7f68adbde740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55eb1aca71f0 con 0x55eb1ad42e80 2024-04-22T13:31:55.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.051+0000 7f68adbde740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55eb1ac444b0 con 0x55eb1ad4ba00 2024-04-22T13:31:55.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.051+0000 7f68adbde740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55eb1aca31b0 con 0x55eb1ad424d0 2024-04-22T13:31:55.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.051+0000 7f68a1ffb640 1 --2- 172.21.15.12:0/232274062 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f688c0a3d20 0x7f688c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.051+0000 7f68a8cb1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb1ad42e80 0x55eb1ad4b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.052+0000 7f68a8cb1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb1ad42e80 0x55eb1ad4b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60954/0 (socket says 172.21.15.12:60954) 2024-04-22T13:31:55.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.052+0000 7f68a8cb1640 1 -- 172.21.15.12:0/4210309515 learned_addr learned my addr 172.21.15.12:0/4210309515 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:55.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.052+0000 7f68a94b2640 1 --2- 172.21.15.12:0/232274062 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f688c0a3d20 0x7f688c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.052+0000 7f68a94b2640 1 --2- 172.21.15.12:0/4210309515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb1ad424d0 0x55eb1ad428b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.052+0000 7f68a1ffb640 1 -- 172.21.15.12:0/232274062 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f68940d09b0 con 0x7f68a4100d50 2024-04-22T13:31:55.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.052+0000 7f68a3fff640 1 --2- 172.21.15.12:0/4210309515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55eb1ad4ba00 0x55eb1ad4de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.052+0000 7f68a8cb1640 1 -- 172.21.15.12:0/4210309515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb1ad424d0 msgr2=0x55eb1ad428b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:55.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.052+0000 7f68a8cb1640 1 --2- 172.21.15.12:0/4210309515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb1ad424d0 0x55eb1ad428b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.052+0000 7f68a8cb1640 1 -- 172.21.15.12:0/4210309515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55eb1ad4ba00 msgr2=0x55eb1ad4de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:55.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.052+0000 7f68a8cb1640 1 --2- 172.21.15.12:0/4210309515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55eb1ad4ba00 0x55eb1ad4de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.052+0000 7f68a8cb1640 1 -- 172.21.15.12:0/4210309515 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb1ac361b0 con 0x55eb1ad42e80 2024-04-22T13:31:55.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.052+0000 7f68a8cb1640 1 --2- 172.21.15.12:0/4210309515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb1ad42e80 0x55eb1ad4b430 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f68940ce090 tx=0x7f6894000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a39417a010e8693b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:55.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.052+0000 7f68a94b2640 1 --2- 172.21.15.12:0/232274062 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f688c0a3d20 0x7f688c0a61e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f689c02c630 tx=0x7f689c009800 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:55.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.052+0000 7f688affd640 1 -- 172.21.15.12:0/4210309515 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68940d0e60 con 0x55eb1ad42e80 2024-04-22T13:31:55.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.052+0000 7f688affd640 1 -- 172.21.15.12:0/4210309515 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68940a0d20 con 0x55eb1ad42e80 2024-04-22T13:31:55.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.052+0000 7f688affd640 1 -- 172.21.15.12:0/4210309515 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6894098fe0 con 0x55eb1ad42e80 2024-04-22T13:31:55.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.053+0000 7f68adbde740 1 -- 172.21.15.12:0/4210309515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb1ad42e80 msgr2=0x55eb1ad4b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:55.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.053+0000 7f68adbde740 1 --2- 172.21.15.12:0/4210309515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb1ad42e80 0x55eb1ad4b430 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f68940ce090 tx=0x7f6894000bc0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.053+0000 7f68adbde740 1 -- 172.21.15.12:0/4210309515 shutdown_connections 2024-04-22T13:31:55.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.053+0000 7f68adbde740 1 --2- 172.21.15.12:0/4210309515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55eb1ad4ba00 0x55eb1ad4de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.053+0000 7f68adbde740 1 --2- 172.21.15.12:0/4210309515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb1ad42e80 0x55eb1ad4b430 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.053+0000 7f68adbde740 1 --2- 172.21.15.12:0/4210309515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb1ad424d0 0x55eb1ad428b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.053+0000 7f68adbde740 1 -- 172.21.15.12:0/4210309515 >> 172.21.15.12:0/4210309515 conn(0x55eb1ad3cba0 msgr2=0x55eb1ad3cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:55.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.053+0000 7f68adbde740 1 -- 172.21.15.12:0/4210309515 shutdown_connections 2024-04-22T13:31:55.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.053+0000 7f68adbde740 1 -- 172.21.15.12:0/4210309515 wait complete. 2024-04-22T13:31:55.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.053+0000 7f68adbde740 1 -- start start 2024-04-22T13:31:55.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.053+0000 7f68adbde740 1 Processor -- start 2024-04-22T13:31:55.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.054+0000 7f68adbde740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55eb1ad424d0 0x55eb1ad4b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.054+0000 7f68adbde740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb1ad42e80 0x55eb1ad48650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.054+0000 7f68adbde740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb1ad4ba00 0x55eb1ad48bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.054+0000 7f68adbde740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55eb1ad4e970 con 0x55eb1ad4ba00 2024-04-22T13:31:55.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.054+0000 7f68adbde740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55eb1ac60500 con 0x55eb1ad424d0 2024-04-22T13:31:55.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.054+0000 7f68adbde740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55eb1ad4eaf0 con 0x55eb1ad42e80 2024-04-22T13:31:55.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.054+0000 7f68adbde740 10 client.0 _ll_register_callbacks cb 0x55eb1ac2eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:55.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.054+0000 7f68a94b2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb1ad4ba00 0x55eb1ad48bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.054+0000 7f68a94b2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb1ad4ba00 0x55eb1ad48bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60966/0 (socket says 172.21.15.12:60966) 2024-04-22T13:31:55.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.054+0000 7f68a94b2640 1 -- 172.21.15.12:0/619139412 learned_addr learned my addr 172.21.15.12:0/619139412 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:55.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.054+0000 7f68a3fff640 1 --2- 172.21.15.12:0/619139412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb1ad42e80 0x55eb1ad48650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.054+0000 7f68a8cb1640 1 --2- 172.21.15.12:0/619139412 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55eb1ad424d0 0x55eb1ad4b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.054+0000 7f68a94b2640 1 -- 172.21.15.12:0/619139412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb1ad42e80 msgr2=0x55eb1ad48650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:55.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.054+0000 7f68a94b2640 1 --2- 172.21.15.12:0/619139412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb1ad42e80 0x55eb1ad48650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.054+0000 7f68a94b2640 1 -- 172.21.15.12:0/619139412 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55eb1ad424d0 msgr2=0x55eb1ad4b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:55.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.054+0000 7f68a94b2640 1 --2- 172.21.15.12:0/619139412 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55eb1ad424d0 0x55eb1ad4b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.054+0000 7f68a94b2640 1 -- 172.21.15.12:0/619139412 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb1a9b93f0 con 0x55eb1ad4ba00 2024-04-22T13:31:55.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.054+0000 7f68a8cb1640 1 --2- 172.21.15.12:0/619139412 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55eb1ad424d0 0x55eb1ad4b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:55.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.054+0000 7f68a94b2640 1 --2- 172.21.15.12:0/619139412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb1ad4ba00 0x55eb1ad48bb0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f689c036000 tx=0x7f689c005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c71f20327c82ec1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:55.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.054+0000 7f6883fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:55.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.055+0000 7f6883fff640 1 -- 172.21.15.12:0/619139412 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c048070 con 0x55eb1ad4ba00 2024-04-22T13:31:55.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.055+0000 7f6883fff640 1 -- 172.21.15.12:0/619139412 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f689c005040 con 0x55eb1ad4ba00 2024-04-22T13:31:55.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.055+0000 7f68adbde740 1 -- 172.21.15.12:0/619139412 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68940a0d20 con 0x55eb1ad4ba00 2024-04-22T13:31:55.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.055+0000 7f6883fff640 1 -- 172.21.15.12:0/619139412 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f689c05a050 con 0x55eb1ad4ba00 2024-04-22T13:31:55.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.055+0000 7f68adbde740 10 client.18429 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:55.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.055+0000 7f68adbde740 1 -- 172.21.15.12:0/619139412 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eb1aa837d0 con 0x55eb1ad4ba00 2024-04-22T13:31:55.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.055+0000 7f6883fff640 1 -- 172.21.15.12:0/619139412 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68940a0d20 con 0x55eb1ad4ba00 2024-04-22T13:31:55.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.055+0000 7f6883fff640 1 -- 172.21.15.12:0/619139412 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f689c034900 con 0x55eb1ad4ba00 2024-04-22T13:31:55.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.055+0000 7f68adbde740 10 client.18429 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:55.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.055+0000 7f68adbde740 10 client.18429 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:31:55.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.055+0000 7f68adbde740 20 client.18429 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:55.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.055+0000 7f68adbde740 1 --2- 172.21.15.12:0/619139412 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55eb1ad5bca0 0x55eb1ad5e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.055+0000 7f68adbde740 4 client.18429 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:55.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.055+0000 7f68adbde740 1 -- 172.21.15.12:0/619139412 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb1ac46930 con 0x55eb1ad5bca0 2024-04-22T13:31:55.072 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.056+0000 7f68a8cb1640 1 --2- 172.21.15.12:0/619139412 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55eb1ad5bca0 0x55eb1ad5e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.056+0000 7f6883fff640 10 client.18429 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:31:55.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.056+0000 7f68a8cb1640 1 --2- 172.21.15.12:0/619139412 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55eb1ad5bca0 0x55eb1ad5e120 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8a400659b9c3fbb9 server_cookie=3e375df2511502b7 in_seq=0 out_seq=0 2024-04-22T13:31:55.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.057+0000 7f6883fff640 1 -- 172.21.15.12:0/619139412 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb1ac46930 con 0x55eb1ad5bca0 2024-04-22T13:31:55.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.057+0000 7f6883fff640 10 client.18429 handle_command_reply: tid=0 2024-04-22T13:31:55.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.057+0000 7f68adbde740 1 client.18429 shutdown 2024-04-22T13:31:55.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.057+0000 7f68adbde740 1 -- 172.21.15.12:0/619139412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb1ad4ba00 msgr2=0x55eb1ad48bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:55.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.057+0000 7f68adbde740 1 --2- 172.21.15.12:0/619139412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb1ad4ba00 0x55eb1ad48bb0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f689c036000 tx=0x7f689c005c50 comp rx=0 tx=0).stop 2024-04-22T13:31:55.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.057+0000 7f68adbde740 1 -- 172.21.15.12:0/619139412 shutdown_connections 2024-04-22T13:31:55.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.057+0000 7f68adbde740 1 --2- 172.21.15.12:0/619139412 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55eb1ad5bca0 0x55eb1ad5e120 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.057+0000 7f68adbde740 1 --2- 172.21.15.12:0/619139412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb1ad4ba00 0x55eb1ad48bb0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.057+0000 7f6883fff640 0 client.18429 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:31:55.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.057+0000 7f68adbde740 1 --2- 172.21.15.12:0/619139412 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb1ad42e80 0x55eb1ad48650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.057+0000 7f68adbde740 1 --2- 172.21.15.12:0/619139412 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55eb1ad424d0 0x55eb1ad4b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.057+0000 7f68adbde740 1 -- 172.21.15.12:0/619139412 >> 172.21.15.12:0/619139412 conn(0x55eb1ad3cba0 msgr2=0x55eb1ad3e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:55.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.057+0000 7f68adbde740 1 -- 172.21.15.12:0/619139412 shutdown_connections 2024-04-22T13:31:55.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.057+0000 7f68adbde740 1 -- 172.21.15.12:0/619139412 wait complete. 2024-04-22T13:31:55.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.057+0000 7f68adbde740 20 client.18429 trim_cache size 0 max 16384 2024-04-22T13:31:55.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.070+0000 7f68adbde740 1 Processor -- start 2024-04-22T13:31:55.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.070+0000 7f68adbde740 1 -- start start 2024-04-22T13:31:55.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.071+0000 7f68adbde740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb1ad424d0 0x55eb1ad57860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.071+0000 7f68adbde740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55eb1ad42e80 0x55eb1ad4ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.071+0000 7f68adbde740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb1ad4ba00 0x55eb1ad4b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.071+0000 7f68adbde740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41afd50 con 0x55eb1ad4ba00 2024-04-22T13:31:55.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.071+0000 7f68adbde740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41b1890 con 0x55eb1ad42e80 2024-04-22T13:31:55.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.071+0000 7f68adbde740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41b0420 con 0x55eb1ad424d0 2024-04-22T13:31:55.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.071+0000 7f68a94b2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb1ad4ba00 0x55eb1ad4b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.071+0000 7f68a94b2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb1ad4ba00 0x55eb1ad4b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60976/0 (socket says 172.21.15.12:60976) 2024-04-22T13:31:55.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.071+0000 7f68a94b2640 1 -- 172.21.15.12:0/2811977180 learned_addr learned my addr 172.21.15.12:0/2811977180 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:55.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.071+0000 7f68a94b2640 1 -- 172.21.15.12:0/2811977180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb1ad424d0 msgr2=0x55eb1ad57860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:55.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.071+0000 7f68a94b2640 1 --2- 172.21.15.12:0/2811977180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb1ad424d0 0x55eb1ad57860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.071+0000 7f68a94b2640 1 -- 172.21.15.12:0/2811977180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55eb1ad42e80 msgr2=0x55eb1ad4ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:55.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.071+0000 7f68a94b2640 1 --2- 172.21.15.12:0/2811977180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55eb1ad42e80 0x55eb1ad4ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.071+0000 7f68a94b2640 1 -- 172.21.15.12:0/2811977180 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb1ad48f20 con 0x55eb1ad4ba00 2024-04-22T13:31:55.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.071+0000 7f68a94b2640 1 --2- 172.21.15.12:0/2811977180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb1ad4ba00 0x55eb1ad4b2c0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f689c034ee0 tx=0x7f689c005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf942f6cd51de1ec server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:55.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.071+0000 7f688a7fc640 1 -- 172.21.15.12:0/2811977180 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c039040 con 0x55eb1ad4ba00 2024-04-22T13:31:55.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.071+0000 7f688a7fc640 1 -- 172.21.15.12:0/2811977180 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f689c063070 con 0x55eb1ad4ba00 2024-04-22T13:31:55.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.071+0000 7f688a7fc640 1 -- 172.21.15.12:0/2811977180 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c0346a0 con 0x55eb1ad4ba00 2024-04-22T13:31:55.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.072+0000 7f68adbde740 1 -- 172.21.15.12:0/2811977180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb1ad4ba00 msgr2=0x55eb1ad4b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:55.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.072+0000 7f68adbde740 1 --2- 172.21.15.12:0/2811977180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb1ad4ba00 0x55eb1ad4b2c0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f689c034ee0 tx=0x7f689c005d80 comp rx=0 tx=0).stop 2024-04-22T13:31:55.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.072+0000 7f68adbde740 1 -- 172.21.15.12:0/2811977180 shutdown_connections 2024-04-22T13:31:55.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.072+0000 7f68adbde740 1 --2- 172.21.15.12:0/2811977180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb1ad4ba00 0x55eb1ad4b2c0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.072+0000 7f68adbde740 1 --2- 172.21.15.12:0/2811977180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55eb1ad42e80 0x55eb1ad4ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.072+0000 7f68adbde740 1 --2- 172.21.15.12:0/2811977180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb1ad424d0 0x55eb1ad57860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.072+0000 7f68adbde740 1 -- 172.21.15.12:0/2811977180 >> 172.21.15.12:0/2811977180 conn(0x55eb1ad3cba0 msgr2=0x55eb1ad3b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:55.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.072+0000 7f68adbde740 1 -- 172.21.15.12:0/2811977180 shutdown_connections 2024-04-22T13:31:55.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.072+0000 7f68adbde740 1 -- 172.21.15.12:0/2811977180 wait complete. 2024-04-22T13:31:55.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.072+0000 7f68adbde740 1 -- start start 2024-04-22T13:31:55.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.072+0000 7f68adbde740 1 Processor -- start 2024-04-22T13:31:55.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.073+0000 7f68adbde740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55eb1ad424d0 0x55eb1ad541a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.073+0000 7f68adbde740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb1ad42e80 0x55eb1ad4f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.073+0000 7f68adbde740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb1ad4ba00 0x55eb1ad4fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.073+0000 7f68adbde740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41b11c0 con 0x55eb1ad42e80 2024-04-22T13:31:55.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.073+0000 7f68adbde740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41b0af0 con 0x55eb1ad424d0 2024-04-22T13:31:55.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.073+0000 7f68adbde740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41add50 con 0x55eb1ad4ba00 2024-04-22T13:31:55.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.073+0000 7f68adbde740 10 client.0 _ll_register_callbacks cb 0x7f689c004f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:55.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.073+0000 7f68a3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb1ad42e80 0x55eb1ad4f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.073+0000 7f68a3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb1ad42e80 0x55eb1ad4f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60988/0 (socket says 172.21.15.12:60988) 2024-04-22T13:31:55.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.073+0000 7f68a3fff640 1 -- 172.21.15.12:0/3886394984 learned_addr learned my addr 172.21.15.12:0/3886394984 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:55.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.073+0000 7f68a94b2640 1 --2- 172.21.15.12:0/3886394984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb1ad4ba00 0x55eb1ad4fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.073+0000 7f68a8cb1640 1 --2- 172.21.15.12:0/3886394984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55eb1ad424d0 0x55eb1ad541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.073+0000 7f68a3fff640 1 -- 172.21.15.12:0/3886394984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb1ad4ba00 msgr2=0x55eb1ad4fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:55.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.073+0000 7f68a3fff640 1 --2- 172.21.15.12:0/3886394984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb1ad4ba00 0x55eb1ad4fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.073+0000 7f68a3fff640 1 -- 172.21.15.12:0/3886394984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55eb1ad424d0 msgr2=0x55eb1ad541a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:55.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.073+0000 7f68a3fff640 1 --2- 172.21.15.12:0/3886394984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55eb1ad424d0 0x55eb1ad541a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.073+0000 7f68a3fff640 1 -- 172.21.15.12:0/3886394984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb1ad502a0 con 0x55eb1ad42e80 2024-04-22T13:31:55.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.073+0000 7f68a3fff640 1 --2- 172.21.15.12:0/3886394984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb1ad42e80 0x55eb1ad4f510 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f6898013490 tx=0x7f6898013950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec1a3b1bac03850 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:55.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.073+0000 7f6883fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:55.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.073+0000 7f6883fff640 1 -- 172.21.15.12:0/3886394984 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68980119d0 con 0x55eb1ad42e80 2024-04-22T13:31:55.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.073+0000 7f6883fff640 1 -- 172.21.15.12:0/3886394984 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f6898011b70 con 0x55eb1ad42e80 2024-04-22T13:31:55.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.073+0000 7f68adbde740 1 -- 172.21.15.12:0/3886394984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f689c063070 con 0x55eb1ad42e80 2024-04-22T13:31:55.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.074+0000 7f6883fff640 1 -- 172.21.15.12:0/3886394984 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6898011d70 con 0x55eb1ad42e80 2024-04-22T13:31:55.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.074+0000 7f68adbde740 10 client.18435 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:55.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.074+0000 7f68adbde740 1 -- 172.21.15.12:0/3886394984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eb1adc2350 con 0x55eb1ad42e80 2024-04-22T13:31:55.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.074+0000 7f6883fff640 1 -- 172.21.15.12:0/3886394984 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f689c063070 con 0x55eb1ad42e80 2024-04-22T13:31:55.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.074+0000 7f6883fff640 1 -- 172.21.15.12:0/3886394984 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6898024020 con 0x55eb1ad42e80 2024-04-22T13:31:55.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.074+0000 7f68adbde740 10 client.18435 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:55.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.074+0000 7f68adbde740 10 client.18435 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:31:55.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.074+0000 7f68adbde740 20 client.18435 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:55.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.074+0000 7f68adbde740 1 --2- 172.21.15.12:0/3886394984 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55eb1adc2b20 0x55eb1adc4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.074+0000 7f68adbde740 4 client.18435 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:31:55.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.074+0000 7f68adbde740 1 -- 172.21.15.12:0/3886394984 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eb1acb4c10 con 0x55eb1adc2b20 2024-04-22T13:31:55.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.074+0000 7f68a8cb1640 1 --2- 172.21.15.12:0/3886394984 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55eb1adc2b20 0x55eb1adc4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.075+0000 7f6883fff640 10 client.18435 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:31:55.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.075+0000 7f68a8cb1640 1 --2- 172.21.15.12:0/3886394984 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55eb1adc2b20 0x55eb1adc4f40 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=897d47d0d9f8cc57 server_cookie=d8218b03a3d8a958 in_seq=0 out_seq=0 2024-04-22T13:31:55.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.075+0000 7f6883fff640 1 -- 172.21.15.12:0/3886394984 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eb1acb4c10 con 0x55eb1adc2b20 2024-04-22T13:31:55.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.075+0000 7f6883fff640 10 client.18435 handle_command_reply: tid=0 2024-04-22T13:31:55.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.075+0000 7f68adbde740 1 client.18435 shutdown 2024-04-22T13:31:55.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.075+0000 7f68adbde740 1 -- 172.21.15.12:0/3886394984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb1ad42e80 msgr2=0x55eb1ad4f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:55.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.075+0000 7f68adbde740 1 --2- 172.21.15.12:0/3886394984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb1ad42e80 0x55eb1ad4f510 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f6898013490 tx=0x7f6898013950 comp rx=0 tx=0).stop 2024-04-22T13:31:55.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.075+0000 7f68adbde740 1 -- 172.21.15.12:0/3886394984 shutdown_connections 2024-04-22T13:31:55.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.075+0000 7f68adbde740 1 --2- 172.21.15.12:0/3886394984 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55eb1adc2b20 0x55eb1adc4f40 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.075+0000 7f68adbde740 1 --2- 172.21.15.12:0/3886394984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55eb1ad4ba00 0x55eb1ad4fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.075+0000 7f68adbde740 1 --2- 172.21.15.12:0/3886394984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55eb1ad42e80 0x55eb1ad4f510 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.075+0000 7f68adbde740 1 --2- 172.21.15.12:0/3886394984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55eb1ad424d0 0x55eb1ad541a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.075+0000 7f6883fff640 0 client.18435 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:31:55.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.075+0000 7f68adbde740 1 -- 172.21.15.12:0/3886394984 >> 172.21.15.12:0/3886394984 conn(0x55eb1ad3cba0 msgr2=0x55eb1ad43af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:55.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.075+0000 7f68adbde740 1 -- 172.21.15.12:0/3886394984 shutdown_connections 2024-04-22T13:31:55.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.075+0000 7f68adbde740 1 -- 172.21.15.12:0/3886394984 wait complete. 2024-04-22T13:31:55.090 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.075+0000 7f68adbde740 20 client.18435 trim_cache size 0 max 16384 2024-04-22T13:31:55.090 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:31:55.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.076+0000 7f68aaf15640 1 -- 172.21.15.12:0/232274062 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f688c0a3d20 msgr2=0x7f688c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:55.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.076+0000 7f68aaf15640 1 --2- 172.21.15.12:0/232274062 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f688c0a3d20 0x7f688c0a61e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f689c02c630 tx=0x7f689c009800 comp rx=0 tx=0).stop 2024-04-22T13:31:55.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.076+0000 7f68aaf15640 1 -- 172.21.15.12:0/232274062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68a4100d50 msgr2=0x7f68a41a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:55.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.076+0000 7f68aaf15640 1 --2- 172.21.15.12:0/232274062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68a4100d50 0x7f68a41a05e0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f689400c920 tx=0x7f689400cde0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.076+0000 7f68aaf15640 1 -- 172.21.15.12:0/232274062 shutdown_connections 2024-04-22T13:31:55.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.076+0000 7f68aaf15640 1 --2- 172.21.15.12:0/232274062 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f688c0a3d20 0x7f688c0a61e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.092 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.076+0000 7f68aaf15640 1 --2- 172.21.15.12:0/232274062 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f68a410e900 0x7f68a419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.076+0000 7f68aaf15640 1 --2- 172.21.15.12:0/232274062 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68a4101670 0x7f68a41a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.076+0000 7f68aaf15640 1 --2- 172.21.15.12:0/232274062 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f68a4100d50 0x7f68a41a05e0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.076+0000 7f68aaf15640 1 -- 172.21.15.12:0/232274062 >> 172.21.15.12:0/232274062 conn(0x7f68a4077f50 msgr2=0x7f68a410fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:55.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.076+0000 7f68aaf15640 1 -- 172.21.15.12:0/232274062 shutdown_connections 2024-04-22T13:31:55.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.076+0000 7f68aaf15640 1 -- 172.21.15.12:0/232274062 wait complete. 2024-04-22T13:31:55.094 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:31:55.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.203+0000 7f5ae9281640 1 -- 172.21.15.12:0/1006386141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ae4101a40 msgr2=0x7f5ae4105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:55.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.203+0000 7f5ae9281640 1 --2- 172.21.15.12:0/1006386141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ae4101a40 0x7f5ae4105a80 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f5acc009900 tx=0x7f5acc02bc30 comp rx=0 tx=0).stop 2024-04-22T13:31:55.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.203+0000 7f5ae9281640 1 -- 172.21.15.12:0/1006386141 shutdown_connections 2024-04-22T13:31:55.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.203+0000 7f5ae9281640 1 --2- 172.21.15.12:0/1006386141 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ae4106160 0x7f5ae410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.203+0000 7f5ae9281640 1 --2- 172.21.15.12:0/1006386141 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ae4101a40 0x7f5ae4105a80 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.203+0000 7f5ae9281640 1 --2- 172.21.15.12:0/1006386141 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ae4101090 0x7f5ae4101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.203+0000 7f5ae9281640 1 -- 172.21.15.12:0/1006386141 >> 172.21.15.12:0/1006386141 conn(0x7f5ae4077f50 msgr2=0x7f5ae4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:55.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.203+0000 7f5ae9281640 1 -- 172.21.15.12:0/1006386141 shutdown_connections 2024-04-22T13:31:55.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.203+0000 7f5ae9281640 1 -- 172.21.15.12:0/1006386141 wait complete. 2024-04-22T13:31:55.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.203+0000 7f5ae9281640 1 Processor -- start 2024-04-22T13:31:55.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.203+0000 7f5ae9281640 1 -- start start 2024-04-22T13:31:55.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.204+0000 7f5ae9281640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ae4101090 0x7f5ae419e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.204+0000 7f5ae9281640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ae4106160 0x7f5ae419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.204+0000 7f5ae9281640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ae41985d0 0x7f5ae4198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.204+0000 7f5ae9281640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae41110a0 con 0x7f5ae4101090 2024-04-22T13:31:55.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.204+0000 7f5ae9281640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae4110f20 con 0x7f5ae4106160 2024-04-22T13:31:55.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.204+0000 7f5ae9281640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae4111220 con 0x7f5ae41985d0 2024-04-22T13:31:55.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.204+0000 7f5ae2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ae4101090 0x7f5ae419e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.204+0000 7f5ae2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ae4101090 0x7f5ae419e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32772/0 (socket says 172.21.15.12:32772) 2024-04-22T13:31:55.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.204+0000 7f5ae2ffd640 1 -- 172.21.15.12:0/1170152090 learned_addr learned my addr 172.21.15.12:0/1170152090 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:55.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.204+0000 7f5ae37fe640 1 --2- 172.21.15.12:0/1170152090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ae41985d0 0x7f5ae4198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.204+0000 7f5ae27fc640 1 --2- 172.21.15.12:0/1170152090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ae4106160 0x7f5ae419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.204+0000 7f5ae2ffd640 1 -- 172.21.15.12:0/1170152090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ae41985d0 msgr2=0x7f5ae4198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:55.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.204+0000 7f5ae2ffd640 1 --2- 172.21.15.12:0/1170152090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ae41985d0 0x7f5ae4198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.204+0000 7f5ae2ffd640 1 -- 172.21.15.12:0/1170152090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ae4106160 msgr2=0x7f5ae419e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:55.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.204+0000 7f5ae2ffd640 1 --2- 172.21.15.12:0/1170152090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ae4106160 0x7f5ae419e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.204+0000 7f5ae2ffd640 1 -- 172.21.15.12:0/1170152090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae41990f0 con 0x7f5ae4101090 2024-04-22T13:31:55.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.204+0000 7f5ae27fc640 1 --2- 172.21.15.12:0/1170152090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ae4106160 0x7f5ae419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:55.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.204+0000 7f5ae37fe640 1 --2- 172.21.15.12:0/1170152090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ae41985d0 0x7f5ae4198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:55.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.204+0000 7f5ae2ffd640 1 --2- 172.21.15.12:0/1170152090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ae4101090 0x7f5ae419e420 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400c970 tx=0x7f5ad400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:55.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.204+0000 7f5adbfff640 1 -- 172.21.15.12:0/1170152090 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4007bb0 con 0x7f5ae4101090 2024-04-22T13:31:55.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.204+0000 7f5adbfff640 1 -- 172.21.15.12:0/1170152090 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ad4007d50 con 0x7f5ae4101090 2024-04-22T13:31:55.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.204+0000 7f5adbfff640 1 -- 172.21.15.12:0/1170152090 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad400f450 con 0x7f5ae4101090 2024-04-22T13:31:55.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.204+0000 7f5ae9281640 1 -- 172.21.15.12:0/1170152090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ae41993e0 con 0x7f5ae4101090 2024-04-22T13:31:55.204 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.205+0000 7f5ae9281640 1 -- 172.21.15.12:0/1170152090 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ae41a55e0 con 0x7f5ae4101090 2024-04-22T13:31:55.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.207+0000 7f5adbfff640 1 -- 172.21.15.12:0/1170152090 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5ad4004f20 con 0x7f5ae4101090 2024-04-22T13:31:55.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.207+0000 7f5aebe75740 1 Processor -- start 2024-04-22T13:31:55.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.207+0000 7f5aebe75740 1 -- start start 2024-04-22T13:31:55.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.207+0000 7f5aebe75740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb403f93b0 0x55cb403f9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.208+0000 7f5aebe75740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb403f9d60 0x55cb40402890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.208+0000 7f5aebe75740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb40402e60 0x55cb404032c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.208+0000 7f5aebe75740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55cb4035e1f0 con 0x55cb403f93b0 2024-04-22T13:31:55.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.208+0000 7f5aebe75740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55cb402fb4b0 con 0x55cb40402e60 2024-04-22T13:31:55.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.208+0000 7f5aebe75740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cb4035a1b0 con 0x55cb403f9d60 2024-04-22T13:31:55.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.208+0000 7f5ae37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb403f93b0 0x55cb403f9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.208+0000 7f5ae37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb403f93b0 0x55cb403f9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32788/0 (socket says 172.21.15.12:32788) 2024-04-22T13:31:55.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.208+0000 7f5ae37fe640 1 -- 172.21.15.12:0/3365498826 learned_addr learned my addr 172.21.15.12:0/3365498826 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:55.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.208+0000 7f5ae2ffd640 1 --2- 172.21.15.12:0/3365498826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb403f9d60 0x55cb40402890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.208+0000 7f5adbfff640 1 --2- 172.21.15.12:0/1170152090 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5ad00a3d20 0x7f5ad00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.208+0000 7f5ae27fc640 1 --2- 172.21.15.12:0/3365498826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb40402e60 0x55cb404032c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.208+0000 7f5ae37fe640 1 -- 172.21.15.12:0/3365498826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb403f9d60 msgr2=0x55cb40402890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:55.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.208+0000 7f5ae37fe640 1 --2- 172.21.15.12:0/3365498826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb403f9d60 0x55cb40402890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.208+0000 7f5ae37fe640 1 -- 172.21.15.12:0/3365498826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb40402e60 msgr2=0x55cb404032c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:55.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.208+0000 7f5ae37fe640 1 --2- 172.21.15.12:0/3365498826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb40402e60 0x55cb404032c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.208+0000 7f5ae37fe640 1 -- 172.21.15.12:0/3365498826 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb402ed1b0 con 0x55cb403f93b0 2024-04-22T13:31:55.217 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.208+0000 7f5adbfff640 1 -- 172.21.15.12:0/1170152090 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f5ad40d0610 con 0x7f5ae4101090 2024-04-22T13:31:55.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.208+0000 7f5ae37fe640 1 --2- 172.21.15.12:0/1170152090 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5ad00a3d20 0x7f5ad00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.208+0000 7f5ae37fe640 1 --2- 172.21.15.12:0/3365498826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb403f93b0 0x55cb403f9790 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f5adc00a2b0 tx=0x7f5adc00e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bcf77dbf45000d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:55.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.208+0000 7f5ad97fa640 1 -- 172.21.15.12:0/3365498826 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5adc022070 con 0x55cb403f93b0 2024-04-22T13:31:55.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.209+0000 7f5ad97fa640 1 -- 172.21.15.12:0/3365498826 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5adc018070 con 0x55cb403f93b0 2024-04-22T13:31:55.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.209+0000 7f5ad97fa640 1 -- 172.21.15.12:0/3365498826 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5adc01cce0 con 0x55cb403f93b0 2024-04-22T13:31:55.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.209+0000 7f5aebe75740 1 -- 172.21.15.12:0/3365498826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb403f93b0 msgr2=0x55cb403f9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:55.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.209+0000 7f5ae37fe640 1 --2- 172.21.15.12:0/1170152090 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5ad00a3d20 0x7f5ad00a61e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5adc01f000 tx=0x7f5adc020000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:55.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.209+0000 7f5aebe75740 1 --2- 172.21.15.12:0/3365498826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb403f93b0 0x55cb403f9790 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f5adc00a2b0 tx=0x7f5adc00e7e0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.209+0000 7f5aebe75740 1 -- 172.21.15.12:0/3365498826 shutdown_connections 2024-04-22T13:31:55.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.209+0000 7f5aebe75740 1 --2- 172.21.15.12:0/3365498826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb40402e60 0x55cb404032c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.209+0000 7f5aebe75740 1 --2- 172.21.15.12:0/3365498826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb403f9d60 0x55cb40402890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.209+0000 7f5aebe75740 1 --2- 172.21.15.12:0/3365498826 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb403f93b0 0x55cb403f9790 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.209+0000 7f5aebe75740 1 -- 172.21.15.12:0/3365498826 >> 172.21.15.12:0/3365498826 conn(0x55cb403f3ba0 msgr2=0x55cb403f3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:55.218 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.209+0000 7f5aebe75740 1 -- 172.21.15.12:0/3365498826 shutdown_connections 2024-04-22T13:31:55.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.209+0000 7f5aebe75740 1 -- 172.21.15.12:0/3365498826 wait complete. 2024-04-22T13:31:55.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.210+0000 7f5aebe75740 1 -- start start 2024-04-22T13:31:55.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.210+0000 7f5aebe75740 1 Processor -- start 2024-04-22T13:31:55.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.210+0000 7f5aebe75740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb403f93b0 0x55cb40409270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.210+0000 7f5aebe75740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb403f9d60 0x55cb40406600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.210+0000 7f5aebe75740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb40402e60 0x55cb40406b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.210+0000 7f5aebe75740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55cb40409da0 con 0x55cb403f9d60 2024-04-22T13:31:55.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.210+0000 7f5aebe75740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55cb40317500 con 0x55cb403f93b0 2024-04-22T13:31:55.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.210+0000 7f5aebe75740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cb40409f20 con 0x55cb40402e60 2024-04-22T13:31:55.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.210+0000 7f5aebe75740 10 client.0 _ll_register_callbacks cb 0x55cb402e5b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:55.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.210+0000 7f5ae27fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb403f9d60 0x55cb40406600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.210+0000 7f5ae27fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb403f9d60 0x55cb40406600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32796/0 (socket says 172.21.15.12:32796) 2024-04-22T13:31:55.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.210+0000 7f5ae27fc640 1 -- 172.21.15.12:0/3962784586 learned_addr learned my addr 172.21.15.12:0/3962784586 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:55.219 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.210+0000 7f5ae2ffd640 1 --2- 172.21.15.12:0/3962784586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb403f93b0 0x55cb40409270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.210+0000 7f5ae37fe640 1 --2- 172.21.15.12:0/3962784586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb40402e60 0x55cb40406b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.210+0000 7f5ae27fc640 1 -- 172.21.15.12:0/3962784586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb40402e60 msgr2=0x55cb40406b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:55.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.210+0000 7f5ae27fc640 1 --2- 172.21.15.12:0/3962784586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb40402e60 0x55cb40406b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.210+0000 7f5ae27fc640 1 -- 172.21.15.12:0/3962784586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb403f93b0 msgr2=0x55cb40409270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:55.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.210+0000 7f5ae27fc640 1 --2- 172.21.15.12:0/3962784586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb403f93b0 0x55cb40409270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.210+0000 7f5ae27fc640 1 -- 172.21.15.12:0/3962784586 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb400703f0 con 0x55cb403f9d60 2024-04-22T13:31:55.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.211+0000 7f5ae27fc640 1 --2- 172.21.15.12:0/3962784586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb403f9d60 0x55cb40406600 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f5acc005ab0 tx=0x7f5acc02c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6a76d69b97d5fb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:55.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.211+0000 7f5aca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:55.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.211+0000 7f5aca7fc640 1 -- 172.21.15.12:0/3962784586 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc005230 con 0x55cb403f9d60 2024-04-22T13:31:55.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.211+0000 7f5aca7fc640 1 -- 172.21.15.12:0/3962784586 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f5acc0053d0 con 0x55cb403f9d60 2024-04-22T13:31:55.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.211+0000 7f5aebe75740 1 -- 172.21.15.12:0/3962784586 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5adc018070 con 0x55cb403f9d60 2024-04-22T13:31:55.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.211+0000 7f5aca7fc640 1 -- 172.21.15.12:0/3962784586 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5acc04f050 con 0x55cb403f9d60 2024-04-22T13:31:55.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.211+0000 7f5aebe75740 10 client.18447 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:55.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.211+0000 7f5aebe75740 1 -- 172.21.15.12:0/3962784586 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cb4013a7d0 con 0x55cb403f9d60 2024-04-22T13:31:55.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.211+0000 7f5aca7fc640 1 -- 172.21.15.12:0/3962784586 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5adc018070 con 0x55cb403f9d60 2024-04-22T13:31:55.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.211+0000 7f5aca7fc640 1 -- 172.21.15.12:0/3962784586 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5acc003710 con 0x55cb403f9d60 2024-04-22T13:31:55.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.211+0000 7f5aebe75740 10 client.18447 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:55.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.211+0000 7f5aebe75740 10 client.18447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:55.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.211+0000 7f5aebe75740 20 client.18447 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:55.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.211+0000 7f5aebe75740 1 --2- 172.21.15.12:0/3962784586 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55cb40417070 0x55cb40419510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.211+0000 7f5aebe75740 4 client.18447 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:55.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.211+0000 7f5aebe75740 1 -- 172.21.15.12:0/3962784586 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb402fd930 con 0x55cb40417070 2024-04-22T13:31:55.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.212+0000 7f5ae2ffd640 1 --2- 172.21.15.12:0/3962784586 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55cb40417070 0x55cb40419510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.212+0000 7f5ae2ffd640 1 --2- 172.21.15.12:0/3962784586 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55cb40417070 0x55cb40419510 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c30c007505b75961 server_cookie=ba4d9a0b904577d in_seq=0 out_seq=0 2024-04-22T13:31:55.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.212+0000 7f5aca7fc640 10 client.18447 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:55.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.213+0000 7f5aca7fc640 1 -- 172.21.15.12:0/3962784586 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cb402fd930 con 0x55cb40417070 2024-04-22T13:31:55.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.213+0000 7f5aca7fc640 10 client.18447 handle_command_reply: tid=0 2024-04-22T13:31:55.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.213+0000 7f5aebe75740 1 client.18447 shutdown 2024-04-22T13:31:55.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.213+0000 7f5aebe75740 1 -- 172.21.15.12:0/3962784586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb403f9d60 msgr2=0x55cb40406600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:55.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.213+0000 7f5aebe75740 1 --2- 172.21.15.12:0/3962784586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb403f9d60 0x55cb40406600 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f5acc005ab0 tx=0x7f5acc02c9f0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.213+0000 7f5aebe75740 1 -- 172.21.15.12:0/3962784586 shutdown_connections 2024-04-22T13:31:55.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.213+0000 7f5aebe75740 1 --2- 172.21.15.12:0/3962784586 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55cb40417070 0x55cb40419510 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.213+0000 7f5aebe75740 1 --2- 172.21.15.12:0/3962784586 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb40402e60 0x55cb40406b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.213+0000 7f5aebe75740 1 --2- 172.21.15.12:0/3962784586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb403f9d60 0x55cb40406600 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.213+0000 7f5aca7fc640 0 client.18447 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:55.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.213+0000 7f5aebe75740 1 --2- 172.21.15.12:0/3962784586 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb403f93b0 0x55cb40409270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.213+0000 7f5aebe75740 1 -- 172.21.15.12:0/3962784586 >> 172.21.15.12:0/3962784586 conn(0x55cb403f3ba0 msgr2=0x55cb403f58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:55.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.213+0000 7f5aebe75740 1 -- 172.21.15.12:0/3962784586 shutdown_connections 2024-04-22T13:31:55.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.213+0000 7f5aebe75740 1 -- 172.21.15.12:0/3962784586 wait complete. 2024-04-22T13:31:55.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.213+0000 7f5aebe75740 20 client.18447 trim_cache size 0 max 16384 2024-04-22T13:31:55.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.226+0000 7f5aebe75740 1 Processor -- start 2024-04-22T13:31:55.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.226+0000 7f5aebe75740 1 -- start start 2024-04-22T13:31:55.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.227+0000 7f5aebe75740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb403f93b0 0x55cb40412ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.227+0000 7f5aebe75740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb403f9d60 0x55cb40404250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.227+0000 7f5aebe75740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb40402e60 0x55cb40408d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.227+0000 7f5aebe75740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae41ab9c0 con 0x55cb403f93b0 2024-04-22T13:31:55.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.227+0000 7f5aebe75740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae41ad500 con 0x55cb403f9d60 2024-04-22T13:31:55.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.227+0000 7f5aebe75740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae41ac090 con 0x55cb40402e60 2024-04-22T13:31:55.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.227+0000 7f5ae2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb403f93b0 0x55cb40412ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.227+0000 7f5ae2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb403f93b0 0x55cb40412ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32804/0 (socket says 172.21.15.12:32804) 2024-04-22T13:31:55.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.227+0000 7f5ae2ffd640 1 -- 172.21.15.12:0/1202508224 learned_addr learned my addr 172.21.15.12:0/1202508224 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:55.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.227+0000 7f5ae37fe640 1 --2- 172.21.15.12:0/1202508224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb40402e60 0x55cb40408d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.227+0000 7f5ae27fc640 1 --2- 172.21.15.12:0/1202508224 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb403f9d60 0x55cb40404250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.227+0000 7f5ae2ffd640 1 -- 172.21.15.12:0/1202508224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb40402e60 msgr2=0x55cb40408d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.227+0000 7f5ae2ffd640 1 --2- 172.21.15.12:0/1202508224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb40402e60 0x55cb40408d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.227+0000 7f5ae2ffd640 1 -- 172.21.15.12:0/1202508224 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb403f9d60 msgr2=0x55cb40404250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.227+0000 7f5ae2ffd640 1 --2- 172.21.15.12:0/1202508224 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb403f9d60 0x55cb40404250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.227+0000 7f5ae2ffd640 1 -- 172.21.15.12:0/1202508224 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb403ef470 con 0x55cb403f93b0 2024-04-22T13:31:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.227+0000 7f5ae37fe640 1 --2- 172.21.15.12:0/1202508224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb40402e60 0x55cb40408d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.227+0000 7f5ae27fc640 1 --2- 172.21.15.12:0/1202508224 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb403f9d60 0x55cb40404250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.227+0000 7f5ae2ffd640 1 --2- 172.21.15.12:0/1202508224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb403f93b0 0x55cb40412ca0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4098fa0 tx=0x7f5ad4087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f77d60ceb2ef5e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.228+0000 7f5ad8ff9640 1 -- 172.21.15.12:0/1202508224 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad401d040 con 0x55cb403f93b0 2024-04-22T13:31:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.228+0000 7f5ad8ff9640 1 -- 172.21.15.12:0/1202508224 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ad409c5d0 con 0x55cb403f93b0 2024-04-22T13:31:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.228+0000 7f5ad8ff9640 1 -- 172.21.15.12:0/1202508224 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4098c40 con 0x55cb403f93b0 2024-04-22T13:31:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.228+0000 7f5aebe75740 1 -- 172.21.15.12:0/1202508224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb403f93b0 msgr2=0x55cb40412ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.228+0000 7f5aebe75740 1 --2- 172.21.15.12:0/1202508224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb403f93b0 0x55cb40412ca0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4098fa0 tx=0x7f5ad4087040 comp rx=0 tx=0).stop 2024-04-22T13:31:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.228+0000 7f5aebe75740 1 -- 172.21.15.12:0/1202508224 shutdown_connections 2024-04-22T13:31:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.228+0000 7f5aebe75740 1 --2- 172.21.15.12:0/1202508224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb40402e60 0x55cb40408d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.228+0000 7f5aebe75740 1 --2- 172.21.15.12:0/1202508224 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb403f9d60 0x55cb40404250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.228+0000 7f5aebe75740 1 --2- 172.21.15.12:0/1202508224 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb403f93b0 0x55cb40412ca0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.228+0000 7f5aebe75740 1 -- 172.21.15.12:0/1202508224 >> 172.21.15.12:0/1202508224 conn(0x55cb403f3ba0 msgr2=0x55cb403f57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:55.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.228+0000 7f5aebe75740 1 -- 172.21.15.12:0/1202508224 shutdown_connections 2024-04-22T13:31:55.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.228+0000 7f5aebe75740 1 -- 172.21.15.12:0/1202508224 wait complete. 2024-04-22T13:31:55.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.229+0000 7f5aebe75740 1 -- start start 2024-04-22T13:31:55.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.229+0000 7f5aebe75740 1 Processor -- start 2024-04-22T13:31:55.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.229+0000 7f5aebe75740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb403f93b0 0x55cb40403c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.229+0000 7f5aebe75740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb403f9d60 0x55cb40404180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.229+0000 7f5aebe75740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb40402e60 0x55cb4047c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.229+0000 7f5aebe75740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae41ace30 con 0x55cb40402e60 2024-04-22T13:31:55.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.229+0000 7f5aebe75740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae41ac760 con 0x55cb403f93b0 2024-04-22T13:31:55.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.229+0000 7f5aebe75740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae41a99c0 con 0x55cb403f9d60 2024-04-22T13:31:55.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.229+0000 7f5aebe75740 10 client.0 _ll_register_callbacks cb 0x7f5acc004590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.229+0000 7f5ae37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb40402e60 0x55cb4047c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.229+0000 7f5ae37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb40402e60 0x55cb4047c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32816/0 (socket says 172.21.15.12:32816) 2024-04-22T13:31:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.229+0000 7f5ae37fe640 1 -- 172.21.15.12:0/1722405977 learned_addr learned my addr 172.21.15.12:0/1722405977 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.229+0000 7f5ae2ffd640 1 --2- 172.21.15.12:0/1722405977 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb403f93b0 0x55cb40403c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.229+0000 7f5ae27fc640 1 --2- 172.21.15.12:0/1722405977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb403f9d60 0x55cb40404180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.229+0000 7f5ae37fe640 1 -- 172.21.15.12:0/1722405977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb403f9d60 msgr2=0x55cb40404180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.229+0000 7f5ae37fe640 1 --2- 172.21.15.12:0/1722405977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb403f9d60 0x55cb40404180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.229+0000 7f5ae37fe640 1 -- 172.21.15.12:0/1722405977 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb403f93b0 msgr2=0x55cb40403c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.229+0000 7f5ae37fe640 1 --2- 172.21.15.12:0/1722405977 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb403f93b0 0x55cb40403c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.229+0000 7f5ae37fe640 1 -- 172.21.15.12:0/1722405977 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb4047cb90 con 0x55cb40402e60 2024-04-22T13:31:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.229+0000 7f5ae27fc640 1 --2- 172.21.15.12:0/1722405977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb403f9d60 0x55cb40404180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.229+0000 7f5ae37fe640 1 --2- 172.21.15.12:0/1722405977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb40402e60 0x55cb4047c650 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f5adc00afd0 tx=0x7f5adc00a800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5758f3b3c49f316 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.229+0000 7f5aca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.230+0000 7f5aca7fc640 1 -- 172.21.15.12:0/1722405977 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5adc022040 con 0x55cb40402e60 2024-04-22T13:31:55.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.230+0000 7f5aca7fc640 1 -- 172.21.15.12:0/1722405977 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f5adc011a30 con 0x55cb40402e60 2024-04-22T13:31:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.230+0000 7f5aebe75740 1 -- 172.21.15.12:0/1722405977 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ad409c5d0 con 0x55cb40402e60 2024-04-22T13:31:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.230+0000 7f5aca7fc640 1 -- 172.21.15.12:0/1722405977 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5adc00ab00 con 0x55cb40402e60 2024-04-22T13:31:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.230+0000 7f5aebe75740 10 client.18453 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.230+0000 7f5aebe75740 1 -- 172.21.15.12:0/1722405977 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cb4047dc00 con 0x55cb40402e60 2024-04-22T13:31:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.230+0000 7f5aca7fc640 1 -- 172.21.15.12:0/1722405977 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ad409c5d0 con 0x55cb40402e60 2024-04-22T13:31:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.230+0000 7f5aca7fc640 1 -- 172.21.15.12:0/1722405977 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5adc038020 con 0x55cb40402e60 2024-04-22T13:31:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.230+0000 7f5aebe75740 10 client.18453 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.230+0000 7f5aebe75740 10 client.18453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.230+0000 7f5aebe75740 20 client.18453 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.230+0000 7f5aebe75740 1 --2- 172.21.15.12:0/1722405977 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55cb4047dfe0 0x55cb40480400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.230+0000 7f5aebe75740 4 client.18453 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:31:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.230+0000 7f5aebe75740 1 -- 172.21.15.12:0/1722405977 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cb4036bc10 con 0x55cb4047dfe0 2024-04-22T13:31:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.231+0000 7f5ae2ffd640 1 --2- 172.21.15.12:0/1722405977 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55cb4047dfe0 0x55cb40480400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:55.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.231+0000 7f5ae2ffd640 1 --2- 172.21.15.12:0/1722405977 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55cb4047dfe0 0x55cb40480400 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57370ae4607d739c server_cookie=5bd7fe171376de1a in_seq=0 out_seq=0 2024-04-22T13:31:55.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.231+0000 7f5aca7fc640 10 client.18453 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:55.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.231+0000 7f5aca7fc640 1 -- 172.21.15.12:0/1722405977 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cb4036bc10 con 0x55cb4047dfe0 2024-04-22T13:31:55.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.231+0000 7f5aca7fc640 10 client.18453 handle_command_reply: tid=0 2024-04-22T13:31:55.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.231+0000 7f5aebe75740 1 client.18453 shutdown 2024-04-22T13:31:55.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.231+0000 7f5aebe75740 1 -- 172.21.15.12:0/1722405977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb40402e60 msgr2=0x55cb4047c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:55.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.231+0000 7f5aebe75740 1 --2- 172.21.15.12:0/1722405977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb40402e60 0x55cb4047c650 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f5adc00afd0 tx=0x7f5adc00a800 comp rx=0 tx=0).stop 2024-04-22T13:31:55.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.231+0000 7f5aebe75740 1 -- 172.21.15.12:0/1722405977 shutdown_connections 2024-04-22T13:31:55.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.231+0000 7f5aebe75740 1 --2- 172.21.15.12:0/1722405977 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55cb4047dfe0 0x55cb40480400 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.231+0000 7f5aebe75740 1 --2- 172.21.15.12:0/1722405977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb40402e60 0x55cb4047c650 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.231+0000 7f5aebe75740 1 --2- 172.21.15.12:0/1722405977 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cb403f9d60 0x55cb40404180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.231+0000 7f5aca7fc640 0 client.18453 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:55.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.231+0000 7f5aebe75740 1 --2- 172.21.15.12:0/1722405977 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cb403f93b0 0x55cb40403c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.231+0000 7f5aebe75740 1 -- 172.21.15.12:0/1722405977 >> 172.21.15.12:0/1722405977 conn(0x55cb403f3ba0 msgr2=0x55cb403f2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:55.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.232+0000 7f5aebe75740 1 -- 172.21.15.12:0/1722405977 shutdown_connections 2024-04-22T13:31:55.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.232+0000 7f5aebe75740 1 -- 172.21.15.12:0/1722405977 wait complete. 2024-04-22T13:31:55.238 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.232+0000 7f5aebe75740 20 client.18453 trim_cache size 0 max 16384 2024-04-22T13:31:55.238 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:31:55.239 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.232+0000 7f5ae9281640 1 -- 172.21.15.12:0/1170152090 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5ad00a3d20 msgr2=0x7f5ad00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:55.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.232+0000 7f5ae9281640 1 --2- 172.21.15.12:0/1170152090 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5ad00a3d20 0x7f5ad00a61e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5adc01f000 tx=0x7f5adc020000 comp rx=0 tx=0).stop 2024-04-22T13:31:55.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.232+0000 7f5ae9281640 1 -- 172.21.15.12:0/1170152090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ae4101090 msgr2=0x7f5ae419e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:55.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.232+0000 7f5ae9281640 1 --2- 172.21.15.12:0/1170152090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ae4101090 0x7f5ae419e420 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400c970 tx=0x7f5ad400ce30 comp rx=0 tx=0).stop 2024-04-22T13:31:55.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.232+0000 7f5ae9281640 1 -- 172.21.15.12:0/1170152090 shutdown_connections 2024-04-22T13:31:55.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.232+0000 7f5ae9281640 1 --2- 172.21.15.12:0/1170152090 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5ad00a3d20 0x7f5ad00a61e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.232+0000 7f5ae9281640 1 --2- 172.21.15.12:0/1170152090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ae41985d0 0x7f5ae4198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.232+0000 7f5ae9281640 1 --2- 172.21.15.12:0/1170152090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ae4106160 0x7f5ae419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.232+0000 7f5ae9281640 1 --2- 172.21.15.12:0/1170152090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5ae4101090 0x7f5ae419e420 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:55.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.232+0000 7f5ae9281640 1 -- 172.21.15.12:0/1170152090 >> 172.21.15.12:0/1170152090 conn(0x7f5ae4077f50 msgr2=0x7f5ae4069550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:55.240 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.232+0000 7f5ae9281640 1 -- 172.21.15.12:0/1170152090 shutdown_connections 2024-04-22T13:31:55.241 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:55.232+0000 7f5ae9281640 1 -- 172.21.15.12:0/1170152090 wait complete. 2024-04-22T13:31:55.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:55 smithi012 ceph-mon[28265]: pgmap v239: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:31:55.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:55 smithi134 ceph-mon[31338]: pgmap v239: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:31:55.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:55 smithi111 ceph-mon[31361]: pgmap v239: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:31:56.242 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:31:56.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:31:56.242 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:31:56.365 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.368+0000 7fb745ee0640 1 -- 172.21.15.12:0/3937645629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb740106160 msgr2=0x7fb74010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:56.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.368+0000 7fb745ee0640 1 --2- 172.21.15.12:0/3937645629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb740106160 0x7fb74010dd30 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fb738009ca0 tx=0x7fb73802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.368+0000 7fb745ee0640 1 -- 172.21.15.12:0/3937645629 shutdown_connections 2024-04-22T13:31:56.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.368+0000 7fb745ee0640 1 --2- 172.21.15.12:0/3937645629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb740106160 0x7fb74010dd30 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.368+0000 7fb745ee0640 1 --2- 172.21.15.12:0/3937645629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb740101a40 0x7fb740105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.368+0000 7fb745ee0640 1 --2- 172.21.15.12:0/3937645629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb740101090 0x7fb740101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.368+0000 7fb745ee0640 1 -- 172.21.15.12:0/3937645629 >> 172.21.15.12:0/3937645629 conn(0x7fb740077f50 msgr2=0x7fb740078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:56.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.368+0000 7fb745ee0640 1 -- 172.21.15.12:0/3937645629 shutdown_connections 2024-04-22T13:31:56.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.368+0000 7fb745ee0640 1 -- 172.21.15.12:0/3937645629 wait complete. 2024-04-22T13:31:56.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.369+0000 7fb745ee0640 1 Processor -- start 2024-04-22T13:31:56.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.369+0000 7fb745ee0640 1 -- start start 2024-04-22T13:31:56.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.369+0000 7fb745ee0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb740101090 0x7fb74010e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.369+0000 7fb745ee0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb740106160 0x7fb74010ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.369+0000 7fb745ee0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb74010f140 0x7fb7401a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.369+0000 7fb745ee0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb7401121e0 con 0x7fb740106160 2024-04-22T13:31:56.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.369+0000 7fb745ee0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb740112060 con 0x7fb74010f140 2024-04-22T13:31:56.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.369+0000 7fb745ee0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb740112360 con 0x7fb740101090 2024-04-22T13:31:56.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.369+0000 7fb73effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb740106160 0x7fb74010ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:56.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.369+0000 7fb73effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb740106160 0x7fb74010ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32828/0 (socket says 172.21.15.12:32828) 2024-04-22T13:31:56.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.369+0000 7fb73effd640 1 -- 172.21.15.12:0/3548158313 learned_addr learned my addr 172.21.15.12:0/3548158313 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:56.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.369+0000 7fb73f7fe640 1 --2- 172.21.15.12:0/3548158313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb740101090 0x7fb74010e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:56.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.369+0000 7fb73ffff640 1 --2- 172.21.15.12:0/3548158313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb74010f140 0x7fb7401a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:56.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.369+0000 7fb73effd640 1 -- 172.21.15.12:0/3548158313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb740101090 msgr2=0x7fb74010e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:56.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.369+0000 7fb73effd640 1 --2- 172.21.15.12:0/3548158313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb740101090 0x7fb74010e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.369+0000 7fb73effd640 1 -- 172.21.15.12:0/3548158313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb74010f140 msgr2=0x7fb7401a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:56.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.369+0000 7fb73effd640 1 --2- 172.21.15.12:0/3548158313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb74010f140 0x7fb7401a68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.370+0000 7fb73effd640 1 -- 172.21.15.12:0/3548158313 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7401a6fb0 con 0x7fb740106160 2024-04-22T13:31:56.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.370+0000 7fb73ffff640 1 --2- 172.21.15.12:0/3548158313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb74010f140 0x7fb7401a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:56.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.370+0000 7fb73effd640 1 --2- 172.21.15.12:0/3548158313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb740106160 0x7fb74010ec00 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fb73400d770 tx=0x7fb73400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:56.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.370+0000 7fb73cff9640 1 -- 172.21.15.12:0/3548158313 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb734017070 con 0x7fb740106160 2024-04-22T13:31:56.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.370+0000 7fb73cff9640 1 -- 172.21.15.12:0/3548158313 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb734009c30 con 0x7fb740106160 2024-04-22T13:31:56.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.370+0000 7fb745ee0640 1 -- 172.21.15.12:0/3548158313 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7401a72a0 con 0x7fb740106160 2024-04-22T13:31:56.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.370+0000 7fb73cff9640 1 -- 172.21.15.12:0/3548158313 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73400f620 con 0x7fb740106160 2024-04-22T13:31:56.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.370+0000 7fb745ee0640 1 -- 172.21.15.12:0/3548158313 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7401a7840 con 0x7fb740106160 2024-04-22T13:31:56.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.372+0000 7fb73cff9640 1 -- 172.21.15.12:0/3548158313 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb73401a070 con 0x7fb740106160 2024-04-22T13:31:56.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.373+0000 7fb748bae740 1 Processor -- start 2024-04-22T13:31:56.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.373+0000 7fb748bae740 1 -- start start 2024-04-22T13:31:56.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.373+0000 7fb748bae740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a284918e0 0x557a28491cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.373+0000 7fb748bae740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a28492290 0x557a2849a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.373+0000 7fb748bae740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a2849ae10 0x557a2849d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.373+0000 7fb748bae740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557a283f5f00 con 0x557a28492290 2024-04-22T13:31:56.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.373+0000 7fb748bae740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557a28393590 con 0x557a2849ae10 2024-04-22T13:31:56.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.373+0000 7fb748bae740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557a283f1ec0 con 0x557a284918e0 2024-04-22T13:31:56.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.373+0000 7fb73cff9640 1 --2- 172.21.15.12:0/3548158313 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb7280a3dd0 0x7fb7280a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.373+0000 7fb73f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a28492290 0x557a2849a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:56.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.373+0000 7fb73f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a28492290 0x557a2849a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32842/0 (socket says 172.21.15.12:32842) 2024-04-22T13:31:56.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.373+0000 7fb73f7fe640 1 -- 172.21.15.12:0/817044426 learned_addr learned my addr 172.21.15.12:0/817044426 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:56.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.373+0000 7fb73ffff640 1 --2- 172.21.15.12:0/3548158313 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb7280a3dd0 0x7fb7280a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:56.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.373+0000 7fb73effd640 1 --2- 172.21.15.12:0/817044426 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a2849ae10 0x557a2849d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:56.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.373+0000 7fb73ffff640 1 --2- 172.21.15.12:0/817044426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a284918e0 0x557a28491cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:56.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.373+0000 7fb73cff9640 1 -- 172.21.15.12:0/3548158313 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fb7340d02f0 con 0x7fb740106160 2024-04-22T13:31:56.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.373+0000 7fb73f7fe640 1 -- 172.21.15.12:0/817044426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a284918e0 msgr2=0x557a28491cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:56.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.373+0000 7fb73f7fe640 1 --2- 172.21.15.12:0/817044426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a284918e0 0x557a28491cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.373+0000 7fb73f7fe640 1 -- 172.21.15.12:0/817044426 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a2849ae10 msgr2=0x557a2849d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:56.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.373+0000 7fb73f7fe640 1 --2- 172.21.15.12:0/817044426 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a2849ae10 0x557a2849d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.373+0000 7fb73f7fe640 1 -- 172.21.15.12:0/817044426 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a281d07e0 con 0x557a28492290 2024-04-22T13:31:56.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.373+0000 7fb73effd640 1 --2- 172.21.15.12:0/817044426 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a2849ae10 0x557a2849d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:56.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.374+0000 7fb73f7fe640 1 --2- 172.21.15.12:0/817044426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a28492290 0x557a2849a840 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fb73000bcf0 tx=0x7fb73000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c735baf017f8f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:56.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.374+0000 7fb73ffff640 1 --2- 172.21.15.12:0/3548158313 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb7280a3dd0 0x7fb7280a6290 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb738009a60 tx=0x7fb738004a60 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:56.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.374+0000 7fb725ffb640 1 -- 172.21.15.12:0/817044426 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb730015070 con 0x557a28492290 2024-04-22T13:31:56.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.374+0000 7fb725ffb640 1 -- 172.21.15.12:0/817044426 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb73001a910 con 0x557a28492290 2024-04-22T13:31:56.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.374+0000 7fb725ffb640 1 -- 172.21.15.12:0/817044426 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7300119a0 con 0x557a28492290 2024-04-22T13:31:56.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.375+0000 7fb748bae740 1 -- 172.21.15.12:0/817044426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a28492290 msgr2=0x557a2849a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:56.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.375+0000 7fb748bae740 1 --2- 172.21.15.12:0/817044426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a28492290 0x557a2849a840 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fb73000bcf0 tx=0x7fb73000f730 comp rx=0 tx=0).stop 2024-04-22T13:31:56.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.375+0000 7fb748bae740 1 -- 172.21.15.12:0/817044426 shutdown_connections 2024-04-22T13:31:56.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.375+0000 7fb748bae740 1 --2- 172.21.15.12:0/817044426 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a2849ae10 0x557a2849d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.375+0000 7fb748bae740 1 --2- 172.21.15.12:0/817044426 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a28492290 0x557a2849a840 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.375+0000 7fb748bae740 1 --2- 172.21.15.12:0/817044426 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a284918e0 0x557a28491cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.375+0000 7fb748bae740 1 -- 172.21.15.12:0/817044426 >> 172.21.15.12:0/817044426 conn(0x557a2848bfb0 msgr2=0x557a2848c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:56.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.375+0000 7fb748bae740 1 -- 172.21.15.12:0/817044426 shutdown_connections 2024-04-22T13:31:56.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.375+0000 7fb748bae740 1 -- 172.21.15.12:0/817044426 wait complete. 2024-04-22T13:31:56.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.375+0000 7fb748bae740 1 -- start start 2024-04-22T13:31:56.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.375+0000 7fb748bae740 1 Processor -- start 2024-04-22T13:31:56.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.375+0000 7fb748bae740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a284918e0 0x557a28497e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.376+0000 7fb748bae740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a28492290 0x557a284983b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.376+0000 7fb748bae740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a2849ae10 0x557a284988f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.376+0000 7fb748bae740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557a2849dd80 con 0x557a2849ae10 2024-04-22T13:31:56.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.376+0000 7fb748bae740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557a283af580 con 0x557a28492290 2024-04-22T13:31:56.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.376+0000 7fb748bae740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557a2849df00 con 0x557a284918e0 2024-04-22T13:31:56.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.376+0000 7fb748bae740 10 client.0 _ll_register_callbacks cb 0x557a283b5650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:56.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.376+0000 7fb73ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a2849ae10 0x557a284988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:56.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.376+0000 7fb73ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a2849ae10 0x557a284988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32848/0 (socket says 172.21.15.12:32848) 2024-04-22T13:31:56.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.376+0000 7fb73ffff640 1 -- 172.21.15.12:0/2575547502 learned_addr learned my addr 172.21.15.12:0/2575547502 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:56.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.376+0000 7fb73f7fe640 1 --2- 172.21.15.12:0/2575547502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a284918e0 0x557a28497e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:56.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.376+0000 7fb73effd640 1 --2- 172.21.15.12:0/2575547502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a28492290 0x557a284983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:56.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.376+0000 7fb73ffff640 1 -- 172.21.15.12:0/2575547502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a284918e0 msgr2=0x557a28497e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:56.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.376+0000 7fb73ffff640 1 --2- 172.21.15.12:0/2575547502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a284918e0 0x557a28497e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.376+0000 7fb73ffff640 1 -- 172.21.15.12:0/2575547502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a28492290 msgr2=0x557a284983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:56.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.376+0000 7fb73ffff640 1 --2- 172.21.15.12:0/2575547502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a28492290 0x557a284983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.376+0000 7fb73ffff640 1 -- 172.21.15.12:0/2575547502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a28230f70 con 0x557a2849ae10 2024-04-22T13:31:56.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.376+0000 7fb73ffff640 1 --2- 172.21.15.12:0/2575547502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a2849ae10 0x557a284988f0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fb738036000 tx=0x7fb7380095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1fb037ea55c25f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:56.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.376+0000 7fb71effd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:56.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.376+0000 7fb71effd640 1 -- 172.21.15.12:0/2575547502 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb738050020 con 0x557a2849ae10 2024-04-22T13:31:56.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.377+0000 7fb71effd640 1 -- 172.21.15.12:0/2575547502 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fb738004240 con 0x557a2849ae10 2024-04-22T13:31:56.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.377+0000 7fb748bae740 1 -- 172.21.15.12:0/2575547502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb73001a910 con 0x557a2849ae10 2024-04-22T13:31:56.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.377+0000 7fb71effd640 1 -- 172.21.15.12:0/2575547502 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb738048040 con 0x557a2849ae10 2024-04-22T13:31:56.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.377+0000 7fb748bae740 10 client.18465 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:56.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.377+0000 7fb748bae740 1 -- 172.21.15.12:0/2575547502 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557a280c8170 con 0x557a2849ae10 2024-04-22T13:31:56.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.377+0000 7fb71effd640 1 -- 172.21.15.12:0/2575547502 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb73001a910 con 0x557a2849ae10 2024-04-22T13:31:56.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.377+0000 7fb71effd640 1 -- 172.21.15.12:0/2575547502 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb73804b030 con 0x557a2849ae10 2024-04-22T13:31:56.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.377+0000 7fb748bae740 10 client.18465 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:56.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.377+0000 7fb748bae740 10 client.18465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:56.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.377+0000 7fb748bae740 20 client.18465 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:56.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.377+0000 7fb748bae740 1 --2- 172.21.15.12:0/2575547502 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557a284ab3d0 0x557a284ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.377+0000 7fb748bae740 4 client.18465 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:56.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.377+0000 7fb748bae740 1 -- 172.21.15.12:0/2575547502 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a28403920 con 0x557a284ab3d0 2024-04-22T13:31:56.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.378+0000 7fb73f7fe640 1 --2- 172.21.15.12:0/2575547502 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557a284ab3d0 0x557a284ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:56.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.378+0000 7fb73f7fe640 1 --2- 172.21.15.12:0/2575547502 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557a284ab3d0 0x557a284ad7c0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3aaae8a4ea746e2c server_cookie=24232ecaf31e100c in_seq=0 out_seq=0 2024-04-22T13:31:56.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.378+0000 7fb71effd640 10 client.18465 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:56.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.379+0000 7fb71effd640 1 -- 172.21.15.12:0/2575547502 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557a28403920 con 0x557a284ab3d0 2024-04-22T13:31:56.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.379+0000 7fb71effd640 10 client.18465 handle_command_reply: tid=0 2024-04-22T13:31:56.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.379+0000 7fb748bae740 1 client.18465 shutdown 2024-04-22T13:31:56.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.379+0000 7fb748bae740 1 -- 172.21.15.12:0/2575547502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a2849ae10 msgr2=0x557a284988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:56.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.379+0000 7fb748bae740 1 --2- 172.21.15.12:0/2575547502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a2849ae10 0x557a284988f0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fb738036000 tx=0x7fb7380095c0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.379+0000 7fb748bae740 1 -- 172.21.15.12:0/2575547502 shutdown_connections 2024-04-22T13:31:56.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.379+0000 7fb748bae740 1 --2- 172.21.15.12:0/2575547502 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557a284ab3d0 0x557a284ad7c0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.379+0000 7fb748bae740 1 --2- 172.21.15.12:0/2575547502 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a2849ae10 0x557a284988f0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.379+0000 7fb748bae740 1 --2- 172.21.15.12:0/2575547502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a28492290 0x557a284983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.379+0000 7fb71effd640 0 client.18465 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:56.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.379+0000 7fb748bae740 1 --2- 172.21.15.12:0/2575547502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a284918e0 0x557a28497e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.379+0000 7fb748bae740 1 -- 172.21.15.12:0/2575547502 >> 172.21.15.12:0/2575547502 conn(0x557a2848bfb0 msgr2=0x557a2848dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:56.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.379+0000 7fb748bae740 1 -- 172.21.15.12:0/2575547502 shutdown_connections 2024-04-22T13:31:56.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.379+0000 7fb748bae740 1 -- 172.21.15.12:0/2575547502 wait complete. 2024-04-22T13:31:56.394 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.379+0000 7fb748bae740 20 client.18465 trim_cache size 0 max 16384 2024-04-22T13:31:56.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.397+0000 7fb748bae740 1 Processor -- start 2024-04-22T13:31:56.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.397+0000 7fb748bae740 1 -- start start 2024-04-22T13:31:56.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.397+0000 7fb748bae740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a284918e0 0x557a2849a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.397+0000 7fb748bae740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a28492290 0x557a284a6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.397+0000 7fb748bae740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a2849ae10 0x557a284b3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.397+0000 7fb748bae740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb7401afd40 con 0x557a28492290 2024-04-22T13:31:56.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.397+0000 7fb748bae740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb7401b1880 con 0x557a2849ae10 2024-04-22T13:31:56.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.397+0000 7fb748bae740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb7401b0410 con 0x557a284918e0 2024-04-22T13:31:56.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.397+0000 7fb73effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a28492290 0x557a284a6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:56.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.397+0000 7fb73effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a28492290 0x557a284a6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32862/0 (socket says 172.21.15.12:32862) 2024-04-22T13:31:56.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.397+0000 7fb73effd640 1 -- 172.21.15.12:0/3266901950 learned_addr learned my addr 172.21.15.12:0/3266901950 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:56.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.397+0000 7fb73f7fe640 1 --2- 172.21.15.12:0/3266901950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a284918e0 0x557a2849a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:56.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.397+0000 7fb73effd640 1 -- 172.21.15.12:0/3266901950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a284918e0 msgr2=0x557a2849a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:56.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.397+0000 7fb73effd640 1 --2- 172.21.15.12:0/3266901950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a284918e0 0x557a2849a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.397+0000 7fb73effd640 1 -- 172.21.15.12:0/3266901950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a2849ae10 msgr2=0x557a284b3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:56.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.397+0000 7fb73effd640 1 --2- 172.21.15.12:0/3266901950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a2849ae10 0x557a284b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.397+0000 7fb73effd640 1 -- 172.21.15.12:0/3266901950 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a284addf0 con 0x557a28492290 2024-04-22T13:31:56.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.397+0000 7fb73effd640 1 --2- 172.21.15.12:0/3266901950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a28492290 0x557a284a6c40 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fb734009610 tx=0x7fb73406ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7ba4ed2d2601a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:56.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.398+0000 7fb7257fa640 1 -- 172.21.15.12:0/3266901950 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb734021040 con 0x557a28492290 2024-04-22T13:31:56.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.398+0000 7fb7257fa640 1 -- 172.21.15.12:0/3266901950 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7340a0600 con 0x557a28492290 2024-04-22T13:31:56.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.398+0000 7fb7257fa640 1 -- 172.21.15.12:0/3266901950 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7340668c0 con 0x557a28492290 2024-04-22T13:31:56.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.398+0000 7fb748bae740 1 -- 172.21.15.12:0/3266901950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a28492290 msgr2=0x557a284a6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:56.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.398+0000 7fb748bae740 1 --2- 172.21.15.12:0/3266901950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a28492290 0x557a284a6c40 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fb734009610 tx=0x7fb73406ac00 comp rx=0 tx=0).stop 2024-04-22T13:31:56.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.398+0000 7fb748bae740 1 -- 172.21.15.12:0/3266901950 shutdown_connections 2024-04-22T13:31:56.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.398+0000 7fb748bae740 1 --2- 172.21.15.12:0/3266901950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a2849ae10 0x557a284b3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.398+0000 7fb748bae740 1 --2- 172.21.15.12:0/3266901950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a28492290 0x557a284a6c40 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.398+0000 7fb748bae740 1 --2- 172.21.15.12:0/3266901950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a284918e0 0x557a2849a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.398+0000 7fb748bae740 1 -- 172.21.15.12:0/3266901950 >> 172.21.15.12:0/3266901950 conn(0x557a2848bfb0 msgr2=0x557a284986c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:56.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.398+0000 7fb748bae740 1 -- 172.21.15.12:0/3266901950 shutdown_connections 2024-04-22T13:31:56.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.398+0000 7fb748bae740 1 -- 172.21.15.12:0/3266901950 wait complete. 2024-04-22T13:31:56.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.399+0000 7fb748bae740 1 -- start start 2024-04-22T13:31:56.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.399+0000 7fb748bae740 1 Processor -- start 2024-04-22T13:31:56.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.399+0000 7fb748bae740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a284918e0 0x557a284a3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.399+0000 7fb748bae740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a28492290 0x557a284a3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.399+0000 7fb748bae740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a2849ae10 0x557a284a0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.399+0000 7fb748bae740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb7401b11b0 con 0x557a284918e0 2024-04-22T13:31:56.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.399+0000 7fb748bae740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb7401b0ae0 con 0x557a28492290 2024-04-22T13:31:56.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.399+0000 7fb748bae740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb7401add40 con 0x557a2849ae10 2024-04-22T13:31:56.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.399+0000 7fb748bae740 10 client.0 _ll_register_callbacks cb 0x7fb738037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:56.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.399+0000 7fb73f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a284918e0 0x557a284a3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:56.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.399+0000 7fb73f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a284918e0 0x557a284a3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32866/0 (socket says 172.21.15.12:32866) 2024-04-22T13:31:56.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.399+0000 7fb73f7fe640 1 -- 172.21.15.12:0/2571984885 learned_addr learned my addr 172.21.15.12:0/2571984885 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:56.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.399+0000 7fb73ffff640 1 --2- 172.21.15.12:0/2571984885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a2849ae10 0x557a284a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:56.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.399+0000 7fb73f7fe640 1 -- 172.21.15.12:0/2571984885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a2849ae10 msgr2=0x557a284a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:56.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.399+0000 7fb73f7fe640 1 --2- 172.21.15.12:0/2571984885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a2849ae10 0x557a284a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.399+0000 7fb73f7fe640 1 -- 172.21.15.12:0/2571984885 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a28492290 msgr2=0x557a284a3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:56.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.399+0000 7fb73f7fe640 1 --2- 172.21.15.12:0/2571984885 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a28492290 0x557a284a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.399+0000 7fb73f7fe640 1 -- 172.21.15.12:0/2571984885 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a284a0860 con 0x557a284918e0 2024-04-22T13:31:56.403 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.400+0000 7fb73ffff640 1 --2- 172.21.15.12:0/2571984885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a2849ae10 0x557a284a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:56.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.400+0000 7fb73f7fe640 1 --2- 172.21.15.12:0/2571984885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a284918e0 0x557a284a3150 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x557a284913b0 tx=0x7fb73000b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e427384505254f1c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:56.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.400+0000 7fb71effd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:56.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.400+0000 7fb71effd640 1 -- 172.21.15.12:0/2571984885 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb730015040 con 0x557a284918e0 2024-04-22T13:31:56.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.400+0000 7fb71effd640 1 -- 172.21.15.12:0/2571984885 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fb730005240 con 0x557a284918e0 2024-04-22T13:31:56.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.400+0000 7fb748bae740 1 -- 172.21.15.12:0/2571984885 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7340a0600 con 0x557a284918e0 2024-04-22T13:31:56.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.400+0000 7fb71effd640 1 -- 172.21.15.12:0/2571984885 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb730036050 con 0x557a284918e0 2024-04-22T13:31:56.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.400+0000 7fb748bae740 10 client.18471 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:56.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.400+0000 7fb748bae740 1 -- 172.21.15.12:0/2571984885 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557a28512320 con 0x557a284918e0 2024-04-22T13:31:56.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.400+0000 7fb71effd640 1 -- 172.21.15.12:0/2571984885 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7340a0600 con 0x557a284918e0 2024-04-22T13:31:56.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.400+0000 7fb71effd640 1 -- 172.21.15.12:0/2571984885 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb730004680 con 0x557a284918e0 2024-04-22T13:31:56.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.400+0000 7fb748bae740 10 client.18471 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:56.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.400+0000 7fb748bae740 10 client.18471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:56.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.400+0000 7fb748bae740 20 client.18471 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:56.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.400+0000 7fb748bae740 1 --2- 172.21.15.12:0/2571984885 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557a28512850 0x557a28514c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.400+0000 7fb748bae740 4 client.18471 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:31:56.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.400+0000 7fb748bae740 1 -- 172.21.15.12:0/2571984885 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557a283fd790 con 0x557a28512850 2024-04-22T13:31:56.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.401+0000 7fb73effd640 1 --2- 172.21.15.12:0/2571984885 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557a28512850 0x557a28514c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:56.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.401+0000 7fb73effd640 1 --2- 172.21.15.12:0/2571984885 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557a28512850 0x557a28514c70 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab2be6fe71e7bdd6 server_cookie=9465f91a13f4b347 in_seq=0 out_seq=0 2024-04-22T13:31:56.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.401+0000 7fb71effd640 10 client.18471 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:56.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.402+0000 7fb71effd640 1 -- 172.21.15.12:0/2571984885 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557a283fd790 con 0x557a28512850 2024-04-22T13:31:56.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.402+0000 7fb71effd640 10 client.18471 handle_command_reply: tid=0 2024-04-22T13:31:56.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.402+0000 7fb748bae740 1 client.18471 shutdown 2024-04-22T13:31:56.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.402+0000 7fb748bae740 1 -- 172.21.15.12:0/2571984885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a284918e0 msgr2=0x557a284a3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:56.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.402+0000 7fb748bae740 1 --2- 172.21.15.12:0/2571984885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a284918e0 0x557a284a3150 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x557a284913b0 tx=0x7fb73000b560 comp rx=0 tx=0).stop 2024-04-22T13:31:56.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.402+0000 7fb748bae740 1 -- 172.21.15.12:0/2571984885 shutdown_connections 2024-04-22T13:31:56.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.402+0000 7fb748bae740 1 --2- 172.21.15.12:0/2571984885 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557a28512850 0x557a28514c70 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.402+0000 7fb748bae740 1 --2- 172.21.15.12:0/2571984885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557a2849ae10 0x557a284a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.402+0000 7fb748bae740 1 --2- 172.21.15.12:0/2571984885 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a28492290 0x557a284a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.402+0000 7fb748bae740 1 --2- 172.21.15.12:0/2571984885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557a284918e0 0x557a284a3150 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.402+0000 7fb71effd640 0 client.18471 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:56.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.402+0000 7fb748bae740 1 -- 172.21.15.12:0/2571984885 >> 172.21.15.12:0/2571984885 conn(0x557a2848bfb0 msgr2=0x557a2848a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:56.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.403+0000 7fb748bae740 1 -- 172.21.15.12:0/2571984885 shutdown_connections 2024-04-22T13:31:56.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.403+0000 7fb748bae740 1 -- 172.21.15.12:0/2571984885 wait complete. 2024-04-22T13:31:56.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.403+0000 7fb748bae740 20 client.18471 trim_cache size 0 max 16384 2024-04-22T13:31:56.406 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:31:56.406 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-04-22T13:31:56.406 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "85c59d30-d952-469a-bead-69057e4a9026", 2024-04-22T13:31:56.406 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-04-22T13:31:56.407 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:31:56.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.403+0000 7fb745ee0640 1 -- 172.21.15.12:0/3548158313 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb7280a3dd0 msgr2=0x7fb7280a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:56.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.403+0000 7fb745ee0640 1 --2- 172.21.15.12:0/3548158313 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb7280a3dd0 0x7fb7280a6290 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb738009a60 tx=0x7fb738004a60 comp rx=0 tx=0).stop 2024-04-22T13:31:56.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.403+0000 7fb745ee0640 1 -- 172.21.15.12:0/3548158313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb740106160 msgr2=0x7fb74010ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:56.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.403+0000 7fb745ee0640 1 --2- 172.21.15.12:0/3548158313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb740106160 0x7fb74010ec00 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fb73400d770 tx=0x7fb73400dc30 comp rx=0 tx=0).stop 2024-04-22T13:31:56.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.403+0000 7fb745ee0640 1 -- 172.21.15.12:0/3548158313 shutdown_connections 2024-04-22T13:31:56.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.403+0000 7fb745ee0640 1 --2- 172.21.15.12:0/3548158313 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb7280a3dd0 0x7fb7280a6290 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.403+0000 7fb745ee0640 1 --2- 172.21.15.12:0/3548158313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb74010f140 0x7fb7401a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.403+0000 7fb745ee0640 1 --2- 172.21.15.12:0/3548158313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb740106160 0x7fb74010ec00 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.403+0000 7fb745ee0640 1 --2- 172.21.15.12:0/3548158313 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb740101090 0x7fb74010e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.403+0000 7fb745ee0640 1 -- 172.21.15.12:0/3548158313 >> 172.21.15.12:0/3548158313 conn(0x7fb740077f50 msgr2=0x7fb7400fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:56.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.403+0000 7fb745ee0640 1 -- 172.21.15.12:0/3548158313 shutdown_connections 2024-04-22T13:31:56.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.403+0000 7fb745ee0640 1 -- 172.21.15.12:0/3548158313 wait complete. 2024-04-22T13:31:56.410 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:31:56.410 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:31:56.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.519+0000 7f74b0950640 1 -- 172.21.15.12:0/2365260884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74ac101080 msgr2=0x7f74ac101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:56.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.519+0000 7f74b0950640 1 --2- 172.21.15.12:0/2365260884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74ac101080 0x7f74ac101460 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f749c009900 tx=0x7f749c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:31:56.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.519+0000 7f74b0950640 1 -- 172.21.15.12:0/2365260884 shutdown_connections 2024-04-22T13:31:56.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.519+0000 7f74b0950640 1 --2- 172.21.15.12:0/2365260884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74ac106150 0x7f74ac10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.519+0000 7f74b0950640 1 --2- 172.21.15.12:0/2365260884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74ac101a30 0x7f74ac105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.519+0000 7f74b0950640 1 --2- 172.21.15.12:0/2365260884 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74ac101080 0x7f74ac101460 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.519+0000 7f74b0950640 1 -- 172.21.15.12:0/2365260884 >> 172.21.15.12:0/2365260884 conn(0x7f74ac078000 msgr2=0x7f74ac078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:56.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.519+0000 7f74b0950640 1 -- 172.21.15.12:0/2365260884 shutdown_connections 2024-04-22T13:31:56.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.519+0000 7f74b0950640 1 -- 172.21.15.12:0/2365260884 wait complete. 2024-04-22T13:31:56.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.520+0000 7f74b0950640 1 Processor -- start 2024-04-22T13:31:56.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.520+0000 7f74b0950640 1 -- start start 2024-04-22T13:31:56.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.520+0000 7f74b0950640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74ac101080 0x7f74ac102b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.520+0000 7f74b0950640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74ac106150 0x7f74ac103050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.520+0000 7f74b0950640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74ac103590 0x7f74ac1a6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.520+0000 7f74b0950640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac111130 con 0x7f74ac103590 2024-04-22T13:31:56.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.520+0000 7f74b0950640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac110fb0 con 0x7f74ac106150 2024-04-22T13:31:56.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.520+0000 7f74b0950640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac1112b0 con 0x7f74ac101080 2024-04-22T13:31:56.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.520+0000 7f74aad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74ac103590 0x7f74ac1a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:56.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.521+0000 7f74aad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74ac103590 0x7f74ac1a6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32880/0 (socket says 172.21.15.12:32880) 2024-04-22T13:31:56.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.521+0000 7f74aad9d640 1 -- 172.21.15.12:0/564498825 learned_addr learned my addr 172.21.15.12:0/564498825 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:56.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.521+0000 7f74aa59c640 1 --2- 172.21.15.12:0/564498825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74ac101080 0x7f74ac102b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:56.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.521+0000 7f74a3fff640 1 --2- 172.21.15.12:0/564498825 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74ac106150 0x7f74ac103050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:56.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.521+0000 7f74aad9d640 1 -- 172.21.15.12:0/564498825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74ac101080 msgr2=0x7f74ac102b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:56.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.521+0000 7f74aad9d640 1 --2- 172.21.15.12:0/564498825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74ac101080 0x7f74ac102b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.521+0000 7f74aad9d640 1 -- 172.21.15.12:0/564498825 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74ac106150 msgr2=0x7f74ac103050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:56.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.521+0000 7f74aad9d640 1 --2- 172.21.15.12:0/564498825 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74ac106150 0x7f74ac103050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.521+0000 7f74aad9d640 1 -- 172.21.15.12:0/564498825 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74ac1a7010 con 0x7f74ac103590 2024-04-22T13:31:56.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.521+0000 7f74aad9d640 1 --2- 172.21.15.12:0/564498825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74ac103590 0x7f74ac1a6970 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f74a400b970 tx=0x7f74a400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:56.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.521+0000 7f74a37fe640 1 -- 172.21.15.12:0/564498825 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a4019070 con 0x7f74ac103590 2024-04-22T13:31:56.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.521+0000 7f74a37fe640 1 -- 172.21.15.12:0/564498825 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74a4014440 con 0x7f74ac103590 2024-04-22T13:31:56.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.521+0000 7f74b0950640 1 -- 172.21.15.12:0/564498825 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74ac1a7300 con 0x7f74ac103590 2024-04-22T13:31:56.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.521+0000 7f74a37fe640 1 -- 172.21.15.12:0/564498825 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a40075d0 con 0x7f74ac103590 2024-04-22T13:31:56.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.521+0000 7f74b0950640 1 -- 172.21.15.12:0/564498825 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74ac1a7850 con 0x7f74ac103590 2024-04-22T13:31:56.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.523+0000 7f74a37fe640 1 -- 172.21.15.12:0/564498825 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f74a401c070 con 0x7f74ac103590 2024-04-22T13:31:56.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.524+0000 7f74b361c740 1 Processor -- start 2024-04-22T13:31:56.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.524+0000 7f74b361c740 1 -- start start 2024-04-22T13:31:56.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.524+0000 7f74b361c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3ae896580 0x55b3ae896960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.524+0000 7f74b361c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ae896ea0 0x55b3ae89f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.524+0000 7f74b361c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3ae89fa80 0x55b3ae8a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.524+0000 7f74b361c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b3ae7fb1e0 con 0x55b3ae896ea0 2024-04-22T13:31:56.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.524+0000 7f74b361c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b3ae798390 con 0x55b3ae89fa80 2024-04-22T13:31:56.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.524+0000 7f74b361c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b3ae7f71a0 con 0x55b3ae896580 2024-04-22T13:31:56.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.524+0000 7f74aa59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ae896ea0 0x55b3ae89f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:56.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.524+0000 7f74aa59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ae896ea0 0x55b3ae89f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32890/0 (socket says 172.21.15.12:32890) 2024-04-22T13:31:56.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.524+0000 7f74aa59c640 1 -- 172.21.15.12:0/3811380699 learned_addr learned my addr 172.21.15.12:0/3811380699 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:56.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.524+0000 7f74a3fff640 1 --2- 172.21.15.12:0/3811380699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3ae896580 0x55b3ae896960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:56.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.524+0000 7f74a37fe640 1 --2- 172.21.15.12:0/564498825 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f74980a3da0 0x7f74980a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.524+0000 7f74a3fff640 1 --2- 172.21.15.12:0/3811380699 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3ae89fa80 0x55b3ae8a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:56.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.525+0000 7f74aad9d640 1 --2- 172.21.15.12:0/564498825 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f74980a3da0 0x7f74980a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:56.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.525+0000 7f74aa59c640 1 -- 172.21.15.12:0/3811380699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3ae896580 msgr2=0x55b3ae896960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:56.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.525+0000 7f74aa59c640 1 --2- 172.21.15.12:0/3811380699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3ae896580 0x55b3ae896960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.525+0000 7f74aa59c640 1 -- 172.21.15.12:0/3811380699 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3ae89fa80 msgr2=0x55b3ae8a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:56.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.525+0000 7f74aa59c640 1 --2- 172.21.15.12:0/3811380699 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3ae89fa80 0x55b3ae8a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.525+0000 7f74aa59c640 1 -- 172.21.15.12:0/3811380699 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3ae5d77d0 con 0x55b3ae896ea0 2024-04-22T13:31:56.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.525+0000 7f74a37fe640 1 -- 172.21.15.12:0/564498825 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f74a40cfc00 con 0x7f74ac103590 2024-04-22T13:31:56.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.525+0000 7f74aa59c640 1 --2- 172.21.15.12:0/3811380699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ae896ea0 0x55b3ae89f540 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f749c009900 tx=0x7f749c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a90374fda33d23 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:56.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.525+0000 7f74aad9d640 1 --2- 172.21.15.12:0/564498825 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f74980a3da0 0x7f74980a6260 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f74a409ba80 tx=0x7f74a4014d00 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:56.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.525+0000 7f74a0ff9640 1 -- 172.21.15.12:0/3811380699 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749c045020 con 0x55b3ae896ea0 2024-04-22T13:31:56.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.525+0000 7f74a0ff9640 1 -- 172.21.15.12:0/3811380699 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f749c02cc90 con 0x55b3ae896ea0 2024-04-22T13:31:56.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.525+0000 7f74a0ff9640 1 -- 172.21.15.12:0/3811380699 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749c0046c0 con 0x55b3ae896ea0 2024-04-22T13:31:56.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.526+0000 7f74b361c740 1 -- 172.21.15.12:0/3811380699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ae896ea0 msgr2=0x55b3ae89f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:56.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.526+0000 7f74b361c740 1 --2- 172.21.15.12:0/3811380699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ae896ea0 0x55b3ae89f540 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f749c009900 tx=0x7f749c004040 comp rx=0 tx=0).stop 2024-04-22T13:31:56.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.526+0000 7f74b361c740 1 -- 172.21.15.12:0/3811380699 shutdown_connections 2024-04-22T13:31:56.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.526+0000 7f74b361c740 1 --2- 172.21.15.12:0/3811380699 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3ae89fa80 0x55b3ae8a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.526+0000 7f74b361c740 1 --2- 172.21.15.12:0/3811380699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ae896ea0 0x55b3ae89f540 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.526+0000 7f74b361c740 1 --2- 172.21.15.12:0/3811380699 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3ae896580 0x55b3ae896960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.526+0000 7f74b361c740 1 -- 172.21.15.12:0/3811380699 >> 172.21.15.12:0/3811380699 conn(0x55b3ae890cb0 msgr2=0x55b3ae891090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:56.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.526+0000 7f74b361c740 1 -- 172.21.15.12:0/3811380699 shutdown_connections 2024-04-22T13:31:56.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.526+0000 7f74b361c740 1 -- 172.21.15.12:0/3811380699 wait complete. 2024-04-22T13:31:56.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.526+0000 7f74b361c740 1 -- start start 2024-04-22T13:31:56.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.526+0000 7f74b361c740 1 Processor -- start 2024-04-22T13:31:56.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.527+0000 7f74b361c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ae896580 0x55b3ae89cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.527+0000 7f74b361c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3ae896ea0 0x55b3ae89d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.527+0000 7f74b361c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3ae89fa80 0x55b3ae89d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.527+0000 7f74b361c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b3ae8a2810 con 0x55b3ae896580 2024-04-22T13:31:56.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.527+0000 7f74b361c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b3ae7b42f0 con 0x55b3ae89fa80 2024-04-22T13:31:56.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.527+0000 7f74b361c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b3ae8a2990 con 0x55b3ae896ea0 2024-04-22T13:31:56.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.527+0000 7f74b361c740 10 client.0 _ll_register_callbacks cb 0x55b3ae7c1580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:56.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.527+0000 7f74a3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ae896580 0x55b3ae89cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:56.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.527+0000 7f74a3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ae896580 0x55b3ae89cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32904/0 (socket says 172.21.15.12:32904) 2024-04-22T13:31:56.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.527+0000 7f74a3fff640 1 -- 172.21.15.12:0/1592887683 learned_addr learned my addr 172.21.15.12:0/1592887683 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:56.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.527+0000 7f74a3fff640 1 -- 172.21.15.12:0/1592887683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3ae896ea0 msgr2=0x55b3ae89d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:56.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.527+0000 7f74a3fff640 1 --2- 172.21.15.12:0/1592887683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3ae896ea0 0x55b3ae89d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.527+0000 7f74a3fff640 1 -- 172.21.15.12:0/1592887683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3ae89fa80 msgr2=0x55b3ae89d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:56.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.527+0000 7f74a3fff640 1 --2- 172.21.15.12:0/1592887683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3ae89fa80 0x55b3ae89d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.527+0000 7f74a3fff640 1 -- 172.21.15.12:0/1592887683 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3ae50d3f0 con 0x55b3ae896580 2024-04-22T13:31:56.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.527+0000 7f74a3fff640 1 --2- 172.21.15.12:0/1592887683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ae896580 0x55b3ae89cae0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f749400c890 tx=0x7f7494011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2dbe8049b4f19a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:56.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.527+0000 7f7491ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:56.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.527+0000 7f7491ffb640 1 -- 172.21.15.12:0/1592887683 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7494005020 con 0x55b3ae896580 2024-04-22T13:31:56.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.527+0000 7f7491ffb640 1 -- 172.21.15.12:0/1592887683 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f74940051c0 con 0x55b3ae896580 2024-04-22T13:31:56.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.527+0000 7f74b361c740 1 -- 172.21.15.12:0/1592887683 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f749c02cc90 con 0x55b3ae896580 2024-04-22T13:31:56.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.527+0000 7f7491ffb640 1 -- 172.21.15.12:0/1592887683 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7494026050 con 0x55b3ae896580 2024-04-22T13:31:56.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.527+0000 7f74b361c740 10 client.18483 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:56.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.527+0000 7f74b361c740 1 -- 172.21.15.12:0/1592887683 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b3ae4cd0f0 con 0x55b3ae896580 2024-04-22T13:31:56.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.528+0000 7f7491ffb640 1 -- 172.21.15.12:0/1592887683 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f749c02cc90 con 0x55b3ae896580 2024-04-22T13:31:56.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.528+0000 7f7491ffb640 1 -- 172.21.15.12:0/1592887683 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7494012450 con 0x55b3ae896580 2024-04-22T13:31:56.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.528+0000 7f74b361c740 10 client.18483 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:56.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.528+0000 7f74b361c740 10 client.18483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:56.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.528+0000 7f74b361c740 20 client.18483 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:56.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.528+0000 7f74b361c740 1 --2- 172.21.15.12:0/1592887683 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b3ae8afa40 0x55b3ae8b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.528+0000 7f74b361c740 4 client.18483 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:31:56.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.528+0000 7f74b361c740 1 -- 172.21.15.12:0/1592887683 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3ae808c00 con 0x55b3ae8afa40 2024-04-22T13:31:56.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.528+0000 7f74aa59c640 1 --2- 172.21.15.12:0/1592887683 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b3ae8afa40 0x55b3ae8b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:56.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.528+0000 7f74aa59c640 1 --2- 172.21.15.12:0/1592887683 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b3ae8afa40 0x55b3ae8b1ea0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=796e9d9d34a2ecd5 server_cookie=6f67a94ebf9bd6b5 in_seq=0 out_seq=0 2024-04-22T13:31:56.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.528+0000 7f7491ffb640 10 client.18483 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:56.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.529+0000 7f7491ffb640 1 -- 172.21.15.12:0/1592887683 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b3ae808c00 con 0x55b3ae8afa40 2024-04-22T13:31:56.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.529+0000 7f7491ffb640 10 client.18483 handle_command_reply: tid=0 2024-04-22T13:31:56.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.529+0000 7f74b361c740 1 client.18483 shutdown 2024-04-22T13:31:56.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.529+0000 7f74b361c740 1 -- 172.21.15.12:0/1592887683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ae896580 msgr2=0x55b3ae89cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:56.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.529+0000 7f74b361c740 1 --2- 172.21.15.12:0/1592887683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ae896580 0x55b3ae89cae0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f749400c890 tx=0x7f7494011810 comp rx=0 tx=0).stop 2024-04-22T13:31:56.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.529+0000 7f74b361c740 1 -- 172.21.15.12:0/1592887683 shutdown_connections 2024-04-22T13:31:56.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.529+0000 7f74b361c740 1 --2- 172.21.15.12:0/1592887683 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b3ae8afa40 0x55b3ae8b1ea0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.529+0000 7f74b361c740 1 --2- 172.21.15.12:0/1592887683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3ae89fa80 0x55b3ae89d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.529+0000 7f7491ffb640 0 client.18483 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:56.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.529+0000 7f74b361c740 1 --2- 172.21.15.12:0/1592887683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3ae896ea0 0x55b3ae89d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.529+0000 7f74b361c740 1 --2- 172.21.15.12:0/1592887683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ae896580 0x55b3ae89cae0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.529+0000 7f74b361c740 1 -- 172.21.15.12:0/1592887683 >> 172.21.15.12:0/1592887683 conn(0x55b3ae890cb0 msgr2=0x55b3ae892a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:56.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.529+0000 7f74b361c740 1 -- 172.21.15.12:0/1592887683 shutdown_connections 2024-04-22T13:31:56.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.529+0000 7f74b361c740 1 -- 172.21.15.12:0/1592887683 wait complete. 2024-04-22T13:31:56.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.529+0000 7f74b361c740 20 client.18483 trim_cache size 0 max 16384 2024-04-22T13:31:56.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.547+0000 7f74b361c740 1 Processor -- start 2024-04-22T13:31:56.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.547+0000 7f74b361c740 1 -- start start 2024-04-22T13:31:56.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.547+0000 7f74b361c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3ae896580 0x55b3ae89f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.547+0000 7f74b361c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3ae896ea0 0x55b3ae89ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.547+0000 7f74b361c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ae89fa80 0x55b3ae8ab6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.547+0000 7f74b361c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac1afca0 con 0x55b3ae89fa80 2024-04-22T13:31:56.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.547+0000 7f74b361c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac1b17e0 con 0x55b3ae896580 2024-04-22T13:31:56.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.547+0000 7f74b361c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac1b0370 con 0x55b3ae896ea0 2024-04-22T13:31:56.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.547+0000 7f74a3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ae89fa80 0x55b3ae8ab6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:56.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.547+0000 7f74a3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ae89fa80 0x55b3ae8ab6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32918/0 (socket says 172.21.15.12:32918) 2024-04-22T13:31:56.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.547+0000 7f74a3fff640 1 -- 172.21.15.12:0/1977917959 learned_addr learned my addr 172.21.15.12:0/1977917959 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:56.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.547+0000 7f74a3fff640 1 --2- 172.21.15.12:0/1977917959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3ae896580 0x55b3ae89f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:56.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.547+0000 7f74aa59c640 1 --2- 172.21.15.12:0/1977917959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3ae896ea0 0x55b3ae89ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:56.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.548+0000 7f74a3fff640 1 -- 172.21.15.12:0/1977917959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3ae896ea0 msgr2=0x55b3ae89ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:56.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.548+0000 7f74a3fff640 1 --2- 172.21.15.12:0/1977917959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3ae896ea0 0x55b3ae89ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.548+0000 7f74a3fff640 1 -- 172.21.15.12:0/1977917959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3ae896580 msgr2=0x55b3ae89f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:56.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.548+0000 7f74a3fff640 1 --2- 172.21.15.12:0/1977917959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3ae896580 0x55b3ae89f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.548+0000 7f74a3fff640 1 -- 172.21.15.12:0/1977917959 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3ae8a1e70 con 0x55b3ae89fa80 2024-04-22T13:31:56.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.548+0000 7f74aa59c640 1 --2- 172.21.15.12:0/1977917959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3ae896ea0 0x55b3ae89ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:31:56.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.548+0000 7f74a3fff640 1 --2- 172.21.15.12:0/1977917959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ae89fa80 0x55b3ae8ab6d0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f749400c900 tx=0x7f7494005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94e0c9b0106c7aa server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:56.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.548+0000 7f7493fff640 1 -- 172.21.15.12:0/1977917959 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7494012450 con 0x55b3ae89fa80 2024-04-22T13:31:56.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.548+0000 7f7493fff640 1 -- 172.21.15.12:0/1977917959 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74940125f0 con 0x55b3ae89fa80 2024-04-22T13:31:56.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.548+0000 7f7493fff640 1 -- 172.21.15.12:0/1977917959 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74940152d0 con 0x55b3ae89fa80 2024-04-22T13:31:56.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.548+0000 7f74b361c740 1 -- 172.21.15.12:0/1977917959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ae89fa80 msgr2=0x55b3ae8ab6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:56.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.548+0000 7f74b361c740 1 --2- 172.21.15.12:0/1977917959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ae89fa80 0x55b3ae8ab6d0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f749400c900 tx=0x7f7494005850 comp rx=0 tx=0).stop 2024-04-22T13:31:56.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.548+0000 7f74b361c740 1 -- 172.21.15.12:0/1977917959 shutdown_connections 2024-04-22T13:31:56.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.548+0000 7f74b361c740 1 --2- 172.21.15.12:0/1977917959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ae89fa80 0x55b3ae8ab6d0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.548+0000 7f74b361c740 1 --2- 172.21.15.12:0/1977917959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3ae896ea0 0x55b3ae89ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.548+0000 7f74b361c740 1 --2- 172.21.15.12:0/1977917959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3ae896580 0x55b3ae89f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.548+0000 7f74b361c740 1 -- 172.21.15.12:0/1977917959 >> 172.21.15.12:0/1977917959 conn(0x55b3ae890cb0 msgr2=0x55b3ae892990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:56.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.549+0000 7f74b361c740 1 -- 172.21.15.12:0/1977917959 shutdown_connections 2024-04-22T13:31:56.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.549+0000 7f74b361c740 1 -- 172.21.15.12:0/1977917959 wait complete. 2024-04-22T13:31:56.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.549+0000 7f74b361c740 1 -- start start 2024-04-22T13:31:56.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.549+0000 7f74b361c740 1 Processor -- start 2024-04-22T13:31:56.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.549+0000 7f74b361c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ae896580 0x55b3ae90f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.549+0000 7f74b361c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3ae896ea0 0x55b3ae90f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.549+0000 7f74b361c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3ae89fa80 0x55b3ae8a3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.549+0000 7f74b361c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac1b1110 con 0x55b3ae896580 2024-04-22T13:31:56.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.549+0000 7f74b361c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac1b0a40 con 0x55b3ae89fa80 2024-04-22T13:31:56.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.549+0000 7f74b361c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac1adcc0 con 0x55b3ae896ea0 2024-04-22T13:31:56.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.549+0000 7f74b361c740 10 client.0 _ll_register_callbacks cb 0x7f749400cd50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:31:56.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.549+0000 7f74a3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ae896580 0x55b3ae90f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:56.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.549+0000 7f74a3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ae896580 0x55b3ae90f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:32932/0 (socket says 172.21.15.12:32932) 2024-04-22T13:31:56.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.549+0000 7f74a3fff640 1 -- 172.21.15.12:0/899197701 learned_addr learned my addr 172.21.15.12:0/899197701 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:31:56.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.549+0000 7f74aa59c640 1 --2- 172.21.15.12:0/899197701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3ae896ea0 0x55b3ae90f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:56.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.549+0000 7f74a3fff640 1 -- 172.21.15.12:0/899197701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3ae896ea0 msgr2=0x55b3ae90f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:56.562 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.549+0000 7f74a3fff640 1 --2- 172.21.15.12:0/899197701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3ae896ea0 0x55b3ae90f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.549+0000 7f74a3fff640 1 -- 172.21.15.12:0/899197701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3ae89fa80 msgr2=0x55b3ae8a3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:31:56.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.550+0000 7f74a3fff640 1 --2- 172.21.15.12:0/899197701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3ae89fa80 0x55b3ae8a3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.550+0000 7f74a3fff640 1 -- 172.21.15.12:0/899197701 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3ae8a44a0 con 0x55b3ae896580 2024-04-22T13:31:56.563 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.550+0000 7f74aa59c640 1 --2- 172.21.15.12:0/899197701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3ae896ea0 0x55b3ae90f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:31:56.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.550+0000 7f74a3fff640 1 --2- 172.21.15.12:0/899197701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ae896580 0x55b3ae90f2f0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f7494015c80 tx=0x7f7494005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b3354d6217e32b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:31:56.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.550+0000 7f7491ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:31:56.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.550+0000 7f7491ffb640 1 -- 172.21.15.12:0/899197701 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7494012450 con 0x55b3ae896580 2024-04-22T13:31:56.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.550+0000 7f7491ffb640 1 -- 172.21.15.12:0/899197701 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f7494012a40 con 0x55b3ae896580 2024-04-22T13:31:56.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.550+0000 7f74b361c740 1 -- 172.21.15.12:0/899197701 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74940125f0 con 0x55b3ae896580 2024-04-22T13:31:56.564 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.550+0000 7f7491ffb640 1 -- 172.21.15.12:0/899197701 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f749403a050 con 0x55b3ae896580 2024-04-22T13:31:56.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.550+0000 7f74b361c740 10 client.18489 fetch_fsmap learned FSMap version 28 2024-04-22T13:31:56.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.550+0000 7f74b361c740 1 -- 172.21.15.12:0/899197701 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b3ae916a20 con 0x55b3ae896580 2024-04-22T13:31:56.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.550+0000 7f7491ffb640 1 -- 172.21.15.12:0/899197701 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74940125f0 con 0x55b3ae896580 2024-04-22T13:31:56.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.551+0000 7f7491ffb640 1 -- 172.21.15.12:0/899197701 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f749401f7d0 con 0x55b3ae896580 2024-04-22T13:31:56.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.551+0000 7f74b361c740 10 client.18489 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:31:56.565 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.551+0000 7f74b361c740 10 client.18489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:31:56.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.551+0000 7f74b361c740 20 client.18489 populate_metadata read hostname 'smithi012' 2024-04-22T13:31:56.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.551+0000 7f74b361c740 1 --2- 172.21.15.12:0/899197701 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b3ae9170d0 0x55b3ae9194f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:31:56.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.551+0000 7f74b361c740 4 client.18489 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:31:56.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.551+0000 7f74b361c740 1 -- 172.21.15.12:0/899197701 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b3ae802a70 con 0x55b3ae9170d0 2024-04-22T13:31:56.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.551+0000 7f74aa59c640 1 --2- 172.21.15.12:0/899197701 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b3ae9170d0 0x55b3ae9194f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:31:56.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.551+0000 7f7491ffb640 10 client.18489 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:56.566 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.551+0000 7f74aa59c640 1 --2- 172.21.15.12:0/899197701 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b3ae9170d0 0x55b3ae9194f0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=292b596c154acabf server_cookie=11b58d4a05c80935 in_seq=0 out_seq=0 2024-04-22T13:31:56.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.551+0000 7f7491ffb640 1 -- 172.21.15.12:0/899197701 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b3ae802a70 con 0x55b3ae9170d0 2024-04-22T13:31:56.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.552+0000 7f7491ffb640 10 client.18489 handle_command_reply: tid=0 2024-04-22T13:31:56.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.552+0000 7f74b361c740 1 client.18489 shutdown 2024-04-22T13:31:56.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.552+0000 7f74b361c740 1 -- 172.21.15.12:0/899197701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ae896580 msgr2=0x55b3ae90f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:56.567 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.552+0000 7f74b361c740 1 --2- 172.21.15.12:0/899197701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ae896580 0x55b3ae90f2f0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f7494015c80 tx=0x7f7494005fa0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.552+0000 7f74b361c740 1 -- 172.21.15.12:0/899197701 shutdown_connections 2024-04-22T13:31:56.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.552+0000 7f74b361c740 1 --2- 172.21.15.12:0/899197701 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b3ae9170d0 0x55b3ae9194f0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.552+0000 7f74b361c740 1 --2- 172.21.15.12:0/899197701 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b3ae89fa80 0x55b3ae8a3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.552+0000 7f74b361c740 1 --2- 172.21.15.12:0/899197701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b3ae896ea0 0x55b3ae90f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.552+0000 7f7491ffb640 0 client.18489 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:31:56.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.552+0000 7f74b361c740 1 --2- 172.21.15.12:0/899197701 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b3ae896580 0x55b3ae90f2f0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.552+0000 7f74b361c740 1 -- 172.21.15.12:0/899197701 >> 172.21.15.12:0/899197701 conn(0x55b3ae890cb0 msgr2=0x55b3ae892090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:56.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.552+0000 7f74b361c740 1 -- 172.21.15.12:0/899197701 shutdown_connections 2024-04-22T13:31:56.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.552+0000 7f74b361c740 1 -- 172.21.15.12:0/899197701 wait complete. 2024-04-22T13:31:56.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.552+0000 7f74b361c740 20 client.18489 trim_cache size 0 max 16384 2024-04-22T13:31:56.570 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:31:56.570 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:31:56.570 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-04-22T13:31:56.570 INFO:teuthology.orchestra.run.smithi012.stdout: "85c59d30-d952-469a-bead-69057e4a9026": { 2024-04-22T13:31:56.570 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-04-22T13:31:56.571 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "85c59d30-d952-469a-bead-69057e4a9026", 2024-04-22T13:31:56.571 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-04-22T13:31:56.571 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:31:56.571 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:31:56.571 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:31:56.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.552+0000 7f74b0950640 1 -- 172.21.15.12:0/564498825 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f74980a3da0 msgr2=0x7f74980a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:56.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.552+0000 7f74b0950640 1 --2- 172.21.15.12:0/564498825 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f74980a3da0 0x7f74980a6260 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f74a409ba80 tx=0x7f74a4014d00 comp rx=0 tx=0).stop 2024-04-22T13:31:56.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.552+0000 7f74b0950640 1 -- 172.21.15.12:0/564498825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74ac103590 msgr2=0x7f74ac1a6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:31:56.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.552+0000 7f74b0950640 1 --2- 172.21.15.12:0/564498825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74ac103590 0x7f74ac1a6970 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f74a400b970 tx=0x7f74a400be30 comp rx=0 tx=0).stop 2024-04-22T13:31:56.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.553+0000 7f74b0950640 1 -- 172.21.15.12:0/564498825 shutdown_connections 2024-04-22T13:31:56.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.553+0000 7f74b0950640 1 --2- 172.21.15.12:0/564498825 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f74980a3da0 0x7f74980a6260 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.553+0000 7f74b0950640 1 --2- 172.21.15.12:0/564498825 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f74ac103590 0x7f74ac1a6970 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.553+0000 7f74b0950640 1 --2- 172.21.15.12:0/564498825 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74ac106150 0x7f74ac103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.553+0000 7f74b0950640 1 --2- 172.21.15.12:0/564498825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f74ac101080 0x7f74ac102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:31:56.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.553+0000 7f74b0950640 1 -- 172.21.15.12:0/564498825 >> 172.21.15.12:0/564498825 conn(0x7f74ac078000 msgr2=0x7f74ac0694e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:31:56.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.553+0000 7f74b0950640 1 -- 172.21.15.12:0/564498825 shutdown_connections 2024-04-22T13:31:56.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:31:56.553+0000 7f74b0950640 1 -- 172.21.15.12:0/564498825 wait complete. 2024-04-22T13:31:56.576 INFO:tasks.cephfs.filesystem:scrub status for tag:85c59d30-d952-469a-bead-69057e4a9026 - {'path': '/', 'tag': '85c59d30-d952-469a-bead-69057e4a9026', 'options': 'recursive,force'} 2024-04-22T13:31:57.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:57 smithi012 ceph-mon[28265]: pgmap v240: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:31:57.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:57 smithi012 ceph-mon[28265]: scrub queued for path: / 2024-04-22T13:31:57.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:57 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:31:57.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:57 smithi012 ceph-mon[28265]: scrub summary: active paths [/] 2024-04-22T13:31:57.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:57 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:31:57.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:57 smithi134 ceph-mon[31338]: pgmap v240: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:31:57.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:57 smithi134 ceph-mon[31338]: scrub queued for path: / 2024-04-22T13:31:57.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:57 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:31:57.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:57 smithi134 ceph-mon[31338]: scrub summary: active paths [/] 2024-04-22T13:31:57.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:57 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:31:57.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:57 smithi111 ceph-mon[31361]: pgmap v240: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-22T13:31:57.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:57 smithi111 ceph-mon[31361]: scrub queued for path: / 2024-04-22T13:31:57.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:57 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:31:57.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:57 smithi111 ceph-mon[31361]: scrub summary: active paths [/] 2024-04-22T13:31:57.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:57 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:31:59.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:31:59 smithi012 ceph-mon[28265]: pgmap v241: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 13 op/s 2024-04-22T13:31:59.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:31:59 smithi134 ceph-mon[31338]: pgmap v241: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 13 op/s 2024-04-22T13:31:59.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:31:59 smithi111 ceph-mon[31361]: pgmap v241: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 13 op/s 2024-04-22T13:32:00.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:00 smithi012 ceph-mon[28265]: scrub summary: idle 2024-04-22T13:32:00.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:00 smithi134 ceph-mon[31338]: scrub summary: idle 2024-04-22T13:32:00.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:00 smithi111 ceph-mon[31361]: scrub summary: idle 2024-04-22T13:32:01.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:01 smithi012 ceph-mon[28265]: pgmap v242: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:32:01.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:01 smithi134 ceph-mon[31338]: pgmap v242: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:32:01.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:01 smithi111 ceph-mon[31361]: pgmap v242: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:32:03.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:03 smithi012 ceph-mon[28265]: pgmap v243: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-22T13:32:03.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:03 smithi134 ceph-mon[31338]: pgmap v243: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-22T13:32:03.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:03 smithi111 ceph-mon[31361]: pgmap v243: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-22T13:32:05.838 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:05 smithi012 ceph-mon[28265]: pgmap v244: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-22T13:32:05.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:05 smithi134 ceph-mon[31338]: pgmap v244: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-22T13:32:05.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:05 smithi111 ceph-mon[31361]: pgmap v244: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-22T13:32:07.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:07 smithi012 ceph-mon[28265]: pgmap v245: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-22T13:32:07.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:07 smithi134 ceph-mon[31338]: pgmap v245: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-22T13:32:07.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:07 smithi111 ceph-mon[31361]: pgmap v245: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-22T13:32:09.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:09 smithi012 ceph-mon[28265]: pgmap v246: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-22T13:32:09.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:09 smithi134 ceph-mon[31338]: pgmap v246: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-22T13:32:09.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:09 smithi111 ceph-mon[31361]: pgmap v246: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-22T13:32:11.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:11 smithi012 ceph-mon[28265]: pgmap v247: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:32:11.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:11 smithi012 ceph-mon[28265]: osdmap e93: 12 total, 12 up, 12 in 2024-04-22T13:32:11.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:11 smithi134 ceph-mon[31338]: pgmap v247: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:32:11.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:11 smithi134 ceph-mon[31338]: osdmap e93: 12 total, 12 up, 12 in 2024-04-22T13:32:11.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:11 smithi111 ceph-mon[31361]: pgmap v247: 97 pgs: 97 active+clean; 259 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:32:11.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:11 smithi111 ceph-mon[31361]: osdmap e93: 12 total, 12 up, 12 in 2024-04-22T13:32:13.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:13 smithi134 ceph-mon[31338]: pgmap v249: 97 pgs: 18 active+clean+snaptrim, 50 active+clean, 29 active+clean+snaptrim_wait; 138 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:32:13.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:13 smithi111 ceph-mon[31361]: pgmap v249: 97 pgs: 18 active+clean+snaptrim, 50 active+clean, 29 active+clean+snaptrim_wait; 138 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:32:14.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:13 smithi012 ceph-mon[28265]: pgmap v249: 97 pgs: 18 active+clean+snaptrim, 50 active+clean, 29 active+clean+snaptrim_wait; 138 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:32:15.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:15 smithi134 ceph-mon[31338]: pgmap v250: 97 pgs: 18 active+clean+snaptrim, 50 active+clean, 29 active+clean+snaptrim_wait; 138 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:32:15.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:15 smithi111 ceph-mon[31361]: pgmap v250: 97 pgs: 18 active+clean+snaptrim, 50 active+clean, 29 active+clean+snaptrim_wait; 138 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:32:16.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:15 smithi012 ceph-mon[28265]: pgmap v250: 97 pgs: 18 active+clean+snaptrim, 50 active+clean, 29 active+clean+snaptrim_wait; 138 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:32:17.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:17 smithi134 ceph-mon[31338]: pgmap v251: 97 pgs: 18 active+clean+snaptrim, 50 active+clean, 29 active+clean+snaptrim_wait; 138 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:32:17.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:17 smithi111 ceph-mon[31361]: pgmap v251: 97 pgs: 18 active+clean+snaptrim, 50 active+clean, 29 active+clean+snaptrim_wait; 138 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:32:18.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:17 smithi012 ceph-mon[28265]: pgmap v251: 97 pgs: 18 active+clean+snaptrim, 50 active+clean, 29 active+clean+snaptrim_wait; 138 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:32:18.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:18 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:32:18.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:18 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:32:19.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:18 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:32:19.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:19 smithi134 ceph-mon[31338]: pgmap v252: 97 pgs: 97 active+clean; 2.3 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:32:19.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:19 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:32:19.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:19 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:32:19.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:19 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:32:19.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:19 smithi134 ceph-mon[31338]: osdmap e94: 12 total, 12 up, 12 in 2024-04-22T13:32:19.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:19 smithi111 ceph-mon[31361]: pgmap v252: 97 pgs: 97 active+clean; 2.3 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:32:19.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:19 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:32:19.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:19 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:32:19.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:19 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:32:19.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:19 smithi111 ceph-mon[31361]: osdmap e94: 12 total, 12 up, 12 in 2024-04-22T13:32:20.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:19 smithi012 ceph-mon[28265]: pgmap v252: 97 pgs: 97 active+clean; 2.3 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:32:20.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:19 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:32:20.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:19 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:32:20.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:19 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:32:20.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:19 smithi012 ceph-mon[28265]: osdmap e94: 12 total, 12 up, 12 in 2024-04-22T13:32:21.888 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:21 smithi012 ceph-mon[28265]: pgmap v254: 97 pgs: 97 active+clean; 2.3 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 210 B/s wr, 10 op/s 2024-04-22T13:32:21.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:21 smithi134 ceph-mon[31338]: pgmap v254: 97 pgs: 97 active+clean; 2.3 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 210 B/s wr, 10 op/s 2024-04-22T13:32:21.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:21 smithi111 ceph-mon[31361]: pgmap v254: 97 pgs: 97 active+clean; 2.3 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 210 B/s wr, 10 op/s 2024-04-22T13:32:23.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:23 smithi134 ceph-mon[31338]: pgmap v255: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-22T13:32:23.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:23 smithi111 ceph-mon[31361]: pgmap v255: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-22T13:32:24.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:23 smithi012 ceph-mon[28265]: pgmap v255: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-22T13:32:25.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:25 smithi134 ceph-mon[31338]: pgmap v256: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-22T13:32:25.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:25 smithi111 ceph-mon[31361]: pgmap v256: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-22T13:32:26.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:25 smithi012 ceph-mon[28265]: pgmap v256: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-22T13:32:26.577 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:32:26.577 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:32:26.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.699+0000 7fc85792e640 1 -- 172.21.15.12:0/191238245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc850106170 msgr2=0x7fc85010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.699+0000 7fc85792e640 1 --2- 172.21.15.12:0/191238245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc850106170 0x7fc85010dd40 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fc848009ca0 tx=0x7fc84802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.699+0000 7fc85792e640 1 -- 172.21.15.12:0/191238245 shutdown_connections 2024-04-22T13:32:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.699+0000 7fc85792e640 1 --2- 172.21.15.12:0/191238245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc850106170 0x7fc85010dd40 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.699+0000 7fc85792e640 1 --2- 172.21.15.12:0/191238245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc850101a50 0x7fc850105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.699+0000 7fc85792e640 1 --2- 172.21.15.12:0/191238245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8501010a0 0x7fc850101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.699+0000 7fc85792e640 1 -- 172.21.15.12:0/191238245 >> 172.21.15.12:0/191238245 conn(0x7fc850077f70 msgr2=0x7fc850078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.699+0000 7fc85792e640 1 -- 172.21.15.12:0/191238245 shutdown_connections 2024-04-22T13:32:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.699+0000 7fc85792e640 1 -- 172.21.15.12:0/191238245 wait complete. 2024-04-22T13:32:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.699+0000 7fc85792e640 1 Processor -- start 2024-04-22T13:32:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.699+0000 7fc85792e640 1 -- start start 2024-04-22T13:32:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.700+0000 7fc85792e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8501010a0 0x7fc850107d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.700+0000 7fc85792e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc850106170 0x7fc8501082a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:26.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.700+0000 7fc85792e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8501087e0 0x7fc8501aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:26.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.700+0000 7fc85792e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc850111150 con 0x7fc850106170 2024-04-22T13:32:26.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.700+0000 7fc85792e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc850110fd0 con 0x7fc8501010a0 2024-04-22T13:32:26.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.700+0000 7fc85792e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc8501112d0 con 0x7fc8501087e0 2024-04-22T13:32:26.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.700+0000 7fc854ec9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc850106170 0x7fc8501082a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:26.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.700+0000 7fc854ec9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc850106170 0x7fc8501082a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33776/0 (socket says 172.21.15.12:33776) 2024-04-22T13:32:26.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.700+0000 7fc854ec9640 1 -- 172.21.15.12:0/2471245139 learned_addr learned my addr 172.21.15.12:0/2471245139 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:26.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.700+0000 7fc8556ca640 1 --2- 172.21.15.12:0/2471245139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8501010a0 0x7fc850107d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:26.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.700+0000 7fc855ecb640 1 --2- 172.21.15.12:0/2471245139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8501087e0 0x7fc8501aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:26.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.700+0000 7fc854ec9640 1 -- 172.21.15.12:0/2471245139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8501087e0 msgr2=0x7fc8501aaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:26.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.700+0000 7fc854ec9640 1 --2- 172.21.15.12:0/2471245139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8501087e0 0x7fc8501aaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.700+0000 7fc854ec9640 1 -- 172.21.15.12:0/2471245139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8501010a0 msgr2=0x7fc850107d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:26.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.700+0000 7fc854ec9640 1 --2- 172.21.15.12:0/2471245139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8501010a0 0x7fc850107d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.700+0000 7fc854ec9640 1 -- 172.21.15.12:0/2471245139 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8501ab410 con 0x7fc850106170 2024-04-22T13:32:26.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.700+0000 7fc854ec9640 1 --2- 172.21.15.12:0/2471245139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc850106170 0x7fc8501082a0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fc84000d740 tx=0x7fc84000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:26.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.700+0000 7fc8467fc640 1 -- 172.21.15.12:0/2471245139 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840017070 con 0x7fc850106170 2024-04-22T13:32:26.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.701+0000 7fc8467fc640 1 -- 172.21.15.12:0/2471245139 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc84000de50 con 0x7fc850106170 2024-04-22T13:32:26.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.701+0000 7fc85792e640 1 -- 172.21.15.12:0/2471245139 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8501ab700 con 0x7fc850106170 2024-04-22T13:32:26.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.701+0000 7fc8467fc640 1 -- 172.21.15.12:0/2471245139 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840004ce0 con 0x7fc850106170 2024-04-22T13:32:26.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.701+0000 7fc85792e640 1 -- 172.21.15.12:0/2471245139 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8501abbd0 con 0x7fc850106170 2024-04-22T13:32:26.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.703+0000 7fc8467fc640 1 -- 172.21.15.12:0/2471245139 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc840004e80 con 0x7fc850106170 2024-04-22T13:32:26.708 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.703+0000 7fc85a5f3740 1 Processor -- start 2024-04-22T13:32:26.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.703+0000 7fc85a5f3740 1 -- start start 2024-04-22T13:32:26.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.703+0000 7fc85a5f3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564da70dd580 0x564da70dd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:26.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.703+0000 7fc85a5f3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564da70ddea0 0x564da70e6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:26.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.703+0000 7fc85a5f3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564da70e6a80 0x564da70e8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:26.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.703+0000 7fc85a5f3740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564da70421e0 con 0x564da70e6a80 2024-04-22T13:32:26.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.703+0000 7fc85a5f3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564da6fdf390 con 0x564da70dd580 2024-04-22T13:32:26.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.703+0000 7fc85a5f3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564da703e1a0 con 0x564da70ddea0 2024-04-22T13:32:26.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.704+0000 7fc8467fc640 1 --2- 172.21.15.12:0/2471245139 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc83c0a3d20 0x7fc83c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:26.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.704+0000 7fc854ec9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564da70e6a80 0x564da70e8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:26.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.704+0000 7fc854ec9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564da70e6a80 0x564da70e8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33792/0 (socket says 172.21.15.12:33792) 2024-04-22T13:32:26.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.704+0000 7fc855ecb640 1 --2- 172.21.15.12:0/2471245139 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc83c0a3d20 0x7fc83c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:26.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.704+0000 7fc854ec9640 1 -- 172.21.15.12:0/3287628022 learned_addr learned my addr 172.21.15.12:0/3287628022 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:26.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.704+0000 7fc8556ca640 1 --2- 172.21.15.12:0/3287628022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564da70ddea0 0x564da70e6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.704+0000 7fc855ecb640 1 --2- 172.21.15.12:0/3287628022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564da70dd580 0x564da70dd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.704+0000 7fc8467fc640 1 -- 172.21.15.12:0/2471245139 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7fc8400d0240 con 0x7fc850106170 2024-04-22T13:32:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.704+0000 7fc854ec9640 1 -- 172.21.15.12:0/3287628022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564da70ddea0 msgr2=0x564da70e6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.704+0000 7fc854ec9640 1 --2- 172.21.15.12:0/3287628022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564da70ddea0 0x564da70e6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.704+0000 7fc854ec9640 1 -- 172.21.15.12:0/3287628022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564da70dd580 msgr2=0x564da70dd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.704+0000 7fc854ec9640 1 --2- 172.21.15.12:0/3287628022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564da70dd580 0x564da70dd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.704+0000 7fc854ec9640 1 -- 172.21.15.12:0/3287628022 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564da6e1e7d0 con 0x564da70e6a80 2024-04-22T13:32:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.704+0000 7fc855ecb640 1 --2- 172.21.15.12:0/2471245139 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc83c0a3d20 0x7fc83c0a61e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc848009430 tx=0x7fc848032340 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.704+0000 7fc854ec9640 1 --2- 172.21.15.12:0/3287628022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564da70e6a80 0x564da70e8e70 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fc840098bd0 tx=0x7fc840004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58157f38768413 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.704+0000 7fc83b7fe640 1 -- 172.21.15.12:0/3287628022 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840020070 con 0x564da70e6a80 2024-04-22T13:32:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.704+0000 7fc83b7fe640 1 -- 172.21.15.12:0/3287628022 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8400a05b0 con 0x564da70e6a80 2024-04-22T13:32:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.705+0000 7fc83b7fe640 1 -- 172.21.15.12:0/3287628022 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84009b8b0 con 0x564da70e6a80 2024-04-22T13:32:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.705+0000 7fc85a5f3740 1 -- 172.21.15.12:0/3287628022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564da70e6a80 msgr2=0x564da70e8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:26.710 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.705+0000 7fc85a5f3740 1 --2- 172.21.15.12:0/3287628022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564da70e6a80 0x564da70e8e70 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fc840098bd0 tx=0x7fc840004790 comp rx=0 tx=0).stop 2024-04-22T13:32:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.705+0000 7fc85a5f3740 1 -- 172.21.15.12:0/3287628022 shutdown_connections 2024-04-22T13:32:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.705+0000 7fc85a5f3740 1 --2- 172.21.15.12:0/3287628022 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564da70e6a80 0x564da70e8e70 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.705+0000 7fc85a5f3740 1 --2- 172.21.15.12:0/3287628022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564da70ddea0 0x564da70e6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.705+0000 7fc85a5f3740 1 --2- 172.21.15.12:0/3287628022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564da70dd580 0x564da70dd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.705+0000 7fc85a5f3740 1 -- 172.21.15.12:0/3287628022 >> 172.21.15.12:0/3287628022 conn(0x564da70d7cb0 msgr2=0x564da70d8090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.705+0000 7fc85a5f3740 1 -- 172.21.15.12:0/3287628022 shutdown_connections 2024-04-22T13:32:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.705+0000 7fc85a5f3740 1 -- 172.21.15.12:0/3287628022 wait complete. 2024-04-22T13:32:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.706+0000 7fc85a5f3740 1 -- start start 2024-04-22T13:32:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.706+0000 7fc85a5f3740 1 Processor -- start 2024-04-22T13:32:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.706+0000 7fc85a5f3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564da70dd580 0x564da70e3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.706+0000 7fc85a5f3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564da70ddea0 0x564da70e3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.706+0000 7fc85a5f3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564da70e6a80 0x564da70e4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.706+0000 7fc85a5f3740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x564da70e97b0 con 0x564da70e6a80 2024-04-22T13:32:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.706+0000 7fc85a5f3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564da6ffb2f0 con 0x564da70ddea0 2024-04-22T13:32:26.711 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.706+0000 7fc85a5f3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564da70e9930 con 0x564da70dd580 2024-04-22T13:32:26.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.706+0000 7fc85a5f3740 10 client.0 _ll_register_callbacks cb 0x564da7008580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:26.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.706+0000 7fc855ecb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564da70e6a80 0x564da70e4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:26.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.706+0000 7fc855ecb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564da70e6a80 0x564da70e4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33794/0 (socket says 172.21.15.12:33794) 2024-04-22T13:32:26.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.706+0000 7fc855ecb640 1 -- 172.21.15.12:0/2274903916 learned_addr learned my addr 172.21.15.12:0/2274903916 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:26.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.706+0000 7fc8556ca640 1 --2- 172.21.15.12:0/2274903916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564da70dd580 0x564da70e3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:26.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.707+0000 7fc854ec9640 1 --2- 172.21.15.12:0/2274903916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564da70ddea0 0x564da70e3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:26.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.707+0000 7fc855ecb640 1 -- 172.21.15.12:0/2274903916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564da70dd580 msgr2=0x564da70e3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:26.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.707+0000 7fc855ecb640 1 --2- 172.21.15.12:0/2274903916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564da70dd580 0x564da70e3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.707+0000 7fc855ecb640 1 -- 172.21.15.12:0/2274903916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564da70ddea0 msgr2=0x564da70e3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:26.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.707+0000 7fc855ecb640 1 --2- 172.21.15.12:0/2274903916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564da70ddea0 0x564da70e3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.707+0000 7fc855ecb640 1 -- 172.21.15.12:0/2274903916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564da6d543f0 con 0x564da70e6a80 2024-04-22T13:32:26.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.707+0000 7fc855ecb640 1 --2- 172.21.15.12:0/2274903916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564da70e6a80 0x564da70e4500 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fc848036000 tx=0x7fc848044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15720a8d0e5d8b68 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:26.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.707+0000 7fc838ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:32:26.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.707+0000 7fc838ff9640 1 -- 172.21.15.12:0/2274903916 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848049040 con 0x564da70e6a80 2024-04-22T13:32:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.707+0000 7fc838ff9640 1 -- 172.21.15.12:0/2274903916 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7fc848003c30 con 0x564da70e6a80 2024-04-22T13:32:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.707+0000 7fc85a5f3740 1 -- 172.21.15.12:0/2274903916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8400a05b0 con 0x564da70e6a80 2024-04-22T13:32:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.707+0000 7fc838ff9640 1 -- 172.21.15.12:0/2274903916 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc84805b050 con 0x564da70e6a80 2024-04-22T13:32:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.707+0000 7fc85a5f3740 10 client.18501 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.707+0000 7fc85a5f3740 1 -- 172.21.15.12:0/2274903916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564da6d140f0 con 0x564da70e6a80 2024-04-22T13:32:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.707+0000 7fc838ff9640 1 -- 172.21.15.12:0/2274903916 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8400a05b0 con 0x564da70e6a80 2024-04-22T13:32:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.708+0000 7fc838ff9640 1 -- 172.21.15.12:0/2274903916 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc848034540 con 0x564da70e6a80 2024-04-22T13:32:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.708+0000 7fc85a5f3740 10 client.18501 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.708+0000 7fc85a5f3740 10 client.18501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.708+0000 7fc85a5f3740 20 client.18501 populate_metadata read hostname 'smithi012' 2024-04-22T13:32:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.708+0000 7fc85a5f3740 1 --2- 172.21.15.12:0/2274903916 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x564da70f69c0 0x564da70f8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.708+0000 7fc85a5f3740 4 client.18501 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.708+0000 7fc85a5f3740 1 -- 172.21.15.12:0/2274903916 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564da704fc00 con 0x564da70f69c0 2024-04-22T13:32:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.708+0000 7fc8556ca640 1 --2- 172.21.15.12:0/2274903916 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x564da70f69c0 0x564da70f8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:26.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.708+0000 7fc8556ca640 1 --2- 172.21.15.12:0/2274903916 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x564da70f69c0 0x564da70f8df0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2b3d12c6c07a4fb server_cookie=7b75e595049162f4 in_seq=0 out_seq=0 2024-04-22T13:32:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.708+0000 7fc838ff9640 10 client.18501 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:32:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.709+0000 7fc838ff9640 1 -- 172.21.15.12:0/2274903916 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564da704fc00 con 0x564da70f69c0 2024-04-22T13:32:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.709+0000 7fc838ff9640 10 client.18501 handle_command_reply: tid=0 2024-04-22T13:32:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.709+0000 7fc85a5f3740 1 client.18501 shutdown 2024-04-22T13:32:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.709+0000 7fc85a5f3740 1 -- 172.21.15.12:0/2274903916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564da70e6a80 msgr2=0x564da70e4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.709+0000 7fc85a5f3740 1 --2- 172.21.15.12:0/2274903916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564da70e6a80 0x564da70e4500 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fc848036000 tx=0x7fc848044000 comp rx=0 tx=0).stop 2024-04-22T13:32:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.709+0000 7fc85a5f3740 1 -- 172.21.15.12:0/2274903916 shutdown_connections 2024-04-22T13:32:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.709+0000 7fc85a5f3740 1 --2- 172.21.15.12:0/2274903916 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x564da70f69c0 0x564da70f8df0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.709+0000 7fc85a5f3740 1 --2- 172.21.15.12:0/2274903916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564da70e6a80 0x564da70e4500 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.709+0000 7fc85a5f3740 1 --2- 172.21.15.12:0/2274903916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564da70ddea0 0x564da70e3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.709+0000 7fc838ff9640 0 client.18501 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:32:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.709+0000 7fc85a5f3740 1 --2- 172.21.15.12:0/2274903916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564da70dd580 0x564da70e3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.709+0000 7fc85a5f3740 1 -- 172.21.15.12:0/2274903916 >> 172.21.15.12:0/2274903916 conn(0x564da70d7cb0 msgr2=0x564da70d9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:26.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.709+0000 7fc85a5f3740 1 -- 172.21.15.12:0/2274903916 shutdown_connections 2024-04-22T13:32:26.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.709+0000 7fc85a5f3740 1 -- 172.21.15.12:0/2274903916 wait complete. 2024-04-22T13:32:26.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.709+0000 7fc85a5f3740 20 client.18501 trim_cache size 0 max 16384 2024-04-22T13:32:26.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.727+0000 7fc85a5f3740 1 Processor -- start 2024-04-22T13:32:26.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.727+0000 7fc85a5f3740 1 -- start start 2024-04-22T13:32:26.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.728+0000 7fc85a5f3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564da70dd580 0x564da70e61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:26.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.728+0000 7fc85a5f3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564da70ddea0 0x564da70e29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:26.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.728+0000 7fc85a5f3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564da70e6a80 0x564da70f2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:26.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.728+0000 7fc85a5f3740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc8501b4190 con 0x564da70e6a80 2024-04-22T13:32:26.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.728+0000 7fc85a5f3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc8501b5cd0 con 0x564da70ddea0 2024-04-22T13:32:26.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.728+0000 7fc85a5f3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc8501b4860 con 0x564da70dd580 2024-04-22T13:32:26.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.728+0000 7fc855ecb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564da70e6a80 0x564da70f2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:26.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.728+0000 7fc855ecb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564da70e6a80 0x564da70f2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33796/0 (socket says 172.21.15.12:33796) 2024-04-22T13:32:26.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.728+0000 7fc855ecb640 1 -- 172.21.15.12:0/3985449591 learned_addr learned my addr 172.21.15.12:0/3985449591 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:26.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.728+0000 7fc8556ca640 1 --2- 172.21.15.12:0/3985449591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564da70dd580 0x564da70e61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:26.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.728+0000 7fc855ecb640 1 -- 172.21.15.12:0/3985449591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564da70dd580 msgr2=0x564da70e61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:26.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.728+0000 7fc855ecb640 1 --2- 172.21.15.12:0/3985449591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564da70dd580 0x564da70e61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.728+0000 7fc855ecb640 1 -- 172.21.15.12:0/3985449591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564da70ddea0 msgr2=0x564da70e29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:26.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.728+0000 7fc854ec9640 1 --2- 172.21.15.12:0/3985449591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564da70ddea0 0x564da70e29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:26.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.728+0000 7fc855ecb640 1 --2- 172.21.15.12:0/3985449591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564da70ddea0 0x564da70e29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.728+0000 7fc855ecb640 1 -- 172.21.15.12:0/3985449591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564da70e8e70 con 0x564da70e6a80 2024-04-22T13:32:26.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.728+0000 7fc855ecb640 1 --2- 172.21.15.12:0/3985449591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564da70e6a80 0x564da70f2670 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fc848034960 tx=0x7fc848032f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a7f0ab0ab57361 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:26.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.728+0000 7fc83affd640 1 -- 172.21.15.12:0/3985449591 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848063070 con 0x564da70e6a80 2024-04-22T13:32:26.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.729+0000 7fc83affd640 1 -- 172.21.15.12:0/3985449591 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc848052070 con 0x564da70e6a80 2024-04-22T13:32:26.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.729+0000 7fc83affd640 1 -- 172.21.15.12:0/3985449591 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848049040 con 0x564da70e6a80 2024-04-22T13:32:26.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.729+0000 7fc85a5f3740 1 -- 172.21.15.12:0/3985449591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564da70e6a80 msgr2=0x564da70f2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:26.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.729+0000 7fc85a5f3740 1 --2- 172.21.15.12:0/3985449591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564da70e6a80 0x564da70f2670 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fc848034960 tx=0x7fc848032f80 comp rx=0 tx=0).stop 2024-04-22T13:32:26.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.729+0000 7fc85a5f3740 1 -- 172.21.15.12:0/3985449591 shutdown_connections 2024-04-22T13:32:26.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.729+0000 7fc85a5f3740 1 --2- 172.21.15.12:0/3985449591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564da70e6a80 0x564da70f2670 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.729+0000 7fc85a5f3740 1 --2- 172.21.15.12:0/3985449591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564da70ddea0 0x564da70e29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.729+0000 7fc85a5f3740 1 --2- 172.21.15.12:0/3985449591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564da70dd580 0x564da70e61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.729+0000 7fc85a5f3740 1 -- 172.21.15.12:0/3985449591 >> 172.21.15.12:0/3985449591 conn(0x564da70d7cb0 msgr2=0x564da70d6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:26.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.729+0000 7fc85a5f3740 1 -- 172.21.15.12:0/3985449591 shutdown_connections 2024-04-22T13:32:26.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.729+0000 7fc85a5f3740 1 -- 172.21.15.12:0/3985449591 wait complete. 2024-04-22T13:32:26.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.729+0000 7fc85a5f3740 1 -- start start 2024-04-22T13:32:26.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.730+0000 7fc85a5f3740 1 Processor -- start 2024-04-22T13:32:26.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.730+0000 7fc85a5f3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564da70dd580 0x564da70eeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:26.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.730+0000 7fc85a5f3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564da70ddea0 0x564da70ef0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:26.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.730+0000 7fc85a5f3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564da70e6a80 0x564da70ebb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:26.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.730+0000 7fc85a5f3740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc8501b5600 con 0x564da70ddea0 2024-04-22T13:32:26.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.730+0000 7fc85a5f3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc8501b4f30 con 0x564da70dd580 2024-04-22T13:32:26.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.730+0000 7fc85a5f3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc8501b2190 con 0x564da70e6a80 2024-04-22T13:32:26.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.730+0000 7fc85a5f3740 10 client.0 _ll_register_callbacks cb 0x7fc848046040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:26.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.730+0000 7fc854ec9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564da70ddea0 0x564da70ef0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:26.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.730+0000 7fc854ec9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564da70ddea0 0x564da70ef0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33812/0 (socket says 172.21.15.12:33812) 2024-04-22T13:32:26.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.730+0000 7fc854ec9640 1 -- 172.21.15.12:0/2253403713 learned_addr learned my addr 172.21.15.12:0/2253403713 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:26.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.730+0000 7fc855ecb640 1 --2- 172.21.15.12:0/2253403713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564da70e6a80 0x564da70ebb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:26.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.730+0000 7fc8556ca640 1 --2- 172.21.15.12:0/2253403713 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564da70dd580 0x564da70eeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:26.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.730+0000 7fc854ec9640 1 -- 172.21.15.12:0/2253403713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564da70e6a80 msgr2=0x564da70ebb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:26.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.730+0000 7fc854ec9640 1 --2- 172.21.15.12:0/2253403713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564da70e6a80 0x564da70ebb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.730+0000 7fc854ec9640 1 -- 172.21.15.12:0/2253403713 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564da70dd580 msgr2=0x564da70eeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:26.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.730+0000 7fc854ec9640 1 --2- 172.21.15.12:0/2253403713 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564da70dd580 0x564da70eeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.730+0000 7fc854ec9640 1 -- 172.21.15.12:0/2253403713 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564da70ec090 con 0x564da70ddea0 2024-04-22T13:32:26.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.731+0000 7fc854ec9640 1 --2- 172.21.15.12:0/2253403713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564da70ddea0 0x564da70ef0c0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fc8400d05f0 tx=0x7fc8400cdee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d4067f409d09a3f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:26.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.731+0000 7fc838ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:32:26.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.731+0000 7fc838ff9640 1 -- 172.21.15.12:0/2253403713 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840020040 con 0x564da70ddea0 2024-04-22T13:32:26.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.731+0000 7fc838ff9640 1 -- 172.21.15.12:0/2253403713 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7fc84009a500 con 0x564da70ddea0 2024-04-22T13:32:26.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.731+0000 7fc85a5f3740 1 -- 172.21.15.12:0/2253403713 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc848052070 con 0x564da70ddea0 2024-04-22T13:32:26.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.731+0000 7fc838ff9640 1 -- 172.21.15.12:0/2253403713 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc8400a05b0 con 0x564da70ddea0 2024-04-22T13:32:26.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.731+0000 7fc85a5f3740 10 client.18507 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:26.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.731+0000 7fc85a5f3740 1 -- 172.21.15.12:0/2253403713 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564da715d790 con 0x564da70ddea0 2024-04-22T13:32:26.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.731+0000 7fc838ff9640 1 -- 172.21.15.12:0/2253403713 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc848052070 con 0x564da70ddea0 2024-04-22T13:32:26.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.731+0000 7fc838ff9640 1 -- 172.21.15.12:0/2253403713 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc840098150 con 0x564da70ddea0 2024-04-22T13:32:26.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.731+0000 7fc85a5f3740 10 client.18507 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:26.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.731+0000 7fc85a5f3740 10 client.18507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:26.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.731+0000 7fc85a5f3740 20 client.18507 populate_metadata read hostname 'smithi012' 2024-04-22T13:32:26.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.731+0000 7fc85a5f3740 1 --2- 172.21.15.12:0/2253403713 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x564da715dad0 0x564da715fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:26.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.731+0000 7fc85a5f3740 4 client.18507 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:32:26.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.731+0000 7fc85a5f3740 1 -- 172.21.15.12:0/2253403713 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564da7049a70 con 0x564da715dad0 2024-04-22T13:32:26.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.732+0000 7fc8556ca640 1 --2- 172.21.15.12:0/2253403713 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x564da715dad0 0x564da715fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:26.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.732+0000 7fc838ff9640 10 client.18507 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:32:26.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.732+0000 7fc8556ca640 1 --2- 172.21.15.12:0/2253403713 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x564da715dad0 0x564da715fec0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4914b70eb9e1c9b server_cookie=a9c77cdea892af28 in_seq=0 out_seq=0 2024-04-22T13:32:26.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.732+0000 7fc838ff9640 1 -- 172.21.15.12:0/2253403713 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564da7049a70 con 0x564da715dad0 2024-04-22T13:32:26.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.732+0000 7fc838ff9640 10 client.18507 handle_command_reply: tid=0 2024-04-22T13:32:26.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.732+0000 7fc85a5f3740 1 client.18507 shutdown 2024-04-22T13:32:26.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.732+0000 7fc85a5f3740 1 -- 172.21.15.12:0/2253403713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564da70ddea0 msgr2=0x564da70ef0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:26.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.732+0000 7fc85a5f3740 1 --2- 172.21.15.12:0/2253403713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564da70ddea0 0x564da70ef0c0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fc8400d05f0 tx=0x7fc8400cdee0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.732+0000 7fc85a5f3740 1 -- 172.21.15.12:0/2253403713 shutdown_connections 2024-04-22T13:32:26.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.732+0000 7fc85a5f3740 1 --2- 172.21.15.12:0/2253403713 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x564da715dad0 0x564da715fec0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.732+0000 7fc85a5f3740 1 --2- 172.21.15.12:0/2253403713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564da70e6a80 0x564da70ebb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.732+0000 7fc85a5f3740 1 --2- 172.21.15.12:0/2253403713 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564da70ddea0 0x564da70ef0c0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.732+0000 7fc85a5f3740 1 --2- 172.21.15.12:0/2253403713 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564da70dd580 0x564da70eeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.732+0000 7fc838ff9640 0 client.18507 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:32:26.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.732+0000 7fc85a5f3740 1 -- 172.21.15.12:0/2253403713 >> 172.21.15.12:0/2253403713 conn(0x564da70d7cb0 msgr2=0x564da70d64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:26.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.732+0000 7fc85a5f3740 1 -- 172.21.15.12:0/2253403713 shutdown_connections 2024-04-22T13:32:26.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.732+0000 7fc85a5f3740 1 -- 172.21.15.12:0/2253403713 wait complete. 2024-04-22T13:32:26.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.733+0000 7fc85a5f3740 20 client.18507 trim_cache size 0 max 16384 2024-04-22T13:32:26.738 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:32:26.738 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-04-22T13:32:26.738 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-04-22T13:32:26.738 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:32:26.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.733+0000 7fc85792e640 1 -- 172.21.15.12:0/2471245139 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc83c0a3d20 msgr2=0x7fc83c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:26.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.733+0000 7fc85792e640 1 --2- 172.21.15.12:0/2471245139 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc83c0a3d20 0x7fc83c0a61e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc848009430 tx=0x7fc848032340 comp rx=0 tx=0).stop 2024-04-22T13:32:26.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.733+0000 7fc85792e640 1 -- 172.21.15.12:0/2471245139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc850106170 msgr2=0x7fc8501082a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:26.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.733+0000 7fc85792e640 1 --2- 172.21.15.12:0/2471245139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc850106170 0x7fc8501082a0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fc84000d740 tx=0x7fc84000dc00 comp rx=0 tx=0).stop 2024-04-22T13:32:26.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.733+0000 7fc85792e640 1 -- 172.21.15.12:0/2471245139 shutdown_connections 2024-04-22T13:32:26.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.733+0000 7fc85792e640 1 --2- 172.21.15.12:0/2471245139 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc83c0a3d20 0x7fc83c0a61e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.733+0000 7fc85792e640 1 --2- 172.21.15.12:0/2471245139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc8501087e0 0x7fc8501aaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.733+0000 7fc85792e640 1 --2- 172.21.15.12:0/2471245139 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc850106170 0x7fc8501082a0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.733+0000 7fc85792e640 1 --2- 172.21.15.12:0/2471245139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8501010a0 0x7fc850107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.733+0000 7fc85792e640 1 -- 172.21.15.12:0/2471245139 >> 172.21.15.12:0/2471245139 conn(0x7fc850077f70 msgr2=0x7fc8500ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:26.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.733+0000 7fc85792e640 1 -- 172.21.15.12:0/2471245139 shutdown_connections 2024-04-22T13:32:26.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.733+0000 7fc85792e640 1 -- 172.21.15.12:0/2471245139 wait complete. 2024-04-22T13:32:26.741 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:32:26.741 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:32:26.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.846+0000 7f4276443640 1 -- 172.21.15.12:0/3565781693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f427006a8b0 msgr2=0x7f427006ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:26.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.846+0000 7f4276443640 1 --2- 172.21.15.12:0/3565781693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f427006a8b0 0x7f427006ac90 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f4258009900 tx=0x7f425802bc30 comp rx=0 tx=0).stop 2024-04-22T13:32:26.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.846+0000 7f4276443640 1 -- 172.21.15.12:0/3565781693 shutdown_connections 2024-04-22T13:32:26.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.846+0000 7f4276443640 1 --2- 172.21.15.12:0/3565781693 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42701072c0 0x7f427010ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.846+0000 7f4276443640 1 --2- 172.21.15.12:0/3565781693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f427006b260 0x7f4270106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.846+0000 7f4276443640 1 --2- 172.21.15.12:0/3565781693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f427006a8b0 0x7f427006ac90 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.846+0000 7f4276443640 1 -- 172.21.15.12:0/3565781693 >> 172.21.15.12:0/3565781693 conn(0x7f4270077e30 msgr2=0x7f4270078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:26.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.847+0000 7f4276443640 1 -- 172.21.15.12:0/3565781693 shutdown_connections 2024-04-22T13:32:26.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.847+0000 7f4276443640 1 -- 172.21.15.12:0/3565781693 wait complete. 2024-04-22T13:32:26.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.847+0000 7f4276443640 1 Processor -- start 2024-04-22T13:32:26.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.847+0000 7f4276443640 1 -- start start 2024-04-22T13:32:26.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.847+0000 7f4276443640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f427006a8b0 0x7f4270198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:26.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.847+0000 7f4276443640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42701072c0 0x7f4270198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:26.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.847+0000 7f4276443640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f427019cfd0 0x7f427019d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:26.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.847+0000 7f4276443640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f42701121d0 con 0x7f42701072c0 2024-04-22T13:32:26.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.847+0000 7f4276443640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4270112050 con 0x7f427006a8b0 2024-04-22T13:32:26.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.847+0000 7f4276443640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4270112350 con 0x7f427019cfd0 2024-04-22T13:32:26.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.848+0000 7f426f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42701072c0 0x7f4270198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:26.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.848+0000 7f426f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42701072c0 0x7f4270198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33836/0 (socket says 172.21.15.12:33836) 2024-04-22T13:32:26.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.848+0000 7f426f7fe640 1 -- 172.21.15.12:0/3660938887 learned_addr learned my addr 172.21.15.12:0/3660938887 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:26.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.848+0000 7f42749e0640 1 --2- 172.21.15.12:0/3660938887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f427019cfd0 0x7f427019d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:26.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.848+0000 7f426ffff640 1 --2- 172.21.15.12:0/3660938887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f427006a8b0 0x7f4270198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:26.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.848+0000 7f426f7fe640 1 -- 172.21.15.12:0/3660938887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f427019cfd0 msgr2=0x7f427019d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:26.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.848+0000 7f426f7fe640 1 --2- 172.21.15.12:0/3660938887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f427019cfd0 0x7f427019d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.848+0000 7f426f7fe640 1 -- 172.21.15.12:0/3660938887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f427006a8b0 msgr2=0x7f4270198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:26.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.848+0000 7f426f7fe640 1 --2- 172.21.15.12:0/3660938887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f427006a8b0 0x7f4270198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:26.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.848+0000 7f426f7fe640 1 -- 172.21.15.12:0/3660938887 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427019dac0 con 0x7f42701072c0 2024-04-22T13:32:26.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.848+0000 7f426ffff640 1 --2- 172.21.15.12:0/3660938887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f427006a8b0 0x7f4270198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:26.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.848+0000 7f426f7fe640 1 --2- 172.21.15.12:0/3660938887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42701072c0 0x7f4270198c50 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f426000e9f0 tx=0x7f426000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:26.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.848+0000 7f426d7fa640 1 -- 172.21.15.12:0/3660938887 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426000ccc0 con 0x7f42701072c0 2024-04-22T13:32:26.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.848+0000 7f426d7fa640 1 -- 172.21.15.12:0/3660938887 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42600108e0 con 0x7f42701072c0 2024-04-22T13:32:26.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.848+0000 7f426d7fa640 1 -- 172.21.15.12:0/3660938887 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4260018860 con 0x7f42701072c0 2024-04-22T13:32:26.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.848+0000 7f4276443640 1 -- 172.21.15.12:0/3660938887 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f427019ddb0 con 0x7f42701072c0 2024-04-22T13:32:26.848 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.848+0000 7f4276443640 1 -- 172.21.15.12:0/3660938887 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42701a5550 con 0x7f42701072c0 2024-04-22T13:32:26.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.849+0000 7f4276443640 1 -- 172.21.15.12:0/3660938887 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4270069230 con 0x7f42701072c0 2024-04-22T13:32:26.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.850+0000 7f426d7fa640 1 -- 172.21.15.12:0/3660938887 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4260010430 con 0x7f42701072c0 2024-04-22T13:32:26.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.851+0000 7f426d7fa640 1 --2- 172.21.15.12:0/3660938887 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f425c0a3d20 0x7f425c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:26.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.851+0000 7f426ffff640 1 --2- 172.21.15.12:0/3660938887 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f425c0a3d20 0x7f425c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:26.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.851+0000 7f426d7fa640 1 -- 172.21.15.12:0/3660938887 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f42600d1050 con 0x7f42701072c0 2024-04-22T13:32:26.849 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.852+0000 7f426ffff640 1 --2- 172.21.15.12:0/3660938887 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f425c0a3d20 0x7f425c0a61e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4258036040 tx=0x7f4258000bc0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:26.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:26.855+0000 7f426d7fa640 1 -- 172.21.15.12:0/3660938887 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f426009d6b0 con 0x7f42701072c0 2024-04-22T13:32:27.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.079+0000 7f4276443640 1 -- 172.21.15.12:0/3660938887 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4270199a60 con 0x7f42701072c0 2024-04-22T13:32:27.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.080+0000 7f426d7fa640 1 -- 172.21.15.12:0/3660938887 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7f426009cfb0 con 0x7f42701072c0 2024-04-22T13:32:27.076 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:32:27.077 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:32:27.077 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:32:27.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.082+0000 7f4276443640 1 -- 172.21.15.12:0/3660938887 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f425c0a3d20 msgr2=0x7f425c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.082+0000 7f4276443640 1 --2- 172.21.15.12:0/3660938887 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f425c0a3d20 0x7f425c0a61e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4258036040 tx=0x7f4258000bc0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.082+0000 7f4276443640 1 -- 172.21.15.12:0/3660938887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42701072c0 msgr2=0x7f4270198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.082+0000 7f4276443640 1 --2- 172.21.15.12:0/3660938887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42701072c0 0x7f4270198c50 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f426000e9f0 tx=0x7f426000eeb0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.082+0000 7f4276443640 1 -- 172.21.15.12:0/3660938887 shutdown_connections 2024-04-22T13:32:27.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.082+0000 7f4276443640 1 --2- 172.21.15.12:0/3660938887 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f425c0a3d20 0x7f425c0a61e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.082+0000 7f4276443640 1 --2- 172.21.15.12:0/3660938887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f427019cfd0 0x7f427019d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.083+0000 7f4276443640 1 --2- 172.21.15.12:0/3660938887 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f42701072c0 0x7f4270198c50 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.083+0000 7f4276443640 1 --2- 172.21.15.12:0/3660938887 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f427006a8b0 0x7f4270198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.083+0000 7f4276443640 1 -- 172.21.15.12:0/3660938887 >> 172.21.15.12:0/3660938887 conn(0x7f4270077e30 msgr2=0x7f42700ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:27.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.083+0000 7f4276443640 1 -- 172.21.15.12:0/3660938887 shutdown_connections 2024-04-22T13:32:27.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.083+0000 7f4276443640 1 -- 172.21.15.12:0/3660938887 wait complete. 2024-04-22T13:32:27.090 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:32:27.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.198+0000 7f3751681640 1 -- 172.21.15.12:0/2269905457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f374c106170 msgr2=0x7f374c10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.198+0000 7f3751681640 1 --2- 172.21.15.12:0/2269905457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f374c106170 0x7f374c10dd40 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f3744009ca0 tx=0x7f374402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.198+0000 7f3751681640 1 -- 172.21.15.12:0/2269905457 shutdown_connections 2024-04-22T13:32:27.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.198+0000 7f3751681640 1 --2- 172.21.15.12:0/2269905457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f374c106170 0x7f374c10dd40 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.198+0000 7f3751681640 1 --2- 172.21.15.12:0/2269905457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f374c101a50 0x7f374c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.198+0000 7f3751681640 1 --2- 172.21.15.12:0/2269905457 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f374c1010a0 0x7f374c101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.198+0000 7f3751681640 1 -- 172.21.15.12:0/2269905457 >> 172.21.15.12:0/2269905457 conn(0x7f374c077eb0 msgr2=0x7f374c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:27.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.198+0000 7f3751681640 1 -- 172.21.15.12:0/2269905457 shutdown_connections 2024-04-22T13:32:27.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.198+0000 7f3751681640 1 -- 172.21.15.12:0/2269905457 wait complete. 2024-04-22T13:32:27.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.199+0000 7f3751681640 1 Processor -- start 2024-04-22T13:32:27.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.199+0000 7f3751681640 1 -- start start 2024-04-22T13:32:27.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.199+0000 7f3751681640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f374c1010a0 0x7f374c102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.199+0000 7f3751681640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f374c106170 0x7f374c103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.199+0000 7f3751681640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f374c1035b0 0x7f374c1a67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.199+0000 7f3751681640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f374c1110c0 con 0x7f374c106170 2024-04-22T13:32:27.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.199+0000 7f3751681640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f374c110f40 con 0x7f374c1035b0 2024-04-22T13:32:27.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.199+0000 7f3751681640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f374c111240 con 0x7f374c1010a0 2024-04-22T13:32:27.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.199+0000 7f3743fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f374c106170 0x7f374c103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.199+0000 7f3743fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f374c106170 0x7f374c103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33846/0 (socket says 172.21.15.12:33846) 2024-04-22T13:32:27.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.199+0000 7f3743fff640 1 -- 172.21.15.12:0/3548196751 learned_addr learned my addr 172.21.15.12:0/3548196751 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:27.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.199+0000 7f374affd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f374c1010a0 0x7f374c102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.199+0000 7f374b7fe640 1 --2- 172.21.15.12:0/3548196751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f374c1035b0 0x7f374c1a67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.199+0000 7f3743fff640 1 -- 172.21.15.12:0/3548196751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f374c1010a0 msgr2=0x7f374c102b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.199+0000 7f3743fff640 1 --2- 172.21.15.12:0/3548196751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f374c1010a0 0x7f374c102b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.199+0000 7f3743fff640 1 -- 172.21.15.12:0/3548196751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f374c1035b0 msgr2=0x7f374c1a67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.199+0000 7f3743fff640 1 --2- 172.21.15.12:0/3548196751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f374c1035b0 0x7f374c1a67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.199+0000 7f3743fff640 1 -- 172.21.15.12:0/3548196751 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374c1a6ec0 con 0x7f374c106170 2024-04-22T13:32:27.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.200+0000 7f3743fff640 1 --2- 172.21.15.12:0/3548196751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f374c106170 0x7f374c103070 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f373400c970 tx=0x7f373400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:27.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.200+0000 7f3748ff9640 1 -- 172.21.15.12:0/3548196751 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3734007bb0 con 0x7f374c106170 2024-04-22T13:32:27.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.200+0000 7f3748ff9640 1 -- 172.21.15.12:0/3548196751 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3734007d50 con 0x7f374c106170 2024-04-22T13:32:27.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.200+0000 7f3748ff9640 1 -- 172.21.15.12:0/3548196751 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3734004dd0 con 0x7f374c106170 2024-04-22T13:32:27.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.200+0000 7f3751681640 1 -- 172.21.15.12:0/3548196751 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f374c1a71b0 con 0x7f374c106170 2024-04-22T13:32:27.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.200+0000 7f3751681640 1 -- 172.21.15.12:0/3548196751 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f374c1a76b0 con 0x7f374c106170 2024-04-22T13:32:27.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.202+0000 7f3748ff9640 1 -- 172.21.15.12:0/3548196751 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3734004fb0 con 0x7f374c106170 2024-04-22T13:32:27.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.202+0000 7f3754274740 1 Processor -- start 2024-04-22T13:32:27.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.202+0000 7f3754274740 1 -- start start 2024-04-22T13:32:27.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.203+0000 7f3754274740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588734874d0 0x5588734878b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.203+0000 7f3754274740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558873487e80 0x558873490430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.203+0000 7f3754274740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558873490a00 0x558873492e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.203+0000 7f3754274740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5588733ec1f0 con 0x558873490a00 2024-04-22T13:32:27.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.203+0000 7f3754274740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5588733894b0 con 0x558873487e80 2024-04-22T13:32:27.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.203+0000 7f3754274740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5588733e81b0 con 0x5588734874d0 2024-04-22T13:32:27.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.203+0000 7f3743fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558873490a00 0x558873492e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.203+0000 7f3743fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558873490a00 0x558873492e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33854/0 (socket says 172.21.15.12:33854) 2024-04-22T13:32:27.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.203+0000 7f3743fff640 1 -- 172.21.15.12:0/196301336 learned_addr learned my addr 172.21.15.12:0/196301336 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:27.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.203+0000 7f374b7fe640 1 --2- 172.21.15.12:0/196301336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588734874d0 0x5588734878b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.203+0000 7f3748ff9640 1 --2- 172.21.15.12:0/3548196751 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f37380a3d50 0x7f37380a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.203+0000 7f374affd640 1 --2- 172.21.15.12:0/196301336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558873487e80 0x558873490430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.203+0000 7f374b7fe640 1 --2- 172.21.15.12:0/3548196751 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f37380a3d50 0x7f37380a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.203+0000 7f3748ff9640 1 -- 172.21.15.12:0/3548196751 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f37340d0450 con 0x7f374c106170 2024-04-22T13:32:27.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.203+0000 7f3743fff640 1 -- 172.21.15.12:0/196301336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588734874d0 msgr2=0x5588734878b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.203+0000 7f3743fff640 1 --2- 172.21.15.12:0/196301336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588734874d0 0x5588734878b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.203+0000 7f3743fff640 1 -- 172.21.15.12:0/196301336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558873487e80 msgr2=0x558873490430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.203+0000 7f3743fff640 1 --2- 172.21.15.12:0/196301336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558873487e80 0x558873490430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.203+0000 7f3743fff640 1 -- 172.21.15.12:0/196301336 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55887337b1b0 con 0x558873490a00 2024-04-22T13:32:27.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.204+0000 7f3743fff640 1 --2- 172.21.15.12:0/196301336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558873490a00 0x558873492e50 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f37340cdad0 tx=0x7f3734000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9740b1f653c6fb72 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:27.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.204+0000 7f37417fa640 1 -- 172.21.15.12:0/196301336 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373401d050 con 0x558873490a00 2024-04-22T13:32:27.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.204+0000 7f374b7fe640 1 --2- 172.21.15.12:0/3548196751 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f37380a3d50 0x7f37380a6210 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f37440098d0 tx=0x7f374402c0f0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:27.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.204+0000 7f37417fa640 1 -- 172.21.15.12:0/196301336 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37340d3b10 con 0x558873490a00 2024-04-22T13:32:27.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.204+0000 7f37417fa640 1 -- 172.21.15.12:0/196301336 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3734098a20 con 0x558873490a00 2024-04-22T13:32:27.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.204+0000 7f3754274740 1 -- 172.21.15.12:0/196301336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558873490a00 msgr2=0x558873492e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.204+0000 7f3754274740 1 --2- 172.21.15.12:0/196301336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558873490a00 0x558873492e50 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f37340cdad0 tx=0x7f3734000bc0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.205+0000 7f3754274740 1 -- 172.21.15.12:0/196301336 shutdown_connections 2024-04-22T13:32:27.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.205+0000 7f3754274740 1 --2- 172.21.15.12:0/196301336 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558873490a00 0x558873492e50 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.205+0000 7f3754274740 1 --2- 172.21.15.12:0/196301336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558873487e80 0x558873490430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.205+0000 7f3754274740 1 --2- 172.21.15.12:0/196301336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5588734874d0 0x5588734878b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.205+0000 7f3754274740 1 -- 172.21.15.12:0/196301336 >> 172.21.15.12:0/196301336 conn(0x558873481ba0 msgr2=0x558873481f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:27.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.205+0000 7f3754274740 1 -- 172.21.15.12:0/196301336 shutdown_connections 2024-04-22T13:32:27.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.205+0000 7f3754274740 1 -- 172.21.15.12:0/196301336 wait complete. 2024-04-22T13:32:27.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.205+0000 7f3754274740 1 -- start start 2024-04-22T13:32:27.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.205+0000 7f3754274740 1 Processor -- start 2024-04-22T13:32:27.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.205+0000 7f3754274740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588734874d0 0x5588734902d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.205+0000 7f3754274740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558873487e80 0x55887348d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.205+0000 7f3754274740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558873490a00 0x55887348dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.205+0000 7f3754274740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5588734938b0 con 0x5588734874d0 2024-04-22T13:32:27.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.205+0000 7f3754274740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5588733a5500 con 0x558873487e80 2024-04-22T13:32:27.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.205+0000 7f3754274740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558873493a30 con 0x558873490a00 2024-04-22T13:32:27.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.205+0000 7f3754274740 10 client.0 _ll_register_callbacks cb 0x558873373b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:27.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.205+0000 7f374affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588734874d0 0x5588734902d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.205+0000 7f374affd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588734874d0 0x5588734902d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33858/0 (socket says 172.21.15.12:33858) 2024-04-22T13:32:27.209 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.205+0000 7f374affd640 1 -- 172.21.15.12:0/1191990682 learned_addr learned my addr 172.21.15.12:0/1191990682 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:27.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.206+0000 7f374b7fe640 1 --2- 172.21.15.12:0/1191990682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558873490a00 0x55887348dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.206+0000 7f3743fff640 1 --2- 172.21.15.12:0/1191990682 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558873487e80 0x55887348d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.206+0000 7f374affd640 1 -- 172.21.15.12:0/1191990682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558873490a00 msgr2=0x55887348dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.206+0000 7f374affd640 1 --2- 172.21.15.12:0/1191990682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558873490a00 0x55887348dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.206+0000 7f374affd640 1 -- 172.21.15.12:0/1191990682 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558873487e80 msgr2=0x55887348d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.206+0000 7f374affd640 1 --2- 172.21.15.12:0/1191990682 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558873487e80 0x55887348d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.206+0000 7f374affd640 1 -- 172.21.15.12:0/1191990682 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588730fe3f0 con 0x5588734874d0 2024-04-22T13:32:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.206+0000 7f374affd640 1 --2- 172.21.15.12:0/1191990682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588734874d0 0x5588734902d0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f373c010820 tx=0x7f373c010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4679ceab33f88d51 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.206+0000 7f37327fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:32:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.206+0000 7f37327fc640 1 -- 172.21.15.12:0/1191990682 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c01e060 con 0x5588734874d0 2024-04-22T13:32:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.206+0000 7f37327fc640 1 -- 172.21.15.12:0/1191990682 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f373c00fb90 con 0x5588734874d0 2024-04-22T13:32:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.206+0000 7f3754274740 1 -- 172.21.15.12:0/1191990682 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55887338b930 con 0x5588734874d0 2024-04-22T13:32:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.206+0000 7f37327fc640 1 -- 172.21.15.12:0/1191990682 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f373c00fd90 con 0x5588734874d0 2024-04-22T13:32:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.206+0000 7f3754274740 10 client.18525 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.206+0000 7f3754274740 1 -- 172.21.15.12:0/1191990682 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5588731c87d0 con 0x5588734874d0 2024-04-22T13:32:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.206+0000 7f37327fc640 1 -- 172.21.15.12:0/1191990682 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55887338b930 con 0x5588734874d0 2024-04-22T13:32:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.207+0000 7f37327fc640 1 -- 172.21.15.12:0/1191990682 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f373c021420 con 0x5588734874d0 2024-04-22T13:32:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.207+0000 7f3754274740 10 client.18525 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.207+0000 7f3754274740 10 client.18525 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:32:27.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.207+0000 7f3754274740 20 client.18525 populate_metadata read hostname 'smithi012' 2024-04-22T13:32:27.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.207+0000 7f3754274740 1 --2- 172.21.15.12:0/1191990682 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5588734a0b70 0x5588734a2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.207+0000 7f3754274740 4 client.18525 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:27.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.207+0000 7f3754274740 1 -- 172.21.15.12:0/1191990682 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588733f9c10 con 0x5588734a0b70 2024-04-22T13:32:27.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.207+0000 7f3743fff640 1 --2- 172.21.15.12:0/1191990682 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5588734a0b70 0x5588734a2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.207+0000 7f3743fff640 1 --2- 172.21.15.12:0/1191990682 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5588734a0b70 0x5588734a2fc0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=196a65234a82c27c server_cookie=8e2bcb3481a6f80d in_seq=0 out_seq=0 2024-04-22T13:32:27.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.207+0000 7f37327fc640 10 client.18525 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:32:27.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.208+0000 7f37327fc640 1 -- 172.21.15.12:0/1191990682 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5588733f9c10 con 0x5588734a0b70 2024-04-22T13:32:27.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.208+0000 7f37327fc640 10 client.18525 handle_command_reply: tid=0 2024-04-22T13:32:27.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.208+0000 7f3754274740 1 client.18525 shutdown 2024-04-22T13:32:27.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.208+0000 7f3754274740 1 -- 172.21.15.12:0/1191990682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588734874d0 msgr2=0x5588734902d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.208+0000 7f3754274740 1 --2- 172.21.15.12:0/1191990682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588734874d0 0x5588734902d0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f373c010820 tx=0x7f373c010ce0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.208+0000 7f3754274740 1 -- 172.21.15.12:0/1191990682 shutdown_connections 2024-04-22T13:32:27.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.208+0000 7f3754274740 1 --2- 172.21.15.12:0/1191990682 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5588734a0b70 0x5588734a2fc0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.208+0000 7f3754274740 1 --2- 172.21.15.12:0/1191990682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558873490a00 0x55887348dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.208+0000 7f3754274740 1 --2- 172.21.15.12:0/1191990682 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558873487e80 0x55887348d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.208+0000 7f37327fc640 0 client.18525 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:32:27.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.208+0000 7f3754274740 1 --2- 172.21.15.12:0/1191990682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5588734874d0 0x5588734902d0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.208+0000 7f3754274740 1 -- 172.21.15.12:0/1191990682 >> 172.21.15.12:0/1191990682 conn(0x558873481ba0 msgr2=0x5588734838f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:27.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.208+0000 7f3754274740 1 -- 172.21.15.12:0/1191990682 shutdown_connections 2024-04-22T13:32:27.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.208+0000 7f3754274740 1 -- 172.21.15.12:0/1191990682 wait complete. 2024-04-22T13:32:27.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.208+0000 7f3754274740 20 client.18525 trim_cache size 0 max 16384 2024-04-22T13:32:27.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.222+0000 7f3754274740 1 Processor -- start 2024-04-22T13:32:27.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.222+0000 7f3754274740 1 -- start start 2024-04-22T13:32:27.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.222+0000 7f3754274740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588734874d0 0x55887349c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.222+0000 7f3754274740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558873487e80 0x55887348fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.222+0000 7f3754274740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558873490a00 0x5588734902d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.222+0000 7f3754274740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f374c1afce0 con 0x558873490a00 2024-04-22T13:32:27.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.222+0000 7f3754274740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f374c1b1820 con 0x5588734874d0 2024-04-22T13:32:27.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.222+0000 7f3754274740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f374c1b03b0 con 0x558873487e80 2024-04-22T13:32:27.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.223+0000 7f374b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558873490a00 0x5588734902d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.223+0000 7f374b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558873490a00 0x5588734902d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33860/0 (socket says 172.21.15.12:33860) 2024-04-22T13:32:27.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.223+0000 7f374b7fe640 1 -- 172.21.15.12:0/127284429 learned_addr learned my addr 172.21.15.12:0/127284429 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:27.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.223+0000 7f374affd640 1 --2- 172.21.15.12:0/127284429 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588734874d0 0x55887349c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.223+0000 7f3743fff640 1 --2- 172.21.15.12:0/127284429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558873487e80 0x55887348fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.223+0000 7f374b7fe640 1 -- 172.21.15.12:0/127284429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558873487e80 msgr2=0x55887348fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.223+0000 7f374b7fe640 1 --2- 172.21.15.12:0/127284429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558873487e80 0x55887348fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.223+0000 7f374b7fe640 1 -- 172.21.15.12:0/127284429 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588734874d0 msgr2=0x55887349c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.225 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.223+0000 7f374b7fe640 1 --2- 172.21.15.12:0/127284429 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588734874d0 0x55887349c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.223+0000 7f374b7fe640 1 -- 172.21.15.12:0/127284429 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55887348df30 con 0x558873490a00 2024-04-22T13:32:27.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.223+0000 7f3743fff640 1 --2- 172.21.15.12:0/127284429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558873487e80 0x55887348fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:27.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.223+0000 7f374affd640 1 --2- 172.21.15.12:0/127284429 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588734874d0 0x55887349c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:27.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.223+0000 7f374b7fe640 1 --2- 172.21.15.12:0/127284429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558873490a00 0x5588734902d0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f3744009ca0 tx=0x7f3744032360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b01a698a6d0197 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:27.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.223+0000 7f3740ff9640 1 -- 172.21.15.12:0/127284429 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3744039040 con 0x558873490a00 2024-04-22T13:32:27.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.223+0000 7f3740ff9640 1 -- 172.21.15.12:0/127284429 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3744054070 con 0x558873490a00 2024-04-22T13:32:27.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.223+0000 7f3740ff9640 1 -- 172.21.15.12:0/127284429 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374404f6b0 con 0x558873490a00 2024-04-22T13:32:27.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.224+0000 7f3754274740 1 -- 172.21.15.12:0/127284429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558873490a00 msgr2=0x5588734902d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.224+0000 7f3754274740 1 --2- 172.21.15.12:0/127284429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558873490a00 0x5588734902d0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f3744009ca0 tx=0x7f3744032360 comp rx=0 tx=0).stop 2024-04-22T13:32:27.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.224+0000 7f3754274740 1 -- 172.21.15.12:0/127284429 shutdown_connections 2024-04-22T13:32:27.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.224+0000 7f3754274740 1 --2- 172.21.15.12:0/127284429 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558873490a00 0x5588734902d0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.224+0000 7f3754274740 1 --2- 172.21.15.12:0/127284429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558873487e80 0x55887348fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.224+0000 7f3754274740 1 --2- 172.21.15.12:0/127284429 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588734874d0 0x55887349c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.224+0000 7f3754274740 1 -- 172.21.15.12:0/127284429 >> 172.21.15.12:0/127284429 conn(0x558873481ba0 msgr2=0x558873480c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:27.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.224+0000 7f3754274740 1 -- 172.21.15.12:0/127284429 shutdown_connections 2024-04-22T13:32:27.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.224+0000 7f3754274740 1 -- 172.21.15.12:0/127284429 wait complete. 2024-04-22T13:32:27.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.224+0000 7f3754274740 1 -- start start 2024-04-22T13:32:27.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.225+0000 7f3754274740 1 Processor -- start 2024-04-22T13:32:27.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.225+0000 7f3754274740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588734874d0 0x558873494d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.225+0000 7f3754274740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558873487e80 0x558873495250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.225+0000 7f3754274740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558873490a00 0x558873506100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.225+0000 7f3754274740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f374c1b1150 con 0x558873490a00 2024-04-22T13:32:27.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.225+0000 7f3754274740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f374c1b0a80 con 0x5588734874d0 2024-04-22T13:32:27.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.225+0000 7f3754274740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f374c1adce0 con 0x558873487e80 2024-04-22T13:32:27.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.225+0000 7f3754274740 10 client.0 _ll_register_callbacks cb 0x7f373c010430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:27.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.225+0000 7f374b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558873490a00 0x558873506100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.225+0000 7f374b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558873490a00 0x558873506100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33866/0 (socket says 172.21.15.12:33866) 2024-04-22T13:32:27.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.225+0000 7f374b7fe640 1 -- 172.21.15.12:0/2543711419 learned_addr learned my addr 172.21.15.12:0/2543711419 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:27.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.225+0000 7f374affd640 1 --2- 172.21.15.12:0/2543711419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588734874d0 0x558873494d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.225+0000 7f3743fff640 1 --2- 172.21.15.12:0/2543711419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558873487e80 0x558873495250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.225+0000 7f374b7fe640 1 -- 172.21.15.12:0/2543711419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558873487e80 msgr2=0x558873495250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.225+0000 7f374b7fe640 1 --2- 172.21.15.12:0/2543711419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558873487e80 0x558873495250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.225+0000 7f374b7fe640 1 -- 172.21.15.12:0/2543711419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588734874d0 msgr2=0x558873494d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.225+0000 7f374b7fe640 1 --2- 172.21.15.12:0/2543711419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588734874d0 0x558873494d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.225+0000 7f374b7fe640 1 -- 172.21.15.12:0/2543711419 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558873506640 con 0x558873490a00 2024-04-22T13:32:27.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.225+0000 7f374b7fe640 1 --2- 172.21.15.12:0/2543711419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558873490a00 0x558873506100 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f374402c630 tx=0x7f3744034a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b44bf250fe8bcf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:27.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.225+0000 7f37327fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:32:27.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.225+0000 7f37327fc640 1 -- 172.21.15.12:0/2543711419 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3744039040 con 0x558873490a00 2024-04-22T13:32:27.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.226+0000 7f37327fc640 1 -- 172.21.15.12:0/2543711419 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f374404f650 con 0x558873490a00 2024-04-22T13:32:27.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.226+0000 7f3754274740 1 -- 172.21.15.12:0/2543711419 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3744054070 con 0x558873490a00 2024-04-22T13:32:27.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.226+0000 7f37327fc640 1 -- 172.21.15.12:0/2543711419 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3744004810 con 0x558873490a00 2024-04-22T13:32:27.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.226+0000 7f3754274740 10 client.18531 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:27.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.226+0000 7f3754274740 1 -- 172.21.15.12:0/2543711419 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5588735078a0 con 0x558873490a00 2024-04-22T13:32:27.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.226+0000 7f37327fc640 1 -- 172.21.15.12:0/2543711419 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3744054070 con 0x558873490a00 2024-04-22T13:32:27.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.226+0000 7f37327fc640 1 -- 172.21.15.12:0/2543711419 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f374403f070 con 0x558873490a00 2024-04-22T13:32:27.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.226+0000 7f3754274740 10 client.18531 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:27.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.226+0000 7f3754274740 10 client.18531 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:32:27.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.226+0000 7f3754274740 20 client.18531 populate_metadata read hostname 'smithi012' 2024-04-22T13:32:27.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.226+0000 7f3754274740 1 --2- 172.21.15.12:0/2543711419 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x558873507fc0 0x55887350a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.226+0000 7f3754274740 4 client.18531 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:32:27.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.226+0000 7f3754274740 1 -- 172.21.15.12:0/2543711419 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5588733f3a80 con 0x558873507fc0 2024-04-22T13:32:27.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.226+0000 7f374affd640 1 --2- 172.21.15.12:0/2543711419 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x558873507fc0 0x55887350a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.227+0000 7f37327fc640 10 client.18531 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:32:27.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.227+0000 7f374affd640 1 --2- 172.21.15.12:0/2543711419 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x558873507fc0 0x55887350a3e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6707e8f4d480261a server_cookie=11f8b6fc77886e93 in_seq=0 out_seq=0 2024-04-22T13:32:27.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.227+0000 7f37327fc640 1 -- 172.21.15.12:0/2543711419 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5588733f3a80 con 0x558873507fc0 2024-04-22T13:32:27.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.227+0000 7f37327fc640 10 client.18531 handle_command_reply: tid=0 2024-04-22T13:32:27.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.227+0000 7f3754274740 1 client.18531 shutdown 2024-04-22T13:32:27.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.227+0000 7f3754274740 1 -- 172.21.15.12:0/2543711419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558873490a00 msgr2=0x558873506100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.227+0000 7f3754274740 1 --2- 172.21.15.12:0/2543711419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558873490a00 0x558873506100 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f374402c630 tx=0x7f3744034a70 comp rx=0 tx=0).stop 2024-04-22T13:32:27.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.227+0000 7f3754274740 1 -- 172.21.15.12:0/2543711419 shutdown_connections 2024-04-22T13:32:27.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.227+0000 7f3754274740 1 --2- 172.21.15.12:0/2543711419 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x558873507fc0 0x55887350a3e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.227+0000 7f3754274740 1 --2- 172.21.15.12:0/2543711419 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558873490a00 0x558873506100 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.227+0000 7f3754274740 1 --2- 172.21.15.12:0/2543711419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558873487e80 0x558873495250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.227+0000 7f37327fc640 0 client.18531 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:32:27.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.227+0000 7f3754274740 1 --2- 172.21.15.12:0/2543711419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588734874d0 0x558873494d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.227+0000 7f3754274740 1 -- 172.21.15.12:0/2543711419 >> 172.21.15.12:0/2543711419 conn(0x558873481ba0 msgr2=0x558873488260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:27.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.227+0000 7f3754274740 1 -- 172.21.15.12:0/2543711419 shutdown_connections 2024-04-22T13:32:27.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.227+0000 7f3754274740 1 -- 172.21.15.12:0/2543711419 wait complete. 2024-04-22T13:32:27.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.227+0000 7f3754274740 20 client.18531 trim_cache size 0 max 16384 2024-04-22T13:32:27.231 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:32:27.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.228+0000 7f3751681640 1 -- 172.21.15.12:0/3548196751 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f37380a3d50 msgr2=0x7f37380a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.228+0000 7f3751681640 1 --2- 172.21.15.12:0/3548196751 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f37380a3d50 0x7f37380a6210 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f37440098d0 tx=0x7f374402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.228+0000 7f3751681640 1 -- 172.21.15.12:0/3548196751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f374c106170 msgr2=0x7f374c103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.228+0000 7f3751681640 1 --2- 172.21.15.12:0/3548196751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f374c106170 0x7f374c103070 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f373400c970 tx=0x7f373400cd70 comp rx=0 tx=0).stop 2024-04-22T13:32:27.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.228+0000 7f3751681640 1 -- 172.21.15.12:0/3548196751 shutdown_connections 2024-04-22T13:32:27.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.228+0000 7f3751681640 1 --2- 172.21.15.12:0/3548196751 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f37380a3d50 0x7f37380a6210 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.228+0000 7f3751681640 1 --2- 172.21.15.12:0/3548196751 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f374c1035b0 0x7f374c1a67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.228+0000 7f3751681640 1 --2- 172.21.15.12:0/3548196751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f374c106170 0x7f374c103070 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.228+0000 7f3751681640 1 --2- 172.21.15.12:0/3548196751 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f374c1010a0 0x7f374c102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.228+0000 7f3751681640 1 -- 172.21.15.12:0/3548196751 >> 172.21.15.12:0/3548196751 conn(0x7f374c077eb0 msgr2=0x7f374c104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:27.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.228+0000 7f3751681640 1 -- 172.21.15.12:0/3548196751 shutdown_connections 2024-04-22T13:32:27.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.228+0000 7f3751681640 1 -- 172.21.15.12:0/3548196751 wait complete. 2024-04-22T13:32:27.235 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:32:27.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.343+0000 7fcc9a9f4640 1 -- 172.21.15.12:0/1859390724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc94106170 msgr2=0x7fcc9410dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.343+0000 7fcc9a9f4640 1 --2- 172.21.15.12:0/1859390724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc94106170 0x7fcc9410dd40 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c009ca0 tx=0x7fcc8c02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.343+0000 7fcc9a9f4640 1 -- 172.21.15.12:0/1859390724 shutdown_connections 2024-04-22T13:32:27.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.343+0000 7fcc9a9f4640 1 --2- 172.21.15.12:0/1859390724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc94106170 0x7fcc9410dd40 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.343+0000 7fcc9a9f4640 1 --2- 172.21.15.12:0/1859390724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc94101a50 0x7fcc94105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.343+0000 7fcc9a9f4640 1 --2- 172.21.15.12:0/1859390724 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc941010a0 0x7fcc94101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.343+0000 7fcc9a9f4640 1 -- 172.21.15.12:0/1859390724 >> 172.21.15.12:0/1859390724 conn(0x7fcc94077eb0 msgr2=0x7fcc940782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:27.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.343+0000 7fcc9a9f4640 1 -- 172.21.15.12:0/1859390724 shutdown_connections 2024-04-22T13:32:27.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.343+0000 7fcc9a9f4640 1 -- 172.21.15.12:0/1859390724 wait complete. 2024-04-22T13:32:27.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.343+0000 7fcc9a9f4640 1 Processor -- start 2024-04-22T13:32:27.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.344+0000 7fcc9a9f4640 1 -- start start 2024-04-22T13:32:27.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.344+0000 7fcc9a9f4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc941010a0 0x7fcc94198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.344+0000 7fcc9a9f4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc94106170 0x7fcc94198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.344+0000 7fcc9a9f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc9419ced0 0x7fcc9419d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.344+0000 7fcc9a9f4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcc941121f0 con 0x7fcc941010a0 2024-04-22T13:32:27.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.344+0000 7fcc9a9f4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcc94112070 con 0x7fcc94106170 2024-04-22T13:32:27.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.344+0000 7fcc9a9f4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcc94112370 con 0x7fcc9419ced0 2024-04-22T13:32:27.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.344+0000 7fcc93fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc941010a0 0x7fcc94198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.344+0000 7fcc93fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc941010a0 0x7fcc94198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33872/0 (socket says 172.21.15.12:33872) 2024-04-22T13:32:27.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.344+0000 7fcc93fff640 1 -- 172.21.15.12:0/1204403682 learned_addr learned my addr 172.21.15.12:0/1204403682 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:27.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.344+0000 7fcc937fe640 1 --2- 172.21.15.12:0/1204403682 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc94106170 0x7fcc94198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.344+0000 7fcc98f91640 1 --2- 172.21.15.12:0/1204403682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc9419ced0 0x7fcc9419d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.344+0000 7fcc93fff640 1 -- 172.21.15.12:0/1204403682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc9419ced0 msgr2=0x7fcc9419d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.344+0000 7fcc93fff640 1 --2- 172.21.15.12:0/1204403682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc9419ced0 0x7fcc9419d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.344+0000 7fcc93fff640 1 -- 172.21.15.12:0/1204403682 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc94106170 msgr2=0x7fcc94198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.344+0000 7fcc93fff640 1 --2- 172.21.15.12:0/1204403682 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc94106170 0x7fcc94198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.344+0000 7fcc93fff640 1 -- 172.21.15.12:0/1204403682 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc9419d9c0 con 0x7fcc941010a0 2024-04-22T13:32:27.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.345+0000 7fcc93fff640 1 --2- 172.21.15.12:0/1204403682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc941010a0 0x7fcc94198610 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400c920 tx=0x7fcc8400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:27.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.345+0000 7fcc917fa640 1 -- 172.21.15.12:0/1204403682 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc84007c70 con 0x7fcc941010a0 2024-04-22T13:32:27.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.345+0000 7fcc917fa640 1 -- 172.21.15.12:0/1204403682 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc84007e10 con 0x7fcc941010a0 2024-04-22T13:32:27.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.345+0000 7fcc9a9f4640 1 -- 172.21.15.12:0/1204403682 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc9419dcb0 con 0x7fcc941010a0 2024-04-22T13:32:27.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.345+0000 7fcc917fa640 1 -- 172.21.15.12:0/1204403682 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc84018820 con 0x7fcc941010a0 2024-04-22T13:32:27.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.345+0000 7fcc9a9f4640 1 -- 172.21.15.12:0/1204403682 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc941a54a0 con 0x7fcc941010a0 2024-04-22T13:32:27.350 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.347+0000 7fcc917fa640 1 -- 172.21.15.12:0/1204403682 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcc8400f450 con 0x7fcc941010a0 2024-04-22T13:32:27.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.347+0000 7fcc9d5e8740 1 Processor -- start 2024-04-22T13:32:27.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.347+0000 7fcc9d5e8740 1 -- start start 2024-04-22T13:32:27.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.348+0000 7fcc9d5e8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a25eab64d0 0x55a25eab68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.348+0000 7fcc9d5e8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a25eab6e80 0x55a25eabf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.348+0000 7fcc9d5e8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a25eabfa00 0x55a25eac1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.348+0000 7fcc9d5e8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a25ea1b1f0 con 0x55a25eab6e80 2024-04-22T13:32:27.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.348+0000 7fcc9d5e8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a25e9b84b0 con 0x55a25eab64d0 2024-04-22T13:32:27.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.348+0000 7fcc9d5e8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a25ea171b0 con 0x55a25eabfa00 2024-04-22T13:32:27.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.348+0000 7fcc93fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a25eab6e80 0x55a25eabf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.348+0000 7fcc93fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a25eab6e80 0x55a25eabf430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33886/0 (socket says 172.21.15.12:33886) 2024-04-22T13:32:27.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.348+0000 7fcc93fff640 1 -- 172.21.15.12:0/2939524895 learned_addr learned my addr 172.21.15.12:0/2939524895 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:27.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.348+0000 7fcc98f91640 1 --2- 172.21.15.12:0/2939524895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a25eab64d0 0x55a25eab68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.351 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.348+0000 7fcc937fe640 1 --2- 172.21.15.12:0/2939524895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a25eabfa00 0x55a25eac1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.348+0000 7fcc917fa640 1 --2- 172.21.15.12:0/1204403682 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fcc7c0a3d20 0x7fcc7c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.348+0000 7fcc937fe640 1 -- 172.21.15.12:0/2939524895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a25eab64d0 msgr2=0x55a25eab68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.348+0000 7fcc937fe640 1 --2- 172.21.15.12:0/2939524895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a25eab64d0 0x55a25eab68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.348+0000 7fcc937fe640 1 -- 172.21.15.12:0/2939524895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a25eab6e80 msgr2=0x55a25eabf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.348+0000 7fcc937fe640 1 --2- 172.21.15.12:0/2939524895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a25eab6e80 0x55a25eabf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.348+0000 7fcc937fe640 1 -- 172.21.15.12:0/2939524895 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a25e9aa1b0 con 0x55a25eabfa00 2024-04-22T13:32:27.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.348+0000 7fcc93fff640 1 --2- 172.21.15.12:0/2939524895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a25eab6e80 0x55a25eabf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:32:27.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.348+0000 7fcc98f91640 1 --2- 172.21.15.12:0/2939524895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a25eab64d0 0x55a25eab68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:32:27.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.348+0000 7fcc98f91640 1 --2- 172.21.15.12:0/1204403682 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fcc7c0a3d20 0x7fcc7c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.348+0000 7fcc917fa640 1 -- 172.21.15.12:0/1204403682 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7fcc840d08b0 con 0x7fcc941010a0 2024-04-22T13:32:27.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.348+0000 7fcc937fe640 1 --2- 172.21.15.12:0/2939524895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a25eabfa00 0x55a25eac1e50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcc9406a420 tx=0x7fcc88011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4ccbab8c6be0f3f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:27.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.349+0000 7fcc7a7fc640 1 -- 172.21.15.12:0/2939524895 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc88004050 con 0x55a25eabfa00 2024-04-22T13:32:27.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.349+0000 7fcc98f91640 1 --2- 172.21.15.12:0/1204403682 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fcc7c0a3d20 0x7fcc7c0a61e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c005e30 tx=0x7fcc8c02e040 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:27.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.349+0000 7fcc7a7fc640 1 -- 172.21.15.12:0/2939524895 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc880054d0 con 0x55a25eabfa00 2024-04-22T13:32:27.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.349+0000 7fcc7a7fc640 1 -- 172.21.15.12:0/2939524895 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc88004b90 con 0x55a25eabfa00 2024-04-22T13:32:27.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.349+0000 7fcc9d5e8740 1 -- 172.21.15.12:0/2939524895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a25eabfa00 msgr2=0x55a25eac1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.349+0000 7fcc9d5e8740 1 --2- 172.21.15.12:0/2939524895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a25eabfa00 0x55a25eac1e50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcc9406a420 tx=0x7fcc88011810 comp rx=0 tx=0).stop 2024-04-22T13:32:27.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.349+0000 7fcc9d5e8740 1 -- 172.21.15.12:0/2939524895 shutdown_connections 2024-04-22T13:32:27.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.349+0000 7fcc9d5e8740 1 --2- 172.21.15.12:0/2939524895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a25eabfa00 0x55a25eac1e50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.349+0000 7fcc9d5e8740 1 --2- 172.21.15.12:0/2939524895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a25eab6e80 0x55a25eabf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.349+0000 7fcc9d5e8740 1 --2- 172.21.15.12:0/2939524895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a25eab64d0 0x55a25eab68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.349+0000 7fcc9d5e8740 1 -- 172.21.15.12:0/2939524895 >> 172.21.15.12:0/2939524895 conn(0x55a25eab0ba0 msgr2=0x55a25eab0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:27.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.349+0000 7fcc9d5e8740 1 -- 172.21.15.12:0/2939524895 shutdown_connections 2024-04-22T13:32:27.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.349+0000 7fcc9d5e8740 1 -- 172.21.15.12:0/2939524895 wait complete. 2024-04-22T13:32:27.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.350+0000 7fcc9d5e8740 1 -- start start 2024-04-22T13:32:27.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.350+0000 7fcc9d5e8740 1 Processor -- start 2024-04-22T13:32:27.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.350+0000 7fcc9d5e8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a25eab64d0 0x55a25eabf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.350+0000 7fcc9d5e8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a25eab6e80 0x55a25eabc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.350+0000 7fcc9d5e8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a25eabfa00 0x55a25eabcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.350+0000 7fcc9d5e8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a25eac2970 con 0x55a25eabfa00 2024-04-22T13:32:27.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.350+0000 7fcc9d5e8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a25e9d4500 con 0x55a25eab64d0 2024-04-22T13:32:27.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.350+0000 7fcc9d5e8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a25eac2af0 con 0x55a25eab6e80 2024-04-22T13:32:27.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.350+0000 7fcc9d5e8740 10 client.0 _ll_register_callbacks cb 0x55a25e9a2b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:27.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.350+0000 7fcc98f91640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a25eabfa00 0x55a25eabcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.350+0000 7fcc98f91640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a25eabfa00 0x55a25eabcbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33902/0 (socket says 172.21.15.12:33902) 2024-04-22T13:32:27.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.350+0000 7fcc98f91640 1 -- 172.21.15.12:0/686190506 learned_addr learned my addr 172.21.15.12:0/686190506 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:27.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.350+0000 7fcc93fff640 1 --2- 172.21.15.12:0/686190506 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a25eab64d0 0x55a25eabf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.350+0000 7fcc937fe640 1 --2- 172.21.15.12:0/686190506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a25eab6e80 0x55a25eabc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.351+0000 7fcc98f91640 1 -- 172.21.15.12:0/686190506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a25eab6e80 msgr2=0x55a25eabc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.351+0000 7fcc98f91640 1 --2- 172.21.15.12:0/686190506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a25eab6e80 0x55a25eabc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.351+0000 7fcc98f91640 1 -- 172.21.15.12:0/686190506 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a25eab64d0 msgr2=0x55a25eabf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.351+0000 7fcc98f91640 1 --2- 172.21.15.12:0/686190506 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a25eab64d0 0x55a25eabf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.351+0000 7fcc98f91640 1 -- 172.21.15.12:0/686190506 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a25e72d3f0 con 0x55a25eabfa00 2024-04-22T13:32:27.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.351+0000 7fcc98f91640 1 --2- 172.21.15.12:0/686190506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a25eabfa00 0x55a25eabcbb0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c036000 tx=0x7fcc8c005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98aee0df9fa208e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:27.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.351+0000 7fcc737fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:32:27.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.351+0000 7fcc737fe640 1 -- 172.21.15.12:0/686190506 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c052060 con 0x55a25eabfa00 2024-04-22T13:32:27.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.351+0000 7fcc737fe640 1 -- 172.21.15.12:0/686190506 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7fcc8c004050 con 0x55a25eabfa00 2024-04-22T13:32:27.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.351+0000 7fcc9d5e8740 1 -- 172.21.15.12:0/686190506 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc880054d0 con 0x55a25eabfa00 2024-04-22T13:32:27.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.351+0000 7fcc737fe640 1 -- 172.21.15.12:0/686190506 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcc8c046070 con 0x55a25eabfa00 2024-04-22T13:32:27.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.351+0000 7fcc9d5e8740 10 client.18543 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:27.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.351+0000 7fcc9d5e8740 1 -- 172.21.15.12:0/686190506 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a25e7f77d0 con 0x55a25eabfa00 2024-04-22T13:32:27.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.351+0000 7fcc737fe640 1 -- 172.21.15.12:0/686190506 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc880054d0 con 0x55a25eabfa00 2024-04-22T13:32:27.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.351+0000 7fcc737fe640 1 -- 172.21.15.12:0/686190506 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fcc8c03f040 con 0x55a25eabfa00 2024-04-22T13:32:27.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.351+0000 7fcc9d5e8740 10 client.18543 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:27.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.351+0000 7fcc9d5e8740 10 client.18543 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:32:27.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.352+0000 7fcc9d5e8740 20 client.18543 populate_metadata read hostname 'smithi012' 2024-04-22T13:32:27.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.352+0000 7fcc9d5e8740 1 --2- 172.21.15.12:0/686190506 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55a25eacfc20 0x55a25ead20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.352+0000 7fcc9d5e8740 4 client.18543 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:27.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.352+0000 7fcc9d5e8740 1 -- 172.21.15.12:0/686190506 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a25e9ba930 con 0x55a25eacfc20 2024-04-22T13:32:27.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.352+0000 7fcc93fff640 1 --2- 172.21.15.12:0/686190506 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55a25eacfc20 0x55a25ead20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.352+0000 7fcc93fff640 1 --2- 172.21.15.12:0/686190506 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55a25eacfc20 0x55a25ead20a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e112941d912da208 server_cookie=d66db43c1ef439c8 in_seq=0 out_seq=0 2024-04-22T13:32:27.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.352+0000 7fcc737fe640 10 client.18543 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:32:27.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.353+0000 7fcc737fe640 1 -- 172.21.15.12:0/686190506 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a25e9ba930 con 0x55a25eacfc20 2024-04-22T13:32:27.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.353+0000 7fcc737fe640 10 client.18543 handle_command_reply: tid=0 2024-04-22T13:32:27.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.353+0000 7fcc9d5e8740 1 client.18543 shutdown 2024-04-22T13:32:27.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.353+0000 7fcc9d5e8740 1 -- 172.21.15.12:0/686190506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a25eabfa00 msgr2=0x55a25eabcbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.353+0000 7fcc9d5e8740 1 --2- 172.21.15.12:0/686190506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a25eabfa00 0x55a25eabcbb0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c036000 tx=0x7fcc8c005d80 comp rx=0 tx=0).stop 2024-04-22T13:32:27.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.353+0000 7fcc9d5e8740 1 -- 172.21.15.12:0/686190506 shutdown_connections 2024-04-22T13:32:27.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.353+0000 7fcc9d5e8740 1 --2- 172.21.15.12:0/686190506 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55a25eacfc20 0x55a25ead20a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.353+0000 7fcc9d5e8740 1 --2- 172.21.15.12:0/686190506 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a25eabfa00 0x55a25eabcbb0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.353+0000 7fcc9d5e8740 1 --2- 172.21.15.12:0/686190506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a25eab6e80 0x55a25eabc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.353+0000 7fcc9d5e8740 1 --2- 172.21.15.12:0/686190506 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a25eab64d0 0x55a25eabf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.353+0000 7fcc9d5e8740 1 -- 172.21.15.12:0/686190506 >> 172.21.15.12:0/686190506 conn(0x55a25eab0ba0 msgr2=0x55a25eab28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:27.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.353+0000 7fcc737fe640 0 client.18543 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:32:27.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.353+0000 7fcc9d5e8740 1 -- 172.21.15.12:0/686190506 shutdown_connections 2024-04-22T13:32:27.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.353+0000 7fcc9d5e8740 1 -- 172.21.15.12:0/686190506 wait complete. 2024-04-22T13:32:27.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.353+0000 7fcc9d5e8740 20 client.18543 trim_cache size 0 max 16384 2024-04-22T13:32:27.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.366+0000 7fcc9d5e8740 1 Processor -- start 2024-04-22T13:32:27.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.366+0000 7fcc9d5e8740 1 -- start start 2024-04-22T13:32:27.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.367+0000 7fcc9d5e8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a25eab64d0 0x55a25eacb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.367+0000 7fcc9d5e8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a25eab6e80 0x55a25eab2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.367+0000 7fcc9d5e8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a25eabfa00 0x55a25eabed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.367+0000 7fcc9d5e8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcc941ad9b0 con 0x55a25eabfa00 2024-04-22T13:32:27.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.367+0000 7fcc9d5e8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcc941af4f0 con 0x55a25eab6e80 2024-04-22T13:32:27.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.367+0000 7fcc9d5e8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcc941ae080 con 0x55a25eab64d0 2024-04-22T13:32:27.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.367+0000 7fcc98f91640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a25eabfa00 0x55a25eabed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.367+0000 7fcc98f91640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a25eabfa00 0x55a25eabed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33914/0 (socket says 172.21.15.12:33914) 2024-04-22T13:32:27.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.367+0000 7fcc98f91640 1 -- 172.21.15.12:0/1230196714 learned_addr learned my addr 172.21.15.12:0/1230196714 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:27.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.367+0000 7fcc93fff640 1 --2- 172.21.15.12:0/1230196714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a25eab64d0 0x55a25eacb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.367+0000 7fcc937fe640 1 --2- 172.21.15.12:0/1230196714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a25eab6e80 0x55a25eab2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.367+0000 7fcc98f91640 1 -- 172.21.15.12:0/1230196714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a25eab64d0 msgr2=0x55a25eacb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.367+0000 7fcc98f91640 1 --2- 172.21.15.12:0/1230196714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a25eab64d0 0x55a25eacb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.367+0000 7fcc98f91640 1 -- 172.21.15.12:0/1230196714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a25eab6e80 msgr2=0x55a25eab2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.367+0000 7fcc98f91640 1 --2- 172.21.15.12:0/1230196714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a25eab6e80 0x55a25eab2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.367+0000 7fcc98f91640 1 -- 172.21.15.12:0/1230196714 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a25ead2400 con 0x55a25eabfa00 2024-04-22T13:32:27.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.367+0000 7fcc93fff640 1 --2- 172.21.15.12:0/1230196714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a25eab64d0 0x55a25eacb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:27.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.367+0000 7fcc937fe640 1 --2- 172.21.15.12:0/1230196714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a25eab6e80 0x55a25eab2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:27.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.367+0000 7fcc98f91640 1 --2- 172.21.15.12:0/1230196714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a25eabfa00 0x55a25eabed80 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c02c630 tx=0x7fcc8c02c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce4db1536451d12 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:27.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.367+0000 7fcc79ffb640 1 -- 172.21.15.12:0/1230196714 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c052040 con 0x55a25eabfa00 2024-04-22T13:32:27.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.367+0000 7fcc79ffb640 1 -- 172.21.15.12:0/1230196714 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc8c032890 con 0x55a25eabfa00 2024-04-22T13:32:27.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.367+0000 7fcc79ffb640 1 -- 172.21.15.12:0/1230196714 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c039040 con 0x55a25eabfa00 2024-04-22T13:32:27.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.368+0000 7fcc9d5e8740 1 -- 172.21.15.12:0/1230196714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a25eabfa00 msgr2=0x55a25eabed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.368+0000 7fcc9d5e8740 1 --2- 172.21.15.12:0/1230196714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a25eabfa00 0x55a25eabed80 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c02c630 tx=0x7fcc8c02c810 comp rx=0 tx=0).stop 2024-04-22T13:32:27.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.368+0000 7fcc9d5e8740 1 -- 172.21.15.12:0/1230196714 shutdown_connections 2024-04-22T13:32:27.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.368+0000 7fcc9d5e8740 1 --2- 172.21.15.12:0/1230196714 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a25eabfa00 0x55a25eabed80 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.368+0000 7fcc9d5e8740 1 --2- 172.21.15.12:0/1230196714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a25eab6e80 0x55a25eab2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.368+0000 7fcc9d5e8740 1 --2- 172.21.15.12:0/1230196714 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a25eab64d0 0x55a25eacb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.368+0000 7fcc9d5e8740 1 -- 172.21.15.12:0/1230196714 >> 172.21.15.12:0/1230196714 conn(0x55a25eab0ba0 msgr2=0x55a25eab1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:27.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.368+0000 7fcc9d5e8740 1 -- 172.21.15.12:0/1230196714 shutdown_connections 2024-04-22T13:32:27.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.368+0000 7fcc9d5e8740 1 -- 172.21.15.12:0/1230196714 wait complete. 2024-04-22T13:32:27.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.368+0000 7fcc9d5e8740 1 -- start start 2024-04-22T13:32:27.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.369+0000 7fcc9d5e8740 1 Processor -- start 2024-04-22T13:32:27.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.369+0000 7fcc9d5e8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a25eab64d0 0x55a25eac7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.369+0000 7fcc9d5e8740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a25eab6e80 0x55a25eab8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.369+0000 7fcc9d5e8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a25eabfa00 0x55a25eab9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.369+0000 7fcc9d5e8740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fcc941aee20 con 0x55a25eab6e80 2024-04-22T13:32:27.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.369+0000 7fcc9d5e8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcc941ae750 con 0x55a25eabfa00 2024-04-22T13:32:27.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.369+0000 7fcc9d5e8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcc941ab9b0 con 0x55a25eab64d0 2024-04-22T13:32:27.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.369+0000 7fcc9d5e8740 10 client.0 _ll_register_callbacks cb 0x7fcc8c032f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:27.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.369+0000 7fcc937fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a25eab6e80 0x55a25eab8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.369+0000 7fcc937fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a25eab6e80 0x55a25eab8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33928/0 (socket says 172.21.15.12:33928) 2024-04-22T13:32:27.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.369+0000 7fcc937fe640 1 -- 172.21.15.12:0/4057000183 learned_addr learned my addr 172.21.15.12:0/4057000183 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:27.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.369+0000 7fcc93fff640 1 --2- 172.21.15.12:0/4057000183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a25eab64d0 0x55a25eac7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.369+0000 7fcc98f91640 1 --2- 172.21.15.12:0/4057000183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a25eabfa00 0x55a25eab9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.369+0000 7fcc937fe640 1 -- 172.21.15.12:0/4057000183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a25eab64d0 msgr2=0x55a25eac7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.369+0000 7fcc937fe640 1 --2- 172.21.15.12:0/4057000183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a25eab64d0 0x55a25eac7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.369+0000 7fcc937fe640 1 -- 172.21.15.12:0/4057000183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a25eabfa00 msgr2=0x55a25eab9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.369+0000 7fcc937fe640 1 --2- 172.21.15.12:0/4057000183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a25eabfa00 0x55a25eab9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.369+0000 7fcc937fe640 1 -- 172.21.15.12:0/4057000183 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a25eab9960 con 0x55a25eab6e80 2024-04-22T13:32:27.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.369+0000 7fcc937fe640 1 --2- 172.21.15.12:0/4057000183 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a25eab6e80 0x55a25eab8ee0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009710 tx=0x7fcc88005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e03de9848081aad server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:27.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.369+0000 7fcc737fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:32:27.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.369+0000 7fcc737fe640 1 -- 172.21.15.12:0/4057000183 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc88028070 con 0x55a25eab6e80 2024-04-22T13:32:27.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.370+0000 7fcc737fe640 1 -- 172.21.15.12:0/4057000183 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7fcc88005030 con 0x55a25eab6e80 2024-04-22T13:32:27.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.370+0000 7fcc9d5e8740 1 -- 172.21.15.12:0/4057000183 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc8c032890 con 0x55a25eab6e80 2024-04-22T13:32:27.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.370+0000 7fcc737fe640 1 -- 172.21.15.12:0/4057000183 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcc88005230 con 0x55a25eab6e80 2024-04-22T13:32:27.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.370+0000 7fcc9d5e8740 10 client.18549 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:27.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.370+0000 7fcc9d5e8740 1 -- 172.21.15.12:0/4057000183 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a25eb35e10 con 0x55a25eab6e80 2024-04-22T13:32:27.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.370+0000 7fcc737fe640 1 -- 172.21.15.12:0/4057000183 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc8c032890 con 0x55a25eab6e80 2024-04-22T13:32:27.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.370+0000 7fcc737fe640 1 -- 172.21.15.12:0/4057000183 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fcc88033070 con 0x55a25eab6e80 2024-04-22T13:32:27.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.370+0000 7fcc9d5e8740 10 client.18549 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:27.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.370+0000 7fcc9d5e8740 10 client.18549 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:32:27.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.370+0000 7fcc9d5e8740 20 client.18549 populate_metadata read hostname 'smithi012' 2024-04-22T13:32:27.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.370+0000 7fcc9d5e8740 1 --2- 172.21.15.12:0/4057000183 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55a25eb36420 0x55a25eb38840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.370+0000 7fcc9d5e8740 4 client.18549 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:32:27.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.370+0000 7fcc9d5e8740 1 -- 172.21.15.12:0/4057000183 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a25ea28c10 con 0x55a25eb36420 2024-04-22T13:32:27.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.370+0000 7fcc93fff640 1 --2- 172.21.15.12:0/4057000183 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55a25eb36420 0x55a25eb38840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.371+0000 7fcc93fff640 1 --2- 172.21.15.12:0/4057000183 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55a25eb36420 0x55a25eb38840 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b1f0893e05c529a2 server_cookie=549670aeef277cf3 in_seq=0 out_seq=0 2024-04-22T13:32:27.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.371+0000 7fcc737fe640 10 client.18549 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:32:27.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.371+0000 7fcc737fe640 1 -- 172.21.15.12:0/4057000183 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a25ea28c10 con 0x55a25eb36420 2024-04-22T13:32:27.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.371+0000 7fcc737fe640 10 client.18549 handle_command_reply: tid=0 2024-04-22T13:32:27.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.371+0000 7fcc9d5e8740 1 client.18549 shutdown 2024-04-22T13:32:27.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.371+0000 7fcc9d5e8740 1 -- 172.21.15.12:0/4057000183 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a25eab6e80 msgr2=0x55a25eab8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.371+0000 7fcc9d5e8740 1 --2- 172.21.15.12:0/4057000183 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a25eab6e80 0x55a25eab8ee0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009710 tx=0x7fcc88005b40 comp rx=0 tx=0).stop 2024-04-22T13:32:27.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.371+0000 7fcc9d5e8740 1 -- 172.21.15.12:0/4057000183 shutdown_connections 2024-04-22T13:32:27.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.371+0000 7fcc9d5e8740 1 --2- 172.21.15.12:0/4057000183 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55a25eb36420 0x55a25eb38840 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.371+0000 7fcc9d5e8740 1 --2- 172.21.15.12:0/4057000183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a25eabfa00 0x55a25eab9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.371+0000 7fcc737fe640 0 client.18549 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:32:27.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.371+0000 7fcc9d5e8740 1 --2- 172.21.15.12:0/4057000183 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a25eab6e80 0x55a25eab8ee0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.371+0000 7fcc9d5e8740 1 --2- 172.21.15.12:0/4057000183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a25eab64d0 0x55a25eac7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.371+0000 7fcc9d5e8740 1 -- 172.21.15.12:0/4057000183 >> 172.21.15.12:0/4057000183 conn(0x55a25eab0ba0 msgr2=0x55a25eaaf370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:27.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.371+0000 7fcc9d5e8740 1 -- 172.21.15.12:0/4057000183 shutdown_connections 2024-04-22T13:32:27.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.371+0000 7fcc9d5e8740 1 -- 172.21.15.12:0/4057000183 wait complete. 2024-04-22T13:32:27.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.371+0000 7fcc9d5e8740 20 client.18549 trim_cache size 0 max 16384 2024-04-22T13:32:27.375 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:32:27.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.372+0000 7fcc9a9f4640 1 -- 172.21.15.12:0/1204403682 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fcc7c0a3d20 msgr2=0x7fcc7c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.372+0000 7fcc9a9f4640 1 --2- 172.21.15.12:0/1204403682 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fcc7c0a3d20 0x7fcc7c0a61e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c005e30 tx=0x7fcc8c02e040 comp rx=0 tx=0).stop 2024-04-22T13:32:27.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.372+0000 7fcc9a9f4640 1 -- 172.21.15.12:0/1204403682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc941010a0 msgr2=0x7fcc94198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.372+0000 7fcc9a9f4640 1 --2- 172.21.15.12:0/1204403682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc941010a0 0x7fcc94198610 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400c920 tx=0x7fcc8400cde0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.372+0000 7fcc9a9f4640 1 -- 172.21.15.12:0/1204403682 shutdown_connections 2024-04-22T13:32:27.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.372+0000 7fcc9a9f4640 1 --2- 172.21.15.12:0/1204403682 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fcc7c0a3d20 0x7fcc7c0a61e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.372+0000 7fcc9a9f4640 1 --2- 172.21.15.12:0/1204403682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcc9419ced0 0x7fcc9419d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.372+0000 7fcc9a9f4640 1 --2- 172.21.15.12:0/1204403682 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcc94106170 0x7fcc94198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.372+0000 7fcc9a9f4640 1 --2- 172.21.15.12:0/1204403682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fcc941010a0 0x7fcc94198610 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.372+0000 7fcc9a9f4640 1 -- 172.21.15.12:0/1204403682 >> 172.21.15.12:0/1204403682 conn(0x7fcc94077eb0 msgr2=0x7fcc940fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:27.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.372+0000 7fcc9a9f4640 1 -- 172.21.15.12:0/1204403682 shutdown_connections 2024-04-22T13:32:27.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.372+0000 7fcc9a9f4640 1 -- 172.21.15.12:0/1204403682 wait complete. 2024-04-22T13:32:27.379 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:32:27.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.487+0000 7fac1e766640 1 -- 172.21.15.12:0/1475550271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac18107270 msgr2=0x7fac1810ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.487+0000 7fac1e766640 1 --2- 172.21.15.12:0/1475550271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac18107270 0x7fac1810ee40 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fac10009ca0 tx=0x7fac1002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.487+0000 7fac1e766640 1 -- 172.21.15.12:0/1475550271 shutdown_connections 2024-04-22T13:32:27.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.487+0000 7fac1e766640 1 --2- 172.21.15.12:0/1475550271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac18107270 0x7fac1810ee40 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.487+0000 7fac1e766640 1 --2- 172.21.15.12:0/1475550271 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac1806b1f0 0x7fac18106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.486 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.487+0000 7fac1e766640 1 --2- 172.21.15.12:0/1475550271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac1806a840 0x7fac1806ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.487+0000 7fac1e766640 1 -- 172.21.15.12:0/1475550271 >> 172.21.15.12:0/1475550271 conn(0x7fac18077eb0 msgr2=0x7fac180782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:27.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.487+0000 7fac1e766640 1 -- 172.21.15.12:0/1475550271 shutdown_connections 2024-04-22T13:32:27.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.487+0000 7fac1e766640 1 -- 172.21.15.12:0/1475550271 wait complete. 2024-04-22T13:32:27.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.488+0000 7fac1e766640 1 Processor -- start 2024-04-22T13:32:27.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.488+0000 7fac1e766640 1 -- start start 2024-04-22T13:32:27.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.488+0000 7fac1e766640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac1806a840 0x7fac181986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.488+0000 7fac1e766640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac18107270 0x7fac18198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.488+0000 7fac1e766640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac1819cf70 0x7fac1819d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.488+0000 7fac1e766640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fac181121d0 con 0x7fac1806a840 2024-04-22T13:32:27.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.488+0000 7fac1e766640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fac18112050 con 0x7fac18107270 2024-04-22T13:32:27.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.488+0000 7fac1e766640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fac18112350 con 0x7fac1819cf70 2024-04-22T13:32:27.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.488+0000 7fac17fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac1806a840 0x7fac181986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.487 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.488+0000 7fac17fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac1806a840 0x7fac181986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33950/0 (socket says 172.21.15.12:33950) 2024-04-22T13:32:27.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.488+0000 7fac17fff640 1 -- 172.21.15.12:0/2910804741 learned_addr learned my addr 172.21.15.12:0/2910804741 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:27.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.489+0000 7fac177fe640 1 --2- 172.21.15.12:0/2910804741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac18107270 0x7fac18198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.489+0000 7fac17fff640 1 -- 172.21.15.12:0/2910804741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac1819cf70 msgr2=0x7fac1819d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.489+0000 7fac1cd03640 1 --2- 172.21.15.12:0/2910804741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac1819cf70 0x7fac1819d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.489+0000 7fac17fff640 1 --2- 172.21.15.12:0/2910804741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac1819cf70 0x7fac1819d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.489+0000 7fac17fff640 1 -- 172.21.15.12:0/2910804741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac18107270 msgr2=0x7fac18198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.489+0000 7fac17fff640 1 --2- 172.21.15.12:0/2910804741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac18107270 0x7fac18198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.489+0000 7fac17fff640 1 -- 172.21.15.12:0/2910804741 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1819da60 con 0x7fac1806a840 2024-04-22T13:32:27.488 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.489+0000 7fac177fe640 1 --2- 172.21.15.12:0/2910804741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac18107270 0x7fac18198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:27.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.489+0000 7fac1cd03640 1 --2- 172.21.15.12:0/2910804741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac1819cf70 0x7fac1819d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:27.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.489+0000 7fac17fff640 1 --2- 172.21.15.12:0/2910804741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac1806a840 0x7fac181986b0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fac0000e9f0 tx=0x7fac0000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:27.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.489+0000 7fac157fa640 1 -- 172.21.15.12:0/2910804741 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0000cd80 con 0x7fac1806a840 2024-04-22T13:32:27.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.489+0000 7fac157fa640 1 -- 172.21.15.12:0/2910804741 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac000054d0 con 0x7fac1806a840 2024-04-22T13:32:27.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.489+0000 7fac1e766640 1 -- 172.21.15.12:0/2910804741 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac1819dd50 con 0x7fac1806a840 2024-04-22T13:32:27.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.489+0000 7fac157fa640 1 -- 172.21.15.12:0/2910804741 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac00010690 con 0x7fac1806a840 2024-04-22T13:32:27.489 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.489+0000 7fac1e766640 1 -- 172.21.15.12:0/2910804741 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac181a54f0 con 0x7fac1806a840 2024-04-22T13:32:27.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.491+0000 7fac157fa640 1 -- 172.21.15.12:0/2910804741 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fac00004e90 con 0x7fac1806a840 2024-04-22T13:32:27.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.492+0000 7fac21431740 1 Processor -- start 2024-04-22T13:32:27.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.492+0000 7fac21431740 1 -- start start 2024-04-22T13:32:27.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.492+0000 7fac21431740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d66cce3250 0x55d66cce3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.492+0000 7fac21431740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d66cce3c00 0x55d66ccf0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.492+0000 7fac21431740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d66ccf0e10 0x55d66ccf3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.492+0000 7fac21431740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d66cc481f0 con 0x55d66cce3c00 2024-04-22T13:32:27.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.492+0000 7fac21431740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d66cbe54b0 con 0x55d66cce3250 2024-04-22T13:32:27.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.492+0000 7fac21431740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d66cc441b0 con 0x55d66ccf0e10 2024-04-22T13:32:27.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.492+0000 7fac157fa640 1 --2- 172.21.15.12:0/2910804741 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fac040a3d20 0x7fac040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.492+0000 7fac17fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d66cce3c00 0x55d66ccf0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.492+0000 7fac17fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d66cce3c00 0x55d66ccf0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33952/0 (socket says 172.21.15.12:33952) 2024-04-22T13:32:27.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.492+0000 7fac17fff640 1 -- 172.21.15.12:0/47960533 learned_addr learned my addr 172.21.15.12:0/47960533 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:27.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.492+0000 7fac1cd03640 1 --2- 172.21.15.12:0/47960533 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d66cce3250 0x55d66cce3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.492+0000 7fac1cd03640 1 --2- 172.21.15.12:0/2910804741 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fac040a3d20 0x7fac040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.492+0000 7fac177fe640 1 --2- 172.21.15.12:0/47960533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d66ccf0e10 0x55d66ccf3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.492+0000 7fac157fa640 1 -- 172.21.15.12:0/2910804741 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7fac000d0da0 con 0x7fac1806a840 2024-04-22T13:32:27.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.493+0000 7fac17fff640 1 -- 172.21.15.12:0/47960533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d66ccf0e10 msgr2=0x55d66ccf3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.493+0000 7fac17fff640 1 --2- 172.21.15.12:0/47960533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d66ccf0e10 0x55d66ccf3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.493+0000 7fac17fff640 1 -- 172.21.15.12:0/47960533 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d66cce3250 msgr2=0x55d66cce3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.493+0000 7fac17fff640 1 --2- 172.21.15.12:0/47960533 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d66cce3250 0x55d66cce3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.493+0000 7fac17fff640 1 -- 172.21.15.12:0/47960533 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d66cbd71b0 con 0x55d66cce3c00 2024-04-22T13:32:27.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.493+0000 7fac1cd03640 1 --2- 172.21.15.12:0/47960533 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d66cce3250 0x55d66cce3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:32:27.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.493+0000 7fac17fff640 1 --2- 172.21.15.12:0/47960533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d66cce3c00 0x55d66ccf0840 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fac000ce510 tx=0x7fac00021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f33787cbb910f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:27.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.493+0000 7fac1cd03640 1 --2- 172.21.15.12:0/2910804741 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fac040a3d20 0x7fac040a61e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fac10033000 tx=0x7fac100095c0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:27.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.493+0000 7fac0e7fc640 1 -- 172.21.15.12:0/47960533 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac00005180 con 0x55d66cce3c00 2024-04-22T13:32:27.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.493+0000 7fac0e7fc640 1 -- 172.21.15.12:0/47960533 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac000a10b0 con 0x55d66cce3c00 2024-04-22T13:32:27.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.493+0000 7fac0e7fc640 1 -- 172.21.15.12:0/47960533 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac00099370 con 0x55d66cce3c00 2024-04-22T13:32:27.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.494+0000 7fac21431740 1 -- 172.21.15.12:0/47960533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d66cce3c00 msgr2=0x55d66ccf0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.494+0000 7fac21431740 1 --2- 172.21.15.12:0/47960533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d66cce3c00 0x55d66ccf0840 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fac000ce510 tx=0x7fac00021000 comp rx=0 tx=0).stop 2024-04-22T13:32:27.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.494+0000 7fac21431740 1 -- 172.21.15.12:0/47960533 shutdown_connections 2024-04-22T13:32:27.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.494+0000 7fac21431740 1 --2- 172.21.15.12:0/47960533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d66ccf0e10 0x55d66ccf3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.494+0000 7fac21431740 1 --2- 172.21.15.12:0/47960533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d66cce3c00 0x55d66ccf0840 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.494+0000 7fac21431740 1 --2- 172.21.15.12:0/47960533 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d66cce3250 0x55d66cce3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.494+0000 7fac21431740 1 -- 172.21.15.12:0/47960533 >> 172.21.15.12:0/47960533 conn(0x55d66ccddba0 msgr2=0x55d66ccddf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:27.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.494+0000 7fac21431740 1 -- 172.21.15.12:0/47960533 shutdown_connections 2024-04-22T13:32:27.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.494+0000 7fac21431740 1 -- 172.21.15.12:0/47960533 wait complete. 2024-04-22T13:32:27.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.494+0000 7fac21431740 1 -- start start 2024-04-22T13:32:27.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.494+0000 7fac21431740 1 Processor -- start 2024-04-22T13:32:27.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.495+0000 7fac21431740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d66cce3250 0x55d66ccf06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.495+0000 7fac21431740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d66cce3c00 0x55d66cceda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.495+0000 7fac21431740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d66ccf0e10 0x55d66ccedfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.495+0000 7fac21431740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55d66ccf3d40 con 0x55d66cce3c00 2024-04-22T13:32:27.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.495+0000 7fac21431740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d66cc01500 con 0x55d66cce3250 2024-04-22T13:32:27.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.495+0000 7fac21431740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d66ccf3ec0 con 0x55d66ccf0e10 2024-04-22T13:32:27.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.495+0000 7fac21431740 10 client.0 _ll_register_callbacks cb 0x55d66cbcfb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:27.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.495+0000 7fac177fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d66cce3c00 0x55d66cceda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.495+0000 7fac177fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d66cce3c00 0x55d66cceda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33958/0 (socket says 172.21.15.12:33958) 2024-04-22T13:32:27.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.495+0000 7fac177fe640 1 -- 172.21.15.12:0/407297397 learned_addr learned my addr 172.21.15.12:0/407297397 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:27.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.495+0000 7fac17fff640 1 --2- 172.21.15.12:0/407297397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d66cce3250 0x55d66ccf06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.495+0000 7fac1cd03640 1 --2- 172.21.15.12:0/407297397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d66ccf0e10 0x55d66ccedfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.495+0000 7fac177fe640 1 -- 172.21.15.12:0/407297397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d66ccf0e10 msgr2=0x55d66ccedfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.495+0000 7fac177fe640 1 --2- 172.21.15.12:0/407297397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d66ccf0e10 0x55d66ccedfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.495+0000 7fac177fe640 1 -- 172.21.15.12:0/407297397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d66cce3250 msgr2=0x55d66ccf06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.495+0000 7fac177fe640 1 --2- 172.21.15.12:0/407297397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d66cce3250 0x55d66ccf06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.495+0000 7fac177fe640 1 -- 172.21.15.12:0/407297397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d66c95a3f0 con 0x55d66cce3c00 2024-04-22T13:32:27.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.495+0000 7fac1cd03640 1 --2- 172.21.15.12:0/407297397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d66ccf0e10 0x55d66ccedfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:27.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.495+0000 7fac177fe640 1 --2- 172.21.15.12:0/407297397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d66cce3c00 0x55d66cceda80 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fac08010820 tx=0x7fac08010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae2cf84b5c42a66 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:27.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.495+0000 7fabf77fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:32:27.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.495+0000 7fabf77fe640 1 -- 172.21.15.12:0/407297397 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0800eda0 con 0x55d66cce3c00 2024-04-22T13:32:27.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.495+0000 7fabf77fe640 1 -- 172.21.15.12:0/407297397 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7fac08005020 con 0x55d66cce3c00 2024-04-22T13:32:27.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.495+0000 7fac21431740 1 -- 172.21.15.12:0/407297397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d66cbe7930 con 0x55d66cce3c00 2024-04-22T13:32:27.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.496+0000 7fabf77fe640 1 -- 172.21.15.12:0/407297397 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fac08005220 con 0x55d66cce3c00 2024-04-22T13:32:27.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.496+0000 7fac21431740 10 client.18561 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:27.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.496+0000 7fac21431740 1 -- 172.21.15.12:0/407297397 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d66ca247d0 con 0x55d66cce3c00 2024-04-22T13:32:27.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.496+0000 7fabf77fe640 1 -- 172.21.15.12:0/407297397 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d66cbe7930 con 0x55d66cce3c00 2024-04-22T13:32:27.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.496+0000 7fabf77fe640 1 -- 172.21.15.12:0/407297397 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fac0801f020 con 0x55d66cce3c00 2024-04-22T13:32:27.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.496+0000 7fac21431740 10 client.18561 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:27.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.496+0000 7fac21431740 10 client.18561 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:32:27.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.496+0000 7fac21431740 20 client.18561 populate_metadata read hostname 'smithi012' 2024-04-22T13:32:27.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.496+0000 7fac21431740 1 --2- 172.21.15.12:0/407297397 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55d66cd01110 0x55d66cd03530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.496+0000 7fac21431740 4 client.18561 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:27.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.496+0000 7fac21431740 1 -- 172.21.15.12:0/407297397 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d66cc55c10 con 0x55d66cd01110 2024-04-22T13:32:27.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.496+0000 7fac17fff640 1 --2- 172.21.15.12:0/407297397 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55d66cd01110 0x55d66cd03530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.497+0000 7fabf77fe640 10 client.18561 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:32:27.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.497+0000 7fac17fff640 1 --2- 172.21.15.12:0/407297397 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55d66cd01110 0x55d66cd03530 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a0fe755bbf9e7147 server_cookie=85ca42d32cab9ec5 in_seq=0 out_seq=0 2024-04-22T13:32:27.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.497+0000 7fabf77fe640 1 -- 172.21.15.12:0/407297397 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d66cc55c10 con 0x55d66cd01110 2024-04-22T13:32:27.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.497+0000 7fabf77fe640 10 client.18561 handle_command_reply: tid=0 2024-04-22T13:32:27.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.497+0000 7fac21431740 1 client.18561 shutdown 2024-04-22T13:32:27.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.498+0000 7fac21431740 1 -- 172.21.15.12:0/407297397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d66cce3c00 msgr2=0x55d66cceda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.498+0000 7fac21431740 1 --2- 172.21.15.12:0/407297397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d66cce3c00 0x55d66cceda80 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fac08010820 tx=0x7fac08010ce0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.498+0000 7fac21431740 1 -- 172.21.15.12:0/407297397 shutdown_connections 2024-04-22T13:32:27.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.498+0000 7fac21431740 1 --2- 172.21.15.12:0/407297397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d66ccf0e10 0x55d66ccedfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.498+0000 7fac21431740 1 --2- 172.21.15.12:0/407297397 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55d66cd01110 0x55d66cd03530 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.498+0000 7fac21431740 1 --2- 172.21.15.12:0/407297397 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d66cce3c00 0x55d66cceda80 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.498+0000 7fabf77fe640 0 client.18561 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:32:27.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.498+0000 7fac21431740 1 --2- 172.21.15.12:0/407297397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d66cce3250 0x55d66ccf06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.498+0000 7fac21431740 1 -- 172.21.15.12:0/407297397 >> 172.21.15.12:0/407297397 conn(0x55d66ccddba0 msgr2=0x55d66ccdf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:27.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.498+0000 7fac21431740 1 -- 172.21.15.12:0/407297397 shutdown_connections 2024-04-22T13:32:27.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.498+0000 7fac21431740 1 -- 172.21.15.12:0/407297397 wait complete. 2024-04-22T13:32:27.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.498+0000 7fac21431740 20 client.18561 trim_cache size 0 max 16384 2024-04-22T13:32:27.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.511+0000 7fac21431740 1 Processor -- start 2024-04-22T13:32:27.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.511+0000 7fac21431740 1 -- start start 2024-04-22T13:32:27.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.511+0000 7fac21431740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d66cce3250 0x55d66ccfcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.511+0000 7fac21431740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d66cce3c00 0x55d66ccf01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.511+0000 7fac21431740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d66ccf0e10 0x55d66ccf06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.511+0000 7fac21431740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fac181ab880 con 0x55d66ccf0e10 2024-04-22T13:32:27.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.511+0000 7fac21431740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fac181ad3c0 con 0x55d66cce3c00 2024-04-22T13:32:27.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.511+0000 7fac21431740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fac181abf50 con 0x55d66cce3250 2024-04-22T13:32:27.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.511+0000 7fac1cd03640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d66ccf0e10 0x55d66ccf06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.512+0000 7fac1cd03640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d66ccf0e10 0x55d66ccf06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33968/0 (socket says 172.21.15.12:33968) 2024-04-22T13:32:27.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.512+0000 7fac1cd03640 1 -- 172.21.15.12:0/667671682 learned_addr learned my addr 172.21.15.12:0/667671682 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:27.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.512+0000 7fac17fff640 1 --2- 172.21.15.12:0/667671682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d66cce3250 0x55d66ccfcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.512+0000 7fac1cd03640 1 -- 172.21.15.12:0/667671682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d66cce3250 msgr2=0x55d66ccfcc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.512+0000 7fac1cd03640 1 --2- 172.21.15.12:0/667671682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d66cce3250 0x55d66ccfcc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.512+0000 7fac177fe640 1 --2- 172.21.15.12:0/667671682 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d66cce3c00 0x55d66ccf01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.512+0000 7fac1cd03640 1 -- 172.21.15.12:0/667671682 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d66cce3c00 msgr2=0x55d66ccf01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.512+0000 7fac1cd03640 1 --2- 172.21.15.12:0/667671682 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d66cce3c00 0x55d66ccf01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.512+0000 7fac1cd03640 1 -- 172.21.15.12:0/667671682 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d66ccebfd0 con 0x55d66ccf0e10 2024-04-22T13:32:27.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.512+0000 7fac17fff640 1 --2- 172.21.15.12:0/667671682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d66cce3250 0x55d66ccfcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:27.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.512+0000 7fac177fe640 1 --2- 172.21.15.12:0/667671682 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d66cce3c00 0x55d66ccf01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:27.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.512+0000 7fac1cd03640 1 --2- 172.21.15.12:0/667671682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d66ccf0e10 0x55d66ccf06f0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fac10036000 tx=0x7fac1003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375316350262465e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:27.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.512+0000 7fac0dffb640 1 -- 172.21.15.12:0/667671682 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10053070 con 0x55d66ccf0e10 2024-04-22T13:32:27.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.512+0000 7fac0dffb640 1 -- 172.21.15.12:0/667671682 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac1003a040 con 0x55d66ccf0e10 2024-04-22T13:32:27.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.512+0000 7fac0dffb640 1 -- 172.21.15.12:0/667671682 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10039070 con 0x55d66ccf0e10 2024-04-22T13:32:27.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.513+0000 7fac21431740 1 -- 172.21.15.12:0/667671682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d66ccf0e10 msgr2=0x55d66ccf06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.513+0000 7fac21431740 1 --2- 172.21.15.12:0/667671682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d66ccf0e10 0x55d66ccf06f0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fac10036000 tx=0x7fac1003e000 comp rx=0 tx=0).stop 2024-04-22T13:32:27.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.513+0000 7fac21431740 1 -- 172.21.15.12:0/667671682 shutdown_connections 2024-04-22T13:32:27.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.513+0000 7fac21431740 1 --2- 172.21.15.12:0/667671682 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d66ccf0e10 0x55d66ccf06f0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.513+0000 7fac21431740 1 --2- 172.21.15.12:0/667671682 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d66cce3c00 0x55d66ccf01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.513+0000 7fac21431740 1 --2- 172.21.15.12:0/667671682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d66cce3250 0x55d66ccfcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.513+0000 7fac21431740 1 -- 172.21.15.12:0/667671682 >> 172.21.15.12:0/667671682 conn(0x55d66ccddba0 msgr2=0x55d66cce9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:27.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.513+0000 7fac21431740 1 -- 172.21.15.12:0/667671682 shutdown_connections 2024-04-22T13:32:27.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.513+0000 7fac21431740 1 -- 172.21.15.12:0/667671682 wait complete. 2024-04-22T13:32:27.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.513+0000 7fac21431740 1 -- start start 2024-04-22T13:32:27.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.513+0000 7fac21431740 1 Processor -- start 2024-04-22T13:32:27.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.514+0000 7fac21431740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d66cce3250 0x55d66ccf4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.514+0000 7fac21431740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d66cce3c00 0x55d66ccf53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.514+0000 7fac21431740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d66ccf0e10 0x55d66ccf5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.514+0000 7fac21431740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fac181accf0 con 0x55d66cce3c00 2024-04-22T13:32:27.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.514+0000 7fac21431740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fac181ac620 con 0x55d66ccf0e10 2024-04-22T13:32:27.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.514+0000 7fac21431740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fac181a98a0 con 0x55d66cce3250 2024-04-22T13:32:27.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.514+0000 7fac21431740 10 client.0 _ll_register_callbacks cb 0x7fac08010430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:27.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.514+0000 7fac177fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d66cce3c00 0x55d66ccf53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.514+0000 7fac177fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d66cce3c00 0x55d66ccf53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33982/0 (socket says 172.21.15.12:33982) 2024-04-22T13:32:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.514+0000 7fac177fe640 1 -- 172.21.15.12:0/1235319745 learned_addr learned my addr 172.21.15.12:0/1235319745 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.514+0000 7fac1cd03640 1 --2- 172.21.15.12:0/1235319745 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d66ccf0e10 0x55d66ccf5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.514+0000 7fac17fff640 1 --2- 172.21.15.12:0/1235319745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d66cce3250 0x55d66ccf4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.514+0000 7fac177fe640 1 -- 172.21.15.12:0/1235319745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d66cce3250 msgr2=0x55d66ccf4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.514+0000 7fac177fe640 1 --2- 172.21.15.12:0/1235319745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d66cce3250 0x55d66ccf4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.514+0000 7fac177fe640 1 -- 172.21.15.12:0/1235319745 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d66ccf0e10 msgr2=0x55d66ccf5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.514+0000 7fac177fe640 1 --2- 172.21.15.12:0/1235319745 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d66ccf0e10 0x55d66ccf5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.514+0000 7fac177fe640 1 -- 172.21.15.12:0/1235319745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d66cd668c0 con 0x55d66cce3c00 2024-04-22T13:32:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.514+0000 7fac177fe640 1 --2- 172.21.15.12:0/1235319745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d66cce3c00 0x55d66ccf53e0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x55d66cced2e0 tx=0x7fac0800e8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7992b537b725cf23 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:27.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.514+0000 7fabf77fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:32:27.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.514+0000 7fabf77fe640 1 -- 172.21.15.12:0/1235319745 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0802e060 con 0x55d66cce3c00 2024-04-22T13:32:27.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.514+0000 7fabf77fe640 1 -- 172.21.15.12:0/1235319745 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7fac080053c0 con 0x55d66cce3c00 2024-04-22T13:32:27.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.514+0000 7fac21431740 1 -- 172.21.15.12:0/1235319745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac1003a040 con 0x55d66cce3c00 2024-04-22T13:32:27.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.515+0000 7fabf77fe640 1 -- 172.21.15.12:0/1235319745 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fac080055e0 con 0x55d66cce3c00 2024-04-22T13:32:27.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.515+0000 7fac21431740 10 client.18567 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:27.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.515+0000 7fac21431740 1 -- 172.21.15.12:0/1235319745 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d66cd67b30 con 0x55d66cce3c00 2024-04-22T13:32:27.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.515+0000 7fabf77fe640 1 -- 172.21.15.12:0/1235319745 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac1003a040 con 0x55d66cce3c00 2024-04-22T13:32:27.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.515+0000 7fabf77fe640 1 -- 172.21.15.12:0/1235319745 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fac08004660 con 0x55d66cce3c00 2024-04-22T13:32:27.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.515+0000 7fac21431740 10 client.18567 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:27.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.515+0000 7fac21431740 10 client.18567 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:32:27.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.515+0000 7fac21431740 20 client.18567 populate_metadata read hostname 'smithi012' 2024-04-22T13:32:27.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.515+0000 7fac21431740 1 --2- 172.21.15.12:0/1235319745 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55d66cd682e0 0x55d66cd6a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.515+0000 7fac21431740 4 client.18567 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:32:27.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.515+0000 7fac21431740 1 -- 172.21.15.12:0/1235319745 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d66cc4fa80 con 0x55d66cd682e0 2024-04-22T13:32:27.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.515+0000 7fac17fff640 1 --2- 172.21.15.12:0/1235319745 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55d66cd682e0 0x55d66cd6a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.516+0000 7fabf77fe640 10 client.18567 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:32:27.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.516+0000 7fac17fff640 1 --2- 172.21.15.12:0/1235319745 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55d66cd682e0 0x55d66cd6a700 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9f78052983586b43 server_cookie=a9c7f3c4e58d014a in_seq=0 out_seq=0 2024-04-22T13:32:27.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.516+0000 7fabf77fe640 1 -- 172.21.15.12:0/1235319745 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d66cc4fa80 con 0x55d66cd682e0 2024-04-22T13:32:27.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.516+0000 7fabf77fe640 10 client.18567 handle_command_reply: tid=0 2024-04-22T13:32:27.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.516+0000 7fac21431740 1 client.18567 shutdown 2024-04-22T13:32:27.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.516+0000 7fac21431740 1 -- 172.21.15.12:0/1235319745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d66cce3c00 msgr2=0x55d66ccf53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.516+0000 7fac21431740 1 --2- 172.21.15.12:0/1235319745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d66cce3c00 0x55d66ccf53e0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x55d66cced2e0 tx=0x7fac0800e8c0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.516+0000 7fac21431740 1 -- 172.21.15.12:0/1235319745 shutdown_connections 2024-04-22T13:32:27.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.516+0000 7fac21431740 1 --2- 172.21.15.12:0/1235319745 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d66ccf0e10 0x55d66ccf5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.516+0000 7fac21431740 1 --2- 172.21.15.12:0/1235319745 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55d66cd682e0 0x55d66cd6a700 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.516+0000 7fac21431740 1 --2- 172.21.15.12:0/1235319745 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d66cce3c00 0x55d66ccf53e0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.516+0000 7fabf77fe640 0 client.18567 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:32:27.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.516+0000 7fac21431740 1 --2- 172.21.15.12:0/1235319745 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d66cce3250 0x55d66ccf4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.516+0000 7fac21431740 1 -- 172.21.15.12:0/1235319745 >> 172.21.15.12:0/1235319745 conn(0x55d66ccddba0 msgr2=0x55d66ccdc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:27.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.516+0000 7fac21431740 1 -- 172.21.15.12:0/1235319745 shutdown_connections 2024-04-22T13:32:27.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.516+0000 7fac21431740 1 -- 172.21.15.12:0/1235319745 wait complete. 2024-04-22T13:32:27.521 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.516+0000 7fac21431740 20 client.18567 trim_cache size 0 max 16384 2024-04-22T13:32:27.522 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:32:27.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.517+0000 7fac1e766640 1 -- 172.21.15.12:0/2910804741 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fac040a3d20 msgr2=0x7fac040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.517+0000 7fac1e766640 1 --2- 172.21.15.12:0/2910804741 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fac040a3d20 0x7fac040a61e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fac10033000 tx=0x7fac100095c0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.517+0000 7fac1e766640 1 -- 172.21.15.12:0/2910804741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac1806a840 msgr2=0x7fac181986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.517+0000 7fac1e766640 1 --2- 172.21.15.12:0/2910804741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac1806a840 0x7fac181986b0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fac0000e9f0 tx=0x7fac0000edf0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.517+0000 7fac1e766640 1 -- 172.21.15.12:0/2910804741 shutdown_connections 2024-04-22T13:32:27.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.517+0000 7fac1e766640 1 --2- 172.21.15.12:0/2910804741 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fac040a3d20 0x7fac040a61e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.517+0000 7fac1e766640 1 --2- 172.21.15.12:0/2910804741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac1819cf70 0x7fac1819d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.517+0000 7fac1e766640 1 --2- 172.21.15.12:0/2910804741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac18107270 0x7fac18198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.517+0000 7fac1e766640 1 --2- 172.21.15.12:0/2910804741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac1806a840 0x7fac181986b0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.517+0000 7fac1e766640 1 -- 172.21.15.12:0/2910804741 >> 172.21.15.12:0/2910804741 conn(0x7fac18077eb0 msgr2=0x7fac180fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.517+0000 7fac1e766640 1 -- 172.21.15.12:0/2910804741 shutdown_connections 2024-04-22T13:32:27.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.517+0000 7fac1e766640 1 -- 172.21.15.12:0/2910804741 wait complete. 2024-04-22T13:32:27.525 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:32:27.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.634+0000 7f64e8da4640 1 -- 172.21.15.12:0/244796737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64e4101a50 msgr2=0x7f64e4105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.630 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.634+0000 7f64e8da4640 1 --2- 172.21.15.12:0/244796737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64e4101a50 0x7f64e4105a90 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f64d8009900 tx=0x7f64d802bc30 comp rx=0 tx=0).stop 2024-04-22T13:32:27.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.634+0000 7f64e8da4640 1 -- 172.21.15.12:0/244796737 shutdown_connections 2024-04-22T13:32:27.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.634+0000 7f64e8da4640 1 --2- 172.21.15.12:0/244796737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64e4106170 0x7f64e410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.634+0000 7f64e8da4640 1 --2- 172.21.15.12:0/244796737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64e4101a50 0x7f64e4105a90 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.634+0000 7f64e8da4640 1 --2- 172.21.15.12:0/244796737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e41010a0 0x7f64e4101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.634+0000 7f64e8da4640 1 -- 172.21.15.12:0/244796737 >> 172.21.15.12:0/244796737 conn(0x7f64e40771d0 msgr2=0x7f64e40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:27.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.634+0000 7f64e8da4640 1 -- 172.21.15.12:0/244796737 shutdown_connections 2024-04-22T13:32:27.631 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.634+0000 7f64e8da4640 1 -- 172.21.15.12:0/244796737 wait complete. 2024-04-22T13:32:27.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.635+0000 7f64e8da4640 1 Processor -- start 2024-04-22T13:32:27.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.635+0000 7f64e8da4640 1 -- start start 2024-04-22T13:32:27.633 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.635+0000 7f64e8da4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64e41010a0 0x7f64e4198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.635+0000 7f64e8da4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e4106170 0x7f64e4198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.635+0000 7f64e8da4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64e419cfd0 0x7f64e419d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.635+0000 7f64e8da4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f64e4111090 con 0x7f64e419cfd0 2024-04-22T13:32:27.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.635+0000 7f64e8da4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f64e4110f10 con 0x7f64e41010a0 2024-04-22T13:32:27.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.635+0000 7f64e8da4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f64e4111210 con 0x7f64e4106170 2024-04-22T13:32:27.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.635+0000 7f64e2d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64e419cfd0 0x7f64e419d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.635+0000 7f64e2d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64e419cfd0 0x7f64e419d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34004/0 (socket says 172.21.15.12:34004) 2024-04-22T13:32:27.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.635+0000 7f64e2d9d640 1 -- 172.21.15.12:0/1668862358 learned_addr learned my addr 172.21.15.12:0/1668862358 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:27.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.635+0000 7f64e259c640 1 --2- 172.21.15.12:0/1668862358 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64e41010a0 0x7f64e4198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.635+0000 7f64e1d9b640 1 --2- 172.21.15.12:0/1668862358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e4106170 0x7f64e4198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.635+0000 7f64e2d9d640 1 -- 172.21.15.12:0/1668862358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e4106170 msgr2=0x7f64e4198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.634 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.635+0000 7f64e2d9d640 1 --2- 172.21.15.12:0/1668862358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e4106170 0x7f64e4198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.635+0000 7f64e2d9d640 1 -- 172.21.15.12:0/1668862358 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64e41010a0 msgr2=0x7f64e4198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.635+0000 7f64e2d9d640 1 --2- 172.21.15.12:0/1668862358 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64e41010a0 0x7f64e4198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.635+0000 7f64e2d9d640 1 -- 172.21.15.12:0/1668862358 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e419dac0 con 0x7f64e419cfd0 2024-04-22T13:32:27.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.636+0000 7f64e2d9d640 1 --2- 172.21.15.12:0/1668862358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64e419cfd0 0x7f64e419d480 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00eaa0 tx=0x7f64dc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:27.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.636+0000 7f64d37fe640 1 -- 172.21.15.12:0/1668862358 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc00cdb0 con 0x7f64e419cfd0 2024-04-22T13:32:27.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.636+0000 7f64d37fe640 1 -- 172.21.15.12:0/1668862358 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64dc005020 con 0x7f64e419cfd0 2024-04-22T13:32:27.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.636+0000 7f64e8da4640 1 -- 172.21.15.12:0/1668862358 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64e419ddb0 con 0x7f64e419cfd0 2024-04-22T13:32:27.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.636+0000 7f64d37fe640 1 -- 172.21.15.12:0/1668862358 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc0116d0 con 0x7f64e419cfd0 2024-04-22T13:32:27.635 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.636+0000 7f64e8da4640 1 -- 172.21.15.12:0/1668862358 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64e41a5520 con 0x7f64e419cfd0 2024-04-22T13:32:27.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.638+0000 7f64d37fe640 1 -- 172.21.15.12:0/1668862358 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f64dc0051d0 con 0x7f64e419cfd0 2024-04-22T13:32:27.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.638+0000 7f64eb996740 1 Processor -- start 2024-04-22T13:32:27.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.639+0000 7f64eb996740 1 -- start start 2024-04-22T13:32:27.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.639+0000 7f64eb996740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dada1b94d0 0x55dada1b98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.639+0000 7f64eb996740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dada1b9e80 0x55dada1c2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.639+0000 7f64eb996740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dada1c2a00 0x55dada1c4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.639+0000 7f64eb996740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55dada11e1f0 con 0x55dada1c2a00 2024-04-22T13:32:27.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.639+0000 7f64eb996740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55dada0bb4b0 con 0x55dada1b94d0 2024-04-22T13:32:27.642 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.639+0000 7f64eb996740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dada11a1b0 con 0x55dada1b9e80 2024-04-22T13:32:27.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.639+0000 7f64e1d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dada1c2a00 0x55dada1c4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.639+0000 7f64e1d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dada1c2a00 0x55dada1c4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34012/0 (socket says 172.21.15.12:34012) 2024-04-22T13:32:27.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.639+0000 7f64e1d9b640 1 -- 172.21.15.12:0/262543262 learned_addr learned my addr 172.21.15.12:0/262543262 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:27.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.639+0000 7f64d37fe640 1 --2- 172.21.15.12:0/1668862358 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f64cc0a3d20 0x7f64cc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.639+0000 7f64e259c640 1 --2- 172.21.15.12:0/262543262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dada1b9e80 0x55dada1c2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.639+0000 7f64e1d9b640 1 --2- 172.21.15.12:0/262543262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dada1b94d0 0x55dada1b98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.639+0000 7f64e2d9d640 1 --2- 172.21.15.12:0/1668862358 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f64cc0a3d20 0x7f64cc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.639+0000 7f64e1d9b640 1 -- 172.21.15.12:0/262543262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dada1b9e80 msgr2=0x55dada1c2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.639+0000 7f64e1d9b640 1 --2- 172.21.15.12:0/262543262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dada1b9e80 0x55dada1c2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.639+0000 7f64e1d9b640 1 -- 172.21.15.12:0/262543262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dada1b94d0 msgr2=0x55dada1b98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.639+0000 7f64e1d9b640 1 --2- 172.21.15.12:0/262543262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dada1b94d0 0x55dada1b98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.639+0000 7f64e1d9b640 1 -- 172.21.15.12:0/262543262 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dada0ad1b0 con 0x55dada1c2a00 2024-04-22T13:32:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.639+0000 7f64d37fe640 1 -- 172.21.15.12:0/1668862358 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f64dc0d08d0 con 0x7f64e419cfd0 2024-04-22T13:32:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.640+0000 7f64e1d9b640 1 --2- 172.21.15.12:0/262543262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dada1c2a00 0x55dada1c4e50 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f64d80096c0 tx=0x7f64d8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3d0549c359788b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.640+0000 7f64e2d9d640 1 --2- 172.21.15.12:0/1668862358 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f64cc0a3d20 0x7f64cc0a61e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f64dc098da0 tx=0x7f64dc06e380 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.640+0000 7f64d0ff9640 1 -- 172.21.15.12:0/262543262 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d8047020 con 0x55dada1c2a00 2024-04-22T13:32:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.640+0000 7f64d0ff9640 1 -- 172.21.15.12:0/262543262 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64d802cc90 con 0x55dada1c2a00 2024-04-22T13:32:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.640+0000 7f64d0ff9640 1 -- 172.21.15.12:0/262543262 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d8004920 con 0x55dada1c2a00 2024-04-22T13:32:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.640+0000 7f64eb996740 1 -- 172.21.15.12:0/262543262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dada1c2a00 msgr2=0x55dada1c4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.640+0000 7f64eb996740 1 --2- 172.21.15.12:0/262543262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dada1c2a00 0x55dada1c4e50 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f64d80096c0 tx=0x7f64d8005e80 comp rx=0 tx=0).stop 2024-04-22T13:32:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.640+0000 7f64eb996740 1 -- 172.21.15.12:0/262543262 shutdown_connections 2024-04-22T13:32:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.640+0000 7f64eb996740 1 --2- 172.21.15.12:0/262543262 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dada1c2a00 0x55dada1c4e50 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.640+0000 7f64eb996740 1 --2- 172.21.15.12:0/262543262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dada1b9e80 0x55dada1c2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.640+0000 7f64eb996740 1 --2- 172.21.15.12:0/262543262 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dada1b94d0 0x55dada1b98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.640+0000 7f64eb996740 1 -- 172.21.15.12:0/262543262 >> 172.21.15.12:0/262543262 conn(0x55dada1b3ba0 msgr2=0x55dada1b3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.641+0000 7f64eb996740 1 -- 172.21.15.12:0/262543262 shutdown_connections 2024-04-22T13:32:27.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.641+0000 7f64eb996740 1 -- 172.21.15.12:0/262543262 wait complete. 2024-04-22T13:32:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.641+0000 7f64eb996740 1 -- start start 2024-04-22T13:32:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.641+0000 7f64eb996740 1 Processor -- start 2024-04-22T13:32:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.641+0000 7f64eb996740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dada1b94d0 0x55dada1c2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.641+0000 7f64eb996740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dada1b9e80 0x55dada1bf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.641+0000 7f64eb996740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dada1c2a00 0x55dada1bfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.641+0000 7f64eb996740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55dada1c5910 con 0x55dada1b94d0 2024-04-22T13:32:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.641+0000 7f64eb996740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55dada0d7500 con 0x55dada1b9e80 2024-04-22T13:32:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.641+0000 7f64eb996740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dada1c5a90 con 0x55dada1c2a00 2024-04-22T13:32:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.641+0000 7f64e1d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dada1b94d0 0x55dada1c2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.641+0000 7f64eb996740 10 client.0 _ll_register_callbacks cb 0x55dada0a5b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.641+0000 7f64e1d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dada1b94d0 0x55dada1c2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34018/0 (socket says 172.21.15.12:34018) 2024-04-22T13:32:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.641+0000 7f64e1d9b640 1 -- 172.21.15.12:0/2734170009 learned_addr learned my addr 172.21.15.12:0/2734170009 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.642+0000 7f64e1d9b640 1 -- 172.21.15.12:0/2734170009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dada1c2a00 msgr2=0x55dada1bfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.642+0000 7f64e1d9b640 1 --2- 172.21.15.12:0/2734170009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dada1c2a00 0x55dada1bfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.642+0000 7f64e1d9b640 1 -- 172.21.15.12:0/2734170009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dada1b9e80 msgr2=0x55dada1bf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.642+0000 7f64e1d9b640 1 --2- 172.21.15.12:0/2734170009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dada1b9e80 0x55dada1bf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.642+0000 7f64e1d9b640 1 -- 172.21.15.12:0/2734170009 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dad9e303f0 con 0x55dada1b94d0 2024-04-22T13:32:27.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.642+0000 7f64e1d9b640 1 --2- 172.21.15.12:0/2734170009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dada1b94d0 0x55dada1c2320 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f64d802c170 tx=0x7f64d802cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a4dc4c3c64979c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:27.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.642+0000 7f64c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:32:27.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.642+0000 7f64c9ffb640 1 -- 172.21.15.12:0/2734170009 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d8051070 con 0x55dada1b94d0 2024-04-22T13:32:27.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.642+0000 7f64c9ffb640 1 -- 172.21.15.12:0/2734170009 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f64d8003c30 con 0x55dada1b94d0 2024-04-22T13:32:27.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.642+0000 7f64eb996740 1 -- 172.21.15.12:0/2734170009 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64d802cc90 con 0x55dada1b94d0 2024-04-22T13:32:27.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.642+0000 7f64c9ffb640 1 -- 172.21.15.12:0/2734170009 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f64d805d050 con 0x55dada1b94d0 2024-04-22T13:32:27.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.642+0000 7f64eb996740 10 client.18579 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:27.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.642+0000 7f64eb996740 1 -- 172.21.15.12:0/2734170009 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dad9efa7d0 con 0x55dada1b94d0 2024-04-22T13:32:27.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.642+0000 7f64c9ffb640 1 -- 172.21.15.12:0/2734170009 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64d802cc90 con 0x55dada1b94d0 2024-04-22T13:32:27.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.642+0000 7f64c9ffb640 1 -- 172.21.15.12:0/2734170009 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f64d804c450 con 0x55dada1b94d0 2024-04-22T13:32:27.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.643+0000 7f64eb996740 10 client.18579 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.643+0000 7f64eb996740 10 client.18579 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:32:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.643+0000 7f64eb996740 20 client.18579 populate_metadata read hostname 'smithi012' 2024-04-22T13:32:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.643+0000 7f64eb996740 1 --2- 172.21.15.12:0/2734170009 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55dada1d2a40 0x55dada1d4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.643+0000 7f64eb996740 4 client.18579 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.643+0000 7f64eb996740 1 -- 172.21.15.12:0/2734170009 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dada0bd930 con 0x55dada1d2a40 2024-04-22T13:32:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.643+0000 7f64e259c640 1 --2- 172.21.15.12:0/2734170009 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55dada1d2a40 0x55dada1d4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.643+0000 7f64e259c640 1 --2- 172.21.15.12:0/2734170009 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55dada1d2a40 0x55dada1d4e30 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=37cf665e4356af5d server_cookie=2f4ace20f31b2c59 in_seq=0 out_seq=0 2024-04-22T13:32:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.643+0000 7f64c9ffb640 10 client.18579 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:32:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.644+0000 7f64c9ffb640 1 -- 172.21.15.12:0/2734170009 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dada0bd930 con 0x55dada1d2a40 2024-04-22T13:32:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.644+0000 7f64c9ffb640 10 client.18579 handle_command_reply: tid=0 2024-04-22T13:32:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.644+0000 7f64eb996740 1 client.18579 shutdown 2024-04-22T13:32:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.644+0000 7f64eb996740 1 -- 172.21.15.12:0/2734170009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dada1b94d0 msgr2=0x55dada1c2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.644+0000 7f64eb996740 1 --2- 172.21.15.12:0/2734170009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dada1b94d0 0x55dada1c2320 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f64d802c170 tx=0x7f64d802cbe0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.644+0000 7f64eb996740 1 -- 172.21.15.12:0/2734170009 shutdown_connections 2024-04-22T13:32:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.644+0000 7f64eb996740 1 --2- 172.21.15.12:0/2734170009 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55dada1d2a40 0x55dada1d4e30 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.644+0000 7f64eb996740 1 --2- 172.21.15.12:0/2734170009 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dada1c2a00 0x55dada1bfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.644+0000 7f64eb996740 1 --2- 172.21.15.12:0/2734170009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dada1b9e80 0x55dada1bf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.644+0000 7f64c9ffb640 0 client.18579 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:32:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.644+0000 7f64eb996740 1 --2- 172.21.15.12:0/2734170009 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dada1b94d0 0x55dada1c2320 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.644+0000 7f64eb996740 1 -- 172.21.15.12:0/2734170009 >> 172.21.15.12:0/2734170009 conn(0x55dada1b3ba0 msgr2=0x55dada1b58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.645+0000 7f64eb996740 1 -- 172.21.15.12:0/2734170009 shutdown_connections 2024-04-22T13:32:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.645+0000 7f64eb996740 1 -- 172.21.15.12:0/2734170009 wait complete. 2024-04-22T13:32:27.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.645+0000 7f64eb996740 20 client.18579 trim_cache size 0 max 16384 2024-04-22T13:32:27.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.658+0000 7f64eb996740 1 Processor -- start 2024-04-22T13:32:27.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.658+0000 7f64eb996740 1 -- start start 2024-04-22T13:32:27.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.658+0000 7f64eb996740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dada1b94d0 0x55dada1c1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.658+0000 7f64eb996740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dada1b9e80 0x55dada1c2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.658+0000 7f64eb996740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dada1c2a00 0x55dada1d7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.658+0000 7f64eb996740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f64e41ad9d0 con 0x55dada1b94d0 2024-04-22T13:32:27.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.658+0000 7f64eb996740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f64e41af510 con 0x55dada1c2a00 2024-04-22T13:32:27.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.658+0000 7f64eb996740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f64e41ae0a0 con 0x55dada1b9e80 2024-04-22T13:32:27.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.658+0000 7f64e1d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dada1b94d0 0x55dada1c1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.658+0000 7f64e1d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dada1b94d0 0x55dada1c1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34022/0 (socket says 172.21.15.12:34022) 2024-04-22T13:32:27.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.658+0000 7f64e1d9b640 1 -- 172.21.15.12:0/2260892194 learned_addr learned my addr 172.21.15.12:0/2260892194 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:27.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.658+0000 7f64e259c640 1 --2- 172.21.15.12:0/2260892194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dada1b9e80 0x55dada1c2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.658+0000 7f64e1d9b640 1 --2- 172.21.15.12:0/2260892194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dada1c2a00 0x55dada1d7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.658+0000 7f64e1d9b640 1 -- 172.21.15.12:0/2260892194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dada1b9e80 msgr2=0x55dada1c2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.658+0000 7f64e1d9b640 1 --2- 172.21.15.12:0/2260892194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dada1b9e80 0x55dada1c2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.659+0000 7f64e1d9b640 1 -- 172.21.15.12:0/2260892194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dada1c2a00 msgr2=0x55dada1d7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.659+0000 7f64e1d9b640 1 --2- 172.21.15.12:0/2260892194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dada1c2a00 0x55dada1d7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.659+0000 7f64e1d9b640 1 -- 172.21.15.12:0/2260892194 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dada1bdc00 con 0x55dada1b94d0 2024-04-22T13:32:27.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.659+0000 7f64e1d9b640 1 --2- 172.21.15.12:0/2260892194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dada1b94d0 0x55dada1c1de0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f64d80058c0 tx=0x7f64d8005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabe9d8a25bb0bb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:27.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.659+0000 7f64cbfff640 1 -- 172.21.15.12:0/2260892194 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d8051040 con 0x55dada1b94d0 2024-04-22T13:32:27.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.659+0000 7f64cbfff640 1 -- 172.21.15.12:0/2260892194 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64d805a070 con 0x55dada1b94d0 2024-04-22T13:32:27.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.659+0000 7f64cbfff640 1 -- 172.21.15.12:0/2260892194 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d8047030 con 0x55dada1b94d0 2024-04-22T13:32:27.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.659+0000 7f64eb996740 1 -- 172.21.15.12:0/2260892194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dada1b94d0 msgr2=0x55dada1c1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.659+0000 7f64eb996740 1 --2- 172.21.15.12:0/2260892194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dada1b94d0 0x55dada1c1de0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f64d80058c0 tx=0x7f64d8005970 comp rx=0 tx=0).stop 2024-04-22T13:32:27.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.659+0000 7f64eb996740 1 -- 172.21.15.12:0/2260892194 shutdown_connections 2024-04-22T13:32:27.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.659+0000 7f64eb996740 1 --2- 172.21.15.12:0/2260892194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dada1c2a00 0x55dada1d7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.659+0000 7f64eb996740 1 --2- 172.21.15.12:0/2260892194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dada1b9e80 0x55dada1c2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.659+0000 7f64eb996740 1 --2- 172.21.15.12:0/2260892194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dada1b94d0 0x55dada1c1de0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.659+0000 7f64eb996740 1 -- 172.21.15.12:0/2260892194 >> 172.21.15.12:0/2260892194 conn(0x55dada1b3ba0 msgr2=0x55dada1ce570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:27.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.659+0000 7f64eb996740 1 -- 172.21.15.12:0/2260892194 shutdown_connections 2024-04-22T13:32:27.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.659+0000 7f64eb996740 1 -- 172.21.15.12:0/2260892194 wait complete. 2024-04-22T13:32:27.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.660+0000 7f64eb996740 1 -- start start 2024-04-22T13:32:27.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.660+0000 7f64eb996740 1 Processor -- start 2024-04-22T13:32:27.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.660+0000 7f64eb996740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dada1b94d0 0x55dada1d5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.660+0000 7f64eb996740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dada1b9e80 0x55dada1d5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.660+0000 7f64eb996740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dada1c2a00 0x55dada1d62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.660+0000 7f64eb996740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f64e41aee40 con 0x55dada1b94d0 2024-04-22T13:32:27.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.660+0000 7f64eb996740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f64e41ae770 con 0x55dada1c2a00 2024-04-22T13:32:27.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.660+0000 7f64eb996740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f64e41ab9d0 con 0x55dada1b9e80 2024-04-22T13:32:27.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.660+0000 7f64eb996740 10 client.0 _ll_register_callbacks cb 0x7f64d802cb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:27.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.660+0000 7f64e1d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dada1b94d0 0x55dada1d5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.660+0000 7f64e1d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dada1b94d0 0x55dada1d5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34038/0 (socket says 172.21.15.12:34038) 2024-04-22T13:32:27.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.660+0000 7f64e1d9b640 1 -- 172.21.15.12:0/2447170037 learned_addr learned my addr 172.21.15.12:0/2447170037 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:27.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.660+0000 7f64e259c640 1 --2- 172.21.15.12:0/2447170037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dada1b9e80 0x55dada1d5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.660+0000 7f64e1d9b640 1 -- 172.21.15.12:0/2447170037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dada1b9e80 msgr2=0x55dada1d5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.660+0000 7f64e1d9b640 1 --2- 172.21.15.12:0/2447170037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dada1b9e80 0x55dada1d5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.660+0000 7f64e1d9b640 1 -- 172.21.15.12:0/2447170037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dada1c2a00 msgr2=0x55dada1d62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.660+0000 7f64e1d9b640 1 --2- 172.21.15.12:0/2447170037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dada1c2a00 0x55dada1d62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.660+0000 7f64e1d9b640 1 -- 172.21.15.12:0/2447170037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dada237c20 con 0x55dada1b94d0 2024-04-22T13:32:27.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.661+0000 7f64e1d9b640 1 --2- 172.21.15.12:0/2447170037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dada1b94d0 0x55dada1d5870 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f64d8063040 tx=0x7f64d8005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54dc466d3fb8038 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:27.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.661+0000 7f64c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:32:27.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.661+0000 7f64c9ffb640 1 -- 172.21.15.12:0/2447170037 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d8047040 con 0x55dada1b94d0 2024-04-22T13:32:27.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.661+0000 7f64c9ffb640 1 -- 172.21.15.12:0/2447170037 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f64d804c650 con 0x55dada1b94d0 2024-04-22T13:32:27.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.661+0000 7f64eb996740 1 -- 172.21.15.12:0/2447170037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64d805a070 con 0x55dada1b94d0 2024-04-22T13:32:27.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.661+0000 7f64c9ffb640 1 -- 172.21.15.12:0/2447170037 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f64d8051040 con 0x55dada1b94d0 2024-04-22T13:32:27.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.661+0000 7f64eb996740 10 client.18585 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:27.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.661+0000 7f64eb996740 1 -- 172.21.15.12:0/2447170037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dada238ef0 con 0x55dada1b94d0 2024-04-22T13:32:27.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.661+0000 7f64c9ffb640 1 -- 172.21.15.12:0/2447170037 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64d805a070 con 0x55dada1b94d0 2024-04-22T13:32:27.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.661+0000 7f64c9ffb640 1 -- 172.21.15.12:0/2447170037 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f64d8041820 con 0x55dada1b94d0 2024-04-22T13:32:27.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.661+0000 7f64eb996740 10 client.18585 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:27.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.661+0000 7f64eb996740 10 client.18585 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:32:27.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.661+0000 7f64eb996740 20 client.18585 populate_metadata read hostname 'smithi012' 2024-04-22T13:32:27.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.661+0000 7f64eb996740 1 --2- 172.21.15.12:0/2447170037 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55dada2394f0 0x55dada23b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.661+0000 7f64eb996740 4 client.18585 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:32:27.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.661+0000 7f64eb996740 1 -- 172.21.15.12:0/2447170037 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dada12bc10 con 0x55dada2394f0 2024-04-22T13:32:27.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.662+0000 7f64e259c640 1 --2- 172.21.15.12:0/2447170037 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55dada2394f0 0x55dada23b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.662+0000 7f64e259c640 1 --2- 172.21.15.12:0/2447170037 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55dada2394f0 0x55dada23b910 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dbaaa91753df591b server_cookie=d6f18a343b8e3da5 in_seq=0 out_seq=0 2024-04-22T13:32:27.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.662+0000 7f64c9ffb640 10 client.18585 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:32:27.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.662+0000 7f64c9ffb640 1 -- 172.21.15.12:0/2447170037 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dada12bc10 con 0x55dada2394f0 2024-04-22T13:32:27.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.662+0000 7f64c9ffb640 10 client.18585 handle_command_reply: tid=0 2024-04-22T13:32:27.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.662+0000 7f64eb996740 1 client.18585 shutdown 2024-04-22T13:32:27.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.662+0000 7f64eb996740 1 -- 172.21.15.12:0/2447170037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dada1b94d0 msgr2=0x55dada1d5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.662+0000 7f64eb996740 1 --2- 172.21.15.12:0/2447170037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dada1b94d0 0x55dada1d5870 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f64d8063040 tx=0x7f64d8005c00 comp rx=0 tx=0).stop 2024-04-22T13:32:27.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.662+0000 7f64eb996740 1 -- 172.21.15.12:0/2447170037 shutdown_connections 2024-04-22T13:32:27.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.662+0000 7f64eb996740 1 --2- 172.21.15.12:0/2447170037 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55dada2394f0 0x55dada23b910 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.662+0000 7f64eb996740 1 --2- 172.21.15.12:0/2447170037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dada1c2a00 0x55dada1d62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.662+0000 7f64eb996740 1 --2- 172.21.15.12:0/2447170037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dada1b9e80 0x55dada1d5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.662+0000 7f64c9ffb640 0 client.18585 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:32:27.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.662+0000 7f64eb996740 1 --2- 172.21.15.12:0/2447170037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dada1b94d0 0x55dada1d5870 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.662+0000 7f64eb996740 1 -- 172.21.15.12:0/2447170037 >> 172.21.15.12:0/2447170037 conn(0x55dada1b3ba0 msgr2=0x55dada1ba2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:27.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.662+0000 7f64eb996740 1 -- 172.21.15.12:0/2447170037 shutdown_connections 2024-04-22T13:32:27.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.662+0000 7f64eb996740 1 -- 172.21.15.12:0/2447170037 wait complete. 2024-04-22T13:32:27.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.663+0000 7f64eb996740 20 client.18585 trim_cache size 0 max 16384 2024-04-22T13:32:27.666 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:32:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.663+0000 7f64e8da4640 1 -- 172.21.15.12:0/1668862358 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f64cc0a3d20 msgr2=0x7f64cc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.663+0000 7f64e8da4640 1 --2- 172.21.15.12:0/1668862358 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f64cc0a3d20 0x7f64cc0a61e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f64dc098da0 tx=0x7f64dc06e380 comp rx=0 tx=0).stop 2024-04-22T13:32:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.663+0000 7f64e8da4640 1 -- 172.21.15.12:0/1668862358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64e419cfd0 msgr2=0x7f64e419d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.663+0000 7f64e8da4640 1 --2- 172.21.15.12:0/1668862358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64e419cfd0 0x7f64e419d480 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00eaa0 tx=0x7f64dc00ef60 comp rx=0 tx=0).stop 2024-04-22T13:32:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.663+0000 7f64e8da4640 1 -- 172.21.15.12:0/1668862358 shutdown_connections 2024-04-22T13:32:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.663+0000 7f64e8da4640 1 --2- 172.21.15.12:0/1668862358 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f64cc0a3d20 0x7f64cc0a61e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.663+0000 7f64e8da4640 1 --2- 172.21.15.12:0/1668862358 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f64e419cfd0 0x7f64e419d480 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.663+0000 7f64e8da4640 1 --2- 172.21.15.12:0/1668862358 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64e4106170 0x7f64e4198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.663+0000 7f64e8da4640 1 --2- 172.21.15.12:0/1668862358 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64e41010a0 0x7f64e4198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.663+0000 7f64e8da4640 1 -- 172.21.15.12:0/1668862358 >> 172.21.15.12:0/1668862358 conn(0x7f64e40771d0 msgr2=0x7f64e4104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.663+0000 7f64e8da4640 1 -- 172.21.15.12:0/1668862358 shutdown_connections 2024-04-22T13:32:27.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.663+0000 7f64e8da4640 1 -- 172.21.15.12:0/1668862358 wait complete. 2024-04-22T13:32:27.669 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:27 smithi012 ceph-mon[28265]: pgmap v257: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-22T13:32:27.669 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:27 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3660938887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:32:27.670 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:32:27.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.778+0000 7f2e48206640 1 -- 172.21.15.12:0/3920191135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e40106160 msgr2=0x7f2e4010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.778+0000 7f2e48206640 1 --2- 172.21.15.12:0/3920191135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e40106160 0x7f2e4010dd30 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009ca0 tx=0x7f2e3802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.778+0000 7f2e48206640 1 -- 172.21.15.12:0/3920191135 shutdown_connections 2024-04-22T13:32:27.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.778+0000 7f2e48206640 1 --2- 172.21.15.12:0/3920191135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e40106160 0x7f2e4010dd30 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.778+0000 7f2e48206640 1 --2- 172.21.15.12:0/3920191135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e40101a40 0x7f2e40105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.778+0000 7f2e48206640 1 --2- 172.21.15.12:0/3920191135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e40101090 0x7f2e40101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.778+0000 7f2e48206640 1 -- 172.21.15.12:0/3920191135 >> 172.21.15.12:0/3920191135 conn(0x7f2e40077f50 msgr2=0x7f2e40078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:27.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.778+0000 7f2e48206640 1 -- 172.21.15.12:0/3920191135 shutdown_connections 2024-04-22T13:32:27.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.778+0000 7f2e48206640 1 -- 172.21.15.12:0/3920191135 wait complete. 2024-04-22T13:32:27.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.779+0000 7f2e48206640 1 Processor -- start 2024-04-22T13:32:27.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.779+0000 7f2e48206640 1 -- start start 2024-04-22T13:32:27.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.779+0000 7f2e48206640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e40101090 0x7f2e40198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.779+0000 7f2e48206640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e40106160 0x7f2e40198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.779+0000 7f2e48206640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e4019cfc0 0x7f2e4019d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.779+0000 7f2e48206640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401121e0 con 0x7f2e4019cfc0 2024-04-22T13:32:27.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.779+0000 7f2e48206640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2e40112060 con 0x7f2e40101090 2024-04-22T13:32:27.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.779+0000 7f2e48206640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2e40112360 con 0x7f2e40106160 2024-04-22T13:32:27.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.779+0000 7f2e467a3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e4019cfc0 0x7f2e4019d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.779+0000 7f2e467a3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e4019cfc0 0x7f2e4019d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34064/0 (socket says 172.21.15.12:34064) 2024-04-22T13:32:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.779+0000 7f2e467a3640 1 -- 172.21.15.12:0/3447498762 learned_addr learned my addr 172.21.15.12:0/3447498762 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.779+0000 7f2e457a1640 1 --2- 172.21.15.12:0/3447498762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e40106160 0x7f2e40198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.779+0000 7f2e45fa2640 1 --2- 172.21.15.12:0/3447498762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e40101090 0x7f2e40198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.779+0000 7f2e467a3640 1 -- 172.21.15.12:0/3447498762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e40106160 msgr2=0x7f2e40198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.779+0000 7f2e467a3640 1 --2- 172.21.15.12:0/3447498762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e40106160 0x7f2e40198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.779+0000 7f2e467a3640 1 -- 172.21.15.12:0/3447498762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e40101090 msgr2=0x7f2e40198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.779+0000 7f2e467a3640 1 --2- 172.21.15.12:0/3447498762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e40101090 0x7f2e40198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.779+0000 7f2e467a3640 1 -- 172.21.15.12:0/3447498762 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4019dab0 con 0x7f2e4019cfc0 2024-04-22T13:32:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.779+0000 7f2e457a1640 1 --2- 172.21.15.12:0/3447498762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e40106160 0x7f2e40198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.779+0000 7f2e467a3640 1 --2- 172.21.15.12:0/3447498762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e4019cfc0 0x7f2e4019d470 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f2e38005e30 tx=0x7f2e38032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.780+0000 7f2e36ffd640 1 -- 172.21.15.12:0/3447498762 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38041020 con 0x7f2e4019cfc0 2024-04-22T13:32:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.780+0000 7f2e48206640 1 -- 172.21.15.12:0/3447498762 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e4019dd40 con 0x7f2e4019cfc0 2024-04-22T13:32:27.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.780+0000 7f2e36ffd640 1 -- 172.21.15.12:0/3447498762 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e38032970 con 0x7f2e4019cfc0 2024-04-22T13:32:27.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.780+0000 7f2e36ffd640 1 -- 172.21.15.12:0/3447498762 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38039040 con 0x7f2e4019cfc0 2024-04-22T13:32:27.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.780+0000 7f2e48206640 1 -- 172.21.15.12:0/3447498762 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e401a5560 con 0x7f2e4019cfc0 2024-04-22T13:32:27.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.782+0000 7f2e36ffd640 1 -- 172.21.15.12:0/3447498762 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2e38044020 con 0x7f2e4019cfc0 2024-04-22T13:32:27.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.782+0000 7f2e4adf5740 1 Processor -- start 2024-04-22T13:32:27.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.783+0000 7f2e4adf5740 1 -- start start 2024-04-22T13:32:27.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.783+0000 7f2e4adf5740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b81113330 0x555b81113710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.783+0000 7f2e4adf5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b81113ce0 0x555b8111c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.783+0000 7f2e4adf5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b8111ce40 0x555b8111d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.783+0000 7f2e4adf5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555b810781f0 con 0x555b8111ce40 2024-04-22T13:32:27.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.783+0000 7f2e4adf5740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555b810154b0 con 0x555b81113330 2024-04-22T13:32:27.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.783+0000 7f2e4adf5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555b810741b0 con 0x555b81113ce0 2024-04-22T13:32:27.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.783+0000 7f2e36ffd640 1 --2- 172.21.15.12:0/3447498762 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2e2c0a3d20 0x7f2e2c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.783+0000 7f2e457a1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b8111ce40 0x555b8111d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.783+0000 7f2e467a3640 1 --2- 172.21.15.12:0/3447498762 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2e2c0a3d20 0x7f2e2c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.783+0000 7f2e457a1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b8111ce40 0x555b8111d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34068/0 (socket says 172.21.15.12:34068) 2024-04-22T13:32:27.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.783+0000 7f2e457a1640 1 -- 172.21.15.12:0/1503308325 learned_addr learned my addr 172.21.15.12:0/1503308325 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:27.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.783+0000 7f2e45fa2640 1 --2- 172.21.15.12:0/1503308325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b81113ce0 0x555b8111c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.783+0000 7f2e457a1640 1 --2- 172.21.15.12:0/1503308325 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b81113330 0x555b81113710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.783+0000 7f2e36ffd640 1 -- 172.21.15.12:0/3447498762 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f2e380f2570 con 0x7f2e4019cfc0 2024-04-22T13:32:27.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.783+0000 7f2e457a1640 1 -- 172.21.15.12:0/1503308325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b81113ce0 msgr2=0x555b8111c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.783+0000 7f2e457a1640 1 --2- 172.21.15.12:0/1503308325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b81113ce0 0x555b8111c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.783+0000 7f2e457a1640 1 -- 172.21.15.12:0/1503308325 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b81113330 msgr2=0x555b81113710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.783+0000 7f2e457a1640 1 --2- 172.21.15.12:0/1503308325 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b81113330 0x555b81113710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.783+0000 7f2e457a1640 1 -- 172.21.15.12:0/1503308325 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b810071b0 con 0x555b8111ce40 2024-04-22T13:32:27.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.783+0000 7f2e467a3640 1 --2- 172.21.15.12:0/3447498762 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2e2c0a3d20 0x7f2e2c0a61e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009dc0 tx=0x7f2e380b6040 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:27.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.784+0000 7f2e457a1640 1 --2- 172.21.15.12:0/1503308325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b8111ce40 0x555b8111d2c0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f2e30012770 tx=0x7f2e30009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57274449494cb9be server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:27.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.784+0000 7f2e2bfff640 1 -- 172.21.15.12:0/1503308325 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e30012cf0 con 0x555b8111ce40 2024-04-22T13:32:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.784+0000 7f2e2bfff640 1 -- 172.21.15.12:0/1503308325 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e30010c80 con 0x555b8111ce40 2024-04-22T13:32:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.784+0000 7f2e2bfff640 1 -- 172.21.15.12:0/1503308325 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3001eaf0 con 0x555b8111ce40 2024-04-22T13:32:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.784+0000 7f2e4adf5740 1 -- 172.21.15.12:0/1503308325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b8111ce40 msgr2=0x555b8111d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.784+0000 7f2e4adf5740 1 --2- 172.21.15.12:0/1503308325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b8111ce40 0x555b8111d2c0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f2e30012770 tx=0x7f2e30009680 comp rx=0 tx=0).stop 2024-04-22T13:32:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.784+0000 7f2e4adf5740 1 -- 172.21.15.12:0/1503308325 shutdown_connections 2024-04-22T13:32:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.784+0000 7f2e4adf5740 1 --2- 172.21.15.12:0/1503308325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b8111ce40 0x555b8111d2c0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.784+0000 7f2e4adf5740 1 --2- 172.21.15.12:0/1503308325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b81113ce0 0x555b8111c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.784+0000 7f2e4adf5740 1 --2- 172.21.15.12:0/1503308325 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b81113330 0x555b81113710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.784+0000 7f2e4adf5740 1 -- 172.21.15.12:0/1503308325 >> 172.21.15.12:0/1503308325 conn(0x555b8110dba0 msgr2=0x555b8110df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.785+0000 7f2e4adf5740 1 -- 172.21.15.12:0/1503308325 shutdown_connections 2024-04-22T13:32:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.785+0000 7f2e4adf5740 1 -- 172.21.15.12:0/1503308325 wait complete. 2024-04-22T13:32:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.785+0000 7f2e4adf5740 1 -- start start 2024-04-22T13:32:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.785+0000 7f2e4adf5740 1 Processor -- start 2024-04-22T13:32:27.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.785+0000 7f2e4adf5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b81113330 0x555b81123250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.785+0000 7f2e4adf5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b81113ce0 0x555b811205e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.785+0000 7f2e4adf5740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b8111ce40 0x555b81120b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.785+0000 7f2e4adf5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555b81123dc0 con 0x555b81113330 2024-04-22T13:32:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.785+0000 7f2e4adf5740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555b81031500 con 0x555b8111ce40 2024-04-22T13:32:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.785+0000 7f2e4adf5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555b81123f40 con 0x555b81113ce0 2024-04-22T13:32:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.785+0000 7f2e4adf5740 10 client.0 _ll_register_callbacks cb 0x555b80fffb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.785+0000 7f2e457a1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b81113330 0x555b81123250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.785+0000 7f2e457a1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b81113330 0x555b81123250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34070/0 (socket says 172.21.15.12:34070) 2024-04-22T13:32:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.785+0000 7f2e457a1640 1 -- 172.21.15.12:0/3630672731 learned_addr learned my addr 172.21.15.12:0/3630672731 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.786+0000 7f2e45fa2640 1 --2- 172.21.15.12:0/3630672731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b81113ce0 0x555b811205e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.786+0000 7f2e457a1640 1 -- 172.21.15.12:0/3630672731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b81113ce0 msgr2=0x555b811205e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.786+0000 7f2e457a1640 1 --2- 172.21.15.12:0/3630672731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b81113ce0 0x555b811205e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.786+0000 7f2e457a1640 1 -- 172.21.15.12:0/3630672731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b8111ce40 msgr2=0x555b81120b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.786+0000 7f2e457a1640 1 --2- 172.21.15.12:0/3630672731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b8111ce40 0x555b81120b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.786+0000 7f2e457a1640 1 -- 172.21.15.12:0/3630672731 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b80d8a3f0 con 0x555b81113330 2024-04-22T13:32:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.786+0000 7f2e457a1640 1 --2- 172.21.15.12:0/3630672731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b81113330 0x555b81123250 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f2e30009860 tx=0x7f2e30009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5afe7b5886da285 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.786+0000 7f2e297fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:32:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.786+0000 7f2e297fa640 1 -- 172.21.15.12:0/3630672731 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e300109e0 con 0x555b81113330 2024-04-22T13:32:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.786+0000 7f2e297fa640 1 -- 172.21.15.12:0/3630672731 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f2e3001e8d0 con 0x555b81113330 2024-04-22T13:32:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.786+0000 7f2e4adf5740 1 -- 172.21.15.12:0/3630672731 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e30010c80 con 0x555b81113330 2024-04-22T13:32:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.786+0000 7f2e297fa640 1 -- 172.21.15.12:0/3630672731 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2e3001ead0 con 0x555b81113330 2024-04-22T13:32:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.786+0000 7f2e4adf5740 10 client.18597 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.786+0000 7f2e4adf5740 1 -- 172.21.15.12:0/3630672731 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555b80e547d0 con 0x555b81113330 2024-04-22T13:32:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.786+0000 7f2e297fa640 1 -- 172.21.15.12:0/3630672731 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e30010c80 con 0x555b81113330 2024-04-22T13:32:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.787+0000 7f2e297fa640 1 -- 172.21.15.12:0/3630672731 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2e30004750 con 0x555b81113330 2024-04-22T13:32:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.787+0000 7f2e4adf5740 10 client.18597 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.787+0000 7f2e4adf5740 10 client.18597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.787+0000 7f2e4adf5740 20 client.18597 populate_metadata read hostname 'smithi012' 2024-04-22T13:32:27.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.787+0000 7f2e4adf5740 1 --2- 172.21.15.12:0/3630672731 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555b81130fa0 0x555b81133390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.787+0000 7f2e4adf5740 4 client.18597 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.787+0000 7f2e4adf5740 1 -- 172.21.15.12:0/3630672731 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b81017930 con 0x555b81130fa0 2024-04-22T13:32:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.787+0000 7f2e45fa2640 1 --2- 172.21.15.12:0/3630672731 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555b81130fa0 0x555b81133390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.787+0000 7f2e45fa2640 1 --2- 172.21.15.12:0/3630672731 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555b81130fa0 0x555b81133390 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f7afebb09e7f6a5 server_cookie=e833c81a9e61277a in_seq=0 out_seq=0 2024-04-22T13:32:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.787+0000 7f2e297fa640 10 client.18597 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:32:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.788+0000 7f2e297fa640 1 -- 172.21.15.12:0/3630672731 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555b81017930 con 0x555b81130fa0 2024-04-22T13:32:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.788+0000 7f2e297fa640 10 client.18597 handle_command_reply: tid=0 2024-04-22T13:32:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.788+0000 7f2e4adf5740 1 client.18597 shutdown 2024-04-22T13:32:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.788+0000 7f2e4adf5740 1 -- 172.21.15.12:0/3630672731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b81113330 msgr2=0x555b81123250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.788+0000 7f2e4adf5740 1 --2- 172.21.15.12:0/3630672731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b81113330 0x555b81123250 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f2e30009860 tx=0x7f2e30009280 comp rx=0 tx=0).stop 2024-04-22T13:32:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.788+0000 7f2e4adf5740 1 -- 172.21.15.12:0/3630672731 shutdown_connections 2024-04-22T13:32:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.788+0000 7f2e4adf5740 1 --2- 172.21.15.12:0/3630672731 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555b81130fa0 0x555b81133390 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.789+0000 7f2e4adf5740 1 --2- 172.21.15.12:0/3630672731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b8111ce40 0x555b81120b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.789+0000 7f2e4adf5740 1 --2- 172.21.15.12:0/3630672731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b81113ce0 0x555b811205e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.789+0000 7f2e4adf5740 1 --2- 172.21.15.12:0/3630672731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b81113330 0x555b81123250 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.789+0000 7f2e297fa640 0 client.18597 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:32:27.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.789+0000 7f2e4adf5740 1 -- 172.21.15.12:0/3630672731 >> 172.21.15.12:0/3630672731 conn(0x555b8110dba0 msgr2=0x555b8110f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:27.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.789+0000 7f2e4adf5740 1 -- 172.21.15.12:0/3630672731 shutdown_connections 2024-04-22T13:32:27.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.789+0000 7f2e4adf5740 1 -- 172.21.15.12:0/3630672731 wait complete. 2024-04-22T13:32:27.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.789+0000 7f2e4adf5740 20 client.18597 trim_cache size 0 max 16384 2024-04-22T13:32:27.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.802+0000 7f2e4adf5740 1 Processor -- start 2024-04-22T13:32:27.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.802+0000 7f2e4adf5740 1 -- start start 2024-04-22T13:32:27.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.802+0000 7f2e4adf5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b81113330 0x555b8111e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.802+0000 7f2e4adf5740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b81113ce0 0x555b8112ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.802+0000 7f2e4adf5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b8111ce40 0x555b81122d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.802+0000 7f2e4adf5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401ab8e0 con 0x555b81113330 2024-04-22T13:32:27.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.802+0000 7f2e4adf5740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401ad420 con 0x555b81113ce0 2024-04-22T13:32:27.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.802+0000 7f2e4adf5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401abfb0 con 0x555b8111ce40 2024-04-22T13:32:27.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.802+0000 7f2e457a1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b81113330 0x555b8111e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.802+0000 7f2e457a1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b81113330 0x555b8111e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34072/0 (socket says 172.21.15.12:34072) 2024-04-22T13:32:27.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.802+0000 7f2e457a1640 1 -- 172.21.15.12:0/2109099924 learned_addr learned my addr 172.21.15.12:0/2109099924 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:27.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.802+0000 7f2e457a1640 1 -- 172.21.15.12:0/2109099924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b8111ce40 msgr2=0x555b81122d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.802+0000 7f2e457a1640 1 --2- 172.21.15.12:0/2109099924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b8111ce40 0x555b81122d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.802+0000 7f2e457a1640 1 -- 172.21.15.12:0/2109099924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b81113ce0 msgr2=0x555b8112ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.802+0000 7f2e45fa2640 1 --2- 172.21.15.12:0/2109099924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b81113ce0 0x555b8112ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.802+0000 7f2e457a1640 1 --2- 172.21.15.12:0/2109099924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b81113ce0 0x555b8112ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.802+0000 7f2e457a1640 1 -- 172.21.15.12:0/2109099924 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b8111eb30 con 0x555b81113330 2024-04-22T13:32:27.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.803+0000 7f2e457a1640 1 --2- 172.21.15.12:0/2109099924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b81113330 0x555b8111e290 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f2e30004b20 tx=0x7f2e30009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce235661ab74269 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:27.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.803+0000 7f2e2b7fe640 1 -- 172.21.15.12:0/2109099924 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e30035070 con 0x555b81113330 2024-04-22T13:32:27.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.803+0000 7f2e2b7fe640 1 -- 172.21.15.12:0/2109099924 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e30009ab0 con 0x555b81113330 2024-04-22T13:32:27.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.803+0000 7f2e2b7fe640 1 -- 172.21.15.12:0/2109099924 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e30010c40 con 0x555b81113330 2024-04-22T13:32:27.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.803+0000 7f2e4adf5740 1 -- 172.21.15.12:0/2109099924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b81113330 msgr2=0x555b8111e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.803+0000 7f2e4adf5740 1 --2- 172.21.15.12:0/2109099924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b81113330 0x555b8111e290 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f2e30004b20 tx=0x7f2e30009cb0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.803+0000 7f2e4adf5740 1 -- 172.21.15.12:0/2109099924 shutdown_connections 2024-04-22T13:32:27.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.803+0000 7f2e4adf5740 1 --2- 172.21.15.12:0/2109099924 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b8111ce40 0x555b81122d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.803+0000 7f2e4adf5740 1 --2- 172.21.15.12:0/2109099924 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b81113ce0 0x555b8112ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.803+0000 7f2e4adf5740 1 --2- 172.21.15.12:0/2109099924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b81113330 0x555b8111e290 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.803+0000 7f2e4adf5740 1 -- 172.21.15.12:0/2109099924 >> 172.21.15.12:0/2109099924 conn(0x555b8110dba0 msgr2=0x555b8110f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:27.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.803+0000 7f2e4adf5740 1 -- 172.21.15.12:0/2109099924 shutdown_connections 2024-04-22T13:32:27.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.803+0000 7f2e4adf5740 1 -- 172.21.15.12:0/2109099924 wait complete. 2024-04-22T13:32:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.804+0000 7f2e4adf5740 1 -- start start 2024-04-22T13:32:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.804+0000 7f2e4adf5740 1 Processor -- start 2024-04-22T13:32:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.804+0000 7f2e4adf5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b81113330 0x555b811295e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.804+0000 7f2e4adf5740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b81113ce0 0x555b81122030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.804+0000 7f2e4adf5740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b8111ce40 0x555b81122570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.804+0000 7f2e4adf5740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401acd50 con 0x555b81113ce0 2024-04-22T13:32:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.804+0000 7f2e4adf5740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401ac680 con 0x555b8111ce40 2024-04-22T13:32:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.804+0000 7f2e4adf5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2e401a9900 con 0x555b81113330 2024-04-22T13:32:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.804+0000 7f2e4adf5740 10 client.0 _ll_register_callbacks cb 0x7f2e30012670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.804+0000 7f2e45fa2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b81113ce0 0x555b81122030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.804+0000 7f2e45fa2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b81113ce0 0x555b81122030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34086/0 (socket says 172.21.15.12:34086) 2024-04-22T13:32:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.804+0000 7f2e45fa2640 1 -- 172.21.15.12:0/224045786 learned_addr learned my addr 172.21.15.12:0/224045786 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.805+0000 7f2e45fa2640 1 -- 172.21.15.12:0/224045786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b81113330 msgr2=0x555b811295e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.805+0000 7f2e457a1640 1 --2- 172.21.15.12:0/224045786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b81113330 0x555b811295e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.805+0000 7f2e457a1640 1 --2- 172.21.15.12:0/224045786 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b8111ce40 0x555b81122570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.805+0000 7f2e45fa2640 1 --2- 172.21.15.12:0/224045786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b81113330 0x555b811295e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.805+0000 7f2e45fa2640 1 -- 172.21.15.12:0/224045786 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b8111ce40 msgr2=0x555b81122570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.805+0000 7f2e45fa2640 1 --2- 172.21.15.12:0/224045786 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b8111ce40 0x555b81122570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.805+0000 7f2e45fa2640 1 -- 172.21.15.12:0/224045786 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b81122da0 con 0x555b81113ce0 2024-04-22T13:32:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.805+0000 7f2e45fa2640 1 --2- 172.21.15.12:0/224045786 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b81113ce0 0x555b81122030 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c005d00 tx=0x7f2e3c01bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea8852280d75fa45 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.805+0000 7f2e297fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:32:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.805+0000 7f2e297fa640 1 -- 172.21.15.12:0/224045786 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c004440 con 0x555b81113ce0 2024-04-22T13:32:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.805+0000 7f2e297fa640 1 -- 172.21.15.12:0/224045786 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f2e3c012920 con 0x555b81113ce0 2024-04-22T13:32:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.805+0000 7f2e4adf5740 1 -- 172.21.15.12:0/224045786 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e30009ab0 con 0x555b81113ce0 2024-04-22T13:32:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.805+0000 7f2e297fa640 1 -- 172.21.15.12:0/224045786 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2e3c02a050 con 0x555b81113ce0 2024-04-22T13:32:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.805+0000 7f2e4adf5740 10 client.18603 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.805+0000 7f2e4adf5740 1 -- 172.21.15.12:0/224045786 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555b811973a0 con 0x555b81113ce0 2024-04-22T13:32:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.805+0000 7f2e297fa640 1 -- 172.21.15.12:0/224045786 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e30009ab0 con 0x555b81113ce0 2024-04-22T13:32:27.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.805+0000 7f2e297fa640 1 -- 172.21.15.12:0/224045786 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2e3c003710 con 0x555b81113ce0 2024-04-22T13:32:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.806+0000 7f2e4adf5740 10 client.18603 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.806+0000 7f2e4adf5740 10 client.18603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.806+0000 7f2e4adf5740 20 client.18603 populate_metadata read hostname 'smithi012' 2024-04-22T13:32:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.806+0000 7f2e4adf5740 1 --2- 172.21.15.12:0/224045786 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555b81197af0 0x555b81199f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.806+0000 7f2e4adf5740 4 client.18603 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:32:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.806+0000 7f2e4adf5740 1 -- 172.21.15.12:0/224045786 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555b81085c10 con 0x555b81197af0 2024-04-22T13:32:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.806+0000 7f2e457a1640 1 --2- 172.21.15.12:0/224045786 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555b81197af0 0x555b81199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.806+0000 7f2e457a1640 1 --2- 172.21.15.12:0/224045786 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555b81197af0 0x555b81199f10 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a1492c5110d9859 server_cookie=f8303342e82deb15 in_seq=0 out_seq=0 2024-04-22T13:32:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.806+0000 7f2e297fa640 10 client.18603 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:32:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.806+0000 7f2e297fa640 1 -- 172.21.15.12:0/224045786 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555b81085c10 con 0x555b81197af0 2024-04-22T13:32:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.806+0000 7f2e297fa640 10 client.18603 handle_command_reply: tid=0 2024-04-22T13:32:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.806+0000 7f2e4adf5740 1 client.18603 shutdown 2024-04-22T13:32:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.806+0000 7f2e4adf5740 1 -- 172.21.15.12:0/224045786 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b81113ce0 msgr2=0x555b81122030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.806+0000 7f2e4adf5740 1 --2- 172.21.15.12:0/224045786 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b81113ce0 0x555b81122030 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c005d00 tx=0x7f2e3c01bf90 comp rx=0 tx=0).stop 2024-04-22T13:32:27.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.807+0000 7f2e4adf5740 1 -- 172.21.15.12:0/224045786 shutdown_connections 2024-04-22T13:32:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.807+0000 7f2e4adf5740 1 --2- 172.21.15.12:0/224045786 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555b81197af0 0x555b81199f10 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.807+0000 7f2e4adf5740 1 --2- 172.21.15.12:0/224045786 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b8111ce40 0x555b81122570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.807+0000 7f2e4adf5740 1 --2- 172.21.15.12:0/224045786 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555b81113ce0 0x555b81122030 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.807+0000 7f2e297fa640 0 client.18603 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:32:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.807+0000 7f2e4adf5740 1 --2- 172.21.15.12:0/224045786 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555b81113330 0x555b811295e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.807+0000 7f2e4adf5740 1 -- 172.21.15.12:0/224045786 >> 172.21.15.12:0/224045786 conn(0x555b8110dba0 msgr2=0x555b8110c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.807+0000 7f2e4adf5740 1 -- 172.21.15.12:0/224045786 shutdown_connections 2024-04-22T13:32:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.807+0000 7f2e4adf5740 1 -- 172.21.15.12:0/224045786 wait complete. 2024-04-22T13:32:27.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.807+0000 7f2e4adf5740 20 client.18603 trim_cache size 0 max 16384 2024-04-22T13:32:27.809 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:32:27.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.807+0000 7f2e48206640 1 -- 172.21.15.12:0/3447498762 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2e2c0a3d20 msgr2=0x7f2e2c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.807+0000 7f2e48206640 1 --2- 172.21.15.12:0/3447498762 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2e2c0a3d20 0x7f2e2c0a61e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009dc0 tx=0x7f2e380b6040 comp rx=0 tx=0).stop 2024-04-22T13:32:27.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.807+0000 7f2e48206640 1 -- 172.21.15.12:0/3447498762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e4019cfc0 msgr2=0x7f2e4019d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.807+0000 7f2e48206640 1 --2- 172.21.15.12:0/3447498762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e4019cfc0 0x7f2e4019d470 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f2e38005e30 tx=0x7f2e38032420 comp rx=0 tx=0).stop 2024-04-22T13:32:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.807+0000 7f2e48206640 1 -- 172.21.15.12:0/3447498762 shutdown_connections 2024-04-22T13:32:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.807+0000 7f2e48206640 1 --2- 172.21.15.12:0/3447498762 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2e2c0a3d20 0x7f2e2c0a61e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.807+0000 7f2e48206640 1 --2- 172.21.15.12:0/3447498762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2e4019cfc0 0x7f2e4019d470 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.807+0000 7f2e48206640 1 --2- 172.21.15.12:0/3447498762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e40106160 0x7f2e40198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.807+0000 7f2e48206640 1 --2- 172.21.15.12:0/3447498762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e40101090 0x7f2e40198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.807+0000 7f2e48206640 1 -- 172.21.15.12:0/3447498762 >> 172.21.15.12:0/3447498762 conn(0x7f2e40077f50 msgr2=0x7f2e400ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.807+0000 7f2e48206640 1 -- 172.21.15.12:0/3447498762 shutdown_connections 2024-04-22T13:32:27.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:27.807+0000 7f2e48206640 1 -- 172.21.15.12:0/3447498762 wait complete. 2024-04-22T13:32:27.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:27 smithi134 ceph-mon[31338]: pgmap v257: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-22T13:32:27.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:27 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/3660938887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:32:27.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:27 smithi111 ceph-mon[31361]: pgmap v257: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-22T13:32:27.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:27 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/3660938887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:32:28.815 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:32:28.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:32:28.815 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:32:28.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.940+0000 7fd74de9a640 1 -- 172.21.15.12:0/915190032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd748107270 msgr2=0x7fd74810ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:28.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.940+0000 7fd74de9a640 1 --2- 172.21.15.12:0/915190032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd748107270 0x7fd74810ee40 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fd740009ca0 tx=0x7fd74002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.940+0000 7fd74de9a640 1 -- 172.21.15.12:0/915190032 shutdown_connections 2024-04-22T13:32:28.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.940+0000 7fd74de9a640 1 --2- 172.21.15.12:0/915190032 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd748107270 0x7fd74810ee40 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.940+0000 7fd74de9a640 1 --2- 172.21.15.12:0/915190032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd74806b1f0 0x7fd748106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.940+0000 7fd74de9a640 1 --2- 172.21.15.12:0/915190032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd74806a840 0x7fd74806ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.940+0000 7fd74de9a640 1 -- 172.21.15.12:0/915190032 >> 172.21.15.12:0/915190032 conn(0x7fd748077eb0 msgr2=0x7fd7480782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:28.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.940+0000 7fd74de9a640 1 -- 172.21.15.12:0/915190032 shutdown_connections 2024-04-22T13:32:28.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.940+0000 7fd74de9a640 1 -- 172.21.15.12:0/915190032 wait complete. 2024-04-22T13:32:28.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.940+0000 7fd74de9a640 1 Processor -- start 2024-04-22T13:32:28.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.941+0000 7fd74de9a640 1 -- start start 2024-04-22T13:32:28.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.941+0000 7fd74de9a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd74806a840 0x7fd748100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:28.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.941+0000 7fd74de9a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd748107270 0x7fd7481013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:28.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.941+0000 7fd74de9a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd748101900 0x7fd7481a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:28.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.941+0000 7fd74de9a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd7481121d0 con 0x7fd748101900 2024-04-22T13:32:28.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.941+0000 7fd74de9a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd748112050 con 0x7fd748107270 2024-04-22T13:32:28.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.941+0000 7fd74de9a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd748112350 con 0x7fd74806a840 2024-04-22T13:32:28.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.941+0000 7fd747fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd748101900 0x7fd7481a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:28.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.941+0000 7fd747fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd748101900 0x7fd7481a68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34112/0 (socket says 172.21.15.12:34112) 2024-04-22T13:32:28.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.941+0000 7fd747fff640 1 -- 172.21.15.12:0/419521963 learned_addr learned my addr 172.21.15.12:0/419521963 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:28.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.941+0000 7fd7477fe640 1 --2- 172.21.15.12:0/419521963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd74806a840 0x7fd748100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:28.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.941+0000 7fd746ffd640 1 --2- 172.21.15.12:0/419521963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd748107270 0x7fd7481013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:28.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.941+0000 7fd747fff640 1 -- 172.21.15.12:0/419521963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd74806a840 msgr2=0x7fd748100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:28.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.941+0000 7fd747fff640 1 --2- 172.21.15.12:0/419521963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd74806a840 0x7fd748100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.941+0000 7fd747fff640 1 -- 172.21.15.12:0/419521963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd748107270 msgr2=0x7fd7481013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:28.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.941+0000 7fd747fff640 1 --2- 172.21.15.12:0/419521963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd748107270 0x7fd7481013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.941+0000 7fd747fff640 1 -- 172.21.15.12:0/419521963 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7481a6f60 con 0x7fd748101900 2024-04-22T13:32:28.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.941+0000 7fd747fff640 1 --2- 172.21.15.12:0/419521963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd748101900 0x7fd7481a68c0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fd7400098d0 tx=0x7fd740035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:28.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.942+0000 7fd744ff9640 1 -- 172.21.15.12:0/419521963 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd740039040 con 0x7fd748101900 2024-04-22T13:32:28.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.942+0000 7fd744ff9640 1 -- 172.21.15.12:0/419521963 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd740044070 con 0x7fd748101900 2024-04-22T13:32:28.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.942+0000 7fd744ff9640 1 -- 172.21.15.12:0/419521963 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7400042a0 con 0x7fd748101900 2024-04-22T13:32:28.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.942+0000 7fd74de9a640 1 -- 172.21.15.12:0/419521963 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7481a71f0 con 0x7fd748101900 2024-04-22T13:32:28.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.942+0000 7fd74de9a640 1 -- 172.21.15.12:0/419521963 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7481a7690 con 0x7fd748101900 2024-04-22T13:32:28.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.944+0000 7fd744ff9640 1 -- 172.21.15.12:0/419521963 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd740004440 con 0x7fd748101900 2024-04-22T13:32:28.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.944+0000 7fd750b6a740 1 Processor -- start 2024-04-22T13:32:28.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.944+0000 7fd750b6a740 1 -- start start 2024-04-22T13:32:28.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.945+0000 7fd750b6a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f6c6e68e0 0x561f6c6e6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:28.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.945+0000 7fd750b6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6c6e7290 0x561f6c6ef840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:28.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.945+0000 7fd744ff9640 1 --2- 172.21.15.12:0/419521963 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd7300a3d20 0x7fd7300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:28.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.945+0000 7fd750b6a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f6c6efe10 0x561f6c6f2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:28.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.945+0000 7fd750b6a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561f6c64af00 con 0x561f6c6e68e0 2024-04-22T13:32:28.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.945+0000 7fd750b6a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561f6c5e8590 con 0x561f6c6efe10 2024-04-22T13:32:28.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.945+0000 7fd750b6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561f6c646ec0 con 0x561f6c6e7290 2024-04-22T13:32:28.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.945+0000 7fd746ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f6c6e68e0 0x561f6c6e6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:28.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.945+0000 7fd746ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f6c6e68e0 0x561f6c6e6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34116/0 (socket says 172.21.15.12:34116) 2024-04-22T13:32:28.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.945+0000 7fd746ffd640 1 -- 172.21.15.12:0/1093027744 learned_addr learned my addr 172.21.15.12:0/1093027744 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:28.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.945+0000 7fd746ffd640 1 --2- 172.21.15.12:0/419521963 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd7300a3d20 0x7fd7300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:28.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.945+0000 7fd744ff9640 1 -- 172.21.15.12:0/419521963 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7fd7400f2c80 con 0x7fd748101900 2024-04-22T13:32:28.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.945+0000 7fd747fff640 1 --2- 172.21.15.12:0/1093027744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f6c6efe10 0x561f6c6f2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:28.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.945+0000 7fd7477fe640 1 --2- 172.21.15.12:0/1093027744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6c6e7290 0x561f6c6ef840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:28.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.945+0000 7fd746ffd640 1 -- 172.21.15.12:0/1093027744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6c6e7290 msgr2=0x561f6c6ef840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:28.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.945+0000 7fd746ffd640 1 --2- 172.21.15.12:0/1093027744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6c6e7290 0x561f6c6ef840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.945+0000 7fd746ffd640 1 -- 172.21.15.12:0/1093027744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f6c6efe10 msgr2=0x561f6c6f2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:28.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.945+0000 7fd746ffd640 1 --2- 172.21.15.12:0/1093027744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f6c6efe10 0x561f6c6f2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.945+0000 7fd746ffd640 1 -- 172.21.15.12:0/1093027744 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f6c4257e0 con 0x561f6c6e68e0 2024-04-22T13:32:28.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.945+0000 7fd746ffd640 1 --2- 172.21.15.12:0/1093027744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f6c6e68e0 0x561f6c6e6cc0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fd73c013ba0 tx=0x7fd73c0103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e790a4f0f43ac054 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:28.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.946+0000 7fd746ffd640 1 --2- 172.21.15.12:0/419521963 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd7300a3d20 0x7fd7300a61e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd73c005940 tx=0x7fd73c015000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:28.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.946+0000 7fd72dffb640 1 -- 172.21.15.12:0/1093027744 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c023070 con 0x561f6c6e68e0 2024-04-22T13:32:28.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.946+0000 7fd72dffb640 1 -- 172.21.15.12:0/1093027744 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd73c00e940 con 0x561f6c6e68e0 2024-04-22T13:32:28.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.946+0000 7fd72dffb640 1 -- 172.21.15.12:0/1093027744 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c01fc00 con 0x561f6c6e68e0 2024-04-22T13:32:28.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.946+0000 7fd750b6a740 1 -- 172.21.15.12:0/1093027744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f6c6e68e0 msgr2=0x561f6c6e6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:28.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.946+0000 7fd750b6a740 1 --2- 172.21.15.12:0/1093027744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f6c6e68e0 0x561f6c6e6cc0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fd73c013ba0 tx=0x7fd73c0103d0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.946+0000 7fd750b6a740 1 -- 172.21.15.12:0/1093027744 shutdown_connections 2024-04-22T13:32:28.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.946+0000 7fd750b6a740 1 --2- 172.21.15.12:0/1093027744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f6c6efe10 0x561f6c6f2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.946+0000 7fd750b6a740 1 --2- 172.21.15.12:0/1093027744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6c6e7290 0x561f6c6ef840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.946+0000 7fd750b6a740 1 --2- 172.21.15.12:0/1093027744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f6c6e68e0 0x561f6c6e6cc0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.946+0000 7fd750b6a740 1 -- 172.21.15.12:0/1093027744 >> 172.21.15.12:0/1093027744 conn(0x561f6c6e0fb0 msgr2=0x561f6c6e1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:28.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.946+0000 7fd750b6a740 1 -- 172.21.15.12:0/1093027744 shutdown_connections 2024-04-22T13:32:28.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.946+0000 7fd750b6a740 1 -- 172.21.15.12:0/1093027744 wait complete. 2024-04-22T13:32:28.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.947+0000 7fd750b6a740 1 -- start start 2024-04-22T13:32:28.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.947+0000 7fd750b6a740 1 Processor -- start 2024-04-22T13:32:28.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.947+0000 7fd750b6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6c6e68e0 0x561f6c6ecec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:28.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.947+0000 7fd750b6a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f6c6e7290 0x561f6c6ed400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:28.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.947+0000 7fd750b6a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f6c6efe10 0x561f6c6ed940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:28.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.947+0000 7fd750b6a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561f6c6f2d20 con 0x561f6c6e7290 2024-04-22T13:32:28.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.947+0000 7fd750b6a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561f6c604580 con 0x561f6c6efe10 2024-04-22T13:32:28.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.947+0000 7fd750b6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561f6c6f2ea0 con 0x561f6c6e68e0 2024-04-22T13:32:28.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.947+0000 7fd750b6a740 10 client.0 _ll_register_callbacks cb 0x561f6c60a650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:28.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.947+0000 7fd746ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f6c6e7290 0x561f6c6ed400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:28.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.948+0000 7fd746ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f6c6e7290 0x561f6c6ed400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34128/0 (socket says 172.21.15.12:34128) 2024-04-22T13:32:28.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.948+0000 7fd746ffd640 1 -- 172.21.15.12:0/251635856 learned_addr learned my addr 172.21.15.12:0/251635856 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:28.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.948+0000 7fd7477fe640 1 --2- 172.21.15.12:0/251635856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6c6e68e0 0x561f6c6ecec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:28.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.948+0000 7fd747fff640 1 --2- 172.21.15.12:0/251635856 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f6c6efe10 0x561f6c6ed940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:28.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.948+0000 7fd746ffd640 1 -- 172.21.15.12:0/251635856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6c6e68e0 msgr2=0x561f6c6ecec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:28.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.948+0000 7fd746ffd640 1 --2- 172.21.15.12:0/251635856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6c6e68e0 0x561f6c6ecec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.948+0000 7fd746ffd640 1 -- 172.21.15.12:0/251635856 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f6c6efe10 msgr2=0x561f6c6ed940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:28.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.948+0000 7fd746ffd640 1 --2- 172.21.15.12:0/251635856 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f6c6efe10 0x561f6c6ed940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.948+0000 7fd746ffd640 1 -- 172.21.15.12:0/251635856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f6c485f70 con 0x561f6c6e7290 2024-04-22T13:32:28.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.948+0000 7fd747fff640 1 --2- 172.21.15.12:0/251635856 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f6c6efe10 0x561f6c6ed940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:28.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.948+0000 7fd746ffd640 1 --2- 172.21.15.12:0/251635856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f6c6e7290 0x561f6c6ed400 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fd73c01f860 tx=0x7fd73c00ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf5d43d60ab309c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:28.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.948+0000 7fd726ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:32:28.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.948+0000 7fd726ffd640 1 -- 172.21.15.12:0/251635856 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c023040 con 0x561f6c6e7290 2024-04-22T13:32:28.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.948+0000 7fd726ffd640 1 -- 172.21.15.12:0/251635856 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7fd73c01f9a0 con 0x561f6c6e7290 2024-04-22T13:32:28.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.948+0000 7fd750b6a740 1 -- 172.21.15.12:0/251635856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd73c00e940 con 0x561f6c6e7290 2024-04-22T13:32:28.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.948+0000 7fd726ffd640 1 -- 172.21.15.12:0/251635856 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd73c01fba0 con 0x561f6c6e7290 2024-04-22T13:32:28.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.948+0000 7fd750b6a740 10 client.18615 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:28.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.948+0000 7fd750b6a740 1 -- 172.21.15.12:0/251635856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561f6c31d170 con 0x561f6c6e7290 2024-04-22T13:32:28.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.949+0000 7fd726ffd640 1 -- 172.21.15.12:0/251635856 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd73c00e940 con 0x561f6c6e7290 2024-04-22T13:32:28.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.949+0000 7fd726ffd640 1 -- 172.21.15.12:0/251635856 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd73c038070 con 0x561f6c6e7290 2024-04-22T13:32:28.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.949+0000 7fd750b6a740 10 client.18615 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:28.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.949+0000 7fd750b6a740 10 client.18615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:28.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.949+0000 7fd750b6a740 20 client.18615 populate_metadata read hostname 'smithi012' 2024-04-22T13:32:28.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.949+0000 7fd750b6a740 1 --2- 172.21.15.12:0/251635856 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x561f6c7003c0 0x561f6c7027b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:28.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.949+0000 7fd750b6a740 4 client.18615 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:28.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.949+0000 7fd750b6a740 1 -- 172.21.15.12:0/251635856 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f6c658920 con 0x561f6c7003c0 2024-04-22T13:32:28.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.949+0000 7fd7477fe640 1 --2- 172.21.15.12:0/251635856 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x561f6c7003c0 0x561f6c7027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:28.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.950+0000 7fd726ffd640 10 client.18615 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:32:28.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.950+0000 7fd7477fe640 1 --2- 172.21.15.12:0/251635856 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x561f6c7003c0 0x561f6c7027b0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a033cabfcfbb4b5f server_cookie=57abf8f8581e81dc in_seq=0 out_seq=0 2024-04-22T13:32:28.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.950+0000 7fd726ffd640 1 -- 172.21.15.12:0/251635856 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561f6c658920 con 0x561f6c7003c0 2024-04-22T13:32:28.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.950+0000 7fd726ffd640 10 client.18615 handle_command_reply: tid=0 2024-04-22T13:32:28.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.950+0000 7fd750b6a740 1 client.18615 shutdown 2024-04-22T13:32:28.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.950+0000 7fd750b6a740 1 -- 172.21.15.12:0/251635856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f6c6e7290 msgr2=0x561f6c6ed400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:28.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.950+0000 7fd750b6a740 1 --2- 172.21.15.12:0/251635856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f6c6e7290 0x561f6c6ed400 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fd73c01f860 tx=0x7fd73c00ac50 comp rx=0 tx=0).stop 2024-04-22T13:32:28.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.951+0000 7fd750b6a740 1 -- 172.21.15.12:0/251635856 shutdown_connections 2024-04-22T13:32:28.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.951+0000 7fd750b6a740 1 --2- 172.21.15.12:0/251635856 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x561f6c7003c0 0x561f6c7027b0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.951+0000 7fd750b6a740 1 --2- 172.21.15.12:0/251635856 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f6c6efe10 0x561f6c6ed940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.951+0000 7fd750b6a740 1 --2- 172.21.15.12:0/251635856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f6c6e7290 0x561f6c6ed400 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.951+0000 7fd750b6a740 1 --2- 172.21.15.12:0/251635856 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6c6e68e0 0x561f6c6ecec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.951+0000 7fd726ffd640 0 client.18615 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:32:28.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.951+0000 7fd750b6a740 1 -- 172.21.15.12:0/251635856 >> 172.21.15.12:0/251635856 conn(0x561f6c6e0fb0 msgr2=0x561f6c6e2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:28.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.951+0000 7fd750b6a740 1 -- 172.21.15.12:0/251635856 shutdown_connections 2024-04-22T13:32:28.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.951+0000 7fd750b6a740 1 -- 172.21.15.12:0/251635856 wait complete. 2024-04-22T13:32:28.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.951+0000 7fd750b6a740 20 client.18615 trim_cache size 0 max 16384 2024-04-22T13:32:28.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.968+0000 7fd750b6a740 1 Processor -- start 2024-04-22T13:32:28.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.968+0000 7fd750b6a740 1 -- start start 2024-04-22T13:32:28.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.969+0000 7fd750b6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6c6e68e0 0x561f6c6ef5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:28.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.969+0000 7fd750b6a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f6c6e7290 0x561f6c6fbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:28.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.969+0000 7fd750b6a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f6c6efe10 0x561f6c6dfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:28.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.969+0000 7fd750b6a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd7481afc30 con 0x561f6c6e7290 2024-04-22T13:32:28.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.969+0000 7fd750b6a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd7481b1770 con 0x561f6c6efe10 2024-04-22T13:32:28.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.969+0000 7fd750b6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd7481b0300 con 0x561f6c6e68e0 2024-04-22T13:32:28.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.969+0000 7fd746ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f6c6e7290 0x561f6c6fbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:28.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.969+0000 7fd746ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f6c6e7290 0x561f6c6fbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34144/0 (socket says 172.21.15.12:34144) 2024-04-22T13:32:28.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.969+0000 7fd746ffd640 1 -- 172.21.15.12:0/581273096 learned_addr learned my addr 172.21.15.12:0/581273096 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:28.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.969+0000 7fd7477fe640 1 --2- 172.21.15.12:0/581273096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6c6e68e0 0x561f6c6ef5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:28.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.969+0000 7fd746ffd640 1 -- 172.21.15.12:0/581273096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6c6e68e0 msgr2=0x561f6c6ef5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:28.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.969+0000 7fd746ffd640 1 --2- 172.21.15.12:0/581273096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6c6e68e0 0x561f6c6ef5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.969+0000 7fd746ffd640 1 -- 172.21.15.12:0/581273096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f6c6efe10 msgr2=0x561f6c6dfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:28.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.969+0000 7fd747fff640 1 --2- 172.21.15.12:0/581273096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f6c6efe10 0x561f6c6dfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:28.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.969+0000 7fd746ffd640 1 --2- 172.21.15.12:0/581273096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f6c6efe10 0x561f6c6dfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.969+0000 7fd746ffd640 1 -- 172.21.15.12:0/581273096 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f6c6e8410 con 0x561f6c6e7290 2024-04-22T13:32:28.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.969+0000 7fd7477fe640 1 --2- 172.21.15.12:0/581273096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6c6e68e0 0x561f6c6ef5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:28.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.969+0000 7fd746ffd640 1 --2- 172.21.15.12:0/581273096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f6c6e7290 0x561f6c6fbbe0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fd73c005a50 tx=0x7fd73c00e710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765de22a41e0e172 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:28.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.969+0000 7fd72d7fa640 1 -- 172.21.15.12:0/581273096 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c03e070 con 0x561f6c6e7290 2024-04-22T13:32:28.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.970+0000 7fd72d7fa640 1 -- 172.21.15.12:0/581273096 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd73c013640 con 0x561f6c6e7290 2024-04-22T13:32:28.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.970+0000 7fd72d7fa640 1 -- 172.21.15.12:0/581273096 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c004aa0 con 0x561f6c6e7290 2024-04-22T13:32:28.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.970+0000 7fd750b6a740 1 -- 172.21.15.12:0/581273096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f6c6e7290 msgr2=0x561f6c6fbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:28.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.970+0000 7fd750b6a740 1 --2- 172.21.15.12:0/581273096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f6c6e7290 0x561f6c6fbbe0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fd73c005a50 tx=0x7fd73c00e710 comp rx=0 tx=0).stop 2024-04-22T13:32:28.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.970+0000 7fd750b6a740 1 -- 172.21.15.12:0/581273096 shutdown_connections 2024-04-22T13:32:28.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.970+0000 7fd750b6a740 1 --2- 172.21.15.12:0/581273096 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f6c6efe10 0x561f6c6dfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.970+0000 7fd750b6a740 1 --2- 172.21.15.12:0/581273096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f6c6e7290 0x561f6c6fbbe0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.970+0000 7fd750b6a740 1 --2- 172.21.15.12:0/581273096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6c6e68e0 0x561f6c6ef5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.970+0000 7fd750b6a740 1 -- 172.21.15.12:0/581273096 >> 172.21.15.12:0/581273096 conn(0x561f6c6e0fb0 msgr2=0x561f6c6df780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:28.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.970+0000 7fd750b6a740 1 -- 172.21.15.12:0/581273096 shutdown_connections 2024-04-22T13:32:28.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.970+0000 7fd750b6a740 1 -- 172.21.15.12:0/581273096 wait complete. 2024-04-22T13:32:28.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.970+0000 7fd750b6a740 1 -- start start 2024-04-22T13:32:28.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.971+0000 7fd750b6a740 1 Processor -- start 2024-04-22T13:32:28.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.971+0000 7fd750b6a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f6c6e68e0 0x561f6c6f3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:28.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.971+0000 7fd750b6a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6c6e7290 0x561f6c6f4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:28.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.971+0000 7fd750b6a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f6c6efe10 0x561f6c6f4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:28.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.971+0000 7fd750b6a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd7481b10a0 con 0x561f6c6e68e0 2024-04-22T13:32:28.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.971+0000 7fd750b6a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd7481b09d0 con 0x561f6c6efe10 2024-04-22T13:32:28.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.971+0000 7fd750b6a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd7481adc30 con 0x561f6c6e7290 2024-04-22T13:32:28.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.971+0000 7fd750b6a740 10 client.0 _ll_register_callbacks cb 0x7fd73800a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:28.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.971+0000 7fd7477fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f6c6e68e0 0x561f6c6f3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:28.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.971+0000 7fd7477fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f6c6e68e0 0x561f6c6f3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34148/0 (socket says 172.21.15.12:34148) 2024-04-22T13:32:28.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.971+0000 7fd7477fe640 1 -- 172.21.15.12:0/1595652284 learned_addr learned my addr 172.21.15.12:0/1595652284 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:28.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.971+0000 7fd747fff640 1 --2- 172.21.15.12:0/1595652284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f6c6efe10 0x561f6c6f4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:28.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.971+0000 7fd746ffd640 1 --2- 172.21.15.12:0/1595652284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6c6e7290 0x561f6c6f4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:28.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.971+0000 7fd7477fe640 1 -- 172.21.15.12:0/1595652284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6c6e7290 msgr2=0x561f6c6f4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:28.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.971+0000 7fd7477fe640 1 --2- 172.21.15.12:0/1595652284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6c6e7290 0x561f6c6f4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.971+0000 7fd7477fe640 1 -- 172.21.15.12:0/1595652284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f6c6efe10 msgr2=0x561f6c6f4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:28.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.971+0000 7fd7477fe640 1 --2- 172.21.15.12:0/1595652284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f6c6efe10 0x561f6c6f4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.971+0000 7fd7477fe640 1 -- 172.21.15.12:0/1595652284 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f6c765b00 con 0x561f6c6e68e0 2024-04-22T13:32:28.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.971+0000 7fd746ffd640 1 --2- 172.21.15.12:0/1595652284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6c6e7290 0x561f6c6f4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:28.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.972+0000 7fd7477fe640 1 --2- 172.21.15.12:0/1595652284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f6c6e68e0 0x561f6c6f3dd0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fd738004700 tx=0x7fd738010af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c9cc50cde59f37 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:28.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.972+0000 7fd726ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:32:28.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.972+0000 7fd726ffd640 1 -- 172.21.15.12:0/1595652284 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd738004220 con 0x561f6c6e68e0 2024-04-22T13:32:28.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.972+0000 7fd726ffd640 1 -- 172.21.15.12:0/1595652284 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7fd7380043c0 con 0x561f6c6e68e0 2024-04-22T13:32:28.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.972+0000 7fd750b6a740 1 -- 172.21.15.12:0/1595652284 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd73c013640 con 0x561f6c6e68e0 2024-04-22T13:32:28.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.972+0000 7fd726ffd640 1 -- 172.21.15.12:0/1595652284 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd738030050 con 0x561f6c6e68e0 2024-04-22T13:32:28.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.972+0000 7fd750b6a740 10 client.18621 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:28.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.972+0000 7fd750b6a740 1 -- 172.21.15.12:0/1595652284 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561f6c766e10 con 0x561f6c6e68e0 2024-04-22T13:32:28.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.972+0000 7fd726ffd640 1 -- 172.21.15.12:0/1595652284 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd73c013640 con 0x561f6c6e68e0 2024-04-22T13:32:28.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.972+0000 7fd726ffd640 1 -- 172.21.15.12:0/1595652284 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd738003c70 con 0x561f6c6e68e0 2024-04-22T13:32:28.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.972+0000 7fd750b6a740 10 client.18621 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:28.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.972+0000 7fd750b6a740 10 client.18621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:28.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.972+0000 7fd750b6a740 20 client.18621 populate_metadata read hostname 'smithi012' 2024-04-22T13:32:28.981 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.972+0000 7fd750b6a740 1 --2- 172.21.15.12:0/1595652284 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x561f6c7674e0 0x561f6c7698d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:28.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.972+0000 7fd750b6a740 4 client.18621 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:32:28.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.972+0000 7fd750b6a740 1 -- 172.21.15.12:0/1595652284 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561f6c652790 con 0x561f6c7674e0 2024-04-22T13:32:28.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.973+0000 7fd746ffd640 1 --2- 172.21.15.12:0/1595652284 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x561f6c7674e0 0x561f6c7698d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:28.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.973+0000 7fd726ffd640 10 client.18621 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:32:28.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.973+0000 7fd746ffd640 1 --2- 172.21.15.12:0/1595652284 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x561f6c7674e0 0x561f6c7698d0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=912637d3039f7c6c server_cookie=495ea35222ef7d8f in_seq=0 out_seq=0 2024-04-22T13:32:28.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.975+0000 7fd726ffd640 1 -- 172.21.15.12:0/1595652284 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561f6c652790 con 0x561f6c7674e0 2024-04-22T13:32:28.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.975+0000 7fd726ffd640 10 client.18621 handle_command_reply: tid=0 2024-04-22T13:32:28.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.975+0000 7fd750b6a740 1 client.18621 shutdown 2024-04-22T13:32:28.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.975+0000 7fd750b6a740 1 -- 172.21.15.12:0/1595652284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f6c6e68e0 msgr2=0x561f6c6f3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:28.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.975+0000 7fd750b6a740 1 --2- 172.21.15.12:0/1595652284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f6c6e68e0 0x561f6c6f3dd0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fd738004700 tx=0x7fd738010af0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.975+0000 7fd750b6a740 1 -- 172.21.15.12:0/1595652284 shutdown_connections 2024-04-22T13:32:28.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.975+0000 7fd750b6a740 1 --2- 172.21.15.12:0/1595652284 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x561f6c7674e0 0x561f6c7698d0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.982 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.975+0000 7fd750b6a740 1 --2- 172.21.15.12:0/1595652284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f6c6efe10 0x561f6c6f4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.975+0000 7fd750b6a740 1 --2- 172.21.15.12:0/1595652284 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561f6c6e7290 0x561f6c6f4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.975+0000 7fd750b6a740 1 --2- 172.21.15.12:0/1595652284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561f6c6e68e0 0x561f6c6f3dd0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.975+0000 7fd750b6a740 1 -- 172.21.15.12:0/1595652284 >> 172.21.15.12:0/1595652284 conn(0x561f6c6e0fb0 msgr2=0x561f6c6e0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:28.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.975+0000 7fd726ffd640 0 client.18621 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:32:28.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.975+0000 7fd750b6a740 1 -- 172.21.15.12:0/1595652284 shutdown_connections 2024-04-22T13:32:28.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.975+0000 7fd750b6a740 1 -- 172.21.15.12:0/1595652284 wait complete. 2024-04-22T13:32:28.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.975+0000 7fd750b6a740 20 client.18621 trim_cache size 0 max 16384 2024-04-22T13:32:28.983 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:32:28.983 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-04-22T13:32:28.984 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "aa515c8b-e412-4be2-802a-5f55c3798af4", 2024-04-22T13:32:28.984 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-04-22T13:32:28.984 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:32:28.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.976+0000 7fd74de9a640 1 -- 172.21.15.12:0/419521963 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd7300a3d20 msgr2=0x7fd7300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:28.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.976+0000 7fd74de9a640 1 --2- 172.21.15.12:0/419521963 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd7300a3d20 0x7fd7300a61e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd73c005940 tx=0x7fd73c015000 comp rx=0 tx=0).stop 2024-04-22T13:32:28.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.976+0000 7fd74de9a640 1 -- 172.21.15.12:0/419521963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd748101900 msgr2=0x7fd7481a68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:28.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.976+0000 7fd74de9a640 1 --2- 172.21.15.12:0/419521963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd748101900 0x7fd7481a68c0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fd7400098d0 tx=0x7fd740035d60 comp rx=0 tx=0).stop 2024-04-22T13:32:28.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.976+0000 7fd74de9a640 1 -- 172.21.15.12:0/419521963 shutdown_connections 2024-04-22T13:32:28.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.976+0000 7fd74de9a640 1 --2- 172.21.15.12:0/419521963 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd7300a3d20 0x7fd7300a61e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.976+0000 7fd74de9a640 1 --2- 172.21.15.12:0/419521963 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd748101900 0x7fd7481a68c0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.976+0000 7fd74de9a640 1 --2- 172.21.15.12:0/419521963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd748107270 0x7fd7481013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.976+0000 7fd74de9a640 1 --2- 172.21.15.12:0/419521963 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd74806a840 0x7fd748100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:28.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.976+0000 7fd74de9a640 1 -- 172.21.15.12:0/419521963 >> 172.21.15.12:0/419521963 conn(0x7fd748077eb0 msgr2=0x7fd7480fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:28.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.976+0000 7fd74de9a640 1 -- 172.21.15.12:0/419521963 shutdown_connections 2024-04-22T13:32:28.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:28.976+0000 7fd74de9a640 1 -- 172.21.15.12:0/419521963 wait complete. 2024-04-22T13:32:28.987 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:32:28.987 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:32:29.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.095+0000 7f778be93640 1 -- 172.21.15.12:0/1360297152 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77841021a0 msgr2=0x7f7784102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:29.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.095+0000 7f778be93640 1 --2- 172.21.15.12:0/1360297152 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77841021a0 0x7f7784102580 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f7780009900 tx=0x7f778002bc30 comp rx=0 tx=0).stop 2024-04-22T13:32:29.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.095+0000 7f778be93640 1 -- 172.21.15.12:0/1360297152 shutdown_connections 2024-04-22T13:32:29.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.095+0000 7f778be93640 1 --2- 172.21.15.12:0/1360297152 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7784107270 0x7f778410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.095+0000 7f778be93640 1 --2- 172.21.15.12:0/1360297152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7784102b50 0x7f7784106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.095+0000 7f778be93640 1 --2- 172.21.15.12:0/1360297152 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77841021a0 0x7f7784102580 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.095+0000 7f778be93640 1 -- 172.21.15.12:0/1360297152 >> 172.21.15.12:0/1360297152 conn(0x7f77840793e0 msgr2=0x7f77840797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:29.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.095+0000 7f778be93640 1 -- 172.21.15.12:0/1360297152 shutdown_connections 2024-04-22T13:32:29.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.095+0000 7f778be93640 1 -- 172.21.15.12:0/1360297152 wait complete. 2024-04-22T13:32:29.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.096+0000 7f778be93640 1 Processor -- start 2024-04-22T13:32:29.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.096+0000 7f778be93640 1 -- start start 2024-04-22T13:32:29.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.096+0000 7f778be93640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77841021a0 0x7f77841986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:29.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.096+0000 7f778be93640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7784107270 0x7f7784198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:29.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.096+0000 7f778be93640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f778419cf70 0x7f778419d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:29.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.096+0000 7f778be93640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f77841121d0 con 0x7f77841021a0 2024-04-22T13:32:29.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.096+0000 7f778be93640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7784112050 con 0x7f7784107270 2024-04-22T13:32:29.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.096+0000 7f778be93640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7784112350 con 0x7f778419cf70 2024-04-22T13:32:29.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.096+0000 7f7789c2f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77841021a0 0x7f77841986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:29.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.096+0000 7f7789c2f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77841021a0 0x7f77841986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34162/0 (socket says 172.21.15.12:34162) 2024-04-22T13:32:29.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.096+0000 7f7789c2f640 1 -- 172.21.15.12:0/1146924239 learned_addr learned my addr 172.21.15.12:0/1146924239 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:29.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.096+0000 7f778a430640 1 --2- 172.21.15.12:0/1146924239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f778419cf70 0x7f778419d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:29.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.097+0000 7f7789c2f640 1 -- 172.21.15.12:0/1146924239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f778419cf70 msgr2=0x7f778419d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:29.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.097+0000 7f7789c2f640 1 --2- 172.21.15.12:0/1146924239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f778419cf70 0x7f778419d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.097+0000 7f7789c2f640 1 -- 172.21.15.12:0/1146924239 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7784107270 msgr2=0x7f7784198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:29.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.097+0000 7f778942e640 1 --2- 172.21.15.12:0/1146924239 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7784107270 0x7f7784198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:29.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.097+0000 7f7789c2f640 1 --2- 172.21.15.12:0/1146924239 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7784107270 0x7f7784198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.097+0000 7f7789c2f640 1 -- 172.21.15.12:0/1146924239 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f778419da60 con 0x7f77841021a0 2024-04-22T13:32:29.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.097+0000 7f778a430640 1 --2- 172.21.15.12:0/1146924239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f778419cf70 0x7f778419d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:29.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.097+0000 7f7789c2f640 1 --2- 172.21.15.12:0/1146924239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77841021a0 0x7f77841986b0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f7780005940 tx=0x7f778002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:29.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.097+0000 7f777affd640 1 -- 172.21.15.12:0/1146924239 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7780043020 con 0x7f77841021a0 2024-04-22T13:32:29.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.097+0000 7f777affd640 1 -- 172.21.15.12:0/1146924239 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f778002cbe0 con 0x7f77841021a0 2024-04-22T13:32:29.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.097+0000 7f778be93640 1 -- 172.21.15.12:0/1146924239 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f778419dcf0 con 0x7f77841021a0 2024-04-22T13:32:29.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.097+0000 7f777affd640 1 -- 172.21.15.12:0/1146924239 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7780003bf0 con 0x7f77841021a0 2024-04-22T13:32:29.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.097+0000 7f778be93640 1 -- 172.21.15.12:0/1146924239 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77841a5510 con 0x7f77841021a0 2024-04-22T13:32:29.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.099+0000 7f777affd640 1 -- 172.21.15.12:0/1146924239 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7780046020 con 0x7f77841021a0 2024-04-22T13:32:29.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.099+0000 7f778eb67740 1 Processor -- start 2024-04-22T13:32:29.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.100+0000 7f778eb67740 1 -- start start 2024-04-22T13:32:29.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.100+0000 7f778eb67740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa680f7580 0x55fa680f7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:29.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.100+0000 7f778eb67740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa680f7ea0 0x55fa68100540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:29.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.100+0000 7f778eb67740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa68100a80 0x55fa68102e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:29.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.100+0000 7f778eb67740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fa6805c1e0 con 0x55fa680f7580 2024-04-22T13:32:29.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.100+0000 7f778eb67740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fa67ff9390 con 0x55fa680f7ea0 2024-04-22T13:32:29.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.100+0000 7f778eb67740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fa680581a0 con 0x55fa68100a80 2024-04-22T13:32:29.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.100+0000 7f778a430640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa680f7580 0x55fa680f7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:29.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.100+0000 7f778a430640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa680f7580 0x55fa680f7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34166/0 (socket says 172.21.15.12:34166) 2024-04-22T13:32:29.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.100+0000 7f778a430640 1 -- 172.21.15.12:0/969060609 learned_addr learned my addr 172.21.15.12:0/969060609 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:29.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.100+0000 7f777affd640 1 --2- 172.21.15.12:0/1146924239 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f77700a3d20 0x7f77700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:29.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.100+0000 7f7789c2f640 1 --2- 172.21.15.12:0/969060609 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa680f7ea0 0x55fa68100540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:29.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.100+0000 7f778942e640 1 --2- 172.21.15.12:0/969060609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa68100a80 0x55fa68102e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:29.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.100+0000 7f778a430640 1 --2- 172.21.15.12:0/1146924239 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f77700a3d20 0x7f77700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:29.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.100+0000 7f778a430640 1 -- 172.21.15.12:0/969060609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa68100a80 msgr2=0x55fa68102e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:29.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.100+0000 7f778a430640 1 --2- 172.21.15.12:0/969060609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa68100a80 0x55fa68102e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.100+0000 7f778a430640 1 -- 172.21.15.12:0/969060609 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa680f7ea0 msgr2=0x55fa68100540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:29.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.100+0000 7f778a430640 1 --2- 172.21.15.12:0/969060609 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa680f7ea0 0x55fa68100540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.100+0000 7f777affd640 1 -- 172.21.15.12:0/1146924239 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f77800f29e0 con 0x7f77841021a0 2024-04-22T13:32:29.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.100+0000 7f778a430640 1 -- 172.21.15.12:0/969060609 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa67e387d0 con 0x55fa680f7580 2024-04-22T13:32:29.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.100+0000 7f778942e640 1 --2- 172.21.15.12:0/969060609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa68100a80 0x55fa68102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:29.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.101+0000 7f778a430640 1 --2- 172.21.15.12:0/969060609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa680f7580 0x55fa680f7960 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f777c0149a0 tx=0x7f777c014e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e836a0b51414d3f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:29.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.101+0000 7f776ffff640 1 -- 172.21.15.12:0/969060609 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777c021070 con 0x55fa680f7580 2024-04-22T13:32:29.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.101+0000 7f776ffff640 1 -- 172.21.15.12:0/969060609 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f777c013460 con 0x55fa680f7580 2024-04-22T13:32:29.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.101+0000 7f776ffff640 1 -- 172.21.15.12:0/969060609 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777c00f5f0 con 0x55fa680f7580 2024-04-22T13:32:29.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.101+0000 7f778eb67740 1 -- 172.21.15.12:0/969060609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa680f7580 msgr2=0x55fa680f7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:29.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.101+0000 7f778a430640 1 --2- 172.21.15.12:0/1146924239 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f77700a3d20 0x7f77700a61e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f777c01e000 tx=0x7f777c019000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:29.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.101+0000 7f778eb67740 1 --2- 172.21.15.12:0/969060609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa680f7580 0x55fa680f7960 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f777c0149a0 tx=0x7f777c014e60 comp rx=0 tx=0).stop 2024-04-22T13:32:29.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.102+0000 7f778eb67740 1 -- 172.21.15.12:0/969060609 shutdown_connections 2024-04-22T13:32:29.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.102+0000 7f778eb67740 1 --2- 172.21.15.12:0/969060609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa68100a80 0x55fa68102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.102+0000 7f778eb67740 1 --2- 172.21.15.12:0/969060609 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa680f7ea0 0x55fa68100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.102+0000 7f778eb67740 1 --2- 172.21.15.12:0/969060609 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa680f7580 0x55fa680f7960 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.102+0000 7f778eb67740 1 -- 172.21.15.12:0/969060609 >> 172.21.15.12:0/969060609 conn(0x55fa680f1cb0 msgr2=0x55fa680f2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:29.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.102+0000 7f778eb67740 1 -- 172.21.15.12:0/969060609 shutdown_connections 2024-04-22T13:32:29.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.102+0000 7f778eb67740 1 -- 172.21.15.12:0/969060609 wait complete. 2024-04-22T13:32:29.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.102+0000 7f778eb67740 1 -- start start 2024-04-22T13:32:29.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.102+0000 7f778eb67740 1 Processor -- start 2024-04-22T13:32:29.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.102+0000 7f778eb67740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa680f7580 0x55fa680fdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:29.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.102+0000 7f778eb67740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa680f7ea0 0x55fa680fdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:29.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.102+0000 7f778eb67740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa68100a80 0x55fa680fe520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:29.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.102+0000 7f778eb67740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55fa68103870 con 0x55fa680f7580 2024-04-22T13:32:29.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.102+0000 7f778eb67740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fa680152f0 con 0x55fa68100a80 2024-04-22T13:32:29.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.102+0000 7f778eb67740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fa681039f0 con 0x55fa680f7ea0 2024-04-22T13:32:29.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.102+0000 7f778eb67740 10 client.0 _ll_register_callbacks cb 0x55fa68022580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:29.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.103+0000 7f7789c2f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa680f7580 0x55fa680fdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:29.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.103+0000 7f7789c2f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa680f7580 0x55fa680fdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34176/0 (socket says 172.21.15.12:34176) 2024-04-22T13:32:29.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.103+0000 7f7789c2f640 1 -- 172.21.15.12:0/764247102 learned_addr learned my addr 172.21.15.12:0/764247102 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:29.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.103+0000 7f778942e640 1 --2- 172.21.15.12:0/764247102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa680f7ea0 0x55fa680fdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:29.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.103+0000 7f7789c2f640 1 -- 172.21.15.12:0/764247102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa680f7ea0 msgr2=0x55fa680fdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:29.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.103+0000 7f7789c2f640 1 --2- 172.21.15.12:0/764247102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa680f7ea0 0x55fa680fdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.103+0000 7f7789c2f640 1 -- 172.21.15.12:0/764247102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa68100a80 msgr2=0x55fa680fe520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:29.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.103+0000 7f778a430640 1 --2- 172.21.15.12:0/764247102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa68100a80 0x55fa680fe520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:29.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.103+0000 7f7789c2f640 1 --2- 172.21.15.12:0/764247102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa68100a80 0x55fa680fe520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.103+0000 7f7789c2f640 1 -- 172.21.15.12:0/764247102 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa67d6e3f0 con 0x55fa680f7580 2024-04-22T13:32:29.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.103+0000 7f778942e640 1 --2- 172.21.15.12:0/764247102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa680f7ea0 0x55fa680fdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:29.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.103+0000 7f7789c2f640 1 --2- 172.21.15.12:0/764247102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa680f7580 0x55fa680fdaa0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f7780006010 tx=0x7f77800be5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c632f3f25d92b5be server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:29.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.103+0000 7f776d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:32:29.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.103+0000 7f776d7fa640 1 -- 172.21.15.12:0/764247102 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778003c350 con 0x55fa680f7580 2024-04-22T13:32:29.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.103+0000 7f776d7fa640 1 -- 172.21.15.12:0/764247102 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f77800054d0 con 0x55fa680f7580 2024-04-22T13:32:29.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.103+0000 7f778eb67740 1 -- 172.21.15.12:0/764247102 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f777c013460 con 0x55fa680f7580 2024-04-22T13:32:29.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.103+0000 7f776d7fa640 1 -- 172.21.15.12:0/764247102 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f77800beab0 con 0x55fa680f7580 2024-04-22T13:32:29.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.103+0000 7f778eb67740 10 client.18633 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:29.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.103+0000 7f778eb67740 1 -- 172.21.15.12:0/764247102 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fa67d2e0f0 con 0x55fa680f7580 2024-04-22T13:32:29.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.104+0000 7f776d7fa640 1 -- 172.21.15.12:0/764247102 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f777c013460 con 0x55fa680f7580 2024-04-22T13:32:29.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.104+0000 7f776d7fa640 1 -- 172.21.15.12:0/764247102 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f77800c7290 con 0x55fa680f7580 2024-04-22T13:32:29.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.104+0000 7f778eb67740 10 client.18633 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:29.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.104+0000 7f778eb67740 10 client.18633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:29.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.104+0000 7f778eb67740 20 client.18633 populate_metadata read hostname 'smithi012' 2024-04-22T13:32:29.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.104+0000 7f778eb67740 1 --2- 172.21.15.12:0/764247102 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55fa68114db0 0x55fa68117250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:29.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.104+0000 7f778eb67740 4 client.18633 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:29.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.104+0000 7f778eb67740 1 -- 172.21.15.12:0/764247102 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa68069c00 con 0x55fa68114db0 2024-04-22T13:32:29.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.104+0000 7f778942e640 1 --2- 172.21.15.12:0/764247102 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55fa68114db0 0x55fa68117250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:29.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.104+0000 7f776d7fa640 10 client.18633 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:32:29.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.104+0000 7f778942e640 1 --2- 172.21.15.12:0/764247102 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55fa68114db0 0x55fa68117250 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a458e6d4ffd46e57 server_cookie=7f712510bcae34e2 in_seq=0 out_seq=0 2024-04-22T13:32:29.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.105+0000 7f776d7fa640 1 -- 172.21.15.12:0/764247102 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fa68069c00 con 0x55fa68114db0 2024-04-22T13:32:29.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.105+0000 7f776d7fa640 10 client.18633 handle_command_reply: tid=0 2024-04-22T13:32:29.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.105+0000 7f778eb67740 1 client.18633 shutdown 2024-04-22T13:32:29.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.105+0000 7f778eb67740 1 -- 172.21.15.12:0/764247102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa680f7580 msgr2=0x55fa680fdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:29.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.105+0000 7f778eb67740 1 --2- 172.21.15.12:0/764247102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa680f7580 0x55fa680fdaa0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f7780006010 tx=0x7f77800be5f0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.105+0000 7f778eb67740 1 -- 172.21.15.12:0/764247102 shutdown_connections 2024-04-22T13:32:29.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.105+0000 7f778eb67740 1 --2- 172.21.15.12:0/764247102 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55fa68114db0 0x55fa68117250 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.105+0000 7f778eb67740 1 --2- 172.21.15.12:0/764247102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa68100a80 0x55fa680fe520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.105+0000 7f776d7fa640 0 client.18633 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:32:29.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.105+0000 7f778eb67740 1 --2- 172.21.15.12:0/764247102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa680f7ea0 0x55fa680fdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.105+0000 7f778eb67740 1 --2- 172.21.15.12:0/764247102 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa680f7580 0x55fa680fdaa0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.105+0000 7f778eb67740 1 -- 172.21.15.12:0/764247102 >> 172.21.15.12:0/764247102 conn(0x55fa680f1cb0 msgr2=0x55fa680f3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:29.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.105+0000 7f778eb67740 1 -- 172.21.15.12:0/764247102 shutdown_connections 2024-04-22T13:32:29.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.106+0000 7f778eb67740 1 -- 172.21.15.12:0/764247102 wait complete. 2024-04-22T13:32:29.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.106+0000 7f778eb67740 20 client.18633 trim_cache size 0 max 16384 2024-04-22T13:32:29.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.123+0000 7f778eb67740 1 Processor -- start 2024-04-22T13:32:29.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.123+0000 7f778eb67740 1 -- start start 2024-04-22T13:32:29.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.123+0000 7f778eb67740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa680f7580 0x55fa681001d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:29.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.123+0000 7f778eb67740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa680f7ea0 0x55fa68110bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:29.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.123+0000 7f778eb67740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa68100a80 0x55fa680fc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:29.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.123+0000 7f778eb67740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f77841ab830 con 0x55fa680f7580 2024-04-22T13:32:29.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.123+0000 7f778eb67740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f77841ad370 con 0x55fa680f7ea0 2024-04-22T13:32:29.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.123+0000 7f778eb67740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f77841abf00 con 0x55fa68100a80 2024-04-22T13:32:29.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.124+0000 7f7789c2f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa680f7580 0x55fa681001d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:29.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.124+0000 7f7789c2f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa680f7580 0x55fa681001d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34186/0 (socket says 172.21.15.12:34186) 2024-04-22T13:32:29.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.124+0000 7f7789c2f640 1 -- 172.21.15.12:0/2501509028 learned_addr learned my addr 172.21.15.12:0/2501509028 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:29.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.124+0000 7f778a430640 1 --2- 172.21.15.12:0/2501509028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa68100a80 0x55fa680fc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:29.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.124+0000 7f7789c2f640 1 -- 172.21.15.12:0/2501509028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa68100a80 msgr2=0x55fa680fc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:29.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.124+0000 7f7789c2f640 1 --2- 172.21.15.12:0/2501509028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa68100a80 0x55fa680fc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.124+0000 7f7789c2f640 1 -- 172.21.15.12:0/2501509028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa680f7ea0 msgr2=0x55fa68110bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:29.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.124+0000 7f778942e640 1 --2- 172.21.15.12:0/2501509028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa680f7ea0 0x55fa68110bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:29.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.124+0000 7f7789c2f640 1 --2- 172.21.15.12:0/2501509028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa680f7ea0 0x55fa68110bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.124+0000 7f7789c2f640 1 -- 172.21.15.12:0/2501509028 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa680f8e00 con 0x55fa680f7580 2024-04-22T13:32:29.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.124+0000 7f778a430640 1 --2- 172.21.15.12:0/2501509028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa68100a80 0x55fa680fc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:29.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.124+0000 7f778942e640 1 --2- 172.21.15.12:0/2501509028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa680f7ea0 0x55fa68110bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:29.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.124+0000 7f7789c2f640 1 --2- 172.21.15.12:0/2501509028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa680f7580 0x55fa681001d0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f77800f2e60 tx=0x7f778002cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22beb17d1ff2ec12 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:29.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.124+0000 7f776f7fe640 1 -- 172.21.15.12:0/2501509028 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77800bab80 con 0x55fa680f7580 2024-04-22T13:32:29.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.124+0000 7f776f7fe640 1 -- 172.21.15.12:0/2501509028 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f778003c350 con 0x55fa680f7580 2024-04-22T13:32:29.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.124+0000 7f776f7fe640 1 -- 172.21.15.12:0/2501509028 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77800c7290 con 0x55fa680f7580 2024-04-22T13:32:29.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.125+0000 7f778eb67740 1 -- 172.21.15.12:0/2501509028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa680f7580 msgr2=0x55fa681001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:29.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.125+0000 7f778eb67740 1 --2- 172.21.15.12:0/2501509028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa680f7580 0x55fa681001d0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f77800f2e60 tx=0x7f778002cd90 comp rx=0 tx=0).stop 2024-04-22T13:32:29.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.125+0000 7f778eb67740 1 -- 172.21.15.12:0/2501509028 shutdown_connections 2024-04-22T13:32:29.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.125+0000 7f778eb67740 1 --2- 172.21.15.12:0/2501509028 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa68100a80 0x55fa680fc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.125+0000 7f778eb67740 1 --2- 172.21.15.12:0/2501509028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa680f7ea0 0x55fa68110bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.125+0000 7f778eb67740 1 --2- 172.21.15.12:0/2501509028 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa680f7580 0x55fa681001d0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.125+0000 7f778eb67740 1 -- 172.21.15.12:0/2501509028 >> 172.21.15.12:0/2501509028 conn(0x55fa680f1cb0 msgr2=0x55fa680f0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:29.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.125+0000 7f778eb67740 1 -- 172.21.15.12:0/2501509028 shutdown_connections 2024-04-22T13:32:29.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.125+0000 7f778eb67740 1 -- 172.21.15.12:0/2501509028 wait complete. 2024-04-22T13:32:29.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.125+0000 7f778eb67740 1 -- start start 2024-04-22T13:32:29.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.125+0000 7f778eb67740 1 Processor -- start 2024-04-22T13:32:29.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.126+0000 7f778eb67740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa680f7580 0x55fa6810d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:29.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.126+0000 7f778eb67740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa680f7ea0 0x55fa6810d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:29.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.126+0000 7f778eb67740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa68100a80 0x55fa6810a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:29.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.126+0000 7f778eb67740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f77841acca0 con 0x55fa68100a80 2024-04-22T13:32:29.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.126+0000 7f778eb67740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f77841ac5d0 con 0x55fa680f7580 2024-04-22T13:32:29.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.126+0000 7f778eb67740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f77841a9850 con 0x55fa680f7ea0 2024-04-22T13:32:29.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.126+0000 7f778eb67740 10 client.0 _ll_register_callbacks cb 0x7f77800057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:29.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.126+0000 7f778a430640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa68100a80 0x55fa6810a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:29.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.126+0000 7f778a430640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa68100a80 0x55fa6810a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:34198/0 (socket says 172.21.15.12:34198) 2024-04-22T13:32:29.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.126+0000 7f778a430640 1 -- 172.21.15.12:0/3311618349 learned_addr learned my addr 172.21.15.12:0/3311618349 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:29.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.126+0000 7f7789c2f640 1 --2- 172.21.15.12:0/3311618349 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa680f7580 0x55fa6810d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:29.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.126+0000 7f778942e640 1 --2- 172.21.15.12:0/3311618349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa680f7ea0 0x55fa6810d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:29.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.126+0000 7f778a430640 1 -- 172.21.15.12:0/3311618349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa680f7ea0 msgr2=0x55fa6810d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:29.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.126+0000 7f778a430640 1 --2- 172.21.15.12:0/3311618349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa680f7ea0 0x55fa6810d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.126+0000 7f778a430640 1 -- 172.21.15.12:0/3311618349 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa680f7580 msgr2=0x55fa6810d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:29.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.126+0000 7f778a430640 1 --2- 172.21.15.12:0/3311618349 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa680f7580 0x55fa6810d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.126+0000 7f778a430640 1 -- 172.21.15.12:0/3311618349 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa6810a8d0 con 0x55fa68100a80 2024-04-22T13:32:29.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.126+0000 7f778a430640 1 --2- 172.21.15.12:0/3311618349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa68100a80 0x55fa6810a0a0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f777c005cc0 tx=0x7f777c00e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f496f1ebf536fe server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:29.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.126+0000 7f776d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:32:29.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.126+0000 7f776d7fa640 1 -- 172.21.15.12:0/3311618349 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777c032060 con 0x55fa68100a80 2024-04-22T13:32:29.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.126+0000 7f776d7fa640 1 -- 172.21.15.12:0/3311618349 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f777c014250 con 0x55fa68100a80 2024-04-22T13:32:29.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.126+0000 7f778eb67740 1 -- 172.21.15.12:0/3311618349 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f778003c350 con 0x55fa68100a80 2024-04-22T13:32:29.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.126+0000 7f776d7fa640 1 -- 172.21.15.12:0/3311618349 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f777c021070 con 0x55fa68100a80 2024-04-22T13:32:29.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.126+0000 7f778eb67740 10 client.18639 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:29.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.126+0000 7f778eb67740 1 -- 172.21.15.12:0/3311618349 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fa6817be40 con 0x55fa68100a80 2024-04-22T13:32:29.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.127+0000 7f776d7fa640 1 -- 172.21.15.12:0/3311618349 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f778003c350 con 0x55fa68100a80 2024-04-22T13:32:29.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.127+0000 7f776d7fa640 1 -- 172.21.15.12:0/3311618349 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f777c013460 con 0x55fa68100a80 2024-04-22T13:32:29.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.127+0000 7f778eb67740 10 client.18639 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:29.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.127+0000 7f778eb67740 10 client.18639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:29.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.127+0000 7f778eb67740 20 client.18639 populate_metadata read hostname 'smithi012' 2024-04-22T13:32:29.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.127+0000 7f778eb67740 1 --2- 172.21.15.12:0/3311618349 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55fa6817c400 0x55fa6817e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:29.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.127+0000 7f778eb67740 4 client.18639 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:32:29.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.127+0000 7f778eb67740 1 -- 172.21.15.12:0/3311618349 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fa68063a70 con 0x55fa6817c400 2024-04-22T13:32:29.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.127+0000 7f7789c2f640 1 --2- 172.21.15.12:0/3311618349 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55fa6817c400 0x55fa6817e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:29.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.127+0000 7f776d7fa640 10 client.18639 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:32:29.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.127+0000 7f7789c2f640 1 --2- 172.21.15.12:0/3311618349 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55fa6817c400 0x55fa6817e820 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ebdd423866e1f98 server_cookie=bd0dc32c30dcff7a in_seq=0 out_seq=0 2024-04-22T13:32:29.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.128+0000 7f776d7fa640 1 -- 172.21.15.12:0/3311618349 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55fa68063a70 con 0x55fa6817c400 2024-04-22T13:32:29.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.128+0000 7f776d7fa640 10 client.18639 handle_command_reply: tid=0 2024-04-22T13:32:29.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.128+0000 7f778eb67740 1 client.18639 shutdown 2024-04-22T13:32:29.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.128+0000 7f778eb67740 1 -- 172.21.15.12:0/3311618349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa68100a80 msgr2=0x55fa6810a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:29.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.128+0000 7f778eb67740 1 --2- 172.21.15.12:0/3311618349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa68100a80 0x55fa6810a0a0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f777c005cc0 tx=0x7f777c00e000 comp rx=0 tx=0).stop 2024-04-22T13:32:29.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.128+0000 7f778eb67740 1 -- 172.21.15.12:0/3311618349 shutdown_connections 2024-04-22T13:32:29.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.128+0000 7f778eb67740 1 --2- 172.21.15.12:0/3311618349 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55fa6817c400 0x55fa6817e820 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.128+0000 7f778eb67740 1 --2- 172.21.15.12:0/3311618349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55fa68100a80 0x55fa6810a0a0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.128+0000 7f778eb67740 1 --2- 172.21.15.12:0/3311618349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fa680f7ea0 0x55fa6810d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.128+0000 7f778eb67740 1 --2- 172.21.15.12:0/3311618349 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fa680f7580 0x55fa6810d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.128+0000 7f776d7fa640 0 client.18639 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:32:29.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.128+0000 7f778eb67740 1 -- 172.21.15.12:0/3311618349 >> 172.21.15.12:0/3311618349 conn(0x55fa680f1cb0 msgr2=0x55fa680f04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:29.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.128+0000 7f778eb67740 1 -- 172.21.15.12:0/3311618349 shutdown_connections 2024-04-22T13:32:29.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.128+0000 7f778eb67740 1 -- 172.21.15.12:0/3311618349 wait complete. 2024-04-22T13:32:29.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.128+0000 7f778eb67740 20 client.18639 trim_cache size 0 max 16384 2024-04-22T13:32:29.132 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:32:29.132 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:32:29.132 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-04-22T13:32:29.132 INFO:teuthology.orchestra.run.smithi012.stdout: "aa515c8b-e412-4be2-802a-5f55c3798af4": { 2024-04-22T13:32:29.132 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-04-22T13:32:29.132 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "aa515c8b-e412-4be2-802a-5f55c3798af4", 2024-04-22T13:32:29.132 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-04-22T13:32:29.132 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:32:29.132 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:32:29.132 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:32:29.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.129+0000 7f778be93640 1 -- 172.21.15.12:0/1146924239 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f77700a3d20 msgr2=0x7f77700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:29.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.129+0000 7f778be93640 1 --2- 172.21.15.12:0/1146924239 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f77700a3d20 0x7f77700a61e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f777c01e000 tx=0x7f777c019000 comp rx=0 tx=0).stop 2024-04-22T13:32:29.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.129+0000 7f778be93640 1 -- 172.21.15.12:0/1146924239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77841021a0 msgr2=0x7f77841986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:29.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.129+0000 7f778be93640 1 --2- 172.21.15.12:0/1146924239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77841021a0 0x7f77841986b0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f7780005940 tx=0x7f778002c7e0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.129+0000 7f778be93640 1 -- 172.21.15.12:0/1146924239 shutdown_connections 2024-04-22T13:32:29.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.129+0000 7f778be93640 1 --2- 172.21.15.12:0/1146924239 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f77700a3d20 0x7f77700a61e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.129+0000 7f778be93640 1 --2- 172.21.15.12:0/1146924239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f778419cf70 0x7f778419d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.129+0000 7f778be93640 1 --2- 172.21.15.12:0/1146924239 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7784107270 0x7f7784198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.129+0000 7f778be93640 1 --2- 172.21.15.12:0/1146924239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f77841021a0 0x7f77841986b0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:29.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.129+0000 7f778be93640 1 -- 172.21.15.12:0/1146924239 >> 172.21.15.12:0/1146924239 conn(0x7f77840793e0 msgr2=0x7f77840ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:29.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.129+0000 7f778be93640 1 -- 172.21.15.12:0/1146924239 shutdown_connections 2024-04-22T13:32:29.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:29.129+0000 7f778be93640 1 -- 172.21.15.12:0/1146924239 wait complete. 2024-04-22T13:32:29.135 INFO:tasks.cephfs.filesystem:scrub status for tag:aa515c8b-e412-4be2-802a-5f55c3798af4 - {'path': '/', 'tag': 'aa515c8b-e412-4be2-802a-5f55c3798af4', 'options': 'recursive,force'} 2024-04-22T13:32:29.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:29 smithi134 ceph-mon[31338]: pgmap v258: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-22T13:32:29.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:29 smithi111 ceph-mon[31361]: pgmap v258: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-22T13:32:30.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:29 smithi012 ceph-mon[28265]: pgmap v258: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-22T13:32:30.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:30 smithi134 ceph-mon[31338]: scrub queued for path: / 2024-04-22T13:32:30.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:30 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:32:30.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:30 smithi134 ceph-mon[31338]: scrub summary: active paths [/] 2024-04-22T13:32:30.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:30 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:32:30.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:30 smithi134 ceph-mon[31338]: scrub summary: idle 2024-04-22T13:32:30.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:30 smithi111 ceph-mon[31361]: scrub queued for path: / 2024-04-22T13:32:30.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:30 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:32:30.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:30 smithi111 ceph-mon[31361]: scrub summary: active paths [/] 2024-04-22T13:32:30.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:30 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:32:30.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:30 smithi111 ceph-mon[31361]: scrub summary: idle 2024-04-22T13:32:31.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:30 smithi012 ceph-mon[28265]: scrub queued for path: / 2024-04-22T13:32:31.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:30 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:32:31.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:30 smithi012 ceph-mon[28265]: scrub summary: active paths [/] 2024-04-22T13:32:31.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:30 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:32:31.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:30 smithi012 ceph-mon[28265]: scrub summary: idle 2024-04-22T13:32:31.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:31 smithi134 ceph-mon[31338]: pgmap v259: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-22T13:32:31.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:31 smithi111 ceph-mon[31361]: pgmap v259: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-22T13:32:32.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:31 smithi012 ceph-mon[28265]: pgmap v259: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-22T13:32:33.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:33 smithi134 ceph-mon[31338]: pgmap v260: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:32:33.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:33 smithi111 ceph-mon[31361]: pgmap v260: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:32:34.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:33 smithi012 ceph-mon[28265]: pgmap v260: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:32:35.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:35 smithi134 ceph-mon[31338]: pgmap v261: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:32:35.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:35 smithi111 ceph-mon[31361]: pgmap v261: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:32:36.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:35 smithi012 ceph-mon[28265]: pgmap v261: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:32:37.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:37 smithi134 ceph-mon[31338]: pgmap v262: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:32:37.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:37 smithi111 ceph-mon[31361]: pgmap v262: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:32:38.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:37 smithi012 ceph-mon[28265]: pgmap v262: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:32:39.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:39 smithi134 ceph-mon[31338]: pgmap v263: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:32:39.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:39 smithi111 ceph-mon[31361]: pgmap v263: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:32:40.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:39 smithi012 ceph-mon[28265]: pgmap v263: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:32:41.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:41 smithi134 ceph-mon[31338]: pgmap v264: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:32:41.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:41 smithi111 ceph-mon[31361]: pgmap v264: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:32:42.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:41 smithi012 ceph-mon[28265]: pgmap v264: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:32:43.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:43 smithi134 ceph-mon[31338]: pgmap v265: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:32:43.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:43 smithi111 ceph-mon[31361]: pgmap v265: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:32:44.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:43 smithi012 ceph-mon[28265]: pgmap v265: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:32:45.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:45 smithi134 ceph-mon[31338]: pgmap v266: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:45.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:45 smithi111 ceph-mon[31361]: pgmap v266: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:46.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:45 smithi012 ceph-mon[28265]: pgmap v266: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:47.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:47 smithi134 ceph-mon[31338]: pgmap v267: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:47.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:47 smithi111 ceph-mon[31361]: pgmap v267: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:48.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:47 smithi012 ceph-mon[28265]: pgmap v267: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:49.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:49 smithi134 ceph-mon[31338]: pgmap v268: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:32:49.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:49 smithi111 ceph-mon[31361]: pgmap v268: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:32:50.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:49 smithi012 ceph-mon[28265]: pgmap v268: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:32:51.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:51 smithi134 ceph-mon[31338]: pgmap v269: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:51.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:51 smithi111 ceph-mon[31361]: pgmap v269: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:52.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:51 smithi012 ceph-mon[28265]: pgmap v269: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:53.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:53 smithi134 ceph-mon[31338]: pgmap v270: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:32:53.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:53 smithi111 ceph-mon[31361]: pgmap v270: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:32:54.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:53 smithi012 ceph-mon[28265]: pgmap v270: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:32:55.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:55 smithi134 ceph-mon[31338]: pgmap v271: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:55.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:55 smithi111 ceph-mon[31361]: pgmap v271: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:56.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:55 smithi012 ceph-mon[28265]: pgmap v271: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:57.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:57 smithi134 ceph-mon[31338]: pgmap v272: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:57.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:57 smithi111 ceph-mon[31361]: pgmap v272: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:58.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:57 smithi012 ceph-mon[28265]: pgmap v272: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:32:59.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:32:59.136 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:32:59.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.259+0000 7f5fb2323640 1 -- 172.21.15.12:0/3094342505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fac100d40 msgr2=0x7f5fac101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.259+0000 7f5fb2323640 1 --2- 172.21.15.12:0/3094342505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fac100d40 0x7f5fac101120 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009900 tx=0x7f5f9c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:32:59.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.259+0000 7f5fb2323640 1 -- 172.21.15.12:0/3094342505 shutdown_connections 2024-04-22T13:32:59.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.259+0000 7f5fb2323640 1 --2- 172.21.15.12:0/3094342505 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fac10e8f0 0x7f5fac110ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.259+0000 7f5fb2323640 1 --2- 172.21.15.12:0/3094342505 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fac101660 0x7f5fac10e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.259+0000 7f5fb2323640 1 --2- 172.21.15.12:0/3094342505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fac100d40 0x7f5fac101120 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.259+0000 7f5fb2323640 1 -- 172.21.15.12:0/3094342505 >> 172.21.15.12:0/3094342505 conn(0x7f5fac078000 msgr2=0x7f5fac078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:59.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.259+0000 7f5fb2323640 1 -- 172.21.15.12:0/3094342505 shutdown_connections 2024-04-22T13:32:59.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.259+0000 7f5fb2323640 1 -- 172.21.15.12:0/3094342505 wait complete. 2024-04-22T13:32:59.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.259+0000 7f5fb2323640 1 Processor -- start 2024-04-22T13:32:59.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.259+0000 7f5fb2323640 1 -- start start 2024-04-22T13:32:59.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.260+0000 7f5fb2323640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fac100d40 0x7f5fac1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.260+0000 7f5fb2323640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fac101660 0x7f5fac1a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.260+0000 7f5fb2323640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fac10e8f0 0x7f5fac19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.260+0000 7f5fb2323640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac1143b0 con 0x7f5fac10e8f0 2024-04-22T13:32:59.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.260+0000 7f5fb2323640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac114230 con 0x7f5fac101660 2024-04-22T13:32:59.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.260+0000 7f5fb2323640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac114530 con 0x7f5fac100d40 2024-04-22T13:32:59.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.260+0000 7f5fb08c0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fac10e8f0 0x7f5fac19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.260+0000 7f5fabfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fac100d40 0x7f5fac1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.260+0000 7f5fabfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fac100d40 0x7f5fac1a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.12:47742/0 (socket says 172.21.15.12:47742) 2024-04-22T13:32:59.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.260+0000 7f5fb08c0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fac10e8f0 0x7f5fac19a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51280/0 (socket says 172.21.15.12:51280) 2024-04-22T13:32:59.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.260+0000 7f5fabfff640 1 -- 172.21.15.12:0/2719854106 learned_addr learned my addr 172.21.15.12:0/2719854106 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:59.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.260+0000 7f5fab7fe640 1 --2- 172.21.15.12:0/2719854106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fac101660 0x7f5fac1a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.260+0000 7f5fabfff640 1 -- 172.21.15.12:0/2719854106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fac101660 msgr2=0x7f5fac1a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.260+0000 7f5fabfff640 1 --2- 172.21.15.12:0/2719854106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fac101660 0x7f5fac1a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.260+0000 7f5fabfff640 1 -- 172.21.15.12:0/2719854106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fac10e8f0 msgr2=0x7f5fac19a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.258 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.260+0000 7f5fabfff640 1 --2- 172.21.15.12:0/2719854106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fac10e8f0 0x7f5fac19a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.260+0000 7f5fabfff640 1 -- 172.21.15.12:0/2719854106 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fac19af90 con 0x7f5fac100d40 2024-04-22T13:32:59.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.260+0000 7f5fb08c0640 1 --2- 172.21.15.12:0/2719854106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fac10e8f0 0x7f5fac19a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:32:59.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.261+0000 7f5fabfff640 1 --2- 172.21.15.12:0/2719854106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fac100d40 0x7f5fac1a0610 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c005940 tx=0x7f5f9c0040e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:59.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.261+0000 7f5fa97fa640 1 -- 172.21.15.12:0/2719854106 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c043020 con 0x7f5fac100d40 2024-04-22T13:32:59.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.261+0000 7f5fb2323640 1 -- 172.21.15.12:0/2719854106 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fac19b220 con 0x7f5fac100d40 2024-04-22T13:32:59.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.261+0000 7f5fa97fa640 1 -- 172.21.15.12:0/2719854106 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f9c0052b0 con 0x7f5fac100d40 2024-04-22T13:32:59.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.261+0000 7f5fa97fa640 1 -- 172.21.15.12:0/2719854106 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c003c00 con 0x7f5fac100d40 2024-04-22T13:32:59.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.261+0000 7f5fb2323640 1 -- 172.21.15.12:0/2719854106 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fac1a7590 con 0x7f5fac100d40 2024-04-22T13:32:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.263+0000 7f5fb4fec740 1 Processor -- start 2024-04-22T13:32:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.263+0000 7f5fb4fec740 1 -- start start 2024-04-22T13:32:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.263+0000 7f5fa97fa640 1 -- 172.21.15.12:0/2719854106 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5f9c046020 con 0x7f5fac100d40 2024-04-22T13:32:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.264+0000 7f5fb4fec740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55904c3b7580 0x55904c3b7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.264+0000 7f5fb4fec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55904c3b7ea0 0x55904c3c0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.264+0000 7f5fb4fec740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904c3c0a80 0x55904c3c2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.264+0000 7f5fb4fec740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55904c31c1e0 con 0x55904c3c0a80 2024-04-22T13:32:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.264+0000 7f5fb4fec740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55904c2b9390 con 0x55904c3b7580 2024-04-22T13:32:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.264+0000 7f5fb4fec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55904c3181a0 con 0x55904c3b7ea0 2024-04-22T13:32:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.264+0000 7f5fab7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904c3c0a80 0x55904c3c2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.264+0000 7f5fab7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904c3c0a80 0x55904c3c2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51294/0 (socket says 172.21.15.12:51294) 2024-04-22T13:32:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.264+0000 7f5fab7fe640 1 -- 172.21.15.12:0/3550559493 learned_addr learned my addr 172.21.15.12:0/3550559493 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.264+0000 7f5fabfff640 1 --2- 172.21.15.12:0/3550559493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55904c3b7ea0 0x55904c3c0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.264+0000 7f5fb08c0640 1 --2- 172.21.15.12:0/3550559493 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55904c3b7580 0x55904c3b7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.264+0000 7f5fab7fe640 1 -- 172.21.15.12:0/3550559493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55904c3b7ea0 msgr2=0x55904c3c0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.264+0000 7f5fab7fe640 1 --2- 172.21.15.12:0/3550559493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55904c3b7ea0 0x55904c3c0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.264+0000 7f5fab7fe640 1 -- 172.21.15.12:0/3550559493 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55904c3b7580 msgr2=0x55904c3b7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.264+0000 7f5fab7fe640 1 --2- 172.21.15.12:0/3550559493 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55904c3b7580 0x55904c3b7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.264+0000 7f5fab7fe640 1 -- 172.21.15.12:0/3550559493 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55904c0f87d0 con 0x55904c3c0a80 2024-04-22T13:32:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.264+0000 7f5fabfff640 1 --2- 172.21.15.12:0/3550559493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55904c3b7ea0 0x55904c3c0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.264+0000 7f5fb08c0640 1 --2- 172.21.15.12:0/3550559493 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55904c3b7580 0x55904c3b7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.264+0000 7f5fa97fa640 1 --2- 172.21.15.12:0/2719854106 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5f940a3d20 0x7f5f940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.264+0000 7f5fab7fe640 1 --2- 172.21.15.12:0/3550559493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904c3c0a80 0x55904c3c2e70 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000aca0 tx=0x7f5fa000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b731a01959481cf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.264+0000 7f5fb08c0640 1 --2- 172.21.15.12:0/2719854106 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5f940a3d20 0x7f5f940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.264+0000 7f5fa97fa640 1 -- 172.21.15.12:0/2719854106 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f5f9c0f29d0 con 0x7f5fac100d40 2024-04-22T13:32:59.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.265+0000 7f5f927fc640 1 -- 172.21.15.12:0/3550559493 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa001d070 con 0x55904c3c0a80 2024-04-22T13:32:59.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.265+0000 7f5f927fc640 1 -- 172.21.15.12:0/3550559493 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fa0018af0 con 0x55904c3c0a80 2024-04-22T13:32:59.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.265+0000 7f5f927fc640 1 -- 172.21.15.12:0/3550559493 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0010bf0 con 0x55904c3c0a80 2024-04-22T13:32:59.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.265+0000 7f5fb4fec740 1 -- 172.21.15.12:0/3550559493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904c3c0a80 msgr2=0x55904c3c2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.265+0000 7f5fb08c0640 1 --2- 172.21.15.12:0/2719854106 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5f940a3d20 0x7f5f940a61e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4005c40 tx=0x7f5fa400a4f0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:59.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.265+0000 7f5fb4fec740 1 --2- 172.21.15.12:0/3550559493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904c3c0a80 0x55904c3c2e70 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000aca0 tx=0x7f5fa000f790 comp rx=0 tx=0).stop 2024-04-22T13:32:59.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.265+0000 7f5fb4fec740 1 -- 172.21.15.12:0/3550559493 shutdown_connections 2024-04-22T13:32:59.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.265+0000 7f5fb4fec740 1 --2- 172.21.15.12:0/3550559493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904c3c0a80 0x55904c3c2e70 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.265+0000 7f5fb4fec740 1 --2- 172.21.15.12:0/3550559493 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55904c3b7ea0 0x55904c3c0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.265+0000 7f5fb4fec740 1 --2- 172.21.15.12:0/3550559493 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55904c3b7580 0x55904c3b7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.265+0000 7f5fb4fec740 1 -- 172.21.15.12:0/3550559493 >> 172.21.15.12:0/3550559493 conn(0x55904c3b1cb0 msgr2=0x55904c3b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:59.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.265+0000 7f5fb4fec740 1 -- 172.21.15.12:0/3550559493 shutdown_connections 2024-04-22T13:32:59.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.265+0000 7f5fb4fec740 1 -- 172.21.15.12:0/3550559493 wait complete. 2024-04-22T13:32:59.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.266+0000 7f5fb4fec740 1 -- start start 2024-04-22T13:32:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.266+0000 7f5fb4fec740 1 Processor -- start 2024-04-22T13:32:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.266+0000 7f5fb4fec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55904c3b7580 0x55904c3bda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.266+0000 7f5fb4fec740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904c3b7ea0 0x55904c3bdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.266+0000 7f5fb4fec740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55904c3c0a80 0x55904c3be4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.266+0000 7f5fb4fec740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55904c3c3870 con 0x55904c3b7ea0 2024-04-22T13:32:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.266+0000 7f5fb4fec740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55904c2d52f0 con 0x55904c3c0a80 2024-04-22T13:32:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.266+0000 7f5fb4fec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55904c3c39f0 con 0x55904c3b7580 2024-04-22T13:32:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.266+0000 7f5fb4fec740 10 client.0 _ll_register_callbacks cb 0x55904c2e2580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.266+0000 7f5fab7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904c3b7ea0 0x55904c3bdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.266+0000 7f5fab7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904c3b7ea0 0x55904c3bdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51304/0 (socket says 172.21.15.12:51304) 2024-04-22T13:32:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.266+0000 7f5fab7fe640 1 -- 172.21.15.12:0/2511886248 learned_addr learned my addr 172.21.15.12:0/2511886248 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.266+0000 7f5fb08c0640 1 --2- 172.21.15.12:0/2511886248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55904c3c0a80 0x55904c3be4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.266+0000 7f5fabfff640 1 --2- 172.21.15.12:0/2511886248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55904c3b7580 0x55904c3bda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.267+0000 7f5fab7fe640 1 -- 172.21.15.12:0/2511886248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55904c3b7580 msgr2=0x55904c3bda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:59.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.267+0000 7f5fab7fe640 1 --2- 172.21.15.12:0/2511886248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55904c3b7580 0x55904c3bda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.267+0000 7f5fab7fe640 1 -- 172.21.15.12:0/2511886248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55904c3c0a80 msgr2=0x55904c3be4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:59.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.267+0000 7f5fab7fe640 1 --2- 172.21.15.12:0/2511886248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55904c3c0a80 0x55904c3be4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.267+0000 7f5fab7fe640 1 -- 172.21.15.12:0/2511886248 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55904c02e3f0 con 0x55904c3b7ea0 2024-04-22T13:32:59.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.267+0000 7f5fabfff640 1 --2- 172.21.15.12:0/2511886248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55904c3b7580 0x55904c3bda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:59.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.267+0000 7f5fab7fe640 1 --2- 172.21.15.12:0/2511886248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904c3b7ea0 0x55904c3bdfb0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000a670 tx=0x7f5fa0019f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19e8b68592e3da0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:59.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.267+0000 7f5f8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:32:59.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.267+0000 7f5f8b7fe640 1 -- 172.21.15.12:0/2511886248 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa001d050 con 0x55904c3b7ea0 2024-04-22T13:32:59.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.267+0000 7f5f8b7fe640 1 -- 172.21.15.12:0/2511886248 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f5fa0010930 con 0x55904c3b7ea0 2024-04-22T13:32:59.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.267+0000 7f5fb4fec740 1 -- 172.21.15.12:0/2511886248 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fa0018af0 con 0x55904c3b7ea0 2024-04-22T13:32:59.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.267+0000 7f5f8b7fe640 1 -- 172.21.15.12:0/2511886248 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5fa0032050 con 0x55904c3b7ea0 2024-04-22T13:32:59.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.267+0000 7f5fb4fec740 10 client.18651 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:59.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.267+0000 7f5fb4fec740 1 -- 172.21.15.12:0/2511886248 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55904bfee0f0 con 0x55904c3b7ea0 2024-04-22T13:32:59.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.267+0000 7f5f8b7fe640 1 -- 172.21.15.12:0/2511886248 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fa0018af0 con 0x55904c3b7ea0 2024-04-22T13:32:59.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.267+0000 7f5f8b7fe640 1 -- 172.21.15.12:0/2511886248 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5fa00046c0 con 0x55904c3b7ea0 2024-04-22T13:32:59.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.268+0000 7f5fb4fec740 10 client.18651 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:59.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.268+0000 7f5fb4fec740 10 client.18651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:59.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.268+0000 7f5fb4fec740 20 client.18651 populate_metadata read hostname 'smithi012' 2024-04-22T13:32:59.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.268+0000 7f5fb4fec740 1 --2- 172.21.15.12:0/2511886248 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55904c3d0960 0x55904c3d2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.268+0000 7f5fb4fec740 4 client.18651 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:59.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.268+0000 7f5fb4fec740 1 -- 172.21.15.12:0/2511886248 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55904c329c00 con 0x55904c3d0960 2024-04-22T13:32:59.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.268+0000 7f5fabfff640 1 --2- 172.21.15.12:0/2511886248 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55904c3d0960 0x55904c3d2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.268+0000 7f5f8b7fe640 10 client.18651 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:32:59.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.268+0000 7f5fabfff640 1 --2- 172.21.15.12:0/2511886248 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55904c3d0960 0x55904c3d2d50 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa386a4b32b6abe server_cookie=86542575e38049cb in_seq=0 out_seq=0 2024-04-22T13:32:59.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.269+0000 7f5f8b7fe640 1 -- 172.21.15.12:0/2511886248 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55904c329c00 con 0x55904c3d0960 2024-04-22T13:32:59.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.269+0000 7f5f8b7fe640 10 client.18651 handle_command_reply: tid=0 2024-04-22T13:32:59.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.269+0000 7f5fb4fec740 1 client.18651 shutdown 2024-04-22T13:32:59.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.269+0000 7f5fb4fec740 1 -- 172.21.15.12:0/2511886248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904c3b7ea0 msgr2=0x55904c3bdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.269+0000 7f5fb4fec740 1 --2- 172.21.15.12:0/2511886248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904c3b7ea0 0x55904c3bdfb0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000a670 tx=0x7f5fa0019f60 comp rx=0 tx=0).stop 2024-04-22T13:32:59.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.269+0000 7f5fb4fec740 1 -- 172.21.15.12:0/2511886248 shutdown_connections 2024-04-22T13:32:59.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.269+0000 7f5fb4fec740 1 --2- 172.21.15.12:0/2511886248 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55904c3d0960 0x55904c3d2d50 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.269+0000 7f5fb4fec740 1 --2- 172.21.15.12:0/2511886248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55904c3c0a80 0x55904c3be4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.269+0000 7f5fb4fec740 1 --2- 172.21.15.12:0/2511886248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904c3b7ea0 0x55904c3bdfb0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.269+0000 7f5fb4fec740 1 --2- 172.21.15.12:0/2511886248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55904c3b7580 0x55904c3bda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.269+0000 7f5f8b7fe640 0 client.18651 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:32:59.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.269+0000 7f5fb4fec740 1 -- 172.21.15.12:0/2511886248 >> 172.21.15.12:0/2511886248 conn(0x55904c3b1cb0 msgr2=0x55904c3b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:59.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.269+0000 7f5fb4fec740 1 -- 172.21.15.12:0/2511886248 shutdown_connections 2024-04-22T13:32:59.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.269+0000 7f5fb4fec740 1 -- 172.21.15.12:0/2511886248 wait complete. 2024-04-22T13:32:59.274 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.270+0000 7f5fb4fec740 20 client.18651 trim_cache size 0 max 16384 2024-04-22T13:32:59.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.287+0000 7f5fb4fec740 1 Processor -- start 2024-04-22T13:32:59.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.287+0000 7f5fb4fec740 1 -- start start 2024-04-22T13:32:59.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.287+0000 7f5fb4fec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55904c3b7580 0x55904c3c01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.287+0000 7f5fb4fec740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55904c3b7ea0 0x55904c3cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.287+0000 7f5fb4fec740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904c3c0a80 0x55904c3d8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.287+0000 7f5fb4fec740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac1afd00 con 0x55904c3c0a80 2024-04-22T13:32:59.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.287+0000 7f5fb4fec740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac1b1840 con 0x55904c3b7ea0 2024-04-22T13:32:59.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.287+0000 7f5fb4fec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac1b03d0 con 0x55904c3b7580 2024-04-22T13:32:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.288+0000 7f5fb08c0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904c3c0a80 0x55904c3d8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.288+0000 7f5fb08c0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904c3c0a80 0x55904c3d8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51310/0 (socket says 172.21.15.12:51310) 2024-04-22T13:32:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.288+0000 7f5fb08c0640 1 -- 172.21.15.12:0/3979246177 learned_addr learned my addr 172.21.15.12:0/3979246177 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.288+0000 7f5fabfff640 1 --2- 172.21.15.12:0/3979246177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55904c3b7580 0x55904c3c01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.288+0000 7f5fab7fe640 1 --2- 172.21.15.12:0/3979246177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55904c3b7ea0 0x55904c3cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.288+0000 7f5fb08c0640 1 -- 172.21.15.12:0/3979246177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55904c3b7580 msgr2=0x55904c3c01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.288+0000 7f5fb08c0640 1 --2- 172.21.15.12:0/3979246177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55904c3b7580 0x55904c3c01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.288+0000 7f5fb08c0640 1 -- 172.21.15.12:0/3979246177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55904c3b7ea0 msgr2=0x55904c3cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.288+0000 7f5fb08c0640 1 --2- 172.21.15.12:0/3979246177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55904c3b7ea0 0x55904c3cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.288+0000 7f5fb08c0640 1 -- 172.21.15.12:0/3979246177 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55904c3b1ae0 con 0x55904c3c0a80 2024-04-22T13:32:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.288+0000 7f5fab7fe640 1 --2- 172.21.15.12:0/3979246177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55904c3b7ea0 0x55904c3cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.288+0000 7f5fb08c0640 1 --2- 172.21.15.12:0/3979246177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904c3c0a80 0x55904c3d8470 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400e990 tx=0x7f5fa4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ebb3e41f306476 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.288+0000 7f5f91ffb640 1 -- 172.21.15.12:0/3979246177 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4017070 con 0x55904c3c0a80 2024-04-22T13:32:59.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.288+0000 7f5f91ffb640 1 -- 172.21.15.12:0/3979246177 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fa4011dd0 con 0x55904c3c0a80 2024-04-22T13:32:59.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.288+0000 7f5f91ffb640 1 -- 172.21.15.12:0/3979246177 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4004380 con 0x55904c3c0a80 2024-04-22T13:32:59.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.289+0000 7f5fb4fec740 1 -- 172.21.15.12:0/3979246177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904c3c0a80 msgr2=0x55904c3d8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.289+0000 7f5fb4fec740 1 --2- 172.21.15.12:0/3979246177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904c3c0a80 0x55904c3d8470 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400e990 tx=0x7f5fa4013000 comp rx=0 tx=0).stop 2024-04-22T13:32:59.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.289+0000 7f5fb4fec740 1 -- 172.21.15.12:0/3979246177 shutdown_connections 2024-04-22T13:32:59.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.289+0000 7f5fb4fec740 1 --2- 172.21.15.12:0/3979246177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904c3c0a80 0x55904c3d8470 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.289+0000 7f5fb4fec740 1 --2- 172.21.15.12:0/3979246177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55904c3b7ea0 0x55904c3cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.289+0000 7f5fb4fec740 1 --2- 172.21.15.12:0/3979246177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55904c3b7580 0x55904c3c01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.289+0000 7f5fb4fec740 1 -- 172.21.15.12:0/3979246177 >> 172.21.15.12:0/3979246177 conn(0x55904c3b1cb0 msgr2=0x55904c3b8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:59.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.289+0000 7f5fb4fec740 1 -- 172.21.15.12:0/3979246177 shutdown_connections 2024-04-22T13:32:59.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.289+0000 7f5fb4fec740 1 -- 172.21.15.12:0/3979246177 wait complete. 2024-04-22T13:32:59.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.289+0000 7f5fb4fec740 1 -- start start 2024-04-22T13:32:59.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.289+0000 7f5fb4fec740 1 Processor -- start 2024-04-22T13:32:59.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.290+0000 7f5fb4fec740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904c3b7580 0x55904c3c4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.290 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.290+0000 7f5fb4fec740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55904c3b7ea0 0x55904c3c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.290+0000 7f5fb4fec740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55904c3c0a80 0x55904c3c5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.290+0000 7f5fb4fec740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac1b1170 con 0x55904c3b7580 2024-04-22T13:32:59.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.290+0000 7f5fb4fec740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac1b0aa0 con 0x55904c3b7ea0 2024-04-22T13:32:59.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.290+0000 7f5fb4fec740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac1add00 con 0x55904c3c0a80 2024-04-22T13:32:59.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.290+0000 7f5fb4fec740 10 client.0 _ll_register_callbacks cb 0x7f5fa001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:59.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.290+0000 7f5fabfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904c3b7580 0x55904c3c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.290+0000 7f5fabfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904c3b7580 0x55904c3c4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51312/0 (socket says 172.21.15.12:51312) 2024-04-22T13:32:59.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.290+0000 7f5fabfff640 1 -- 172.21.15.12:0/3675884662 learned_addr learned my addr 172.21.15.12:0/3675884662 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:59.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.290+0000 7f5fb08c0640 1 --2- 172.21.15.12:0/3675884662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55904c3c0a80 0x55904c3c5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.290+0000 7f5fab7fe640 1 --2- 172.21.15.12:0/3675884662 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55904c3b7ea0 0x55904c3c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.290+0000 7f5fabfff640 1 -- 172.21.15.12:0/3675884662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55904c3c0a80 msgr2=0x55904c3c5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:59.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.290+0000 7f5fabfff640 1 --2- 172.21.15.12:0/3675884662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55904c3c0a80 0x55904c3c5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.290+0000 7f5fabfff640 1 -- 172.21.15.12:0/3675884662 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55904c3b7ea0 msgr2=0x55904c3c4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:59.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.290+0000 7f5fabfff640 1 --2- 172.21.15.12:0/3675884662 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55904c3b7ea0 0x55904c3c4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.290+0000 7f5fabfff640 1 -- 172.21.15.12:0/3675884662 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55904c435ce0 con 0x55904c3b7580 2024-04-22T13:32:59.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.290+0000 7f5fb08c0640 1 --2- 172.21.15.12:0/3675884662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55904c3c0a80 0x55904c3c5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:59.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.290+0000 7f5fabfff640 1 --2- 172.21.15.12:0/3675884662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904c3b7580 0x55904c3c4680 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c0baf00 tx=0x7f5f9c0df040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee323eed10bd3df9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:59.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.290+0000 7f5f8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:32:59.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.290+0000 7f5f8b7fe640 1 -- 172.21.15.12:0/3675884662 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c0044e0 con 0x55904c3b7580 2024-04-22T13:32:59.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.291+0000 7f5f8b7fe640 1 -- 172.21.15.12:0/3675884662 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f5f9c005460 con 0x55904c3b7580 2024-04-22T13:32:59.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.291+0000 7f5fb4fec740 1 -- 172.21.15.12:0/3675884662 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fa4011dd0 con 0x55904c3b7580 2024-04-22T13:32:59.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.291+0000 7f5f8b7fe640 1 -- 172.21.15.12:0/3675884662 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5f9c0bf1a0 con 0x55904c3b7580 2024-04-22T13:32:59.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.291+0000 7f5fb4fec740 10 client.18657 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:59.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.291+0000 7f5fb4fec740 1 -- 172.21.15.12:0/3675884662 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55904c436f00 con 0x55904c3b7580 2024-04-22T13:32:59.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.291+0000 7f5f8b7fe640 1 -- 172.21.15.12:0/3675884662 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fa4011dd0 con 0x55904c3b7580 2024-04-22T13:32:59.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.291+0000 7f5f8b7fe640 1 -- 172.21.15.12:0/3675884662 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5f9c0f03c0 con 0x55904c3b7580 2024-04-22T13:32:59.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.291+0000 7f5fb4fec740 10 client.18657 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:59.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.291+0000 7f5fb4fec740 10 client.18657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:32:59.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.291+0000 7f5fb4fec740 20 client.18657 populate_metadata read hostname 'smithi012' 2024-04-22T13:32:59.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.291+0000 7f5fb4fec740 1 --2- 172.21.15.12:0/3675884662 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55904c4375b0 0x55904c4399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.291+0000 7f5fb4fec740 4 client.18657 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:32:59.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.291+0000 7f5fb4fec740 1 -- 172.21.15.12:0/3675884662 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55904c323a70 con 0x55904c4375b0 2024-04-22T13:32:59.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.291+0000 7f5fab7fe640 1 --2- 172.21.15.12:0/3675884662 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55904c4375b0 0x55904c4399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.292+0000 7f5f8b7fe640 10 client.18657 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:32:59.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.292+0000 7f5fab7fe640 1 --2- 172.21.15.12:0/3675884662 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55904c4375b0 0x55904c4399d0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ad2ebb8cf9942a6 server_cookie=b53e672052d237ff in_seq=0 out_seq=0 2024-04-22T13:32:59.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.292+0000 7f5f8b7fe640 1 -- 172.21.15.12:0/3675884662 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55904c323a70 con 0x55904c4375b0 2024-04-22T13:32:59.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.292+0000 7f5f8b7fe640 10 client.18657 handle_command_reply: tid=0 2024-04-22T13:32:59.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.292+0000 7f5fb4fec740 1 client.18657 shutdown 2024-04-22T13:32:59.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.292+0000 7f5fb4fec740 1 -- 172.21.15.12:0/3675884662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904c3b7580 msgr2=0x55904c3c4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.292+0000 7f5fb4fec740 1 --2- 172.21.15.12:0/3675884662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904c3b7580 0x55904c3c4680 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c0baf00 tx=0x7f5f9c0df040 comp rx=0 tx=0).stop 2024-04-22T13:32:59.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.292+0000 7f5fb4fec740 1 -- 172.21.15.12:0/3675884662 shutdown_connections 2024-04-22T13:32:59.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.292+0000 7f5fb4fec740 1 --2- 172.21.15.12:0/3675884662 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55904c4375b0 0x55904c4399d0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.292+0000 7f5fb4fec740 1 --2- 172.21.15.12:0/3675884662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55904c3c0a80 0x55904c3c5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.292+0000 7f5fb4fec740 1 --2- 172.21.15.12:0/3675884662 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55904c3b7ea0 0x55904c3c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.292+0000 7f5fb4fec740 1 --2- 172.21.15.12:0/3675884662 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55904c3b7580 0x55904c3c4680 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.292+0000 7f5f8b7fe640 0 client.18657 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:32:59.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.292+0000 7f5fb4fec740 1 -- 172.21.15.12:0/3675884662 >> 172.21.15.12:0/3675884662 conn(0x55904c3b1cb0 msgr2=0x55904c3bda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:59.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.292+0000 7f5fb4fec740 1 -- 172.21.15.12:0/3675884662 shutdown_connections 2024-04-22T13:32:59.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.292+0000 7f5fb4fec740 1 -- 172.21.15.12:0/3675884662 wait complete. 2024-04-22T13:32:59.294 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.292+0000 7f5fb4fec740 20 client.18657 trim_cache size 0 max 16384 2024-04-22T13:32:59.294 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:32:59.295 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-04-22T13:32:59.295 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-04-22T13:32:59.295 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:32:59.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.293+0000 7f5fb2323640 1 -- 172.21.15.12:0/2719854106 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5f940a3d20 msgr2=0x7f5f940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.293+0000 7f5fb2323640 1 --2- 172.21.15.12:0/2719854106 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5f940a3d20 0x7f5f940a61e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4005c40 tx=0x7f5fa400a4f0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.293+0000 7f5fb2323640 1 -- 172.21.15.12:0/2719854106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fac100d40 msgr2=0x7f5fac1a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.293+0000 7f5fb2323640 1 --2- 172.21.15.12:0/2719854106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fac100d40 0x7f5fac1a0610 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c005940 tx=0x7f5f9c0040e0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.293+0000 7f5fb2323640 1 -- 172.21.15.12:0/2719854106 shutdown_connections 2024-04-22T13:32:59.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.293+0000 7f5fb2323640 1 --2- 172.21.15.12:0/2719854106 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5f940a3d20 0x7f5f940a61e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.293+0000 7f5fb2323640 1 --2- 172.21.15.12:0/2719854106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5fac10e8f0 0x7f5fac19a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.293+0000 7f5fb2323640 1 --2- 172.21.15.12:0/2719854106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fac101660 0x7f5fac1a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.293+0000 7f5fb2323640 1 --2- 172.21.15.12:0/2719854106 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fac100d40 0x7f5fac1a0610 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.293+0000 7f5fb2323640 1 -- 172.21.15.12:0/2719854106 >> 172.21.15.12:0/2719854106 conn(0x7f5fac078000 msgr2=0x7f5fac10ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:59.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.293+0000 7f5fb2323640 1 -- 172.21.15.12:0/2719854106 shutdown_connections 2024-04-22T13:32:59.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.293+0000 7f5fb2323640 1 -- 172.21.15.12:0/2719854106 wait complete. 2024-04-22T13:32:59.299 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:32:59.299 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:32:59.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.406+0000 7f6d6052e640 1 -- 172.21.15.12:0/2667363033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d58103af0 msgr2=0x7f6d5810bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.406+0000 7f6d6052e640 1 --2- 172.21.15.12:0/2667363033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d58103af0 0x7f6d5810bfd0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f6d48009900 tx=0x7f6d4802bcd0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.406+0000 7f6d6052e640 1 -- 172.21.15.12:0/2667363033 shutdown_connections 2024-04-22T13:32:59.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.406+0000 7f6d6052e640 1 --2- 172.21.15.12:0/2667363033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d5810c5a0 0x7f6d5810e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.406+0000 7f6d6052e640 1 --2- 172.21.15.12:0/2667363033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d58103af0 0x7f6d5810bfd0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.406+0000 7f6d6052e640 1 --2- 172.21.15.12:0/2667363033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d581031d0 0x7f6d581035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.406+0000 7f6d6052e640 1 -- 172.21.15.12:0/2667363033 >> 172.21.15.12:0/2667363033 conn(0x7f6d580771d0 msgr2=0x7f6d580775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:59.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.406+0000 7f6d6052e640 1 -- 172.21.15.12:0/2667363033 shutdown_connections 2024-04-22T13:32:59.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.406+0000 7f6d6052e640 1 -- 172.21.15.12:0/2667363033 wait complete. 2024-04-22T13:32:59.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.407+0000 7f6d6052e640 1 Processor -- start 2024-04-22T13:32:59.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.407+0000 7f6d6052e640 1 -- start start 2024-04-22T13:32:59.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.407+0000 7f6d6052e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d581031d0 0x7f6d58100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.407+0000 7f6d6052e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d58103af0 0x7f6d58101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.407+0000 7f6d6052e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d5810c5a0 0x7f6d581018a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.407+0000 7f6d6052e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6d581121e0 con 0x7f6d5810c5a0 2024-04-22T13:32:59.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.407+0000 7f6d6052e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6d58112060 con 0x7f6d581031d0 2024-04-22T13:32:59.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.407+0000 7f6d6052e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6d58112360 con 0x7f6d58103af0 2024-04-22T13:32:59.405 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.407+0000 7f6d5eacb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d5810c5a0 0x7f6d581018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.407+0000 7f6d5eacb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d5810c5a0 0x7f6d581018a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51328/0 (socket says 172.21.15.12:51328) 2024-04-22T13:32:59.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.407+0000 7f6d5eacb640 1 -- 172.21.15.12:0/432117285 learned_addr learned my addr 172.21.15.12:0/432117285 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:59.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.408+0000 7f6d5e2ca640 1 --2- 172.21.15.12:0/432117285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d581031d0 0x7f6d58100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.408+0000 7f6d5dac9640 1 --2- 172.21.15.12:0/432117285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d58103af0 0x7f6d58101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.408+0000 7f6d5eacb640 1 -- 172.21.15.12:0/432117285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d58103af0 msgr2=0x7f6d58101360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.408+0000 7f6d5eacb640 1 --2- 172.21.15.12:0/432117285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d58103af0 0x7f6d58101360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.408+0000 7f6d5eacb640 1 -- 172.21.15.12:0/432117285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d581031d0 msgr2=0x7f6d58100e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.408+0000 7f6d5eacb640 1 --2- 172.21.15.12:0/432117285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d581031d0 0x7f6d58100e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.408+0000 7f6d5eacb640 1 -- 172.21.15.12:0/432117285 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d581a6be0 con 0x7f6d5810c5a0 2024-04-22T13:32:59.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.408+0000 7f6d5e2ca640 1 --2- 172.21.15.12:0/432117285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d581031d0 0x7f6d58100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:59.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.408+0000 7f6d5eacb640 1 --2- 172.21.15.12:0/432117285 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d5810c5a0 0x7f6d581018a0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000ccb0 tx=0x7f6d5000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:59.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.408+0000 7f6d4f7fe640 1 -- 172.21.15.12:0/432117285 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5000f450 con 0x7f6d5810c5a0 2024-04-22T13:32:59.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.408+0000 7f6d6052e640 1 -- 172.21.15.12:0/432117285 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d581a6ed0 con 0x7f6d5810c5a0 2024-04-22T13:32:59.406 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.408+0000 7f6d4f7fe640 1 -- 172.21.15.12:0/432117285 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d5000faa0 con 0x7f6d5810c5a0 2024-04-22T13:32:59.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.408+0000 7f6d4f7fe640 1 -- 172.21.15.12:0/432117285 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50018b00 con 0x7f6d5810c5a0 2024-04-22T13:32:59.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.408+0000 7f6d6052e640 1 -- 172.21.15.12:0/432117285 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d581a73a0 con 0x7f6d5810c5a0 2024-04-22T13:32:59.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.409+0000 7f6d6052e640 1 -- 172.21.15.12:0/432117285 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d58069230 con 0x7f6d5810c5a0 2024-04-22T13:32:59.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.410+0000 7f6d4f7fe640 1 -- 172.21.15.12:0/432117285 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6d5000fc50 con 0x7f6d5810c5a0 2024-04-22T13:32:59.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.411+0000 7f6d4f7fe640 1 --2- 172.21.15.12:0/432117285 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6d440a3d20 0x7f6d440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.411+0000 7f6d5e2ca640 1 --2- 172.21.15.12:0/432117285 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6d440a3d20 0x7f6d440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.411+0000 7f6d4f7fe640 1 -- 172.21.15.12:0/432117285 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f6d500d0f60 con 0x7f6d5810c5a0 2024-04-22T13:32:59.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.411+0000 7f6d5e2ca640 1 --2- 172.21.15.12:0/432117285 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6d440a3d20 0x7f6d440a61e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6d540058c0 tx=0x7f6d54005850 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:59.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.415+0000 7f6d4f7fe640 1 -- 172.21.15.12:0/432117285 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6d50014030 con 0x7f6d5810c5a0 2024-04-22T13:32:59.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.645+0000 7f6d6052e640 1 -- 172.21.15.12:0/432117285 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d58107690 con 0x7f6d5810c5a0 2024-04-22T13:32:59.640 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.645+0000 7f6d4f7fe640 1 -- 172.21.15.12:0/432117285 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7f6d5009d620 con 0x7f6d5810c5a0 2024-04-22T13:32:59.641 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:32:59.641 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:32:59.641 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:32:59.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.647+0000 7f6d6052e640 1 -- 172.21.15.12:0/432117285 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6d440a3d20 msgr2=0x7f6d440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.647+0000 7f6d6052e640 1 --2- 172.21.15.12:0/432117285 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6d440a3d20 0x7f6d440a61e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6d540058c0 tx=0x7f6d54005850 comp rx=0 tx=0).stop 2024-04-22T13:32:59.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.647+0000 7f6d6052e640 1 -- 172.21.15.12:0/432117285 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d5810c5a0 msgr2=0x7f6d581018a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.647+0000 7f6d6052e640 1 --2- 172.21.15.12:0/432117285 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d5810c5a0 0x7f6d581018a0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000ccb0 tx=0x7f6d5000a620 comp rx=0 tx=0).stop 2024-04-22T13:32:59.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.647+0000 7f6d6052e640 1 -- 172.21.15.12:0/432117285 shutdown_connections 2024-04-22T13:32:59.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.647+0000 7f6d6052e640 1 --2- 172.21.15.12:0/432117285 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f6d440a3d20 0x7f6d440a61e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.647+0000 7f6d6052e640 1 --2- 172.21.15.12:0/432117285 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6d5810c5a0 0x7f6d581018a0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.647+0000 7f6d6052e640 1 --2- 172.21.15.12:0/432117285 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d58103af0 0x7f6d58101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.647+0000 7f6d6052e640 1 --2- 172.21.15.12:0/432117285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6d581031d0 0x7f6d58100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.647+0000 7f6d6052e640 1 -- 172.21.15.12:0/432117285 >> 172.21.15.12:0/432117285 conn(0x7f6d580771d0 msgr2=0x7f6d580fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:59.643 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.647+0000 7f6d6052e640 1 -- 172.21.15.12:0/432117285 shutdown_connections 2024-04-22T13:32:59.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.647+0000 7f6d6052e640 1 -- 172.21.15.12:0/432117285 wait complete. 2024-04-22T13:32:59.654 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:32:59.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.763+0000 7f899e00f640 1 -- 172.21.15.12:0/4004454803 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f899806b1f0 msgr2=0x7f8998106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.763+0000 7f899e00f640 1 --2- 172.21.15.12:0/4004454803 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f899806b1f0 0x7f8998106b90 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f8988009900 tx=0x7f898802bc30 comp rx=0 tx=0).stop 2024-04-22T13:32:59.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.763+0000 7f899e00f640 1 -- 172.21.15.12:0/4004454803 shutdown_connections 2024-04-22T13:32:59.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.763+0000 7f899e00f640 1 --2- 172.21.15.12:0/4004454803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8998107270 0x7f899810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.763+0000 7f899e00f640 1 --2- 172.21.15.12:0/4004454803 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f899806b1f0 0x7f8998106b90 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.763+0000 7f899e00f640 1 --2- 172.21.15.12:0/4004454803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f899806a840 0x7f899806ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.763+0000 7f899e00f640 1 -- 172.21.15.12:0/4004454803 >> 172.21.15.12:0/4004454803 conn(0x7f8998077eb0 msgr2=0x7f89980782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:59.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.763+0000 7f899e00f640 1 -- 172.21.15.12:0/4004454803 shutdown_connections 2024-04-22T13:32:59.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.763+0000 7f899e00f640 1 -- 172.21.15.12:0/4004454803 wait complete. 2024-04-22T13:32:59.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.764+0000 7f899e00f640 1 Processor -- start 2024-04-22T13:32:59.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.764+0000 7f899e00f640 1 -- start start 2024-04-22T13:32:59.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.764+0000 7f899e00f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f899806a840 0x7f8998100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.764+0000 7f899e00f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8998107270 0x7f89981013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.764+0000 7f899e00f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8998101900 0x7f89981a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.764+0000 7f899e00f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f89981121b0 con 0x7f8998101900 2024-04-22T13:32:59.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.764+0000 7f899e00f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8998112030 con 0x7f8998107270 2024-04-22T13:32:59.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.764+0000 7f899e00f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8998112330 con 0x7f899806a840 2024-04-22T13:32:59.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.765+0000 7f8997fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8998101900 0x7f89981a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.765+0000 7f8997fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8998101900 0x7f89981a68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51342/0 (socket says 172.21.15.12:51342) 2024-04-22T13:32:59.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.765+0000 7f8997fff640 1 -- 172.21.15.12:0/2892716437 learned_addr learned my addr 172.21.15.12:0/2892716437 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:59.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.765+0000 7f89977fe640 1 --2- 172.21.15.12:0/2892716437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f899806a840 0x7f8998100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.765+0000 7f8996ffd640 1 --2- 172.21.15.12:0/2892716437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8998107270 0x7f89981013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.765+0000 7f8997fff640 1 -- 172.21.15.12:0/2892716437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f899806a840 msgr2=0x7f8998100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.765+0000 7f8997fff640 1 --2- 172.21.15.12:0/2892716437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f899806a840 0x7f8998100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.765+0000 7f8997fff640 1 -- 172.21.15.12:0/2892716437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8998107270 msgr2=0x7f89981013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.765+0000 7f8997fff640 1 --2- 172.21.15.12:0/2892716437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8998107270 0x7f89981013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.765+0000 7f8997fff640 1 -- 172.21.15.12:0/2892716437 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89981a6fe0 con 0x7f8998101900 2024-04-22T13:32:59.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.765+0000 7f8996ffd640 1 --2- 172.21.15.12:0/2892716437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8998107270 0x7f89981013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:59.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.765+0000 7f89977fe640 1 --2- 172.21.15.12:0/2892716437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f899806a840 0x7f8998100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:59.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.765+0000 7f8997fff640 1 --2- 172.21.15.12:0/2892716437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8998101900 0x7f89981a68e0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f899000eaa0 tx=0x7f899000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:59.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.765+0000 7f8994ff9640 1 -- 172.21.15.12:0/2892716437 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899000cdb0 con 0x7f8998101900 2024-04-22T13:32:59.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.765+0000 7f8994ff9640 1 -- 172.21.15.12:0/2892716437 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8990005020 con 0x7f8998101900 2024-04-22T13:32:59.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.765+0000 7f899e00f640 1 -- 172.21.15.12:0/2892716437 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89981a72d0 con 0x7f8998101900 2024-04-22T13:32:59.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.765+0000 7f8994ff9640 1 -- 172.21.15.12:0/2892716437 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89900116d0 con 0x7f8998101900 2024-04-22T13:32:59.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.765+0000 7f899e00f640 1 -- 172.21.15.12:0/2892716437 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89981a77d0 con 0x7f8998101900 2024-04-22T13:32:59.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.768+0000 7f8994ff9640 1 -- 172.21.15.12:0/2892716437 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f89900051d0 con 0x7f8998101900 2024-04-22T13:32:59.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.768+0000 7f89a0bfd740 1 Processor -- start 2024-04-22T13:32:59.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.768+0000 7f89a0bfd740 1 -- start start 2024-04-22T13:32:59.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.768+0000 7f89a0bfd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583cc5e34d0 0x5583cc5e38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.768+0000 7f89a0bfd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583cc5e3e80 0x5583cc5ec430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.768+0000 7f89a0bfd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583cc5eca00 0x5583cc5eee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.768+0000 7f89a0bfd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5583cc5481f0 con 0x5583cc5eca00 2024-04-22T13:32:59.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.768+0000 7f8994ff9640 1 --2- 172.21.15.12:0/2892716437 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f89800a3d50 0x7f89800a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.768+0000 7f89a0bfd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5583cc4e54b0 con 0x5583cc5e3e80 2024-04-22T13:32:59.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.768+0000 7f89a0bfd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5583cc5441b0 con 0x5583cc5e34d0 2024-04-22T13:32:59.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.768+0000 7f8996ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583cc5eca00 0x5583cc5eee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.768+0000 7f8997fff640 1 --2- 172.21.15.12:0/2892716437 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f89800a3d50 0x7f89800a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.772 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.768+0000 7f8996ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583cc5eca00 0x5583cc5eee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51344/0 (socket says 172.21.15.12:51344) 2024-04-22T13:32:59.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.768+0000 7f8996ffd640 1 -- 172.21.15.12:0/3220357016 learned_addr learned my addr 172.21.15.12:0/3220357016 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:59.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.768+0000 7f89977fe640 1 --2- 172.21.15.12:0/3220357016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583cc5e3e80 0x5583cc5ec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.768+0000 7f8994ff9640 1 -- 172.21.15.12:0/2892716437 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f89900d09e0 con 0x7f8998101900 2024-04-22T13:32:59.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.769+0000 7f8996ffd640 1 --2- 172.21.15.12:0/3220357016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583cc5e34d0 0x5583cc5e38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.769+0000 7f8996ffd640 1 -- 172.21.15.12:0/3220357016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583cc5e34d0 msgr2=0x5583cc5e38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:59.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.769+0000 7f8996ffd640 1 --2- 172.21.15.12:0/3220357016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583cc5e34d0 0x5583cc5e38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.769+0000 7f8996ffd640 1 -- 172.21.15.12:0/3220357016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583cc5e3e80 msgr2=0x5583cc5ec430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:59.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.769+0000 7f8996ffd640 1 --2- 172.21.15.12:0/3220357016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583cc5e3e80 0x5583cc5ec430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.769+0000 7f8996ffd640 1 -- 172.21.15.12:0/3220357016 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583cc4d71b0 con 0x5583cc5eca00 2024-04-22T13:32:59.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.769+0000 7f8997fff640 1 --2- 172.21.15.12:0/2892716437 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f89800a3d50 0x7f89800a6210 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f899000efd0 tx=0x7f8990087040 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:59.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.769+0000 7f8996ffd640 1 --2- 172.21.15.12:0/3220357016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583cc5eca00 0x5583cc5eee50 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f8988009900 tx=0x7f8988000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d295f9c51e2d6a71 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:59.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.769+0000 7f897dffb640 1 -- 172.21.15.12:0/3220357016 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8988047070 con 0x5583cc5eca00 2024-04-22T13:32:59.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.769+0000 7f897dffb640 1 -- 172.21.15.12:0/3220357016 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f898802cc90 con 0x5583cc5eca00 2024-04-22T13:32:59.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.769+0000 7f897dffb640 1 -- 172.21.15.12:0/3220357016 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89880046c0 con 0x5583cc5eca00 2024-04-22T13:32:59.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.770+0000 7f89a0bfd740 1 -- 172.21.15.12:0/3220357016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583cc5eca00 msgr2=0x5583cc5eee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.770+0000 7f89a0bfd740 1 --2- 172.21.15.12:0/3220357016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583cc5eca00 0x5583cc5eee50 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f8988009900 tx=0x7f8988000bc0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.770+0000 7f89a0bfd740 1 -- 172.21.15.12:0/3220357016 shutdown_connections 2024-04-22T13:32:59.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.770+0000 7f89a0bfd740 1 --2- 172.21.15.12:0/3220357016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583cc5eca00 0x5583cc5eee50 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.770+0000 7f89a0bfd740 1 --2- 172.21.15.12:0/3220357016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583cc5e3e80 0x5583cc5ec430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.770+0000 7f89a0bfd740 1 --2- 172.21.15.12:0/3220357016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583cc5e34d0 0x5583cc5e38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.770+0000 7f89a0bfd740 1 -- 172.21.15.12:0/3220357016 >> 172.21.15.12:0/3220357016 conn(0x5583cc5ddba0 msgr2=0x5583cc5ddf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:59.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.770+0000 7f89a0bfd740 1 -- 172.21.15.12:0/3220357016 shutdown_connections 2024-04-22T13:32:59.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.770+0000 7f89a0bfd740 1 -- 172.21.15.12:0/3220357016 wait complete. 2024-04-22T13:32:59.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.770+0000 7f89a0bfd740 1 -- start start 2024-04-22T13:32:59.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.770+0000 7f89a0bfd740 1 Processor -- start 2024-04-22T13:32:59.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.771+0000 7f89a0bfd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583cc5e34d0 0x5583cc5ec340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.771+0000 7f89a0bfd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583cc5e3e80 0x5583cc5e96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.771+0000 7f89a0bfd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583cc5eca00 0x5583cc5e9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.771+0000 7f89a0bfd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5583cc5ef910 con 0x5583cc5e3e80 2024-04-22T13:32:59.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.771+0000 7f89a0bfd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5583cc501500 con 0x5583cc5eca00 2024-04-22T13:32:59.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.771+0000 7f89a0bfd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5583cc5efa90 con 0x5583cc5e34d0 2024-04-22T13:32:59.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.771+0000 7f89a0bfd740 10 client.0 _ll_register_callbacks cb 0x5583cc4cfb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:59.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.771+0000 7f89977fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583cc5e3e80 0x5583cc5e96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.771+0000 7f89977fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583cc5e3e80 0x5583cc5e96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51354/0 (socket says 172.21.15.12:51354) 2024-04-22T13:32:59.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.771+0000 7f89977fe640 1 -- 172.21.15.12:0/791301593 learned_addr learned my addr 172.21.15.12:0/791301593 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:59.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.771+0000 7f8996ffd640 1 --2- 172.21.15.12:0/791301593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583cc5eca00 0x5583cc5e9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.771+0000 7f89977fe640 1 -- 172.21.15.12:0/791301593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583cc5e34d0 msgr2=0x5583cc5ec340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:59.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.771+0000 7f89977fe640 1 --2- 172.21.15.12:0/791301593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583cc5e34d0 0x5583cc5ec340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.771+0000 7f89977fe640 1 -- 172.21.15.12:0/791301593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583cc5eca00 msgr2=0x5583cc5e9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:59.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.771+0000 7f89977fe640 1 --2- 172.21.15.12:0/791301593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583cc5eca00 0x5583cc5e9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.771+0000 7f89977fe640 1 -- 172.21.15.12:0/791301593 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583cc25a3f0 con 0x5583cc5e3e80 2024-04-22T13:32:59.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.771+0000 7f8996ffd640 1 --2- 172.21.15.12:0/791301593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583cc5eca00 0x5583cc5e9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:59.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.771+0000 7f89977fe640 1 --2- 172.21.15.12:0/791301593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583cc5e3e80 0x5583cc5e96d0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f898c009f60 tx=0x7f898c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e58b1f548b5cae6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:59.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.771+0000 7f8976ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:32:59.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.771+0000 7f8976ffd640 1 -- 172.21.15.12:0/791301593 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c012920 con 0x5583cc5e3e80 2024-04-22T13:32:59.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.771+0000 7f8976ffd640 1 -- 172.21.15.12:0/791301593 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f898c012ac0 con 0x5583cc5e3e80 2024-04-22T13:32:59.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.771+0000 7f89a0bfd740 1 -- 172.21.15.12:0/791301593 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f898802cc90 con 0x5583cc5e3e80 2024-04-22T13:32:59.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.771+0000 7f8976ffd640 1 -- 172.21.15.12:0/791301593 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f898c012cc0 con 0x5583cc5e3e80 2024-04-22T13:32:59.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.771+0000 7f89a0bfd740 10 client.18675 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:59.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.771+0000 7f89a0bfd740 1 -- 172.21.15.12:0/791301593 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5583cc3247d0 con 0x5583cc5e3e80 2024-04-22T13:32:59.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.772+0000 7f8976ffd640 1 -- 172.21.15.12:0/791301593 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f898802cc90 con 0x5583cc5e3e80 2024-04-22T13:32:59.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.772+0000 7f8976ffd640 1 -- 172.21.15.12:0/791301593 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f898c021020 con 0x5583cc5e3e80 2024-04-22T13:32:59.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.772+0000 7f89a0bfd740 10 client.18675 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:59.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.772+0000 7f89a0bfd740 10 client.18675 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:32:59.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.772+0000 7f89a0bfd740 20 client.18675 populate_metadata read hostname 'smithi012' 2024-04-22T13:32:59.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.772+0000 7f89a0bfd740 1 --2- 172.21.15.12:0/791301593 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5583cc5fccd0 0x5583cc5ff130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.772+0000 7f89a0bfd740 4 client.18675 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:59.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.772+0000 7f89a0bfd740 1 -- 172.21.15.12:0/791301593 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583cc4e7930 con 0x5583cc5fccd0 2024-04-22T13:32:59.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.772+0000 7f8996ffd640 1 --2- 172.21.15.12:0/791301593 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5583cc5fccd0 0x5583cc5ff130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.772+0000 7f8976ffd640 10 client.18675 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:32:59.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.772+0000 7f8996ffd640 1 --2- 172.21.15.12:0/791301593 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5583cc5fccd0 0x5583cc5ff130 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=33791e5adb8c0f67 server_cookie=53346b8a6030517b in_seq=0 out_seq=0 2024-04-22T13:32:59.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.773+0000 7f8976ffd640 1 -- 172.21.15.12:0/791301593 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5583cc4e7930 con 0x5583cc5fccd0 2024-04-22T13:32:59.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.773+0000 7f8976ffd640 10 client.18675 handle_command_reply: tid=0 2024-04-22T13:32:59.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.773+0000 7f89a0bfd740 1 client.18675 shutdown 2024-04-22T13:32:59.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.773+0000 7f89a0bfd740 1 -- 172.21.15.12:0/791301593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583cc5e3e80 msgr2=0x5583cc5e96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.773+0000 7f89a0bfd740 1 --2- 172.21.15.12:0/791301593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583cc5e3e80 0x5583cc5e96d0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f898c009f60 tx=0x7f898c009f90 comp rx=0 tx=0).stop 2024-04-22T13:32:59.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.773+0000 7f89a0bfd740 1 -- 172.21.15.12:0/791301593 shutdown_connections 2024-04-22T13:32:59.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.773+0000 7f89a0bfd740 1 --2- 172.21.15.12:0/791301593 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5583cc5fccd0 0x5583cc5ff130 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.773+0000 7f89a0bfd740 1 --2- 172.21.15.12:0/791301593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583cc5eca00 0x5583cc5e9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.773+0000 7f89a0bfd740 1 --2- 172.21.15.12:0/791301593 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583cc5e3e80 0x5583cc5e96d0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.773+0000 7f8976ffd640 0 client.18675 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:32:59.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.773+0000 7f89a0bfd740 1 --2- 172.21.15.12:0/791301593 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583cc5e34d0 0x5583cc5ec340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.773+0000 7f89a0bfd740 1 -- 172.21.15.12:0/791301593 >> 172.21.15.12:0/791301593 conn(0x5583cc5ddba0 msgr2=0x5583cc5df8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:59.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.774+0000 7f89a0bfd740 1 -- 172.21.15.12:0/791301593 shutdown_connections 2024-04-22T13:32:59.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.774+0000 7f89a0bfd740 1 -- 172.21.15.12:0/791301593 wait complete. 2024-04-22T13:32:59.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.774+0000 7f89a0bfd740 20 client.18675 trim_cache size 0 max 16384 2024-04-22T13:32:59.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.787+0000 7f89a0bfd740 1 Processor -- start 2024-04-22T13:32:59.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.787+0000 7f89a0bfd740 1 -- start start 2024-04-22T13:32:59.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.787+0000 7f89a0bfd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583cc5e34d0 0x5583cc5f87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.787+0000 7f89a0bfd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583cc5e3e80 0x5583cc5df600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.787+0000 7f89a0bfd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583cc5eca00 0x5583cc5ebe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.787+0000 7f89a0bfd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f89981afd40 con 0x5583cc5e34d0 2024-04-22T13:32:59.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.787+0000 7f89a0bfd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f89981b1880 con 0x5583cc5eca00 2024-04-22T13:32:59.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.787+0000 7f89a0bfd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f89981b0410 con 0x5583cc5e3e80 2024-04-22T13:32:59.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.787+0000 7f8996ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583cc5e34d0 0x5583cc5f87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.787+0000 7f8996ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583cc5e34d0 0x5583cc5f87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51368/0 (socket says 172.21.15.12:51368) 2024-04-22T13:32:59.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.787+0000 7f8996ffd640 1 -- 172.21.15.12:0/808748199 learned_addr learned my addr 172.21.15.12:0/808748199 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:59.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.787+0000 7f8996ffd640 1 --2- 172.21.15.12:0/808748199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583cc5eca00 0x5583cc5ebe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.787+0000 7f8996ffd640 1 -- 172.21.15.12:0/808748199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583cc5e3e80 msgr2=0x5583cc5df600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:59.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.787+0000 7f89977fe640 1 --2- 172.21.15.12:0/808748199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583cc5e3e80 0x5583cc5df600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.788+0000 7f8996ffd640 1 --2- 172.21.15.12:0/808748199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583cc5e3e80 0x5583cc5df600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.788+0000 7f8996ffd640 1 -- 172.21.15.12:0/808748199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583cc5eca00 msgr2=0x5583cc5ebe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:59.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.788+0000 7f8996ffd640 1 --2- 172.21.15.12:0/808748199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583cc5eca00 0x5583cc5ebe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.788+0000 7f8996ffd640 1 -- 172.21.15.12:0/808748199 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583cc5e8680 con 0x5583cc5e34d0 2024-04-22T13:32:59.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.788+0000 7f89977fe640 1 --2- 172.21.15.12:0/808748199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583cc5e3e80 0x5583cc5df600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:59.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.788+0000 7f8996ffd640 1 --2- 172.21.15.12:0/808748199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583cc5e34d0 0x5583cc5f87d0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f8988045040 tx=0x7f89880052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be750290ee3ddc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:59.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.788+0000 7f897d7fa640 1 -- 172.21.15.12:0/808748199 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8988047040 con 0x5583cc5e34d0 2024-04-22T13:32:59.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.788+0000 7f897d7fa640 1 -- 172.21.15.12:0/808748199 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f898802cc90 con 0x5583cc5e34d0 2024-04-22T13:32:59.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.788+0000 7f897d7fa640 1 -- 172.21.15.12:0/808748199 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898804f860 con 0x5583cc5e34d0 2024-04-22T13:32:59.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.788+0000 7f89a0bfd740 1 -- 172.21.15.12:0/808748199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583cc5e34d0 msgr2=0x5583cc5f87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.788+0000 7f89a0bfd740 1 --2- 172.21.15.12:0/808748199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583cc5e34d0 0x5583cc5f87d0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f8988045040 tx=0x7f89880052b0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.788+0000 7f89a0bfd740 1 -- 172.21.15.12:0/808748199 shutdown_connections 2024-04-22T13:32:59.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.789+0000 7f89a0bfd740 1 --2- 172.21.15.12:0/808748199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583cc5eca00 0x5583cc5ebe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.789+0000 7f89a0bfd740 1 --2- 172.21.15.12:0/808748199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583cc5e3e80 0x5583cc5df600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.789+0000 7f89a0bfd740 1 --2- 172.21.15.12:0/808748199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583cc5e34d0 0x5583cc5f87d0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.789+0000 7f89a0bfd740 1 -- 172.21.15.12:0/808748199 >> 172.21.15.12:0/808748199 conn(0x5583cc5ddba0 msgr2=0x5583cc5deef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:59.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.789+0000 7f89a0bfd740 1 -- 172.21.15.12:0/808748199 shutdown_connections 2024-04-22T13:32:59.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.789+0000 7f89a0bfd740 1 -- 172.21.15.12:0/808748199 wait complete. 2024-04-22T13:32:59.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.789+0000 7f89a0bfd740 1 -- start start 2024-04-22T13:32:59.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.789+0000 7f89a0bfd740 1 Processor -- start 2024-04-22T13:32:59.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.789+0000 7f89a0bfd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583cc5e34d0 0x5583cc5f4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.789+0000 7f89a0bfd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583cc5e3e80 0x5583cc5f02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.789+0000 7f89a0bfd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583cc5eca00 0x5583cc5f07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.789+0000 7f89a0bfd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f89981b11b0 con 0x5583cc5e34d0 2024-04-22T13:32:59.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.789+0000 7f89a0bfd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f89981b0ae0 con 0x5583cc5eca00 2024-04-22T13:32:59.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.789+0000 7f89a0bfd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f89981add40 con 0x5583cc5e3e80 2024-04-22T13:32:59.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.789+0000 7f89a0bfd740 10 client.0 _ll_register_callbacks cb 0x7f898c009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:59.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.789+0000 7f8996ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583cc5e34d0 0x5583cc5f4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.789+0000 7f8996ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583cc5e34d0 0x5583cc5f4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51378/0 (socket says 172.21.15.12:51378) 2024-04-22T13:32:59.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.789+0000 7f8996ffd640 1 -- 172.21.15.12:0/3703212130 learned_addr learned my addr 172.21.15.12:0/3703212130 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:59.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.790+0000 7f89977fe640 1 --2- 172.21.15.12:0/3703212130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583cc5e3e80 0x5583cc5f02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.790+0000 7f8996ffd640 1 -- 172.21.15.12:0/3703212130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583cc5e3e80 msgr2=0x5583cc5f02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:59.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.790+0000 7f8996ffd640 1 --2- 172.21.15.12:0/3703212130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583cc5e3e80 0x5583cc5f02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.790+0000 7f8996ffd640 1 -- 172.21.15.12:0/3703212130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583cc5eca00 msgr2=0x5583cc5f07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:59.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.790+0000 7f8996ffd640 1 --2- 172.21.15.12:0/3703212130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583cc5eca00 0x5583cc5f07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.790+0000 7f8996ffd640 1 -- 172.21.15.12:0/3703212130 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583cc5ea050 con 0x5583cc5e34d0 2024-04-22T13:32:59.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.790+0000 7f8996ffd640 1 --2- 172.21.15.12:0/3703212130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583cc5e34d0 0x5583cc5f4f00 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f8988009840 tx=0x7f8988005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a10dcbe29d62a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:59.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.790+0000 7f8976ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:32:59.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.790+0000 7f8976ffd640 1 -- 172.21.15.12:0/3703212130 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898805e020 con 0x5583cc5e34d0 2024-04-22T13:32:59.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.790+0000 7f89a0bfd740 1 -- 172.21.15.12:0/3703212130 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f898802cc90 con 0x5583cc5e34d0 2024-04-22T13:32:59.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.790+0000 7f8976ffd640 1 -- 172.21.15.12:0/3703212130 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f8988004390 con 0x5583cc5e34d0 2024-04-22T13:32:59.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.790+0000 7f8976ffd640 1 -- 172.21.15.12:0/3703212130 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8988047040 con 0x5583cc5e34d0 2024-04-22T13:32:59.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.790+0000 7f89a0bfd740 10 client.18681 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:59.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.790+0000 7f89a0bfd740 1 -- 172.21.15.12:0/3703212130 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5583cc662e80 con 0x5583cc5e34d0 2024-04-22T13:32:59.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.790+0000 7f8976ffd640 1 -- 172.21.15.12:0/3703212130 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f898802cc90 con 0x5583cc5e34d0 2024-04-22T13:32:59.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.791+0000 7f8976ffd640 1 -- 172.21.15.12:0/3703212130 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f89880035f0 con 0x5583cc5e34d0 2024-04-22T13:32:59.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.791+0000 7f89a0bfd740 10 client.18681 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:59.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.791+0000 7f89a0bfd740 10 client.18681 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:32:59.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.791+0000 7f89a0bfd740 20 client.18681 populate_metadata read hostname 'smithi012' 2024-04-22T13:32:59.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.791+0000 7f89a0bfd740 1 --2- 172.21.15.12:0/3703212130 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5583cc6630b0 0x5583cc6654a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.791+0000 7f89a0bfd740 4 client.18681 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:32:59.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.791+0000 7f89a0bfd740 1 -- 172.21.15.12:0/3703212130 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5583cc555c10 con 0x5583cc6630b0 2024-04-22T13:32:59.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.791+0000 7f89977fe640 1 --2- 172.21.15.12:0/3703212130 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5583cc6630b0 0x5583cc6654a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.791+0000 7f89977fe640 1 --2- 172.21.15.12:0/3703212130 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5583cc6630b0 0x5583cc6654a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6c6a58804208cc73 server_cookie=cfffc60902493f72 in_seq=0 out_seq=0 2024-04-22T13:32:59.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.791+0000 7f8976ffd640 10 client.18681 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:32:59.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.792+0000 7f8976ffd640 1 -- 172.21.15.12:0/3703212130 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5583cc555c10 con 0x5583cc6630b0 2024-04-22T13:32:59.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.792+0000 7f8976ffd640 10 client.18681 handle_command_reply: tid=0 2024-04-22T13:32:59.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.792+0000 7f89a0bfd740 1 client.18681 shutdown 2024-04-22T13:32:59.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.792+0000 7f89a0bfd740 1 -- 172.21.15.12:0/3703212130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583cc5e34d0 msgr2=0x5583cc5f4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.792+0000 7f89a0bfd740 1 --2- 172.21.15.12:0/3703212130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583cc5e34d0 0x5583cc5f4f00 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f8988009840 tx=0x7f8988005e70 comp rx=0 tx=0).stop 2024-04-22T13:32:59.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.792+0000 7f89a0bfd740 1 -- 172.21.15.12:0/3703212130 shutdown_connections 2024-04-22T13:32:59.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.792+0000 7f89a0bfd740 1 --2- 172.21.15.12:0/3703212130 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5583cc6630b0 0x5583cc6654a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.792+0000 7f89a0bfd740 1 --2- 172.21.15.12:0/3703212130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583cc5eca00 0x5583cc5f07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.792+0000 7f89a0bfd740 1 --2- 172.21.15.12:0/3703212130 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5583cc5e3e80 0x5583cc5f02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.792+0000 7f8976ffd640 0 client.18681 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:32:59.794 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.792+0000 7f89a0bfd740 1 --2- 172.21.15.12:0/3703212130 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5583cc5e34d0 0x5583cc5f4f00 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.792+0000 7f89a0bfd740 1 -- 172.21.15.12:0/3703212130 >> 172.21.15.12:0/3703212130 conn(0x5583cc5ddba0 msgr2=0x5583cc5df780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:59.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.792+0000 7f89a0bfd740 1 -- 172.21.15.12:0/3703212130 shutdown_connections 2024-04-22T13:32:59.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.792+0000 7f89a0bfd740 1 -- 172.21.15.12:0/3703212130 wait complete. 2024-04-22T13:32:59.795 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.792+0000 7f89a0bfd740 20 client.18681 trim_cache size 0 max 16384 2024-04-22T13:32:59.795 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:32:59.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.792+0000 7f899e00f640 1 -- 172.21.15.12:0/2892716437 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f89800a3d50 msgr2=0x7f89800a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.793+0000 7f899e00f640 1 --2- 172.21.15.12:0/2892716437 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f89800a3d50 0x7f89800a6210 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f899000efd0 tx=0x7f8990087040 comp rx=0 tx=0).stop 2024-04-22T13:32:59.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.793+0000 7f899e00f640 1 -- 172.21.15.12:0/2892716437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8998101900 msgr2=0x7f89981a68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.793+0000 7f899e00f640 1 --2- 172.21.15.12:0/2892716437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8998101900 0x7f89981a68e0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f899000eaa0 tx=0x7f899000ef60 comp rx=0 tx=0).stop 2024-04-22T13:32:59.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.793+0000 7f899e00f640 1 -- 172.21.15.12:0/2892716437 shutdown_connections 2024-04-22T13:32:59.796 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.793+0000 7f899e00f640 1 --2- 172.21.15.12:0/2892716437 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f89800a3d50 0x7f89800a6210 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.793+0000 7f899e00f640 1 --2- 172.21.15.12:0/2892716437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8998101900 0x7f89981a68e0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.793+0000 7f899e00f640 1 --2- 172.21.15.12:0/2892716437 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8998107270 0x7f89981013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.793+0000 7f899e00f640 1 --2- 172.21.15.12:0/2892716437 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f899806a840 0x7f8998100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.793+0000 7f899e00f640 1 -- 172.21.15.12:0/2892716437 >> 172.21.15.12:0/2892716437 conn(0x7f8998077eb0 msgr2=0x7f8998108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:59.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.793+0000 7f899e00f640 1 -- 172.21.15.12:0/2892716437 shutdown_connections 2024-04-22T13:32:59.797 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.793+0000 7f899e00f640 1 -- 172.21.15.12:0/2892716437 wait complete. 2024-04-22T13:32:59.799 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:32:59.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.907+0000 7fb1a5ed2640 1 -- 172.21.15.12:0/4131929846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1a0102b40 msgr2=0x7fb1a0106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.907+0000 7fb1a5ed2640 1 --2- 172.21.15.12:0/4131929846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1a0102b40 0x7fb1a0106b80 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fb194009900 tx=0x7fb19402bc30 comp rx=0 tx=0).stop 2024-04-22T13:32:59.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.907+0000 7fb1a5ed2640 1 -- 172.21.15.12:0/4131929846 shutdown_connections 2024-04-22T13:32:59.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.907+0000 7fb1a5ed2640 1 --2- 172.21.15.12:0/4131929846 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1a0107260 0x7fb1a010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.907+0000 7fb1a5ed2640 1 --2- 172.21.15.12:0/4131929846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1a0102b40 0x7fb1a0106b80 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.907+0000 7fb1a5ed2640 1 --2- 172.21.15.12:0/4131929846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1a0102190 0x7fb1a0102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.905 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.907+0000 7fb1a5ed2640 1 -- 172.21.15.12:0/4131929846 >> 172.21.15.12:0/4131929846 conn(0x7fb1a00fd990 msgr2=0x7fb1a00ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:59.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.907+0000 7fb1a5ed2640 1 -- 172.21.15.12:0/4131929846 shutdown_connections 2024-04-22T13:32:59.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.907+0000 7fb1a5ed2640 1 -- 172.21.15.12:0/4131929846 wait complete. 2024-04-22T13:32:59.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.908+0000 7fb1a5ed2640 1 Processor -- start 2024-04-22T13:32:59.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.908+0000 7fb1a5ed2640 1 -- start start 2024-04-22T13:32:59.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.908+0000 7fb1a5ed2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1a0102190 0x7fb1a01a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.908+0000 7fb1a5ed2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1a0107260 0x7fb1a01a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.908+0000 7fb1a5ed2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1a019a710 0x7fb1a019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.908+0000 7fb1a5ed2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a01121f0 con 0x7fb1a0107260 2024-04-22T13:32:59.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.908+0000 7fb1a5ed2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a0112070 con 0x7fb1a019a710 2024-04-22T13:32:59.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.908+0000 7fb1a5ed2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a0112370 con 0x7fb1a0102190 2024-04-22T13:32:59.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.908+0000 7fb19effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1a0107260 0x7fb1a01a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.908+0000 7fb19effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1a0107260 0x7fb1a01a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51404/0 (socket says 172.21.15.12:51404) 2024-04-22T13:32:59.906 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.908+0000 7fb19effd640 1 -- 172.21.15.12:0/3627350956 learned_addr learned my addr 172.21.15.12:0/3627350956 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:59.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.909+0000 7fb19ffff640 1 --2- 172.21.15.12:0/3627350956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1a019a710 0x7fb1a019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.909+0000 7fb19f7fe640 1 --2- 172.21.15.12:0/3627350956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1a0102190 0x7fb1a01a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.909+0000 7fb19effd640 1 -- 172.21.15.12:0/3627350956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1a0102190 msgr2=0x7fb1a01a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.909+0000 7fb19effd640 1 --2- 172.21.15.12:0/3627350956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1a0102190 0x7fb1a01a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.909+0000 7fb19effd640 1 -- 172.21.15.12:0/3627350956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1a019a710 msgr2=0x7fb1a019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.909+0000 7fb19effd640 1 --2- 172.21.15.12:0/3627350956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1a019a710 0x7fb1a019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.909+0000 7fb19effd640 1 -- 172.21.15.12:0/3627350956 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a019b310 con 0x7fb1a0107260 2024-04-22T13:32:59.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.909+0000 7fb19effd640 1 --2- 172.21.15.12:0/3627350956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1a0107260 0x7fb1a01a0b90 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fb194000c00 tx=0x7fb1940045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:59.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.909+0000 7fb19cff9640 1 -- 172.21.15.12:0/3627350956 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb194043020 con 0x7fb1a0107260 2024-04-22T13:32:59.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.909+0000 7fb19cff9640 1 -- 172.21.15.12:0/3627350956 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb19402cb70 con 0x7fb1a0107260 2024-04-22T13:32:59.907 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.909+0000 7fb19cff9640 1 -- 172.21.15.12:0/3627350956 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb194003b50 con 0x7fb1a0107260 2024-04-22T13:32:59.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.909+0000 7fb1a5ed2640 1 -- 172.21.15.12:0/3627350956 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a019b5a0 con 0x7fb1a0107260 2024-04-22T13:32:59.908 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.909+0000 7fb1a5ed2640 1 -- 172.21.15.12:0/3627350956 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a01a7790 con 0x7fb1a0107260 2024-04-22T13:32:59.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.912+0000 7fb19cff9640 1 -- 172.21.15.12:0/3627350956 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb194046020 con 0x7fb1a0107260 2024-04-22T13:32:59.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.912+0000 7fb1a8ba1740 1 Processor -- start 2024-04-22T13:32:59.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.912+0000 7fb1a8ba1740 1 -- start start 2024-04-22T13:32:59.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.912+0000 7fb1a8ba1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3c69ab4d0 0x55a3c69ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.912+0000 7fb1a8ba1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3c69abe80 0x55a3c69b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.912+0000 7fb1a8ba1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c69b4a00 0x55a3c69b6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.912+0000 7fb1a8ba1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a3c69101f0 con 0x55a3c69ab4d0 2024-04-22T13:32:59.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.912+0000 7fb1a8ba1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a3c68ad4b0 con 0x55a3c69b4a00 2024-04-22T13:32:59.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.912+0000 7fb1a8ba1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a3c690c1b0 con 0x55a3c69abe80 2024-04-22T13:32:59.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.912+0000 7fb19ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3c69ab4d0 0x55a3c69ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.912+0000 7fb19ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3c69ab4d0 0x55a3c69ab8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51420/0 (socket says 172.21.15.12:51420) 2024-04-22T13:32:59.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.912+0000 7fb19cff9640 1 --2- 172.21.15.12:0/3627350956 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb1880a3da0 0x7fb1880a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.912+0000 7fb19ffff640 1 -- 172.21.15.12:0/3656441882 learned_addr learned my addr 172.21.15.12:0/3656441882 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:59.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.912+0000 7fb19ffff640 1 --2- 172.21.15.12:0/3627350956 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb1880a3da0 0x7fb1880a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.912+0000 7fb19cff9640 1 -- 172.21.15.12:0/3627350956 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7fb1940f27f0 con 0x7fb1a0107260 2024-04-22T13:32:59.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.912+0000 7fb19effd640 1 --2- 172.21.15.12:0/3656441882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c69b4a00 0x55a3c69b6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.912+0000 7fb19f7fe640 1 --2- 172.21.15.12:0/3656441882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3c69abe80 0x55a3c69b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.913+0000 7fb19ffff640 1 -- 172.21.15.12:0/3656441882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3c69abe80 msgr2=0x55a3c69b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:59.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.913+0000 7fb19ffff640 1 --2- 172.21.15.12:0/3656441882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3c69abe80 0x55a3c69b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.913+0000 7fb19ffff640 1 -- 172.21.15.12:0/3656441882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c69b4a00 msgr2=0x55a3c69b6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:59.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.913+0000 7fb19ffff640 1 --2- 172.21.15.12:0/3656441882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c69b4a00 0x55a3c69b6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.913+0000 7fb19ffff640 1 -- 172.21.15.12:0/3656441882 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3c689f1b0 con 0x55a3c69ab4d0 2024-04-22T13:32:59.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.913+0000 7fb19effd640 1 --2- 172.21.15.12:0/3656441882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c69b4a00 0x55a3c69b6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:59.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.913+0000 7fb19ffff640 1 --2- 172.21.15.12:0/3656441882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3c69ab4d0 0x55a3c69ab8b0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fb198012890 tx=0x7fb198012d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8635734ca575ced2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:59.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.913+0000 7fb185ffb640 1 -- 172.21.15.12:0/3656441882 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb198005020 con 0x55a3c69ab4d0 2024-04-22T13:32:59.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.913+0000 7fb185ffb640 1 -- 172.21.15.12:0/3656441882 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb198005670 con 0x55a3c69ab4d0 2024-04-22T13:32:59.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.913+0000 7fb185ffb640 1 -- 172.21.15.12:0/3656441882 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19801c750 con 0x55a3c69ab4d0 2024-04-22T13:32:59.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.913+0000 7fb19ffff640 1 --2- 172.21.15.12:0/3627350956 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb1880a3da0 0x7fb1880a6260 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb19801f000 tx=0x7fb19801e000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:59.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.914+0000 7fb1a8ba1740 1 -- 172.21.15.12:0/3656441882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3c69ab4d0 msgr2=0x55a3c69ab8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.914+0000 7fb1a8ba1740 1 --2- 172.21.15.12:0/3656441882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3c69ab4d0 0x55a3c69ab8b0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fb198012890 tx=0x7fb198012d50 comp rx=0 tx=0).stop 2024-04-22T13:32:59.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.914+0000 7fb1a8ba1740 1 -- 172.21.15.12:0/3656441882 shutdown_connections 2024-04-22T13:32:59.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.914+0000 7fb1a8ba1740 1 --2- 172.21.15.12:0/3656441882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c69b4a00 0x55a3c69b6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.914+0000 7fb1a8ba1740 1 --2- 172.21.15.12:0/3656441882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3c69abe80 0x55a3c69b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.914+0000 7fb1a8ba1740 1 --2- 172.21.15.12:0/3656441882 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3c69ab4d0 0x55a3c69ab8b0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.914+0000 7fb1a8ba1740 1 -- 172.21.15.12:0/3656441882 >> 172.21.15.12:0/3656441882 conn(0x55a3c69a5ba0 msgr2=0x55a3c69a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:59.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.914+0000 7fb1a8ba1740 1 -- 172.21.15.12:0/3656441882 shutdown_connections 2024-04-22T13:32:59.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.914+0000 7fb1a8ba1740 1 -- 172.21.15.12:0/3656441882 wait complete. 2024-04-22T13:32:59.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.914+0000 7fb1a8ba1740 1 -- start start 2024-04-22T13:32:59.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.914+0000 7fb1a8ba1740 1 Processor -- start 2024-04-22T13:32:59.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.914+0000 7fb1a8ba1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3c69ab4d0 0x55a3c69b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.914+0000 7fb1a8ba1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c69abe80 0x55a3c69b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.914+0000 7fb1a8ba1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3c69b4a00 0x55a3c69b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.914+0000 7fb1a8ba1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a3c69b7970 con 0x55a3c69ab4d0 2024-04-22T13:32:59.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.915+0000 7fb1a8ba1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a3c68c9500 con 0x55a3c69abe80 2024-04-22T13:32:59.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.915+0000 7fb1a8ba1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a3c69b7af0 con 0x55a3c69b4a00 2024-04-22T13:32:59.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.915+0000 7fb19f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3c69ab4d0 0x55a3c69b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.915+0000 7fb1a8ba1740 10 client.0 _ll_register_callbacks cb 0x55a3c6897b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:59.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.915+0000 7fb19f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3c69ab4d0 0x55a3c69b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51426/0 (socket says 172.21.15.12:51426) 2024-04-22T13:32:59.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.915+0000 7fb19f7fe640 1 -- 172.21.15.12:0/100473058 learned_addr learned my addr 172.21.15.12:0/100473058 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:59.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.915+0000 7fb19effd640 1 --2- 172.21.15.12:0/100473058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c69abe80 0x55a3c69b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.915+0000 7fb19ffff640 1 --2- 172.21.15.12:0/100473058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3c69b4a00 0x55a3c69b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.915+0000 7fb19f7fe640 1 -- 172.21.15.12:0/100473058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3c69b4a00 msgr2=0x55a3c69b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:59.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.915+0000 7fb19f7fe640 1 --2- 172.21.15.12:0/100473058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3c69b4a00 0x55a3c69b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.915+0000 7fb19f7fe640 1 -- 172.21.15.12:0/100473058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c69abe80 msgr2=0x55a3c69b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:59.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.915+0000 7fb19f7fe640 1 --2- 172.21.15.12:0/100473058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c69abe80 0x55a3c69b16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.915+0000 7fb19f7fe640 1 -- 172.21.15.12:0/100473058 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3c66223f0 con 0x55a3c69ab4d0 2024-04-22T13:32:59.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.915+0000 7fb19ffff640 1 --2- 172.21.15.12:0/100473058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3c69b4a00 0x55a3c69b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:32:59.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.915+0000 7fb19f7fe640 1 --2- 172.21.15.12:0/100473058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3c69ab4d0 0x55a3c69b4330 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fb190013710 tx=0x7fb190013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31da9a89c6460054 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:59.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.915+0000 7fb17effd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:32:59.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.915+0000 7fb17effd640 1 -- 172.21.15.12:0/100473058 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb190004460 con 0x55a3c69ab4d0 2024-04-22T13:32:59.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.915+0000 7fb17effd640 1 -- 172.21.15.12:0/100473058 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7fb1900119c0 con 0x55a3c69ab4d0 2024-04-22T13:32:59.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.915+0000 7fb1a8ba1740 1 -- 172.21.15.12:0/100473058 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb198005670 con 0x55a3c69ab4d0 2024-04-22T13:32:59.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.915+0000 7fb17effd640 1 -- 172.21.15.12:0/100473058 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb190011bc0 con 0x55a3c69ab4d0 2024-04-22T13:32:59.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.915+0000 7fb1a8ba1740 10 client.18693 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:59.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.915+0000 7fb1a8ba1740 1 -- 172.21.15.12:0/100473058 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a3c66ec7d0 con 0x55a3c69ab4d0 2024-04-22T13:32:59.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.916+0000 7fb17effd640 1 -- 172.21.15.12:0/100473058 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb198005670 con 0x55a3c69ab4d0 2024-04-22T13:32:59.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.916+0000 7fb17effd640 1 -- 172.21.15.12:0/100473058 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb19001a070 con 0x55a3c69ab4d0 2024-04-22T13:32:59.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.916+0000 7fb1a8ba1740 10 client.18693 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:59.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.916+0000 7fb1a8ba1740 10 client.18693 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:32:59.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.916+0000 7fb1a8ba1740 20 client.18693 populate_metadata read hostname 'smithi012' 2024-04-22T13:32:59.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.916+0000 7fb1a8ba1740 1 --2- 172.21.15.12:0/100473058 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55a3c69c4cd0 0x55a3c69c7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.916+0000 7fb1a8ba1740 4 client.18693 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:32:59.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.916+0000 7fb1a8ba1740 1 -- 172.21.15.12:0/100473058 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3c68af930 con 0x55a3c69c4cd0 2024-04-22T13:32:59.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.916+0000 7fb19effd640 1 --2- 172.21.15.12:0/100473058 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55a3c69c4cd0 0x55a3c69c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.917+0000 7fb19effd640 1 --2- 172.21.15.12:0/100473058 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55a3c69c4cd0 0x55a3c69c7140 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=253c44579a653149 server_cookie=35ada7db6118924f in_seq=0 out_seq=0 2024-04-22T13:32:59.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.917+0000 7fb17effd640 10 client.18693 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:32:59.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.917+0000 7fb17effd640 1 -- 172.21.15.12:0/100473058 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a3c68af930 con 0x55a3c69c4cd0 2024-04-22T13:32:59.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.917+0000 7fb17effd640 10 client.18693 handle_command_reply: tid=0 2024-04-22T13:32:59.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.917+0000 7fb1a8ba1740 1 client.18693 shutdown 2024-04-22T13:32:59.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.917+0000 7fb1a8ba1740 1 -- 172.21.15.12:0/100473058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3c69ab4d0 msgr2=0x55a3c69b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.917+0000 7fb1a8ba1740 1 --2- 172.21.15.12:0/100473058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3c69ab4d0 0x55a3c69b4330 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fb190013710 tx=0x7fb190013bd0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.917+0000 7fb1a8ba1740 1 -- 172.21.15.12:0/100473058 shutdown_connections 2024-04-22T13:32:59.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.917+0000 7fb1a8ba1740 1 --2- 172.21.15.12:0/100473058 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55a3c69c4cd0 0x55a3c69c7140 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.917+0000 7fb1a8ba1740 1 --2- 172.21.15.12:0/100473058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3c69b4a00 0x55a3c69b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.917+0000 7fb1a8ba1740 1 --2- 172.21.15.12:0/100473058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c69abe80 0x55a3c69b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.917+0000 7fb1a8ba1740 1 --2- 172.21.15.12:0/100473058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3c69ab4d0 0x55a3c69b4330 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.917+0000 7fb17effd640 0 client.18693 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:32:59.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.917+0000 7fb1a8ba1740 1 -- 172.21.15.12:0/100473058 >> 172.21.15.12:0/100473058 conn(0x55a3c69a5ba0 msgr2=0x55a3c69a78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:59.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.918+0000 7fb1a8ba1740 1 -- 172.21.15.12:0/100473058 shutdown_connections 2024-04-22T13:32:59.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.918+0000 7fb1a8ba1740 1 -- 172.21.15.12:0/100473058 wait complete. 2024-04-22T13:32:59.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.918+0000 7fb1a8ba1740 20 client.18693 trim_cache size 0 max 16384 2024-04-22T13:32:59.933 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:59 smithi134 ceph-mon[31338]: pgmap v273: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:32:59.933 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:32:59 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/432117285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:32:59.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.931+0000 7fb1a8ba1740 1 Processor -- start 2024-04-22T13:32:59.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.931+0000 7fb1a8ba1740 1 -- start start 2024-04-22T13:32:59.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.931+0000 7fb1a8ba1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3c69ab4d0 0x55a3c69c0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.931+0000 7fb1a8ba1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c69abe80 0x55a3c69afe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.931+0000 7fb1a8ba1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3c69b4a00 0x55a3c69b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.931+0000 7fb1a8ba1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a01afbc0 con 0x55a3c69ab4d0 2024-04-22T13:32:59.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.931+0000 7fb1a8ba1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a01b1700 con 0x55a3c69abe80 2024-04-22T13:32:59.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.931+0000 7fb1a8ba1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a01b0290 con 0x55a3c69b4a00 2024-04-22T13:32:59.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.931+0000 7fb19f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3c69ab4d0 0x55a3c69c0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.931+0000 7fb19f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3c69ab4d0 0x55a3c69c0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51438/0 (socket says 172.21.15.12:51438) 2024-04-22T13:32:59.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.931+0000 7fb19f7fe640 1 -- 172.21.15.12:0/3878263219 learned_addr learned my addr 172.21.15.12:0/3878263219 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:59.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.931+0000 7fb19ffff640 1 --2- 172.21.15.12:0/3878263219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3c69b4a00 0x55a3c69b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.931+0000 7fb19effd640 1 --2- 172.21.15.12:0/3878263219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c69abe80 0x55a3c69afe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.931+0000 7fb19f7fe640 1 -- 172.21.15.12:0/3878263219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3c69b4a00 msgr2=0x55a3c69b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:59.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.931+0000 7fb19f7fe640 1 --2- 172.21.15.12:0/3878263219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3c69b4a00 0x55a3c69b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.931+0000 7fb19f7fe640 1 -- 172.21.15.12:0/3878263219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c69abe80 msgr2=0x55a3c69afe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:59.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.931+0000 7fb19f7fe640 1 --2- 172.21.15.12:0/3878263219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c69abe80 0x55a3c69afe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.931+0000 7fb19f7fe640 1 -- 172.21.15.12:0/3878263219 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3c69c3820 con 0x55a3c69ab4d0 2024-04-22T13:32:59.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.932+0000 7fb19f7fe640 1 --2- 172.21.15.12:0/3878263219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3c69ab4d0 0x55a3c69c0860 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fb1900136e0 tx=0x7fb190011a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88f97d454377206 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:59.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.932+0000 7fb1857fa640 1 -- 172.21.15.12:0/3878263219 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb190032070 con 0x55a3c69ab4d0 2024-04-22T13:32:59.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.932+0000 7fb1857fa640 1 -- 172.21.15.12:0/3878263219 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb190003670 con 0x55a3c69ab4d0 2024-04-22T13:32:59.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.932+0000 7fb1857fa640 1 -- 172.21.15.12:0/3878263219 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb190028070 con 0x55a3c69ab4d0 2024-04-22T13:32:59.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.932+0000 7fb1a8ba1740 1 -- 172.21.15.12:0/3878263219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3c69ab4d0 msgr2=0x55a3c69c0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.932+0000 7fb1a8ba1740 1 --2- 172.21.15.12:0/3878263219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3c69ab4d0 0x55a3c69c0860 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fb1900136e0 tx=0x7fb190011a40 comp rx=0 tx=0).stop 2024-04-22T13:32:59.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.932+0000 7fb1a8ba1740 1 -- 172.21.15.12:0/3878263219 shutdown_connections 2024-04-22T13:32:59.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.932+0000 7fb1a8ba1740 1 --2- 172.21.15.12:0/3878263219 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3c69b4a00 0x55a3c69b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.932+0000 7fb1a8ba1740 1 --2- 172.21.15.12:0/3878263219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c69abe80 0x55a3c69afe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.932+0000 7fb1a8ba1740 1 --2- 172.21.15.12:0/3878263219 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3c69ab4d0 0x55a3c69c0860 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.936 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.932+0000 7fb1a8ba1740 1 -- 172.21.15.12:0/3878263219 >> 172.21.15.12:0/3878263219 conn(0x55a3c69a5ba0 msgr2=0x55a3c69a7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:59.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.932+0000 7fb1a8ba1740 1 -- 172.21.15.12:0/3878263219 shutdown_connections 2024-04-22T13:32:59.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.932+0000 7fb1a8ba1740 1 -- 172.21.15.12:0/3878263219 wait complete. 2024-04-22T13:32:59.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.933+0000 7fb1a8ba1740 1 -- start start 2024-04-22T13:32:59.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.933+0000 7fb1a8ba1740 1 Processor -- start 2024-04-22T13:32:59.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.933+0000 7fb1a8ba1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3c69ab4d0 0x55a3c69bd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.933+0000 7fb1a8ba1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c69abe80 0x55a3c69b8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.933+0000 7fb1a8ba1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3c69b4a00 0x55a3c69b8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.933+0000 7fb1a8ba1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a01b1030 con 0x55a3c69ab4d0 2024-04-22T13:32:59.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.933+0000 7fb1a8ba1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a01b0960 con 0x55a3c69abe80 2024-04-22T13:32:59.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.933+0000 7fb1a8ba1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a01adbc0 con 0x55a3c69b4a00 2024-04-22T13:32:59.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.933+0000 7fb1a8ba1740 10 client.0 _ll_register_callbacks cb 0x7fb190009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:32:59.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.933+0000 7fb19f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3c69ab4d0 0x55a3c69bd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.933+0000 7fb19f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3c69ab4d0 0x55a3c69bd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51454/0 (socket says 172.21.15.12:51454) 2024-04-22T13:32:59.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.933+0000 7fb19f7fe640 1 -- 172.21.15.12:0/1799555159 learned_addr learned my addr 172.21.15.12:0/1799555159 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:32:59.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.933+0000 7fb19ffff640 1 --2- 172.21.15.12:0/1799555159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3c69b4a00 0x55a3c69b8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.933+0000 7fb19effd640 1 --2- 172.21.15.12:0/1799555159 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c69abe80 0x55a3c69b8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.934+0000 7fb19f7fe640 1 -- 172.21.15.12:0/1799555159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3c69b4a00 msgr2=0x55a3c69b8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:59.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.934+0000 7fb19f7fe640 1 --2- 172.21.15.12:0/1799555159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3c69b4a00 0x55a3c69b8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.934+0000 7fb19f7fe640 1 -- 172.21.15.12:0/1799555159 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c69abe80 msgr2=0x55a3c69b8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:32:59.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.934+0000 7fb19f7fe640 1 --2- 172.21.15.12:0/1799555159 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c69abe80 0x55a3c69b8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.934+0000 7fb19f7fe640 1 -- 172.21.15.12:0/1799555159 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3c69b92f0 con 0x55a3c69ab4d0 2024-04-22T13:32:59.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.934+0000 7fb19ffff640 1 --2- 172.21.15.12:0/1799555159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3c69b4a00 0x55a3c69b8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:32:59.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.934+0000 7fb19f7fe640 1 --2- 172.21.15.12:0/1799555159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3c69ab4d0 0x55a3c69bd1a0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fb190009830 tx=0x7fb190009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b6d5736216831 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:32:59.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.934+0000 7fb17effd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:32:59.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.934+0000 7fb17effd640 1 -- 172.21.15.12:0/1799555159 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb190028070 con 0x55a3c69ab4d0 2024-04-22T13:32:59.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.934+0000 7fb17effd640 1 -- 172.21.15.12:0/1799555159 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7fb190003c30 con 0x55a3c69ab4d0 2024-04-22T13:32:59.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.934+0000 7fb1a8ba1740 1 -- 172.21.15.12:0/1799555159 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb190003670 con 0x55a3c69ab4d0 2024-04-22T13:32:59.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.934+0000 7fb17effd640 1 -- 172.21.15.12:0/1799555159 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb190032070 con 0x55a3c69ab4d0 2024-04-22T13:32:59.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.934+0000 7fb1a8ba1740 10 client.18699 fetch_fsmap learned FSMap version 28 2024-04-22T13:32:59.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.934+0000 7fb1a8ba1740 1 -- 172.21.15.12:0/1799555159 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a3c6a2b310 con 0x55a3c69ab4d0 2024-04-22T13:32:59.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.934+0000 7fb17effd640 1 -- 172.21.15.12:0/1799555159 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb190003670 con 0x55a3c69ab4d0 2024-04-22T13:32:59.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.934+0000 7fb17effd640 1 -- 172.21.15.12:0/1799555159 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb19003e030 con 0x55a3c69ab4d0 2024-04-22T13:32:59.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.934+0000 7fb1a8ba1740 10 client.18699 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:32:59.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.934+0000 7fb1a8ba1740 10 client.18699 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:32:59.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.934+0000 7fb1a8ba1740 20 client.18699 populate_metadata read hostname 'smithi012' 2024-04-22T13:32:59.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.935+0000 7fb1a8ba1740 1 --2- 172.21.15.12:0/1799555159 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55a3c6a2b960 0x55a3c6a2dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:32:59.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.935+0000 7fb1a8ba1740 4 client.18699 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:32:59.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.935+0000 7fb1a8ba1740 1 -- 172.21.15.12:0/1799555159 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a3c691dc10 con 0x55a3c6a2b960 2024-04-22T13:32:59.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.935+0000 7fb19effd640 1 --2- 172.21.15.12:0/1799555159 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55a3c6a2b960 0x55a3c6a2dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:32:59.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.935+0000 7fb17effd640 10 client.18699 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:32:59.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.935+0000 7fb19effd640 1 --2- 172.21.15.12:0/1799555159 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55a3c6a2b960 0x55a3c6a2dd50 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b56786b369dd91a7 server_cookie=42a34904d19278c in_seq=0 out_seq=0 2024-04-22T13:32:59.940 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.935+0000 7fb17effd640 1 -- 172.21.15.12:0/1799555159 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a3c691dc10 con 0x55a3c6a2b960 2024-04-22T13:32:59.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.935+0000 7fb17effd640 10 client.18699 handle_command_reply: tid=0 2024-04-22T13:32:59.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.935+0000 7fb1a8ba1740 1 client.18699 shutdown 2024-04-22T13:32:59.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.935+0000 7fb1a8ba1740 1 -- 172.21.15.12:0/1799555159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3c69ab4d0 msgr2=0x55a3c69bd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.935+0000 7fb1a8ba1740 1 --2- 172.21.15.12:0/1799555159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3c69ab4d0 0x55a3c69bd1a0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fb190009830 tx=0x7fb190009fa0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.936+0000 7fb1a8ba1740 1 -- 172.21.15.12:0/1799555159 shutdown_connections 2024-04-22T13:32:59.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.936+0000 7fb1a8ba1740 1 --2- 172.21.15.12:0/1799555159 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55a3c6a2b960 0x55a3c6a2dd50 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.936+0000 7fb1a8ba1740 1 --2- 172.21.15.12:0/1799555159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3c69b4a00 0x55a3c69b8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.936+0000 7fb1a8ba1740 1 --2- 172.21.15.12:0/1799555159 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c69abe80 0x55a3c69b8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.936+0000 7fb1a8ba1740 1 --2- 172.21.15.12:0/1799555159 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3c69ab4d0 0x55a3c69bd1a0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.936+0000 7fb17effd640 0 client.18699 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:32:59.941 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.936+0000 7fb1a8ba1740 1 -- 172.21.15.12:0/1799555159 >> 172.21.15.12:0/1799555159 conn(0x55a3c69a5ba0 msgr2=0x55a3c69ac260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:59.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.936+0000 7fb1a8ba1740 1 -- 172.21.15.12:0/1799555159 shutdown_connections 2024-04-22T13:32:59.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.936+0000 7fb1a8ba1740 1 -- 172.21.15.12:0/1799555159 wait complete. 2024-04-22T13:32:59.942 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.936+0000 7fb1a8ba1740 20 client.18699 trim_cache size 0 max 16384 2024-04-22T13:32:59.942 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:32:59.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.936+0000 7fb1a5ed2640 1 -- 172.21.15.12:0/3627350956 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb1880a3da0 msgr2=0x7fb1880a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.936+0000 7fb1a5ed2640 1 --2- 172.21.15.12:0/3627350956 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb1880a3da0 0x7fb1880a6260 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb19801f000 tx=0x7fb19801e000 comp rx=0 tx=0).stop 2024-04-22T13:32:59.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.936+0000 7fb1a5ed2640 1 -- 172.21.15.12:0/3627350956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1a0107260 msgr2=0x7fb1a01a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:32:59.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.936+0000 7fb1a5ed2640 1 --2- 172.21.15.12:0/3627350956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1a0107260 0x7fb1a01a0b90 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fb194000c00 tx=0x7fb1940045e0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.943 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.936+0000 7fb1a5ed2640 1 -- 172.21.15.12:0/3627350956 shutdown_connections 2024-04-22T13:32:59.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.936+0000 7fb1a5ed2640 1 --2- 172.21.15.12:0/3627350956 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb1880a3da0 0x7fb1880a6260 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.936+0000 7fb1a5ed2640 1 --2- 172.21.15.12:0/3627350956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb1a019a710 0x7fb1a019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.936+0000 7fb1a5ed2640 1 --2- 172.21.15.12:0/3627350956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb1a0107260 0x7fb1a01a0b90 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.936+0000 7fb1a5ed2640 1 --2- 172.21.15.12:0/3627350956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1a0102190 0x7fb1a01a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:32:59.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.936+0000 7fb1a5ed2640 1 -- 172.21.15.12:0/3627350956 >> 172.21.15.12:0/3627350956 conn(0x7fb1a00fd990 msgr2=0x7fb1a00ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:32:59.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.936+0000 7fb1a5ed2640 1 -- 172.21.15.12:0/3627350956 shutdown_connections 2024-04-22T13:32:59.944 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:32:59.936+0000 7fb1a5ed2640 1 -- 172.21.15.12:0/3627350956 wait complete. 2024-04-22T13:32:59.944 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:59 smithi012 ceph-mon[28265]: pgmap v273: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:32:59.944 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:32:59 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/432117285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:32:59.945 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:32:59.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:59 smithi111 ceph-mon[31361]: pgmap v273: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:32:59.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:32:59 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/432117285' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:33:00.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.055+0000 7f6557443640 1 -- 172.21.15.12:0/238073420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65501011a0 msgr2=0x7f6550101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:00.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.055+0000 7f6557443640 1 --2- 172.21.15.12:0/238073420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65501011a0 0x7f6550101600 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f6540009930 tx=0x7f654002bc30 comp rx=0 tx=0).stop 2024-04-22T13:33:00.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.055+0000 7f6557443640 1 -- 172.21.15.12:0/238073420 shutdown_connections 2024-04-22T13:33:00.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.055+0000 7f6557443640 1 --2- 172.21.15.12:0/238073420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6550101b40 0x7f655010dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.055+0000 7f6557443640 1 --2- 172.21.15.12:0/238073420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f65501011a0 0x7f6550101600 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.055+0000 7f6557443640 1 --2- 172.21.15.12:0/238073420 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6550107030 0x7f6550107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.055+0000 7f6557443640 1 -- 172.21.15.12:0/238073420 >> 172.21.15.12:0/238073420 conn(0x7f6550076c50 msgr2=0x7f6550077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:00.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.055+0000 7f6557443640 1 -- 172.21.15.12:0/238073420 shutdown_connections 2024-04-22T13:33:00.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.055+0000 7f6557443640 1 -- 172.21.15.12:0/238073420 wait complete. 2024-04-22T13:33:00.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.056+0000 7f6557443640 1 Processor -- start 2024-04-22T13:33:00.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.056+0000 7f6557443640 1 -- start start 2024-04-22T13:33:00.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.056+0000 7f6557443640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65501011a0 0x7f65501995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.056+0000 7f6557443640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6550101b40 0x7f6550199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.056+0000 7f6557443640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6550107030 0x7f65501a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.056+0000 7f6557443640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f65501110e0 con 0x7f6550107030 2024-04-22T13:33:00.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.056+0000 7f6557443640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6550110f60 con 0x7f65501011a0 2024-04-22T13:33:00.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.056+0000 7f6557443640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6550111260 con 0x7f6550101b40 2024-04-22T13:33:00.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.056+0000 7f65559e0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6550107030 0x7f65501a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.056+0000 7f65559e0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6550107030 0x7f65501a4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51472/0 (socket says 172.21.15.12:51472) 2024-04-22T13:33:00.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.056+0000 7f65559e0640 1 -- 172.21.15.12:0/1008389916 learned_addr learned my addr 172.21.15.12:0/1008389916 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:00.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.057+0000 7f65549de640 1 --2- 172.21.15.12:0/1008389916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6550101b40 0x7f6550199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.057+0000 7f65551df640 1 --2- 172.21.15.12:0/1008389916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65501011a0 0x7f65501995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.057+0000 7f65559e0640 1 -- 172.21.15.12:0/1008389916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6550101b40 msgr2=0x7f6550199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:00.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.057+0000 7f65559e0640 1 --2- 172.21.15.12:0/1008389916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6550101b40 0x7f6550199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.057+0000 7f65559e0640 1 -- 172.21.15.12:0/1008389916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65501011a0 msgr2=0x7f65501995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:00.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.057+0000 7f65559e0640 1 --2- 172.21.15.12:0/1008389916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65501011a0 0x7f65501995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.057+0000 7f65559e0640 1 -- 172.21.15.12:0/1008389916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65501a4b60 con 0x7f6550107030 2024-04-22T13:33:00.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.057+0000 7f65559e0640 1 --2- 172.21.15.12:0/1008389916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6550107030 0x7f65501a4620 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f654800eaa0 tx=0x7f654800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:00.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.057+0000 7f65467fc640 1 -- 172.21.15.12:0/1008389916 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654800cdb0 con 0x7f6550107030 2024-04-22T13:33:00.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.057+0000 7f6557443640 1 -- 172.21.15.12:0/1008389916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65501a4d30 con 0x7f6550107030 2024-04-22T13:33:00.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.057+0000 7f65467fc640 1 -- 172.21.15.12:0/1008389916 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6548005020 con 0x7f6550107030 2024-04-22T13:33:00.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.057+0000 7f65467fc640 1 -- 172.21.15.12:0/1008389916 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65480116d0 con 0x7f6550107030 2024-04-22T13:33:00.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.057+0000 7f6557443640 1 -- 172.21.15.12:0/1008389916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65501a5250 con 0x7f6550107030 2024-04-22T13:33:00.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.059+0000 7f65467fc640 1 -- 172.21.15.12:0/1008389916 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f65480054a0 con 0x7f6550107030 2024-04-22T13:33:00.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.060+0000 7f655a035740 1 Processor -- start 2024-04-22T13:33:00.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.060+0000 7f655a035740 1 -- start start 2024-04-22T13:33:00.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.060+0000 7f655a035740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615656823b0 0x561565682790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.060+0000 7f655a035740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561565682d60 0x56156568b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.063 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.060+0000 7f655a035740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156568be60 0x56156568c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.060+0000 7f655a035740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5615655e71f0 con 0x56156568be60 2024-04-22T13:33:00.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.060+0000 7f655a035740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5615655844b0 con 0x561565682d60 2024-04-22T13:33:00.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.060+0000 7f655a035740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5615655e31b0 con 0x5615656823b0 2024-04-22T13:33:00.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.060+0000 7f65467fc640 1 --2- 172.21.15.12:0/1008389916 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f653c0a3d20 0x7f653c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.060+0000 7f65549de640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156568be60 0x56156568c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.060+0000 7f65549de640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156568be60 0x56156568c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51476/0 (socket says 172.21.15.12:51476) 2024-04-22T13:33:00.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.060+0000 7f65549de640 1 -- 172.21.15.12:0/3674738275 learned_addr learned my addr 172.21.15.12:0/3674738275 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:00.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.060+0000 7f65559e0640 1 --2- 172.21.15.12:0/1008389916 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f653c0a3d20 0x7f653c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.060+0000 7f65551df640 1 --2- 172.21.15.12:0/3674738275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561565682d60 0x56156568b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.060+0000 7f65549de640 1 --2- 172.21.15.12:0/3674738275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615656823b0 0x561565682790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.060+0000 7f65467fc640 1 -- 172.21.15.12:0/1008389916 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f65480d09d0 con 0x7f6550107030 2024-04-22T13:33:00.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.061+0000 7f65549de640 1 -- 172.21.15.12:0/3674738275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615656823b0 msgr2=0x561565682790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:00.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.061+0000 7f65549de640 1 --2- 172.21.15.12:0/3674738275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615656823b0 0x561565682790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.061+0000 7f65549de640 1 -- 172.21.15.12:0/3674738275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561565682d60 msgr2=0x56156568b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:00.064 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.061+0000 7f65549de640 1 --2- 172.21.15.12:0/3674738275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561565682d60 0x56156568b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.061+0000 7f65549de640 1 -- 172.21.15.12:0/3674738275 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615655761b0 con 0x56156568be60 2024-04-22T13:33:00.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.061+0000 7f65559e0640 1 --2- 172.21.15.12:0/1008389916 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f653c0a3d20 0x7f653c0a61e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f654809c630 tx=0x7f654806e370 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:00.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.061+0000 7f65549de640 1 --2- 172.21.15.12:0/3674738275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156568be60 0x56156568c2c0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f6540005e00 tx=0x7f6540005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb2e6deb5de4f69 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:00.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.061+0000 7f653b7fe640 1 -- 172.21.15.12:0/3674738275 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654004a070 con 0x56156568be60 2024-04-22T13:33:00.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.061+0000 7f653b7fe640 1 -- 172.21.15.12:0/3674738275 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6540005050 con 0x56156568be60 2024-04-22T13:33:00.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.061+0000 7f653b7fe640 1 -- 172.21.15.12:0/3674738275 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65400048e0 con 0x56156568be60 2024-04-22T13:33:00.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.062+0000 7f655a035740 1 -- 172.21.15.12:0/3674738275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156568be60 msgr2=0x56156568c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:00.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.062+0000 7f655a035740 1 --2- 172.21.15.12:0/3674738275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156568be60 0x56156568c2c0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f6540005e00 tx=0x7f6540005d90 comp rx=0 tx=0).stop 2024-04-22T13:33:00.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.062+0000 7f655a035740 1 -- 172.21.15.12:0/3674738275 shutdown_connections 2024-04-22T13:33:00.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.062+0000 7f655a035740 1 --2- 172.21.15.12:0/3674738275 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56156568be60 0x56156568c2c0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.062+0000 7f655a035740 1 --2- 172.21.15.12:0/3674738275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561565682d60 0x56156568b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.062+0000 7f655a035740 1 --2- 172.21.15.12:0/3674738275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615656823b0 0x561565682790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.062+0000 7f655a035740 1 -- 172.21.15.12:0/3674738275 >> 172.21.15.12:0/3674738275 conn(0x56156567cba0 msgr2=0x56156567cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:00.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.062+0000 7f655a035740 1 -- 172.21.15.12:0/3674738275 shutdown_connections 2024-04-22T13:33:00.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.062+0000 7f655a035740 1 -- 172.21.15.12:0/3674738275 wait complete. 2024-04-22T13:33:00.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.062+0000 7f655a035740 1 -- start start 2024-04-22T13:33:00.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.062+0000 7f655a035740 1 Processor -- start 2024-04-22T13:33:00.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.063+0000 7f655a035740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615656823b0 0x5615656922d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.063+0000 7f655a035740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561565682d60 0x56156568f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.063+0000 7f655a035740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56156568be60 0x56156568fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.063+0000 7f655a035740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561565692d40 con 0x5615656823b0 2024-04-22T13:33:00.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.063+0000 7f655a035740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5615655a0500 con 0x561565682d60 2024-04-22T13:33:00.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.063+0000 7f655a035740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561565692ec0 con 0x56156568be60 2024-04-22T13:33:00.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.063+0000 7f655a035740 10 client.0 _ll_register_callbacks cb 0x56156556eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:00.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.063+0000 7f65549de640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615656823b0 0x5615656922d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.063+0000 7f65549de640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615656823b0 0x5615656922d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51478/0 (socket says 172.21.15.12:51478) 2024-04-22T13:33:00.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.063+0000 7f65549de640 1 -- 172.21.15.12:0/470198082 learned_addr learned my addr 172.21.15.12:0/470198082 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:00.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.063+0000 7f65549de640 1 --2- 172.21.15.12:0/470198082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56156568be60 0x56156568fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.063+0000 7f65551df640 1 --2- 172.21.15.12:0/470198082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561565682d60 0x56156568f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.063+0000 7f65549de640 1 -- 172.21.15.12:0/470198082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56156568be60 msgr2=0x56156568fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:00.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.063+0000 7f65549de640 1 --2- 172.21.15.12:0/470198082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56156568be60 0x56156568fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.063+0000 7f65549de640 1 -- 172.21.15.12:0/470198082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561565682d60 msgr2=0x56156568f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:00.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.063+0000 7f65549de640 1 --2- 172.21.15.12:0/470198082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561565682d60 0x56156568f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.063+0000 7f65549de640 1 -- 172.21.15.12:0/470198082 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615652f93f0 con 0x5615656823b0 2024-04-22T13:33:00.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.063+0000 7f65549de640 1 --2- 172.21.15.12:0/470198082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615656823b0 0x5615656922d0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f6540048040 tx=0x7f654002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76f94dc9f70a696 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:00.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.063+0000 7f6538ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:33:00.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.063+0000 7f6538ff9640 1 -- 172.21.15.12:0/470198082 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654004a040 con 0x5615656823b0 2024-04-22T13:33:00.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.063+0000 7f655a035740 1 -- 172.21.15.12:0/470198082 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6540005050 con 0x5615656823b0 2024-04-22T13:33:00.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.063+0000 7f6538ff9640 1 -- 172.21.15.12:0/470198082 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f65400046c0 con 0x5615656823b0 2024-04-22T13:33:00.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.064+0000 7f6538ff9640 1 -- 172.21.15.12:0/470198082 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f65400048c0 con 0x5615656823b0 2024-04-22T13:33:00.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.064+0000 7f655a035740 10 client.18711 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:00.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.064+0000 7f655a035740 1 -- 172.21.15.12:0/470198082 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5615653c37d0 con 0x5615656823b0 2024-04-22T13:33:00.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.064+0000 7f6538ff9640 1 -- 172.21.15.12:0/470198082 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6540005050 con 0x5615656823b0 2024-04-22T13:33:00.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.064+0000 7f6538ff9640 1 -- 172.21.15.12:0/470198082 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6540059070 con 0x5615656823b0 2024-04-22T13:33:00.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.064+0000 7f655a035740 10 client.18711 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:00.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.064+0000 7f655a035740 10 client.18711 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:33:00.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.064+0000 7f655a035740 20 client.18711 populate_metadata read hostname 'smithi012' 2024-04-22T13:33:00.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.064+0000 7f655a035740 1 --2- 172.21.15.12:0/470198082 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x56156569fec0 0x5615656a22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.068 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.064+0000 7f655a035740 4 client.18711 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:33:00.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.064+0000 7f655a035740 1 -- 172.21.15.12:0/470198082 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561565586930 con 0x56156569fec0 2024-04-22T13:33:00.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.064+0000 7f65551df640 1 --2- 172.21.15.12:0/470198082 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x56156569fec0 0x5615656a22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.065+0000 7f6538ff9640 10 client.18711 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:33:00.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.065+0000 7f65551df640 1 --2- 172.21.15.12:0/470198082 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x56156569fec0 0x5615656a22b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9014f724e089bf08 server_cookie=67414958c1692f3d in_seq=0 out_seq=0 2024-04-22T13:33:00.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.066+0000 7f6538ff9640 1 -- 172.21.15.12:0/470198082 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561565586930 con 0x56156569fec0 2024-04-22T13:33:00.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.066+0000 7f6538ff9640 10 client.18711 handle_command_reply: tid=0 2024-04-22T13:33:00.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.066+0000 7f655a035740 1 client.18711 shutdown 2024-04-22T13:33:00.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.066+0000 7f655a035740 1 -- 172.21.15.12:0/470198082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615656823b0 msgr2=0x5615656922d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:00.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.066+0000 7f655a035740 1 --2- 172.21.15.12:0/470198082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615656823b0 0x5615656922d0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f6540048040 tx=0x7f654002bc30 comp rx=0 tx=0).stop 2024-04-22T13:33:00.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.066+0000 7f655a035740 1 -- 172.21.15.12:0/470198082 shutdown_connections 2024-04-22T13:33:00.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.066+0000 7f655a035740 1 --2- 172.21.15.12:0/470198082 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56156568be60 0x56156568fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.066+0000 7f655a035740 1 --2- 172.21.15.12:0/470198082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561565682d60 0x56156568f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.066+0000 7f655a035740 1 --2- 172.21.15.12:0/470198082 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x56156569fec0 0x5615656a22b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.066+0000 7f655a035740 1 --2- 172.21.15.12:0/470198082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615656823b0 0x5615656922d0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.066+0000 7f6538ff9640 0 client.18711 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:33:00.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.066+0000 7f655a035740 1 -- 172.21.15.12:0/470198082 >> 172.21.15.12:0/470198082 conn(0x56156567cba0 msgr2=0x56156567e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:00.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.066+0000 7f655a035740 1 -- 172.21.15.12:0/470198082 shutdown_connections 2024-04-22T13:33:00.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.066+0000 7f655a035740 1 -- 172.21.15.12:0/470198082 wait complete. 2024-04-22T13:33:00.070 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.066+0000 7f655a035740 20 client.18711 trim_cache size 0 max 16384 2024-04-22T13:33:00.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.079+0000 7f655a035740 1 Processor -- start 2024-04-22T13:33:00.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.079+0000 7f655a035740 1 -- start start 2024-04-22T13:33:00.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.080+0000 7f655a035740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615656823b0 0x56156568e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.080+0000 7f655a035740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561565682d60 0x56156568d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.080+0000 7f655a035740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56156568be60 0x56156569b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.080+0000 7f655a035740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f65501ab9b0 con 0x561565682d60 2024-04-22T13:33:00.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.080+0000 7f655a035740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f65501ad4f0 con 0x56156568be60 2024-04-22T13:33:00.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.080+0000 7f655a035740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f65501ac080 con 0x5615656823b0 2024-04-22T13:33:00.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.080+0000 7f65551df640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561565682d60 0x56156568d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.080+0000 7f65551df640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561565682d60 0x56156568d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51482/0 (socket says 172.21.15.12:51482) 2024-04-22T13:33:00.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.080+0000 7f65551df640 1 -- 172.21.15.12:0/412600442 learned_addr learned my addr 172.21.15.12:0/412600442 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:00.081 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.080+0000 7f65551df640 1 -- 172.21.15.12:0/412600442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615656823b0 msgr2=0x56156568e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:00.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.080+0000 7f65549de640 1 --2- 172.21.15.12:0/412600442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615656823b0 0x56156568e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.080+0000 7f65551df640 1 --2- 172.21.15.12:0/412600442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615656823b0 0x56156568e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.080+0000 7f65551df640 1 -- 172.21.15.12:0/412600442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56156568be60 msgr2=0x56156569b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:00.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.080+0000 7f65551df640 1 --2- 172.21.15.12:0/412600442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56156568be60 0x56156569b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.080+0000 7f65551df640 1 -- 172.21.15.12:0/412600442 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56156568ff50 con 0x561565682d60 2024-04-22T13:33:00.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.080+0000 7f65551df640 1 --2- 172.21.15.12:0/412600442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561565682d60 0x56156568d270 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x5615656831a0 tx=0x7f654c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=358e1f4f351f4e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:00.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.080+0000 7f653affd640 1 -- 172.21.15.12:0/412600442 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c01c430 con 0x561565682d60 2024-04-22T13:33:00.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.080+0000 7f653affd640 1 -- 172.21.15.12:0/412600442 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f654c005020 con 0x561565682d60 2024-04-22T13:33:00.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.080+0000 7f653affd640 1 -- 172.21.15.12:0/412600442 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c004a90 con 0x561565682d60 2024-04-22T13:33:00.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.081+0000 7f655a035740 1 -- 172.21.15.12:0/412600442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561565682d60 msgr2=0x56156568d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:00.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.081+0000 7f655a035740 1 --2- 172.21.15.12:0/412600442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561565682d60 0x56156568d270 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x5615656831a0 tx=0x7f654c012820 comp rx=0 tx=0).stop 2024-04-22T13:33:00.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.081+0000 7f655a035740 1 -- 172.21.15.12:0/412600442 shutdown_connections 2024-04-22T13:33:00.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.081+0000 7f655a035740 1 --2- 172.21.15.12:0/412600442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56156568be60 0x56156569b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.081+0000 7f655a035740 1 --2- 172.21.15.12:0/412600442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561565682d60 0x56156568d270 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.081+0000 7f655a035740 1 --2- 172.21.15.12:0/412600442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5615656823b0 0x56156568e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.081+0000 7f655a035740 1 -- 172.21.15.12:0/412600442 >> 172.21.15.12:0/412600442 conn(0x56156567cba0 msgr2=0x56156569b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:00.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.081+0000 7f655a035740 1 -- 172.21.15.12:0/412600442 shutdown_connections 2024-04-22T13:33:00.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.081+0000 7f655a035740 1 -- 172.21.15.12:0/412600442 wait complete. 2024-04-22T13:33:00.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.081+0000 7f655a035740 1 -- start start 2024-04-22T13:33:00.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.081+0000 7f655a035740 1 Processor -- start 2024-04-22T13:33:00.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.082+0000 7f655a035740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615656823b0 0x5615656a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.082+0000 7f655a035740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561565682d60 0x5615656a1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.082+0000 7f655a035740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56156568be60 0x5615656a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.082+0000 7f655a035740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f65501ace20 con 0x5615656823b0 2024-04-22T13:33:00.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.082+0000 7f655a035740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f65501ac750 con 0x56156568be60 2024-04-22T13:33:00.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.082+0000 7f655a035740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f65501a99b0 con 0x561565682d60 2024-04-22T13:33:00.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.082+0000 7f655a035740 10 client.0 _ll_register_callbacks cb 0x7f6540004270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:00.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.082+0000 7f65549de640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615656823b0 0x5615656a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.083 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.082+0000 7f65549de640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615656823b0 0x5615656a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51490/0 (socket says 172.21.15.12:51490) 2024-04-22T13:33:00.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.082+0000 7f65549de640 1 -- 172.21.15.12:0/3491778060 learned_addr learned my addr 172.21.15.12:0/3491778060 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:00.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.082+0000 7f65551df640 1 --2- 172.21.15.12:0/3491778060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561565682d60 0x5615656a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.082+0000 7f65549de640 1 --2- 172.21.15.12:0/3491778060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56156568be60 0x5615656a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.082+0000 7f65549de640 1 -- 172.21.15.12:0/3491778060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561565682d60 msgr2=0x5615656a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:00.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.082+0000 7f65549de640 1 --2- 172.21.15.12:0/3491778060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561565682d60 0x5615656a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.082+0000 7f65549de640 1 -- 172.21.15.12:0/3491778060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56156568be60 msgr2=0x5615656a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:00.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.082+0000 7f65549de640 1 --2- 172.21.15.12:0/3491778060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56156568be60 0x5615656a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.082+0000 7f65549de640 1 -- 172.21.15.12:0/3491778060 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561565705090 con 0x5615656823b0 2024-04-22T13:33:00.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.082+0000 7f65549de640 1 --2- 172.21.15.12:0/3491778060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615656823b0 0x5615656a0c20 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f654002c730 tx=0x7f654002cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c760b7b29ff8ce19 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:00.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.082+0000 7f6538ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:33:00.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.083+0000 7f6538ff9640 1 -- 172.21.15.12:0/3491778060 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654005f040 con 0x5615656823b0 2024-04-22T13:33:00.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.083+0000 7f6538ff9640 1 -- 172.21.15.12:0/3491778060 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f654002cba0 con 0x5615656823b0 2024-04-22T13:33:00.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.083+0000 7f655a035740 1 -- 172.21.15.12:0/3491778060 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f654c005020 con 0x5615656823b0 2024-04-22T13:33:00.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.083+0000 7f6538ff9640 1 -- 172.21.15.12:0/3491778060 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f654004a040 con 0x5615656823b0 2024-04-22T13:33:00.084 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.083+0000 7f655a035740 10 client.18717 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:00.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.083+0000 7f655a035740 1 -- 172.21.15.12:0/3491778060 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5615657062a0 con 0x5615656823b0 2024-04-22T13:33:00.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.083+0000 7f6538ff9640 1 -- 172.21.15.12:0/3491778060 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f654c005020 con 0x5615656823b0 2024-04-22T13:33:00.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.083+0000 7f6538ff9640 1 -- 172.21.15.12:0/3491778060 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6540059070 con 0x5615656823b0 2024-04-22T13:33:00.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.083+0000 7f655a035740 10 client.18717 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:00.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.083+0000 7f655a035740 10 client.18717 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:33:00.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.083+0000 7f655a035740 20 client.18717 populate_metadata read hostname 'smithi012' 2024-04-22T13:33:00.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.083+0000 7f655a035740 1 --2- 172.21.15.12:0/3491778060 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x561565706900 0x561565708d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.083+0000 7f655a035740 4 client.18717 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:33:00.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.083+0000 7f655a035740 1 -- 172.21.15.12:0/3491778060 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5615655f4c10 con 0x561565706900 2024-04-22T13:33:00.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.083+0000 7f65551df640 1 --2- 172.21.15.12:0/3491778060 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x561565706900 0x561565708d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.084+0000 7f6538ff9640 10 client.18717 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:33:00.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.084+0000 7f65551df640 1 --2- 172.21.15.12:0/3491778060 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x561565706900 0x561565708d20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e03082c9a0f7263 server_cookie=41b273eb4ee80cea in_seq=0 out_seq=0 2024-04-22T13:33:00.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.084+0000 7f6538ff9640 1 -- 172.21.15.12:0/3491778060 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5615655f4c10 con 0x561565706900 2024-04-22T13:33:00.085 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.084+0000 7f6538ff9640 10 client.18717 handle_command_reply: tid=0 2024-04-22T13:33:00.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.084+0000 7f655a035740 1 client.18717 shutdown 2024-04-22T13:33:00.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.084+0000 7f655a035740 1 -- 172.21.15.12:0/3491778060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615656823b0 msgr2=0x5615656a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:00.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.084+0000 7f655a035740 1 --2- 172.21.15.12:0/3491778060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615656823b0 0x5615656a0c20 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f654002c730 tx=0x7f654002cdb0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.084+0000 7f655a035740 1 -- 172.21.15.12:0/3491778060 shutdown_connections 2024-04-22T13:33:00.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.084+0000 7f655a035740 1 --2- 172.21.15.12:0/3491778060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56156568be60 0x5615656a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.084+0000 7f655a035740 1 --2- 172.21.15.12:0/3491778060 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561565682d60 0x5615656a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.084+0000 7f655a035740 1 --2- 172.21.15.12:0/3491778060 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x561565706900 0x561565708d20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.084+0000 7f655a035740 1 --2- 172.21.15.12:0/3491778060 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5615656823b0 0x5615656a0c20 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.084+0000 7f6538ff9640 0 client.18717 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:33:00.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.084+0000 7f655a035740 1 -- 172.21.15.12:0/3491778060 >> 172.21.15.12:0/3491778060 conn(0x56156567cba0 msgr2=0x56156567dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:00.086 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.084+0000 7f655a035740 1 -- 172.21.15.12:0/3491778060 shutdown_connections 2024-04-22T13:33:00.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.084+0000 7f655a035740 1 -- 172.21.15.12:0/3491778060 wait complete. 2024-04-22T13:33:00.087 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.084+0000 7f655a035740 20 client.18717 trim_cache size 0 max 16384 2024-04-22T13:33:00.087 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:33:00.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.085+0000 7f6557443640 1 -- 172.21.15.12:0/1008389916 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f653c0a3d20 msgr2=0x7f653c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:00.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.085+0000 7f6557443640 1 --2- 172.21.15.12:0/1008389916 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f653c0a3d20 0x7f653c0a61e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f654809c630 tx=0x7f654806e370 comp rx=0 tx=0).stop 2024-04-22T13:33:00.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.085+0000 7f6557443640 1 -- 172.21.15.12:0/1008389916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6550107030 msgr2=0x7f65501a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:00.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.085+0000 7f6557443640 1 --2- 172.21.15.12:0/1008389916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6550107030 0x7f65501a4620 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f654800eaa0 tx=0x7f654800ef60 comp rx=0 tx=0).stop 2024-04-22T13:33:00.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.085+0000 7f6557443640 1 -- 172.21.15.12:0/1008389916 shutdown_connections 2024-04-22T13:33:00.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.085+0000 7f6557443640 1 --2- 172.21.15.12:0/1008389916 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f653c0a3d20 0x7f653c0a61e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.085+0000 7f6557443640 1 --2- 172.21.15.12:0/1008389916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6550107030 0x7f65501a4620 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.085+0000 7f6557443640 1 --2- 172.21.15.12:0/1008389916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6550101b40 0x7f6550199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.085+0000 7f6557443640 1 --2- 172.21.15.12:0/1008389916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65501011a0 0x7f65501995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.088 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.085+0000 7f6557443640 1 -- 172.21.15.12:0/1008389916 >> 172.21.15.12:0/1008389916 conn(0x7f6550076c50 msgr2=0x7f65500fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:00.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.085+0000 7f6557443640 1 -- 172.21.15.12:0/1008389916 shutdown_connections 2024-04-22T13:33:00.089 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.085+0000 7f6557443640 1 -- 172.21.15.12:0/1008389916 wait complete. 2024-04-22T13:33:00.091 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:33:00.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.199+0000 7f7b0a606640 1 -- 172.21.15.12:0/2650467433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b0406b1f0 msgr2=0x7f7b04106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:00.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.199+0000 7f7b0a606640 1 --2- 172.21.15.12:0/2650467433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b0406b1f0 0x7f7b04106b90 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009900 tx=0x7f7aec02bc30 comp rx=0 tx=0).stop 2024-04-22T13:33:00.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.199+0000 7f7b0a606640 1 -- 172.21.15.12:0/2650467433 shutdown_connections 2024-04-22T13:33:00.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.199+0000 7f7b0a606640 1 --2- 172.21.15.12:0/2650467433 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b04107270 0x7f7b0410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.199+0000 7f7b0a606640 1 --2- 172.21.15.12:0/2650467433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b0406b1f0 0x7f7b04106b90 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.199+0000 7f7b0a606640 1 --2- 172.21.15.12:0/2650467433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b0406a840 0x7f7b0406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.199+0000 7f7b0a606640 1 -- 172.21.15.12:0/2650467433 >> 172.21.15.12:0/2650467433 conn(0x7f7b04077eb0 msgr2=0x7f7b040782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:00.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.199+0000 7f7b0a606640 1 -- 172.21.15.12:0/2650467433 shutdown_connections 2024-04-22T13:33:00.195 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.199+0000 7f7b0a606640 1 -- 172.21.15.12:0/2650467433 wait complete. 2024-04-22T13:33:00.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.200+0000 7f7b0a606640 1 Processor -- start 2024-04-22T13:33:00.196 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.200+0000 7f7b0a606640 1 -- start start 2024-04-22T13:33:00.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.200+0000 7f7b0a606640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b0406a840 0x7f7b041a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.200+0000 7f7b0a606640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b04107270 0x7f7b041a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.200+0000 7f7b0a606640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b0419a810 0x7f7b0419acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.200+0000 7f7b0a606640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041121b0 con 0x7f7b0419a810 2024-04-22T13:33:00.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.200+0000 7f7b0a606640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7b04112030 con 0x7f7b0406a840 2024-04-22T13:33:00.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.200+0000 7f7b0a606640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7b04112330 con 0x7f7b04107270 2024-04-22T13:33:00.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.200+0000 7f7b08ba3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b0419a810 0x7f7b0419acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.200+0000 7f7b08ba3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b0419a810 0x7f7b0419acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51512/0 (socket says 172.21.15.12:51512) 2024-04-22T13:33:00.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.200+0000 7f7b08ba3640 1 -- 172.21.15.12:0/789735178 learned_addr learned my addr 172.21.15.12:0/789735178 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:00.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.201+0000 7f7b03fff640 1 --2- 172.21.15.12:0/789735178 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b0406a840 0x7f7b041a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.201+0000 7f7b037fe640 1 --2- 172.21.15.12:0/789735178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b04107270 0x7f7b041a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.201+0000 7f7b08ba3640 1 -- 172.21.15.12:0/789735178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b04107270 msgr2=0x7f7b041a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:00.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.201+0000 7f7b08ba3640 1 --2- 172.21.15.12:0/789735178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b04107270 0x7f7b041a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.201+0000 7f7b08ba3640 1 -- 172.21.15.12:0/789735178 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b0406a840 msgr2=0x7f7b041a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:00.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.201+0000 7f7b08ba3640 1 --2- 172.21.15.12:0/789735178 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b0406a840 0x7f7b041a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.201+0000 7f7b08ba3640 1 -- 172.21.15.12:0/789735178 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b0419b3a0 con 0x7f7b0419a810 2024-04-22T13:33:00.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.201+0000 7f7b037fe640 1 --2- 172.21.15.12:0/789735178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b04107270 0x7f7b041a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:00.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.201+0000 7f7b08ba3640 1 --2- 172.21.15.12:0/789735178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b0419a810 0x7f7b0419acc0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00cad0 tx=0x7f7afc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:00.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.201+0000 7f7b017fa640 1 -- 172.21.15.12:0/789735178 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc00ae20 con 0x7f7b0419a810 2024-04-22T13:33:00.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.201+0000 7f7b017fa640 1 -- 172.21.15.12:0/789735178 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7afc005020 con 0x7f7b0419a810 2024-04-22T13:33:00.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.201+0000 7f7b0a606640 1 -- 172.21.15.12:0/789735178 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b0419b690 con 0x7f7b0419a810 2024-04-22T13:33:00.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.201+0000 7f7b017fa640 1 -- 172.21.15.12:0/789735178 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc00f6b0 con 0x7f7b0419a810 2024-04-22T13:33:00.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.201+0000 7f7b0a606640 1 -- 172.21.15.12:0/789735178 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b041a7840 con 0x7f7b0419a810 2024-04-22T13:33:00.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.204+0000 7f7b017fa640 1 -- 172.21.15.12:0/789735178 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7afc0051d0 con 0x7f7b0419a810 2024-04-22T13:33:00.207 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.204+0000 7f7b0d1f3740 1 Processor -- start 2024-04-22T13:33:00.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.204+0000 7f7b0d1f3740 1 -- start start 2024-04-22T13:33:00.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.204+0000 7f7b0d1f3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630281ca4d0 0x5630281ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.204+0000 7f7b0d1f3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630281cae80 0x5630281d3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.204+0000 7f7b0d1f3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630281d3a00 0x5630281d5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.204+0000 7f7b0d1f3740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56302812f1f0 con 0x5630281ca4d0 2024-04-22T13:33:00.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.204+0000 7f7b0d1f3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5630280cc4b0 con 0x5630281d3a00 2024-04-22T13:33:00.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.204+0000 7f7b0d1f3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56302812b1b0 con 0x5630281cae80 2024-04-22T13:33:00.208 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.204+0000 7f7b037fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630281ca4d0 0x5630281ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.204+0000 7f7b037fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630281ca4d0 0x5630281ca8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51524/0 (socket says 172.21.15.12:51524) 2024-04-22T13:33:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.204+0000 7f7b037fe640 1 -- 172.21.15.12:0/4134905135 learned_addr learned my addr 172.21.15.12:0/4134905135 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.204+0000 7f7b03fff640 1 --2- 172.21.15.12:0/4134905135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630281cae80 0x5630281d3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.204+0000 7f7b037fe640 1 --2- 172.21.15.12:0/4134905135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630281d3a00 0x5630281d5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.204+0000 7f7b017fa640 1 --2- 172.21.15.12:0/789735178 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7af00a3da0 0x7f7af00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.204+0000 7f7b037fe640 1 -- 172.21.15.12:0/4134905135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630281cae80 msgr2=0x5630281d3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.204+0000 7f7b037fe640 1 --2- 172.21.15.12:0/4134905135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630281cae80 0x5630281d3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.204+0000 7f7b037fe640 1 -- 172.21.15.12:0/4134905135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630281d3a00 msgr2=0x5630281d5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.204+0000 7f7b037fe640 1 --2- 172.21.15.12:0/4134905135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630281d3a00 0x5630281d5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.204+0000 7f7b037fe640 1 -- 172.21.15.12:0/4134905135 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630280be1b0 con 0x5630281ca4d0 2024-04-22T13:33:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.204+0000 7f7b03fff640 1 --2- 172.21.15.12:0/4134905135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630281cae80 0x5630281d3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:00.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.204+0000 7f7b08ba3640 1 --2- 172.21.15.12:0/789735178 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7af00a3da0 0x7f7af00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.204+0000 7f7b037fe640 1 --2- 172.21.15.12:0/4134905135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630281ca4d0 0x5630281ca8b0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f7aec02c780 tx=0x7f7aec009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea4802f2928e182 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.205+0000 7f7b017fa640 1 -- 172.21.15.12:0/789735178 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f7afc0d07a0 con 0x7f7b0419a810 2024-04-22T13:33:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.205+0000 7f7afa7fc640 1 -- 172.21.15.12:0/4134905135 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec02c9e0 con 0x5630281ca4d0 2024-04-22T13:33:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.205+0000 7f7afa7fc640 1 -- 172.21.15.12:0/4134905135 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aec048070 con 0x5630281ca4d0 2024-04-22T13:33:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.205+0000 7f7afa7fc640 1 -- 172.21.15.12:0/4134905135 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec004ab0 con 0x5630281ca4d0 2024-04-22T13:33:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.205+0000 7f7b0d1f3740 1 -- 172.21.15.12:0/4134905135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630281ca4d0 msgr2=0x5630281ca8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.205+0000 7f7b08ba3640 1 --2- 172.21.15.12:0/789735178 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7af00a3da0 0x7f7af00a6260 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00a6e0 tx=0x7f7afc087040 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.205+0000 7f7b0d1f3740 1 --2- 172.21.15.12:0/4134905135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630281ca4d0 0x5630281ca8b0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f7aec02c780 tx=0x7f7aec009560 comp rx=0 tx=0).stop 2024-04-22T13:33:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.205+0000 7f7b0d1f3740 1 -- 172.21.15.12:0/4134905135 shutdown_connections 2024-04-22T13:33:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.205+0000 7f7b0d1f3740 1 --2- 172.21.15.12:0/4134905135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630281d3a00 0x5630281d5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.205+0000 7f7b0d1f3740 1 --2- 172.21.15.12:0/4134905135 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630281cae80 0x5630281d3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.205+0000 7f7b0d1f3740 1 --2- 172.21.15.12:0/4134905135 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630281ca4d0 0x5630281ca8b0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.205+0000 7f7b0d1f3740 1 -- 172.21.15.12:0/4134905135 >> 172.21.15.12:0/4134905135 conn(0x5630281c4ba0 msgr2=0x5630281c4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.206+0000 7f7b0d1f3740 1 -- 172.21.15.12:0/4134905135 shutdown_connections 2024-04-22T13:33:00.211 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.206+0000 7f7b0d1f3740 1 -- 172.21.15.12:0/4134905135 wait complete. 2024-04-22T13:33:00.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.206+0000 7f7b0d1f3740 1 -- start start 2024-04-22T13:33:00.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.206+0000 7f7b0d1f3740 1 Processor -- start 2024-04-22T13:33:00.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.206+0000 7f7b0d1f3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630281ca4d0 0x5630281d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.206+0000 7f7b0d1f3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630281cae80 0x5630281d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.206+0000 7f7b0d1f3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630281d3a00 0x5630281d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.206+0000 7f7b0d1f3740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5630281d6910 con 0x5630281d3a00 2024-04-22T13:33:00.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.206+0000 7f7b0d1f3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5630280e8500 con 0x5630281ca4d0 2024-04-22T13:33:00.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.206+0000 7f7b0d1f3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5630281d6a90 con 0x5630281cae80 2024-04-22T13:33:00.212 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.206+0000 7f7b0d1f3740 10 client.0 _ll_register_callbacks cb 0x5630280b6b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:00.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.206+0000 7f7b037fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630281d3a00 0x5630281d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.207+0000 7f7b037fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630281d3a00 0x5630281d0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51536/0 (socket says 172.21.15.12:51536) 2024-04-22T13:33:00.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.207+0000 7f7b037fe640 1 -- 172.21.15.12:0/3148025961 learned_addr learned my addr 172.21.15.12:0/3148025961 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:00.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.207+0000 7f7b03fff640 1 --2- 172.21.15.12:0/3148025961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630281cae80 0x5630281d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.207+0000 7f7b037fe640 1 --2- 172.21.15.12:0/3148025961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630281ca4d0 0x5630281d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.207+0000 7f7b037fe640 1 -- 172.21.15.12:0/3148025961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630281cae80 msgr2=0x5630281d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:00.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.207+0000 7f7b037fe640 1 --2- 172.21.15.12:0/3148025961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630281cae80 0x5630281d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.207+0000 7f7b037fe640 1 -- 172.21.15.12:0/3148025961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630281ca4d0 msgr2=0x5630281d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:00.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.207+0000 7f7b037fe640 1 --2- 172.21.15.12:0/3148025961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630281ca4d0 0x5630281d3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.207+0000 7f7b037fe640 1 -- 172.21.15.12:0/3148025961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563027e413f0 con 0x5630281d3a00 2024-04-22T13:33:00.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.207+0000 7f7b037fe640 1 --2- 172.21.15.12:0/3148025961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630281d3a00 0x5630281d0d30 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f7aec02c320 tx=0x7f7aec02c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392ca1a95da13571 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:00.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.207+0000 7f7ae37fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:33:00.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.207+0000 7f7ae37fe640 1 -- 172.21.15.12:0/3148025961 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec054070 con 0x5630281d3a00 2024-04-22T13:33:00.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.207+0000 7f7ae37fe640 1 -- 172.21.15.12:0/3148025961 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f7aec005550 con 0x5630281d3a00 2024-04-22T13:33:00.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.207+0000 7f7b0d1f3740 1 -- 172.21.15.12:0/3148025961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7aec048070 con 0x5630281d3a00 2024-04-22T13:33:00.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.207+0000 7f7ae37fe640 1 -- 172.21.15.12:0/3148025961 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7aec004830 con 0x5630281d3a00 2024-04-22T13:33:00.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.207+0000 7f7b0d1f3740 10 client.18729 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:00.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.207+0000 7f7b0d1f3740 1 -- 172.21.15.12:0/3148025961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563027f0b7d0 con 0x5630281d3a00 2024-04-22T13:33:00.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.208+0000 7f7ae37fe640 1 -- 172.21.15.12:0/3148025961 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aec048070 con 0x5630281d3a00 2024-04-22T13:33:00.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.208+0000 7f7ae37fe640 1 -- 172.21.15.12:0/3148025961 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7aec043a50 con 0x5630281d3a00 2024-04-22T13:33:00.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.208+0000 7f7b0d1f3740 10 client.18729 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:00.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.208+0000 7f7b0d1f3740 10 client.18729 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:33:00.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.208+0000 7f7b0d1f3740 20 client.18729 populate_metadata read hostname 'smithi012' 2024-04-22T13:33:00.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.208+0000 7f7b0d1f3740 1 --2- 172.21.15.12:0/3148025961 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5630281e3ac0 0x5630281e5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.208+0000 7f7b0d1f3740 4 client.18729 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:33:00.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.208+0000 7f7b0d1f3740 1 -- 172.21.15.12:0/3148025961 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630280ce930 con 0x5630281e3ac0 2024-04-22T13:33:00.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.208+0000 7f7b03fff640 1 --2- 172.21.15.12:0/3148025961 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5630281e3ac0 0x5630281e5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.208+0000 7f7b03fff640 1 --2- 172.21.15.12:0/3148025961 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5630281e3ac0 0x5630281e5eb0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6240382a434f672a server_cookie=7d20e3fb002d2026 in_seq=0 out_seq=0 2024-04-22T13:33:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.208+0000 7f7ae37fe640 10 client.18729 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:33:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.209+0000 7f7ae37fe640 1 -- 172.21.15.12:0/3148025961 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5630280ce930 con 0x5630281e3ac0 2024-04-22T13:33:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.209+0000 7f7ae37fe640 10 client.18729 handle_command_reply: tid=0 2024-04-22T13:33:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.209+0000 7f7b0d1f3740 1 client.18729 shutdown 2024-04-22T13:33:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.209+0000 7f7b0d1f3740 1 -- 172.21.15.12:0/3148025961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630281d3a00 msgr2=0x5630281d0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.209+0000 7f7b0d1f3740 1 --2- 172.21.15.12:0/3148025961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630281d3a00 0x5630281d0d30 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f7aec02c320 tx=0x7f7aec02c2b0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.209+0000 7f7b0d1f3740 1 -- 172.21.15.12:0/3148025961 shutdown_connections 2024-04-22T13:33:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.209+0000 7f7b0d1f3740 1 --2- 172.21.15.12:0/3148025961 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5630281e3ac0 0x5630281e5eb0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.209+0000 7f7b0d1f3740 1 --2- 172.21.15.12:0/3148025961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630281d3a00 0x5630281d0d30 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.209+0000 7f7b0d1f3740 1 --2- 172.21.15.12:0/3148025961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630281cae80 0x5630281d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.209+0000 7f7b0d1f3740 1 --2- 172.21.15.12:0/3148025961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630281ca4d0 0x5630281d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.209+0000 7f7b0d1f3740 1 -- 172.21.15.12:0/3148025961 >> 172.21.15.12:0/3148025961 conn(0x5630281c4ba0 msgr2=0x5630281c68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.209+0000 7f7ae37fe640 0 client.18729 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:33:00.215 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.210+0000 7f7b0d1f3740 1 -- 172.21.15.12:0/3148025961 shutdown_connections 2024-04-22T13:33:00.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.210+0000 7f7b0d1f3740 1 -- 172.21.15.12:0/3148025961 wait complete. 2024-04-22T13:33:00.216 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.210+0000 7f7b0d1f3740 20 client.18729 trim_cache size 0 max 16384 2024-04-22T13:33:00.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.223+0000 7f7b0d1f3740 1 Processor -- start 2024-04-22T13:33:00.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.223+0000 7f7b0d1f3740 1 -- start start 2024-04-22T13:33:00.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.223+0000 7f7b0d1f3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630281ca4d0 0x5630281c3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.223+0000 7f7b0d1f3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630281cae80 0x5630281d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.223+0000 7f7b0d1f3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630281d3a00 0x5630281d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.223+0000 7f7b0d1f3740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041afc20 con 0x5630281cae80 2024-04-22T13:33:00.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.223+0000 7f7b0d1f3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041b1760 con 0x5630281ca4d0 2024-04-22T13:33:00.220 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.223+0000 7f7b0d1f3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041b02f0 con 0x5630281d3a00 2024-04-22T13:33:00.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.223+0000 7f7b03fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630281cae80 0x5630281d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.223+0000 7f7b03fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630281cae80 0x5630281d2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51548/0 (socket says 172.21.15.12:51548) 2024-04-22T13:33:00.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.223+0000 7f7b03fff640 1 -- 172.21.15.12:0/1062771467 learned_addr learned my addr 172.21.15.12:0/1062771467 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:00.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.223+0000 7f7b037fe640 1 --2- 172.21.15.12:0/1062771467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630281ca4d0 0x5630281c3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.223+0000 7f7b03fff640 1 -- 172.21.15.12:0/1062771467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630281d3a00 msgr2=0x5630281d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:00.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.224+0000 7f7b03fff640 1 --2- 172.21.15.12:0/1062771467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630281d3a00 0x5630281d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.224+0000 7f7b03fff640 1 -- 172.21.15.12:0/1062771467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630281ca4d0 msgr2=0x5630281c3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:00.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.224+0000 7f7b03fff640 1 --2- 172.21.15.12:0/1062771467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630281ca4d0 0x5630281c3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.224+0000 7f7b03fff640 1 -- 172.21.15.12:0/1062771467 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630281cecd0 con 0x5630281cae80 2024-04-22T13:33:00.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.224+0000 7f7b03fff640 1 --2- 172.21.15.12:0/1062771467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630281cae80 0x5630281d2e50 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x5630281c9fa0 tx=0x7f7af400bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769016fc016b5ab4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:00.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.224+0000 7f7af9ffb640 1 -- 172.21.15.12:0/1062771467 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af4003e40 con 0x5630281cae80 2024-04-22T13:33:00.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.224+0000 7f7af9ffb640 1 -- 172.21.15.12:0/1062771467 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7af401d920 con 0x5630281cae80 2024-04-22T13:33:00.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.224+0000 7f7af9ffb640 1 -- 172.21.15.12:0/1062771467 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af4004b20 con 0x5630281cae80 2024-04-22T13:33:00.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.224+0000 7f7b0d1f3740 1 -- 172.21.15.12:0/1062771467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630281cae80 msgr2=0x5630281d2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:00.221 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.224+0000 7f7b0d1f3740 1 --2- 172.21.15.12:0/1062771467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630281cae80 0x5630281d2e50 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x5630281c9fa0 tx=0x7f7af400bf30 comp rx=0 tx=0).stop 2024-04-22T13:33:00.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.225+0000 7f7b0d1f3740 1 -- 172.21.15.12:0/1062771467 shutdown_connections 2024-04-22T13:33:00.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.225+0000 7f7b0d1f3740 1 --2- 172.21.15.12:0/1062771467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630281d3a00 0x5630281d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.225+0000 7f7b0d1f3740 1 --2- 172.21.15.12:0/1062771467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630281cae80 0x5630281d2e50 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.225+0000 7f7b0d1f3740 1 --2- 172.21.15.12:0/1062771467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630281ca4d0 0x5630281c3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.225+0000 7f7b0d1f3740 1 -- 172.21.15.12:0/1062771467 >> 172.21.15.12:0/1062771467 conn(0x5630281c4ba0 msgr2=0x5630281cf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:00.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.225+0000 7f7b0d1f3740 1 -- 172.21.15.12:0/1062771467 shutdown_connections 2024-04-22T13:33:00.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.225+0000 7f7b0d1f3740 1 -- 172.21.15.12:0/1062771467 wait complete. 2024-04-22T13:33:00.222 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.225+0000 7f7b0d1f3740 1 -- start start 2024-04-22T13:33:00.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.225+0000 7f7b0d1f3740 1 Processor -- start 2024-04-22T13:33:00.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.225+0000 7f7b0d1f3740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630281ca4d0 0x5630281e6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.225+0000 7f7b0d1f3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630281cae80 0x5630281e6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.225+0000 7f7b0d1f3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630281d3a00 0x5630281e72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.225+0000 7f7b0d1f3740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041b1090 con 0x5630281ca4d0 2024-04-22T13:33:00.226 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.225+0000 7f7b0d1f3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041b09c0 con 0x5630281d3a00 2024-04-22T13:33:00.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.225+0000 7f7b0d1f3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041adc20 con 0x5630281cae80 2024-04-22T13:33:00.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.225+0000 7f7b0d1f3740 10 client.0 _ll_register_callbacks cb 0x7f7aec02ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:00.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.225+0000 7f7b037fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630281ca4d0 0x5630281e6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.225+0000 7f7b037fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630281ca4d0 0x5630281e6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51552/0 (socket says 172.21.15.12:51552) 2024-04-22T13:33:00.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.226+0000 7f7b037fe640 1 -- 172.21.15.12:0/2743335086 learned_addr learned my addr 172.21.15.12:0/2743335086 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:00.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.226+0000 7f7b037fe640 1 --2- 172.21.15.12:0/2743335086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630281d3a00 0x5630281e72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.226+0000 7f7b03fff640 1 --2- 172.21.15.12:0/2743335086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630281cae80 0x5630281e6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.226+0000 7f7b037fe640 1 -- 172.21.15.12:0/2743335086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630281cae80 msgr2=0x5630281e6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:00.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.226+0000 7f7b037fe640 1 --2- 172.21.15.12:0/2743335086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630281cae80 0x5630281e6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.226+0000 7f7b037fe640 1 -- 172.21.15.12:0/2743335086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630281d3a00 msgr2=0x5630281e72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:00.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.226+0000 7f7b037fe640 1 --2- 172.21.15.12:0/2743335086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630281d3a00 0x5630281e72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.226+0000 7f7b037fe640 1 -- 172.21.15.12:0/2743335086 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563028248ca0 con 0x5630281ca4d0 2024-04-22T13:33:00.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.226+0000 7f7b037fe640 1 --2- 172.21.15.12:0/2743335086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630281ca4d0 0x5630281e6870 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009340 tx=0x7f7aec02cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290d51612e7ed672 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:00.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.226+0000 7f7ae37fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:33:00.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.226+0000 7f7ae37fe640 1 -- 172.21.15.12:0/2743335086 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec062030 con 0x5630281ca4d0 2024-04-22T13:33:00.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.226+0000 7f7ae37fe640 1 -- 172.21.15.12:0/2743335086 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f7aec0049d0 con 0x5630281ca4d0 2024-04-22T13:33:00.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.226+0000 7f7b0d1f3740 1 -- 172.21.15.12:0/2743335086 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56302813cc10 con 0x5630281ca4d0 2024-04-22T13:33:00.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.226+0000 7f7ae37fe640 1 -- 172.21.15.12:0/2743335086 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7aec06c050 con 0x5630281ca4d0 2024-04-22T13:33:00.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.226+0000 7f7b0d1f3740 10 client.18735 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:00.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.226+0000 7f7b0d1f3740 1 -- 172.21.15.12:0/2743335086 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563028249f10 con 0x5630281ca4d0 2024-04-22T13:33:00.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.226+0000 7f7ae37fe640 1 -- 172.21.15.12:0/2743335086 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56302813cc10 con 0x5630281ca4d0 2024-04-22T13:33:00.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.227+0000 7f7ae37fe640 1 -- 172.21.15.12:0/2743335086 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7aec003e40 con 0x5630281ca4d0 2024-04-22T13:33:00.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.227+0000 7f7b0d1f3740 10 client.18735 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:00.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.227+0000 7f7b0d1f3740 10 client.18735 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:33:00.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.227+0000 7f7b0d1f3740 20 client.18735 populate_metadata read hostname 'smithi012' 2024-04-22T13:33:00.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.227+0000 7f7b0d1f3740 1 --2- 172.21.15.12:0/2743335086 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56302824a630 0x56302824ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.227+0000 7f7b0d1f3740 4 client.18735 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:33:00.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.227+0000 7f7b0d1f3740 1 -- 172.21.15.12:0/2743335086 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563028136a80 con 0x56302824a630 2024-04-22T13:33:00.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.227+0000 7f7b03fff640 1 --2- 172.21.15.12:0/2743335086 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56302824a630 0x56302824ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.227+0000 7f7ae37fe640 10 client.18735 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:33:00.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.227+0000 7f7b03fff640 1 --2- 172.21.15.12:0/2743335086 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56302824a630 0x56302824ca50 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c91a2f0f07a9a61b server_cookie=c9dd904f3ae78363 in_seq=0 out_seq=0 2024-04-22T13:33:00.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.228+0000 7f7ae37fe640 1 -- 172.21.15.12:0/2743335086 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563028136a80 con 0x56302824a630 2024-04-22T13:33:00.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.228+0000 7f7ae37fe640 10 client.18735 handle_command_reply: tid=0 2024-04-22T13:33:00.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.228+0000 7f7b0d1f3740 1 client.18735 shutdown 2024-04-22T13:33:00.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.228+0000 7f7b0d1f3740 1 -- 172.21.15.12:0/2743335086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630281ca4d0 msgr2=0x5630281e6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:00.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.228+0000 7f7b0d1f3740 1 --2- 172.21.15.12:0/2743335086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630281ca4d0 0x5630281e6870 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009340 tx=0x7f7aec02cab0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.228+0000 7f7b0d1f3740 1 -- 172.21.15.12:0/2743335086 shutdown_connections 2024-04-22T13:33:00.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.228+0000 7f7b0d1f3740 1 --2- 172.21.15.12:0/2743335086 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56302824a630 0x56302824ca50 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.228+0000 7f7b0d1f3740 1 --2- 172.21.15.12:0/2743335086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630281d3a00 0x5630281e72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.228+0000 7f7b0d1f3740 1 --2- 172.21.15.12:0/2743335086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5630281cae80 0x5630281e6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.228+0000 7f7ae37fe640 0 client.18735 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:33:00.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.228+0000 7f7b0d1f3740 1 --2- 172.21.15.12:0/2743335086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5630281ca4d0 0x5630281e6870 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.228+0000 7f7b0d1f3740 1 -- 172.21.15.12:0/2743335086 >> 172.21.15.12:0/2743335086 conn(0x5630281c4ba0 msgr2=0x5630281cef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:00.229 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.228+0000 7f7b0d1f3740 1 -- 172.21.15.12:0/2743335086 shutdown_connections 2024-04-22T13:33:00.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.228+0000 7f7b0d1f3740 1 -- 172.21.15.12:0/2743335086 wait complete. 2024-04-22T13:33:00.230 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.228+0000 7f7b0d1f3740 20 client.18735 trim_cache size 0 max 16384 2024-04-22T13:33:00.230 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:33:00.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.228+0000 7f7b0a606640 1 -- 172.21.15.12:0/789735178 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7af00a3da0 msgr2=0x7f7af00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:00.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.228+0000 7f7b0a606640 1 --2- 172.21.15.12:0/789735178 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7af00a3da0 0x7f7af00a6260 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00a6e0 tx=0x7f7afc087040 comp rx=0 tx=0).stop 2024-04-22T13:33:00.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.228+0000 7f7b0a606640 1 -- 172.21.15.12:0/789735178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b0419a810 msgr2=0x7f7b0419acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:00.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.228+0000 7f7b0a606640 1 --2- 172.21.15.12:0/789735178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b0419a810 0x7f7b0419acc0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00cad0 tx=0x7f7afc00cf90 comp rx=0 tx=0).stop 2024-04-22T13:33:00.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.228+0000 7f7b0a606640 1 -- 172.21.15.12:0/789735178 shutdown_connections 2024-04-22T13:33:00.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.228+0000 7f7b0a606640 1 --2- 172.21.15.12:0/789735178 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7af00a3da0 0x7f7af00a6260 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.228+0000 7f7b0a606640 1 --2- 172.21.15.12:0/789735178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7b0419a810 0x7f7b0419acc0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.228+0000 7f7b0a606640 1 --2- 172.21.15.12:0/789735178 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b04107270 0x7f7b041a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.228+0000 7f7b0a606640 1 --2- 172.21.15.12:0/789735178 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b0406a840 0x7f7b041a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.229+0000 7f7b0a606640 1 -- 172.21.15.12:0/789735178 >> 172.21.15.12:0/789735178 conn(0x7f7b04077eb0 msgr2=0x7f7b040fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:00.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.229+0000 7f7b0a606640 1 -- 172.21.15.12:0/789735178 shutdown_connections 2024-04-22T13:33:00.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.229+0000 7f7b0a606640 1 -- 172.21.15.12:0/789735178 wait complete. 2024-04-22T13:33:00.234 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:33:00.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.344+0000 7fe41297e640 1 -- 172.21.15.12:0/144728469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe40c1031d0 msgr2=0x7fe40c1035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:00.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.344+0000 7fe41297e640 1 --2- 172.21.15.12:0/144728469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe40c1031d0 0x7fe40c1035b0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc009900 tx=0x7fe3fc02bd60 comp rx=0 tx=0).stop 2024-04-22T13:33:00.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.344+0000 7fe41297e640 1 -- 172.21.15.12:0/144728469 shutdown_connections 2024-04-22T13:33:00.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.344+0000 7fe41297e640 1 --2- 172.21.15.12:0/144728469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe40c10c510 0x7fe40c10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.344+0000 7fe41297e640 1 --2- 172.21.15.12:0/144728469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe40c103af0 0x7fe40c10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.344+0000 7fe41297e640 1 --2- 172.21.15.12:0/144728469 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe40c1031d0 0x7fe40c1035b0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.344+0000 7fe41297e640 1 -- 172.21.15.12:0/144728469 >> 172.21.15.12:0/144728469 conn(0x7fe40c077f50 msgr2=0x7fe40c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:00.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.344+0000 7fe41297e640 1 -- 172.21.15.12:0/144728469 shutdown_connections 2024-04-22T13:33:00.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.344+0000 7fe41297e640 1 -- 172.21.15.12:0/144728469 wait complete. 2024-04-22T13:33:00.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.345+0000 7fe41297e640 1 Processor -- start 2024-04-22T13:33:00.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.345+0000 7fe41297e640 1 -- start start 2024-04-22T13:33:00.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.345+0000 7fe41297e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe40c1031d0 0x7fe40c19e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.345+0000 7fe41297e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe40c103af0 0x7fe40c19e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.345+0000 7fe41297e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe40c10c510 0x7fe40c198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.345+0000 7fe41297e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c112200 con 0x7fe40c103af0 2024-04-22T13:33:00.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.345+0000 7fe41297e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c112080 con 0x7fe40c10c510 2024-04-22T13:33:00.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.345+0000 7fe41297e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c112380 con 0x7fe40c1031d0 2024-04-22T13:33:00.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.346+0000 7fe40b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe40c103af0 0x7fe40c19e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.346+0000 7fe40b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe40c103af0 0x7fe40c19e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51568/0 (socket says 172.21.15.12:51568) 2024-04-22T13:33:00.343 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.346+0000 7fe40b7fe640 1 -- 172.21.15.12:0/3493246568 learned_addr learned my addr 172.21.15.12:0/3493246568 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:00.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.346+0000 7fe40bfff640 1 --2- 172.21.15.12:0/3493246568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe40c1031d0 0x7fe40c19e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.346+0000 7fe410f1b640 1 --2- 172.21.15.12:0/3493246568 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe40c10c510 0x7fe40c198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.346+0000 7fe40b7fe640 1 -- 172.21.15.12:0/3493246568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe40c1031d0 msgr2=0x7fe40c19e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:00.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.346+0000 7fe40b7fe640 1 --2- 172.21.15.12:0/3493246568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe40c1031d0 0x7fe40c19e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.346+0000 7fe40b7fe640 1 -- 172.21.15.12:0/3493246568 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe40c10c510 msgr2=0x7fe40c198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:00.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.346+0000 7fe40b7fe640 1 --2- 172.21.15.12:0/3493246568 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe40c10c510 0x7fe40c198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.346+0000 7fe40b7fe640 1 -- 172.21.15.12:0/3493246568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe40c198d80 con 0x7fe40c103af0 2024-04-22T13:33:00.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.346+0000 7fe410f1b640 1 --2- 172.21.15.12:0/3493246568 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe40c10c510 0x7fe40c198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:00.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.346+0000 7fe40b7fe640 1 --2- 172.21.15.12:0/3493246568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe40c103af0 0x7fe40c19e990 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fe40000c920 tx=0x7fe40000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:00.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.346+0000 7fe4097fa640 1 -- 172.21.15.12:0/3493246568 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe400007c70 con 0x7fe40c103af0 2024-04-22T13:33:00.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.346+0000 7fe4097fa640 1 -- 172.21.15.12:0/3493246568 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe400007e10 con 0x7fe40c103af0 2024-04-22T13:33:00.344 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.346+0000 7fe41297e640 1 -- 172.21.15.12:0/3493246568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe40c199070 con 0x7fe40c103af0 2024-04-22T13:33:00.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.346+0000 7fe4097fa640 1 -- 172.21.15.12:0/3493246568 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe400018820 con 0x7fe40c103af0 2024-04-22T13:33:00.345 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.346+0000 7fe41297e640 1 -- 172.21.15.12:0/3493246568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe40c1a5310 con 0x7fe40c103af0 2024-04-22T13:33:00.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.348+0000 7fe4097fa640 1 -- 172.21.15.12:0/3493246568 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe40000f450 con 0x7fe40c103af0 2024-04-22T13:33:00.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.349+0000 7fe415569740 1 Processor -- start 2024-04-22T13:33:00.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.349+0000 7fe415569740 1 -- start start 2024-04-22T13:33:00.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.349+0000 7fe415569740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614efea24d0 0x5614efea28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.352 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.349+0000 7fe415569740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614efea2e80 0x5614efeab430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.349+0000 7fe415569740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efeaba00 0x5614efeade50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.349+0000 7fe415569740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5614efe071f0 con 0x5614efeaba00 2024-04-22T13:33:00.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.349+0000 7fe415569740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5614efda44b0 con 0x5614efea2e80 2024-04-22T13:33:00.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.349+0000 7fe415569740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5614efe031b0 con 0x5614efea24d0 2024-04-22T13:33:00.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.349+0000 7fe40b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efeaba00 0x5614efeade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.349+0000 7fe4097fa640 1 --2- 172.21.15.12:0/3493246568 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe3f40a3d20 0x7fe3f40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.349+0000 7fe40b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efeaba00 0x5614efeade50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51572/0 (socket says 172.21.15.12:51572) 2024-04-22T13:33:00.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.349+0000 7fe40b7fe640 1 -- 172.21.15.12:0/2846758324 learned_addr learned my addr 172.21.15.12:0/2846758324 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:00.353 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.349+0000 7fe410f1b640 1 --2- 172.21.15.12:0/2846758324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614efea24d0 0x5614efea28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.349+0000 7fe40bfff640 1 --2- 172.21.15.12:0/2846758324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614efea2e80 0x5614efeab430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.349+0000 7fe410f1b640 1 --2- 172.21.15.12:0/3493246568 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe3f40a3d20 0x7fe3f40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.350+0000 7fe4097fa640 1 -- 172.21.15.12:0/3493246568 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7fe4000d0930 con 0x7fe40c103af0 2024-04-22T13:33:00.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.350+0000 7fe410f1b640 1 -- 172.21.15.12:0/2846758324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614efea2e80 msgr2=0x5614efeab430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:00.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.350+0000 7fe410f1b640 1 --2- 172.21.15.12:0/2846758324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614efea2e80 0x5614efeab430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.350+0000 7fe410f1b640 1 -- 172.21.15.12:0/2846758324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efeaba00 msgr2=0x5614efeade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:00.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.350+0000 7fe410f1b640 1 --2- 172.21.15.12:0/2846758324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efeaba00 0x5614efeade50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.350+0000 7fe410f1b640 1 -- 172.21.15.12:0/2846758324 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614efd961b0 con 0x5614efea24d0 2024-04-22T13:33:00.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.350+0000 7fe40b7fe640 1 --2- 172.21.15.12:0/2846758324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efeaba00 0x5614efeade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:33:00.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.350+0000 7fe410f1b640 1 --2- 172.21.15.12:0/3493246568 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe3f40a3d20 0x7fe3f40a61e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe404016b40 tx=0x7fe40401a000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:00.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.350+0000 7fe410f1b640 1 --2- 172.21.15.12:0/2846758324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614efea24d0 0x5614efea28b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe40400aee0 tx=0x7fe404016640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50ffd32ad303651d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:00.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.350+0000 7fe3f27fc640 1 -- 172.21.15.12:0/2846758324 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe404026070 con 0x5614efea24d0 2024-04-22T13:33:00.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.350+0000 7fe3f27fc640 1 -- 172.21.15.12:0/2846758324 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe404009c90 con 0x5614efea24d0 2024-04-22T13:33:00.354 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.350+0000 7fe3f27fc640 1 -- 172.21.15.12:0/2846758324 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe404013040 con 0x5614efea24d0 2024-04-22T13:33:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.351+0000 7fe415569740 1 -- 172.21.15.12:0/2846758324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614efea24d0 msgr2=0x5614efea28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.351+0000 7fe415569740 1 --2- 172.21.15.12:0/2846758324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614efea24d0 0x5614efea28b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe40400aee0 tx=0x7fe404016640 comp rx=0 tx=0).stop 2024-04-22T13:33:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.351+0000 7fe415569740 1 -- 172.21.15.12:0/2846758324 shutdown_connections 2024-04-22T13:33:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.351+0000 7fe415569740 1 --2- 172.21.15.12:0/2846758324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efeaba00 0x5614efeade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.351+0000 7fe415569740 1 --2- 172.21.15.12:0/2846758324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614efea2e80 0x5614efeab430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.351+0000 7fe415569740 1 --2- 172.21.15.12:0/2846758324 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614efea24d0 0x5614efea28b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.351+0000 7fe415569740 1 -- 172.21.15.12:0/2846758324 >> 172.21.15.12:0/2846758324 conn(0x5614efe9cba0 msgr2=0x5614efe9cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.351+0000 7fe415569740 1 -- 172.21.15.12:0/2846758324 shutdown_connections 2024-04-22T13:33:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.351+0000 7fe415569740 1 -- 172.21.15.12:0/2846758324 wait complete. 2024-04-22T13:33:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.351+0000 7fe415569740 1 -- start start 2024-04-22T13:33:00.355 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.351+0000 7fe415569740 1 Processor -- start 2024-04-22T13:33:00.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.352+0000 7fe415569740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efea24d0 0x5614efeab2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.352+0000 7fe415569740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614efea2e80 0x5614efea8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.352+0000 7fe415569740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614efeaba00 0x5614efea8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.352+0000 7fe415569740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5614efeae910 con 0x5614efea24d0 2024-04-22T13:33:00.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.352+0000 7fe415569740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5614efdc0500 con 0x5614efeaba00 2024-04-22T13:33:00.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.352+0000 7fe415569740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5614efeaea90 con 0x5614efea2e80 2024-04-22T13:33:00.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.352+0000 7fe415569740 10 client.0 _ll_register_callbacks cb 0x5614efd8eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:00.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.352+0000 7fe40bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efea24d0 0x5614efeab2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.352+0000 7fe40bfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efea24d0 0x5614efeab2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51586/0 (socket says 172.21.15.12:51586) 2024-04-22T13:33:00.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.352+0000 7fe40bfff640 1 -- 172.21.15.12:0/752900950 learned_addr learned my addr 172.21.15.12:0/752900950 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:00.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.352+0000 7fe40b7fe640 1 --2- 172.21.15.12:0/752900950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614efea2e80 0x5614efea8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.352+0000 7fe410f1b640 1 --2- 172.21.15.12:0/752900950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614efeaba00 0x5614efea8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.352+0000 7fe40bfff640 1 -- 172.21.15.12:0/752900950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614efea2e80 msgr2=0x5614efea8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.352+0000 7fe40bfff640 1 --2- 172.21.15.12:0/752900950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614efea2e80 0x5614efea8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.352+0000 7fe40bfff640 1 -- 172.21.15.12:0/752900950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614efeaba00 msgr2=0x5614efea8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.352+0000 7fe40bfff640 1 --2- 172.21.15.12:0/752900950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614efeaba00 0x5614efea8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.352+0000 7fe40bfff640 1 -- 172.21.15.12:0/752900950 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614efb193f0 con 0x5614efea24d0 2024-04-22T13:33:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.352+0000 7fe40b7fe640 1 --2- 172.21.15.12:0/752900950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614efea2e80 0x5614efea8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:33:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.352+0000 7fe40bfff640 1 --2- 172.21.15.12:0/752900950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efea24d0 0x5614efeab2d0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc02c2a0 tx=0x7fe3fc02dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a505666f76fd6131 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.352+0000 7fe3eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:33:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.352+0000 7fe3eb7fe640 1 -- 172.21.15.12:0/752900950 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc044030 con 0x5614efea24d0 2024-04-22T13:33:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.353+0000 7fe415569740 1 -- 172.21.15.12:0/752900950 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5614efda6930 con 0x5614efea24d0 2024-04-22T13:33:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.353+0000 7fe3eb7fe640 1 -- 172.21.15.12:0/752900950 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7fe3fc03a040 con 0x5614efea24d0 2024-04-22T13:33:00.357 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.353+0000 7fe3eb7fe640 1 -- 172.21.15.12:0/752900950 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe3fc005310 con 0x5614efea24d0 2024-04-22T13:33:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.353+0000 7fe415569740 10 client.18747 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.353+0000 7fe415569740 1 -- 172.21.15.12:0/752900950 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5614efbe37d0 con 0x5614efea24d0 2024-04-22T13:33:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.353+0000 7fe3eb7fe640 1 -- 172.21.15.12:0/752900950 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5614efda6930 con 0x5614efea24d0 2024-04-22T13:33:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.353+0000 7fe3eb7fe640 1 -- 172.21.15.12:0/752900950 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe3fc050020 con 0x5614efea24d0 2024-04-22T13:33:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.353+0000 7fe415569740 10 client.18747 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.353+0000 7fe415569740 10 client.18747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:33:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.353+0000 7fe415569740 20 client.18747 populate_metadata read hostname 'smithi012' 2024-04-22T13:33:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.353+0000 7fe415569740 1 --2- 172.21.15.12:0/752900950 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5614efebff70 0x5614efec2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.353+0000 7fe415569740 4 client.18747 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:33:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.353+0000 7fe415569740 1 -- 172.21.15.12:0/752900950 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614efe14c10 con 0x5614efebff70 2024-04-22T13:33:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.353+0000 7fe40b7fe640 1 --2- 172.21.15.12:0/752900950 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5614efebff70 0x5614efec2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.354+0000 7fe40b7fe640 1 --2- 172.21.15.12:0/752900950 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5614efebff70 0x5614efec2380 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6fb6e9b0e4b1948 server_cookie=833c553f4f12335e in_seq=0 out_seq=0 2024-04-22T13:33:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.354+0000 7fe3eb7fe640 10 client.18747 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:33:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.354+0000 7fe3eb7fe640 1 -- 172.21.15.12:0/752900950 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5614efe14c10 con 0x5614efebff70 2024-04-22T13:33:00.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.355+0000 7fe3eb7fe640 10 client.18747 handle_command_reply: tid=0 2024-04-22T13:33:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.355+0000 7fe415569740 1 client.18747 shutdown 2024-04-22T13:33:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.355+0000 7fe415569740 1 -- 172.21.15.12:0/752900950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efea24d0 msgr2=0x5614efeab2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.355+0000 7fe415569740 1 --2- 172.21.15.12:0/752900950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efea24d0 0x5614efeab2d0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc02c2a0 tx=0x7fe3fc02dbc0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.355+0000 7fe415569740 1 -- 172.21.15.12:0/752900950 shutdown_connections 2024-04-22T13:33:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.355+0000 7fe415569740 1 --2- 172.21.15.12:0/752900950 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5614efebff70 0x5614efec2380 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.355+0000 7fe415569740 1 --2- 172.21.15.12:0/752900950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614efeaba00 0x5614efea8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.355+0000 7fe415569740 1 --2- 172.21.15.12:0/752900950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614efea2e80 0x5614efea8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.355+0000 7fe415569740 1 --2- 172.21.15.12:0/752900950 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efea24d0 0x5614efeab2d0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.355+0000 7fe3eb7fe640 0 client.18747 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:33:00.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.355+0000 7fe415569740 1 -- 172.21.15.12:0/752900950 >> 172.21.15.12:0/752900950 conn(0x5614efe9cba0 msgr2=0x5614efe9e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:00.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.355+0000 7fe415569740 1 -- 172.21.15.12:0/752900950 shutdown_connections 2024-04-22T13:33:00.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.355+0000 7fe415569740 1 -- 172.21.15.12:0/752900950 wait complete. 2024-04-22T13:33:00.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.355+0000 7fe415569740 20 client.18747 trim_cache size 0 max 16384 2024-04-22T13:33:00.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.368+0000 7fe415569740 1 Processor -- start 2024-04-22T13:33:00.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.368+0000 7fe415569740 1 -- start start 2024-04-22T13:33:00.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.368+0000 7fe415569740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614efea24d0 0x5614efe9ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.368+0000 7fe415569740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614efea2e80 0x5614efeaad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.368+0000 7fe415569740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efeaba00 0x5614efeab2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.368+0000 7fe415569740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c1ab810 con 0x5614efeaba00 2024-04-22T13:33:00.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.368+0000 7fe415569740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c1ad350 con 0x5614efea24d0 2024-04-22T13:33:00.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.368+0000 7fe415569740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c1abee0 con 0x5614efea2e80 2024-04-22T13:33:00.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.368+0000 7fe410f1b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efeaba00 0x5614efeab2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.369+0000 7fe410f1b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efeaba00 0x5614efeab2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51590/0 (socket says 172.21.15.12:51590) 2024-04-22T13:33:00.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.369+0000 7fe410f1b640 1 -- 172.21.15.12:0/2360381234 learned_addr learned my addr 172.21.15.12:0/2360381234 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:00.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.369+0000 7fe40bfff640 1 --2- 172.21.15.12:0/2360381234 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614efea24d0 0x5614efe9ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.369+0000 7fe40b7fe640 1 --2- 172.21.15.12:0/2360381234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614efea2e80 0x5614efeaad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.369+0000 7fe410f1b640 1 -- 172.21.15.12:0/2360381234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614efea2e80 msgr2=0x5614efeaad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:00.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.369+0000 7fe410f1b640 1 --2- 172.21.15.12:0/2360381234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614efea2e80 0x5614efeaad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.369+0000 7fe410f1b640 1 -- 172.21.15.12:0/2360381234 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614efea24d0 msgr2=0x5614efe9ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:00.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.369+0000 7fe410f1b640 1 --2- 172.21.15.12:0/2360381234 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614efea24d0 0x5614efe9ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.369+0000 7fe410f1b640 1 -- 172.21.15.12:0/2360381234 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614efea2c20 con 0x5614efeaba00 2024-04-22T13:33:00.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.369+0000 7fe410f1b640 1 --2- 172.21.15.12:0/2360381234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efeaba00 0x5614efeab2d0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fe40400aee0 tx=0x7fe404016bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127b99c9976ce55b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:00.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.369+0000 7fe3f1ffb640 1 -- 172.21.15.12:0/2360381234 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe404026050 con 0x5614efeaba00 2024-04-22T13:33:00.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.369+0000 7fe3f1ffb640 1 -- 172.21.15.12:0/2360381234 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe404032070 con 0x5614efeaba00 2024-04-22T13:33:00.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.369+0000 7fe3f1ffb640 1 -- 172.21.15.12:0/2360381234 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40402db00 con 0x5614efeaba00 2024-04-22T13:33:00.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.370+0000 7fe415569740 1 -- 172.21.15.12:0/2360381234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efeaba00 msgr2=0x5614efeab2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:00.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.370+0000 7fe415569740 1 --2- 172.21.15.12:0/2360381234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efeaba00 0x5614efeab2d0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fe40400aee0 tx=0x7fe404016bf0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.370+0000 7fe415569740 1 -- 172.21.15.12:0/2360381234 shutdown_connections 2024-04-22T13:33:00.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.370+0000 7fe415569740 1 --2- 172.21.15.12:0/2360381234 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efeaba00 0x5614efeab2d0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.370+0000 7fe415569740 1 --2- 172.21.15.12:0/2360381234 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614efea2e80 0x5614efeaad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.370+0000 7fe415569740 1 --2- 172.21.15.12:0/2360381234 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614efea24d0 0x5614efe9ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.370+0000 7fe415569740 1 -- 172.21.15.12:0/2360381234 >> 172.21.15.12:0/2360381234 conn(0x5614efe9cba0 msgr2=0x5614efe9b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:00.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.370+0000 7fe415569740 1 -- 172.21.15.12:0/2360381234 shutdown_connections 2024-04-22T13:33:00.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.370+0000 7fe415569740 1 -- 172.21.15.12:0/2360381234 wait complete. 2024-04-22T13:33:00.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.370+0000 7fe415569740 1 -- start start 2024-04-22T13:33:00.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.370+0000 7fe415569740 1 Processor -- start 2024-04-22T13:33:00.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.371+0000 7fe415569740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614efea24d0 0x5614efeb3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.371+0000 7fe415569740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efea2e80 0x5614efeb3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.371+0000 7fe415569740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614efeaba00 0x5614eff244b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.371+0000 7fe415569740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c1acc80 con 0x5614efea2e80 2024-04-22T13:33:00.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.371+0000 7fe415569740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c1ac5b0 con 0x5614efeaba00 2024-04-22T13:33:00.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.371+0000 7fe415569740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c1a9810 con 0x5614efea24d0 2024-04-22T13:33:00.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.371+0000 7fe415569740 10 client.0 _ll_register_callbacks cb 0x7fe3fc0045a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:00.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.371+0000 7fe40b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efea2e80 0x5614efeb3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.371+0000 7fe40b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efea2e80 0x5614efeb3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51598/0 (socket says 172.21.15.12:51598) 2024-04-22T13:33:00.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.371+0000 7fe40b7fe640 1 -- 172.21.15.12:0/3238451683 learned_addr learned my addr 172.21.15.12:0/3238451683 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:00.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.371+0000 7fe40bfff640 1 --2- 172.21.15.12:0/3238451683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614efea24d0 0x5614efeb3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.371+0000 7fe410f1b640 1 --2- 172.21.15.12:0/3238451683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614efeaba00 0x5614eff244b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.371+0000 7fe40b7fe640 1 -- 172.21.15.12:0/3238451683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614efea24d0 msgr2=0x5614efeb3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:00.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.371+0000 7fe40b7fe640 1 --2- 172.21.15.12:0/3238451683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614efea24d0 0x5614efeb3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.371+0000 7fe40b7fe640 1 -- 172.21.15.12:0/3238451683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614efeaba00 msgr2=0x5614eff244b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:00.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.371+0000 7fe40b7fe640 1 --2- 172.21.15.12:0/3238451683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614efeaba00 0x5614eff244b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.371+0000 7fe40b7fe640 1 -- 172.21.15.12:0/3238451683 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614eff249f0 con 0x5614efea2e80 2024-04-22T13:33:00.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.371+0000 7fe410f1b640 1 --2- 172.21.15.12:0/3238451683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614efeaba00 0x5614eff244b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:00.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.371+0000 7fe40b7fe640 1 --2- 172.21.15.12:0/3238451683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efea2e80 0x5614efeb3670 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fe400007590 tx=0x7fe400087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c98c72c92171000 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:00.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.371+0000 7fe3eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:33:00.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.371+0000 7fe3eb7fe640 1 -- 172.21.15.12:0/3238451683 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40009c8d0 con 0x5614efea2e80 2024-04-22T13:33:00.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.371+0000 7fe3eb7fe640 1 -- 172.21.15.12:0/3238451683 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7fe400098f40 con 0x5614efea2e80 2024-04-22T13:33:00.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.371+0000 7fe415569740 1 -- 172.21.15.12:0/3238451683 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe404032070 con 0x5614efea2e80 2024-04-22T13:33:00.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.371+0000 7fe3eb7fe640 1 -- 172.21.15.12:0/3238451683 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe4000a0c80 con 0x5614efea2e80 2024-04-22T13:33:00.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.371+0000 7fe415569740 10 client.18753 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:00.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.371+0000 7fe415569740 1 -- 172.21.15.12:0/3238451683 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5614eff25c30 con 0x5614efea2e80 2024-04-22T13:33:00.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.372+0000 7fe3eb7fe640 1 -- 172.21.15.12:0/3238451683 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe404032070 con 0x5614efea2e80 2024-04-22T13:33:00.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.372+0000 7fe3eb7fe640 1 -- 172.21.15.12:0/3238451683 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe400087140 con 0x5614efea2e80 2024-04-22T13:33:00.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.372+0000 7fe415569740 10 client.18753 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:00.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.372+0000 7fe415569740 10 client.18753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:33:00.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.372+0000 7fe415569740 20 client.18753 populate_metadata read hostname 'smithi012' 2024-04-22T13:33:00.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.372+0000 7fe415569740 1 --2- 172.21.15.12:0/3238451683 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5614eff26380 0x5614eff287a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:00.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.372+0000 7fe415569740 4 client.18753 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:33:00.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.372+0000 7fe415569740 1 -- 172.21.15.12:0/3238451683 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5614efe0ea80 con 0x5614eff26380 2024-04-22T13:33:00.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.372+0000 7fe40bfff640 1 --2- 172.21.15.12:0/3238451683 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5614eff26380 0x5614eff287a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:00.374 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.372+0000 7fe3eb7fe640 10 client.18753 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:33:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.372+0000 7fe40bfff640 1 --2- 172.21.15.12:0/3238451683 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5614eff26380 0x5614eff287a0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baacb4e4567fc2a server_cookie=c73b5bc877237e4f in_seq=0 out_seq=0 2024-04-22T13:33:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.373+0000 7fe3eb7fe640 1 -- 172.21.15.12:0/3238451683 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5614efe0ea80 con 0x5614eff26380 2024-04-22T13:33:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.373+0000 7fe3eb7fe640 10 client.18753 handle_command_reply: tid=0 2024-04-22T13:33:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.373+0000 7fe415569740 1 client.18753 shutdown 2024-04-22T13:33:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.373+0000 7fe415569740 1 -- 172.21.15.12:0/3238451683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efea2e80 msgr2=0x5614efeb3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.373+0000 7fe415569740 1 --2- 172.21.15.12:0/3238451683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efea2e80 0x5614efeb3670 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fe400007590 tx=0x7fe400087040 comp rx=0 tx=0).stop 2024-04-22T13:33:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.373+0000 7fe415569740 1 -- 172.21.15.12:0/3238451683 shutdown_connections 2024-04-22T13:33:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.373+0000 7fe415569740 1 --2- 172.21.15.12:0/3238451683 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5614eff26380 0x5614eff287a0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.373+0000 7fe415569740 1 --2- 172.21.15.12:0/3238451683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614efeaba00 0x5614eff244b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.373+0000 7fe415569740 1 --2- 172.21.15.12:0/3238451683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5614efea2e80 0x5614efeb3670 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.373+0000 7fe3eb7fe640 0 client.18753 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:33:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.373+0000 7fe415569740 1 --2- 172.21.15.12:0/3238451683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5614efea24d0 0x5614efeb3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.373+0000 7fe415569740 1 -- 172.21.15.12:0/3238451683 >> 172.21.15.12:0/3238451683 conn(0x5614efe9cba0 msgr2=0x5614efe9e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.373+0000 7fe415569740 1 -- 172.21.15.12:0/3238451683 shutdown_connections 2024-04-22T13:33:00.375 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.373+0000 7fe415569740 1 -- 172.21.15.12:0/3238451683 wait complete. 2024-04-22T13:33:00.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.373+0000 7fe415569740 20 client.18753 trim_cache size 0 max 16384 2024-04-22T13:33:00.376 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:33:00.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.373+0000 7fe41297e640 1 -- 172.21.15.12:0/3493246568 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe3f40a3d20 msgr2=0x7fe3f40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:00.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.373+0000 7fe41297e640 1 --2- 172.21.15.12:0/3493246568 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe3f40a3d20 0x7fe3f40a61e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe404016b40 tx=0x7fe40401a000 comp rx=0 tx=0).stop 2024-04-22T13:33:00.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.373+0000 7fe41297e640 1 -- 172.21.15.12:0/3493246568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe40c103af0 msgr2=0x7fe40c19e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:00.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.373+0000 7fe41297e640 1 --2- 172.21.15.12:0/3493246568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe40c103af0 0x7fe40c19e990 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fe40000c920 tx=0x7fe40000cde0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.374+0000 7fe41297e640 1 -- 172.21.15.12:0/3493246568 shutdown_connections 2024-04-22T13:33:00.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.374+0000 7fe41297e640 1 --2- 172.21.15.12:0/3493246568 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe3f40a3d20 0x7fe3f40a61e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.374+0000 7fe41297e640 1 --2- 172.21.15.12:0/3493246568 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe40c10c510 0x7fe40c198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.374+0000 7fe41297e640 1 --2- 172.21.15.12:0/3493246568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe40c103af0 0x7fe40c19e990 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.374+0000 7fe41297e640 1 --2- 172.21.15.12:0/3493246568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe40c1031d0 0x7fe40c19e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:00.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.374+0000 7fe41297e640 1 -- 172.21.15.12:0/3493246568 >> 172.21.15.12:0/3493246568 conn(0x7fe40c077f50 msgr2=0x7fe40c0ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:00.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.374+0000 7fe41297e640 1 -- 172.21.15.12:0/3493246568 shutdown_connections 2024-04-22T13:33:00.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:00.374+0000 7fe41297e640 1 -- 172.21.15.12:0/3493246568 wait complete. 2024-04-22T13:33:01.381 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:33:01.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:33:01.381 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:33:01.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.496+0000 7fce33aba640 1 -- 172.21.15.12:0/1165162534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce2c105ea0 msgr2=0x7fce2c106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:01.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.496+0000 7fce33aba640 1 --2- 172.21.15.12:0/1165162534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce2c105ea0 0x7fce2c106280 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fce28009900 tx=0x7fce2802bc80 comp rx=0 tx=0).stop 2024-04-22T13:33:01.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.496+0000 7fce33aba640 1 -- 172.21.15.12:0/1165162534 shutdown_connections 2024-04-22T13:33:01.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.496+0000 7fce33aba640 1 --2- 172.21.15.12:0/1165162534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce2c101ad0 0x7fce2c10dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.496+0000 7fce33aba640 1 --2- 172.21.15.12:0/1165162534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce2c101130 0x7fce2c101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.496+0000 7fce33aba640 1 --2- 172.21.15.12:0/1165162534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce2c105ea0 0x7fce2c106280 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.496+0000 7fce33aba640 1 -- 172.21.15.12:0/1165162534 >> 172.21.15.12:0/1165162534 conn(0x7fce2c077e30 msgr2=0x7fce2c078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:01.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.496+0000 7fce33aba640 1 -- 172.21.15.12:0/1165162534 shutdown_connections 2024-04-22T13:33:01.494 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.496+0000 7fce33aba640 1 -- 172.21.15.12:0/1165162534 wait complete. 2024-04-22T13:33:01.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.497+0000 7fce33aba640 1 Processor -- start 2024-04-22T13:33:01.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.497+0000 7fce33aba640 1 -- start start 2024-04-22T13:33:01.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.497+0000 7fce33aba640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce2c101130 0x7fce2c19e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.497+0000 7fce33aba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce2c101ad0 0x7fce2c19eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.497+0000 7fce33aba640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce2c105ea0 0x7fce2c198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.497+0000 7fce33aba640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fce2c112350 con 0x7fce2c101130 2024-04-22T13:33:01.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.497+0000 7fce33aba640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fce2c1121d0 con 0x7fce2c105ea0 2024-04-22T13:33:01.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.497+0000 7fce33aba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fce2c1124d0 con 0x7fce2c101ad0 2024-04-22T13:33:01.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.497+0000 7fce31856640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce2c101130 0x7fce2c19e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.497+0000 7fce31856640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce2c101130 0x7fce2c19e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51612/0 (socket says 172.21.15.12:51612) 2024-04-22T13:33:01.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.497+0000 7fce31856640 1 -- 172.21.15.12:0/1242472900 learned_addr learned my addr 172.21.15.12:0/1242472900 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:01.495 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.497+0000 7fce31055640 1 --2- 172.21.15.12:0/1242472900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce2c101ad0 0x7fce2c19eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.497+0000 7fce32057640 1 --2- 172.21.15.12:0/1242472900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce2c105ea0 0x7fce2c198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.497+0000 7fce31856640 1 -- 172.21.15.12:0/1242472900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce2c101ad0 msgr2=0x7fce2c19eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:01.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.497+0000 7fce31856640 1 --2- 172.21.15.12:0/1242472900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce2c101ad0 0x7fce2c19eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.497+0000 7fce31856640 1 -- 172.21.15.12:0/1242472900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce2c105ea0 msgr2=0x7fce2c198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:01.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.497+0000 7fce31856640 1 --2- 172.21.15.12:0/1242472900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce2c105ea0 0x7fce2c198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.497+0000 7fce31856640 1 -- 172.21.15.12:0/1242472900 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce2c198f50 con 0x7fce2c101130 2024-04-22T13:33:01.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.497+0000 7fce32057640 1 --2- 172.21.15.12:0/1242472900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce2c105ea0 0x7fce2c198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:01.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.497+0000 7fce31055640 1 --2- 172.21.15.12:0/1242472900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce2c101ad0 0x7fce2c19eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:01.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.497+0000 7fce31856640 1 --2- 172.21.15.12:0/1242472900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce2c101130 0x7fce2c19e5d0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fce28006010 tx=0x7fce28004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:01.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.497+0000 7fce22ffd640 1 -- 172.21.15.12:0/1242472900 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28043020 con 0x7fce2c101130 2024-04-22T13:33:01.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.498+0000 7fce22ffd640 1 -- 172.21.15.12:0/1242472900 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce28004440 con 0x7fce2c101130 2024-04-22T13:33:01.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.498+0000 7fce33aba640 1 -- 172.21.15.12:0/1242472900 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce2c1991e0 con 0x7fce2c101130 2024-04-22T13:33:01.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.498+0000 7fce22ffd640 1 -- 172.21.15.12:0/1242472900 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28003b50 con 0x7fce2c101130 2024-04-22T13:33:01.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.498+0000 7fce33aba640 1 -- 172.21.15.12:0/1242472900 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce2c1a5550 con 0x7fce2c101130 2024-04-22T13:33:01.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.500+0000 7fce22ffd640 1 -- 172.21.15.12:0/1242472900 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fce28046020 con 0x7fce2c101130 2024-04-22T13:33:01.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.500+0000 7fce36783740 1 Processor -- start 2024-04-22T13:33:01.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.500+0000 7fce36783740 1 -- start start 2024-04-22T13:33:01.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.501+0000 7fce36783740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d79f538e0 0x559d79f53cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.501+0000 7fce36783740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d79f54290 0x559d79f5c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.501+0000 7fce36783740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d79f5ce10 0x559d79f5f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.501+0000 7fce36783740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559d79eb7f00 con 0x559d79f54290 2024-04-22T13:33:01.506 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.501+0000 7fce36783740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559d79e55590 con 0x559d79f538e0 2024-04-22T13:33:01.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.501+0000 7fce36783740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559d79eb3ec0 con 0x559d79f5ce10 2024-04-22T13:33:01.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.501+0000 7fce31856640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d79f54290 0x559d79f5c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.501+0000 7fce22ffd640 1 --2- 172.21.15.12:0/1242472900 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fce180a3da0 0x7fce180a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.501+0000 7fce31856640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d79f54290 0x559d79f5c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51628/0 (socket says 172.21.15.12:51628) 2024-04-22T13:33:01.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.501+0000 7fce31856640 1 -- 172.21.15.12:0/2471304602 learned_addr learned my addr 172.21.15.12:0/2471304602 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:01.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.501+0000 7fce31055640 1 --2- 172.21.15.12:0/2471304602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d79f5ce10 0x559d79f5f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.501+0000 7fce32057640 1 --2- 172.21.15.12:0/2471304602 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d79f538e0 0x559d79f53cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.501+0000 7fce32057640 1 --2- 172.21.15.12:0/1242472900 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fce180a3da0 0x7fce180a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.507 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.501+0000 7fce22ffd640 1 -- 172.21.15.12:0/1242472900 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7fce280f23e0 con 0x7fce2c101130 2024-04-22T13:33:01.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.501+0000 7fce31856640 1 -- 172.21.15.12:0/2471304602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d79f5ce10 msgr2=0x559d79f5f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:01.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.501+0000 7fce31856640 1 --2- 172.21.15.12:0/2471304602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d79f5ce10 0x559d79f5f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.501+0000 7fce31856640 1 -- 172.21.15.12:0/2471304602 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d79f538e0 msgr2=0x559d79f53cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:01.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.501+0000 7fce31856640 1 --2- 172.21.15.12:0/2471304602 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d79f538e0 0x559d79f53cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.501+0000 7fce31856640 1 -- 172.21.15.12:0/2471304602 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d79c927e0 con 0x559d79f54290 2024-04-22T13:33:01.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.501+0000 7fce31055640 1 --2- 172.21.15.12:0/2471304602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d79f5ce10 0x559d79f5f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:33:01.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.502+0000 7fce31856640 1 --2- 172.21.15.12:0/2471304602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d79f54290 0x559d79f5c840 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fce280bae60 tx=0x7fce280a8fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca0e06d195c5af4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:01.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.502+0000 7fce32057640 1 --2- 172.21.15.12:0/1242472900 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fce180a3da0 0x7fce180a6260 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fce24009270 tx=0x7fce24005bd0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:01.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.502+0000 7fce17fff640 1 -- 172.21.15.12:0/2471304602 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce280be490 con 0x559d79f54290 2024-04-22T13:33:01.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.502+0000 7fce17fff640 1 -- 172.21.15.12:0/2471304602 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce280c2840 con 0x559d79f54290 2024-04-22T13:33:01.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.502+0000 7fce17fff640 1 -- 172.21.15.12:0/2471304602 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce280efdb0 con 0x559d79f54290 2024-04-22T13:33:01.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.502+0000 7fce36783740 1 -- 172.21.15.12:0/2471304602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d79f54290 msgr2=0x559d79f5c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:01.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.502+0000 7fce36783740 1 --2- 172.21.15.12:0/2471304602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d79f54290 0x559d79f5c840 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fce280bae60 tx=0x7fce280a8fc0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.503+0000 7fce36783740 1 -- 172.21.15.12:0/2471304602 shutdown_connections 2024-04-22T13:33:01.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.503+0000 7fce36783740 1 --2- 172.21.15.12:0/2471304602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d79f5ce10 0x559d79f5f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.503+0000 7fce36783740 1 --2- 172.21.15.12:0/2471304602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d79f54290 0x559d79f5c840 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.503+0000 7fce36783740 1 --2- 172.21.15.12:0/2471304602 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d79f538e0 0x559d79f53cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.503+0000 7fce36783740 1 -- 172.21.15.12:0/2471304602 >> 172.21.15.12:0/2471304602 conn(0x559d79f4dfb0 msgr2=0x559d79f4e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:01.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.503+0000 7fce36783740 1 -- 172.21.15.12:0/2471304602 shutdown_connections 2024-04-22T13:33:01.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.503+0000 7fce36783740 1 -- 172.21.15.12:0/2471304602 wait complete. 2024-04-22T13:33:01.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.503+0000 7fce36783740 1 -- start start 2024-04-22T13:33:01.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.503+0000 7fce36783740 1 Processor -- start 2024-04-22T13:33:01.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.503+0000 7fce36783740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d79f538e0 0x559d79f59e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.503+0000 7fce36783740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d79f54290 0x559d79f5a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.504+0000 7fce36783740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d79f5ce10 0x559d79f5a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.504+0000 7fce36783740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559d79eb7f00 con 0x559d79f5ce10 2024-04-22T13:33:01.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.504+0000 7fce36783740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559d79e71580 con 0x559d79f54290 2024-04-22T13:33:01.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.504+0000 7fce36783740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559d79f5fb40 con 0x559d79f538e0 2024-04-22T13:33:01.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.504+0000 7fce36783740 10 client.0 _ll_register_callbacks cb 0x559d79e77650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:01.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.504+0000 7fce32057640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d79f5ce10 0x559d79f5a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.504+0000 7fce32057640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d79f5ce10 0x559d79f5a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51644/0 (socket says 172.21.15.12:51644) 2024-04-22T13:33:01.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.504+0000 7fce32057640 1 -- 172.21.15.12:0/3178759281 learned_addr learned my addr 172.21.15.12:0/3178759281 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:01.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.504+0000 7fce31055640 1 --2- 172.21.15.12:0/3178759281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d79f54290 0x559d79f5a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.504+0000 7fce31856640 1 --2- 172.21.15.12:0/3178759281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d79f538e0 0x559d79f59e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.504+0000 7fce32057640 1 -- 172.21.15.12:0/3178759281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d79f538e0 msgr2=0x559d79f59e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:01.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.504+0000 7fce32057640 1 --2- 172.21.15.12:0/3178759281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d79f538e0 0x559d79f59e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.504+0000 7fce32057640 1 -- 172.21.15.12:0/3178759281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d79f54290 msgr2=0x559d79f5a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:01.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.504+0000 7fce32057640 1 --2- 172.21.15.12:0/3178759281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d79f54290 0x559d79f5a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.504+0000 7fce32057640 1 -- 172.21.15.12:0/3178759281 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d79cf2f70 con 0x559d79f5ce10 2024-04-22T13:33:01.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.504+0000 7fce31055640 1 --2- 172.21.15.12:0/3178759281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d79f54290 0x559d79f5a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:01.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.504+0000 7fce31856640 1 --2- 172.21.15.12:0/3178759281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d79f538e0 0x559d79f59e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:01.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.504+0000 7fce32057640 1 --2- 172.21.15.12:0/3178759281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d79f5ce10 0x559d79f5a8a0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fce24009d50 tx=0x7fce24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=885593eb33711f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:01.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.504+0000 7fce157fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:33:01.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.504+0000 7fce157fa640 1 -- 172.21.15.12:0/3178759281 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2400fd70 con 0x559d79f5ce10 2024-04-22T13:33:01.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.504+0000 7fce157fa640 1 -- 172.21.15.12:0/3178759281 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7fce24004b20 con 0x559d79f5ce10 2024-04-22T13:33:01.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.504+0000 7fce36783740 1 -- 172.21.15.12:0/3178759281 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce280c2840 con 0x559d79f5ce10 2024-04-22T13:33:01.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.505+0000 7fce157fa640 1 -- 172.21.15.12:0/3178759281 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fce24004d60 con 0x559d79f5ce10 2024-04-22T13:33:01.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.505+0000 7fce36783740 10 client.18765 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:01.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.505+0000 7fce36783740 1 -- 172.21.15.12:0/3178759281 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559d79b8a170 con 0x559d79f5ce10 2024-04-22T13:33:01.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.505+0000 7fce157fa640 1 -- 172.21.15.12:0/3178759281 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce280c2840 con 0x559d79f5ce10 2024-04-22T13:33:01.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.505+0000 7fce157fa640 1 -- 172.21.15.12:0/3178759281 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fce24030020 con 0x559d79f5ce10 2024-04-22T13:33:01.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.505+0000 7fce36783740 10 client.18765 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:01.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.505+0000 7fce36783740 10 client.18765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:33:01.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.505+0000 7fce36783740 20 client.18765 populate_metadata read hostname 'smithi012' 2024-04-22T13:33:01.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.505+0000 7fce36783740 1 --2- 172.21.15.12:0/3178759281 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559d79f6cf70 0x559d79f6f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.505+0000 7fce36783740 4 client.18765 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:33:01.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.505+0000 7fce36783740 1 -- 172.21.15.12:0/3178759281 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d79ec5920 con 0x559d79f6cf70 2024-04-22T13:33:01.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.505+0000 7fce31856640 1 --2- 172.21.15.12:0/3178759281 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559d79f6cf70 0x559d79f6f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.506+0000 7fce157fa640 10 client.18765 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:33:01.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.506+0000 7fce31856640 1 --2- 172.21.15.12:0/3178759281 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559d79f6cf70 0x559d79f6f3f0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23b9ef5379379095 server_cookie=c96fcbecefae989f in_seq=0 out_seq=0 2024-04-22T13:33:01.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.507+0000 7fce157fa640 1 -- 172.21.15.12:0/3178759281 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559d79ec5920 con 0x559d79f6cf70 2024-04-22T13:33:01.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.507+0000 7fce157fa640 10 client.18765 handle_command_reply: tid=0 2024-04-22T13:33:01.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.507+0000 7fce36783740 1 client.18765 shutdown 2024-04-22T13:33:01.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.507+0000 7fce36783740 1 -- 172.21.15.12:0/3178759281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d79f5ce10 msgr2=0x559d79f5a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:01.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.507+0000 7fce36783740 1 --2- 172.21.15.12:0/3178759281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d79f5ce10 0x559d79f5a8a0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fce24009d50 tx=0x7fce24004a40 comp rx=0 tx=0).stop 2024-04-22T13:33:01.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.507+0000 7fce36783740 1 -- 172.21.15.12:0/3178759281 shutdown_connections 2024-04-22T13:33:01.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.507+0000 7fce36783740 1 --2- 172.21.15.12:0/3178759281 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559d79f6cf70 0x559d79f6f3f0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.507+0000 7fce36783740 1 --2- 172.21.15.12:0/3178759281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d79f5ce10 0x559d79f5a8a0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.507+0000 7fce36783740 1 --2- 172.21.15.12:0/3178759281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d79f54290 0x559d79f5a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.507+0000 7fce157fa640 0 client.18765 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:33:01.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.507+0000 7fce36783740 1 --2- 172.21.15.12:0/3178759281 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d79f538e0 0x559d79f59e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.507+0000 7fce36783740 1 -- 172.21.15.12:0/3178759281 >> 172.21.15.12:0/3178759281 conn(0x559d79f4dfb0 msgr2=0x559d79f4fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:01.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.507+0000 7fce36783740 1 -- 172.21.15.12:0/3178759281 shutdown_connections 2024-04-22T13:33:01.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.507+0000 7fce36783740 1 -- 172.21.15.12:0/3178759281 wait complete. 2024-04-22T13:33:01.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.507+0000 7fce36783740 20 client.18765 trim_cache size 0 max 16384 2024-04-22T13:33:01.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.525+0000 7fce36783740 1 Processor -- start 2024-04-22T13:33:01.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.525+0000 7fce36783740 1 -- start start 2024-04-22T13:33:01.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.525+0000 7fce36783740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d79f538e0 0x559d79f5c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.525+0000 7fce36783740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d79f54290 0x559d79f687b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.525+0000 7fce36783740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d79f5ce10 0x559d79f5a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.525+0000 7fce36783740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fce2c1adaa0 con 0x559d79f5ce10 2024-04-22T13:33:01.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.525+0000 7fce36783740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fce2c1af5e0 con 0x559d79f54290 2024-04-22T13:33:01.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.525+0000 7fce36783740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fce2c1ae170 con 0x559d79f538e0 2024-04-22T13:33:01.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.525+0000 7fce32057640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d79f5ce10 0x559d79f5a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.525+0000 7fce32057640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d79f5ce10 0x559d79f5a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51648/0 (socket says 172.21.15.12:51648) 2024-04-22T13:33:01.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.525+0000 7fce32057640 1 -- 172.21.15.12:0/3507255905 learned_addr learned my addr 172.21.15.12:0/3507255905 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:01.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.525+0000 7fce31856640 1 --2- 172.21.15.12:0/3507255905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d79f538e0 0x559d79f5c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.525+0000 7fce32057640 1 -- 172.21.15.12:0/3507255905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d79f538e0 msgr2=0x559d79f5c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:01.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.525+0000 7fce32057640 1 --2- 172.21.15.12:0/3507255905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d79f538e0 0x559d79f5c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.525+0000 7fce31055640 1 --2- 172.21.15.12:0/3507255905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d79f54290 0x559d79f687b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.525+0000 7fce32057640 1 -- 172.21.15.12:0/3507255905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d79f54290 msgr2=0x559d79f687b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:01.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.525+0000 7fce32057640 1 --2- 172.21.15.12:0/3507255905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d79f54290 0x559d79f687b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.525+0000 7fce32057640 1 -- 172.21.15.12:0/3507255905 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d79f68cf0 con 0x559d79f5ce10 2024-04-22T13:33:01.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.525+0000 7fce32057640 1 --2- 172.21.15.12:0/3507255905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d79f5ce10 0x559d79f5a4f0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fce24006010 tx=0x7fce240098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9959aaf1368feba server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:01.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.526+0000 7fce177fe640 1 -- 172.21.15.12:0/3507255905 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce24041070 con 0x559d79f5ce10 2024-04-22T13:33:01.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.526+0000 7fce177fe640 1 -- 172.21.15.12:0/3507255905 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce2400f7b0 con 0x559d79f5ce10 2024-04-22T13:33:01.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.526+0000 7fce177fe640 1 -- 172.21.15.12:0/3507255905 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce24005070 con 0x559d79f5ce10 2024-04-22T13:33:01.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.526+0000 7fce36783740 1 -- 172.21.15.12:0/3507255905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d79f5ce10 msgr2=0x559d79f5a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:01.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.526+0000 7fce36783740 1 --2- 172.21.15.12:0/3507255905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d79f5ce10 0x559d79f5a4f0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fce24006010 tx=0x7fce240098f0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.526+0000 7fce36783740 1 -- 172.21.15.12:0/3507255905 shutdown_connections 2024-04-22T13:33:01.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.526+0000 7fce36783740 1 --2- 172.21.15.12:0/3507255905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d79f5ce10 0x559d79f5a4f0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.526+0000 7fce36783740 1 --2- 172.21.15.12:0/3507255905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d79f54290 0x559d79f687b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.526+0000 7fce36783740 1 --2- 172.21.15.12:0/3507255905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d79f538e0 0x559d79f5c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.526+0000 7fce36783740 1 -- 172.21.15.12:0/3507255905 >> 172.21.15.12:0/3507255905 conn(0x559d79f4dfb0 msgr2=0x559d79f59de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:01.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.526+0000 7fce36783740 1 -- 172.21.15.12:0/3507255905 shutdown_connections 2024-04-22T13:33:01.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.526+0000 7fce36783740 1 -- 172.21.15.12:0/3507255905 wait complete. 2024-04-22T13:33:01.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.527+0000 7fce36783740 1 -- start start 2024-04-22T13:33:01.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.527+0000 7fce36783740 1 Processor -- start 2024-04-22T13:33:01.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.527+0000 7fce36783740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d79f538e0 0x559d79f64cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.527+0000 7fce36783740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d79f54290 0x559d79f65200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.527+0000 7fce36783740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d79f5ce10 0x559d79f61c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.527+0000 7fce36783740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fce2c1aef10 con 0x559d79f54290 2024-04-22T13:33:01.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.527+0000 7fce36783740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fce2c1ae840 con 0x559d79f538e0 2024-04-22T13:33:01.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.527+0000 7fce36783740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fce2c1abaa0 con 0x559d79f5ce10 2024-04-22T13:33:01.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.527+0000 7fce36783740 10 client.0 _ll_register_callbacks cb 0x7fce24009c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:01.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.527+0000 7fce31055640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d79f54290 0x559d79f65200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.527+0000 7fce31055640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d79f54290 0x559d79f65200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:51654/0 (socket says 172.21.15.12:51654) 2024-04-22T13:33:01.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.527+0000 7fce31055640 1 -- 172.21.15.12:0/2304497509 learned_addr learned my addr 172.21.15.12:0/2304497509 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:01.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.527+0000 7fce31856640 1 --2- 172.21.15.12:0/2304497509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d79f538e0 0x559d79f64cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.527+0000 7fce32057640 1 --2- 172.21.15.12:0/2304497509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d79f5ce10 0x559d79f61c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.528+0000 7fce31055640 1 -- 172.21.15.12:0/2304497509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d79f5ce10 msgr2=0x559d79f61c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:01.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.528+0000 7fce31055640 1 --2- 172.21.15.12:0/2304497509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d79f5ce10 0x559d79f61c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.528+0000 7fce31055640 1 -- 172.21.15.12:0/2304497509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d79f538e0 msgr2=0x559d79f64cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:01.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.528+0000 7fce31055640 1 --2- 172.21.15.12:0/2304497509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d79f538e0 0x559d79f64cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.528+0000 7fce31055640 1 -- 172.21.15.12:0/2304497509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d79f62490 con 0x559d79f54290 2024-04-22T13:33:01.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.528+0000 7fce31856640 1 --2- 172.21.15.12:0/2304497509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d79f538e0 0x559d79f64cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:01.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.528+0000 7fce31055640 1 --2- 172.21.15.12:0/2304497509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d79f54290 0x559d79f65200 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fce1c005fd0 tx=0x7fce1c0138f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69fa9b45dfe76930 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:01.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.528+0000 7fce157fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:33:01.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.528+0000 7fce157fa640 1 -- 172.21.15.12:0/2304497509 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1c005020 con 0x559d79f54290 2024-04-22T13:33:01.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.528+0000 7fce157fa640 1 -- 172.21.15.12:0/2304497509 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7fce1c0051c0 con 0x559d79f54290 2024-04-22T13:33:01.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.528+0000 7fce36783740 1 -- 172.21.15.12:0/2304497509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce2400f7b0 con 0x559d79f54290 2024-04-22T13:33:01.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.528+0000 7fce157fa640 1 -- 172.21.15.12:0/2304497509 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fce1c0053c0 con 0x559d79f54290 2024-04-22T13:33:01.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.528+0000 7fce36783740 10 client.18771 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:01.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.528+0000 7fce36783740 1 -- 172.21.15.12:0/2304497509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559d79fd4030 con 0x559d79f54290 2024-04-22T13:33:01.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.528+0000 7fce157fa640 1 -- 172.21.15.12:0/2304497509 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce2400f7b0 con 0x559d79f54290 2024-04-22T13:33:01.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.528+0000 7fce157fa640 1 -- 172.21.15.12:0/2304497509 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fce1c010620 con 0x559d79f54290 2024-04-22T13:33:01.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.528+0000 7fce36783740 10 client.18771 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:01.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.528+0000 7fce36783740 10 client.18771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:33:01.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.528+0000 7fce36783740 20 client.18771 populate_metadata read hostname 'smithi012' 2024-04-22T13:33:01.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.529+0000 7fce36783740 1 --2- 172.21.15.12:0/2304497509 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559d79fd4810 0x559d79fd6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.529+0000 7fce36783740 4 client.18771 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:33:01.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.529+0000 7fce36783740 1 -- 172.21.15.12:0/2304497509 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559d79ebf790 con 0x559d79fd4810 2024-04-22T13:33:01.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.529+0000 7fce31856640 1 --2- 172.21.15.12:0/2304497509 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559d79fd4810 0x559d79fd6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.529+0000 7fce31856640 1 --2- 172.21.15.12:0/2304497509 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559d79fd4810 0x559d79fd6c30 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95cedc2e28799481 server_cookie=fefd5eea5cd346fa in_seq=0 out_seq=0 2024-04-22T13:33:01.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.529+0000 7fce157fa640 10 client.18771 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:33:01.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.530+0000 7fce157fa640 1 -- 172.21.15.12:0/2304497509 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559d79ebf790 con 0x559d79fd4810 2024-04-22T13:33:01.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.530+0000 7fce157fa640 10 client.18771 handle_command_reply: tid=0 2024-04-22T13:33:01.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.530+0000 7fce36783740 1 client.18771 shutdown 2024-04-22T13:33:01.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.530+0000 7fce36783740 1 -- 172.21.15.12:0/2304497509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d79f54290 msgr2=0x559d79f65200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:01.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.530+0000 7fce36783740 1 --2- 172.21.15.12:0/2304497509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d79f54290 0x559d79f65200 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fce1c005fd0 tx=0x7fce1c0138f0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.530+0000 7fce36783740 1 -- 172.21.15.12:0/2304497509 shutdown_connections 2024-04-22T13:33:01.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.530+0000 7fce36783740 1 --2- 172.21.15.12:0/2304497509 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559d79fd4810 0x559d79fd6c30 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.530+0000 7fce36783740 1 --2- 172.21.15.12:0/2304497509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559d79f5ce10 0x559d79f61c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.530+0000 7fce36783740 1 --2- 172.21.15.12:0/2304497509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559d79f54290 0x559d79f65200 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.530+0000 7fce157fa640 0 client.18771 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:33:01.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.530+0000 7fce36783740 1 --2- 172.21.15.12:0/2304497509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559d79f538e0 0x559d79f64cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.530+0000 7fce36783740 1 -- 172.21.15.12:0/2304497509 >> 172.21.15.12:0/2304497509 conn(0x559d79f4dfb0 msgr2=0x559d79f4fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:01.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.531+0000 7fce36783740 1 -- 172.21.15.12:0/2304497509 shutdown_connections 2024-04-22T13:33:01.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.531+0000 7fce36783740 1 -- 172.21.15.12:0/2304497509 wait complete. 2024-04-22T13:33:01.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.531+0000 7fce36783740 20 client.18771 trim_cache size 0 max 16384 2024-04-22T13:33:01.538 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:33:01.538 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-04-22T13:33:01.538 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "872e28f7-70ae-412e-a360-1e6233cd104c", 2024-04-22T13:33:01.538 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-04-22T13:33:01.538 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:33:01.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.531+0000 7fce33aba640 1 -- 172.21.15.12:0/1242472900 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fce180a3da0 msgr2=0x7fce180a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:01.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.531+0000 7fce33aba640 1 --2- 172.21.15.12:0/1242472900 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fce180a3da0 0x7fce180a6260 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fce24009270 tx=0x7fce24005bd0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.531+0000 7fce33aba640 1 -- 172.21.15.12:0/1242472900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce2c101130 msgr2=0x7fce2c19e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:01.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.531+0000 7fce33aba640 1 --2- 172.21.15.12:0/1242472900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce2c101130 0x7fce2c19e5d0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fce28006010 tx=0x7fce28004040 comp rx=0 tx=0).stop 2024-04-22T13:33:01.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.531+0000 7fce33aba640 1 -- 172.21.15.12:0/1242472900 shutdown_connections 2024-04-22T13:33:01.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.531+0000 7fce33aba640 1 --2- 172.21.15.12:0/1242472900 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fce180a3da0 0x7fce180a6260 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.531+0000 7fce33aba640 1 --2- 172.21.15.12:0/1242472900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce2c105ea0 0x7fce2c198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.531+0000 7fce33aba640 1 --2- 172.21.15.12:0/1242472900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fce2c101ad0 0x7fce2c19eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.531+0000 7fce33aba640 1 --2- 172.21.15.12:0/1242472900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fce2c101130 0x7fce2c19e5d0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.531+0000 7fce33aba640 1 -- 172.21.15.12:0/1242472900 >> 172.21.15.12:0/1242472900 conn(0x7fce2c077e30 msgr2=0x7fce2c0fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:01.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.531+0000 7fce33aba640 1 -- 172.21.15.12:0/1242472900 shutdown_connections 2024-04-22T13:33:01.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.531+0000 7fce33aba640 1 -- 172.21.15.12:0/1242472900 wait complete. 2024-04-22T13:33:01.541 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:33:01.542 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:33:01.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.651+0000 7ff1970ce640 1 -- 172.21.15.12:0/2912777795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff190101a30 msgr2=0x7ff190105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:01.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.651+0000 7ff1970ce640 1 --2- 172.21.15.12:0/2912777795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff190101a30 0x7ff190105a70 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7ff184009900 tx=0x7ff18402bc30 comp rx=0 tx=0).stop 2024-04-22T13:33:01.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.651+0000 7ff1970ce640 1 -- 172.21.15.12:0/2912777795 shutdown_connections 2024-04-22T13:33:01.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.651+0000 7ff1970ce640 1 --2- 172.21.15.12:0/2912777795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff190106150 0x7ff19010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.651+0000 7ff1970ce640 1 --2- 172.21.15.12:0/2912777795 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff190101a30 0x7ff190105a70 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.651+0000 7ff1970ce640 1 --2- 172.21.15.12:0/2912777795 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff190101080 0x7ff190101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.651+0000 7ff1970ce640 1 -- 172.21.15.12:0/2912777795 >> 172.21.15.12:0/2912777795 conn(0x7ff190078000 msgr2=0x7ff190078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:01.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.651+0000 7ff1970ce640 1 -- 172.21.15.12:0/2912777795 shutdown_connections 2024-04-22T13:33:01.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.651+0000 7ff1970ce640 1 -- 172.21.15.12:0/2912777795 wait complete. 2024-04-22T13:33:01.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.651+0000 7ff1970ce640 1 Processor -- start 2024-04-22T13:33:01.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.651+0000 7ff1970ce640 1 -- start start 2024-04-22T13:33:01.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.652+0000 7ff1970ce640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff190101080 0x7ff190102b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.652+0000 7ff1970ce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff190106150 0x7ff190103050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.652+0000 7ff1970ce640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff190103590 0x7ff1901a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.652+0000 7ff1970ce640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff1901110e0 con 0x7ff190101080 2024-04-22T13:33:01.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.652+0000 7ff1970ce640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff190110f60 con 0x7ff190106150 2024-04-22T13:33:01.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.652+0000 7ff1970ce640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff190111260 con 0x7ff190103590 2024-04-22T13:33:01.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.652+0000 7ff194e6a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff190101080 0x7ff190102b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.652+0000 7ff194e6a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff190101080 0x7ff190102b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33712/0 (socket says 172.21.15.12:33712) 2024-04-22T13:33:01.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.652+0000 7ff194e6a640 1 -- 172.21.15.12:0/2065567681 learned_addr learned my addr 172.21.15.12:0/2065567681 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:01.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.652+0000 7ff19566b640 1 --2- 172.21.15.12:0/2065567681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff190103590 0x7ff1901a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.652+0000 7ff18ffff640 1 --2- 172.21.15.12:0/2065567681 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff190106150 0x7ff190103050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.652+0000 7ff194e6a640 1 -- 172.21.15.12:0/2065567681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff190103590 msgr2=0x7ff1901a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:01.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.652+0000 7ff194e6a640 1 --2- 172.21.15.12:0/2065567681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff190103590 0x7ff1901a68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.652+0000 7ff194e6a640 1 -- 172.21.15.12:0/2065567681 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff190106150 msgr2=0x7ff190103050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:01.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.652+0000 7ff194e6a640 1 --2- 172.21.15.12:0/2065567681 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff190106150 0x7ff190103050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.652+0000 7ff194e6a640 1 -- 172.21.15.12:0/2065567681 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1901a6fb0 con 0x7ff190101080 2024-04-22T13:33:01.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.652+0000 7ff194e6a640 1 --2- 172.21.15.12:0/2065567681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff190101080 0x7ff190102b10 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7ff18000e9f0 tx=0x7ff18000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:01.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.652+0000 7ff18dffb640 1 -- 172.21.15.12:0/2065567681 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18000cd80 con 0x7ff190101080 2024-04-22T13:33:01.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.652+0000 7ff18dffb640 1 -- 172.21.15.12:0/2065567681 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1800108e0 con 0x7ff190101080 2024-04-22T13:33:01.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.652+0000 7ff1970ce640 1 -- 172.21.15.12:0/2065567681 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1901a72a0 con 0x7ff190101080 2024-04-22T13:33:01.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.652+0000 7ff18dffb640 1 -- 172.21.15.12:0/2065567681 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff180018980 con 0x7ff190101080 2024-04-22T13:33:01.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.653+0000 7ff1970ce640 1 -- 172.21.15.12:0/2065567681 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1901a7810 con 0x7ff190101080 2024-04-22T13:33:01.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.655+0000 7ff18dffb640 1 -- 172.21.15.12:0/2065567681 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff180010430 con 0x7ff190101080 2024-04-22T13:33:01.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.655+0000 7ff199d95740 1 Processor -- start 2024-04-22T13:33:01.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.655+0000 7ff199d95740 1 -- start start 2024-04-22T13:33:01.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.655+0000 7ff199d95740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563ce3eb2580 0x563ce3eb2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.655+0000 7ff199d95740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ce3eb2ea0 0x563ce3ebb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.655+0000 7ff199d95740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ce3ebba80 0x563ce3ebde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.655+0000 7ff199d95740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563ce3e171e0 con 0x563ce3eb2ea0 2024-04-22T13:33:01.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.655+0000 7ff199d95740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x563ce3db4390 con 0x563ce3eb2580 2024-04-22T13:33:01.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.655+0000 7ff199d95740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563ce3e131a0 con 0x563ce3ebba80 2024-04-22T13:33:01.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.655+0000 7ff194e6a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ce3eb2ea0 0x563ce3ebb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.655+0000 7ff194e6a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ce3eb2ea0 0x563ce3ebb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33728/0 (socket says 172.21.15.12:33728) 2024-04-22T13:33:01.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.655+0000 7ff194e6a640 1 -- 172.21.15.12:0/3864707828 learned_addr learned my addr 172.21.15.12:0/3864707828 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:01.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.656+0000 7ff19566b640 1 --2- 172.21.15.12:0/3864707828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563ce3eb2580 0x563ce3eb2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.656+0000 7ff18dffb640 1 --2- 172.21.15.12:0/2065567681 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff1780a3dd0 0x7ff1780a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.656+0000 7ff18ffff640 1 --2- 172.21.15.12:0/3864707828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ce3ebba80 0x563ce3ebde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.656+0000 7ff19566b640 1 --2- 172.21.15.12:0/2065567681 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff1780a3dd0 0x7ff1780a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.656+0000 7ff194e6a640 1 -- 172.21.15.12:0/3864707828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ce3ebba80 msgr2=0x563ce3ebde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:01.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.656+0000 7ff194e6a640 1 --2- 172.21.15.12:0/3864707828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ce3ebba80 0x563ce3ebde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.656+0000 7ff194e6a640 1 -- 172.21.15.12:0/3864707828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563ce3eb2580 msgr2=0x563ce3eb2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.656+0000 7ff194e6a640 1 --2- 172.21.15.12:0/3864707828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563ce3eb2580 0x563ce3eb2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.656+0000 7ff18dffb640 1 -- 172.21.15.12:0/2065567681 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7ff1800d0a10 con 0x7ff190101080 2024-04-22T13:33:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.656+0000 7ff194e6a640 1 -- 172.21.15.12:0/3864707828 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ce3bf37d0 con 0x563ce3eb2ea0 2024-04-22T13:33:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.656+0000 7ff194e6a640 1 --2- 172.21.15.12:0/3864707828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ce3eb2ea0 0x563ce3ebb540 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7ff1800ce0f0 tx=0x7ff18000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffe067bbcc0bce8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.656+0000 7ff19566b640 1 --2- 172.21.15.12:0/2065567681 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff1780a3dd0 0x7ff1780a6290 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff188005fd0 tx=0x7ff188004ab0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.656+0000 7ff176ffd640 1 -- 172.21.15.12:0/3864707828 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1800d9020 con 0x563ce3eb2ea0 2024-04-22T13:33:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.656+0000 7ff176ffd640 1 -- 172.21.15.12:0/3864707828 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1800da040 con 0x563ce3eb2ea0 2024-04-22T13:33:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.656+0000 7ff176ffd640 1 -- 172.21.15.12:0/3864707828 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1800a0d80 con 0x563ce3eb2ea0 2024-04-22T13:33:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.657+0000 7ff199d95740 1 -- 172.21.15.12:0/3864707828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ce3eb2ea0 msgr2=0x563ce3ebb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:01.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.657+0000 7ff199d95740 1 --2- 172.21.15.12:0/3864707828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ce3eb2ea0 0x563ce3ebb540 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7ff1800ce0f0 tx=0x7ff18000cf20 comp rx=0 tx=0).stop 2024-04-22T13:33:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.657+0000 7ff199d95740 1 -- 172.21.15.12:0/3864707828 shutdown_connections 2024-04-22T13:33:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.657+0000 7ff199d95740 1 --2- 172.21.15.12:0/3864707828 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ce3ebba80 0x563ce3ebde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.657+0000 7ff199d95740 1 --2- 172.21.15.12:0/3864707828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ce3eb2ea0 0x563ce3ebb540 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.657+0000 7ff199d95740 1 --2- 172.21.15.12:0/3864707828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563ce3eb2580 0x563ce3eb2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.657+0000 7ff199d95740 1 -- 172.21.15.12:0/3864707828 >> 172.21.15.12:0/3864707828 conn(0x563ce3eaccb0 msgr2=0x563ce3ead090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.657+0000 7ff199d95740 1 -- 172.21.15.12:0/3864707828 shutdown_connections 2024-04-22T13:33:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.657+0000 7ff199d95740 1 -- 172.21.15.12:0/3864707828 wait complete. 2024-04-22T13:33:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.657+0000 7ff199d95740 1 -- start start 2024-04-22T13:33:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.657+0000 7ff199d95740 1 Processor -- start 2024-04-22T13:33:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.658+0000 7ff199d95740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563ce3eb2580 0x563ce3eb8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.658+0000 7ff199d95740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ce3eb2ea0 0x563ce3eb8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.658+0000 7ff199d95740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ce3ebba80 0x563ce3eb9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.658+0000 7ff199d95740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563ce3ebe7b0 con 0x563ce3eb2ea0 2024-04-22T13:33:01.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.658+0000 7ff199d95740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x563ce3dd02f0 con 0x563ce3eb2580 2024-04-22T13:33:01.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.658+0000 7ff199d95740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563ce3ebe930 con 0x563ce3ebba80 2024-04-22T13:33:01.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.658+0000 7ff199d95740 10 client.0 _ll_register_callbacks cb 0x563ce3ddd580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:01.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.658+0000 7ff18ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ce3eb2ea0 0x563ce3eb8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.658+0000 7ff18ffff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ce3eb2ea0 0x563ce3eb8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33732/0 (socket says 172.21.15.12:33732) 2024-04-22T13:33:01.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.658+0000 7ff18ffff640 1 -- 172.21.15.12:0/2740983280 learned_addr learned my addr 172.21.15.12:0/2740983280 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:01.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.658+0000 7ff19566b640 1 --2- 172.21.15.12:0/2740983280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ce3ebba80 0x563ce3eb9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.658+0000 7ff194e6a640 1 --2- 172.21.15.12:0/2740983280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563ce3eb2580 0x563ce3eb8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.658+0000 7ff18ffff640 1 -- 172.21.15.12:0/2740983280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ce3ebba80 msgr2=0x563ce3eb9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:01.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.658+0000 7ff18ffff640 1 --2- 172.21.15.12:0/2740983280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ce3ebba80 0x563ce3eb9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.658+0000 7ff18ffff640 1 -- 172.21.15.12:0/2740983280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563ce3eb2580 msgr2=0x563ce3eb8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:01.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.658+0000 7ff18ffff640 1 --2- 172.21.15.12:0/2740983280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563ce3eb2580 0x563ce3eb8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.658+0000 7ff18ffff640 1 -- 172.21.15.12:0/2740983280 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ce3b293f0 con 0x563ce3eb2ea0 2024-04-22T13:33:01.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.658+0000 7ff18ffff640 1 --2- 172.21.15.12:0/2740983280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ce3eb2ea0 0x563ce3eb8f40 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7ff184000c00 tx=0x7ff18402ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b574ed2f4951c77 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:01.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.658+0000 7ff16ffff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:33:01.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.659+0000 7ff16ffff640 1 -- 172.21.15.12:0/2740983280 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18402c5d0 con 0x563ce3eb2ea0 2024-04-22T13:33:01.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.659+0000 7ff16ffff640 1 -- 172.21.15.12:0/2740983280 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7ff184005030 con 0x563ce3eb2ea0 2024-04-22T13:33:01.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.659+0000 7ff199d95740 1 -- 172.21.15.12:0/2740983280 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1800da040 con 0x563ce3eb2ea0 2024-04-22T13:33:01.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.659+0000 7ff16ffff640 1 -- 172.21.15.12:0/2740983280 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff184005230 con 0x563ce3eb2ea0 2024-04-22T13:33:01.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.659+0000 7ff199d95740 10 client.18783 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:01.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.659+0000 7ff199d95740 1 -- 172.21.15.12:0/2740983280 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563ce3ae90f0 con 0x563ce3eb2ea0 2024-04-22T13:33:01.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.659+0000 7ff16ffff640 1 -- 172.21.15.12:0/2740983280 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1800da040 con 0x563ce3eb2ea0 2024-04-22T13:33:01.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.659+0000 7ff16ffff640 1 -- 172.21.15.12:0/2740983280 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff184050020 con 0x563ce3eb2ea0 2024-04-22T13:33:01.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.659+0000 7ff199d95740 10 client.18783 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:01.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.659+0000 7ff199d95740 10 client.18783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:33:01.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.659+0000 7ff199d95740 20 client.18783 populate_metadata read hostname 'smithi012' 2024-04-22T13:33:01.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.659+0000 7ff199d95740 1 --2- 172.21.15.12:0/2740983280 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x563ce3ecb930 0x563ce3ecdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.659+0000 7ff199d95740 4 client.18783 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:33:01.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.659+0000 7ff199d95740 1 -- 172.21.15.12:0/2740983280 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ce3e24c00 con 0x563ce3ecb930 2024-04-22T13:33:01.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.659+0000 7ff194e6a640 1 --2- 172.21.15.12:0/2740983280 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x563ce3ecb930 0x563ce3ecdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.660+0000 7ff16ffff640 10 client.18783 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:33:01.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.660+0000 7ff194e6a640 1 --2- 172.21.15.12:0/2740983280 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x563ce3ecb930 0x563ce3ecdd70 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=374a6c79110601fd server_cookie=fecaa7305a25fd02 in_seq=0 out_seq=0 2024-04-22T13:33:01.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.660+0000 7ff16ffff640 1 -- 172.21.15.12:0/2740983280 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563ce3e24c00 con 0x563ce3ecb930 2024-04-22T13:33:01.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.660+0000 7ff16ffff640 10 client.18783 handle_command_reply: tid=0 2024-04-22T13:33:01.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.661+0000 7ff199d95740 1 client.18783 shutdown 2024-04-22T13:33:01.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.661+0000 7ff199d95740 1 -- 172.21.15.12:0/2740983280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ce3eb2ea0 msgr2=0x563ce3eb8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:01.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.661+0000 7ff199d95740 1 --2- 172.21.15.12:0/2740983280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ce3eb2ea0 0x563ce3eb8f40 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7ff184000c00 tx=0x7ff18402ca10 comp rx=0 tx=0).stop 2024-04-22T13:33:01.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.661+0000 7ff199d95740 1 -- 172.21.15.12:0/2740983280 shutdown_connections 2024-04-22T13:33:01.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.661+0000 7ff199d95740 1 --2- 172.21.15.12:0/2740983280 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x563ce3ecb930 0x563ce3ecdd70 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.661+0000 7ff199d95740 1 --2- 172.21.15.12:0/2740983280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ce3ebba80 0x563ce3eb9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.661+0000 7ff199d95740 1 --2- 172.21.15.12:0/2740983280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ce3eb2ea0 0x563ce3eb8f40 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.661+0000 7ff16ffff640 0 client.18783 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:33:01.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.661+0000 7ff199d95740 1 --2- 172.21.15.12:0/2740983280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563ce3eb2580 0x563ce3eb8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.661+0000 7ff199d95740 1 -- 172.21.15.12:0/2740983280 >> 172.21.15.12:0/2740983280 conn(0x563ce3eaccb0 msgr2=0x563ce3eaea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:01.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.661+0000 7ff199d95740 1 -- 172.21.15.12:0/2740983280 shutdown_connections 2024-04-22T13:33:01.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.661+0000 7ff199d95740 1 -- 172.21.15.12:0/2740983280 wait complete. 2024-04-22T13:33:01.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.661+0000 7ff199d95740 20 client.18783 trim_cache size 0 max 16384 2024-04-22T13:33:01.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.679+0000 7ff199d95740 1 Processor -- start 2024-04-22T13:33:01.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.679+0000 7ff199d95740 1 -- start start 2024-04-22T13:33:01.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.679+0000 7ff199d95740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ce3eb2580 0x563ce3ebb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.679+0000 7ff199d95740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563ce3eb2ea0 0x563ce3ec7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.679+0000 7ff199d95740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ce3ebba80 0x563ce3eae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.679+0000 7ff199d95740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff1901afcf0 con 0x563ce3eb2580 2024-04-22T13:33:01.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.679+0000 7ff199d95740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff1901b1830 con 0x563ce3eb2ea0 2024-04-22T13:33:01.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.679+0000 7ff199d95740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff1901b03c0 con 0x563ce3ebba80 2024-04-22T13:33:01.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.679+0000 7ff194e6a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ce3eb2580 0x563ce3ebb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.679+0000 7ff194e6a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ce3eb2580 0x563ce3ebb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33738/0 (socket says 172.21.15.12:33738) 2024-04-22T13:33:01.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.679+0000 7ff194e6a640 1 -- 172.21.15.12:0/4041493462 learned_addr learned my addr 172.21.15.12:0/4041493462 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:01.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.680+0000 7ff19566b640 1 --2- 172.21.15.12:0/4041493462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ce3ebba80 0x563ce3eae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.680+0000 7ff18ffff640 1 --2- 172.21.15.12:0/4041493462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563ce3eb2ea0 0x563ce3ec7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.680+0000 7ff194e6a640 1 -- 172.21.15.12:0/4041493462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ce3ebba80 msgr2=0x563ce3eae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:01.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.680+0000 7ff194e6a640 1 --2- 172.21.15.12:0/4041493462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ce3ebba80 0x563ce3eae7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.680+0000 7ff194e6a640 1 -- 172.21.15.12:0/4041493462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563ce3eb2ea0 msgr2=0x563ce3ec7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:01.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.680+0000 7ff194e6a640 1 --2- 172.21.15.12:0/4041493462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563ce3eb2ea0 0x563ce3ec7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.680+0000 7ff194e6a640 1 -- 172.21.15.12:0/4041493462 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ce3eacae0 con 0x563ce3eb2580 2024-04-22T13:33:01.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.680+0000 7ff194e6a640 1 --2- 172.21.15.12:0/4041493462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ce3eb2580 0x563ce3ebb130 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7ff18009ad30 tx=0x7ff1800d4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f2a6988164dee2 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:01.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.680+0000 7ff1767fc640 1 -- 172.21.15.12:0/4041493462 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1800d9040 con 0x563ce3eb2580 2024-04-22T13:33:01.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.680+0000 7ff1767fc640 1 -- 172.21.15.12:0/4041493462 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1800d7070 con 0x563ce3eb2580 2024-04-22T13:33:01.686 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.680+0000 7ff1767fc640 1 -- 172.21.15.12:0/4041493462 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1800990f0 con 0x563ce3eb2580 2024-04-22T13:33:01.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.681+0000 7ff199d95740 1 -- 172.21.15.12:0/4041493462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ce3eb2580 msgr2=0x563ce3ebb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:01.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.681+0000 7ff199d95740 1 --2- 172.21.15.12:0/4041493462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ce3eb2580 0x563ce3ebb130 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7ff18009ad30 tx=0x7ff1800d4040 comp rx=0 tx=0).stop 2024-04-22T13:33:01.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.681+0000 7ff199d95740 1 -- 172.21.15.12:0/4041493462 shutdown_connections 2024-04-22T13:33:01.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.681+0000 7ff199d95740 1 --2- 172.21.15.12:0/4041493462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ce3ebba80 0x563ce3eae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.681+0000 7ff199d95740 1 --2- 172.21.15.12:0/4041493462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563ce3eb2ea0 0x563ce3ec7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.681+0000 7ff199d95740 1 --2- 172.21.15.12:0/4041493462 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ce3eb2580 0x563ce3ebb130 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.681+0000 7ff199d95740 1 -- 172.21.15.12:0/4041493462 >> 172.21.15.12:0/4041493462 conn(0x563ce3eaccb0 msgr2=0x563ce3eae0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:01.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.681+0000 7ff199d95740 1 -- 172.21.15.12:0/4041493462 shutdown_connections 2024-04-22T13:33:01.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.681+0000 7ff199d95740 1 -- 172.21.15.12:0/4041493462 wait complete. 2024-04-22T13:33:01.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.681+0000 7ff199d95740 1 -- start start 2024-04-22T13:33:01.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.681+0000 7ff199d95740 1 Processor -- start 2024-04-22T13:33:01.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.682+0000 7ff199d95740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563ce3eb2580 0x563ce3eb5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.682+0000 7ff199d95740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ce3eb2ea0 0x563ce3eb5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.687 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.682+0000 7ff199d95740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ce3ebba80 0x563ce3eb60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.682+0000 7ff199d95740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff1901b1160 con 0x563ce3ebba80 2024-04-22T13:33:01.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.682+0000 7ff199d95740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff1901b0a90 con 0x563ce3eb2580 2024-04-22T13:33:01.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.682+0000 7ff199d95740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff1901adcf0 con 0x563ce3eb2ea0 2024-04-22T13:33:01.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.682+0000 7ff199d95740 10 client.0 _ll_register_callbacks cb 0x7ff1880093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:01.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.682+0000 7ff19566b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ce3ebba80 0x563ce3eb60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.682+0000 7ff19566b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ce3ebba80 0x563ce3eb60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:33748/0 (socket says 172.21.15.12:33748) 2024-04-22T13:33:01.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.682+0000 7ff19566b640 1 -- 172.21.15.12:0/3036424037 learned_addr learned my addr 172.21.15.12:0/3036424037 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:01.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.682+0000 7ff18ffff640 1 --2- 172.21.15.12:0/3036424037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ce3eb2ea0 0x563ce3eb5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.682+0000 7ff194e6a640 1 --2- 172.21.15.12:0/3036424037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563ce3eb2580 0x563ce3eb5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.682+0000 7ff19566b640 1 -- 172.21.15.12:0/3036424037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ce3eb2ea0 msgr2=0x563ce3eb5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:01.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.682+0000 7ff19566b640 1 --2- 172.21.15.12:0/3036424037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ce3eb2ea0 0x563ce3eb5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.682+0000 7ff19566b640 1 -- 172.21.15.12:0/3036424037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563ce3eb2580 msgr2=0x563ce3eb5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:01.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.682+0000 7ff19566b640 1 --2- 172.21.15.12:0/3036424037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563ce3eb2580 0x563ce3eb5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.688 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.682+0000 7ff19566b640 1 -- 172.21.15.12:0/3036424037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ce3f30eb0 con 0x563ce3ebba80 2024-04-22T13:33:01.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.682+0000 7ff19566b640 1 --2- 172.21.15.12:0/3036424037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ce3ebba80 0x563ce3eb60b0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7ff18800fec0 tx=0x7ff188004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc235c3d1a9d6c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:01.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.682+0000 7ff16ffff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:33:01.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.682+0000 7ff16ffff640 1 -- 172.21.15.12:0/3036424037 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff188019070 con 0x563ce3ebba80 2024-04-22T13:33:01.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.682+0000 7ff16ffff640 1 -- 172.21.15.12:0/3036424037 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7ff188010cc0 con 0x563ce3ebba80 2024-04-22T13:33:01.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.682+0000 7ff199d95740 1 -- 172.21.15.12:0/3036424037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1800d7070 con 0x563ce3ebba80 2024-04-22T13:33:01.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.683+0000 7ff16ffff640 1 -- 172.21.15.12:0/3036424037 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff188004b20 con 0x563ce3ebba80 2024-04-22T13:33:01.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.683+0000 7ff199d95740 10 client.18789 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:01.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.683+0000 7ff199d95740 1 -- 172.21.15.12:0/3036424037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563ce3f32180 con 0x563ce3ebba80 2024-04-22T13:33:01.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.683+0000 7ff16ffff640 1 -- 172.21.15.12:0/3036424037 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1800d7070 con 0x563ce3ebba80 2024-04-22T13:33:01.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.683+0000 7ff16ffff640 1 -- 172.21.15.12:0/3036424037 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff18801f420 con 0x563ce3ebba80 2024-04-22T13:33:01.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.683+0000 7ff199d95740 10 client.18789 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:01.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.683+0000 7ff199d95740 10 client.18789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:33:01.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.683+0000 7ff199d95740 20 client.18789 populate_metadata read hostname 'smithi012' 2024-04-22T13:33:01.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.683+0000 7ff199d95740 1 --2- 172.21.15.12:0/3036424037 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x563ce3f32980 0x563ce3f34da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:01.689 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.683+0000 7ff199d95740 4 client.18789 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:33:01.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.683+0000 7ff199d95740 1 -- 172.21.15.12:0/3036424037 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563ce3e1ea70 con 0x563ce3f32980 2024-04-22T13:33:01.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.683+0000 7ff194e6a640 1 --2- 172.21.15.12:0/3036424037 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x563ce3f32980 0x563ce3f34da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:01.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.684+0000 7ff194e6a640 1 --2- 172.21.15.12:0/3036424037 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x563ce3f32980 0x563ce3f34da0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c449994066b56654 server_cookie=6a3f9ff84541daea in_seq=0 out_seq=0 2024-04-22T13:33:01.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.684+0000 7ff16ffff640 10 client.18789 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:33:01.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.684+0000 7ff16ffff640 1 -- 172.21.15.12:0/3036424037 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563ce3e1ea70 con 0x563ce3f32980 2024-04-22T13:33:01.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.684+0000 7ff16ffff640 10 client.18789 handle_command_reply: tid=0 2024-04-22T13:33:01.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.684+0000 7ff199d95740 1 client.18789 shutdown 2024-04-22T13:33:01.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.684+0000 7ff199d95740 1 -- 172.21.15.12:0/3036424037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ce3ebba80 msgr2=0x563ce3eb60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:01.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.684+0000 7ff199d95740 1 --2- 172.21.15.12:0/3036424037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ce3ebba80 0x563ce3eb60b0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7ff18800fec0 tx=0x7ff188004a40 comp rx=0 tx=0).stop 2024-04-22T13:33:01.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.684+0000 7ff199d95740 1 -- 172.21.15.12:0/3036424037 shutdown_connections 2024-04-22T13:33:01.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.684+0000 7ff199d95740 1 --2- 172.21.15.12:0/3036424037 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x563ce3f32980 0x563ce3f34da0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.684+0000 7ff199d95740 1 --2- 172.21.15.12:0/3036424037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563ce3ebba80 0x563ce3eb60b0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.684+0000 7ff199d95740 1 --2- 172.21.15.12:0/3036424037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563ce3eb2ea0 0x563ce3eb5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.690 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.684+0000 7ff16ffff640 0 client.18789 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:33:01.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.684+0000 7ff199d95740 1 --2- 172.21.15.12:0/3036424037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563ce3eb2580 0x563ce3eb5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.691 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.684+0000 7ff199d95740 1 -- 172.21.15.12:0/3036424037 >> 172.21.15.12:0/3036424037 conn(0x563ce3eaccb0 msgr2=0x563ce3eab4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:01.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.684+0000 7ff199d95740 1 -- 172.21.15.12:0/3036424037 shutdown_connections 2024-04-22T13:33:01.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.684+0000 7ff199d95740 1 -- 172.21.15.12:0/3036424037 wait complete. 2024-04-22T13:33:01.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.684+0000 7ff199d95740 20 client.18789 trim_cache size 0 max 16384 2024-04-22T13:33:01.692 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:33:01.692 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:33:01.692 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-04-22T13:33:01.692 INFO:teuthology.orchestra.run.smithi012.stdout: "872e28f7-70ae-412e-a360-1e6233cd104c": { 2024-04-22T13:33:01.692 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-04-22T13:33:01.693 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "872e28f7-70ae-412e-a360-1e6233cd104c", 2024-04-22T13:33:01.693 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-04-22T13:33:01.693 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:33:01.693 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:33:01.693 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:33:01.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.685+0000 7ff1970ce640 1 -- 172.21.15.12:0/2065567681 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff1780a3dd0 msgr2=0x7ff1780a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:01.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.685+0000 7ff1970ce640 1 --2- 172.21.15.12:0/2065567681 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff1780a3dd0 0x7ff1780a6290 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff188005fd0 tx=0x7ff188004ab0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.685+0000 7ff1970ce640 1 -- 172.21.15.12:0/2065567681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff190101080 msgr2=0x7ff190102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:01.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.685+0000 7ff1970ce640 1 --2- 172.21.15.12:0/2065567681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff190101080 0x7ff190102b10 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7ff18000e9f0 tx=0x7ff18000eeb0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.685+0000 7ff1970ce640 1 -- 172.21.15.12:0/2065567681 shutdown_connections 2024-04-22T13:33:01.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.685+0000 7ff1970ce640 1 --2- 172.21.15.12:0/2065567681 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff1780a3dd0 0x7ff1780a6290 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.685+0000 7ff1970ce640 1 --2- 172.21.15.12:0/2065567681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff190103590 0x7ff1901a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.685+0000 7ff1970ce640 1 --2- 172.21.15.12:0/2065567681 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff190106150 0x7ff190103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.685+0000 7ff1970ce640 1 --2- 172.21.15.12:0/2065567681 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff190101080 0x7ff190102b10 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:01.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.685+0000 7ff1970ce640 1 -- 172.21.15.12:0/2065567681 >> 172.21.15.12:0/2065567681 conn(0x7ff190078000 msgr2=0x7ff1900693f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:01.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.685+0000 7ff1970ce640 1 -- 172.21.15.12:0/2065567681 shutdown_connections 2024-04-22T13:33:01.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:01.685+0000 7ff1970ce640 1 -- 172.21.15.12:0/2065567681 wait complete. 2024-04-22T13:33:01.695 INFO:tasks.cephfs.filesystem:scrub status for tag:872e28f7-70ae-412e-a360-1e6233cd104c - {'path': '/', 'tag': '872e28f7-70ae-412e-a360-1e6233cd104c', 'options': 'recursive,force'} 2024-04-22T13:33:01.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:01 smithi134 ceph-mon[31338]: pgmap v274: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:01.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:01 smithi111 ceph-mon[31361]: pgmap v274: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:02.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:01 smithi012 ceph-mon[28265]: pgmap v274: 97 pgs: 97 active+clean; 2.3 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:02.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:02 smithi134 ceph-mon[31338]: scrub queued for path: / 2024-04-22T13:33:02.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:02 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:33:02.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:02 smithi134 ceph-mon[31338]: scrub summary: active paths [/] 2024-04-22T13:33:02.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:02 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:33:02.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:02 smithi111 ceph-mon[31361]: scrub queued for path: / 2024-04-22T13:33:02.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:02 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:33:02.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:02 smithi111 ceph-mon[31361]: scrub summary: active paths [/] 2024-04-22T13:33:02.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:02 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:33:03.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:02 smithi012 ceph-mon[28265]: scrub queued for path: / 2024-04-22T13:33:03.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:02 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:33:03.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:02 smithi012 ceph-mon[28265]: scrub summary: active paths [/] 2024-04-22T13:33:03.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:02 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:33:03.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:03 smithi134 ceph-mon[31338]: pgmap v275: 97 pgs: 97 active+clean; 2.3 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-22T13:33:03.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:03 smithi111 ceph-mon[31361]: pgmap v275: 97 pgs: 97 active+clean; 2.3 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-22T13:33:04.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:03 smithi012 ceph-mon[28265]: pgmap v275: 97 pgs: 97 active+clean; 2.3 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-22T13:33:05.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:05 smithi111 ceph-mon[31361]: pgmap v276: 97 pgs: 97 active+clean; 2.3 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-22T13:33:05.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:05 smithi111 ceph-mon[31361]: scrub summary: idle 2024-04-22T13:33:06.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:05 smithi012 ceph-mon[28265]: pgmap v276: 97 pgs: 97 active+clean; 2.3 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-22T13:33:06.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:05 smithi012 ceph-mon[28265]: scrub summary: idle 2024-04-22T13:33:06.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:05 smithi134 ceph-mon[31338]: pgmap v276: 97 pgs: 97 active+clean; 2.3 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-22T13:33:06.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:05 smithi134 ceph-mon[31338]: scrub summary: idle 2024-04-22T13:33:07.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:07 smithi111 ceph-mon[31361]: pgmap v277: 97 pgs: 97 active+clean; 2.3 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-22T13:33:08.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:07 smithi012 ceph-mon[28265]: pgmap v277: 97 pgs: 97 active+clean; 2.3 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-22T13:33:08.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:07 smithi134 ceph-mon[31338]: pgmap v277: 97 pgs: 97 active+clean; 2.3 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-22T13:33:09.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:09 smithi111 ceph-mon[31361]: pgmap v278: 97 pgs: 97 active+clean; 2.3 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-22T13:33:10.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:09 smithi012 ceph-mon[28265]: pgmap v278: 97 pgs: 97 active+clean; 2.3 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-22T13:33:10.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:09 smithi134 ceph-mon[31338]: pgmap v278: 97 pgs: 97 active+clean; 2.3 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-22T13:33:11.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:11 smithi111 ceph-mon[31361]: pgmap v279: 97 pgs: 97 active+clean; 2.3 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-22T13:33:11.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:11 smithi111 ceph-mon[31361]: osdmap e95: 12 total, 12 up, 12 in 2024-04-22T13:33:12.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:11 smithi012 ceph-mon[28265]: pgmap v279: 97 pgs: 97 active+clean; 2.3 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-22T13:33:12.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:11 smithi012 ceph-mon[28265]: osdmap e95: 12 total, 12 up, 12 in 2024-04-22T13:33:12.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:11 smithi134 ceph-mon[31338]: pgmap v279: 97 pgs: 97 active+clean; 2.3 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-22T13:33:12.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:11 smithi134 ceph-mon[31338]: osdmap e95: 12 total, 12 up, 12 in 2024-04-22T13:33:13.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:13 smithi111 ceph-mon[31361]: pgmap v281: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:33:14.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:13 smithi012 ceph-mon[28265]: pgmap v281: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:33:14.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:13 smithi134 ceph-mon[31338]: pgmap v281: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:33:15.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:15 smithi012 ceph-mon[28265]: pgmap v282: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:33:15.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:15 smithi012 ceph-mon[28265]: osdmap e96: 12 total, 12 up, 12 in 2024-04-22T13:33:15.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:15 smithi134 ceph-mon[31338]: pgmap v282: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:33:15.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:15 smithi134 ceph-mon[31338]: osdmap e96: 12 total, 12 up, 12 in 2024-04-22T13:33:15.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:15 smithi111 ceph-mon[31361]: pgmap v282: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:33:15.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:15 smithi111 ceph-mon[31361]: osdmap e96: 12 total, 12 up, 12 in 2024-04-22T13:33:17.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:17 smithi012 ceph-mon[28265]: pgmap v284: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-22T13:33:17.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:17 smithi134 ceph-mon[31338]: pgmap v284: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-22T13:33:17.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:17 smithi111 ceph-mon[31361]: pgmap v284: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-22T13:33:19.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:19 smithi012 ceph-mon[28265]: pgmap v285: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-22T13:33:19.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:19 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:33:19.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:19 smithi134 ceph-mon[31338]: pgmap v285: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-22T13:33:19.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:19 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:33:19.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:19 smithi111 ceph-mon[31361]: pgmap v285: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-22T13:33:19.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:19 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:33:20.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:20 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:33:20.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:20 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:33:20.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:20 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:33:20.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:20 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:33:20.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:20 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:33:20.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:20 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:33:20.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:20 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:33:20.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:20 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:33:20.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:20 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:33:21.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:21 smithi134 ceph-mon[31338]: pgmap v286: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 212 B/s wr, 10 op/s 2024-04-22T13:33:21.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:21 smithi111 ceph-mon[31361]: pgmap v286: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 212 B/s wr, 10 op/s 2024-04-22T13:33:22.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:21 smithi012 ceph-mon[28265]: pgmap v286: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 212 B/s wr, 10 op/s 2024-04-22T13:33:23.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:23 smithi134 ceph-mon[31338]: pgmap v287: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-22T13:33:23.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:23 smithi111 ceph-mon[31361]: pgmap v287: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-22T13:33:24.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:23 smithi012 ceph-mon[28265]: pgmap v287: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-22T13:33:25.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:25 smithi134 ceph-mon[31338]: pgmap v288: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-22T13:33:25.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:25 smithi111 ceph-mon[31361]: pgmap v288: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-22T13:33:26.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:25 smithi012 ceph-mon[28265]: pgmap v288: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-22T13:33:27.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:27 smithi134 ceph-mon[31338]: pgmap v289: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 86 B/s wr, 8 op/s 2024-04-22T13:33:27.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:27 smithi111 ceph-mon[31361]: pgmap v289: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 86 B/s wr, 8 op/s 2024-04-22T13:33:28.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:27 smithi012 ceph-mon[28265]: pgmap v289: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 86 B/s wr, 8 op/s 2024-04-22T13:33:29.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:29 smithi134 ceph-mon[31338]: pgmap v290: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-22T13:33:29.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:29 smithi111 ceph-mon[31361]: pgmap v290: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-22T13:33:30.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:29 smithi012 ceph-mon[28265]: pgmap v290: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-22T13:33:31.696 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:33:31.696 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:33:31.814 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.819+0000 7f1ff299f640 1 -- 172.21.15.12:0/2539587576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fec10c350 msgr2=0x7f1fec10e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:31.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.819+0000 7f1ff299f640 1 --2- 172.21.15.12:0/2539587576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fec10c350 0x7f1fec10e7a0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4009ca0 tx=0x7f1fe402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.819+0000 7f1ff299f640 1 -- 172.21.15.12:0/2539587576 shutdown_connections 2024-04-22T13:33:31.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.819+0000 7f1ff299f640 1 --2- 172.21.15.12:0/2539587576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fec10c350 0x7f1fec10e7a0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.819+0000 7f1ff299f640 1 --2- 172.21.15.12:0/2539587576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fec103800 0x7f1fec10bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.819+0000 7f1ff299f640 1 --2- 172.21.15.12:0/2539587576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1fec102ee0 0x7f1fec1032c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.819+0000 7f1ff299f640 1 -- 172.21.15.12:0/2539587576 >> 172.21.15.12:0/2539587576 conn(0x7f1fec0fc6e0 msgr2=0x7f1fec0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:31.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.819+0000 7f1ff299f640 1 -- 172.21.15.12:0/2539587576 shutdown_connections 2024-04-22T13:33:31.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.819+0000 7f1ff299f640 1 -- 172.21.15.12:0/2539587576 wait complete. 2024-04-22T13:33:31.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.819+0000 7f1ff299f640 1 Processor -- start 2024-04-22T13:33:31.815 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.820+0000 7f1ff299f640 1 -- start start 2024-04-22T13:33:31.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.820+0000 7f1ff299f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fec102ee0 0x7f1fec100b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:31.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.820+0000 7f1ff299f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fec103800 0x7f1fec101070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:31.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.820+0000 7f1ff299f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1fec10c350 0x7f1fec1015b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:31.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.820+0000 7f1ff299f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec111f90 con 0x7f1fec103800 2024-04-22T13:33:31.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.820+0000 7f1ff299f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec111e10 con 0x7f1fec10c350 2024-04-22T13:33:31.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.820+0000 7f1ff299f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec112110 con 0x7f1fec102ee0 2024-04-22T13:33:31.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.820+0000 7f1ff119c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fec103800 0x7f1fec101070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:31.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.820+0000 7f1ff119c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fec103800 0x7f1fec101070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60424/0 (socket says 172.21.15.12:60424) 2024-04-22T13:33:31.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.820+0000 7f1ff199d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fec102ee0 0x7f1fec100b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:31.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.820+0000 7f1ff119c640 1 -- 172.21.15.12:0/2206558618 learned_addr learned my addr 172.21.15.12:0/2206558618 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:31.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.820+0000 7f1ff219e640 1 --2- 172.21.15.12:0/2206558618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1fec10c350 0x7f1fec1015b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:31.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.820+0000 7f1ff119c640 1 -- 172.21.15.12:0/2206558618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fec102ee0 msgr2=0x7f1fec100b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:31.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.820+0000 7f1ff119c640 1 --2- 172.21.15.12:0/2206558618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fec102ee0 0x7f1fec100b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.820+0000 7f1ff119c640 1 -- 172.21.15.12:0/2206558618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1fec10c350 msgr2=0x7f1fec1015b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:31.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.820+0000 7f1ff119c640 1 --2- 172.21.15.12:0/2206558618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1fec10c350 0x7f1fec1015b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.820+0000 7f1ff119c640 1 -- 172.21.15.12:0/2206558618 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fec1a6950 con 0x7f1fec103800 2024-04-22T13:33:31.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.821+0000 7f1ff119c640 1 --2- 172.21.15.12:0/2206558618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fec103800 0x7f1fec101070 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc00d950 tx=0x7f1fdc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:31.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.821+0000 7f1fe2ffd640 1 -- 172.21.15.12:0/2206558618 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc017070 con 0x7f1fec103800 2024-04-22T13:33:31.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.821+0000 7f1fe2ffd640 1 -- 172.21.15.12:0/2206558618 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fdc009c30 con 0x7f1fec103800 2024-04-22T13:33:31.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.821+0000 7f1fe2ffd640 1 -- 172.21.15.12:0/2206558618 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc00f620 con 0x7f1fec103800 2024-04-22T13:33:31.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.821+0000 7f1ff299f640 1 -- 172.21.15.12:0/2206558618 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fec1a6c40 con 0x7f1fec103800 2024-04-22T13:33:31.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.821+0000 7f1ff299f640 1 -- 172.21.15.12:0/2206558618 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fec1a7190 con 0x7f1fec103800 2024-04-22T13:33:31.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.823+0000 7f1fe2ffd640 1 -- 172.21.15.12:0/2206558618 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1fdc01a070 con 0x7f1fec103800 2024-04-22T13:33:31.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.824+0000 7f1ff67ea740 1 Processor -- start 2024-04-22T13:33:31.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.824+0000 7f1ff67ea740 1 -- start start 2024-04-22T13:33:31.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.824+0000 7f1ff67ea740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314245b710 0x56314245baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:31.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.824+0000 7f1ff67ea740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56314245c030 0x5631424646d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:31.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.824+0000 7f1ff67ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563142464c10 0x563142467000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:31.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.824+0000 7f1ff67ea740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5631423c01e0 con 0x56314245b710 2024-04-22T13:33:31.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.824+0000 7f1ff67ea740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56314235d390 con 0x56314245c030 2024-04-22T13:33:31.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.824+0000 7f1ff67ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5631423bc1a0 con 0x563142464c10 2024-04-22T13:33:31.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.824+0000 7f1fe2ffd640 1 --2- 172.21.15.12:0/2206558618 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f1fd80a3dd0 0x7f1fd80a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:31.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.825+0000 7f1ff219e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314245b710 0x56314245baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:31.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.825+0000 7f1ff219e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314245b710 0x56314245baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60438/0 (socket says 172.21.15.12:60438) 2024-04-22T13:33:31.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.825+0000 7f1ff219e640 1 -- 172.21.15.12:0/935994736 learned_addr learned my addr 172.21.15.12:0/935994736 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:31.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.825+0000 7f1ff219e640 1 --2- 172.21.15.12:0/2206558618 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f1fd80a3dd0 0x7f1fd80a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:31.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.825+0000 7f1ff199d640 1 --2- 172.21.15.12:0/935994736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56314245c030 0x5631424646d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:31.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.825+0000 7f1ff119c640 1 --2- 172.21.15.12:0/935994736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563142464c10 0x563142467000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:31.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.825+0000 7f1fe2ffd640 1 -- 172.21.15.12:0/2206558618 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f1fdc0d0100 con 0x7f1fec103800 2024-04-22T13:33:31.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.825+0000 7f1ff219e640 1 -- 172.21.15.12:0/935994736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563142464c10 msgr2=0x563142467000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:31.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.825+0000 7f1ff219e640 1 --2- 172.21.15.12:0/935994736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563142464c10 0x563142467000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.825+0000 7f1ff219e640 1 -- 172.21.15.12:0/935994736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56314245c030 msgr2=0x5631424646d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:31.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.825+0000 7f1ff219e640 1 --2- 172.21.15.12:0/935994736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56314245c030 0x5631424646d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.825+0000 7f1ff219e640 1 -- 172.21.15.12:0/935994736 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56314219c7d0 con 0x56314245b710 2024-04-22T13:33:31.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.825+0000 7f1ff199d640 1 --2- 172.21.15.12:0/935994736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56314245c030 0x5631424646d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:33:31.831 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.825+0000 7f1ff219e640 1 --2- 172.21.15.12:0/935994736 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314245b710 0x56314245baf0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4037040 tx=0x7f1fe40324c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9019ed716c0216aa server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:31.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.825+0000 7f1ff219e640 1 --2- 172.21.15.12:0/2206558618 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f1fd80a3dd0 0x7f1fd80a6290 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4009ca0 tx=0x7f1fe4042000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:31.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.825+0000 7f1fd7fff640 1 -- 172.21.15.12:0/935994736 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4051070 con 0x56314245b710 2024-04-22T13:33:31.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.826+0000 7f1fd7fff640 1 -- 172.21.15.12:0/935994736 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fe4032770 con 0x56314245b710 2024-04-22T13:33:31.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.826+0000 7f1fd7fff640 1 -- 172.21.15.12:0/935994736 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4039040 con 0x56314245b710 2024-04-22T13:33:31.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.826+0000 7f1ff67ea740 1 -- 172.21.15.12:0/935994736 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314245b710 msgr2=0x56314245baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:31.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.826+0000 7f1ff67ea740 1 --2- 172.21.15.12:0/935994736 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314245b710 0x56314245baf0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4037040 tx=0x7f1fe40324c0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.826+0000 7f1ff67ea740 1 -- 172.21.15.12:0/935994736 shutdown_connections 2024-04-22T13:33:31.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.826+0000 7f1ff67ea740 1 --2- 172.21.15.12:0/935994736 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563142464c10 0x563142467000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.826+0000 7f1ff67ea740 1 --2- 172.21.15.12:0/935994736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56314245c030 0x5631424646d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.826+0000 7f1ff67ea740 1 --2- 172.21.15.12:0/935994736 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314245b710 0x56314245baf0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.826+0000 7f1ff67ea740 1 -- 172.21.15.12:0/935994736 >> 172.21.15.12:0/935994736 conn(0x563142455e40 msgr2=0x563142456220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:31.832 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.826+0000 7f1ff67ea740 1 -- 172.21.15.12:0/935994736 shutdown_connections 2024-04-22T13:33:31.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.826+0000 7f1ff67ea740 1 -- 172.21.15.12:0/935994736 wait complete. 2024-04-22T13:33:31.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.827+0000 7f1ff67ea740 1 -- start start 2024-04-22T13:33:31.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.827+0000 7f1ff67ea740 1 Processor -- start 2024-04-22T13:33:31.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.827+0000 7f1ff67ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56314245b710 0x563142461c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:31.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.827+0000 7f1ff67ea740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314245c030 0x563142462150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:31.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.827+0000 7f1ff67ea740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563142464c10 0x563142462690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:31.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.827+0000 7f1ff67ea740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5631424679a0 con 0x56314245c030 2024-04-22T13:33:31.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.827+0000 7f1ff67ea740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5631423792f0 con 0x563142464c10 2024-04-22T13:33:31.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.827+0000 7f1ff67ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563142467b20 con 0x56314245b710 2024-04-22T13:33:31.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.827+0000 7f1ff67ea740 10 client.0 _ll_register_callbacks cb 0x563142386580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:31.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.827+0000 7f1ff119c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314245c030 0x563142462150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:31.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.827+0000 7f1ff119c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314245c030 0x563142462150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60444/0 (socket says 172.21.15.12:60444) 2024-04-22T13:33:31.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.827+0000 7f1ff119c640 1 -- 172.21.15.12:0/1147836752 learned_addr learned my addr 172.21.15.12:0/1147836752 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:31.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.828+0000 7f1ff199d640 1 --2- 172.21.15.12:0/1147836752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56314245b710 0x563142461c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:31.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.828+0000 7f1ff119c640 1 -- 172.21.15.12:0/1147836752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56314245b710 msgr2=0x563142461c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:31.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.828+0000 7f1ff119c640 1 --2- 172.21.15.12:0/1147836752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56314245b710 0x563142461c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.828+0000 7f1ff119c640 1 -- 172.21.15.12:0/1147836752 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563142464c10 msgr2=0x563142462690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:31.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.828+0000 7f1ff219e640 1 --2- 172.21.15.12:0/1147836752 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563142464c10 0x563142462690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:31.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.828+0000 7f1ff119c640 1 --2- 172.21.15.12:0/1147836752 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563142464c10 0x563142462690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.828+0000 7f1ff119c640 1 -- 172.21.15.12:0/1147836752 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631420d23f0 con 0x56314245c030 2024-04-22T13:33:31.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.828+0000 7f1ff219e640 1 --2- 172.21.15.12:0/1147836752 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563142464c10 0x563142462690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:31.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.828+0000 7f1ff119c640 1 --2- 172.21.15.12:0/1147836752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314245c030 0x563142462150 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc098570 tx=0x7f1fdc09ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315344655e126bbc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:31.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.828+0000 7f1fd57fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:33:31.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.828+0000 7f1fd57fa640 1 -- 172.21.15.12:0/1147836752 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc021040 con 0x56314245c030 2024-04-22T13:33:31.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.828+0000 7f1fd57fa640 1 -- 172.21.15.12:0/1147836752 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f1fdc0cda90 con 0x56314245c030 2024-04-22T13:33:31.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.828+0000 7f1ff67ea740 1 -- 172.21.15.12:0/1147836752 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fe4032770 con 0x56314245c030 2024-04-22T13:33:31.834 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.828+0000 7f1fd57fa640 1 -- 172.21.15.12:0/1147836752 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1fdc0cdc90 con 0x56314245c030 2024-04-22T13:33:31.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.828+0000 7f1ff67ea740 10 client.18801 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:31.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.828+0000 7f1ff67ea740 1 -- 172.21.15.12:0/1147836752 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5631420920f0 con 0x56314245c030 2024-04-22T13:33:31.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.828+0000 7f1fd57fa640 1 -- 172.21.15.12:0/1147836752 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fe4032770 con 0x56314245c030 2024-04-22T13:33:31.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.829+0000 7f1fd57fa640 1 -- 172.21.15.12:0/1147836752 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1fdc097ff0 con 0x56314245c030 2024-04-22T13:33:31.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.829+0000 7f1ff67ea740 10 client.18801 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:31.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.829+0000 7f1ff67ea740 10 client.18801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:33:31.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.829+0000 7f1ff67ea740 20 client.18801 populate_metadata read hostname 'smithi012' 2024-04-22T13:33:31.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.829+0000 7f1ff67ea740 1 --2- 172.21.15.12:0/1147836752 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x563142474c40 0x563142477030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:31.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.829+0000 7f1ff67ea740 4 client.18801 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:33:31.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.829+0000 7f1ff67ea740 1 -- 172.21.15.12:0/1147836752 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631423cdc00 con 0x563142474c40 2024-04-22T13:33:31.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.829+0000 7f1ff199d640 1 --2- 172.21.15.12:0/1147836752 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x563142474c40 0x563142477030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:31.835 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.829+0000 7f1ff199d640 1 --2- 172.21.15.12:0/1147836752 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x563142474c40 0x563142477030 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28fee28dc79c92c0 server_cookie=abc497be407cd44d in_seq=0 out_seq=0 2024-04-22T13:33:31.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.829+0000 7f1fd57fa640 10 client.18801 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:33:31.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.830+0000 7f1fd57fa640 1 -- 172.21.15.12:0/1147836752 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5631423cdc00 con 0x563142474c40 2024-04-22T13:33:31.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.830+0000 7f1fd57fa640 10 client.18801 handle_command_reply: tid=0 2024-04-22T13:33:31.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.830+0000 7f1ff67ea740 1 client.18801 shutdown 2024-04-22T13:33:31.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.830+0000 7f1ff67ea740 1 -- 172.21.15.12:0/1147836752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314245c030 msgr2=0x563142462150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:31.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.830+0000 7f1ff67ea740 1 --2- 172.21.15.12:0/1147836752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314245c030 0x563142462150 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc098570 tx=0x7f1fdc09ba70 comp rx=0 tx=0).stop 2024-04-22T13:33:31.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.830+0000 7f1ff67ea740 1 -- 172.21.15.12:0/1147836752 shutdown_connections 2024-04-22T13:33:31.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.830+0000 7f1ff67ea740 1 --2- 172.21.15.12:0/1147836752 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x563142474c40 0x563142477030 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.830+0000 7f1ff67ea740 1 --2- 172.21.15.12:0/1147836752 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563142464c10 0x563142462690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.830+0000 7f1ff67ea740 1 --2- 172.21.15.12:0/1147836752 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314245c030 0x563142462150 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.830+0000 7f1fd57fa640 0 client.18801 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:33:31.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.830+0000 7f1ff67ea740 1 --2- 172.21.15.12:0/1147836752 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56314245b710 0x563142461c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.830+0000 7f1ff67ea740 1 -- 172.21.15.12:0/1147836752 >> 172.21.15.12:0/1147836752 conn(0x563142455e40 msgr2=0x563142455120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:31.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.830+0000 7f1ff67ea740 1 -- 172.21.15.12:0/1147836752 shutdown_connections 2024-04-22T13:33:31.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.830+0000 7f1ff67ea740 1 -- 172.21.15.12:0/1147836752 wait complete. 2024-04-22T13:33:31.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.831+0000 7f1ff67ea740 20 client.18801 trim_cache size 0 max 16384 2024-04-22T13:33:31.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.848+0000 7f1ff67ea740 1 Processor -- start 2024-04-22T13:33:31.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.848+0000 7f1ff67ea740 1 -- start start 2024-04-22T13:33:31.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.848+0000 7f1ff67ea740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56314245b710 0x563142464340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:31.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.848+0000 7f1ff67ea740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314245c030 0x563142470890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:31.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.848+0000 7f1ff67ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563142464c10 0x563142454fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:31.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.848+0000 7f1ff67ea740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec1afac0 con 0x56314245c030 2024-04-22T13:33:31.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.848+0000 7f1ff67ea740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec1b1600 con 0x56314245b710 2024-04-22T13:33:31.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.848+0000 7f1ff67ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec1b0190 con 0x563142464c10 2024-04-22T13:33:31.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.848+0000 7f1ff119c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314245c030 0x563142470890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:31.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.848+0000 7f1ff119c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314245c030 0x563142470890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60452/0 (socket says 172.21.15.12:60452) 2024-04-22T13:33:31.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.848+0000 7f1ff119c640 1 -- 172.21.15.12:0/1048359897 learned_addr learned my addr 172.21.15.12:0/1048359897 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:31.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.849+0000 7f1ff219e640 1 --2- 172.21.15.12:0/1048359897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563142464c10 0x563142454fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:31.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.849+0000 7f1ff199d640 1 --2- 172.21.15.12:0/1048359897 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56314245b710 0x563142464340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:31.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.849+0000 7f1ff119c640 1 -- 172.21.15.12:0/1048359897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563142464c10 msgr2=0x563142454fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:31.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.849+0000 7f1ff119c640 1 --2- 172.21.15.12:0/1048359897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563142464c10 0x563142454fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.845 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.849+0000 7f1ff119c640 1 -- 172.21.15.12:0/1048359897 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56314245b710 msgr2=0x563142464340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:31.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.849+0000 7f1ff119c640 1 --2- 172.21.15.12:0/1048359897 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56314245b710 0x563142464340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.849+0000 7f1ff119c640 1 -- 172.21.15.12:0/1048359897 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631424737e0 con 0x56314245c030 2024-04-22T13:33:31.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.849+0000 7f1ff199d640 1 --2- 172.21.15.12:0/1048359897 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56314245b710 0x563142464340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:31.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.849+0000 7f1ff119c640 1 --2- 172.21.15.12:0/1048359897 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314245c030 0x563142470890 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc098ae0 tx=0x7f1fdc09b290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e322cf767850d43 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:31.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.849+0000 7f1fd77fe640 1 -- 172.21.15.12:0/1048359897 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc0a0410 con 0x56314245c030 2024-04-22T13:33:31.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.849+0000 7f1fd77fe640 1 -- 172.21.15.12:0/1048359897 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fdc0a4cd0 con 0x56314245c030 2024-04-22T13:33:31.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.849+0000 7f1fd77fe640 1 -- 172.21.15.12:0/1048359897 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc0a1800 con 0x56314245c030 2024-04-22T13:33:31.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.850+0000 7f1ff67ea740 1 -- 172.21.15.12:0/1048359897 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314245c030 msgr2=0x563142470890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:31.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.850+0000 7f1ff67ea740 1 --2- 172.21.15.12:0/1048359897 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314245c030 0x563142470890 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc098ae0 tx=0x7f1fdc09b290 comp rx=0 tx=0).stop 2024-04-22T13:33:31.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.850+0000 7f1ff67ea740 1 -- 172.21.15.12:0/1048359897 shutdown_connections 2024-04-22T13:33:31.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.850+0000 7f1ff67ea740 1 --2- 172.21.15.12:0/1048359897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563142464c10 0x563142454fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.850+0000 7f1ff67ea740 1 --2- 172.21.15.12:0/1048359897 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314245c030 0x563142470890 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.850+0000 7f1ff67ea740 1 --2- 172.21.15.12:0/1048359897 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56314245b710 0x563142464340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.850+0000 7f1ff67ea740 1 -- 172.21.15.12:0/1048359897 >> 172.21.15.12:0/1048359897 conn(0x563142455e40 msgr2=0x563142454890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:31.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.850+0000 7f1ff67ea740 1 -- 172.21.15.12:0/1048359897 shutdown_connections 2024-04-22T13:33:31.851 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.850+0000 7f1ff67ea740 1 -- 172.21.15.12:0/1048359897 wait complete. 2024-04-22T13:33:31.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.850+0000 7f1ff67ea740 1 -- start start 2024-04-22T13:33:31.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.850+0000 7f1ff67ea740 1 Processor -- start 2024-04-22T13:33:31.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.851+0000 7f1ff67ea740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314245b710 0x563142469080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:31.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.851+0000 7f1ff67ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56314245c030 0x5631424695c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:31.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.851+0000 7f1ff67ea740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563142464c10 0x5631424d4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:31.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.851+0000 7f1ff67ea740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec1b0f30 con 0x56314245b710 2024-04-22T13:33:31.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.851+0000 7f1ff67ea740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec1b0860 con 0x563142464c10 2024-04-22T13:33:31.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.851+0000 7f1ff67ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec1adac0 con 0x56314245c030 2024-04-22T13:33:31.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.851+0000 7f1ff67ea740 10 client.0 _ll_register_callbacks cb 0x7f1fdc09bf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:31.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.851+0000 7f1ff199d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314245b710 0x563142469080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:31.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.851+0000 7f1ff199d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314245b710 0x563142469080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60460/0 (socket says 172.21.15.12:60460) 2024-04-22T13:33:31.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.851+0000 7f1ff199d640 1 -- 172.21.15.12:0/4232712268 learned_addr learned my addr 172.21.15.12:0/4232712268 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:31.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.851+0000 7f1ff119c640 1 --2- 172.21.15.12:0/4232712268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56314245c030 0x5631424695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:31.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.851+0000 7f1ff219e640 1 --2- 172.21.15.12:0/4232712268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563142464c10 0x5631424d4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:31.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.851+0000 7f1ff199d640 1 -- 172.21.15.12:0/4232712268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56314245c030 msgr2=0x5631424695c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:31.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.851+0000 7f1ff199d640 1 --2- 172.21.15.12:0/4232712268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56314245c030 0x5631424695c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.851+0000 7f1ff199d640 1 -- 172.21.15.12:0/4232712268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563142464c10 msgr2=0x5631424d4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:31.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.851+0000 7f1ff199d640 1 --2- 172.21.15.12:0/4232712268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563142464c10 0x5631424d4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.851+0000 7f1ff199d640 1 -- 172.21.15.12:0/4232712268 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56314245ca80 con 0x56314245b710 2024-04-22T13:33:31.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.851+0000 7f1ff199d640 1 --2- 172.21.15.12:0/4232712268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314245b710 0x563142469080 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x563142477880 tx=0x7f1fe80138f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425f4be5f363ab84 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:31.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.851+0000 7f1fd57fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:33:31.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.851+0000 7f1fd57fa640 1 -- 172.21.15.12:0/4232712268 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8004220 con 0x56314245b710 2024-04-22T13:33:31.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.851+0000 7f1fd57fa640 1 -- 172.21.15.12:0/4232712268 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f1fe80043c0 con 0x56314245b710 2024-04-22T13:33:31.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.851+0000 7f1ff67ea740 1 -- 172.21.15.12:0/4232712268 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fdc0a4cd0 con 0x56314245b710 2024-04-22T13:33:31.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.852+0000 7f1fd57fa640 1 -- 172.21.15.12:0/4232712268 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1fe802f050 con 0x56314245b710 2024-04-22T13:33:31.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.852+0000 7f1ff67ea740 10 client.18807 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:31.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.852+0000 7f1ff67ea740 1 -- 172.21.15.12:0/4232712268 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5631424dba10 con 0x56314245b710 2024-04-22T13:33:31.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.852+0000 7f1fd57fa640 1 -- 172.21.15.12:0/4232712268 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fdc0a4cd0 con 0x56314245b710 2024-04-22T13:33:31.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.852+0000 7f1fd57fa640 1 -- 172.21.15.12:0/4232712268 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1fe8003780 con 0x56314245b710 2024-04-22T13:33:31.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.852+0000 7f1ff67ea740 10 client.18807 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:31.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.852+0000 7f1ff67ea740 10 client.18807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:33:31.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.852+0000 7f1ff67ea740 20 client.18807 populate_metadata read hostname 'smithi012' 2024-04-22T13:33:31.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.852+0000 7f1ff67ea740 1 --2- 172.21.15.12:0/4232712268 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5631424dc190 0x5631424de5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:31.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.852+0000 7f1ff67ea740 4 client.18807 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:33:31.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.852+0000 7f1ff67ea740 1 -- 172.21.15.12:0/4232712268 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5631423c7a70 con 0x5631424dc190 2024-04-22T13:33:31.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.852+0000 7f1ff119c640 1 --2- 172.21.15.12:0/4232712268 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5631424dc190 0x5631424de5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:31.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.853+0000 7f1fd57fa640 10 client.18807 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:33:31.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.853+0000 7f1ff119c640 1 --2- 172.21.15.12:0/4232712268 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5631424dc190 0x5631424de5b0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3df89f1ba34a14f6 server_cookie=6d12e845b4546cb7 in_seq=0 out_seq=0 2024-04-22T13:33:31.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.853+0000 7f1fd57fa640 1 -- 172.21.15.12:0/4232712268 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5631423c7a70 con 0x5631424dc190 2024-04-22T13:33:31.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.853+0000 7f1fd57fa640 10 client.18807 handle_command_reply: tid=0 2024-04-22T13:33:31.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.853+0000 7f1ff67ea740 1 client.18807 shutdown 2024-04-22T13:33:31.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.853+0000 7f1ff67ea740 1 -- 172.21.15.12:0/4232712268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314245b710 msgr2=0x563142469080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:31.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.853+0000 7f1ff67ea740 1 --2- 172.21.15.12:0/4232712268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314245b710 0x563142469080 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x563142477880 tx=0x7f1fe80138f0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.853+0000 7f1ff67ea740 1 -- 172.21.15.12:0/4232712268 shutdown_connections 2024-04-22T13:33:31.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.853+0000 7f1ff67ea740 1 --2- 172.21.15.12:0/4232712268 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5631424dc190 0x5631424de5b0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.853+0000 7f1ff67ea740 1 --2- 172.21.15.12:0/4232712268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563142464c10 0x5631424d4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.853+0000 7f1ff67ea740 1 --2- 172.21.15.12:0/4232712268 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56314245c030 0x5631424695c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.853+0000 7f1fd57fa640 0 client.18807 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:33:31.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.853+0000 7f1ff67ea740 1 --2- 172.21.15.12:0/4232712268 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56314245b710 0x563142469080 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.853+0000 7f1ff67ea740 1 -- 172.21.15.12:0/4232712268 >> 172.21.15.12:0/4232712268 conn(0x563142455e40 msgr2=0x563142455120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:31.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.853+0000 7f1ff67ea740 1 -- 172.21.15.12:0/4232712268 shutdown_connections 2024-04-22T13:33:31.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.853+0000 7f1ff67ea740 1 -- 172.21.15.12:0/4232712268 wait complete. 2024-04-22T13:33:31.855 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.853+0000 7f1ff67ea740 20 client.18807 trim_cache size 0 max 16384 2024-04-22T13:33:31.855 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:33:31.856 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-04-22T13:33:31.856 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-04-22T13:33:31.856 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:33:31.856 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.854+0000 7f1ff299f640 1 -- 172.21.15.12:0/2206558618 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f1fd80a3dd0 msgr2=0x7f1fd80a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:31.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.854+0000 7f1ff299f640 1 --2- 172.21.15.12:0/2206558618 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f1fd80a3dd0 0x7f1fd80a6290 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4009ca0 tx=0x7f1fe4042000 comp rx=0 tx=0).stop 2024-04-22T13:33:31.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.854+0000 7f1ff299f640 1 -- 172.21.15.12:0/2206558618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fec103800 msgr2=0x7f1fec101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:31.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.854+0000 7f1ff299f640 1 --2- 172.21.15.12:0/2206558618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fec103800 0x7f1fec101070 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc00d950 tx=0x7f1fdc00de10 comp rx=0 tx=0).stop 2024-04-22T13:33:31.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.854+0000 7f1ff299f640 1 -- 172.21.15.12:0/2206558618 shutdown_connections 2024-04-22T13:33:31.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.854+0000 7f1ff299f640 1 --2- 172.21.15.12:0/2206558618 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f1fd80a3dd0 0x7f1fd80a6290 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.854+0000 7f1ff299f640 1 --2- 172.21.15.12:0/2206558618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1fec10c350 0x7f1fec1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.854+0000 7f1ff299f640 1 --2- 172.21.15.12:0/2206558618 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f1fec103800 0x7f1fec101070 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.854+0000 7f1ff299f640 1 --2- 172.21.15.12:0/2206558618 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1fec102ee0 0x7f1fec100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.854+0000 7f1ff299f640 1 -- 172.21.15.12:0/2206558618 >> 172.21.15.12:0/2206558618 conn(0x7f1fec0fc6e0 msgr2=0x7f1fec0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:31.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.854+0000 7f1ff299f640 1 -- 172.21.15.12:0/2206558618 shutdown_connections 2024-04-22T13:33:31.857 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.854+0000 7f1ff299f640 1 -- 172.21.15.12:0/2206558618 wait complete. 2024-04-22T13:33:31.860 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:33:31.860 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:33:31.881 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:31 smithi012 ceph-mon[28265]: pgmap v291: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:31.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:31 smithi134 ceph-mon[31338]: pgmap v291: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:31.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.967+0000 7f093856d640 1 -- 172.21.15.12:0/2852233126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09301031d0 msgr2=0x7f09301035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.967+0000 7f093856d640 1 --2- 172.21.15.12:0/2852233126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09301031d0 0x7f09301035b0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f0920009ca0 tx=0x7f092002c180 comp rx=0 tx=0).stop 2024-04-22T13:33:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.967+0000 7f093856d640 1 -- 172.21.15.12:0/2852233126 shutdown_connections 2024-04-22T13:33:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.967+0000 7f093856d640 1 --2- 172.21.15.12:0/2852233126 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f093010c5a0 0x7f093010e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.967+0000 7f093856d640 1 --2- 172.21.15.12:0/2852233126 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0930103af0 0x7f093010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.967+0000 7f093856d640 1 --2- 172.21.15.12:0/2852233126 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09301031d0 0x7f09301035b0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.967+0000 7f093856d640 1 -- 172.21.15.12:0/2852233126 >> 172.21.15.12:0/2852233126 conn(0x7f09300771d0 msgr2=0x7f09300775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.967+0000 7f093856d640 1 -- 172.21.15.12:0/2852233126 shutdown_connections 2024-04-22T13:33:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.967+0000 7f093856d640 1 -- 172.21.15.12:0/2852233126 wait complete. 2024-04-22T13:33:31.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.968+0000 7f093856d640 1 Processor -- start 2024-04-22T13:33:31.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.968+0000 7f093856d640 1 -- start start 2024-04-22T13:33:31.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.968+0000 7f093856d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09301031d0 0x7f093019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:31.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.968+0000 7f093856d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0930103af0 0x7f093019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:31.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.968+0000 7f093856d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f093010c5a0 0x7f09301985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:31.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.968+0000 7f093856d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f09301121f0 con 0x7f09301031d0 2024-04-22T13:33:31.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.968+0000 7f093856d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0930112070 con 0x7f0930103af0 2024-04-22T13:33:31.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.968+0000 7f093856d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0930112370 con 0x7f093010c5a0 2024-04-22T13:33:31.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.968+0000 7f0936309640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09301031d0 0x7f093019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:31.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.968+0000 7f0936309640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09301031d0 0x7f093019e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60478/0 (socket says 172.21.15.12:60478) 2024-04-22T13:33:31.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.968+0000 7f0936309640 1 -- 172.21.15.12:0/1268139744 learned_addr learned my addr 172.21.15.12:0/1268139744 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:31.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.968+0000 7f0935b08640 1 --2- 172.21.15.12:0/1268139744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0930103af0 0x7f093019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:31.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.968+0000 7f0936b0a640 1 --2- 172.21.15.12:0/1268139744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f093010c5a0 0x7f09301985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:31.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.968+0000 7f0936309640 1 -- 172.21.15.12:0/1268139744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f093010c5a0 msgr2=0x7f09301985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:31.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.968+0000 7f0936309640 1 --2- 172.21.15.12:0/1268139744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f093010c5a0 0x7f09301985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.968+0000 7f0936309640 1 -- 172.21.15.12:0/1268139744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0930103af0 msgr2=0x7f093019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:31.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.968+0000 7f0936309640 1 --2- 172.21.15.12:0/1268139744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0930103af0 0x7f093019e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:31.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.968+0000 7f0936309640 1 -- 172.21.15.12:0/1268139744 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0930198d30 con 0x7f09301031d0 2024-04-22T13:33:31.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.969+0000 7f0936309640 1 --2- 172.21.15.12:0/1268139744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09301031d0 0x7f093019e400 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f0920005e30 tx=0x7f0920003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:31.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.969+0000 7f092f7fe640 1 -- 172.21.15.12:0/1268139744 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0920039040 con 0x7f09301031d0 2024-04-22T13:33:31.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.969+0000 7f093856d640 1 -- 172.21.15.12:0/1268139744 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0930198fc0 con 0x7f09301031d0 2024-04-22T13:33:31.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.969+0000 7f092f7fe640 1 -- 172.21.15.12:0/1268139744 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0920044070 con 0x7f09301031d0 2024-04-22T13:33:31.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.969+0000 7f092f7fe640 1 -- 172.21.15.12:0/1268139744 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0920032920 con 0x7f09301031d0 2024-04-22T13:33:31.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.969+0000 7f093856d640 1 -- 172.21.15.12:0/1268139744 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09301a52c0 con 0x7f09301031d0 2024-04-22T13:33:31.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.970+0000 7f093856d640 1 -- 172.21.15.12:0/1268139744 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0930069230 con 0x7f09301031d0 2024-04-22T13:33:31.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.971+0000 7f092f7fe640 1 -- 172.21.15.12:0/1268139744 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0920032ac0 con 0x7f09301031d0 2024-04-22T13:33:31.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.972+0000 7f092f7fe640 1 --2- 172.21.15.12:0/1268139744 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f09180a3d20 0x7f09180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:31.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.972+0000 7f0935b08640 1 --2- 172.21.15.12:0/1268139744 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f09180a3d20 0x7f09180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:31.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.972+0000 7f092f7fe640 1 -- 172.21.15.12:0/1268139744 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f09200f3670 con 0x7f09301031d0 2024-04-22T13:33:31.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.972+0000 7f0935b08640 1 --2- 172.21.15.12:0/1268139744 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f09180a3d20 0x7f09180a61e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f093010e310 tx=0x7f0924005f90 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:31.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:31.976+0000 7f092f7fe640 1 -- 172.21.15.12:0/1268139744 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f09200bfe40 con 0x7f09301031d0 2024-04-22T13:33:31.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:31 smithi111 ceph-mon[31361]: pgmap v291: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:32.197 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.202+0000 7f093856d640 1 -- 172.21.15.12:0/1268139744 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0930107690 con 0x7f09301031d0 2024-04-22T13:33:32.198 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.203+0000 7f092f7fe640 1 -- 172.21.15.12:0/1268139744 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7f09200bf740 con 0x7f09301031d0 2024-04-22T13:33:32.198 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:33:32.198 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:33:32.199 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:33:32.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.205+0000 7f093856d640 1 -- 172.21.15.12:0/1268139744 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f09180a3d20 msgr2=0x7f09180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.205+0000 7f093856d640 1 --2- 172.21.15.12:0/1268139744 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f09180a3d20 0x7f09180a61e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f093010e310 tx=0x7f0924005f90 comp rx=0 tx=0).stop 2024-04-22T13:33:32.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.205+0000 7f093856d640 1 -- 172.21.15.12:0/1268139744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09301031d0 msgr2=0x7f093019e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.205+0000 7f093856d640 1 --2- 172.21.15.12:0/1268139744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09301031d0 0x7f093019e400 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f0920005e30 tx=0x7f0920003c20 comp rx=0 tx=0).stop 2024-04-22T13:33:32.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.205+0000 7f093856d640 1 -- 172.21.15.12:0/1268139744 shutdown_connections 2024-04-22T13:33:32.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.205+0000 7f093856d640 1 --2- 172.21.15.12:0/1268139744 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f09180a3d20 0x7f09180a61e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.205+0000 7f093856d640 1 --2- 172.21.15.12:0/1268139744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f093010c5a0 0x7f09301985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.205+0000 7f093856d640 1 --2- 172.21.15.12:0/1268139744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0930103af0 0x7f093019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.205+0000 7f093856d640 1 --2- 172.21.15.12:0/1268139744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f09301031d0 0x7f093019e400 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.205+0000 7f093856d640 1 -- 172.21.15.12:0/1268139744 >> 172.21.15.12:0/1268139744 conn(0x7f09300771d0 msgr2=0x7f09300ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:32.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.205+0000 7f093856d640 1 -- 172.21.15.12:0/1268139744 shutdown_connections 2024-04-22T13:33:32.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.205+0000 7f093856d640 1 -- 172.21.15.12:0/1268139744 wait complete. 2024-04-22T13:33:32.211 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:33:32.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.360+0000 7fe07cbef640 1 -- 172.21.15.12:0/1016567731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe07806b1f0 msgr2=0x7fe078106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.360+0000 7fe07cbef640 1 --2- 172.21.15.12:0/1016567731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe07806b1f0 0x7fe078106b90 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fe060009900 tx=0x7fe06002bc30 comp rx=0 tx=0).stop 2024-04-22T13:33:32.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.360+0000 7fe07cbef640 1 -- 172.21.15.12:0/1016567731 shutdown_connections 2024-04-22T13:33:32.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.360+0000 7fe07cbef640 1 --2- 172.21.15.12:0/1016567731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe078107270 0x7fe07810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.360+0000 7fe07cbef640 1 --2- 172.21.15.12:0/1016567731 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe07806b1f0 0x7fe078106b90 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.360+0000 7fe07cbef640 1 --2- 172.21.15.12:0/1016567731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe07806a840 0x7fe07806ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.360+0000 7fe07cbef640 1 -- 172.21.15.12:0/1016567731 >> 172.21.15.12:0/1016567731 conn(0x7fe078077eb0 msgr2=0x7fe0780782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:32.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.360+0000 7fe07cbef640 1 -- 172.21.15.12:0/1016567731 shutdown_connections 2024-04-22T13:33:32.356 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.360+0000 7fe07cbef640 1 -- 172.21.15.12:0/1016567731 wait complete. 2024-04-22T13:33:32.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.361+0000 7fe07cbef640 1 Processor -- start 2024-04-22T13:33:32.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.361+0000 7fe07cbef640 1 -- start start 2024-04-22T13:33:32.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.361+0000 7fe07cbef640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe07806a840 0x7fe078198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.361+0000 7fe07cbef640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe078107270 0x7fe078198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.358 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.361+0000 7fe07cbef640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe07819cff0 0x7fe07819d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.361+0000 7fe07cbef640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe0781121b0 con 0x7fe07806a840 2024-04-22T13:33:32.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.361+0000 7fe07cbef640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe078112030 con 0x7fe07819cff0 2024-04-22T13:33:32.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.361+0000 7fe07cbef640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe078112330 con 0x7fe078107270 2024-04-22T13:33:32.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.361+0000 7fe07659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe07806a840 0x7fe078198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.362+0000 7fe07659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe07806a840 0x7fe078198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60506/0 (socket says 172.21.15.12:60506) 2024-04-22T13:33:32.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.362+0000 7fe07659c640 1 -- 172.21.15.12:0/3015320928 learned_addr learned my addr 172.21.15.12:0/3015320928 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:32.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.362+0000 7fe06dd9b640 1 --2- 172.21.15.12:0/3015320928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe078107270 0x7fe078198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.362+0000 7fe076d9d640 1 --2- 172.21.15.12:0/3015320928 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe07819cff0 0x7fe07819d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.362+0000 7fe07659c640 1 -- 172.21.15.12:0/3015320928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe078107270 msgr2=0x7fe078198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.362+0000 7fe07659c640 1 --2- 172.21.15.12:0/3015320928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe078107270 0x7fe078198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.362+0000 7fe07659c640 1 -- 172.21.15.12:0/3015320928 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe07819cff0 msgr2=0x7fe07819d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.362+0000 7fe07659c640 1 --2- 172.21.15.12:0/3015320928 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe07819cff0 0x7fe07819d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.362+0000 7fe07659c640 1 -- 172.21.15.12:0/3015320928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe07819dae0 con 0x7fe07806a840 2024-04-22T13:33:32.359 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.362+0000 7fe07659c640 1 --2- 172.21.15.12:0/3015320928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe07806a840 0x7fe078198730 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fe06800d770 tx=0x7fe06800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:32.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.362+0000 7fe06ffff640 1 -- 172.21.15.12:0/3015320928 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe068017070 con 0x7fe07806a840 2024-04-22T13:33:32.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.362+0000 7fe07cbef640 1 -- 172.21.15.12:0/3015320928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe07819ddd0 con 0x7fe07806a840 2024-04-22T13:33:32.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.362+0000 7fe06ffff640 1 -- 172.21.15.12:0/3015320928 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0680054d0 con 0x7fe07806a840 2024-04-22T13:33:32.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.362+0000 7fe06ffff640 1 -- 172.21.15.12:0/3015320928 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe068004c30 con 0x7fe07806a840 2024-04-22T13:33:32.360 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.362+0000 7fe07cbef640 1 -- 172.21.15.12:0/3015320928 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0781a5540 con 0x7fe07806a840 2024-04-22T13:33:32.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.365+0000 7fe06ffff640 1 -- 172.21.15.12:0/3015320928 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe068004e10 con 0x7fe07806a840 2024-04-22T13:33:32.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.365+0000 7fe07f7d6740 1 Processor -- start 2024-04-22T13:33:32.366 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.365+0000 7fe07f7d6740 1 -- start start 2024-04-22T13:33:32.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.365+0000 7fe07f7d6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5638058003b0 0x563805800790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.365+0000 7fe07f7d6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563805800d60 0x563805809890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.365+0000 7fe07f7d6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563805809e60 0x56380580a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.365+0000 7fe07f7d6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5638057651f0 con 0x563805800d60 2024-04-22T13:33:32.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.365+0000 7fe07f7d6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5638057024b0 con 0x5638058003b0 2024-04-22T13:33:32.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.365+0000 7fe07f7d6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5638057611b0 con 0x563805809e60 2024-04-22T13:33:32.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.365+0000 7fe07659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563805800d60 0x563805809890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.365+0000 7fe07659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563805800d60 0x563805809890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60520/0 (socket says 172.21.15.12:60520) 2024-04-22T13:33:32.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.365+0000 7fe07659c640 1 -- 172.21.15.12:0/3796981250 learned_addr learned my addr 172.21.15.12:0/3796981250 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:32.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.365+0000 7fe06ffff640 1 --2- 172.21.15.12:0/3015320928 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe0640a3d20 0x7fe0640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.365+0000 7fe06dd9b640 1 --2- 172.21.15.12:0/3796981250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563805809e60 0x56380580a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.366+0000 7fe076d9d640 1 --2- 172.21.15.12:0/3796981250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5638058003b0 0x563805800790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.367 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.366+0000 7fe076d9d640 1 --2- 172.21.15.12:0/3015320928 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe0640a3d20 0x7fe0640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.366+0000 7fe06ffff640 1 -- 172.21.15.12:0/3015320928 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7fe0680d0360 con 0x7fe07806a840 2024-04-22T13:33:32.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.366+0000 7fe07659c640 1 -- 172.21.15.12:0/3796981250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563805809e60 msgr2=0x56380580a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.366+0000 7fe07659c640 1 --2- 172.21.15.12:0/3796981250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563805809e60 0x56380580a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.366+0000 7fe07659c640 1 -- 172.21.15.12:0/3796981250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5638058003b0 msgr2=0x563805800790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.366+0000 7fe07659c640 1 --2- 172.21.15.12:0/3796981250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5638058003b0 0x563805800790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.366+0000 7fe07659c640 1 -- 172.21.15.12:0/3796981250 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638056f41b0 con 0x563805800d60 2024-04-22T13:33:32.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.366+0000 7fe076d9d640 1 --2- 172.21.15.12:0/3796981250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5638058003b0 0x563805800790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:32.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.366+0000 7fe07659c640 1 --2- 172.21.15.12:0/3796981250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563805800d60 0x563805809890 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fe06809c020 tx=0x7fe068009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5388913b07f928 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:32.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.366+0000 7fe076d9d640 1 --2- 172.21.15.12:0/3015320928 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe0640a3d20 0x7fe0640a61e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe07000afa0 tx=0x7fe070005c80 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:32.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.366+0000 7fe06cd99640 1 -- 172.21.15.12:0/3796981250 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe068020050 con 0x563805800d60 2024-04-22T13:33:32.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.366+0000 7fe06cd99640 1 -- 172.21.15.12:0/3796981250 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe06809ca20 con 0x563805800d60 2024-04-22T13:33:32.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.366+0000 7fe06cd99640 1 -- 172.21.15.12:0/3796981250 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06809a700 con 0x563805800d60 2024-04-22T13:33:32.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.367+0000 7fe07f7d6740 1 -- 172.21.15.12:0/3796981250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563805800d60 msgr2=0x563805809890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.367+0000 7fe07f7d6740 1 --2- 172.21.15.12:0/3796981250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563805800d60 0x563805809890 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fe06809c020 tx=0x7fe068009f90 comp rx=0 tx=0).stop 2024-04-22T13:33:32.368 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.367+0000 7fe07f7d6740 1 -- 172.21.15.12:0/3796981250 shutdown_connections 2024-04-22T13:33:32.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.367+0000 7fe07f7d6740 1 --2- 172.21.15.12:0/3796981250 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563805809e60 0x56380580a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.367+0000 7fe07f7d6740 1 --2- 172.21.15.12:0/3796981250 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563805800d60 0x563805809890 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.367+0000 7fe07f7d6740 1 --2- 172.21.15.12:0/3796981250 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5638058003b0 0x563805800790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.367+0000 7fe07f7d6740 1 -- 172.21.15.12:0/3796981250 >> 172.21.15.12:0/3796981250 conn(0x5638057faba0 msgr2=0x5638057faf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:32.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.367+0000 7fe07f7d6740 1 -- 172.21.15.12:0/3796981250 shutdown_connections 2024-04-22T13:33:32.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.367+0000 7fe07f7d6740 1 -- 172.21.15.12:0/3796981250 wait complete. 2024-04-22T13:33:32.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.367+0000 7fe07f7d6740 1 -- start start 2024-04-22T13:33:32.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.367+0000 7fe07f7d6740 1 Processor -- start 2024-04-22T13:33:32.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.368+0000 7fe07f7d6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638058003b0 0x5638058102a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.368+0000 7fe07f7d6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563805800d60 0x56380580d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.368+0000 7fe07f7d6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563805809e60 0x56380580db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.368+0000 7fe07f7d6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x563805810d40 con 0x5638058003b0 2024-04-22T13:33:32.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.368+0000 7fe07f7d6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56380571e500 con 0x563805809e60 2024-04-22T13:33:32.369 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.368+0000 7fe07f7d6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563805810ec0 con 0x563805800d60 2024-04-22T13:33:32.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.368+0000 7fe07f7d6740 10 client.0 _ll_register_callbacks cb 0x5638056ecb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:32.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.368+0000 7fe07659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638058003b0 0x5638058102a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.368+0000 7fe07659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638058003b0 0x5638058102a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60522/0 (socket says 172.21.15.12:60522) 2024-04-22T13:33:32.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.368+0000 7fe07659c640 1 -- 172.21.15.12:0/3141964994 learned_addr learned my addr 172.21.15.12:0/3141964994 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:32.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.368+0000 7fe06dd9b640 1 --2- 172.21.15.12:0/3141964994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563805800d60 0x56380580d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.368+0000 7fe076d9d640 1 --2- 172.21.15.12:0/3141964994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563805809e60 0x56380580db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.368+0000 7fe07659c640 1 -- 172.21.15.12:0/3141964994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563805800d60 msgr2=0x56380580d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.368+0000 7fe07659c640 1 --2- 172.21.15.12:0/3141964994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563805800d60 0x56380580d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.368+0000 7fe07659c640 1 -- 172.21.15.12:0/3141964994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563805809e60 msgr2=0x56380580db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.368+0000 7fe07659c640 1 --2- 172.21.15.12:0/3141964994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563805809e60 0x56380580db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.368+0000 7fe07659c640 1 -- 172.21.15.12:0/3141964994 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638054773f0 con 0x5638058003b0 2024-04-22T13:33:32.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.368+0000 7fe07659c640 1 --2- 172.21.15.12:0/3141964994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638058003b0 0x5638058102a0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fe0680cdb80 tx=0x7fe06809c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133a6c1bf435e978 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:32.370 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.368+0000 7fe05dffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:33:32.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.368+0000 7fe05dffb640 1 -- 172.21.15.12:0/3141964994 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0680a4f90 con 0x5638058003b0 2024-04-22T13:33:32.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.368+0000 7fe05dffb640 1 -- 172.21.15.12:0/3141964994 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7fe0680a4b00 con 0x5638058003b0 2024-04-22T13:33:32.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.368+0000 7fe07f7d6740 1 -- 172.21.15.12:0/3141964994 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe06809ca20 con 0x5638058003b0 2024-04-22T13:33:32.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.368+0000 7fe05dffb640 1 -- 172.21.15.12:0/3141964994 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe0680ba050 con 0x5638058003b0 2024-04-22T13:33:32.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.369+0000 7fe07f7d6740 10 client.18825 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:32.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.369+0000 7fe07f7d6740 1 -- 172.21.15.12:0/3141964994 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5638055417d0 con 0x5638058003b0 2024-04-22T13:33:32.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.369+0000 7fe05dffb640 1 -- 172.21.15.12:0/3141964994 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe06809ca20 con 0x5638058003b0 2024-04-22T13:33:32.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.369+0000 7fe05dffb640 1 -- 172.21.15.12:0/3141964994 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe0680a0170 con 0x5638058003b0 2024-04-22T13:33:32.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.369+0000 7fe07f7d6740 10 client.18825 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:32.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.369+0000 7fe07f7d6740 10 client.18825 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:33:32.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.369+0000 7fe07f7d6740 20 client.18825 populate_metadata read hostname 'smithi012' 2024-04-22T13:33:32.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.369+0000 7fe07f7d6740 1 --2- 172.21.15.12:0/3141964994 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x56380581de70 0x563805820260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.369+0000 7fe07f7d6740 4 client.18825 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:33:32.371 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.369+0000 7fe07f7d6740 1 -- 172.21.15.12:0/3141964994 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563805704930 con 0x56380581de70 2024-04-22T13:33:32.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.369+0000 7fe06dd9b640 1 --2- 172.21.15.12:0/3141964994 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x56380581de70 0x563805820260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.370+0000 7fe06dd9b640 1 --2- 172.21.15.12:0/3141964994 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x56380581de70 0x563805820260 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=89851018d7b5892f server_cookie=5a0c96f478e42324 in_seq=0 out_seq=0 2024-04-22T13:33:32.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.370+0000 7fe05dffb640 10 client.18825 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:33:32.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.370+0000 7fe05dffb640 1 -- 172.21.15.12:0/3141964994 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563805704930 con 0x56380581de70 2024-04-22T13:33:32.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.370+0000 7fe05dffb640 10 client.18825 handle_command_reply: tid=0 2024-04-22T13:33:32.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.370+0000 7fe07f7d6740 1 client.18825 shutdown 2024-04-22T13:33:32.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.371+0000 7fe07f7d6740 1 -- 172.21.15.12:0/3141964994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638058003b0 msgr2=0x5638058102a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.371+0000 7fe07f7d6740 1 --2- 172.21.15.12:0/3141964994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638058003b0 0x5638058102a0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fe0680cdb80 tx=0x7fe06809c410 comp rx=0 tx=0).stop 2024-04-22T13:33:32.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.371+0000 7fe07f7d6740 1 -- 172.21.15.12:0/3141964994 shutdown_connections 2024-04-22T13:33:32.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.371+0000 7fe07f7d6740 1 --2- 172.21.15.12:0/3141964994 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x56380581de70 0x563805820260 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.371+0000 7fe07f7d6740 1 --2- 172.21.15.12:0/3141964994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563805809e60 0x56380580db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.371+0000 7fe07f7d6740 1 --2- 172.21.15.12:0/3141964994 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563805800d60 0x56380580d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.371+0000 7fe05dffb640 0 client.18825 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:33:32.372 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.371+0000 7fe07f7d6740 1 --2- 172.21.15.12:0/3141964994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638058003b0 0x5638058102a0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.371+0000 7fe07f7d6740 1 -- 172.21.15.12:0/3141964994 >> 172.21.15.12:0/3141964994 conn(0x5638057faba0 msgr2=0x5638057fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:32.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.371+0000 7fe07f7d6740 1 -- 172.21.15.12:0/3141964994 shutdown_connections 2024-04-22T13:33:32.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.371+0000 7fe07f7d6740 1 -- 172.21.15.12:0/3141964994 wait complete. 2024-04-22T13:33:32.373 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.371+0000 7fe07f7d6740 20 client.18825 trim_cache size 0 max 16384 2024-04-22T13:33:32.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.384+0000 7fe07f7d6740 1 Processor -- start 2024-04-22T13:33:32.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.384+0000 7fe07f7d6740 1 -- start start 2024-04-22T13:33:32.384 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.384+0000 7fe07f7d6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638058003b0 0x56380580b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.384+0000 7fe07f7d6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563805800d60 0x56380580fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.384+0000 7fe07f7d6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563805809e60 0x5638058102a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.384+0000 7fe07f7d6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe0781ab8e0 con 0x563805800d60 2024-04-22T13:33:32.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.384+0000 7fe07f7d6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe0781ad420 con 0x563805809e60 2024-04-22T13:33:32.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.384+0000 7fe07f7d6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe0781abfb0 con 0x5638058003b0 2024-04-22T13:33:32.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.384+0000 7fe06dd9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563805800d60 0x56380580fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.384+0000 7fe06dd9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563805800d60 0x56380580fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60524/0 (socket says 172.21.15.12:60524) 2024-04-22T13:33:32.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.384+0000 7fe06dd9b640 1 -- 172.21.15.12:0/3424086137 learned_addr learned my addr 172.21.15.12:0/3424086137 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:32.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.384+0000 7fe06dd9b640 1 -- 172.21.15.12:0/3424086137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638058003b0 msgr2=0x56380580b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.385+0000 7fe07659c640 1 --2- 172.21.15.12:0/3424086137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638058003b0 0x56380580b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.385+0000 7fe06dd9b640 1 --2- 172.21.15.12:0/3424086137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638058003b0 0x56380580b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.385+0000 7fe076d9d640 1 --2- 172.21.15.12:0/3424086137 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563805809e60 0x5638058102a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.385+0000 7fe06dd9b640 1 -- 172.21.15.12:0/3424086137 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563805809e60 msgr2=0x5638058102a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.385 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.385+0000 7fe06dd9b640 1 --2- 172.21.15.12:0/3424086137 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563805809e60 0x5638058102a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.385+0000 7fe06dd9b640 1 -- 172.21.15.12:0/3424086137 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638057ff560 con 0x563805800d60 2024-04-22T13:33:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.385+0000 7fe076d9d640 1 --2- 172.21.15.12:0/3424086137 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563805809e60 0x5638058102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.385+0000 7fe06dd9b640 1 --2- 172.21.15.12:0/3424086137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563805800d60 0x56380580fd60 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fe060005ab0 tx=0x7fe060005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70e6aff2af0f57 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.385+0000 7fe05ffff640 1 -- 172.21.15.12:0/3424086137 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060047050 con 0x563805800d60 2024-04-22T13:33:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.385+0000 7fe05ffff640 1 -- 172.21.15.12:0/3424086137 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe06002cc90 con 0x563805800d60 2024-04-22T13:33:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.385+0000 7fe05ffff640 1 -- 172.21.15.12:0/3424086137 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060005330 con 0x563805800d60 2024-04-22T13:33:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.385+0000 7fe07f7d6740 1 -- 172.21.15.12:0/3424086137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563805800d60 msgr2=0x56380580fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.385+0000 7fe07f7d6740 1 --2- 172.21.15.12:0/3424086137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563805800d60 0x56380580fd60 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fe060005ab0 tx=0x7fe060005a20 comp rx=0 tx=0).stop 2024-04-22T13:33:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.385+0000 7fe07f7d6740 1 -- 172.21.15.12:0/3424086137 shutdown_connections 2024-04-22T13:33:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.385+0000 7fe07f7d6740 1 --2- 172.21.15.12:0/3424086137 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563805809e60 0x5638058102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.385+0000 7fe07f7d6740 1 --2- 172.21.15.12:0/3424086137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563805800d60 0x56380580fd60 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.385+0000 7fe07f7d6740 1 --2- 172.21.15.12:0/3424086137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5638058003b0 0x56380580b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.385+0000 7fe07f7d6740 1 -- 172.21.15.12:0/3424086137 >> 172.21.15.12:0/3424086137 conn(0x5638057faba0 msgr2=0x56380580c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:32.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.386+0000 7fe07f7d6740 1 -- 172.21.15.12:0/3424086137 shutdown_connections 2024-04-22T13:33:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.386+0000 7fe07f7d6740 1 -- 172.21.15.12:0/3424086137 wait complete. 2024-04-22T13:33:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.386+0000 7fe07f7d6740 1 -- start start 2024-04-22T13:33:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.386+0000 7fe07f7d6740 1 Processor -- start 2024-04-22T13:33:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.386+0000 7fe07f7d6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638058003b0 0x563805820ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.386+0000 7fe07f7d6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563805800d60 0x563805821430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.386+0000 7fe07f7d6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563805809e60 0x563805821970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.386+0000 7fe07f7d6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe0781acd50 con 0x5638058003b0 2024-04-22T13:33:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.386+0000 7fe07f7d6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe0781ac680 con 0x563805809e60 2024-04-22T13:33:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.386+0000 7fe07f7d6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe0781a98e0 con 0x563805800d60 2024-04-22T13:33:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.386+0000 7fe07f7d6740 10 client.0 _ll_register_callbacks cb 0x7fe06809a790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.386+0000 7fe07659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638058003b0 0x563805820ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.386+0000 7fe07659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638058003b0 0x563805820ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60532/0 (socket says 172.21.15.12:60532) 2024-04-22T13:33:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.386+0000 7fe07659c640 1 -- 172.21.15.12:0/3611523170 learned_addr learned my addr 172.21.15.12:0/3611523170 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:32.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.386+0000 7fe076d9d640 1 --2- 172.21.15.12:0/3611523170 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563805809e60 0x563805821970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.386+0000 7fe06dd9b640 1 --2- 172.21.15.12:0/3611523170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563805800d60 0x563805821430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.386+0000 7fe07659c640 1 -- 172.21.15.12:0/3611523170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563805800d60 msgr2=0x563805821430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.386+0000 7fe07659c640 1 --2- 172.21.15.12:0/3611523170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563805800d60 0x563805821430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.386+0000 7fe07659c640 1 -- 172.21.15.12:0/3611523170 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563805809e60 msgr2=0x563805821970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.386+0000 7fe07659c640 1 --2- 172.21.15.12:0/3611523170 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563805809e60 0x563805821970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.386+0000 7fe07659c640 1 -- 172.21.15.12:0/3611523170 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638058832a0 con 0x5638058003b0 2024-04-22T13:33:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.386+0000 7fe06dd9b640 1 --2- 172.21.15.12:0/3611523170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563805800d60 0x563805821430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.387+0000 7fe07659c640 1 --2- 172.21.15.12:0/3611523170 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638058003b0 0x563805820ef0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fe06809a5d0 tx=0x7fe068098110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70958a81911c09a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.387+0000 7fe05dffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:33:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.387+0000 7fe05dffb640 1 -- 172.21.15.12:0/3611523170 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe068020070 con 0x5638058003b0 2024-04-22T13:33:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.387+0000 7fe05dffb640 1 -- 172.21.15.12:0/3611523170 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7fe068098990 con 0x5638058003b0 2024-04-22T13:33:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.387+0000 7fe07f7d6740 1 -- 172.21.15.12:0/3611523170 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe06002cc90 con 0x5638058003b0 2024-04-22T13:33:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.387+0000 7fe05dffb640 1 -- 172.21.15.12:0/3611523170 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe06809be60 con 0x5638058003b0 2024-04-22T13:33:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.387+0000 7fe07f7d6740 10 client.18831 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:32.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.387+0000 7fe07f7d6740 1 -- 172.21.15.12:0/3611523170 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5638058844d0 con 0x5638058003b0 2024-04-22T13:33:32.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.387+0000 7fe05dffb640 1 -- 172.21.15.12:0/3611523170 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe06002cc90 con 0x5638058003b0 2024-04-22T13:33:32.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.387+0000 7fe05dffb640 1 -- 172.21.15.12:0/3611523170 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe06809b1e0 con 0x5638058003b0 2024-04-22T13:33:32.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.387+0000 7fe07f7d6740 10 client.18831 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:32.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.387+0000 7fe07f7d6740 10 client.18831 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:33:32.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.387+0000 7fe07f7d6740 20 client.18831 populate_metadata read hostname 'smithi012' 2024-04-22T13:33:32.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.387+0000 7fe07f7d6740 1 --2- 172.21.15.12:0/3611523170 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x563805884a90 0x563805886eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.387+0000 7fe07f7d6740 4 client.18831 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:33:32.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.387+0000 7fe07f7d6740 1 -- 172.21.15.12:0/3611523170 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563805772c10 con 0x563805884a90 2024-04-22T13:33:32.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.388+0000 7fe06dd9b640 1 --2- 172.21.15.12:0/3611523170 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x563805884a90 0x563805886eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.388+0000 7fe06dd9b640 1 --2- 172.21.15.12:0/3611523170 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x563805884a90 0x563805886eb0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7bc7078720644ae1 server_cookie=2859bdc4160cd358 in_seq=0 out_seq=0 2024-04-22T13:33:32.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.388+0000 7fe05dffb640 10 client.18831 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:33:32.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.388+0000 7fe05dffb640 1 -- 172.21.15.12:0/3611523170 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563805772c10 con 0x563805884a90 2024-04-22T13:33:32.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.388+0000 7fe05dffb640 10 client.18831 handle_command_reply: tid=0 2024-04-22T13:33:32.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.388+0000 7fe07f7d6740 1 client.18831 shutdown 2024-04-22T13:33:32.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.388+0000 7fe07f7d6740 1 -- 172.21.15.12:0/3611523170 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638058003b0 msgr2=0x563805820ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.388+0000 7fe07f7d6740 1 --2- 172.21.15.12:0/3611523170 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638058003b0 0x563805820ef0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fe06809a5d0 tx=0x7fe068098110 comp rx=0 tx=0).stop 2024-04-22T13:33:32.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.388+0000 7fe07f7d6740 1 -- 172.21.15.12:0/3611523170 shutdown_connections 2024-04-22T13:33:32.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.388+0000 7fe07f7d6740 1 --2- 172.21.15.12:0/3611523170 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x563805884a90 0x563805886eb0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.388+0000 7fe07f7d6740 1 --2- 172.21.15.12:0/3611523170 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563805809e60 0x563805821970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.388+0000 7fe05dffb640 0 client.18831 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:33:32.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.388+0000 7fe07f7d6740 1 --2- 172.21.15.12:0/3611523170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563805800d60 0x563805821430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.388+0000 7fe07f7d6740 1 --2- 172.21.15.12:0/3611523170 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5638058003b0 0x563805820ef0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.388+0000 7fe07f7d6740 1 -- 172.21.15.12:0/3611523170 >> 172.21.15.12:0/3611523170 conn(0x5638057faba0 msgr2=0x5638057f9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:32.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.388+0000 7fe07f7d6740 1 -- 172.21.15.12:0/3611523170 shutdown_connections 2024-04-22T13:33:32.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.388+0000 7fe07f7d6740 1 -- 172.21.15.12:0/3611523170 wait complete. 2024-04-22T13:33:32.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.389+0000 7fe07f7d6740 20 client.18831 trim_cache size 0 max 16384 2024-04-22T13:33:32.391 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:33:32.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.389+0000 7fe07cbef640 1 -- 172.21.15.12:0/3015320928 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe0640a3d20 msgr2=0x7fe0640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.389+0000 7fe07cbef640 1 --2- 172.21.15.12:0/3015320928 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe0640a3d20 0x7fe0640a61e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe07000afa0 tx=0x7fe070005c80 comp rx=0 tx=0).stop 2024-04-22T13:33:32.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.389+0000 7fe07cbef640 1 -- 172.21.15.12:0/3015320928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe07806a840 msgr2=0x7fe078198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.389+0000 7fe07cbef640 1 --2- 172.21.15.12:0/3015320928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe07806a840 0x7fe078198730 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fe06800d770 tx=0x7fe06800dc30 comp rx=0 tx=0).stop 2024-04-22T13:33:32.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.389+0000 7fe07cbef640 1 -- 172.21.15.12:0/3015320928 shutdown_connections 2024-04-22T13:33:32.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.389+0000 7fe07cbef640 1 --2- 172.21.15.12:0/3015320928 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe0640a3d20 0x7fe0640a61e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.389+0000 7fe07cbef640 1 --2- 172.21.15.12:0/3015320928 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe07819cff0 0x7fe07819d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.389+0000 7fe07cbef640 1 --2- 172.21.15.12:0/3015320928 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe078107270 0x7fe078198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.389+0000 7fe07cbef640 1 --2- 172.21.15.12:0/3015320928 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe07806a840 0x7fe078198730 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.389+0000 7fe07cbef640 1 -- 172.21.15.12:0/3015320928 >> 172.21.15.12:0/3015320928 conn(0x7fe078077eb0 msgr2=0x7fe0780ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:32.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.389+0000 7fe07cbef640 1 -- 172.21.15.12:0/3015320928 shutdown_connections 2024-04-22T13:33:32.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.389+0000 7fe07cbef640 1 -- 172.21.15.12:0/3015320928 wait complete. 2024-04-22T13:33:32.395 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:33:32.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.504+0000 7f824c7a4640 1 -- 172.21.15.12:0/3199403180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82441010a0 msgr2=0x7f8244101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.504+0000 7f824c7a4640 1 --2- 172.21.15.12:0/3199403180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82441010a0 0x7f8244101480 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f8240009900 tx=0x7f824002bc30 comp rx=0 tx=0).stop 2024-04-22T13:33:32.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.504+0000 7f824c7a4640 1 -- 172.21.15.12:0/3199403180 shutdown_connections 2024-04-22T13:33:32.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.504+0000 7f824c7a4640 1 --2- 172.21.15.12:0/3199403180 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8244106170 0x7f824410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.504+0000 7f824c7a4640 1 --2- 172.21.15.12:0/3199403180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8244101a50 0x7f8244105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.504+0000 7f824c7a4640 1 --2- 172.21.15.12:0/3199403180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82441010a0 0x7f8244101480 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.504+0000 7f824c7a4640 1 -- 172.21.15.12:0/3199403180 >> 172.21.15.12:0/3199403180 conn(0x7f8244077eb0 msgr2=0x7f82440782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:32.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.504+0000 7f824c7a4640 1 -- 172.21.15.12:0/3199403180 shutdown_connections 2024-04-22T13:33:32.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.504+0000 7f824c7a4640 1 -- 172.21.15.12:0/3199403180 wait complete. 2024-04-22T13:33:32.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.504+0000 7f824c7a4640 1 Processor -- start 2024-04-22T13:33:32.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.504+0000 7f824c7a4640 1 -- start start 2024-04-22T13:33:32.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.504+0000 7f824c7a4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82441010a0 0x7f824419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.504+0000 7f824c7a4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8244106170 0x7f824419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.505+0000 7f824c7a4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82441985a0 0x7f8244198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.505+0000 7f824c7a4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8244111150 con 0x7f82441985a0 2024-04-22T13:33:32.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.505+0000 7f824c7a4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8244110fd0 con 0x7f82441010a0 2024-04-22T13:33:32.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.505+0000 7f824c7a4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f82441112d0 con 0x7f8244106170 2024-04-22T13:33:32.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.505+0000 7f824ad41640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82441985a0 0x7f8244198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.505+0000 7f824ad41640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82441985a0 0x7f8244198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60540/0 (socket says 172.21.15.12:60540) 2024-04-22T13:33:32.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.505+0000 7f824ad41640 1 -- 172.21.15.12:0/738853237 learned_addr learned my addr 172.21.15.12:0/738853237 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:32.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.505+0000 7f824a540640 1 --2- 172.21.15.12:0/738853237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82441010a0 0x7f824419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.505+0000 7f8249d3f640 1 --2- 172.21.15.12:0/738853237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8244106170 0x7f824419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.505+0000 7f824ad41640 1 -- 172.21.15.12:0/738853237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8244106170 msgr2=0x7f824419e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.505+0000 7f824ad41640 1 --2- 172.21.15.12:0/738853237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8244106170 0x7f824419e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.505+0000 7f824ad41640 1 -- 172.21.15.12:0/738853237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82441010a0 msgr2=0x7f824419e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.505+0000 7f824ad41640 1 --2- 172.21.15.12:0/738853237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82441010a0 0x7f824419e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.505+0000 7f824ad41640 1 -- 172.21.15.12:0/738853237 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82441990c0 con 0x7f82441985a0 2024-04-22T13:33:32.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.505+0000 7f824ad41640 1 --2- 172.21.15.12:0/738853237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82441985a0 0x7f8244198a50 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f823c00eaa0 tx=0x7f823c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:32.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.505+0000 7f823b7fe640 1 -- 172.21.15.12:0/738853237 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c004d20 con 0x7f82441985a0 2024-04-22T13:33:32.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.505+0000 7f823b7fe640 1 -- 172.21.15.12:0/738853237 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f823c005020 con 0x7f82441985a0 2024-04-22T13:33:32.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.505+0000 7f824c7a4640 1 -- 172.21.15.12:0/738853237 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82441993b0 con 0x7f82441985a0 2024-04-22T13:33:32.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.505+0000 7f823b7fe640 1 -- 172.21.15.12:0/738853237 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c0116b0 con 0x7f82441985a0 2024-04-22T13:33:32.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.505+0000 7f824c7a4640 1 -- 172.21.15.12:0/738853237 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82441a5650 con 0x7f82441985a0 2024-04-22T13:33:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.508+0000 7f823b7fe640 1 -- 172.21.15.12:0/738853237 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f823c0051d0 con 0x7f82441985a0 2024-04-22T13:33:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.508+0000 7f824f39a740 1 Processor -- start 2024-04-22T13:33:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.508+0000 7f824f39a740 1 -- start start 2024-04-22T13:33:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.509+0000 7f824f39a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f1db87a3b0 0x55f1db87a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.509+0000 7f824f39a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1db87ad60 0x55f1db883890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.509+0000 7f824f39a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1db883e60 0x55f1db8842c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.509+0000 7f824f39a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f1db7df1f0 con 0x55f1db87a3b0 2024-04-22T13:33:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.509+0000 7f824f39a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f1db77c4b0 con 0x55f1db883e60 2024-04-22T13:33:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.509+0000 7f824f39a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f1db7db1b0 con 0x55f1db87ad60 2024-04-22T13:33:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.509+0000 7f8249d3f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f1db87a3b0 0x55f1db87a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.509+0000 7f8249d3f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f1db87a3b0 0x55f1db87a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60556/0 (socket says 172.21.15.12:60556) 2024-04-22T13:33:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.509+0000 7f8249d3f640 1 -- 172.21.15.12:0/1587458132 learned_addr learned my addr 172.21.15.12:0/1587458132 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:32.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.509+0000 7f824a540640 1 --2- 172.21.15.12:0/1587458132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1db87ad60 0x55f1db883890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.509+0000 7f823b7fe640 1 --2- 172.21.15.12:0/738853237 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f82300a3da0 0x7f82300a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.509+0000 7f8249d3f640 1 --2- 172.21.15.12:0/1587458132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1db883e60 0x55f1db8842c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.509+0000 7f8249d3f640 1 -- 172.21.15.12:0/1587458132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1db87ad60 msgr2=0x55f1db883890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.509+0000 7f8249d3f640 1 --2- 172.21.15.12:0/1587458132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1db87ad60 0x55f1db883890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.509+0000 7f8249d3f640 1 -- 172.21.15.12:0/1587458132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1db883e60 msgr2=0x55f1db8842c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.509+0000 7f8249d3f640 1 --2- 172.21.15.12:0/1587458132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1db883e60 0x55f1db8842c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.509+0000 7f8249d3f640 1 -- 172.21.15.12:0/1587458132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1db76e1b0 con 0x55f1db87a3b0 2024-04-22T13:33:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.509+0000 7f824ad41640 1 --2- 172.21.15.12:0/738853237 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f82300a3da0 0x7f82300a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.509+0000 7f823b7fe640 1 -- 172.21.15.12:0/738853237 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f823c0d08d0 con 0x7f82441985a0 2024-04-22T13:33:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.509+0000 7f8249d3f640 1 --2- 172.21.15.12:0/1587458132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f1db87a3b0 0x55f1db87a790 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f8234009f80 tx=0x7f8234009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f339d556e7026d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.510+0000 7f8238ff9640 1 -- 172.21.15.12:0/1587458132 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234017610 con 0x55f1db87a3b0 2024-04-22T13:33:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.510+0000 7f8238ff9640 1 -- 172.21.15.12:0/1587458132 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f823401c070 con 0x55f1db87a3b0 2024-04-22T13:33:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.510+0000 7f8238ff9640 1 -- 172.21.15.12:0/1587458132 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82340157e0 con 0x55f1db87a3b0 2024-04-22T13:33:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.510+0000 7f824f39a740 1 -- 172.21.15.12:0/1587458132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f1db87a3b0 msgr2=0x55f1db87a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.510+0000 7f824ad41640 1 --2- 172.21.15.12:0/738853237 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f82300a3da0 0x7f82300a6260 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f823c009270 tx=0x7f823c087040 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:32.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.510+0000 7f824f39a740 1 --2- 172.21.15.12:0/1587458132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f1db87a3b0 0x55f1db87a790 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f8234009f80 tx=0x7f8234009570 comp rx=0 tx=0).stop 2024-04-22T13:33:32.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.510+0000 7f824f39a740 1 -- 172.21.15.12:0/1587458132 shutdown_connections 2024-04-22T13:33:32.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.510+0000 7f824f39a740 1 --2- 172.21.15.12:0/1587458132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1db883e60 0x55f1db8842c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.510+0000 7f824f39a740 1 --2- 172.21.15.12:0/1587458132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1db87ad60 0x55f1db883890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.510+0000 7f824f39a740 1 --2- 172.21.15.12:0/1587458132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f1db87a3b0 0x55f1db87a790 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.510+0000 7f824f39a740 1 -- 172.21.15.12:0/1587458132 >> 172.21.15.12:0/1587458132 conn(0x55f1db874ba0 msgr2=0x55f1db874f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:32.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.510+0000 7f824f39a740 1 -- 172.21.15.12:0/1587458132 shutdown_connections 2024-04-22T13:33:32.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.510+0000 7f824f39a740 1 -- 172.21.15.12:0/1587458132 wait complete. 2024-04-22T13:33:32.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.511+0000 7f824f39a740 1 -- start start 2024-04-22T13:33:32.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.511+0000 7f824f39a740 1 Processor -- start 2024-04-22T13:33:32.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.511+0000 7f824f39a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1db87a3b0 0x55f1db88a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.511+0000 7f824f39a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f1db87ad60 0x55f1db8875a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.511+0000 7f824f39a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1db883e60 0x55f1db887ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.511+0000 7f824f39a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f1db88ad40 con 0x55f1db87ad60 2024-04-22T13:33:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.511+0000 7f824f39a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f1db798500 con 0x55f1db87a3b0 2024-04-22T13:33:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.511+0000 7f824f39a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f1db88aec0 con 0x55f1db883e60 2024-04-22T13:33:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.511+0000 7f824f39a740 10 client.0 _ll_register_callbacks cb 0x55f1db766b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.511+0000 7f824a540640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f1db87ad60 0x55f1db8875a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.511+0000 7f824a540640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f1db87ad60 0x55f1db8875a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60566/0 (socket says 172.21.15.12:60566) 2024-04-22T13:33:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.511+0000 7f824a540640 1 -- 172.21.15.12:0/1647879436 learned_addr learned my addr 172.21.15.12:0/1647879436 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.511+0000 7f8249d3f640 1 --2- 172.21.15.12:0/1647879436 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1db87a3b0 0x55f1db88a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.511+0000 7f8249d3f640 1 --2- 172.21.15.12:0/1647879436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1db883e60 0x55f1db887ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.511+0000 7f824a540640 1 -- 172.21.15.12:0/1647879436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1db883e60 msgr2=0x55f1db887ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.511+0000 7f824a540640 1 --2- 172.21.15.12:0/1647879436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1db883e60 0x55f1db887ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.512+0000 7f824a540640 1 -- 172.21.15.12:0/1647879436 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1db87a3b0 msgr2=0x55f1db88a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.512+0000 7f824a540640 1 --2- 172.21.15.12:0/1647879436 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1db87a3b0 0x55f1db88a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.515 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.512+0000 7f824a540640 1 -- 172.21.15.12:0/1647879436 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1db4f13f0 con 0x55f1db87ad60 2024-04-22T13:33:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.512+0000 7f8249d3f640 1 --2- 172.21.15.12:0/1647879436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1db883e60 0x55f1db887ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.512+0000 7f824a540640 1 --2- 172.21.15.12:0/1647879436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f1db87ad60 0x55f1db8875a0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f8240009900 tx=0x7f824002c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb9fa9636fe89bc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.512+0000 7f822dffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:33:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.512+0000 7f822dffb640 1 -- 172.21.15.12:0/1647879436 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240005030 con 0x55f1db87ad60 2024-04-22T13:33:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.512+0000 7f824f39a740 1 -- 172.21.15.12:0/1647879436 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f823401c070 con 0x55f1db87ad60 2024-04-22T13:33:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.512+0000 7f822dffb640 1 -- 172.21.15.12:0/1647879436 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f82400051d0 con 0x55f1db87ad60 2024-04-22T13:33:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.512+0000 7f822dffb640 1 -- 172.21.15.12:0/1647879436 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f82400053d0 con 0x55f1db87ad60 2024-04-22T13:33:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.512+0000 7f824f39a740 10 client.18843 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.512+0000 7f824f39a740 1 -- 172.21.15.12:0/1647879436 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f1db5bb7d0 con 0x55f1db87ad60 2024-04-22T13:33:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.512+0000 7f822dffb640 1 -- 172.21.15.12:0/1647879436 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f823401c070 con 0x55f1db87ad60 2024-04-22T13:33:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.512+0000 7f822dffb640 1 -- 172.21.15.12:0/1647879436 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8240051050 con 0x55f1db87ad60 2024-04-22T13:33:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.513+0000 7f824f39a740 10 client.18843 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.513+0000 7f824f39a740 10 client.18843 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:33:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.513+0000 7f824f39a740 20 client.18843 populate_metadata read hostname 'smithi012' 2024-04-22T13:33:32.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.513+0000 7f824f39a740 1 --2- 172.21.15.12:0/1647879436 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55f1db897f40 0x55f1db89a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.513+0000 7f824f39a740 4 client.18843 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:33:32.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.513+0000 7f824f39a740 1 -- 172.21.15.12:0/1647879436 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1db77e930 con 0x55f1db897f40 2024-04-22T13:33:32.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.513+0000 7f8249d3f640 1 --2- 172.21.15.12:0/1647879436 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55f1db897f40 0x55f1db89a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.513+0000 7f8249d3f640 1 --2- 172.21.15.12:0/1647879436 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55f1db897f40 0x55f1db89a3c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c92aaa7f7b97e631 server_cookie=7ce286eb04870946 in_seq=0 out_seq=0 2024-04-22T13:33:32.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.513+0000 7f822dffb640 10 client.18843 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:33:32.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.514+0000 7f822dffb640 1 -- 172.21.15.12:0/1647879436 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f1db77e930 con 0x55f1db897f40 2024-04-22T13:33:32.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.514+0000 7f822dffb640 10 client.18843 handle_command_reply: tid=0 2024-04-22T13:33:32.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.514+0000 7f824f39a740 1 client.18843 shutdown 2024-04-22T13:33:32.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.514+0000 7f824f39a740 1 -- 172.21.15.12:0/1647879436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f1db87ad60 msgr2=0x55f1db8875a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.514+0000 7f824f39a740 1 --2- 172.21.15.12:0/1647879436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f1db87ad60 0x55f1db8875a0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f8240009900 tx=0x7f824002c9f0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.514+0000 7f824f39a740 1 -- 172.21.15.12:0/1647879436 shutdown_connections 2024-04-22T13:33:32.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.514+0000 7f824f39a740 1 --2- 172.21.15.12:0/1647879436 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55f1db897f40 0x55f1db89a3c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.514+0000 7f824f39a740 1 --2- 172.21.15.12:0/1647879436 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1db883e60 0x55f1db887ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.514+0000 7f824f39a740 1 --2- 172.21.15.12:0/1647879436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f1db87ad60 0x55f1db8875a0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.514+0000 7f822dffb640 0 client.18843 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:33:32.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.514+0000 7f824f39a740 1 --2- 172.21.15.12:0/1647879436 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1db87a3b0 0x55f1db88a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.514+0000 7f824f39a740 1 -- 172.21.15.12:0/1647879436 >> 172.21.15.12:0/1647879436 conn(0x55f1db874ba0 msgr2=0x55f1db8768f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:32.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.514+0000 7f824f39a740 1 -- 172.21.15.12:0/1647879436 shutdown_connections 2024-04-22T13:33:32.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.514+0000 7f824f39a740 1 -- 172.21.15.12:0/1647879436 wait complete. 2024-04-22T13:33:32.518 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.514+0000 7f824f39a740 20 client.18843 trim_cache size 0 max 16384 2024-04-22T13:33:32.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.527+0000 7f824f39a740 1 Processor -- start 2024-04-22T13:33:32.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.528+0000 7f824f39a740 1 -- start start 2024-04-22T13:33:32.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.528+0000 7f824f39a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1db87a3b0 0x55f1db893c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.528+0000 7f824f39a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f1db87ad60 0x55f1db89fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.528+0000 7f824f39a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1db883e60 0x55f1db8a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.528+0000 7f824f39a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f82441ab9b0 con 0x55f1db87ad60 2024-04-22T13:33:32.529 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.528+0000 7f824f39a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f82441ad4f0 con 0x55f1db883e60 2024-04-22T13:33:32.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.528+0000 7f824f39a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f82441ac080 con 0x55f1db87a3b0 2024-04-22T13:33:32.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.528+0000 7f824a540640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f1db87ad60 0x55f1db89fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.528+0000 7f824a540640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f1db87ad60 0x55f1db89fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60576/0 (socket says 172.21.15.12:60576) 2024-04-22T13:33:32.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.528+0000 7f824a540640 1 -- 172.21.15.12:0/2095384040 learned_addr learned my addr 172.21.15.12:0/2095384040 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:32.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.528+0000 7f8249d3f640 1 --2- 172.21.15.12:0/2095384040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1db87a3b0 0x55f1db893c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.528+0000 7f8249d3f640 1 --2- 172.21.15.12:0/2095384040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1db883e60 0x55f1db8a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.528+0000 7f824a540640 1 -- 172.21.15.12:0/2095384040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1db87a3b0 msgr2=0x55f1db893c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.528+0000 7f824a540640 1 --2- 172.21.15.12:0/2095384040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1db87a3b0 0x55f1db893c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.528+0000 7f824a540640 1 -- 172.21.15.12:0/2095384040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1db883e60 msgr2=0x55f1db8a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.528+0000 7f824a540640 1 --2- 172.21.15.12:0/2095384040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1db883e60 0x55f1db8a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.528+0000 7f824a540640 1 -- 172.21.15.12:0/2095384040 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1db8845b0 con 0x55f1db87ad60 2024-04-22T13:33:32.530 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.528+0000 7f8249d3f640 1 --2- 172.21.15.12:0/2095384040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1db87a3b0 0x55f1db893c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:32.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.528+0000 7f824a540640 1 --2- 172.21.15.12:0/2095384040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f1db87ad60 0x55f1db89fc90 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f824002c170 tx=0x7f824002cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d46d981b7c8774d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:32.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.528+0000 7f822ffff640 1 -- 172.21.15.12:0/2095384040 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240005850 con 0x55f1db87ad60 2024-04-22T13:33:32.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.528+0000 7f822ffff640 1 -- 172.21.15.12:0/2095384040 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82400035f0 con 0x55f1db87ad60 2024-04-22T13:33:32.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.528+0000 7f822ffff640 1 -- 172.21.15.12:0/2095384040 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240052430 con 0x55f1db87ad60 2024-04-22T13:33:32.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.529+0000 7f824f39a740 1 -- 172.21.15.12:0/2095384040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f1db87ad60 msgr2=0x55f1db89fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.529+0000 7f824f39a740 1 --2- 172.21.15.12:0/2095384040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f1db87ad60 0x55f1db89fc90 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f824002c170 tx=0x7f824002cd70 comp rx=0 tx=0).stop 2024-04-22T13:33:32.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.529+0000 7f824f39a740 1 -- 172.21.15.12:0/2095384040 shutdown_connections 2024-04-22T13:33:32.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.529+0000 7f824f39a740 1 --2- 172.21.15.12:0/2095384040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1db883e60 0x55f1db8a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.529+0000 7f824f39a740 1 --2- 172.21.15.12:0/2095384040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f1db87ad60 0x55f1db89fc90 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.529+0000 7f824f39a740 1 --2- 172.21.15.12:0/2095384040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1db87a3b0 0x55f1db893c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.529+0000 7f824f39a740 1 -- 172.21.15.12:0/2095384040 >> 172.21.15.12:0/2095384040 conn(0x55f1db874ba0 msgr2=0x55f1db887560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:32.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.529+0000 7f824f39a740 1 -- 172.21.15.12:0/2095384040 shutdown_connections 2024-04-22T13:33:32.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.529+0000 7f824f39a740 1 -- 172.21.15.12:0/2095384040 wait complete. 2024-04-22T13:33:32.531 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.529+0000 7f824f39a740 1 -- start start 2024-04-22T13:33:32.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.529+0000 7f824f39a740 1 Processor -- start 2024-04-22T13:33:32.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.530+0000 7f824f39a740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f1db87a3b0 0x55f1db8905a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.530+0000 7f824f39a740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1db87ad60 0x55f1db89f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.530+0000 7f824f39a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1db883e60 0x55f1db89f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.530+0000 7f824f39a740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f82441ace20 con 0x55f1db87a3b0 2024-04-22T13:33:32.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.530+0000 7f824f39a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f82441ac750 con 0x55f1db883e60 2024-04-22T13:33:32.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.530+0000 7f824f39a740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f82441a99b0 con 0x55f1db87ad60 2024-04-22T13:33:32.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.530+0000 7f824f39a740 10 client.0 _ll_register_callbacks cb 0x7f8240004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:32.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.530+0000 7f8249d3f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f1db87a3b0 0x55f1db8905a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.530+0000 7f8249d3f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f1db87a3b0 0x55f1db8905a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60590/0 (socket says 172.21.15.12:60590) 2024-04-22T13:33:32.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.530+0000 7f8249d3f640 1 -- 172.21.15.12:0/237247920 learned_addr learned my addr 172.21.15.12:0/237247920 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:32.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.530+0000 7f824a540640 1 --2- 172.21.15.12:0/237247920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1db87ad60 0x55f1db89f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.532 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.530+0000 7f8249d3f640 1 -- 172.21.15.12:0/237247920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1db87ad60 msgr2=0x55f1db89f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.530+0000 7f8249d3f640 1 --2- 172.21.15.12:0/237247920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1db87ad60 0x55f1db89f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.530+0000 7f8249d3f640 1 -- 172.21.15.12:0/237247920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1db883e60 msgr2=0x55f1db89f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.530+0000 7f8249d3f640 1 --2- 172.21.15.12:0/237247920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1db883e60 0x55f1db89f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.530+0000 7f8249d3f640 1 -- 172.21.15.12:0/237247920 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1db89fe20 con 0x55f1db87a3b0 2024-04-22T13:33:32.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.530+0000 7f8249d3f640 1 --2- 172.21.15.12:0/237247920 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f1db87a3b0 0x55f1db8905a0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x55f1db86f190 tx=0x7f8234009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6af88d141ebdce server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:32.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.530+0000 7f822dffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:33:32.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.530+0000 7f822dffb640 1 -- 172.21.15.12:0/237247920 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823402c570 con 0x55f1db87a3b0 2024-04-22T13:33:32.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.530+0000 7f822dffb640 1 -- 172.21.15.12:0/237247920 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f823402c710 con 0x55f1db87a3b0 2024-04-22T13:33:32.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.530+0000 7f824f39a740 1 -- 172.21.15.12:0/237247920 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82400035f0 con 0x55f1db87a3b0 2024-04-22T13:33:32.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.531+0000 7f822dffb640 1 -- 172.21.15.12:0/237247920 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f823402c910 con 0x55f1db87a3b0 2024-04-22T13:33:32.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.531+0000 7f824f39a740 10 client.18849 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:32.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.531+0000 7f824f39a740 1 -- 172.21.15.12:0/237247920 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f1db8fe580 con 0x55f1db87a3b0 2024-04-22T13:33:32.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.531+0000 7f822dffb640 1 -- 172.21.15.12:0/237247920 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82400035f0 con 0x55f1db87a3b0 2024-04-22T13:33:32.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.531+0000 7f822dffb640 1 -- 172.21.15.12:0/237247920 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8234015750 con 0x55f1db87a3b0 2024-04-22T13:33:32.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.531+0000 7f824f39a740 10 client.18849 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:32.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.531+0000 7f824f39a740 10 client.18849 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:33:32.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.531+0000 7f824f39a740 20 client.18849 populate_metadata read hostname 'smithi012' 2024-04-22T13:33:32.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.531+0000 7f824f39a740 1 --2- 172.21.15.12:0/237247920 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55f1db8fe9c0 0x55f1db900de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.531+0000 7f824f39a740 4 client.18849 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:33:32.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.531+0000 7f824f39a740 1 -- 172.21.15.12:0/237247920 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f1db7ecc10 con 0x55f1db8fe9c0 2024-04-22T13:33:32.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.531+0000 7f824a540640 1 --2- 172.21.15.12:0/237247920 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55f1db8fe9c0 0x55f1db900de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.532+0000 7f822dffb640 10 client.18849 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:33:32.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.532+0000 7f824a540640 1 --2- 172.21.15.12:0/237247920 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55f1db8fe9c0 0x55f1db900de0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=82da4f2c89a0fa21 server_cookie=70cf8b01d021f08a in_seq=0 out_seq=0 2024-04-22T13:33:32.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.532+0000 7f822dffb640 1 -- 172.21.15.12:0/237247920 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f1db7ecc10 con 0x55f1db8fe9c0 2024-04-22T13:33:32.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.532+0000 7f822dffb640 10 client.18849 handle_command_reply: tid=0 2024-04-22T13:33:32.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.532+0000 7f824f39a740 1 client.18849 shutdown 2024-04-22T13:33:32.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.532+0000 7f824f39a740 1 -- 172.21.15.12:0/237247920 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f1db87a3b0 msgr2=0x55f1db8905a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.532+0000 7f824f39a740 1 --2- 172.21.15.12:0/237247920 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f1db87a3b0 0x55f1db8905a0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x55f1db86f190 tx=0x7f8234009110 comp rx=0 tx=0).stop 2024-04-22T13:33:32.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.532+0000 7f824f39a740 1 -- 172.21.15.12:0/237247920 shutdown_connections 2024-04-22T13:33:32.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.532+0000 7f824f39a740 1 --2- 172.21.15.12:0/237247920 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55f1db8fe9c0 0x55f1db900de0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.532+0000 7f824f39a740 1 --2- 172.21.15.12:0/237247920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1db883e60 0x55f1db89f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.532+0000 7f824f39a740 1 --2- 172.21.15.12:0/237247920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f1db87ad60 0x55f1db89f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.532+0000 7f822dffb640 0 client.18849 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:33:32.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.532+0000 7f824f39a740 1 --2- 172.21.15.12:0/237247920 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f1db87a3b0 0x55f1db8905a0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.532+0000 7f824f39a740 1 -- 172.21.15.12:0/237247920 >> 172.21.15.12:0/237247920 conn(0x55f1db874ba0 msgr2=0x55f1db8817f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:32.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.532+0000 7f824f39a740 1 -- 172.21.15.12:0/237247920 shutdown_connections 2024-04-22T13:33:32.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.532+0000 7f824f39a740 1 -- 172.21.15.12:0/237247920 wait complete. 2024-04-22T13:33:32.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.532+0000 7f824f39a740 20 client.18849 trim_cache size 0 max 16384 2024-04-22T13:33:32.536 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:33:32.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.532+0000 7f824c7a4640 1 -- 172.21.15.12:0/738853237 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f82300a3da0 msgr2=0x7f82300a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.532+0000 7f824c7a4640 1 --2- 172.21.15.12:0/738853237 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f82300a3da0 0x7f82300a6260 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f823c009270 tx=0x7f823c087040 comp rx=0 tx=0).stop 2024-04-22T13:33:32.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.532+0000 7f824c7a4640 1 -- 172.21.15.12:0/738853237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82441985a0 msgr2=0x7f8244198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.532+0000 7f824c7a4640 1 --2- 172.21.15.12:0/738853237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82441985a0 0x7f8244198a50 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f823c00eaa0 tx=0x7f823c00eea0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.533+0000 7f824c7a4640 1 -- 172.21.15.12:0/738853237 shutdown_connections 2024-04-22T13:33:32.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.533+0000 7f824c7a4640 1 --2- 172.21.15.12:0/738853237 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f82300a3da0 0x7f82300a6260 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.533+0000 7f824c7a4640 1 --2- 172.21.15.12:0/738853237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f82441985a0 0x7f8244198a50 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.533+0000 7f824c7a4640 1 --2- 172.21.15.12:0/738853237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8244106170 0x7f824419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.533+0000 7f824c7a4640 1 --2- 172.21.15.12:0/738853237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82441010a0 0x7f824419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.533+0000 7f824c7a4640 1 -- 172.21.15.12:0/738853237 >> 172.21.15.12:0/738853237 conn(0x7f8244077eb0 msgr2=0x7f824406a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:32.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.533+0000 7f824c7a4640 1 -- 172.21.15.12:0/738853237 shutdown_connections 2024-04-22T13:33:32.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.533+0000 7f824c7a4640 1 -- 172.21.15.12:0/738853237 wait complete. 2024-04-22T13:33:32.539 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:33:32.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.648+0000 7f8e7d7c7640 1 -- 172.21.15.12:0/284337387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e78102b30 msgr2=0x7f8e78106b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.648+0000 7f8e7d7c7640 1 --2- 172.21.15.12:0/284337387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e78102b30 0x7f8e78106b70 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c009900 tx=0x7f8e6c02bc30 comp rx=0 tx=0).stop 2024-04-22T13:33:32.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.648+0000 7f8e7d7c7640 1 -- 172.21.15.12:0/284337387 shutdown_connections 2024-04-22T13:33:32.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.648+0000 7f8e7d7c7640 1 --2- 172.21.15.12:0/284337387 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e78107250 0x7f8e7810ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.648+0000 7f8e7d7c7640 1 --2- 172.21.15.12:0/284337387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e78102b30 0x7f8e78106b70 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.648+0000 7f8e7d7c7640 1 --2- 172.21.15.12:0/284337387 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e78102180 0x7f8e78102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.648+0000 7f8e7d7c7640 1 -- 172.21.15.12:0/284337387 >> 172.21.15.12:0/284337387 conn(0x7f8e780fd9c0 msgr2=0x7f8e780ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:32.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.649+0000 7f8e7d7c7640 1 -- 172.21.15.12:0/284337387 shutdown_connections 2024-04-22T13:33:32.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.649+0000 7f8e7d7c7640 1 -- 172.21.15.12:0/284337387 wait complete. 2024-04-22T13:33:32.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.649+0000 7f8e7d7c7640 1 Processor -- start 2024-04-22T13:33:32.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.649+0000 7f8e7d7c7640 1 -- start start 2024-04-22T13:33:32.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.649+0000 7f8e7d7c7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e78102180 0x7f8e7819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.649+0000 7f8e7d7c7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e78107250 0x7f8e7819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.649+0000 7f8e7d7c7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e781985f0 0x7f8e78198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.649+0000 7f8e7d7c7640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8e78112170 con 0x7f8e78107250 2024-04-22T13:33:32.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.649+0000 7f8e7d7c7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8e78111ff0 con 0x7f8e78102180 2024-04-22T13:33:32.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.649+0000 7f8e7d7c7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8e781122f0 con 0x7f8e781985f0 2024-04-22T13:33:32.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.650+0000 7f8e767fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e78107250 0x7f8e7819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.650+0000 7f8e767fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e78107250 0x7f8e7819e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60600/0 (socket says 172.21.15.12:60600) 2024-04-22T13:33:32.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.650+0000 7f8e767fc640 1 -- 172.21.15.12:0/3017253637 learned_addr learned my addr 172.21.15.12:0/3017253637 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:32.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.650+0000 7f8e777fe640 1 --2- 172.21.15.12:0/3017253637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e781985f0 0x7f8e78198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.650+0000 7f8e76ffd640 1 --2- 172.21.15.12:0/3017253637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e78102180 0x7f8e7819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.650+0000 7f8e767fc640 1 -- 172.21.15.12:0/3017253637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e781985f0 msgr2=0x7f8e78198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.657 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.650+0000 7f8e767fc640 1 --2- 172.21.15.12:0/3017253637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e781985f0 0x7f8e78198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.650+0000 7f8e767fc640 1 -- 172.21.15.12:0/3017253637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e78102180 msgr2=0x7f8e7819e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.650+0000 7f8e767fc640 1 --2- 172.21.15.12:0/3017253637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e78102180 0x7f8e7819e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.650+0000 7f8e767fc640 1 -- 172.21.15.12:0/3017253637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e78199110 con 0x7f8e78107250 2024-04-22T13:33:32.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.650+0000 7f8e767fc640 1 --2- 172.21.15.12:0/3017253637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e78107250 0x7f8e7819e930 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c036040 tx=0x7f8e6c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:32.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.650+0000 7f8e5ffff640 1 -- 172.21.15.12:0/3017253637 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c043020 con 0x7f8e78107250 2024-04-22T13:33:32.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.650+0000 7f8e5ffff640 1 -- 172.21.15.12:0/3017253637 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e6c02cb70 con 0x7f8e78107250 2024-04-22T13:33:32.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.650+0000 7f8e7d7c7640 1 -- 172.21.15.12:0/3017253637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e781993a0 con 0x7f8e78107250 2024-04-22T13:33:32.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.650+0000 7f8e5ffff640 1 -- 172.21.15.12:0/3017253637 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c003bf0 con 0x7f8e78107250 2024-04-22T13:33:32.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.650+0000 7f8e7d7c7640 1 -- 172.21.15.12:0/3017253637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e781a55a0 con 0x7f8e78107250 2024-04-22T13:33:32.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.653+0000 7f8e5ffff640 1 -- 172.21.15.12:0/3017253637 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8e6c046020 con 0x7f8e78107250 2024-04-22T13:33:32.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.653+0000 7f8e803b9740 1 Processor -- start 2024-04-22T13:33:32.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.653+0000 7f8e803b9740 1 -- start start 2024-04-22T13:33:32.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.654+0000 7f8e803b9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d0e7a34d0 0x561d0e7a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.658 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.654+0000 7f8e803b9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d0e7a3e80 0x561d0e7ac430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.654+0000 7f8e803b9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d0e7aca00 0x561d0e7aee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.654+0000 7f8e803b9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561d0e7081f0 con 0x561d0e7aca00 2024-04-22T13:33:32.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.654+0000 7f8e803b9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561d0e6a54b0 con 0x561d0e7a3e80 2024-04-22T13:33:32.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.654+0000 7f8e803b9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561d0e7041b0 con 0x561d0e7a34d0 2024-04-22T13:33:32.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.654+0000 7f8e767fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d0e7aca00 0x561d0e7aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.654+0000 7f8e767fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d0e7aca00 0x561d0e7aee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60608/0 (socket says 172.21.15.12:60608) 2024-04-22T13:33:32.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.654+0000 7f8e767fc640 1 -- 172.21.15.12:0/1433114683 learned_addr learned my addr 172.21.15.12:0/1433114683 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:32.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.654+0000 7f8e777fe640 1 --2- 172.21.15.12:0/1433114683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d0e7a34d0 0x561d0e7a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.654+0000 7f8e5ffff640 1 --2- 172.21.15.12:0/3017253637 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f8e600a3d20 0x7f8e600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.654+0000 7f8e767fc640 1 -- 172.21.15.12:0/1433114683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d0e7a34d0 msgr2=0x561d0e7a38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.654+0000 7f8e76ffd640 1 --2- 172.21.15.12:0/1433114683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d0e7a3e80 0x561d0e7ac430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.654+0000 7f8e767fc640 1 --2- 172.21.15.12:0/1433114683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d0e7a34d0 0x561d0e7a38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.659 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.654+0000 7f8e767fc640 1 -- 172.21.15.12:0/1433114683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d0e7a3e80 msgr2=0x561d0e7ac430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.654+0000 7f8e767fc640 1 --2- 172.21.15.12:0/1433114683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d0e7a3e80 0x561d0e7ac430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.654+0000 7f8e767fc640 1 -- 172.21.15.12:0/1433114683 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d0e6971b0 con 0x561d0e7aca00 2024-04-22T13:33:32.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.654+0000 7f8e777fe640 1 --2- 172.21.15.12:0/3017253637 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f8e600a3d20 0x7f8e600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.654+0000 7f8e5ffff640 1 -- 172.21.15.12:0/3017253637 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f8e6c0f27d0 con 0x7f8e78107250 2024-04-22T13:33:32.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.654+0000 7f8e767fc640 1 --2- 172.21.15.12:0/1433114683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d0e7aca00 0x561d0e7aee50 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0effc0 tx=0x7f8e6c0e99b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7c2ff1ec6ed1396 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:32.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.654+0000 7f8e5d7fa640 1 -- 172.21.15.12:0/1433114683 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c0be3e0 con 0x561d0e7aca00 2024-04-22T13:33:32.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.655+0000 7f8e5d7fa640 1 -- 172.21.15.12:0/1433114683 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e6c0befa0 con 0x561d0e7aca00 2024-04-22T13:33:32.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.655+0000 7f8e5d7fa640 1 -- 172.21.15.12:0/1433114683 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c0bce60 con 0x561d0e7aca00 2024-04-22T13:33:32.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.655+0000 7f8e803b9740 1 -- 172.21.15.12:0/1433114683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d0e7aca00 msgr2=0x561d0e7aee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.655+0000 7f8e777fe640 1 --2- 172.21.15.12:0/3017253637 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f8e600a3d20 0x7f8e600a61e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8e70005fd0 tx=0x7f8e70014040 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:32.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.655+0000 7f8e803b9740 1 --2- 172.21.15.12:0/1433114683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d0e7aca00 0x561d0e7aee50 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0effc0 tx=0x7f8e6c0e99b0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.655+0000 7f8e803b9740 1 -- 172.21.15.12:0/1433114683 shutdown_connections 2024-04-22T13:33:32.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.655+0000 7f8e803b9740 1 --2- 172.21.15.12:0/1433114683 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d0e7aca00 0x561d0e7aee50 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.660 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.655+0000 7f8e803b9740 1 --2- 172.21.15.12:0/1433114683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d0e7a3e80 0x561d0e7ac430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.655+0000 7f8e803b9740 1 --2- 172.21.15.12:0/1433114683 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d0e7a34d0 0x561d0e7a38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.655+0000 7f8e803b9740 1 -- 172.21.15.12:0/1433114683 >> 172.21.15.12:0/1433114683 conn(0x561d0e79dba0 msgr2=0x561d0e79df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:32.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.655+0000 7f8e803b9740 1 -- 172.21.15.12:0/1433114683 shutdown_connections 2024-04-22T13:33:32.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.655+0000 7f8e803b9740 1 -- 172.21.15.12:0/1433114683 wait complete. 2024-04-22T13:33:32.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.656+0000 7f8e803b9740 1 -- start start 2024-04-22T13:33:32.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.656+0000 7f8e803b9740 1 Processor -- start 2024-04-22T13:33:32.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.656+0000 7f8e803b9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d0e7a34d0 0x561d0e7ac2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.656+0000 7f8e803b9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d0e7a3e80 0x561d0e7a9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.656+0000 7f8e803b9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d0e7aca00 0x561d0e7a9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.656+0000 7f8e803b9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561d0e7af730 con 0x561d0e7aca00 2024-04-22T13:33:32.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.656+0000 7f8e803b9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561d0e6c1500 con 0x561d0e7a3e80 2024-04-22T13:33:32.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.656+0000 7f8e803b9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561d0e7af8b0 con 0x561d0e7a34d0 2024-04-22T13:33:32.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.656+0000 7f8e803b9740 10 client.0 _ll_register_callbacks cb 0x561d0e68fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:32.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.656+0000 7f8e777fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d0e7aca00 0x561d0e7a9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.661 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.656+0000 7f8e76ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d0e7a34d0 0x561d0e7ac2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.656+0000 7f8e777fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d0e7aca00 0x561d0e7a9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60610/0 (socket says 172.21.15.12:60610) 2024-04-22T13:33:32.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.656+0000 7f8e777fe640 1 -- 172.21.15.12:0/600086959 learned_addr learned my addr 172.21.15.12:0/600086959 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:32.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.656+0000 7f8e767fc640 1 --2- 172.21.15.12:0/600086959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d0e7a3e80 0x561d0e7a9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.656+0000 7f8e777fe640 1 -- 172.21.15.12:0/600086959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d0e7a34d0 msgr2=0x561d0e7ac2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.656+0000 7f8e777fe640 1 --2- 172.21.15.12:0/600086959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d0e7a34d0 0x561d0e7ac2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.656+0000 7f8e777fe640 1 -- 172.21.15.12:0/600086959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d0e7a3e80 msgr2=0x561d0e7a9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.656+0000 7f8e777fe640 1 --2- 172.21.15.12:0/600086959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d0e7a3e80 0x561d0e7a9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.656+0000 7f8e777fe640 1 -- 172.21.15.12:0/600086959 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d0e41a3f0 con 0x561d0e7aca00 2024-04-22T13:33:32.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.656+0000 7f8e767fc640 1 --2- 172.21.15.12:0/600086959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d0e7a3e80 0x561d0e7a9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:32.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.656+0000 7f8e777fe640 1 --2- 172.21.15.12:0/600086959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d0e7aca00 0x561d0e7a9c00 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f8e70009ee0 tx=0x7f8e70009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ffe5e77a9e08239 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:32.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.656+0000 7f8e5aff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:33:32.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.657+0000 7f8e5aff5640 1 -- 172.21.15.12:0/600086959 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7001c890 con 0x561d0e7aca00 2024-04-22T13:33:32.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.657+0000 7f8e5aff5640 1 -- 172.21.15.12:0/600086959 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f8e7001ca30 con 0x561d0e7aca00 2024-04-22T13:33:32.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.657+0000 7f8e803b9740 1 -- 172.21.15.12:0/600086959 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e6c0befa0 con 0x561d0e7aca00 2024-04-22T13:33:32.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.657+0000 7f8e5aff5640 1 -- 172.21.15.12:0/600086959 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8e7001cc30 con 0x561d0e7aca00 2024-04-22T13:33:32.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.657+0000 7f8e803b9740 10 client.18861 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:32.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.657+0000 7f8e803b9740 1 -- 172.21.15.12:0/600086959 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561d0e4e47d0 con 0x561d0e7aca00 2024-04-22T13:33:32.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.657+0000 7f8e5aff5640 1 -- 172.21.15.12:0/600086959 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e6c0befa0 con 0x561d0e7aca00 2024-04-22T13:33:32.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.657+0000 7f8e5aff5640 1 -- 172.21.15.12:0/600086959 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8e70025020 con 0x561d0e7aca00 2024-04-22T13:33:32.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.657+0000 7f8e803b9740 10 client.18861 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:32.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.657+0000 7f8e803b9740 10 client.18861 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:33:32.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.657+0000 7f8e803b9740 20 client.18861 populate_metadata read hostname 'smithi012' 2024-04-22T13:33:32.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.657+0000 7f8e803b9740 1 --2- 172.21.15.12:0/600086959 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x561d0e7bc850 0x561d0e7becb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.657+0000 7f8e803b9740 4 client.18861 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:33:32.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.657+0000 7f8e803b9740 1 -- 172.21.15.12:0/600086959 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d0e6a7930 con 0x561d0e7bc850 2024-04-22T13:33:32.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.658+0000 7f8e76ffd640 1 --2- 172.21.15.12:0/600086959 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x561d0e7bc850 0x561d0e7becb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.658+0000 7f8e76ffd640 1 --2- 172.21.15.12:0/600086959 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x561d0e7bc850 0x561d0e7becb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b2f883b0e75ec5e server_cookie=6369cafbdf76141 in_seq=0 out_seq=0 2024-04-22T13:33:32.663 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.658+0000 7f8e5aff5640 10 client.18861 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:33:32.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.659+0000 7f8e5aff5640 1 -- 172.21.15.12:0/600086959 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561d0e6a7930 con 0x561d0e7bc850 2024-04-22T13:33:32.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.659+0000 7f8e5aff5640 10 client.18861 handle_command_reply: tid=0 2024-04-22T13:33:32.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.659+0000 7f8e803b9740 1 client.18861 shutdown 2024-04-22T13:33:32.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.659+0000 7f8e803b9740 1 -- 172.21.15.12:0/600086959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d0e7aca00 msgr2=0x561d0e7a9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.659+0000 7f8e803b9740 1 --2- 172.21.15.12:0/600086959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d0e7aca00 0x561d0e7a9c00 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f8e70009ee0 tx=0x7f8e70009500 comp rx=0 tx=0).stop 2024-04-22T13:33:32.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.659+0000 7f8e803b9740 1 -- 172.21.15.12:0/600086959 shutdown_connections 2024-04-22T13:33:32.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.659+0000 7f8e803b9740 1 --2- 172.21.15.12:0/600086959 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x561d0e7bc850 0x561d0e7becb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.659+0000 7f8e803b9740 1 --2- 172.21.15.12:0/600086959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d0e7aca00 0x561d0e7a9c00 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.659+0000 7f8e803b9740 1 --2- 172.21.15.12:0/600086959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d0e7a3e80 0x561d0e7a9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.659+0000 7f8e803b9740 1 --2- 172.21.15.12:0/600086959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d0e7a34d0 0x561d0e7ac2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.659+0000 7f8e5aff5640 0 client.18861 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:33:32.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.659+0000 7f8e803b9740 1 -- 172.21.15.12:0/600086959 >> 172.21.15.12:0/600086959 conn(0x561d0e79dba0 msgr2=0x561d0e79f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:32.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.659+0000 7f8e803b9740 1 -- 172.21.15.12:0/600086959 shutdown_connections 2024-04-22T13:33:32.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.659+0000 7f8e803b9740 1 -- 172.21.15.12:0/600086959 wait complete. 2024-04-22T13:33:32.664 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.659+0000 7f8e803b9740 20 client.18861 trim_cache size 0 max 16384 2024-04-22T13:33:32.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.672+0000 7f8e803b9740 1 Processor -- start 2024-04-22T13:33:32.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.672+0000 7f8e803b9740 1 -- start start 2024-04-22T13:33:32.672 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.672+0000 7f8e803b9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d0e7a34d0 0x561d0e7b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.672+0000 7f8e803b9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d0e7a3e80 0x561d0e7abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.672+0000 7f8e803b9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d0e7aca00 0x561d0e7ac2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.672+0000 7f8e803b9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8e781adb90 con 0x561d0e7a34d0 2024-04-22T13:33:32.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.672+0000 7f8e803b9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8e781af6d0 con 0x561d0e7aca00 2024-04-22T13:33:32.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.672+0000 7f8e803b9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8e781ae260 con 0x561d0e7a3e80 2024-04-22T13:33:32.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.673+0000 7f8e76ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d0e7a34d0 0x561d0e7b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.673+0000 7f8e76ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d0e7a34d0 0x561d0e7b4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60626/0 (socket says 172.21.15.12:60626) 2024-04-22T13:33:32.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.673+0000 7f8e76ffd640 1 -- 172.21.15.12:0/1002017511 learned_addr learned my addr 172.21.15.12:0/1002017511 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:32.673 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.673+0000 7f8e767fc640 1 --2- 172.21.15.12:0/1002017511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d0e7a3e80 0x561d0e7abdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.673+0000 7f8e76ffd640 1 -- 172.21.15.12:0/1002017511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d0e7a3e80 msgr2=0x561d0e7abdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.673+0000 7f8e76ffd640 1 --2- 172.21.15.12:0/1002017511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d0e7a3e80 0x561d0e7abdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.673+0000 7f8e76ffd640 1 -- 172.21.15.12:0/1002017511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d0e7aca00 msgr2=0x561d0e7ac2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.673+0000 7f8e76ffd640 1 --2- 172.21.15.12:0/1002017511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d0e7aca00 0x561d0e7ac2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.673+0000 7f8e76ffd640 1 -- 172.21.15.12:0/1002017511 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d0e7af1c0 con 0x561d0e7a34d0 2024-04-22T13:33:32.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.673+0000 7f8e767fc640 1 --2- 172.21.15.12:0/1002017511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d0e7a3e80 0x561d0e7abdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:32.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.673+0000 7f8e76ffd640 1 --2- 172.21.15.12:0/1002017511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d0e7a34d0 0x561d0e7b4e50 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f8e68000c00 tx=0x7f8e6800bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1883bdf482bb9b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:32.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.673+0000 7f8e5cff9640 1 -- 172.21.15.12:0/1002017511 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e68003d20 con 0x561d0e7a34d0 2024-04-22T13:33:32.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.673+0000 7f8e5cff9640 1 -- 172.21.15.12:0/1002017511 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e6801e520 con 0x561d0e7a34d0 2024-04-22T13:33:32.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.673+0000 7f8e5cff9640 1 -- 172.21.15.12:0/1002017511 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e68012a80 con 0x561d0e7a34d0 2024-04-22T13:33:32.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.674+0000 7f8e803b9740 1 -- 172.21.15.12:0/1002017511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d0e7a34d0 msgr2=0x561d0e7b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.674+0000 7f8e803b9740 1 --2- 172.21.15.12:0/1002017511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d0e7a34d0 0x561d0e7b4e50 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f8e68000c00 tx=0x7f8e6800bcf0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.674+0000 7f8e803b9740 1 -- 172.21.15.12:0/1002017511 shutdown_connections 2024-04-22T13:33:32.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.674+0000 7f8e803b9740 1 --2- 172.21.15.12:0/1002017511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d0e7aca00 0x561d0e7ac2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.674+0000 7f8e803b9740 1 --2- 172.21.15.12:0/1002017511 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d0e7a3e80 0x561d0e7abdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.674+0000 7f8e803b9740 1 --2- 172.21.15.12:0/1002017511 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d0e7a34d0 0x561d0e7b4e50 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.674+0000 7f8e803b9740 1 -- 172.21.15.12:0/1002017511 >> 172.21.15.12:0/1002017511 conn(0x561d0e79dba0 msgr2=0x561d0e7a9640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:32.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.674+0000 7f8e803b9740 1 -- 172.21.15.12:0/1002017511 shutdown_connections 2024-04-22T13:33:32.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.674+0000 7f8e803b9740 1 -- 172.21.15.12:0/1002017511 wait complete. 2024-04-22T13:33:32.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.674+0000 7f8e803b9740 1 -- start start 2024-04-22T13:33:32.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.674+0000 7f8e803b9740 1 Processor -- start 2024-04-22T13:33:32.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.674+0000 7f8e803b9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d0e7a34d0 0x561d0e7a7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.674+0000 7f8e803b9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d0e7a3e80 0x561d0e7a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.674+0000 7f8e803b9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d0e7aca00 0x561d0e7a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.674+0000 7f8e803b9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f8e781af000 con 0x561d0e7aca00 2024-04-22T13:33:32.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.674+0000 7f8e803b9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8e781ae930 con 0x561d0e7a34d0 2024-04-22T13:33:32.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.674+0000 7f8e803b9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8e781abb90 con 0x561d0e7a3e80 2024-04-22T13:33:32.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.674+0000 7f8e803b9740 10 client.0 _ll_register_callbacks cb 0x7f8e700104e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:32.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.674+0000 7f8e777fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d0e7aca00 0x561d0e7a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.674+0000 7f8e777fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d0e7aca00 0x561d0e7a60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60636/0 (socket says 172.21.15.12:60636) 2024-04-22T13:33:32.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.674+0000 7f8e777fe640 1 -- 172.21.15.12:0/2050810381 learned_addr learned my addr 172.21.15.12:0/2050810381 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:32.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.675+0000 7f8e777fe640 1 -- 172.21.15.12:0/2050810381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d0e7a3e80 msgr2=0x561d0e7a5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.675+0000 7f8e777fe640 1 --2- 172.21.15.12:0/2050810381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d0e7a3e80 0x561d0e7a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.675+0000 7f8e777fe640 1 -- 172.21.15.12:0/2050810381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d0e7a34d0 msgr2=0x561d0e7a7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.675+0000 7f8e777fe640 1 --2- 172.21.15.12:0/2050810381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d0e7a34d0 0x561d0e7a7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.675+0000 7f8e777fe640 1 -- 172.21.15.12:0/2050810381 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d0e7a6760 con 0x561d0e7aca00 2024-04-22T13:33:32.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.675+0000 7f8e777fe640 1 --2- 172.21.15.12:0/2050810381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d0e7aca00 0x561d0e7a60c0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f8e70004410 tx=0x7f8e70010d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712c7a957997ac0c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:32.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.675+0000 7f8e5aff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:33:32.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.675+0000 7f8e5aff5640 1 -- 172.21.15.12:0/2050810381 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7001c8a0 con 0x561d0e7aca00 2024-04-22T13:33:32.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.675+0000 7f8e5aff5640 1 -- 172.21.15.12:0/2050810381 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f8e7001d900 con 0x561d0e7aca00 2024-04-22T13:33:32.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.675+0000 7f8e803b9740 1 -- 172.21.15.12:0/2050810381 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e6801e520 con 0x561d0e7aca00 2024-04-22T13:33:32.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.675+0000 7f8e5aff5640 1 -- 172.21.15.12:0/2050810381 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8e70040050 con 0x561d0e7aca00 2024-04-22T13:33:32.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.675+0000 7f8e803b9740 10 client.18867 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:32.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.675+0000 7f8e803b9740 1 -- 172.21.15.12:0/2050810381 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561d0e826c40 con 0x561d0e7aca00 2024-04-22T13:33:32.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.675+0000 7f8e5aff5640 1 -- 172.21.15.12:0/2050810381 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e6801e520 con 0x561d0e7aca00 2024-04-22T13:33:32.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.676+0000 7f8e5aff5640 1 -- 172.21.15.12:0/2050810381 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f8e70025070 con 0x561d0e7aca00 2024-04-22T13:33:32.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.676+0000 7f8e803b9740 10 client.18867 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:32.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.676+0000 7f8e803b9740 10 client.18867 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:33:32.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.676+0000 7f8e803b9740 20 client.18867 populate_metadata read hostname 'smithi012' 2024-04-22T13:33:32.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.676+0000 7f8e803b9740 1 --2- 172.21.15.12:0/2050810381 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x561d0e8273e0 0x561d0e829840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.676+0000 7f8e803b9740 4 client.18867 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:33:32.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.676+0000 7f8e803b9740 1 -- 172.21.15.12:0/2050810381 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561d0e715c10 con 0x561d0e8273e0 2024-04-22T13:33:32.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.676+0000 7f8e76ffd640 1 --2- 172.21.15.12:0/2050810381 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x561d0e8273e0 0x561d0e829840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.676+0000 7f8e76ffd640 1 --2- 172.21.15.12:0/2050810381 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x561d0e8273e0 0x561d0e829840 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1854010684cef9fc server_cookie=38b8a8e7e3d0488b in_seq=0 out_seq=0 2024-04-22T13:33:32.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.676+0000 7f8e5aff5640 10 client.18867 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:33:32.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.677+0000 7f8e5aff5640 1 -- 172.21.15.12:0/2050810381 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561d0e715c10 con 0x561d0e8273e0 2024-04-22T13:33:32.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.677+0000 7f8e5aff5640 10 client.18867 handle_command_reply: tid=0 2024-04-22T13:33:32.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.677+0000 7f8e803b9740 1 client.18867 shutdown 2024-04-22T13:33:32.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.677+0000 7f8e803b9740 1 -- 172.21.15.12:0/2050810381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d0e7aca00 msgr2=0x561d0e7a60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.677+0000 7f8e803b9740 1 --2- 172.21.15.12:0/2050810381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d0e7aca00 0x561d0e7a60c0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f8e70004410 tx=0x7f8e70010d40 comp rx=0 tx=0).stop 2024-04-22T13:33:32.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.677+0000 7f8e803b9740 1 -- 172.21.15.12:0/2050810381 shutdown_connections 2024-04-22T13:33:32.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.677+0000 7f8e803b9740 1 --2- 172.21.15.12:0/2050810381 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x561d0e8273e0 0x561d0e829840 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.677+0000 7f8e803b9740 1 --2- 172.21.15.12:0/2050810381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d0e7aca00 0x561d0e7a60c0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.677+0000 7f8e803b9740 1 --2- 172.21.15.12:0/2050810381 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561d0e7a3e80 0x561d0e7a5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.677+0000 7f8e803b9740 1 --2- 172.21.15.12:0/2050810381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561d0e7a34d0 0x561d0e7a7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.677+0000 7f8e5aff5640 0 client.18867 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:33:32.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.677+0000 7f8e803b9740 1 -- 172.21.15.12:0/2050810381 >> 172.21.15.12:0/2050810381 conn(0x561d0e79dba0 msgr2=0x561d0e7a4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:32.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.677+0000 7f8e803b9740 1 -- 172.21.15.12:0/2050810381 shutdown_connections 2024-04-22T13:33:32.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.677+0000 7f8e803b9740 1 -- 172.21.15.12:0/2050810381 wait complete. 2024-04-22T13:33:32.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.677+0000 7f8e803b9740 20 client.18867 trim_cache size 0 max 16384 2024-04-22T13:33:32.680 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:33:32.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.677+0000 7f8e7d7c7640 1 -- 172.21.15.12:0/3017253637 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f8e600a3d20 msgr2=0x7f8e600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.677+0000 7f8e7d7c7640 1 --2- 172.21.15.12:0/3017253637 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f8e600a3d20 0x7f8e600a61e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8e70005fd0 tx=0x7f8e70014040 comp rx=0 tx=0).stop 2024-04-22T13:33:32.681 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.677+0000 7f8e7d7c7640 1 -- 172.21.15.12:0/3017253637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e78107250 msgr2=0x7f8e7819e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.677+0000 7f8e7d7c7640 1 --2- 172.21.15.12:0/3017253637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e78107250 0x7f8e7819e930 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c036040 tx=0x7f8e6c0045e0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.677+0000 7f8e7d7c7640 1 -- 172.21.15.12:0/3017253637 shutdown_connections 2024-04-22T13:33:32.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.677+0000 7f8e7d7c7640 1 --2- 172.21.15.12:0/3017253637 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f8e600a3d20 0x7f8e600a61e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.677+0000 7f8e7d7c7640 1 --2- 172.21.15.12:0/3017253637 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e781985f0 0x7f8e78198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.677+0000 7f8e7d7c7640 1 --2- 172.21.15.12:0/3017253637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8e78107250 0x7f8e7819e930 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.677+0000 7f8e7d7c7640 1 --2- 172.21.15.12:0/3017253637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e78102180 0x7f8e7819e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.677+0000 7f8e7d7c7640 1 -- 172.21.15.12:0/3017253637 >> 172.21.15.12:0/3017253637 conn(0x7f8e780fd9c0 msgr2=0x7f8e780fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:32.682 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.678+0000 7f8e7d7c7640 1 -- 172.21.15.12:0/3017253637 shutdown_connections 2024-04-22T13:33:32.683 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.678+0000 7f8e7d7c7640 1 -- 172.21.15.12:0/3017253637 wait complete. 2024-04-22T13:33:32.684 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:33:32.790 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.792+0000 7f7fcc04a640 1 -- 172.21.15.12:0/1329770484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fc41031b0 msgr2=0x7f7fc4103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.792+0000 7f7fcc04a640 1 --2- 172.21.15.12:0/1329770484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fc41031b0 0x7f7fc4103590 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0009900 tx=0x7f7fc002bcd0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.793+0000 7f7fcc04a640 1 -- 172.21.15.12:0/1329770484 shutdown_connections 2024-04-22T13:33:32.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.793+0000 7f7fcc04a640 1 --2- 172.21.15.12:0/1329770484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7fc410c580 0x7f7fc410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.793+0000 7f7fcc04a640 1 --2- 172.21.15.12:0/1329770484 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fc4103ad0 0x7f7fc410bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.793+0000 7f7fcc04a640 1 --2- 172.21.15.12:0/1329770484 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fc41031b0 0x7f7fc4103590 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.793+0000 7f7fcc04a640 1 -- 172.21.15.12:0/1329770484 >> 172.21.15.12:0/1329770484 conn(0x7f7fc40771d0 msgr2=0x7f7fc40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:32.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.793+0000 7f7fcc04a640 1 -- 172.21.15.12:0/1329770484 shutdown_connections 2024-04-22T13:33:32.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.793+0000 7f7fcc04a640 1 -- 172.21.15.12:0/1329770484 wait complete. 2024-04-22T13:33:32.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.793+0000 7f7fcc04a640 1 Processor -- start 2024-04-22T13:33:32.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.793+0000 7f7fcc04a640 1 -- start start 2024-04-22T13:33:32.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.793+0000 7f7fcc04a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fc41031b0 0x7f7fc419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.791 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.793+0000 7f7fcc04a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7fc4103ad0 0x7f7fc419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.793+0000 7f7fcc04a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fc410c580 0x7f7fc41985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.794+0000 7f7fcc04a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc41121b0 con 0x7f7fc41031b0 2024-04-22T13:33:32.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.794+0000 7f7fcc04a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc4112030 con 0x7f7fc410c580 2024-04-22T13:33:32.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.794+0000 7f7fcc04a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc4112330 con 0x7f7fc4103ad0 2024-04-22T13:33:32.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.794+0000 7f7fc9de6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fc41031b0 0x7f7fc419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.794+0000 7f7fc9de6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fc41031b0 0x7f7fc419e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60652/0 (socket says 172.21.15.12:60652) 2024-04-22T13:33:32.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.794+0000 7f7fc9de6640 1 -- 172.21.15.12:0/1447802230 learned_addr learned my addr 172.21.15.12:0/1447802230 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:32.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.794+0000 7f7fc95e5640 1 --2- 172.21.15.12:0/1447802230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7fc4103ad0 0x7f7fc419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.794+0000 7f7fca5e7640 1 --2- 172.21.15.12:0/1447802230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fc410c580 0x7f7fc41985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.794+0000 7f7fc9de6640 1 -- 172.21.15.12:0/1447802230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7fc4103ad0 msgr2=0x7f7fc419e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.792 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.794+0000 7f7fc9de6640 1 --2- 172.21.15.12:0/1447802230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7fc4103ad0 0x7f7fc419e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.794+0000 7f7fc9de6640 1 -- 172.21.15.12:0/1447802230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fc410c580 msgr2=0x7f7fc41985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.794+0000 7f7fc9de6640 1 --2- 172.21.15.12:0/1447802230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fc410c580 0x7f7fc41985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.794+0000 7f7fc9de6640 1 -- 172.21.15.12:0/1447802230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc4198d20 con 0x7f7fc41031b0 2024-04-22T13:33:32.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.794+0000 7f7fc9de6640 1 --2- 172.21.15.12:0/1447802230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fc41031b0 0x7f7fc419e3f0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0005940 tx=0x7f7fc00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:32.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.794+0000 7f7fbaffd640 1 -- 172.21.15.12:0/1447802230 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0043020 con 0x7f7fc41031b0 2024-04-22T13:33:32.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.794+0000 7f7fbaffd640 1 -- 172.21.15.12:0/1447802230 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fc002dba0 con 0x7f7fc41031b0 2024-04-22T13:33:32.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.794+0000 7f7fcc04a640 1 -- 172.21.15.12:0/1447802230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fc4198fb0 con 0x7f7fc41031b0 2024-04-22T13:33:32.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.794+0000 7f7fbaffd640 1 -- 172.21.15.12:0/1447802230 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0003930 con 0x7f7fc41031b0 2024-04-22T13:33:32.793 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.794+0000 7f7fcc04a640 1 -- 172.21.15.12:0/1447802230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fc41a52b0 con 0x7f7fc41031b0 2024-04-22T13:33:32.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.797+0000 7f7fbaffd640 1 -- 172.21.15.12:0/1447802230 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7fc0046020 con 0x7f7fc41031b0 2024-04-22T13:33:32.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.797+0000 7f7fcec3f740 1 Processor -- start 2024-04-22T13:33:32.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.797+0000 7f7fcec3f740 1 -- start start 2024-04-22T13:33:32.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.797+0000 7f7fcec3f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644ae6aa4d0 0x5644ae6aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.800 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.797+0000 7f7fcec3f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644ae6aae80 0x5644ae6b3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.797+0000 7f7fcec3f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644ae6b3a00 0x5644ae6b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.798+0000 7f7fcec3f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5644ae60f1f0 con 0x5644ae6aae80 2024-04-22T13:33:32.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.798+0000 7f7fcec3f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5644ae5ac4b0 con 0x5644ae6b3a00 2024-04-22T13:33:32.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.798+0000 7f7fcec3f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5644ae60b1b0 con 0x5644ae6aa4d0 2024-04-22T13:33:32.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.798+0000 7f7fc9de6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644ae6aae80 0x5644ae6b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.798+0000 7f7fc9de6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644ae6aae80 0x5644ae6b3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60666/0 (socket says 172.21.15.12:60666) 2024-04-22T13:33:32.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.798+0000 7f7fc9de6640 1 -- 172.21.15.12:0/2461962855 learned_addr learned my addr 172.21.15.12:0/2461962855 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:32.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.798+0000 7f7fbaffd640 1 --2- 172.21.15.12:0/1447802230 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7fb00a3d20 0x7f7fb00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.801 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.798+0000 7f7fca5e7640 1 --2- 172.21.15.12:0/2461962855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644ae6aa4d0 0x5644ae6aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.798+0000 7f7fc95e5640 1 --2- 172.21.15.12:0/2461962855 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644ae6b3a00 0x5644ae6b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.798+0000 7f7fca5e7640 1 --2- 172.21.15.12:0/1447802230 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7fb00a3d20 0x7f7fb00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.798+0000 7f7fbaffd640 1 -- 172.21.15.12:0/1447802230 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f7fc00f2780 con 0x7f7fc41031b0 2024-04-22T13:33:32.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.798+0000 7f7fc9de6640 1 -- 172.21.15.12:0/2461962855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644ae6aa4d0 msgr2=0x5644ae6aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.798+0000 7f7fc9de6640 1 --2- 172.21.15.12:0/2461962855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644ae6aa4d0 0x5644ae6aa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.798+0000 7f7fc9de6640 1 -- 172.21.15.12:0/2461962855 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644ae6b3a00 msgr2=0x5644ae6b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.798+0000 7f7fc9de6640 1 --2- 172.21.15.12:0/2461962855 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644ae6b3a00 0x5644ae6b5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.798+0000 7f7fc9de6640 1 -- 172.21.15.12:0/2461962855 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644ae59e1b0 con 0x5644ae6aae80 2024-04-22T13:33:32.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.798+0000 7f7fc9de6640 1 --2- 172.21.15.12:0/2461962855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644ae6aae80 0x5644ae6b3430 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00be550 tx=0x7f7fc002bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13bd08357015a18 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:32.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.798+0000 7f7faffff640 1 -- 172.21.15.12:0/2461962855 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc00bef50 con 0x5644ae6aae80 2024-04-22T13:33:32.802 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.798+0000 7f7fca5e7640 1 --2- 172.21.15.12:0/1447802230 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7fb00a3d20 0x7f7fb00a61e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009370 tx=0x7f7fbc009240 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:32.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.798+0000 7f7faffff640 1 -- 172.21.15.12:0/2461962855 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fc00be850 con 0x5644ae6aae80 2024-04-22T13:33:32.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.798+0000 7f7faffff640 1 -- 172.21.15.12:0/2461962855 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc00baec0 con 0x5644ae6aae80 2024-04-22T13:33:32.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.799+0000 7f7fcec3f740 1 -- 172.21.15.12:0/2461962855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644ae6aae80 msgr2=0x5644ae6b3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.799+0000 7f7fcec3f740 1 --2- 172.21.15.12:0/2461962855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644ae6aae80 0x5644ae6b3430 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00be550 tx=0x7f7fc002bcd0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.799+0000 7f7fcec3f740 1 -- 172.21.15.12:0/2461962855 shutdown_connections 2024-04-22T13:33:32.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.799+0000 7f7fcec3f740 1 --2- 172.21.15.12:0/2461962855 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644ae6b3a00 0x5644ae6b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.799+0000 7f7fcec3f740 1 --2- 172.21.15.12:0/2461962855 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644ae6aae80 0x5644ae6b3430 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.799+0000 7f7fcec3f740 1 --2- 172.21.15.12:0/2461962855 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644ae6aa4d0 0x5644ae6aa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.799+0000 7f7fcec3f740 1 -- 172.21.15.12:0/2461962855 >> 172.21.15.12:0/2461962855 conn(0x5644ae6a4ba0 msgr2=0x5644ae6a4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:32.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.799+0000 7f7fcec3f740 1 -- 172.21.15.12:0/2461962855 shutdown_connections 2024-04-22T13:33:32.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.799+0000 7f7fcec3f740 1 -- 172.21.15.12:0/2461962855 wait complete. 2024-04-22T13:33:32.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.800+0000 7f7fcec3f740 1 -- start start 2024-04-22T13:33:32.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.800+0000 7f7fcec3f740 1 Processor -- start 2024-04-22T13:33:32.803 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.800+0000 7f7fcec3f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644ae6aa4d0 0x5644ae6b3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.800+0000 7f7fcec3f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644ae6aae80 0x5644ae6b0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.800+0000 7f7fcec3f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644ae6b3a00 0x5644ae6b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.800+0000 7f7fcec3f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5644ae6b6730 con 0x5644ae6aae80 2024-04-22T13:33:32.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.800+0000 7f7fcec3f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5644ae5c8500 con 0x5644ae6aa4d0 2024-04-22T13:33:32.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.800+0000 7f7fcec3f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5644ae6b68b0 con 0x5644ae6b3a00 2024-04-22T13:33:32.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.800+0000 7f7fcec3f740 10 client.0 _ll_register_callbacks cb 0x5644ae596b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:32.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.800+0000 7f7fc95e5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644ae6aae80 0x5644ae6b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.800+0000 7f7fc95e5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644ae6aae80 0x5644ae6b0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60668/0 (socket says 172.21.15.12:60668) 2024-04-22T13:33:32.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.800+0000 7f7fc95e5640 1 -- 172.21.15.12:0/1516132824 learned_addr learned my addr 172.21.15.12:0/1516132824 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:32.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.800+0000 7f7fc9de6640 1 --2- 172.21.15.12:0/1516132824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644ae6aa4d0 0x5644ae6b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.801+0000 7f7fca5e7640 1 --2- 172.21.15.12:0/1516132824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644ae6b3a00 0x5644ae6b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.804 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.801+0000 7f7fc95e5640 1 -- 172.21.15.12:0/1516132824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644ae6b3a00 msgr2=0x5644ae6b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.801+0000 7f7fc95e5640 1 --2- 172.21.15.12:0/1516132824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644ae6b3a00 0x5644ae6b0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.801+0000 7f7fc95e5640 1 -- 172.21.15.12:0/1516132824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644ae6aa4d0 msgr2=0x5644ae6b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.801+0000 7f7fc95e5640 1 --2- 172.21.15.12:0/1516132824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644ae6aa4d0 0x5644ae6b3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.801+0000 7f7fc95e5640 1 -- 172.21.15.12:0/1516132824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644ae3213f0 con 0x5644ae6aae80 2024-04-22T13:33:32.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.801+0000 7f7fca5e7640 1 --2- 172.21.15.12:0/1516132824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644ae6b3a00 0x5644ae6b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:32.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.801+0000 7f7fc9de6640 1 --2- 172.21.15.12:0/1516132824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644ae6aa4d0 0x5644ae6b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:32.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.801+0000 7f7fc95e5640 1 --2- 172.21.15.12:0/1516132824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644ae6aae80 0x5644ae6b0610 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4012700 tx=0x7f7fb4012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f633c0d5a6a6cce4 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:32.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.801+0000 7f7fad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:33:32.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.801+0000 7f7fad7fa640 1 -- 172.21.15.12:0/1516132824 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4004050 con 0x5644ae6aae80 2024-04-22T13:33:32.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.801+0000 7f7fad7fa640 1 -- 172.21.15.12:0/1516132824 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f7fb40041f0 con 0x5644ae6aae80 2024-04-22T13:33:32.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.801+0000 7f7fcec3f740 1 -- 172.21.15.12:0/1516132824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fc00be850 con 0x5644ae6aae80 2024-04-22T13:33:32.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.801+0000 7f7fad7fa640 1 -- 172.21.15.12:0/1516132824 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7fb40043f0 con 0x5644ae6aae80 2024-04-22T13:33:32.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.801+0000 7f7fcec3f740 10 client.18879 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:32.805 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.801+0000 7f7fcec3f740 1 -- 172.21.15.12:0/1516132824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5644ae3eb7d0 con 0x5644ae6aae80 2024-04-22T13:33:32.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.801+0000 7f7fad7fa640 1 -- 172.21.15.12:0/1516132824 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fc00be850 con 0x5644ae6aae80 2024-04-22T13:33:32.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.801+0000 7f7fad7fa640 1 -- 172.21.15.12:0/1516132824 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7fb4003b20 con 0x5644ae6aae80 2024-04-22T13:33:32.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.802+0000 7f7fcec3f740 10 client.18879 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:32.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.802+0000 7f7fcec3f740 10 client.18879 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:33:32.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.802+0000 7f7fcec3f740 20 client.18879 populate_metadata read hostname 'smithi012' 2024-04-22T13:33:32.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.802+0000 7f7fcec3f740 1 --2- 172.21.15.12:0/1516132824 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5644ae6c3920 0x5644ae6c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.802+0000 7f7fcec3f740 4 client.18879 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:33:32.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.802+0000 7f7fcec3f740 1 -- 172.21.15.12:0/1516132824 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644ae5ae930 con 0x5644ae6c3920 2024-04-22T13:33:32.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.802+0000 7f7fc9de6640 1 --2- 172.21.15.12:0/1516132824 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5644ae6c3920 0x5644ae6c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.802+0000 7f7fc9de6640 1 --2- 172.21.15.12:0/1516132824 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5644ae6c3920 0x5644ae6c5da0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d405d5c7c6188dc9 server_cookie=c04f73883f195310 in_seq=0 out_seq=0 2024-04-22T13:33:32.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.802+0000 7f7fad7fa640 10 client.18879 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:33:32.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.803+0000 7f7fad7fa640 1 -- 172.21.15.12:0/1516132824 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5644ae5ae930 con 0x5644ae6c3920 2024-04-22T13:33:32.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.803+0000 7f7fad7fa640 10 client.18879 handle_command_reply: tid=0 2024-04-22T13:33:32.806 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.803+0000 7f7fcec3f740 1 client.18879 shutdown 2024-04-22T13:33:32.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.803+0000 7f7fcec3f740 1 -- 172.21.15.12:0/1516132824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644ae6aae80 msgr2=0x5644ae6b0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.803+0000 7f7fcec3f740 1 --2- 172.21.15.12:0/1516132824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644ae6aae80 0x5644ae6b0610 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4012700 tx=0x7f7fb4012bc0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.803+0000 7f7fcec3f740 1 -- 172.21.15.12:0/1516132824 shutdown_connections 2024-04-22T13:33:32.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.803+0000 7f7fcec3f740 1 --2- 172.21.15.12:0/1516132824 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5644ae6c3920 0x5644ae6c5da0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.803+0000 7f7fcec3f740 1 --2- 172.21.15.12:0/1516132824 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644ae6b3a00 0x5644ae6b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.803+0000 7f7fcec3f740 1 --2- 172.21.15.12:0/1516132824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644ae6aae80 0x5644ae6b0610 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.803+0000 7f7fcec3f740 1 --2- 172.21.15.12:0/1516132824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644ae6aa4d0 0x5644ae6b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.803+0000 7f7fad7fa640 0 client.18879 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:33:32.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.803+0000 7f7fcec3f740 1 -- 172.21.15.12:0/1516132824 >> 172.21.15.12:0/1516132824 conn(0x5644ae6a4ba0 msgr2=0x5644ae6a68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:32.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.803+0000 7f7fcec3f740 1 -- 172.21.15.12:0/1516132824 shutdown_connections 2024-04-22T13:33:32.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.803+0000 7f7fcec3f740 1 -- 172.21.15.12:0/1516132824 wait complete. 2024-04-22T13:33:32.807 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.803+0000 7f7fcec3f740 20 client.18879 trim_cache size 0 max 16384 2024-04-22T13:33:32.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.817+0000 7f7fcec3f740 1 Processor -- start 2024-04-22T13:33:32.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.817+0000 7f7fcec3f740 1 -- start start 2024-04-22T13:33:32.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.817+0000 7f7fcec3f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644ae6aa4d0 0x5644ae6bf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.817+0000 7f7fcec3f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644ae6aae80 0x5644ae6af770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.817+0000 7f7fcec3f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644ae6b3a00 0x5644ae6abae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.817+0000 7f7fcec3f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc41adb80 con 0x5644ae6b3a00 2024-04-22T13:33:32.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.817+0000 7f7fcec3f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc41af6c0 con 0x5644ae6aa4d0 2024-04-22T13:33:32.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.817+0000 7f7fcec3f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc41ae250 con 0x5644ae6aae80 2024-04-22T13:33:32.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.817+0000 7f7fca5e7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644ae6b3a00 0x5644ae6abae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.817+0000 7f7fca5e7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644ae6b3a00 0x5644ae6abae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60684/0 (socket says 172.21.15.12:60684) 2024-04-22T13:33:32.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.817+0000 7f7fca5e7640 1 -- 172.21.15.12:0/934914902 learned_addr learned my addr 172.21.15.12:0/934914902 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:32.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.817+0000 7f7fc9de6640 1 --2- 172.21.15.12:0/934914902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644ae6aa4d0 0x5644ae6bf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.817+0000 7f7fc95e5640 1 --2- 172.21.15.12:0/934914902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644ae6aae80 0x5644ae6af770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.817+0000 7f7fca5e7640 1 -- 172.21.15.12:0/934914902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644ae6aae80 msgr2=0x5644ae6af770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.817+0000 7f7fca5e7640 1 --2- 172.21.15.12:0/934914902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644ae6aae80 0x5644ae6af770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.817+0000 7f7fca5e7640 1 -- 172.21.15.12:0/934914902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644ae6aa4d0 msgr2=0x5644ae6bf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.817+0000 7f7fca5e7640 1 --2- 172.21.15.12:0/934914902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644ae6aa4d0 0x5644ae6bf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.817+0000 7f7fca5e7640 1 -- 172.21.15.12:0/934914902 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644ae6a4380 con 0x5644ae6b3a00 2024-04-22T13:33:32.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.818+0000 7f7fca5e7640 1 --2- 172.21.15.12:0/934914902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644ae6b3a00 0x5644ae6abae0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009df0 tx=0x7f7fbc0096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c485b020b7c8e4f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:32.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.818+0000 7f7faf7fe640 1 -- 172.21.15.12:0/934914902 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc017070 con 0x5644ae6b3a00 2024-04-22T13:33:32.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.818+0000 7f7faf7fe640 1 -- 172.21.15.12:0/934914902 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fbc018040 con 0x5644ae6b3a00 2024-04-22T13:33:32.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.818+0000 7f7faf7fe640 1 -- 172.21.15.12:0/934914902 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc020480 con 0x5644ae6b3a00 2024-04-22T13:33:32.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.818+0000 7f7fcec3f740 1 -- 172.21.15.12:0/934914902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644ae6b3a00 msgr2=0x5644ae6abae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.818+0000 7f7fcec3f740 1 --2- 172.21.15.12:0/934914902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644ae6b3a00 0x5644ae6abae0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009df0 tx=0x7f7fbc0096d0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.818+0000 7f7fcec3f740 1 -- 172.21.15.12:0/934914902 shutdown_connections 2024-04-22T13:33:32.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.818+0000 7f7fcec3f740 1 --2- 172.21.15.12:0/934914902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644ae6b3a00 0x5644ae6abae0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.818+0000 7f7fcec3f740 1 --2- 172.21.15.12:0/934914902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644ae6aae80 0x5644ae6af770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.818+0000 7f7fcec3f740 1 --2- 172.21.15.12:0/934914902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644ae6aa4d0 0x5644ae6bf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.818+0000 7f7fcec3f740 1 -- 172.21.15.12:0/934914902 >> 172.21.15.12:0/934914902 conn(0x5644ae6a4ba0 msgr2=0x5644ae6ab3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:32.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.818+0000 7f7fcec3f740 1 -- 172.21.15.12:0/934914902 shutdown_connections 2024-04-22T13:33:32.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.818+0000 7f7fcec3f740 1 -- 172.21.15.12:0/934914902 wait complete. 2024-04-22T13:33:32.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.819+0000 7f7fcec3f740 1 -- start start 2024-04-22T13:33:32.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.819+0000 7f7fcec3f740 1 Processor -- start 2024-04-22T13:33:32.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.819+0000 7f7fcec3f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644ae6aa4d0 0x5644ae6bbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.819+0000 7f7fcec3f740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644ae6aae80 0x5644ae6acf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.819+0000 7f7fcec3f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644ae6b3a00 0x5644ae6ad4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.819+0000 7f7fcec3f740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc41aeff0 con 0x5644ae6aae80 2024-04-22T13:33:32.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.819+0000 7f7fcec3f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc41ae920 con 0x5644ae6b3a00 2024-04-22T13:33:32.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.819+0000 7f7fcec3f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc41abb80 con 0x5644ae6aa4d0 2024-04-22T13:33:32.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.819+0000 7f7fcec3f740 10 client.0 _ll_register_callbacks cb 0x7f7fb4009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:32.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.819+0000 7f7fc95e5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644ae6aae80 0x5644ae6acf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.819+0000 7f7fc95e5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644ae6aae80 0x5644ae6acf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60694/0 (socket says 172.21.15.12:60694) 2024-04-22T13:33:32.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.819+0000 7f7fc95e5640 1 -- 172.21.15.12:0/2565372574 learned_addr learned my addr 172.21.15.12:0/2565372574 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:32.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.819+0000 7f7fca5e7640 1 --2- 172.21.15.12:0/2565372574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644ae6b3a00 0x5644ae6ad4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.819+0000 7f7fc9de6640 1 --2- 172.21.15.12:0/2565372574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644ae6aa4d0 0x5644ae6bbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.819+0000 7f7fc95e5640 1 -- 172.21.15.12:0/2565372574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644ae6aa4d0 msgr2=0x5644ae6bbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.819+0000 7f7fc95e5640 1 --2- 172.21.15.12:0/2565372574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644ae6aa4d0 0x5644ae6bbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.819+0000 7f7fc95e5640 1 -- 172.21.15.12:0/2565372574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644ae6b3a00 msgr2=0x5644ae6ad4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.819+0000 7f7fc95e5640 1 --2- 172.21.15.12:0/2565372574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644ae6b3a00 0x5644ae6ad4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.820+0000 7f7fc95e5640 1 -- 172.21.15.12:0/2565372574 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644ae6adb60 con 0x5644ae6aae80 2024-04-22T13:33:32.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.820+0000 7f7fc9de6640 1 --2- 172.21.15.12:0/2565372574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644ae6aa4d0 0x5644ae6bbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:32.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.820+0000 7f7fc95e5640 1 --2- 172.21.15.12:0/2565372574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644ae6aae80 0x5644ae6acf60 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x5644ae6b1be0 tx=0x7f7fb40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b3aa815e31f723 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:32.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.820+0000 7f7fad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:33:32.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.820+0000 7f7fad7fa640 1 -- 172.21.15.12:0/2565372574 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4029040 con 0x5644ae6aae80 2024-04-22T13:33:32.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.820+0000 7f7fad7fa640 1 -- 172.21.15.12:0/2565372574 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f7fb400f9e0 con 0x5644ae6aae80 2024-04-22T13:33:32.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.820+0000 7f7fcec3f740 1 -- 172.21.15.12:0/2565372574 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fbc018040 con 0x5644ae6aae80 2024-04-22T13:33:32.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.820+0000 7f7fad7fa640 1 -- 172.21.15.12:0/2565372574 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7fb400fbe0 con 0x5644ae6aae80 2024-04-22T13:33:32.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.820+0000 7f7fcec3f740 10 client.18885 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:32.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.820+0000 7f7fcec3f740 1 -- 172.21.15.12:0/2565372574 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5644ae729e00 con 0x5644ae6aae80 2024-04-22T13:33:32.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.820+0000 7f7fad7fa640 1 -- 172.21.15.12:0/2565372574 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fbc018040 con 0x5644ae6aae80 2024-04-22T13:33:32.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.820+0000 7f7fad7fa640 1 -- 172.21.15.12:0/2565372574 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7fb402c070 con 0x5644ae6aae80 2024-04-22T13:33:32.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.820+0000 7f7fcec3f740 10 client.18885 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:32.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.820+0000 7f7fcec3f740 10 client.18885 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:33:32.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.820+0000 7f7fcec3f740 20 client.18885 populate_metadata read hostname 'smithi012' 2024-04-22T13:33:32.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.820+0000 7f7fcec3f740 1 --2- 172.21.15.12:0/2565372574 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5644ae72a600 0x5644ae72ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.821+0000 7f7fcec3f740 4 client.18885 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:33:32.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.821+0000 7f7fcec3f740 1 -- 172.21.15.12:0/2565372574 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5644ae61cc10 con 0x5644ae72a600 2024-04-22T13:33:32.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.821+0000 7f7fc9de6640 1 --2- 172.21.15.12:0/2565372574 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5644ae72a600 0x5644ae72ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.821+0000 7f7fad7fa640 10 client.18885 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:33:32.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.821+0000 7f7fc9de6640 1 --2- 172.21.15.12:0/2565372574 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5644ae72a600 0x5644ae72ca20 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f9da3b90e5943d56 server_cookie=30163a7646583e61 in_seq=0 out_seq=0 2024-04-22T13:33:32.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.821+0000 7f7fad7fa640 1 -- 172.21.15.12:0/2565372574 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5644ae61cc10 con 0x5644ae72a600 2024-04-22T13:33:32.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.821+0000 7f7fad7fa640 10 client.18885 handle_command_reply: tid=0 2024-04-22T13:33:32.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.821+0000 7f7fcec3f740 1 client.18885 shutdown 2024-04-22T13:33:32.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.822+0000 7f7fcec3f740 1 -- 172.21.15.12:0/2565372574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644ae6aae80 msgr2=0x5644ae6acf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.822+0000 7f7fcec3f740 1 --2- 172.21.15.12:0/2565372574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644ae6aae80 0x5644ae6acf60 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x5644ae6b1be0 tx=0x7f7fb40042f0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.825 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.822+0000 7f7fcec3f740 1 -- 172.21.15.12:0/2565372574 shutdown_connections 2024-04-22T13:33:32.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.822+0000 7f7fcec3f740 1 --2- 172.21.15.12:0/2565372574 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x5644ae72a600 0x5644ae72ca20 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.822+0000 7f7fcec3f740 1 --2- 172.21.15.12:0/2565372574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5644ae6b3a00 0x5644ae6ad4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.822+0000 7f7fad7fa640 0 client.18885 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:33:32.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.822+0000 7f7fcec3f740 1 --2- 172.21.15.12:0/2565372574 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5644ae6aae80 0x5644ae6acf60 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.822+0000 7f7fcec3f740 1 --2- 172.21.15.12:0/2565372574 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5644ae6aa4d0 0x5644ae6bbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.822+0000 7f7fcec3f740 1 -- 172.21.15.12:0/2565372574 >> 172.21.15.12:0/2565372574 conn(0x5644ae6a4ba0 msgr2=0x5644ae6a3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:32.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.822+0000 7f7fcec3f740 1 -- 172.21.15.12:0/2565372574 shutdown_connections 2024-04-22T13:33:32.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.822+0000 7f7fcec3f740 1 -- 172.21.15.12:0/2565372574 wait complete. 2024-04-22T13:33:32.826 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.822+0000 7f7fcec3f740 20 client.18885 trim_cache size 0 max 16384 2024-04-22T13:33:32.826 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:33:32.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.822+0000 7f7fcc04a640 1 -- 172.21.15.12:0/1447802230 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7fb00a3d20 msgr2=0x7f7fb00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.827 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.822+0000 7f7fcc04a640 1 --2- 172.21.15.12:0/1447802230 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7fb00a3d20 0x7f7fb00a61e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009370 tx=0x7f7fbc009240 comp rx=0 tx=0).stop 2024-04-22T13:33:32.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.822+0000 7f7fcc04a640 1 -- 172.21.15.12:0/1447802230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fc41031b0 msgr2=0x7f7fc419e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.822+0000 7f7fcc04a640 1 --2- 172.21.15.12:0/1447802230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fc41031b0 0x7f7fc419e3f0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0005940 tx=0x7f7fc00045e0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.822+0000 7f7fcc04a640 1 -- 172.21.15.12:0/1447802230 shutdown_connections 2024-04-22T13:33:32.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.822+0000 7f7fcc04a640 1 --2- 172.21.15.12:0/1447802230 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7fb00a3d20 0x7f7fb00a61e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.822+0000 7f7fcc04a640 1 --2- 172.21.15.12:0/1447802230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fc410c580 0x7f7fc41985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.822+0000 7f7fcc04a640 1 --2- 172.21.15.12:0/1447802230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7fc4103ad0 0x7f7fc419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.822+0000 7f7fcc04a640 1 --2- 172.21.15.12:0/1447802230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7fc41031b0 0x7f7fc419e3f0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.822+0000 7f7fcc04a640 1 -- 172.21.15.12:0/1447802230 >> 172.21.15.12:0/1447802230 conn(0x7f7fc40771d0 msgr2=0x7f7fc40fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:32.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.822+0000 7f7fcc04a640 1 -- 172.21.15.12:0/1447802230 shutdown_connections 2024-04-22T13:33:32.828 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.822+0000 7f7fcc04a640 1 -- 172.21.15.12:0/1447802230 wait complete. 2024-04-22T13:33:32.829 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:32 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1268139744' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:33:32.829 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:33:32.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:32 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1268139744' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:33:32.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.939+0000 7fda0e7f4640 1 -- 172.21.15.12:0/3321311730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda08101090 msgr2=0x7fda08101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.934 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.939+0000 7fda0e7f4640 1 --2- 172.21.15.12:0/3321311730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda08101090 0x7fda08101470 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009900 tx=0x7fd9f802bc30 comp rx=0 tx=0).stop 2024-04-22T13:33:32.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.939+0000 7fda0e7f4640 1 -- 172.21.15.12:0/3321311730 shutdown_connections 2024-04-22T13:33:32.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.939+0000 7fda0e7f4640 1 --2- 172.21.15.12:0/3321311730 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda08106160 0x7fda0810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.939+0000 7fda0e7f4640 1 --2- 172.21.15.12:0/3321311730 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda08101a40 0x7fda08105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.939+0000 7fda0e7f4640 1 --2- 172.21.15.12:0/3321311730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda08101090 0x7fda08101470 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.939+0000 7fda0e7f4640 1 -- 172.21.15.12:0/3321311730 >> 172.21.15.12:0/3321311730 conn(0x7fda08077f50 msgr2=0x7fda08078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:32.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.939+0000 7fda0e7f4640 1 -- 172.21.15.12:0/3321311730 shutdown_connections 2024-04-22T13:33:32.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.939+0000 7fda0e7f4640 1 -- 172.21.15.12:0/3321311730 wait complete. 2024-04-22T13:33:32.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.939+0000 7fda0e7f4640 1 Processor -- start 2024-04-22T13:33:32.935 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.939+0000 7fda0e7f4640 1 -- start start 2024-04-22T13:33:32.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.940+0000 7fda0e7f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda08101090 0x7fda08102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.940+0000 7fda0e7f4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda08106160 0x7fda08103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.940+0000 7fda0e7f4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda081035a0 0x7fda081a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.937 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.940+0000 7fda0e7f4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fda081110a0 con 0x7fda08106160 2024-04-22T13:33:32.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.940+0000 7fda0e7f4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fda08110f20 con 0x7fda081035a0 2024-04-22T13:33:32.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.940+0000 7fda0e7f4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fda08111220 con 0x7fda08101090 2024-04-22T13:33:32.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.940+0000 7fda077fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda08106160 0x7fda08103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.940+0000 7fda077fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda08106160 0x7fda08103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60714/0 (socket says 172.21.15.12:60714) 2024-04-22T13:33:32.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.940+0000 7fda077fe640 1 -- 172.21.15.12:0/3366300782 learned_addr learned my addr 172.21.15.12:0/3366300782 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:32.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.940+0000 7fda0cd91640 1 --2- 172.21.15.12:0/3366300782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda081035a0 0x7fda081a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.940+0000 7fda07fff640 1 --2- 172.21.15.12:0/3366300782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda08101090 0x7fda08102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.940+0000 7fda077fe640 1 -- 172.21.15.12:0/3366300782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda08101090 msgr2=0x7fda08102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.940+0000 7fda077fe640 1 --2- 172.21.15.12:0/3366300782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda08101090 0x7fda08102b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.940+0000 7fda077fe640 1 -- 172.21.15.12:0/3366300782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda081035a0 msgr2=0x7fda081a6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.940+0000 7fda077fe640 1 --2- 172.21.15.12:0/3366300782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda081035a0 0x7fda081a6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.940+0000 7fda077fe640 1 -- 172.21.15.12:0/3366300782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda081a7020 con 0x7fda08106160 2024-04-22T13:33:32.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.940+0000 7fda077fe640 1 --2- 172.21.15.12:0/3366300782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda08106160 0x7fda08103060 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00e9c0 tx=0x7fd9fc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:32.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.941+0000 7fda057fa640 1 -- 172.21.15.12:0/3366300782 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc00cd80 con 0x7fda08106160 2024-04-22T13:33:32.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.941+0000 7fda0e7f4640 1 -- 172.21.15.12:0/3366300782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda081a7310 con 0x7fda08106160 2024-04-22T13:33:32.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.941+0000 7fda057fa640 1 -- 172.21.15.12:0/3366300782 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9fc0054d0 con 0x7fda08106160 2024-04-22T13:33:32.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.941+0000 7fda057fa640 1 -- 172.21.15.12:0/3366300782 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc010430 con 0x7fda08106160 2024-04-22T13:33:32.939 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.941+0000 7fda0e7f4640 1 -- 172.21.15.12:0/3366300782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda081a78b0 con 0x7fda08106160 2024-04-22T13:33:32.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.943+0000 7fda113e9740 1 Processor -- start 2024-04-22T13:33:32.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.943+0000 7fda113e9740 1 -- start start 2024-04-22T13:33:32.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.943+0000 7fda057fa640 1 -- 172.21.15.12:0/3366300782 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd9fc004f90 con 0x7fda08106160 2024-04-22T13:33:32.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.943+0000 7fda113e9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611ca5ba4d0 0x5611ca5ba8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.943+0000 7fda113e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611ca5bae80 0x5611ca5c3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.943+0000 7fda113e9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5611ca5c3a00 0x5611ca5c5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.943+0000 7fda113e9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5611ca51f1f0 con 0x5611ca5ba4d0 2024-04-22T13:33:32.946 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.943+0000 7fda113e9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5611ca4bc4b0 con 0x5611ca5c3a00 2024-04-22T13:33:32.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.943+0000 7fda113e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5611ca51b1b0 con 0x5611ca5bae80 2024-04-22T13:33:32.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.943+0000 7fda0cd91640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611ca5ba4d0 0x5611ca5ba8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.943+0000 7fda0cd91640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611ca5ba4d0 0x5611ca5ba8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60718/0 (socket says 172.21.15.12:60718) 2024-04-22T13:33:32.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.943+0000 7fda0cd91640 1 -- 172.21.15.12:0/3806103233 learned_addr learned my addr 172.21.15.12:0/3806103233 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:32.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.944+0000 7fda07fff640 1 --2- 172.21.15.12:0/3806103233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611ca5bae80 0x5611ca5c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.944+0000 7fda0cd91640 1 -- 172.21.15.12:0/3806103233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611ca5bae80 msgr2=0x5611ca5c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.944+0000 7fda077fe640 1 --2- 172.21.15.12:0/3806103233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5611ca5c3a00 0x5611ca5c5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.944+0000 7fda0cd91640 1 --2- 172.21.15.12:0/3806103233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611ca5bae80 0x5611ca5c3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.944+0000 7fda0cd91640 1 -- 172.21.15.12:0/3806103233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5611ca5c3a00 msgr2=0x5611ca5c5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.944+0000 7fda0cd91640 1 --2- 172.21.15.12:0/3806103233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5611ca5c3a00 0x5611ca5c5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.944+0000 7fda0cd91640 1 -- 172.21.15.12:0/3806103233 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611ca4ae1b0 con 0x5611ca5ba4d0 2024-04-22T13:33:32.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.944+0000 7fda077fe640 1 --2- 172.21.15.12:0/3806103233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5611ca5c3a00 0x5611ca5c5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:32.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.944+0000 7fda057fa640 1 --2- 172.21.15.12:0/3366300782 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd9f00a3d50 0x7fd9f00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.944+0000 7fda07fff640 1 --2- 172.21.15.12:0/3366300782 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd9f00a3d50 0x7fd9f00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.947 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.944+0000 7fda0cd91640 1 --2- 172.21.15.12:0/3806103233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611ca5ba4d0 0x5611ca5ba8b0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fda00009f90 tx=0x7fda00010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3bffc91b3d22f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:32.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.944+0000 7fda057fa640 1 -- 172.21.15.12:0/3366300782 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7fd9fc0d0b70 con 0x7fda08106160 2024-04-22T13:33:32.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.944+0000 7fd9ee7fc640 1 -- 172.21.15.12:0/3806103233 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00011700 con 0x5611ca5ba4d0 2024-04-22T13:33:32.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.944+0000 7fd9ee7fc640 1 -- 172.21.15.12:0/3806103233 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda00011d50 con 0x5611ca5ba4d0 2024-04-22T13:33:32.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.944+0000 7fd9ee7fc640 1 -- 172.21.15.12:0/3806103233 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00019d40 con 0x5611ca5ba4d0 2024-04-22T13:33:32.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.945+0000 7fda113e9740 1 -- 172.21.15.12:0/3806103233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611ca5ba4d0 msgr2=0x5611ca5ba8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.948 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.945+0000 7fda07fff640 1 --2- 172.21.15.12:0/3366300782 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd9f00a3d50 0x7fd9f00a6210 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8006010 tx=0x7fd9f8005e40 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:32.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.945+0000 7fda113e9740 1 --2- 172.21.15.12:0/3806103233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611ca5ba4d0 0x5611ca5ba8b0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fda00009f90 tx=0x7fda00010860 comp rx=0 tx=0).stop 2024-04-22T13:33:32.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.945+0000 7fda113e9740 1 -- 172.21.15.12:0/3806103233 shutdown_connections 2024-04-22T13:33:32.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.945+0000 7fda113e9740 1 --2- 172.21.15.12:0/3806103233 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5611ca5c3a00 0x5611ca5c5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.945+0000 7fda113e9740 1 --2- 172.21.15.12:0/3806103233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611ca5bae80 0x5611ca5c3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.945+0000 7fda113e9740 1 --2- 172.21.15.12:0/3806103233 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611ca5ba4d0 0x5611ca5ba8b0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.945+0000 7fda113e9740 1 -- 172.21.15.12:0/3806103233 >> 172.21.15.12:0/3806103233 conn(0x5611ca5b4ba0 msgr2=0x5611ca5b4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:32.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.945+0000 7fda113e9740 1 -- 172.21.15.12:0/3806103233 shutdown_connections 2024-04-22T13:33:32.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.945+0000 7fda113e9740 1 -- 172.21.15.12:0/3806103233 wait complete. 2024-04-22T13:33:32.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.945+0000 7fda113e9740 1 -- start start 2024-04-22T13:33:32.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.945+0000 7fda113e9740 1 Processor -- start 2024-04-22T13:33:32.949 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.946+0000 7fda113e9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5611ca5ba4d0 0x5611ca5c3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.946+0000 7fda113e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611ca5bae80 0x5611ca5c0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.946+0000 7fda113e9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611ca5c3a00 0x5611ca5c0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.946+0000 7fda113e9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5611ca5c6970 con 0x5611ca5c3a00 2024-04-22T13:33:32.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.946+0000 7fda113e9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5611ca4d8500 con 0x5611ca5ba4d0 2024-04-22T13:33:32.950 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.946+0000 7fda113e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5611ca5c6af0 con 0x5611ca5bae80 2024-04-22T13:33:32.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.946+0000 7fda113e9740 10 client.0 _ll_register_callbacks cb 0x5611ca4a6b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:32.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.946+0000 7fda0cd91640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611ca5c3a00 0x5611ca5c0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.946+0000 7fda0cd91640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611ca5c3a00 0x5611ca5c0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60730/0 (socket says 172.21.15.12:60730) 2024-04-22T13:33:32.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.946+0000 7fda0cd91640 1 -- 172.21.15.12:0/1190419777 learned_addr learned my addr 172.21.15.12:0/1190419777 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:32.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.946+0000 7fda07fff640 1 --2- 172.21.15.12:0/1190419777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5611ca5ba4d0 0x5611ca5c3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.946+0000 7fda077fe640 1 --2- 172.21.15.12:0/1190419777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611ca5bae80 0x5611ca5c0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.946+0000 7fda0cd91640 1 -- 172.21.15.12:0/1190419777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611ca5bae80 msgr2=0x5611ca5c0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.946+0000 7fda0cd91640 1 --2- 172.21.15.12:0/1190419777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611ca5bae80 0x5611ca5c0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.946+0000 7fda0cd91640 1 -- 172.21.15.12:0/1190419777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5611ca5ba4d0 msgr2=0x5611ca5c3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.946+0000 7fda0cd91640 1 --2- 172.21.15.12:0/1190419777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5611ca5ba4d0 0x5611ca5c3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.946+0000 7fda0cd91640 1 -- 172.21.15.12:0/1190419777 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611ca2313f0 con 0x5611ca5c3a00 2024-04-22T13:33:32.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.946+0000 7fda077fe640 1 --2- 172.21.15.12:0/1190419777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611ca5bae80 0x5611ca5c0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:32.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.946+0000 7fda0cd91640 1 --2- 172.21.15.12:0/1190419777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611ca5c3a00 0x5611ca5c0d20 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fda00005f40 tx=0x7fda000194e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e705c6a4c00925 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:32.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.946+0000 7fd9e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:33:32.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.947+0000 7fd9e77fe640 1 -- 172.21.15.12:0/1190419777 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00027030 con 0x5611ca5c3a00 2024-04-22T13:33:32.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.947+0000 7fd9e77fe640 1 -- 172.21.15.12:0/1190419777 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7fda0001f070 con 0x5611ca5c3a00 2024-04-22T13:33:32.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.947+0000 7fda113e9740 1 -- 172.21.15.12:0/1190419777 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda00011d50 con 0x5611ca5c3a00 2024-04-22T13:33:32.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.947+0000 7fd9e77fe640 1 -- 172.21.15.12:0/1190419777 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fda00031050 con 0x5611ca5c3a00 2024-04-22T13:33:32.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.947+0000 7fda113e9740 10 client.18897 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:32.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.947+0000 7fda113e9740 1 -- 172.21.15.12:0/1190419777 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5611ca2fb7d0 con 0x5611ca5c3a00 2024-04-22T13:33:32.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.947+0000 7fd9e77fe640 1 -- 172.21.15.12:0/1190419777 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda00011d50 con 0x5611ca5c3a00 2024-04-22T13:33:32.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.947+0000 7fd9e77fe640 1 -- 172.21.15.12:0/1190419777 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fda00003a20 con 0x5611ca5c3a00 2024-04-22T13:33:32.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.947+0000 7fda113e9740 10 client.18897 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:32.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.947+0000 7fda113e9740 10 client.18897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:33:32.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.947+0000 7fda113e9740 20 client.18897 populate_metadata read hostname 'smithi012' 2024-04-22T13:33:32.952 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.947+0000 7fda113e9740 1 --2- 172.21.15.12:0/1190419777 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5611ca5d3b40 0x5611ca5d5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.947+0000 7fda113e9740 4 client.18897 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:33:32.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.947+0000 7fda113e9740 1 -- 172.21.15.12:0/1190419777 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611ca4be930 con 0x5611ca5d3b40 2024-04-22T13:33:32.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.947+0000 7fda07fff640 1 --2- 172.21.15.12:0/1190419777 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5611ca5d3b40 0x5611ca5d5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.948+0000 7fda07fff640 1 --2- 172.21.15.12:0/1190419777 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5611ca5d3b40 0x5611ca5d5f30 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17fa1c1d6aaf9346 server_cookie=c6f9f78a6df0ca20 in_seq=0 out_seq=0 2024-04-22T13:33:32.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.948+0000 7fd9e77fe640 10 client.18897 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:33:32.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.948+0000 7fd9e77fe640 1 -- 172.21.15.12:0/1190419777 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5611ca4be930 con 0x5611ca5d3b40 2024-04-22T13:33:32.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.948+0000 7fd9e77fe640 10 client.18897 handle_command_reply: tid=0 2024-04-22T13:33:32.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.949+0000 7fda113e9740 1 client.18897 shutdown 2024-04-22T13:33:32.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.949+0000 7fda113e9740 1 -- 172.21.15.12:0/1190419777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611ca5c3a00 msgr2=0x5611ca5c0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.949+0000 7fda113e9740 1 --2- 172.21.15.12:0/1190419777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611ca5c3a00 0x5611ca5c0d20 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fda00005f40 tx=0x7fda000194e0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.949+0000 7fda113e9740 1 -- 172.21.15.12:0/1190419777 shutdown_connections 2024-04-22T13:33:32.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.949+0000 7fda113e9740 1 --2- 172.21.15.12:0/1190419777 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5611ca5d3b40 0x5611ca5d5f30 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.949+0000 7fda113e9740 1 --2- 172.21.15.12:0/1190419777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611ca5c3a00 0x5611ca5c0d20 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.949+0000 7fda113e9740 1 --2- 172.21.15.12:0/1190419777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611ca5bae80 0x5611ca5c0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.949+0000 7fda113e9740 1 --2- 172.21.15.12:0/1190419777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5611ca5ba4d0 0x5611ca5c3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.949+0000 7fd9e77fe640 0 client.18897 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:33:32.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.949+0000 7fda113e9740 1 -- 172.21.15.12:0/1190419777 >> 172.21.15.12:0/1190419777 conn(0x5611ca5b4ba0 msgr2=0x5611ca5b68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:32.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.949+0000 7fda113e9740 1 -- 172.21.15.12:0/1190419777 shutdown_connections 2024-04-22T13:33:32.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.949+0000 7fda113e9740 1 -- 172.21.15.12:0/1190419777 wait complete. 2024-04-22T13:33:32.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.949+0000 7fda113e9740 20 client.18897 trim_cache size 0 max 16384 2024-04-22T13:33:32.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.962+0000 7fda113e9740 1 Processor -- start 2024-04-22T13:33:32.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.962+0000 7fda113e9740 1 -- start start 2024-04-22T13:33:32.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.962+0000 7fda113e9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5611ca5ba4d0 0x5611ca5cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.962+0000 7fda113e9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611ca5bae80 0x5611ca5bb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.962+0000 7fda113e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611ca5c3a00 0x5611ca5c2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.962+0000 7fda113e9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fda081afe20 con 0x5611ca5bae80 2024-04-22T13:33:32.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.962+0000 7fda113e9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fda081b1960 con 0x5611ca5ba4d0 2024-04-22T13:33:32.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.962+0000 7fda113e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fda081b04f0 con 0x5611ca5c3a00 2024-04-22T13:33:32.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.963+0000 7fda077fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611ca5bae80 0x5611ca5bb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.963+0000 7fda077fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611ca5bae80 0x5611ca5bb3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60734/0 (socket says 172.21.15.12:60734) 2024-04-22T13:33:32.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.963+0000 7fda077fe640 1 -- 172.21.15.12:0/3209111984 learned_addr learned my addr 172.21.15.12:0/3209111984 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:32.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.963+0000 7fda07fff640 1 --2- 172.21.15.12:0/3209111984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5611ca5ba4d0 0x5611ca5cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.963+0000 7fda077fe640 1 -- 172.21.15.12:0/3209111984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611ca5c3a00 msgr2=0x5611ca5c2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.963+0000 7fda077fe640 1 --2- 172.21.15.12:0/3209111984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611ca5c3a00 0x5611ca5c2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.963+0000 7fda077fe640 1 -- 172.21.15.12:0/3209111984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5611ca5ba4d0 msgr2=0x5611ca5cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.963+0000 7fda077fe640 1 --2- 172.21.15.12:0/3209111984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5611ca5ba4d0 0x5611ca5cf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.963+0000 7fda077fe640 1 -- 172.21.15.12:0/3209111984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611ca5d62a0 con 0x5611ca5bae80 2024-04-22T13:33:32.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.963+0000 7fda07fff640 1 --2- 172.21.15.12:0/3209111984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5611ca5ba4d0 0x5611ca5cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:32.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.963+0000 7fda077fe640 1 --2- 172.21.15.12:0/3209111984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611ca5bae80 0x5611ca5bb3d0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc09cac0 tx=0x7fd9fc098dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a46b87508ec810 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:32.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.963+0000 7fd9edffb640 1 -- 172.21.15.12:0/3209111984 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc09de60 con 0x5611ca5bae80 2024-04-22T13:33:32.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.963+0000 7fd9edffb640 1 -- 172.21.15.12:0/3209111984 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9fc0046c0 con 0x5611ca5bae80 2024-04-22T13:33:32.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.963+0000 7fd9edffb640 1 -- 172.21.15.12:0/3209111984 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc0a57a0 con 0x5611ca5bae80 2024-04-22T13:33:32.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.964+0000 7fda113e9740 1 -- 172.21.15.12:0/3209111984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611ca5bae80 msgr2=0x5611ca5bb3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.964+0000 7fda113e9740 1 --2- 172.21.15.12:0/3209111984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611ca5bae80 0x5611ca5bb3d0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc09cac0 tx=0x7fd9fc098dd0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.964+0000 7fda113e9740 1 -- 172.21.15.12:0/3209111984 shutdown_connections 2024-04-22T13:33:32.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.964+0000 7fda113e9740 1 --2- 172.21.15.12:0/3209111984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611ca5c3a00 0x5611ca5c2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.964+0000 7fda113e9740 1 --2- 172.21.15.12:0/3209111984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611ca5bae80 0x5611ca5bb3d0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.964+0000 7fda113e9740 1 --2- 172.21.15.12:0/3209111984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5611ca5ba4d0 0x5611ca5cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.964+0000 7fda113e9740 1 -- 172.21.15.12:0/3209111984 >> 172.21.15.12:0/3209111984 conn(0x5611ca5b4ba0 msgr2=0x5611ca5b6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:32.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.964+0000 7fda113e9740 1 -- 172.21.15.12:0/3209111984 shutdown_connections 2024-04-22T13:33:32.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.964+0000 7fda113e9740 1 -- 172.21.15.12:0/3209111984 wait complete. 2024-04-22T13:33:32.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.964+0000 7fda113e9740 1 -- start start 2024-04-22T13:33:32.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.964+0000 7fda113e9740 1 Processor -- start 2024-04-22T13:33:32.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.964+0000 7fda113e9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5611ca5ba4d0 0x5611ca5c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.964+0000 7fda113e9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611ca5bae80 0x5611ca5c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.965+0000 7fda113e9740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611ca5c3a00 0x5611ca5c8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.965+0000 7fda113e9740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fda081b1290 con 0x5611ca5c3a00 2024-04-22T13:33:32.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.965+0000 7fda113e9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fda081b0bc0 con 0x5611ca5ba4d0 2024-04-22T13:33:32.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.965+0000 7fda113e9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fda081ade20 con 0x5611ca5bae80 2024-04-22T13:33:32.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.965+0000 7fda113e9740 10 client.0 _ll_register_callbacks cb 0x7fda00009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:32.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.965+0000 7fda0cd91640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611ca5c3a00 0x5611ca5c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.965+0000 7fda0cd91640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611ca5c3a00 0x5611ca5c8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60744/0 (socket says 172.21.15.12:60744) 2024-04-22T13:33:32.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.965+0000 7fda0cd91640 1 -- 172.21.15.12:0/2927978750 learned_addr learned my addr 172.21.15.12:0/2927978750 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:32.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.965+0000 7fda0cd91640 1 -- 172.21.15.12:0/2927978750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611ca5bae80 msgr2=0x5611ca5c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.965+0000 7fda0cd91640 1 --2- 172.21.15.12:0/2927978750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611ca5bae80 0x5611ca5c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.965+0000 7fda07fff640 1 --2- 172.21.15.12:0/2927978750 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5611ca5ba4d0 0x5611ca5c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.965+0000 7fda0cd91640 1 -- 172.21.15.12:0/2927978750 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5611ca5ba4d0 msgr2=0x5611ca5c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:32.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.965+0000 7fda0cd91640 1 --2- 172.21.15.12:0/2927978750 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5611ca5ba4d0 0x5611ca5c77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.965+0000 7fda0cd91640 1 -- 172.21.15.12:0/2927978750 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611ca638df0 con 0x5611ca5c3a00 2024-04-22T13:33:32.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.965+0000 7fda0cd91640 1 --2- 172.21.15.12:0/2927978750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611ca5c3a00 0x5611ca5c8260 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fda00002230 tx=0x7fda00009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2bbcefe453cf00 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:32.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.965+0000 7fd9e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:33:32.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.965+0000 7fd9e77fe640 1 -- 172.21.15.12:0/2927978750 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0003a060 con 0x5611ca5c3a00 2024-04-22T13:33:32.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.965+0000 7fd9e77fe640 1 -- 172.21.15.12:0/2927978750 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7fda000117e0 con 0x5611ca5c3a00 2024-04-22T13:33:32.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.965+0000 7fda113e9740 1 -- 172.21.15.12:0/2927978750 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5611ca52cc10 con 0x5611ca5c3a00 2024-04-22T13:33:32.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.965+0000 7fd9e77fe640 1 -- 172.21.15.12:0/2927978750 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fda00027070 con 0x5611ca5c3a00 2024-04-22T13:33:32.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.965+0000 7fda113e9740 10 client.18903 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:32.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.965+0000 7fda113e9740 1 -- 172.21.15.12:0/2927978750 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5611ca63a040 con 0x5611ca5c3a00 2024-04-22T13:33:32.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.966+0000 7fd9e77fe640 1 -- 172.21.15.12:0/2927978750 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5611ca52cc10 con 0x5611ca5c3a00 2024-04-22T13:33:32.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.966+0000 7fd9e77fe640 1 -- 172.21.15.12:0/2927978750 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fda00019650 con 0x5611ca5c3a00 2024-04-22T13:33:32.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.966+0000 7fda113e9740 10 client.18903 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:32.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.966+0000 7fda113e9740 10 client.18903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:33:32.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.966+0000 7fda113e9740 20 client.18903 populate_metadata read hostname 'smithi012' 2024-04-22T13:33:32.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.966+0000 7fda113e9740 1 --2- 172.21.15.12:0/2927978750 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5611ca63a790 0x5611ca63cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:32.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.966+0000 7fda113e9740 4 client.18903 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:33:32.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.966+0000 7fda113e9740 1 -- 172.21.15.12:0/2927978750 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5611ca526a80 con 0x5611ca63a790 2024-04-22T13:33:32.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.966+0000 7fda07fff640 1 --2- 172.21.15.12:0/2927978750 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5611ca63a790 0x5611ca63cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:32.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.966+0000 7fda07fff640 1 --2- 172.21.15.12:0/2927978750 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5611ca63a790 0x5611ca63cbb0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8975fa751b469d5d server_cookie=2560dfd7608c62c9 in_seq=0 out_seq=0 2024-04-22T13:33:32.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.966+0000 7fd9e77fe640 10 client.18903 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:33:32.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.966+0000 7fd9e77fe640 1 -- 172.21.15.12:0/2927978750 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5611ca526a80 con 0x5611ca63a790 2024-04-22T13:33:32.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.966+0000 7fd9e77fe640 10 client.18903 handle_command_reply: tid=0 2024-04-22T13:33:32.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.966+0000 7fda113e9740 1 client.18903 shutdown 2024-04-22T13:33:32.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.967+0000 7fda113e9740 1 -- 172.21.15.12:0/2927978750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611ca5c3a00 msgr2=0x5611ca5c8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.967+0000 7fda113e9740 1 --2- 172.21.15.12:0/2927978750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611ca5c3a00 0x5611ca5c8260 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fda00002230 tx=0x7fda00009ec0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.967+0000 7fda113e9740 1 -- 172.21.15.12:0/2927978750 shutdown_connections 2024-04-22T13:33:32.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.967+0000 7fda113e9740 1 --2- 172.21.15.12:0/2927978750 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x5611ca63a790 0x5611ca63cbb0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.967+0000 7fda113e9740 1 --2- 172.21.15.12:0/2927978750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5611ca5c3a00 0x5611ca5c8260 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.967+0000 7fda113e9740 1 --2- 172.21.15.12:0/2927978750 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5611ca5bae80 0x5611ca5c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.967+0000 7fda113e9740 1 --2- 172.21.15.12:0/2927978750 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5611ca5ba4d0 0x5611ca5c77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.967+0000 7fd9e77fe640 0 client.18903 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:33:32.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.967+0000 7fda113e9740 1 -- 172.21.15.12:0/2927978750 >> 172.21.15.12:0/2927978750 conn(0x5611ca5b4ba0 msgr2=0x5611ca5b5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:32.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.967+0000 7fda113e9740 1 -- 172.21.15.12:0/2927978750 shutdown_connections 2024-04-22T13:33:32.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.967+0000 7fda113e9740 1 -- 172.21.15.12:0/2927978750 wait complete. 2024-04-22T13:33:32.968 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.967+0000 7fda113e9740 20 client.18903 trim_cache size 0 max 16384 2024-04-22T13:33:32.969 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:33:32.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.967+0000 7fda0e7f4640 1 -- 172.21.15.12:0/3366300782 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd9f00a3d50 msgr2=0x7fd9f00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.969 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.967+0000 7fda0e7f4640 1 --2- 172.21.15.12:0/3366300782 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd9f00a3d50 0x7fd9f00a6210 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8006010 tx=0x7fd9f8005e40 comp rx=0 tx=0).stop 2024-04-22T13:33:32.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.967+0000 7fda0e7f4640 1 -- 172.21.15.12:0/3366300782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda08106160 msgr2=0x7fda08103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:32.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.967+0000 7fda0e7f4640 1 --2- 172.21.15.12:0/3366300782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda08106160 0x7fda08103060 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00e9c0 tx=0x7fd9fc00ee80 comp rx=0 tx=0).stop 2024-04-22T13:33:32.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.967+0000 7fda0e7f4640 1 -- 172.21.15.12:0/3366300782 shutdown_connections 2024-04-22T13:33:32.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.967+0000 7fda0e7f4640 1 --2- 172.21.15.12:0/3366300782 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd9f00a3d50 0x7fd9f00a6210 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.967+0000 7fda0e7f4640 1 --2- 172.21.15.12:0/3366300782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda081035a0 0x7fda081a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.967+0000 7fda0e7f4640 1 --2- 172.21.15.12:0/3366300782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fda08106160 0x7fda08103060 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.967+0000 7fda0e7f4640 1 --2- 172.21.15.12:0/3366300782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda08101090 0x7fda08102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:32.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.967+0000 7fda0e7f4640 1 -- 172.21.15.12:0/3366300782 >> 172.21.15.12:0/3366300782 conn(0x7fda08077f50 msgr2=0x7fda08100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:32.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.967+0000 7fda0e7f4640 1 -- 172.21.15.12:0/3366300782 shutdown_connections 2024-04-22T13:33:32.970 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:32.968+0000 7fda0e7f4640 1 -- 172.21.15.12:0/3366300782 wait complete. 2024-04-22T13:33:32.975 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:32 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1268139744' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:33:33.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:33 smithi134 ceph-mon[31338]: pgmap v292: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:33:33.974 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:33:33.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:33:33.975 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:33:33.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:33 smithi111 ceph-mon[31361]: pgmap v292: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:33:34.011 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:33 smithi012 ceph-mon[28265]: pgmap v292: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:33:34.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.107+0000 7fe9f319d640 1 -- 172.21.15.12:0/2171770471 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9ec06a840 msgr2=0x7fe9ec06ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:34.105 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.107+0000 7fe9f319d640 1 --2- 172.21.15.12:0/2171770471 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9ec06a840 0x7fe9ec06ac20 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc009900 tx=0x7fe9dc02bc30 comp rx=0 tx=0).stop 2024-04-22T13:33:34.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.107+0000 7fe9f319d640 1 -- 172.21.15.12:0/2171770471 shutdown_connections 2024-04-22T13:33:34.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.107+0000 7fe9f319d640 1 --2- 172.21.15.12:0/2171770471 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9ec107270 0x7fe9ec10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.107+0000 7fe9f319d640 1 --2- 172.21.15.12:0/2171770471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9ec06b1f0 0x7fe9ec106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.107+0000 7fe9f319d640 1 --2- 172.21.15.12:0/2171770471 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9ec06a840 0x7fe9ec06ac20 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.107+0000 7fe9f319d640 1 -- 172.21.15.12:0/2171770471 >> 172.21.15.12:0/2171770471 conn(0x7fe9ec077eb0 msgr2=0x7fe9ec0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:34.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.107+0000 7fe9f319d640 1 -- 172.21.15.12:0/2171770471 shutdown_connections 2024-04-22T13:33:34.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.108+0000 7fe9f319d640 1 -- 172.21.15.12:0/2171770471 wait complete. 2024-04-22T13:33:34.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.108+0000 7fe9f319d640 1 Processor -- start 2024-04-22T13:33:34.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.108+0000 7fe9f319d640 1 -- start start 2024-04-22T13:33:34.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.108+0000 7fe9f319d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9ec06a840 0x7fe9ec198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.108+0000 7fe9f319d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9ec107270 0x7fe9ec198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.108+0000 7fe9f319d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9ec19cfd0 0x7fe9ec19d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.108+0000 7fe9f319d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec1121d0 con 0x7fe9ec06a840 2024-04-22T13:33:34.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.108+0000 7fe9f319d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec112050 con 0x7fe9ec107270 2024-04-22T13:33:34.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.108+0000 7fe9f319d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec112350 con 0x7fe9ec19cfd0 2024-04-22T13:33:34.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.108+0000 7fe9f0f39640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9ec06a840 0x7fe9ec198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.108+0000 7fe9f0f39640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9ec06a840 0x7fe9ec198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60764/0 (socket says 172.21.15.12:60764) 2024-04-22T13:33:34.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.108+0000 7fe9f0f39640 1 -- 172.21.15.12:0/2447131132 learned_addr learned my addr 172.21.15.12:0/2447131132 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:34.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.109+0000 7fe9f173a640 1 --2- 172.21.15.12:0/2447131132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9ec19cfd0 0x7fe9ec19d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.109+0000 7fe9ebfff640 1 --2- 172.21.15.12:0/2447131132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9ec107270 0x7fe9ec198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.109+0000 7fe9f0f39640 1 -- 172.21.15.12:0/2447131132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9ec19cfd0 msgr2=0x7fe9ec19d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:34.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.109+0000 7fe9f0f39640 1 --2- 172.21.15.12:0/2447131132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9ec19cfd0 0x7fe9ec19d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.109+0000 7fe9f0f39640 1 -- 172.21.15.12:0/2447131132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9ec107270 msgr2=0x7fe9ec198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:34.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.109+0000 7fe9f0f39640 1 --2- 172.21.15.12:0/2447131132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9ec107270 0x7fe9ec198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.109+0000 7fe9f0f39640 1 -- 172.21.15.12:0/2447131132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ec19dac0 con 0x7fe9ec06a840 2024-04-22T13:33:34.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.109+0000 7fe9f173a640 1 --2- 172.21.15.12:0/2447131132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9ec19cfd0 0x7fe9ec19d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:34.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.109+0000 7fe9f0f39640 1 --2- 172.21.15.12:0/2447131132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9ec06a840 0x7fe9ec198710 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc005940 tx=0x7fe9dc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:34.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.109+0000 7fe9e9ffb640 1 -- 172.21.15.12:0/2447131132 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc043020 con 0x7fe9ec06a840 2024-04-22T13:33:34.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.109+0000 7fe9f319d640 1 -- 172.21.15.12:0/2447131132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9ec19dd50 con 0x7fe9ec06a840 2024-04-22T13:33:34.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.109+0000 7fe9e9ffb640 1 -- 172.21.15.12:0/2447131132 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9dc02caf0 con 0x7fe9ec06a840 2024-04-22T13:33:34.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.109+0000 7fe9e9ffb640 1 -- 172.21.15.12:0/2447131132 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc003af0 con 0x7fe9ec06a840 2024-04-22T13:33:34.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.109+0000 7fe9f319d640 1 -- 172.21.15.12:0/2447131132 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9ec1a5570 con 0x7fe9ec06a840 2024-04-22T13:33:34.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.112+0000 7fe9e9ffb640 1 -- 172.21.15.12:0/2447131132 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe9dc046020 con 0x7fe9ec06a840 2024-04-22T13:33:34.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.112+0000 7fe9f5d8c740 1 Processor -- start 2024-04-22T13:33:34.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.112+0000 7fe9f5d8c740 1 -- start start 2024-04-22T13:33:34.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.112+0000 7fe9f5d8c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af3d517c0 0x559af3d51ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.112+0000 7fe9f5d8c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559af3d52170 0x559af3d5aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.112+0000 7fe9f5d8c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559af3d5b270 0x559af3d5b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.112+0000 7fe9f5d8c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559af3cb5f00 con 0x559af3d517c0 2024-04-22T13:33:34.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.112+0000 7fe9f5d8c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559af3c53590 con 0x559af3d52170 2024-04-22T13:33:34.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.112+0000 7fe9f5d8c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559af3cb1ec0 con 0x559af3d5b270 2024-04-22T13:33:34.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.112+0000 7fe9f173a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af3d517c0 0x559af3d51ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.113+0000 7fe9f173a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af3d517c0 0x559af3d51ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60766/0 (socket says 172.21.15.12:60766) 2024-04-22T13:33:34.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.113+0000 7fe9f173a640 1 -- 172.21.15.12:0/595640199 learned_addr learned my addr 172.21.15.12:0/595640199 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:34.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.113+0000 7fe9ebfff640 1 --2- 172.21.15.12:0/595640199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559af3d5b270 0x559af3d5b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.113+0000 7fe9e9ffb640 1 --2- 172.21.15.12:0/2447131132 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe9d40a3d20 0x7fe9d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.113+0000 7fe9f0f39640 1 --2- 172.21.15.12:0/595640199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559af3d52170 0x559af3d5aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.113+0000 7fe9f173a640 1 -- 172.21.15.12:0/595640199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559af3d5b270 msgr2=0x559af3d5b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:34.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.113+0000 7fe9f173a640 1 --2- 172.21.15.12:0/595640199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559af3d5b270 0x559af3d5b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.113+0000 7fe9f173a640 1 -- 172.21.15.12:0/595640199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559af3d52170 msgr2=0x559af3d5aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:34.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.113+0000 7fe9f173a640 1 --2- 172.21.15.12:0/595640199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559af3d52170 0x559af3d5aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.113+0000 7fe9f173a640 1 -- 172.21.15.12:0/595640199 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559af3a907e0 con 0x559af3d517c0 2024-04-22T13:33:34.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.113+0000 7fe9f0f39640 1 --2- 172.21.15.12:0/595640199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559af3d52170 0x559af3d5aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:34.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.113+0000 7fe9f173a640 1 --2- 172.21.15.12:0/2447131132 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe9d40a3d20 0x7fe9d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.116 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.113+0000 7fe9e9ffb640 1 -- 172.21.15.12:0/2447131132 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7fe9dc0f2970 con 0x7fe9ec06a840 2024-04-22T13:33:34.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.113+0000 7fe9f173a640 1 --2- 172.21.15.12:0/595640199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af3d517c0 0x559af3d51ba0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4009fa0 tx=0x7fe9e400f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ad6c58c2949dae server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:34.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.113+0000 7fe9d2ffd640 1 -- 172.21.15.12:0/595640199 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e401a610 con 0x559af3d517c0 2024-04-22T13:33:34.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.113+0000 7fe9d2ffd640 1 -- 172.21.15.12:0/595640199 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9e401ac60 con 0x559af3d517c0 2024-04-22T13:33:34.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.113+0000 7fe9d2ffd640 1 -- 172.21.15.12:0/595640199 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e400d9b0 con 0x559af3d517c0 2024-04-22T13:33:34.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.114+0000 7fe9f5d8c740 1 -- 172.21.15.12:0/595640199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af3d517c0 msgr2=0x559af3d51ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:34.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.114+0000 7fe9f5d8c740 1 --2- 172.21.15.12:0/595640199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af3d517c0 0x559af3d51ba0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4009fa0 tx=0x7fe9e400f7f0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.114+0000 7fe9f173a640 1 --2- 172.21.15.12:0/2447131132 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe9d40a3d20 0x7fe9d40a61e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4020000 tx=0x7fe9e401d000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:34.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.114+0000 7fe9f5d8c740 1 -- 172.21.15.12:0/595640199 shutdown_connections 2024-04-22T13:33:34.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.114+0000 7fe9f5d8c740 1 --2- 172.21.15.12:0/595640199 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559af3d5b270 0x559af3d5b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.114+0000 7fe9f5d8c740 1 --2- 172.21.15.12:0/595640199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559af3d52170 0x559af3d5aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.114+0000 7fe9f5d8c740 1 --2- 172.21.15.12:0/595640199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af3d517c0 0x559af3d51ba0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.114+0000 7fe9f5d8c740 1 -- 172.21.15.12:0/595640199 >> 172.21.15.12:0/595640199 conn(0x559af3d4bfb0 msgr2=0x559af3d4c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:34.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.114+0000 7fe9f5d8c740 1 -- 172.21.15.12:0/595640199 shutdown_connections 2024-04-22T13:33:34.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.114+0000 7fe9f5d8c740 1 -- 172.21.15.12:0/595640199 wait complete. 2024-04-22T13:33:34.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.115+0000 7fe9f5d8c740 1 -- start start 2024-04-22T13:33:34.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.115+0000 7fe9f5d8c740 1 Processor -- start 2024-04-22T13:33:34.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.115+0000 7fe9f5d8c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559af3d517c0 0x559af3d5ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.115+0000 7fe9f5d8c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559af3d52170 0x559af3d5f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.115+0000 7fe9f5d8c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af3d5b270 0x559af3d5f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.115+0000 7fe9f5d8c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x559af3d621b0 con 0x559af3d5b270 2024-04-22T13:33:34.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.115+0000 7fe9f5d8c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559af3c6f580 con 0x559af3d517c0 2024-04-22T13:33:34.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.115+0000 7fe9f5d8c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559af3d62330 con 0x559af3d52170 2024-04-22T13:33:34.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.115+0000 7fe9f5d8c740 10 client.0 _ll_register_callbacks cb 0x559af3c75650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:34.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.115+0000 7fe9f173a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af3d5b270 0x559af3d5f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.115+0000 7fe9ebfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559af3d52170 0x559af3d5f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.115+0000 7fe9f173a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af3d5b270 0x559af3d5f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60780/0 (socket says 172.21.15.12:60780) 2024-04-22T13:33:34.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.115+0000 7fe9f173a640 1 -- 172.21.15.12:0/4174873090 learned_addr learned my addr 172.21.15.12:0/4174873090 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:34.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.115+0000 7fe9f0f39640 1 --2- 172.21.15.12:0/4174873090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559af3d517c0 0x559af3d5ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.116+0000 7fe9ebfff640 1 -- 172.21.15.12:0/4174873090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559af3d517c0 msgr2=0x559af3d5ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:34.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.116+0000 7fe9ebfff640 1 --2- 172.21.15.12:0/4174873090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559af3d517c0 0x559af3d5ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.116+0000 7fe9ebfff640 1 -- 172.21.15.12:0/4174873090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af3d5b270 msgr2=0x559af3d5f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:34.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.116+0000 7fe9ebfff640 1 --2- 172.21.15.12:0/4174873090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af3d5b270 0x559af3d5f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.116+0000 7fe9ebfff640 1 -- 172.21.15.12:0/4174873090 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559af3af0f70 con 0x559af3d52170 2024-04-22T13:33:34.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.116+0000 7fe9f173a640 1 --2- 172.21.15.12:0/4174873090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af3d5b270 0x559af3d5f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:33:34.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.116+0000 7fe9ebfff640 1 --2- 172.21.15.12:0/4174873090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559af3d52170 0x559af3d5f200 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec06a420 tx=0x7fe9e0013b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5ee3e3c60b968bc server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:34.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.116+0000 7fe9cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-04-22T13:33:34.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.116+0000 7fe9cbfff640 1 -- 172.21.15.12:0/4174873090 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0004440 con 0x559af3d52170 2024-04-22T13:33:34.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.116+0000 7fe9f5d8c740 1 -- 172.21.15.12:0/4174873090 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9e401ac60 con 0x559af3d52170 2024-04-22T13:33:34.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.116+0000 7fe9cbfff640 1 -- 172.21.15.12:0/4174873090 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7fe9e00119c0 con 0x559af3d52170 2024-04-22T13:33:34.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.116+0000 7fe9cbfff640 1 -- 172.21.15.12:0/4174873090 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe9e0011bc0 con 0x559af3d52170 2024-04-22T13:33:34.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.116+0000 7fe9f5d8c740 10 client.27817 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:34.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.116+0000 7fe9f5d8c740 1 -- 172.21.15.12:0/4174873090 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559af3988170 con 0x559af3d52170 2024-04-22T13:33:34.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.117+0000 7fe9cbfff640 1 -- 172.21.15.12:0/4174873090 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9e401ac60 con 0x559af3d52170 2024-04-22T13:33:34.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.117+0000 7fe9cbfff640 1 -- 172.21.15.12:0/4174873090 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe9e001a020 con 0x559af3d52170 2024-04-22T13:33:34.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.117+0000 7fe9f5d8c740 10 client.27817 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:34.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.117+0000 7fe9f5d8c740 10 client.27817 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:33:34.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.117+0000 7fe9f5d8c740 20 client.27817 populate_metadata read hostname 'smithi012' 2024-04-22T13:33:34.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.117+0000 7fe9f5d8c740 1 --2- 172.21.15.12:0/4174873090 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559af3d6f7b0 0x559af3d71bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.117+0000 7fe9f5d8c740 4 client.27817 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:33:34.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.117+0000 7fe9f5d8c740 1 -- 172.21.15.12:0/4174873090 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559af3cc3920 con 0x559af3d6f7b0 2024-04-22T13:33:34.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.117+0000 7fe9f0f39640 1 --2- 172.21.15.12:0/4174873090 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559af3d6f7b0 0x559af3d71bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.118+0000 7fe9cbfff640 10 client.27817 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:33:34.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.118+0000 7fe9f0f39640 1 --2- 172.21.15.12:0/4174873090 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559af3d6f7b0 0x559af3d71bc0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7187f1e3624c771 server_cookie=252928f2cdda9703 in_seq=0 out_seq=0 2024-04-22T13:33:34.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.118+0000 7fe9cbfff640 1 -- 172.21.15.12:0/4174873090 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559af3cc3920 con 0x559af3d6f7b0 2024-04-22T13:33:34.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.118+0000 7fe9cbfff640 10 client.27817 handle_command_reply: tid=0 2024-04-22T13:33:34.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.118+0000 7fe9f5d8c740 1 client.27817 shutdown 2024-04-22T13:33:34.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.118+0000 7fe9f5d8c740 1 -- 172.21.15.12:0/4174873090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559af3d52170 msgr2=0x559af3d5f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:34.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.118+0000 7fe9f5d8c740 1 --2- 172.21.15.12:0/4174873090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559af3d52170 0x559af3d5f200 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec06a420 tx=0x7fe9e0013b70 comp rx=0 tx=0).stop 2024-04-22T13:33:34.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.119+0000 7fe9f5d8c740 1 -- 172.21.15.12:0/4174873090 shutdown_connections 2024-04-22T13:33:34.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.119+0000 7fe9f5d8c740 1 --2- 172.21.15.12:0/4174873090 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559af3d6f7b0 0x559af3d71bc0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.119+0000 7fe9f5d8c740 1 --2- 172.21.15.12:0/4174873090 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af3d5b270 0x559af3d5f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.119+0000 7fe9f5d8c740 1 --2- 172.21.15.12:0/4174873090 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559af3d52170 0x559af3d5f200 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.119+0000 7fe9cbfff640 0 client.27817 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:33:34.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.119+0000 7fe9f5d8c740 1 --2- 172.21.15.12:0/4174873090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559af3d517c0 0x559af3d5ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.119+0000 7fe9f5d8c740 1 -- 172.21.15.12:0/4174873090 >> 172.21.15.12:0/4174873090 conn(0x559af3d4bfb0 msgr2=0x559af3d4dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:34.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.119+0000 7fe9f5d8c740 1 -- 172.21.15.12:0/4174873090 shutdown_connections 2024-04-22T13:33:34.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.119+0000 7fe9f5d8c740 1 -- 172.21.15.12:0/4174873090 wait complete. 2024-04-22T13:33:34.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.119+0000 7fe9f5d8c740 20 client.27817 trim_cache size 0 max 16384 2024-04-22T13:33:34.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.136+0000 7fe9f5d8c740 1 Processor -- start 2024-04-22T13:33:34.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.136+0000 7fe9f5d8c740 1 -- start start 2024-04-22T13:33:34.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.137+0000 7fe9f5d8c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559af3d517c0 0x559af3d613f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.137+0000 7fe9f5d8c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559af3d52170 0x559af3d6b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.137+0000 7fe9f5d8c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af3d5b270 0x559af3d5c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.137+0000 7fe9f5d8c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec1ab8e0 con 0x559af3d5b270 2024-04-22T13:33:34.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.137+0000 7fe9f5d8c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec1ad420 con 0x559af3d517c0 2024-04-22T13:33:34.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.137+0000 7fe9f5d8c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec1abfb0 con 0x559af3d52170 2024-04-22T13:33:34.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.137+0000 7fe9f173a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af3d5b270 0x559af3d5c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.137+0000 7fe9f173a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af3d5b270 0x559af3d5c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60784/0 (socket says 172.21.15.12:60784) 2024-04-22T13:33:34.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.137+0000 7fe9f173a640 1 -- 172.21.15.12:0/7702385 learned_addr learned my addr 172.21.15.12:0/7702385 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:34.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.137+0000 7fe9f0f39640 1 --2- 172.21.15.12:0/7702385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559af3d517c0 0x559af3d613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.137+0000 7fe9ebfff640 1 --2- 172.21.15.12:0/7702385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559af3d52170 0x559af3d6b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.137+0000 7fe9f173a640 1 -- 172.21.15.12:0/7702385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559af3d52170 msgr2=0x559af3d6b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:34.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.137+0000 7fe9f173a640 1 --2- 172.21.15.12:0/7702385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559af3d52170 0x559af3d6b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.137+0000 7fe9f173a640 1 -- 172.21.15.12:0/7702385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559af3d517c0 msgr2=0x559af3d613f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:34.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.137+0000 7fe9f173a640 1 --2- 172.21.15.12:0/7702385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559af3d517c0 0x559af3d613f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.137+0000 7fe9f173a640 1 -- 172.21.15.12:0/7702385 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559af3d6e300 con 0x559af3d5b270 2024-04-22T13:33:34.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.137+0000 7fe9f173a640 1 --2- 172.21.15.12:0/7702385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af3d5b270 0x559af3d5c660 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fe9e400d670 tx=0x7fe9e4026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b12bfed4ba3151c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:34.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.137+0000 7fe9d27fc640 1 -- 172.21.15.12:0/7702385 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4009af0 con 0x559af3d5b270 2024-04-22T13:33:34.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.137+0000 7fe9d27fc640 1 -- 172.21.15.12:0/7702385 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9e4029040 con 0x559af3d5b270 2024-04-22T13:33:34.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.137+0000 7fe9d27fc640 1 -- 172.21.15.12:0/7702385 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e40122a0 con 0x559af3d5b270 2024-04-22T13:33:34.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.138+0000 7fe9f5d8c740 1 -- 172.21.15.12:0/7702385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af3d5b270 msgr2=0x559af3d5c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:34.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.138+0000 7fe9f5d8c740 1 --2- 172.21.15.12:0/7702385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af3d5b270 0x559af3d5c660 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fe9e400d670 tx=0x7fe9e4026000 comp rx=0 tx=0).stop 2024-04-22T13:33:34.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.138+0000 7fe9f5d8c740 1 -- 172.21.15.12:0/7702385 shutdown_connections 2024-04-22T13:33:34.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.138+0000 7fe9f5d8c740 1 --2- 172.21.15.12:0/7702385 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af3d5b270 0x559af3d5c660 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.138+0000 7fe9f5d8c740 1 --2- 172.21.15.12:0/7702385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559af3d52170 0x559af3d6b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.138+0000 7fe9f5d8c740 1 --2- 172.21.15.12:0/7702385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559af3d517c0 0x559af3d613f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.138+0000 7fe9f5d8c740 1 -- 172.21.15.12:0/7702385 >> 172.21.15.12:0/7702385 conn(0x559af3d4bfb0 msgr2=0x559af3d4b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:34.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.138+0000 7fe9f5d8c740 1 -- 172.21.15.12:0/7702385 shutdown_connections 2024-04-22T13:33:34.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.138+0000 7fe9f5d8c740 1 -- 172.21.15.12:0/7702385 wait complete. 2024-04-22T13:33:34.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.138+0000 7fe9f5d8c740 1 -- start start 2024-04-22T13:33:34.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.138+0000 7fe9f5d8c740 1 Processor -- start 2024-04-22T13:33:34.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.139+0000 7fe9f5d8c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559af3d517c0 0x559af3d67aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.139+0000 7fe9f5d8c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559af3d52170 0x559af3d64590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.139+0000 7fe9f5d8c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af3d5b270 0x559af3d64b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.139+0000 7fe9f5d8c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec1acd50 con 0x559af3d5b270 2024-04-22T13:33:34.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.139+0000 7fe9f5d8c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec1ac680 con 0x559af3d52170 2024-04-22T13:33:34.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.139+0000 7fe9f5d8c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec1a9900 con 0x559af3d517c0 2024-04-22T13:33:34.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.139+0000 7fe9f5d8c740 10 client.0 _ll_register_callbacks cb 0x7fe9e00096e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:34.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.139+0000 7fe9f173a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af3d5b270 0x559af3d64b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.139+0000 7fe9f173a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af3d5b270 0x559af3d64b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60786/0 (socket says 172.21.15.12:60786) 2024-04-22T13:33:34.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.139+0000 7fe9f173a640 1 -- 172.21.15.12:0/3118062185 learned_addr learned my addr 172.21.15.12:0/3118062185 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:34.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.139+0000 7fe9ebfff640 1 --2- 172.21.15.12:0/3118062185 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559af3d52170 0x559af3d64590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.139+0000 7fe9f0f39640 1 --2- 172.21.15.12:0/3118062185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559af3d517c0 0x559af3d67aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.139+0000 7fe9f173a640 1 -- 172.21.15.12:0/3118062185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559af3d517c0 msgr2=0x559af3d67aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:34.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.139+0000 7fe9f173a640 1 --2- 172.21.15.12:0/3118062185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559af3d517c0 0x559af3d67aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.139+0000 7fe9f173a640 1 -- 172.21.15.12:0/3118062185 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559af3d52170 msgr2=0x559af3d64590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:34.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.139+0000 7fe9f173a640 1 --2- 172.21.15.12:0/3118062185 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559af3d52170 0x559af3d64590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.139+0000 7fe9f173a640 1 -- 172.21.15.12:0/3118062185 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559af3d65340 con 0x559af3d5b270 2024-04-22T13:33:34.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.140+0000 7fe9f173a640 1 --2- 172.21.15.12:0/3118062185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af3d5b270 0x559af3d64b10 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4009280 tx=0x7fe9e4009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d22197dfee70eec8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:34.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.140+0000 7fe9cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:33:34.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.140+0000 7fe9cbfff640 1 -- 172.21.15.12:0/3118062185 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e403b030 con 0x559af3d5b270 2024-04-22T13:33:34.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.140+0000 7fe9cbfff640 1 -- 172.21.15.12:0/3118062185 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7fe9e4033050 con 0x559af3d5b270 2024-04-22T13:33:34.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.140+0000 7fe9f5d8c740 1 -- 172.21.15.12:0/3118062185 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9e4029040 con 0x559af3d5b270 2024-04-22T13:33:34.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.140+0000 7fe9cbfff640 1 -- 172.21.15.12:0/3118062185 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe9e400d420 con 0x559af3d5b270 2024-04-22T13:33:34.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.140+0000 7fe9f5d8c740 10 client.18921 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:34.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.140+0000 7fe9f5d8c740 1 -- 172.21.15.12:0/3118062185 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559af3dd6e30 con 0x559af3d5b270 2024-04-22T13:33:34.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.140+0000 7fe9cbfff640 1 -- 172.21.15.12:0/3118062185 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9e4029040 con 0x559af3d5b270 2024-04-22T13:33:34.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.140+0000 7fe9cbfff640 1 -- 172.21.15.12:0/3118062185 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe9e40377e0 con 0x559af3d5b270 2024-04-22T13:33:34.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.140+0000 7fe9f5d8c740 10 client.18921 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:34.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.140+0000 7fe9f5d8c740 10 client.18921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:33:34.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.140+0000 7fe9f5d8c740 20 client.18921 populate_metadata read hostname 'smithi012' 2024-04-22T13:33:34.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.140+0000 7fe9f5d8c740 1 --2- 172.21.15.12:0/3118062185 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559af3dd73a0 0x559af3dd9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.140+0000 7fe9f5d8c740 4 client.18921 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:33:34.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.140+0000 7fe9f5d8c740 1 -- 172.21.15.12:0/3118062185 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559af3cbd790 con 0x559af3dd73a0 2024-04-22T13:33:34.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.141+0000 7fe9f0f39640 1 --2- 172.21.15.12:0/3118062185 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559af3dd73a0 0x559af3dd9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.141+0000 7fe9f0f39640 1 --2- 172.21.15.12:0/3118062185 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559af3dd73a0 0x559af3dd9790 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c08d3cbdeeebdd46 server_cookie=198c02248791a7e8 in_seq=0 out_seq=0 2024-04-22T13:33:34.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.141+0000 7fe9cbfff640 10 client.18921 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:33:34.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.142+0000 7fe9cbfff640 1 -- 172.21.15.12:0/3118062185 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559af3cbd790 con 0x559af3dd73a0 2024-04-22T13:33:34.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.142+0000 7fe9cbfff640 10 client.18921 handle_command_reply: tid=0 2024-04-22T13:33:34.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.142+0000 7fe9f5d8c740 1 client.18921 shutdown 2024-04-22T13:33:34.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.142+0000 7fe9f5d8c740 1 -- 172.21.15.12:0/3118062185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af3d5b270 msgr2=0x559af3d64b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:34.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.142+0000 7fe9f5d8c740 1 --2- 172.21.15.12:0/3118062185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af3d5b270 0x559af3d64b10 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4009280 tx=0x7fe9e4009510 comp rx=0 tx=0).stop 2024-04-22T13:33:34.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.142+0000 7fe9f5d8c740 1 -- 172.21.15.12:0/3118062185 shutdown_connections 2024-04-22T13:33:34.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.142+0000 7fe9f5d8c740 1 --2- 172.21.15.12:0/3118062185 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x559af3dd73a0 0x559af3dd9790 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.142+0000 7fe9cbfff640 0 client.18921 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:33:34.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.142+0000 7fe9f5d8c740 1 --2- 172.21.15.12:0/3118062185 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559af3d5b270 0x559af3d64b10 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.142+0000 7fe9f5d8c740 1 --2- 172.21.15.12:0/3118062185 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559af3d52170 0x559af3d64590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.142+0000 7fe9f5d8c740 1 --2- 172.21.15.12:0/3118062185 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559af3d517c0 0x559af3d67aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.142+0000 7fe9f5d8c740 1 -- 172.21.15.12:0/3118062185 >> 172.21.15.12:0/3118062185 conn(0x559af3d4bfb0 msgr2=0x559af3d5ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:34.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.143+0000 7fe9f5d8c740 1 -- 172.21.15.12:0/3118062185 shutdown_connections 2024-04-22T13:33:34.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.143+0000 7fe9f5d8c740 1 -- 172.21.15.12:0/3118062185 wait complete. 2024-04-22T13:33:34.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.143+0000 7fe9f5d8c740 20 client.18921 trim_cache size 0 max 16384 2024-04-22T13:33:34.144 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:33:34.144 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-04-22T13:33:34.144 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "ba11c3e2-f7d1-4574-9a4a-81c595744c9f", 2024-04-22T13:33:34.144 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-04-22T13:33:34.144 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:33:34.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.143+0000 7fe9f319d640 1 -- 172.21.15.12:0/2447131132 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe9d40a3d20 msgr2=0x7fe9d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:34.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.143+0000 7fe9f319d640 1 --2- 172.21.15.12:0/2447131132 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe9d40a3d20 0x7fe9d40a61e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4020000 tx=0x7fe9e401d000 comp rx=0 tx=0).stop 2024-04-22T13:33:34.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.143+0000 7fe9f319d640 1 -- 172.21.15.12:0/2447131132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9ec06a840 msgr2=0x7fe9ec198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:34.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.143+0000 7fe9f319d640 1 --2- 172.21.15.12:0/2447131132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9ec06a840 0x7fe9ec198710 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc005940 tx=0x7fe9dc0045e0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.143+0000 7fe9f319d640 1 -- 172.21.15.12:0/2447131132 shutdown_connections 2024-04-22T13:33:34.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.143+0000 7fe9f319d640 1 --2- 172.21.15.12:0/2447131132 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fe9d40a3d20 0x7fe9d40a61e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.145 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.143+0000 7fe9f319d640 1 --2- 172.21.15.12:0/2447131132 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9ec19cfd0 0x7fe9ec19d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.143+0000 7fe9f319d640 1 --2- 172.21.15.12:0/2447131132 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe9ec107270 0x7fe9ec198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.143+0000 7fe9f319d640 1 --2- 172.21.15.12:0/2447131132 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe9ec06a840 0x7fe9ec198710 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.143+0000 7fe9f319d640 1 -- 172.21.15.12:0/2447131132 >> 172.21.15.12:0/2447131132 conn(0x7fe9ec077eb0 msgr2=0x7fe9ec108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:34.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.143+0000 7fe9f319d640 1 -- 172.21.15.12:0/2447131132 shutdown_connections 2024-04-22T13:33:34.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.143+0000 7fe9f319d640 1 -- 172.21.15.12:0/2447131132 wait complete. 2024-04-22T13:33:34.150 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:33:34.150 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:33:34.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.303+0000 7fc7be66d640 1 -- 172.21.15.12:0/3733269646 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7b81010a0 msgr2=0x7fc7b8101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:34.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.303+0000 7fc7be66d640 1 --2- 172.21.15.12:0/3733269646 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7b81010a0 0x7fc7b8101500 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac009900 tx=0x7fc7ac02bc30 comp rx=0 tx=0).stop 2024-04-22T13:33:34.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.303+0000 7fc7be66d640 1 -- 172.21.15.12:0/3733269646 shutdown_connections 2024-04-22T13:33:34.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.303+0000 7fc7be66d640 1 --2- 172.21.15.12:0/3733269646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7b8101a40 0x7fc7b810dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.303+0000 7fc7be66d640 1 --2- 172.21.15.12:0/3733269646 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7b81010a0 0x7fc7b8101500 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.303+0000 7fc7be66d640 1 --2- 172.21.15.12:0/3733269646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b8105dd0 0x7fc7b81061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.303+0000 7fc7be66d640 1 -- 172.21.15.12:0/3733269646 >> 172.21.15.12:0/3733269646 conn(0x7fc7b8077c40 msgr2=0x7fc7b8078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:34.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.303+0000 7fc7be66d640 1 -- 172.21.15.12:0/3733269646 shutdown_connections 2024-04-22T13:33:34.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.303+0000 7fc7be66d640 1 -- 172.21.15.12:0/3733269646 wait complete. 2024-04-22T13:33:34.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.304+0000 7fc7be66d640 1 Processor -- start 2024-04-22T13:33:34.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.304+0000 7fc7be66d640 1 -- start start 2024-04-22T13:33:34.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.304+0000 7fc7be66d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b81010a0 0x7fc7b8197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.304+0000 7fc7be66d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7b8101a40 0x7fc7b8197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.304+0000 7fc7be66d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7b8105dd0 0x7fc7b819bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.304+0000 7fc7be66d640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b8111150 con 0x7fc7b8101a40 2024-04-22T13:33:34.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.304+0000 7fc7be66d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b8110fd0 con 0x7fc7b8105dd0 2024-04-22T13:33:34.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.304+0000 7fc7be66d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b81112d0 con 0x7fc7b81010a0 2024-04-22T13:33:34.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.304+0000 7fc7b77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7b8101a40 0x7fc7b8197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.304+0000 7fc7b77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7b8101a40 0x7fc7b8197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60790/0 (socket says 172.21.15.12:60790) 2024-04-22T13:33:34.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.304+0000 7fc7b77fe640 1 -- 172.21.15.12:0/944789037 learned_addr learned my addr 172.21.15.12:0/944789037 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:34.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.304+0000 7fc7b7fff640 1 --2- 172.21.15.12:0/944789037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b81010a0 0x7fc7b8197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.304+0000 7fc7bcc0a640 1 --2- 172.21.15.12:0/944789037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7b8105dd0 0x7fc7b819bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.304+0000 7fc7b77fe640 1 -- 172.21.15.12:0/944789037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b81010a0 msgr2=0x7fc7b8197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:34.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.304+0000 7fc7b77fe640 1 --2- 172.21.15.12:0/944789037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b81010a0 0x7fc7b8197690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.304+0000 7fc7b77fe640 1 -- 172.21.15.12:0/944789037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7b8105dd0 msgr2=0x7fc7b819bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:34.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.304+0000 7fc7b77fe640 1 --2- 172.21.15.12:0/944789037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7b8105dd0 0x7fc7b819bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.305+0000 7fc7b77fe640 1 -- 172.21.15.12:0/944789037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b819c680 con 0x7fc7b8101a40 2024-04-22T13:33:34.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.305+0000 7fc7bcc0a640 1 --2- 172.21.15.12:0/944789037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7b8105dd0 0x7fc7b819bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:34.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.305+0000 7fc7b77fe640 1 --2- 172.21.15.12:0/944789037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7b8101a40 0x7fc7b8197bd0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac005940 tx=0x7fc7ac02c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:34.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.305+0000 7fc7b57fa640 1 -- 172.21.15.12:0/944789037 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac043020 con 0x7fc7b8101a40 2024-04-22T13:33:34.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.305+0000 7fc7b57fa640 1 -- 172.21.15.12:0/944789037 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7ac02cbe0 con 0x7fc7b8101a40 2024-04-22T13:33:34.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.305+0000 7fc7be66d640 1 -- 172.21.15.12:0/944789037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b819c910 con 0x7fc7b8101a40 2024-04-22T13:33:34.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.305+0000 7fc7b57fa640 1 -- 172.21.15.12:0/944789037 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac003bf0 con 0x7fc7b8101a40 2024-04-22T13:33:34.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.305+0000 7fc7be66d640 1 -- 172.21.15.12:0/944789037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b81a4200 con 0x7fc7b8101a40 2024-04-22T13:33:34.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.308+0000 7fc7b57fa640 1 -- 172.21.15.12:0/944789037 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc7ac046020 con 0x7fc7b8101a40 2024-04-22T13:33:34.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.308+0000 7fc7c1263740 1 Processor -- start 2024-04-22T13:33:34.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.308+0000 7fc7c1263740 1 -- start start 2024-04-22T13:33:34.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.308+0000 7fc7c1263740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3bfe10e90 0x55a3bfe11270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.308+0000 7fc7c1263740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3bfe117b0 0x55a3bfe1eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.308+0000 7fc7c1263740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bfe1f080 0x55a3bfe21470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.308+0000 7fc7c1263740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a3bfd761e0 con 0x55a3bfe10e90 2024-04-22T13:33:34.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.308+0000 7fc7c1263740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a3bfd13390 con 0x55a3bfe1f080 2024-04-22T13:33:34.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.308+0000 7fc7c1263740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a3bfd721a0 con 0x55a3bfe117b0 2024-04-22T13:33:34.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.308+0000 7fc7bcc0a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3bfe10e90 0x55a3bfe11270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.308+0000 7fc7bcc0a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3bfe10e90 0x55a3bfe11270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60806/0 (socket says 172.21.15.12:60806) 2024-04-22T13:33:34.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.308+0000 7fc7bcc0a640 1 -- 172.21.15.12:0/4090131998 learned_addr learned my addr 172.21.15.12:0/4090131998 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:34.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.308+0000 7fc7b57fa640 1 --2- 172.21.15.12:0/944789037 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc7a00a3d20 0x7fc7a00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.308+0000 7fc7b77fe640 1 --2- 172.21.15.12:0/4090131998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bfe1f080 0x55a3bfe21470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.309+0000 7fc7b7fff640 1 --2- 172.21.15.12:0/4090131998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3bfe117b0 0x55a3bfe1eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.309+0000 7fc7bcc0a640 1 -- 172.21.15.12:0/4090131998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3bfe117b0 msgr2=0x55a3bfe1eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:34.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.309+0000 7fc7bcc0a640 1 --2- 172.21.15.12:0/4090131998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3bfe117b0 0x55a3bfe1eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.309+0000 7fc7bcc0a640 1 -- 172.21.15.12:0/4090131998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bfe1f080 msgr2=0x55a3bfe21470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:34.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.309+0000 7fc7bcc0a640 1 --2- 172.21.15.12:0/4090131998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bfe1f080 0x55a3bfe21470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.309+0000 7fc7bcc0a640 1 -- 172.21.15.12:0/4090131998 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3bfb527d0 con 0x55a3bfe10e90 2024-04-22T13:33:34.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.309+0000 7fc7b7fff640 1 --2- 172.21.15.12:0/4090131998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3bfe117b0 0x55a3bfe1eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:33:34.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.309+0000 7fc7b57fa640 1 -- 172.21.15.12:0/944789037 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7fc7ac0f2950 con 0x7fc7b8101a40 2024-04-22T13:33:34.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.309+0000 7fc7bcc0a640 1 --2- 172.21.15.12:0/944789037 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc7a00a3d20 0x7fc7a00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.309+0000 7fc7bcc0a640 1 --2- 172.21.15.12:0/4090131998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3bfe10e90 0x55a3bfe11270 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000f390 tx=0x7fc7b000f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4099c03f5a0f19d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:34.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.309+0000 7fc79e7fc640 1 -- 172.21.15.12:0/4090131998 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b001a610 con 0x55a3bfe10e90 2024-04-22T13:33:34.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.309+0000 7fc79e7fc640 1 -- 172.21.15.12:0/4090131998 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7b001ac60 con 0x55a3bfe10e90 2024-04-22T13:33:34.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.309+0000 7fc79e7fc640 1 -- 172.21.15.12:0/4090131998 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b000d950 con 0x55a3bfe10e90 2024-04-22T13:33:34.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.310+0000 7fc7c1263740 1 -- 172.21.15.12:0/4090131998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3bfe10e90 msgr2=0x55a3bfe11270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:34.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.310+0000 7fc7c1263740 1 --2- 172.21.15.12:0/4090131998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3bfe10e90 0x55a3bfe11270 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000f390 tx=0x7fc7b000f850 comp rx=0 tx=0).stop 2024-04-22T13:33:34.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.310+0000 7fc7bcc0a640 1 --2- 172.21.15.12:0/944789037 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc7a00a3d20 0x7fc7a00a61e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0020000 tx=0x7fc7b001d000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:34.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.310+0000 7fc7c1263740 1 -- 172.21.15.12:0/4090131998 shutdown_connections 2024-04-22T13:33:34.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.310+0000 7fc7c1263740 1 --2- 172.21.15.12:0/4090131998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bfe1f080 0x55a3bfe21470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.310+0000 7fc7c1263740 1 --2- 172.21.15.12:0/4090131998 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3bfe117b0 0x55a3bfe1eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.310+0000 7fc7c1263740 1 --2- 172.21.15.12:0/4090131998 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3bfe10e90 0x55a3bfe11270 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.310+0000 7fc7c1263740 1 -- 172.21.15.12:0/4090131998 >> 172.21.15.12:0/4090131998 conn(0x55a3bfe0bcb0 msgr2=0x55a3bfe0c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:34.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.310+0000 7fc7c1263740 1 -- 172.21.15.12:0/4090131998 shutdown_connections 2024-04-22T13:33:34.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.310+0000 7fc7c1263740 1 -- 172.21.15.12:0/4090131998 wait complete. 2024-04-22T13:33:34.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.310+0000 7fc7c1263740 1 -- start start 2024-04-22T13:33:34.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.310+0000 7fc7c1263740 1 Processor -- start 2024-04-22T13:33:34.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.311+0000 7fc7c1263740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bfe10e90 0x55a3bfe1bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.311+0000 7fc7c1263740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3bfe117b0 0x55a3bfe1c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.311+0000 7fc7c1263740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3bfe1f080 0x55a3bfe1ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.311+0000 7fc7c1263740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a3bfe21cb0 con 0x55a3bfe117b0 2024-04-22T13:33:34.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.311+0000 7fc7c1263740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a3bfd2f2f0 con 0x55a3bfe10e90 2024-04-22T13:33:34.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.311+0000 7fc7c1263740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a3bfe21e30 con 0x55a3bfe1f080 2024-04-22T13:33:34.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.311+0000 7fc7c1263740 10 client.0 _ll_register_callbacks cb 0x55a3bfd3c580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:34.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.311+0000 7fc7b77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3bfe117b0 0x55a3bfe1c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.311+0000 7fc7b77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3bfe117b0 0x55a3bfe1c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60814/0 (socket says 172.21.15.12:60814) 2024-04-22T13:33:34.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.311+0000 7fc7b77fe640 1 -- 172.21.15.12:0/266636784 learned_addr learned my addr 172.21.15.12:0/266636784 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:34.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.311+0000 7fc7bcc0a640 1 --2- 172.21.15.12:0/266636784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3bfe1f080 0x55a3bfe1ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.311+0000 7fc7b7fff640 1 --2- 172.21.15.12:0/266636784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bfe10e90 0x55a3bfe1bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.311+0000 7fc7b77fe640 1 -- 172.21.15.12:0/266636784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3bfe1f080 msgr2=0x55a3bfe1ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:34.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.311+0000 7fc7b77fe640 1 --2- 172.21.15.12:0/266636784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3bfe1f080 0x55a3bfe1ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.311+0000 7fc7b77fe640 1 -- 172.21.15.12:0/266636784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bfe10e90 msgr2=0x55a3bfe1bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:34.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.311+0000 7fc7b77fe640 1 --2- 172.21.15.12:0/266636784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bfe10e90 0x55a3bfe1bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.311+0000 7fc7b77fe640 1 -- 172.21.15.12:0/266636784 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3bfa883f0 con 0x55a3bfe117b0 2024-04-22T13:33:34.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.311+0000 7fc7b77fe640 1 --2- 172.21.15.12:0/266636784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3bfe117b0 0x55a3bfe1c510 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac0be720 tx=0x7fc7ac0be630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a790497ac551a5fa server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:34.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.311+0000 7fc7977fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:33:34.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.311+0000 7fc7977fe640 1 -- 172.21.15.12:0/266636784 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac0bf120 con 0x55a3bfe117b0 2024-04-22T13:33:34.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.311+0000 7fc7977fe640 1 -- 172.21.15.12:0/266636784 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7fc7ac0054d0 con 0x55a3bfe117b0 2024-04-22T13:33:34.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.311+0000 7fc7c1263740 1 -- 172.21.15.12:0/266636784 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7b001ac60 con 0x55a3bfe117b0 2024-04-22T13:33:34.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.312+0000 7fc7977fe640 1 -- 172.21.15.12:0/266636784 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc7ac0bea20 con 0x55a3bfe117b0 2024-04-22T13:33:34.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.312+0000 7fc7c1263740 10 client.18933 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:34.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.312+0000 7fc7c1263740 1 -- 172.21.15.12:0/266636784 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a3bfa480f0 con 0x55a3bfe117b0 2024-04-22T13:33:34.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.312+0000 7fc7977fe640 1 -- 172.21.15.12:0/266636784 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7b001ac60 con 0x55a3bfe117b0 2024-04-22T13:33:34.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.312+0000 7fc7977fe640 1 -- 172.21.15.12:0/266636784 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc7ac0b9160 con 0x55a3bfe117b0 2024-04-22T13:33:34.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.312+0000 7fc7c1263740 10 client.18933 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:34.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.312+0000 7fc7c1263740 10 client.18933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:33:34.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.312+0000 7fc7c1263740 20 client.18933 populate_metadata read hostname 'smithi012' 2024-04-22T13:33:34.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.312+0000 7fc7c1263740 1 --2- 172.21.15.12:0/266636784 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55a3bfe2ee60 0x55a3bfe31280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.312+0000 7fc7c1263740 4 client.18933 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:33:34.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.312+0000 7fc7c1263740 1 -- 172.21.15.12:0/266636784 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3bfd83c00 con 0x55a3bfe2ee60 2024-04-22T13:33:34.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.312+0000 7fc7b7fff640 1 --2- 172.21.15.12:0/266636784 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55a3bfe2ee60 0x55a3bfe31280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.313+0000 7fc7977fe640 10 client.18933 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:33:34.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.313+0000 7fc7b7fff640 1 --2- 172.21.15.12:0/266636784 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55a3bfe2ee60 0x55a3bfe31280 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=440fa87fafc13fc4 server_cookie=b4df579e1267f464 in_seq=0 out_seq=0 2024-04-22T13:33:34.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.313+0000 7fc7977fe640 1 -- 172.21.15.12:0/266636784 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a3bfd83c00 con 0x55a3bfe2ee60 2024-04-22T13:33:34.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.313+0000 7fc7977fe640 10 client.18933 handle_command_reply: tid=0 2024-04-22T13:33:34.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.313+0000 7fc7c1263740 1 client.18933 shutdown 2024-04-22T13:33:34.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.314+0000 7fc7c1263740 1 -- 172.21.15.12:0/266636784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3bfe117b0 msgr2=0x55a3bfe1c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:34.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.314+0000 7fc7c1263740 1 --2- 172.21.15.12:0/266636784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3bfe117b0 0x55a3bfe1c510 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac0be720 tx=0x7fc7ac0be630 comp rx=0 tx=0).stop 2024-04-22T13:33:34.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.314+0000 7fc7c1263740 1 -- 172.21.15.12:0/266636784 shutdown_connections 2024-04-22T13:33:34.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.314+0000 7fc7c1263740 1 --2- 172.21.15.12:0/266636784 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55a3bfe2ee60 0x55a3bfe31280 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.314+0000 7fc7c1263740 1 --2- 172.21.15.12:0/266636784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3bfe1f080 0x55a3bfe1ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.314+0000 7fc7c1263740 1 --2- 172.21.15.12:0/266636784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3bfe117b0 0x55a3bfe1c510 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.314+0000 7fc7c1263740 1 --2- 172.21.15.12:0/266636784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bfe10e90 0x55a3bfe1bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.314+0000 7fc7977fe640 0 client.18933 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:33:34.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.314+0000 7fc7c1263740 1 -- 172.21.15.12:0/266636784 >> 172.21.15.12:0/266636784 conn(0x55a3bfe0bcb0 msgr2=0x55a3bfe0da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:34.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.314+0000 7fc7c1263740 1 -- 172.21.15.12:0/266636784 shutdown_connections 2024-04-22T13:33:34.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.314+0000 7fc7c1263740 1 -- 172.21.15.12:0/266636784 wait complete. 2024-04-22T13:33:34.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.314+0000 7fc7c1263740 20 client.18933 trim_cache size 0 max 16384 2024-04-22T13:33:34.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.331+0000 7fc7c1263740 1 Processor -- start 2024-04-22T13:33:34.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.332+0000 7fc7c1263740 1 -- start start 2024-04-22T13:33:34.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.332+0000 7fc7c1263740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3bfe10e90 0x55a3bfe1e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.332+0000 7fc7c1263740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bfe117b0 0x55a3bfe2abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.332+0000 7fc7c1263740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3bfe1f080 0x55a3bfe19b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.332+0000 7fc7c1263740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b81ab7f0 con 0x55a3bfe1f080 2024-04-22T13:33:34.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.332+0000 7fc7c1263740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b81ad330 con 0x55a3bfe117b0 2024-04-22T13:33:34.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.332+0000 7fc7c1263740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b81abec0 con 0x55a3bfe10e90 2024-04-22T13:33:34.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.332+0000 7fc7bcc0a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3bfe1f080 0x55a3bfe19b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.332+0000 7fc7bcc0a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3bfe1f080 0x55a3bfe19b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60816/0 (socket says 172.21.15.12:60816) 2024-04-22T13:33:34.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.332+0000 7fc7bcc0a640 1 -- 172.21.15.12:0/2889233008 learned_addr learned my addr 172.21.15.12:0/2889233008 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:34.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.332+0000 7fc7b7fff640 1 --2- 172.21.15.12:0/2889233008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3bfe10e90 0x55a3bfe1e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.332+0000 7fc7b77fe640 1 --2- 172.21.15.12:0/2889233008 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bfe117b0 0x55a3bfe2abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.332+0000 7fc7bcc0a640 1 -- 172.21.15.12:0/2889233008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3bfe10e90 msgr2=0x55a3bfe1e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:34.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.332+0000 7fc7bcc0a640 1 --2- 172.21.15.12:0/2889233008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3bfe10e90 0x55a3bfe1e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.332+0000 7fc7bcc0a640 1 -- 172.21.15.12:0/2889233008 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bfe117b0 msgr2=0x55a3bfe2abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:34.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.332+0000 7fc7bcc0a640 1 --2- 172.21.15.12:0/2889233008 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bfe117b0 0x55a3bfe2abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.332+0000 7fc7bcc0a640 1 -- 172.21.15.12:0/2889233008 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3bfe31900 con 0x55a3bfe1f080 2024-04-22T13:33:34.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.332+0000 7fc7b77fe640 1 --2- 172.21.15.12:0/2889233008 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bfe117b0 0x55a3bfe2abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:33:34.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.332+0000 7fc7bcc0a640 1 --2- 172.21.15.12:0/2889233008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3bfe1f080 0x55a3bfe19b10 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000f360 tx=0x7fc7b0027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0a2a17a27e49a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:34.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.332+0000 7fc79dffb640 1 -- 172.21.15.12:0/2889233008 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0009710 con 0x55a3bfe1f080 2024-04-22T13:33:34.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.333+0000 7fc79dffb640 1 -- 172.21.15.12:0/2889233008 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7b0034070 con 0x55a3bfe1f080 2024-04-22T13:33:34.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.333+0000 7fc79dffb640 1 -- 172.21.15.12:0/2889233008 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b00256a0 con 0x55a3bfe1f080 2024-04-22T13:33:34.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.333+0000 7fc7c1263740 1 -- 172.21.15.12:0/2889233008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3bfe1f080 msgr2=0x55a3bfe19b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:34.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.333+0000 7fc7c1263740 1 --2- 172.21.15.12:0/2889233008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3bfe1f080 0x55a3bfe19b10 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000f360 tx=0x7fc7b0027000 comp rx=0 tx=0).stop 2024-04-22T13:33:34.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.333+0000 7fc7c1263740 1 -- 172.21.15.12:0/2889233008 shutdown_connections 2024-04-22T13:33:34.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.333+0000 7fc7c1263740 1 --2- 172.21.15.12:0/2889233008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3bfe1f080 0x55a3bfe19b10 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.333+0000 7fc7c1263740 1 --2- 172.21.15.12:0/2889233008 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bfe117b0 0x55a3bfe2abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.333+0000 7fc7c1263740 1 --2- 172.21.15.12:0/2889233008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3bfe10e90 0x55a3bfe1e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.333+0000 7fc7c1263740 1 -- 172.21.15.12:0/2889233008 >> 172.21.15.12:0/2889233008 conn(0x55a3bfe0bcb0 msgr2=0x55a3bfe0a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:34.335 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.333+0000 7fc7c1263740 1 -- 172.21.15.12:0/2889233008 shutdown_connections 2024-04-22T13:33:34.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.333+0000 7fc7c1263740 1 -- 172.21.15.12:0/2889233008 wait complete. 2024-04-22T13:33:34.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.333+0000 7fc7c1263740 1 -- start start 2024-04-22T13:33:34.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.334+0000 7fc7c1263740 1 Processor -- start 2024-04-22T13:33:34.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.334+0000 7fc7c1263740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3bfe10e90 0x55a3bfe364a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.334+0000 7fc7c1263740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3bfe117b0 0x55a3bfe369e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.334+0000 7fc7c1263740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bfe1f080 0x55a3bfe256a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.334+0000 7fc7c1263740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b81acc60 con 0x55a3bfe117b0 2024-04-22T13:33:34.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.334+0000 7fc7c1263740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b81ac590 con 0x55a3bfe1f080 2024-04-22T13:33:34.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.334+0000 7fc7c1263740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b81a97f0 con 0x55a3bfe10e90 2024-04-22T13:33:34.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.334+0000 7fc7c1263740 10 client.0 _ll_register_callbacks cb 0x7fc7ac0057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:33:34.336 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.334+0000 7fc7b77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3bfe117b0 0x55a3bfe369e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.334+0000 7fc7b77fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3bfe117b0 0x55a3bfe369e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:60818/0 (socket says 172.21.15.12:60818) 2024-04-22T13:33:34.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.334+0000 7fc7b77fe640 1 -- 172.21.15.12:0/800438197 learned_addr learned my addr 172.21.15.12:0/800438197 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:33:34.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.334+0000 7fc7bcc0a640 1 --2- 172.21.15.12:0/800438197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bfe1f080 0x55a3bfe256a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.334+0000 7fc7b7fff640 1 --2- 172.21.15.12:0/800438197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3bfe10e90 0x55a3bfe364a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.334+0000 7fc7b77fe640 1 -- 172.21.15.12:0/800438197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3bfe10e90 msgr2=0x55a3bfe364a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:34.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.334+0000 7fc7b77fe640 1 --2- 172.21.15.12:0/800438197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3bfe10e90 0x55a3bfe364a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.334+0000 7fc7b77fe640 1 -- 172.21.15.12:0/800438197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bfe1f080 msgr2=0x55a3bfe256a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:33:34.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.334+0000 7fc7b77fe640 1 --2- 172.21.15.12:0/800438197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bfe1f080 0x55a3bfe256a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.334+0000 7fc7b77fe640 1 -- 172.21.15.12:0/800438197 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3bfe21b90 con 0x55a3bfe117b0 2024-04-22T13:33:34.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.335+0000 7fc7b77fe640 1 --2- 172.21.15.12:0/800438197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3bfe117b0 0x55a3bfe369e0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac0f06c0 tx=0x7fc7ac02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af6ad1145debb57 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:33:34.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.335+0000 7fc7977fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:33:34.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.335+0000 7fc7977fe640 1 -- 172.21.15.12:0/800438197 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac0bf120 con 0x55a3bfe117b0 2024-04-22T13:33:34.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.335+0000 7fc7977fe640 1 -- 172.21.15.12:0/800438197 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7fc7ac0c7200 con 0x55a3bfe117b0 2024-04-22T13:33:34.337 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.335+0000 7fc7c1263740 1 -- 172.21.15.12:0/800438197 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7b0034070 con 0x55a3bfe117b0 2024-04-22T13:33:34.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.335+0000 7fc7977fe640 1 -- 172.21.15.12:0/800438197 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc7ac0c7690 con 0x55a3bfe117b0 2024-04-22T13:33:34.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.335+0000 7fc7c1263740 10 client.18939 fetch_fsmap learned FSMap version 28 2024-04-22T13:33:34.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.335+0000 7fc7c1263740 1 -- 172.21.15.12:0/800438197 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a3bfe21b90 con 0x55a3bfe117b0 2024-04-22T13:33:34.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.335+0000 7fc7977fe640 1 -- 172.21.15.12:0/800438197 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7b0034070 con 0x55a3bfe117b0 2024-04-22T13:33:34.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.335+0000 7fc7977fe640 1 -- 172.21.15.12:0/800438197 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc7ac0f2ce0 con 0x55a3bfe117b0 2024-04-22T13:33:34.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.335+0000 7fc7c1263740 10 client.18939 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:33:34.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.335+0000 7fc7c1263740 10 client.18939 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:33:34.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.335+0000 7fc7c1263740 20 client.18939 populate_metadata read hostname 'smithi012' 2024-04-22T13:33:34.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.335+0000 7fc7c1263740 1 --2- 172.21.15.12:0/800438197 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55a3bfe95d30 0x55a3bfe98150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:33:34.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.335+0000 7fc7c1263740 4 client.18939 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:33:34.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.335+0000 7fc7c1263740 1 -- 172.21.15.12:0/800438197 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a3bfd7da70 con 0x55a3bfe95d30 2024-04-22T13:33:34.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.336+0000 7fc7b7fff640 1 --2- 172.21.15.12:0/800438197 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55a3bfe95d30 0x55a3bfe98150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:33:34.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.336+0000 7fc7977fe640 10 client.18939 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:33:34.338 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.336+0000 7fc7b7fff640 1 --2- 172.21.15.12:0/800438197 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55a3bfe95d30 0x55a3bfe98150 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae7cc06cc68a26ae server_cookie=4dab2d980bd14c34 in_seq=0 out_seq=0 2024-04-22T13:33:34.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.336+0000 7fc7977fe640 1 -- 172.21.15.12:0/800438197 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a3bfd7da70 con 0x55a3bfe95d30 2024-04-22T13:33:34.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.336+0000 7fc7977fe640 10 client.18939 handle_command_reply: tid=0 2024-04-22T13:33:34.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.336+0000 7fc7c1263740 1 client.18939 shutdown 2024-04-22T13:33:34.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.336+0000 7fc7c1263740 1 -- 172.21.15.12:0/800438197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3bfe117b0 msgr2=0x55a3bfe369e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:34.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.336+0000 7fc7c1263740 1 --2- 172.21.15.12:0/800438197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3bfe117b0 0x55a3bfe369e0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac0f06c0 tx=0x7fc7ac02bc30 comp rx=0 tx=0).stop 2024-04-22T13:33:34.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.336+0000 7fc7c1263740 1 -- 172.21.15.12:0/800438197 shutdown_connections 2024-04-22T13:33:34.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.336+0000 7fc7c1263740 1 --2- 172.21.15.12:0/800438197 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55a3bfe95d30 0x55a3bfe98150 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.336+0000 7fc7c1263740 1 --2- 172.21.15.12:0/800438197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3bfe1f080 0x55a3bfe256a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.336+0000 7fc7c1263740 1 --2- 172.21.15.12:0/800438197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a3bfe117b0 0x55a3bfe369e0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.336+0000 7fc7c1263740 1 --2- 172.21.15.12:0/800438197 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a3bfe10e90 0x55a3bfe364a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.336+0000 7fc7c1263740 1 -- 172.21.15.12:0/800438197 >> 172.21.15.12:0/800438197 conn(0x55a3bfe0bcb0 msgr2=0x55a3bfe1c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:34.339 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.336+0000 7fc7977fe640 0 client.18939 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:33:34.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.337+0000 7fc7c1263740 1 -- 172.21.15.12:0/800438197 shutdown_connections 2024-04-22T13:33:34.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.337+0000 7fc7c1263740 1 -- 172.21.15.12:0/800438197 wait complete. 2024-04-22T13:33:34.340 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.337+0000 7fc7c1263740 20 client.18939 trim_cache size 0 max 16384 2024-04-22T13:33:34.340 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:33:34.340 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:33:34.340 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-04-22T13:33:34.340 INFO:teuthology.orchestra.run.smithi012.stdout: "ba11c3e2-f7d1-4574-9a4a-81c595744c9f": { 2024-04-22T13:33:34.340 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-04-22T13:33:34.340 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "ba11c3e2-f7d1-4574-9a4a-81c595744c9f", 2024-04-22T13:33:34.340 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-04-22T13:33:34.341 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:33:34.341 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:33:34.341 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:33:34.341 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.337+0000 7fc7be66d640 1 -- 172.21.15.12:0/944789037 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc7a00a3d20 msgr2=0x7fc7a00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:34.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.337+0000 7fc7be66d640 1 --2- 172.21.15.12:0/944789037 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc7a00a3d20 0x7fc7a00a61e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0020000 tx=0x7fc7b001d000 comp rx=0 tx=0).stop 2024-04-22T13:33:34.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.337+0000 7fc7be66d640 1 -- 172.21.15.12:0/944789037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7b8101a40 msgr2=0x7fc7b8197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:33:34.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.337+0000 7fc7be66d640 1 --2- 172.21.15.12:0/944789037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7b8101a40 0x7fc7b8197bd0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac005940 tx=0x7fc7ac02c7e0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.337+0000 7fc7be66d640 1 -- 172.21.15.12:0/944789037 shutdown_connections 2024-04-22T13:33:34.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.337+0000 7fc7be66d640 1 --2- 172.21.15.12:0/944789037 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc7a00a3d20 0x7fc7a00a61e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.337+0000 7fc7be66d640 1 --2- 172.21.15.12:0/944789037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7b8105dd0 0x7fc7b819bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.337+0000 7fc7be66d640 1 --2- 172.21.15.12:0/944789037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc7b8101a40 0x7fc7b8197bd0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.337+0000 7fc7be66d640 1 --2- 172.21.15.12:0/944789037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc7b81010a0 0x7fc7b8197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:33:34.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.337+0000 7fc7be66d640 1 -- 172.21.15.12:0/944789037 >> 172.21.15.12:0/944789037 conn(0x7fc7b8077c40 msgr2=0x7fc7b80fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:33:34.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.337+0000 7fc7be66d640 1 -- 172.21.15.12:0/944789037 shutdown_connections 2024-04-22T13:33:34.342 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:33:34.337+0000 7fc7be66d640 1 -- 172.21.15.12:0/944789037 wait complete. 2024-04-22T13:33:34.343 INFO:tasks.cephfs.filesystem:scrub status for tag:ba11c3e2-f7d1-4574-9a4a-81c595744c9f - {'path': '/', 'tag': 'ba11c3e2-f7d1-4574-9a4a-81c595744c9f', 'options': 'recursive,force'} 2024-04-22T13:33:35.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:35 smithi134 ceph-mon[31338]: scrub queued for path: / 2024-04-22T13:33:35.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:35 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:33:35.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:35 smithi134 ceph-mon[31338]: scrub summary: active paths [/] 2024-04-22T13:33:35.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:35 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:33:35.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:35 smithi134 ceph-mon[31338]: pgmap v293: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:35.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:35 smithi134 ceph-mon[31338]: scrub summary: idle 2024-04-22T13:33:35.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:35 smithi111 ceph-mon[31361]: scrub queued for path: / 2024-04-22T13:33:35.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:35 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:33:35.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:35 smithi111 ceph-mon[31361]: scrub summary: active paths [/] 2024-04-22T13:33:35.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:35 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:33:35.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:35 smithi111 ceph-mon[31361]: pgmap v293: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:35.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:35 smithi111 ceph-mon[31361]: scrub summary: idle 2024-04-22T13:33:36.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:35 smithi012 ceph-mon[28265]: scrub queued for path: / 2024-04-22T13:33:36.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:35 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:33:36.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:35 smithi012 ceph-mon[28265]: scrub summary: active paths [/] 2024-04-22T13:33:36.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:35 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:33:36.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:35 smithi012 ceph-mon[28265]: pgmap v293: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:36.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:35 smithi012 ceph-mon[28265]: scrub summary: idle 2024-04-22T13:33:37.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:37 smithi134 ceph-mon[31338]: pgmap v294: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:37.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:37 smithi111 ceph-mon[31361]: pgmap v294: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:38.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:37 smithi012 ceph-mon[28265]: pgmap v294: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:39.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:39 smithi134 ceph-mon[31338]: pgmap v295: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:33:39.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:39 smithi111 ceph-mon[31361]: pgmap v295: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:33:40.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:39 smithi012 ceph-mon[28265]: pgmap v295: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:33:41.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:41 smithi134 ceph-mon[31338]: pgmap v296: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:33:41.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:41 smithi111 ceph-mon[31361]: pgmap v296: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:33:42.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:41 smithi012 ceph-mon[28265]: pgmap v296: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:33:43.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:43 smithi134 ceph-mon[31338]: pgmap v297: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:33:43.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:43 smithi111 ceph-mon[31361]: pgmap v297: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:33:44.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:43 smithi012 ceph-mon[28265]: pgmap v297: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:33:45.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:45 smithi134 ceph-mon[31338]: pgmap v298: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:33:45.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:45 smithi111 ceph-mon[31361]: pgmap v298: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:33:46.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:45 smithi012 ceph-mon[28265]: pgmap v298: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:33:47.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:47 smithi134 ceph-mon[31338]: pgmap v299: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:33:47.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:47 smithi111 ceph-mon[31361]: pgmap v299: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:33:48.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:47 smithi012 ceph-mon[28265]: pgmap v299: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-04-22T13:33:49.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:49 smithi134 ceph-mon[31338]: pgmap v300: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:33:49.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:49 smithi111 ceph-mon[31361]: pgmap v300: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:33:50.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:49 smithi012 ceph-mon[28265]: pgmap v300: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-04-22T13:33:51.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:51 smithi134 ceph-mon[31338]: pgmap v301: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:51.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:51 smithi111 ceph-mon[31361]: pgmap v301: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:52.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:51 smithi012 ceph-mon[28265]: pgmap v301: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:53.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:53 smithi134 ceph-mon[31338]: pgmap v302: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:33:53.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:53 smithi111 ceph-mon[31361]: pgmap v302: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:33:54.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:53 smithi012 ceph-mon[28265]: pgmap v302: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:33:55.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:55 smithi134 ceph-mon[31338]: pgmap v303: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:55.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:55 smithi111 ceph-mon[31361]: pgmap v303: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:56.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:55 smithi012 ceph-mon[28265]: pgmap v303: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:57.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:57 smithi134 ceph-mon[31338]: pgmap v304: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:57.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:57 smithi111 ceph-mon[31361]: pgmap v304: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:58.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:57 smithi012 ceph-mon[28265]: pgmap v304: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:33:59.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:33:59 smithi134 ceph-mon[31338]: pgmap v305: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:33:59.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:33:59 smithi111 ceph-mon[31361]: pgmap v305: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:34:00.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:33:59 smithi012 ceph-mon[28265]: pgmap v305: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:34:01.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:01 smithi134 ceph-mon[31338]: pgmap v306: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:01.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:01 smithi111 ceph-mon[31361]: pgmap v306: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:02.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:01 smithi012 ceph-mon[28265]: pgmap v306: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:03.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:03 smithi134 ceph-mon[31338]: pgmap v307: 97 pgs: 97 active+clean; 2.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-22T13:34:03.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:03 smithi111 ceph-mon[31361]: pgmap v307: 97 pgs: 97 active+clean; 2.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-22T13:34:04.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:03 smithi012 ceph-mon[28265]: pgmap v307: 97 pgs: 97 active+clean; 2.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-22T13:34:04.344 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:34:04.345 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:34:04.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.465+0000 7fac2f58f640 1 -- 172.21.15.12:0/4167476705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac28106150 msgr2=0x7fac2810dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:04.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.465+0000 7fac2f58f640 1 --2- 172.21.15.12:0/4167476705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac28106150 0x7fac2810dd20 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fac24009ca0 tx=0x7fac2402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.465+0000 7fac2f58f640 1 -- 172.21.15.12:0/4167476705 shutdown_connections 2024-04-22T13:34:04.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.465+0000 7fac2f58f640 1 --2- 172.21.15.12:0/4167476705 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac28106150 0x7fac2810dd20 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.465+0000 7fac2f58f640 1 --2- 172.21.15.12:0/4167476705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac28101a30 0x7fac28105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.465+0000 7fac2f58f640 1 --2- 172.21.15.12:0/4167476705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac28101080 0x7fac28101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.465+0000 7fac2f58f640 1 -- 172.21.15.12:0/4167476705 >> 172.21.15.12:0/4167476705 conn(0x7fac28078000 msgr2=0x7fac28078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:04.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.465+0000 7fac2f58f640 1 -- 172.21.15.12:0/4167476705 shutdown_connections 2024-04-22T13:34:04.464 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.465+0000 7fac2f58f640 1 -- 172.21.15.12:0/4167476705 wait complete. 2024-04-22T13:34:04.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.466+0000 7fac2f58f640 1 Processor -- start 2024-04-22T13:34:04.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.466+0000 7fac2f58f640 1 -- start start 2024-04-22T13:34:04.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.466+0000 7fac2f58f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac28101080 0x7fac28199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.466+0000 7fac2f58f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac28106150 0x7fac28199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.466+0000 7fac2f58f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac28199fc0 0x7fac281a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.466+0000 7fac2f58f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fac281110a0 con 0x7fac28101080 2024-04-22T13:34:04.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.466+0000 7fac2f58f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fac28110f20 con 0x7fac28106150 2024-04-22T13:34:04.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.466+0000 7fac2f58f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fac28111220 con 0x7fac28199fc0 2024-04-22T13:34:04.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.466+0000 7fac2d32b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac28101080 0x7fac28199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.466+0000 7fac2d32b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac28101080 0x7fac28199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36458/0 (socket says 172.21.15.12:36458) 2024-04-22T13:34:04.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.466+0000 7fac2d32b640 1 -- 172.21.15.12:0/809553810 learned_addr learned my addr 172.21.15.12:0/809553810 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:04.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.466+0000 7fac2cb2a640 1 --2- 172.21.15.12:0/809553810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac28106150 0x7fac28199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.466+0000 7fac2db2c640 1 --2- 172.21.15.12:0/809553810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac28199fc0 0x7fac281a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.465 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.466+0000 7fac2d32b640 1 -- 172.21.15.12:0/809553810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac28199fc0 msgr2=0x7fac281a45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:04.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.466+0000 7fac2d32b640 1 --2- 172.21.15.12:0/809553810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac28199fc0 0x7fac281a45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.466+0000 7fac2d32b640 1 -- 172.21.15.12:0/809553810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac28106150 msgr2=0x7fac28199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:04.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.466+0000 7fac2d32b640 1 --2- 172.21.15.12:0/809553810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac28106150 0x7fac28199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.466+0000 7fac2d32b640 1 -- 172.21.15.12:0/809553810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac281a4d40 con 0x7fac28101080 2024-04-22T13:34:04.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.467+0000 7fac2d32b640 1 --2- 172.21.15.12:0/809553810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac28101080 0x7fac28199540 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fac2000d950 tx=0x7fac2000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:04.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.467+0000 7fac1e7fc640 1 -- 172.21.15.12:0/809553810 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac20017070 con 0x7fac28101080 2024-04-22T13:34:04.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.467+0000 7fac1e7fc640 1 -- 172.21.15.12:0/809553810 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac20009c30 con 0x7fac28101080 2024-04-22T13:34:04.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.467+0000 7fac2f58f640 1 -- 172.21.15.12:0/809553810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac281a4fd0 con 0x7fac28101080 2024-04-22T13:34:04.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.467+0000 7fac1e7fc640 1 -- 172.21.15.12:0/809553810 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2000f620 con 0x7fac28101080 2024-04-22T13:34:04.466 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.467+0000 7fac2f58f640 1 -- 172.21.15.12:0/809553810 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac281a54f0 con 0x7fac28101080 2024-04-22T13:34:04.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.470+0000 7fac1e7fc640 1 -- 172.21.15.12:0/809553810 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fac20010460 con 0x7fac28101080 2024-04-22T13:34:04.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.470+0000 7fac3217d740 1 Processor -- start 2024-04-22T13:34:04.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.470+0000 7fac3217d740 1 -- start start 2024-04-22T13:34:04.473 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.470+0000 7fac3217d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b65d3d2580 0x55b65d3d2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.470+0000 7fac3217d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b65d3d2ea0 0x55b65d3db540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.471+0000 7fac3217d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b65d3dba80 0x55b65d3dde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.471+0000 7fac3217d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b65d3371e0 con 0x55b65d3d2ea0 2024-04-22T13:34:04.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.471+0000 7fac3217d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b65d2d4390 con 0x55b65d3dba80 2024-04-22T13:34:04.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.471+0000 7fac3217d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b65d3331a0 con 0x55b65d3d2580 2024-04-22T13:34:04.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.471+0000 7fac2d32b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b65d3d2ea0 0x55b65d3db540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.471+0000 7fac2d32b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b65d3d2ea0 0x55b65d3db540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36472/0 (socket says 172.21.15.12:36472) 2024-04-22T13:34:04.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.471+0000 7fac2d32b640 1 -- 172.21.15.12:0/1756519633 learned_addr learned my addr 172.21.15.12:0/1756519633 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:04.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.471+0000 7fac2db2c640 1 --2- 172.21.15.12:0/1756519633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b65d3d2580 0x55b65d3d2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.471+0000 7fac2cb2a640 1 --2- 172.21.15.12:0/1756519633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b65d3dba80 0x55b65d3dde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.471+0000 7fac1e7fc640 1 --2- 172.21.15.12:0/809553810 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fac140a3d20 0x7fac140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.471+0000 7fac2d32b640 1 -- 172.21.15.12:0/1756519633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b65d3d2580 msgr2=0x55b65d3d2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:04.474 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.471+0000 7fac2d32b640 1 --2- 172.21.15.12:0/1756519633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b65d3d2580 0x55b65d3d2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.471+0000 7fac2d32b640 1 -- 172.21.15.12:0/1756519633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b65d3dba80 msgr2=0x55b65d3dde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:04.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.471+0000 7fac2d32b640 1 --2- 172.21.15.12:0/1756519633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b65d3dba80 0x55b65d3dde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.471+0000 7fac2d32b640 1 -- 172.21.15.12:0/1756519633 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b65d1137d0 con 0x55b65d3d2ea0 2024-04-22T13:34:04.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.471+0000 7fac2cb2a640 1 --2- 172.21.15.12:0/1756519633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b65d3dba80 0x55b65d3dde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:04.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.471+0000 7fac2db2c640 1 --2- 172.21.15.12:0/809553810 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fac140a3d20 0x7fac140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.471+0000 7fac1e7fc640 1 -- 172.21.15.12:0/809553810 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7fac200d0350 con 0x7fac28101080 2024-04-22T13:34:04.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.471+0000 7fac2d32b640 1 --2- 172.21.15.12:0/1756519633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b65d3d2ea0 0x55b65d3db540 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fac200cda60 tx=0x7fac20066a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7760cf3f0c46a80 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:04.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.472+0000 7fac137fe640 1 -- 172.21.15.12:0/1756519633 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2009bf00 con 0x55b65d3d2ea0 2024-04-22T13:34:04.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.472+0000 7fac137fe640 1 -- 172.21.15.12:0/1756519633 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac2006d0c0 con 0x55b65d3d2ea0 2024-04-22T13:34:04.475 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.472+0000 7fac2db2c640 1 --2- 172.21.15.12:0/809553810 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fac140a3d20 0x7fac140a61e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fac24005e30 tx=0x7fac2402e040 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:04.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.472+0000 7fac137fe640 1 -- 172.21.15.12:0/1756519633 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2009bd20 con 0x55b65d3d2ea0 2024-04-22T13:34:04.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.472+0000 7fac3217d740 1 -- 172.21.15.12:0/1756519633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b65d3d2ea0 msgr2=0x55b65d3db540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:04.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.472+0000 7fac3217d740 1 --2- 172.21.15.12:0/1756519633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b65d3d2ea0 0x55b65d3db540 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fac200cda60 tx=0x7fac20066a70 comp rx=0 tx=0).stop 2024-04-22T13:34:04.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.472+0000 7fac3217d740 1 -- 172.21.15.12:0/1756519633 shutdown_connections 2024-04-22T13:34:04.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.472+0000 7fac3217d740 1 --2- 172.21.15.12:0/1756519633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b65d3dba80 0x55b65d3dde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.472+0000 7fac3217d740 1 --2- 172.21.15.12:0/1756519633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b65d3d2ea0 0x55b65d3db540 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.472+0000 7fac3217d740 1 --2- 172.21.15.12:0/1756519633 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b65d3d2580 0x55b65d3d2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.472+0000 7fac3217d740 1 -- 172.21.15.12:0/1756519633 >> 172.21.15.12:0/1756519633 conn(0x55b65d3cccb0 msgr2=0x55b65d3cd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:04.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.472+0000 7fac3217d740 1 -- 172.21.15.12:0/1756519633 shutdown_connections 2024-04-22T13:34:04.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.472+0000 7fac3217d740 1 -- 172.21.15.12:0/1756519633 wait complete. 2024-04-22T13:34:04.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.473+0000 7fac3217d740 1 -- start start 2024-04-22T13:34:04.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.473+0000 7fac3217d740 1 Processor -- start 2024-04-22T13:34:04.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.473+0000 7fac3217d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b65d3d2580 0x55b65d3d8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.473+0000 7fac3217d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b65d3d2ea0 0x55b65d3d8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.476 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.473+0000 7fac3217d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b65d3dba80 0x55b65d3d94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.473+0000 7fac3217d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b65d3de7b0 con 0x55b65d3d2580 2024-04-22T13:34:04.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.473+0000 7fac3217d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b65d2f02f0 con 0x55b65d3d2ea0 2024-04-22T13:34:04.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.473+0000 7fac3217d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b65d3de930 con 0x55b65d3dba80 2024-04-22T13:34:04.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.473+0000 7fac3217d740 10 client.0 _ll_register_callbacks cb 0x55b65d2fd580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:04.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.473+0000 7fac2d32b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b65d3d2580 0x55b65d3d8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.473+0000 7fac2d32b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b65d3d2580 0x55b65d3d8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36474/0 (socket says 172.21.15.12:36474) 2024-04-22T13:34:04.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.473+0000 7fac2d32b640 1 -- 172.21.15.12:0/2078872737 learned_addr learned my addr 172.21.15.12:0/2078872737 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:04.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.473+0000 7fac2db2c640 1 --2- 172.21.15.12:0/2078872737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b65d3dba80 0x55b65d3d94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.474+0000 7fac2cb2a640 1 --2- 172.21.15.12:0/2078872737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b65d3d2ea0 0x55b65d3d8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.474+0000 7fac2d32b640 1 -- 172.21.15.12:0/2078872737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b65d3dba80 msgr2=0x55b65d3d94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:04.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.474+0000 7fac2d32b640 1 --2- 172.21.15.12:0/2078872737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b65d3dba80 0x55b65d3d94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.474+0000 7fac2d32b640 1 -- 172.21.15.12:0/2078872737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b65d3d2ea0 msgr2=0x55b65d3d8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:04.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.474+0000 7fac2d32b640 1 --2- 172.21.15.12:0/2078872737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b65d3d2ea0 0x55b65d3d8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.474+0000 7fac2d32b640 1 -- 172.21.15.12:0/2078872737 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b65d0493f0 con 0x55b65d3d2580 2024-04-22T13:34:04.477 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.474+0000 7fac2db2c640 1 --2- 172.21.15.12:0/2078872737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b65d3dba80 0x55b65d3d94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:04.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.474+0000 7fac2d32b640 1 --2- 172.21.15.12:0/2078872737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b65d3d2580 0x55b65d3d8a50 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fac2006d090 tx=0x7fac20010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694ead40336788c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:04.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.474+0000 7fac10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:34:04.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.474+0000 7fac10ff9640 1 -- 172.21.15.12:0/2078872737 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac20020070 con 0x55b65d3d2580 2024-04-22T13:34:04.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.474+0000 7fac10ff9640 1 -- 172.21.15.12:0/2078872737 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7fac200cdd40 con 0x55b65d3d2580 2024-04-22T13:34:04.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.474+0000 7fac3217d740 1 -- 172.21.15.12:0/2078872737 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b65d344c00 con 0x55b65d3d2580 2024-04-22T13:34:04.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.474+0000 7fac10ff9640 1 -- 172.21.15.12:0/2078872737 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fac2009c190 con 0x55b65d3d2580 2024-04-22T13:34:04.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.474+0000 7fac3217d740 10 client.18951 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:04.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.474+0000 7fac3217d740 1 -- 172.21.15.12:0/2078872737 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b65d0090f0 con 0x55b65d3d2580 2024-04-22T13:34:04.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.474+0000 7fac10ff9640 1 -- 172.21.15.12:0/2078872737 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b65d344c00 con 0x55b65d3d2580 2024-04-22T13:34:04.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.475+0000 7fac10ff9640 1 -- 172.21.15.12:0/2078872737 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fac20098a30 con 0x55b65d3d2580 2024-04-22T13:34:04.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.475+0000 7fac3217d740 10 client.18951 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:04.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.475+0000 7fac3217d740 10 client.18951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:34:04.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.475+0000 7fac3217d740 20 client.18951 populate_metadata read hostname 'smithi012' 2024-04-22T13:34:04.478 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.475+0000 7fac3217d740 1 --2- 172.21.15.12:0/2078872737 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b65d3eb810 0x55b65d3edc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.475+0000 7fac3217d740 4 client.18951 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:34:04.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.475+0000 7fac3217d740 1 -- 172.21.15.12:0/2078872737 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b65d33ea70 con 0x55b65d3eb810 2024-04-22T13:34:04.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.475+0000 7fac2cb2a640 1 --2- 172.21.15.12:0/2078872737 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b65d3eb810 0x55b65d3edc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.475+0000 7fac10ff9640 10 client.18951 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:34:04.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.475+0000 7fac2cb2a640 1 --2- 172.21.15.12:0/2078872737 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b65d3eb810 0x55b65d3edc00 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2de5db92e3ce63df server_cookie=ec82f55361686960 in_seq=0 out_seq=0 2024-04-22T13:34:04.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.476+0000 7fac10ff9640 1 -- 172.21.15.12:0/2078872737 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b65d33ea70 con 0x55b65d3eb810 2024-04-22T13:34:04.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.476+0000 7fac10ff9640 10 client.18951 handle_command_reply: tid=0 2024-04-22T13:34:04.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.476+0000 7fac3217d740 1 client.18951 shutdown 2024-04-22T13:34:04.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.476+0000 7fac3217d740 1 -- 172.21.15.12:0/2078872737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b65d3d2580 msgr2=0x55b65d3d8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:04.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.476+0000 7fac3217d740 1 --2- 172.21.15.12:0/2078872737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b65d3d2580 0x55b65d3d8a50 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fac2006d090 tx=0x7fac20010cb0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.476+0000 7fac3217d740 1 -- 172.21.15.12:0/2078872737 shutdown_connections 2024-04-22T13:34:04.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.476+0000 7fac3217d740 1 --2- 172.21.15.12:0/2078872737 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b65d3eb810 0x55b65d3edc00 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.476+0000 7fac3217d740 1 --2- 172.21.15.12:0/2078872737 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b65d3dba80 0x55b65d3d94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.476+0000 7fac3217d740 1 --2- 172.21.15.12:0/2078872737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b65d3d2ea0 0x55b65d3d8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.479 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.476+0000 7fac10ff9640 0 client.18951 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:34:04.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.476+0000 7fac3217d740 1 --2- 172.21.15.12:0/2078872737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b65d3d2580 0x55b65d3d8a50 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.476+0000 7fac3217d740 1 -- 172.21.15.12:0/2078872737 >> 172.21.15.12:0/2078872737 conn(0x55b65d3cccb0 msgr2=0x55b65d3cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:04.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.477+0000 7fac3217d740 1 -- 172.21.15.12:0/2078872737 shutdown_connections 2024-04-22T13:34:04.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.477+0000 7fac3217d740 1 -- 172.21.15.12:0/2078872737 wait complete. 2024-04-22T13:34:04.480 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.477+0000 7fac3217d740 20 client.18951 trim_cache size 0 max 16384 2024-04-22T13:34:04.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.494+0000 7fac3217d740 1 Processor -- start 2024-04-22T13:34:04.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.494+0000 7fac3217d740 1 -- start start 2024-04-22T13:34:04.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.495+0000 7fac3217d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b65d3d2580 0x55b65d3db180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.495+0000 7fac3217d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b65d3d2ea0 0x55b65d3e72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.495+0000 7fac3217d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b65d3dba80 0x55b65d3f0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.496 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.495+0000 7fac3217d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fac281adbc0 con 0x55b65d3d2580 2024-04-22T13:34:04.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.495+0000 7fac3217d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fac281af700 con 0x55b65d3dba80 2024-04-22T13:34:04.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.495+0000 7fac3217d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fac281ae290 con 0x55b65d3d2ea0 2024-04-22T13:34:04.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.495+0000 7fac2d32b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b65d3d2580 0x55b65d3db180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.495+0000 7fac2d32b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b65d3d2580 0x55b65d3db180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36482/0 (socket says 172.21.15.12:36482) 2024-04-22T13:34:04.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.495+0000 7fac2d32b640 1 -- 172.21.15.12:0/2048953495 learned_addr learned my addr 172.21.15.12:0/2048953495 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:04.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.495+0000 7fac2db2c640 1 --2- 172.21.15.12:0/2048953495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b65d3dba80 0x55b65d3f0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.495+0000 7fac2cb2a640 1 --2- 172.21.15.12:0/2048953495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b65d3d2ea0 0x55b65d3e72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.495+0000 7fac2d32b640 1 -- 172.21.15.12:0/2048953495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b65d3d2ea0 msgr2=0x55b65d3e72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:04.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.495+0000 7fac2d32b640 1 --2- 172.21.15.12:0/2048953495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b65d3d2ea0 0x55b65d3e72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.495+0000 7fac2d32b640 1 -- 172.21.15.12:0/2048953495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b65d3dba80 msgr2=0x55b65d3f0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:04.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.495+0000 7fac2d32b640 1 --2- 172.21.15.12:0/2048953495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b65d3dba80 0x55b65d3f0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.495+0000 7fac2d32b640 1 -- 172.21.15.12:0/2048953495 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b65d3dde70 con 0x55b65d3d2580 2024-04-22T13:34:04.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.495+0000 7fac2cb2a640 1 --2- 172.21.15.12:0/2048953495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b65d3d2ea0 0x55b65d3e72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:04.497 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.495+0000 7fac2d32b640 1 --2- 172.21.15.12:0/2048953495 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b65d3d2580 0x55b65d3db180 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fac20005c40 tx=0x7fac2009a8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319fd68822fc5183 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:04.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.495+0000 7fac12ffd640 1 -- 172.21.15.12:0/2048953495 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac20014070 con 0x55b65d3d2580 2024-04-22T13:34:04.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.495+0000 7fac12ffd640 1 -- 172.21.15.12:0/2048953495 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac20009de0 con 0x55b65d3d2580 2024-04-22T13:34:04.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.495+0000 7fac12ffd640 1 -- 172.21.15.12:0/2048953495 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac20020070 con 0x55b65d3d2580 2024-04-22T13:34:04.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.496+0000 7fac3217d740 1 -- 172.21.15.12:0/2048953495 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b65d3d2580 msgr2=0x55b65d3db180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:04.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.496+0000 7fac3217d740 1 --2- 172.21.15.12:0/2048953495 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b65d3d2580 0x55b65d3db180 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fac20005c40 tx=0x7fac2009a8d0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.496+0000 7fac3217d740 1 -- 172.21.15.12:0/2048953495 shutdown_connections 2024-04-22T13:34:04.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.496+0000 7fac3217d740 1 --2- 172.21.15.12:0/2048953495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b65d3dba80 0x55b65d3f0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.496+0000 7fac3217d740 1 --2- 172.21.15.12:0/2048953495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b65d3d2ea0 0x55b65d3e72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.496+0000 7fac3217d740 1 --2- 172.21.15.12:0/2048953495 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b65d3d2580 0x55b65d3db180 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.496+0000 7fac3217d740 1 -- 172.21.15.12:0/2048953495 >> 172.21.15.12:0/2048953495 conn(0x55b65d3cccb0 msgr2=0x55b65d3e6bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:04.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.496+0000 7fac3217d740 1 -- 172.21.15.12:0/2048953495 shutdown_connections 2024-04-22T13:34:04.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.496+0000 7fac3217d740 1 -- 172.21.15.12:0/2048953495 wait complete. 2024-04-22T13:34:04.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.496+0000 7fac3217d740 1 -- start start 2024-04-22T13:34:04.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.496+0000 7fac3217d740 1 Processor -- start 2024-04-22T13:34:04.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.497+0000 7fac3217d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b65d3d2580 0x55b65d3f10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.497+0000 7fac3217d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b65d3d2ea0 0x55b65d3ee4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.497+0000 7fac3217d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b65d3dba80 0x55b65d3eeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.497+0000 7fac3217d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fac281af030 con 0x55b65d3d2ea0 2024-04-22T13:34:04.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.497+0000 7fac3217d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fac281ae960 con 0x55b65d3d2580 2024-04-22T13:34:04.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.497+0000 7fac3217d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fac281abbc0 con 0x55b65d3dba80 2024-04-22T13:34:04.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.497+0000 7fac3217d740 10 client.0 _ll_register_callbacks cb 0x7fac24009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:04.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.497+0000 7fac2cb2a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b65d3d2ea0 0x55b65d3ee4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.497+0000 7fac2cb2a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b65d3d2ea0 0x55b65d3ee4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36494/0 (socket says 172.21.15.12:36494) 2024-04-22T13:34:04.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.497+0000 7fac2cb2a640 1 -- 172.21.15.12:0/2698784084 learned_addr learned my addr 172.21.15.12:0/2698784084 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:04.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.497+0000 7fac2d32b640 1 --2- 172.21.15.12:0/2698784084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b65d3d2580 0x55b65d3f10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.497+0000 7fac2db2c640 1 --2- 172.21.15.12:0/2698784084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b65d3dba80 0x55b65d3eeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.497+0000 7fac2cb2a640 1 -- 172.21.15.12:0/2698784084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b65d3dba80 msgr2=0x55b65d3eeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:04.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.497+0000 7fac2cb2a640 1 --2- 172.21.15.12:0/2698784084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b65d3dba80 0x55b65d3eeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.499 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.497+0000 7fac2cb2a640 1 -- 172.21.15.12:0/2698784084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b65d3d2580 msgr2=0x55b65d3f10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:04.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.497+0000 7fac2cb2a640 1 --2- 172.21.15.12:0/2698784084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b65d3d2580 0x55b65d3f10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.497+0000 7fac2cb2a640 1 -- 172.21.15.12:0/2698784084 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b65d3ef2b0 con 0x55b65d3d2ea0 2024-04-22T13:34:04.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.497+0000 7fac2d32b640 1 --2- 172.21.15.12:0/2698784084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b65d3d2580 0x55b65d3f10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:04.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.497+0000 7fac2cb2a640 1 --2- 172.21.15.12:0/2698784084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b65d3d2ea0 0x55b65d3ee4a0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fac180044d0 tx=0x7fac1800ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781450de9b42ff3b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:04.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.497+0000 7fac10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:34:04.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.498+0000 7fac10ff9640 1 -- 172.21.15.12:0/2698784084 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1801b6d0 con 0x55b65d3d2ea0 2024-04-22T13:34:04.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.498+0000 7fac10ff9640 1 -- 172.21.15.12:0/2698784084 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7fac1801b870 con 0x55b65d3d2ea0 2024-04-22T13:34:04.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.498+0000 7fac3217d740 1 -- 172.21.15.12:0/2698784084 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac20009de0 con 0x55b65d3d2ea0 2024-04-22T13:34:04.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.498+0000 7fac10ff9640 1 -- 172.21.15.12:0/2698784084 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fac180035f0 con 0x55b65d3d2ea0 2024-04-22T13:34:04.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.498+0000 7fac3217d740 10 client.18957 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:04.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.498+0000 7fac3217d740 1 -- 172.21.15.12:0/2698784084 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b65d451ad0 con 0x55b65d3d2ea0 2024-04-22T13:34:04.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.498+0000 7fac10ff9640 1 -- 172.21.15.12:0/2698784084 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac20009de0 con 0x55b65d3d2ea0 2024-04-22T13:34:04.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.498+0000 7fac10ff9640 1 -- 172.21.15.12:0/2698784084 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fac1802b020 con 0x55b65d3d2ea0 2024-04-22T13:34:04.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.498+0000 7fac3217d740 10 client.18957 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:04.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.498+0000 7fac3217d740 10 client.18957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:34:04.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.498+0000 7fac3217d740 20 client.18957 populate_metadata read hostname 'smithi012' 2024-04-22T13:34:04.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.498+0000 7fac3217d740 1 --2- 172.21.15.12:0/2698784084 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b65d452230 0x55b65d454650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.498+0000 7fac3217d740 4 client.18957 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:34:04.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.498+0000 7fac3217d740 1 -- 172.21.15.12:0/2698784084 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b65d2ba590 con 0x55b65d452230 2024-04-22T13:34:04.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.498+0000 7fac2d32b640 1 --2- 172.21.15.12:0/2698784084 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b65d452230 0x55b65d454650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.499+0000 7fac10ff9640 10 client.18957 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:34:04.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.499+0000 7fac2d32b640 1 --2- 172.21.15.12:0/2698784084 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b65d452230 0x55b65d454650 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=960efc8b0b9286a5 server_cookie=ad2e5f0bca0d1aa7 in_seq=0 out_seq=0 2024-04-22T13:34:04.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.499+0000 7fac10ff9640 1 -- 172.21.15.12:0/2698784084 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b65d2ba590 con 0x55b65d452230 2024-04-22T13:34:04.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.499+0000 7fac10ff9640 10 client.18957 handle_command_reply: tid=0 2024-04-22T13:34:04.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.499+0000 7fac3217d740 1 client.18957 shutdown 2024-04-22T13:34:04.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.499+0000 7fac3217d740 1 -- 172.21.15.12:0/2698784084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b65d3d2ea0 msgr2=0x55b65d3ee4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:04.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.499+0000 7fac3217d740 1 --2- 172.21.15.12:0/2698784084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b65d3d2ea0 0x55b65d3ee4a0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fac180044d0 tx=0x7fac1800ba60 comp rx=0 tx=0).stop 2024-04-22T13:34:04.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.499+0000 7fac3217d740 1 -- 172.21.15.12:0/2698784084 shutdown_connections 2024-04-22T13:34:04.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.499+0000 7fac3217d740 1 --2- 172.21.15.12:0/2698784084 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b65d452230 0x55b65d454650 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.499+0000 7fac3217d740 1 --2- 172.21.15.12:0/2698784084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b65d3dba80 0x55b65d3eeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.499+0000 7fac3217d740 1 --2- 172.21.15.12:0/2698784084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b65d3d2ea0 0x55b65d3ee4a0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.499+0000 7fac10ff9640 0 client.18957 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:34:04.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.499+0000 7fac3217d740 1 --2- 172.21.15.12:0/2698784084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b65d3d2580 0x55b65d3f10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.499+0000 7fac3217d740 1 -- 172.21.15.12:0/2698784084 >> 172.21.15.12:0/2698784084 conn(0x55b65d3cccb0 msgr2=0x55b65d3d91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:04.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.499+0000 7fac3217d740 1 -- 172.21.15.12:0/2698784084 shutdown_connections 2024-04-22T13:34:04.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.499+0000 7fac3217d740 1 -- 172.21.15.12:0/2698784084 wait complete. 2024-04-22T13:34:04.502 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.499+0000 7fac3217d740 20 client.18957 trim_cache size 0 max 16384 2024-04-22T13:34:04.502 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:34:04.502 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-04-22T13:34:04.503 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-04-22T13:34:04.503 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:34:04.503 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.500+0000 7fac2f58f640 1 -- 172.21.15.12:0/809553810 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fac140a3d20 msgr2=0x7fac140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:04.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.500+0000 7fac2f58f640 1 --2- 172.21.15.12:0/809553810 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fac140a3d20 0x7fac140a61e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fac24005e30 tx=0x7fac2402e040 comp rx=0 tx=0).stop 2024-04-22T13:34:04.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.500+0000 7fac2f58f640 1 -- 172.21.15.12:0/809553810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac28101080 msgr2=0x7fac28199540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:04.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.500+0000 7fac2f58f640 1 --2- 172.21.15.12:0/809553810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac28101080 0x7fac28199540 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fac2000d950 tx=0x7fac2000de10 comp rx=0 tx=0).stop 2024-04-22T13:34:04.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.500+0000 7fac2f58f640 1 -- 172.21.15.12:0/809553810 shutdown_connections 2024-04-22T13:34:04.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.500+0000 7fac2f58f640 1 --2- 172.21.15.12:0/809553810 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fac140a3d20 0x7fac140a61e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.500+0000 7fac2f58f640 1 --2- 172.21.15.12:0/809553810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fac28199fc0 0x7fac281a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.500+0000 7fac2f58f640 1 --2- 172.21.15.12:0/809553810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fac28106150 0x7fac28199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.500+0000 7fac2f58f640 1 --2- 172.21.15.12:0/809553810 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fac28101080 0x7fac28199540 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.500+0000 7fac2f58f640 1 -- 172.21.15.12:0/809553810 >> 172.21.15.12:0/809553810 conn(0x7fac28078000 msgr2=0x7fac280fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:04.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.500+0000 7fac2f58f640 1 -- 172.21.15.12:0/809553810 shutdown_connections 2024-04-22T13:34:04.504 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.500+0000 7fac2f58f640 1 -- 172.21.15.12:0/809553810 wait complete. 2024-04-22T13:34:04.507 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:34:04.507 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:34:04.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.613+0000 7f8500ae0640 1 -- 172.21.15.12:0/535336519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84fc1021a0 msgr2=0x7f84fc102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:04.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.613+0000 7f8500ae0640 1 --2- 172.21.15.12:0/535336519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84fc1021a0 0x7f84fc102580 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f84ec009900 tx=0x7f84ec02bc30 comp rx=0 tx=0).stop 2024-04-22T13:34:04.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.613+0000 7f8500ae0640 1 -- 172.21.15.12:0/535336519 shutdown_connections 2024-04-22T13:34:04.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.613+0000 7f8500ae0640 1 --2- 172.21.15.12:0/535336519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84fc107270 0x7f84fc10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.613+0000 7f8500ae0640 1 --2- 172.21.15.12:0/535336519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84fc102b50 0x7f84fc106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.613+0000 7f8500ae0640 1 --2- 172.21.15.12:0/535336519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84fc1021a0 0x7f84fc102580 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.613+0000 7f8500ae0640 1 -- 172.21.15.12:0/535336519 >> 172.21.15.12:0/535336519 conn(0x7f84fc0793a0 msgr2=0x7f84fc0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:04.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.613+0000 7f8500ae0640 1 -- 172.21.15.12:0/535336519 shutdown_connections 2024-04-22T13:34:04.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.613+0000 7f8500ae0640 1 -- 172.21.15.12:0/535336519 wait complete. 2024-04-22T13:34:04.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.614+0000 7f8500ae0640 1 Processor -- start 2024-04-22T13:34:04.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.614+0000 7f8500ae0640 1 -- start start 2024-04-22T13:34:04.612 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.614+0000 7f8500ae0640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84fc1021a0 0x7f84fc1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.614+0000 7f8500ae0640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84fc107270 0x7f84fc198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.614+0000 7f8500ae0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84fc19cf90 0x7f84fc19d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.614+0000 7f8500ae0640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f84fc1121b0 con 0x7f84fc19cf90 2024-04-22T13:34:04.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.614+0000 7f8500ae0640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f84fc112030 con 0x7f84fc1021a0 2024-04-22T13:34:04.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.614+0000 7f8500ae0640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f84fc112330 con 0x7f84fc107270 2024-04-22T13:34:04.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.614+0000 7f84fad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84fc19cf90 0x7f84fc19d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.614+0000 7f84fad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84fc19cf90 0x7f84fc19d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36508/0 (socket says 172.21.15.12:36508) 2024-04-22T13:34:04.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.614+0000 7f84fad9d640 1 -- 172.21.15.12:0/3409522628 learned_addr learned my addr 172.21.15.12:0/3409522628 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:04.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.615+0000 7f84fa59c640 1 --2- 172.21.15.12:0/3409522628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84fc1021a0 0x7f84fc1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.615+0000 7f84f3fff640 1 --2- 172.21.15.12:0/3409522628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84fc107270 0x7f84fc198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.615+0000 7f84fad9d640 1 -- 172.21.15.12:0/3409522628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84fc107270 msgr2=0x7f84fc198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:04.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.615+0000 7f84fad9d640 1 --2- 172.21.15.12:0/3409522628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84fc107270 0x7f84fc198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.615+0000 7f84fad9d640 1 -- 172.21.15.12:0/3409522628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84fc1021a0 msgr2=0x7f84fc1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:04.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.615+0000 7f84fad9d640 1 --2- 172.21.15.12:0/3409522628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84fc1021a0 0x7f84fc1986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.615+0000 7f84fad9d640 1 -- 172.21.15.12:0/3409522628 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84fc19da80 con 0x7f84fc19cf90 2024-04-22T13:34:04.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.615+0000 7f84fad9d640 1 --2- 172.21.15.12:0/3409522628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84fc19cf90 0x7f84fc19d440 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f84f400eaa0 tx=0x7f84f400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:04.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.615+0000 7f84f37fe640 1 -- 172.21.15.12:0/3409522628 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f400cdb0 con 0x7f84fc19cf90 2024-04-22T13:34:04.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.615+0000 7f8500ae0640 1 -- 172.21.15.12:0/3409522628 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84fc19dd70 con 0x7f84fc19cf90 2024-04-22T13:34:04.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.615+0000 7f84f37fe640 1 -- 172.21.15.12:0/3409522628 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84f4005020 con 0x7f84fc19cf90 2024-04-22T13:34:04.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.615+0000 7f84f37fe640 1 -- 172.21.15.12:0/3409522628 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f40116d0 con 0x7f84fc19cf90 2024-04-22T13:34:04.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.615+0000 7f8500ae0640 1 -- 172.21.15.12:0/3409522628 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84fc1a5510 con 0x7f84fc19cf90 2024-04-22T13:34:04.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.616+0000 7f8500ae0640 1 -- 172.21.15.12:0/3409522628 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84fc069230 con 0x7f84fc19cf90 2024-04-22T13:34:04.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.617+0000 7f84f37fe640 1 -- 172.21.15.12:0/3409522628 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f84f40051d0 con 0x7f84fc19cf90 2024-04-22T13:34:04.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.618+0000 7f84f37fe640 1 --2- 172.21.15.12:0/3409522628 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f84e80a3d20 0x7f84e80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.618+0000 7f84fa59c640 1 --2- 172.21.15.12:0/3409522628 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f84e80a3d20 0x7f84e80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.618+0000 7f84f37fe640 1 -- 172.21.15.12:0/3409522628 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f84f40d09e0 con 0x7f84fc19cf90 2024-04-22T13:34:04.615 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.618+0000 7f84fa59c640 1 --2- 172.21.15.12:0/3409522628 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f84e80a3d20 0x7f84e80a61e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f84ec006010 tx=0x7f84ec005e40 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:04.618 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.623+0000 7f84f37fe640 1 -- 172.21.15.12:0/3409522628 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f84f4016030 con 0x7f84fc19cf90 2024-04-22T13:34:04.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.847+0000 7f8500ae0640 1 -- 172.21.15.12:0/3409522628 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84fc10a500 con 0x7f84fc19cf90 2024-04-22T13:34:04.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.848+0000 7f84f37fe640 1 -- 172.21.15.12:0/3409522628 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7f84f409d040 con 0x7f84fc19cf90 2024-04-22T13:34:04.845 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:34:04.845 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:34:04.845 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:34:04.846 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.850+0000 7f8500ae0640 1 -- 172.21.15.12:0/3409522628 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f84e80a3d20 msgr2=0x7f84e80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:04.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.850+0000 7f8500ae0640 1 --2- 172.21.15.12:0/3409522628 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f84e80a3d20 0x7f84e80a61e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f84ec006010 tx=0x7f84ec005e40 comp rx=0 tx=0).stop 2024-04-22T13:34:04.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.850+0000 7f8500ae0640 1 -- 172.21.15.12:0/3409522628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84fc19cf90 msgr2=0x7f84fc19d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:04.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.850+0000 7f8500ae0640 1 --2- 172.21.15.12:0/3409522628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84fc19cf90 0x7f84fc19d440 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f84f400eaa0 tx=0x7f84f400ef60 comp rx=0 tx=0).stop 2024-04-22T13:34:04.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.850+0000 7f8500ae0640 1 -- 172.21.15.12:0/3409522628 shutdown_connections 2024-04-22T13:34:04.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.850+0000 7f8500ae0640 1 --2- 172.21.15.12:0/3409522628 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f84e80a3d20 0x7f84e80a61e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.850+0000 7f8500ae0640 1 --2- 172.21.15.12:0/3409522628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f84fc19cf90 0x7f84fc19d440 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.850+0000 7f8500ae0640 1 --2- 172.21.15.12:0/3409522628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84fc107270 0x7f84fc198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.850+0000 7f8500ae0640 1 --2- 172.21.15.12:0/3409522628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84fc1021a0 0x7f84fc1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.850+0000 7f8500ae0640 1 -- 172.21.15.12:0/3409522628 >> 172.21.15.12:0/3409522628 conn(0x7f84fc0793a0 msgr2=0x7f84fc0ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:04.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.850+0000 7f8500ae0640 1 -- 172.21.15.12:0/3409522628 shutdown_connections 2024-04-22T13:34:04.847 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.850+0000 7f8500ae0640 1 -- 172.21.15.12:0/3409522628 wait complete. 2024-04-22T13:34:04.858 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:34:04.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.965+0000 7f415d1fd640 1 -- 172.21.15.12:0/2083044007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4158107260 msgr2=0x7f415810ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:04.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.965+0000 7f415d1fd640 1 --2- 172.21.15.12:0/2083044007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4158107260 0x7f415810ee30 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f4150009ca0 tx=0x7f415002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.965+0000 7f415d1fd640 1 -- 172.21.15.12:0/2083044007 shutdown_connections 2024-04-22T13:34:04.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.965+0000 7f415d1fd640 1 --2- 172.21.15.12:0/2083044007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4158107260 0x7f415810ee30 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.965+0000 7f415d1fd640 1 --2- 172.21.15.12:0/2083044007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4158102b40 0x7f4158106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.965+0000 7f415d1fd640 1 --2- 172.21.15.12:0/2083044007 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4158102190 0x7f4158102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.965+0000 7f415d1fd640 1 -- 172.21.15.12:0/2083044007 >> 172.21.15.12:0/2083044007 conn(0x7f41580fd990 msgr2=0x7f41580ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:04.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.966+0000 7f415d1fd640 1 -- 172.21.15.12:0/2083044007 shutdown_connections 2024-04-22T13:34:04.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.966+0000 7f415d1fd640 1 -- 172.21.15.12:0/2083044007 wait complete. 2024-04-22T13:34:04.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.966+0000 7f415d1fd640 1 Processor -- start 2024-04-22T13:34:04.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.966+0000 7f415d1fd640 1 -- start start 2024-04-22T13:34:04.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.967+0000 7f415d1fd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4158102190 0x7f4158074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.967+0000 7f415d1fd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4158107260 0x7f4158074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.967+0000 7f415d1fd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4158074cd0 0x7f41581a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.967+0000 7f415d1fd640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f41581121d0 con 0x7f4158102190 2024-04-22T13:34:04.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.967+0000 7f415d1fd640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4158112050 con 0x7f4158107260 2024-04-22T13:34:04.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.967+0000 7f415d1fd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4158112350 con 0x7f4158074cd0 2024-04-22T13:34:04.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.967+0000 7f4156d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4158102190 0x7f4158074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.967+0000 7f4156d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4158102190 0x7f4158074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36522/0 (socket says 172.21.15.12:36522) 2024-04-22T13:34:04.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.967+0000 7f4156d9d640 1 -- 172.21.15.12:0/1121064036 learned_addr learned my addr 172.21.15.12:0/1121064036 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:04.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.967+0000 7f415759e640 1 --2- 172.21.15.12:0/1121064036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4158074cd0 0x7f41581a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.967+0000 7f415659c640 1 --2- 172.21.15.12:0/1121064036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4158107260 0x7f4158074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.967+0000 7f4156d9d640 1 -- 172.21.15.12:0/1121064036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4158074cd0 msgr2=0x7f41581a6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:04.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.967+0000 7f4156d9d640 1 --2- 172.21.15.12:0/1121064036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4158074cd0 0x7f41581a6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.967+0000 7f4156d9d640 1 -- 172.21.15.12:0/1121064036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4158107260 msgr2=0x7f4158074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:04.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.967+0000 7f4156d9d640 1 --2- 172.21.15.12:0/1121064036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4158107260 0x7f4158074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.967+0000 7f4156d9d640 1 -- 172.21.15.12:0/1121064036 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41581a7020 con 0x7f4158102190 2024-04-22T13:34:04.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.967+0000 7f4156d9d640 1 --2- 172.21.15.12:0/1121064036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4158102190 0x7f4158074250 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f414800d770 tx=0x7f414800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:04.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.967+0000 7f413ffff640 1 -- 172.21.15.12:0/1121064036 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4148017070 con 0x7f4158102190 2024-04-22T13:34:04.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.967+0000 7f413ffff640 1 -- 172.21.15.12:0/1121064036 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41480054d0 con 0x7f4158102190 2024-04-22T13:34:04.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.967+0000 7f413ffff640 1 -- 172.21.15.12:0/1121064036 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4148004c30 con 0x7f4158102190 2024-04-22T13:34:04.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.968+0000 7f415d1fd640 1 -- 172.21.15.12:0/1121064036 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41581a7310 con 0x7f4158102190 2024-04-22T13:34:04.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.968+0000 7f415d1fd640 1 -- 172.21.15.12:0/1121064036 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41581a7810 con 0x7f4158102190 2024-04-22T13:34:04.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.970+0000 7f413ffff640 1 -- 172.21.15.12:0/1121064036 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4148004dd0 con 0x7f4158102190 2024-04-22T13:34:04.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.970+0000 7f415fddf740 1 Processor -- start 2024-04-22T13:34:04.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.970+0000 7f415fddf740 1 -- start start 2024-04-22T13:34:04.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.970+0000 7f415fddf740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5631cde504d0 0x5631cde508b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.970+0000 7f415fddf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631cde50e80 0x5631cde59430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.970+0000 7f415fddf740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5631cde59a00 0x5631cde5be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.970+0000 7f415fddf740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5631cddb51f0 con 0x5631cde59a00 2024-04-22T13:34:04.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.970+0000 7f415fddf740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5631cdd524b0 con 0x5631cde504d0 2024-04-22T13:34:04.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.970+0000 7f415fddf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5631cddb11b0 con 0x5631cde50e80 2024-04-22T13:34:04.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.970+0000 7f415659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5631cde59a00 0x5631cde5be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.970+0000 7f415659c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5631cde59a00 0x5631cde5be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36530/0 (socket says 172.21.15.12:36530) 2024-04-22T13:34:04.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.970+0000 7f415659c640 1 -- 172.21.15.12:0/2175049156 learned_addr learned my addr 172.21.15.12:0/2175049156 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:04.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.971+0000 7f4156d9d640 1 --2- 172.21.15.12:0/2175049156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631cde50e80 0x5631cde59430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.971+0000 7f415759e640 1 --2- 172.21.15.12:0/2175049156 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5631cde504d0 0x5631cde508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.971+0000 7f413ffff640 1 --2- 172.21.15.12:0/1121064036 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f41400a3d50 0x7f41400a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.971+0000 7f415659c640 1 -- 172.21.15.12:0/2175049156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631cde50e80 msgr2=0x5631cde59430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:04.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.971+0000 7f415659c640 1 --2- 172.21.15.12:0/2175049156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631cde50e80 0x5631cde59430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.971+0000 7f415659c640 1 -- 172.21.15.12:0/2175049156 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5631cde504d0 msgr2=0x5631cde508b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:04.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.971+0000 7f415659c640 1 --2- 172.21.15.12:0/2175049156 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5631cde504d0 0x5631cde508b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.971+0000 7f415659c640 1 -- 172.21.15.12:0/2175049156 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631cdd441b0 con 0x5631cde59a00 2024-04-22T13:34:04.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.971+0000 7f415759e640 1 --2- 172.21.15.12:0/2175049156 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5631cde504d0 0x5631cde508b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:04.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.971+0000 7f4156d9d640 1 --2- 172.21.15.12:0/2175049156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631cde50e80 0x5631cde59430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:04.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.971+0000 7f415759e640 1 --2- 172.21.15.12:0/1121064036 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f41400a3d50 0x7f41400a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.971+0000 7f413ffff640 1 -- 172.21.15.12:0/1121064036 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f41480d0250 con 0x7f4158102190 2024-04-22T13:34:04.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.971+0000 7f415659c640 1 --2- 172.21.15.12:0/2175049156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5631cde59a00 0x5631cde5be50 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f414c00bbf0 tx=0x7f414c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c5b9d507431d35 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:04.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.971+0000 7f413d7fa640 1 -- 172.21.15.12:0/2175049156 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c005020 con 0x5631cde59a00 2024-04-22T13:34:04.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.971+0000 7f413d7fa640 1 -- 172.21.15.12:0/2175049156 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f414c0047a0 con 0x5631cde59a00 2024-04-22T13:34:04.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.971+0000 7f413d7fa640 1 -- 172.21.15.12:0/2175049156 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c0116a0 con 0x5631cde59a00 2024-04-22T13:34:04.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.972+0000 7f415fddf740 1 -- 172.21.15.12:0/2175049156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5631cde59a00 msgr2=0x5631cde5be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:04.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.972+0000 7f415759e640 1 --2- 172.21.15.12:0/1121064036 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f41400a3d50 0x7f41400a6210 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4150004af0 tx=0x7f415002e040 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:04.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.972+0000 7f415fddf740 1 --2- 172.21.15.12:0/2175049156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5631cde59a00 0x5631cde5be50 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f414c00bbf0 tx=0x7f414c0107a0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.972+0000 7f415fddf740 1 -- 172.21.15.12:0/2175049156 shutdown_connections 2024-04-22T13:34:04.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.972+0000 7f415fddf740 1 --2- 172.21.15.12:0/2175049156 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5631cde59a00 0x5631cde5be50 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.972+0000 7f415fddf740 1 --2- 172.21.15.12:0/2175049156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631cde50e80 0x5631cde59430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.972+0000 7f415fddf740 1 --2- 172.21.15.12:0/2175049156 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5631cde504d0 0x5631cde508b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.972+0000 7f415fddf740 1 -- 172.21.15.12:0/2175049156 >> 172.21.15.12:0/2175049156 conn(0x5631cde4aba0 msgr2=0x5631cde4af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:04.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.972+0000 7f415fddf740 1 -- 172.21.15.12:0/2175049156 shutdown_connections 2024-04-22T13:34:04.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.972+0000 7f415fddf740 1 -- 172.21.15.12:0/2175049156 wait complete. 2024-04-22T13:34:04.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.972+0000 7f415fddf740 1 -- start start 2024-04-22T13:34:04.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.972+0000 7f415fddf740 1 Processor -- start 2024-04-22T13:34:04.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.973+0000 7f415fddf740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5631cde504d0 0x5631cde59320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.973+0000 7f415fddf740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5631cde50e80 0x5631cde566b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.973+0000 7f415fddf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631cde59a00 0x5631cde56c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.973+0000 7f415fddf740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5631cde5c970 con 0x5631cde504d0 2024-04-22T13:34:04.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.973+0000 7f415fddf740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5631cdd6e500 con 0x5631cde50e80 2024-04-22T13:34:04.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.973+0000 7f415fddf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5631cde5caf0 con 0x5631cde59a00 2024-04-22T13:34:04.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.973+0000 7f415fddf740 10 client.0 _ll_register_callbacks cb 0x5631cdd3cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:04.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.973+0000 7f4156d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5631cde504d0 0x5631cde59320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.973+0000 7f4156d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5631cde504d0 0x5631cde59320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36544/0 (socket says 172.21.15.12:36544) 2024-04-22T13:34:04.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.973+0000 7f4156d9d640 1 -- 172.21.15.12:0/2142665876 learned_addr learned my addr 172.21.15.12:0/2142665876 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:04.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.973+0000 7f415759e640 1 --2- 172.21.15.12:0/2142665876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631cde59a00 0x5631cde56c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.973+0000 7f415659c640 1 --2- 172.21.15.12:0/2142665876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5631cde50e80 0x5631cde566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.973+0000 7f4156d9d640 1 -- 172.21.15.12:0/2142665876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631cde59a00 msgr2=0x5631cde56c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:04.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.973+0000 7f4156d9d640 1 --2- 172.21.15.12:0/2142665876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631cde59a00 0x5631cde56c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.973+0000 7f4156d9d640 1 -- 172.21.15.12:0/2142665876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5631cde50e80 msgr2=0x5631cde566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:04.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.973+0000 7f4156d9d640 1 --2- 172.21.15.12:0/2142665876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5631cde50e80 0x5631cde566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.973+0000 7f4156d9d640 1 -- 172.21.15.12:0/2142665876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631cdac73f0 con 0x5631cde504d0 2024-04-22T13:34:04.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.973+0000 7f415759e640 1 --2- 172.21.15.12:0/2142665876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631cde59a00 0x5631cde56c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:04.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.973+0000 7f415659c640 1 --2- 172.21.15.12:0/2142665876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5631cde50e80 0x5631cde566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:04.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.973+0000 7f4156d9d640 1 --2- 172.21.15.12:0/2142665876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5631cde504d0 0x5631cde59320 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f41480cd9c0 tx=0x7f414809be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2063a3ae59c6dfa server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:04.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.973+0000 7f413aff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:34:04.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.973+0000 7f413aff5640 1 -- 172.21.15.12:0/2142665876 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4148020050 con 0x5631cde504d0 2024-04-22T13:34:04.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.973+0000 7f413aff5640 1 -- 172.21.15.12:0/2142665876 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f41480988d0 con 0x5631cde504d0 2024-04-22T13:34:04.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.973+0000 7f415fddf740 1 -- 172.21.15.12:0/2142665876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f414c0047a0 con 0x5631cde504d0 2024-04-22T13:34:04.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.973+0000 7f413aff5640 1 -- 172.21.15.12:0/2142665876 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4148005680 con 0x5631cde504d0 2024-04-22T13:34:04.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.973+0000 7f415fddf740 10 client.18975 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:04.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.973+0000 7f415fddf740 1 -- 172.21.15.12:0/2142665876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5631cdb917d0 con 0x5631cde504d0 2024-04-22T13:34:04.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.974+0000 7f413aff5640 1 -- 172.21.15.12:0/2142665876 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f414c0047a0 con 0x5631cde504d0 2024-04-22T13:34:04.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.974+0000 7f413aff5640 1 -- 172.21.15.12:0/2142665876 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f41480a00b0 con 0x5631cde504d0 2024-04-22T13:34:04.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.974+0000 7f415fddf740 10 client.18975 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:04.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.974+0000 7f415fddf740 10 client.18975 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:34:04.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.974+0000 7f415fddf740 20 client.18975 populate_metadata read hostname 'smithi012' 2024-04-22T13:34:04.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.974+0000 7f415fddf740 1 --2- 172.21.15.12:0/2142665876 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5631cde69c10 0x5631cde6c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.974+0000 7f415fddf740 4 client.18975 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:34:04.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.974+0000 7f415fddf740 1 -- 172.21.15.12:0/2142665876 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631cdd54930 con 0x5631cde69c10 2024-04-22T13:34:04.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.974+0000 7f415659c640 1 --2- 172.21.15.12:0/2142665876 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5631cde69c10 0x5631cde6c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.974+0000 7f415659c640 1 --2- 172.21.15.12:0/2142665876 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5631cde69c10 0x5631cde6c080 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c90ee1c3bbd7abbc server_cookie=25e26d1a514881ac in_seq=0 out_seq=0 2024-04-22T13:34:04.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.974+0000 7f413aff5640 10 client.18975 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:34:04.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.975+0000 7f413aff5640 1 -- 172.21.15.12:0/2142665876 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5631cdd54930 con 0x5631cde69c10 2024-04-22T13:34:04.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.975+0000 7f413aff5640 10 client.18975 handle_command_reply: tid=0 2024-04-22T13:34:04.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.975+0000 7f415fddf740 1 client.18975 shutdown 2024-04-22T13:34:04.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.975+0000 7f415fddf740 1 -- 172.21.15.12:0/2142665876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5631cde504d0 msgr2=0x5631cde59320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:04.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.975+0000 7f415fddf740 1 --2- 172.21.15.12:0/2142665876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5631cde504d0 0x5631cde59320 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f41480cd9c0 tx=0x7f414809be90 comp rx=0 tx=0).stop 2024-04-22T13:34:04.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.975+0000 7f415fddf740 1 -- 172.21.15.12:0/2142665876 shutdown_connections 2024-04-22T13:34:04.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.975+0000 7f415fddf740 1 --2- 172.21.15.12:0/2142665876 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5631cde69c10 0x5631cde6c080 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.975+0000 7f415fddf740 1 --2- 172.21.15.12:0/2142665876 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631cde59a00 0x5631cde56c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.975+0000 7f415fddf740 1 --2- 172.21.15.12:0/2142665876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5631cde50e80 0x5631cde566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.975+0000 7f413aff5640 0 client.18975 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:34:04.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.975+0000 7f415fddf740 1 --2- 172.21.15.12:0/2142665876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5631cde504d0 0x5631cde59320 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.975+0000 7f415fddf740 1 -- 172.21.15.12:0/2142665876 >> 172.21.15.12:0/2142665876 conn(0x5631cde4aba0 msgr2=0x5631cde4c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:04.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.975+0000 7f415fddf740 1 -- 172.21.15.12:0/2142665876 shutdown_connections 2024-04-22T13:34:04.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.975+0000 7f415fddf740 1 -- 172.21.15.12:0/2142665876 wait complete. 2024-04-22T13:34:04.980 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.976+0000 7f415fddf740 20 client.18975 trim_cache size 0 max 16384 2024-04-22T13:34:04.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.989+0000 7f415fddf740 1 Processor -- start 2024-04-22T13:34:04.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.989+0000 7f415fddf740 1 -- start start 2024-04-22T13:34:04.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.989+0000 7f415fddf740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5631cde504d0 0x5631cde4ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.989+0000 7f415fddf740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5631cde50e80 0x5631cde49a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.989+0000 7f415fddf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631cde59a00 0x5631cde58de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.989+0000 7f415fddf740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f41581afd20 con 0x5631cde504d0 2024-04-22T13:34:04.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.989+0000 7f415fddf740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f41581b1860 con 0x5631cde50e80 2024-04-22T13:34:04.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.989+0000 7f415fddf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f41581b03f0 con 0x5631cde59a00 2024-04-22T13:34:04.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.989+0000 7f4156d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5631cde504d0 0x5631cde4ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.991 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.989+0000 7f4156d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5631cde504d0 0x5631cde4ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36558/0 (socket says 172.21.15.12:36558) 2024-04-22T13:34:04.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.989+0000 7f4156d9d640 1 -- 172.21.15.12:0/27506272 learned_addr learned my addr 172.21.15.12:0/27506272 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:04.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.989+0000 7f415759e640 1 --2- 172.21.15.12:0/27506272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631cde59a00 0x5631cde58de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.989+0000 7f415659c640 1 --2- 172.21.15.12:0/27506272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5631cde50e80 0x5631cde49a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.989+0000 7f4156d9d640 1 -- 172.21.15.12:0/27506272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631cde59a00 msgr2=0x5631cde58de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:04.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.989+0000 7f4156d9d640 1 --2- 172.21.15.12:0/27506272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631cde59a00 0x5631cde58de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.989+0000 7f4156d9d640 1 -- 172.21.15.12:0/27506272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5631cde50e80 msgr2=0x5631cde49a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:04.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.989+0000 7f4156d9d640 1 --2- 172.21.15.12:0/27506272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5631cde50e80 0x5631cde49a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.989+0000 7f4156d9d640 1 -- 172.21.15.12:0/27506272 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631cde50c20 con 0x5631cde504d0 2024-04-22T13:34:04.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.989+0000 7f415759e640 1 --2- 172.21.15.12:0/27506272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631cde59a00 0x5631cde58de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:04.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.990+0000 7f4156d9d640 1 --2- 172.21.15.12:0/27506272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5631cde504d0 0x5631cde4ca80 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f414809bf60 tx=0x7f4148005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf49dfc7591a6b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:04.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.990+0000 7f413cff9640 1 -- 172.21.15.12:0/27506272 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4148020070 con 0x5631cde504d0 2024-04-22T13:34:04.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.990+0000 7f413cff9640 1 -- 172.21.15.12:0/27506272 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f414809c260 con 0x5631cde504d0 2024-04-22T13:34:04.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.990+0000 7f413cff9640 1 -- 172.21.15.12:0/27506272 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414809b120 con 0x5631cde504d0 2024-04-22T13:34:04.992 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.990+0000 7f415fddf740 1 -- 172.21.15.12:0/27506272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5631cde504d0 msgr2=0x5631cde4ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:04.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.990+0000 7f415fddf740 1 --2- 172.21.15.12:0/27506272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5631cde504d0 0x5631cde4ca80 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f414809bf60 tx=0x7f4148005bb0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.990+0000 7f415fddf740 1 -- 172.21.15.12:0/27506272 shutdown_connections 2024-04-22T13:34:04.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.990+0000 7f415fddf740 1 --2- 172.21.15.12:0/27506272 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631cde59a00 0x5631cde58de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.990+0000 7f415fddf740 1 --2- 172.21.15.12:0/27506272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5631cde50e80 0x5631cde49a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.990+0000 7f415fddf740 1 --2- 172.21.15.12:0/27506272 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5631cde504d0 0x5631cde4ca80 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.990+0000 7f415fddf740 1 -- 172.21.15.12:0/27506272 >> 172.21.15.12:0/27506272 conn(0x5631cde4aba0 msgr2=0x5631cde49370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:04.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.990+0000 7f415fddf740 1 -- 172.21.15.12:0/27506272 shutdown_connections 2024-04-22T13:34:04.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.990+0000 7f415fddf740 1 -- 172.21.15.12:0/27506272 wait complete. 2024-04-22T13:34:04.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.991+0000 7f415fddf740 1 -- start start 2024-04-22T13:34:04.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.991+0000 7f415fddf740 1 Processor -- start 2024-04-22T13:34:04.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.991+0000 7f415fddf740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5631cde504d0 0x5631cde61fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.991+0000 7f415fddf740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5631cde50e80 0x5631cde52f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.991+0000 7f415fddf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631cde59a00 0x5631cde53460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.993 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.991+0000 7f415fddf740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f41581b1190 con 0x5631cde504d0 2024-04-22T13:34:04.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.991+0000 7f415fddf740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f41581b0ac0 con 0x5631cde50e80 2024-04-22T13:34:04.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.991+0000 7f415fddf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f41581add20 con 0x5631cde59a00 2024-04-22T13:34:04.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.991+0000 7f415fddf740 10 client.0 _ll_register_callbacks cb 0x7f414809c0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:04.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.991+0000 7f4156d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5631cde504d0 0x5631cde61fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.991+0000 7f4156d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5631cde504d0 0x5631cde61fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36570/0 (socket says 172.21.15.12:36570) 2024-04-22T13:34:04.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.991+0000 7f4156d9d640 1 -- 172.21.15.12:0/3686147792 learned_addr learned my addr 172.21.15.12:0/3686147792 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:04.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.991+0000 7f415659c640 1 --2- 172.21.15.12:0/3686147792 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5631cde50e80 0x5631cde52f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.991+0000 7f415759e640 1 --2- 172.21.15.12:0/3686147792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631cde59a00 0x5631cde53460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.991+0000 7f4156d9d640 1 -- 172.21.15.12:0/3686147792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631cde59a00 msgr2=0x5631cde53460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:04.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.991+0000 7f4156d9d640 1 --2- 172.21.15.12:0/3686147792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631cde59a00 0x5631cde53460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.991+0000 7f4156d9d640 1 -- 172.21.15.12:0/3686147792 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5631cde50e80 msgr2=0x5631cde52f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:04.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.991+0000 7f4156d9d640 1 --2- 172.21.15.12:0/3686147792 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5631cde50e80 0x5631cde52f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.994 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.991+0000 7f4156d9d640 1 -- 172.21.15.12:0/3686147792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631cde53b00 con 0x5631cde504d0 2024-04-22T13:34:04.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.991+0000 7f415659c640 1 --2- 172.21.15.12:0/3686147792 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5631cde50e80 0x5631cde52f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:04.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.991+0000 7f415759e640 1 --2- 172.21.15.12:0/3686147792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631cde59a00 0x5631cde53460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:04.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.992+0000 7f4156d9d640 1 --2- 172.21.15.12:0/3686147792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5631cde504d0 0x5631cde61fa0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f4148009610 tx=0x7f41480a4f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9796fc5814c86079 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:04.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.992+0000 7f413aff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:34:04.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.992+0000 7f413aff5640 1 -- 172.21.15.12:0/3686147792 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4148020070 con 0x5631cde504d0 2024-04-22T13:34:04.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.992+0000 7f413aff5640 1 -- 172.21.15.12:0/3686147792 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f4148015020 con 0x5631cde504d0 2024-04-22T13:34:04.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.992+0000 7f415fddf740 1 -- 172.21.15.12:0/3686147792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f414809c260 con 0x5631cde504d0 2024-04-22T13:34:04.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.992+0000 7f413aff5640 1 -- 172.21.15.12:0/3686147792 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f41480cdbe0 con 0x5631cde504d0 2024-04-22T13:34:04.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.992+0000 7f415fddf740 10 client.18981 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:04.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.992+0000 7f415fddf740 1 -- 172.21.15.12:0/3686147792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5631cded0050 con 0x5631cde504d0 2024-04-22T13:34:04.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.992+0000 7f413aff5640 1 -- 172.21.15.12:0/3686147792 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f414809c260 con 0x5631cde504d0 2024-04-22T13:34:04.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.992+0000 7f413aff5640 1 -- 172.21.15.12:0/3686147792 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f41480a17f0 con 0x5631cde504d0 2024-04-22T13:34:04.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.992+0000 7f415fddf740 10 client.18981 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:04.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.992+0000 7f415fddf740 10 client.18981 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:34:04.995 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.992+0000 7f415fddf740 20 client.18981 populate_metadata read hostname 'smithi012' 2024-04-22T13:34:04.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.992+0000 7f415fddf740 1 --2- 172.21.15.12:0/3686147792 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5631cded03e0 0x5631cded27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:04.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.992+0000 7f415fddf740 4 client.18981 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:34:04.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.992+0000 7f415fddf740 1 -- 172.21.15.12:0/3686147792 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5631cddc2c10 con 0x5631cded03e0 2024-04-22T13:34:04.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.993+0000 7f415659c640 1 --2- 172.21.15.12:0/3686147792 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5631cded03e0 0x5631cded27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:04.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.993+0000 7f415659c640 1 --2- 172.21.15.12:0/3686147792 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5631cded03e0 0x5631cded27d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1b8f4512c7c30e6a server_cookie=b929ce926acd78f8 in_seq=0 out_seq=0 2024-04-22T13:34:04.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.993+0000 7f413aff5640 10 client.18981 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:34:04.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.993+0000 7f413aff5640 1 -- 172.21.15.12:0/3686147792 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5631cddc2c10 con 0x5631cded03e0 2024-04-22T13:34:04.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.993+0000 7f413aff5640 10 client.18981 handle_command_reply: tid=0 2024-04-22T13:34:04.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.993+0000 7f415fddf740 1 client.18981 shutdown 2024-04-22T13:34:04.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.993+0000 7f415fddf740 1 -- 172.21.15.12:0/3686147792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5631cde504d0 msgr2=0x5631cde61fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:04.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.993+0000 7f415fddf740 1 --2- 172.21.15.12:0/3686147792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5631cde504d0 0x5631cde61fa0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f4148009610 tx=0x7f41480a4f10 comp rx=0 tx=0).stop 2024-04-22T13:34:04.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.993+0000 7f415fddf740 1 -- 172.21.15.12:0/3686147792 shutdown_connections 2024-04-22T13:34:04.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.993+0000 7f415fddf740 1 --2- 172.21.15.12:0/3686147792 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x5631cded03e0 0x5631cded27d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.996 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.993+0000 7f415fddf740 1 --2- 172.21.15.12:0/3686147792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5631cde59a00 0x5631cde53460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.993+0000 7f413aff5640 0 client.18981 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:34:04.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.993+0000 7f415fddf740 1 --2- 172.21.15.12:0/3686147792 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5631cde50e80 0x5631cde52f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.993+0000 7f415fddf740 1 --2- 172.21.15.12:0/3686147792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5631cde504d0 0x5631cde61fa0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.993+0000 7f415fddf740 1 -- 172.21.15.12:0/3686147792 >> 172.21.15.12:0/3686147792 conn(0x5631cde4aba0 msgr2=0x5631cde49c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:04.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.993+0000 7f415fddf740 1 -- 172.21.15.12:0/3686147792 shutdown_connections 2024-04-22T13:34:04.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.994+0000 7f415fddf740 1 -- 172.21.15.12:0/3686147792 wait complete. 2024-04-22T13:34:04.997 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.994+0000 7f415fddf740 20 client.18981 trim_cache size 0 max 16384 2024-04-22T13:34:04.997 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:34:04.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.994+0000 7f415d1fd640 1 -- 172.21.15.12:0/1121064036 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f41400a3d50 msgr2=0x7f41400a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:04.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.994+0000 7f415d1fd640 1 --2- 172.21.15.12:0/1121064036 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f41400a3d50 0x7f41400a6210 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4150004af0 tx=0x7f415002e040 comp rx=0 tx=0).stop 2024-04-22T13:34:04.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.994+0000 7f415d1fd640 1 -- 172.21.15.12:0/1121064036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4158102190 msgr2=0x7f4158074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:04.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.994+0000 7f415d1fd640 1 --2- 172.21.15.12:0/1121064036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4158102190 0x7f4158074250 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f414800d770 tx=0x7f414800dc30 comp rx=0 tx=0).stop 2024-04-22T13:34:04.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.994+0000 7f415d1fd640 1 -- 172.21.15.12:0/1121064036 shutdown_connections 2024-04-22T13:34:04.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.994+0000 7f415d1fd640 1 --2- 172.21.15.12:0/1121064036 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f41400a3d50 0x7f41400a6210 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.994+0000 7f415d1fd640 1 --2- 172.21.15.12:0/1121064036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4158074cd0 0x7f41581a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.994+0000 7f415d1fd640 1 --2- 172.21.15.12:0/1121064036 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4158107260 0x7f4158074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.994+0000 7f415d1fd640 1 --2- 172.21.15.12:0/1121064036 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4158102190 0x7f4158074250 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:04.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.994+0000 7f415d1fd640 1 -- 172.21.15.12:0/1121064036 >> 172.21.15.12:0/1121064036 conn(0x7f41580fd990 msgr2=0x7f41580ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:04.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.994+0000 7f415d1fd640 1 -- 172.21.15.12:0/1121064036 shutdown_connections 2024-04-22T13:34:04.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:04.994+0000 7f415d1fd640 1 -- 172.21.15.12:0/1121064036 wait complete. 2024-04-22T13:34:05.001 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:34:05.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.109+0000 7f3b2a338640 1 -- 172.21.15.12:0/649974363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b241014a0 msgr2=0x7f3b24106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.109+0000 7f3b2a338640 1 --2- 172.21.15.12:0/649974363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b241014a0 0x7f3b24106bd0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f3b18009900 tx=0x7f3b1802bc30 comp rx=0 tx=0).stop 2024-04-22T13:34:05.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.109+0000 7f3b2a338640 1 -- 172.21.15.12:0/649974363 shutdown_connections 2024-04-22T13:34:05.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.109+0000 7f3b2a338640 1 --2- 172.21.15.12:0/649974363 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b241072b0 0x7f3b2410eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.109+0000 7f3b2a338640 1 --2- 172.21.15.12:0/649974363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b241014a0 0x7f3b24106bd0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.109+0000 7f3b2a338640 1 --2- 172.21.15.12:0/649974363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b24100af0 0x7f3b24100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.109+0000 7f3b2a338640 1 -- 172.21.15.12:0/649974363 >> 172.21.15.12:0/649974363 conn(0x7f3b240760e0 msgr2=0x7f3b240764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:05.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.109+0000 7f3b2a338640 1 -- 172.21.15.12:0/649974363 shutdown_connections 2024-04-22T13:34:05.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.109+0000 7f3b2a338640 1 -- 172.21.15.12:0/649974363 wait complete. 2024-04-22T13:34:05.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.110+0000 7f3b2a338640 1 Processor -- start 2024-04-22T13:34:05.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.110+0000 7f3b2a338640 1 -- start start 2024-04-22T13:34:05.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.110+0000 7f3b2a338640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b24100af0 0x7f3b24101a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.110+0000 7f3b2a338640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b241072b0 0x7f3b24101fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.110+0000 7f3b2a338640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b241024e0 0x7f3b241a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.110+0000 7f3b2a338640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3b24112230 con 0x7f3b241072b0 2024-04-22T13:34:05.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.110+0000 7f3b2a338640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3b241120b0 con 0x7f3b241024e0 2024-04-22T13:34:05.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.110+0000 7f3b2a338640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3b241123b0 con 0x7f3b24100af0 2024-04-22T13:34:05.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.110+0000 7f3b237fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b241072b0 0x7f3b24101fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.110+0000 7f3b237fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b241072b0 0x7f3b24101fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36592/0 (socket says 172.21.15.12:36592) 2024-04-22T13:34:05.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.110+0000 7f3b237fe640 1 -- 172.21.15.12:0/991956716 learned_addr learned my addr 172.21.15.12:0/991956716 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:05.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.111+0000 7f3b23fff640 1 --2- 172.21.15.12:0/991956716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b24100af0 0x7f3b24101a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.111+0000 7f3b288d5640 1 --2- 172.21.15.12:0/991956716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b241024e0 0x7f3b241a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.111+0000 7f3b237fe640 1 -- 172.21.15.12:0/991956716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b24100af0 msgr2=0x7f3b24101a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.111+0000 7f3b237fe640 1 --2- 172.21.15.12:0/991956716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b24100af0 0x7f3b24101a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.111+0000 7f3b237fe640 1 -- 172.21.15.12:0/991956716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b241024e0 msgr2=0x7f3b241a69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.111+0000 7f3b237fe640 1 --2- 172.21.15.12:0/991956716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b241024e0 0x7f3b241a69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.111+0000 7f3b237fe640 1 -- 172.21.15.12:0/991956716 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b241a70c0 con 0x7f3b241072b0 2024-04-22T13:34:05.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.111+0000 7f3b237fe640 1 --2- 172.21.15.12:0/991956716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b241072b0 0x7f3b24101fa0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f3b18036040 tx=0x7f3b1802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:05.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.111+0000 7f3b217fa640 1 -- 172.21.15.12:0/991956716 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b18043020 con 0x7f3b241072b0 2024-04-22T13:34:05.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.111+0000 7f3b2a338640 1 -- 172.21.15.12:0/991956716 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b241a7350 con 0x7f3b241072b0 2024-04-22T13:34:05.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.111+0000 7f3b217fa640 1 -- 172.21.15.12:0/991956716 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b1802cbe0 con 0x7f3b241072b0 2024-04-22T13:34:05.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.111+0000 7f3b217fa640 1 -- 172.21.15.12:0/991956716 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b18003bf0 con 0x7f3b241072b0 2024-04-22T13:34:05.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.111+0000 7f3b2a338640 1 -- 172.21.15.12:0/991956716 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b241a78a0 con 0x7f3b241072b0 2024-04-22T13:34:05.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.113+0000 7f3b217fa640 1 -- 172.21.15.12:0/991956716 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3b18046020 con 0x7f3b241072b0 2024-04-22T13:34:05.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.114+0000 7f3b2cffd740 1 Processor -- start 2024-04-22T13:34:05.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.114+0000 7f3b2cffd740 1 -- start start 2024-04-22T13:34:05.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.114+0000 7f3b2cffd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561511dd54d0 0x561511dd58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.114+0000 7f3b2cffd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561511dd5e80 0x561511dde430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.114+0000 7f3b2cffd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561511ddea00 0x561511de0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.114+0000 7f3b2cffd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561511d3a1f0 con 0x561511dd5e80 2024-04-22T13:34:05.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.114+0000 7f3b2cffd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561511cd74b0 con 0x561511dd54d0 2024-04-22T13:34:05.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.114+0000 7f3b2cffd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561511d361b0 con 0x561511ddea00 2024-04-22T13:34:05.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.114+0000 7f3b217fa640 1 --2- 172.21.15.12:0/991956716 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3b0c0a3d50 0x7f3b0c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.114+0000 7f3b23fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561511dd5e80 0x561511dde430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.114+0000 7f3b23fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561511dd5e80 0x561511dde430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36596/0 (socket says 172.21.15.12:36596) 2024-04-22T13:34:05.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.114+0000 7f3b23fff640 1 -- 172.21.15.12:0/1512748380 learned_addr learned my addr 172.21.15.12:0/1512748380 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:05.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.114+0000 7f3b288d5640 1 --2- 172.21.15.12:0/1512748380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561511dd54d0 0x561511dd58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.114+0000 7f3b288d5640 1 --2- 172.21.15.12:0/991956716 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3b0c0a3d50 0x7f3b0c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.114+0000 7f3b237fe640 1 --2- 172.21.15.12:0/1512748380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561511ddea00 0x561511de0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.114+0000 7f3b217fa640 1 -- 172.21.15.12:0/991956716 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f3b180f2a50 con 0x7f3b241072b0 2024-04-22T13:34:05.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.115+0000 7f3b23fff640 1 -- 172.21.15.12:0/1512748380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561511ddea00 msgr2=0x561511de0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.115+0000 7f3b23fff640 1 --2- 172.21.15.12:0/1512748380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561511ddea00 0x561511de0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.115+0000 7f3b23fff640 1 -- 172.21.15.12:0/1512748380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561511dd54d0 msgr2=0x561511dd58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.115+0000 7f3b23fff640 1 --2- 172.21.15.12:0/1512748380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561511dd54d0 0x561511dd58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.115+0000 7f3b23fff640 1 -- 172.21.15.12:0/1512748380 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561511cc91b0 con 0x561511dd5e80 2024-04-22T13:34:05.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.115+0000 7f3b288d5640 1 --2- 172.21.15.12:0/1512748380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561511dd54d0 0x561511dd58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:34:05.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.115+0000 7f3b237fe640 1 --2- 172.21.15.12:0/1512748380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561511ddea00 0x561511de0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:05.119 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.115+0000 7f3b23fff640 1 --2- 172.21.15.12:0/1512748380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561511dd5e80 0x561511dde430 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f3b1400bcb0 tx=0x7f3b1400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0050670d3e4bdf server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:05.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.115+0000 7f3b288d5640 1 --2- 172.21.15.12:0/991956716 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3b0c0a3d50 0x7f3b0c0a6210 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c009fd0 tx=0x7f3b1c004a40 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:05.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.115+0000 7f3b0a7fc640 1 -- 172.21.15.12:0/1512748380 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b14015070 con 0x561511dd5e80 2024-04-22T13:34:05.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.115+0000 7f3b0a7fc640 1 -- 172.21.15.12:0/1512748380 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b140118f0 con 0x561511dd5e80 2024-04-22T13:34:05.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.115+0000 7f3b0a7fc640 1 -- 172.21.15.12:0/1512748380 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b140108b0 con 0x561511dd5e80 2024-04-22T13:34:05.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.116+0000 7f3b2cffd740 1 -- 172.21.15.12:0/1512748380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561511dd5e80 msgr2=0x561511dde430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.116+0000 7f3b2cffd740 1 --2- 172.21.15.12:0/1512748380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561511dd5e80 0x561511dde430 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f3b1400bcb0 tx=0x7f3b1400f730 comp rx=0 tx=0).stop 2024-04-22T13:34:05.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.116+0000 7f3b2cffd740 1 -- 172.21.15.12:0/1512748380 shutdown_connections 2024-04-22T13:34:05.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.116+0000 7f3b2cffd740 1 --2- 172.21.15.12:0/1512748380 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561511ddea00 0x561511de0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.116+0000 7f3b2cffd740 1 --2- 172.21.15.12:0/1512748380 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561511dd5e80 0x561511dde430 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.116+0000 7f3b2cffd740 1 --2- 172.21.15.12:0/1512748380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561511dd54d0 0x561511dd58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.116+0000 7f3b2cffd740 1 -- 172.21.15.12:0/1512748380 >> 172.21.15.12:0/1512748380 conn(0x561511dcfba0 msgr2=0x561511dcff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:05.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.116+0000 7f3b2cffd740 1 -- 172.21.15.12:0/1512748380 shutdown_connections 2024-04-22T13:34:05.120 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.116+0000 7f3b2cffd740 1 -- 172.21.15.12:0/1512748380 wait complete. 2024-04-22T13:34:05.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.116+0000 7f3b2cffd740 1 -- start start 2024-04-22T13:34:05.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.116+0000 7f3b2cffd740 1 Processor -- start 2024-04-22T13:34:05.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.117+0000 7f3b2cffd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561511dd54d0 0x561511dde300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.117+0000 7f3b2cffd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561511dd5e80 0x561511ddb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.117+0000 7f3b2cffd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561511ddea00 0x561511ddbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.117+0000 7f3b2cffd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x561511de1970 con 0x561511ddea00 2024-04-22T13:34:05.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.117+0000 7f3b2cffd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561511cf3500 con 0x561511dd54d0 2024-04-22T13:34:05.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.117+0000 7f3b2cffd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x561511de1af0 con 0x561511dd5e80 2024-04-22T13:34:05.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.117+0000 7f3b2cffd740 10 client.0 _ll_register_callbacks cb 0x561511cc1b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:05.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.117+0000 7f3b288d5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561511ddea00 0x561511ddbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.117+0000 7f3b288d5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561511ddea00 0x561511ddbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36610/0 (socket says 172.21.15.12:36610) 2024-04-22T13:34:05.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.117+0000 7f3b288d5640 1 -- 172.21.15.12:0/2678009546 learned_addr learned my addr 172.21.15.12:0/2678009546 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:05.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.117+0000 7f3b23fff640 1 --2- 172.21.15.12:0/2678009546 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561511dd54d0 0x561511dde300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.117+0000 7f3b237fe640 1 --2- 172.21.15.12:0/2678009546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561511dd5e80 0x561511ddb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.117+0000 7f3b288d5640 1 -- 172.21.15.12:0/2678009546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561511dd5e80 msgr2=0x561511ddb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.117+0000 7f3b288d5640 1 --2- 172.21.15.12:0/2678009546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561511dd5e80 0x561511ddb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.117+0000 7f3b288d5640 1 -- 172.21.15.12:0/2678009546 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561511dd54d0 msgr2=0x561511dde300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.117+0000 7f3b288d5640 1 --2- 172.21.15.12:0/2678009546 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561511dd54d0 0x561511dde300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.117+0000 7f3b288d5640 1 -- 172.21.15.12:0/2678009546 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561511a4c3f0 con 0x561511ddea00 2024-04-22T13:34:05.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.117+0000 7f3b288d5640 1 --2- 172.21.15.12:0/2678009546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561511ddea00 0x561511ddbc10 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c014810 tx=0x7f3b1c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d859b4d87183ee5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:05.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.117+0000 7f3b037fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:34:05.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.117+0000 7f3b037fe640 1 -- 172.21.15.12:0/2678009546 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c00fae0 con 0x561511ddea00 2024-04-22T13:34:05.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.117+0000 7f3b037fe640 1 -- 172.21.15.12:0/2678009546 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f3b1c00fc80 con 0x561511ddea00 2024-04-22T13:34:05.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.117+0000 7f3b2cffd740 1 -- 172.21.15.12:0/2678009546 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b140118f0 con 0x561511ddea00 2024-04-22T13:34:05.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.118+0000 7f3b037fe640 1 -- 172.21.15.12:0/2678009546 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3b1c00fe80 con 0x561511ddea00 2024-04-22T13:34:05.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.118+0000 7f3b2cffd740 10 client.18993 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:05.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.118+0000 7f3b2cffd740 1 -- 172.21.15.12:0/2678009546 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561511b167d0 con 0x561511ddea00 2024-04-22T13:34:05.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.118+0000 7f3b037fe640 1 -- 172.21.15.12:0/2678009546 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b140118f0 con 0x561511ddea00 2024-04-22T13:34:05.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.118+0000 7f3b037fe640 1 -- 172.21.15.12:0/2678009546 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3b1c031020 con 0x561511ddea00 2024-04-22T13:34:05.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.118+0000 7f3b2cffd740 10 client.18993 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:05.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.118+0000 7f3b2cffd740 10 client.18993 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:34:05.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.118+0000 7f3b2cffd740 20 client.18993 populate_metadata read hostname 'smithi012' 2024-04-22T13:34:05.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.118+0000 7f3b2cffd740 1 --2- 172.21.15.12:0/2678009546 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x561511deeb40 0x561511df0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.118+0000 7f3b2cffd740 4 client.18993 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:34:05.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.118+0000 7f3b2cffd740 1 -- 172.21.15.12:0/2678009546 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561511cd9930 con 0x561511deeb40 2024-04-22T13:34:05.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.118+0000 7f3b23fff640 1 --2- 172.21.15.12:0/2678009546 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x561511deeb40 0x561511df0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.119+0000 7f3b037fe640 10 client.18993 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:34:05.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.119+0000 7f3b23fff640 1 --2- 172.21.15.12:0/2678009546 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x561511deeb40 0x561511df0f90 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3b6ff9adfb22f690 server_cookie=36c8e62223d6a752 in_seq=0 out_seq=0 2024-04-22T13:34:05.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.119+0000 7f3b037fe640 1 -- 172.21.15.12:0/2678009546 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561511cd9930 con 0x561511deeb40 2024-04-22T13:34:05.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.119+0000 7f3b037fe640 10 client.18993 handle_command_reply: tid=0 2024-04-22T13:34:05.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.119+0000 7f3b2cffd740 1 client.18993 shutdown 2024-04-22T13:34:05.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.120+0000 7f3b2cffd740 1 -- 172.21.15.12:0/2678009546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561511ddea00 msgr2=0x561511ddbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.120+0000 7f3b2cffd740 1 --2- 172.21.15.12:0/2678009546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561511ddea00 0x561511ddbc10 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c014810 tx=0x7f3b1c004ab0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.123 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.120+0000 7f3b2cffd740 1 -- 172.21.15.12:0/2678009546 shutdown_connections 2024-04-22T13:34:05.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.120+0000 7f3b2cffd740 1 --2- 172.21.15.12:0/2678009546 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x561511deeb40 0x561511df0f90 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.120+0000 7f3b2cffd740 1 --2- 172.21.15.12:0/2678009546 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561511ddea00 0x561511ddbc10 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.120+0000 7f3b037fe640 0 client.18993 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:34:05.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.120+0000 7f3b2cffd740 1 --2- 172.21.15.12:0/2678009546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561511dd5e80 0x561511ddb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.120+0000 7f3b2cffd740 1 --2- 172.21.15.12:0/2678009546 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561511dd54d0 0x561511dde300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.120+0000 7f3b2cffd740 1 -- 172.21.15.12:0/2678009546 >> 172.21.15.12:0/2678009546 conn(0x561511dcfba0 msgr2=0x561511dd18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:05.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.120+0000 7f3b2cffd740 1 -- 172.21.15.12:0/2678009546 shutdown_connections 2024-04-22T13:34:05.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.120+0000 7f3b2cffd740 1 -- 172.21.15.12:0/2678009546 wait complete. 2024-04-22T13:34:05.124 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.120+0000 7f3b2cffd740 20 client.18993 trim_cache size 0 max 16384 2024-04-22T13:34:05.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.133+0000 7f3b2cffd740 1 Processor -- start 2024-04-22T13:34:05.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.133+0000 7f3b2cffd740 1 -- start start 2024-04-22T13:34:05.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.133+0000 7f3b2cffd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561511dd54d0 0x561511dea860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.133+0000 7f3b2cffd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561511dd5e80 0x561511ddddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.133+0000 7f3b2cffd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561511ddea00 0x561511dde300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.133+0000 7f3b2cffd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3b241afd60 con 0x561511dd54d0 2024-04-22T13:34:05.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.133+0000 7f3b2cffd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3b241b18a0 con 0x561511dd5e80 2024-04-22T13:34:05.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.133+0000 7f3b2cffd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3b241b0430 con 0x561511ddea00 2024-04-22T13:34:05.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.133+0000 7f3b23fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561511dd54d0 0x561511dea860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.133+0000 7f3b23fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561511dd54d0 0x561511dea860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36622/0 (socket says 172.21.15.12:36622) 2024-04-22T13:34:05.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.133+0000 7f3b23fff640 1 -- 172.21.15.12:0/3986732204 learned_addr learned my addr 172.21.15.12:0/3986732204 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:05.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.134+0000 7f3b288d5640 1 --2- 172.21.15.12:0/3986732204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561511ddea00 0x561511dde300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.136 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.134+0000 7f3b23fff640 1 -- 172.21.15.12:0/3986732204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561511ddea00 msgr2=0x561511dde300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.134+0000 7f3b23fff640 1 --2- 172.21.15.12:0/3986732204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561511ddea00 0x561511dde300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.134+0000 7f3b237fe640 1 --2- 172.21.15.12:0/3986732204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561511dd5e80 0x561511ddddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.134+0000 7f3b23fff640 1 -- 172.21.15.12:0/3986732204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561511dd5e80 msgr2=0x561511ddddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.134+0000 7f3b23fff640 1 --2- 172.21.15.12:0/3986732204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561511dd5e80 0x561511ddddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.134+0000 7f3b23fff640 1 -- 172.21.15.12:0/3986732204 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561511dd9be0 con 0x561511dd54d0 2024-04-22T13:34:05.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.134+0000 7f3b288d5640 1 --2- 172.21.15.12:0/3986732204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561511ddea00 0x561511dde300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:05.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.134+0000 7f3b23fff640 1 --2- 172.21.15.12:0/3986732204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561511dd54d0 0x561511dea860 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f3b14014040 tx=0x7f3b14010470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9537c428ab821375 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:05.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.134+0000 7f3b09ffb640 1 -- 172.21.15.12:0/3986732204 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b14015070 con 0x561511dd54d0 2024-04-22T13:34:05.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.134+0000 7f3b09ffb640 1 -- 172.21.15.12:0/3986732204 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b140054f0 con 0x561511dd54d0 2024-04-22T13:34:05.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.134+0000 7f3b09ffb640 1 -- 172.21.15.12:0/3986732204 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b14004b70 con 0x561511dd54d0 2024-04-22T13:34:05.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.135+0000 7f3b2cffd740 1 -- 172.21.15.12:0/3986732204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561511dd54d0 msgr2=0x561511dea860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.135+0000 7f3b2cffd740 1 --2- 172.21.15.12:0/3986732204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561511dd54d0 0x561511dea860 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f3b14014040 tx=0x7f3b14010470 comp rx=0 tx=0).stop 2024-04-22T13:34:05.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.135+0000 7f3b2cffd740 1 -- 172.21.15.12:0/3986732204 shutdown_connections 2024-04-22T13:34:05.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.135+0000 7f3b2cffd740 1 --2- 172.21.15.12:0/3986732204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561511ddea00 0x561511dde300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.135+0000 7f3b2cffd740 1 --2- 172.21.15.12:0/3986732204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561511dd5e80 0x561511ddddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.135+0000 7f3b2cffd740 1 --2- 172.21.15.12:0/3986732204 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561511dd54d0 0x561511dea860 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.135+0000 7f3b2cffd740 1 -- 172.21.15.12:0/3986732204 >> 172.21.15.12:0/3986732204 conn(0x561511dcfba0 msgr2=0x561511dd1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:05.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.135+0000 7f3b2cffd740 1 -- 172.21.15.12:0/3986732204 shutdown_connections 2024-04-22T13:34:05.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.135+0000 7f3b2cffd740 1 -- 172.21.15.12:0/3986732204 wait complete. 2024-04-22T13:34:05.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.135+0000 7f3b2cffd740 1 -- start start 2024-04-22T13:34:05.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.135+0000 7f3b2cffd740 1 Processor -- start 2024-04-22T13:34:05.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.135+0000 7f3b2cffd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561511dd54d0 0x561511de7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.135+0000 7f3b2cffd740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561511dd5e80 0x561511dd8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.135+0000 7f3b2cffd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561511ddea00 0x561511dd85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.135+0000 7f3b2cffd740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3b241b11d0 con 0x561511dd5e80 2024-04-22T13:34:05.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.135+0000 7f3b2cffd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3b241b0b00 con 0x561511dd54d0 2024-04-22T13:34:05.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.135+0000 7f3b2cffd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3b241add60 con 0x561511ddea00 2024-04-22T13:34:05.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.135+0000 7f3b2cffd740 10 client.0 _ll_register_callbacks cb 0x7f3b1c014720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:05.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.136+0000 7f3b237fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561511dd5e80 0x561511dd8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.136+0000 7f3b237fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561511dd5e80 0x561511dd8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36634/0 (socket says 172.21.15.12:36634) 2024-04-22T13:34:05.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.136+0000 7f3b237fe640 1 -- 172.21.15.12:0/996201267 learned_addr learned my addr 172.21.15.12:0/996201267 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:05.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.136+0000 7f3b23fff640 1 --2- 172.21.15.12:0/996201267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561511dd54d0 0x561511de7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.136+0000 7f3b237fe640 1 -- 172.21.15.12:0/996201267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561511ddea00 msgr2=0x561511dd85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.136+0000 7f3b288d5640 1 --2- 172.21.15.12:0/996201267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561511ddea00 0x561511dd85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.136+0000 7f3b237fe640 1 --2- 172.21.15.12:0/996201267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561511ddea00 0x561511dd85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.136+0000 7f3b237fe640 1 -- 172.21.15.12:0/996201267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561511dd54d0 msgr2=0x561511de7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.136+0000 7f3b237fe640 1 --2- 172.21.15.12:0/996201267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561511dd54d0 0x561511de7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.136+0000 7f3b237fe640 1 -- 172.21.15.12:0/996201267 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561511dd8df0 con 0x561511dd5e80 2024-04-22T13:34:05.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.136+0000 7f3b23fff640 1 --2- 172.21.15.12:0/996201267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561511dd54d0 0x561511de7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:05.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.136+0000 7f3b288d5640 1 --2- 172.21.15.12:0/996201267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561511ddea00 0x561511dd85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:05.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.136+0000 7f3b237fe640 1 --2- 172.21.15.12:0/996201267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561511dd5e80 0x561511dd8080 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f3b180f02d0 tx=0x7f3b180036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9713cd60cdd4678d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:05.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.136+0000 7f3b037fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:34:05.139 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.136+0000 7f3b037fe640 1 -- 172.21.15.12:0/996201267 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b180bf1c0 con 0x561511dd5e80 2024-04-22T13:34:05.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.136+0000 7f3b037fe640 1 -- 172.21.15.12:0/996201267 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f3b180bcdc0 con 0x561511dd5e80 2024-04-22T13:34:05.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.136+0000 7f3b2cffd740 1 -- 172.21.15.12:0/996201267 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b140054f0 con 0x561511dd5e80 2024-04-22T13:34:05.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.136+0000 7f3b037fe640 1 -- 172.21.15.12:0/996201267 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3b180c2e70 con 0x561511dd5e80 2024-04-22T13:34:05.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.136+0000 7f3b2cffd740 10 client.18999 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:05.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.136+0000 7f3b2cffd740 1 -- 172.21.15.12:0/996201267 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561511e55040 con 0x561511dd5e80 2024-04-22T13:34:05.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.137+0000 7f3b037fe640 1 -- 172.21.15.12:0/996201267 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b140054f0 con 0x561511dd5e80 2024-04-22T13:34:05.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.137+0000 7f3b037fe640 1 -- 172.21.15.12:0/996201267 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3b180b9270 con 0x561511dd5e80 2024-04-22T13:34:05.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.137+0000 7f3b2cffd740 10 client.18999 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:05.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.137+0000 7f3b2cffd740 10 client.18999 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:34:05.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.137+0000 7f3b2cffd740 20 client.18999 populate_metadata read hostname 'smithi012' 2024-04-22T13:34:05.140 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.137+0000 7f3b2cffd740 1 --2- 172.21.15.12:0/996201267 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x561511e557c0 0x561511e57be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.137+0000 7f3b2cffd740 4 client.18999 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:34:05.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.137+0000 7f3b2cffd740 1 -- 172.21.15.12:0/996201267 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561511d47c10 con 0x561511e557c0 2024-04-22T13:34:05.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.137+0000 7f3b23fff640 1 --2- 172.21.15.12:0/996201267 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x561511e557c0 0x561511e57be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.137+0000 7f3b23fff640 1 --2- 172.21.15.12:0/996201267 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x561511e557c0 0x561511e57be0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=919183930a66a25a server_cookie=f754e171edea50ca in_seq=0 out_seq=0 2024-04-22T13:34:05.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.137+0000 7f3b037fe640 10 client.18999 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:34:05.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.138+0000 7f3b037fe640 1 -- 172.21.15.12:0/996201267 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561511d47c10 con 0x561511e557c0 2024-04-22T13:34:05.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.138+0000 7f3b037fe640 10 client.18999 handle_command_reply: tid=0 2024-04-22T13:34:05.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.138+0000 7f3b2cffd740 1 client.18999 shutdown 2024-04-22T13:34:05.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.138+0000 7f3b2cffd740 1 -- 172.21.15.12:0/996201267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561511dd5e80 msgr2=0x561511dd8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.138+0000 7f3b2cffd740 1 --2- 172.21.15.12:0/996201267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561511dd5e80 0x561511dd8080 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f3b180f02d0 tx=0x7f3b180036f0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.138+0000 7f3b2cffd740 1 -- 172.21.15.12:0/996201267 shutdown_connections 2024-04-22T13:34:05.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.138+0000 7f3b2cffd740 1 --2- 172.21.15.12:0/996201267 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x561511e557c0 0x561511e57be0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.138+0000 7f3b2cffd740 1 --2- 172.21.15.12:0/996201267 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x561511ddea00 0x561511dd85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.141 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.138+0000 7f3b2cffd740 1 --2- 172.21.15.12:0/996201267 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561511dd5e80 0x561511dd8080 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.138+0000 7f3b037fe640 0 client.18999 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:34:05.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.138+0000 7f3b2cffd740 1 --2- 172.21.15.12:0/996201267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561511dd54d0 0x561511de7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.138+0000 7f3b2cffd740 1 -- 172.21.15.12:0/996201267 >> 172.21.15.12:0/996201267 conn(0x561511dcfba0 msgr2=0x561511dce370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:05.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.138+0000 7f3b2cffd740 1 -- 172.21.15.12:0/996201267 shutdown_connections 2024-04-22T13:34:05.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.138+0000 7f3b2cffd740 1 -- 172.21.15.12:0/996201267 wait complete. 2024-04-22T13:34:05.142 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.138+0000 7f3b2cffd740 20 client.18999 trim_cache size 0 max 16384 2024-04-22T13:34:05.142 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:34:05.143 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.138+0000 7f3b2a338640 1 -- 172.21.15.12:0/991956716 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3b0c0a3d50 msgr2=0x7f3b0c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.138+0000 7f3b2a338640 1 --2- 172.21.15.12:0/991956716 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3b0c0a3d50 0x7f3b0c0a6210 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c009fd0 tx=0x7f3b1c004a40 comp rx=0 tx=0).stop 2024-04-22T13:34:05.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.138+0000 7f3b2a338640 1 -- 172.21.15.12:0/991956716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b241072b0 msgr2=0x7f3b24101fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.138+0000 7f3b2a338640 1 --2- 172.21.15.12:0/991956716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b241072b0 0x7f3b24101fa0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f3b18036040 tx=0x7f3b1802c7e0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.138+0000 7f3b2a338640 1 -- 172.21.15.12:0/991956716 shutdown_connections 2024-04-22T13:34:05.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.138+0000 7f3b2a338640 1 --2- 172.21.15.12:0/991956716 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3b0c0a3d50 0x7f3b0c0a6210 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.138+0000 7f3b2a338640 1 --2- 172.21.15.12:0/991956716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b241024e0 0x7f3b241a69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.138+0000 7f3b2a338640 1 --2- 172.21.15.12:0/991956716 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b241072b0 0x7f3b24101fa0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.138+0000 7f3b2a338640 1 --2- 172.21.15.12:0/991956716 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b24100af0 0x7f3b24101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.139+0000 7f3b2a338640 1 -- 172.21.15.12:0/991956716 >> 172.21.15.12:0/991956716 conn(0x7f3b240760e0 msgr2=0x7f3b240fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:05.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.139+0000 7f3b2a338640 1 -- 172.21.15.12:0/991956716 shutdown_connections 2024-04-22T13:34:05.144 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.139+0000 7f3b2a338640 1 -- 172.21.15.12:0/991956716 wait complete. 2024-04-22T13:34:05.145 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:34:05.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.253+0000 7f5d49b8f640 1 -- 172.21.15.12:0/632516761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d4410e960 msgr2=0x7f5d44110d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.253+0000 7f5d49b8f640 1 --2- 172.21.15.12:0/632516761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d4410e960 0x7f5d44110d50 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009ca0 tx=0x7f5d3c02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.253+0000 7f5d49b8f640 1 -- 172.21.15.12:0/632516761 shutdown_connections 2024-04-22T13:34:05.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.253+0000 7f5d49b8f640 1 --2- 172.21.15.12:0/632516761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d4410e960 0x7f5d44110d50 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.253+0000 7f5d49b8f640 1 --2- 172.21.15.12:0/632516761 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d4406aef0 0x7f5d4410e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.253+0000 7f5d49b8f640 1 --2- 172.21.15.12:0/632516761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d4406a5d0 0x7f5d4406a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.253+0000 7f5d49b8f640 1 -- 172.21.15.12:0/632516761 >> 172.21.15.12:0/632516761 conn(0x7f5d44077eb0 msgr2=0x7f5d440782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:05.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.253+0000 7f5d49b8f640 1 -- 172.21.15.12:0/632516761 shutdown_connections 2024-04-22T13:34:05.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.253+0000 7f5d49b8f640 1 -- 172.21.15.12:0/632516761 wait complete. 2024-04-22T13:34:05.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.254+0000 7f5d49b8f640 1 Processor -- start 2024-04-22T13:34:05.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.254+0000 7f5d49b8f640 1 -- start start 2024-04-22T13:34:05.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.254+0000 7f5d49b8f640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d4406a5d0 0x7f5d441a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.254+0000 7f5d49b8f640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d4406aef0 0x7f5d441a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.254+0000 7f5d49b8f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d4410e960 0x7f5d4419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.254+0000 7f5d49b8f640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5d44114390 con 0x7f5d4406a5d0 2024-04-22T13:34:05.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.254+0000 7f5d49b8f640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5d44114210 con 0x7f5d4406aef0 2024-04-22T13:34:05.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.254+0000 7f5d49b8f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5d44114510 con 0x7f5d4410e960 2024-04-22T13:34:05.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.254+0000 7f5d437fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d4406a5d0 0x7f5d441a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.254+0000 7f5d437fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d4406a5d0 0x7f5d441a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36650/0 (socket says 172.21.15.12:36650) 2024-04-22T13:34:05.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.254+0000 7f5d437fe640 1 -- 172.21.15.12:0/1026396659 learned_addr learned my addr 172.21.15.12:0/1026396659 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:05.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.254+0000 7f5d42ffd640 1 --2- 172.21.15.12:0/1026396659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d4406aef0 0x7f5d441a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.254+0000 7f5d43fff640 1 --2- 172.21.15.12:0/1026396659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d4410e960 0x7f5d4419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.254+0000 7f5d437fe640 1 -- 172.21.15.12:0/1026396659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d4410e960 msgr2=0x7f5d4419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.255+0000 7f5d437fe640 1 --2- 172.21.15.12:0/1026396659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d4410e960 0x7f5d4419a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.255+0000 7f5d437fe640 1 -- 172.21.15.12:0/1026396659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d4406aef0 msgr2=0x7f5d441a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.255+0000 7f5d437fe640 1 --2- 172.21.15.12:0/1026396659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d4406aef0 0x7f5d441a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.255+0000 7f5d437fe640 1 -- 172.21.15.12:0/1026396659 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4419af90 con 0x7f5d4406a5d0 2024-04-22T13:34:05.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.255+0000 7f5d42ffd640 1 --2- 172.21.15.12:0/1026396659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d4406aef0 0x7f5d441a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:05.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.255+0000 7f5d437fe640 1 --2- 172.21.15.12:0/1026396659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d4406a5d0 0x7f5d441a0610 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400d950 tx=0x7f5d3400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:05.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.255+0000 7f5d40ff9640 1 -- 172.21.15.12:0/1026396659 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d34017070 con 0x7f5d4406a5d0 2024-04-22T13:34:05.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.255+0000 7f5d40ff9640 1 -- 172.21.15.12:0/1026396659 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d34009c30 con 0x7f5d4406a5d0 2024-04-22T13:34:05.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.255+0000 7f5d49b8f640 1 -- 172.21.15.12:0/1026396659 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d4419b280 con 0x7f5d4406a5d0 2024-04-22T13:34:05.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.255+0000 7f5d40ff9640 1 -- 172.21.15.12:0/1026396659 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3400f620 con 0x7f5d4406a5d0 2024-04-22T13:34:05.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.255+0000 7f5d49b8f640 1 -- 172.21.15.12:0/1026396659 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d441a7590 con 0x7f5d4406a5d0 2024-04-22T13:34:05.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.257+0000 7f5d40ff9640 1 -- 172.21.15.12:0/1026396659 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5d34010460 con 0x7f5d4406a5d0 2024-04-22T13:34:05.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.257+0000 7f5d4c77d740 1 Processor -- start 2024-04-22T13:34:05.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.258+0000 7f5d4c77d740 1 -- start start 2024-04-22T13:34:05.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.258+0000 7f5d4c77d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55935da894d0 0x55935da898b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.258+0000 7f5d4c77d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55935da89e80 0x55935da92430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.258+0000 7f5d4c77d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55935da92a00 0x55935da94e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.258+0000 7f5d4c77d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55935d9ee1f0 con 0x55935da89e80 2024-04-22T13:34:05.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.258+0000 7f5d4c77d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55935d98b4b0 con 0x55935da894d0 2024-04-22T13:34:05.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.258+0000 7f5d4c77d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55935d9ea1b0 con 0x55935da92a00 2024-04-22T13:34:05.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.258+0000 7f5d40ff9640 1 --2- 172.21.15.12:0/1026396659 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5d2c0a3d20 0x7f5d2c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.258+0000 7f5d437fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55935da89e80 0x55935da92430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.258+0000 7f5d437fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55935da89e80 0x55935da92430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36654/0 (socket says 172.21.15.12:36654) 2024-04-22T13:34:05.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.258+0000 7f5d437fe640 1 -- 172.21.15.12:0/4278081554 learned_addr learned my addr 172.21.15.12:0/4278081554 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:05.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.258+0000 7f5d43fff640 1 --2- 172.21.15.12:0/1026396659 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5d2c0a3d20 0x7f5d2c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.258+0000 7f5d42ffd640 1 --2- 172.21.15.12:0/4278081554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55935da92a00 0x55935da94e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.258+0000 7f5d43fff640 1 --2- 172.21.15.12:0/4278081554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55935da894d0 0x55935da898b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.258+0000 7f5d40ff9640 1 -- 172.21.15.12:0/1026396659 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f5d340d0550 con 0x7f5d4406a5d0 2024-04-22T13:34:05.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.258+0000 7f5d437fe640 1 -- 172.21.15.12:0/4278081554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55935da92a00 msgr2=0x55935da94e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.258+0000 7f5d437fe640 1 --2- 172.21.15.12:0/4278081554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55935da92a00 0x55935da94e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.258+0000 7f5d437fe640 1 -- 172.21.15.12:0/4278081554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55935da894d0 msgr2=0x55935da898b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.258+0000 7f5d437fe640 1 --2- 172.21.15.12:0/4278081554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55935da894d0 0x55935da898b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.258+0000 7f5d437fe640 1 -- 172.21.15.12:0/4278081554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55935d97d1b0 con 0x55935da89e80 2024-04-22T13:34:05.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.258+0000 7f5d42ffd640 1 --2- 172.21.15.12:0/4278081554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55935da92a00 0x55935da94e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:05.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.258+0000 7f5d43fff640 1 --2- 172.21.15.12:0/1026396659 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5d2c0a3d20 0x7f5d2c0a61e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c005e30 tx=0x7f5d3c009880 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:05.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.258+0000 7f5d437fe640 1 --2- 172.21.15.12:0/4278081554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55935da89e80 0x55935da92430 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f5d340cdbd0 tx=0x7f5d3401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20cbadd534d4e382 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:05.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.259+0000 7f5d29ffb640 1 -- 172.21.15.12:0/4278081554 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d34020030 con 0x55935da89e80 2024-04-22T13:34:05.262 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.259+0000 7f5d29ffb640 1 -- 172.21.15.12:0/4278081554 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d34009de0 con 0x55935da89e80 2024-04-22T13:34:05.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.259+0000 7f5d29ffb640 1 -- 172.21.15.12:0/4278081554 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d340a4c90 con 0x55935da89e80 2024-04-22T13:34:05.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.259+0000 7f5d4c77d740 1 -- 172.21.15.12:0/4278081554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55935da89e80 msgr2=0x55935da92430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.259+0000 7f5d4c77d740 1 --2- 172.21.15.12:0/4278081554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55935da89e80 0x55935da92430 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f5d340cdbd0 tx=0x7f5d3401e000 comp rx=0 tx=0).stop 2024-04-22T13:34:05.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.259+0000 7f5d4c77d740 1 -- 172.21.15.12:0/4278081554 shutdown_connections 2024-04-22T13:34:05.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.259+0000 7f5d4c77d740 1 --2- 172.21.15.12:0/4278081554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55935da92a00 0x55935da94e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.259+0000 7f5d4c77d740 1 --2- 172.21.15.12:0/4278081554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55935da89e80 0x55935da92430 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.259+0000 7f5d4c77d740 1 --2- 172.21.15.12:0/4278081554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55935da894d0 0x55935da898b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.259+0000 7f5d4c77d740 1 -- 172.21.15.12:0/4278081554 >> 172.21.15.12:0/4278081554 conn(0x55935da83ba0 msgr2=0x55935da83f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:05.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.259+0000 7f5d4c77d740 1 -- 172.21.15.12:0/4278081554 shutdown_connections 2024-04-22T13:34:05.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.259+0000 7f5d4c77d740 1 -- 172.21.15.12:0/4278081554 wait complete. 2024-04-22T13:34:05.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.260+0000 7f5d4c77d740 1 -- start start 2024-04-22T13:34:05.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.260+0000 7f5d4c77d740 1 Processor -- start 2024-04-22T13:34:05.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.260+0000 7f5d4c77d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55935da894d0 0x55935da92270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.263 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.260+0000 7f5d4c77d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55935da89e80 0x55935da8f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.260+0000 7f5d4c77d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55935da92a00 0x55935da8fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.260+0000 7f5d4c77d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55935da95910 con 0x55935da894d0 2024-04-22T13:34:05.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.260+0000 7f5d4c77d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55935d9a7500 con 0x55935da89e80 2024-04-22T13:34:05.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.260+0000 7f5d4c77d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55935da95a90 con 0x55935da92a00 2024-04-22T13:34:05.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.260+0000 7f5d437fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55935da894d0 0x55935da92270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.260+0000 7f5d4c77d740 10 client.0 _ll_register_callbacks cb 0x55935d975b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:05.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.260+0000 7f5d437fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55935da894d0 0x55935da92270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36664/0 (socket says 172.21.15.12:36664) 2024-04-22T13:34:05.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.260+0000 7f5d437fe640 1 -- 172.21.15.12:0/2339658842 learned_addr learned my addr 172.21.15.12:0/2339658842 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:05.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.260+0000 7f5d42ffd640 1 --2- 172.21.15.12:0/2339658842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55935da89e80 0x55935da8f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.261+0000 7f5d43fff640 1 --2- 172.21.15.12:0/2339658842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55935da92a00 0x55935da8fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.261+0000 7f5d437fe640 1 -- 172.21.15.12:0/2339658842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55935da92a00 msgr2=0x55935da8fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.261+0000 7f5d437fe640 1 --2- 172.21.15.12:0/2339658842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55935da92a00 0x55935da8fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.261+0000 7f5d437fe640 1 -- 172.21.15.12:0/2339658842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55935da89e80 msgr2=0x55935da8f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.261+0000 7f5d437fe640 1 --2- 172.21.15.12:0/2339658842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55935da89e80 0x55935da8f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.261+0000 7f5d437fe640 1 -- 172.21.15.12:0/2339658842 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55935d7003f0 con 0x55935da894d0 2024-04-22T13:34:05.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.261+0000 7f5d43fff640 1 --2- 172.21.15.12:0/2339658842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55935da92a00 0x55935da8fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:05.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.261+0000 7f5d437fe640 1 --2- 172.21.15.12:0/2339658842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55935da894d0 0x55935da92270 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f5d34000c00 tx=0x7f5d3409c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de4881135b1c0e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:05.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.261+0000 7f5d22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:34:05.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.261+0000 7f5d22ffd640 1 -- 172.21.15.12:0/2339658842 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d34020070 con 0x55935da894d0 2024-04-22T13:34:05.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.261+0000 7f5d22ffd640 1 -- 172.21.15.12:0/2339658842 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f5d340a4c90 con 0x55935da894d0 2024-04-22T13:34:05.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.261+0000 7f5d4c77d740 1 -- 172.21.15.12:0/2339658842 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d34009de0 con 0x55935da894d0 2024-04-22T13:34:05.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.261+0000 7f5d22ffd640 1 -- 172.21.15.12:0/2339658842 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5d340ce0f0 con 0x55935da894d0 2024-04-22T13:34:05.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.261+0000 7f5d4c77d740 10 client.19011 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:05.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.261+0000 7f5d4c77d740 1 -- 172.21.15.12:0/2339658842 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55935d7ca7d0 con 0x55935da894d0 2024-04-22T13:34:05.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.261+0000 7f5d22ffd640 1 -- 172.21.15.12:0/2339658842 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d34009de0 con 0x55935da894d0 2024-04-22T13:34:05.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.261+0000 7f5d22ffd640 1 -- 172.21.15.12:0/2339658842 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5d34098200 con 0x55935da894d0 2024-04-22T13:34:05.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.261+0000 7f5d4c77d740 10 client.19011 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:05.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.261+0000 7f5d4c77d740 10 client.19011 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:34:05.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.261+0000 7f5d4c77d740 20 client.19011 populate_metadata read hostname 'smithi012' 2024-04-22T13:34:05.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.262+0000 7f5d4c77d740 1 --2- 172.21.15.12:0/2339658842 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55935daa29d0 0x55935daa4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.262+0000 7f5d4c77d740 4 client.19011 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:34:05.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.262+0000 7f5d4c77d740 1 -- 172.21.15.12:0/2339658842 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55935d98d930 con 0x55935daa29d0 2024-04-22T13:34:05.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.262+0000 7f5d42ffd640 1 --2- 172.21.15.12:0/2339658842 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55935daa29d0 0x55935daa4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.262+0000 7f5d22ffd640 10 client.19011 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:34:05.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.262+0000 7f5d42ffd640 1 --2- 172.21.15.12:0/2339658842 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55935daa29d0 0x55935daa4dc0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8ab452e4de8d7f04 server_cookie=2f6e9cc850349f69 in_seq=0 out_seq=0 2024-04-22T13:34:05.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.263+0000 7f5d22ffd640 1 -- 172.21.15.12:0/2339658842 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55935d98d930 con 0x55935daa29d0 2024-04-22T13:34:05.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.263+0000 7f5d22ffd640 10 client.19011 handle_command_reply: tid=0 2024-04-22T13:34:05.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.263+0000 7f5d4c77d740 1 client.19011 shutdown 2024-04-22T13:34:05.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.263+0000 7f5d4c77d740 1 -- 172.21.15.12:0/2339658842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55935da894d0 msgr2=0x55935da92270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.263+0000 7f5d4c77d740 1 --2- 172.21.15.12:0/2339658842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55935da894d0 0x55935da92270 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f5d34000c00 tx=0x7f5d3409c530 comp rx=0 tx=0).stop 2024-04-22T13:34:05.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.263+0000 7f5d4c77d740 1 -- 172.21.15.12:0/2339658842 shutdown_connections 2024-04-22T13:34:05.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.263+0000 7f5d4c77d740 1 --2- 172.21.15.12:0/2339658842 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55935da92a00 0x55935da8fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.263+0000 7f5d4c77d740 1 --2- 172.21.15.12:0/2339658842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55935da89e80 0x55935da8f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.263+0000 7f5d4c77d740 1 --2- 172.21.15.12:0/2339658842 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55935daa29d0 0x55935daa4dc0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.263+0000 7f5d4c77d740 1 --2- 172.21.15.12:0/2339658842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55935da894d0 0x55935da92270 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.263+0000 7f5d4c77d740 1 -- 172.21.15.12:0/2339658842 >> 172.21.15.12:0/2339658842 conn(0x55935da83ba0 msgr2=0x55935da858f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:05.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.264+0000 7f5d22ffd640 0 client.19011 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:34:05.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.264+0000 7f5d4c77d740 1 -- 172.21.15.12:0/2339658842 shutdown_connections 2024-04-22T13:34:05.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.264+0000 7f5d4c77d740 1 -- 172.21.15.12:0/2339658842 wait complete. 2024-04-22T13:34:05.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.264+0000 7f5d4c77d740 20 client.19011 trim_cache size 0 max 16384 2024-04-22T13:34:05.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.279+0000 7f5d4c77d740 1 Processor -- start 2024-04-22T13:34:05.277 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.279+0000 7f5d4c77d740 1 -- start start 2024-04-22T13:34:05.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.279+0000 7f5d4c77d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55935da894d0 0x55935da9e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.279+0000 7f5d4c77d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55935da89e80 0x55935da82e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.279+0000 7f5d4c77d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55935da92a00 0x55935da91d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.279+0000 7f5d4c77d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5d441afd70 con 0x55935da894d0 2024-04-22T13:34:05.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.279+0000 7f5d4c77d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5d441b18b0 con 0x55935da89e80 2024-04-22T13:34:05.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.279+0000 7f5d4c77d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5d441b0440 con 0x55935da92a00 2024-04-22T13:34:05.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.280+0000 7f5d437fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55935da894d0 0x55935da9e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.280+0000 7f5d437fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55935da894d0 0x55935da9e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36676/0 (socket says 172.21.15.12:36676) 2024-04-22T13:34:05.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.280+0000 7f5d437fe640 1 -- 172.21.15.12:0/3816568629 learned_addr learned my addr 172.21.15.12:0/3816568629 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:05.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.280+0000 7f5d42ffd640 1 --2- 172.21.15.12:0/3816568629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55935da89e80 0x55935da82e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.280+0000 7f5d43fff640 1 --2- 172.21.15.12:0/3816568629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55935da92a00 0x55935da91d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.280+0000 7f5d437fe640 1 -- 172.21.15.12:0/3816568629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55935da92a00 msgr2=0x55935da91d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.280+0000 7f5d437fe640 1 --2- 172.21.15.12:0/3816568629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55935da92a00 0x55935da91d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.280+0000 7f5d437fe640 1 -- 172.21.15.12:0/3816568629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55935da89e80 msgr2=0x55935da82e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.280+0000 7f5d437fe640 1 --2- 172.21.15.12:0/3816568629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55935da89e80 0x55935da82e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.280+0000 7f5d437fe640 1 -- 172.21.15.12:0/3816568629 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55935da8fea0 con 0x55935da894d0 2024-04-22T13:34:05.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.280+0000 7f5d42ffd640 1 --2- 172.21.15.12:0/3816568629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55935da89e80 0x55935da82e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:34:05.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.280+0000 7f5d437fe640 1 --2- 172.21.15.12:0/3816568629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55935da894d0 0x55935da9e800 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f5d340058c0 tx=0x7f5d34010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0efca86b9f610a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:05.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.280+0000 7f5d297fa640 1 -- 172.21.15.12:0/3816568629 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d34020070 con 0x55935da894d0 2024-04-22T13:34:05.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.280+0000 7f5d297fa640 1 -- 172.21.15.12:0/3816568629 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d340a5120 con 0x55935da894d0 2024-04-22T13:34:05.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.280+0000 7f5d297fa640 1 -- 172.21.15.12:0/3816568629 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d340108c0 con 0x55935da894d0 2024-04-22T13:34:05.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.281+0000 7f5d4c77d740 1 -- 172.21.15.12:0/3816568629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55935da894d0 msgr2=0x55935da9e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.281+0000 7f5d4c77d740 1 --2- 172.21.15.12:0/3816568629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55935da894d0 0x55935da9e800 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f5d340058c0 tx=0x7f5d34010cb0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.281+0000 7f5d4c77d740 1 -- 172.21.15.12:0/3816568629 shutdown_connections 2024-04-22T13:34:05.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.281+0000 7f5d4c77d740 1 --2- 172.21.15.12:0/3816568629 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55935da92a00 0x55935da91d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.281+0000 7f5d4c77d740 1 --2- 172.21.15.12:0/3816568629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55935da89e80 0x55935da82e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.281+0000 7f5d4c77d740 1 --2- 172.21.15.12:0/3816568629 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55935da894d0 0x55935da9e800 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.281+0000 7f5d4c77d740 1 -- 172.21.15.12:0/3816568629 >> 172.21.15.12:0/3816568629 conn(0x55935da83ba0 msgr2=0x55935da8aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:05.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.281+0000 7f5d4c77d740 1 -- 172.21.15.12:0/3816568629 shutdown_connections 2024-04-22T13:34:05.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.281+0000 7f5d4c77d740 1 -- 172.21.15.12:0/3816568629 wait complete. 2024-04-22T13:34:05.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.281+0000 7f5d4c77d740 1 -- start start 2024-04-22T13:34:05.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.281+0000 7f5d4c77d740 1 Processor -- start 2024-04-22T13:34:05.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.282+0000 7f5d4c77d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55935da894d0 0x55935da95fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.282+0000 7f5d4c77d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55935da89e80 0x55935da964e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.282+0000 7f5d4c77d740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55935da92a00 0x55935db00c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.282+0000 7f5d4c77d740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f5d441b11e0 con 0x55935da92a00 2024-04-22T13:34:05.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.282+0000 7f5d4c77d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5d441b0b10 con 0x55935da894d0 2024-04-22T13:34:05.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.282+0000 7f5d4c77d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5d441add70 con 0x55935da89e80 2024-04-22T13:34:05.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.282+0000 7f5d4c77d740 10 client.0 _ll_register_callbacks cb 0x7f5d34098750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:05.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.282+0000 7f5d43fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55935da92a00 0x55935db00c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.282+0000 7f5d43fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55935da92a00 0x55935db00c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36688/0 (socket says 172.21.15.12:36688) 2024-04-22T13:34:05.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.282+0000 7f5d43fff640 1 -- 172.21.15.12:0/2707808092 learned_addr learned my addr 172.21.15.12:0/2707808092 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:05.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.282+0000 7f5d42ffd640 1 --2- 172.21.15.12:0/2707808092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55935da89e80 0x55935da964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.282+0000 7f5d437fe640 1 --2- 172.21.15.12:0/2707808092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55935da894d0 0x55935da95fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.282+0000 7f5d43fff640 1 -- 172.21.15.12:0/2707808092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55935da89e80 msgr2=0x55935da964e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.282+0000 7f5d43fff640 1 --2- 172.21.15.12:0/2707808092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55935da89e80 0x55935da964e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.282+0000 7f5d43fff640 1 -- 172.21.15.12:0/2707808092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55935da894d0 msgr2=0x55935da95fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.282+0000 7f5d43fff640 1 --2- 172.21.15.12:0/2707808092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55935da894d0 0x55935da95fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.282+0000 7f5d43fff640 1 -- 172.21.15.12:0/2707808092 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55935db011f0 con 0x55935da92a00 2024-04-22T13:34:05.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.282+0000 7f5d437fe640 1 --2- 172.21.15.12:0/2707808092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55935da894d0 0x55935da95fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:05.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.282+0000 7f5d43fff640 1 --2- 172.21.15.12:0/2707808092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55935da92a00 0x55935db00c30 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009ca0 tx=0x7f5d3c045070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3a63fc540d894f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:05.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.282+0000 7f5d22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:34:05.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.282+0000 7f5d22ffd640 1 -- 172.21.15.12:0/2707808092 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c053030 con 0x55935da92a00 2024-04-22T13:34:05.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.282+0000 7f5d22ffd640 1 -- 172.21.15.12:0/2707808092 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f5d3c049040 con 0x55935da92a00 2024-04-22T13:34:05.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.283+0000 7f5d4c77d740 1 -- 172.21.15.12:0/2707808092 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d340a5120 con 0x55935da92a00 2024-04-22T13:34:05.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.283+0000 7f5d22ffd640 1 -- 172.21.15.12:0/2707808092 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5d3c039070 con 0x55935da92a00 2024-04-22T13:34:05.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.283+0000 7f5d4c77d740 10 client.19017 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:05.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.283+0000 7f5d4c77d740 1 -- 172.21.15.12:0/2707808092 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55935db08560 con 0x55935da92a00 2024-04-22T13:34:05.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.283+0000 7f5d22ffd640 1 -- 172.21.15.12:0/2707808092 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d340a5120 con 0x55935da92a00 2024-04-22T13:34:05.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.283+0000 7f5d22ffd640 1 -- 172.21.15.12:0/2707808092 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5d3c05f020 con 0x55935da92a00 2024-04-22T13:34:05.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.283+0000 7f5d4c77d740 10 client.19017 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:05.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.283+0000 7f5d4c77d740 10 client.19017 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:34:05.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.283+0000 7f5d4c77d740 20 client.19017 populate_metadata read hostname 'smithi012' 2024-04-22T13:34:05.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.283+0000 7f5d4c77d740 1 --2- 172.21.15.12:0/2707808092 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55935db08d10 0x55935db0b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.283+0000 7f5d4c77d740 4 client.19017 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:34:05.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.283+0000 7f5d4c77d740 1 -- 172.21.15.12:0/2707808092 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55935d9fbc10 con 0x55935db08d10 2024-04-22T13:34:05.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.283+0000 7f5d437fe640 1 --2- 172.21.15.12:0/2707808092 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55935db08d10 0x55935db0b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.284+0000 7f5d437fe640 1 --2- 172.21.15.12:0/2707808092 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55935db08d10 0x55935db0b130 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b9f405701f54eff4 server_cookie=8dfbce91658087be in_seq=0 out_seq=0 2024-04-22T13:34:05.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.284+0000 7f5d22ffd640 10 client.19017 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:34:05.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.284+0000 7f5d22ffd640 1 -- 172.21.15.12:0/2707808092 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55935d9fbc10 con 0x55935db08d10 2024-04-22T13:34:05.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.284+0000 7f5d22ffd640 10 client.19017 handle_command_reply: tid=0 2024-04-22T13:34:05.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.284+0000 7f5d4c77d740 1 client.19017 shutdown 2024-04-22T13:34:05.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.284+0000 7f5d4c77d740 1 -- 172.21.15.12:0/2707808092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55935da92a00 msgr2=0x55935db00c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.284+0000 7f5d4c77d740 1 --2- 172.21.15.12:0/2707808092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55935da92a00 0x55935db00c30 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009ca0 tx=0x7f5d3c045070 comp rx=0 tx=0).stop 2024-04-22T13:34:05.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.284+0000 7f5d4c77d740 1 -- 172.21.15.12:0/2707808092 shutdown_connections 2024-04-22T13:34:05.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.284+0000 7f5d4c77d740 1 --2- 172.21.15.12:0/2707808092 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55935db08d10 0x55935db0b130 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.284+0000 7f5d4c77d740 1 --2- 172.21.15.12:0/2707808092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55935da92a00 0x55935db00c30 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.284+0000 7f5d4c77d740 1 --2- 172.21.15.12:0/2707808092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55935da89e80 0x55935da964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.284+0000 7f5d22ffd640 0 client.19017 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:34:05.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.284+0000 7f5d4c77d740 1 --2- 172.21.15.12:0/2707808092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55935da894d0 0x55935da95fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.284+0000 7f5d4c77d740 1 -- 172.21.15.12:0/2707808092 >> 172.21.15.12:0/2707808092 conn(0x55935da83ba0 msgr2=0x55935da8a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:05.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.284+0000 7f5d4c77d740 1 -- 172.21.15.12:0/2707808092 shutdown_connections 2024-04-22T13:34:05.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.284+0000 7f5d4c77d740 1 -- 172.21.15.12:0/2707808092 wait complete. 2024-04-22T13:34:05.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.284+0000 7f5d4c77d740 20 client.19017 trim_cache size 0 max 16384 2024-04-22T13:34:05.288 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:34:05.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.285+0000 7f5d49b8f640 1 -- 172.21.15.12:0/1026396659 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5d2c0a3d20 msgr2=0x7f5d2c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.285+0000 7f5d49b8f640 1 --2- 172.21.15.12:0/1026396659 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5d2c0a3d20 0x7f5d2c0a61e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c005e30 tx=0x7f5d3c009880 comp rx=0 tx=0).stop 2024-04-22T13:34:05.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.285+0000 7f5d49b8f640 1 -- 172.21.15.12:0/1026396659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d4406a5d0 msgr2=0x7f5d441a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.285+0000 7f5d49b8f640 1 --2- 172.21.15.12:0/1026396659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d4406a5d0 0x7f5d441a0610 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400d950 tx=0x7f5d3400de10 comp rx=0 tx=0).stop 2024-04-22T13:34:05.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.285+0000 7f5d49b8f640 1 -- 172.21.15.12:0/1026396659 shutdown_connections 2024-04-22T13:34:05.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.285+0000 7f5d49b8f640 1 --2- 172.21.15.12:0/1026396659 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f5d2c0a3d20 0x7f5d2c0a61e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.285+0000 7f5d49b8f640 1 --2- 172.21.15.12:0/1026396659 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5d4410e960 0x7f5d4419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.285+0000 7f5d49b8f640 1 --2- 172.21.15.12:0/1026396659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d4406aef0 0x7f5d441a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.285+0000 7f5d49b8f640 1 --2- 172.21.15.12:0/1026396659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d4406a5d0 0x7f5d441a0610 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.285+0000 7f5d49b8f640 1 -- 172.21.15.12:0/1026396659 >> 172.21.15.12:0/1026396659 conn(0x7f5d44077eb0 msgr2=0x7f5d4410fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:05.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.285+0000 7f5d49b8f640 1 -- 172.21.15.12:0/1026396659 shutdown_connections 2024-04-22T13:34:05.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.285+0000 7f5d49b8f640 1 -- 172.21.15.12:0/1026396659 wait complete. 2024-04-22T13:34:05.292 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:34:05.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.400+0000 7f518b7b6640 1 -- 172.21.15.12:0/2600598624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5184102e20 msgr2=0x7f518406a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.400+0000 7f518b7b6640 1 --2- 172.21.15.12:0/2600598624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5184102e20 0x7f518406a550 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f5180009900 tx=0x7f518002bc30 comp rx=0 tx=0).stop 2024-04-22T13:34:05.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.400+0000 7f518b7b6640 1 -- 172.21.15.12:0/2600598624 shutdown_connections 2024-04-22T13:34:05.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.400+0000 7f518b7b6640 1 --2- 172.21.15.12:0/2600598624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f518410e810 0x7f5184110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.398 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.400+0000 7f518b7b6640 1 --2- 172.21.15.12:0/2600598624 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f518406aa90 0x7f518406aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.400+0000 7f518b7b6640 1 --2- 172.21.15.12:0/2600598624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5184102e20 0x7f518406a550 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.400+0000 7f518b7b6640 1 -- 172.21.15.12:0/2600598624 >> 172.21.15.12:0/2600598624 conn(0x7f5184076c50 msgr2=0x7f5184077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:05.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.400+0000 7f518b7b6640 1 -- 172.21.15.12:0/2600598624 shutdown_connections 2024-04-22T13:34:05.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.400+0000 7f518b7b6640 1 -- 172.21.15.12:0/2600598624 wait complete. 2024-04-22T13:34:05.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.400+0000 7f518b7b6640 1 Processor -- start 2024-04-22T13:34:05.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.400+0000 7f518b7b6640 1 -- start start 2024-04-22T13:34:05.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.400+0000 7f518b7b6640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f518406aa90 0x7f51841a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.401+0000 7f518b7b6640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5184102e20 0x7f51841a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.401+0000 7f518b7b6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f518410e810 0x7f518419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.401+0000 7f518b7b6640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f51841143b0 con 0x7f5184102e20 2024-04-22T13:34:05.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.401+0000 7f518b7b6640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5184114230 con 0x7f518406aa90 2024-04-22T13:34:05.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.401+0000 7f518b7b6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5184114530 con 0x7f518410e810 2024-04-22T13:34:05.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.401+0000 7f5188d51640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5184102e20 0x7f51841a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.399 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.401+0000 7f5188d51640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5184102e20 0x7f51841a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36706/0 (socket says 172.21.15.12:36706) 2024-04-22T13:34:05.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.401+0000 7f5188d51640 1 -- 172.21.15.12:0/2855648967 learned_addr learned my addr 172.21.15.12:0/2855648967 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:05.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.401+0000 7f5189552640 1 --2- 172.21.15.12:0/2855648967 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f518406aa90 0x7f51841a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.401+0000 7f5189d53640 1 --2- 172.21.15.12:0/2855648967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f518410e810 0x7f518419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.401+0000 7f5188d51640 1 -- 172.21.15.12:0/2855648967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f518410e810 msgr2=0x7f518419a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.401+0000 7f5188d51640 1 --2- 172.21.15.12:0/2855648967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f518410e810 0x7f518419a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.401+0000 7f5188d51640 1 -- 172.21.15.12:0/2855648967 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f518406aa90 msgr2=0x7f51841a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.401+0000 7f5188d51640 1 --2- 172.21.15.12:0/2855648967 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f518406aa90 0x7f51841a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.401+0000 7f5188d51640 1 -- 172.21.15.12:0/2855648967 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518419b030 con 0x7f5184102e20 2024-04-22T13:34:05.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.401+0000 7f5188d51640 1 --2- 172.21.15.12:0/2855648967 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5184102e20 0x7f51841a0bf0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f517400c920 tx=0x7f517400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:05.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.401+0000 7f517a7fc640 1 -- 172.21.15.12:0/2855648967 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5174007c70 con 0x7f5184102e20 2024-04-22T13:34:05.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.401+0000 7f517a7fc640 1 -- 172.21.15.12:0/2855648967 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5174007e10 con 0x7f5184102e20 2024-04-22T13:34:05.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.401+0000 7f518b7b6640 1 -- 172.21.15.12:0/2855648967 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f518419b320 con 0x7f5184102e20 2024-04-22T13:34:05.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.401+0000 7f517a7fc640 1 -- 172.21.15.12:0/2855648967 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5174018820 con 0x7f5184102e20 2024-04-22T13:34:05.400 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.401+0000 7f518b7b6640 1 -- 172.21.15.12:0/2855648967 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51841a7630 con 0x7f5184102e20 2024-04-22T13:34:05.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.404+0000 7f517a7fc640 1 -- 172.21.15.12:0/2855648967 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f517400f450 con 0x7f5184102e20 2024-04-22T13:34:05.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.404+0000 7f518e3ab740 1 Processor -- start 2024-04-22T13:34:05.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.404+0000 7f518e3ab740 1 -- start start 2024-04-22T13:34:05.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.404+0000 7f518e3ab740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7ef0794d0 0x55a7ef0798b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.404+0000 7f518e3ab740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7ef079e80 0x55a7ef082430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.404+0000 7f518e3ab740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7ef082a00 0x55a7ef084e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.404+0000 7f518e3ab740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a7eefde1f0 con 0x55a7ef079e80 2024-04-22T13:34:05.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.404+0000 7f518e3ab740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a7eef7b4b0 con 0x55a7ef0794d0 2024-04-22T13:34:05.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.404+0000 7f518e3ab740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a7eefda1b0 con 0x55a7ef082a00 2024-04-22T13:34:05.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.404+0000 7f517a7fc640 1 --2- 172.21.15.12:0/2855648967 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f51700a3da0 0x7f51700a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.404+0000 7f5189552640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7ef079e80 0x55a7ef082430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.404+0000 7f5189552640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7ef079e80 0x55a7ef082430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36712/0 (socket says 172.21.15.12:36712) 2024-04-22T13:34:05.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.404+0000 7f5189552640 1 -- 172.21.15.12:0/370170900 learned_addr learned my addr 172.21.15.12:0/370170900 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:05.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.404+0000 7f5189d53640 1 --2- 172.21.15.12:0/370170900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7ef0794d0 0x55a7ef0798b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.404+0000 7f517a7fc640 1 -- 172.21.15.12:0/2855648967 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f51740d08a0 con 0x7f5184102e20 2024-04-22T13:34:05.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.405+0000 7f5188d51640 1 --2- 172.21.15.12:0/370170900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7ef082a00 0x55a7ef084e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.405+0000 7f5189d53640 1 --2- 172.21.15.12:0/2855648967 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f51700a3da0 0x7f51700a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.405+0000 7f5189552640 1 -- 172.21.15.12:0/370170900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7ef082a00 msgr2=0x55a7ef084e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.405+0000 7f5189552640 1 --2- 172.21.15.12:0/370170900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7ef082a00 0x55a7ef084e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.405+0000 7f5189552640 1 -- 172.21.15.12:0/370170900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7ef0794d0 msgr2=0x55a7ef0798b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.405+0000 7f5189552640 1 --2- 172.21.15.12:0/370170900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7ef0794d0 0x55a7ef0798b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.405+0000 7f5189552640 1 -- 172.21.15.12:0/370170900 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7eef6d1b0 con 0x55a7ef079e80 2024-04-22T13:34:05.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.405+0000 7f5189d53640 1 --2- 172.21.15.12:0/2855648967 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f51700a3da0 0x7f51700a6260 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f517c005c40 tx=0x7f517c005bd0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:05.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.405+0000 7f5189552640 1 --2- 172.21.15.12:0/370170900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7ef079e80 0x55a7ef082430 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f5180009900 tx=0x7f5180004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988c30de558a63d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:05.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.405+0000 7f516f7fe640 1 -- 172.21.15.12:0/370170900 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5180045020 con 0x55a7ef079e80 2024-04-22T13:34:05.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.405+0000 7f516f7fe640 1 -- 172.21.15.12:0/370170900 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f518002cc90 con 0x55a7ef079e80 2024-04-22T13:34:05.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.405+0000 7f516f7fe640 1 -- 172.21.15.12:0/370170900 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51800046c0 con 0x55a7ef079e80 2024-04-22T13:34:05.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.406+0000 7f518e3ab740 1 -- 172.21.15.12:0/370170900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7ef079e80 msgr2=0x55a7ef082430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.406+0000 7f518e3ab740 1 --2- 172.21.15.12:0/370170900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7ef079e80 0x55a7ef082430 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f5180009900 tx=0x7f5180004040 comp rx=0 tx=0).stop 2024-04-22T13:34:05.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.406+0000 7f518e3ab740 1 -- 172.21.15.12:0/370170900 shutdown_connections 2024-04-22T13:34:05.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.406+0000 7f518e3ab740 1 --2- 172.21.15.12:0/370170900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7ef082a00 0x55a7ef084e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.406+0000 7f518e3ab740 1 --2- 172.21.15.12:0/370170900 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7ef079e80 0x55a7ef082430 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.406+0000 7f518e3ab740 1 --2- 172.21.15.12:0/370170900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7ef0794d0 0x55a7ef0798b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.406+0000 7f518e3ab740 1 -- 172.21.15.12:0/370170900 >> 172.21.15.12:0/370170900 conn(0x55a7ef073ba0 msgr2=0x55a7ef073f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:05.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.406+0000 7f518e3ab740 1 -- 172.21.15.12:0/370170900 shutdown_connections 2024-04-22T13:34:05.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.406+0000 7f518e3ab740 1 -- 172.21.15.12:0/370170900 wait complete. 2024-04-22T13:34:05.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.406+0000 7f518e3ab740 1 -- start start 2024-04-22T13:34:05.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.406+0000 7f518e3ab740 1 Processor -- start 2024-04-22T13:34:05.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.407+0000 7f518e3ab740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7ef0794d0 0x55a7ef0822d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.407+0000 7f518e3ab740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7ef079e80 0x55a7ef07f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.407+0000 7f518e3ab740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7ef082a00 0x55a7ef07fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.407+0000 7f518e3ab740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a7ef085910 con 0x55a7ef079e80 2024-04-22T13:34:05.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.407+0000 7f518e3ab740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a7eef97500 con 0x55a7ef082a00 2024-04-22T13:34:05.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.407+0000 7f518e3ab740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a7ef085a90 con 0x55a7ef0794d0 2024-04-22T13:34:05.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.407+0000 7f518e3ab740 10 client.0 _ll_register_callbacks cb 0x55a7eef65b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:05.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.407+0000 7f5188d51640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7ef079e80 0x55a7ef07f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.407+0000 7f5188d51640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7ef079e80 0x55a7ef07f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36724/0 (socket says 172.21.15.12:36724) 2024-04-22T13:34:05.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.407+0000 7f5188d51640 1 -- 172.21.15.12:0/2875171885 learned_addr learned my addr 172.21.15.12:0/2875171885 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:05.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.407+0000 7f5189d53640 1 --2- 172.21.15.12:0/2875171885 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7ef082a00 0x55a7ef07fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.407+0000 7f5188d51640 1 -- 172.21.15.12:0/2875171885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7ef0794d0 msgr2=0x55a7ef0822d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.413 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.407+0000 7f5188d51640 1 --2- 172.21.15.12:0/2875171885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7ef0794d0 0x55a7ef0822d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.407+0000 7f5188d51640 1 -- 172.21.15.12:0/2875171885 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7ef082a00 msgr2=0x55a7ef07fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.407+0000 7f5188d51640 1 --2- 172.21.15.12:0/2875171885 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7ef082a00 0x55a7ef07fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.407+0000 7f5188d51640 1 -- 172.21.15.12:0/2875171885 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7eecf03f0 con 0x55a7ef079e80 2024-04-22T13:34:05.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.407+0000 7f5188d51640 1 --2- 172.21.15.12:0/2875171885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7ef079e80 0x55a7ef07f660 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f51740ce0c0 tx=0x7f517401def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d8c4659d2c1145 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:05.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.407+0000 7f516cff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:34:05.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.407+0000 7f516cff9640 1 -- 172.21.15.12:0/2875171885 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517409c860 con 0x55a7ef079e80 2024-04-22T13:34:05.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.407+0000 7f516cff9640 1 -- 172.21.15.12:0/2875171885 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f517400f740 con 0x55a7ef079e80 2024-04-22T13:34:05.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.407+0000 7f518e3ab740 1 -- 172.21.15.12:0/2875171885 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f518002cc90 con 0x55a7ef079e80 2024-04-22T13:34:05.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.408+0000 7f516cff9640 1 -- 172.21.15.12:0/2875171885 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f51740a0c10 con 0x55a7ef079e80 2024-04-22T13:34:05.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.408+0000 7f518e3ab740 10 client.19029 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:05.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.408+0000 7f518e3ab740 1 -- 172.21.15.12:0/2875171885 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a7eedba7d0 con 0x55a7ef079e80 2024-04-22T13:34:05.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.408+0000 7f516cff9640 1 -- 172.21.15.12:0/2875171885 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f518002cc90 con 0x55a7ef079e80 2024-04-22T13:34:05.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.408+0000 7f516cff9640 1 -- 172.21.15.12:0/2875171885 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f51740870b0 con 0x55a7ef079e80 2024-04-22T13:34:05.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.408+0000 7f518e3ab740 10 client.19029 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:05.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.408+0000 7f518e3ab740 10 client.19029 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:34:05.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.408+0000 7f518e3ab740 20 client.19029 populate_metadata read hostname 'smithi012' 2024-04-22T13:34:05.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.408+0000 7f518e3ab740 1 --2- 172.21.15.12:0/2875171885 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55a7ef092af0 0x55a7ef094f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.408+0000 7f518e3ab740 4 client.19029 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:34:05.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.408+0000 7f518e3ab740 1 -- 172.21.15.12:0/2875171885 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7eef7d930 con 0x55a7ef092af0 2024-04-22T13:34:05.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.408+0000 7f5189552640 1 --2- 172.21.15.12:0/2875171885 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55a7ef092af0 0x55a7ef094f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.409+0000 7f516cff9640 10 client.19029 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:34:05.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.409+0000 7f5189552640 1 --2- 172.21.15.12:0/2875171885 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55a7ef092af0 0x55a7ef094f40 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e47f52a2341a8558 server_cookie=f1e205d5145578d1 in_seq=0 out_seq=0 2024-04-22T13:34:05.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.409+0000 7f516cff9640 1 -- 172.21.15.12:0/2875171885 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a7eef7d930 con 0x55a7ef092af0 2024-04-22T13:34:05.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.409+0000 7f516cff9640 10 client.19029 handle_command_reply: tid=0 2024-04-22T13:34:05.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.409+0000 7f518e3ab740 1 client.19029 shutdown 2024-04-22T13:34:05.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.410+0000 7f518e3ab740 1 -- 172.21.15.12:0/2875171885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7ef079e80 msgr2=0x55a7ef07f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.410+0000 7f518e3ab740 1 --2- 172.21.15.12:0/2875171885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7ef079e80 0x55a7ef07f660 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f51740ce0c0 tx=0x7f517401def0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.410+0000 7f518e3ab740 1 -- 172.21.15.12:0/2875171885 shutdown_connections 2024-04-22T13:34:05.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.410+0000 7f518e3ab740 1 --2- 172.21.15.12:0/2875171885 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55a7ef092af0 0x55a7ef094f40 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.410+0000 7f518e3ab740 1 --2- 172.21.15.12:0/2875171885 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7ef082a00 0x55a7ef07fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.410+0000 7f518e3ab740 1 --2- 172.21.15.12:0/2875171885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7ef079e80 0x55a7ef07f660 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.410+0000 7f516cff9640 0 client.19029 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:34:05.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.410+0000 7f518e3ab740 1 --2- 172.21.15.12:0/2875171885 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7ef0794d0 0x55a7ef0822d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.410+0000 7f518e3ab740 1 -- 172.21.15.12:0/2875171885 >> 172.21.15.12:0/2875171885 conn(0x55a7ef073ba0 msgr2=0x55a7ef0758f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:05.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.410+0000 7f518e3ab740 1 -- 172.21.15.12:0/2875171885 shutdown_connections 2024-04-22T13:34:05.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.410+0000 7f518e3ab740 1 -- 172.21.15.12:0/2875171885 wait complete. 2024-04-22T13:34:05.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.410+0000 7f518e3ab740 20 client.19029 trim_cache size 0 max 16384 2024-04-22T13:34:05.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.423+0000 7f518e3ab740 1 Processor -- start 2024-04-22T13:34:05.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.423+0000 7f518e3ab740 1 -- start start 2024-04-22T13:34:05.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.423+0000 7f518e3ab740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7ef0794d0 0x55a7ef08e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.423+0000 7f518e3ab740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7ef079e80 0x55a7ef081d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.423+0000 7f518e3ab740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7ef082a00 0x55a7ef0822d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.423+0000 7f518e3ab740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f51841afc20 con 0x55a7ef079e80 2024-04-22T13:34:05.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.423+0000 7f518e3ab740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f51841b1760 con 0x55a7ef082a00 2024-04-22T13:34:05.425 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.423+0000 7f518e3ab740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f51841b02f0 con 0x55a7ef0794d0 2024-04-22T13:34:05.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.423+0000 7f5188d51640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7ef079e80 0x55a7ef081d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.423+0000 7f5188d51640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7ef079e80 0x55a7ef081d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36740/0 (socket says 172.21.15.12:36740) 2024-04-22T13:34:05.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.423+0000 7f5188d51640 1 -- 172.21.15.12:0/2619645473 learned_addr learned my addr 172.21.15.12:0/2619645473 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:05.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.424+0000 7f5189552640 1 --2- 172.21.15.12:0/2619645473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7ef0794d0 0x55a7ef08e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.424+0000 7f5188d51640 1 -- 172.21.15.12:0/2619645473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7ef0794d0 msgr2=0x55a7ef08e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.424+0000 7f5188d51640 1 --2- 172.21.15.12:0/2619645473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7ef0794d0 0x55a7ef08e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.424+0000 7f5188d51640 1 -- 172.21.15.12:0/2619645473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7ef082a00 msgr2=0x55a7ef0822d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.426 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.424+0000 7f5188d51640 1 --2- 172.21.15.12:0/2619645473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7ef082a00 0x55a7ef0822d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.424+0000 7f5188d51640 1 -- 172.21.15.12:0/2619645473 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7ef07dbb0 con 0x55a7ef079e80 2024-04-22T13:34:05.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.424+0000 7f5189552640 1 --2- 172.21.15.12:0/2619645473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7ef0794d0 0x55a7ef08e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:05.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.424+0000 7f5188d51640 1 --2- 172.21.15.12:0/2619645473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7ef079e80 0x55a7ef081d90 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f51740ce610 tx=0x7f517400a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8ec9637254d244 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:05.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.424+0000 7f516effd640 1 -- 172.21.15.12:0/2619645473 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5174098ed0 con 0x55a7ef079e80 2024-04-22T13:34:05.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.424+0000 7f516effd640 1 -- 172.21.15.12:0/2619645473 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f517400f740 con 0x55a7ef079e80 2024-04-22T13:34:05.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.424+0000 7f516effd640 1 -- 172.21.15.12:0/2619645473 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517409db90 con 0x55a7ef079e80 2024-04-22T13:34:05.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.425+0000 7f518e3ab740 1 -- 172.21.15.12:0/2619645473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7ef079e80 msgr2=0x55a7ef081d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.425+0000 7f518e3ab740 1 --2- 172.21.15.12:0/2619645473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7ef079e80 0x55a7ef081d90 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f51740ce610 tx=0x7f517400a560 comp rx=0 tx=0).stop 2024-04-22T13:34:05.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.425+0000 7f518e3ab740 1 -- 172.21.15.12:0/2619645473 shutdown_connections 2024-04-22T13:34:05.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.425+0000 7f518e3ab740 1 --2- 172.21.15.12:0/2619645473 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7ef082a00 0x55a7ef0822d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.425+0000 7f518e3ab740 1 --2- 172.21.15.12:0/2619645473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7ef079e80 0x55a7ef081d90 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.425+0000 7f518e3ab740 1 --2- 172.21.15.12:0/2619645473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7ef0794d0 0x55a7ef08e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.425+0000 7f518e3ab740 1 -- 172.21.15.12:0/2619645473 >> 172.21.15.12:0/2619645473 conn(0x55a7ef073ba0 msgr2=0x55a7ef072c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:05.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.425+0000 7f518e3ab740 1 -- 172.21.15.12:0/2619645473 shutdown_connections 2024-04-22T13:34:05.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.425+0000 7f518e3ab740 1 -- 172.21.15.12:0/2619645473 wait complete. 2024-04-22T13:34:05.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.425+0000 7f518e3ab740 1 -- start start 2024-04-22T13:34:05.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.425+0000 7f518e3ab740 1 Processor -- start 2024-04-22T13:34:05.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.425+0000 7f518e3ab740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7ef0794d0 0x55a7ef0867e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.425+0000 7f518e3ab740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7ef079e80 0x55a7ef086d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.425+0000 7f518e3ab740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7ef082a00 0x55a7ef087260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.425+0000 7f518e3ab740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f51841b1090 con 0x55a7ef079e80 2024-04-22T13:34:05.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.425+0000 7f518e3ab740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f51841b09c0 con 0x55a7ef0794d0 2024-04-22T13:34:05.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.425+0000 7f518e3ab740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f51841adc20 con 0x55a7ef082a00 2024-04-22T13:34:05.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.425+0000 7f518e3ab740 10 client.0 _ll_register_callbacks cb 0x7f5174098bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:05.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.425+0000 7f5188d51640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7ef079e80 0x55a7ef086d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.425+0000 7f5188d51640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7ef079e80 0x55a7ef086d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36752/0 (socket says 172.21.15.12:36752) 2024-04-22T13:34:05.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.425+0000 7f5188d51640 1 -- 172.21.15.12:0/181356515 learned_addr learned my addr 172.21.15.12:0/181356515 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:05.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.426+0000 7f5189552640 1 --2- 172.21.15.12:0/181356515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7ef0794d0 0x55a7ef0867e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.426+0000 7f5189d53640 1 --2- 172.21.15.12:0/181356515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7ef082a00 0x55a7ef087260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.426+0000 7f5188d51640 1 -- 172.21.15.12:0/181356515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7ef082a00 msgr2=0x55a7ef087260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.426+0000 7f5188d51640 1 --2- 172.21.15.12:0/181356515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7ef082a00 0x55a7ef087260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.426+0000 7f5188d51640 1 -- 172.21.15.12:0/181356515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7ef0794d0 msgr2=0x55a7ef0867e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.426+0000 7f5188d51640 1 --2- 172.21.15.12:0/181356515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7ef0794d0 0x55a7ef0867e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.426+0000 7f5188d51640 1 -- 172.21.15.12:0/181356515 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7ef0f8100 con 0x55a7ef079e80 2024-04-22T13:34:05.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.426+0000 7f5189552640 1 --2- 172.21.15.12:0/181356515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7ef0794d0 0x55a7ef0867e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:05.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.426+0000 7f5188d51640 1 --2- 172.21.15.12:0/181356515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7ef079e80 0x55a7ef086d20 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f51740a4fa0 tx=0x7f517400f8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c79ffce112a9994b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:05.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.426+0000 7f516cff9640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:34:05.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.426+0000 7f516cff9640 1 -- 172.21.15.12:0/181356515 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51740a5240 con 0x55a7ef079e80 2024-04-22T13:34:05.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.426+0000 7f516cff9640 1 -- 172.21.15.12:0/181356515 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f517409db90 con 0x55a7ef079e80 2024-04-22T13:34:05.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.426+0000 7f518e3ab740 1 -- 172.21.15.12:0/181356515 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f517400f740 con 0x55a7ef079e80 2024-04-22T13:34:05.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.426+0000 7f516cff9640 1 -- 172.21.15.12:0/181356515 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f517409dd90 con 0x55a7ef079e80 2024-04-22T13:34:05.429 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.426+0000 7f518e3ab740 10 client.19035 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:05.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.426+0000 7f518e3ab740 1 -- 172.21.15.12:0/181356515 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a7ef0f8100 con 0x55a7ef079e80 2024-04-22T13:34:05.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.426+0000 7f516cff9640 1 -- 172.21.15.12:0/181356515 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f517400f740 con 0x55a7ef079e80 2024-04-22T13:34:05.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.427+0000 7f516cff9640 1 -- 172.21.15.12:0/181356515 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f51740ac030 con 0x55a7ef079e80 2024-04-22T13:34:05.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.427+0000 7f518e3ab740 10 client.19035 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:05.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.427+0000 7f518e3ab740 10 client.19035 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:34:05.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.427+0000 7f518e3ab740 20 client.19035 populate_metadata read hostname 'smithi012' 2024-04-22T13:34:05.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.427+0000 7f518e3ab740 1 --2- 172.21.15.12:0/181356515 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55a7ef0f9770 0x55a7ef0fbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.427+0000 7f518e3ab740 4 client.19035 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:34:05.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.427+0000 7f518e3ab740 1 -- 172.21.15.12:0/181356515 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a7eefebc10 con 0x55a7ef0f9770 2024-04-22T13:34:05.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.427+0000 7f5189552640 1 --2- 172.21.15.12:0/181356515 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55a7ef0f9770 0x55a7ef0fbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.427+0000 7f516cff9640 10 client.19035 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:34:05.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.427+0000 7f5189552640 1 --2- 172.21.15.12:0/181356515 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55a7ef0f9770 0x55a7ef0fbb90 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8dd97a8c4a95af29 server_cookie=7a7be05e4ba0fcb3 in_seq=0 out_seq=0 2024-04-22T13:34:05.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.427+0000 7f516cff9640 1 -- 172.21.15.12:0/181356515 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a7eefebc10 con 0x55a7ef0f9770 2024-04-22T13:34:05.430 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.428+0000 7f516cff9640 10 client.19035 handle_command_reply: tid=0 2024-04-22T13:34:05.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.428+0000 7f518e3ab740 1 client.19035 shutdown 2024-04-22T13:34:05.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.428+0000 7f518e3ab740 1 -- 172.21.15.12:0/181356515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7ef079e80 msgr2=0x55a7ef086d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.428+0000 7f518e3ab740 1 --2- 172.21.15.12:0/181356515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7ef079e80 0x55a7ef086d20 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f51740a4fa0 tx=0x7f517400f8f0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.428+0000 7f518e3ab740 1 -- 172.21.15.12:0/181356515 shutdown_connections 2024-04-22T13:34:05.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.428+0000 7f518e3ab740 1 --2- 172.21.15.12:0/181356515 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x55a7ef0f9770 0x55a7ef0fbb90 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.428+0000 7f518e3ab740 1 --2- 172.21.15.12:0/181356515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7ef082a00 0x55a7ef087260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.428+0000 7f516cff9640 0 client.19035 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:34:05.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.428+0000 7f518e3ab740 1 --2- 172.21.15.12:0/181356515 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7ef079e80 0x55a7ef086d20 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.428+0000 7f518e3ab740 1 --2- 172.21.15.12:0/181356515 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7ef0794d0 0x55a7ef0867e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.428+0000 7f518e3ab740 1 -- 172.21.15.12:0/181356515 >> 172.21.15.12:0/181356515 conn(0x55a7ef073ba0 msgr2=0x55a7ef07a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:05.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.428+0000 7f518e3ab740 1 -- 172.21.15.12:0/181356515 shutdown_connections 2024-04-22T13:34:05.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.428+0000 7f518e3ab740 1 -- 172.21.15.12:0/181356515 wait complete. 2024-04-22T13:34:05.431 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.428+0000 7f518e3ab740 20 client.19035 trim_cache size 0 max 16384 2024-04-22T13:34:05.432 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:34:05.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.428+0000 7f518b7b6640 1 -- 172.21.15.12:0/2855648967 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f51700a3da0 msgr2=0x7f51700a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.428+0000 7f518b7b6640 1 --2- 172.21.15.12:0/2855648967 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f51700a3da0 0x7f51700a6260 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f517c005c40 tx=0x7f517c005bd0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.428+0000 7f518b7b6640 1 -- 172.21.15.12:0/2855648967 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5184102e20 msgr2=0x7f51841a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.428+0000 7f518b7b6640 1 --2- 172.21.15.12:0/2855648967 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5184102e20 0x7f51841a0bf0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f517400c920 tx=0x7f517400cde0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.428+0000 7f518b7b6640 1 -- 172.21.15.12:0/2855648967 shutdown_connections 2024-04-22T13:34:05.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.428+0000 7f518b7b6640 1 --2- 172.21.15.12:0/2855648967 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f51700a3da0 0x7f51700a6260 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.428+0000 7f518b7b6640 1 --2- 172.21.15.12:0/2855648967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f518410e810 0x7f518419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.428+0000 7f518b7b6640 1 --2- 172.21.15.12:0/2855648967 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5184102e20 0x7f51841a0bf0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.428+0000 7f518b7b6640 1 --2- 172.21.15.12:0/2855648967 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f518406aa90 0x7f51841a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.428+0000 7f518b7b6640 1 -- 172.21.15.12:0/2855648967 >> 172.21.15.12:0/2855648967 conn(0x7f5184076c50 msgr2=0x7f5184100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:05.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.429+0000 7f518b7b6640 1 -- 172.21.15.12:0/2855648967 shutdown_connections 2024-04-22T13:34:05.433 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.429+0000 7f518b7b6640 1 -- 172.21.15.12:0/2855648967 wait complete. 2024-04-22T13:34:05.436 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:34:05.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.543+0000 7f3256cd5640 1 -- 172.21.15.12:0/572603320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3250103af0 msgr2=0x7f325010bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.543+0000 7f3256cd5640 1 --2- 172.21.15.12:0/572603320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3250103af0 0x7f325010bfd0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f3240009900 tx=0x7f324002bd60 comp rx=0 tx=0).stop 2024-04-22T13:34:05.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.543+0000 7f3256cd5640 1 -- 172.21.15.12:0/572603320 shutdown_connections 2024-04-22T13:34:05.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.543+0000 7f3256cd5640 1 --2- 172.21.15.12:0/572603320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f325010c510 0x7f325010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.543+0000 7f3256cd5640 1 --2- 172.21.15.12:0/572603320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3250103af0 0x7f325010bfd0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.543+0000 7f3256cd5640 1 --2- 172.21.15.12:0/572603320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32501031d0 0x7f32501035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.543+0000 7f3256cd5640 1 -- 172.21.15.12:0/572603320 >> 172.21.15.12:0/572603320 conn(0x7f3250077f50 msgr2=0x7f3250078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:05.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.543+0000 7f3256cd5640 1 -- 172.21.15.12:0/572603320 shutdown_connections 2024-04-22T13:34:05.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.543+0000 7f3256cd5640 1 -- 172.21.15.12:0/572603320 wait complete. 2024-04-22T13:34:05.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.544+0000 7f3256cd5640 1 Processor -- start 2024-04-22T13:34:05.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.544+0000 7f3256cd5640 1 -- start start 2024-04-22T13:34:05.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.544+0000 7f3256cd5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32501031d0 0x7f3250100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.544+0000 7f3256cd5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3250103af0 0x7f3250101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.544+0000 7f3256cd5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f325010c510 0x7f32501018a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.544+0000 7f3256cd5640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f32501121a0 con 0x7f32501031d0 2024-04-22T13:34:05.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.544+0000 7f3256cd5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3250112020 con 0x7f3250103af0 2024-04-22T13:34:05.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.544+0000 7f3256cd5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3250112320 con 0x7f325010c510 2024-04-22T13:34:05.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.544+0000 7f3254a71640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32501031d0 0x7f3250100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.544+0000 7f3254a71640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32501031d0 0x7f3250100e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36776/0 (socket says 172.21.15.12:36776) 2024-04-22T13:34:05.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.544+0000 7f3254a71640 1 -- 172.21.15.12:0/3837310990 learned_addr learned my addr 172.21.15.12:0/3837310990 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:05.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.544+0000 7f3247fff640 1 --2- 172.21.15.12:0/3837310990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3250103af0 0x7f3250101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.544+0000 7f3255272640 1 --2- 172.21.15.12:0/3837310990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f325010c510 0x7f32501018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.543 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.544+0000 7f3254a71640 1 -- 172.21.15.12:0/3837310990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f325010c510 msgr2=0x7f32501018a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.544+0000 7f3254a71640 1 --2- 172.21.15.12:0/3837310990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f325010c510 0x7f32501018a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.544+0000 7f3254a71640 1 -- 172.21.15.12:0/3837310990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3250103af0 msgr2=0x7f3250101360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.545+0000 7f3254a71640 1 --2- 172.21.15.12:0/3837310990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3250103af0 0x7f3250101360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.545+0000 7f3254a71640 1 -- 172.21.15.12:0/3837310990 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32501a6c40 con 0x7f32501031d0 2024-04-22T13:34:05.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.545+0000 7f3255272640 1 --2- 172.21.15.12:0/3837310990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f325010c510 0x7f32501018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:05.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.545+0000 7f3254a71640 1 --2- 172.21.15.12:0/3837310990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32501031d0 0x7f3250100e20 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f324c00d770 tx=0x7f324c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:05.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.545+0000 7f3245ffb640 1 -- 172.21.15.12:0/3837310990 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c017070 con 0x7f32501031d0 2024-04-22T13:34:05.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.545+0000 7f3245ffb640 1 -- 172.21.15.12:0/3837310990 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f324c0054d0 con 0x7f32501031d0 2024-04-22T13:34:05.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.545+0000 7f3256cd5640 1 -- 172.21.15.12:0/3837310990 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32501a6f30 con 0x7f32501031d0 2024-04-22T13:34:05.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.545+0000 7f3245ffb640 1 -- 172.21.15.12:0/3837310990 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c004c30 con 0x7f32501031d0 2024-04-22T13:34:05.544 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.545+0000 7f3256cd5640 1 -- 172.21.15.12:0/3837310990 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32501a73a0 con 0x7f32501031d0 2024-04-22T13:34:05.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.547+0000 7f3245ffb640 1 -- 172.21.15.12:0/3837310990 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f324c004e10 con 0x7f32501031d0 2024-04-22T13:34:05.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.548+0000 7f325999c740 1 Processor -- start 2024-04-22T13:34:05.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.548+0000 7f325999c740 1 -- start start 2024-04-22T13:34:05.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.548+0000 7f325999c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1ed16c4d0 0x55b1ed16c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.548+0000 7f325999c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1ed16ce80 0x55b1ed175430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.548+0000 7f325999c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1ed175a00 0x55b1ed177e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.548+0000 7f325999c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b1ed0d11f0 con 0x55b1ed175a00 2024-04-22T13:34:05.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.548+0000 7f325999c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b1ed06e4b0 con 0x55b1ed16c4d0 2024-04-22T13:34:05.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.548+0000 7f325999c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b1ed0cd1b0 con 0x55b1ed16ce80 2024-04-22T13:34:05.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.548+0000 7f3245ffb640 1 --2- 172.21.15.12:0/3837310990 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f32380a3d20 0x7f32380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.548+0000 7f3247fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1ed175a00 0x55b1ed177e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.548+0000 7f3247fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1ed175a00 0x55b1ed177e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36788/0 (socket says 172.21.15.12:36788) 2024-04-22T13:34:05.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.548+0000 7f3247fff640 1 -- 172.21.15.12:0/111835588 learned_addr learned my addr 172.21.15.12:0/111835588 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:05.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.548+0000 7f3255272640 1 --2- 172.21.15.12:0/3837310990 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f32380a3d20 0x7f32380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.548+0000 7f3245ffb640 1 -- 172.21.15.12:0/3837310990 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f324c0d02e0 con 0x7f32501031d0 2024-04-22T13:34:05.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.548+0000 7f3255272640 1 --2- 172.21.15.12:0/111835588 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1ed16c4d0 0x55b1ed16c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.548+0000 7f3254a71640 1 --2- 172.21.15.12:0/111835588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1ed16ce80 0x55b1ed175430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.548+0000 7f3247fff640 1 -- 172.21.15.12:0/111835588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1ed16ce80 msgr2=0x55b1ed175430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.548+0000 7f3247fff640 1 --2- 172.21.15.12:0/111835588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1ed16ce80 0x55b1ed175430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.548+0000 7f3247fff640 1 -- 172.21.15.12:0/111835588 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1ed16c4d0 msgr2=0x55b1ed16c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.548+0000 7f3247fff640 1 --2- 172.21.15.12:0/111835588 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1ed16c4d0 0x55b1ed16c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.548+0000 7f3247fff640 1 -- 172.21.15.12:0/111835588 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1ed0601b0 con 0x55b1ed175a00 2024-04-22T13:34:05.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.549+0000 7f3247fff640 1 --2- 172.21.15.12:0/111835588 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1ed175a00 0x55b1ed177e50 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f3240009900 tx=0x7f324002c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b376a74b9810f94 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:05.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.549+0000 7f3255272640 1 --2- 172.21.15.12:0/3837310990 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f32380a3d20 0x7f32380a61e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3248005c40 tx=0x7f32480094a0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:05.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.549+0000 7f3236ffd640 1 -- 172.21.15.12:0/111835588 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3240004180 con 0x55b1ed175a00 2024-04-22T13:34:05.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.549+0000 7f3236ffd640 1 -- 172.21.15.12:0/111835588 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f324002cdc0 con 0x55b1ed175a00 2024-04-22T13:34:05.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.549+0000 7f3236ffd640 1 -- 172.21.15.12:0/111835588 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32400046c0 con 0x55b1ed175a00 2024-04-22T13:34:05.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.549+0000 7f325999c740 1 -- 172.21.15.12:0/111835588 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1ed175a00 msgr2=0x55b1ed177e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.549+0000 7f325999c740 1 --2- 172.21.15.12:0/111835588 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1ed175a00 0x55b1ed177e50 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f3240009900 tx=0x7f324002c7f0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.549+0000 7f325999c740 1 -- 172.21.15.12:0/111835588 shutdown_connections 2024-04-22T13:34:05.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.549+0000 7f325999c740 1 --2- 172.21.15.12:0/111835588 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1ed175a00 0x55b1ed177e50 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.549+0000 7f325999c740 1 --2- 172.21.15.12:0/111835588 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1ed16ce80 0x55b1ed175430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.549+0000 7f325999c740 1 --2- 172.21.15.12:0/111835588 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1ed16c4d0 0x55b1ed16c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.549+0000 7f325999c740 1 -- 172.21.15.12:0/111835588 >> 172.21.15.12:0/111835588 conn(0x55b1ed166ba0 msgr2=0x55b1ed166f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:05.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.550+0000 7f325999c740 1 -- 172.21.15.12:0/111835588 shutdown_connections 2024-04-22T13:34:05.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.550+0000 7f325999c740 1 -- 172.21.15.12:0/111835588 wait complete. 2024-04-22T13:34:05.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.550+0000 7f325999c740 1 -- start start 2024-04-22T13:34:05.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.550+0000 7f325999c740 1 Processor -- start 2024-04-22T13:34:05.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.550+0000 7f325999c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1ed16c4d0 0x55b1ed1752a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.550+0000 7f325999c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1ed16ce80 0x55b1ed172630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.550+0000 7f325999c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1ed175a00 0x55b1ed172b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.550+0000 7f325999c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55b1ed178910 con 0x55b1ed16ce80 2024-04-22T13:34:05.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.550+0000 7f325999c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b1ed08a500 con 0x55b1ed175a00 2024-04-22T13:34:05.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.550+0000 7f325999c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b1ed178a90 con 0x55b1ed16c4d0 2024-04-22T13:34:05.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.550+0000 7f325999c740 10 client.0 _ll_register_callbacks cb 0x55b1ed058b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:05.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.550+0000 7f3247fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1ed16ce80 0x55b1ed172630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.550+0000 7f3247fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1ed16ce80 0x55b1ed172630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36798/0 (socket says 172.21.15.12:36798) 2024-04-22T13:34:05.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.550+0000 7f3247fff640 1 -- 172.21.15.12:0/4134199122 learned_addr learned my addr 172.21.15.12:0/4134199122 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:05.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.551+0000 7f3255272640 1 --2- 172.21.15.12:0/4134199122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1ed175a00 0x55b1ed172b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.551+0000 7f3254a71640 1 --2- 172.21.15.12:0/4134199122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1ed16c4d0 0x55b1ed1752a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.551+0000 7f3247fff640 1 -- 172.21.15.12:0/4134199122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1ed16c4d0 msgr2=0x55b1ed1752a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.551+0000 7f3247fff640 1 --2- 172.21.15.12:0/4134199122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1ed16c4d0 0x55b1ed1752a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.551+0000 7f3247fff640 1 -- 172.21.15.12:0/4134199122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1ed175a00 msgr2=0x55b1ed172b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.551+0000 7f3247fff640 1 --2- 172.21.15.12:0/4134199122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1ed175a00 0x55b1ed172b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.551+0000 7f3247fff640 1 -- 172.21.15.12:0/4134199122 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1ecde33f0 con 0x55b1ed16ce80 2024-04-22T13:34:05.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.551+0000 7f3247fff640 1 --2- 172.21.15.12:0/4134199122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1ed16ce80 0x55b1ed172630 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f3240005740 tx=0x7f3240005310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c53a5039dc1cce server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:05.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.551+0000 7f322ffff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:34:05.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.551+0000 7f322ffff640 1 -- 172.21.15.12:0/4134199122 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32400037f0 con 0x55b1ed16ce80 2024-04-22T13:34:05.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.551+0000 7f322ffff640 1 -- 172.21.15.12:0/4134199122 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f3240003990 con 0x55b1ed16ce80 2024-04-22T13:34:05.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.551+0000 7f325999c740 1 -- 172.21.15.12:0/4134199122 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f324002cdc0 con 0x55b1ed16ce80 2024-04-22T13:34:05.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.551+0000 7f322ffff640 1 -- 172.21.15.12:0/4134199122 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f324005a050 con 0x55b1ed16ce80 2024-04-22T13:34:05.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.551+0000 7f325999c740 10 client.19047 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:05.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.551+0000 7f325999c740 1 -- 172.21.15.12:0/4134199122 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b1ecead7d0 con 0x55b1ed16ce80 2024-04-22T13:34:05.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.552+0000 7f322ffff640 1 -- 172.21.15.12:0/4134199122 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f324002cdc0 con 0x55b1ed16ce80 2024-04-22T13:34:05.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.552+0000 7f322ffff640 1 -- 172.21.15.12:0/4134199122 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3240034c00 con 0x55b1ed16ce80 2024-04-22T13:34:05.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.552+0000 7f325999c740 10 client.19047 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:05.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.552+0000 7f325999c740 10 client.19047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:34:05.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.552+0000 7f325999c740 20 client.19047 populate_metadata read hostname 'smithi012' 2024-04-22T13:34:05.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.552+0000 7f325999c740 1 --2- 172.21.15.12:0/4134199122 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b1ed1859e0 0x55b1ed187dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.552+0000 7f325999c740 4 client.19047 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:34:05.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.552+0000 7f325999c740 1 -- 172.21.15.12:0/4134199122 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1ed070930 con 0x55b1ed1859e0 2024-04-22T13:34:05.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.552+0000 7f3254a71640 1 --2- 172.21.15.12:0/4134199122 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b1ed1859e0 0x55b1ed187dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.552+0000 7f322ffff640 10 client.19047 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:34:05.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.552+0000 7f3254a71640 1 --2- 172.21.15.12:0/4134199122 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b1ed1859e0 0x55b1ed187dd0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18bd98f92c163fa7 server_cookie=7b1cc794507474f4 in_seq=0 out_seq=0 2024-04-22T13:34:05.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.553+0000 7f322ffff640 1 -- 172.21.15.12:0/4134199122 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b1ed070930 con 0x55b1ed1859e0 2024-04-22T13:34:05.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.553+0000 7f322ffff640 10 client.19047 handle_command_reply: tid=0 2024-04-22T13:34:05.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.553+0000 7f325999c740 1 client.19047 shutdown 2024-04-22T13:34:05.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.553+0000 7f325999c740 1 -- 172.21.15.12:0/4134199122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1ed16ce80 msgr2=0x55b1ed172630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.553+0000 7f325999c740 1 --2- 172.21.15.12:0/4134199122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1ed16ce80 0x55b1ed172630 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f3240005740 tx=0x7f3240005310 comp rx=0 tx=0).stop 2024-04-22T13:34:05.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.553+0000 7f325999c740 1 -- 172.21.15.12:0/4134199122 shutdown_connections 2024-04-22T13:34:05.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.553+0000 7f325999c740 1 --2- 172.21.15.12:0/4134199122 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b1ed1859e0 0x55b1ed187dd0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.553+0000 7f325999c740 1 --2- 172.21.15.12:0/4134199122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1ed175a00 0x55b1ed172b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.553+0000 7f325999c740 1 --2- 172.21.15.12:0/4134199122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1ed16ce80 0x55b1ed172630 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.553+0000 7f322ffff640 0 client.19047 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:34:05.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.553+0000 7f325999c740 1 --2- 172.21.15.12:0/4134199122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1ed16c4d0 0x55b1ed1752a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.553+0000 7f325999c740 1 -- 172.21.15.12:0/4134199122 >> 172.21.15.12:0/4134199122 conn(0x55b1ed166ba0 msgr2=0x55b1ed1688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:05.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.553+0000 7f325999c740 1 -- 172.21.15.12:0/4134199122 shutdown_connections 2024-04-22T13:34:05.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.553+0000 7f325999c740 1 -- 172.21.15.12:0/4134199122 wait complete. 2024-04-22T13:34:05.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.554+0000 7f325999c740 20 client.19047 trim_cache size 0 max 16384 2024-04-22T13:34:05.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.567+0000 7f325999c740 1 Processor -- start 2024-04-22T13:34:05.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.567+0000 7f325999c740 1 -- start start 2024-04-22T13:34:05.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.567+0000 7f325999c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1ed16c4d0 0x55b1ed1817d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.567+0000 7f325999c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1ed16ce80 0x55b1ed165a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.567+0000 7f325999c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1ed175a00 0x55b1ed174d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.567+0000 7f325999c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f32501afc10 con 0x55b1ed16c4d0 2024-04-22T13:34:05.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.567+0000 7f325999c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f32501b1750 con 0x55b1ed175a00 2024-04-22T13:34:05.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.567+0000 7f325999c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f32501b02e0 con 0x55b1ed16ce80 2024-04-22T13:34:05.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.567+0000 7f3254a71640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1ed16c4d0 0x55b1ed1817d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.567+0000 7f3254a71640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1ed16c4d0 0x55b1ed1817d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36812/0 (socket says 172.21.15.12:36812) 2024-04-22T13:34:05.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.567+0000 7f3254a71640 1 -- 172.21.15.12:0/613267512 learned_addr learned my addr 172.21.15.12:0/613267512 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:05.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.567+0000 7f3255272640 1 --2- 172.21.15.12:0/613267512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1ed175a00 0x55b1ed174d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.567+0000 7f3254a71640 1 -- 172.21.15.12:0/613267512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1ed16ce80 msgr2=0x55b1ed165a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.567+0000 7f3254a71640 1 --2- 172.21.15.12:0/613267512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1ed16ce80 0x55b1ed165a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.567+0000 7f3254a71640 1 -- 172.21.15.12:0/613267512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1ed175a00 msgr2=0x55b1ed174d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.569 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.567+0000 7f3254a71640 1 --2- 172.21.15.12:0/613267512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1ed175a00 0x55b1ed174d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.567+0000 7f3254a71640 1 -- 172.21.15.12:0/613267512 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1ed162470 con 0x55b1ed16c4d0 2024-04-22T13:34:05.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.568+0000 7f3254a71640 1 --2- 172.21.15.12:0/613267512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1ed16c4d0 0x55b1ed1817d0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f324c009610 tx=0x7f324c06abf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bab33efdb129a79 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:05.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.568+0000 7f32367fc640 1 -- 172.21.15.12:0/613267512 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c0ba070 con 0x55b1ed16c4d0 2024-04-22T13:34:05.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.568+0000 7f32367fc640 1 -- 172.21.15.12:0/613267512 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f324c0a4eb0 con 0x55b1ed16c4d0 2024-04-22T13:34:05.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.568+0000 7f32367fc640 1 -- 172.21.15.12:0/613267512 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c0b5070 con 0x55b1ed16c4d0 2024-04-22T13:34:05.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.568+0000 7f325999c740 1 -- 172.21.15.12:0/613267512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1ed16c4d0 msgr2=0x55b1ed1817d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.568+0000 7f325999c740 1 --2- 172.21.15.12:0/613267512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1ed16c4d0 0x55b1ed1817d0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f324c009610 tx=0x7f324c06abf0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.568+0000 7f325999c740 1 -- 172.21.15.12:0/613267512 shutdown_connections 2024-04-22T13:34:05.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.568+0000 7f325999c740 1 --2- 172.21.15.12:0/613267512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1ed175a00 0x55b1ed174d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.568+0000 7f325999c740 1 --2- 172.21.15.12:0/613267512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1ed16ce80 0x55b1ed165a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.568+0000 7f325999c740 1 --2- 172.21.15.12:0/613267512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1ed16c4d0 0x55b1ed1817d0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.570 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.568+0000 7f325999c740 1 -- 172.21.15.12:0/613267512 >> 172.21.15.12:0/613267512 conn(0x55b1ed166ba0 msgr2=0x55b1ed165370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:05.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.568+0000 7f325999c740 1 -- 172.21.15.12:0/613267512 shutdown_connections 2024-04-22T13:34:05.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.568+0000 7f325999c740 1 -- 172.21.15.12:0/613267512 wait complete. 2024-04-22T13:34:05.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.569+0000 7f325999c740 1 -- start start 2024-04-22T13:34:05.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.569+0000 7f325999c740 1 Processor -- start 2024-04-22T13:34:05.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.569+0000 7f325999c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1ed16c4d0 0x55b1ed17e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.569+0000 7f325999c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1ed16ce80 0x55b1ed179420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.569+0000 7f325999c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1ed175a00 0x55b1ed179960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.569+0000 7f325999c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f32501b1080 con 0x55b1ed175a00 2024-04-22T13:34:05.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.569+0000 7f325999c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f32501b09b0 con 0x55b1ed16ce80 2024-04-22T13:34:05.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.569+0000 7f325999c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f32501adc10 con 0x55b1ed16c4d0 2024-04-22T13:34:05.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.569+0000 7f325999c740 10 client.0 _ll_register_callbacks cb 0x7f324002ccd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:05.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.569+0000 7f3255272640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1ed175a00 0x55b1ed179960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.569+0000 7f3255272640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1ed175a00 0x55b1ed179960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36816/0 (socket says 172.21.15.12:36816) 2024-04-22T13:34:05.571 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.569+0000 7f3255272640 1 -- 172.21.15.12:0/994015368 learned_addr learned my addr 172.21.15.12:0/994015368 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:05.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.569+0000 7f3254a71640 1 --2- 172.21.15.12:0/994015368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1ed16c4d0 0x55b1ed17e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.569+0000 7f3247fff640 1 --2- 172.21.15.12:0/994015368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1ed16ce80 0x55b1ed179420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.570+0000 7f3255272640 1 -- 172.21.15.12:0/994015368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1ed16c4d0 msgr2=0x55b1ed17e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.570+0000 7f3255272640 1 --2- 172.21.15.12:0/994015368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1ed16c4d0 0x55b1ed17e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.570+0000 7f3255272640 1 -- 172.21.15.12:0/994015368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1ed16ce80 msgr2=0x55b1ed179420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:05.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.570+0000 7f3255272640 1 --2- 172.21.15.12:0/994015368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1ed16ce80 0x55b1ed179420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.570+0000 7f3255272640 1 -- 172.21.15.12:0/994015368 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1ed17a140 con 0x55b1ed175a00 2024-04-22T13:34:05.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.570+0000 7f3255272640 1 --2- 172.21.15.12:0/994015368 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1ed175a00 0x55b1ed179960 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f3248005fd0 tx=0x7f3248004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8daed9296daee95 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:05.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.570+0000 7f322ffff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:34:05.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.570+0000 7f322ffff640 1 -- 172.21.15.12:0/994015368 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3248004d20 con 0x55b1ed175a00 2024-04-22T13:34:05.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.570+0000 7f322ffff640 1 -- 172.21.15.12:0/994015368 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f3248021970 con 0x55b1ed175a00 2024-04-22T13:34:05.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.570+0000 7f325999c740 1 -- 172.21.15.12:0/994015368 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f324c0a4eb0 con 0x55b1ed175a00 2024-04-22T13:34:05.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.570+0000 7f322ffff640 1 -- 172.21.15.12:0/994015368 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3248021b70 con 0x55b1ed175a00 2024-04-22T13:34:05.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.570+0000 7f325999c740 10 client.19053 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:05.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.570+0000 7f325999c740 1 -- 172.21.15.12:0/994015368 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b1ed1ebd10 con 0x55b1ed175a00 2024-04-22T13:34:05.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.570+0000 7f322ffff640 1 -- 172.21.15.12:0/994015368 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f324c0a4eb0 con 0x55b1ed175a00 2024-04-22T13:34:05.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.570+0000 7f322ffff640 1 -- 172.21.15.12:0/994015368 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3248030020 con 0x55b1ed175a00 2024-04-22T13:34:05.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.570+0000 7f325999c740 10 client.19053 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:05.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.570+0000 7f325999c740 10 client.19053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:34:05.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.570+0000 7f325999c740 20 client.19053 populate_metadata read hostname 'smithi012' 2024-04-22T13:34:05.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.570+0000 7f325999c740 1 --2- 172.21.15.12:0/994015368 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b1ed1ec4e0 0x55b1ed1ee900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:05.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.570+0000 7f325999c740 4 client.19053 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:34:05.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.570+0000 7f325999c740 1 -- 172.21.15.12:0/994015368 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b1ed172f00 con 0x55b1ed1ec4e0 2024-04-22T13:34:05.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.571+0000 7f3254a71640 1 --2- 172.21.15.12:0/994015368 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b1ed1ec4e0 0x55b1ed1ee900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:05.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.571+0000 7f322ffff640 10 client.19053 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:34:05.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.571+0000 7f3254a71640 1 --2- 172.21.15.12:0/994015368 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b1ed1ec4e0 0x55b1ed1ee900 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ad15344be20793c server_cookie=1b75e214c2c476e4 in_seq=0 out_seq=0 2024-04-22T13:34:05.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.571+0000 7f322ffff640 1 -- 172.21.15.12:0/994015368 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b1ed172f00 con 0x55b1ed1ec4e0 2024-04-22T13:34:05.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.571+0000 7f322ffff640 10 client.19053 handle_command_reply: tid=0 2024-04-22T13:34:05.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.571+0000 7f325999c740 1 client.19053 shutdown 2024-04-22T13:34:05.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.571+0000 7f325999c740 1 -- 172.21.15.12:0/994015368 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1ed175a00 msgr2=0x55b1ed179960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.571+0000 7f325999c740 1 --2- 172.21.15.12:0/994015368 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1ed175a00 0x55b1ed179960 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f3248005fd0 tx=0x7f3248004ab0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.571+0000 7f325999c740 1 -- 172.21.15.12:0/994015368 shutdown_connections 2024-04-22T13:34:05.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.571+0000 7f325999c740 1 --2- 172.21.15.12:0/994015368 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55b1ed1ec4e0 0x55b1ed1ee900 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.571+0000 7f325999c740 1 --2- 172.21.15.12:0/994015368 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55b1ed175a00 0x55b1ed179960 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.571+0000 7f325999c740 1 --2- 172.21.15.12:0/994015368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b1ed16ce80 0x55b1ed179420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.571+0000 7f325999c740 1 --2- 172.21.15.12:0/994015368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1ed16c4d0 0x55b1ed17e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.571+0000 7f322ffff640 0 client.19053 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:34:05.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.571+0000 7f325999c740 1 -- 172.21.15.12:0/994015368 >> 172.21.15.12:0/994015368 conn(0x55b1ed166ba0 msgr2=0x55b1ed168860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:05.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.571+0000 7f325999c740 1 -- 172.21.15.12:0/994015368 shutdown_connections 2024-04-22T13:34:05.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.572+0000 7f325999c740 1 -- 172.21.15.12:0/994015368 wait complete. 2024-04-22T13:34:05.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.572+0000 7f325999c740 20 client.19053 trim_cache size 0 max 16384 2024-04-22T13:34:05.575 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:34:05.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.572+0000 7f3256cd5640 1 -- 172.21.15.12:0/3837310990 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f32380a3d20 msgr2=0x7f32380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.572+0000 7f3256cd5640 1 --2- 172.21.15.12:0/3837310990 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f32380a3d20 0x7f32380a61e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3248005c40 tx=0x7f32480094a0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.572+0000 7f3256cd5640 1 -- 172.21.15.12:0/3837310990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32501031d0 msgr2=0x7f3250100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:05.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.572+0000 7f3256cd5640 1 --2- 172.21.15.12:0/3837310990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32501031d0 0x7f3250100e20 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f324c00d770 tx=0x7f324c00dc30 comp rx=0 tx=0).stop 2024-04-22T13:34:05.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.572+0000 7f3256cd5640 1 -- 172.21.15.12:0/3837310990 shutdown_connections 2024-04-22T13:34:05.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.572+0000 7f3256cd5640 1 --2- 172.21.15.12:0/3837310990 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f32380a3d20 0x7f32380a61e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.572+0000 7f3256cd5640 1 --2- 172.21.15.12:0/3837310990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f325010c510 0x7f32501018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.572+0000 7f3256cd5640 1 --2- 172.21.15.12:0/3837310990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3250103af0 0x7f3250101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.572+0000 7f3256cd5640 1 --2- 172.21.15.12:0/3837310990 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32501031d0 0x7f3250100e20 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:05.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.572+0000 7f3256cd5640 1 -- 172.21.15.12:0/3837310990 >> 172.21.15.12:0/3837310990 conn(0x7f3250077f50 msgr2=0x7f32500fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:05.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.572+0000 7f3256cd5640 1 -- 172.21.15.12:0/3837310990 shutdown_connections 2024-04-22T13:34:05.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:05.572+0000 7f3256cd5640 1 -- 172.21.15.12:0/3837310990 wait complete. 2024-04-22T13:34:05.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:05 smithi012 ceph-mon[28265]: pgmap v308: 97 pgs: 97 active+clean; 2.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:34:05.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:05 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3409522628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:34:05.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:05 smithi111 ceph-mon[31361]: pgmap v308: 97 pgs: 97 active+clean; 2.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:34:05.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:05 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/3409522628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:34:06.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:05 smithi134 ceph-mon[31338]: pgmap v308: 97 pgs: 97 active+clean; 2.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:34:06.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:05 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/3409522628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:34:06.579 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-22T13:34:06.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:34:06.580 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-22T13:34:06.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.704+0000 7f6135332640 1 -- 172.21.15.12:0/3370817249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6130102210 msgr2=0x7f61301025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:06.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.704+0000 7f6135332640 1 --2- 172.21.15.12:0/3370817249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6130102210 0x7f61301025f0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f6128009900 tx=0x7f612802bc30 comp rx=0 tx=0).stop 2024-04-22T13:34:06.702 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.704+0000 7f6135332640 1 -- 172.21.15.12:0/3370817249 shutdown_connections 2024-04-22T13:34:06.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.704+0000 7f6135332640 1 --2- 172.21.15.12:0/3370817249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61301072e0 0x7f613010eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.704+0000 7f6135332640 1 --2- 172.21.15.12:0/3370817249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6130102bc0 0x7f6130106c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.704+0000 7f6135332640 1 --2- 172.21.15.12:0/3370817249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6130102210 0x7f61301025f0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.704+0000 7f6135332640 1 -- 172.21.15.12:0/3370817249 >> 172.21.15.12:0/3370817249 conn(0x7f61300fdab0 msgr2=0x7f61300ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:06.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.704+0000 7f6135332640 1 -- 172.21.15.12:0/3370817249 shutdown_connections 2024-04-22T13:34:06.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.704+0000 7f6135332640 1 -- 172.21.15.12:0/3370817249 wait complete. 2024-04-22T13:34:06.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.704+0000 7f6135332640 1 Processor -- start 2024-04-22T13:34:06.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.705+0000 7f6135332640 1 -- start start 2024-04-22T13:34:06.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.705+0000 7f6135332640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6130102210 0x7f613019e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.705+0000 7f6135332640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61301072e0 0x7f613019ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.705+0000 7f6135332640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6130198650 0x7f6130198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.705+0000 7f6135332640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6130112270 con 0x7f6130198650 2024-04-22T13:34:06.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.705+0000 7f6135332640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f61301120f0 con 0x7f6130102210 2024-04-22T13:34:06.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.705+0000 7f6135332640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f61301123f0 con 0x7f61301072e0 2024-04-22T13:34:06.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.705+0000 7f612f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6130198650 0x7f6130198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.705+0000 7f612f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6130198650 0x7f6130198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36832/0 (socket says 172.21.15.12:36832) 2024-04-22T13:34:06.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.705+0000 7f612f7fe640 1 -- 172.21.15.12:0/2581239472 learned_addr learned my addr 172.21.15.12:0/2581239472 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:06.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.705+0000 7f612effd640 1 --2- 172.21.15.12:0/2581239472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6130102210 0x7f613019e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.705+0000 7f612e7fc640 1 --2- 172.21.15.12:0/2581239472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61301072e0 0x7f613019ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.705+0000 7f612f7fe640 1 -- 172.21.15.12:0/2581239472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61301072e0 msgr2=0x7f613019ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:06.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.705+0000 7f612f7fe640 1 --2- 172.21.15.12:0/2581239472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61301072e0 0x7f613019ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.705+0000 7f612f7fe640 1 -- 172.21.15.12:0/2581239472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6130102210 msgr2=0x7f613019e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:06.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.705+0000 7f612f7fe640 1 --2- 172.21.15.12:0/2581239472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6130102210 0x7f613019e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.705+0000 7f612f7fe640 1 -- 172.21.15.12:0/2581239472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6130199250 con 0x7f6130198650 2024-04-22T13:34:06.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.706+0000 7f612f7fe640 1 --2- 172.21.15.12:0/2581239472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6130198650 0x7f6130198ad0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f6120009fd0 tx=0x7f612000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:06.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.706+0000 7f6117fff640 1 -- 172.21.15.12:0/2581239472 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120017070 con 0x7f6130198650 2024-04-22T13:34:06.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.706+0000 7f6135332640 1 -- 172.21.15.12:0/2581239472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61301a5330 con 0x7f6130198650 2024-04-22T13:34:06.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.706+0000 7f6117fff640 1 -- 172.21.15.12:0/2581239472 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6120003e20 con 0x7f6130198650 2024-04-22T13:34:06.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.706+0000 7f6117fff640 1 -- 172.21.15.12:0/2581239472 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120010460 con 0x7f6130198650 2024-04-22T13:34:06.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.706+0000 7f6135332640 1 -- 172.21.15.12:0/2581239472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61301a56e0 con 0x7f6130198650 2024-04-22T13:34:06.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.708+0000 7f6117fff640 1 -- 172.21.15.12:0/2581239472 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6120003970 con 0x7f6130198650 2024-04-22T13:34:06.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.708+0000 7f6138002740 1 Processor -- start 2024-04-22T13:34:06.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.708+0000 7f6138002740 1 -- start start 2024-04-22T13:34:06.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.709+0000 7f6117fff640 1 --2- 172.21.15.12:0/2581239472 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f61180a3d20 0x7f61180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.709+0000 7f6138002740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e93f8c8e0 0x557e93f8ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.709+0000 7f6138002740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e93f8d290 0x557e93f95840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.709+0000 7f6138002740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e93f95e10 0x557e93f98260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.709+0000 7f6138002740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557e93ef0f00 con 0x557e93f8c8e0 2024-04-22T13:34:06.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.709+0000 7f6138002740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557e93e8e590 con 0x557e93f8d290 2024-04-22T13:34:06.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.709+0000 7f6138002740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557e93eecec0 con 0x557e93f95e10 2024-04-22T13:34:06.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.709+0000 7f612e7fc640 1 --2- 172.21.15.12:0/2581239472 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f61180a3d20 0x7f61180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.709+0000 7f6117fff640 1 -- 172.21.15.12:0/2581239472 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f61200d0490 con 0x7f6130198650 2024-04-22T13:34:06.714 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.709+0000 7f612effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e93f8c8e0 0x557e93f8ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.709+0000 7f612effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e93f8c8e0 0x557e93f8ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36848/0 (socket says 172.21.15.12:36848) 2024-04-22T13:34:06.715 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.709+0000 7f612effd640 1 -- 172.21.15.12:0/3587433414 learned_addr learned my addr 172.21.15.12:0/3587433414 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:06.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.709+0000 7f612e7fc640 1 --2- 172.21.15.12:0/3587433414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e93f8d290 0x557e93f95840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.709+0000 7f612f7fe640 1 --2- 172.21.15.12:0/3587433414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e93f95e10 0x557e93f98260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.709+0000 7f612effd640 1 -- 172.21.15.12:0/3587433414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e93f95e10 msgr2=0x557e93f98260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:06.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.709+0000 7f612effd640 1 --2- 172.21.15.12:0/3587433414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e93f95e10 0x557e93f98260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.709+0000 7f612effd640 1 -- 172.21.15.12:0/3587433414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e93f8d290 msgr2=0x557e93f95840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:06.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.709+0000 7f612effd640 1 --2- 172.21.15.12:0/3587433414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e93f8d290 0x557e93f95840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.709+0000 7f612effd640 1 -- 172.21.15.12:0/3587433414 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e93ccb7e0 con 0x557e93f8c8e0 2024-04-22T13:34:06.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.709+0000 7f612f7fe640 1 --2- 172.21.15.12:0/3587433414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e93f95e10 0x557e93f98260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:06.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.709+0000 7f612e7fc640 1 --2- 172.21.15.12:0/2581239472 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f61180a3d20 0x7f61180a61e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6130199ea0 tx=0x7f6124013880 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:06.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.710+0000 7f612effd640 1 --2- 172.21.15.12:0/3587433414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e93f8c8e0 0x557e93f8ccc0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f6128009230 tx=0x7f612802c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f834c5574040a287 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:06.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.710+0000 7f61157fa640 1 -- 172.21.15.12:0/3587433414 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6128045020 con 0x557e93f8c8e0 2024-04-22T13:34:06.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.710+0000 7f61157fa640 1 -- 172.21.15.12:0/3587433414 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f612802c7e0 con 0x557e93f8c8e0 2024-04-22T13:34:06.716 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.710+0000 7f61157fa640 1 -- 172.21.15.12:0/3587433414 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61280046c0 con 0x557e93f8c8e0 2024-04-22T13:34:06.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.710+0000 7f6138002740 1 -- 172.21.15.12:0/3587433414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e93f8c8e0 msgr2=0x557e93f8ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:06.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.710+0000 7f6138002740 1 --2- 172.21.15.12:0/3587433414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e93f8c8e0 0x557e93f8ccc0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f6128009230 tx=0x7f612802c720 comp rx=0 tx=0).stop 2024-04-22T13:34:06.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.710+0000 7f6138002740 1 -- 172.21.15.12:0/3587433414 shutdown_connections 2024-04-22T13:34:06.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.710+0000 7f6138002740 1 --2- 172.21.15.12:0/3587433414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e93f95e10 0x557e93f98260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.710+0000 7f6138002740 1 --2- 172.21.15.12:0/3587433414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e93f8d290 0x557e93f95840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.710+0000 7f6138002740 1 --2- 172.21.15.12:0/3587433414 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e93f8c8e0 0x557e93f8ccc0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.710+0000 7f6138002740 1 -- 172.21.15.12:0/3587433414 >> 172.21.15.12:0/3587433414 conn(0x557e93f86fb0 msgr2=0x557e93f87390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:06.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.711+0000 7f6138002740 1 -- 172.21.15.12:0/3587433414 shutdown_connections 2024-04-22T13:34:06.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.711+0000 7f6138002740 1 -- 172.21.15.12:0/3587433414 wait complete. 2024-04-22T13:34:06.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.711+0000 7f6138002740 1 -- start start 2024-04-22T13:34:06.717 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.711+0000 7f6138002740 1 Processor -- start 2024-04-22T13:34:06.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.711+0000 7f6138002740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e93f8c8e0 0x557e93f92df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.711+0000 7f6138002740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e93f8d290 0x557e93f93330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.711+0000 7f6138002740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e93f95e10 0x557e93f93870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.711+0000 7f6138002740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x557e93f98d20 con 0x557e93f8d290 2024-04-22T13:34:06.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.711+0000 7f6138002740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557e93eaa580 con 0x557e93f8c8e0 2024-04-22T13:34:06.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.711+0000 7f6138002740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557e93f98ea0 con 0x557e93f95e10 2024-04-22T13:34:06.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.711+0000 7f6138002740 10 client.0 _ll_register_callbacks cb 0x557e93eb0650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:06.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.711+0000 7f612e7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e93f8d290 0x557e93f93330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.718 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.711+0000 7f612e7fc640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e93f8d290 0x557e93f93330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36864/0 (socket says 172.21.15.12:36864) 2024-04-22T13:34:06.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.711+0000 7f612e7fc640 1 -- 172.21.15.12:0/1275294870 learned_addr learned my addr 172.21.15.12:0/1275294870 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:06.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.711+0000 7f612effd640 1 --2- 172.21.15.12:0/1275294870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e93f8c8e0 0x557e93f92df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.711+0000 7f612f7fe640 1 --2- 172.21.15.12:0/1275294870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e93f95e10 0x557e93f93870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.712+0000 7f612e7fc640 1 -- 172.21.15.12:0/1275294870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e93f95e10 msgr2=0x557e93f93870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:06.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.712+0000 7f612e7fc640 1 --2- 172.21.15.12:0/1275294870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e93f95e10 0x557e93f93870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.712+0000 7f612e7fc640 1 -- 172.21.15.12:0/1275294870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e93f8c8e0 msgr2=0x557e93f92df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:06.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.712+0000 7f612e7fc640 1 --2- 172.21.15.12:0/1275294870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e93f8c8e0 0x557e93f92df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.712+0000 7f612e7fc640 1 -- 172.21.15.12:0/1275294870 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e93d2bf70 con 0x557e93f8d290 2024-04-22T13:34:06.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.712+0000 7f612e7fc640 1 --2- 172.21.15.12:0/1275294870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e93f8d290 0x557e93f93330 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f613006b610 tx=0x7f6124019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467fe75a545d3ced server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:06.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.712+0000 7f6112ff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:34:06.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.712+0000 7f6112ff5640 1 -- 172.21.15.12:0/1275294870 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612400ee40 con 0x557e93f8d290 2024-04-22T13:34:06.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.712+0000 7f6112ff5640 1 -- 172.21.15.12:0/1275294870 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f6124005020 con 0x557e93f8d290 2024-04-22T13:34:06.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.712+0000 7f6138002740 1 -- 172.21.15.12:0/1275294870 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f612802c7e0 con 0x557e93f8d290 2024-04-22T13:34:06.719 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.712+0000 7f6112ff5640 1 -- 172.21.15.12:0/1275294870 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6124005220 con 0x557e93f8d290 2024-04-22T13:34:06.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.712+0000 7f6138002740 10 client.19065 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:06.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.712+0000 7f6138002740 1 -- 172.21.15.12:0/1275294870 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557e93bc3170 con 0x557e93f8d290 2024-04-22T13:34:06.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.712+0000 7f6112ff5640 1 -- 172.21.15.12:0/1275294870 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f612802c7e0 con 0x557e93f8d290 2024-04-22T13:34:06.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.712+0000 7f6112ff5640 1 -- 172.21.15.12:0/1275294870 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6124030020 con 0x557e93f8d290 2024-04-22T13:34:06.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.713+0000 7f6138002740 10 client.19065 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:06.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.713+0000 7f6138002740 10 client.19065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:34:06.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.713+0000 7f6138002740 20 client.19065 populate_metadata read hostname 'smithi012' 2024-04-22T13:34:06.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.713+0000 7f6138002740 1 --2- 172.21.15.12:0/1275294870 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557e93fa61f0 0x557e93fa8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.713+0000 7f6138002740 4 client.19065 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:34:06.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.713+0000 7f6138002740 1 -- 172.21.15.12:0/1275294870 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e93efe920 con 0x557e93fa61f0 2024-04-22T13:34:06.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.713+0000 7f612effd640 1 --2- 172.21.15.12:0/1275294870 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557e93fa61f0 0x557e93fa8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.713+0000 7f6112ff5640 10 client.19065 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:34:06.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.713+0000 7f612effd640 1 --2- 172.21.15.12:0/1275294870 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557e93fa61f0 0x557e93fa8660 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bd2388a9db3d75f server_cookie=fcf1676025b218fd in_seq=0 out_seq=0 2024-04-22T13:34:06.720 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.714+0000 7f6112ff5640 1 -- 172.21.15.12:0/1275294870 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557e93efe920 con 0x557e93fa61f0 2024-04-22T13:34:06.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.714+0000 7f6112ff5640 10 client.19065 handle_command_reply: tid=0 2024-04-22T13:34:06.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.714+0000 7f6138002740 1 client.19065 shutdown 2024-04-22T13:34:06.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.714+0000 7f6138002740 1 -- 172.21.15.12:0/1275294870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e93f8d290 msgr2=0x557e93f93330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:06.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.714+0000 7f6138002740 1 --2- 172.21.15.12:0/1275294870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e93f8d290 0x557e93f93330 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f613006b610 tx=0x7f6124019000 comp rx=0 tx=0).stop 2024-04-22T13:34:06.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.714+0000 7f6138002740 1 -- 172.21.15.12:0/1275294870 shutdown_connections 2024-04-22T13:34:06.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.714+0000 7f6138002740 1 --2- 172.21.15.12:0/1275294870 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557e93fa61f0 0x557e93fa8660 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.714+0000 7f6138002740 1 --2- 172.21.15.12:0/1275294870 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e93f95e10 0x557e93f93870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.714+0000 7f6138002740 1 --2- 172.21.15.12:0/1275294870 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e93f8d290 0x557e93f93330 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.714+0000 7f6138002740 1 --2- 172.21.15.12:0/1275294870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e93f8c8e0 0x557e93f92df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.714+0000 7f6112ff5640 0 client.19065 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:34:06.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.714+0000 7f6138002740 1 -- 172.21.15.12:0/1275294870 >> 172.21.15.12:0/1275294870 conn(0x557e93f86fb0 msgr2=0x557e93f88d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:06.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.714+0000 7f6138002740 1 -- 172.21.15.12:0/1275294870 shutdown_connections 2024-04-22T13:34:06.721 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.714+0000 7f6138002740 1 -- 172.21.15.12:0/1275294870 wait complete. 2024-04-22T13:34:06.722 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.714+0000 7f6138002740 20 client.19065 trim_cache size 0 max 16384 2024-04-22T13:34:06.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.732+0000 7f6138002740 1 Processor -- start 2024-04-22T13:34:06.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.732+0000 7f6138002740 1 -- start start 2024-04-22T13:34:06.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.732+0000 7f6138002740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e93f8c8e0 0x557e93f95520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.732+0000 7f6138002740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e93f8d290 0x557e93fa1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.732+0000 7f6138002740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e93f95e10 0x557e93f88b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.732+0000 7f6138002740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f61301adb70 con 0x557e93f95e10 2024-04-22T13:34:06.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.732+0000 7f6138002740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f61301af6b0 con 0x557e93f8d290 2024-04-22T13:34:06.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.732+0000 7f6138002740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f61301ae240 con 0x557e93f8c8e0 2024-04-22T13:34:06.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.732+0000 7f612f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e93f95e10 0x557e93f88b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.732+0000 7f612f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e93f95e10 0x557e93f88b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36872/0 (socket says 172.21.15.12:36872) 2024-04-22T13:34:06.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.732+0000 7f612f7fe640 1 -- 172.21.15.12:0/1254065590 learned_addr learned my addr 172.21.15.12:0/1254065590 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:06.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.732+0000 7f612e7fc640 1 --2- 172.21.15.12:0/1254065590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e93f8d290 0x557e93fa1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.732+0000 7f612f7fe640 1 -- 172.21.15.12:0/1254065590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e93f8c8e0 msgr2=0x557e93f95520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:06.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.732+0000 7f612f7fe640 1 --2- 172.21.15.12:0/1254065590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e93f8c8e0 0x557e93f95520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.732+0000 7f612f7fe640 1 -- 172.21.15.12:0/1254065590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e93f8d290 msgr2=0x557e93fa1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:06.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.732+0000 7f612f7fe640 1 --2- 172.21.15.12:0/1254065590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e93f8d290 0x557e93fa1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.732+0000 7f612f7fe640 1 -- 172.21.15.12:0/1254065590 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e93f8d030 con 0x557e93f95e10 2024-04-22T13:34:06.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.732+0000 7f612e7fc640 1 --2- 172.21.15.12:0/1254065590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e93f8d290 0x557e93fa1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:06.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.733+0000 7f612f7fe640 1 --2- 172.21.15.12:0/1254065590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e93f95e10 0x557e93f88b10 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f612000b760 tx=0x7f6120003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eebd28abceaff8 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:06.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.733+0000 7f6114ff9640 1 -- 172.21.15.12:0/1254065590 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61200b5070 con 0x557e93f95e10 2024-04-22T13:34:06.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.733+0000 7f6114ff9640 1 -- 172.21.15.12:0/1254065590 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61200a5060 con 0x557e93f95e10 2024-04-22T13:34:06.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.733+0000 7f6114ff9640 1 -- 172.21.15.12:0/1254065590 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120020040 con 0x557e93f95e10 2024-04-22T13:34:06.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.733+0000 7f6138002740 1 -- 172.21.15.12:0/1254065590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e93f95e10 msgr2=0x557e93f88b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:06.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.733+0000 7f6138002740 1 --2- 172.21.15.12:0/1254065590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e93f95e10 0x557e93f88b10 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f612000b760 tx=0x7f6120003c60 comp rx=0 tx=0).stop 2024-04-22T13:34:06.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.733+0000 7f6138002740 1 -- 172.21.15.12:0/1254065590 shutdown_connections 2024-04-22T13:34:06.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.733+0000 7f6138002740 1 --2- 172.21.15.12:0/1254065590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e93f95e10 0x557e93f88b10 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.733+0000 7f6138002740 1 --2- 172.21.15.12:0/1254065590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e93f8d290 0x557e93fa1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.733+0000 7f6138002740 1 --2- 172.21.15.12:0/1254065590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e93f8c8e0 0x557e93f95520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.733+0000 7f6138002740 1 -- 172.21.15.12:0/1254065590 >> 172.21.15.12:0/1254065590 conn(0x557e93f86fb0 msgr2=0x557e93f88400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:06.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.733+0000 7f6138002740 1 -- 172.21.15.12:0/1254065590 shutdown_connections 2024-04-22T13:34:06.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.733+0000 7f6138002740 1 -- 172.21.15.12:0/1254065590 wait complete. 2024-04-22T13:34:06.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.734+0000 7f6138002740 1 -- start start 2024-04-22T13:34:06.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.734+0000 7f6138002740 1 Processor -- start 2024-04-22T13:34:06.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.734+0000 7f6138002740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e93f8c8e0 0x557e93f99f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.734+0000 7f6138002740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e93f8d290 0x557e93f9a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.734+0000 7f6138002740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e93f95e10 0x557e9400b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.734+0000 7f6138002740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f61301aefe0 con 0x557e93f8c8e0 2024-04-22T13:34:06.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.734+0000 7f6138002740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f61301ae910 con 0x557e93f95e10 2024-04-22T13:34:06.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.734+0000 7f6138002740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f61301abb70 con 0x557e93f8d290 2024-04-22T13:34:06.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.734+0000 7f6138002740 10 client.0 _ll_register_callbacks cb 0x7f612400aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:06.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.734+0000 7f612effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e93f8c8e0 0x557e93f99f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.734+0000 7f612effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e93f8c8e0 0x557e93f99f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36876/0 (socket says 172.21.15.12:36876) 2024-04-22T13:34:06.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.734+0000 7f612effd640 1 -- 172.21.15.12:0/123528533 learned_addr learned my addr 172.21.15.12:0/123528533 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:06.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.734+0000 7f612f7fe640 1 --2- 172.21.15.12:0/123528533 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e93f95e10 0x557e9400b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.734+0000 7f612effd640 1 -- 172.21.15.12:0/123528533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e93f8d290 msgr2=0x557e93f9a490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-22T13:34:06.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.734+0000 7f612effd640 1 --2- 172.21.15.12:0/123528533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e93f8d290 0x557e93f9a490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.734+0000 7f612effd640 1 -- 172.21.15.12:0/123528533 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e93f95e10 msgr2=0x557e9400b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:06.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.734+0000 7f612effd640 1 --2- 172.21.15.12:0/123528533 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e93f95e10 0x557e9400b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.734+0000 7f612effd640 1 -- 172.21.15.12:0/123528533 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e9400bd20 con 0x557e93f8c8e0 2024-04-22T13:34:06.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.735+0000 7f612effd640 1 --2- 172.21.15.12:0/123528533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e93f8c8e0 0x557e93f99f50 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f6128009900 tx=0x7f612802c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bbf71bffc8d8635 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:06.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.735+0000 7f6112ff5640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:34:06.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.735+0000 7f6112ff5640 1 -- 172.21.15.12:0/123528533 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6128045040 con 0x557e93f8c8e0 2024-04-22T13:34:06.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.735+0000 7f6112ff5640 1 -- 172.21.15.12:0/123528533 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7f6128004040 con 0x557e93f8c8e0 2024-04-22T13:34:06.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.735+0000 7f6138002740 1 -- 172.21.15.12:0/123528533 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61200a5060 con 0x557e93f8c8e0 2024-04-22T13:34:06.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.735+0000 7f6112ff5640 1 -- 172.21.15.12:0/123528533 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6128004240 con 0x557e93f8c8e0 2024-04-22T13:34:06.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.735+0000 7f6138002740 10 client.19071 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:06.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.735+0000 7f6138002740 1 -- 172.21.15.12:0/123528533 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557e9400cf20 con 0x557e93f8c8e0 2024-04-22T13:34:06.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.735+0000 7f6112ff5640 1 -- 172.21.15.12:0/123528533 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61200a5060 con 0x557e93f8c8e0 2024-04-22T13:34:06.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.735+0000 7f6112ff5640 1 -- 172.21.15.12:0/123528533 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6128058070 con 0x557e93f8c8e0 2024-04-22T13:34:06.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.735+0000 7f6138002740 10 client.19071 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:06.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.735+0000 7f6138002740 10 client.19071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:34:06.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.735+0000 7f6138002740 20 client.19071 populate_metadata read hostname 'smithi012' 2024-04-22T13:34:06.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.735+0000 7f6138002740 1 --2- 172.21.15.12:0/123528533 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557e9400d1b0 0x557e9400f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.735+0000 7f6138002740 4 client.19071 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-22T13:34:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.735+0000 7f6138002740 1 -- 172.21.15.12:0/123528533 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557e93ef8790 con 0x557e9400d1b0 2024-04-22T13:34:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.736+0000 7f612e7fc640 1 --2- 172.21.15.12:0/123528533 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557e9400d1b0 0x557e9400f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.736+0000 7f6112ff5640 10 client.19071 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:34:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.736+0000 7f612e7fc640 1 --2- 172.21.15.12:0/123528533 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557e9400d1b0 0x557e9400f5a0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e6049110d3d0b8 server_cookie=32331b6ea9d071f in_seq=0 out_seq=0 2024-04-22T13:34:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.737+0000 7f6112ff5640 1 -- 172.21.15.12:0/123528533 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557e93ef8790 con 0x557e9400d1b0 2024-04-22T13:34:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.737+0000 7f6112ff5640 10 client.19071 handle_command_reply: tid=0 2024-04-22T13:34:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.737+0000 7f6138002740 1 client.19071 shutdown 2024-04-22T13:34:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.737+0000 7f6138002740 1 -- 172.21.15.12:0/123528533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e93f8c8e0 msgr2=0x557e93f99f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.737+0000 7f6138002740 1 --2- 172.21.15.12:0/123528533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e93f8c8e0 0x557e93f99f50 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f6128009900 tx=0x7f612802c3d0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.737+0000 7f6138002740 1 -- 172.21.15.12:0/123528533 shutdown_connections 2024-04-22T13:34:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.737+0000 7f6138002740 1 --2- 172.21.15.12:0/123528533 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x557e9400d1b0 0x557e9400f5a0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.737+0000 7f6138002740 1 --2- 172.21.15.12:0/123528533 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557e93f95e10 0x557e9400b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.737+0000 7f6138002740 1 --2- 172.21.15.12:0/123528533 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557e93f8d290 0x557e93f9a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.737+0000 7f6138002740 1 --2- 172.21.15.12:0/123528533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e93f8c8e0 0x557e93f99f50 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.737+0000 7f6138002740 1 -- 172.21.15.12:0/123528533 >> 172.21.15.12:0/123528533 conn(0x557e93f86fb0 msgr2=0x557e93f88c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.737+0000 7f6112ff5640 0 client.19071 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:34:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.737+0000 7f6138002740 1 -- 172.21.15.12:0/123528533 shutdown_connections 2024-04-22T13:34:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.737+0000 7f6138002740 1 -- 172.21.15.12:0/123528533 wait complete. 2024-04-22T13:34:06.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.738+0000 7f6138002740 20 client.19071 trim_cache size 0 max 16384 2024-04-22T13:34:06.740 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:34:06.740 INFO:teuthology.orchestra.run.smithi012.stdout: "return_code": 0, 2024-04-22T13:34:06.740 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_tag": "c2f84b8c-e51d-410e-8d94-26a97ead20e9", 2024-04-22T13:34:06.740 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": "asynchronous" 2024-04-22T13:34:06.740 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:34:06.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.738+0000 7f6135332640 1 -- 172.21.15.12:0/2581239472 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f61180a3d20 msgr2=0x7f61180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:06.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.738+0000 7f6135332640 1 --2- 172.21.15.12:0/2581239472 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f61180a3d20 0x7f61180a61e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6130199ea0 tx=0x7f6124013880 comp rx=0 tx=0).stop 2024-04-22T13:34:06.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.738+0000 7f6135332640 1 -- 172.21.15.12:0/2581239472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6130198650 msgr2=0x7f6130198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:06.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.738+0000 7f6135332640 1 --2- 172.21.15.12:0/2581239472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6130198650 0x7f6130198ad0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f6120009fd0 tx=0x7f612000b260 comp rx=0 tx=0).stop 2024-04-22T13:34:06.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.738+0000 7f6135332640 1 -- 172.21.15.12:0/2581239472 shutdown_connections 2024-04-22T13:34:06.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.738+0000 7f6135332640 1 --2- 172.21.15.12:0/2581239472 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f61180a3d20 0x7f61180a61e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.738+0000 7f6135332640 1 --2- 172.21.15.12:0/2581239472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6130198650 0x7f6130198ad0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.738+0000 7f6135332640 1 --2- 172.21.15.12:0/2581239472 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61301072e0 0x7f613019ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.738+0000 7f6135332640 1 --2- 172.21.15.12:0/2581239472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6130102210 0x7f613019e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.738+0000 7f6135332640 1 -- 172.21.15.12:0/2581239472 >> 172.21.15.12:0/2581239472 conn(0x7f61300fdab0 msgr2=0x7f61300fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:06.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.738+0000 7f6135332640 1 -- 172.21.15.12:0/2581239472 shutdown_connections 2024-04-22T13:34:06.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.738+0000 7f6135332640 1 -- 172.21.15.12:0/2581239472 wait complete. 2024-04-22T13:34:06.745 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:34:06.745 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:34:06.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.853+0000 7fb4a8315640 1 -- 172.21.15.12:0/2256538920 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4a006a5f0 msgr2=0x7fb4a006aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:06.852 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.853+0000 7fb4a8315640 1 --2- 172.21.15.12:0/2256538920 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4a006a5f0 0x7fb4a006aa70 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fb490009930 tx=0x7fb49002bc30 comp rx=0 tx=0).stop 2024-04-22T13:34:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.853+0000 7fb4a8315640 1 -- 172.21.15.12:0/2256538920 shutdown_connections 2024-04-22T13:34:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.853+0000 7fb4a8315640 1 --2- 172.21.15.12:0/2256538920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4a006afb0 0x7fb4a010edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.853+0000 7fb4a8315640 1 --2- 172.21.15.12:0/2256538920 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4a006a5f0 0x7fb4a006aa70 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.853+0000 7fb4a8315640 1 --2- 172.21.15.12:0/2256538920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4a0106f00 0x7fb4a01072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.853+0000 7fb4a8315640 1 -- 172.21.15.12:0/2256538920 >> 172.21.15.12:0/2256538920 conn(0x7fb4a0077c40 msgr2=0x7fb4a0078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.853+0000 7fb4a8315640 1 -- 172.21.15.12:0/2256538920 shutdown_connections 2024-04-22T13:34:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.853+0000 7fb4a8315640 1 -- 172.21.15.12:0/2256538920 wait complete. 2024-04-22T13:34:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.854+0000 7fb4a8315640 1 Processor -- start 2024-04-22T13:34:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.854+0000 7fb4a8315640 1 -- start start 2024-04-22T13:34:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.854+0000 7fb4a8315640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4a006a5f0 0x7fb4a0101170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.854+0000 7fb4a8315640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4a006afb0 0x7fb4a01016b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.854+0000 7fb4a8315640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4a0106f00 0x7fb4a0101bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.853 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.854+0000 7fb4a8315640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a0112170 con 0x7fb4a0106f00 2024-04-22T13:34:06.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.854+0000 7fb4a8315640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a0111ff0 con 0x7fb4a006a5f0 2024-04-22T13:34:06.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.854+0000 7fb4a8315640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01122f0 con 0x7fb4a006afb0 2024-04-22T13:34:06.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.854+0000 7fb4a68b2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4a0106f00 0x7fb4a0101bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.854+0000 7fb4a68b2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4a0106f00 0x7fb4a0101bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36886/0 (socket says 172.21.15.12:36886) 2024-04-22T13:34:06.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.854+0000 7fb4a68b2640 1 -- 172.21.15.12:0/244293014 learned_addr learned my addr 172.21.15.12:0/244293014 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:06.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.854+0000 7fb4a58b0640 1 --2- 172.21.15.12:0/244293014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4a006afb0 0x7fb4a01016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.854+0000 7fb4a60b1640 1 --2- 172.21.15.12:0/244293014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4a006a5f0 0x7fb4a0101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.855+0000 7fb4a68b2640 1 -- 172.21.15.12:0/244293014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4a006afb0 msgr2=0x7fb4a01016b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:06.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.855+0000 7fb4a68b2640 1 --2- 172.21.15.12:0/244293014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4a006afb0 0x7fb4a01016b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.855+0000 7fb4a68b2640 1 -- 172.21.15.12:0/244293014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4a006a5f0 msgr2=0x7fb4a0101170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:06.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.855+0000 7fb4a68b2640 1 --2- 172.21.15.12:0/244293014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4a006a5f0 0x7fb4a0101170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.854 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.855+0000 7fb4a68b2640 1 -- 172.21.15.12:0/244293014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a01a6b90 con 0x7fb4a0106f00 2024-04-22T13:34:06.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.855+0000 7fb4a60b1640 1 --2- 172.21.15.12:0/244293014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4a006a5f0 0x7fb4a0101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:06.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.855+0000 7fb4a68b2640 1 --2- 172.21.15.12:0/244293014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4a0106f00 0x7fb4a0101bf0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fb49800eaa0 tx=0x7fb49800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:06.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.855+0000 7fb4977fe640 1 -- 172.21.15.12:0/244293014 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49800cdb0 con 0x7fb4a0106f00 2024-04-22T13:34:06.858 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.855+0000 7fb4977fe640 1 -- 172.21.15.12:0/244293014 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb498005020 con 0x7fb4a0106f00 2024-04-22T13:34:06.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.855+0000 7fb4a8315640 1 -- 172.21.15.12:0/244293014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a01a6e80 con 0x7fb4a0106f00 2024-04-22T13:34:06.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.855+0000 7fb4977fe640 1 -- 172.21.15.12:0/244293014 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4980116d0 con 0x7fb4a0106f00 2024-04-22T13:34:06.859 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.855+0000 7fb4a8315640 1 -- 172.21.15.12:0/244293014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a01a7350 con 0x7fb4a0106f00 2024-04-22T13:34:06.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.858+0000 7fb4aafe1740 1 Processor -- start 2024-04-22T13:34:06.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.858+0000 7fb4977fe640 1 -- 172.21.15.12:0/244293014 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb4980051d0 con 0x7fb4a0106f00 2024-04-22T13:34:06.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.859+0000 7fb4aafe1740 1 -- start start 2024-04-22T13:34:06.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.859+0000 7fb4aafe1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dff8bc2580 0x55dff8bc2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.859+0000 7fb4aafe1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff8bc2ea0 0x55dff8bcb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.859+0000 7fb4aafe1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dff8bcba80 0x55dff8bcde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.859+0000 7fb4aafe1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55dff8b271e0 con 0x55dff8bcba80 2024-04-22T13:34:06.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.859+0000 7fb4aafe1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55dff8ac4390 con 0x55dff8bc2580 2024-04-22T13:34:06.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.859+0000 7fb4aafe1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dff8b231a0 con 0x55dff8bc2ea0 2024-04-22T13:34:06.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.859+0000 7fb4a58b0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dff8bcba80 0x55dff8bcde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.869 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.859+0000 7fb4a60b1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff8bc2ea0 0x55dff8bcb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.859+0000 7fb4a58b0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dff8bcba80 0x55dff8bcde70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36902/0 (socket says 172.21.15.12:36902) 2024-04-22T13:34:06.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.859+0000 7fb4a58b0640 1 -- 172.21.15.12:0/905366775 learned_addr learned my addr 172.21.15.12:0/905366775 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:06.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.859+0000 7fb4977fe640 1 --2- 172.21.15.12:0/244293014 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb48c0a3d50 0x7fb48c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.859+0000 7fb4a58b0640 1 --2- 172.21.15.12:0/905366775 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dff8bc2580 0x55dff8bc2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.859+0000 7fb4a60b1640 1 -- 172.21.15.12:0/905366775 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dff8bc2580 msgr2=0x55dff8bc2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:06.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.859+0000 7fb4a60b1640 1 --2- 172.21.15.12:0/905366775 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dff8bc2580 0x55dff8bc2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.859+0000 7fb4a60b1640 1 -- 172.21.15.12:0/905366775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dff8bcba80 msgr2=0x55dff8bcde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:06.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.859+0000 7fb4a60b1640 1 --2- 172.21.15.12:0/905366775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dff8bcba80 0x55dff8bcde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.859+0000 7fb4a68b2640 1 --2- 172.21.15.12:0/244293014 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb48c0a3d50 0x7fb48c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.859+0000 7fb4a60b1640 1 -- 172.21.15.12:0/905366775 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff89037d0 con 0x55dff8bc2ea0 2024-04-22T13:34:06.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.859+0000 7fb4a58b0640 1 --2- 172.21.15.12:0/905366775 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dff8bc2580 0x55dff8bc2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:06.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.859+0000 7fb4977fe640 1 -- 172.21.15.12:0/244293014 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7fb4980d09b0 con 0x7fb4a0106f00 2024-04-22T13:34:06.870 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.860+0000 7fb4a60b1640 1 --2- 172.21.15.12:0/905366775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff8bc2ea0 0x55dff8bcb540 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00bc30 tx=0x7fb49c00f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80121bbd49700a8b server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:06.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.860+0000 7fb4a68b2640 1 --2- 172.21.15.12:0/244293014 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb48c0a3d50 0x7fb48c0a6210 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb49800efd0 tx=0x7fb49801eb40 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:06.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.860+0000 7fb494bfe640 1 -- 172.21.15.12:0/905366775 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c015070 con 0x55dff8bc2ea0 2024-04-22T13:34:06.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.860+0000 7fb494bfe640 1 -- 172.21.15.12:0/905366775 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb49c0054d0 con 0x55dff8bc2ea0 2024-04-22T13:34:06.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.860+0000 7fb494bfe640 1 -- 172.21.15.12:0/905366775 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c011440 con 0x55dff8bc2ea0 2024-04-22T13:34:06.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.860+0000 7fb4aafe1740 1 -- 172.21.15.12:0/905366775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff8bc2ea0 msgr2=0x55dff8bcb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:06.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.860+0000 7fb4aafe1740 1 --2- 172.21.15.12:0/905366775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff8bc2ea0 0x55dff8bcb540 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00bc30 tx=0x7fb49c00f730 comp rx=0 tx=0).stop 2024-04-22T13:34:06.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.861+0000 7fb4aafe1740 1 -- 172.21.15.12:0/905366775 shutdown_connections 2024-04-22T13:34:06.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.861+0000 7fb4aafe1740 1 --2- 172.21.15.12:0/905366775 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dff8bcba80 0x55dff8bcde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.861+0000 7fb4aafe1740 1 --2- 172.21.15.12:0/905366775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff8bc2ea0 0x55dff8bcb540 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.861+0000 7fb4aafe1740 1 --2- 172.21.15.12:0/905366775 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dff8bc2580 0x55dff8bc2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.861+0000 7fb4aafe1740 1 -- 172.21.15.12:0/905366775 >> 172.21.15.12:0/905366775 conn(0x55dff8bbccb0 msgr2=0x55dff8bbd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:06.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.861+0000 7fb4aafe1740 1 -- 172.21.15.12:0/905366775 shutdown_connections 2024-04-22T13:34:06.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.861+0000 7fb4aafe1740 1 -- 172.21.15.12:0/905366775 wait complete. 2024-04-22T13:34:06.871 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.861+0000 7fb4aafe1740 1 -- start start 2024-04-22T13:34:06.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.861+0000 7fb4aafe1740 1 Processor -- start 2024-04-22T13:34:06.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.861+0000 7fb4aafe1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dff8bc2580 0x55dff8bc8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.861+0000 7fb4aafe1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff8bc2ea0 0x55dff8bc9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.861+0000 7fb4aafe1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dff8bcba80 0x55dff8bc9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.861+0000 7fb4aafe1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55dff8bce870 con 0x55dff8bc2580 2024-04-22T13:34:06.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.861+0000 7fb4aafe1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55dff8ae02f0 con 0x55dff8bcba80 2024-04-22T13:34:06.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.861+0000 7fb4aafe1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dff8bce9f0 con 0x55dff8bc2ea0 2024-04-22T13:34:06.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.861+0000 7fb4aafe1740 10 client.0 _ll_register_callbacks cb 0x55dff8aed580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:06.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.861+0000 7fb4a58b0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dff8bc2580 0x55dff8bc8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.861+0000 7fb4a58b0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dff8bc2580 0x55dff8bc8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36910/0 (socket says 172.21.15.12:36910) 2024-04-22T13:34:06.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.861+0000 7fb4a58b0640 1 -- 172.21.15.12:0/322291434 learned_addr learned my addr 172.21.15.12:0/322291434 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:06.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.862+0000 7fb4a60b1640 1 --2- 172.21.15.12:0/322291434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff8bc2ea0 0x55dff8bc9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.872 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.862+0000 7fb4a58b0640 1 -- 172.21.15.12:0/322291434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff8bc2ea0 msgr2=0x55dff8bc9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:06.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.862+0000 7fb4a58b0640 1 --2- 172.21.15.12:0/322291434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff8bc2ea0 0x55dff8bc9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.862+0000 7fb4a58b0640 1 -- 172.21.15.12:0/322291434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dff8bcba80 msgr2=0x55dff8bc9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:06.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.862+0000 7fb4a58b0640 1 --2- 172.21.15.12:0/322291434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dff8bcba80 0x55dff8bc9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.862+0000 7fb4a58b0640 1 -- 172.21.15.12:0/322291434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dff88393f0 con 0x55dff8bc2580 2024-04-22T13:34:06.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.862+0000 7fb4a60b1640 1 --2- 172.21.15.12:0/322291434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff8bc2ea0 0x55dff8bc9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:06.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.862+0000 7fb4a58b0640 1 --2- 172.21.15.12:0/322291434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dff8bc2580 0x55dff8bc8ad0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fb490035000 tx=0x7fb490005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f17fba29ab32d659 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:06.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.862+0000 7fb489ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:34:06.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.862+0000 7fb489ffb640 1 -- 172.21.15.12:0/322291434 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb490047020 con 0x55dff8bc2580 2024-04-22T13:34:06.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.862+0000 7fb4aafe1740 1 -- 172.21.15.12:0/322291434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb49c0054d0 con 0x55dff8bc2580 2024-04-22T13:34:06.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.862+0000 7fb489ffb640 1 -- 172.21.15.12:0/322291434 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7fb490005050 con 0x55dff8bc2580 2024-04-22T13:34:06.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.862+0000 7fb489ffb640 1 -- 172.21.15.12:0/322291434 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb490052050 con 0x55dff8bc2580 2024-04-22T13:34:06.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.862+0000 7fb4aafe1740 10 client.19083 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:06.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.862+0000 7fb4aafe1740 1 -- 172.21.15.12:0/322291434 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dff87f90f0 con 0x55dff8bc2580 2024-04-22T13:34:06.873 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.862+0000 7fb489ffb640 1 -- 172.21.15.12:0/322291434 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb49c0054d0 con 0x55dff8bc2580 2024-04-22T13:34:06.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.863+0000 7fb489ffb640 1 -- 172.21.15.12:0/322291434 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb4900035f0 con 0x55dff8bc2580 2024-04-22T13:34:06.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.863+0000 7fb4aafe1740 10 client.19083 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:06.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.863+0000 7fb4aafe1740 10 client.19083 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:34:06.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.863+0000 7fb4aafe1740 20 client.19083 populate_metadata read hostname 'smithi012' 2024-04-22T13:34:06.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.863+0000 7fb4aafe1740 1 --2- 172.21.15.12:0/322291434 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55dff8bdbb10 0x55dff8bddf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.863+0000 7fb4aafe1740 4 client.19083 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:34:06.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.863+0000 7fb4aafe1740 1 -- 172.21.15.12:0/322291434 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dff8b34c00 con 0x55dff8bdbb10 2024-04-22T13:34:06.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.863+0000 7fb4a60b1640 1 --2- 172.21.15.12:0/322291434 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55dff8bdbb10 0x55dff8bddf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.863+0000 7fb4a60b1640 1 --2- 172.21.15.12:0/322291434 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55dff8bdbb10 0x55dff8bddf00 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=711c9086d038984b server_cookie=fa6bfdee262c4573 in_seq=0 out_seq=0 2024-04-22T13:34:06.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.863+0000 7fb489ffb640 10 client.19083 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:34:06.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.864+0000 7fb489ffb640 1 -- 172.21.15.12:0/322291434 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dff8b34c00 con 0x55dff8bdbb10 2024-04-22T13:34:06.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.864+0000 7fb489ffb640 10 client.19083 handle_command_reply: tid=0 2024-04-22T13:34:06.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.864+0000 7fb4aafe1740 1 client.19083 shutdown 2024-04-22T13:34:06.874 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.864+0000 7fb4aafe1740 1 -- 172.21.15.12:0/322291434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dff8bc2580 msgr2=0x55dff8bc8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:06.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.864+0000 7fb4aafe1740 1 --2- 172.21.15.12:0/322291434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dff8bc2580 0x55dff8bc8ad0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fb490035000 tx=0x7fb490005850 comp rx=0 tx=0).stop 2024-04-22T13:34:06.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.864+0000 7fb4aafe1740 1 -- 172.21.15.12:0/322291434 shutdown_connections 2024-04-22T13:34:06.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.864+0000 7fb4aafe1740 1 --2- 172.21.15.12:0/322291434 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55dff8bdbb10 0x55dff8bddf00 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.864+0000 7fb4aafe1740 1 --2- 172.21.15.12:0/322291434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dff8bcba80 0x55dff8bc9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.864+0000 7fb489ffb640 0 client.19083 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:34:06.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.864+0000 7fb4aafe1740 1 --2- 172.21.15.12:0/322291434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff8bc2ea0 0x55dff8bc9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.864+0000 7fb4aafe1740 1 --2- 172.21.15.12:0/322291434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dff8bc2580 0x55dff8bc8ad0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.864+0000 7fb4aafe1740 1 -- 172.21.15.12:0/322291434 >> 172.21.15.12:0/322291434 conn(0x55dff8bbccb0 msgr2=0x55dff8bbea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:06.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.864+0000 7fb4aafe1740 1 -- 172.21.15.12:0/322291434 shutdown_connections 2024-04-22T13:34:06.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.864+0000 7fb4aafe1740 1 -- 172.21.15.12:0/322291434 wait complete. 2024-04-22T13:34:06.875 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.864+0000 7fb4aafe1740 20 client.19083 trim_cache size 0 max 16384 2024-04-22T13:34:06.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.882+0000 7fb4aafe1740 1 Processor -- start 2024-04-22T13:34:06.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.882+0000 7fb4aafe1740 1 -- start start 2024-04-22T13:34:06.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.882+0000 7fb4aafe1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dff8bc2580 0x55dff8bcb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.882+0000 7fb4aafe1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dff8bc2ea0 0x55dff8bd7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.882+0000 7fb4aafe1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff8bcba80 0x55dff8bc91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.882+0000 7fb4aafe1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01afd00 con 0x55dff8bc2ea0 2024-04-22T13:34:06.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.882+0000 7fb4aafe1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01b1840 con 0x55dff8bc2580 2024-04-22T13:34:06.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.882+0000 7fb4aafe1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01b03d0 con 0x55dff8bcba80 2024-04-22T13:34:06.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.882+0000 7fb4a60b1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dff8bc2ea0 0x55dff8bd7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.882+0000 7fb4a60b1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dff8bc2ea0 0x55dff8bd7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36926/0 (socket says 172.21.15.12:36926) 2024-04-22T13:34:06.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.882+0000 7fb4a60b1640 1 -- 172.21.15.12:0/680878258 learned_addr learned my addr 172.21.15.12:0/680878258 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:06.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.882+0000 7fb4a60b1640 1 -- 172.21.15.12:0/680878258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff8bcba80 msgr2=0x55dff8bc91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:06.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.882+0000 7fb4a58b0640 1 --2- 172.21.15.12:0/680878258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff8bcba80 0x55dff8bc91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.882+0000 7fb4a60b1640 1 --2- 172.21.15.12:0/680878258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff8bcba80 0x55dff8bc91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.882+0000 7fb4a60b1640 1 -- 172.21.15.12:0/680878258 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dff8bc2580 msgr2=0x55dff8bcb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:06.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.882+0000 7fb4a60b1640 1 --2- 172.21.15.12:0/680878258 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dff8bc2580 0x55dff8bcb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.882+0000 7fb4a60b1640 1 -- 172.21.15.12:0/680878258 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff8bc2cd0 con 0x55dff8bc2ea0 2024-04-22T13:34:06.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.883+0000 7fb4a60b1640 1 --2- 172.21.15.12:0/680878258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dff8bc2ea0 0x55dff8bd7760 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x55dff8bc2050 tx=0x7fb49c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376350b4d502af6f server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:06.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.883+0000 7fb48bfff640 1 -- 172.21.15.12:0/680878258 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c02d070 con 0x55dff8bc2ea0 2024-04-22T13:34:06.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.883+0000 7fb48bfff640 1 -- 172.21.15.12:0/680878258 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb49c0285c0 con 0x55dff8bc2ea0 2024-04-22T13:34:06.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.883+0000 7fb48bfff640 1 -- 172.21.15.12:0/680878258 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c015040 con 0x55dff8bc2ea0 2024-04-22T13:34:06.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.883+0000 7fb4aafe1740 1 -- 172.21.15.12:0/680878258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dff8bc2ea0 msgr2=0x55dff8bd7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:06.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.883+0000 7fb4aafe1740 1 --2- 172.21.15.12:0/680878258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dff8bc2ea0 0x55dff8bd7760 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x55dff8bc2050 tx=0x7fb49c00f730 comp rx=0 tx=0).stop 2024-04-22T13:34:06.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.883+0000 7fb4aafe1740 1 -- 172.21.15.12:0/680878258 shutdown_connections 2024-04-22T13:34:06.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.883+0000 7fb4aafe1740 1 --2- 172.21.15.12:0/680878258 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff8bcba80 0x55dff8bc91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.883+0000 7fb4aafe1740 1 --2- 172.21.15.12:0/680878258 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dff8bc2ea0 0x55dff8bd7760 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.883+0000 7fb4aafe1740 1 --2- 172.21.15.12:0/680878258 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dff8bc2580 0x55dff8bcb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.883+0000 7fb4aafe1740 1 -- 172.21.15.12:0/680878258 >> 172.21.15.12:0/680878258 conn(0x55dff8bbccb0 msgr2=0x55dff8bc8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:06.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.883+0000 7fb4aafe1740 1 -- 172.21.15.12:0/680878258 shutdown_connections 2024-04-22T13:34:06.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.883+0000 7fb4aafe1740 1 -- 172.21.15.12:0/680878258 wait complete. 2024-04-22T13:34:06.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.884+0000 7fb4aafe1740 1 -- start start 2024-04-22T13:34:06.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.884+0000 7fb4aafe1740 1 Processor -- start 2024-04-22T13:34:06.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.884+0000 7fb4aafe1740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dff8bc2580 0x55dff8bcf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.884+0000 7fb4aafe1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff8bc2ea0 0x55dff8bcfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.884+0000 7fb4aafe1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dff8bcba80 0x55dff8bd0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.884+0000 7fb4aafe1740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01b1170 con 0x55dff8bc2580 2024-04-22T13:34:06.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.884+0000 7fb4aafe1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01b0aa0 con 0x55dff8bcba80 2024-04-22T13:34:06.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.884+0000 7fb4aafe1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a01add00 con 0x55dff8bc2ea0 2024-04-22T13:34:06.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.884+0000 7fb4aafe1740 10 client.0 _ll_register_callbacks cb 0x7fb490004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:06.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.884+0000 7fb4a58b0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dff8bc2580 0x55dff8bcf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.884+0000 7fb4a58b0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dff8bc2580 0x55dff8bcf5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:36934/0 (socket says 172.21.15.12:36934) 2024-04-22T13:34:06.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.884+0000 7fb4a58b0640 1 -- 172.21.15.12:0/3674849320 learned_addr learned my addr 172.21.15.12:0/3674849320 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:06.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.884+0000 7fb4a60b1640 1 --2- 172.21.15.12:0/3674849320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff8bc2ea0 0x55dff8bcfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.884+0000 7fb4a58b0640 1 --2- 172.21.15.12:0/3674849320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dff8bcba80 0x55dff8bd0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.884+0000 7fb4a58b0640 1 -- 172.21.15.12:0/3674849320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff8bc2ea0 msgr2=0x55dff8bcfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:06.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.884+0000 7fb4a58b0640 1 --2- 172.21.15.12:0/3674849320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff8bc2ea0 0x55dff8bcfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.884+0000 7fb4a58b0640 1 -- 172.21.15.12:0/3674849320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dff8bcba80 msgr2=0x55dff8bd0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:06.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.884+0000 7fb4a58b0640 1 --2- 172.21.15.12:0/3674849320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dff8bcba80 0x55dff8bd0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.884+0000 7fb4a58b0640 1 -- 172.21.15.12:0/3674849320 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dff8c41010 con 0x55dff8bc2580 2024-04-22T13:34:06.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.884+0000 7fb4a60b1640 1 --2- 172.21.15.12:0/3674849320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff8bc2ea0 0x55dff8bcfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:06.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.885+0000 7fb4a58b0640 1 --2- 172.21.15.12:0/3674849320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dff8bc2580 0x55dff8bcf5d0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fb490050040 tx=0x7fb490005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944354134f7eabc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:06.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.885+0000 7fb489ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:34:06.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.885+0000 7fb489ffb640 1 -- 172.21.15.12:0/3674849320 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb490047040 con 0x55dff8bc2580 2024-04-22T13:34:06.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.885+0000 7fb489ffb640 1 -- 172.21.15.12:0/3674849320 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8484+0+0 (secure 0 0 0) 0x7fb490005040 con 0x55dff8bc2580 2024-04-22T13:34:06.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.885+0000 7fb4aafe1740 1 -- 172.21.15.12:0/3674849320 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb49c0285c0 con 0x55dff8bc2580 2024-04-22T13:34:06.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.885+0000 7fb489ffb640 1 -- 172.21.15.12:0/3674849320 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb490005240 con 0x55dff8bc2580 2024-04-22T13:34:06.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.885+0000 7fb4aafe1740 10 client.19089 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:06.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.885+0000 7fb4aafe1740 1 -- 172.21.15.12:0/3674849320 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dff8c42260 con 0x55dff8bc2580 2024-04-22T13:34:06.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.885+0000 7fb489ffb640 1 -- 172.21.15.12:0/3674849320 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb49c0285c0 con 0x55dff8bc2580 2024-04-22T13:34:06.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.885+0000 7fb489ffb640 1 -- 172.21.15.12:0/3674849320 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb49005f030 con 0x55dff8bc2580 2024-04-22T13:34:06.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.885+0000 7fb4aafe1740 10 client.19089 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:06.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.885+0000 7fb4aafe1740 10 client.19089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:34:06.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.885+0000 7fb4aafe1740 20 client.19089 populate_metadata read hostname 'smithi012' 2024-04-22T13:34:06.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.885+0000 7fb4aafe1740 1 --2- 172.21.15.12:0/3674849320 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55dff8c429e0 0x55dff8c44e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:06.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.885+0000 7fb4aafe1740 4 client.19089 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:34:06.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.885+0000 7fb4aafe1740 1 -- 172.21.15.12:0/3674849320 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dff8b2ea70 con 0x55dff8c429e0 2024-04-22T13:34:06.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.885+0000 7fb4a60b1640 1 --2- 172.21.15.12:0/3674849320 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55dff8c429e0 0x55dff8c44e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:06.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.886+0000 7fb489ffb640 10 client.19089 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:34:06.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.886+0000 7fb4a60b1640 1 --2- 172.21.15.12:0/3674849320 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55dff8c429e0 0x55dff8c44e00 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95da99722db95e8 server_cookie=9ace8ad670c24fd7 in_seq=0 out_seq=0 2024-04-22T13:34:06.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.886+0000 7fb489ffb640 1 -- 172.21.15.12:0/3674849320 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55dff8b2ea70 con 0x55dff8c429e0 2024-04-22T13:34:06.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.886+0000 7fb489ffb640 10 client.19089 handle_command_reply: tid=0 2024-04-22T13:34:06.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.886+0000 7fb4aafe1740 1 client.19089 shutdown 2024-04-22T13:34:06.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.886+0000 7fb4aafe1740 1 -- 172.21.15.12:0/3674849320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dff8bc2580 msgr2=0x55dff8bcf5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:06.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.886+0000 7fb4aafe1740 1 --2- 172.21.15.12:0/3674849320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dff8bc2580 0x55dff8bcf5d0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fb490050040 tx=0x7fb490005a20 comp rx=0 tx=0).stop 2024-04-22T13:34:06.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.886+0000 7fb4aafe1740 1 -- 172.21.15.12:0/3674849320 shutdown_connections 2024-04-22T13:34:06.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.886+0000 7fb4aafe1740 1 --2- 172.21.15.12:0/3674849320 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x55dff8c429e0 0x55dff8c44e00 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.886+0000 7fb4aafe1740 1 --2- 172.21.15.12:0/3674849320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55dff8bcba80 0x55dff8bd0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.886+0000 7fb489ffb640 0 client.19089 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:34:06.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.886+0000 7fb4aafe1740 1 --2- 172.21.15.12:0/3674849320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dff8bc2ea0 0x55dff8bcfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.886+0000 7fb4aafe1740 1 --2- 172.21.15.12:0/3674849320 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55dff8bc2580 0x55dff8bcf5d0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.886+0000 7fb4aafe1740 1 -- 172.21.15.12:0/3674849320 >> 172.21.15.12:0/3674849320 conn(0x55dff8bbccb0 msgr2=0x55dff8bbe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:06.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.886+0000 7fb4aafe1740 1 -- 172.21.15.12:0/3674849320 shutdown_connections 2024-04-22T13:34:06.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.886+0000 7fb4aafe1740 1 -- 172.21.15.12:0/3674849320 wait complete. 2024-04-22T13:34:06.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.886+0000 7fb4aafe1740 20 client.19089 trim_cache size 0 max 16384 2024-04-22T13:34:06.890 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:34:06.891 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-22T13:34:06.891 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": { 2024-04-22T13:34:06.891 INFO:teuthology.orchestra.run.smithi012.stdout: "c2f84b8c-e51d-410e-8d94-26a97ead20e9": { 2024-04-22T13:34:06.891 INFO:teuthology.orchestra.run.smithi012.stdout: "path": "/", 2024-04-22T13:34:06.891 INFO:teuthology.orchestra.run.smithi012.stdout: "tag": "c2f84b8c-e51d-410e-8d94-26a97ead20e9", 2024-04-22T13:34:06.891 INFO:teuthology.orchestra.run.smithi012.stdout: "options": "recursive,force" 2024-04-22T13:34:06.891 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:06.891 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:06.891 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:34:06.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.887+0000 7fb4a8315640 1 -- 172.21.15.12:0/244293014 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb48c0a3d50 msgr2=0x7fb48c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:06.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.887+0000 7fb4a8315640 1 --2- 172.21.15.12:0/244293014 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb48c0a3d50 0x7fb48c0a6210 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb49800efd0 tx=0x7fb49801eb40 comp rx=0 tx=0).stop 2024-04-22T13:34:06.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.887+0000 7fb4a8315640 1 -- 172.21.15.12:0/244293014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4a0106f00 msgr2=0x7fb4a0101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:06.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.887+0000 7fb4a8315640 1 --2- 172.21.15.12:0/244293014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4a0106f00 0x7fb4a0101bf0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fb49800eaa0 tx=0x7fb49800ef60 comp rx=0 tx=0).stop 2024-04-22T13:34:06.892 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.887+0000 7fb4a8315640 1 -- 172.21.15.12:0/244293014 shutdown_connections 2024-04-22T13:34:06.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.887+0000 7fb4a8315640 1 --2- 172.21.15.12:0/244293014 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb48c0a3d50 0x7fb48c0a6210 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.887+0000 7fb4a8315640 1 --2- 172.21.15.12:0/244293014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb4a0106f00 0x7fb4a0101bf0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.887+0000 7fb4a8315640 1 --2- 172.21.15.12:0/244293014 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4a006afb0 0x7fb4a01016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.887+0000 7fb4a8315640 1 --2- 172.21.15.12:0/244293014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4a006a5f0 0x7fb4a0101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:06.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.887+0000 7fb4a8315640 1 -- 172.21.15.12:0/244293014 >> 172.21.15.12:0/244293014 conn(0x7fb4a0077c40 msgr2=0x7fb4a00fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:06.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.887+0000 7fb4a8315640 1 -- 172.21.15.12:0/244293014 shutdown_connections 2024-04-22T13:34:06.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:06.887+0000 7fb4a8315640 1 -- 172.21.15.12:0/244293014 wait complete. 2024-04-22T13:34:06.894 INFO:tasks.cephfs.filesystem:scrub status for tag:c2f84b8c-e51d-410e-8d94-26a97ead20e9 - {'path': '/', 'tag': 'c2f84b8c-e51d-410e-8d94-26a97ead20e9', 'options': 'recursive,force'} 2024-04-22T13:34:07.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:07 smithi111 ceph-mon[31361]: pgmap v309: 97 pgs: 97 active+clean; 2.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:34:07.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:07 smithi111 ceph-mon[31361]: scrub queued for path: / 2024-04-22T13:34:07.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:07 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:34:07.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:07 smithi111 ceph-mon[31361]: scrub summary: active paths [/] 2024-04-22T13:34:07.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:07 smithi111 ceph-mon[31361]: scrub summary: idle+waiting paths [/] 2024-04-22T13:34:08.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:07 smithi012 ceph-mon[28265]: pgmap v309: 97 pgs: 97 active+clean; 2.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:34:08.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:07 smithi012 ceph-mon[28265]: scrub queued for path: / 2024-04-22T13:34:08.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:07 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:34:08.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:07 smithi012 ceph-mon[28265]: scrub summary: active paths [/] 2024-04-22T13:34:08.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:07 smithi012 ceph-mon[28265]: scrub summary: idle+waiting paths [/] 2024-04-22T13:34:08.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:07 smithi134 ceph-mon[31338]: pgmap v309: 97 pgs: 97 active+clean; 2.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:34:08.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:07 smithi134 ceph-mon[31338]: scrub queued for path: / 2024-04-22T13:34:08.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:07 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:34:08.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:07 smithi134 ceph-mon[31338]: scrub summary: active paths [/] 2024-04-22T13:34:08.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:07 smithi134 ceph-mon[31338]: scrub summary: idle+waiting paths [/] 2024-04-22T13:34:09.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:09 smithi111 ceph-mon[31361]: pgmap v310: 97 pgs: 97 active+clean; 2.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-22T13:34:10.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:09 smithi012 ceph-mon[28265]: pgmap v310: 97 pgs: 97 active+clean; 2.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-22T13:34:10.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:09 smithi134 ceph-mon[31338]: pgmap v310: 97 pgs: 97 active+clean; 2.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-22T13:34:10.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:10 smithi111 ceph-mon[31361]: scrub summary: idle 2024-04-22T13:34:11.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:10 smithi012 ceph-mon[28265]: scrub summary: idle 2024-04-22T13:34:11.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:10 smithi134 ceph-mon[31338]: scrub summary: idle 2024-04-22T13:34:11.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:11 smithi111 ceph-mon[31361]: pgmap v311: 97 pgs: 97 active+clean; 2.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-22T13:34:11.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:11 smithi111 ceph-mon[31361]: osdmap e97: 12 total, 12 up, 12 in 2024-04-22T13:34:12.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:11 smithi012 ceph-mon[28265]: pgmap v311: 97 pgs: 97 active+clean; 2.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-22T13:34:12.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:11 smithi012 ceph-mon[28265]: osdmap e97: 12 total, 12 up, 12 in 2024-04-22T13:34:12.176 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:11 smithi134 ceph-mon[31338]: pgmap v311: 97 pgs: 97 active+clean; 2.3 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-22T13:34:12.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:11 smithi134 ceph-mon[31338]: osdmap e97: 12 total, 12 up, 12 in 2024-04-22T13:34:13.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:13 smithi111 ceph-mon[31361]: pgmap v313: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-22T13:34:14.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:13 smithi012 ceph-mon[28265]: pgmap v313: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-22T13:34:14.177 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:13 smithi134 ceph-mon[31338]: pgmap v313: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-22T13:34:14.766 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-22T13:34:14.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.886+0000 7f2c265c7640 1 -- 172.21.15.12:0/3944696818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c2010e7f0 msgr2=0x7f2c20110c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:14.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.886+0000 7f2c265c7640 1 --2- 172.21.15.12:0/3944696818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c2010e7f0 0x7f2c20110c00 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f2c18009ca0 tx=0x7f2c1802c190 comp rx=0 tx=0).stop 2024-04-22T13:34:14.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.886+0000 7f2c265c7640 1 -- 172.21.15.12:0/3944696818 shutdown_connections 2024-04-22T13:34:14.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.886+0000 7f2c265c7640 1 --2- 172.21.15.12:0/3944696818 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c2010e7f0 0x7f2c20110c00 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:14.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.886+0000 7f2c265c7640 1 --2- 172.21.15.12:0/3944696818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c20074c40 0x7f2c2010e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:14.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.886+0000 7f2c265c7640 1 --2- 172.21.15.12:0/3944696818 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c20074320 0x7f2c20074700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:14.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.886+0000 7f2c265c7640 1 -- 172.21.15.12:0/3944696818 >> 172.21.15.12:0/3944696818 conn(0x7f2c200fda40 msgr2=0x7f2c200ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:14.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.886+0000 7f2c265c7640 1 -- 172.21.15.12:0/3944696818 shutdown_connections 2024-04-22T13:34:14.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.886+0000 7f2c265c7640 1 -- 172.21.15.12:0/3944696818 wait complete. 2024-04-22T13:34:14.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.886+0000 7f2c265c7640 1 Processor -- start 2024-04-22T13:34:14.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.886+0000 7f2c265c7640 1 -- start start 2024-04-22T13:34:14.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.887+0000 7f2c265c7640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c20074320 0x7f2c201a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:14.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.887+0000 7f2c265c7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c20074c40 0x7f2c201a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:14.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.887+0000 7f2c265c7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c2010e7f0 0x7f2c2019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:14.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.887+0000 7f2c265c7640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f2c201143c0 con 0x7f2c20074c40 2024-04-22T13:34:14.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.887+0000 7f2c265c7640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2c20114240 con 0x7f2c2010e7f0 2024-04-22T13:34:14.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.887+0000 7f2c265c7640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2c20114540 con 0x7f2c20074320 2024-04-22T13:34:14.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.887+0000 7f2c1f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c20074c40 0x7f2c201a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:14.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.887+0000 7f2c1f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c20074c40 0x7f2c201a0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55490/0 (socket says 172.21.15.12:55490) 2024-04-22T13:34:14.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.887+0000 7f2c1f7fe640 1 -- 172.21.15.12:0/3945732966 learned_addr learned my addr 172.21.15.12:0/3945732966 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:14.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.887+0000 7f2c1ffff640 1 --2- 172.21.15.12:0/3945732966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c20074320 0x7f2c201a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:14.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.887+0000 7f2c24b64640 1 --2- 172.21.15.12:0/3945732966 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c2010e7f0 0x7f2c2019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:14.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.887+0000 7f2c1f7fe640 1 -- 172.21.15.12:0/3945732966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c20074320 msgr2=0x7f2c201a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:14.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.887+0000 7f2c1f7fe640 1 --2- 172.21.15.12:0/3945732966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c20074320 0x7f2c201a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:14.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.887+0000 7f2c1f7fe640 1 -- 172.21.15.12:0/3945732966 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c2010e7f0 msgr2=0x7f2c2019a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:14.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.887+0000 7f2c1f7fe640 1 --2- 172.21.15.12:0/3945732966 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c2010e7f0 0x7f2c2019a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:14.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.887+0000 7f2c1f7fe640 1 -- 172.21.15.12:0/3945732966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c2019afa0 con 0x7f2c20074c40 2024-04-22T13:34:14.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.888+0000 7f2c1f7fe640 1 --2- 172.21.15.12:0/3945732966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c20074c40 0x7f2c201a0b10 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f2c1400c920 tx=0x7f2c1400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:14.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.888+0000 7f2c1d7fa640 1 -- 172.21.15.12:0/3945732966 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c14007c70 con 0x7f2c20074c40 2024-04-22T13:34:14.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.888+0000 7f2c1d7fa640 1 -- 172.21.15.12:0/3945732966 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c14007e10 con 0x7f2c20074c40 2024-04-22T13:34:14.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.888+0000 7f2c265c7640 1 -- 172.21.15.12:0/3945732966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c2019b290 con 0x7f2c20074c40 2024-04-22T13:34:14.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.888+0000 7f2c1d7fa640 1 -- 172.21.15.12:0/3945732966 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c14018820 con 0x7f2c20074c40 2024-04-22T13:34:14.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.888+0000 7f2c265c7640 1 -- 172.21.15.12:0/3945732966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c201a7550 con 0x7f2c20074c40 2024-04-22T13:34:14.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.889+0000 7f2c265c7640 1 -- 172.21.15.12:0/3945732966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c200759d0 con 0x7f2c20074c40 2024-04-22T13:34:14.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.891+0000 7f2c1d7fa640 1 -- 172.21.15.12:0/3945732966 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2c1400f450 con 0x7f2c20074c40 2024-04-22T13:34:14.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.892+0000 7f2c1d7fa640 1 --2- 172.21.15.12:0/3945732966 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2c080a3dc0 0x7f2c080a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:14.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.892+0000 7f2c1ffff640 1 --2- 172.21.15.12:0/3945732966 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2c080a3dc0 0x7f2c080a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:14.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.892+0000 7f2c1d7fa640 1 -- 172.21.15.12:0/3945732966 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f2c140d0890 con 0x7f2c20074c40 2024-04-22T13:34:14.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.893+0000 7f2c1ffff640 1 --2- 172.21.15.12:0/3945732966 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2c080a3dc0 0x7f2c080a6280 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2c10005d00 tx=0x7f2c1000b410 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:14.893 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:14.897+0000 7f2c1d7fa640 1 -- 172.21.15.12:0/3945732966 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2c14014030 con 0x7f2c20074c40 2024-04-22T13:34:15.104 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.108+0000 7f2c265c7640 1 -- 172.21.15.12:0/3945732966 --> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f2c20062390 con 0x7f2c080a3dc0 2024-04-22T13:34:15.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.116+0000 7f2c1d7fa640 1 -- 172.21.15.12:0/3945732966 <== mgr.24968 v2:172.21.15.12:6800/1615661596 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f2c20062390 con 0x7f2c080a3dc0 2024-04-22T13:34:15.113 INFO:teuthology.orchestra.run.smithi012.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-22T00:00:00", "created": "2024-04-22T13:24:13", "first": "2024-04-22T13:25:00", "last": "2024-04-22T13:34:00", "last_pruned": "2024-04-22T13:34:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-22T13:34:15.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.118+0000 7f2c265c7640 1 -- 172.21.15.12:0/3945732966 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2c080a3dc0 msgr2=0x7f2c080a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:15.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.118+0000 7f2c265c7640 1 --2- 172.21.15.12:0/3945732966 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2c080a3dc0 0x7f2c080a6280 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2c10005d00 tx=0x7f2c1000b410 comp rx=0 tx=0).stop 2024-04-22T13:34:15.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.118+0000 7f2c265c7640 1 -- 172.21.15.12:0/3945732966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c20074c40 msgr2=0x7f2c201a0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:15.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.118+0000 7f2c265c7640 1 --2- 172.21.15.12:0/3945732966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c20074c40 0x7f2c201a0b10 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f2c1400c920 tx=0x7f2c1400cde0 comp rx=0 tx=0).stop 2024-04-22T13:34:15.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.118+0000 7f2c265c7640 1 -- 172.21.15.12:0/3945732966 shutdown_connections 2024-04-22T13:34:15.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.118+0000 7f2c265c7640 1 --2- 172.21.15.12:0/3945732966 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f2c080a3dc0 0x7f2c080a6280 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:15.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.118+0000 7f2c265c7640 1 --2- 172.21.15.12:0/3945732966 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c2010e7f0 0x7f2c2019a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:15.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.118+0000 7f2c265c7640 1 --2- 172.21.15.12:0/3945732966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2c20074c40 0x7f2c201a0b10 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:15.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.118+0000 7f2c265c7640 1 --2- 172.21.15.12:0/3945732966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c20074320 0x7f2c201a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:15.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.118+0000 7f2c265c7640 1 -- 172.21.15.12:0/3945732966 >> 172.21.15.12:0/3945732966 conn(0x7f2c200fda40 msgr2=0x7f2c200fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:15.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.118+0000 7f2c265c7640 1 -- 172.21.15.12:0/3945732966 shutdown_connections 2024-04-22T13:34:15.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.118+0000 7f2c265c7640 1 -- 172.21.15.12:0/3945732966 wait complete. 2024-04-22T13:34:15.125 DEBUG:teuthology.orchestra.run.smithi012:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-22T13:34:15.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.232+0000 7ff54784e640 1 -- 172.21.15.12:0/503261687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff540103200 msgr2=0x7ff5401035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:15.231 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.232+0000 7ff54784e640 1 --2- 172.21.15.12:0/503261687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff540103200 0x7ff5401035e0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009900 tx=0x7ff53c02bcd0 comp rx=0 tx=0).stop 2024-04-22T13:34:15.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.232+0000 7ff54784e640 1 -- 172.21.15.12:0/503261687 shutdown_connections 2024-04-22T13:34:15.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.232+0000 7ff54784e640 1 --2- 172.21.15.12:0/503261687 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff54010c5d0 0x7ff54010ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:15.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.232+0000 7ff54784e640 1 --2- 172.21.15.12:0/503261687 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff540103b20 0x7ff54010c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:15.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.232+0000 7ff54784e640 1 --2- 172.21.15.12:0/503261687 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff540103200 0x7ff5401035e0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:15.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.232+0000 7ff54784e640 1 -- 172.21.15.12:0/503261687 >> 172.21.15.12:0/503261687 conn(0x7ff5400771d0 msgr2=0x7ff5400775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:15.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.232+0000 7ff54784e640 1 -- 172.21.15.12:0/503261687 shutdown_connections 2024-04-22T13:34:15.232 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.232+0000 7ff54784e640 1 -- 172.21.15.12:0/503261687 wait complete. 2024-04-22T13:34:15.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.233+0000 7ff54784e640 1 Processor -- start 2024-04-22T13:34:15.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.233+0000 7ff54784e640 1 -- start start 2024-04-22T13:34:15.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.233+0000 7ff54784e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff540103200 0x7ff5401a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:15.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.233+0000 7ff54784e640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff540103b20 0x7ff5401a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:15.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.233+0000 7ff54784e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff54010c5d0 0x7ff54019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:15.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.233+0000 7ff54784e640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff540112270 con 0x7ff54010c5d0 2024-04-22T13:34:15.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.233+0000 7ff54784e640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff5401120f0 con 0x7ff540103b20 2024-04-22T13:34:15.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.233+0000 7ff54784e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff5401123f0 con 0x7ff540103200 2024-04-22T13:34:15.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.233+0000 7ff545deb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff54010c5d0 0x7ff54019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:15.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.233+0000 7ff545deb640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff54010c5d0 0x7ff54019a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55498/0 (socket says 172.21.15.12:55498) 2024-04-22T13:34:15.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.233+0000 7ff545deb640 1 -- 172.21.15.12:0/759356809 learned_addr learned my addr 172.21.15.12:0/759356809 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:15.233 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.234+0000 7ff544de9640 1 --2- 172.21.15.12:0/759356809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff540103b20 0x7ff5401a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:15.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.234+0000 7ff5455ea640 1 --2- 172.21.15.12:0/759356809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff540103200 0x7ff5401a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:15.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.234+0000 7ff545deb640 1 -- 172.21.15.12:0/759356809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff540103200 msgr2=0x7ff5401a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:15.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.234+0000 7ff545deb640 1 --2- 172.21.15.12:0/759356809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff540103200 0x7ff5401a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:15.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.234+0000 7ff545deb640 1 -- 172.21.15.12:0/759356809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff540103b20 msgr2=0x7ff5401a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:15.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.234+0000 7ff545deb640 1 --2- 172.21.15.12:0/759356809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff540103b20 0x7ff5401a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:15.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.234+0000 7ff545deb640 1 -- 172.21.15.12:0/759356809 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff54019af60 con 0x7ff54010c5d0 2024-04-22T13:34:15.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.234+0000 7ff545deb640 1 --2- 172.21.15.12:0/759356809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff54010c5d0 0x7ff54019a790 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7ff53800b9e0 tx=0x7ff53800bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:15.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.234+0000 7ff5367fc640 1 -- 172.21.15.12:0/759356809 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff538019070 con 0x7ff54010c5d0 2024-04-22T13:34:15.234 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.234+0000 7ff54784e640 1 -- 172.21.15.12:0/759356809 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff54019b250 con 0x7ff54010c5d0 2024-04-22T13:34:15.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.234+0000 7ff5367fc640 1 -- 172.21.15.12:0/759356809 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff538014440 con 0x7ff54010c5d0 2024-04-22T13:34:15.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.234+0000 7ff5367fc640 1 -- 172.21.15.12:0/759356809 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff538012580 con 0x7ff54010c5d0 2024-04-22T13:34:15.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.234+0000 7ff54784e640 1 -- 172.21.15.12:0/759356809 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5401a7470 con 0x7ff54010c5d0 2024-04-22T13:34:15.235 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.235+0000 7ff54784e640 1 -- 172.21.15.12:0/759356809 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff54006afa0 con 0x7ff54010c5d0 2024-04-22T13:34:15.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.236+0000 7ff5367fc640 1 -- 172.21.15.12:0/759356809 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff53801c070 con 0x7ff54010c5d0 2024-04-22T13:34:15.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.237+0000 7ff5367fc640 1 --2- 172.21.15.12:0/759356809 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff52c0a3dc0 0x7ff52c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:15.236 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.237+0000 7ff5455ea640 1 --2- 172.21.15.12:0/759356809 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff52c0a3dc0 0x7ff52c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:15.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.237+0000 7ff5367fc640 1 -- 172.21.15.12:0/759356809 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7ff5380cffd0 con 0x7ff54010c5d0 2024-04-22T13:34:15.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.238+0000 7ff5455ea640 1 --2- 172.21.15.12:0/759356809 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff52c0a3dc0 0x7ff52c0a6280 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009230 tx=0x7ff53c005fd0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:15.237 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.241+0000 7ff5367fc640 1 -- 172.21.15.12:0/759356809 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff53809c150 con 0x7ff54010c5d0 2024-04-22T13:34:15.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.449+0000 7ff54784e640 1 -- 172.21.15.12:0/759356809 --> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7ff5401025f0 con 0x7ff52c0a3dc0 2024-04-22T13:34:15.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.456+0000 7ff5367fc640 1 -- 172.21.15.12:0/759356809 <== mgr.24968 v2:172.21.15.12:6800/1615661596 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7ff5401025f0 con 0x7ff52c0a3dc0 2024-04-22T13:34:15.452 INFO:teuthology.orchestra.run.smithi012.stdout:/ 1m 6m3h 2024-04-22T13:34:15.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.458+0000 7ff54784e640 1 -- 172.21.15.12:0/759356809 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff52c0a3dc0 msgr2=0x7ff52c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:15.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.458+0000 7ff54784e640 1 --2- 172.21.15.12:0/759356809 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff52c0a3dc0 0x7ff52c0a6280 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009230 tx=0x7ff53c005fd0 comp rx=0 tx=0).stop 2024-04-22T13:34:15.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.458+0000 7ff54784e640 1 -- 172.21.15.12:0/759356809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff54010c5d0 msgr2=0x7ff54019a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:15.454 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.458+0000 7ff54784e640 1 --2- 172.21.15.12:0/759356809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff54010c5d0 0x7ff54019a790 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7ff53800b9e0 tx=0x7ff53800bde0 comp rx=0 tx=0).stop 2024-04-22T13:34:15.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.458+0000 7ff54784e640 1 -- 172.21.15.12:0/759356809 shutdown_connections 2024-04-22T13:34:15.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.458+0000 7ff54784e640 1 --2- 172.21.15.12:0/759356809 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff52c0a3dc0 0x7ff52c0a6280 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:15.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.458+0000 7ff54784e640 1 --2- 172.21.15.12:0/759356809 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff54010c5d0 0x7ff54019a790 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:15.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.458+0000 7ff54784e640 1 --2- 172.21.15.12:0/759356809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff540103b20 0x7ff5401a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:15.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.458+0000 7ff54784e640 1 --2- 172.21.15.12:0/759356809 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff540103200 0x7ff5401a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:15.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.458+0000 7ff54784e640 1 -- 172.21.15.12:0/759356809 >> 172.21.15.12:0/759356809 conn(0x7ff5400771d0 msgr2=0x7ff540101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:15.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.458+0000 7ff54784e640 1 -- 172.21.15.12:0/759356809 shutdown_connections 2024-04-22T13:34:15.455 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:15.458+0000 7ff54784e640 1 -- 172.21.15.12:0/759356809 wait complete. 2024-04-22T13:34:15.465 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-22T13:34:15.479 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-22T13:34:15.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:15 smithi012 ceph-mon[28265]: pgmap v314: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-22T13:34:15.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:15 smithi012 ceph-mon[28265]: osdmap e98: 12 total, 12 up, 12 in 2024-04-22T13:34:15.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:15 smithi134 ceph-mon[31338]: pgmap v314: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-22T13:34:15.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:15 smithi134 ceph-mon[31338]: osdmap e98: 12 total, 12 up, 12 in 2024-04-22T13:34:15.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:15 smithi111 ceph-mon[31361]: pgmap v314: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-04-22T13:34:15.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:15 smithi111 ceph-mon[31361]: osdmap e98: 12 total, 12 up, 12 in 2024-04-22T13:34:16.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:16 smithi012 ceph-mon[28265]: from='client.19095 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:34:16.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:16 smithi012 ceph-mon[28265]: from='client.19101 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:34:16.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:16 smithi134 ceph-mon[31338]: from='client.19095 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:34:16.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:16 smithi134 ceph-mon[31338]: from='client.19101 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:34:16.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:16 smithi111 ceph-mon[31361]: from='client.19095 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:34:16.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:16 smithi111 ceph-mon[31361]: from='client.19101 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-22T13:34:17.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:17 smithi012 ceph-mon[28265]: pgmap v316: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-22T13:34:17.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:17 smithi134 ceph-mon[31338]: pgmap v316: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-22T13:34:17.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:17 smithi111 ceph-mon[31361]: pgmap v316: 97 pgs: 97 active+clean; 2.3 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-22T13:34:19.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:19 smithi012 ceph-mon[28265]: pgmap v317: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 255 B/s wr, 16 op/s 2024-04-22T13:34:19.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:19 smithi134 ceph-mon[31338]: pgmap v317: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 255 B/s wr, 16 op/s 2024-04-22T13:34:19.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:19 smithi111 ceph-mon[31361]: pgmap v317: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 255 B/s wr, 16 op/s 2024-04-22T13:34:20.809 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:20 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:34:20.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:20 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:34:20.848 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:20 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-22T13:34:21.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:21 smithi012 ceph-mon[28265]: pgmap v318: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 212 B/s wr, 13 op/s 2024-04-22T13:34:21.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:21 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:34:21.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:21 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:34:21.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:21 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:34:21.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:21 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:34:21.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:21 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:34:21.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:21 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:34:21.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:21 smithi134 ceph-mon[31338]: pgmap v318: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 212 B/s wr, 13 op/s 2024-04-22T13:34:21.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:21 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:34:21.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:21 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:34:21.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:21 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:34:21.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:21 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:34:21.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:21 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:34:21.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:21 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:34:21.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:21 smithi111 ceph-mon[31361]: pgmap v318: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 212 B/s wr, 13 op/s 2024-04-22T13:34:21.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:21 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:34:21.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:21 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:34:21.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:21 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:34:21.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:21 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:34:21.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:21 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:34:21.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:21 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:34:22.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:22 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:22 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:22 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:22 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:22 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:22 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:22 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:22 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:22 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:22 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:22 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:22 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:22 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:34:22.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:22 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:34:22.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:22 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:34:22.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:22 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:22 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:22 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:22 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:22 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:22 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:22 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:22 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:22 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:22 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:22 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:22 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:22 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:34:22.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:22 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:34:22.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:22 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:34:22.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:22 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:22 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:22 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:22 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:22 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:22 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:22 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:22 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:22 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:22 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:22 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:22 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-22T13:34:22.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:22 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-22T13:34:22.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:22 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-22T13:34:22.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:22 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' 2024-04-22T13:34:23.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:23 smithi012 ceph-mon[28265]: Adjusting osd_memory_target on smithi134 to 176.2M 2024-04-22T13:34:23.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:23 smithi012 ceph-mon[28265]: Unable to set osd_memory_target on smithi134 to 184792473: error parsing value: Value '184792473' is below minimum 939524096 2024-04-22T13:34:23.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:23 smithi012 ceph-mon[28265]: Adjusting osd_memory_target on smithi111 to 176.2M 2024-04-22T13:34:23.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:23 smithi012 ceph-mon[28265]: Unable to set osd_memory_target on smithi111 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-22T13:34:23.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:23 smithi012 ceph-mon[28265]: Adjusting osd_memory_target on smithi012 to 177.2M 2024-04-22T13:34:23.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:23 smithi012 ceph-mon[28265]: Unable to set osd_memory_target on smithi012 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-04-22T13:34:23.837 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:23 smithi012 ceph-mon[28265]: pgmap v319: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-22T13:34:23.877 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:23 smithi111 ceph-mon[31361]: Adjusting osd_memory_target on smithi134 to 176.2M 2024-04-22T13:34:23.877 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:23 smithi111 ceph-mon[31361]: Unable to set osd_memory_target on smithi134 to 184792473: error parsing value: Value '184792473' is below minimum 939524096 2024-04-22T13:34:23.878 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:23 smithi111 ceph-mon[31361]: Adjusting osd_memory_target on smithi111 to 176.2M 2024-04-22T13:34:23.878 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:23 smithi111 ceph-mon[31361]: Unable to set osd_memory_target on smithi111 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-22T13:34:23.878 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:23 smithi111 ceph-mon[31361]: Adjusting osd_memory_target on smithi012 to 177.2M 2024-04-22T13:34:23.878 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:23 smithi111 ceph-mon[31361]: Unable to set osd_memory_target on smithi012 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-04-22T13:34:23.878 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:23 smithi111 ceph-mon[31361]: pgmap v319: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-22T13:34:23.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:23 smithi134 ceph-mon[31338]: Adjusting osd_memory_target on smithi134 to 176.2M 2024-04-22T13:34:23.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:23 smithi134 ceph-mon[31338]: Unable to set osd_memory_target on smithi134 to 184792473: error parsing value: Value '184792473' is below minimum 939524096 2024-04-22T13:34:23.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:23 smithi134 ceph-mon[31338]: Adjusting osd_memory_target on smithi111 to 176.2M 2024-04-22T13:34:23.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:23 smithi134 ceph-mon[31338]: Unable to set osd_memory_target on smithi111 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-22T13:34:23.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:23 smithi134 ceph-mon[31338]: Adjusting osd_memory_target on smithi012 to 177.2M 2024-04-22T13:34:23.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:23 smithi134 ceph-mon[31338]: Unable to set osd_memory_target on smithi012 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-04-22T13:34:23.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:23 smithi134 ceph-mon[31338]: pgmap v319: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-22T13:34:25.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:25 smithi012 ceph-mon[28265]: pgmap v320: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-22T13:34:25.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:25 smithi134 ceph-mon[31338]: pgmap v320: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-22T13:34:25.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:25 smithi111 ceph-mon[31361]: pgmap v320: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 102 B/s wr, 13 op/s 2024-04-22T13:34:27.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:27 smithi012 ceph-mon[28265]: pgmap v321: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 86 B/s wr, 11 op/s 2024-04-22T13:34:27.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:27 smithi134 ceph-mon[31338]: pgmap v321: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 86 B/s wr, 11 op/s 2024-04-22T13:34:27.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:27 smithi111 ceph-mon[31361]: pgmap v321: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 86 B/s wr, 11 op/s 2024-04-22T13:34:29.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:29 smithi012 ceph-mon[28265]: pgmap v322: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 85 B/s wr, 15 op/s 2024-04-22T13:34:29.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:29 smithi134 ceph-mon[31338]: pgmap v322: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 85 B/s wr, 15 op/s 2024-04-22T13:34:29.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:29 smithi111 ceph-mon[31361]: pgmap v322: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 85 B/s wr, 15 op/s 2024-04-22T13:34:31.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:31 smithi012 ceph-mon[28265]: pgmap v323: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:31.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:31 smithi134 ceph-mon[31338]: pgmap v323: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:31.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:31 smithi111 ceph-mon[31361]: pgmap v323: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:33.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:33 smithi012 ceph-mon[28265]: pgmap v324: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:34:33.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:33 smithi134 ceph-mon[31338]: pgmap v324: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:34:33.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:33 smithi111 ceph-mon[31361]: pgmap v324: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:34:35.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:35 smithi012 ceph-mon[28265]: pgmap v325: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:35.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:35 smithi134 ceph-mon[31338]: pgmap v325: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:35.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:35 smithi111 ceph-mon[31361]: pgmap v325: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:36.895 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-22T13:34:36.895 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-22T13:34:37.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.021+0000 7f9285367640 1 -- 172.21.15.12:0/572094311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9280101a40 msgr2=0x7f928010dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.021+0000 7f9285367640 1 --2- 172.21.15.12:0/572094311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9280101a40 0x7f928010dca0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f9278009ca0 tx=0x7f927802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.021+0000 7f9285367640 1 -- 172.21.15.12:0/572094311 shutdown_connections 2024-04-22T13:34:37.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.021+0000 7f9285367640 1 --2- 172.21.15.12:0/572094311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9280101a40 0x7f928010dca0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.021+0000 7f9285367640 1 --2- 172.21.15.12:0/572094311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92801010a0 0x7f9280101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.021+0000 7f9285367640 1 --2- 172.21.15.12:0/572094311 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9280105dd0 0x7f92801061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.021+0000 7f9285367640 1 -- 172.21.15.12:0/572094311 >> 172.21.15.12:0/572094311 conn(0x7f9280077c40 msgr2=0x7f9280078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.021+0000 7f9285367640 1 -- 172.21.15.12:0/572094311 shutdown_connections 2024-04-22T13:34:37.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.021+0000 7f9285367640 1 -- 172.21.15.12:0/572094311 wait complete. 2024-04-22T13:34:37.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.021+0000 7f9285367640 1 Processor -- start 2024-04-22T13:34:37.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.021+0000 7f9285367640 1 -- start start 2024-04-22T13:34:37.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.022+0000 7f9285367640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92801010a0 0x7f9280198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.022+0000 7f9285367640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9280101a40 0x7f9280198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.022+0000 7f9285367640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9280105dd0 0x7f928019cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.022+0000 7f9285367640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f9280111160 con 0x7f92801010a0 2024-04-22T13:34:37.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.022+0000 7f9285367640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9280110fe0 con 0x7f9280105dd0 2024-04-22T13:34:37.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.022+0000 7f9285367640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f92801112e0 con 0x7f9280101a40 2024-04-22T13:34:37.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.022+0000 7f927effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92801010a0 0x7f9280198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.022+0000 7f927effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92801010a0 0x7f9280198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49770/0 (socket says 172.21.15.12:49770) 2024-04-22T13:34:37.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.022+0000 7f927effd640 1 -- 172.21.15.12:0/1275211495 learned_addr learned my addr 172.21.15.12:0/1275211495 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:37.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.022+0000 7f9277fff640 1 --2- 172.21.15.12:0/1275211495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9280101a40 0x7f9280198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.022+0000 7f927f7fe640 1 --2- 172.21.15.12:0/1275211495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9280105dd0 0x7f928019cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.022+0000 7f927effd640 1 -- 172.21.15.12:0/1275211495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9280101a40 msgr2=0x7f9280198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.022+0000 7f927effd640 1 --2- 172.21.15.12:0/1275211495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9280101a40 0x7f9280198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.022+0000 7f927effd640 1 -- 172.21.15.12:0/1275211495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9280105dd0 msgr2=0x7f928019cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.022+0000 7f927effd640 1 --2- 172.21.15.12:0/1275211495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9280105dd0 0x7f928019cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.022+0000 7f927effd640 1 -- 172.21.15.12:0/1275211495 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f928019d700 con 0x7f92801010a0 2024-04-22T13:34:37.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.022+0000 7f9277fff640 1 --2- 172.21.15.12:0/1275211495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9280101a40 0x7f9280198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:37.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.022+0000 7f927f7fe640 1 --2- 172.21.15.12:0/1275211495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9280105dd0 0x7f928019cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:37.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.022+0000 7f927effd640 1 --2- 172.21.15.12:0/1275211495 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92801010a0 0x7f9280198710 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f927000d950 tx=0x7f927000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.020 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.022+0000 7f927cff9640 1 -- 172.21.15.12:0/1275211495 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9270017030 con 0x7f92801010a0 2024-04-22T13:34:37.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.022+0000 7f927cff9640 1 -- 172.21.15.12:0/1275211495 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9270004020 con 0x7f92801010a0 2024-04-22T13:34:37.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.022+0000 7f927cff9640 1 -- 172.21.15.12:0/1275211495 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92700047a0 con 0x7f92801010a0 2024-04-22T13:34:37.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.023+0000 7f9285367640 1 -- 172.21.15.12:0/1275211495 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f928019d9f0 con 0x7f92801010a0 2024-04-22T13:34:37.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.023+0000 7f9285367640 1 -- 172.21.15.12:0/1275211495 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92801a5280 con 0x7f92801010a0 2024-04-22T13:34:37.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.025+0000 7f927cff9640 1 -- 172.21.15.12:0/1275211495 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9270009a30 con 0x7f92801010a0 2024-04-22T13:34:37.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.025+0000 7f9288038740 1 Processor -- start 2024-04-22T13:34:37.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.025+0000 7f9288038740 1 -- start start 2024-04-22T13:34:37.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.026+0000 7f9288038740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e15ddf580 0x560e15ddf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.026+0000 7f9288038740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e15ddfea0 0x560e15de8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.026+0000 7f9288038740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e15de8a80 0x560e15deae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.026+0000 7f9288038740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560e15d441e0 con 0x560e15ddfea0 2024-04-22T13:34:37.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.026+0000 7f9288038740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560e15ce1390 con 0x560e15ddf580 2024-04-22T13:34:37.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.026+0000 7f9288038740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560e15d401a0 con 0x560e15de8a80 2024-04-22T13:34:37.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.026+0000 7f927cff9640 1 --2- 172.21.15.12:0/1275211495 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f926c0a3d20 0x7f926c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.026+0000 7f927effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e15ddfea0 0x560e15de8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.026+0000 7f927effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e15ddfea0 0x560e15de8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49784/0 (socket says 172.21.15.12:49784) 2024-04-22T13:34:37.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.026+0000 7f927effd640 1 -- 172.21.15.12:0/3756328228 learned_addr learned my addr 172.21.15.12:0/3756328228 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:37.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.026+0000 7f927f7fe640 1 --2- 172.21.15.12:0/3756328228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e15ddf580 0x560e15ddf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.026+0000 7f927f7fe640 1 --2- 172.21.15.12:0/1275211495 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f926c0a3d20 0x7f926c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.026+0000 7f927cff9640 1 -- 172.21.15.12:0/1275211495 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f92700d02d0 con 0x7f92801010a0 2024-04-22T13:34:37.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.026+0000 7f927effd640 1 -- 172.21.15.12:0/3756328228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e15de8a80 msgr2=0x560e15deae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.026+0000 7f927effd640 1 --2- 172.21.15.12:0/3756328228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e15de8a80 0x560e15deae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.026+0000 7f927effd640 1 -- 172.21.15.12:0/3756328228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e15ddf580 msgr2=0x560e15ddf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.026+0000 7f927effd640 1 --2- 172.21.15.12:0/3756328228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e15ddf580 0x560e15ddf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.026+0000 7f927effd640 1 -- 172.21.15.12:0/3756328228 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e15b207d0 con 0x560e15ddfea0 2024-04-22T13:34:37.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.026+0000 7f927effd640 1 --2- 172.21.15.12:0/3756328228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e15ddfea0 0x560e15de8540 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f92700cdaa0 tx=0x7f9270021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b02ae10cdfea0c server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.026+0000 7f927f7fe640 1 --2- 172.21.15.12:0/1275211495 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f926c0a3d20 0x7f926c0a61e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9278033000 tx=0x7f92780095c0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.027+0000 7f92757fa640 1 -- 172.21.15.12:0/3756328228 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927009c290 con 0x560e15ddfea0 2024-04-22T13:34:37.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.027+0000 7f92757fa640 1 -- 172.21.15.12:0/3756328228 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f927009cc20 con 0x560e15ddfea0 2024-04-22T13:34:37.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.027+0000 7f92757fa640 1 -- 172.21.15.12:0/3756328228 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927009d260 con 0x560e15ddfea0 2024-04-22T13:34:37.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.027+0000 7f9288038740 1 -- 172.21.15.12:0/3756328228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e15ddfea0 msgr2=0x560e15de8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.027+0000 7f9288038740 1 --2- 172.21.15.12:0/3756328228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e15ddfea0 0x560e15de8540 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f92700cdaa0 tx=0x7f9270021000 comp rx=0 tx=0).stop 2024-04-22T13:34:37.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.027+0000 7f9288038740 1 -- 172.21.15.12:0/3756328228 shutdown_connections 2024-04-22T13:34:37.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.027+0000 7f9288038740 1 --2- 172.21.15.12:0/3756328228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e15de8a80 0x560e15deae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.027+0000 7f9288038740 1 --2- 172.21.15.12:0/3756328228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e15ddfea0 0x560e15de8540 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.028+0000 7f9288038740 1 --2- 172.21.15.12:0/3756328228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e15ddf580 0x560e15ddf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.028+0000 7f9288038740 1 -- 172.21.15.12:0/3756328228 >> 172.21.15.12:0/3756328228 conn(0x560e15dd9cb0 msgr2=0x560e15dda090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.028+0000 7f9288038740 1 -- 172.21.15.12:0/3756328228 shutdown_connections 2024-04-22T13:34:37.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.028+0000 7f9288038740 1 -- 172.21.15.12:0/3756328228 wait complete. 2024-04-22T13:34:37.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.028+0000 7f9288038740 1 -- start start 2024-04-22T13:34:37.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.028+0000 7f9288038740 1 Processor -- start 2024-04-22T13:34:37.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.028+0000 7f9288038740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e15ddf580 0x560e15de5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.028+0000 7f9288038740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e15ddfea0 0x560e15de6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.028+0000 7f9288038740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e15de8a80 0x560e15de6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.028+0000 7f9288038740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x560e15deb7b0 con 0x560e15de8a80 2024-04-22T13:34:37.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.028+0000 7f9288038740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560e15cfd2f0 con 0x560e15ddf580 2024-04-22T13:34:37.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.028+0000 7f9288038740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560e15deb930 con 0x560e15ddfea0 2024-04-22T13:34:37.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.028+0000 7f9288038740 10 client.0 _ll_register_callbacks cb 0x560e15d0a580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:37.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.028+0000 7f927f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e15de8a80 0x560e15de6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.028+0000 7f927f7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e15de8a80 0x560e15de6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49788/0 (socket says 172.21.15.12:49788) 2024-04-22T13:34:37.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.028+0000 7f927f7fe640 1 -- 172.21.15.12:0/1404794479 learned_addr learned my addr 172.21.15.12:0/1404794479 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:37.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.028+0000 7f927effd640 1 --2- 172.21.15.12:0/1404794479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e15ddf580 0x560e15de5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.029+0000 7f9277fff640 1 --2- 172.21.15.12:0/1404794479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e15ddfea0 0x560e15de6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.029+0000 7f927f7fe640 1 -- 172.21.15.12:0/1404794479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e15ddfea0 msgr2=0x560e15de6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.029+0000 7f927f7fe640 1 --2- 172.21.15.12:0/1404794479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e15ddfea0 0x560e15de6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.029+0000 7f927f7fe640 1 -- 172.21.15.12:0/1404794479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e15ddf580 msgr2=0x560e15de5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.029+0000 7f927f7fe640 1 --2- 172.21.15.12:0/1404794479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e15ddf580 0x560e15de5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.029+0000 7f927f7fe640 1 -- 172.21.15.12:0/1404794479 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e15a563f0 con 0x560e15de8a80 2024-04-22T13:34:37.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.029+0000 7f927f7fe640 1 --2- 172.21.15.12:0/1404794479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e15de8a80 0x560e15de6580 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f9278036000 tx=0x7f9278009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263530f84373515 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.029+0000 7f92667fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:34:37.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.029+0000 7f92667fc640 1 -- 172.21.15.12:0/1404794479 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9278053060 con 0x560e15de8a80 2024-04-22T13:34:37.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.029+0000 7f92667fc640 1 -- 172.21.15.12:0/1404794479 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f9278004450 con 0x560e15de8a80 2024-04-22T13:34:37.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.029+0000 7f9288038740 1 -- 172.21.15.12:0/1404794479 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560e15d51c00 con 0x560e15de8a80 2024-04-22T13:34:37.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.029+0000 7f92667fc640 1 -- 172.21.15.12:0/1404794479 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9278039070 con 0x560e15de8a80 2024-04-22T13:34:37.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.029+0000 7f9288038740 10 client.19113 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:37.033 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.029+0000 7f9288038740 1 -- 172.21.15.12:0/1404794479 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560e15a160f0 con 0x560e15de8a80 2024-04-22T13:34:37.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.029+0000 7f92667fc640 1 -- 172.21.15.12:0/1404794479 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560e15d51c00 con 0x560e15de8a80 2024-04-22T13:34:37.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.030+0000 7f92667fc640 1 -- 172.21.15.12:0/1404794479 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f927804d070 con 0x560e15de8a80 2024-04-22T13:34:37.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.030+0000 7f9288038740 10 client.19113 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:37.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.030+0000 7f9288038740 10 client.19113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:34:37.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.030+0000 7f9288038740 20 client.19113 populate_metadata read hostname 'smithi012' 2024-04-22T13:34:37.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.030+0000 7f9288038740 1 --2- 172.21.15.12:0/1404794479 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560e15df8a80 0x560e15dfaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.030+0000 7f9288038740 4 client.19113 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:34:37.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.030+0000 7f9288038740 1 -- 172.21.15.12:0/1404794479 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e15d4ba70 con 0x560e15df8a80 2024-04-22T13:34:37.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.030+0000 7f927effd640 1 --2- 172.21.15.12:0/1404794479 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560e15df8a80 0x560e15dfaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.030+0000 7f927effd640 1 --2- 172.21.15.12:0/1404794479 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560e15df8a80 0x560e15dfaee0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5aa6373dc2521581 server_cookie=752d3741825a3ddf in_seq=0 out_seq=0 2024-04-22T13:34:37.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.030+0000 7f92667fc640 10 client.19113 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:34:37.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.031+0000 7f92667fc640 1 -- 172.21.15.12:0/1404794479 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560e15d4ba70 con 0x560e15df8a80 2024-04-22T13:34:37.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.031+0000 7f92667fc640 10 client.19113 handle_command_reply: tid=0 2024-04-22T13:34:37.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.031+0000 7f9288038740 1 client.19113 shutdown 2024-04-22T13:34:37.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.031+0000 7f9288038740 1 -- 172.21.15.12:0/1404794479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e15de8a80 msgr2=0x560e15de6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.035 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.031+0000 7f9288038740 1 --2- 172.21.15.12:0/1404794479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e15de8a80 0x560e15de6580 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f9278036000 tx=0x7f9278009380 comp rx=0 tx=0).stop 2024-04-22T13:34:37.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.031+0000 7f9288038740 1 -- 172.21.15.12:0/1404794479 shutdown_connections 2024-04-22T13:34:37.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.031+0000 7f9288038740 1 --2- 172.21.15.12:0/1404794479 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560e15df8a80 0x560e15dfaee0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.031+0000 7f9288038740 1 --2- 172.21.15.12:0/1404794479 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e15de8a80 0x560e15de6580 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.031+0000 7f9288038740 1 --2- 172.21.15.12:0/1404794479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e15ddfea0 0x560e15de6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.031+0000 7f92667fc640 0 client.19113 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:34:37.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.031+0000 7f9288038740 1 --2- 172.21.15.12:0/1404794479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e15ddf580 0x560e15de5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.031+0000 7f9288038740 1 -- 172.21.15.12:0/1404794479 >> 172.21.15.12:0/1404794479 conn(0x560e15dd9cb0 msgr2=0x560e15ddba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.031+0000 7f9288038740 1 -- 172.21.15.12:0/1404794479 shutdown_connections 2024-04-22T13:34:37.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.031+0000 7f9288038740 1 -- 172.21.15.12:0/1404794479 wait complete. 2024-04-22T13:34:37.036 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.032+0000 7f9288038740 20 client.19113 trim_cache size 0 max 16384 2024-04-22T13:34:37.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.049+0000 7f9288038740 1 Processor -- start 2024-04-22T13:34:37.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.049+0000 7f9288038740 1 -- start start 2024-04-22T13:34:37.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.049+0000 7f9288038740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e15ddf580 0x560e15de8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.049+0000 7f9288038740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e15ddfea0 0x560e15df46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.050+0000 7f9288038740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e15de8a80 0x560e15dd8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.051 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.050+0000 7f9288038740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f92801adb80 con 0x560e15ddf580 2024-04-22T13:34:37.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.050+0000 7f9288038740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f92801af6c0 con 0x560e15ddfea0 2024-04-22T13:34:37.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.050+0000 7f9288038740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f92801ae250 con 0x560e15de8a80 2024-04-22T13:34:37.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.050+0000 7f927effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e15ddf580 0x560e15de8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.050+0000 7f927effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e15ddf580 0x560e15de8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49802/0 (socket says 172.21.15.12:49802) 2024-04-22T13:34:37.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.050+0000 7f927effd640 1 -- 172.21.15.12:0/2361651447 learned_addr learned my addr 172.21.15.12:0/2361651447 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:37.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.050+0000 7f927f7fe640 1 --2- 172.21.15.12:0/2361651447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e15de8a80 0x560e15dd8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.050+0000 7f9277fff640 1 --2- 172.21.15.12:0/2361651447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e15ddfea0 0x560e15df46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.050+0000 7f927effd640 1 -- 172.21.15.12:0/2361651447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e15de8a80 msgr2=0x560e15dd8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.050+0000 7f927effd640 1 --2- 172.21.15.12:0/2361651447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e15de8a80 0x560e15dd8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.050+0000 7f927effd640 1 -- 172.21.15.12:0/2361651447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e15ddfea0 msgr2=0x560e15df46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.050+0000 7f927effd640 1 --2- 172.21.15.12:0/2361651447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e15ddfea0 0x560e15df46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.050+0000 7f927effd640 1 -- 172.21.15.12:0/2361651447 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e15ddfcd0 con 0x560e15ddf580 2024-04-22T13:34:37.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.050+0000 7f9277fff640 1 --2- 172.21.15.12:0/2361651447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e15ddfea0 0x560e15df46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:37.052 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.050+0000 7f927effd640 1 --2- 172.21.15.12:0/2361651447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e15ddf580 0x560e15de8230 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f927009bd60 tx=0x7f9270000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a5f3108f5a6b85 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.050+0000 7f9274ff9640 1 -- 172.21.15.12:0/2361651447 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92700b5070 con 0x560e15ddf580 2024-04-22T13:34:37.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.050+0000 7f9274ff9640 1 -- 172.21.15.12:0/2361651447 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92700a0640 con 0x560e15ddf580 2024-04-22T13:34:37.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.050+0000 7f9274ff9640 1 -- 172.21.15.12:0/2361651447 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9270098900 con 0x560e15ddf580 2024-04-22T13:34:37.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.051+0000 7f9288038740 1 -- 172.21.15.12:0/2361651447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e15ddf580 msgr2=0x560e15de8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.051+0000 7f9288038740 1 --2- 172.21.15.12:0/2361651447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e15ddf580 0x560e15de8230 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f927009bd60 tx=0x7f9270000bc0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.051+0000 7f9288038740 1 -- 172.21.15.12:0/2361651447 shutdown_connections 2024-04-22T13:34:37.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.051+0000 7f9288038740 1 --2- 172.21.15.12:0/2361651447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e15de8a80 0x560e15dd8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.051+0000 7f9288038740 1 --2- 172.21.15.12:0/2361651447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e15ddfea0 0x560e15df46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.051+0000 7f9288038740 1 --2- 172.21.15.12:0/2361651447 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e15ddf580 0x560e15de8230 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.051+0000 7f9288038740 1 -- 172.21.15.12:0/2361651447 >> 172.21.15.12:0/2361651447 conn(0x560e15dd9cb0 msgr2=0x560e15de4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.051+0000 7f9288038740 1 -- 172.21.15.12:0/2361651447 shutdown_connections 2024-04-22T13:34:37.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.051+0000 7f9288038740 1 -- 172.21.15.12:0/2361651447 wait complete. 2024-04-22T13:34:37.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.051+0000 7f9288038740 1 -- start start 2024-04-22T13:34:37.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.052+0000 7f9288038740 1 Processor -- start 2024-04-22T13:34:37.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.052+0000 7f9288038740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e15ddf580 0x560e15df1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.053 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.052+0000 7f9288038740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e15ddfea0 0x560e15de2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.052+0000 7f9288038740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e15de8a80 0x560e15de2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.052+0000 7f9288038740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f92801aeff0 con 0x560e15ddf580 2024-04-22T13:34:37.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.052+0000 7f9288038740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f92801ae920 con 0x560e15de8a80 2024-04-22T13:34:37.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.052+0000 7f9288038740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f92801abb80 con 0x560e15ddfea0 2024-04-22T13:34:37.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.052+0000 7f9288038740 10 client.0 _ll_register_callbacks cb 0x7f9278009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:37.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.052+0000 7f927effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e15ddf580 0x560e15df1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.052+0000 7f927effd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e15ddf580 0x560e15df1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49808/0 (socket says 172.21.15.12:49808) 2024-04-22T13:34:37.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.052+0000 7f927effd640 1 -- 172.21.15.12:0/2649058922 learned_addr learned my addr 172.21.15.12:0/2649058922 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:37.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.052+0000 7f9277fff640 1 --2- 172.21.15.12:0/2649058922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e15ddfea0 0x560e15de2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.052+0000 7f927f7fe640 1 --2- 172.21.15.12:0/2649058922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e15de8a80 0x560e15de2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.052+0000 7f927effd640 1 -- 172.21.15.12:0/2649058922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e15ddfea0 msgr2=0x560e15de2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.054 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.052+0000 7f927effd640 1 --2- 172.21.15.12:0/2649058922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e15ddfea0 0x560e15de2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.052+0000 7f927effd640 1 -- 172.21.15.12:0/2649058922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e15de8a80 msgr2=0x560e15de2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.052+0000 7f927effd640 1 --2- 172.21.15.12:0/2649058922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e15de8a80 0x560e15de2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.052+0000 7f927effd640 1 -- 172.21.15.12:0/2649058922 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e15de32f0 con 0x560e15ddf580 2024-04-22T13:34:37.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.052+0000 7f9277fff640 1 --2- 172.21.15.12:0/2649058922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e15ddfea0 0x560e15de2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:37.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.052+0000 7f927f7fe640 1 --2- 172.21.15.12:0/2649058922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e15de8a80 0x560e15de2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:37.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.052+0000 7f927effd640 1 --2- 172.21.15.12:0/2649058922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e15ddf580 0x560e15df1070 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f92700a49d0 tx=0x7f927009c290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a22d02cbe994b99 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.052+0000 7f92667fc640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:34:37.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.052+0000 7f92667fc640 1 -- 172.21.15.12:0/2649058922 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92700b5040 con 0x560e15ddf580 2024-04-22T13:34:37.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.053+0000 7f92667fc640 1 -- 172.21.15.12:0/2649058922 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f92700cdcc0 con 0x560e15ddf580 2024-04-22T13:34:37.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.053+0000 7f9288038740 1 -- 172.21.15.12:0/2649058922 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560e15cc7590 con 0x560e15ddf580 2024-04-22T13:34:37.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.053+0000 7f92667fc640 1 -- 172.21.15.12:0/2649058922 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f927009a590 con 0x560e15ddf580 2024-04-22T13:34:37.055 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.053+0000 7f9288038740 10 client.19119 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:37.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.053+0000 7f9288038740 1 -- 172.21.15.12:0/2649058922 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560e15e5f010 con 0x560e15ddf580 2024-04-22T13:34:37.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.053+0000 7f92667fc640 1 -- 172.21.15.12:0/2649058922 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560e15cc7590 con 0x560e15ddf580 2024-04-22T13:34:37.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.053+0000 7f92667fc640 1 -- 172.21.15.12:0/2649058922 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f927009c4f0 con 0x560e15ddf580 2024-04-22T13:34:37.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.053+0000 7f9288038740 10 client.19119 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:37.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.053+0000 7f9288038740 10 client.19119 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:34:37.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.053+0000 7f9288038740 20 client.19119 populate_metadata read hostname 'smithi012' 2024-04-22T13:34:37.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.053+0000 7f9288038740 1 --2- 172.21.15.12:0/2649058922 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560e15e5f240 0x560e15e61630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.053+0000 7f9288038740 4 client.19119 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-22T13:34:37.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.053+0000 7f9288038740 1 -- 172.21.15.12:0/2649058922 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560e15d14a00 con 0x560e15e5f240 2024-04-22T13:34:37.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.053+0000 7f9277fff640 1 --2- 172.21.15.12:0/2649058922 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560e15e5f240 0x560e15e61630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.054+0000 7f9277fff640 1 --2- 172.21.15.12:0/2649058922 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560e15e5f240 0x560e15e61630 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=340945afc98073 server_cookie=4e9b96f5573a9ab2 in_seq=0 out_seq=0 2024-04-22T13:34:37.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.054+0000 7f92667fc640 10 client.19119 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:34:37.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.054+0000 7f92667fc640 1 -- 172.21.15.12:0/2649058922 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560e15d14a00 con 0x560e15e5f240 2024-04-22T13:34:37.056 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.054+0000 7f92667fc640 10 client.19119 handle_command_reply: tid=0 2024-04-22T13:34:37.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.054+0000 7f9288038740 1 client.19119 shutdown 2024-04-22T13:34:37.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.054+0000 7f9288038740 1 -- 172.21.15.12:0/2649058922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e15ddf580 msgr2=0x560e15df1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.054+0000 7f9288038740 1 --2- 172.21.15.12:0/2649058922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e15ddf580 0x560e15df1070 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f92700a49d0 tx=0x7f927009c290 comp rx=0 tx=0).stop 2024-04-22T13:34:37.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.054+0000 7f9288038740 1 -- 172.21.15.12:0/2649058922 shutdown_connections 2024-04-22T13:34:37.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.054+0000 7f9288038740 1 --2- 172.21.15.12:0/2649058922 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x560e15e5f240 0x560e15e61630 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.054+0000 7f9288038740 1 --2- 172.21.15.12:0/2649058922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560e15de8a80 0x560e15de2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.054+0000 7f9288038740 1 --2- 172.21.15.12:0/2649058922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560e15ddfea0 0x560e15de2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.054+0000 7f9288038740 1 --2- 172.21.15.12:0/2649058922 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560e15ddf580 0x560e15df1070 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.054+0000 7f9288038740 1 -- 172.21.15.12:0/2649058922 >> 172.21.15.12:0/2649058922 conn(0x560e15dd9cb0 msgr2=0x560e15de6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.054+0000 7f92667fc640 0 client.19119 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:34:37.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.054+0000 7f9288038740 1 -- 172.21.15.12:0/2649058922 shutdown_connections 2024-04-22T13:34:37.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.054+0000 7f9288038740 1 -- 172.21.15.12:0/2649058922 wait complete. 2024-04-22T13:34:37.057 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.054+0000 7f9288038740 20 client.19119 trim_cache size 0 max 16384 2024-04-22T13:34:37.058 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:34:37.058 INFO:teuthology.orchestra.run.smithi012.stdout: "status": "no active scrubs running", 2024-04-22T13:34:37.058 INFO:teuthology.orchestra.run.smithi012.stdout: "scrubs": {} 2024-04-22T13:34:37.058 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:34:37.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.055+0000 7f9285367640 1 -- 172.21.15.12:0/1275211495 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f926c0a3d20 msgr2=0x7f926c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.055+0000 7f9285367640 1 --2- 172.21.15.12:0/1275211495 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f926c0a3d20 0x7f926c0a61e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9278033000 tx=0x7f92780095c0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.055+0000 7f9285367640 1 -- 172.21.15.12:0/1275211495 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92801010a0 msgr2=0x7f9280198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.055+0000 7f9285367640 1 --2- 172.21.15.12:0/1275211495 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92801010a0 0x7f9280198710 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f927000d950 tx=0x7f927000de10 comp rx=0 tx=0).stop 2024-04-22T13:34:37.059 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.055+0000 7f9285367640 1 -- 172.21.15.12:0/1275211495 shutdown_connections 2024-04-22T13:34:37.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.055+0000 7f9285367640 1 --2- 172.21.15.12:0/1275211495 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f926c0a3d20 0x7f926c0a61e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.055+0000 7f9285367640 1 --2- 172.21.15.12:0/1275211495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9280105dd0 0x7f928019cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.055+0000 7f9285367640 1 --2- 172.21.15.12:0/1275211495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9280101a40 0x7f9280198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.055+0000 7f9285367640 1 --2- 172.21.15.12:0/1275211495 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f92801010a0 0x7f9280198710 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.055+0000 7f9285367640 1 -- 172.21.15.12:0/1275211495 >> 172.21.15.12:0/1275211495 conn(0x7f9280077c40 msgr2=0x7f9280104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.055+0000 7f9285367640 1 -- 172.21.15.12:0/1275211495 shutdown_connections 2024-04-22T13:34:37.060 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.055+0000 7f9285367640 1 -- 172.21.15.12:0/1275211495 wait complete. 2024-04-22T13:34:37.061 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-22T13:34:37.061 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:34:37.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.168+0000 7fb3d0504640 1 -- 172.21.15.12:0/1423077352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3c8100360 msgr2=0x7fb3c8100740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.166 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.168+0000 7fb3d0504640 1 --2- 172.21.15.12:0/1423077352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3c8100360 0x7fb3c8100740 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc009900 tx=0x7fb3bc02bc30 comp rx=0 tx=0).stop 2024-04-22T13:34:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.168+0000 7fb3d0504640 1 -- 172.21.15.12:0/1423077352 shutdown_connections 2024-04-22T13:34:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.168+0000 7fb3d0504640 1 --2- 172.21.15.12:0/1423077352 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3c8108430 0x7fb3c810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.168+0000 7fb3d0504640 1 --2- 172.21.15.12:0/1423077352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3c8100d10 0x7fb3c8107d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.168+0000 7fb3d0504640 1 --2- 172.21.15.12:0/1423077352 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3c8100360 0x7fb3c8100740 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.168+0000 7fb3d0504640 1 -- 172.21.15.12:0/1423077352 >> 172.21.15.12:0/1423077352 conn(0x7fb3c80770b0 msgr2=0x7fb3c80fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.168+0000 7fb3d0504640 1 -- 172.21.15.12:0/1423077352 shutdown_connections 2024-04-22T13:34:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.168+0000 7fb3d0504640 1 -- 172.21.15.12:0/1423077352 wait complete. 2024-04-22T13:34:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.169+0000 7fb3d0504640 1 Processor -- start 2024-04-22T13:34:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.169+0000 7fb3d0504640 1 -- start start 2024-04-22T13:34:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.169+0000 7fb3d0504640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3c8100360 0x7fb3c81986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.169+0000 7fb3d0504640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3c8108430 0x7fb3c8198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.167 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.169+0000 7fb3d0504640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3c819cfb0 0x7fb3c819d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.169+0000 7fb3d0504640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c8112170 con 0x7fb3c8100360 2024-04-22T13:34:37.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.169+0000 7fb3d0504640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c8111ff0 con 0x7fb3c8108430 2024-04-22T13:34:37.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.169+0000 7fb3d0504640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c81122f0 con 0x7fb3c819cfb0 2024-04-22T13:34:37.168 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.169+0000 7fb3ce2a0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3c8100360 0x7fb3c81986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.169+0000 7fb3ce2a0640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3c8100360 0x7fb3c81986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49828/0 (socket says 172.21.15.12:49828) 2024-04-22T13:34:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.169+0000 7fb3ce2a0640 1 -- 172.21.15.12:0/1095992961 learned_addr learned my addr 172.21.15.12:0/1095992961 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.169+0000 7fb3ceaa1640 1 --2- 172.21.15.12:0/1095992961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3c819cfb0 0x7fb3c819d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.169+0000 7fb3cda9f640 1 --2- 172.21.15.12:0/1095992961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3c8108430 0x7fb3c8198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.169+0000 7fb3ce2a0640 1 -- 172.21.15.12:0/1095992961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3c819cfb0 msgr2=0x7fb3c819d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.169 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.169+0000 7fb3ce2a0640 1 --2- 172.21.15.12:0/1095992961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3c819cfb0 0x7fb3c819d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.169+0000 7fb3ce2a0640 1 -- 172.21.15.12:0/1095992961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3c8108430 msgr2=0x7fb3c8198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.169+0000 7fb3ce2a0640 1 --2- 172.21.15.12:0/1095992961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3c8108430 0x7fb3c8198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.169+0000 7fb3ce2a0640 1 -- 172.21.15.12:0/1095992961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c819daa0 con 0x7fb3c8100360 2024-04-22T13:34:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.169+0000 7fb3ce2a0640 1 --2- 172.21.15.12:0/1095992961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3c8100360 0x7fb3c81986f0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc005940 tx=0x7fb3bc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.170+0000 7fb3bb7fe640 1 -- 172.21.15.12:0/1095992961 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc043020 con 0x7fb3c8100360 2024-04-22T13:34:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.170+0000 7fb3d0504640 1 -- 172.21.15.12:0/1095992961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3c819dd30 con 0x7fb3c8100360 2024-04-22T13:34:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.170+0000 7fb3bb7fe640 1 -- 172.21.15.12:0/1095992961 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3bc02caf0 con 0x7fb3c8100360 2024-04-22T13:34:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.170+0000 7fb3bb7fe640 1 -- 172.21.15.12:0/1095992961 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc003af0 con 0x7fb3c8100360 2024-04-22T13:34:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.170+0000 7fb3d0504640 1 -- 172.21.15.12:0/1095992961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3c81a5580 con 0x7fb3c8100360 2024-04-22T13:34:37.170 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.171+0000 7fb3d0504640 1 -- 172.21.15.12:0/1095992961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3c8069230 con 0x7fb3c8100360 2024-04-22T13:34:37.172 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.172+0000 7fb3bb7fe640 1 -- 172.21.15.12:0/1095992961 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb3bc046020 con 0x7fb3c8100360 2024-04-22T13:34:37.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.172+0000 7fb3bb7fe640 1 --2- 172.21.15.12:0/1095992961 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb3b40a3d20 0x7fb3b40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.173+0000 7fb3cda9f640 1 --2- 172.21.15.12:0/1095992961 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb3b40a3d20 0x7fb3b40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.173+0000 7fb3bb7fe640 1 -- 172.21.15.12:0/1095992961 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7fb3bc0f2eb0 con 0x7fb3c8100360 2024-04-22T13:34:37.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.173+0000 7fb3cda9f640 1 --2- 172.21.15.12:0/1095992961 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb3b40a3d20 0x7fb3b40a61e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb3c80ff760 tx=0x7fb3c400b390 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.173 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.176+0000 7fb3bb7fe640 1 -- 172.21.15.12:0/1095992961 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb3bc04d020 con 0x7fb3c8100360 2024-04-22T13:34:37.401 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.406+0000 7fb3d0504640 1 -- 172.21.15.12:0/1095992961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3c810a440 con 0x7fb3c8100360 2024-04-22T13:34:37.402 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.406+0000 7fb3bb7fe640 1 -- 172.21.15.12:0/1095992961 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7fb3bc0bf620 con 0x7fb3c8100360 2024-04-22T13:34:37.402 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:34:37.402 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:34:37.402 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:34:37.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.408+0000 7fb3d0504640 1 -- 172.21.15.12:0/1095992961 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb3b40a3d20 msgr2=0x7fb3b40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.408+0000 7fb3d0504640 1 --2- 172.21.15.12:0/1095992961 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb3b40a3d20 0x7fb3b40a61e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb3c80ff760 tx=0x7fb3c400b390 comp rx=0 tx=0).stop 2024-04-22T13:34:37.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.408+0000 7fb3d0504640 1 -- 172.21.15.12:0/1095992961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3c8100360 msgr2=0x7fb3c81986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.408+0000 7fb3d0504640 1 --2- 172.21.15.12:0/1095992961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3c8100360 0x7fb3c81986f0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc005940 tx=0x7fb3bc0045e0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.408+0000 7fb3d0504640 1 -- 172.21.15.12:0/1095992961 shutdown_connections 2024-04-22T13:34:37.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.408+0000 7fb3d0504640 1 --2- 172.21.15.12:0/1095992961 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb3b40a3d20 0x7fb3b40a61e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.408+0000 7fb3d0504640 1 --2- 172.21.15.12:0/1095992961 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3c819cfb0 0x7fb3c819d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.408+0000 7fb3d0504640 1 --2- 172.21.15.12:0/1095992961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3c8108430 0x7fb3c8198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.408+0000 7fb3d0504640 1 --2- 172.21.15.12:0/1095992961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb3c8100360 0x7fb3c81986f0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.408+0000 7fb3d0504640 1 -- 172.21.15.12:0/1095992961 >> 172.21.15.12:0/1095992961 conn(0x7fb3c80770b0 msgr2=0x7fb3c8106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.408+0000 7fb3d0504640 1 -- 172.21.15.12:0/1095992961 shutdown_connections 2024-04-22T13:34:37.404 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.409+0000 7fb3d0504640 1 -- 172.21.15.12:0/1095992961 wait complete. 2024-04-22T13:34:37.415 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-22T13:34:37.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.524+0000 7f3b7045c640 1 -- 172.21.15.12:0/2474561901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b68102b40 msgr2=0x7f3b68106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.524+0000 7f3b7045c640 1 --2- 172.21.15.12:0/2474561901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b68102b40 0x7f3b68106b80 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f3b58009900 tx=0x7f3b5802bc30 comp rx=0 tx=0).stop 2024-04-22T13:34:37.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.524+0000 7f3b7045c640 1 -- 172.21.15.12:0/2474561901 shutdown_connections 2024-04-22T13:34:37.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.524+0000 7f3b7045c640 1 --2- 172.21.15.12:0/2474561901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b68107260 0x7f3b6810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.524+0000 7f3b7045c640 1 --2- 172.21.15.12:0/2474561901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b68102b40 0x7f3b68106b80 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.524+0000 7f3b7045c640 1 --2- 172.21.15.12:0/2474561901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b68102190 0x7f3b68102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.524+0000 7f3b7045c640 1 -- 172.21.15.12:0/2474561901 >> 172.21.15.12:0/2474561901 conn(0x7f3b680793a0 msgr2=0x7f3b680797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.524+0000 7f3b7045c640 1 -- 172.21.15.12:0/2474561901 shutdown_connections 2024-04-22T13:34:37.522 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.524+0000 7f3b7045c640 1 -- 172.21.15.12:0/2474561901 wait complete. 2024-04-22T13:34:37.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.524+0000 7f3b7045c640 1 Processor -- start 2024-04-22T13:34:37.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.524+0000 7f3b7045c640 1 -- start start 2024-04-22T13:34:37.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.524+0000 7f3b7045c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b68102190 0x7f3b6819e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.524+0000 7f3b7045c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b68107260 0x7f3b6819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.525+0000 7f3b7045c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b681985b0 0x7f3b68198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.525+0000 7f3b7045c640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3b68112250 con 0x7f3b681985b0 2024-04-22T13:34:37.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.525+0000 7f3b7045c640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3b681120d0 con 0x7f3b68107260 2024-04-22T13:34:37.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.525+0000 7f3b7045c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3b681123d0 con 0x7f3b68102190 2024-04-22T13:34:37.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.525+0000 7f3b6e9f9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b681985b0 0x7f3b68198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.523 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.525+0000 7f3b6e9f9640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b681985b0 0x7f3b68198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49846/0 (socket says 172.21.15.12:49846) 2024-04-22T13:34:37.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.525+0000 7f3b6e9f9640 1 -- 172.21.15.12:0/2641067632 learned_addr learned my addr 172.21.15.12:0/2641067632 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:37.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.525+0000 7f3b6d9f7640 1 --2- 172.21.15.12:0/2641067632 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b68107260 0x7f3b6819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.525+0000 7f3b6e1f8640 1 --2- 172.21.15.12:0/2641067632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b68102190 0x7f3b6819e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.525+0000 7f3b6e9f9640 1 -- 172.21.15.12:0/2641067632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b68102190 msgr2=0x7f3b6819e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.525+0000 7f3b6e9f9640 1 --2- 172.21.15.12:0/2641067632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b68102190 0x7f3b6819e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.525+0000 7f3b6e9f9640 1 -- 172.21.15.12:0/2641067632 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b68107260 msgr2=0x7f3b6819e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.525+0000 7f3b6e9f9640 1 --2- 172.21.15.12:0/2641067632 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b68107260 0x7f3b6819e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.525+0000 7f3b6e9f9640 1 -- 172.21.15.12:0/2641067632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b681990d0 con 0x7f3b681985b0 2024-04-22T13:34:37.524 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.525+0000 7f3b6e9f9640 1 --2- 172.21.15.12:0/2641067632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b681985b0 0x7f3b68198a60 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f3b6000cce0 tx=0x7f3b6000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.525+0000 7f3b5f7fe640 1 -- 172.21.15.12:0/2641067632 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6000f450 con 0x7f3b681985b0 2024-04-22T13:34:37.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.525+0000 7f3b7045c640 1 -- 172.21.15.12:0/2641067632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b681993c0 con 0x7f3b681985b0 2024-04-22T13:34:37.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.525+0000 7f3b5f7fe640 1 -- 172.21.15.12:0/2641067632 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b6000faa0 con 0x7f3b681985b0 2024-04-22T13:34:37.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.525+0000 7f3b5f7fe640 1 -- 172.21.15.12:0/2641067632 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b60018b70 con 0x7f3b681985b0 2024-04-22T13:34:37.525 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.525+0000 7f3b7045c640 1 -- 172.21.15.12:0/2641067632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b681a55c0 con 0x7f3b681985b0 2024-04-22T13:34:37.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.528+0000 7f3b5f7fe640 1 -- 172.21.15.12:0/2641067632 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3b60018d10 con 0x7f3b681985b0 2024-04-22T13:34:37.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.528+0000 7f3b73050740 1 Processor -- start 2024-04-22T13:34:37.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.528+0000 7f3b73050740 1 -- start start 2024-04-22T13:34:37.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.529+0000 7f3b73050740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7edd17250 0x55a7edd17630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.529+0000 7f3b73050740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7edd17c00 0x55a7edd24840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.529+0000 7f3b73050740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7edd24e10 0x55a7edd27220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.529+0000 7f3b73050740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a7edc7c1f0 con 0x55a7edd24e10 2024-04-22T13:34:37.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.529+0000 7f3b73050740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a7edc194b0 con 0x55a7edd17250 2024-04-22T13:34:37.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.529+0000 7f3b73050740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a7edc781b0 con 0x55a7edd17c00 2024-04-22T13:34:37.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.529+0000 7f3b6d9f7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7edd17250 0x55a7edd17630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.529+0000 7f3b6d9f7640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7edd17250 0x55a7edd17630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:55216/0 (socket says 172.21.15.12:55216) 2024-04-22T13:34:37.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.529+0000 7f3b6e1f8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7edd17c00 0x55a7edd24840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.529+0000 7f3b6d9f7640 1 -- 172.21.15.12:0/3228581904 learned_addr learned my addr 172.21.15.12:0/3228581904 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:37.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.529+0000 7f3b5f7fe640 1 --2- 172.21.15.12:0/2641067632 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3b540a3d20 0x7f3b540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.529+0000 7f3b6d9f7640 1 --2- 172.21.15.12:0/3228581904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7edd24e10 0x55a7edd27220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.529+0000 7f3b6e9f9640 1 --2- 172.21.15.12:0/2641067632 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3b540a3d20 0x7f3b540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.529+0000 7f3b6e1f8640 1 -- 172.21.15.12:0/3228581904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7edd17250 msgr2=0x55a7edd17630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.529+0000 7f3b6e1f8640 1 --2- 172.21.15.12:0/3228581904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7edd17250 0x55a7edd17630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.529+0000 7f3b5f7fe640 1 -- 172.21.15.12:0/2641067632 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f3b600d0c70 con 0x7f3b681985b0 2024-04-22T13:34:37.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.529+0000 7f3b6e1f8640 1 -- 172.21.15.12:0/3228581904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7edd24e10 msgr2=0x55a7edd27220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.529+0000 7f3b6e1f8640 1 --2- 172.21.15.12:0/3228581904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7edd24e10 0x55a7edd27220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.529+0000 7f3b6e1f8640 1 -- 172.21.15.12:0/3228581904 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7edc0b1b0 con 0x55a7edd17c00 2024-04-22T13:34:37.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.529+0000 7f3b6d9f7640 1 --2- 172.21.15.12:0/3228581904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7edd17250 0x55a7edd17630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:34:37.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.529+0000 7f3b6e1f8640 1 --2- 172.21.15.12:0/3228581904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7edd17c00 0x55a7edd24840 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400b760 tx=0x7f3b6400bc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84528007775f2bf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.530+0000 7f3b6e9f9640 1 --2- 172.21.15.12:0/2641067632 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3b540a3d20 0x7f3b540a61e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3b600ce490 tx=0x7f3b600676d0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.530+0000 7f3b5cbfe640 1 -- 172.21.15.12:0/3228581904 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6400d950 con 0x55a7edd17c00 2024-04-22T13:34:37.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.530+0000 7f3b5cbfe640 1 -- 172.21.15.12:0/3228581904 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b64019070 con 0x55a7edd17c00 2024-04-22T13:34:37.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.530+0000 7f3b5cbfe640 1 -- 172.21.15.12:0/3228581904 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b64014d30 con 0x55a7edd17c00 2024-04-22T13:34:37.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.530+0000 7f3b73050740 1 -- 172.21.15.12:0/3228581904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7edd17c00 msgr2=0x55a7edd24840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.530+0000 7f3b73050740 1 --2- 172.21.15.12:0/3228581904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7edd17c00 0x55a7edd24840 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400b760 tx=0x7f3b6400bc20 comp rx=0 tx=0).stop 2024-04-22T13:34:37.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.530+0000 7f3b73050740 1 -- 172.21.15.12:0/3228581904 shutdown_connections 2024-04-22T13:34:37.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.530+0000 7f3b73050740 1 --2- 172.21.15.12:0/3228581904 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7edd24e10 0x55a7edd27220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.530+0000 7f3b73050740 1 --2- 172.21.15.12:0/3228581904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7edd17c00 0x55a7edd24840 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.535 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.530+0000 7f3b73050740 1 --2- 172.21.15.12:0/3228581904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7edd17250 0x55a7edd17630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.530+0000 7f3b73050740 1 -- 172.21.15.12:0/3228581904 >> 172.21.15.12:0/3228581904 conn(0x55a7edd11ba0 msgr2=0x55a7edd11f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.531+0000 7f3b73050740 1 -- 172.21.15.12:0/3228581904 shutdown_connections 2024-04-22T13:34:37.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.531+0000 7f3b73050740 1 -- 172.21.15.12:0/3228581904 wait complete. 2024-04-22T13:34:37.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.531+0000 7f3b73050740 1 -- start start 2024-04-22T13:34:37.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.531+0000 7f3b73050740 1 Processor -- start 2024-04-22T13:34:37.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.531+0000 7f3b73050740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7edd17250 0x55a7edd24660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.531+0000 7f3b73050740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7edd17c00 0x55a7edd21a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.531+0000 7f3b73050740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7edd24e10 0x55a7edd21fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.531+0000 7f3b73050740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55a7edd27d40 con 0x55a7edd17250 2024-04-22T13:34:37.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.531+0000 7f3b73050740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a7edc35500 con 0x55a7edd17c00 2024-04-22T13:34:37.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.531+0000 7f3b73050740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a7edd27ec0 con 0x55a7edd24e10 2024-04-22T13:34:37.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.531+0000 7f3b73050740 10 client.0 _ll_register_callbacks cb 0x55a7edc03b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:37.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.531+0000 7f3b6d9f7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7edd17250 0x55a7edd24660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.531+0000 7f3b6d9f7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7edd17250 0x55a7edd24660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49856/0 (socket says 172.21.15.12:49856) 2024-04-22T13:34:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.531+0000 7f3b6d9f7640 1 -- 172.21.15.12:0/2778335299 learned_addr learned my addr 172.21.15.12:0/2778335299 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.532+0000 7f3b6e1f8640 1 --2- 172.21.15.12:0/2778335299 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7edd17c00 0x55a7edd21a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.532+0000 7f3b6d9f7640 1 -- 172.21.15.12:0/2778335299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7edd24e10 msgr2=0x55a7edd21fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.532+0000 7f3b6d9f7640 1 --2- 172.21.15.12:0/2778335299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7edd24e10 0x55a7edd21fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.532+0000 7f3b6d9f7640 1 -- 172.21.15.12:0/2778335299 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7edd17c00 msgr2=0x55a7edd21a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.532+0000 7f3b6d9f7640 1 --2- 172.21.15.12:0/2778335299 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7edd17c00 0x55a7edd21a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.532+0000 7f3b6d9f7640 1 -- 172.21.15.12:0/2778335299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7ed98e3f0 con 0x55a7edd17250 2024-04-22T13:34:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.532+0000 7f3b6d9f7640 1 --2- 172.21.15.12:0/2778335299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7edd17250 0x55a7edd24660 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f3b5802c240 tx=0x7f3b5802cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b4c6abefee1ca server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.532+0000 7f3b51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:34:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.532+0000 7f3b51ffb640 1 -- 172.21.15.12:0/2778335299 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b58047030 con 0x55a7edd17250 2024-04-22T13:34:37.537 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.532+0000 7f3b51ffb640 1 -- 172.21.15.12:0/2778335299 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f3b5803a040 con 0x55a7edd17250 2024-04-22T13:34:37.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.532+0000 7f3b73050740 1 -- 172.21.15.12:0/2778335299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b64019070 con 0x55a7edd17250 2024-04-22T13:34:37.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.532+0000 7f3b51ffb640 1 -- 172.21.15.12:0/2778335299 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3b58005480 con 0x55a7edd17250 2024-04-22T13:34:37.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.532+0000 7f3b73050740 10 client.19137 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:37.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.532+0000 7f3b73050740 1 -- 172.21.15.12:0/2778335299 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a7eda587d0 con 0x55a7edd17250 2024-04-22T13:34:37.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.532+0000 7f3b51ffb640 1 -- 172.21.15.12:0/2778335299 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b64019070 con 0x55a7edd17250 2024-04-22T13:34:37.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.532+0000 7f3b51ffb640 1 -- 172.21.15.12:0/2778335299 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3b58053020 con 0x55a7edd17250 2024-04-22T13:34:37.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.533+0000 7f3b73050740 10 client.19137 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:37.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.533+0000 7f3b73050740 10 client.19137 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:34:37.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.533+0000 7f3b73050740 20 client.19137 populate_metadata read hostname 'smithi012' 2024-04-22T13:34:37.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.533+0000 7f3b73050740 1 --2- 172.21.15.12:0/2778335299 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55a7edd34f10 0x55a7edd37360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.533+0000 7f3b73050740 4 client.19137 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:34:37.538 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.533+0000 7f3b73050740 1 -- 172.21.15.12:0/2778335299 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7edc1b930 con 0x55a7edd34f10 2024-04-22T13:34:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.533+0000 7f3b6e1f8640 1 --2- 172.21.15.12:0/2778335299 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55a7edd34f10 0x55a7edd37360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.533+0000 7f3b51ffb640 10 client.19137 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:34:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.533+0000 7f3b6e1f8640 1 --2- 172.21.15.12:0/2778335299 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55a7edd34f10 0x55a7edd37360 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=63ceadfb1cc76a2d server_cookie=515d1f222bdabfe2 in_seq=0 out_seq=0 2024-04-22T13:34:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.534+0000 7f3b51ffb640 1 -- 172.21.15.12:0/2778335299 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a7edc1b930 con 0x55a7edd34f10 2024-04-22T13:34:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.534+0000 7f3b51ffb640 10 client.19137 handle_command_reply: tid=0 2024-04-22T13:34:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.534+0000 7f3b73050740 1 client.19137 shutdown 2024-04-22T13:34:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.534+0000 7f3b73050740 1 -- 172.21.15.12:0/2778335299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7edd17250 msgr2=0x55a7edd24660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.534+0000 7f3b73050740 1 --2- 172.21.15.12:0/2778335299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7edd17250 0x55a7edd24660 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f3b5802c240 tx=0x7f3b5802cdc0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.534+0000 7f3b73050740 1 -- 172.21.15.12:0/2778335299 shutdown_connections 2024-04-22T13:34:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.534+0000 7f3b73050740 1 --2- 172.21.15.12:0/2778335299 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55a7edd34f10 0x55a7edd37360 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.534+0000 7f3b73050740 1 --2- 172.21.15.12:0/2778335299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7edd24e10 0x55a7edd21fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.534+0000 7f3b73050740 1 --2- 172.21.15.12:0/2778335299 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7edd17c00 0x55a7edd21a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.534+0000 7f3b51ffb640 0 client.19137 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:34:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.534+0000 7f3b73050740 1 --2- 172.21.15.12:0/2778335299 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7edd17250 0x55a7edd24660 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.534+0000 7f3b73050740 1 -- 172.21.15.12:0/2778335299 >> 172.21.15.12:0/2778335299 conn(0x55a7edd11ba0 msgr2=0x55a7edd138f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.534+0000 7f3b73050740 1 -- 172.21.15.12:0/2778335299 shutdown_connections 2024-04-22T13:34:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.534+0000 7f3b73050740 1 -- 172.21.15.12:0/2778335299 wait complete. 2024-04-22T13:34:37.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.534+0000 7f3b73050740 20 client.19137 trim_cache size 0 max 16384 2024-04-22T13:34:37.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.547+0000 7f3b73050740 1 Processor -- start 2024-04-22T13:34:37.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.547+0000 7f3b73050740 1 -- start start 2024-04-22T13:34:37.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.548+0000 7f3b73050740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7edd17250 0x55a7edd30c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.548+0000 7f3b73050740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7edd17c00 0x55a7edd21230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.548+0000 7f3b73050740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7edd24e10 0x55a7edd24190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.549 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.548+0000 7f3b73050740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3b681ab940 con 0x55a7edd24e10 2024-04-22T13:34:37.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.548+0000 7f3b73050740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3b681ad480 con 0x55a7edd17250 2024-04-22T13:34:37.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.548+0000 7f3b73050740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3b681ac010 con 0x55a7edd17c00 2024-04-22T13:34:37.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.548+0000 7f3b6d9f7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7edd24e10 0x55a7edd24190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.548+0000 7f3b6d9f7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7edd24e10 0x55a7edd24190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49860/0 (socket says 172.21.15.12:49860) 2024-04-22T13:34:37.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.548+0000 7f3b6d9f7640 1 -- 172.21.15.12:0/3784994328 learned_addr learned my addr 172.21.15.12:0/3784994328 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:37.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.548+0000 7f3b6d9f7640 1 --2- 172.21.15.12:0/3784994328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7edd17250 0x55a7edd30c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.548+0000 7f3b6d9f7640 1 -- 172.21.15.12:0/3784994328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7edd17c00 msgr2=0x55a7edd21230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.548+0000 7f3b6d9f7640 1 --2- 172.21.15.12:0/3784994328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7edd17c00 0x55a7edd21230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.548+0000 7f3b6d9f7640 1 -- 172.21.15.12:0/3784994328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7edd17250 msgr2=0x55a7edd30c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.548+0000 7f3b6d9f7640 1 --2- 172.21.15.12:0/3784994328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7edd17250 0x55a7edd30c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.548+0000 7f3b6d9f7640 1 -- 172.21.15.12:0/3784994328 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7edd0bfe0 con 0x55a7edd24e10 2024-04-22T13:34:37.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.548+0000 7f3b6d9f7640 1 --2- 172.21.15.12:0/3784994328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7edd24e10 0x55a7edd24190 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f3b58006010 tx=0x7f3b58001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e5886c532b7c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.548+0000 7f3b53fff640 1 -- 172.21.15.12:0/3784994328 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b58047040 con 0x55a7edd24e10 2024-04-22T13:34:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.548+0000 7f3b53fff640 1 -- 172.21.15.12:0/3784994328 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b580038c0 con 0x55a7edd24e10 2024-04-22T13:34:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.548+0000 7f3b53fff640 1 -- 172.21.15.12:0/3784994328 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b58055460 con 0x55a7edd24e10 2024-04-22T13:34:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.549+0000 7f3b73050740 1 -- 172.21.15.12:0/3784994328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7edd24e10 msgr2=0x55a7edd24190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.549+0000 7f3b73050740 1 --2- 172.21.15.12:0/3784994328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7edd24e10 0x55a7edd24190 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f3b58006010 tx=0x7f3b58001e50 comp rx=0 tx=0).stop 2024-04-22T13:34:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.549+0000 7f3b73050740 1 -- 172.21.15.12:0/3784994328 shutdown_connections 2024-04-22T13:34:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.549+0000 7f3b73050740 1 --2- 172.21.15.12:0/3784994328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7edd24e10 0x55a7edd24190 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.549+0000 7f3b73050740 1 --2- 172.21.15.12:0/3784994328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7edd17c00 0x55a7edd21230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.549+0000 7f3b73050740 1 --2- 172.21.15.12:0/3784994328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7edd17250 0x55a7edd30c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.549+0000 7f3b73050740 1 -- 172.21.15.12:0/3784994328 >> 172.21.15.12:0/3784994328 conn(0x55a7edd11ba0 msgr2=0x55a7edd10c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.549+0000 7f3b73050740 1 -- 172.21.15.12:0/3784994328 shutdown_connections 2024-04-22T13:34:37.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.549+0000 7f3b73050740 1 -- 172.21.15.12:0/3784994328 wait complete. 2024-04-22T13:34:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.549+0000 7f3b73050740 1 -- start start 2024-04-22T13:34:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.549+0000 7f3b73050740 1 Processor -- start 2024-04-22T13:34:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.549+0000 7f3b73050740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7edd17250 0x55a7edd2d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.550+0000 7f3b73050740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7edd17c00 0x55a7edd2d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.550+0000 7f3b73050740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7edd24e10 0x55a7edd1e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.550+0000 7f3b73050740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f3b681acdb0 con 0x55a7edd24e10 2024-04-22T13:34:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.550+0000 7f3b73050740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3b681ac6e0 con 0x55a7edd17c00 2024-04-22T13:34:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.550+0000 7f3b73050740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3b681a9940 con 0x55a7edd17250 2024-04-22T13:34:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.550+0000 7f3b73050740 10 client.0 _ll_register_callbacks cb 0x7f3b5802cc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.550+0000 7f3b6d9f7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7edd24e10 0x55a7edd1e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.550+0000 7f3b6d9f7640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7edd24e10 0x55a7edd1e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49870/0 (socket says 172.21.15.12:49870) 2024-04-22T13:34:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.550+0000 7f3b6d9f7640 1 -- 172.21.15.12:0/3538085416 learned_addr learned my addr 172.21.15.12:0/3538085416 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.550+0000 7f3b6e1f8640 1 --2- 172.21.15.12:0/3538085416 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7edd17c00 0x55a7edd2d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.550+0000 7f3b6d9f7640 1 --2- 172.21.15.12:0/3538085416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7edd17250 0x55a7edd2d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.550+0000 7f3b6d9f7640 1 -- 172.21.15.12:0/3538085416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7edd17250 msgr2=0x55a7edd2d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.550+0000 7f3b6d9f7640 1 --2- 172.21.15.12:0/3538085416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7edd17250 0x55a7edd2d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.550+0000 7f3b6d9f7640 1 -- 172.21.15.12:0/3538085416 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7edd17c00 msgr2=0x55a7edd2d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.550+0000 7f3b6d9f7640 1 --2- 172.21.15.12:0/3538085416 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7edd17c00 0x55a7edd2d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.550+0000 7f3b6d9f7640 1 -- 172.21.15.12:0/3538085416 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7edd1e9c0 con 0x55a7edd24e10 2024-04-22T13:34:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.550+0000 7f3b6d9f7640 1 --2- 172.21.15.12:0/3538085416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7edd24e10 0x55a7edd1e1f0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f3b58000c00 tx=0x7f3b58005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db706c0bf5d16ec server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.550+0000 7f3b51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:34:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.550+0000 7f3b51ffb640 1 -- 172.21.15.12:0/3538085416 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b58047070 con 0x55a7edd24e10 2024-04-22T13:34:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.550+0000 7f3b51ffb640 1 -- 172.21.15.12:0/3538085416 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f3b58004de0 con 0x55a7edd24e10 2024-04-22T13:34:37.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.550+0000 7f3b73050740 1 -- 172.21.15.12:0/3538085416 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b580038c0 con 0x55a7edd24e10 2024-04-22T13:34:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.550+0000 7f3b51ffb640 1 -- 172.21.15.12:0/3538085416 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3b5805a040 con 0x55a7edd24e10 2024-04-22T13:34:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.550+0000 7f3b73050740 10 client.19143 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.550+0000 7f3b73050740 1 -- 172.21.15.12:0/3538085416 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a7edd9b030 con 0x55a7edd24e10 2024-04-22T13:34:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.551+0000 7f3b51ffb640 1 -- 172.21.15.12:0/3538085416 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b580038c0 con 0x55a7edd24e10 2024-04-22T13:34:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.551+0000 7f3b51ffb640 1 -- 172.21.15.12:0/3538085416 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3b58053070 con 0x55a7edd24e10 2024-04-22T13:34:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.551+0000 7f3b73050740 10 client.19143 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.551+0000 7f3b73050740 10 client.19143 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-04-22T13:34:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.551+0000 7f3b73050740 20 client.19143 populate_metadata read hostname 'smithi012' 2024-04-22T13:34:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.551+0000 7f3b73050740 1 --2- 172.21.15.12:0/3538085416 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55a7edd9b670 0x55a7edd9da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.551+0000 7f3b73050740 4 client.19143 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:34:37.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.551+0000 7f3b73050740 1 -- 172.21.15.12:0/3538085416 --> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a7edc89c10 con 0x55a7edd9b670 2024-04-22T13:34:37.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.551+0000 7f3b6e1f8640 1 --2- 172.21.15.12:0/3538085416 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55a7edd9b670 0x55a7edd9da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.551+0000 7f3b6e1f8640 1 --2- 172.21.15.12:0/3538085416 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55a7edd9b670 0x55a7edd9da60 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1d6d5b62f7607374 server_cookie=9122de06cbb6f5a in_seq=0 out_seq=0 2024-04-22T13:34:37.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.551+0000 7f3b51ffb640 10 client.19143 ms_handle_connect on v2:172.21.15.111:6832/1902687060 2024-04-22T13:34:37.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.552+0000 7f3b51ffb640 1 -- 172.21.15.12:0/3538085416 <== mds.4 v2:172.21.15.111:6832/1902687060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a7edc89c10 con 0x55a7edd9b670 2024-04-22T13:34:37.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.552+0000 7f3b51ffb640 10 client.19143 handle_command_reply: tid=0 2024-04-22T13:34:37.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.552+0000 7f3b73050740 1 client.19143 shutdown 2024-04-22T13:34:37.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.552+0000 7f3b73050740 1 -- 172.21.15.12:0/3538085416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7edd24e10 msgr2=0x55a7edd1e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.552+0000 7f3b73050740 1 --2- 172.21.15.12:0/3538085416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7edd24e10 0x55a7edd1e1f0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f3b58000c00 tx=0x7f3b58005960 comp rx=0 tx=0).stop 2024-04-22T13:34:37.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.552+0000 7f3b73050740 1 -- 172.21.15.12:0/3538085416 shutdown_connections 2024-04-22T13:34:37.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.552+0000 7f3b73050740 1 --2- 172.21.15.12:0/3538085416 >> [v2:172.21.15.111:6832/1902687060,v1:172.21.15.111:6833/1902687060] conn(0x55a7edd9b670 0x55a7edd9da60 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.552+0000 7f3b73050740 1 --2- 172.21.15.12:0/3538085416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a7edd24e10 0x55a7edd1e1f0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.552+0000 7f3b73050740 1 --2- 172.21.15.12:0/3538085416 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a7edd17c00 0x55a7edd2d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.552+0000 7f3b51ffb640 0 client.19143 ms_handle_reset on v2:172.21.15.111:6832/1902687060 2024-04-22T13:34:37.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.552+0000 7f3b73050740 1 --2- 172.21.15.12:0/3538085416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a7edd17250 0x55a7edd2d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.552+0000 7f3b73050740 1 -- 172.21.15.12:0/3538085416 >> 172.21.15.12:0/3538085416 conn(0x55a7edd11ba0 msgr2=0x55a7edd1c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.552+0000 7f3b73050740 1 -- 172.21.15.12:0/3538085416 shutdown_connections 2024-04-22T13:34:37.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.552+0000 7f3b73050740 1 -- 172.21.15.12:0/3538085416 wait complete. 2024-04-22T13:34:37.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.552+0000 7f3b73050740 20 client.19143 trim_cache size 0 max 16384 2024-04-22T13:34:37.556 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:34:37.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.552+0000 7f3b7045c640 1 -- 172.21.15.12:0/2641067632 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3b540a3d20 msgr2=0x7f3b540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.552+0000 7f3b7045c640 1 --2- 172.21.15.12:0/2641067632 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3b540a3d20 0x7f3b540a61e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3b600ce490 tx=0x7f3b600676d0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.552+0000 7f3b7045c640 1 -- 172.21.15.12:0/2641067632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b681985b0 msgr2=0x7f3b68198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.552+0000 7f3b7045c640 1 --2- 172.21.15.12:0/2641067632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b681985b0 0x7f3b68198a60 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f3b6000cce0 tx=0x7f3b6000a620 comp rx=0 tx=0).stop 2024-04-22T13:34:37.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.552+0000 7f3b7045c640 1 -- 172.21.15.12:0/2641067632 shutdown_connections 2024-04-22T13:34:37.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.552+0000 7f3b7045c640 1 --2- 172.21.15.12:0/2641067632 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f3b540a3d20 0x7f3b540a61e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.552+0000 7f3b7045c640 1 --2- 172.21.15.12:0/2641067632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3b681985b0 0x7f3b68198a60 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.552+0000 7f3b7045c640 1 --2- 172.21.15.12:0/2641067632 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b68107260 0x7f3b6819e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.552+0000 7f3b7045c640 1 --2- 172.21.15.12:0/2641067632 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b68102190 0x7f3b6819e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.552+0000 7f3b7045c640 1 -- 172.21.15.12:0/2641067632 >> 172.21.15.12:0/2641067632 conn(0x7f3b680793a0 msgr2=0x7f3b680fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.553+0000 7f3b7045c640 1 -- 172.21.15.12:0/2641067632 shutdown_connections 2024-04-22T13:34:37.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.553+0000 7f3b7045c640 1 -- 172.21.15.12:0/2641067632 wait complete. 2024-04-22T13:34:37.559 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-22T13:34:37.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.667+0000 7ff3d8d75640 1 -- 172.21.15.12:0/2300001644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3d4106170 msgr2=0x7ff3d410dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.667+0000 7ff3d8d75640 1 --2- 172.21.15.12:0/2300001644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3d4106170 0x7ff3d410dd40 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc009ca0 tx=0x7ff3cc02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.667+0000 7ff3d8d75640 1 -- 172.21.15.12:0/2300001644 shutdown_connections 2024-04-22T13:34:37.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.667+0000 7ff3d8d75640 1 --2- 172.21.15.12:0/2300001644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3d4106170 0x7ff3d410dd40 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.667+0000 7ff3d8d75640 1 --2- 172.21.15.12:0/2300001644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3d4101a50 0x7ff3d4105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.667+0000 7ff3d8d75640 1 --2- 172.21.15.12:0/2300001644 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3d41010a0 0x7ff3d4101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.667+0000 7ff3d8d75640 1 -- 172.21.15.12:0/2300001644 >> 172.21.15.12:0/2300001644 conn(0x7ff3d4077eb0 msgr2=0x7ff3d40782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.667+0000 7ff3d8d75640 1 -- 172.21.15.12:0/2300001644 shutdown_connections 2024-04-22T13:34:37.665 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.667+0000 7ff3d8d75640 1 -- 172.21.15.12:0/2300001644 wait complete. 2024-04-22T13:34:37.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.668+0000 7ff3d8d75640 1 Processor -- start 2024-04-22T13:34:37.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.668+0000 7ff3d8d75640 1 -- start start 2024-04-22T13:34:37.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.668+0000 7ff3d8d75640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3d41010a0 0x7ff3d4197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.668+0000 7ff3d8d75640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3d4106170 0x7ff3d4197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.668+0000 7ff3d8d75640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3d419be40 0x7ff3d419c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.668+0000 7ff3d8d75640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d41110e0 con 0x7ff3d41010a0 2024-04-22T13:34:37.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.668+0000 7ff3d8d75640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d4110f60 con 0x7ff3d419be40 2024-04-22T13:34:37.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.668+0000 7ff3d8d75640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d4111260 con 0x7ff3d4106170 2024-04-22T13:34:37.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.668+0000 7ff3d259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3d41010a0 0x7ff3d4197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.668+0000 7ff3d259c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3d41010a0 0x7ff3d4197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49892/0 (socket says 172.21.15.12:49892) 2024-04-22T13:34:37.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.668+0000 7ff3d259c640 1 -- 172.21.15.12:0/4240182532 learned_addr learned my addr 172.21.15.12:0/4240182532 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:37.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.668+0000 7ff3cbfff640 1 --2- 172.21.15.12:0/4240182532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3d4106170 0x7ff3d4197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.666 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.669+0000 7ff3d2d9d640 1 --2- 172.21.15.12:0/4240182532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3d419be40 0x7ff3d419c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.669+0000 7ff3d259c640 1 -- 172.21.15.12:0/4240182532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3d4106170 msgr2=0x7ff3d4197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.669+0000 7ff3d259c640 1 --2- 172.21.15.12:0/4240182532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3d4106170 0x7ff3d4197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.669+0000 7ff3d259c640 1 -- 172.21.15.12:0/4240182532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3d419be40 msgr2=0x7ff3d419c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.669+0000 7ff3d259c640 1 --2- 172.21.15.12:0/4240182532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3d419be40 0x7ff3d419c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.669+0000 7ff3d259c640 1 -- 172.21.15.12:0/4240182532 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d419c930 con 0x7ff3d41010a0 2024-04-22T13:34:37.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.669+0000 7ff3d2d9d640 1 --2- 172.21.15.12:0/4240182532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3d419be40 0x7ff3d419c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:37.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.669+0000 7ff3d259c640 1 --2- 172.21.15.12:0/4240182532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3d41010a0 0x7ff3d4197580 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400c920 tx=0x7ff3c400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.669+0000 7ff3cb7fe640 1 -- 172.21.15.12:0/4240182532 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4007c70 con 0x7ff3d41010a0 2024-04-22T13:34:37.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.669+0000 7ff3cb7fe640 1 -- 172.21.15.12:0/4240182532 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3c4007e10 con 0x7ff3d41010a0 2024-04-22T13:34:37.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.669+0000 7ff3cb7fe640 1 -- 172.21.15.12:0/4240182532 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4018820 con 0x7ff3d41010a0 2024-04-22T13:34:37.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.669+0000 7ff3d8d75640 1 -- 172.21.15.12:0/4240182532 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d419cc20 con 0x7ff3d41010a0 2024-04-22T13:34:37.667 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.669+0000 7ff3d8d75640 1 -- 172.21.15.12:0/4240182532 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d406a8c0 con 0x7ff3d41010a0 2024-04-22T13:34:37.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.671+0000 7ff3cb7fe640 1 -- 172.21.15.12:0/4240182532 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff3c400f450 con 0x7ff3d41010a0 2024-04-22T13:34:37.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.672+0000 7ff3db966740 1 Processor -- start 2024-04-22T13:34:37.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.672+0000 7ff3db966740 1 -- start start 2024-04-22T13:34:37.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.672+0000 7ff3db966740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9008d03b0 0x55f9008d0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.674 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.672+0000 7ff3db966740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f9008d0d60 0x55f9008d9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.672+0000 7ff3db966740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9008d9e60 0x55f9008da2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.672+0000 7ff3db966740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f9008351f0 con 0x55f9008d03b0 2024-04-22T13:34:37.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.672+0000 7ff3db966740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f9007d24b0 con 0x55f9008d0d60 2024-04-22T13:34:37.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.672+0000 7ff3db966740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f9008311b0 con 0x55f9008d9e60 2024-04-22T13:34:37.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.672+0000 7ff3d2d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9008d03b0 0x55f9008d0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.672+0000 7ff3d2d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9008d03b0 0x55f9008d0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49906/0 (socket says 172.21.15.12:49906) 2024-04-22T13:34:37.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.672+0000 7ff3d2d9d640 1 -- 172.21.15.12:0/2871192560 learned_addr learned my addr 172.21.15.12:0/2871192560 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:37.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.672+0000 7ff3cb7fe640 1 --2- 172.21.15.12:0/4240182532 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff3c00a3da0 0x7ff3c00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.672+0000 7ff3cbfff640 1 --2- 172.21.15.12:0/2871192560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9008d9e60 0x55f9008da2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.672+0000 7ff3d259c640 1 --2- 172.21.15.12:0/2871192560 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f9008d0d60 0x55f9008d9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.672+0000 7ff3d2d9d640 1 --2- 172.21.15.12:0/4240182532 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff3c00a3da0 0x7ff3c00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.672+0000 7ff3d2d9d640 1 -- 172.21.15.12:0/2871192560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9008d9e60 msgr2=0x55f9008da2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.673+0000 7ff3d2d9d640 1 --2- 172.21.15.12:0/2871192560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9008d9e60 0x55f9008da2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.675 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.673+0000 7ff3d2d9d640 1 -- 172.21.15.12:0/2871192560 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f9008d0d60 msgr2=0x55f9008d9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.673+0000 7ff3d2d9d640 1 --2- 172.21.15.12:0/2871192560 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f9008d0d60 0x55f9008d9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.673+0000 7ff3d2d9d640 1 -- 172.21.15.12:0/2871192560 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9007c41b0 con 0x55f9008d03b0 2024-04-22T13:34:37.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.673+0000 7ff3cb7fe640 1 -- 172.21.15.12:0/4240182532 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7ff3c40d0980 con 0x7ff3d41010a0 2024-04-22T13:34:37.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.673+0000 7ff3d2d9d640 1 --2- 172.21.15.12:0/2871192560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9008d03b0 0x55f9008d0790 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc009430 tx=0x7ff3cc032f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a90259c28e9f3446 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.673+0000 7ff3c8ff9640 1 -- 172.21.15.12:0/2871192560 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc0327a0 con 0x55f9008d03b0 2024-04-22T13:34:37.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.673+0000 7ff3c8ff9640 1 -- 172.21.15.12:0/2871192560 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3cc032940 con 0x55f9008d03b0 2024-04-22T13:34:37.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.673+0000 7ff3c8ff9640 1 -- 172.21.15.12:0/2871192560 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc034900 con 0x55f9008d03b0 2024-04-22T13:34:37.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.674+0000 7ff3db966740 1 -- 172.21.15.12:0/2871192560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9008d03b0 msgr2=0x55f9008d0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.674+0000 7ff3d2d9d640 1 --2- 172.21.15.12:0/4240182532 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff3c00a3da0 0x7ff3c00a6260 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc04c000 tx=0x7ff3cc048000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.674+0000 7ff3db966740 1 --2- 172.21.15.12:0/2871192560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9008d03b0 0x55f9008d0790 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc009430 tx=0x7ff3cc032f30 comp rx=0 tx=0).stop 2024-04-22T13:34:37.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.674+0000 7ff3db966740 1 -- 172.21.15.12:0/2871192560 shutdown_connections 2024-04-22T13:34:37.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.674+0000 7ff3db966740 1 --2- 172.21.15.12:0/2871192560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9008d9e60 0x55f9008da2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.674+0000 7ff3db966740 1 --2- 172.21.15.12:0/2871192560 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f9008d0d60 0x55f9008d9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.674+0000 7ff3db966740 1 --2- 172.21.15.12:0/2871192560 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9008d03b0 0x55f9008d0790 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.676 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.674+0000 7ff3db966740 1 -- 172.21.15.12:0/2871192560 >> 172.21.15.12:0/2871192560 conn(0x55f9008caba0 msgr2=0x55f9008caf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.674+0000 7ff3db966740 1 -- 172.21.15.12:0/2871192560 shutdown_connections 2024-04-22T13:34:37.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.674+0000 7ff3db966740 1 -- 172.21.15.12:0/2871192560 wait complete. 2024-04-22T13:34:37.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.674+0000 7ff3db966740 1 -- start start 2024-04-22T13:34:37.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.674+0000 7ff3db966740 1 Processor -- start 2024-04-22T13:34:37.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.674+0000 7ff3db966740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f9008d03b0 0x55f9008e0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.674+0000 7ff3db966740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9008d0d60 0x55f9008dd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.674+0000 7ff3db966740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9008d9e60 0x55f9008ddb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.674+0000 7ff3db966740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55f9008e0da0 con 0x55f9008d9e60 2024-04-22T13:34:37.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.674+0000 7ff3db966740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f9007ee500 con 0x55f9008d03b0 2024-04-22T13:34:37.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.674+0000 7ff3db966740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f9008e0f20 con 0x55f9008d0d60 2024-04-22T13:34:37.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.674+0000 7ff3db966740 10 client.0 _ll_register_callbacks cb 0x55f9007bcb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:37.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.675+0000 7ff3d2d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9008d9e60 0x55f9008ddb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.675+0000 7ff3d2d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9008d9e60 0x55f9008ddb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49916/0 (socket says 172.21.15.12:49916) 2024-04-22T13:34:37.677 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.675+0000 7ff3d2d9d640 1 -- 172.21.15.12:0/2252071342 learned_addr learned my addr 172.21.15.12:0/2252071342 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:37.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.675+0000 7ff3d259c640 1 --2- 172.21.15.12:0/2252071342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f9008d03b0 0x55f9008e0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.675+0000 7ff3cbfff640 1 --2- 172.21.15.12:0/2252071342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9008d0d60 0x55f9008dd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.675+0000 7ff3d2d9d640 1 -- 172.21.15.12:0/2252071342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9008d0d60 msgr2=0x55f9008dd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.675+0000 7ff3d2d9d640 1 --2- 172.21.15.12:0/2252071342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9008d0d60 0x55f9008dd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.675+0000 7ff3d2d9d640 1 -- 172.21.15.12:0/2252071342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f9008d03b0 msgr2=0x55f9008e0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.675+0000 7ff3d2d9d640 1 --2- 172.21.15.12:0/2252071342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f9008d03b0 0x55f9008e0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.675+0000 7ff3d2d9d640 1 -- 172.21.15.12:0/2252071342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9005473f0 con 0x55f9008d9e60 2024-04-22T13:34:37.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.675+0000 7ff3d2d9d640 1 --2- 172.21.15.12:0/2252071342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9008d9e60 0x55f9008ddb10 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc02c130 tx=0x7ff3cc004650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff1bcb22f70a7fd server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.675+0000 7ff3b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:34:37.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.675+0000 7ff3b9ffb640 1 -- 172.21.15.12:0/2252071342 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc05b060 con 0x55f9008d9e60 2024-04-22T13:34:37.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.675+0000 7ff3b9ffb640 1 -- 172.21.15.12:0/2252071342 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7ff3cc003d40 con 0x55f9008d9e60 2024-04-22T13:34:37.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.675+0000 7ff3db966740 1 -- 172.21.15.12:0/2252071342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3cc032940 con 0x55f9008d9e60 2024-04-22T13:34:37.678 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.675+0000 7ff3b9ffb640 1 -- 172.21.15.12:0/2252071342 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff3cc039050 con 0x55f9008d9e60 2024-04-22T13:34:37.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.675+0000 7ff3db966740 10 client.19155 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:37.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.675+0000 7ff3db966740 1 -- 172.21.15.12:0/2252071342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f9006117d0 con 0x55f9008d9e60 2024-04-22T13:34:37.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.676+0000 7ff3b9ffb640 1 -- 172.21.15.12:0/2252071342 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3cc032940 con 0x55f9008d9e60 2024-04-22T13:34:37.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.676+0000 7ff3b9ffb640 1 -- 172.21.15.12:0/2252071342 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff3cc0438e0 con 0x55f9008d9e60 2024-04-22T13:34:37.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.676+0000 7ff3db966740 10 client.19155 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:37.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.676+0000 7ff3db966740 10 client.19155 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:34:37.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.676+0000 7ff3db966740 20 client.19155 populate_metadata read hostname 'smithi012' 2024-04-22T13:34:37.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.676+0000 7ff3db966740 1 --2- 172.21.15.12:0/2252071342 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55f9008edde0 0x55f9008f01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.676+0000 7ff3db966740 4 client.19155 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:34:37.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.676+0000 7ff3db966740 1 -- 172.21.15.12:0/2252071342 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9007d4930 con 0x55f9008edde0 2024-04-22T13:34:37.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.676+0000 7ff3d259c640 1 --2- 172.21.15.12:0/2252071342 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55f9008edde0 0x55f9008f01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.676+0000 7ff3b9ffb640 10 client.19155 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:34:37.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.676+0000 7ff3d259c640 1 --2- 172.21.15.12:0/2252071342 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55f9008edde0 0x55f9008f01d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e6d06eae60e6c730 server_cookie=a5be7ab961dedbad in_seq=0 out_seq=0 2024-04-22T13:34:37.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.677+0000 7ff3b9ffb640 1 -- 172.21.15.12:0/2252071342 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f9007d4930 con 0x55f9008edde0 2024-04-22T13:34:37.679 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.677+0000 7ff3b9ffb640 10 client.19155 handle_command_reply: tid=0 2024-04-22T13:34:37.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.677+0000 7ff3db966740 1 client.19155 shutdown 2024-04-22T13:34:37.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.677+0000 7ff3db966740 1 -- 172.21.15.12:0/2252071342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9008d9e60 msgr2=0x55f9008ddb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.677+0000 7ff3db966740 1 --2- 172.21.15.12:0/2252071342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9008d9e60 0x55f9008ddb10 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc02c130 tx=0x7ff3cc004650 comp rx=0 tx=0).stop 2024-04-22T13:34:37.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.677+0000 7ff3db966740 1 -- 172.21.15.12:0/2252071342 shutdown_connections 2024-04-22T13:34:37.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.677+0000 7ff3db966740 1 --2- 172.21.15.12:0/2252071342 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55f9008edde0 0x55f9008f01d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.677+0000 7ff3db966740 1 --2- 172.21.15.12:0/2252071342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9008d9e60 0x55f9008ddb10 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.677+0000 7ff3db966740 1 --2- 172.21.15.12:0/2252071342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9008d0d60 0x55f9008dd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.677+0000 7ff3b9ffb640 0 client.19155 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:34:37.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.677+0000 7ff3db966740 1 --2- 172.21.15.12:0/2252071342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f9008d03b0 0x55f9008e0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.677+0000 7ff3db966740 1 -- 172.21.15.12:0/2252071342 >> 172.21.15.12:0/2252071342 conn(0x55f9008caba0 msgr2=0x55f9008cc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.677+0000 7ff3db966740 1 -- 172.21.15.12:0/2252071342 shutdown_connections 2024-04-22T13:34:37.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.677+0000 7ff3db966740 1 -- 172.21.15.12:0/2252071342 wait complete. 2024-04-22T13:34:37.680 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.677+0000 7ff3db966740 20 client.19155 trim_cache size 0 max 16384 2024-04-22T13:34:37.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.691+0000 7ff3db966740 1 Processor -- start 2024-04-22T13:34:37.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.691+0000 7ff3db966740 1 -- start start 2024-04-22T13:34:37.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.691+0000 7ff3db966740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9008d03b0 0x55f9008e9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.691+0000 7ff3db966740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9008d0d60 0x55f9008cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.691+0000 7ff3db966740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f9008d9e60 0x55f9008dfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.691+0000 7ff3db966740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d41ab750 con 0x55f9008d0d60 2024-04-22T13:34:37.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.691+0000 7ff3db966740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d41ad290 con 0x55f9008d9e60 2024-04-22T13:34:37.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.691+0000 7ff3db966740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d41abe20 con 0x55f9008d03b0 2024-04-22T13:34:37.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.691+0000 7ff3cbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9008d0d60 0x55f9008cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.691+0000 7ff3cbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9008d0d60 0x55f9008cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49932/0 (socket says 172.21.15.12:49932) 2024-04-22T13:34:37.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.691+0000 7ff3cbfff640 1 -- 172.21.15.12:0/3101994901 learned_addr learned my addr 172.21.15.12:0/3101994901 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:37.692 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.691+0000 7ff3d259c640 1 --2- 172.21.15.12:0/3101994901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9008d03b0 0x55f9008e9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.691+0000 7ff3d2d9d640 1 --2- 172.21.15.12:0/3101994901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f9008d9e60 0x55f9008dfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.691+0000 7ff3cbfff640 1 -- 172.21.15.12:0/3101994901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9008d03b0 msgr2=0x55f9008e9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.691+0000 7ff3cbfff640 1 --2- 172.21.15.12:0/3101994901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9008d03b0 0x55f9008e9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.691+0000 7ff3cbfff640 1 -- 172.21.15.12:0/3101994901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f9008d9e60 msgr2=0x55f9008dfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.691+0000 7ff3cbfff640 1 --2- 172.21.15.12:0/3101994901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f9008d9e60 0x55f9008dfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.692+0000 7ff3cbfff640 1 -- 172.21.15.12:0/3101994901 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9008e0f10 con 0x55f9008d0d60 2024-04-22T13:34:37.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.692+0000 7ff3d2d9d640 1 --2- 172.21.15.12:0/3101994901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f9008d9e60 0x55f9008dfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:37.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.692+0000 7ff3cbfff640 1 --2- 172.21.15.12:0/3101994901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9008d0d60 0x55f9008cc6d0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009a70 tx=0x7ff3bc009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5379dac0fc2b5b03 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.692+0000 7ff3bbfff640 1 -- 172.21.15.12:0/3101994901 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc021070 con 0x55f9008d0d60 2024-04-22T13:34:37.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.692+0000 7ff3bbfff640 1 -- 172.21.15.12:0/3101994901 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3bc012cb0 con 0x55f9008d0d60 2024-04-22T13:34:37.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.692+0000 7ff3bbfff640 1 -- 172.21.15.12:0/3101994901 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc00f450 con 0x55f9008d0d60 2024-04-22T13:34:37.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.692+0000 7ff3db966740 1 -- 172.21.15.12:0/3101994901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9008d0d60 msgr2=0x55f9008cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.692+0000 7ff3db966740 1 --2- 172.21.15.12:0/3101994901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9008d0d60 0x55f9008cc6d0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc009a70 tx=0x7ff3bc009f30 comp rx=0 tx=0).stop 2024-04-22T13:34:37.693 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.692+0000 7ff3db966740 1 -- 172.21.15.12:0/3101994901 shutdown_connections 2024-04-22T13:34:37.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.692+0000 7ff3db966740 1 --2- 172.21.15.12:0/3101994901 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f9008d9e60 0x55f9008dfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.692+0000 7ff3db966740 1 --2- 172.21.15.12:0/3101994901 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9008d0d60 0x55f9008cc6d0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.692+0000 7ff3db966740 1 --2- 172.21.15.12:0/3101994901 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9008d03b0 0x55f9008e9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.692+0000 7ff3db966740 1 -- 172.21.15.12:0/3101994901 >> 172.21.15.12:0/3101994901 conn(0x55f9008caba0 msgr2=0x55f9008cbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.693+0000 7ff3db966740 1 -- 172.21.15.12:0/3101994901 shutdown_connections 2024-04-22T13:34:37.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.693+0000 7ff3db966740 1 -- 172.21.15.12:0/3101994901 wait complete. 2024-04-22T13:34:37.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.693+0000 7ff3db966740 1 -- start start 2024-04-22T13:34:37.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.693+0000 7ff3db966740 1 Processor -- start 2024-04-22T13:34:37.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.693+0000 7ff3db966740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9008d03b0 0x55f9008e6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.693+0000 7ff3db966740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f9008d0d60 0x55f9008dee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.693+0000 7ff3db966740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9008d9e60 0x55f9008df380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.693+0000 7ff3db966740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d41acbc0 con 0x55f9008d9e60 2024-04-22T13:34:37.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.693+0000 7ff3db966740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d41ac4f0 con 0x55f9008d0d60 2024-04-22T13:34:37.694 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.693+0000 7ff3db966740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d41a9750 con 0x55f9008d03b0 2024-04-22T13:34:37.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.693+0000 7ff3db966740 10 client.0 _ll_register_callbacks cb 0x7ff3cc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:37.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.693+0000 7ff3d2d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9008d9e60 0x55f9008df380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.693+0000 7ff3d2d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9008d9e60 0x55f9008df380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49938/0 (socket says 172.21.15.12:49938) 2024-04-22T13:34:37.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.693+0000 7ff3d2d9d640 1 -- 172.21.15.12:0/2756027622 learned_addr learned my addr 172.21.15.12:0/2756027622 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:37.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.693+0000 7ff3d259c640 1 --2- 172.21.15.12:0/2756027622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9008d03b0 0x55f9008e6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.693+0000 7ff3cbfff640 1 --2- 172.21.15.12:0/2756027622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f9008d0d60 0x55f9008dee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.693+0000 7ff3d2d9d640 1 -- 172.21.15.12:0/2756027622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9008d03b0 msgr2=0x55f9008e6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.693+0000 7ff3d2d9d640 1 --2- 172.21.15.12:0/2756027622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9008d03b0 0x55f9008e6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.693+0000 7ff3d2d9d640 1 -- 172.21.15.12:0/2756027622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f9008d0d60 msgr2=0x55f9008dee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.693+0000 7ff3d2d9d640 1 --2- 172.21.15.12:0/2756027622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f9008d0d60 0x55f9008dee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.693+0000 7ff3d2d9d640 1 -- 172.21.15.12:0/2756027622 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9008dfac0 con 0x55f9008d9e60 2024-04-22T13:34:37.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.693+0000 7ff3cbfff640 1 --2- 172.21.15.12:0/2756027622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f9008d0d60 0x55f9008dee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:37.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.694+0000 7ff3d2d9d640 1 --2- 172.21.15.12:0/2756027622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9008d9e60 0x55f9008df380 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc02c6d0 tx=0x7ff3cc040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f675dfd40f9ffe39 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.695 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.694+0000 7ff3b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:34:37.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.694+0000 7ff3b9ffb640 1 -- 172.21.15.12:0/2756027622 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc05b070 con 0x55f9008d9e60 2024-04-22T13:34:37.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.694+0000 7ff3b9ffb640 1 -- 172.21.15.12:0/2756027622 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7ff3cc003d40 con 0x55f9008d9e60 2024-04-22T13:34:37.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.694+0000 7ff3db966740 1 -- 172.21.15.12:0/2756027622 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3bc012cb0 con 0x55f9008d9e60 2024-04-22T13:34:37.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.694+0000 7ff3b9ffb640 1 -- 172.21.15.12:0/2756027622 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff3cc06e050 con 0x55f9008d9e60 2024-04-22T13:34:37.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.694+0000 7ff3db966740 10 client.19161 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:37.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.694+0000 7ff3db966740 1 -- 172.21.15.12:0/2756027622 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f900953fb0 con 0x55f9008d9e60 2024-04-22T13:34:37.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.694+0000 7ff3b9ffb640 1 -- 172.21.15.12:0/2756027622 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3bc012cb0 con 0x55f9008d9e60 2024-04-22T13:34:37.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.694+0000 7ff3b9ffb640 1 -- 172.21.15.12:0/2756027622 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ff3cc04d610 con 0x55f9008d9e60 2024-04-22T13:34:37.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.694+0000 7ff3db966740 10 client.19161 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:37.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.694+0000 7ff3db966740 10 client.19161 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-04-22T13:34:37.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.694+0000 7ff3db966740 20 client.19161 populate_metadata read hostname 'smithi012' 2024-04-22T13:34:37.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.694+0000 7ff3db966740 1 --2- 172.21.15.12:0/2756027622 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55f9009546d0 0x55f900956af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.694+0000 7ff3db966740 4 client.19161 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:34:37.696 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.694+0000 7ff3db966740 1 -- 172.21.15.12:0/2756027622 --> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f9008ddea0 con 0x55f9009546d0 2024-04-22T13:34:37.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.695+0000 7ff3d259c640 1 --2- 172.21.15.12:0/2756027622 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55f9009546d0 0x55f900956af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.695+0000 7ff3d259c640 1 --2- 172.21.15.12:0/2756027622 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55f9009546d0 0x55f900956af0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7dda4bb895db57d1 server_cookie=d344b8613c3a99e2 in_seq=0 out_seq=0 2024-04-22T13:34:37.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.695+0000 7ff3b9ffb640 10 client.19161 ms_handle_connect on v2:172.21.15.111:6834/2665640170 2024-04-22T13:34:37.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.695+0000 7ff3b9ffb640 1 -- 172.21.15.12:0/2756027622 <== mds.3 v2:172.21.15.111:6834/2665640170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f9008ddea0 con 0x55f9009546d0 2024-04-22T13:34:37.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.695+0000 7ff3b9ffb640 10 client.19161 handle_command_reply: tid=0 2024-04-22T13:34:37.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.695+0000 7ff3db966740 1 client.19161 shutdown 2024-04-22T13:34:37.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.695+0000 7ff3db966740 1 -- 172.21.15.12:0/2756027622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9008d9e60 msgr2=0x55f9008df380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.695+0000 7ff3db966740 1 --2- 172.21.15.12:0/2756027622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9008d9e60 0x55f9008df380 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc02c6d0 tx=0x7ff3cc040000 comp rx=0 tx=0).stop 2024-04-22T13:34:37.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.695+0000 7ff3db966740 1 -- 172.21.15.12:0/2756027622 shutdown_connections 2024-04-22T13:34:37.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.695+0000 7ff3db966740 1 --2- 172.21.15.12:0/2756027622 >> [v2:172.21.15.111:6834/2665640170,v1:172.21.15.111:6835/2665640170] conn(0x55f9009546d0 0x55f900956af0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.695+0000 7ff3db966740 1 --2- 172.21.15.12:0/2756027622 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f9008d9e60 0x55f9008df380 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.695+0000 7ff3db966740 1 --2- 172.21.15.12:0/2756027622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f9008d0d60 0x55f9008dee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.697 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.695+0000 7ff3b9ffb640 0 client.19161 ms_handle_reset on v2:172.21.15.111:6834/2665640170 2024-04-22T13:34:37.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.695+0000 7ff3db966740 1 --2- 172.21.15.12:0/2756027622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f9008d03b0 0x55f9008e6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.695+0000 7ff3db966740 1 -- 172.21.15.12:0/2756027622 >> 172.21.15.12:0/2756027622 conn(0x55f9008caba0 msgr2=0x55f9008c9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.696+0000 7ff3db966740 1 -- 172.21.15.12:0/2756027622 shutdown_connections 2024-04-22T13:34:37.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.696+0000 7ff3db966740 1 -- 172.21.15.12:0/2756027622 wait complete. 2024-04-22T13:34:37.698 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.696+0000 7ff3db966740 20 client.19161 trim_cache size 0 max 16384 2024-04-22T13:34:37.698 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:34:37.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.696+0000 7ff3d8d75640 1 -- 172.21.15.12:0/4240182532 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff3c00a3da0 msgr2=0x7ff3c00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.696+0000 7ff3d8d75640 1 --2- 172.21.15.12:0/4240182532 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff3c00a3da0 0x7ff3c00a6260 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc04c000 tx=0x7ff3cc048000 comp rx=0 tx=0).stop 2024-04-22T13:34:37.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.696+0000 7ff3d8d75640 1 -- 172.21.15.12:0/4240182532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3d41010a0 msgr2=0x7ff3d4197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.696+0000 7ff3d8d75640 1 --2- 172.21.15.12:0/4240182532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3d41010a0 0x7ff3d4197580 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400c920 tx=0x7ff3c400cde0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.696+0000 7ff3d8d75640 1 -- 172.21.15.12:0/4240182532 shutdown_connections 2024-04-22T13:34:37.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.696+0000 7ff3d8d75640 1 --2- 172.21.15.12:0/4240182532 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ff3c00a3da0 0x7ff3c00a6260 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.696+0000 7ff3d8d75640 1 --2- 172.21.15.12:0/4240182532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff3d419be40 0x7ff3d419c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.696+0000 7ff3d8d75640 1 --2- 172.21.15.12:0/4240182532 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3d4106170 0x7ff3d4197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.699 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.696+0000 7ff3d8d75640 1 --2- 172.21.15.12:0/4240182532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff3d41010a0 0x7ff3d4197580 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.696+0000 7ff3d8d75640 1 -- 172.21.15.12:0/4240182532 >> 172.21.15.12:0/4240182532 conn(0x7ff3d4077eb0 msgr2=0x7ff3d40fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.696+0000 7ff3d8d75640 1 -- 172.21.15.12:0/4240182532 shutdown_connections 2024-04-22T13:34:37.700 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.696+0000 7ff3d8d75640 1 -- 172.21.15.12:0/4240182532 wait complete. 2024-04-22T13:34:37.701 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:37 smithi012 ceph-mon[28265]: pgmap v326: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:37.701 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:37 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1095992961' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:34:37.702 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-22T13:34:37.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.811+0000 7f32f9af1640 1 -- 172.21.15.12:0/2472330956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32f4107270 msgr2=0x7f32f410ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.811+0000 7f32f9af1640 1 --2- 172.21.15.12:0/2472330956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32f4107270 0x7f32f410ee40 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009ca0 tx=0x7f32ec02c0f0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.811+0000 7f32f9af1640 1 -- 172.21.15.12:0/2472330956 shutdown_connections 2024-04-22T13:34:37.808 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.811+0000 7f32f9af1640 1 --2- 172.21.15.12:0/2472330956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32f4107270 0x7f32f410ee40 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.811+0000 7f32f9af1640 1 --2- 172.21.15.12:0/2472330956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32f406b1f0 0x7f32f4106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.811+0000 7f32f9af1640 1 --2- 172.21.15.12:0/2472330956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32f406a840 0x7f32f406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.811+0000 7f32f9af1640 1 -- 172.21.15.12:0/2472330956 >> 172.21.15.12:0/2472330956 conn(0x7f32f4077eb0 msgr2=0x7f32f40782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.811+0000 7f32f9af1640 1 -- 172.21.15.12:0/2472330956 shutdown_connections 2024-04-22T13:34:37.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.811+0000 7f32f9af1640 1 -- 172.21.15.12:0/2472330956 wait complete. 2024-04-22T13:34:37.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.812+0000 7f32f9af1640 1 Processor -- start 2024-04-22T13:34:37.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.812+0000 7f32f9af1640 1 -- start start 2024-04-22T13:34:37.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.812+0000 7f32f9af1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32f406a840 0x7f32f41986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.812+0000 7f32f9af1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32f4107270 0x7f32f4198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.812+0000 7f32f9af1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32f419cf90 0x7f32f419d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.812+0000 7f32f9af1640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f32f41121d0 con 0x7f32f4107270 2024-04-22T13:34:37.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.812+0000 7f32f9af1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f32f4112050 con 0x7f32f406a840 2024-04-22T13:34:37.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.812+0000 7f32f9af1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f32f4112350 con 0x7f32f419cf90 2024-04-22T13:34:37.809 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.812+0000 7f32f2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32f4107270 0x7f32f4198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.812+0000 7f32f2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32f4107270 0x7f32f4198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49958/0 (socket says 172.21.15.12:49958) 2024-04-22T13:34:37.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.812+0000 7f32f2ffd640 1 -- 172.21.15.12:0/3580578169 learned_addr learned my addr 172.21.15.12:0/3580578169 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:37.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.812+0000 7f32f37fe640 1 --2- 172.21.15.12:0/3580578169 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32f406a840 0x7f32f41986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.812+0000 7f32f3fff640 1 --2- 172.21.15.12:0/3580578169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32f419cf90 0x7f32f419d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.812+0000 7f32f2ffd640 1 -- 172.21.15.12:0/3580578169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32f419cf90 msgr2=0x7f32f419d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.812+0000 7f32f2ffd640 1 --2- 172.21.15.12:0/3580578169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32f419cf90 0x7f32f419d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.812+0000 7f32f2ffd640 1 -- 172.21.15.12:0/3580578169 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32f406a840 msgr2=0x7f32f41986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.812+0000 7f32f2ffd640 1 --2- 172.21.15.12:0/3580578169 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32f406a840 0x7f32f41986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.812+0000 7f32f2ffd640 1 -- 172.21.15.12:0/3580578169 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f419da80 con 0x7f32f4107270 2024-04-22T13:34:37.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.813+0000 7f32f2ffd640 1 --2- 172.21.15.12:0/3580578169 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32f4107270 0x7f32f4198c10 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f32dc00d740 tx=0x7f32dc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.813+0000 7f32f0ff9640 1 -- 172.21.15.12:0/3580578169 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc017070 con 0x7f32f4107270 2024-04-22T13:34:37.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.813+0000 7f32f0ff9640 1 -- 172.21.15.12:0/3580578169 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32dc00de50 con 0x7f32f4107270 2024-04-22T13:34:37.810 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.813+0000 7f32f9af1640 1 -- 172.21.15.12:0/3580578169 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32f419dd70 con 0x7f32f4107270 2024-04-22T13:34:37.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.813+0000 7f32f0ff9640 1 -- 172.21.15.12:0/3580578169 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc004ce0 con 0x7f32f4107270 2024-04-22T13:34:37.811 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.813+0000 7f32f9af1640 1 -- 172.21.15.12:0/3580578169 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32f41a5510 con 0x7f32f4107270 2024-04-22T13:34:37.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.815+0000 7f32f0ff9640 1 -- 172.21.15.12:0/3580578169 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f32dc004e80 con 0x7f32f4107270 2024-04-22T13:34:37.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.815+0000 7f32fc7b6740 1 Processor -- start 2024-04-22T13:34:37.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.815+0000 7f32fc7b6740 1 -- start start 2024-04-22T13:34:37.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.815+0000 7f32fc7b6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c304d973b0 0x55c304d97790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.815+0000 7f32fc7b6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c304d97d60 0x55c304da0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.817 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.815+0000 7f32fc7b6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c304da0e60 0x55c304da12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.815+0000 7f32fc7b6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c304cfc1f0 con 0x55c304d973b0 2024-04-22T13:34:37.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.815+0000 7f32fc7b6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c304c994b0 con 0x55c304da0e60 2024-04-22T13:34:37.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.815+0000 7f32fc7b6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c304cf81b0 con 0x55c304d97d60 2024-04-22T13:34:37.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.815+0000 7f32f0ff9640 1 --2- 172.21.15.12:0/3580578169 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f32e00a3d20 0x7f32e00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.816+0000 7f32f3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c304d973b0 0x55c304d97790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.816+0000 7f32f3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c304d973b0 0x55c304d97790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49962/0 (socket says 172.21.15.12:49962) 2024-04-22T13:34:37.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.816+0000 7f32f3fff640 1 -- 172.21.15.12:0/4118644676 learned_addr learned my addr 172.21.15.12:0/4118644676 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:37.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.816+0000 7f32f3fff640 1 --2- 172.21.15.12:0/3580578169 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f32e00a3d20 0x7f32e00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.816+0000 7f32f2ffd640 1 --2- 172.21.15.12:0/4118644676 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c304da0e60 0x55c304da12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.816+0000 7f32f0ff9640 1 -- 172.21.15.12:0/3580578169 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f32dc0d00c0 con 0x7f32f4107270 2024-04-22T13:34:37.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.816+0000 7f32f37fe640 1 --2- 172.21.15.12:0/4118644676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c304d97d60 0x55c304da0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.816+0000 7f32f3fff640 1 -- 172.21.15.12:0/4118644676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c304d97d60 msgr2=0x55c304da0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.816+0000 7f32f3fff640 1 --2- 172.21.15.12:0/4118644676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c304d97d60 0x55c304da0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.818 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.816+0000 7f32f3fff640 1 -- 172.21.15.12:0/4118644676 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c304da0e60 msgr2=0x55c304da12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.816+0000 7f32f3fff640 1 --2- 172.21.15.12:0/4118644676 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c304da0e60 0x55c304da12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.816+0000 7f32f3fff640 1 -- 172.21.15.12:0/4118644676 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c304c8b1b0 con 0x55c304d973b0 2024-04-22T13:34:37.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.816+0000 7f32f3fff640 1 --2- 172.21.15.12:0/4118644676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c304d973b0 0x55c304d97790 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f32ec037040 tx=0x7f32ec0324c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd2917c0b534fb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.816+0000 7f32f3fff640 1 --2- 172.21.15.12:0/3580578169 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f32e00a3d20 0x7f32e00a61e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009ca0 tx=0x7f32ec042000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.816+0000 7f32e9ffb640 1 -- 172.21.15.12:0/4118644676 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec051070 con 0x55c304d973b0 2024-04-22T13:34:37.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.816+0000 7f32e9ffb640 1 -- 172.21.15.12:0/4118644676 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32ec032830 con 0x55c304d973b0 2024-04-22T13:34:37.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.816+0000 7f32e9ffb640 1 -- 172.21.15.12:0/4118644676 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec039040 con 0x55c304d973b0 2024-04-22T13:34:37.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.817+0000 7f32fc7b6740 1 -- 172.21.15.12:0/4118644676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c304d973b0 msgr2=0x55c304d97790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.817+0000 7f32fc7b6740 1 --2- 172.21.15.12:0/4118644676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c304d973b0 0x55c304d97790 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f32ec037040 tx=0x7f32ec0324c0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.817+0000 7f32fc7b6740 1 -- 172.21.15.12:0/4118644676 shutdown_connections 2024-04-22T13:34:37.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.817+0000 7f32fc7b6740 1 --2- 172.21.15.12:0/4118644676 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c304da0e60 0x55c304da12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.817+0000 7f32fc7b6740 1 --2- 172.21.15.12:0/4118644676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c304d97d60 0x55c304da0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.817+0000 7f32fc7b6740 1 --2- 172.21.15.12:0/4118644676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c304d973b0 0x55c304d97790 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.819 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.817+0000 7f32fc7b6740 1 -- 172.21.15.12:0/4118644676 >> 172.21.15.12:0/4118644676 conn(0x55c304d91ba0 msgr2=0x55c304d91f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.817+0000 7f32fc7b6740 1 -- 172.21.15.12:0/4118644676 shutdown_connections 2024-04-22T13:34:37.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.817+0000 7f32fc7b6740 1 -- 172.21.15.12:0/4118644676 wait complete. 2024-04-22T13:34:37.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.817+0000 7f32fc7b6740 1 -- start start 2024-04-22T13:34:37.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.818+0000 7f32fc7b6740 1 Processor -- start 2024-04-22T13:34:37.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.818+0000 7f32fc7b6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c304d973b0 0x55c304da7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.818+0000 7f32fc7b6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c304d97d60 0x55c304da4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.818+0000 7f32fc7b6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c304da0e60 0x55c304da4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.818+0000 7f32fc7b6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x55c304da7d40 con 0x55c304da0e60 2024-04-22T13:34:37.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.818+0000 7f32fc7b6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c304cb5500 con 0x55c304d97d60 2024-04-22T13:34:37.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.818+0000 7f32fc7b6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c304da7ec0 con 0x55c304d973b0 2024-04-22T13:34:37.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.818+0000 7f32fc7b6740 10 client.0 _ll_register_callbacks cb 0x55c304c83b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:37.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.818+0000 7f32f3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c304da0e60 0x55c304da4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.818+0000 7f32f3fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c304da0e60 0x55c304da4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49974/0 (socket says 172.21.15.12:49974) 2024-04-22T13:34:37.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.818+0000 7f32f37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c304d973b0 0x55c304da7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.820 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.818+0000 7f32f3fff640 1 -- 172.21.15.12:0/844856463 learned_addr learned my addr 172.21.15.12:0/844856463 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:37.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.818+0000 7f32f2ffd640 1 --2- 172.21.15.12:0/844856463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c304d97d60 0x55c304da4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.818+0000 7f32f3fff640 1 -- 172.21.15.12:0/844856463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c304d973b0 msgr2=0x55c304da7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.821 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.818+0000 7f32f3fff640 1 --2- 172.21.15.12:0/844856463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c304d973b0 0x55c304da7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.818+0000 7f32f3fff640 1 -- 172.21.15.12:0/844856463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c304d97d60 msgr2=0x55c304da4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.818+0000 7f32f3fff640 1 --2- 172.21.15.12:0/844856463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c304d97d60 0x55c304da4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.818+0000 7f32f3fff640 1 -- 172.21.15.12:0/844856463 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c304a0e3f0 con 0x55c304da0e60 2024-04-22T13:34:37.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.819+0000 7f32f3fff640 1 --2- 172.21.15.12:0/844856463 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c304da0e60 0x55c304da4b40 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f32ec02c7e0 tx=0x7f32ec004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90b6c1db5a41a35 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.819+0000 7f32d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:34:37.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.819+0000 7f32d2ffd640 1 -- 172.21.15.12:0/844856463 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec05a060 con 0x55c304da0e60 2024-04-22T13:34:37.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.819+0000 7f32d2ffd640 1 -- 172.21.15.12:0/844856463 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f32ec0043e0 con 0x55c304da0e60 2024-04-22T13:34:37.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.819+0000 7f32fc7b6740 1 -- 172.21.15.12:0/844856463 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32ec032830 con 0x55c304da0e60 2024-04-22T13:34:37.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.819+0000 7f32d2ffd640 1 -- 172.21.15.12:0/844856463 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f32ec039070 con 0x55c304da0e60 2024-04-22T13:34:37.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.819+0000 7f32fc7b6740 10 client.19173 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:37.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.819+0000 7f32fc7b6740 1 -- 172.21.15.12:0/844856463 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c304ad87d0 con 0x55c304da0e60 2024-04-22T13:34:37.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.819+0000 7f32d2ffd640 1 -- 172.21.15.12:0/844856463 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32ec032830 con 0x55c304da0e60 2024-04-22T13:34:37.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.819+0000 7f32d2ffd640 1 -- 172.21.15.12:0/844856463 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f32ec04b070 con 0x55c304da0e60 2024-04-22T13:34:37.822 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.819+0000 7f32fc7b6740 10 client.19173 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:37.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.819+0000 7f32fc7b6740 10 client.19173 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:34:37.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.819+0000 7f32fc7b6740 20 client.19173 populate_metadata read hostname 'smithi012' 2024-04-22T13:34:37.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.819+0000 7f32fc7b6740 1 --2- 172.21.15.12:0/844856463 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55c304db4e80 0x55c304db7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.819+0000 7f32fc7b6740 4 client.19173 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:34:37.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.819+0000 7f32fc7b6740 1 -- 172.21.15.12:0/844856463 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c304c9b930 con 0x55c304db4e80 2024-04-22T13:34:37.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.820+0000 7f32f37fe640 1 --2- 172.21.15.12:0/844856463 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55c304db4e80 0x55c304db7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.820+0000 7f32d2ffd640 10 client.19173 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:34:37.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.820+0000 7f32f37fe640 1 --2- 172.21.15.12:0/844856463 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55c304db4e80 0x55c304db7270 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=84d67fdea972259e server_cookie=109d0f9d2c65756 in_seq=0 out_seq=0 2024-04-22T13:34:37.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.821+0000 7f32d2ffd640 1 -- 172.21.15.12:0/844856463 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c304c9b930 con 0x55c304db4e80 2024-04-22T13:34:37.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.821+0000 7f32d2ffd640 10 client.19173 handle_command_reply: tid=0 2024-04-22T13:34:37.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.821+0000 7f32fc7b6740 1 client.19173 shutdown 2024-04-22T13:34:37.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.821+0000 7f32fc7b6740 1 -- 172.21.15.12:0/844856463 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c304da0e60 msgr2=0x55c304da4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.821+0000 7f32fc7b6740 1 --2- 172.21.15.12:0/844856463 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c304da0e60 0x55c304da4b40 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f32ec02c7e0 tx=0x7f32ec004f90 comp rx=0 tx=0).stop 2024-04-22T13:34:37.823 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.821+0000 7f32fc7b6740 1 -- 172.21.15.12:0/844856463 shutdown_connections 2024-04-22T13:34:37.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.821+0000 7f32fc7b6740 1 --2- 172.21.15.12:0/844856463 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55c304db4e80 0x55c304db7270 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.821+0000 7f32fc7b6740 1 --2- 172.21.15.12:0/844856463 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c304da0e60 0x55c304da4b40 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.821+0000 7f32fc7b6740 1 --2- 172.21.15.12:0/844856463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c304d97d60 0x55c304da4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.821+0000 7f32fc7b6740 1 --2- 172.21.15.12:0/844856463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c304d973b0 0x55c304da7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.821+0000 7f32d2ffd640 0 client.19173 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:34:37.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.821+0000 7f32fc7b6740 1 -- 172.21.15.12:0/844856463 >> 172.21.15.12:0/844856463 conn(0x55c304d91ba0 msgr2=0x55c304d938f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.821+0000 7f32fc7b6740 1 -- 172.21.15.12:0/844856463 shutdown_connections 2024-04-22T13:34:37.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.821+0000 7f32fc7b6740 1 -- 172.21.15.12:0/844856463 wait complete. 2024-04-22T13:34:37.824 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.821+0000 7f32fc7b6740 20 client.19173 trim_cache size 0 max 16384 2024-04-22T13:34:37.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.834+0000 7f32fc7b6740 1 Processor -- start 2024-04-22T13:34:37.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.834+0000 7f32fc7b6740 1 -- start start 2024-04-22T13:34:37.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.835+0000 7f32fc7b6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c304d973b0 0x55c304da2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.835+0000 7f32fc7b6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c304d97d60 0x55c304db0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.835+0000 7f32fc7b6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c304da0e60 0x55c304d93a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.835+0000 7f32fc7b6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f32f41ab930 con 0x55c304d973b0 2024-04-22T13:34:37.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.835+0000 7f32fc7b6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f32f41ad470 con 0x55c304da0e60 2024-04-22T13:34:37.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.835+0000 7f32fc7b6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f32f41ac000 con 0x55c304d97d60 2024-04-22T13:34:37.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.835+0000 7f32f37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c304d973b0 0x55c304da2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.835+0000 7f32f37fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c304d973b0 0x55c304da2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49984/0 (socket says 172.21.15.12:49984) 2024-04-22T13:34:37.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.835+0000 7f32f37fe640 1 -- 172.21.15.12:0/334244148 learned_addr learned my addr 172.21.15.12:0/334244148 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:37.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.835+0000 7f32f2ffd640 1 --2- 172.21.15.12:0/334244148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c304d97d60 0x55c304db0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.835+0000 7f32f37fe640 1 -- 172.21.15.12:0/334244148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c304d97d60 msgr2=0x55c304db0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.835+0000 7f32f37fe640 1 --2- 172.21.15.12:0/334244148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c304d97d60 0x55c304db0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.835+0000 7f32f37fe640 1 -- 172.21.15.12:0/334244148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c304da0e60 msgr2=0x55c304d93a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.835+0000 7f32f37fe640 1 --2- 172.21.15.12:0/334244148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c304da0e60 0x55c304d93a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.835+0000 7f32f37fe640 1 -- 172.21.15.12:0/334244148 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c304d981d0 con 0x55c304d973b0 2024-04-22T13:34:37.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.835+0000 7f32f2ffd640 1 --2- 172.21.15.12:0/334244148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c304d97d60 0x55c304db0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:37.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.835+0000 7f32f37fe640 1 --2- 172.21.15.12:0/334244148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c304d973b0 0x55c304da2250 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f32e40045f0 tx=0x7f32e400bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4575a1274578d81e server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.835+0000 7f32e97fa640 1 -- 172.21.15.12:0/334244148 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e4003ea0 con 0x55c304d973b0 2024-04-22T13:34:37.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.835+0000 7f32e97fa640 1 -- 172.21.15.12:0/334244148 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32e401d8e0 con 0x55c304d973b0 2024-04-22T13:34:37.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.835+0000 7f32e97fa640 1 -- 172.21.15.12:0/334244148 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e4010430 con 0x55c304d973b0 2024-04-22T13:34:37.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.836+0000 7f32fc7b6740 1 -- 172.21.15.12:0/334244148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c304d973b0 msgr2=0x55c304da2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.836+0000 7f32fc7b6740 1 --2- 172.21.15.12:0/334244148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c304d973b0 0x55c304da2250 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f32e40045f0 tx=0x7f32e400bf50 comp rx=0 tx=0).stop 2024-04-22T13:34:37.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.836+0000 7f32fc7b6740 1 -- 172.21.15.12:0/334244148 shutdown_connections 2024-04-22T13:34:37.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.836+0000 7f32fc7b6740 1 --2- 172.21.15.12:0/334244148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c304da0e60 0x55c304d93a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.836+0000 7f32fc7b6740 1 --2- 172.21.15.12:0/334244148 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c304d97d60 0x55c304db0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.836+0000 7f32fc7b6740 1 --2- 172.21.15.12:0/334244148 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c304d973b0 0x55c304da2250 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.836+0000 7f32fc7b6740 1 -- 172.21.15.12:0/334244148 >> 172.21.15.12:0/334244148 conn(0x55c304d91ba0 msgr2=0x55c304d90c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.836+0000 7f32fc7b6740 1 -- 172.21.15.12:0/334244148 shutdown_connections 2024-04-22T13:34:37.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.836+0000 7f32fc7b6740 1 -- 172.21.15.12:0/334244148 wait complete. 2024-04-22T13:34:37.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.836+0000 7f32fc7b6740 1 -- start start 2024-04-22T13:34:37.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.836+0000 7f32fc7b6740 1 Processor -- start 2024-04-22T13:34:37.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.837+0000 7f32fc7b6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c304d973b0 0x55c304da6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.838 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.837+0000 7f32fc7b6740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c304d97d60 0x55c304da6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.837+0000 7f32fc7b6740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c304da0e60 0x55c304e1a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.837+0000 7f32fc7b6740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f32f41acda0 con 0x55c304d97d60 2024-04-22T13:34:37.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.837+0000 7f32fc7b6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f32f41ac6d0 con 0x55c304d973b0 2024-04-22T13:34:37.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.837+0000 7f32fc7b6740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f32f41a9930 con 0x55c304da0e60 2024-04-22T13:34:37.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.837+0000 7f32fc7b6740 10 client.0 _ll_register_callbacks cb 0x7f32ec032e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:37.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.837+0000 7f32f2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c304d97d60 0x55c304da6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.837+0000 7f32f2ffd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c304d97d60 0x55c304da6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:49998/0 (socket says 172.21.15.12:49998) 2024-04-22T13:34:37.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.837+0000 7f32f2ffd640 1 -- 172.21.15.12:0/3452536172 learned_addr learned my addr 172.21.15.12:0/3452536172 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:37.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.837+0000 7f32f37fe640 1 --2- 172.21.15.12:0/3452536172 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c304d973b0 0x55c304da6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.837+0000 7f32f2ffd640 1 -- 172.21.15.12:0/3452536172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c304da0e60 msgr2=0x55c304e1a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.837+0000 7f32f2ffd640 1 --2- 172.21.15.12:0/3452536172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c304da0e60 0x55c304e1a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.837+0000 7f32f2ffd640 1 -- 172.21.15.12:0/3452536172 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c304d973b0 msgr2=0x55c304da6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.837+0000 7f32f2ffd640 1 --2- 172.21.15.12:0/3452536172 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c304d973b0 0x55c304da6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.837+0000 7f32f2ffd640 1 -- 172.21.15.12:0/3452536172 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c304e1a5c0 con 0x55c304d97d60 2024-04-22T13:34:37.839 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.837+0000 7f32f2ffd640 1 --2- 172.21.15.12:0/3452536172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c304d97d60 0x55c304da6b40 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f32dc005d80 tx=0x7f32dc018ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e5d88f3243c929 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.837+0000 7f32d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:34:37.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.837+0000 7f32d2ffd640 1 -- 172.21.15.12:0/3452536172 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc09d110 con 0x55c304d97d60 2024-04-22T13:34:37.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.838+0000 7f32d2ffd640 1 -- 172.21.15.12:0/3452536172 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f32dc009c30 con 0x55c304d97d60 2024-04-22T13:34:37.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.838+0000 7f32fc7b6740 1 -- 172.21.15.12:0/3452536172 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32e401d8e0 con 0x55c304d97d60 2024-04-22T13:34:37.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.838+0000 7f32d2ffd640 1 -- 172.21.15.12:0/3452536172 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f32dc09c0d0 con 0x55c304d97d60 2024-04-22T13:34:37.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.838+0000 7f32fc7b6740 10 client.19179 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:37.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.838+0000 7f32fc7b6740 1 -- 172.21.15.12:0/3452536172 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c304e1b6d0 con 0x55c304d97d60 2024-04-22T13:34:37.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.838+0000 7f32d2ffd640 1 -- 172.21.15.12:0/3452536172 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32e401d8e0 con 0x55c304d97d60 2024-04-22T13:34:37.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.838+0000 7f32d2ffd640 1 -- 172.21.15.12:0/3452536172 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f32dc0cda50 con 0x55c304d97d60 2024-04-22T13:34:37.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.838+0000 7f32fc7b6740 10 client.19179 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:37.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.838+0000 7f32fc7b6740 10 client.19179 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-04-22T13:34:37.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.838+0000 7f32fc7b6740 20 client.19179 populate_metadata read hostname 'smithi012' 2024-04-22T13:34:37.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.838+0000 7f32fc7b6740 1 --2- 172.21.15.12:0/3452536172 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55c304e1be80 0x55c304e1e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.840 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.838+0000 7f32fc7b6740 4 client.19179 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:34:37.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.838+0000 7f32fc7b6740 1 -- 172.21.15.12:0/3452536172 --> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c304d09c10 con 0x55c304e1be80 2024-04-22T13:34:37.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.838+0000 7f32f37fe640 1 --2- 172.21.15.12:0/3452536172 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55c304e1be80 0x55c304e1e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.839+0000 7f32d2ffd640 10 client.19179 ms_handle_connect on v2:172.21.15.134:6834/3927126579 2024-04-22T13:34:37.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.839+0000 7f32f37fe640 1 --2- 172.21.15.12:0/3452536172 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55c304e1be80 0x55c304e1e2a0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b1c80c92c2459f0b server_cookie=1d38f6141863ff0e in_seq=0 out_seq=0 2024-04-22T13:34:37.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.839+0000 7f32d2ffd640 1 -- 172.21.15.12:0/3452536172 <== mds.2 v2:172.21.15.134:6834/3927126579 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c304d09c10 con 0x55c304e1be80 2024-04-22T13:34:37.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.839+0000 7f32d2ffd640 10 client.19179 handle_command_reply: tid=0 2024-04-22T13:34:37.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.839+0000 7f32fc7b6740 1 client.19179 shutdown 2024-04-22T13:34:37.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.839+0000 7f32fc7b6740 1 -- 172.21.15.12:0/3452536172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c304d97d60 msgr2=0x55c304da6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.839+0000 7f32fc7b6740 1 --2- 172.21.15.12:0/3452536172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c304d97d60 0x55c304da6b40 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f32dc005d80 tx=0x7f32dc018ab0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.839+0000 7f32fc7b6740 1 -- 172.21.15.12:0/3452536172 shutdown_connections 2024-04-22T13:34:37.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.839+0000 7f32fc7b6740 1 --2- 172.21.15.12:0/3452536172 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c304da0e60 0x55c304e1a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.839+0000 7f32fc7b6740 1 --2- 172.21.15.12:0/3452536172 >> [v2:172.21.15.134:6834/3927126579,v1:172.21.15.134:6835/3927126579] conn(0x55c304e1be80 0x55c304e1e2a0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.839+0000 7f32fc7b6740 1 --2- 172.21.15.12:0/3452536172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c304d97d60 0x55c304da6b40 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.839+0000 7f32fc7b6740 1 --2- 172.21.15.12:0/3452536172 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c304d973b0 0x55c304da6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.841 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.839+0000 7f32fc7b6740 1 -- 172.21.15.12:0/3452536172 >> 172.21.15.12:0/3452536172 conn(0x55c304d91ba0 msgr2=0x55c304da3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.839+0000 7f32d2ffd640 0 client.19179 ms_handle_reset on v2:172.21.15.134:6834/3927126579 2024-04-22T13:34:37.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.839+0000 7f32fc7b6740 1 -- 172.21.15.12:0/3452536172 shutdown_connections 2024-04-22T13:34:37.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.839+0000 7f32fc7b6740 1 -- 172.21.15.12:0/3452536172 wait complete. 2024-04-22T13:34:37.842 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.839+0000 7f32fc7b6740 20 client.19179 trim_cache size 0 max 16384 2024-04-22T13:34:37.842 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:34:37.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.840+0000 7f32f9af1640 1 -- 172.21.15.12:0/3580578169 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f32e00a3d20 msgr2=0x7f32e00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.840+0000 7f32f9af1640 1 --2- 172.21.15.12:0/3580578169 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f32e00a3d20 0x7f32e00a61e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009ca0 tx=0x7f32ec042000 comp rx=0 tx=0).stop 2024-04-22T13:34:37.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.840+0000 7f32f9af1640 1 -- 172.21.15.12:0/3580578169 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32f4107270 msgr2=0x7f32f4198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.840+0000 7f32f9af1640 1 --2- 172.21.15.12:0/3580578169 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32f4107270 0x7f32f4198c10 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f32dc00d740 tx=0x7f32dc00dc00 comp rx=0 tx=0).stop 2024-04-22T13:34:37.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.840+0000 7f32f9af1640 1 -- 172.21.15.12:0/3580578169 shutdown_connections 2024-04-22T13:34:37.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.840+0000 7f32f9af1640 1 --2- 172.21.15.12:0/3580578169 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f32e00a3d20 0x7f32e00a61e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.840+0000 7f32f9af1640 1 --2- 172.21.15.12:0/3580578169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32f419cf90 0x7f32f419d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.840+0000 7f32f9af1640 1 --2- 172.21.15.12:0/3580578169 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f32f4107270 0x7f32f4198c10 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.840+0000 7f32f9af1640 1 --2- 172.21.15.12:0/3580578169 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32f406a840 0x7f32f41986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.843 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.840+0000 7f32f9af1640 1 -- 172.21.15.12:0/3580578169 >> 172.21.15.12:0/3580578169 conn(0x7f32f4077eb0 msgr2=0x7f32f4100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.840+0000 7f32f9af1640 1 -- 172.21.15.12:0/3580578169 shutdown_connections 2024-04-22T13:34:37.844 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.840+0000 7f32f9af1640 1 -- 172.21.15.12:0/3580578169 wait complete. 2024-04-22T13:34:37.846 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-22T13:34:37.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:37 smithi134 ceph-mon[31338]: pgmap v326: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:37.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:37 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1095992961' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:34:37.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.955+0000 7fb98f146640 1 -- 172.21.15.12:0/1110907619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb988102b40 msgr2=0x7fb988106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.955+0000 7fb98f146640 1 --2- 172.21.15.12:0/1110907619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb988102b40 0x7fb988106b80 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fb974009900 tx=0x7fb97402bc30 comp rx=0 tx=0).stop 2024-04-22T13:34:37.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.955+0000 7fb98f146640 1 -- 172.21.15.12:0/1110907619 shutdown_connections 2024-04-22T13:34:37.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.955+0000 7fb98f146640 1 --2- 172.21.15.12:0/1110907619 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb988107260 0x7fb98810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.955+0000 7fb98f146640 1 --2- 172.21.15.12:0/1110907619 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb988102b40 0x7fb988106b80 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.955+0000 7fb98f146640 1 --2- 172.21.15.12:0/1110907619 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb988102190 0x7fb988102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.955+0000 7fb98f146640 1 -- 172.21.15.12:0/1110907619 >> 172.21.15.12:0/1110907619 conn(0x7fb9880fd990 msgr2=0x7fb9880ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.955+0000 7fb98f146640 1 -- 172.21.15.12:0/1110907619 shutdown_connections 2024-04-22T13:34:37.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.955+0000 7fb98f146640 1 -- 172.21.15.12:0/1110907619 wait complete. 2024-04-22T13:34:37.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.956+0000 7fb98f146640 1 Processor -- start 2024-04-22T13:34:37.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.956+0000 7fb98f146640 1 -- start start 2024-04-22T13:34:37.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.956+0000 7fb98f146640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb988102190 0x7fb988198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.956+0000 7fb98f146640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb988107260 0x7fb988198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.956+0000 7fb98f146640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb98819d020 0x7fb98819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.956+0000 7fb98f146640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb988112180 con 0x7fb988107260 2024-04-22T13:34:37.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.956+0000 7fb98f146640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb988112000 con 0x7fb98819d020 2024-04-22T13:34:37.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.956+0000 7fb98f146640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb988112300 con 0x7fb988102190 2024-04-22T13:34:37.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.956+0000 7fb987fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb988107260 0x7fb988198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.956+0000 7fb987fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb988107260 0x7fb988198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50016/0 (socket says 172.21.15.12:50016) 2024-04-22T13:34:37.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.956+0000 7fb987fff640 1 -- 172.21.15.12:0/3924223561 learned_addr learned my addr 172.21.15.12:0/3924223561 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:37.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.956+0000 7fb98cee2640 1 --2- 172.21.15.12:0/3924223561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb988102190 0x7fb988198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.956+0000 7fb98d6e3640 1 --2- 172.21.15.12:0/3924223561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb98819d020 0x7fb98819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.956+0000 7fb987fff640 1 -- 172.21.15.12:0/3924223561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb988102190 msgr2=0x7fb988198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.956+0000 7fb987fff640 1 --2- 172.21.15.12:0/3924223561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb988102190 0x7fb988198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.956+0000 7fb987fff640 1 -- 172.21.15.12:0/3924223561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb98819d020 msgr2=0x7fb98819d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.956+0000 7fb987fff640 1 --2- 172.21.15.12:0/3924223561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb98819d020 0x7fb98819d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.956+0000 7fb987fff640 1 -- 172.21.15.12:0/3924223561 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98819db10 con 0x7fb988107260 2024-04-22T13:34:37.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.956+0000 7fb98cee2640 1 --2- 172.21.15.12:0/3924223561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb988102190 0x7fb988198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:37.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.957+0000 7fb987fff640 1 --2- 172.21.15.12:0/3924223561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb988107260 0x7fb988198ca0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fb974036040 tx=0x7fb97402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.957+0000 7fb985ffb640 1 -- 172.21.15.12:0/3924223561 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb974043020 con 0x7fb988107260 2024-04-22T13:34:37.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.957+0000 7fb985ffb640 1 -- 172.21.15.12:0/3924223561 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb97402cbe0 con 0x7fb988107260 2024-04-22T13:34:37.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.957+0000 7fb985ffb640 1 -- 172.21.15.12:0/3924223561 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb974003bf0 con 0x7fb988107260 2024-04-22T13:34:37.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.957+0000 7fb98f146640 1 -- 172.21.15.12:0/3924223561 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb98819dda0 con 0x7fb988107260 2024-04-22T13:34:37.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.957+0000 7fb98f146640 1 -- 172.21.15.12:0/3924223561 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9881a5570 con 0x7fb988107260 2024-04-22T13:34:37.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.960+0000 7fb985ffb640 1 -- 172.21.15.12:0/3924223561 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb974046020 con 0x7fb988107260 2024-04-22T13:34:37.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.960+0000 7fb991e0c740 1 Processor -- start 2024-04-22T13:34:37.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.960+0000 7fb991e0c740 1 -- start start 2024-04-22T13:34:37.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.960+0000 7fb991e0c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621549824d0 0x5621549828b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.960+0000 7fb991e0c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562154982e80 0x56215498b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.960+0000 7fb991e0c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56215498ba00 0x56215498de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.960+0000 7fb991e0c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x5621548e71f0 con 0x56215498ba00 2024-04-22T13:34:37.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.960+0000 7fb991e0c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5621548844b0 con 0x562154982e80 2024-04-22T13:34:37.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.960+0000 7fb991e0c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5621548e31b0 con 0x5621549824d0 2024-04-22T13:34:37.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.960+0000 7fb987fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56215498ba00 0x56215498de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.960+0000 7fb987fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56215498ba00 0x56215498de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50024/0 (socket says 172.21.15.12:50024) 2024-04-22T13:34:37.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.960+0000 7fb987fff640 1 -- 172.21.15.12:0/943183964 learned_addr learned my addr 172.21.15.12:0/943183964 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:37.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.960+0000 7fb98cee2640 1 --2- 172.21.15.12:0/943183964 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562154982e80 0x56215498b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.960+0000 7fb98d6e3640 1 --2- 172.21.15.12:0/943183964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621549824d0 0x5621549828b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.960+0000 7fb985ffb640 1 --2- 172.21.15.12:0/3924223561 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb9780a3da0 0x7fb9780a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.960+0000 7fb98d6e3640 1 --2- 172.21.15.12:0/3924223561 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb9780a3da0 0x7fb9780a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.961+0000 7fb98cee2640 1 -- 172.21.15.12:0/943183964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621549824d0 msgr2=0x5621549828b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.961+0000 7fb98cee2640 1 --2- 172.21.15.12:0/943183964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621549824d0 0x5621549828b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.961+0000 7fb98cee2640 1 -- 172.21.15.12:0/943183964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56215498ba00 msgr2=0x56215498de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.961+0000 7fb98cee2640 1 --2- 172.21.15.12:0/943183964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56215498ba00 0x56215498de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.961+0000 7fb98cee2640 1 -- 172.21.15.12:0/943183964 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621548761b0 con 0x562154982e80 2024-04-22T13:34:37.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.961+0000 7fb985ffb640 1 -- 172.21.15.12:0/3924223561 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7fb9740f2990 con 0x7fb988107260 2024-04-22T13:34:37.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.961+0000 7fb98cee2640 1 --2- 172.21.15.12:0/943183964 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562154982e80 0x56215498b430 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00cbd0 tx=0x7fb97c00e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d24d55fa5d7e86d server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.961+0000 7fb98d6e3640 1 --2- 172.21.15.12:0/3924223561 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb9780a3da0 0x7fb9780a6260 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb980005fd0 tx=0x7fb980004ab0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.961+0000 7fb972ffd640 1 -- 172.21.15.12:0/943183964 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c005220 con 0x562154982e80 2024-04-22T13:34:37.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.961+0000 7fb972ffd640 1 -- 172.21.15.12:0/943183964 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb97c004c50 con 0x562154982e80 2024-04-22T13:34:37.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.961+0000 7fb972ffd640 1 -- 172.21.15.12:0/943183964 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c010720 con 0x562154982e80 2024-04-22T13:34:37.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.962+0000 7fb991e0c740 1 -- 172.21.15.12:0/943183964 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562154982e80 msgr2=0x56215498b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.962+0000 7fb991e0c740 1 --2- 172.21.15.12:0/943183964 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562154982e80 0x56215498b430 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00cbd0 tx=0x7fb97c00e720 comp rx=0 tx=0).stop 2024-04-22T13:34:37.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.962+0000 7fb991e0c740 1 -- 172.21.15.12:0/943183964 shutdown_connections 2024-04-22T13:34:37.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.962+0000 7fb991e0c740 1 --2- 172.21.15.12:0/943183964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56215498ba00 0x56215498de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.962+0000 7fb991e0c740 1 --2- 172.21.15.12:0/943183964 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562154982e80 0x56215498b430 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.962+0000 7fb991e0c740 1 --2- 172.21.15.12:0/943183964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621549824d0 0x5621549828b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.962+0000 7fb991e0c740 1 -- 172.21.15.12:0/943183964 >> 172.21.15.12:0/943183964 conn(0x56215497cba0 msgr2=0x56215497cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.962+0000 7fb991e0c740 1 -- 172.21.15.12:0/943183964 shutdown_connections 2024-04-22T13:34:37.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.962+0000 7fb991e0c740 1 -- 172.21.15.12:0/943183964 wait complete. 2024-04-22T13:34:37.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.962+0000 7fb991e0c740 1 -- start start 2024-04-22T13:34:37.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.962+0000 7fb991e0c740 1 Processor -- start 2024-04-22T13:34:37.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.963+0000 7fb991e0c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5621549824d0 0x56215498b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.963+0000 7fb991e0c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562154982e80 0x5621549885f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.963+0000 7fb991e0c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56215498ba00 0x562154988b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.963+0000 7fb991e0c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x56215498e970 con 0x56215498ba00 2024-04-22T13:34:37.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.963+0000 7fb991e0c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5621548a0500 con 0x5621549824d0 2024-04-22T13:34:37.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.963+0000 7fb991e0c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56215498eaf0 con 0x562154982e80 2024-04-22T13:34:37.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.963+0000 7fb991e0c740 10 client.0 _ll_register_callbacks cb 0x56215486eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:37.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.963+0000 7fb98cee2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5621549824d0 0x56215498b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.963+0000 7fb98d6e3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56215498ba00 0x562154988b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.963+0000 7fb98cee2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5621549824d0 0x56215498b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.12:55398/0 (socket says 172.21.15.12:55398) 2024-04-22T13:34:37.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.963+0000 7fb98cee2640 1 -- 172.21.15.12:0/2591928425 learned_addr learned my addr 172.21.15.12:0/2591928425 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:37.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.963+0000 7fb987fff640 1 --2- 172.21.15.12:0/2591928425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562154982e80 0x5621549885f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.963+0000 7fb98cee2640 1 -- 172.21.15.12:0/2591928425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562154982e80 msgr2=0x5621549885f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.963+0000 7fb98cee2640 1 --2- 172.21.15.12:0/2591928425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562154982e80 0x5621549885f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.963+0000 7fb98cee2640 1 -- 172.21.15.12:0/2591928425 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56215498ba00 msgr2=0x562154988b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.963+0000 7fb98cee2640 1 --2- 172.21.15.12:0/2591928425 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56215498ba00 0x562154988b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.963+0000 7fb98cee2640 1 -- 172.21.15.12:0/2591928425 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621545f93f0 con 0x5621549824d0 2024-04-22T13:34:37.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.963+0000 7fb98d6e3640 1 --2- 172.21.15.12:0/2591928425 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56215498ba00 0x562154988b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:34:37.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.963+0000 7fb98cee2640 1 --2- 172.21.15.12:0/2591928425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5621549824d0 0x56215498b260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb97c018000 tx=0x7fb97c00c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5b6a0f83b2d1d0a server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.963+0000 7fb967fff640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-04-22T13:34:37.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.963+0000 7fb967fff640 1 -- 172.21.15.12:0/2591928425 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c02a060 con 0x5621549824d0 2024-04-22T13:34:37.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.964+0000 7fb991e0c740 1 -- 172.21.15.12:0/2591928425 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb97c004c50 con 0x5621549824d0 2024-04-22T13:34:37.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.964+0000 7fb967fff640 1 -- 172.21.15.12:0/2591928425 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7fb97c005010 con 0x5621549824d0 2024-04-22T13:34:37.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.964+0000 7fb967fff640 1 -- 172.21.15.12:0/2591928425 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb97c005210 con 0x5621549824d0 2024-04-22T13:34:37.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.964+0000 7fb991e0c740 10 client.27872 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:37.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.964+0000 7fb991e0c740 1 -- 172.21.15.12:0/2591928425 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5621546c37d0 con 0x5621549824d0 2024-04-22T13:34:37.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.964+0000 7fb967fff640 1 -- 172.21.15.12:0/2591928425 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb97c004c50 con 0x5621549824d0 2024-04-22T13:34:37.964 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.964+0000 7fb967fff640 1 -- 172.21.15.12:0/2591928425 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb97c01a460 con 0x5621549824d0 2024-04-22T13:34:37.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.964+0000 7fb991e0c740 10 client.27872 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:37.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.964+0000 7fb991e0c740 10 client.27872 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:34:37.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.964+0000 7fb991e0c740 20 client.27872 populate_metadata read hostname 'smithi012' 2024-04-22T13:34:37.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.964+0000 7fb991e0c740 1 --2- 172.21.15.12:0/2591928425 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56215499bb40 0x56215499df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.964+0000 7fb991e0c740 4 client.27872 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:34:37.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.964+0000 7fb991e0c740 1 -- 172.21.15.12:0/2591928425 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562154886930 con 0x56215499bb40 2024-04-22T13:34:37.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.965+0000 7fb987fff640 1 --2- 172.21.15.12:0/2591928425 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56215499bb40 0x56215499df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.965+0000 7fb967fff640 10 client.27872 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:34:37.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.965+0000 7fb987fff640 1 --2- 172.21.15.12:0/2591928425 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56215499bb40 0x56215499df30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=205420f7653090db server_cookie=6fa609003c9b54c4 in_seq=0 out_seq=0 2024-04-22T13:34:37.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.966+0000 7fb967fff640 1 -- 172.21.15.12:0/2591928425 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562154886930 con 0x56215499bb40 2024-04-22T13:34:37.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.966+0000 7fb967fff640 10 client.27872 handle_command_reply: tid=0 2024-04-22T13:34:37.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.966+0000 7fb991e0c740 1 client.27872 shutdown 2024-04-22T13:34:37.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.966+0000 7fb991e0c740 1 -- 172.21.15.12:0/2591928425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5621549824d0 msgr2=0x56215498b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.966+0000 7fb991e0c740 1 --2- 172.21.15.12:0/2591928425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5621549824d0 0x56215498b260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb97c018000 tx=0x7fb97c00c940 comp rx=0 tx=0).stop 2024-04-22T13:34:37.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.966+0000 7fb991e0c740 1 -- 172.21.15.12:0/2591928425 shutdown_connections 2024-04-22T13:34:37.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.966+0000 7fb991e0c740 1 --2- 172.21.15.12:0/2591928425 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x56215499bb40 0x56215499df30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.966+0000 7fb991e0c740 1 --2- 172.21.15.12:0/2591928425 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56215498ba00 0x562154988b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.966+0000 7fb991e0c740 1 --2- 172.21.15.12:0/2591928425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562154982e80 0x5621549885f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.966+0000 7fb991e0c740 1 --2- 172.21.15.12:0/2591928425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5621549824d0 0x56215498b260 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.966+0000 7fb967fff640 0 client.27872 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:34:37.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.966+0000 7fb991e0c740 1 -- 172.21.15.12:0/2591928425 >> 172.21.15.12:0/2591928425 conn(0x56215497cba0 msgr2=0x56215497e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.966+0000 7fb991e0c740 1 -- 172.21.15.12:0/2591928425 shutdown_connections 2024-04-22T13:34:37.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.966+0000 7fb991e0c740 1 -- 172.21.15.12:0/2591928425 wait complete. 2024-04-22T13:34:37.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.966+0000 7fb991e0c740 20 client.27872 trim_cache size 0 max 16384 2024-04-22T13:34:37.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.979+0000 7fb991e0c740 1 Processor -- start 2024-04-22T13:34:37.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.979+0000 7fb991e0c740 1 -- start start 2024-04-22T13:34:37.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.980+0000 7fb991e0c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621549824d0 0x562154997860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.980+0000 7fb991e0c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562154982e80 0x562154987880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.980+0000 7fb991e0c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56215498ba00 0x56215498ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.980+0000 7fb991e0c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb9881adb50 con 0x56215498ba00 2024-04-22T13:34:37.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.980+0000 7fb991e0c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb9881af690 con 0x562154982e80 2024-04-22T13:34:37.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.980+0000 7fb991e0c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb9881ae220 con 0x5621549824d0 2024-04-22T13:34:37.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.980+0000 7fb98d6e3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56215498ba00 0x56215498ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.980+0000 7fb98d6e3640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56215498ba00 0x56215498ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50042/0 (socket says 172.21.15.12:50042) 2024-04-22T13:34:37.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.980+0000 7fb98d6e3640 1 -- 172.21.15.12:0/1215081228 learned_addr learned my addr 172.21.15.12:0/1215081228 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:37.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.980+0000 7fb987fff640 1 --2- 172.21.15.12:0/1215081228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562154982e80 0x562154987880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.980+0000 7fb98cee2640 1 --2- 172.21.15.12:0/1215081228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621549824d0 0x562154997860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.980+0000 7fb98d6e3640 1 -- 172.21.15.12:0/1215081228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621549824d0 msgr2=0x562154997860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.980+0000 7fb98d6e3640 1 --2- 172.21.15.12:0/1215081228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621549824d0 0x562154997860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.980+0000 7fb98d6e3640 1 -- 172.21.15.12:0/1215081228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562154982e80 msgr2=0x562154987880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.980+0000 7fb98d6e3640 1 --2- 172.21.15.12:0/1215081228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562154982e80 0x562154987880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.980+0000 7fb98d6e3640 1 -- 172.21.15.12:0/1215081228 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56215497c9d0 con 0x56215498ba00 2024-04-22T13:34:37.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.980+0000 7fb98d6e3640 1 --2- 172.21.15.12:0/1215081228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56215498ba00 0x56215498ad20 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fb980014310 tx=0x7fb9800097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5260ce3559825710 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.980+0000 7fb9727fc640 1 -- 172.21.15.12:0/1215081228 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98001a040 con 0x56215498ba00 2024-04-22T13:34:37.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.980+0000 7fb9727fc640 1 -- 172.21.15.12:0/1215081228 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9800119e0 con 0x56215498ba00 2024-04-22T13:34:37.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.980+0000 7fb9727fc640 1 -- 172.21.15.12:0/1215081228 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9800053a0 con 0x56215498ba00 2024-04-22T13:34:37.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.981+0000 7fb991e0c740 1 -- 172.21.15.12:0/1215081228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56215498ba00 msgr2=0x56215498ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.981+0000 7fb991e0c740 1 --2- 172.21.15.12:0/1215081228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56215498ba00 0x56215498ad20 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fb980014310 tx=0x7fb9800097c0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.981+0000 7fb991e0c740 1 -- 172.21.15.12:0/1215081228 shutdown_connections 2024-04-22T13:34:37.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.981+0000 7fb991e0c740 1 --2- 172.21.15.12:0/1215081228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56215498ba00 0x56215498ad20 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.981+0000 7fb991e0c740 1 --2- 172.21.15.12:0/1215081228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562154982e80 0x562154987880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.981+0000 7fb991e0c740 1 --2- 172.21.15.12:0/1215081228 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621549824d0 0x562154997860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.981+0000 7fb991e0c740 1 -- 172.21.15.12:0/1215081228 >> 172.21.15.12:0/1215081228 conn(0x56215497cba0 msgr2=0x56215497e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.981+0000 7fb991e0c740 1 -- 172.21.15.12:0/1215081228 shutdown_connections 2024-04-22T13:34:37.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.981+0000 7fb991e0c740 1 -- 172.21.15.12:0/1215081228 wait complete. 2024-04-22T13:34:37.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.981+0000 7fb991e0c740 1 -- start start 2024-04-22T13:34:37.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.981+0000 7fb991e0c740 1 Processor -- start 2024-04-22T13:34:37.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:37 smithi111 ceph-mon[31361]: pgmap v326: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:37.979 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:37 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1095992961' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:34:37.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.982+0000 7fb991e0c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621549824d0 0x562154993d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.982+0000 7fb991e0c740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562154982e80 0x5621549942b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.982+0000 7fb991e0c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56215498ba00 0x562154984bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.982+0000 7fb991e0c740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb9881aefc0 con 0x562154982e80 2024-04-22T13:34:37.983 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.982+0000 7fb991e0c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb9881ae8f0 con 0x56215498ba00 2024-04-22T13:34:37.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.982+0000 7fb991e0c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb9881abb50 con 0x5621549824d0 2024-04-22T13:34:37.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.982+0000 7fb991e0c740 10 client.0 _ll_register_callbacks cb 0x7fb980009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:37.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.982+0000 7fb987fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562154982e80 0x5621549942b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.982+0000 7fb987fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562154982e80 0x5621549942b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50046/0 (socket says 172.21.15.12:50046) 2024-04-22T13:34:37.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.982+0000 7fb987fff640 1 -- 172.21.15.12:0/2571830975 learned_addr learned my addr 172.21.15.12:0/2571830975 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:37.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.982+0000 7fb98d6e3640 1 --2- 172.21.15.12:0/2571830975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56215498ba00 0x562154984bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.982+0000 7fb98cee2640 1 --2- 172.21.15.12:0/2571830975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621549824d0 0x562154993d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.985 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.982+0000 7fb987fff640 1 -- 172.21.15.12:0/2571830975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621549824d0 msgr2=0x562154993d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.982+0000 7fb987fff640 1 --2- 172.21.15.12:0/2571830975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621549824d0 0x562154993d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.982+0000 7fb987fff640 1 -- 172.21.15.12:0/2571830975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56215498ba00 msgr2=0x562154984bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:37.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.982+0000 7fb987fff640 1 --2- 172.21.15.12:0/2571830975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56215498ba00 0x562154984bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.982+0000 7fb987fff640 1 -- 172.21.15.12:0/2571830975 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562154985420 con 0x562154982e80 2024-04-22T13:34:37.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.982+0000 7fb987fff640 1 --2- 172.21.15.12:0/2571830975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562154982e80 0x5621549942b0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fb974005ec0 tx=0x7fb9740a9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9701aae38127178 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:37.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.982+0000 7fb967fff640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:34:37.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.982+0000 7fb967fff640 1 -- 172.21.15.12:0/2571830975 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9740df0b0 con 0x562154982e80 2024-04-22T13:34:37.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.983+0000 7fb967fff640 1 -- 172.21.15.12:0/2571830975 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7fb9740054d0 con 0x562154982e80 2024-04-22T13:34:37.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.983+0000 7fb991e0c740 1 -- 172.21.15.12:0/2571830975 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9800119e0 con 0x562154982e80 2024-04-22T13:34:37.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.983+0000 7fb967fff640 1 -- 172.21.15.12:0/2571830975 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb9740bf160 con 0x562154982e80 2024-04-22T13:34:37.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.983+0000 7fb991e0c740 10 client.19197 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:37.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.983+0000 7fb991e0c740 1 -- 172.21.15.12:0/2571830975 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562154a01920 con 0x562154982e80 2024-04-22T13:34:37.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.983+0000 7fb967fff640 1 -- 172.21.15.12:0/2571830975 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9800119e0 con 0x562154982e80 2024-04-22T13:34:37.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.983+0000 7fb967fff640 1 -- 172.21.15.12:0/2571830975 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fb9740f0380 con 0x562154982e80 2024-04-22T13:34:37.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.983+0000 7fb991e0c740 10 client.19197 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:37.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.983+0000 7fb991e0c740 10 client.19197 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-22T13:34:37.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.983+0000 7fb991e0c740 20 client.19197 populate_metadata read hostname 'smithi012' 2024-04-22T13:34:37.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.983+0000 7fb991e0c740 1 --2- 172.21.15.12:0/2571830975 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x562154a020c0 0x562154a044b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:37.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.983+0000 7fb991e0c740 4 client.19197 mds_command: new command op to 24484 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:34:37.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.983+0000 7fb991e0c740 1 -- 172.21.15.12:0/2571830975 --> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5621548f4c10 con 0x562154a020c0 2024-04-22T13:34:37.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.983+0000 7fb98cee2640 1 --2- 172.21.15.12:0/2571830975 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x562154a020c0 0x562154a044b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:37.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.984+0000 7fb98cee2640 1 --2- 172.21.15.12:0/2571830975 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x562154a020c0 0x562154a044b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ea820a1554c36bc0 server_cookie=ce641e96649b3f33 in_seq=0 out_seq=0 2024-04-22T13:34:37.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.984+0000 7fb967fff640 10 client.19197 ms_handle_connect on v2:172.21.15.134:6838/1488513960 2024-04-22T13:34:37.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.984+0000 7fb967fff640 1 -- 172.21.15.12:0/2571830975 <== mds.1 v2:172.21.15.134:6838/1488513960 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5621548f4c10 con 0x562154a020c0 2024-04-22T13:34:37.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.984+0000 7fb967fff640 10 client.19197 handle_command_reply: tid=0 2024-04-22T13:34:37.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.984+0000 7fb991e0c740 1 client.19197 shutdown 2024-04-22T13:34:37.987 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.984+0000 7fb991e0c740 1 -- 172.21.15.12:0/2571830975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562154982e80 msgr2=0x5621549942b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.984+0000 7fb991e0c740 1 --2- 172.21.15.12:0/2571830975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562154982e80 0x5621549942b0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fb974005ec0 tx=0x7fb9740a9040 comp rx=0 tx=0).stop 2024-04-22T13:34:37.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.984+0000 7fb991e0c740 1 -- 172.21.15.12:0/2571830975 shutdown_connections 2024-04-22T13:34:37.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.984+0000 7fb991e0c740 1 --2- 172.21.15.12:0/2571830975 >> [v2:172.21.15.134:6838/1488513960,v1:172.21.15.134:6839/1488513960] conn(0x562154a020c0 0x562154a044b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.984+0000 7fb991e0c740 1 --2- 172.21.15.12:0/2571830975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56215498ba00 0x562154984bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.984+0000 7fb967fff640 0 client.19197 ms_handle_reset on v2:172.21.15.134:6838/1488513960 2024-04-22T13:34:37.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.984+0000 7fb991e0c740 1 --2- 172.21.15.12:0/2571830975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562154982e80 0x5621549942b0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.984+0000 7fb991e0c740 1 --2- 172.21.15.12:0/2571830975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621549824d0 0x562154993d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.984+0000 7fb991e0c740 1 -- 172.21.15.12:0/2571830975 >> 172.21.15.12:0/2571830975 conn(0x56215497cba0 msgr2=0x56215497df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.984+0000 7fb991e0c740 1 -- 172.21.15.12:0/2571830975 shutdown_connections 2024-04-22T13:34:37.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.984+0000 7fb991e0c740 1 -- 172.21.15.12:0/2571830975 wait complete. 2024-04-22T13:34:37.988 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.984+0000 7fb991e0c740 20 client.19197 trim_cache size 0 max 16384 2024-04-22T13:34:37.988 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:34:37.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.985+0000 7fb98f146640 1 -- 172.21.15.12:0/3924223561 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb9780a3da0 msgr2=0x7fb9780a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.985+0000 7fb98f146640 1 --2- 172.21.15.12:0/3924223561 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb9780a3da0 0x7fb9780a6260 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb980005fd0 tx=0x7fb980004ab0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.989 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.985+0000 7fb98f146640 1 -- 172.21.15.12:0/3924223561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb988107260 msgr2=0x7fb988198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:37.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.985+0000 7fb98f146640 1 --2- 172.21.15.12:0/3924223561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb988107260 0x7fb988198ca0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fb974036040 tx=0x7fb97402c7e0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.985+0000 7fb98f146640 1 -- 172.21.15.12:0/3924223561 shutdown_connections 2024-04-22T13:34:37.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.985+0000 7fb98f146640 1 --2- 172.21.15.12:0/3924223561 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb9780a3da0 0x7fb9780a6260 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.985+0000 7fb98f146640 1 --2- 172.21.15.12:0/3924223561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb98819d020 0x7fb98819d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.985+0000 7fb98f146640 1 --2- 172.21.15.12:0/3924223561 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb988107260 0x7fb988198ca0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.985+0000 7fb98f146640 1 --2- 172.21.15.12:0/3924223561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb988102190 0x7fb988198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:37.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.985+0000 7fb98f146640 1 -- 172.21.15.12:0/3924223561 >> 172.21.15.12:0/3924223561 conn(0x7fb9880fd990 msgr2=0x7fb9880fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:37.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.985+0000 7fb98f146640 1 -- 172.21.15.12:0/3924223561 shutdown_connections 2024-04-22T13:34:37.990 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:37.985+0000 7fb98f146640 1 -- 172.21.15.12:0/3924223561 wait complete. 2024-04-22T13:34:37.991 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-22T13:34:38.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.100+0000 7faeae398640 1 -- 172.21.15.12:0/1980991305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faea8106160 msgr2=0x7faea810dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:38.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.100+0000 7faeae398640 1 --2- 172.21.15.12:0/1980991305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faea8106160 0x7faea810dd30 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7faea0009ca0 tx=0x7faea002c0f0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.100+0000 7faeae398640 1 -- 172.21.15.12:0/1980991305 shutdown_connections 2024-04-22T13:34:38.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.100+0000 7faeae398640 1 --2- 172.21.15.12:0/1980991305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faea8106160 0x7faea810dd30 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.100+0000 7faeae398640 1 --2- 172.21.15.12:0/1980991305 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faea8101a40 0x7faea8105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.100+0000 7faeae398640 1 --2- 172.21.15.12:0/1980991305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faea8101090 0x7faea8101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.100+0000 7faeae398640 1 -- 172.21.15.12:0/1980991305 >> 172.21.15.12:0/1980991305 conn(0x7faea8077f50 msgr2=0x7faea8078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:38.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.100+0000 7faeae398640 1 -- 172.21.15.12:0/1980991305 shutdown_connections 2024-04-22T13:34:38.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.100+0000 7faeae398640 1 -- 172.21.15.12:0/1980991305 wait complete. 2024-04-22T13:34:38.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.101+0000 7faeae398640 1 Processor -- start 2024-04-22T13:34:38.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.101+0000 7faeae398640 1 -- start start 2024-04-22T13:34:38.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.101+0000 7faeae398640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faea8101090 0x7faea81986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:38.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.101+0000 7faeae398640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faea8106160 0x7faea8198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:38.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.101+0000 7faeae398640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faea819cf30 0x7faea819d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:38.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.101+0000 7faeae398640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7faea81121e0 con 0x7faea8101090 2024-04-22T13:34:38.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.101+0000 7faeae398640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faea8112060 con 0x7faea819cf30 2024-04-22T13:34:38.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.101+0000 7faeae398640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faea8112360 con 0x7faea8106160 2024-04-22T13:34:38.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.101+0000 7faea7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faea8101090 0x7faea81986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:38.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.101+0000 7faea7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faea8101090 0x7faea81986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50066/0 (socket says 172.21.15.12:50066) 2024-04-22T13:34:38.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.101+0000 7faea7fff640 1 -- 172.21.15.12:0/4179012857 learned_addr learned my addr 172.21.15.12:0/4179012857 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:38.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.101+0000 7faeac935640 1 --2- 172.21.15.12:0/4179012857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faea819cf30 0x7faea819d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:38.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.101+0000 7faea77fe640 1 --2- 172.21.15.12:0/4179012857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faea8106160 0x7faea8198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:38.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.101+0000 7faea7fff640 1 -- 172.21.15.12:0/4179012857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faea8106160 msgr2=0x7faea8198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:38.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.101+0000 7faea7fff640 1 --2- 172.21.15.12:0/4179012857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faea8106160 0x7faea8198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.101+0000 7faea7fff640 1 -- 172.21.15.12:0/4179012857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faea819cf30 msgr2=0x7faea819d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:38.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.101+0000 7faea7fff640 1 --2- 172.21.15.12:0/4179012857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faea819cf30 0x7faea819d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.101+0000 7faea7fff640 1 -- 172.21.15.12:0/4179012857 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea819da20 con 0x7faea8101090 2024-04-22T13:34:38.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.102+0000 7faea7fff640 1 --2- 172.21.15.12:0/4179012857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faea8101090 0x7faea81986c0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fae9800d770 tx=0x7fae9800db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:38.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.102+0000 7faea57fa640 1 -- 172.21.15.12:0/4179012857 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98017070 con 0x7faea8101090 2024-04-22T13:34:38.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.102+0000 7faeae398640 1 -- 172.21.15.12:0/4179012857 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faea819dd10 con 0x7faea8101090 2024-04-22T13:34:38.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.102+0000 7faea57fa640 1 -- 172.21.15.12:0/4179012857 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae98009820 con 0x7faea8101090 2024-04-22T13:34:38.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.102+0000 7faea57fa640 1 -- 172.21.15.12:0/4179012857 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98004bc0 con 0x7faea8101090 2024-04-22T13:34:38.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.102+0000 7faeae398640 1 -- 172.21.15.12:0/4179012857 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faea81a5500 con 0x7faea8101090 2024-04-22T13:34:38.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.105+0000 7faea57fa640 1 -- 172.21.15.12:0/4179012857 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fae980099d0 con 0x7faea8101090 2024-04-22T13:34:38.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.105+0000 7faeb0f81740 1 Processor -- start 2024-04-22T13:34:38.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.105+0000 7faeb0f81740 1 -- start start 2024-04-22T13:34:38.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.105+0000 7faeb0f81740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555fbbfd44d0 0x555fbbfd48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:38.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.105+0000 7faeb0f81740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fbbfd4e80 0x555fbbfdd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:38.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.105+0000 7faeb0f81740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555fbbfdda00 0x555fbbfdfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:38.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.105+0000 7faeb0f81740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555fbbf391f0 con 0x555fbbfd44d0 2024-04-22T13:34:38.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.105+0000 7faeb0f81740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555fbbed64b0 con 0x555fbbfdda00 2024-04-22T13:34:38.107 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.105+0000 7faeb0f81740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555fbbf351b0 con 0x555fbbfd4e80 2024-04-22T13:34:38.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.105+0000 7faeac935640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555fbbfd44d0 0x555fbbfd48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:38.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.105+0000 7faeac935640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555fbbfd44d0 0x555fbbfd48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50080/0 (socket says 172.21.15.12:50080) 2024-04-22T13:34:38.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.105+0000 7faeac935640 1 -- 172.21.15.12:0/1123458915 learned_addr learned my addr 172.21.15.12:0/1123458915 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:38.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.105+0000 7faea77fe640 1 --2- 172.21.15.12:0/1123458915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555fbbfdda00 0x555fbbfdfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:38.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.105+0000 7faea7fff640 1 --2- 172.21.15.12:0/1123458915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fbbfd4e80 0x555fbbfdd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:38.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.106+0000 7faea57fa640 1 --2- 172.21.15.12:0/4179012857 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fae900a3d20 0x7fae900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:38.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.106+0000 7faeac935640 1 -- 172.21.15.12:0/1123458915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fbbfd4e80 msgr2=0x555fbbfdd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:38.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.106+0000 7faeac935640 1 --2- 172.21.15.12:0/1123458915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fbbfd4e80 0x555fbbfdd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.106+0000 7faeac935640 1 -- 172.21.15.12:0/1123458915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555fbbfdda00 msgr2=0x555fbbfdfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:38.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.106+0000 7faeac935640 1 --2- 172.21.15.12:0/1123458915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555fbbfdda00 0x555fbbfdfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.106+0000 7faeac935640 1 -- 172.21.15.12:0/1123458915 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fbbec81b0 con 0x555fbbfd44d0 2024-04-22T13:34:38.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.106+0000 7faea77fe640 1 --2- 172.21.15.12:0/1123458915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555fbbfdda00 0x555fbbfdfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:38.108 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.106+0000 7faea57fa640 1 -- 172.21.15.12:0/4179012857 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7fae980d0200 con 0x7faea8101090 2024-04-22T13:34:38.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.106+0000 7faeac935640 1 --2- 172.21.15.12:0/4179012857 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fae900a3d20 0x7fae900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:38.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.106+0000 7faeac935640 1 --2- 172.21.15.12:0/1123458915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555fbbfd44d0 0x555fbbfd48b0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7faea0009ca0 tx=0x7faea0005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66299abb52361c17 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:38.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.106+0000 7fae8e7fc640 1 -- 172.21.15.12:0/1123458915 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0039040 con 0x555fbbfd44d0 2024-04-22T13:34:38.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.106+0000 7fae8e7fc640 1 -- 172.21.15.12:0/1123458915 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faea0043050 con 0x555fbbfd44d0 2024-04-22T13:34:38.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.106+0000 7fae8e7fc640 1 -- 172.21.15.12:0/1123458915 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0051070 con 0x555fbbfd44d0 2024-04-22T13:34:38.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.107+0000 7faeb0f81740 1 -- 172.21.15.12:0/1123458915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555fbbfd44d0 msgr2=0x555fbbfd48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:38.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.107+0000 7faeb0f81740 1 --2- 172.21.15.12:0/1123458915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555fbbfd44d0 0x555fbbfd48b0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7faea0009ca0 tx=0x7faea0005fa0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.107+0000 7faeac935640 1 --2- 172.21.15.12:0/4179012857 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fae900a3d20 0x7fae900a61e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7faea004a000 tx=0x7faea004b000 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:38.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.107+0000 7faeb0f81740 1 -- 172.21.15.12:0/1123458915 shutdown_connections 2024-04-22T13:34:38.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.107+0000 7faeb0f81740 1 --2- 172.21.15.12:0/1123458915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555fbbfdda00 0x555fbbfdfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.107+0000 7faeb0f81740 1 --2- 172.21.15.12:0/1123458915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fbbfd4e80 0x555fbbfdd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.107+0000 7faeb0f81740 1 --2- 172.21.15.12:0/1123458915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555fbbfd44d0 0x555fbbfd48b0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.107+0000 7faeb0f81740 1 -- 172.21.15.12:0/1123458915 >> 172.21.15.12:0/1123458915 conn(0x555fbbfceba0 msgr2=0x555fbbfcef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:38.109 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.107+0000 7faeb0f81740 1 -- 172.21.15.12:0/1123458915 shutdown_connections 2024-04-22T13:34:38.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.107+0000 7faeb0f81740 1 -- 172.21.15.12:0/1123458915 wait complete. 2024-04-22T13:34:38.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.107+0000 7faeb0f81740 1 -- start start 2024-04-22T13:34:38.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.108+0000 7faeb0f81740 1 Processor -- start 2024-04-22T13:34:38.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.108+0000 7faeb0f81740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fbbfd44d0 0x555fbbfdd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:38.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.108+0000 7faeb0f81740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555fbbfd4e80 0x555fbbfda6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:38.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.108+0000 7faeb0f81740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555fbbfdda00 0x555fbbfdacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:38.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.108+0000 7faeb0f81740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x555fbbfe0970 con 0x555fbbfdda00 2024-04-22T13:34:38.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.108+0000 7faeb0f81740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555fbbef2500 con 0x555fbbfd4e80 2024-04-22T13:34:38.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.108+0000 7faeb0f81740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555fbbfe0af0 con 0x555fbbfd44d0 2024-04-22T13:34:38.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.108+0000 7faeb0f81740 10 client.0 _ll_register_callbacks cb 0x555fbbec0b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:38.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.108+0000 7faeac935640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555fbbfdda00 0x555fbbfdacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:38.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.108+0000 7faeac935640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555fbbfdda00 0x555fbbfdacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50086/0 (socket says 172.21.15.12:50086) 2024-04-22T13:34:38.110 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.108+0000 7faea7fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fbbfd44d0 0x555fbbfdd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:38.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.108+0000 7faeac935640 1 -- 172.21.15.12:0/847888308 learned_addr learned my addr 172.21.15.12:0/847888308 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:38.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.108+0000 7faea77fe640 1 --2- 172.21.15.12:0/847888308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555fbbfd4e80 0x555fbbfda6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:38.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.108+0000 7faeac935640 1 -- 172.21.15.12:0/847888308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fbbfd44d0 msgr2=0x555fbbfdd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:38.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.108+0000 7faeac935640 1 --2- 172.21.15.12:0/847888308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fbbfd44d0 0x555fbbfdd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.108+0000 7faeac935640 1 -- 172.21.15.12:0/847888308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555fbbfd4e80 msgr2=0x555fbbfda6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:38.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.108+0000 7faeac935640 1 --2- 172.21.15.12:0/847888308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555fbbfd4e80 0x555fbbfda6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.108+0000 7faeac935640 1 -- 172.21.15.12:0/847888308 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fbbc4b3f0 con 0x555fbbfdda00 2024-04-22T13:34:38.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.108+0000 7faea7fff640 1 --2- 172.21.15.12:0/847888308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fbbfd44d0 0x555fbbfdd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-22T13:34:38.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.108+0000 7faeac935640 1 --2- 172.21.15.12:0/847888308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555fbbfdda00 0x555fbbfdacd0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7faea0005e30 tx=0x7faea0003dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f15d575c88bc9b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:38.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.108+0000 7fae877fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:34:38.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.109+0000 7fae877fe640 1 -- 172.21.15.12:0/847888308 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0059020 con 0x555fbbfdda00 2024-04-22T13:34:38.111 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.109+0000 7fae877fe640 1 -- 172.21.15.12:0/847888308 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7faea0032c40 con 0x555fbbfdda00 2024-04-22T13:34:38.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.109+0000 7faeb0f81740 1 -- 172.21.15.12:0/847888308 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faea0043050 con 0x555fbbfdda00 2024-04-22T13:34:38.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.109+0000 7fae877fe640 1 -- 172.21.15.12:0/847888308 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faea0064050 con 0x555fbbfdda00 2024-04-22T13:34:38.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.109+0000 7faeb0f81740 10 client.19209 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:38.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.109+0000 7faeb0f81740 1 -- 172.21.15.12:0/847888308 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555fbbd157d0 con 0x555fbbfdda00 2024-04-22T13:34:38.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.109+0000 7fae877fe640 1 -- 172.21.15.12:0/847888308 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faea0043050 con 0x555fbbfdda00 2024-04-22T13:34:38.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.109+0000 7fae877fe640 1 -- 172.21.15.12:0/847888308 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7faea0055b40 con 0x555fbbfdda00 2024-04-22T13:34:38.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.109+0000 7faeb0f81740 10 client.19209 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:38.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.109+0000 7faeb0f81740 10 client.19209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:34:38.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.109+0000 7faeb0f81740 20 client.19209 populate_metadata read hostname 'smithi012' 2024-04-22T13:34:38.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.109+0000 7faeb0f81740 1 --2- 172.21.15.12:0/847888308 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555fbbfedac0 0x555fbbfefeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:38.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.109+0000 7faeb0f81740 4 client.19209 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-22T13:34:38.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.109+0000 7faeb0f81740 1 -- 172.21.15.12:0/847888308 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fbbed8930 con 0x555fbbfedac0 2024-04-22T13:34:38.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.109+0000 7faea7fff640 1 --2- 172.21.15.12:0/847888308 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555fbbfedac0 0x555fbbfefeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:38.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.110+0000 7faea7fff640 1 --2- 172.21.15.12:0/847888308 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555fbbfedac0 0x555fbbfefeb0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e92e796e2f96251e server_cookie=b12e469f6586cb8a in_seq=0 out_seq=0 2024-04-22T13:34:38.112 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.110+0000 7fae877fe640 10 client.19209 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:34:38.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.111+0000 7fae877fe640 1 -- 172.21.15.12:0/847888308 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555fbbed8930 con 0x555fbbfedac0 2024-04-22T13:34:38.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.111+0000 7fae877fe640 10 client.19209 handle_command_reply: tid=0 2024-04-22T13:34:38.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.111+0000 7faeb0f81740 1 client.19209 shutdown 2024-04-22T13:34:38.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.111+0000 7faeb0f81740 1 -- 172.21.15.12:0/847888308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555fbbfdda00 msgr2=0x555fbbfdacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:38.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.111+0000 7faeb0f81740 1 --2- 172.21.15.12:0/847888308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555fbbfdda00 0x555fbbfdacd0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7faea0005e30 tx=0x7faea0003dd0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.111+0000 7faeb0f81740 1 -- 172.21.15.12:0/847888308 shutdown_connections 2024-04-22T13:34:38.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.111+0000 7faeb0f81740 1 --2- 172.21.15.12:0/847888308 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555fbbfedac0 0x555fbbfefeb0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.111+0000 7faeb0f81740 1 --2- 172.21.15.12:0/847888308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555fbbfdda00 0x555fbbfdacd0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.111+0000 7fae877fe640 0 client.19209 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:34:38.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.111+0000 7faeb0f81740 1 --2- 172.21.15.12:0/847888308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555fbbfd4e80 0x555fbbfda6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.111+0000 7faeb0f81740 1 --2- 172.21.15.12:0/847888308 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fbbfd44d0 0x555fbbfdd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.111+0000 7faeb0f81740 1 -- 172.21.15.12:0/847888308 >> 172.21.15.12:0/847888308 conn(0x555fbbfceba0 msgr2=0x555fbbfd08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:38.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.111+0000 7faeb0f81740 1 -- 172.21.15.12:0/847888308 shutdown_connections 2024-04-22T13:34:38.113 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.111+0000 7faeb0f81740 1 -- 172.21.15.12:0/847888308 wait complete. 2024-04-22T13:34:38.114 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.111+0000 7faeb0f81740 20 client.19209 trim_cache size 0 max 16384 2024-04-22T13:34:38.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.124+0000 7faeb0f81740 1 Processor -- start 2024-04-22T13:34:38.125 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.124+0000 7faeb0f81740 1 -- start start 2024-04-22T13:34:38.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.124+0000 7faeb0f81740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555fbbfd44d0 0x555fbbfe9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:38.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.124+0000 7faeb0f81740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fbbfd4e80 0x555fbbfdcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:38.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.124+0000 7faeb0f81740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555fbbfdda00 0x555fbbfdd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:38.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.124+0000 7faeb0f81740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7faea81adab0 con 0x555fbbfdda00 2024-04-22T13:34:38.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.124+0000 7faeb0f81740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faea81af5f0 con 0x555fbbfd44d0 2024-04-22T13:34:38.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.124+0000 7faeb0f81740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faea81ae180 con 0x555fbbfd4e80 2024-04-22T13:34:38.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.124+0000 7faeac935640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555fbbfdda00 0x555fbbfdd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:38.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.124+0000 7faeac935640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555fbbfdda00 0x555fbbfdd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50102/0 (socket says 172.21.15.12:50102) 2024-04-22T13:34:38.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.124+0000 7faeac935640 1 -- 172.21.15.12:0/1852213910 learned_addr learned my addr 172.21.15.12:0/1852213910 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:38.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.125+0000 7faea7fff640 1 --2- 172.21.15.12:0/1852213910 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555fbbfd44d0 0x555fbbfe9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:38.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.125+0000 7faea77fe640 1 --2- 172.21.15.12:0/1852213910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fbbfd4e80 0x555fbbfdcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:38.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.125+0000 7faeac935640 1 -- 172.21.15.12:0/1852213910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fbbfd4e80 msgr2=0x555fbbfdcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:38.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.125+0000 7faeac935640 1 --2- 172.21.15.12:0/1852213910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fbbfd4e80 0x555fbbfdcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.125+0000 7faeac935640 1 -- 172.21.15.12:0/1852213910 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555fbbfd44d0 msgr2=0x555fbbfe9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:38.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.125+0000 7faeac935640 1 --2- 172.21.15.12:0/1852213910 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555fbbfd44d0 0x555fbbfe9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.125+0000 7faeac935640 1 -- 172.21.15.12:0/1852213910 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fbbfe02b0 con 0x555fbbfdda00 2024-04-22T13:34:38.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.125+0000 7faea77fe640 1 --2- 172.21.15.12:0/1852213910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fbbfd4e80 0x555fbbfdcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:38.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.125+0000 7faea7fff640 1 --2- 172.21.15.12:0/1852213910 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555fbbfd44d0 0x555fbbfe9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:38.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.125+0000 7faeac935640 1 --2- 172.21.15.12:0/1852213910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555fbbfdda00 0x555fbbfdd330 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7faea004b510 tx=0x7faea0032a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fef67275cfec2fb server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:38.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.125+0000 7fae8dffb640 1 -- 172.21.15.12:0/1852213910 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0039030 con 0x555fbbfdda00 2024-04-22T13:34:38.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.125+0000 7fae8dffb640 1 -- 172.21.15.12:0/1852213910 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faea0043040 con 0x555fbbfdda00 2024-04-22T13:34:38.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.125+0000 7fae8dffb640 1 -- 172.21.15.12:0/1852213910 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0059040 con 0x555fbbfdda00 2024-04-22T13:34:38.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.126+0000 7faeb0f81740 1 -- 172.21.15.12:0/1852213910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555fbbfdda00 msgr2=0x555fbbfdd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:38.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.126+0000 7faeb0f81740 1 --2- 172.21.15.12:0/1852213910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555fbbfdda00 0x555fbbfdd330 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7faea004b510 tx=0x7faea0032a50 comp rx=0 tx=0).stop 2024-04-22T13:34:38.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.126+0000 7faeb0f81740 1 -- 172.21.15.12:0/1852213910 shutdown_connections 2024-04-22T13:34:38.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.126+0000 7faeb0f81740 1 --2- 172.21.15.12:0/1852213910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555fbbfdda00 0x555fbbfdd330 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.126+0000 7faeb0f81740 1 --2- 172.21.15.12:0/1852213910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fbbfd4e80 0x555fbbfdcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.126+0000 7faeb0f81740 1 --2- 172.21.15.12:0/1852213910 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555fbbfd44d0 0x555fbbfe9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.126+0000 7faeb0f81740 1 -- 172.21.15.12:0/1852213910 >> 172.21.15.12:0/1852213910 conn(0x555fbbfceba0 msgr2=0x555fbbfd0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:38.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.126+0000 7faeb0f81740 1 -- 172.21.15.12:0/1852213910 shutdown_connections 2024-04-22T13:34:38.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.126+0000 7faeb0f81740 1 -- 172.21.15.12:0/1852213910 wait complete. 2024-04-22T13:34:38.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.126+0000 7faeb0f81740 1 -- start start 2024-04-22T13:34:38.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.126+0000 7faeb0f81740 1 Processor -- start 2024-04-22T13:34:38.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.126+0000 7faeb0f81740 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555fbbfd44d0 0x555fbbfe61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:38.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.126+0000 7faeb0f81740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fbbfd4e80 0x555fbbfe1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:38.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.126+0000 7faeb0f81740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555fbbfdda00 0x555fbbfe1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:38.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.126+0000 7faeb0f81740 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7faea81aef20 con 0x555fbbfd44d0 2024-04-22T13:34:38.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.126+0000 7faeb0f81740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faea81ae850 con 0x555fbbfdda00 2024-04-22T13:34:38.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.126+0000 7faeb0f81740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faea81abab0 con 0x555fbbfd4e80 2024-04-22T13:34:38.128 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.127+0000 7faeb0f81740 10 client.0 _ll_register_callbacks cb 0x7faea0003ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-22T13:34:38.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.127+0000 7faea7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555fbbfd44d0 0x555fbbfe61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:38.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.127+0000 7faea7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555fbbfd44d0 0x555fbbfe61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50104/0 (socket says 172.21.15.12:50104) 2024-04-22T13:34:38.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.127+0000 7faea7fff640 1 -- 172.21.15.12:0/1039525871 learned_addr learned my addr 172.21.15.12:0/1039525871 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:38.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.127+0000 7faeac935640 1 --2- 172.21.15.12:0/1039525871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555fbbfdda00 0x555fbbfe1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:38.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.127+0000 7faea77fe640 1 --2- 172.21.15.12:0/1039525871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fbbfd4e80 0x555fbbfe1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:38.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.127+0000 7faea7fff640 1 -- 172.21.15.12:0/1039525871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fbbfd4e80 msgr2=0x555fbbfe1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:38.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.127+0000 7faea7fff640 1 --2- 172.21.15.12:0/1039525871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fbbfd4e80 0x555fbbfe1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.127+0000 7faea7fff640 1 -- 172.21.15.12:0/1039525871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555fbbfdda00 msgr2=0x555fbbfe1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-22T13:34:38.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.127+0000 7faea7fff640 1 --2- 172.21.15.12:0/1039525871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555fbbfdda00 0x555fbbfe1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.127+0000 7faea7fff640 1 -- 172.21.15.12:0/1039525871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fbbfe2350 con 0x555fbbfd44d0 2024-04-22T13:34:38.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.127+0000 7faea77fe640 1 --2- 172.21.15.12:0/1039525871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fbbfd4e80 0x555fbbfe1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:38.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.127+0000 7faea7fff640 1 --2- 172.21.15.12:0/1039525871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555fbbfd44d0 0x555fbbfe61c0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fae980cd9d0 tx=0x7fae980d0400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d7ee48995a11e36 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:38.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.127+0000 7fae877fe640 10 client.0 ms_handle_connect on v2:172.21.15.12:3300/0 2024-04-22T13:34:38.129 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.127+0000 7fae877fe640 1 -- 172.21.15.12:0/1039525871 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98020040 con 0x555fbbfd44d0 2024-04-22T13:34:38.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.127+0000 7fae877fe640 1 -- 172.21.15.12:0/1039525871 <== mon.0 v2:172.21.15.12:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7fae98009cf0 con 0x555fbbfd44d0 2024-04-22T13:34:38.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.127+0000 7faeb0f81740 1 -- 172.21.15.12:0/1039525871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faea0043040 con 0x555fbbfd44d0 2024-04-22T13:34:38.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.127+0000 7fae877fe640 1 -- 172.21.15.12:0/1039525871 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fae980a4e30 con 0x555fbbfd44d0 2024-04-22T13:34:38.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.127+0000 7faeb0f81740 10 client.19215 fetch_fsmap learned FSMap version 28 2024-04-22T13:34:38.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.127+0000 7faeb0f81740 1 -- 172.21.15.12:0/1039525871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555fbc053d90 con 0x555fbbfd44d0 2024-04-22T13:34:38.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.128+0000 7fae877fe640 1 -- 172.21.15.12:0/1039525871 <== mon.0 v2:172.21.15.12:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faea0043040 con 0x555fbbfd44d0 2024-04-22T13:34:38.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.128+0000 7fae877fe640 1 -- 172.21.15.12:0/1039525871 <== mon.0 v2:172.21.15.12:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fae9809b610 con 0x555fbbfd44d0 2024-04-22T13:34:38.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.128+0000 7faeb0f81740 10 client.19215 fetch_fsmap finished waiting for FSMap version 28 2024-04-22T13:34:38.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.128+0000 7faeb0f81740 10 client.19215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-22T13:34:38.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.128+0000 7faeb0f81740 20 client.19215 populate_metadata read hostname 'smithi012' 2024-04-22T13:34:38.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.128+0000 7faeb0f81740 1 --2- 172.21.15.12:0/1039525871 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555fbc0544b0 0x555fbc0568d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:38.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.128+0000 7faeb0f81740 4 client.19215 mds_command: new command op to 24509 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-22T13:34:38.130 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.128+0000 7faeb0f81740 1 -- 172.21.15.12:0/1039525871 --> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555fbbf46c10 con 0x555fbc0544b0 2024-04-22T13:34:38.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.128+0000 7faea77fe640 1 --2- 172.21.15.12:0/1039525871 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555fbc0544b0 0x555fbc0568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:38.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.129+0000 7faea77fe640 1 --2- 172.21.15.12:0/1039525871 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555fbc0544b0 0x555fbc0568d0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e25516437700b44f server_cookie=9c712ef9a3c3ee48 in_seq=0 out_seq=0 2024-04-22T13:34:38.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.129+0000 7fae877fe640 10 client.19215 ms_handle_connect on v2:172.21.15.111:6838/2126656770 2024-04-22T13:34:38.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.129+0000 7fae877fe640 1 -- 172.21.15.12:0/1039525871 <== mds.0 v2:172.21.15.111:6838/2126656770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555fbbf46c10 con 0x555fbc0544b0 2024-04-22T13:34:38.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.129+0000 7fae877fe640 10 client.19215 handle_command_reply: tid=0 2024-04-22T13:34:38.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.129+0000 7faeb0f81740 1 client.19215 shutdown 2024-04-22T13:34:38.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.129+0000 7faeb0f81740 1 -- 172.21.15.12:0/1039525871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555fbbfd44d0 msgr2=0x555fbbfe61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:38.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.129+0000 7faeb0f81740 1 --2- 172.21.15.12:0/1039525871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555fbbfd44d0 0x555fbbfe61c0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fae980cd9d0 tx=0x7fae980d0400 comp rx=0 tx=0).stop 2024-04-22T13:34:38.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.129+0000 7faeb0f81740 1 -- 172.21.15.12:0/1039525871 shutdown_connections 2024-04-22T13:34:38.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.129+0000 7faeb0f81740 1 --2- 172.21.15.12:0/1039525871 >> [v2:172.21.15.111:6838/2126656770,v1:172.21.15.111:6839/2126656770] conn(0x555fbc0544b0 0x555fbc0568d0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.129+0000 7faeb0f81740 1 --2- 172.21.15.12:0/1039525871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555fbbfdda00 0x555fbbfe1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.129+0000 7faeb0f81740 1 --2- 172.21.15.12:0/1039525871 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555fbbfd4e80 0x555fbbfe1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.129+0000 7fae877fe640 0 client.19215 ms_handle_reset on v2:172.21.15.111:6838/2126656770 2024-04-22T13:34:38.131 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.129+0000 7faeb0f81740 1 --2- 172.21.15.12:0/1039525871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555fbbfd44d0 0x555fbbfe61c0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.129+0000 7faeb0f81740 1 -- 172.21.15.12:0/1039525871 >> 172.21.15.12:0/1039525871 conn(0x555fbbfceba0 msgr2=0x555fbbfd5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:38.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.129+0000 7faeb0f81740 1 -- 172.21.15.12:0/1039525871 shutdown_connections 2024-04-22T13:34:38.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.129+0000 7faeb0f81740 1 -- 172.21.15.12:0/1039525871 wait complete. 2024-04-22T13:34:38.132 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.129+0000 7faeb0f81740 20 client.19215 trim_cache size 0 max 16384 2024-04-22T13:34:38.132 INFO:teuthology.orchestra.run.smithi012.stdout:[] 2024-04-22T13:34:38.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.130+0000 7faeae398640 1 -- 172.21.15.12:0/4179012857 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fae900a3d20 msgr2=0x7fae900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:38.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.130+0000 7faeae398640 1 --2- 172.21.15.12:0/4179012857 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fae900a3d20 0x7fae900a61e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7faea004a000 tx=0x7faea004b000 comp rx=0 tx=0).stop 2024-04-22T13:34:38.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.130+0000 7faeae398640 1 -- 172.21.15.12:0/4179012857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faea8101090 msgr2=0x7faea81986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:38.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.130+0000 7faeae398640 1 --2- 172.21.15.12:0/4179012857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faea8101090 0x7faea81986c0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fae9800d770 tx=0x7fae9800db70 comp rx=0 tx=0).stop 2024-04-22T13:34:38.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.130+0000 7faeae398640 1 -- 172.21.15.12:0/4179012857 shutdown_connections 2024-04-22T13:34:38.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.130+0000 7faeae398640 1 --2- 172.21.15.12:0/4179012857 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fae900a3d20 0x7fae900a61e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.130+0000 7faeae398640 1 --2- 172.21.15.12:0/4179012857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faea819cf30 0x7faea819d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.133 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.130+0000 7faeae398640 1 --2- 172.21.15.12:0/4179012857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faea8106160 0x7faea8198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.130+0000 7faeae398640 1 --2- 172.21.15.12:0/4179012857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7faea8101090 0x7faea81986c0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.130+0000 7faeae398640 1 -- 172.21.15.12:0/4179012857 >> 172.21.15.12:0/4179012857 conn(0x7faea8077f50 msgr2=0x7faea80feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:38.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.130+0000 7faeae398640 1 -- 172.21.15.12:0/4179012857 shutdown_connections 2024-04-22T13:34:38.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.130+0000 7faeae398640 1 -- 172.21.15.12:0/4179012857 wait complete. 2024-04-22T13:34:38.136 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_d69f13d827952bb07bb9b7973bca14f97d764d6f/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_d69f13d827952bb07bb9b7973bca14f97d764d6f/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_d69f13d827952bb07bb9b7973bca14f97d764d6f/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-04-22T13:34:38.136 INFO:tasks.fwd_scrub:done joining 2024-04-22T13:34:38.137 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-22T13:34:38.149 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:34:38.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.255+0000 7f659c967640 1 -- 172.21.15.12:0/1657274773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6598106f50 msgr2=0x7f6598107330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:38.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.255+0000 7f659c967640 1 --2- 172.21.15.12:0/1657274773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6598106f50 0x7f6598107330 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f6588009930 tx=0x7f658802bc30 comp rx=0 tx=0).stop 2024-04-22T13:34:38.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.255+0000 7f659c967640 1 -- 172.21.15.12:0/1657274773 shutdown_connections 2024-04-22T13:34:38.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.255+0000 7f659c967640 1 --2- 172.21.15.12:0/1657274773 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6598100e80 0x7f659810ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.255+0000 7f659c967640 1 --2- 172.21.15.12:0/1657274773 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65981004c0 0x7f6598100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.255+0000 7f659c967640 1 --2- 172.21.15.12:0/1657274773 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6598106f50 0x7f6598107330 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.255+0000 7f659c967640 1 -- 172.21.15.12:0/1657274773 >> 172.21.15.12:0/1657274773 conn(0x7f6598077450 msgr2=0x7f65980fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:38.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.255+0000 7f659c967640 1 -- 172.21.15.12:0/1657274773 shutdown_connections 2024-04-22T13:34:38.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.255+0000 7f659c967640 1 -- 172.21.15.12:0/1657274773 wait complete. 2024-04-22T13:34:38.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.256+0000 7f659c967640 1 Processor -- start 2024-04-22T13:34:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.256+0000 7f659c967640 1 -- start start 2024-04-22T13:34:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.256+0000 7f659c967640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65981004c0 0x7f6598198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.256+0000 7f659c967640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6598100e80 0x7f6598198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.256+0000 7f659c967640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6598106f50 0x7f659819cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.256+0000 7f659c967640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f6598112280 con 0x7f6598106f50 2024-04-22T13:34:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.256+0000 7f659c967640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6598112100 con 0x7f6598100e80 2024-04-22T13:34:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.256+0000 7f659c967640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6598112400 con 0x7f65981004c0 2024-04-22T13:34:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.257+0000 7f6596d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6598106f50 0x7f659819cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.257+0000 7f6596d9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6598106f50 0x7f659819cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50126/0 (socket says 172.21.15.12:50126) 2024-04-22T13:34:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.257+0000 7f6596d9d640 1 -- 172.21.15.12:0/563311455 learned_addr learned my addr 172.21.15.12:0/563311455 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.257+0000 7f659659c640 1 --2- 172.21.15.12:0/563311455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65981004c0 0x7f6598198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:38.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.257+0000 7f658ffff640 1 --2- 172.21.15.12:0/563311455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6598100e80 0x7f6598198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:38.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.257+0000 7f6596d9d640 1 -- 172.21.15.12:0/563311455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65981004c0 msgr2=0x7f6598198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:38.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.257+0000 7f6596d9d640 1 --2- 172.21.15.12:0/563311455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65981004c0 0x7f6598198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.257+0000 7f6596d9d640 1 -- 172.21.15.12:0/563311455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6598100e80 msgr2=0x7f6598198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:38.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.257+0000 7f6596d9d640 1 --2- 172.21.15.12:0/563311455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6598100e80 0x7f6598198cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.257+0000 7f6596d9d640 1 -- 172.21.15.12:0/563311455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f659819d720 con 0x7f6598106f50 2024-04-22T13:34:38.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.257+0000 7f6596d9d640 1 --2- 172.21.15.12:0/563311455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6598106f50 0x7f659819cff0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f659000cce0 tx=0x7f659000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:38.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.257+0000 7f658f7fe640 1 -- 172.21.15.12:0/563311455 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659000f450 con 0x7f6598106f50 2024-04-22T13:34:38.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.257+0000 7f658f7fe640 1 -- 172.21.15.12:0/563311455 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f659000faa0 con 0x7f6598106f50 2024-04-22T13:34:38.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.257+0000 7f659c967640 1 -- 172.21.15.12:0/563311455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f659819da10 con 0x7f6598106f50 2024-04-22T13:34:38.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.257+0000 7f658f7fe640 1 -- 172.21.15.12:0/563311455 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6590018b70 con 0x7f6598106f50 2024-04-22T13:34:38.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.257+0000 7f659c967640 1 -- 172.21.15.12:0/563311455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65981a52a0 con 0x7f6598106f50 2024-04-22T13:34:38.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.258+0000 7f659c967640 1 -- 172.21.15.12:0/563311455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6598069230 con 0x7f6598106f50 2024-04-22T13:34:38.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.259+0000 7f658f7fe640 1 -- 172.21.15.12:0/563311455 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6590018d10 con 0x7f6598106f50 2024-04-22T13:34:38.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.260+0000 7f658f7fe640 1 --2- 172.21.15.12:0/563311455 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f65840a3da0 0x7f65840a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:38.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.260+0000 7f659659c640 1 --2- 172.21.15.12:0/563311455 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f65840a3da0 0x7f65840a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:38.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.261+0000 7f658f7fe640 1 -- 172.21.15.12:0/563311455 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7f65900d0c60 con 0x7f6598106f50 2024-04-22T13:34:38.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.261+0000 7f659659c640 1 --2- 172.21.15.12:0/563311455 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f65840a3da0 0x7f65840a6260 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6588005ab0 tx=0x7f6588005fd0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:38.259 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.264+0000 7f658f7fe640 1 -- 172.21.15.12:0/563311455 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6590014030 con 0x7f6598106f50 2024-04-22T13:34:38.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.453+0000 7f659c967640 1 -- 172.21.15.12:0/563311455 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f659810a480 con 0x7f6598106f50 2024-04-22T13:34:38.450 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.454+0000 7f658f7fe640 1 -- 172.21.15.12:0/563311455 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+23087 (secure 0 0 0) 0x7f659009d410 con 0x7f6598106f50 2024-04-22T13:34:38.450 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:34:38.450 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":98,"fsid":"0aede5ea-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:14:09.362918+0000","modified":"2024-04-22T13:34:14.525607+0000","last_up_change":"2024-04-22T13:21:15.263393+0000","last_in_change":"2024-04-22T13:20:54.174685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:17:32.372434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:22:02.768799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:22:03.578730+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d9e2365-3a64-4b13-96fc-c217b1cee2ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6803","nonce":1191350632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6805","nonce":1191350632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6809","nonce":1191350632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6807","nonce":1191350632}]},"public_addr":"172.21.15.12:6803/1191350632","cluster_addr":"172.21.15.12:6805/1191350632","heartbeat_back_addr":"172.21.15.12:6809/1191350632","heartbeat_front_addr":"172.21.15.12:6807/1191350632","state":["exists","up"]},{"osd":1,"uuid":"1499c783-2bd7-48ce-a444-34add42bcf31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6801","nonce":417496465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6803","nonce":417496465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6807","nonce":417496465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6805","nonce":417496465}]},"public_addr":"172.21.15.111:6801/417496465","cluster_addr":"172.21.15.111:6803/417496465","heartbeat_back_addr":"172.21.15.111:6807/417496465","heartbeat_front_addr":"172.21.15.111:6805/417496465","state":["exists","up"]},{"osd":2,"uuid":"71a9ebc3-c622-434e-a6c2-f48358f384f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6801","nonce":1403153998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6803","nonce":1403153998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6807","nonce":1403153998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6805","nonce":1403153998}]},"public_addr":"172.21.15.134:6801/1403153998","cluster_addr":"172.21.15.134:6803/1403153998","heartbeat_back_addr":"172.21.15.134:6807/1403153998","heartbeat_front_addr":"172.21.15.134:6805/1403153998","state":["exists","up"]},{"osd":3,"uuid":"6dd00db2-a286-48ba-8666-56ffdd5f35ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6811","nonce":3359066681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6813","nonce":3359066681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6817","nonce":3359066681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6815","nonce":3359066681}]},"public_addr":"172.21.15.12:6811/3359066681","cluster_addr":"172.21.15.12:6813/3359066681","heartbeat_back_addr":"172.21.15.12:6817/3359066681","heartbeat_front_addr":"172.21.15.12:6815/3359066681","state":["exists","up"]},{"osd":4,"uuid":"1e29afd7-354c-4506-aaa0-b43eb1a19a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6809","nonce":2886349872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6811","nonce":2886349872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6815","nonce":2886349872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6813","nonce":2886349872}]},"public_addr":"172.21.15.111:6809/2886349872","cluster_addr":"172.21.15.111:6811/2886349872","heartbeat_back_addr":"172.21.15.111:6815/2886349872","heartbeat_front_addr":"172.21.15.111:6813/2886349872","state":["exists","up"]},{"osd":5,"uuid":"8f40f02d-0ba2-4842-a03d-2a1a04d1010d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6809","nonce":3071977602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6811","nonce":3071977602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6815","nonce":3071977602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6813","nonce":3071977602}]},"public_addr":"172.21.15.134:6809/3071977602","cluster_addr":"172.21.15.134:6811/3071977602","heartbeat_back_addr":"172.21.15.134:6815/3071977602","heartbeat_front_addr":"172.21.15.134:6813/3071977602","state":["exists","up"]},{"osd":6,"uuid":"bb0c17a9-6a8e-4be6-8660-f73cfa8200ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6819","nonce":3819619299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6821","nonce":3819619299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6825","nonce":3819619299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6823","nonce":3819619299}]},"public_addr":"172.21.15.12:6819/3819619299","cluster_addr":"172.21.15.12:6821/3819619299","heartbeat_back_addr":"172.21.15.12:6825/3819619299","heartbeat_front_addr":"172.21.15.12:6823/3819619299","state":["exists","up"]},{"osd":7,"uuid":"0a93d6af-569e-4e45-b697-9e978c1c1004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6817","nonce":3113856282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6819","nonce":3113856282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6823","nonce":3113856282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6821","nonce":3113856282}]},"public_addr":"172.21.15.111:6817/3113856282","cluster_addr":"172.21.15.111:6819/3113856282","heartbeat_back_addr":"172.21.15.111:6823/3113856282","heartbeat_front_addr":"172.21.15.111:6821/3113856282","state":["exists","up"]},{"osd":8,"uuid":"e21acc68-cf50-4f4f-b1ba-cf6c195b4955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6817","nonce":1689362191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6819","nonce":1689362191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6823","nonce":1689362191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6821","nonce":1689362191}]},"public_addr":"172.21.15.134:6817/1689362191","cluster_addr":"172.21.15.134:6819/1689362191","heartbeat_back_addr":"172.21.15.134:6823/1689362191","heartbeat_front_addr":"172.21.15.134:6821/1689362191","state":["exists","up"]},{"osd":9,"uuid":"fd745869-7f5f-48a8-b412-bb9c92b586e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6827","nonce":4292385779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6829","nonce":4292385779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6833","nonce":4292385779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6831","nonce":4292385779}]},"public_addr":"172.21.15.12:6827/4292385779","cluster_addr":"172.21.15.12:6829/4292385779","heartbeat_back_addr":"172.21.15.12:6833/4292385779","heartbeat_front_addr":"172.21.15.12:6831/4292385779","state":["exists","up"]},{"osd":10,"uuid":"798b1abd-f6c7-4a5f-8da0-fdae49e50d2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6825","nonce":3338566976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6827","nonce":3338566976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6831","nonce":3338566976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6829","nonce":3338566976}]},"public_addr":"172.21.15.111:6825/3338566976","cluster_addr":"172.21.15.111:6827/3338566976","heartbeat_back_addr":"172.21.15.111:6831/3338566976","heartbeat_front_addr":"172.21.15.111:6829/3338566976","state":["exists","up"]},{"osd":11,"uuid":"b18fc5f3-1049-4871-b060-e99290273cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6825","nonce":578640428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6827","nonce":578640428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6831","nonce":578640428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6829","nonce":578640428}]},"public_addr":"172.21.15.134:6825/578640428","cluster_addr":"172.21.15.134:6827/578640428","heartbeat_back_addr":"172.21.15.134:6831/578640428","heartbeat_front_addr":"172.21.15.134:6829/578640428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:16:40.828608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:05.052528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:28.297316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:53.496497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:17.947259+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:41.712894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:07.105945+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:32.129688+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:56.008516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:22.239867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:47.334692+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:12.301276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/3535299028":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/2837479598":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/3010129696":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/2593601362":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/2519382069":"2024-04-23T13:15:04.279990+0000","172.21.15.12:6801/2009831940":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/856202072":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/3509412187":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/1802873236":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/1007749062":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3588480095":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1904962176":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/1802873236":"2024-04-23T13:24:06.155564+0000","172.21.15.12:6800/2009831940":"2024-04-23T13:15:04.279990+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:34:38.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.456+0000 7f659c967640 1 -- 172.21.15.12:0/563311455 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f65840a3da0 msgr2=0x7f65840a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:38.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.456+0000 7f659c967640 1 --2- 172.21.15.12:0/563311455 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f65840a3da0 0x7f65840a6260 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6588005ab0 tx=0x7f6588005fd0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.456+0000 7f659c967640 1 -- 172.21.15.12:0/563311455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6598106f50 msgr2=0x7f659819cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:38.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.456+0000 7f659c967640 1 --2- 172.21.15.12:0/563311455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6598106f50 0x7f659819cff0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f659000cce0 tx=0x7f659000a620 comp rx=0 tx=0).stop 2024-04-22T13:34:38.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.456+0000 7f659c967640 1 -- 172.21.15.12:0/563311455 shutdown_connections 2024-04-22T13:34:38.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.456+0000 7f659c967640 1 --2- 172.21.15.12:0/563311455 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f65840a3da0 0x7f65840a6260 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.456+0000 7f659c967640 1 --2- 172.21.15.12:0/563311455 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f6598106f50 0x7f659819cff0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.456+0000 7f659c967640 1 --2- 172.21.15.12:0/563311455 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6598100e80 0x7f6598198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.456+0000 7f659c967640 1 --2- 172.21.15.12:0/563311455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65981004c0 0x7f6598198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.456+0000 7f659c967640 1 -- 172.21.15.12:0/563311455 >> 172.21.15.12:0/563311455 conn(0x7f6598077450 msgr2=0x7f6598104f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:38.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.457+0000 7f659c967640 1 -- 172.21.15.12:0/563311455 shutdown_connections 2024-04-22T13:34:38.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.457+0000 7f659c967640 1 -- 172.21.15.12:0/563311455 wait complete. 2024-04-22T13:34:38.463 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T13:17:32.372434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T13:22:02.768799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T13:22:03.578730+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 97, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T13:34:38.463 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T13:34:38.572 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.575+0000 7fc9a189a640 1 -- 172.21.15.12:0/3789084762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc99c102b30 msgr2=0x7fc99c106b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:38.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.575+0000 7fc9a189a640 1 --2- 172.21.15.12:0/3789084762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc99c102b30 0x7fc99c106b70 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fc990009900 tx=0x7fc99002bc30 comp rx=0 tx=0).stop 2024-04-22T13:34:38.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.575+0000 7fc9a189a640 1 -- 172.21.15.12:0/3789084762 shutdown_connections 2024-04-22T13:34:38.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.575+0000 7fc9a189a640 1 --2- 172.21.15.12:0/3789084762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc99c107250 0x7fc99c10ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.575+0000 7fc9a189a640 1 --2- 172.21.15.12:0/3789084762 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc99c102b30 0x7fc99c106b70 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.575+0000 7fc9a189a640 1 --2- 172.21.15.12:0/3789084762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc99c102180 0x7fc99c102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.575+0000 7fc9a189a640 1 -- 172.21.15.12:0/3789084762 >> 172.21.15.12:0/3789084762 conn(0x7fc99c0fd9c0 msgr2=0x7fc99c0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:38.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.575+0000 7fc9a189a640 1 -- 172.21.15.12:0/3789084762 shutdown_connections 2024-04-22T13:34:38.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.575+0000 7fc9a189a640 1 -- 172.21.15.12:0/3789084762 wait complete. 2024-04-22T13:34:38.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.576+0000 7fc9a189a640 1 Processor -- start 2024-04-22T13:34:38.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.576+0000 7fc9a189a640 1 -- start start 2024-04-22T13:34:38.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.576+0000 7fc9a189a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc99c102180 0x7fc99c19e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:38.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.576+0000 7fc9a189a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc99c107250 0x7fc99c19e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:38.573 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.576+0000 7fc9a189a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc99c1985c0 0x7fc99c198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:38.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.576+0000 7fc9a189a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c1121e0 con 0x7fc99c1985c0 2024-04-22T13:34:38.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.576+0000 7fc9a189a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c112060 con 0x7fc99c102180 2024-04-22T13:34:38.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.576+0000 7fc9a189a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c112360 con 0x7fc99c107250 2024-04-22T13:34:38.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.576+0000 7fc99b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc99c1985c0 0x7fc99c198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:38.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.576+0000 7fc99b7fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc99c1985c0 0x7fc99c198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50148/0 (socket says 172.21.15.12:50148) 2024-04-22T13:34:38.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.576+0000 7fc99b7fe640 1 -- 172.21.15.12:0/699254260 learned_addr learned my addr 172.21.15.12:0/699254260 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:38.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.576+0000 7fc99affd640 1 --2- 172.21.15.12:0/699254260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc99c102180 0x7fc99c19e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:38.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.576+0000 7fc99a7fc640 1 --2- 172.21.15.12:0/699254260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc99c107250 0x7fc99c19e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:38.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.577+0000 7fc99b7fe640 1 -- 172.21.15.12:0/699254260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc99c107250 msgr2=0x7fc99c19e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:38.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.577+0000 7fc99b7fe640 1 --2- 172.21.15.12:0/699254260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc99c107250 0x7fc99c19e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.577+0000 7fc99b7fe640 1 -- 172.21.15.12:0/699254260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc99c102180 msgr2=0x7fc99c19e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:38.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.577+0000 7fc99b7fe640 1 --2- 172.21.15.12:0/699254260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc99c102180 0x7fc99c19e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.577+0000 7fc99b7fe640 1 -- 172.21.15.12:0/699254260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99c1990e0 con 0x7fc99c1985c0 2024-04-22T13:34:38.574 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.577+0000 7fc99b7fe640 1 --2- 172.21.15.12:0/699254260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc99c1985c0 0x7fc99c198a70 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fc99400e970 tx=0x7fc99400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:38.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.577+0000 7fc9a0898640 1 -- 172.21.15.12:0/699254260 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99400ccc0 con 0x7fc99c1985c0 2024-04-22T13:34:38.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.577+0000 7fc9a189a640 1 -- 172.21.15.12:0/699254260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99c1993d0 con 0x7fc99c1985c0 2024-04-22T13:34:38.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.577+0000 7fc9a0898640 1 -- 172.21.15.12:0/699254260 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc99400ce60 con 0x7fc99c1985c0 2024-04-22T13:34:38.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.577+0000 7fc9a0898640 1 -- 172.21.15.12:0/699254260 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc994010430 con 0x7fc99c1985c0 2024-04-22T13:34:38.575 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.577+0000 7fc9a189a640 1 -- 172.21.15.12:0/699254260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc99c1a5620 con 0x7fc99c1985c0 2024-04-22T13:34:38.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.578+0000 7fc9a189a640 1 -- 172.21.15.12:0/699254260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc99c068d70 con 0x7fc99c1985c0 2024-04-22T13:34:38.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.579+0000 7fc9a0898640 1 -- 172.21.15.12:0/699254260 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc994004ff0 con 0x7fc99c1985c0 2024-04-22T13:34:38.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.580+0000 7fc9a0898640 1 --2- 172.21.15.12:0/699254260 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc9840a3d20 0x7fc9840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:38.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.580+0000 7fc99affd640 1 --2- 172.21.15.12:0/699254260 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc9840a3d20 0x7fc9840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:38.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.580+0000 7fc9a0898640 1 -- 172.21.15.12:0/699254260 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7fc9940d0b70 con 0x7fc99c1985c0 2024-04-22T13:34:38.576 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.580+0000 7fc99affd640 1 --2- 172.21.15.12:0/699254260 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc9840a3d20 0x7fc9840a61e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc98c005fd0 tx=0x7fc98c009500 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:38.580 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.585+0000 7fc9a0898640 1 -- 172.21.15.12:0/699254260 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc99409d230 con 0x7fc99c1985c0 2024-04-22T13:34:38.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.772+0000 7fc9a189a640 1 -- 172.21.15.12:0/699254260 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc99c102b30 con 0x7fc99c1985c0 2024-04-22T13:34:38.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.773+0000 7fc9a0898640 1 -- 172.21.15.12:0/699254260 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) ==== 93+0+10 (secure 0 0 0) 0x7fc99409cb30 con 0x7fc99c1985c0 2024-04-22T13:34:38.768 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-22T13:34:38.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.774+0000 7fc9a189a640 1 -- 172.21.15.12:0/699254260 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc9840a3d20 msgr2=0x7fc9840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:38.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.774+0000 7fc9a189a640 1 --2- 172.21.15.12:0/699254260 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc9840a3d20 0x7fc9840a61e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc98c005fd0 tx=0x7fc98c009500 comp rx=0 tx=0).stop 2024-04-22T13:34:38.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.774+0000 7fc9a189a640 1 -- 172.21.15.12:0/699254260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc99c1985c0 msgr2=0x7fc99c198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:38.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.774+0000 7fc9a189a640 1 --2- 172.21.15.12:0/699254260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc99c1985c0 0x7fc99c198a70 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fc99400e970 tx=0x7fc99400ed70 comp rx=0 tx=0).stop 2024-04-22T13:34:38.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.775+0000 7fc9a189a640 1 -- 172.21.15.12:0/699254260 shutdown_connections 2024-04-22T13:34:38.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.775+0000 7fc9a189a640 1 --2- 172.21.15.12:0/699254260 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc9840a3d20 0x7fc9840a61e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.775+0000 7fc9a189a640 1 --2- 172.21.15.12:0/699254260 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc99c1985c0 0x7fc99c198a70 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.775+0000 7fc9a189a640 1 --2- 172.21.15.12:0/699254260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc99c107250 0x7fc99c19e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.770 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.775+0000 7fc9a189a640 1 --2- 172.21.15.12:0/699254260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc99c102180 0x7fc99c19e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.775+0000 7fc9a189a640 1 -- 172.21.15.12:0/699254260 >> 172.21.15.12:0/699254260 conn(0x7fc99c0fd9c0 msgr2=0x7fc99c0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:38.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.775+0000 7fc9a189a640 1 -- 172.21.15.12:0/699254260 shutdown_connections 2024-04-22T13:34:38.771 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.775+0000 7fc9a189a640 1 -- 172.21.15.12:0/699254260 wait complete. 2024-04-22T13:34:38.780 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:38 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/563311455' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:34:38.781 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T13:34:38.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.890+0000 7fd20a49a640 1 -- 172.21.15.12:0/3740981971 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd20406a570 msgr2=0x7fd20406a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:38.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.890+0000 7fd20a49a640 1 --2- 172.21.15.12:0/3740981971 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd20406a570 0x7fd20406a9f0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8009930 tx=0x7fd1f802bc30 comp rx=0 tx=0).stop 2024-04-22T13:34:38.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.890+0000 7fd20a49a640 1 -- 172.21.15.12:0/3740981971 shutdown_connections 2024-04-22T13:34:38.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.890+0000 7fd20a49a640 1 --2- 172.21.15.12:0/3740981971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd20406af30 0x7fd20410ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.890+0000 7fd20a49a640 1 --2- 172.21.15.12:0/3740981971 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd20406a570 0x7fd20406a9f0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.890+0000 7fd20a49a640 1 --2- 172.21.15.12:0/3740981971 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd204106f90 0x7fd204107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.890+0000 7fd20a49a640 1 -- 172.21.15.12:0/3740981971 >> 172.21.15.12:0/3740981971 conn(0x7fd204077e30 msgr2=0x7fd204078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:38.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.890+0000 7fd20a49a640 1 -- 172.21.15.12:0/3740981971 shutdown_connections 2024-04-22T13:34:38.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.890+0000 7fd20a49a640 1 -- 172.21.15.12:0/3740981971 wait complete. 2024-04-22T13:34:38.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.891+0000 7fd20a49a640 1 Processor -- start 2024-04-22T13:34:38.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.891+0000 7fd20a49a640 1 -- start start 2024-04-22T13:34:38.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.891+0000 7fd20a49a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd20406a570 0x7fd204198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:38.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.891+0000 7fd20a49a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd20406af30 0x7fd204198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:38.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.891+0000 7fd20a49a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd204106f90 0x7fd20419cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:38.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.891+0000 7fd20a49a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fd2041121f0 con 0x7fd20406af30 2024-04-22T13:34:38.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.891+0000 7fd20a49a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd204112070 con 0x7fd20406a570 2024-04-22T13:34:38.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.891+0000 7fd20a49a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd204112370 con 0x7fd204106f90 2024-04-22T13:34:38.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.891+0000 7fd2037fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd20406af30 0x7fd204198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:38.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.891+0000 7fd2037fe640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd20406af30 0x7fd204198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50166/0 (socket says 172.21.15.12:50166) 2024-04-22T13:34:38.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.891+0000 7fd2037fe640 1 -- 172.21.15.12:0/4176406739 learned_addr learned my addr 172.21.15.12:0/4176406739 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:38.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.891+0000 7fd203fff640 1 --2- 172.21.15.12:0/4176406739 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd20406a570 0x7fd204198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:38.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.891+0000 7fd208a37640 1 --2- 172.21.15.12:0/4176406739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd204106f90 0x7fd20419cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:38.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.892+0000 7fd2037fe640 1 -- 172.21.15.12:0/4176406739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd204106f90 msgr2=0x7fd20419cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:38.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.892+0000 7fd2037fe640 1 --2- 172.21.15.12:0/4176406739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd204106f90 0x7fd20419cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.892+0000 7fd2037fe640 1 -- 172.21.15.12:0/4176406739 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd20406a570 msgr2=0x7fd204198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:38.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.892+0000 7fd2037fe640 1 --2- 172.21.15.12:0/4176406739 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd20406a570 0x7fd204198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:38.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.892+0000 7fd2037fe640 1 -- 172.21.15.12:0/4176406739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd20419d6f0 con 0x7fd20406af30 2024-04-22T13:34:38.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.892+0000 7fd208a37640 1 --2- 172.21.15.12:0/4176406739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd204106f90 0x7fd20419cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:34:38.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.892+0000 7fd2037fe640 1 --2- 172.21.15.12:0/4176406739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd20406af30 0x7fd204198c90 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8005940 tx=0x7fd1f802ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:38.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.892+0000 7fd2017fa640 1 -- 172.21.15.12:0/4176406739 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8043020 con 0x7fd20406af30 2024-04-22T13:34:38.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.892+0000 7fd2017fa640 1 -- 172.21.15.12:0/4176406739 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1f802cc80 con 0x7fd20406af30 2024-04-22T13:34:38.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.892+0000 7fd20a49a640 1 -- 172.21.15.12:0/4176406739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd20419d980 con 0x7fd20406af30 2024-04-22T13:34:38.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.892+0000 7fd2017fa640 1 -- 172.21.15.12:0/4176406739 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8003bf0 con 0x7fd20406af30 2024-04-22T13:34:38.890 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.892+0000 7fd20a49a640 1 -- 172.21.15.12:0/4176406739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2041a5270 con 0x7fd20406af30 2024-04-22T13:34:38.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.893+0000 7fd20a49a640 1 -- 172.21.15.12:0/4176406739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2040691c0 con 0x7fd20406af30 2024-04-22T13:34:38.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.894+0000 7fd2017fa640 1 -- 172.21.15.12:0/4176406739 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd1f8046020 con 0x7fd20406af30 2024-04-22T13:34:38.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.895+0000 7fd2017fa640 1 --2- 172.21.15.12:0/4176406739 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd1ec0a3d20 0x7fd1ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:38.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.895+0000 7fd203fff640 1 --2- 172.21.15.12:0/4176406739 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd1ec0a3d20 0x7fd1ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:38.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.895+0000 7fd2017fa640 1 -- 172.21.15.12:0/4176406739 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7fd1f80f2a10 con 0x7fd20406af30 2024-04-22T13:34:38.891 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.895+0000 7fd203fff640 1 --2- 172.21.15.12:0/4176406739 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd1ec0a3d20 0x7fd1ec0a61e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400abe0 tx=0x7fd1f400a2d0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:38.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:38.900+0000 7fd2017fa640 1 -- 172.21.15.12:0/4176406739 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd1f804d020 con 0x7fd20406af30 2024-04-22T13:34:38.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:38 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/563311455' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:34:38.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:38 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/563311455' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:34:39.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.084+0000 7fd20a49a640 1 -- 172.21.15.12:0/4176406739 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd20410a520 con 0x7fd20406af30 2024-04-22T13:34:39.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.084+0000 7fd2017fa640 1 -- 172.21.15.12:0/4176406739 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) ==== 104+0+11 (secure 0 0 0) 0x7fd1f80bf180 con 0x7fd20406af30 2024-04-22T13:34:39.080 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-22T13:34:39.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.086+0000 7fd20a49a640 1 -- 172.21.15.12:0/4176406739 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd1ec0a3d20 msgr2=0x7fd1ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:39.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.086+0000 7fd20a49a640 1 --2- 172.21.15.12:0/4176406739 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd1ec0a3d20 0x7fd1ec0a61e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400abe0 tx=0x7fd1f400a2d0 comp rx=0 tx=0).stop 2024-04-22T13:34:39.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.086+0000 7fd20a49a640 1 -- 172.21.15.12:0/4176406739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd20406af30 msgr2=0x7fd204198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:39.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.086+0000 7fd20a49a640 1 --2- 172.21.15.12:0/4176406739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd20406af30 0x7fd204198c90 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8005940 tx=0x7fd1f802ca90 comp rx=0 tx=0).stop 2024-04-22T13:34:39.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.086+0000 7fd20a49a640 1 -- 172.21.15.12:0/4176406739 shutdown_connections 2024-04-22T13:34:39.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.086+0000 7fd20a49a640 1 --2- 172.21.15.12:0/4176406739 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fd1ec0a3d20 0x7fd1ec0a61e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:39.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.086+0000 7fd20a49a640 1 --2- 172.21.15.12:0/4176406739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd204106f90 0x7fd20419cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:39.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.086+0000 7fd20a49a640 1 --2- 172.21.15.12:0/4176406739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd20406af30 0x7fd204198c90 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:39.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.086+0000 7fd20a49a640 1 --2- 172.21.15.12:0/4176406739 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd20406a570 0x7fd204198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:39.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.086+0000 7fd20a49a640 1 -- 172.21.15.12:0/4176406739 >> 172.21.15.12:0/4176406739 conn(0x7fd204077e30 msgr2=0x7fd2040feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:39.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.087+0000 7fd20a49a640 1 -- 172.21.15.12:0/4176406739 shutdown_connections 2024-04-22T13:34:39.082 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.087+0000 7fd20a49a640 1 -- 172.21.15.12:0/4176406739 wait complete. 2024-04-22T13:34:39.093 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T13:34:39.199 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.202+0000 7ffbe2385640 1 -- 172.21.15.12:0/3520318641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbdc102b40 msgr2=0x7ffbdc106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:39.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.202+0000 7ffbe2385640 1 --2- 172.21.15.12:0/3520318641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbdc102b40 0x7ffbdc106b80 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0009900 tx=0x7ffbd002bc30 comp rx=0 tx=0).stop 2024-04-22T13:34:39.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.202+0000 7ffbe2385640 1 -- 172.21.15.12:0/3520318641 shutdown_connections 2024-04-22T13:34:39.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.202+0000 7ffbe2385640 1 --2- 172.21.15.12:0/3520318641 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffbdc107260 0x7ffbdc10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:39.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.202+0000 7ffbe2385640 1 --2- 172.21.15.12:0/3520318641 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbdc102b40 0x7ffbdc106b80 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:39.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.202+0000 7ffbe2385640 1 --2- 172.21.15.12:0/3520318641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbdc102190 0x7ffbdc102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:39.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.202+0000 7ffbe2385640 1 -- 172.21.15.12:0/3520318641 >> 172.21.15.12:0/3520318641 conn(0x7ffbdc0fd990 msgr2=0x7ffbdc0ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:39.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.202+0000 7ffbe2385640 1 -- 172.21.15.12:0/3520318641 shutdown_connections 2024-04-22T13:34:39.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.202+0000 7ffbe2385640 1 -- 172.21.15.12:0/3520318641 wait complete. 2024-04-22T13:34:39.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.203+0000 7ffbe2385640 1 Processor -- start 2024-04-22T13:34:39.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.203+0000 7ffbe2385640 1 -- start start 2024-04-22T13:34:39.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.203+0000 7ffbe2385640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbdc102190 0x7ffbdc074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:39.200 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.203+0000 7ffbe2385640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffbdc107260 0x7ffbdc074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:39.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.203+0000 7ffbe2385640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbdc074cd0 0x7ffbdc1a6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:39.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.203+0000 7ffbe2385640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7ffbdc1121a0 con 0x7ffbdc102190 2024-04-22T13:34:39.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.203+0000 7ffbe2385640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ffbdc112020 con 0x7ffbdc107260 2024-04-22T13:34:39.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.203+0000 7ffbe2385640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffbdc112320 con 0x7ffbdc074cd0 2024-04-22T13:34:39.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.203+0000 7ffbdbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbdc102190 0x7ffbdc074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:39.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.203+0000 7ffbdbfff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbdc102190 0x7ffbdc074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50178/0 (socket says 172.21.15.12:50178) 2024-04-22T13:34:39.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.203+0000 7ffbdbfff640 1 -- 172.21.15.12:0/1348419328 learned_addr learned my addr 172.21.15.12:0/1348419328 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:39.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.203+0000 7ffbdb7fe640 1 --2- 172.21.15.12:0/1348419328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffbdc107260 0x7ffbdc074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:39.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.203+0000 7ffbe0922640 1 --2- 172.21.15.12:0/1348419328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbdc074cd0 0x7ffbdc1a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:39.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.203+0000 7ffbdbfff640 1 -- 172.21.15.12:0/1348419328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbdc074cd0 msgr2=0x7ffbdc1a6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:39.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.203+0000 7ffbdbfff640 1 --2- 172.21.15.12:0/1348419328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbdc074cd0 0x7ffbdc1a6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:39.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.203+0000 7ffbdbfff640 1 -- 172.21.15.12:0/1348419328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffbdc107260 msgr2=0x7ffbdc074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:39.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.203+0000 7ffbdbfff640 1 --2- 172.21.15.12:0/1348419328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffbdc107260 0x7ffbdc074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:39.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.204+0000 7ffbdbfff640 1 -- 172.21.15.12:0/1348419328 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbdc1a7050 con 0x7ffbdc102190 2024-04-22T13:34:39.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.204+0000 7ffbdbfff640 1 --2- 172.21.15.12:0/1348419328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbdc102190 0x7ffbdc074250 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00e9f0 tx=0x7ffbcc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:39.201 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.204+0000 7ffbd97fa640 1 -- 172.21.15.12:0/1348419328 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc00cd80 con 0x7ffbdc102190 2024-04-22T13:34:39.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.204+0000 7ffbd97fa640 1 -- 172.21.15.12:0/1348419328 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbcc0054d0 con 0x7ffbdc102190 2024-04-22T13:34:39.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.204+0000 7ffbe2385640 1 -- 172.21.15.12:0/1348419328 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbdc1a7340 con 0x7ffbdc102190 2024-04-22T13:34:39.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.204+0000 7ffbd97fa640 1 -- 172.21.15.12:0/1348419328 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc010690 con 0x7ffbdc102190 2024-04-22T13:34:39.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.204+0000 7ffbe2385640 1 -- 172.21.15.12:0/1348419328 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbdc1a7810 con 0x7ffbdc102190 2024-04-22T13:34:39.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.205+0000 7ffbe2385640 1 -- 172.21.15.12:0/1348419328 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbdc0691c0 con 0x7ffbdc102190 2024-04-22T13:34:39.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.207+0000 7ffbd97fa640 1 -- 172.21.15.12:0/1348419328 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ffbcc004e30 con 0x7ffbdc102190 2024-04-22T13:34:39.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.207+0000 7ffbd97fa640 1 --2- 172.21.15.12:0/1348419328 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ffbc40a3d50 0x7ffbc40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:39.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.207+0000 7ffbdb7fe640 1 --2- 172.21.15.12:0/1348419328 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ffbc40a3d50 0x7ffbc40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:39.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.207+0000 7ffbd97fa640 1 -- 172.21.15.12:0/1348419328 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7ffbcc0d0dd0 con 0x7ffbdc102190 2024-04-22T13:34:39.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.208+0000 7ffbdb7fe640 1 --2- 172.21.15.12:0/1348419328 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ffbc40a3d50 0x7ffbc40a6210 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0009230 tx=0x7ffbd0005fd0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:39.206 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.211+0000 7ffbd97fa640 1 -- 172.21.15.12:0/1348419328 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffbcc053ff0 con 0x7ffbdc102190 2024-04-22T13:34:39.392 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.397+0000 7ffbe2385640 1 -- 172.21.15.12:0/1348419328 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ffbdc10a490 con 0x7ffbdc102190 2024-04-22T13:34:39.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.397+0000 7ffbd97fa640 1 -- 172.21.15.12:0/1348419328 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) ==== 100+0+11 (secure 0 0 0) 0x7ffbcc09d490 con 0x7ffbdc102190 2024-04-22T13:34:39.393 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-22T13:34:39.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.399+0000 7ffbe2385640 1 -- 172.21.15.12:0/1348419328 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ffbc40a3d50 msgr2=0x7ffbc40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:39.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.399+0000 7ffbe2385640 1 --2- 172.21.15.12:0/1348419328 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ffbc40a3d50 0x7ffbc40a6210 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0009230 tx=0x7ffbd0005fd0 comp rx=0 tx=0).stop 2024-04-22T13:34:39.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.399+0000 7ffbe2385640 1 -- 172.21.15.12:0/1348419328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbdc102190 msgr2=0x7ffbdc074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:39.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.399+0000 7ffbe2385640 1 --2- 172.21.15.12:0/1348419328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbdc102190 0x7ffbdc074250 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00e9f0 tx=0x7ffbcc00edf0 comp rx=0 tx=0).stop 2024-04-22T13:34:39.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.399+0000 7ffbe2385640 1 -- 172.21.15.12:0/1348419328 shutdown_connections 2024-04-22T13:34:39.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.399+0000 7ffbe2385640 1 --2- 172.21.15.12:0/1348419328 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7ffbc40a3d50 0x7ffbc40a6210 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:39.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.399+0000 7ffbe2385640 1 --2- 172.21.15.12:0/1348419328 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffbdc074cd0 0x7ffbdc1a6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:39.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.399+0000 7ffbe2385640 1 --2- 172.21.15.12:0/1348419328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffbdc107260 0x7ffbdc074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:39.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.399+0000 7ffbe2385640 1 --2- 172.21.15.12:0/1348419328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ffbdc102190 0x7ffbdc074250 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:39.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.399+0000 7ffbe2385640 1 -- 172.21.15.12:0/1348419328 >> 172.21.15.12:0/1348419328 conn(0x7ffbdc0fd990 msgr2=0x7ffbdc0fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:39.395 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.399+0000 7ffbe2385640 1 -- 172.21.15.12:0/1348419328 shutdown_connections 2024-04-22T13:34:39.396 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.399+0000 7ffbe2385640 1 -- 172.21.15.12:0/1348419328 wait complete. 2024-04-22T13:34:39.405 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-22T13:34:39.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.513+0000 7fa298cc1640 1 -- 172.21.15.12:0/3784359717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa294102b40 msgr2=0x7fa294106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:39.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.513+0000 7fa298cc1640 1 --2- 172.21.15.12:0/3784359717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa294102b40 0x7fa294106b80 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fa288009900 tx=0x7fa28802bc30 comp rx=0 tx=0).stop 2024-04-22T13:34:39.510 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.513+0000 7fa298cc1640 1 -- 172.21.15.12:0/3784359717 shutdown_connections 2024-04-22T13:34:39.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.513+0000 7fa298cc1640 1 --2- 172.21.15.12:0/3784359717 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa294107260 0x7fa29410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:39.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.513+0000 7fa298cc1640 1 --2- 172.21.15.12:0/3784359717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa294102b40 0x7fa294106b80 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:39.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.513+0000 7fa298cc1640 1 --2- 172.21.15.12:0/3784359717 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa294102190 0x7fa294102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:39.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.513+0000 7fa298cc1640 1 -- 172.21.15.12:0/3784359717 >> 172.21.15.12:0/3784359717 conn(0x7fa294079310 msgr2=0x7fa294079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:39.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.513+0000 7fa298cc1640 1 -- 172.21.15.12:0/3784359717 shutdown_connections 2024-04-22T13:34:39.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.513+0000 7fa298cc1640 1 -- 172.21.15.12:0/3784359717 wait complete. 2024-04-22T13:34:39.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.513+0000 7fa298cc1640 1 Processor -- start 2024-04-22T13:34:39.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.514+0000 7fa298cc1640 1 -- start start 2024-04-22T13:34:39.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.514+0000 7fa298cc1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa294102190 0x7fa294198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:39.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.514+0000 7fa298cc1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa294107260 0x7fa294198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:39.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.514+0000 7fa298cc1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa29419d000 0x7fa29419d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:39.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.514+0000 7fa298cc1640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa294112180 con 0x7fa294107260 2024-04-22T13:34:39.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.514+0000 7fa298cc1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa294112000 con 0x7fa29419d000 2024-04-22T13:34:39.511 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.514+0000 7fa298cc1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa294112300 con 0x7fa294102190 2024-04-22T13:34:39.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.514+0000 7fa291d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa294107260 0x7fa294198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:39.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.514+0000 7fa291d9b640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa294107260 0x7fa294198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:50196/0 (socket says 172.21.15.12:50196) 2024-04-22T13:34:39.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.514+0000 7fa291d9b640 1 -- 172.21.15.12:0/2000617103 learned_addr learned my addr 172.21.15.12:0/2000617103 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:34:39.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.514+0000 7fa29259c640 1 --2- 172.21.15.12:0/2000617103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa294102190 0x7fa294198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:39.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.514+0000 7fa292d9d640 1 --2- 172.21.15.12:0/2000617103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa29419d000 0x7fa29419d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:39.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.514+0000 7fa291d9b640 1 -- 172.21.15.12:0/2000617103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa294102190 msgr2=0x7fa294198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:39.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.514+0000 7fa291d9b640 1 --2- 172.21.15.12:0/2000617103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa294102190 0x7fa294198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:39.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.514+0000 7fa291d9b640 1 -- 172.21.15.12:0/2000617103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa29419d000 msgr2=0x7fa29419d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:39.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.514+0000 7fa291d9b640 1 --2- 172.21.15.12:0/2000617103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa29419d000 0x7fa29419d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:39.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.514+0000 7fa291d9b640 1 -- 172.21.15.12:0/2000617103 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa29419daf0 con 0x7fa294107260 2024-04-22T13:34:39.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.514+0000 7fa292d9d640 1 --2- 172.21.15.12:0/2000617103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa29419d000 0x7fa29419d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:34:39.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.514+0000 7fa291d9b640 1 --2- 172.21.15.12:0/2000617103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa294107260 0x7fa294198c80 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fa288036040 tx=0x7fa28802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:39.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.514+0000 7fa2837fe640 1 -- 172.21.15.12:0/2000617103 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa288043020 con 0x7fa294107260 2024-04-22T13:34:39.512 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.515+0000 7fa298cc1640 1 -- 172.21.15.12:0/2000617103 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa29419dd80 con 0x7fa294107260 2024-04-22T13:34:39.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.515+0000 7fa2837fe640 1 -- 172.21.15.12:0/2000617103 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa28802cbe0 con 0x7fa294107260 2024-04-22T13:34:39.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.515+0000 7fa2837fe640 1 -- 172.21.15.12:0/2000617103 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa288003bf0 con 0x7fa294107260 2024-04-22T13:34:39.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.515+0000 7fa298cc1640 1 -- 172.21.15.12:0/2000617103 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2941a55d0 con 0x7fa294107260 2024-04-22T13:34:39.513 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.516+0000 7fa298cc1640 1 -- 172.21.15.12:0/2000617103 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2940691c0 con 0x7fa294107260 2024-04-22T13:34:39.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.517+0000 7fa2837fe640 1 -- 172.21.15.12:0/2000617103 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa288046020 con 0x7fa294107260 2024-04-22T13:34:39.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.517+0000 7fa2837fe640 1 --2- 172.21.15.12:0/2000617103 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa27c0a3d20 0x7fa27c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:34:39.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.518+0000 7fa2837fe640 1 -- 172.21.15.12:0/2000617103 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8484+0+0 (secure 0 0 0) 0x7fa2880f2f90 con 0x7fa294107260 2024-04-22T13:34:39.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.518+0000 7fa29259c640 1 --2- 172.21.15.12:0/2000617103 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa27c0a3d20 0x7fa27c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:34:39.514 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.518+0000 7fa29259c640 1 --2- 172.21.15.12:0/2000617103 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa27c0a3d20 0x7fa27c0a61e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa2840058c0 tx=0x7fa28400a2d0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:34:39.516 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.521+0000 7fa2837fe640 1 -- 172.21.15.12:0/2000617103 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa28804d020 con 0x7fa294107260 2024-04-22T13:34:39.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.758+0000 7fa298cc1640 1 -- 172.21.15.12:0/2000617103 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fa29410a4c0 con 0x7fa294107260 2024-04-22T13:34:39.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.766+0000 7fa2837fe640 1 -- 172.21.15.12:0/2000617103 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746057 (secure 0 0 0) 0x7fa2880bf760 con 0x7fa294107260 2024-04-22T13:34:39.772 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:34:39.773 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:34:39.773 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 26, 2024-04-22T13:34:39.773 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.773 INFO:teuthology.orchestra.run.smithi012.stdout: "active_gid": 24968, 2024-04-22T13:34:39.773 INFO:teuthology.orchestra.run.smithi012.stdout: "active_name": "x", 2024-04-22T13:34:39.773 INFO:teuthology.orchestra.run.smithi012.stdout: "active_addrs": { 2024-04-22T13:34:39.773 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-22T13:34:39.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:39.773 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-22T13:34:39.773 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6800", 2024-04-22T13:34:39.773 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1615661596 2024-04-22T13:34:39.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.773 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:39.774 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-22T13:34:39.774 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6801", 2024-04-22T13:34:39.774 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1615661596 2024-04-22T13:34:39.774 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:39.774 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-22T13:34:39.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.774 INFO:teuthology.orchestra.run.smithi012.stdout: "active_addr": "172.21.15.12:6801/1615661596", 2024-04-22T13:34:39.774 INFO:teuthology.orchestra.run.smithi012.stdout: "active_change": "2024-04-22T13:24:06.155794+0000", 2024-04-22T13:34:39.774 INFO:teuthology.orchestra.run.smithi012.stdout: "active_mgr_features": 4540701547738038271, 2024-04-22T13:34:39.774 INFO:teuthology.orchestra.run.smithi012.stdout: "available": true, 2024-04-22T13:34:39.774 INFO:teuthology.orchestra.run.smithi012.stdout: "standbys": [ 2024-04-22T13:34:39.774 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:39.774 INFO:teuthology.orchestra.run.smithi012.stdout: "gid": 24928, 2024-04-22T13:34:39.775 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "z", 2024-04-22T13:34:39.775 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_features": 4540701547738038271, 2024-04-22T13:34:39.775 INFO:teuthology.orchestra.run.smithi012.stdout: "available_modules": [ 2024-04-22T13:34:39.775 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:39.775 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "alerts", 2024-04-22T13:34:39.775 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:39.775 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:39.775 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:39.775 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-04-22T13:34:39.775 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-04-22T13:34:39.775 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:39.775 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.775 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.775 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-04-22T13:34:39.775 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.776 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.776 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.776 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How frequently to reexamine health status", 2024-04-22T13:34:39.776 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.776 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.776 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.776 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:39.776 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:39.776 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.776 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.776 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.776 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.776 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.777 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.777 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:39.777 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:39.777 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:39.777 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:39.777 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:39.777 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:39.777 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:39.777 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:39.777 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.777 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.777 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.777 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.777 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:39.777 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:39.778 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.778 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.778 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.778 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.778 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.778 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.778 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.778 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.778 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.778 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.778 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.778 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:39.779 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:39.779 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.779 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.779 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.779 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:39.779 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.779 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.779 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:39.779 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:39.779 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:39.779 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:39.779 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:39.779 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:39.779 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:39.779 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:39.779 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.780 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.780 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.780 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.780 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.780 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:39.780 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:39.780 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.780 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.780 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.780 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.780 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.780 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.780 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.780 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.780 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.780 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.781 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.781 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_destination": { 2024-04-22T13:34:39.781 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_destination", 2024-04-22T13:34:39.781 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.781 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.781 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.781 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.781 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.781 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.781 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.781 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email address to send alerts to", 2024-04-22T13:34:39.781 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.781 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.781 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.782 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_from_name": { 2024-04-22T13:34:39.782 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_from_name", 2024-04-22T13:34:39.782 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.782 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.782 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.782 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "Ceph", 2024-04-22T13:34:39.782 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.782 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.782 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.782 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email From: name", 2024-04-22T13:34:39.782 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.782 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.782 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.783 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_host": { 2024-04-22T13:34:39.783 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_host", 2024-04-22T13:34:39.783 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.783 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.783 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.783 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.783 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.783 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.783 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.783 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP server", 2024-04-22T13:34:39.783 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.783 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.783 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.783 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_password": { 2024-04-22T13:34:39.784 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_password", 2024-04-22T13:34:39.784 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.784 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.784 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.784 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.784 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.784 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.784 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.784 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Password to authenticate with", 2024-04-22T13:34:39.784 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.784 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.784 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.784 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_port": { 2024-04-22T13:34:39.784 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_port", 2024-04-22T13:34:39.784 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:39.785 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.785 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.785 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "465", 2024-04-22T13:34:39.785 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.785 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.785 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.785 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP port", 2024-04-22T13:34:39.785 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.785 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.785 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.785 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_sender": { 2024-04-22T13:34:39.785 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_sender", 2024-04-22T13:34:39.785 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.785 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.785 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.786 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.786 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.786 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.786 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.786 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP envelope sender", 2024-04-22T13:34:39.786 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.786 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.786 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.786 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_ssl": { 2024-04-22T13:34:39.786 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_ssl", 2024-04-22T13:34:39.786 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.786 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.787 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.787 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:39.787 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.787 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.787 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.787 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-22T13:34:39.787 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.787 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.787 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.787 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_user": { 2024-04-22T13:34:39.787 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_user", 2024-04-22T13:34:39.787 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.787 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.787 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.787 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.788 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.788 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.788 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.788 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "User to authenticate as", 2024-04-22T13:34:39.788 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.788 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.788 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.788 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:39.788 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:39.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.788 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:39.788 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "balancer", 2024-04-22T13:34:39.788 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:39.788 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:39.788 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:39.789 INFO:teuthology.orchestra.run.smithi012.stdout: "active": { 2024-04-22T13:34:39.789 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "active", 2024-04-22T13:34:39.789 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.789 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.789 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.789 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:39.789 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.789 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.789 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.789 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically balance PGs across cluster", 2024-04-22T13:34:39.789 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.789 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.789 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.789 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_time": { 2024-04-22T13:34:39.790 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_time", 2024-04-22T13:34:39.790 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.790 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.790 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.790 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0000", 2024-04-22T13:34:39.790 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.790 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.790 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.790 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "beginning time of day to automatically balance", 2024-04-22T13:34:39.790 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-22T13:34:39.790 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.790 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.790 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_weekday": { 2024-04-22T13:34:39.790 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_weekday", 2024-04-22T13:34:39.791 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-04-22T13:34:39.791 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.791 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.791 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-04-22T13:34:39.791 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-04-22T13:34:39.791 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-04-22T13:34:39.791 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.791 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-22T13:34:39.791 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-22T13:34:39.791 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.791 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.791 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.791 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_max_iterations": { 2024-04-22T13:34:39.791 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_max_iterations", 2024-04-22T13:34:39.791 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-04-22T13:34:39.792 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.792 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.792 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "25", 2024-04-22T13:34:39.792 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-04-22T13:34:39.792 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "250", 2024-04-22T13:34:39.792 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.792 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-22T13:34:39.792 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.792 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.792 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.792 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.792 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_metrics": { 2024-04-22T13:34:39.792 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_metrics", 2024-04-22T13:34:39.792 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.793 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.793 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.793 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "pgs,objects,bytes", 2024-04-22T13:34:39.793 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.793 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.793 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.793 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-22T13:34:39.793 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-22T13:34:39.793 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.793 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.793 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.793 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_step": { 2024-04-22T13:34:39.793 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_step", 2024-04-22T13:34:39.793 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-22T13:34:39.793 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.794 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.794 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.5", 2024-04-22T13:34:39.794 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0.001", 2024-04-22T13:34:39.794 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "0.999", 2024-04-22T13:34:39.794 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.794 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "aggressiveness of optimization", 2024-04-22T13:34:39.794 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-22T13:34:39.794 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.794 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.794 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.794 INFO:teuthology.orchestra.run.smithi012.stdout: "end_time": { 2024-04-22T13:34:39.794 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_time", 2024-04-22T13:34:39.794 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.794 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.795 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.795 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2359", 2024-04-22T13:34:39.795 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.795 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.795 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.795 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ending time of day to automatically balance", 2024-04-22T13:34:39.795 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-22T13:34:39.795 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.795 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.795 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.795 INFO:teuthology.orchestra.run.smithi012.stdout: "end_weekday": { 2024-04-22T13:34:39.795 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_weekday", 2024-04-22T13:34:39.795 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-04-22T13:34:39.795 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.796 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.796 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-04-22T13:34:39.796 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-04-22T13:34:39.796 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-04-22T13:34:39.796 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.796 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-22T13:34:39.796 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-22T13:34:39.796 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.796 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.796 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.796 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:39.796 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:39.796 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.796 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.796 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.796 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.797 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.797 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.797 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:39.797 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:39.797 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:39.797 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:39.797 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:39.797 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:39.797 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:39.797 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:39.797 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.797 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.797 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.797 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.798 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.798 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:39.798 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:39.798 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.798 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.798 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.798 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.798 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.798 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.798 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.798 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.798 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.798 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.799 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.799 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.799 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:39.799 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:39.799 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.799 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.799 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.799 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:39.799 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.799 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.799 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:39.799 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:39.799 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:39.799 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:39.800 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:39.800 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:39.800 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:39.800 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:39.800 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.800 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.800 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.800 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.800 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:39.800 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:39.800 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.800 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.800 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.800 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.801 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.801 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.801 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.801 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.801 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.801 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.801 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.801 INFO:teuthology.orchestra.run.smithi012.stdout: "min_score": { 2024-04-22T13:34:39.801 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_score", 2024-04-22T13:34:39.801 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-22T13:34:39.801 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.801 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.801 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-04-22T13:34:39.802 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.802 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.802 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.802 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-22T13:34:39.802 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.802 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.802 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.802 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.802 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-04-22T13:34:39.802 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-04-22T13:34:39.802 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.802 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.802 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.802 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "upmap", 2024-04-22T13:34:39.803 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.803 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.803 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:39.803 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-compat", 2024-04-22T13:34:39.803 INFO:teuthology.orchestra.run.smithi012.stdout: "none", 2024-04-22T13:34:39.803 INFO:teuthology.orchestra.run.smithi012.stdout: "read", 2024-04-22T13:34:39.803 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap", 2024-04-22T13:34:39.803 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap-read" 2024-04-22T13:34:39.803 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:39.803 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Balancer mode", 2024-04-22T13:34:39.803 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.803 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.803 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.803 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.803 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_ids": { 2024-04-22T13:34:39.804 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_ids", 2024-04-22T13:34:39.804 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.804 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.804 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.804 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.804 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.804 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.804 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.804 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-22T13:34:39.804 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.804 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.804 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.805 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-04-22T13:34:39.805 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-04-22T13:34:39.805 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:39.805 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.805 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.805 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-04-22T13:34:39.805 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.805 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.805 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.805 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-22T13:34:39.805 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.805 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.805 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.805 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_deviation": { 2024-04-22T13:34:39.806 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_deviation", 2024-04-22T13:34:39.806 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:39.806 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.806 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.806 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-04-22T13:34:39.806 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-04-22T13:34:39.806 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.806 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.806 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-22T13:34:39.806 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-22T13:34:39.806 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.806 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.806 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.806 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_optimizations": { 2024-04-22T13:34:39.806 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_optimizations", 2024-04-22T13:34:39.807 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-04-22T13:34:39.807 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.807 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.807 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-22T13:34:39.807 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.807 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.807 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.807 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-22T13:34:39.807 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.807 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.807 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.807 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:39.807 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:39.807 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.808 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:39.808 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm", 2024-04-22T13:34:39.808 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:39.808 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:39.808 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:39.808 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_down_multiplier": { 2024-04-22T13:34:39.808 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_down_multiplier", 2024-04-22T13:34:39.808 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-22T13:34:39.808 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.808 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.808 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-04-22T13:34:39.808 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.808 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.808 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.809 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-22T13:34:39.809 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.809 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.809 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.809 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.809 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_refresh_rate": { 2024-04-22T13:34:39.809 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_refresh_rate", 2024-04-22T13:34:39.809 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:39.809 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.809 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.809 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "20", 2024-04-22T13:34:39.809 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.809 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.809 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.809 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-22T13:34:39.809 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.810 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.810 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.810 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.810 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_starting_port": { 2024-04-22T13:34:39.810 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_starting_port", 2024-04-22T13:34:39.810 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:39.810 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.810 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.810 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4721", 2024-04-22T13:34:39.810 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.810 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.810 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.810 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-22T13:34:39.810 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.811 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.811 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.811 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.811 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_ptrace": { 2024-04-22T13:34:39.811 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_ptrace", 2024-04-22T13:34:39.811 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.811 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.811 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.811 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.811 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.811 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.811 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.811 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-22T13:34:39.811 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-22T13:34:39.811 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.812 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.812 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.812 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_interval": { 2024-04-22T13:34:39.812 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_interval", 2024-04-22T13:34:39.812 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:39.812 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.812 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.812 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-04-22T13:34:39.812 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.812 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.812 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.812 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-22T13:34:39.812 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.812 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.812 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.812 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.813 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_memory_target_ratio": { 2024-04-22T13:34:39.813 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_memory_target_ratio", 2024-04-22T13:34:39.813 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-22T13:34:39.813 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.813 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.813 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.7", 2024-04-22T13:34:39.813 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.813 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.813 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.813 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-22T13:34:39.813 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.813 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.813 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.813 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.813 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm_log_destination": { 2024-04-22T13:34:39.813 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm_log_destination", 2024-04-22T13:34:39.814 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.814 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.814 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.814 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.814 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.814 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.814 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:39.814 INFO:teuthology.orchestra.run.smithi012.stdout: "file", 2024-04-22T13:34:39.814 INFO:teuthology.orchestra.run.smithi012.stdout: "file,syslog", 2024-04-22T13:34:39.814 INFO:teuthology.orchestra.run.smithi012.stdout: "syslog" 2024-04-22T13:34:39.814 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:39.814 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-22T13:34:39.814 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.814 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.815 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.815 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.815 INFO:teuthology.orchestra.run.smithi012.stdout: "cgroups_split": { 2024-04-22T13:34:39.815 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cgroups_split", 2024-04-22T13:34:39.815 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.815 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.815 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.815 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:39.815 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.815 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.815 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.815 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-22T13:34:39.815 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.816 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.816 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.816 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.816 INFO:teuthology.orchestra.run.smithi012.stdout: "config_checks_enabled": { 2024-04-22T13:34:39.816 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_checks_enabled", 2024-04-22T13:34:39.816 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.816 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.816 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.816 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.816 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.816 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.816 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.816 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-22T13:34:39.816 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.816 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.817 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.817 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.817 INFO:teuthology.orchestra.run.smithi012.stdout: "config_dashboard": { 2024-04-22T13:34:39.817 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_dashboard", 2024-04-22T13:34:39.817 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.817 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.817 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.817 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:39.817 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.817 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.817 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.817 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-22T13:34:39.817 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.817 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.818 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.818 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.818 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_alertmanager": { 2024-04-22T13:34:39.818 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_alertmanager", 2024-04-22T13:34:39.818 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.818 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.818 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.818 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-22T13:34:39.818 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.818 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.818 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.818 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-04-22T13:34:39.818 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.818 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.818 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.818 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.819 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_base": { 2024-04-22T13:34:39.819 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_base", 2024-04-22T13:34:39.819 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.819 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.819 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.819 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-22T13:34:39.819 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.819 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.819 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.819 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Container image name, without the tag", 2024-04-22T13:34:39.819 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.819 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.819 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.819 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.819 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_elasticsearch": { 2024-04-22T13:34:39.820 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_elasticsearch", 2024-04-22T13:34:39.820 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.820 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.820 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.820 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-22T13:34:39.820 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.820 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.820 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.820 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "elasticsearch container image", 2024-04-22T13:34:39.820 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.820 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.820 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.820 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.820 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_grafana": { 2024-04-22T13:34:39.820 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_grafana", 2024-04-22T13:34:39.821 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.821 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.821 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.821 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-22T13:34:39.821 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.821 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.821 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.821 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-04-22T13:34:39.821 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.821 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.821 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.821 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.821 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_haproxy": { 2024-04-22T13:34:39.821 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_haproxy", 2024-04-22T13:34:39.822 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.822 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.822 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.822 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-22T13:34:39.822 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.822 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.822 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.822 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "HAproxy container image", 2024-04-22T13:34:39.822 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.822 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.822 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.822 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.822 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_agent": { 2024-04-22T13:34:39.822 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_agent", 2024-04-22T13:34:39.823 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.823 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.823 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.823 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-22T13:34:39.823 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.823 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.823 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.823 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger agent container image", 2024-04-22T13:34:39.823 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.823 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.823 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.823 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.823 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_collector": { 2024-04-22T13:34:39.823 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_collector", 2024-04-22T13:34:39.823 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.824 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.824 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.824 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-22T13:34:39.824 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.824 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.824 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.824 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger collector container image", 2024-04-22T13:34:39.824 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.824 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.824 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.824 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.824 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_query": { 2024-04-22T13:34:39.824 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_query", 2024-04-22T13:34:39.824 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.824 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.825 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.825 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-22T13:34:39.825 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.825 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.825 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.825 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger query container image", 2024-04-22T13:34:39.825 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.825 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.825 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.825 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.825 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_keepalived": { 2024-04-22T13:34:39.825 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_keepalived", 2024-04-22T13:34:39.825 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.825 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.826 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.826 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-22T13:34:39.826 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.826 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.826 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.826 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Keepalived container image", 2024-04-22T13:34:39.826 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.826 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.826 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.826 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.826 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_loki": { 2024-04-22T13:34:39.826 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_loki", 2024-04-22T13:34:39.826 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.826 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.827 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.827 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-22T13:34:39.827 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.827 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.827 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.827 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Loki container image", 2024-04-22T13:34:39.827 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.827 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.827 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.827 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.827 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_node_exporter": { 2024-04-22T13:34:39.827 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_node_exporter", 2024-04-22T13:34:39.827 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.827 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.827 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.828 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-22T13:34:39.828 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.828 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.828 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.828 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-04-22T13:34:39.828 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.828 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.828 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.828 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.828 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_nvmeof": { 2024-04-22T13:34:39.828 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_nvmeof", 2024-04-22T13:34:39.828 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.828 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.828 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.828 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-22T13:34:39.829 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.829 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.829 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.829 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Nvme-of container image", 2024-04-22T13:34:39.829 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.829 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.829 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.829 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.829 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_prometheus": { 2024-04-22T13:34:39.829 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_prometheus", 2024-04-22T13:34:39.829 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.829 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.829 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.829 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-22T13:34:39.830 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.830 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.830 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.830 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-04-22T13:34:39.830 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.830 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.830 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.830 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.830 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_promtail": { 2024-04-22T13:34:39.830 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_promtail", 2024-04-22T13:34:39.830 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.830 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.830 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.830 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-22T13:34:39.831 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.831 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.831 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.831 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Promtail container image", 2024-04-22T13:34:39.831 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.831 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.831 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.831 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.831 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_samba": { 2024-04-22T13:34:39.831 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_samba", 2024-04-22T13:34:39.831 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.831 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.831 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.831 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-22T13:34:39.832 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.832 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.832 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.832 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Samba/SMB container image", 2024-04-22T13:34:39.832 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.832 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.832 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.832 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.832 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_snmp_gateway": { 2024-04-22T13:34:39.832 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_snmp_gateway", 2024-04-22T13:34:39.832 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.832 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.832 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.832 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-22T13:34:39.832 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.832 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.833 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.833 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SNMP Gateway container image", 2024-04-22T13:34:39.833 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.833 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.833 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.833 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.833 INFO:teuthology.orchestra.run.smithi012.stdout: "container_init": { 2024-04-22T13:34:39.833 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_init", 2024-04-22T13:34:39.833 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.833 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.833 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.833 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:39.833 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.834 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.834 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.834 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Run podman/docker with `--init`", 2024-04-22T13:34:39.834 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.834 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.834 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.834 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.834 INFO:teuthology.orchestra.run.smithi012.stdout: "daemon_cache_timeout": { 2024-04-22T13:34:39.834 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "daemon_cache_timeout", 2024-04-22T13:34:39.834 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:39.834 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.834 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.834 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-04-22T13:34:39.834 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.835 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.835 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.835 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-22T13:34:39.835 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.835 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.835 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.835 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.835 INFO:teuthology.orchestra.run.smithi012.stdout: "default_cephadm_command_timeout": { 2024-04-22T13:34:39.835 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_cephadm_command_timeout", 2024-04-22T13:34:39.835 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:39.835 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.835 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.835 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "900", 2024-04-22T13:34:39.836 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.836 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.836 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.836 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-22T13:34:39.836 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.836 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.836 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.836 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.836 INFO:teuthology.orchestra.run.smithi012.stdout: "default_registry": { 2024-04-22T13:34:39.836 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_registry", 2024-04-22T13:34:39.836 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.836 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.836 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.836 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io", 2024-04-22T13:34:39.837 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.837 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.837 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.837 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-22T13:34:39.837 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.837 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.837 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.837 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.837 INFO:teuthology.orchestra.run.smithi012.stdout: "device_cache_timeout": { 2024-04-22T13:34:39.837 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_cache_timeout", 2024-04-22T13:34:39.837 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:39.837 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.837 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.837 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1800", 2024-04-22T13:34:39.838 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.838 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.838 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.838 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache device inventory", 2024-04-22T13:34:39.838 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.838 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.838 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.838 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.838 INFO:teuthology.orchestra.run.smithi012.stdout: "device_enhanced_scan": { 2024-04-22T13:34:39.838 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_enhanced_scan", 2024-04-22T13:34:39.838 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.838 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.838 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.838 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.838 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.839 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.839 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.839 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-22T13:34:39.839 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.839 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.839 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.839 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.839 INFO:teuthology.orchestra.run.smithi012.stdout: "facts_cache_timeout": { 2024-04-22T13:34:39.839 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "facts_cache_timeout", 2024-04-22T13:34:39.839 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:39.839 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.839 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.839 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-04-22T13:34:39.839 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.840 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.840 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.840 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache host facts data", 2024-04-22T13:34:39.840 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.840 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.840 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.840 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.840 INFO:teuthology.orchestra.run.smithi012.stdout: "grafana_dashboards_path": { 2024-04-22T13:34:39.840 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "grafana_dashboards_path", 2024-04-22T13:34:39.840 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.840 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.840 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.840 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-22T13:34:39.840 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.840 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.841 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.841 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-22T13:34:39.841 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.841 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.841 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.841 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.841 INFO:teuthology.orchestra.run.smithi012.stdout: "host_check_interval": { 2024-04-22T13:34:39.841 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "host_check_interval", 2024-04-22T13:34:39.841 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:39.841 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.841 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.841 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-04-22T13:34:39.841 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.841 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.841 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.842 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to perform a host check", 2024-04-22T13:34:39.842 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.842 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.842 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.842 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.842 INFO:teuthology.orchestra.run.smithi012.stdout: "hw_monitoring": { 2024-04-22T13:34:39.842 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hw_monitoring", 2024-04-22T13:34:39.842 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.842 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.842 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.842 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.842 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.842 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.842 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.843 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-22T13:34:39.843 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.843 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.843 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.843 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.843 INFO:teuthology.orchestra.run.smithi012.stdout: "inventory_list_all": { 2024-04-22T13:34:39.843 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "inventory_list_all", 2024-04-22T13:34:39.843 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.843 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.843 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.843 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.843 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.843 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.843 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.843 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-22T13:34:39.844 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.844 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.844 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.844 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.844 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:39.844 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:39.844 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.844 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.844 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.844 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.844 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.844 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.844 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:39.844 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:39.845 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:39.845 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:39.845 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:39.845 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:39.845 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:39.845 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:39.845 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.845 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.845 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.845 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.845 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.845 INFO:teuthology.orchestra.run.smithi012.stdout: "log_refresh_metadata": { 2024-04-22T13:34:39.845 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_refresh_metadata", 2024-04-22T13:34:39.846 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.846 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.846 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.846 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.846 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.846 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.846 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.846 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-22T13:34:39.846 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.846 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.846 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.846 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.846 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:39.847 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:39.847 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.847 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.847 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.847 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:39.847 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.847 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.847 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.847 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-22T13:34:39.847 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.847 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.847 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.847 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.847 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:39.848 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:39.848 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.848 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.848 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.848 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:39.848 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.848 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.848 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:39.848 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:39.848 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:39.848 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:39.848 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:39.848 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:39.848 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:39.848 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:39.848 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.849 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.849 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.849 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.849 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.849 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:39.849 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:39.849 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.849 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.849 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.849 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.849 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.849 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.849 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.849 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.849 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.849 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.850 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.850 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.850 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-22T13:34:39.850 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-22T13:34:39.850 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.850 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.850 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.850 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.850 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.850 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.850 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.850 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-22T13:34:39.850 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.850 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.851 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.851 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.851 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-22T13:34:39.851 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-22T13:34:39.851 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.851 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.851 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.851 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "*", 2024-04-22T13:34:39.851 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.851 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.851 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.851 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-22T13:34:39.851 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.851 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.851 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.851 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.852 INFO:teuthology.orchestra.run.smithi012.stdout: "max_count_per_host": { 2024-04-22T13:34:39.852 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_count_per_host", 2024-04-22T13:34:39.852 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:39.852 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.852 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.852 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-22T13:34:39.852 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.852 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.852 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.852 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of daemons per service per host", 2024-04-22T13:34:39.852 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.852 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.852 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.852 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.852 INFO:teuthology.orchestra.run.smithi012.stdout: "max_osd_draining_count": { 2024-04-22T13:34:39.852 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_osd_draining_count", 2024-04-22T13:34:39.852 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:39.853 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.853 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.853 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-22T13:34:39.853 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.853 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.853 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.853 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-22T13:34:39.853 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.853 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.853 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.853 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.853 INFO:teuthology.orchestra.run.smithi012.stdout: "migration_current": { 2024-04-22T13:34:39.853 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "migration_current", 2024-04-22T13:34:39.853 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:39.853 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.853 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.854 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.854 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.854 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.854 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.854 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "internal - do not modify", 2024-04-22T13:34:39.854 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.854 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.854 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.854 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.854 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-04-22T13:34:39.854 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-04-22T13:34:39.854 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.854 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.855 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.855 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "root", 2024-04-22T13:34:39.855 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.855 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.855 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:39.855 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm-package", 2024-04-22T13:34:39.855 INFO:teuthology.orchestra.run.smithi012.stdout: "root" 2024-04-22T13:34:39.855 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:39.855 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "mode for remote execution of cephadm", 2024-04-22T13:34:39.855 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.855 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.855 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.855 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.855 INFO:teuthology.orchestra.run.smithi012.stdout: "oob_default_addr": { 2024-04-22T13:34:39.855 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "oob_default_addr", 2024-04-22T13:34:39.856 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.856 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.856 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.856 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "169.254.1.1", 2024-04-22T13:34:39.856 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.856 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.856 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.856 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-22T13:34:39.856 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.856 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.856 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.856 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.856 INFO:teuthology.orchestra.run.smithi012.stdout: "prometheus_alerts_path": { 2024-04-22T13:34:39.856 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus_alerts_path", 2024-04-22T13:34:39.856 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.856 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.857 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.857 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-22T13:34:39.857 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.857 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.857 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.857 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-22T13:34:39.857 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.857 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.857 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.857 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.857 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_insecure": { 2024-04-22T13:34:39.857 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_insecure", 2024-04-22T13:34:39.857 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.858 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.858 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.858 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.858 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.858 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.858 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.858 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-22T13:34:39.858 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.858 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.858 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.858 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.858 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_password": { 2024-04-22T13:34:39.858 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_password", 2024-04-22T13:34:39.858 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.859 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.859 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.859 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.859 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.859 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.859 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.859 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-22T13:34:39.859 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.859 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.859 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.859 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.859 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_url": { 2024-04-22T13:34:39.859 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_url", 2024-04-22T13:34:39.859 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.860 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.860 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.860 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.860 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.860 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.860 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.860 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-22T13:34:39.860 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.860 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.860 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.860 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.860 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_username": { 2024-04-22T13:34:39.860 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_username", 2024-04-22T13:34:39.860 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.860 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.861 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.861 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.861 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.861 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.861 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.861 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-22T13:34:39.861 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.861 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.861 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.861 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.861 INFO:teuthology.orchestra.run.smithi012.stdout: "secure_monitoring_stack": { 2024-04-22T13:34:39.861 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "secure_monitoring_stack", 2024-04-22T13:34:39.861 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.861 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.861 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.862 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.862 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.862 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.862 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.862 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-22T13:34:39.862 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.862 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.862 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.862 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.862 INFO:teuthology.orchestra.run.smithi012.stdout: "service_discovery_port": { 2024-04-22T13:34:39.862 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "service_discovery_port", 2024-04-22T13:34:39.862 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:39.862 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.862 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.863 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8765", 2024-04-22T13:34:39.863 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.863 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.863 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.863 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "cephadm service discovery port", 2024-04-22T13:34:39.863 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.863 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.863 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.863 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.863 INFO:teuthology.orchestra.run.smithi012.stdout: "ssh_config_file": { 2024-04-22T13:34:39.863 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssh_config_file", 2024-04-22T13:34:39.863 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.863 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.863 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.863 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.864 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.864 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.864 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.864 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-22T13:34:39.864 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.864 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.864 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.864 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.864 INFO:teuthology.orchestra.run.smithi012.stdout: "use_agent": { 2024-04-22T13:34:39.864 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_agent", 2024-04-22T13:34:39.864 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.864 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.864 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.864 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.864 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.865 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.865 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.865 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-22T13:34:39.865 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.865 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.865 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.865 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.865 INFO:teuthology.orchestra.run.smithi012.stdout: "use_repo_digest": { 2024-04-22T13:34:39.865 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_repo_digest", 2024-04-22T13:34:39.865 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.865 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.865 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.865 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:39.865 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.865 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.866 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.866 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-22T13:34:39.866 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.866 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.866 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.866 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.866 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_failed_host_check": { 2024-04-22T13:34:39.866 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_failed_host_check", 2024-04-22T13:34:39.866 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.866 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.866 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.866 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:39.866 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.866 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.866 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.867 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if the host check fails", 2024-04-22T13:34:39.867 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.867 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.867 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.867 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.867 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_daemons": { 2024-04-22T13:34:39.867 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_daemons", 2024-04-22T13:34:39.867 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.867 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.867 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.867 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:39.867 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.867 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.867 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.867 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-22T13:34:39.868 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.868 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.868 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.868 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.868 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_hosts": { 2024-04-22T13:34:39.868 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_hosts", 2024-04-22T13:34:39.868 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.868 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.868 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.868 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:39.868 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.868 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.868 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.868 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-22T13:34:39.868 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.868 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.868 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.869 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:39.869 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:39.869 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.869 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:39.869 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crash", 2024-04-22T13:34:39.869 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:39.869 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:39.869 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:39.869 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:39.869 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:39.869 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.869 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.869 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.869 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.869 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.869 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.870 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:39.870 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:39.870 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:39.870 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:39.870 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:39.870 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:39.870 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:39.870 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:39.870 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.870 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.870 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.870 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.870 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.870 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:39.871 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:39.871 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.871 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.871 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.871 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.871 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.871 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.871 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.871 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.871 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.871 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.871 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.871 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.871 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:39.871 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:39.872 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.872 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.872 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.872 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:39.872 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.872 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.872 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:39.872 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:39.872 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:39.872 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:39.872 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:39.872 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:39.872 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:39.872 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:39.872 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.873 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.873 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.873 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.873 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.873 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:39.873 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:39.873 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.873 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.873 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.873 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.873 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.873 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.873 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.873 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.873 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.873 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.873 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.874 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.874 INFO:teuthology.orchestra.run.smithi012.stdout: "retain_interval": { 2024-04-22T13:34:39.874 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retain_interval", 2024-04-22T13:34:39.874 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:39.874 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.874 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.874 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "31536000", 2024-04-22T13:34:39.874 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.874 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.874 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.874 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-22T13:34:39.874 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.874 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.874 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.875 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.875 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_recent_interval": { 2024-04-22T13:34:39.875 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_recent_interval", 2024-04-22T13:34:39.875 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:39.875 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.875 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.875 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1209600", 2024-04-22T13:34:39.875 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.875 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.875 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.875 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-22T13:34:39.875 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.875 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.875 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.876 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:39.876 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:39.876 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.876 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:39.876 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dashboard", 2024-04-22T13:34:39.876 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:39.876 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:39.876 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:39.876 INFO:teuthology.orchestra.run.smithi012.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-22T13:34:39.876 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-22T13:34:39.876 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:39.876 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.876 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.876 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-22T13:34:39.876 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.876 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.876 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.877 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.877 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.877 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.877 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.877 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.877 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_HOST": { 2024-04-22T13:34:39.877 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-22T13:34:39.877 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.877 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.877 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.877 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.877 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.877 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.877 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.877 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.878 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.878 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.878 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.878 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.878 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-22T13:34:39.878 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-22T13:34:39.878 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.878 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.878 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.878 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:39.878 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.878 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.878 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.878 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.879 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.879 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.879 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.879 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.879 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_ENABLED": { 2024-04-22T13:34:39.879 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_ENABLED", 2024-04-22T13:34:39.879 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.879 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.879 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.879 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.879 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.879 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.879 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.879 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.879 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.880 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.880 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.880 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.880 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-22T13:34:39.880 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-22T13:34:39.880 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.880 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.880 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.880 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:39.880 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.880 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.880 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.880 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.880 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.880 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.880 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.881 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.881 INFO:teuthology.orchestra.run.smithi012.stdout: "ENABLE_BROWSABLE_API": { 2024-04-22T13:34:39.881 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-22T13:34:39.881 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.881 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.881 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.881 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:39.881 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.881 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.881 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.881 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.881 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.881 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.881 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.881 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.882 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-22T13:34:39.882 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-22T13:34:39.882 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.882 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.882 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.882 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:39.882 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.882 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.882 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.882 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.882 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.882 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.882 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.882 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.883 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-22T13:34:39.883 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-22T13:34:39.883 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.883 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.883 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.883 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:39.883 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.883 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.883 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.883 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.883 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.883 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.883 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.883 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.883 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-22T13:34:39.884 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-22T13:34:39.884 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.884 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.884 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.884 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:39.884 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.884 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.884 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.884 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.884 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.884 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.884 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.884 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.884 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-22T13:34:39.884 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-22T13:34:39.885 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.885 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.885 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.885 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:39.885 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.885 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.885 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.885 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.885 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.885 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.885 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.885 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.885 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-22T13:34:39.885 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-22T13:34:39.885 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.886 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.886 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.886 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:39.886 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.886 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.886 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.886 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.886 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.886 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.886 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.886 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.886 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-22T13:34:39.886 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-22T13:34:39.886 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.887 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.887 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.887 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:39.887 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.887 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.887 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.887 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.887 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.887 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.887 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.887 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.887 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-22T13:34:39.887 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-22T13:34:39.887 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.887 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.888 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.888 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:39.888 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.888 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.888 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.888 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.888 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.888 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.888 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.888 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.888 INFO:teuthology.orchestra.run.smithi012.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-22T13:34:39.888 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-22T13:34:39.888 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.888 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.888 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.889 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.889 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.889 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.889 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.889 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.889 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.889 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.889 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.889 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.889 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_PASSWORD": { 2024-04-22T13:34:39.889 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-22T13:34:39.889 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.889 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.889 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.889 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-04-22T13:34:39.890 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.890 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.890 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.890 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.890 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.890 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.890 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.890 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.890 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-22T13:34:39.890 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-22T13:34:39.890 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.890 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.890 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.890 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:39.891 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.891 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.891 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.891 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.891 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.891 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.891 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.891 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.891 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_URL": { 2024-04-22T13:34:39.891 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_URL", 2024-04-22T13:34:39.891 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.891 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.891 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.891 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.891 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.891 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.891 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.892 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.892 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.892 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.892 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.892 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.892 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_USERNAME": { 2024-04-22T13:34:39.892 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-22T13:34:39.892 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.892 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.892 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.892 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-04-22T13:34:39.892 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.892 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.892 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.893 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.893 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.893 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.893 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.893 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-22T13:34:39.893 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-22T13:34:39.893 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.893 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.893 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.893 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.893 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.893 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.893 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.893 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.893 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.894 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.894 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.894 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.894 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-22T13:34:39.894 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-22T13:34:39.894 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.894 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.894 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.894 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.894 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.894 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.894 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.894 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.895 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.895 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.895 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.895 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.895 INFO:teuthology.orchestra.run.smithi012.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-22T13:34:39.895 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-22T13:34:39.895 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.895 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.895 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.895 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:39.895 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.895 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.895 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.895 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.896 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.896 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.896 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.896 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.896 INFO:teuthology.orchestra.run.smithi012.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-22T13:34:39.896 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-22T13:34:39.896 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.896 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.896 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.896 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.896 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.896 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.896 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.896 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.897 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.897 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.897 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.897 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.897 INFO:teuthology.orchestra.run.smithi012.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-22T13:34:39.897 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-22T13:34:39.897 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.897 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.897 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.897 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "[]", 2024-04-22T13:34:39.897 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.897 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.897 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.897 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.898 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.898 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.898 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.898 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.898 INFO:teuthology.orchestra.run.smithi012.stdout: "MULTICLUSTER_CONFIG": { 2024-04-22T13:34:39.898 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-22T13:34:39.898 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.898 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.898 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.898 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "{}", 2024-04-22T13:34:39.898 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.898 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.898 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.898 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.899 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.899 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.899 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.899 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.899 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_HOST": { 2024-04-22T13:34:39.899 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-22T13:34:39.899 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.899 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.899 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.899 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.899 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.899 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.899 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.899 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.899 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.900 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.900 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.900 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.900 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-22T13:34:39.900 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-22T13:34:39.900 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.900 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.900 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.900 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:39.900 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.900 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.900 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.900 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.900 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.901 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.901 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.901 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.901 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-22T13:34:39.901 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-22T13:34:39.901 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.901 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.901 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.901 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.901 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.901 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.901 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.901 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.901 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.901 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.902 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.902 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.902 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-22T13:34:39.902 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-22T13:34:39.902 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.902 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.902 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.902 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.902 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.902 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.902 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.902 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.902 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.903 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.903 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.903 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-22T13:34:39.903 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-22T13:34:39.903 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.903 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.903 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.903 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:39.903 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.903 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.903 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.903 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.903 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.903 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.903 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.904 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-22T13:34:39.904 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-22T13:34:39.904 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.904 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.904 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.904 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:39.904 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.904 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.904 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.904 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.904 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.904 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.904 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.905 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-22T13:34:39.905 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-22T13:34:39.905 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.905 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.905 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.905 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.905 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.905 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.905 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.905 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.905 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.905 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.905 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.906 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-22T13:34:39.906 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-22T13:34:39.906 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.906 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.906 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.906 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.906 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.906 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.906 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.906 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.906 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.906 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.906 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.906 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.907 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-22T13:34:39.907 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-22T13:34:39.907 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.907 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.907 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.907 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.907 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.907 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.907 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.907 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.907 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.907 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.907 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.908 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_ENABLED": { 2024-04-22T13:34:39.908 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-22T13:34:39.908 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.908 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.908 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.908 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:39.908 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.908 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.908 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.908 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.908 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.908 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.908 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.909 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.909 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-22T13:34:39.909 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-22T13:34:39.909 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.909 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.909 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.909 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-22T13:34:39.909 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.909 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.909 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.909 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.909 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.909 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.909 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.909 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.909 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-22T13:34:39.910 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-22T13:34:39.910 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:39.910 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.910 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.910 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-22T13:34:39.910 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.910 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.910 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.910 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.910 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.910 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.910 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.910 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.910 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-22T13:34:39.910 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-22T13:34:39.911 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:39.911 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.911 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.911 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8", 2024-04-22T13:34:39.911 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.911 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.911 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.911 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.911 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.911 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.911 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.911 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.911 INFO:teuthology.orchestra.run.smithi012.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-22T13:34:39.911 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-22T13:34:39.911 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:39.912 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.912 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.912 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "45", 2024-04-22T13:34:39.912 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.912 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.912 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.912 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.912 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.912 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.912 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.912 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.912 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ACCESS_KEY": { 2024-04-22T13:34:39.912 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-22T13:34:39.912 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.912 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.913 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.913 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.913 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.913 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.913 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.913 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.913 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.913 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.913 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.913 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-22T13:34:39.913 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-22T13:34:39.913 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.913 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.913 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.913 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-04-22T13:34:39.914 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.914 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.914 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.914 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.914 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.914 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.914 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.914 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SECRET_KEY": { 2024-04-22T13:34:39.914 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-22T13:34:39.914 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.914 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.914 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.914 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.914 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.915 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.915 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.915 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.915 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.915 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.915 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.915 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.915 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SSL_VERIFY": { 2024-04-22T13:34:39.915 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-22T13:34:39.915 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.915 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.915 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.915 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:39.915 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.915 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.916 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.916 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.916 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.916 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.916 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.916 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.916 INFO:teuthology.orchestra.run.smithi012.stdout: "UNSAFE_TLS_v1_2": { 2024-04-22T13:34:39.916 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-22T13:34:39.916 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.916 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.916 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.916 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.916 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.916 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.916 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.917 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.917 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.917 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.917 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.917 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-22T13:34:39.917 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-22T13:34:39.917 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:39.917 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.917 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.917 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-04-22T13:34:39.917 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.917 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.917 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.917 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.917 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.918 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.918 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.918 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-22T13:34:39.918 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-22T13:34:39.918 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:39.918 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.918 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.918 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-22T13:34:39.918 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.918 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.918 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.918 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.918 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.918 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.919 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.919 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.919 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-22T13:34:39.919 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-22T13:34:39.919 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:39.919 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.919 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.919 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-04-22T13:34:39.919 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.919 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.919 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.919 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.919 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.919 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.919 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.920 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.920 INFO:teuthology.orchestra.run.smithi012.stdout: "cross_origin_url": { 2024-04-22T13:34:39.920 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cross_origin_url", 2024-04-22T13:34:39.920 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.920 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.920 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.920 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.920 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.920 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.920 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.920 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.920 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.920 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.920 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.920 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.921 INFO:teuthology.orchestra.run.smithi012.stdout: "crt_file": { 2024-04-22T13:34:39.921 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crt_file", 2024-04-22T13:34:39.921 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.921 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.921 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.921 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.921 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.921 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.921 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.921 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.921 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.921 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.921 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.921 INFO:teuthology.orchestra.run.smithi012.stdout: "debug": { 2024-04-22T13:34:39.922 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "debug", 2024-04-22T13:34:39.922 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.922 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.922 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.922 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.922 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.922 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.922 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.922 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable/disable debug options", 2024-04-22T13:34:39.922 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.922 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.922 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.922 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.922 INFO:teuthology.orchestra.run.smithi012.stdout: "jwt_token_ttl": { 2024-04-22T13:34:39.922 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "jwt_token_ttl", 2024-04-22T13:34:39.923 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:39.923 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.923 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.923 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "28800", 2024-04-22T13:34:39.923 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.923 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.923 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.923 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.923 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.923 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.923 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.923 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-04-22T13:34:39.923 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-04-22T13:34:39.923 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.924 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.924 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.924 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.924 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.924 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.924 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.924 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.924 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.924 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.924 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.924 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:39.924 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:39.924 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.925 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.925 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.925 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.925 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.925 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.925 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:39.925 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:39.925 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:39.925 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:39.925 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:39.925 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:39.925 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:39.925 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:39.925 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.926 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.926 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.926 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.926 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:39.926 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:39.926 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.926 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.926 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.926 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.926 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.926 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.926 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.926 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.927 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.927 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.927 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.927 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:39.927 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:39.927 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.927 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.927 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.927 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:39.927 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.927 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.927 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:39.927 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:39.927 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:39.928 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:39.928 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:39.928 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:39.928 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:39.928 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:39.928 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.928 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.928 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.928 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.928 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:39.928 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:39.928 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.928 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.928 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.928 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.929 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.929 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.929 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.929 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.929 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.929 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.929 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.929 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.929 INFO:teuthology.orchestra.run.smithi012.stdout: "motd": { 2024-04-22T13:34:39.929 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "motd", 2024-04-22T13:34:39.929 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.929 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.929 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.929 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.929 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.929 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.929 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.930 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "The message of the day", 2024-04-22T13:34:39.930 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.930 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.930 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.930 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.930 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect_resolve_ip_addr": { 2024-04-22T13:34:39.930 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "redirect_resolve_ip_addr", 2024-04-22T13:34:39.930 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.930 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.930 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.930 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.930 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.930 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.930 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.930 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.931 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.931 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.931 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.931 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-04-22T13:34:39.931 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-04-22T13:34:39.931 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.931 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.931 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.931 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-04-22T13:34:39.931 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.931 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.931 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.931 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.931 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.932 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.932 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.932 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-04-22T13:34:39.932 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-04-22T13:34:39.932 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:39.932 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.932 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.932 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8080", 2024-04-22T13:34:39.932 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.932 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.932 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.932 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.932 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.932 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.932 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.933 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-04-22T13:34:39.933 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-04-22T13:34:39.933 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.933 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.933 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.933 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:39.933 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.933 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.933 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.933 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.933 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.933 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.933 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.933 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl_server_port": { 2024-04-22T13:34:39.934 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl_server_port", 2024-04-22T13:34:39.934 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:39.934 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.934 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.934 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8443", 2024-04-22T13:34:39.934 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.934 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.934 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.934 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.934 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.934 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.934 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.934 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-04-22T13:34:39.935 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-04-22T13:34:39.935 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.935 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.935 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.935 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "redirect", 2024-04-22T13:34:39.935 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.935 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.935 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:39.935 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:39.935 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect" 2024-04-22T13:34:39.935 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:39.935 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.935 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.935 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.935 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.936 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-04-22T13:34:39.936 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-04-22T13:34:39.936 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:39.936 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.936 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.936 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-04-22T13:34:39.936 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-04-22T13:34:39.936 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-04-22T13:34:39.936 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.936 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.936 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.936 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.936 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.937 INFO:teuthology.orchestra.run.smithi012.stdout: "url_prefix": { 2024-04-22T13:34:39.937 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url_prefix", 2024-04-22T13:34:39.937 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.937 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.937 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.937 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.937 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.937 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.937 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.937 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.937 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.937 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.937 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.937 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:39.937 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:39.938 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.938 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:39.938 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "devicehealth", 2024-04-22T13:34:39.938 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:39.938 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:39.938 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:39.938 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_monitoring": { 2024-04-22T13:34:39.938 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_monitoring", 2024-04-22T13:34:39.938 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.938 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.938 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.938 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:39.938 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.938 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.938 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.939 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "monitor device health metrics", 2024-04-22T13:34:39.939 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.939 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.939 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.939 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:39.939 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:39.939 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.939 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.939 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.939 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.939 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.939 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.939 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:39.939 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:39.940 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:39.940 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:39.940 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:39.940 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:39.940 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:39.940 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:39.940 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.940 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.940 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.940 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.940 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:39.940 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:39.940 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.940 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.940 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.941 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.941 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.941 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.941 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.941 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.941 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.941 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.941 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.941 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:39.941 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:39.941 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.941 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.941 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.941 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:39.941 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.941 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.942 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:39.942 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:39.942 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:39.942 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:39.942 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:39.942 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:39.942 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:39.942 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:39.942 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.942 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.942 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.942 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.942 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:39.942 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:39.943 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.943 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.943 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.943 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.943 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.943 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.943 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.943 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.943 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.943 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.943 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.943 INFO:teuthology.orchestra.run.smithi012.stdout: "mark_out_threshold": { 2024-04-22T13:34:39.943 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mark_out_threshold", 2024-04-22T13:34:39.943 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:39.944 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.944 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.944 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2419200", 2024-04-22T13:34:39.944 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.944 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.944 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.944 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-22T13:34:39.944 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.944 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.944 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.944 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.944 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": { 2024-04-22T13:34:39.944 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_name", 2024-04-22T13:34:39.944 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.944 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.945 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.945 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "device_health_metrics", 2024-04-22T13:34:39.945 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.945 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.945 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.945 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-22T13:34:39.945 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.945 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.945 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.945 INFO:teuthology.orchestra.run.smithi012.stdout: "retention_period": { 2024-04-22T13:34:39.945 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retention_period", 2024-04-22T13:34:39.945 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:39.945 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.945 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.946 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15552000", 2024-04-22T13:34:39.946 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.946 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.946 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.946 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain device health metrics", 2024-04-22T13:34:39.946 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.946 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.946 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.946 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_frequency": { 2024-04-22T13:34:39.946 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_frequency", 2024-04-22T13:34:39.946 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:39.946 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.947 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.947 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-04-22T13:34:39.947 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.947 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.947 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.947 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-22T13:34:39.947 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.947 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.947 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.947 INFO:teuthology.orchestra.run.smithi012.stdout: "self_heal": { 2024-04-22T13:34:39.947 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "self_heal", 2024-04-22T13:34:39.947 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.947 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.947 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.948 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:39.948 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.948 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.948 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.948 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-22T13:34:39.948 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.948 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.948 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.948 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.948 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-04-22T13:34:39.948 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-04-22T13:34:39.948 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:39.948 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.948 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.948 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-04-22T13:34:39.949 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.949 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.949 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.949 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and check device health", 2024-04-22T13:34:39.949 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.949 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.949 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.949 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_threshold": { 2024-04-22T13:34:39.949 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_threshold", 2024-04-22T13:34:39.949 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:39.949 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.949 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.949 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7257600", 2024-04-22T13:34:39.949 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.950 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.950 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.950 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-22T13:34:39.950 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.950 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.950 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.950 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:39.950 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:39.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.950 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:39.950 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "diskprediction_local", 2024-04-22T13:34:39.950 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:39.950 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:39.950 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:39.951 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:39.951 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:39.951 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.951 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.951 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.951 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.951 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.951 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.951 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:39.951 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:39.951 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:39.951 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:39.951 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:39.951 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:39.952 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:39.952 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:39.952 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.952 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.952 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.952 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.952 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:39.952 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:39.952 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.952 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.952 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.952 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.952 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.953 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.953 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.953 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.953 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.953 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.953 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.953 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.953 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:39.953 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:39.953 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.953 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.953 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.953 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:39.953 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.953 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.954 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:39.954 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:39.954 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:39.954 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:39.954 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:39.954 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:39.954 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:39.954 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:39.954 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.954 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.954 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.954 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.954 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:39.955 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:39.955 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.955 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.955 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.955 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.955 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.955 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.955 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.955 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.955 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.955 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.955 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.955 INFO:teuthology.orchestra.run.smithi012.stdout: "predict_interval": { 2024-04-22T13:34:39.955 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predict_interval", 2024-04-22T13:34:39.956 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.956 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.956 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.956 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-04-22T13:34:39.956 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.956 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.956 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.956 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.956 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.956 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.956 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.956 INFO:teuthology.orchestra.run.smithi012.stdout: "predictor_model": { 2024-04-22T13:34:39.956 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predictor_model", 2024-04-22T13:34:39.956 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.957 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.957 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.957 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "prophetstor", 2024-04-22T13:34:39.957 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.957 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.957 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.957 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.957 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.957 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.957 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.957 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-04-22T13:34:39.957 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-04-22T13:34:39.957 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.958 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.958 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.958 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-04-22T13:34:39.958 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.958 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.958 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.958 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.958 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.958 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.958 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.958 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:39.958 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:39.958 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.959 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:39.959 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "influx", 2024-04-22T13:34:39.959 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": false, 2024-04-22T13:34:39.959 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "influxdb python module not found", 2024-04-22T13:34:39.959 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:39.959 INFO:teuthology.orchestra.run.smithi012.stdout: "batch_size": { 2024-04-22T13:34:39.959 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "batch_size", 2024-04-22T13:34:39.959 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:39.959 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.959 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.959 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5000", 2024-04-22T13:34:39.959 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.959 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.959 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.959 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-22T13:34:39.960 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.960 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.960 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.960 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.960 INFO:teuthology.orchestra.run.smithi012.stdout: "database": { 2024-04-22T13:34:39.960 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "database", 2024-04-22T13:34:39.960 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.960 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.960 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.960 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "ceph", 2024-04-22T13:34:39.960 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.960 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.960 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.960 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-22T13:34:39.960 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.961 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.961 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.961 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.961 INFO:teuthology.orchestra.run.smithi012.stdout: "hostname": { 2024-04-22T13:34:39.961 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hostname", 2024-04-22T13:34:39.961 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.961 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.961 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.961 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.961 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.961 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.961 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.961 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server hostname", 2024-04-22T13:34:39.961 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.962 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.962 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.962 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.962 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-04-22T13:34:39.962 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-04-22T13:34:39.962 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:39.962 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.962 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.962 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "30", 2024-04-22T13:34:39.962 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "5", 2024-04-22T13:34:39.962 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.962 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.962 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-22T13:34:39.962 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.963 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.963 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.963 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:39.963 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:39.963 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.963 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.963 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.963 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.963 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.963 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.963 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:39.963 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:39.963 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:39.963 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:39.964 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:39.964 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:39.964 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:39.964 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:39.964 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.964 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.964 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.964 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.964 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.964 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:39.964 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:39.964 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.964 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.964 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.965 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.965 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.965 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.965 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.965 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.965 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.965 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.965 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.965 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:39.965 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:39.965 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.965 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.965 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.966 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:39.966 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.966 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.966 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:39.966 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:39.966 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:39.966 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:39.966 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:39.966 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:39.966 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:39.966 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:39.966 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.966 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.966 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.966 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.967 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.967 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:39.967 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:39.967 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.967 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.967 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.967 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.967 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.967 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.967 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.967 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.967 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.967 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.967 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.967 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.968 INFO:teuthology.orchestra.run.smithi012.stdout: "password": { 2024-04-22T13:34:39.968 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "password", 2024-04-22T13:34:39.968 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.968 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.968 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.968 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.968 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.968 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.968 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.968 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "password of InfluxDB server user", 2024-04-22T13:34:39.968 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.968 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.968 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.968 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.968 INFO:teuthology.orchestra.run.smithi012.stdout: "port": { 2024-04-22T13:34:39.969 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "port", 2024-04-22T13:34:39.969 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:39.969 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.969 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.969 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8086", 2024-04-22T13:34:39.969 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.969 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.969 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.969 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server port", 2024-04-22T13:34:39.969 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.969 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.969 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.969 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-04-22T13:34:39.969 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-04-22T13:34:39.970 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.970 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.970 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.970 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "false", 2024-04-22T13:34:39.970 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.970 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.970 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.970 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-22T13:34:39.970 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.970 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.970 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.970 INFO:teuthology.orchestra.run.smithi012.stdout: "threads": { 2024-04-22T13:34:39.971 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threads", 2024-04-22T13:34:39.971 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:39.971 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.971 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.971 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-04-22T13:34:39.971 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-04-22T13:34:39.971 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "32", 2024-04-22T13:34:39.971 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.971 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-22T13:34:39.971 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.971 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.971 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.971 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.971 INFO:teuthology.orchestra.run.smithi012.stdout: "username": { 2024-04-22T13:34:39.971 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "username", 2024-04-22T13:34:39.972 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.972 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.972 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.972 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.972 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.972 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.972 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.972 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "username of InfluxDB server user", 2024-04-22T13:34:39.972 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.972 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.972 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.972 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.972 INFO:teuthology.orchestra.run.smithi012.stdout: "verify_ssl": { 2024-04-22T13:34:39.972 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "verify_ssl", 2024-04-22T13:34:39.972 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.973 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.973 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.973 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "true", 2024-04-22T13:34:39.973 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.973 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.973 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.973 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-22T13:34:39.973 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.973 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.973 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.973 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:39.973 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:39.973 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.973 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:39.974 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "insights", 2024-04-22T13:34:39.974 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:39.974 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:39.974 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:39.974 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:39.974 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:39.974 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.974 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.974 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.974 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.974 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.974 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.974 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:39.974 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:39.975 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:39.975 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:39.975 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:39.975 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:39.975 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:39.975 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:39.975 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.975 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.975 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.975 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.975 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:39.975 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:39.975 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.976 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.976 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.976 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.976 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.976 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.976 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.976 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.976 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.976 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.976 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.976 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:39.976 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:39.976 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.976 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.976 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.976 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:39.977 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.977 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.977 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:39.977 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:39.977 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:39.977 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:39.977 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:39.977 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:39.977 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:39.977 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:39.977 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.977 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.977 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.977 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.977 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:39.978 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:39.978 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.978 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.978 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.978 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.978 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.978 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.978 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.978 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.978 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.978 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.978 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.978 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:39.978 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:39.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.979 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:39.979 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "iostat", 2024-04-22T13:34:39.979 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:39.979 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:39.979 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:39.979 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:39.979 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:39.979 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.979 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.979 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.979 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.979 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.979 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.979 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:39.979 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:39.980 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:39.980 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:39.980 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:39.980 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:39.980 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:39.980 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:39.980 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.980 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.980 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.980 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.980 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:39.980 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:39.980 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.980 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.980 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:39.981 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.982 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:39.983 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:39.983 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.983 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.983 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.983 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.983 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.983 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.983 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.983 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.983 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.983 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.983 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.983 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:39.983 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:39.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:39.984 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "k8sevents", 2024-04-22T13:34:39.984 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:39.984 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:39.984 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:39.984 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_event_retention_days": { 2024-04-22T13:34:39.984 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ceph_event_retention_days", 2024-04-22T13:34:39.984 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:39.984 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.984 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.984 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7", 2024-04-22T13:34:39.984 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.984 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.984 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.984 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-22T13:34:39.985 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.985 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.985 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.985 INFO:teuthology.orchestra.run.smithi012.stdout: "config_check_secs": { 2024-04-22T13:34:39.985 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_check_secs", 2024-04-22T13:34:39.985 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:39.985 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.985 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:39.985 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-22T13:34:39.985 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "10", 2024-04-22T13:34:39.985 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.985 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.985 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-22T13:34:39.985 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.986 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.986 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.986 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:39.986 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:39.986 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.986 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.986 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.986 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.986 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.986 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.986 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:39.986 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:39.986 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:39.987 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:39.987 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:39.987 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:39.987 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:39.987 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:39.987 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.987 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.987 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.987 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.987 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.987 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:39.987 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:39.987 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.987 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.987 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.988 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.988 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.988 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.988 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.988 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.988 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.988 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.988 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.988 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:39.988 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:39.988 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.988 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.988 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.989 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:39.989 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.989 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.989 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:39.989 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:39.989 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:39.989 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:39.989 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:39.989 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:39.989 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:39.989 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:39.989 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.989 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.989 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.989 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.990 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:39.990 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:39.990 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.990 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.990 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.990 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.990 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.990 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.990 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.990 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.990 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.990 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.990 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.990 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:39.991 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:39.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:39.991 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "localpool", 2024-04-22T13:34:39.991 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:39.991 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:39.991 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:39.991 INFO:teuthology.orchestra.run.smithi012.stdout: "failure_domain": { 2024-04-22T13:34:39.991 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "failure_domain", 2024-04-22T13:34:39.991 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.991 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.991 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.991 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "host", 2024-04-22T13:34:39.991 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.991 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.992 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.992 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "failure domain for any created local pool", 2024-04-22T13:34:39.992 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-22T13:34:39.992 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.992 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.992 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:39.992 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:39.992 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.992 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.992 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.992 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.992 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.992 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.992 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:39.993 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:39.993 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:39.993 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:39.993 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:39.993 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:39.993 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:39.993 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:39.993 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.993 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.993 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.993 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.993 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:39.993 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:39.993 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.994 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.994 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.994 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.994 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.994 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.994 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.994 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.994 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.994 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.994 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.994 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:39.994 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:39.995 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:39.995 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.995 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.995 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:39.995 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.995 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.995 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:39.995 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:39.995 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:39.995 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:39.995 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:39.995 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:39.995 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:39.995 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:39.996 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.996 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.996 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.996 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.996 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:39.996 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:39.996 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:39.996 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.996 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.996 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:39.996 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.996 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.997 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.997 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:39.997 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.997 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.997 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.997 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": { 2024-04-22T13:34:39.997 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_size", 2024-04-22T13:34:39.997 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:39.997 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.997 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.997 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:39.997 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.997 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.997 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.997 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default min_size for any created local pool", 2024-04-22T13:34:39.998 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-22T13:34:39.998 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.998 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.998 INFO:teuthology.orchestra.run.smithi012.stdout: "num_rep": { 2024-04-22T13:34:39.998 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "num_rep", 2024-04-22T13:34:39.998 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:39.998 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.998 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.998 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3", 2024-04-22T13:34:39.998 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.998 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.998 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.998 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default replica count for any created local pool", 2024-04-22T13:34:39.999 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:39.999 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:39.999 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:39.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:39.999 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": { 2024-04-22T13:34:39.999 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_num", 2024-04-22T13:34:39.999 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:39.999 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:39.999 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:39.999 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "128", 2024-04-22T13:34:39.999 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:39.999 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:39.999 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:39.999 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default pg_num for any created local pool", 2024-04-22T13:34:40.000 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.000 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.000 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.000 INFO:teuthology.orchestra.run.smithi012.stdout: "prefix": { 2024-04-22T13:34:40.000 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prefix", 2024-04-22T13:34:40.000 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.000 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.000 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.000 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.000 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.000 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.000 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.000 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name prefix for any created local pool", 2024-04-22T13:34:40.001 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.001 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.001 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.001 INFO:teuthology.orchestra.run.smithi012.stdout: "subtree": { 2024-04-22T13:34:40.001 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "subtree", 2024-04-22T13:34:40.001 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.001 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.001 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.001 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "rack", 2024-04-22T13:34:40.001 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.001 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.001 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.001 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-22T13:34:40.002 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-22T13:34:40.002 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.002 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.002 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.002 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.002 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.002 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mds_autoscaler", 2024-04-22T13:34:40.002 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.002 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.002 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.002 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.002 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.002 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.003 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.003 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.003 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.003 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.003 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.003 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.003 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.003 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.003 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.003 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.003 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.003 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.003 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.003 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.004 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.004 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.004 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.004 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.004 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.004 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.004 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.004 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.004 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.004 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.004 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.004 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.004 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.004 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.005 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.005 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.005 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.005 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.005 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.005 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.005 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.005 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.005 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.005 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.005 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.005 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.005 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.005 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.006 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.006 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.006 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.006 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.006 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.006 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.006 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.006 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.006 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.006 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.006 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.006 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.006 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.007 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.007 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.007 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.007 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.007 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.007 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.007 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.007 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.007 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.007 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.007 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirroring", 2024-04-22T13:34:40.007 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.008 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.008 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.008 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.008 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.008 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.008 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.008 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.008 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.008 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.008 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.008 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.008 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.008 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.008 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.009 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.009 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.009 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.009 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.009 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.009 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.009 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.009 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.009 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.009 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.009 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.009 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.010 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.010 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.010 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.010 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.010 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.010 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.010 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.010 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.010 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.010 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.010 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.010 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.010 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.011 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.011 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.011 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.011 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.011 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.011 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.011 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.011 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.011 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.011 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.011 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.011 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.011 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.011 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.012 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.012 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.012 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.012 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.012 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.012 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.012 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.012 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.012 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.012 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.012 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.012 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.012 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.012 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.013 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.013 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.013 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.013 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.013 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "nfs", 2024-04-22T13:34:40.013 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.013 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.013 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.013 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.013 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.013 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.013 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.013 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.013 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.014 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.014 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.014 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.014 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.014 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.014 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.014 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.014 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.014 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.014 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.014 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.014 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.014 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.015 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.015 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.015 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.015 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.015 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.015 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.015 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.015 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.015 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.015 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.015 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.015 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.015 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.016 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.016 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.016 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.016 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.016 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.016 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.016 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.016 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.016 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.016 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.016 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.016 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.016 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.017 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.017 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.017 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.017 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.017 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.017 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.017 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.017 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.017 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.017 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.017 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.017 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.017 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.017 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.018 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.018 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.018 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.018 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.018 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.018 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.018 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.018 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.018 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.018 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.018 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-04-22T13:34:40.018 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.018 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.019 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.019 INFO:teuthology.orchestra.run.smithi012.stdout: "fail_fs": { 2024-04-22T13:34:40.019 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "fail_fs", 2024-04-22T13:34:40.019 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.019 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.019 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.019 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.019 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.019 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.019 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.019 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-22T13:34:40.019 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.019 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.019 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.020 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.020 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.020 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.020 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.020 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.020 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.020 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.020 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.020 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.020 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.020 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.020 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.020 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.020 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.021 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.021 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.021 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.021 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.021 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.021 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.021 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.021 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.021 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.021 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.021 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.021 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.021 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.021 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.022 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.022 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.022 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.022 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.022 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.022 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.022 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.022 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.022 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.022 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.022 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.022 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.023 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.023 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.023 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.023 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.023 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.023 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.023 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.023 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.023 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.023 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.023 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.023 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.023 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.023 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.024 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.024 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.024 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.024 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.024 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.024 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.024 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.024 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.024 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.024 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.024 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.024 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.024 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator": { 2024-04-22T13:34:40.025 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-04-22T13:34:40.025 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.025 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.025 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.025 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.025 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.025 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.025 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.025 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm", 2024-04-22T13:34:40.025 INFO:teuthology.orchestra.run.smithi012.stdout: "rook", 2024-04-22T13:34:40.025 INFO:teuthology.orchestra.run.smithi012.stdout: "test_orchestrator" 2024-04-22T13:34:40.025 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.025 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Orchestrator backend", 2024-04-22T13:34:40.025 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.025 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.026 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.026 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.026 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.026 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.026 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_perf_query", 2024-04-22T13:34:40.026 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.026 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.026 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.026 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.026 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.026 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.026 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.027 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.027 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.027 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.027 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.027 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.027 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.027 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.027 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.027 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.027 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.027 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.027 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.027 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.027 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.028 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.028 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.028 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.028 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.028 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.028 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.028 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.028 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.028 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.028 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.028 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.028 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.028 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.028 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.028 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.029 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.029 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.029 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.029 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.029 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.029 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.029 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.029 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.029 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.029 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.029 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.029 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.029 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.029 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.030 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.030 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.030 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.030 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.030 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.030 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.030 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.030 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.030 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.030 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.030 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.030 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.030 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.030 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.031 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.031 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.031 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.031 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.031 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.031 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.031 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.031 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.031 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_support", 2024-04-22T13:34:40.031 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.031 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.031 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.031 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.031 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.032 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.032 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.032 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.032 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.032 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.032 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.032 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.032 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.032 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.032 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.032 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.032 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.032 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.032 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.032 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.033 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.033 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.033 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.033 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.033 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.033 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.033 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.033 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.033 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.033 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.033 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.033 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.033 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.033 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.034 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.034 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.034 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.034 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.034 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.034 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.034 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.034 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.034 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.034 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.034 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.034 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.034 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.035 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.035 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.035 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.035 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.035 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.035 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.035 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.035 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.035 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.035 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.035 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.035 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.035 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.035 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.036 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.036 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.036 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.036 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.036 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.036 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.036 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.036 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.036 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.036 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.036 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.036 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_autoscaler", 2024-04-22T13:34:40.036 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.037 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.037 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.037 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.037 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.037 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.037 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.037 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.037 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.037 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.037 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.037 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.037 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.037 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.037 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.038 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.038 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.038 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.038 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.038 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.038 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.038 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.038 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.038 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.038 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.038 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.038 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.038 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.039 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.039 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.039 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.039 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.039 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.039 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.039 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.039 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.039 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.039 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.039 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.039 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.039 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.040 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.040 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.040 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.040 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.040 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.040 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.040 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.040 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.040 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.040 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.040 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.040 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.040 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.040 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.040 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.041 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.041 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.041 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.041 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.041 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.041 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.041 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.041 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.041 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.041 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.041 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.041 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.041 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.042 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-04-22T13:34:40.042 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-04-22T13:34:40.042 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.042 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.042 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.042 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-04-22T13:34:40.042 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.042 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.042 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.042 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.042 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.042 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.042 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.043 INFO:teuthology.orchestra.run.smithi012.stdout: "threshold": { 2024-04-22T13:34:40.043 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threshold", 2024-04-22T13:34:40.043 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-22T13:34:40.043 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.043 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.043 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-04-22T13:34:40.043 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1.0", 2024-04-22T13:34:40.043 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.043 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.043 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "scaling threshold", 2024-04-22T13:34:40.043 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-22T13:34:40.043 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.043 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.043 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.043 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.044 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.044 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "progress", 2024-04-22T13:34:40.044 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.044 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.044 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.044 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_pg_recovery_event": { 2024-04-22T13:34:40.044 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_pg_recovery_event", 2024-04-22T13:34:40.044 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.044 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.044 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.044 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.044 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.044 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.044 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.044 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-22T13:34:40.045 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.045 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.045 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.045 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-04-22T13:34:40.045 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-04-22T13:34:40.045 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.045 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.045 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.045 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.045 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.045 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.045 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.045 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.045 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.045 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.046 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.046 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.046 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.046 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.046 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.046 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.046 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.046 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.046 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.046 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.046 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.046 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.047 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.047 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.047 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.047 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.047 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.047 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.047 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.047 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.047 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.047 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.047 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.047 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.047 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.048 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.048 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.048 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.048 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.048 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.048 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.048 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.048 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.048 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.048 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.048 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.048 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.048 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.048 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.049 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.049 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.049 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.049 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.049 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.049 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.049 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.049 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.049 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.049 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.049 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.049 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.049 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.049 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.049 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.050 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.050 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.050 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.050 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.050 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.050 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.050 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.050 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.050 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.050 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.050 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.050 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.051 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.051 INFO:teuthology.orchestra.run.smithi012.stdout: "max_completed_events": { 2024-04-22T13:34:40.051 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_completed_events", 2024-04-22T13:34:40.051 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.051 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.051 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.051 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "50", 2024-04-22T13:34:40.051 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.051 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.051 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.051 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "number of past completed events to remember", 2024-04-22T13:34:40.051 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.052 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.052 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.052 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-04-22T13:34:40.052 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-04-22T13:34:40.052 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.052 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.052 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.052 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-04-22T13:34:40.052 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.052 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.052 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.052 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long the module is going to sleep", 2024-04-22T13:34:40.052 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.053 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.053 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.053 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.053 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.053 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus", 2024-04-22T13:34:40.053 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.053 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.053 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.053 INFO:teuthology.orchestra.run.smithi012.stdout: "cache": { 2024-04-22T13:34:40.053 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cache", 2024-04-22T13:34:40.053 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.053 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.053 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.054 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.054 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.054 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.054 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.054 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.054 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.054 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.054 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.054 INFO:teuthology.orchestra.run.smithi012.stdout: "exclude_perf_counters": { 2024-04-22T13:34:40.054 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "exclude_perf_counters", 2024-04-22T13:34:40.054 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.054 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.054 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.054 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.055 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.055 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.055 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.055 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-22T13:34:40.055 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-22T13:34:40.055 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.055 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.055 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.055 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.055 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.055 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.055 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.055 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.055 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.056 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.056 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.056 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.056 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.056 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.056 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.056 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.056 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.056 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.056 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.056 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.056 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.056 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.056 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.056 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.057 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.057 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.057 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.057 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.057 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.057 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.057 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.057 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.057 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.057 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.057 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.057 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.057 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.057 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.057 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.058 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.058 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.058 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.058 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.058 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.058 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.058 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.058 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.058 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.058 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.058 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.058 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.058 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.058 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.058 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.059 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.059 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.059 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.059 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.059 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.059 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.059 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.059 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.059 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.059 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.059 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.059 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.059 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.059 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.060 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools": { 2024-04-22T13:34:40.060 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools", 2024-04-22T13:34:40.060 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.060 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.060 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.060 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.060 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.060 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.060 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.060 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.060 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.060 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.060 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.061 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-22T13:34:40.061 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-22T13:34:40.061 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.061 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.061 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.061 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "300", 2024-04-22T13:34:40.061 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.061 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.061 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.061 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.061 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.061 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.061 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.061 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_interval": { 2024-04-22T13:34:40.062 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_interval", 2024-04-22T13:34:40.062 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-22T13:34:40.062 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.062 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.062 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15.0", 2024-04-22T13:34:40.062 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.062 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.062 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.062 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.062 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.062 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.062 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.063 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-04-22T13:34:40.063 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-04-22T13:34:40.063 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.063 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.063 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.063 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-04-22T13:34:40.063 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.063 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.063 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.063 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-22T13:34:40.063 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.063 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.063 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.063 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-04-22T13:34:40.063 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-04-22T13:34:40.064 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.064 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.064 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.064 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "9283", 2024-04-22T13:34:40.064 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.064 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.064 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.064 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-22T13:34:40.064 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.064 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.064 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.064 INFO:teuthology.orchestra.run.smithi012.stdout: "stale_cache_strategy": { 2024-04-22T13:34:40.064 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stale_cache_strategy", 2024-04-22T13:34:40.065 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.065 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.065 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.065 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "log", 2024-04-22T13:34:40.065 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.065 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.065 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.065 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.065 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.065 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.065 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.065 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-04-22T13:34:40.065 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-04-22T13:34:40.066 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.066 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.066 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.066 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "default", 2024-04-22T13:34:40.066 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.066 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.066 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.066 INFO:teuthology.orchestra.run.smithi012.stdout: "default", 2024-04-22T13:34:40.066 INFO:teuthology.orchestra.run.smithi012.stdout: "error" 2024-04-22T13:34:40.066 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.066 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.066 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.066 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.066 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.067 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-04-22T13:34:40.067 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-04-22T13:34:40.067 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.067 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.067 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.067 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-04-22T13:34:40.067 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-04-22T13:34:40.067 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-04-22T13:34:40.067 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.067 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.067 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.067 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.067 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.067 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.068 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.068 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.068 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_support", 2024-04-22T13:34:40.068 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.068 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.068 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.068 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.068 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.068 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.068 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.068 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.068 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.068 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.069 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.069 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.069 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.069 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.069 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.069 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.069 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.069 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.069 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.069 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.069 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.069 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.069 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.069 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.070 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.070 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.070 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.070 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.070 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.070 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.070 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.070 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.070 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.070 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.070 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.070 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.070 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.071 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.071 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.071 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.071 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.071 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.071 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.071 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.071 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.071 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.071 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.071 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.071 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.071 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.071 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.072 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.072 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.072 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.072 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.072 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.072 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.072 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.072 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.072 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.072 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.072 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.072 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.072 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.073 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.073 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.073 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.073 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.073 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.073 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_snap_create": { 2024-04-22T13:34:40.073 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_snap_create", 2024-04-22T13:34:40.073 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.073 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.073 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.073 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-22T13:34:40.073 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.073 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.073 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.073 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.074 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.074 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.074 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.074 INFO:teuthology.orchestra.run.smithi012.stdout: "mirror_snapshot_schedule": { 2024-04-22T13:34:40.074 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirror_snapshot_schedule", 2024-04-22T13:34:40.074 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.074 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.074 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.074 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.074 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.074 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.074 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.074 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.075 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.075 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.075 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.075 INFO:teuthology.orchestra.run.smithi012.stdout: "trash_purge_schedule": { 2024-04-22T13:34:40.075 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "trash_purge_schedule", 2024-04-22T13:34:40.075 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.075 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.075 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.075 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.076 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.076 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.076 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.076 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.076 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.076 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.076 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.076 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.076 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.076 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.076 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "restful", 2024-04-22T13:34:40.076 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.076 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.076 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.077 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_auth": { 2024-04-22T13:34:40.077 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_auth", 2024-04-22T13:34:40.077 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.077 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.077 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.077 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.077 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.077 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.077 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.077 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.077 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.077 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.077 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.077 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-04-22T13:34:40.077 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-04-22T13:34:40.078 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.078 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.078 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.078 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.078 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.078 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.078 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.078 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.078 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.078 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.078 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.078 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.078 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.079 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.079 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.079 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.079 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.079 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.079 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.079 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.079 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.079 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.079 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.079 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.079 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.079 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.079 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.079 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.080 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.080 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.080 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.080 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.080 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.080 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.080 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.080 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.080 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.080 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.080 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.080 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.080 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.081 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.081 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.081 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.081 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.081 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.081 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.081 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.081 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.081 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.081 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.081 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.081 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.081 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.081 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.082 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.082 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.082 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.082 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.082 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.082 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.082 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.082 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.082 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.082 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.082 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.082 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.082 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.083 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.083 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.083 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.083 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.083 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.083 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.083 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.083 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.083 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.083 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.083 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-04-22T13:34:40.083 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-04-22T13:34:40.083 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.083 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.084 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.084 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.084 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.084 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.084 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.084 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.084 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.084 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.084 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.084 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.084 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-04-22T13:34:40.084 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-04-22T13:34:40.084 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.084 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.084 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.085 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.085 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.085 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.085 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.085 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.085 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.085 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.085 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.085 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.085 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.085 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.085 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.085 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rgw", 2024-04-22T13:34:40.085 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.085 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.086 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.086 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.086 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.086 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.086 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.086 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.086 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.086 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.086 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.086 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.086 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.086 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.086 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.086 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.087 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.087 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.087 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.087 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.087 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.087 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.087 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.087 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.087 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.087 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.087 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.087 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.087 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.087 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.088 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.088 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.088 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.088 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.088 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.088 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.088 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.088 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.088 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.088 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.088 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.088 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.088 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.088 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.088 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.089 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.089 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.089 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.089 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.089 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.089 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.089 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.089 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.089 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.089 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.089 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.089 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.089 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.089 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.090 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.090 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.090 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.090 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.090 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.090 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.090 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.090 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.090 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.090 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.090 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.090 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.090 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.090 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.091 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.091 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.091 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.091 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rook", 2024-04-22T13:34:40.091 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.091 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.091 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.091 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.091 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.091 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.091 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.091 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.091 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.091 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.091 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.092 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.092 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.092 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.092 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.092 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.092 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.092 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.092 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.092 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.092 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.092 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.092 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.092 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.092 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.092 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.093 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.093 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.093 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.093 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.093 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.093 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.093 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.093 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.093 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.093 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.093 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.093 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.093 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.093 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.094 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.094 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.094 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.094 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.094 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.094 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.094 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.094 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.094 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.094 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.094 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.094 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.094 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.094 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.095 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.095 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.095 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.095 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.095 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.095 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.095 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.095 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.095 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.095 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.095 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.095 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.095 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.095 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.096 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.096 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.096 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.096 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.096 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.096 INFO:teuthology.orchestra.run.smithi012.stdout: "storage_class": { 2024-04-22T13:34:40.096 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "storage_class", 2024-04-22T13:34:40.096 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.096 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.096 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.096 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "local", 2024-04-22T13:34:40.096 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.096 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.096 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.097 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-22T13:34:40.097 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.097 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.097 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.097 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.097 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.097 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.097 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.097 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "selftest", 2024-04-22T13:34:40.097 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.097 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.097 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.097 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.097 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.097 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.098 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.098 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.098 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.098 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.098 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.098 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.098 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.098 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.098 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.098 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.098 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.098 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.098 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.099 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.099 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.099 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.099 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.099 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.099 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.099 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.099 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.099 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.099 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.099 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.099 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.099 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.099 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.100 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.100 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.100 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.100 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.100 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.100 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.100 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.100 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.100 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.100 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.100 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.100 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.100 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.100 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.100 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.101 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.101 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.101 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.101 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.101 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.101 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.101 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.101 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.101 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.101 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.101 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.101 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.101 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.101 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.102 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.102 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.102 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.102 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.102 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.102 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.102 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.102 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.102 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.102 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.102 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.102 INFO:teuthology.orchestra.run.smithi012.stdout: "roption1": { 2024-04-22T13:34:40.102 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption1", 2024-04-22T13:34:40.102 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.103 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.103 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.103 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.103 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.103 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.103 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.103 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.103 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.103 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.103 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.103 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.103 INFO:teuthology.orchestra.run.smithi012.stdout: "roption2": { 2024-04-22T13:34:40.104 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption2", 2024-04-22T13:34:40.104 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.104 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.104 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.104 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "xyz", 2024-04-22T13:34:40.104 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.104 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.104 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.104 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.104 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.104 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.104 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.104 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.104 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption1": { 2024-04-22T13:34:40.105 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption1", 2024-04-22T13:34:40.105 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.105 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.105 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.105 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.105 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.105 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.105 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.105 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.105 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.105 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.105 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.105 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.105 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption2": { 2024-04-22T13:34:40.106 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption2", 2024-04-22T13:34:40.106 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.106 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.106 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.106 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.106 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.106 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.106 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.106 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.106 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.106 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.106 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.106 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.106 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption3": { 2024-04-22T13:34:40.107 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption3", 2024-04-22T13:34:40.107 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-22T13:34:40.107 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.107 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.107 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.107 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.107 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.107 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.107 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.107 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.107 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.107 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.107 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.108 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption4": { 2024-04-22T13:34:40.108 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption4", 2024-04-22T13:34:40.108 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.108 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.108 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.108 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.108 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.108 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.108 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.108 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.108 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.108 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.108 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.108 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.108 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption5": { 2024-04-22T13:34:40.109 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption5", 2024-04-22T13:34:40.109 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.109 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.109 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.109 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.109 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.109 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.109 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.109 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.109 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.109 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.109 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.109 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.110 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption6": { 2024-04-22T13:34:40.110 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption6", 2024-04-22T13:34:40.110 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.110 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.110 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.110 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.110 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.110 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.110 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.110 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.110 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.110 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.110 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.110 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.111 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption7": { 2024-04-22T13:34:40.111 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption7", 2024-04-22T13:34:40.111 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.111 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.111 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.111 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.111 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-04-22T13:34:40.111 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "42", 2024-04-22T13:34:40.111 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.111 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.111 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.111 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.111 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.111 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.112 INFO:teuthology.orchestra.run.smithi012.stdout: "testkey": { 2024-04-22T13:34:40.112 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testkey", 2024-04-22T13:34:40.112 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.112 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.112 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.112 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.112 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.112 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.112 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.112 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.112 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.112 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.112 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.112 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.113 INFO:teuthology.orchestra.run.smithi012.stdout: "testlkey": { 2024-04-22T13:34:40.113 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testlkey", 2024-04-22T13:34:40.113 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.113 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.113 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.113 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.113 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.113 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.113 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.113 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.113 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.113 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.113 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.113 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.114 INFO:teuthology.orchestra.run.smithi012.stdout: "testnewline": { 2024-04-22T13:34:40.114 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testnewline", 2024-04-22T13:34:40.114 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.114 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.114 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.114 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.114 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.114 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.114 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.114 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.114 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.114 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.114 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.114 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.115 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.115 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.115 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.115 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snap_schedule", 2024-04-22T13:34:40.115 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.115 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.115 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.115 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_m_granularity": { 2024-04-22T13:34:40.115 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_m_granularity", 2024-04-22T13:34:40.115 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.115 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.115 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.115 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.115 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.115 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.116 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.116 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow minute scheduled snapshots", 2024-04-22T13:34:40.116 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.116 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.116 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.116 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.116 INFO:teuthology.orchestra.run.smithi012.stdout: "dump_on_update": { 2024-04-22T13:34:40.116 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dump_on_update", 2024-04-22T13:34:40.116 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.116 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.116 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.116 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.116 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.116 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.117 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.117 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "dump database to debug log on update", 2024-04-22T13:34:40.117 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.117 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.117 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.117 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.117 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.117 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.117 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.117 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.117 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.117 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.117 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.117 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.117 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.117 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.118 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.118 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.118 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.118 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.118 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.118 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.118 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.118 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.118 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.118 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.118 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.118 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.118 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.118 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.119 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.119 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.119 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.119 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.119 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.119 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.119 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.119 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.119 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.119 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.119 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.119 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.119 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.119 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.119 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.120 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.120 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.120 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.120 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.120 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.120 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.120 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.120 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.120 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.120 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.120 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.120 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.120 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.120 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.120 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.121 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.121 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.121 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.121 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.121 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.121 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.121 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.121 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.121 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.121 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.121 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.121 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.121 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.121 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.121 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.122 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.122 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.122 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.122 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.122 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stats", 2024-04-22T13:34:40.122 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.122 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.122 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.122 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.122 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.122 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.122 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.122 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.123 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.123 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.123 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.123 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.123 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.123 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.123 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.123 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.123 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.123 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.123 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.123 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.123 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.123 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.123 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.124 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.124 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.124 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.124 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.124 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.124 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.124 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.124 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.124 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.124 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.124 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.124 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.124 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.124 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.125 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.125 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.125 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.125 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.125 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.125 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.125 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.125 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.125 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.125 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.125 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.125 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.125 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.125 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.125 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.126 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.126 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.126 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.126 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.126 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.126 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.126 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.126 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.126 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.126 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.126 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.126 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.126 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.126 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.127 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.127 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.127 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.127 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.127 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.127 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.127 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.127 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.127 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.127 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.127 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "status", 2024-04-22T13:34:40.127 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.127 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.127 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.127 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.128 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.128 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.128 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.128 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.128 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.128 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.128 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.128 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.128 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.128 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.128 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.128 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.128 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.128 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.128 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.129 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.129 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.129 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.129 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.129 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.129 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.129 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.129 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.129 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.129 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.129 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.129 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.129 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.129 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.129 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.129 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.130 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.130 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.130 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.130 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.130 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.130 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.130 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.130 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.130 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.130 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.130 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.130 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.131 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.131 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.131 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.131 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.131 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.131 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.131 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.131 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.131 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.131 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.131 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.131 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.131 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.131 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.131 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.132 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.132 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.132 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.132 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.132 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.132 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.132 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.132 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.132 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.132 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.132 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.132 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.132 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.132 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.132 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telegraf", 2024-04-22T13:34:40.133 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.133 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.133 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.133 INFO:teuthology.orchestra.run.smithi012.stdout: "address": { 2024-04-22T13:34:40.133 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "address", 2024-04-22T13:34:40.133 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.133 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.133 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.133 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-22T13:34:40.133 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.133 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.133 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.133 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.133 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.134 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.134 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.134 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.134 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-04-22T13:34:40.134 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-04-22T13:34:40.134 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.134 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.134 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.134 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15", 2024-04-22T13:34:40.134 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.134 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.134 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.134 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.135 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.135 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.135 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.135 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.135 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.135 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.135 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.135 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.135 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.135 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.135 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.135 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.135 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.135 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.135 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.136 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.136 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.136 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.136 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.136 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.136 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.136 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.136 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.136 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.136 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.136 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.136 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.136 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.137 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.137 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.137 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.137 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.137 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.137 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.137 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.137 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.137 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.137 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.137 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.137 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.137 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.137 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.137 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.138 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.138 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.138 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.138 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.138 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.138 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.138 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.138 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.138 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.138 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.138 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.138 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.138 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.138 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.139 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.139 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.139 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.139 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.139 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.139 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.139 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.139 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.139 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.139 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.139 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.139 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.139 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.139 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.140 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.140 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.140 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.140 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.140 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.140 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telemetry", 2024-04-22T13:34:40.140 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.140 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.140 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.140 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_basic": { 2024-04-22T13:34:40.140 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_basic", 2024-04-22T13:34:40.140 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.140 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.141 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.141 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.141 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.141 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.141 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.141 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-22T13:34:40.141 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.141 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.141 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.141 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_crash": { 2024-04-22T13:34:40.141 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_crash", 2024-04-22T13:34:40.141 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.141 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.142 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.142 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.142 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.142 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.142 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.142 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-22T13:34:40.142 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.142 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.142 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.142 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.142 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_device": { 2024-04-22T13:34:40.142 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_device", 2024-04-22T13:34:40.142 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.142 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.143 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.143 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.143 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.143 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.143 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.143 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-22T13:34:40.143 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.143 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.143 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.143 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_ident": { 2024-04-22T13:34:40.143 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_ident", 2024-04-22T13:34:40.143 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.144 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.144 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.144 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.144 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.144 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.144 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.144 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-22T13:34:40.144 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.144 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.144 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.144 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.144 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_perf": { 2024-04-22T13:34:40.144 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_perf", 2024-04-22T13:34:40.144 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.145 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.145 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.145 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.145 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.145 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.145 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.145 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-22T13:34:40.145 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.145 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.145 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.145 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.145 INFO:teuthology.orchestra.run.smithi012.stdout: "contact": { 2024-04-22T13:34:40.145 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "contact", 2024-04-22T13:34:40.146 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.146 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.146 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.146 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.146 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.146 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.146 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.146 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.146 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.146 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.146 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.146 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.146 INFO:teuthology.orchestra.run.smithi012.stdout: "description": { 2024-04-22T13:34:40.146 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "description", 2024-04-22T13:34:40.147 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.147 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.147 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.147 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.147 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.147 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.147 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.147 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.147 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.147 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.147 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.147 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.147 INFO:teuthology.orchestra.run.smithi012.stdout: "device_url": { 2024-04-22T13:34:40.147 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_url", 2024-04-22T13:34:40.148 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.148 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.148 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.148 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-22T13:34:40.148 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.148 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.148 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.148 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.148 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.148 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.148 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.148 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.148 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-04-22T13:34:40.148 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-04-22T13:34:40.149 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.149 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.149 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.149 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.149 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.149 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.149 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.149 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.149 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.149 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.149 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.149 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.149 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-04-22T13:34:40.149 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-04-22T13:34:40.149 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.150 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.150 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.150 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "24", 2024-04-22T13:34:40.150 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "8", 2024-04-22T13:34:40.150 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.150 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.150 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.150 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.150 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.150 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.150 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.150 INFO:teuthology.orchestra.run.smithi012.stdout: "last_opt_revision": { 2024-04-22T13:34:40.150 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "last_opt_revision", 2024-04-22T13:34:40.150 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.151 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.151 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.151 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1", 2024-04-22T13:34:40.151 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.151 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.151 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.151 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.151 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.151 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.151 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.151 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.151 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard": { 2024-04-22T13:34:40.151 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard", 2024-04-22T13:34:40.151 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.151 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.152 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.152 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.152 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.152 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.152 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.152 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.152 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.152 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.152 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.152 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.152 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard_description": { 2024-04-22T13:34:40.152 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard_description", 2024-04-22T13:34:40.152 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.152 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.152 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.152 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.153 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.153 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.153 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.153 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.153 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.153 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.153 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.153 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.153 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.153 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.153 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.153 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.153 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.153 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.153 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.153 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.154 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.154 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.154 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.154 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.154 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.154 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.154 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.154 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.154 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.154 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.154 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.154 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.154 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.155 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.155 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.155 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.155 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.155 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.155 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.155 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.155 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.155 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.155 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.155 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.155 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.155 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.155 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.155 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.156 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.156 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.156 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.156 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.156 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.156 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.156 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.156 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.156 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.156 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.156 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.156 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.156 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.156 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.157 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.157 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.157 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.157 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.157 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.157 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.157 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.157 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.157 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.157 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.157 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.157 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.157 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.157 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.158 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.158 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.158 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.158 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.158 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.158 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.158 INFO:teuthology.orchestra.run.smithi012.stdout: "organization": { 2024-04-22T13:34:40.158 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "organization", 2024-04-22T13:34:40.158 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.158 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.158 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.158 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.158 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.158 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.159 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.159 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.159 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.159 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.159 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.159 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.159 INFO:teuthology.orchestra.run.smithi012.stdout: "proxy": { 2024-04-22T13:34:40.159 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "proxy", 2024-04-22T13:34:40.159 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.159 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.159 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.159 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.159 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.159 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.160 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.160 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.160 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.160 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.160 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.160 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.160 INFO:teuthology.orchestra.run.smithi012.stdout: "url": { 2024-04-22T13:34:40.160 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url", 2024-04-22T13:34:40.160 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.160 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.160 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.160 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-22T13:34:40.160 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.160 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.160 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.161 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.161 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.161 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.161 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.161 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.161 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.161 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.161 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.161 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "test_orchestrator", 2024-04-22T13:34:40.161 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.161 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.161 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.161 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.161 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.162 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.162 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.162 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.162 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.162 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.162 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.162 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.162 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.162 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.162 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.162 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.162 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.162 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.162 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.163 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.163 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.163 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.163 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.163 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.163 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.163 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.163 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.163 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.163 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.163 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.163 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.163 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.163 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.163 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.164 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.164 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.164 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.164 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.164 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.164 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.164 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.164 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.164 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.164 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.164 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.164 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.164 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.164 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.165 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.165 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.165 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.165 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.165 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.165 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.165 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.165 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.165 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.165 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.165 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.165 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.165 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.165 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.165 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.165 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.166 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.166 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.166 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.166 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.166 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.166 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.166 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.166 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.166 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.166 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.166 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.166 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.166 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "volumes", 2024-04-22T13:34:40.166 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.167 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.167 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.167 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.167 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.167 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.167 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.167 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.167 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.167 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.167 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.167 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.167 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.167 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.167 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.167 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.168 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.168 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.168 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.168 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.168 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.168 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.168 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.168 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.168 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.168 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.168 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.168 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.168 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.168 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.168 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.169 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.169 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.169 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.169 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.169 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.169 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.169 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.169 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.169 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.169 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.169 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.169 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.169 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.169 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.169 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.170 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.170 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.170 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.170 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.170 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.170 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.170 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.170 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.170 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.170 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.170 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.170 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.170 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.171 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.171 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.171 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.171 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.171 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.171 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.171 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.171 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.171 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.171 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.171 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.171 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.171 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.171 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.172 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_clones": { 2024-04-22T13:34:40.172 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_clones", 2024-04-22T13:34:40.172 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.172 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.172 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.172 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4", 2024-04-22T13:34:40.172 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.172 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.172 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.172 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-22T13:34:40.172 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.172 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.172 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.172 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.172 INFO:teuthology.orchestra.run.smithi012.stdout: "periodic_async_work": { 2024-04-22T13:34:40.173 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "periodic_async_work", 2024-04-22T13:34:40.173 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.173 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.173 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.173 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.173 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.173 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.173 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.173 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Periodically check for async work", 2024-04-22T13:34:40.173 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.173 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.173 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.173 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.173 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_delay": { 2024-04-22T13:34:40.174 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_delay", 2024-04-22T13:34:40.174 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.174 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.174 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.174 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-04-22T13:34:40.174 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.174 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.174 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.174 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-22T13:34:40.174 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.174 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.174 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.174 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.175 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_no_wait": { 2024-04-22T13:34:40.175 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_no_wait", 2024-04-22T13:34:40.175 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.175 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.175 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.175 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.175 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.175 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.175 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.175 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-22T13:34:40.175 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.175 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.175 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.175 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.176 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.176 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.176 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.176 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix", 2024-04-22T13:34:40.176 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.176 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.176 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.176 INFO:teuthology.orchestra.run.smithi012.stdout: "discovery_interval": { 2024-04-22T13:34:40.176 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "discovery_interval", 2024-04-22T13:34:40.176 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-04-22T13:34:40.176 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.176 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.176 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "100", 2024-04-22T13:34:40.176 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.176 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.177 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.177 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.177 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.177 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.177 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.177 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.177 INFO:teuthology.orchestra.run.smithi012.stdout: "identifier": { 2024-04-22T13:34:40.177 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "identifier", 2024-04-22T13:34:40.177 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.177 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.177 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.177 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.177 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.177 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.177 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.178 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.178 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.178 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.178 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.178 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.178 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-04-22T13:34:40.178 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-04-22T13:34:40.178 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.178 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.178 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.178 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-04-22T13:34:40.178 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.178 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.178 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.179 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.179 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.179 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.179 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.179 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.179 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.179 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.179 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.179 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.179 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.179 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.179 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.179 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.179 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.179 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.180 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.180 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.180 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.180 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.180 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.180 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.180 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.180 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.180 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.180 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.180 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.180 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.180 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.180 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.181 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.181 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.181 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.181 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.181 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.181 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.181 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.181 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.181 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.181 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.181 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.181 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.181 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.181 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.181 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.182 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.182 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.182 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.182 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.182 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.182 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.182 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.182 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.182 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.182 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.182 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.182 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.182 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.182 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.182 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.183 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.183 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.183 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.183 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.183 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.183 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.183 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.183 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.183 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.183 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.183 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.183 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.183 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.183 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.183 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.184 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_host": { 2024-04-22T13:34:40.184 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_host", 2024-04-22T13:34:40.184 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.184 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.184 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.184 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.184 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.184 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.184 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.184 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.184 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.184 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.184 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.184 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.184 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_port": { 2024-04-22T13:34:40.185 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_port", 2024-04-22T13:34:40.185 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.185 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.185 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.185 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10051", 2024-04-22T13:34:40.185 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.185 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.185 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.185 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.185 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.185 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.185 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.185 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_sender": { 2024-04-22T13:34:40.186 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_sender", 2024-04-22T13:34:40.186 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.186 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.186 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.186 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-22T13:34:40.186 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.186 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.186 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.186 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.186 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.186 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.186 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.186 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.186 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.187 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.187 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-22T13:34:40.187 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.187 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.187 INFO:teuthology.orchestra.run.smithi012.stdout: "gid": 24931, 2024-04-22T13:34:40.187 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "y", 2024-04-22T13:34:40.187 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_features": 4540701547738038271, 2024-04-22T13:34:40.187 INFO:teuthology.orchestra.run.smithi012.stdout: "available_modules": [ 2024-04-22T13:34:40.187 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.187 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "alerts", 2024-04-22T13:34:40.187 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.187 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.187 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.187 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-04-22T13:34:40.187 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-04-22T13:34:40.187 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.188 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.188 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.188 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-04-22T13:34:40.188 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.188 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.188 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.188 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How frequently to reexamine health status", 2024-04-22T13:34:40.188 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.188 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.188 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.188 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.188 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.188 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.188 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.189 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.189 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.189 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.189 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.189 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.189 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.189 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.189 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.189 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.189 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.189 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.189 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.189 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.189 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.190 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.190 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.190 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.190 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.190 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.190 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.190 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.190 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.190 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.190 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.190 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.190 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.190 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.191 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.191 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.191 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.191 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.191 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.191 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.191 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.191 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.191 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.191 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.191 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.191 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.191 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.191 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.192 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.192 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.192 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.192 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.192 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.192 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.192 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.192 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.192 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.192 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.192 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.192 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.192 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.192 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.192 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.193 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.193 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.193 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.193 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.193 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.193 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.193 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.193 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.193 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.193 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.193 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.193 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_destination": { 2024-04-22T13:34:40.193 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_destination", 2024-04-22T13:34:40.193 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.194 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.194 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.194 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.194 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.194 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.194 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.194 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email address to send alerts to", 2024-04-22T13:34:40.194 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.194 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.194 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.194 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.194 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_from_name": { 2024-04-22T13:34:40.194 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_from_name", 2024-04-22T13:34:40.194 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.195 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.195 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.195 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "Ceph", 2024-04-22T13:34:40.195 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.195 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.195 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.195 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email From: name", 2024-04-22T13:34:40.195 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.195 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.195 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.195 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_host": { 2024-04-22T13:34:40.195 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_host", 2024-04-22T13:34:40.195 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.196 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.196 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.196 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.196 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.196 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.196 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.196 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP server", 2024-04-22T13:34:40.196 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.196 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.196 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.196 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.196 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_password": { 2024-04-22T13:34:40.196 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_password", 2024-04-22T13:34:40.196 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.197 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.197 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.197 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.197 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.197 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.197 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.197 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Password to authenticate with", 2024-04-22T13:34:40.197 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.197 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.197 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.197 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.197 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_port": { 2024-04-22T13:34:40.197 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_port", 2024-04-22T13:34:40.197 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.198 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.198 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.198 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "465", 2024-04-22T13:34:40.198 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.198 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.198 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.198 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP port", 2024-04-22T13:34:40.198 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.198 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.198 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.198 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.198 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_sender": { 2024-04-22T13:34:40.198 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_sender", 2024-04-22T13:34:40.199 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.199 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.199 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.199 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.199 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.199 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.199 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.199 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP envelope sender", 2024-04-22T13:34:40.199 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.199 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.199 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.199 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.199 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_ssl": { 2024-04-22T13:34:40.199 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_ssl", 2024-04-22T13:34:40.200 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.200 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.200 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.200 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.200 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.200 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.200 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.200 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-22T13:34:40.200 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.200 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.200 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.200 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_user": { 2024-04-22T13:34:40.200 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_user", 2024-04-22T13:34:40.201 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.201 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.201 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.201 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.201 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.201 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.201 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.201 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "User to authenticate as", 2024-04-22T13:34:40.201 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.201 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.201 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.201 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.201 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.201 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.202 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "balancer", 2024-04-22T13:34:40.202 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.202 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.202 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.202 INFO:teuthology.orchestra.run.smithi012.stdout: "active": { 2024-04-22T13:34:40.202 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "active", 2024-04-22T13:34:40.202 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.202 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.202 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.202 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.202 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.202 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.202 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.202 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically balance PGs across cluster", 2024-04-22T13:34:40.203 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.203 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.203 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.203 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_time": { 2024-04-22T13:34:40.203 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_time", 2024-04-22T13:34:40.203 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.203 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.203 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.203 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0000", 2024-04-22T13:34:40.203 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.203 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.203 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.203 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "beginning time of day to automatically balance", 2024-04-22T13:34:40.203 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-22T13:34:40.204 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.204 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.204 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_weekday": { 2024-04-22T13:34:40.204 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_weekday", 2024-04-22T13:34:40.204 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-04-22T13:34:40.204 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.204 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.204 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-04-22T13:34:40.204 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-04-22T13:34:40.204 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-04-22T13:34:40.204 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.204 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-22T13:34:40.205 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-22T13:34:40.205 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.205 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.205 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_max_iterations": { 2024-04-22T13:34:40.205 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_max_iterations", 2024-04-22T13:34:40.205 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-04-22T13:34:40.205 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.205 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.205 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "25", 2024-04-22T13:34:40.205 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-04-22T13:34:40.205 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "250", 2024-04-22T13:34:40.205 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.205 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-22T13:34:40.206 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.206 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.206 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.206 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.206 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_metrics": { 2024-04-22T13:34:40.206 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_metrics", 2024-04-22T13:34:40.206 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.206 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.206 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.206 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "pgs,objects,bytes", 2024-04-22T13:34:40.206 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.206 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.206 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.206 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-22T13:34:40.207 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-22T13:34:40.207 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.207 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.207 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_step": { 2024-04-22T13:34:40.207 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_step", 2024-04-22T13:34:40.207 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-22T13:34:40.207 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.207 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.207 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.5", 2024-04-22T13:34:40.207 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0.001", 2024-04-22T13:34:40.207 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "0.999", 2024-04-22T13:34:40.207 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.207 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "aggressiveness of optimization", 2024-04-22T13:34:40.207 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-22T13:34:40.208 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.208 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.208 INFO:teuthology.orchestra.run.smithi012.stdout: "end_time": { 2024-04-22T13:34:40.208 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_time", 2024-04-22T13:34:40.208 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.208 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.208 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.208 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2359", 2024-04-22T13:34:40.208 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.208 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.208 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.208 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ending time of day to automatically balance", 2024-04-22T13:34:40.208 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-22T13:34:40.209 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.209 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.209 INFO:teuthology.orchestra.run.smithi012.stdout: "end_weekday": { 2024-04-22T13:34:40.209 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_weekday", 2024-04-22T13:34:40.209 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-04-22T13:34:40.209 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.209 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.209 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-04-22T13:34:40.209 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-04-22T13:34:40.209 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-04-22T13:34:40.209 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.209 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-22T13:34:40.209 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-22T13:34:40.210 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.210 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.210 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.210 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.210 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.210 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.210 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.210 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.210 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.210 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.210 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.210 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.210 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.211 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.211 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.211 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.211 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.211 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.211 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.211 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.211 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.211 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.211 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.211 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.211 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.211 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.211 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.211 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.212 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.212 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.212 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.212 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.212 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.212 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.212 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.212 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.212 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.212 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.212 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.212 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.212 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.212 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.213 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.213 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.213 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.213 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.213 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.213 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.213 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.213 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.213 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.213 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.213 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.213 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.213 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.213 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.213 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.214 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.214 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.214 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.214 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.214 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.214 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.214 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.214 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.214 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.214 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.214 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.214 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.214 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.215 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.215 INFO:teuthology.orchestra.run.smithi012.stdout: "min_score": { 2024-04-22T13:34:40.215 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_score", 2024-04-22T13:34:40.215 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-22T13:34:40.215 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.215 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.215 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-04-22T13:34:40.215 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.215 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.215 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.215 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-22T13:34:40.215 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.215 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.215 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.216 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-04-22T13:34:40.216 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-04-22T13:34:40.216 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.216 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.216 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.216 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "upmap", 2024-04-22T13:34:40.216 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.216 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.216 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.216 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-compat", 2024-04-22T13:34:40.216 INFO:teuthology.orchestra.run.smithi012.stdout: "none", 2024-04-22T13:34:40.216 INFO:teuthology.orchestra.run.smithi012.stdout: "read", 2024-04-22T13:34:40.216 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap", 2024-04-22T13:34:40.216 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap-read" 2024-04-22T13:34:40.216 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.217 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Balancer mode", 2024-04-22T13:34:40.217 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.217 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.217 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.217 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_ids": { 2024-04-22T13:34:40.217 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_ids", 2024-04-22T13:34:40.217 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.217 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.217 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.217 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.217 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.217 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.217 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.217 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-22T13:34:40.218 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.218 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.218 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.218 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-04-22T13:34:40.218 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-04-22T13:34:40.218 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.218 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.218 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.218 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-04-22T13:34:40.218 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.218 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.218 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.218 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-22T13:34:40.219 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.219 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.219 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.219 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_deviation": { 2024-04-22T13:34:40.219 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_deviation", 2024-04-22T13:34:40.219 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.219 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.219 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.219 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-04-22T13:34:40.219 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-04-22T13:34:40.219 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.219 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.219 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-22T13:34:40.220 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-22T13:34:40.220 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.220 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.220 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.220 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_optimizations": { 2024-04-22T13:34:40.220 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_optimizations", 2024-04-22T13:34:40.220 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-04-22T13:34:40.220 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.220 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.220 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-22T13:34:40.220 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.220 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.220 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.220 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-22T13:34:40.220 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.221 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.221 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.221 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.221 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.221 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.221 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm", 2024-04-22T13:34:40.221 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.221 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.221 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.221 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_down_multiplier": { 2024-04-22T13:34:40.221 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_down_multiplier", 2024-04-22T13:34:40.221 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-22T13:34:40.221 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.221 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.222 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-04-22T13:34:40.222 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.222 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.222 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.222 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-22T13:34:40.222 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.222 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.222 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.222 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_refresh_rate": { 2024-04-22T13:34:40.222 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_refresh_rate", 2024-04-22T13:34:40.222 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.222 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.223 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.223 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "20", 2024-04-22T13:34:40.223 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.223 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.223 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.223 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-22T13:34:40.223 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.223 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.223 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.223 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_starting_port": { 2024-04-22T13:34:40.223 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_starting_port", 2024-04-22T13:34:40.223 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.223 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.223 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.224 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4721", 2024-04-22T13:34:40.224 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.224 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.224 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.224 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-22T13:34:40.224 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.224 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.224 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.224 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_ptrace": { 2024-04-22T13:34:40.224 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_ptrace", 2024-04-22T13:34:40.224 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.224 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.224 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.225 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.225 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.225 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.225 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.225 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-22T13:34:40.225 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-22T13:34:40.225 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.225 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.225 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_interval": { 2024-04-22T13:34:40.225 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_interval", 2024-04-22T13:34:40.225 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.225 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.225 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.226 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-04-22T13:34:40.226 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.226 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.226 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.226 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-22T13:34:40.226 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.226 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.226 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.226 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_memory_target_ratio": { 2024-04-22T13:34:40.226 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_memory_target_ratio", 2024-04-22T13:34:40.226 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-22T13:34:40.226 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.226 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.227 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.7", 2024-04-22T13:34:40.227 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.227 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.227 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.227 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-22T13:34:40.227 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.227 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.227 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.227 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm_log_destination": { 2024-04-22T13:34:40.227 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm_log_destination", 2024-04-22T13:34:40.227 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.227 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.227 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.227 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.228 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.228 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.228 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.228 INFO:teuthology.orchestra.run.smithi012.stdout: "file", 2024-04-22T13:34:40.228 INFO:teuthology.orchestra.run.smithi012.stdout: "file,syslog", 2024-04-22T13:34:40.228 INFO:teuthology.orchestra.run.smithi012.stdout: "syslog" 2024-04-22T13:34:40.228 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.228 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-22T13:34:40.228 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.228 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.228 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.228 INFO:teuthology.orchestra.run.smithi012.stdout: "cgroups_split": { 2024-04-22T13:34:40.228 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cgroups_split", 2024-04-22T13:34:40.229 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.229 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.229 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.229 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.229 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.229 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.229 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.229 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-22T13:34:40.229 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.229 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.229 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.229 INFO:teuthology.orchestra.run.smithi012.stdout: "config_checks_enabled": { 2024-04-22T13:34:40.229 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_checks_enabled", 2024-04-22T13:34:40.229 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.230 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.230 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.230 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.230 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.230 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.230 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.230 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-22T13:34:40.230 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.230 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.230 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.230 INFO:teuthology.orchestra.run.smithi012.stdout: "config_dashboard": { 2024-04-22T13:34:40.230 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_dashboard", 2024-04-22T13:34:40.230 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.231 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.231 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.231 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.231 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.231 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.231 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.231 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-22T13:34:40.231 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.231 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.231 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.231 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_alertmanager": { 2024-04-22T13:34:40.231 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_alertmanager", 2024-04-22T13:34:40.231 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.231 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.231 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.232 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-22T13:34:40.232 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.232 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.232 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.232 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-04-22T13:34:40.232 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.232 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.232 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.232 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_base": { 2024-04-22T13:34:40.232 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_base", 2024-04-22T13:34:40.232 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.232 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.232 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.232 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-22T13:34:40.232 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.233 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.233 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.233 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Container image name, without the tag", 2024-04-22T13:34:40.233 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.233 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.233 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.233 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_elasticsearch": { 2024-04-22T13:34:40.233 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_elasticsearch", 2024-04-22T13:34:40.233 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.233 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.233 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.233 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-22T13:34:40.233 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.233 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.234 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.234 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "elasticsearch container image", 2024-04-22T13:34:40.234 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.234 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.234 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.234 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_grafana": { 2024-04-22T13:34:40.234 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_grafana", 2024-04-22T13:34:40.234 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.234 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.234 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.234 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-22T13:34:40.234 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.234 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.235 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.235 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-04-22T13:34:40.235 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.235 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.235 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.235 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_haproxy": { 2024-04-22T13:34:40.235 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_haproxy", 2024-04-22T13:34:40.235 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.235 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.235 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.235 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-22T13:34:40.235 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.235 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.236 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.237 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "HAproxy container image", 2024-04-22T13:34:40.237 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.237 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.237 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.237 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_agent": { 2024-04-22T13:34:40.237 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_agent", 2024-04-22T13:34:40.237 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.238 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.238 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.238 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-22T13:34:40.238 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.238 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.238 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.238 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger agent container image", 2024-04-22T13:34:40.238 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.238 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.238 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.238 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_collector": { 2024-04-22T13:34:40.238 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_collector", 2024-04-22T13:34:40.238 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.238 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.239 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.239 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-22T13:34:40.239 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.239 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.239 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.239 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger collector container image", 2024-04-22T13:34:40.239 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.239 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.239 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.239 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_query": { 2024-04-22T13:34:40.239 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_query", 2024-04-22T13:34:40.239 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.239 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.239 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.240 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-22T13:34:40.240 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.240 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.240 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.240 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger query container image", 2024-04-22T13:34:40.240 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.240 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.240 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.240 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_keepalived": { 2024-04-22T13:34:40.240 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_keepalived", 2024-04-22T13:34:40.240 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.240 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.240 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.241 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-22T13:34:40.241 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.241 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.241 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.241 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Keepalived container image", 2024-04-22T13:34:40.241 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.241 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.241 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.241 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_loki": { 2024-04-22T13:34:40.241 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_loki", 2024-04-22T13:34:40.241 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.241 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.242 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.242 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-22T13:34:40.242 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.242 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.242 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.242 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Loki container image", 2024-04-22T13:34:40.242 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.242 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.242 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.242 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_node_exporter": { 2024-04-22T13:34:40.242 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_node_exporter", 2024-04-22T13:34:40.242 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.242 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.243 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.243 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-22T13:34:40.243 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.243 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.243 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.243 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-04-22T13:34:40.243 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.243 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.243 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.243 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_nvmeof": { 2024-04-22T13:34:40.243 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_nvmeof", 2024-04-22T13:34:40.243 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.243 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.243 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.244 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-22T13:34:40.244 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.244 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.244 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.244 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Nvme-of container image", 2024-04-22T13:34:40.244 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.244 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.244 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.244 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_prometheus": { 2024-04-22T13:34:40.244 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_prometheus", 2024-04-22T13:34:40.244 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.244 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.244 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.244 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-22T13:34:40.245 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.245 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.245 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.245 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-04-22T13:34:40.245 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.245 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.245 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.245 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_promtail": { 2024-04-22T13:34:40.245 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_promtail", 2024-04-22T13:34:40.245 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.245 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.245 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.245 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-22T13:34:40.246 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.246 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.246 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.246 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Promtail container image", 2024-04-22T13:34:40.246 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.246 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.246 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.246 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_samba": { 2024-04-22T13:34:40.246 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_samba", 2024-04-22T13:34:40.246 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.246 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.246 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.246 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-22T13:34:40.247 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.247 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.247 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.247 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Samba/SMB container image", 2024-04-22T13:34:40.247 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.247 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.247 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.247 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_snmp_gateway": { 2024-04-22T13:34:40.247 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_snmp_gateway", 2024-04-22T13:34:40.247 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.247 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.247 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.247 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-22T13:34:40.247 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.248 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.248 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.248 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SNMP Gateway container image", 2024-04-22T13:34:40.248 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.248 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.248 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.248 INFO:teuthology.orchestra.run.smithi012.stdout: "container_init": { 2024-04-22T13:34:40.248 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_init", 2024-04-22T13:34:40.248 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.248 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.248 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.248 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.248 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.249 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.249 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.249 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Run podman/docker with `--init`", 2024-04-22T13:34:40.249 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.249 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.249 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.249 INFO:teuthology.orchestra.run.smithi012.stdout: "daemon_cache_timeout": { 2024-04-22T13:34:40.249 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "daemon_cache_timeout", 2024-04-22T13:34:40.249 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.249 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.249 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.249 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-04-22T13:34:40.249 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.249 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.250 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.250 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-22T13:34:40.250 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.250 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.250 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.250 INFO:teuthology.orchestra.run.smithi012.stdout: "default_cephadm_command_timeout": { 2024-04-22T13:34:40.250 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_cephadm_command_timeout", 2024-04-22T13:34:40.250 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.250 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.250 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.250 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "900", 2024-04-22T13:34:40.250 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.250 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.251 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.251 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-22T13:34:40.251 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.251 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.251 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.251 INFO:teuthology.orchestra.run.smithi012.stdout: "default_registry": { 2024-04-22T13:34:40.251 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_registry", 2024-04-22T13:34:40.251 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.251 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.251 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.251 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io", 2024-04-22T13:34:40.251 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.252 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.252 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.252 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-22T13:34:40.252 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.252 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.252 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.252 INFO:teuthology.orchestra.run.smithi012.stdout: "device_cache_timeout": { 2024-04-22T13:34:40.253 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_cache_timeout", 2024-04-22T13:34:40.253 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.253 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.253 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.253 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1800", 2024-04-22T13:34:40.253 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.253 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.253 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.254 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache device inventory", 2024-04-22T13:34:40.254 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.254 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.254 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.254 INFO:teuthology.orchestra.run.smithi012.stdout: "device_enhanced_scan": { 2024-04-22T13:34:40.254 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_enhanced_scan", 2024-04-22T13:34:40.255 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.255 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.255 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.255 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.255 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.255 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.255 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.255 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-22T13:34:40.255 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.255 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.256 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.256 INFO:teuthology.orchestra.run.smithi012.stdout: "facts_cache_timeout": { 2024-04-22T13:34:40.256 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "facts_cache_timeout", 2024-04-22T13:34:40.256 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.256 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.256 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.256 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-04-22T13:34:40.256 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.256 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.256 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.256 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache host facts data", 2024-04-22T13:34:40.256 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.256 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.256 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.257 INFO:teuthology.orchestra.run.smithi012.stdout: "grafana_dashboards_path": { 2024-04-22T13:34:40.257 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "grafana_dashboards_path", 2024-04-22T13:34:40.257 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.257 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.257 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.257 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-22T13:34:40.257 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.257 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.257 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.257 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-22T13:34:40.257 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.257 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.257 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.258 INFO:teuthology.orchestra.run.smithi012.stdout: "host_check_interval": { 2024-04-22T13:34:40.258 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "host_check_interval", 2024-04-22T13:34:40.258 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.258 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.258 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.258 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-04-22T13:34:40.258 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.258 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.258 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.258 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to perform a host check", 2024-04-22T13:34:40.258 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.258 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.258 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.259 INFO:teuthology.orchestra.run.smithi012.stdout: "hw_monitoring": { 2024-04-22T13:34:40.259 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hw_monitoring", 2024-04-22T13:34:40.259 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.259 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.259 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.259 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.259 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.259 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.259 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.259 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-22T13:34:40.259 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.259 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.259 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.260 INFO:teuthology.orchestra.run.smithi012.stdout: "inventory_list_all": { 2024-04-22T13:34:40.260 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "inventory_list_all", 2024-04-22T13:34:40.260 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.260 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.260 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.260 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.260 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.260 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.260 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.260 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-22T13:34:40.260 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.260 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.260 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.261 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.261 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.261 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.261 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.261 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.261 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.261 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.261 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.261 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.261 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.261 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.261 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.261 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.261 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.261 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.262 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.262 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.262 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.262 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.262 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.262 INFO:teuthology.orchestra.run.smithi012.stdout: "log_refresh_metadata": { 2024-04-22T13:34:40.262 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_refresh_metadata", 2024-04-22T13:34:40.262 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.262 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.262 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.262 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.262 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.262 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.263 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.263 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-22T13:34:40.263 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.263 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.263 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.263 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.263 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.263 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.263 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.263 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.263 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.263 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.263 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.263 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.264 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-22T13:34:40.264 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.264 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.264 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.264 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.264 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.264 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.264 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.264 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.264 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.264 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.264 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.264 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.265 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.265 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.265 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.265 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.265 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.265 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.265 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.265 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.265 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.265 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.265 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.265 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.265 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.265 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.266 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.266 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.266 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.266 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.266 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.266 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.266 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.266 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.266 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.266 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.266 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-22T13:34:40.266 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-22T13:34:40.266 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.266 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.267 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.267 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.267 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.267 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.267 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.267 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-22T13:34:40.267 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.267 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.267 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.267 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-22T13:34:40.267 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-22T13:34:40.267 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.267 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.267 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.268 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "*", 2024-04-22T13:34:40.268 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.268 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.268 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.268 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-22T13:34:40.268 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.268 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.268 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.268 INFO:teuthology.orchestra.run.smithi012.stdout: "max_count_per_host": { 2024-04-22T13:34:40.268 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_count_per_host", 2024-04-22T13:34:40.268 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.268 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.268 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.268 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-22T13:34:40.269 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.269 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.269 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.269 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of daemons per service per host", 2024-04-22T13:34:40.269 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.269 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.269 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.269 INFO:teuthology.orchestra.run.smithi012.stdout: "max_osd_draining_count": { 2024-04-22T13:34:40.269 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_osd_draining_count", 2024-04-22T13:34:40.269 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.269 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.269 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.269 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-22T13:34:40.270 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.270 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.270 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.270 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-22T13:34:40.270 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.270 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.270 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.270 INFO:teuthology.orchestra.run.smithi012.stdout: "migration_current": { 2024-04-22T13:34:40.270 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "migration_current", 2024-04-22T13:34:40.270 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.270 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.270 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.270 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.270 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.271 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.271 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.271 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "internal - do not modify", 2024-04-22T13:34:40.271 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.271 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.271 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.271 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-04-22T13:34:40.271 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-04-22T13:34:40.271 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.271 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.271 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.271 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "root", 2024-04-22T13:34:40.271 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.271 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.272 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.272 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm-package", 2024-04-22T13:34:40.272 INFO:teuthology.orchestra.run.smithi012.stdout: "root" 2024-04-22T13:34:40.272 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.272 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "mode for remote execution of cephadm", 2024-04-22T13:34:40.272 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.272 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.272 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.272 INFO:teuthology.orchestra.run.smithi012.stdout: "oob_default_addr": { 2024-04-22T13:34:40.272 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "oob_default_addr", 2024-04-22T13:34:40.272 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.272 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.273 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.273 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "169.254.1.1", 2024-04-22T13:34:40.273 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.273 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.273 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.273 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-22T13:34:40.273 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.273 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.273 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.273 INFO:teuthology.orchestra.run.smithi012.stdout: "prometheus_alerts_path": { 2024-04-22T13:34:40.273 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus_alerts_path", 2024-04-22T13:34:40.273 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.273 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.273 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.274 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-22T13:34:40.274 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.274 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.274 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.274 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-22T13:34:40.274 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.274 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.274 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.274 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_insecure": { 2024-04-22T13:34:40.274 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_insecure", 2024-04-22T13:34:40.274 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.274 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.274 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.275 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.275 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.275 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.275 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.275 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-22T13:34:40.275 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.275 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.275 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.275 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_password": { 2024-04-22T13:34:40.275 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_password", 2024-04-22T13:34:40.275 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.275 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.275 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.276 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.276 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.276 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.276 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.276 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-22T13:34:40.276 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.276 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.276 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.276 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_url": { 2024-04-22T13:34:40.276 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_url", 2024-04-22T13:34:40.276 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.276 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.276 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.276 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.277 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.277 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.277 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.277 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-22T13:34:40.277 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.277 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.277 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.277 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_username": { 2024-04-22T13:34:40.277 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_username", 2024-04-22T13:34:40.277 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.277 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.277 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.277 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.278 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.278 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.278 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.278 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-22T13:34:40.278 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.278 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.278 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.278 INFO:teuthology.orchestra.run.smithi012.stdout: "secure_monitoring_stack": { 2024-04-22T13:34:40.278 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "secure_monitoring_stack", 2024-04-22T13:34:40.278 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.278 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.278 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.278 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.279 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.279 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.279 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.279 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-22T13:34:40.279 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.279 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.279 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.279 INFO:teuthology.orchestra.run.smithi012.stdout: "service_discovery_port": { 2024-04-22T13:34:40.279 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "service_discovery_port", 2024-04-22T13:34:40.279 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.279 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.279 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.279 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8765", 2024-04-22T13:34:40.279 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.280 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.280 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.280 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "cephadm service discovery port", 2024-04-22T13:34:40.280 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.280 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.280 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.280 INFO:teuthology.orchestra.run.smithi012.stdout: "ssh_config_file": { 2024-04-22T13:34:40.280 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssh_config_file", 2024-04-22T13:34:40.280 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.280 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.280 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.280 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.280 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.281 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.281 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.281 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-22T13:34:40.281 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.281 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.281 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.281 INFO:teuthology.orchestra.run.smithi012.stdout: "use_agent": { 2024-04-22T13:34:40.281 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_agent", 2024-04-22T13:34:40.281 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.281 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.281 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.281 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.281 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.282 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.282 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.282 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-22T13:34:40.282 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.282 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.282 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.282 INFO:teuthology.orchestra.run.smithi012.stdout: "use_repo_digest": { 2024-04-22T13:34:40.282 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_repo_digest", 2024-04-22T13:34:40.282 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.282 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.282 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.282 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.282 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.283 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.283 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.283 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-22T13:34:40.283 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.283 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.283 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.283 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_failed_host_check": { 2024-04-22T13:34:40.283 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_failed_host_check", 2024-04-22T13:34:40.283 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.283 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.283 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.283 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.283 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.283 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.284 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.284 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if the host check fails", 2024-04-22T13:34:40.284 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.284 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.284 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.284 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_daemons": { 2024-04-22T13:34:40.284 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_daemons", 2024-04-22T13:34:40.284 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.284 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.284 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.284 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.284 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.284 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.285 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.285 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-22T13:34:40.285 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.285 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.285 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.285 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_hosts": { 2024-04-22T13:34:40.285 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_hosts", 2024-04-22T13:34:40.285 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.285 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.285 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.285 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.285 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.285 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.286 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.286 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-22T13:34:40.286 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.286 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.286 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.286 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.286 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.286 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crash", 2024-04-22T13:34:40.286 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.286 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.286 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.286 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.287 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.287 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.287 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.287 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.287 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.287 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.287 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.287 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.287 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.287 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.287 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.287 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.287 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.287 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.288 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.288 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.288 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.288 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.288 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.288 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.288 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.288 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.288 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.288 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.288 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.288 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.288 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.288 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.289 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.289 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.289 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.289 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.289 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.289 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.289 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.289 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.289 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.289 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.289 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.290 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.290 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.290 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.290 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.290 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.290 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.290 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.290 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.290 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.290 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.290 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.290 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.290 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.291 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.291 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.291 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.291 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.291 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.291 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.291 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.291 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.291 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.291 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.291 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.291 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.291 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.292 INFO:teuthology.orchestra.run.smithi012.stdout: "retain_interval": { 2024-04-22T13:34:40.292 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retain_interval", 2024-04-22T13:34:40.292 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.292 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.292 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.292 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "31536000", 2024-04-22T13:34:40.292 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.292 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.292 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.292 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-22T13:34:40.292 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.292 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.292 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.292 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_recent_interval": { 2024-04-22T13:34:40.292 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_recent_interval", 2024-04-22T13:34:40.293 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.293 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.293 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.293 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1209600", 2024-04-22T13:34:40.293 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.293 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.293 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.293 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-22T13:34:40.293 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.293 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.293 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.293 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.293 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.294 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dashboard", 2024-04-22T13:34:40.294 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.294 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.294 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.294 INFO:teuthology.orchestra.run.smithi012.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-22T13:34:40.294 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-22T13:34:40.294 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.294 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.294 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.294 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-22T13:34:40.294 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.294 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.294 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.294 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.295 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.295 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.295 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.295 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_HOST": { 2024-04-22T13:34:40.295 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-22T13:34:40.295 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.295 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.295 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.295 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.295 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.295 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.295 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.295 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.295 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.296 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.296 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.296 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-22T13:34:40.296 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-22T13:34:40.296 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.296 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.296 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.296 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.296 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.296 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.296 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.296 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.296 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.297 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.297 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.297 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_ENABLED": { 2024-04-22T13:34:40.297 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_ENABLED", 2024-04-22T13:34:40.297 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.297 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.297 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.297 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.297 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.297 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.297 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.297 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.297 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.297 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.298 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.298 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-22T13:34:40.298 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-22T13:34:40.298 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.298 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.298 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.298 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.298 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.298 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.298 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.298 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.298 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.298 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.299 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.299 INFO:teuthology.orchestra.run.smithi012.stdout: "ENABLE_BROWSABLE_API": { 2024-04-22T13:34:40.299 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-22T13:34:40.299 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.299 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.299 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.299 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.299 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.299 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.299 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.299 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.299 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.299 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.300 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.300 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-22T13:34:40.300 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-22T13:34:40.300 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.300 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.300 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.300 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.300 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.300 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.300 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.300 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.300 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.300 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.301 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.301 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-22T13:34:40.301 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-22T13:34:40.301 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.301 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.301 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.301 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.301 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.301 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.301 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.301 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.301 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.301 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.302 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.302 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-22T13:34:40.302 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-22T13:34:40.302 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.302 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.302 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.302 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.302 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.302 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.302 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.302 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.302 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.302 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.303 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.303 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-22T13:34:40.303 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-22T13:34:40.303 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.303 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.303 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.303 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.303 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.303 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.303 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.303 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.303 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.303 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.303 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.304 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-22T13:34:40.304 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-22T13:34:40.304 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.304 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.304 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.304 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.304 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.304 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.304 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.304 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.304 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.304 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.304 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.305 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-22T13:34:40.305 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-22T13:34:40.305 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.305 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.305 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.305 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.305 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.305 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.305 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.305 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.305 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.305 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.305 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.305 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.305 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-22T13:34:40.306 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-22T13:34:40.306 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.306 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.306 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.306 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.306 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.306 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.306 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.306 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.306 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.306 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.306 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.306 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.306 INFO:teuthology.orchestra.run.smithi012.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-22T13:34:40.307 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-22T13:34:40.307 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.307 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.307 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.307 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.307 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.307 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.307 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.307 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.307 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.307 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.307 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.307 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.307 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_PASSWORD": { 2024-04-22T13:34:40.307 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-22T13:34:40.308 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.308 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.308 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.308 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-04-22T13:34:40.308 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.308 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.308 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.308 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.308 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.308 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.308 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.308 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.308 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-22T13:34:40.308 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-22T13:34:40.308 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.309 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.309 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.309 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.309 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.309 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.309 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.309 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.309 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.309 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.309 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.309 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.309 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_URL": { 2024-04-22T13:34:40.309 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_URL", 2024-04-22T13:34:40.309 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.310 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.310 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.310 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.310 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.310 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.310 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.310 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.310 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.310 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.310 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.310 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.310 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_USERNAME": { 2024-04-22T13:34:40.310 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-22T13:34:40.311 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.311 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.311 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.311 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-04-22T13:34:40.311 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.311 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.311 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.311 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.311 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.311 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.311 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.311 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-22T13:34:40.311 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-22T13:34:40.312 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.312 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.312 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.312 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.312 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.312 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.312 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.312 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.312 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.312 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.312 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.312 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.312 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-22T13:34:40.312 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-22T13:34:40.312 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.313 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.313 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.313 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.313 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.313 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.313 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.313 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.313 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.313 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.313 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.313 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.313 INFO:teuthology.orchestra.run.smithi012.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-22T13:34:40.313 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-22T13:34:40.313 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.314 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.314 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.314 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.314 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.314 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.314 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.314 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.314 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.314 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.314 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.314 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.314 INFO:teuthology.orchestra.run.smithi012.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-22T13:34:40.314 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-22T13:34:40.314 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.315 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.315 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.315 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.315 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.315 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.315 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.315 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.315 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.315 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.315 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.315 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.315 INFO:teuthology.orchestra.run.smithi012.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-22T13:34:40.315 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-22T13:34:40.315 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.315 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.316 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.316 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "[]", 2024-04-22T13:34:40.316 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.316 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.316 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.316 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.316 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.316 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.316 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.316 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.316 INFO:teuthology.orchestra.run.smithi012.stdout: "MULTICLUSTER_CONFIG": { 2024-04-22T13:34:40.316 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-22T13:34:40.316 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.316 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.317 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.317 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "{}", 2024-04-22T13:34:40.317 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.317 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.317 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.317 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.317 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.317 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.317 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.317 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.317 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_HOST": { 2024-04-22T13:34:40.317 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-22T13:34:40.317 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.317 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.318 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.318 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.318 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.318 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.318 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.318 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.318 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.318 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.318 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.318 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.318 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-22T13:34:40.318 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-22T13:34:40.318 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.318 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.318 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.319 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.319 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.319 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.319 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.319 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.319 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.319 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.319 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.319 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.319 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-22T13:34:40.319 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-22T13:34:40.319 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.319 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.319 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.319 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.319 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.320 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.320 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.320 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.320 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.320 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.320 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.320 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.320 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-22T13:34:40.320 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-22T13:34:40.320 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.320 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.320 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.320 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.320 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.320 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.321 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.321 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.321 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.321 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.321 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.321 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.321 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-22T13:34:40.321 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-22T13:34:40.321 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.321 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.321 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.321 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.321 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.321 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.322 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.322 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.322 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.322 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.322 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.322 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.322 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-22T13:34:40.322 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-22T13:34:40.322 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.322 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.322 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.322 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.322 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.322 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.323 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.323 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.323 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.323 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.323 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.323 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.323 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-22T13:34:40.323 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-22T13:34:40.323 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.323 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.323 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.323 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.323 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.323 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.324 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.324 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.324 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.324 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.324 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.324 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.324 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-22T13:34:40.324 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-22T13:34:40.324 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.324 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.324 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.324 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.324 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.324 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.324 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.325 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.325 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.325 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.325 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.325 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.325 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-22T13:34:40.325 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-22T13:34:40.325 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.325 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.325 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.325 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.325 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.325 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.325 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.325 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.326 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.326 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.326 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.326 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.326 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_ENABLED": { 2024-04-22T13:34:40.326 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-22T13:34:40.326 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.326 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.326 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.326 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.326 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.326 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.326 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.327 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.327 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.327 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.327 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.327 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.327 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-22T13:34:40.327 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-22T13:34:40.327 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.327 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.327 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.327 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-22T13:34:40.327 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.327 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.327 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.327 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.328 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.328 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.328 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.328 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.328 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-22T13:34:40.328 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-22T13:34:40.328 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.328 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.328 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.328 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-22T13:34:40.328 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.328 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.328 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.328 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.329 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.329 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.329 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.329 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.329 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-22T13:34:40.329 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-22T13:34:40.329 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.329 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.329 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.329 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8", 2024-04-22T13:34:40.329 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.329 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.329 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.329 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.329 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.330 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.330 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.330 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.330 INFO:teuthology.orchestra.run.smithi012.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-22T13:34:40.330 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-22T13:34:40.330 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.330 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.330 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.330 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "45", 2024-04-22T13:34:40.330 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.330 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.330 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.330 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.330 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.331 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.331 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.331 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.331 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ACCESS_KEY": { 2024-04-22T13:34:40.331 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-22T13:34:40.331 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.331 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.331 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.331 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.331 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.331 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.331 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.331 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.331 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-22T13:34:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-22T13:34:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-04-22T13:34:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.332 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.333 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.333 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SECRET_KEY": { 2024-04-22T13:34:40.333 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-22T13:34:40.333 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.333 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.333 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.333 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.333 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.333 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.333 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.333 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.333 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.333 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.333 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.334 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.334 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SSL_VERIFY": { 2024-04-22T13:34:40.334 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-22T13:34:40.334 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.334 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.334 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.334 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.334 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.334 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.334 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.334 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.334 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.334 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.335 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.335 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.335 INFO:teuthology.orchestra.run.smithi012.stdout: "UNSAFE_TLS_v1_2": { 2024-04-22T13:34:40.335 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-22T13:34:40.335 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.335 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.335 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.335 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.335 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.335 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.335 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.335 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.335 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.335 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.335 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.336 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.336 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-22T13:34:40.336 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-22T13:34:40.336 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.336 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.336 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.336 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-04-22T13:34:40.336 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.336 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.336 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.336 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.336 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.336 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.336 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.337 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.337 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-22T13:34:40.337 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-22T13:34:40.337 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.337 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.337 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.337 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-22T13:34:40.337 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.337 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.337 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.337 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.337 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.337 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.337 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.338 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.338 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-22T13:34:40.338 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-22T13:34:40.338 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.338 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.338 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.338 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-04-22T13:34:40.338 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.338 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.338 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.338 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.338 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.338 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.338 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.339 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.339 INFO:teuthology.orchestra.run.smithi012.stdout: "cross_origin_url": { 2024-04-22T13:34:40.339 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cross_origin_url", 2024-04-22T13:34:40.339 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.339 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.339 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.339 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.339 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.339 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.339 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.339 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.339 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.339 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.339 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.340 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.340 INFO:teuthology.orchestra.run.smithi012.stdout: "crt_file": { 2024-04-22T13:34:40.340 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crt_file", 2024-04-22T13:34:40.340 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.340 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.340 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.340 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.340 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.340 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.340 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.340 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.340 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.340 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.340 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.340 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.341 INFO:teuthology.orchestra.run.smithi012.stdout: "debug": { 2024-04-22T13:34:40.341 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "debug", 2024-04-22T13:34:40.341 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.341 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.341 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.341 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.341 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.341 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.341 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.341 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable/disable debug options", 2024-04-22T13:34:40.341 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.341 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.341 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.341 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.342 INFO:teuthology.orchestra.run.smithi012.stdout: "jwt_token_ttl": { 2024-04-22T13:34:40.342 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "jwt_token_ttl", 2024-04-22T13:34:40.342 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.342 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.342 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.342 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "28800", 2024-04-22T13:34:40.342 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.342 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.342 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.342 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.342 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.342 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.342 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.342 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.343 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-04-22T13:34:40.343 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-04-22T13:34:40.343 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.343 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.343 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.343 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.343 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.343 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.343 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.343 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.343 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.343 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.343 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.343 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.343 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.344 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.344 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.344 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.344 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.344 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.344 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.344 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.344 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.344 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.344 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.344 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.344 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.344 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.344 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.344 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.345 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.345 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.345 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.345 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.345 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.345 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.345 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.345 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.345 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.345 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.345 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.345 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.345 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.345 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.346 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.346 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.346 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.346 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.346 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.346 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.346 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.346 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.346 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.346 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.346 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.346 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.346 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.346 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.347 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.347 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.347 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.347 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.347 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.347 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.347 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.347 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.347 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.347 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.347 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.347 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.347 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.347 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.347 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.348 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.348 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.348 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.348 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.348 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.348 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.348 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.348 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.348 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.348 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.348 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.348 INFO:teuthology.orchestra.run.smithi012.stdout: "motd": { 2024-04-22T13:34:40.348 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "motd", 2024-04-22T13:34:40.348 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.349 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.349 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.349 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.349 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.349 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.349 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.349 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "The message of the day", 2024-04-22T13:34:40.349 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.349 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.349 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.349 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.349 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect_resolve_ip_addr": { 2024-04-22T13:34:40.349 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "redirect_resolve_ip_addr", 2024-04-22T13:34:40.349 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.350 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.350 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.350 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.350 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.350 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.350 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.350 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.350 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.350 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.350 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.350 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.350 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-04-22T13:34:40.350 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-04-22T13:34:40.350 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.351 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.351 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.351 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-04-22T13:34:40.351 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.351 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.351 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.351 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.351 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.351 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.351 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.351 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.352 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-04-22T13:34:40.352 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-04-22T13:34:40.352 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.352 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.352 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.352 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8080", 2024-04-22T13:34:40.352 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.352 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.352 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.352 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.352 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.352 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.352 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.352 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.353 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-04-22T13:34:40.353 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-04-22T13:34:40.353 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.353 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.353 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.353 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.353 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.353 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.353 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.353 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.353 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.353 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.353 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.353 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.354 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl_server_port": { 2024-04-22T13:34:40.354 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl_server_port", 2024-04-22T13:34:40.354 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.354 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.354 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.354 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8443", 2024-04-22T13:34:40.354 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.354 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.354 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.354 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.354 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.354 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.354 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.354 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.355 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-04-22T13:34:40.355 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-04-22T13:34:40.355 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.355 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.355 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.355 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "redirect", 2024-04-22T13:34:40.355 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.355 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.355 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.355 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.355 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect" 2024-04-22T13:34:40.355 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.355 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.355 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.355 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.356 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.356 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.356 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-04-22T13:34:40.356 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-04-22T13:34:40.356 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.356 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.356 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.356 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-04-22T13:34:40.356 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-04-22T13:34:40.356 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-04-22T13:34:40.356 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.356 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.356 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.356 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.357 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.357 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.357 INFO:teuthology.orchestra.run.smithi012.stdout: "url_prefix": { 2024-04-22T13:34:40.357 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url_prefix", 2024-04-22T13:34:40.357 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.357 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.357 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.357 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.357 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.357 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.357 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.357 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.357 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.357 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.357 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.358 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.358 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.358 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.358 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.358 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "devicehealth", 2024-04-22T13:34:40.358 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.358 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.358 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.358 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_monitoring": { 2024-04-22T13:34:40.358 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_monitoring", 2024-04-22T13:34:40.358 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.358 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.358 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.359 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.359 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.359 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.359 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.359 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "monitor device health metrics", 2024-04-22T13:34:40.359 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.359 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.359 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.359 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.359 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.359 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.359 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.359 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.359 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.360 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.360 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.360 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.360 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.360 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.360 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.360 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.360 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.360 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.360 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.360 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.360 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.360 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.360 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.360 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.361 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.361 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.361 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.361 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.361 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.361 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.361 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.361 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.361 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.361 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.361 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.361 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.361 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.361 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.362 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.362 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.362 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.362 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.362 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.362 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.362 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.362 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.362 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.362 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.362 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.362 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.362 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.362 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.363 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.363 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.363 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.363 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.363 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.363 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.363 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.363 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.363 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.363 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.363 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.363 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.363 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.363 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.364 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.364 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.364 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.364 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.364 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.364 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.364 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.364 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.364 INFO:teuthology.orchestra.run.smithi012.stdout: "mark_out_threshold": { 2024-04-22T13:34:40.364 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mark_out_threshold", 2024-04-22T13:34:40.364 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.364 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.364 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.364 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2419200", 2024-04-22T13:34:40.364 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.365 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.365 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.365 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-22T13:34:40.365 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.365 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.365 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.365 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.365 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": { 2024-04-22T13:34:40.365 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_name", 2024-04-22T13:34:40.365 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.365 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.365 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.365 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "device_health_metrics", 2024-04-22T13:34:40.365 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.365 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.366 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.366 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-22T13:34:40.366 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.366 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.366 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.366 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.366 INFO:teuthology.orchestra.run.smithi012.stdout: "retention_period": { 2024-04-22T13:34:40.366 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retention_period", 2024-04-22T13:34:40.366 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.366 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.366 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.366 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15552000", 2024-04-22T13:34:40.366 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.366 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.367 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.367 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain device health metrics", 2024-04-22T13:34:40.367 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.367 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.367 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.367 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.367 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_frequency": { 2024-04-22T13:34:40.367 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_frequency", 2024-04-22T13:34:40.367 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.367 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.367 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.367 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-04-22T13:34:40.367 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.367 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.368 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.368 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-22T13:34:40.368 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.368 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.368 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.368 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.368 INFO:teuthology.orchestra.run.smithi012.stdout: "self_heal": { 2024-04-22T13:34:40.368 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "self_heal", 2024-04-22T13:34:40.368 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.368 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.368 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.368 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.368 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.368 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.368 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.369 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-22T13:34:40.369 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.369 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.369 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.369 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.369 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-04-22T13:34:40.369 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-04-22T13:34:40.369 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.369 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.369 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.369 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-04-22T13:34:40.369 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.369 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.369 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.370 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and check device health", 2024-04-22T13:34:40.370 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.370 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.370 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.370 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.370 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_threshold": { 2024-04-22T13:34:40.370 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_threshold", 2024-04-22T13:34:40.370 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.370 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.370 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.370 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7257600", 2024-04-22T13:34:40.370 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.370 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.370 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.370 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-22T13:34:40.371 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.371 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.371 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.371 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.371 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.371 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.371 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.371 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "diskprediction_local", 2024-04-22T13:34:40.371 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.371 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.371 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.371 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.371 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.371 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.372 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.372 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.372 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.372 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.372 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.372 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.372 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.372 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.372 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.372 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.372 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.372 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.372 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.372 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.372 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.373 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.373 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.373 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.373 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.373 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.373 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.373 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.373 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.373 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.373 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.373 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.373 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.373 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.373 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.374 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.374 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.374 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.374 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.374 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.374 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.374 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.374 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.374 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.374 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.374 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.374 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.374 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.374 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.375 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.375 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.375 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.375 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.375 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.375 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.375 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.375 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.375 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.375 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.375 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.375 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.375 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.375 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.376 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.376 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.376 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.376 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.376 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.376 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.376 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.376 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.376 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.376 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.376 INFO:teuthology.orchestra.run.smithi012.stdout: "predict_interval": { 2024-04-22T13:34:40.376 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predict_interval", 2024-04-22T13:34:40.376 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.376 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.377 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.377 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-04-22T13:34:40.377 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.377 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.377 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.377 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.377 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.377 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.377 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.377 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.377 INFO:teuthology.orchestra.run.smithi012.stdout: "predictor_model": { 2024-04-22T13:34:40.377 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predictor_model", 2024-04-22T13:34:40.377 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.377 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.377 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.378 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "prophetstor", 2024-04-22T13:34:40.378 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.378 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.378 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.378 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.378 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.378 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.378 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.378 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.378 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-04-22T13:34:40.378 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-04-22T13:34:40.378 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.378 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.378 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.379 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-04-22T13:34:40.379 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.379 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.379 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.379 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.379 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.379 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.379 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.379 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.379 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.379 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.379 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.379 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "influx", 2024-04-22T13:34:40.379 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": false, 2024-04-22T13:34:40.379 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "influxdb python module not found", 2024-04-22T13:34:40.380 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.380 INFO:teuthology.orchestra.run.smithi012.stdout: "batch_size": { 2024-04-22T13:34:40.380 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "batch_size", 2024-04-22T13:34:40.380 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.380 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.380 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.380 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5000", 2024-04-22T13:34:40.380 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.380 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.380 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.380 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-22T13:34:40.380 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.380 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.380 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.380 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.381 INFO:teuthology.orchestra.run.smithi012.stdout: "database": { 2024-04-22T13:34:40.381 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "database", 2024-04-22T13:34:40.381 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.381 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.381 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.381 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "ceph", 2024-04-22T13:34:40.381 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.381 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.381 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.381 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-22T13:34:40.381 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.381 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.381 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.381 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.382 INFO:teuthology.orchestra.run.smithi012.stdout: "hostname": { 2024-04-22T13:34:40.382 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hostname", 2024-04-22T13:34:40.382 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.382 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.382 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.382 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.382 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.382 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.382 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.382 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server hostname", 2024-04-22T13:34:40.382 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.382 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.382 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.382 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.382 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-04-22T13:34:40.383 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-04-22T13:34:40.383 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.383 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.383 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.383 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "30", 2024-04-22T13:34:40.383 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "5", 2024-04-22T13:34:40.383 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.383 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.383 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-22T13:34:40.383 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.383 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.383 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.383 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.383 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.383 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.384 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.384 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.384 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.384 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.384 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.384 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.384 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.384 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.384 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.384 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.384 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.384 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.384 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.384 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.384 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.385 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.385 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.385 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.385 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.385 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.385 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.385 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.385 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.385 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.385 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.385 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.385 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.385 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.385 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.385 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.386 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.386 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.386 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.386 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.386 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.386 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.386 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.386 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.386 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.386 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.386 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.386 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.386 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.386 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.386 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.387 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.387 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.387 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.387 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.387 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.387 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.387 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.387 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.387 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.387 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.387 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.387 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.387 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.387 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.387 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.388 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.388 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.388 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.388 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.388 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.388 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.388 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.388 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.388 INFO:teuthology.orchestra.run.smithi012.stdout: "password": { 2024-04-22T13:34:40.388 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "password", 2024-04-22T13:34:40.388 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.388 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.388 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.389 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.389 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.389 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.389 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.389 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "password of InfluxDB server user", 2024-04-22T13:34:40.389 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.389 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.389 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.389 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.389 INFO:teuthology.orchestra.run.smithi012.stdout: "port": { 2024-04-22T13:34:40.389 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "port", 2024-04-22T13:34:40.389 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.389 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.389 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.390 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8086", 2024-04-22T13:34:40.390 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.390 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.390 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.390 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server port", 2024-04-22T13:34:40.390 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.390 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.390 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.390 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-04-22T13:34:40.390 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-04-22T13:34:40.390 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.390 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.390 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.391 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "false", 2024-04-22T13:34:40.391 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.391 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.391 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.391 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-22T13:34:40.391 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.391 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.391 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.391 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.391 INFO:teuthology.orchestra.run.smithi012.stdout: "threads": { 2024-04-22T13:34:40.391 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threads", 2024-04-22T13:34:40.391 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.391 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.391 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.392 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-04-22T13:34:40.392 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-04-22T13:34:40.392 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "32", 2024-04-22T13:34:40.392 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.392 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-22T13:34:40.392 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.392 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.392 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.392 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.392 INFO:teuthology.orchestra.run.smithi012.stdout: "username": { 2024-04-22T13:34:40.392 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "username", 2024-04-22T13:34:40.392 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.392 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.392 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.393 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.393 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.393 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.393 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.393 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "username of InfluxDB server user", 2024-04-22T13:34:40.393 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.393 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.393 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.393 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.393 INFO:teuthology.orchestra.run.smithi012.stdout: "verify_ssl": { 2024-04-22T13:34:40.393 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "verify_ssl", 2024-04-22T13:34:40.393 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.393 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.393 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.393 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "true", 2024-04-22T13:34:40.394 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.394 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.394 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.394 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-22T13:34:40.394 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.394 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.394 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.394 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.394 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.394 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.394 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.394 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "insights", 2024-04-22T13:34:40.395 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.395 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.395 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.395 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.395 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.395 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.395 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.395 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.395 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.395 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.395 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.395 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.395 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.395 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.396 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.396 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.396 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.396 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.396 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.396 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.396 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.396 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.396 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.396 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.396 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.396 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.396 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.396 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.396 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.397 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.397 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.397 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.397 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.397 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.397 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.397 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.397 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.397 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.397 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.397 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.397 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.397 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.397 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.397 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.398 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.398 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.398 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.398 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.398 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.398 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.398 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.398 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.398 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.398 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.398 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.398 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.398 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.398 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.399 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.399 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.399 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.399 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.399 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.399 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.399 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.399 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.399 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.399 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.399 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.399 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.399 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.399 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.399 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.400 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.400 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.400 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "iostat", 2024-04-22T13:34:40.400 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.400 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.400 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.400 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.400 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.400 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.400 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.400 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.400 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.400 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.400 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.400 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.401 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.401 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.401 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.401 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.401 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.401 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.401 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.401 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.401 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.401 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.401 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.401 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.401 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.401 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.402 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.402 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.402 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.402 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.402 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.402 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.402 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.402 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.402 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.402 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.402 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.402 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.402 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.403 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.403 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.403 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.403 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.403 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.403 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.403 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.403 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.403 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.403 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.403 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.403 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.403 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.403 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.404 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.404 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.404 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.404 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.404 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.404 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.404 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.404 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.404 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.404 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.404 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.404 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.404 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.404 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.405 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.405 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.405 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.405 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.405 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.405 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "k8sevents", 2024-04-22T13:34:40.405 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.405 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.405 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.405 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_event_retention_days": { 2024-04-22T13:34:40.405 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ceph_event_retention_days", 2024-04-22T13:34:40.405 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.405 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.405 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.406 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7", 2024-04-22T13:34:40.406 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.406 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.406 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.406 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-22T13:34:40.406 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.406 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.406 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.406 INFO:teuthology.orchestra.run.smithi012.stdout: "config_check_secs": { 2024-04-22T13:34:40.406 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_check_secs", 2024-04-22T13:34:40.406 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.406 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.406 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.407 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-22T13:34:40.407 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "10", 2024-04-22T13:34:40.407 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.407 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.407 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-22T13:34:40.407 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.407 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.407 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.407 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.407 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.407 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.407 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.407 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.407 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.408 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.408 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.408 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.408 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.408 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.408 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.408 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.408 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.408 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.408 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.408 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.408 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.408 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.408 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.409 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.409 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.409 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.409 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.409 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.409 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.409 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.409 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.409 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.409 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.409 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.409 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.409 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.409 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.409 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.410 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.410 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.410 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.410 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.410 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.410 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.410 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.410 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.410 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.410 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.410 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.410 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.410 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.410 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.410 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.411 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.411 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.411 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.411 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.411 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.411 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.411 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.411 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.411 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.411 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.411 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.411 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.411 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.411 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.412 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.412 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.412 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.412 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.412 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.412 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.412 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.412 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "localpool", 2024-04-22T13:34:40.412 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.412 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.412 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.412 INFO:teuthology.orchestra.run.smithi012.stdout: "failure_domain": { 2024-04-22T13:34:40.412 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "failure_domain", 2024-04-22T13:34:40.412 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.413 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.413 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.413 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "host", 2024-04-22T13:34:40.413 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.413 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.413 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.413 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "failure domain for any created local pool", 2024-04-22T13:34:40.413 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-22T13:34:40.413 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.413 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.413 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.413 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.413 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.414 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.414 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.414 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.414 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.414 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.414 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.414 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.414 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.414 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.414 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.414 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.414 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.414 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.414 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.415 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.415 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.415 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.415 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.415 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.415 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.415 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.415 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.415 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.416 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.416 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.416 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.416 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.416 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.416 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.416 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.416 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.416 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.416 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.416 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.416 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.416 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.416 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.417 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.417 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.417 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.417 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.417 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.417 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.417 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.417 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.417 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.417 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.417 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.417 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.417 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.418 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.418 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.418 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.418 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.418 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.418 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.418 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.418 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.418 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.418 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.418 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.418 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.418 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.419 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": { 2024-04-22T13:34:40.419 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_size", 2024-04-22T13:34:40.419 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.419 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.419 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.419 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.419 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.419 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.419 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.419 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default min_size for any created local pool", 2024-04-22T13:34:40.419 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-22T13:34:40.419 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.419 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.419 INFO:teuthology.orchestra.run.smithi012.stdout: "num_rep": { 2024-04-22T13:34:40.420 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "num_rep", 2024-04-22T13:34:40.420 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.420 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.420 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.420 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3", 2024-04-22T13:34:40.420 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.420 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.420 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.420 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default replica count for any created local pool", 2024-04-22T13:34:40.420 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.420 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.420 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.420 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": { 2024-04-22T13:34:40.421 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_num", 2024-04-22T13:34:40.421 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.421 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.421 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.421 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "128", 2024-04-22T13:34:40.421 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.421 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.421 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.421 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default pg_num for any created local pool", 2024-04-22T13:34:40.421 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.421 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.421 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.421 INFO:teuthology.orchestra.run.smithi012.stdout: "prefix": { 2024-04-22T13:34:40.422 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prefix", 2024-04-22T13:34:40.422 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.422 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.422 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.422 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.422 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.422 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.422 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.422 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name prefix for any created local pool", 2024-04-22T13:34:40.422 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.422 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.422 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.422 INFO:teuthology.orchestra.run.smithi012.stdout: "subtree": { 2024-04-22T13:34:40.422 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "subtree", 2024-04-22T13:34:40.423 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.423 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.423 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.423 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "rack", 2024-04-22T13:34:40.423 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.423 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.423 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.423 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-22T13:34:40.423 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-22T13:34:40.423 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.423 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.423 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.423 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.424 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.424 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mds_autoscaler", 2024-04-22T13:34:40.424 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.424 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.424 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.424 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.424 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.424 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.424 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.424 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.424 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.424 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.424 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.424 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.424 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.425 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.425 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.425 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.425 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.425 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.425 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.425 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.425 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.425 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.425 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.425 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.425 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.425 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.425 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.426 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.426 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.426 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.426 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.426 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.426 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.426 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.426 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.426 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.426 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.426 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.426 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.426 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.427 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.427 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.427 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.427 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.427 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.427 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.427 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.427 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.427 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.427 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.427 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.427 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.427 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.427 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.427 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.428 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.428 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.428 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.428 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.428 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.428 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.428 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.428 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.428 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.428 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.428 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.428 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.429 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.429 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.429 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.429 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.429 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.429 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirroring", 2024-04-22T13:34:40.429 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.429 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.429 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.429 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.429 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.429 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.429 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.429 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.430 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.430 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.430 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.430 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.430 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.430 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.430 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.430 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.430 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.430 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.430 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.430 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.430 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.430 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.431 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.431 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.431 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.431 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.431 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.431 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.431 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.431 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.431 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.431 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.431 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.431 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.431 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.431 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.432 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.432 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.432 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.432 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.432 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.432 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.432 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.432 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.432 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.432 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.432 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.432 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.432 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.433 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.433 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.433 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.433 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.433 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.433 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.433 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.433 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.433 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.433 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.433 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.433 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.433 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.433 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.434 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.434 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.434 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.434 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.434 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.434 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.434 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.434 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.434 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "nfs", 2024-04-22T13:34:40.434 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.434 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.435 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.435 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.435 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.435 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.435 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.435 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.435 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.435 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.435 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.435 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.435 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.435 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.435 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.435 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.435 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.436 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.436 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.436 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.436 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.436 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.436 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.436 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.436 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.436 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.436 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.436 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.436 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.436 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.437 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.437 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.437 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.437 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.437 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.437 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.437 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.437 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.437 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.437 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.437 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.438 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.438 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.438 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.438 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.438 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.438 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.438 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.438 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.438 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.438 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.439 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.439 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.439 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.439 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.439 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.439 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.439 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.439 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.439 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.439 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.440 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.440 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.440 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.440 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.440 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.440 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.440 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.440 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.440 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.440 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.440 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-04-22T13:34:40.440 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.440 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.441 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.441 INFO:teuthology.orchestra.run.smithi012.stdout: "fail_fs": { 2024-04-22T13:34:40.441 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "fail_fs", 2024-04-22T13:34:40.441 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.441 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.441 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.441 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.441 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.441 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.441 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.441 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-22T13:34:40.442 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.442 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.442 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.442 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.442 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.442 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.442 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.442 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.442 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.442 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.442 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.442 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.442 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.443 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.443 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.443 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.443 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.443 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.443 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.443 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.443 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.443 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.443 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.443 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.443 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.444 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.444 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.444 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.444 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.444 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.444 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.444 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.444 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.444 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.444 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.444 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.444 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.444 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.445 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.445 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.445 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.445 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.445 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.445 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.445 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.445 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.445 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.445 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.445 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.445 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.445 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.445 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.445 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.446 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.446 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.446 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.446 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.446 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.446 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.446 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.446 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.446 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.446 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.446 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.446 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.446 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.447 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.447 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.447 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.447 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator": { 2024-04-22T13:34:40.447 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-04-22T13:34:40.447 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.447 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.447 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.447 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.447 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.447 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.447 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.447 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm", 2024-04-22T13:34:40.447 INFO:teuthology.orchestra.run.smithi012.stdout: "rook", 2024-04-22T13:34:40.448 INFO:teuthology.orchestra.run.smithi012.stdout: "test_orchestrator" 2024-04-22T13:34:40.448 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.448 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Orchestrator backend", 2024-04-22T13:34:40.448 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.448 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.448 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.448 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.448 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.448 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_perf_query", 2024-04-22T13:34:40.448 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.448 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.448 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.449 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.449 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.449 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.449 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.449 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.449 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.449 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.449 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.449 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.449 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.449 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.449 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.449 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.449 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.450 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.450 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.450 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.450 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.450 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.450 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.450 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.450 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.450 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.450 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.450 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.450 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.450 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.451 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.451 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.451 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.451 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.451 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.451 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.451 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.451 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.451 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.451 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.451 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.451 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.451 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.451 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.452 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.452 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.452 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.452 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.452 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.452 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.452 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.452 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.452 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.452 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.452 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.452 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.452 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.453 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.453 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.453 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.453 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.453 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.453 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.453 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.453 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.453 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.453 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.453 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.453 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.453 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.453 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.454 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.454 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.454 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_support", 2024-04-22T13:34:40.454 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.454 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.454 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.454 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.454 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.454 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.454 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.454 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.454 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.454 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.454 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.455 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.455 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.455 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.455 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.455 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.455 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.455 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.455 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.455 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.455 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.455 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.455 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.455 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.456 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.456 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.456 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.456 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.456 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.456 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.456 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.456 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.456 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.456 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.456 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.456 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.456 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.456 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.456 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.457 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.457 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.457 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.457 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.457 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.457 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.457 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.457 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.457 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.457 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.457 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.457 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.457 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.458 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.458 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.458 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.458 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.458 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.458 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.458 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.458 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.458 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.458 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.458 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.458 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.458 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.458 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.459 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.459 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.459 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.459 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.459 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.459 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_autoscaler", 2024-04-22T13:34:40.459 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.459 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.459 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.459 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.459 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.459 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.460 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.460 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.460 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.460 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.460 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.460 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.460 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.460 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.460 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.460 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.460 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.460 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.460 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.460 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.460 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.461 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.461 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.461 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.461 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.461 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.461 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.461 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.461 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.461 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.461 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.461 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.461 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.461 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.461 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.462 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.462 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.462 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.462 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.462 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.462 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.462 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.462 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.462 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.462 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.462 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.462 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.462 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.463 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.463 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.463 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.463 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.463 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.463 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.463 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.463 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.463 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.463 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.463 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.463 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.463 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.464 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.464 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.464 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.464 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.464 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.464 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.464 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.464 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.464 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-04-22T13:34:40.464 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-04-22T13:34:40.464 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.464 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.464 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.464 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-04-22T13:34:40.465 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.465 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.465 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.465 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.465 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.465 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.465 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.465 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.465 INFO:teuthology.orchestra.run.smithi012.stdout: "threshold": { 2024-04-22T13:34:40.465 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threshold", 2024-04-22T13:34:40.465 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-22T13:34:40.465 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.465 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.465 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-04-22T13:34:40.466 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1.0", 2024-04-22T13:34:40.466 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.466 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.466 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "scaling threshold", 2024-04-22T13:34:40.466 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-22T13:34:40.466 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.466 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.466 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.466 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.466 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "progress", 2024-04-22T13:34:40.466 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.466 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.467 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.467 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_pg_recovery_event": { 2024-04-22T13:34:40.467 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_pg_recovery_event", 2024-04-22T13:34:40.467 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.467 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.467 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.467 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.467 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.467 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.467 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.467 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-22T13:34:40.467 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.467 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.467 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.468 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-04-22T13:34:40.468 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-04-22T13:34:40.468 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.468 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.468 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.468 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.468 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.468 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.468 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.468 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.468 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.468 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.468 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.468 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.469 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.469 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.469 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.469 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.469 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.469 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.469 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.469 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.469 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.469 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.469 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.469 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.469 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.469 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.469 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.470 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.470 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.470 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.470 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.470 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.470 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.470 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.470 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.470 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.470 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.470 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.470 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.471 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.471 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.471 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.471 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.471 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.471 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.471 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.471 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.471 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.471 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.471 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.471 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.471 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.471 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.472 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.472 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.472 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.472 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.472 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.472 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.472 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.472 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.472 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.472 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.472 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.472 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.472 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.473 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.473 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.473 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.473 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.473 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.473 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.473 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.473 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.473 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.473 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.473 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.473 INFO:teuthology.orchestra.run.smithi012.stdout: "max_completed_events": { 2024-04-22T13:34:40.473 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_completed_events", 2024-04-22T13:34:40.474 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.474 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.474 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.474 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "50", 2024-04-22T13:34:40.474 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.474 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.474 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.474 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "number of past completed events to remember", 2024-04-22T13:34:40.474 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.474 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.474 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.474 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-04-22T13:34:40.474 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-04-22T13:34:40.475 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.475 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.475 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.475 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-04-22T13:34:40.475 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.475 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.475 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.475 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long the module is going to sleep", 2024-04-22T13:34:40.475 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.475 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.475 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.475 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.475 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus", 2024-04-22T13:34:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: "cache": { 2024-04-22T13:34:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cache", 2024-04-22T13:34:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.476 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "exclude_perf_counters": { 2024-04-22T13:34:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "exclude_perf_counters", 2024-04-22T13:34:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-22T13:34:40.477 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-22T13:34:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.478 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.479 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.480 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.481 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.482 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools": { 2024-04-22T13:34:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools", 2024-04-22T13:34:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.483 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-22T13:34:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-22T13:34:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "300", 2024-04-22T13:34:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.484 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_interval": { 2024-04-22T13:34:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_interval", 2024-04-22T13:34:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-22T13:34:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15.0", 2024-04-22T13:34:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.485 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-04-22T13:34:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-04-22T13:34:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-04-22T13:34:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.486 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-22T13:34:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-04-22T13:34:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-04-22T13:34:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "9283", 2024-04-22T13:34:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-22T13:34:40.487 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: "stale_cache_strategy": { 2024-04-22T13:34:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stale_cache_strategy", 2024-04-22T13:34:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "log", 2024-04-22T13:34:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.488 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.489 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.489 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-04-22T13:34:40.489 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-04-22T13:34:40.489 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.489 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.489 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.489 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "default", 2024-04-22T13:34:40.489 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.489 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.489 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.489 INFO:teuthology.orchestra.run.smithi012.stdout: "default", 2024-04-22T13:34:40.489 INFO:teuthology.orchestra.run.smithi012.stdout: "error" 2024-04-22T13:34:40.489 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.489 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.489 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.490 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.490 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.490 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-04-22T13:34:40.490 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-04-22T13:34:40.490 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.490 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.490 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.490 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-04-22T13:34:40.490 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-04-22T13:34:40.490 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-04-22T13:34:40.490 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.490 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.490 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.491 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.491 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.491 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.491 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.491 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.491 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_support", 2024-04-22T13:34:40.491 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.491 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.491 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.491 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.491 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.491 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.491 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.491 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.492 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.492 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.492 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.492 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.492 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.492 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.492 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.492 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.492 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.492 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.492 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.492 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.492 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.492 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.492 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.493 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.493 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.493 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.493 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.493 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.493 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.493 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.493 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.493 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.493 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.493 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.493 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.493 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.494 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.494 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.494 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.494 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.494 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.494 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.494 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.494 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.494 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.494 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.494 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.494 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.494 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.494 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.495 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.495 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.495 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.495 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.495 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.495 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.495 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.495 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.495 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.495 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.495 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.495 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.495 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.496 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.496 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.496 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.496 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.496 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.496 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.496 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_snap_create": { 2024-04-22T13:34:40.496 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_snap_create", 2024-04-22T13:34:40.496 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.496 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.496 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.496 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-22T13:34:40.496 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: "mirror_snapshot_schedule": { 2024-04-22T13:34:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirror_snapshot_schedule", 2024-04-22T13:34:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.497 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.498 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.498 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.498 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.498 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.498 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.498 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.498 INFO:teuthology.orchestra.run.smithi012.stdout: "trash_purge_schedule": { 2024-04-22T13:34:40.498 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "trash_purge_schedule", 2024-04-22T13:34:40.498 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.498 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.498 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.498 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.498 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.499 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.499 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.499 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.499 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.499 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.499 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.499 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.499 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.499 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.499 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.499 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "restful", 2024-04-22T13:34:40.499 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.499 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.499 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.499 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_auth": { 2024-04-22T13:34:40.500 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_auth", 2024-04-22T13:34:40.500 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.500 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.500 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.500 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.500 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.500 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.500 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.500 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.500 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.500 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.500 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.500 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-04-22T13:34:40.501 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-04-22T13:34:40.501 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.501 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.501 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.501 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.501 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.501 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.501 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.501 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.501 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.501 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.501 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.501 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.502 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.502 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.502 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.502 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.502 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.502 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.502 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.502 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.502 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.502 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.502 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.502 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.502 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.502 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.503 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.503 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.503 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.503 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.503 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.503 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.503 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.503 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.503 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.503 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.503 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.503 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.503 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.503 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.504 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.504 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.504 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.504 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.504 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.504 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.504 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.504 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.504 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.504 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.504 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.504 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.504 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.504 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.505 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.505 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.505 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.505 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.505 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.505 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.505 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.505 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.505 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.505 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.505 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.505 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.505 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.505 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.506 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.506 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.506 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.506 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.506 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.506 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.506 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.506 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.506 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.506 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.506 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.506 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.506 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.507 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-04-22T13:34:40.507 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-04-22T13:34:40.507 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.507 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.507 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.507 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.507 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.507 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.507 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.507 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.507 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.507 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.507 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.507 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.507 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-04-22T13:34:40.508 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-04-22T13:34:40.508 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.508 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.508 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.508 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.508 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.508 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.508 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.508 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.508 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.508 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.508 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.508 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.508 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.508 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.508 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.509 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rgw", 2024-04-22T13:34:40.509 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.509 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.509 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.509 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.509 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.509 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.509 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.509 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.509 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.509 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.509 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.509 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.509 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.510 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.510 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.510 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.510 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.510 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.510 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.510 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.510 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.510 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.510 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.510 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.510 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.510 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.511 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.511 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.511 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.511 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.511 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.511 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.511 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.511 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.511 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.511 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.511 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.511 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.511 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.511 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.512 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.512 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.512 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.512 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.512 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.512 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.512 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.512 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.512 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.512 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.512 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.512 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.512 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.512 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.513 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.513 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.513 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.513 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.513 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.513 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.513 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.513 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.513 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.513 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.513 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.513 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.513 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.513 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.513 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.514 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.514 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.514 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.514 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.514 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.514 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.514 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.514 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rook", 2024-04-22T13:34:40.514 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.514 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.514 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.514 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.514 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.514 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.515 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.515 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.515 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.515 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.515 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.515 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.515 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.515 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.515 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.515 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.515 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.515 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.515 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.515 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.516 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.516 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.516 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.516 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.516 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.516 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.516 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.516 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.516 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.516 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.516 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.516 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.516 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.516 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.516 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.517 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.517 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.517 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.517 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.517 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.517 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.517 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.517 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.517 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.517 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.517 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.517 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.517 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.517 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.518 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.518 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.518 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.518 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.518 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.518 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.518 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.518 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.518 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.518 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.518 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.518 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.518 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.518 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.519 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.519 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.519 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.519 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.519 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.519 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.519 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.519 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.519 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.519 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.519 INFO:teuthology.orchestra.run.smithi012.stdout: "storage_class": { 2024-04-22T13:34:40.519 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "storage_class", 2024-04-22T13:34:40.519 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.519 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.519 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.520 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "local", 2024-04-22T13:34:40.520 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.520 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.520 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.520 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-22T13:34:40.520 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.520 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.520 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.520 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.520 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.520 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.520 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.520 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "selftest", 2024-04-22T13:34:40.520 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.521 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.521 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.521 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.521 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.521 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.521 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.521 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.521 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.521 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.521 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.521 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.521 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.521 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.521 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.521 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.522 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.522 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.522 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.522 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.522 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.522 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.522 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.522 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.522 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.522 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.522 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.522 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.522 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.522 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.522 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.523 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.523 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.523 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.523 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.523 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.523 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.523 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.523 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.523 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.523 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.523 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.523 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.523 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.523 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.524 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.524 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.524 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.524 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.524 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.524 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.524 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.524 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.524 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.524 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.524 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.524 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.524 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.524 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.525 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.525 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.525 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.525 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.525 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.525 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.525 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.525 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.525 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.525 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.525 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.525 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.525 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.526 INFO:teuthology.orchestra.run.smithi012.stdout: "roption1": { 2024-04-22T13:34:40.526 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption1", 2024-04-22T13:34:40.526 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.526 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.526 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.526 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.526 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.526 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.526 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.526 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.526 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.526 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.526 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.527 INFO:teuthology.orchestra.run.smithi012.stdout: "roption2": { 2024-04-22T13:34:40.527 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption2", 2024-04-22T13:34:40.527 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.527 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.527 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.527 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "xyz", 2024-04-22T13:34:40.527 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.527 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.527 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.527 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.527 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.527 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.527 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.528 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption1": { 2024-04-22T13:34:40.528 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption1", 2024-04-22T13:34:40.528 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.528 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.528 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.528 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.528 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.528 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.528 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.528 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.528 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.528 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.528 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.528 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.529 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption2": { 2024-04-22T13:34:40.529 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption2", 2024-04-22T13:34:40.529 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.529 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.529 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.529 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.529 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.529 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.529 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.529 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.529 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.529 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.529 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.530 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption3": { 2024-04-22T13:34:40.530 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption3", 2024-04-22T13:34:40.530 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-22T13:34:40.530 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.530 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.530 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.530 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.530 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.530 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.530 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.530 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.530 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.530 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.530 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.531 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption4": { 2024-04-22T13:34:40.531 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption4", 2024-04-22T13:34:40.531 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.531 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.531 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.531 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.531 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.531 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.531 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.531 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.531 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.531 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.531 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.532 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption5": { 2024-04-22T13:34:40.532 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption5", 2024-04-22T13:34:40.532 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.532 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.532 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.532 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.532 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.532 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.532 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.532 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.532 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.532 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.532 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.533 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption6": { 2024-04-22T13:34:40.533 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption6", 2024-04-22T13:34:40.533 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.533 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.533 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.533 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.533 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.533 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.533 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.533 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.533 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.533 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.533 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.534 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption7": { 2024-04-22T13:34:40.534 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption7", 2024-04-22T13:34:40.534 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.534 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.534 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.534 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.534 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-04-22T13:34:40.534 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "42", 2024-04-22T13:34:40.534 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.534 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.534 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.534 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.534 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.534 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.534 INFO:teuthology.orchestra.run.smithi012.stdout: "testkey": { 2024-04-22T13:34:40.535 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testkey", 2024-04-22T13:34:40.535 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.535 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.535 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.535 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.535 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.535 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.535 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.535 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.535 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.535 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.535 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.535 INFO:teuthology.orchestra.run.smithi012.stdout: "testlkey": { 2024-04-22T13:34:40.535 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testlkey", 2024-04-22T13:34:40.536 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.536 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.536 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.536 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.536 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.536 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.536 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.536 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.536 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.536 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.536 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.536 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.536 INFO:teuthology.orchestra.run.smithi012.stdout: "testnewline": { 2024-04-22T13:34:40.536 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testnewline", 2024-04-22T13:34:40.537 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.537 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.537 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.537 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.537 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.537 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.537 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.537 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.537 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.537 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.537 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.537 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.537 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.538 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.538 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snap_schedule", 2024-04-22T13:34:40.538 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.538 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.538 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.538 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_m_granularity": { 2024-04-22T13:34:40.538 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_m_granularity", 2024-04-22T13:34:40.538 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.538 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.538 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.538 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.538 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.538 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.539 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.539 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow minute scheduled snapshots", 2024-04-22T13:34:40.539 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.539 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.539 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.539 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.539 INFO:teuthology.orchestra.run.smithi012.stdout: "dump_on_update": { 2024-04-22T13:34:40.539 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dump_on_update", 2024-04-22T13:34:40.539 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.539 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.539 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.539 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.539 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.539 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.540 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.540 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "dump database to debug log on update", 2024-04-22T13:34:40.540 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.540 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.540 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.540 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.540 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.540 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.540 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.540 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.540 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.540 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.540 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.540 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.541 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.541 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.541 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.541 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.541 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.541 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.541 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.541 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.541 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.541 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.541 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.541 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.541 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.541 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.542 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.542 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.542 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.542 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.542 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.542 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.542 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.542 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.542 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.542 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.542 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.542 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.542 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.543 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.543 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.543 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.543 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.543 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.543 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.543 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.543 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.543 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.543 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.543 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.543 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.543 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.543 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.544 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.544 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.544 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.544 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.544 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.544 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.544 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.544 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.544 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.544 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.544 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.544 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.544 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.544 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.545 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.545 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.545 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.545 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.545 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stats", 2024-04-22T13:34:40.545 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.545 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.545 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.545 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.545 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.545 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.545 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.545 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.546 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.546 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.546 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.546 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.546 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.546 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.546 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.546 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.546 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.546 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.546 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.546 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.546 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.546 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.547 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.547 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.547 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.547 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.547 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.547 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.547 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.547 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.547 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.547 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.547 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.547 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.547 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.547 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.548 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.548 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.548 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.548 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.548 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.548 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.548 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.548 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.548 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.548 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.548 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.548 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.548 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.548 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.549 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.549 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.549 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.549 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.549 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.549 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.549 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.549 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.549 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.549 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.549 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.549 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.549 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.550 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.550 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.550 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.550 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.550 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.550 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.550 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.550 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.550 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.550 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "status", 2024-04-22T13:34:40.550 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.550 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.550 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.551 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.551 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.551 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.551 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.551 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.551 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.551 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.551 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.551 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.551 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.551 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.551 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.551 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.551 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.552 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.552 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.552 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.552 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.552 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.552 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.552 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.552 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.552 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.552 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.552 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.552 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.552 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.552 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.553 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.553 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.553 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.553 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.553 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.553 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.553 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.553 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.553 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.553 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.553 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.553 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.553 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.553 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.554 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.554 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.554 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.554 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.554 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.554 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.554 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.554 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.554 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.554 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.554 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.554 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.554 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.555 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.555 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.555 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.555 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.555 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.555 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.555 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.555 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.555 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.555 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.555 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.555 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.555 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.555 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.556 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telegraf", 2024-04-22T13:34:40.556 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.556 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.556 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.556 INFO:teuthology.orchestra.run.smithi012.stdout: "address": { 2024-04-22T13:34:40.556 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "address", 2024-04-22T13:34:40.556 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.556 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.556 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.556 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-22T13:34:40.556 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.556 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.556 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.556 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.557 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.557 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.557 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.557 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-04-22T13:34:40.557 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-04-22T13:34:40.557 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.557 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.557 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.557 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15", 2024-04-22T13:34:40.557 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.557 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.557 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.557 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.557 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.558 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.558 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.558 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.558 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.558 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.558 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.558 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.558 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.558 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.558 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.558 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.558 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.558 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.559 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.559 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.559 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.559 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.559 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.559 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.559 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.559 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.559 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.559 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.559 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.559 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.559 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.559 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.560 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.560 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.560 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.560 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.560 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.560 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.560 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.560 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.560 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.560 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.560 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.560 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.560 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.561 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.561 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.561 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.561 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.561 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.561 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.561 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.561 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.561 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.561 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.561 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.561 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.561 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.561 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.561 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.562 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.562 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.562 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.562 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.562 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.562 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.562 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.562 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.562 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.562 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.562 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.562 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.562 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.562 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.563 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.563 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.563 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telemetry", 2024-04-22T13:34:40.563 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.563 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.563 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.563 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_basic": { 2024-04-22T13:34:40.563 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_basic", 2024-04-22T13:34:40.563 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.563 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.563 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.563 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.563 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.564 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.564 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.564 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-22T13:34:40.564 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.564 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.564 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.564 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_crash": { 2024-04-22T13:34:40.564 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_crash", 2024-04-22T13:34:40.564 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.564 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.564 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.564 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.564 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.565 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.565 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.565 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-22T13:34:40.565 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.565 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.565 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.565 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_device": { 2024-04-22T13:34:40.565 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_device", 2024-04-22T13:34:40.565 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.565 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.565 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.565 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.566 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.566 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.566 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.566 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-22T13:34:40.566 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.566 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.566 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.566 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_ident": { 2024-04-22T13:34:40.566 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_ident", 2024-04-22T13:34:40.566 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.566 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.566 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.566 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.567 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.567 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.567 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.567 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-22T13:34:40.567 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.567 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.567 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.567 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_perf": { 2024-04-22T13:34:40.567 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_perf", 2024-04-22T13:34:40.567 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.567 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.567 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.567 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.567 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.568 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.568 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.568 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-22T13:34:40.568 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.568 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.568 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.568 INFO:teuthology.orchestra.run.smithi012.stdout: "contact": { 2024-04-22T13:34:40.568 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "contact", 2024-04-22T13:34:40.568 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.568 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.568 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.568 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.568 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.569 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.569 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.569 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.569 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.569 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.569 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.569 INFO:teuthology.orchestra.run.smithi012.stdout: "description": { 2024-04-22T13:34:40.569 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "description", 2024-04-22T13:34:40.569 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.569 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.569 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.569 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.569 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.569 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.570 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.570 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.570 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.570 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.570 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.570 INFO:teuthology.orchestra.run.smithi012.stdout: "device_url": { 2024-04-22T13:34:40.570 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_url", 2024-04-22T13:34:40.570 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.570 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.570 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.570 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-22T13:34:40.570 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.570 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.571 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.571 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.571 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.571 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.571 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.571 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-04-22T13:34:40.571 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-04-22T13:34:40.571 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.571 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.571 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.571 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.571 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.571 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.571 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.572 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.572 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.572 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.572 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.572 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-04-22T13:34:40.572 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-04-22T13:34:40.572 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.572 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.572 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.572 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "24", 2024-04-22T13:34:40.572 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "8", 2024-04-22T13:34:40.572 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.572 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.573 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.573 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.573 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.573 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.573 INFO:teuthology.orchestra.run.smithi012.stdout: "last_opt_revision": { 2024-04-22T13:34:40.573 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "last_opt_revision", 2024-04-22T13:34:40.573 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.573 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.573 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.573 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1", 2024-04-22T13:34:40.573 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.573 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.574 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.574 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.574 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.574 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.574 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.574 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard": { 2024-04-22T13:34:40.574 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard", 2024-04-22T13:34:40.574 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.574 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.574 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.574 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.574 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.574 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.575 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.575 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.575 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.575 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.575 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.575 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard_description": { 2024-04-22T13:34:40.575 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard_description", 2024-04-22T13:34:40.575 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.575 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.575 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.575 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.575 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.575 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.576 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.576 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.576 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.576 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.576 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.576 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.576 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.576 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.576 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.576 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.576 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.576 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.576 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.576 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.577 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.577 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.577 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.577 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.577 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.577 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.577 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.577 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.577 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.577 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.577 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.577 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.577 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.578 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.578 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.578 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.578 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.578 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.578 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.578 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.578 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.578 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.578 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.578 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.579 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.579 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.579 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.579 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.579 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.579 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.579 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.579 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.579 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.579 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.579 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.579 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.579 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.579 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.579 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.580 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.580 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.580 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.580 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.580 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.580 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.580 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.580 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.580 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.580 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.580 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.580 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.580 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.580 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.581 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.581 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.581 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.581 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.581 INFO:teuthology.orchestra.run.smithi012.stdout: "organization": { 2024-04-22T13:34:40.581 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "organization", 2024-04-22T13:34:40.581 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.581 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.581 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.581 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.581 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.581 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.581 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.582 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.582 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.582 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.582 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.582 INFO:teuthology.orchestra.run.smithi012.stdout: "proxy": { 2024-04-22T13:34:40.582 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "proxy", 2024-04-22T13:34:40.582 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.582 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.582 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.582 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.582 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.582 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.582 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.583 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.583 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.583 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.583 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.583 INFO:teuthology.orchestra.run.smithi012.stdout: "url": { 2024-04-22T13:34:40.583 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url", 2024-04-22T13:34:40.583 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.583 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.583 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.583 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-22T13:34:40.583 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.583 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.583 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.583 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.584 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.584 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.584 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.584 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.584 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.584 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "test_orchestrator", 2024-04-22T13:34:40.584 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.584 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.584 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.584 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.584 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.584 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.584 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.585 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.585 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.585 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.585 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.585 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.585 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.585 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.585 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.585 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.585 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.585 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.585 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.585 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.585 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.585 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.586 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.586 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.586 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.586 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.586 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.586 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.586 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.586 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.586 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.586 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.586 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.586 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.586 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.587 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.587 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.587 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.587 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.587 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.587 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.587 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.587 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.587 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.587 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.587 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.587 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.587 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.587 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.588 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.588 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.588 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.588 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.588 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.588 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.588 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.588 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.588 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.588 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.588 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.588 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.588 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.588 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.589 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.589 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.589 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.589 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.589 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.589 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.589 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.589 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.589 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "volumes", 2024-04-22T13:34:40.589 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.589 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.589 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.589 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.590 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.590 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.590 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.590 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.590 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.590 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.590 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.590 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.590 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.590 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.590 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.590 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.590 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.590 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.591 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.591 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.591 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.591 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.591 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.591 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.591 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.591 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.591 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.591 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.591 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.591 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.591 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.592 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.592 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.592 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.592 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.592 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.592 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.592 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.592 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.592 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.592 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.592 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.592 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.592 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.592 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.593 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.593 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.593 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.593 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.593 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.593 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.593 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.593 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.593 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.593 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.593 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.593 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.593 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.594 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.594 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.594 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.594 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.594 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.594 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.594 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.594 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.594 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.594 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.594 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.594 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_clones": { 2024-04-22T13:34:40.594 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_clones", 2024-04-22T13:34:40.595 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.595 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.595 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.595 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4", 2024-04-22T13:34:40.595 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.595 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.595 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.595 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-22T13:34:40.595 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.595 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.595 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.595 INFO:teuthology.orchestra.run.smithi012.stdout: "periodic_async_work": { 2024-04-22T13:34:40.595 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "periodic_async_work", 2024-04-22T13:34:40.596 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.596 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.596 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.596 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.596 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.596 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.596 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.596 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Periodically check for async work", 2024-04-22T13:34:40.596 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.596 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.596 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.596 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_delay": { 2024-04-22T13:34:40.596 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_delay", 2024-04-22T13:34:40.597 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.597 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.597 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.597 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-04-22T13:34:40.597 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.597 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.597 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.597 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-22T13:34:40.597 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.597 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.597 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.597 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_no_wait": { 2024-04-22T13:34:40.597 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_no_wait", 2024-04-22T13:34:40.598 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.598 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.598 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.598 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.598 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.598 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.598 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.598 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-22T13:34:40.598 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.598 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.598 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.598 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.598 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.599 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.599 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix", 2024-04-22T13:34:40.599 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.599 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.599 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.599 INFO:teuthology.orchestra.run.smithi012.stdout: "discovery_interval": { 2024-04-22T13:34:40.599 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "discovery_interval", 2024-04-22T13:34:40.599 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-04-22T13:34:40.599 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.599 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.599 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "100", 2024-04-22T13:34:40.599 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.599 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.600 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.600 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.600 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.600 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.600 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.600 INFO:teuthology.orchestra.run.smithi012.stdout: "identifier": { 2024-04-22T13:34:40.600 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "identifier", 2024-04-22T13:34:40.600 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.600 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.600 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.600 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.600 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.600 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.600 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.601 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.601 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.601 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.601 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.601 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.601 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-04-22T13:34:40.601 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-04-22T13:34:40.601 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.601 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.601 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.601 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-04-22T13:34:40.601 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.601 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.601 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.602 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.602 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.602 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.602 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.602 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.602 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.602 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.602 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.602 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.602 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.602 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.602 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.602 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.603 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.603 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.603 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.603 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.603 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.603 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.603 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.603 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.603 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.603 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.603 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.603 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.603 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.603 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.603 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.604 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.604 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.604 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.604 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.604 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.604 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.604 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.604 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.604 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.604 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.604 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.604 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.604 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.604 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.605 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.605 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.605 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.605 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.605 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.605 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.605 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.605 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.605 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.605 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.605 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.605 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.605 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.605 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.606 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.606 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.606 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.606 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.606 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.606 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.606 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.606 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.606 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.606 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.606 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.606 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.606 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.607 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.607 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.607 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.607 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.607 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_host": { 2024-04-22T13:34:40.607 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_host", 2024-04-22T13:34:40.607 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.607 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.607 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.607 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.607 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.607 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.607 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.607 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.608 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.608 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.608 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.608 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_port": { 2024-04-22T13:34:40.608 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_port", 2024-04-22T13:34:40.608 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.608 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.608 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.608 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10051", 2024-04-22T13:34:40.608 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.608 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.608 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.608 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.608 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.609 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.609 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.609 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_sender": { 2024-04-22T13:34:40.609 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_sender", 2024-04-22T13:34:40.609 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.609 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.609 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.609 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-22T13:34:40.609 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.609 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.609 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.609 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.609 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.610 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.610 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.610 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.610 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.610 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.610 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-22T13:34:40.610 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.610 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.610 INFO:teuthology.orchestra.run.smithi012.stdout: "modules": [ 2024-04-22T13:34:40.610 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm", 2024-04-22T13:34:40.610 INFO:teuthology.orchestra.run.smithi012.stdout: "dashboard", 2024-04-22T13:34:40.610 INFO:teuthology.orchestra.run.smithi012.stdout: "iostat", 2024-04-22T13:34:40.610 INFO:teuthology.orchestra.run.smithi012.stdout: "nfs", 2024-04-22T13:34:40.610 INFO:teuthology.orchestra.run.smithi012.stdout: "restful", 2024-04-22T13:34:40.611 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_schedule" 2024-04-22T13:34:40.611 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.611 INFO:teuthology.orchestra.run.smithi012.stdout: "available_modules": [ 2024-04-22T13:34:40.611 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.611 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "alerts", 2024-04-22T13:34:40.611 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.611 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.611 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.611 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-04-22T13:34:40.611 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-04-22T13:34:40.611 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.611 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.611 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.611 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-04-22T13:34:40.611 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.612 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.612 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.612 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How frequently to reexamine health status", 2024-04-22T13:34:40.612 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.612 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.612 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.612 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.612 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.612 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.612 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.612 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.612 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.612 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.612 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.612 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.612 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.613 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.613 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.613 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.613 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.613 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.613 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.613 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.613 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.613 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.613 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.613 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.613 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.613 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.613 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.614 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.614 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.614 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.614 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.614 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.614 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.614 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.614 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.614 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.614 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.614 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.614 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.614 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.615 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.615 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.615 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.615 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.615 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.615 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.615 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.615 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.615 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.615 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.615 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.615 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.615 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.615 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.616 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.616 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.616 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.616 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.616 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.616 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.616 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.616 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.616 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.616 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.616 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.616 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.616 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.616 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.616 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.617 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_destination": { 2024-04-22T13:34:40.617 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_destination", 2024-04-22T13:34:40.617 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.617 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.617 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.617 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.617 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.617 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.617 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.617 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email address to send alerts to", 2024-04-22T13:34:40.617 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.617 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.617 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.618 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_from_name": { 2024-04-22T13:34:40.618 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_from_name", 2024-04-22T13:34:40.618 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.618 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.618 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.618 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "Ceph", 2024-04-22T13:34:40.618 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.618 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.618 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.618 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Email From: name", 2024-04-22T13:34:40.618 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.618 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.618 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.619 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_host": { 2024-04-22T13:34:40.619 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_host", 2024-04-22T13:34:40.619 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.619 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.619 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.619 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.619 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.619 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.619 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.619 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP server", 2024-04-22T13:34:40.619 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.619 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.619 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.620 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_password": { 2024-04-22T13:34:40.620 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_password", 2024-04-22T13:34:40.620 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.620 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.620 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.620 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.620 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.620 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.620 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.620 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Password to authenticate with", 2024-04-22T13:34:40.620 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.621 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.621 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.621 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_port": { 2024-04-22T13:34:40.621 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_port", 2024-04-22T13:34:40.621 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.621 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.621 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.621 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "465", 2024-04-22T13:34:40.621 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.621 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.621 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.621 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP port", 2024-04-22T13:34:40.621 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.621 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.622 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.622 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_sender": { 2024-04-22T13:34:40.622 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_sender", 2024-04-22T13:34:40.622 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.622 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.622 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.622 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.622 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.622 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.622 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.622 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SMTP envelope sender", 2024-04-22T13:34:40.622 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.623 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.623 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.623 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_ssl": { 2024-04-22T13:34:40.623 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_ssl", 2024-04-22T13:34:40.623 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.623 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.623 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.623 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.623 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.623 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.623 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.623 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-22T13:34:40.624 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.624 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.624 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.624 INFO:teuthology.orchestra.run.smithi012.stdout: "smtp_user": { 2024-04-22T13:34:40.624 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "smtp_user", 2024-04-22T13:34:40.624 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.624 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.624 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.625 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.625 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.625 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.625 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.625 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "User to authenticate as", 2024-04-22T13:34:40.625 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.625 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.625 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.625 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.625 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.625 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.626 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "balancer", 2024-04-22T13:34:40.626 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.626 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.626 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.626 INFO:teuthology.orchestra.run.smithi012.stdout: "active": { 2024-04-22T13:34:40.626 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "active", 2024-04-22T13:34:40.626 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.626 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.626 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.626 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.626 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.626 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.626 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.627 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically balance PGs across cluster", 2024-04-22T13:34:40.627 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.627 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.627 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.627 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_time": { 2024-04-22T13:34:40.627 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_time", 2024-04-22T13:34:40.627 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.627 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.627 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.627 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0000", 2024-04-22T13:34:40.627 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.627 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.628 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.628 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "beginning time of day to automatically balance", 2024-04-22T13:34:40.628 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-22T13:34:40.628 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.628 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.628 INFO:teuthology.orchestra.run.smithi012.stdout: "begin_weekday": { 2024-04-22T13:34:40.628 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "begin_weekday", 2024-04-22T13:34:40.628 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-04-22T13:34:40.628 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.628 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.628 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-04-22T13:34:40.628 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-04-22T13:34:40.628 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-04-22T13:34:40.629 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.629 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-22T13:34:40.629 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-22T13:34:40.629 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.629 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.629 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_max_iterations": { 2024-04-22T13:34:40.629 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_max_iterations", 2024-04-22T13:34:40.629 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-04-22T13:34:40.629 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.629 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.629 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "25", 2024-04-22T13:34:40.629 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-04-22T13:34:40.629 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "250", 2024-04-22T13:34:40.629 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.630 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-22T13:34:40.630 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.630 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.630 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.630 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_metrics": { 2024-04-22T13:34:40.630 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_metrics", 2024-04-22T13:34:40.630 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.630 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.630 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.630 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "pgs,objects,bytes", 2024-04-22T13:34:40.630 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.630 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.630 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.631 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-22T13:34:40.631 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-22T13:34:40.631 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.631 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.631 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_compat_step": { 2024-04-22T13:34:40.631 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crush_compat_step", 2024-04-22T13:34:40.631 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-22T13:34:40.631 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.631 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.631 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.5", 2024-04-22T13:34:40.631 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0.001", 2024-04-22T13:34:40.631 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "0.999", 2024-04-22T13:34:40.631 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.631 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "aggressiveness of optimization", 2024-04-22T13:34:40.632 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-22T13:34:40.632 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.632 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.632 INFO:teuthology.orchestra.run.smithi012.stdout: "end_time": { 2024-04-22T13:34:40.632 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_time", 2024-04-22T13:34:40.632 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.632 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.632 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.632 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2359", 2024-04-22T13:34:40.632 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.632 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.632 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.632 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ending time of day to automatically balance", 2024-04-22T13:34:40.633 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-22T13:34:40.633 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.633 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.633 INFO:teuthology.orchestra.run.smithi012.stdout: "end_weekday": { 2024-04-22T13:34:40.633 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "end_weekday", 2024-04-22T13:34:40.633 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-04-22T13:34:40.633 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.633 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.633 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-04-22T13:34:40.633 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "0", 2024-04-22T13:34:40.633 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "6", 2024-04-22T13:34:40.633 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.633 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-22T13:34:40.633 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-22T13:34:40.634 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.634 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.634 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.634 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.634 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.634 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.634 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.634 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.634 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.634 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.634 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.634 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.634 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.634 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.635 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.635 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.635 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.635 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.635 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.635 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.635 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.635 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.635 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.635 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.635 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.635 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.635 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.636 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.636 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.636 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.636 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.636 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.636 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.636 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.636 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.636 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.636 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.636 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.636 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.636 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.637 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.637 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.637 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.637 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.637 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.637 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.637 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.637 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.637 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.637 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.637 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.637 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.637 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.637 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.637 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.638 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.638 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.638 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.638 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.638 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.638 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.638 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.638 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.638 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.638 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.638 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.638 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.639 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.639 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.639 INFO:teuthology.orchestra.run.smithi012.stdout: "min_score": { 2024-04-22T13:34:40.639 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_score", 2024-04-22T13:34:40.639 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-22T13:34:40.639 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.639 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.639 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-04-22T13:34:40.639 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.639 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.639 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.639 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-22T13:34:40.639 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.640 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.640 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.640 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-04-22T13:34:40.640 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-04-22T13:34:40.640 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.640 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.640 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.640 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "upmap", 2024-04-22T13:34:40.640 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.640 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.640 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.640 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-compat", 2024-04-22T13:34:40.640 INFO:teuthology.orchestra.run.smithi012.stdout: "none", 2024-04-22T13:34:40.641 INFO:teuthology.orchestra.run.smithi012.stdout: "read", 2024-04-22T13:34:40.641 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap", 2024-04-22T13:34:40.641 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap-read" 2024-04-22T13:34:40.641 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.641 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Balancer mode", 2024-04-22T13:34:40.641 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.641 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.641 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.641 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_ids": { 2024-04-22T13:34:40.641 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_ids", 2024-04-22T13:34:40.641 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.641 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.641 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.641 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.642 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.642 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.642 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.642 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-22T13:34:40.642 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.642 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.642 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.642 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.642 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-04-22T13:34:40.642 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-04-22T13:34:40.642 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.642 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.642 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.643 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-04-22T13:34:40.643 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.643 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.643 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.643 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-22T13:34:40.643 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.643 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.643 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.643 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_deviation": { 2024-04-22T13:34:40.643 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_deviation", 2024-04-22T13:34:40.643 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.643 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.643 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.643 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-04-22T13:34:40.643 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-04-22T13:34:40.644 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.644 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.644 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-22T13:34:40.644 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-22T13:34:40.644 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.644 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.644 INFO:teuthology.orchestra.run.smithi012.stdout: "upmap_max_optimizations": { 2024-04-22T13:34:40.644 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "upmap_max_optimizations", 2024-04-22T13:34:40.644 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-04-22T13:34:40.644 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.644 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.644 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-22T13:34:40.644 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.645 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.645 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.645 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-22T13:34:40.645 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.645 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.645 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.645 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.645 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.645 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.645 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm", 2024-04-22T13:34:40.645 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.645 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.645 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.645 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_down_multiplier": { 2024-04-22T13:34:40.646 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_down_multiplier", 2024-04-22T13:34:40.646 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-22T13:34:40.646 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.646 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.646 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-04-22T13:34:40.646 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.646 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.646 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.646 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-22T13:34:40.646 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.646 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.646 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.646 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.647 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_refresh_rate": { 2024-04-22T13:34:40.647 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_refresh_rate", 2024-04-22T13:34:40.647 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.647 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.647 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.647 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "20", 2024-04-22T13:34:40.647 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.647 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.647 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.647 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-22T13:34:40.647 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.647 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.647 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.648 INFO:teuthology.orchestra.run.smithi012.stdout: "agent_starting_port": { 2024-04-22T13:34:40.648 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "agent_starting_port", 2024-04-22T13:34:40.648 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.648 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.648 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.648 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4721", 2024-04-22T13:34:40.648 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.648 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.648 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.648 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-22T13:34:40.648 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.648 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.648 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.648 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_ptrace": { 2024-04-22T13:34:40.649 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_ptrace", 2024-04-22T13:34:40.649 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.649 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.649 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.649 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.649 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.649 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.649 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.649 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-22T13:34:40.649 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-22T13:34:40.649 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.649 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.649 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.649 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_interval": { 2024-04-22T13:34:40.650 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_interval", 2024-04-22T13:34:40.650 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.650 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.650 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.650 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-04-22T13:34:40.650 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.650 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.650 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.650 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-22T13:34:40.650 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.650 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.650 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.650 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.650 INFO:teuthology.orchestra.run.smithi012.stdout: "autotune_memory_target_ratio": { 2024-04-22T13:34:40.651 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "autotune_memory_target_ratio", 2024-04-22T13:34:40.651 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-22T13:34:40.651 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.651 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.651 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0.7", 2024-04-22T13:34:40.651 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.651 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.651 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.651 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-22T13:34:40.651 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.651 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.651 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.651 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.651 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm_log_destination": { 2024-04-22T13:34:40.652 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cephadm_log_destination", 2024-04-22T13:34:40.652 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.652 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.652 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.652 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.652 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.652 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.652 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.652 INFO:teuthology.orchestra.run.smithi012.stdout: "file", 2024-04-22T13:34:40.652 INFO:teuthology.orchestra.run.smithi012.stdout: "file,syslog", 2024-04-22T13:34:40.652 INFO:teuthology.orchestra.run.smithi012.stdout: "syslog" 2024-04-22T13:34:40.652 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.652 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-22T13:34:40.652 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.653 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.653 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.653 INFO:teuthology.orchestra.run.smithi012.stdout: "cgroups_split": { 2024-04-22T13:34:40.653 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cgroups_split", 2024-04-22T13:34:40.653 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.653 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.653 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.653 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.653 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.653 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.653 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.653 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-22T13:34:40.653 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.653 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.654 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.654 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.654 INFO:teuthology.orchestra.run.smithi012.stdout: "config_checks_enabled": { 2024-04-22T13:34:40.654 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_checks_enabled", 2024-04-22T13:34:40.654 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.654 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.654 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.654 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.654 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.654 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.654 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.654 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-22T13:34:40.654 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.655 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.655 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.655 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.655 INFO:teuthology.orchestra.run.smithi012.stdout: "config_dashboard": { 2024-04-22T13:34:40.655 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_dashboard", 2024-04-22T13:34:40.655 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.655 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.655 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.655 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.655 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.655 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.655 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.655 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-22T13:34:40.655 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.656 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.656 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.656 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.656 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_alertmanager": { 2024-04-22T13:34:40.656 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_alertmanager", 2024-04-22T13:34:40.656 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.656 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.656 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.656 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-22T13:34:40.656 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.656 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.656 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.656 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-04-22T13:34:40.656 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.656 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.657 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.657 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.657 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_base": { 2024-04-22T13:34:40.657 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_base", 2024-04-22T13:34:40.657 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.657 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.657 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.657 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-22T13:34:40.657 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.657 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.657 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.657 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Container image name, without the tag", 2024-04-22T13:34:40.657 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.657 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.657 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.658 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_elasticsearch": { 2024-04-22T13:34:40.658 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_elasticsearch", 2024-04-22T13:34:40.658 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.658 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.658 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.658 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-22T13:34:40.658 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.658 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.658 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.658 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "elasticsearch container image", 2024-04-22T13:34:40.658 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.658 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.659 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.659 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_grafana": { 2024-04-22T13:34:40.659 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_grafana", 2024-04-22T13:34:40.659 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.659 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.659 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.659 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-22T13:34:40.659 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.659 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.659 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.659 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-04-22T13:34:40.659 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.659 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.659 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.660 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.660 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_haproxy": { 2024-04-22T13:34:40.660 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_haproxy", 2024-04-22T13:34:40.660 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.660 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.660 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.660 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-22T13:34:40.660 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.660 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.660 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.660 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "HAproxy container image", 2024-04-22T13:34:40.660 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.660 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.660 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.661 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_agent": { 2024-04-22T13:34:40.661 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_agent", 2024-04-22T13:34:40.661 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.661 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.661 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.661 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-22T13:34:40.661 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.661 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.661 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.661 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger agent container image", 2024-04-22T13:34:40.661 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.661 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.661 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.661 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_collector": { 2024-04-22T13:34:40.662 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_collector", 2024-04-22T13:34:40.662 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.662 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.662 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.662 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-22T13:34:40.662 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.662 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.662 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.662 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger collector container image", 2024-04-22T13:34:40.662 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.662 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.662 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.662 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_jaeger_query": { 2024-04-22T13:34:40.663 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_jaeger_query", 2024-04-22T13:34:40.663 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.663 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.663 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.663 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-22T13:34:40.663 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.663 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.663 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.663 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Jaeger query container image", 2024-04-22T13:34:40.663 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.663 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.663 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.663 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_keepalived": { 2024-04-22T13:34:40.663 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_keepalived", 2024-04-22T13:34:40.664 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.664 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.664 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.664 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-22T13:34:40.664 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.664 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.664 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.664 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Keepalived container image", 2024-04-22T13:34:40.664 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.664 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.664 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.664 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.664 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_loki": { 2024-04-22T13:34:40.664 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_loki", 2024-04-22T13:34:40.665 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.665 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.665 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.665 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-22T13:34:40.665 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.665 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.665 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.665 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Loki container image", 2024-04-22T13:34:40.665 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.665 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.665 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.665 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_node_exporter": { 2024-04-22T13:34:40.666 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_node_exporter", 2024-04-22T13:34:40.666 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.666 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.666 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.666 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-22T13:34:40.666 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.666 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.666 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.666 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-04-22T13:34:40.666 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.666 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.666 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.667 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_nvmeof": { 2024-04-22T13:34:40.667 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_nvmeof", 2024-04-22T13:34:40.667 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.667 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.667 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.667 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-22T13:34:40.667 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.667 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.667 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.667 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Nvme-of container image", 2024-04-22T13:34:40.667 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.667 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.667 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.668 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_prometheus": { 2024-04-22T13:34:40.668 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_prometheus", 2024-04-22T13:34:40.668 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.668 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.668 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.668 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-22T13:34:40.668 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.668 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.668 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.668 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Prometheus container image", 2024-04-22T13:34:40.668 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.668 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.668 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.668 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_promtail": { 2024-04-22T13:34:40.669 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_promtail", 2024-04-22T13:34:40.669 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.669 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.669 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.669 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-22T13:34:40.669 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.669 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.669 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.669 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Promtail container image", 2024-04-22T13:34:40.669 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.669 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.669 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.669 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_samba": { 2024-04-22T13:34:40.670 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_samba", 2024-04-22T13:34:40.670 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.670 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.670 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.670 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-22T13:34:40.670 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.670 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.670 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.670 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Samba/SMB container image", 2024-04-22T13:34:40.670 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.670 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.670 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.670 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.671 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image_snmp_gateway": { 2024-04-22T13:34:40.671 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_image_snmp_gateway", 2024-04-22T13:34:40.671 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.671 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.671 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.671 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-22T13:34:40.671 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.671 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.671 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.671 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "SNMP Gateway container image", 2024-04-22T13:34:40.671 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.671 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.671 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.671 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.672 INFO:teuthology.orchestra.run.smithi012.stdout: "container_init": { 2024-04-22T13:34:40.672 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "container_init", 2024-04-22T13:34:40.672 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.672 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.672 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.672 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.672 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.672 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.672 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.672 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Run podman/docker with `--init`", 2024-04-22T13:34:40.672 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.672 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.672 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.672 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.672 INFO:teuthology.orchestra.run.smithi012.stdout: "daemon_cache_timeout": { 2024-04-22T13:34:40.673 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "daemon_cache_timeout", 2024-04-22T13:34:40.673 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.673 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.673 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.673 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-04-22T13:34:40.673 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.673 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.673 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.673 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-22T13:34:40.673 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.673 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.673 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.673 INFO:teuthology.orchestra.run.smithi012.stdout: "default_cephadm_command_timeout": { 2024-04-22T13:34:40.674 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_cephadm_command_timeout", 2024-04-22T13:34:40.674 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.674 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.674 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.674 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "900", 2024-04-22T13:34:40.674 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.674 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.674 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.674 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-22T13:34:40.674 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.674 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.674 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.674 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.674 INFO:teuthology.orchestra.run.smithi012.stdout: "default_registry": { 2024-04-22T13:34:40.675 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "default_registry", 2024-04-22T13:34:40.675 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.675 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.675 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.675 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "docker.io", 2024-04-22T13:34:40.675 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.675 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.675 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.675 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-22T13:34:40.675 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.675 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.675 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.676 INFO:teuthology.orchestra.run.smithi012.stdout: "device_cache_timeout": { 2024-04-22T13:34:40.676 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_cache_timeout", 2024-04-22T13:34:40.676 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.676 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.676 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.676 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1800", 2024-04-22T13:34:40.676 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.676 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.676 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.676 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache device inventory", 2024-04-22T13:34:40.676 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.676 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.676 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.676 INFO:teuthology.orchestra.run.smithi012.stdout: "device_enhanced_scan": { 2024-04-22T13:34:40.677 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_enhanced_scan", 2024-04-22T13:34:40.677 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.677 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.677 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.677 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.677 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.677 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.677 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.677 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-22T13:34:40.677 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.677 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.677 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.677 INFO:teuthology.orchestra.run.smithi012.stdout: "facts_cache_timeout": { 2024-04-22T13:34:40.677 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "facts_cache_timeout", 2024-04-22T13:34:40.677 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.678 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.678 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.678 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-04-22T13:34:40.678 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.678 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.678 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.678 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "seconds to cache host facts data", 2024-04-22T13:34:40.678 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.678 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.678 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.678 INFO:teuthology.orchestra.run.smithi012.stdout: "grafana_dashboards_path": { 2024-04-22T13:34:40.678 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "grafana_dashboards_path", 2024-04-22T13:34:40.678 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.679 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.679 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.679 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-22T13:34:40.679 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.679 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.679 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.679 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-22T13:34:40.679 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.679 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.679 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.679 INFO:teuthology.orchestra.run.smithi012.stdout: "host_check_interval": { 2024-04-22T13:34:40.679 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "host_check_interval", 2024-04-22T13:34:40.679 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.680 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.680 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.680 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-04-22T13:34:40.680 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.680 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.680 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.680 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to perform a host check", 2024-04-22T13:34:40.680 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.680 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.680 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.680 INFO:teuthology.orchestra.run.smithi012.stdout: "hw_monitoring": { 2024-04-22T13:34:40.680 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hw_monitoring", 2024-04-22T13:34:40.680 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.680 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.681 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.681 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.681 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.681 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.681 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.681 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-22T13:34:40.681 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.681 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.681 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.681 INFO:teuthology.orchestra.run.smithi012.stdout: "inventory_list_all": { 2024-04-22T13:34:40.681 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "inventory_list_all", 2024-04-22T13:34:40.681 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.681 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.681 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.681 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.682 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.682 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.682 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.682 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-22T13:34:40.682 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.682 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.682 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.682 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.682 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.682 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.682 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.682 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.682 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.682 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.683 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.683 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.683 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.683 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.683 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.683 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.683 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.683 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.683 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.683 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.683 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.683 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.683 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.683 INFO:teuthology.orchestra.run.smithi012.stdout: "log_refresh_metadata": { 2024-04-22T13:34:40.684 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_refresh_metadata", 2024-04-22T13:34:40.684 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.684 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.684 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.684 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.684 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.684 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.684 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.684 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-22T13:34:40.684 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.684 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.684 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.684 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.685 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.685 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.685 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.685 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.685 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.685 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.685 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.685 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.685 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-22T13:34:40.685 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.685 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.685 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.685 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.685 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.685 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.686 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.686 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.686 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.686 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.686 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.686 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.686 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.686 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.686 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.686 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.686 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.686 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.686 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.686 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.686 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.687 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.687 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.687 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.687 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.687 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.687 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.687 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.687 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.687 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.687 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.687 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.687 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.687 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.687 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.688 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.688 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.688 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-22T13:34:40.688 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-22T13:34:40.688 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.688 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.688 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.688 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.688 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.688 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.688 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.688 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-22T13:34:40.688 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.688 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.689 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.689 INFO:teuthology.orchestra.run.smithi012.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-22T13:34:40.689 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-22T13:34:40.689 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.689 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.689 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.689 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "*", 2024-04-22T13:34:40.689 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.689 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.689 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.689 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-22T13:34:40.689 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.689 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.689 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.690 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.690 INFO:teuthology.orchestra.run.smithi012.stdout: "max_count_per_host": { 2024-04-22T13:34:40.690 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_count_per_host", 2024-04-22T13:34:40.690 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.690 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.690 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.690 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-22T13:34:40.690 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.690 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.690 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.690 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of daemons per service per host", 2024-04-22T13:34:40.690 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.690 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.690 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.691 INFO:teuthology.orchestra.run.smithi012.stdout: "max_osd_draining_count": { 2024-04-22T13:34:40.691 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_osd_draining_count", 2024-04-22T13:34:40.691 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.691 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.691 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.691 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-22T13:34:40.691 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.691 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.691 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.691 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-22T13:34:40.691 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.691 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.691 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.692 INFO:teuthology.orchestra.run.smithi012.stdout: "migration_current": { 2024-04-22T13:34:40.692 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "migration_current", 2024-04-22T13:34:40.692 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.692 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.692 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.692 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.692 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.692 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.692 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.692 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "internal - do not modify", 2024-04-22T13:34:40.692 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.692 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.692 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.692 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.692 INFO:teuthology.orchestra.run.smithi012.stdout: "mode": { 2024-04-22T13:34:40.693 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mode", 2024-04-22T13:34:40.693 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.693 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.693 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.693 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "root", 2024-04-22T13:34:40.693 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.693 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.693 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.693 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm-package", 2024-04-22T13:34:40.693 INFO:teuthology.orchestra.run.smithi012.stdout: "root" 2024-04-22T13:34:40.693 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.693 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "mode for remote execution of cephadm", 2024-04-22T13:34:40.693 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.693 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.693 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.694 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.694 INFO:teuthology.orchestra.run.smithi012.stdout: "oob_default_addr": { 2024-04-22T13:34:40.694 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "oob_default_addr", 2024-04-22T13:34:40.694 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.694 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.694 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.694 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "169.254.1.1", 2024-04-22T13:34:40.694 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.694 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.694 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.694 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-22T13:34:40.694 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.694 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.695 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.695 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.695 INFO:teuthology.orchestra.run.smithi012.stdout: "prometheus_alerts_path": { 2024-04-22T13:34:40.695 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus_alerts_path", 2024-04-22T13:34:40.695 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.695 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.695 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.695 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-22T13:34:40.695 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.695 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.695 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.695 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-22T13:34:40.695 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.695 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.695 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.696 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_insecure": { 2024-04-22T13:34:40.696 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_insecure", 2024-04-22T13:34:40.696 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.696 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.696 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.696 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.696 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.696 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.696 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.696 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-22T13:34:40.696 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.696 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.696 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.696 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.697 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_password": { 2024-04-22T13:34:40.697 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_password", 2024-04-22T13:34:40.697 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.697 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.697 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.697 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.697 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.697 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.697 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.697 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-22T13:34:40.697 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.697 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.697 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.697 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.697 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_url": { 2024-04-22T13:34:40.697 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_url", 2024-04-22T13:34:40.698 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.698 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.698 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.698 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.698 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.698 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.698 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.698 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-22T13:34:40.698 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.698 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.698 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.698 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.698 INFO:teuthology.orchestra.run.smithi012.stdout: "registry_username": { 2024-04-22T13:34:40.698 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "registry_username", 2024-04-22T13:34:40.699 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.699 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.699 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.699 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.699 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.699 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.699 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.699 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-22T13:34:40.699 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.699 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.699 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.699 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.699 INFO:teuthology.orchestra.run.smithi012.stdout: "secure_monitoring_stack": { 2024-04-22T13:34:40.699 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "secure_monitoring_stack", 2024-04-22T13:34:40.700 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.700 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.700 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.700 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.700 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.700 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.700 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.700 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-22T13:34:40.700 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.700 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.700 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.700 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.700 INFO:teuthology.orchestra.run.smithi012.stdout: "service_discovery_port": { 2024-04-22T13:34:40.700 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "service_discovery_port", 2024-04-22T13:34:40.700 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.700 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.701 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.701 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8765", 2024-04-22T13:34:40.701 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.701 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.701 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.701 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "cephadm service discovery port", 2024-04-22T13:34:40.701 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.701 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.701 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.701 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.701 INFO:teuthology.orchestra.run.smithi012.stdout: "ssh_config_file": { 2024-04-22T13:34:40.701 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssh_config_file", 2024-04-22T13:34:40.701 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.701 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.701 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.701 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.702 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.702 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.702 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.702 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-22T13:34:40.702 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.702 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.702 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.702 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.702 INFO:teuthology.orchestra.run.smithi012.stdout: "use_agent": { 2024-04-22T13:34:40.702 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_agent", 2024-04-22T13:34:40.702 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.702 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.702 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.703 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.703 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.703 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.703 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.703 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-22T13:34:40.703 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.703 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.703 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.703 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.703 INFO:teuthology.orchestra.run.smithi012.stdout: "use_repo_digest": { 2024-04-22T13:34:40.703 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "use_repo_digest", 2024-04-22T13:34:40.703 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.703 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.703 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.703 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.704 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.704 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.704 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.704 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-22T13:34:40.704 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.704 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.704 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.704 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.704 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_failed_host_check": { 2024-04-22T13:34:40.704 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_failed_host_check", 2024-04-22T13:34:40.704 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.704 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.704 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.704 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.704 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.705 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.705 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.705 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if the host check fails", 2024-04-22T13:34:40.705 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.705 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.705 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.705 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.705 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_daemons": { 2024-04-22T13:34:40.705 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_daemons", 2024-04-22T13:34:40.705 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.705 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.705 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.705 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.705 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.706 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.706 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.706 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-22T13:34:40.706 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.706 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.706 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.706 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.706 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_on_stray_hosts": { 2024-04-22T13:34:40.706 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_on_stray_hosts", 2024-04-22T13:34:40.706 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.706 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.706 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.706 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.706 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.707 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.707 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.707 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-22T13:34:40.707 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.707 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.707 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.707 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.707 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.707 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.707 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.707 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crash", 2024-04-22T13:34:40.707 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.707 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.707 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.707 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.707 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.708 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.708 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.708 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.708 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.708 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.708 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.708 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.708 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.708 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.708 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.708 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.708 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.708 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.708 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.708 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.709 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.709 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.709 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.709 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.709 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.709 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.709 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.709 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.709 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.709 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.709 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.709 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.709 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.710 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.710 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.710 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.710 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.710 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.710 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.710 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.710 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.710 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.710 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.710 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.710 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.710 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.710 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.711 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.711 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.711 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.711 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.711 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.711 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.711 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.711 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.711 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.711 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.711 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.711 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.711 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.711 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.711 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.712 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.712 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.712 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.712 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.712 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.712 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.712 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.712 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.712 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.712 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.712 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.712 INFO:teuthology.orchestra.run.smithi012.stdout: "retain_interval": { 2024-04-22T13:34:40.712 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retain_interval", 2024-04-22T13:34:40.712 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.713 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.713 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.713 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "31536000", 2024-04-22T13:34:40.713 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.713 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.713 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.713 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-22T13:34:40.713 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.713 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.713 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.713 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.713 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_recent_interval": { 2024-04-22T13:34:40.713 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_recent_interval", 2024-04-22T13:34:40.713 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.714 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.714 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.714 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1209600", 2024-04-22T13:34:40.714 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.714 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.714 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.714 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-22T13:34:40.714 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.714 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.714 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.714 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.714 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.714 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.714 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.715 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dashboard", 2024-04-22T13:34:40.715 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.715 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.715 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.715 INFO:teuthology.orchestra.run.smithi012.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-22T13:34:40.715 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-22T13:34:40.715 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.715 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.715 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.715 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-22T13:34:40.715 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.715 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.715 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.715 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.716 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.716 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.716 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.716 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.716 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_HOST": { 2024-04-22T13:34:40.716 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-22T13:34:40.716 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.716 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.716 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.716 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.716 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.716 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.716 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.716 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.716 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.717 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.717 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.717 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.717 INFO:teuthology.orchestra.run.smithi012.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-22T13:34:40.717 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-22T13:34:40.717 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.717 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.717 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.717 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.717 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.717 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.717 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.717 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.717 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.717 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.718 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.718 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.718 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_ENABLED": { 2024-04-22T13:34:40.718 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_ENABLED", 2024-04-22T13:34:40.718 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.718 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.718 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.718 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.718 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.718 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.718 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.718 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.718 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.719 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.719 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.719 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.719 INFO:teuthology.orchestra.run.smithi012.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-22T13:34:40.719 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-22T13:34:40.719 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.719 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.719 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.719 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.719 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.719 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.719 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.719 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.719 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.719 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.720 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.720 INFO:teuthology.orchestra.run.smithi012.stdout: "ENABLE_BROWSABLE_API": { 2024-04-22T13:34:40.720 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-22T13:34:40.720 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.720 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.720 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.720 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.720 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.720 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.720 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.720 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.720 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.720 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.720 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.720 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.721 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-22T13:34:40.721 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-22T13:34:40.721 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.721 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.721 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.721 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.721 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.721 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.721 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.721 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.721 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.721 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.721 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.721 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.721 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-22T13:34:40.722 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-22T13:34:40.722 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.722 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.722 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.722 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.722 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.722 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.722 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.722 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.722 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.722 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.722 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.722 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.722 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-22T13:34:40.722 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-22T13:34:40.723 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.723 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.723 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.723 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.723 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.723 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.723 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.723 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.723 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.723 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.723 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.723 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.723 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-22T13:34:40.723 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-22T13:34:40.724 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.724 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.724 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.724 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.724 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.724 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.724 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.724 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.724 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.724 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.724 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.724 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.724 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-22T13:34:40.724 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-22T13:34:40.725 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.725 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.725 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.725 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.725 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.725 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.725 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.725 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.725 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.725 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.725 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.725 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.725 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-22T13:34:40.725 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-22T13:34:40.726 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.726 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.726 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.726 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.726 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.726 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.726 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.726 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.726 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.726 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.726 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.726 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.726 INFO:teuthology.orchestra.run.smithi012.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-22T13:34:40.727 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-22T13:34:40.727 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.727 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.727 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.727 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.727 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.727 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.727 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.727 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.727 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.727 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.727 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.727 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.727 INFO:teuthology.orchestra.run.smithi012.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-22T13:34:40.727 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-22T13:34:40.728 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.728 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.728 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.728 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.728 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.728 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.728 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.728 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.728 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.728 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.728 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.728 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.728 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_PASSWORD": { 2024-04-22T13:34:40.728 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-22T13:34:40.728 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.729 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.729 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.729 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-04-22T13:34:40.729 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.729 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.729 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.729 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.729 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.729 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.729 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.729 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.729 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-22T13:34:40.729 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-22T13:34:40.729 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.730 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.730 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.730 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.730 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.730 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.730 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.730 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.730 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.730 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.730 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.730 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.730 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_URL": { 2024-04-22T13:34:40.730 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_URL", 2024-04-22T13:34:40.730 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.731 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.731 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.731 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.731 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.731 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.731 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.731 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.731 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.731 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.731 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.731 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.731 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_API_USERNAME": { 2024-04-22T13:34:40.731 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-22T13:34:40.731 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.732 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.732 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.732 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-04-22T13:34:40.732 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.732 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.732 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.732 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.732 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.732 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.732 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.732 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.732 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-22T13:34:40.732 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-22T13:34:40.732 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.732 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.733 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.733 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.733 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.733 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.733 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.733 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.733 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.733 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.733 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.733 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.733 INFO:teuthology.orchestra.run.smithi012.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-22T13:34:40.733 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-22T13:34:40.733 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.733 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.733 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.734 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.734 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.734 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.734 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.734 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.734 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.734 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.734 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.734 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.734 INFO:teuthology.orchestra.run.smithi012.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-22T13:34:40.734 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-22T13:34:40.734 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.734 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.734 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.734 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.735 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.735 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.735 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.735 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.735 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.735 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.735 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.735 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.735 INFO:teuthology.orchestra.run.smithi012.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-22T13:34:40.735 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-22T13:34:40.735 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.735 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.735 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.735 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.735 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.736 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.736 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.736 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.736 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.736 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.736 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.736 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.736 INFO:teuthology.orchestra.run.smithi012.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-22T13:34:40.736 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-22T13:34:40.736 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.736 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.736 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.736 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "[]", 2024-04-22T13:34:40.736 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.736 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.737 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.737 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.737 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.737 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.737 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.737 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.737 INFO:teuthology.orchestra.run.smithi012.stdout: "MULTICLUSTER_CONFIG": { 2024-04-22T13:34:40.737 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-22T13:34:40.737 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.737 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.737 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.737 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "{}", 2024-04-22T13:34:40.738 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.738 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.738 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.738 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.738 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.738 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.738 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.738 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.738 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_HOST": { 2024-04-22T13:34:40.738 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-22T13:34:40.738 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.738 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.738 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.738 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.739 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.739 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.739 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.739 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.739 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.739 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.739 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.739 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.739 INFO:teuthology.orchestra.run.smithi012.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-22T13:34:40.739 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-22T13:34:40.739 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.739 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.739 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.739 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.740 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.740 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.740 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.740 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.740 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.740 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.740 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.740 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.740 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-22T13:34:40.740 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-22T13:34:40.740 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.740 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.740 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.740 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.740 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.741 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.741 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.741 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.741 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.741 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.741 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.741 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.741 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-22T13:34:40.741 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-22T13:34:40.741 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.741 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.741 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.741 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.741 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.742 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.742 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.742 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.742 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.742 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.742 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.742 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.742 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-22T13:34:40.742 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-22T13:34:40.742 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.742 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.742 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.742 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.743 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.743 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.743 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.743 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.743 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.743 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.743 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.743 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.743 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-22T13:34:40.743 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-22T13:34:40.743 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.743 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.743 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.743 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.743 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.744 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.744 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.744 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.744 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.744 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.744 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.744 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.744 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-22T13:34:40.744 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-22T13:34:40.744 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.744 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.744 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.744 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.744 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.744 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.745 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.745 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.745 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.745 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.745 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.745 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.745 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-22T13:34:40.745 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-22T13:34:40.745 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.745 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.745 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.745 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.745 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.745 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.746 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.746 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.746 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.746 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.746 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.746 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.746 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-22T13:34:40.746 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-22T13:34:40.746 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.746 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.746 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.746 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.746 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.746 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.747 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.747 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.747 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.747 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.747 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.747 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.747 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_ENABLED": { 2024-04-22T13:34:40.747 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-22T13:34:40.747 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.747 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.747 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.747 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.747 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.747 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.747 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.748 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.748 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.748 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.748 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.748 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.748 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-22T13:34:40.748 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-22T13:34:40.748 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.748 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.748 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.748 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-22T13:34:40.748 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.748 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.748 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.749 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.749 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.749 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.749 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.749 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.749 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-22T13:34:40.749 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-22T13:34:40.749 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.749 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.749 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.749 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-22T13:34:40.749 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.749 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.749 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.749 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.749 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.750 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.750 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.750 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.750 INFO:teuthology.orchestra.run.smithi012.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-22T13:34:40.750 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-22T13:34:40.750 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.750 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.750 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.750 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8", 2024-04-22T13:34:40.750 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.750 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.750 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.750 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.750 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.751 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.751 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.751 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.751 INFO:teuthology.orchestra.run.smithi012.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-22T13:34:40.751 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-22T13:34:40.751 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.751 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.751 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.751 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "45", 2024-04-22T13:34:40.751 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.751 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.751 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.751 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.751 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.751 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.752 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.752 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.752 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ACCESS_KEY": { 2024-04-22T13:34:40.752 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-22T13:34:40.752 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.752 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.752 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.752 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.752 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.752 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.752 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.752 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.752 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.752 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.753 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.753 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-22T13:34:40.753 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-22T13:34:40.753 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.753 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.753 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.753 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "admin", 2024-04-22T13:34:40.753 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.753 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.753 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.753 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.753 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.753 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.753 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.753 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.754 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SECRET_KEY": { 2024-04-22T13:34:40.754 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-22T13:34:40.754 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.754 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.754 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.754 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.754 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.754 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.754 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.754 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.754 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.754 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.754 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.754 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.755 INFO:teuthology.orchestra.run.smithi012.stdout: "RGW_API_SSL_VERIFY": { 2024-04-22T13:34:40.755 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-22T13:34:40.755 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.755 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.755 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.755 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.755 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.755 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.755 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.755 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.755 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.755 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.755 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.755 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.756 INFO:teuthology.orchestra.run.smithi012.stdout: "UNSAFE_TLS_v1_2": { 2024-04-22T13:34:40.756 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-22T13:34:40.756 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.756 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.756 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.756 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.756 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.756 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.756 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.756 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.756 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.756 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.756 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.756 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.756 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-22T13:34:40.757 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-22T13:34:40.757 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.757 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.757 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.757 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-04-22T13:34:40.757 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.757 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.757 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.757 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.757 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.757 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.757 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.757 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.757 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-22T13:34:40.757 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-22T13:34:40.758 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.758 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.758 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.758 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-22T13:34:40.758 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.758 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.758 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.758 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.758 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.758 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.758 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.758 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.758 INFO:teuthology.orchestra.run.smithi012.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-22T13:34:40.758 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-22T13:34:40.759 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.759 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.759 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.759 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-04-22T13:34:40.759 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.759 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.759 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.759 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.759 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.759 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.759 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.759 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.759 INFO:teuthology.orchestra.run.smithi012.stdout: "cross_origin_url": { 2024-04-22T13:34:40.759 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cross_origin_url", 2024-04-22T13:34:40.759 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.760 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.760 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.760 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.760 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.760 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.760 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.760 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.760 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.760 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.760 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.760 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.760 INFO:teuthology.orchestra.run.smithi012.stdout: "crt_file": { 2024-04-22T13:34:40.760 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "crt_file", 2024-04-22T13:34:40.760 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.760 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.761 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.761 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.761 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.761 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.761 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.761 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.761 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.761 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.761 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.761 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.761 INFO:teuthology.orchestra.run.smithi012.stdout: "debug": { 2024-04-22T13:34:40.761 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "debug", 2024-04-22T13:34:40.761 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.761 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.761 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.762 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.762 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.762 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.762 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.762 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Enable/disable debug options", 2024-04-22T13:34:40.762 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.762 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.762 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.762 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.762 INFO:teuthology.orchestra.run.smithi012.stdout: "jwt_token_ttl": { 2024-04-22T13:34:40.762 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "jwt_token_ttl", 2024-04-22T13:34:40.762 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.762 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.762 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.763 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "28800", 2024-04-22T13:34:40.763 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.763 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.763 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.763 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.763 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.763 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.763 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.763 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.763 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-04-22T13:34:40.763 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-04-22T13:34:40.763 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.763 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.763 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.764 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.764 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.764 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.764 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.764 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.764 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.764 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.764 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.764 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.764 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.764 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.764 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.764 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.764 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.764 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.764 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.765 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.765 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.765 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.765 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.765 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.765 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.765 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.765 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.765 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.765 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.765 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.765 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.765 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.765 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.766 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.766 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.766 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.766 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.766 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.766 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.766 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.766 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.766 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.766 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.766 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.766 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.766 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.767 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.767 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.767 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.767 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.767 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.767 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.767 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.767 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.767 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.767 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.767 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.767 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.767 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.767 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.768 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.768 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.768 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.768 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.768 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.768 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.768 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.768 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.768 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.768 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.768 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.768 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.768 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.768 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.768 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.769 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.769 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.769 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.769 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.769 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.769 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.769 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.769 INFO:teuthology.orchestra.run.smithi012.stdout: "motd": { 2024-04-22T13:34:40.769 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "motd", 2024-04-22T13:34:40.769 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.769 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.769 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.769 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.769 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.769 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.770 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.770 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "The message of the day", 2024-04-22T13:34:40.770 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.770 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.770 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.770 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.770 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect_resolve_ip_addr": { 2024-04-22T13:34:40.770 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "redirect_resolve_ip_addr", 2024-04-22T13:34:40.770 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.770 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.770 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.770 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.770 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.770 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.771 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.771 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.771 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.771 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.771 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.771 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.771 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-04-22T13:34:40.771 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-04-22T13:34:40.771 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.771 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.771 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.771 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-04-22T13:34:40.771 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.772 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.772 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.772 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.772 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.772 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.772 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.772 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.772 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-04-22T13:34:40.772 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-04-22T13:34:40.772 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.772 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.772 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.772 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8080", 2024-04-22T13:34:40.772 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.772 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.773 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.773 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.773 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.773 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.773 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.773 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.773 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-04-22T13:34:40.773 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-04-22T13:34:40.773 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.773 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.773 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.773 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.773 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.773 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.773 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.774 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.774 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.774 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.774 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.774 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.774 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl_server_port": { 2024-04-22T13:34:40.774 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl_server_port", 2024-04-22T13:34:40.774 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.774 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.774 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.774 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8443", 2024-04-22T13:34:40.774 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.774 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.774 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.775 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.775 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.775 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.775 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.775 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.775 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-04-22T13:34:40.775 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-04-22T13:34:40.775 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.775 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.775 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.775 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "redirect", 2024-04-22T13:34:40.775 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.775 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.775 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.776 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.776 INFO:teuthology.orchestra.run.smithi012.stdout: "redirect" 2024-04-22T13:34:40.776 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.776 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.776 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.776 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.776 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.776 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.776 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-04-22T13:34:40.776 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-04-22T13:34:40.776 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.776 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.776 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.776 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-04-22T13:34:40.776 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-04-22T13:34:40.777 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-04-22T13:34:40.777 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.777 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.777 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.777 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.777 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.777 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.777 INFO:teuthology.orchestra.run.smithi012.stdout: "url_prefix": { 2024-04-22T13:34:40.777 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url_prefix", 2024-04-22T13:34:40.777 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.777 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.777 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.777 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.777 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.778 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.778 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.778 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.778 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.778 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.778 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.778 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.778 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.778 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.778 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.778 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "devicehealth", 2024-04-22T13:34:40.778 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.778 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.778 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.778 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_monitoring": { 2024-04-22T13:34:40.779 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_monitoring", 2024-04-22T13:34:40.779 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.779 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.779 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.779 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.779 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.779 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.779 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.779 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "monitor device health metrics", 2024-04-22T13:34:40.779 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.779 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.779 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.779 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.779 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.780 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.780 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.780 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.780 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.780 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.780 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.780 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.780 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.780 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.780 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.780 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.780 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.780 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.780 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.780 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.781 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.781 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.781 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.781 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.781 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.781 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.781 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.781 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.781 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.781 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.781 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.781 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.781 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.781 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.781 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.782 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.782 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.782 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.782 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.782 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.782 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.782 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.782 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.782 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.782 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.782 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.782 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.782 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.782 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.783 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.783 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.783 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.783 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.783 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.783 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.783 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.783 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.783 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.783 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.783 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.783 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.783 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.783 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.783 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.784 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.784 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.784 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.784 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.784 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.784 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.784 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.784 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.784 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.784 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.784 INFO:teuthology.orchestra.run.smithi012.stdout: "mark_out_threshold": { 2024-04-22T13:34:40.784 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mark_out_threshold", 2024-04-22T13:34:40.784 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.784 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.785 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.785 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "2419200", 2024-04-22T13:34:40.785 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.785 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.785 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.785 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-22T13:34:40.785 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.785 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.785 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.785 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.785 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": { 2024-04-22T13:34:40.785 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pool_name", 2024-04-22T13:34:40.785 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.785 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.785 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.786 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "device_health_metrics", 2024-04-22T13:34:40.786 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.786 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.786 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.786 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-22T13:34:40.786 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.786 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.786 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.786 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.786 INFO:teuthology.orchestra.run.smithi012.stdout: "retention_period": { 2024-04-22T13:34:40.786 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "retention_period", 2024-04-22T13:34:40.786 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.786 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.786 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.786 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15552000", 2024-04-22T13:34:40.787 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.787 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.787 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.787 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long to retain device health metrics", 2024-04-22T13:34:40.787 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.787 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.787 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.787 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.787 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_frequency": { 2024-04-22T13:34:40.787 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_frequency", 2024-04-22T13:34:40.787 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.788 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.788 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.788 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-04-22T13:34:40.788 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.788 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.788 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.788 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-22T13:34:40.788 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.788 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.788 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.788 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.788 INFO:teuthology.orchestra.run.smithi012.stdout: "self_heal": { 2024-04-22T13:34:40.788 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "self_heal", 2024-04-22T13:34:40.788 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.789 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.789 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.789 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.789 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.789 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.789 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.789 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-22T13:34:40.789 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.789 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.789 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.789 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.789 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-04-22T13:34:40.789 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-04-22T13:34:40.789 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.790 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.790 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.790 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-04-22T13:34:40.790 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.790 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.790 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.790 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how frequently to wake up and check device health", 2024-04-22T13:34:40.790 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.790 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.790 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.790 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.790 INFO:teuthology.orchestra.run.smithi012.stdout: "warn_threshold": { 2024-04-22T13:34:40.790 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "warn_threshold", 2024-04-22T13:34:40.790 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.791 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.791 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.791 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7257600", 2024-04-22T13:34:40.791 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.791 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.791 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.791 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-22T13:34:40.791 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.791 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.791 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.791 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.791 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.791 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.791 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.792 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "diskprediction_local", 2024-04-22T13:34:40.792 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.792 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.792 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.792 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.792 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.792 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.792 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.792 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.792 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.792 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.792 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.792 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.792 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.792 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.793 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.793 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.793 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.793 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.793 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.793 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.793 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.793 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.793 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.793 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.793 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.793 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.793 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.793 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.794 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.794 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.794 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.794 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.794 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.794 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.794 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.794 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.794 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.794 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.794 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.794 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.794 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.794 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.794 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.795 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.795 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.795 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.795 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.795 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.795 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.795 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.795 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.795 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.795 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.795 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.795 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.795 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.795 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.796 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.796 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.796 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.796 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.796 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.796 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.796 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.796 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.796 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.796 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.796 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.796 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.796 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.796 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.796 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.797 INFO:teuthology.orchestra.run.smithi012.stdout: "predict_interval": { 2024-04-22T13:34:40.797 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predict_interval", 2024-04-22T13:34:40.797 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.797 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.797 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.797 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "86400", 2024-04-22T13:34:40.797 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.797 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.797 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.797 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.797 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.797 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.797 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.797 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.798 INFO:teuthology.orchestra.run.smithi012.stdout: "predictor_model": { 2024-04-22T13:34:40.798 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "predictor_model", 2024-04-22T13:34:40.798 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.798 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.798 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.798 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "prophetstor", 2024-04-22T13:34:40.798 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.798 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.798 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.798 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.798 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.798 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.798 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.798 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.799 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-04-22T13:34:40.799 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-04-22T13:34:40.799 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.799 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.799 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.799 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "600", 2024-04-22T13:34:40.799 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.799 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.799 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.799 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.799 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.799 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.799 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.799 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.800 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.800 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.800 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.800 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "influx", 2024-04-22T13:34:40.800 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": false, 2024-04-22T13:34:40.800 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "influxdb python module not found", 2024-04-22T13:34:40.800 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.800 INFO:teuthology.orchestra.run.smithi012.stdout: "batch_size": { 2024-04-22T13:34:40.800 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "batch_size", 2024-04-22T13:34:40.800 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.800 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.800 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.800 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5000", 2024-04-22T13:34:40.800 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.801 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.801 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.801 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-22T13:34:40.801 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.801 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.801 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.801 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.801 INFO:teuthology.orchestra.run.smithi012.stdout: "database": { 2024-04-22T13:34:40.801 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "database", 2024-04-22T13:34:40.801 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.801 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.801 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.801 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "ceph", 2024-04-22T13:34:40.802 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.802 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.802 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.802 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-22T13:34:40.802 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.802 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.802 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.802 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.802 INFO:teuthology.orchestra.run.smithi012.stdout: "hostname": { 2024-04-22T13:34:40.802 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "hostname", 2024-04-22T13:34:40.802 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.802 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.802 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.802 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.803 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.803 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.803 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.803 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server hostname", 2024-04-22T13:34:40.803 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.803 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.803 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.803 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.803 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-04-22T13:34:40.803 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-04-22T13:34:40.803 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.803 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.803 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.804 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "30", 2024-04-22T13:34:40.804 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "5", 2024-04-22T13:34:40.804 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.804 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.804 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-22T13:34:40.804 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.804 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.804 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.804 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.804 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.804 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.804 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.804 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.804 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.804 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.805 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.805 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.805 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.805 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.805 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.805 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.805 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.805 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.805 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.805 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.805 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.805 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.805 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.805 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.805 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.806 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.806 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.806 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.806 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.806 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.806 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.806 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.806 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.806 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.806 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.806 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.806 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.806 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.807 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.807 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.807 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.807 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.807 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.807 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.807 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.807 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.807 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.807 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.807 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.807 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.807 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.807 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.808 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.808 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.808 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.808 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.808 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.808 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.808 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.808 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.808 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.808 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.808 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.808 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.808 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.808 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.808 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.809 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.809 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.809 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.809 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.809 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.809 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.809 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.809 INFO:teuthology.orchestra.run.smithi012.stdout: "password": { 2024-04-22T13:34:40.809 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "password", 2024-04-22T13:34:40.809 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.809 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.809 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.809 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.809 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.810 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.810 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.810 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "password of InfluxDB server user", 2024-04-22T13:34:40.810 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.810 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.810 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.810 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.810 INFO:teuthology.orchestra.run.smithi012.stdout: "port": { 2024-04-22T13:34:40.810 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "port", 2024-04-22T13:34:40.810 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.810 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.810 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.810 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "8086", 2024-04-22T13:34:40.810 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.811 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.811 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.811 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "InfluxDB server port", 2024-04-22T13:34:40.811 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.811 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.811 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.811 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.811 INFO:teuthology.orchestra.run.smithi012.stdout: "ssl": { 2024-04-22T13:34:40.811 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ssl", 2024-04-22T13:34:40.811 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.811 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.811 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.811 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "false", 2024-04-22T13:34:40.812 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.812 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.812 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.812 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-22T13:34:40.812 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.812 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.812 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.812 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.813 INFO:teuthology.orchestra.run.smithi012.stdout: "threads": { 2024-04-22T13:34:40.813 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threads", 2024-04-22T13:34:40.813 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.813 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.813 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.813 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-04-22T13:34:40.813 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-04-22T13:34:40.813 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "32", 2024-04-22T13:34:40.813 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.813 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-22T13:34:40.813 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.813 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.813 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.813 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.813 INFO:teuthology.orchestra.run.smithi012.stdout: "username": { 2024-04-22T13:34:40.814 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "username", 2024-04-22T13:34:40.814 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.814 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.814 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.814 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.814 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.814 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.814 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.814 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "username of InfluxDB server user", 2024-04-22T13:34:40.814 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.814 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.814 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.814 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.814 INFO:teuthology.orchestra.run.smithi012.stdout: "verify_ssl": { 2024-04-22T13:34:40.815 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "verify_ssl", 2024-04-22T13:34:40.815 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.815 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.815 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.815 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "true", 2024-04-22T13:34:40.815 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.815 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.815 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.815 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-22T13:34:40.815 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.815 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.815 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.815 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.815 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.816 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.816 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.816 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "insights", 2024-04-22T13:34:40.816 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.816 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.816 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.816 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.816 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.816 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.816 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.816 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.816 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.816 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.816 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.816 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.817 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.817 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.817 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.817 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.817 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.817 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.817 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.817 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.817 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.817 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.817 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.817 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.817 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.817 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.817 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.818 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.818 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.818 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.818 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.818 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.818 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.818 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.818 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.818 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.818 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.818 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.818 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.818 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.818 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.819 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.819 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.819 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.819 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.819 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.819 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.819 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.819 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.819 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.819 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.819 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.819 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.819 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.819 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.820 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.820 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.820 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.820 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.820 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.820 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.820 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.820 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.820 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.820 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.820 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.820 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.820 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.820 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.821 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.821 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.821 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.821 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.821 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.821 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.821 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.821 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "iostat", 2024-04-22T13:34:40.821 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.821 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.821 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.821 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.821 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.821 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.821 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.822 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.822 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.822 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.822 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.822 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.822 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.822 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.822 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.822 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.822 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.822 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.822 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.822 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.822 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.823 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.823 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.823 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.823 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.823 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.823 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.823 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.823 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.823 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.823 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.823 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.823 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.823 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.823 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.824 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.824 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.824 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.824 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.824 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.824 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.824 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.824 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.824 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.824 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.824 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.824 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.824 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.824 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.825 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.825 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.825 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.825 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.825 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.825 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.825 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.825 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.825 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.825 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.825 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.825 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.825 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.826 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.826 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.826 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.826 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.826 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.826 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.826 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.826 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.826 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.826 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.826 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.826 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.826 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.826 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.827 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "k8sevents", 2024-04-22T13:34:40.827 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.827 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.827 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.827 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_event_retention_days": { 2024-04-22T13:34:40.827 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "ceph_event_retention_days", 2024-04-22T13:34:40.827 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.827 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.827 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.827 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "7", 2024-04-22T13:34:40.827 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.827 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.827 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.828 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-22T13:34:40.828 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.828 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.828 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.828 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.828 INFO:teuthology.orchestra.run.smithi012.stdout: "config_check_secs": { 2024-04-22T13:34:40.828 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "config_check_secs", 2024-04-22T13:34:40.828 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.828 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.828 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.828 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-22T13:34:40.828 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "10", 2024-04-22T13:34:40.828 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.828 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.829 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-22T13:34:40.829 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.829 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.829 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.829 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.829 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.829 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.829 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.829 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.829 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.829 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.829 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.829 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.829 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.830 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.830 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.830 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.830 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.830 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.830 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.830 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.830 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.830 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.830 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.830 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.830 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.830 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.830 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.830 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.831 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.831 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.831 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.831 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.831 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.831 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.831 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.831 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.831 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.831 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.831 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.831 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.831 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.832 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.832 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.832 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.832 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.832 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.832 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.832 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.832 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.832 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.832 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.832 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.832 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.832 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.832 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.832 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.833 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.833 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.833 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.833 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.833 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.833 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.833 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.833 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.833 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.833 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.833 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.833 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.833 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.833 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.833 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.834 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.834 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.834 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.834 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.834 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.834 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.834 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "localpool", 2024-04-22T13:34:40.834 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.834 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.834 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.834 INFO:teuthology.orchestra.run.smithi012.stdout: "failure_domain": { 2024-04-22T13:34:40.834 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "failure_domain", 2024-04-22T13:34:40.834 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.835 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.835 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.835 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "host", 2024-04-22T13:34:40.835 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.835 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.835 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.835 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "failure domain for any created local pool", 2024-04-22T13:34:40.835 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-22T13:34:40.835 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.835 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.835 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.835 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.835 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.835 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.835 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.836 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.836 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.836 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.836 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.836 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.836 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.836 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.836 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.836 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.836 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.836 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.836 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.836 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.836 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.837 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.837 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.837 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.837 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.837 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.837 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.837 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.837 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.837 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.838 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.838 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.838 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.838 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.838 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.838 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.838 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.838 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.838 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.838 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.838 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.838 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.838 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.838 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.839 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.839 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.839 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.839 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.839 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.839 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.839 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.839 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.839 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.839 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.839 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.839 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.839 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.839 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.839 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.840 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.840 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.840 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.840 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.840 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.840 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.840 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.840 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.840 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.840 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.840 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.840 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.840 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.840 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.840 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": { 2024-04-22T13:34:40.841 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "min_size", 2024-04-22T13:34:40.841 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.841 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.841 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.841 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.841 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.841 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.841 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.841 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default min_size for any created local pool", 2024-04-22T13:34:40.841 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-22T13:34:40.841 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.841 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.841 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.841 INFO:teuthology.orchestra.run.smithi012.stdout: "num_rep": { 2024-04-22T13:34:40.842 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "num_rep", 2024-04-22T13:34:40.842 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.842 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.842 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.842 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3", 2024-04-22T13:34:40.842 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.842 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.842 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.842 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default replica count for any created local pool", 2024-04-22T13:34:40.842 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.842 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.842 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.842 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.842 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": { 2024-04-22T13:34:40.843 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_num", 2024-04-22T13:34:40.843 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.843 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.843 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.843 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "128", 2024-04-22T13:34:40.843 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.843 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.843 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.843 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "default pg_num for any created local pool", 2024-04-22T13:34:40.843 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.843 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.843 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.843 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.843 INFO:teuthology.orchestra.run.smithi012.stdout: "prefix": { 2024-04-22T13:34:40.844 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prefix", 2024-04-22T13:34:40.844 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.844 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.844 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.844 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.844 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.844 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.844 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.844 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "name prefix for any created local pool", 2024-04-22T13:34:40.844 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.844 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.844 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.844 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.844 INFO:teuthology.orchestra.run.smithi012.stdout: "subtree": { 2024-04-22T13:34:40.844 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "subtree", 2024-04-22T13:34:40.845 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.845 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.845 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.845 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "rack", 2024-04-22T13:34:40.845 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.845 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.845 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.845 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-22T13:34:40.845 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-22T13:34:40.845 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.845 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.845 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.845 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.845 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.845 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.846 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mds_autoscaler", 2024-04-22T13:34:40.846 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.846 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.846 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.846 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.846 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.846 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.846 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.846 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.846 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.846 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.846 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.846 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.846 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.847 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.847 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.847 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.847 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.847 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.847 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.847 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.847 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.847 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.847 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.847 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.847 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.847 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.847 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.847 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.848 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.848 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.848 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.848 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.848 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.848 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.848 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.848 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.848 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.848 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.848 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.848 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.848 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.848 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.848 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.849 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.849 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.849 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.849 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.849 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.849 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.849 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.849 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.849 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.849 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.849 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.849 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.849 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.849 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.849 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.850 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.850 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.850 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.850 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.850 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.850 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.850 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.850 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.850 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.850 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.850 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.850 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.850 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.850 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.851 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.851 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.851 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.851 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.851 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirroring", 2024-04-22T13:34:40.851 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.851 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.851 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.851 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.851 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.851 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.851 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.851 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.851 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.851 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.852 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.852 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.852 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.852 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.852 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.852 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.852 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.852 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.852 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.852 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.852 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.852 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.852 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.852 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.852 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.853 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.853 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.853 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.853 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.853 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.853 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.853 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.853 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.853 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.853 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.853 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.853 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.853 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.853 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.853 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.853 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.854 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.854 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.854 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.854 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.854 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.854 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.854 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.854 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.854 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.854 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.854 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.854 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.854 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.854 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.855 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.855 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.855 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.855 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.855 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.855 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.855 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.855 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.855 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.855 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.855 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.855 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.855 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.855 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.856 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.856 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.856 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.856 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.856 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.856 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.856 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.856 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "nfs", 2024-04-22T13:34:40.856 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.856 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.856 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.856 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.856 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.856 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.856 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.857 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.857 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.857 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.857 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.857 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.857 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.857 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.857 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.857 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.857 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.857 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.857 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.857 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.857 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.857 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.857 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.858 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.858 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.858 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.858 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.858 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.858 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.858 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.858 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.858 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.858 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.858 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.858 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.858 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.859 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.859 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.859 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.859 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.859 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.859 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.859 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.859 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.859 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.859 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.859 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.859 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.859 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.859 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.860 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.860 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.860 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.860 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.860 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.860 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.860 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.860 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.860 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.860 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.860 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.860 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.860 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.860 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.861 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.861 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.861 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.861 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.861 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.861 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.861 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.861 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.861 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.861 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.861 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.861 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.861 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-04-22T13:34:40.861 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.861 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.862 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.862 INFO:teuthology.orchestra.run.smithi012.stdout: "fail_fs": { 2024-04-22T13:34:40.862 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "fail_fs", 2024-04-22T13:34:40.862 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.862 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.862 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.862 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.862 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.862 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.862 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.862 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-22T13:34:40.862 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.862 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.862 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.862 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.863 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.863 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.863 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.863 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.863 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.863 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.863 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.863 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.863 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.863 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.863 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.863 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.863 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.863 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.863 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.864 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.864 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.864 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.864 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.864 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.864 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.864 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.864 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.864 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.864 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.864 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.864 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.864 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.864 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.864 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.865 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.865 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.865 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.865 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.865 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.865 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.865 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.865 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.865 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.865 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.865 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.865 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.865 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.865 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.865 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.866 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.866 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.866 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.866 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.866 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.866 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.866 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.866 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.866 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.866 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.866 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.866 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.866 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.867 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.867 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.867 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.867 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.867 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.867 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.867 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.867 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.867 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.867 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.867 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.867 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.867 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator": { 2024-04-22T13:34:40.867 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "orchestrator", 2024-04-22T13:34:40.867 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.867 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.868 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.868 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.868 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.868 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.868 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.868 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm", 2024-04-22T13:34:40.868 INFO:teuthology.orchestra.run.smithi012.stdout: "rook", 2024-04-22T13:34:40.868 INFO:teuthology.orchestra.run.smithi012.stdout: "test_orchestrator" 2024-04-22T13:34:40.868 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.868 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Orchestrator backend", 2024-04-22T13:34:40.868 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.868 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.868 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.868 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.868 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.869 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.869 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.869 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_perf_query", 2024-04-22T13:34:40.869 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.869 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.869 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.869 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.869 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.869 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.869 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.869 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.869 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.869 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.869 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.870 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.870 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.870 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.870 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.870 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.870 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.870 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.870 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.870 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.870 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.870 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.870 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.870 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.870 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.871 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.871 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.871 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.871 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.871 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.871 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.871 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.871 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.871 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.871 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.871 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.871 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.871 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.871 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.871 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.872 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.872 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.872 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.872 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.872 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.872 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.872 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.872 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.872 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.872 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.872 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.872 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.872 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.872 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.873 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.873 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.873 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.873 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.873 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.873 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.873 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.873 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.873 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.873 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.873 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.873 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.873 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.873 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.874 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.874 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.874 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.874 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.874 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.874 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.874 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.874 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.874 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "osd_support", 2024-04-22T13:34:40.874 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.874 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.874 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.874 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.875 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.875 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.875 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.875 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.875 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.875 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.875 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.875 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.875 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.875 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.875 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.875 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.875 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.876 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.876 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.876 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.876 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.876 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.876 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.876 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.876 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.876 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.876 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.876 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.876 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.876 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.876 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.876 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.877 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.877 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.877 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.877 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.877 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.877 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.877 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.877 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.877 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.877 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.877 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.877 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.877 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.877 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.877 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.877 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.878 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.878 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.878 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.878 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.878 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.878 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.878 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.878 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.878 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.878 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.878 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.878 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.878 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.879 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.879 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.879 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.879 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.879 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.879 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.879 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.879 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.879 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.879 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.879 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.879 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.879 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.879 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.879 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.880 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "pg_autoscaler", 2024-04-22T13:34:40.880 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.880 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.880 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.880 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.880 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.880 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.880 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.880 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.880 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.880 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.880 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.880 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.880 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.880 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.881 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.881 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.881 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.881 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.881 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.881 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.881 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.881 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.881 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.881 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.881 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.881 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.881 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.881 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.881 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.882 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.882 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.882 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.882 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.882 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.882 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.882 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.882 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.882 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.882 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.882 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.882 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.882 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.882 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.883 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.883 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.883 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.883 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.883 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.883 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.883 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.883 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.883 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.883 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.883 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.883 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.883 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.883 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.883 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.883 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.884 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.884 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.884 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.884 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.884 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.884 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.884 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.884 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.884 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.884 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.884 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.884 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.884 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.884 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.884 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-04-22T13:34:40.885 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-04-22T13:34:40.885 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.885 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.885 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.885 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-04-22T13:34:40.885 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.885 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.885 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.885 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.885 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.885 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.885 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.885 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.885 INFO:teuthology.orchestra.run.smithi012.stdout: "threshold": { 2024-04-22T13:34:40.886 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "threshold", 2024-04-22T13:34:40.886 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-22T13:34:40.886 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.886 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.886 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "3.0", 2024-04-22T13:34:40.886 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1.0", 2024-04-22T13:34:40.886 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.886 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.886 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "scaling threshold", 2024-04-22T13:34:40.886 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-22T13:34:40.886 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.886 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.886 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.886 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.887 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.887 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.887 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "progress", 2024-04-22T13:34:40.887 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.887 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.887 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.887 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_pg_recovery_event": { 2024-04-22T13:34:40.887 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_pg_recovery_event", 2024-04-22T13:34:40.887 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.887 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.887 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.887 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.887 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.887 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.887 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.888 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-22T13:34:40.888 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.888 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.888 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.888 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.888 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-04-22T13:34:40.888 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-04-22T13:34:40.888 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.888 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.888 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.888 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.888 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.888 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.888 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.888 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.888 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.889 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.889 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.889 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.889 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.889 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.889 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.889 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.889 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.889 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.889 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.889 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.889 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.889 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.889 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.889 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.890 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.890 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.890 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.890 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.890 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.890 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.890 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.890 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.890 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.890 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.890 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.890 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.890 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.890 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.891 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.891 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.891 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.891 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.891 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.891 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.891 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.891 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.891 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.891 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.891 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.891 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.891 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.891 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.892 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.892 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.892 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.892 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.892 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.892 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.892 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.892 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.892 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.892 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.892 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.892 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.892 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.892 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.892 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.893 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.893 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.893 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.893 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.893 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.893 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.893 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.893 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.893 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.893 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.893 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.893 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.893 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.893 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.894 INFO:teuthology.orchestra.run.smithi012.stdout: "max_completed_events": { 2024-04-22T13:34:40.894 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_completed_events", 2024-04-22T13:34:40.894 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.894 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.894 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.894 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "50", 2024-04-22T13:34:40.894 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.894 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.894 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.894 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "number of past completed events to remember", 2024-04-22T13:34:40.894 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.894 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.894 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.895 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.895 INFO:teuthology.orchestra.run.smithi012.stdout: "sleep_interval": { 2024-04-22T13:34:40.895 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "sleep_interval", 2024-04-22T13:34:40.895 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.895 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.895 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.895 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "5", 2024-04-22T13:34:40.895 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.895 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.895 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.895 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "how long the module is going to sleep", 2024-04-22T13:34:40.895 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.895 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.895 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.896 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.896 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.896 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.896 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.896 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "prometheus", 2024-04-22T13:34:40.896 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.896 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.896 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.896 INFO:teuthology.orchestra.run.smithi012.stdout: "cache": { 2024-04-22T13:34:40.896 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "cache", 2024-04-22T13:34:40.896 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.896 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.896 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.896 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.897 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.897 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.897 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.897 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.897 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.897 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.897 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.897 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.897 INFO:teuthology.orchestra.run.smithi012.stdout: "exclude_perf_counters": { 2024-04-22T13:34:40.897 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "exclude_perf_counters", 2024-04-22T13:34:40.897 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.897 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.897 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.897 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.897 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.898 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.898 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.898 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-22T13:34:40.898 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-22T13:34:40.898 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.898 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.898 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.898 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.898 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.898 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.898 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.898 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.898 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.899 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.899 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.899 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.899 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.899 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.899 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.899 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.899 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.899 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.899 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.899 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.899 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.899 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.900 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.900 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.900 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.900 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.900 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.900 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.900 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.900 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.900 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.900 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.900 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.900 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.900 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.900 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.901 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.901 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.901 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.901 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.901 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.901 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.901 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.901 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.901 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.901 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.901 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.901 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.901 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.901 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.901 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.902 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.902 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.902 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.902 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.902 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.902 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.902 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.902 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.902 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.902 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.902 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.902 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.902 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.902 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.903 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.903 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.903 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.903 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.903 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.903 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.903 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.903 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.903 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools": { 2024-04-22T13:34:40.903 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools", 2024-04-22T13:34:40.903 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.903 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.903 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.903 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.903 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.904 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.904 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.904 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.904 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.904 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.904 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.904 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.904 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-22T13:34:40.904 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-22T13:34:40.904 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.904 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.904 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.904 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "300", 2024-04-22T13:34:40.904 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.905 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.905 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.905 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.905 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.905 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.905 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.905 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.905 INFO:teuthology.orchestra.run.smithi012.stdout: "scrape_interval": { 2024-04-22T13:34:40.905 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "scrape_interval", 2024-04-22T13:34:40.905 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-22T13:34:40.905 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.905 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.905 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15.0", 2024-04-22T13:34:40.905 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.905 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.906 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.906 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.906 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.906 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.906 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.906 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.906 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-04-22T13:34:40.906 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-04-22T13:34:40.906 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.906 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.906 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.906 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "::", 2024-04-22T13:34:40.906 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.906 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.907 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.907 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-22T13:34:40.907 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.907 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.907 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.907 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.907 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-04-22T13:34:40.907 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-04-22T13:34:40.907 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.907 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.907 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.907 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "9283", 2024-04-22T13:34:40.907 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.907 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.908 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.908 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-22T13:34:40.908 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.908 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.908 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.908 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.908 INFO:teuthology.orchestra.run.smithi012.stdout: "stale_cache_strategy": { 2024-04-22T13:34:40.908 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stale_cache_strategy", 2024-04-22T13:34:40.908 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.908 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.908 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.908 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "log", 2024-04-22T13:34:40.908 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.908 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.908 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.909 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.909 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.909 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.909 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.909 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.909 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_behaviour": { 2024-04-22T13:34:40.909 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_behaviour", 2024-04-22T13:34:40.909 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.909 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.909 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.909 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "default", 2024-04-22T13:34:40.909 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.909 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.909 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.909 INFO:teuthology.orchestra.run.smithi012.stdout: "default", 2024-04-22T13:34:40.910 INFO:teuthology.orchestra.run.smithi012.stdout: "error" 2024-04-22T13:34:40.910 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.910 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.910 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.910 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.910 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.910 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.910 INFO:teuthology.orchestra.run.smithi012.stdout: "standby_error_status_code": { 2024-04-22T13:34:40.910 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "standby_error_status_code", 2024-04-22T13:34:40.910 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.910 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.910 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.910 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "500", 2024-04-22T13:34:40.911 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "400", 2024-04-22T13:34:40.911 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "599", 2024-04-22T13:34:40.911 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.911 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.911 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.911 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.911 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.911 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.911 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.911 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.911 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.911 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_support", 2024-04-22T13:34:40.911 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.911 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.911 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.912 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.912 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.912 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.912 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.912 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.912 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.912 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.912 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.912 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.912 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.912 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.912 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.912 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.912 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.912 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.913 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.913 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.913 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.913 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.913 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.913 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.913 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.913 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.913 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.913 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.913 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.913 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.913 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.913 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.913 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.914 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.914 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.914 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.914 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.914 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.914 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.914 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.914 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.914 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.914 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.914 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.914 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.914 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.914 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.914 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.915 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.915 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.915 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.915 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.915 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.915 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.915 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.915 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.915 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.915 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.915 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.915 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.915 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.915 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.916 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.916 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.916 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.916 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.916 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.916 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.916 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.916 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.916 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.916 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.916 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.916 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_snap_create": { 2024-04-22T13:34:40.916 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_snap_create", 2024-04-22T13:34:40.916 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.917 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.917 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.917 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10", 2024-04-22T13:34:40.917 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.917 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.917 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.917 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.917 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.917 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.917 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.917 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.917 INFO:teuthology.orchestra.run.smithi012.stdout: "mirror_snapshot_schedule": { 2024-04-22T13:34:40.917 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mirror_snapshot_schedule", 2024-04-22T13:34:40.917 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.917 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.918 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.918 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.918 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.918 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.918 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.918 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.918 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.918 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.918 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.918 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.918 INFO:teuthology.orchestra.run.smithi012.stdout: "trash_purge_schedule": { 2024-04-22T13:34:40.918 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "trash_purge_schedule", 2024-04-22T13:34:40.918 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.918 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.919 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.919 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.919 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.919 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.919 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.919 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.919 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.919 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.919 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.919 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.919 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.919 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.919 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.919 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "restful", 2024-04-22T13:34:40.920 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.920 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.920 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.920 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_auth": { 2024-04-22T13:34:40.920 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enable_auth", 2024-04-22T13:34:40.920 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.920 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.920 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.920 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.920 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.920 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.920 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.920 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.920 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.921 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.921 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.921 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.921 INFO:teuthology.orchestra.run.smithi012.stdout: "key_file": { 2024-04-22T13:34:40.921 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "key_file", 2024-04-22T13:34:40.921 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.921 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.921 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.921 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.921 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.921 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.921 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.921 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.921 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.922 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.922 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.922 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.922 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.922 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.922 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.922 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.922 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.922 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.922 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.922 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.922 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.922 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.923 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.923 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.923 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.923 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.923 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.923 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.923 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.923 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.923 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.923 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.923 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.923 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.923 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.924 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.924 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.924 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.924 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.924 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.924 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.924 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.924 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.924 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.924 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.924 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.924 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.924 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.924 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.925 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.925 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.925 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.925 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.925 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.925 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.925 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.925 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.925 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.925 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.925 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.925 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.925 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.925 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.926 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.926 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.926 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.926 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.926 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.926 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.926 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.926 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.926 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.926 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.926 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.926 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.926 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.927 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.927 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.927 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.927 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.927 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.927 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.927 INFO:teuthology.orchestra.run.smithi012.stdout: "server_addr": { 2024-04-22T13:34:40.927 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_addr", 2024-04-22T13:34:40.927 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.927 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.927 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.927 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.927 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.927 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.927 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.928 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.928 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.928 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.928 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.928 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.928 INFO:teuthology.orchestra.run.smithi012.stdout: "server_port": { 2024-04-22T13:34:40.928 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "server_port", 2024-04-22T13:34:40.928 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.928 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.928 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.928 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.928 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.928 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.928 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.929 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.929 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.929 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.929 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.929 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.929 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.929 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.929 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.929 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rgw", 2024-04-22T13:34:40.929 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.929 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.929 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.929 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.929 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.930 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.930 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.930 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.930 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.930 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.930 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.930 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.930 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.930 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.930 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.930 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.930 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.930 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.930 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.931 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.931 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.931 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.931 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.931 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.931 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.931 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.931 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.931 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.931 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.931 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.931 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.931 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.931 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.931 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.932 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.932 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.932 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.932 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.932 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.932 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.932 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.932 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.932 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.932 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.932 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.932 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.932 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.932 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.933 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.933 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.933 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.933 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.933 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.933 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.933 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.933 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.933 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.933 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.933 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.933 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.933 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.933 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.933 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.934 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.934 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.934 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.934 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.934 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.934 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.934 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.934 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.934 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.934 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.934 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.934 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.934 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.934 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rook", 2024-04-22T13:34:40.935 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.935 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.935 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.935 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.935 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.935 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.935 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.935 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.935 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.935 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.935 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.935 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.935 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.935 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.936 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.936 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.936 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.936 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.936 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.936 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.936 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.936 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.936 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.936 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.936 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.936 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.936 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.936 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.937 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.937 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.937 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.937 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.937 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.937 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.937 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.937 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.937 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.937 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.937 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.937 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.937 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.937 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.937 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.937 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.938 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.938 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.938 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.938 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.938 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.938 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.938 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.938 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.938 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.938 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.938 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.938 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.938 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.939 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.939 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.939 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.939 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.939 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.939 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.939 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.939 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.939 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.939 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.939 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.939 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.939 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.939 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.939 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.940 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.940 INFO:teuthology.orchestra.run.smithi012.stdout: "storage_class": { 2024-04-22T13:34:40.940 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "storage_class", 2024-04-22T13:34:40.940 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.940 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.940 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.940 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "local", 2024-04-22T13:34:40.940 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.940 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.940 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.940 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-22T13:34:40.940 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.940 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.940 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.941 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.941 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.941 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.941 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.941 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "selftest", 2024-04-22T13:34:40.941 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.941 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.941 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.941 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.941 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.941 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.941 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.941 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.941 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.941 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.942 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.942 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.942 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.942 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.942 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.942 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.942 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.942 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.942 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.942 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.942 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.942 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.942 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.942 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.943 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.943 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.943 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.943 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.943 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.943 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.943 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.943 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.943 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.943 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.943 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.943 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.943 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.943 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.943 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.944 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.944 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.944 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.944 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.944 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.944 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.944 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.944 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.944 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.944 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.944 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.944 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.944 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.944 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.945 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.945 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.945 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.945 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.945 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.945 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.945 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.945 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.945 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.945 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.945 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.945 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.945 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.945 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.946 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.946 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.946 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.946 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.946 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.946 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.946 INFO:teuthology.orchestra.run.smithi012.stdout: "roption1": { 2024-04-22T13:34:40.946 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption1", 2024-04-22T13:34:40.946 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.946 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.946 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.946 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.946 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.946 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.946 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.947 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.947 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.947 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.947 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.947 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.947 INFO:teuthology.orchestra.run.smithi012.stdout: "roption2": { 2024-04-22T13:34:40.947 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "roption2", 2024-04-22T13:34:40.947 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.947 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.947 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.947 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "xyz", 2024-04-22T13:34:40.947 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.947 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.947 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.948 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.948 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.948 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.948 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.948 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.948 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption1": { 2024-04-22T13:34:40.948 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption1", 2024-04-22T13:34:40.948 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.948 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.948 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.948 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.948 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.948 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.948 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.949 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.949 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.949 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.949 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.949 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.949 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption2": { 2024-04-22T13:34:40.949 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption2", 2024-04-22T13:34:40.949 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.949 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.949 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.949 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.949 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.949 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.949 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.949 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.950 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.950 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.950 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.950 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.950 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption3": { 2024-04-22T13:34:40.950 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption3", 2024-04-22T13:34:40.950 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "float", 2024-04-22T13:34:40.950 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.950 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.950 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.950 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.950 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.950 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.950 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.951 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.951 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.951 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.951 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.951 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption4": { 2024-04-22T13:34:40.951 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption4", 2024-04-22T13:34:40.951 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.951 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.951 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.951 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.951 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.951 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.951 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.951 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.951 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.952 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.952 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.952 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.952 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption5": { 2024-04-22T13:34:40.952 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption5", 2024-04-22T13:34:40.952 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.952 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.952 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.952 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.952 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.952 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.952 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.952 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.952 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.953 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.953 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.953 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.953 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption6": { 2024-04-22T13:34:40.953 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption6", 2024-04-22T13:34:40.953 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.953 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.953 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.953 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.953 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.953 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.953 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.953 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.954 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.954 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.954 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.954 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.954 INFO:teuthology.orchestra.run.smithi012.stdout: "rwoption7": { 2024-04-22T13:34:40.954 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rwoption7", 2024-04-22T13:34:40.954 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.954 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.954 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.954 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.954 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "1", 2024-04-22T13:34:40.954 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "42", 2024-04-22T13:34:40.954 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.954 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.955 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.955 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.955 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.955 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.955 INFO:teuthology.orchestra.run.smithi012.stdout: "testkey": { 2024-04-22T13:34:40.955 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testkey", 2024-04-22T13:34:40.955 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.955 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.955 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.955 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.955 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.955 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.955 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.955 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.956 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.956 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.956 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.956 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.956 INFO:teuthology.orchestra.run.smithi012.stdout: "testlkey": { 2024-04-22T13:34:40.956 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testlkey", 2024-04-22T13:34:40.956 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.956 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.956 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.956 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.956 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.956 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.956 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.956 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.956 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.957 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.957 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.957 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.957 INFO:teuthology.orchestra.run.smithi012.stdout: "testnewline": { 2024-04-22T13:34:40.957 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "testnewline", 2024-04-22T13:34:40.957 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.957 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.957 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.957 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.957 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.957 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.957 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.957 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.957 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.958 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.958 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.958 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.958 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.958 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.958 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.958 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snap_schedule", 2024-04-22T13:34:40.958 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.958 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.958 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.958 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_m_granularity": { 2024-04-22T13:34:40.958 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "allow_m_granularity", 2024-04-22T13:34:40.958 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.958 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.959 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.959 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.959 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.959 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.959 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.959 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "allow minute scheduled snapshots", 2024-04-22T13:34:40.959 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.959 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.959 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.959 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.959 INFO:teuthology.orchestra.run.smithi012.stdout: "dump_on_update": { 2024-04-22T13:34:40.959 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "dump_on_update", 2024-04-22T13:34:40.959 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.959 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.960 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.960 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.960 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.960 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.960 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.960 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "dump database to debug log on update", 2024-04-22T13:34:40.960 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.960 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.960 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.960 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.960 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.960 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.960 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.960 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.960 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.961 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.961 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.961 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.961 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.961 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.961 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.961 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.961 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.961 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.961 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.961 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.961 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.961 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.961 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.962 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.962 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.962 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.962 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.962 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.962 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.962 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.962 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.962 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.962 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.962 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.962 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.962 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.962 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.963 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.963 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.963 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.963 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.963 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.963 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.963 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.963 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.963 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.963 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.963 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.963 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.963 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.963 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.963 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.964 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.964 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.964 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.964 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.964 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.964 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.964 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.964 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.964 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.964 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.964 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.964 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.964 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.964 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.964 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.965 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.965 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.965 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.965 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.965 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.965 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.965 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.965 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.965 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.965 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.965 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "stats", 2024-04-22T13:34:40.965 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.965 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.965 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.966 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.966 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.966 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.966 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.966 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.966 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.966 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.966 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.966 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.966 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.966 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.966 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.966 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.966 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.967 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.967 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.967 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.967 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.967 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.967 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.967 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.967 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.967 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.967 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.967 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.967 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.967 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.967 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.967 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.968 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.968 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.968 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.968 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.968 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.968 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.968 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.968 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.968 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.968 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.968 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.968 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.968 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.968 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.968 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.968 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.969 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.969 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.969 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.969 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.969 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.969 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.969 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.969 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.969 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.969 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.969 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.969 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.969 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.969 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.969 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.970 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.970 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.970 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.970 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.970 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.970 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.970 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.970 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.970 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.970 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.970 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.970 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.970 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.970 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "status", 2024-04-22T13:34:40.971 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.971 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.971 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.971 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.971 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.971 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.971 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.971 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.971 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.971 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.971 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.971 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.971 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.971 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.972 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.972 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.972 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.972 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.972 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.972 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.972 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.972 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.972 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.972 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.972 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.972 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.972 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.972 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.973 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.973 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.973 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.973 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.973 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.973 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.973 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.973 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.973 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.973 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.973 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.973 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.973 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.974 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.974 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.974 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.974 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.974 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.974 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.974 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.974 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.974 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.974 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.974 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.974 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.974 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.974 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.975 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.975 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.975 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.975 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.975 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.975 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.975 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.975 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.975 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.975 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.975 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.975 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.975 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.976 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.976 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.976 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.976 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.976 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.976 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.976 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telegraf", 2024-04-22T13:34:40.976 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.976 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.976 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.976 INFO:teuthology.orchestra.run.smithi012.stdout: "address": { 2024-04-22T13:34:40.976 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "address", 2024-04-22T13:34:40.976 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.976 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.977 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.977 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-22T13:34:40.977 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.977 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.977 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.977 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.977 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.977 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.977 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.977 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-04-22T13:34:40.977 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-04-22T13:34:40.977 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:40.977 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.977 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.978 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "15", 2024-04-22T13:34:40.978 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.978 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.978 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.978 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.978 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.978 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.978 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.978 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.978 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.978 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.978 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.978 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.979 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.979 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.979 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.979 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.979 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.979 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.979 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.979 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.979 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.979 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.979 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.979 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.979 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.979 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.980 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.980 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.980 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.980 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.980 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.980 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.980 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.980 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.980 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.980 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.980 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.980 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.980 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.980 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.981 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.981 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.981 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.981 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.981 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.981 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.981 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.981 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.981 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.981 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.981 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.981 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.981 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.981 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.982 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.982 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.982 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.982 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.982 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.982 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.982 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:40.982 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:40.982 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.982 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.982 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.982 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.982 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.983 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.983 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.983 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.983 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.983 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.983 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.983 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.983 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:40.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.983 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:40.983 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "telemetry", 2024-04-22T13:34:40.983 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:40.983 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:40.983 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:40.983 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_basic": { 2024-04-22T13:34:40.984 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_basic", 2024-04-22T13:34:40.984 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.984 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.984 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.984 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.984 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.984 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.984 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.984 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-22T13:34:40.984 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.984 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.984 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.984 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_crash": { 2024-04-22T13:34:40.984 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_crash", 2024-04-22T13:34:40.985 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.985 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.985 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.985 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.985 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.985 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.985 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.985 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-22T13:34:40.985 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.985 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.985 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.985 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_device": { 2024-04-22T13:34:40.985 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_device", 2024-04-22T13:34:40.985 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.986 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.986 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.986 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:40.986 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.986 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.986 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.986 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-22T13:34:40.986 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.986 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.986 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.986 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_ident": { 2024-04-22T13:34:40.986 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_ident", 2024-04-22T13:34:40.987 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.987 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.987 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.987 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.987 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.987 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.987 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.987 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-22T13:34:40.987 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.987 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.987 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.987 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.987 INFO:teuthology.orchestra.run.smithi012.stdout: "channel_perf": { 2024-04-22T13:34:40.987 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "channel_perf", 2024-04-22T13:34:40.987 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.988 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.988 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.988 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.988 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.988 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.988 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.988 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-22T13:34:40.988 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.988 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.988 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.988 INFO:teuthology.orchestra.run.smithi012.stdout: "contact": { 2024-04-22T13:34:40.988 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "contact", 2024-04-22T13:34:40.988 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.989 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.989 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.989 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.989 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.989 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.989 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.989 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.989 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.989 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.989 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.989 INFO:teuthology.orchestra.run.smithi012.stdout: "description": { 2024-04-22T13:34:40.989 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "description", 2024-04-22T13:34:40.989 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.990 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.990 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.990 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.990 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.990 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.990 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.990 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.990 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.990 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.990 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.990 INFO:teuthology.orchestra.run.smithi012.stdout: "device_url": { 2024-04-22T13:34:40.990 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "device_url", 2024-04-22T13:34:40.990 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.991 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.991 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.991 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-22T13:34:40.991 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.991 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.991 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.991 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.991 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.991 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.991 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.991 INFO:teuthology.orchestra.run.smithi012.stdout: "enabled": { 2024-04-22T13:34:40.991 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "enabled", 2024-04-22T13:34:40.991 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.991 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.992 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.992 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.992 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.992 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.992 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.992 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.992 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.992 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.992 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.992 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-04-22T13:34:40.992 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-04-22T13:34:40.992 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.992 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.993 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.993 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "24", 2024-04-22T13:34:40.993 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "8", 2024-04-22T13:34:40.993 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.993 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.993 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.993 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.993 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.993 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.993 INFO:teuthology.orchestra.run.smithi012.stdout: "last_opt_revision": { 2024-04-22T13:34:40.993 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "last_opt_revision", 2024-04-22T13:34:40.993 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:40.993 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.993 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.994 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "1", 2024-04-22T13:34:40.994 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.994 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.994 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.994 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.994 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.994 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.994 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.994 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard": { 2024-04-22T13:34:40.994 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard", 2024-04-22T13:34:40.994 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.994 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.994 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.995 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.995 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.995 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.995 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.995 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.995 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.995 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.995 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.995 INFO:teuthology.orchestra.run.smithi012.stdout: "leaderboard_description": { 2024-04-22T13:34:40.995 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "leaderboard_description", 2024-04-22T13:34:40.995 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.995 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.995 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:40.995 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.996 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.996 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.996 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.996 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.996 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.996 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.996 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.996 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:40.996 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:40.996 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.996 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.996 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.996 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:40.997 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.997 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.997 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.997 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.997 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.997 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.997 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.997 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.997 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.997 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:40.997 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.997 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.997 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.997 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.998 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:40.998 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:40.998 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:40.998 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.998 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.998 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:40.998 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.998 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.998 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:40.998 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:40.998 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:40.998 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:40.998 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:40.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:40.998 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:40.999 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:40.999 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:40.999 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:40.999 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:40.999 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:40.999 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:40.999 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:40.999 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:40.999 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:40.999 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:40.999 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:40.999 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:40.999 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:40.999 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:40.999 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:41.000 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:41.000 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:41.000 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:41.000 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:41.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.000 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:41.000 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:41.000 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:41.000 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:41.000 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:41.000 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:41.000 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:41.000 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:41.001 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:41.001 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:41.001 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:41.001 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:41.001 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:41.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.001 INFO:teuthology.orchestra.run.smithi012.stdout: "organization": { 2024-04-22T13:34:41.001 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "organization", 2024-04-22T13:34:41.001 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:41.001 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:41.001 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:41.001 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:41.001 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:41.001 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:41.002 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:41.002 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:41.002 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:41.002 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:41.002 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:41.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.002 INFO:teuthology.orchestra.run.smithi012.stdout: "proxy": { 2024-04-22T13:34:41.002 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "proxy", 2024-04-22T13:34:41.002 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:41.002 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:41.002 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:41.002 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:41.002 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:41.003 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:41.003 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:41.003 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:41.003 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:41.003 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:41.003 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:41.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.003 INFO:teuthology.orchestra.run.smithi012.stdout: "url": { 2024-04-22T13:34:41.003 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "url", 2024-04-22T13:34:41.003 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:41.003 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:41.003 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:41.003 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-22T13:34:41.003 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:41.004 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:41.004 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:41.004 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:41.004 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:41.004 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:41.004 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:41.004 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:41.004 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:41.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.004 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:41.004 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "test_orchestrator", 2024-04-22T13:34:41.004 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:41.004 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:41.005 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:41.005 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:41.005 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:41.005 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:41.005 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:41.005 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:41.005 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:41.005 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:41.005 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:41.005 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:41.005 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:41.005 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:41.005 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:41.005 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:41.005 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:41.006 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:41.006 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:41.006 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:41.006 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:41.006 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:41.006 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:41.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.006 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:41.006 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:41.006 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:41.006 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:41.006 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:41.006 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:41.006 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:41.007 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:41.007 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:41.007 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:41.007 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:41.007 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:41.007 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:41.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.007 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:41.007 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:41.007 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:41.007 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:41.007 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:41.007 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:41.007 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:41.008 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:41.008 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:41.008 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:41.008 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:41.008 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:41.008 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:41.008 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:41.008 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:41.008 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:41.008 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:41.008 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:41.008 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:41.008 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:41.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.008 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:41.009 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:41.009 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:41.009 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:41.009 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:41.009 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:41.009 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:41.009 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:41.009 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:41.009 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:41.009 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:41.009 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:41.009 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:41.009 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:41.009 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:41.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.010 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:41.010 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "volumes", 2024-04-22T13:34:41.010 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:41.010 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:41.010 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:41.010 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:41.010 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:41.010 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:41.010 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:41.010 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:41.010 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:41.010 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:41.010 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:41.010 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:41.011 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:41.011 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:41.011 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:41.011 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:41.011 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:41.011 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:41.011 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:41.011 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:41.011 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:41.011 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:41.011 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:41.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.011 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:41.011 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:41.011 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:41.012 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:41.012 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:41.012 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:41.012 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:41.012 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:41.012 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:41.012 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:41.012 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:41.012 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:41.012 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:41.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.012 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:41.012 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:41.013 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:41.013 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:41.013 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:41.013 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:41.013 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:41.013 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:41.013 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:41.013 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:41.013 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:41.013 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:41.013 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:41.013 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:41.013 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:41.013 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:41.013 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:41.014 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:41.014 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:41.014 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:41.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.014 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:41.014 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:41.014 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:41.014 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:41.014 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:41.014 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:41.014 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:41.014 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:41.014 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:41.014 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:41.015 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:41.015 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:41.015 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:41.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.015 INFO:teuthology.orchestra.run.smithi012.stdout: "max_concurrent_clones": { 2024-04-22T13:34:41.015 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "max_concurrent_clones", 2024-04-22T13:34:41.015 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:41.015 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:41.015 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:41.015 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "4", 2024-04-22T13:34:41.015 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:41.015 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:41.015 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:41.015 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-22T13:34:41.016 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:41.016 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:41.016 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:41.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.016 INFO:teuthology.orchestra.run.smithi012.stdout: "periodic_async_work": { 2024-04-22T13:34:41.016 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "periodic_async_work", 2024-04-22T13:34:41.016 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:41.016 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:41.016 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:41.016 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:41.016 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:41.016 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:41.017 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:41.017 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Periodically check for async work", 2024-04-22T13:34:41.017 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:41.017 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:41.017 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:41.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.017 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_delay": { 2024-04-22T13:34:41.017 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_delay", 2024-04-22T13:34:41.018 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:41.018 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:41.018 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:41.018 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "0", 2024-04-22T13:34:41.018 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:41.018 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:41.018 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:41.019 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-22T13:34:41.019 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:41.019 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:41.019 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:41.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.019 INFO:teuthology.orchestra.run.smithi012.stdout: "snapshot_clone_no_wait": { 2024-04-22T13:34:41.020 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snapshot_clone_no_wait", 2024-04-22T13:34:41.020 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:41.020 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:41.020 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:41.020 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "True", 2024-04-22T13:34:41.020 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:41.020 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:41.021 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:41.021 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-22T13:34:41.021 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:41.021 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:41.021 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:41.021 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:41.021 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:41.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.021 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:41.021 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix", 2024-04-22T13:34:41.021 INFO:teuthology.orchestra.run.smithi012.stdout: "can_run": true, 2024-04-22T13:34:41.022 INFO:teuthology.orchestra.run.smithi012.stdout: "error_string": "", 2024-04-22T13:34:41.022 INFO:teuthology.orchestra.run.smithi012.stdout: "module_options": { 2024-04-22T13:34:41.022 INFO:teuthology.orchestra.run.smithi012.stdout: "discovery_interval": { 2024-04-22T13:34:41.022 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "discovery_interval", 2024-04-22T13:34:41.022 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "uint", 2024-04-22T13:34:41.022 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:41.022 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:41.022 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "100", 2024-04-22T13:34:41.022 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:41.022 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:41.022 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:41.022 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:41.022 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:41.022 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:41.023 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:41.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.023 INFO:teuthology.orchestra.run.smithi012.stdout: "identifier": { 2024-04-22T13:34:41.023 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "identifier", 2024-04-22T13:34:41.023 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:41.023 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:41.023 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:41.023 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:41.023 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:41.023 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:41.023 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:41.023 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:41.023 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:41.023 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:41.023 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:41.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.024 INFO:teuthology.orchestra.run.smithi012.stdout: "interval": { 2024-04-22T13:34:41.024 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "interval", 2024-04-22T13:34:41.024 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "secs", 2024-04-22T13:34:41.024 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:41.024 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:41.024 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "60", 2024-04-22T13:34:41.024 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:41.024 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:41.024 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:41.024 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:41.024 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:41.024 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:41.024 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:41.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.025 INFO:teuthology.orchestra.run.smithi012.stdout: "log_level": { 2024-04-22T13:34:41.025 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_level", 2024-04-22T13:34:41.025 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:41.025 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:41.025 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:41.025 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:41.025 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:41.025 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:41.025 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:41.025 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:41.025 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:41.025 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:41.025 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:41.026 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:41.026 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:41.026 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:41.026 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:41.026 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:41.026 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:41.026 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:41.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.026 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster": { 2024-04-22T13:34:41.026 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster", 2024-04-22T13:34:41.026 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:41.026 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:41.026 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:41.026 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:41.027 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:41.027 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:41.027 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:41.027 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:41.027 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:41.027 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:41.027 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:41.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.027 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_cluster_level": { 2024-04-22T13:34:41.027 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_cluster_level", 2024-04-22T13:34:41.027 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:41.027 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:41.027 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:41.027 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "info", 2024-04-22T13:34:41.028 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:41.028 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:41.028 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [ 2024-04-22T13:34:41.028 INFO:teuthology.orchestra.run.smithi012.stdout: "", 2024-04-22T13:34:41.028 INFO:teuthology.orchestra.run.smithi012.stdout: "critical", 2024-04-22T13:34:41.028 INFO:teuthology.orchestra.run.smithi012.stdout: "debug", 2024-04-22T13:34:41.028 INFO:teuthology.orchestra.run.smithi012.stdout: "error", 2024-04-22T13:34:41.028 INFO:teuthology.orchestra.run.smithi012.stdout: "info", 2024-04-22T13:34:41.028 INFO:teuthology.orchestra.run.smithi012.stdout: "warning" 2024-04-22T13:34:41.028 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:41.028 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:41.028 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:41.028 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:41.028 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:41.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.029 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": { 2024-04-22T13:34:41.029 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "log_to_file", 2024-04-22T13:34:41.029 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "bool", 2024-04-22T13:34:41.029 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:41.029 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-22T13:34:41.029 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "False", 2024-04-22T13:34:41.029 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:41.029 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:41.029 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:41.029 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:41.029 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:41.029 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:41.029 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:41.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.029 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_host": { 2024-04-22T13:34:41.029 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_host", 2024-04-22T13:34:41.029 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:41.030 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:41.030 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:41.030 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "", 2024-04-22T13:34:41.030 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:41.030 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:41.030 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:41.030 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:41.030 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:41.030 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:41.030 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:41.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.030 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_port": { 2024-04-22T13:34:41.030 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_port", 2024-04-22T13:34:41.030 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "int", 2024-04-22T13:34:41.030 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:41.031 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:41.031 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "10051", 2024-04-22T13:34:41.031 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:41.031 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:41.031 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:41.031 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:41.031 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:41.031 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:41.031 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:41.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.031 INFO:teuthology.orchestra.run.smithi012.stdout: "zabbix_sender": { 2024-04-22T13:34:41.031 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "zabbix_sender", 2024-04-22T13:34:41.031 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "str", 2024-04-22T13:34:41.031 INFO:teuthology.orchestra.run.smithi012.stdout: "level": "advanced", 2024-04-22T13:34:41.032 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 0, 2024-04-22T13:34:41.032 INFO:teuthology.orchestra.run.smithi012.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-22T13:34:41.032 INFO:teuthology.orchestra.run.smithi012.stdout: "min": "", 2024-04-22T13:34:41.032 INFO:teuthology.orchestra.run.smithi012.stdout: "max": "", 2024-04-22T13:34:41.032 INFO:teuthology.orchestra.run.smithi012.stdout: "enum_allowed": [], 2024-04-22T13:34:41.032 INFO:teuthology.orchestra.run.smithi012.stdout: "desc": "", 2024-04-22T13:34:41.032 INFO:teuthology.orchestra.run.smithi012.stdout: "long_desc": "", 2024-04-22T13:34:41.032 INFO:teuthology.orchestra.run.smithi012.stdout: "tags": [], 2024-04-22T13:34:41.032 INFO:teuthology.orchestra.run.smithi012.stdout: "see_also": [] 2024-04-22T13:34:41.032 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:41.032 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:41.032 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:41.032 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:41.032 INFO:teuthology.orchestra.run.smithi012.stdout: "services": { 2024-04-22T13:34:41.032 INFO:teuthology.orchestra.run.smithi012.stdout: "dashboard": "https://172.21.15.12:8443/" 2024-04-22T13:34:41.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.033 INFO:teuthology.orchestra.run.smithi012.stdout: "always_on_modules": { 2024-04-22T13:34:41.033 INFO:teuthology.orchestra.run.smithi012.stdout: "octopus": [ 2024-04-22T13:34:41.033 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-04-22T13:34:41.033 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-04-22T13:34:41.033 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-04-22T13:34:41.033 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-04-22T13:34:41.033 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-04-22T13:34:41.033 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-04-22T13:34:41.033 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-04-22T13:34:41.033 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-04-22T13:34:41.033 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-04-22T13:34:41.033 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-04-22T13:34:41.033 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:41.034 INFO:teuthology.orchestra.run.smithi012.stdout: "pacific": [ 2024-04-22T13:34:41.034 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-04-22T13:34:41.034 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-04-22T13:34:41.034 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-04-22T13:34:41.034 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-04-22T13:34:41.034 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-04-22T13:34:41.034 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-04-22T13:34:41.034 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-04-22T13:34:41.034 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-04-22T13:34:41.034 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-04-22T13:34:41.034 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-04-22T13:34:41.034 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:41.034 INFO:teuthology.orchestra.run.smithi012.stdout: "quincy": [ 2024-04-22T13:34:41.034 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-04-22T13:34:41.035 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-04-22T13:34:41.035 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-04-22T13:34:41.035 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-04-22T13:34:41.035 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-04-22T13:34:41.035 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-04-22T13:34:41.035 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-04-22T13:34:41.035 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-04-22T13:34:41.035 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-04-22T13:34:41.035 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-04-22T13:34:41.035 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:41.035 INFO:teuthology.orchestra.run.smithi012.stdout: "reef": [ 2024-04-22T13:34:41.035 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-04-22T13:34:41.035 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-04-22T13:34:41.035 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-04-22T13:34:41.036 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-04-22T13:34:41.036 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-04-22T13:34:41.036 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-04-22T13:34:41.036 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-04-22T13:34:41.036 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-04-22T13:34:41.036 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-04-22T13:34:41.036 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-04-22T13:34:41.036 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-22T13:34:41.036 INFO:teuthology.orchestra.run.smithi012.stdout: "squid": [ 2024-04-22T13:34:41.036 INFO:teuthology.orchestra.run.smithi012.stdout: "balancer", 2024-04-22T13:34:41.036 INFO:teuthology.orchestra.run.smithi012.stdout: "crash", 2024-04-22T13:34:41.036 INFO:teuthology.orchestra.run.smithi012.stdout: "devicehealth", 2024-04-22T13:34:41.036 INFO:teuthology.orchestra.run.smithi012.stdout: "orchestrator", 2024-04-22T13:34:41.036 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscaler", 2024-04-22T13:34:41.036 INFO:teuthology.orchestra.run.smithi012.stdout: "progress", 2024-04-22T13:34:41.037 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_support", 2024-04-22T13:34:41.037 INFO:teuthology.orchestra.run.smithi012.stdout: "status", 2024-04-22T13:34:41.037 INFO:teuthology.orchestra.run.smithi012.stdout: "telemetry", 2024-04-22T13:34:41.037 INFO:teuthology.orchestra.run.smithi012.stdout: "volumes" 2024-04-22T13:34:41.037 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-22T13:34:41.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.037 INFO:teuthology.orchestra.run.smithi012.stdout: "last_failure_osd_epoch": 90, 2024-04-22T13:34:41.037 INFO:teuthology.orchestra.run.smithi012.stdout: "active_clients": [ 2024-04-22T13:34:41.037 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:41.037 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "devicehealth", 2024-04-22T13:34:41.037 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-22T13:34:41.037 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:41.037 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-22T13:34:41.038 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-04-22T13:34:41.038 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2353208714 2024-04-22T13:34:41.038 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:41.038 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-22T13:34:41.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.038 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:41.038 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "libcephsqlite", 2024-04-22T13:34:41.038 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-22T13:34:41.038 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:41.038 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-22T13:34:41.038 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-04-22T13:34:41.038 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 8129225 2024-04-22T13:34:41.038 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:41.038 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-22T13:34:41.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.039 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:41.039 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "rbd_support", 2024-04-22T13:34:41.039 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-22T13:34:41.039 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:41.039 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-22T13:34:41.039 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-04-22T13:34:41.039 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 435386301 2024-04-22T13:34:41.039 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:41.039 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-22T13:34:41.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.039 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:41.039 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snap_schedule", 2024-04-22T13:34:41.039 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-22T13:34:41.039 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:41.040 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-22T13:34:41.040 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-04-22T13:34:41.040 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 33855450 2024-04-22T13:34:41.040 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:41.040 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-22T13:34:41.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.040 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:41.040 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "snap_schedule", 2024-04-22T13:34:41.040 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-22T13:34:41.040 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:41.040 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-22T13:34:41.040 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-04-22T13:34:41.040 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1486523455 2024-04-22T13:34:41.040 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:41.040 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-22T13:34:41.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:41.041 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:41.041 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "volumes", 2024-04-22T13:34:41.041 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-22T13:34:41.041 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-22T13:34:41.041 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-22T13:34:41.041 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:0", 2024-04-22T13:34:41.041 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1544776777 2024-04-22T13:34:41.041 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:41.041 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-22T13:34:41.041 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:41.041 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-22T13:34:41.041 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:34:41.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:39 smithi111 ceph-mon[31361]: pgmap v327: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:34:41.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:39 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/699254260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:34:41.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:39 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/4176406739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:34:41.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:39 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1348419328' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:34:41.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:40 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2000617103' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-22T13:34:41.043 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:39 smithi134 ceph-mon[31338]: pgmap v327: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:34:41.044 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:39 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/699254260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:34:41.044 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:39 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/4176406739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:34:41.044 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:39 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1348419328' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:34:41.044 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:40 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2000617103' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-22T13:34:41.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.772+0000 7fa298cc1640 1 -- 172.21.15.12:0/2000617103 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa27c0a3d20 msgr2=0x7fa27c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:41.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.772+0000 7fa298cc1640 1 --2- 172.21.15.12:0/2000617103 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa27c0a3d20 0x7fa27c0a61e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa2840058c0 tx=0x7fa28400a2d0 comp rx=0 tx=0).stop 2024-04-22T13:34:41.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.772+0000 7fa298cc1640 1 -- 172.21.15.12:0/2000617103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa294107260 msgr2=0x7fa294198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:34:41.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.772+0000 7fa298cc1640 1 --2- 172.21.15.12:0/2000617103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa294107260 0x7fa294198c80 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fa288036040 tx=0x7fa28802c7e0 comp rx=0 tx=0).stop 2024-04-22T13:34:41.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.772+0000 7fa298cc1640 1 -- 172.21.15.12:0/2000617103 shutdown_connections 2024-04-22T13:34:41.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.772+0000 7fa298cc1640 1 --2- 172.21.15.12:0/2000617103 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa27c0a3d20 0x7fa27c0a61e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:41.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.772+0000 7fa298cc1640 1 --2- 172.21.15.12:0/2000617103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa29419d000 0x7fa29419d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:41.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.772+0000 7fa298cc1640 1 --2- 172.21.15.12:0/2000617103 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa294107260 0x7fa294198c80 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:41.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.772+0000 7fa298cc1640 1 --2- 172.21.15.12:0/2000617103 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa294102190 0x7fa294198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:34:41.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.772+0000 7fa298cc1640 1 -- 172.21.15.12:0/2000617103 >> 172.21.15.12:0/2000617103 conn(0x7fa294079310 msgr2=0x7fa2941002e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:34:41.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.772+0000 7fa298cc1640 1 -- 172.21.15.12:0/2000617103 shutdown_connections 2024-04-22T13:34:41.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:34:39.772+0000 7fa298cc1640 1 -- 172.21.15.12:0/2000617103 wait complete. 2024-04-22T13:34:41.045 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:39 smithi012 ceph-mon[28265]: pgmap v327: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:34:41.045 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:39 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/699254260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:34:41.045 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:39 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/4176406739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:34:41.045 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:39 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1348419328' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:34:41.053 DEBUG:tasks.check_counter:Getting stats from a 2024-04-22T13:34:41.053 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-22T13:34:41.054 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:40 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2000617103' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-22T13:34:41.312 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:34:41.689 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:41 smithi012 ceph-mon[28265]: pgmap v328: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:41.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:41 smithi134 ceph-mon[31338]: pgmap v328: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:41.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:41 smithi111 ceph-mon[31361]: pgmap v328: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:43.122 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:34:43.122 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T13:34:43.122 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 1511, 2024-04-22T13:34:43.122 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 1511, 2024-04-22T13:34:43.122 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 455600, 2024-04-22T13:34:43.122 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 479675, 2024-04-22T13:34:43.122 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 6, 2024-04-22T13:34:43.122 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 2, 2024-04-22T13:34:43.122 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.151383342, 2024-04-22T13:34:43.122 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.048923733, 2024-04-22T13:34:43.122 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.039984158, 2024-04-22T13:34:43.123 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.047884616, 2024-04-22T13:34:43.123 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:34:43.123 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 1510, 2024-04-22T13:34:43.123 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.056880624, 2024-04-22T13:34:43.123 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000037669 2024-04-22T13:34:43.123 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.123 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:34:43.123 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.123 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.123 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.123 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.123 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-22T13:34:43.123 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 448 2024-04-22T13:34:43.123 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.124 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T13:34:43.124 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 643, 2024-04-22T13:34:43.124 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 799, 2024-04-22T13:34:43.124 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 158658, 2024-04-22T13:34:43.124 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 647170, 2024-04-22T13:34:43.124 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 7, 2024-04-22T13:34:43.124 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 5, 2024-04-22T13:34:43.124 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.082092252, 2024-04-22T13:34:43.124 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.027347121, 2024-04-22T13:34:43.124 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.018945055, 2024-04-22T13:34:43.124 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.027284418, 2024-04-22T13:34:43.124 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:34:43.124 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 796, 2024-04-22T13:34:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.038690973, 2024-04-22T13:34:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000048606 2024-04-22T13:34:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:34:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-22T13:34:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 443264 2024-04-22T13:34:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T13:34:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 236, 2024-04-22T13:34:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 224, 2024-04-22T13:34:43.125 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 1618268, 2024-04-22T13:34:43.126 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 129870, 2024-04-22T13:34:43.126 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 3, 2024-04-22T13:34:43.126 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 2, 2024-04-22T13:34:43.126 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.043229990, 2024-04-22T13:34:43.126 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.014365949, 2024-04-22T13:34:43.126 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.034361242, 2024-04-22T13:34:43.126 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.004120523, 2024-04-22T13:34:43.126 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:34:43.126 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 223, 2024-04-22T13:34:43.126 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.010745583, 2024-04-22T13:34:43.126 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000048186 2024-04-22T13:34:43.127 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.127 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:34:43.127 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.127 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.127 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.127 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.127 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 1618000, 2024-04-22T13:34:43.127 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 129552 2024-04-22T13:34:43.127 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.127 INFO:teuthology.orchestra.run.smithi012.stdout: "cct": { 2024-04-22T13:34:43.127 INFO:teuthology.orchestra.run.smithi012.stdout: "total_workers": 1, 2024-04-22T13:34:43.127 INFO:teuthology.orchestra.run.smithi012.stdout: "unhealthy_workers": 0 2024-04-22T13:34:43.127 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.127 INFO:teuthology.orchestra.run.smithi012.stdout: "finisher-MDSRank": { 2024-04-22T13:34:43.127 INFO:teuthology.orchestra.run.smithi012.stdout: "queue_len": 0, 2024-04-22T13:34:43.128 INFO:teuthology.orchestra.run.smithi012.stdout: "complete_latency": { 2024-04-22T13:34:43.128 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 3446, 2024-04-22T13:34:43.128 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.106928301, 2024-04-22T13:34:43.128 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000031029 2024-04-22T13:34:43.128 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:43.128 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.128 INFO:teuthology.orchestra.run.smithi012.stdout: "finisher-PurgeQueue": { 2024-04-22T13:34:43.128 INFO:teuthology.orchestra.run.smithi012.stdout: "queue_len": 0, 2024-04-22T13:34:43.128 INFO:teuthology.orchestra.run.smithi012.stdout: "complete_latency": { 2024-04-22T13:34:43.128 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.128 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.128 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.128 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:43.128 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.128 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": { 2024-04-22T13:34:43.129 INFO:teuthology.orchestra.run.smithi012.stdout: "request": 0, 2024-04-22T13:34:43.129 INFO:teuthology.orchestra.run.smithi012.stdout: "reply": 0, 2024-04-22T13:34:43.129 INFO:teuthology.orchestra.run.smithi012.stdout: "reply_latency": { 2024-04-22T13:34:43.129 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.129 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.129 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.129 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.129 INFO:teuthology.orchestra.run.smithi012.stdout: "slow_reply": 0, 2024-04-22T13:34:43.129 INFO:teuthology.orchestra.run.smithi012.stdout: "forward": 0, 2024-04-22T13:34:43.129 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_fetch_complete": 0, 2024-04-22T13:34:43.129 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_fetch_keys": 0, 2024-04-22T13:34:43.129 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_commit": 0, 2024-04-22T13:34:43.129 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_split": 0, 2024-04-22T13:34:43.130 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_merge": 0, 2024-04-22T13:34:43.130 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes": 0, 2024-04-22T13:34:43.130 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_top": 0, 2024-04-22T13:34:43.130 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_bottom": 0, 2024-04-22T13:34:43.130 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_pin_tail": 0, 2024-04-22T13:34:43.130 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_pinned": 0, 2024-04-22T13:34:43.130 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_expired": 0, 2024-04-22T13:34:43.130 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_with_caps": 0, 2024-04-22T13:34:43.130 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": 0, 2024-04-22T13:34:43.130 INFO:teuthology.orchestra.run.smithi012.stdout: "subtrees": 1, 2024-04-22T13:34:43.130 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse": 0, 2024-04-22T13:34:43.131 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_hit": 0, 2024-04-22T13:34:43.131 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_forward": 0, 2024-04-22T13:34:43.131 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_discover": 0, 2024-04-22T13:34:43.131 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_dir_fetch": 0, 2024-04-22T13:34:43.131 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_remote_ino": 0, 2024-04-22T13:34:43.131 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_lock": 0, 2024-04-22T13:34:43.131 INFO:teuthology.orchestra.run.smithi012.stdout: "load_cent": 0, 2024-04-22T13:34:43.131 INFO:teuthology.orchestra.run.smithi012.stdout: "q": 0, 2024-04-22T13:34:43.131 INFO:teuthology.orchestra.run.smithi012.stdout: "exported": 0, 2024-04-22T13:34:43.131 INFO:teuthology.orchestra.run.smithi012.stdout: "exported_inodes": 0, 2024-04-22T13:34:43.131 INFO:teuthology.orchestra.run.smithi012.stdout: "imported": 0, 2024-04-22T13:34:43.131 INFO:teuthology.orchestra.run.smithi012.stdout: "imported_inodes": 0, 2024-04-22T13:34:43.132 INFO:teuthology.orchestra.run.smithi012.stdout: "openino_dir_fetch": 0, 2024-04-22T13:34:43.132 INFO:teuthology.orchestra.run.smithi012.stdout: "openino_backtrace_fetch": 0, 2024-04-22T13:34:43.132 INFO:teuthology.orchestra.run.smithi012.stdout: "openino_peer_discover": 0, 2024-04-22T13:34:43.132 INFO:teuthology.orchestra.run.smithi012.stdout: "root_rfiles": 0, 2024-04-22T13:34:43.132 INFO:teuthology.orchestra.run.smithi012.stdout: "root_rbytes": 0, 2024-04-22T13:34:43.132 INFO:teuthology.orchestra.run.smithi012.stdout: "root_rsnaps": 0, 2024-04-22T13:34:43.132 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_backtrace_fetch": 0, 2024-04-22T13:34:43.132 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_set_tag": 0, 2024-04-22T13:34:43.132 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_backtrace_repaired": 0, 2024-04-22T13:34:43.132 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_inotable_repaired": 0, 2024-04-22T13:34:43.132 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_dir_inodes": 0, 2024-04-22T13:34:43.132 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_dir_base_inodes": 0, 2024-04-22T13:34:43.132 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_dirfrag_rstats": 0, 2024-04-22T13:34:43.132 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_file_inodes": 0, 2024-04-22T13:34:43.132 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_inode_file_caps": 0, 2024-04-22T13:34:43.133 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_revoke": 0, 2024-04-22T13:34:43.133 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_grant": 0, 2024-04-22T13:34:43.133 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_trunc": 0, 2024-04-22T13:34:43.133 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-22T13:34:43.133 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-22T13:34:43.133 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_caps": 0, 2024-04-22T13:34:43.133 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_caps_dirty": 0, 2024-04-22T13:34:43.133 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_cap_release": 0, 2024-04-22T13:34:43.133 INFO:teuthology.orchestra.run.smithi012.stdout: "process_request_cap_release": 0 2024-04-22T13:34:43.133 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.133 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_cache": { 2024-04-22T13:34:43.133 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_update": 0, 2024-04-22T13:34:43.133 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_update_receipt": 0, 2024-04-22T13:34:43.133 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_try_discover": 0, 2024-04-22T13:34:43.133 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_send_discover": 0, 2024-04-22T13:34:43.134 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_handle_discover": 0, 2024-04-22T13:34:43.134 INFO:teuthology.orchestra.run.smithi012.stdout: "num_strays": 0, 2024-04-22T13:34:43.134 INFO:teuthology.orchestra.run.smithi012.stdout: "num_strays_delayed": 0, 2024-04-22T13:34:43.134 INFO:teuthology.orchestra.run.smithi012.stdout: "num_strays_enqueuing": 0, 2024-04-22T13:34:43.134 INFO:teuthology.orchestra.run.smithi012.stdout: "strays_created": 0, 2024-04-22T13:34:43.134 INFO:teuthology.orchestra.run.smithi012.stdout: "strays_enqueued": 0, 2024-04-22T13:34:43.134 INFO:teuthology.orchestra.run.smithi012.stdout: "strays_reintegrated": 0, 2024-04-22T13:34:43.134 INFO:teuthology.orchestra.run.smithi012.stdout: "strays_migrated": 0, 2024-04-22T13:34:43.134 INFO:teuthology.orchestra.run.smithi012.stdout: "num_recovering_processing": 0, 2024-04-22T13:34:43.134 INFO:teuthology.orchestra.run.smithi012.stdout: "num_recovering_enqueued": 0, 2024-04-22T13:34:43.134 INFO:teuthology.orchestra.run.smithi012.stdout: "num_recovering_prioritized": 0, 2024-04-22T13:34:43.134 INFO:teuthology.orchestra.run.smithi012.stdout: "recovery_started": 0, 2024-04-22T13:34:43.134 INFO:teuthology.orchestra.run.smithi012.stdout: "recovery_completed": 0, 2024-04-22T13:34:43.134 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_quiesce_path": 0, 2024-04-22T13:34:43.135 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_quiesce_inode": 0, 2024-04-22T13:34:43.135 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_enqueue_scrub": 0, 2024-04-22T13:34:43.135 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_exportdir": 0, 2024-04-22T13:34:43.135 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_flush": 0, 2024-04-22T13:34:43.135 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_fragmentdir": 0, 2024-04-22T13:34:43.135 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_fragstats": 0, 2024-04-22T13:34:43.135 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_inodestats": 0 2024-04-22T13:34:43.135 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.135 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_log": { 2024-04-22T13:34:43.135 INFO:teuthology.orchestra.run.smithi012.stdout: "evlrg": 0, 2024-04-22T13:34:43.135 INFO:teuthology.orchestra.run.smithi012.stdout: "evadd": 0, 2024-04-22T13:34:43.135 INFO:teuthology.orchestra.run.smithi012.stdout: "evex": 0, 2024-04-22T13:34:43.135 INFO:teuthology.orchestra.run.smithi012.stdout: "evtrm": 0, 2024-04-22T13:34:43.136 INFO:teuthology.orchestra.run.smithi012.stdout: "ev": 8, 2024-04-22T13:34:43.136 INFO:teuthology.orchestra.run.smithi012.stdout: "evexg": 0, 2024-04-22T13:34:43.136 INFO:teuthology.orchestra.run.smithi012.stdout: "evexd": 0, 2024-04-22T13:34:43.136 INFO:teuthology.orchestra.run.smithi012.stdout: "segadd": 0, 2024-04-22T13:34:43.136 INFO:teuthology.orchestra.run.smithi012.stdout: "segex": 0, 2024-04-22T13:34:43.136 INFO:teuthology.orchestra.run.smithi012.stdout: "segtrm": 0, 2024-04-22T13:34:43.136 INFO:teuthology.orchestra.run.smithi012.stdout: "seg": 2, 2024-04-22T13:34:43.136 INFO:teuthology.orchestra.run.smithi012.stdout: "segmjr": 2, 2024-04-22T13:34:43.136 INFO:teuthology.orchestra.run.smithi012.stdout: "segexg": 0, 2024-04-22T13:34:43.136 INFO:teuthology.orchestra.run.smithi012.stdout: "segexd": 0, 2024-04-22T13:34:43.136 INFO:teuthology.orchestra.run.smithi012.stdout: "expos": 4194304, 2024-04-22T13:34:43.136 INFO:teuthology.orchestra.run.smithi012.stdout: "wrpos": 4196655, 2024-04-22T13:34:43.136 INFO:teuthology.orchestra.run.smithi012.stdout: "rdpos": 4196514, 2024-04-22T13:34:43.136 INFO:teuthology.orchestra.run.smithi012.stdout: "jlat": { 2024-04-22T13:34:43.136 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.136 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.137 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.137 INFO:teuthology.orchestra.run.smithi012.stdout: "replayed": 8 2024-04-22T13:34:43.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.137 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_mem": { 2024-04-22T13:34:43.137 INFO:teuthology.orchestra.run.smithi012.stdout: "ino": 3, 2024-04-22T13:34:43.137 INFO:teuthology.orchestra.run.smithi012.stdout: "ino+": 3, 2024-04-22T13:34:43.137 INFO:teuthology.orchestra.run.smithi012.stdout: "ino-": 0, 2024-04-22T13:34:43.137 INFO:teuthology.orchestra.run.smithi012.stdout: "dir": 1, 2024-04-22T13:34:43.137 INFO:teuthology.orchestra.run.smithi012.stdout: "dir+": 1, 2024-04-22T13:34:43.137 INFO:teuthology.orchestra.run.smithi012.stdout: "dir-": 0, 2024-04-22T13:34:43.137 INFO:teuthology.orchestra.run.smithi012.stdout: "dn": 0, 2024-04-22T13:34:43.137 INFO:teuthology.orchestra.run.smithi012.stdout: "dn+": 0, 2024-04-22T13:34:43.137 INFO:teuthology.orchestra.run.smithi012.stdout: "dn-": 0, 2024-04-22T13:34:43.137 INFO:teuthology.orchestra.run.smithi012.stdout: "cap": 0, 2024-04-22T13:34:43.138 INFO:teuthology.orchestra.run.smithi012.stdout: "cap+": 0, 2024-04-22T13:34:43.138 INFO:teuthology.orchestra.run.smithi012.stdout: "cap-": 0, 2024-04-22T13:34:43.138 INFO:teuthology.orchestra.run.smithi012.stdout: "rss": 53032, 2024-04-22T13:34:43.138 INFO:teuthology.orchestra.run.smithi012.stdout: "heap": 215416 2024-04-22T13:34:43.138 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.138 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_server": { 2024-04-22T13:34:43.138 INFO:teuthology.orchestra.run.smithi012.stdout: "dispatch_client_request": 0, 2024-04-22T13:34:43.138 INFO:teuthology.orchestra.run.smithi012.stdout: "dispatch_server_request": 0, 2024-04-22T13:34:43.138 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_request": 0, 2024-04-22T13:34:43.138 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_session": 0, 2024-04-22T13:34:43.138 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_peer_request": 0, 2024-04-22T13:34:43.138 INFO:teuthology.orchestra.run.smithi012.stdout: "req_create_latency": { 2024-04-22T13:34:43.138 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.138 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.138 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.139 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.139 INFO:teuthology.orchestra.run.smithi012.stdout: "req_getattr_latency": { 2024-04-22T13:34:43.139 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.139 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.139 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.139 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.139 INFO:teuthology.orchestra.run.smithi012.stdout: "req_getfilelock_latency": { 2024-04-22T13:34:43.139 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.139 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.139 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.139 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.139 INFO:teuthology.orchestra.run.smithi012.stdout: "req_link_latency": { 2024-04-22T13:34:43.139 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.139 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.139 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.140 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.140 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookup_latency": { 2024-04-22T13:34:43.140 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.140 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.140 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.140 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.140 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookuphash_latency": { 2024-04-22T13:34:43.140 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.140 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.140 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.140 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.140 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookupino_latency": { 2024-04-22T13:34:43.140 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.140 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.140 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.140 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookupname_latency": { 2024-04-22T13:34:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookupparent_latency": { 2024-04-22T13:34:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookupsnap_latency": { 2024-04-22T13:34:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.142 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lssnap_latency": { 2024-04-22T13:34:43.142 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.142 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.142 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.142 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.142 INFO:teuthology.orchestra.run.smithi012.stdout: "req_mkdir_latency": { 2024-04-22T13:34:43.142 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.143 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.143 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.143 INFO:teuthology.orchestra.run.smithi012.stdout: "req_mknod_latency": { 2024-04-22T13:34:43.143 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.143 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.143 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.143 INFO:teuthology.orchestra.run.smithi012.stdout: "req_mksnap_latency": { 2024-04-22T13:34:43.143 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.143 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.143 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.144 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.144 INFO:teuthology.orchestra.run.smithi012.stdout: "req_open_latency": { 2024-04-22T13:34:43.144 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.144 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.144 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.144 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.144 INFO:teuthology.orchestra.run.smithi012.stdout: "req_readdir_latency": { 2024-04-22T13:34:43.144 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.144 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.144 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.144 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.144 INFO:teuthology.orchestra.run.smithi012.stdout: "req_rename_latency": { 2024-04-22T13:34:43.144 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.145 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.145 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.145 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.145 INFO:teuthology.orchestra.run.smithi012.stdout: "req_renamesnap_latency": { 2024-04-22T13:34:43.145 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.145 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.145 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.145 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.145 INFO:teuthology.orchestra.run.smithi012.stdout: "req_snapdiff_latency": { 2024-04-22T13:34:43.145 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.145 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.145 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.145 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.145 INFO:teuthology.orchestra.run.smithi012.stdout: "req_rmdir_latency": { 2024-04-22T13:34:43.146 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.146 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.146 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.146 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.146 INFO:teuthology.orchestra.run.smithi012.stdout: "req_rmsnap_latency": { 2024-04-22T13:34:43.146 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.146 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.146 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.146 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.146 INFO:teuthology.orchestra.run.smithi012.stdout: "req_rmxattr_latency": { 2024-04-22T13:34:43.146 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.146 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.146 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.146 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.147 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setattr_latency": { 2024-04-22T13:34:43.147 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.147 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.147 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.147 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.147 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setdirlayout_latency": { 2024-04-22T13:34:43.147 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.147 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.147 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.147 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.147 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setfilelock_latency": { 2024-04-22T13:34:43.147 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.147 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.147 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.147 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.148 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setlayout_latency": { 2024-04-22T13:34:43.148 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.148 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.148 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.148 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.148 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setxattr_latency": { 2024-04-22T13:34:43.148 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.148 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.148 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.148 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.148 INFO:teuthology.orchestra.run.smithi012.stdout: "req_symlink_latency": { 2024-04-22T13:34:43.148 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.148 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.148 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.148 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.148 INFO:teuthology.orchestra.run.smithi012.stdout: "req_unlink_latency": { 2024-04-22T13:34:43.149 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.149 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.149 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.149 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.149 INFO:teuthology.orchestra.run.smithi012.stdout: "cap_revoke_eviction": 0, 2024-04-22T13:34:43.149 INFO:teuthology.orchestra.run.smithi012.stdout: "cap_acquisition_throttle": 0, 2024-04-22T13:34:43.149 INFO:teuthology.orchestra.run.smithi012.stdout: "req_getvxattr_latency": { 2024-04-22T13:34:43.149 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.149 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.149 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.149 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:43.149 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.149 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_sessions": { 2024-04-22T13:34:43.149 INFO:teuthology.orchestra.run.smithi012.stdout: "session_count": 2, 2024-04-22T13:34:43.150 INFO:teuthology.orchestra.run.smithi012.stdout: "session_add": 4, 2024-04-22T13:34:43.150 INFO:teuthology.orchestra.run.smithi012.stdout: "session_remove": 2, 2024-04-22T13:34:43.150 INFO:teuthology.orchestra.run.smithi012.stdout: "sessions_open": 4, 2024-04-22T13:34:43.150 INFO:teuthology.orchestra.run.smithi012.stdout: "sessions_stale": 0, 2024-04-22T13:34:43.150 INFO:teuthology.orchestra.run.smithi012.stdout: "total_load": 0, 2024-04-22T13:34:43.150 INFO:teuthology.orchestra.run.smithi012.stdout: "average_load": 0, 2024-04-22T13:34:43.150 INFO:teuthology.orchestra.run.smithi012.stdout: "avg_session_uptime": 3031, 2024-04-22T13:34:43.150 INFO:teuthology.orchestra.run.smithi012.stdout: "mdthresh_evicted": 0 2024-04-22T13:34:43.150 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.150 INFO:teuthology.orchestra.run.smithi012.stdout: "mempool": { 2024-04-22T13:34:43.150 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_bytes": 0, 2024-04-22T13:34:43.150 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_items": 0, 2024-04-22T13:34:43.150 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T13:34:43.150 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_items": 0, 2024-04-22T13:34:43.151 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T13:34:43.151 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_items": 0, 2024-04-22T13:34:43.151 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T13:34:43.151 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T13:34:43.151 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T13:34:43.151 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T13:34:43.151 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T13:34:43.151 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_items": 0, 2024-04-22T13:34:43.151 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T13:34:43.151 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T13:34:43.151 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_bytes": 0, 2024-04-22T13:34:43.151 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_items": 0, 2024-04-22T13:34:43.151 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_bytes": 0, 2024-04-22T13:34:43.151 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_items": 0, 2024-04-22T13:34:43.151 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T13:34:43.152 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T13:34:43.152 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T13:34:43.152 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T13:34:43.152 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T13:34:43.152 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_items": 0, 2024-04-22T13:34:43.152 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_bytes": 0, 2024-04-22T13:34:43.152 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_items": 0, 2024-04-22T13:34:43.152 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T13:34:43.152 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T13:34:43.152 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_bytes": 0, 2024-04-22T13:34:43.152 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_items": 0, 2024-04-22T13:34:43.152 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_bytes": 0, 2024-04-22T13:34:43.152 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_items": 0, 2024-04-22T13:34:43.152 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T13:34:43.152 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_items": 0, 2024-04-22T13:34:43.152 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T13:34:43.153 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_items": 0, 2024-04-22T13:34:43.153 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_bytes": 189433, 2024-04-22T13:34:43.153 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_items": 52, 2024-04-22T13:34:43.153 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_bytes": 0, 2024-04-22T13:34:43.153 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_items": 0, 2024-04-22T13:34:43.153 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bytes": 0, 2024-04-22T13:34:43.153 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_items": 0, 2024-04-22T13:34:43.153 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_bytes": 0, 2024-04-22T13:34:43.153 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_items": 0, 2024-04-22T13:34:43.153 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_bytes": 0, 2024-04-22T13:34:43.153 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_items": 0, 2024-04-22T13:34:43.153 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_bytes": 7936, 2024-04-22T13:34:43.153 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_items": 165, 2024-04-22T13:34:43.153 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T13:34:43.153 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_items": 0, 2024-04-22T13:34:43.154 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_bytes": 0, 2024-04-22T13:34:43.154 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_items": 0, 2024-04-22T13:34:43.154 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_bytes": 8512, 2024-04-22T13:34:43.154 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_items": 13, 2024-04-22T13:34:43.154 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_bytes": 0, 2024-04-22T13:34:43.154 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_items": 0, 2024-04-22T13:34:43.154 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_bytes": 0, 2024-04-22T13:34:43.154 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_items": 0 2024-04-22T13:34:43.154 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.154 INFO:teuthology.orchestra.run.smithi012.stdout: "objecter": { 2024-04-22T13:34:43.154 INFO:teuthology.orchestra.run.smithi012.stdout: "op_active": 0, 2024-04-22T13:34:43.154 INFO:teuthology.orchestra.run.smithi012.stdout: "op_laggy": 0, 2024-04-22T13:34:43.154 INFO:teuthology.orchestra.run.smithi012.stdout: "op_send": 2149, 2024-04-22T13:34:43.154 INFO:teuthology.orchestra.run.smithi012.stdout: "op_send_bytes": 21, 2024-04-22T13:34:43.154 INFO:teuthology.orchestra.run.smithi012.stdout: "op_resend": 3, 2024-04-22T13:34:43.155 INFO:teuthology.orchestra.run.smithi012.stdout: "op_reply": 2149, 2024-04-22T13:34:43.155 INFO:teuthology.orchestra.run.smithi012.stdout: "op_latency": { 2024-04-22T13:34:43.155 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 2149, 2024-04-22T13:34:43.155 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 1.303988980, 2024-04-22T13:34:43.155 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000606788 2024-04-22T13:34:43.155 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.155 INFO:teuthology.orchestra.run.smithi012.stdout: "op_inflight": 0, 2024-04-22T13:34:43.155 INFO:teuthology.orchestra.run.smithi012.stdout: "oplen_avg": { 2024-04-22T13:34:43.155 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 2149, 2024-04-22T13:34:43.155 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 2150 2024-04-22T13:34:43.155 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.155 INFO:teuthology.orchestra.run.smithi012.stdout: "op": 2149, 2024-04-22T13:34:43.155 INFO:teuthology.orchestra.run.smithi012.stdout: "op_r": 2149, 2024-04-22T13:34:43.155 INFO:teuthology.orchestra.run.smithi012.stdout: "op_w": 0, 2024-04-22T13:34:43.156 INFO:teuthology.orchestra.run.smithi012.stdout: "op_rmw": 0, 2024-04-22T13:34:43.156 INFO:teuthology.orchestra.run.smithi012.stdout: "op_pg": 0, 2024-04-22T13:34:43.156 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_stat": 1388, 2024-04-22T13:34:43.156 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_create": 0, 2024-04-22T13:34:43.156 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_read": 759, 2024-04-22T13:34:43.156 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_write": 0, 2024-04-22T13:34:43.156 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_writefull": 0, 2024-04-22T13:34:43.156 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_writesame": 0, 2024-04-22T13:34:43.156 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_append": 0, 2024-04-22T13:34:43.156 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_zero": 0, 2024-04-22T13:34:43.156 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_truncate": 0, 2024-04-22T13:34:43.156 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_delete": 0, 2024-04-22T13:34:43.156 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_mapext": 0, 2024-04-22T13:34:43.156 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_sparse_read": 0, 2024-04-22T13:34:43.157 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_clonerange": 0, 2024-04-22T13:34:43.157 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_getxattr": 1, 2024-04-22T13:34:43.157 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_setxattr": 0, 2024-04-22T13:34:43.157 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_cmpxattr": 0, 2024-04-22T13:34:43.157 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_rmxattr": 0, 2024-04-22T13:34:43.157 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_resetxattrs": 0, 2024-04-22T13:34:43.157 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_call": 0, 2024-04-22T13:34:43.157 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_watch": 0, 2024-04-22T13:34:43.157 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_notify": 0, 2024-04-22T13:34:43.157 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_src_cmpxattr": 0, 2024-04-22T13:34:43.157 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_pgls": 0, 2024-04-22T13:34:43.157 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_pgls_filter": 0, 2024-04-22T13:34:43.157 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_other": 0, 2024-04-22T13:34:43.157 INFO:teuthology.orchestra.run.smithi012.stdout: "linger_active": 0, 2024-04-22T13:34:43.157 INFO:teuthology.orchestra.run.smithi012.stdout: "linger_send": 0, 2024-04-22T13:34:43.158 INFO:teuthology.orchestra.run.smithi012.stdout: "linger_resend": 0, 2024-04-22T13:34:43.158 INFO:teuthology.orchestra.run.smithi012.stdout: "linger_ping": 0, 2024-04-22T13:34:43.158 INFO:teuthology.orchestra.run.smithi012.stdout: "poolop_active": 0, 2024-04-22T13:34:43.158 INFO:teuthology.orchestra.run.smithi012.stdout: "poolop_send": 0, 2024-04-22T13:34:43.158 INFO:teuthology.orchestra.run.smithi012.stdout: "poolop_resend": 0, 2024-04-22T13:34:43.158 INFO:teuthology.orchestra.run.smithi012.stdout: "poolstat_active": 0, 2024-04-22T13:34:43.158 INFO:teuthology.orchestra.run.smithi012.stdout: "poolstat_send": 0, 2024-04-22T13:34:43.158 INFO:teuthology.orchestra.run.smithi012.stdout: "poolstat_resend": 0, 2024-04-22T13:34:43.158 INFO:teuthology.orchestra.run.smithi012.stdout: "statfs_active": 0, 2024-04-22T13:34:43.158 INFO:teuthology.orchestra.run.smithi012.stdout: "statfs_send": 0, 2024-04-22T13:34:43.158 INFO:teuthology.orchestra.run.smithi012.stdout: "statfs_resend": 0, 2024-04-22T13:34:43.158 INFO:teuthology.orchestra.run.smithi012.stdout: "command_active": 0, 2024-04-22T13:34:43.158 INFO:teuthology.orchestra.run.smithi012.stdout: "command_send": 0, 2024-04-22T13:34:43.158 INFO:teuthology.orchestra.run.smithi012.stdout: "command_resend": 0, 2024-04-22T13:34:43.159 INFO:teuthology.orchestra.run.smithi012.stdout: "map_epoch": 98, 2024-04-22T13:34:43.159 INFO:teuthology.orchestra.run.smithi012.stdout: "map_full": 0, 2024-04-22T13:34:43.159 INFO:teuthology.orchestra.run.smithi012.stdout: "map_inc": 11, 2024-04-22T13:34:43.159 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_sessions": 5, 2024-04-22T13:34:43.159 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_session_open": 5, 2024-04-22T13:34:43.159 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_session_close": 0, 2024-04-22T13:34:43.159 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_laggy": 0, 2024-04-22T13:34:43.159 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_wr": 0, 2024-04-22T13:34:43.159 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_rd": 2, 2024-04-22T13:34:43.159 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_del": 0 2024-04-22T13:34:43.159 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.159 INFO:teuthology.orchestra.run.smithi012.stdout: "oft": { 2024-04-22T13:34:43.159 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_total_objs": 0, 2024-04-22T13:34:43.159 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_total_kv_pairs": 0, 2024-04-22T13:34:43.159 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_total_updates": 0, 2024-04-22T13:34:43.159 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_total_removes": 0 2024-04-22T13:34:43.160 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.160 INFO:teuthology.orchestra.run.smithi012.stdout: "osd-slow-ops": { 2024-04-22T13:34:43.160 INFO:teuthology.orchestra.run.smithi012.stdout: "slow_ops_count": 0 2024-04-22T13:34:43.160 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.160 INFO:teuthology.orchestra.run.smithi012.stdout: "purge_queue": { 2024-04-22T13:34:43.160 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executing_ops": 0, 2024-04-22T13:34:43.160 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executing_ops_high_water": 0, 2024-04-22T13:34:43.160 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executing": 0, 2024-04-22T13:34:43.160 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executing_high_water": 0, 2024-04-22T13:34:43.160 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executed_ops": 0, 2024-04-22T13:34:43.160 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executed": 0, 2024-04-22T13:34:43.160 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_item_in_journal": 0 2024-04-22T13:34:43.160 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.160 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T13:34:43.160 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-04-22T13:34:43.161 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-04-22T13:34:43.161 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-04-22T13:34:43.161 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 2386, 2024-04-22T13:34:43.161 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 2002738, 2024-04-22T13:34:43.161 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-04-22T13:34:43.161 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 2386, 2024-04-22T13:34:43.161 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-04-22T13:34:43.161 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-04-22T13:34:43.161 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 2386, 2024-04-22T13:34:43.161 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 2002738, 2024-04-22T13:34:43.161 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-04-22T13:34:43.161 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.161 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.161 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.161 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:43.162 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.162 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-objecter_bytes": { 2024-04-22T13:34:43.162 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-04-22T13:34:43.162 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-04-22T13:34:43.162 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-04-22T13:34:43.162 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 0, 2024-04-22T13:34:43.162 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 0, 2024-04-22T13:34:43.162 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-04-22T13:34:43.162 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 0, 2024-04-22T13:34:43.162 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 2149, 2024-04-22T13:34:43.162 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 2356, 2024-04-22T13:34:43.162 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 2149, 2024-04-22T13:34:43.162 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 2356, 2024-04-22T13:34:43.162 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-04-22T13:34:43.163 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.163 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.163 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.163 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:43.163 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.163 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-objecter_ops": { 2024-04-22T13:34:43.163 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-04-22T13:34:43.163 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 1024, 2024-04-22T13:34:43.163 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-04-22T13:34:43.163 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 0, 2024-04-22T13:34:43.163 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 0, 2024-04-22T13:34:43.163 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-04-22T13:34:43.163 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 0, 2024-04-22T13:34:43.163 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 2149, 2024-04-22T13:34:43.163 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 2149, 2024-04-22T13:34:43.164 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 2149, 2024-04-22T13:34:43.164 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 2149, 2024-04-22T13:34:43.164 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-04-22T13:34:43.164 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.164 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.164 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.164 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:43.164 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.164 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-write_buf_throttle": { 2024-04-22T13:34:43.164 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-04-22T13:34:43.164 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 3758096384, 2024-04-22T13:34:43.164 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-04-22T13:34:43.164 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 0, 2024-04-22T13:34:43.164 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 0, 2024-04-22T13:34:43.164 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-04-22T13:34:43.165 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 0, 2024-04-22T13:34:43.165 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-04-22T13:34:43.165 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-04-22T13:34:43.165 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 0, 2024-04-22T13:34:43.165 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 0, 2024-04-22T13:34:43.165 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-04-22T13:34:43.165 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.165 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.165 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.165 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:43.165 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:43.165 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-write_buf_throttle-0x55719851e6e0": { 2024-04-22T13:34:43.165 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-04-22T13:34:43.165 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 3758096384, 2024-04-22T13:34:43.166 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-04-22T13:34:43.166 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 0, 2024-04-22T13:34:43.166 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 0, 2024-04-22T13:34:43.166 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-04-22T13:34:43.166 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 0, 2024-04-22T13:34:43.166 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-04-22T13:34:43.166 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-04-22T13:34:43.166 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 0, 2024-04-22T13:34:43.166 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 0, 2024-04-22T13:34:43.166 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-04-22T13:34:43.166 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:43.166 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:43.166 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:43.166 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:43.167 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:43.167 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:34:43.748 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:43 smithi012 ceph-mon[28265]: pgmap v329: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:34:43.896 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-04-22T13:34:43.896 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-04-22T13:34:43.896 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-04-22T13:34:43.896 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-04-22T13:34:43.897 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-04-22T13:34:43.897 DEBUG:tasks.check_counter:Getting stats from d 2024-04-22T13:34:43.897 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-22T13:34:43.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:43 smithi134 ceph-mon[31338]: pgmap v329: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:34:43.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:43 smithi111 ceph-mon[31361]: pgmap v329: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:34:44.147 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:34:45.878 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:45 smithi012 ceph-mon[28265]: pgmap v330: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:45.879 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:34:45.879 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T13:34:45.879 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 0, 2024-04-22T13:34:45.879 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 2, 2024-04-22T13:34:45.879 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 0, 2024-04-22T13:34:45.879 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 0, 2024-04-22T13:34:45.879 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 4, 2024-04-22T13:34:45.879 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 0, 2024-04-22T13:34:45.879 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.001490728, 2024-04-22T13:34:45.879 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.000000000, 2024-04-22T13:34:45.880 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.000390608, 2024-04-22T13:34:45.880 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-22T13:34:45.880 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:34:45.880 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:45.880 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:45.880 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:45.880 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:45.880 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:34:45.880 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:45.880 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:45.880 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:45.880 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:45.880 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-22T13:34:45.881 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 0 2024-04-22T13:34:45.881 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:45.881 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T13:34:45.881 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 2, 2024-04-22T13:34:45.881 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 162, 2024-04-22T13:34:45.881 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 320, 2024-04-22T13:34:45.881 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 184624, 2024-04-22T13:34:45.881 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 5, 2024-04-22T13:34:45.881 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 3, 2024-04-22T13:34:45.881 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.014006412, 2024-04-22T13:34:45.881 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.009606863, 2024-04-22T13:34:45.881 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.001838741, 2024-04-22T13:34:45.881 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-22T13:34:45.881 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:34:45.881 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 159, 2024-04-22T13:34:45.882 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.014079127, 2024-04-22T13:34:45.882 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000088547 2024-04-22T13:34:45.882 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:45.882 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:34:45.882 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:45.882 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:45.882 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:45.882 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:45.882 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-22T13:34:45.882 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 184624 2024-04-22T13:34:45.883 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:45.883 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T13:34:45.883 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 228, 2024-04-22T13:34:45.883 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 215, 2024-04-22T13:34:45.883 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 1593472, 2024-04-22T13:34:45.883 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 131072, 2024-04-22T13:34:45.883 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 2, 2024-04-22T13:34:45.883 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 1, 2024-04-22T13:34:45.883 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.046340332, 2024-04-22T13:34:45.883 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.013899133, 2024-04-22T13:34:45.883 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.037489051, 2024-04-22T13:34:45.883 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.004173707, 2024-04-22T13:34:45.883 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:34:45.883 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 215, 2024-04-22T13:34:45.883 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.010709767, 2024-04-22T13:34:45.883 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000049812 2024-04-22T13:34:45.884 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:45.884 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:34:45.884 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:45.884 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:45.884 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:45.884 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:45.884 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 1593472, 2024-04-22T13:34:45.884 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 131072 2024-04-22T13:34:45.884 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:45.884 INFO:teuthology.orchestra.run.smithi012.stdout: "cct": { 2024-04-22T13:34:45.884 INFO:teuthology.orchestra.run.smithi012.stdout: "total_workers": 0, 2024-04-22T13:34:45.884 INFO:teuthology.orchestra.run.smithi012.stdout: "unhealthy_workers": 0 2024-04-22T13:34:45.884 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:45.884 INFO:teuthology.orchestra.run.smithi012.stdout: "mempool": { 2024-04-22T13:34:45.884 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_bytes": 0, 2024-04-22T13:34:45.885 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_items": 0, 2024-04-22T13:34:45.885 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T13:34:45.885 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_items": 0, 2024-04-22T13:34:45.885 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T13:34:45.885 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_items": 0, 2024-04-22T13:34:45.885 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T13:34:45.885 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T13:34:45.885 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T13:34:45.885 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T13:34:45.885 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T13:34:45.885 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_items": 0, 2024-04-22T13:34:45.885 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T13:34:45.885 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T13:34:45.885 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_bytes": 0, 2024-04-22T13:34:45.885 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_items": 0, 2024-04-22T13:34:45.885 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_bytes": 0, 2024-04-22T13:34:45.886 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_items": 0, 2024-04-22T13:34:45.886 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T13:34:45.886 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T13:34:45.886 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T13:34:45.886 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T13:34:45.886 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T13:34:45.886 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_items": 0, 2024-04-22T13:34:45.886 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_bytes": 0, 2024-04-22T13:34:45.886 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_items": 0, 2024-04-22T13:34:45.886 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T13:34:45.886 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T13:34:45.886 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_bytes": 0, 2024-04-22T13:34:45.886 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_items": 0, 2024-04-22T13:34:45.886 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_bytes": 0, 2024-04-22T13:34:45.887 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_items": 0, 2024-04-22T13:34:45.887 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T13:34:45.887 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_items": 0, 2024-04-22T13:34:45.887 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T13:34:45.887 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_items": 0, 2024-04-22T13:34:45.887 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_bytes": 148994, 2024-04-22T13:34:45.887 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_items": 31, 2024-04-22T13:34:45.887 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_bytes": 0, 2024-04-22T13:34:45.887 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_items": 0, 2024-04-22T13:34:45.887 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bytes": 0, 2024-04-22T13:34:45.887 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_items": 0, 2024-04-22T13:34:45.887 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_bytes": 0, 2024-04-22T13:34:45.887 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_items": 0, 2024-04-22T13:34:45.887 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_bytes": 0, 2024-04-22T13:34:45.887 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_items": 0, 2024-04-22T13:34:45.887 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_bytes": 0, 2024-04-22T13:34:45.887 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_items": 0, 2024-04-22T13:34:45.888 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T13:34:45.888 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_items": 0, 2024-04-22T13:34:45.888 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_bytes": 0, 2024-04-22T13:34:45.888 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_items": 0, 2024-04-22T13:34:45.888 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_bytes": 928, 2024-04-22T13:34:45.888 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_items": 2, 2024-04-22T13:34:45.888 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_bytes": 0, 2024-04-22T13:34:45.888 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_items": 0, 2024-04-22T13:34:45.888 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_bytes": 0, 2024-04-22T13:34:45.888 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_items": 0 2024-04-22T13:34:45.888 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:45.888 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T13:34:45.888 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-04-22T13:34:45.888 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-04-22T13:34:45.888 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-04-22T13:34:45.888 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 226, 2024-04-22T13:34:45.889 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 1558884, 2024-04-22T13:34:45.889 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-04-22T13:34:45.889 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 226, 2024-04-22T13:34:45.889 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-04-22T13:34:45.889 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-04-22T13:34:45.889 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 226, 2024-04-22T13:34:45.889 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 1558884, 2024-04-22T13:34:45.889 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-04-22T13:34:45.889 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:45.889 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:45.889 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:45.889 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:45.890 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:45.890 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:34:45.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:45 smithi134 ceph-mon[31338]: pgmap v330: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:45.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:45 smithi111 ceph-mon[31361]: pgmap v330: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:46.557 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-22T13:34:46.558 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-22T13:34:46.558 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-22T13:34:46.558 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-22T13:34:46.558 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-22T13:34:46.558 DEBUG:tasks.check_counter:Getting stats from g 2024-04-22T13:34:46.558 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-22T13:34:46.814 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:34:47.856 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:47 smithi012 ceph-mon[28265]: pgmap v331: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:47.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:47 smithi134 ceph-mon[31338]: pgmap v331: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:47.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:47 smithi111 ceph-mon[31361]: pgmap v331: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:48.398 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:34:48.398 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T13:34:48.398 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 996, 2024-04-22T13:34:48.398 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 982, 2024-04-22T13:34:48.398 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 1875216, 2024-04-22T13:34:48.398 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 371370, 2024-04-22T13:34:48.398 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 5, 2024-04-22T13:34:48.399 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 1, 2024-04-22T13:34:48.399 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.132818628, 2024-04-22T13:34:48.399 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.051269365, 2024-04-22T13:34:48.399 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.054826986, 2024-04-22T13:34:48.399 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.027023337, 2024-04-22T13:34:48.399 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:34:48.399 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 982, 2024-04-22T13:34:48.399 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.051648471, 2024-04-22T13:34:48.399 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000052595 2024-04-22T13:34:48.399 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.399 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:34:48.399 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.399 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.400 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.400 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 1621536, 2024-04-22T13:34:48.400 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 131280 2024-04-22T13:34:48.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.400 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T13:34:48.400 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 1402, 2024-04-22T13:34:48.400 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 1559, 2024-04-22T13:34:48.400 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 358516, 2024-04-22T13:34:48.400 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 890313, 2024-04-22T13:34:48.400 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 7, 2024-04-22T13:34:48.400 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 5, 2024-04-22T13:34:48.400 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.103974792, 2024-04-22T13:34:48.400 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.034625151, 2024-04-22T13:34:48.401 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.025303949, 2024-04-22T13:34:48.401 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.034833749, 2024-04-22T13:34:48.401 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:34:48.401 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 1556, 2024-04-22T13:34:48.401 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.056245734, 2024-04-22T13:34:48.401 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000036147 2024-04-22T13:34:48.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.401 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:34:48.401 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.401 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.401 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.401 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-22T13:34:48.401 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 445056 2024-04-22T13:34:48.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.402 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T13:34:48.402 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 4, 2024-04-22T13:34:48.402 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 6, 2024-04-22T13:34:48.402 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 1629, 2024-04-22T13:34:48.402 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 1284, 2024-04-22T13:34:48.402 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 5, 2024-04-22T13:34:48.402 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 4, 2024-04-22T13:34:48.402 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.001335082, 2024-04-22T13:34:48.402 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.000064961, 2024-04-22T13:34:48.402 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.001105076, 2024-04-22T13:34:48.402 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000071495, 2024-04-22T13:34:48.402 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:34:48.402 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 4, 2024-04-22T13:34:48.402 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.002289267, 2024-04-22T13:34:48.403 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000572316 2024-04-22T13:34:48.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.403 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:34:48.403 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.403 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.403 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.403 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-22T13:34:48.403 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 0 2024-04-22T13:34:48.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.403 INFO:teuthology.orchestra.run.smithi012.stdout: "cct": { 2024-04-22T13:34:48.403 INFO:teuthology.orchestra.run.smithi012.stdout: "total_workers": 1, 2024-04-22T13:34:48.403 INFO:teuthology.orchestra.run.smithi012.stdout: "unhealthy_workers": 0 2024-04-22T13:34:48.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.403 INFO:teuthology.orchestra.run.smithi012.stdout: "finisher-MDSRank": { 2024-04-22T13:34:48.403 INFO:teuthology.orchestra.run.smithi012.stdout: "queue_len": 0, 2024-04-22T13:34:48.404 INFO:teuthology.orchestra.run.smithi012.stdout: "complete_latency": { 2024-04-22T13:34:48.404 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 3665, 2024-04-22T13:34:48.404 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.091690141, 2024-04-22T13:34:48.404 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000025017 2024-04-22T13:34:48.404 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:48.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.404 INFO:teuthology.orchestra.run.smithi012.stdout: "finisher-PurgeQueue": { 2024-04-22T13:34:48.404 INFO:teuthology.orchestra.run.smithi012.stdout: "queue_len": 0, 2024-04-22T13:34:48.404 INFO:teuthology.orchestra.run.smithi012.stdout: "complete_latency": { 2024-04-22T13:34:48.404 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.404 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.404 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.404 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:48.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.404 INFO:teuthology.orchestra.run.smithi012.stdout: "mds": { 2024-04-22T13:34:48.405 INFO:teuthology.orchestra.run.smithi012.stdout: "request": 0, 2024-04-22T13:34:48.405 INFO:teuthology.orchestra.run.smithi012.stdout: "reply": 0, 2024-04-22T13:34:48.405 INFO:teuthology.orchestra.run.smithi012.stdout: "reply_latency": { 2024-04-22T13:34:48.405 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.405 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.405 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.405 INFO:teuthology.orchestra.run.smithi012.stdout: "slow_reply": 0, 2024-04-22T13:34:48.405 INFO:teuthology.orchestra.run.smithi012.stdout: "forward": 0, 2024-04-22T13:34:48.405 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_fetch_complete": 0, 2024-04-22T13:34:48.405 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_fetch_keys": 0, 2024-04-22T13:34:48.405 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_commit": 0, 2024-04-22T13:34:48.405 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_split": 0, 2024-04-22T13:34:48.405 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_merge": 0, 2024-04-22T13:34:48.405 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes": 0, 2024-04-22T13:34:48.405 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_top": 0, 2024-04-22T13:34:48.406 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_bottom": 0, 2024-04-22T13:34:48.406 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_pin_tail": 0, 2024-04-22T13:34:48.406 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_pinned": 0, 2024-04-22T13:34:48.406 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_expired": 0, 2024-04-22T13:34:48.406 INFO:teuthology.orchestra.run.smithi012.stdout: "inodes_with_caps": 0, 2024-04-22T13:34:48.406 INFO:teuthology.orchestra.run.smithi012.stdout: "caps": 0, 2024-04-22T13:34:48.406 INFO:teuthology.orchestra.run.smithi012.stdout: "subtrees": 1, 2024-04-22T13:34:48.406 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse": 0, 2024-04-22T13:34:48.406 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_hit": 0, 2024-04-22T13:34:48.406 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_forward": 0, 2024-04-22T13:34:48.406 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_discover": 0, 2024-04-22T13:34:48.406 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_dir_fetch": 0, 2024-04-22T13:34:48.406 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_remote_ino": 0, 2024-04-22T13:34:48.406 INFO:teuthology.orchestra.run.smithi012.stdout: "traverse_lock": 0, 2024-04-22T13:34:48.407 INFO:teuthology.orchestra.run.smithi012.stdout: "load_cent": 0, 2024-04-22T13:34:48.407 INFO:teuthology.orchestra.run.smithi012.stdout: "q": 0, 2024-04-22T13:34:48.407 INFO:teuthology.orchestra.run.smithi012.stdout: "exported": 0, 2024-04-22T13:34:48.407 INFO:teuthology.orchestra.run.smithi012.stdout: "exported_inodes": 0, 2024-04-22T13:34:48.407 INFO:teuthology.orchestra.run.smithi012.stdout: "imported": 0, 2024-04-22T13:34:48.407 INFO:teuthology.orchestra.run.smithi012.stdout: "imported_inodes": 0, 2024-04-22T13:34:48.407 INFO:teuthology.orchestra.run.smithi012.stdout: "openino_dir_fetch": 0, 2024-04-22T13:34:48.407 INFO:teuthology.orchestra.run.smithi012.stdout: "openino_backtrace_fetch": 0, 2024-04-22T13:34:48.407 INFO:teuthology.orchestra.run.smithi012.stdout: "openino_peer_discover": 0, 2024-04-22T13:34:48.407 INFO:teuthology.orchestra.run.smithi012.stdout: "root_rfiles": 0, 2024-04-22T13:34:48.407 INFO:teuthology.orchestra.run.smithi012.stdout: "root_rbytes": 0, 2024-04-22T13:34:48.407 INFO:teuthology.orchestra.run.smithi012.stdout: "root_rsnaps": 0, 2024-04-22T13:34:48.407 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_backtrace_fetch": 0, 2024-04-22T13:34:48.407 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_set_tag": 0, 2024-04-22T13:34:48.408 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_backtrace_repaired": 0, 2024-04-22T13:34:48.408 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_inotable_repaired": 0, 2024-04-22T13:34:48.408 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_dir_inodes": 0, 2024-04-22T13:34:48.408 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_dir_base_inodes": 0, 2024-04-22T13:34:48.408 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_dirfrag_rstats": 0, 2024-04-22T13:34:48.408 INFO:teuthology.orchestra.run.smithi012.stdout: "scrub_file_inodes": 0, 2024-04-22T13:34:48.408 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_inode_file_caps": 0, 2024-04-22T13:34:48.408 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_revoke": 0, 2024-04-22T13:34:48.408 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_grant": 0, 2024-04-22T13:34:48.408 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_trunc": 0, 2024-04-22T13:34:48.408 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-22T13:34:48.408 INFO:teuthology.orchestra.run.smithi012.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-22T13:34:48.408 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_caps": 0, 2024-04-22T13:34:48.408 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_caps_dirty": 0, 2024-04-22T13:34:48.408 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_cap_release": 0, 2024-04-22T13:34:48.408 INFO:teuthology.orchestra.run.smithi012.stdout: "process_request_cap_release": 0 2024-04-22T13:34:48.409 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.409 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_cache": { 2024-04-22T13:34:48.409 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_update": 0, 2024-04-22T13:34:48.409 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_update_receipt": 0, 2024-04-22T13:34:48.409 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_try_discover": 0, 2024-04-22T13:34:48.409 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_send_discover": 0, 2024-04-22T13:34:48.409 INFO:teuthology.orchestra.run.smithi012.stdout: "dir_handle_discover": 0, 2024-04-22T13:34:48.409 INFO:teuthology.orchestra.run.smithi012.stdout: "num_strays": 0, 2024-04-22T13:34:48.409 INFO:teuthology.orchestra.run.smithi012.stdout: "num_strays_delayed": 0, 2024-04-22T13:34:48.409 INFO:teuthology.orchestra.run.smithi012.stdout: "num_strays_enqueuing": 0, 2024-04-22T13:34:48.409 INFO:teuthology.orchestra.run.smithi012.stdout: "strays_created": 0, 2024-04-22T13:34:48.409 INFO:teuthology.orchestra.run.smithi012.stdout: "strays_enqueued": 0, 2024-04-22T13:34:48.409 INFO:teuthology.orchestra.run.smithi012.stdout: "strays_reintegrated": 0, 2024-04-22T13:34:48.409 INFO:teuthology.orchestra.run.smithi012.stdout: "strays_migrated": 0, 2024-04-22T13:34:48.409 INFO:teuthology.orchestra.run.smithi012.stdout: "num_recovering_processing": 0, 2024-04-22T13:34:48.410 INFO:teuthology.orchestra.run.smithi012.stdout: "num_recovering_enqueued": 0, 2024-04-22T13:34:48.410 INFO:teuthology.orchestra.run.smithi012.stdout: "num_recovering_prioritized": 0, 2024-04-22T13:34:48.410 INFO:teuthology.orchestra.run.smithi012.stdout: "recovery_started": 0, 2024-04-22T13:34:48.410 INFO:teuthology.orchestra.run.smithi012.stdout: "recovery_completed": 0, 2024-04-22T13:34:48.410 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_quiesce_path": 0, 2024-04-22T13:34:48.410 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_quiesce_inode": 0, 2024-04-22T13:34:48.410 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_enqueue_scrub": 0, 2024-04-22T13:34:48.410 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_exportdir": 0, 2024-04-22T13:34:48.410 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_flush": 0, 2024-04-22T13:34:48.410 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_fragmentdir": 0, 2024-04-22T13:34:48.410 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_fragstats": 0, 2024-04-22T13:34:48.410 INFO:teuthology.orchestra.run.smithi012.stdout: "ireq_inodestats": 0 2024-04-22T13:34:48.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.410 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_log": { 2024-04-22T13:34:48.410 INFO:teuthology.orchestra.run.smithi012.stdout: "evlrg": 0, 2024-04-22T13:34:48.411 INFO:teuthology.orchestra.run.smithi012.stdout: "evadd": 0, 2024-04-22T13:34:48.411 INFO:teuthology.orchestra.run.smithi012.stdout: "evex": 0, 2024-04-22T13:34:48.411 INFO:teuthology.orchestra.run.smithi012.stdout: "evtrm": 0, 2024-04-22T13:34:48.411 INFO:teuthology.orchestra.run.smithi012.stdout: "ev": 6, 2024-04-22T13:34:48.411 INFO:teuthology.orchestra.run.smithi012.stdout: "evexg": 0, 2024-04-22T13:34:48.411 INFO:teuthology.orchestra.run.smithi012.stdout: "evexd": 0, 2024-04-22T13:34:48.411 INFO:teuthology.orchestra.run.smithi012.stdout: "segadd": 0, 2024-04-22T13:34:48.411 INFO:teuthology.orchestra.run.smithi012.stdout: "segex": 0, 2024-04-22T13:34:48.411 INFO:teuthology.orchestra.run.smithi012.stdout: "segtrm": 0, 2024-04-22T13:34:48.411 INFO:teuthology.orchestra.run.smithi012.stdout: "seg": 2, 2024-04-22T13:34:48.411 INFO:teuthology.orchestra.run.smithi012.stdout: "segmjr": 2, 2024-04-22T13:34:48.411 INFO:teuthology.orchestra.run.smithi012.stdout: "segexg": 0, 2024-04-22T13:34:48.411 INFO:teuthology.orchestra.run.smithi012.stdout: "segexd": 0, 2024-04-22T13:34:48.411 INFO:teuthology.orchestra.run.smithi012.stdout: "expos": 4194304, 2024-04-22T13:34:48.411 INFO:teuthology.orchestra.run.smithi012.stdout: "wrpos": 4195956, 2024-04-22T13:34:48.412 INFO:teuthology.orchestra.run.smithi012.stdout: "rdpos": 4195815, 2024-04-22T13:34:48.412 INFO:teuthology.orchestra.run.smithi012.stdout: "jlat": { 2024-04-22T13:34:48.412 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.412 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.412 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.412 INFO:teuthology.orchestra.run.smithi012.stdout: "replayed": 6 2024-04-22T13:34:48.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.412 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_mem": { 2024-04-22T13:34:48.412 INFO:teuthology.orchestra.run.smithi012.stdout: "ino": 3, 2024-04-22T13:34:48.412 INFO:teuthology.orchestra.run.smithi012.stdout: "ino+": 3, 2024-04-22T13:34:48.412 INFO:teuthology.orchestra.run.smithi012.stdout: "ino-": 0, 2024-04-22T13:34:48.412 INFO:teuthology.orchestra.run.smithi012.stdout: "dir": 1, 2024-04-22T13:34:48.412 INFO:teuthology.orchestra.run.smithi012.stdout: "dir+": 1, 2024-04-22T13:34:48.413 INFO:teuthology.orchestra.run.smithi012.stdout: "dir-": 0, 2024-04-22T13:34:48.413 INFO:teuthology.orchestra.run.smithi012.stdout: "dn": 0, 2024-04-22T13:34:48.413 INFO:teuthology.orchestra.run.smithi012.stdout: "dn+": 0, 2024-04-22T13:34:48.413 INFO:teuthology.orchestra.run.smithi012.stdout: "dn-": 0, 2024-04-22T13:34:48.413 INFO:teuthology.orchestra.run.smithi012.stdout: "cap": 0, 2024-04-22T13:34:48.413 INFO:teuthology.orchestra.run.smithi012.stdout: "cap+": 0, 2024-04-22T13:34:48.413 INFO:teuthology.orchestra.run.smithi012.stdout: "cap-": 0, 2024-04-22T13:34:48.413 INFO:teuthology.orchestra.run.smithi012.stdout: "rss": 51104, 2024-04-22T13:34:48.413 INFO:teuthology.orchestra.run.smithi012.stdout: "heap": 215416 2024-04-22T13:34:48.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.413 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_server": { 2024-04-22T13:34:48.413 INFO:teuthology.orchestra.run.smithi012.stdout: "dispatch_client_request": 0, 2024-04-22T13:34:48.413 INFO:teuthology.orchestra.run.smithi012.stdout: "dispatch_server_request": 0, 2024-04-22T13:34:48.413 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_request": 0, 2024-04-22T13:34:48.413 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_client_session": 0, 2024-04-22T13:34:48.413 INFO:teuthology.orchestra.run.smithi012.stdout: "handle_peer_request": 0, 2024-04-22T13:34:48.414 INFO:teuthology.orchestra.run.smithi012.stdout: "req_create_latency": { 2024-04-22T13:34:48.414 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.414 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.414 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.414 INFO:teuthology.orchestra.run.smithi012.stdout: "req_getattr_latency": { 2024-04-22T13:34:48.414 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.414 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.414 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.414 INFO:teuthology.orchestra.run.smithi012.stdout: "req_getfilelock_latency": { 2024-04-22T13:34:48.414 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.414 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.414 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.415 INFO:teuthology.orchestra.run.smithi012.stdout: "req_link_latency": { 2024-04-22T13:34:48.415 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.415 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.415 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.415 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookup_latency": { 2024-04-22T13:34:48.415 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.415 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.415 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.415 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookuphash_latency": { 2024-04-22T13:34:48.415 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.415 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.416 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.416 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookupino_latency": { 2024-04-22T13:34:48.416 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.416 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.416 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.416 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookupname_latency": { 2024-04-22T13:34:48.416 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.416 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.416 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.416 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookupparent_latency": { 2024-04-22T13:34:48.416 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.417 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.418 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.418 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lookupsnap_latency": { 2024-04-22T13:34:48.418 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.418 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.418 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.418 INFO:teuthology.orchestra.run.smithi012.stdout: "req_lssnap_latency": { 2024-04-22T13:34:48.418 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.419 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.419 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.419 INFO:teuthology.orchestra.run.smithi012.stdout: "req_mkdir_latency": { 2024-04-22T13:34:48.419 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.419 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.420 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.420 INFO:teuthology.orchestra.run.smithi012.stdout: "req_mknod_latency": { 2024-04-22T13:34:48.420 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.420 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.420 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.420 INFO:teuthology.orchestra.run.smithi012.stdout: "req_mksnap_latency": { 2024-04-22T13:34:48.420 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.420 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.420 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.420 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.420 INFO:teuthology.orchestra.run.smithi012.stdout: "req_open_latency": { 2024-04-22T13:34:48.420 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.421 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.421 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.421 INFO:teuthology.orchestra.run.smithi012.stdout: "req_readdir_latency": { 2024-04-22T13:34:48.421 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.421 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.421 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.421 INFO:teuthology.orchestra.run.smithi012.stdout: "req_rename_latency": { 2024-04-22T13:34:48.421 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.421 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.421 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.421 INFO:teuthology.orchestra.run.smithi012.stdout: "req_renamesnap_latency": { 2024-04-22T13:34:48.421 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.422 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.422 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.422 INFO:teuthology.orchestra.run.smithi012.stdout: "req_snapdiff_latency": { 2024-04-22T13:34:48.422 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.422 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.422 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.422 INFO:teuthology.orchestra.run.smithi012.stdout: "req_rmdir_latency": { 2024-04-22T13:34:48.422 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.422 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.422 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.422 INFO:teuthology.orchestra.run.smithi012.stdout: "req_rmsnap_latency": { 2024-04-22T13:34:48.423 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.423 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.423 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.423 INFO:teuthology.orchestra.run.smithi012.stdout: "req_rmxattr_latency": { 2024-04-22T13:34:48.423 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.423 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.423 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.423 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setattr_latency": { 2024-04-22T13:34:48.423 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.423 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.423 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.423 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setdirlayout_latency": { 2024-04-22T13:34:48.424 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.424 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.424 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.424 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setfilelock_latency": { 2024-04-22T13:34:48.424 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.424 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.424 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.424 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setlayout_latency": { 2024-04-22T13:34:48.425 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.425 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.425 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.425 INFO:teuthology.orchestra.run.smithi012.stdout: "req_setxattr_latency": { 2024-04-22T13:34:48.425 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.425 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.425 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.425 INFO:teuthology.orchestra.run.smithi012.stdout: "req_symlink_latency": { 2024-04-22T13:34:48.425 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.425 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.425 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.425 INFO:teuthology.orchestra.run.smithi012.stdout: "req_unlink_latency": { 2024-04-22T13:34:48.426 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.426 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.426 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.426 INFO:teuthology.orchestra.run.smithi012.stdout: "cap_revoke_eviction": 0, 2024-04-22T13:34:48.426 INFO:teuthology.orchestra.run.smithi012.stdout: "cap_acquisition_throttle": 0, 2024-04-22T13:34:48.426 INFO:teuthology.orchestra.run.smithi012.stdout: "req_getvxattr_latency": { 2024-04-22T13:34:48.426 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.426 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.426 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.426 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:48.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.427 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_sessions": { 2024-04-22T13:34:48.427 INFO:teuthology.orchestra.run.smithi012.stdout: "session_count": 0, 2024-04-22T13:34:48.427 INFO:teuthology.orchestra.run.smithi012.stdout: "session_add": 2, 2024-04-22T13:34:48.427 INFO:teuthology.orchestra.run.smithi012.stdout: "session_remove": 2, 2024-04-22T13:34:48.427 INFO:teuthology.orchestra.run.smithi012.stdout: "sessions_open": 2, 2024-04-22T13:34:48.427 INFO:teuthology.orchestra.run.smithi012.stdout: "sessions_stale": 0, 2024-04-22T13:34:48.427 INFO:teuthology.orchestra.run.smithi012.stdout: "total_load": 0, 2024-04-22T13:34:48.427 INFO:teuthology.orchestra.run.smithi012.stdout: "average_load": 0, 2024-04-22T13:34:48.427 INFO:teuthology.orchestra.run.smithi012.stdout: "avg_session_uptime": 1285, 2024-04-22T13:34:48.427 INFO:teuthology.orchestra.run.smithi012.stdout: "mdthresh_evicted": 0 2024-04-22T13:34:48.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.427 INFO:teuthology.orchestra.run.smithi012.stdout: "mempool": { 2024-04-22T13:34:48.428 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_bytes": 0, 2024-04-22T13:34:48.428 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_items": 0, 2024-04-22T13:34:48.428 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T13:34:48.428 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_items": 0, 2024-04-22T13:34:48.428 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T13:34:48.428 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_items": 0, 2024-04-22T13:34:48.428 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T13:34:48.428 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T13:34:48.428 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T13:34:48.428 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T13:34:48.428 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T13:34:48.428 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_items": 0, 2024-04-22T13:34:48.428 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T13:34:48.428 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T13:34:48.429 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_bytes": 0, 2024-04-22T13:34:48.429 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_items": 0, 2024-04-22T13:34:48.429 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_bytes": 0, 2024-04-22T13:34:48.429 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_items": 0, 2024-04-22T13:34:48.429 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T13:34:48.429 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T13:34:48.429 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T13:34:48.429 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T13:34:48.429 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T13:34:48.429 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_items": 0, 2024-04-22T13:34:48.429 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_bytes": 0, 2024-04-22T13:34:48.429 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_items": 0, 2024-04-22T13:34:48.429 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T13:34:48.429 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T13:34:48.430 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_bytes": 0, 2024-04-22T13:34:48.430 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_items": 0, 2024-04-22T13:34:48.430 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_bytes": 0, 2024-04-22T13:34:48.430 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_items": 0, 2024-04-22T13:34:48.430 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T13:34:48.430 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_items": 0, 2024-04-22T13:34:48.430 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T13:34:48.430 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_items": 0, 2024-04-22T13:34:48.430 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_bytes": 197530, 2024-04-22T13:34:48.430 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_items": 57, 2024-04-22T13:34:48.430 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_bytes": 0, 2024-04-22T13:34:48.430 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_items": 0, 2024-04-22T13:34:48.430 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bytes": 0, 2024-04-22T13:34:48.430 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_items": 0, 2024-04-22T13:34:48.431 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_bytes": 0, 2024-04-22T13:34:48.431 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_items": 0, 2024-04-22T13:34:48.431 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_bytes": 0, 2024-04-22T13:34:48.431 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_items": 0, 2024-04-22T13:34:48.431 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_bytes": 7936, 2024-04-22T13:34:48.431 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_items": 165, 2024-04-22T13:34:48.431 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T13:34:48.431 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_items": 0, 2024-04-22T13:34:48.431 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_bytes": 0, 2024-04-22T13:34:48.431 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_items": 0, 2024-04-22T13:34:48.431 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_bytes": 8512, 2024-04-22T13:34:48.431 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_items": 13, 2024-04-22T13:34:48.431 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_bytes": 0, 2024-04-22T13:34:48.431 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_items": 0, 2024-04-22T13:34:48.431 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_bytes": 0, 2024-04-22T13:34:48.431 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_items": 0 2024-04-22T13:34:48.432 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.432 INFO:teuthology.orchestra.run.smithi012.stdout: "objecter": { 2024-04-22T13:34:48.432 INFO:teuthology.orchestra.run.smithi012.stdout: "op_active": 0, 2024-04-22T13:34:48.432 INFO:teuthology.orchestra.run.smithi012.stdout: "op_laggy": 0, 2024-04-22T13:34:48.432 INFO:teuthology.orchestra.run.smithi012.stdout: "op_send": 2159, 2024-04-22T13:34:48.432 INFO:teuthology.orchestra.run.smithi012.stdout: "op_send_bytes": 21, 2024-04-22T13:34:48.433 INFO:teuthology.orchestra.run.smithi012.stdout: "op_resend": 3, 2024-04-22T13:34:48.433 INFO:teuthology.orchestra.run.smithi012.stdout: "op_reply": 2159, 2024-04-22T13:34:48.433 INFO:teuthology.orchestra.run.smithi012.stdout: "op_latency": { 2024-04-22T13:34:48.433 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 2159, 2024-04-22T13:34:48.433 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 1.119990550, 2024-04-22T13:34:48.433 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000518754 2024-04-22T13:34:48.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.433 INFO:teuthology.orchestra.run.smithi012.stdout: "op_inflight": 0, 2024-04-22T13:34:48.433 INFO:teuthology.orchestra.run.smithi012.stdout: "oplen_avg": { 2024-04-22T13:34:48.433 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 2159, 2024-04-22T13:34:48.434 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 2160 2024-04-22T13:34:48.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.434 INFO:teuthology.orchestra.run.smithi012.stdout: "op": 2159, 2024-04-22T13:34:48.434 INFO:teuthology.orchestra.run.smithi012.stdout: "op_r": 2159, 2024-04-22T13:34:48.434 INFO:teuthology.orchestra.run.smithi012.stdout: "op_w": 0, 2024-04-22T13:34:48.434 INFO:teuthology.orchestra.run.smithi012.stdout: "op_rmw": 0, 2024-04-22T13:34:48.434 INFO:teuthology.orchestra.run.smithi012.stdout: "op_pg": 0, 2024-04-22T13:34:48.434 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_stat": 1396, 2024-04-22T13:34:48.434 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_create": 0, 2024-04-22T13:34:48.434 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_read": 761, 2024-04-22T13:34:48.434 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_write": 0, 2024-04-22T13:34:48.434 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_writefull": 0, 2024-04-22T13:34:48.434 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_writesame": 0, 2024-04-22T13:34:48.434 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_append": 0, 2024-04-22T13:34:48.434 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_zero": 0, 2024-04-22T13:34:48.435 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_truncate": 0, 2024-04-22T13:34:48.435 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_delete": 0, 2024-04-22T13:34:48.435 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_mapext": 0, 2024-04-22T13:34:48.435 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_sparse_read": 0, 2024-04-22T13:34:48.435 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_clonerange": 0, 2024-04-22T13:34:48.435 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_getxattr": 1, 2024-04-22T13:34:48.435 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_setxattr": 0, 2024-04-22T13:34:48.435 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_cmpxattr": 0, 2024-04-22T13:34:48.435 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_rmxattr": 0, 2024-04-22T13:34:48.435 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_resetxattrs": 0, 2024-04-22T13:34:48.435 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_call": 0, 2024-04-22T13:34:48.435 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_watch": 0, 2024-04-22T13:34:48.435 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_notify": 0, 2024-04-22T13:34:48.435 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_src_cmpxattr": 0, 2024-04-22T13:34:48.436 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_pgls": 0, 2024-04-22T13:34:48.436 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_pgls_filter": 0, 2024-04-22T13:34:48.436 INFO:teuthology.orchestra.run.smithi012.stdout: "osdop_other": 0, 2024-04-22T13:34:48.436 INFO:teuthology.orchestra.run.smithi012.stdout: "linger_active": 0, 2024-04-22T13:34:48.436 INFO:teuthology.orchestra.run.smithi012.stdout: "linger_send": 0, 2024-04-22T13:34:48.436 INFO:teuthology.orchestra.run.smithi012.stdout: "linger_resend": 0, 2024-04-22T13:34:48.436 INFO:teuthology.orchestra.run.smithi012.stdout: "linger_ping": 0, 2024-04-22T13:34:48.436 INFO:teuthology.orchestra.run.smithi012.stdout: "poolop_active": 0, 2024-04-22T13:34:48.436 INFO:teuthology.orchestra.run.smithi012.stdout: "poolop_send": 0, 2024-04-22T13:34:48.436 INFO:teuthology.orchestra.run.smithi012.stdout: "poolop_resend": 0, 2024-04-22T13:34:48.436 INFO:teuthology.orchestra.run.smithi012.stdout: "poolstat_active": 0, 2024-04-22T13:34:48.436 INFO:teuthology.orchestra.run.smithi012.stdout: "poolstat_send": 0, 2024-04-22T13:34:48.437 INFO:teuthology.orchestra.run.smithi012.stdout: "poolstat_resend": 0, 2024-04-22T13:34:48.437 INFO:teuthology.orchestra.run.smithi012.stdout: "statfs_active": 0, 2024-04-22T13:34:48.437 INFO:teuthology.orchestra.run.smithi012.stdout: "statfs_send": 0, 2024-04-22T13:34:48.437 INFO:teuthology.orchestra.run.smithi012.stdout: "statfs_resend": 0, 2024-04-22T13:34:48.437 INFO:teuthology.orchestra.run.smithi012.stdout: "command_active": 0, 2024-04-22T13:34:48.437 INFO:teuthology.orchestra.run.smithi012.stdout: "command_send": 0, 2024-04-22T13:34:48.437 INFO:teuthology.orchestra.run.smithi012.stdout: "command_resend": 0, 2024-04-22T13:34:48.437 INFO:teuthology.orchestra.run.smithi012.stdout: "map_epoch": 98, 2024-04-22T13:34:48.437 INFO:teuthology.orchestra.run.smithi012.stdout: "map_full": 0, 2024-04-22T13:34:48.437 INFO:teuthology.orchestra.run.smithi012.stdout: "map_inc": 11, 2024-04-22T13:34:48.438 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_sessions": 6, 2024-04-22T13:34:48.438 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_session_open": 6, 2024-04-22T13:34:48.438 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_session_close": 0, 2024-04-22T13:34:48.438 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_laggy": 0, 2024-04-22T13:34:48.438 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_wr": 0, 2024-04-22T13:34:48.438 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_rd": 2, 2024-04-22T13:34:48.438 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_del": 0 2024-04-22T13:34:48.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.438 INFO:teuthology.orchestra.run.smithi012.stdout: "oft": { 2024-04-22T13:34:48.438 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_total_objs": 0, 2024-04-22T13:34:48.438 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_total_kv_pairs": 0, 2024-04-22T13:34:48.438 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_total_updates": 0, 2024-04-22T13:34:48.438 INFO:teuthology.orchestra.run.smithi012.stdout: "omap_total_removes": 0 2024-04-22T13:34:48.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.439 INFO:teuthology.orchestra.run.smithi012.stdout: "osd-slow-ops": { 2024-04-22T13:34:48.439 INFO:teuthology.orchestra.run.smithi012.stdout: "slow_ops_count": 0 2024-04-22T13:34:48.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.439 INFO:teuthology.orchestra.run.smithi012.stdout: "purge_queue": { 2024-04-22T13:34:48.439 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executing_ops": 0, 2024-04-22T13:34:48.439 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executing_ops_high_water": 0, 2024-04-22T13:34:48.439 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executing": 0, 2024-04-22T13:34:48.439 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executing_high_water": 0, 2024-04-22T13:34:48.439 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executed_ops": 0, 2024-04-22T13:34:48.439 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_executed": 0, 2024-04-22T13:34:48.439 INFO:teuthology.orchestra.run.smithi012.stdout: "pq_item_in_journal": 0 2024-04-22T13:34:48.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.439 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T13:34:48.439 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-04-22T13:34:48.439 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-04-22T13:34:48.440 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-04-22T13:34:48.440 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 2398, 2024-04-22T13:34:48.440 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 2004389, 2024-04-22T13:34:48.440 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-04-22T13:34:48.440 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 2398, 2024-04-22T13:34:48.440 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-04-22T13:34:48.440 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-04-22T13:34:48.440 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 2398, 2024-04-22T13:34:48.440 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 2004389, 2024-04-22T13:34:48.440 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-04-22T13:34:48.440 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.440 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.440 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.440 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:48.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.441 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-objecter_bytes": { 2024-04-22T13:34:48.441 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-04-22T13:34:48.441 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-04-22T13:34:48.441 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-04-22T13:34:48.441 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 0, 2024-04-22T13:34:48.441 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 0, 2024-04-22T13:34:48.441 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-04-22T13:34:48.441 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 0, 2024-04-22T13:34:48.441 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 2159, 2024-04-22T13:34:48.441 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 1657, 2024-04-22T13:34:48.441 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 2159, 2024-04-22T13:34:48.441 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 1657, 2024-04-22T13:34:48.441 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-04-22T13:34:48.441 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.442 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.442 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.442 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:48.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.442 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-objecter_ops": { 2024-04-22T13:34:48.442 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-04-22T13:34:48.442 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 1024, 2024-04-22T13:34:48.442 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-04-22T13:34:48.442 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 0, 2024-04-22T13:34:48.442 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 0, 2024-04-22T13:34:48.442 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-04-22T13:34:48.442 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 0, 2024-04-22T13:34:48.442 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 2159, 2024-04-22T13:34:48.442 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 2159, 2024-04-22T13:34:48.443 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 2159, 2024-04-22T13:34:48.443 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 2159, 2024-04-22T13:34:48.443 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-04-22T13:34:48.443 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.443 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.443 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.443 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:48.443 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.443 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-write_buf_throttle": { 2024-04-22T13:34:48.443 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-04-22T13:34:48.443 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 3758096384, 2024-04-22T13:34:48.443 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-04-22T13:34:48.443 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 0, 2024-04-22T13:34:48.443 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 0, 2024-04-22T13:34:48.443 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-04-22T13:34:48.444 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 0, 2024-04-22T13:34:48.444 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-04-22T13:34:48.444 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-04-22T13:34:48.444 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 0, 2024-04-22T13:34:48.444 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 0, 2024-04-22T13:34:48.444 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-04-22T13:34:48.444 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.444 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.444 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.444 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:48.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:48.444 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-write_buf_throttle-0x55b6d576e6e0": { 2024-04-22T13:34:48.444 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-04-22T13:34:48.444 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 3758096384, 2024-04-22T13:34:48.444 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-04-22T13:34:48.444 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 0, 2024-04-22T13:34:48.445 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 0, 2024-04-22T13:34:48.445 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-04-22T13:34:48.445 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 0, 2024-04-22T13:34:48.445 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-04-22T13:34:48.445 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-04-22T13:34:48.445 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 0, 2024-04-22T13:34:48.445 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 0, 2024-04-22T13:34:48.445 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-04-22T13:34:48.445 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:48.445 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:48.445 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:48.445 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:48.445 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:48.445 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:34:49.366 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-22T13:34:49.367 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-22T13:34:49.367 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-04-22T13:34:49.367 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-04-22T13:34:49.367 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-04-22T13:34:49.367 DEBUG:tasks.check_counter:Getting stats from j 2024-04-22T13:34:49.367 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-22T13:34:49.624 INFO:teuthology.orchestra.run.smithi012.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.a/config 2024-04-22T13:34:49.860 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:49 smithi012 ceph-mon[28265]: pgmap v332: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:34:49.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:49 smithi134 ceph-mon[31338]: pgmap v332: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:34:49.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:49 smithi111 ceph-mon[31361]: pgmap v332: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:34:51.298 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-22T13:34:51.298 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T13:34:51.298 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 0, 2024-04-22T13:34:51.298 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 2, 2024-04-22T13:34:51.298 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 0, 2024-04-22T13:34:51.298 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 0, 2024-04-22T13:34:51.299 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 4, 2024-04-22T13:34:51.299 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 0, 2024-04-22T13:34:51.299 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.001362584, 2024-04-22T13:34:51.299 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.000000000, 2024-04-22T13:34:51.299 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.000355511, 2024-04-22T13:34:51.299 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-22T13:34:51.299 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:34:51.299 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:51.299 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:51.299 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:51.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:51.299 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:34:51.299 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:51.299 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:51.300 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:51.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:51.300 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-22T13:34:51.300 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 0 2024-04-22T13:34:51.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:51.300 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T13:34:51.300 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 230, 2024-04-22T13:34:51.300 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 217, 2024-04-22T13:34:51.300 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 1594016, 2024-04-22T13:34:51.300 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 132352, 2024-04-22T13:34:51.300 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 2, 2024-04-22T13:34:51.300 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 1, 2024-04-22T13:34:51.300 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.043248062, 2024-04-22T13:34:51.300 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.014053546, 2024-04-22T13:34:51.300 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.034103099, 2024-04-22T13:34:51.301 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.004370927, 2024-04-22T13:34:51.301 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:34:51.301 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 217, 2024-04-22T13:34:51.301 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.011922906, 2024-04-22T13:34:51.301 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000054944 2024-04-22T13:34:51.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:51.301 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:34:51.301 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:51.301 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:51.301 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:51.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:51.301 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 1594016, 2024-04-22T13:34:51.301 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 132352 2024-04-22T13:34:51.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:51.301 INFO:teuthology.orchestra.run.smithi012.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T13:34:51.302 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_messages": 2, 2024-04-22T13:34:51.302 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages": 163, 2024-04-22T13:34:51.302 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_bytes": 320, 2024-04-22T13:34:51.302 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_bytes": 184800, 2024-04-22T13:34:51.302 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_created_connections": 5, 2024-04-22T13:34:51.302 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_active_connections": 3, 2024-04-22T13:34:51.302 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_total_time": 0.014884875, 2024-04-22T13:34:51.302 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_send_time": 0.010210548, 2024-04-22T13:34:51.302 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_recv_time": 0.002326706, 2024-04-22T13:34:51.302 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-22T13:34:51.302 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:34:51.302 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 160, 2024-04-22T13:34:51.302 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.009106049, 2024-04-22T13:34:51.302 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000056912 2024-04-22T13:34:51.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:51.303 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:34:51.303 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:51.303 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:51.303 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:51.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:51.303 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-22T13:34:51.303 INFO:teuthology.orchestra.run.smithi012.stdout: "msgr_send_encrypted_bytes": 184800 2024-04-22T13:34:51.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:51.303 INFO:teuthology.orchestra.run.smithi012.stdout: "cct": { 2024-04-22T13:34:51.303 INFO:teuthology.orchestra.run.smithi012.stdout: "total_workers": 0, 2024-04-22T13:34:51.303 INFO:teuthology.orchestra.run.smithi012.stdout: "unhealthy_workers": 0 2024-04-22T13:34:51.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:51.303 INFO:teuthology.orchestra.run.smithi012.stdout: "mempool": { 2024-04-22T13:34:51.304 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_bytes": 0, 2024-04-22T13:34:51.304 INFO:teuthology.orchestra.run.smithi012.stdout: "bloom_filter_items": 0, 2024-04-22T13:34:51.304 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T13:34:51.304 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_items": 0, 2024-04-22T13:34:51.304 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T13:34:51.304 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_data_items": 0, 2024-04-22T13:34:51.304 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T13:34:51.304 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T13:34:51.304 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T13:34:51.305 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T13:34:51.305 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T13:34:51.305 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_other_items": 0, 2024-04-22T13:34:51.305 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T13:34:51.305 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T13:34:51.305 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_bytes": 0, 2024-04-22T13:34:51.305 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_items": 0, 2024-04-22T13:34:51.305 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_bytes": 0, 2024-04-22T13:34:51.305 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blob_items": 0, 2024-04-22T13:34:51.305 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T13:34:51.305 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T13:34:51.306 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T13:34:51.306 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T13:34:51.306 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T13:34:51.306 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_items": 0, 2024-04-22T13:34:51.306 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_bytes": 0, 2024-04-22T13:34:51.306 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_txc_items": 0, 2024-04-22T13:34:51.306 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T13:34:51.306 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T13:34:51.306 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_bytes": 0, 2024-04-22T13:34:51.306 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_writing_items": 0, 2024-04-22T13:34:51.306 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_bytes": 0, 2024-04-22T13:34:51.306 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_items": 0, 2024-04-22T13:34:51.306 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T13:34:51.306 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_reader_items": 0, 2024-04-22T13:34:51.307 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T13:34:51.307 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_file_writer_items": 0, 2024-04-22T13:34:51.307 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_bytes": 149057, 2024-04-22T13:34:51.307 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_anon_items": 32, 2024-04-22T13:34:51.307 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_bytes": 0, 2024-04-22T13:34:51.307 INFO:teuthology.orchestra.run.smithi012.stdout: "buffer_meta_items": 0, 2024-04-22T13:34:51.307 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bytes": 0, 2024-04-22T13:34:51.307 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_items": 0, 2024-04-22T13:34:51.307 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_bytes": 0, 2024-04-22T13:34:51.307 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mapbl_items": 0, 2024-04-22T13:34:51.307 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_bytes": 0, 2024-04-22T13:34:51.307 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pglog_items": 0, 2024-04-22T13:34:51.307 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_bytes": 0, 2024-04-22T13:34:51.307 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_items": 0, 2024-04-22T13:34:51.307 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T13:34:51.310 INFO:teuthology.orchestra.run.smithi012.stdout: "osdmap_mapping_items": 0, 2024-04-22T13:34:51.310 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_bytes": 0, 2024-04-22T13:34:51.310 INFO:teuthology.orchestra.run.smithi012.stdout: "pgmap_items": 0, 2024-04-22T13:34:51.310 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_bytes": 928, 2024-04-22T13:34:51.310 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_co_items": 2, 2024-04-22T13:34:51.310 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_bytes": 0, 2024-04-22T13:34:51.310 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_1_items": 0, 2024-04-22T13:34:51.310 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_bytes": 0, 2024-04-22T13:34:51.310 INFO:teuthology.orchestra.run.smithi012.stdout: "unittest_2_items": 0 2024-04-22T13:34:51.311 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-22T13:34:51.311 INFO:teuthology.orchestra.run.smithi012.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T13:34:51.311 INFO:teuthology.orchestra.run.smithi012.stdout: "val": 0, 2024-04-22T13:34:51.311 INFO:teuthology.orchestra.run.smithi012.stdout: "max": 104857600, 2024-04-22T13:34:51.311 INFO:teuthology.orchestra.run.smithi012.stdout: "get_started": 0, 2024-04-22T13:34:51.311 INFO:teuthology.orchestra.run.smithi012.stdout: "get": 228, 2024-04-22T13:34:51.311 INFO:teuthology.orchestra.run.smithi012.stdout: "get_sum": 1559144, 2024-04-22T13:34:51.311 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_fail": 0, 2024-04-22T13:34:51.311 INFO:teuthology.orchestra.run.smithi012.stdout: "get_or_fail_success": 228, 2024-04-22T13:34:51.311 INFO:teuthology.orchestra.run.smithi012.stdout: "take": 0, 2024-04-22T13:34:51.311 INFO:teuthology.orchestra.run.smithi012.stdout: "take_sum": 0, 2024-04-22T13:34:51.311 INFO:teuthology.orchestra.run.smithi012.stdout: "put": 228, 2024-04-22T13:34:51.311 INFO:teuthology.orchestra.run.smithi012.stdout: "put_sum": 1559144, 2024-04-22T13:34:51.311 INFO:teuthology.orchestra.run.smithi012.stdout: "wait": { 2024-04-22T13:34:51.311 INFO:teuthology.orchestra.run.smithi012.stdout: "avgcount": 0, 2024-04-22T13:34:51.312 INFO:teuthology.orchestra.run.smithi012.stdout: "sum": 0.000000000, 2024-04-22T13:34:51.312 INFO:teuthology.orchestra.run.smithi012.stdout: "avgtime": 0.000000000 2024-04-22T13:34:51.312 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:51.312 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-22T13:34:51.312 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-22T13:34:51.879 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:51 smithi012 ceph-mon[28265]: pgmap v333: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:51.909 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-22T13:34:51.909 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-22T13:34:51.909 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-22T13:34:51.909 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-22T13:34:51.909 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-22T13:34:51.910 DEBUG:tasks.check_counter:Getting stats from b 2024-04-22T13:34:51.910 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-22T13:34:51.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:51 smithi134 ceph-mon[31338]: pgmap v333: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:51.936 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:51 smithi111 ceph-mon[31361]: pgmap v333: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:52.158 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T13:34:53.835 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:53 smithi111 ceph-mon[31361]: pgmap v334: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:34:53.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:53 smithi134 ceph-mon[31338]: pgmap v334: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:34:53.960 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-04-22T13:34:53.961 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T13:34:53.961 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 1924, 2024-04-22T13:34:53.961 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 1537, 2024-04-22T13:34:53.961 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 721257, 2024-04-22T13:34:53.961 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 3400760, 2024-04-22T13:34:53.961 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 169, 2024-04-22T13:34:53.961 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 5, 2024-04-22T13:34:53.961 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.230077079, 2024-04-22T13:34:53.961 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.059816651, 2024-04-22T13:34:53.961 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.099799587, 2024-04-22T13:34:53.961 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.050336848, 2024-04-22T13:34:53.961 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:34:53.961 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 1535, 2024-04-22T13:34:53.961 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.105061520, 2024-04-22T13:34:53.962 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000068443 2024-04-22T13:34:53.962 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.962 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:34:53.962 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2052, 2024-04-22T13:34:53.962 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000410273, 2024-04-22T13:34:53.962 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000199 2024-04-22T13:34:53.962 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.962 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 44672, 2024-04-22T13:34:53.962 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 5616 2024-04-22T13:34:53.962 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.962 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T13:34:53.962 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 2088, 2024-04-22T13:34:53.962 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 806, 2024-04-22T13:34:53.962 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 819693, 2024-04-22T13:34:53.963 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 771232, 2024-04-22T13:34:53.963 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 41, 2024-04-22T13:34:53.963 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 5, 2024-04-22T13:34:53.963 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.255734493, 2024-04-22T13:34:53.963 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.053288588, 2024-04-22T13:34:53.963 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.097663309, 2024-04-22T13:34:53.963 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.100976343, 2024-04-22T13:34:53.963 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:34:53.963 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 804, 2024-04-22T13:34:53.963 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.148206908, 2024-04-22T13:34:53.963 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000184336 2024-04-22T13:34:53.963 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.963 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:34:53.964 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2139, 2024-04-22T13:34:53.964 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000372597, 2024-04-22T13:34:53.964 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000174 2024-04-22T13:34:53.964 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.964 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 413248, 2024-04-22T13:34:53.964 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 248368 2024-04-22T13:34:53.964 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.964 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T13:34:53.964 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 1782, 2024-04-22T13:34:53.964 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 1664, 2024-04-22T13:34:53.964 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 2319569, 2024-04-22T13:34:53.964 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 1108692, 2024-04-22T13:34:53.964 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 28, 2024-04-22T13:34:53.964 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 7, 2024-04-22T13:34:53.964 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.205907103, 2024-04-22T13:34:53.965 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.069429406, 2024-04-22T13:34:53.965 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.096753431, 2024-04-22T13:34:53.965 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.045927942, 2024-04-22T13:34:53.965 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:34:53.965 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 1663, 2024-04-22T13:34:53.965 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.116271116, 2024-04-22T13:34:53.965 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000069916 2024-04-22T13:34:53.965 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.965 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:34:53.965 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2047, 2024-04-22T13:34:53.965 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000426649, 2024-04-22T13:34:53.965 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000208 2024-04-22T13:34:53.965 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.965 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 1667648, 2024-04-22T13:34:53.966 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 483552 2024-04-22T13:34:53.966 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.966 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2024-04-22T13:34:53.966 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 1, 2024-04-22T13:34:53.966 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2024-04-22T13:34:53.966 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.966 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-MDSRank": { 2024-04-22T13:34:53.966 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-04-22T13:34:53.966 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-04-22T13:34:53.966 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 762, 2024-04-22T13:34:53.966 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.199692083, 2024-04-22T13:34:53.966 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000262063 2024-04-22T13:34:53.966 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:34:53.966 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.967 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-PurgeQueue": { 2024-04-22T13:34:53.967 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-04-22T13:34:53.967 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-04-22T13:34:53.967 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 87, 2024-04-22T13:34:53.967 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.003205617, 2024-04-22T13:34:53.967 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000036846 2024-04-22T13:34:53.967 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:34:53.967 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.967 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": { 2024-04-22T13:34:53.967 INFO:teuthology.orchestra.run.smithi111.stdout: "request": 166, 2024-04-22T13:34:53.967 INFO:teuthology.orchestra.run.smithi111.stdout: "reply": 166, 2024-04-22T13:34:53.967 INFO:teuthology.orchestra.run.smithi111.stdout: "reply_latency": { 2024-04-22T13:34:53.967 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 166, 2024-04-22T13:34:53.967 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.733881931, 2024-04-22T13:34:53.968 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.004420975 2024-04-22T13:34:53.968 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.968 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_reply": 0, 2024-04-22T13:34:53.968 INFO:teuthology.orchestra.run.smithi111.stdout: "forward": 0, 2024-04-22T13:34:53.968 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_complete": 0, 2024-04-22T13:34:53.968 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_keys": 0, 2024-04-22T13:34:53.968 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_commit": 53, 2024-04-22T13:34:53.968 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_split": 0, 2024-04-22T13:34:53.968 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_merge": 0, 2024-04-22T13:34:53.968 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes": 39, 2024-04-22T13:34:53.968 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_top": 9, 2024-04-22T13:34:53.968 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_bottom": 30, 2024-04-22T13:34:53.968 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pin_tail": 0, 2024-04-22T13:34:53.968 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pinned": 25, 2024-04-22T13:34:53.969 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_expired": 0, 2024-04-22T13:34:53.969 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_with_caps": 3, 2024-04-22T13:34:53.969 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": 4, 2024-04-22T13:34:53.969 INFO:teuthology.orchestra.run.smithi111.stdout: "subtrees": 2, 2024-04-22T13:34:53.969 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse": 231, 2024-04-22T13:34:53.969 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_hit": 221, 2024-04-22T13:34:53.969 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_forward": 0, 2024-04-22T13:34:53.969 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_discover": 0, 2024-04-22T13:34:53.969 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_dir_fetch": 0, 2024-04-22T13:34:53.969 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_remote_ino": 0, 2024-04-22T13:34:53.969 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_lock": 0, 2024-04-22T13:34:53.969 INFO:teuthology.orchestra.run.smithi111.stdout: "load_cent": 0, 2024-04-22T13:34:53.969 INFO:teuthology.orchestra.run.smithi111.stdout: "q": 0, 2024-04-22T13:34:53.969 INFO:teuthology.orchestra.run.smithi111.stdout: "exported": 0, 2024-04-22T13:34:53.970 INFO:teuthology.orchestra.run.smithi111.stdout: "exported_inodes": 0, 2024-04-22T13:34:53.970 INFO:teuthology.orchestra.run.smithi111.stdout: "imported": 0, 2024-04-22T13:34:53.970 INFO:teuthology.orchestra.run.smithi111.stdout: "imported_inodes": 0, 2024-04-22T13:34:53.970 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_dir_fetch": 0, 2024-04-22T13:34:53.970 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_backtrace_fetch": 245, 2024-04-22T13:34:53.970 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_peer_discover": 0, 2024-04-22T13:34:53.970 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rfiles": 4, 2024-04-22T13:34:53.970 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rbytes": 234, 2024-04-22T13:34:53.970 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rsnaps": 6, 2024-04-22T13:34:53.970 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_fetch": 245, 2024-04-22T13:34:53.970 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_set_tag": 0, 2024-04-22T13:34:53.970 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_repaired": 0, 2024-04-22T13:34:53.970 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_inotable_repaired": 0, 2024-04-22T13:34:53.970 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_inodes": 153, 2024-04-22T13:34:53.971 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_base_inodes": 21, 2024-04-22T13:34:53.971 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dirfrag_rstats": 132, 2024-04-22T13:34:53.971 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_file_inodes": 92, 2024-04-22T13:34:53.971 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_inode_file_caps": 1, 2024-04-22T13:34:53.971 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_revoke": 27, 2024-04-22T13:34:53.971 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_grant": 38, 2024-04-22T13:34:53.971 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_trunc": 0, 2024-04-22T13:34:53.971 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-22T13:34:53.971 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-22T13:34:53.971 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps": 117, 2024-04-22T13:34:53.971 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps_dirty": 85, 2024-04-22T13:34:53.971 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_cap_release": 4, 2024-04-22T13:34:53.971 INFO:teuthology.orchestra.run.smithi111.stdout: "process_request_cap_release": 29 2024-04-22T13:34:53.971 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.971 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_cache": { 2024-04-22T13:34:53.972 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update": 0, 2024-04-22T13:34:53.972 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update_receipt": 0, 2024-04-22T13:34:53.973 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_try_discover": 0, 2024-04-22T13:34:53.973 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_send_discover": 0, 2024-04-22T13:34:53.973 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_handle_discover": 7, 2024-04-22T13:34:53.973 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays": 2, 2024-04-22T13:34:53.973 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_delayed": 0, 2024-04-22T13:34:53.973 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_enqueuing": 0, 2024-04-22T13:34:53.974 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_created": 6, 2024-04-22T13:34:53.974 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_enqueued": 4, 2024-04-22T13:34:53.974 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_reintegrated": 0, 2024-04-22T13:34:53.974 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_migrated": 0, 2024-04-22T13:34:53.974 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_processing": 0, 2024-04-22T13:34:53.974 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_enqueued": 0, 2024-04-22T13:34:53.974 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_prioritized": 0, 2024-04-22T13:34:53.974 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_started": 0, 2024-04-22T13:34:53.974 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_completed": 0, 2024-04-22T13:34:53.974 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_path": 2, 2024-04-22T13:34:53.974 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_inode": 24, 2024-04-22T13:34:53.974 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_enqueue_scrub": 21, 2024-04-22T13:34:53.974 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_exportdir": 0, 2024-04-22T13:34:53.974 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_flush": 0, 2024-04-22T13:34:53.975 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragmentdir": 0, 2024-04-22T13:34:53.975 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragstats": 0, 2024-04-22T13:34:53.975 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_inodestats": 0 2024-04-22T13:34:53.975 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.975 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_log": { 2024-04-22T13:34:53.975 INFO:teuthology.orchestra.run.smithi111.stdout: "evlrg": 0, 2024-04-22T13:34:53.975 INFO:teuthology.orchestra.run.smithi111.stdout: "evadd": 251, 2024-04-22T13:34:53.975 INFO:teuthology.orchestra.run.smithi111.stdout: "evex": 158, 2024-04-22T13:34:53.975 INFO:teuthology.orchestra.run.smithi111.stdout: "evtrm": 158, 2024-04-22T13:34:53.975 INFO:teuthology.orchestra.run.smithi111.stdout: "ev": 93, 2024-04-22T13:34:53.975 INFO:teuthology.orchestra.run.smithi111.stdout: "evexg": 0, 2024-04-22T13:34:53.975 INFO:teuthology.orchestra.run.smithi111.stdout: "evexd": 21, 2024-04-22T13:34:53.975 INFO:teuthology.orchestra.run.smithi111.stdout: "segadd": 13, 2024-04-22T13:34:53.975 INFO:teuthology.orchestra.run.smithi111.stdout: "segex": 12, 2024-04-22T13:34:53.975 INFO:teuthology.orchestra.run.smithi111.stdout: "segtrm": 12, 2024-04-22T13:34:53.976 INFO:teuthology.orchestra.run.smithi111.stdout: "seg": 1, 2024-04-22T13:34:53.976 INFO:teuthology.orchestra.run.smithi111.stdout: "segmjr": 1, 2024-04-22T13:34:53.976 INFO:teuthology.orchestra.run.smithi111.stdout: "segexg": 0, 2024-04-22T13:34:53.976 INFO:teuthology.orchestra.run.smithi111.stdout: "segexd": 1, 2024-04-22T13:34:53.976 INFO:teuthology.orchestra.run.smithi111.stdout: "expos": 5055868, 2024-04-22T13:34:53.976 INFO:teuthology.orchestra.run.smithi111.stdout: "wrpos": 5314079, 2024-04-22T13:34:53.976 INFO:teuthology.orchestra.run.smithi111.stdout: "rdpos": 0, 2024-04-22T13:34:53.976 INFO:teuthology.orchestra.run.smithi111.stdout: "jlat": { 2024-04-22T13:34:53.976 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 204, 2024-04-22T13:34:53.976 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 1.880404346, 2024-04-22T13:34:53.976 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.009217668 2024-04-22T13:34:53.976 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.976 INFO:teuthology.orchestra.run.smithi111.stdout: "replayed": 0 2024-04-22T13:34:53.976 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.977 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_mem": { 2024-04-22T13:34:53.977 INFO:teuthology.orchestra.run.smithi111.stdout: "ino": 29, 2024-04-22T13:34:53.977 INFO:teuthology.orchestra.run.smithi111.stdout: "ino+": 54, 2024-04-22T13:34:53.977 INFO:teuthology.orchestra.run.smithi111.stdout: "ino-": 25, 2024-04-22T13:34:53.977 INFO:teuthology.orchestra.run.smithi111.stdout: "dir": 20, 2024-04-22T13:34:53.977 INFO:teuthology.orchestra.run.smithi111.stdout: "dir+": 20, 2024-04-22T13:34:53.977 INFO:teuthology.orchestra.run.smithi111.stdout: "dir-": 0, 2024-04-22T13:34:53.977 INFO:teuthology.orchestra.run.smithi111.stdout: "dn": 39, 2024-04-22T13:34:53.977 INFO:teuthology.orchestra.run.smithi111.stdout: "dn+": 43, 2024-04-22T13:34:53.977 INFO:teuthology.orchestra.run.smithi111.stdout: "dn-": 4, 2024-04-22T13:34:53.977 INFO:teuthology.orchestra.run.smithi111.stdout: "cap": 4, 2024-04-22T13:34:53.977 INFO:teuthology.orchestra.run.smithi111.stdout: "cap+": 27, 2024-04-22T13:34:53.977 INFO:teuthology.orchestra.run.smithi111.stdout: "cap-": 23, 2024-04-22T13:34:53.977 INFO:teuthology.orchestra.run.smithi111.stdout: "rss": 56996, 2024-04-22T13:34:53.977 INFO:teuthology.orchestra.run.smithi111.stdout: "heap": 231800 2024-04-22T13:34:53.978 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.978 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_server": { 2024-04-22T13:34:53.978 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_client_request": 215, 2024-04-22T13:34:53.978 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_server_request": 18, 2024-04-22T13:34:53.978 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_request": 166, 2024-04-22T13:34:53.978 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_session": 32, 2024-04-22T13:34:53.978 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_peer_request": 36, 2024-04-22T13:34:53.978 INFO:teuthology.orchestra.run.smithi111.stdout: "req_create_latency": { 2024-04-22T13:34:53.978 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 8, 2024-04-22T13:34:53.978 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.006139866, 2024-04-22T13:34:53.978 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000767483 2024-04-22T13:34:53.978 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.978 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getattr_latency": { 2024-04-22T13:34:53.978 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 99, 2024-04-22T13:34:53.979 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.021233171, 2024-04-22T13:34:53.979 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000214476 2024-04-22T13:34:53.979 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.979 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getfilelock_latency": { 2024-04-22T13:34:53.979 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:53.979 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:53.979 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:53.979 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.979 INFO:teuthology.orchestra.run.smithi111.stdout: "req_link_latency": { 2024-04-22T13:34:53.979 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:53.979 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:53.979 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:53.979 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.979 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookup_latency": { 2024-04-22T13:34:53.980 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 5, 2024-04-22T13:34:53.980 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.001318885, 2024-04-22T13:34:53.980 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000263777 2024-04-22T13:34:53.980 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.980 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookuphash_latency": { 2024-04-22T13:34:53.980 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:53.980 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:53.980 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:53.980 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.980 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupino_latency": { 2024-04-22T13:34:53.980 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2, 2024-04-22T13:34:53.980 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000222748, 2024-04-22T13:34:53.980 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000111374 2024-04-22T13:34:53.980 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.981 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupname_latency": { 2024-04-22T13:34:53.981 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:53.981 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:53.981 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:53.981 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.981 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupparent_latency": { 2024-04-22T13:34:53.981 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:53.981 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:53.981 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:53.981 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.981 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupsnap_latency": { 2024-04-22T13:34:53.981 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:53.981 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:53.981 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:53.981 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.982 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lssnap_latency": { 2024-04-22T13:34:53.982 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 10, 2024-04-22T13:34:53.982 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.001654522, 2024-04-22T13:34:53.982 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000165452 2024-04-22T13:34:53.982 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.983 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mkdir_latency": { 2024-04-22T13:34:53.983 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 8, 2024-04-22T13:34:53.983 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.011547647, 2024-04-22T13:34:53.983 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.001443455 2024-04-22T13:34:53.983 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.983 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mknod_latency": { 2024-04-22T13:34:53.983 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:53.983 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:53.983 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:53.983 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.983 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mksnap_latency": { 2024-04-22T13:34:53.983 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 10, 2024-04-22T13:34:53.983 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.050814422, 2024-04-22T13:34:53.984 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.005081442 2024-04-22T13:34:53.984 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.984 INFO:teuthology.orchestra.run.smithi111.stdout: "req_open_latency": { 2024-04-22T13:34:53.984 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 1, 2024-04-22T13:34:53.984 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000212340, 2024-04-22T13:34:53.984 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000212340 2024-04-22T13:34:53.984 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.984 INFO:teuthology.orchestra.run.smithi111.stdout: "req_readdir_latency": { 2024-04-22T13:34:53.984 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2, 2024-04-22T13:34:53.984 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.578621841, 2024-04-22T13:34:53.984 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.289310920 2024-04-22T13:34:53.984 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.984 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rename_latency": { 2024-04-22T13:34:53.984 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2, 2024-04-22T13:34:53.985 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.002028626, 2024-04-22T13:34:53.985 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.001014313 2024-04-22T13:34:53.985 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.985 INFO:teuthology.orchestra.run.smithi111.stdout: "req_renamesnap_latency": { 2024-04-22T13:34:53.985 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:53.985 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:53.985 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:53.985 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.985 INFO:teuthology.orchestra.run.smithi111.stdout: "req_snapdiff_latency": { 2024-04-22T13:34:53.985 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:53.985 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:53.985 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:53.985 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.985 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmdir_latency": { 2024-04-22T13:34:53.986 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2, 2024-04-22T13:34:53.986 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.011289063, 2024-04-22T13:34:53.986 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.005644531 2024-04-22T13:34:53.986 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.986 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmsnap_latency": { 2024-04-22T13:34:53.986 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 4, 2024-04-22T13:34:53.986 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.018595548, 2024-04-22T13:34:53.986 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.004648887 2024-04-22T13:34:53.986 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.986 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmxattr_latency": { 2024-04-22T13:34:53.986 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:53.986 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:53.986 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:53.987 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.987 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setattr_latency": { 2024-04-22T13:34:53.987 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 1, 2024-04-22T13:34:53.987 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000974659, 2024-04-22T13:34:53.987 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000974659 2024-04-22T13:34:53.987 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.987 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setdirlayout_latency": { 2024-04-22T13:34:53.987 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:53.987 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:53.987 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:53.987 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.987 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setfilelock_latency": { 2024-04-22T13:34:53.987 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:53.988 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:53.988 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:53.988 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.988 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setlayout_latency": { 2024-04-22T13:34:53.988 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:53.988 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:53.988 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:53.988 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.988 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setxattr_latency": { 2024-04-22T13:34:53.988 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 8, 2024-04-22T13:34:53.988 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.022962092, 2024-04-22T13:34:53.988 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.002870261 2024-04-22T13:34:53.988 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.988 INFO:teuthology.orchestra.run.smithi111.stdout: "req_symlink_latency": { 2024-04-22T13:34:53.989 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:53.989 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:53.989 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:53.989 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.989 INFO:teuthology.orchestra.run.smithi111.stdout: "req_unlink_latency": { 2024-04-22T13:34:53.989 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 4, 2024-04-22T13:34:53.989 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.006266501, 2024-04-22T13:34:53.989 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.001566625 2024-04-22T13:34:53.989 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.989 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_revoke_eviction": 0, 2024-04-22T13:34:53.989 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_acquisition_throttle": 0, 2024-04-22T13:34:53.989 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getvxattr_latency": { 2024-04-22T13:34:53.989 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:53.989 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:53.990 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:53.990 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:34:53.990 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.990 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_sessions": { 2024-04-22T13:34:53.990 INFO:teuthology.orchestra.run.smithi111.stdout: "session_count": 2, 2024-04-22T13:34:53.990 INFO:teuthology.orchestra.run.smithi111.stdout: "session_add": 5, 2024-04-22T13:34:53.990 INFO:teuthology.orchestra.run.smithi111.stdout: "session_remove": 3, 2024-04-22T13:34:53.990 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_open": 2, 2024-04-22T13:34:53.990 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_stale": 0, 2024-04-22T13:34:53.990 INFO:teuthology.orchestra.run.smithi111.stdout: "total_load": 27, 2024-04-22T13:34:53.990 INFO:teuthology.orchestra.run.smithi111.stdout: "average_load": 13, 2024-04-22T13:34:53.990 INFO:teuthology.orchestra.run.smithi111.stdout: "avg_session_uptime": 617, 2024-04-22T13:34:53.990 INFO:teuthology.orchestra.run.smithi111.stdout: "mdthresh_evicted": 0 2024-04-22T13:34:53.990 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.991 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2024-04-22T13:34:53.991 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 0, 2024-04-22T13:34:53.991 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 0, 2024-04-22T13:34:53.991 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T13:34:53.991 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2024-04-22T13:34:53.991 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T13:34:53.991 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2024-04-22T13:34:53.991 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T13:34:53.991 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T13:34:53.991 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T13:34:53.991 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T13:34:53.991 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T13:34:53.991 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2024-04-22T13:34:53.992 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T13:34:53.992 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T13:34:53.992 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2024-04-22T13:34:53.992 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2024-04-22T13:34:53.992 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2024-04-22T13:34:53.992 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2024-04-22T13:34:53.992 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T13:34:53.992 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T13:34:53.992 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T13:34:53.992 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T13:34:53.992 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T13:34:53.992 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2024-04-22T13:34:53.992 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2024-04-22T13:34:53.992 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2024-04-22T13:34:53.993 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T13:34:53.993 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T13:34:53.993 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2024-04-22T13:34:53.993 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2024-04-22T13:34:53.993 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2024-04-22T13:34:53.993 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2024-04-22T13:34:53.993 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T13:34:53.993 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2024-04-22T13:34:53.993 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T13:34:53.993 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2024-04-22T13:34:53.993 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 335221, 2024-04-22T13:34:53.993 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 123, 2024-04-22T13:34:53.993 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 0, 2024-04-22T13:34:53.993 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 0, 2024-04-22T13:34:53.993 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2024-04-22T13:34:53.994 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2024-04-22T13:34:53.994 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2024-04-22T13:34:53.994 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2024-04-22T13:34:53.994 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2024-04-22T13:34:53.994 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2024-04-22T13:34:53.994 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 7936, 2024-04-22T13:34:53.994 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 165, 2024-04-22T13:34:53.994 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T13:34:53.994 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2024-04-22T13:34:53.994 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2024-04-22T13:34:53.994 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2024-04-22T13:34:53.994 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 136605, 2024-04-22T13:34:53.994 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 1193, 2024-04-22T13:34:53.995 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2024-04-22T13:34:53.995 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2024-04-22T13:34:53.995 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2024-04-22T13:34:53.995 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2024-04-22T13:34:53.995 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.995 INFO:teuthology.orchestra.run.smithi111.stdout: "objecter": { 2024-04-22T13:34:53.995 INFO:teuthology.orchestra.run.smithi111.stdout: "op_active": 0, 2024-04-22T13:34:53.995 INFO:teuthology.orchestra.run.smithi111.stdout: "op_laggy": 0, 2024-04-22T13:34:53.995 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send": 1251, 2024-04-22T13:34:53.995 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send_bytes": 1217698, 2024-04-22T13:34:53.995 INFO:teuthology.orchestra.run.smithi111.stdout: "op_resend": 31, 2024-04-22T13:34:53.995 INFO:teuthology.orchestra.run.smithi111.stdout: "op_reply": 1251, 2024-04-22T13:34:53.995 INFO:teuthology.orchestra.run.smithi111.stdout: "op_latency": { 2024-04-22T13:34:53.995 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 1251, 2024-04-22T13:34:53.996 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 24.568540743, 2024-04-22T13:34:53.996 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.019639121 2024-04-22T13:34:53.996 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.996 INFO:teuthology.orchestra.run.smithi111.stdout: "op_inflight": 0, 2024-04-22T13:34:53.996 INFO:teuthology.orchestra.run.smithi111.stdout: "oplen_avg": { 2024-04-22T13:34:53.996 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 1251, 2024-04-22T13:34:53.996 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 1409 2024-04-22T13:34:53.996 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:53.996 INFO:teuthology.orchestra.run.smithi111.stdout: "op": 1251, 2024-04-22T13:34:53.996 INFO:teuthology.orchestra.run.smithi111.stdout: "op_r": 289, 2024-04-22T13:34:53.996 INFO:teuthology.orchestra.run.smithi111.stdout: "op_w": 962, 2024-04-22T13:34:53.996 INFO:teuthology.orchestra.run.smithi111.stdout: "op_rmw": 0, 2024-04-22T13:34:53.996 INFO:teuthology.orchestra.run.smithi111.stdout: "op_pg": 0, 2024-04-22T13:34:53.996 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_stat": 33, 2024-04-22T13:34:53.996 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_create": 28, 2024-04-22T13:34:53.997 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_read": 23, 2024-04-22T13:34:53.997 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_write": 206, 2024-04-22T13:34:53.997 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writefull": 57, 2024-04-22T13:34:53.997 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writesame": 0, 2024-04-22T13:34:53.997 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_append": 0, 2024-04-22T13:34:53.997 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_zero": 0, 2024-04-22T13:34:53.997 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_truncate": 0, 2024-04-22T13:34:53.997 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_delete": 579, 2024-04-22T13:34:53.997 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_mapext": 0, 2024-04-22T13:34:53.997 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_sparse_read": 0, 2024-04-22T13:34:53.997 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_clonerange": 0, 2024-04-22T13:34:53.997 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_getxattr": 266, 2024-04-22T13:34:53.997 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_setxattr": 56, 2024-04-22T13:34:53.997 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_cmpxattr": 0, 2024-04-22T13:34:53.998 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_rmxattr": 0, 2024-04-22T13:34:53.998 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_resetxattrs": 0, 2024-04-22T13:34:53.998 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_call": 0, 2024-04-22T13:34:53.998 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_watch": 0, 2024-04-22T13:34:53.998 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_notify": 0, 2024-04-22T13:34:53.998 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_src_cmpxattr": 0, 2024-04-22T13:34:53.998 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls": 0, 2024-04-22T13:34:53.998 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls_filter": 0, 2024-04-22T13:34:53.998 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_other": 0, 2024-04-22T13:34:53.998 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_active": 0, 2024-04-22T13:34:53.998 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_send": 0, 2024-04-22T13:34:53.998 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_resend": 0, 2024-04-22T13:34:53.998 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_ping": 0, 2024-04-22T13:34:53.998 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_active": 0, 2024-04-22T13:34:53.999 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_send": 0, 2024-04-22T13:34:53.999 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_resend": 0, 2024-04-22T13:34:53.999 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_active": 0, 2024-04-22T13:34:53.999 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_send": 0, 2024-04-22T13:34:53.999 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_resend": 0, 2024-04-22T13:34:53.999 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_active": 0, 2024-04-22T13:34:53.999 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_send": 0, 2024-04-22T13:34:53.999 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_resend": 0, 2024-04-22T13:34:53.999 INFO:teuthology.orchestra.run.smithi111.stdout: "command_active": 0, 2024-04-22T13:34:53.999 INFO:teuthology.orchestra.run.smithi111.stdout: "command_send": 0, 2024-04-22T13:34:53.999 INFO:teuthology.orchestra.run.smithi111.stdout: "command_resend": 0, 2024-04-22T13:34:53.999 INFO:teuthology.orchestra.run.smithi111.stdout: "map_epoch": 98, 2024-04-22T13:34:53.999 INFO:teuthology.orchestra.run.smithi111.stdout: "map_full": 0, 2024-04-22T13:34:53.999 INFO:teuthology.orchestra.run.smithi111.stdout: "map_inc": 12, 2024-04-22T13:34:54.000 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_sessions": 12, 2024-04-22T13:34:54.000 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_open": 12, 2024-04-22T13:34:54.000 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_close": 0, 2024-04-22T13:34:54.000 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_laggy": 0, 2024-04-22T13:34:54.000 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_wr": 147, 2024-04-22T13:34:54.000 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_rd": 0, 2024-04-22T13:34:54.000 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_del": 14 2024-04-22T13:34:54.000 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:54.000 INFO:teuthology.orchestra.run.smithi111.stdout: "oft": { 2024-04-22T13:34:54.000 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_objs": 1, 2024-04-22T13:34:54.000 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_kv_pairs": 0, 2024-04-22T13:34:54.000 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_updates": 0, 2024-04-22T13:34:54.000 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_removes": 0 2024-04-22T13:34:54.000 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:54.000 INFO:teuthology.orchestra.run.smithi111.stdout: "osd-slow-ops": { 2024-04-22T13:34:54.001 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_ops_count": 44 2024-04-22T13:34:54.001 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:54.001 INFO:teuthology.orchestra.run.smithi111.stdout: "purge_queue": { 2024-04-22T13:34:54.001 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops": 0, 2024-04-22T13:34:54.001 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops_high_water": 64, 2024-04-22T13:34:54.001 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing": 0, 2024-04-22T13:34:54.001 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_high_water": 3, 2024-04-22T13:34:54.001 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed_ops": 67, 2024-04-22T13:34:54.001 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed": 4, 2024-04-22T13:34:54.001 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_item_in_journal": 0 2024-04-22T13:34:54.001 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:54.001 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T13:34:54.001 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-22T13:34:54.001 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-04-22T13:34:54.002 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-22T13:34:54.002 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 5790, 2024-04-22T13:34:54.002 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 3272958, 2024-04-22T13:34:54.002 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-22T13:34:54.002 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 5790, 2024-04-22T13:34:54.002 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-04-22T13:34:54.002 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-04-22T13:34:54.002 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 5790, 2024-04-22T13:34:54.002 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 3272958, 2024-04-22T13:34:54.002 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-22T13:34:54.002 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:54.002 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:54.002 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:54.002 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:34:54.003 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:54.003 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_bytes": { 2024-04-22T13:34:54.003 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-22T13:34:54.003 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-04-22T13:34:54.003 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-22T13:34:54.003 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-04-22T13:34:54.003 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-04-22T13:34:54.003 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-22T13:34:54.003 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-04-22T13:34:54.003 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 1251, 2024-04-22T13:34:54.003 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 1218134, 2024-04-22T13:34:54.003 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 1251, 2024-04-22T13:34:54.003 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 1218134, 2024-04-22T13:34:54.003 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-22T13:34:54.003 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:54.004 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:54.004 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:54.004 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:34:54.004 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:54.004 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_ops": { 2024-04-22T13:34:54.004 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-22T13:34:54.004 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 1024, 2024-04-22T13:34:54.004 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-22T13:34:54.004 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-04-22T13:34:54.004 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-04-22T13:34:54.004 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-22T13:34:54.004 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-04-22T13:34:54.004 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 1251, 2024-04-22T13:34:54.005 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 1251, 2024-04-22T13:34:54.005 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 1251, 2024-04-22T13:34:54.005 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 1251, 2024-04-22T13:34:54.005 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-22T13:34:54.005 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:54.005 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:54.005 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:54.005 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:34:54.005 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:54.005 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle": { 2024-04-22T13:34:54.005 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-22T13:34:54.005 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-04-22T13:34:54.005 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-22T13:34:54.005 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 4, 2024-04-22T13:34:54.006 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 436, 2024-04-22T13:34:54.006 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-22T13:34:54.006 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 4, 2024-04-22T13:34:54.006 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-04-22T13:34:54.006 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-04-22T13:34:54.006 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 2, 2024-04-22T13:34:54.006 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 436, 2024-04-22T13:34:54.006 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-22T13:34:54.006 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:54.006 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:54.006 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:54.006 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:34:54.006 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:54.007 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle-0x5575e46d2500": { 2024-04-22T13:34:54.007 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-22T13:34:54.007 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-04-22T13:34:54.007 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-22T13:34:54.007 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 251, 2024-04-22T13:34:54.007 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 1119775, 2024-04-22T13:34:54.007 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-22T13:34:54.007 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 251, 2024-04-22T13:34:54.007 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-04-22T13:34:54.007 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-04-22T13:34:54.007 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 204, 2024-04-22T13:34:54.007 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 1119775, 2024-04-22T13:34:54.007 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-22T13:34:54.007 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:54.007 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:54.008 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:54.008 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:34:54.008 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:34:54.008 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-04-22T13:34:54.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:53 smithi012 ceph-mon[28265]: pgmap v334: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:34:54.742 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-22T13:34:54.742 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-22T13:34:54.742 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-22T13:34:54.742 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-04-22T13:34:54.742 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-04-22T13:34:54.742 DEBUG:tasks.check_counter:Getting stats from e 2024-04-22T13:34:54.742 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-22T13:34:55.030 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T13:34:55.871 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:55 smithi111 ceph-mon[31361]: pgmap v335: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:55.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:55 smithi134 ceph-mon[31338]: pgmap v335: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:56.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:55 smithi012 ceph-mon[28265]: pgmap v335: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:56.744 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-04-22T13:34:56.744 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T13:34:56.745 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 812, 2024-04-22T13:34:56.745 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 893, 2024-04-22T13:34:56.745 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 1788855, 2024-04-22T13:34:56.745 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 540355, 2024-04-22T13:34:56.745 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 12, 2024-04-22T13:34:56.745 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 3, 2024-04-22T13:34:56.745 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.118614560, 2024-04-22T13:34:56.745 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.047476157, 2024-04-22T13:34:56.745 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.066551893, 2024-04-22T13:34:56.745 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.009861665, 2024-04-22T13:34:56.745 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:34:56.745 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 894, 2024-04-22T13:34:56.745 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.049103702, 2024-04-22T13:34:56.746 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000054925 2024-04-22T13:34:56.746 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.746 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:34:56.746 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 1103, 2024-04-22T13:34:56.746 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000310678, 2024-04-22T13:34:56.746 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000281 2024-04-22T13:34:56.746 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.746 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 1645264, 2024-04-22T13:34:56.746 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 144624 2024-04-22T13:34:56.746 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.746 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T13:34:56.746 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 62, 2024-04-22T13:34:56.746 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 222, 2024-04-22T13:34:56.746 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 13311, 2024-04-22T13:34:56.747 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 924987, 2024-04-22T13:34:56.747 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 49, 2024-04-22T13:34:56.747 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 8, 2024-04-22T13:34:56.747 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.024932683, 2024-04-22T13:34:56.747 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.007332647, 2024-04-22T13:34:56.747 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.014108281, 2024-04-22T13:34:56.747 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000801274, 2024-04-22T13:34:56.747 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:34:56.747 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 220, 2024-04-22T13:34:56.747 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.024826414, 2024-04-22T13:34:56.747 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000112847 2024-04-22T13:34:56.747 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.747 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:34:56.747 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 48, 2024-04-22T13:34:56.748 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000004770, 2024-04-22T13:34:56.748 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000099 2024-04-22T13:34:56.748 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.748 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 4240, 2024-04-22T13:34:56.748 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 450416 2024-04-22T13:34:56.748 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.748 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T13:34:56.748 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 37, 2024-04-22T13:34:56.748 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 75, 2024-04-22T13:34:56.748 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 8359, 2024-04-22T13:34:56.748 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 71514, 2024-04-22T13:34:56.748 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 12, 2024-04-22T13:34:56.748 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 5, 2024-04-22T13:34:56.748 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.012710924, 2024-04-22T13:34:56.749 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.004136115, 2024-04-22T13:34:56.749 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.007535178, 2024-04-22T13:34:56.749 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000245427, 2024-04-22T13:34:56.749 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:34:56.749 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 71, 2024-04-22T13:34:56.749 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.009321070, 2024-04-22T13:34:56.749 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000131282 2024-04-22T13:34:56.749 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.749 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:34:56.749 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 68, 2024-04-22T13:34:56.749 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000017363, 2024-04-22T13:34:56.749 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000255 2024-04-22T13:34:56.749 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.749 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 928, 2024-04-22T13:34:56.749 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 976 2024-04-22T13:34:56.750 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.750 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2024-04-22T13:34:56.750 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 1, 2024-04-22T13:34:56.750 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2024-04-22T13:34:56.750 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.750 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-MDSRank": { 2024-04-22T13:34:56.750 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-04-22T13:34:56.750 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-04-22T13:34:56.750 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 53, 2024-04-22T13:34:56.750 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.001277252, 2024-04-22T13:34:56.750 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000024099 2024-04-22T13:34:56.750 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:34:56.750 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.750 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-PurgeQueue": { 2024-04-22T13:34:56.751 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-04-22T13:34:56.751 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-04-22T13:34:56.751 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2, 2024-04-22T13:34:56.751 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000004740, 2024-04-22T13:34:56.751 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000002370 2024-04-22T13:34:56.751 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:34:56.751 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.751 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": { 2024-04-22T13:34:56.751 INFO:teuthology.orchestra.run.smithi111.stdout: "request": 2, 2024-04-22T13:34:56.751 INFO:teuthology.orchestra.run.smithi111.stdout: "reply": 1, 2024-04-22T13:34:56.751 INFO:teuthology.orchestra.run.smithi111.stdout: "reply_latency": { 2024-04-22T13:34:56.751 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 1, 2024-04-22T13:34:56.751 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000704002, 2024-04-22T13:34:56.751 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000704002 2024-04-22T13:34:56.751 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.751 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_reply": 0, 2024-04-22T13:34:56.752 INFO:teuthology.orchestra.run.smithi111.stdout: "forward": 1, 2024-04-22T13:34:56.752 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_complete": 0, 2024-04-22T13:34:56.752 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_keys": 0, 2024-04-22T13:34:56.752 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_commit": 11, 2024-04-22T13:34:56.752 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_split": 0, 2024-04-22T13:34:56.752 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_merge": 0, 2024-04-22T13:34:56.752 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes": 14, 2024-04-22T13:34:56.752 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_top": 0, 2024-04-22T13:34:56.752 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_bottom": 14, 2024-04-22T13:34:56.752 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pin_tail": 0, 2024-04-22T13:34:56.752 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pinned": 14, 2024-04-22T13:34:56.752 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_expired": 0, 2024-04-22T13:34:56.752 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_with_caps": 1, 2024-04-22T13:34:56.752 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": 1, 2024-04-22T13:34:56.752 INFO:teuthology.orchestra.run.smithi111.stdout: "subtrees": 2, 2024-04-22T13:34:56.753 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse": 7, 2024-04-22T13:34:56.753 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_hit": 5, 2024-04-22T13:34:56.753 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_forward": 0, 2024-04-22T13:34:56.753 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_discover": 1, 2024-04-22T13:34:56.753 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_dir_fetch": 0, 2024-04-22T13:34:56.753 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_remote_ino": 0, 2024-04-22T13:34:56.753 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_lock": 0, 2024-04-22T13:34:56.753 INFO:teuthology.orchestra.run.smithi111.stdout: "load_cent": 0, 2024-04-22T13:34:56.753 INFO:teuthology.orchestra.run.smithi111.stdout: "q": 0, 2024-04-22T13:34:56.753 INFO:teuthology.orchestra.run.smithi111.stdout: "exported": 0, 2024-04-22T13:34:56.753 INFO:teuthology.orchestra.run.smithi111.stdout: "exported_inodes": 0, 2024-04-22T13:34:56.753 INFO:teuthology.orchestra.run.smithi111.stdout: "imported": 0, 2024-04-22T13:34:56.753 INFO:teuthology.orchestra.run.smithi111.stdout: "imported_inodes": 0, 2024-04-22T13:34:56.753 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_dir_fetch": 0, 2024-04-22T13:34:56.753 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_backtrace_fetch": 0, 2024-04-22T13:34:56.754 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_peer_discover": 0, 2024-04-22T13:34:56.754 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rfiles": 4, 2024-04-22T13:34:56.754 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rbytes": 234, 2024-04-22T13:34:56.754 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rsnaps": 6, 2024-04-22T13:34:56.754 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_fetch": 0, 2024-04-22T13:34:56.754 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_set_tag": 0, 2024-04-22T13:34:56.754 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_repaired": 0, 2024-04-22T13:34:56.754 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_inotable_repaired": 0, 2024-04-22T13:34:56.754 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_inodes": 0, 2024-04-22T13:34:56.754 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_base_inodes": 0, 2024-04-22T13:34:56.754 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dirfrag_rstats": 0, 2024-04-22T13:34:56.754 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_file_inodes": 0, 2024-04-22T13:34:56.754 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_inode_file_caps": 0, 2024-04-22T13:34:56.755 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_revoke": 4, 2024-04-22T13:34:56.755 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_grant": 5, 2024-04-22T13:34:56.755 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_trunc": 0, 2024-04-22T13:34:56.755 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-22T13:34:56.755 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-22T13:34:56.755 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps": 4, 2024-04-22T13:34:56.755 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps_dirty": 0, 2024-04-22T13:34:56.755 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_cap_release": 0, 2024-04-22T13:34:56.755 INFO:teuthology.orchestra.run.smithi111.stdout: "process_request_cap_release": 1 2024-04-22T13:34:56.755 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.755 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_cache": { 2024-04-22T13:34:56.755 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update": 0, 2024-04-22T13:34:56.755 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update_receipt": 0, 2024-04-22T13:34:56.755 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_try_discover": 0, 2024-04-22T13:34:56.755 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_send_discover": 2, 2024-04-22T13:34:56.755 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_handle_discover": 0, 2024-04-22T13:34:56.756 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays": 0, 2024-04-22T13:34:56.756 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_delayed": 0, 2024-04-22T13:34:56.756 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_enqueuing": 0, 2024-04-22T13:34:56.756 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_created": 0, 2024-04-22T13:34:56.756 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_enqueued": 0, 2024-04-22T13:34:56.756 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_reintegrated": 0, 2024-04-22T13:34:56.756 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_migrated": 0, 2024-04-22T13:34:56.756 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_processing": 0, 2024-04-22T13:34:56.756 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_enqueued": 0, 2024-04-22T13:34:56.756 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_prioritized": 0, 2024-04-22T13:34:56.756 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_started": 0, 2024-04-22T13:34:56.756 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_completed": 0, 2024-04-22T13:34:56.756 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_path": 2, 2024-04-22T13:34:56.756 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_inode": 10, 2024-04-22T13:34:56.757 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_enqueue_scrub": 0, 2024-04-22T13:34:56.757 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_exportdir": 0, 2024-04-22T13:34:56.757 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_flush": 0, 2024-04-22T13:34:56.757 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragmentdir": 0, 2024-04-22T13:34:56.757 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragstats": 0, 2024-04-22T13:34:56.757 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_inodestats": 0 2024-04-22T13:34:56.757 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.757 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_log": { 2024-04-22T13:34:56.757 INFO:teuthology.orchestra.run.smithi111.stdout: "evlrg": 0, 2024-04-22T13:34:56.757 INFO:teuthology.orchestra.run.smithi111.stdout: "evadd": 8, 2024-04-22T13:34:56.757 INFO:teuthology.orchestra.run.smithi111.stdout: "evex": 0, 2024-04-22T13:34:56.757 INFO:teuthology.orchestra.run.smithi111.stdout: "evtrm": 0, 2024-04-22T13:34:56.757 INFO:teuthology.orchestra.run.smithi111.stdout: "ev": 8, 2024-04-22T13:34:56.757 INFO:teuthology.orchestra.run.smithi111.stdout: "evexg": 0, 2024-04-22T13:34:56.757 INFO:teuthology.orchestra.run.smithi111.stdout: "evexd": 0, 2024-04-22T13:34:56.757 INFO:teuthology.orchestra.run.smithi111.stdout: "segadd": 2, 2024-04-22T13:34:56.757 INFO:teuthology.orchestra.run.smithi111.stdout: "segex": 0, 2024-04-22T13:34:56.758 INFO:teuthology.orchestra.run.smithi111.stdout: "segtrm": 0, 2024-04-22T13:34:56.758 INFO:teuthology.orchestra.run.smithi111.stdout: "seg": 2, 2024-04-22T13:34:56.758 INFO:teuthology.orchestra.run.smithi111.stdout: "segmjr": 2, 2024-04-22T13:34:56.758 INFO:teuthology.orchestra.run.smithi111.stdout: "segexg": 0, 2024-04-22T13:34:56.758 INFO:teuthology.orchestra.run.smithi111.stdout: "segexd": 0, 2024-04-22T13:34:56.758 INFO:teuthology.orchestra.run.smithi111.stdout: "expos": 4194304, 2024-04-22T13:34:56.758 INFO:teuthology.orchestra.run.smithi111.stdout: "wrpos": 4196655, 2024-04-22T13:34:56.758 INFO:teuthology.orchestra.run.smithi111.stdout: "rdpos": 0, 2024-04-22T13:34:56.758 INFO:teuthology.orchestra.run.smithi111.stdout: "jlat": { 2024-04-22T13:34:56.758 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 7, 2024-04-22T13:34:56.758 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.012480458, 2024-04-22T13:34:56.758 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.001782922 2024-04-22T13:34:56.758 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.758 INFO:teuthology.orchestra.run.smithi111.stdout: "replayed": 0 2024-04-22T13:34:56.758 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.759 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_mem": { 2024-04-22T13:34:56.759 INFO:teuthology.orchestra.run.smithi111.stdout: "ino": 17, 2024-04-22T13:34:56.759 INFO:teuthology.orchestra.run.smithi111.stdout: "ino+": 17, 2024-04-22T13:34:56.759 INFO:teuthology.orchestra.run.smithi111.stdout: "ino-": 0, 2024-04-22T13:34:56.759 INFO:teuthology.orchestra.run.smithi111.stdout: "dir": 15, 2024-04-22T13:34:56.759 INFO:teuthology.orchestra.run.smithi111.stdout: "dir+": 16, 2024-04-22T13:34:56.759 INFO:teuthology.orchestra.run.smithi111.stdout: "dir-": 1, 2024-04-22T13:34:56.759 INFO:teuthology.orchestra.run.smithi111.stdout: "dn": 14, 2024-04-22T13:34:56.759 INFO:teuthology.orchestra.run.smithi111.stdout: "dn+": 14, 2024-04-22T13:34:56.759 INFO:teuthology.orchestra.run.smithi111.stdout: "dn-": 0, 2024-04-22T13:34:56.759 INFO:teuthology.orchestra.run.smithi111.stdout: "cap": 1, 2024-04-22T13:34:56.759 INFO:teuthology.orchestra.run.smithi111.stdout: "cap+": 1, 2024-04-22T13:34:56.759 INFO:teuthology.orchestra.run.smithi111.stdout: "cap-": 0, 2024-04-22T13:34:56.759 INFO:teuthology.orchestra.run.smithi111.stdout: "rss": 52788, 2024-04-22T13:34:56.759 INFO:teuthology.orchestra.run.smithi111.stdout: "heap": 223608 2024-04-22T13:34:56.760 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.760 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_server": { 2024-04-22T13:34:56.760 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_client_request": 3, 2024-04-22T13:34:56.760 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_server_request": 0, 2024-04-22T13:34:56.760 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_request": 2, 2024-04-22T13:34:56.760 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_session": 25, 2024-04-22T13:34:56.760 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_peer_request": 10, 2024-04-22T13:34:56.760 INFO:teuthology.orchestra.run.smithi111.stdout: "req_create_latency": { 2024-04-22T13:34:56.760 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.760 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.760 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.760 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.760 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getattr_latency": { 2024-04-22T13:34:56.760 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 1, 2024-04-22T13:34:56.760 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000704002, 2024-04-22T13:34:56.761 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000704002 2024-04-22T13:34:56.761 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.761 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getfilelock_latency": { 2024-04-22T13:34:56.761 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.761 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.761 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.761 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.761 INFO:teuthology.orchestra.run.smithi111.stdout: "req_link_latency": { 2024-04-22T13:34:56.761 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.761 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.761 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.761 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.761 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookup_latency": { 2024-04-22T13:34:56.761 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.761 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.761 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.762 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.762 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookuphash_latency": { 2024-04-22T13:34:56.762 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.762 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.762 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.762 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.762 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupino_latency": { 2024-04-22T13:34:56.762 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.762 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.762 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.762 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.762 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupname_latency": { 2024-04-22T13:34:56.762 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.762 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.763 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.763 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.763 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupparent_latency": { 2024-04-22T13:34:56.763 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.763 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.763 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.763 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.763 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupsnap_latency": { 2024-04-22T13:34:56.763 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.763 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.763 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.763 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.763 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lssnap_latency": { 2024-04-22T13:34:56.763 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.763 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.764 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.764 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.764 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mkdir_latency": { 2024-04-22T13:34:56.764 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.764 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.764 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.764 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.765 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mknod_latency": { 2024-04-22T13:34:56.765 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.765 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.765 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.765 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.765 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mksnap_latency": { 2024-04-22T13:34:56.765 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.765 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.765 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.765 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.765 INFO:teuthology.orchestra.run.smithi111.stdout: "req_open_latency": { 2024-04-22T13:34:56.765 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.765 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.765 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.766 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.766 INFO:teuthology.orchestra.run.smithi111.stdout: "req_readdir_latency": { 2024-04-22T13:34:56.766 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.766 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.766 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.766 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.766 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rename_latency": { 2024-04-22T13:34:56.766 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.766 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.766 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.766 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.766 INFO:teuthology.orchestra.run.smithi111.stdout: "req_renamesnap_latency": { 2024-04-22T13:34:56.766 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.767 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.767 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.767 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.767 INFO:teuthology.orchestra.run.smithi111.stdout: "req_snapdiff_latency": { 2024-04-22T13:34:56.767 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.767 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.767 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.767 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.767 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmdir_latency": { 2024-04-22T13:34:56.767 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.767 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.767 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.767 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.767 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmsnap_latency": { 2024-04-22T13:34:56.768 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.768 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.768 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.768 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.768 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmxattr_latency": { 2024-04-22T13:34:56.768 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.768 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.768 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.768 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.768 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setattr_latency": { 2024-04-22T13:34:56.768 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.768 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.768 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.768 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.768 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setdirlayout_latency": { 2024-04-22T13:34:56.769 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.769 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.769 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.769 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.769 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setfilelock_latency": { 2024-04-22T13:34:56.769 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.769 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.769 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.769 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.769 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setlayout_latency": { 2024-04-22T13:34:56.769 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.769 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.769 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.769 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.770 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setxattr_latency": { 2024-04-22T13:34:56.770 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.770 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.770 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.770 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.770 INFO:teuthology.orchestra.run.smithi111.stdout: "req_symlink_latency": { 2024-04-22T13:34:56.770 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.770 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.770 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.770 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.770 INFO:teuthology.orchestra.run.smithi111.stdout: "req_unlink_latency": { 2024-04-22T13:34:56.770 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.770 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.770 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.771 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.771 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_revoke_eviction": 0, 2024-04-22T13:34:56.771 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_acquisition_throttle": 0, 2024-04-22T13:34:56.771 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getvxattr_latency": { 2024-04-22T13:34:56.771 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.771 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.771 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.771 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:34:56.771 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.771 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_sessions": { 2024-04-22T13:34:56.771 INFO:teuthology.orchestra.run.smithi111.stdout: "session_count": 2, 2024-04-22T13:34:56.771 INFO:teuthology.orchestra.run.smithi111.stdout: "session_add": 4, 2024-04-22T13:34:56.771 INFO:teuthology.orchestra.run.smithi111.stdout: "session_remove": 2, 2024-04-22T13:34:56.771 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_open": 2, 2024-04-22T13:34:56.772 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_stale": 0, 2024-04-22T13:34:56.772 INFO:teuthology.orchestra.run.smithi111.stdout: "total_load": 1, 2024-04-22T13:34:56.772 INFO:teuthology.orchestra.run.smithi111.stdout: "average_load": 1, 2024-04-22T13:34:56.772 INFO:teuthology.orchestra.run.smithi111.stdout: "avg_session_uptime": 623, 2024-04-22T13:34:56.772 INFO:teuthology.orchestra.run.smithi111.stdout: "mdthresh_evicted": 0 2024-04-22T13:34:56.772 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.772 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2024-04-22T13:34:56.772 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 0, 2024-04-22T13:34:56.772 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 0, 2024-04-22T13:34:56.772 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T13:34:56.772 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2024-04-22T13:34:56.772 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T13:34:56.772 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2024-04-22T13:34:56.772 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T13:34:56.772 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T13:34:56.773 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T13:34:56.773 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T13:34:56.773 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T13:34:56.773 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2024-04-22T13:34:56.773 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T13:34:56.773 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T13:34:56.773 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2024-04-22T13:34:56.773 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2024-04-22T13:34:56.773 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2024-04-22T13:34:56.773 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2024-04-22T13:34:56.773 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T13:34:56.773 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T13:34:56.773 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T13:34:56.773 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T13:34:56.773 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T13:34:56.774 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2024-04-22T13:34:56.774 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2024-04-22T13:34:56.774 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2024-04-22T13:34:56.774 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T13:34:56.774 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T13:34:56.774 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2024-04-22T13:34:56.774 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2024-04-22T13:34:56.774 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2024-04-22T13:34:56.774 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2024-04-22T13:34:56.774 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T13:34:56.774 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2024-04-22T13:34:56.774 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T13:34:56.774 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2024-04-22T13:34:56.775 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 255015, 2024-04-22T13:34:56.775 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 87, 2024-04-22T13:34:56.775 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 0, 2024-04-22T13:34:56.775 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 0, 2024-04-22T13:34:56.775 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2024-04-22T13:34:56.775 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2024-04-22T13:34:56.775 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2024-04-22T13:34:56.775 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2024-04-22T13:34:56.776 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2024-04-22T13:34:56.776 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2024-04-22T13:34:56.776 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 7936, 2024-04-22T13:34:56.776 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 165, 2024-04-22T13:34:56.776 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T13:34:56.776 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2024-04-22T13:34:56.776 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2024-04-22T13:34:56.776 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2024-04-22T13:34:56.776 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 68077, 2024-04-22T13:34:56.777 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 293, 2024-04-22T13:34:56.777 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2024-04-22T13:34:56.777 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2024-04-22T13:34:56.777 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2024-04-22T13:34:56.777 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2024-04-22T13:34:56.777 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.777 INFO:teuthology.orchestra.run.smithi111.stdout: "objecter": { 2024-04-22T13:34:56.777 INFO:teuthology.orchestra.run.smithi111.stdout: "op_active": 0, 2024-04-22T13:34:56.777 INFO:teuthology.orchestra.run.smithi111.stdout: "op_laggy": 0, 2024-04-22T13:34:56.777 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send": 46, 2024-04-22T13:34:56.777 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send_bytes": 13148, 2024-04-22T13:34:56.777 INFO:teuthology.orchestra.run.smithi111.stdout: "op_resend": 33, 2024-04-22T13:34:56.777 INFO:teuthology.orchestra.run.smithi111.stdout: "op_reply": 46, 2024-04-22T13:34:56.777 INFO:teuthology.orchestra.run.smithi111.stdout: "op_latency": { 2024-04-22T13:34:56.778 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 46, 2024-04-22T13:34:56.778 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.215995991, 2024-04-22T13:34:56.778 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.004695565 2024-04-22T13:34:56.778 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.778 INFO:teuthology.orchestra.run.smithi111.stdout: "op_inflight": 0, 2024-04-22T13:34:56.778 INFO:teuthology.orchestra.run.smithi111.stdout: "oplen_avg": { 2024-04-22T13:34:56.778 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 46, 2024-04-22T13:34:56.778 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 70 2024-04-22T13:34:56.778 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.778 INFO:teuthology.orchestra.run.smithi111.stdout: "op": 46, 2024-04-22T13:34:56.778 INFO:teuthology.orchestra.run.smithi111.stdout: "op_r": 0, 2024-04-22T13:34:56.778 INFO:teuthology.orchestra.run.smithi111.stdout: "op_w": 46, 2024-04-22T13:34:56.778 INFO:teuthology.orchestra.run.smithi111.stdout: "op_rmw": 0, 2024-04-22T13:34:56.778 INFO:teuthology.orchestra.run.smithi111.stdout: "op_pg": 0, 2024-04-22T13:34:56.779 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_stat": 0, 2024-04-22T13:34:56.779 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_create": 10, 2024-04-22T13:34:56.779 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_read": 0, 2024-04-22T13:34:56.779 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_write": 7, 2024-04-22T13:34:56.779 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writefull": 8, 2024-04-22T13:34:56.779 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writesame": 0, 2024-04-22T13:34:56.779 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_append": 0, 2024-04-22T13:34:56.779 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_zero": 0, 2024-04-22T13:34:56.779 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_truncate": 0, 2024-04-22T13:34:56.779 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_delete": 6, 2024-04-22T13:34:56.779 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_mapext": 0, 2024-04-22T13:34:56.779 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_sparse_read": 0, 2024-04-22T13:34:56.779 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_clonerange": 0, 2024-04-22T13:34:56.779 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_getxattr": 0, 2024-04-22T13:34:56.779 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_setxattr": 20, 2024-04-22T13:34:56.780 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_cmpxattr": 0, 2024-04-22T13:34:56.780 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_rmxattr": 0, 2024-04-22T13:34:56.780 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_resetxattrs": 0, 2024-04-22T13:34:56.780 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_call": 0, 2024-04-22T13:34:56.780 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_watch": 0, 2024-04-22T13:34:56.780 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_notify": 0, 2024-04-22T13:34:56.780 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_src_cmpxattr": 0, 2024-04-22T13:34:56.780 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls": 0, 2024-04-22T13:34:56.780 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls_filter": 0, 2024-04-22T13:34:56.780 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_other": 0, 2024-04-22T13:34:56.780 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_active": 0, 2024-04-22T13:34:56.780 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_send": 0, 2024-04-22T13:34:56.780 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_resend": 0, 2024-04-22T13:34:56.780 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_ping": 0, 2024-04-22T13:34:56.780 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_active": 0, 2024-04-22T13:34:56.781 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_send": 0, 2024-04-22T13:34:56.781 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_resend": 0, 2024-04-22T13:34:56.781 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_active": 0, 2024-04-22T13:34:56.781 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_send": 0, 2024-04-22T13:34:56.781 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_resend": 0, 2024-04-22T13:34:56.781 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_active": 0, 2024-04-22T13:34:56.781 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_send": 0, 2024-04-22T13:34:56.781 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_resend": 0, 2024-04-22T13:34:56.781 INFO:teuthology.orchestra.run.smithi111.stdout: "command_active": 0, 2024-04-22T13:34:56.781 INFO:teuthology.orchestra.run.smithi111.stdout: "command_send": 0, 2024-04-22T13:34:56.781 INFO:teuthology.orchestra.run.smithi111.stdout: "command_resend": 0, 2024-04-22T13:34:56.781 INFO:teuthology.orchestra.run.smithi111.stdout: "map_epoch": 98, 2024-04-22T13:34:56.781 INFO:teuthology.orchestra.run.smithi111.stdout: "map_full": 0, 2024-04-22T13:34:56.781 INFO:teuthology.orchestra.run.smithi111.stdout: "map_inc": 11, 2024-04-22T13:34:56.781 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_sessions": 8, 2024-04-22T13:34:56.781 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_open": 8, 2024-04-22T13:34:56.782 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_close": 0, 2024-04-22T13:34:56.782 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_laggy": 0, 2024-04-22T13:34:56.782 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_wr": 18, 2024-04-22T13:34:56.782 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_rd": 0, 2024-04-22T13:34:56.782 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_del": 1 2024-04-22T13:34:56.782 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.782 INFO:teuthology.orchestra.run.smithi111.stdout: "oft": { 2024-04-22T13:34:56.782 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_objs": 1, 2024-04-22T13:34:56.782 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_kv_pairs": 0, 2024-04-22T13:34:56.782 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_updates": 0, 2024-04-22T13:34:56.782 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_removes": 0 2024-04-22T13:34:56.782 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.782 INFO:teuthology.orchestra.run.smithi111.stdout: "osd-slow-ops": { 2024-04-22T13:34:56.782 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_ops_count": 12 2024-04-22T13:34:56.783 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.783 INFO:teuthology.orchestra.run.smithi111.stdout: "purge_queue": { 2024-04-22T13:34:56.783 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops": 0, 2024-04-22T13:34:56.783 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops_high_water": 0, 2024-04-22T13:34:56.783 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing": 0, 2024-04-22T13:34:56.783 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_high_water": 0, 2024-04-22T13:34:56.783 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed_ops": 0, 2024-04-22T13:34:56.783 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed": 0, 2024-04-22T13:34:56.783 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_item_in_journal": 0 2024-04-22T13:34:56.783 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.783 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T13:34:56.783 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-22T13:34:56.783 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-04-22T13:34:56.783 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-22T13:34:56.783 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 907, 2024-04-22T13:34:56.784 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 1711181, 2024-04-22T13:34:56.784 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-22T13:34:56.784 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 907, 2024-04-22T13:34:56.784 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-04-22T13:34:56.784 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-04-22T13:34:56.784 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 907, 2024-04-22T13:34:56.784 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 1711181, 2024-04-22T13:34:56.784 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-22T13:34:56.784 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.784 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.784 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.784 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:34:56.784 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.784 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_bytes": { 2024-04-22T13:34:56.784 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-22T13:34:56.785 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-04-22T13:34:56.785 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-22T13:34:56.785 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-04-22T13:34:56.785 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-04-22T13:34:56.785 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-22T13:34:56.785 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-04-22T13:34:56.785 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 46, 2024-04-22T13:34:56.785 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 13148, 2024-04-22T13:34:56.785 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 46, 2024-04-22T13:34:56.785 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 13148, 2024-04-22T13:34:56.785 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-22T13:34:56.785 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.785 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.785 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.785 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:34:56.786 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.786 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_ops": { 2024-04-22T13:34:56.786 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-22T13:34:56.786 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 1024, 2024-04-22T13:34:56.786 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-22T13:34:56.786 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-04-22T13:34:56.786 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-04-22T13:34:56.786 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-22T13:34:56.786 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-04-22T13:34:56.786 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 46, 2024-04-22T13:34:56.786 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 46, 2024-04-22T13:34:56.786 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 46, 2024-04-22T13:34:56.786 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 46, 2024-04-22T13:34:56.786 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-22T13:34:56.786 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.787 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.787 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.787 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:34:56.787 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.787 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle": { 2024-04-22T13:34:56.787 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-22T13:34:56.787 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-04-22T13:34:56.787 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-22T13:34:56.787 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-04-22T13:34:56.787 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-04-22T13:34:56.787 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-22T13:34:56.787 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-04-22T13:34:56.787 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-04-22T13:34:56.787 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-04-22T13:34:56.787 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 0, 2024-04-22T13:34:56.788 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 0, 2024-04-22T13:34:56.788 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-22T13:34:56.788 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.788 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.788 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.788 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:34:56.788 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:56.788 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle-0x56206e8c8500": { 2024-04-22T13:34:56.788 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-22T13:34:56.788 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-04-22T13:34:56.788 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-22T13:34:56.788 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 8, 2024-04-22T13:34:56.788 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 2351, 2024-04-22T13:34:56.788 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-22T13:34:56.788 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 8, 2024-04-22T13:34:56.788 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-04-22T13:34:56.789 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-04-22T13:34:56.789 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 7, 2024-04-22T13:34:56.789 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 2351, 2024-04-22T13:34:56.789 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-22T13:34:56.789 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:56.789 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:56.789 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:56.789 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:34:56.789 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:34:56.789 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-04-22T13:34:57.636 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-22T13:34:57.636 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-22T13:34:57.636 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-04-22T13:34:57.636 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-22T13:34:57.636 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-22T13:34:57.636 DEBUG:tasks.check_counter:Getting stats from h 2024-04-22T13:34:57.636 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-22T13:34:57.875 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T13:34:57.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:57 smithi134 ceph-mon[31338]: pgmap v336: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:57.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:57 smithi111 ceph-mon[31361]: pgmap v336: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:58.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:57 smithi012 ceph-mon[28265]: pgmap v336: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:34:59.553 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-04-22T13:34:59.554 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T13:34:59.554 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 1658, 2024-04-22T13:34:59.554 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 1660, 2024-04-22T13:34:59.554 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 1609778, 2024-04-22T13:34:59.554 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 527254, 2024-04-22T13:34:59.554 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 6, 2024-04-22T13:34:59.554 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 2, 2024-04-22T13:34:59.554 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.141213831, 2024-04-22T13:34:59.554 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.056223090, 2024-04-22T13:34:59.554 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.031739188, 2024-04-22T13:34:59.554 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.039748481, 2024-04-22T13:34:59.554 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:34:59.554 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 1658, 2024-04-22T13:34:59.555 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.058945028, 2024-04-22T13:34:59.555 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000035551 2024-04-22T13:34:59.555 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.555 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:34:59.555 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.555 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.555 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.555 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.555 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-22T13:34:59.555 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 0 2024-04-22T13:34:59.555 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.555 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T13:34:59.555 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 253, 2024-04-22T13:34:59.555 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 234, 2024-04-22T13:34:59.556 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 1650840, 2024-04-22T13:34:59.556 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 134890, 2024-04-22T13:34:59.556 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 4, 2024-04-22T13:34:59.556 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 3, 2024-04-22T13:34:59.556 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.046048497, 2024-04-22T13:34:59.556 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.013888602, 2024-04-22T13:34:59.556 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.035972461, 2024-04-22T13:34:59.556 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.005486408, 2024-04-22T13:34:59.556 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:34:59.556 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 233, 2024-04-22T13:34:59.556 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.015651104, 2024-04-22T13:34:59.556 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000067172 2024-04-22T13:34:59.557 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.557 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:34:59.557 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.557 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.557 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.557 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.557 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 1649184, 2024-04-22T13:34:59.557 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 133552 2024-04-22T13:34:59.557 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.557 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T13:34:59.557 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 755, 2024-04-22T13:34:59.557 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 913, 2024-04-22T13:34:59.557 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 187637, 2024-04-22T13:34:59.557 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 687447, 2024-04-22T13:34:59.557 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 6, 2024-04-22T13:34:59.558 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 4, 2024-04-22T13:34:59.558 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.071683709, 2024-04-22T13:34:59.558 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.023233561, 2024-04-22T13:34:59.558 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.016363893, 2024-04-22T13:34:59.558 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.024715719, 2024-04-22T13:34:59.558 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:34:59.558 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 911, 2024-04-22T13:34:59.558 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.036720858, 2024-04-22T13:34:59.558 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000040308 2024-04-22T13:34:59.558 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.558 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:34:59.558 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.558 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.558 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.559 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.559 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-22T13:34:59.559 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 449264 2024-04-22T13:34:59.559 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.559 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2024-04-22T13:34:59.559 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 1, 2024-04-22T13:34:59.559 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2024-04-22T13:34:59.559 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.559 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-MDSRank": { 2024-04-22T13:34:59.559 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-04-22T13:34:59.559 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-04-22T13:34:59.559 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 4058, 2024-04-22T13:34:59.559 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.098964667, 2024-04-22T13:34:59.559 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000024387 2024-04-22T13:34:59.559 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:34:59.560 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.560 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-PurgeQueue": { 2024-04-22T13:34:59.560 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-04-22T13:34:59.560 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-04-22T13:34:59.560 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.560 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.560 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.560 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:34:59.560 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.560 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": { 2024-04-22T13:34:59.560 INFO:teuthology.orchestra.run.smithi111.stdout: "request": 0, 2024-04-22T13:34:59.560 INFO:teuthology.orchestra.run.smithi111.stdout: "reply": 0, 2024-04-22T13:34:59.561 INFO:teuthology.orchestra.run.smithi111.stdout: "reply_latency": { 2024-04-22T13:34:59.561 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.561 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.561 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.561 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.561 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_reply": 0, 2024-04-22T13:34:59.561 INFO:teuthology.orchestra.run.smithi111.stdout: "forward": 0, 2024-04-22T13:34:59.561 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_complete": 0, 2024-04-22T13:34:59.561 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_keys": 0, 2024-04-22T13:34:59.561 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_commit": 0, 2024-04-22T13:34:59.561 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_split": 0, 2024-04-22T13:34:59.561 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_merge": 0, 2024-04-22T13:34:59.561 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes": 31, 2024-04-22T13:34:59.561 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_top": 8, 2024-04-22T13:34:59.561 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_bottom": 23, 2024-04-22T13:34:59.562 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pin_tail": 0, 2024-04-22T13:34:59.562 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pinned": 20, 2024-04-22T13:34:59.562 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_expired": 0, 2024-04-22T13:34:59.562 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_with_caps": 0, 2024-04-22T13:34:59.562 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": 0, 2024-04-22T13:34:59.562 INFO:teuthology.orchestra.run.smithi111.stdout: "subtrees": 2, 2024-04-22T13:34:59.562 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse": 0, 2024-04-22T13:34:59.562 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_hit": 0, 2024-04-22T13:34:59.562 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_forward": 0, 2024-04-22T13:34:59.562 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_discover": 0, 2024-04-22T13:34:59.563 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_dir_fetch": 0, 2024-04-22T13:34:59.563 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_remote_ino": 0, 2024-04-22T13:34:59.563 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_lock": 0, 2024-04-22T13:34:59.563 INFO:teuthology.orchestra.run.smithi111.stdout: "load_cent": 0, 2024-04-22T13:34:59.563 INFO:teuthology.orchestra.run.smithi111.stdout: "q": 0, 2024-04-22T13:34:59.563 INFO:teuthology.orchestra.run.smithi111.stdout: "exported": 0, 2024-04-22T13:34:59.563 INFO:teuthology.orchestra.run.smithi111.stdout: "exported_inodes": 0, 2024-04-22T13:34:59.563 INFO:teuthology.orchestra.run.smithi111.stdout: "imported": 0, 2024-04-22T13:34:59.563 INFO:teuthology.orchestra.run.smithi111.stdout: "imported_inodes": 0, 2024-04-22T13:34:59.563 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_dir_fetch": 0, 2024-04-22T13:34:59.563 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_backtrace_fetch": 0, 2024-04-22T13:34:59.564 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_peer_discover": 0, 2024-04-22T13:34:59.564 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rfiles": 4, 2024-04-22T13:34:59.564 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rbytes": 234, 2024-04-22T13:34:59.564 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rsnaps": 6, 2024-04-22T13:34:59.564 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_fetch": 0, 2024-04-22T13:34:59.564 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_set_tag": 0, 2024-04-22T13:34:59.564 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_repaired": 0, 2024-04-22T13:34:59.564 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_inotable_repaired": 0, 2024-04-22T13:34:59.564 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_inodes": 0, 2024-04-22T13:34:59.564 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_base_inodes": 0, 2024-04-22T13:34:59.564 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dirfrag_rstats": 0, 2024-04-22T13:34:59.564 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_file_inodes": 0, 2024-04-22T13:34:59.564 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_inode_file_caps": 0, 2024-04-22T13:34:59.564 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_revoke": 0, 2024-04-22T13:34:59.564 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_grant": 0, 2024-04-22T13:34:59.565 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_trunc": 0, 2024-04-22T13:34:59.565 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-22T13:34:59.565 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-22T13:34:59.565 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps": 0, 2024-04-22T13:34:59.565 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps_dirty": 0, 2024-04-22T13:34:59.565 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_cap_release": 0, 2024-04-22T13:34:59.565 INFO:teuthology.orchestra.run.smithi111.stdout: "process_request_cap_release": 0 2024-04-22T13:34:59.565 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.565 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_cache": { 2024-04-22T13:34:59.565 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update": 0, 2024-04-22T13:34:59.565 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update_receipt": 0, 2024-04-22T13:34:59.565 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_try_discover": 0, 2024-04-22T13:34:59.565 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_send_discover": 0, 2024-04-22T13:34:59.565 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_handle_discover": 0, 2024-04-22T13:34:59.565 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays": 0, 2024-04-22T13:34:59.566 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_delayed": 0, 2024-04-22T13:34:59.566 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_enqueuing": 0, 2024-04-22T13:34:59.566 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_created": 0, 2024-04-22T13:34:59.566 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_enqueued": 0, 2024-04-22T13:34:59.566 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_reintegrated": 0, 2024-04-22T13:34:59.566 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_migrated": 0, 2024-04-22T13:34:59.566 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_processing": 0, 2024-04-22T13:34:59.566 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_enqueued": 0, 2024-04-22T13:34:59.566 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_prioritized": 0, 2024-04-22T13:34:59.566 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_started": 0, 2024-04-22T13:34:59.566 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_completed": 0, 2024-04-22T13:34:59.566 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_path": 0, 2024-04-22T13:34:59.566 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_inode": 0, 2024-04-22T13:34:59.566 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_enqueue_scrub": 0, 2024-04-22T13:34:59.567 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_exportdir": 0, 2024-04-22T13:34:59.567 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_flush": 0, 2024-04-22T13:34:59.567 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragmentdir": 0, 2024-04-22T13:34:59.567 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragstats": 0, 2024-04-22T13:34:59.567 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_inodestats": 0 2024-04-22T13:34:59.567 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.567 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_log": { 2024-04-22T13:34:59.567 INFO:teuthology.orchestra.run.smithi111.stdout: "evlrg": 0, 2024-04-22T13:34:59.567 INFO:teuthology.orchestra.run.smithi111.stdout: "evadd": 0, 2024-04-22T13:34:59.567 INFO:teuthology.orchestra.run.smithi111.stdout: "evex": 0, 2024-04-22T13:34:59.567 INFO:teuthology.orchestra.run.smithi111.stdout: "evtrm": 0, 2024-04-22T13:34:59.567 INFO:teuthology.orchestra.run.smithi111.stdout: "ev": 251, 2024-04-22T13:34:59.567 INFO:teuthology.orchestra.run.smithi111.stdout: "evexg": 0, 2024-04-22T13:34:59.567 INFO:teuthology.orchestra.run.smithi111.stdout: "evexd": 0, 2024-04-22T13:34:59.567 INFO:teuthology.orchestra.run.smithi111.stdout: "segadd": 0, 2024-04-22T13:34:59.567 INFO:teuthology.orchestra.run.smithi111.stdout: "segex": 0, 2024-04-22T13:34:59.568 INFO:teuthology.orchestra.run.smithi111.stdout: "segtrm": 0, 2024-04-22T13:34:59.568 INFO:teuthology.orchestra.run.smithi111.stdout: "seg": 2, 2024-04-22T13:34:59.568 INFO:teuthology.orchestra.run.smithi111.stdout: "segmjr": 13, 2024-04-22T13:34:59.568 INFO:teuthology.orchestra.run.smithi111.stdout: "segexg": 0, 2024-04-22T13:34:59.568 INFO:teuthology.orchestra.run.smithi111.stdout: "segexd": 0, 2024-04-22T13:34:59.568 INFO:teuthology.orchestra.run.smithi111.stdout: "expos": 5055868, 2024-04-22T13:34:59.568 INFO:teuthology.orchestra.run.smithi111.stdout: "wrpos": 5314079, 2024-04-22T13:34:59.568 INFO:teuthology.orchestra.run.smithi111.stdout: "rdpos": 5313967, 2024-04-22T13:34:59.568 INFO:teuthology.orchestra.run.smithi111.stdout: "jlat": { 2024-04-22T13:34:59.568 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.568 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.568 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.568 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.568 INFO:teuthology.orchestra.run.smithi111.stdout: "replayed": 251 2024-04-22T13:34:59.568 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.569 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_mem": { 2024-04-22T13:34:59.569 INFO:teuthology.orchestra.run.smithi111.stdout: "ino": 20, 2024-04-22T13:34:59.569 INFO:teuthology.orchestra.run.smithi111.stdout: "ino+": 24, 2024-04-22T13:34:59.569 INFO:teuthology.orchestra.run.smithi111.stdout: "ino-": 4, 2024-04-22T13:34:59.569 INFO:teuthology.orchestra.run.smithi111.stdout: "dir": 11, 2024-04-22T13:34:59.569 INFO:teuthology.orchestra.run.smithi111.stdout: "dir+": 11, 2024-04-22T13:34:59.569 INFO:teuthology.orchestra.run.smithi111.stdout: "dir-": 0, 2024-04-22T13:34:59.569 INFO:teuthology.orchestra.run.smithi111.stdout: "dn": 31, 2024-04-22T13:34:59.569 INFO:teuthology.orchestra.run.smithi111.stdout: "dn+": 31, 2024-04-22T13:34:59.569 INFO:teuthology.orchestra.run.smithi111.stdout: "dn-": 0, 2024-04-22T13:34:59.569 INFO:teuthology.orchestra.run.smithi111.stdout: "cap": 0, 2024-04-22T13:34:59.569 INFO:teuthology.orchestra.run.smithi111.stdout: "cap+": 0, 2024-04-22T13:34:59.569 INFO:teuthology.orchestra.run.smithi111.stdout: "cap-": 0, 2024-04-22T13:34:59.569 INFO:teuthology.orchestra.run.smithi111.stdout: "rss": 51520, 2024-04-22T13:34:59.570 INFO:teuthology.orchestra.run.smithi111.stdout: "heap": 215416 2024-04-22T13:34:59.570 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.570 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_server": { 2024-04-22T13:34:59.570 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_client_request": 0, 2024-04-22T13:34:59.570 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_server_request": 0, 2024-04-22T13:34:59.570 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_request": 0, 2024-04-22T13:34:59.570 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_session": 0, 2024-04-22T13:34:59.570 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_peer_request": 0, 2024-04-22T13:34:59.570 INFO:teuthology.orchestra.run.smithi111.stdout: "req_create_latency": { 2024-04-22T13:34:59.570 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.570 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.570 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.570 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.570 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getattr_latency": { 2024-04-22T13:34:59.571 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.571 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.571 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.571 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.571 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getfilelock_latency": { 2024-04-22T13:34:59.571 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.571 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.571 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.571 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.571 INFO:teuthology.orchestra.run.smithi111.stdout: "req_link_latency": { 2024-04-22T13:34:59.571 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.571 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.571 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.571 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.571 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookup_latency": { 2024-04-22T13:34:59.572 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.572 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.572 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.572 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.572 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookuphash_latency": { 2024-04-22T13:34:59.572 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.572 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.572 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.572 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.572 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupino_latency": { 2024-04-22T13:34:59.572 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.572 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.572 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.572 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.573 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupname_latency": { 2024-04-22T13:34:59.573 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.573 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.573 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.573 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.573 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupparent_latency": { 2024-04-22T13:34:59.573 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.573 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.573 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.573 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.573 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupsnap_latency": { 2024-04-22T13:34:59.573 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.573 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.573 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.573 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.573 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lssnap_latency": { 2024-04-22T13:34:59.574 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.574 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.574 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.574 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.574 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mkdir_latency": { 2024-04-22T13:34:59.574 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.575 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.575 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.575 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.575 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mknod_latency": { 2024-04-22T13:34:59.575 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.575 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.575 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.575 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.575 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mksnap_latency": { 2024-04-22T13:34:59.575 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.575 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.575 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.576 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.576 INFO:teuthology.orchestra.run.smithi111.stdout: "req_open_latency": { 2024-04-22T13:34:59.576 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.576 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.576 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.576 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.576 INFO:teuthology.orchestra.run.smithi111.stdout: "req_readdir_latency": { 2024-04-22T13:34:59.576 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.576 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.576 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.576 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.576 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rename_latency": { 2024-04-22T13:34:59.576 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.576 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.576 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.577 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.577 INFO:teuthology.orchestra.run.smithi111.stdout: "req_renamesnap_latency": { 2024-04-22T13:34:59.577 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.577 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.577 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.577 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.577 INFO:teuthology.orchestra.run.smithi111.stdout: "req_snapdiff_latency": { 2024-04-22T13:34:59.577 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.577 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.577 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.577 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.577 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmdir_latency": { 2024-04-22T13:34:59.577 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.577 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.577 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.578 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.578 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmsnap_latency": { 2024-04-22T13:34:59.578 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.578 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.578 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.578 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.578 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmxattr_latency": { 2024-04-22T13:34:59.578 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.578 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.578 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.578 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.578 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setattr_latency": { 2024-04-22T13:34:59.578 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.578 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.578 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.579 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.579 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setdirlayout_latency": { 2024-04-22T13:34:59.579 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.579 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.579 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.579 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.579 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setfilelock_latency": { 2024-04-22T13:34:59.579 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.579 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.579 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.579 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.579 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setlayout_latency": { 2024-04-22T13:34:59.579 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.579 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.580 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.580 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.580 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setxattr_latency": { 2024-04-22T13:34:59.580 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.580 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.580 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.580 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.580 INFO:teuthology.orchestra.run.smithi111.stdout: "req_symlink_latency": { 2024-04-22T13:34:59.580 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.580 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.580 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.580 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.580 INFO:teuthology.orchestra.run.smithi111.stdout: "req_unlink_latency": { 2024-04-22T13:34:59.580 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.580 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.581 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.581 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.581 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_revoke_eviction": 0, 2024-04-22T13:34:59.581 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_acquisition_throttle": 0, 2024-04-22T13:34:59.581 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getvxattr_latency": { 2024-04-22T13:34:59.581 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.581 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.581 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.581 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:34:59.581 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.581 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_sessions": { 2024-04-22T13:34:59.581 INFO:teuthology.orchestra.run.smithi111.stdout: "session_count": 2, 2024-04-22T13:34:59.581 INFO:teuthology.orchestra.run.smithi111.stdout: "session_add": 5, 2024-04-22T13:34:59.581 INFO:teuthology.orchestra.run.smithi111.stdout: "session_remove": 3, 2024-04-22T13:34:59.581 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_open": 4, 2024-04-22T13:34:59.582 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_stale": 0, 2024-04-22T13:34:59.582 INFO:teuthology.orchestra.run.smithi111.stdout: "total_load": 0, 2024-04-22T13:34:59.582 INFO:teuthology.orchestra.run.smithi111.stdout: "average_load": 0, 2024-04-22T13:34:59.582 INFO:teuthology.orchestra.run.smithi111.stdout: "avg_session_uptime": 2934, 2024-04-22T13:34:59.582 INFO:teuthology.orchestra.run.smithi111.stdout: "mdthresh_evicted": 0 2024-04-22T13:34:59.582 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.582 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2024-04-22T13:34:59.582 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 0, 2024-04-22T13:34:59.582 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 0, 2024-04-22T13:34:59.582 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T13:34:59.582 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2024-04-22T13:34:59.582 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T13:34:59.582 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2024-04-22T13:34:59.583 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T13:34:59.583 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T13:34:59.583 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T13:34:59.583 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T13:34:59.583 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T13:34:59.583 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2024-04-22T13:34:59.583 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T13:34:59.583 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T13:34:59.583 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2024-04-22T13:34:59.583 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2024-04-22T13:34:59.583 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2024-04-22T13:34:59.583 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2024-04-22T13:34:59.583 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T13:34:59.583 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T13:34:59.584 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T13:34:59.584 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T13:34:59.584 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T13:34:59.584 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2024-04-22T13:34:59.584 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2024-04-22T13:34:59.584 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2024-04-22T13:34:59.584 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T13:34:59.584 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T13:34:59.584 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2024-04-22T13:34:59.584 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2024-04-22T13:34:59.584 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2024-04-22T13:34:59.584 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2024-04-22T13:34:59.584 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T13:34:59.584 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2024-04-22T13:34:59.584 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T13:34:59.584 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2024-04-22T13:34:59.584 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 189726, 2024-04-22T13:34:59.585 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 60, 2024-04-22T13:34:59.585 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 0, 2024-04-22T13:34:59.585 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 0, 2024-04-22T13:34:59.585 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2024-04-22T13:34:59.585 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2024-04-22T13:34:59.585 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2024-04-22T13:34:59.585 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2024-04-22T13:34:59.585 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2024-04-22T13:34:59.585 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2024-04-22T13:34:59.585 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 7936, 2024-04-22T13:34:59.585 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 165, 2024-04-22T13:34:59.585 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T13:34:59.585 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2024-04-22T13:34:59.585 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2024-04-22T13:34:59.585 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2024-04-22T13:34:59.585 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 94397, 2024-04-22T13:34:59.586 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 754, 2024-04-22T13:34:59.586 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2024-04-22T13:34:59.586 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2024-04-22T13:34:59.586 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2024-04-22T13:34:59.586 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2024-04-22T13:34:59.586 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.586 INFO:teuthology.orchestra.run.smithi111.stdout: "objecter": { 2024-04-22T13:34:59.586 INFO:teuthology.orchestra.run.smithi111.stdout: "op_active": 0, 2024-04-22T13:34:59.586 INFO:teuthology.orchestra.run.smithi111.stdout: "op_laggy": 0, 2024-04-22T13:34:59.586 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send": 2411, 2024-04-22T13:34:59.586 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send_bytes": 26, 2024-04-22T13:34:59.586 INFO:teuthology.orchestra.run.smithi111.stdout: "op_resend": 4, 2024-04-22T13:34:59.586 INFO:teuthology.orchestra.run.smithi111.stdout: "op_reply": 2411, 2024-04-22T13:34:59.586 INFO:teuthology.orchestra.run.smithi111.stdout: "op_latency": { 2024-04-22T13:34:59.586 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2411, 2024-04-22T13:34:59.587 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 1.215976874, 2024-04-22T13:34:59.587 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000504345 2024-04-22T13:34:59.587 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.587 INFO:teuthology.orchestra.run.smithi111.stdout: "op_inflight": 0, 2024-04-22T13:34:59.587 INFO:teuthology.orchestra.run.smithi111.stdout: "oplen_avg": { 2024-04-22T13:34:59.587 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2411, 2024-04-22T13:34:59.587 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 2412 2024-04-22T13:34:59.587 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.587 INFO:teuthology.orchestra.run.smithi111.stdout: "op": 2411, 2024-04-22T13:34:59.587 INFO:teuthology.orchestra.run.smithi111.stdout: "op_r": 2411, 2024-04-22T13:34:59.587 INFO:teuthology.orchestra.run.smithi111.stdout: "op_w": 0, 2024-04-22T13:34:59.587 INFO:teuthology.orchestra.run.smithi111.stdout: "op_rmw": 0, 2024-04-22T13:34:59.587 INFO:teuthology.orchestra.run.smithi111.stdout: "op_pg": 0, 2024-04-22T13:34:59.587 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_stat": 1517, 2024-04-22T13:34:59.587 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_create": 0, 2024-04-22T13:34:59.587 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_read": 891, 2024-04-22T13:34:59.588 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_write": 0, 2024-04-22T13:34:59.588 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writefull": 0, 2024-04-22T13:34:59.588 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writesame": 0, 2024-04-22T13:34:59.588 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_append": 0, 2024-04-22T13:34:59.588 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_zero": 0, 2024-04-22T13:34:59.588 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_truncate": 0, 2024-04-22T13:34:59.588 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_delete": 0, 2024-04-22T13:34:59.588 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_mapext": 0, 2024-04-22T13:34:59.588 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_sparse_read": 0, 2024-04-22T13:34:59.588 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_clonerange": 0, 2024-04-22T13:34:59.589 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_getxattr": 2, 2024-04-22T13:34:59.589 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_setxattr": 0, 2024-04-22T13:34:59.589 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_cmpxattr": 0, 2024-04-22T13:34:59.589 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_rmxattr": 0, 2024-04-22T13:34:59.589 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_resetxattrs": 0, 2024-04-22T13:34:59.589 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_call": 0, 2024-04-22T13:34:59.589 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_watch": 0, 2024-04-22T13:34:59.589 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_notify": 0, 2024-04-22T13:34:59.589 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_src_cmpxattr": 0, 2024-04-22T13:34:59.589 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls": 0, 2024-04-22T13:34:59.589 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls_filter": 0, 2024-04-22T13:34:59.589 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_other": 0, 2024-04-22T13:34:59.589 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_active": 0, 2024-04-22T13:34:59.589 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_send": 0, 2024-04-22T13:34:59.589 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_resend": 0, 2024-04-22T13:34:59.590 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_ping": 0, 2024-04-22T13:34:59.590 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_active": 0, 2024-04-22T13:34:59.590 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_send": 0, 2024-04-22T13:34:59.590 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_resend": 0, 2024-04-22T13:34:59.590 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_active": 0, 2024-04-22T13:34:59.590 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_send": 0, 2024-04-22T13:34:59.590 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_resend": 0, 2024-04-22T13:34:59.590 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_active": 0, 2024-04-22T13:34:59.590 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_send": 0, 2024-04-22T13:34:59.590 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_resend": 0, 2024-04-22T13:34:59.590 INFO:teuthology.orchestra.run.smithi111.stdout: "command_active": 0, 2024-04-22T13:34:59.590 INFO:teuthology.orchestra.run.smithi111.stdout: "command_send": 0, 2024-04-22T13:34:59.590 INFO:teuthology.orchestra.run.smithi111.stdout: "command_resend": 0, 2024-04-22T13:34:59.591 INFO:teuthology.orchestra.run.smithi111.stdout: "map_epoch": 98, 2024-04-22T13:34:59.591 INFO:teuthology.orchestra.run.smithi111.stdout: "map_full": 0, 2024-04-22T13:34:59.591 INFO:teuthology.orchestra.run.smithi111.stdout: "map_inc": 11, 2024-04-22T13:34:59.591 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_sessions": 5, 2024-04-22T13:34:59.591 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_open": 5, 2024-04-22T13:34:59.591 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_close": 0, 2024-04-22T13:34:59.591 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_laggy": 0, 2024-04-22T13:34:59.591 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_wr": 0, 2024-04-22T13:34:59.591 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_rd": 2, 2024-04-22T13:34:59.591 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_del": 0 2024-04-22T13:34:59.591 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.591 INFO:teuthology.orchestra.run.smithi111.stdout: "oft": { 2024-04-22T13:34:59.591 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_objs": 0, 2024-04-22T13:34:59.591 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_kv_pairs": 0, 2024-04-22T13:34:59.592 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_updates": 0, 2024-04-22T13:34:59.592 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_removes": 0 2024-04-22T13:34:59.592 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.592 INFO:teuthology.orchestra.run.smithi111.stdout: "osd-slow-ops": { 2024-04-22T13:34:59.592 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_ops_count": 0 2024-04-22T13:34:59.592 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.592 INFO:teuthology.orchestra.run.smithi111.stdout: "purge_queue": { 2024-04-22T13:34:59.592 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops": 0, 2024-04-22T13:34:59.592 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops_high_water": 0, 2024-04-22T13:34:59.592 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing": 0, 2024-04-22T13:34:59.592 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_high_water": 0, 2024-04-22T13:34:59.592 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed_ops": 0, 2024-04-22T13:34:59.592 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed": 0, 2024-04-22T13:34:59.592 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_item_in_journal": 0 2024-04-22T13:34:59.592 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.593 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T13:34:59.593 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-22T13:34:59.593 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-04-22T13:34:59.593 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-22T13:34:59.593 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 2662, 2024-04-22T13:34:59.593 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 3192944, 2024-04-22T13:34:59.593 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-22T13:34:59.593 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 2662, 2024-04-22T13:34:59.593 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-04-22T13:34:59.593 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-04-22T13:34:59.593 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 2662, 2024-04-22T13:34:59.593 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 3192944, 2024-04-22T13:34:59.593 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-22T13:34:59.593 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.593 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.593 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.593 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:34:59.594 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.594 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_bytes": { 2024-04-22T13:34:59.594 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-22T13:34:59.594 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-04-22T13:34:59.594 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-22T13:34:59.594 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-04-22T13:34:59.594 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-04-22T13:34:59.594 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-22T13:34:59.594 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-04-22T13:34:59.594 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 2411, 2024-04-22T13:34:59.594 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 1119785, 2024-04-22T13:34:59.594 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 2411, 2024-04-22T13:34:59.594 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 1119785, 2024-04-22T13:34:59.594 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-22T13:34:59.594 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.595 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.595 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.595 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:34:59.595 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.595 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_ops": { 2024-04-22T13:34:59.595 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-22T13:34:59.595 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 1024, 2024-04-22T13:34:59.595 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-22T13:34:59.595 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-04-22T13:34:59.595 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-04-22T13:34:59.595 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-22T13:34:59.595 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-04-22T13:34:59.595 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 2411, 2024-04-22T13:34:59.595 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 2411, 2024-04-22T13:34:59.595 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 2411, 2024-04-22T13:34:59.595 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 2411, 2024-04-22T13:34:59.595 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-22T13:34:59.596 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.596 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.596 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.596 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:34:59.596 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.596 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle": { 2024-04-22T13:34:59.596 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-22T13:34:59.596 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-04-22T13:34:59.596 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-22T13:34:59.596 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-04-22T13:34:59.596 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-04-22T13:34:59.596 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-22T13:34:59.596 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-04-22T13:34:59.596 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-04-22T13:34:59.596 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-04-22T13:34:59.596 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 0, 2024-04-22T13:34:59.596 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 0, 2024-04-22T13:34:59.597 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-22T13:34:59.597 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.597 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.597 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.597 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:34:59.597 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:34:59.597 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle-0x55a1914d06e0": { 2024-04-22T13:34:59.597 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-22T13:34:59.597 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-04-22T13:34:59.597 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-22T13:34:59.597 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-04-22T13:34:59.597 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-04-22T13:34:59.597 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-22T13:34:59.597 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-04-22T13:34:59.597 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-04-22T13:34:59.597 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-04-22T13:34:59.598 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 0, 2024-04-22T13:34:59.598 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 0, 2024-04-22T13:34:59.598 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-22T13:34:59.598 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:34:59.598 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:34:59.598 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:34:59.598 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:34:59.598 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:34:59.598 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-04-22T13:34:59.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:34:59 smithi134 ceph-mon[31338]: pgmap v337: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:34:59.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:34:59 smithi111 ceph-mon[31361]: pgmap v337: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:35:00.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:34:59 smithi012 ceph-mon[28265]: pgmap v337: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-22T13:35:00.324 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-22T13:35:00.324 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-22T13:35:00.324 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-22T13:35:00.324 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-22T13:35:00.324 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-22T13:35:00.324 DEBUG:tasks.check_counter:Getting stats from k 2024-04-22T13:35:00.325 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-22T13:35:00.564 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.b/config 2024-04-22T13:35:01.721 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:01 smithi111 ceph-mon[31361]: pgmap v338: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:35:01.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:01 smithi134 ceph-mon[31338]: pgmap v338: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:35:02.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:01 smithi012 ceph-mon[28265]: pgmap v338: 97 pgs: 97 active+clean; 2.3 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-22T13:35:02.294 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-04-22T13:35:02.294 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T13:35:02.294 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 73, 2024-04-22T13:35:02.295 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 79, 2024-04-22T13:35:02.295 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 15281, 2024-04-22T13:35:02.295 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 573761, 2024-04-22T13:35:02.295 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 58, 2024-04-22T13:35:02.295 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 7, 2024-04-22T13:35:02.295 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.023281810, 2024-04-22T13:35:02.295 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.001734773, 2024-04-22T13:35:02.295 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.016101005, 2024-04-22T13:35:02.295 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000319895, 2024-04-22T13:35:02.295 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:35:02.295 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 73, 2024-04-22T13:35:02.295 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.008815413, 2024-04-22T13:35:02.295 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000120759 2024-04-22T13:35:02.296 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.296 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:35:02.296 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 56, 2024-04-22T13:35:02.296 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000005805, 2024-04-22T13:35:02.296 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000103 2024-04-22T13:35:02.296 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.296 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 3920, 2024-04-22T13:35:02.296 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 1424 2024-04-22T13:35:02.296 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.296 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T13:35:02.296 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 388, 2024-04-22T13:35:02.296 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 568, 2024-04-22T13:35:02.296 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 108363, 2024-04-22T13:35:02.296 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 252272, 2024-04-22T13:35:02.297 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 8, 2024-04-22T13:35:02.297 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 3, 2024-04-22T13:35:02.297 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.060313914, 2024-04-22T13:35:02.297 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.028876016, 2024-04-22T13:35:02.297 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.025503199, 2024-04-22T13:35:02.297 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.004123282, 2024-04-22T13:35:02.297 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:35:02.297 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 570, 2024-04-22T13:35:02.297 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.037156392, 2024-04-22T13:35:02.297 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000065186 2024-04-22T13:35:02.297 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.297 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:35:02.297 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 891, 2024-04-22T13:35:02.297 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000229380, 2024-04-22T13:35:02.298 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000257 2024-04-22T13:35:02.298 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.298 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-04-22T13:35:02.298 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 129584 2024-04-22T13:35:02.298 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.298 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T13:35:02.298 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 294, 2024-04-22T13:35:02.298 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 412, 2024-04-22T13:35:02.298 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 1640001, 2024-04-22T13:35:02.298 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 473077, 2024-04-22T13:35:02.298 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 6, 2024-04-22T13:35:02.298 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 5, 2024-04-22T13:35:02.298 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.059118617, 2024-04-22T13:35:02.298 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.020355059, 2024-04-22T13:35:02.299 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.043501636, 2024-04-22T13:35:02.299 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.005735187, 2024-04-22T13:35:02.299 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:35:02.299 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 411, 2024-04-22T13:35:02.299 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.077007635, 2024-04-22T13:35:02.299 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000187366 2024-04-22T13:35:02.299 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.299 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:35:02.299 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.299 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.299 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.299 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.299 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 1635920, 2024-04-22T13:35:02.299 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 465712 2024-04-22T13:35:02.299 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.300 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2024-04-22T13:35:02.300 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 1, 2024-04-22T13:35:02.300 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2024-04-22T13:35:02.300 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.300 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-MDSRank": { 2024-04-22T13:35:02.300 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-04-22T13:35:02.300 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-04-22T13:35:02.300 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 52, 2024-04-22T13:35:02.300 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.001520264, 2024-04-22T13:35:02.300 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000029235 2024-04-22T13:35:02.300 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:35:02.300 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.300 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-PurgeQueue": { 2024-04-22T13:35:02.300 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2024-04-22T13:35:02.301 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2024-04-22T13:35:02.301 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2, 2024-04-22T13:35:02.301 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000005627, 2024-04-22T13:35:02.301 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000002813 2024-04-22T13:35:02.301 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:35:02.301 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.301 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": { 2024-04-22T13:35:02.301 INFO:teuthology.orchestra.run.smithi111.stdout: "request": 0, 2024-04-22T13:35:02.301 INFO:teuthology.orchestra.run.smithi111.stdout: "reply": 0, 2024-04-22T13:35:02.301 INFO:teuthology.orchestra.run.smithi111.stdout: "reply_latency": { 2024-04-22T13:35:02.301 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.301 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.301 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.301 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.301 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_reply": 0, 2024-04-22T13:35:02.301 INFO:teuthology.orchestra.run.smithi111.stdout: "forward": 0, 2024-04-22T13:35:02.302 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_complete": 0, 2024-04-22T13:35:02.302 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_keys": 0, 2024-04-22T13:35:02.302 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_commit": 11, 2024-04-22T13:35:02.302 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_split": 0, 2024-04-22T13:35:02.302 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_merge": 0, 2024-04-22T13:35:02.302 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes": 10, 2024-04-22T13:35:02.302 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_top": 0, 2024-04-22T13:35:02.302 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_bottom": 10, 2024-04-22T13:35:02.302 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pin_tail": 0, 2024-04-22T13:35:02.302 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pinned": 10, 2024-04-22T13:35:02.302 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_expired": 0, 2024-04-22T13:35:02.302 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_with_caps": 0, 2024-04-22T13:35:02.302 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": 0, 2024-04-22T13:35:02.302 INFO:teuthology.orchestra.run.smithi111.stdout: "subtrees": 1, 2024-04-22T13:35:02.303 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse": 4, 2024-04-22T13:35:02.303 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_hit": 4, 2024-04-22T13:35:02.303 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_forward": 0, 2024-04-22T13:35:02.303 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_discover": 0, 2024-04-22T13:35:02.303 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_dir_fetch": 0, 2024-04-22T13:35:02.303 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_remote_ino": 0, 2024-04-22T13:35:02.303 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_lock": 0, 2024-04-22T13:35:02.303 INFO:teuthology.orchestra.run.smithi111.stdout: "load_cent": 0, 2024-04-22T13:35:02.303 INFO:teuthology.orchestra.run.smithi111.stdout: "q": 0, 2024-04-22T13:35:02.303 INFO:teuthology.orchestra.run.smithi111.stdout: "exported": 0, 2024-04-22T13:35:02.303 INFO:teuthology.orchestra.run.smithi111.stdout: "exported_inodes": 0, 2024-04-22T13:35:02.303 INFO:teuthology.orchestra.run.smithi111.stdout: "imported": 0, 2024-04-22T13:35:02.303 INFO:teuthology.orchestra.run.smithi111.stdout: "imported_inodes": 0, 2024-04-22T13:35:02.303 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_dir_fetch": 0, 2024-04-22T13:35:02.303 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_backtrace_fetch": 0, 2024-04-22T13:35:02.304 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_peer_discover": 0, 2024-04-22T13:35:02.304 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rfiles": 4, 2024-04-22T13:35:02.304 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rbytes": 234, 2024-04-22T13:35:02.304 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rsnaps": 6, 2024-04-22T13:35:02.304 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_fetch": 0, 2024-04-22T13:35:02.304 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_set_tag": 0, 2024-04-22T13:35:02.304 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_repaired": 0, 2024-04-22T13:35:02.304 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_inotable_repaired": 0, 2024-04-22T13:35:02.304 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_inodes": 0, 2024-04-22T13:35:02.304 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_base_inodes": 0, 2024-04-22T13:35:02.304 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dirfrag_rstats": 0, 2024-04-22T13:35:02.304 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_file_inodes": 0, 2024-04-22T13:35:02.304 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_inode_file_caps": 0, 2024-04-22T13:35:02.304 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_revoke": 0, 2024-04-22T13:35:02.305 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_grant": 0, 2024-04-22T13:35:02.305 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_trunc": 0, 2024-04-22T13:35:02.305 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-22T13:35:02.305 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-22T13:35:02.305 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps": 0, 2024-04-22T13:35:02.305 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps_dirty": 0, 2024-04-22T13:35:02.305 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_cap_release": 0, 2024-04-22T13:35:02.305 INFO:teuthology.orchestra.run.smithi111.stdout: "process_request_cap_release": 0 2024-04-22T13:35:02.305 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.305 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_cache": { 2024-04-22T13:35:02.305 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update": 0, 2024-04-22T13:35:02.305 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update_receipt": 0, 2024-04-22T13:35:02.305 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_try_discover": 0, 2024-04-22T13:35:02.306 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_send_discover": 1, 2024-04-22T13:35:02.306 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_handle_discover": 0, 2024-04-22T13:35:02.306 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays": 0, 2024-04-22T13:35:02.306 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_delayed": 0, 2024-04-22T13:35:02.306 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_enqueuing": 0, 2024-04-22T13:35:02.306 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_created": 0, 2024-04-22T13:35:02.306 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_enqueued": 0, 2024-04-22T13:35:02.306 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_reintegrated": 0, 2024-04-22T13:35:02.306 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_migrated": 0, 2024-04-22T13:35:02.306 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_processing": 0, 2024-04-22T13:35:02.306 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_enqueued": 0, 2024-04-22T13:35:02.306 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_prioritized": 0, 2024-04-22T13:35:02.306 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_started": 0, 2024-04-22T13:35:02.306 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_completed": 0, 2024-04-22T13:35:02.307 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_path": 2, 2024-04-22T13:35:02.307 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_quiesce_inode": 2, 2024-04-22T13:35:02.307 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_enqueue_scrub": 0, 2024-04-22T13:35:02.307 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_exportdir": 0, 2024-04-22T13:35:02.307 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_flush": 0, 2024-04-22T13:35:02.307 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragmentdir": 0, 2024-04-22T13:35:02.307 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragstats": 0, 2024-04-22T13:35:02.307 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_inodestats": 0 2024-04-22T13:35:02.307 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.307 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_log": { 2024-04-22T13:35:02.307 INFO:teuthology.orchestra.run.smithi111.stdout: "evlrg": 0, 2024-04-22T13:35:02.307 INFO:teuthology.orchestra.run.smithi111.stdout: "evadd": 6, 2024-04-22T13:35:02.307 INFO:teuthology.orchestra.run.smithi111.stdout: "evex": 0, 2024-04-22T13:35:02.307 INFO:teuthology.orchestra.run.smithi111.stdout: "evtrm": 0, 2024-04-22T13:35:02.308 INFO:teuthology.orchestra.run.smithi111.stdout: "ev": 6, 2024-04-22T13:35:02.308 INFO:teuthology.orchestra.run.smithi111.stdout: "evexg": 0, 2024-04-22T13:35:02.308 INFO:teuthology.orchestra.run.smithi111.stdout: "evexd": 0, 2024-04-22T13:35:02.308 INFO:teuthology.orchestra.run.smithi111.stdout: "segadd": 2, 2024-04-22T13:35:02.308 INFO:teuthology.orchestra.run.smithi111.stdout: "segex": 0, 2024-04-22T13:35:02.308 INFO:teuthology.orchestra.run.smithi111.stdout: "segtrm": 0, 2024-04-22T13:35:02.308 INFO:teuthology.orchestra.run.smithi111.stdout: "seg": 2, 2024-04-22T13:35:02.308 INFO:teuthology.orchestra.run.smithi111.stdout: "segmjr": 2, 2024-04-22T13:35:02.308 INFO:teuthology.orchestra.run.smithi111.stdout: "segexg": 0, 2024-04-22T13:35:02.308 INFO:teuthology.orchestra.run.smithi111.stdout: "segexd": 0, 2024-04-22T13:35:02.308 INFO:teuthology.orchestra.run.smithi111.stdout: "expos": 4194304, 2024-04-22T13:35:02.308 INFO:teuthology.orchestra.run.smithi111.stdout: "wrpos": 4195956, 2024-04-22T13:35:02.308 INFO:teuthology.orchestra.run.smithi111.stdout: "rdpos": 0, 2024-04-22T13:35:02.308 INFO:teuthology.orchestra.run.smithi111.stdout: "jlat": { 2024-04-22T13:35:02.308 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 5, 2024-04-22T13:35:02.309 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.010729665, 2024-04-22T13:35:02.309 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.002145933 2024-04-22T13:35:02.309 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.309 INFO:teuthology.orchestra.run.smithi111.stdout: "replayed": 0 2024-04-22T13:35:02.309 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.309 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_mem": { 2024-04-22T13:35:02.309 INFO:teuthology.orchestra.run.smithi111.stdout: "ino": 13, 2024-04-22T13:35:02.309 INFO:teuthology.orchestra.run.smithi111.stdout: "ino+": 13, 2024-04-22T13:35:02.309 INFO:teuthology.orchestra.run.smithi111.stdout: "ino-": 0, 2024-04-22T13:35:02.309 INFO:teuthology.orchestra.run.smithi111.stdout: "dir": 11, 2024-04-22T13:35:02.309 INFO:teuthology.orchestra.run.smithi111.stdout: "dir+": 12, 2024-04-22T13:35:02.309 INFO:teuthology.orchestra.run.smithi111.stdout: "dir-": 1, 2024-04-22T13:35:02.309 INFO:teuthology.orchestra.run.smithi111.stdout: "dn": 10, 2024-04-22T13:35:02.309 INFO:teuthology.orchestra.run.smithi111.stdout: "dn+": 10, 2024-04-22T13:35:02.309 INFO:teuthology.orchestra.run.smithi111.stdout: "dn-": 0, 2024-04-22T13:35:02.310 INFO:teuthology.orchestra.run.smithi111.stdout: "cap": 0, 2024-04-22T13:35:02.310 INFO:teuthology.orchestra.run.smithi111.stdout: "cap+": 0, 2024-04-22T13:35:02.310 INFO:teuthology.orchestra.run.smithi111.stdout: "cap-": 0, 2024-04-22T13:35:02.310 INFO:teuthology.orchestra.run.smithi111.stdout: "rss": 51856, 2024-04-22T13:35:02.310 INFO:teuthology.orchestra.run.smithi111.stdout: "heap": 223608 2024-04-22T13:35:02.310 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.310 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_server": { 2024-04-22T13:35:02.310 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_client_request": 0, 2024-04-22T13:35:02.310 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_server_request": 0, 2024-04-22T13:35:02.310 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_request": 0, 2024-04-22T13:35:02.310 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_session": 8, 2024-04-22T13:35:02.310 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_peer_request": 2, 2024-04-22T13:35:02.310 INFO:teuthology.orchestra.run.smithi111.stdout: "req_create_latency": { 2024-04-22T13:35:02.310 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.311 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.311 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.311 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.311 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getattr_latency": { 2024-04-22T13:35:02.311 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.311 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.311 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.311 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.311 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getfilelock_latency": { 2024-04-22T13:35:02.311 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.311 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.311 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.311 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.311 INFO:teuthology.orchestra.run.smithi111.stdout: "req_link_latency": { 2024-04-22T13:35:02.311 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.311 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.311 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.312 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.312 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookup_latency": { 2024-04-22T13:35:02.312 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.312 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.312 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.312 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.312 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookuphash_latency": { 2024-04-22T13:35:02.312 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.312 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.312 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.312 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.312 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupino_latency": { 2024-04-22T13:35:02.312 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.312 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.312 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.313 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.313 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupname_latency": { 2024-04-22T13:35:02.313 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.313 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.313 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.313 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.313 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupparent_latency": { 2024-04-22T13:35:02.313 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.313 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.313 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.313 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.313 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupsnap_latency": { 2024-04-22T13:35:02.313 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.313 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.313 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.313 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.314 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lssnap_latency": { 2024-04-22T13:35:02.314 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.314 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.314 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.314 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.314 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mkdir_latency": { 2024-04-22T13:35:02.314 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.314 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.315 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.315 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.315 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mknod_latency": { 2024-04-22T13:35:02.315 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.315 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.315 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.315 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.315 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mksnap_latency": { 2024-04-22T13:35:02.315 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.315 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.315 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.315 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.315 INFO:teuthology.orchestra.run.smithi111.stdout: "req_open_latency": { 2024-04-22T13:35:02.315 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.315 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.316 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.316 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.316 INFO:teuthology.orchestra.run.smithi111.stdout: "req_readdir_latency": { 2024-04-22T13:35:02.316 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.316 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.316 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.316 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.316 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rename_latency": { 2024-04-22T13:35:02.316 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.316 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.316 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.316 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.316 INFO:teuthology.orchestra.run.smithi111.stdout: "req_renamesnap_latency": { 2024-04-22T13:35:02.316 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.316 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.317 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.317 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.317 INFO:teuthology.orchestra.run.smithi111.stdout: "req_snapdiff_latency": { 2024-04-22T13:35:02.317 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.317 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.317 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.317 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.317 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmdir_latency": { 2024-04-22T13:35:02.317 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.317 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.317 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.317 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.317 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmsnap_latency": { 2024-04-22T13:35:02.317 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.317 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.317 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.318 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.318 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmxattr_latency": { 2024-04-22T13:35:02.318 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.318 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.318 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.318 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.318 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setattr_latency": { 2024-04-22T13:35:02.318 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.318 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.318 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.318 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.318 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setdirlayout_latency": { 2024-04-22T13:35:02.318 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.318 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.319 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.319 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.319 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setfilelock_latency": { 2024-04-22T13:35:02.319 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.319 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.319 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.319 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.319 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setlayout_latency": { 2024-04-22T13:35:02.319 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.319 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.319 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.319 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.319 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setxattr_latency": { 2024-04-22T13:35:02.319 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.319 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.319 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.320 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.320 INFO:teuthology.orchestra.run.smithi111.stdout: "req_symlink_latency": { 2024-04-22T13:35:02.320 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.320 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.320 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.320 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.320 INFO:teuthology.orchestra.run.smithi111.stdout: "req_unlink_latency": { 2024-04-22T13:35:02.320 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.320 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.320 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.320 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.320 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_revoke_eviction": 0, 2024-04-22T13:35:02.320 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_acquisition_throttle": 0, 2024-04-22T13:35:02.320 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getvxattr_latency": { 2024-04-22T13:35:02.320 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.320 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.320 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.321 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:35:02.321 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.321 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_sessions": { 2024-04-22T13:35:02.321 INFO:teuthology.orchestra.run.smithi111.stdout: "session_count": 0, 2024-04-22T13:35:02.321 INFO:teuthology.orchestra.run.smithi111.stdout: "session_add": 2, 2024-04-22T13:35:02.321 INFO:teuthology.orchestra.run.smithi111.stdout: "session_remove": 2, 2024-04-22T13:35:02.321 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_open": 0, 2024-04-22T13:35:02.321 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_stale": 0, 2024-04-22T13:35:02.321 INFO:teuthology.orchestra.run.smithi111.stdout: "total_load": 0, 2024-04-22T13:35:02.321 INFO:teuthology.orchestra.run.smithi111.stdout: "average_load": 0, 2024-04-22T13:35:02.321 INFO:teuthology.orchestra.run.smithi111.stdout: "avg_session_uptime": 85, 2024-04-22T13:35:02.321 INFO:teuthology.orchestra.run.smithi111.stdout: "mdthresh_evicted": 0 2024-04-22T13:35:02.321 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.321 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2024-04-22T13:35:02.321 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 0, 2024-04-22T13:35:02.322 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 0, 2024-04-22T13:35:02.322 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T13:35:02.322 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2024-04-22T13:35:02.322 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T13:35:02.322 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2024-04-22T13:35:02.322 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T13:35:02.322 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T13:35:02.322 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T13:35:02.322 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T13:35:02.322 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T13:35:02.322 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2024-04-22T13:35:02.322 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T13:35:02.322 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T13:35:02.322 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2024-04-22T13:35:02.322 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2024-04-22T13:35:02.323 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2024-04-22T13:35:02.323 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2024-04-22T13:35:02.323 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T13:35:02.323 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T13:35:02.323 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T13:35:02.323 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T13:35:02.323 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T13:35:02.323 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2024-04-22T13:35:02.323 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2024-04-22T13:35:02.323 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2024-04-22T13:35:02.323 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T13:35:02.323 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T13:35:02.323 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2024-04-22T13:35:02.323 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2024-04-22T13:35:02.323 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2024-04-22T13:35:02.324 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2024-04-22T13:35:02.324 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T13:35:02.324 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2024-04-22T13:35:02.324 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T13:35:02.324 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2024-04-22T13:35:02.324 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 238279, 2024-04-22T13:35:02.324 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 77, 2024-04-22T13:35:02.324 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 0, 2024-04-22T13:35:02.324 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 0, 2024-04-22T13:35:02.324 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2024-04-22T13:35:02.324 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2024-04-22T13:35:02.324 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2024-04-22T13:35:02.324 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2024-04-22T13:35:02.324 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2024-04-22T13:35:02.324 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2024-04-22T13:35:02.325 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 7936, 2024-04-22T13:35:02.325 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 165, 2024-04-22T13:35:02.325 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T13:35:02.325 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2024-04-22T13:35:02.325 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2024-04-22T13:35:02.325 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2024-04-22T13:35:02.325 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 50880, 2024-04-22T13:35:02.325 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 199, 2024-04-22T13:35:02.325 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2024-04-22T13:35:02.325 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2024-04-22T13:35:02.325 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2024-04-22T13:35:02.325 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2024-04-22T13:35:02.325 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.325 INFO:teuthology.orchestra.run.smithi111.stdout: "objecter": { 2024-04-22T13:35:02.325 INFO:teuthology.orchestra.run.smithi111.stdout: "op_active": 0, 2024-04-22T13:35:02.326 INFO:teuthology.orchestra.run.smithi111.stdout: "op_laggy": 0, 2024-04-22T13:35:02.326 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send": 41, 2024-04-22T13:35:02.326 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send_bytes": 11954, 2024-04-22T13:35:02.326 INFO:teuthology.orchestra.run.smithi111.stdout: "op_resend": 33, 2024-04-22T13:35:02.326 INFO:teuthology.orchestra.run.smithi111.stdout: "op_reply": 41, 2024-04-22T13:35:02.326 INFO:teuthology.orchestra.run.smithi111.stdout: "op_latency": { 2024-04-22T13:35:02.326 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 41, 2024-04-22T13:35:02.326 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.219995923, 2024-04-22T13:35:02.326 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.005365754 2024-04-22T13:35:02.326 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.326 INFO:teuthology.orchestra.run.smithi111.stdout: "op_inflight": 0, 2024-04-22T13:35:02.326 INFO:teuthology.orchestra.run.smithi111.stdout: "oplen_avg": { 2024-04-22T13:35:02.326 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 41, 2024-04-22T13:35:02.326 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 63 2024-04-22T13:35:02.327 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.327 INFO:teuthology.orchestra.run.smithi111.stdout: "op": 41, 2024-04-22T13:35:02.327 INFO:teuthology.orchestra.run.smithi111.stdout: "op_r": 0, 2024-04-22T13:35:02.327 INFO:teuthology.orchestra.run.smithi111.stdout: "op_w": 41, 2024-04-22T13:35:02.327 INFO:teuthology.orchestra.run.smithi111.stdout: "op_rmw": 0, 2024-04-22T13:35:02.327 INFO:teuthology.orchestra.run.smithi111.stdout: "op_pg": 0, 2024-04-22T13:35:02.327 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_stat": 0, 2024-04-22T13:35:02.327 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_create": 10, 2024-04-22T13:35:02.327 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_read": 0, 2024-04-22T13:35:02.327 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_write": 5, 2024-04-22T13:35:02.327 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writefull": 7, 2024-04-22T13:35:02.327 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writesame": 0, 2024-04-22T13:35:02.327 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_append": 0, 2024-04-22T13:35:02.327 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_zero": 0, 2024-04-22T13:35:02.328 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_truncate": 0, 2024-04-22T13:35:02.328 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_delete": 6, 2024-04-22T13:35:02.328 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_mapext": 0, 2024-04-22T13:35:02.328 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_sparse_read": 0, 2024-04-22T13:35:02.328 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_clonerange": 0, 2024-04-22T13:35:02.328 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_getxattr": 0, 2024-04-22T13:35:02.328 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_setxattr": 20, 2024-04-22T13:35:02.328 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_cmpxattr": 0, 2024-04-22T13:35:02.328 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_rmxattr": 0, 2024-04-22T13:35:02.328 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_resetxattrs": 0, 2024-04-22T13:35:02.328 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_call": 0, 2024-04-22T13:35:02.328 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_watch": 0, 2024-04-22T13:35:02.328 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_notify": 0, 2024-04-22T13:35:02.328 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_src_cmpxattr": 0, 2024-04-22T13:35:02.328 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls": 0, 2024-04-22T13:35:02.328 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls_filter": 0, 2024-04-22T13:35:02.329 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_other": 0, 2024-04-22T13:35:02.329 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_active": 0, 2024-04-22T13:35:02.329 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_send": 0, 2024-04-22T13:35:02.329 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_resend": 0, 2024-04-22T13:35:02.329 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_ping": 0, 2024-04-22T13:35:02.329 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_active": 0, 2024-04-22T13:35:02.329 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_send": 0, 2024-04-22T13:35:02.329 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_resend": 0, 2024-04-22T13:35:02.329 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_active": 0, 2024-04-22T13:35:02.329 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_send": 0, 2024-04-22T13:35:02.329 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_resend": 0, 2024-04-22T13:35:02.329 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_active": 0, 2024-04-22T13:35:02.329 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_send": 0, 2024-04-22T13:35:02.329 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_resend": 0, 2024-04-22T13:35:02.329 INFO:teuthology.orchestra.run.smithi111.stdout: "command_active": 0, 2024-04-22T13:35:02.330 INFO:teuthology.orchestra.run.smithi111.stdout: "command_send": 0, 2024-04-22T13:35:02.330 INFO:teuthology.orchestra.run.smithi111.stdout: "command_resend": 0, 2024-04-22T13:35:02.330 INFO:teuthology.orchestra.run.smithi111.stdout: "map_epoch": 98, 2024-04-22T13:35:02.330 INFO:teuthology.orchestra.run.smithi111.stdout: "map_full": 0, 2024-04-22T13:35:02.330 INFO:teuthology.orchestra.run.smithi111.stdout: "map_inc": 11, 2024-04-22T13:35:02.330 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_sessions": 9, 2024-04-22T13:35:02.330 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_open": 9, 2024-04-22T13:35:02.330 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_close": 0, 2024-04-22T13:35:02.330 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_laggy": 0, 2024-04-22T13:35:02.330 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_wr": 14, 2024-04-22T13:35:02.330 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_rd": 0, 2024-04-22T13:35:02.330 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_del": 1 2024-04-22T13:35:02.330 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.330 INFO:teuthology.orchestra.run.smithi111.stdout: "oft": { 2024-04-22T13:35:02.331 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_objs": 1, 2024-04-22T13:35:02.331 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_kv_pairs": 0, 2024-04-22T13:35:02.331 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_updates": 0, 2024-04-22T13:35:02.331 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_removes": 0 2024-04-22T13:35:02.331 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.331 INFO:teuthology.orchestra.run.smithi111.stdout: "osd-slow-ops": { 2024-04-22T13:35:02.331 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_ops_count": 4 2024-04-22T13:35:02.331 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.331 INFO:teuthology.orchestra.run.smithi111.stdout: "purge_queue": { 2024-04-22T13:35:02.331 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops": 0, 2024-04-22T13:35:02.331 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops_high_water": 0, 2024-04-22T13:35:02.331 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing": 0, 2024-04-22T13:35:02.331 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_high_water": 0, 2024-04-22T13:35:02.331 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed_ops": 0, 2024-04-22T13:35:02.331 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed": 0, 2024-04-22T13:35:02.332 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_item_in_journal": 0 2024-04-22T13:35:02.332 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.332 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T13:35:02.332 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-22T13:35:02.332 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-04-22T13:35:02.332 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-22T13:35:02.332 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 751, 2024-04-22T13:35:02.332 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 1678623, 2024-04-22T13:35:02.332 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-22T13:35:02.332 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 751, 2024-04-22T13:35:02.332 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-04-22T13:35:02.332 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-04-22T13:35:02.332 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 751, 2024-04-22T13:35:02.332 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 1678623, 2024-04-22T13:35:02.333 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-22T13:35:02.333 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.333 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.333 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.333 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:35:02.333 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.333 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_bytes": { 2024-04-22T13:35:02.333 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-22T13:35:02.333 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-04-22T13:35:02.333 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-22T13:35:02.333 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-04-22T13:35:02.333 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-04-22T13:35:02.333 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-22T13:35:02.333 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-04-22T13:35:02.333 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 41, 2024-04-22T13:35:02.333 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 11954, 2024-04-22T13:35:02.334 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 41, 2024-04-22T13:35:02.334 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 11954, 2024-04-22T13:35:02.334 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-22T13:35:02.334 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.334 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.334 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.334 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:35:02.334 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.334 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_ops": { 2024-04-22T13:35:02.334 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-22T13:35:02.334 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 1024, 2024-04-22T13:35:02.334 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-22T13:35:02.334 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-04-22T13:35:02.335 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-04-22T13:35:02.335 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-22T13:35:02.335 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-04-22T13:35:02.335 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 41, 2024-04-22T13:35:02.335 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 41, 2024-04-22T13:35:02.335 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 41, 2024-04-22T13:35:02.335 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 41, 2024-04-22T13:35:02.335 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-22T13:35:02.335 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.335 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.335 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.335 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:35:02.335 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.335 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle": { 2024-04-22T13:35:02.335 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-22T13:35:02.336 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-04-22T13:35:02.336 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-22T13:35:02.336 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2024-04-22T13:35:02.336 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2024-04-22T13:35:02.336 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-22T13:35:02.336 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2024-04-22T13:35:02.336 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-04-22T13:35:02.336 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-04-22T13:35:02.336 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 0, 2024-04-22T13:35:02.336 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 0, 2024-04-22T13:35:02.336 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-22T13:35:02.336 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.336 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.336 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.336 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:35:02.337 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-04-22T13:35:02.337 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle-0x561dafe36500": { 2024-04-22T13:35:02.337 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-04-22T13:35:02.337 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2024-04-22T13:35:02.337 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-04-22T13:35:02.337 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 6, 2024-04-22T13:35:02.337 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 1652, 2024-04-22T13:35:02.337 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-04-22T13:35:02.337 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 6, 2024-04-22T13:35:02.337 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-04-22T13:35:02.337 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-04-22T13:35:02.337 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 5, 2024-04-22T13:35:02.337 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 1652, 2024-04-22T13:35:02.337 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-04-22T13:35:02.337 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-04-22T13:35:02.337 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-04-22T13:35:02.338 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-04-22T13:35:02.338 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:35:02.338 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-04-22T13:35:02.338 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-04-22T13:35:03.123 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-04-22T13:35:03.123 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-04-22T13:35:03.123 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=6 2024-04-22T13:35:03.123 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-04-22T13:35:03.123 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-04-22T13:35:03.124 DEBUG:tasks.check_counter:Getting stats from c 2024-04-22T13:35:03.124 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-22T13:35:03.372 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:35:03.746 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:03 smithi134 ceph-mon[31338]: pgmap v339: 97 pgs: 97 active+clean; 2.3 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-22T13:35:03.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:03 smithi111 ceph-mon[31361]: pgmap v339: 97 pgs: 97 active+clean; 2.3 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-22T13:35:04.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:03 smithi012 ceph-mon[28265]: pgmap v339: 97 pgs: 97 active+clean; 2.3 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-22T13:35:04.956 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-04-22T13:35:04.956 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T13:35:04.956 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 1025, 2024-04-22T13:35:04.956 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 1006, 2024-04-22T13:35:04.956 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 1909968, 2024-04-22T13:35:04.956 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 381122, 2024-04-22T13:35:04.956 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 5, 2024-04-22T13:35:04.956 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 1, 2024-04-22T13:35:04.956 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.137605587, 2024-04-22T13:35:04.956 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.044714592, 2024-04-22T13:35:04.957 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.058963516, 2024-04-22T13:35:04.957 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.033597582, 2024-04-22T13:35:04.957 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:35:04.957 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1005, 2024-04-22T13:35:04.957 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.059293434, 2024-04-22T13:35:04.957 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000058998 2024-04-22T13:35:04.957 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.957 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:35:04.957 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.957 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.957 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.957 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.957 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1649568, 2024-04-22T13:35:04.957 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 134672 2024-04-22T13:35:04.957 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.957 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T13:35:04.958 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 8, 2024-04-22T13:35:04.958 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 168, 2024-04-22T13:35:04.958 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 3868, 2024-04-22T13:35:04.958 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 453546, 2024-04-22T13:35:04.958 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 6, 2024-04-22T13:35:04.958 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 4, 2024-04-22T13:35:04.958 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.010834801, 2024-04-22T13:35:04.958 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.006725105, 2024-04-22T13:35:04.958 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.002197539, 2024-04-22T13:35:04.958 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000094975, 2024-04-22T13:35:04.958 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:35:04.958 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 166, 2024-04-22T13:35:04.958 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.015106754, 2024-04-22T13:35:04.958 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000091004 2024-04-22T13:35:04.959 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.959 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:35:04.959 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.959 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.959 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.959 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.959 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-22T13:35:04.959 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 452848 2024-04-22T13:35:04.959 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.959 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T13:35:04.959 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 1441, 2024-04-22T13:35:04.959 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 1443, 2024-04-22T13:35:04.959 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 369574, 2024-04-22T13:35:04.959 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 458245, 2024-04-22T13:35:04.959 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 4, 2024-04-22T13:35:04.960 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 3, 2024-04-22T13:35:04.960 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.096228292, 2024-04-22T13:35:04.960 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.030066685, 2024-04-22T13:35:04.960 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.024312230, 2024-04-22T13:35:04.960 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.035188021, 2024-04-22T13:35:04.960 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:35:04.960 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1441, 2024-04-22T13:35:04.960 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.047330397, 2024-04-22T13:35:04.960 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000032845 2024-04-22T13:35:04.960 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.960 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:35:04.960 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.960 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.960 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.960 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.960 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-22T13:35:04.960 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 0 2024-04-22T13:35:04.961 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.961 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-04-22T13:35:04.961 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-04-22T13:35:04.961 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-04-22T13:35:04.961 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.961 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-04-22T13:35:04.961 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-04-22T13:35:04.961 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-04-22T13:35:04.961 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3771, 2024-04-22T13:35:04.961 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.097052189, 2024-04-22T13:35:04.961 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000025736 2024-04-22T13:35:04.961 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:04.961 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.961 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-04-22T13:35:04.961 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-04-22T13:35:04.961 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-04-22T13:35:04.962 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.962 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.962 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.962 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:04.962 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.962 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-04-22T13:35:04.962 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 0, 2024-04-22T13:35:04.962 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 0, 2024-04-22T13:35:04.962 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-04-22T13:35:04.962 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.962 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.962 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.962 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.962 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-04-22T13:35:04.962 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 0, 2024-04-22T13:35:04.963 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2024-04-22T13:35:04.963 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-04-22T13:35:04.963 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 0, 2024-04-22T13:35:04.963 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2024-04-22T13:35:04.963 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-04-22T13:35:04.963 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 0, 2024-04-22T13:35:04.963 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 0, 2024-04-22T13:35:04.963 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 0, 2024-04-22T13:35:04.963 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 0, 2024-04-22T13:35:04.963 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 0, 2024-04-22T13:35:04.963 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 0, 2024-04-22T13:35:04.963 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 0, 2024-04-22T13:35:04.963 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 0, 2024-04-22T13:35:04.963 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 1, 2024-04-22T13:35:04.963 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 0, 2024-04-22T13:35:04.963 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 0, 2024-04-22T13:35:04.963 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 0, 2024-04-22T13:35:04.964 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 0, 2024-04-22T13:35:04.964 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-04-22T13:35:04.964 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-04-22T13:35:04.964 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2024-04-22T13:35:04.964 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2024-04-22T13:35:04.964 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-04-22T13:35:04.964 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2024-04-22T13:35:04.964 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2024-04-22T13:35:04.964 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 0, 2024-04-22T13:35:04.964 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 0, 2024-04-22T13:35:04.964 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-04-22T13:35:04.964 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 0, 2024-04-22T13:35:04.964 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2024-04-22T13:35:04.964 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 0, 2024-04-22T13:35:04.964 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 0, 2024-04-22T13:35:04.964 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 0, 2024-04-22T13:35:04.965 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 0, 2024-04-22T13:35:04.965 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-04-22T13:35:04.965 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-04-22T13:35:04.965 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-04-22T13:35:04.965 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2024-04-22T13:35:04.965 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2024-04-22T13:35:04.965 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2024-04-22T13:35:04.965 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 0, 2024-04-22T13:35:04.965 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2024-04-22T13:35:04.965 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 0, 2024-04-22T13:35:04.965 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 0, 2024-04-22T13:35:04.965 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-04-22T13:35:04.965 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-22T13:35:04.965 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-22T13:35:04.965 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 0, 2024-04-22T13:35:04.965 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 0, 2024-04-22T13:35:04.966 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 0, 2024-04-22T13:35:04.966 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 0 2024-04-22T13:35:04.966 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.966 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-04-22T13:35:04.966 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2024-04-22T13:35:04.966 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2024-04-22T13:35:04.966 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2024-04-22T13:35:04.966 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 0, 2024-04-22T13:35:04.966 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 0, 2024-04-22T13:35:04.966 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2024-04-22T13:35:04.966 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-04-22T13:35:04.966 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-04-22T13:35:04.966 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 0, 2024-04-22T13:35:04.966 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 0, 2024-04-22T13:35:04.966 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-04-22T13:35:04.967 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-04-22T13:35:04.967 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-04-22T13:35:04.967 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-04-22T13:35:04.967 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-04-22T13:35:04.967 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-04-22T13:35:04.967 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-04-22T13:35:04.967 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_path": 0, 2024-04-22T13:35:04.967 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_inode": 0, 2024-04-22T13:35:04.967 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2024-04-22T13:35:04.967 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2024-04-22T13:35:04.967 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-04-22T13:35:04.967 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2024-04-22T13:35:04.967 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-04-22T13:35:04.967 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2024-04-22T13:35:04.967 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.968 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-04-22T13:35:04.968 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 0, 2024-04-22T13:35:04.968 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 0, 2024-04-22T13:35:04.968 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-04-22T13:35:04.968 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-04-22T13:35:04.968 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 7, 2024-04-22T13:35:04.968 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-04-22T13:35:04.968 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-04-22T13:35:04.968 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 0, 2024-04-22T13:35:04.968 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-04-22T13:35:04.968 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-04-22T13:35:04.968 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 2, 2024-04-22T13:35:04.968 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 2, 2024-04-22T13:35:04.968 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-04-22T13:35:04.968 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-04-22T13:35:04.969 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 4194304, 2024-04-22T13:35:04.969 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 4196347, 2024-04-22T13:35:04.969 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 4196206, 2024-04-22T13:35:04.969 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-04-22T13:35:04.969 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.969 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.969 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.969 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.969 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 7 2024-04-22T13:35:04.969 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.969 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-04-22T13:35:04.969 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 3, 2024-04-22T13:35:04.969 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 3, 2024-04-22T13:35:04.969 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 0, 2024-04-22T13:35:04.969 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 1, 2024-04-22T13:35:04.970 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 1, 2024-04-22T13:35:04.970 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 0, 2024-04-22T13:35:04.970 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 0, 2024-04-22T13:35:04.970 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 0, 2024-04-22T13:35:04.970 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 0, 2024-04-22T13:35:04.970 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 0, 2024-04-22T13:35:04.970 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 0, 2024-04-22T13:35:04.970 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 0, 2024-04-22T13:35:04.970 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 50320, 2024-04-22T13:35:04.970 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 215416 2024-04-22T13:35:04.970 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.970 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-04-22T13:35:04.970 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 0, 2024-04-22T13:35:04.970 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 0, 2024-04-22T13:35:04.971 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 0, 2024-04-22T13:35:04.971 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 0, 2024-04-22T13:35:04.971 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 0, 2024-04-22T13:35:04.971 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-04-22T13:35:04.971 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.971 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.971 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.971 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.971 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-04-22T13:35:04.971 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.971 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.971 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.971 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.971 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-04-22T13:35:04.971 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.971 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.972 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.972 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.972 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-04-22T13:35:04.972 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.972 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.972 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.972 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.972 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-04-22T13:35:04.972 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.972 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.972 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.972 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.972 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-04-22T13:35:04.972 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.972 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.973 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.973 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.973 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-04-22T13:35:04.973 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.973 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.973 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.973 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.973 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-04-22T13:35:04.973 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.973 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.973 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.973 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.973 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-04-22T13:35:04.973 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.973 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.973 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.974 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.974 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-04-22T13:35:04.974 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.974 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.974 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.974 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.974 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-04-22T13:35:04.974 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.974 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.974 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.974 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.974 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-04-22T13:35:04.974 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.975 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.975 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.975 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.975 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-04-22T13:35:04.975 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.975 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.975 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.975 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.975 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-04-22T13:35:04.975 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.975 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.976 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.976 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.976 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-04-22T13:35:04.976 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.976 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.976 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.976 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.976 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-04-22T13:35:04.976 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.976 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.976 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.976 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.976 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-04-22T13:35:04.976 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.976 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.976 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.977 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.977 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-04-22T13:35:04.977 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.977 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.977 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.977 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.977 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-04-22T13:35:04.977 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.977 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.977 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.977 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.977 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-04-22T13:35:04.977 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.977 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.977 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.977 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.978 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-04-22T13:35:04.978 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.978 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.978 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.978 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.978 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-04-22T13:35:04.978 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.978 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.978 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.978 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.978 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-04-22T13:35:04.978 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.978 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.978 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.978 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.978 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-04-22T13:35:04.979 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.979 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.979 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.979 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.979 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-04-22T13:35:04.979 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.979 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.979 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.979 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.979 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-04-22T13:35:04.979 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.979 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.979 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.979 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.979 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-04-22T13:35:04.980 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.980 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.980 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.980 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.980 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-04-22T13:35:04.980 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.980 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.980 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.980 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.980 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-04-22T13:35:04.980 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.980 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.980 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.980 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.981 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-04-22T13:35:04.981 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-04-22T13:35:04.981 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-04-22T13:35:04.981 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.981 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.981 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.981 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:04.981 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.981 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-04-22T13:35:04.981 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 1, 2024-04-22T13:35:04.981 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 3, 2024-04-22T13:35:04.981 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 2, 2024-04-22T13:35:04.981 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 3, 2024-04-22T13:35:04.981 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-04-22T13:35:04.981 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 0, 2024-04-22T13:35:04.981 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 0, 2024-04-22T13:35:04.982 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 4182, 2024-04-22T13:35:04.982 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-04-22T13:35:04.982 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.982 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-04-22T13:35:04.982 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-04-22T13:35:04.982 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-04-22T13:35:04.982 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T13:35:04.982 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-04-22T13:35:04.982 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T13:35:04.982 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-04-22T13:35:04.982 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T13:35:04.982 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T13:35:04.982 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T13:35:04.982 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T13:35:04.982 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T13:35:04.983 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-04-22T13:35:04.983 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T13:35:04.983 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T13:35:04.983 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-04-22T13:35:04.983 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-04-22T13:35:04.983 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-04-22T13:35:04.983 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-04-22T13:35:04.983 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T13:35:04.983 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T13:35:04.983 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T13:35:04.983 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T13:35:04.983 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T13:35:04.983 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-04-22T13:35:04.983 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-04-22T13:35:04.983 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-04-22T13:35:04.983 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T13:35:04.984 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T13:35:04.984 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-04-22T13:35:04.984 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-04-22T13:35:04.984 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-04-22T13:35:04.984 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-04-22T13:35:04.984 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T13:35:04.984 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-04-22T13:35:04.984 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T13:35:04.984 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-04-22T13:35:04.984 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 181393, 2024-04-22T13:35:04.984 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 49, 2024-04-22T13:35:04.984 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-04-22T13:35:04.984 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-04-22T13:35:04.984 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-04-22T13:35:04.984 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-04-22T13:35:04.984 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-04-22T13:35:04.984 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-04-22T13:35:04.985 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-04-22T13:35:04.985 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-04-22T13:35:04.985 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 7936, 2024-04-22T13:35:04.985 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 165, 2024-04-22T13:35:04.985 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T13:35:04.985 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-04-22T13:35:04.985 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-04-22T13:35:04.985 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-04-22T13:35:04.985 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 8512, 2024-04-22T13:35:04.985 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 13, 2024-04-22T13:35:04.985 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-04-22T13:35:04.985 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-04-22T13:35:04.985 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-04-22T13:35:04.985 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-04-22T13:35:04.985 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.986 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-04-22T13:35:04.986 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-04-22T13:35:04.986 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-04-22T13:35:04.986 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 2218, 2024-04-22T13:35:04.986 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 21, 2024-04-22T13:35:04.986 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 3, 2024-04-22T13:35:04.986 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 2218, 2024-04-22T13:35:04.986 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-04-22T13:35:04.986 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2218, 2024-04-22T13:35:04.986 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1.030973905, 2024-04-22T13:35:04.986 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000464821 2024-04-22T13:35:04.986 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.986 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-04-22T13:35:04.986 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-04-22T13:35:04.986 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2218, 2024-04-22T13:35:04.986 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 2219 2024-04-22T13:35:04.987 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.987 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 2218, 2024-04-22T13:35:04.987 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 2218, 2024-04-22T13:35:04.987 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 0, 2024-04-22T13:35:04.987 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-04-22T13:35:04.987 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-04-22T13:35:04.987 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 1433, 2024-04-22T13:35:04.987 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 0, 2024-04-22T13:35:04.987 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 783, 2024-04-22T13:35:04.987 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 0, 2024-04-22T13:35:04.987 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 0, 2024-04-22T13:35:04.987 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-04-22T13:35:04.987 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-04-22T13:35:04.987 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-04-22T13:35:04.987 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2024-04-22T13:35:04.987 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 0, 2024-04-22T13:35:04.988 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-04-22T13:35:04.988 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-04-22T13:35:04.988 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-04-22T13:35:04.988 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 1, 2024-04-22T13:35:04.988 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 0, 2024-04-22T13:35:04.988 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-04-22T13:35:04.988 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-04-22T13:35:04.988 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-04-22T13:35:04.988 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-04-22T13:35:04.988 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-04-22T13:35:04.988 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-04-22T13:35:04.988 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-04-22T13:35:04.988 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-04-22T13:35:04.988 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-04-22T13:35:04.988 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-04-22T13:35:04.988 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-04-22T13:35:04.989 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-04-22T13:35:04.989 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-04-22T13:35:04.989 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-04-22T13:35:04.989 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-04-22T13:35:04.989 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-04-22T13:35:04.989 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-04-22T13:35:04.989 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-04-22T13:35:04.989 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-04-22T13:35:04.989 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-04-22T13:35:04.989 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-04-22T13:35:04.989 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-04-22T13:35:04.989 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-04-22T13:35:04.989 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-04-22T13:35:04.989 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-04-22T13:35:04.989 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-04-22T13:35:04.989 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 98, 2024-04-22T13:35:04.989 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-04-22T13:35:04.990 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 11, 2024-04-22T13:35:04.990 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 4, 2024-04-22T13:35:04.990 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 4, 2024-04-22T13:35:04.990 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2024-04-22T13:35:04.990 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-04-22T13:35:04.990 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 0, 2024-04-22T13:35:04.990 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 2, 2024-04-22T13:35:04.990 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 0 2024-04-22T13:35:04.990 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.990 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-04-22T13:35:04.990 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 0, 2024-04-22T13:35:04.990 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-04-22T13:35:04.990 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-04-22T13:35:04.990 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-04-22T13:35:04.990 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.991 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2024-04-22T13:35:04.991 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 0 2024-04-22T13:35:04.991 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.991 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-04-22T13:35:04.991 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-04-22T13:35:04.991 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 0, 2024-04-22T13:35:04.991 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-04-22T13:35:04.991 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 0, 2024-04-22T13:35:04.991 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 0, 2024-04-22T13:35:04.991 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 0, 2024-04-22T13:35:04.991 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-04-22T13:35:04.991 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.991 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T13:35:04.991 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-22T13:35:04.991 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-04-22T13:35:04.991 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-22T13:35:04.991 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 2470, 2024-04-22T13:35:04.992 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 2045321, 2024-04-22T13:35:04.992 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-22T13:35:04.992 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 2470, 2024-04-22T13:35:04.992 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-22T13:35:04.992 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-22T13:35:04.992 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2470, 2024-04-22T13:35:04.992 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 2045321, 2024-04-22T13:35:04.992 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-22T13:35:04.992 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.992 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.992 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.992 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:04.992 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.992 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-04-22T13:35:04.992 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-22T13:35:04.993 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-04-22T13:35:04.993 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-22T13:35:04.993 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-22T13:35:04.993 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-22T13:35:04.993 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-22T13:35:04.993 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-22T13:35:04.993 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 2218, 2024-04-22T13:35:04.993 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 2048, 2024-04-22T13:35:04.993 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2218, 2024-04-22T13:35:04.993 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 2048, 2024-04-22T13:35:04.993 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-22T13:35:04.993 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.993 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.993 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.993 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:04.993 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.994 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-04-22T13:35:04.994 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-22T13:35:04.994 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-04-22T13:35:04.994 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-22T13:35:04.994 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-22T13:35:04.994 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-22T13:35:04.994 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-22T13:35:04.994 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-22T13:35:04.994 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 2218, 2024-04-22T13:35:04.994 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 2218, 2024-04-22T13:35:04.994 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2218, 2024-04-22T13:35:04.994 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 2218, 2024-04-22T13:35:04.994 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-22T13:35:04.994 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.994 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.994 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.995 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:04.995 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.995 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-04-22T13:35:04.995 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-22T13:35:04.995 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-04-22T13:35:04.995 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-22T13:35:04.995 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-22T13:35:04.995 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-22T13:35:04.995 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-22T13:35:04.995 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-22T13:35:04.995 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-22T13:35:04.995 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-22T13:35:04.995 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-04-22T13:35:04.995 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-04-22T13:35:04.995 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-22T13:35:04.996 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.996 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.996 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.996 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:04.996 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:04.996 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x563decc986e0": { 2024-04-22T13:35:04.996 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-22T13:35:04.996 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-04-22T13:35:04.996 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-22T13:35:04.996 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-22T13:35:04.996 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-22T13:35:04.996 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-22T13:35:04.996 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-22T13:35:04.996 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-22T13:35:04.996 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-22T13:35:04.996 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-04-22T13:35:04.996 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-04-22T13:35:04.997 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-22T13:35:04.997 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:04.997 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:04.997 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:04.997 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:04.997 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:04.997 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-04-22T13:35:05.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:05 smithi111 ceph-mon[31361]: pgmap v340: 97 pgs: 97 active+clean; 2.3 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:35:05.756 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-22T13:35:05.756 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-22T13:35:05.756 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-04-22T13:35:05.756 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-22T13:35:05.757 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-22T13:35:05.757 DEBUG:tasks.check_counter:Getting stats from f 2024-04-22T13:35:05.757 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-22T13:35:05.780 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:05 smithi134 ceph-mon[31338]: pgmap v340: 97 pgs: 97 active+clean; 2.3 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:35:05.994 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:35:06.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:05 smithi012 ceph-mon[28265]: pgmap v340: 97 pgs: 97 active+clean; 2.3 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:35:07.654 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:07 smithi134 ceph-mon[31338]: pgmap v341: 97 pgs: 97 active+clean; 2.3 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:35:07.655 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-04-22T13:35:07.655 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T13:35:07.655 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 714, 2024-04-22T13:35:07.655 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 874, 2024-04-22T13:35:07.655 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 1773018, 2024-04-22T13:35:07.655 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 554974, 2024-04-22T13:35:07.655 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 12, 2024-04-22T13:35:07.655 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 3, 2024-04-22T13:35:07.655 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.113050074, 2024-04-22T13:35:07.655 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.039009543, 2024-04-22T13:35:07.656 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.066459126, 2024-04-22T13:35:07.656 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.011105863, 2024-04-22T13:35:07.656 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:35:07.656 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 877, 2024-04-22T13:35:07.656 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.051779184, 2024-04-22T13:35:07.656 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000059041 2024-04-22T13:35:07.656 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.656 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:35:07.656 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 986, 2024-04-22T13:35:07.656 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000265438, 2024-04-22T13:35:07.656 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000269 2024-04-22T13:35:07.656 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.656 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1655456, 2024-04-22T13:35:07.656 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 153344 2024-04-22T13:35:07.657 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.657 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T13:35:07.657 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 69, 2024-04-22T13:35:07.657 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 100, 2024-04-22T13:35:07.657 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 14864, 2024-04-22T13:35:07.657 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 635428, 2024-04-22T13:35:07.657 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 51, 2024-04-22T13:35:07.657 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 3, 2024-04-22T13:35:07.657 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.021753667, 2024-04-22T13:35:07.657 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.002838513, 2024-04-22T13:35:07.657 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.016825259, 2024-04-22T13:35:07.657 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000396174, 2024-04-22T13:35:07.657 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:35:07.657 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 98, 2024-04-22T13:35:07.657 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.012270413, 2024-04-22T13:35:07.658 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000125208 2024-04-22T13:35:07.658 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.658 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:35:07.658 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 58, 2024-04-22T13:35:07.658 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000006825, 2024-04-22T13:35:07.658 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000117 2024-04-22T13:35:07.658 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.658 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 5008, 2024-04-22T13:35:07.658 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 130656 2024-04-22T13:35:07.658 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.658 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T13:35:07.658 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 31, 2024-04-22T13:35:07.658 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 186, 2024-04-22T13:35:07.658 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 6635, 2024-04-22T13:35:07.659 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 361859, 2024-04-22T13:35:07.659 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 11, 2024-04-22T13:35:07.659 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 6, 2024-04-22T13:35:07.659 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.018766972, 2024-04-22T13:35:07.659 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.008465734, 2024-04-22T13:35:07.659 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.008030487, 2024-04-22T13:35:07.659 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000331980, 2024-04-22T13:35:07.659 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:35:07.659 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 180, 2024-04-22T13:35:07.659 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.022148826, 2024-04-22T13:35:07.659 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000123049 2024-04-22T13:35:07.659 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.659 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:35:07.660 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 44, 2024-04-22T13:35:07.660 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000009449, 2024-04-22T13:35:07.660 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000214 2024-04-22T13:35:07.660 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.660 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-04-22T13:35:07.660 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 326848 2024-04-22T13:35:07.660 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.660 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-04-22T13:35:07.660 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-04-22T13:35:07.660 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-04-22T13:35:07.660 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.660 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-04-22T13:35:07.660 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-04-22T13:35:07.660 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-04-22T13:35:07.660 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 62, 2024-04-22T13:35:07.660 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.002017574, 2024-04-22T13:35:07.661 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000032541 2024-04-22T13:35:07.661 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:07.661 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.661 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-04-22T13:35:07.661 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-04-22T13:35:07.661 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-04-22T13:35:07.661 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2, 2024-04-22T13:35:07.661 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000008537, 2024-04-22T13:35:07.661 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000004268 2024-04-22T13:35:07.661 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:07.661 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.661 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-04-22T13:35:07.661 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 6, 2024-04-22T13:35:07.661 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 3, 2024-04-22T13:35:07.661 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-04-22T13:35:07.662 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3, 2024-04-22T13:35:07.662 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000798030, 2024-04-22T13:35:07.662 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000266010 2024-04-22T13:35:07.662 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.662 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-04-22T13:35:07.662 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 3, 2024-04-22T13:35:07.662 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2024-04-22T13:35:07.662 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-04-22T13:35:07.662 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 11, 2024-04-22T13:35:07.662 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2024-04-22T13:35:07.662 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-04-22T13:35:07.662 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 10, 2024-04-22T13:35:07.662 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 0, 2024-04-22T13:35:07.662 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 10, 2024-04-22T13:35:07.663 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 0, 2024-04-22T13:35:07.663 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 10, 2024-04-22T13:35:07.663 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 0, 2024-04-22T13:35:07.663 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 1, 2024-04-22T13:35:07.663 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 1, 2024-04-22T13:35:07.663 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 1, 2024-04-22T13:35:07.663 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 11, 2024-04-22T13:35:07.663 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 10, 2024-04-22T13:35:07.663 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 0, 2024-04-22T13:35:07.663 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 0, 2024-04-22T13:35:07.663 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-04-22T13:35:07.663 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-04-22T13:35:07.663 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2024-04-22T13:35:07.663 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2024-04-22T13:35:07.663 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-04-22T13:35:07.664 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2024-04-22T13:35:07.664 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2024-04-22T13:35:07.664 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 0, 2024-04-22T13:35:07.664 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 0, 2024-04-22T13:35:07.664 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-04-22T13:35:07.664 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 0, 2024-04-22T13:35:07.664 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 1, 2024-04-22T13:35:07.664 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 4, 2024-04-22T13:35:07.664 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 234, 2024-04-22T13:35:07.664 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 6, 2024-04-22T13:35:07.664 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 0, 2024-04-22T13:35:07.664 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-04-22T13:35:07.664 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-04-22T13:35:07.664 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-04-22T13:35:07.664 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2024-04-22T13:35:07.665 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2024-04-22T13:35:07.665 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2024-04-22T13:35:07.665 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 0, 2024-04-22T13:35:07.665 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2024-04-22T13:35:07.665 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 1, 2024-04-22T13:35:07.665 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 0, 2024-04-22T13:35:07.665 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-04-22T13:35:07.665 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-22T13:35:07.665 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-22T13:35:07.665 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 1, 2024-04-22T13:35:07.665 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 0, 2024-04-22T13:35:07.665 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 0, 2024-04-22T13:35:07.665 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 0 2024-04-22T13:35:07.665 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.665 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-04-22T13:35:07.666 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2024-04-22T13:35:07.666 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2024-04-22T13:35:07.666 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2024-04-22T13:35:07.666 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 1, 2024-04-22T13:35:07.666 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 0, 2024-04-22T13:35:07.666 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2024-04-22T13:35:07.666 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-04-22T13:35:07.666 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-04-22T13:35:07.666 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 0, 2024-04-22T13:35:07.666 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 0, 2024-04-22T13:35:07.666 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-04-22T13:35:07.666 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-04-22T13:35:07.666 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-04-22T13:35:07.666 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-04-22T13:35:07.667 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-04-22T13:35:07.667 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-04-22T13:35:07.667 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-04-22T13:35:07.667 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_path": 2, 2024-04-22T13:35:07.667 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_inode": 2, 2024-04-22T13:35:07.667 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2024-04-22T13:35:07.667 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2024-04-22T13:35:07.667 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-04-22T13:35:07.667 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2024-04-22T13:35:07.667 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-04-22T13:35:07.667 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2024-04-22T13:35:07.667 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.667 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-04-22T13:35:07.667 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 0, 2024-04-22T13:35:07.667 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 10, 2024-04-22T13:35:07.668 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-04-22T13:35:07.668 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-04-22T13:35:07.668 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 10, 2024-04-22T13:35:07.668 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-04-22T13:35:07.668 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-04-22T13:35:07.668 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 2, 2024-04-22T13:35:07.668 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-04-22T13:35:07.668 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-04-22T13:35:07.668 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 2, 2024-04-22T13:35:07.668 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 2, 2024-04-22T13:35:07.668 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-04-22T13:35:07.668 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-04-22T13:35:07.668 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 4194304, 2024-04-22T13:35:07.668 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 4197187, 2024-04-22T13:35:07.668 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 0, 2024-04-22T13:35:07.668 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-04-22T13:35:07.669 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 9, 2024-04-22T13:35:07.669 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.017681965, 2024-04-22T13:35:07.669 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001964662 2024-04-22T13:35:07.669 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.669 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 0 2024-04-22T13:35:07.669 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.669 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-04-22T13:35:07.669 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 13, 2024-04-22T13:35:07.669 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 13, 2024-04-22T13:35:07.669 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 0, 2024-04-22T13:35:07.669 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 11, 2024-04-22T13:35:07.669 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 12, 2024-04-22T13:35:07.669 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 1, 2024-04-22T13:35:07.669 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 10, 2024-04-22T13:35:07.669 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 10, 2024-04-22T13:35:07.670 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 0, 2024-04-22T13:35:07.670 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 1, 2024-04-22T13:35:07.670 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 1, 2024-04-22T13:35:07.670 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 0, 2024-04-22T13:35:07.670 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 52184, 2024-04-22T13:35:07.670 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 223608 2024-04-22T13:35:07.670 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.670 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-04-22T13:35:07.670 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 6, 2024-04-22T13:35:07.670 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 0, 2024-04-22T13:35:07.670 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 6, 2024-04-22T13:35:07.670 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 30, 2024-04-22T13:35:07.670 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 2, 2024-04-22T13:35:07.670 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-04-22T13:35:07.671 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.671 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.671 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.671 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.671 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-04-22T13:35:07.671 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3, 2024-04-22T13:35:07.671 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000798030, 2024-04-22T13:35:07.671 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000266010 2024-04-22T13:35:07.671 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.671 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-04-22T13:35:07.671 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.671 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.671 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.671 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.671 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-04-22T13:35:07.671 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.672 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.672 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.672 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.672 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-04-22T13:35:07.672 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.672 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.672 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.672 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.672 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-04-22T13:35:07.672 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.672 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.672 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.672 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.672 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-04-22T13:35:07.672 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.673 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.673 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.673 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.673 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-04-22T13:35:07.673 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.673 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.673 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.673 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.673 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-04-22T13:35:07.673 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.673 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.673 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.673 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.673 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-04-22T13:35:07.673 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.674 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.674 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.674 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.674 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-04-22T13:35:07.674 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.674 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.674 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.674 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.674 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-04-22T13:35:07.674 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.675 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.675 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.675 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.675 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-04-22T13:35:07.675 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.675 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.675 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.675 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.676 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-04-22T13:35:07.676 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.676 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.676 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.676 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.676 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-04-22T13:35:07.676 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.676 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.676 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.676 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.676 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-04-22T13:35:07.676 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.676 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.676 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.676 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.677 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-04-22T13:35:07.677 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.677 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.677 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.677 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.677 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-04-22T13:35:07.677 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.677 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.677 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.677 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.677 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-04-22T13:35:07.677 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.677 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.677 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.677 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.678 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-04-22T13:35:07.678 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.678 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.678 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.678 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.678 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-04-22T13:35:07.678 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.678 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.678 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.678 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.678 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-04-22T13:35:07.678 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.678 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.679 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.679 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.679 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-04-22T13:35:07.679 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.679 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.679 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.679 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.679 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-04-22T13:35:07.679 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.679 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.679 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.679 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.679 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-04-22T13:35:07.679 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.679 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.680 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.680 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.680 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-04-22T13:35:07.680 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.680 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.680 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.680 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.680 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-04-22T13:35:07.680 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.680 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.680 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.680 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.680 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-04-22T13:35:07.680 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.681 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.681 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.681 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.681 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-04-22T13:35:07.681 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.681 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.681 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.681 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.681 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-04-22T13:35:07.681 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-04-22T13:35:07.681 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-04-22T13:35:07.681 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.681 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.681 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.682 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:07.682 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.682 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-04-22T13:35:07.682 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 2, 2024-04-22T13:35:07.682 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 5, 2024-04-22T13:35:07.682 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 3, 2024-04-22T13:35:07.682 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 2, 2024-04-22T13:35:07.682 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-04-22T13:35:07.682 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 2, 2024-04-22T13:35:07.682 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 0, 2024-04-22T13:35:07.682 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 622, 2024-04-22T13:35:07.682 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-04-22T13:35:07.682 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.682 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-04-22T13:35:07.683 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-04-22T13:35:07.683 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-04-22T13:35:07.683 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T13:35:07.683 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-04-22T13:35:07.683 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T13:35:07.683 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-04-22T13:35:07.683 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T13:35:07.683 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T13:35:07.683 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T13:35:07.683 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T13:35:07.683 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T13:35:07.683 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-04-22T13:35:07.683 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T13:35:07.683 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T13:35:07.683 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-04-22T13:35:07.683 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-04-22T13:35:07.684 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-04-22T13:35:07.684 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-04-22T13:35:07.684 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T13:35:07.684 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T13:35:07.684 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T13:35:07.684 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T13:35:07.684 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T13:35:07.684 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-04-22T13:35:07.684 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-04-22T13:35:07.684 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-04-22T13:35:07.684 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T13:35:07.684 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T13:35:07.684 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-04-22T13:35:07.684 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-04-22T13:35:07.684 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-04-22T13:35:07.684 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-04-22T13:35:07.685 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T13:35:07.685 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-04-22T13:35:07.685 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T13:35:07.685 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-04-22T13:35:07.685 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 239095, 2024-04-22T13:35:07.685 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 80, 2024-04-22T13:35:07.685 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 88, 2024-04-22T13:35:07.685 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 1, 2024-04-22T13:35:07.685 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-04-22T13:35:07.685 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-04-22T13:35:07.685 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-04-22T13:35:07.685 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-04-22T13:35:07.685 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-04-22T13:35:07.685 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-04-22T13:35:07.685 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 7936, 2024-04-22T13:35:07.686 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 165, 2024-04-22T13:35:07.686 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T13:35:07.686 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-04-22T13:35:07.686 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-04-22T13:35:07.686 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-04-22T13:35:07.686 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 51248, 2024-04-22T13:35:07.686 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 201, 2024-04-22T13:35:07.686 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-04-22T13:35:07.686 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-04-22T13:35:07.686 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-04-22T13:35:07.686 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-04-22T13:35:07.686 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.686 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-04-22T13:35:07.687 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-04-22T13:35:07.687 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-04-22T13:35:07.687 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 47, 2024-04-22T13:35:07.687 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 13365, 2024-04-22T13:35:07.687 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 33, 2024-04-22T13:35:07.687 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 47, 2024-04-22T13:35:07.687 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-04-22T13:35:07.687 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 47, 2024-04-22T13:35:07.687 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.238994008, 2024-04-22T13:35:07.687 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.005084978 2024-04-22T13:35:07.687 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.687 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-04-22T13:35:07.687 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-04-22T13:35:07.687 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 47, 2024-04-22T13:35:07.687 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 69 2024-04-22T13:35:07.687 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.688 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 47, 2024-04-22T13:35:07.688 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 0, 2024-04-22T13:35:07.688 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 47, 2024-04-22T13:35:07.688 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-04-22T13:35:07.688 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-04-22T13:35:07.688 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 0, 2024-04-22T13:35:07.688 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 10, 2024-04-22T13:35:07.688 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 0, 2024-04-22T13:35:07.688 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 9, 2024-04-22T13:35:07.688 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 9, 2024-04-22T13:35:07.688 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-04-22T13:35:07.688 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-04-22T13:35:07.688 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-04-22T13:35:07.688 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2024-04-22T13:35:07.688 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 6, 2024-04-22T13:35:07.689 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-04-22T13:35:07.689 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-04-22T13:35:07.689 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-04-22T13:35:07.689 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 0, 2024-04-22T13:35:07.689 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 20, 2024-04-22T13:35:07.689 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-04-22T13:35:07.689 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-04-22T13:35:07.689 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-04-22T13:35:07.689 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-04-22T13:35:07.689 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-04-22T13:35:07.689 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-04-22T13:35:07.689 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-04-22T13:35:07.689 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-04-22T13:35:07.689 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-04-22T13:35:07.689 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-04-22T13:35:07.690 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-04-22T13:35:07.690 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-04-22T13:35:07.690 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-04-22T13:35:07.690 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-04-22T13:35:07.690 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-04-22T13:35:07.690 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-04-22T13:35:07.690 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-04-22T13:35:07.690 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-04-22T13:35:07.690 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-04-22T13:35:07.690 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-04-22T13:35:07.690 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-04-22T13:35:07.690 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-04-22T13:35:07.690 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-04-22T13:35:07.690 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-04-22T13:35:07.690 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-04-22T13:35:07.690 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-04-22T13:35:07.691 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 98, 2024-04-22T13:35:07.691 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-04-22T13:35:07.691 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 11, 2024-04-22T13:35:07.691 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 8, 2024-04-22T13:35:07.691 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 8, 2024-04-22T13:35:07.691 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2024-04-22T13:35:07.691 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-04-22T13:35:07.691 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 14, 2024-04-22T13:35:07.691 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 0, 2024-04-22T13:35:07.691 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 1 2024-04-22T13:35:07.691 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.691 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-04-22T13:35:07.691 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 1, 2024-04-22T13:35:07.691 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-04-22T13:35:07.691 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-04-22T13:35:07.691 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-04-22T13:35:07.691 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.692 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2024-04-22T13:35:07.692 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 4 2024-04-22T13:35:07.692 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.692 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-04-22T13:35:07.692 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-04-22T13:35:07.692 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 0, 2024-04-22T13:35:07.692 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-04-22T13:35:07.692 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 0, 2024-04-22T13:35:07.692 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 0, 2024-04-22T13:35:07.692 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 0, 2024-04-22T13:35:07.692 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-04-22T13:35:07.692 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.692 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T13:35:07.692 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-22T13:35:07.692 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-04-22T13:35:07.692 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-22T13:35:07.692 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 810, 2024-04-22T13:35:07.693 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 1703209, 2024-04-22T13:35:07.693 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-22T13:35:07.693 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 810, 2024-04-22T13:35:07.693 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-22T13:35:07.693 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-22T13:35:07.693 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 810, 2024-04-22T13:35:07.693 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 1703209, 2024-04-22T13:35:07.693 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-22T13:35:07.693 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.693 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.693 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.693 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:07.693 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.693 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-04-22T13:35:07.693 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-22T13:35:07.694 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-04-22T13:35:07.694 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-22T13:35:07.694 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-22T13:35:07.694 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-22T13:35:07.694 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-22T13:35:07.694 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-22T13:35:07.694 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 47, 2024-04-22T13:35:07.694 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 13365, 2024-04-22T13:35:07.694 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 47, 2024-04-22T13:35:07.694 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 13365, 2024-04-22T13:35:07.694 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-22T13:35:07.694 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.694 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.694 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.695 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:07.695 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.695 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-04-22T13:35:07.695 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-22T13:35:07.695 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-04-22T13:35:07.695 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-22T13:35:07.695 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-22T13:35:07.695 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-22T13:35:07.695 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-22T13:35:07.695 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-22T13:35:07.695 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 47, 2024-04-22T13:35:07.695 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 47, 2024-04-22T13:35:07.695 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 47, 2024-04-22T13:35:07.695 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 47, 2024-04-22T13:35:07.695 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-22T13:35:07.696 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.696 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.696 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.696 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:07.696 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.696 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-04-22T13:35:07.696 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-22T13:35:07.696 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-04-22T13:35:07.696 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-22T13:35:07.696 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-22T13:35:07.696 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-22T13:35:07.696 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-22T13:35:07.696 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-22T13:35:07.696 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-22T13:35:07.696 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-22T13:35:07.696 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-04-22T13:35:07.697 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-04-22T13:35:07.697 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-22T13:35:07.697 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.697 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.697 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.697 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:07.697 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:07.697 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x561e17666500": { 2024-04-22T13:35:07.697 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-22T13:35:07.697 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-04-22T13:35:07.697 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-22T13:35:07.697 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 10, 2024-04-22T13:35:07.697 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 2883, 2024-04-22T13:35:07.697 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-22T13:35:07.697 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 10, 2024-04-22T13:35:07.698 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-22T13:35:07.698 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-22T13:35:07.698 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 9, 2024-04-22T13:35:07.698 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 2883, 2024-04-22T13:35:07.698 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-22T13:35:07.698 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:07.698 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:07.698 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:07.698 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:07.698 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:07.698 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-04-22T13:35:07.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:07 smithi111 ceph-mon[31361]: pgmap v341: 97 pgs: 97 active+clean; 2.3 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:35:08.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:07 smithi012 ceph-mon[28265]: pgmap v341: 97 pgs: 97 active+clean; 2.3 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:35:08.465 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-22T13:35:08.465 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-22T13:35:08.465 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-04-22T13:35:08.465 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-22T13:35:08.465 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-22T13:35:08.465 DEBUG:tasks.check_counter:Getting stats from i 2024-04-22T13:35:08.465 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-22T13:35:08.755 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:35:09.919 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:09 smithi134 ceph-mon[31338]: pgmap v342: 97 pgs: 97 active+clean; 2.3 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-22T13:35:09.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:09 smithi111 ceph-mon[31361]: pgmap v342: 97 pgs: 97 active+clean; 2.3 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-22T13:35:10.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:09 smithi012 ceph-mon[28265]: pgmap v342: 97 pgs: 97 active+clean; 2.3 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-22T13:35:10.410 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-04-22T13:35:10.410 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T13:35:10.410 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 733, 2024-04-22T13:35:10.410 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 869, 2024-04-22T13:35:10.410 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 1785794, 2024-04-22T13:35:10.410 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 429279, 2024-04-22T13:35:10.410 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 12, 2024-04-22T13:35:10.410 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 3, 2024-04-22T13:35:10.410 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.113124404, 2024-04-22T13:35:10.410 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.039881190, 2024-04-22T13:35:10.411 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.065762075, 2024-04-22T13:35:10.411 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.010563908, 2024-04-22T13:35:10.411 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:35:10.411 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 872, 2024-04-22T13:35:10.411 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.055194470, 2024-04-22T13:35:10.411 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000063296 2024-04-22T13:35:10.411 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.411 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:35:10.411 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1003, 2024-04-22T13:35:10.411 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000263933, 2024-04-22T13:35:10.411 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000263 2024-04-22T13:35:10.411 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.411 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1658032, 2024-04-22T13:35:10.411 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 145664 2024-04-22T13:35:10.411 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.412 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T13:35:10.412 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 69, 2024-04-22T13:35:10.412 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 235, 2024-04-22T13:35:10.412 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 15116, 2024-04-22T13:35:10.412 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 979595, 2024-04-22T13:35:10.412 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 54, 2024-04-22T13:35:10.412 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 9, 2024-04-22T13:35:10.412 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.028225867, 2024-04-22T13:35:10.412 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.008276308, 2024-04-22T13:35:10.412 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.016822995, 2024-04-22T13:35:10.412 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000236514, 2024-04-22T13:35:10.412 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:35:10.412 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 229, 2024-04-22T13:35:10.412 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.024563256, 2024-04-22T13:35:10.413 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000107263 2024-04-22T13:35:10.413 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.413 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:35:10.413 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 63, 2024-04-22T13:35:10.413 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000006355, 2024-04-22T13:35:10.413 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000100 2024-04-22T13:35:10.413 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.413 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 5968, 2024-04-22T13:35:10.413 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 458144 2024-04-22T13:35:10.413 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.413 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T13:35:10.413 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 22, 2024-04-22T13:35:10.413 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 25, 2024-04-22T13:35:10.414 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 5875, 2024-04-22T13:35:10.414 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 14712, 2024-04-22T13:35:10.414 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 7, 2024-04-22T13:35:10.414 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 4, 2024-04-22T13:35:10.414 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.003972818, 2024-04-22T13:35:10.414 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.000537367, 2024-04-22T13:35:10.414 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.002794732, 2024-04-22T13:35:10.414 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.000495860, 2024-04-22T13:35:10.414 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:35:10.414 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 23, 2024-04-22T13:35:10.414 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.022576055, 2024-04-22T13:35:10.414 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000981567 2024-04-22T13:35:10.414 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.414 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:35:10.415 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 8, 2024-04-22T13:35:10.415 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000001641, 2024-04-22T13:35:10.415 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000205 2024-04-22T13:35:10.415 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.415 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 928, 2024-04-22T13:35:10.415 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 976 2024-04-22T13:35:10.415 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.415 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-04-22T13:35:10.415 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-04-22T13:35:10.415 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-04-22T13:35:10.415 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.415 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-04-22T13:35:10.415 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-04-22T13:35:10.415 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-04-22T13:35:10.416 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 57, 2024-04-22T13:35:10.416 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.002180776, 2024-04-22T13:35:10.416 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000038259 2024-04-22T13:35:10.416 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:10.416 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.416 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-04-22T13:35:10.416 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-04-22T13:35:10.416 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-04-22T13:35:10.416 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2, 2024-04-22T13:35:10.416 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000008389, 2024-04-22T13:35:10.416 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000004194 2024-04-22T13:35:10.417 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:10.417 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.417 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-04-22T13:35:10.417 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 3, 2024-04-22T13:35:10.417 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 3, 2024-04-22T13:35:10.417 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-04-22T13:35:10.417 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3, 2024-04-22T13:35:10.417 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.001937230, 2024-04-22T13:35:10.417 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000645743 2024-04-22T13:35:10.417 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.417 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-04-22T13:35:10.417 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 0, 2024-04-22T13:35:10.417 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2024-04-22T13:35:10.417 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-04-22T13:35:10.418 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 11, 2024-04-22T13:35:10.418 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2024-04-22T13:35:10.418 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-04-22T13:35:10.418 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 11, 2024-04-22T13:35:10.418 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 0, 2024-04-22T13:35:10.418 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 11, 2024-04-22T13:35:10.418 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 0, 2024-04-22T13:35:10.418 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 10, 2024-04-22T13:35:10.418 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 1, 2024-04-22T13:35:10.418 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 0, 2024-04-22T13:35:10.418 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 0, 2024-04-22T13:35:10.418 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 2, 2024-04-22T13:35:10.418 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 12, 2024-04-22T13:35:10.419 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 8, 2024-04-22T13:35:10.419 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 0, 2024-04-22T13:35:10.419 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 2, 2024-04-22T13:35:10.419 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-04-22T13:35:10.419 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-04-22T13:35:10.419 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2024-04-22T13:35:10.419 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2024-04-22T13:35:10.419 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-04-22T13:35:10.419 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2024-04-22T13:35:10.419 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2024-04-22T13:35:10.419 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 0, 2024-04-22T13:35:10.419 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 0, 2024-04-22T13:35:10.420 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-04-22T13:35:10.420 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 0, 2024-04-22T13:35:10.420 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2024-04-22T13:35:10.420 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 4, 2024-04-22T13:35:10.420 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 234, 2024-04-22T13:35:10.420 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 6, 2024-04-22T13:35:10.420 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 0, 2024-04-22T13:35:10.420 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-04-22T13:35:10.420 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-04-22T13:35:10.420 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-04-22T13:35:10.420 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2024-04-22T13:35:10.420 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2024-04-22T13:35:10.420 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2024-04-22T13:35:10.420 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 0, 2024-04-22T13:35:10.421 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2024-04-22T13:35:10.421 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 2, 2024-04-22T13:35:10.421 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 0, 2024-04-22T13:35:10.421 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-04-22T13:35:10.421 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-22T13:35:10.421 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-22T13:35:10.421 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 2, 2024-04-22T13:35:10.421 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 0, 2024-04-22T13:35:10.421 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 0, 2024-04-22T13:35:10.421 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 0 2024-04-22T13:35:10.421 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.421 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-04-22T13:35:10.421 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2024-04-22T13:35:10.421 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2024-04-22T13:35:10.422 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2024-04-22T13:35:10.422 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 3, 2024-04-22T13:35:10.422 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 0, 2024-04-22T13:35:10.422 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2024-04-22T13:35:10.422 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-04-22T13:35:10.422 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-04-22T13:35:10.422 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 0, 2024-04-22T13:35:10.422 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 0, 2024-04-22T13:35:10.422 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-04-22T13:35:10.422 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-04-22T13:35:10.422 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-04-22T13:35:10.422 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-04-22T13:35:10.422 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-04-22T13:35:10.422 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-04-22T13:35:10.423 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-04-22T13:35:10.423 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_path": 2, 2024-04-22T13:35:10.423 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_inode": 4, 2024-04-22T13:35:10.423 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2024-04-22T13:35:10.423 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2024-04-22T13:35:10.423 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-04-22T13:35:10.423 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2024-04-22T13:35:10.423 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-04-22T13:35:10.423 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2024-04-22T13:35:10.423 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.423 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-04-22T13:35:10.423 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 0, 2024-04-22T13:35:10.423 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 7, 2024-04-22T13:35:10.423 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-04-22T13:35:10.423 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-04-22T13:35:10.424 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 7, 2024-04-22T13:35:10.424 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-04-22T13:35:10.424 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-04-22T13:35:10.424 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 2, 2024-04-22T13:35:10.424 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-04-22T13:35:10.424 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-04-22T13:35:10.424 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 2, 2024-04-22T13:35:10.424 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 2, 2024-04-22T13:35:10.424 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-04-22T13:35:10.424 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-04-22T13:35:10.424 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 4194304, 2024-04-22T13:35:10.424 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 4196347, 2024-04-22T13:35:10.424 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 0, 2024-04-22T13:35:10.424 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-04-22T13:35:10.425 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 6, 2024-04-22T13:35:10.425 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.010927537, 2024-04-22T13:35:10.425 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.001821256 2024-04-22T13:35:10.425 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.425 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 0 2024-04-22T13:35:10.425 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.425 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-04-22T13:35:10.425 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 14, 2024-04-22T13:35:10.425 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 14, 2024-04-22T13:35:10.425 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 0, 2024-04-22T13:35:10.425 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 13, 2024-04-22T13:35:10.425 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 14, 2024-04-22T13:35:10.425 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 1, 2024-04-22T13:35:10.425 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 11, 2024-04-22T13:35:10.426 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 12, 2024-04-22T13:35:10.426 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 1, 2024-04-22T13:35:10.426 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 0, 2024-04-22T13:35:10.426 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 2, 2024-04-22T13:35:10.426 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 2, 2024-04-22T13:35:10.426 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 55488, 2024-04-22T13:35:10.426 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 223608 2024-04-22T13:35:10.426 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.426 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-04-22T13:35:10.426 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 5, 2024-04-22T13:35:10.426 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 0, 2024-04-22T13:35:10.426 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 3, 2024-04-22T13:35:10.426 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 17, 2024-04-22T13:35:10.426 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 4, 2024-04-22T13:35:10.427 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-04-22T13:35:10.427 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.427 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.427 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.427 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.427 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-04-22T13:35:10.427 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1, 2024-04-22T13:35:10.427 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000292384, 2024-04-22T13:35:10.427 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000292384 2024-04-22T13:35:10.427 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.427 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-04-22T13:35:10.427 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.427 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.427 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.427 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.428 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-04-22T13:35:10.428 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.428 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.428 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.428 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.428 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-04-22T13:35:10.428 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2, 2024-04-22T13:35:10.428 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.001644846, 2024-04-22T13:35:10.428 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000822423 2024-04-22T13:35:10.428 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.428 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-04-22T13:35:10.428 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.428 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.428 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.429 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.429 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-04-22T13:35:10.429 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.429 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.429 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.429 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.429 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-04-22T13:35:10.429 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.429 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.429 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.429 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.429 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-04-22T13:35:10.429 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.429 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.429 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.430 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.430 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-04-22T13:35:10.430 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.430 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.430 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.430 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.430 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-04-22T13:35:10.430 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.430 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.430 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.430 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.430 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-04-22T13:35:10.430 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.431 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.431 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.431 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.431 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-04-22T13:35:10.431 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.431 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.431 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.432 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.432 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-04-22T13:35:10.432 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.432 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.432 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.432 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.432 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-04-22T13:35:10.432 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.432 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.432 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.432 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.432 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-04-22T13:35:10.432 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.432 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.432 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.433 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.433 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-04-22T13:35:10.433 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.433 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.433 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.433 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.433 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-04-22T13:35:10.433 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.433 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.433 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.433 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.433 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-04-22T13:35:10.433 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.433 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.434 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.434 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.434 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-04-22T13:35:10.434 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.434 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.434 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.434 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.434 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-04-22T13:35:10.434 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.434 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.434 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.434 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.434 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-04-22T13:35:10.435 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.435 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.435 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.435 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.435 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-04-22T13:35:10.435 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.435 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.435 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.435 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.435 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-04-22T13:35:10.435 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.435 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.435 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.435 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.436 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-04-22T13:35:10.436 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.436 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.436 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.436 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.436 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-04-22T13:35:10.436 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.436 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.436 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.436 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.436 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-04-22T13:35:10.436 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.436 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.436 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.436 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.437 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-04-22T13:35:10.437 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.437 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.437 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.437 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.437 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-04-22T13:35:10.437 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.437 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.437 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.437 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.437 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-04-22T13:35:10.437 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-04-22T13:35:10.437 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-04-22T13:35:10.437 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.438 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.438 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.438 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:10.438 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.438 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-04-22T13:35:10.438 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 1, 2024-04-22T13:35:10.438 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 3, 2024-04-22T13:35:10.438 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 2, 2024-04-22T13:35:10.438 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 1, 2024-04-22T13:35:10.438 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-04-22T13:35:10.438 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 2, 2024-04-22T13:35:10.438 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 2, 2024-04-22T13:35:10.438 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 605, 2024-04-22T13:35:10.438 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-04-22T13:35:10.438 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.439 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-04-22T13:35:10.439 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-04-22T13:35:10.439 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-04-22T13:35:10.439 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T13:35:10.439 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-04-22T13:35:10.439 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T13:35:10.439 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-04-22T13:35:10.439 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T13:35:10.439 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T13:35:10.439 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T13:35:10.439 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T13:35:10.439 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T13:35:10.439 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-04-22T13:35:10.439 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T13:35:10.440 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T13:35:10.440 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-04-22T13:35:10.440 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-04-22T13:35:10.440 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-04-22T13:35:10.440 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-04-22T13:35:10.440 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T13:35:10.440 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T13:35:10.440 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T13:35:10.440 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T13:35:10.440 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T13:35:10.440 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-04-22T13:35:10.440 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-04-22T13:35:10.440 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-04-22T13:35:10.440 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T13:35:10.440 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T13:35:10.440 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-04-22T13:35:10.441 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-04-22T13:35:10.441 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-04-22T13:35:10.441 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-04-22T13:35:10.441 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T13:35:10.441 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-04-22T13:35:10.441 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T13:35:10.441 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-04-22T13:35:10.441 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 247271, 2024-04-22T13:35:10.441 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 87, 2024-04-22T13:35:10.441 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-04-22T13:35:10.441 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-04-22T13:35:10.441 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-04-22T13:35:10.441 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-04-22T13:35:10.441 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-04-22T13:35:10.441 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-04-22T13:35:10.441 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-04-22T13:35:10.442 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-04-22T13:35:10.442 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 7936, 2024-04-22T13:35:10.442 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 165, 2024-04-22T13:35:10.442 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T13:35:10.442 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-04-22T13:35:10.442 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-04-22T13:35:10.442 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-04-22T13:35:10.442 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 56544, 2024-04-22T13:35:10.442 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 224, 2024-04-22T13:35:10.442 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-04-22T13:35:10.442 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-04-22T13:35:10.442 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-04-22T13:35:10.442 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-04-22T13:35:10.442 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.442 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-04-22T13:35:10.443 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-04-22T13:35:10.443 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-04-22T13:35:10.443 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 45, 2024-04-22T13:35:10.443 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 12580, 2024-04-22T13:35:10.443 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 31, 2024-04-22T13:35:10.443 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 45, 2024-04-22T13:35:10.443 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-04-22T13:35:10.443 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 45, 2024-04-22T13:35:10.443 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.244993864, 2024-04-22T13:35:10.443 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.005444308 2024-04-22T13:35:10.443 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.443 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-04-22T13:35:10.443 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-04-22T13:35:10.443 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 45, 2024-04-22T13:35:10.443 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 69 2024-04-22T13:35:10.443 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.444 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 45, 2024-04-22T13:35:10.444 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 0, 2024-04-22T13:35:10.444 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 45, 2024-04-22T13:35:10.444 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-04-22T13:35:10.444 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-04-22T13:35:10.444 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 0, 2024-04-22T13:35:10.444 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 10, 2024-04-22T13:35:10.444 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 0, 2024-04-22T13:35:10.444 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 6, 2024-04-22T13:35:10.444 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 8, 2024-04-22T13:35:10.444 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-04-22T13:35:10.444 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-04-22T13:35:10.444 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-04-22T13:35:10.444 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2024-04-22T13:35:10.444 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 6, 2024-04-22T13:35:10.444 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-04-22T13:35:10.444 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-04-22T13:35:10.445 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-04-22T13:35:10.445 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 0, 2024-04-22T13:35:10.445 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 20, 2024-04-22T13:35:10.445 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-04-22T13:35:10.445 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-04-22T13:35:10.445 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-04-22T13:35:10.445 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-04-22T13:35:10.445 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-04-22T13:35:10.445 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-04-22T13:35:10.445 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-04-22T13:35:10.445 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-04-22T13:35:10.445 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-04-22T13:35:10.445 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-04-22T13:35:10.445 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-04-22T13:35:10.445 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-04-22T13:35:10.445 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-04-22T13:35:10.446 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-04-22T13:35:10.446 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-04-22T13:35:10.446 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-04-22T13:35:10.446 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-04-22T13:35:10.446 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-04-22T13:35:10.446 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-04-22T13:35:10.446 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-04-22T13:35:10.446 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-04-22T13:35:10.446 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-04-22T13:35:10.446 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-04-22T13:35:10.446 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-04-22T13:35:10.446 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-04-22T13:35:10.446 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-04-22T13:35:10.446 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 98, 2024-04-22T13:35:10.446 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-04-22T13:35:10.446 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 11, 2024-04-22T13:35:10.447 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 9, 2024-04-22T13:35:10.447 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 9, 2024-04-22T13:35:10.447 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2024-04-22T13:35:10.447 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-04-22T13:35:10.447 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 17, 2024-04-22T13:35:10.447 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 0, 2024-04-22T13:35:10.447 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 2 2024-04-22T13:35:10.447 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.447 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-04-22T13:35:10.447 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 1, 2024-04-22T13:35:10.447 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-04-22T13:35:10.447 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-04-22T13:35:10.447 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-04-22T13:35:10.447 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.447 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2024-04-22T13:35:10.447 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 6 2024-04-22T13:35:10.448 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.448 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-04-22T13:35:10.448 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-04-22T13:35:10.448 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 0, 2024-04-22T13:35:10.448 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-04-22T13:35:10.448 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 0, 2024-04-22T13:35:10.448 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 0, 2024-04-22T13:35:10.448 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 0, 2024-04-22T13:35:10.448 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-04-22T13:35:10.448 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.448 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T13:35:10.448 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-22T13:35:10.448 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-04-22T13:35:10.448 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-22T13:35:10.448 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 820, 2024-04-22T13:35:10.448 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 1714843, 2024-04-22T13:35:10.449 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-22T13:35:10.449 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 820, 2024-04-22T13:35:10.449 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-22T13:35:10.449 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-22T13:35:10.449 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 820, 2024-04-22T13:35:10.449 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 1714843, 2024-04-22T13:35:10.449 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-22T13:35:10.449 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.449 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.449 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.449 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:10.449 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.449 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-04-22T13:35:10.449 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-22T13:35:10.449 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-04-22T13:35:10.449 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-22T13:35:10.449 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-22T13:35:10.449 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-22T13:35:10.450 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-22T13:35:10.450 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-22T13:35:10.450 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 45, 2024-04-22T13:35:10.450 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 12580, 2024-04-22T13:35:10.450 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 45, 2024-04-22T13:35:10.450 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 12580, 2024-04-22T13:35:10.450 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-22T13:35:10.450 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.450 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.450 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.450 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:10.450 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.450 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-04-22T13:35:10.450 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-22T13:35:10.451 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-04-22T13:35:10.451 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-22T13:35:10.451 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-22T13:35:10.451 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-22T13:35:10.451 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-22T13:35:10.451 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-22T13:35:10.451 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 45, 2024-04-22T13:35:10.451 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 45, 2024-04-22T13:35:10.451 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 45, 2024-04-22T13:35:10.451 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 45, 2024-04-22T13:35:10.451 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-22T13:35:10.451 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.451 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.451 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.451 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:10.451 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.451 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-04-22T13:35:10.451 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-22T13:35:10.452 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-04-22T13:35:10.452 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-22T13:35:10.452 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-22T13:35:10.452 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-22T13:35:10.452 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-22T13:35:10.452 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-22T13:35:10.452 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-22T13:35:10.452 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-22T13:35:10.452 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-04-22T13:35:10.452 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-04-22T13:35:10.452 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-22T13:35:10.452 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.452 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.452 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.452 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:10.452 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:10.452 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x55968b55c500": { 2024-04-22T13:35:10.453 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-22T13:35:10.453 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-04-22T13:35:10.453 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-22T13:35:10.453 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 7, 2024-04-22T13:35:10.453 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 2043, 2024-04-22T13:35:10.453 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-22T13:35:10.453 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 7, 2024-04-22T13:35:10.453 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-22T13:35:10.453 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-22T13:35:10.453 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 6, 2024-04-22T13:35:10.453 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 2043, 2024-04-22T13:35:10.453 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-22T13:35:10.453 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:10.453 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:10.453 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:10.454 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:10.454 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:10.454 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-04-22T13:35:11.185 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-22T13:35:11.185 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-22T13:35:11.185 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-22T13:35:11.185 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-22T13:35:11.185 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-22T13:35:11.185 DEBUG:tasks.check_counter:Getting stats from l 2024-04-22T13:35:11.185 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-22T13:35:11.421 INFO:teuthology.orchestra.run.smithi134.stderr:Inferring config /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/mon.c/config 2024-04-22T13:35:11.809 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:11 smithi134 ceph-mon[31338]: pgmap v343: 97 pgs: 97 active+clean; 2.3 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:35:11.809 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:11 smithi134 ceph-mon[31338]: osdmap e99: 12 total, 12 up, 12 in 2024-04-22T13:35:11.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:11 smithi111 ceph-mon[31361]: pgmap v343: 97 pgs: 97 active+clean; 2.3 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:35:11.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:11 smithi111 ceph-mon[31361]: osdmap e99: 12 total, 12 up, 12 in 2024-04-22T13:35:12.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:11 smithi012 ceph-mon[28265]: pgmap v343: 97 pgs: 97 active+clean; 2.3 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-22T13:35:12.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:11 smithi012 ceph-mon[28265]: osdmap e99: 12 total, 12 up, 12 in 2024-04-22T13:35:13.084 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-04-22T13:35:13.084 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-0": { 2024-04-22T13:35:13.085 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 794, 2024-04-22T13:35:13.085 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 794, 2024-04-22T13:35:13.085 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 213382, 2024-04-22T13:35:13.085 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 251732, 2024-04-22T13:35:13.085 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 6, 2024-04-22T13:35:13.085 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 2, 2024-04-22T13:35:13.085 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.079364329, 2024-04-22T13:35:13.085 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.025089761, 2024-04-22T13:35:13.085 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.021404488, 2024-04-22T13:35:13.085 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.024398065, 2024-04-22T13:35:13.085 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:35:13.085 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 793, 2024-04-22T13:35:13.085 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.033788179, 2024-04-22T13:35:13.086 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000042608 2024-04-22T13:35:13.086 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.086 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:35:13.086 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.086 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.086 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.086 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.086 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-22T13:35:13.086 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 448 2024-04-22T13:35:13.086 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.086 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-1": { 2024-04-22T13:35:13.086 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 702, 2024-04-22T13:35:13.086 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 869, 2024-04-22T13:35:13.087 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 172542, 2024-04-22T13:35:13.087 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 677501, 2024-04-22T13:35:13.087 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 7, 2024-04-22T13:35:13.087 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 5, 2024-04-22T13:35:13.087 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.086044775, 2024-04-22T13:35:13.087 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.026901803, 2024-04-22T13:35:13.087 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.020199265, 2024-04-22T13:35:13.087 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.030578114, 2024-04-22T13:35:13.087 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:35:13.087 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 866, 2024-04-22T13:35:13.087 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.040706731, 2024-04-22T13:35:13.087 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000047005 2024-04-22T13:35:13.088 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.088 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:35:13.088 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.088 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.088 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.088 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.088 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-22T13:35:13.088 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 454896 2024-04-22T13:35:13.088 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.088 INFO:teuthology.orchestra.run.smithi134.stdout: "AsyncMessenger::Worker-2": { 2024-04-22T13:35:13.088 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_messages": 1030, 2024-04-22T13:35:13.088 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages": 1018, 2024-04-22T13:35:13.089 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_bytes": 1884752, 2024-04-22T13:35:13.089 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_bytes": 386728, 2024-04-22T13:35:13.089 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_created_connections": 3, 2024-04-22T13:35:13.089 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_active_connections": 2, 2024-04-22T13:35:13.089 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_total_time": 0.147679973, 2024-04-22T13:35:13.089 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_send_time": 0.044339525, 2024-04-22T13:35:13.089 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_recv_time": 0.063674580, 2024-04-22T13:35:13.089 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_running_fast_dispatch_time": 0.039995862, 2024-04-22T13:35:13.089 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_messages_queue_lat": { 2024-04-22T13:35:13.089 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 1017, 2024-04-22T13:35:13.089 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.055729391, 2024-04-22T13:35:13.089 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000054797 2024-04-22T13:35:13.089 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.089 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_handle_ack_lat": { 2024-04-22T13:35:13.089 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.089 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.089 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.090 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.090 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_recv_encrypted_bytes": 1622672, 2024-04-22T13:35:13.090 INFO:teuthology.orchestra.run.smithi134.stdout: "msgr_send_encrypted_bytes": 138688 2024-04-22T13:35:13.090 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.090 INFO:teuthology.orchestra.run.smithi134.stdout: "cct": { 2024-04-22T13:35:13.090 INFO:teuthology.orchestra.run.smithi134.stdout: "total_workers": 1, 2024-04-22T13:35:13.090 INFO:teuthology.orchestra.run.smithi134.stdout: "unhealthy_workers": 0 2024-04-22T13:35:13.090 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.090 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-MDSRank": { 2024-04-22T13:35:13.090 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-04-22T13:35:13.090 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-04-22T13:35:13.090 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 3763, 2024-04-22T13:35:13.090 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.118798626, 2024-04-22T13:35:13.090 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000031570 2024-04-22T13:35:13.091 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:13.091 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.091 INFO:teuthology.orchestra.run.smithi134.stdout: "finisher-PurgeQueue": { 2024-04-22T13:35:13.091 INFO:teuthology.orchestra.run.smithi134.stdout: "queue_len": 0, 2024-04-22T13:35:13.091 INFO:teuthology.orchestra.run.smithi134.stdout: "complete_latency": { 2024-04-22T13:35:13.091 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.091 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.091 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.091 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:13.091 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.091 INFO:teuthology.orchestra.run.smithi134.stdout: "mds": { 2024-04-22T13:35:13.091 INFO:teuthology.orchestra.run.smithi134.stdout: "request": 0, 2024-04-22T13:35:13.091 INFO:teuthology.orchestra.run.smithi134.stdout: "reply": 0, 2024-04-22T13:35:13.092 INFO:teuthology.orchestra.run.smithi134.stdout: "reply_latency": { 2024-04-22T13:35:13.092 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.092 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.092 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.092 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.092 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_reply": 0, 2024-04-22T13:35:13.092 INFO:teuthology.orchestra.run.smithi134.stdout: "forward": 0, 2024-04-22T13:35:13.092 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_complete": 0, 2024-04-22T13:35:13.092 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_fetch_keys": 0, 2024-04-22T13:35:13.092 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_commit": 0, 2024-04-22T13:35:13.092 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_split": 0, 2024-04-22T13:35:13.093 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_merge": 0, 2024-04-22T13:35:13.093 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes": 0, 2024-04-22T13:35:13.093 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_top": 0, 2024-04-22T13:35:13.093 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_bottom": 0, 2024-04-22T13:35:13.093 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pin_tail": 0, 2024-04-22T13:35:13.093 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_pinned": 0, 2024-04-22T13:35:13.093 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_expired": 0, 2024-04-22T13:35:13.093 INFO:teuthology.orchestra.run.smithi134.stdout: "inodes_with_caps": 0, 2024-04-22T13:35:13.093 INFO:teuthology.orchestra.run.smithi134.stdout: "caps": 0, 2024-04-22T13:35:13.093 INFO:teuthology.orchestra.run.smithi134.stdout: "subtrees": 1, 2024-04-22T13:35:13.093 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse": 0, 2024-04-22T13:35:13.093 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_hit": 0, 2024-04-22T13:35:13.093 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_forward": 0, 2024-04-22T13:35:13.093 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_discover": 0, 2024-04-22T13:35:13.093 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_dir_fetch": 0, 2024-04-22T13:35:13.093 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_remote_ino": 0, 2024-04-22T13:35:13.094 INFO:teuthology.orchestra.run.smithi134.stdout: "traverse_lock": 0, 2024-04-22T13:35:13.094 INFO:teuthology.orchestra.run.smithi134.stdout: "load_cent": 0, 2024-04-22T13:35:13.094 INFO:teuthology.orchestra.run.smithi134.stdout: "q": 0, 2024-04-22T13:35:13.094 INFO:teuthology.orchestra.run.smithi134.stdout: "exported": 0, 2024-04-22T13:35:13.094 INFO:teuthology.orchestra.run.smithi134.stdout: "exported_inodes": 0, 2024-04-22T13:35:13.094 INFO:teuthology.orchestra.run.smithi134.stdout: "imported": 0, 2024-04-22T13:35:13.094 INFO:teuthology.orchestra.run.smithi134.stdout: "imported_inodes": 0, 2024-04-22T13:35:13.094 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_dir_fetch": 0, 2024-04-22T13:35:13.094 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_backtrace_fetch": 0, 2024-04-22T13:35:13.094 INFO:teuthology.orchestra.run.smithi134.stdout: "openino_peer_discover": 0, 2024-04-22T13:35:13.095 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rfiles": 0, 2024-04-22T13:35:13.095 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rbytes": 0, 2024-04-22T13:35:13.095 INFO:teuthology.orchestra.run.smithi134.stdout: "root_rsnaps": 0, 2024-04-22T13:35:13.095 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_fetch": 0, 2024-04-22T13:35:13.095 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_set_tag": 0, 2024-04-22T13:35:13.095 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_backtrace_repaired": 0, 2024-04-22T13:35:13.095 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_inotable_repaired": 0, 2024-04-22T13:35:13.095 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_inodes": 0, 2024-04-22T13:35:13.095 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dir_base_inodes": 0, 2024-04-22T13:35:13.095 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_dirfrag_rstats": 0, 2024-04-22T13:35:13.095 INFO:teuthology.orchestra.run.smithi134.stdout: "scrub_file_inodes": 0, 2024-04-22T13:35:13.095 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_inode_file_caps": 0, 2024-04-22T13:35:13.095 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_revoke": 0, 2024-04-22T13:35:13.095 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_grant": 0, 2024-04-22T13:35:13.095 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_trunc": 0, 2024-04-22T13:35:13.095 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-22T13:35:13.096 INFO:teuthology.orchestra.run.smithi134.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-22T13:35:13.096 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps": 0, 2024-04-22T13:35:13.096 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_caps_dirty": 0, 2024-04-22T13:35:13.096 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_cap_release": 0, 2024-04-22T13:35:13.096 INFO:teuthology.orchestra.run.smithi134.stdout: "process_request_cap_release": 0 2024-04-22T13:35:13.096 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.096 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_cache": { 2024-04-22T13:35:13.096 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update": 0, 2024-04-22T13:35:13.096 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_update_receipt": 0, 2024-04-22T13:35:13.096 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_try_discover": 0, 2024-04-22T13:35:13.096 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_send_discover": 0, 2024-04-22T13:35:13.096 INFO:teuthology.orchestra.run.smithi134.stdout: "dir_handle_discover": 0, 2024-04-22T13:35:13.096 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays": 0, 2024-04-22T13:35:13.096 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_delayed": 0, 2024-04-22T13:35:13.096 INFO:teuthology.orchestra.run.smithi134.stdout: "num_strays_enqueuing": 0, 2024-04-22T13:35:13.097 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_created": 0, 2024-04-22T13:35:13.097 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_enqueued": 0, 2024-04-22T13:35:13.097 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_reintegrated": 0, 2024-04-22T13:35:13.097 INFO:teuthology.orchestra.run.smithi134.stdout: "strays_migrated": 0, 2024-04-22T13:35:13.097 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_processing": 0, 2024-04-22T13:35:13.097 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_enqueued": 0, 2024-04-22T13:35:13.097 INFO:teuthology.orchestra.run.smithi134.stdout: "num_recovering_prioritized": 0, 2024-04-22T13:35:13.097 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_started": 0, 2024-04-22T13:35:13.097 INFO:teuthology.orchestra.run.smithi134.stdout: "recovery_completed": 0, 2024-04-22T13:35:13.097 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_path": 0, 2024-04-22T13:35:13.097 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_quiesce_inode": 0, 2024-04-22T13:35:13.097 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_enqueue_scrub": 0, 2024-04-22T13:35:13.097 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_exportdir": 0, 2024-04-22T13:35:13.097 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_flush": 0, 2024-04-22T13:35:13.097 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragmentdir": 0, 2024-04-22T13:35:13.098 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_fragstats": 0, 2024-04-22T13:35:13.098 INFO:teuthology.orchestra.run.smithi134.stdout: "ireq_inodestats": 0 2024-04-22T13:35:13.098 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.098 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_log": { 2024-04-22T13:35:13.098 INFO:teuthology.orchestra.run.smithi134.stdout: "evlrg": 0, 2024-04-22T13:35:13.098 INFO:teuthology.orchestra.run.smithi134.stdout: "evadd": 0, 2024-04-22T13:35:13.098 INFO:teuthology.orchestra.run.smithi134.stdout: "evex": 0, 2024-04-22T13:35:13.098 INFO:teuthology.orchestra.run.smithi134.stdout: "evtrm": 0, 2024-04-22T13:35:13.098 INFO:teuthology.orchestra.run.smithi134.stdout: "ev": 10, 2024-04-22T13:35:13.098 INFO:teuthology.orchestra.run.smithi134.stdout: "evexg": 0, 2024-04-22T13:35:13.098 INFO:teuthology.orchestra.run.smithi134.stdout: "evexd": 0, 2024-04-22T13:35:13.098 INFO:teuthology.orchestra.run.smithi134.stdout: "segadd": 0, 2024-04-22T13:35:13.098 INFO:teuthology.orchestra.run.smithi134.stdout: "segex": 0, 2024-04-22T13:35:13.098 INFO:teuthology.orchestra.run.smithi134.stdout: "segtrm": 0, 2024-04-22T13:35:13.098 INFO:teuthology.orchestra.run.smithi134.stdout: "seg": 2, 2024-04-22T13:35:13.099 INFO:teuthology.orchestra.run.smithi134.stdout: "segmjr": 2, 2024-04-22T13:35:13.099 INFO:teuthology.orchestra.run.smithi134.stdout: "segexg": 0, 2024-04-22T13:35:13.099 INFO:teuthology.orchestra.run.smithi134.stdout: "segexd": 0, 2024-04-22T13:35:13.099 INFO:teuthology.orchestra.run.smithi134.stdout: "expos": 4194304, 2024-04-22T13:35:13.099 INFO:teuthology.orchestra.run.smithi134.stdout: "wrpos": 4197187, 2024-04-22T13:35:13.099 INFO:teuthology.orchestra.run.smithi134.stdout: "rdpos": 4197046, 2024-04-22T13:35:13.099 INFO:teuthology.orchestra.run.smithi134.stdout: "jlat": { 2024-04-22T13:35:13.099 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.099 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.099 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.099 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.099 INFO:teuthology.orchestra.run.smithi134.stdout: "replayed": 10 2024-04-22T13:35:13.099 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.099 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_mem": { 2024-04-22T13:35:13.099 INFO:teuthology.orchestra.run.smithi134.stdout: "ino": 3, 2024-04-22T13:35:13.100 INFO:teuthology.orchestra.run.smithi134.stdout: "ino+": 3, 2024-04-22T13:35:13.100 INFO:teuthology.orchestra.run.smithi134.stdout: "ino-": 0, 2024-04-22T13:35:13.100 INFO:teuthology.orchestra.run.smithi134.stdout: "dir": 1, 2024-04-22T13:35:13.100 INFO:teuthology.orchestra.run.smithi134.stdout: "dir+": 1, 2024-04-22T13:35:13.100 INFO:teuthology.orchestra.run.smithi134.stdout: "dir-": 0, 2024-04-22T13:35:13.100 INFO:teuthology.orchestra.run.smithi134.stdout: "dn": 0, 2024-04-22T13:35:13.100 INFO:teuthology.orchestra.run.smithi134.stdout: "dn+": 0, 2024-04-22T13:35:13.100 INFO:teuthology.orchestra.run.smithi134.stdout: "dn-": 0, 2024-04-22T13:35:13.100 INFO:teuthology.orchestra.run.smithi134.stdout: "cap": 0, 2024-04-22T13:35:13.100 INFO:teuthology.orchestra.run.smithi134.stdout: "cap+": 0, 2024-04-22T13:35:13.100 INFO:teuthology.orchestra.run.smithi134.stdout: "cap-": 0, 2024-04-22T13:35:13.100 INFO:teuthology.orchestra.run.smithi134.stdout: "rss": 52808, 2024-04-22T13:35:13.100 INFO:teuthology.orchestra.run.smithi134.stdout: "heap": 215416 2024-04-22T13:35:13.100 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.100 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_server": { 2024-04-22T13:35:13.101 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_client_request": 0, 2024-04-22T13:35:13.101 INFO:teuthology.orchestra.run.smithi134.stdout: "dispatch_server_request": 0, 2024-04-22T13:35:13.101 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_request": 0, 2024-04-22T13:35:13.101 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_client_session": 0, 2024-04-22T13:35:13.101 INFO:teuthology.orchestra.run.smithi134.stdout: "handle_peer_request": 0, 2024-04-22T13:35:13.101 INFO:teuthology.orchestra.run.smithi134.stdout: "req_create_latency": { 2024-04-22T13:35:13.101 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.101 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.101 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.101 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.102 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getattr_latency": { 2024-04-22T13:35:13.102 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.102 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.102 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.102 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.102 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getfilelock_latency": { 2024-04-22T13:35:13.102 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.102 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.102 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.102 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.102 INFO:teuthology.orchestra.run.smithi134.stdout: "req_link_latency": { 2024-04-22T13:35:13.102 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.102 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.102 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.102 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.103 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookup_latency": { 2024-04-22T13:35:13.103 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.103 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.103 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.103 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.103 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookuphash_latency": { 2024-04-22T13:35:13.103 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.103 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.103 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.103 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.103 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupino_latency": { 2024-04-22T13:35:13.103 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.103 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.103 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.103 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.104 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupname_latency": { 2024-04-22T13:35:13.104 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.104 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.104 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.104 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.104 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupparent_latency": { 2024-04-22T13:35:13.104 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.104 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.104 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.104 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.104 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lookupsnap_latency": { 2024-04-22T13:35:13.104 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.104 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.104 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.104 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.105 INFO:teuthology.orchestra.run.smithi134.stdout: "req_lssnap_latency": { 2024-04-22T13:35:13.105 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.105 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.105 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.105 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.105 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mkdir_latency": { 2024-04-22T13:35:13.105 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.106 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.106 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.106 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.106 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mknod_latency": { 2024-04-22T13:35:13.106 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.106 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.106 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.106 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.106 INFO:teuthology.orchestra.run.smithi134.stdout: "req_mksnap_latency": { 2024-04-22T13:35:13.106 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.106 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.107 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.107 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.107 INFO:teuthology.orchestra.run.smithi134.stdout: "req_open_latency": { 2024-04-22T13:35:13.107 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.107 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.107 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.107 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.107 INFO:teuthology.orchestra.run.smithi134.stdout: "req_readdir_latency": { 2024-04-22T13:35:13.107 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.107 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.107 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.107 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.107 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rename_latency": { 2024-04-22T13:35:13.107 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.108 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.108 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.108 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.108 INFO:teuthology.orchestra.run.smithi134.stdout: "req_renamesnap_latency": { 2024-04-22T13:35:13.108 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.108 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.108 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.108 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.108 INFO:teuthology.orchestra.run.smithi134.stdout: "req_snapdiff_latency": { 2024-04-22T13:35:13.108 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.108 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.108 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.108 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.108 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmdir_latency": { 2024-04-22T13:35:13.109 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.109 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.109 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.109 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.109 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmsnap_latency": { 2024-04-22T13:35:13.109 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.109 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.109 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.109 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.109 INFO:teuthology.orchestra.run.smithi134.stdout: "req_rmxattr_latency": { 2024-04-22T13:35:13.109 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.109 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.109 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.110 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.110 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setattr_latency": { 2024-04-22T13:35:13.110 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.110 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.110 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.110 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.110 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setdirlayout_latency": { 2024-04-22T13:35:13.110 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.110 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.110 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.110 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.110 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setfilelock_latency": { 2024-04-22T13:35:13.110 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.110 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.111 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.111 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.111 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setlayout_latency": { 2024-04-22T13:35:13.111 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.111 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.111 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.111 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.111 INFO:teuthology.orchestra.run.smithi134.stdout: "req_setxattr_latency": { 2024-04-22T13:35:13.111 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.111 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.111 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.111 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.111 INFO:teuthology.orchestra.run.smithi134.stdout: "req_symlink_latency": { 2024-04-22T13:35:13.111 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.112 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.112 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.112 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.112 INFO:teuthology.orchestra.run.smithi134.stdout: "req_unlink_latency": { 2024-04-22T13:35:13.112 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.112 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.112 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.112 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.112 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_revoke_eviction": 0, 2024-04-22T13:35:13.112 INFO:teuthology.orchestra.run.smithi134.stdout: "cap_acquisition_throttle": 0, 2024-04-22T13:35:13.112 INFO:teuthology.orchestra.run.smithi134.stdout: "req_getvxattr_latency": { 2024-04-22T13:35:13.112 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.112 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.112 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.113 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:13.113 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.113 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_sessions": { 2024-04-22T13:35:13.113 INFO:teuthology.orchestra.run.smithi134.stdout: "session_count": 2, 2024-04-22T13:35:13.113 INFO:teuthology.orchestra.run.smithi134.stdout: "session_add": 5, 2024-04-22T13:35:13.113 INFO:teuthology.orchestra.run.smithi134.stdout: "session_remove": 3, 2024-04-22T13:35:13.113 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_open": 4, 2024-04-22T13:35:13.113 INFO:teuthology.orchestra.run.smithi134.stdout: "sessions_stale": 0, 2024-04-22T13:35:13.113 INFO:teuthology.orchestra.run.smithi134.stdout: "total_load": 0, 2024-04-22T13:35:13.113 INFO:teuthology.orchestra.run.smithi134.stdout: "average_load": 0, 2024-04-22T13:35:13.113 INFO:teuthology.orchestra.run.smithi134.stdout: "avg_session_uptime": 3015, 2024-04-22T13:35:13.113 INFO:teuthology.orchestra.run.smithi134.stdout: "mdthresh_evicted": 0 2024-04-22T13:35:13.113 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.113 INFO:teuthology.orchestra.run.smithi134.stdout: "mempool": { 2024-04-22T13:35:13.114 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_bytes": 0, 2024-04-22T13:35:13.114 INFO:teuthology.orchestra.run.smithi134.stdout: "bloom_filter_items": 0, 2024-04-22T13:35:13.114 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_bytes": 0, 2024-04-22T13:35:13.114 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_alloc_items": 0, 2024-04-22T13:35:13.114 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_bytes": 0, 2024-04-22T13:35:13.114 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_data_items": 0, 2024-04-22T13:35:13.114 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-22T13:35:13.114 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_onode_items": 0, 2024-04-22T13:35:13.114 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-22T13:35:13.114 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_meta_items": 0, 2024-04-22T13:35:13.114 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_bytes": 0, 2024-04-22T13:35:13.114 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_other_items": 0, 2024-04-22T13:35:13.114 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-22T13:35:13.114 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_cache_buffer_items": 0, 2024-04-22T13:35:13.114 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_bytes": 0, 2024-04-22T13:35:13.115 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_extent_items": 0, 2024-04-22T13:35:13.115 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_bytes": 0, 2024-04-22T13:35:13.115 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_blob_items": 0, 2024-04-22T13:35:13.115 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-22T13:35:13.115 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_shared_blob_items": 0, 2024-04-22T13:35:13.115 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-22T13:35:13.115 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_inline_bl_items": 0, 2024-04-22T13:35:13.115 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_bytes": 0, 2024-04-22T13:35:13.115 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_fsck_items": 0, 2024-04-22T13:35:13.115 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_bytes": 0, 2024-04-22T13:35:13.115 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_txc_items": 0, 2024-04-22T13:35:13.115 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-22T13:35:13.115 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_deferred_items": 0, 2024-04-22T13:35:13.116 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_bytes": 0, 2024-04-22T13:35:13.116 INFO:teuthology.orchestra.run.smithi134.stdout: "bluestore_writing_items": 0, 2024-04-22T13:35:13.116 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_bytes": 0, 2024-04-22T13:35:13.116 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_items": 0, 2024-04-22T13:35:13.116 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_bytes": 0, 2024-04-22T13:35:13.116 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_reader_items": 0, 2024-04-22T13:35:13.116 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_bytes": 0, 2024-04-22T13:35:13.116 INFO:teuthology.orchestra.run.smithi134.stdout: "bluefs_file_writer_items": 0, 2024-04-22T13:35:13.116 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_bytes": 189411, 2024-04-22T13:35:13.116 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_anon_items": 52, 2024-04-22T13:35:13.116 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_bytes": 0, 2024-04-22T13:35:13.116 INFO:teuthology.orchestra.run.smithi134.stdout: "buffer_meta_items": 0, 2024-04-22T13:35:13.116 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_bytes": 0, 2024-04-22T13:35:13.116 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_items": 0, 2024-04-22T13:35:13.116 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_bytes": 0, 2024-04-22T13:35:13.117 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_mapbl_items": 0, 2024-04-22T13:35:13.117 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_bytes": 0, 2024-04-22T13:35:13.117 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_pglog_items": 0, 2024-04-22T13:35:13.117 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_bytes": 7936, 2024-04-22T13:35:13.117 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_items": 165, 2024-04-22T13:35:13.117 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_bytes": 0, 2024-04-22T13:35:13.117 INFO:teuthology.orchestra.run.smithi134.stdout: "osdmap_mapping_items": 0, 2024-04-22T13:35:13.117 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_bytes": 0, 2024-04-22T13:35:13.117 INFO:teuthology.orchestra.run.smithi134.stdout: "pgmap_items": 0, 2024-04-22T13:35:13.117 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_bytes": 8512, 2024-04-22T13:35:13.117 INFO:teuthology.orchestra.run.smithi134.stdout: "mds_co_items": 13, 2024-04-22T13:35:13.117 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_bytes": 0, 2024-04-22T13:35:13.117 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_1_items": 0, 2024-04-22T13:35:13.117 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_bytes": 0, 2024-04-22T13:35:13.117 INFO:teuthology.orchestra.run.smithi134.stdout: "unittest_2_items": 0 2024-04-22T13:35:13.117 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.117 INFO:teuthology.orchestra.run.smithi134.stdout: "objecter": { 2024-04-22T13:35:13.118 INFO:teuthology.orchestra.run.smithi134.stdout: "op_active": 0, 2024-04-22T13:35:13.118 INFO:teuthology.orchestra.run.smithi134.stdout: "op_laggy": 0, 2024-04-22T13:35:13.118 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send": 2270, 2024-04-22T13:35:13.118 INFO:teuthology.orchestra.run.smithi134.stdout: "op_send_bytes": 21, 2024-04-22T13:35:13.118 INFO:teuthology.orchestra.run.smithi134.stdout: "op_resend": 3, 2024-04-22T13:35:13.118 INFO:teuthology.orchestra.run.smithi134.stdout: "op_reply": 2270, 2024-04-22T13:35:13.118 INFO:teuthology.orchestra.run.smithi134.stdout: "op_latency": { 2024-04-22T13:35:13.118 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2270, 2024-04-22T13:35:13.118 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 1.513961721, 2024-04-22T13:35:13.118 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000666943 2024-04-22T13:35:13.118 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.118 INFO:teuthology.orchestra.run.smithi134.stdout: "op_inflight": 0, 2024-04-22T13:35:13.119 INFO:teuthology.orchestra.run.smithi134.stdout: "oplen_avg": { 2024-04-22T13:35:13.119 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 2270, 2024-04-22T13:35:13.119 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 2271 2024-04-22T13:35:13.119 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.119 INFO:teuthology.orchestra.run.smithi134.stdout: "op": 2270, 2024-04-22T13:35:13.119 INFO:teuthology.orchestra.run.smithi134.stdout: "op_r": 2270, 2024-04-22T13:35:13.119 INFO:teuthology.orchestra.run.smithi134.stdout: "op_w": 0, 2024-04-22T13:35:13.119 INFO:teuthology.orchestra.run.smithi134.stdout: "op_rmw": 0, 2024-04-22T13:35:13.119 INFO:teuthology.orchestra.run.smithi134.stdout: "op_pg": 0, 2024-04-22T13:35:13.119 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_stat": 1478, 2024-04-22T13:35:13.119 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_create": 0, 2024-04-22T13:35:13.119 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_read": 790, 2024-04-22T13:35:13.119 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_write": 0, 2024-04-22T13:35:13.119 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writefull": 0, 2024-04-22T13:35:13.119 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_writesame": 0, 2024-04-22T13:35:13.120 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_append": 0, 2024-04-22T13:35:13.120 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_zero": 0, 2024-04-22T13:35:13.120 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_truncate": 0, 2024-04-22T13:35:13.120 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_delete": 0, 2024-04-22T13:35:13.120 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_mapext": 0, 2024-04-22T13:35:13.120 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_sparse_read": 0, 2024-04-22T13:35:13.120 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_clonerange": 0, 2024-04-22T13:35:13.120 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_getxattr": 1, 2024-04-22T13:35:13.120 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_setxattr": 0, 2024-04-22T13:35:13.120 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_cmpxattr": 0, 2024-04-22T13:35:13.120 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_rmxattr": 0, 2024-04-22T13:35:13.120 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_resetxattrs": 0, 2024-04-22T13:35:13.120 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_call": 0, 2024-04-22T13:35:13.120 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_watch": 0, 2024-04-22T13:35:13.120 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_notify": 0, 2024-04-22T13:35:13.120 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_src_cmpxattr": 0, 2024-04-22T13:35:13.121 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls": 0, 2024-04-22T13:35:13.121 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_pgls_filter": 0, 2024-04-22T13:35:13.121 INFO:teuthology.orchestra.run.smithi134.stdout: "osdop_other": 0, 2024-04-22T13:35:13.121 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_active": 0, 2024-04-22T13:35:13.121 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_send": 0, 2024-04-22T13:35:13.121 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_resend": 0, 2024-04-22T13:35:13.121 INFO:teuthology.orchestra.run.smithi134.stdout: "linger_ping": 0, 2024-04-22T13:35:13.121 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_active": 0, 2024-04-22T13:35:13.121 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_send": 0, 2024-04-22T13:35:13.121 INFO:teuthology.orchestra.run.smithi134.stdout: "poolop_resend": 0, 2024-04-22T13:35:13.121 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_active": 0, 2024-04-22T13:35:13.121 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_send": 0, 2024-04-22T13:35:13.121 INFO:teuthology.orchestra.run.smithi134.stdout: "poolstat_resend": 0, 2024-04-22T13:35:13.121 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_active": 0, 2024-04-22T13:35:13.121 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_send": 0, 2024-04-22T13:35:13.121 INFO:teuthology.orchestra.run.smithi134.stdout: "statfs_resend": 0, 2024-04-22T13:35:13.122 INFO:teuthology.orchestra.run.smithi134.stdout: "command_active": 0, 2024-04-22T13:35:13.122 INFO:teuthology.orchestra.run.smithi134.stdout: "command_send": 0, 2024-04-22T13:35:13.122 INFO:teuthology.orchestra.run.smithi134.stdout: "command_resend": 0, 2024-04-22T13:35:13.122 INFO:teuthology.orchestra.run.smithi134.stdout: "map_epoch": 99, 2024-04-22T13:35:13.122 INFO:teuthology.orchestra.run.smithi134.stdout: "map_full": 0, 2024-04-22T13:35:13.122 INFO:teuthology.orchestra.run.smithi134.stdout: "map_inc": 12, 2024-04-22T13:35:13.122 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_sessions": 5, 2024-04-22T13:35:13.122 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_open": 5, 2024-04-22T13:35:13.122 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_session_close": 0, 2024-04-22T13:35:13.122 INFO:teuthology.orchestra.run.smithi134.stdout: "osd_laggy": 0, 2024-04-22T13:35:13.122 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_wr": 0, 2024-04-22T13:35:13.122 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_rd": 2, 2024-04-22T13:35:13.122 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_del": 0 2024-04-22T13:35:13.122 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.122 INFO:teuthology.orchestra.run.smithi134.stdout: "oft": { 2024-04-22T13:35:13.122 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_objs": 0, 2024-04-22T13:35:13.123 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_kv_pairs": 0, 2024-04-22T13:35:13.123 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_updates": 0, 2024-04-22T13:35:13.123 INFO:teuthology.orchestra.run.smithi134.stdout: "omap_total_removes": 0 2024-04-22T13:35:13.123 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.123 INFO:teuthology.orchestra.run.smithi134.stdout: "osd-slow-ops": { 2024-04-22T13:35:13.123 INFO:teuthology.orchestra.run.smithi134.stdout: "slow_ops_count": 0 2024-04-22T13:35:13.123 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.123 INFO:teuthology.orchestra.run.smithi134.stdout: "purge_queue": { 2024-04-22T13:35:13.123 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops": 0, 2024-04-22T13:35:13.123 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_ops_high_water": 0, 2024-04-22T13:35:13.123 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing": 0, 2024-04-22T13:35:13.123 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executing_high_water": 0, 2024-04-22T13:35:13.123 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed_ops": 0, 2024-04-22T13:35:13.123 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_executed": 0, 2024-04-22T13:35:13.123 INFO:teuthology.orchestra.run.smithi134.stdout: "pq_item_in_journal": 0 2024-04-22T13:35:13.123 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.124 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-22T13:35:13.124 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-22T13:35:13.124 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-04-22T13:35:13.124 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-22T13:35:13.124 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 2522, 2024-04-22T13:35:13.124 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 2027869, 2024-04-22T13:35:13.124 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-22T13:35:13.124 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 2522, 2024-04-22T13:35:13.124 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-22T13:35:13.124 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-22T13:35:13.124 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2522, 2024-04-22T13:35:13.124 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 2027869, 2024-04-22T13:35:13.124 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-22T13:35:13.124 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.124 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.124 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.125 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:13.125 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.125 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_bytes": { 2024-04-22T13:35:13.125 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-22T13:35:13.125 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 104857600, 2024-04-22T13:35:13.125 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-22T13:35:13.125 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-22T13:35:13.125 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-22T13:35:13.125 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-22T13:35:13.125 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-22T13:35:13.125 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 2270, 2024-04-22T13:35:13.125 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 2888, 2024-04-22T13:35:13.125 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2270, 2024-04-22T13:35:13.125 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 2888, 2024-04-22T13:35:13.125 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-22T13:35:13.126 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.126 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.126 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.126 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:13.126 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.126 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-objecter_ops": { 2024-04-22T13:35:13.126 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-22T13:35:13.126 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 1024, 2024-04-22T13:35:13.126 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-22T13:35:13.126 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-22T13:35:13.126 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-22T13:35:13.126 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-22T13:35:13.126 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-22T13:35:13.126 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 2270, 2024-04-22T13:35:13.127 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 2270, 2024-04-22T13:35:13.127 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 2270, 2024-04-22T13:35:13.127 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 2270, 2024-04-22T13:35:13.127 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-22T13:35:13.127 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.127 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.127 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.127 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:13.127 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.127 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle": { 2024-04-22T13:35:13.127 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-22T13:35:13.127 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-04-22T13:35:13.127 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-22T13:35:13.127 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-22T13:35:13.127 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-22T13:35:13.128 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-22T13:35:13.128 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-22T13:35:13.128 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-22T13:35:13.128 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-22T13:35:13.128 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-04-22T13:35:13.128 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-04-22T13:35:13.128 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-22T13:35:13.128 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.128 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.128 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.128 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:13.128 INFO:teuthology.orchestra.run.smithi134.stdout: }, 2024-04-22T13:35:13.128 INFO:teuthology.orchestra.run.smithi134.stdout: "throttle-write_buf_throttle-0x55b7dab3e6e0": { 2024-04-22T13:35:13.128 INFO:teuthology.orchestra.run.smithi134.stdout: "val": 0, 2024-04-22T13:35:13.128 INFO:teuthology.orchestra.run.smithi134.stdout: "max": 3758096384, 2024-04-22T13:35:13.128 INFO:teuthology.orchestra.run.smithi134.stdout: "get_started": 0, 2024-04-22T13:35:13.128 INFO:teuthology.orchestra.run.smithi134.stdout: "get": 0, 2024-04-22T13:35:13.129 INFO:teuthology.orchestra.run.smithi134.stdout: "get_sum": 0, 2024-04-22T13:35:13.129 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_fail": 0, 2024-04-22T13:35:13.129 INFO:teuthology.orchestra.run.smithi134.stdout: "get_or_fail_success": 0, 2024-04-22T13:35:13.129 INFO:teuthology.orchestra.run.smithi134.stdout: "take": 0, 2024-04-22T13:35:13.129 INFO:teuthology.orchestra.run.smithi134.stdout: "take_sum": 0, 2024-04-22T13:35:13.129 INFO:teuthology.orchestra.run.smithi134.stdout: "put": 0, 2024-04-22T13:35:13.129 INFO:teuthology.orchestra.run.smithi134.stdout: "put_sum": 0, 2024-04-22T13:35:13.129 INFO:teuthology.orchestra.run.smithi134.stdout: "wait": { 2024-04-22T13:35:13.129 INFO:teuthology.orchestra.run.smithi134.stdout: "avgcount": 0, 2024-04-22T13:35:13.129 INFO:teuthology.orchestra.run.smithi134.stdout: "sum": 0.000000000, 2024-04-22T13:35:13.129 INFO:teuthology.orchestra.run.smithi134.stdout: "avgtime": 0.000000000 2024-04-22T13:35:13.129 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:13.129 INFO:teuthology.orchestra.run.smithi134.stdout: } 2024-04-22T13:35:13.129 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-04-22T13:35:13.745 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:13 smithi134 ceph-mon[31338]: pgmap v345: 97 pgs: 97 active+clean; 2.3 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:35:13.773 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-04-22T13:35:13.773 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-04-22T13:35:13.774 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-04-22T13:35:13.774 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-04-22T13:35:13.774 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-04-22T13:35:13.774 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-22T13:35:13.785 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-22T13:35:13.786 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-22T13:35:13.786 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-22T13:35:13.808 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-22T13:35:13.808 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-22T13:35:13.868 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:35:13.934 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:13 smithi012 ceph-mon[28265]: pgmap v345: 97 pgs: 97 active+clean; 2.3 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:35:13.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:13 smithi111 ceph-mon[31361]: pgmap v345: 97 pgs: 97 active+clean; 2.3 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:35:14.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.029+0000 7f0ba0c9a640 1 -- 172.21.15.12:0/3853763524 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b9c106180 msgr2=0x7f0b9c10dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:14.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.029+0000 7f0ba0c9a640 1 --2- 172.21.15.12:0/3853763524 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b9c106180 0x7f0b9c10dd50 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f0b94009ca0 tx=0x7f0b9402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:35:14.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.030+0000 7f0ba0c9a640 1 -- 172.21.15.12:0/3853763524 shutdown_connections 2024-04-22T13:35:14.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.030+0000 7f0ba0c9a640 1 --2- 172.21.15.12:0/3853763524 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b9c106180 0x7f0b9c10dd50 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:14.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.030+0000 7f0ba0c9a640 1 --2- 172.21.15.12:0/3853763524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b9c101a60 0x7f0b9c105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:14.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.030+0000 7f0ba0c9a640 1 --2- 172.21.15.12:0/3853763524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b9c1010b0 0x7f0b9c101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:14.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.030+0000 7f0ba0c9a640 1 -- 172.21.15.12:0/3853763524 >> 172.21.15.12:0/3853763524 conn(0x7f0b9c0771d0 msgr2=0x7f0b9c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:14.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.030+0000 7f0ba0c9a640 1 -- 172.21.15.12:0/3853763524 shutdown_connections 2024-04-22T13:35:14.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.030+0000 7f0ba0c9a640 1 -- 172.21.15.12:0/3853763524 wait complete. 2024-04-22T13:35:14.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.030+0000 7f0ba0c9a640 1 Processor -- start 2024-04-22T13:35:14.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.030+0000 7f0ba0c9a640 1 -- start start 2024-04-22T13:35:14.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.031+0000 7f0ba0c9a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b9c1010b0 0x7f0b9c197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:14.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.031+0000 7f0ba0c9a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b9c106180 0x7f0b9c197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:14.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.031+0000 7f0ba0c9a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b9c19be50 0x7f0b9c19c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:14.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.031+0000 7f0ba0c9a640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c1110d0 con 0x7f0b9c1010b0 2024-04-22T13:35:14.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.031+0000 7f0ba0c9a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c110f50 con 0x7f0b9c19be50 2024-04-22T13:35:14.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.031+0000 7f0ba0c9a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c111250 con 0x7f0b9c106180 2024-04-22T13:35:14.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.031+0000 7f0b9a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b9c1010b0 0x7f0b9c197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:14.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.031+0000 7f0b9a59c640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b9c1010b0 0x7f0b9c197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55598/0 (socket says 172.21.15.12:55598) 2024-04-22T13:35:14.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.031+0000 7f0b9a59c640 1 -- 172.21.15.12:0/2562088388 learned_addr learned my addr 172.21.15.12:0/2562088388 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:35:14.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.031+0000 7f0b99d9b640 1 --2- 172.21.15.12:0/2562088388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b9c106180 0x7f0b9c197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:14.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.031+0000 7f0b9ad9d640 1 --2- 172.21.15.12:0/2562088388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b9c19be50 0x7f0b9c19c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:14.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.031+0000 7f0b9a59c640 1 -- 172.21.15.12:0/2562088388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b9c106180 msgr2=0x7f0b9c197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:14.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.031+0000 7f0b9a59c640 1 --2- 172.21.15.12:0/2562088388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b9c106180 0x7f0b9c197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:14.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.031+0000 7f0b9a59c640 1 -- 172.21.15.12:0/2562088388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b9c19be50 msgr2=0x7f0b9c19c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:14.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.031+0000 7f0b9a59c640 1 --2- 172.21.15.12:0/2562088388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b9c19be50 0x7f0b9c19c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:14.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.031+0000 7f0b9a59c640 1 -- 172.21.15.12:0/2562088388 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b9c19c940 con 0x7f0b9c1010b0 2024-04-22T13:35:14.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.031+0000 7f0b9ad9d640 1 --2- 172.21.15.12:0/2562088388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b9c19be50 0x7f0b9c19c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:35:14.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.031+0000 7f0b9a59c640 1 --2- 172.21.15.12:0/2562088388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b9c1010b0 0x7f0b9c197590 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c00eb00 tx=0x7f0b8c009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:14.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.032+0000 7f0b8b7fe640 1 -- 172.21.15.12:0/2562088388 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c01c070 con 0x7f0b9c1010b0 2024-04-22T13:35:14.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.032+0000 7f0b8b7fe640 1 -- 172.21.15.12:0/2562088388 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b8c00bc30 con 0x7f0b9c1010b0 2024-04-22T13:35:14.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.032+0000 7f0ba0c9a640 1 -- 172.21.15.12:0/2562088388 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b9c19cc30 con 0x7f0b9c1010b0 2024-04-22T13:35:14.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.032+0000 7f0b8b7fe640 1 -- 172.21.15.12:0/2562088388 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c016610 con 0x7f0b9c1010b0 2024-04-22T13:35:14.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.032+0000 7f0ba0c9a640 1 -- 172.21.15.12:0/2562088388 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b9c06a960 con 0x7f0b9c1010b0 2024-04-22T13:35:14.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.033+0000 7f0ba0c9a640 1 -- 172.21.15.12:0/2562088388 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b9c069230 con 0x7f0b9c1010b0 2024-04-22T13:35:14.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.034+0000 7f0b8b7fe640 1 -- 172.21.15.12:0/2562088388 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0b8c01f070 con 0x7f0b9c1010b0 2024-04-22T13:35:14.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.035+0000 7f0b8b7fe640 1 --2- 172.21.15.12:0/2562088388 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f0b840a3d20 0x7f0b840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:14.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.035+0000 7f0b99d9b640 1 --2- 172.21.15.12:0/2562088388 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f0b840a3d20 0x7f0b840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:14.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.035+0000 7f0b8b7fe640 1 -- 172.21.15.12:0/2562088388 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8528+0+0 (secure 0 0 0) 0x7f0b8c0d2060 con 0x7f0b9c1010b0 2024-04-22T13:35:14.031 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.035+0000 7f0b99d9b640 1 --2- 172.21.15.12:0/2562088388 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f0b840a3d20 0x7f0b840a61e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c100670 tx=0x7f0b9000b390 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:14.034 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.039+0000 7f0b8b7fe640 1 -- 172.21.15.12:0/2562088388 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0b8c00bde0 con 0x7f0b9c1010b0 2024-04-22T13:35:14.223 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.228+0000 7f0ba0c9a640 1 -- 172.21.15.12:0/2562088388 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b9c1988e0 con 0x7f0b9c1010b0 2024-04-22T13:35:14.224 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.229+0000 7f0b8b7fe640 1 -- 172.21.15.12:0/2562088388 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+23154 (secure 0 0 0) 0x7f0b8c09e070 con 0x7f0b9c1010b0 2024-04-22T13:35:14.225 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:35:14.225 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":99,"fsid":"0aede5ea-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:14:09.362918+0000","modified":"2024-04-22T13:35:10.629460+0000","last_up_change":"2024-04-22T13:21:15.263393+0000","last_in_change":"2024-04-22T13:20:54.174685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:17:32.372434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:22:02.768799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:22:03.578730+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d9e2365-3a64-4b13-96fc-c217b1cee2ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6803","nonce":1191350632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6805","nonce":1191350632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6809","nonce":1191350632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6807","nonce":1191350632}]},"public_addr":"172.21.15.12:6803/1191350632","cluster_addr":"172.21.15.12:6805/1191350632","heartbeat_back_addr":"172.21.15.12:6809/1191350632","heartbeat_front_addr":"172.21.15.12:6807/1191350632","state":["exists","up"]},{"osd":1,"uuid":"1499c783-2bd7-48ce-a444-34add42bcf31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6801","nonce":417496465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6803","nonce":417496465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6807","nonce":417496465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6805","nonce":417496465}]},"public_addr":"172.21.15.111:6801/417496465","cluster_addr":"172.21.15.111:6803/417496465","heartbeat_back_addr":"172.21.15.111:6807/417496465","heartbeat_front_addr":"172.21.15.111:6805/417496465","state":["exists","up"]},{"osd":2,"uuid":"71a9ebc3-c622-434e-a6c2-f48358f384f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6801","nonce":1403153998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6803","nonce":1403153998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6807","nonce":1403153998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6805","nonce":1403153998}]},"public_addr":"172.21.15.134:6801/1403153998","cluster_addr":"172.21.15.134:6803/1403153998","heartbeat_back_addr":"172.21.15.134:6807/1403153998","heartbeat_front_addr":"172.21.15.134:6805/1403153998","state":["exists","up"]},{"osd":3,"uuid":"6dd00db2-a286-48ba-8666-56ffdd5f35ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6811","nonce":3359066681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6813","nonce":3359066681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6817","nonce":3359066681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6815","nonce":3359066681}]},"public_addr":"172.21.15.12:6811/3359066681","cluster_addr":"172.21.15.12:6813/3359066681","heartbeat_back_addr":"172.21.15.12:6817/3359066681","heartbeat_front_addr":"172.21.15.12:6815/3359066681","state":["exists","up"]},{"osd":4,"uuid":"1e29afd7-354c-4506-aaa0-b43eb1a19a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6809","nonce":2886349872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6811","nonce":2886349872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6815","nonce":2886349872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6813","nonce":2886349872}]},"public_addr":"172.21.15.111:6809/2886349872","cluster_addr":"172.21.15.111:6811/2886349872","heartbeat_back_addr":"172.21.15.111:6815/2886349872","heartbeat_front_addr":"172.21.15.111:6813/2886349872","state":["exists","up"]},{"osd":5,"uuid":"8f40f02d-0ba2-4842-a03d-2a1a04d1010d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6809","nonce":3071977602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6811","nonce":3071977602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6815","nonce":3071977602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6813","nonce":3071977602}]},"public_addr":"172.21.15.134:6809/3071977602","cluster_addr":"172.21.15.134:6811/3071977602","heartbeat_back_addr":"172.21.15.134:6815/3071977602","heartbeat_front_addr":"172.21.15.134:6813/3071977602","state":["exists","up"]},{"osd":6,"uuid":"bb0c17a9-6a8e-4be6-8660-f73cfa8200ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6819","nonce":3819619299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6821","nonce":3819619299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6825","nonce":3819619299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6823","nonce":3819619299}]},"public_addr":"172.21.15.12:6819/3819619299","cluster_addr":"172.21.15.12:6821/3819619299","heartbeat_back_addr":"172.21.15.12:6825/3819619299","heartbeat_front_addr":"172.21.15.12:6823/3819619299","state":["exists","up"]},{"osd":7,"uuid":"0a93d6af-569e-4e45-b697-9e978c1c1004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6817","nonce":3113856282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6819","nonce":3113856282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6823","nonce":3113856282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6821","nonce":3113856282}]},"public_addr":"172.21.15.111:6817/3113856282","cluster_addr":"172.21.15.111:6819/3113856282","heartbeat_back_addr":"172.21.15.111:6823/3113856282","heartbeat_front_addr":"172.21.15.111:6821/3113856282","state":["exists","up"]},{"osd":8,"uuid":"e21acc68-cf50-4f4f-b1ba-cf6c195b4955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6817","nonce":1689362191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6819","nonce":1689362191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6823","nonce":1689362191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6821","nonce":1689362191}]},"public_addr":"172.21.15.134:6817/1689362191","cluster_addr":"172.21.15.134:6819/1689362191","heartbeat_back_addr":"172.21.15.134:6823/1689362191","heartbeat_front_addr":"172.21.15.134:6821/1689362191","state":["exists","up"]},{"osd":9,"uuid":"fd745869-7f5f-48a8-b412-bb9c92b586e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6827","nonce":4292385779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6829","nonce":4292385779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6833","nonce":4292385779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6831","nonce":4292385779}]},"public_addr":"172.21.15.12:6827/4292385779","cluster_addr":"172.21.15.12:6829/4292385779","heartbeat_back_addr":"172.21.15.12:6833/4292385779","heartbeat_front_addr":"172.21.15.12:6831/4292385779","state":["exists","up"]},{"osd":10,"uuid":"798b1abd-f6c7-4a5f-8da0-fdae49e50d2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6825","nonce":3338566976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6827","nonce":3338566976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6831","nonce":3338566976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6829","nonce":3338566976}]},"public_addr":"172.21.15.111:6825/3338566976","cluster_addr":"172.21.15.111:6827/3338566976","heartbeat_back_addr":"172.21.15.111:6831/3338566976","heartbeat_front_addr":"172.21.15.111:6829/3338566976","state":["exists","up"]},{"osd":11,"uuid":"b18fc5f3-1049-4871-b060-e99290273cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6825","nonce":578640428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6827","nonce":578640428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6831","nonce":578640428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6829","nonce":578640428}]},"public_addr":"172.21.15.134:6825/578640428","cluster_addr":"172.21.15.134:6827/578640428","heartbeat_back_addr":"172.21.15.134:6831/578640428","heartbeat_front_addr":"172.21.15.134:6829/578640428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:16:40.828608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:05.052528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:28.297316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:53.496497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:17.947259+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:41.712894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:07.105945+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:32.129688+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:56.008516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:22.239867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:47.334692+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:12.301276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/3535299028":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/2837479598":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/3010129696":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/2593601362":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/2519382069":"2024-04-23T13:15:04.279990+0000","172.21.15.12:6801/2009831940":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/856202072":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/3509412187":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/1802873236":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/1007749062":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3588480095":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1904962176":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/1802873236":"2024-04-23T13:24:06.155564+0000","172.21.15.12:6800/2009831940":"2024-04-23T13:15:04.279990+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:35:14.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.231+0000 7f0ba0c9a640 1 -- 172.21.15.12:0/2562088388 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f0b840a3d20 msgr2=0x7f0b840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:14.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.231+0000 7f0ba0c9a640 1 --2- 172.21.15.12:0/2562088388 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f0b840a3d20 0x7f0b840a61e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c100670 tx=0x7f0b9000b390 comp rx=0 tx=0).stop 2024-04-22T13:35:14.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.231+0000 7f0ba0c9a640 1 -- 172.21.15.12:0/2562088388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b9c1010b0 msgr2=0x7f0b9c197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:14.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.231+0000 7f0ba0c9a640 1 --2- 172.21.15.12:0/2562088388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b9c1010b0 0x7f0b9c197590 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c00eb00 tx=0x7f0b8c009f20 comp rx=0 tx=0).stop 2024-04-22T13:35:14.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.231+0000 7f0ba0c9a640 1 -- 172.21.15.12:0/2562088388 shutdown_connections 2024-04-22T13:35:14.227 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.231+0000 7f0ba0c9a640 1 --2- 172.21.15.12:0/2562088388 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f0b840a3d20 0x7f0b840a61e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:14.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.231+0000 7f0ba0c9a640 1 --2- 172.21.15.12:0/2562088388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0b9c19be50 0x7f0b9c19c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:14.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.231+0000 7f0ba0c9a640 1 --2- 172.21.15.12:0/2562088388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b9c106180 0x7f0b9c197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:14.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.231+0000 7f0ba0c9a640 1 --2- 172.21.15.12:0/2562088388 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0b9c1010b0 0x7f0b9c197590 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:14.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.231+0000 7f0ba0c9a640 1 -- 172.21.15.12:0/2562088388 >> 172.21.15.12:0/2562088388 conn(0x7f0b9c0771d0 msgr2=0x7f0b9c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:14.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.231+0000 7f0ba0c9a640 1 -- 172.21.15.12:0/2562088388 shutdown_connections 2024-04-22T13:35:14.228 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.231+0000 7f0ba0c9a640 1 -- 172.21.15.12:0/2562088388 wait complete. 2024-04-22T13:35:14.238 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-22T13:17:32.372434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-22T13:22:02.768799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-22T13:22:03.578730+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-22T13:35:14.239 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-22T13:35:14.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.389+0000 7fa8783fd640 1 -- 172.21.15.12:0/450628323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa870102b40 msgr2=0x7fa870106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:14.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.389+0000 7fa8783fd640 1 --2- 172.21.15.12:0/450628323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa870102b40 0x7fa870106b80 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fa860009900 tx=0x7fa86002bc30 comp rx=0 tx=0).stop 2024-04-22T13:35:14.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.389+0000 7fa8783fd640 1 -- 172.21.15.12:0/450628323 shutdown_connections 2024-04-22T13:35:14.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.389+0000 7fa8783fd640 1 --2- 172.21.15.12:0/450628323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa870107260 0x7fa87010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:14.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.389+0000 7fa8783fd640 1 --2- 172.21.15.12:0/450628323 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa870102b40 0x7fa870106b80 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:14.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.389+0000 7fa8783fd640 1 --2- 172.21.15.12:0/450628323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa870102190 0x7fa870102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:14.386 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.389+0000 7fa8783fd640 1 -- 172.21.15.12:0/450628323 >> 172.21.15.12:0/450628323 conn(0x7fa8700fd990 msgr2=0x7fa8700ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:14.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.389+0000 7fa8783fd640 1 -- 172.21.15.12:0/450628323 shutdown_connections 2024-04-22T13:35:14.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.389+0000 7fa8783fd640 1 -- 172.21.15.12:0/450628323 wait complete. 2024-04-22T13:35:14.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.389+0000 7fa8783fd640 1 Processor -- start 2024-04-22T13:35:14.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.389+0000 7fa8783fd640 1 -- start start 2024-04-22T13:35:14.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.389+0000 7fa8783fd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa870102190 0x7fa870198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:14.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.389+0000 7fa8783fd640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa870107260 0x7fa870198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:14.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.389+0000 7fa8783fd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa87019d000 0x7fa87019d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:14.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.390+0000 7fa8783fd640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa870112180 con 0x7fa870107260 2024-04-22T13:35:14.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.390+0000 7fa8783fd640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa870112000 con 0x7fa87019d000 2024-04-22T13:35:14.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.390+0000 7fa8783fd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa870112300 con 0x7fa870102190 2024-04-22T13:35:14.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.390+0000 7fa875998640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa870107260 0x7fa870198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:14.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.390+0000 7fa875998640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa870107260 0x7fa870198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55604/0 (socket says 172.21.15.12:55604) 2024-04-22T13:35:14.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.390+0000 7fa875998640 1 -- 172.21.15.12:0/886042931 learned_addr learned my addr 172.21.15.12:0/886042931 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:35:14.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.390+0000 7fa87699a640 1 --2- 172.21.15.12:0/886042931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa87019d000 0x7fa87019d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:14.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.390+0000 7fa876199640 1 --2- 172.21.15.12:0/886042931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa870102190 0x7fa870198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:14.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.390+0000 7fa875998640 1 -- 172.21.15.12:0/886042931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa870102190 msgr2=0x7fa870198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:14.387 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.390+0000 7fa875998640 1 --2- 172.21.15.12:0/886042931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa870102190 0x7fa870198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:14.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.390+0000 7fa875998640 1 -- 172.21.15.12:0/886042931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa87019d000 msgr2=0x7fa87019d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:14.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.390+0000 7fa875998640 1 --2- 172.21.15.12:0/886042931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa87019d000 0x7fa87019d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:14.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.390+0000 7fa875998640 1 -- 172.21.15.12:0/886042931 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87019daf0 con 0x7fa870107260 2024-04-22T13:35:14.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.390+0000 7fa876199640 1 --2- 172.21.15.12:0/886042931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa870102190 0x7fa870198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:35:14.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.390+0000 7fa875998640 1 --2- 172.21.15.12:0/886042931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa870107260 0x7fa870198c80 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fa860036040 tx=0x7fa86002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:14.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.390+0000 7fa8677fe640 1 -- 172.21.15.12:0/886042931 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa860043020 con 0x7fa870107260 2024-04-22T13:35:14.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.390+0000 7fa8677fe640 1 -- 172.21.15.12:0/886042931 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa86002cbe0 con 0x7fa870107260 2024-04-22T13:35:14.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.390+0000 7fa8677fe640 1 -- 172.21.15.12:0/886042931 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa860003bf0 con 0x7fa870107260 2024-04-22T13:35:14.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.390+0000 7fa8783fd640 1 -- 172.21.15.12:0/886042931 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa87019dd80 con 0x7fa870107260 2024-04-22T13:35:14.388 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.390+0000 7fa8783fd640 1 -- 172.21.15.12:0/886042931 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8701a55d0 con 0x7fa870107260 2024-04-22T13:35:14.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.391+0000 7fa8783fd640 1 -- 172.21.15.12:0/886042931 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8700691c0 con 0x7fa870107260 2024-04-22T13:35:14.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.392+0000 7fa8677fe640 1 -- 172.21.15.12:0/886042931 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa860046020 con 0x7fa870107260 2024-04-22T13:35:14.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.393+0000 7fa8677fe640 1 --2- 172.21.15.12:0/886042931 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa85c0a3da0 0x7fa85c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:14.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.393+0000 7fa876199640 1 --2- 172.21.15.12:0/886042931 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa85c0a3da0 0x7fa85c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:14.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.393+0000 7fa8677fe640 1 -- 172.21.15.12:0/886042931 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8528+0+0 (secure 0 0 0) 0x7fa8600f2910 con 0x7fa870107260 2024-04-22T13:35:14.389 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.394+0000 7fa876199640 1 --2- 172.21.15.12:0/886042931 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa85c0a3da0 0x7fa85c0a6260 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa86c00abe0 tx=0x7fa86c00a5c0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:14.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.398+0000 7fa8677fe640 1 -- 172.21.15.12:0/886042931 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa86004d020 con 0x7fa870107260 2024-04-22T13:35:14.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.585+0000 7fa8783fd640 1 -- 172.21.15.12:0/886042931 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa870102b40 con 0x7fa870107260 2024-04-22T13:35:14.581 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.586+0000 7fa8677fe640 1 -- 172.21.15.12:0/886042931 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7fa8600bf0c0 con 0x7fa870107260 2024-04-22T13:35:14.581 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 1 2024-04-22T13:35:14.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.588+0000 7fa8783fd640 1 -- 172.21.15.12:0/886042931 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa85c0a3da0 msgr2=0x7fa85c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:14.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.588+0000 7fa8783fd640 1 --2- 172.21.15.12:0/886042931 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa85c0a3da0 0x7fa85c0a6260 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa86c00abe0 tx=0x7fa86c00a5c0 comp rx=0 tx=0).stop 2024-04-22T13:35:14.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.588+0000 7fa8783fd640 1 -- 172.21.15.12:0/886042931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa870107260 msgr2=0x7fa870198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:14.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.588+0000 7fa8783fd640 1 --2- 172.21.15.12:0/886042931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa870107260 0x7fa870198c80 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fa860036040 tx=0x7fa86002c7e0 comp rx=0 tx=0).stop 2024-04-22T13:35:14.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.588+0000 7fa8783fd640 1 -- 172.21.15.12:0/886042931 shutdown_connections 2024-04-22T13:35:14.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.588+0000 7fa8783fd640 1 --2- 172.21.15.12:0/886042931 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa85c0a3da0 0x7fa85c0a6260 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:14.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.588+0000 7fa8783fd640 1 --2- 172.21.15.12:0/886042931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa87019d000 0x7fa87019d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:14.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.588+0000 7fa8783fd640 1 --2- 172.21.15.12:0/886042931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa870107260 0x7fa870198c80 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:14.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.588+0000 7fa8783fd640 1 --2- 172.21.15.12:0/886042931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa870102190 0x7fa870198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:14.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.588+0000 7fa8783fd640 1 -- 172.21.15.12:0/886042931 >> 172.21.15.12:0/886042931 conn(0x7fa8700fd990 msgr2=0x7fa8700fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:14.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.588+0000 7fa8783fd640 1 -- 172.21.15.12:0/886042931 shutdown_connections 2024-04-22T13:35:14.584 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.588+0000 7fa8783fd640 1 -- 172.21.15.12:0/886042931 wait complete. 2024-04-22T13:35:14.594 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-22T13:35:14.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.706+0000 7fbd851b1640 1 -- 172.21.15.12:0/909480439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd80076da0 msgr2=0x7fbd801128f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:14.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.706+0000 7fbd851b1640 1 --2- 172.21.15.12:0/909480439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd80076da0 0x7fbd801128f0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009900 tx=0x7fbd6802bc30 comp rx=0 tx=0).stop 2024-04-22T13:35:14.703 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.706+0000 7fbd851b1640 1 -- 172.21.15.12:0/909480439 shutdown_connections 2024-04-22T13:35:14.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.706+0000 7fbd851b1640 1 --2- 172.21.15.12:0/909480439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd80112e30 0x7fbd80115220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:14.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.706+0000 7fbd851b1640 1 --2- 172.21.15.12:0/909480439 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd80076da0 0x7fbd801128f0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:14.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.706+0000 7fbd851b1640 1 --2- 172.21.15.12:0/909480439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd80076480 0x7fbd80076860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:14.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.706+0000 7fbd851b1640 1 -- 172.21.15.12:0/909480439 >> 172.21.15.12:0/909480439 conn(0x7fbd800ffce0 msgr2=0x7fbd80102100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:14.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.706+0000 7fbd851b1640 1 -- 172.21.15.12:0/909480439 shutdown_connections 2024-04-22T13:35:14.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.706+0000 7fbd851b1640 1 -- 172.21.15.12:0/909480439 wait complete. 2024-04-22T13:35:14.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.706+0000 7fbd851b1640 1 Processor -- start 2024-04-22T13:35:14.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.706+0000 7fbd851b1640 1 -- start start 2024-04-22T13:35:14.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.707+0000 7fbd851b1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd80076480 0x7fbd8019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:14.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.707+0000 7fbd851b1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd80076da0 0x7fbd8019f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:14.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.707+0000 7fbd851b1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd80112e30 0x7fbd801a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:14.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.707+0000 7fbd851b1640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fbd80118840 con 0x7fbd80076da0 2024-04-22T13:35:14.704 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.707+0000 7fbd851b1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbd801186c0 con 0x7fbd80112e30 2024-04-22T13:35:14.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.707+0000 7fbd851b1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbd801189c0 con 0x7fbd80076480 2024-04-22T13:35:14.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.707+0000 7fbd77fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd80076da0 0x7fbd8019f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:14.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.707+0000 7fbd77fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd80076da0 0x7fbd8019f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55624/0 (socket says 172.21.15.12:55624) 2024-04-22T13:35:14.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.707+0000 7fbd77fff640 1 -- 172.21.15.12:0/4082156852 learned_addr learned my addr 172.21.15.12:0/4082156852 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:35:14.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.707+0000 7fbd7f59e640 1 --2- 172.21.15.12:0/4082156852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd80112e30 0x7fbd801a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:14.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.707+0000 7fbd7ed9d640 1 --2- 172.21.15.12:0/4082156852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd80076480 0x7fbd8019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:14.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.707+0000 7fbd77fff640 1 -- 172.21.15.12:0/4082156852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd80076480 msgr2=0x7fbd8019ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:14.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.707+0000 7fbd77fff640 1 --2- 172.21.15.12:0/4082156852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd80076480 0x7fbd8019ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:14.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.707+0000 7fbd77fff640 1 -- 172.21.15.12:0/4082156852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd80112e30 msgr2=0x7fbd801a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:14.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.707+0000 7fbd77fff640 1 --2- 172.21.15.12:0/4082156852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd80112e30 0x7fbd801a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:14.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.707+0000 7fbd77fff640 1 -- 172.21.15.12:0/4082156852 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd801a3e30 con 0x7fbd80076da0 2024-04-22T13:35:14.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.707+0000 7fbd7ed9d640 1 --2- 172.21.15.12:0/4082156852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd80076480 0x7fbd8019ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:35:14.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.707+0000 7fbd77fff640 1 --2- 172.21.15.12:0/4082156852 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd80076da0 0x7fbd8019f380 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fbd68006010 tx=0x7fbd68005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:14.705 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.708+0000 7fbd7cd99640 1 -- 172.21.15.12:0/4082156852 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68043020 con 0x7fbd80076da0 2024-04-22T13:35:14.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.708+0000 7fbd851b1640 1 -- 172.21.15.12:0/4082156852 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd801a40c0 con 0x7fbd80076da0 2024-04-22T13:35:14.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.708+0000 7fbd7cd99640 1 -- 172.21.15.12:0/4082156852 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd68004350 con 0x7fbd80076da0 2024-04-22T13:35:14.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.708+0000 7fbd7cd99640 1 -- 172.21.15.12:0/4082156852 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68003a10 con 0x7fbd80076da0 2024-04-22T13:35:14.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.708+0000 7fbd851b1640 1 -- 172.21.15.12:0/4082156852 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd801ab9b0 con 0x7fbd80076da0 2024-04-22T13:35:14.706 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.709+0000 7fbd851b1640 1 -- 172.21.15.12:0/4082156852 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd80068d70 con 0x7fbd80076da0 2024-04-22T13:35:14.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.710+0000 7fbd7cd99640 1 -- 172.21.15.12:0/4082156852 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbd68046020 con 0x7fbd80076da0 2024-04-22T13:35:14.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.710+0000 7fbd7cd99640 1 --2- 172.21.15.12:0/4082156852 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbd6c0a3d20 0x7fbd6c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:14.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.711+0000 7fbd7ed9d640 1 --2- 172.21.15.12:0/4082156852 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbd6c0a3d20 0x7fbd6c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:14.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.711+0000 7fbd7cd99640 1 -- 172.21.15.12:0/4082156852 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7fbd680f26d0 con 0x7fbd80076da0 2024-04-22T13:35:14.707 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.711+0000 7fbd7ed9d640 1 --2- 172.21.15.12:0/4082156852 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbd6c0a3d20 0x7fbd6c0a61e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009c20 tx=0x7fbd70005f90 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:14.709 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.714+0000 7fbd7cd99640 1 -- 172.21.15.12:0/4082156852 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd6804d070 con 0x7fbd80076da0 2024-04-22T13:35:14.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.899+0000 7fbd851b1640 1 -- 172.21.15.12:0/4082156852 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbd8010dfc0 con 0x7fbd80076da0 2024-04-22T13:35:14.895 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.899+0000 7fbd7cd99640 1 -- 172.21.15.12:0/4082156852 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) ==== 104+0+11 (secure 0 0 0) 0x7fbd680beea0 con 0x7fbd80076da0 2024-04-22T13:35:14.895 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 32 2024-04-22T13:35:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.901+0000 7fbd851b1640 1 -- 172.21.15.12:0/4082156852 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbd6c0a3d20 msgr2=0x7fbd6c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.901+0000 7fbd851b1640 1 --2- 172.21.15.12:0/4082156852 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbd6c0a3d20 0x7fbd6c0a61e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009c20 tx=0x7fbd70005f90 comp rx=0 tx=0).stop 2024-04-22T13:35:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.901+0000 7fbd851b1640 1 -- 172.21.15.12:0/4082156852 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd80076da0 msgr2=0x7fbd8019f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.901+0000 7fbd851b1640 1 --2- 172.21.15.12:0/4082156852 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd80076da0 0x7fbd8019f380 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fbd68006010 tx=0x7fbd68005d90 comp rx=0 tx=0).stop 2024-04-22T13:35:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.901+0000 7fbd851b1640 1 -- 172.21.15.12:0/4082156852 shutdown_connections 2024-04-22T13:35:14.897 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.901+0000 7fbd851b1640 1 --2- 172.21.15.12:0/4082156852 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fbd6c0a3d20 0x7fbd6c0a61e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.901+0000 7fbd851b1640 1 --2- 172.21.15.12:0/4082156852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd80112e30 0x7fbd801a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.901+0000 7fbd851b1640 1 --2- 172.21.15.12:0/4082156852 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbd80076da0 0x7fbd8019f380 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.901+0000 7fbd851b1640 1 --2- 172.21.15.12:0/4082156852 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd80076480 0x7fbd8019ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.901+0000 7fbd851b1640 1 -- 172.21.15.12:0/4082156852 >> 172.21.15.12:0/4082156852 conn(0x7fbd800ffce0 msgr2=0x7fbd801020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.902+0000 7fbd851b1640 1 -- 172.21.15.12:0/4082156852 shutdown_connections 2024-04-22T13:35:14.898 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:14.902+0000 7fbd851b1640 1 -- 172.21.15.12:0/4082156852 wait complete. 2024-04-22T13:35:14.907 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:14 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2562088388' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:35:14.907 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:14 smithi012 ceph-mon[28265]: osdmap e100: 12 total, 12 up, 12 in 2024-04-22T13:35:14.907 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:14 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/886042931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:35:14.908 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-22T13:35:14.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:14 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2562088388' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:35:14.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:14 smithi134 ceph-mon[31338]: osdmap e100: 12 total, 12 up, 12 in 2024-04-22T13:35:14.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:14 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/886042931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:35:14.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:14 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2562088388' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:35:14.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:14 smithi111 ceph-mon[31361]: osdmap e100: 12 total, 12 up, 12 in 2024-04-22T13:35:14.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:14 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/886042931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-22T13:35:15.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.017+0000 7f71ab032640 1 -- 172.21.15.12:0/2100795000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71a4102190 msgr2=0x7f71a4102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:15.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.017+0000 7f71ab032640 1 --2- 172.21.15.12:0/2100795000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71a4102190 0x7f71a4102570 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f7194009900 tx=0x7f719402bc30 comp rx=0 tx=0).stop 2024-04-22T13:35:15.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.017+0000 7f71ab032640 1 -- 172.21.15.12:0/2100795000 shutdown_connections 2024-04-22T13:35:15.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.017+0000 7f71ab032640 1 --2- 172.21.15.12:0/2100795000 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71a4107260 0x7f71a410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:15.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.017+0000 7f71ab032640 1 --2- 172.21.15.12:0/2100795000 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a4102b40 0x7f71a4106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:15.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.017+0000 7f71ab032640 1 --2- 172.21.15.12:0/2100795000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71a4102190 0x7f71a4102570 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:15.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.017+0000 7f71ab032640 1 -- 172.21.15.12:0/2100795000 >> 172.21.15.12:0/2100795000 conn(0x7f71a40fd990 msgr2=0x7f71a40ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:15.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.017+0000 7f71ab032640 1 -- 172.21.15.12:0/2100795000 shutdown_connections 2024-04-22T13:35:15.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.017+0000 7f71ab032640 1 -- 172.21.15.12:0/2100795000 wait complete. 2024-04-22T13:35:15.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.018+0000 7f71ab032640 1 Processor -- start 2024-04-22T13:35:15.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.018+0000 7f71ab032640 1 -- start start 2024-04-22T13:35:15.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.018+0000 7f71ab032640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71a4102190 0x7f71a4198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:15.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.018+0000 7f71ab032640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a4107260 0x7f71a4198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:15.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.018+0000 7f71ab032640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71a419cfe0 0x7f71a419d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:15.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.018+0000 7f71ab032640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f71a41121c0 con 0x7f71a4102190 2024-04-22T13:35:15.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.018+0000 7f71ab032640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f71a4112040 con 0x7f71a419cfe0 2024-04-22T13:35:15.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.018+0000 7f71ab032640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f71a4112340 con 0x7f71a4107260 2024-04-22T13:35:15.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.018+0000 7f71a8dce640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71a4102190 0x7f71a4198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:15.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.018+0000 7f71a8dce640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71a4102190 0x7f71a4198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55632/0 (socket says 172.21.15.12:55632) 2024-04-22T13:35:15.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.018+0000 7f71a8dce640 1 -- 172.21.15.12:0/1136139726 learned_addr learned my addr 172.21.15.12:0/1136139726 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:35:15.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.019+0000 7f71a3fff640 1 --2- 172.21.15.12:0/1136139726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a4107260 0x7f71a4198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:15.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.019+0000 7f71a8dce640 1 -- 172.21.15.12:0/1136139726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a4107260 msgr2=0x7f71a4198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:15.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.019+0000 7f71a95cf640 1 --2- 172.21.15.12:0/1136139726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71a419cfe0 0x7f71a419d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:15.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.019+0000 7f71a8dce640 1 --2- 172.21.15.12:0/1136139726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a4107260 0x7f71a4198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:15.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.019+0000 7f71a8dce640 1 -- 172.21.15.12:0/1136139726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71a419cfe0 msgr2=0x7f71a419d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:15.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.019+0000 7f71a8dce640 1 --2- 172.21.15.12:0/1136139726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71a419cfe0 0x7f71a419d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:15.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.019+0000 7f71a8dce640 1 -- 172.21.15.12:0/1136139726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a419dad0 con 0x7f71a4102190 2024-04-22T13:35:15.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.019+0000 7f71a3fff640 1 --2- 172.21.15.12:0/1136139726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a4107260 0x7f71a4198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:35:15.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.019+0000 7f71a8dce640 1 --2- 172.21.15.12:0/1136139726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71a4102190 0x7f71a4198720 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f7194005940 tx=0x7f719402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:15.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.019+0000 7f71a1ffb640 1 -- 172.21.15.12:0/1136139726 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7194043020 con 0x7f71a4102190 2024-04-22T13:35:15.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.019+0000 7f71a1ffb640 1 -- 172.21.15.12:0/1136139726 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f719402cbe0 con 0x7f71a4102190 2024-04-22T13:35:15.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.019+0000 7f71ab032640 1 -- 172.21.15.12:0/1136139726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71a419dd60 con 0x7f71a4102190 2024-04-22T13:35:15.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.019+0000 7f71a1ffb640 1 -- 172.21.15.12:0/1136139726 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7194003bf0 con 0x7f71a4102190 2024-04-22T13:35:15.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.019+0000 7f71ab032640 1 -- 172.21.15.12:0/1136139726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71a41a5580 con 0x7f71a4102190 2024-04-22T13:35:15.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.020+0000 7f71ab032640 1 -- 172.21.15.12:0/1136139726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71a40691c0 con 0x7f71a4102190 2024-04-22T13:35:15.018 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.021+0000 7f71a1ffb640 1 -- 172.21.15.12:0/1136139726 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7194046020 con 0x7f71a4102190 2024-04-22T13:35:15.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.022+0000 7f71a1ffb640 1 --2- 172.21.15.12:0/1136139726 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f718c0a3d20 0x7f718c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:15.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.022+0000 7f71a3fff640 1 --2- 172.21.15.12:0/1136139726 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f718c0a3d20 0x7f718c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:15.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.022+0000 7f71a1ffb640 1 -- 172.21.15.12:0/1136139726 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7f71940f3030 con 0x7f71a4102190 2024-04-22T13:35:15.019 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.023+0000 7f71a3fff640 1 --2- 172.21.15.12:0/1136139726 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f718c0a3d20 0x7f718c0a61e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f71a41087b0 tx=0x7f7198005e50 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:15.021 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.026+0000 7f71a1ffb640 1 -- 172.21.15.12:0/1136139726 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f719404d020 con 0x7f71a4102190 2024-04-22T13:35:15.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.214+0000 7f71ab032640 1 -- 172.21.15.12:0/1136139726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f71a4102b40 con 0x7f71a4102190 2024-04-22T13:35:15.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.215+0000 7f71a1ffb640 1 -- 172.21.15.12:0/1136139726 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) ==== 100+0+11 (secure 0 0 0) 0x7f71940bf800 con 0x7f71a4102190 2024-04-22T13:35:15.211 INFO:teuthology.orchestra.run.smithi012.stdout:pg_num: 64 2024-04-22T13:35:15.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.217+0000 7f71ab032640 1 -- 172.21.15.12:0/1136139726 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f718c0a3d20 msgr2=0x7f718c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:15.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.217+0000 7f71ab032640 1 --2- 172.21.15.12:0/1136139726 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f718c0a3d20 0x7f718c0a61e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f71a41087b0 tx=0x7f7198005e50 comp rx=0 tx=0).stop 2024-04-22T13:35:15.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.217+0000 7f71ab032640 1 -- 172.21.15.12:0/1136139726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71a4102190 msgr2=0x7f71a4198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:15.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.217+0000 7f71ab032640 1 --2- 172.21.15.12:0/1136139726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71a4102190 0x7f71a4198720 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f7194005940 tx=0x7f719402c7e0 comp rx=0 tx=0).stop 2024-04-22T13:35:15.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.217+0000 7f71ab032640 1 -- 172.21.15.12:0/1136139726 shutdown_connections 2024-04-22T13:35:15.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.217+0000 7f71ab032640 1 --2- 172.21.15.12:0/1136139726 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f718c0a3d20 0x7f718c0a61e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:15.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.217+0000 7f71ab032640 1 --2- 172.21.15.12:0/1136139726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f71a419cfe0 0x7f71a419d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:15.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.217+0000 7f71ab032640 1 --2- 172.21.15.12:0/1136139726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71a4107260 0x7f71a4198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:15.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.217+0000 7f71ab032640 1 --2- 172.21.15.12:0/1136139726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f71a4102190 0x7f71a4198720 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:15.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.217+0000 7f71ab032640 1 -- 172.21.15.12:0/1136139726 >> 172.21.15.12:0/1136139726 conn(0x7f71a40fd990 msgr2=0x7f71a40fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:15.213 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.217+0000 7f71ab032640 1 -- 172.21.15.12:0/1136139726 shutdown_connections 2024-04-22T13:35:15.214 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.217+0000 7f71ab032640 1 -- 172.21.15.12:0/1136139726 wait complete. 2024-04-22T13:35:15.224 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-22T13:35:15.328 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.331+0000 7efeb47ef640 1 -- 172.21.15.12:0/1917594395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efeac06a840 msgr2=0x7efeac06ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:15.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.331+0000 7efeb47ef640 1 --2- 172.21.15.12:0/1917594395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efeac06a840 0x7efeac06ac20 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7efea8009ca0 tx=0x7efea802c0f0 comp rx=0 tx=0).stop 2024-04-22T13:35:15.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.331+0000 7efeb47ef640 1 -- 172.21.15.12:0/1917594395 shutdown_connections 2024-04-22T13:35:15.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.331+0000 7efeb47ef640 1 --2- 172.21.15.12:0/1917594395 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efeac107270 0x7efeac10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:15.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.331+0000 7efeb47ef640 1 --2- 172.21.15.12:0/1917594395 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeac06b1f0 0x7efeac106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:15.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.331+0000 7efeb47ef640 1 --2- 172.21.15.12:0/1917594395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efeac06a840 0x7efeac06ac20 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:15.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.331+0000 7efeb47ef640 1 -- 172.21.15.12:0/1917594395 >> 172.21.15.12:0/1917594395 conn(0x7efeac077eb0 msgr2=0x7efeac0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:15.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.331+0000 7efeb47ef640 1 -- 172.21.15.12:0/1917594395 shutdown_connections 2024-04-22T13:35:15.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.331+0000 7efeb47ef640 1 -- 172.21.15.12:0/1917594395 wait complete. 2024-04-22T13:35:15.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.332+0000 7efeb47ef640 1 Processor -- start 2024-04-22T13:35:15.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.332+0000 7efeb47ef640 1 -- start start 2024-04-22T13:35:15.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.332+0000 7efeb47ef640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeac06a840 0x7efeac100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:15.329 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.332+0000 7efeb47ef640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efeac107270 0x7efeac1013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:15.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.332+0000 7efeb47ef640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efeac101900 0x7efeac1a6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:15.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.332+0000 7efeb47ef640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7efeac1121b0 con 0x7efeac107270 2024-04-22T13:35:15.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.332+0000 7efeb47ef640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efeac112030 con 0x7efeac101900 2024-04-22T13:35:15.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.332+0000 7efeb47ef640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efeac112330 con 0x7efeac06a840 2024-04-22T13:35:15.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.332+0000 7efeb1d8a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efeac107270 0x7efeac1013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:15.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.332+0000 7efeb1d8a640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efeac107270 0x7efeac1013c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55656/0 (socket says 172.21.15.12:55656) 2024-04-22T13:35:15.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.332+0000 7efeb1d8a640 1 -- 172.21.15.12:0/300074088 learned_addr learned my addr 172.21.15.12:0/300074088 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:35:15.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.332+0000 7efeb2d8c640 1 --2- 172.21.15.12:0/300074088 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efeac101900 0x7efeac1a6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:15.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.332+0000 7efeb258b640 1 --2- 172.21.15.12:0/300074088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeac06a840 0x7efeac100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:15.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.332+0000 7efeb1d8a640 1 -- 172.21.15.12:0/300074088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeac06a840 msgr2=0x7efeac100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:15.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.332+0000 7efeb1d8a640 1 --2- 172.21.15.12:0/300074088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeac06a840 0x7efeac100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:15.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.332+0000 7efeb1d8a640 1 -- 172.21.15.12:0/300074088 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efeac101900 msgr2=0x7efeac1a6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:15.330 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.332+0000 7efeb1d8a640 1 --2- 172.21.15.12:0/300074088 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efeac101900 0x7efeac1a6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:15.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.332+0000 7efeb1d8a640 1 -- 172.21.15.12:0/300074088 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeac1a6f40 con 0x7efeac107270 2024-04-22T13:35:15.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.332+0000 7efeb2d8c640 1 --2- 172.21.15.12:0/300074088 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efeac101900 0x7efeac1a6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-22T13:35:15.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.333+0000 7efeb1d8a640 1 --2- 172.21.15.12:0/300074088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efeac107270 0x7efeac1013c0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00c970 tx=0x7efe9c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:15.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.333+0000 7efea37fe640 1 -- 172.21.15.12:0/300074088 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c014070 con 0x7efeac107270 2024-04-22T13:35:15.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.333+0000 7efea37fe640 1 -- 172.21.15.12:0/300074088 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe9c004440 con 0x7efeac107270 2024-04-22T13:35:15.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.333+0000 7efea37fe640 1 -- 172.21.15.12:0/300074088 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c003e50 con 0x7efeac107270 2024-04-22T13:35:15.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.333+0000 7efeb47ef640 1 -- 172.21.15.12:0/300074088 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeac1a7230 con 0x7efeac107270 2024-04-22T13:35:15.331 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.333+0000 7efeb47ef640 1 -- 172.21.15.12:0/300074088 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeac1a7730 con 0x7efeac107270 2024-04-22T13:35:15.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.334+0000 7efeb47ef640 1 -- 172.21.15.12:0/300074088 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeac0691c0 con 0x7efeac107270 2024-04-22T13:35:15.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.335+0000 7efea37fe640 1 -- 172.21.15.12:0/300074088 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efe9c017070 con 0x7efeac107270 2024-04-22T13:35:15.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.336+0000 7efea37fe640 1 --2- 172.21.15.12:0/300074088 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7efe980a3ef0 0x7efe980a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:15.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.336+0000 7efeb258b640 1 --2- 172.21.15.12:0/300074088 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7efe980a3ef0 0x7efe980a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:15.332 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.336+0000 7efea37fe640 1 -- 172.21.15.12:0/300074088 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7efe9c0d0b70 con 0x7efeac107270 2024-04-22T13:35:15.333 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.336+0000 7efeb258b640 1 --2- 172.21.15.12:0/300074088 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7efe980a3ef0 0x7efe980a63b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7efea8006010 tx=0x7efea802cca0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:15.334 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.339+0000 7efea37fe640 1 -- 172.21.15.12:0/300074088 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efe9c01e030 con 0x7efeac107270 2024-04-22T13:35:15.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.537+0000 7efeb47ef640 1 -- 172.21.15.12:0/300074088 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efeac10a4d0 con 0x7efeac107270 2024-04-22T13:35:15.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.538+0000 7efea37fe640 1 -- 172.21.15.12:0/300074088 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v100) ==== 81+0+945 (secure 0 0 0) 0x7efe9c09d230 con 0x7efeac107270 2024-04-22T13:35:15.533 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3535299028 2024-04-23T13:24:06.155564+0000 2024-04-22T13:35:15.534 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2837479598 2024-04-23T13:24:06.155564+0000 2024-04-22T13:35:15.534 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3010129696 2024-04-23T13:24:06.155564+0000 2024-04-22T13:35:15.534 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2593601362 2024-04-23T13:15:04.279990+0000 2024-04-22T13:35:15.534 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/2519382069 2024-04-23T13:15:04.279990+0000 2024-04-22T13:35:15.534 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/2009831940 2024-04-23T13:15:04.279990+0000 2024-04-22T13:35:15.534 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/856202072 2024-04-23T13:24:06.155564+0000 2024-04-22T13:35:15.534 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3509412187 2024-04-23T13:14:36.603653+0000 2024-04-22T13:35:15.534 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/1802873236 2024-04-23T13:24:06.155564+0000 2024-04-22T13:35:15.534 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/1007749062 2024-04-23T13:15:04.279990+0000 2024-04-22T13:35:15.534 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/3588480095 2024-04-23T13:14:36.603653+0000 2024-04-22T13:35:15.534 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/2091570832 2024-04-23T13:14:36.603653+0000 2024-04-22T13:35:15.534 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:0/1904962176 2024-04-23T13:14:36.603653+0000 2024-04-22T13:35:15.534 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6801/2091570832 2024-04-23T13:14:36.603653+0000 2024-04-22T13:35:15.534 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/1802873236 2024-04-23T13:24:06.155564+0000 2024-04-22T13:35:15.535 INFO:teuthology.orchestra.run.smithi012.stdout:172.21.15.12:6800/2009831940 2024-04-23T13:15:04.279990+0000 2024-04-22T13:35:15.535 INFO:teuthology.orchestra.run.smithi012.stderr:listed 16 entries 2024-04-22T13:35:15.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.540+0000 7efeb47ef640 1 -- 172.21.15.12:0/300074088 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7efe980a3ef0 msgr2=0x7efe980a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:15.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.540+0000 7efeb47ef640 1 --2- 172.21.15.12:0/300074088 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7efe980a3ef0 0x7efe980a63b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7efea8006010 tx=0x7efea802cca0 comp rx=0 tx=0).stop 2024-04-22T13:35:15.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.540+0000 7efeb47ef640 1 -- 172.21.15.12:0/300074088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efeac107270 msgr2=0x7efeac1013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:15.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.540+0000 7efeb47ef640 1 --2- 172.21.15.12:0/300074088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efeac107270 0x7efeac1013c0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00c970 tx=0x7efe9c00ce30 comp rx=0 tx=0).stop 2024-04-22T13:35:15.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.540+0000 7efeb47ef640 1 -- 172.21.15.12:0/300074088 shutdown_connections 2024-04-22T13:35:15.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.540+0000 7efeb47ef640 1 --2- 172.21.15.12:0/300074088 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7efe980a3ef0 0x7efe980a63b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:15.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.540+0000 7efeb47ef640 1 --2- 172.21.15.12:0/300074088 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efeac101900 0x7efeac1a6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:15.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.540+0000 7efeb47ef640 1 --2- 172.21.15.12:0/300074088 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efeac107270 0x7efeac1013c0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:15.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.540+0000 7efeb47ef640 1 --2- 172.21.15.12:0/300074088 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efeac06a840 0x7efeac100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:15.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.540+0000 7efeb47ef640 1 -- 172.21.15.12:0/300074088 >> 172.21.15.12:0/300074088 conn(0x7efeac077eb0 msgr2=0x7efeac100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:15.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.540+0000 7efeb47ef640 1 -- 172.21.15.12:0/300074088 shutdown_connections 2024-04-22T13:35:15.536 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:15.540+0000 7efeb47ef640 1 -- 172.21.15.12:0/300074088 wait complete. 2024-04-22T13:35:15.546 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-22T13:35:15.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:35:15.546 DEBUG:teuthology.orchestra.run.smithi012:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-22T13:35:15.584 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi012.front.sepia.ceph.com 2024-04-22T13:35:15.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:35:15.584 DEBUG:teuthology.orchestra.run.smithi012:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-22T13:35:15.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:35:15.660 DEBUG:teuthology.orchestra.run.smithi012:> ip netns list 2024-04-22T13:35:15.715 INFO:teuthology.orchestra.run.smithi012.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-22T13:35:15.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:35:15.716 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-22T13:35:15.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-22T13:35:15.781 DEBUG:teuthology.orchestra.run.smithi012:> sudo ip link delete ceph-brx 2024-04-22T13:35:15.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:15 smithi012 ceph-mon[28265]: pgmap v346: 97 pgs: 97 active+clean; 2.3 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:35:15.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:15 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/4082156852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:35:15.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:15 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1136139726' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:35:15.836 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:15 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/300074088' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-22T13:35:15.879 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-22T13:35:15.891 INFO:tasks.cephadm:Teardown begin 2024-04-22T13:35:15.891 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-22T13:35:15.891 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:35:15.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:15 smithi134 ceph-mon[31338]: pgmap v346: 97 pgs: 97 active+clean; 2.3 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:35:15.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:15 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/4082156852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:35:15.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:15 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1136139726' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:35:15.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:15 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/300074088' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-22T13:35:15.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:15 smithi111 ceph-mon[31361]: pgmap v346: 97 pgs: 97 active+clean; 2.3 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-22T13:35:15.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:15 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/4082156852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-22T13:35:15.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:15 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1136139726' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-22T13:35:15.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:15 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/300074088' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-22T13:35:16.025 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.027+0000 7f214f7b1640 1 -- 172.21.15.12:0/640178292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2148103af0 msgr2=0x7f214810bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:16.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.027+0000 7f214f7b1640 1 --2- 172.21.15.12:0/640178292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2148103af0 0x7f214810bfd0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f2138009900 tx=0x7f213802bcd0 comp rx=0 tx=0).stop 2024-04-22T13:35:16.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.028+0000 7f214f7b1640 1 -- 172.21.15.12:0/640178292 shutdown_connections 2024-04-22T13:35:16.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.028+0000 7f214f7b1640 1 --2- 172.21.15.12:0/640178292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f214810c5a0 0x7f214810e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:16.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.028+0000 7f214f7b1640 1 --2- 172.21.15.12:0/640178292 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2148103af0 0x7f214810bfd0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:16.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.028+0000 7f214f7b1640 1 --2- 172.21.15.12:0/640178292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f21481031d0 0x7f21481035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:16.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.028+0000 7f214f7b1640 1 -- 172.21.15.12:0/640178292 >> 172.21.15.12:0/640178292 conn(0x7f21480771d0 msgr2=0x7f21480775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:16.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.028+0000 7f214f7b1640 1 -- 172.21.15.12:0/640178292 shutdown_connections 2024-04-22T13:35:16.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.028+0000 7f214f7b1640 1 -- 172.21.15.12:0/640178292 wait complete. 2024-04-22T13:35:16.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.028+0000 7f214f7b1640 1 Processor -- start 2024-04-22T13:35:16.026 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.028+0000 7f214f7b1640 1 -- start start 2024-04-22T13:35:16.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.028+0000 7f214f7b1640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21481031d0 0x7f214819e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:16.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.028+0000 7f214f7b1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2148103af0 0x7f214819e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:16.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.028+0000 7f214f7b1640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f214810c5a0 0x7f21481985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:16.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.028+0000 7f214f7b1640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f21481121e0 con 0x7f21481031d0 2024-04-22T13:35:16.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.028+0000 7f214f7b1640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2148112060 con 0x7f214810c5a0 2024-04-22T13:35:16.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.028+0000 7f214f7b1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2148112360 con 0x7f2148103af0 2024-04-22T13:35:16.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.029+0000 7f214d54d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21481031d0 0x7f214819e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:16.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.029+0000 7f214d54d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21481031d0 0x7f214819e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55682/0 (socket says 172.21.15.12:55682) 2024-04-22T13:35:16.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.029+0000 7f214d54d640 1 -- 172.21.15.12:0/1334316782 learned_addr learned my addr 172.21.15.12:0/1334316782 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:35:16.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.029+0000 7f214dd4e640 1 --2- 172.21.15.12:0/1334316782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f214810c5a0 0x7f21481985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:16.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.029+0000 7f214d54d640 1 -- 172.21.15.12:0/1334316782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2148103af0 msgr2=0x7f214819e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:16.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.029+0000 7f214cd4c640 1 --2- 172.21.15.12:0/1334316782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2148103af0 0x7f214819e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:16.027 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.029+0000 7f214d54d640 1 --2- 172.21.15.12:0/1334316782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2148103af0 0x7f214819e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:16.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.029+0000 7f214d54d640 1 -- 172.21.15.12:0/1334316782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f214810c5a0 msgr2=0x7f21481985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:16.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.029+0000 7f214d54d640 1 --2- 172.21.15.12:0/1334316782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f214810c5a0 0x7f21481985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:16.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.029+0000 7f214d54d640 1 -- 172.21.15.12:0/1334316782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2148198d60 con 0x7f21481031d0 2024-04-22T13:35:16.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.029+0000 7f214dd4e640 1 --2- 172.21.15.12:0/1334316782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f214810c5a0 0x7f21481985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:35:16.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.029+0000 7f214cd4c640 1 --2- 172.21.15.12:0/1334316782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2148103af0 0x7f214819e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:35:16.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.029+0000 7f214d54d640 1 --2- 172.21.15.12:0/1334316782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21481031d0 0x7f214819e430 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f214400e9f0 tx=0x7f214400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:16.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.029+0000 7f213e7fc640 1 -- 172.21.15.12:0/1334316782 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214400cd80 con 0x7f21481031d0 2024-04-22T13:35:16.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.029+0000 7f213e7fc640 1 -- 172.21.15.12:0/1334316782 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21440108e0 con 0x7f21481031d0 2024-04-22T13:35:16.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.029+0000 7f214f7b1640 1 -- 172.21.15.12:0/1334316782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2148199050 con 0x7f21481031d0 2024-04-22T13:35:16.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.029+0000 7f213e7fc640 1 -- 172.21.15.12:0/1334316782 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2144018980 con 0x7f21481031d0 2024-04-22T13:35:16.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.029+0000 7f214f7b1640 1 -- 172.21.15.12:0/1334316782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21481a52f0 con 0x7f21481031d0 2024-04-22T13:35:16.028 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.030+0000 7f214f7b1640 1 -- 172.21.15.12:0/1334316782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2148069230 con 0x7f21481031d0 2024-04-22T13:35:16.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.032+0000 7f213e7fc640 1 -- 172.21.15.12:0/1334316782 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2144010430 con 0x7f21481031d0 2024-04-22T13:35:16.029 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.033+0000 7f213e7fc640 1 --2- 172.21.15.12:0/1334316782 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f21340a3da0 0x7f21340a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:16.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.033+0000 7f214cd4c640 1 --2- 172.21.15.12:0/1334316782 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f21340a3da0 0x7f21340a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:16.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.033+0000 7f213e7fc640 1 -- 172.21.15.12:0/1334316782 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7f21440d14a0 con 0x7f21481031d0 2024-04-22T13:35:16.030 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.033+0000 7f214cd4c640 1 --2- 172.21.15.12:0/1334316782 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f21340a3da0 0x7f21340a6260 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2138036040 tx=0x7f2138000bc0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:16.032 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.036+0000 7f213e7fc640 1 -- 172.21.15.12:0/1334316782 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f214409db60 con 0x7f21481031d0 2024-04-22T13:35:16.256 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.261+0000 7f214f7b1640 1 -- 172.21.15.12:0/1334316782 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2148199b10 con 0x7f21481031d0 2024-04-22T13:35:16.257 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.262+0000 7f213e7fc640 1 -- 172.21.15.12:0/1334316782 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7f214409d460 con 0x7f21481031d0 2024-04-22T13:35:16.257 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:35:16.258 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:35:16.258 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:35:16.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.265+0000 7f214f7b1640 1 -- 172.21.15.12:0/1334316782 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f21340a3da0 msgr2=0x7f21340a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:16.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.265+0000 7f214f7b1640 1 --2- 172.21.15.12:0/1334316782 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f21340a3da0 0x7f21340a6260 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2138036040 tx=0x7f2138000bc0 comp rx=0 tx=0).stop 2024-04-22T13:35:16.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.265+0000 7f214f7b1640 1 -- 172.21.15.12:0/1334316782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21481031d0 msgr2=0x7f214819e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:16.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.265+0000 7f214f7b1640 1 --2- 172.21.15.12:0/1334316782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21481031d0 0x7f214819e430 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f214400e9f0 tx=0x7f214400eeb0 comp rx=0 tx=0).stop 2024-04-22T13:35:16.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.265+0000 7f214f7b1640 1 -- 172.21.15.12:0/1334316782 shutdown_connections 2024-04-22T13:35:16.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.265+0000 7f214f7b1640 1 --2- 172.21.15.12:0/1334316782 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f21340a3da0 0x7f21340a6260 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:16.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.265+0000 7f214f7b1640 1 --2- 172.21.15.12:0/1334316782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f214810c5a0 0x7f21481985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:16.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.265+0000 7f214f7b1640 1 --2- 172.21.15.12:0/1334316782 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2148103af0 0x7f214819e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:16.260 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.265+0000 7f214f7b1640 1 --2- 172.21.15.12:0/1334316782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f21481031d0 0x7f214819e430 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:16.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.265+0000 7f214f7b1640 1 -- 172.21.15.12:0/1334316782 >> 172.21.15.12:0/1334316782 conn(0x7f21480771d0 msgr2=0x7f21480ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:16.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.265+0000 7f214f7b1640 1 -- 172.21.15.12:0/1334316782 shutdown_connections 2024-04-22T13:35:16.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.265+0000 7f214f7b1640 1 -- 172.21.15.12:0/1334316782 wait complete. 2024-04-22T13:35:16.272 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-22T13:35:16.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.379+0000 7fa3541f4640 1 -- 172.21.15.12:0/2840164205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa34c102b40 msgr2=0x7fa34c106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:16.376 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.379+0000 7fa3541f4640 1 --2- 172.21.15.12:0/2840164205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa34c102b40 0x7fa34c106b80 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fa340009900 tx=0x7fa34002bc30 comp rx=0 tx=0).stop 2024-04-22T13:35:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.379+0000 7fa3541f4640 1 -- 172.21.15.12:0/2840164205 shutdown_connections 2024-04-22T13:35:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.379+0000 7fa3541f4640 1 --2- 172.21.15.12:0/2840164205 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa34c107260 0x7fa34c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.379+0000 7fa3541f4640 1 --2- 172.21.15.12:0/2840164205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa34c102b40 0x7fa34c106b80 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.379+0000 7fa3541f4640 1 --2- 172.21.15.12:0/2840164205 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa34c102190 0x7fa34c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.379+0000 7fa3541f4640 1 -- 172.21.15.12:0/2840164205 >> 172.21.15.12:0/2840164205 conn(0x7fa34c0793e0 msgr2=0x7fa34c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.379+0000 7fa3541f4640 1 -- 172.21.15.12:0/2840164205 shutdown_connections 2024-04-22T13:35:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.379+0000 7fa3541f4640 1 -- 172.21.15.12:0/2840164205 wait complete. 2024-04-22T13:35:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.379+0000 7fa3541f4640 1 Processor -- start 2024-04-22T13:35:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.379+0000 7fa3541f4640 1 -- start start 2024-04-22T13:35:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.379+0000 7fa3541f4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa34c102190 0x7fa34c1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.379+0000 7fa3541f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa34c107260 0x7fa34c1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.379+0000 7fa3541f4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa34c19a7e0 0x7fa34c19ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.379+0000 7fa3541f4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c112180 con 0x7fa34c102190 2024-04-22T13:35:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.379+0000 7fa3541f4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c112000 con 0x7fa34c19a7e0 2024-04-22T13:35:16.377 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.379+0000 7fa3541f4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c112300 con 0x7fa34c107260 2024-04-22T13:35:16.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.380+0000 7fa351f90640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa34c102190 0x7fa34c1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:16.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.380+0000 7fa351f90640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa34c102190 0x7fa34c1a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55694/0 (socket says 172.21.15.12:55694) 2024-04-22T13:35:16.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.380+0000 7fa351f90640 1 -- 172.21.15.12:0/2445042290 learned_addr learned my addr 172.21.15.12:0/2445042290 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:35:16.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.380+0000 7fa352791640 1 --2- 172.21.15.12:0/2445042290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa34c19a7e0 0x7fa34c19ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:16.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.380+0000 7fa35178f640 1 --2- 172.21.15.12:0/2445042290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa34c107260 0x7fa34c1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:16.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.380+0000 7fa351f90640 1 -- 172.21.15.12:0/2445042290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa34c107260 msgr2=0x7fa34c1a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:16.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.380+0000 7fa351f90640 1 --2- 172.21.15.12:0/2445042290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa34c107260 0x7fa34c1a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:16.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.380+0000 7fa351f90640 1 -- 172.21.15.12:0/2445042290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa34c19a7e0 msgr2=0x7fa34c19ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:16.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.380+0000 7fa351f90640 1 --2- 172.21.15.12:0/2445042290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa34c19a7e0 0x7fa34c19ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:16.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.380+0000 7fa351f90640 1 -- 172.21.15.12:0/2445042290 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa34c19b370 con 0x7fa34c102190 2024-04-22T13:35:16.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.380+0000 7fa351f90640 1 --2- 172.21.15.12:0/2445042290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa34c102190 0x7fa34c1a0670 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fa33c00eab0 tx=0x7fa33c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:16.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.380+0000 7fa34affd640 1 -- 172.21.15.12:0/2445042290 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c00ce40 con 0x7fa34c102190 2024-04-22T13:35:16.378 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.380+0000 7fa3541f4640 1 -- 172.21.15.12:0/2445042290 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa34c19b660 con 0x7fa34c102190 2024-04-22T13:35:16.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.380+0000 7fa34affd640 1 -- 172.21.15.12:0/2445042290 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa33c011920 con 0x7fa34c102190 2024-04-22T13:35:16.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.380+0000 7fa3541f4640 1 -- 172.21.15.12:0/2445042290 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa34c1a7860 con 0x7fa34c102190 2024-04-22T13:35:16.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.380+0000 7fa34affd640 1 -- 172.21.15.12:0/2445042290 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c0109d0 con 0x7fa34c102190 2024-04-22T13:35:16.379 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.381+0000 7fa3541f4640 1 -- 172.21.15.12:0/2445042290 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa34c103660 con 0x7fa34c102190 2024-04-22T13:35:16.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.382+0000 7fa34affd640 1 -- 172.21.15.12:0/2445042290 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa33c011470 con 0x7fa34c102190 2024-04-22T13:35:16.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.383+0000 7fa34affd640 1 --2- 172.21.15.12:0/2445042290 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa3340a3d20 0x7fa3340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:16.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.383+0000 7fa35178f640 1 --2- 172.21.15.12:0/2445042290 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa3340a3d20 0x7fa3340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:16.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.383+0000 7fa34affd640 1 -- 172.21.15.12:0/2445042290 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7fa33c0d0f20 con 0x7fa34c102190 2024-04-22T13:35:16.380 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.384+0000 7fa35178f640 1 --2- 172.21.15.12:0/2445042290 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa3340a3d20 0x7fa3340a61e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa340006010 tx=0x7fa34002c690 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:16.382 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.387+0000 7fa34affd640 1 -- 172.21.15.12:0/2445042290 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa34c19b660 con 0x7fa34c102190 2024-04-22T13:35:16.610 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.615+0000 7fa3541f4640 1 -- 172.21.15.12:0/2445042290 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa34c069230 con 0x7fa34c102190 2024-04-22T13:35:16.611 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.615+0000 7fa34affd640 1 -- 172.21.15.12:0/2445042290 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7fa33c016030 con 0x7fa34c102190 2024-04-22T13:35:16.611 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:35:16.611 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6835/1590002838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6834","nonce":1590002838},{"type":"v1","addr":"172.21.15.12:6835","nonce":1590002838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.12:6837/3677153272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6836","nonce":3677153272},{"type":"v1","addr":"172.21.15.12:6837","nonce":3677153272}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-22T13:22:04.602553+0000","modified":"2024-04-22T13:24:04.600025+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24509,"mds_1":24484,"mds_2":24448,"mds_3":24443,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14631":{"gid":14631,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6839/4083188176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6838","nonce":4083188176},{"type":"v1","addr":"172.21.15.12:6839","nonce":4083188176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14643":{"gid":14643,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.12:6841/763294965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6840","nonce":763294965},{"type":"v1","addr":"172.21.15.12:6841","nonce":763294965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6833/4235830781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":4235830781},{"type":"v1","addr":"172.21.15.134:6833","nonce":4235830781}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.111:6833/1902687060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1902687060},{"type":"v1","addr":"172.21.15.111:6833","nonce":1902687060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.111:6835/2665640170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":2665640170},{"type":"v1","addr":"172.21.15.111:6835","nonce":2665640170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.134:6835/3927126579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3927126579},{"type":"v1","addr":"172.21.15.134:6835","nonce":3927126579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.134:6837/3631577545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6836","nonce":3631577545},{"type":"v1","addr":"172.21.15.134:6837","nonce":3631577545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6837/4218407470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":4218407470},{"type":"v1","addr":"172.21.15.111:6837","nonce":4218407470}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.134:6839/1488513960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1488513960},{"type":"v1","addr":"172.21.15.134:6839","nonce":1488513960}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2126656770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2126656770},{"type":"v1","addr":"172.21.15.111:6839","nonce":2126656770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24509,"qdb_cluster":[24407,24484,24443,24448,24509]},"id":1}]} 2024-04-22T13:35:16.611 INFO:teuthology.orchestra.run.smithi012.stderr:dumped fsmap epoch 28 2024-04-22T13:35:16.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.618+0000 7fa3541f4640 1 -- 172.21.15.12:0/2445042290 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa3340a3d20 msgr2=0x7fa3340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:16.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.618+0000 7fa3541f4640 1 --2- 172.21.15.12:0/2445042290 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa3340a3d20 0x7fa3340a61e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa340006010 tx=0x7fa34002c690 comp rx=0 tx=0).stop 2024-04-22T13:35:16.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.618+0000 7fa3541f4640 1 -- 172.21.15.12:0/2445042290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa34c102190 msgr2=0x7fa34c1a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:16.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.618+0000 7fa3541f4640 1 --2- 172.21.15.12:0/2445042290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa34c102190 0x7fa34c1a0670 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fa33c00eab0 tx=0x7fa33c00ef70 comp rx=0 tx=0).stop 2024-04-22T13:35:16.613 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.618+0000 7fa3541f4640 1 -- 172.21.15.12:0/2445042290 shutdown_connections 2024-04-22T13:35:16.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.618+0000 7fa3541f4640 1 --2- 172.21.15.12:0/2445042290 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa3340a3d20 0x7fa3340a61e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:16.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.618+0000 7fa3541f4640 1 --2- 172.21.15.12:0/2445042290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa34c19a7e0 0x7fa34c19ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:16.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.618+0000 7fa3541f4640 1 --2- 172.21.15.12:0/2445042290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa34c107260 0x7fa34c1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:16.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.618+0000 7fa3541f4640 1 --2- 172.21.15.12:0/2445042290 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa34c102190 0x7fa34c1a0670 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:16.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.618+0000 7fa3541f4640 1 -- 172.21.15.12:0/2445042290 >> 172.21.15.12:0/2445042290 conn(0x7fa34c0793e0 msgr2=0x7fa34c1000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:16.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.618+0000 7fa3541f4640 1 -- 172.21.15.12:0/2445042290 shutdown_connections 2024-04-22T13:35:16.614 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.618+0000 7fa3541f4640 1 -- 172.21.15.12:0/2445042290 wait complete. 2024-04-22T13:35:16.624 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T13:35:16.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.731+0000 7fc5cdb32640 1 -- 172.21.15.12:0/2043880480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5c8102180 msgr2=0x7fc5c8102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:16.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.731+0000 7fc5cdb32640 1 --2- 172.21.15.12:0/2043880480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5c8102180 0x7fc5c8102560 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8009900 tx=0x7fc5b802bc30 comp rx=0 tx=0).stop 2024-04-22T13:35:16.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.731+0000 7fc5cdb32640 1 -- 172.21.15.12:0/2043880480 shutdown_connections 2024-04-22T13:35:16.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.731+0000 7fc5cdb32640 1 --2- 172.21.15.12:0/2043880480 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5c8107250 0x7fc5c810ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:16.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.731+0000 7fc5cdb32640 1 --2- 172.21.15.12:0/2043880480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5c8102b30 0x7fc5c8106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:16.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.731+0000 7fc5cdb32640 1 --2- 172.21.15.12:0/2043880480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5c8102180 0x7fc5c8102560 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:16.729 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.731+0000 7fc5cdb32640 1 -- 172.21.15.12:0/2043880480 >> 172.21.15.12:0/2043880480 conn(0x7fc5c8079350 msgr2=0x7fc5c8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:16.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.731+0000 7fc5cdb32640 1 -- 172.21.15.12:0/2043880480 shutdown_connections 2024-04-22T13:35:16.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.731+0000 7fc5cdb32640 1 -- 172.21.15.12:0/2043880480 wait complete. 2024-04-22T13:35:16.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.732+0000 7fc5cdb32640 1 Processor -- start 2024-04-22T13:35:16.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.732+0000 7fc5cdb32640 1 -- start start 2024-04-22T13:35:16.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.732+0000 7fc5cdb32640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5c8102180 0x7fc5c8074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:16.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.732+0000 7fc5cdb32640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5c8107250 0x7fc5c8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:16.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.732+0000 7fc5cdb32640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5c8074cd0 0x7fc5c81a6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:16.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.732+0000 7fc5cdb32640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c8112230 con 0x7fc5c8074cd0 2024-04-22T13:35:16.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.732+0000 7fc5cdb32640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c81120b0 con 0x7fc5c8107250 2024-04-22T13:35:16.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.732+0000 7fc5cdb32640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c81123b0 con 0x7fc5c8102180 2024-04-22T13:35:16.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.732+0000 7fc5c7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5c8074cd0 0x7fc5c81a6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:16.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.733+0000 7fc5c7fff640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5c8074cd0 0x7fc5c81a6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55716/0 (socket says 172.21.15.12:55716) 2024-04-22T13:35:16.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.733+0000 7fc5c7fff640 1 -- 172.21.15.12:0/2377969895 learned_addr learned my addr 172.21.15.12:0/2377969895 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:35:16.730 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.733+0000 7fc5c77fe640 1 --2- 172.21.15.12:0/2377969895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5c8102180 0x7fc5c8074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:16.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.733+0000 7fc5c6ffd640 1 --2- 172.21.15.12:0/2377969895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5c8107250 0x7fc5c8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:16.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.733+0000 7fc5c7fff640 1 -- 172.21.15.12:0/2377969895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5c8102180 msgr2=0x7fc5c8074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:16.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.733+0000 7fc5c7fff640 1 --2- 172.21.15.12:0/2377969895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5c8102180 0x7fc5c8074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:16.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.733+0000 7fc5c7fff640 1 -- 172.21.15.12:0/2377969895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5c8107250 msgr2=0x7fc5c8074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:16.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.733+0000 7fc5c7fff640 1 --2- 172.21.15.12:0/2377969895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5c8107250 0x7fc5c8074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:16.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.733+0000 7fc5c7fff640 1 -- 172.21.15.12:0/2377969895 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c81a7060 con 0x7fc5c8074cd0 2024-04-22T13:35:16.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.733+0000 7fc5c6ffd640 1 --2- 172.21.15.12:0/2377969895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5c8107250 0x7fc5c8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:35:16.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.733+0000 7fc5c7fff640 1 --2- 172.21.15.12:0/2377969895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5c8074cd0 0x7fc5c81a6960 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fc5c000b970 tx=0x7fc5c000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:16.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.733+0000 7fc5c4ff9640 1 -- 172.21.15.12:0/2377969895 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0019070 con 0x7fc5c8074cd0 2024-04-22T13:35:16.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.733+0000 7fc5c4ff9640 1 -- 172.21.15.12:0/2377969895 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5c0014440 con 0x7fc5c8074cd0 2024-04-22T13:35:16.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.733+0000 7fc5c4ff9640 1 -- 172.21.15.12:0/2377969895 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c00075d0 con 0x7fc5c8074cd0 2024-04-22T13:35:16.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.733+0000 7fc5cdb32640 1 -- 172.21.15.12:0/2377969895 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5c81a7350 con 0x7fc5c8074cd0 2024-04-22T13:35:16.731 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.733+0000 7fc5cdb32640 1 -- 172.21.15.12:0/2377969895 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5c81a78a0 con 0x7fc5c8074cd0 2024-04-22T13:35:16.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.734+0000 7fc5cdb32640 1 -- 172.21.15.12:0/2377969895 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5c8068de0 con 0x7fc5c8074cd0 2024-04-22T13:35:16.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.736+0000 7fc5c4ff9640 1 -- 172.21.15.12:0/2377969895 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc5c001c070 con 0x7fc5c8074cd0 2024-04-22T13:35:16.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.736+0000 7fc5c4ff9640 1 --2- 172.21.15.12:0/2377969895 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc5b00a3dd0 0x7fc5b00a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:16.732 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.737+0000 7fc5c77fe640 1 --2- 172.21.15.12:0/2377969895 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc5b00a3dd0 0x7fc5b00a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:16.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.737+0000 7fc5c4ff9640 1 -- 172.21.15.12:0/2377969895 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7fc5c00d0100 con 0x7fc5c8074cd0 2024-04-22T13:35:16.733 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.737+0000 7fc5c77fe640 1 --2- 172.21.15.12:0/2377969895 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc5b00a3dd0 0x7fc5b00a6290 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8006010 tx=0x7fc5b8005e40 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:16.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.740+0000 7fc5c4ff9640 1 -- 172.21.15.12:0/2377969895 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc5c009c280 con 0x7fc5c8074cd0 2024-04-22T13:35:16.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.928+0000 7fc5cdb32640 1 -- 172.21.15.12:0/2377969895 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc5c810a480 con 0x7fc5c8074cd0 2024-04-22T13:35:16.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.930+0000 7fc5c4ff9640 1 -- 172.21.15.12:0/2377969895 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+23088 (secure 0 0 0) 0x7fc5c00a0580 con 0x7fc5c8074cd0 2024-04-22T13:35:16.925 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-22T13:35:16.925 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":100,"fsid":"0aede5ea-00aa-11ef-bc93-c7b262605968","created":"2024-04-22T13:14:09.362918+0000","modified":"2024-04-22T13:35:14.539576+0000","last_up_change":"2024-04-22T13:21:15.263393+0000","last_in_change":"2024-04-22T13:20:54.174685+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-22T13:17:32.372434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-22T13:22:02.768799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-22T13:22:03.578730+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d9e2365-3a64-4b13-96fc-c217b1cee2ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6803","nonce":1191350632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6805","nonce":1191350632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6809","nonce":1191350632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":1191350632},{"type":"v1","addr":"172.21.15.12:6807","nonce":1191350632}]},"public_addr":"172.21.15.12:6803/1191350632","cluster_addr":"172.21.15.12:6805/1191350632","heartbeat_back_addr":"172.21.15.12:6809/1191350632","heartbeat_front_addr":"172.21.15.12:6807/1191350632","state":["exists","up"]},{"osd":1,"uuid":"1499c783-2bd7-48ce-a444-34add42bcf31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6801","nonce":417496465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6803","nonce":417496465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6807","nonce":417496465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":417496465},{"type":"v1","addr":"172.21.15.111:6805","nonce":417496465}]},"public_addr":"172.21.15.111:6801/417496465","cluster_addr":"172.21.15.111:6803/417496465","heartbeat_back_addr":"172.21.15.111:6807/417496465","heartbeat_front_addr":"172.21.15.111:6805/417496465","state":["exists","up"]},{"osd":2,"uuid":"71a9ebc3-c622-434e-a6c2-f48358f384f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6801","nonce":1403153998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6803","nonce":1403153998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6807","nonce":1403153998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1403153998},{"type":"v1","addr":"172.21.15.134:6805","nonce":1403153998}]},"public_addr":"172.21.15.134:6801/1403153998","cluster_addr":"172.21.15.134:6803/1403153998","heartbeat_back_addr":"172.21.15.134:6807/1403153998","heartbeat_front_addr":"172.21.15.134:6805/1403153998","state":["exists","up"]},{"osd":3,"uuid":"6dd00db2-a286-48ba-8666-56ffdd5f35ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6811","nonce":3359066681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6813","nonce":3359066681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6817","nonce":3359066681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3359066681},{"type":"v1","addr":"172.21.15.12:6815","nonce":3359066681}]},"public_addr":"172.21.15.12:6811/3359066681","cluster_addr":"172.21.15.12:6813/3359066681","heartbeat_back_addr":"172.21.15.12:6817/3359066681","heartbeat_front_addr":"172.21.15.12:6815/3359066681","state":["exists","up"]},{"osd":4,"uuid":"1e29afd7-354c-4506-aaa0-b43eb1a19a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6809","nonce":2886349872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6811","nonce":2886349872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6815","nonce":2886349872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2886349872},{"type":"v1","addr":"172.21.15.111:6813","nonce":2886349872}]},"public_addr":"172.21.15.111:6809/2886349872","cluster_addr":"172.21.15.111:6811/2886349872","heartbeat_back_addr":"172.21.15.111:6815/2886349872","heartbeat_front_addr":"172.21.15.111:6813/2886349872","state":["exists","up"]},{"osd":5,"uuid":"8f40f02d-0ba2-4842-a03d-2a1a04d1010d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6809","nonce":3071977602}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6811","nonce":3071977602}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6815","nonce":3071977602}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":3071977602},{"type":"v1","addr":"172.21.15.134:6813","nonce":3071977602}]},"public_addr":"172.21.15.134:6809/3071977602","cluster_addr":"172.21.15.134:6811/3071977602","heartbeat_back_addr":"172.21.15.134:6815/3071977602","heartbeat_front_addr":"172.21.15.134:6813/3071977602","state":["exists","up"]},{"osd":6,"uuid":"bb0c17a9-6a8e-4be6-8660-f73cfa8200ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6819","nonce":3819619299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6821","nonce":3819619299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6825","nonce":3819619299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":3819619299},{"type":"v1","addr":"172.21.15.12:6823","nonce":3819619299}]},"public_addr":"172.21.15.12:6819/3819619299","cluster_addr":"172.21.15.12:6821/3819619299","heartbeat_back_addr":"172.21.15.12:6825/3819619299","heartbeat_front_addr":"172.21.15.12:6823/3819619299","state":["exists","up"]},{"osd":7,"uuid":"0a93d6af-569e-4e45-b697-9e978c1c1004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6817","nonce":3113856282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6819","nonce":3113856282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6823","nonce":3113856282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":3113856282},{"type":"v1","addr":"172.21.15.111:6821","nonce":3113856282}]},"public_addr":"172.21.15.111:6817/3113856282","cluster_addr":"172.21.15.111:6819/3113856282","heartbeat_back_addr":"172.21.15.111:6823/3113856282","heartbeat_front_addr":"172.21.15.111:6821/3113856282","state":["exists","up"]},{"osd":8,"uuid":"e21acc68-cf50-4f4f-b1ba-cf6c195b4955","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6817","nonce":1689362191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6819","nonce":1689362191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6823","nonce":1689362191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1689362191},{"type":"v1","addr":"172.21.15.134:6821","nonce":1689362191}]},"public_addr":"172.21.15.134:6817/1689362191","cluster_addr":"172.21.15.134:6819/1689362191","heartbeat_back_addr":"172.21.15.134:6823/1689362191","heartbeat_front_addr":"172.21.15.134:6821/1689362191","state":["exists","up"]},{"osd":9,"uuid":"fd745869-7f5f-48a8-b412-bb9c92b586e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6827","nonce":4292385779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6829","nonce":4292385779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6832","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6833","nonce":4292385779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":4292385779},{"type":"v1","addr":"172.21.15.12:6831","nonce":4292385779}]},"public_addr":"172.21.15.12:6827/4292385779","cluster_addr":"172.21.15.12:6829/4292385779","heartbeat_back_addr":"172.21.15.12:6833/4292385779","heartbeat_front_addr":"172.21.15.12:6831/4292385779","state":["exists","up"]},{"osd":10,"uuid":"798b1abd-f6c7-4a5f-8da0-fdae49e50d2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6825","nonce":3338566976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6827","nonce":3338566976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6831","nonce":3338566976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3338566976},{"type":"v1","addr":"172.21.15.111:6829","nonce":3338566976}]},"public_addr":"172.21.15.111:6825/3338566976","cluster_addr":"172.21.15.111:6827/3338566976","heartbeat_back_addr":"172.21.15.111:6831/3338566976","heartbeat_front_addr":"172.21.15.111:6829/3338566976","state":["exists","up"]},{"osd":11,"uuid":"b18fc5f3-1049-4871-b060-e99290273cfb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6825","nonce":578640428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6827","nonce":578640428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6831","nonce":578640428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":578640428},{"type":"v1","addr":"172.21.15.134:6829","nonce":578640428}]},"public_addr":"172.21.15.134:6825/578640428","cluster_addr":"172.21.15.134:6827/578640428","heartbeat_back_addr":"172.21.15.134:6831/578640428","heartbeat_front_addr":"172.21.15.134:6829/578640428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:16:40.828608+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:05.052528+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:28.297316+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:17:53.496497+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:17.947259+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:18:41.712894+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:07.105945+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:32.129688+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:19:56.008516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:22.239867+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:20:47.334692+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T13:21:12.301276+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.12:0/3535299028":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/2837479598":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/3010129696":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/2593601362":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/2519382069":"2024-04-23T13:15:04.279990+0000","172.21.15.12:6801/2009831940":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/856202072":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/3509412187":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/1802873236":"2024-04-23T13:24:06.155564+0000","172.21.15.12:0/1007749062":"2024-04-23T13:15:04.279990+0000","172.21.15.12:0/3588480095":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:0/1904962176":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6801/2091570832":"2024-04-23T13:14:36.603653+0000","172.21.15.12:6800/1802873236":"2024-04-23T13:24:06.155564+0000","172.21.15.12:6800/2009831940":"2024-04-23T13:15:04.279990+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T13:35:16.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:16 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1334316782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:35:16.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:16 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2445042290' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:35:16.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.931+0000 7fc5cdb32640 1 -- 172.21.15.12:0/2377969895 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc5b00a3dd0 msgr2=0x7fc5b00a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:16.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.931+0000 7fc5cdb32640 1 --2- 172.21.15.12:0/2377969895 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc5b00a3dd0 0x7fc5b00a6290 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8006010 tx=0x7fc5b8005e40 comp rx=0 tx=0).stop 2024-04-22T13:35:16.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.931+0000 7fc5cdb32640 1 -- 172.21.15.12:0/2377969895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5c8074cd0 msgr2=0x7fc5c81a6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:16.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.931+0000 7fc5cdb32640 1 --2- 172.21.15.12:0/2377969895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5c8074cd0 0x7fc5c81a6960 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fc5c000b970 tx=0x7fc5c000be30 comp rx=0 tx=0).stop 2024-04-22T13:35:16.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.932+0000 7fc5cdb32640 1 -- 172.21.15.12:0/2377969895 shutdown_connections 2024-04-22T13:35:16.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.932+0000 7fc5cdb32640 1 --2- 172.21.15.12:0/2377969895 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fc5b00a3dd0 0x7fc5b00a6290 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:16.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.932+0000 7fc5cdb32640 1 --2- 172.21.15.12:0/2377969895 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc5c8074cd0 0x7fc5c81a6960 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:16.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.932+0000 7fc5cdb32640 1 --2- 172.21.15.12:0/2377969895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5c8107250 0x7fc5c8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:16.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.932+0000 7fc5cdb32640 1 --2- 172.21.15.12:0/2377969895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5c8102180 0x7fc5c8074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:16.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.932+0000 7fc5cdb32640 1 -- 172.21.15.12:0/2377969895 >> 172.21.15.12:0/2377969895 conn(0x7fc5c8079350 msgr2=0x7fc5c8100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:16.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.932+0000 7fc5cdb32640 1 -- 172.21.15.12:0/2377969895 shutdown_connections 2024-04-22T13:35:16.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:16.932+0000 7fc5cdb32640 1 -- 172.21.15.12:0/2377969895 wait complete. 2024-04-22T13:35:16.937 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:16 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1334316782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:35:16.937 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:16 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2445042290' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:35:16.938 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-22T13:35:16.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:16 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1334316782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:35:16.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:16 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2445042290' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-22T13:35:17.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.046+0000 7fb8d70e4640 1 -- 172.21.15.12:0/2264800833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8d0101a50 msgr2=0x7fb8d0105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:17.043 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.046+0000 7fb8d70e4640 1 --2- 172.21.15.12:0/2264800833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8d0101a50 0x7fb8d0105a90 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009900 tx=0x7fb8b802bc30 comp rx=0 tx=0).stop 2024-04-22T13:35:17.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.046+0000 7fb8d70e4640 1 -- 172.21.15.12:0/2264800833 shutdown_connections 2024-04-22T13:35:17.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.046+0000 7fb8d70e4640 1 --2- 172.21.15.12:0/2264800833 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8d0106170 0x7fb8d010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:17.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.046+0000 7fb8d70e4640 1 --2- 172.21.15.12:0/2264800833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8d0101a50 0x7fb8d0105a90 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:17.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.046+0000 7fb8d70e4640 1 --2- 172.21.15.12:0/2264800833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8d01010a0 0x7fb8d0101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:17.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.046+0000 7fb8d70e4640 1 -- 172.21.15.12:0/2264800833 >> 172.21.15.12:0/2264800833 conn(0x7fb8d0077eb0 msgr2=0x7fb8d00782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:17.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.046+0000 7fb8d70e4640 1 -- 172.21.15.12:0/2264800833 shutdown_connections 2024-04-22T13:35:17.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.046+0000 7fb8d70e4640 1 -- 172.21.15.12:0/2264800833 wait complete. 2024-04-22T13:35:17.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.047+0000 7fb8d70e4640 1 Processor -- start 2024-04-22T13:35:17.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.047+0000 7fb8d70e4640 1 -- start start 2024-04-22T13:35:17.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.047+0000 7fb8d70e4640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8d01010a0 0x7fb8d0198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:17.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.047+0000 7fb8d70e4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8d0106170 0x7fb8d0198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:17.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.047+0000 7fb8d70e4640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8d019d020 0x7fb8d019d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:17.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.047+0000 7fb8d70e4640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d01110b0 con 0x7fb8d01010a0 2024-04-22T13:35:17.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.047+0000 7fb8d70e4640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d0110f30 con 0x7fb8d019d020 2024-04-22T13:35:17.044 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.047+0000 7fb8d70e4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d0111230 con 0x7fb8d0106170 2024-04-22T13:35:17.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.047+0000 7fb8d4e80640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8d01010a0 0x7fb8d0198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:17.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.047+0000 7fb8d4e80640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8d01010a0 0x7fb8d0198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55736/0 (socket says 172.21.15.12:55736) 2024-04-22T13:35:17.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.047+0000 7fb8d4e80640 1 -- 172.21.15.12:0/3945587582 learned_addr learned my addr 172.21.15.12:0/3945587582 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:35:17.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.047+0000 7fb8cffff640 1 --2- 172.21.15.12:0/3945587582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8d0106170 0x7fb8d0198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:17.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.047+0000 7fb8d5681640 1 --2- 172.21.15.12:0/3945587582 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8d019d020 0x7fb8d019d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:17.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.047+0000 7fb8d4e80640 1 -- 172.21.15.12:0/3945587582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8d0106170 msgr2=0x7fb8d0198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:17.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.047+0000 7fb8d4e80640 1 --2- 172.21.15.12:0/3945587582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8d0106170 0x7fb8d0198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:17.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.047+0000 7fb8d4e80640 1 -- 172.21.15.12:0/3945587582 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8d019d020 msgr2=0x7fb8d019d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:17.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.047+0000 7fb8d4e80640 1 --2- 172.21.15.12:0/3945587582 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8d019d020 0x7fb8d019d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:17.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.047+0000 7fb8d4e80640 1 -- 172.21.15.12:0/3945587582 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d019db10 con 0x7fb8d01010a0 2024-04-22T13:35:17.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.048+0000 7fb8d4e80640 1 --2- 172.21.15.12:0/3945587582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8d01010a0 0x7fb8d0198760 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000d770 tx=0x7fb8c000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:17.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.048+0000 7fb8cdffb640 1 -- 172.21.15.12:0/3945587582 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0017070 con 0x7fb8d01010a0 2024-04-22T13:35:17.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.048+0000 7fb8cdffb640 1 -- 172.21.15.12:0/3945587582 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8c00054d0 con 0x7fb8d01010a0 2024-04-22T13:35:17.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.048+0000 7fb8d70e4640 1 -- 172.21.15.12:0/3945587582 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d019de00 con 0x7fb8d01010a0 2024-04-22T13:35:17.045 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.048+0000 7fb8cdffb640 1 -- 172.21.15.12:0/3945587582 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0004c30 con 0x7fb8d01010a0 2024-04-22T13:35:17.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.048+0000 7fb8d70e4640 1 -- 172.21.15.12:0/3945587582 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d01a55c0 con 0x7fb8d01010a0 2024-04-22T13:35:17.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.049+0000 7fb8d70e4640 1 -- 172.21.15.12:0/3945587582 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8d00691c0 con 0x7fb8d01010a0 2024-04-22T13:35:17.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.050+0000 7fb8cdffb640 1 -- 172.21.15.12:0/3945587582 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb8c0004e10 con 0x7fb8d01010a0 2024-04-22T13:35:17.046 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.051+0000 7fb8cdffb640 1 --2- 172.21.15.12:0/3945587582 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb8bc0a3da0 0x7fb8bc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:17.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.051+0000 7fb8cffff640 1 --2- 172.21.15.12:0/3945587582 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb8bc0a3da0 0x7fb8bc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:17.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.051+0000 7fb8cdffb640 1 -- 172.21.15.12:0/3945587582 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8484+0+0 (secure 0 0 0) 0x7fb8c00d0290 con 0x7fb8d01010a0 2024-04-22T13:35:17.047 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.051+0000 7fb8cffff640 1 --2- 172.21.15.12:0/3945587582 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb8bc0a3da0 0x7fb8bc0a6260 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009230 tx=0x7fb8b8005fd0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:17.050 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.055+0000 7fb8cdffb640 1 -- 172.21.15.12:0/3945587582 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb8c009c8f0 con 0x7fb8d01010a0 2024-04-22T13:35:17.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.284+0000 7fb8d70e4640 1 -- 172.21.15.12:0/3945587582 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb8d0101a50 con 0x7fb8d01010a0 2024-04-22T13:35:17.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.286+0000 7fb8cdffb640 1 -- 172.21.15.12:0/3945587582 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) ==== 161+0+0 (secure 0 0 0) 0x7fb8c009c1f0 con 0x7fb8d01010a0 2024-04-22T13:35:17.281 INFO:teuthology.orchestra.run.smithi012.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-22T13:35:17.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.289+0000 7fb8d70e4640 1 -- 172.21.15.12:0/3945587582 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb8bc0a3da0 msgr2=0x7fb8bc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:17.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.289+0000 7fb8d70e4640 1 --2- 172.21.15.12:0/3945587582 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb8bc0a3da0 0x7fb8bc0a6260 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8009230 tx=0x7fb8b8005fd0 comp rx=0 tx=0).stop 2024-04-22T13:35:17.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.289+0000 7fb8d70e4640 1 -- 172.21.15.12:0/3945587582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8d01010a0 msgr2=0x7fb8d0198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:17.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.289+0000 7fb8d70e4640 1 --2- 172.21.15.12:0/3945587582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8d01010a0 0x7fb8d0198760 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000d770 tx=0x7fb8c000dc30 comp rx=0 tx=0).stop 2024-04-22T13:35:17.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.289+0000 7fb8d70e4640 1 -- 172.21.15.12:0/3945587582 shutdown_connections 2024-04-22T13:35:17.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.289+0000 7fb8d70e4640 1 --2- 172.21.15.12:0/3945587582 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fb8bc0a3da0 0x7fb8bc0a6260 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:17.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.289+0000 7fb8d70e4640 1 --2- 172.21.15.12:0/3945587582 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8d019d020 0x7fb8d019d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:17.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.289+0000 7fb8d70e4640 1 --2- 172.21.15.12:0/3945587582 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8d0106170 0x7fb8d0198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:17.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.289+0000 7fb8d70e4640 1 --2- 172.21.15.12:0/3945587582 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb8d01010a0 0x7fb8d0198760 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:17.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.289+0000 7fb8d70e4640 1 -- 172.21.15.12:0/3945587582 >> 172.21.15.12:0/3945587582 conn(0x7fb8d0077eb0 msgr2=0x7fb8d006a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:17.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.290+0000 7fb8d70e4640 1 -- 172.21.15.12:0/3945587582 shutdown_connections 2024-04-22T13:35:17.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.290+0000 7fb8d70e4640 1 -- 172.21.15.12:0/3945587582 wait complete. 2024-04-22T13:35:17.296 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-22T13:35:17.407 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.409+0000 7fa5742d2640 1 -- 172.21.15.12:0/2130222624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa56c107270 msgr2=0x7fa56c10ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:17.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.409+0000 7fa5742d2640 1 --2- 172.21.15.12:0/2130222624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa56c107270 0x7fa56c10ee40 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fa564009ca0 tx=0x7fa56402c0f0 comp rx=0 tx=0).stop 2024-04-22T13:35:17.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.409+0000 7fa5742d2640 1 -- 172.21.15.12:0/2130222624 shutdown_connections 2024-04-22T13:35:17.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.409+0000 7fa5742d2640 1 --2- 172.21.15.12:0/2130222624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa56c107270 0x7fa56c10ee40 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:17.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.409+0000 7fa5742d2640 1 --2- 172.21.15.12:0/2130222624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa56c102b50 0x7fa56c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:17.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.409+0000 7fa5742d2640 1 --2- 172.21.15.12:0/2130222624 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa56c1021a0 0x7fa56c102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:17.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.409+0000 7fa5742d2640 1 -- 172.21.15.12:0/2130222624 >> 172.21.15.12:0/2130222624 conn(0x7fa56c0fda40 msgr2=0x7fa56c0ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:17.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.410+0000 7fa5742d2640 1 -- 172.21.15.12:0/2130222624 shutdown_connections 2024-04-22T13:35:17.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.410+0000 7fa5742d2640 1 -- 172.21.15.12:0/2130222624 wait complete. 2024-04-22T13:35:17.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.410+0000 7fa5742d2640 1 Processor -- start 2024-04-22T13:35:17.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.410+0000 7fa5742d2640 1 -- start start 2024-04-22T13:35:17.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.410+0000 7fa5742d2640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa56c1021a0 0x7fa56c1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:17.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.410+0000 7fa5742d2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa56c107270 0x7fa56c1a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:17.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.410+0000 7fa5742d2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa56c19a7a0 0x7fa56c19ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:17.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.410+0000 7fa5742d2640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c1121e0 con 0x7fa56c1021a0 2024-04-22T13:35:17.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.410+0000 7fa5742d2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c112060 con 0x7fa56c19a7a0 2024-04-22T13:35:17.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.410+0000 7fa5742d2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c112360 con 0x7fa56c107270 2024-04-22T13:35:17.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.410+0000 7fa57206e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa56c1021a0 0x7fa56c1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:17.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.410+0000 7fa57206e640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa56c1021a0 0x7fa56c1a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55784/0 (socket says 172.21.15.12:55784) 2024-04-22T13:35:17.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.410+0000 7fa57206e640 1 -- 172.21.15.12:0/4054133784 learned_addr learned my addr 172.21.15.12:0/4054133784 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:35:17.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.411+0000 7fa57286f640 1 --2- 172.21.15.12:0/4054133784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa56c19a7a0 0x7fa56c19ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:17.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.411+0000 7fa57186d640 1 --2- 172.21.15.12:0/4054133784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa56c107270 0x7fa56c1a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:17.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.411+0000 7fa57206e640 1 -- 172.21.15.12:0/4054133784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa56c107270 msgr2=0x7fa56c1a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:17.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.411+0000 7fa57206e640 1 --2- 172.21.15.12:0/4054133784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa56c107270 0x7fa56c1a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:17.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.411+0000 7fa57206e640 1 -- 172.21.15.12:0/4054133784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa56c19a7a0 msgr2=0x7fa56c19ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:17.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.411+0000 7fa57206e640 1 --2- 172.21.15.12:0/4054133784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa56c19a7a0 0x7fa56c19ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:17.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.411+0000 7fa57206e640 1 -- 172.21.15.12:0/4054133784 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56c19b3a0 con 0x7fa56c1021a0 2024-04-22T13:35:17.409 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.411+0000 7fa57206e640 1 --2- 172.21.15.12:0/4054133784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa56c1021a0 0x7fa56c1a0690 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fa56800e9f0 tx=0x7fa56800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:17.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.411+0000 7fa5637fe640 1 -- 172.21.15.12:0/4054133784 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56800cd80 con 0x7fa56c1021a0 2024-04-22T13:35:17.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.411+0000 7fa5742d2640 1 -- 172.21.15.12:0/4054133784 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa56c1a7430 con 0x7fa56c1021a0 2024-04-22T13:35:17.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.411+0000 7fa5637fe640 1 -- 172.21.15.12:0/4054133784 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5680108e0 con 0x7fa56c1021a0 2024-04-22T13:35:17.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.411+0000 7fa5637fe640 1 -- 172.21.15.12:0/4054133784 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa568018980 con 0x7fa56c1021a0 2024-04-22T13:35:17.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.411+0000 7fa5742d2640 1 -- 172.21.15.12:0/4054133784 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa56c1a7890 con 0x7fa56c1021a0 2024-04-22T13:35:17.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.412+0000 7fa5742d2640 1 -- 172.21.15.12:0/4054133784 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa56c103670 con 0x7fa56c1021a0 2024-04-22T13:35:17.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.413+0000 7fa5637fe640 1 -- 172.21.15.12:0/4054133784 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa568010430 con 0x7fa56c1021a0 2024-04-22T13:35:17.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.414+0000 7fa5637fe640 1 --2- 172.21.15.12:0/4054133784 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa5580a3d20 0x7fa5580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:17.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.414+0000 7fa57186d640 1 --2- 172.21.15.12:0/4054133784 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa5580a3d20 0x7fa5580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:17.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.414+0000 7fa5637fe640 1 -- 172.21.15.12:0/4054133784 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8870+0+0 (secure 0 0 0) 0x7fa5680d1150 con 0x7fa56c1021a0 2024-04-22T13:35:17.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.414+0000 7fa57186d640 1 --2- 172.21.15.12:0/4054133784 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa5580a3d20 0x7fa5580a61e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa56c19bfa0 tx=0x7fa55c005f90 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:17.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.417+0000 7fa5637fe640 1 -- 172.21.15.12:0/4054133784 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa56809cfe0 con 0x7fa56c1021a0 2024-04-22T13:35:17.586 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:17 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mon-a[28261]: 2024-04-22T13:35:17.284+0000 7fe07fc48700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-22T13:35:17.586 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 22 13:35:17 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-a[70376]: ignoring --setuser ceph since I am not root 2024-04-22T13:35:17.586 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 22 13:35:17 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-a[70376]: ignoring --setgroup ceph since I am not root 2024-04-22T13:35:17.586 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 22 13:35:17 smithi012 ceph-mds[70380]: -- 172.21.15.12:0/905633612 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5613f46cba40 con 0x5613f469e800 2024-04-22T13:35:17.587 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 22 13:35:17 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-a[70376]: starting mds.a at 2024-04-22T13:35:17.587 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 22 13:35:17 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-g[69989]: ignoring --setuser ceph since I am not root 2024-04-22T13:35:17.587 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 22 13:35:17 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-g[69989]: ignoring --setgroup ceph since I am not root 2024-04-22T13:35:17.587 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 22 13:35:17 smithi012 ceph-mds[70015]: -- 172.21.15.12:0/3182925852 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c8da0b24e0 con 0x55c8da089000 2024-04-22T13:35:17.587 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 22 13:35:17 smithi012 ceph-mds[70015]: -- 172.21.15.12:0/3182925852 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c8da0b5a40 con 0x55c8da089000 2024-04-22T13:35:17.587 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 22 13:35:17 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-g[69989]: starting mds.g at 2024-04-22T13:35:17.641 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:17.646+0000 7fa5742d2640 1 -- 172.21.15.12:0/4054133784 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa56c102be0 con 0x7fa56c1021a0 2024-04-22T13:35:17.647 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 22 13:35:17 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-e[61566]: ignoring --setuser ceph since I am not root 2024-04-22T13:35:17.647 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 22 13:35:17 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-e[61566]: ignoring --setgroup ceph since I am not root 2024-04-22T13:35:17.647 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 22 13:35:17 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-e[61566]: starting mds.e at 2024-04-22T13:35:17.648 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 22 13:35:17 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-c[62477]: ignoring --setuser ceph since I am not root 2024-04-22T13:35:17.648 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 22 13:35:17 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-c[62477]: ignoring --setgroup ceph since I am not root 2024-04-22T13:35:17.648 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 22 13:35:17 smithi134 ceph-mds[62481]: -- 172.21.15.134:0/500613698 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560ff2b21a40 con 0x560ff2af5000 2024-04-22T13:35:17.648 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 22 13:35:17 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-c[62477]: starting mds.c at 2024-04-22T13:35:17.648 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 22 13:35:17 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-b[62191]: ignoring --setuser ceph since I am not root 2024-04-22T13:35:17.648 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 22 13:35:17 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-b[62191]: ignoring --setgroup ceph since I am not root 2024-04-22T13:35:17.648 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 22 13:35:17 smithi111 ceph-mds[62195]: -- 172.21.15.111:0/4184717312 <== mon.0 v2:172.21.15.12:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d8e54c7a40 con 0x55d8e549b000 2024-04-22T13:35:17.648 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 22 13:35:17 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-b[62191]: starting mds.b at 2024-04-22T13:35:17.649 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 22 13:35:17 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-f[62166]: ignoring --setuser ceph since I am not root 2024-04-22T13:35:17.649 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 22 13:35:17 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-f[62166]: ignoring --setgroup ceph since I am not root 2024-04-22T13:35:17.649 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 22 13:35:17 smithi134 ceph-mds[62170]: -- 172.21.15.134:0/3220246030 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f426e67a40 con 0x55f426e3a800 2024-04-22T13:35:17.649 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 22 13:35:17 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-f[62166]: starting mds.f at 2024-04-22T13:35:17.649 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 22 13:35:17 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-h[61871]: ignoring --setuser ceph since I am not root 2024-04-22T13:35:17.649 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 22 13:35:17 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-h[61871]: ignoring --setgroup ceph since I am not root 2024-04-22T13:35:17.649 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 22 13:35:17 smithi111 ceph-mds[61875]: -- 172.21.15.111:0/3423570061 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5567a761da40 con 0x5567a75f0800 2024-04-22T13:35:17.649 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 22 13:35:17 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-h[61871]: starting mds.h at 2024-04-22T13:35:17.650 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 22 13:35:17 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-l[61858]: ignoring --setuser ceph since I am not root 2024-04-22T13:35:17.650 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 22 13:35:17 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-l[61858]: ignoring --setgroup ceph since I am not root 2024-04-22T13:35:17.650 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 22 13:35:17 smithi134 ceph-mds[61862]: -- 172.21.15.134:0/1374610410 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560ed7a804e0 con 0x560ed7a56800 2024-04-22T13:35:17.650 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 22 13:35:17 smithi134 ceph-mds[61862]: -- 172.21.15.134:0/1374610410 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560ed7a83a40 con 0x560ed7a56800 2024-04-22T13:35:17.650 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 22 13:35:17 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-l[61858]: starting mds.l at 2024-04-22T13:35:17.650 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 22 13:35:17 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-k[61243]: ignoring --setuser ceph since I am not root 2024-04-22T13:35:17.650 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 22 13:35:17 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-k[61243]: ignoring --setgroup ceph since I am not root 2024-04-22T13:35:17.650 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 22 13:35:17 smithi111 ceph-mds[61247]: -- 172.21.15.111:0/3791751745 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e4168fda40 con 0x55e4168d0c00 2024-04-22T13:35:17.650 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 22 13:35:17 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-k[61243]: starting mds.k at 2024-04-22T13:35:17.650 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 22 13:35:17 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-i[62801]: ignoring --setuser ceph since I am not root 2024-04-22T13:35:17.650 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 22 13:35:17 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-i[62801]: ignoring --setgroup ceph since I am not root 2024-04-22T13:35:17.650 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 22 13:35:17 smithi134 ceph-mds[62805]: -- 172.21.15.134:0/439096206 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5651ea54c4e0 con 0x5651ea522800 2024-04-22T13:35:17.651 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 22 13:35:17 smithi134 ceph-mds[62805]: -- 172.21.15.134:0/439096206 <== mon.1 v2:172.21.15.134:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5651ea54fa40 con 0x5651ea522800 2024-04-22T13:35:17.651 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 22 13:35:17 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-i[62801]: starting mds.i at 2024-04-22T13:35:17.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:17 smithi134 ceph-mon[31338]: pgmap v348: 97 pgs: 97 active+clean; 2.3 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-22T13:35:17.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:17 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2377969895' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:35:17.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:17 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/3945587582' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-22T13:35:17.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:17 smithi134 ceph-mon[31338]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-22T13:35:17.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:17 smithi134 ceph-mon[31338]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-22T13:35:17.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:17 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/3945587582' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-22T13:35:17.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:17 smithi134 ceph-mon[31338]: osdmap e101: 12 total, 12 up, 12 in 2024-04-22T13:35:17.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:17 smithi134 ceph-mon[31338]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-22T13:35:17.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:17 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/4054133784' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-22T13:35:17.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:17 smithi111 ceph-mon[31361]: pgmap v348: 97 pgs: 97 active+clean; 2.3 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-22T13:35:17.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:17 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2377969895' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:35:17.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:17 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/3945587582' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-22T13:35:17.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:17 smithi111 ceph-mon[31361]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-22T13:35:17.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:17 smithi111 ceph-mon[31361]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-22T13:35:17.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:17 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/3945587582' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-22T13:35:17.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:17 smithi111 ceph-mon[31361]: osdmap e101: 12 total, 12 up, 12 in 2024-04-22T13:35:17.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:17 smithi111 ceph-mon[31361]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-22T13:35:17.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:17 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/4054133784' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-22T13:35:18.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:17 smithi012 ceph-mon[28265]: pgmap v348: 97 pgs: 97 active+clean; 2.3 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-22T13:35:18.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:17 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2377969895' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-22T13:35:18.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:17 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3945587582' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-22T13:35:18.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:17 smithi012 ceph-mon[28265]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-22T13:35:18.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:17 smithi012 ceph-mon[28265]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-22T13:35:18.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:17 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/3945587582' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-22T13:35:18.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:17 smithi012 ceph-mon[28265]: osdmap e101: 12 total, 12 up, 12 in 2024-04-22T13:35:18.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:17 smithi012 ceph-mon[28265]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-22T13:35:18.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:17 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/4054133784' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-22T13:35:18.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.288+0000 7fa5637fe640 1 -- 172.21.15.12:0/4054133784 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v30) ==== 104+0+0 (secure 0 0 0) 0x7fa5680a12e0 con 0x7fa56c1021a0 2024-04-22T13:35:18.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.291+0000 7fa5742d2640 1 -- 172.21.15.12:0/4054133784 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa5580a3d20 msgr2=0x7fa5580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:18.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.291+0000 7fa5742d2640 1 --2- 172.21.15.12:0/4054133784 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa5580a3d20 0x7fa5580a61e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa56c19bfa0 tx=0x7fa55c005f90 comp rx=0 tx=0).stop 2024-04-22T13:35:18.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.291+0000 7fa5742d2640 1 -- 172.21.15.12:0/4054133784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa56c1021a0 msgr2=0x7fa56c1a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:18.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.291+0000 7fa5742d2640 1 --2- 172.21.15.12:0/4054133784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa56c1021a0 0x7fa56c1a0690 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fa56800e9f0 tx=0x7fa56800eeb0 comp rx=0 tx=0).stop 2024-04-22T13:35:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.291+0000 7fa5742d2640 1 -- 172.21.15.12:0/4054133784 shutdown_connections 2024-04-22T13:35:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.291+0000 7fa5742d2640 1 --2- 172.21.15.12:0/4054133784 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7fa5580a3d20 0x7fa5580a61e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.291+0000 7fa5742d2640 1 --2- 172.21.15.12:0/4054133784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa56c19a7a0 0x7fa56c19ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.291+0000 7fa5742d2640 1 --2- 172.21.15.12:0/4054133784 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa56c107270 0x7fa56c1a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.291+0000 7fa5742d2640 1 --2- 172.21.15.12:0/4054133784 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fa56c1021a0 0x7fa56c1a0690 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.291+0000 7fa5742d2640 1 -- 172.21.15.12:0/4054133784 >> 172.21.15.12:0/4054133784 conn(0x7fa56c0fda40 msgr2=0x7fa56c0ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.291+0000 7fa5742d2640 1 -- 172.21.15.12:0/4054133784 shutdown_connections 2024-04-22T13:35:18.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.291+0000 7fa5742d2640 1 -- 172.21.15.12:0/4054133784 wait complete. 2024-04-22T13:35:18.305 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-22T13:35:18.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.423+0000 7f511fe48640 1 -- 172.21.15.12:0/2662413401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5118074300 msgr2=0x7f51180746e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:18.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.423+0000 7f511fe48640 1 --2- 172.21.15.12:0/2662413401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5118074300 0x7f51180746e0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f5110009900 tx=0x7f511002bcd0 comp rx=0 tx=0).stop 2024-04-22T13:35:18.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.424+0000 7f511fe48640 1 -- 172.21.15.12:0/2662413401 shutdown_connections 2024-04-22T13:35:18.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.424+0000 7f511fe48640 1 --2- 172.21.15.12:0/2662413401 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f511810e7f0 0x7f5118110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:18.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.424+0000 7f511fe48640 1 --2- 172.21.15.12:0/2662413401 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5118074c20 0x7f511810e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:18.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.424+0000 7f511fe48640 1 --2- 172.21.15.12:0/2662413401 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5118074300 0x7f51180746e0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:18.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.424+0000 7f511fe48640 1 -- 172.21.15.12:0/2662413401 >> 172.21.15.12:0/2662413401 conn(0x7f51180fda40 msgr2=0x7f51180ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:18.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.424+0000 7f511fe48640 1 -- 172.21.15.12:0/2662413401 shutdown_connections 2024-04-22T13:35:18.420 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.424+0000 7f511fe48640 1 -- 172.21.15.12:0/2662413401 wait complete. 2024-04-22T13:35:18.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.424+0000 7f511fe48640 1 Processor -- start 2024-04-22T13:35:18.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.424+0000 7f511fe48640 1 -- start start 2024-04-22T13:35:18.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.425+0000 7f511fe48640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5118074300 0x7f51181a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:18.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.425+0000 7f511fe48640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5118074c20 0x7f51181a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:18.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.425+0000 7f511fe48640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f511810e7f0 0x7f511819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:18.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.425+0000 7f511fe48640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f51181143b0 con 0x7f511810e7f0 2024-04-22T13:35:18.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.425+0000 7f511fe48640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5118114230 con 0x7f5118074300 2024-04-22T13:35:18.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.425+0000 7f511fe48640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5118114530 con 0x7f5118074c20 2024-04-22T13:35:18.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.425+0000 7f511e3e5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f511810e7f0 0x7f511819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:18.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.425+0000 7f511e3e5640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f511810e7f0 0x7f511819a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55804/0 (socket says 172.21.15.12:55804) 2024-04-22T13:35:18.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.425+0000 7f511e3e5640 1 -- 172.21.15.12:0/2888149409 learned_addr learned my addr 172.21.15.12:0/2888149409 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:35:18.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.425+0000 7f511dbe4640 1 --2- 172.21.15.12:0/2888149409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5118074300 0x7f51181a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:18.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.425+0000 7f511d3e3640 1 --2- 172.21.15.12:0/2888149409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5118074c20 0x7f51181a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:18.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.425+0000 7f511e3e5640 1 -- 172.21.15.12:0/2888149409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5118074c20 msgr2=0x7f51181a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:18.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.425+0000 7f511e3e5640 1 --2- 172.21.15.12:0/2888149409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5118074c20 0x7f51181a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:18.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.425+0000 7f511e3e5640 1 -- 172.21.15.12:0/2888149409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5118074300 msgr2=0x7f51181a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:18.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.425+0000 7f511e3e5640 1 --2- 172.21.15.12:0/2888149409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5118074300 0x7f51181a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:18.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.425+0000 7f511e3e5640 1 -- 172.21.15.12:0/2888149409 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f511819b030 con 0x7f511810e7f0 2024-04-22T13:35:18.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.425+0000 7f511e3e5640 1 --2- 172.21.15.12:0/2888149409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f511810e7f0 0x7f511819a770 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f510800eaa0 tx=0x7f510800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:18.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.425+0000 7f5116ffd640 1 -- 172.21.15.12:0/2888149409 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510800cdb0 con 0x7f511810e7f0 2024-04-22T13:35:18.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.425+0000 7f5116ffd640 1 -- 172.21.15.12:0/2888149409 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5108005020 con 0x7f511810e7f0 2024-04-22T13:35:18.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.425+0000 7f511fe48640 1 -- 172.21.15.12:0/2888149409 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f511819b320 con 0x7f511810e7f0 2024-04-22T13:35:18.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.425+0000 7f5116ffd640 1 -- 172.21.15.12:0/2888149409 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51080116d0 con 0x7f511810e7f0 2024-04-22T13:35:18.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.425+0000 7f511fe48640 1 -- 172.21.15.12:0/2888149409 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51181a7630 con 0x7f511810e7f0 2024-04-22T13:35:18.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.427+0000 7f511fe48640 1 -- 172.21.15.12:0/2888149409 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51180759b0 con 0x7f511810e7f0 2024-04-22T13:35:18.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.427+0000 7f5116ffd640 1 -- 172.21.15.12:0/2888149409 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f51080051d0 con 0x7f511810e7f0 2024-04-22T13:35:18.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.428+0000 7f5116ffd640 1 --2- 172.21.15.12:0/2888149409 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f51000a3d20 0x7f51000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:18.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.428+0000 7f511dbe4640 1 --2- 172.21.15.12:0/2888149409 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f51000a3d20 0x7f51000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:18.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.428+0000 7f5116ffd640 1 -- 172.21.15.12:0/2888149409 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8870+0+0 (secure 0 0 0) 0x7f51080d0a20 con 0x7f511810e7f0 2024-04-22T13:35:18.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.429+0000 7f511dbe4640 1 --2- 172.21.15.12:0/2888149409 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f51000a3d20 0x7f51000a61e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5110006010 tx=0x7f5110005e40 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:18.428 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.433+0000 7f5116ffd640 1 -- 172.21.15.12:0/2888149409 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5108016030 con 0x7f511810e7f0 2024-04-22T13:35:18.616 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:18.621+0000 7f511fe48640 1 -- 172.21.15.12:0/2888149409 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5118109970 con 0x7f511810e7f0 2024-04-22T13:35:18.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:18 smithi134 ceph-mon[31338]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-22T13:35:18.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:18 smithi134 ceph-mon[31338]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-22T13:35:18.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:18 smithi134 ceph-mon[31338]: Cluster is now healthy 2024-04-22T13:35:18.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:18 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/4054133784' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-22T13:35:18.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:18 smithi134 ceph-mon[31338]: mds.? [v2:172.21.15.12:6838/1265470917,v1:172.21.15.12:6840/1265470917] up:boot 2024-04-22T13:35:18.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:18 smithi134 ceph-mon[31338]: mds.? [v2:172.21.15.12:6839/2927111531,v1:172.21.15.12:6841/2927111531] up:boot 2024-04-22T13:35:18.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:18 smithi134 ceph-mon[31338]: mds.? [v2:172.21.15.111:6832/3294346577,v1:172.21.15.111:6833/3294346577] up:boot 2024-04-22T13:35:18.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:18 smithi134 ceph-mon[31338]: mds.? [v2:172.21.15.134:6833/1706198213,v1:172.21.15.134:6837/1706198213] up:boot 2024-04-22T13:35:18.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:18 smithi134 ceph-mon[31338]: mds.? [v2:172.21.15.134:6832/974187404,v1:172.21.15.134:6836/974187404] up:boot 2024-04-22T13:35:18.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:18 smithi134 ceph-mon[31338]: mds.? [v2:172.21.15.134:6835/2495853212,v1:172.21.15.134:6838/2495853212] up:boot 2024-04-22T13:35:18.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:18 smithi134 ceph-mon[31338]: mds.? [v2:172.21.15.134:6834/1527605640,v1:172.21.15.134:6839/1527605640] up:boot 2024-04-22T13:35:18.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:18 smithi134 ceph-mon[31338]: mds.? [v2:172.21.15.111:6834/1428280052,v1:172.21.15.111:6835/1428280052] up:boot 2024-04-22T13:35:18.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:18 smithi134 ceph-mon[31338]: mds.? [v2:172.21.15.111:6836/924862231,v1:172.21.15.111:6837/924862231] up:boot 2024-04-22T13:35:18.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:18 smithi134 ceph-mon[31338]: mds.? [v2:172.21.15.111:6838/291740583,v1:172.21.15.111:6839/291740583] up:boot 2024-04-22T13:35:18.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:18 smithi134 ceph-mon[31338]: fsmap 12 up:standby 2024-04-22T13:35:18.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:18 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-22T13:35:18.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:18 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-22T13:35:18.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:18 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-22T13:35:18.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:18 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-22T13:35:18.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:18 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-22T13:35:18.928 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:18 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-22T13:35:18.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:18 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-22T13:35:18.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:18 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-22T13:35:18.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:18 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T13:35:18.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:18 smithi134 ceph-mon[31338]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-22T13:35:18.929 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:18 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2888149409' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-22T13:35:18.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:18 smithi111 ceph-mon[31361]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-22T13:35:18.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:18 smithi111 ceph-mon[31361]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-22T13:35:18.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:18 smithi111 ceph-mon[31361]: Cluster is now healthy 2024-04-22T13:35:18.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:18 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/4054133784' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-22T13:35:18.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:18 smithi111 ceph-mon[31361]: mds.? [v2:172.21.15.12:6838/1265470917,v1:172.21.15.12:6840/1265470917] up:boot 2024-04-22T13:35:18.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:18 smithi111 ceph-mon[31361]: mds.? [v2:172.21.15.12:6839/2927111531,v1:172.21.15.12:6841/2927111531] up:boot 2024-04-22T13:35:18.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:18 smithi111 ceph-mon[31361]: mds.? [v2:172.21.15.111:6832/3294346577,v1:172.21.15.111:6833/3294346577] up:boot 2024-04-22T13:35:18.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:18 smithi111 ceph-mon[31361]: mds.? [v2:172.21.15.134:6833/1706198213,v1:172.21.15.134:6837/1706198213] up:boot 2024-04-22T13:35:18.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:18 smithi111 ceph-mon[31361]: mds.? [v2:172.21.15.134:6832/974187404,v1:172.21.15.134:6836/974187404] up:boot 2024-04-22T13:35:18.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:18 smithi111 ceph-mon[31361]: mds.? [v2:172.21.15.134:6835/2495853212,v1:172.21.15.134:6838/2495853212] up:boot 2024-04-22T13:35:18.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:18 smithi111 ceph-mon[31361]: mds.? [v2:172.21.15.134:6834/1527605640,v1:172.21.15.134:6839/1527605640] up:boot 2024-04-22T13:35:18.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:18 smithi111 ceph-mon[31361]: mds.? [v2:172.21.15.111:6834/1428280052,v1:172.21.15.111:6835/1428280052] up:boot 2024-04-22T13:35:18.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:18 smithi111 ceph-mon[31361]: mds.? [v2:172.21.15.111:6836/924862231,v1:172.21.15.111:6837/924862231] up:boot 2024-04-22T13:35:18.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:18 smithi111 ceph-mon[31361]: mds.? [v2:172.21.15.111:6838/291740583,v1:172.21.15.111:6839/291740583] up:boot 2024-04-22T13:35:18.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:18 smithi111 ceph-mon[31361]: fsmap 12 up:standby 2024-04-22T13:35:18.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:18 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-22T13:35:18.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:18 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-22T13:35:18.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:18 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-22T13:35:18.977 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:18 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-22T13:35:18.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:18 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-22T13:35:18.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:18 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-22T13:35:18.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:18 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-22T13:35:18.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:18 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-22T13:35:18.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:18 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T13:35:18.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:18 smithi111 ceph-mon[31361]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-22T13:35:18.978 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:18 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2888149409' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-22T13:35:19.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:18 smithi012 ceph-mon[28265]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-22T13:35:19.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:18 smithi012 ceph-mon[28265]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-22T13:35:19.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:18 smithi012 ceph-mon[28265]: Cluster is now healthy 2024-04-22T13:35:19.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:18 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/4054133784' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-22T13:35:19.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:18 smithi012 ceph-mon[28265]: mds.? [v2:172.21.15.12:6838/1265470917,v1:172.21.15.12:6840/1265470917] up:boot 2024-04-22T13:35:19.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:18 smithi012 ceph-mon[28265]: mds.? [v2:172.21.15.12:6839/2927111531,v1:172.21.15.12:6841/2927111531] up:boot 2024-04-22T13:35:19.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:18 smithi012 ceph-mon[28265]: mds.? [v2:172.21.15.111:6832/3294346577,v1:172.21.15.111:6833/3294346577] up:boot 2024-04-22T13:35:19.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:18 smithi012 ceph-mon[28265]: mds.? [v2:172.21.15.134:6833/1706198213,v1:172.21.15.134:6837/1706198213] up:boot 2024-04-22T13:35:19.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:18 smithi012 ceph-mon[28265]: mds.? [v2:172.21.15.134:6832/974187404,v1:172.21.15.134:6836/974187404] up:boot 2024-04-22T13:35:19.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:18 smithi012 ceph-mon[28265]: mds.? [v2:172.21.15.134:6835/2495853212,v1:172.21.15.134:6838/2495853212] up:boot 2024-04-22T13:35:19.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:18 smithi012 ceph-mon[28265]: mds.? [v2:172.21.15.134:6834/1527605640,v1:172.21.15.134:6839/1527605640] up:boot 2024-04-22T13:35:19.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:18 smithi012 ceph-mon[28265]: mds.? [v2:172.21.15.111:6834/1428280052,v1:172.21.15.111:6835/1428280052] up:boot 2024-04-22T13:35:19.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:18 smithi012 ceph-mon[28265]: mds.? [v2:172.21.15.111:6836/924862231,v1:172.21.15.111:6837/924862231] up:boot 2024-04-22T13:35:19.087 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:18 smithi012 ceph-mon[28265]: mds.? [v2:172.21.15.111:6838/291740583,v1:172.21.15.111:6839/291740583] up:boot 2024-04-22T13:35:19.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:18 smithi012 ceph-mon[28265]: fsmap 12 up:standby 2024-04-22T13:35:19.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:18 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-22T13:35:19.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:18 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-22T13:35:19.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:18 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-22T13:35:19.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:18 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-22T13:35:19.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:18 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-22T13:35:19.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:18 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-22T13:35:19.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:18 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-22T13:35:19.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:18 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-22T13:35:19.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:18 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-22T13:35:19.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:18 smithi012 ceph-mon[28265]: from='mgr.24968 172.21.15.12:0/3950955042' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-22T13:35:19.088 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:18 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2888149409' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-22T13:35:19.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.290+0000 7f5116ffd640 1 -- 172.21.15.12:0/2888149409 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v102) ==== 181+0+0 (secure 0 0 0) 0x7f510809c910 con 0x7f511810e7f0 2024-04-22T13:35:19.285 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_metadata' removed 2024-04-22T13:35:19.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.293+0000 7f511fe48640 1 -- 172.21.15.12:0/2888149409 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f51000a3d20 msgr2=0x7f51000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:19.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.293+0000 7f511fe48640 1 --2- 172.21.15.12:0/2888149409 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f51000a3d20 0x7f51000a61e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5110006010 tx=0x7f5110005e40 comp rx=0 tx=0).stop 2024-04-22T13:35:19.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.293+0000 7f511fe48640 1 -- 172.21.15.12:0/2888149409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f511810e7f0 msgr2=0x7f511819a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:19.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.293+0000 7f511fe48640 1 --2- 172.21.15.12:0/2888149409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f511810e7f0 0x7f511819a770 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f510800eaa0 tx=0x7f510800ef60 comp rx=0 tx=0).stop 2024-04-22T13:35:19.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.293+0000 7f511fe48640 1 -- 172.21.15.12:0/2888149409 shutdown_connections 2024-04-22T13:35:19.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.293+0000 7f511fe48640 1 --2- 172.21.15.12:0/2888149409 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f51000a3d20 0x7f51000a61e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:19.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.293+0000 7f511fe48640 1 --2- 172.21.15.12:0/2888149409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f511810e7f0 0x7f511819a770 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:19.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.293+0000 7f511fe48640 1 --2- 172.21.15.12:0/2888149409 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5118074c20 0x7f51181a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:19.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.293+0000 7f511fe48640 1 --2- 172.21.15.12:0/2888149409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5118074300 0x7f51181a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:19.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.293+0000 7f511fe48640 1 -- 172.21.15.12:0/2888149409 >> 172.21.15.12:0/2888149409 conn(0x7f51180fda40 msgr2=0x7f51180ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:19.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.293+0000 7f511fe48640 1 -- 172.21.15.12:0/2888149409 shutdown_connections 2024-04-22T13:35:19.289 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.293+0000 7f511fe48640 1 -- 172.21.15.12:0/2888149409 wait complete. 2024-04-22T13:35:19.306 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-22T13:35:19.418 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.422+0000 7f7f10870640 1 -- 172.21.15.12:0/3190308236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f0c06a840 msgr2=0x7f7f0c06ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:19.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.422+0000 7f7f10870640 1 --2- 172.21.15.12:0/3190308236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f0c06a840 0x7f7f0c06ac20 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009900 tx=0x7f7efc02bc50 comp rx=0 tx=0).stop 2024-04-22T13:35:19.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.423+0000 7f7f10870640 1 -- 172.21.15.12:0/3190308236 shutdown_connections 2024-04-22T13:35:19.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.423+0000 7f7f10870640 1 --2- 172.21.15.12:0/3190308236 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f0c107270 0x7f7f0c10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:19.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.423+0000 7f7f10870640 1 --2- 172.21.15.12:0/3190308236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f0c06b1f0 0x7f7f0c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:19.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.423+0000 7f7f10870640 1 --2- 172.21.15.12:0/3190308236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f0c06a840 0x7f7f0c06ac20 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:19.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.423+0000 7f7f10870640 1 -- 172.21.15.12:0/3190308236 >> 172.21.15.12:0/3190308236 conn(0x7f7f0c077eb0 msgr2=0x7f7f0c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:19.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.423+0000 7f7f10870640 1 -- 172.21.15.12:0/3190308236 shutdown_connections 2024-04-22T13:35:19.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.423+0000 7f7f10870640 1 -- 172.21.15.12:0/3190308236 wait complete. 2024-04-22T13:35:19.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.423+0000 7f7f10870640 1 Processor -- start 2024-04-22T13:35:19.419 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.423+0000 7f7f10870640 1 -- start start 2024-04-22T13:35:19.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.424+0000 7f7f10870640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f0c06a840 0x7f7f0c198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:19.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.424+0000 7f7f10870640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f0c107270 0x7f7f0c198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:19.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.424+0000 7f7f10870640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f0c19d020 0x7f7f0c19d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:19.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.424+0000 7f7f10870640 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0c112280 con 0x7f7f0c19d020 2024-04-22T13:35:19.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.424+0000 7f7f10870640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0c112100 con 0x7f7f0c107270 2024-04-22T13:35:19.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.424+0000 7f7f10870640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0c112400 con 0x7f7f0c06a840 2024-04-22T13:35:19.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.424+0000 7f7f0ad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f0c19d020 0x7f7f0c19d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:19.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.424+0000 7f7f0ad9d640 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f0c19d020 0x7f7f0c19d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:55822/0 (socket says 172.21.15.12:55822) 2024-04-22T13:35:19.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.424+0000 7f7f0ad9d640 1 -- 172.21.15.12:0/1130803002 learned_addr learned my addr 172.21.15.12:0/1130803002 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-04-22T13:35:19.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.424+0000 7f7f09d9b640 1 --2- 172.21.15.12:0/1130803002 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f0c107270 0x7f7f0c198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:19.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.424+0000 7f7f0a59c640 1 --2- 172.21.15.12:0/1130803002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f0c06a840 0x7f7f0c198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:19.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.424+0000 7f7f0ad9d640 1 -- 172.21.15.12:0/1130803002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f0c06a840 msgr2=0x7f7f0c198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:19.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.424+0000 7f7f0ad9d640 1 --2- 172.21.15.12:0/1130803002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f0c06a840 0x7f7f0c198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:19.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.424+0000 7f7f0ad9d640 1 -- 172.21.15.12:0/1130803002 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f0c107270 msgr2=0x7f7f0c198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:19.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.424+0000 7f7f0ad9d640 1 --2- 172.21.15.12:0/1130803002 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f0c107270 0x7f7f0c198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:19.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.424+0000 7f7f0ad9d640 1 -- 172.21.15.12:0/1130803002 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f0c19db10 con 0x7f7f0c19d020 2024-04-22T13:35:19.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.424+0000 7f7f0a59c640 1 --2- 172.21.15.12:0/1130803002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f0c06a840 0x7f7f0c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-22T13:35:19.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.424+0000 7f7f0ad9d640 1 --2- 172.21.15.12:0/1130803002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f0c19d020 0x7f7f0c19d4d0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400eaa0 tx=0x7f7f0400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:19.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.424+0000 7f7efb7fe640 1 -- 172.21.15.12:0/1130803002 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0400cdb0 con 0x7f7f0c19d020 2024-04-22T13:35:19.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.424+0000 7f7efb7fe640 1 -- 172.21.15.12:0/1130803002 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f04005020 con 0x7f7f0c19d020 2024-04-22T13:35:19.422 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.424+0000 7f7f10870640 1 -- 172.21.15.12:0/1130803002 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f0c19de00 con 0x7f7f0c19d020 2024-04-22T13:35:19.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.424+0000 7f7efb7fe640 1 -- 172.21.15.12:0/1130803002 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f040116d0 con 0x7f7f0c19d020 2024-04-22T13:35:19.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.424+0000 7f7f10870640 1 -- 172.21.15.12:0/1130803002 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f0c1a55a0 con 0x7f7f0c19d020 2024-04-22T13:35:19.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.425+0000 7f7f10870640 1 -- 172.21.15.12:0/1130803002 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f0c0691c0 con 0x7f7f0c19d020 2024-04-22T13:35:19.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.426+0000 7f7efb7fe640 1 -- 172.21.15.12:0/1130803002 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7f040051d0 con 0x7f7f0c19d020 2024-04-22T13:35:19.423 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.427+0000 7f7efb7fe640 1 --2- 172.21.15.12:0/1130803002 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7ef40a3d20 0x7f7ef40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-22T13:35:19.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.427+0000 7f7f0a59c640 1 --2- 172.21.15.12:0/1130803002 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7ef40a3d20 0x7f7ef40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-22T13:35:19.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.427+0000 7f7efb7fe640 1 -- 172.21.15.12:0/1130803002 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8374+0+0 (secure 0 0 0) 0x7f7f040d0a30 con 0x7f7f0c19d020 2024-04-22T13:35:19.424 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.428+0000 7f7f0a59c640 1 --2- 172.21.15.12:0/1130803002 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7ef40a3d20 0x7f7ef40a61e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7efc036040 tx=0x7f7efc005fd0 comp rx=0 tx=0).ready entity=mgr.24968 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-22T13:35:19.427 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.432+0000 7f7efb7fe640 1 -- 172.21.15.12:0/1130803002 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7f04016030 con 0x7f7f0c19d020 2024-04-22T13:35:19.620 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:19.624+0000 7f7f10870640 1 -- 172.21.15.12:0/1130803002 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7f0c06b1f0 con 0x7f7f0c19d020 2024-04-22T13:35:19.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:19 smithi134 ceph-mon[31338]: pgmap v350: 97 pgs: 97 active+clean; 2.3 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.0 KiB/s wr, 15 op/s 2024-04-22T13:35:19.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:19 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/2888149409' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-22T13:35:19.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:19 smithi134 ceph-mon[31338]: osdmap e102: 12 total, 12 up, 12 in 2024-04-22T13:35:19.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:19 smithi134 ceph-mon[31338]: from='client.? 172.21.15.12:0/1130803002' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-22T13:35:19.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:19 smithi111 ceph-mon[31361]: pgmap v350: 97 pgs: 97 active+clean; 2.3 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.0 KiB/s wr, 15 op/s 2024-04-22T13:35:19.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:19 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/2888149409' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-22T13:35:19.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:19 smithi111 ceph-mon[31361]: osdmap e102: 12 total, 12 up, 12 in 2024-04-22T13:35:19.976 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:19 smithi111 ceph-mon[31361]: from='client.? 172.21.15.12:0/1130803002' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-22T13:35:20.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:19 smithi012 ceph-mon[28265]: pgmap v350: 97 pgs: 97 active+clean; 2.3 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.0 KiB/s wr, 15 op/s 2024-04-22T13:35:20.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:19 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/2888149409' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-22T13:35:20.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:19 smithi012 ceph-mon[28265]: osdmap e102: 12 total, 12 up, 12 in 2024-04-22T13:35:20.086 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:19 smithi012 ceph-mon[28265]: from='client.? 172.21.15.12:0/1130803002' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-22T13:35:20.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:20.301+0000 7f7efb7fe640 1 -- 172.21.15.12:0/1130803002 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v103) ==== 169+0+0 (secure 0 0 0) 0x7f7f0409d160 con 0x7f7f0c19d020 2024-04-22T13:35:20.296 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'cephfs_data' removed 2024-04-22T13:35:20.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:20.305+0000 7f7f10870640 1 -- 172.21.15.12:0/1130803002 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7ef40a3d20 msgr2=0x7f7ef40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:20.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:20.305+0000 7f7f10870640 1 --2- 172.21.15.12:0/1130803002 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7ef40a3d20 0x7f7ef40a61e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7efc036040 tx=0x7f7efc005fd0 comp rx=0 tx=0).stop 2024-04-22T13:35:20.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:20.305+0000 7f7f10870640 1 -- 172.21.15.12:0/1130803002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f0c19d020 msgr2=0x7f7f0c19d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-22T13:35:20.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:20.305+0000 7f7f10870640 1 --2- 172.21.15.12:0/1130803002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f0c19d020 0x7f7f0c19d4d0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400eaa0 tx=0x7f7f0400ef60 comp rx=0 tx=0).stop 2024-04-22T13:35:20.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:20.305+0000 7f7f10870640 1 -- 172.21.15.12:0/1130803002 shutdown_connections 2024-04-22T13:35:20.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:20.305+0000 7f7f10870640 1 --2- 172.21.15.12:0/1130803002 >> [v2:172.21.15.12:6800/1615661596,v1:172.21.15.12:6801/1615661596] conn(0x7f7ef40a3d20 0x7f7ef40a61e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:20.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:20.305+0000 7f7f10870640 1 --2- 172.21.15.12:0/1130803002 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f7f0c19d020 0x7f7f0c19d4d0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:20.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:20.305+0000 7f7f10870640 1 --2- 172.21.15.12:0/1130803002 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f0c107270 0x7f7f0c198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:20.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:20.305+0000 7f7f10870640 1 --2- 172.21.15.12:0/1130803002 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f0c06a840 0x7f7f0c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-22T13:35:20.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:20.305+0000 7f7f10870640 1 -- 172.21.15.12:0/1130803002 >> 172.21.15.12:0/1130803002 conn(0x7f7f0c077eb0 msgr2=0x7f7f0c108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-22T13:35:20.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:20.306+0000 7f7f10870640 1 -- 172.21.15.12:0/1130803002 shutdown_connections 2024-04-22T13:35:20.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-22T13:35:20.306+0000 7f7f10870640 1 -- 172.21.15.12:0/1130803002 wait complete. 2024-04-22T13:35:20.319 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-22T13:35:20.346 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-22T13:35:20.377 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-22T13:35:20.404 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-22T13:35:20.404 DEBUG:teuthology.orchestra.run.smithi012:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-22T13:35:20.421 DEBUG:teuthology.orchestra.run.smithi111:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-22T13:35:20.437 DEBUG:teuthology.orchestra.run.smithi134:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-22T13:35:20.459 INFO:tasks.cephadm:Stopping all daemons... 2024-04-22T13:35:20.459 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-22T13:35:20.459 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mon.a 2024-04-22T13:35:20.586 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:20 smithi012 systemd[1]: Stopping Ceph mon.a for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:35:21.107 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:20 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mon-a[28261]: 2024-04-22T13:35:20.847+0000 7fe085453700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:35:21.108 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:20 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mon-a[28261]: 2024-04-22T13:35:20.847+0000 7fe085453700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-22T13:35:21.391 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:21 smithi012 podman[101157]: 2024-04-22 13:35:21.112956907 +0000 UTC m=+0.451185416 container died f2fedfae476176dd590a89c0a4f08f37920f761ca34ffa424bc61e3b5b5e8b84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mon-a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T13:35:21.758 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:21 smithi012 podman[101157]: 2024-04-22 13:35:21.503930227 +0000 UTC m=+0.842158723 container cleanup f2fedfae476176dd590a89c0a4f08f37920f761ca34ffa424bc61e3b5b5e8b84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS) 2024-04-22T13:35:21.758 INFO:journalctl@ceph.mon.a.smithi012.stdout:Apr 22 13:35:21 smithi012 bash[101157]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mon-a 2024-04-22T13:35:21.885 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mon.a.service' 2024-04-22T13:35:21.935 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:35:21.935 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-22T13:35:21.935 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-22T13:35:21.935 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mon.b 2024-04-22T13:35:22.226 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:21 smithi111 systemd[1]: Stopping Ceph mon.b for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:35:22.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:22 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mon-b[31357]: 2024-04-22T13:35:22.241+0000 7fd9340a0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:35:22.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:22 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mon-b[31357]: 2024-04-22T13:35:22.241+0000 7fd9340a0700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-22T13:35:22.726 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:22 smithi111 podman[67678]: 2024-04-22 13:35:22.488043595 +0000 UTC m=+0.355123722 container died d0da6c57256953ecb5efa867720986477ff1731b90a87301f532a7caa2c06099 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mon-b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-22T13:35:23.130 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:22 smithi111 podman[67678]: 2024-04-22 13:35:22.867332553 +0000 UTC m=+0.734412670 container cleanup d0da6c57256953ecb5efa867720986477ff1731b90a87301f532a7caa2c06099 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mon-b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True) 2024-04-22T13:35:23.130 INFO:journalctl@ceph.mon.b.smithi111.stdout:Apr 22 13:35:22 smithi111 bash[67678]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mon-b 2024-04-22T13:35:23.277 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mon.b.service' 2024-04-22T13:35:23.322 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:35:23.323 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-22T13:35:23.323 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-22T13:35:23.323 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mon.c 2024-04-22T13:35:23.616 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:23 smithi134 systemd[1]: Stopping Ceph mon.c for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:35:23.926 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:23 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mon-c[31320]: 2024-04-22T13:35:23.620+0000 7fe8072a5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:35:23.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:23 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mon-c[31320]: 2024-04-22T13:35:23.620+0000 7fe8072a5700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-22T13:35:23.927 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:23 smithi134 podman[68144]: 2024-04-22 13:35:23.753703393 +0000 UTC m=+0.240142773 container died b5335653a227532dc122aefd6e6dcf0398436f989f552bbf1bdcd8f870592afa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mon-c, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-22T13:35:24.426 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:24 smithi134 podman[68144]: 2024-04-22 13:35:24.129093379 +0000 UTC m=+0.615532755 container cleanup b5335653a227532dc122aefd6e6dcf0398436f989f552bbf1bdcd8f870592afa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mon-c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-04-22T13:35:24.427 INFO:journalctl@ceph.mon.c.smithi134.stdout:Apr 22 13:35:24 smithi134 bash[68144]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mon-c 2024-04-22T13:35:24.593 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mon.c.service' 2024-04-22T13:35:24.637 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:35:24.637 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-22T13:35:24.637 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-22T13:35:24.638 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mgr.x 2024-04-22T13:35:24.971 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:35:24 smithi012 systemd[1]: Stopping Ceph mgr.x for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:35:25.279 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:35:24 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x[28500]: 2024-04-22T13:35:24.973+0000 7fdcc4acb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:35:25.279 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:35:25 smithi012 podman[101378]: 2024-04-22 13:35:25.00598795 +0000 UTC m=+0.160009908 container died 822680d55dcd56b91d5bc4f10bb989d0f7f38bc01c560d035e060fbb4006c424 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-22T13:35:25.586 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:35:25 smithi012 podman[101378]: 2024-04-22 13:35:25.391622717 +0000 UTC m=+0.545644666 container cleanup 822680d55dcd56b91d5bc4f10bb989d0f7f38bc01c560d035e060fbb4006c424 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-22T13:35:25.586 INFO:journalctl@ceph.mgr.x.smithi012.stdout:Apr 22 13:35:25 smithi012 bash[101378]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-x 2024-04-22T13:35:25.752 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-22T13:35:25.788 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:35:25.788 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-22T13:35:25.788 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-22T13:35:25.788 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mgr.y 2024-04-22T13:35:26.112 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:35:25 smithi111 systemd[1]: Stopping Ceph mgr.y for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:35:26.475 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:35:26 smithi111 podman[67810]: 2024-04-22 13:35:26.117568902 +0000 UTC m=+0.139909024 container died 094e71b40f5b49861bc7c6225f1af2107604d8ed523d62c1af2de85ceabda89d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-22T13:35:26.726 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:35:26 smithi111 podman[67810]: 2024-04-22 13:35:26.480238578 +0000 UTC m=+0.502578684 container cleanup 094e71b40f5b49861bc7c6225f1af2107604d8ed523d62c1af2de85ceabda89d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-22T13:35:26.726 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:35:26 smithi111 bash[67810]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y 2024-04-22T13:35:26.726 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:35:26 smithi111 podman[67824]: 2024-04-22 13:35:26.673122529 +0000 UTC m=+0.551120495 container remove 094e71b40f5b49861bc7c6225f1af2107604d8ed523d62c1af2de85ceabda89d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-y, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-22T13:35:26.726 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Apr 22 13:35:26 smithi111 systemd[1]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-04-22T13:35:26.884 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-22T13:35:26.920 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:35:26.920 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-22T13:35:26.920 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-22T13:35:26.920 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mgr.z 2024-04-22T13:35:27.241 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:35:26 smithi134 systemd[1]: Stopping Ceph mgr.z for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:35:27.529 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:35:27 smithi134 podman[68269]: 2024-04-22 13:35:27.246044802 +0000 UTC m=+0.126345710 container died bed71e9456c44b97207188b4a9e0f9f0ef79b9efc40b917c76cd66a7bdfd979d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-04-22T13:35:27.863 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:35:27 smithi134 podman[68269]: 2024-04-22 13:35:27.616991633 +0000 UTC m=+0.497292542 container cleanup bed71e9456c44b97207188b4a9e0f9f0ef79b9efc40b917c76cd66a7bdfd979d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True) 2024-04-22T13:35:27.863 INFO:journalctl@ceph.mgr.z.smithi134.stdout:Apr 22 13:35:27 smithi134 bash[68269]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mgr-z 2024-04-22T13:35:28.025 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-22T13:35:28.060 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:35:28.060 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-22T13:35:28.060 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-22T13:35:28.060 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.0 2024-04-22T13:35:28.382 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 22 13:35:28 smithi012 systemd[1]: Stopping Ceph osd.0 for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:35:28.836 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 22 13:35:28 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-0[42725]: 2024-04-22T13:35:28.386+0000 7f9655467700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:35:28.836 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 22 13:35:28 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-0[42725]: 2024-04-22T13:35:28.386+0000 7f9655467700 -1 osd.0 103 *** Got signal Terminated *** 2024-04-22T13:35:28.836 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 22 13:35:28 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-0[42725]: 2024-04-22T13:35:28.386+0000 7f9655467700 -1 osd.0 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T13:35:34.033 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 22 13:35:33 smithi012 podman[101501]: 2024-04-22 13:35:33.748889221 +0000 UTC m=+5.486549028 container died 8bc0836a9f030f336f76a68bf61f93118478597149c8e9de4a5635e5ffaacb11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-22T13:35:34.336 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 22 13:35:34 smithi012 podman[101501]: 2024-04-22 13:35:34.154381836 +0000 UTC m=+5.892041632 container cleanup 8bc0836a9f030f336f76a68bf61f93118478597149c8e9de4a5635e5ffaacb11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True) 2024-04-22T13:35:34.336 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 22 13:35:34 smithi012 bash[101501]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-0 2024-04-22T13:35:34.784 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 22 13:35:34 smithi012 podman[101520]: 2024-04-22 13:35:34.430503085 +0000 UTC m=+0.670087965 container remove 8bc0836a9f030f336f76a68bf61f93118478597149c8e9de4a5635e5ffaacb11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T13:35:35.086 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 22 13:35:34 smithi012 podman[101613]: 2024-04-22 13:35:34.789464786 +0000 UTC m=+0.113425495 container create 9442ec9dbd5cf9e4dcc0ee36145d978a56819d94146e27a401321da43ff93b30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-04-22T13:35:35.086 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 22 13:35:34 smithi012 podman[101613]: 2024-04-22 13:35:34.69456434 +0000 UTC m=+0.018525076 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:35:35.086 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 22 13:35:35 smithi012 podman[101613]: 2024-04-22 13:35:35.016865977 +0000 UTC m=+0.340826687 container init 9442ec9dbd5cf9e4dcc0ee36145d978a56819d94146e27a401321da43ff93b30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-22T13:35:35.086 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 22 13:35:35 smithi012 podman[101613]: 2024-04-22 13:35:35.020023749 +0000 UTC m=+0.343984475 container start 9442ec9dbd5cf9e4dcc0ee36145d978a56819d94146e27a401321da43ff93b30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T13:35:35.587 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 22 13:35:35 smithi012 podman[101613]: 2024-04-22 13:35:35.099991032 +0000 UTC m=+0.423951742 container attach 9442ec9dbd5cf9e4dcc0ee36145d978a56819d94146e27a401321da43ff93b30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T13:35:35.587 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 22 13:35:35 smithi012 podman[101613]: 2024-04-22 13:35:35.328739449 +0000 UTC m=+0.652700168 container died 9442ec9dbd5cf9e4dcc0ee36145d978a56819d94146e27a401321da43ff93b30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T13:35:35.867 INFO:journalctl@ceph.osd.0.smithi012.stdout:Apr 22 13:35:35 smithi012 podman[101642]: 2024-04-22 13:35:35.870159308 +0000 UTC m=+0.531519589 container remove 9442ec9dbd5cf9e4dcc0ee36145d978a56819d94146e27a401321da43ff93b30 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True) 2024-04-22T13:35:35.887 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.0.service' 2024-04-22T13:35:35.929 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:35:35.930 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-22T13:35:35.930 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-22T13:35:35.930 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.1 2024-04-22T13:35:36.226 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 22 13:35:35 smithi111 systemd[1]: Stopping Ceph osd.1 for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:35:36.726 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 22 13:35:36 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-1[35956]: 2024-04-22T13:35:36.283+0000 7f5b70abe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:35:36.726 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 22 13:35:36 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-1[35956]: 2024-04-22T13:35:36.283+0000 7f5b70abe700 -1 osd.1 103 *** Got signal Terminated *** 2024-04-22T13:35:36.726 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 22 13:35:36 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-1[35956]: 2024-04-22T13:35:36.283+0000 7f5b70abe700 -1 osd.1 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T13:35:41.976 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 22 13:35:41 smithi111 podman[67933]: 2024-04-22 13:35:41.711724746 +0000 UTC m=+5.584965484 container died 673de813ef9294c8c637f1272fd52ea8a10489c644a8b6f5bf588544939b5062 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-1, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-22T13:35:42.354 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 22 13:35:42 smithi111 podman[67933]: 2024-04-22 13:35:42.067067045 +0000 UTC m=+5.940307782 container cleanup 673de813ef9294c8c637f1272fd52ea8a10489c644a8b6f5bf588544939b5062 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-1, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-22T13:35:42.355 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 22 13:35:42 smithi111 bash[67933]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-1 2024-04-22T13:35:42.683 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 22 13:35:42 smithi111 podman[67951]: 2024-04-22 13:35:42.359904732 +0000 UTC m=+0.650870722 container remove 673de813ef9294c8c637f1272fd52ea8a10489c644a8b6f5bf588544939b5062 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-22T13:35:42.975 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 22 13:35:42 smithi111 podman[68041]: 2024-04-22 13:35:42.591315521 +0000 UTC m=+0.017855488 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:35:42.975 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 22 13:35:42 smithi111 podman[68041]: 2024-04-22 13:35:42.727915319 +0000 UTC m=+0.154455312 container create f0d347dfc334df56b70006010fe0c91717f332ef2152e4ee2ca033321cdbb7ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-22T13:35:42.975 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 22 13:35:42 smithi111 podman[68041]: 2024-04-22 13:35:42.98052041 +0000 UTC m=+0.407060364 container init f0d347dfc334df56b70006010fe0c91717f332ef2152e4ee2ca033321cdbb7ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-22T13:35:43.226 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 22 13:35:42 smithi111 podman[68041]: 2024-04-22 13:35:42.983980843 +0000 UTC m=+0.410520797 container start f0d347dfc334df56b70006010fe0c91717f332ef2152e4ee2ca033321cdbb7ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-22T13:35:43.226 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 22 13:35:43 smithi111 podman[68041]: 2024-04-22 13:35:43.072848586 +0000 UTC m=+0.499388541 container attach f0d347dfc334df56b70006010fe0c91717f332ef2152e4ee2ca033321cdbb7ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-22T13:35:43.556 INFO:journalctl@ceph.osd.1.smithi111.stdout:Apr 22 13:35:43 smithi111 podman[68041]: 2024-04-22 13:35:43.283223335 +0000 UTC m=+0.709763295 container died f0d347dfc334df56b70006010fe0c91717f332ef2152e4ee2ca033321cdbb7ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-22T13:35:43.914 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.1.service' 2024-04-22T13:35:43.952 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:35:43.952 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-22T13:35:43.952 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-22T13:35:43.952 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.2 2024-04-22T13:35:44.232 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 22 13:35:44 smithi134 systemd[1]: Stopping Ceph osd.2 for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:35:44.677 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 22 13:35:44 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-2[36460]: 2024-04-22T13:35:44.236+0000 7ff2ee048700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:35:44.677 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 22 13:35:44 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-2[36460]: 2024-04-22T13:35:44.236+0000 7ff2ee048700 -1 osd.2 103 *** Got signal Terminated *** 2024-04-22T13:35:44.677 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 22 13:35:44 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-2[36460]: 2024-04-22T13:35:44.236+0000 7ff2ee048700 -1 osd.2 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T13:35:50.017 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 22 13:35:49 smithi134 podman[68392]: 2024-04-22 13:35:49.732343715 +0000 UTC m=+5.587066997 container died 948065a127b10016897da8fea9251c525eb50e5edfbbac4d5b5774d0612a8876 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-22T13:35:50.358 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 22 13:35:50 smithi134 podman[68392]: 2024-04-22 13:35:50.111724472 +0000 UTC m=+5.966447765 container cleanup 948065a127b10016897da8fea9251c525eb50e5edfbbac4d5b5774d0612a8876 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0) 2024-04-22T13:35:50.358 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 22 13:35:50 smithi134 bash[68392]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-2 2024-04-22T13:35:50.676 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 22 13:35:50 smithi134 podman[68409]: 2024-04-22 13:35:50.363062722 +0000 UTC m=+0.628817175 container remove 948065a127b10016897da8fea9251c525eb50e5edfbbac4d5b5774d0612a8876 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T13:35:50.978 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 22 13:35:50 smithi134 podman[68499]: 2024-04-22 13:35:50.714076361 +0000 UTC m=+0.105828889 container create 222cd779f92ea5207c29433ca5353aae5d370ad02ea87305b4ebb273951ea571 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-22T13:35:50.979 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 22 13:35:50 smithi134 podman[68499]: 2024-04-22 13:35:50.626270016 +0000 UTC m=+0.018022544 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:35:50.979 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 22 13:35:50 smithi134 podman[68499]: 2024-04-22 13:35:50.924682003 +0000 UTC m=+0.316434531 container init 222cd779f92ea5207c29433ca5353aae5d370ad02ea87305b4ebb273951ea571 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T13:35:50.979 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 22 13:35:50 smithi134 podman[68499]: 2024-04-22 13:35:50.927913081 +0000 UTC m=+0.319665610 container start 222cd779f92ea5207c29433ca5353aae5d370ad02ea87305b4ebb273951ea571 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-22T13:35:50.979 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 22 13:35:50 smithi134 podman[68499]: 2024-04-22 13:35:50.983047279 +0000 UTC m=+0.374799817 container attach 222cd779f92ea5207c29433ca5353aae5d370ad02ea87305b4ebb273951ea571 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T13:35:51.243 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 22 13:35:51 smithi134 podman[68499]: 2024-04-22 13:35:51.225300902 +0000 UTC m=+0.617053437 container died 222cd779f92ea5207c29433ca5353aae5d370ad02ea87305b4ebb273951ea571 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True) 2024-04-22T13:35:51.808 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.2.service' 2024-04-22T13:35:51.832 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 22 13:35:51 smithi134 podman[68525]: 2024-04-22 13:35:51.795601458 +0000 UTC m=+0.564611225 container remove 222cd779f92ea5207c29433ca5353aae5d370ad02ea87305b4ebb273951ea571 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-22T13:35:51.832 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 22 13:35:51 smithi134 systemd[1]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.2.service: Deactivated successfully. 2024-04-22T13:35:51.832 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 22 13:35:51 smithi134 systemd[1]: Stopped Ceph osd.2 for 0aede5ea-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:35:51.832 INFO:journalctl@ceph.osd.2.smithi134.stdout:Apr 22 13:35:51 smithi134 systemd[1]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.2.service: Consumed 19.431s CPU time. 2024-04-22T13:35:51.844 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:35:51.845 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-22T13:35:51.845 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-22T13:35:51.845 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.3 2024-04-22T13:35:52.157 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 22 13:35:51 smithi012 systemd[1]: Stopping Ceph osd.3 for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:35:52.435 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 22 13:35:52 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-3[49663]: 2024-04-22T13:35:52.162+0000 7fb4a0117700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:35:52.436 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 22 13:35:52 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-3[49663]: 2024-04-22T13:35:52.162+0000 7fb4a0117700 -1 osd.3 103 *** Got signal Terminated *** 2024-04-22T13:35:52.436 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 22 13:35:52 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-3[49663]: 2024-04-22T13:35:52.162+0000 7fb4a0117700 -1 osd.3 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T13:35:54.586 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:35:54 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:35:54.312+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:30.866019+0000 front 2024-04-22T13:35:30.866082+0000 (oldest deadline 2024-04-22T13:35:53.766157+0000) 2024-04-22T13:35:55.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:35:54 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:35:54.721+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:35:55.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:35:55 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:35:55.168+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:35:55.677 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:35:55 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:35:55.349+0000 7fcf4af2c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:28.669896+0000 front 2024-04-22T13:35:28.670069+0000 (oldest deadline 2024-04-22T13:35:54.569846+0000) 2024-04-22T13:35:55.706 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:35:55 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:35:55.346+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:30.866019+0000 front 2024-04-22T13:35:30.866082+0000 (oldest deadline 2024-04-22T13:35:53.766157+0000) 2024-04-22T13:35:56.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:35:55 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:35:55.710+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:35:56.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:35:56 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:35:56.150+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:35:56.675 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:35:56 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:35:56.385+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:30.866019+0000 front 2024-04-22T13:35:30.866082+0000 (oldest deadline 2024-04-22T13:35:53.766157+0000) 2024-04-22T13:35:56.676 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:35:56 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:35:56.338+0000 7fcf4af2c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:28.669896+0000 front 2024-04-22T13:35:28.670069+0000 (oldest deadline 2024-04-22T13:35:54.569846+0000) 2024-04-22T13:35:57.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:35:56 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:35:56.679+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:35:57.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:35:57 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:35:57.198+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:35:57.669 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 22 13:35:57 smithi012 podman[101721]: 2024-04-22 13:35:57.494181927 +0000 UTC m=+5.438262854 container died 5f98e37eebf9601121245c960b498901d3410187a45089a362e82b60e10f202d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-22T13:35:57.669 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:35:57 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:35:57.359+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:30.866019+0000 front 2024-04-22T13:35:30.866082+0000 (oldest deadline 2024-04-22T13:35:53.766157+0000) 2024-04-22T13:35:57.676 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:35:57 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:35:57.361+0000 7fcf4af2c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:28.669896+0000 front 2024-04-22T13:35:28.670069+0000 (oldest deadline 2024-04-22T13:35:54.569846+0000) 2024-04-22T13:35:58.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:35:57 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:35:57.673+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:35:58.087 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 22 13:35:57 smithi012 podman[101721]: 2024-04-22 13:35:57.845063173 +0000 UTC m=+5.789144101 container cleanup 5f98e37eebf9601121245c960b498901d3410187a45089a362e82b60e10f202d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-22T13:35:58.087 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 22 13:35:57 smithi012 bash[101721]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-3 2024-04-22T13:35:58.469 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 22 13:35:58 smithi012 podman[101750]: 2024-04-22 13:35:58.137903074 +0000 UTC m=+0.641651110 container remove 5f98e37eebf9601121245c960b498901d3410187a45089a362e82b60e10f202d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-3, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T13:35:58.469 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:35:58 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:35:58.356+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:30.866019+0000 front 2024-04-22T13:35:30.866082+0000 (oldest deadline 2024-04-22T13:35:53.766157+0000) 2024-04-22T13:35:58.676 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:35:58 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:35:58.375+0000 7fcf4af2c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:28.669896+0000 front 2024-04-22T13:35:28.670069+0000 (oldest deadline 2024-04-22T13:35:54.569846+0000) 2024-04-22T13:35:58.677 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:35:58 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:35:58.298+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:35:58.726 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:35:58 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:35:58.233+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:35:58.769 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:35:58 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:35:58.694+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:35:58.769 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 22 13:35:58 smithi012 podman[101841]: 2024-04-22 13:35:58.377909349 +0000 UTC m=+0.018339849 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:35:58.770 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 22 13:35:58 smithi012 podman[101841]: 2024-04-22 13:35:58.505404807 +0000 UTC m=+0.145835305 container create 58b7b3e7e3626898f3e84772ebc837c92625eb61a4f40c80ed55c4b8d66b2bff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-22T13:35:58.770 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 22 13:35:58 smithi012 podman[101841]: 2024-04-22 13:35:58.691307379 +0000 UTC m=+0.331737877 container init 58b7b3e7e3626898f3e84772ebc837c92625eb61a4f40c80ed55c4b8d66b2bff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-22T13:35:58.770 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 22 13:35:58 smithi012 podman[101841]: 2024-04-22 13:35:58.694394975 +0000 UTC m=+0.334825473 container start 58b7b3e7e3626898f3e84772ebc837c92625eb61a4f40c80ed55c4b8d66b2bff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-22T13:35:58.770 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 22 13:35:58 smithi012 podman[101841]: 2024-04-22 13:35:58.774646791 +0000 UTC m=+0.415077301 container attach 58b7b3e7e3626898f3e84772ebc837c92625eb61a4f40c80ed55c4b8d66b2bff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-22T13:35:59.086 INFO:journalctl@ceph.osd.3.smithi012.stdout:Apr 22 13:35:58 smithi012 podman[101841]: 2024-04-22 13:35:58.995282579 +0000 UTC m=+0.635713076 container died 58b7b3e7e3626898f3e84772ebc837c92625eb61a4f40c80ed55c4b8d66b2bff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416) 2024-04-22T13:35:59.564 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:35:59 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:35:59.382+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:30.866019+0000 front 2024-04-22T13:35:30.866082+0000 (oldest deadline 2024-04-22T13:35:53.766157+0000) 2024-04-22T13:35:59.586 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.3.service' 2024-04-22T13:35:59.627 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:35:59.627 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-22T13:35:59.627 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-22T13:35:59.627 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.4 2024-04-22T13:35:59.653 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:35:59 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:35:59.258+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:35:59.676 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:35:59 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:35:59.353+0000 7fcf4af2c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:28.669896+0000 front 2024-04-22T13:35:28.670069+0000 (oldest deadline 2024-04-22T13:35:54.569846+0000) 2024-04-22T13:35:59.676 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:35:59 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:35:59.248+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:35:59.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:35:59 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:35:59.699+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:35:59.933 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 22 13:35:59 smithi111 systemd[1]: Stopping Ceph osd.4 for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:36:00.226 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 22 13:35:59 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-4[42939]: 2024-04-22T13:35:59.937+0000 7fd695d66700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:36:00.226 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 22 13:35:59 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-4[42939]: 2024-04-22T13:35:59.937+0000 7fd695d66700 -1 osd.4 103 *** Got signal Terminated *** 2024-04-22T13:36:00.226 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 22 13:35:59 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-4[42939]: 2024-04-22T13:35:59.937+0000 7fd695d66700 -1 osd.4 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T13:36:00.676 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:00 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:36:00.387+0000 7fcf4af2c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:28.669896+0000 front 2024-04-22T13:35:28.670069+0000 (oldest deadline 2024-04-22T13:35:54.569846+0000) 2024-04-22T13:36:00.677 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:00 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:00.229+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:00.688 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:00 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:00.346+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:30.866019+0000 front 2024-04-22T13:35:30.866082+0000 (oldest deadline 2024-04-22T13:35:53.766157+0000) 2024-04-22T13:36:00.726 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:00 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:00.278+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:01.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:00 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:00.692+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:01.654 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:01 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:01.383+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:30.866019+0000 front 2024-04-22T13:35:30.866082+0000 (oldest deadline 2024-04-22T13:35:53.766157+0000) 2024-04-22T13:36:01.677 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:01 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:36:01.433+0000 7fcf4af2c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:28.669896+0000 front 2024-04-22T13:35:28.670069+0000 (oldest deadline 2024-04-22T13:35:54.569846+0000) 2024-04-22T13:36:01.677 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:01 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:01.236+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:01.726 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:01 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:01.274+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:02.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:01 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:01.658+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:02.500 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:02 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:36:02.427+0000 7fcf4af2c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:28.669896+0000 front 2024-04-22T13:35:28.670069+0000 (oldest deadline 2024-04-22T13:35:54.569846+0000) 2024-04-22T13:36:02.500 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:02 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:02.251+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:02.726 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 22 13:36:02 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-4[42939]: 2024-04-22T13:36:02.492+0000 7fd68e352700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.321402+0000 front 2024-04-22T13:35:39.321429+0000 (oldest deadline 2024-04-22T13:36:02.221543+0000) 2024-04-22T13:36:02.726 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:02 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:02.271+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:02.836 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:02 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:02.421+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:30.866019+0000 front 2024-04-22T13:35:30.866082+0000 (oldest deadline 2024-04-22T13:35:53.766157+0000) 2024-04-22T13:36:02.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:02 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:02.633+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:02.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:02 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:02.633+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:03.459 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:03 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:03.184+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:03.459 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:03 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:03.240+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:03.459 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:03 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:03.240+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:03.626 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:03 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:03.374+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:30.866019+0000 front 2024-04-22T13:35:30.866082+0000 (oldest deadline 2024-04-22T13:35:53.766157+0000) 2024-04-22T13:36:03.658 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 22 13:36:03 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-4[42939]: 2024-04-22T13:36:03.528+0000 7fd68e352700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.321402+0000 front 2024-04-22T13:35:39.321429+0000 (oldest deadline 2024-04-22T13:36:02.221543+0000) 2024-04-22T13:36:03.658 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:03 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:03.295+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:03.926 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:03 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:36:03.463+0000 7fcf4af2c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:28.669896+0000 front 2024-04-22T13:35:28.670069+0000 (oldest deadline 2024-04-22T13:35:54.569846+0000) 2024-04-22T13:36:03.976 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:03 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:03.662+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:38.436756+0000 front 2024-04-22T13:35:38.436613+0000 (oldest deadline 2024-04-22T13:36:03.136757+0000) 2024-04-22T13:36:04.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:03 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:03.631+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:04.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:03 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:03.631+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:04.426 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:04 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:04.153+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:04.427 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:04 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:04.248+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:04.427 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:04 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:04.248+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:04.663 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:04 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:04.281+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:04.664 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 22 13:36:04 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-4[42939]: 2024-04-22T13:36:04.505+0000 7fd68e352700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.321402+0000 front 2024-04-22T13:35:39.321429+0000 (oldest deadline 2024-04-22T13:36:02.221543+0000) 2024-04-22T13:36:04.836 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:04 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:04.383+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:30.866019+0000 front 2024-04-22T13:35:30.866082+0000 (oldest deadline 2024-04-22T13:35:53.766157+0000) 2024-04-22T13:36:04.836 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:04 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:04.383+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.067439+0000 front 2024-04-22T13:35:39.067262+0000 (oldest deadline 2024-04-22T13:36:03.767273+0000) 2024-04-22T13:36:04.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:04 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:04.609+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:04.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:04 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:04.609+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:04.926 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:04 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:36:04.459+0000 7fcf4af2c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:28.669896+0000 front 2024-04-22T13:35:28.670069+0000 (oldest deadline 2024-04-22T13:35:54.569846+0000) 2024-04-22T13:36:04.976 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:04 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:04.667+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:38.436756+0000 front 2024-04-22T13:35:38.436613+0000 (oldest deadline 2024-04-22T13:36:03.136757+0000) 2024-04-22T13:36:05.426 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:05 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:05.145+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:05.427 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:05 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:05.255+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:05.427 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:05 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:05.255+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:05.651 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 22 13:36:05 smithi111 podman[68169]: 2024-04-22 13:36:05.383288113 +0000 UTC m=+5.548173366 container died a2f7b06a1201bff971c6f45b19d8d8aadba80b14f14f5b1859f99c78da7b0ef2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-22T13:36:05.652 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:05 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:05.301+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:05.652 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:05 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:05.301+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:05.836 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:05 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:05.404+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:30.866019+0000 front 2024-04-22T13:35:30.866082+0000 (oldest deadline 2024-04-22T13:35:53.766157+0000) 2024-04-22T13:36:05.836 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:05 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:05.404+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.067439+0000 front 2024-04-22T13:35:39.067262+0000 (oldest deadline 2024-04-22T13:36:03.767273+0000) 2024-04-22T13:36:05.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:05 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:05.593+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:05.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:05 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:05.593+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:05.926 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:05 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:36:05.447+0000 7fcf4af2c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:28.669896+0000 front 2024-04-22T13:35:28.670069+0000 (oldest deadline 2024-04-22T13:35:54.569846+0000) 2024-04-22T13:36:05.926 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:05 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:36:05.447+0000 7fcf4af2c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:40.371877+0000 front 2024-04-22T13:35:40.371666+0000 (oldest deadline 2024-04-22T13:36:04.471635+0000) 2024-04-22T13:36:05.976 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:05 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:05.691+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:38.436756+0000 front 2024-04-22T13:35:38.436613+0000 (oldest deadline 2024-04-22T13:36:03.136757+0000) 2024-04-22T13:36:05.976 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 22 13:36:05 smithi111 podman[68169]: 2024-04-22 13:36:05.755430073 +0000 UTC m=+5.920315323 container cleanup a2f7b06a1201bff971c6f45b19d8d8aadba80b14f14f5b1859f99c78da7b0ef2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-4, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-22T13:36:05.976 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 22 13:36:05 smithi111 bash[68169]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-4 2024-04-22T13:36:06.372 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 22 13:36:06 smithi111 podman[68186]: 2024-04-22 13:36:06.031650717 +0000 UTC m=+0.643311421 container remove a2f7b06a1201bff971c6f45b19d8d8aadba80b14f14f5b1859f99c78da7b0ef2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T13:36:06.372 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:06 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:06.269+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:06.372 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:06 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:06.269+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:06.426 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:06 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:06.163+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:06.427 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:06 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:36:06.407+0000 7fcf4af2c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:28.669896+0000 front 2024-04-22T13:35:28.670069+0000 (oldest deadline 2024-04-22T13:35:54.569846+0000) 2024-04-22T13:36:06.427 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:06 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:36:06.407+0000 7fcf4af2c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:40.371877+0000 front 2024-04-22T13:35:40.371666+0000 (oldest deadline 2024-04-22T13:36:04.471635+0000) 2024-04-22T13:36:06.427 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:06 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:06.260+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:06.427 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:06 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:06.260+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:06.632 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 22 13:36:06 smithi111 podman[68281]: 2024-04-22 13:36:06.280382212 +0000 UTC m=+0.017897962 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:36:06.633 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 22 13:36:06 smithi111 podman[68281]: 2024-04-22 13:36:06.401354586 +0000 UTC m=+0.138870324 container create b5f92a4165aecfc626dbeb57d4d70740e0bdea98cd013beed1d87e792eba1f12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-22T13:36:06.633 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 22 13:36:06 smithi111 podman[68281]: 2024-04-22 13:36:06.637595399 +0000 UTC m=+0.375111132 container init b5f92a4165aecfc626dbeb57d4d70740e0bdea98cd013beed1d87e792eba1f12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-22T13:36:06.836 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:06 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:06.402+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:30.866019+0000 front 2024-04-22T13:35:30.866082+0000 (oldest deadline 2024-04-22T13:35:53.766157+0000) 2024-04-22T13:36:06.836 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:06 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:06.402+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.067439+0000 front 2024-04-22T13:35:39.067262+0000 (oldest deadline 2024-04-22T13:36:03.767273+0000) 2024-04-22T13:36:06.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:06 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:06.564+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:06.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:06 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:06.564+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:06.918 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:06 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:06.649+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:38.436756+0000 front 2024-04-22T13:35:38.436613+0000 (oldest deadline 2024-04-22T13:36:03.136757+0000) 2024-04-22T13:36:06.918 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 22 13:36:06 smithi111 podman[68281]: 2024-04-22 13:36:06.640680053 +0000 UTC m=+0.378195786 container start b5f92a4165aecfc626dbeb57d4d70740e0bdea98cd013beed1d87e792eba1f12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-22T13:36:06.919 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 22 13:36:06 smithi111 podman[68281]: 2024-04-22 13:36:06.737631654 +0000 UTC m=+0.475147404 container attach b5f92a4165aecfc626dbeb57d4d70740e0bdea98cd013beed1d87e792eba1f12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-4-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-22T13:36:06.919 INFO:journalctl@ceph.osd.4.smithi111.stdout:Apr 22 13:36:06 smithi111 podman[68281]: 2024-04-22 13:36:06.923416566 +0000 UTC m=+0.660932298 container died b5f92a4165aecfc626dbeb57d4d70740e0bdea98cd013beed1d87e792eba1f12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-22T13:36:07.426 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:07 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:07.157+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:07.427 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:07 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:07.274+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:07.427 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:07 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:07.274+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:07.464 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:07 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:07.281+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:07.464 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:07 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:07.281+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:07.480 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.4.service' 2024-04-22T13:36:07.556 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:36:07.557 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-22T13:36:07.557 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-22T13:36:07.557 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.5 2024-04-22T13:36:07.726 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:07 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:07.643+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:38.436756+0000 front 2024-04-22T13:35:38.436613+0000 (oldest deadline 2024-04-22T13:36:03.136757+0000) 2024-04-22T13:36:07.836 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:07 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:07.384+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:30.866019+0000 front 2024-04-22T13:35:30.866082+0000 (oldest deadline 2024-04-22T13:35:53.766157+0000) 2024-04-22T13:36:07.837 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:07 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:07.384+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.067439+0000 front 2024-04-22T13:35:39.067262+0000 (oldest deadline 2024-04-22T13:36:03.767273+0000) 2024-04-22T13:36:07.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:07 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:07.608+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:07.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:07 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:07.608+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:07.859 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:07 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:36:07.438+0000 7fcf4af2c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:28.669896+0000 front 2024-04-22T13:35:28.670069+0000 (oldest deadline 2024-04-22T13:35:54.569846+0000) 2024-04-22T13:36:07.860 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:07 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:36:07.438+0000 7fcf4af2c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:40.371877+0000 front 2024-04-22T13:35:40.371666+0000 (oldest deadline 2024-04-22T13:36:04.471635+0000) 2024-04-22T13:36:07.860 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:07 smithi134 systemd[1]: Stopping Ceph osd.5 for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:36:07.860 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:07 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:36:07.863+0000 7fcf4e938700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:36:07.860 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:07 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:36:07.863+0000 7fcf4e938700 -1 osd.5 103 *** Got signal Terminated *** 2024-04-22T13:36:07.860 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:07 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:36:07.863+0000 7fcf4e938700 -1 osd.5 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T13:36:08.405 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:08 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:08.144+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:08.405 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:08 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:08.250+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:08.405 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:08 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:08.250+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:08.621 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:08 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:08.256+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:08.621 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:08 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:08.256+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:08.676 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:08 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:36:08.409+0000 7fcf4af2c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:28.669896+0000 front 2024-04-22T13:35:28.670069+0000 (oldest deadline 2024-04-22T13:35:54.569846+0000) 2024-04-22T13:36:08.677 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:08 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:36:08.409+0000 7fcf4af2c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:40.371877+0000 front 2024-04-22T13:35:40.371666+0000 (oldest deadline 2024-04-22T13:36:04.471635+0000) 2024-04-22T13:36:08.836 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:08 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:08.427+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:30.866019+0000 front 2024-04-22T13:35:30.866082+0000 (oldest deadline 2024-04-22T13:35:53.766157+0000) 2024-04-22T13:36:08.836 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:08 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:08.427+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.067439+0000 front 2024-04-22T13:35:39.067262+0000 (oldest deadline 2024-04-22T13:36:03.767273+0000) 2024-04-22T13:36:08.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:08 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:08.603+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:08.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:08 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:08.603+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:08.976 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:08 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:08.625+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:38.436756+0000 front 2024-04-22T13:35:38.436613+0000 (oldest deadline 2024-04-22T13:36:03.136757+0000) 2024-04-22T13:36:09.372 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:09 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:09.102+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:09.373 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:09 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:09.224+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:09.373 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:09 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:09.224+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:09.645 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:09 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:09.292+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:09.645 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:09 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:09.292+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:09.676 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:09 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:36:09.377+0000 7fcf4af2c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:28.669896+0000 front 2024-04-22T13:35:28.670069+0000 (oldest deadline 2024-04-22T13:35:54.569846+0000) 2024-04-22T13:36:09.676 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:09 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:36:09.377+0000 7fcf4af2c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:40.371877+0000 front 2024-04-22T13:35:40.371666+0000 (oldest deadline 2024-04-22T13:36:04.471635+0000) 2024-04-22T13:36:09.836 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:09 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:09.403+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:30.866019+0000 front 2024-04-22T13:35:30.866082+0000 (oldest deadline 2024-04-22T13:35:53.766157+0000) 2024-04-22T13:36:09.836 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:09 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:09.403+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.067439+0000 front 2024-04-22T13:35:39.067262+0000 (oldest deadline 2024-04-22T13:36:03.767273+0000) 2024-04-22T13:36:09.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:09 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:09.604+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:09.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:09 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:09.604+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:09.976 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:09 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:09.649+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:38.436756+0000 front 2024-04-22T13:35:38.436613+0000 (oldest deadline 2024-04-22T13:36:03.136757+0000) 2024-04-22T13:36:10.416 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:10 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:10.071+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:10.416 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:10 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:10.228+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:10.417 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:10 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:10.228+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:10.606 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:10 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:10.259+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:10.607 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:10 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:10.259+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:10.607 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:10 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:10.259+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:10.676 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:10 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:36:10.421+0000 7fcf4af2c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:28.669896+0000 front 2024-04-22T13:35:28.670069+0000 (oldest deadline 2024-04-22T13:35:54.569846+0000) 2024-04-22T13:36:10.677 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:10 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:36:10.421+0000 7fcf4af2c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:40.371877+0000 front 2024-04-22T13:35:40.371666+0000 (oldest deadline 2024-04-22T13:36:04.471635+0000) 2024-04-22T13:36:10.677 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:10 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:36:10.421+0000 7fcf4af2c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:44.472029+0000 front 2024-04-22T13:35:44.472053+0000 (oldest deadline 2024-04-22T13:36:09.772218+0000) 2024-04-22T13:36:10.836 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:10 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:10.386+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:30.866019+0000 front 2024-04-22T13:35:30.866082+0000 (oldest deadline 2024-04-22T13:35:53.766157+0000) 2024-04-22T13:36:10.836 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:10 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:10.386+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.067439+0000 front 2024-04-22T13:35:39.067262+0000 (oldest deadline 2024-04-22T13:36:03.767273+0000) 2024-04-22T13:36:10.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:10 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:10.561+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:10.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:10 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:10.561+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:10.976 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:10 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:10.610+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:38.436756+0000 front 2024-04-22T13:35:38.436613+0000 (oldest deadline 2024-04-22T13:36:03.136757+0000) 2024-04-22T13:36:11.426 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:11 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:11.062+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:11.427 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:11 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:11.189+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:11.427 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:11 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:11.189+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:11.567 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:11 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:11.277+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:11.567 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:11 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:11.277+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:11.567 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:11 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:11.277+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:11.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:11 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:11.540+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:11.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:11 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:11.540+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:11.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:11 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:11.540+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:11.837 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:11 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:11.354+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:30.866019+0000 front 2024-04-22T13:35:30.866082+0000 (oldest deadline 2024-04-22T13:35:53.766157+0000) 2024-04-22T13:36:11.837 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:11 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:11.354+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.067439+0000 front 2024-04-22T13:35:39.067262+0000 (oldest deadline 2024-04-22T13:36:03.767273+0000) 2024-04-22T13:36:11.837 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:11 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:11.354+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:46.970435+0000 front 2024-04-22T13:35:46.970345+0000 (oldest deadline 2024-04-22T13:36:10.470288+0000) 2024-04-22T13:36:11.926 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:11 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:36:11.446+0000 7fcf4af2c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:28.669896+0000 front 2024-04-22T13:35:28.670069+0000 (oldest deadline 2024-04-22T13:35:54.569846+0000) 2024-04-22T13:36:11.927 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:11 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:36:11.446+0000 7fcf4af2c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:40.371877+0000 front 2024-04-22T13:35:40.371666+0000 (oldest deadline 2024-04-22T13:36:04.471635+0000) 2024-04-22T13:36:11.927 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:11 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:36:11.446+0000 7fcf4af2c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:44.472029+0000 front 2024-04-22T13:35:44.472053+0000 (oldest deadline 2024-04-22T13:36:09.772218+0000) 2024-04-22T13:36:11.976 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:11 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:11.571+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:38.436756+0000 front 2024-04-22T13:35:38.436613+0000 (oldest deadline 2024-04-22T13:36:03.136757+0000) 2024-04-22T13:36:12.426 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:12 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:12.036+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:12.427 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:12 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:12.223+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:12.427 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:12 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:12.223+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:12.582 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:12 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:12.287+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:12.583 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:12 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:12.287+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:12.583 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:12 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:12.287+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:12.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:12 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:12.508+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:12.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:12 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:12.508+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:12.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:12 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:12.508+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:12.836 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:12 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:12.402+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:30.866019+0000 front 2024-04-22T13:35:30.866082+0000 (oldest deadline 2024-04-22T13:35:53.766157+0000) 2024-04-22T13:36:12.837 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:12 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:12.402+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.067439+0000 front 2024-04-22T13:35:39.067262+0000 (oldest deadline 2024-04-22T13:36:03.767273+0000) 2024-04-22T13:36:12.837 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:12 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:12.402+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:46.970435+0000 front 2024-04-22T13:35:46.970345+0000 (oldest deadline 2024-04-22T13:36:10.470288+0000) 2024-04-22T13:36:12.927 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:12 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:36:12.444+0000 7fcf4af2c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:28.669896+0000 front 2024-04-22T13:35:28.670069+0000 (oldest deadline 2024-04-22T13:35:54.569846+0000) 2024-04-22T13:36:12.927 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:12 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:36:12.444+0000 7fcf4af2c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:40.371877+0000 front 2024-04-22T13:35:40.371666+0000 (oldest deadline 2024-04-22T13:36:04.471635+0000) 2024-04-22T13:36:12.927 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:12 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5[43415]: 2024-04-22T13:36:12.444+0000 7fcf4af2c700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:44.472029+0000 front 2024-04-22T13:35:44.472053+0000 (oldest deadline 2024-04-22T13:36:09.772218+0000) 2024-04-22T13:36:12.976 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:12 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:12.587+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:38.436756+0000 front 2024-04-22T13:35:38.436613+0000 (oldest deadline 2024-04-22T13:36:03.136757+0000) 2024-04-22T13:36:12.976 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:12 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:12.587+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:46.037898+0000 front 2024-04-22T13:35:46.037862+0000 (oldest deadline 2024-04-22T13:36:11.937748+0000) 2024-04-22T13:36:13.427 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:13 smithi134 podman[68615]: 2024-04-22 13:36:13.270136741 +0000 UTC m=+5.513189611 container died 3fb7405bf104f115868d9dc6686b359c56d61dd5a9281c7308c774077e386bd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-22T13:36:13.427 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:13 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:13.262+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:13.427 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:13 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:13.262+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:13.427 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:13 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:13.053+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:13.591 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:13 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:13.256+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:13.592 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:13 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:13.256+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:13.592 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:13 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:13.256+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:13.794 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:13 smithi134 podman[68615]: 2024-04-22 13:36:13.589197232 +0000 UTC m=+5.832250104 container cleanup 3fb7405bf104f115868d9dc6686b359c56d61dd5a9281c7308c774077e386bd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=) 2024-04-22T13:36:13.794 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:13 smithi134 bash[68615]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5 2024-04-22T13:36:13.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:13 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:13.515+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:13.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:13 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:13.515+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:13.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:13 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:13.515+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:13.837 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:13 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:13.368+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:30.866019+0000 front 2024-04-22T13:35:30.866082+0000 (oldest deadline 2024-04-22T13:35:53.766157+0000) 2024-04-22T13:36:13.837 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:13 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:13.368+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.067439+0000 front 2024-04-22T13:35:39.067262+0000 (oldest deadline 2024-04-22T13:36:03.767273+0000) 2024-04-22T13:36:13.837 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:13 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:13.368+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:46.970435+0000 front 2024-04-22T13:35:46.970345+0000 (oldest deadline 2024-04-22T13:36:10.470288+0000) 2024-04-22T13:36:13.976 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:13 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:13.596+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:38.436756+0000 front 2024-04-22T13:35:38.436613+0000 (oldest deadline 2024-04-22T13:36:03.136757+0000) 2024-04-22T13:36:13.976 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:13 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:13.596+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:46.037898+0000 front 2024-04-22T13:35:46.037862+0000 (oldest deadline 2024-04-22T13:36:11.937748+0000) 2024-04-22T13:36:14.177 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:13 smithi134 podman[68632]: 2024-04-22 13:36:13.798643401 +0000 UTC m=+0.529844927 container remove 3fb7405bf104f115868d9dc6686b359c56d61dd5a9281c7308c774077e386bd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-22T13:36:14.177 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:14 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:14.011+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:14.177 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:14 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:14.011+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:45.259185+0000 front 2024-04-22T13:35:45.259222+0000 (oldest deadline 2024-04-22T13:36:13.459981+0000) 2024-04-22T13:36:14.455 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:14 smithi134 podman[68721]: 2024-04-22 13:36:14.108829457 +0000 UTC m=+0.018042598 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:36:14.455 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:14 smithi134 podman[68721]: 2024-04-22 13:36:14.216239926 +0000 UTC m=+0.125453039 container create 401662fd38de247d600db59352f7d4d611742778c37ce6f63fd65c9e9b4769f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-22T13:36:14.455 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:14 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:14.234+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:14.455 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:14 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:14.234+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:14.455 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:14 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:14.234+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:14.576 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:14 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:14.293+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:14.576 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:14 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:14.293+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:14.576 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:14 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:14.293+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:14.750 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:14 smithi134 podman[68721]: 2024-04-22 13:36:14.460415993 +0000 UTC m=+0.369629106 container init 401662fd38de247d600db59352f7d4d611742778c37ce6f63fd65c9e9b4769f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-22T13:36:14.750 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:14 smithi134 podman[68721]: 2024-04-22 13:36:14.463603822 +0000 UTC m=+0.372816936 container start 401662fd38de247d600db59352f7d4d611742778c37ce6f63fd65c9e9b4769f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=) 2024-04-22T13:36:14.750 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:14 smithi134 podman[68721]: 2024-04-22 13:36:14.535428571 +0000 UTC m=+0.444641693 container attach 401662fd38de247d600db59352f7d4d611742778c37ce6f63fd65c9e9b4769f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-22T13:36:14.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:14 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:14.526+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:14.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:14 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:14.526+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:14.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:14 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:14.526+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:14.837 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:14 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:14.384+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:30.866019+0000 front 2024-04-22T13:35:30.866082+0000 (oldest deadline 2024-04-22T13:35:53.766157+0000) 2024-04-22T13:36:14.837 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:14 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:14.384+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.067439+0000 front 2024-04-22T13:35:39.067262+0000 (oldest deadline 2024-04-22T13:36:03.767273+0000) 2024-04-22T13:36:14.837 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:14 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:14.384+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:46.970435+0000 front 2024-04-22T13:35:46.970345+0000 (oldest deadline 2024-04-22T13:36:10.470288+0000) 2024-04-22T13:36:14.976 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:14 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:14.580+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:38.436756+0000 front 2024-04-22T13:35:38.436613+0000 (oldest deadline 2024-04-22T13:36:03.136757+0000) 2024-04-22T13:36:14.976 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:14 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:14.580+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:46.037898+0000 front 2024-04-22T13:35:46.037862+0000 (oldest deadline 2024-04-22T13:36:11.937748+0000) 2024-04-22T13:36:15.028 INFO:journalctl@ceph.osd.5.smithi134.stdout:Apr 22 13:36:14 smithi134 podman[68721]: 2024-04-22 13:36:14.755052442 +0000 UTC m=+0.664265561 container died 401662fd38de247d600db59352f7d4d611742778c37ce6f63fd65c9e9b4769f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T13:36:15.309 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:15 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:15.033+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:15.309 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:15 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:15.033+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:45.259185+0000 front 2024-04-22T13:35:45.259222+0000 (oldest deadline 2024-04-22T13:36:13.459981+0000) 2024-04-22T13:36:15.309 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:15 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:15.252+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:15.309 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:15 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:15.252+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:15.309 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:15 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:15.252+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:15.333 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.5.service' 2024-04-22T13:36:15.372 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:36:15.372 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-22T13:36:15.372 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-22T13:36:15.372 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.6 2024-04-22T13:36:15.618 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:15 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:15.321+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:15.618 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:15 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:15.321+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:15.618 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:15 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:15.321+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:15.665 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:15 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:15.407+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:30.866019+0000 front 2024-04-22T13:35:30.866082+0000 (oldest deadline 2024-04-22T13:35:53.766157+0000) 2024-04-22T13:36:15.666 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:15 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:15.407+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.067439+0000 front 2024-04-22T13:35:39.067262+0000 (oldest deadline 2024-04-22T13:36:03.767273+0000) 2024-04-22T13:36:15.666 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:15 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:15.407+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:46.970435+0000 front 2024-04-22T13:35:46.970345+0000 (oldest deadline 2024-04-22T13:36:10.470288+0000) 2024-04-22T13:36:15.666 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:15 smithi012 systemd[1]: Stopping Ceph osd.6 for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:36:15.666 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:15 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:15.670+0000 7fb75f8c4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:36:15.666 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:15 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:15.670+0000 7fb75f8c4700 -1 osd.6 103 *** Got signal Terminated *** 2024-04-22T13:36:15.666 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:15 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:15.670+0000 7fb75f8c4700 -1 osd.6 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T13:36:15.666 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:15 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:15.576+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:15.666 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:15 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:15.576+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:15.666 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:15 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:15.576+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:15.976 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:15 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:15.622+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:38.436756+0000 front 2024-04-22T13:35:38.436613+0000 (oldest deadline 2024-04-22T13:36:03.136757+0000) 2024-04-22T13:36:15.976 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:15 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:15.622+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:46.037898+0000 front 2024-04-22T13:35:46.037862+0000 (oldest deadline 2024-04-22T13:36:11.937748+0000) 2024-04-22T13:36:16.426 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:15 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:15.990+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:16.427 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:15 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:15.990+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:45.259185+0000 front 2024-04-22T13:35:45.259222+0000 (oldest deadline 2024-04-22T13:36:13.459981+0000) 2024-04-22T13:36:16.427 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:16 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:16.225+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:16.427 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:16 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:16.225+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:16.427 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:16 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:16.225+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:16.726 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:16 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:16.596+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:38.436756+0000 front 2024-04-22T13:35:38.436613+0000 (oldest deadline 2024-04-22T13:36:03.136757+0000) 2024-04-22T13:36:16.726 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:16 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:16.596+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:46.037898+0000 front 2024-04-22T13:35:46.037862+0000 (oldest deadline 2024-04-22T13:36:11.937748+0000) 2024-04-22T13:36:16.726 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:16 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:16.350+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:16.726 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:16 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:16.350+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:16.726 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:16 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:16.350+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:16.836 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:16 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:16.393+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:30.866019+0000 front 2024-04-22T13:35:30.866082+0000 (oldest deadline 2024-04-22T13:35:53.766157+0000) 2024-04-22T13:36:16.837 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:16 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:16.393+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.067439+0000 front 2024-04-22T13:35:39.067262+0000 (oldest deadline 2024-04-22T13:36:03.767273+0000) 2024-04-22T13:36:16.837 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:16 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:16.393+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:46.970435+0000 front 2024-04-22T13:35:46.970345+0000 (oldest deadline 2024-04-22T13:36:10.470288+0000) 2024-04-22T13:36:16.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:16 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:16.623+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:16.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:16 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:16.623+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:16.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:16 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:16.623+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:17.265 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:17 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:17.005+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:17.265 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:17 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:17.005+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:45.259185+0000 front 2024-04-22T13:35:45.259222+0000 (oldest deadline 2024-04-22T13:36:13.459981+0000) 2024-04-22T13:36:17.677 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:17 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:17.269+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:17.677 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:17 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:17.269+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:17.677 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:17 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:17.269+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:17.726 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:17 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:17.572+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:38.436756+0000 front 2024-04-22T13:35:38.436613+0000 (oldest deadline 2024-04-22T13:36:03.136757+0000) 2024-04-22T13:36:17.726 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:17 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:17.572+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:46.037898+0000 front 2024-04-22T13:35:46.037862+0000 (oldest deadline 2024-04-22T13:36:11.937748+0000) 2024-04-22T13:36:17.726 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:17 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:17.336+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:17.726 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:17 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:17.336+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:17.726 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:17 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:17.336+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:17.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:17 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:17.603+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:17.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:17 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:17.603+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:17.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:17 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:17.603+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:17.837 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:17 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:17.380+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:30.866019+0000 front 2024-04-22T13:35:30.866082+0000 (oldest deadline 2024-04-22T13:35:53.766157+0000) 2024-04-22T13:36:17.837 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:17 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:17.380+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.067439+0000 front 2024-04-22T13:35:39.067262+0000 (oldest deadline 2024-04-22T13:36:03.767273+0000) 2024-04-22T13:36:17.837 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:17 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:17.380+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:46.970435+0000 front 2024-04-22T13:35:46.970345+0000 (oldest deadline 2024-04-22T13:36:10.470288+0000) 2024-04-22T13:36:18.314 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:18 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:18.030+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:18.314 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:18 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:18.030+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:45.259185+0000 front 2024-04-22T13:35:45.259222+0000 (oldest deadline 2024-04-22T13:36:13.459981+0000) 2024-04-22T13:36:18.676 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:18 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:18.319+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:18.677 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:18 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:18.319+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:18.677 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:18 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:18.319+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:18.726 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:18 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:18.565+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:38.436756+0000 front 2024-04-22T13:35:38.436613+0000 (oldest deadline 2024-04-22T13:36:03.136757+0000) 2024-04-22T13:36:18.726 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:18 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:18.565+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:46.037898+0000 front 2024-04-22T13:35:46.037862+0000 (oldest deadline 2024-04-22T13:36:11.937748+0000) 2024-04-22T13:36:18.726 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:18 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:18.327+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:18.726 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:18 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:18.327+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:18.726 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:18 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:18.327+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:18.836 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:18 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:18.402+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:30.866019+0000 front 2024-04-22T13:35:30.866082+0000 (oldest deadline 2024-04-22T13:35:53.766157+0000) 2024-04-22T13:36:18.836 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:18 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:18.402+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.067439+0000 front 2024-04-22T13:35:39.067262+0000 (oldest deadline 2024-04-22T13:36:03.767273+0000) 2024-04-22T13:36:18.836 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:18 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:18.402+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:46.970435+0000 front 2024-04-22T13:35:46.970345+0000 (oldest deadline 2024-04-22T13:36:10.470288+0000) 2024-04-22T13:36:18.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:18 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:18.570+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:18.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:18 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:18.570+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:18.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:18 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:18.570+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:19.332 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:18 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:18.993+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:19.332 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:18 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:18.993+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:45.259185+0000 front 2024-04-22T13:35:45.259222+0000 (oldest deadline 2024-04-22T13:36:13.459981+0000) 2024-04-22T13:36:19.332 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:18 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:18.993+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.361108+0000 front 2024-04-22T13:35:56.360977+0000 (oldest deadline 2024-04-22T13:36:18.661036+0000) 2024-04-22T13:36:19.676 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:19 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:19.336+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:19.677 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:19 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:19.336+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:19.677 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:19 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:19.336+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:19.726 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:19 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:19.536+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:38.436756+0000 front 2024-04-22T13:35:38.436613+0000 (oldest deadline 2024-04-22T13:36:03.136757+0000) 2024-04-22T13:36:19.726 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:19 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:19.536+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:46.037898+0000 front 2024-04-22T13:35:46.037862+0000 (oldest deadline 2024-04-22T13:36:11.937748+0000) 2024-04-22T13:36:19.726 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:19 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:19.331+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:19.726 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:19 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:19.331+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:19.726 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:19 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:19.331+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:19.726 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:19 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:19.331+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:19.836 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:19 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:19.449+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:30.866019+0000 front 2024-04-22T13:35:30.866082+0000 (oldest deadline 2024-04-22T13:35:53.766157+0000) 2024-04-22T13:36:19.836 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:19 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:19.449+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.067439+0000 front 2024-04-22T13:35:39.067262+0000 (oldest deadline 2024-04-22T13:36:03.767273+0000) 2024-04-22T13:36:19.836 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:19 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:19.449+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:46.970435+0000 front 2024-04-22T13:35:46.970345+0000 (oldest deadline 2024-04-22T13:36:10.470288+0000) 2024-04-22T13:36:19.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:19 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:19.560+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:19.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:19 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:19.560+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:19.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:19 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:19.560+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:19.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:19 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:19.560+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:54.709908+0000 front 2024-04-22T13:35:54.709763+0000 (oldest deadline 2024-04-22T13:36:18.809896+0000) 2024-04-22T13:36:20.378 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:20 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:20.030+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:20.378 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:20 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:20.030+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:45.259185+0000 front 2024-04-22T13:35:45.259222+0000 (oldest deadline 2024-04-22T13:36:13.459981+0000) 2024-04-22T13:36:20.378 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:20 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:20.030+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.361108+0000 front 2024-04-22T13:35:56.360977+0000 (oldest deadline 2024-04-22T13:36:18.661036+0000) 2024-04-22T13:36:20.676 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:20 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:20.383+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:20.677 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:20 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:20.383+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:20.677 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:20 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:20.383+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:20.726 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:20 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:20.545+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:38.436756+0000 front 2024-04-22T13:35:38.436613+0000 (oldest deadline 2024-04-22T13:36:03.136757+0000) 2024-04-22T13:36:20.726 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:20 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:20.545+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:46.037898+0000 front 2024-04-22T13:35:46.037862+0000 (oldest deadline 2024-04-22T13:36:11.937748+0000) 2024-04-22T13:36:20.726 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:20 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:20.331+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:20.726 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:20 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:20.331+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:20.726 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:20 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:20.331+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:20.726 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:20 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:20.331+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:20.836 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:20 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:20.463+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:30.866019+0000 front 2024-04-22T13:35:30.866082+0000 (oldest deadline 2024-04-22T13:35:53.766157+0000) 2024-04-22T13:36:20.836 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:20 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:20.463+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.067439+0000 front 2024-04-22T13:35:39.067262+0000 (oldest deadline 2024-04-22T13:36:03.767273+0000) 2024-04-22T13:36:20.836 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:20 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6[56517]: 2024-04-22T13:36:20.463+0000 7fb757eb0700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:46.970435+0000 front 2024-04-22T13:35:46.970345+0000 (oldest deadline 2024-04-22T13:36:10.470288+0000) 2024-04-22T13:36:20.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:20 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:20.603+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:20.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:20 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:20.603+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:20.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:20 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:20.603+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:20.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:20 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:20.603+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:54.709908+0000 front 2024-04-22T13:35:54.709763+0000 (oldest deadline 2024-04-22T13:36:18.809896+0000) 2024-04-22T13:36:21.290 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:21 smithi012 podman[101948]: 2024-04-22 13:36:21.024868483 +0000 UTC m=+5.450410811 container died 0c2b48670cd5e5f6fb9fd3b990f2708582ba35ac7bd4caf4defa32d1903e515b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-22T13:36:21.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:20 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:20.992+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:21.381 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:20 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:20.992+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:45.259185+0000 front 2024-04-22T13:35:45.259222+0000 (oldest deadline 2024-04-22T13:36:13.459981+0000) 2024-04-22T13:36:21.382 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:20 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:20.992+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.361108+0000 front 2024-04-22T13:35:56.360977+0000 (oldest deadline 2024-04-22T13:36:18.661036+0000) 2024-04-22T13:36:21.546 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:21 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:21.298+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:21.546 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:21 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:21.298+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:21.546 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:21 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:21.298+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:21.546 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:21 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:21.298+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:21.586 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:21 smithi012 podman[101948]: 2024-04-22 13:36:21.394828033 +0000 UTC m=+5.820370369 container cleanup 0c2b48670cd5e5f6fb9fd3b990f2708582ba35ac7bd4caf4defa32d1903e515b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-22T13:36:21.586 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:21 smithi012 bash[101948]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6 2024-04-22T13:36:21.658 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:21 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:21.386+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:21.658 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:21 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:21.386+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:21.658 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:21 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:21.386+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:21.976 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:21 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:21.551+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:38.436756+0000 front 2024-04-22T13:35:38.436613+0000 (oldest deadline 2024-04-22T13:36:03.136757+0000) 2024-04-22T13:36:21.976 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:21 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:21.551+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:46.037898+0000 front 2024-04-22T13:35:46.037862+0000 (oldest deadline 2024-04-22T13:36:11.937748+0000) 2024-04-22T13:36:22.022 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:21 smithi012 podman[101965]: 2024-04-22 13:36:21.687612284 +0000 UTC m=+0.659449244 container remove 0c2b48670cd5e5f6fb9fd3b990f2708582ba35ac7bd4caf4defa32d1903e515b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T13:36:22.022 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:21 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:21.591+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:22.022 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:21 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:21.591+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:22.022 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:21 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:21.591+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:22.022 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:21 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:21.591+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:54.709908+0000 front 2024-04-22T13:35:54.709763+0000 (oldest deadline 2024-04-22T13:36:18.809896+0000) 2024-04-22T13:36:22.327 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:22 smithi012 podman[102054]: 2024-04-22 13:36:21.930792527 +0000 UTC m=+0.018422529 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:36:22.327 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:22 smithi012 podman[102054]: 2024-04-22 13:36:22.029750994 +0000 UTC m=+0.117380981 container create 3f29819048d8fef70a18c97099976f2e273d9e62d07ed8eeefb9c0161ee72d36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2) 2024-04-22T13:36:22.327 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:22 smithi012 podman[102054]: 2024-04-22 13:36:22.24911315 +0000 UTC m=+0.336743137 container init 3f29819048d8fef70a18c97099976f2e273d9e62d07ed8eeefb9c0161ee72d36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-04-22T13:36:22.327 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:22 smithi012 podman[102054]: 2024-04-22 13:36:22.252238855 +0000 UTC m=+0.339868843 container start 3f29819048d8fef70a18c97099976f2e273d9e62d07ed8eeefb9c0161ee72d36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-22T13:36:22.327 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:22 smithi012 podman[102054]: 2024-04-22 13:36:22.332228185 +0000 UTC m=+0.419858174 container attach 3f29819048d8fef70a18c97099976f2e273d9e62d07ed8eeefb9c0161ee72d36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T13:36:22.426 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:22 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:22.031+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:22.427 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:22 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:22.031+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:45.259185+0000 front 2024-04-22T13:35:45.259222+0000 (oldest deadline 2024-04-22T13:36:13.459981+0000) 2024-04-22T13:36:22.427 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:22 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:22.031+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.361108+0000 front 2024-04-22T13:35:56.360977+0000 (oldest deadline 2024-04-22T13:36:18.661036+0000) 2024-04-22T13:36:22.540 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:22 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:22.280+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:22.541 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:22 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:22.280+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:22.541 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:22 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:22.280+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:22.541 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:22 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:22.280+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:22.586 INFO:journalctl@ceph.osd.6.smithi012.stdout:Apr 22 13:36:22 smithi012 podman[102054]: 2024-04-22 13:36:22.564781173 +0000 UTC m=+0.652411163 container died 3f29819048d8fef70a18c97099976f2e273d9e62d07ed8eeefb9c0161ee72d36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-04-22T13:36:22.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:22 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:22.433+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:22.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:22 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:22.433+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:22.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:22 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:22.433+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:22.976 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:22 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:22.545+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:38.436756+0000 front 2024-04-22T13:35:38.436613+0000 (oldest deadline 2024-04-22T13:36:03.136757+0000) 2024-04-22T13:36:22.976 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:22 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:22.545+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:46.037898+0000 front 2024-04-22T13:35:46.037862+0000 (oldest deadline 2024-04-22T13:36:11.937748+0000) 2024-04-22T13:36:22.976 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:22 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:22.545+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.638820+0000 front 2024-04-22T13:35:56.638794+0000 (oldest deadline 2024-04-22T13:36:21.938817+0000) 2024-04-22T13:36:23.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:22 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:22.638+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:23.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:22 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:22.638+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:23.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:22 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:22.638+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:23.087 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:22 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:22.638+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:54.709908+0000 front 2024-04-22T13:35:54.709763+0000 (oldest deadline 2024-04-22T13:36:18.809896+0000) 2024-04-22T13:36:23.134 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.6.service' 2024-04-22T13:36:23.175 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:36:23.175 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-22T13:36:23.175 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-22T13:36:23.175 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.7 2024-04-22T13:36:23.426 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:23 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:23.038+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:23.427 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:23 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:23.038+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:45.259185+0000 front 2024-04-22T13:35:45.259222+0000 (oldest deadline 2024-04-22T13:36:13.459981+0000) 2024-04-22T13:36:23.427 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:23 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:23.038+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.361108+0000 front 2024-04-22T13:35:56.360977+0000 (oldest deadline 2024-04-22T13:36:18.661036+0000) 2024-04-22T13:36:23.476 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:23 smithi111 systemd[1]: Stopping Ceph osd.7 for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:36:23.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:23 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:23.282+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:23.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:23 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:23.282+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:23.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:23 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:23.282+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:23.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:23 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:23.282+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:23.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:23 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:23.614+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:23.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:23 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:23.614+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:23.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:23 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:23.614+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:23.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:23 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:23.614+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:54.709908+0000 front 2024-04-22T13:35:54.709763+0000 (oldest deadline 2024-04-22T13:36:18.809896+0000) 2024-04-22T13:36:23.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:23 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:23.461+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:23.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:23 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:23.461+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:23.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:23 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:23.461+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:23.976 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:23 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:23.504+0000 7f11b69de700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:36:23.976 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:23 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:23.504+0000 7f11b69de700 -1 osd.7 103 *** Got signal Terminated *** 2024-04-22T13:36:23.976 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:23 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:23.504+0000 7f11b69de700 -1 osd.7 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T13:36:23.977 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:23 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:23.562+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:38.436756+0000 front 2024-04-22T13:35:38.436613+0000 (oldest deadline 2024-04-22T13:36:03.136757+0000) 2024-04-22T13:36:23.977 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:23 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:23.562+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:46.037898+0000 front 2024-04-22T13:35:46.037862+0000 (oldest deadline 2024-04-22T13:36:11.937748+0000) 2024-04-22T13:36:23.977 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:23 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:23.562+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.638820+0000 front 2024-04-22T13:35:56.638794+0000 (oldest deadline 2024-04-22T13:36:21.938817+0000) 2024-04-22T13:36:24.426 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:24 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:24.063+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:24.427 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:24 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:24.063+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:45.259185+0000 front 2024-04-22T13:35:45.259222+0000 (oldest deadline 2024-04-22T13:36:13.459981+0000) 2024-04-22T13:36:24.427 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:24 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:24.063+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.361108+0000 front 2024-04-22T13:35:56.360977+0000 (oldest deadline 2024-04-22T13:36:18.661036+0000) 2024-04-22T13:36:24.570 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:24 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:24.265+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:24.570 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:24 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:24.265+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:24.570 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:24 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:24.265+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:24.571 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:24 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:24.265+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:24.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:24 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:24.599+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:24.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:24 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:24.599+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:24.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:24 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:24.599+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:24.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:24 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:24.599+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:54.709908+0000 front 2024-04-22T13:35:54.709763+0000 (oldest deadline 2024-04-22T13:36:18.809896+0000) 2024-04-22T13:36:24.926 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:24 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:24.496+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:24.926 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:24 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:24.496+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:24.926 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:24 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:24.496+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:24.976 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:24 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:24.575+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:38.436756+0000 front 2024-04-22T13:35:38.436613+0000 (oldest deadline 2024-04-22T13:36:03.136757+0000) 2024-04-22T13:36:24.976 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:24 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:24.575+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:46.037898+0000 front 2024-04-22T13:35:46.037862+0000 (oldest deadline 2024-04-22T13:36:11.937748+0000) 2024-04-22T13:36:24.976 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:24 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:24.575+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.638820+0000 front 2024-04-22T13:35:56.638794+0000 (oldest deadline 2024-04-22T13:36:21.938817+0000) 2024-04-22T13:36:25.426 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:25 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:25.028+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:25.427 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:25 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:25.028+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:45.259185+0000 front 2024-04-22T13:35:45.259222+0000 (oldest deadline 2024-04-22T13:36:13.459981+0000) 2024-04-22T13:36:25.427 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:25 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:25.028+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.361108+0000 front 2024-04-22T13:35:56.360977+0000 (oldest deadline 2024-04-22T13:36:18.661036+0000) 2024-04-22T13:36:25.547 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:25 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:25.239+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:25.547 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:25 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:25.239+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:25.547 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:25 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:25.239+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:25.547 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:25 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:25.239+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:25.846 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:25 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:25.551+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:38.436756+0000 front 2024-04-22T13:35:38.436613+0000 (oldest deadline 2024-04-22T13:36:03.136757+0000) 2024-04-22T13:36:25.846 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:25 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:25.551+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:46.037898+0000 front 2024-04-22T13:35:46.037862+0000 (oldest deadline 2024-04-22T13:36:11.937748+0000) 2024-04-22T13:36:25.846 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:25 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:25.551+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.638820+0000 front 2024-04-22T13:35:56.638794+0000 (oldest deadline 2024-04-22T13:36:21.938817+0000) 2024-04-22T13:36:25.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:25 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:25.535+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:25.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:25 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:25.535+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:25.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:25 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:25.535+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:26.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:25 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:25.647+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:26.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:25 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:25.647+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:26.087 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:25 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:25.647+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:26.087 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:25 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:25.647+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:54.709908+0000 front 2024-04-22T13:35:54.709763+0000 (oldest deadline 2024-04-22T13:36:18.809896+0000) 2024-04-22T13:36:26.427 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:26 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:26.009+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:26.427 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:26 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:26.009+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:45.259185+0000 front 2024-04-22T13:35:45.259222+0000 (oldest deadline 2024-04-22T13:36:13.459981+0000) 2024-04-22T13:36:26.427 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:26 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:26.009+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.361108+0000 front 2024-04-22T13:35:56.360977+0000 (oldest deadline 2024-04-22T13:36:18.661036+0000) 2024-04-22T13:36:26.526 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:26 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:26.279+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:26.527 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:26 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:26.279+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:26.527 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:26 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:26.279+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:26.527 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:26 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:26.279+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:26.783 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:26 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:26.530+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:38.436756+0000 front 2024-04-22T13:35:38.436613+0000 (oldest deadline 2024-04-22T13:36:03.136757+0000) 2024-04-22T13:36:26.784 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:26 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:26.530+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:46.037898+0000 front 2024-04-22T13:35:46.037862+0000 (oldest deadline 2024-04-22T13:36:11.937748+0000) 2024-04-22T13:36:26.784 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:26 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:26.530+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.638820+0000 front 2024-04-22T13:35:56.638794+0000 (oldest deadline 2024-04-22T13:36:21.938817+0000) 2024-04-22T13:36:26.926 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:26 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:26.564+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:26.926 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:26 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:26.564+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:26.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:26 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:26.564+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:26.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:26 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:26.564+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:27.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:26 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:26.661+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:27.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:26 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:26.661+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:27.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:26 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:26.661+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:27.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:26 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:26.661+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:54.709908+0000 front 2024-04-22T13:35:54.709763+0000 (oldest deadline 2024-04-22T13:36:18.809896+0000) 2024-04-22T13:36:27.426 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:27 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:27.035+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:27.427 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:27 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:27.035+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:45.259185+0000 front 2024-04-22T13:35:45.259222+0000 (oldest deadline 2024-04-22T13:36:13.459981+0000) 2024-04-22T13:36:27.427 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:27 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:27.035+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.361108+0000 front 2024-04-22T13:35:56.360977+0000 (oldest deadline 2024-04-22T13:36:18.661036+0000) 2024-04-22T13:36:27.427 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:27 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:27.035+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:00.961980+0000 front 2024-04-22T13:36:00.961935+0000 (oldest deadline 2024-04-22T13:36:26.261844+0000) 2024-04-22T13:36:27.523 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:27 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:27.231+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:27.523 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:27 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:27.231+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:27.523 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:27 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:27.231+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:27.523 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:27 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:27.231+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:27.926 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:27 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:27.516+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:27.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:27 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:27.516+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:27.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:27 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:27.516+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:27.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:27 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:27.516+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:27.976 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:27 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:27.527+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:38.436756+0000 front 2024-04-22T13:35:38.436613+0000 (oldest deadline 2024-04-22T13:36:03.136757+0000) 2024-04-22T13:36:27.976 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:27 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:27.527+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:46.037898+0000 front 2024-04-22T13:35:46.037862+0000 (oldest deadline 2024-04-22T13:36:11.937748+0000) 2024-04-22T13:36:27.976 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:27 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:27.527+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.638820+0000 front 2024-04-22T13:35:56.638794+0000 (oldest deadline 2024-04-22T13:36:21.938817+0000) 2024-04-22T13:36:27.976 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:27 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7[49799]: 2024-04-22T13:36:27.527+0000 7f11aefca700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:01.939203+0000 front 2024-04-22T13:36:01.939156+0000 (oldest deadline 2024-04-22T13:36:27.239389+0000) 2024-04-22T13:36:28.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:27 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:27.679+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:28.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:27 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:27.679+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:28.087 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:27 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:27.679+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:28.087 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:27 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:27.679+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:54.709908+0000 front 2024-04-22T13:35:54.709763+0000 (oldest deadline 2024-04-22T13:36:18.809896+0000) 2024-04-22T13:36:28.427 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:28 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:28.033+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:28.427 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:28 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:28.033+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:45.259185+0000 front 2024-04-22T13:35:45.259222+0000 (oldest deadline 2024-04-22T13:36:13.459981+0000) 2024-04-22T13:36:28.427 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:28 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:28.033+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.361108+0000 front 2024-04-22T13:35:56.360977+0000 (oldest deadline 2024-04-22T13:36:18.661036+0000) 2024-04-22T13:36:28.427 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:28 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:28.033+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:00.961980+0000 front 2024-04-22T13:36:00.961935+0000 (oldest deadline 2024-04-22T13:36:26.261844+0000) 2024-04-22T13:36:28.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:28 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:28.215+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:28.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:28 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:28.215+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:28.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:28 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:28.215+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:28.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:28 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:28.215+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:28.926 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:28 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:28.518+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:28.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:28 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:28.518+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:28.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:28 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:28.518+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:28.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:28 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:28.518+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:29.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:28 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:28.654+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:29.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:28 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:28.654+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:29.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:28 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:28.654+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:29.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:28 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:28.654+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:54.709908+0000 front 2024-04-22T13:35:54.709763+0000 (oldest deadline 2024-04-22T13:36:18.809896+0000) 2024-04-22T13:36:29.133 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:28 smithi111 podman[68389]: 2024-04-22 13:36:28.871111616 +0000 UTC m=+5.487842270 container died 20104481e22c42d5f2ec57d4dcdbb43904e91730c515f1326cd1a18f625104e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-22T13:36:29.427 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:28 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:28.998+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:29.427 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:28 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:28.998+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:45.259185+0000 front 2024-04-22T13:35:45.259222+0000 (oldest deadline 2024-04-22T13:36:13.459981+0000) 2024-04-22T13:36:29.427 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:28 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:28.998+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.361108+0000 front 2024-04-22T13:35:56.360977+0000 (oldest deadline 2024-04-22T13:36:18.661036+0000) 2024-04-22T13:36:29.427 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:28 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:28.998+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:00.961980+0000 front 2024-04-22T13:36:00.961935+0000 (oldest deadline 2024-04-22T13:36:26.261844+0000) 2024-04-22T13:36:29.476 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:29 smithi111 podman[68389]: 2024-04-22 13:36:29.229074149 +0000 UTC m=+5.845804795 container cleanup 20104481e22c42d5f2ec57d4dcdbb43904e91730c515f1326cd1a18f625104e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T13:36:29.476 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:29 smithi111 bash[68389]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7 2024-04-22T13:36:29.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:29 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:29.169+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:29.477 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:29 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:29.169+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:29.477 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:29 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:29.169+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:29.477 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:29 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:29.169+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:29.477 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:29 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:29.169+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.500117+0000 front 2024-04-22T13:36:04.500141+0000 (oldest deadline 2024-04-22T13:36:28.600151+0000) 2024-04-22T13:36:29.841 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:29 smithi111 podman[68420]: 2024-04-22 13:36:29.513548252 +0000 UTC m=+0.639492785 container remove 20104481e22c42d5f2ec57d4dcdbb43904e91730c515f1326cd1a18f625104e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-22T13:36:29.926 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:29 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:29.513+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:29.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:29 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:29.513+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:29.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:29 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:29.513+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:29.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:29 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:29.513+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:30.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:29 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:29.698+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:30.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:29 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:29.698+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:30.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:29 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:29.698+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:30.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:29 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:29.698+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:54.709908+0000 front 2024-04-22T13:35:54.709763+0000 (oldest deadline 2024-04-22T13:36:18.809896+0000) 2024-04-22T13:36:30.120 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:29 smithi111 podman[68508]: 2024-04-22 13:36:29.749076142 +0000 UTC m=+0.018215988 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:36:30.120 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:29 smithi111 podman[68508]: 2024-04-22 13:36:29.864695401 +0000 UTC m=+0.133835239 container create a10d375954ba7f42d3df58135f8eb4922df7ecc944535ac78c7e3cb493764e5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-22T13:36:30.120 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:30 smithi111 podman[68508]: 2024-04-22 13:36:30.066953591 +0000 UTC m=+0.336093429 container init a10d375954ba7f42d3df58135f8eb4922df7ecc944535ac78c7e3cb493764e5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-22T13:36:30.120 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:30 smithi111 podman[68508]: 2024-04-22 13:36:30.070261345 +0000 UTC m=+0.339401188 container start a10d375954ba7f42d3df58135f8eb4922df7ecc944535ac78c7e3cb493764e5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-22T13:36:30.120 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:30 smithi111 podman[68508]: 2024-04-22 13:36:30.125356683 +0000 UTC m=+0.394496547 container attach a10d375954ba7f42d3df58135f8eb4922df7ecc944535ac78c7e3cb493764e5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T13:36:30.426 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:30 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:30.000+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:30.427 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:30 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:30.000+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:45.259185+0000 front 2024-04-22T13:35:45.259222+0000 (oldest deadline 2024-04-22T13:36:13.459981+0000) 2024-04-22T13:36:30.427 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:30 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:30.000+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.361108+0000 front 2024-04-22T13:35:56.360977+0000 (oldest deadline 2024-04-22T13:36:18.661036+0000) 2024-04-22T13:36:30.427 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:30 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:30.000+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:00.961980+0000 front 2024-04-22T13:36:00.961935+0000 (oldest deadline 2024-04-22T13:36:26.261844+0000) 2024-04-22T13:36:30.476 INFO:journalctl@ceph.osd.7.smithi111.stdout:Apr 22 13:36:30 smithi111 podman[68508]: 2024-04-22 13:36:30.367345275 +0000 UTC m=+0.636485107 container died a10d375954ba7f42d3df58135f8eb4922df7ecc944535ac78c7e3cb493764e5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-04-22T13:36:30.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:30 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:30.167+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:30.477 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:30 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:30.167+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:30.477 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:30 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:30.167+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:30.477 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:30 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:30.167+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:30.477 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:30 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:30.167+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.500117+0000 front 2024-04-22T13:36:04.500141+0000 (oldest deadline 2024-04-22T13:36:28.600151+0000) 2024-04-22T13:36:30.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:30 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:30.546+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:30.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:30 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:30.546+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:30.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:30 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:30.546+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:30.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:30 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:30.546+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:30.995 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.7.service' 2024-04-22T13:36:31.034 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:36:31.034 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-22T13:36:31.035 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-22T13:36:31.035 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.8 2024-04-22T13:36:31.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:30 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:30.664+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:31.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:30 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:30.664+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:31.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:30 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:30.664+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:31.087 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:30 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:30.664+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:54.709908+0000 front 2024-04-22T13:35:54.709763+0000 (oldest deadline 2024-04-22T13:36:18.809896+0000) 2024-04-22T13:36:31.087 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:30 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:30.664+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.711148+0000 front 2024-04-22T13:36:04.711181+0000 (oldest deadline 2024-04-22T13:36:30.010919+0000) 2024-04-22T13:36:31.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:31 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:31.128+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:31.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:31 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:31.128+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:31.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:31 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:31.128+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:31.227 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:31 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:31.128+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:31.227 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:31 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:31.128+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.500117+0000 front 2024-04-22T13:36:04.500141+0000 (oldest deadline 2024-04-22T13:36:28.600151+0000) 2024-04-22T13:36:31.345 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:30 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:30.957+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:31.345 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:30 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:30.957+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:45.259185+0000 front 2024-04-22T13:35:45.259222+0000 (oldest deadline 2024-04-22T13:36:13.459981+0000) 2024-04-22T13:36:31.345 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:30 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:30.957+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.361108+0000 front 2024-04-22T13:35:56.360977+0000 (oldest deadline 2024-04-22T13:36:18.661036+0000) 2024-04-22T13:36:31.345 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:30 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:30.957+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:00.961980+0000 front 2024-04-22T13:36:00.961935+0000 (oldest deadline 2024-04-22T13:36:26.261844+0000) 2024-04-22T13:36:31.345 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:31 smithi134 systemd[1]: Stopping Ceph osd.8 for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:36:31.677 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:31 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:31.348+0000 7f4bf7e3f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:36:31.677 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:31 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:31.348+0000 7f4bf7e3f700 -1 osd.8 103 *** Got signal Terminated *** 2024-04-22T13:36:31.677 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:31 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:31.348+0000 7f4bf7e3f700 -1 osd.8 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T13:36:31.677 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:31 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:31.534+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:31.677 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:31 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:31.534+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:31.677 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:31 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:31.534+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:31.677 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:31 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:31.534+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:32.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:31 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:31.626+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:32.087 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:31 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:31.626+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:32.087 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:31 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:31.626+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:32.087 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:31 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:31.626+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:54.709908+0000 front 2024-04-22T13:35:54.709763+0000 (oldest deadline 2024-04-22T13:36:18.809896+0000) 2024-04-22T13:36:32.087 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:31 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:31.626+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.711148+0000 front 2024-04-22T13:36:04.711181+0000 (oldest deadline 2024-04-22T13:36:30.010919+0000) 2024-04-22T13:36:32.177 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:31 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:31.909+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:32.177 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:31 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:31.909+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:45.259185+0000 front 2024-04-22T13:35:45.259222+0000 (oldest deadline 2024-04-22T13:36:13.459981+0000) 2024-04-22T13:36:32.177 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:31 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:31.909+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.361108+0000 front 2024-04-22T13:35:56.360977+0000 (oldest deadline 2024-04-22T13:36:18.661036+0000) 2024-04-22T13:36:32.177 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:31 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:31.909+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:00.961980+0000 front 2024-04-22T13:36:00.961935+0000 (oldest deadline 2024-04-22T13:36:26.261844+0000) 2024-04-22T13:36:32.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:32 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:32.122+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:32.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:32 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:32.122+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:32.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:32 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:32.122+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:32.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:32 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:32.122+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:32.477 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:32 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:32.122+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.500117+0000 front 2024-04-22T13:36:04.500141+0000 (oldest deadline 2024-04-22T13:36:28.600151+0000) 2024-04-22T13:36:32.874 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:32 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:32.568+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:32.874 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:32 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:32.568+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:32.874 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:32 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:32.568+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:32.874 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:32 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:32.568+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:33.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:32 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:32.658+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:33.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:32 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:32.658+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:33.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:32 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:32.658+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:33.087 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:32 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:32.658+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:54.709908+0000 front 2024-04-22T13:35:54.709763+0000 (oldest deadline 2024-04-22T13:36:18.809896+0000) 2024-04-22T13:36:33.087 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:32 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:32.658+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.711148+0000 front 2024-04-22T13:36:04.711181+0000 (oldest deadline 2024-04-22T13:36:30.010919+0000) 2024-04-22T13:36:33.177 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:32 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:32.878+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:33.177 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:32 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:32.878+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:45.259185+0000 front 2024-04-22T13:35:45.259222+0000 (oldest deadline 2024-04-22T13:36:13.459981+0000) 2024-04-22T13:36:33.177 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:32 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:32.878+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.361108+0000 front 2024-04-22T13:35:56.360977+0000 (oldest deadline 2024-04-22T13:36:18.661036+0000) 2024-04-22T13:36:33.177 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:32 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:32.878+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:00.961980+0000 front 2024-04-22T13:36:00.961935+0000 (oldest deadline 2024-04-22T13:36:26.261844+0000) 2024-04-22T13:36:33.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:33 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:33.094+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:33.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:33 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:33.094+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:33.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:33 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:33.094+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:33.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:33 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:33.094+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:33.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:33 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:33.094+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.500117+0000 front 2024-04-22T13:36:04.500141+0000 (oldest deadline 2024-04-22T13:36:28.600151+0000) 2024-04-22T13:36:33.899 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:33 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:33.531+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:33.899 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:33 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:33.531+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:33.899 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:33 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:33.531+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:33.899 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:33 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:33.531+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:34.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:33 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:33.668+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:34.087 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:33 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:33.668+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:34.087 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:33 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:33.668+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:34.087 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:33 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:33.668+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:54.709908+0000 front 2024-04-22T13:35:54.709763+0000 (oldest deadline 2024-04-22T13:36:18.809896+0000) 2024-04-22T13:36:34.087 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:33 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:33.668+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.711148+0000 front 2024-04-22T13:36:04.711181+0000 (oldest deadline 2024-04-22T13:36:30.010919+0000) 2024-04-22T13:36:34.176 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:33 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:33.903+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:34.177 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:33 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:33.903+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:45.259185+0000 front 2024-04-22T13:35:45.259222+0000 (oldest deadline 2024-04-22T13:36:13.459981+0000) 2024-04-22T13:36:34.177 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:33 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:33.903+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.361108+0000 front 2024-04-22T13:35:56.360977+0000 (oldest deadline 2024-04-22T13:36:18.661036+0000) 2024-04-22T13:36:34.177 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:33 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:33.903+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:00.961980+0000 front 2024-04-22T13:36:00.961935+0000 (oldest deadline 2024-04-22T13:36:26.261844+0000) 2024-04-22T13:36:34.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:34 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:34.120+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:34.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:34 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:34.120+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:34.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:34 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:34.120+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:34.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:34 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:34.120+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:34.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:34 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:34.120+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.500117+0000 front 2024-04-22T13:36:04.500141+0000 (oldest deadline 2024-04-22T13:36:28.600151+0000) 2024-04-22T13:36:34.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:34 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:34.632+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:34.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:34 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:34.632+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:34.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:34 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:34.632+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:34.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:34 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:34.632+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:54.709908+0000 front 2024-04-22T13:35:54.709763+0000 (oldest deadline 2024-04-22T13:36:18.809896+0000) 2024-04-22T13:36:34.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:34 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:34.632+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.711148+0000 front 2024-04-22T13:36:04.711181+0000 (oldest deadline 2024-04-22T13:36:30.010919+0000) 2024-04-22T13:36:34.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:34 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:34.632+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:10.011551+0000 front 2024-04-22T13:36:10.011569+0000 (oldest deadline 2024-04-22T13:36:34.111381+0000) 2024-04-22T13:36:34.925 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:34 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:34.571+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:34.925 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:34 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:34.571+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:34.925 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:34 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:34.571+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:34.925 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:34 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:34.571+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:34.925 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:34 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:34.571+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.666758+0000 front 2024-04-22T13:36:12.666682+0000 (oldest deadline 2024-04-22T13:36:34.366802+0000) 2024-04-22T13:36:35.177 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:34 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:34.929+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:35.177 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:34 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:34.929+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:45.259185+0000 front 2024-04-22T13:35:45.259222+0000 (oldest deadline 2024-04-22T13:36:13.459981+0000) 2024-04-22T13:36:35.177 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:34 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:34.929+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.361108+0000 front 2024-04-22T13:35:56.360977+0000 (oldest deadline 2024-04-22T13:36:18.661036+0000) 2024-04-22T13:36:35.177 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:34 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:34.929+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:00.961980+0000 front 2024-04-22T13:36:00.961935+0000 (oldest deadline 2024-04-22T13:36:26.261844+0000) 2024-04-22T13:36:35.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:35 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:35.104+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:35.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:35 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:35.104+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:35.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:35 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:35.104+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:35.477 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:35 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:35.104+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:35.477 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:35 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:35.104+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.500117+0000 front 2024-04-22T13:36:04.500141+0000 (oldest deadline 2024-04-22T13:36:28.600151+0000) 2024-04-22T13:36:35.477 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:35 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:35.104+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.701241+0000 front 2024-04-22T13:36:12.701180+0000 (oldest deadline 2024-04-22T13:36:34.901637+0000) 2024-04-22T13:36:35.926 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:35 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:35.547+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:35.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:35 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:35.547+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:35.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:35 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:35.547+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:35.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:35 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:35.547+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:35.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:35 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:35.547+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.666758+0000 front 2024-04-22T13:36:12.666682+0000 (oldest deadline 2024-04-22T13:36:34.366802+0000) 2024-04-22T13:36:36.086 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:35 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:35.608+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:36.087 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:35 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:35.608+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:36.087 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:35 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:35.608+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:36.087 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:35 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:35.608+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:54.709908+0000 front 2024-04-22T13:35:54.709763+0000 (oldest deadline 2024-04-22T13:36:18.809896+0000) 2024-04-22T13:36:36.087 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:35 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:35.608+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.711148+0000 front 2024-04-22T13:36:04.711181+0000 (oldest deadline 2024-04-22T13:36:30.010919+0000) 2024-04-22T13:36:36.087 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:35 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:35.608+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:10.011551+0000 front 2024-04-22T13:36:10.011569+0000 (oldest deadline 2024-04-22T13:36:34.111381+0000) 2024-04-22T13:36:36.427 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:35 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:35.957+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:37.058290+0000 front 2024-04-22T13:35:37.058491+0000 (oldest deadline 2024-04-22T13:36:02.958392+0000) 2024-04-22T13:36:36.427 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:35 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:35.957+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:45.259185+0000 front 2024-04-22T13:35:45.259222+0000 (oldest deadline 2024-04-22T13:36:13.459981+0000) 2024-04-22T13:36:36.427 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:35 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:35.957+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.361108+0000 front 2024-04-22T13:35:56.360977+0000 (oldest deadline 2024-04-22T13:36:18.661036+0000) 2024-04-22T13:36:36.427 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:35 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8[50462]: 2024-04-22T13:36:35.957+0000 7f4bf4433700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:00.961980+0000 front 2024-04-22T13:36:00.961935+0000 (oldest deadline 2024-04-22T13:36:26.261844+0000) 2024-04-22T13:36:36.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:36 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:36.102+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:36.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:36 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:36.102+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:36.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:36 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:36.102+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:36.477 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:36 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:36.102+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:36.477 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:36 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:36.102+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.500117+0000 front 2024-04-22T13:36:04.500141+0000 (oldest deadline 2024-04-22T13:36:28.600151+0000) 2024-04-22T13:36:36.477 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:36 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:36.102+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.701241+0000 front 2024-04-22T13:36:12.701180+0000 (oldest deadline 2024-04-22T13:36:34.901637+0000) 2024-04-22T13:36:36.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:36 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:36.583+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:36.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:36 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:36.583+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:36.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:36 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:36.583+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:36.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:36 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:36.583+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:54.709908+0000 front 2024-04-22T13:35:54.709763+0000 (oldest deadline 2024-04-22T13:36:18.809896+0000) 2024-04-22T13:36:36.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:36 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:36.583+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.711148+0000 front 2024-04-22T13:36:04.711181+0000 (oldest deadline 2024-04-22T13:36:30.010919+0000) 2024-04-22T13:36:36.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:36 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:36.583+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:10.011551+0000 front 2024-04-22T13:36:10.011569+0000 (oldest deadline 2024-04-22T13:36:34.111381+0000) 2024-04-22T13:36:36.874 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:36 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:36.569+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:36.874 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:36 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:36.569+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:36.874 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:36 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:36.569+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:36.874 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:36 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:36.569+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:36.874 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:36 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:36.569+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.666758+0000 front 2024-04-22T13:36:12.666682+0000 (oldest deadline 2024-04-22T13:36:34.366802+0000) 2024-04-22T13:36:37.176 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:36 smithi134 podman[68837]: 2024-04-22 13:36:36.879418478 +0000 UTC m=+5.645487114 container died dcfeaa679a8e650d897a6a9c9d1532f784b83000066b8f9e181401d11b92f941 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-22T13:36:37.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:37 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:37.069+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:37.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:37 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:37.069+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:37.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:37 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:37.069+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:37.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:37 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:37.069+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:37.477 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:37 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:37.069+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.500117+0000 front 2024-04-22T13:36:04.500141+0000 (oldest deadline 2024-04-22T13:36:28.600151+0000) 2024-04-22T13:36:37.477 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:37 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:37.069+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.701241+0000 front 2024-04-22T13:36:12.701180+0000 (oldest deadline 2024-04-22T13:36:34.901637+0000) 2024-04-22T13:36:37.487 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:37 smithi134 podman[68837]: 2024-04-22 13:36:37.299277413 +0000 UTC m=+6.065346042 container cleanup dcfeaa679a8e650d897a6a9c9d1532f784b83000066b8f9e181401d11b92f941 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-22T13:36:37.487 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:37 smithi134 bash[68837]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8 2024-04-22T13:36:37.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:37 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:37.561+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:37.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:37 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:37.561+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:37.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:37 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:37.561+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:37.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:37 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:37.561+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:54.709908+0000 front 2024-04-22T13:35:54.709763+0000 (oldest deadline 2024-04-22T13:36:18.809896+0000) 2024-04-22T13:36:37.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:37 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:37.561+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.711148+0000 front 2024-04-22T13:36:04.711181+0000 (oldest deadline 2024-04-22T13:36:30.010919+0000) 2024-04-22T13:36:37.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:37 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:37.561+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:10.011551+0000 front 2024-04-22T13:36:10.011569+0000 (oldest deadline 2024-04-22T13:36:34.111381+0000) 2024-04-22T13:36:37.910 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:37 smithi134 podman[68855]: 2024-04-22 13:36:37.492194905 +0000 UTC m=+0.612170250 container remove dcfeaa679a8e650d897a6a9c9d1532f784b83000066b8f9e181401d11b92f941 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-22T13:36:37.910 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:37 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:37.543+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:37.910 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:37 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:37.543+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:37.911 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:37 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:37.543+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:37.911 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:37 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:37.543+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:37.911 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:37 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:37.543+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.666758+0000 front 2024-04-22T13:36:12.666682+0000 (oldest deadline 2024-04-22T13:36:34.366802+0000) 2024-04-22T13:36:38.177 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:37 smithi134 podman[68947]: 2024-04-22 13:36:37.818703646 +0000 UTC m=+0.017675868 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:36:38.177 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:37 smithi134 podman[68947]: 2024-04-22 13:36:37.926290427 +0000 UTC m=+0.125262633 container create c743f7ad46fa64a45c18e9b431f71fb68dfc18776e659e316020384a32a05043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-22T13:36:38.177 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:38 smithi134 podman[68947]: 2024-04-22 13:36:38.15427885 +0000 UTC m=+0.353251057 container init c743f7ad46fa64a45c18e9b431f71fb68dfc18776e659e316020384a32a05043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-22T13:36:38.177 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:38 smithi134 podman[68947]: 2024-04-22 13:36:38.157302376 +0000 UTC m=+0.356274584 container start c743f7ad46fa64a45c18e9b431f71fb68dfc18776e659e316020384a32a05043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-22T13:36:38.441 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:38 smithi134 podman[68947]: 2024-04-22 13:36:38.22924347 +0000 UTC m=+0.428215692 container attach c743f7ad46fa64a45c18e9b431f71fb68dfc18776e659e316020384a32a05043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-22T13:36:38.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:38 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:38.041+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:38.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:38 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:38.041+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:38.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:38 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:38.041+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:38.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:38 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:38.041+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:38.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:38 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:38.041+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.500117+0000 front 2024-04-22T13:36:04.500141+0000 (oldest deadline 2024-04-22T13:36:28.600151+0000) 2024-04-22T13:36:38.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:38 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:38.041+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.701241+0000 front 2024-04-22T13:36:12.701180+0000 (oldest deadline 2024-04-22T13:36:34.901637+0000) 2024-04-22T13:36:38.735 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:38 smithi134 podman[68947]: 2024-04-22 13:36:38.44680687 +0000 UTC m=+0.645779095 container died c743f7ad46fa64a45c18e9b431f71fb68dfc18776e659e316020384a32a05043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True) 2024-04-22T13:36:38.735 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:38 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:38.587+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:38.735 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:38 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:38.587+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:38.735 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:38 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:38.587+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:38.735 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:38 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:38.587+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:38.735 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:38 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:38.587+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.666758+0000 front 2024-04-22T13:36:12.666682+0000 (oldest deadline 2024-04-22T13:36:34.366802+0000) 2024-04-22T13:36:38.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:38 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:38.518+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:38.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:38 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:38.518+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:38.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:38 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:38.518+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:38.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:38 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:38.518+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:54.709908+0000 front 2024-04-22T13:35:54.709763+0000 (oldest deadline 2024-04-22T13:36:18.809896+0000) 2024-04-22T13:36:38.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:38 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:38.518+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.711148+0000 front 2024-04-22T13:36:04.711181+0000 (oldest deadline 2024-04-22T13:36:30.010919+0000) 2024-04-22T13:36:38.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:38 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:38.518+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:10.011551+0000 front 2024-04-22T13:36:10.011569+0000 (oldest deadline 2024-04-22T13:36:34.111381+0000) 2024-04-22T13:36:38.961 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.8.service' 2024-04-22T13:36:38.985 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:38 smithi134 podman[68973]: 2024-04-22 13:36:38.949237834 +0000 UTC m=+0.497218221 container remove c743f7ad46fa64a45c18e9b431f71fb68dfc18776e659e316020384a32a05043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-22T13:36:38.985 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:38 smithi134 systemd[1]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.8.service: Deactivated successfully. 2024-04-22T13:36:38.986 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:38 smithi134 systemd[1]: Stopped Ceph osd.8 for 0aede5ea-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:36:38.986 INFO:journalctl@ceph.osd.8.smithi134.stdout:Apr 22 13:36:38 smithi134 systemd[1]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.8.service: Consumed 13.046s CPU time. 2024-04-22T13:36:38.998 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:36:38.998 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-22T13:36:38.999 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-22T13:36:38.999 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.9 2024-04-22T13:36:39.298 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:39 smithi012 systemd[1]: Stopping Ceph osd.9 for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:36:39.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:39 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:39.031+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:39.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:39 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:39.031+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:39.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:39 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:39.031+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:39.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:39 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:39.031+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:39.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:39 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:39.031+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.500117+0000 front 2024-04-22T13:36:04.500141+0000 (oldest deadline 2024-04-22T13:36:28.600151+0000) 2024-04-22T13:36:39.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:39 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:39.031+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.701241+0000 front 2024-04-22T13:36:12.701180+0000 (oldest deadline 2024-04-22T13:36:34.901637+0000) 2024-04-22T13:36:39.586 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:39 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:39.302+0000 7f52011bf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:36:39.586 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:39 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:39.302+0000 7f52011bf700 -1 osd.9 103 *** Got signal Terminated *** 2024-04-22T13:36:39.587 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:39 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:39.302+0000 7f52011bf700 -1 osd.9 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T13:36:39.587 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:39 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:39.543+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:39.587 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:39 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:39.543+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:39.587 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:39 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:39.543+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:39.587 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:39 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:39.543+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:54.709908+0000 front 2024-04-22T13:35:54.709763+0000 (oldest deadline 2024-04-22T13:36:18.809896+0000) 2024-04-22T13:36:39.587 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:39 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:39.543+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.711148+0000 front 2024-04-22T13:36:04.711181+0000 (oldest deadline 2024-04-22T13:36:30.010919+0000) 2024-04-22T13:36:39.587 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:39 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:39.543+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:10.011551+0000 front 2024-04-22T13:36:10.011569+0000 (oldest deadline 2024-04-22T13:36:34.111381+0000) 2024-04-22T13:36:39.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:39 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:39.578+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:39.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:39 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:39.578+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:39.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:39 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:39.578+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:39.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:39 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:39.578+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:39.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:39 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:39.578+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.666758+0000 front 2024-04-22T13:36:12.666682+0000 (oldest deadline 2024-04-22T13:36:34.366802+0000) 2024-04-22T13:36:40.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:40 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:40.028+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:40.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:40 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:40.028+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:40.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:40 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:40.028+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:40.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:40 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:40.028+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:40.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:40 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:40.028+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.500117+0000 front 2024-04-22T13:36:04.500141+0000 (oldest deadline 2024-04-22T13:36:28.600151+0000) 2024-04-22T13:36:40.477 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:40 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:40.028+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.701241+0000 front 2024-04-22T13:36:12.701180+0000 (oldest deadline 2024-04-22T13:36:34.901637+0000) 2024-04-22T13:36:40.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:40 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:40.527+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:40.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:40 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:40.527+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:40.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:40 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:40.527+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:40.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:40 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:40.527+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:54.709908+0000 front 2024-04-22T13:35:54.709763+0000 (oldest deadline 2024-04-22T13:36:18.809896+0000) 2024-04-22T13:36:40.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:40 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:40.527+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.711148+0000 front 2024-04-22T13:36:04.711181+0000 (oldest deadline 2024-04-22T13:36:30.010919+0000) 2024-04-22T13:36:40.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:40 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:40.527+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:10.011551+0000 front 2024-04-22T13:36:10.011569+0000 (oldest deadline 2024-04-22T13:36:34.111381+0000) 2024-04-22T13:36:40.926 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:40 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:40.585+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:40.926 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:40 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:40.585+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:40.926 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:40 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:40.585+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:40.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:40 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:40.585+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:40.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:40 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:40.585+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.666758+0000 front 2024-04-22T13:36:12.666682+0000 (oldest deadline 2024-04-22T13:36:34.366802+0000) 2024-04-22T13:36:41.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:41 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:41.014+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:41.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:41 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:41.014+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:41.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:41 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:41.014+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:41.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:41 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:41.014+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:41.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:41 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:41.014+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.500117+0000 front 2024-04-22T13:36:04.500141+0000 (oldest deadline 2024-04-22T13:36:28.600151+0000) 2024-04-22T13:36:41.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:41 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:41.014+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.701241+0000 front 2024-04-22T13:36:12.701180+0000 (oldest deadline 2024-04-22T13:36:34.901637+0000) 2024-04-22T13:36:41.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:41 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:41.537+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:41.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:41 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:41.537+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:41.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:41 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:41.537+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:41.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:41 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:41.537+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:54.709908+0000 front 2024-04-22T13:35:54.709763+0000 (oldest deadline 2024-04-22T13:36:18.809896+0000) 2024-04-22T13:36:41.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:41 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:41.537+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.711148+0000 front 2024-04-22T13:36:04.711181+0000 (oldest deadline 2024-04-22T13:36:30.010919+0000) 2024-04-22T13:36:41.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:41 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:41.537+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:10.011551+0000 front 2024-04-22T13:36:10.011569+0000 (oldest deadline 2024-04-22T13:36:34.111381+0000) 2024-04-22T13:36:41.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:41 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:41.620+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:41.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:41 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:41.620+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:41.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:41 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:41.620+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:41.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:41 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:41.620+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:41.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:41 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:41.620+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.666758+0000 front 2024-04-22T13:36:12.666682+0000 (oldest deadline 2024-04-22T13:36:34.366802+0000) 2024-04-22T13:36:42.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:42 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:42.023+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:42.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:42 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:42.023+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:42.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:42 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:42.023+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:42.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:42 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:42.023+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:42.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:42 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:42.023+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.500117+0000 front 2024-04-22T13:36:04.500141+0000 (oldest deadline 2024-04-22T13:36:28.600151+0000) 2024-04-22T13:36:42.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:42 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:42.023+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.701241+0000 front 2024-04-22T13:36:12.701180+0000 (oldest deadline 2024-04-22T13:36:34.901637+0000) 2024-04-22T13:36:42.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:42 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:42.497+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:42.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:42 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:42.497+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:42.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:42 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:42.497+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:42.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:42 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:42.497+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:54.709908+0000 front 2024-04-22T13:35:54.709763+0000 (oldest deadline 2024-04-22T13:36:18.809896+0000) 2024-04-22T13:36:42.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:42 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:42.497+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.711148+0000 front 2024-04-22T13:36:04.711181+0000 (oldest deadline 2024-04-22T13:36:30.010919+0000) 2024-04-22T13:36:42.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:42 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:42.497+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:10.011551+0000 front 2024-04-22T13:36:10.011569+0000 (oldest deadline 2024-04-22T13:36:34.111381+0000) 2024-04-22T13:36:42.926 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:42 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:42.575+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:42.926 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:42 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:42.575+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:42.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:42 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:42.575+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:42.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:42 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:42.575+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:42.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:42 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:42.575+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.666758+0000 front 2024-04-22T13:36:12.666682+0000 (oldest deadline 2024-04-22T13:36:34.366802+0000) 2024-04-22T13:36:42.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:42 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:42.575+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-22T13:36:20.267735+0000 front 2024-04-22T13:36:20.267751+0000 (oldest deadline 2024-04-22T13:36:42.567700+0000) 2024-04-22T13:36:43.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:43 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:43.018+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:43.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:43 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:43.018+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:43.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:43 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:43.018+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:43.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:43 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:43.018+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:43.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:43 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:43.018+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.500117+0000 front 2024-04-22T13:36:04.500141+0000 (oldest deadline 2024-04-22T13:36:28.600151+0000) 2024-04-22T13:36:43.477 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:43 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:43.018+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.701241+0000 front 2024-04-22T13:36:12.701180+0000 (oldest deadline 2024-04-22T13:36:34.901637+0000) 2024-04-22T13:36:43.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:43 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:43.461+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.705102+0000 front 2024-04-22T13:35:32.705176+0000 (oldest deadline 2024-04-22T13:35:53.805263+0000) 2024-04-22T13:36:43.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:43 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:43.461+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.007297+0000 front 2024-04-22T13:35:39.007312+0000 (oldest deadline 2024-04-22T13:36:01.907108+0000) 2024-04-22T13:36:43.836 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:43 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:43.461+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.808368+0000 front 2024-04-22T13:35:47.808234+0000 (oldest deadline 2024-04-22T13:36:11.308366+0000) 2024-04-22T13:36:43.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:43 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:43.461+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:54.709908+0000 front 2024-04-22T13:35:54.709763+0000 (oldest deadline 2024-04-22T13:36:18.809896+0000) 2024-04-22T13:36:43.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:43 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:43.461+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.711148+0000 front 2024-04-22T13:36:04.711181+0000 (oldest deadline 2024-04-22T13:36:30.010919+0000) 2024-04-22T13:36:43.837 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:43 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9[63392]: 2024-04-22T13:36:43.461+0000 7f51fd7b3700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:10.011551+0000 front 2024-04-22T13:36:10.011569+0000 (oldest deadline 2024-04-22T13:36:34.111381+0000) 2024-04-22T13:36:43.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:43 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:43.594+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:43.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:43 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:43.594+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:43.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:43 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:43.594+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:43.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:43 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:43.594+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:43.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:43 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:43.594+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.666758+0000 front 2024-04-22T13:36:12.666682+0000 (oldest deadline 2024-04-22T13:36:34.366802+0000) 2024-04-22T13:36:43.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:43 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:43.594+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-22T13:36:20.267735+0000 front 2024-04-22T13:36:20.267751+0000 (oldest deadline 2024-04-22T13:36:42.567700+0000) 2024-04-22T13:36:44.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:43 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:43.998+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:44.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:43 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:43.998+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:44.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:43 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:43.998+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:44.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:43 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:43.998+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:44.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:43 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:43.998+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.500117+0000 front 2024-04-22T13:36:04.500141+0000 (oldest deadline 2024-04-22T13:36:28.600151+0000) 2024-04-22T13:36:44.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:43 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:43.998+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.701241+0000 front 2024-04-22T13:36:12.701180+0000 (oldest deadline 2024-04-22T13:36:34.901637+0000) 2024-04-22T13:36:44.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:44 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:44.644+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:44.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:44 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:44.644+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:44.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:44 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:44.644+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:44.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:44 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:44.644+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:44.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:44 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:44.644+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.666758+0000 front 2024-04-22T13:36:12.666682+0000 (oldest deadline 2024-04-22T13:36:34.366802+0000) 2024-04-22T13:36:44.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:44 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:44.644+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-22T13:36:20.267735+0000 front 2024-04-22T13:36:20.267751+0000 (oldest deadline 2024-04-22T13:36:42.567700+0000) 2024-04-22T13:36:44.940 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:44 smithi012 podman[102172]: 2024-04-22 13:36:44.691931149 +0000 UTC m=+5.489590324 container died e72530e1e36c64b7032870416d2fe50903e7bbb2cfb483f248fe4b43736bcdad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-22T13:36:45.206 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:45 smithi012 podman[102172]: 2024-04-22 13:36:45.027135284 +0000 UTC m=+5.824794479 container cleanup e72530e1e36c64b7032870416d2fe50903e7bbb2cfb483f248fe4b43736bcdad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-22T13:36:45.206 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:45 smithi012 bash[102172]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9 2024-04-22T13:36:45.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:44 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:44.960+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:45.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:44 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:44.960+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:45.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:44 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:44.960+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:45.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:44 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:44.960+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:45.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:44 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:44.960+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.500117+0000 front 2024-04-22T13:36:04.500141+0000 (oldest deadline 2024-04-22T13:36:28.600151+0000) 2024-04-22T13:36:45.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:44 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:44.960+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.701241+0000 front 2024-04-22T13:36:12.701180+0000 (oldest deadline 2024-04-22T13:36:34.901637+0000) 2024-04-22T13:36:45.586 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:45 smithi012 podman[102190]: 2024-04-22 13:36:45.211574855 +0000 UTC m=+0.516150012 container remove e72530e1e36c64b7032870416d2fe50903e7bbb2cfb483f248fe4b43736bcdad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T13:36:45.926 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:45 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:45.677+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:45.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:45 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:45.677+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:45.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:45 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:45.677+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:45.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:45 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:45.677+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:45.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:45 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:45.677+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.666758+0000 front 2024-04-22T13:36:12.666682+0000 (oldest deadline 2024-04-22T13:36:34.366802+0000) 2024-04-22T13:36:45.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:45 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:45.677+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-22T13:36:20.267735+0000 front 2024-04-22T13:36:20.267751+0000 (oldest deadline 2024-04-22T13:36:42.567700+0000) 2024-04-22T13:36:45.935 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:45 smithi012 podman[102281]: 2024-04-22 13:36:45.654112527 +0000 UTC m=+0.113224771 container create 23b269545494be7675e29b73ed384d28f6e39cd7d1e2c4045ae46d428a824976 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T13:36:45.935 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:45 smithi012 podman[102281]: 2024-04-22 13:36:45.559628994 +0000 UTC m=+0.018741243 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:36:45.935 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:45 smithi012 podman[102281]: 2024-04-22 13:36:45.856655835 +0000 UTC m=+0.315768080 container init 23b269545494be7675e29b73ed384d28f6e39cd7d1e2c4045ae46d428a824976 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-22T13:36:45.935 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:45 smithi012 podman[102281]: 2024-04-22 13:36:45.859803283 +0000 UTC m=+0.318915536 container start 23b269545494be7675e29b73ed384d28f6e39cd7d1e2c4045ae46d428a824976 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-22T13:36:45.935 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:45 smithi012 podman[102281]: 2024-04-22 13:36:45.939883379 +0000 UTC m=+0.398995627 container attach 23b269545494be7675e29b73ed384d28f6e39cd7d1e2c4045ae46d428a824976 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T13:36:46.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:45 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:45.922+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:46.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:45 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:45.922+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:46.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:45 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:45.922+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:46.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:45 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:45.922+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:46.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:45 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:45.922+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.500117+0000 front 2024-04-22T13:36:04.500141+0000 (oldest deadline 2024-04-22T13:36:28.600151+0000) 2024-04-22T13:36:46.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:45 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:45.922+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.701241+0000 front 2024-04-22T13:36:12.701180+0000 (oldest deadline 2024-04-22T13:36:34.901637+0000) 2024-04-22T13:36:46.336 INFO:journalctl@ceph.osd.9.smithi012.stdout:Apr 22 13:36:46 smithi012 podman[102281]: 2024-04-22 13:36:46.166122953 +0000 UTC m=+0.625235193 container died 23b269545494be7675e29b73ed384d28f6e39cd7d1e2c4045ae46d428a824976 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-22T13:36:46.745 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.9.service' 2024-04-22T13:36:46.783 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:36:46.783 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-22T13:36:46.783 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-22T13:36:46.784 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.10 2024-04-22T13:36:47.084 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:46 smithi111 systemd[1]: Stopping Ceph osd.10 for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:36:47.084 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:46 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:46.925+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:47.085 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:46 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:46.925+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:47.085 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:46 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:46.925+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:47.085 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:46 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:46.925+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:47.085 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:46 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:46.925+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.500117+0000 front 2024-04-22T13:36:04.500141+0000 (oldest deadline 2024-04-22T13:36:28.600151+0000) 2024-04-22T13:36:47.085 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:46 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:46.925+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.701241+0000 front 2024-04-22T13:36:12.701180+0000 (oldest deadline 2024-04-22T13:36:34.901637+0000) 2024-04-22T13:36:47.085 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:46 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:46.925+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-22T13:36:14.902188+0000 front 2024-04-22T13:36:14.902107+0000 (oldest deadline 2024-04-22T13:36:46.702713+0000) 2024-04-22T13:36:47.085 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:47 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:47.088+0000 7f2639e73700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:36:47.085 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:47 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:47.088+0000 7f2639e73700 -1 osd.10 103 *** Got signal Terminated *** 2024-04-22T13:36:47.085 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:47 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:47.088+0000 7f2639e73700 -1 osd.10 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T13:36:47.176 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:46 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:46.714+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:47.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:46 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:46.714+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:47.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:46 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:46.714+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:47.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:46 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:46.714+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:47.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:46 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:46.714+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.666758+0000 front 2024-04-22T13:36:12.666682+0000 (oldest deadline 2024-04-22T13:36:34.366802+0000) 2024-04-22T13:36:47.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:46 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:46.714+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-22T13:36:20.267735+0000 front 2024-04-22T13:36:20.267751+0000 (oldest deadline 2024-04-22T13:36:42.567700+0000) 2024-04-22T13:36:48.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:47 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:47.701+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:48.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:47 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:47.701+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:48.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:47 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:47.701+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:48.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:47 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:47.701+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:48.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:47 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:47.701+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.666758+0000 front 2024-04-22T13:36:12.666682+0000 (oldest deadline 2024-04-22T13:36:34.366802+0000) 2024-04-22T13:36:48.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:47 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:47.701+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-22T13:36:20.267735+0000 front 2024-04-22T13:36:20.267751+0000 (oldest deadline 2024-04-22T13:36:42.567700+0000) 2024-04-22T13:36:48.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:47 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:47.944+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:48.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:47 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:47.944+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:48.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:47 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:47.944+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:48.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:47 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:47.944+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:48.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:47 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:47.944+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.500117+0000 front 2024-04-22T13:36:04.500141+0000 (oldest deadline 2024-04-22T13:36:28.600151+0000) 2024-04-22T13:36:48.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:47 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:47.944+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.701241+0000 front 2024-04-22T13:36:12.701180+0000 (oldest deadline 2024-04-22T13:36:34.901637+0000) 2024-04-22T13:36:48.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:47 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:47.944+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-22T13:36:14.902188+0000 front 2024-04-22T13:36:14.902107+0000 (oldest deadline 2024-04-22T13:36:46.702713+0000) 2024-04-22T13:36:48.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:48 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:48.679+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:48.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:48 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:48.679+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:48.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:48 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:48.679+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:48.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:48 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:48.679+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:48.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:48 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:48.679+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.666758+0000 front 2024-04-22T13:36:12.666682+0000 (oldest deadline 2024-04-22T13:36:34.366802+0000) 2024-04-22T13:36:48.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:48 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:48.679+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-22T13:36:20.267735+0000 front 2024-04-22T13:36:20.267751+0000 (oldest deadline 2024-04-22T13:36:42.567700+0000) 2024-04-22T13:36:49.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:48 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:48.980+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:49.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:48 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:48.980+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:49.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:48 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:48.980+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:49.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:48 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:48.980+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:49.477 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:48 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:48.980+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.500117+0000 front 2024-04-22T13:36:04.500141+0000 (oldest deadline 2024-04-22T13:36:28.600151+0000) 2024-04-22T13:36:49.477 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:48 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:48.980+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.701241+0000 front 2024-04-22T13:36:12.701180+0000 (oldest deadline 2024-04-22T13:36:34.901637+0000) 2024-04-22T13:36:49.477 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:48 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:48.980+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-22T13:36:14.902188+0000 front 2024-04-22T13:36:14.902107+0000 (oldest deadline 2024-04-22T13:36:46.702713+0000) 2024-04-22T13:36:49.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:49 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:49.666+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:49.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:49 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:49.666+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:49.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:49 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:49.666+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:49.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:49 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:49.666+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:49.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:49 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:49.666+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.666758+0000 front 2024-04-22T13:36:12.666682+0000 (oldest deadline 2024-04-22T13:36:34.366802+0000) 2024-04-22T13:36:49.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:49 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:49.666+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-22T13:36:20.267735+0000 front 2024-04-22T13:36:20.267751+0000 (oldest deadline 2024-04-22T13:36:42.567700+0000) 2024-04-22T13:36:50.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:49 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:49.966+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:50.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:49 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:49.966+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:50.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:49 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:49.966+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:50.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:49 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:49.966+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:50.227 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:49 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:49.966+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.500117+0000 front 2024-04-22T13:36:04.500141+0000 (oldest deadline 2024-04-22T13:36:28.600151+0000) 2024-04-22T13:36:50.227 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:49 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:49.966+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.701241+0000 front 2024-04-22T13:36:12.701180+0000 (oldest deadline 2024-04-22T13:36:34.901637+0000) 2024-04-22T13:36:50.227 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:49 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:49.966+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-22T13:36:14.902188+0000 front 2024-04-22T13:36:14.902107+0000 (oldest deadline 2024-04-22T13:36:46.702713+0000) 2024-04-22T13:36:50.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:50 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:50.670+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:50.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:50 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:50.670+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:50.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:50 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:50.670+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:50.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:50 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:50.670+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:50.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:50 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:50.670+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.666758+0000 front 2024-04-22T13:36:12.666682+0000 (oldest deadline 2024-04-22T13:36:34.366802+0000) 2024-04-22T13:36:50.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:50 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:50.670+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-22T13:36:20.267735+0000 front 2024-04-22T13:36:20.267751+0000 (oldest deadline 2024-04-22T13:36:42.567700+0000) 2024-04-22T13:36:51.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:50 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:50.929+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:51.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:50 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:50.929+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:51.227 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:50 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:50.929+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:51.227 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:50 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:50.929+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:51.227 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:50 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:50.929+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.500117+0000 front 2024-04-22T13:36:04.500141+0000 (oldest deadline 2024-04-22T13:36:28.600151+0000) 2024-04-22T13:36:51.227 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:50 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:50.929+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.701241+0000 front 2024-04-22T13:36:12.701180+0000 (oldest deadline 2024-04-22T13:36:34.901637+0000) 2024-04-22T13:36:51.227 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:50 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:50.929+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-22T13:36:14.902188+0000 front 2024-04-22T13:36:14.902107+0000 (oldest deadline 2024-04-22T13:36:46.702713+0000) 2024-04-22T13:36:51.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:51 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:51.669+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:51.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:51 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:51.669+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:51.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:51 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:51.669+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:51.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:51 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:51.669+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:51.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:51 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:51.669+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.666758+0000 front 2024-04-22T13:36:12.666682+0000 (oldest deadline 2024-04-22T13:36:34.366802+0000) 2024-04-22T13:36:51.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:51 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:51.669+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-22T13:36:20.267735+0000 front 2024-04-22T13:36:20.267751+0000 (oldest deadline 2024-04-22T13:36:42.567700+0000) 2024-04-22T13:36:51.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:51 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:51.669+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2024-04-22T13:36:27.268804+0000 front 2024-04-22T13:36:27.268886+0000 (oldest deadline 2024-04-22T13:36:50.768673+0000) 2024-04-22T13:36:52.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:51 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:51.927+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.193534+0000 front 2024-04-22T13:35:32.193429+0000 (oldest deadline 2024-04-22T13:35:54.493513+0000) 2024-04-22T13:36:52.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:51 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:51.927+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.695160+0000 front 2024-04-22T13:35:39.695182+0000 (oldest deadline 2024-04-22T13:36:04.395251+0000) 2024-04-22T13:36:52.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:51 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:51.927+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:47.796672+0000 front 2024-04-22T13:35:47.796585+0000 (oldest deadline 2024-04-22T13:36:10.096561+0000) 2024-04-22T13:36:52.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:51 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:51.927+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6814 osd.3 since back 2024-04-22T13:35:56.899148+0000 front 2024-04-22T13:35:56.899292+0000 (oldest deadline 2024-04-22T13:36:19.199185+0000) 2024-04-22T13:36:52.227 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:51 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:51.927+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:04.500117+0000 front 2024-04-22T13:36:04.500141+0000 (oldest deadline 2024-04-22T13:36:28.600151+0000) 2024-04-22T13:36:52.227 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:51 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:51.927+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.701241+0000 front 2024-04-22T13:36:12.701180+0000 (oldest deadline 2024-04-22T13:36:34.901637+0000) 2024-04-22T13:36:52.227 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:51 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10[56819]: 2024-04-22T13:36:51.927+0000 7f2636467700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-22T13:36:14.902188+0000 front 2024-04-22T13:36:14.902107+0000 (oldest deadline 2024-04-22T13:36:46.702713+0000) 2024-04-22T13:36:52.827 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:52 smithi111 podman[68614]: 2024-04-22 13:36:52.574533638 +0000 UTC m=+5.587303910 container died 99f24f1b918dcda411ee9e1480ec4ae916b892fd391a109b7e90d196dfa531cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T13:36:53.176 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:52 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:52.713+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:53.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:52 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:52.713+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:53.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:52 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:52.713+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:53.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:52 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:52.714+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:53.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:52 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:52.714+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.666758+0000 front 2024-04-22T13:36:12.666682+0000 (oldest deadline 2024-04-22T13:36:34.366802+0000) 2024-04-22T13:36:53.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:52 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:52.714+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-22T13:36:20.267735+0000 front 2024-04-22T13:36:20.267751+0000 (oldest deadline 2024-04-22T13:36:42.567700+0000) 2024-04-22T13:36:53.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:52 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:52.714+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2024-04-22T13:36:27.268804+0000 front 2024-04-22T13:36:27.268886+0000 (oldest deadline 2024-04-22T13:36:50.768673+0000) 2024-04-22T13:36:53.184 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:52 smithi111 podman[68614]: 2024-04-22 13:36:52.930433459 +0000 UTC m=+5.943203731 container cleanup 99f24f1b918dcda411ee9e1480ec4ae916b892fd391a109b7e90d196dfa531cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T13:36:53.184 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:52 smithi111 bash[68614]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10 2024-04-22T13:36:53.476 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:53 smithi111 podman[68631]: 2024-04-22 13:36:53.189731938 +0000 UTC m=+0.608153037 container remove 99f24f1b918dcda411ee9e1480ec4ae916b892fd391a109b7e90d196dfa531cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-22T13:36:53.838 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:53 smithi111 podman[68721]: 2024-04-22 13:36:53.449132158 +0000 UTC m=+0.017654011 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:36:53.838 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:53 smithi111 podman[68721]: 2024-04-22 13:36:53.557473325 +0000 UTC m=+0.125995196 container create 4e4d4f8c359f7347cc94033ba4df56e89d6d71062f56ab8a6c115efebbeffc32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-22T13:36:53.838 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:53 smithi111 podman[68721]: 2024-04-22 13:36:53.759891807 +0000 UTC m=+0.328413646 container init 4e4d4f8c359f7347cc94033ba4df56e89d6d71062f56ab8a6c115efebbeffc32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-22T13:36:53.838 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:53 smithi111 podman[68721]: 2024-04-22 13:36:53.763351313 +0000 UTC m=+0.331873144 container start 4e4d4f8c359f7347cc94033ba4df56e89d6d71062f56ab8a6c115efebbeffc32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-22T13:36:53.838 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:53 smithi111 podman[68721]: 2024-04-22 13:36:53.842892467 +0000 UTC m=+0.411414296 container attach 4e4d4f8c359f7347cc94033ba4df56e89d6d71062f56ab8a6c115efebbeffc32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T13:36:54.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:53 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:53.762+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:54.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:53 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:53.762+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:54.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:53 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:53.762+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:54.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:53 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:53.762+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:54.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:53 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:53.762+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.666758+0000 front 2024-04-22T13:36:12.666682+0000 (oldest deadline 2024-04-22T13:36:34.366802+0000) 2024-04-22T13:36:54.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:53 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:53.762+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-22T13:36:20.267735+0000 front 2024-04-22T13:36:20.267751+0000 (oldest deadline 2024-04-22T13:36:42.567700+0000) 2024-04-22T13:36:54.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:53 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:53.762+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2024-04-22T13:36:27.268804+0000 front 2024-04-22T13:36:27.268886+0000 (oldest deadline 2024-04-22T13:36:50.768673+0000) 2024-04-22T13:36:54.226 INFO:journalctl@ceph.osd.10.smithi111.stdout:Apr 22 13:36:54 smithi111 podman[68721]: 2024-04-22 13:36:54.05429548 +0000 UTC m=+0.622817311 container died 4e4d4f8c359f7347cc94033ba4df56e89d6d71062f56ab8a6c115efebbeffc32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-10-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True) 2024-04-22T13:36:54.612 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.10.service' 2024-04-22T13:36:54.649 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:36:54.649 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-22T13:36:54.649 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-22T13:36:54.649 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.11 2024-04-22T13:36:54.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:54 smithi134 systemd[1]: Stopping Ceph osd.11 for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:36:54.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:54 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:54.732+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:54.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:54 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:54.732+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:54.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:54 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:54.732+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:54.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:54 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:54.732+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:54.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:54 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:54.732+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.666758+0000 front 2024-04-22T13:36:12.666682+0000 (oldest deadline 2024-04-22T13:36:34.366802+0000) 2024-04-22T13:36:54.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:54 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:54.732+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-22T13:36:20.267735+0000 front 2024-04-22T13:36:20.267751+0000 (oldest deadline 2024-04-22T13:36:42.567700+0000) 2024-04-22T13:36:54.927 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:54 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:54.732+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2024-04-22T13:36:27.268804+0000 front 2024-04-22T13:36:27.268886+0000 (oldest deadline 2024-04-22T13:36:50.768673+0000) 2024-04-22T13:36:55.426 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:54 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:54.946+0000 7f4c924b2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:36:55.427 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:54 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:54.946+0000 7f4c924b2700 -1 osd.11 103 *** Got signal Terminated *** 2024-04-22T13:36:55.427 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:54 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:54.946+0000 7f4c924b2700 -1 osd.11 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-22T13:36:56.176 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:55 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:55.749+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:56.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:55 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:55.749+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:56.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:55 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:55.749+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:56.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:55 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:55.749+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:56.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:55 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:55.749+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.666758+0000 front 2024-04-22T13:36:12.666682+0000 (oldest deadline 2024-04-22T13:36:34.366802+0000) 2024-04-22T13:36:56.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:55 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:55.749+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-22T13:36:20.267735+0000 front 2024-04-22T13:36:20.267751+0000 (oldest deadline 2024-04-22T13:36:42.567700+0000) 2024-04-22T13:36:56.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:55 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:55.749+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2024-04-22T13:36:27.268804+0000 front 2024-04-22T13:36:27.268886+0000 (oldest deadline 2024-04-22T13:36:50.768673+0000) 2024-04-22T13:36:57.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:56 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:56.789+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:57.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:56 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:56.789+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:57.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:56 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:56.789+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:57.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:56 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:56.789+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:57.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:56 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:56.789+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.666758+0000 front 2024-04-22T13:36:12.666682+0000 (oldest deadline 2024-04-22T13:36:34.366802+0000) 2024-04-22T13:36:57.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:56 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:56.789+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-22T13:36:20.267735+0000 front 2024-04-22T13:36:20.267751+0000 (oldest deadline 2024-04-22T13:36:42.567700+0000) 2024-04-22T13:36:57.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:56 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:56.789+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2024-04-22T13:36:27.268804+0000 front 2024-04-22T13:36:27.268886+0000 (oldest deadline 2024-04-22T13:36:50.768673+0000) 2024-04-22T13:36:58.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:57 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:57.806+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:58.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:57 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:57.806+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:58.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:57 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:57.806+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:58.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:57 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:57.806+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:58.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:57 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:57.806+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.666758+0000 front 2024-04-22T13:36:12.666682+0000 (oldest deadline 2024-04-22T13:36:34.366802+0000) 2024-04-22T13:36:58.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:57 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:57.806+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-22T13:36:20.267735+0000 front 2024-04-22T13:36:20.267751+0000 (oldest deadline 2024-04-22T13:36:42.567700+0000) 2024-04-22T13:36:58.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:57 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:57.806+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2024-04-22T13:36:27.268804+0000 front 2024-04-22T13:36:27.268886+0000 (oldest deadline 2024-04-22T13:36:50.768673+0000) 2024-04-22T13:36:59.176 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:58 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:58.818+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:36:59.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:58 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:58.818+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:36:59.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:58 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:58.818+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:36:59.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:58 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:58.818+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:36:59.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:58 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:58.818+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.666758+0000 front 2024-04-22T13:36:12.666682+0000 (oldest deadline 2024-04-22T13:36:34.366802+0000) 2024-04-22T13:36:59.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:58 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:58.818+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-22T13:36:20.267735+0000 front 2024-04-22T13:36:20.267751+0000 (oldest deadline 2024-04-22T13:36:42.567700+0000) 2024-04-22T13:36:59.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:58 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:58.818+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2024-04-22T13:36:27.268804+0000 front 2024-04-22T13:36:27.268886+0000 (oldest deadline 2024-04-22T13:36:50.768673+0000) 2024-04-22T13:37:00.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:59 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:59.831+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6806 osd.0 since back 2024-04-22T13:35:32.659227+0000 front 2024-04-22T13:35:32.659292+0000 (oldest deadline 2024-04-22T13:35:57.359352+0000) 2024-04-22T13:37:00.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:59 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:59.831+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2024-04-22T13:35:39.060473+0000 front 2024-04-22T13:35:39.060442+0000 (oldest deadline 2024-04-22T13:36:02.560441+0000) 2024-04-22T13:37:00.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:59 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:59.831+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6804 osd.2 since back 2024-04-22T13:35:48.862624+0000 front 2024-04-22T13:35:48.862497+0000 (oldest deadline 2024-04-22T13:36:14.162730+0000) 2024-04-22T13:37:00.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:59 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:59.831+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2024-04-22T13:36:02.165849+0000 front 2024-04-22T13:36:02.165586+0000 (oldest deadline 2024-04-22T13:36:25.665488+0000) 2024-04-22T13:37:00.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:59 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:59.831+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6812 osd.5 since back 2024-04-22T13:36:12.666758+0000 front 2024-04-22T13:36:12.666682+0000 (oldest deadline 2024-04-22T13:36:34.366802+0000) 2024-04-22T13:37:00.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:59 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:59.831+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.12:6822 osd.6 since back 2024-04-22T13:36:20.267735+0000 front 2024-04-22T13:36:20.267751+0000 (oldest deadline 2024-04-22T13:36:42.567700+0000) 2024-04-22T13:37:00.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:59 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:59.831+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2024-04-22T13:36:27.268804+0000 front 2024-04-22T13:36:27.268886+0000 (oldest deadline 2024-04-22T13:36:50.768673+0000) 2024-04-22T13:37:00.177 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:36:59 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11[57487]: 2024-04-22T13:36:59.831+0000 7f4c8aa9e700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.134:6820 osd.8 since back 2024-04-22T13:36:36.069732+0000 front 2024-04-22T13:36:36.069779+0000 (oldest deadline 2024-04-22T13:36:59.470337+0000) 2024-04-22T13:37:00.739 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:37:00 smithi134 podman[69064]: 2024-04-22 13:37:00.481110096 +0000 UTC m=+5.631437111 container died 23ebe1ae87f2e25f15741fca7372da4ced3ed4c795c85f165c442702ea75ae47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-22T13:37:01.105 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:37:00 smithi134 podman[69064]: 2024-04-22 13:37:00.842480289 +0000 UTC m=+5.992807312 container cleanup 23ebe1ae87f2e25f15741fca7372da4ced3ed4c795c85f165c442702ea75ae47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T13:37:01.105 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:37:00 smithi134 bash[69064]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11 2024-04-22T13:37:01.426 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:37:01 smithi134 podman[69080]: 2024-04-22 13:37:01.110265597 +0000 UTC m=+0.629260819 container remove 23ebe1ae87f2e25f15741fca7372da4ced3ed4c795c85f165c442702ea75ae47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-22T13:37:01.750 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:37:01 smithi134 podman[69171]: 2024-04-22 13:37:01.453062149 +0000 UTC m=+0.112861823 container create 85090cc2ffc9da6614e833839151d6a0593fd7502e8c7c5b010c671de6a39fbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-22T13:37:01.750 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:37:01 smithi134 podman[69171]: 2024-04-22 13:37:01.357942463 +0000 UTC m=+0.017742096 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-22T13:37:01.750 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:37:01 smithi134 podman[69171]: 2024-04-22 13:37:01.680290604 +0000 UTC m=+0.340090232 container init 85090cc2ffc9da6614e833839151d6a0593fd7502e8c7c5b010c671de6a39fbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-22T13:37:01.750 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:37:01 smithi134 podman[69171]: 2024-04-22 13:37:01.683402409 +0000 UTC m=+0.343202040 container start 85090cc2ffc9da6614e833839151d6a0593fd7502e8c7c5b010c671de6a39fbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-22T13:37:01.750 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:37:01 smithi134 podman[69171]: 2024-04-22 13:37:01.755145714 +0000 UTC m=+0.414945355 container attach 85090cc2ffc9da6614e833839151d6a0593fd7502e8c7c5b010c671de6a39fbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T13:37:02.176 INFO:journalctl@ceph.osd.11.smithi134.stdout:Apr 22 13:37:01 smithi134 podman[69171]: 2024-04-22 13:37:01.971163812 +0000 UTC m=+0.630963445 container died 85090cc2ffc9da6614e833839151d6a0593fd7502e8c7c5b010c671de6a39fbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-22T13:37:02.577 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@osd.11.service' 2024-04-22T13:37:02.619 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:37:02.620 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-22T13:37:02.620 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-22T13:37:02.620 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.a 2024-04-22T13:37:02.922 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 22 13:37:02 smithi012 systemd[1]: Stopping Ceph mds.a for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:37:03.336 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 22 13:37:02 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-a[70376]: 2024-04-22T13:37:02.925+0000 7fbc523e0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:37:03.337 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 22 13:37:02 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-a[70376]: 2024-04-22T13:37:02.925+0000 7fbc523e0700 -1 mds.a *** got signal Terminated *** 2024-04-22T13:37:05.633 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 22 13:37:05 smithi012 podman[102401]: 2024-04-22 13:37:05.350107296 +0000 UTC m=+2.525643827 container died 130122b5463070ae43d225b0e631940da7b15627ae7a59665d73b58d86f0d4bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-a, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-22T13:37:05.983 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 22 13:37:05 smithi012 podman[102401]: 2024-04-22 13:37:05.737166075 +0000 UTC m=+2.912702607 container cleanup 130122b5463070ae43d225b0e631940da7b15627ae7a59665d73b58d86f0d4bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-a, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-22T13:37:05.983 INFO:journalctl@ceph.mds.a.smithi012.stdout:Apr 22 13:37:05 smithi012 bash[102401]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-a 2024-04-22T13:37:06.111 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.a.service' 2024-04-22T13:37:06.146 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:37:06.146 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-22T13:37:06.146 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-22T13:37:06.146 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.d 2024-04-22T13:37:06.336 INFO:journalctl@ceph.mds.d.smithi012.stdout:Apr 22 13:37:06 smithi012 systemd[1]: Stopping Ceph mds.d for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:37:06.836 INFO:journalctl@ceph.mds.d.smithi012.stdout:Apr 22 13:37:06 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-d[69402]: 2024-04-22T13:37:06.481+0000 7fe3682de700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:37:06.837 INFO:journalctl@ceph.mds.d.smithi012.stdout:Apr 22 13:37:06 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-d[69402]: 2024-04-22T13:37:06.481+0000 7fe3682de700 -1 mds.d *** got signal Terminated *** 2024-04-22T13:37:10.011 INFO:journalctl@ceph.mds.d.smithi012.stdout:Apr 22 13:37:09 smithi012 podman[102522]: 2024-04-22 13:37:09.765048355 +0000 UTC m=+3.387705808 container died b96099ca848db41e053410667ba075056942af1501b3173c097e07841d8804e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T13:37:10.338 INFO:journalctl@ceph.mds.d.smithi012.stdout:Apr 22 13:37:10 smithi012 podman[102522]: 2024-04-22 13:37:10.11567844 +0000 UTC m=+3.738335889 container cleanup b96099ca848db41e053410667ba075056942af1501b3173c097e07841d8804e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-22T13:37:10.338 INFO:journalctl@ceph.mds.d.smithi012.stdout:Apr 22 13:37:10 smithi012 bash[102522]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-d 2024-04-22T13:37:10.547 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.d.service' 2024-04-22T13:37:10.580 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:37:10.580 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-22T13:37:10.581 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-22T13:37:10.581 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.g 2024-04-22T13:37:10.898 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 22 13:37:10 smithi012 systemd[1]: Stopping Ceph mds.g for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:37:11.336 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 22 13:37:10 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-g[69989]: 2024-04-22T13:37:10.901+0000 7fe8ec828700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:37:11.336 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 22 13:37:10 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-g[69989]: 2024-04-22T13:37:10.901+0000 7fe8ec828700 -1 mds.g *** got signal Terminated *** 2024-04-22T13:37:13.639 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 22 13:37:13 smithi012 podman[102642]: 2024-04-22 13:37:13.348008721 +0000 UTC m=+2.544540486 container died 2b9a2ed95a8dd6e490b8b24d126f5ad764b53b2ee98999a99d1219efcba650be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-g, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-22T13:37:13.990 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 22 13:37:13 smithi012 podman[102642]: 2024-04-22 13:37:13.744209025 +0000 UTC m=+2.940740791 container cleanup 2b9a2ed95a8dd6e490b8b24d126f5ad764b53b2ee98999a99d1219efcba650be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416) 2024-04-22T13:37:13.990 INFO:journalctl@ceph.mds.g.smithi012.stdout:Apr 22 13:37:13 smithi012 bash[102642]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-g 2024-04-22T13:37:14.116 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.g.service' 2024-04-22T13:37:14.148 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:37:14.148 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-22T13:37:14.148 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-22T13:37:14.148 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.j 2024-04-22T13:37:14.336 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 22 13:37:14 smithi012 systemd[1]: Stopping Ceph mds.j for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:37:14.836 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 22 13:37:14 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-j[69039]: 2024-04-22T13:37:14.471+0000 7fb0e9b02700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:37:14.836 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 22 13:37:14 smithi012 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-j[69039]: 2024-04-22T13:37:14.471+0000 7fb0e9b02700 -1 mds.j *** got signal Terminated *** 2024-04-22T13:37:18.735 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 22 13:37:18 smithi012 podman[102761]: 2024-04-22 13:37:18.486145217 +0000 UTC m=+4.115953450 container died 4322eb25c6620ba43d35597f19a3b3ede7e464016affd1e04f6b6d8f2fa5a38c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-j, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-22T13:37:19.011 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 22 13:37:18 smithi012 podman[102761]: 2024-04-22 13:37:18.830367448 +0000 UTC m=+4.460175686 container cleanup 4322eb25c6620ba43d35597f19a3b3ede7e464016affd1e04f6b6d8f2fa5a38c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-22T13:37:19.012 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 22 13:37:18 smithi012 bash[102761]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-j 2024-04-22T13:37:19.242 DEBUG:teuthology.orchestra.run.smithi012:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.j.service' 2024-04-22T13:37:19.263 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 22 13:37:19 smithi012 podman[102775]: 2024-04-22 13:37:19.123258951 +0000 UTC m=+0.639181938 container remove 4322eb25c6620ba43d35597f19a3b3ede7e464016affd1e04f6b6d8f2fa5a38c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-j, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-22T13:37:19.263 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 22 13:37:19 smithi012 systemd[1]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.j.service: Deactivated successfully. 2024-04-22T13:37:19.263 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 22 13:37:19 smithi012 systemd[1]: Stopped Ceph mds.j for 0aede5ea-00aa-11ef-bc93-c7b262605968. 2024-04-22T13:37:19.263 INFO:journalctl@ceph.mds.j.smithi012.stdout:Apr 22 13:37:19 smithi012 systemd[1]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.j.service: Consumed 1.100s CPU time. 2024-04-22T13:37:19.274 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:37:19.274 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-22T13:37:19.274 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-22T13:37:19.274 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.b 2024-04-22T13:37:19.584 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 22 13:37:19 smithi111 systemd[1]: Stopping Ceph mds.b for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:37:19.976 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 22 13:37:19 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-b[62191]: 2024-04-22T13:37:19.588+0000 7f20304a3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:37:19.976 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 22 13:37:19 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-b[62191]: 2024-04-22T13:37:19.588+0000 7f20304a3700 -1 mds.b *** got signal Terminated *** 2024-04-22T13:37:21.701 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 22 13:37:21 smithi111 podman[68840]: 2024-04-22 13:37:21.350579275 +0000 UTC m=+1.872783244 container died ee7dfcb8429a7318584ae12e241b31777c839341a5a86c9838940194ceec7517 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True) 2024-04-22T13:37:21.976 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 22 13:37:21 smithi111 podman[68840]: 2024-04-22 13:37:21.796451106 +0000 UTC m=+2.318655072 container cleanup ee7dfcb8429a7318584ae12e241b31777c839341a5a86c9838940194ceec7517 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5) 2024-04-22T13:37:21.976 INFO:journalctl@ceph.mds.b.smithi111.stdout:Apr 22 13:37:21 smithi111 bash[68840]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-b 2024-04-22T13:37:22.220 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.b.service' 2024-04-22T13:37:22.254 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:37:22.254 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-22T13:37:22.254 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-22T13:37:22.254 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.e 2024-04-22T13:37:22.570 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 22 13:37:22 smithi111 systemd[1]: Stopping Ceph mds.e for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:37:22.976 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 22 13:37:22 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-e[61566]: 2024-04-22T13:37:22.575+0000 7f3e582ee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:37:22.976 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 22 13:37:22 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-e[61566]: 2024-04-22T13:37:22.575+0000 7f3e582ee700 -1 mds.e *** got signal Terminated *** 2024-04-22T13:37:25.646 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 22 13:37:25 smithi111 podman[68960]: 2024-04-22 13:37:25.364274991 +0000 UTC m=+2.885662827 container died 933813d49ecada9e59885b27ff94001926ff0147d5570b56874d2ebc34c87f6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-e, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS) 2024-04-22T13:37:25.976 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 22 13:37:25 smithi111 podman[68960]: 2024-04-22 13:37:25.742153852 +0000 UTC m=+3.263541695 container cleanup 933813d49ecada9e59885b27ff94001926ff0147d5570b56874d2ebc34c87f6c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T13:37:25.976 INFO:journalctl@ceph.mds.e.smithi111.stdout:Apr 22 13:37:25 smithi111 bash[68960]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-e 2024-04-22T13:37:26.112 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.e.service' 2024-04-22T13:37:26.143 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:37:26.143 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-22T13:37:26.144 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-22T13:37:26.144 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.h 2024-04-22T13:37:26.466 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 22 13:37:26 smithi111 systemd[1]: Stopping Ceph mds.h for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:37:26.726 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 22 13:37:26 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-h[61871]: 2024-04-22T13:37:26.470+0000 7f3ebf973700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:37:26.726 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 22 13:37:26 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-h[61871]: 2024-04-22T13:37:26.470+0000 7f3ebf973700 -1 mds.h *** got signal Terminated *** 2024-04-22T13:37:29.726 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 22 13:37:29 smithi111 podman[69081]: 2024-04-22 13:37:29.363469291 +0000 UTC m=+2.999437832 container died 2081b13c1b65210248b8024205cffa724f493ca66d7d41c96e396ec89ca336d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-22T13:37:30.208 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 22 13:37:29 smithi111 podman[69081]: 2024-04-22 13:37:29.937157864 +0000 UTC m=+3.573126413 container cleanup 2081b13c1b65210248b8024205cffa724f493ca66d7d41c96e396ec89ca336d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-h, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-22T13:37:30.208 INFO:journalctl@ceph.mds.h.smithi111.stdout:Apr 22 13:37:29 smithi111 bash[69081]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-h 2024-04-22T13:37:30.316 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.h.service' 2024-04-22T13:37:30.349 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:37:30.349 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-22T13:37:30.349 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-22T13:37:30.349 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.k 2024-04-22T13:37:30.476 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 22 13:37:30 smithi111 systemd[1]: Stopping Ceph mds.k for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:37:30.976 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 22 13:37:30 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-k[61243]: 2024-04-22T13:37:30.682+0000 7f06f04ee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:37:30.976 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 22 13:37:30 smithi111 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-k[61243]: 2024-04-22T13:37:30.682+0000 7f06f04ee700 -1 mds.k *** got signal Terminated *** 2024-04-22T13:37:33.726 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 22 13:37:33 smithi111 podman[69213]: 2024-04-22 13:37:33.35739705 +0000 UTC m=+2.790795958 container died 363efbebf019617358b97ce9e3fb7b339e9262b6e82851d8ca37f80787cfe553 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True) 2024-04-22T13:37:34.143 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 22 13:37:33 smithi111 podman[69213]: 2024-04-22 13:37:33.807808475 +0000 UTC m=+3.241207391 container cleanup 363efbebf019617358b97ce9e3fb7b339e9262b6e82851d8ca37f80787cfe553 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-k, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-22T13:37:34.144 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 22 13:37:33 smithi111 bash[69213]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-k 2024-04-22T13:37:34.144 INFO:journalctl@ceph.mds.k.smithi111.stdout:Apr 22 13:37:33 smithi111 podman[69226]: 2024-04-22 13:37:33.958636774 +0000 UTC m=+0.594811420 container remove 363efbebf019617358b97ce9e3fb7b339e9262b6e82851d8ca37f80787cfe553 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-22T13:37:34.162 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.k.service' 2024-04-22T13:37:34.194 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:37:34.195 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-22T13:37:34.195 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-22T13:37:34.195 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.c 2024-04-22T13:37:34.478 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 22 13:37:34 smithi134 systemd[1]: Stopping Ceph mds.c for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:37:34.886 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 22 13:37:34 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-c[62477]: 2024-04-22T13:37:34.491+0000 7f14030b6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:37:34.886 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 22 13:37:34 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-c[62477]: 2024-04-22T13:37:34.491+0000 7f14030b6700 -1 mds.c *** got signal Terminated *** 2024-04-22T13:37:37.604 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 22 13:37:37 smithi134 podman[69277]: 2024-04-22 13:37:37.351081059 +0000 UTC m=+2.955093817 container died 43ee81f18aaa61c7ab7be2a43d45adec793586eca87d518c0cae167334e629d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-c, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-22T13:37:37.926 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 22 13:37:37 smithi134 podman[69277]: 2024-04-22 13:37:37.699413508 +0000 UTC m=+3.303426266 container cleanup 43ee81f18aaa61c7ab7be2a43d45adec793586eca87d518c0cae167334e629d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-c, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-22T13:37:37.927 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 22 13:37:37 smithi134 bash[69277]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-c 2024-04-22T13:37:37.927 INFO:journalctl@ceph.mds.c.smithi134.stdout:Apr 22 13:37:37 smithi134 podman[69304]: 2024-04-22 13:37:37.858864596 +0000 UTC m=+0.503122850 container remove 43ee81f18aaa61c7ab7be2a43d45adec793586eca87d518c0cae167334e629d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-c, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-22T13:37:38.065 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.c.service' 2024-04-22T13:37:38.100 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:37:38.100 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-22T13:37:38.100 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-22T13:37:38.100 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.f 2024-04-22T13:37:38.408 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 22 13:37:38 smithi134 systemd[1]: Stopping Ceph mds.f for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:37:38.676 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 22 13:37:38 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-f[62166]: 2024-04-22T13:37:38.411+0000 7f1d8d4bc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:37:38.677 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 22 13:37:38 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-f[62166]: 2024-04-22T13:37:38.411+0000 7f1d8d4bc700 -1 mds.f *** got signal Terminated *** 2024-04-22T13:37:41.616 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 22 13:37:41 smithi134 podman[69412]: 2024-04-22 13:37:41.351354785 +0000 UTC m=+3.030795719 container died 8ad0c7dc27649152d1bfde057e2cb35d7b4ab1053c5a9c18a98159bb142eec21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-22T13:37:41.926 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 22 13:37:41 smithi134 podman[69412]: 2024-04-22 13:37:41.704571279 +0000 UTC m=+3.384012212 container cleanup 8ad0c7dc27649152d1bfde057e2cb35d7b4ab1053c5a9c18a98159bb142eec21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-22T13:37:41.927 INFO:journalctl@ceph.mds.f.smithi134.stdout:Apr 22 13:37:41 smithi134 bash[69412]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-f 2024-04-22T13:37:42.136 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.f.service' 2024-04-22T13:37:42.169 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:37:42.169 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-22T13:37:42.169 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-22T13:37:42.169 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.i 2024-04-22T13:37:42.545 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 22 13:37:42 smithi134 systemd[1]: Stopping Ceph mds.i for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:37:42.926 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 22 13:37:42 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-i[62801]: 2024-04-22T13:37:42.548+0000 7f866e630700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:37:42.927 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 22 13:37:42 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-i[62801]: 2024-04-22T13:37:42.548+0000 7f866e630700 -1 mds.i *** got signal Terminated *** 2024-04-22T13:37:45.426 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 22 13:37:45 smithi134 podman[69534]: 2024-04-22 13:37:45.352493483 +0000 UTC m=+2.965581359 container died 59128799688f5b0448c78803b026b3666e987dc24535d5ed386fef2f9cde26c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-i, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-22T13:37:45.829 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 22 13:37:45 smithi134 podman[69534]: 2024-04-22 13:37:45.649894175 +0000 UTC m=+3.262982060 container cleanup 59128799688f5b0448c78803b026b3666e987dc24535d5ed386fef2f9cde26c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416) 2024-04-22T13:37:45.829 INFO:journalctl@ceph.mds.i.smithi134.stdout:Apr 22 13:37:45 smithi134 bash[69534]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-i 2024-04-22T13:37:45.950 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.i.service' 2024-04-22T13:37:45.980 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:37:45.980 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-22T13:37:45.981 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-22T13:37:45.981 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.l 2024-04-22T13:37:46.176 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 22 13:37:46 smithi134 systemd[1]: Stopping Ceph mds.l for 0aede5ea-00aa-11ef-bc93-c7b262605968... 2024-04-22T13:37:46.676 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 22 13:37:46 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-l[61858]: 2024-04-22T13:37:46.302+0000 7f5a7c9fa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-22T13:37:46.677 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 22 13:37:46 smithi134 ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-l[61858]: 2024-04-22T13:37:46.302+0000 7f5a7c9fa700 -1 mds.l *** got signal Terminated *** 2024-04-22T13:37:49.677 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 22 13:37:49 smithi134 podman[69653]: 2024-04-22 13:37:49.355658438 +0000 UTC m=+3.156158850 container died a9d4a04af81444ff2a58793619ac037e6593743062f91453f7580721b291faab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-22T13:37:50.025 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 22 13:37:49 smithi134 podman[69653]: 2024-04-22 13:37:49.820947243 +0000 UTC m=+3.621447712 container cleanup a9d4a04af81444ff2a58793619ac037e6593743062f91453f7580721b291faab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-l, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-22T13:37:50.025 INFO:journalctl@ceph.mds.l.smithi134.stdout:Apr 22 13:37:49 smithi134 bash[69653]: ceph-0aede5ea-00aa-11ef-bc93-c7b262605968-mds-l 2024-04-22T13:37:50.175 DEBUG:teuthology.orchestra.run.smithi134:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0aede5ea-00aa-11ef-bc93-c7b262605968@mds.l.service' 2024-04-22T13:37:50.204 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-22T13:37:50.205 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-22T13:37:50.205 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-22T13:37:50.406 INFO:teuthology.orchestra.run.smithi012.stdout:Deleting cluster with fsid: 0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:37:52.975 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-22T13:37:53.166 INFO:teuthology.orchestra.run.smithi111.stdout:Deleting cluster with fsid: 0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:37:55.541 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-22T13:37:55.731 INFO:teuthology.orchestra.run.smithi134.stdout:Deleting cluster with fsid: 0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:37:58.099 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-22T13:37:58.132 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-22T13:37:58.157 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-22T13:37:58.186 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-22T13:37:58.189 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/7668268/remote/smithi012/crash 2024-04-22T13:37:58.190 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/crash -- . 2024-04-22T13:37:58.225 INFO:teuthology.orchestra.run.smithi012.stderr:tar: /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-22T13:37:58.225 INFO:teuthology.orchestra.run.smithi012.stderr:tar: Error is not recoverable: exiting now 2024-04-22T13:37:58.227 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/7668268/remote/smithi111/crash 2024-04-22T13:37:58.228 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/crash -- . 2024-04-22T13:37:58.300 INFO:teuthology.orchestra.run.smithi111.stderr:tar: /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-22T13:37:58.300 INFO:teuthology.orchestra.run.smithi111.stderr:tar: Error is not recoverable: exiting now 2024-04-22T13:37:58.302 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/7668268/remote/smithi134/crash 2024-04-22T13:37:58.303 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/crash -- . 2024-04-22T13:37:58.339 INFO:teuthology.orchestra.run.smithi134.stderr:tar: /var/lib/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-22T13:37:58.339 INFO:teuthology.orchestra.run.smithi134.stderr:tar: Error is not recoverable: exiting now 2024-04-22T13:37:58.340 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-22T13:37:58.340 DEBUG:teuthology.orchestra.run.smithi012:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-22T13:37:58.374 INFO:tasks.cephadm:Compressing logs... 2024-04-22T13:37:58.375 DEBUG:teuthology.orchestra.run.smithi012:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-22T13:37:58.418 DEBUG:teuthology.orchestra.run.smithi111:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-22T13:37:58.420 DEBUG:teuthology.orchestra.run.smithi134:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-22T13:37:58.443 INFO:teuthology.orchestra.run.smithi111.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory 2024-04-22T13:37:58.443 INFO:teuthology.orchestra.run.smithi111.stderr: /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph.log 2024-04-22T13:37:58.443 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-22T13:37:58.443 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-22T13:37:58.443 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-22T13:37:58.444 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-22T13:37:58.444 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-22T13:37:58.444 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-22T13:37:58.444 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-22T13:37:58.449 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mgr.y.log: 87.5% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-22T13:37:58.449 INFO:teuthology.orchestra.run.smithi111.stderr: 90.4% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-22T13:37:58.449 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-22T13:37:58.449 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-22T13:37:58.452 INFO:teuthology.orchestra.run.smithi134.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph.log 2024-04-22T13:37:58.452 INFO:teuthology.orchestra.run.smithi134.stderr:: No such file or directory 2024-04-22T13:37:58.452 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-22T13:37:58.452 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-22T13:37:58.452 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-22T13:37:58.452 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-22T13:37:58.453 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-22T13:37:58.453 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-22T13:37:58.453 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-22T13:37:58.453 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-22T13:37:58.453 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mgr.z.log: 87.6% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-22T13:37:58.453 INFO:teuthology.orchestra.run.smithi134.stderr: 90.4% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-22T13:37:58.457 INFO:teuthology.orchestra.run.smithi012.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-22T13:37:58.458 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph.log 2024-04-22T13:37:58.458 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-22T13:37:58.458 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-22T13:37:58.458 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-22T13:37:58.458 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-22T13:37:58.458 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-22T13:37:58.458 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-22T13:37:58.458 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-22T13:37:58.458 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-22T13:37:58.458 INFO:teuthology.orchestra.run.smithi012.stderr: 87.5% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-22T13:37:58.458 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mon.a.log: 90.3% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-22T13:37:58.458 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-22T13:37:58.458 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-22T13:37:58.458 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph.cephadm.log: 84.2% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-22T13:37:58.459 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-volume.log 94.5% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-22T13:37:58.459 INFO:teuthology.orchestra.run.smithi012.stderr: 2024-04-22T13:37:58.459 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-22T13:37:58.461 INFO:teuthology.orchestra.run.smithi134.stderr: 91.9% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-22T13:37:58.461 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-22T13:37:58.461 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-22T13:37:58.461 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-22T13:37:58.461 INFO:teuthology.orchestra.run.smithi134.stderr: 83.8% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-22T13:37:58.461 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.10.log: 91.9% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-22T13:37:58.461 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-22T13:37:58.461 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.9% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-22T13:37:58.461 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-22T13:37:58.462 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-22T13:37:58.462 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-22T13:37:58.464 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.d.log: 91.1% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-22T13:37:58.464 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-22T13:37:58.465 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.9.log: 94.6% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-22T13:37:58.476 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-22T13:37:58.477 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-22T13:37:58.487 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-22T13:37:58.488 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-22T13:37:58.496 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-volume.log: 92.8% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-22T13:37:58.496 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-22T13:37:58.497 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/cephadm.log: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-22T13:37:58.497 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-22T13:37:58.502 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-22T13:37:58.502 INFO:teuthology.orchestra.run.smithi111.stderr: 91.0% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-22T13:37:58.505 INFO:teuthology.orchestra.run.smithi134.stderr: 91.0% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-22T13:37:58.508 INFO:teuthology.orchestra.run.smithi134.stderr: 94.6% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-22T13:37:58.510 INFO:teuthology.orchestra.run.smithi012.stderr: 94.5% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-22T13:37:58.521 INFO:teuthology.orchestra.run.smithi012.stderr: 94.5% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-22T13:37:58.532 INFO:teuthology.orchestra.run.smithi134.stderr: 92.9% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-22T13:37:58.536 INFO:teuthology.orchestra.run.smithi111.stderr: 93.9% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-22T13:37:58.563 INFO:teuthology.orchestra.run.smithi134.stderr: 94.4% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-22T13:37:58.570 INFO:teuthology.orchestra.run.smithi111.stderr: 93.1% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-22T13:37:58.576 INFO:teuthology.orchestra.run.smithi111.stderr: 92.8% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-22T13:37:58.716 INFO:teuthology.orchestra.run.smithi012.stderr: 90.2% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-22T13:37:58.918 INFO:teuthology.orchestra.run.smithi134.stderr: 92.1% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-22T13:37:58.939 INFO:teuthology.orchestra.run.smithi111.stderr: 92.9% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-22T13:37:58.951 INFO:teuthology.orchestra.run.smithi111.stderr: 92.3% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-22T13:37:59.340 INFO:teuthology.orchestra.run.smithi134.stderr: 93.1% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-22T13:37:59.584 INFO:teuthology.orchestra.run.smithi012.stderr: 93.2% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-22T13:37:59.635 INFO:teuthology.orchestra.run.smithi012.stderr: 93.3% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-22T13:37:59.660 INFO:teuthology.orchestra.run.smithi111.stderr: 93.3% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-22T13:37:59.673 INFO:teuthology.orchestra.run.smithi134.stderr: 93.9% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-22T13:37:59.712 INFO:teuthology.orchestra.run.smithi111.stderr: 93.8% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-22T13:37:59.716 INFO:teuthology.orchestra.run.smithi012.stderr: 90.7% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-22T13:37:59.722 INFO:teuthology.orchestra.run.smithi134.stderr: 93.2% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-22T13:37:59.818 INFO:teuthology.orchestra.run.smithi111.stderr: 93.3% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-22T13:37:59.867 INFO:teuthology.orchestra.run.smithi012.stderr: 93.2% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-22T13:37:59.916 INFO:teuthology.orchestra.run.smithi111.stderr: 93.3% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-22T13:37:59.918 INFO:teuthology.orchestra.run.smithi111.stderr: 2024-04-22T13:37:59.918 INFO:teuthology.orchestra.run.smithi111.stderr:real 0m1.487s 2024-04-22T13:37:59.918 INFO:teuthology.orchestra.run.smithi111.stderr:user 0m6.249s 2024-04-22T13:37:59.918 INFO:teuthology.orchestra.run.smithi111.stderr:sys 0m0.232s 2024-04-22T13:38:00.087 INFO:teuthology.orchestra.run.smithi134.stderr: 93.3% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-22T13:38:00.090 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-04-22T13:38:00.090 INFO:teuthology.orchestra.run.smithi134.stderr:real 0m1.656s 2024-04-22T13:38:00.090 INFO:teuthology.orchestra.run.smithi134.stderr:user 0m5.555s 2024-04-22T13:38:00.090 INFO:teuthology.orchestra.run.smithi134.stderr:sys 0m0.213s 2024-04-22T13:38:00.252 INFO:teuthology.orchestra.run.smithi012.stderr: 93.6% -- replaced with /var/log/ceph/0aede5ea-00aa-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-22T13:38:00.253 INFO:teuthology.orchestra.run.smithi012.stderr: 2024-04-22T13:38:00.253 INFO:teuthology.orchestra.run.smithi012.stderr:real 0m1.823s 2024-04-22T13:38:00.254 INFO:teuthology.orchestra.run.smithi012.stderr:user 0m7.014s 2024-04-22T13:38:00.254 INFO:teuthology.orchestra.run.smithi012.stderr:sys 0m0.257s 2024-04-22T13:38:00.254 INFO:tasks.cephadm:Archiving logs... 2024-04-22T13:38:00.254 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/7668268/remote/smithi012/log 2024-04-22T13:38:00.255 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-22T13:38:01.255 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/7668268/remote/smithi111/log 2024-04-22T13:38:01.256 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-22T13:38:02.121 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/7668268/remote/smithi134/log 2024-04-22T13:38:02.122 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-22T13:38:02.969 INFO:tasks.cephadm:Removing cluster... 2024-04-22T13:38:02.969 DEBUG:teuthology.orchestra.run.smithi012:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 --force 2024-04-22T13:38:03.170 INFO:teuthology.orchestra.run.smithi012.stdout:Deleting cluster with fsid: 0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:38:03.445 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 --force 2024-04-22T13:38:03.635 INFO:teuthology.orchestra.run.smithi111.stdout:Deleting cluster with fsid: 0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:38:04.010 DEBUG:teuthology.orchestra.run.smithi134:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0aede5ea-00aa-11ef-bc93-c7b262605968 --force 2024-04-22T13:38:04.199 INFO:teuthology.orchestra.run.smithi134.stdout:Deleting cluster with fsid: 0aede5ea-00aa-11ef-bc93-c7b262605968 2024-04-22T13:38:04.456 INFO:tasks.cephadm:Removing cephadm ... 2024-04-22T13:38:04.456 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-22T13:38:04.475 DEBUG:teuthology.orchestra.run.smithi111:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-22T13:38:04.491 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-22T13:38:04.505 INFO:tasks.cephadm:Teardown complete 2024-04-22T13:38:04.506 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-22T13:38:04.519 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-22T13:38:04.520 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-22T13:38:04.522 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-22T13:38:04.533 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-22T13:38:04.639 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-22T13:38:04.650 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-22T13:38:04.650 DEBUG:teuthology.orchestra.run.smithi012:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T13:38:04.653 DEBUG:teuthology.orchestra.run.smithi111:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T13:38:04.655 DEBUG:teuthology.orchestra.run.smithi134:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T13:38:04.667 INFO:teuthology.orchestra.run.smithi111.stderr:bash: line 1: ntpq: command not found 2024-04-22T13:38:04.667 INFO:teuthology.orchestra.run.smithi012.stderr:bash: line 1: ntpq: command not found 2024-04-22T13:38:04.669 INFO:teuthology.orchestra.run.smithi134.stderr:bash: line 1: ntpq: command not found 2024-04-22T13:38:04.671 INFO:teuthology.orchestra.run.smithi111.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T13:38:04.671 INFO:teuthology.orchestra.run.smithi111.stdout:=============================================================================== 2024-04-22T13:38:04.671 INFO:teuthology.orchestra.run.smithi111.stdout:^- hv01.front.sepia.ceph.com 2 6 377 23 +465us[ +389us] +/- 590ms 2024-04-22T13:38:04.671 INFO:teuthology.orchestra.run.smithi111.stdout:^* hv02.front.sepia.ceph.com 3 6 377 21 -2135us[-2211us] +/- 64ms 2024-04-22T13:38:04.672 INFO:teuthology.orchestra.run.smithi111.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 20 +2897us[+2897us] +/- 86ms 2024-04-22T13:38:04.672 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T13:38:04.673 INFO:teuthology.orchestra.run.smithi012.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T13:38:04.673 INFO:teuthology.orchestra.run.smithi012.stdout:=============================================================================== 2024-04-22T13:38:04.673 INFO:teuthology.orchestra.run.smithi012.stdout:^- hv01.front.sepia.ceph.com 2 6 377 19 +443us[ +443us] +/- 590ms 2024-04-22T13:38:04.673 INFO:teuthology.orchestra.run.smithi012.stdout:^* hv02.front.sepia.ceph.com 3 6 377 23 -2138us[-2112us] +/- 64ms 2024-04-22T13:38:04.674 INFO:teuthology.orchestra.run.smithi012.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 23 +2877us[+2877us] +/- 86ms 2024-04-22T13:38:04.674 INFO:teuthology.orchestra.run.smithi012.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T13:38:04.675 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T13:38:04.675 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2024-04-22T13:38:04.675 INFO:teuthology.orchestra.run.smithi134.stdout:^- hv01.front.sepia.ceph.com 2 6 377 21 +424us[ +390us] +/- 590ms 2024-04-22T13:38:04.675 INFO:teuthology.orchestra.run.smithi134.stdout:^* hv02.front.sepia.ceph.com 3 6 377 19 -2158us[-2192us] +/- 64ms 2024-04-22T13:38:04.675 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 21 +2893us[+2859us] +/- 86ms 2024-04-22T13:38:04.675 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T13:38:04.675 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-22T13:38:04.686 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-22T13:38:04.687 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-22T13:38:04.698 DEBUG:teuthology.orchestra.run.smithi012:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-22T13:38:04.800 DEBUG:teuthology.orchestra.run.smithi111:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-22T13:38:04.866 DEBUG:teuthology.orchestra.run.smithi134:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-22T13:38:04.930 DEBUG:teuthology.orchestra.run.smithi012:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T13:38:04.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T13:38:04.960 DEBUG:teuthology.orchestra.run.smithi111:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T13:38:04.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T13:38:04.987 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T13:38:05.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T13:38:05.013 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-22T13:38:05.026 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-22T13:38:05.037 INFO:teuthology.task.internal:Duration was 1885.009365 seconds 2024-04-22T13:38:05.038 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-22T13:38:05.049 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-22T13:38:05.049 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-22T13:38:05.052 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-22T13:38:05.055 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-22T13:38:05.089 INFO:teuthology.orchestra.run.smithi111.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T13:38:05.090 INFO:teuthology.orchestra.run.smithi012.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T13:38:05.091 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T13:38:05.513 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-22T13:38:05.514 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi012.front.sepia.ceph.com 2024-04-22T13:38:05.514 DEBUG:teuthology.orchestra.run.smithi012:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-22T13:38:05.540 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi111.front.sepia.ceph.com 2024-04-22T13:38:05.540 DEBUG:teuthology.orchestra.run.smithi111:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-22T13:38:05.580 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi134.front.sepia.ceph.com 2024-04-22T13:38:05.580 DEBUG:teuthology.orchestra.run.smithi134:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-22T13:38:05.609 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-22T13:38:05.609 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-22T13:38:05.612 DEBUG:teuthology.orchestra.run.smithi111:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-22T13:38:05.623 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-22T13:38:05.677 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-22T13:38:05.677 DEBUG:teuthology.orchestra.run.smithi012:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-22T13:38:05.680 DEBUG:teuthology.orchestra.run.smithi111:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-22T13:38:05.687 DEBUG:teuthology.orchestra.run.smithi134:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-22T13:38:06.094 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-22T13:38:06.107 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-22T13:38:06.107 DEBUG:teuthology.orchestra.run.smithi012:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-22T13:38:06.131 DEBUG:teuthology.orchestra.run.smithi111:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-22T13:38:06.156 DEBUG:teuthology.orchestra.run.smithi134:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-22T13:38:06.180 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-22T13:38:06.192 DEBUG:teuthology.orchestra.run.smithi012:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-22T13:38:06.195 DEBUG:teuthology.orchestra.run.smithi111:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-22T13:38:06.198 DEBUG:teuthology.orchestra.run.smithi134:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-22T13:38:06.215 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = core 2024-04-22T13:38:06.217 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern = core 2024-04-22T13:38:06.242 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = core 2024-04-22T13:38:06.263 DEBUG:teuthology.orchestra.run.smithi012:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-22T13:38:06.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T13:38:06.283 DEBUG:teuthology.orchestra.run.smithi111:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-22T13:38:06.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T13:38:06.296 DEBUG:teuthology.orchestra.run.smithi134:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-22T13:38:06.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T13:38:06.317 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-22T13:38:06.333 INFO:teuthology.task.internal:Transferring archived files... 2024-04-22T13:38:06.334 DEBUG:teuthology.misc:Transferring archived files from smithi012:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/7668268/remote/smithi012 2024-04-22T13:38:06.334 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-22T13:38:06.377 DEBUG:teuthology.misc:Transferring archived files from smithi111:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/7668268/remote/smithi111 2024-04-22T13:38:06.378 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-22T13:38:06.412 DEBUG:teuthology.misc:Transferring archived files from smithi134:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/7668268/remote/smithi134 2024-04-22T13:38:06.412 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-22T13:38:06.447 INFO:teuthology.task.internal:Removing archive directory... 2024-04-22T13:38:06.447 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-22T13:38:06.450 DEBUG:teuthology.orchestra.run.smithi111:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-22T13:38:06.452 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-22T13:38:06.497 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-22T13:38:06.511 INFO:teuthology.task.internal:Not uploading archives. 2024-04-22T13:38:06.511 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-22T13:38:06.524 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-22T13:38:06.524 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-22T13:38:06.526 DEBUG:teuthology.orchestra.run.smithi111:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-22T13:38:06.529 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-22T13:38:06.538 INFO:teuthology.orchestra.run.smithi012.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 13:38 /home/ubuntu/cephtest 2024-04-22T13:38:06.541 INFO:teuthology.orchestra.run.smithi111.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 13:38 /home/ubuntu/cephtest 2024-04-22T13:38:06.552 INFO:teuthology.orchestra.run.smithi134.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 13:38 /home/ubuntu/cephtest 2024-04-22T13:38:06.553 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-22T13:38:06.566 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-22T13:38:06.628 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsync-tester}} duration: 1885.0093653202057 flavor: default owner: scheduled_leonidus@teuthology success: true 2024-04-22T13:38:06.628 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-22T13:38:06.749 INFO:teuthology.run:pass